2023-08-27T11:04:53.514 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc 2023-08-27T11:04:53.515 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381536 --owner scheduled_rishabh@teuthology --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} --name rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi --verbose -- /home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381536/orig.config.yaml 2023-08-27T11:04:53.564 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-27T11:04:53.658 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381536 branch: wip-rishabh-xiubo-wip-57154 description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} email: null first_in_suite: false job_id: '7381536' kernel: client: sha1: distro flavor: default kdb: true sha1: a6a69670761025e039dc5ae86619b0cf04c46dc2 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-xiubo-wip-57154 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 1da267a3afa9ab8f9b1d164359066f8bd682ffd8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1da267a3afa9ab8f9b1d164359066f8bd682ffd8 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-xiubo-wip-57154 sha1: 1da267a3afa9ab8f9b1d164359066f8bd682ffd8 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 1da267a3afa9ab8f9b1d164359066f8bd682ffd8 sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-xiubo-wip-57154 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1da267a3afa9ab8f9b1d164359066f8bd682ffd8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1da267a3afa9ab8f9b1d164359066f8bd682ffd8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi112.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi138.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestVolumes - tasks.cephfs.test_volumes.TestSubvolumeGroups - tasks.cephfs.test_volumes.TestSubvolumes - tasks.cephfs.test_subvolume.TestSubvolume teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 449a1bc2027504e7b3c3d7b30fa4178906581da7 timestamp: 2023-08-26_02:16:12 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1553 2023-08-27T11:04:53.658 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1da267a3afa9ab8f9b1d164359066f8bd682ffd8/qa; will attempt to use it 2023-08-27T11:04:53.659 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1da267a3afa9ab8f9b1d164359066f8bd682ffd8/qa/tasks 2023-08-27T11:04:53.659 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-08-27T11:04:53.661 INFO:teuthology.task.internal:Checking packages... 2023-08-27T11:04:53.688 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '1da267a3afa9ab8f9b1d164359066f8bd682ffd8' 2023-08-27T11:04:53.688 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-08-27T11:04:53.689 INFO:teuthology.packaging:ref: None 2023-08-27T11:04:53.689 INFO:teuthology.packaging:tag: None 2023-08-27T11:04:53.690 INFO:teuthology.packaging:branch: wip-rishabh-xiubo-wip-57154 2023-08-27T11:04:53.690 INFO:teuthology.packaging:sha1: 1da267a3afa9ab8f9b1d164359066f8bd682ffd8 2023-08-27T11:04:53.691 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-xiubo-wip-57154 2023-08-27T11:04:53.848 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5707.g1da267a3 2023-08-27T11:04:53.848 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-08-27T11:04:53.854 INFO:teuthology.task.internal:no buildpackages task found 2023-08-27T11:04:53.855 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-08-27T11:04:53.866 INFO:teuthology.task.internal:Saving configuration 2023-08-27T11:04:53.884 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-08-27T11:04:53.895 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-27T11:04:53.921 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381536', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-27 10:59:39.321338', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-27T11:04:53.944 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381536', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-27 10:59:39.319671', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-27T11:04:53.945 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-08-27T11:04:53.956 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-08-27T11:04:53.956 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-08-27T11:04:53.956 INFO:teuthology.run_tasks:Running task console_log... 2023-08-27T11:04:54.035 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5218ef38b0>, signals=[15]) 2023-08-27T11:04:54.036 INFO:teuthology.run_tasks:Running task internal.connect... 2023-08-27T11:04:54.044 INFO:teuthology.task.internal:Opening connections... 2023-08-27T11:04:54.045 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:04:54.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-27T11:04:54.116 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:04:54.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-27T11:04:54.186 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-08-27T11:04:54.198 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2023-08-27T11:04:54.222 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2023-08-27T11:04:54.222 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2023-08-27T11:04:54.280 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="Red Hat Enterprise Linux" 2023-08-27T11:04:54.281 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="8.6 (Ootpa)" 2023-08-27T11:04:54.281 INFO:teuthology.orchestra.run.smithi112.stdout:ID="rhel" 2023-08-27T11:04:54.281 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="fedora" 2023-08-27T11:04:54.282 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="8.6" 2023-08-27T11:04:54.282 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el8" 2023-08-27T11:04:54.282 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-27T11:04:54.282 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2023-08-27T11:04:54.282 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-27T11:04:54.283 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.redhat.com/" 2023-08-27T11:04:54.283 INFO:teuthology.orchestra.run.smithi112.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-27T11:04:54.283 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-27T11:04:54.283 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:04:54.283 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-27T11:04:54.283 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-27T11:04:54.284 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-27T11:04:54.284 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-27T11:04:54.285 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2023-08-27T11:04:54.311 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2023-08-27T11:04:54.331 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2023-08-27T11:04:54.332 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2023-08-27T11:04:54.390 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="Red Hat Enterprise Linux" 2023-08-27T11:04:54.391 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="8.6 (Ootpa)" 2023-08-27T11:04:54.391 INFO:teuthology.orchestra.run.smithi138.stdout:ID="rhel" 2023-08-27T11:04:54.391 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="fedora" 2023-08-27T11:04:54.391 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="8.6" 2023-08-27T11:04:54.392 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el8" 2023-08-27T11:04:54.392 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-27T11:04:54.392 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2023-08-27T11:04:54.392 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-27T11:04:54.392 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://www.redhat.com/" 2023-08-27T11:04:54.393 INFO:teuthology.orchestra.run.smithi138.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-27T11:04:54.393 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-27T11:04:54.393 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:04:54.394 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-27T11:04:54.394 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-27T11:04:54.394 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-27T11:04:54.394 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-27T11:04:54.395 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2023-08-27T11:04:54.419 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-08-27T11:04:54.433 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-08-27T11:04:54.445 INFO:teuthology.task.internal:Checking for old test directory... 2023-08-27T11:04:54.445 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2023-08-27T11:04:54.448 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2023-08-27T11:04:54.466 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-08-27T11:04:54.477 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-08-27T11:04:54.478 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2023-08-27T11:04:54.507 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2023-08-27T11:04:54.534 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-08-27T11:04:54.591 INFO:teuthology.run_tasks:Running task kernel... 2023-08-27T11:04:54.609 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'} 2023-08-27T11:04:54.610 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-08-27T11:04:54.610 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-08-27T11:04:54.610 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-27T11:04:54.611 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-27T11:04:54.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:04:54.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:04:54.635 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2023-08-27T11:04:54.636 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2023-08-27T11:04:54.695 INFO:teuthology.orchestra.run.smithi112.stdout:4.18.0-372.9.1.el8.x86_64 2023-08-27T11:04:54.696 INFO:teuthology.orchestra.run.smithi138.stdout:4.18.0-372.9.1.el8.x86_64 2023-08-27T11:04:54.696 INFO:teuthology.task.kernel:Running kernel on smithi112: 4.18.0-372.9.1.el8.x86_64 2023-08-27T11:04:54.696 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2023-08-27T11:04:54.697 INFO:teuthology.task.kernel:Running kernel on smithi138: 4.18.0-372.9.1.el8.x86_64 2023-08-27T11:04:54.697 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum install -y kernel 2023-08-27T11:04:55.929 INFO:teuthology.orchestra.run.smithi112.stdout:Updating Subscription Management repositories. 2023-08-27T11:04:55.929 INFO:teuthology.orchestra.run.smithi112.stdout:Unable to read consumer identity 2023-08-27T11:04:55.930 INFO:teuthology.orchestra.run.smithi138.stdout:Updating Subscription Management repositories. 2023-08-27T11:04:55.931 INFO:teuthology.orchestra.run.smithi138.stdout:Unable to read consumer identity 2023-08-27T11:04:57.369 INFO:teuthology.orchestra.run.smithi112.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2023-08-27T11:04:57.429 INFO:teuthology.orchestra.run.smithi138.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-08-27T11:04:58.328 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-08-27T11:04:58.538 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-08-27T11:05:01.778 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 606 kB/s | 24 kB 00:00 2023-08-27T11:05:02.005 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 498 kB/s | 24 kB 00:00 2023-08-27T11:05:02.422 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-08-27T11:05:02.472 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-08-27T11:05:02.473 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2023-08-27T11:05:02.474 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-08-27T11:05:02.590 DEBUG:teuthology.orchestra.run.smithi112:> echo no | sudo yum reinstall kernel || true 2023-08-27T11:05:02.690 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-08-27T11:05:02.743 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-08-27T11:05:02.744 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2023-08-27T11:05:02.744 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-08-27T11:05:02.890 DEBUG:teuthology.orchestra.run.smithi138:> echo no | sudo yum reinstall kernel || true 2023-08-27T11:05:02.919 INFO:teuthology.orchestra.run.smithi112.stdout:Updating Subscription Management repositories. 2023-08-27T11:05:02.919 INFO:teuthology.orchestra.run.smithi112.stdout:Unable to read consumer identity 2023-08-27T11:05:03.175 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Aug 2023 11:05:01 AM UTC. 2023-08-27T11:05:03.215 INFO:teuthology.orchestra.run.smithi112.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-08-27T11:05:03.222 INFO:teuthology.orchestra.run.smithi138.stdout:Updating Subscription Management repositories. 2023-08-27T11:05:03.223 INFO:teuthology.orchestra.run.smithi138.stdout:Unable to read consumer identity 2023-08-27T11:05:03.256 INFO:teuthology.orchestra.run.smithi112.stderr:Error: No packages marked for reinstall. 2023-08-27T11:05:03.303 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum reinstall -y kernel || true 2023-08-27T11:05:03.478 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Aug 2023 11:05:01 AM UTC. 2023-08-27T11:05:03.520 INFO:teuthology.orchestra.run.smithi138.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-08-27T11:05:03.564 INFO:teuthology.orchestra.run.smithi138.stderr:Error: No packages marked for reinstall. 2023-08-27T11:05:03.612 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum reinstall -y kernel || true 2023-08-27T11:05:03.634 INFO:teuthology.orchestra.run.smithi112.stdout:Updating Subscription Management repositories. 2023-08-27T11:05:03.635 INFO:teuthology.orchestra.run.smithi112.stdout:Unable to read consumer identity 2023-08-27T11:05:03.888 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Aug 2023 11:05:01 AM UTC. 2023-08-27T11:05:03.926 INFO:teuthology.orchestra.run.smithi112.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-08-27T11:05:03.943 INFO:teuthology.orchestra.run.smithi138.stdout:Updating Subscription Management repositories. 2023-08-27T11:05:03.943 INFO:teuthology.orchestra.run.smithi138.stdout:Unable to read consumer identity 2023-08-27T11:05:03.963 INFO:teuthology.orchestra.run.smithi112.stderr:Error: No packages marked for reinstall. 2023-08-27T11:05:04.009 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2023-08-27T11:05:04.089 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-08-27T11:05:04.090 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-08-27T11:05:04.090 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-08-27T11:05:04.091 DEBUG:teuthology.parallel:result is None 2023-08-27T11:05:04.197 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:03 ago on Sun 27 Aug 2023 11:05:01 AM UTC. 2023-08-27T11:05:04.237 INFO:teuthology.orchestra.run.smithi138.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-08-27T11:05:04.267 INFO:teuthology.orchestra.run.smithi138.stderr:Error: No packages marked for reinstall. 2023-08-27T11:05:04.316 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2023-08-27T11:05:04.437 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-08-27T11:05:04.438 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-08-27T11:05:04.439 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-08-27T11:05:04.439 DEBUG:teuthology.parallel:result is None 2023-08-27T11:05:04.439 INFO:teuthology.run_tasks:Running task internal.base... 2023-08-27T11:05:04.456 INFO:teuthology.task.internal:Creating test directory... 2023-08-27T11:05:04.456 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-27T11:05:04.459 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-27T11:05:04.476 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-08-27T11:05:04.484 INFO:teuthology.run_tasks:Running task internal.archive... 2023-08-27T11:05:04.520 INFO:teuthology.task.internal:Creating archive directory... 2023-08-27T11:05:04.520 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-27T11:05:04.523 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-27T11:05:04.546 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-08-27T11:05:04.553 INFO:teuthology.task.internal:Enabling coredump saving... 2023-08-27T11:05:04.554 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-27T11:05:04.582 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-27T11:05:04.617 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-27T11:05:04.626 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-27T11:05:04.642 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-27T11:05:04.664 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-27T11:05:04.665 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-08-27T11:05:04.676 INFO:teuthology.task.internal:Configuring sudo... 2023-08-27T11:05:04.676 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-27T11:05:04.708 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-27T11:05:04.735 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-08-27T11:05:04.755 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-08-27T11:05:04.755 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-27T11:05:04.776 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-27T11:05:04.793 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-27T11:05:04.872 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-27T11:05:04.920 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-27T11:05:05.004 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-27T11:05:05.044 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:05:05.045 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-27T11:05:05.124 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-27T11:05:05.166 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-27T11:05:05.229 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-27T11:05:05.308 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-27T11:05:05.354 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:05:05.354 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-27T11:05:05.421 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2023-08-27T11:05:05.423 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2023-08-27T11:05:05.473 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-27T11:05:05.518 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-27T11:05:05.788 INFO:teuthology.run_tasks:Running task internal.timer... 2023-08-27T11:05:05.796 INFO:teuthology.task.internal:Starting timer... 2023-08-27T11:05:05.797 INFO:teuthology.run_tasks:Running task pcp... 2023-08-27T11:05:05.813 INFO:teuthology.run_tasks:Running task selinux... 2023-08-27T11:05:05.826 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-08-27T11:05:05.827 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2023-08-27T11:05:05.905 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2023-08-27T11:05:05.907 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2023-08-27T11:05:05.966 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2023-08-27T11:05:05.968 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-08-27T11:05:05.969 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2023-08-27T11:05:05.995 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2023-08-27T11:05:05.996 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2023-08-27T11:05:06.014 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2023-08-27T11:05:06.015 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive'} 2023-08-27T11:05:06.015 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-27T11:05:06.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:05:06.063 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-27T11:05:06.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:05:06.090 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-08-27T11:05:06.090 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2023-08-27T11:05:06.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2023-08-27T11:05:06.166 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-08-27T11:05:06.186 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-08-27T11:05:06.191 INFO:teuthology.repo_utils:Fetching main from origin 2023-08-27T11:05:06.295 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-08-27T11:05:06.311 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-08-27T11:05:06.313 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi112.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-08-27T11:13:45.018 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2023-08-27T11:13:45.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-08-27T11:13:45.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-27T11:13:45.126 DEBUG:teuthology.orchestra.run.smithi112:> true 2023-08-27T11:13:45.150 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-08-27T11:13:45.151 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2023-08-27T11:13:45.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-27T11:13:45.220 DEBUG:teuthology.orchestra.run.smithi138:> true 2023-08-27T11:13:45.286 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2023-08-27T11:13:45.286 INFO:teuthology.run_tasks:Running task clock... 2023-08-27T11:13:45.301 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-08-27T11:13:45.301 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-27T11:13:45.302 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-27T11:13:45.304 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-27T11:13:45.305 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-27T11:13:45.333 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-27T11:13:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-27T11:13:45.357 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-27T11:13:45.370 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-27T11:13:45.415 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2023-08-27T11:13:45.426 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2023-08-27T11:13:45.437 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2023-08-27T11:13:45.440 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-27T11:13:45.450 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2023-08-27T11:13:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-27T11:13:45.463 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-27T11:13:45.477 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-27T11:13:45.523 INFO:teuthology.orchestra.run.smithi112.stderr:bash: ntpq: command not found 2023-08-27T11:13:45.528 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-27T11:13:45.528 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2023-08-27T11:13:45.528 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:13:45.528 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:13:45.529 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:13:45.529 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:13:45.539 INFO:teuthology.orchestra.run.smithi138.stderr:bash: ntpq: command not found 2023-08-27T11:13:45.542 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-27T11:13:45.542 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2023-08-27T11:13:45.543 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:13:45.543 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:13:45.543 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:13:45.549 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:13:45.550 INFO:teuthology.run_tasks:Running task install... 2023-08-27T11:13:45.560 DEBUG:teuthology.task.install:project ceph 2023-08-27T11:13:45.560 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8'}} 2023-08-27T11:13:45.561 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8'} 2023-08-27T11:13:45.561 INFO:teuthology.task.install:Using flavor: default 2023-08-27T11:13:45.568 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-08-27T11:13:45.568 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-08-27T11:13:45.569 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8', 'tag': None, 'wait_for_package': False} 2023-08-27T11:13:45.569 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1da267a3afa9ab8f9b1d164359066f8bd682ffd8 2023-08-27T11:13:45.571 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8', 'tag': None, 'wait_for_package': False} 2023-08-27T11:13:45.572 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1da267a3afa9ab8f9b1d164359066f8bd682ffd8 2023-08-27T11:13:45.791 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/ 2023-08-27T11:13:45.791 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5707.g1da267a3 2023-08-27T11:13:45.806 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/ 2023-08-27T11:13:45.806 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5707.g1da267a3 2023-08-27T11:13:45.984 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-27T11:13:45.985 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:13:45.985 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-27T11:13:45.987 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-27T11:13:45.988 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:13:45.988 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-27T11:13:46.015 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-27T11:13:46.016 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-27T11:13:46.017 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-27T11:13:46.017 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-27T11:13:46.083 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-27T11:13:46.085 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-27T11:13:46.162 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-27T11:13:46.164 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-27T11:13:46.228 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2023-08-27T11:13:46.230 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2023-08-27T11:13:46.231 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2023-08-27T11:13:46.232 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2023-08-27T11:13:46.600 INFO:teuthology.orchestra.run.smithi112.stdout:Updating Subscription Management repositories. 2023-08-27T11:13:46.602 INFO:teuthology.orchestra.run.smithi138.stdout:Updating Subscription Management repositories. 2023-08-27T11:13:46.832 INFO:teuthology.orchestra.run.smithi112.stdout:44 files removed 2023-08-27T11:13:46.840 INFO:teuthology.orchestra.run.smithi138.stdout:44 files removed 2023-08-27T11:13:46.878 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-27T11:13:46.887 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-27T11:13:47.207 INFO:teuthology.orchestra.run.smithi112.stdout:Updating Subscription Management repositories. 2023-08-27T11:13:47.216 INFO:teuthology.orchestra.run.smithi138.stdout:Updating Subscription Management repositories. 2023-08-27T11:13:47.880 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 254 kB/s | 81 kB 00:00 2023-08-27T11:13:47.916 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 233 kB/s | 81 kB 00:00 2023-08-27T11:13:48.242 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 32 kB/s | 9.4 kB 00:00 2023-08-27T11:13:48.272 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 26 kB/s | 9.4 kB 00:00 2023-08-27T11:13:48.581 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 5.2 kB/s | 1.5 kB 00:00 2023-08-27T11:13:48.618 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2023-08-27T11:13:48.740 INFO:teuthology.orchestra.run.smithi112.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-08-27T11:13:48.752 INFO:teuthology.orchestra.run.smithi138.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-08-27T11:13:50.099 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-08-27T11:13:50.328 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-08-27T11:13:53.499 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 596 kB/s | 24 kB 00:00 2023-08-27T11:13:53.878 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 362 kB/s | 24 kB 00:00 2023-08-27T11:13:54.393 INFO:teuthology.orchestra.run.smithi112.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-08-27T11:13:54.763 INFO:teuthology.orchestra.run.smithi138.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 65 MB/s | 53 MB 00:00 2023-08-27T11:14:02.261 INFO:teuthology.orchestra.run.smithi112.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 60 MB/s | 53 MB 00:00 2023-08-27T11:14:03.022 INFO:teuthology.orchestra.run.smithi138.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-08-27T11:14:13.326 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-27T11:14:13.327 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-27T11:14:13.341 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-27T11:14:13.342 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-27T11:14:13.344 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-27T11:14:13.345 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-27T11:14:13.347 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-27T11:14:13.350 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-27T11:14:13.352 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-27T11:14:13.481 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-08-27T11:14:13.493 INFO:teuthology.orchestra.run.smithi112.stdout:=========================================================================================================================== 2023-08-27T11:14:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2023-08-27T11:14:13.494 INFO:teuthology.orchestra.run.smithi112.stdout:=========================================================================================================================== 2023-08-27T11:14:13.494 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-08-27T11:14:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-27T11:14:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-27T11:14:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 6.4 k 2023-08-27T11:14:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 5.4 M 2023-08-27T11:14:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 867 k 2023-08-27T11:14:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 167 k 2023-08-27T11:14:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 1.6 M 2023-08-27T11:14:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 132 k 2023-08-27T11:14:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 1.7 M 2023-08-27T11:14:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 7.4 M 2023-08-27T11:14:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 47 k 2023-08-27T11:14:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 14 M 2023-08-27T11:14:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 48 M 2023-08-27T11:14:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 261 k 2023-08-27T11:14:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 171 k 2023-08-27T11:14:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 260 k 2023-08-27T11:14:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 18 k 2023-08-27T11:14:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-27T11:14:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-27T11:14:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-27T11:14:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 31 k 2023-08-27T11:14:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 789 k 2023-08-27T11:14:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 139 k 2023-08-27T11:14:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-27T11:14:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-27T11:14:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-27T11:14:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-27T11:14:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-27T11:14:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-27T11:14:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 197 k 2023-08-27T11:14:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 371 k 2023-08-27T11:14:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 364 k 2023-08-27T11:14:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 107 k 2023-08-27T11:14:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 94 k 2023-08-27T11:14:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 3.5 M 2023-08-27T11:14:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 179 k 2023-08-27T11:14:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-27T11:14:13.506 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2023-08-27T11:14:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-27T11:14:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 3.7 M 2023-08-27T11:14:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 3.6 M 2023-08-27T11:14:13.507 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-08-27T11:14:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-27T11:14:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 22 M 2023-08-27T11:14:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 23 k 2023-08-27T11:14:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 2.3 M 2023-08-27T11:14:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 242 k 2023-08-27T11:14:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 4.6 M 2023-08-27T11:14:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 18 M 2023-08-27T11:14:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 15 k 2023-08-27T11:14:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 24 k 2023-08-27T11:14:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-27T11:14:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-27T11:14:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-27T11:14:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-27T11:14:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-27T11:14:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 139 k 2023-08-27T11:14:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-27T11:14:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-27T11:14:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-27T11:14:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-27T11:14:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-27T11:14:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-27T11:14:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 473 k 2023-08-27T11:14:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-27T11:14:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 6.8 M 2023-08-27T11:14:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-27T11:14:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-27T11:14:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-27T11:14:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-27T11:14:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-27T11:14:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-27T11:14:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-27T11:14:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-27T11:14:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-27T11:14:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-27T11:14:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-27T11:14:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-27T11:14:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-27T11:14:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-27T11:14:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-27T11:14:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-27T11:14:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 45 k 2023-08-27T11:14:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 126 k 2023-08-27T11:14:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-27T11:14:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-27T11:14:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-27T11:14:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-27T11:14:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-27T11:14:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-27T11:14:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-27T11:14:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-27T11:14:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-27T11:14:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-27T11:14:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-27T11:14:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-27T11:14:13.519 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-27T11:14:13.519 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-27T11:14:13.519 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-27T11:14:13.519 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-27T11:14:13.519 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-27T11:14:13.520 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-27T11:14:13.520 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-27T11:14:13.520 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-27T11:14:13.520 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-27T11:14:13.520 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-27T11:14:13.521 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-27T11:14:13.521 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-27T11:14:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-27T11:14:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-27T11:14:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-27T11:14:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-27T11:14:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-27T11:14:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-27T11:14:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-27T11:14:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-27T11:14:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-27T11:14:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-27T11:14:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-27T11:14:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-27T11:14:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-27T11:14:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-27T11:14:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-27T11:14:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-27T11:14:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-27T11:14:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-27T11:14:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-27T11:14:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-27T11:14:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-27T11:14:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-27T11:14:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-27T11:14:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-27T11:14:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-27T11:14:13.527 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling module streams: 2023-08-27T11:14:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql 10 2023-08-27T11:14:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:14:13.527 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-08-27T11:14:13.528 INFO:teuthology.orchestra.run.smithi112.stdout:=========================================================================================================================== 2023-08-27T11:14:13.528 INFO:teuthology.orchestra.run.smithi112.stdout:Install 128 Packages 2023-08-27T11:14:13.528 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 3 Packages 2023-08-27T11:14:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:14:13.529 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 206 M 2023-08-27T11:14:13.529 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-08-27T11:14:13.785 INFO:teuthology.orchestra.run.smithi112.stdout:(1/131): ceph-18.0.0-5707.g1da267a3.el8.x86_64. 28 kB/s | 6.4 kB 00:00 2023-08-27T11:14:14.129 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-27T11:14:14.130 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-27T11:14:14.144 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-27T11:14:14.145 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-27T11:14:14.146 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-27T11:14:14.147 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-27T11:14:14.149 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-27T11:14:14.152 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-27T11:14:14.153 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-27T11:14:14.186 INFO:teuthology.orchestra.run.smithi112.stdout:(2/131): ceph-base-18.0.0-5707.g1da267a3.el8.x8 8.5 MB/s | 5.4 MB 00:00 2023-08-27T11:14:14.211 INFO:teuthology.orchestra.run.smithi112.stdout:(3/131): ceph-fuse-18.0.0-5707.g1da267a3.el8.x8 2.0 MB/s | 867 kB 00:00 2023-08-27T11:14:14.237 INFO:teuthology.orchestra.run.smithi112.stdout:(4/131): ceph-immutable-object-cache-18.0.0-570 3.2 MB/s | 167 kB 00:00 2023-08-27T11:14:14.284 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-08-27T11:14:14.296 INFO:teuthology.orchestra.run.smithi138.stdout:=========================================================================================================================== 2023-08-27T11:14:14.297 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2023-08-27T11:14:14.297 INFO:teuthology.orchestra.run.smithi138.stdout:=========================================================================================================================== 2023-08-27T11:14:14.297 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-08-27T11:14:14.298 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-27T11:14:14.298 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-27T11:14:14.298 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 6.4 k 2023-08-27T11:14:14.298 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 5.4 M 2023-08-27T11:14:14.299 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 867 k 2023-08-27T11:14:14.299 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 167 k 2023-08-27T11:14:14.299 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 1.6 M 2023-08-27T11:14:14.299 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 132 k 2023-08-27T11:14:14.299 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 1.7 M 2023-08-27T11:14:14.300 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 7.4 M 2023-08-27T11:14:14.300 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 47 k 2023-08-27T11:14:14.300 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 14 M 2023-08-27T11:14:14.300 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 48 M 2023-08-27T11:14:14.301 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 261 k 2023-08-27T11:14:14.301 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 171 k 2023-08-27T11:14:14.301 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 260 k 2023-08-27T11:14:14.301 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 18 k 2023-08-27T11:14:14.301 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-27T11:14:14.302 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-27T11:14:14.302 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-27T11:14:14.302 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 31 k 2023-08-27T11:14:14.302 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 789 k 2023-08-27T11:14:14.303 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 139 k 2023-08-27T11:14:14.303 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-27T11:14:14.303 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-27T11:14:14.303 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-27T11:14:14.304 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-27T11:14:14.304 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-27T11:14:14.304 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-27T11:14:14.305 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 197 k 2023-08-27T11:14:14.305 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 371 k 2023-08-27T11:14:14.305 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 364 k 2023-08-27T11:14:14.306 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 107 k 2023-08-27T11:14:14.306 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 94 k 2023-08-27T11:14:14.306 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 3.5 M 2023-08-27T11:14:14.307 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 179 k 2023-08-27T11:14:14.307 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-27T11:14:14.307 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2023-08-27T11:14:14.308 INFO:teuthology.orchestra.run.smithi138.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-27T11:14:14.308 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 3.7 M 2023-08-27T11:14:14.308 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 3.6 M 2023-08-27T11:14:14.309 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2023-08-27T11:14:14.309 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-27T11:14:14.310 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 22 M 2023-08-27T11:14:14.310 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 23 k 2023-08-27T11:14:14.310 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 2.3 M 2023-08-27T11:14:14.310 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 242 k 2023-08-27T11:14:14.311 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 4.6 M 2023-08-27T11:14:14.311 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 18 M 2023-08-27T11:14:14.312 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 15 k 2023-08-27T11:14:14.312 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 24 k 2023-08-27T11:14:14.312 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-27T11:14:14.313 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-27T11:14:14.313 INFO:teuthology.orchestra.run.smithi138.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-27T11:14:14.313 INFO:teuthology.orchestra.run.smithi138.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-27T11:14:14.314 INFO:teuthology.orchestra.run.smithi138.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-27T11:14:14.314 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 139 k 2023-08-27T11:14:14.314 INFO:teuthology.orchestra.run.smithi138.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-27T11:14:14.314 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-27T11:14:14.315 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-27T11:14:14.315 INFO:teuthology.orchestra.run.smithi138.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-27T11:14:14.315 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-27T11:14:14.315 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-27T11:14:14.315 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 473 k 2023-08-27T11:14:14.316 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-27T11:14:14.316 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 6.8 M 2023-08-27T11:14:14.316 INFO:teuthology.orchestra.run.smithi138.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-27T11:14:14.316 INFO:teuthology.orchestra.run.smithi138.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-27T11:14:14.317 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-27T11:14:14.317 INFO:teuthology.orchestra.run.smithi138.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-27T11:14:14.317 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-27T11:14:14.317 INFO:teuthology.orchestra.run.smithi138.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-27T11:14:14.317 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-27T11:14:14.318 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-27T11:14:14.318 INFO:teuthology.orchestra.run.smithi138.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-27T11:14:14.318 INFO:teuthology.orchestra.run.smithi138.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-27T11:14:14.318 INFO:teuthology.orchestra.run.smithi138.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-27T11:14:14.319 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-27T11:14:14.319 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-27T11:14:14.319 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-27T11:14:14.319 INFO:teuthology.orchestra.run.smithi138.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-27T11:14:14.319 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-27T11:14:14.319 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 45 k 2023-08-27T11:14:14.320 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 126 k 2023-08-27T11:14:14.320 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-27T11:14:14.320 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-27T11:14:14.320 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-27T11:14:14.320 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-27T11:14:14.321 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-27T11:14:14.321 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-27T11:14:14.321 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-27T11:14:14.321 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-27T11:14:14.321 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-27T11:14:14.322 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-27T11:14:14.322 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-27T11:14:14.322 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-27T11:14:14.322 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-27T11:14:14.322 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-27T11:14:14.323 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-27T11:14:14.323 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-27T11:14:14.323 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-27T11:14:14.323 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-27T11:14:14.323 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-27T11:14:14.324 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-27T11:14:14.324 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-27T11:14:14.324 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-27T11:14:14.324 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-27T11:14:14.324 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-27T11:14:14.325 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-27T11:14:14.325 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-27T11:14:14.325 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-27T11:14:14.325 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-27T11:14:14.325 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-27T11:14:14.326 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-27T11:14:14.326 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-27T11:14:14.326 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-27T11:14:14.326 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-27T11:14:14.326 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-27T11:14:14.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-27T11:14:14.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-27T11:14:14.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-27T11:14:14.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-27T11:14:14.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-27T11:14:14.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-27T11:14:14.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-27T11:14:14.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-27T11:14:14.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-27T11:14:14.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-27T11:14:14.329 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-27T11:14:14.329 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-27T11:14:14.329 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-27T11:14:14.329 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-27T11:14:14.329 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-27T11:14:14.330 INFO:teuthology.orchestra.run.smithi138.stdout:Enabling module streams: 2023-08-27T11:14:14.330 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql 10 2023-08-27T11:14:14.330 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:14:14.330 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-08-27T11:14:14.330 INFO:teuthology.orchestra.run.smithi138.stdout:=========================================================================================================================== 2023-08-27T11:14:14.331 INFO:teuthology.orchestra.run.smithi138.stdout:Install 128 Packages 2023-08-27T11:14:14.331 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 3 Packages 2023-08-27T11:14:14.331 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:14:14.332 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 206 M 2023-08-27T11:14:14.332 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-08-27T11:14:14.404 INFO:teuthology.orchestra.run.smithi112.stdout:(5/131): ceph-mgr-18.0.0-5707.g1da267a3.el8.x86 9.4 MB/s | 1.6 MB 00:00 2023-08-27T11:14:14.762 INFO:teuthology.orchestra.run.smithi112.stdout:(6/131): ceph-mds-18.0.0-5707.g1da267a3.el8.x86 4.1 MB/s | 2.3 MB 00:00 2023-08-27T11:14:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:(7/131): ceph-mon-18.0.0-5707.g1da267a3.el8.x86 11 MB/s | 4.6 MB 00:00 2023-08-27T11:14:14.906 INFO:teuthology.orchestra.run.smithi138.stdout:(1/131): ceph-18.0.0-5707.g1da267a3.el8.x86_64. 30 kB/s | 6.4 kB 00:00 2023-08-27T11:14:16.063 INFO:teuthology.orchestra.run.smithi112.stdout:(8/131): ceph-radosgw-18.0.0-5707.g1da267a3.el8 11 MB/s | 14 MB 00:01 2023-08-27T11:14:16.105 INFO:teuthology.orchestra.run.smithi112.stdout:(9/131): ceph-selinux-18.0.0-5707.g1da267a3.el8 586 kB/s | 24 kB 00:00 2023-08-27T11:14:16.330 INFO:teuthology.orchestra.run.smithi112.stdout:(10/131): ceph-common-18.0.0-5707.g1da267a3.el8 8.0 MB/s | 22 MB 00:02 2023-08-27T11:14:16.390 INFO:teuthology.orchestra.run.smithi138.stdout:(2/131): ceph-fuse-18.0.0-5707.g1da267a3.el8.x8 584 kB/s | 867 kB 00:01 2023-08-27T11:14:16.422 INFO:teuthology.orchestra.run.smithi112.stdout:(11/131): cephfs-mirror-18.0.0-5707.g1da267a3.e 2.8 MB/s | 260 kB 00:00 2023-08-27T11:14:16.498 INFO:teuthology.orchestra.run.smithi112.stdout:(12/131): libcephfs-devel-18.0.0-5707.g1da267a3 413 kB/s | 31 kB 00:00 2023-08-27T11:14:16.632 INFO:teuthology.orchestra.run.smithi138.stdout:(3/131): ceph-immutable-object-cache-18.0.0-570 692 kB/s | 167 kB 00:00 2023-08-27T11:14:16.723 INFO:teuthology.orchestra.run.smithi112.stdout:(13/131): libcephfs2-18.0.0-5707.g1da267a3.el8. 3.4 MB/s | 789 kB 00:00 2023-08-27T11:14:16.815 INFO:teuthology.orchestra.run.smithi112.stdout:(14/131): libcephsqlite-18.0.0-5707.g1da267a3.e 1.5 MB/s | 139 kB 00:00 2023-08-27T11:14:16.907 INFO:teuthology.orchestra.run.smithi112.stdout:(15/131): librados-devel-18.0.0-5707.g1da267a3. 1.5 MB/s | 139 kB 00:00 2023-08-27T11:14:17.065 INFO:teuthology.orchestra.run.smithi112.stdout:(16/131): libradosstriper1-18.0.0-5707.g1da267a 2.9 MB/s | 473 kB 00:00 2023-08-27T11:14:17.799 INFO:teuthology.orchestra.run.smithi138.stdout:(4/131): ceph-base-18.0.0-5707.g1da267a3.el8.x8 1.7 MB/s | 5.4 MB 00:03 2023-08-27T11:14:18.466 INFO:teuthology.orchestra.run.smithi112.stdout:(17/131): librgw2-18.0.0-5707.g1da267a3.el8.x86 4.9 MB/s | 6.8 MB 00:01 2023-08-27T11:14:18.558 INFO:teuthology.orchestra.run.smithi112.stdout:(18/131): python3-ceph-argparse-18.0.0-5707.g1d 488 kB/s | 45 kB 00:00 2023-08-27T11:14:18.607 INFO:teuthology.orchestra.run.smithi138.stdout:(5/131): ceph-mgr-18.0.0-5707.g1da267a3.el8.x86 1.9 MB/s | 1.6 MB 00:00 2023-08-27T11:14:18.649 INFO:teuthology.orchestra.run.smithi112.stdout:(19/131): python3-ceph-common-18.0.0-5707.g1da2 1.3 MB/s | 126 kB 00:00 2023-08-27T11:14:18.741 INFO:teuthology.orchestra.run.smithi112.stdout:(20/131): python3-cephfs-18.0.0-5707.g1da267a3. 2.1 MB/s | 197 kB 00:00 2023-08-27T11:14:18.842 INFO:teuthology.orchestra.run.smithi112.stdout:(21/131): python3-rados-18.0.0-5707.g1da267a3.e 3.6 MB/s | 371 kB 00:00 2023-08-27T11:14:19.033 INFO:teuthology.orchestra.run.smithi138.stdout:(6/131): ceph-mds-18.0.0-5707.g1da267a3.el8.x86 967 kB/s | 2.3 MB 00:02 2023-08-27T11:14:19.284 INFO:teuthology.orchestra.run.smithi112.stdout:(22/131): ceph-test-18.0.0-5707.g1da267a3.el8.x 15 MB/s | 48 MB 00:03 2023-08-27T11:14:19.309 INFO:teuthology.orchestra.run.smithi112.stdout:(23/131): python3-rbd-18.0.0-5707.g1da267a3.el8 779 kB/s | 364 kB 00:00 2023-08-27T11:14:19.334 INFO:teuthology.orchestra.run.smithi112.stdout:(24/131): python3-rgw-18.0.0-5707.g1da267a3.el8 2.1 MB/s | 107 kB 00:00 2023-08-27T11:14:19.526 INFO:teuthology.orchestra.run.smithi112.stdout:(25/131): ceph-osd-18.0.0-5707.g1da267a3.el8.x8 3.7 MB/s | 18 MB 00:04 2023-08-27T11:14:19.543 INFO:teuthology.orchestra.run.smithi112.stdout:(26/131): rbd-fuse-18.0.0-5707.g1da267a3.el8.x8 400 kB/s | 94 kB 00:00 2023-08-27T11:14:19.635 INFO:teuthology.orchestra.run.smithi112.stdout:(27/131): rbd-mirror-18.0.0-5707.g1da267a3.el8. 12 MB/s | 3.5 MB 00:00 2023-08-27T11:14:19.660 INFO:teuthology.orchestra.run.smithi112.stdout:(28/131): ceph-grafana-dashboards-18.0.0-5707.g 197 kB/s | 23 kB 00:00 2023-08-27T11:14:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:(29/131): ceph-mgr-cephadm-18.0.0-5707.g1da267a 2.6 MB/s | 132 kB 00:00 2023-08-27T11:14:19.708 INFO:teuthology.orchestra.run.smithi138.stdout:(7/131): ceph-mon-18.0.0-5707.g1da267a3.el8.x86 4.2 MB/s | 4.6 MB 00:01 2023-08-27T11:14:19.711 INFO:teuthology.orchestra.run.smithi112.stdout:(30/131): rbd-nbd-18.0.0-5707.g1da267a3.el8.x86 973 kB/s | 179 kB 00:00 2023-08-27T11:14:19.770 INFO:teuthology.orchestra.run.smithi112.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5707.g1d 4.0 MB/s | 242 kB 00:00 2023-08-27T11:14:19.828 INFO:teuthology.orchestra.run.smithi112.stdout:(32/131): ceph-mgr-rook-18.0.0-5707.g1da267a3.e 814 kB/s | 47 kB 00:00 2023-08-27T11:14:19.887 INFO:teuthology.orchestra.run.smithi112.stdout:(33/131): ceph-prometheus-alerts-18.0.0-5707.g1 249 kB/s | 15 kB 00:00 2023-08-27T11:14:19.954 INFO:teuthology.orchestra.run.smithi112.stdout:(34/131): ceph-volume-18.0.0-5707.g1da267a3.el8 3.8 MB/s | 261 kB 00:00 2023-08-27T11:14:19.987 INFO:teuthology.orchestra.run.smithi112.stdout:(35/131): ceph-mgr-dashboard-18.0.0-5707.g1da26 5.4 MB/s | 1.7 MB 00:00 2023-08-27T11:14:20.013 INFO:teuthology.orchestra.run.smithi112.stdout:(36/131): cephadm-18.0.0-5707.g1da267a3.el8.noa 2.9 MB/s | 171 kB 00:00 2023-08-27T11:14:20.046 INFO:teuthology.orchestra.run.smithi112.stdout:(37/131): cephfs-top-18.0.0-5707.g1da267a3.el8. 308 kB/s | 18 kB 00:00 2023-08-27T11:14:20.388 INFO:teuthology.orchestra.run.smithi112.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 172 kB/s | 59 kB 00:00 2023-08-27T11:14:20.547 INFO:teuthology.orchestra.run.smithi112.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 574 kB/s | 306 kB 00:00 2023-08-27T11:14:20.610 INFO:teuthology.orchestra.run.smithi112.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 338 kB/s | 75 kB 00:00 2023-08-27T11:14:20.757 INFO:teuthology.orchestra.run.smithi112.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 299 kB/s | 44 kB 00:00 2023-08-27T11:14:20.815 INFO:teuthology.orchestra.run.smithi112.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-08-27T11:14:20.990 INFO:teuthology.orchestra.run.smithi112.stdout:(43/131): ceph-mgr-diskprediction-local-18.0.0- 5.7 MB/s | 7.4 MB 00:01 2023-08-27T11:14:21.016 INFO:teuthology.orchestra.run.smithi112.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 716 kB/s | 185 kB 00:00 2023-08-27T11:14:21.032 INFO:teuthology.orchestra.run.smithi112.stdout:(45/131): python3-cachetools-3.1.1-4.el8.noarch 152 kB/s | 33 kB 00:00 2023-08-27T11:14:21.141 INFO:teuthology.orchestra.run.smithi112.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2023-08-27T11:14:21.224 INFO:teuthology.orchestra.run.smithi112.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.0 MB/s | 384 kB 00:00 2023-08-27T11:14:21.242 INFO:teuthology.orchestra.run.smithi112.stdout:(48/131): python3-certifi-2018.10.15-7.el8.noar 65 kB/s | 16 kB 00:00 2023-08-27T11:14:21.267 INFO:teuthology.orchestra.run.smithi112.stdout:(49/131): python3-google-auth-1.1.1-10.el8.noar 647 kB/s | 81 kB 00:00 2023-08-27T11:14:21.309 INFO:teuthology.orchestra.run.smithi112.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 132 kB/s | 11 kB 00:00 2023-08-27T11:14:21.367 INFO:teuthology.orchestra.run.smithi112.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2023-08-27T11:14:21.434 INFO:teuthology.orchestra.run.smithi112.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 389 kB/s | 49 kB 00:00 2023-08-27T11:14:21.468 INFO:teuthology.orchestra.run.smithi112.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 5.8 MB/s | 1.2 MB 00:00 2023-08-27T11:14:21.535 INFO:teuthology.orchestra.run.smithi112.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 602 kB/s | 60 kB 00:00 2023-08-27T11:14:21.577 INFO:teuthology.orchestra.run.smithi112.stdout:(55/131): python3-more-itertools-7.2.0-3.el8.no 281 kB/s | 59 kB 00:00 2023-08-27T11:14:21.602 INFO:teuthology.orchestra.run.smithi112.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-08-27T11:14:21.616 INFO:teuthology.orchestra.run.smithi138.stdout:(8/131): ceph-radosgw-18.0.0-5707.g1da267a3.el8 7.1 MB/s | 14 MB 00:01 2023-08-27T11:14:21.636 INFO:teuthology.orchestra.run.smithi112.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-08-27T11:14:21.667 INFO:teuthology.orchestra.run.smithi138.stdout:(9/131): ceph-selinux-18.0.0-5707.g1da267a3.el8 488 kB/s | 24 kB 00:00 2023-08-27T11:14:21.720 INFO:teuthology.orchestra.run.smithi112.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 236 kB/s | 34 kB 00:00 2023-08-27T11:14:21.745 INFO:teuthology.orchestra.run.smithi112.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.3 MB/s | 196 kB 00:00 2023-08-27T11:14:21.770 INFO:teuthology.orchestra.run.smithi112.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 480 kB/s | 64 kB 00:00 2023-08-27T11:14:21.853 INFO:teuthology.orchestra.run.smithi112.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 140 kB/s | 19 kB 00:00 2023-08-27T11:14:21.879 INFO:teuthology.orchestra.run.smithi112.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 179 kB/s | 24 kB 00:00 2023-08-27T11:14:21.904 INFO:teuthology.orchestra.run.smithi112.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 218 kB/s | 29 kB 00:00 2023-08-27T11:14:21.996 INFO:teuthology.orchestra.run.smithi112.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 193 kB/s | 27 kB 00:00 2023-08-27T11:14:22.021 INFO:teuthology.orchestra.run.smithi112.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.8 MB/s | 255 kB 00:00 2023-08-27T11:14:22.080 INFO:teuthology.orchestra.run.smithi112.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.4 MB/s | 251 kB 00:00 2023-08-27T11:14:22.172 INFO:teuthology.orchestra.run.smithi112.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 568 kB/s | 85 kB 00:00 2023-08-27T11:14:22.197 INFO:teuthology.orchestra.run.smithi112.stdout:(68/131): python3-websocket-client-0.56.0-5.el8 305 kB/s | 61 kB 00:00 2023-08-27T11:14:22.223 INFO:teuthology.orchestra.run.smithi112.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 161 kB/s | 23 kB 00:00 2023-08-27T11:14:22.248 INFO:teuthology.orchestra.run.smithi112.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-08-27T11:14:22.348 INFO:teuthology.orchestra.run.smithi112.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-08-27T11:14:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 723 kB/s | 54 kB 00:00 2023-08-27T11:14:22.549 INFO:teuthology.orchestra.run.smithi112.stdout:(73/131): thrift-0.13.0-2.el8.x86_64.rpm 4.6 MB/s | 1.7 MB 00:00 2023-08-27T11:14:22.574 INFO:teuthology.orchestra.run.smithi112.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.7 MB/s | 259 kB 00:00 2023-08-27T11:14:22.749 INFO:teuthology.orchestra.run.smithi112.stdout:(75/131): python3-scipy-1.0.0-21.module+el8.5.0 26 MB/s | 14 MB 00:00 2023-08-27T11:14:22.774 INFO:teuthology.orchestra.run.smithi112.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 199 kB/s | 39 kB 00:00 2023-08-27T11:14:22.800 INFO:teuthology.orchestra.run.smithi112.stdout:(77/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2023-08-27T11:14:22.883 INFO:teuthology.orchestra.run.smithi112.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 5.0 MB/s | 688 kB 00:00 2023-08-27T11:14:22.909 INFO:teuthology.orchestra.run.smithi112.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.4 MB/s | 157 kB 00:00 2023-08-27T11:14:22.942 INFO:teuthology.orchestra.run.smithi112.stdout:(80/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.1 MB/s | 538 kB 00:00 2023-08-27T11:14:23.043 INFO:teuthology.orchestra.run.smithi112.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2023-08-27T11:14:23.084 INFO:teuthology.orchestra.run.smithi112.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 4.0 MB/s | 713 kB 00:00 2023-08-27T11:14:23.110 INFO:teuthology.orchestra.run.smithi112.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 397 kB/s | 27 kB 00:00 2023-08-27T11:14:23.143 INFO:teuthology.orchestra.run.smithi112.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 5.3 MB/s | 320 kB 00:00 2023-08-27T11:14:23.160 INFO:teuthology.orchestra.run.smithi112.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 879 kB/s | 44 kB 00:00 2023-08-27T11:14:23.277 INFO:teuthology.orchestra.run.smithi112.stdout:(86/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-08-27T11:14:23.311 INFO:teuthology.orchestra.run.smithi112.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 3.0 MB/s | 457 kB 00:00 2023-08-27T11:14:23.386 INFO:teuthology.orchestra.run.smithi112.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.0 MB/s | 225 kB 00:00 2023-08-27T11:14:23.461 INFO:teuthology.orchestra.run.smithi112.stdout:(89/131): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2023-08-27T11:14:23.478 INFO:teuthology.orchestra.run.smithi112.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 383 kB/s | 64 kB 00:00 2023-08-27T11:14:23.521 INFO:teuthology.orchestra.run.smithi112.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 342 kB/s | 20 kB 00:00 2023-08-27T11:14:23.546 INFO:teuthology.orchestra.run.smithi112.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.5 MB/s | 103 kB 00:00 2023-08-27T11:14:23.563 INFO:teuthology.orchestra.run.smithi112.stdout:(93/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 718 kB/s | 126 kB 00:00 2023-08-27T11:14:23.713 INFO:teuthology.orchestra.run.smithi112.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 1.3 MB/s | 198 kB 00:00 2023-08-27T11:14:23.738 INFO:teuthology.orchestra.run.smithi112.stdout:(95/131): libtool-2.4.6-25.el8.x86_64.rpm 3.2 MB/s | 709 kB 00:00 2023-08-27T11:14:23.838 INFO:teuthology.orchestra.run.smithi112.stdout:(96/131): openblas-threads-0.3.15-3.el8.x86_64. 17 MB/s | 4.9 MB 00:00 2023-08-27T11:14:23.881 INFO:teuthology.orchestra.run.smithi112.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2023-08-27T11:14:23.922 INFO:teuthology.orchestra.run.smithi112.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 4.1 MB/s | 354 kB 00:00 2023-08-27T11:14:24.023 INFO:teuthology.orchestra.run.smithi112.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-08-27T11:14:24.090 INFO:teuthology.orchestra.run.smithi112.stdout:(100/131): postgresql-contrib-10.21-2.module+el 4.7 MB/s | 808 kB 00:00 2023-08-27T11:14:24.165 INFO:teuthology.orchestra.run.smithi112.stdout:(101/131): postgresql-server-10.21-2.module+el8 18 MB/s | 5.1 MB 00:00 2023-08-27T11:14:24.215 INFO:teuthology.orchestra.run.smithi112.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 7.8 MB/s | 1.5 MB 00:00 2023-08-27T11:14:24.249 INFO:teuthology.orchestra.run.smithi112.stdout:(103/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 295 kB/s | 47 kB 00:00 2023-08-27T11:14:24.332 INFO:teuthology.orchestra.run.smithi112.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.2 MB/s | 200 kB 00:00 2023-08-27T11:14:24.358 INFO:teuthology.orchestra.run.smithi112.stdout:(105/131): python3-jwt-1.6.1-2.el8.noarch.rpm 299 kB/s | 43 kB 00:00 2023-08-27T11:14:24.400 INFO:teuthology.orchestra.run.smithi112.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2023-08-27T11:14:24.550 INFO:teuthology.orchestra.run.smithi112.stdout:(107/131): keyutils-libs-devel-1.5.10-9.el8.x86 221 kB/s | 48 kB 00:00 2023-08-27T11:14:24.650 INFO:teuthology.orchestra.run.smithi112.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 298 kB/s | 87 kB 00:00 2023-08-27T11:14:24.676 INFO:teuthology.orchestra.run.smithi112.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 155 kB/s | 43 kB 00:00 2023-08-27T11:14:24.701 INFO:teuthology.orchestra.run.smithi112.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 725 kB/s | 109 kB 00:00 2023-08-27T11:14:24.759 INFO:teuthology.orchestra.run.smithi112.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2023-08-27T11:14:24.868 INFO:teuthology.orchestra.run.smithi112.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.8 MB/s | 560 kB 00:00 2023-08-27T11:14:24.902 INFO:teuthology.orchestra.run.smithi112.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 93 kB/s | 19 kB 00:00 2023-08-27T11:14:24.993 INFO:teuthology.orchestra.run.smithi138.stdout:(10/131): ceph-osd-18.0.0-5707.g1da267a3.el8.x8 3.0 MB/s | 18 MB 00:05 2023-08-27T11:14:25.010 INFO:teuthology.orchestra.run.smithi112.stdout:(114/131): libverto-devel-0.3.0-5.el8.x86_64.rp 165 kB/s | 18 kB 00:00 2023-08-27T11:14:25.036 INFO:teuthology.orchestra.run.smithi112.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 3.3 MB/s | 559 kB 00:00 2023-08-27T11:14:25.061 INFO:teuthology.orchestra.run.smithi112.stdout:(116/131): python3-oauthlib-2.1.0-1.el8.noarch. 515 kB/s | 155 kB 00:00 2023-08-27T11:14:25.161 INFO:teuthology.orchestra.run.smithi112.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.9 MB/s | 238 kB 00:00 2023-08-27T11:14:25.203 INFO:teuthology.orchestra.run.smithi112.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.1 MB/s | 605 kB 00:00 2023-08-27T11:14:25.242 INFO:teuthology.orchestra.run.smithi138.stdout:(11/131): ceph-common-18.0.0-5707.g1da267a3.el8 2.1 MB/s | 22 MB 00:10 2023-08-27T11:14:25.268 INFO:teuthology.orchestra.run.smithi138.stdout:(12/131): cephfs-mirror-18.0.0-5707.g1da267a3.e 946 kB/s | 260 kB 00:00 2023-08-27T11:14:25.301 INFO:teuthology.orchestra.run.smithi138.stdout:(13/131): libcephfs-devel-18.0.0-5707.g1da267a3 534 kB/s | 31 kB 00:00 2023-08-27T11:14:25.303 INFO:teuthology.orchestra.run.smithi112.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 9.6 MB/s | 2.3 MB 00:00 2023-08-27T11:14:25.328 INFO:teuthology.orchestra.run.smithi112.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 432 kB/s | 72 kB 00:00 2023-08-27T11:14:25.362 INFO:teuthology.orchestra.run.smithi112.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 521 kB/s | 83 kB 00:00 2023-08-27T11:14:25.401 INFO:teuthology.orchestra.run.smithi138.stdout:(14/131): libcephsqlite-18.0.0-5707.g1da267a3.e 1.4 MB/s | 139 kB 00:00 2023-08-27T11:14:25.437 INFO:teuthology.orchestra.run.smithi112.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 904 kB/s | 98 kB 00:00 2023-08-27T11:14:25.454 INFO:teuthology.orchestra.run.smithi112.stdout:(123/131): elfutils-libelf-devel-0.186-1.el8.x8 397 kB/s | 60 kB 00:00 2023-08-27T11:14:25.471 INFO:teuthology.orchestra.run.smithi112.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 531 kB/s | 58 kB 00:00 2023-08-27T11:14:25.476 INFO:teuthology.orchestra.run.smithi138.stdout:(15/131): librados-devel-18.0.0-5707.g1da267a3. 1.8 MB/s | 139 kB 00:00 2023-08-27T11:14:25.502 INFO:teuthology.orchestra.run.smithi138.stdout:(16/131): libcephfs2-18.0.0-5707.g1da267a3.el8. 3.3 MB/s | 789 kB 00:00 2023-08-27T11:14:25.571 INFO:teuthology.orchestra.run.smithi112.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 5.4 MB/s | 644 kB 00:00 2023-08-27T11:14:25.588 INFO:teuthology.orchestra.run.smithi112.stdout:(126/131): libcom_err-devel-1.45.6-4.el8.x86_64 256 kB/s | 39 kB 00:00 2023-08-27T11:14:25.615 INFO:teuthology.orchestra.run.smithi112.stdout:(127/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.2 MB/s | 171 kB 00:00 2023-08-27T11:14:25.619 INFO:teuthology.orchestra.run.smithi138.stdout:(17/131): libradosstriper1-18.0.0-5707.g1da267a 3.3 MB/s | 473 kB 00:00 2023-08-27T11:14:25.711 INFO:teuthology.orchestra.run.smithi138.stdout:(18/131): python3-ceph-argparse-18.0.0-5707.g1d 487 kB/s | 45 kB 00:00 2023-08-27T11:14:25.773 INFO:teuthology.orchestra.run.smithi112.stdout:(128/131): NetworkManager-initscripts-updown-1. 685 kB/s | 138 kB 00:00 2023-08-27T11:14:25.779 INFO:teuthology.orchestra.run.smithi138.stdout:(19/131): python3-ceph-common-18.0.0-5707.g1da2 1.8 MB/s | 126 kB 00:00 2023-08-27T11:14:25.981 INFO:teuthology.orchestra.run.smithi112.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 3.7 MB/s | 798 kB 00:00 2023-08-27T11:14:26.040 INFO:teuthology.orchestra.run.smithi112.stdout:(130/131): librbd1-18.0.0-5707.g1da267a3.el8.x8 8.5 MB/s | 3.6 MB 00:00 2023-08-27T11:14:26.262 INFO:teuthology.orchestra.run.smithi138.stdout:(20/131): ceph-test-18.0.0-5707.g1da267a3.el8.x 10 MB/s | 48 MB 00:04 2023-08-27T11:14:26.287 INFO:teuthology.orchestra.run.smithi138.stdout:(21/131): python3-cephfs-18.0.0-5707.g1da267a3. 387 kB/s | 197 kB 00:00 2023-08-27T11:14:26.337 INFO:teuthology.orchestra.run.smithi138.stdout:(22/131): python3-rados-18.0.0-5707.g1da267a3.e 4.8 MB/s | 371 kB 00:00 2023-08-27T11:14:26.379 INFO:teuthology.orchestra.run.smithi138.stdout:(23/131): python3-rgw-18.0.0-5707.g1da267a3.el8 2.5 MB/s | 107 kB 00:00 2023-08-27T11:14:26.421 INFO:teuthology.orchestra.run.smithi138.stdout:(24/131): rbd-fuse-18.0.0-5707.g1da267a3.el8.x8 2.2 MB/s | 94 kB 00:00 2023-08-27T11:14:26.549 INFO:teuthology.orchestra.run.smithi112.stdout:(131/131): librados2-18.0.0-5707.g1da267a3.el8. 3.9 MB/s | 3.7 MB 00:00 2023-08-27T11:14:26.552 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-08-27T11:14:26.552 INFO:teuthology.orchestra.run.smithi112.stdout:Total 16 MB/s | 206 MB 00:13 2023-08-27T11:14:26.655 INFO:teuthology.orchestra.run.smithi138.stdout:(25/131): python3-rbd-18.0.0-5707.g1da267a3.el8 989 kB/s | 364 kB 00:00 2023-08-27T11:14:26.730 INFO:teuthology.orchestra.run.smithi138.stdout:(26/131): rbd-nbd-18.0.0-5707.g1da267a3.el8.x86 2.3 MB/s | 179 kB 00:00 2023-08-27T11:14:26.814 INFO:teuthology.orchestra.run.smithi138.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5707.g 277 kB/s | 23 kB 00:00 2023-08-27T11:14:26.872 INFO:teuthology.orchestra.run.smithi138.stdout:(28/131): rbd-mirror-18.0.0-5707.g1da267a3.el8. 7.8 MB/s | 3.5 MB 00:00 2023-08-27T11:14:26.898 INFO:teuthology.orchestra.run.smithi138.stdout:(29/131): ceph-mgr-cephadm-18.0.0-5707.g1da267a 1.5 MB/s | 132 kB 00:00 2023-08-27T11:14:27.048 INFO:teuthology.orchestra.run.smithi138.stdout:(30/131): ceph-mgr-dashboard-18.0.0-5707.g1da26 10 MB/s | 1.7 MB 00:00 2023-08-27T11:14:27.107 INFO:teuthology.orchestra.run.smithi138.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5707.g1d 4.1 MB/s | 242 kB 00:00 2023-08-27T11:14:27.157 INFO:teuthology.orchestra.run.smithi138.stdout:(32/131): ceph-mgr-rook-18.0.0-5707.g1da267a3.e 952 kB/s | 47 kB 00:00 2023-08-27T11:14:27.207 INFO:teuthology.orchestra.run.smithi138.stdout:(33/131): ceph-prometheus-alerts-18.0.0-5707.g1 291 kB/s | 15 kB 00:00 2023-08-27T11:14:27.266 INFO:teuthology.orchestra.run.smithi138.stdout:(34/131): ceph-volume-18.0.0-5707.g1da267a3.el8 4.4 MB/s | 261 kB 00:00 2023-08-27T11:14:27.326 INFO:teuthology.orchestra.run.smithi138.stdout:(35/131): cephadm-18.0.0-5707.g1da267a3.el8.noa 2.9 MB/s | 171 kB 00:00 2023-08-27T11:14:27.375 INFO:teuthology.orchestra.run.smithi138.stdout:(36/131): cephfs-top-18.0.0-5707.g1da267a3.el8. 360 kB/s | 18 kB 00:00 2023-08-27T11:14:27.377 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-08-27T11:14:27.495 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-08-27T11:14:27.495 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-08-27T11:14:27.542 INFO:teuthology.orchestra.run.smithi138.stdout:(37/131): librgw2-18.0.0-5707.g1da267a3.el8.x86 3.3 MB/s | 6.8 MB 00:02 2023-08-27T11:14:27.834 INFO:teuthology.orchestra.run.smithi138.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 668 kB/s | 306 kB 00:00 2023-08-27T11:14:27.859 INFO:teuthology.orchestra.run.smithi138.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 185 kB/s | 59 kB 00:00 2023-08-27T11:14:27.926 INFO:teuthology.orchestra.run.smithi138.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 817 kB/s | 75 kB 00:00 2023-08-27T11:14:28.026 INFO:teuthology.orchestra.run.smithi138.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 436 kB/s | 44 kB 00:00 2023-08-27T11:14:28.129 INFO:teuthology.orchestra.run.smithi138.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-08-27T11:14:28.194 INFO:teuthology.orchestra.run.smithi138.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.4 MB/s | 476 kB 00:00 2023-08-27T11:14:28.228 INFO:teuthology.orchestra.run.smithi138.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 329 kB/s | 33 kB 00:00 2023-08-27T11:14:28.286 INFO:teuthology.orchestra.run.smithi138.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 178 kB/s | 16 kB 00:00 2023-08-27T11:14:28.328 INFO:teuthology.orchestra.run.smithi138.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-08-27T11:14:28.420 INFO:teuthology.orchestra.run.smithi138.stdout:(47/131): ceph-mgr-diskprediction-local-18.0.0- 4.9 MB/s | 7.4 MB 00:01 2023-08-27T11:14:28.446 INFO:teuthology.orchestra.run.smithi138.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 694 kB/s | 81 kB 00:00 2023-08-27T11:14:28.471 INFO:teuthology.orchestra.run.smithi138.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.0 MB/s | 384 kB 00:00 2023-08-27T11:14:28.538 INFO:teuthology.orchestra.run.smithi138.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 191 kB/s | 18 kB 00:00 2023-08-27T11:14:28.588 INFO:teuthology.orchestra.run.smithi138.stdout:(51/131): python3-jaraco-6.2-6.el8.noarch.rpm 66 kB/s | 11 kB 00:00 2023-08-27T11:14:28.614 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-08-27T11:14:28.618 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-08-27T11:14:28.630 INFO:teuthology.orchestra.run.smithi138.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 530 kB/s | 49 kB 00:00 2023-08-27T11:14:28.672 INFO:teuthology.orchestra.run.smithi138.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 5.8 MB/s | 1.2 MB 00:00 2023-08-27T11:14:28.722 INFO:teuthology.orchestra.run.smithi138.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 655 kB/s | 60 kB 00:00 2023-08-27T11:14:28.781 INFO:teuthology.orchestra.run.smithi138.stdout:(55/131): python3-more-itertools-7.2.0-3.el8.no 304 kB/s | 59 kB 00:00 2023-08-27T11:14:28.806 INFO:teuthology.orchestra.run.smithi138.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-08-27T11:14:28.832 INFO:teuthology.orchestra.run.smithi138.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 143 kB/s | 16 kB 00:00 2023-08-27T11:14:28.874 INFO:teuthology.orchestra.run.smithi138.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 364 kB/s | 34 kB 00:00 2023-08-27T11:14:28.899 INFO:teuthology.orchestra.run.smithi138.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2023-08-27T11:14:28.924 INFO:teuthology.orchestra.run.smithi138.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 697 kB/s | 64 kB 00:00 2023-08-27T11:14:28.966 INFO:teuthology.orchestra.run.smithi138.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 203 kB/s | 19 kB 00:00 2023-08-27T11:14:29.008 INFO:teuthology.orchestra.run.smithi138.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2023-08-27T11:14:29.034 INFO:teuthology.orchestra.run.smithi138.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 268 kB/s | 29 kB 00:00 2023-08-27T11:14:29.067 INFO:teuthology.orchestra.run.smithi138.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 273 kB/s | 27 kB 00:00 2023-08-27T11:14:29.109 INFO:teuthology.orchestra.run.smithi138.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-08-27T11:14:29.134 INFO:teuthology.orchestra.run.smithi138.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-08-27T11:14:29.168 INFO:teuthology.orchestra.run.smithi138.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 609 kB/s | 61 kB 00:00 2023-08-27T11:14:29.202 INFO:teuthology.orchestra.run.smithi138.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 925 kB/s | 85 kB 00:00 2023-08-27T11:14:29.228 INFO:teuthology.orchestra.run.smithi138.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 248 kB/s | 23 kB 00:00 2023-08-27T11:14:29.252 INFO:teuthology.orchestra.run.smithi138.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-08-27T11:14:29.328 INFO:teuthology.orchestra.run.smithi138.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 466 kB/s | 47 kB 00:00 2023-08-27T11:14:29.345 INFO:teuthology.orchestra.run.smithi138.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.1 MB/s | 200 kB 00:00 2023-08-27T11:14:29.370 INFO:teuthology.orchestra.run.smithi138.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-08-27T11:14:29.395 INFO:teuthology.orchestra.run.smithi138.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-08-27T11:14:29.479 INFO:teuthology.orchestra.run.smithi138.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 443 kB/s | 48 kB 00:00 2023-08-27T11:14:29.521 INFO:teuthology.orchestra.run.smithi138.stdout:(76/131): thrift-0.13.0-2.el8.x86_64.rpm 4.9 MB/s | 1.7 MB 00:00 2023-08-27T11:14:29.579 INFO:teuthology.orchestra.run.smithi138.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 473 kB/s | 87 kB 00:00 2023-08-27T11:14:29.680 INFO:teuthology.orchestra.run.smithi138.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 688 kB/s | 109 kB 00:00 2023-08-27T11:14:29.705 INFO:teuthology.orchestra.run.smithi138.stdout:(79/131): python3-requests-oauthlib-1.0.0-1.el8 189 kB/s | 43 kB 00:00 2023-08-27T11:14:29.730 INFO:teuthology.orchestra.run.smithi138.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2023-08-27T11:14:29.795 INFO:teuthology.orchestra.run.smithi138.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.7 MB/s | 560 kB 00:00 2023-08-27T11:14:29.831 INFO:teuthology.orchestra.run.smithi138.stdout:(82/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 149 kB/s | 19 kB 00:00 2023-08-27T11:14:29.906 INFO:teuthology.orchestra.run.smithi138.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 883 kB/s | 155 kB 00:00 2023-08-27T11:14:29.931 INFO:teuthology.orchestra.run.smithi138.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 178 kB/s | 18 kB 00:00 2023-08-27T11:14:29.990 INFO:teuthology.orchestra.run.smithi138.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 2.8 MB/s | 559 kB 00:00 2023-08-27T11:14:30.032 INFO:teuthology.orchestra.run.smithi138.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-08-27T11:14:30.057 INFO:teuthology.orchestra.run.smithi138.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.8 MB/s | 238 kB 00:00 2023-08-27T11:14:30.124 INFO:teuthology.orchestra.run.smithi138.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 787 kB/s | 72 kB 00:00 2023-08-27T11:14:30.216 INFO:teuthology.orchestra.run.smithi138.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 10 MB/s | 2.3 MB 00:00 2023-08-27T11:14:30.242 INFO:teuthology.orchestra.run.smithi138.stdout:(90/131): fuse-2.9.7-15.el8.x86_64.rpm 449 kB/s | 83 kB 00:00 2023-08-27T11:14:30.267 INFO:teuthology.orchestra.run.smithi138.stdout:(91/131): elfutils-libelf-devel-0.186-1.el8.x86 421 kB/s | 60 kB 00:00 2023-08-27T11:14:30.367 INFO:teuthology.orchestra.run.smithi138.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 652 kB/s | 98 kB 00:00 2023-08-27T11:14:30.384 INFO:teuthology.orchestra.run.smithi138.stdout:(93/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 406 kB/s | 58 kB 00:00 2023-08-27T11:14:30.402 INFO:teuthology.orchestra.run.smithi138.stdout:(94/131): libcom_err-devel-1.45.6-4.el8.x86_64. 288 kB/s | 39 kB 00:00 2023-08-27T11:14:30.501 INFO:teuthology.orchestra.run.smithi138.stdout:(95/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.7 MB/s | 644 kB 00:00 2023-08-27T11:14:30.535 INFO:teuthology.orchestra.run.smithi138.stdout:(96/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.1 MB/s | 171 kB 00:00 2023-08-27T11:14:30.560 INFO:teuthology.orchestra.run.smithi138.stdout:(97/131): NetworkManager-initscripts-updown-1.3 867 kB/s | 138 kB 00:00 2023-08-27T11:14:30.686 INFO:teuthology.orchestra.run.smithi138.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 433 kB/s | 54 kB 00:00 2023-08-27T11:14:30.720 INFO:teuthology.orchestra.run.smithi138.stdout:(99/131): python3-pyasn1-modules-0.3.7-6.el8.no 598 kB/s | 110 kB 00:00 2023-08-27T11:14:30.828 INFO:teuthology.orchestra.run.smithi138.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.8 MB/s | 259 kB 00:00 2023-08-27T11:14:30.867 INFO:teuthology.orchestra.run.smithi138.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.2 MB/s | 323 kB 00:00 2023-08-27T11:14:31.048 INFO:teuthology.orchestra.run.smithi138.stdout:(102/131): python3-scipy-1.0.0-21.module+el8.5. 25 MB/s | 14 MB 00:00 2023-08-27T11:14:31.070 INFO:teuthology.orchestra.run.smithi138.stdout:(103/131): python3-markupsafe-0.23-19.el8.x86_6 160 kB/s | 39 kB 00:00 2023-08-27T11:14:31.104 INFO:teuthology.orchestra.run.smithi138.stdout:(104/131): bison-3.0.4-10.el8.x86_64.rpm 2.8 MB/s | 688 kB 00:00 2023-08-27T11:14:31.155 INFO:teuthology.orchestra.run.smithi138.stdout:(105/131): python3-jinja2-2.10.1-3.el8.noarch.r 4.8 MB/s | 538 kB 00:00 2023-08-27T11:14:31.205 INFO:teuthology.orchestra.run.smithi138.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2023-08-27T11:14:31.223 INFO:teuthology.orchestra.run.smithi138.stdout:(107/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2023-08-27T11:14:31.289 INFO:teuthology.orchestra.run.smithi138.stdout:(108/131): automake-1.16.1-7.el8.noarch.rpm 5.2 MB/s | 713 kB 00:00 2023-08-27T11:14:31.356 INFO:teuthology.orchestra.run.smithi138.stdout:(109/131): python3-jsonpatch-1.21-2.el8.noarch. 199 kB/s | 27 kB 00:00 2023-08-27T11:14:31.423 INFO:teuthology.orchestra.run.smithi138.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2023-08-27T11:14:31.465 INFO:teuthology.orchestra.run.smithi138.stdout:(111/131): flex-2.6.1-9.el8.x86_64.rpm 1.8 MB/s | 320 kB 00:00 2023-08-27T11:14:31.490 INFO:teuthology.orchestra.run.smithi138.stdout:(112/131): python3-prettytable-0.7.2-14.el8.noa 328 kB/s | 44 kB 00:00 2023-08-27T11:14:31.607 INFO:teuthology.orchestra.run.smithi138.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 20 MB/s | 3.7 MB 00:00 2023-08-27T11:14:31.640 INFO:teuthology.orchestra.run.smithi138.stdout:(114/131): python3-werkzeug-0.12.2-4.el8.noarch 2.5 MB/s | 457 kB 00:00 2023-08-27T11:14:31.666 INFO:teuthology.orchestra.run.smithi138.stdout:(115/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.3 MB/s | 225 kB 00:00 2023-08-27T11:14:31.732 INFO:teuthology.orchestra.run.smithi138.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 510 kB/s | 64 kB 00:00 2023-08-27T11:14:31.783 INFO:teuthology.orchestra.run.smithi138.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 171 kB/s | 20 kB 00:00 2023-08-27T11:14:31.833 INFO:teuthology.orchestra.run.smithi138.stdout:(118/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 656 kB/s | 126 kB 00:00 2023-08-27T11:14:31.858 INFO:teuthology.orchestra.run.smithi138.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 823 kB/s | 103 kB 00:00 2023-08-27T11:14:31.917 INFO:teuthology.orchestra.run.smithi138.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2023-08-27T11:14:32.018 INFO:teuthology.orchestra.run.smithi138.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-08-27T11:14:32.268 INFO:teuthology.orchestra.run.smithi138.stdout:(122/131): openblas-threads-0.3.15-3.el8.x86_64 11 MB/s | 4.9 MB 00:00 2023-08-27T11:14:32.368 INFO:teuthology.orchestra.run.smithi138.stdout:(123/131): jq-1.6-3.el8.x86_64.rpm 577 kB/s | 202 kB 00:00 2023-08-27T11:14:32.435 INFO:teuthology.orchestra.run.smithi138.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.1 MB/s | 354 kB 00:00 2023-08-27T11:14:32.502 INFO:teuthology.orchestra.run.smithi138.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 8.1 MB/s | 4.7 MB 00:00 2023-08-27T11:14:32.636 INFO:teuthology.orchestra.run.smithi138.stdout:(126/131): postgresql-server-10.21-2.module+el8 19 MB/s | 5.1 MB 00:00 2023-08-27T11:14:32.669 INFO:teuthology.orchestra.run.smithi138.stdout:(127/131): postgresql-contrib-10.21-2.module+el 3.4 MB/s | 808 kB 00:00 2023-08-27T11:14:32.720 INFO:teuthology.orchestra.run.smithi138.stdout:(128/131): postgresql-10.21-2.module+el8.6.0+15 6.9 MB/s | 1.5 MB 00:00 2023-08-27T11:14:32.937 INFO:teuthology.orchestra.run.smithi138.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 3.6 MB/s | 798 kB 00:00 2023-08-27T11:14:33.087 INFO:teuthology.orchestra.run.smithi138.stdout:(130/131): librbd1-18.0.0-5707.g1da267a3.el8.x8 8.7 MB/s | 3.6 MB 00:00 2023-08-27T11:14:33.487 INFO:teuthology.orchestra.run.smithi138.stdout:(131/131): librados2-18.0.0-5707.g1da267a3.el8. 4.4 MB/s | 3.7 MB 00:00 2023-08-27T11:14:33.490 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-08-27T11:14:33.490 INFO:teuthology.orchestra.run.smithi138.stdout:Total 11 MB/s | 206 MB 00:19 2023-08-27T11:14:33.813 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-08-27T11:14:33.994 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-27T11:14:34.173 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-27T11:14:34.353 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-08-27T11:14:34.427 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-08-27T11:14:34.473 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-08-27T11:14:34.473 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-08-27T11:14:34.604 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-27T11:14:34.729 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-27T11:14:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-27T11:14:35.358 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-27T11:14:35.594 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-08-27T11:14:35.598 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 6/134 2023-08-27T11:14:35.598 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-08-27T11:14:35.994 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 6/134 2023-08-27T11:14:36.229 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 7/134 2023-08-27T11:14:36.358 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 7/134 2023-08-27T11:14:36.545 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 8/134 2023-08-27T11:14:36.630 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 8/134 2023-08-27T11:14:36.864 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:18.0.0-5707.g1da267a3.el8.x86_64 9/134 2023-08-27T11:14:37.177 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-27T11:14:37.398 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-27T11:14:37.832 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-27T11:14:38.115 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-27T11:14:38.279 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 14/134 2023-08-27T11:14:38.385 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 14/134 2023-08-27T11:14:38.554 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86 15/134 2023-08-27T11:14:38.629 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86 15/134 2023-08-27T11:14:38.787 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-27T11:14:38.916 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-27T11:14:39.079 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-27T11:14:39.690 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-27T11:14:39.870 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-27T11:14:40.465 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-27T11:14:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-08-27T11:14:40.670 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-08-27T11:14:40.696 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-27T11:14:40.896 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-27T11:14:41.107 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-27T11:14:41.243 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-27T11:14:41.385 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-08-27T11:14:41.448 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-27T11:14:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-08-27T11:14:41.855 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-27T11:14:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-08-27T11:14:42.076 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 5/134 2023-08-27T11:14:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-08-27T11:14:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 5/134 2023-08-27T11:14:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-08-27T11:14:42.648 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 6/134 2023-08-27T11:14:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-08-27T11:14:42.776 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 6/134 2023-08-27T11:14:42.891 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-27T11:14:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:18.0.0-5707.g1da267a3.el8. 26/134 2023-08-27T11:14:43.164 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:18.0.0-5707.g1da267a3.el 27/134 2023-08-27T11:14:43.289 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 7/134 2023-08-27T11:14:43.394 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 7/134 2023-08-27T11:14:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:18.0.0-5707.g1da267a3.el8.x86_64 8/134 2023-08-27T11:14:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-08-27T11:14:44.018 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-27T11:14:44.093 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:18.0.0-5707.g1da267a3.el8.x86_6 28/134 2023-08-27T11:14:44.269 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-27T11:14:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-08-27T11:14:44.666 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-08-27T11:14:44.692 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-27T11:14:44.921 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-08-27T11:14:44.943 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-27T11:14:45.122 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 14/134 2023-08-27T11:14:45.144 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-27T11:14:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 14/134 2023-08-27T11:14:45.406 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86 15/134 2023-08-27T11:14:45.551 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86 15/134 2023-08-27T11:14:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-08-27T11:14:45.928 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-08-27T11:14:46.114 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-08-27T11:14:46.280 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-27T11:14:46.425 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-27T11:14:46.572 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-27T11:14:46.903 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-27T11:14:47.174 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-27T11:14:47.212 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-08-27T11:14:47.381 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-27T11:14:47.414 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-27T11:14:47.612 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-27T11:14:47.981 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-27T11:14:48.133 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:18.0.0-5707.g1da267a3.el8.x86_64 36/134 2023-08-27T11:14:48.180 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-08-27T11:14:48.318 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:18.0.0-5707.g1da267a3.el8.x86_6 37/134 2023-08-27T11:14:48.486 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-08-27T11:14:48.666 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-08-27T11:14:48.855 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-08-27T11:14:48.901 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-08-27T11:14:49.005 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-08-27T11:14:49.191 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-08-27T11:14:49.199 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-08-27T11:14:49.391 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-08-27T11:14:49.412 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-08-27T11:14:49.632 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-27T11:14:49.772 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-08-27T11:14:49.820 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:18.0.0-5707.g1da267a3.el8. 26/134 2023-08-27T11:14:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:18.0.0-5707.g1da267a3.el 27/134 2023-08-27T11:14:50.030 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-08-27T11:14:50.202 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:18.0.0-5707.g1da267a3.el8.x86_6 28/134 2023-08-27T11:14:50.273 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-08-27T11:14:50.431 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-27T11:14:50.490 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-08-27T11:14:50.731 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-08-27T11:14:51.059 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-08-27T11:14:51.318 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-08-27T11:14:51.539 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-08-27T11:14:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-08-27T11:14:51.898 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-08-27T11:14:52.071 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-08-27T11:14:52.184 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-27T11:14:52.427 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-08-27T11:14:52.689 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-08-27T11:14:52.939 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-08-27T11:14:53.167 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-08-27T11:14:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-08-27T11:14:53.342 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-08-27T11:14:53.520 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-08-27T11:14:53.610 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-08-27T11:14:53.693 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-08-27T11:14:53.766 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-08-27T11:14:53.890 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-08-27T11:14:53.991 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-08-27T11:14:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-08-27T11:14:54.196 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-08-27T11:14:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-27T11:14:54.343 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-27T11:14:54.419 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-27T11:14:54.557 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-27T11:14:54.631 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-08-27T11:14:54.727 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:18.0.0-5707.g1da267a3.el8.x86_64 36/134 2023-08-27T11:14:54.811 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-08-27T11:14:54.869 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:18.0.0-5707.g1da267a3.el8.x86_6 37/134 2023-08-27T11:14:54.960 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-27T11:14:55.053 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-08-27T11:14:55.114 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-27T11:14:55.230 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-08-27T11:14:55.382 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-27T11:14:55.421 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-08-27T11:14:55.503 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-08-27T11:14:55.600 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-27T11:14:55.748 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-27T11:14:55.749 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-08-27T11:14:55.888 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-27T11:14:55.991 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-08-27T11:14:56.103 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-27T11:14:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-27T11:14:56.215 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-27T11:14:56.306 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-27T11:14:56.416 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-08-27T11:14:56.488 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-27T11:14:56.608 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-08-27T11:14:56.654 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-27T11:14:56.777 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-08-27T11:14:56.799 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-27T11:14:56.982 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-08-27T11:14:57.006 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-27T11:14:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-08-27T11:14:57.205 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-27T11:14:57.324 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-08-27T11:14:57.407 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-27T11:14:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-08-27T11:14:57.640 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-27T11:14:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-08-27T11:14:57.868 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-27T11:14:58.059 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-27T11:14:58.094 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-08-27T11:14:58.259 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-27T11:14:58.355 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-08-27T11:14:58.598 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-08-27T11:14:58.841 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-08-27T11:14:59.058 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-08-27T11:14:59.385 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-08-27T11:14:59.424 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-27T11:14:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-08-27T11:14:59.680 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-27T11:14:59.862 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-08-27T11:14:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-27T11:15:00.074 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-08-27T11:15:00.193 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-27T11:15:00.269 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-08-27T11:15:00.440 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-08-27T11:15:00.451 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-27T11:15:00.659 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-27T11:15:00.767 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-08-27T11:15:01.049 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-08-27T11:15:01.291 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-08-27T11:15:01.462 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-27T11:15:01.521 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-08-27T11:15:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 92/134 2023-08-27T11:15:01.719 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-27T11:15:01.721 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 92/134 2023-08-27T11:15:01.914 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-27T11:15:01.935 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:18.0.0-5707.g1da267a3.el8.x86_64 93/134 2023-08-27T11:15:02.204 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-27T11:15:02.427 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-27T11:15:02.624 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-27T11:15:02.797 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-27T11:15:02.994 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-27T11:15:03.197 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-27T11:15:03.379 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-27T11:15:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-27T11:15:03.717 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-27T11:15:03.975 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-27T11:15:04.199 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-27T11:15:04.392 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-27T11:15:04.650 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-27T11:15:04.894 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-27T11:15:05.085 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-27T11:15:05.312 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-27T11:15:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 94/134 2023-08-27T11:15:05.850 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 94/134 2023-08-27T11:15:06.408 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-27T11:15:06.440 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 94/134 2023-08-27T11:15:06.682 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-27T11:15:06.731 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 95/134 2023-08-27T11:15:06.816 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 95/134 2023-08-27T11:15:06.944 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-27T11:15:06.975 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 96/134 2023-08-27T11:15:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-27T11:15:07.517 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-27T11:15:07.720 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-27T11:15:08.535 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-27T11:15:08.719 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 92/134 2023-08-27T11:15:08.824 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 92/134 2023-08-27T11:15:08.994 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:18.0.0-5707.g1da267a3.el8.x86_64 93/134 2023-08-27T11:15:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 94/134 2023-08-27T11:15:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 94/134 2023-08-27T11:15:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 94/134 2023-08-27T11:15:13.841 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 95/134 2023-08-27T11:15:13.936 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 95/134 2023-08-27T11:15:14.234 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 96/134 2023-08-27T11:15:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 96/134 2023-08-27T11:15:26.284 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2023-08-27T11:15:26.284 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2023-08-27T11:15:26.284 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /dev 2023-08-27T11:15:26.284 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /run 2023-08-27T11:15:26.285 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2023-08-27T11:15:26.285 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2023-08-27T11:15:26.285 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2023-08-27T11:15:26.285 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2023-08-27T11:15:26.286 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /dev 2023-08-27T11:15:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:15:26.614 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 97/134 2023-08-27T11:15:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 97/134 2023-08-27T11:15:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 98/134 2023-08-27T11:15:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 98/134 2023-08-27T11:15:28.755 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 99/134 2023-08-27T11:15:29.208 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 99/134 2023-08-27T11:15:29.405 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 100/134 2023-08-27T11:15:29.423 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 100/134 2023-08-27T11:15:31.128 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 101/134 2023-08-27T11:15:31.338 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 101/134 2023-08-27T11:15:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5707.g1da267a3.e 102/134 2023-08-27T11:15:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5707.g1da267a3. 103/134 2023-08-27T11:15:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noa 104/134 2023-08-27T11:15:32.032 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noa 104/134 2023-08-27T11:15:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.n 105/134 2023-08-27T11:15:33.274 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 96/134 2023-08-27T11:15:33.275 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2023-08-27T11:15:33.275 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2023-08-27T11:15:33.275 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /dev 2023-08-27T11:15:33.275 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /run 2023-08-27T11:15:33.276 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2023-08-27T11:15:33.276 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2023-08-27T11:15:33.276 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2023-08-27T11:15:33.276 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2023-08-27T11:15:33.276 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /dev 2023-08-27T11:15:33.277 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:15:33.284 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.n 105/134 2023-08-27T11:15:33.429 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da 106/134 2023-08-27T11:15:33.521 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da 106/134 2023-08-27T11:15:33.624 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 97/134 2023-08-27T11:15:33.873 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5707.g1da267a3.el 107/134 2023-08-27T11:15:34.046 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 108/134 2023-08-27T11:15:34.079 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 108/134 2023-08-27T11:15:34.260 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 109/134 2023-08-27T11:15:34.265 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 109/134 2023-08-27T11:15:35.216 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 97/134 2023-08-27T11:15:35.387 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 98/134 2023-08-27T11:15:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:18.0.0-5707.g1da267a3.el8.x86_64 110/134 2023-08-27T11:15:35.603 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 98/134 2023-08-27T11:15:35.757 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 99/134 2023-08-27T11:15:35.779 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 111/134 2023-08-27T11:15:35.813 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 111/134 2023-08-27T11:15:35.955 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5707.g1da26 112/134 2023-08-27T11:15:35.997 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5707.g1da26 112/134 2023-08-27T11:15:36.146 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 113/134 2023-08-27T11:15:36.209 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 99/134 2023-08-27T11:15:36.398 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 100/134 2023-08-27T11:15:36.415 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 100/134 2023-08-27T11:15:36.497 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 113/134 2023-08-27T11:15:36.621 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 114/134 2023-08-27T11:15:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 101/134 2023-08-27T11:15:38.289 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 101/134 2023-08-27T11:15:38.420 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5707.g1da267a3.e 102/134 2023-08-27T11:15:38.614 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5707.g1da267a3. 103/134 2023-08-27T11:15:38.823 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noa 104/134 2023-08-27T11:15:39.042 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noa 104/134 2023-08-27T11:15:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.n 105/134 2023-08-27T11:15:40.321 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.n 105/134 2023-08-27T11:15:40.497 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da 106/134 2023-08-27T11:15:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da 106/134 2023-08-27T11:15:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5707.g1da267a3.el 107/134 2023-08-27T11:15:41.140 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 114/134 2023-08-27T11:15:41.186 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 108/134 2023-08-27T11:15:41.222 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 108/134 2023-08-27T11:15:41.414 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 109/134 2023-08-27T11:15:41.417 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:18.0.0-5707.g1da267a3.el8.x86_64 115/134 2023-08-27T11:15:41.425 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 109/134 2023-08-27T11:15:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-08-27T11:15:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-08-27T11:15:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-08-27T11:15:42.731 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:18.0.0-5707.g1da267a3.el8.x86_64 110/134 2023-08-27T11:15:42.989 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 111/134 2023-08-27T11:15:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-08-27T11:15:43.022 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 111/134 2023-08-27T11:15:43.189 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5707.g1da26 112/134 2023-08-27T11:15:43.190 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 119/134 2023-08-27T11:15:43.231 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5707.g1da26 112/134 2023-08-27T11:15:43.383 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-27T11:15:43.397 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 113/134 2023-08-27T11:15:43.566 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:18.0.0-5707.g1da267a3.el8.x86_ 121/134 2023-08-27T11:15:43.721 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-27T11:15:43.749 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 113/134 2023-08-27T11:15:43.938 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 114/134 2023-08-27T11:15:45.899 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-27T11:15:46.170 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-27T11:15:46.187 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-27T11:15:46.329 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:18.0.0-5707.g1da267a3.el8.noarch 124/134 2023-08-27T11:15:46.595 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 125/134 2023-08-27T11:15:46.874 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:18.0.0-5707.g1da267a3.el8.x86_64 126/134 2023-08-27T11:15:47.180 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-27T11:15:47.344 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-27T11:15:47.353 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-27T11:15:47.530 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-08-27T11:15:47.702 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-08-27T11:15:47.781 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-08-27T11:15:48.011 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-08-27T11:15:48.023 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-08-27T11:15:48.023 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-27T11:15:48.239 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-27T11:15:48.239 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-27T11:15:48.457 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-27T11:15:48.486 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 114/134 2023-08-27T11:15:48.676 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-27T11:15:49.166 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-27T11:15:49.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:18.0.0-5707.g1da267a3.el8.x86_64 1/134 2023-08-27T11:15:49.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 2/134 2023-08-27T11:15:49.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 3/134 2023-08-27T11:15:49.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 4/134 2023-08-27T11:15:49.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5707.g1da26 5/134 2023-08-27T11:15:49.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 6/134 2023-08-27T11:15:49.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 7/134 2023-08-27T11:15:49.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 8/134 2023-08-27T11:15:49.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 9/134 2023-08-27T11:15:49.169 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 10/134 2023-08-27T11:15:49.169 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 11/134 2023-08-27T11:15:49.180 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:18.0.0-5707.g1da267a3.el8.x86_64 12/134 2023-08-27T11:15:49.181 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 13/134 2023-08-27T11:15:49.181 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:18.0.0-5707.g1da267a3.el8.x86_ 14/134 2023-08-27T11:15:49.181 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 15/134 2023-08-27T11:15:49.181 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 16/134 2023-08-27T11:15:49.181 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:18.0.0-5707.g1da267a3.el8.x86_6 17/134 2023-08-27T11:15:49.182 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86 18/134 2023-08-27T11:15:49.182 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 19/134 2023-08-27T11:15:49.182 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5707.g1da267a3.el 20/134 2023-08-27T11:15:49.182 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:18.0.0-5707.g1da267a3.el8. 21/134 2023-08-27T11:15:49.182 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:18.0.0-5707.g1da267a3.el8.x86_6 22/134 2023-08-27T11:15:49.183 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:18.0.0-5707.g1da267a3.el8.x86_64 23/134 2023-08-27T11:15:49.183 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:18.0.0-5707.g1da267a3.el8.x86_64 24/134 2023-08-27T11:15:49.183 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:18.0.0-5707.g1da267a3.el8.x86_64 25/134 2023-08-27T11:15:49.183 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 26/134 2023-08-27T11:15:49.183 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 27/134 2023-08-27T11:15:49.184 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:18.0.0-5707.g1da267a3.el8.x86_64 28/134 2023-08-27T11:15:49.184 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5707.g1da267a3. 29/134 2023-08-27T11:15:49.184 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noa 30/134 2023-08-27T11:15:49.184 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.n 31/134 2023-08-27T11:15:49.184 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da 32/134 2023-08-27T11:15:49.185 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5707.g1da267a3.el 33/134 2023-08-27T11:15:49.185 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 34/134 2023-08-27T11:15:49.185 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5707.g1da267a3.e 35/134 2023-08-27T11:15:49.185 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 36/134 2023-08-27T11:15:49.185 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 37/134 2023-08-27T11:15:49.186 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:18.0.0-5707.g1da267a3.el8.noarch 38/134 2023-08-27T11:15:49.186 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-27T11:15:49.186 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-27T11:15:49.186 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-27T11:15:49.188 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-27T11:15:49.188 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-27T11:15:49.188 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-27T11:15:49.188 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-27T11:15:49.188 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-27T11:15:49.189 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-27T11:15:49.189 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-27T11:15:49.189 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-27T11:15:49.189 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-27T11:15:49.189 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-27T11:15:49.190 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-27T11:15:49.190 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-27T11:15:49.190 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-27T11:15:49.190 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-27T11:15:49.190 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-27T11:15:49.191 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-27T11:15:49.191 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-27T11:15:49.191 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-27T11:15:49.191 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-27T11:15:49.191 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-27T11:15:49.192 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-27T11:15:49.192 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-27T11:15:49.192 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-27T11:15:49.192 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-27T11:15:49.192 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-27T11:15:49.193 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-27T11:15:49.193 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-27T11:15:49.193 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-27T11:15:49.193 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-27T11:15:49.193 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-27T11:15:49.194 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-08-27T11:15:49.194 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-08-27T11:15:49.194 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-08-27T11:15:49.194 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-08-27T11:15:49.194 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-08-27T11:15:49.194 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-08-27T11:15:49.195 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-08-27T11:15:49.195 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-08-27T11:15:49.195 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-08-27T11:15:49.195 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-08-27T11:15:49.195 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-08-27T11:15:49.196 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-08-27T11:15:49.196 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-08-27T11:15:49.196 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-08-27T11:15:49.196 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-08-27T11:15:49.196 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-08-27T11:15:49.197 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-08-27T11:15:49.197 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-08-27T11:15:49.197 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-08-27T11:15:49.197 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-08-27T11:15:49.197 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-08-27T11:15:49.198 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-08-27T11:15:49.198 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-08-27T11:15:49.198 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-08-27T11:15:49.198 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-08-27T11:15:49.198 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-08-27T11:15:49.198 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-08-27T11:15:49.199 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-08-27T11:15:49.199 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-08-27T11:15:49.199 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-08-27T11:15:49.199 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-08-27T11:15:49.199 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-08-27T11:15:49.200 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-08-27T11:15:49.200 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-08-27T11:15:49.200 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-08-27T11:15:49.200 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-08-27T11:15:49.201 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-08-27T11:15:49.201 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-08-27T11:15:49.201 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-08-27T11:15:49.201 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-08-27T11:15:49.201 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-08-27T11:15:49.202 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-08-27T11:15:49.202 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-08-27T11:15:49.202 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-08-27T11:15:49.202 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-08-27T11:15:49.202 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-08-27T11:15:49.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-08-27T11:15:49.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-08-27T11:15:49.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-27T11:15:49.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-08-27T11:15:49.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-27T11:15:49.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-08-27T11:15:49.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-08-27T11:15:49.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-08-27T11:15:49.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-08-27T11:15:49.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-08-27T11:15:49.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-27T11:15:49.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 129/134 2023-08-27T11:15:49.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-27T11:15:49.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 131/134 2023-08-27T11:15:49.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-27T11:15:49.206 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-27T11:15:49.370 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:18.0.0-5707.g1da267a3.el8.x86_64 115/134 2023-08-27T11:15:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-08-27T11:15:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 117/134 2023-08-27T11:15:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-08-27T11:15:50.312 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-08-27T11:15:50.514 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-08-27T11:15:50.758 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-08-27T11:15:50.933 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:18.0.0-5707.g1da267a3.el8.x86_ 121/134 2023-08-27T11:15:51.113 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-27T11:15:51.939 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-27T11:15:51.940 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:50 ago on Sun 27 Aug 2023 11:14:01 AM UTC. 2023-08-27T11:15:53.302 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-27T11:15:53.579 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-27T11:15:53.597 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-27T11:15:53.754 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:18.0.0-5707.g1da267a3.el8.noarch 124/134 2023-08-27T11:15:54.045 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 125/134 2023-08-27T11:15:54.369 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:18.0.0-5707.g1da267a3.el8.x86_64 126/134 2023-08-27T11:15:54.696 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-27T11:15:54.870 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-08-27T11:15:54.948 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-08-27T11:15:55.187 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-08-27T11:15:55.199 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-08-27T11:15:55.353 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-27T11:15:55.361 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-27T11:15:55.497 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-08-27T11:15:55.497 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-27T11:15:55.673 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-27T11:15:55.673 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-27T11:15:55.908 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-27T11:15:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-27T11:15:56.606 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-27T11:15:56.607 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:18.0.0-5707.g1da267a3.el8.x86_64 1/134 2023-08-27T11:15:56.607 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 2/134 2023-08-27T11:15:56.607 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 3/134 2023-08-27T11:15:56.607 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 4/134 2023-08-27T11:15:56.608 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5707.g1da26 5/134 2023-08-27T11:15:56.608 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 6/134 2023-08-27T11:15:56.608 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 7/134 2023-08-27T11:15:56.608 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 8/134 2023-08-27T11:15:56.608 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 9/134 2023-08-27T11:15:56.609 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 10/134 2023-08-27T11:15:56.609 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 11/134 2023-08-27T11:15:56.609 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:18.0.0-5707.g1da267a3.el8.x86_64 12/134 2023-08-27T11:15:56.609 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 13/134 2023-08-27T11:15:56.610 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:18.0.0-5707.g1da267a3.el8.x86_ 14/134 2023-08-27T11:15:56.610 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 15/134 2023-08-27T11:15:56.610 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 16/134 2023-08-27T11:15:56.610 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:18.0.0-5707.g1da267a3.el8.x86_6 17/134 2023-08-27T11:15:56.610 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86 18/134 2023-08-27T11:15:56.611 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 19/134 2023-08-27T11:15:56.611 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5707.g1da267a3.el 20/134 2023-08-27T11:15:56.611 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:18.0.0-5707.g1da267a3.el8. 21/134 2023-08-27T11:15:56.611 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:18.0.0-5707.g1da267a3.el8.x86_6 22/134 2023-08-27T11:15:56.611 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:18.0.0-5707.g1da267a3.el8.x86_64 23/134 2023-08-27T11:15:56.611 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:18.0.0-5707.g1da267a3.el8.x86_64 24/134 2023-08-27T11:15:56.612 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:18.0.0-5707.g1da267a3.el8.x86_64 25/134 2023-08-27T11:15:56.612 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 26/134 2023-08-27T11:15:56.612 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 27/134 2023-08-27T11:15:56.621 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:18.0.0-5707.g1da267a3.el8.x86_64 28/134 2023-08-27T11:15:56.621 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5707.g1da267a3. 29/134 2023-08-27T11:15:56.621 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noa 30/134 2023-08-27T11:15:56.622 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.n 31/134 2023-08-27T11:15:56.622 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da 32/134 2023-08-27T11:15:56.622 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5707.g1da267a3.el 33/134 2023-08-27T11:15:56.622 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 34/134 2023-08-27T11:15:56.622 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5707.g1da267a3.e 35/134 2023-08-27T11:15:56.623 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 36/134 2023-08-27T11:15:56.623 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 37/134 2023-08-27T11:15:56.623 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:18.0.0-5707.g1da267a3.el8.noarch 38/134 2023-08-27T11:15:56.623 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-27T11:15:56.623 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-27T11:15:56.624 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-27T11:15:56.624 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-27T11:15:56.624 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-27T11:15:56.624 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-27T11:15:56.624 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-27T11:15:56.625 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-27T11:15:56.625 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-27T11:15:56.625 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-27T11:15:56.625 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-27T11:15:56.625 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-27T11:15:56.626 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-27T11:15:56.626 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-27T11:15:56.626 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-27T11:15:56.626 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-27T11:15:56.626 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-27T11:15:56.627 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-27T11:15:56.627 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-27T11:15:56.627 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-27T11:15:56.627 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-27T11:15:56.627 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-27T11:15:56.628 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-27T11:15:56.628 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-27T11:15:56.628 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-27T11:15:56.628 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-27T11:15:56.628 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-27T11:15:56.629 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-27T11:15:56.629 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-27T11:15:56.629 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-27T11:15:56.631 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-27T11:15:56.631 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-27T11:15:56.631 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-27T11:15:56.631 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-08-27T11:15:56.632 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-08-27T11:15:56.632 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-08-27T11:15:56.632 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-08-27T11:15:56.633 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-08-27T11:15:56.633 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-08-27T11:15:56.633 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-08-27T11:15:56.633 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-08-27T11:15:56.634 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-08-27T11:15:56.634 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-08-27T11:15:56.634 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-08-27T11:15:56.634 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-08-27T11:15:56.634 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-08-27T11:15:56.635 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-08-27T11:15:56.635 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-08-27T11:15:56.635 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-08-27T11:15:56.635 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-08-27T11:15:56.636 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-08-27T11:15:56.636 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-08-27T11:15:56.636 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-08-27T11:15:56.636 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-08-27T11:15:56.636 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-08-27T11:15:56.637 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-08-27T11:15:56.637 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-08-27T11:15:56.637 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-08-27T11:15:56.637 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-08-27T11:15:56.638 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-08-27T11:15:56.638 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-08-27T11:15:56.638 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-08-27T11:15:56.638 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-08-27T11:15:56.638 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-08-27T11:15:56.639 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-08-27T11:15:56.640 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-08-27T11:15:56.640 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-08-27T11:15:56.640 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-08-27T11:15:56.640 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-08-27T11:15:56.641 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-08-27T11:15:56.641 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-08-27T11:15:56.641 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-08-27T11:15:56.642 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-08-27T11:15:56.642 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-08-27T11:15:56.642 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-08-27T11:15:56.642 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-08-27T11:15:56.642 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-08-27T11:15:56.643 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-08-27T11:15:56.643 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-08-27T11:15:56.643 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-08-27T11:15:56.643 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-08-27T11:15:56.644 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-08-27T11:15:56.644 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-08-27T11:15:56.644 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-08-27T11:15:56.644 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-08-27T11:15:56.645 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-08-27T11:15:56.645 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-08-27T11:15:56.645 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-08-27T11:15:56.645 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-08-27T11:15:56.646 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-08-27T11:15:56.646 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 129/134 2023-08-27T11:15:56.646 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-27T11:15:56.646 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 131/134 2023-08-27T11:15:56.646 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-27T11:15:56.647 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-27T11:15:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-27T11:15:59.301 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:57 ago on Sun 27 Aug 2023 11:14:02 AM UTC. 2023-08-27T11:16:00.338 INFO:teuthology.orchestra.run.smithi112.stdout:Installed products updated. 2023-08-27T11:16:00.556 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:00.556 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2023-08-27T11:16:00.556 INFO:teuthology.orchestra.run.smithi112.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-27T11:16:00.557 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.557 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.557 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-08-27T11:16:00.557 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-27T11:16:00.557 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.1-7.el8.noarch 2023-08-27T11:16:00.558 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.0.4-10.el8.x86_64 2023-08-27T11:16:00.558 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.558 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.558 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.558 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.559 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:00.559 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.559 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.559 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.560 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:00.560 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:00.560 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:00.560 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:00.560 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:00.561 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.561 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.561 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:00.561 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.561 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.562 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.562 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:00.562 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:00.562 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.562 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:00.563 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-27T11:16:00.563 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.1-9.el8.x86_64 2023-08-27T11:16:00.563 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-27T11:16:00.563 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-27T11:16:00.563 INFO:teuthology.orchestra.run.smithi112.stdout: jq-1.6-3.el8.x86_64 2023-08-27T11:16:00.564 INFO:teuthology.orchestra.run.smithi112.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-27T11:16:00.564 INFO:teuthology.orchestra.run.smithi112.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-27T11:16:00.564 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-27T11:16:00.564 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.565 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.565 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.565 INFO:teuthology.orchestra.run.smithi112.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-27T11:16:00.565 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-27T11:16:00.566 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-27T11:16:00.566 INFO:teuthology.orchestra.run.smithi112.stdout: libpq-13.5-1.el8.x86_64 2023-08-27T11:16:00.566 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-27T11:16:00.566 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-27T11:16:00.566 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.567 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.567 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-27T11:16:00.567 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.567 INFO:teuthology.orchestra.run.smithi112.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-27T11:16:00.567 INFO:teuthology.orchestra.run.smithi112.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-27T11:16:00.568 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-27T11:16:00.568 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-27T11:16:00.568 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-27T11:16:00.568 INFO:teuthology.orchestra.run.smithi112.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-27T11:16:00.568 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-27T11:16:00.569 INFO:teuthology.orchestra.run.smithi112.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-27T11:16:00.569 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-27T11:16:00.569 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-27T11:16:00.570 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-27T11:16:00.570 INFO:teuthology.orchestra.run.smithi112.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-27T11:16:00.570 INFO:teuthology.orchestra.run.smithi112.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-27T11:16:00.570 INFO:teuthology.orchestra.run.smithi112.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-27T11:16:00.570 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-27T11:16:00.571 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-27T11:16:00.571 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-27T11:16:00.571 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-27T11:16:00.572 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-27T11:16:00.572 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-27T11:16:00.572 INFO:teuthology.orchestra.run.smithi112.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-27T11:16:00.573 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-27T11:16:00.573 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.573 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.574 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.574 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-27T11:16:00.574 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-27T11:16:00.574 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-27T11:16:00.575 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-27T11:16:00.575 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-27T11:16:00.575 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-27T11:16:00.576 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-27T11:16:00.576 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-27T11:16:00.576 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-27T11:16:00.576 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-27T11:16:00.577 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-27T11:16:00.577 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-27T11:16:00.577 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-27T11:16:00.578 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-27T11:16:00.578 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-27T11:16:00.578 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-27T11:16:00.578 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-27T11:16:00.579 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-27T11:16:00.579 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-27T11:16:00.579 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-27T11:16:00.579 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-27T11:16:00.579 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-27T11:16:00.580 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-2.6-1.el8.noarch 2023-08-27T11:16:00.580 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-27T11:16:00.580 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-27T11:16:00.580 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-27T11:16:00.580 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-27T11:16:00.581 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-27T11:16:00.581 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-27T11:16:00.581 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.581 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.581 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-27T11:16:00.582 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-27T11:16:00.582 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.582 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-27T11:16:00.582 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-27T11:16:00.583 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-27T11:16:00.583 INFO:teuthology.orchestra.run.smithi112.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-27T11:16:00.583 INFO:teuthology.orchestra.run.smithi112.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-27T11:16:00.583 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-27T11:16:00.583 INFO:teuthology.orchestra.run.smithi112.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-27T11:16:00.584 INFO:teuthology.orchestra.run.smithi112.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-27T11:16:00.584 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-27T11:16:00.584 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-27T11:16:00.584 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-27T11:16:00.584 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-27T11:16:00.585 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-27T11:16:00.585 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.585 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.585 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:00.585 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-27T11:16:00.586 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-27T11:16:00.586 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-27T11:16:00.586 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-27T11:16:00.586 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-27T11:16:00.587 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-27T11:16:00.587 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:00.587 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-08-27T11:16:00.747 DEBUG:teuthology.parallel:result is None 2023-08-27T11:16:02.869 INFO:teuthology.orchestra.run.smithi138.stdout:Installed products updated. 2023-08-27T11:16:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:03.092 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2023-08-27T11:16:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-27T11:16:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.093 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-08-27T11:16:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-27T11:16:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.1-7.el8.noarch 2023-08-27T11:16:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.0.4-10.el8.x86_64 2023-08-27T11:16:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:16:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-27T11:16:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.1-9.el8.x86_64 2023-08-27T11:16:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-27T11:16:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-27T11:16:03.102 INFO:teuthology.orchestra.run.smithi138.stdout: jq-1.6-3.el8.x86_64 2023-08-27T11:16:03.102 INFO:teuthology.orchestra.run.smithi138.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-27T11:16:03.102 INFO:teuthology.orchestra.run.smithi138.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-27T11:16:03.102 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-27T11:16:03.103 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.103 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.103 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.103 INFO:teuthology.orchestra.run.smithi138.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-27T11:16:03.103 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-27T11:16:03.104 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-27T11:16:03.104 INFO:teuthology.orchestra.run.smithi138.stdout: libpq-13.5-1.el8.x86_64 2023-08-27T11:16:03.104 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-27T11:16:03.104 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-27T11:16:03.104 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.105 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.105 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-27T11:16:03.105 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.105 INFO:teuthology.orchestra.run.smithi138.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-27T11:16:03.106 INFO:teuthology.orchestra.run.smithi138.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-27T11:16:03.106 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-27T11:16:03.106 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-27T11:16:03.106 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-27T11:16:03.107 INFO:teuthology.orchestra.run.smithi138.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-27T11:16:03.107 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-27T11:16:03.107 INFO:teuthology.orchestra.run.smithi138.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-27T11:16:03.107 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-27T11:16:03.107 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-27T11:16:03.108 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-27T11:16:03.108 INFO:teuthology.orchestra.run.smithi138.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-27T11:16:03.108 INFO:teuthology.orchestra.run.smithi138.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-27T11:16:03.108 INFO:teuthology.orchestra.run.smithi138.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-27T11:16:03.109 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-27T11:16:03.109 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-27T11:16:03.109 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-27T11:16:03.111 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-27T11:16:03.111 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-27T11:16:03.111 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-27T11:16:03.112 INFO:teuthology.orchestra.run.smithi138.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-27T11:16:03.112 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-27T11:16:03.112 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.113 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.113 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.113 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-27T11:16:03.113 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-27T11:16:03.114 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-27T11:16:03.114 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-27T11:16:03.114 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-27T11:16:03.115 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-27T11:16:03.115 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-27T11:16:03.115 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-27T11:16:03.115 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-27T11:16:03.116 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-27T11:16:03.116 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-27T11:16:03.116 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-27T11:16:03.117 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-27T11:16:03.117 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-27T11:16:03.117 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-27T11:16:03.117 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-27T11:16:03.118 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-27T11:16:03.118 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-27T11:16:03.118 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-27T11:16:03.119 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-27T11:16:03.119 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-27T11:16:03.119 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-27T11:16:03.119 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-2.6-1.el8.noarch 2023-08-27T11:16:03.119 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-27T11:16:03.120 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-27T11:16:03.120 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-27T11:16:03.120 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-27T11:16:03.120 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-27T11:16:03.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-27T11:16:03.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-27T11:16:03.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-27T11:16:03.122 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.122 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-27T11:16:03.122 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-27T11:16:03.122 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-27T11:16:03.123 INFO:teuthology.orchestra.run.smithi138.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-27T11:16:03.123 INFO:teuthology.orchestra.run.smithi138.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-27T11:16:03.123 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-27T11:16:03.123 INFO:teuthology.orchestra.run.smithi138.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-27T11:16:03.123 INFO:teuthology.orchestra.run.smithi138.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-27T11:16:03.124 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-27T11:16:03.124 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-27T11:16:03.124 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-27T11:16:03.125 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-27T11:16:03.125 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-27T11:16:03.125 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.125 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.126 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:16:03.126 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-27T11:16:03.126 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-27T11:16:03.126 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-27T11:16:03.126 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-27T11:16:03.127 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-27T11:16:03.127 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-27T11:16:03.127 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:03.128 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-08-27T11:16:03.285 DEBUG:teuthology.parallel:result is None 2023-08-27T11:16:03.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1da267a3afa9ab8f9b1d164359066f8bd682ffd8 2023-08-27T11:16:03.504 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-27T11:16:03.533 INFO:teuthology.orchestra.run.smithi112.stdout:18.0.0-5707.g1da267a3.el8 2023-08-27T11:16:03.534 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5707.g1da267a3.el8 2023-08-27T11:16:03.534 INFO:teuthology.task.install:The correct ceph version 18.0.0-5707.g1da267a3 is installed. 2023-08-27T11:16:03.534 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1da267a3afa9ab8f9b1d164359066f8bd682ffd8 2023-08-27T11:16:03.677 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-27T11:16:03.701 INFO:teuthology.orchestra.run.smithi138.stdout:18.0.0-5707.g1da267a3.el8 2023-08-27T11:16:03.702 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5707.g1da267a3.el8 2023-08-27T11:16:03.702 INFO:teuthology.task.install:The correct ceph version 18.0.0-5707.g1da267a3 is installed. 2023-08-27T11:16:03.703 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-08-27T11:16:03.704 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:03.704 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-27T11:16:03.752 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:03.753 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-27T11:16:03.792 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-08-27T11:16:03.793 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:03.793 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2023-08-27T11:16:03.824 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-27T11:16:03.892 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:03.893 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2023-08-27T11:16:03.919 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-27T11:16:03.986 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-08-27T11:16:03.986 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:03.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-27T11:16:04.014 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-27T11:16:04.082 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:04.083 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-27T11:16:04.109 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-27T11:16:04.175 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-08-27T11:16:04.176 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:04.176 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2023-08-27T11:16:04.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-27T11:16:04.270 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:04.270 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2023-08-27T11:16:04.298 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-27T11:16:04.373 INFO:teuthology.run_tasks:Running task ceph... 2023-08-27T11:16:04.478 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-08-27T11:16:04.480 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /var/log/ceph 2023-08-27T11:16:04.483 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2023-08-27T11:16:04.510 INFO:tasks.ceph:Disabling ceph logrotate... 2023-08-27T11:16:04.511 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-27T11:16:04.564 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-27T11:16:04.600 INFO:tasks.ceph:Creating extra log directories... 2023-08-27T11:16:04.602 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-27T11:16:04.636 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-27T11:16:04.691 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-08-27T11:16:04.692 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-08-27T11:16:04.692 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-08-27T11:16:04.692 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:04.693 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-08-27T11:16:04.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-08-27T11:16:04.861 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-08-27T11:16:04.892 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:04.892 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-08-27T11:16:04.910 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-08-27T11:16:05.051 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-08-27T11:16:05.083 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-08-27T11:16:05.084 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-08-27T11:16:05.084 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381536', 'branch': 'wip-rishabh-xiubo-wip-57154', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}', 'email': None, 'first_in_suite': False, 'job_id': '7381536', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-xiubo-wip-57154'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-xiubo-wip-57154', 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-xiubo-wip-57154', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1da267a3afa9ab8f9b1d164359066f8bd682ffd8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi112.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi138.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestVolumes', 'tasks.cephfs.test_volumes.TestSubvolumeGroups', 'tasks.cephfs.test_volumes.TestSubvolumes', 'tasks.cephfs.test_subvolume.TestSubvolume']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '449a1bc2027504e7b3c3d7b30fa4178906581da7', 'timestamp': '2023-08-26_02:16:12', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1553'} 2023-08-27T11:16:05.085 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-27T11:16:05.088 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-27T11:16:05.135 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/run/ceph 2023-08-27T11:16:05.145 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2023-08-27T11:16:05.202 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:05.202 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2023-08-27T11:16:05.226 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-27T11:16:05.226 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2023-08-27T11:16:05.286 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-27T11:16:05.286 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:16:05.286 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 389826 Links: 1 2023-08-27T11:16:05.286 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:16:05.287 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:16:05.287 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-08-27 11:15:48.873706739 +0000 2023-08-27T11:16:05.287 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-08-27 11:13:14.965160945 +0000 2023-08-27T11:16:05.287 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-08-27 11:13:14.965160945 +0000 2023-08-27T11:16:05.287 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2023-08-27T11:16:05.288 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-27T11:16:05.352 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-08-27T11:16:05.357 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-08-27T11:16:05.357 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000270835 s, 1.9 MB/s 2023-08-27T11:16:05.358 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-27T11:16:05.412 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2023-08-27T11:16:05.470 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-27T11:16:05.470 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:16:05.470 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 389920 Links: 1 2023-08-27T11:16:05.471 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:16:05.471 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:16:05.471 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-08-27 11:15:48.873706739 +0000 2023-08-27T11:16:05.471 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-08-27 11:13:15.483147333 +0000 2023-08-27T11:16:05.471 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-08-27 11:13:15.483147333 +0000 2023-08-27T11:16:05.472 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2023-08-27T11:16:05.472 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-27T11:16:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-08-27T11:16:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-08-27T11:16:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000223132 s, 2.3 MB/s 2023-08-27T11:16:05.537 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-27T11:16:05.595 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2023-08-27T11:16:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-27T11:16:05.654 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:16:05.654 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 388616 Links: 1 2023-08-27T11:16:05.654 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:16:05.654 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:16:05.654 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-08-27 11:15:48.873706739 +0000 2023-08-27T11:16:05.655 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-08-27 11:13:15.983134193 +0000 2023-08-27T11:16:05.655 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-08-27 11:13:15.983134193 +0000 2023-08-27T11:16:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2023-08-27T11:16:05.655 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-27T11:16:05.719 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-08-27T11:16:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-08-27T11:16:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000281588 s, 1.8 MB/s 2023-08-27T11:16:05.721 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-27T11:16:05.780 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2023-08-27T11:16:05.837 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-27T11:16:05.838 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:16:05.838 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 384817 Links: 1 2023-08-27T11:16:05.838 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:16:05.838 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:16:05.839 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-08-27 11:15:48.873706739 +0000 2023-08-27T11:16:05.839 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-08-27 11:13:16.502120556 +0000 2023-08-27T11:16:05.839 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-08-27 11:13:16.502120556 +0000 2023-08-27T11:16:05.839 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2023-08-27T11:16:05.840 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-27T11:16:05.902 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-08-27T11:16:05.903 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-08-27T11:16:05.903 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000281765 s, 1.8 MB/s 2023-08-27T11:16:05.904 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-27T11:16:05.961 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-27T11:16:05.962 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:05.962 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2023-08-27T11:16:05.982 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-27T11:16:05.982 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2023-08-27T11:16:06.042 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-27T11:16:06.043 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:16:06.043 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 385576 Links: 1 2023-08-27T11:16:06.043 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:16:06.043 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:16:06.043 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2023-08-27 11:15:56.309597425 +0000 2023-08-27T11:16:06.044 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2023-08-27 11:13:16.611085174 +0000 2023-08-27T11:16:06.044 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2023-08-27 11:13:16.611085174 +0000 2023-08-27T11:16:06.044 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2023-08-27T11:16:06.044 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-27T11:16:06.111 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2023-08-27T11:16:06.111 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2023-08-27T11:16:06.111 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000266408 s, 1.9 MB/s 2023-08-27T11:16:06.112 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-27T11:16:06.172 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2023-08-27T11:16:06.234 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-27T11:16:06.234 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:16:06.234 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 385752 Links: 1 2023-08-27T11:16:06.235 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:16:06.235 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:16:06.235 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2023-08-27 11:15:56.309597425 +0000 2023-08-27T11:16:06.235 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2023-08-27 11:13:17.137074217 +0000 2023-08-27T11:16:06.235 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2023-08-27 11:13:17.137074217 +0000 2023-08-27T11:16:06.236 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2023-08-27T11:16:06.236 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-27T11:16:06.302 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2023-08-27T11:16:06.302 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2023-08-27T11:16:06.302 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000226765 s, 2.3 MB/s 2023-08-27T11:16:06.303 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-27T11:16:06.362 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2023-08-27T11:16:06.420 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-27T11:16:06.421 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:16:06.421 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 385830 Links: 1 2023-08-27T11:16:06.421 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:16:06.421 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:16:06.422 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2023-08-27 11:15:56.309597425 +0000 2023-08-27T11:16:06.422 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2023-08-27 11:13:17.652063490 +0000 2023-08-27T11:16:06.422 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2023-08-27 11:13:17.652063490 +0000 2023-08-27T11:16:06.422 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2023-08-27T11:16:06.423 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-27T11:16:06.487 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2023-08-27T11:16:06.487 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2023-08-27T11:16:06.488 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.00020064 s, 2.6 MB/s 2023-08-27T11:16:06.489 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-27T11:16:06.547 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2023-08-27T11:16:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-27T11:16:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:16:06.606 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 393447 Links: 1 2023-08-27T11:16:06.606 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:16:06.606 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:16:06.606 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2023-08-27 11:15:56.309597425 +0000 2023-08-27T11:16:06.606 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2023-08-27 11:13:18.163052845 +0000 2023-08-27T11:16:06.607 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2023-08-27 11:13:18.163052845 +0000 2023-08-27T11:16:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2023-08-27T11:16:06.607 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-27T11:16:06.672 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2023-08-27T11:16:06.672 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2023-08-27T11:16:06.672 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000197082 s, 2.6 MB/s 2023-08-27T11:16:06.673 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-27T11:16:06.729 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-27T11:16:06.730 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-08-27T11:16:06.730 INFO:tasks.ceph:Generating config... 2023-08-27T11:16:06.731 INFO:tasks.ceph:[client] client mount timeout = 600 2023-08-27T11:16:06.732 INFO:tasks.ceph:[client] debug client = 20 2023-08-27T11:16:06.732 INFO:tasks.ceph:[client] debug ms = 1 2023-08-27T11:16:06.732 INFO:tasks.ceph:[client] pid file = 2023-08-27T11:16:06.732 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-08-27T11:16:06.733 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-08-27T11:16:06.733 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-08-27T11:16:06.733 INFO:tasks.ceph:[mds] debug mds = 20 2023-08-27T11:16:06.733 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-08-27T11:16:06.734 INFO:tasks.ceph:[mds] debug ms = 1 2023-08-27T11:16:06.734 INFO:tasks.ceph:[mds] mds debug frag = True 2023-08-27T11:16:06.734 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-08-27T11:16:06.734 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-08-27T11:16:06.735 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-08-27T11:16:06.735 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-08-27T11:16:06.736 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-08-27T11:16:06.736 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-08-27T11:16:06.736 INFO:tasks.ceph:[mgr] debug client = 20 2023-08-27T11:16:06.736 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-08-27T11:16:06.737 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-08-27T11:16:06.737 INFO:tasks.ceph:[mgr] debug ms = 1 2023-08-27T11:16:06.737 INFO:tasks.ceph:[mon] debug mon = 20 2023-08-27T11:16:06.738 INFO:tasks.ceph:[mon] debug ms = 1 2023-08-27T11:16:06.738 INFO:tasks.ceph:[mon] debug paxos = 20 2023-08-27T11:16:06.738 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-08-27T11:16:06.738 INFO:tasks.ceph:[osd] bdev async discard = True 2023-08-27T11:16:06.739 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-08-27T11:16:06.739 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-08-27T11:16:06.739 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-08-27T11:16:06.739 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-08-27T11:16:06.740 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-08-27T11:16:06.740 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-08-27T11:16:06.741 INFO:tasks.ceph:[osd] debug ms = 1 2023-08-27T11:16:06.741 INFO:tasks.ceph:[osd] debug osd = 20 2023-08-27T11:16:06.741 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-08-27T11:16:06.741 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-08-27T11:16:06.742 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-08-27T11:16:06.742 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-08-27T11:16:06.742 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-08-27T11:16:06.742 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-08-27T11:16:06.743 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-08-27T11:16:06.743 INFO:tasks.ceph:Setting up mon.a... 2023-08-27T11:16:06.744 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-08-27T11:16:06.799 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.keyring 2023-08-27T11:16:06.802 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-08-27T11:16:06.848 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-27T11:16:06.874 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.112'), ('mon.b', '172.21.15.138'), ('mon.c', '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]')] 2023-08-27T11:16:06.875 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.112,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-08-27T11:16:06.876 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:06.877 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-08-27T11:16:06.932 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.112 --add b 172.21.15.138 --addv c '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:07.020 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:07.021 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool: generated fsid e099a24f-c1cd-4f11-ac4f-ecacc10e5e98 2023-08-27T11:16:07.021 INFO:teuthology.orchestra.run.smithi112.stdout:setting min_mon_release = pacific 2023-08-27T11:16:07.021 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 0 2023-08-27T11:16:07.022 INFO:teuthology.orchestra.run.smithi112.stdout:fsid e099a24f-c1cd-4f11-ac4f-ecacc10e5e98 2023-08-27T11:16:07.022 INFO:teuthology.orchestra.run.smithi112.stdout:last_changed 2023-08-27T11:16:07.020344+0000 2023-08-27T11:16:07.023 INFO:teuthology.orchestra.run.smithi112.stdout:created 2023-08-27T11:16:07.020344+0000 2023-08-27T11:16:07.023 INFO:teuthology.orchestra.run.smithi112.stdout:min_mon_release 16 (pacific) 2023-08-27T11:16:07.023 INFO:teuthology.orchestra.run.smithi112.stdout:election_strategy: 1 2023-08-27T11:16:07.023 INFO:teuthology.orchestra.run.smithi112.stdout:0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2023-08-27T11:16:07.024 INFO:teuthology.orchestra.run.smithi112.stdout:1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2023-08-27T11:16:07.024 INFO:teuthology.orchestra.run.smithi112.stdout:2: [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] mon.c 2023-08-27T11:16:07.024 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-08-27T11:16:07.026 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-08-27T11:16:07.043 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e099a24f-c1cd-4f11-ac4f-ecacc10e5e98... 2023-08-27T11:16:07.044 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-27T11:16:07.085 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-27T11:16:07.128 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2023-08-27T11:16:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2023-08-27T11:16:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-27T11:16:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2023-08-27T11:16:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2023-08-27T11:16:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2023-08-27T11:16:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2023-08-27T11:16:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2023-08-27T11:16:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2023-08-27T11:16:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2023-08-27T11:16:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-27T11:16:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2023-08-27T11:16:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2023-08-27T11:16:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-27T11:16:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2023-08-27T11:16:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2023-08-27T11:16:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2023-08-27T11:16:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2023-08-27T11:16:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2023-08-27T11:16:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2023-08-27T11:16:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2023-08-27T11:16:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2023-08-27T11:16:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-27T11:16:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-27T11:16:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2023-08-27T11:16:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-27T11:16:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2023-08-27T11:16:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2023-08-27T11:16:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2023-08-27T11:16:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2023-08-27T11:16:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2023-08-27T11:16:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.112,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2023-08-27T11:16:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on skipped message = False 2023-08-27T11:16:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = e099a24f-c1cd-4f11-ac4f-ecacc10e5e98 2023-08-27T11:16:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.137 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2023-08-27T11:16:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2023-08-27T11:16:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2023-08-27T11:16:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2023-08-27T11:16:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2023-08-27T11:16:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2023-08-27T11:16:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2023-08-27T11:16:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2023-08-27T11:16:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2023-08-27T11:16:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2023-08-27T11:16:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2023-08-27T11:16:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2023-08-27T11:16:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2023-08-27T11:16:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2023-08-27T11:16:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2023-08-27T11:16:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2023-08-27T11:16:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2023-08-27T11:16:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2023-08-27T11:16:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2023-08-27T11:16:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: bdev async discard = True 2023-08-27T11:16:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: bdev enable discard = True 2023-08-27T11:16:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore allocator = bitmap 2023-08-27T11:16:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2023-08-27T11:16:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2023-08-27T11:16:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 1/20 2023-08-27T11:16:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 1/20 2023-08-27T11:16:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-08-27T11:16:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2023-08-27T11:16:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 4/10 2023-08-27T11:16:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2023-08-27T11:16:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2023-08-27T11:16:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2023-08-27T11:16:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2023-08-27T11:16:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2023-08-27T11:16:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2023-08-27T11:16:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.147 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2023-08-27T11:16:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-08-27T11:16:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2023-08-27T11:16:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2023-08-27T11:16:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2023-08-27T11:16:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2023-08-27T11:16:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2023-08-27T11:16:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2023-08-27T11:16:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2023-08-27T11:16:07.149 INFO:teuthology.orchestra.run.smithi112.stdout: debug finisher = 20 2023-08-27T11:16:07.149 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.149 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2023-08-27T11:16:07.149 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-08-27T11:16:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2023-08-27T11:16:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2023-08-27T11:16:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2023-08-27T11:16:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2023-08-27T11:16:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2023-08-27T11:16:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2023-08-27T11:16:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2023-08-27T11:16:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2023-08-27T11:16:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2023-08-27T11:16:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-27T11:16:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2023-08-27T11:16:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2023-08-27T11:16:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2023-08-27T11:16:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-27T11:16:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-27T11:16:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2023-08-27T11:16:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2023-08-27T11:16:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2023-08-27T11:16:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: mon op complaint time = 120 2023-08-27T11:16:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:07.155 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2023-08-27T11:16:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2023-08-27T11:16:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2023-08-27T11:16:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2023-08-27T11:16:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-27T11:16:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-27T11:16:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: client mount timeout = 600 2023-08-27T11:16:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2023-08-27T11:16:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-08-27T11:16:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2023-08-27T11:16:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2023-08-27T11:16:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2023-08-27T11:16:07.157 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2023-08-27T11:16:07.158 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2023-08-27T11:16:07.158 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2023-08-27T11:16:07.158 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2023-08-27T11:16:07.158 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2023-08-27T11:16:07.158 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2023-08-27T11:16:07.159 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2023-08-27T11:16:07.159 INFO:teuthology.orchestra.run.smithi112.stdout:[mds] 2023-08-27T11:16:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds = 20 2023-08-27T11:16:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds balancer = 20 2023-08-27T11:16:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-08-27T11:16:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug frag = True 2023-08-27T11:16:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug scatterstat = True 2023-08-27T11:16:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: mds op complaint time = 180 2023-08-27T11:16:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: mds verify scatter = True 2023-08-27T11:16:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2023-08-27T11:16:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2023-08-27T11:16:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2023-08-27T11:16:07.162 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2023-08-27T11:16:07.163 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2023-08-27T11:16:07.163 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-27T11:16:07.163 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2023-08-27T11:16:07.163 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.163 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2023-08-27T11:16:07.163 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.164 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2023-08-27T11:16:07.164 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.164 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2023-08-27T11:16:07.164 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2023-08-27T11:16:07.164 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.165 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2023-08-27T11:16:07.165 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2023-08-27T11:16:07.165 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.165 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-27T11:16:07.165 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2023-08-27T11:16:07.166 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.166 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2023-08-27T11:16:07.166 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-27T11:16:07.166 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.166 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2023-08-27T11:16:07.167 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.167 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2023-08-27T11:16:07.168 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2023-08-27T11:16:07.168 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2023-08-27T11:16:07.168 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2023-08-27T11:16:07.168 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2023-08-27T11:16:07.169 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2023-08-27T11:16:07.169 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2023-08-27T11:16:07.170 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-27T11:16:07.170 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-27T11:16:07.170 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2023-08-27T11:16:07.170 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.170 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-27T11:16:07.171 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.171 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2023-08-27T11:16:07.171 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.171 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2023-08-27T11:16:07.171 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.172 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2023-08-27T11:16:07.172 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2023-08-27T11:16:07.172 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2023-08-27T11:16:07.172 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.112,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2023-08-27T11:16:07.172 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on skipped message = False 2023-08-27T11:16:07.173 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = e099a24f-c1cd-4f11-ac4f-ecacc10e5e98 2023-08-27T11:16:07.173 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.173 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2023-08-27T11:16:07.173 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2023-08-27T11:16:07.173 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.174 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2023-08-27T11:16:07.174 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2023-08-27T11:16:07.174 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2023-08-27T11:16:07.174 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.175 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2023-08-27T11:16:07.175 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2023-08-27T11:16:07.175 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.175 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2023-08-27T11:16:07.175 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2023-08-27T11:16:07.176 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2023-08-27T11:16:07.176 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.176 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2023-08-27T11:16:07.176 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2023-08-27T11:16:07.177 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.177 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2023-08-27T11:16:07.177 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.178 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2023-08-27T11:16:07.178 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.178 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2023-08-27T11:16:07.178 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.178 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2023-08-27T11:16:07.179 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2023-08-27T11:16:07.179 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.179 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2023-08-27T11:16:07.179 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2023-08-27T11:16:07.180 INFO:teuthology.orchestra.run.smithi138.stdout: bdev async discard = True 2023-08-27T11:16:07.180 INFO:teuthology.orchestra.run.smithi138.stdout: bdev enable discard = True 2023-08-27T11:16:07.180 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore allocator = bitmap 2023-08-27T11:16:07.180 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2023-08-27T11:16:07.180 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2023-08-27T11:16:07.181 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 1/20 2023-08-27T11:16:07.181 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 1/20 2023-08-27T11:16:07.181 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2023-08-27T11:16:07.181 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2023-08-27T11:16:07.182 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 4/10 2023-08-27T11:16:07.182 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2023-08-27T11:16:07.182 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2023-08-27T11:16:07.182 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2023-08-27T11:16:07.183 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2023-08-27T11:16:07.185 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2023-08-27T11:16:07.186 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2023-08-27T11:16:07.186 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.186 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2023-08-27T11:16:07.186 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2023-08-27T11:16:07.187 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2023-08-27T11:16:07.187 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2023-08-27T11:16:07.187 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2023-08-27T11:16:07.187 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2023-08-27T11:16:07.188 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2023-08-27T11:16:07.188 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2023-08-27T11:16:07.188 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2023-08-27T11:16:07.188 INFO:teuthology.orchestra.run.smithi138.stdout: debug finisher = 20 2023-08-27T11:16:07.189 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.189 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2023-08-27T11:16:07.189 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2023-08-27T11:16:07.190 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2023-08-27T11:16:07.190 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2023-08-27T11:16:07.190 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2023-08-27T11:16:07.190 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2023-08-27T11:16:07.190 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2023-08-27T11:16:07.191 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2023-08-27T11:16:07.191 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2023-08-27T11:16:07.191 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2023-08-27T11:16:07.191 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2023-08-27T11:16:07.192 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.192 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-27T11:16:07.192 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2023-08-27T11:16:07.192 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2023-08-27T11:16:07.192 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.193 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2023-08-27T11:16:07.193 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-27T11:16:07.193 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-27T11:16:07.193 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.193 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2023-08-27T11:16:07.194 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2023-08-27T11:16:07.194 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.194 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2023-08-27T11:16:07.194 INFO:teuthology.orchestra.run.smithi138.stdout: mon op complaint time = 120 2023-08-27T11:16:07.194 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-08-27T11:16:07.195 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2023-08-27T11:16:07.195 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2023-08-27T11:16:07.195 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2023-08-27T11:16:07.195 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2023-08-27T11:16:07.195 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-27T11:16:07.196 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-27T11:16:07.196 INFO:teuthology.orchestra.run.smithi138.stdout: client mount timeout = 600 2023-08-27T11:16:07.196 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2023-08-27T11:16:07.196 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2023-08-27T11:16:07.196 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = "" 2023-08-27T11:16:07.196 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2023-08-27T11:16:07.197 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2023-08-27T11:16:07.197 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2023-08-27T11:16:07.197 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2023-08-27T11:16:07.197 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2023-08-27T11:16:07.197 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2023-08-27T11:16:07.198 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2023-08-27T11:16:07.198 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2023-08-27T11:16:07.198 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2023-08-27T11:16:07.198 INFO:teuthology.orchestra.run.smithi138.stdout:[mds] 2023-08-27T11:16:07.198 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds = 20 2023-08-27T11:16:07.199 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds balancer = 20 2023-08-27T11:16:07.199 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2023-08-27T11:16:07.199 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug frag = True 2023-08-27T11:16:07.199 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug scatterstat = True 2023-08-27T11:16:07.199 INFO:teuthology.orchestra.run.smithi138.stdout: mds op complaint time = 180 2023-08-27T11:16:07.200 INFO:teuthology.orchestra.run.smithi138.stdout: mds verify scatter = True 2023-08-27T11:16:07.200 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2023-08-27T11:16:07.200 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2023-08-27T11:16:07.200 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2023-08-27T11:16:07.200 INFO:tasks.ceph:Creating admin key on mon.a... 2023-08-27T11:16:07.201 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-08-27T11:16:07.264 INFO:tasks.ceph:Copying monmap to all nodes... 2023-08-27T11:16:07.265 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:07.265 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-08-27T11:16:07.282 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:07.282 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-08-27T11:16:07.339 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:16:07.340 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:07.340 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-27T11:16:07.340 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-27T11:16:07.456 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:07.456 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:07.483 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:16:07.484 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:07.484 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-27T11:16:07.484 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-27T11:16:07.521 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:07.522 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:07.578 INFO:tasks.ceph:Setting up mon nodes... 2023-08-27T11:16:07.578 INFO:tasks.ceph:Setting up mgr nodes... 2023-08-27T11:16:07.579 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-08-27T11:16:07.633 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-08-27T11:16:07.636 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-08-27T11:16:07.696 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-08-27T11:16:07.699 INFO:tasks.ceph:Setting up mds nodes... 2023-08-27T11:16:07.700 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-08-27T11:16:07.753 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-08-27T11:16:07.755 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-08-27T11:16:07.781 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-08-27T11:16:07.873 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-08-27T11:16:07.875 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-08-27T11:16:07.901 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-08-27T11:16:07.956 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-08-27T11:16:07.958 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-08-27T11:16:07.984 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-08-27T11:16:08.080 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-08-27T11:16:08.082 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-08-27T11:16:08.109 INFO:tasks.ceph_client:Setting up client nodes... 2023-08-27T11:16:08.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-08-27T11:16:08.154 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-08-27T11:16:08.167 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-08-27T11:16:08.210 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-08-27T11:16:08.223 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-08-27T11:16:08.266 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-08-27T11:16:08.279 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-08-27T11:16:08.361 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-08-27T11:16:08.374 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-08-27T11:16:08.374 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-08-27T11:16:08.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-08-27T11:16:08.402 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-27T11:16:08.403 INFO:tasks.ceph:role: osd.0 2023-08-27T11:16:08.403 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:16:08.403 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-27T11:16:08.477 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:16:08.477 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:16:08.477 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:16:08.478 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:16:08.478 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:16:08.478 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:16:08.478 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:16:08.478 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:16:08.479 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:16:08.479 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:16:08.908 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-08-27T11:16:08.910 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-08-27T11:16:08.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-08-27T11:16:09.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-08-27T11:16:09.086 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-08-27T11:16:09.156 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-27T11:16:09.156 INFO:tasks.ceph:role: osd.1 2023-08-27T11:16:09.157 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:16:09.157 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-27T11:16:09.232 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:16:09.232 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:16:09.232 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:16:09.233 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:16:09.233 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:16:09.233 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:16:09.233 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:16:09.233 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:16:09.234 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:16:09.234 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:16:09.652 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-08-27T11:16:09.654 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-08-27T11:16:09.655 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-08-27T11:16:09.751 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-08-27T11:16:09.785 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-08-27T11:16:09.860 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-27T11:16:09.861 INFO:tasks.ceph:role: osd.2 2023-08-27T11:16:09.861 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:16:09.862 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-08-27T11:16:09.941 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:16:09.942 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:16:09.942 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:16:09.942 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:16:09.943 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:16:09.943 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:16:09.943 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:16:09.943 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:16:09.944 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:16:09.944 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:16:10.392 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-08-27T11:16:10.394 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-08-27T11:16:10.395 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-08-27T11:16:10.536 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-08-27T11:16:10.574 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-08-27T11:16:10.647 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-27T11:16:10.648 INFO:tasks.ceph:role: osd.3 2023-08-27T11:16:10.648 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:16:10.648 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-08-27T11:16:10.729 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:16:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:16:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:16:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:16:10.730 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:16:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:16:10.731 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:16:10.731 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:16:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:16:10.731 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:16:11.176 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-08-27T11:16:11.178 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-08-27T11:16:11.178 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-08-27T11:16:11.359 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-08-27T11:16:11.396 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:11.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:11.503+0000 7fbbb1f0b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-08-27T11:16:11.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:11.506+0000 7fbbb1f0b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-08-27T11:16:11.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:11.506+0000 7fbbb1f0b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-27T11:16:11.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:11.506+0000 7fbbb1f0b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-27T11:16:11.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:11.506+0000 7fbbb1f0b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-27T11:16:11.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:11.508+0000 7fbbb1f0b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-08-27T11:16:14.373 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-08-27T11:16:14.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:14.494+0000 7f3b41c0a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-08-27T11:16:14.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:14.494+0000 7f3b41c0a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-08-27T11:16:14.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:14.494+0000 7f3b41c0a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-27T11:16:14.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:14.494+0000 7f3b41c0a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-27T11:16:14.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:14.494+0000 7f3b41c0a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-27T11:16:14.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:14.494+0000 7f3b41c0a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-08-27T11:16:17.361 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-08-27T11:16:17.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:17.483+0000 7f80d2f02640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-08-27T11:16:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:17.483+0000 7f80d2f02640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-08-27T11:16:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:17.483+0000 7f80d2f02640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-27T11:16:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:17.483+0000 7f80d2f02640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-27T11:16:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:17.483+0000 7f80d2f02640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-27T11:16:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:17.483+0000 7f80d2f02640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-08-27T11:16:20.360 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-08-27T11:16:20.443 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:20.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:20.494+0000 7f49e2d7e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-08-27T11:16:20.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:20.494+0000 7f49e2d7e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-08-27T11:16:20.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:20.494+0000 7f49e2d7e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-27T11:16:20.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:20.494+0000 7f49e2d7e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-27T11:16:20.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:20.494+0000 7f49e2d7e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-27T11:16:20.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:20.495+0000 7f49e2d7e640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-08-27T11:16:23.362 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-08-27T11:16:23.394 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-08-27T11:16:23.428 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-27T11:16:23.428 INFO:tasks.ceph:role: osd.4 2023-08-27T11:16:23.429 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:16:23.429 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-27T11:16:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:16:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:16:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:16:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:16:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:16:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:16:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:16:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:16:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:16:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:16:23.933 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2023-08-27T11:16:23.935 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2023-08-27T11:16:23.935 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-08-27T11:16:24.079 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-08-27T11:16:24.116 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-08-27T11:16:24.186 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-27T11:16:24.186 INFO:tasks.ceph:role: osd.5 2023-08-27T11:16:24.187 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:16:24.187 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-27T11:16:24.261 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:16:24.262 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:16:24.262 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:16:24.262 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:16:24.262 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:16:24.263 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:16:24.263 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:16:24.263 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:16:24.263 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:16:24.263 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:16:24.685 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2023-08-27T11:16:24.687 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2023-08-27T11:16:24.687 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-08-27T11:16:24.793 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-08-27T11:16:24.825 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-08-27T11:16:24.894 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-27T11:16:24.894 INFO:tasks.ceph:role: osd.6 2023-08-27T11:16:24.894 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:16:24.895 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-08-27T11:16:24.969 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:16:24.970 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:16:24.970 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:16:24.970 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:16:24.970 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:16:24.971 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:16:24.971 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:16:24.971 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:16:24.971 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:16:24.971 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:16:25.417 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2023-08-27T11:16:25.419 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2023-08-27T11:16:25.419 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-08-27T11:16:25.565 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-08-27T11:16:25.597 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-08-27T11:16:25.664 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-27T11:16:25.664 INFO:tasks.ceph:role: osd.7 2023-08-27T11:16:25.664 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:16:25.665 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-08-27T11:16:25.738 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:16:25.738 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:16:25.738 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:16:25.739 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:16:25.739 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:16:25.739 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:16:25.739 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:16:25.739 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:16:25.740 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:16:25.740 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:16:26.180 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2023-08-27T11:16:26.182 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2023-08-27T11:16:26.182 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-08-27T11:16:26.323 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-08-27T11:16:26.356 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:26.470 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:26.467+0000 7f326a6b8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-08-27T11:16:26.470 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:26.467+0000 7f326a6b8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-08-27T11:16:26.470 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:26.467+0000 7f326a6b8640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-27T11:16:26.471 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:26.467+0000 7f326a6b8640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-27T11:16:26.471 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:26.467+0000 7f326a6b8640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-27T11:16:26.471 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:26.468+0000 7f326a6b8640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-08-27T11:16:29.339 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-08-27T11:16:29.372 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:29.463 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:29.461+0000 7f96147ae640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-08-27T11:16:29.464 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:29.461+0000 7f96147ae640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-08-27T11:16:29.464 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:29.461+0000 7f96147ae640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-27T11:16:29.464 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:29.461+0000 7f96147ae640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-27T11:16:29.464 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:29.461+0000 7f96147ae640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-27T11:16:29.464 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:29.462+0000 7f96147ae640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-08-27T11:16:32.321 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-08-27T11:16:32.354 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:32.446 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:32.444+0000 7fb328cd8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-08-27T11:16:32.446 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:32.444+0000 7fb328cd8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-08-27T11:16:32.446 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:32.444+0000 7fb328cd8640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-27T11:16:32.446 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:32.444+0000 7fb328cd8640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-27T11:16:32.447 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:32.444+0000 7fb328cd8640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-27T11:16:32.447 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:32.444+0000 7fb328cd8640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-08-27T11:16:35.086 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:16:35.094 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:16:35.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-08-27T11:16:35.351 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:35.441+0000 7fde5dc4d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-08-27T11:16:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:35.441+0000 7fde5dc4d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-08-27T11:16:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:35.441+0000 7fde5dc4d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-27T11:16:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:35.441+0000 7fde5dc4d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-27T11:16:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:35.441+0000 7fde5dc4d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-27T11:16:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:35.442+0000 7fde5dc4d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-08-27T11:16:38.323 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-08-27T11:16:38.354 INFO:tasks.ceph:Reading keys from all nodes... 2023-08-27T11:16:38.355 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:38.355 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-08-27T11:16:38.387 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:38.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-08-27T11:16:38.457 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:38.458 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-08-27T11:16:38.527 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:38.528 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-08-27T11:16:38.595 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:38.596 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-08-27T11:16:38.663 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:38.663 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-08-27T11:16:38.730 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:38.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-08-27T11:16:38.796 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:38.797 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-08-27T11:16:38.829 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:38.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-08-27T11:16:38.901 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:38.901 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-08-27T11:16:38.973 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:38.973 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-08-27T11:16:39.042 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:39.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-08-27T11:16:39.111 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:39.111 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-08-27T11:16:39.178 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:39.179 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-08-27T11:16:39.245 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:39.246 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-08-27T11:16:39.265 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:39.266 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-08-27T11:16:39.322 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:39.323 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-08-27T11:16:39.340 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:39.341 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-08-27T11:16:39.397 INFO:tasks.ceph:Adding keys to all mons... 2023-08-27T11:16:39.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo tee -a /etc/ceph/ceph.keyring 2023-08-27T11:16:39.401 DEBUG:teuthology.orchestra.run.smithi138:> sudo tee -a /etc/ceph/ceph.keyring 2023-08-27T11:16:39.441 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.y] 2023-08-27T11:16:39.441 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB3MOtkrRjBJRAAq3ftmIh81LKadpliVUiJ6g== 2023-08-27T11:16:39.441 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2023-08-27T11:16:39.441 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB3MOtk2bnkLBAA+vxcTeJzucKk9iSF/Zw3Lg== 2023-08-27T11:16:39.442 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2023-08-27T11:16:39.442 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB3MOtki7wJNBAAsIGgDLM82sWNcKnCDlSBMw== 2023-08-27T11:16:39.442 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.0] 2023-08-27T11:16:39.442 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB7MOtkR/cSHhAAR8e8VRd+e0ESJkqQDYWbEw== 2023-08-27T11:16:39.442 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.1] 2023-08-27T11:16:39.443 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB+MOtksBF3HRAA1OIXiIkhgPDYpQyYPaWOXg== 2023-08-27T11:16:39.443 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.2] 2023-08-27T11:16:39.443 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCBMOtkWsrdHBAAkfWbiE6wAFzpt9+BxRh8fw== 2023-08-27T11:16:39.443 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.3] 2023-08-27T11:16:39.443 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCEMOtkIkiKHRAARDoh/FH9ZdqUlZZ29Mc/Dw== 2023-08-27T11:16:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.x] 2023-08-27T11:16:39.444 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB3MOtkXguHKRAALHwzk8diVXqsMvTKr+AzLQ== 2023-08-27T11:16:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2023-08-27T11:16:39.444 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB3MOtk3zD5OBAASVnwG6yn/YQzfYyeMakFpA== 2023-08-27T11:16:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2023-08-27T11:16:39.445 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkCsW/BBAAFTPiTOpBo1L+rLo9px+AjQ== 2023-08-27T11:16:39.445 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.4] 2023-08-27T11:16:39.445 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCKMOtkPgDzGxAA+U4kbwD0d4lS8aNPG2CgHw== 2023-08-27T11:16:39.445 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.5] 2023-08-27T11:16:39.446 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCNMOtkKomSGxAAGAQUcIH2H/uyZIU8vlq1pw== 2023-08-27T11:16:39.446 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.6] 2023-08-27T11:16:39.446 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCQMOtkHI2MGhAA7sgbi8JXO8P9wnVvMy3PqA== 2023-08-27T11:16:39.446 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.7] 2023-08-27T11:16:39.447 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCTMOtkVHpiGhAA4C4YJMWbQ0mmGgB4i8G/AA== 2023-08-27T11:16:39.447 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-08-27T11:16:39.447 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA== 2023-08-27T11:16:39.447 INFO:teuthology.orchestra.run.smithi112.stdout:[client.2] 2023-08-27T11:16:39.448 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkViKJDBAAF2ofwO+78er9OyFfn2bO+g== 2023-08-27T11:16:39.448 INFO:teuthology.orchestra.run.smithi112.stdout:[client.1] 2023-08-27T11:16:39.448 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtk66DbDxAAWZ5StELux0lriJ2gEN9icg== 2023-08-27T11:16:39.448 INFO:teuthology.orchestra.run.smithi112.stdout:[client.3] 2023-08-27T11:16:39.449 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkwt+QFRAAowWE69VqeRhQz0896rxx9Q== 2023-08-27T11:16:39.462 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.y] 2023-08-27T11:16:39.463 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB3MOtkrRjBJRAAq3ftmIh81LKadpliVUiJ6g== 2023-08-27T11:16:39.463 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2023-08-27T11:16:39.463 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB3MOtk2bnkLBAA+vxcTeJzucKk9iSF/Zw3Lg== 2023-08-27T11:16:39.463 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2023-08-27T11:16:39.464 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB3MOtki7wJNBAAsIGgDLM82sWNcKnCDlSBMw== 2023-08-27T11:16:39.464 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.0] 2023-08-27T11:16:39.464 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB7MOtkR/cSHhAAR8e8VRd+e0ESJkqQDYWbEw== 2023-08-27T11:16:39.464 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.1] 2023-08-27T11:16:39.465 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB+MOtksBF3HRAA1OIXiIkhgPDYpQyYPaWOXg== 2023-08-27T11:16:39.465 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.2] 2023-08-27T11:16:39.465 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCBMOtkWsrdHBAAkfWbiE6wAFzpt9+BxRh8fw== 2023-08-27T11:16:39.466 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.3] 2023-08-27T11:16:39.466 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCEMOtkIkiKHRAARDoh/FH9ZdqUlZZ29Mc/Dw== 2023-08-27T11:16:39.466 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.x] 2023-08-27T11:16:39.466 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB3MOtkXguHKRAALHwzk8diVXqsMvTKr+AzLQ== 2023-08-27T11:16:39.467 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2023-08-27T11:16:39.467 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB3MOtk3zD5OBAASVnwG6yn/YQzfYyeMakFpA== 2023-08-27T11:16:39.467 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2023-08-27T11:16:39.467 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB4MOtkCsW/BBAAFTPiTOpBo1L+rLo9px+AjQ== 2023-08-27T11:16:39.468 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.4] 2023-08-27T11:16:39.468 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCKMOtkPgDzGxAA+U4kbwD0d4lS8aNPG2CgHw== 2023-08-27T11:16:39.468 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.5] 2023-08-27T11:16:39.468 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCNMOtkKomSGxAAGAQUcIH2H/uyZIU8vlq1pw== 2023-08-27T11:16:39.469 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.6] 2023-08-27T11:16:39.469 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCQMOtkHI2MGhAA7sgbi8JXO8P9wnVvMy3PqA== 2023-08-27T11:16:39.469 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.7] 2023-08-27T11:16:39.470 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCTMOtkVHpiGhAA4C4YJMWbQ0mmGgB4i8G/AA== 2023-08-27T11:16:39.470 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2023-08-27T11:16:39.470 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA== 2023-08-27T11:16:39.470 INFO:teuthology.orchestra.run.smithi138.stdout:[client.2] 2023-08-27T11:16:39.471 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB4MOtkViKJDBAAF2ofwO+78er9OyFfn2bO+g== 2023-08-27T11:16:39.471 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2023-08-27T11:16:39.471 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB4MOtk66DbDxAAWZ5StELux0lriJ2gEN9icg== 2023-08-27T11:16:39.472 INFO:teuthology.orchestra.run.smithi138.stdout:[client.3] 2023-08-27T11:16:39.472 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB4MOtkwt+QFRAAowWE69VqeRhQz0896rxx9Q== 2023-08-27T11:16:39.473 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-27T11:16:39.492 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-27T11:16:39.566 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:16:39.569 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:16:39.626 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:16:39.629 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:16:39.684 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:39.687 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:39.740 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:39.742 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:39.795 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:39.797 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:39.849 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:39.852 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:39.904 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-27T11:16:39.907 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-27T11:16:39.962 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:16:39.965 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:16:40.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:16:40.023 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:16:40.079 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:40.081 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:40.136 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:40.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:40.191 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:40.194 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:40.247 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:40.249 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:16:40.302 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:16:40.304 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:16:40.361 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:16:40.363 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:16:40.421 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:16:40.424 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:16:40.480 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:16:40.483 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:16:40.539 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-08-27T11:16:40.539 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-08-27T11:16:40.565 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-27T11:16:40.664 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-08-27T11:16:40.691 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-08-27T11:16:40.718 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-27T11:16:40.817 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-08-27T11:16:40.843 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-08-27T11:16:40.915 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-27T11:16:41.026 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-08-27T11:16:41.059 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:41.062 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:16:41.120 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-08-27T11:16:41.121 INFO:tasks.ceph.mon.a:Restarting daemon 2023-08-27T11:16:41.122 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-08-27T11:16:41.127 INFO:tasks.ceph.mon.a:Started 2023-08-27T11:16:41.127 INFO:tasks.ceph.mon.b:Restarting daemon 2023-08-27T11:16:41.128 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-08-27T11:16:41.165 INFO:tasks.ceph.mon.b:Started 2023-08-27T11:16:41.165 INFO:tasks.ceph.mon.c:Restarting daemon 2023-08-27T11:16:41.165 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-08-27T11:16:41.172 INFO:tasks.ceph.mon.c:Started 2023-08-27T11:16:41.172 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-08-27T11:16:41.173 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-08-27T11:16:41.173 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-08-27T11:16:41.177 INFO:tasks.ceph.mgr.y:Started 2023-08-27T11:16:41.177 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-08-27T11:16:41.178 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-08-27T11:16:41.182 INFO:tasks.ceph.mgr.x:Started 2023-08-27T11:16:41.183 INFO:tasks.ceph:Setting crush tunables to default 2023-08-27T11:16:41.183 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd crush tunables default 2023-08-27T11:16:41.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.308+0000 7fa31829f700 1 Processor -- start 2023-08-27T11:16:41.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.308+0000 7fa31829f700 1 -- start start 2023-08-27T11:16:41.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31829f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:41.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31829f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:41.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31829f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31015eda0 con 0x7fa31013e0e0 2023-08-27T11:16:41.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31829f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3100b3e10 con 0x7fa3100b52b0 2023-08-27T11:16:41.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31829f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31015f080 con 0x7fa3100b3440 2023-08-27T11:16:41.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31603b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa31013e0e0 0x7fa310140fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48446/0 (socket says 172.21.15.112:48446) 2023-08-27T11:16:41.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31683c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:41.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31603b700 1 -- 172.21.15.112:0/3556188546 learned_addr learned my addr 172.21.15.112:0/3556188546 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:16:41.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31683c700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 msgr2=0x7fa3100b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-08-27T11:16:41.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31683c700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 msgr2=0x7fa3100b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:16:41.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31683c700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:16:41.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31683c700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:16:41.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:41.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 msgr2=0x7fa3100b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-08-27T11:16:41.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 msgr2=0x7fa3100b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:16:41.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:16:41.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.309+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:16:41.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.510+0000 7fa31683c700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:41.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.510+0000 7fa31683c700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 msgr2=0x7fa3100b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-08-27T11:16:41.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.510+0000 7fa31683c700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 msgr2=0x7fa3100b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:16:41.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.510+0000 7fa31683c700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:16:41.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.510+0000 7fa31683c700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:16:41.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.510+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:41.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.510+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 msgr2=0x7fa3100b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-08-27T11:16:41.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.510+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 msgr2=0x7fa3100b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:16:41.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.510+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:16:41.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.510+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:16:41.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.911+0000 7fa31683c700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:41.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.911+0000 7fa31683c700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 msgr2=0x7fa3100b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-08-27T11:16:41.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.911+0000 7fa31683c700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 msgr2=0x7fa3100b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:16:41.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.911+0000 7fa31683c700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:16:41.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.911+0000 7fa31683c700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-27T11:16:41.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.911+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:41.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.911+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 msgr2=0x7fa3100b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-08-27T11:16:41.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.911+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 msgr2=0x7fa3100b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:16:41.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.911+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:16:41.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:41.911+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-27T11:16:42.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:42.712+0000 7fa31683c700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:42.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:42.712+0000 7fa31683c700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 msgr2=0x7fa3100b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-08-27T11:16:42.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:42.712+0000 7fa31683c700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 msgr2=0x7fa3100b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:16:42.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:42.712+0000 7fa31683c700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:16:42.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:42.712+0000 7fa31683c700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-27T11:16:42.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:42.712+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:42.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:42.712+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 msgr2=0x7fa3100b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-08-27T11:16:42.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:42.712+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 msgr2=0x7fa3100b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:16:42.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:42.712+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:16:42.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:42.712+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-27T11:16:44.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.309+0000 7fa307fff700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 msgr2=0x7fa3100b38d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-27T11:16:44.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.309+0000 7fa307fff700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.138:3300/0 conn(0x7fa3100b3440 0x7fa3100b38d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:44.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.309+0000 7fa307fff700 1 -- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 msgr2=0x7fa3100b5740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-27T11:16:44.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.309+0000 7fa307fff700 1 --2- 172.21.15.112:0/3556188546 >> v2:172.21.15.112:3300/0 conn(0x7fa3100b52b0 0x7fa3100b5740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:44.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.309+0000 7fa307fff700 1 -- 172.21.15.112:0/3556188546 >> v1:172.21.15.112:6789/0 conn(0x7fa31013e0e0 legacy=0x7fa310140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:44.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.309+0000 7fa307fff700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:44.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.309+0000 7fa307fff700 1 -- 172.21.15.112:0/3556188546 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31015f360 con 0x7fa2f4003740 2023-08-27T11:16:44.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.309+0000 7fa307fff700 1 -- 172.21.15.112:0/3556188546 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100b3e10 con 0x7fa2f4007350 2023-08-27T11:16:44.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.309+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:44.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.309+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 msgr2=0x7fa2f4009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:16:44.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.309+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 msgr2=0x7fa2f4009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:16:44.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.309+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:16:44.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.309+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:16:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.511+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.511+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 msgr2=0x7fa2f4009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:16:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.511+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 msgr2=0x7fa2f4009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:16:44.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.511+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:16:44.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.511+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:16:44.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.912+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:44.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.912+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 msgr2=0x7fa2f4009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:16:44.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.912+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 msgr2=0x7fa2f4009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:16:44.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.912+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:16:44.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:44.912+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-27T11:16:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:45.713+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:45.713+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 msgr2=0x7fa2f4009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:16:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:45.713+0000 7fa31583a700 1 -- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 msgr2=0x7fa2f4009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:16:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:45.713+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:16:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:45.713+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-27T11:16:46.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.529+0000 7fa315039700 1 -- 172.21.15.112:0/3556188546 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196656678 0 0) 0x7fa31015f360 con 0x7fa2f4003740 2023-08-27T11:16:46.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.529+0000 7fa315039700 1 -- 172.21.15.112:0/3556188546 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2fc0035b0 con 0x7fa2f4003740 2023-08-27T11:16:46.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa315039700 1 -- 172.21.15.112:0/3556188546 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1635062960 0 0) 0x7fa2fc0035b0 con 0x7fa2f4003740 2023-08-27T11:16:46.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa315039700 1 -- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 msgr2=0x7fa2f4009810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-27T11:16:46.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa315039700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2f4007350 0x7fa2f4009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:46.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa315039700 1 -- 172.21.15.112:0/3556188546 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f400afd0 con 0x7fa2f4003740 2023-08-27T11:16:46.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa315039700 1 -- 172.21.15.112:0/3556188546 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fa30c0016f0 con 0x7fa2f4003740 2023-08-27T11:16:46.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa315039700 1 -- 172.21.15.112:0/3556188546 >> v1:172.21.15.138:6789/0 conn(0x7fa2f4003740 legacy=0x7fa2f4005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:46.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa31683c700 1 -- 172.21.15.112:0/3556188546 reap_dead start 2023-08-27T11:16:46.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa315039700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2fc003a00 0x7fa2fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:46.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa315039700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2fc004850 0x7fa2fc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:46.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa315039700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2fc005330 0x7fa2fc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:46.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa315039700 1 -- 172.21.15.112:0/3556188546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100b3e10 con 0x7fa2fc003a00 2023-08-27T11:16:46.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa315039700 1 -- 172.21.15.112:0/3556188546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc00c280 con 0x7fa2fc004850 2023-08-27T11:16:46.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa315039700 1 -- 172.21.15.112:0/3556188546 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc00c440 con 0x7fa2fc005330 2023-08-27T11:16:46.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa31603b700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2fc003a00 0x7fa2fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:46.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.534+0000 7fa31583a700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2fc004850 0x7fa2fc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:46.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.535+0000 7fa31683c700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2fc005330 0x7fa2fc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:46.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.535+0000 7fa31603b700 1 -- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2fc005330 msgr2=0x7fa2fc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:46.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.535+0000 7fa31603b700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2fc005330 0x7fa2fc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:46.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.535+0000 7fa31603b700 1 -- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2fc004850 msgr2=0x7fa2fc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:46.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.535+0000 7fa31603b700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2fc004850 0x7fa2fc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:46.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.535+0000 7fa31603b700 1 -- 172.21.15.112:0/3556188546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc0035b0 con 0x7fa2fc003a00 2023-08-27T11:16:46.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.535+0000 7fa31603b700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2fc003a00 0x7fa2fc003e00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00c1e0 tx=0x7fa2fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f064b65c8671d4f5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:46.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.535+0000 7fa315039700 1 -- 172.21.15.112:0/3556188546 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c007b70 con 0x7fa2fc003a00 2023-08-27T11:16:46.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.535+0000 7fa315039700 1 -- 172.21.15.112:0/3556188546 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c00cd80 con 0x7fa2fc003a00 2023-08-27T11:16:46.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.535+0000 7fa315039700 1 -- 172.21.15.112:0/3556188546 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c005ba0 con 0x7fa2fc003a00 2023-08-27T11:16:46.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.536+0000 7fa31829f700 1 -- 172.21.15.112:0/3556188546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2fc003a00 msgr2=0x7fa2fc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:46.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.536+0000 7fa31829f700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2fc003a00 0x7fa2fc003e00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00c1e0 tx=0x7fa2fc005880 comp rx=0 tx=0).stop 2023-08-27T11:16:46.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.536+0000 7fa31829f700 1 -- 172.21.15.112:0/3556188546 shutdown_connections 2023-08-27T11:16:46.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.536+0000 7fa31829f700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2fc005330 0x7fa2fc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:46.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.536+0000 7fa31829f700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2fc004850 0x7fa2fc004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:46.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.536+0000 7fa31829f700 1 --2- 172.21.15.112:0/3556188546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2fc003a00 0x7fa2fc003e00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:46.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.536+0000 7fa31829f700 1 -- 172.21.15.112:0/3556188546 >> 172.21.15.112:0/3556188546 conn(0x7fa31006adb0 msgr2=0x7fa31006b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:46.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.536+0000 7fa31829f700 1 -- 172.21.15.112:0/3556188546 shutdown_connections 2023-08-27T11:16:46.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.536+0000 7fa31829f700 1 -- 172.21.15.112:0/3556188546 wait complete. 2023-08-27T11:16:46.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.538+0000 7fa31829f700 1 Processor -- start 2023-08-27T11:16:46.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.538+0000 7fa31829f700 1 -- start start 2023-08-27T11:16:46.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.538+0000 7fa31829f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2fc003a00 0x7fa3101528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:46.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.538+0000 7fa31829f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2fc004850 0x7fa31013ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:46.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.538+0000 7fa31829f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2fc005330 0x7fa3100ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:46.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.538+0000 7fa31829f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100cc320 con 0x7fa2fc005330 2023-08-27T11:16:46.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31829f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100cc600 con 0x7fa2fc004850 2023-08-27T11:16:46.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31829f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100cc8e0 con 0x7fa2fc003a00 2023-08-27T11:16:46.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31683c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2fc005330 0x7fa3100ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:46.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31683c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2fc005330 0x7fa3100ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54128/0 (socket says 172.21.15.112:54128) 2023-08-27T11:16:46.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31683c700 1 -- 172.21.15.112:0/3383158832 learned_addr learned my addr 172.21.15.112:0/3383158832 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:16:46.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31583a700 1 --2- 172.21.15.112:0/3383158832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2fc004850 0x7fa31013ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:46.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31603b700 1 --2- 172.21.15.112:0/3383158832 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2fc003a00 0x7fa3101528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:46.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31683c700 1 -- 172.21.15.112:0/3383158832 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2fc003a00 msgr2=0x7fa3101528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:46.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31683c700 1 --2- 172.21.15.112:0/3383158832 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2fc003a00 0x7fa3101528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:46.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31683c700 1 -- 172.21.15.112:0/3383158832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2fc004850 msgr2=0x7fa31013ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:46.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31683c700 1 --2- 172.21.15.112:0/3383158832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2fc004850 0x7fa31013ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:46.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31683c700 1 -- 172.21.15.112:0/3383158832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3100ccbc0 con 0x7fa2fc005330 2023-08-27T11:16:46.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31683c700 1 --2- 172.21.15.112:0/3383158832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2fc005330 0x7fa3100ca510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4007320 tx=0x7fa3080048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:46.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa306ffd700 1 -- 172.21.15.112:0/3383158832 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308003e70 con 0x7fa2fc005330 2023-08-27T11:16:46.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa306ffd700 1 -- 172.21.15.112:0/3383158832 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa308016a10 con 0x7fa2fc005330 2023-08-27T11:16:46.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31829f700 1 -- 172.21.15.112:0/3383158832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3100ccea0 con 0x7fa2fc005330 2023-08-27T11:16:46.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa306ffd700 1 -- 172.21.15.112:0/3383158832 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308016c00 con 0x7fa2fc005330 2023-08-27T11:16:46.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.539+0000 7fa31829f700 1 -- 172.21.15.112:0/3383158832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31014fd60 con 0x7fa2fc005330 2023-08-27T11:16:46.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.540+0000 7fa306ffd700 1 -- 172.21.15.112:0/3383158832 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa308018720 con 0x7fa2fc005330 2023-08-27T11:16:46.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.540+0000 7fa306ffd700 1 -- 172.21.15.112:0/3383158832 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa308022830 con 0x7fa2fc005330 2023-08-27T11:16:46.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.540+0000 7fa31829f700 1 -- 172.21.15.112:0/3383158832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31014ff20 con 0x7fa2fc005330 2023-08-27T11:16:46.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.543+0000 7fa306ffd700 1 -- 172.21.15.112:0/3383158832 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fa30801d030 con 0x7fa2fc005330 2023-08-27T11:16:46.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.618+0000 7fa31829f700 1 -- 172.21.15.112:0/3383158832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa310067a20 con 0x7fa2fc005330 2023-08-27T11:16:46.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.625+0000 7fa306ffd700 1 -- 172.21.15.112:0/3383158832 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa308018c10 con 0x7fa2fc005330 2023-08-27T11:16:46.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.626+0000 7fa31829f700 1 -- 172.21.15.112:0/3383158832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2fc005330 msgr2=0x7fa3100ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:46.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.626+0000 7fa31829f700 1 --2- 172.21.15.112:0/3383158832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2fc005330 0x7fa3100ca510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4007320 tx=0x7fa3080048c0 comp rx=0 tx=0).stop 2023-08-27T11:16:46.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.626+0000 7fa31829f700 1 -- 172.21.15.112:0/3383158832 shutdown_connections 2023-08-27T11:16:46.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.626+0000 7fa31829f700 1 --2- 172.21.15.112:0/3383158832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2fc005330 0x7fa3100ca510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:46.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.626+0000 7fa31829f700 1 --2- 172.21.15.112:0/3383158832 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2fc003a00 0x7fa3101528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:46.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.626+0000 7fa31829f700 1 --2- 172.21.15.112:0/3383158832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2fc004850 0x7fa31013ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:46.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.626+0000 7fa31829f700 1 -- 172.21.15.112:0/3383158832 >> 172.21.15.112:0/3383158832 conn(0x7fa31006adb0 msgr2=0x7fa31015a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:46.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.626+0000 7fa31829f700 1 -- 172.21.15.112:0/3383158832 shutdown_connections 2023-08-27T11:16:46.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:46.626+0000 7fa31829f700 1 -- 172.21.15.112:0/3383158832 wait complete. 2023-08-27T11:16:46.629 INFO:teuthology.orchestra.run.smithi112.stderr:adjusted tunables profile to default 2023-08-27T11:16:46.639 INFO:tasks.ceph:check_enable_crimson: False 2023-08-27T11:16:46.640 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-08-27T11:16:46.640 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:46.640 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-08-27T11:16:46.656 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:46.654+0000 7f28e4477200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:16:46.667 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:46.668 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-08-27T11:16:46.720 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:46.719+0000 7f28e4477200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:16:46.733 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:46.734 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-08-27T11:16:46.795 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:46.794+0000 7f28e4477200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:16:46.800 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:16:46.800 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-08-27T11:16:46.866 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:46.867 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-08-27T11:16:46.873 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:46.872+0000 7f28e4477200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:16:46.894 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:46.895 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-08-27T11:16:46.962 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:46.963 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-08-27T11:16:47.025 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:47.024+0000 7f28e4477200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:16:47.028 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:16:47.028 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-08-27T11:16:47.096 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 913e8c21-bec3-440c-915a-0b3155c4c1cd 0 2023-08-27T11:16:47.257 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.255+0000 7f7cae79d700 1 Processor -- start 2023-08-27T11:16:47.257 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.255+0000 7f7cae79d700 1 -- start start 2023-08-27T11:16:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7cae79d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7ca8161dd0 0x7f7ca80b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7cae79d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7ca80cc980 0x7f7ca80cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7cae79d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca80b46c0 con 0x7f7ca815e190 2023-08-27T11:16:47.261 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7cae79d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80cd370 con 0x7f7ca80cc980 2023-08-27T11:16:47.261 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7cae79d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80b49a0 con 0x7f7ca8161dd0 2023-08-27T11:16:47.261 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7ca77fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7ca8161dd0 0x7f7ca80b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.262 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7ca77fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7ca8161dd0 0x7f7ca80b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37882/0 (socket says 172.21.15.138:37882) 2023-08-27T11:16:47.262 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7ca77fe700 1 -- 172.21.15.138:0/3853831436 learned_addr learned my addr 172.21.15.138:0/3853831436 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:47.262 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7ca7fff700 1 --1- 172.21.15.138:0/3853831436 >> v1:172.21.15.112:6789/0 conn(0x7f7ca815e190 0x7f7ca815e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.138:38410/0 (socket says 172.21.15.138:38410) 2023-08-27T11:16:47.262 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7cacd3a700 1 --2- 172.21.15.138:0/3853831436 >> v2:172.21.15.112:3300/0 conn(0x7f7ca80cc980 0x7f7ca80cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.262 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7ca77fe700 1 -- 172.21.15.138:0/3853831436 >> v2:172.21.15.112:3300/0 conn(0x7f7ca80cc980 msgr2=0x7f7ca80cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:47.263 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7ca77fe700 1 --2- 172.21.15.138:0/3853831436 >> v2:172.21.15.112:3300/0 conn(0x7f7ca80cc980 0x7f7ca80cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.263 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7ca77fe700 1 -- 172.21.15.138:0/3853831436 >> v1:172.21.15.112:6789/0 conn(0x7f7ca815e190 legacy=0x7f7ca815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.263 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7ca77fe700 1 -- 172.21.15.138:0/3853831436 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca81612f0 con 0x7f7ca8161dd0 2023-08-27T11:16:47.263 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.256+0000 7f7ca6ffd700 1 -- 172.21.15.138:0/3853831436 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496932971 0 0) 0x7f7ca80b46c0 con 0x7f7ca815e190 2023-08-27T11:16:47.263 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.257+0000 7f7ca77fe700 1 --2- 172.21.15.138:0/3853831436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca8161dd0 0x7f7ca80b3dc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80cc820 tx=0x7f7c9c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b85044205a131d25 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.257+0000 7f7ca6ffd700 1 -- 172.21.15.138:0/3853831436 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c011830 con 0x7f7ca8161dd0 2023-08-27T11:16:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.257+0000 7f7ca6ffd700 1 -- 172.21.15.138:0/3853831436 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c011990 con 0x7f7ca8161dd0 2023-08-27T11:16:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.257+0000 7f7ca6ffd700 1 -- 172.21.15.138:0/3853831436 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c011d20 con 0x7f7ca8161dd0 2023-08-27T11:16:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.257+0000 7f7cae79d700 1 -- 172.21.15.138:0/3853831436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca8161dd0 msgr2=0x7f7ca80b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.257+0000 7f7cae79d700 1 --2- 172.21.15.138:0/3853831436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca8161dd0 0x7f7ca80b3dc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80cc820 tx=0x7f7c9c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.265 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.257+0000 7f7cae79d700 1 -- 172.21.15.138:0/3853831436 shutdown_connections 2023-08-27T11:16:47.265 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.257+0000 7f7cae79d700 1 --2- 172.21.15.138:0/3853831436 >> v2:172.21.15.112:3300/0 conn(0x7f7ca80cc980 0x7f7ca80cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.265 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.257+0000 7f7cae79d700 1 --2- 172.21.15.138:0/3853831436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca8161dd0 0x7f7ca80b3dc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.266 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.257+0000 7f7cae79d700 1 -- 172.21.15.138:0/3853831436 >> 172.21.15.138:0/3853831436 conn(0x7f7ca806bbf0 msgr2=0x7f7ca80cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:47.266 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.257+0000 7f7cae79d700 1 -- 172.21.15.138:0/3853831436 shutdown_connections 2023-08-27T11:16:47.267 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.257+0000 7f7cae79d700 1 -- 172.21.15.138:0/3853831436 wait complete. 2023-08-27T11:16:47.267 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.259+0000 7f7cae79d700 1 Processor -- start 2023-08-27T11:16:47.267 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.259+0000 7f7cae79d700 1 -- start start 2023-08-27T11:16:47.270 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.259+0000 7f7cae79d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca80cc980 0x7f7ca814fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.270 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.259+0000 7f7cae79d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ca815e190 0x7f7ca8154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.270 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.259+0000 7f7cae79d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca8161dd0 0x7f7ca8154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.270 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.259+0000 7f7cae79d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8151da0 con 0x7f7ca815e190 2023-08-27T11:16:47.271 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.259+0000 7f7cae79d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca81524a0 con 0x7f7ca8161dd0 2023-08-27T11:16:47.271 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.259+0000 7f7cae79d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80aca30 con 0x7f7ca80cc980 2023-08-27T11:16:47.271 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7ca7fff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca80cc980 0x7f7ca814fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.271 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7ca7fff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca80cc980 0x7f7ca814fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:37854/0 (socket says 172.21.15.138:37854) 2023-08-27T11:16:47.271 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7ca7fff700 1 -- 172.21.15.138:0/1277517245 learned_addr learned my addr 172.21.15.138:0/1277517245 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:47.271 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7cacd3a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca8161dd0 0x7f7ca8154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.272 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7ca77fe700 1 --2- 172.21.15.138:0/1277517245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ca815e190 0x7f7ca8154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.272 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7cacd3a700 1 -- 172.21.15.138:0/1277517245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca80cc980 msgr2=0x7f7ca814fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.272 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7cacd3a700 1 --2- 172.21.15.138:0/1277517245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca80cc980 0x7f7ca814fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.272 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7cacd3a700 1 -- 172.21.15.138:0/1277517245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ca815e190 msgr2=0x7f7ca8154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.272 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7cacd3a700 1 --2- 172.21.15.138:0/1277517245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ca815e190 0x7f7ca8154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.273 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7cacd3a700 1 -- 172.21.15.138:0/1277517245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c009550 con 0x7f7ca8161dd0 2023-08-27T11:16:47.273 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7cacd3a700 1 --2- 172.21.15.138:0/1277517245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca8161dd0 0x7f7ca8154990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009d90 tx=0x7f7ca000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:47.273 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7ca4ff9700 1 -- 172.21.15.138:0/1277517245 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0009c00 con 0x7f7ca8161dd0 2023-08-27T11:16:47.273 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7ca4ff9700 1 -- 172.21.15.138:0/1277517245 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca000f040 con 0x7f7ca8161dd0 2023-08-27T11:16:47.273 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7cae79d700 1 -- 172.21.15.138:0/1277517245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca8151ac0 con 0x7f7ca8161dd0 2023-08-27T11:16:47.274 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7ca4ff9700 1 -- 172.21.15.138:0/1277517245 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0014480 con 0x7f7ca8161dd0 2023-08-27T11:16:47.274 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7cae79d700 1 -- 172.21.15.138:0/1277517245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca80acfb0 con 0x7f7ca8161dd0 2023-08-27T11:16:47.274 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7ca4ff9700 1 -- 172.21.15.138:0/1277517245 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7ca00148e0 con 0x7f7ca8161dd0 2023-08-27T11:16:47.274 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.260+0000 7f7ca4ff9700 1 -- 172.21.15.138:0/1277517245 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7ca001da60 con 0x7f7ca8161dd0 2023-08-27T11:16:47.274 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.261+0000 7f7cae79d700 1 -- 172.21.15.138:0/1277517245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca8066fa0 con 0x7f7ca8161dd0 2023-08-27T11:16:47.275 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.263+0000 7f7ca4ff9700 1 -- 172.21.15.138:0/1277517245 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f7ca003bb00 con 0x7f7ca8161dd0 2023-08-27T11:16:47.348 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.347+0000 7f7cae79d700 1 -- 172.21.15.138:0/1277517245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "913e8c21-bec3-440c-915a-0b3155c4c1cd", "id": 0} v 0) v1 -- 0x7f7ca80ac890 con 0x7f7ca8161dd0 2023-08-27T11:16:47.354 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.353+0000 7f7ca4ff9700 1 -- 172.21.15.138:0/1277517245 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "913e8c21-bec3-440c-915a-0b3155c4c1cd", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7ca80ac890 con 0x7f7ca8161dd0 2023-08-27T11:16:47.355 INFO:teuthology.orchestra.run.smithi138.stdout:0 2023-08-27T11:16:47.356 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.354+0000 7f7cae79d700 1 -- 172.21.15.138:0/1277517245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca8161dd0 msgr2=0x7f7ca8154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.356 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.354+0000 7f7cae79d700 1 --2- 172.21.15.138:0/1277517245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca8161dd0 0x7f7ca8154990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009d90 tx=0x7f7ca000c3a0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.356 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.354+0000 7f7cae79d700 1 -- 172.21.15.138:0/1277517245 shutdown_connections 2023-08-27T11:16:47.356 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.354+0000 7f7cae79d700 1 --2- 172.21.15.138:0/1277517245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca8161dd0 0x7f7ca8154990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.356 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.354+0000 7f7cae79d700 1 --2- 172.21.15.138:0/1277517245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca80cc980 0x7f7ca814fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.357 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.354+0000 7f7cae79d700 1 --2- 172.21.15.138:0/1277517245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ca815e190 0x7f7ca8154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.357 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.354+0000 7f7cae79d700 1 -- 172.21.15.138:0/1277517245 >> 172.21.15.138:0/1277517245 conn(0x7f7ca806bbf0 msgr2=0x7f7ca815e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:47.357 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.354+0000 7f7cae79d700 1 -- 172.21.15.138:0/1277517245 shutdown_connections 2023-08-27T11:16:47.357 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.354+0000 7f7cae79d700 1 -- 172.21.15.138:0/1277517245 wait complete. 2023-08-27T11:16:47.368 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 28afd136-6896-4ad4-a7ba-3d9ff30bf460 1 2023-08-27T11:16:47.369 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:47.365+0000 7fcfe221f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:16:47.431 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:47.430+0000 7fcfe221f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:16:47.437 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:47.436+0000 7f28e4477200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:16:47.491 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.489+0000 7f8fdc9cb700 1 Processor -- start 2023-08-27T11:16:47.491 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.489+0000 7f8fdc9cb700 1 -- start start 2023-08-27T11:16:47.493 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.490+0000 7f8fdc9cb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8fd80aecf0 0x7f8fd80712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.494 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.490+0000 7f8fdc9cb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8fd815dda0 0x7f8fd815e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.494 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.490+0000 7f8fdc9cb700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd815e740 con 0x7f8fd815a7b0 2023-08-27T11:16:47.494 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.490+0000 7f8fdc9cb700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd815bce0 con 0x7f8fd815dda0 2023-08-27T11:16:47.494 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.490+0000 7f8fdc9cb700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd80af0d0 con 0x7f8fd80aecf0 2023-08-27T11:16:47.494 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.490+0000 7f8fd659c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8fd80aecf0 0x7f8fd80712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.495 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.490+0000 7f8fd659c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8fd80aecf0 0x7f8fd80712b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37892/0 (socket says 172.21.15.138:37892) 2023-08-27T11:16:47.495 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.490+0000 7f8fd659c700 1 -- 172.21.15.138:0/4115407670 learned_addr learned my addr 172.21.15.138:0/4115407670 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:47.495 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.490+0000 7f8fd5d9b700 1 --2- 172.21.15.138:0/4115407670 >> v2:172.21.15.112:3300/0 conn(0x7f8fd815dda0 0x7f8fd815e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.495 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fd659c700 1 -- 172.21.15.138:0/4115407670 >> v2:172.21.15.112:3300/0 conn(0x7f8fd815dda0 msgr2=0x7f8fd815e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:47.495 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fd659c700 1 --2- 172.21.15.138:0/4115407670 >> v2:172.21.15.112:3300/0 conn(0x7f8fd815dda0 0x7f8fd815e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.496 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fd659c700 1 -- 172.21.15.138:0/4115407670 >> v1:172.21.15.112:6789/0 conn(0x7f8fd815a7b0 legacy=0x7f8fd815ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.496 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fd659c700 1 -- 172.21.15.138:0/4115407670 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8071c20 con 0x7f8fd80aecf0 2023-08-27T11:16:47.496 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fd5d9b700 1 --2- 172.21.15.138:0/4115407670 >> v2:172.21.15.112:3300/0 conn(0x7f8fd815dda0 0x7f8fd815e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:16:47.496 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fd659c700 1 --2- 172.21.15.138:0/4115407670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80aecf0 0x7f8fd80712b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8fd806c030 tx=0x7f8fc800be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a51af990878bbef server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:47.496 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fd559a700 1 -- 172.21.15.138:0/4115407670 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc800dd30 con 0x7f8fd80aecf0 2023-08-27T11:16:47.497 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fd559a700 1 -- 172.21.15.138:0/4115407670 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc800de90 con 0x7f8fd80aecf0 2023-08-27T11:16:47.497 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fd559a700 1 -- 172.21.15.138:0/4115407670 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8010690 con 0x7f8fd80aecf0 2023-08-27T11:16:47.497 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/4115407670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80aecf0 msgr2=0x7f8fd80712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.497 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fdc9cb700 1 --2- 172.21.15.138:0/4115407670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80aecf0 0x7f8fd80712b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8fd806c030 tx=0x7f8fc800be70 comp rx=0 tx=0).stop 2023-08-27T11:16:47.497 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/4115407670 shutdown_connections 2023-08-27T11:16:47.497 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fdc9cb700 1 --2- 172.21.15.138:0/4115407670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80aecf0 0x7f8fd80712b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.498 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fdc9cb700 1 --2- 172.21.15.138:0/4115407670 >> v2:172.21.15.112:3300/0 conn(0x7f8fd815dda0 0x7f8fd815e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.498 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.491+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/4115407670 >> 172.21.15.138:0/4115407670 conn(0x7f8fd80b1380 msgr2=0x7f8fd80b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:47.498 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.492+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/4115407670 shutdown_connections 2023-08-27T11:16:47.498 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.492+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/4115407670 wait complete. 2023-08-27T11:16:47.501 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.493+0000 7f8fdc9cb700 1 Processor -- start 2023-08-27T11:16:47.501 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.493+0000 7f8fdc9cb700 1 -- start start 2023-08-27T11:16:47.501 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fdc9cb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80aecf0 0x7f8fd80c2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.501 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fdc9cb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd815a7b0 0x7f8fd81f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.501 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fdc9cb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fd815dda0 0x7f8fd81514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.502 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fdc9cb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd80acd40 con 0x7f8fd815a7b0 2023-08-27T11:16:47.502 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fdc9cb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd80aca30 con 0x7f8fd80aecf0 2023-08-27T11:16:47.502 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fdc9cb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd81f6570 con 0x7f8fd815dda0 2023-08-27T11:16:47.502 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fd659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80aecf0 0x7f8fd80c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.503 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fd659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80aecf0 0x7f8fd80c2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37896/0 (socket says 172.21.15.138:37896) 2023-08-27T11:16:47.503 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fd659c700 1 -- 172.21.15.138:0/2978553632 learned_addr learned my addr 172.21.15.138:0/2978553632 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:47.503 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fd6d9d700 1 --2- 172.21.15.138:0/2978553632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fd815dda0 0x7f8fd81514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.503 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fd5d9b700 1 --2- 172.21.15.138:0/2978553632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd815a7b0 0x7f8fd81f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.504 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fd659c700 1 -- 172.21.15.138:0/2978553632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fd815dda0 msgr2=0x7f8fd81514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.504 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fd659c700 1 --2- 172.21.15.138:0/2978553632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fd815dda0 0x7f8fd81514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.504 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fd659c700 1 -- 172.21.15.138:0/2978553632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd815a7b0 msgr2=0x7f8fd81f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.504 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fd659c700 1 --2- 172.21.15.138:0/2978553632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd815a7b0 0x7f8fd81f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.504 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fd659c700 1 -- 172.21.15.138:0/2978553632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc800b7f0 con 0x7f8fd80aecf0 2023-08-27T11:16:47.505 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fd659c700 1 --2- 172.21.15.138:0/2978553632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80aecf0 0x7f8fd80c2140 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8000c00 tx=0x7f8fc8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:47.505 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fc6ffd700 1 -- 172.21.15.138:0/2978553632 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8003c70 con 0x7f8fd80aecf0 2023-08-27T11:16:47.505 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fc6ffd700 1 -- 172.21.15.138:0/2978553632 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc8003df0 con 0x7f8fd80aecf0 2023-08-27T11:16:47.505 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/2978553632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd81f67f0 con 0x7f8fd80aecf0 2023-08-27T11:16:47.506 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fc6ffd700 1 -- 172.21.15.138:0/2978553632 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc80106b0 con 0x7f8fd80aecf0 2023-08-27T11:16:47.506 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/2978553632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd81f6db0 con 0x7f8fd80aecf0 2023-08-27T11:16:47.506 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.494+0000 7f8fc6ffd700 1 -- 172.21.15.138:0/2978553632 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8fc8010b10 con 0x7f8fd80aecf0 2023-08-27T11:16:47.506 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.495+0000 7f8fc6ffd700 1 -- 172.21.15.138:0/2978553632 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f8fc80233f0 con 0x7f8fd80aecf0 2023-08-27T11:16:47.506 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.495+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/2978553632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd80ace80 con 0x7f8fd80aecf0 2023-08-27T11:16:47.506 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.497+0000 7f8fc6ffd700 1 -- 172.21.15.138:0/2978553632 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f8fc8019400 con 0x7f8fd80aecf0 2023-08-27T11:16:47.507 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:47.505+0000 7fcfe221f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:16:47.561 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:47.559+0000 7f28e4477200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:16:47.584 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.583+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/2978553632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "28afd136-6896-4ad4-a7ba-3d9ff30bf460", "id": 1} v 0) v1 -- 0x7f8fd8067a20 con 0x7f8fd80aecf0 2023-08-27T11:16:47.585 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:47.584+0000 7fcfe221f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:16:47.592 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.590+0000 7f8fc6ffd700 1 -- 172.21.15.138:0/2978553632 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "28afd136-6896-4ad4-a7ba-3d9ff30bf460", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8fc80195a0 con 0x7f8fd80aecf0 2023-08-27T11:16:47.592 INFO:teuthology.orchestra.run.smithi138.stdout:1 2023-08-27T11:16:47.594 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.591+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/2978553632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80aecf0 msgr2=0x7f8fd80c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.594 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.591+0000 7f8fdc9cb700 1 --2- 172.21.15.138:0/2978553632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80aecf0 0x7f8fd80c2140 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8000c00 tx=0x7f8fc8004750 comp rx=0 tx=0).stop 2023-08-27T11:16:47.594 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.591+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/2978553632 shutdown_connections 2023-08-27T11:16:47.594 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.591+0000 7f8fdc9cb700 1 --2- 172.21.15.138:0/2978553632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fd815dda0 0x7f8fd81514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.594 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.591+0000 7f8fdc9cb700 1 --2- 172.21.15.138:0/2978553632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80aecf0 0x7f8fd80c2140 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.595 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.591+0000 7f8fdc9cb700 1 --2- 172.21.15.138:0/2978553632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd815a7b0 0x7f8fd81f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.595 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.591+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/2978553632 >> 172.21.15.138:0/2978553632 conn(0x7f8fd80b1380 msgr2=0x7f8fd81596c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:47.595 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.592+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/2978553632 shutdown_connections 2023-08-27T11:16:47.595 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.592+0000 7f8fdc9cb700 1 -- 172.21.15.138:0/2978553632 wait complete. 2023-08-27T11:16:47.605 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new ae5cac36-4cb0-4286-83ee-18eac4623cff 2 2023-08-27T11:16:47.671 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:47.670+0000 7f28e4477200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:16:47.726 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.724+0000 7fe13bca6700 1 Processor -- start 2023-08-27T11:16:47.726 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.724+0000 7fe13bca6700 1 -- start start 2023-08-27T11:16:47.734 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:47.727+0000 7fcfe221f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:16:47.737 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.725+0000 7fe13bca6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe1340cedb0 0x7fe1341413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.737 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.725+0000 7fe13bca6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe13406ddb0 0x7fe13415b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.738 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.725+0000 7fe13bca6700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1341418f0 con 0x7fe1340cf190 2023-08-27T11:16:47.738 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.725+0000 7fe13bca6700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13415c220 con 0x7fe1340cedb0 2023-08-27T11:16:47.738 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.725+0000 7fe13bca6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13415c500 con 0x7fe13406ddb0 2023-08-27T11:16:47.739 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.725+0000 7fe139a42700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe1340cedb0 0x7fe1341413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.739 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.725+0000 7fe139241700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe13406ddb0 0x7fe13415b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.739 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.725+0000 7fe139241700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe13406ddb0 0x7fe13415b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:37872/0 (socket says 172.21.15.138:37872) 2023-08-27T11:16:47.740 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.725+0000 7fe139a42700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe1340cedb0 0x7fe1341413b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37902/0 (socket says 172.21.15.138:37902) 2023-08-27T11:16:47.740 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.725+0000 7fe139241700 1 -- 172.21.15.138:0/465223799 learned_addr learned my addr 172.21.15.138:0/465223799 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:47.740 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.726+0000 7fe139241700 1 -- 172.21.15.138:0/465223799 >> v2:172.21.15.138:3300/0 conn(0x7fe1340cedb0 msgr2=0x7fe1341413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:47.740 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.726+0000 7fe139241700 1 --2- 172.21.15.138:0/465223799 >> v2:172.21.15.138:3300/0 conn(0x7fe1340cedb0 0x7fe1341413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.741 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.726+0000 7fe139241700 1 -- 172.21.15.138:0/465223799 >> v1:172.21.15.138:6789/0 conn(0x7fe1340cf190 legacy=0x7fe13406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.741 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.726+0000 7fe139241700 1 -- 172.21.15.138:0/465223799 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe134140bb0 con 0x7fe13406ddb0 2023-08-27T11:16:47.742 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.726+0000 7fe139a42700 1 --2- 172.21.15.138:0/465223799 >> v2:172.21.15.138:3300/0 conn(0x7fe1340cedb0 0x7fe1341413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:16:47.742 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.726+0000 7fe139241700 1 --2- 172.21.15.138:0/465223799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe13406ddb0 0x7fe13415b7d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe1300098f0 tx=0x7fe130009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bd3030f85923c5c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:47.742 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.726+0000 7fe138a40700 1 -- 172.21.15.138:0/465223799 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130011830 con 0x7fe13406ddb0 2023-08-27T11:16:47.742 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.726+0000 7fe138a40700 1 -- 172.21.15.138:0/465223799 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe130011990 con 0x7fe13406ddb0 2023-08-27T11:16:47.742 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.726+0000 7fe138a40700 1 -- 172.21.15.138:0/465223799 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130011d20 con 0x7fe13406ddb0 2023-08-27T11:16:47.743 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.726+0000 7fe13bca6700 1 -- 172.21.15.138:0/465223799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe13406ddb0 msgr2=0x7fe13415b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.743 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.726+0000 7fe13bca6700 1 --2- 172.21.15.138:0/465223799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe13406ddb0 0x7fe13415b7d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe1300098f0 tx=0x7fe130009c00 comp rx=0 tx=0).stop 2023-08-27T11:16:47.743 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.727+0000 7fe13bca6700 1 -- 172.21.15.138:0/465223799 shutdown_connections 2023-08-27T11:16:47.743 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.727+0000 7fe13bca6700 1 --2- 172.21.15.138:0/465223799 >> v2:172.21.15.138:3300/0 conn(0x7fe1340cedb0 0x7fe1341413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.743 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.727+0000 7fe13bca6700 1 --2- 172.21.15.138:0/465223799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe13406ddb0 0x7fe13415b7d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.744 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.727+0000 7fe13bca6700 1 -- 172.21.15.138:0/465223799 >> 172.21.15.138:0/465223799 conn(0x7fe1340b1380 msgr2=0x7fe1340b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:47.744 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.727+0000 7fe13bca6700 1 -- 172.21.15.138:0/465223799 shutdown_connections 2023-08-27T11:16:47.744 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.727+0000 7fe13bca6700 1 -- 172.21.15.138:0/465223799 wait complete. 2023-08-27T11:16:47.744 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.728+0000 7fe13bca6700 1 Processor -- start 2023-08-27T11:16:47.744 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.728+0000 7fe13bca6700 1 -- start start 2023-08-27T11:16:47.745 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.728+0000 7fe13bca6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe13406ddb0 0x7fe13414d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.745 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.728+0000 7fe13bca6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1340cedb0 0x7fe13415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.745 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.728+0000 7fe13bca6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1340cf190 0x7fe13415a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.746 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.728+0000 7fe13bca6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13414def0 con 0x7fe1340cf190 2023-08-27T11:16:47.746 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe13bca6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1340a96f0 con 0x7fe1340cedb0 2023-08-27T11:16:47.746 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe13bca6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1340a99d0 con 0x7fe13406ddb0 2023-08-27T11:16:47.746 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe139241700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1340cedb0 0x7fe13415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.746 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe139a42700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe13406ddb0 0x7fe13414d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.747 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe139241700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1340cedb0 0x7fe13415a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37910/0 (socket says 172.21.15.138:37910) 2023-08-27T11:16:47.747 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe139241700 1 -- 172.21.15.138:0/2056683467 learned_addr learned my addr 172.21.15.138:0/2056683467 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:47.747 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe13a243700 1 --2- 172.21.15.138:0/2056683467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1340cf190 0x7fe13415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.747 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe139a42700 1 -- 172.21.15.138:0/2056683467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1340cedb0 msgr2=0x7fe13415a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.748 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe139a42700 1 --2- 172.21.15.138:0/2056683467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1340cedb0 0x7fe13415a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.748 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe139a42700 1 -- 172.21.15.138:0/2056683467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1340cf190 msgr2=0x7fe13415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.748 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe139a42700 1 --2- 172.21.15.138:0/2056683467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1340cf190 0x7fe13415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.749 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe139a42700 1 -- 172.21.15.138:0/2056683467 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1240094a0 con 0x7fe13406ddb0 2023-08-27T11:16:47.749 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe139241700 1 --2- 172.21.15.138:0/2056683467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1340cedb0 0x7fe13415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:16:47.749 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe139a42700 1 --2- 172.21.15.138:0/2056683467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe13406ddb0 0x7fe13414d9b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe12400e9b0 tx=0x7fe12400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:47.750 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe12a7fc700 1 -- 172.21.15.138:0/2056683467 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12400c850 con 0x7fe13406ddb0 2023-08-27T11:16:47.750 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe12a7fc700 1 -- 172.21.15.138:0/2056683467 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12400c9b0 con 0x7fe13406ddb0 2023-08-27T11:16:47.750 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe13bca6700 1 -- 172.21.15.138:0/2056683467 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe130009550 con 0x7fe13406ddb0 2023-08-27T11:16:47.751 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe12a7fc700 1 -- 172.21.15.138:0/2056683467 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12400cd60 con 0x7fe13406ddb0 2023-08-27T11:16:47.752 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe13bca6700 1 -- 172.21.15.138:0/2056683467 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1340a9f90 con 0x7fe13406ddb0 2023-08-27T11:16:47.752 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe12a7fc700 1 -- 172.21.15.138:0/2056683467 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe124013770 con 0x7fe13406ddb0 2023-08-27T11:16:47.752 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.729+0000 7fe12a7fc700 1 -- 172.21.15.138:0/2056683467 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fe12401caa0 con 0x7fe13406ddb0 2023-08-27T11:16:47.753 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.730+0000 7fe13bca6700 1 -- 172.21.15.138:0/2056683467 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1340aa300 con 0x7fe13406ddb0 2023-08-27T11:16:47.753 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.732+0000 7fe12a7fc700 1 -- 172.21.15.138:0/2056683467 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fe124021060 con 0x7fe13406ddb0 2023-08-27T11:16:47.758 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:47.756+0000 7f28e4477200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:16:47.817 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.815+0000 7fe13bca6700 1 -- 172.21.15.138:0/2056683467 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ae5cac36-4cb0-4286-83ee-18eac4623cff", "id": 2} v 0) v1 -- 0x7fe134067a20 con 0x7fe13406ddb0 2023-08-27T11:16:47.824 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.823+0000 7fe12a7fc700 1 -- 172.21.15.138:0/2056683467 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ae5cac36-4cb0-4286-83ee-18eac4623cff", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe124017070 con 0x7fe13406ddb0 2023-08-27T11:16:47.825 INFO:teuthology.orchestra.run.smithi138.stdout:2 2023-08-27T11:16:47.826 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.824+0000 7fe13bca6700 1 -- 172.21.15.138:0/2056683467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe13406ddb0 msgr2=0x7fe13414d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.826 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.824+0000 7fe13bca6700 1 --2- 172.21.15.138:0/2056683467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe13406ddb0 0x7fe13414d9b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe12400e9b0 tx=0x7fe12400ecc0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.826 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.824+0000 7fe13bca6700 1 -- 172.21.15.138:0/2056683467 shutdown_connections 2023-08-27T11:16:47.827 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.824+0000 7fe13bca6700 1 --2- 172.21.15.138:0/2056683467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1340cf190 0x7fe13415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.827 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.824+0000 7fe13bca6700 1 --2- 172.21.15.138:0/2056683467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe13406ddb0 0x7fe13414d9b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.827 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.824+0000 7fe13bca6700 1 --2- 172.21.15.138:0/2056683467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1340cedb0 0x7fe13415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.827 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.824+0000 7fe13bca6700 1 -- 172.21.15.138:0/2056683467 >> 172.21.15.138:0/2056683467 conn(0x7fe1340b1380 msgr2=0x7fe13413dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:47.828 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.824+0000 7fe13bca6700 1 -- 172.21.15.138:0/2056683467 shutdown_connections 2023-08-27T11:16:47.828 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.824+0000 7fe13bca6700 1 -- 172.21.15.138:0/2056683467 wait complete. 2023-08-27T11:16:47.837 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new fea7bd81-941f-413d-be1a-54a4578417b4 3 2023-08-27T11:16:47.961 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.959+0000 7fd27c9c3700 1 Processor -- start 2023-08-27T11:16:47.961 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.959+0000 7fd27c9c3700 1 -- start start 2023-08-27T11:16:47.963 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.960+0000 7fd27c9c3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd2780cd070 0x7fd27815a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.964 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.960+0000 7fd27c9c3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27815ea00 0x7fd27815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.964 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.960+0000 7fd27c9c3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd27815b510 con 0x7fd278071980 2023-08-27T11:16:47.964 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.960+0000 7fd27c9c3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd27815e5e0 con 0x7fd2780cd070 2023-08-27T11:16:47.964 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.960+0000 7fd27c9c3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27815df20 con 0x7fd27815ea00 2023-08-27T11:16:47.964 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.960+0000 7fd276d9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27815ea00 0x7fd27815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.965 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.961+0000 7fd276d9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27815ea00 0x7fd27815aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:37886/0 (socket says 172.21.15.138:37886) 2023-08-27T11:16:47.966 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.961+0000 7fd276d9d700 1 -- 172.21.15.138:0/3701276691 learned_addr learned my addr 172.21.15.138:0/3701276691 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:47.966 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.961+0000 7fd275d9b700 1 --2- 172.21.15.138:0/3701276691 >> v2:172.21.15.112:3300/0 conn(0x7fd2780cd070 0x7fd27815a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.966 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.961+0000 7fd276d9d700 1 -- 172.21.15.138:0/3701276691 >> v2:172.21.15.112:3300/0 conn(0x7fd2780cd070 msgr2=0x7fd27815a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:47.967 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.961+0000 7fd276d9d700 1 --2- 172.21.15.138:0/3701276691 >> v2:172.21.15.112:3300/0 conn(0x7fd2780cd070 0x7fd27815a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.967 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.961+0000 7fd276d9d700 1 -- 172.21.15.138:0/3701276691 >> v1:172.21.15.112:6789/0 conn(0x7fd278071980 legacy=0x7fd2780cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.967 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.961+0000 7fd276d9d700 1 -- 172.21.15.138:0/3701276691 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27815e200 con 0x7fd27815ea00 2023-08-27T11:16:47.967 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.961+0000 7fd276d9d700 1 --2- 172.21.15.138:0/3701276691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27815ea00 0x7fd27815aca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd268009e50 tx=0x7fd26800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b94bf6f86f595897 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:47.967 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.961+0000 7fd27559a700 1 -- 172.21.15.138:0/3701276691 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26800e070 con 0x7fd27815ea00 2023-08-27T11:16:47.968 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.961+0000 7fd27559a700 1 -- 172.21.15.138:0/3701276691 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26800b910 con 0x7fd27815ea00 2023-08-27T11:16:47.968 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.961+0000 7fd27559a700 1 -- 172.21.15.138:0/3701276691 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26800bcc0 con 0x7fd27815ea00 2023-08-27T11:16:47.968 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.962+0000 7fd27c9c3700 1 -- 172.21.15.138:0/3701276691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27815ea00 msgr2=0x7fd27815aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.968 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.962+0000 7fd27c9c3700 1 --2- 172.21.15.138:0/3701276691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27815ea00 0x7fd27815aca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd268009e50 tx=0x7fd26800b330 comp rx=0 tx=0).stop 2023-08-27T11:16:47.968 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.962+0000 7fd27c9c3700 1 -- 172.21.15.138:0/3701276691 shutdown_connections 2023-08-27T11:16:47.969 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.962+0000 7fd27c9c3700 1 --2- 172.21.15.138:0/3701276691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27815ea00 0x7fd27815aca0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.969 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.962+0000 7fd27c9c3700 1 --2- 172.21.15.138:0/3701276691 >> v2:172.21.15.112:3300/0 conn(0x7fd2780cd070 0x7fd27815a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.969 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.962+0000 7fd27c9c3700 1 -- 172.21.15.138:0/3701276691 >> 172.21.15.138:0/3701276691 conn(0x7fd27806bbf0 msgr2=0x7fd2780cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:47.969 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.962+0000 7fd27c9c3700 1 -- 172.21.15.138:0/3701276691 shutdown_connections 2023-08-27T11:16:47.969 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.962+0000 7fd27c9c3700 1 -- 172.21.15.138:0/3701276691 wait complete. 2023-08-27T11:16:47.972 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.963+0000 7fd27c9c3700 1 Processor -- start 2023-08-27T11:16:47.972 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.963+0000 7fd27c9c3700 1 -- start start 2023-08-27T11:16:47.972 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd27c9c3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd278071980 0x7fd278140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.972 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd27c9c3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2780cd070 0x7fd278143e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.973 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd27c9c3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd27815ea00 0x7fd278159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:47.973 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd27c9c3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2781f0c10 con 0x7fd27815ea00 2023-08-27T11:16:47.973 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd27c9c3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd278140b30 con 0x7fd278071980 2023-08-27T11:16:47.973 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd27c9c3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd278141500 con 0x7fd2780cd070 2023-08-27T11:16:47.973 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd275d9b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2780cd070 0x7fd278143e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.974 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd27659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd278071980 0x7fd278140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.974 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd275d9b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2780cd070 0x7fd278143e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:37890/0 (socket says 172.21.15.138:37890) 2023-08-27T11:16:47.974 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd27659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd278071980 0x7fd278140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37920/0 (socket says 172.21.15.138:37920) 2023-08-27T11:16:47.974 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd275d9b700 1 -- 172.21.15.138:0/744713374 learned_addr learned my addr 172.21.15.138:0/744713374 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:47.974 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd276d9d700 1 --2- 172.21.15.138:0/744713374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd27815ea00 0x7fd278159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:47.975 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd275d9b700 1 -- 172.21.15.138:0/744713374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd278071980 msgr2=0x7fd278140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.975 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd275d9b700 1 --2- 172.21.15.138:0/744713374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd278071980 0x7fd278140fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.975 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd275d9b700 1 -- 172.21.15.138:0/744713374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd27815ea00 msgr2=0x7fd278159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:47.975 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd275d9b700 1 --2- 172.21.15.138:0/744713374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd27815ea00 0x7fd278159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:47.975 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd275d9b700 1 -- 172.21.15.138:0/744713374 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd268009b30 con 0x7fd2780cd070 2023-08-27T11:16:47.976 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.964+0000 7fd27659c700 1 --2- 172.21.15.138:0/744713374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd278071980 0x7fd278140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:16:47.976 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.965+0000 7fd275d9b700 1 --2- 172.21.15.138:0/744713374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2780cd070 0x7fd278143e10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd26000e9b0 tx=0x7fd26000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:47.976 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.965+0000 7fd26effd700 1 -- 172.21.15.138:0/744713374 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2600096b0 con 0x7fd2780cd070 2023-08-27T11:16:47.976 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.965+0000 7fd26effd700 1 -- 172.21.15.138:0/744713374 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26000ee50 con 0x7fd2780cd070 2023-08-27T11:16:47.976 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.965+0000 7fd26effd700 1 -- 172.21.15.138:0/744713374 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26000c9f0 con 0x7fd2780cd070 2023-08-27T11:16:47.977 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.965+0000 7fd27c9c3700 1 -- 172.21.15.138:0/744713374 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2781f0650 con 0x7fd2780cd070 2023-08-27T11:16:47.977 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.965+0000 7fd27c9c3700 1 -- 172.21.15.138:0/744713374 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2781407c0 con 0x7fd2780cd070 2023-08-27T11:16:47.977 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.965+0000 7fd26effd700 1 -- 172.21.15.138:0/744713374 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd260005a20 con 0x7fd2780cd070 2023-08-27T11:16:47.977 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.965+0000 7fd26effd700 1 -- 172.21.15.138:0/744713374 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fd2600066b0 con 0x7fd2780cd070 2023-08-27T11:16:47.977 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.966+0000 7fd27c9c3700 1 -- 172.21.15.138:0/744713374 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd278066fa0 con 0x7fd2780cd070 2023-08-27T11:16:47.978 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:47.968+0000 7fd26effd700 1 -- 172.21.15.138:0/744713374 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fd260021060 con 0x7fd2780cd070 2023-08-27T11:16:48.052 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.051+0000 7fd27c9c3700 1 -- 172.21.15.138:0/744713374 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fea7bd81-941f-413d-be1a-54a4578417b4", "id": 3} v 0) v1 -- 0x7fd278140620 con 0x7fd2780cd070 2023-08-27T11:16:48.060 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.059+0000 7fd26effd700 1 -- 172.21.15.138:0/744713374 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fea7bd81-941f-413d-be1a-54a4578417b4", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd278140620 con 0x7fd2780cd070 2023-08-27T11:16:48.061 INFO:teuthology.orchestra.run.smithi138.stdout:3 2023-08-27T11:16:48.062 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.060+0000 7fd27c9c3700 1 -- 172.21.15.138:0/744713374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2780cd070 msgr2=0x7fd278143e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.062 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.060+0000 7fd27c9c3700 1 --2- 172.21.15.138:0/744713374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2780cd070 0x7fd278143e10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd26000e9b0 tx=0x7fd26000ecc0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.062 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.060+0000 7fd27c9c3700 1 -- 172.21.15.138:0/744713374 shutdown_connections 2023-08-27T11:16:48.062 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.060+0000 7fd27c9c3700 1 --2- 172.21.15.138:0/744713374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd27815ea00 0x7fd278159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.063 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.060+0000 7fd27c9c3700 1 --2- 172.21.15.138:0/744713374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd278071980 0x7fd278140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.063 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.060+0000 7fd27c9c3700 1 --2- 172.21.15.138:0/744713374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2780cd070 0x7fd278143e10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.063 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.060+0000 7fd27c9c3700 1 -- 172.21.15.138:0/744713374 >> 172.21.15.138:0/744713374 conn(0x7fd27806bbf0 msgr2=0x7fd27815a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:48.063 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.060+0000 7fd27c9c3700 1 -- 172.21.15.138:0/744713374 shutdown_connections 2023-08-27T11:16:48.064 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.060+0000 7fd27c9c3700 1 -- 172.21.15.138:0/744713374 wait complete. 2023-08-27T11:16:48.072 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new d260a2ae-e869-4f76-a0cd-1289be7e12e5 4 2023-08-27T11:16:48.122 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:48.120+0000 7fcfe221f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:16:48.193 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.192+0000 7f28531dc700 1 Processor -- start 2023-08-27T11:16:48.194 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.192+0000 7f28531dc700 1 -- start start 2023-08-27T11:16:48.196 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.193+0000 7f28531dc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f284c0cfa90 0x7f284c0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.196 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.193+0000 7f28531dc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c06e1d0 0x7f284c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.197 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.193+0000 7f28531dc700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284c06e770 con 0x7f284c0b5270 2023-08-27T11:16:48.197 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.193+0000 7f28531dc700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284c1444b0 con 0x7f284c0cfa90 2023-08-27T11:16:48.197 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.193+0000 7f28531dc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c15b910 con 0x7f284c06e1d0 2023-08-27T11:16:48.197 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.193+0000 7f284bfff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c06e1d0 0x7f284c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.198 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.193+0000 7f284bfff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c06e1d0 0x7f284c0b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:37896/0 (socket says 172.21.15.138:37896) 2023-08-27T11:16:48.198 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.193+0000 7f284bfff700 1 -- 172.21.15.138:0/54206742 learned_addr learned my addr 172.21.15.138:0/54206742 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:48.198 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.193+0000 7f2850f78700 1 --2- 172.21.15.138:0/54206742 >> v2:172.21.15.112:3300/0 conn(0x7f284c0cfa90 0x7f284c0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.198 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.193+0000 7f284bfff700 1 -- 172.21.15.138:0/54206742 >> v2:172.21.15.112:3300/0 conn(0x7f284c0cfa90 msgr2=0x7f284c0b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:48.198 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.193+0000 7f284bfff700 1 --2- 172.21.15.138:0/54206742 >> v2:172.21.15.112:3300/0 conn(0x7f284c0cfa90 0x7f284c0b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.199 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.193+0000 7f284bfff700 1 -- 172.21.15.138:0/54206742 >> v1:172.21.15.112:6789/0 conn(0x7f284c0b5270 legacy=0x7f284c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.199 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.194+0000 7f284bfff700 1 -- 172.21.15.138:0/54206742 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c144e10 con 0x7f284c06e1d0 2023-08-27T11:16:48.199 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.194+0000 7f284bfff700 1 --2- 172.21.15.138:0/54206742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c06e1d0 0x7f284c0b4d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f28380098f0 tx=0x7f2838009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3788dec61790737f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:48.199 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.194+0000 7f284b7fe700 1 -- 172.21.15.138:0/54206742 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283800b8a0 con 0x7f284c06e1d0 2023-08-27T11:16:48.199 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.194+0000 7f284b7fe700 1 -- 172.21.15.138:0/54206742 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283800ba00 con 0x7f284c06e1d0 2023-08-27T11:16:48.200 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.194+0000 7f284b7fe700 1 -- 172.21.15.138:0/54206742 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283800bd90 con 0x7f284c06e1d0 2023-08-27T11:16:48.200 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.194+0000 7f28531dc700 1 -- 172.21.15.138:0/54206742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c06e1d0 msgr2=0x7f284c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.200 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.194+0000 7f28531dc700 1 --2- 172.21.15.138:0/54206742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c06e1d0 0x7f284c0b4d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f28380098f0 tx=0x7f2838009c00 comp rx=0 tx=0).stop 2023-08-27T11:16:48.200 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.194+0000 7f28531dc700 1 -- 172.21.15.138:0/54206742 shutdown_connections 2023-08-27T11:16:48.201 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.194+0000 7f28531dc700 1 --2- 172.21.15.138:0/54206742 >> v2:172.21.15.112:3300/0 conn(0x7f284c0cfa90 0x7f284c0b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.201 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.194+0000 7f28531dc700 1 --2- 172.21.15.138:0/54206742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c06e1d0 0x7f284c0b4d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.201 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.194+0000 7f28531dc700 1 -- 172.21.15.138:0/54206742 >> 172.21.15.138:0/54206742 conn(0x7f284c06bbf0 msgr2=0x7f284c0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:48.201 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.194+0000 7f28531dc700 1 -- 172.21.15.138:0/54206742 shutdown_connections 2023-08-27T11:16:48.202 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.194+0000 7f28531dc700 1 -- 172.21.15.138:0/54206742 wait complete. 2023-08-27T11:16:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.196+0000 7f28531dc700 1 Processor -- start 2023-08-27T11:16:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.196+0000 7f28531dc700 1 -- start start 2023-08-27T11:16:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.196+0000 7f28531dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c06e1d0 0x7f284c1f1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.196+0000 7f28531dc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c0b5270 0x7f284c1f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.206 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.196+0000 7f28531dc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c1391b0 0x7f284c159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.206 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.196+0000 7f28531dc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c15eba0 con 0x7f284c0b5270 2023-08-27T11:16:48.206 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.196+0000 7f28531dc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c1f43a0 con 0x7f284c06e1d0 2023-08-27T11:16:48.206 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.196+0000 7f28531dc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c1f4640 con 0x7f284c1391b0 2023-08-27T11:16:48.206 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f2851779700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c1391b0 0x7f284c159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.207 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f2850f78700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c06e1d0 0x7f284c1f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.207 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f2851779700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c1391b0 0x7f284c159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:37904/0 (socket says 172.21.15.138:37904) 2023-08-27T11:16:48.207 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f2851779700 1 -- 172.21.15.138:0/1999702095 learned_addr learned my addr 172.21.15.138:0/1999702095 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:48.207 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f284bfff700 1 --2- 172.21.15.138:0/1999702095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c0b5270 0x7f284c1f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.207 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f2851779700 1 -- 172.21.15.138:0/1999702095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c06e1d0 msgr2=0x7f284c1f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.208 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f2851779700 1 --2- 172.21.15.138:0/1999702095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c06e1d0 0x7f284c1f1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.208 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f2851779700 1 -- 172.21.15.138:0/1999702095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c0b5270 msgr2=0x7f284c1f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.208 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f2851779700 1 --2- 172.21.15.138:0/1999702095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c0b5270 0x7f284c1f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.208 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f2851779700 1 -- 172.21.15.138:0/1999702095 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2838009550 con 0x7f284c1391b0 2023-08-27T11:16:48.208 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f2850f78700 1 --2- 172.21.15.138:0/1999702095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c06e1d0 0x7f284c1f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:16:48.208 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f2851779700 1 --2- 172.21.15.138:0/1999702095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c1391b0 0x7f284c159a70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f284400cbc0 tx=0x7f284400ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:48.209 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f28497fa700 1 -- 172.21.15.138:0/1999702095 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2844003c00 con 0x7f284c1391b0 2023-08-27T11:16:48.209 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f28497fa700 1 -- 172.21.15.138:0/1999702095 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2844003d60 con 0x7f284c1391b0 2023-08-27T11:16:48.209 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f28531dc700 1 -- 172.21.15.138:0/1999702095 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284c1f4920 con 0x7f284c1391b0 2023-08-27T11:16:48.209 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f28497fa700 1 -- 172.21.15.138:0/1999702095 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400aae0 con 0x7f284c1391b0 2023-08-27T11:16:48.209 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.197+0000 7f28531dc700 1 -- 172.21.15.138:0/1999702095 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284c15edb0 con 0x7f284c1391b0 2023-08-27T11:16:48.210 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.198+0000 7f28497fa700 1 -- 172.21.15.138:0/1999702095 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f28440050d0 con 0x7f284c1391b0 2023-08-27T11:16:48.210 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.198+0000 7f28497fa700 1 -- 172.21.15.138:0/1999702095 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f2844012a70 con 0x7f284c1391b0 2023-08-27T11:16:48.210 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.198+0000 7f28531dc700 1 -- 172.21.15.138:0/1999702095 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284c066fa0 con 0x7f284c1391b0 2023-08-27T11:16:48.211 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.200+0000 7f28497fa700 1 -- 172.21.15.138:0/1999702095 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f2844012cd0 con 0x7f284c1391b0 2023-08-27T11:16:48.223 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:48.221+0000 7f28e4477200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:16:48.236 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:48.235+0000 7fcfe221f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:16:48.284 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.283+0000 7f28531dc700 1 -- 172.21.15.138:0/1999702095 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d260a2ae-e869-4f76-a0cd-1289be7e12e5", "id": 4} v 0) v1 -- 0x7f284c067a20 con 0x7f284c1391b0 2023-08-27T11:16:48.295 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.294+0000 7f28497fa700 1 -- 172.21.15.138:0/1999702095 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d260a2ae-e869-4f76-a0cd-1289be7e12e5", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2844012560 con 0x7f284c1391b0 2023-08-27T11:16:48.295 INFO:teuthology.orchestra.run.smithi138.stdout:4 2023-08-27T11:16:48.296 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.295+0000 7f28531dc700 1 -- 172.21.15.138:0/1999702095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c1391b0 msgr2=0x7f284c159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.297 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.295+0000 7f28531dc700 1 --2- 172.21.15.138:0/1999702095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c1391b0 0x7f284c159a70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f284400cbc0 tx=0x7f284400ced0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.297 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.295+0000 7f28531dc700 1 -- 172.21.15.138:0/1999702095 shutdown_connections 2023-08-27T11:16:48.297 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.295+0000 7f28531dc700 1 --2- 172.21.15.138:0/1999702095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f284c1391b0 0x7f284c159a70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.297 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.295+0000 7f28531dc700 1 --2- 172.21.15.138:0/1999702095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c06e1d0 0x7f284c1f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.298 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.295+0000 7f28531dc700 1 --2- 172.21.15.138:0/1999702095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c0b5270 0x7f284c1f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.298 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.295+0000 7f28531dc700 1 -- 172.21.15.138:0/1999702095 >> 172.21.15.138:0/1999702095 conn(0x7f284c06bbf0 msgr2=0x7f284c0ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:48.298 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.295+0000 7f28531dc700 1 -- 172.21.15.138:0/1999702095 shutdown_connections 2023-08-27T11:16:48.298 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.295+0000 7f28531dc700 1 -- 172.21.15.138:0/1999702095 wait complete. 2023-08-27T11:16:48.308 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new a1d24950-2c63-4da5-88b4-b694c0650c80 5 2023-08-27T11:16:48.325 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:48.323+0000 7fcfe221f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:16:48.405 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:48.403+0000 7fcfe221f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:16:48.428 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.427+0000 7f9c4cd87700 1 Processor -- start 2023-08-27T11:16:48.429 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.427+0000 7f9c4cd87700 1 -- start start 2023-08-27T11:16:48.432 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4cd87700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9c4815b250 0x7f9c4815db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.432 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4cd87700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9c48066010 0x7f9c48066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.432 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4cd87700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c48066990 con 0x7f9c480ce590 2023-08-27T11:16:48.432 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4cd87700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4815b630 con 0x7f9c48066010 2023-08-27T11:16:48.432 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4cd87700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4815f440 con 0x7f9c4815b250 2023-08-27T11:16:48.433 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4659c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9c4815b250 0x7f9c4815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.433 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4659c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9c4815b250 0x7f9c4815db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37938/0 (socket says 172.21.15.138:37938) 2023-08-27T11:16:48.433 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4659c700 1 -- 172.21.15.138:0/2718466687 learned_addr learned my addr 172.21.15.138:0/2718466687 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:48.433 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c46d9d700 1 --2- 172.21.15.138:0/2718466687 >> v2:172.21.15.112:3300/0 conn(0x7f9c48066010 0x7f9c48066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.433 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4659c700 1 -- 172.21.15.138:0/2718466687 >> v2:172.21.15.112:3300/0 conn(0x7f9c48066010 msgr2=0x7f9c48066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:48.434 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4659c700 1 --2- 172.21.15.138:0/2718466687 >> v2:172.21.15.112:3300/0 conn(0x7f9c48066010 0x7f9c48066450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.434 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4659c700 1 -- 172.21.15.138:0/2718466687 >> v1:172.21.15.112:6789/0 conn(0x7f9c480ce590 legacy=0x7f9c4815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.434 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4659c700 1 -- 172.21.15.138:0/2718466687 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4815eb10 con 0x7f9c4815b250 2023-08-27T11:16:48.434 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c46d9d700 1 --2- 172.21.15.138:0/2718466687 >> v2:172.21.15.112:3300/0 conn(0x7f9c48066010 0x7f9c48066450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:16:48.434 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4659c700 1 --2- 172.21.15.138:0/2718466687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c4815b250 0x7f9c4815db70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9c480d1680 tx=0x7f9c38009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=482cda1ed095f8f2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:48.435 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4559a700 1 -- 172.21.15.138:0/2718466687 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800b870 con 0x7f9c4815b250 2023-08-27T11:16:48.435 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4559a700 1 -- 172.21.15.138:0/2718466687 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3800b9d0 con 0x7f9c4815b250 2023-08-27T11:16:48.435 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.428+0000 7f9c4559a700 1 -- 172.21.15.138:0/2718466687 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800bd60 con 0x7f9c4815b250 2023-08-27T11:16:48.435 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.429+0000 7f9c4cd87700 1 -- 172.21.15.138:0/2718466687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c4815b250 msgr2=0x7f9c4815db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.435 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.429+0000 7f9c4cd87700 1 --2- 172.21.15.138:0/2718466687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c4815b250 0x7f9c4815db70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9c480d1680 tx=0x7f9c38009bd0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.436 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.429+0000 7f9c4cd87700 1 -- 172.21.15.138:0/2718466687 shutdown_connections 2023-08-27T11:16:48.436 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.429+0000 7f9c4cd87700 1 --2- 172.21.15.138:0/2718466687 >> v2:172.21.15.112:3300/0 conn(0x7f9c48066010 0x7f9c48066450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.436 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.429+0000 7f9c4cd87700 1 --2- 172.21.15.138:0/2718466687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c4815b250 0x7f9c4815db70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.436 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.429+0000 7f9c4cd87700 1 -- 172.21.15.138:0/2718466687 >> 172.21.15.138:0/2718466687 conn(0x7f9c4806f790 msgr2=0x7f9c4806fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:48.436 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.429+0000 7f9c4cd87700 1 -- 172.21.15.138:0/2718466687 shutdown_connections 2023-08-27T11:16:48.437 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.429+0000 7f9c4cd87700 1 -- 172.21.15.138:0/2718466687 wait complete. 2023-08-27T11:16:48.437 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.430+0000 7f9c4cd87700 1 Processor -- start 2023-08-27T11:16:48.437 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.431+0000 7f9c4cd87700 1 -- start start 2023-08-27T11:16:48.439 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.431+0000 7f9c4cd87700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c48066010 0x7f9c48147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.439 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.431+0000 7f9c4cd87700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c480ce590 0x7f9c480cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.440 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.431+0000 7f9c4cd87700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c4815b250 0x7f9c48159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.440 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.431+0000 7f9c4cd87700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4815aab0 con 0x7f9c48066010 2023-08-27T11:16:48.440 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.431+0000 7f9c4cd87700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4815a210 con 0x7f9c480ce590 2023-08-27T11:16:48.440 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.431+0000 7f9c4cd87700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4815a4f0 con 0x7f9c4815b250 2023-08-27T11:16:48.440 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.431+0000 7f9c45d9b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c480ce590 0x7f9c480cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.441 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.431+0000 7f9c46d9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c4815b250 0x7f9c48159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.441 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.431+0000 7f9c46d9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c4815b250 0x7f9c48159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:37916/0 (socket says 172.21.15.138:37916) 2023-08-27T11:16:48.441 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.431+0000 7f9c46d9d700 1 -- 172.21.15.138:0/3522482165 learned_addr learned my addr 172.21.15.138:0/3522482165 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:48.441 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.431+0000 7f9c4659c700 1 --2- 172.21.15.138:0/3522482165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c48066010 0x7f9c48147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.442 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c46d9d700 1 -- 172.21.15.138:0/3522482165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c480ce590 msgr2=0x7f9c480cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.442 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c46d9d700 1 --2- 172.21.15.138:0/3522482165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c480ce590 0x7f9c480cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.442 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c46d9d700 1 -- 172.21.15.138:0/3522482165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c48066010 msgr2=0x7f9c48147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.442 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c46d9d700 1 --2- 172.21.15.138:0/3522482165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c48066010 0x7f9c48147aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.442 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c46d9d700 1 -- 172.21.15.138:0/3522482165 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c38009550 con 0x7f9c4815b250 2023-08-27T11:16:48.443 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c45d9b700 1 --2- 172.21.15.138:0/3522482165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c480ce590 0x7f9c480cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:16:48.443 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c46d9d700 1 --2- 172.21.15.138:0/3522482165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c4815b250 0x7f9c48159400 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9c40007ef0 tx=0x7f9c4000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:48.443 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c36ffd700 1 -- 172.21.15.138:0/3522482165 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4000dd80 con 0x7f9c4815b250 2023-08-27T11:16:48.443 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c36ffd700 1 -- 172.21.15.138:0/3522482165 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4000f040 con 0x7f9c4815b250 2023-08-27T11:16:48.443 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c4cd87700 1 -- 172.21.15.138:0/3522482165 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4815a7d0 con 0x7f9c4815b250 2023-08-27T11:16:48.444 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c36ffd700 1 -- 172.21.15.138:0/3522482165 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40003c60 con 0x7f9c4815b250 2023-08-27T11:16:48.444 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c4cd87700 1 -- 172.21.15.138:0/3522482165 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c480cd6b0 con 0x7f9c4815b250 2023-08-27T11:16:48.444 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c36ffd700 1 -- 172.21.15.138:0/3522482165 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9c40004330 con 0x7f9c4815b250 2023-08-27T11:16:48.444 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.432+0000 7f9c36ffd700 1 -- 172.21.15.138:0/3522482165 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f9c400159c0 con 0x7f9c4815b250 2023-08-27T11:16:48.444 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.433+0000 7f9c4cd87700 1 -- 172.21.15.138:0/3522482165 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c48070710 con 0x7f9c4815b250 2023-08-27T11:16:48.445 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.435+0000 7f9c36ffd700 1 -- 172.21.15.138:0/3522482165 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f9c4001a070 con 0x7f9c4815b250 2023-08-27T11:16:48.518 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.517+0000 7f9c4cd87700 1 -- 172.21.15.138:0/3522482165 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a1d24950-2c63-4da5-88b4-b694c0650c80", "id": 5} v 0) v1 -- 0x7f9c480ca960 con 0x7f9c4815b250 2023-08-27T11:16:48.529 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.525+0000 7f9c36ffd700 1 -- 172.21.15.138:0/3522482165 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a1d24950-2c63-4da5-88b4-b694c0650c80", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9c480ca960 con 0x7f9c4815b250 2023-08-27T11:16:48.529 INFO:teuthology.orchestra.run.smithi138.stdout:5 2023-08-27T11:16:48.530 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.526+0000 7f9c4cd87700 1 -- 172.21.15.138:0/3522482165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c4815b250 msgr2=0x7f9c48159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.531 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.526+0000 7f9c4cd87700 1 --2- 172.21.15.138:0/3522482165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c4815b250 0x7f9c48159400 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9c40007ef0 tx=0x7f9c4000d3b0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.531 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.526+0000 7f9c4cd87700 1 -- 172.21.15.138:0/3522482165 shutdown_connections 2023-08-27T11:16:48.531 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.526+0000 7f9c4cd87700 1 --2- 172.21.15.138:0/3522482165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c4815b250 0x7f9c48159400 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.531 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.526+0000 7f9c4cd87700 1 --2- 172.21.15.138:0/3522482165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c48066010 0x7f9c48147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.531 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.526+0000 7f9c4cd87700 1 --2- 172.21.15.138:0/3522482165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c480ce590 0x7f9c480cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.532 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.526+0000 7f9c4cd87700 1 -- 172.21.15.138:0/3522482165 >> 172.21.15.138:0/3522482165 conn(0x7f9c4806f790 msgr2=0x7f9c48070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:48.532 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.526+0000 7f9c4cd87700 1 -- 172.21.15.138:0/3522482165 shutdown_connections 2023-08-27T11:16:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.526+0000 7f9c4cd87700 1 -- 172.21.15.138:0/3522482165 wait complete. 2023-08-27T11:16:48.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 0dc87f47-0452-42e0-bc6e-48d04491f63a 6 2023-08-27T11:16:48.695 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.694+0000 7f4e7821d700 1 Processor -- start 2023-08-27T11:16:48.696 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.694+0000 7f4e7821d700 1 -- start start 2023-08-27T11:16:48.699 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.695+0000 7f4e7821d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4e7015e190 0x7f4e7015e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.699 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.695+0000 7f4e7821d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e70161dd0 0x7f4e700b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.695+0000 7f4e7821d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e700b46c0 con 0x7f4e700cc980 2023-08-27T11:16:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.695+0000 7f4e7821d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e700b49a0 con 0x7f4e7015e190 2023-08-27T11:16:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.695+0000 7f4e7821d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e701612f0 con 0x7f4e70161dd0 2023-08-27T11:16:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.695+0000 7f4e757b8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e70161dd0 0x7f4e700b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.695+0000 7f4e75fb9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4e7015e190 0x7f4e7015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.701 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.695+0000 7f4e75fb9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4e7015e190 0x7f4e7015e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37950/0 (socket says 172.21.15.138:37950) 2023-08-27T11:16:48.701 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.695+0000 7f4e757b8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e70161dd0 0x7f4e700b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:37920/0 (socket says 172.21.15.138:37920) 2023-08-27T11:16:48.701 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.695+0000 7f4e75fb9700 1 -- 172.21.15.138:0/3154924450 learned_addr learned my addr 172.21.15.138:0/3154924450 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:48.701 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.696+0000 7f4e75fb9700 1 -- 172.21.15.138:0/3154924450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e70161dd0 msgr2=0x7f4e700b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:48.701 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.696+0000 7f4e75fb9700 1 --2- 172.21.15.138:0/3154924450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e70161dd0 0x7f4e700b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.702 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.696+0000 7f4e75fb9700 1 -- 172.21.15.138:0/3154924450 >> v1:172.21.15.138:6789/0 conn(0x7f4e700cc980 legacy=0x7f4e700ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.702 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.696+0000 7f4e75fb9700 1 -- 172.21.15.138:0/3154924450 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e701615d0 con 0x7f4e7015e190 2023-08-27T11:16:48.702 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.696+0000 7f4e74fb7700 1 -- 172.21.15.138:0/3154924450 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940139691 0 0) 0x7f4e700b46c0 con 0x7f4e700cc980 2023-08-27T11:16:48.702 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.696+0000 7f4e757b8700 1 --2- 172.21.15.138:0/3154924450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e70161dd0 0x7f4e700b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:16:48.702 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.696+0000 7f4e75fb9700 1 --2- 172.21.15.138:0/3154924450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e7015e190 0x7f4e7015e570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0098f0 tx=0x7f4e6c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64c964e430d98588 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.696+0000 7f4e74fb7700 1 -- 172.21.15.138:0/3154924450 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c011830 con 0x7f4e7015e190 2023-08-27T11:16:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.696+0000 7f4e74fb7700 1 -- 172.21.15.138:0/3154924450 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c011990 con 0x7f4e7015e190 2023-08-27T11:16:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.696+0000 7f4e74fb7700 1 -- 172.21.15.138:0/3154924450 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c011d20 con 0x7f4e7015e190 2023-08-27T11:16:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.697+0000 7f4e7821d700 1 -- 172.21.15.138:0/3154924450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e7015e190 msgr2=0x7f4e7015e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.697+0000 7f4e7821d700 1 --2- 172.21.15.138:0/3154924450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e7015e190 0x7f4e7015e570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0098f0 tx=0x7f4e6c009c00 comp rx=0 tx=0).stop 2023-08-27T11:16:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.697+0000 7f4e7821d700 1 -- 172.21.15.138:0/3154924450 shutdown_connections 2023-08-27T11:16:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.697+0000 7f4e7821d700 1 --2- 172.21.15.138:0/3154924450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e7015e190 0x7f4e7015e570 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.697+0000 7f4e7821d700 1 --2- 172.21.15.138:0/3154924450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e70161dd0 0x7f4e700b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.697+0000 7f4e7821d700 1 -- 172.21.15.138:0/3154924450 >> 172.21.15.138:0/3154924450 conn(0x7f4e7006bbf0 msgr2=0x7f4e700cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.697+0000 7f4e7821d700 1 -- 172.21.15.138:0/3154924450 shutdown_connections 2023-08-27T11:16:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.697+0000 7f4e7821d700 1 -- 172.21.15.138:0/3154924450 wait complete. 2023-08-27T11:16:48.705 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.698+0000 7f4e7821d700 1 Processor -- start 2023-08-27T11:16:48.705 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.698+0000 7f4e7821d700 1 -- start start 2023-08-27T11:16:48.707 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e7821d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700cc980 0x7f4e700720c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.707 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e7821d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e7015e190 0x7f4e700ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.707 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e7821d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e70161dd0 0x7f4e700b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e7821d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700af670 con 0x7f4e7015e190 2023-08-27T11:16:48.709 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e7821d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e70152470 con 0x7f4e70161dd0 2023-08-27T11:16:48.709 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e7821d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e70152750 con 0x7f4e700cc980 2023-08-27T11:16:48.709 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e75fb9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700cc980 0x7f4e700720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.709 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e767ba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e70161dd0 0x7f4e700b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e75fb9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700cc980 0x7f4e700720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:37924/0 (socket says 172.21.15.138:37924) 2023-08-27T11:16:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e75fb9700 1 -- 172.21.15.138:0/2741095448 learned_addr learned my addr 172.21.15.138:0/2741095448 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e767ba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e70161dd0 0x7f4e700b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37960/0 (socket says 172.21.15.138:37960) 2023-08-27T11:16:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e75fb9700 1 -- 172.21.15.138:0/2741095448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e70161dd0 msgr2=0x7f4e700b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e75fb9700 1 --2- 172.21.15.138:0/2741095448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e70161dd0 0x7f4e700b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e75fb9700 1 -- 172.21.15.138:0/2741095448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e7015e190 msgr2=0x7f4e700ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e75fb9700 1 --2- 172.21.15.138:0/2741095448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e7015e190 0x7f4e700ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e75fb9700 1 -- 172.21.15.138:0/2741095448 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e600094a0 con 0x7f4e700cc980 2023-08-27T11:16:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e75fb9700 1 --2- 172.21.15.138:0/2741095448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700cc980 0x7f4e700720c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009c40 tx=0x7f4e6c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e667fc700 1 -- 172.21.15.138:0/2741095448 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c004400 con 0x7f4e700cc980 2023-08-27T11:16:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e667fc700 1 -- 172.21.15.138:0/2741095448 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c01f070 con 0x7f4e700cc980 2023-08-27T11:16:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e7821d700 1 -- 172.21.15.138:0/2741095448 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c009550 con 0x7f4e700cc980 2023-08-27T11:16:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e667fc700 1 -- 172.21.15.138:0/2741095448 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0038a0 con 0x7f4e700cc980 2023-08-27T11:16:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.699+0000 7f4e7821d700 1 -- 172.21.15.138:0/2741095448 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e70072600 con 0x7f4e700cc980 2023-08-27T11:16:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.700+0000 7f4e667fc700 1 -- 172.21.15.138:0/2741095448 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4e6c021030 con 0x7f4e700cc980 2023-08-27T11:16:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.700+0000 7f4e667fc700 1 -- 172.21.15.138:0/2741095448 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f4e6c024430 con 0x7f4e700cc980 2023-08-27T11:16:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.700+0000 7f4e7821d700 1 -- 172.21.15.138:0/2741095448 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e70066fa0 con 0x7f4e700cc980 2023-08-27T11:16:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.702+0000 7f4e667fc700 1 -- 172.21.15.138:0/2741095448 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f4e6c015020 con 0x7f4e700cc980 2023-08-27T11:16:48.787 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.786+0000 7f4e7821d700 1 -- 172.21.15.138:0/2741095448 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0dc87f47-0452-42e0-bc6e-48d04491f63a", "id": 6} v 0) v1 -- 0x7f4e700b62e0 con 0x7f4e700cc980 2023-08-27T11:16:48.793 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.792+0000 7f4e667fc700 1 -- 172.21.15.138:0/2741095448 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0dc87f47-0452-42e0-bc6e-48d04491f63a", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4e6c01a450 con 0x7f4e700cc980 2023-08-27T11:16:48.794 INFO:teuthology.orchestra.run.smithi138.stdout:6 2023-08-27T11:16:48.795 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.793+0000 7f4e7821d700 1 -- 172.21.15.138:0/2741095448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700cc980 msgr2=0x7f4e700720c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.795 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.793+0000 7f4e7821d700 1 --2- 172.21.15.138:0/2741095448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700cc980 0x7f4e700720c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009c40 tx=0x7f4e6c004750 comp rx=0 tx=0).stop 2023-08-27T11:16:48.795 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.793+0000 7f4e7821d700 1 -- 172.21.15.138:0/2741095448 shutdown_connections 2023-08-27T11:16:48.795 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.793+0000 7f4e7821d700 1 --2- 172.21.15.138:0/2741095448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e70161dd0 0x7f4e700b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.796 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.793+0000 7f4e7821d700 1 --2- 172.21.15.138:0/2741095448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700cc980 0x7f4e700720c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.796 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.793+0000 7f4e7821d700 1 --2- 172.21.15.138:0/2741095448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e7015e190 0x7f4e700ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.796 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.793+0000 7f4e7821d700 1 -- 172.21.15.138:0/2741095448 >> 172.21.15.138:0/2741095448 conn(0x7f4e7006bbf0 msgr2=0x7f4e70155b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:48.796 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.793+0000 7f4e7821d700 1 -- 172.21.15.138:0/2741095448 shutdown_connections 2023-08-27T11:16:48.796 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.793+0000 7f4e7821d700 1 -- 172.21.15.138:0/2741095448 wait complete. 2023-08-27T11:16:48.808 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 23ba72a6-c505-4abf-9362-dba0e87146c2 7 2023-08-27T11:16:48.845 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:48.844+0000 7fcfe221f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:16:48.895 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:48.893+0000 7f28e4477200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:16:48.931 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.929+0000 7f8bdf2ca700 1 Processor -- start 2023-08-27T11:16:48.932 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.930+0000 7f8bdf2ca700 1 -- start start 2023-08-27T11:16:48.934 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.930+0000 7f8bdf2ca700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8bd81447a0 0x7f8bd80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.934 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.930+0000 7f8bdf2ca700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 0x7f8bd80b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.934 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.930+0000 7f8bdf2ca700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd80d00a0 con 0x7f8bd80d0920 2023-08-27T11:16:48.935 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.930+0000 7f8bdf2ca700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd80b3670 con 0x7f8bd81447a0 2023-08-27T11:16:48.935 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.930+0000 7f8bdf2ca700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8144b80 con 0x7f8bd8066720 2023-08-27T11:16:48.935 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bdd066700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8bd81447a0 0x7f8bd80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.935 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bdd867700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 0x7f8bd80b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.935 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bdd066700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8bd81447a0 0x7f8bd80cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37962/0 (socket says 172.21.15.138:37962) 2023-08-27T11:16:48.936 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bdd867700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 0x7f8bd80b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:37934/0 (socket says 172.21.15.138:37934) 2023-08-27T11:16:48.936 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bdd066700 1 -- 172.21.15.138:0/2258715789 learned_addr learned my addr 172.21.15.138:0/2258715789 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:48.936 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bdd066700 1 -- 172.21.15.138:0/2258715789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 msgr2=0x7f8bd80b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:48.936 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bdd066700 1 --2- 172.21.15.138:0/2258715789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 0x7f8bd80b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.937 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bdd066700 1 -- 172.21.15.138:0/2258715789 >> v1:172.21.15.112:6789/0 conn(0x7f8bd80d0920 legacy=0x7f8bd8066010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.937 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bdd066700 1 -- 172.21.15.138:0/2258715789 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd80b3ef0 con 0x7f8bd81447a0 2023-08-27T11:16:48.937 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bcffff700 1 -- 172.21.15.138:0/2258715789 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451199987 0 0) 0x7f8bd80d00a0 con 0x7f8bd80d0920 2023-08-27T11:16:48.937 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bdd867700 1 --2- 172.21.15.138:0/2258715789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 0x7f8bd80b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:16:48.937 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bdd066700 1 --2- 172.21.15.138:0/2258715789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bd81447a0 0x7f8bd80cfa70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009920 tx=0x7f8bd4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7478ca3d7a69adfe server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:48.938 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bcffff700 1 -- 172.21.15.138:0/2258715789 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4011830 con 0x7f8bd81447a0 2023-08-27T11:16:48.938 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bcffff700 1 -- 172.21.15.138:0/2258715789 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd4011990 con 0x7f8bd81447a0 2023-08-27T11:16:48.938 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.931+0000 7f8bcffff700 1 -- 172.21.15.138:0/2258715789 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4011d20 con 0x7f8bd81447a0 2023-08-27T11:16:48.938 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.932+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2258715789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bd81447a0 msgr2=0x7f8bd80cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.938 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.932+0000 7f8bdf2ca700 1 --2- 172.21.15.138:0/2258715789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bd81447a0 0x7f8bd80cfa70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009920 tx=0x7f8bd4009c30 comp rx=0 tx=0).stop 2023-08-27T11:16:48.939 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.932+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2258715789 shutdown_connections 2023-08-27T11:16:48.939 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.932+0000 7f8bdf2ca700 1 --2- 172.21.15.138:0/2258715789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 0x7f8bd80b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.939 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.932+0000 7f8bdf2ca700 1 --2- 172.21.15.138:0/2258715789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bd81447a0 0x7f8bd80cfa70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.940 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.932+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2258715789 >> 172.21.15.138:0/2258715789 conn(0x7f8bd806f790 msgr2=0x7f8bd806fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:48.940 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.932+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2258715789 shutdown_connections 2023-08-27T11:16:48.940 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.932+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2258715789 wait complete. 2023-08-27T11:16:48.944 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.934+0000 7f8bdf2ca700 1 Processor -- start 2023-08-27T11:16:48.944 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.934+0000 7f8bdf2ca700 1 -- start start 2023-08-27T11:16:48.945 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.934+0000 7f8bdf2ca700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 0x7f8bd8070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.945 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.934+0000 7f8bdf2ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd80d0920 0x7f8bd8071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.945 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.934+0000 7f8bdf2ca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bd81447a0 0x7f8bd8071860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:48.946 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.934+0000 7f8bdf2ca700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd813ed20 con 0x7f8bd80d0920 2023-08-27T11:16:48.946 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.934+0000 7f8bdf2ca700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd813e0a0 con 0x7f8bd81447a0 2023-08-27T11:16:48.946 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.934+0000 7f8bdf2ca700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd813e380 con 0x7f8bd8066720 2023-08-27T11:16:48.947 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bdd066700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 0x7f8bd8070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.947 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bdd867700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bd81447a0 0x7f8bd8071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.947 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bdd066700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 0x7f8bd8070de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:37936/0 (socket says 172.21.15.138:37936) 2023-08-27T11:16:48.948 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bdd066700 1 -- 172.21.15.138:0/2636117197 learned_addr learned my addr 172.21.15.138:0/2636117197 (peer_addr_for_me v2:172.21.15.138:0/0) 2023-08-27T11:16:48.948 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bdc865700 1 --2- 172.21.15.138:0/2636117197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd80d0920 0x7f8bd8071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:48.948 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bdd066700 1 -- 172.21.15.138:0/2636117197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bd81447a0 msgr2=0x7f8bd8071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.948 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bdd066700 1 --2- 172.21.15.138:0/2636117197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bd81447a0 0x7f8bd8071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.949 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bdd066700 1 -- 172.21.15.138:0/2636117197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd80d0920 msgr2=0x7f8bd8071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:48.949 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bdd066700 1 --2- 172.21.15.138:0/2636117197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd80d0920 0x7f8bd8071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:48.949 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bdd066700 1 -- 172.21.15.138:0/2636117197 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd0009a70 con 0x7f8bd8066720 2023-08-27T11:16:48.950 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bdd066700 1 --2- 172.21.15.138:0/2636117197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 0x7f8bd8070de0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4014040 tx=0x7f8bd4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:48.950 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bcdffb700 1 -- 172.21.15.138:0/2636117197 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd40044d0 con 0x7f8bd8066720 2023-08-27T11:16:48.950 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bcdffb700 1 -- 172.21.15.138:0/2636117197 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd401e070 con 0x7f8bd8066720 2023-08-27T11:16:48.951 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2636117197 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd4009550 con 0x7f8bd8066720 2023-08-27T11:16:48.951 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bcdffb700 1 -- 172.21.15.138:0/2636117197 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd40038e0 con 0x7f8bd8066720 2023-08-27T11:16:48.951 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.935+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2636117197 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd813e940 con 0x7f8bd8066720 2023-08-27T11:16:48.952 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.936+0000 7f8bcdffb700 1 -- 172.21.15.138:0/2636117197 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8bd4003d40 con 0x7f8bd8066720 2023-08-27T11:16:48.952 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.936+0000 7f8bcdffb700 1 -- 172.21.15.138:0/2636117197 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f8bd402db80 con 0x7f8bd8066720 2023-08-27T11:16:48.952 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.937+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2636117197 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd8070710 con 0x7f8bd8066720 2023-08-27T11:16:48.953 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:48.939+0000 7f8bcdffb700 1 -- 172.21.15.138:0/2636117197 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f8bd4020070 con 0x7f8bd8066720 2023-08-27T11:16:48.971 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:48.970+0000 7f28e4477200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:16:49.027 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:49.025+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2636117197 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "23ba72a6-c505-4abf-9362-dba0e87146c2", "id": 7} v 0) v1 -- 0x7f8bd8067a20 con 0x7f8bd8066720 2023-08-27T11:16:49.035 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:49.034+0000 7f8bcdffb700 1 -- 172.21.15.138:0/2636117197 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "23ba72a6-c505-4abf-9362-dba0e87146c2", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8bd4015020 con 0x7f8bd8066720 2023-08-27T11:16:49.035 INFO:teuthology.orchestra.run.smithi138.stdout:7 2023-08-27T11:16:49.036 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:49.034+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2636117197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 msgr2=0x7f8bd8070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:49.036 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:49.034+0000 7f8bdf2ca700 1 --2- 172.21.15.138:0/2636117197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 0x7f8bd8070de0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4014040 tx=0x7f8bd4004750 comp rx=0 tx=0).stop 2023-08-27T11:16:49.037 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:49.035+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2636117197 shutdown_connections 2023-08-27T11:16:49.037 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:49.035+0000 7f8bdf2ca700 1 --2- 172.21.15.138:0/2636117197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bd81447a0 0x7f8bd8071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:49.037 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:49.035+0000 7f8bdf2ca700 1 --2- 172.21.15.138:0/2636117197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8066720 0x7f8bd8070de0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:49.038 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:49.035+0000 7f8bdf2ca700 1 --2- 172.21.15.138:0/2636117197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd80d0920 0x7f8bd8071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:49.038 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:49.035+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2636117197 >> 172.21.15.138:0/2636117197 conn(0x7f8bd806f790 msgr2=0x7f8bd81435f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:49.038 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:49.035+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2636117197 shutdown_connections 2023-08-27T11:16:49.038 INFO:teuthology.orchestra.run.smithi138.stderr:2023-08-27T11:16:49.035+0000 7f8bdf2ca700 1 -- 172.21.15.138:0/2636117197 wait complete. 2023-08-27T11:16:49.048 INFO:tasks.ceph.osd.0:Restarting daemon 2023-08-27T11:16:49.048 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-08-27T11:16:49.051 INFO:tasks.ceph.osd.0:Started 2023-08-27T11:16:49.051 INFO:tasks.ceph.osd.1:Restarting daemon 2023-08-27T11:16:49.052 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-08-27T11:16:49.055 INFO:tasks.ceph.osd.1:Started 2023-08-27T11:16:49.055 INFO:tasks.ceph.osd.2:Restarting daemon 2023-08-27T11:16:49.056 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-08-27T11:16:49.058 INFO:tasks.ceph.osd.2:Started 2023-08-27T11:16:49.059 INFO:tasks.ceph.osd.3:Restarting daemon 2023-08-27T11:16:49.059 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-08-27T11:16:49.061 INFO:tasks.ceph.osd.3:Started 2023-08-27T11:16:49.061 INFO:tasks.ceph.osd.4:Restarting daemon 2023-08-27T11:16:49.062 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-08-27T11:16:49.063 INFO:tasks.ceph.osd.4:Started 2023-08-27T11:16:49.064 INFO:tasks.ceph.osd.5:Restarting daemon 2023-08-27T11:16:49.064 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-08-27T11:16:49.066 INFO:tasks.ceph.osd.5:Started 2023-08-27T11:16:49.066 INFO:tasks.ceph.osd.6:Restarting daemon 2023-08-27T11:16:49.067 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-08-27T11:16:49.069 INFO:tasks.ceph.osd.6:Started 2023-08-27T11:16:49.069 INFO:tasks.ceph.osd.7:Restarting daemon 2023-08-27T11:16:49.069 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-08-27T11:16:49.071 INFO:tasks.ceph.osd.7:Started 2023-08-27T11:16:49.072 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:16:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.238+0000 7f8352eed700 1 Processor -- start 2023-08-27T11:16:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.238+0000 7f8352eed700 1 -- start start 2023-08-27T11:16:49.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.239+0000 7f8352eed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b17e0 0x7f834c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.239+0000 7f8352eed700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f834c161ce0 0x7f834c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.239+0000 7f8352eed700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f834c161850 con 0x7f834c0b1400 2023-08-27T11:16:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.239+0000 7f8352eed700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f834c0cfc10 con 0x7f834c161ce0 2023-08-27T11:16:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.239+0000 7f8352eed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834c161200 con 0x7f834c0b17e0 2023-08-27T11:16:49.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.239+0000 7f834bfff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b17e0 0x7f834c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:49.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.239+0000 7f834bfff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b17e0 0x7f834c0cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:37590/0 (socket says 172.21.15.112:37590) 2023-08-27T11:16:49.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.239+0000 7f834bfff700 1 -- 172.21.15.112:0/3478616905 learned_addr learned my addr 172.21.15.112:0/3478616905 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:16:49.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.239+0000 7f835148a700 1 --2- 172.21.15.112:0/3478616905 >> v2:172.21.15.138:3300/0 conn(0x7f834c161ce0 0x7f834c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:49.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.240+0000 7f834bfff700 1 -- 172.21.15.112:0/3478616905 >> v2:172.21.15.138:3300/0 conn(0x7f834c161ce0 msgr2=0x7f834c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:49.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.240+0000 7f834bfff700 1 --2- 172.21.15.112:0/3478616905 >> v2:172.21.15.138:3300/0 conn(0x7f834c161ce0 0x7f834c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:49.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.240+0000 7f834bfff700 1 -- 172.21.15.112:0/3478616905 >> v1:172.21.15.138:6789/0 conn(0x7f834c0b1400 legacy=0x7f834c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:49.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.240+0000 7f834bfff700 1 -- 172.21.15.112:0/3478616905 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c1614e0 con 0x7f834c0b17e0 2023-08-27T11:16:49.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.240+0000 7f835148a700 1 --2- 172.21.15.112:0/3478616905 >> v2:172.21.15.138:3300/0 conn(0x7f834c161ce0 0x7f834c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:16:49.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.240+0000 7f834bfff700 1 --2- 172.21.15.112:0/3478616905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b17e0 0x7f834c0cf5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f83380098f0 tx=0x7f8338009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54b0327ca4c5aaf4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:49.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.240+0000 7f834b7fe700 1 -- 172.21.15.112:0/3478616905 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338011830 con 0x7f834c0b17e0 2023-08-27T11:16:49.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.240+0000 7f834b7fe700 1 -- 172.21.15.112:0/3478616905 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8338011990 con 0x7f834c0b17e0 2023-08-27T11:16:49.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.240+0000 7f834b7fe700 1 -- 172.21.15.112:0/3478616905 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338011d20 con 0x7f834c0b17e0 2023-08-27T11:16:49.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.241+0000 7f8352eed700 1 -- 172.21.15.112:0/3478616905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b17e0 msgr2=0x7f834c0cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:49.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.241+0000 7f8352eed700 1 --2- 172.21.15.112:0/3478616905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b17e0 0x7f834c0cf5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f83380098f0 tx=0x7f8338009c00 comp rx=0 tx=0).stop 2023-08-27T11:16:49.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.241+0000 7f8352eed700 1 -- 172.21.15.112:0/3478616905 shutdown_connections 2023-08-27T11:16:49.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.241+0000 7f8352eed700 1 --2- 172.21.15.112:0/3478616905 >> v2:172.21.15.138:3300/0 conn(0x7f834c161ce0 0x7f834c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:49.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.241+0000 7f8352eed700 1 --2- 172.21.15.112:0/3478616905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b17e0 0x7f834c0cf5a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:49.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.241+0000 7f8352eed700 1 -- 172.21.15.112:0/3478616905 >> 172.21.15.112:0/3478616905 conn(0x7f834c0d0000 msgr2=0x7f834c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:49.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.241+0000 7f8352eed700 1 -- 172.21.15.112:0/3478616905 shutdown_connections 2023-08-27T11:16:49.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.241+0000 7f8352eed700 1 -- 172.21.15.112:0/3478616905 wait complete. 2023-08-27T11:16:49.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.242+0000 7f8352eed700 1 Processor -- start 2023-08-27T11:16:49.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.242+0000 7f8352eed700 1 -- start start 2023-08-27T11:16:49.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.243+0000 7f8352eed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b1400 0x7f834c1f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:49.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.243+0000 7f8352eed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f834c0b17e0 0x7f834c1f3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:49.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.243+0000 7f8352eed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f834c161ce0 0x7f834c14d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:49.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.243+0000 7f8352eed700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c0c92e0 con 0x7f834c0b17e0 2023-08-27T11:16:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.243+0000 7f8352eed700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c1f3da0 con 0x7f834c161ce0 2023-08-27T11:16:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.243+0000 7f8352eed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834c0c83d0 con 0x7f834c0b1400 2023-08-27T11:16:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.243+0000 7f8350c89700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b1400 0x7f834c1f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.243+0000 7f834bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f834c0b17e0 0x7f834c1f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.243+0000 7f8350c89700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b1400 0x7f834c1f6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:37594/0 (socket says 172.21.15.112:37594) 2023-08-27T11:16:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.243+0000 7f8350c89700 1 -- 172.21.15.112:0/538528168 learned_addr learned my addr 172.21.15.112:0/538528168 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:16:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f835148a700 1 --2- 172.21.15.112:0/538528168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f834c161ce0 0x7f834c14d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f834bfff700 1 -- 172.21.15.112:0/538528168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b1400 msgr2=0x7f834c1f6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f834bfff700 1 --2- 172.21.15.112:0/538528168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b1400 0x7f834c1f6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f834bfff700 1 -- 172.21.15.112:0/538528168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f834c161ce0 msgr2=0x7f834c14d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f834bfff700 1 --2- 172.21.15.112:0/538528168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f834c161ce0 0x7f834c14d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f834bfff700 1 -- 172.21.15.112:0/538528168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8344009a70 con 0x7f834c0b17e0 2023-08-27T11:16:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f8350c89700 1 --2- 172.21.15.112:0/538528168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b1400 0x7f834c1f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:16:49.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f834bfff700 1 --2- 172.21.15.112:0/538528168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f834c0b17e0 0x7f834c1f3860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8338016000 tx=0x7f8338004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:49.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f83497fa700 1 -- 172.21.15.112:0/538528168 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83380043c0 con 0x7f834c0b17e0 2023-08-27T11:16:49.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f83497fa700 1 -- 172.21.15.112:0/538528168 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833801f070 con 0x7f834c0b17e0 2023-08-27T11:16:49.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f8352eed700 1 -- 172.21.15.112:0/538528168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8338009550 con 0x7f834c0b17e0 2023-08-27T11:16:49.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f83497fa700 1 -- 172.21.15.112:0/538528168 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83380036a0 con 0x7f834c0b17e0 2023-08-27T11:16:49.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f8352eed700 1 -- 172.21.15.112:0/538528168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834c1f63b0 con 0x7f834c0b17e0 2023-08-27T11:16:49.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f83497fa700 1 -- 172.21.15.112:0/538528168 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8338021030 con 0x7f834c0b17e0 2023-08-27T11:16:49.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.244+0000 7f83497fa700 1 -- 172.21.15.112:0/538528168 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f833802d440 con 0x7f834c0b17e0 2023-08-27T11:16:49.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.245+0000 7f8352eed700 1 -- 172.21.15.112:0/538528168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834c14da40 con 0x7f834c0b17e0 2023-08-27T11:16:49.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.247+0000 7f83497fa700 1 -- 172.21.15.112:0/538528168 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f8338015020 con 0x7f834c0b17e0 2023-08-27T11:16:49.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.350+0000 7f8352eed700 1 -- 172.21.15.112:0/538528168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f834c067a20 con 0x7f834c0b17e0 2023-08-27T11:16:49.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.351+0000 7f83497fa700 1 -- 172.21.15.112:0/538528168 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f833801a450 con 0x7f834c0b17e0 2023-08-27T11:16:49.353 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:49.353 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":10,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:16:49.027963+0000","last_up_change":"0.000000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:16:49.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.352+0000 7f8352eed700 1 -- 172.21.15.112:0/538528168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f834c0b17e0 msgr2=0x7f834c1f3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:49.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.352+0000 7f8352eed700 1 --2- 172.21.15.112:0/538528168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f834c0b17e0 0x7f834c1f3860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8338016000 tx=0x7f8338004750 comp rx=0 tx=0).stop 2023-08-27T11:16:49.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.352+0000 7f8352eed700 1 -- 172.21.15.112:0/538528168 shutdown_connections 2023-08-27T11:16:49.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.352+0000 7f8352eed700 1 --2- 172.21.15.112:0/538528168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f834c161ce0 0x7f834c14d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:49.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.352+0000 7f8352eed700 1 --2- 172.21.15.112:0/538528168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834c0b1400 0x7f834c1f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:49.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.352+0000 7f8352eed700 1 --2- 172.21.15.112:0/538528168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f834c0b17e0 0x7f834c1f3860 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:49.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.352+0000 7f8352eed700 1 -- 172.21.15.112:0/538528168 >> 172.21.15.112:0/538528168 conn(0x7f834c0d0000 msgr2=0x7f834c1579c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:49.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.352+0000 7f8352eed700 1 -- 172.21.15.112:0/538528168 shutdown_connections 2023-08-27T11:16:49.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:49.352+0000 7f8352eed700 1 -- 172.21.15.112:0/538528168 wait complete. 2023-08-27T11:16:49.365 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-08-27T11:16:49.366 INFO:tasks.ceph:Waiting for OSDs to come up 2023-08-27T11:16:49.528 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:49.527+0000 7fcfe221f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:16:49.596 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:49.595+0000 7fcfe221f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:16:49.669 INFO:tasks.ceph.osd.1.smithi112.stderr:2023-08-27T11:16:49.667+0000 7f3ef7eb5640 -1 Falling back to public interface 2023-08-27T11:16:49.671 INFO:tasks.ceph.osd.5.smithi138.stderr:2023-08-27T11:16:49.669+0000 7f0997c10640 -1 Falling back to public interface 2023-08-27T11:16:49.671 INFO:tasks.ceph.osd.2.smithi112.stderr:2023-08-27T11:16:49.670+0000 7f793b96a640 -1 Falling back to public interface 2023-08-27T11:16:49.675 INFO:tasks.ceph.osd.0.smithi112.stderr:2023-08-27T11:16:49.674+0000 7f45a3242640 -1 Falling back to public interface 2023-08-27T11:16:49.678 INFO:tasks.ceph.osd.6.smithi138.stderr:2023-08-27T11:16:49.677+0000 7fa5f2dc7640 -1 Falling back to public interface 2023-08-27T11:16:49.680 INFO:tasks.ceph.osd.3.smithi112.stderr:2023-08-27T11:16:49.679+0000 7f9b38e63640 -1 Falling back to public interface 2023-08-27T11:16:49.688 INFO:tasks.ceph.osd.7.smithi138.stderr:2023-08-27T11:16:49.686+0000 7fa6a2ee0640 -1 Falling back to public interface 2023-08-27T11:16:49.699 INFO:tasks.ceph.osd.4.smithi138.stderr:2023-08-27T11:16:49.697+0000 7febf9b80640 -1 Falling back to public interface 2023-08-27T11:16:49.708 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:49.706+0000 7f28e4477200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:16:49.786 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:49.784+0000 7f28e4477200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:16:49.925 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:49.923+0000 7f28e4477200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:16:50.160 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:50.158+0000 7f28e4477200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:16:50.172 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-27T11:16:50.297 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.296+0000 7f5e30dbf700 1 Processor -- start 2023-08-27T11:16:50.297 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.296+0000 7f5e30dbf700 1 -- start start 2023-08-27T11:16:50.298 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:50.296+0000 7fcfe221f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:16:50.300 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.297+0000 7f5e30dbf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2c06b260 0x7f5e2c0ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:50.300 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.297+0000 7f5e30dbf700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c0b32e0 con 0x7f5e2c161c30 2023-08-27T11:16:50.301 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.297+0000 7f5e30dbf700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c15e000 con 0x7f5e2c0cc490 2023-08-27T11:16:50.301 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.297+0000 7f5e30dbf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c1620b0 con 0x7f5e2c06b260 2023-08-27T11:16:50.301 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.297+0000 7f5e29d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5e2c161c30 0x7f5e2c0cbd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48562/0 (socket says 172.21.15.112:48562) 2023-08-27T11:16:50.301 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.297+0000 7f5e29d9b700 1 -- 172.21.15.112:0/3073557600 learned_addr learned my addr 172.21.15.112:0/3073557600 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:16:50.301 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.297+0000 7f5e2a59c700 1 --2- 172.21.15.112:0/3073557600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2c06b260 0x7f5e2c0ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:50.302 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.297+0000 7f5e2959a700 1 -- 172.21.15.112:0/3073557600 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3734549684 0 0) 0x7f5e2c0b32e0 con 0x7f5e2c161c30 2023-08-27T11:16:50.302 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.297+0000 7f5e2959a700 1 -- 172.21.15.112:0/3073557600 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e140035b0 con 0x7f5e2c161c30 2023-08-27T11:16:50.302 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.298+0000 7f5e2959a700 1 -- 172.21.15.112:0/3073557600 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235452085 0 0) 0x7f5e2c15e000 con 0x7f5e2c0cc490 2023-08-27T11:16:50.302 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.298+0000 7f5e2959a700 1 -- 172.21.15.112:0/3073557600 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e2c0b32e0 con 0x7f5e2c0cc490 2023-08-27T11:16:50.302 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.298+0000 7f5e2a59c700 1 -- 172.21.15.112:0/3073557600 >> v1:172.21.15.138:6789/0 conn(0x7f5e2c0cc490 legacy=0x7f5e2c0b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:50.302 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.298+0000 7f5e2a59c700 1 -- 172.21.15.112:0/3073557600 >> v1:172.21.15.112:6789/0 conn(0x7f5e2c161c30 legacy=0x7f5e2c0cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:50.303 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.298+0000 7f5e2959a700 1 -- 172.21.15.112:0/3073557600 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 155170392 0 0) 0x7f5e140035b0 con 0x7f5e2c161c30 2023-08-27T11:16:50.303 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.298+0000 7f5e2a59c700 1 -- 172.21.15.112:0/3073557600 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c15e2e0 con 0x7f5e2c06b260 2023-08-27T11:16:50.303 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.298+0000 7f5e2a59c700 1 --2- 172.21.15.112:0/3073557600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2c06b260 0x7f5e2c0ae6a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009870 tx=0x7f5e1c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8c0279ebbd49ba9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:50.303 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.298+0000 7f5e2959a700 1 -- 172.21.15.112:0/3073557600 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20001670 con 0x7f5e2c06b260 2023-08-27T11:16:50.303 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.298+0000 7f5e2959a700 1 -- 172.21.15.112:0/3073557600 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c004030 con 0x7f5e2c06b260 2023-08-27T11:16:50.304 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.298+0000 7f5e2959a700 1 -- 172.21.15.112:0/3073557600 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0043c0 con 0x7f5e2c06b260 2023-08-27T11:16:50.304 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.299+0000 7f5e30dbf700 1 -- 172.21.15.112:0/3073557600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2c06b260 msgr2=0x7f5e2c0ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:50.304 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.299+0000 7f5e30dbf700 1 --2- 172.21.15.112:0/3073557600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2c06b260 0x7f5e2c0ae6a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009870 tx=0x7f5e1c009b80 comp rx=0 tx=0).stop 2023-08-27T11:16:50.304 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.299+0000 7f5e30dbf700 1 -- 172.21.15.112:0/3073557600 shutdown_connections 2023-08-27T11:16:50.304 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.299+0000 7f5e30dbf700 1 --2- 172.21.15.112:0/3073557600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2c06b260 0x7f5e2c0ae6a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:50.304 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.299+0000 7f5e30dbf700 1 -- 172.21.15.112:0/3073557600 >> 172.21.15.112:0/3073557600 conn(0x7f5e2c0b24e0 msgr2=0x7f5e2c0b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:50.305 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.299+0000 7f5e30dbf700 1 -- 172.21.15.112:0/3073557600 shutdown_connections 2023-08-27T11:16:50.305 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.299+0000 7f5e30dbf700 1 -- 172.21.15.112:0/3073557600 wait complete. 2023-08-27T11:16:50.307 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.300+0000 7f5e30dbf700 1 Processor -- start 2023-08-27T11:16:50.307 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.300+0000 7f5e30dbf700 1 -- start start 2023-08-27T11:16:50.308 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e30dbf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2c06b260 0x7f5e2c098a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:50.308 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e30dbf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2c0cc490 0x7f5e2c0ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:50.308 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e30dbf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e2c161c30 0x7f5e2c0aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:50.308 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e30dbf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0ae7a0 con 0x7f5e2c161c30 2023-08-27T11:16:50.308 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e30dbf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0adb90 con 0x7f5e2c0cc490 2023-08-27T11:16:50.309 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e30dbf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0ade70 con 0x7f5e2c06b260 2023-08-27T11:16:50.309 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e2ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e2c161c30 0x7f5e2c0aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:50.309 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e2ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e2c161c30 0x7f5e2c0aeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54188/0 (socket says 172.21.15.112:54188) 2023-08-27T11:16:50.309 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e2ad9d700 1 -- 172.21.15.112:0/641911272 learned_addr learned my addr 172.21.15.112:0/641911272 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:16:50.310 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e29d9b700 1 --2- 172.21.15.112:0/641911272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2c0cc490 0x7f5e2c0ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:50.310 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e2a59c700 1 --2- 172.21.15.112:0/641911272 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2c06b260 0x7f5e2c098a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:50.310 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e2ad9d700 1 -- 172.21.15.112:0/641911272 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2c06b260 msgr2=0x7f5e2c098a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:50.310 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e2ad9d700 1 --2- 172.21.15.112:0/641911272 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2c06b260 0x7f5e2c098a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:50.310 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e2ad9d700 1 -- 172.21.15.112:0/641911272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2c0cc490 msgr2=0x7f5e2c0ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:50.311 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e2ad9d700 1 --2- 172.21.15.112:0/641911272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2c0cc490 0x7f5e2c0ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:50.311 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.301+0000 7f5e2ad9d700 1 -- 172.21.15.112:0/641911272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c0094a0 con 0x7f5e2c161c30 2023-08-27T11:16:50.311 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.302+0000 7f5e2ad9d700 1 --2- 172.21.15.112:0/641911272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e2c161c30 0x7f5e2c0aeff0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400cbc0 tx=0x7f5e2400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:50.311 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.302+0000 7f5e1affd700 1 -- 172.21.15.112:0/641911272 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24003c00 con 0x7f5e2c161c30 2023-08-27T11:16:50.311 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.302+0000 7f5e1affd700 1 -- 172.21.15.112:0/641911272 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e24003d60 con 0x7f5e2c161c30 2023-08-27T11:16:50.311 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.302+0000 7f5e30dbf700 1 -- 172.21.15.112:0/641911272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c0ae150 con 0x7f5e2c161c30 2023-08-27T11:16:50.312 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.302+0000 7f5e1affd700 1 -- 172.21.15.112:0/641911272 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400aa20 con 0x7f5e2c161c30 2023-08-27T11:16:50.312 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.302+0000 7f5e30dbf700 1 -- 172.21.15.112:0/641911272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c0c6e10 con 0x7f5e2c161c30 2023-08-27T11:16:50.312 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.302+0000 7f5e1affd700 1 -- 172.21.15.112:0/641911272 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5e240050d0 con 0x7f5e2c161c30 2023-08-27T11:16:50.312 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.302+0000 7f5e1affd700 1 -- 172.21.15.112:0/641911272 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f5e24012c30 con 0x7f5e2c161c30 2023-08-27T11:16:50.312 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.303+0000 7f5e30dbf700 1 -- 172.21.15.112:0/641911272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c0ae910 con 0x7f5e2c161c30 2023-08-27T11:16:50.313 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.305+0000 7f5e1affd700 1 -- 172.21.15.112:0/641911272 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f5e24005560 con 0x7f5e2c161c30 2023-08-27T11:16:50.374 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:50.373+0000 7f28e4477200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:16:50.375 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.374+0000 7f5e30dbf700 1 -- 172.21.15.112:0/641911272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e2c0aeb30 con 0x7f5e2c161c30 2023-08-27T11:16:50.376 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.375+0000 7f5e1affd700 1 -- 172.21.15.112:0/641911272 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f5e24005700 con 0x7f5e2c161c30 2023-08-27T11:16:50.376 INFO:teuthology.misc.health.smithi112.stdout: 2023-08-27T11:16:50.376 INFO:teuthology.misc.health.smithi112.stdout:{"epoch":10,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:16:49.027963+0000","last_up_change":"0.000000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:16:50.377 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.376+0000 7f5e30dbf700 1 -- 172.21.15.112:0/641911272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e2c161c30 msgr2=0x7f5e2c0aeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:50.378 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.376+0000 7f5e30dbf700 1 --2- 172.21.15.112:0/641911272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e2c161c30 0x7f5e2c0aeff0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400cbc0 tx=0x7f5e2400ced0 comp rx=0 tx=0).stop 2023-08-27T11:16:50.378 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.376+0000 7f5e30dbf700 1 -- 172.21.15.112:0/641911272 shutdown_connections 2023-08-27T11:16:50.378 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.376+0000 7f5e30dbf700 1 --2- 172.21.15.112:0/641911272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e2c161c30 0x7f5e2c0aeff0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:50.378 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.376+0000 7f5e30dbf700 1 --2- 172.21.15.112:0/641911272 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2c06b260 0x7f5e2c098a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:50.378 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.376+0000 7f5e30dbf700 1 --2- 172.21.15.112:0/641911272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2c0cc490 0x7f5e2c0ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:50.379 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.376+0000 7f5e30dbf700 1 -- 172.21.15.112:0/641911272 >> 172.21.15.112:0/641911272 conn(0x7f5e2c0b24e0 msgr2=0x7f5e2c0cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:50.379 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.376+0000 7f5e30dbf700 1 -- 172.21.15.112:0/641911272 shutdown_connections 2023-08-27T11:16:50.379 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:50.376+0000 7f5e30dbf700 1 -- 172.21.15.112:0/641911272 wait complete. 2023-08-27T11:16:50.379 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:50.377+0000 7fcfe221f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:16:50.388 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-08-27T11:16:50.507 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:50.506+0000 7f28e4477200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:16:50.520 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:50.519+0000 7fcfe221f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:16:50.725 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:50.724+0000 7f28e4477200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:16:50.755 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:50.753+0000 7fcfe221f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:16:50.881 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:50.880+0000 7f28e4477200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:16:50.965 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:50.963+0000 7fcfe221f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:16:50.994 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:50.992+0000 7f28e4477200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:16:51.096 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:51.095+0000 7fcfe221f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:16:51.312 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:51.311+0000 7fcfe221f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:16:51.320 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:51.318+0000 7f28e4477200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:16:51.419 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:51.418+0000 7f28e4477200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:16:51.468 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:51.465+0000 7fcfe221f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:16:51.581 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:51.579+0000 7fcfe221f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:16:51.605 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:51.604+0000 7f28e4477200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:16:51.900 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:51.900+0000 7fcfe221f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:16:51.998 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:51.998+0000 7fcfe221f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:16:52.190 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:52.189+0000 7fcfe221f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:16:52.204 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:52.203+0000 7f28e4477200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:16:52.835 INFO:tasks.ceph.mgr.y.smithi112.stderr:2023-08-27T11:16:52.835+0000 7fcfe221f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:16:53.176 INFO:tasks.ceph.osd.1.smithi112.stderr:2023-08-27T11:16:53.176+0000 7f3ef7eb5640 -1 osd.1 0 log_to_monitors true 2023-08-27T11:16:53.185 INFO:tasks.ceph.osd.2.smithi112.stderr:2023-08-27T11:16:53.185+0000 7f793b96a640 -1 osd.2 0 log_to_monitors true 2023-08-27T11:16:53.197 INFO:tasks.ceph.osd.6.smithi138.stderr:2023-08-27T11:16:53.196+0000 7fa5f2dc7640 -1 osd.6 0 log_to_monitors true 2023-08-27T11:16:53.203 INFO:tasks.ceph.osd.0.smithi112.stderr:2023-08-27T11:16:53.203+0000 7f45a3242640 -1 osd.0 0 log_to_monitors true 2023-08-27T11:16:53.208 INFO:tasks.ceph.osd.5.smithi138.stderr:2023-08-27T11:16:53.206+0000 7f0997c10640 -1 osd.5 0 log_to_monitors true 2023-08-27T11:16:53.212 INFO:tasks.ceph.osd.7.smithi138.stderr:2023-08-27T11:16:53.211+0000 7fa6a2ee0640 -1 osd.7 0 log_to_monitors true 2023-08-27T11:16:53.212 INFO:tasks.ceph.osd.3.smithi112.stderr:2023-08-27T11:16:53.212+0000 7f9b38e63640 -1 osd.3 0 log_to_monitors true 2023-08-27T11:16:53.218 INFO:tasks.ceph.osd.4.smithi138.stderr:2023-08-27T11:16:53.217+0000 7febf9b80640 -1 osd.4 0 log_to_monitors true 2023-08-27T11:16:53.229 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:53.227+0000 7f289c488700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-08-27T11:16:53.229 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:16:53.228+0000 7f289c488700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-08-27T11:16:56.234 INFO:tasks.ceph.osd.5.smithi138.stderr:2023-08-27T11:16:56.233+0000 7f098b7c7700 -1 osd.5 0 waiting for initial osdmap 2023-08-27T11:16:56.250 INFO:tasks.ceph.osd.4.smithi138.stderr:2023-08-27T11:16:56.249+0000 7febed737700 -1 osd.4 0 waiting for initial osdmap 2023-08-27T11:16:56.251 INFO:tasks.ceph.osd.6.smithi138.stderr:2023-08-27T11:16:56.250+0000 7fa5e8181700 -1 osd.6 0 waiting for initial osdmap 2023-08-27T11:16:56.257 INFO:tasks.ceph.osd.0.smithi112.stderr:2023-08-27T11:16:56.253+0000 7f4596df9700 -1 osd.0 0 waiting for initial osdmap 2023-08-27T11:16:56.257 INFO:tasks.ceph.osd.5.smithi138.stderr:2023-08-27T11:16:56.255+0000 7f0983ba2700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:16:56.262 INFO:tasks.ceph.osd.4.smithi138.stderr:2023-08-27T11:16:56.261+0000 7febe5b12700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:16:56.262 INFO:tasks.ceph.osd.6.smithi138.stderr:2023-08-27T11:16:56.261+0000 7fa5ded59700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:16:56.268 INFO:tasks.ceph.osd.0.smithi112.stderr:2023-08-27T11:16:56.267+0000 7f458f1d4700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:16:56.313 INFO:tasks.ceph.osd.1.smithi112.stderr:2023-08-27T11:16:56.311+0000 7f3eeba6c700 -1 osd.1 0 waiting for initial osdmap 2023-08-27T11:16:56.322 INFO:tasks.ceph.osd.3.smithi112.stderr:2023-08-27T11:16:56.320+0000 7f9b2ca1a700 -1 osd.3 0 waiting for initial osdmap 2023-08-27T11:16:56.325 INFO:tasks.ceph.osd.1.smithi112.stderr:2023-08-27T11:16:56.324+0000 7f3ee3e47700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:16:56.332 INFO:tasks.ceph.osd.2.smithi112.stderr:2023-08-27T11:16:56.331+0000 7f792f521700 -1 osd.2 0 waiting for initial osdmap 2023-08-27T11:16:56.334 INFO:tasks.ceph.osd.3.smithi112.stderr:2023-08-27T11:16:56.333+0000 7f9b24df5700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:16:56.336 INFO:tasks.ceph.osd.7.smithi138.stderr:2023-08-27T11:16:56.335+0000 7fa696a97700 -1 osd.7 0 waiting for initial osdmap 2023-08-27T11:16:56.347 INFO:tasks.ceph.osd.2.smithi112.stderr:2023-08-27T11:16:56.346+0000 7f79278fc700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:16:56.354 INFO:tasks.ceph.osd.7.smithi138.stderr:2023-08-27T11:16:56.353+0000 7fa68ee72700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:16:57.195 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-27T11:16:57.324 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.323+0000 7f7c07328700 1 Processor -- start 2023-08-27T11:16:57.325 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.323+0000 7f7c07328700 1 -- start start 2023-08-27T11:16:57.326 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.324+0000 7f7c07328700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001596d0 0x7f7c000cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.326 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.324+0000 7f7c07328700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7c000700e0 0x7f7c00070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.327 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.324+0000 7f7c07328700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c0006d2f0 con 0x7f7c000cd580 2023-08-27T11:16:57.327 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.324+0000 7f7c07328700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c00071840 con 0x7f7c000700e0 2023-08-27T11:16:57.327 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.324+0000 7f7c07328700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0006d100 con 0x7f7c001596d0 2023-08-27T11:16:57.327 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.324+0000 7f7c048c3700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7c000cd580 0x7f7c0006d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48756/0 (socket says 172.21.15.112:48756) 2023-08-27T11:16:57.327 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.324+0000 7f7c048c3700 1 -- 172.21.15.112:0/2967321063 learned_addr learned my addr 172.21.15.112:0/2967321063 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:16:57.328 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.324+0000 7f7c058c5700 1 --2- 172.21.15.112:0/2967321063 >> v2:172.21.15.138:3300/0 conn(0x7f7c000700e0 0x7f7c00070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.328 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.324+0000 7f7c050c4700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001596d0 0x7f7c000cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.328 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.324+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 730633162 0 0) 0x7f7c0006d2f0 con 0x7f7c000cd580 2023-08-27T11:16:57.328 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.324+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bec0035b0 con 0x7f7c000cd580 2023-08-27T11:16:57.328 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2521701570 0 0) 0x7f7bec0035b0 con 0x7f7c000cd580 2023-08-27T11:16:57.329 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001596d0 msgr2=0x7f7c000cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:57.329 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001596d0 0x7f7c000cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.329 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 >> v2:172.21.15.138:3300/0 conn(0x7f7c000700e0 msgr2=0x7f7c00070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:57.329 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 --2- 172.21.15.112:0/2967321063 >> v2:172.21.15.138:3300/0 conn(0x7f7c000700e0 0x7f7c00070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.329 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c000ca5c0 con 0x7f7c000cd580 2023-08-27T11:16:57.334 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f7bfc001670 con 0x7f7c000cd580 2023-08-27T11:16:57.334 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 >> v1:172.21.15.112:6789/0 conn(0x7f7c000cd580 legacy=0x7f7c0006d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:57.334 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bec003a00 0x7f7bec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.334 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001596d0 0x7f7bec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.335 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bec004d90 0x7f7bec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.335 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c00071840 con 0x7f7bec003a00 2023-08-27T11:16:57.335 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec00ba10 con 0x7f7bec004d90 2023-08-27T11:16:57.335 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec00bb50 con 0x7f7c001596d0 2023-08-27T11:16:57.335 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7c050c4700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bec003a00 0x7f7bec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.336 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7c048c3700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001596d0 0x7f7bec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.336 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7c058c5700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bec004d90 0x7f7bec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.336 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7c050c4700 1 -- 172.21.15.112:0/2967321063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001596d0 msgr2=0x7f7bec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:57.336 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7c050c4700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001596d0 0x7f7bec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.336 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7c050c4700 1 -- 172.21.15.112:0/2967321063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bec004d90 msgr2=0x7f7bec005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:57.337 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7c050c4700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bec004d90 0x7f7bec005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.337 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7c050c4700 1 -- 172.21.15.112:0/2967321063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec0035b0 con 0x7f7bec003a00 2023-08-27T11:16:57.337 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7c048c3700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001596d0 0x7f7bec004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:16:57.337 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7c058c5700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bec004d90 0x7f7bec005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:16:57.337 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7c050c4700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bec003a00 0x7f7bec003e20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009b60 tx=0x7f7bec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a62078eea5a3a3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:57.338 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400e6c0 con 0x7f7bec003a00 2023-08-27T11:16:57.338 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf400e820 con 0x7f7bec003a00 2023-08-27T11:16:57.338 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.325+0000 7f7bf3fff700 1 -- 172.21.15.112:0/2967321063 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400eb90 con 0x7f7bec003a00 2023-08-27T11:16:57.338 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.326+0000 7f7c07328700 1 -- 172.21.15.112:0/2967321063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bec003a00 msgr2=0x7f7bec003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:57.339 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.326+0000 7f7c07328700 1 --2- 172.21.15.112:0/2967321063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bec003a00 0x7f7bec003e20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009b60 tx=0x7f7bec005880 comp rx=0 tx=0).stop 2023-08-27T11:16:57.339 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.326+0000 7f7c058c5700 1 -- 172.21.15.112:0/2967321063 reap_dead start 2023-08-27T11:16:57.339 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.326+0000 7f7c07328700 1 -- 172.21.15.112:0/2967321063 shutdown_connections 2023-08-27T11:16:57.339 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.326+0000 7f7c07328700 1 -- 172.21.15.112:0/2967321063 >> 172.21.15.112:0/2967321063 conn(0x7f7c000cee00 msgr2=0x7f7c000cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:57.339 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.326+0000 7f7c07328700 1 -- 172.21.15.112:0/2967321063 shutdown_connections 2023-08-27T11:16:57.340 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.326+0000 7f7c07328700 1 -- 172.21.15.112:0/2967321063 wait complete. 2023-08-27T11:16:57.340 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.327+0000 7f7c07328700 1 Processor -- start 2023-08-27T11:16:57.340 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.327+0000 7f7c07328700 1 -- start start 2023-08-27T11:16:57.340 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c07328700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bec003a00 0x7f7c001537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.340 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c07328700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c00150920 0x7f7c001f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.340 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c07328700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001f0900 0x7f7c001f0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.341 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c07328700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0015ab20 con 0x7f7c00150920 2023-08-27T11:16:57.341 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c07328700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c00159ea0 con 0x7f7bec003a00 2023-08-27T11:16:57.341 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c07328700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0015a180 con 0x7f7c001f0900 2023-08-27T11:16:57.341 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c048c3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c00150920 0x7f7c001f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.341 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c048c3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c00150920 0x7f7c001f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54388/0 (socket says 172.21.15.112:54388) 2023-08-27T11:16:57.342 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c048c3700 1 -- 172.21.15.112:0/1282922892 learned_addr learned my addr 172.21.15.112:0/1282922892 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:16:57.342 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c050c4700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bec003a00 0x7f7c001537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.342 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c058c5700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001f0900 0x7f7c001f0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.342 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c048c3700 1 -- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001f0900 msgr2=0x7f7c001f0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:57.342 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c048c3700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001f0900 0x7f7c001f0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.343 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c048c3700 1 -- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bec003a00 msgr2=0x7f7c001537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:57.343 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c048c3700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bec003a00 0x7f7c001537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.343 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.328+0000 7f7c048c3700 1 -- 172.21.15.112:0/1282922892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0015a460 con 0x7f7c00150920 2023-08-27T11:16:57.343 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.329+0000 7f7c048c3700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c00150920 0x7f7c001f3580 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00eaf0 tx=0x7f7bfc00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:57.343 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.329+0000 7f7bf1ffb700 1 -- 172.21.15.112:0/1282922892 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc005e00 con 0x7f7c00150920 2023-08-27T11:16:57.344 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.329+0000 7f7bf1ffb700 1 -- 172.21.15.112:0/1282922892 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bfc00c7e0 con 0x7f7c00150920 2023-08-27T11:16:57.344 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.329+0000 7f7c07328700 1 -- 172.21.15.112:0/1282922892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0015a740 con 0x7f7c00150920 2023-08-27T11:16:57.344 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.329+0000 7f7bf1ffb700 1 -- 172.21.15.112:0/1282922892 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc00cb70 con 0x7f7c00150920 2023-08-27T11:16:57.344 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.329+0000 7f7c07328700 1 -- 172.21.15.112:0/1282922892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c00153ef0 con 0x7f7c00150920 2023-08-27T11:16:57.344 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.330+0000 7f7c07328700 1 -- 172.21.15.112:0/1282922892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c00074850 con 0x7f7c00150920 2023-08-27T11:16:57.344 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.330+0000 7f7bf1ffb700 1 -- 172.21.15.112:0/1282922892 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7f7bfc00ccd0 con 0x7f7c00150920 2023-08-27T11:16:57.345 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.330+0000 7f7bf1ffb700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7be406a080 0x7f7be406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.345 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.331+0000 7f7bf1ffb700 1 -- 172.21.15.112:0/1282922892 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f7bfc087950 con 0x7f7c00150920 2023-08-27T11:16:57.345 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.331+0000 7f7c050c4700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7be406a080 0x7f7be406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.345 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.331+0000 7f7c050c4700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7be406a080 0x7f7be406c540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40090b0 tx=0x7f7bf4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:57.346 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.335+0000 7f7bf1ffb700 1 -- 172.21.15.112:0/1282922892 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7bfc0537e0 con 0x7f7c00150920 2023-08-27T11:16:57.526 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.525+0000 7f7c07328700 1 -- 172.21.15.112:0/1282922892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c00153ef0 con 0x7f7c00150920 2023-08-27T11:16:57.527 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.525+0000 7f7bf1ffb700 1 -- 172.21.15.112:0/1282922892 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10932 (secure 0 0 0) 0x7f7c00153ef0 con 0x7f7c00150920 2023-08-27T11:16:57.527 INFO:teuthology.misc.health.smithi112.stdout: 2023-08-27T11:16:57.527 INFO:teuthology.misc.health.smithi112.stdout:{"epoch":13,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:16:57.262608+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:16:57.529 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.527+0000 7f7c07328700 1 -- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7be406a080 msgr2=0x7f7be406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:57.529 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.527+0000 7f7c07328700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7be406a080 0x7f7be406c540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40090b0 tx=0x7f7bf4008030 comp rx=0 tx=0).stop 2023-08-27T11:16:57.530 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.527+0000 7f7c07328700 1 -- 172.21.15.112:0/1282922892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c00150920 msgr2=0x7f7c001f3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:57.530 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.527+0000 7f7c07328700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c00150920 0x7f7c001f3580 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00eaf0 tx=0x7f7bfc00ee00 comp rx=0 tx=0).stop 2023-08-27T11:16:57.530 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.528+0000 7f7c07328700 1 -- 172.21.15.112:0/1282922892 shutdown_connections 2023-08-27T11:16:57.530 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.528+0000 7f7c07328700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7be406a080 0x7f7be406c540 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.530 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.528+0000 7f7c07328700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c001f0900 0x7f7c001f0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.531 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.528+0000 7f7c07328700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bec003a00 0x7f7c001537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.531 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.528+0000 7f7c07328700 1 --2- 172.21.15.112:0/1282922892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c00150920 0x7f7c001f3580 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.531 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.528+0000 7f7c07328700 1 -- 172.21.15.112:0/1282922892 >> 172.21.15.112:0/1282922892 conn(0x7f7c000cee00 msgr2=0x7f7c0006c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:57.531 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.528+0000 7f7c07328700 1 -- 172.21.15.112:0/1282922892 shutdown_connections 2023-08-27T11:16:57.531 INFO:teuthology.misc.health.smithi112.stderr:2023-08-27T11:16:57.528+0000 7f7c07328700 1 -- 172.21.15.112:0/1282922892 wait complete. 2023-08-27T11:16:57.540 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-08-27T11:16:57.541 INFO:tasks.ceph:Creating RBD pool 2023-08-27T11:16:57.541 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd pool create rbd 8 2023-08-27T11:16:57.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.702+0000 7f032e3d6700 1 Processor -- start 2023-08-27T11:16:57.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.702+0000 7f032e3d6700 1 -- start start 2023-08-27T11:16:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f032e3d6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0328146c20 0x7f0328160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f032e3d6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03280b3440 0x7f032815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f032e3d6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f032815e6e0 0x7f032806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f032e3d6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03280b5070 con 0x7f032815e6e0 2023-08-27T11:16:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f032e3d6700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0328147000 con 0x7f0328146c20 2023-08-27T11:16:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f032e3d6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03281614a0 con 0x7f03280b3440 2023-08-27T11:16:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f032c973700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f032815e6e0 0x7f032806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f032c973700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f032815e6e0 0x7f032806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54398/0 (socket says 172.21.15.112:54398) 2023-08-27T11:16:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f032c973700 1 -- 172.21.15.112:0/2959803794 learned_addr learned my addr 172.21.15.112:0/2959803794 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:16:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f0327fff700 1 --2- 172.21.15.112:0/2959803794 >> v2:172.21.15.138:3300/0 conn(0x7f0328146c20 0x7f0328160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f03277fe700 1 --2- 172.21.15.112:0/2959803794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03280b3440 0x7f032815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f032c973700 1 -- 172.21.15.112:0/2959803794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03280b3440 msgr2=0x7f032815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f032c973700 1 --2- 172.21.15.112:0/2959803794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03280b3440 0x7f032815e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f032c973700 1 -- 172.21.15.112:0/2959803794 >> v2:172.21.15.138:3300/0 conn(0x7f0328146c20 msgr2=0x7f0328160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.703+0000 7f032c973700 1 --2- 172.21.15.112:0/2959803794 >> v2:172.21.15.138:3300/0 conn(0x7f0328146c20 0x7f0328160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f032c973700 1 -- 172.21.15.112:0/2959803794 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0328161740 con 0x7f032815e6e0 2023-08-27T11:16:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f032c973700 1 --2- 172.21.15.112:0/2959803794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f032815e6e0 0x7f032806f790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f03280b2d00 tx=0x7f032000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07e267032274c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f0326ffd700 1 -- 172.21.15.112:0/2959803794 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032000e070 con 0x7f032815e6e0 2023-08-27T11:16:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f0326ffd700 1 -- 172.21.15.112:0/2959803794 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032000b910 con 0x7f032815e6e0 2023-08-27T11:16:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f0326ffd700 1 -- 172.21.15.112:0/2959803794 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032000bcc0 con 0x7f032815e6e0 2023-08-27T11:16:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f032e3d6700 1 -- 172.21.15.112:0/2959803794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f032815e6e0 msgr2=0x7f032806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f032e3d6700 1 --2- 172.21.15.112:0/2959803794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f032815e6e0 0x7f032806f790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f03280b2d00 tx=0x7f032000b330 comp rx=0 tx=0).stop 2023-08-27T11:16:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f032e3d6700 1 -- 172.21.15.112:0/2959803794 shutdown_connections 2023-08-27T11:16:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f032e3d6700 1 --2- 172.21.15.112:0/2959803794 >> v2:172.21.15.138:3300/0 conn(0x7f0328146c20 0x7f0328160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f032e3d6700 1 --2- 172.21.15.112:0/2959803794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f032815e6e0 0x7f032806f790 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f032e3d6700 1 --2- 172.21.15.112:0/2959803794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03280b3440 0x7f032815e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f032e3d6700 1 -- 172.21.15.112:0/2959803794 >> 172.21.15.112:0/2959803794 conn(0x7f03280b1e90 msgr2=0x7f03280b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f032e3d6700 1 -- 172.21.15.112:0/2959803794 shutdown_connections 2023-08-27T11:16:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.704+0000 7f032e3d6700 1 -- 172.21.15.112:0/2959803794 wait complete. 2023-08-27T11:16:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.706+0000 7f032e3d6700 1 Processor -- start 2023-08-27T11:16:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.706+0000 7f032e3d6700 1 -- start start 2023-08-27T11:16:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f032e3d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03280b3440 0x7f03280cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f032e3d6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0328146c20 0x7f0328145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f032e3d6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f032815e6e0 0x7f03281f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f032e3d6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03281f6c20 con 0x7f03280b3440 2023-08-27T11:16:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f032e3d6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0328146280 con 0x7f032815e6e0 2023-08-27T11:16:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f032e3d6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0328140f00 con 0x7f0328146c20 2023-08-27T11:16:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f0327fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03280b3440 0x7f03280cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f0327fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03280b3440 0x7f03280cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54400/0 (socket says 172.21.15.112:54400) 2023-08-27T11:16:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f0327fff700 1 -- 172.21.15.112:0/665218711 learned_addr learned my addr 172.21.15.112:0/665218711 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:16:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f0327fff700 1 -- 172.21.15.112:0/665218711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0328146c20 msgr2=0x7f0328145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f0327fff700 1 --2- 172.21.15.112:0/665218711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0328146c20 0x7f0328145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f0327fff700 1 -- 172.21.15.112:0/665218711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f032815e6e0 msgr2=0x7f03281f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f0327fff700 1 --2- 172.21.15.112:0/665218711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f032815e6e0 0x7f03281f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.707+0000 7f0327fff700 1 -- 172.21.15.112:0/665218711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0320009b30 con 0x7f03280b3440 2023-08-27T11:16:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.708+0000 7f0327fff700 1 --2- 172.21.15.112:0/665218711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03280b3440 0x7f03280cd900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f031c00e9b0 tx=0x7f031c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.708+0000 7f0324ff9700 1 -- 172.21.15.112:0/665218711 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c009710 con 0x7f03280b3440 2023-08-27T11:16:57.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.708+0000 7f0324ff9700 1 -- 172.21.15.112:0/665218711 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c00c850 con 0x7f03280b3440 2023-08-27T11:16:57.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.708+0000 7f032e3d6700 1 -- 172.21.15.112:0/665218711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032813f760 con 0x7f03280b3440 2023-08-27T11:16:57.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.708+0000 7f0324ff9700 1 -- 172.21.15.112:0/665218711 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c00cbc0 con 0x7f03280b3440 2023-08-27T11:16:57.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.708+0000 7f032e3d6700 1 -- 172.21.15.112:0/665218711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03281411a0 con 0x7f03280b3440 2023-08-27T11:16:57.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.709+0000 7f032e3d6700 1 -- 172.21.15.112:0/665218711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03280af990 con 0x7f03280b3440 2023-08-27T11:16:57.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.709+0000 7f0324ff9700 1 -- 172.21.15.112:0/665218711 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7f031c00cd20 con 0x7f03280b3440 2023-08-27T11:16:57.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.709+0000 7f0324ff9700 1 --2- 172.21.15.112:0/665218711 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f031406a050 0x7f031406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:57.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.709+0000 7f0324ff9700 1 -- 172.21.15.112:0/665218711 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f031c0882f0 con 0x7f03280b3440 2023-08-27T11:16:57.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.710+0000 7f03277fe700 1 --2- 172.21.15.112:0/665218711 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f031406a050 0x7f031406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:57.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.710+0000 7f03277fe700 1 --2- 172.21.15.112:0/665218711 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f031406a050 0x7f031406c510 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0318009a40 tx=0x7f0318009500 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:57.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.714+0000 7f0324ff9700 1 -- 172.21.15.112:0/665218711 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f031c0540f0 con 0x7f03280b3440 2023-08-27T11:16:57.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:57.909+0000 7f032e3d6700 1 -- 172.21.15.112:0/665218711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f03281f6ef0 con 0x7f03280b3440 2023-08-27T11:16:58.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.314+0000 7f0324ff9700 1 -- 172.21.15.112:0/665218711 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f031c057710 con 0x7f03280b3440 2023-08-27T11:16:58.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.317+0000 7f032e3d6700 1 -- 172.21.15.112:0/665218711 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f031406a050 msgr2=0x7f031406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:58.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.317+0000 7f032e3d6700 1 --2- 172.21.15.112:0/665218711 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f031406a050 0x7f031406c510 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0318009a40 tx=0x7f0318009500 comp rx=0 tx=0).stop 2023-08-27T11:16:58.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.317+0000 7f032e3d6700 1 -- 172.21.15.112:0/665218711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03280b3440 msgr2=0x7f03280cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:58.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.317+0000 7f032e3d6700 1 --2- 172.21.15.112:0/665218711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03280b3440 0x7f03280cd900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f031c00e9b0 tx=0x7f031c00ed70 comp rx=0 tx=0).stop 2023-08-27T11:16:58.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.317+0000 7f032e3d6700 1 -- 172.21.15.112:0/665218711 shutdown_connections 2023-08-27T11:16:58.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.317+0000 7f032e3d6700 1 --2- 172.21.15.112:0/665218711 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f031406a050 0x7f031406c510 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:58.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.317+0000 7f032e3d6700 1 --2- 172.21.15.112:0/665218711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f032815e6e0 0x7f03281f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:58.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.317+0000 7f032e3d6700 1 --2- 172.21.15.112:0/665218711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03280b3440 0x7f03280cd900 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:58.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.317+0000 7f032e3d6700 1 --2- 172.21.15.112:0/665218711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0328146c20 0x7f0328145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:58.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.317+0000 7f032e3d6700 1 -- 172.21.15.112:0/665218711 >> 172.21.15.112:0/665218711 conn(0x7f03280b1e90 msgr2=0x7f032806d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:58.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.317+0000 7f032e3d6700 1 -- 172.21.15.112:0/665218711 shutdown_connections 2023-08-27T11:16:58.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.317+0000 7f032e3d6700 1 -- 172.21.15.112:0/665218711 wait complete. 2023-08-27T11:16:58.322 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'rbd' created 2023-08-27T11:16:58.330 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-08-27T11:16:58.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.450+0000 7f42f13fb700 1 Processor -- start 2023-08-27T11:16:58.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.450+0000 7f42f13fb700 1 -- start start 2023-08-27T11:16:58.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42f13fb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ec1438b0 0x7f42ec0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:58.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42f13fb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f42ec0b3980 0x7f42ec0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:58.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42f13fb700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ec0b1680 con 0x7f42ec0b1230 2023-08-27T11:16:58.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42f13fb700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ec0b3e70 con 0x7f42ec0b3980 2023-08-27T11:16:58.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42f13fb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec0b4150 con 0x7f42ec1438b0 2023-08-27T11:16:58.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42eaffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ec1438b0 0x7f42ec0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:58.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42ea7fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f42ec0b3980 0x7f42ec0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:58.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42eb7fe700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f42ec0b1230 0x7f42ec0b4cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:56354/0 (socket says 172.21.15.112:56354) 2023-08-27T11:16:58.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42ea7fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f42ec0b3980 0x7f42ec0b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:54872/0 (socket says 172.21.15.112:54872) 2023-08-27T11:16:58.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42eb7fe700 1 -- 172.21.15.112:0/2677643190 learned_addr learned my addr 172.21.15.112:0/2677643190 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:16:58.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42eaffd700 1 -- 172.21.15.112:0/2677643190 >> v2:172.21.15.138:3300/0 conn(0x7f42ec0b3980 msgr2=0x7f42ec0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:58.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42eaffd700 1 --2- 172.21.15.112:0/2677643190 >> v2:172.21.15.138:3300/0 conn(0x7f42ec0b3980 0x7f42ec0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:58.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42eaffd700 1 -- 172.21.15.112:0/2677643190 >> v1:172.21.15.138:6789/0 conn(0x7f42ec0b1230 legacy=0x7f42ec0b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:58.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42eaffd700 1 -- 172.21.15.112:0/2677643190 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec0b4430 con 0x7f42ec1438b0 2023-08-27T11:16:58.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.451+0000 7f42ea7fc700 1 --2- 172.21.15.112:0/2677643190 >> v2:172.21.15.138:3300/0 conn(0x7f42ec0b3980 0x7f42ec0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:16:58.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.452+0000 7f42eaffd700 1 --2- 172.21.15.112:0/2677643190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ec1438b0 0x7f42ec0b3440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009840 tx=0x7f42dc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbe5f0ec7ac377ca server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:58.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.452+0000 7f42e9ffb700 1 -- 172.21.15.112:0/2677643190 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc011830 con 0x7f42ec1438b0 2023-08-27T11:16:58.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.452+0000 7f42e9ffb700 1 -- 172.21.15.112:0/2677643190 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42dc011990 con 0x7f42ec1438b0 2023-08-27T11:16:58.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.452+0000 7f42e9ffb700 1 -- 172.21.15.112:0/2677643190 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc011d20 con 0x7f42ec1438b0 2023-08-27T11:16:58.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.452+0000 7f42f13fb700 1 -- 172.21.15.112:0/2677643190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ec1438b0 msgr2=0x7f42ec0b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:58.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.452+0000 7f42f13fb700 1 --2- 172.21.15.112:0/2677643190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ec1438b0 0x7f42ec0b3440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009840 tx=0x7f42dc009b50 comp rx=0 tx=0).stop 2023-08-27T11:16:58.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.452+0000 7f42f13fb700 1 -- 172.21.15.112:0/2677643190 shutdown_connections 2023-08-27T11:16:58.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.452+0000 7f42f13fb700 1 --2- 172.21.15.112:0/2677643190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ec1438b0 0x7f42ec0b3440 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:58.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.452+0000 7f42f13fb700 1 --2- 172.21.15.112:0/2677643190 >> v2:172.21.15.138:3300/0 conn(0x7f42ec0b3980 0x7f42ec0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:58.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.452+0000 7f42f13fb700 1 -- 172.21.15.112:0/2677643190 >> 172.21.15.112:0/2677643190 conn(0x7f42ec0b1be0 msgr2=0x7f42ec0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:58.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.452+0000 7f42f13fb700 1 -- 172.21.15.112:0/2677643190 shutdown_connections 2023-08-27T11:16:58.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.453+0000 7f42f13fb700 1 -- 172.21.15.112:0/2677643190 wait complete. 2023-08-27T11:16:58.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.454+0000 7f42f13fb700 1 Processor -- start 2023-08-27T11:16:58.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.454+0000 7f42f13fb700 1 -- start start 2023-08-27T11:16:58.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.454+0000 7f42f13fb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ec0b1230 0x7f42ec1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:58.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.454+0000 7f42f13fb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42ec0b3980 0x7f42ec1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:58.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.454+0000 7f42f13fb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ec1438b0 0x7f42ec139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:58.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42f13fb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec1392e0 con 0x7f42ec0b3980 2023-08-27T11:16:58.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42f13fb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec139850 con 0x7f42ec1438b0 2023-08-27T11:16:58.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42f13fb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec0cc5d0 con 0x7f42ec0b1230 2023-08-27T11:16:58.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42ea7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42ec0b3980 0x7f42ec1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42ea7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42ec0b3980 0x7f42ec1548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54452/0 (socket says 172.21.15.112:54452) 2023-08-27T11:16:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42ea7fc700 1 -- 172.21.15.112:0/1102746660 learned_addr learned my addr 172.21.15.112:0/1102746660 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:16:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42eaffd700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ec0b1230 0x7f42ec1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42eb7fe700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ec1438b0 0x7f42ec139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42ea7fc700 1 -- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ec0b1230 msgr2=0x7f42ec1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42ea7fc700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ec0b1230 0x7f42ec1565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42ea7fc700 1 -- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ec1438b0 msgr2=0x7f42ec139ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42ea7fc700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ec1438b0 0x7f42ec139ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42ea7fc700 1 -- 172.21.15.112:0/1102746660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d40094a0 con 0x7f42ec0b3980 2023-08-27T11:16:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42eaffd700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ec0b1230 0x7f42ec1565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:16:58.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42ea7fc700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42ec0b3980 0x7f42ec1548c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f42d400ebe0 tx=0x7f42d4009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:58.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42e37fe700 1 -- 172.21.15.112:0/1102746660 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d400c890 con 0x7f42ec0b3980 2023-08-27T11:16:58.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42e37fe700 1 -- 172.21.15.112:0/1102746660 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d400c9f0 con 0x7f42ec0b3980 2023-08-27T11:16:58.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42f13fb700 1 -- 172.21.15.112:0/1102746660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc0094a0 con 0x7f42ec0b3980 2023-08-27T11:16:58.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42e37fe700 1 -- 172.21.15.112:0/1102746660 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d400cda0 con 0x7f42ec0b3980 2023-08-27T11:16:58.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.455+0000 7f42f13fb700 1 -- 172.21.15.112:0/1102746660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ec1394e0 con 0x7f42ec0b3980 2023-08-27T11:16:58.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.456+0000 7f42f13fb700 1 -- 172.21.15.112:0/1102746660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ec0ccaf0 con 0x7f42ec0b3980 2023-08-27T11:16:58.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.457+0000 7f42e37fe700 1 -- 172.21.15.112:0/1102746660 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7f42d4013470 con 0x7f42ec0b3980 2023-08-27T11:16:58.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.457+0000 7f42e37fe700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f42cc06a050 0x7f42cc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:58.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.457+0000 7f42e37fe700 1 -- 172.21.15.112:0/1102746660 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 5188+0+0 (secure 0 0 0) 0x7f42d4087ff0 con 0x7f42ec0b3980 2023-08-27T11:16:58.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.457+0000 7f42eaffd700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f42cc06a050 0x7f42cc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:58.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.458+0000 7f42eaffd700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f42cc06a050 0x7f42cc06c510 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f42dc000c00 tx=0x7f42dc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:58.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.462+0000 7f42e37fe700 1 -- 172.21.15.112:0/1102746660 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f42ec0ccaf0 con 0x7f42ec0b3980 2023-08-27T11:16:58.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:58.652+0000 7f42f13fb700 1 -- 172.21.15.112:0/1102746660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f42ec0ccaf0 con 0x7f42ec0b3980 2023-08-27T11:16:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.317+0000 7f42e37fe700 1 -- 172.21.15.112:0/1102746660 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f42ec0ccaf0 con 0x7f42ec0b3980 2023-08-27T11:16:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.320+0000 7f42f13fb700 1 -- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f42cc06a050 msgr2=0x7f42cc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.320+0000 7f42f13fb700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f42cc06a050 0x7f42cc06c510 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f42dc000c00 tx=0x7f42dc008030 comp rx=0 tx=0).stop 2023-08-27T11:16:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.320+0000 7f42f13fb700 1 -- 172.21.15.112:0/1102746660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42ec0b3980 msgr2=0x7f42ec1548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.320+0000 7f42f13fb700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42ec0b3980 0x7f42ec1548c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f42d400ebe0 tx=0x7f42d4009e60 comp rx=0 tx=0).stop 2023-08-27T11:16:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.320+0000 7f42f13fb700 1 -- 172.21.15.112:0/1102746660 shutdown_connections 2023-08-27T11:16:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.320+0000 7f42f13fb700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f42cc06a050 0x7f42cc06c510 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.320+0000 7f42f13fb700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ec1438b0 0x7f42ec139ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.320+0000 7f42f13fb700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ec0b1230 0x7f42ec1565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.320+0000 7f42f13fb700 1 --2- 172.21.15.112:0/1102746660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42ec0b3980 0x7f42ec1548c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.320+0000 7f42f13fb700 1 -- 172.21.15.112:0/1102746660 >> 172.21.15.112:0/1102746660 conn(0x7f42ec0b1be0 msgr2=0x7f42ec0b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.321+0000 7f42f13fb700 1 -- 172.21.15.112:0/1102746660 shutdown_connections 2023-08-27T11:16:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.321+0000 7f42f13fb700 1 -- 172.21.15.112:0/1102746660 wait complete. 2023-08-27T11:16:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:enabled application 'rbd' on pool 'rbd' 2023-08-27T11:16:59.336 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-08-27T11:16:59.337 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:16:59.337 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:16:59.339 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:16:59.339 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:16:59.340 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:16:59.342 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:16:59.342 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:16:59.342 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:16:59.344 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:16:59.345 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:16:59.345 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:16:59.347 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:16:59.348 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-08-27T11:16:59.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:16:59.428 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:16:59.429 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:16:59.442 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2023-08-27T11:16:59.444 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:16:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.484+0000 7f366317f700 1 Processor -- start 2023-08-27T11:16:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.484+0000 7f366317f700 1 -- start start 2023-08-27T11:16:59.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f366317f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f365c151060 0x7f365c15e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:59.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f366317f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f365c15e990 0x7f365c06adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:59.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f366317f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365c06b710 con 0x7f365c06b2f0 2023-08-27T11:16:59.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f366317f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365c0b02f0 con 0x7f365c151060 2023-08-27T11:16:59.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f366317f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365c0b05d0 con 0x7f365c15e990 2023-08-27T11:16:59.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f3660f1b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f365c151060 0x7f365c15e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:59.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f3660f1b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f365c151060 0x7f365c15e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54492/0 (socket says 172.21.15.112:54492) 2023-08-27T11:16:59.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f3660f1b700 1 -- 172.21.15.112:0/3556340363 learned_addr learned my addr 172.21.15.112:0/3556340363 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:16:59.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f365bfff700 1 --2- 172.21.15.112:0/3556340363 >> v2:172.21.15.138:3300/0 conn(0x7f365c15e990 0x7f365c06adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:59.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f3660f1b700 1 -- 172.21.15.112:0/3556340363 >> v2:172.21.15.138:3300/0 conn(0x7f365c15e990 msgr2=0x7f365c06adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:59.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f3660f1b700 1 --2- 172.21.15.112:0/3556340363 >> v2:172.21.15.138:3300/0 conn(0x7f365c15e990 0x7f365c06adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f3660f1b700 1 -- 172.21.15.112:0/3556340363 >> v1:172.21.15.138:6789/0 conn(0x7f365c06b2f0 legacy=0x7f365c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:16:59.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f3660f1b700 1 -- 172.21.15.112:0/3556340363 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0b08b0 con 0x7f365c151060 2023-08-27T11:16:59.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f365bfff700 1 --2- 172.21.15.112:0/3556340363 >> v2:172.21.15.138:3300/0 conn(0x7f365c15e990 0x7f365c06adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:16:59.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f3660f1b700 1 --2- 172.21.15.112:0/3556340363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c151060 0x7f365c15e450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f365c06ccb0 tx=0x7f3650009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08e50bb1c766d78 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:59.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.485+0000 7f365b7fe700 1 -- 172.21.15.112:0/3556340363 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3650004230 con 0x7f365c151060 2023-08-27T11:16:59.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.486+0000 7f365b7fe700 1 -- 172.21.15.112:0/3556340363 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3650004390 con 0x7f365c151060 2023-08-27T11:16:59.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.486+0000 7f365b7fe700 1 -- 172.21.15.112:0/3556340363 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36500036a0 con 0x7f365c151060 2023-08-27T11:16:59.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.486+0000 7f366317f700 1 -- 172.21.15.112:0/3556340363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c151060 msgr2=0x7f365c15e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:59.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.486+0000 7f366317f700 1 --2- 172.21.15.112:0/3556340363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c151060 0x7f365c15e450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f365c06ccb0 tx=0x7f3650009bd0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.486+0000 7f366317f700 1 -- 172.21.15.112:0/3556340363 shutdown_connections 2023-08-27T11:16:59.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.486+0000 7f366317f700 1 --2- 172.21.15.112:0/3556340363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c151060 0x7f365c15e450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.486+0000 7f366317f700 1 --2- 172.21.15.112:0/3556340363 >> v2:172.21.15.138:3300/0 conn(0x7f365c15e990 0x7f365c06adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.486+0000 7f366317f700 1 -- 172.21.15.112:0/3556340363 >> 172.21.15.112:0/3556340363 conn(0x7f365c0d0000 msgr2=0x7f365c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:59.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.486+0000 7f366317f700 1 -- 172.21.15.112:0/3556340363 shutdown_connections 2023-08-27T11:16:59.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.486+0000 7f366317f700 1 -- 172.21.15.112:0/3556340363 wait complete. 2023-08-27T11:16:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.487+0000 7f366317f700 1 Processor -- start 2023-08-27T11:16:59.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.487+0000 7f366317f700 1 -- start start 2023-08-27T11:16:59.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.488+0000 7f366317f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f365c06b2f0 0x7f365c1f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:59.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.488+0000 7f366317f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c151060 0x7f365c0c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:59.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.488+0000 7f366317f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f365c15e990 0x7f365c14bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:59.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.488+0000 7f366317f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c1f53f0 con 0x7f365c151060 2023-08-27T11:16:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.488+0000 7f366317f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c1f4e30 con 0x7f365c15e990 2023-08-27T11:16:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.488+0000 7f366317f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365c1f5110 con 0x7f365c06b2f0 2023-08-27T11:16:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.488+0000 7f365bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c151060 0x7f365c0c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.488+0000 7f365bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c151060 0x7f365c0c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54498/0 (socket says 172.21.15.112:54498) 2023-08-27T11:16:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.488+0000 7f365bfff700 1 -- 172.21.15.112:0/662725878 learned_addr learned my addr 172.21.15.112:0/662725878 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:16:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.488+0000 7f366171c700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f365c15e990 0x7f365c14bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.488+0000 7f3660f1b700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f365c06b2f0 0x7f365c1f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.489+0000 7f365bfff700 1 -- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f365c06b2f0 msgr2=0x7f365c1f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.489+0000 7f365bfff700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f365c06b2f0 0x7f365c1f1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.489+0000 7f365bfff700 1 -- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f365c15e990 msgr2=0x7f365c14bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:59.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.489+0000 7f365bfff700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f365c15e990 0x7f365c14bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.489+0000 7f365bfff700 1 -- 172.21.15.112:0/662725878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3650009550 con 0x7f365c151060 2023-08-27T11:16:59.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.489+0000 7f365bfff700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c151060 0x7f365c0c9ea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f364800c750 tx=0x7f364800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:59.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.489+0000 7f36597fa700 1 -- 172.21.15.112:0/662725878 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648011800 con 0x7f365c151060 2023-08-27T11:16:59.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.489+0000 7f36597fa700 1 -- 172.21.15.112:0/662725878 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3648011960 con 0x7f365c151060 2023-08-27T11:16:59.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.489+0000 7f366317f700 1 -- 172.21.15.112:0/662725878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365c14b750 con 0x7f365c151060 2023-08-27T11:16:59.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.489+0000 7f36597fa700 1 -- 172.21.15.112:0/662725878 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648011cd0 con 0x7f365c151060 2023-08-27T11:16:59.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.489+0000 7f366317f700 1 -- 172.21.15.112:0/662725878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365c0ca3e0 con 0x7f365c151060 2023-08-27T11:16:59.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.490+0000 7f366317f700 1 -- 172.21.15.112:0/662725878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365c14bc70 con 0x7f365c151060 2023-08-27T11:16:59.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.490+0000 7f36597fa700 1 -- 172.21.15.112:0/662725878 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7f3648021350 con 0x7f365c151060 2023-08-27T11:16:59.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.491+0000 7f36597fa700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f364006a050 0x7f364006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:59.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.491+0000 7f36597fa700 1 -- 172.21.15.112:0/662725878 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3648087fc0 con 0x7f365c151060 2023-08-27T11:16:59.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.491+0000 7f3660f1b700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f364006a050 0x7f364006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:59.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.491+0000 7f3660f1b700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f364006a050 0x7f364006c510 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3650009130 tx=0x7f3650008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:59.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.495+0000 7f36597fa700 1 -- 172.21.15.112:0/662725878 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f36480ba990 con 0x7f365c151060 2023-08-27T11:16:59.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.702+0000 7f366317f700 1 -- 172.21.15.112:0/662725878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f365c067a20 con 0x7f365c151060 2023-08-27T11:16:59.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.703+0000 7f36597fa700 1 -- 172.21.15.112:0/662725878 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+14629 (secure 0 0 0) 0x7f365c14bc70 con 0x7f365c151060 2023-08-27T11:16:59.704 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:16:59.705 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":15,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:16:59.311384+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:16:59.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.705+0000 7f366317f700 1 -- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f364006a050 msgr2=0x7f364006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:59.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.705+0000 7f366317f700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f364006a050 0x7f364006c510 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3650009130 tx=0x7f3650008030 comp rx=0 tx=0).stop 2023-08-27T11:16:59.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.705+0000 7f366317f700 1 -- 172.21.15.112:0/662725878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c151060 msgr2=0x7f365c0c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:59.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.705+0000 7f366317f700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c151060 0x7f365c0c9ea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f364800c750 tx=0x7f364800cb10 comp rx=0 tx=0).stop 2023-08-27T11:16:59.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.705+0000 7f366317f700 1 -- 172.21.15.112:0/662725878 shutdown_connections 2023-08-27T11:16:59.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.705+0000 7f366317f700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f364006a050 0x7f364006c510 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.705+0000 7f366317f700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f365c15e990 0x7f365c14bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.705+0000 7f366317f700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f365c06b2f0 0x7f365c1f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.705+0000 7f366317f700 1 --2- 172.21.15.112:0/662725878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c151060 0x7f365c0c9ea0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.705+0000 7f366317f700 1 -- 172.21.15.112:0/662725878 >> 172.21.15.112:0/662725878 conn(0x7f365c0d0000 msgr2=0x7f365c0b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:59.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.705+0000 7f366317f700 1 -- 172.21.15.112:0/662725878 shutdown_connections 2023-08-27T11:16:59.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.705+0000 7f366317f700 1 -- 172.21.15.112:0/662725878 wait complete. 2023-08-27T11:16:59.719 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:16:59.720 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:16:59.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.845+0000 7f035ad54700 1 Processor -- start 2023-08-27T11:16:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.845+0000 7f035ad54700 1 -- start start 2023-08-27T11:16:59.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.846+0000 7f035ad54700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0354146c20 0x7f0354160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:59.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.846+0000 7f035ad54700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03540b5070 con 0x7f03540b3440 2023-08-27T11:16:59.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.846+0000 7f035ad54700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0354147000 con 0x7f035406f790 2023-08-27T11:16:59.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.846+0000 7f035ad54700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03541614a0 con 0x7f0354146c20 2023-08-27T11:16:59.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.846+0000 7f0353fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f03540b3440 0x7f035415e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48890/0 (socket says 172.21.15.112:48890) 2023-08-27T11:16:59.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.846+0000 7f0353fff700 1 -- 172.21.15.112:0/1668823624 learned_addr learned my addr 172.21.15.112:0/1668823624 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:16:59.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.846+0000 7f0358af0700 1 --2- 172.21.15.112:0/1668823624 >> v2:172.21.15.138:3300/0 conn(0x7f0354146c20 0x7f0354160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:59.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.846+0000 7f03537fe700 1 -- 172.21.15.112:0/1668823624 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723055004 0 0) 0x7f0340001110 con 0x7f03540b3440 2023-08-27T11:16:59.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.846+0000 7f03537fe700 1 -- 172.21.15.112:0/1668823624 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03440035b0 con 0x7f03540b3440 2023-08-27T11:16:59.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.846+0000 7f0358af0700 1 -- 172.21.15.112:0/1668823624 >> v1:172.21.15.138:6789/0 conn(0x7f035406f790 legacy=0x7f03540b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:59.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.846+0000 7f0358af0700 1 -- 172.21.15.112:0/1668823624 >> v1:172.21.15.112:6789/0 conn(0x7f03540b3440 legacy=0x7f035415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:59.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.846+0000 7f0358af0700 1 -- 172.21.15.112:0/1668823624 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0354161740 con 0x7f0354146c20 2023-08-27T11:16:59.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.847+0000 7f0358af0700 1 --2- 172.21.15.112:0/1668823624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0354146c20 0x7f0354160f60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0348009840 tx=0x7f0348009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e89acc427369a68 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:59.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.847+0000 7f03537fe700 1 -- 172.21.15.112:0/1668823624 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348011830 con 0x7f0354146c20 2023-08-27T11:16:59.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.847+0000 7f03537fe700 1 -- 172.21.15.112:0/1668823624 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0348011990 con 0x7f0354146c20 2023-08-27T11:16:59.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.847+0000 7f03537fe700 1 -- 172.21.15.112:0/1668823624 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348011d20 con 0x7f0354146c20 2023-08-27T11:16:59.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.847+0000 7f035ad54700 1 -- 172.21.15.112:0/1668823624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0354146c20 msgr2=0x7f0354160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.847+0000 7f035ad54700 1 --2- 172.21.15.112:0/1668823624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0354146c20 0x7f0354160f60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0348009840 tx=0x7f0348009b50 comp rx=0 tx=0).stop 2023-08-27T11:16:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.847+0000 7f035ad54700 1 -- 172.21.15.112:0/1668823624 shutdown_connections 2023-08-27T11:16:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.847+0000 7f035ad54700 1 --2- 172.21.15.112:0/1668823624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0354146c20 0x7f0354160f60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.847+0000 7f035ad54700 1 -- 172.21.15.112:0/1668823624 >> 172.21.15.112:0/1668823624 conn(0x7f03540b1e90 msgr2=0x7f03540b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:16:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.847+0000 7f035ad54700 1 -- 172.21.15.112:0/1668823624 shutdown_connections 2023-08-27T11:16:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.847+0000 7f035ad54700 1 -- 172.21.15.112:0/1668823624 wait complete. 2023-08-27T11:16:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.848+0000 7f035ad54700 1 Processor -- start 2023-08-27T11:16:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.848+0000 7f035ad54700 1 -- start start 2023-08-27T11:16:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.849+0000 7f035ad54700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f035406f790 0x7f03541f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.849+0000 7f035ad54700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03540b3440 0x7f03541f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.849+0000 7f035ad54700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0354146c20 0x7f03541409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.849+0000 7f035ad54700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03541f5300 con 0x7f03540b3440 2023-08-27T11:16:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.849+0000 7f035ad54700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0354140f00 con 0x7f035406f790 2023-08-27T11:16:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.849+0000 7f035ad54700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03541f46f0 con 0x7f0354146c20 2023-08-27T11:16:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.849+0000 7f0353fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03540b3440 0x7f03541f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.849+0000 7f0353fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03540b3440 0x7f03541f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54514/0 (socket says 172.21.15.112:54514) 2023-08-27T11:16:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.849+0000 7f0353fff700 1 -- 172.21.15.112:0/3410187598 learned_addr learned my addr 172.21.15.112:0/3410187598 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:16:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.849+0000 7f0358af0700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f035406f790 0x7f03541f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.849+0000 7f03592f1700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0354146c20 0x7f03541409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.850+0000 7f0353fff700 1 -- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0354146c20 msgr2=0x7f03541409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.850+0000 7f0353fff700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0354146c20 0x7f03541409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.850+0000 7f0353fff700 1 -- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f035406f790 msgr2=0x7f03541f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:16:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.850+0000 7f0353fff700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f035406f790 0x7f03541f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:16:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.850+0000 7f0353fff700 1 -- 172.21.15.112:0/3410187598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03480094a0 con 0x7f03540b3440 2023-08-27T11:16:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.850+0000 7f0358af0700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f035406f790 0x7f03541f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:16:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.850+0000 7f0353fff700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03540b3440 0x7f03541f3ff0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f034000c520 tx=0x7f034000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:16:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.850+0000 7f03517fa700 1 -- 172.21.15.112:0/3410187598 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340005d70 con 0x7f03540b3440 2023-08-27T11:16:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.850+0000 7f03517fa700 1 -- 172.21.15.112:0/3410187598 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03400070d0 con 0x7f03540b3440 2023-08-27T11:16:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.850+0000 7f035ad54700 1 -- 172.21.15.112:0/3410187598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03541f49d0 con 0x7f03540b3440 2023-08-27T11:16:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.850+0000 7f03517fa700 1 -- 172.21.15.112:0/3410187598 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340007460 con 0x7f03540b3440 2023-08-27T11:16:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.850+0000 7f035ad54700 1 -- 172.21.15.112:0/3410187598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03541f4f90 con 0x7f03540b3440 2023-08-27T11:16:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.851+0000 7f035ad54700 1 -- 172.21.15.112:0/3410187598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0354067e90 con 0x7f03540b3440 2023-08-27T11:16:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.851+0000 7f03517fa700 1 -- 172.21.15.112:0/3410187598 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7f03400075c0 con 0x7f03540b3440 2023-08-27T11:16:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.852+0000 7f03517fa700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f033806a050 0x7f033806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:16:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.852+0000 7f03517fa700 1 -- 172.21.15.112:0/3410187598 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0340087380 con 0x7f03540b3440 2023-08-27T11:16:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.856+0000 7f03517fa700 1 -- 172.21.15.112:0/3410187598 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0340052eb0 con 0x7f03540b3440 2023-08-27T11:16:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.866+0000 7f0358af0700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f033806a050 0x7f033806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:16:59.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:16:59.866+0000 7f0358af0700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f033806a050 0x7f033806c510 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f03480090b0 tx=0x7f0348008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.045+0000 7f035ad54700 1 -- 172.21.15.112:0/3410187598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f03540b5070 con 0x7f03540b3440 2023-08-27T11:17:00.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.045+0000 7f03517fa700 1 -- 172.21.15.112:0/3410187598 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7f03540b5070 con 0x7f03540b3440 2023-08-27T11:17:00.048 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.048+0000 7f035ad54700 1 -- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f033806a050 msgr2=0x7f033806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.048+0000 7f035ad54700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f033806a050 0x7f033806c510 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f03480090b0 tx=0x7f0348008030 comp rx=0 tx=0).stop 2023-08-27T11:17:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.048+0000 7f035ad54700 1 -- 172.21.15.112:0/3410187598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03540b3440 msgr2=0x7f03541f3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.048+0000 7f035ad54700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03540b3440 0x7f03541f3ff0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f034000c520 tx=0x7f034000c830 comp rx=0 tx=0).stop 2023-08-27T11:17:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.048+0000 7f035ad54700 1 -- 172.21.15.112:0/3410187598 shutdown_connections 2023-08-27T11:17:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.048+0000 7f035ad54700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f033806a050 0x7f033806c510 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.048+0000 7f035ad54700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0354146c20 0x7f03541409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.048+0000 7f035ad54700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f035406f790 0x7f03541f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.048+0000 7f035ad54700 1 --2- 172.21.15.112:0/3410187598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03540b3440 0x7f03541f3ff0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.048+0000 7f035ad54700 1 -- 172.21.15.112:0/3410187598 >> 172.21.15.112:0/3410187598 conn(0x7f03540b1e90 msgr2=0x7f03540b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:00.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.048+0000 7f035ad54700 1 -- 172.21.15.112:0/3410187598 shutdown_connections 2023-08-27T11:17:00.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.048+0000 7f035ad54700 1 -- 172.21.15.112:0/3410187598 wait complete. 2023-08-27T11:17:00.061 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:00.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.188+0000 7f186ee2c700 1 Processor -- start 2023-08-27T11:17:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.188+0000 7f186ee2c700 1 -- start start 2023-08-27T11:17:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f186ee2c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1868146c20 0x7f1868160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f186ee2c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18680b3440 0x7f186815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f186ee2c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18680b5070 con 0x7f186815e6e0 2023-08-27T11:17:00.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f186ee2c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1868147000 con 0x7f1868146c20 2023-08-27T11:17:00.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f186ee2c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18681614a0 con 0x7f18680b3440 2023-08-27T11:17:00.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f186cbc8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1868146c20 0x7f1868160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f186d3c9700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f186815e6e0 0x7f186806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48906/0 (socket says 172.21.15.112:48906) 2023-08-27T11:17:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f186d3c9700 1 -- 172.21.15.112:0/4278826346 learned_addr learned my addr 172.21.15.112:0/4278826346 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f1867fff700 1 --2- 172.21.15.112:0/4278826346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18680b3440 0x7f186815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f186cbc8700 1 -- 172.21.15.112:0/4278826346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18680b3440 msgr2=0x7f186815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f186cbc8700 1 --2- 172.21.15.112:0/4278826346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18680b3440 0x7f186815e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f186cbc8700 1 -- 172.21.15.112:0/4278826346 >> v1:172.21.15.112:6789/0 conn(0x7f186815e6e0 legacy=0x7f186806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f186cbc8700 1 -- 172.21.15.112:0/4278826346 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1868161740 con 0x7f1868146c20 2023-08-27T11:17:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f18677fe700 1 -- 172.21.15.112:0/4278826346 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527754849 0 0) 0x7f1860001490 con 0x7f186815e6e0 2023-08-27T11:17:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f186cbc8700 1 --2- 172.21.15.112:0/4278826346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868146c20 0x7f1868160f60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f185c009840 tx=0x7f185c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196ea75bf9a0d055 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f18677fe700 1 -- 172.21.15.112:0/4278826346 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c011830 con 0x7f1868146c20 2023-08-27T11:17:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f18677fe700 1 -- 172.21.15.112:0/4278826346 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c011990 con 0x7f1868146c20 2023-08-27T11:17:00.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.189+0000 7f18677fe700 1 -- 172.21.15.112:0/4278826346 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c011d20 con 0x7f1868146c20 2023-08-27T11:17:00.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.190+0000 7f186ee2c700 1 -- 172.21.15.112:0/4278826346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868146c20 msgr2=0x7f1868160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.190+0000 7f186ee2c700 1 --2- 172.21.15.112:0/4278826346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868146c20 0x7f1868160f60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f185c009840 tx=0x7f185c009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:00.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.190+0000 7f186ee2c700 1 -- 172.21.15.112:0/4278826346 shutdown_connections 2023-08-27T11:17:00.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.190+0000 7f186ee2c700 1 --2- 172.21.15.112:0/4278826346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868146c20 0x7f1868160f60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.190+0000 7f186ee2c700 1 --2- 172.21.15.112:0/4278826346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18680b3440 0x7f186815e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.190+0000 7f186ee2c700 1 -- 172.21.15.112:0/4278826346 >> 172.21.15.112:0/4278826346 conn(0x7f18680b1e90 msgr2=0x7f18680b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:00.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.190+0000 7f186ee2c700 1 -- 172.21.15.112:0/4278826346 shutdown_connections 2023-08-27T11:17:00.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.190+0000 7f186ee2c700 1 -- 172.21.15.112:0/4278826346 wait complete. 2023-08-27T11:17:00.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.191+0000 7f186ee2c700 1 Processor -- start 2023-08-27T11:17:00.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.191+0000 7f186ee2c700 1 -- start start 2023-08-27T11:17:00.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f186ee2c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18680b3440 0x7f186806e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f186ee2c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868146c20 0x7f186814f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f186ee2c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186815e6e0 0x7f1868070750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f186ee2c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186813d320 con 0x7f1868146c20 2023-08-27T11:17:00.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f186ee2c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186814fc60 con 0x7f18680b3440 2023-08-27T11:17:00.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f186ee2c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18681f3850 con 0x7f186815e6e0 2023-08-27T11:17:00.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f1867fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868146c20 0x7f186814f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f1867fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868146c20 0x7f186814f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54528/0 (socket says 172.21.15.112:54528) 2023-08-27T11:17:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f1867fff700 1 -- 172.21.15.112:0/1591466098 learned_addr learned my addr 172.21.15.112:0/1591466098 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f186d3c9700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186815e6e0 0x7f1868070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f186cbc8700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18680b3440 0x7f186806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f1867fff700 1 -- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186815e6e0 msgr2=0x7f1868070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f1867fff700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186815e6e0 0x7f1868070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f1867fff700 1 -- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18680b3440 msgr2=0x7f186806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f1867fff700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18680b3440 0x7f186806e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.192+0000 7f1867fff700 1 -- 172.21.15.112:0/1591466098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c0094a0 con 0x7f1868146c20 2023-08-27T11:17:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.193+0000 7f1867fff700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868146c20 0x7f186814f720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f185400e9b0 tx=0x7f185400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.193+0000 7f18657fa700 1 -- 172.21.15.112:0/1591466098 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854009710 con 0x7f1868146c20 2023-08-27T11:17:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.193+0000 7f18657fa700 1 -- 172.21.15.112:0/1591466098 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185400c850 con 0x7f1868146c20 2023-08-27T11:17:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.193+0000 7f186ee2c700 1 -- 172.21.15.112:0/1591466098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1868150110 con 0x7f1868146c20 2023-08-27T11:17:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.193+0000 7f18657fa700 1 -- 172.21.15.112:0/1591466098 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185400cbc0 con 0x7f1868146c20 2023-08-27T11:17:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.193+0000 7f186ee2c700 1 -- 172.21.15.112:0/1591466098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18681f2cd0 con 0x7f1868146c20 2023-08-27T11:17:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.194+0000 7f186ee2c700 1 -- 172.21.15.112:0/1591466098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186813d5e0 con 0x7f1868146c20 2023-08-27T11:17:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.194+0000 7f18657fa700 1 -- 172.21.15.112:0/1591466098 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7f185400cd20 con 0x7f1868146c20 2023-08-27T11:17:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.195+0000 7f18657fa700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f184c06a050 0x7f184c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.195+0000 7f18657fa700 1 -- 172.21.15.112:0/1591466098 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1854088050 con 0x7f1868146c20 2023-08-27T11:17:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.195+0000 7f186cbc8700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f184c06a050 0x7f184c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.195+0000 7f186cbc8700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f184c06a050 0x7f184c06c510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f185c0090b0 tx=0x7f185c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.199+0000 7f18657fa700 1 -- 172.21.15.112:0/1591466098 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1854053c00 con 0x7f1868146c20 2023-08-27T11:17:00.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.392+0000 7f186ee2c700 1 -- 172.21.15.112:0/1591466098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1868161cd0 con 0x7f1868146c20 2023-08-27T11:17:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.392+0000 7f18657fa700 1 -- 172.21.15.112:0/1591466098 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1854057220 con 0x7f1868146c20 2023-08-27T11:17:00.394 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:00.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.394+0000 7f186ee2c700 1 -- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f184c06a050 msgr2=0x7f184c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.394+0000 7f186ee2c700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f184c06a050 0x7f184c06c510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f185c0090b0 tx=0x7f185c008030 comp rx=0 tx=0).stop 2023-08-27T11:17:00.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.394+0000 7f186ee2c700 1 -- 172.21.15.112:0/1591466098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868146c20 msgr2=0x7f186814f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.394+0000 7f186ee2c700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868146c20 0x7f186814f720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f185400e9b0 tx=0x7f185400ecc0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.395+0000 7f186ee2c700 1 -- 172.21.15.112:0/1591466098 shutdown_connections 2023-08-27T11:17:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.395+0000 7f186ee2c700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f184c06a050 0x7f184c06c510 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.395+0000 7f186ee2c700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186815e6e0 0x7f1868070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.395+0000 7f186ee2c700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18680b3440 0x7f186806e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.395+0000 7f186ee2c700 1 --2- 172.21.15.112:0/1591466098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868146c20 0x7f186814f720 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.395+0000 7f186ee2c700 1 -- 172.21.15.112:0/1591466098 >> 172.21.15.112:0/1591466098 conn(0x7f18680b1e90 msgr2=0x7f18680713f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.395+0000 7f186ee2c700 1 -- 172.21.15.112:0/1591466098 shutdown_connections 2023-08-27T11:17:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.395+0000 7f186ee2c700 1 -- 172.21.15.112:0/1591466098 wait complete. 2023-08-27T11:17:00.408 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:17:00.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.528+0000 7fd5a7113700 1 Processor -- start 2023-08-27T11:17:00.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.528+0000 7fd5a7113700 1 -- start start 2023-08-27T11:17:00.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a7113700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd5a0143190 0x7fd5a0066010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a7113700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5a00ceca0 0x7fd5a00cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a7113700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5a006e150 0x7fd5a006e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a7113700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a006eb20 con 0x7fd5a006e150 2023-08-27T11:17:00.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a7113700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a00cc200 con 0x7fd5a0143190 2023-08-27T11:17:00.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a7113700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a00cc4e0 con 0x7fd5a00ceca0 2023-08-27T11:17:00.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a56b0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5a006e150 0x7fd5a006e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a56b0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5a006e150 0x7fd5a006e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54538/0 (socket says 172.21.15.112:54538) 2023-08-27T11:17:00.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a56b0700 1 -- 172.21.15.112:0/3304370836 learned_addr learned my addr 172.21.15.112:0/3304370836 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:00.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a4eaf700 1 --2- 172.21.15.112:0/3304370836 >> v2:172.21.15.138:3300/0 conn(0x7fd5a0143190 0x7fd5a0066010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd59ffff700 1 --2- 172.21.15.112:0/3304370836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5a00ceca0 0x7fd5a00cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a56b0700 1 -- 172.21.15.112:0/3304370836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5a00ceca0 msgr2=0x7fd5a00cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:00.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a56b0700 1 --2- 172.21.15.112:0/3304370836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5a00ceca0 0x7fd5a00cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a56b0700 1 -- 172.21.15.112:0/3304370836 >> v2:172.21.15.138:3300/0 conn(0x7fd5a0143190 msgr2=0x7fd5a0066010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:00.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a56b0700 1 --2- 172.21.15.112:0/3304370836 >> v2:172.21.15.138:3300/0 conn(0x7fd5a0143190 0x7fd5a0066010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a56b0700 1 -- 172.21.15.112:0/3304370836 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a00cc7c0 con 0x7fd5a006e150 2023-08-27T11:17:00.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.529+0000 7fd5a56b0700 1 --2- 172.21.15.112:0/3304370836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a006e150 0x7fd5a006e5e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0072060 tx=0x7fd59800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514a1b7ceb7e431d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:00.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.530+0000 7fd59f7fe700 1 -- 172.21.15.112:0/3304370836 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598010070 con 0x7fd5a006e150 2023-08-27T11:17:00.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.530+0000 7fd59f7fe700 1 -- 172.21.15.112:0/3304370836 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59800dce0 con 0x7fd5a006e150 2023-08-27T11:17:00.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.530+0000 7fd59f7fe700 1 -- 172.21.15.112:0/3304370836 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59800b310 con 0x7fd5a006e150 2023-08-27T11:17:00.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.530+0000 7fd5a7113700 1 -- 172.21.15.112:0/3304370836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a006e150 msgr2=0x7fd5a006e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.530+0000 7fd5a7113700 1 --2- 172.21.15.112:0/3304370836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a006e150 0x7fd5a006e5e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0072060 tx=0x7fd59800d430 comp rx=0 tx=0).stop 2023-08-27T11:17:00.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.530+0000 7fd5a7113700 1 -- 172.21.15.112:0/3304370836 shutdown_connections 2023-08-27T11:17:00.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.530+0000 7fd5a7113700 1 --2- 172.21.15.112:0/3304370836 >> v2:172.21.15.138:3300/0 conn(0x7fd5a0143190 0x7fd5a0066010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.530+0000 7fd5a7113700 1 --2- 172.21.15.112:0/3304370836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a006e150 0x7fd5a006e5e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.530+0000 7fd5a7113700 1 --2- 172.21.15.112:0/3304370836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5a00ceca0 0x7fd5a00cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.530+0000 7fd5a7113700 1 -- 172.21.15.112:0/3304370836 >> 172.21.15.112:0/3304370836 conn(0x7fd5a006cd30 msgr2=0x7fd5a00717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:00.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.530+0000 7fd5a7113700 1 -- 172.21.15.112:0/3304370836 shutdown_connections 2023-08-27T11:17:00.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.530+0000 7fd5a7113700 1 -- 172.21.15.112:0/3304370836 wait complete. 2023-08-27T11:17:00.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.531+0000 7fd5a7113700 1 Processor -- start 2023-08-27T11:17:00.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.531+0000 7fd5a7113700 1 -- start start 2023-08-27T11:17:00.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.532+0000 7fd5a7113700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5a006e150 0x7fd5a012fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.532+0000 7fd5a7113700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a00ceca0 0x7fd5a013c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.532+0000 7fd5a7113700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a0143190 0x7fd5a01f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.532+0000 7fd5a7113700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a013cb00 con 0x7fd5a0143190 2023-08-27T11:17:00.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.532+0000 7fd5a7113700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a01300c0 con 0x7fd5a00ceca0 2023-08-27T11:17:00.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.532+0000 7fd5a7113700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a01303a0 con 0x7fd5a006e150 2023-08-27T11:17:00.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.532+0000 7fd5a56b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a0143190 0x7fd5a01f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.532+0000 7fd5a56b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a0143190 0x7fd5a01f25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54544/0 (socket says 172.21.15.112:54544) 2023-08-27T11:17:00.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.532+0000 7fd5a56b0700 1 -- 172.21.15.112:0/2406013699 learned_addr learned my addr 172.21.15.112:0/2406013699 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:00.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd59ffff700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a00ceca0 0x7fd5a013c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd5a4eaf700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5a006e150 0x7fd5a012fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd5a56b0700 1 -- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5a006e150 msgr2=0x7fd5a012fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd5a56b0700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5a006e150 0x7fd5a012fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd5a56b0700 1 -- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a00ceca0 msgr2=0x7fd5a013c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd5a56b0700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a00ceca0 0x7fd5a013c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd5a56b0700 1 -- 172.21.15.112:0/2406013699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59800bd40 con 0x7fd5a0143190 2023-08-27T11:17:00.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd59ffff700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a00ceca0 0x7fd5a013c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:00.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd5a56b0700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a0143190 0x7fd5a01f25b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5a006eeb0 tx=0x7fd598004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:00.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd59d7fa700 1 -- 172.21.15.112:0/2406013699 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598009a10 con 0x7fd5a0143190 2023-08-27T11:17:00.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd59d7fa700 1 -- 172.21.15.112:0/2406013699 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd598009b70 con 0x7fd5a0143190 2023-08-27T11:17:00.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd5a7113700 1 -- 172.21.15.112:0/2406013699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a0130680 con 0x7fd5a0143190 2023-08-27T11:17:00.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd59d7fa700 1 -- 172.21.15.112:0/2406013699 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59800b4c0 con 0x7fd5a0143190 2023-08-27T11:17:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.533+0000 7fd5a7113700 1 -- 172.21.15.112:0/2406013699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a0130c40 con 0x7fd5a0143190 2023-08-27T11:17:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.534+0000 7fd5a7113700 1 -- 172.21.15.112:0/2406013699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a0067a20 con 0x7fd5a0143190 2023-08-27T11:17:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.534+0000 7fd59d7fa700 1 -- 172.21.15.112:0/2406013699 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7fd598004040 con 0x7fd5a0143190 2023-08-27T11:17:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.535+0000 7fd59d7fa700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd58406a050 0x7fd58406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.535+0000 7fd59d7fa700 1 -- 172.21.15.112:0/2406013699 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd598089a40 con 0x7fd5a0143190 2023-08-27T11:17:00.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.535+0000 7fd5a4eaf700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd58406a050 0x7fd58406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.535+0000 7fd5a4eaf700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd58406a050 0x7fd58406c510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd594009ad0 tx=0x7fd594008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:00.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.539+0000 7fd59d7fa700 1 -- 172.21.15.112:0/2406013699 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd598055710 con 0x7fd5a0143190 2023-08-27T11:17:00.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.769+0000 7fd5a7113700 1 -- 172.21.15.112:0/2406013699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5a00cd950 con 0x7fd5a0143190 2023-08-27T11:17:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.769+0000 7fd59d7fa700 1 -- 172.21.15.112:0/2406013699 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3706 (secure 0 0 0) 0x7fd598058d30 con 0x7fd5a0143190 2023-08-27T11:17:00.771 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:00.771 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3289488908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3289488908},{"type":"v1","addr":"172.21.15.112:6835","nonce":3289488908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3991653721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3991653721},{"type":"v1","addr":"172.21.15.112:6834","nonce":3991653721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1923303457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1923303457},{"type":"v1","addr":"172.21.15.138:6836","nonce":1923303457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1827312637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1827312637},{"type":"v1","addr":"172.21.15.138:6837","nonce":1827312637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-08-27T11:17:00.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.771+0000 7fd5a7113700 1 -- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd58406a050 msgr2=0x7fd58406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.771+0000 7fd5a7113700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd58406a050 0x7fd58406c510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd594009ad0 tx=0x7fd594008040 comp rx=0 tx=0).stop 2023-08-27T11:17:00.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.771+0000 7fd5a7113700 1 -- 172.21.15.112:0/2406013699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a0143190 msgr2=0x7fd5a01f25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.771+0000 7fd5a7113700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a0143190 0x7fd5a01f25b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5a006eeb0 tx=0x7fd598004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.771+0000 7fd5a7113700 1 -- 172.21.15.112:0/2406013699 shutdown_connections 2023-08-27T11:17:00.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.771+0000 7fd5a7113700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd58406a050 0x7fd58406c510 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.771+0000 7fd5a7113700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a0143190 0x7fd5a01f25b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.771+0000 7fd5a7113700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5a006e150 0x7fd5a012fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.771+0000 7fd5a7113700 1 --2- 172.21.15.112:0/2406013699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a00ceca0 0x7fd5a013c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.771+0000 7fd5a7113700 1 -- 172.21.15.112:0/2406013699 >> 172.21.15.112:0/2406013699 conn(0x7fd5a006cd30 msgr2=0x7fd5a0141330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:00.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.771+0000 7fd5a7113700 1 -- 172.21.15.112:0/2406013699 shutdown_connections 2023-08-27T11:17:00.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.771+0000 7fd5a7113700 1 -- 172.21.15.112:0/2406013699 wait complete. 2023-08-27T11:17:00.776 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 3 2023-08-27T11:17:00.785 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:00.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.905+0000 7f9e5ef96700 1 Processor -- start 2023-08-27T11:17:00.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.905+0000 7f9e5ef96700 1 -- start start 2023-08-27T11:17:00.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5ef96700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e58144570 0x7f9e5815ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5ef96700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9e580b3e50 0x7f9e5815f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5ef96700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e580b5ae0 con 0x7f9e5815f8a0 2023-08-27T11:17:00.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5ef96700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e5815fcf0 con 0x7f9e58144570 2023-08-27T11:17:00.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5ef96700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e58144950 con 0x7f9e580b3e50 2023-08-27T11:17:00.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5cd32700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e58144570 0x7f9e5815ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5d533700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9e5815f8a0 0x7f9e580ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48932/0 (socket says 172.21.15.112:48932) 2023-08-27T11:17:00.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5cd32700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e58144570 0x7f9e5815ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54548/0 (socket says 172.21.15.112:54548) 2023-08-27T11:17:00.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5d533700 1 -- 172.21.15.112:0/416843513 learned_addr learned my addr 172.21.15.112:0/416843513 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:00.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e57fff700 1 --2- 172.21.15.112:0/416843513 >> v2:172.21.15.138:3300/0 conn(0x7f9e580b3e50 0x7f9e5815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5cd32700 1 -- 172.21.15.112:0/416843513 >> v2:172.21.15.138:3300/0 conn(0x7f9e580b3e50 msgr2=0x7f9e5815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:00.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5cd32700 1 --2- 172.21.15.112:0/416843513 >> v2:172.21.15.138:3300/0 conn(0x7f9e580b3e50 0x7f9e5815f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5cd32700 1 -- 172.21.15.112:0/416843513 >> v1:172.21.15.112:6789/0 conn(0x7f9e5815f8a0 legacy=0x7f9e580ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5cd32700 1 -- 172.21.15.112:0/416843513 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e580b45f0 con 0x7f9e58144570 2023-08-27T11:17:00.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e577fe700 1 -- 172.21.15.112:0/416843513 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727467388 0 0) 0x7f9e50001490 con 0x7f9e5815f8a0 2023-08-27T11:17:00.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e5cd32700 1 --2- 172.21.15.112:0/416843513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e58144570 0x7f9e5815ee20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9e580b33f0 tx=0x7f9e4c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ba50e76b43ea6f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:00.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e577fe700 1 -- 172.21.15.112:0/416843513 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c011830 con 0x7f9e58144570 2023-08-27T11:17:00.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e577fe700 1 -- 172.21.15.112:0/416843513 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c011990 con 0x7f9e58144570 2023-08-27T11:17:00.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.906+0000 7f9e577fe700 1 -- 172.21.15.112:0/416843513 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c011d20 con 0x7f9e58144570 2023-08-27T11:17:00.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.907+0000 7f9e5ef96700 1 -- 172.21.15.112:0/416843513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e58144570 msgr2=0x7f9e5815ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.907+0000 7f9e5ef96700 1 --2- 172.21.15.112:0/416843513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e58144570 0x7f9e5815ee20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9e580b33f0 tx=0x7f9e4c009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:00.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.907+0000 7f9e5ef96700 1 -- 172.21.15.112:0/416843513 shutdown_connections 2023-08-27T11:17:00.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.907+0000 7f9e5ef96700 1 --2- 172.21.15.112:0/416843513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e58144570 0x7f9e5815ee20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.907+0000 7f9e5ef96700 1 --2- 172.21.15.112:0/416843513 >> v2:172.21.15.138:3300/0 conn(0x7f9e580b3e50 0x7f9e5815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.907+0000 7f9e5ef96700 1 -- 172.21.15.112:0/416843513 >> 172.21.15.112:0/416843513 conn(0x7f9e5806f970 msgr2=0x7f9e5806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:00.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.907+0000 7f9e5ef96700 1 -- 172.21.15.112:0/416843513 shutdown_connections 2023-08-27T11:17:00.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.907+0000 7f9e5ef96700 1 -- 172.21.15.112:0/416843513 wait complete. 2023-08-27T11:17:00.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.908+0000 7f9e5ef96700 1 Processor -- start 2023-08-27T11:17:00.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.908+0000 7f9e5ef96700 1 -- start start 2023-08-27T11:17:00.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5ef96700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e580b3e50 0x7f9e581f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5ef96700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e58144570 0x7f9e580c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5ef96700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e5815f8a0 0x7f9e580a86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5ef96700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580c8cb0 con 0x7f9e5815f8a0 2023-08-27T11:17:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5ef96700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e581f51f0 con 0x7f9e58144570 2023-08-27T11:17:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5ef96700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580c80a0 con 0x7f9e580b3e50 2023-08-27T11:17:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5d533700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e5815f8a0 0x7f9e580a86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5d533700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e5815f8a0 0x7f9e580a86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54556/0 (socket says 172.21.15.112:54556) 2023-08-27T11:17:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5d533700 1 -- 172.21.15.112:0/1693632682 learned_addr learned my addr 172.21.15.112:0/1693632682 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:00.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5cd32700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e580b3e50 0x7f9e581f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e57fff700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e58144570 0x7f9e580c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5d533700 1 -- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e580b3e50 msgr2=0x7f9e581f4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5d533700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e580b3e50 0x7f9e581f4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5d533700 1 -- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e58144570 msgr2=0x7f9e580c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:00.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5d533700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e58144570 0x7f9e580c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:00.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.909+0000 7f9e5d533700 1 -- 172.21.15.112:0/1693632682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c0094a0 con 0x7f9e5815f8a0 2023-08-27T11:17:00.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.910+0000 7f9e5d533700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e5815f8a0 0x7f9e580a86f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000cba0 tx=0x7f9e5000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:00.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.910+0000 7f9e557fa700 1 -- 172.21.15.112:0/1693632682 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000aaa0 con 0x7f9e5815f8a0 2023-08-27T11:17:00.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.910+0000 7f9e557fa700 1 -- 172.21.15.112:0/1693632682 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5000ac20 con 0x7f9e5815f8a0 2023-08-27T11:17:00.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.910+0000 7f9e5ef96700 1 -- 172.21.15.112:0/1693632682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e580c8380 con 0x7f9e5815f8a0 2023-08-27T11:17:00.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.910+0000 7f9e557fa700 1 -- 172.21.15.112:0/1693632682 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e500070d0 con 0x7f9e5815f8a0 2023-08-27T11:17:00.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.910+0000 7f9e5ef96700 1 -- 172.21.15.112:0/1693632682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e580c8940 con 0x7f9e5815f8a0 2023-08-27T11:17:00.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.911+0000 7f9e5ef96700 1 -- 172.21.15.112:0/1693632682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e580c8e60 con 0x7f9e5815f8a0 2023-08-27T11:17:00.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.911+0000 7f9e557fa700 1 -- 172.21.15.112:0/1693632682 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7f9e50007230 con 0x7f9e5815f8a0 2023-08-27T11:17:00.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.911+0000 7f9e557fa700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e3c06a050 0x7f9e3c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.912+0000 7f9e557fa700 1 -- 172.21.15.112:0/1693632682 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9e50087c90 con 0x7f9e5815f8a0 2023-08-27T11:17:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.912+0000 7f9e5cd32700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e3c06a050 0x7f9e3c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.912+0000 7f9e5cd32700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e3c06a050 0x7f9e3c06c510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0090b0 tx=0x7f9e4c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:00.916+0000 7f9e557fa700 1 -- 172.21.15.112:0/1693632682 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9e580c8e60 con 0x7f9e5815f8a0 2023-08-27T11:17:01.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.106+0000 7f9e5ef96700 1 -- 172.21.15.112:0/1693632682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e580b5ae0 con 0x7f9e5815f8a0 2023-08-27T11:17:01.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.107+0000 7f9e557fa700 1 -- 172.21.15.112:0/1693632682 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14603 (secure 0 0 0) 0x7f9e580b5ae0 con 0x7f9e5815f8a0 2023-08-27T11:17:01.108 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:01.108 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":16,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:00.315006+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.109+0000 7f9e5ef96700 1 -- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e3c06a050 msgr2=0x7f9e3c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.109+0000 7f9e5ef96700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e3c06a050 0x7f9e3c06c510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0090b0 tx=0x7f9e4c01c040 comp rx=0 tx=0).stop 2023-08-27T11:17:01.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.109+0000 7f9e5ef96700 1 -- 172.21.15.112:0/1693632682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e5815f8a0 msgr2=0x7f9e580a86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.109+0000 7f9e5ef96700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e5815f8a0 0x7f9e580a86f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000cba0 tx=0x7f9e5000ceb0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.109+0000 7f9e5ef96700 1 -- 172.21.15.112:0/1693632682 shutdown_connections 2023-08-27T11:17:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.109+0000 7f9e5ef96700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e3c06a050 0x7f9e3c06c510 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.109+0000 7f9e5ef96700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e5815f8a0 0x7f9e580a86f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.109+0000 7f9e5ef96700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e580b3e50 0x7f9e581f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.109+0000 7f9e5ef96700 1 --2- 172.21.15.112:0/1693632682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e58144570 0x7f9e580c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.109+0000 7f9e5ef96700 1 -- 172.21.15.112:0/1693632682 >> 172.21.15.112:0/1693632682 conn(0x7f9e5806f970 msgr2=0x7f9e5815b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:01.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.109+0000 7f9e5ef96700 1 -- 172.21.15.112:0/1693632682 shutdown_connections 2023-08-27T11:17:01.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.109+0000 7f9e5ef96700 1 -- 172.21.15.112:0/1693632682 wait complete. 2023-08-27T11:17:01.122 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:17:01.123 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:17:01.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.245+0000 7ff1d1de6700 1 Processor -- start 2023-08-27T11:17:01.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.245+0000 7ff1d1de6700 1 -- start start 2023-08-27T11:17:01.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.245+0000 7ff1d1de6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff1cc1437c0 0x7ff1cc0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.245+0000 7ff1d1de6700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1cc0cfd50 con 0x7ff1cc143ba0 2023-08-27T11:17:01.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.245+0000 7ff1d1de6700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1cc0cf140 con 0x7ff1cc06d5b0 2023-08-27T11:17:01.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.245+0000 7ff1d1de6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1cc0cf420 con 0x7ff1cc1437c0 2023-08-27T11:17:01.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.246+0000 7ff1cb7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff1cc1437c0 0x7ff1cc0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.246+0000 7ff1caffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff1cc143ba0 0x7ff1cc0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48944/0 (socket says 172.21.15.112:48944) 2023-08-27T11:17:01.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.246+0000 7ff1caffd700 1 -- 172.21.15.112:0/4237666724 learned_addr learned my addr 172.21.15.112:0/4237666724 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:01.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.246+0000 7ff1cb7fe700 1 -- 172.21.15.112:0/4237666724 >> v1:172.21.15.138:6789/0 conn(0x7ff1cc06d5b0 legacy=0x7ff1cc15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.246+0000 7ff1cb7fe700 1 -- 172.21.15.112:0/4237666724 >> v1:172.21.15.112:6789/0 conn(0x7ff1cc143ba0 legacy=0x7ff1cc0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.246+0000 7ff1cb7fe700 1 -- 172.21.15.112:0/4237666724 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc0cf700 con 0x7ff1cc1437c0 2023-08-27T11:17:01.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.246+0000 7ff1ca7fc700 1 -- 172.21.15.112:0/4237666724 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105315945 0 0) 0x7ff1cc0cfd50 con 0x7ff1cc143ba0 2023-08-27T11:17:01.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.246+0000 7ff1cb7fe700 1 --2- 172.21.15.112:0/4237666724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc1437c0 0x7ff1cc0b4670 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009840 tx=0x7ff1bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6e60cf93758caa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:01.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.246+0000 7ff1ca7fc700 1 -- 172.21.15.112:0/4237666724 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc011830 con 0x7ff1cc1437c0 2023-08-27T11:17:01.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.246+0000 7ff1ca7fc700 1 -- 172.21.15.112:0/4237666724 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc011990 con 0x7ff1cc1437c0 2023-08-27T11:17:01.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.246+0000 7ff1ca7fc700 1 -- 172.21.15.112:0/4237666724 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc011d20 con 0x7ff1cc1437c0 2023-08-27T11:17:01.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.247+0000 7ff1d1de6700 1 -- 172.21.15.112:0/4237666724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc1437c0 msgr2=0x7ff1cc0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.247+0000 7ff1d1de6700 1 --2- 172.21.15.112:0/4237666724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc1437c0 0x7ff1cc0b4670 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009840 tx=0x7ff1bc009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:01.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.247+0000 7ff1d1de6700 1 -- 172.21.15.112:0/4237666724 shutdown_connections 2023-08-27T11:17:01.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.247+0000 7ff1d1de6700 1 --2- 172.21.15.112:0/4237666724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc1437c0 0x7ff1cc0b4670 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.247+0000 7ff1d1de6700 1 -- 172.21.15.112:0/4237666724 >> 172.21.15.112:0/4237666724 conn(0x7ff1cc0b1e90 msgr2=0x7ff1cc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:01.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.247+0000 7ff1d1de6700 1 -- 172.21.15.112:0/4237666724 shutdown_connections 2023-08-27T11:17:01.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.247+0000 7ff1d1de6700 1 -- 172.21.15.112:0/4237666724 wait complete. 2023-08-27T11:17:01.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.248+0000 7ff1d1de6700 1 Processor -- start 2023-08-27T11:17:01.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.248+0000 7ff1d1de6700 1 -- start start 2023-08-27T11:17:01.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1d1de6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc06d5b0 0x7ff1cc1460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1d1de6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1cc1437c0 0x7ff1cc141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1d1de6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc143ba0 0x7ff1cc141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1d1de6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc143fa0 con 0x7ff1cc143ba0 2023-08-27T11:17:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1d1de6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1441f0 con 0x7ff1cc06d5b0 2023-08-27T11:17:01.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1d1de6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1444d0 con 0x7ff1cc1437c0 2023-08-27T11:17:01.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1cbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc143ba0 0x7ff1cc141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1cbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc143ba0 0x7ff1cc141d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54570/0 (socket says 172.21.15.112:54570) 2023-08-27T11:17:01.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1cbfff700 1 -- 172.21.15.112:0/164014023 learned_addr learned my addr 172.21.15.112:0/164014023 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1cb7fe700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc06d5b0 0x7ff1cc1460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1caffd700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1cc1437c0 0x7ff1cc141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1cbfff700 1 -- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1cc1437c0 msgr2=0x7ff1cc141850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1cbfff700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1cc1437c0 0x7ff1cc141850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1cbfff700 1 -- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc06d5b0 msgr2=0x7ff1cc1460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1cbfff700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc06d5b0 0x7ff1cc1460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1cbfff700 1 -- 172.21.15.112:0/164014023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc0094a0 con 0x7ff1cc143ba0 2023-08-27T11:17:01.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1cbfff700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc143ba0 0x7ff1cc141d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400cc50 tx=0x7ff1c400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:01.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1b3fff700 1 -- 172.21.15.112:0/164014023 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4003c00 con 0x7ff1cc143ba0 2023-08-27T11:17:01.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1b3fff700 1 -- 172.21.15.112:0/164014023 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c4003d60 con 0x7ff1cc143ba0 2023-08-27T11:17:01.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1d1de6700 1 -- 172.21.15.112:0/164014023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc1447b0 con 0x7ff1cc143ba0 2023-08-27T11:17:01.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1b3fff700 1 -- 172.21.15.112:0/164014023 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400aab0 con 0x7ff1cc143ba0 2023-08-27T11:17:01.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.249+0000 7ff1d1de6700 1 -- 172.21.15.112:0/164014023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc144d70 con 0x7ff1cc143ba0 2023-08-27T11:17:01.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.250+0000 7ff1d1de6700 1 -- 172.21.15.112:0/164014023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc067e90 con 0x7ff1cc143ba0 2023-08-27T11:17:01.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.251+0000 7ff1b3fff700 1 -- 172.21.15.112:0/164014023 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7ff1c400ac10 con 0x7ff1cc143ba0 2023-08-27T11:17:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.251+0000 7ff1b3fff700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff1b806a050 0x7ff1b806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.251+0000 7ff1b3fff700 1 -- 172.21.15.112:0/164014023 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff1c4088080 con 0x7ff1cc143ba0 2023-08-27T11:17:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.251+0000 7ff1cb7fe700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff1b806a050 0x7ff1b806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.252+0000 7ff1cb7fe700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff1b806a050 0x7ff1b806c510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000c00 tx=0x7ff1bc01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.255+0000 7ff1b3fff700 1 -- 172.21.15.112:0/164014023 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff1c4053ba0 con 0x7ff1cc143ba0 2023-08-27T11:17:01.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.332+0000 7ff1b3fff700 1 -- 172.21.15.112:0/164014023 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff1c4053f70 con 0x7ff1cc143ba0 2023-08-27T11:17:01.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.448+0000 7ff1d1de6700 1 -- 172.21.15.112:0/164014023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff1cc067a20 con 0x7ff1cc143ba0 2023-08-27T11:17:01.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.448+0000 7ff1b3fff700 1 -- 172.21.15.112:0/164014023 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff1c4057630 con 0x7ff1cc143ba0 2023-08-27T11:17:01.450 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:01.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.450+0000 7ff1d1de6700 1 -- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff1b806a050 msgr2=0x7ff1b806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.450+0000 7ff1d1de6700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff1b806a050 0x7ff1b806c510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000c00 tx=0x7ff1bc01c040 comp rx=0 tx=0).stop 2023-08-27T11:17:01.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.450+0000 7ff1d1de6700 1 -- 172.21.15.112:0/164014023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc143ba0 msgr2=0x7ff1cc141d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.450+0000 7ff1d1de6700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc143ba0 0x7ff1cc141d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400cc50 tx=0x7ff1c400cf60 comp rx=0 tx=0).stop 2023-08-27T11:17:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.450+0000 7ff1d1de6700 1 -- 172.21.15.112:0/164014023 shutdown_connections 2023-08-27T11:17:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.450+0000 7ff1d1de6700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff1b806a050 0x7ff1b806c510 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.450+0000 7ff1d1de6700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1cc143ba0 0x7ff1cc141d90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.450+0000 7ff1d1de6700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc06d5b0 0x7ff1cc1460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.450+0000 7ff1d1de6700 1 --2- 172.21.15.112:0/164014023 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1cc1437c0 0x7ff1cc141850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.450+0000 7ff1d1de6700 1 -- 172.21.15.112:0/164014023 >> 172.21.15.112:0/164014023 conn(0x7ff1cc0b1e90 msgr2=0x7ff1cc13cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.450+0000 7ff1d1de6700 1 -- 172.21.15.112:0/164014023 shutdown_connections 2023-08-27T11:17:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.451+0000 7ff1d1de6700 1 -- 172.21.15.112:0/164014023 wait complete. 2023-08-27T11:17:01.464 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:01.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.584+0000 7f17cdf7e700 1 Processor -- start 2023-08-27T11:17:01.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.585+0000 7f17cdf7e700 1 -- start start 2023-08-27T11:17:01.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.585+0000 7f17cdf7e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17c8143390 0x7f17c80723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.585+0000 7f17cdf7e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f17c806e4c0 0x7f17c80af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.585+0000 7f17cdf7e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c80b16e0 con 0x7f17c806f820 2023-08-27T11:17:01.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.585+0000 7f17cdf7e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c8070000 con 0x7f17c806e4c0 2023-08-27T11:17:01.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.585+0000 7f17cdf7e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8143770 con 0x7f17c8143390 2023-08-27T11:17:01.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.586+0000 7f17c7fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f17c806e4c0 0x7f17c80af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.586+0000 7f17c7fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f17c806e4c0 0x7f17c80af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54578/0 (socket says 172.21.15.112:54578) 2023-08-27T11:17:01.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.586+0000 7f17c7fff700 1 -- 172.21.15.112:0/3926228178 learned_addr learned my addr 172.21.15.112:0/3926228178 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:01.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.586+0000 7f17c77fe700 1 --2- 172.21.15.112:0/3926228178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17c8143390 0x7f17c80723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.586+0000 7f17c7fff700 1 -- 172.21.15.112:0/3926228178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17c8143390 msgr2=0x7f17c80723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:01.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.586+0000 7f17c7fff700 1 --2- 172.21.15.112:0/3926228178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17c8143390 0x7f17c80723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.586+0000 7f17c7fff700 1 -- 172.21.15.112:0/3926228178 >> v1:172.21.15.138:6789/0 conn(0x7f17c806f820 legacy=0x7f17c806ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.586+0000 7f17c7fff700 1 -- 172.21.15.112:0/3926228178 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c8070f40 con 0x7f17c806e4c0 2023-08-27T11:17:01.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.586+0000 7f17c7fff700 1 --2- 172.21.15.112:0/3926228178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c806e4c0 0x7f17c80af000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009e50 tx=0x7f17c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6132c5e513f30f24 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:01.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.586+0000 7f17c67fc700 1 -- 172.21.15.112:0/3926228178 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c000e070 con 0x7f17c806e4c0 2023-08-27T11:17:01.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.586+0000 7f17c67fc700 1 -- 172.21.15.112:0/3926228178 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c000b9c0 con 0x7f17c806e4c0 2023-08-27T11:17:01.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.586+0000 7f17c67fc700 1 -- 172.21.15.112:0/3926228178 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c000bd70 con 0x7f17c806e4c0 2023-08-27T11:17:01.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.587+0000 7f17cdf7e700 1 -- 172.21.15.112:0/3926228178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c806e4c0 msgr2=0x7f17c80af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.587+0000 7f17cdf7e700 1 --2- 172.21.15.112:0/3926228178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c806e4c0 0x7f17c80af000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009e50 tx=0x7f17c000b330 comp rx=0 tx=0).stop 2023-08-27T11:17:01.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.587+0000 7f17cdf7e700 1 -- 172.21.15.112:0/3926228178 shutdown_connections 2023-08-27T11:17:01.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.587+0000 7f17cdf7e700 1 --2- 172.21.15.112:0/3926228178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c806e4c0 0x7f17c80af000 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.587+0000 7f17cdf7e700 1 --2- 172.21.15.112:0/3926228178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17c8143390 0x7f17c80723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.587+0000 7f17cdf7e700 1 -- 172.21.15.112:0/3926228178 >> 172.21.15.112:0/3926228178 conn(0x7f17c80cc240 msgr2=0x7f17c80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:01.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.587+0000 7f17cdf7e700 1 -- 172.21.15.112:0/3926228178 shutdown_connections 2023-08-27T11:17:01.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.587+0000 7f17cdf7e700 1 -- 172.21.15.112:0/3926228178 wait complete. 2023-08-27T11:17:01.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.588+0000 7f17cdf7e700 1 Processor -- start 2023-08-27T11:17:01.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.588+0000 7f17cdf7e700 1 -- start start 2023-08-27T11:17:01.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17cdf7e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17c806e4c0 0x7f17c81564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17cdf7e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c806f820 0x7f17c8159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17cdf7e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17c8143390 0x7f17c806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17cdf7e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80ac9c0 con 0x7f17c806f820 2023-08-27T11:17:01.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17cdf7e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8156a20 con 0x7f17c806e4c0 2023-08-27T11:17:01.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17cdf7e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c815aab0 con 0x7f17c8143390 2023-08-27T11:17:01.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17c6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c806f820 0x7f17c8159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17c6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c806f820 0x7f17c8159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54582/0 (socket says 172.21.15.112:54582) 2023-08-27T11:17:01.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17c6ffd700 1 -- 172.21.15.112:0/915873827 learned_addr learned my addr 172.21.15.112:0/915873827 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:01.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17c77fe700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17c806e4c0 0x7f17c81564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17c7fff700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17c8143390 0x7f17c806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17c6ffd700 1 -- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17c8143390 msgr2=0x7f17c806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17c6ffd700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17c8143390 0x7f17c806d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17c6ffd700 1 -- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17c806e4c0 msgr2=0x7f17c81564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17c6ffd700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17c806e4c0 0x7f17c81564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17c6ffd700 1 -- 172.21.15.112:0/915873827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c0009b30 con 0x7f17c806f820 2023-08-27T11:17:01.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17c7fff700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17c8143390 0x7f17c806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:01.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17c6ffd700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c806f820 0x7f17c8159ff0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f17b000c4e0 tx=0x7f17b000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:01.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17bffff700 1 -- 172.21.15.112:0/915873827 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b00052d0 con 0x7f17c806f820 2023-08-27T11:17:01.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17bffff700 1 -- 172.21.15.112:0/915873827 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b0005430 con 0x7f17c806f820 2023-08-27T11:17:01.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17cdf7e700 1 -- 172.21.15.112:0/915873827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c80acc60 con 0x7f17c806f820 2023-08-27T11:17:01.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17bffff700 1 -- 172.21.15.112:0/915873827 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b00038c0 con 0x7f17c806f820 2023-08-27T11:17:01.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.589+0000 7f17cdf7e700 1 -- 172.21.15.112:0/915873827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c815ad90 con 0x7f17c806f820 2023-08-27T11:17:01.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.590+0000 7f17cdf7e700 1 -- 172.21.15.112:0/915873827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c80ad180 con 0x7f17c806f820 2023-08-27T11:17:01.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.591+0000 7f17bffff700 1 -- 172.21.15.112:0/915873827 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f17b0003a20 con 0x7f17c806f820 2023-08-27T11:17:01.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.591+0000 7f17bffff700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f17a806a0e0 0x7f17a806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.591+0000 7f17bffff700 1 -- 172.21.15.112:0/915873827 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f17b0087660 con 0x7f17c806f820 2023-08-27T11:17:01.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.591+0000 7f17c77fe700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f17a806a0e0 0x7f17a806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.592+0000 7f17c77fe700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f17a806a0e0 0x7f17a806c5a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b9b0 tx=0x7f17b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:01.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.595+0000 7f17bffff700 1 -- 172.21.15.112:0/915873827 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f17c80ad180 con 0x7f17c806f820 2023-08-27T11:17:01.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.782+0000 7f17cdf7e700 1 -- 172.21.15.112:0/915873827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17c80ad180 con 0x7f17c806f820 2023-08-27T11:17:01.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.782+0000 7f17bffff700 1 -- 172.21.15.112:0/915873827 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17c80ad180 con 0x7f17c806f820 2023-08-27T11:17:01.784 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:01.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.784+0000 7f17cdf7e700 1 -- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f17a806a0e0 msgr2=0x7f17a806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.784+0000 7f17cdf7e700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f17a806a0e0 0x7f17a806c5a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b9b0 tx=0x7f17b8008040 comp rx=0 tx=0).stop 2023-08-27T11:17:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.784+0000 7f17cdf7e700 1 -- 172.21.15.112:0/915873827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c806f820 msgr2=0x7f17c8159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.784+0000 7f17cdf7e700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c806f820 0x7f17c8159ff0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f17b000c4e0 tx=0x7f17b000c8a0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.784+0000 7f17cdf7e700 1 -- 172.21.15.112:0/915873827 shutdown_connections 2023-08-27T11:17:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.784+0000 7f17cdf7e700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f17a806a0e0 0x7f17a806c5a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.784+0000 7f17cdf7e700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17c8143390 0x7f17c806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.784+0000 7f17cdf7e700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17c806e4c0 0x7f17c81564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.784+0000 7f17cdf7e700 1 --2- 172.21.15.112:0/915873827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c806f820 0x7f17c8159ff0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.784+0000 7f17cdf7e700 1 -- 172.21.15.112:0/915873827 >> 172.21.15.112:0/915873827 conn(0x7f17c80cc240 msgr2=0x7f17c80cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.785+0000 7f17cdf7e700 1 -- 172.21.15.112:0/915873827 shutdown_connections 2023-08-27T11:17:01.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.785+0000 7f17cdf7e700 1 -- 172.21.15.112:0/915873827 wait complete. 2023-08-27T11:17:01.798 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:17:01.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.918+0000 7f812f490700 1 Processor -- start 2023-08-27T11:17:01.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.919+0000 7f812f490700 1 -- start start 2023-08-27T11:17:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812f490700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f812806adb0 0x7f812806b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812f490700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f81280cdbd0 0x7f81280ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812f490700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812815eb60 con 0x7f8128151060 2023-08-27T11:17:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812f490700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812806b770 con 0x7f812806adb0 2023-08-27T11:17:01.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812f490700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81280b02f0 con 0x7f81280cdbd0 2023-08-27T11:17:01.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812ca2b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f812806adb0 0x7f812806b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812d22c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8128151060 0x7f812815e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48970/0 (socket says 172.21.15.112:48970) 2023-08-27T11:17:01.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812d22c700 1 -- 172.21.15.112:0/4032720214 learned_addr learned my addr 172.21.15.112:0/4032720214 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:01.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812da2d700 1 --2- 172.21.15.112:0/4032720214 >> v2:172.21.15.138:3300/0 conn(0x7f81280cdbd0 0x7f81280ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812ca2b700 1 -- 172.21.15.112:0/4032720214 >> v2:172.21.15.138:3300/0 conn(0x7f81280cdbd0 msgr2=0x7f81280ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:01.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812ca2b700 1 --2- 172.21.15.112:0/4032720214 >> v2:172.21.15.138:3300/0 conn(0x7f81280cdbd0 0x7f81280ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812ca2b700 1 -- 172.21.15.112:0/4032720214 >> v1:172.21.15.112:6789/0 conn(0x7f8128151060 legacy=0x7f812815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812ca2b700 1 -- 172.21.15.112:0/4032720214 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81280b05d0 con 0x7f812806adb0 2023-08-27T11:17:01.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f811ffff700 1 -- 172.21.15.112:0/4032720214 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220221135 0 0) 0x7f812815eb60 con 0x7f8128151060 2023-08-27T11:17:01.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f812ca2b700 1 --2- 172.21.15.112:0/4032720214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f812806adb0 0x7f812806b230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f81280b1960 tx=0x7f8124009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1bd027398ec37b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:01.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f811ffff700 1 -- 172.21.15.112:0/4032720214 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124011830 con 0x7f812806adb0 2023-08-27T11:17:01.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f811ffff700 1 -- 172.21.15.112:0/4032720214 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8124011990 con 0x7f812806adb0 2023-08-27T11:17:01.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.920+0000 7f811ffff700 1 -- 172.21.15.112:0/4032720214 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124011d20 con 0x7f812806adb0 2023-08-27T11:17:01.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.921+0000 7f812f490700 1 -- 172.21.15.112:0/4032720214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f812806adb0 msgr2=0x7f812806b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.921+0000 7f812f490700 1 --2- 172.21.15.112:0/4032720214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f812806adb0 0x7f812806b230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f81280b1960 tx=0x7f8124009b20 comp rx=0 tx=0).stop 2023-08-27T11:17:01.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.921+0000 7f812f490700 1 -- 172.21.15.112:0/4032720214 shutdown_connections 2023-08-27T11:17:01.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.921+0000 7f812f490700 1 --2- 172.21.15.112:0/4032720214 >> v2:172.21.15.138:3300/0 conn(0x7f81280cdbd0 0x7f81280ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.921+0000 7f812f490700 1 --2- 172.21.15.112:0/4032720214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f812806adb0 0x7f812806b230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.921+0000 7f812f490700 1 -- 172.21.15.112:0/4032720214 >> 172.21.15.112:0/4032720214 conn(0x7f81280d0000 msgr2=0x7f81280d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:01.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.921+0000 7f812f490700 1 -- 172.21.15.112:0/4032720214 shutdown_connections 2023-08-27T11:17:01.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.921+0000 7f812f490700 1 -- 172.21.15.112:0/4032720214 wait complete. 2023-08-27T11:17:01.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.922+0000 7f812f490700 1 Processor -- start 2023-08-27T11:17:01.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.922+0000 7f812f490700 1 -- start start 2023-08-27T11:17:01.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812f490700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f812806adb0 0x7f812806f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812f490700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81280cdbd0 0x7f8128147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812f490700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8128151060 0x7f812815a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812f490700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81280c39f0 con 0x7f81280cdbd0 2023-08-27T11:17:01.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812f490700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812806fb40 con 0x7f812806adb0 2023-08-27T11:17:01.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812f490700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81280c67f0 con 0x7f8128151060 2023-08-27T11:17:01.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812ca2b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81280cdbd0 0x7f8128147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812ca2b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81280cdbd0 0x7f8128147280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54596/0 (socket says 172.21.15.112:54596) 2023-08-27T11:17:01.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812ca2b700 1 -- 172.21.15.112:0/3338541400 learned_addr learned my addr 172.21.15.112:0/3338541400 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:01.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812d22c700 1 --2- 172.21.15.112:0/3338541400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f812806adb0 0x7f812806f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812ca2b700 1 -- 172.21.15.112:0/3338541400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8128151060 msgr2=0x7f812815a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812ca2b700 1 --2- 172.21.15.112:0/3338541400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8128151060 0x7f812815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812ca2b700 1 -- 172.21.15.112:0/3338541400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f812806adb0 msgr2=0x7f812806f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:01.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812ca2b700 1 --2- 172.21.15.112:0/3338541400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f812806adb0 0x7f812806f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:01.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812ca2b700 1 -- 172.21.15.112:0/3338541400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81240094a0 con 0x7f81280cdbd0 2023-08-27T11:17:01.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.923+0000 7f812ca2b700 1 --2- 172.21.15.112:0/3338541400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81280cdbd0 0x7f8128147280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8124014040 tx=0x7f8124004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:01.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.924+0000 7f811dffb700 1 -- 172.21.15.112:0/3338541400 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81240038a0 con 0x7f81280cdbd0 2023-08-27T11:17:01.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.924+0000 7f811dffb700 1 -- 172.21.15.112:0/3338541400 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812401e070 con 0x7f81280cdbd0 2023-08-27T11:17:01.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.924+0000 7f812f490700 1 -- 172.21.15.112:0/3338541400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8128072280 con 0x7f81280cdbd0 2023-08-27T11:17:01.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.924+0000 7f811dffb700 1 -- 172.21.15.112:0/3338541400 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124003c60 con 0x7f81280cdbd0 2023-08-27T11:17:01.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.924+0000 7f812f490700 1 -- 172.21.15.112:0/3338541400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81280c64c0 con 0x7f81280cdbd0 2023-08-27T11:17:01.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.925+0000 7f812f490700 1 -- 172.21.15.112:0/3338541400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81280c3ba0 con 0x7f81280cdbd0 2023-08-27T11:17:01.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.926+0000 7f811dffb700 1 -- 172.21.15.112:0/3338541400 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f812402d450 con 0x7f81280cdbd0 2023-08-27T11:17:01.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.926+0000 7f811dffb700 1 --2- 172.21.15.112:0/3338541400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f810c06a0e0 0x7f810c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:01.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.926+0000 7f811dffb700 1 -- 172.21.15.112:0/3338541400 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8124089330 con 0x7f81280cdbd0 2023-08-27T11:17:01.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.926+0000 7f812d22c700 1 --2- 172.21.15.112:0/3338541400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f810c06a0e0 0x7f810c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:01.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.927+0000 7f812d22c700 1 --2- 172.21.15.112:0/3338541400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f810c06a0e0 0x7f810c06c5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8118000f60 tx=0x7f8118009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:01.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:01.930+0000 7f811dffb700 1 -- 172.21.15.112:0/3338541400 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8124054ee0 con 0x7f81280cdbd0 2023-08-27T11:17:02.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.122+0000 7f812f490700 1 -- 172.21.15.112:0/3338541400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f81280c3ba0 con 0x7f81280cdbd0 2023-08-27T11:17:02.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.123+0000 7f811dffb700 1 -- 172.21.15.112:0/3338541400 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8124020090 con 0x7f81280cdbd0 2023-08-27T11:17:02.125 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:02.125 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:17:02.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:02.125 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2023-08-27T11:17:02.126 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "rbd" 2023-08-27T11:17:02.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:02.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:02.126 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 2, 2023-08-27T11:17:02.126 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2023-08-27T11:17:02.127 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:02.127 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:17:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.125+0000 7f812f490700 1 -- 172.21.15.112:0/3338541400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f810c06a0e0 msgr2=0x7f810c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.125+0000 7f812f490700 1 --2- 172.21.15.112:0/3338541400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f810c06a0e0 0x7f810c06c5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8118000f60 tx=0x7f8118009040 comp rx=0 tx=0).stop 2023-08-27T11:17:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.125+0000 7f812f490700 1 -- 172.21.15.112:0/3338541400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81280cdbd0 msgr2=0x7f8128147280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.125+0000 7f812f490700 1 --2- 172.21.15.112:0/3338541400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81280cdbd0 0x7f8128147280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8124014040 tx=0x7f8124004750 comp rx=0 tx=0).stop 2023-08-27T11:17:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.125+0000 7f812f490700 1 -- 172.21.15.112:0/3338541400 shutdown_connections 2023-08-27T11:17:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.125+0000 7f812f490700 1 --2- 172.21.15.112:0/3338541400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f810c06a0e0 0x7f810c06c5a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.125+0000 7f812f490700 1 --2- 172.21.15.112:0/3338541400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8128151060 0x7f812815a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.125+0000 7f812f490700 1 --2- 172.21.15.112:0/3338541400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f812806adb0 0x7f812806f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.125+0000 7f812f490700 1 --2- 172.21.15.112:0/3338541400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81280cdbd0 0x7f8128147280 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:02.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.125+0000 7f812f490700 1 -- 172.21.15.112:0/3338541400 >> 172.21.15.112:0/3338541400 conn(0x7f81280d0000 msgr2=0x7f81280d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:02.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.125+0000 7f812f490700 1 -- 172.21.15.112:0/3338541400 shutdown_connections 2023-08-27T11:17:02.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.125+0000 7f812f490700 1 -- 172.21.15.112:0/3338541400 wait complete. 2023-08-27T11:17:02.138 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:17:02.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:17:02.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.260+0000 7f873b5e7700 1 Processor -- start 2023-08-27T11:17:02.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.260+0000 7f873b5e7700 1 -- start start 2023-08-27T11:17:02.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.261+0000 7f873b5e7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87340b0cf0 0x7f87340b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:02.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.261+0000 7f873b5e7700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87340b3e70 con 0x7f87340ce070 2023-08-27T11:17:02.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.261+0000 7f873b5e7700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87340b4150 con 0x7f87341438b0 2023-08-27T11:17:02.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.261+0000 7f873b5e7700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87340b16e0 con 0x7f87340b0cf0 2023-08-27T11:17:02.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.261+0000 7f8738b82700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87340b0cf0 0x7f87340b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:02.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.261+0000 7f8739b84700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f87340ce070 0x7f87340ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48988/0 (socket says 172.21.15.112:48988) 2023-08-27T11:17:02.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.261+0000 7f8739b84700 1 -- 172.21.15.112:0/35887889 learned_addr learned my addr 172.21.15.112:0/35887889 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:02.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.262+0000 7f8738b82700 1 -- 172.21.15.112:0/35887889 >> v1:172.21.15.138:6789/0 conn(0x7f87341438b0 legacy=0x7f87340b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:02.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.262+0000 7f8738b82700 1 -- 172.21.15.112:0/35887889 >> v1:172.21.15.112:6789/0 conn(0x7f87340ce070 legacy=0x7f87340ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:02.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.262+0000 7f8738b82700 1 -- 172.21.15.112:0/35887889 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87340b4430 con 0x7f87340b0cf0 2023-08-27T11:17:02.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.262+0000 7f872bfff700 1 -- 172.21.15.112:0/35887889 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418150691 0 0) 0x7f87340b3e70 con 0x7f87340ce070 2023-08-27T11:17:02.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.262+0000 7f8738b82700 1 --2- 172.21.15.112:0/35887889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340b0cf0 0x7f87340b11a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8730009840 tx=0x7f8730009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878086266fced12a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:02.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.262+0000 7f872bfff700 1 -- 172.21.15.112:0/35887889 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730011830 con 0x7f87340b0cf0 2023-08-27T11:17:02.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.262+0000 7f872bfff700 1 -- 172.21.15.112:0/35887889 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8730011990 con 0x7f87340b0cf0 2023-08-27T11:17:02.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.262+0000 7f872bfff700 1 -- 172.21.15.112:0/35887889 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730011d20 con 0x7f87340b0cf0 2023-08-27T11:17:02.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.263+0000 7f873b5e7700 1 -- 172.21.15.112:0/35887889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340b0cf0 msgr2=0x7f87340b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:02.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.263+0000 7f873b5e7700 1 --2- 172.21.15.112:0/35887889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340b0cf0 0x7f87340b11a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8730009840 tx=0x7f8730009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:02.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.263+0000 7f873b5e7700 1 -- 172.21.15.112:0/35887889 shutdown_connections 2023-08-27T11:17:02.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.263+0000 7f873b5e7700 1 --2- 172.21.15.112:0/35887889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340b0cf0 0x7f87340b11a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:02.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.263+0000 7f873b5e7700 1 -- 172.21.15.112:0/35887889 >> 172.21.15.112:0/35887889 conn(0x7f87340b1be0 msgr2=0x7f87340b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:02.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.263+0000 7f873b5e7700 1 -- 172.21.15.112:0/35887889 shutdown_connections 2023-08-27T11:17:02.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.263+0000 7f873b5e7700 1 -- 172.21.15.112:0/35887889 wait complete. 2023-08-27T11:17:02.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.264+0000 7f873b5e7700 1 Processor -- start 2023-08-27T11:17:02.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.264+0000 7f873b5e7700 1 -- start start 2023-08-27T11:17:02.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.265+0000 7f873b5e7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340b0cf0 0x7f873413db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:02.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.265+0000 7f873b5e7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87340ce070 0x7f8734156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:02.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.265+0000 7f873b5e7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87341438b0 0x7f873413cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:02.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.265+0000 7f873b5e7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873413e0a0 con 0x7f87340b0cf0 2023-08-27T11:17:02.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.265+0000 7f873b5e7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8734071610 con 0x7f87340ce070 2023-08-27T11:17:02.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.265+0000 7f873b5e7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8734156560 con 0x7f87341438b0 2023-08-27T11:17:02.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.265+0000 7f8739383700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340b0cf0 0x7f873413db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:02.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.265+0000 7f8739383700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340b0cf0 0x7f873413db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54608/0 (socket says 172.21.15.112:54608) 2023-08-27T11:17:02.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.265+0000 7f8739383700 1 -- 172.21.15.112:0/376108721 learned_addr learned my addr 172.21.15.112:0/376108721 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:02.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.265+0000 7f8738b82700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87340ce070 0x7f8734156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:02.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.265+0000 7f8739b84700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87341438b0 0x7f873413cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:02.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.266+0000 7f8739383700 1 -- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87341438b0 msgr2=0x7f873413cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:02.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.266+0000 7f8739383700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87341438b0 0x7f873413cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:02.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.266+0000 7f8739383700 1 -- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87340ce070 msgr2=0x7f8734156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:02.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.266+0000 7f8739383700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87340ce070 0x7f8734156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:02.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.266+0000 7f8739383700 1 -- 172.21.15.112:0/376108721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87300094a0 con 0x7f87340b0cf0 2023-08-27T11:17:02.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.266+0000 7f8738b82700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87340ce070 0x7f8734156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:02.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.266+0000 7f8739383700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340b0cf0 0x7f873413db60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f872400c700 tx=0x7f872400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:02.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.266+0000 7f8729ffb700 1 -- 172.21.15.112:0/376108721 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87240052d0 con 0x7f87340b0cf0 2023-08-27T11:17:02.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.266+0000 7f8729ffb700 1 -- 172.21.15.112:0/376108721 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8724005430 con 0x7f87340b0cf0 2023-08-27T11:17:02.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.266+0000 7f873b5e7700 1 -- 172.21.15.112:0/376108721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873413bf60 con 0x7f87340b0cf0 2023-08-27T11:17:02.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.266+0000 7f8729ffb700 1 -- 172.21.15.112:0/376108721 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724003880 con 0x7f87340b0cf0 2023-08-27T11:17:02.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.266+0000 7f873b5e7700 1 -- 172.21.15.112:0/376108721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873413c520 con 0x7f87340b0cf0 2023-08-27T11:17:02.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.267+0000 7f873b5e7700 1 -- 172.21.15.112:0/376108721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8734067a20 con 0x7f87340b0cf0 2023-08-27T11:17:02.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.267+0000 7f8729ffb700 1 -- 172.21.15.112:0/376108721 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f87240039e0 con 0x7f87340b0cf0 2023-08-27T11:17:02.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.268+0000 7f8729ffb700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f871806a0e0 0x7f871806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:02.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.268+0000 7f8729ffb700 1 -- 172.21.15.112:0/376108721 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f87240875d0 con 0x7f87340b0cf0 2023-08-27T11:17:02.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.268+0000 7f8738b82700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f871806a0e0 0x7f871806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:02.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.268+0000 7f8738b82700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f871806a0e0 0x7f871806c5a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f873415d620 tx=0x7f873001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:02.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.272+0000 7f8729ffb700 1 -- 172.21.15.112:0/376108721 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f87240530d0 con 0x7f87340b0cf0 2023-08-27T11:17:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:02.465+0000 7f873b5e7700 1 -- 172.21.15.112:0/376108721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8734142e20 con 0x7f87340b0cf0 2023-08-27T11:17:03.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.352+0000 7f8729ffb700 1 -- 172.21.15.112:0/376108721 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f87240566f0 con 0x7f87340b0cf0 2023-08-27T11:17:03.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.357+0000 7f873b5e7700 1 -- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f871806a0e0 msgr2=0x7f871806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:03.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.357+0000 7f873b5e7700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f871806a0e0 0x7f871806c5a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f873415d620 tx=0x7f873001c040 comp rx=0 tx=0).stop 2023-08-27T11:17:03.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.357+0000 7f873b5e7700 1 -- 172.21.15.112:0/376108721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340b0cf0 msgr2=0x7f873413db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:03.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.357+0000 7f873b5e7700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340b0cf0 0x7f873413db60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f872400c700 tx=0x7f872400cac0 comp rx=0 tx=0).stop 2023-08-27T11:17:03.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.357+0000 7f873b5e7700 1 -- 172.21.15.112:0/376108721 shutdown_connections 2023-08-27T11:17:03.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.357+0000 7f873b5e7700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f871806a0e0 0x7f871806c5a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:03.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.357+0000 7f873b5e7700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87341438b0 0x7f873413cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:03.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.357+0000 7f873b5e7700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340b0cf0 0x7f873413db60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:03.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.357+0000 7f873b5e7700 1 --2- 172.21.15.112:0/376108721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87340ce070 0x7f8734156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:03.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.357+0000 7f873b5e7700 1 -- 172.21.15.112:0/376108721 >> 172.21.15.112:0/376108721 conn(0x7f87340b1be0 msgr2=0x7f87340b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:03.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.358+0000 7f873b5e7700 1 -- 172.21.15.112:0/376108721 shutdown_connections 2023-08-27T11:17:03.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.358+0000 7f873b5e7700 1 -- 172.21.15.112:0/376108721 wait complete. 2023-08-27T11:17:03.361 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2023-08-27T11:17:03.374 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:17:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.506+0000 7fdb55420700 1 Processor -- start 2023-08-27T11:17:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.507+0000 7fdb55420700 1 -- start start 2023-08-27T11:17:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.507+0000 7fdb55420700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdb50146fd0 0x7fdb500b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.507+0000 7fdb55420700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdb50070eb0 0x7fdb50071320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.507+0000 7fdb55420700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb500d1500 con 0x7fdb50146bf0 2023-08-27T11:17:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.507+0000 7fdb55420700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb50071860 con 0x7fdb50146fd0 2023-08-27T11:17:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.507+0000 7fdb55420700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb500d0460 con 0x7fdb50070eb0 2023-08-27T11:17:03.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.507+0000 7fdb4e7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdb50146fd0 0x7fdb500b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:03.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.508+0000 7fdb4e7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdb50146fd0 0x7fdb500b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54618/0 (socket says 172.21.15.112:54618) 2023-08-27T11:17:03.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.508+0000 7fdb4e7fc700 1 -- 172.21.15.112:0/1749291160 learned_addr learned my addr 172.21.15.112:0/1749291160 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:03.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.508+0000 7fdb4f7fe700 1 --2- 172.21.15.112:0/1749291160 >> v2:172.21.15.138:3300/0 conn(0x7fdb50070eb0 0x7fdb50071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:03.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.508+0000 7fdb4e7fc700 1 -- 172.21.15.112:0/1749291160 >> v2:172.21.15.138:3300/0 conn(0x7fdb50070eb0 msgr2=0x7fdb50071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:03.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.508+0000 7fdb4e7fc700 1 --2- 172.21.15.112:0/1749291160 >> v2:172.21.15.138:3300/0 conn(0x7fdb50070eb0 0x7fdb50071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:03.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.508+0000 7fdb4e7fc700 1 -- 172.21.15.112:0/1749291160 >> v1:172.21.15.138:6789/0 conn(0x7fdb50146bf0 legacy=0x7fdb500cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:03.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.508+0000 7fdb4e7fc700 1 -- 172.21.15.112:0/1749291160 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb500d0710 con 0x7fdb50146fd0 2023-08-27T11:17:03.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.508+0000 7fdb4e7fc700 1 --2- 172.21.15.112:0/1749291160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb50146fd0 0x7fdb500b5db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb500b58f0 tx=0x7fdb40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175ab0a9289e271b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:03.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.508+0000 7fdb4dffb700 1 -- 172.21.15.112:0/1749291160 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40004230 con 0x7fdb50146fd0 2023-08-27T11:17:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.508+0000 7fdb4dffb700 1 -- 172.21.15.112:0/1749291160 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb40004390 con 0x7fdb50146fd0 2023-08-27T11:17:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.508+0000 7fdb4dffb700 1 -- 172.21.15.112:0/1749291160 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb400036a0 con 0x7fdb50146fd0 2023-08-27T11:17:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.509+0000 7fdb55420700 1 -- 172.21.15.112:0/1749291160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb50146fd0 msgr2=0x7fdb500b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.509+0000 7fdb55420700 1 --2- 172.21.15.112:0/1749291160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb50146fd0 0x7fdb500b5db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb500b58f0 tx=0x7fdb40009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.509+0000 7fdb55420700 1 -- 172.21.15.112:0/1749291160 shutdown_connections 2023-08-27T11:17:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.509+0000 7fdb55420700 1 --2- 172.21.15.112:0/1749291160 >> v2:172.21.15.138:3300/0 conn(0x7fdb50070eb0 0x7fdb50071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.509+0000 7fdb55420700 1 --2- 172.21.15.112:0/1749291160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb50146fd0 0x7fdb500b5db0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.509+0000 7fdb55420700 1 -- 172.21.15.112:0/1749291160 >> 172.21.15.112:0/1749291160 conn(0x7fdb5006c450 msgr2=0x7fdb5006f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.509+0000 7fdb55420700 1 -- 172.21.15.112:0/1749291160 shutdown_connections 2023-08-27T11:17:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.509+0000 7fdb55420700 1 -- 172.21.15.112:0/1749291160 wait complete. 2023-08-27T11:17:03.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.510+0000 7fdb55420700 1 Processor -- start 2023-08-27T11:17:03.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.510+0000 7fdb55420700 1 -- start start 2023-08-27T11:17:03.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb55420700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb50070eb0 0x7fdb50157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:03.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb55420700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb50146bf0 0x7fdb5013bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:03.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb55420700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50146fd0 0x7fdb5013c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:03.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb55420700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb501592c0 con 0x7fdb50070eb0 2023-08-27T11:17:03.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb55420700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb501595a0 con 0x7fdb50146fd0 2023-08-27T11:17:03.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb55420700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb50159880 con 0x7fdb50146bf0 2023-08-27T11:17:03.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb4effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb50070eb0 0x7fdb50157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:03.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb4effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb50070eb0 0x7fdb50157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54622/0 (socket says 172.21.15.112:54622) 2023-08-27T11:17:03.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb4effd700 1 -- 172.21.15.112:0/2642786986 learned_addr learned my addr 172.21.15.112:0/2642786986 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:03.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb4f7fe700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50146fd0 0x7fdb5013c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:03.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb4e7fc700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb50146bf0 0x7fdb5013bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:03.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb4effd700 1 -- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb50146bf0 msgr2=0x7fdb5013bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:03.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb4effd700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb50146bf0 0x7fdb5013bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:03.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb4effd700 1 -- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50146fd0 msgr2=0x7fdb5013c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:03.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb4effd700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50146fd0 0x7fdb5013c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:03.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb4effd700 1 -- 172.21.15.112:0/2642786986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb40009550 con 0x7fdb50070eb0 2023-08-27T11:17:03.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb4f7fe700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50146fd0 0x7fdb5013c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:03.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb4effd700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb50070eb0 0x7fdb50157da0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400c700 tx=0x7fdb4400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:03.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb377fe700 1 -- 172.21.15.112:0/2642786986 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb44011790 con 0x7fdb50070eb0 2023-08-27T11:17:03.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb377fe700 1 -- 172.21.15.112:0/2642786986 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb440118f0 con 0x7fdb50070eb0 2023-08-27T11:17:03.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb55420700 1 -- 172.21.15.112:0/2642786986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb50159b60 con 0x7fdb50070eb0 2023-08-27T11:17:03.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb377fe700 1 -- 172.21.15.112:0/2642786986 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb44011ca0 con 0x7fdb50070eb0 2023-08-27T11:17:03.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.511+0000 7fdb55420700 1 -- 172.21.15.112:0/2642786986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb501582e0 con 0x7fdb50070eb0 2023-08-27T11:17:03.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.512+0000 7fdb55420700 1 -- 172.21.15.112:0/2642786986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5015a080 con 0x7fdb50070eb0 2023-08-27T11:17:03.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.513+0000 7fdb377fe700 1 -- 172.21.15.112:0/2642786986 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdb44005080 con 0x7fdb50070eb0 2023-08-27T11:17:03.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.513+0000 7fdb377fe700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdb3c06a0e0 0x7fdb3c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:03.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.513+0000 7fdb377fe700 1 -- 172.21.15.112:0/2642786986 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fdb44087e70 con 0x7fdb50070eb0 2023-08-27T11:17:03.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.513+0000 7fdb4e7fc700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdb3c06a0e0 0x7fdb3c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:03.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.514+0000 7fdb4e7fc700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdb3c06a0e0 0x7fdb3c06c5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009130 tx=0x7fdb40008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:03.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.518+0000 7fdb377fe700 1 -- 172.21.15.112:0/2642786986 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdb44053740 con 0x7fdb50070eb0 2023-08-27T11:17:03.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:03.704+0000 7fdb55420700 1 -- 172.21.15.112:0/2642786986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdb50067e90 con 0x7fdb50070eb0 2023-08-27T11:17:04.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.359+0000 7fdb377fe700 1 -- 172.21.15.112:0/2642786986 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdb44056d60 con 0x7fdb50070eb0 2023-08-27T11:17:04.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.364+0000 7fdb55420700 1 -- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdb3c06a0e0 msgr2=0x7fdb3c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:04.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.364+0000 7fdb55420700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdb3c06a0e0 0x7fdb3c06c5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009130 tx=0x7fdb40008030 comp rx=0 tx=0).stop 2023-08-27T11:17:04.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.364+0000 7fdb55420700 1 -- 172.21.15.112:0/2642786986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb50070eb0 msgr2=0x7fdb50157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:04.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.364+0000 7fdb55420700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb50070eb0 0x7fdb50157da0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400c700 tx=0x7fdb4400cac0 comp rx=0 tx=0).stop 2023-08-27T11:17:04.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.365+0000 7fdb55420700 1 -- 172.21.15.112:0/2642786986 shutdown_connections 2023-08-27T11:17:04.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.365+0000 7fdb55420700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdb3c06a0e0 0x7fdb3c06c5a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:04.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.365+0000 7fdb55420700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50146fd0 0x7fdb5013c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:04.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.365+0000 7fdb55420700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb50070eb0 0x7fdb50157da0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400c700 tx=0x7fdb4400cac0 comp rx=0 tx=0).stop 2023-08-27T11:17:04.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.365+0000 7fdb55420700 1 --2- 172.21.15.112:0/2642786986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb50146bf0 0x7fdb5013bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:04.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.365+0000 7fdb55420700 1 -- 172.21.15.112:0/2642786986 >> 172.21.15.112:0/2642786986 conn(0x7fdb5006c450 msgr2=0x7fdb5006de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:04.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.367+0000 7fdb55420700 1 -- 172.21.15.112:0/2642786986 shutdown_connections 2023-08-27T11:17:04.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.367+0000 7fdb55420700 1 -- 172.21.15.112:0/2642786986 wait complete. 2023-08-27T11:17:04.372 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2023-08-27T11:17:04.390 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:17:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.516+0000 7f8afa75e700 1 Processor -- start 2023-08-27T11:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.516+0000 7f8afa75e700 1 -- start start 2023-08-27T11:17:04.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8afa75e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8af40b3440 0x7f8af415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:04.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8afa75e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8af415e6e0 0x7f8af406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:04.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8afa75e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af40b5070 con 0x7f8af4146c20 2023-08-27T11:17:04.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8afa75e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af4147000 con 0x7f8af415e6e0 2023-08-27T11:17:04.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8afa75e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4161740 con 0x7f8af40b3440 2023-08-27T11:17:04.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af8cfb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8af415e6e0 0x7f8af406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:04.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af3fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8af4146c20 0x7f8af4160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49012/0 (socket says 172.21.15.112:49012) 2023-08-27T11:17:04.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af8cfb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8af415e6e0 0x7f8af406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54634/0 (socket says 172.21.15.112:54634) 2023-08-27T11:17:04.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af3fff700 1 -- 172.21.15.112:0/2893519600 learned_addr learned my addr 172.21.15.112:0/2893519600 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:04.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af37fe700 1 --2- 172.21.15.112:0/2893519600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8af40b3440 0x7f8af415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:04.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af8cfb700 1 -- 172.21.15.112:0/2893519600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8af40b3440 msgr2=0x7f8af415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:04.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af8cfb700 1 --2- 172.21.15.112:0/2893519600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8af40b3440 0x7f8af415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:04.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af8cfb700 1 -- 172.21.15.112:0/2893519600 >> v1:172.21.15.112:6789/0 conn(0x7f8af4146c20 legacy=0x7f8af4160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:04.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af8cfb700 1 -- 172.21.15.112:0/2893519600 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af4161a20 con 0x7f8af415e6e0 2023-08-27T11:17:04.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af2ffd700 1 -- 172.21.15.112:0/2893519600 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728360692 0 0) 0x7f8ae8001110 con 0x7f8af4146c20 2023-08-27T11:17:04.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af37fe700 1 --2- 172.21.15.112:0/2893519600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8af40b3440 0x7f8af415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:04.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af8cfb700 1 --2- 172.21.15.112:0/2893519600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415e6e0 0x7f8af406f790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009d90 tx=0x7f8aec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774ea57cbb6c8fc2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:04.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af2ffd700 1 -- 172.21.15.112:0/2893519600 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec00e070 con 0x7f8af415e6e0 2023-08-27T11:17:04.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af2ffd700 1 -- 172.21.15.112:0/2893519600 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec00bbd0 con 0x7f8af415e6e0 2023-08-27T11:17:04.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.517+0000 7f8af2ffd700 1 -- 172.21.15.112:0/2893519600 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec012430 con 0x7f8af415e6e0 2023-08-27T11:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.518+0000 7f8afa75e700 1 -- 172.21.15.112:0/2893519600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415e6e0 msgr2=0x7f8af406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.518+0000 7f8afa75e700 1 --2- 172.21.15.112:0/2893519600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415e6e0 0x7f8af406f790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009d90 tx=0x7f8aec00b330 comp rx=0 tx=0).stop 2023-08-27T11:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.518+0000 7f8afa75e700 1 -- 172.21.15.112:0/2893519600 shutdown_connections 2023-08-27T11:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.518+0000 7f8afa75e700 1 --2- 172.21.15.112:0/2893519600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415e6e0 0x7f8af406f790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.518+0000 7f8afa75e700 1 --2- 172.21.15.112:0/2893519600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8af40b3440 0x7f8af415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.518+0000 7f8afa75e700 1 -- 172.21.15.112:0/2893519600 >> 172.21.15.112:0/2893519600 conn(0x7f8af40b1e90 msgr2=0x7f8af40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.518+0000 7f8afa75e700 1 -- 172.21.15.112:0/2893519600 shutdown_connections 2023-08-27T11:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.518+0000 7f8afa75e700 1 -- 172.21.15.112:0/2893519600 wait complete. 2023-08-27T11:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.520+0000 7f8afa75e700 1 Processor -- start 2023-08-27T11:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.520+0000 7f8afa75e700 1 -- start start 2023-08-27T11:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8afa75e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8af40b3440 0x7f8af4160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8afa75e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8af4146c20 0x7f8af4145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8afa75e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415e6e0 0x7f8af413eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8afa75e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af415f4a0 con 0x7f8af415e6e0 2023-08-27T11:17:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8afa75e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af415f780 con 0x7f8af40b3440 2023-08-27T11:17:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8afa75e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af415fa60 con 0x7f8af4146c20 2023-08-27T11:17:04.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af8cfb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415e6e0 0x7f8af413eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:04.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af8cfb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415e6e0 0x7f8af413eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54640/0 (socket says 172.21.15.112:54640) 2023-08-27T11:17:04.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af8cfb700 1 -- 172.21.15.112:0/3416219329 learned_addr learned my addr 172.21.15.112:0/3416219329 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:04.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af37fe700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8af4146c20 0x7f8af4145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:04.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af3fff700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8af40b3440 0x7f8af4160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:04.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af8cfb700 1 -- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8af4146c20 msgr2=0x7f8af4145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:04.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af8cfb700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8af4146c20 0x7f8af4145ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:04.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af8cfb700 1 -- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8af40b3440 msgr2=0x7f8af4160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:04.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af8cfb700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8af40b3440 0x7f8af4160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:04.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af8cfb700 1 -- 172.21.15.112:0/3416219329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec009a70 con 0x7f8af415e6e0 2023-08-27T11:17:04.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af3fff700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8af40b3440 0x7f8af4160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:04.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af8cfb700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415e6e0 0x7f8af413eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8af406e8e0 tx=0x7f8aec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:04.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af0ff9700 1 -- 172.21.15.112:0/3416219329 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec012430 con 0x7f8af415e6e0 2023-08-27T11:17:04.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af0ff9700 1 -- 172.21.15.112:0/3416219329 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec020070 con 0x7f8af415e6e0 2023-08-27T11:17:04.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8afa75e700 1 -- 172.21.15.112:0/3416219329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af415fd40 con 0x7f8af415e6e0 2023-08-27T11:17:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8af0ff9700 1 -- 172.21.15.112:0/3416219329 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec004280 con 0x7f8af415e6e0 2023-08-27T11:17:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.521+0000 7f8afa75e700 1 -- 172.21.15.112:0/3416219329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af4146410 con 0x7f8af415e6e0 2023-08-27T11:17:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.522+0000 7f8afa75e700 1 -- 172.21.15.112:0/3416219329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af4160260 con 0x7f8af415e6e0 2023-08-27T11:17:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.523+0000 7f8af0ff9700 1 -- 172.21.15.112:0/3416219329 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8aec0043e0 con 0x7f8af415e6e0 2023-08-27T11:17:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.523+0000 7f8af0ff9700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8ae006a0e0 0x7f8ae006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:04.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.523+0000 7f8af0ff9700 1 -- 172.21.15.112:0/3416219329 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f8aec022070 con 0x7f8af415e6e0 2023-08-27T11:17:04.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.523+0000 7f8af3fff700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8ae006a0e0 0x7f8ae006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:04.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.524+0000 7f8af3fff700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8ae006a0e0 0x7f8ae006c5a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8007fd0 tx=0x7f8ae8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:04.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.528+0000 7f8af0ff9700 1 -- 172.21.15.112:0/3416219329 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8aec055fe0 con 0x7f8af415e6e0 2023-08-27T11:17:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:04.759+0000 7f8afa75e700 1 -- 172.21.15.112:0/3416219329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8af4067e90 con 0x7f8af415e6e0 2023-08-27T11:17:05.186 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:17:05.189 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:17:05.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.374+0000 7f8af0ff9700 1 -- 172.21.15.112:0/3416219329 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8aec059600 con 0x7f8af415e6e0 2023-08-27T11:17:05.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.378+0000 7f8afa75e700 1 -- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8ae006a0e0 msgr2=0x7f8ae006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:05.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.378+0000 7f8afa75e700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8ae006a0e0 0x7f8ae006c5a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8007fd0 tx=0x7f8ae8009040 comp rx=0 tx=0).stop 2023-08-27T11:17:05.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.378+0000 7f8afa75e700 1 -- 172.21.15.112:0/3416219329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415e6e0 msgr2=0x7f8af413eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:05.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.378+0000 7f8afa75e700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415e6e0 0x7f8af413eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8af406e8e0 tx=0x7f8aec004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.379+0000 7f8afa75e700 1 -- 172.21.15.112:0/3416219329 shutdown_connections 2023-08-27T11:17:05.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.379+0000 7f8afa75e700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8ae006a0e0 0x7f8ae006c5a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.379+0000 7f8afa75e700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415e6e0 0x7f8af413eb30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.379+0000 7f8afa75e700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8af40b3440 0x7f8af4160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.379+0000 7f8afa75e700 1 --2- 172.21.15.112:0/3416219329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8af4146c20 0x7f8af4145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.379+0000 7f8afa75e700 1 -- 172.21.15.112:0/3416219329 >> 172.21.15.112:0/3416219329 conn(0x7f8af40b1e90 msgr2=0x7f8af406ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:05.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.380+0000 7f8afa75e700 1 -- 172.21.15.112:0/3416219329 shutdown_connections 2023-08-27T11:17:05.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.380+0000 7f8afa75e700 1 -- 172.21.15.112:0/3416219329 wait complete. 2023-08-27T11:17:05.384 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 3 and data pool 4 2023-08-27T11:17:05.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.518+0000 7f06536bd700 1 Processor -- start 2023-08-27T11:17:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.518+0000 7f06536bd700 1 -- start start 2023-08-27T11:17:05.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.519+0000 7f06536bd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c06e600 0x7f064c0a9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.519+0000 7f06536bd700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064c06ea70 con 0x7f064c06e150 2023-08-27T11:17:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.519+0000 7f06536bd700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064c0cc200 con 0x7f064c143190 2023-08-27T11:17:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.519+0000 7f06536bd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0cc4e0 con 0x7f064c06e600 2023-08-27T11:17:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.519+0000 7f0650c58700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f064c06e150 0x7f064c0a8b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49026/0 (socket says 172.21.15.112:49026) 2023-08-27T11:17:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.519+0000 7f0650c58700 1 -- 172.21.15.112:0/202334996 learned_addr learned my addr 172.21.15.112:0/202334996 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.519+0000 7f0651c5a700 1 --2- 172.21.15.112:0/202334996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c06e600 0x7f064c0a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.519+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946727967 0 0) 0x7f064c06ea70 con 0x7f064c06e150 2023-08-27T11:17:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06380035b0 con 0x7f064c06e150 2023-08-27T11:17:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749713128 0 0) 0x7f064c0cc200 con 0x7f064c143190 2023-08-27T11:17:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f064c06ea70 con 0x7f064c143190 2023-08-27T11:17:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3382149968 0 0) 0x7f06380035b0 con 0x7f064c06e150 2023-08-27T11:17:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c06e600 msgr2=0x7f064c0a9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 --2- 172.21.15.112:0/202334996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c06e600 0x7f064c0a9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 >> v1:172.21.15.138:6789/0 conn(0x7f064c143190 legacy=0x7f064c0ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c0cc7c0 con 0x7f064c06e150 2023-08-27T11:17:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f06480017f0 con 0x7f064c06e150 2023-08-27T11:17:05.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 >> v1:172.21.15.112:6789/0 conn(0x7f064c06e150 legacy=0x7f064c0a8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:05.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 --2- 172.21.15.112:0/202334996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638003bc0 0x7f0638003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 --2- 172.21.15.112:0/202334996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638004850 0x7f0638004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 --2- 172.21.15.112:0/202334996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c06e600 0x7f0638005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0cc4e0 con 0x7f0638004850 2023-08-27T11:17:05.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063800bac0 con 0x7f0638003bc0 2023-08-27T11:17:05.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063800bc80 con 0x7f064c06e600 2023-08-27T11:17:05.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0650c58700 1 --2- 172.21.15.112:0/202334996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638004850 0x7f0638004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0651459700 1 --2- 172.21.15.112:0/202334996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638003bc0 0x7f0638003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0651c5a700 1 --2- 172.21.15.112:0/202334996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c06e600 0x7f0638005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0650c58700 1 -- 172.21.15.112:0/202334996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c06e600 msgr2=0x7f0638005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0650c58700 1 --2- 172.21.15.112:0/202334996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c06e600 0x7f0638005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0650c58700 1 -- 172.21.15.112:0/202334996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638003bc0 msgr2=0x7f0638003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0650c58700 1 --2- 172.21.15.112:0/202334996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638003bc0 0x7f0638003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0650c58700 1 -- 172.21.15.112:0/202334996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06380035b0 con 0x7f0638004850 2023-08-27T11:17:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0651c5a700 1 --2- 172.21.15.112:0/202334996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c06e600 0x7f0638005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0650c58700 1 --2- 172.21.15.112:0/202334996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638004850 0x7f0638004cd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f064800c110 tx=0x7f0638005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c48ad725dc51fdb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064800ccb0 con 0x7f0638004850 2023-08-27T11:17:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064800ce10 con 0x7f0638004850 2023-08-27T11:17:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.520+0000 7f0643fff700 1 -- 172.21.15.112:0/202334996 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648007310 con 0x7f0638004850 2023-08-27T11:17:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.521+0000 7f06536bd700 1 -- 172.21.15.112:0/202334996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638004850 msgr2=0x7f0638004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.521+0000 7f06536bd700 1 --2- 172.21.15.112:0/202334996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638004850 0x7f0638004cd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f064800c110 tx=0x7f0638005880 comp rx=0 tx=0).stop 2023-08-27T11:17:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.521+0000 7f0651c5a700 1 -- 172.21.15.112:0/202334996 reap_dead start 2023-08-27T11:17:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.521+0000 7f06536bd700 1 -- 172.21.15.112:0/202334996 shutdown_connections 2023-08-27T11:17:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.521+0000 7f06536bd700 1 -- 172.21.15.112:0/202334996 >> 172.21.15.112:0/202334996 conn(0x7f064c06cd30 msgr2=0x7f064c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.521+0000 7f06536bd700 1 -- 172.21.15.112:0/202334996 shutdown_connections 2023-08-27T11:17:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.521+0000 7f06536bd700 1 -- 172.21.15.112:0/202334996 wait complete. 2023-08-27T11:17:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.522+0000 7f06536bd700 1 Processor -- start 2023-08-27T11:17:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.522+0000 7f06536bd700 1 -- start start 2023-08-27T11:17:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f06536bd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638004850 0x7f064c153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f06536bd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c159520 0x7f064c158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f06536bd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f064c139030 0x7f064c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f06536bd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c157f20 con 0x7f0638004850 2023-08-27T11:17:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f06536bd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c1584a0 con 0x7f064c139030 2023-08-27T11:17:05.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f06536bd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c158e70 con 0x7f064c159520 2023-08-27T11:17:05.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0651459700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638004850 0x7f064c153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0651459700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638004850 0x7f064c153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54656/0 (socket says 172.21.15.112:54656) 2023-08-27T11:17:05.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0651459700 1 -- 172.21.15.112:0/2192670083 learned_addr learned my addr 172.21.15.112:0/2192670083 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:05.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0651c5a700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f064c139030 0x7f064c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0650c58700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c159520 0x7f064c158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0651459700 1 -- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c159520 msgr2=0x7f064c158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:05.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0651459700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c159520 0x7f064c158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0651459700 1 -- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f064c139030 msgr2=0x7f064c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:05.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0651459700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f064c139030 0x7f064c0cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0651459700 1 -- 172.21.15.112:0/2192670083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c159ce0 con 0x7f0638004850 2023-08-27T11:17:05.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0650c58700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c159520 0x7f064c158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:05.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0651c5a700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f064c139030 0x7f064c0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:05.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0651459700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638004850 0x7f064c153c10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f063c00f810 tx=0x7f063c00fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:05.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.523+0000 7f0641ffb700 1 -- 172.21.15.112:0/2192670083 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c005320 con 0x7f0638004850 2023-08-27T11:17:05.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.524+0000 7f0641ffb700 1 -- 172.21.15.112:0/2192670083 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c0054a0 con 0x7f0638004850 2023-08-27T11:17:05.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.524+0000 7f06536bd700 1 -- 172.21.15.112:0/2192670083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064c159fc0 con 0x7f0638004850 2023-08-27T11:17:05.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.524+0000 7f0641ffb700 1 -- 172.21.15.112:0/2192670083 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c006aa0 con 0x7f0638004850 2023-08-27T11:17:05.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.524+0000 7f06536bd700 1 -- 172.21.15.112:0/2192670083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064c1599a0 con 0x7f0638004850 2023-08-27T11:17:05.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.525+0000 7f06536bd700 1 -- 172.21.15.112:0/2192670083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064c15a1b0 con 0x7f0638004850 2023-08-27T11:17:05.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.525+0000 7f0641ffb700 1 -- 172.21.15.112:0/2192670083 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f063c006c00 con 0x7f0638004850 2023-08-27T11:17:05.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.525+0000 7f0641ffb700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063006a070 0x7f063006c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.525+0000 7f0641ffb700 1 -- 172.21.15.112:0/2192670083 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f063c087e60 con 0x7f0638004850 2023-08-27T11:17:05.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.526+0000 7f0650c58700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063006a070 0x7f063006c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.526+0000 7f0650c58700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063006a070 0x7f063006c530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f064c154950 tx=0x7f0648007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:05.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.530+0000 7f0641ffb700 1 -- 172.21.15.112:0/2192670083 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f063c053550 con 0x7f0638004850 2023-08-27T11:17:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.719+0000 7f06536bd700 1 -- 172.21.15.112:0/2192670083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f064c067a20 con 0x7f0638004850 2023-08-27T11:17:05.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.721+0000 7f0641ffb700 1 -- 172.21.15.112:0/2192670083 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+18258 (secure 0 0 0) 0x7f063c01a090 con 0x7f0638004850 2023-08-27T11:17:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:05.722 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":19,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:04.761339+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:05.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.723+0000 7f06536bd700 1 -- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063006a070 msgr2=0x7f063006c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:05.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.723+0000 7f06536bd700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063006a070 0x7f063006c530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f064c154950 tx=0x7f0648007000 comp rx=0 tx=0).stop 2023-08-27T11:17:05.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.723+0000 7f06536bd700 1 -- 172.21.15.112:0/2192670083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638004850 msgr2=0x7f064c153c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:05.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.723+0000 7f06536bd700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638004850 0x7f064c153c10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f063c00f810 tx=0x7f063c00fbd0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.723+0000 7f06536bd700 1 -- 172.21.15.112:0/2192670083 shutdown_connections 2023-08-27T11:17:05.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.723+0000 7f06536bd700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063006a070 0x7f063006c530 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.723+0000 7f06536bd700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f064c139030 0x7f064c0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.723+0000 7f06536bd700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638004850 0x7f064c153c10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.723+0000 7f06536bd700 1 --2- 172.21.15.112:0/2192670083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064c159520 0x7f064c158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.723+0000 7f06536bd700 1 -- 172.21.15.112:0/2192670083 >> 172.21.15.112:0/2192670083 conn(0x7f064c06cd30 msgr2=0x7f064c06f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:05.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.723+0000 7f06536bd700 1 -- 172.21.15.112:0/2192670083 shutdown_connections 2023-08-27T11:17:05.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.723+0000 7f06536bd700 1 -- 172.21.15.112:0/2192670083 wait complete. 2023-08-27T11:17:05.737 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:05.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.856+0000 7fa930308700 1 Processor -- start 2023-08-27T11:17:05.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.856+0000 7fa930308700 1 -- start start 2023-08-27T11:17:05.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.857+0000 7fa930308700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa9280b1400 0x7fa9280cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.857+0000 7fa930308700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa928161ce0 0x7fa92806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.857+0000 7fa930308700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa928161850 con 0x7fa9280b17e0 2023-08-27T11:17:05.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.857+0000 7fa930308700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9280cfc10 con 0x7fa9280b1400 2023-08-27T11:17:05.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.857+0000 7fa930308700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa928161200 con 0x7fa928161ce0 2023-08-27T11:17:05.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.857+0000 7fa92e0a4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa9280b1400 0x7fa9280cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.857+0000 7fa92e0a4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa9280b1400 0x7fa9280cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55090/0 (socket says 172.21.15.112:55090) 2023-08-27T11:17:05.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.857+0000 7fa92d8a3700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa9280b17e0 0x7fa92806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:56570/0 (socket says 172.21.15.112:56570) 2023-08-27T11:17:05.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.857+0000 7fa92d8a3700 1 -- 172.21.15.112:0/2096490660 learned_addr learned my addr 172.21.15.112:0/2096490660 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:05.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.857+0000 7fa92e8a5700 1 --2- 172.21.15.112:0/2096490660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa928161ce0 0x7fa92806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.858+0000 7fa92d0a2700 1 -- 172.21.15.112:0/2096490660 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596680415 0 0) 0x7fa928161850 con 0x7fa9280b17e0 2023-08-27T11:17:05.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.858+0000 7fa92e8a5700 1 -- 172.21.15.112:0/2096490660 >> v2:172.21.15.138:3300/0 conn(0x7fa9280b1400 msgr2=0x7fa9280cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:05.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.858+0000 7fa92e8a5700 1 --2- 172.21.15.112:0/2096490660 >> v2:172.21.15.138:3300/0 conn(0x7fa9280b1400 0x7fa9280cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.858+0000 7fa92e8a5700 1 -- 172.21.15.112:0/2096490660 >> v1:172.21.15.138:6789/0 conn(0x7fa9280b17e0 legacy=0x7fa92806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:05.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.858+0000 7fa92e8a5700 1 -- 172.21.15.112:0/2096490660 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9281614e0 con 0x7fa928161ce0 2023-08-27T11:17:05.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.858+0000 7fa92e8a5700 1 --2- 172.21.15.112:0/2096490660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa928161ce0 0x7fa92806f790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa920009d90 tx=0x7fa92000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab3cd1ae1ee36322 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:05.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.858+0000 7fa92d0a2700 1 -- 172.21.15.112:0/2096490660 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92000e070 con 0x7fa928161ce0 2023-08-27T11:17:05.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.858+0000 7fa92d0a2700 1 -- 172.21.15.112:0/2096490660 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92000bb20 con 0x7fa928161ce0 2023-08-27T11:17:05.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.858+0000 7fa92d0a2700 1 -- 172.21.15.112:0/2096490660 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920012430 con 0x7fa928161ce0 2023-08-27T11:17:05.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.859+0000 7fa930308700 1 -- 172.21.15.112:0/2096490660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa928161ce0 msgr2=0x7fa92806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:05.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.859+0000 7fa930308700 1 --2- 172.21.15.112:0/2096490660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa928161ce0 0x7fa92806f790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa920009d90 tx=0x7fa92000b330 comp rx=0 tx=0).stop 2023-08-27T11:17:05.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.859+0000 7fa930308700 1 -- 172.21.15.112:0/2096490660 shutdown_connections 2023-08-27T11:17:05.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.859+0000 7fa930308700 1 --2- 172.21.15.112:0/2096490660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa928161ce0 0x7fa92806f790 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.859+0000 7fa930308700 1 --2- 172.21.15.112:0/2096490660 >> v2:172.21.15.138:3300/0 conn(0x7fa9280b1400 0x7fa9280cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.859+0000 7fa930308700 1 -- 172.21.15.112:0/2096490660 >> 172.21.15.112:0/2096490660 conn(0x7fa9280d0000 msgr2=0x7fa9280d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:05.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.859+0000 7fa930308700 1 -- 172.21.15.112:0/2096490660 shutdown_connections 2023-08-27T11:17:05.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.859+0000 7fa930308700 1 -- 172.21.15.112:0/2096490660 wait complete. 2023-08-27T11:17:05.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.860+0000 7fa930308700 1 Processor -- start 2023-08-27T11:17:05.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.860+0000 7fa930308700 1 -- start start 2023-08-27T11:17:05.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa930308700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9280b1400 0x7fa9280c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa930308700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9280b17e0 0x7fa928071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa930308700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa928161ce0 0x7fa928152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa930308700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280c3f00 con 0x7fa9280b1400 2023-08-27T11:17:05.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa930308700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280c41a0 con 0x7fa9280b17e0 2023-08-27T11:17:05.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa930308700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280c4480 con 0x7fa928161ce0 2023-08-27T11:17:05.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa92e0a4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9280b1400 0x7fa9280c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa92e0a4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9280b1400 0x7fa9280c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54670/0 (socket says 172.21.15.112:54670) 2023-08-27T11:17:05.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa92e0a4700 1 -- 172.21.15.112:0/3714876170 learned_addr learned my addr 172.21.15.112:0/3714876170 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:05.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa92d8a3700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9280b17e0 0x7fa928071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa92e8a5700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa928161ce0 0x7fa928152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa92e0a4700 1 -- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa928161ce0 msgr2=0x7fa928152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:05.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa92e0a4700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa928161ce0 0x7fa928152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa92e0a4700 1 -- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9280b17e0 msgr2=0x7fa928071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:05.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa92e0a4700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9280b17e0 0x7fa928071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:05.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa92e0a4700 1 -- 172.21.15.112:0/3714876170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa920009a70 con 0x7fa9280b1400 2023-08-27T11:17:05.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa92e0a4700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9280b1400 0x7fa9280c6200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa91800e9b0 tx=0x7fa91800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:05.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa91effd700 1 -- 172.21.15.112:0/3714876170 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918009710 con 0x7fa9280b1400 2023-08-27T11:17:05.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa91effd700 1 -- 172.21.15.112:0/3714876170 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91800c850 con 0x7fa9280b1400 2023-08-27T11:17:05.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa930308700 1 -- 172.21.15.112:0/3714876170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9280c4760 con 0x7fa9280b1400 2023-08-27T11:17:05.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa91effd700 1 -- 172.21.15.112:0/3714876170 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91800cbc0 con 0x7fa9280b1400 2023-08-27T11:17:05.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.861+0000 7fa930308700 1 -- 172.21.15.112:0/3714876170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa928152b50 con 0x7fa9280b1400 2023-08-27T11:17:05.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.862+0000 7fa930308700 1 -- 172.21.15.112:0/3714876170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa928067a20 con 0x7fa9280b1400 2023-08-27T11:17:05.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.863+0000 7fa91effd700 1 -- 172.21.15.112:0/3714876170 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa91800cd20 con 0x7fa9280b1400 2023-08-27T11:17:05.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.863+0000 7fa91effd700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa90c06a0e0 0x7fa90c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:05.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.863+0000 7fa91effd700 1 -- 172.21.15.112:0/3714876170 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa918088680 con 0x7fa9280b1400 2023-08-27T11:17:05.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.863+0000 7fa92d8a3700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa90c06a0e0 0x7fa90c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:05.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.864+0000 7fa92d8a3700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa90c06a0e0 0x7fa90c06c5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa924000f60 tx=0x7fa924007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:05.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:05.867+0000 7fa91effd700 1 -- 172.21.15.112:0/3714876170 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa918057800 con 0x7fa9280b1400 2023-08-27T11:17:06.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.056+0000 7fa930308700 1 -- 172.21.15.112:0/3714876170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa92806db60 con 0x7fa9280b1400 2023-08-27T11:17:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.057+0000 7fa91effd700 1 -- 172.21.15.112:0/3714876170 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+18258 (secure 0 0 0) 0x7fa918057390 con 0x7fa9280b1400 2023-08-27T11:17:06.059 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:06.059 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":19,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:04.761339+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:06.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.059+0000 7fa930308700 1 -- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa90c06a0e0 msgr2=0x7fa90c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:06.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.059+0000 7fa930308700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa90c06a0e0 0x7fa90c06c5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa924000f60 tx=0x7fa924007c00 comp rx=0 tx=0).stop 2023-08-27T11:17:06.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.059+0000 7fa930308700 1 -- 172.21.15.112:0/3714876170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9280b1400 msgr2=0x7fa9280c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:06.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.059+0000 7fa930308700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9280b1400 0x7fa9280c6200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa91800e9b0 tx=0x7fa91800ed70 comp rx=0 tx=0).stop 2023-08-27T11:17:06.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.059+0000 7fa930308700 1 -- 172.21.15.112:0/3714876170 shutdown_connections 2023-08-27T11:17:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.059+0000 7fa930308700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa90c06a0e0 0x7fa90c06c5a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.059+0000 7fa930308700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa928161ce0 0x7fa928152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.059+0000 7fa930308700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9280b1400 0x7fa9280c6200 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.059+0000 7fa930308700 1 --2- 172.21.15.112:0/3714876170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9280b17e0 0x7fa928071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:06.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.059+0000 7fa930308700 1 -- 172.21.15.112:0/3714876170 >> 172.21.15.112:0/3714876170 conn(0x7fa9280d0000 msgr2=0x7fa9280cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:06.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.059+0000 7fa930308700 1 -- 172.21.15.112:0/3714876170 shutdown_connections 2023-08-27T11:17:06.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.059+0000 7fa930308700 1 -- 172.21.15.112:0/3714876170 wait complete. 2023-08-27T11:17:06.073 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:17:06.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.195+0000 7f3fd1101700 1 Processor -- start 2023-08-27T11:17:06.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.195+0000 7f3fd1101700 1 -- start start 2023-08-27T11:17:06.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.195+0000 7f3fd1101700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3fcc146b60 0x7f3fcc06ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:06.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fd1101700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fcc06e2f0 0x7f3fcc0acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:06.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fd1101700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fcc160fa0 con 0x7f3fcc0ad000 2023-08-27T11:17:06.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fd1101700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fcc161b70 con 0x7f3fcc146b60 2023-08-27T11:17:06.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fd1101700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc143ad0 con 0x7f3fcc06e2f0 2023-08-27T11:17:06.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fcad9d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3fcc146b60 0x7f3fcc06ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:06.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fca59c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fcc06e2f0 0x7f3fcc0acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:06.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fcb59e700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3fcc0ad000 0x7f3fcc143f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:56588/0 (socket says 172.21.15.112:56588) 2023-08-27T11:17:06.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fcb59e700 1 -- 172.21.15.112:0/2678305432 learned_addr learned my addr 172.21.15.112:0/2678305432 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:06.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fcad9d700 1 -- 172.21.15.112:0/2678305432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fcc06e2f0 msgr2=0x7f3fcc0acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:06.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fcad9d700 1 --2- 172.21.15.112:0/2678305432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fcc06e2f0 0x7f3fcc0acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:06.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fcad9d700 1 -- 172.21.15.112:0/2678305432 >> v1:172.21.15.138:6789/0 conn(0x7f3fcc0ad000 legacy=0x7f3fcc143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:06.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fcad9d700 1 -- 172.21.15.112:0/2678305432 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fcc06e750 con 0x7f3fcc146b60 2023-08-27T11:17:06.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fca59c700 1 --2- 172.21.15.112:0/2678305432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fcc06e2f0 0x7f3fcc0acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:06.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.196+0000 7f3fcad9d700 1 --2- 172.21.15.112:0/2678305432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc146b60 0x7f3fcc06ddb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009840 tx=0x7f3fc0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=289556c82697b1e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:06.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.197+0000 7f3fc9d9b700 1 -- 172.21.15.112:0/2678305432 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0011830 con 0x7f3fcc146b60 2023-08-27T11:17:06.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.197+0000 7f3fc9d9b700 1 -- 172.21.15.112:0/2678305432 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc0011990 con 0x7f3fcc146b60 2023-08-27T11:17:06.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.197+0000 7f3fc9d9b700 1 -- 172.21.15.112:0/2678305432 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0011d20 con 0x7f3fcc146b60 2023-08-27T11:17:06.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.197+0000 7f3fd1101700 1 -- 172.21.15.112:0/2678305432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc146b60 msgr2=0x7f3fcc06ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:06.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.197+0000 7f3fd1101700 1 --2- 172.21.15.112:0/2678305432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc146b60 0x7f3fcc06ddb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009840 tx=0x7f3fc0009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:06.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.197+0000 7f3fd1101700 1 -- 172.21.15.112:0/2678305432 shutdown_connections 2023-08-27T11:17:06.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.197+0000 7f3fd1101700 1 --2- 172.21.15.112:0/2678305432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc146b60 0x7f3fcc06ddb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:06.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.197+0000 7f3fd1101700 1 --2- 172.21.15.112:0/2678305432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fcc06e2f0 0x7f3fcc0acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:06.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.197+0000 7f3fd1101700 1 -- 172.21.15.112:0/2678305432 >> 172.21.15.112:0/2678305432 conn(0x7f3fcc0cc240 msgr2=0x7f3fcc0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:06.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.197+0000 7f3fd1101700 1 -- 172.21.15.112:0/2678305432 shutdown_connections 2023-08-27T11:17:06.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.197+0000 7f3fd1101700 1 -- 172.21.15.112:0/2678305432 wait complete. 2023-08-27T11:17:06.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.199+0000 7f3fd1101700 1 Processor -- start 2023-08-27T11:17:06.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.199+0000 7f3fd1101700 1 -- start start 2023-08-27T11:17:06.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.199+0000 7f3fd1101700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fcc06e2f0 0x7f3fcc145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:06.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.199+0000 7f3fd1101700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fcc0ad000 0x7f3fcc06f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:06.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.199+0000 7f3fd1101700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc146b60 0x7f3fcc071250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:06.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.199+0000 7f3fd1101700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc146300 con 0x7f3fcc06e2f0 2023-08-27T11:17:06.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.199+0000 7f3fd1101700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc06ff70 con 0x7f3fcc146b60 2023-08-27T11:17:06.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.199+0000 7f3fd1101700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc070250 con 0x7f3fcc0ad000 2023-08-27T11:17:06.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.199+0000 7f3fcad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fcc06e2f0 0x7f3fcc145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:06.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.199+0000 7f3fcad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fcc06e2f0 0x7f3fcc145dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54684/0 (socket says 172.21.15.112:54684) 2023-08-27T11:17:06.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.199+0000 7f3fcad9d700 1 -- 172.21.15.112:0/3995707661 learned_addr learned my addr 172.21.15.112:0/3995707661 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:06.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fcb59e700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc146b60 0x7f3fcc071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:06.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fca59c700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fcc0ad000 0x7f3fcc06f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:06.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fcad9d700 1 -- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fcc0ad000 msgr2=0x7f3fcc06f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:06.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fcad9d700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fcc0ad000 0x7f3fcc06f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:06.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fcad9d700 1 -- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc146b60 msgr2=0x7f3fcc071250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:06.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fcad9d700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc146b60 0x7f3fcc071250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:06.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fcad9d700 1 -- 172.21.15.112:0/3995707661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc0094a0 con 0x7f3fcc06e2f0 2023-08-27T11:17:06.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fcb59e700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc146b60 0x7f3fcc071250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:06.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fcad9d700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fcc06e2f0 0x7f3fcc145dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009fd0 tx=0x7f3fc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:06.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fb37fe700 1 -- 172.21.15.112:0/3995707661 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc00036a0 con 0x7f3fcc06e2f0 2023-08-27T11:17:06.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fb37fe700 1 -- 172.21.15.112:0/3995707661 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc001e070 con 0x7f3fcc06e2f0 2023-08-27T11:17:06.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fd1101700 1 -- 172.21.15.112:0/3995707661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc00094a0 con 0x7f3fcc06e2f0 2023-08-27T11:17:06.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fb37fe700 1 -- 172.21.15.112:0/3995707661 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0003a60 con 0x7f3fcc06e2f0 2023-08-27T11:17:06.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.200+0000 7f3fd1101700 1 -- 172.21.15.112:0/3995707661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc070810 con 0x7f3fcc06e2f0 2023-08-27T11:17:06.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.201+0000 7f3fd1101700 1 -- 172.21.15.112:0/3995707661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fcc143c10 con 0x7f3fcc06e2f0 2023-08-27T11:17:06.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.201+0000 7f3fb37fe700 1 -- 172.21.15.112:0/3995707661 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3fc0003bc0 con 0x7f3fcc06e2f0 2023-08-27T11:17:06.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.202+0000 7f3fb37fe700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fb806a0e0 0x7f3fb806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:06.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.202+0000 7f3fb37fe700 1 -- 172.21.15.112:0/3995707661 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3fc0089530 con 0x7f3fcc06e2f0 2023-08-27T11:17:06.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.202+0000 7f3fca59c700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fb806a0e0 0x7f3fb806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:06.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.202+0000 7f3fca59c700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fb806a0e0 0x7f3fb806c5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc070fd0 tx=0x7f3fbc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:06.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.206+0000 7f3fb37fe700 1 -- 172.21.15.112:0/3995707661 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3fcc143c10 con 0x7f3fcc06e2f0 2023-08-27T11:17:06.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:06.443+0000 7f3fd1101700 1 -- 172.21.15.112:0/3995707661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3fcc143c10 con 0x7f3fcc06e2f0 2023-08-27T11:17:07.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.372+0000 7f3fb37fe700 1 -- 172.21.15.112:0/3995707661 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3fcc143c10 con 0x7f3fcc06e2f0 2023-08-27T11:17:07.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.374+0000 7f3fd1101700 1 -- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fb806a0e0 msgr2=0x7f3fb806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.374+0000 7f3fd1101700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fb806a0e0 0x7f3fb806c5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc070fd0 tx=0x7f3fbc008040 comp rx=0 tx=0).stop 2023-08-27T11:17:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.374+0000 7f3fd1101700 1 -- 172.21.15.112:0/3995707661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fcc06e2f0 msgr2=0x7f3fcc145dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.374+0000 7f3fd1101700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fcc06e2f0 0x7f3fcc145dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009fd0 tx=0x7f3fc0004750 comp rx=0 tx=0).stop 2023-08-27T11:17:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.374+0000 7f3fd1101700 1 -- 172.21.15.112:0/3995707661 shutdown_connections 2023-08-27T11:17:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.374+0000 7f3fd1101700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fb806a0e0 0x7f3fb806c5a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.374+0000 7f3fd1101700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc146b60 0x7f3fcc071250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.374+0000 7f3fd1101700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fcc06e2f0 0x7f3fcc145dc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.374+0000 7f3fd1101700 1 --2- 172.21.15.112:0/3995707661 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fcc0ad000 0x7f3fcc06f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.374+0000 7f3fd1101700 1 -- 172.21.15.112:0/3995707661 >> 172.21.15.112:0/3995707661 conn(0x7f3fcc0cc240 msgr2=0x7f3fcc142e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.375+0000 7f3fd1101700 1 -- 172.21.15.112:0/3995707661 shutdown_connections 2023-08-27T11:17:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.375+0000 7f3fd1101700 1 -- 172.21.15.112:0/3995707661 wait complete. 2023-08-27T11:17:07.390 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:17:07.391 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:17:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.518+0000 7f4b301ad700 1 Processor -- start 2023-08-27T11:17:07.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.518+0000 7f4b301ad700 1 -- start start 2023-08-27T11:17:07.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b301ad700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4b28144b80 0x7f4b280b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:07.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b301ad700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4b280b3de0 0x7f4b2806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:07.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b301ad700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2806c960 con 0x7f4b2806c510 2023-08-27T11:17:07.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b301ad700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b280b4b40 con 0x7f4b28144b80 2023-08-27T11:17:07.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b301ad700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b280cef20 con 0x7f4b280b3de0 2023-08-27T11:17:07.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b2df49700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4b28144b80 0x7f4b280b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:07.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b2df49700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4b28144b80 0x7f4b280b6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54692/0 (socket says 172.21.15.112:54692) 2023-08-27T11:17:07.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b2df49700 1 -- 172.21.15.112:0/1606515324 learned_addr learned my addr 172.21.15.112:0/1606515324 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:07.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b2d748700 1 --2- 172.21.15.112:0/1606515324 >> v2:172.21.15.138:3300/0 conn(0x7f4b280b3de0 0x7f4b2806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:07.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b2df49700 1 -- 172.21.15.112:0/1606515324 >> v2:172.21.15.138:3300/0 conn(0x7f4b280b3de0 msgr2=0x7f4b2806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:07.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b2df49700 1 --2- 172.21.15.112:0/1606515324 >> v2:172.21.15.138:3300/0 conn(0x7f4b280b3de0 0x7f4b2806bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:07.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b2df49700 1 -- 172.21.15.112:0/1606515324 >> v1:172.21.15.138:6789/0 conn(0x7f4b2806c510 legacy=0x7f4b2815eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:07.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b2df49700 1 -- 172.21.15.112:0/1606515324 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b280b42a0 con 0x7f4b28144b80 2023-08-27T11:17:07.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b2df49700 1 --2- 172.21.15.112:0/1606515324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b28144b80 0x7f4b280b6c00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4b280d0d50 tx=0x7f4b24009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6570d94f11d3ee22 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:07.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b2cf47700 1 -- 172.21.15.112:0/1606515324 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24004230 con 0x7f4b28144b80 2023-08-27T11:17:07.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.519+0000 7f4b2cf47700 1 -- 172.21.15.112:0/1606515324 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b24004390 con 0x7f4b28144b80 2023-08-27T11:17:07.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.520+0000 7f4b2cf47700 1 -- 172.21.15.112:0/1606515324 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b240036a0 con 0x7f4b28144b80 2023-08-27T11:17:07.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.520+0000 7f4b301ad700 1 -- 172.21.15.112:0/1606515324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b28144b80 msgr2=0x7f4b280b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:07.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.520+0000 7f4b301ad700 1 --2- 172.21.15.112:0/1606515324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b28144b80 0x7f4b280b6c00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4b280d0d50 tx=0x7f4b24009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:07.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.520+0000 7f4b301ad700 1 -- 172.21.15.112:0/1606515324 shutdown_connections 2023-08-27T11:17:07.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.520+0000 7f4b301ad700 1 --2- 172.21.15.112:0/1606515324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b28144b80 0x7f4b280b6c00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:07.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.520+0000 7f4b301ad700 1 --2- 172.21.15.112:0/1606515324 >> v2:172.21.15.138:3300/0 conn(0x7f4b280b3de0 0x7f4b2806bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:07.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.520+0000 7f4b301ad700 1 -- 172.21.15.112:0/1606515324 >> 172.21.15.112:0/1606515324 conn(0x7f4b280cdc50 msgr2=0x7f4b280ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:07.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.520+0000 7f4b301ad700 1 -- 172.21.15.112:0/1606515324 shutdown_connections 2023-08-27T11:17:07.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.520+0000 7f4b301ad700 1 -- 172.21.15.112:0/1606515324 wait complete. 2023-08-27T11:17:07.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.521+0000 7f4b301ad700 1 Processor -- start 2023-08-27T11:17:07.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.521+0000 7f4b301ad700 1 -- start start 2023-08-27T11:17:07.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b301ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b2806c510 0x7f4b280cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:07.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b301ad700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b280b3de0 0x7f4b281579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:07.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b301ad700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b28144b80 0x7f4b280ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:07.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b301ad700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b28157f20 con 0x7f4b280b3de0 2023-08-27T11:17:07.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b301ad700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b280cd490 con 0x7f4b2806c510 2023-08-27T11:17:07.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b301ad700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b280cc4f0 con 0x7f4b28144b80 2023-08-27T11:17:07.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b2d748700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b280b3de0 0x7f4b281579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:07.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b2d748700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b280b3de0 0x7f4b281579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54698/0 (socket says 172.21.15.112:54698) 2023-08-27T11:17:07.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b2d748700 1 -- 172.21.15.112:0/385277923 learned_addr learned my addr 172.21.15.112:0/385277923 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:07.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b2df49700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b2806c510 0x7f4b280cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:07.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b2e74a700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b28144b80 0x7f4b280ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:07.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b2d748700 1 -- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b28144b80 msgr2=0x7f4b280ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:07.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b2d748700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b28144b80 0x7f4b280ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:07.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b2d748700 1 -- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b2806c510 msgr2=0x7f4b280cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:07.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b2d748700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b2806c510 0x7f4b280cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:07.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.522+0000 7f4b2d748700 1 -- 172.21.15.112:0/385277923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b24009550 con 0x7f4b280b3de0 2023-08-27T11:17:07.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.523+0000 7f4b2d748700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b280b3de0 0x7f4b281579e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800e810 tx=0x7f4b1800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:07.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.523+0000 7f4b1e7fc700 1 -- 172.21.15.112:0/385277923 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18013610 con 0x7f4b280b3de0 2023-08-27T11:17:07.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.523+0000 7f4b1e7fc700 1 -- 172.21.15.112:0/385277923 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b18013770 con 0x7f4b280b3de0 2023-08-27T11:17:07.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.523+0000 7f4b301ad700 1 -- 172.21.15.112:0/385277923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b280cb8e0 con 0x7f4b280b3de0 2023-08-27T11:17:07.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.523+0000 7f4b1e7fc700 1 -- 172.21.15.112:0/385277923 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18013b20 con 0x7f4b280b3de0 2023-08-27T11:17:07.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.523+0000 7f4b301ad700 1 -- 172.21.15.112:0/385277923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b280cbea0 con 0x7f4b280b3de0 2023-08-27T11:17:07.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.524+0000 7f4b301ad700 1 -- 172.21.15.112:0/385277923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b280cc630 con 0x7f4b280b3de0 2023-08-27T11:17:07.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.524+0000 7f4b1e7fc700 1 -- 172.21.15.112:0/385277923 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4b18013c80 con 0x7f4b280b3de0 2023-08-27T11:17:07.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.524+0000 7f4b1e7fc700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4b0c06a0e0 0x7f4b0c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:07.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.524+0000 7f4b1e7fc700 1 -- 172.21.15.112:0/385277923 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4b18088190 con 0x7f4b280b3de0 2023-08-27T11:17:07.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.525+0000 7f4b2df49700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4b0c06a0e0 0x7f4b0c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:07.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.525+0000 7f4b2df49700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4b0c06a0e0 0x7f4b0c06c5a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009130 tx=0x7f4b24008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:07.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.529+0000 7f4b1e7fc700 1 -- 172.21.15.112:0/385277923 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4b280cc630 con 0x7f4b280b3de0 2023-08-27T11:17:07.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:07.762+0000 7f4b301ad700 1 -- 172.21.15.112:0/385277923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4b280cc630 con 0x7f4b280b3de0 2023-08-27T11:17:08.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.376+0000 7f4b1e7fc700 1 -- 172.21.15.112:0/385277923 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4b280cc630 con 0x7f4b280b3de0 2023-08-27T11:17:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.378+0000 7f4b301ad700 1 -- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4b0c06a0e0 msgr2=0x7f4b0c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.378+0000 7f4b301ad700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4b0c06a0e0 0x7f4b0c06c5a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009130 tx=0x7f4b24008030 comp rx=0 tx=0).stop 2023-08-27T11:17:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.378+0000 7f4b301ad700 1 -- 172.21.15.112:0/385277923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b280b3de0 msgr2=0x7f4b281579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.378+0000 7f4b301ad700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b280b3de0 0x7f4b281579e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800e810 tx=0x7f4b1800eb20 comp rx=0 tx=0).stop 2023-08-27T11:17:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.378+0000 7f4b301ad700 1 -- 172.21.15.112:0/385277923 shutdown_connections 2023-08-27T11:17:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.378+0000 7f4b301ad700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4b0c06a0e0 0x7f4b0c06c5a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.378+0000 7f4b301ad700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b28144b80 0x7f4b280ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.378+0000 7f4b301ad700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b2806c510 0x7f4b280cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.378+0000 7f4b301ad700 1 --2- 172.21.15.112:0/385277923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b280b3de0 0x7f4b281579e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.378+0000 7f4b301ad700 1 -- 172.21.15.112:0/385277923 >> 172.21.15.112:0/385277923 conn(0x7f4b280cdc50 msgr2=0x7f4b280d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:08.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.378+0000 7f4b301ad700 1 -- 172.21.15.112:0/385277923 shutdown_connections 2023-08-27T11:17:08.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.378+0000 7f4b301ad700 1 -- 172.21.15.112:0/385277923 wait complete. 2023-08-27T11:17:08.393 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:17:08.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:17:08.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.514+0000 7f622e440700 1 Processor -- start 2023-08-27T11:17:08.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.514+0000 7f622e440700 1 -- start start 2023-08-27T11:17:08.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.515+0000 7f622e440700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f62280b1400 0x7f62280cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.515+0000 7f622e440700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62280b17e0 0x7f622806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.515+0000 7f622e440700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6228161850 con 0x7f6228161ce0 2023-08-27T11:17:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.515+0000 7f622e440700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622806e7d0 con 0x7f62280b1400 2023-08-27T11:17:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.515+0000 7f622e440700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62280cfc10 con 0x7f62280b17e0 2023-08-27T11:17:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.515+0000 7f622c9dd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6228161ce0 0x7f622806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49090/0 (socket says 172.21.15.112:49090) 2023-08-27T11:17:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.515+0000 7f622c9dd700 1 -- 172.21.15.112:0/3294539560 learned_addr learned my addr 172.21.15.112:0/3294539560 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.515+0000 7f6227fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f62280b1400 0x7f62280cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f62277fe700 1 --2- 172.21.15.112:0/3294539560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62280b17e0 0x7f622806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f6227fff700 1 -- 172.21.15.112:0/3294539560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62280b17e0 msgr2=0x7f622806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f6227fff700 1 --2- 172.21.15.112:0/3294539560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62280b17e0 0x7f622806e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f6226ffd700 1 -- 172.21.15.112:0/3294539560 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349781634 0 0) 0x7f6228161850 con 0x7f6228161ce0 2023-08-27T11:17:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f6227fff700 1 -- 172.21.15.112:0/3294539560 >> v1:172.21.15.112:6789/0 conn(0x7f6228161ce0 legacy=0x7f622806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f6227fff700 1 -- 172.21.15.112:0/3294539560 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6228161200 con 0x7f62280b1400 2023-08-27T11:17:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f6227fff700 1 --2- 172.21.15.112:0/3294539560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62280b1400 0x7f62280cf5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6218009840 tx=0x7f6218009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20661dbd9ebdfad server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:08.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f6226ffd700 1 -- 172.21.15.112:0/3294539560 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218011830 con 0x7f62280b1400 2023-08-27T11:17:08.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f6226ffd700 1 -- 172.21.15.112:0/3294539560 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6218011990 con 0x7f62280b1400 2023-08-27T11:17:08.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f6226ffd700 1 -- 172.21.15.112:0/3294539560 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218011d20 con 0x7f62280b1400 2023-08-27T11:17:08.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f622e440700 1 -- 172.21.15.112:0/3294539560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62280b1400 msgr2=0x7f62280cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:08.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f622e440700 1 --2- 172.21.15.112:0/3294539560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62280b1400 0x7f62280cf5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6218009840 tx=0x7f6218009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:08.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f622e440700 1 -- 172.21.15.112:0/3294539560 shutdown_connections 2023-08-27T11:17:08.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f622e440700 1 --2- 172.21.15.112:0/3294539560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62280b1400 0x7f62280cf5a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f622e440700 1 --2- 172.21.15.112:0/3294539560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62280b17e0 0x7f622806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.516+0000 7f622e440700 1 -- 172.21.15.112:0/3294539560 >> 172.21.15.112:0/3294539560 conn(0x7f62280d0000 msgr2=0x7f62280d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:08.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.517+0000 7f622e440700 1 -- 172.21.15.112:0/3294539560 shutdown_connections 2023-08-27T11:17:08.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.517+0000 7f622e440700 1 -- 172.21.15.112:0/3294539560 wait complete. 2023-08-27T11:17:08.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.518+0000 7f622e440700 1 Processor -- start 2023-08-27T11:17:08.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.518+0000 7f622e440700 1 -- start start 2023-08-27T11:17:08.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f622e440700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62280b1400 0x7f62281545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:08.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f622e440700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62280b17e0 0x7f62280c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:08.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f622e440700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6228161ce0 0x7f622806ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:08.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f622e440700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62280c58f0 con 0x7f62280b17e0 2023-08-27T11:17:08.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f622e440700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62281f6100 con 0x7f62280b1400 2023-08-27T11:17:08.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f622e440700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62281f6930 con 0x7f6228161ce0 2023-08-27T11:17:08.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f62277fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62280b17e0 0x7f62280c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:08.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f62277fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62280b17e0 0x7f62280c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54712/0 (socket says 172.21.15.112:54712) 2023-08-27T11:17:08.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f62277fe700 1 -- 172.21.15.112:0/2007456903 learned_addr learned my addr 172.21.15.112:0/2007456903 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:08.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f6227fff700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62280b1400 0x7f62281545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:08.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f622c9dd700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6228161ce0 0x7f622806ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:08.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f62277fe700 1 -- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6228161ce0 msgr2=0x7f622806ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:08.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f62277fe700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6228161ce0 0x7f622806ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f62277fe700 1 -- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62280b1400 msgr2=0x7f62281545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:08.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f62277fe700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62280b1400 0x7f62281545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f62277fe700 1 -- 172.21.15.112:0/2007456903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62180094a0 con 0x7f62280b17e0 2023-08-27T11:17:08.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f62277fe700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62280b17e0 0x7f62280c8870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c820 tx=0x7f621c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:08.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f6224ff9700 1 -- 172.21.15.112:0/2007456903 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c011800 con 0x7f62280b17e0 2023-08-27T11:17:08.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f6224ff9700 1 -- 172.21.15.112:0/2007456903 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c011960 con 0x7f62280b17e0 2023-08-27T11:17:08.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f622e440700 1 -- 172.21.15.112:0/2007456903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62280c9330 con 0x7f62280b17e0 2023-08-27T11:17:08.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f6224ff9700 1 -- 172.21.15.112:0/2007456903 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c011d10 con 0x7f62280b17e0 2023-08-27T11:17:08.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.519+0000 7f622e440700 1 -- 172.21.15.112:0/2007456903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62281f65c0 con 0x7f62280b17e0 2023-08-27T11:17:08.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.520+0000 7f622e440700 1 -- 172.21.15.112:0/2007456903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62281f6ae0 con 0x7f62280b17e0 2023-08-27T11:17:08.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.521+0000 7f6224ff9700 1 -- 172.21.15.112:0/2007456903 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f621c018360 con 0x7f62280b17e0 2023-08-27T11:17:08.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.521+0000 7f6224ff9700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f621406a0e0 0x7f621406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:08.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.521+0000 7f6224ff9700 1 -- 172.21.15.112:0/2007456903 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f621c088560 con 0x7f62280b17e0 2023-08-27T11:17:08.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.521+0000 7f6227fff700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f621406a0e0 0x7f621406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:08.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.522+0000 7f6227fff700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f621406a0e0 0x7f621406c5a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6218000c00 tx=0x7f6218008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:08.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.525+0000 7f6224ff9700 1 -- 172.21.15.112:0/2007456903 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f62281f6ae0 con 0x7f62280b17e0 2023-08-27T11:17:08.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.755+0000 7f622e440700 1 -- 172.21.15.112:0/2007456903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62281f6aa0 con 0x7f62280b17e0 2023-08-27T11:17:08.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.755+0000 7f6224ff9700 1 -- 172.21.15.112:0/2007456903 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4881 (secure 0 0 0) 0x7f621c057650 con 0x7f62280b17e0 2023-08-27T11:17:08.757 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:08.757 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3289488908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3289488908},{"type":"v1","addr":"172.21.15.112:6835","nonce":3289488908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3991653721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3991653721},{"type":"v1","addr":"172.21.15.112:6834","nonce":3991653721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1923303457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1923303457},{"type":"v1","addr":"172.21.15.138:6836","nonce":1923303457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:17:04.761658+0000","modified":"2023-08-27T11:17:08.372245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/1827312637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1827312637},{"type":"v1","addr":"172.21.15.138:6837","nonce":1827312637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:17:08.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.757+0000 7f622e440700 1 -- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f621406a0e0 msgr2=0x7f621406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:08.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.757+0000 7f622e440700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f621406a0e0 0x7f621406c5a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6218000c00 tx=0x7f6218008030 comp rx=0 tx=0).stop 2023-08-27T11:17:08.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.757+0000 7f622e440700 1 -- 172.21.15.112:0/2007456903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62280b17e0 msgr2=0x7f62280c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:08.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.757+0000 7f622e440700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62280b17e0 0x7f62280c8870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c820 tx=0x7f621c00cbe0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.757+0000 7f622e440700 1 -- 172.21.15.112:0/2007456903 shutdown_connections 2023-08-27T11:17:08.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.757+0000 7f622e440700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f621406a0e0 0x7f621406c5a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.757+0000 7f622e440700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6228161ce0 0x7f622806ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.757+0000 7f622e440700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62280b1400 0x7f62281545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.757+0000 7f622e440700 1 --2- 172.21.15.112:0/2007456903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62280b17e0 0x7f62280c8870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.757+0000 7f622e440700 1 -- 172.21.15.112:0/2007456903 >> 172.21.15.112:0/2007456903 conn(0x7f62280d0000 msgr2=0x7f62280cb630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:08.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.757+0000 7f622e440700 1 -- 172.21.15.112:0/2007456903 shutdown_connections 2023-08-27T11:17:08.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.757+0000 7f622e440700 1 -- 172.21.15.112:0/2007456903 wait complete. 2023-08-27T11:17:08.761 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2023-08-27T11:17:08.771 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:08.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.889+0000 7f4a85d65700 1 Processor -- start 2023-08-27T11:17:08.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.889+0000 7f4a85d65700 1 -- start start 2023-08-27T11:17:08.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.889+0000 7f4a85d65700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4a800b3e50 0x7f4a8015f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:08.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.889+0000 7f4a85d65700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a800b45f0 con 0x7f4a80144570 2023-08-27T11:17:08.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.889+0000 7f4a85d65700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a800b4310 con 0x7f4a800b4ad0 2023-08-27T11:17:08.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.889+0000 7f4a85d65700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a800cf7a0 con 0x7f4a800b3e50 2023-08-27T11:17:08.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.890+0000 7f4a7effd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4a800b3e50 0x7f4a8015f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:08.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.890+0000 7f4a7f7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4a80144570 0x7f4a8015ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49100/0 (socket says 172.21.15.112:49100) 2023-08-27T11:17:08.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.890+0000 7f4a7f7fe700 1 -- 172.21.15.112:0/2419572876 learned_addr learned my addr 172.21.15.112:0/2419572876 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:08.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.890+0000 7f4a7effd700 1 -- 172.21.15.112:0/2419572876 >> v1:172.21.15.138:6789/0 conn(0x7f4a800b4ad0 legacy=0x7f4a800b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:08.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.890+0000 7f4a7effd700 1 -- 172.21.15.112:0/2419572876 >> v1:172.21.15.112:6789/0 conn(0x7f4a80144570 legacy=0x7f4a8015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:08.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.890+0000 7f4a7e7fc700 1 -- 172.21.15.112:0/2419572876 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362959493 0 0) 0x7f4a800b45f0 con 0x7f4a80144570 2023-08-27T11:17:08.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.890+0000 7f4a7effd700 1 -- 172.21.15.112:0/2419572876 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a800cdcb0 con 0x7f4a800b3e50 2023-08-27T11:17:08.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.890+0000 7f4a7effd700 1 --2- 172.21.15.112:0/2419572876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a800b3e50 0x7f4a8015f530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009840 tx=0x7f4a74009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153c00165763e170 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:08.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.890+0000 7f4a7e7fc700 1 -- 172.21.15.112:0/2419572876 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74004230 con 0x7f4a800b3e50 2023-08-27T11:17:08.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.890+0000 7f4a7e7fc700 1 -- 172.21.15.112:0/2419572876 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a74004390 con 0x7f4a800b3e50 2023-08-27T11:17:08.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.890+0000 7f4a7e7fc700 1 -- 172.21.15.112:0/2419572876 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a740036a0 con 0x7f4a800b3e50 2023-08-27T11:17:08.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.891+0000 7f4a85d65700 1 -- 172.21.15.112:0/2419572876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a800b3e50 msgr2=0x7f4a8015f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:08.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.891+0000 7f4a85d65700 1 --2- 172.21.15.112:0/2419572876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a800b3e50 0x7f4a8015f530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009840 tx=0x7f4a74009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:08.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.891+0000 7f4a85d65700 1 -- 172.21.15.112:0/2419572876 shutdown_connections 2023-08-27T11:17:08.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.891+0000 7f4a85d65700 1 --2- 172.21.15.112:0/2419572876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a800b3e50 0x7f4a8015f530 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.891+0000 7f4a85d65700 1 -- 172.21.15.112:0/2419572876 >> 172.21.15.112:0/2419572876 conn(0x7f4a8006f970 msgr2=0x7f4a8006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:08.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.891+0000 7f4a85d65700 1 -- 172.21.15.112:0/2419572876 shutdown_connections 2023-08-27T11:17:08.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.891+0000 7f4a85d65700 1 -- 172.21.15.112:0/2419572876 wait complete. 2023-08-27T11:17:08.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.892+0000 7f4a85d65700 1 Processor -- start 2023-08-27T11:17:08.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.892+0000 7f4a85d65700 1 -- start start 2023-08-27T11:17:08.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a85d65700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a800b3e50 0x7f4a8013e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:08.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a85d65700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a800b4ad0 0x7f4a80071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:08.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a85d65700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a80144570 0x7f4a80158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:08.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a85d65700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a801f2d00 con 0x7f4a800b4ad0 2023-08-27T11:17:08.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a85d65700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a80071c20 con 0x7f4a800b3e50 2023-08-27T11:17:08.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a85d65700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a800ca580 con 0x7f4a80144570 2023-08-27T11:17:08.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a7effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a800b4ad0 0x7f4a80071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:08.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a7effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a800b4ad0 0x7f4a80071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54726/0 (socket says 172.21.15.112:54726) 2023-08-27T11:17:08.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a7effd700 1 -- 172.21.15.112:0/4221670820 learned_addr learned my addr 172.21.15.112:0/4221670820 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:08.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a7f7fe700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a800b3e50 0x7f4a8013e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:08.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a7ffff700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a80144570 0x7f4a80158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:08.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a7effd700 1 -- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a80144570 msgr2=0x7f4a80158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:08.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a7effd700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a80144570 0x7f4a80158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a7effd700 1 -- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a800b3e50 msgr2=0x7f4a8013e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:08.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a7effd700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a800b3e50 0x7f4a8013e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:08.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a7effd700 1 -- 172.21.15.112:0/4221670820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a740094a0 con 0x7f4a800b4ad0 2023-08-27T11:17:08.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a7effd700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a800b4ad0 0x7f4a80071280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a740090b0 tx=0x7f4a74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:08.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a67fff700 1 -- 172.21.15.112:0/4221670820 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74011830 con 0x7f4a800b4ad0 2023-08-27T11:17:08.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a67fff700 1 -- 172.21.15.112:0/4221670820 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7401e070 con 0x7f4a800b4ad0 2023-08-27T11:17:08.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a85d65700 1 -- 172.21.15.112:0/4221670820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a800ca860 con 0x7f4a800b4ad0 2023-08-27T11:17:08.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a67fff700 1 -- 172.21.15.112:0/4221670820 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74011bf0 con 0x7f4a800b4ad0 2023-08-27T11:17:08.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.893+0000 7f4a85d65700 1 -- 172.21.15.112:0/4221670820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a800cae20 con 0x7f4a800b4ad0 2023-08-27T11:17:08.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.894+0000 7f4a85d65700 1 -- 172.21.15.112:0/4221670820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a800cb010 con 0x7f4a800b4ad0 2023-08-27T11:17:08.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.895+0000 7f4a67fff700 1 -- 172.21.15.112:0/4221670820 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4a74011d50 con 0x7f4a800b4ad0 2023-08-27T11:17:08.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.895+0000 7f4a67fff700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4a6c06a0e0 0x7f4a6c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:08.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.895+0000 7f4a67fff700 1 -- 172.21.15.112:0/4221670820 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4a74089860 con 0x7f4a800b4ad0 2023-08-27T11:17:08.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.895+0000 7f4a7f7fe700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4a6c06a0e0 0x7f4a6c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:08.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.896+0000 7f4a7f7fe700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4a6c06a0e0 0x7f4a6c06c5a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4a70000f60 tx=0x7f4a70007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:08.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:08.899+0000 7f4a67fff700 1 -- 172.21.15.112:0/4221670820 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4a74054fe0 con 0x7f4a800b4ad0 2023-08-27T11:17:09.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.093+0000 7f4a85d65700 1 -- 172.21.15.112:0/4221670820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a80067a20 con 0x7f4a800b4ad0 2023-08-27T11:17:09.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.094+0000 7f4a67fff700 1 -- 172.21.15.112:0/4221670820 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7f4a74020090 con 0x7f4a800b4ad0 2023-08-27T11:17:09.096 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:09.096 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:09.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.096+0000 7f4a85d65700 1 -- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4a6c06a0e0 msgr2=0x7f4a6c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.096+0000 7f4a85d65700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4a6c06a0e0 0x7f4a6c06c5a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4a70000f60 tx=0x7f4a70007c90 comp rx=0 tx=0).stop 2023-08-27T11:17:09.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.096+0000 7f4a85d65700 1 -- 172.21.15.112:0/4221670820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a800b4ad0 msgr2=0x7f4a80071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.096+0000 7f4a85d65700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a800b4ad0 0x7f4a80071280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a740090b0 tx=0x7f4a74004750 comp rx=0 tx=0).stop 2023-08-27T11:17:09.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.096+0000 7f4a85d65700 1 -- 172.21.15.112:0/4221670820 shutdown_connections 2023-08-27T11:17:09.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.096+0000 7f4a85d65700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4a6c06a0e0 0x7f4a6c06c5a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.096+0000 7f4a85d65700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a80144570 0x7f4a80158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.096+0000 7f4a85d65700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a800b3e50 0x7f4a8013e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.096+0000 7f4a85d65700 1 --2- 172.21.15.112:0/4221670820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a800b4ad0 0x7f4a80071280 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.096+0000 7f4a85d65700 1 -- 172.21.15.112:0/4221670820 >> 172.21.15.112:0/4221670820 conn(0x7f4a8006f970 msgr2=0x7f4a80066870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:09.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.096+0000 7f4a85d65700 1 -- 172.21.15.112:0/4221670820 shutdown_connections 2023-08-27T11:17:09.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.096+0000 7f4a85d65700 1 -- 172.21.15.112:0/4221670820 wait complete. 2023-08-27T11:17:09.110 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:17:09.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:17:09.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.228+0000 7f893e8db700 1 Processor -- start 2023-08-27T11:17:09.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.228+0000 7f893e8db700 1 -- start start 2023-08-27T11:17:09.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.229+0000 7f893e8db700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8938151060 0x7f893815e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.229+0000 7f893e8db700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f893815e990 0x7f893806adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.229+0000 7f893e8db700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893806b710 con 0x7f893806b2f0 2023-08-27T11:17:09.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.229+0000 7f893e8db700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89380b02f0 con 0x7f893815e990 2023-08-27T11:17:09.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.229+0000 7f893e8db700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89380b05d0 con 0x7f8938151060 2023-08-27T11:17:09.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.229+0000 7f89377fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f893815e990 0x7f893806adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.229+0000 7f89377fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f893815e990 0x7f893806adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54732/0 (socket says 172.21.15.112:54732) 2023-08-27T11:17:09.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.229+0000 7f89377fe700 1 -- 172.21.15.112:0/1592684440 learned_addr learned my addr 172.21.15.112:0/1592684440 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:09.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.229+0000 7f8937fff700 1 --2- 172.21.15.112:0/1592684440 >> v2:172.21.15.138:3300/0 conn(0x7f8938151060 0x7f893815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.229+0000 7f89377fe700 1 -- 172.21.15.112:0/1592684440 >> v2:172.21.15.138:3300/0 conn(0x7f8938151060 msgr2=0x7f893815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:09.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.229+0000 7f89377fe700 1 --2- 172.21.15.112:0/1592684440 >> v2:172.21.15.138:3300/0 conn(0x7f8938151060 0x7f893815e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.229+0000 7f89377fe700 1 -- 172.21.15.112:0/1592684440 >> v1:172.21.15.138:6789/0 conn(0x7f893806b2f0 legacy=0x7f893806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.229+0000 7f89377fe700 1 -- 172.21.15.112:0/1592684440 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380b08b0 con 0x7f893815e990 2023-08-27T11:17:09.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.230+0000 7f89377fe700 1 --2- 172.21.15.112:0/1592684440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f893815e990 0x7f893806adb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f893806e8c0 tx=0x7f8928009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed14cef72deffdb8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:09.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.230+0000 7f8936ffd700 1 -- 172.21.15.112:0/1592684440 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892800b870 con 0x7f893815e990 2023-08-27T11:17:09.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.230+0000 7f8936ffd700 1 -- 172.21.15.112:0/1592684440 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892800b9d0 con 0x7f893815e990 2023-08-27T11:17:09.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.230+0000 7f8936ffd700 1 -- 172.21.15.112:0/1592684440 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892800bd60 con 0x7f893815e990 2023-08-27T11:17:09.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.230+0000 7f893e8db700 1 -- 172.21.15.112:0/1592684440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f893815e990 msgr2=0x7f893806adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.230+0000 7f893e8db700 1 --2- 172.21.15.112:0/1592684440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f893815e990 0x7f893806adb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f893806e8c0 tx=0x7f8928009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.230+0000 7f893e8db700 1 -- 172.21.15.112:0/1592684440 shutdown_connections 2023-08-27T11:17:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.230+0000 7f893e8db700 1 --2- 172.21.15.112:0/1592684440 >> v2:172.21.15.138:3300/0 conn(0x7f8938151060 0x7f893815e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.230+0000 7f893e8db700 1 --2- 172.21.15.112:0/1592684440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f893815e990 0x7f893806adb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.230+0000 7f893e8db700 1 -- 172.21.15.112:0/1592684440 >> 172.21.15.112:0/1592684440 conn(0x7f89380d0000 msgr2=0x7f89380d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:09.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.230+0000 7f893e8db700 1 -- 172.21.15.112:0/1592684440 shutdown_connections 2023-08-27T11:17:09.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.230+0000 7f893e8db700 1 -- 172.21.15.112:0/1592684440 wait complete. 2023-08-27T11:17:09.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.231+0000 7f893e8db700 1 Processor -- start 2023-08-27T11:17:09.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.232+0000 7f893e8db700 1 -- start start 2023-08-27T11:17:09.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.232+0000 7f893e8db700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893806b2f0 0x7f89380c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.232+0000 7f893e8db700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8938151060 0x7f893814f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.232+0000 7f893e8db700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f893815e990 0x7f893814fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.232+0000 7f893e8db700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893814a1b0 con 0x7f8938151060 2023-08-27T11:17:09.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.232+0000 7f893e8db700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893814a450 con 0x7f893806b2f0 2023-08-27T11:17:09.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.232+0000 7f893e8db700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893814a730 con 0x7f893815e990 2023-08-27T11:17:09.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.232+0000 7f89377fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8938151060 0x7f893814f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f89377fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8938151060 0x7f893814f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54740/0 (socket says 172.21.15.112:54740) 2023-08-27T11:17:09.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f89377fe700 1 -- 172.21.15.112:0/1127245337 learned_addr learned my addr 172.21.15.112:0/1127245337 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:09.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f8937fff700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893806b2f0 0x7f89380c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f893ce78700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f893815e990 0x7f893814fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f89377fe700 1 -- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f893815e990 msgr2=0x7f893814fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f89377fe700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f893815e990 0x7f893814fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f89377fe700 1 -- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893806b2f0 msgr2=0x7f89380c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f89377fe700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893806b2f0 0x7f89380c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f89377fe700 1 -- 172.21.15.112:0/1127245337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8928009550 con 0x7f8938151060 2023-08-27T11:17:09.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f89377fe700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8938151060 0x7f893814f600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8928000c00 tx=0x7f8928004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:09.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f8934ff9700 1 -- 172.21.15.112:0/1127245337 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8928004540 con 0x7f8938151060 2023-08-27T11:17:09.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f8934ff9700 1 -- 172.21.15.112:0/1127245337 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89280036a0 con 0x7f8938151060 2023-08-27T11:17:09.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f893e8db700 1 -- 172.21.15.112:0/1127245337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893814aa10 con 0x7f8938151060 2023-08-27T11:17:09.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f8934ff9700 1 -- 172.21.15.112:0/1127245337 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8928003a10 con 0x7f8938151060 2023-08-27T11:17:09.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.233+0000 7f893e8db700 1 -- 172.21.15.112:0/1127245337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89380c72f0 con 0x7f8938151060 2023-08-27T11:17:09.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.234+0000 7f893e8db700 1 -- 172.21.15.112:0/1127245337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893814af30 con 0x7f8938151060 2023-08-27T11:17:09.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.234+0000 7f8934ff9700 1 -- 172.21.15.112:0/1127245337 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8928021070 con 0x7f8938151060 2023-08-27T11:17:09.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.235+0000 7f8934ff9700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f892406a0e0 0x7f892406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.235+0000 7f8934ff9700 1 -- 172.21.15.112:0/1127245337 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8928089bb0 con 0x7f8938151060 2023-08-27T11:17:09.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.235+0000 7f8937fff700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f892406a0e0 0x7f892406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.235+0000 7f8937fff700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f892406a0e0 0x7f892406c5a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f892c006b20 tx=0x7f892c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:09.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.239+0000 7f8934ff9700 1 -- 172.21.15.112:0/1127245337 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f893814af30 con 0x7f8938151060 2023-08-27T11:17:09.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.426+0000 7f893e8db700 1 -- 172.21.15.112:0/1127245337 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89380b0e70 con 0x7f892406a0e0 2023-08-27T11:17:09.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.435+0000 7f8934ff9700 1 -- 172.21.15.112:0/1127245337 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287020 (secure 0 0 0) 0x7f892c0076b0 con 0x7f892406a0e0 2023-08-27T11:17:09.440 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:09.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.438+0000 7f893e8db700 1 -- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f892406a0e0 msgr2=0x7f892406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.438+0000 7f893e8db700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f892406a0e0 0x7f892406c5a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f892c006b20 tx=0x7f892c008040 comp rx=0 tx=0).stop 2023-08-27T11:17:09.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.438+0000 7f893e8db700 1 -- 172.21.15.112:0/1127245337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8938151060 msgr2=0x7f893814f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.438+0000 7f893e8db700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8938151060 0x7f893814f600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8928000c00 tx=0x7f8928004750 comp rx=0 tx=0).stop 2023-08-27T11:17:09.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.439+0000 7f893e8db700 1 -- 172.21.15.112:0/1127245337 shutdown_connections 2023-08-27T11:17:09.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.439+0000 7f893e8db700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f892406a0e0 0x7f892406c5a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.439+0000 7f893e8db700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f893815e990 0x7f893814fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.439+0000 7f893e8db700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893806b2f0 0x7f89380c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.439+0000 7f893e8db700 1 --2- 172.21.15.112:0/1127245337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8938151060 0x7f893814f600 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.439+0000 7f893e8db700 1 -- 172.21.15.112:0/1127245337 >> 172.21.15.112:0/1127245337 conn(0x7f89380d0000 msgr2=0x7f893815d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:09.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.439+0000 7f893e8db700 1 -- 172.21.15.112:0/1127245337 shutdown_connections 2023-08-27T11:17:09.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.439+0000 7f893e8db700 1 -- 172.21.15.112:0/1127245337 wait complete. 2023-08-27T11:17:09.445 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:17:09.452 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-08-27T11:17:08.224670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":198,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218980,"kb_used_data":2772,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755740,"statfs":{"total":773094113280,"available":772869877760,"internally_reserved":0,"allocated":2838528,"data_stored":1753007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.872765"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190109+0000","last_change":"2023-08-27T11:17:05.373064+0000","last_active":"2023-08-27T11:17:07.190109+0000","last_peered":"2023-08-27T11:17:07.190109+0000","last_clean":"2023-08-27T11:17:07.190109+0000","last_became_active":"2023-08-27T11:17:05.372771+0000","last_became_peered":"2023-08-27T11:17:05.372771+0000","last_unstale":"2023-08-27T11:17:07.190109+0000","last_undegraded":"2023-08-27T11:17:07.190109+0000","last_fullsized":"2023-08-27T11:17:07.190109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:12:06.488460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202664+0000","last_change":"2023-08-27T11:17:05.380926+0000","last_active":"2023-08-27T11:17:07.202664+0000","last_peered":"2023-08-27T11:17:07.202664+0000","last_clean":"2023-08-27T11:17:07.202664+0000","last_became_active":"2023-08-27T11:17:05.380623+0000","last_became_peered":"2023-08-27T11:17:05.380623+0000","last_unstale":"2023-08-27T11:17:07.202664+0000","last_undegraded":"2023-08-27T11:17:07.202664+0000","last_fullsized":"2023-08-27T11:17:07.202664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368869+0000","last_change":"2023-08-27T11:17:05.377335+0000","last_active":"2023-08-27T11:17:06.368869+0000","last_peered":"2023-08-27T11:17:06.368869+0000","last_clean":"2023-08-27T11:17:06.368869+0000","last_became_active":"2023-08-27T11:17:05.376963+0000","last_became_peered":"2023-08-27T11:17:05.376963+0000","last_unstale":"2023-08-27T11:17:06.368869+0000","last_undegraded":"2023-08-27T11:17:06.368869+0000","last_fullsized":"2023-08-27T11:17:06.368869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:54.962968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369836+0000","last_change":"2023-08-27T11:17:05.371708+0000","last_active":"2023-08-27T11:17:06.369836+0000","last_peered":"2023-08-27T11:17:06.369836+0000","last_clean":"2023-08-27T11:17:06.369836+0000","last_became_active":"2023-08-27T11:17:05.371385+0000","last_became_peered":"2023-08-27T11:17:05.371385+0000","last_unstale":"2023-08-27T11:17:06.369836+0000","last_undegraded":"2023-08-27T11:17:06.369836+0000","last_fullsized":"2023-08-27T11:17:06.369836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607644+0000","last_change":"2023-08-27T11:17:05.377806+0000","last_active":"2023-08-27T11:17:06.607644+0000","last_peered":"2023-08-27T11:17:06.607644+0000","last_clean":"2023-08-27T11:17:06.607644+0000","last_became_active":"2023-08-27T11:17:05.377546+0000","last_became_peered":"2023-08-27T11:17:05.377546+0000","last_unstale":"2023-08-27T11:17:06.607644+0000","last_undegraded":"2023-08-27T11:17:06.607644+0000","last_fullsized":"2023-08-27T11:17:06.607644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:28.711930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369347+0000","last_change":"2023-08-27T11:17:05.371766+0000","last_active":"2023-08-27T11:17:06.369347+0000","last_peered":"2023-08-27T11:17:06.369347+0000","last_clean":"2023-08-27T11:17:06.369347+0000","last_became_active":"2023-08-27T11:17:05.370938+0000","last_became_peered":"2023-08-27T11:17:05.370938+0000","last_unstale":"2023-08-27T11:17:06.369347+0000","last_undegraded":"2023-08-27T11:17:06.369347+0000","last_fullsized":"2023-08-27T11:17:06.369347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:55.731983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203009+0000","last_change":"2023-08-27T11:17:05.383079+0000","last_active":"2023-08-27T11:17:07.203009+0000","last_peered":"2023-08-27T11:17:07.203009+0000","last_clean":"2023-08-27T11:17:07.203009+0000","last_became_active":"2023-08-27T11:17:05.382810+0000","last_became_peered":"2023-08-27T11:17:05.382810+0000","last_unstale":"2023-08-27T11:17:07.203009+0000","last_undegraded":"2023-08-27T11:17:07.203009+0000","last_fullsized":"2023-08-27T11:17:07.203009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:46.743604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202512+0000","last_change":"2023-08-27T11:17:05.376722+0000","last_active":"2023-08-27T11:17:07.202512+0000","last_peered":"2023-08-27T11:17:07.202512+0000","last_clean":"2023-08-27T11:17:07.202512+0000","last_became_active":"2023-08-27T11:17:05.376399+0000","last_became_peered":"2023-08-27T11:17:05.376399+0000","last_unstale":"2023-08-27T11:17:07.202512+0000","last_undegraded":"2023-08-27T11:17:07.202512+0000","last_fullsized":"2023-08-27T11:17:07.202512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607858+0000","last_change":"2023-08-27T11:17:05.371306+0000","last_active":"2023-08-27T11:17:06.607858+0000","last_peered":"2023-08-27T11:17:06.607858+0000","last_clean":"2023-08-27T11:17:06.607858+0000","last_became_active":"2023-08-27T11:17:05.370962+0000","last_became_peered":"2023-08-27T11:17:05.370962+0000","last_unstale":"2023-08-27T11:17:06.607858+0000","last_undegraded":"2023-08-27T11:17:06.607858+0000","last_fullsized":"2023-08-27T11:17:06.607858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:39.209301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202986+0000","last_change":"2023-08-27T11:17:05.372994+0000","last_active":"2023-08-27T11:17:07.202986+0000","last_peered":"2023-08-27T11:17:07.202986+0000","last_clean":"2023-08-27T11:17:07.202986+0000","last_became_active":"2023-08-27T11:17:05.372278+0000","last_became_peered":"2023-08-27T11:17:05.372278+0000","last_unstale":"2023-08-27T11:17:07.202986+0000","last_undegraded":"2023-08-27T11:17:07.202986+0000","last_fullsized":"2023-08-27T11:17:07.202986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369607+0000","last_change":"2023-08-27T11:17:05.376929+0000","last_active":"2023-08-27T11:17:06.369607+0000","last_peered":"2023-08-27T11:17:06.369607+0000","last_clean":"2023-08-27T11:17:06.369607+0000","last_became_active":"2023-08-27T11:17:05.376592+0000","last_became_peered":"2023-08-27T11:17:05.376592+0000","last_unstale":"2023-08-27T11:17:06.369607+0000","last_undegraded":"2023-08-27T11:17:06.369607+0000","last_fullsized":"2023-08-27T11:17:06.369607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:24:05.167634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190152+0000","last_change":"2023-08-27T11:17:05.377608+0000","last_active":"2023-08-27T11:17:07.190152+0000","last_peered":"2023-08-27T11:17:07.190152+0000","last_clean":"2023-08-27T11:17:07.190152+0000","last_became_active":"2023-08-27T11:17:05.376987+0000","last_became_peered":"2023-08-27T11:17:05.376987+0000","last_unstale":"2023-08-27T11:17:07.190152+0000","last_undegraded":"2023-08-27T11:17:07.190152+0000","last_fullsized":"2023-08-27T11:17:07.190152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:30:54.430091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.189991+0000","last_change":"2023-08-27T11:17:05.377602+0000","last_active":"2023-08-27T11:17:07.189991+0000","last_peered":"2023-08-27T11:17:07.189991+0000","last_clean":"2023-08-27T11:17:07.189991+0000","last_became_active":"2023-08-27T11:17:05.377023+0000","last_became_peered":"2023-08-27T11:17:05.377023+0000","last_unstale":"2023-08-27T11:17:07.189991+0000","last_undegraded":"2023-08-27T11:17:07.189991+0000","last_fullsized":"2023-08-27T11:17:07.189991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:18:00.607553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202965+0000","last_change":"2023-08-27T11:17:05.376491+0000","last_active":"2023-08-27T11:17:07.202965+0000","last_peered":"2023-08-27T11:17:07.202965+0000","last_clean":"2023-08-27T11:17:07.202965+0000","last_became_active":"2023-08-27T11:17:05.376149+0000","last_became_peered":"2023-08-27T11:17:05.376149+0000","last_unstale":"2023-08-27T11:17:07.202965+0000","last_undegraded":"2023-08-27T11:17:07.202965+0000","last_fullsized":"2023-08-27T11:17:07.202965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:43.313210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203191+0000","last_change":"2023-08-27T11:17:05.387318+0000","last_active":"2023-08-27T11:17:07.203191+0000","last_peered":"2023-08-27T11:17:07.203191+0000","last_clean":"2023-08-27T11:17:07.203191+0000","last_became_active":"2023-08-27T11:17:05.387002+0000","last_became_peered":"2023-08-27T11:17:05.387002+0000","last_unstale":"2023-08-27T11:17:07.203191+0000","last_undegraded":"2023-08-27T11:17:07.203191+0000","last_fullsized":"2023-08-27T11:17:07.203191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:13.528078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368880+0000","last_change":"2023-08-27T11:17:05.372919+0000","last_active":"2023-08-27T11:17:06.368880+0000","last_peered":"2023-08-27T11:17:06.368880+0000","last_clean":"2023-08-27T11:17:06.368880+0000","last_became_active":"2023-08-27T11:17:05.371149+0000","last_became_peered":"2023-08-27T11:17:05.371149+0000","last_unstale":"2023-08-27T11:17:06.368880+0000","last_undegraded":"2023-08-27T11:17:06.368880+0000","last_fullsized":"2023-08-27T11:17:06.368880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:46.743604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607880+0000","last_change":"2023-08-27T11:17:05.371456+0000","last_active":"2023-08-27T11:17:06.607880+0000","last_peered":"2023-08-27T11:17:06.607880+0000","last_clean":"2023-08-27T11:17:06.607880+0000","last_became_active":"2023-08-27T11:17:05.371193+0000","last_became_peered":"2023-08-27T11:17:05.371193+0000","last_unstale":"2023-08-27T11:17:06.607880+0000","last_undegraded":"2023-08-27T11:17:06.607880+0000","last_fullsized":"2023-08-27T11:17:06.607880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369084+0000","last_change":"2023-08-27T11:17:05.373957+0000","last_active":"2023-08-27T11:17:06.369084+0000","last_peered":"2023-08-27T11:17:06.369084+0000","last_clean":"2023-08-27T11:17:06.369084+0000","last_became_active":"2023-08-27T11:17:05.373507+0000","last_became_peered":"2023-08-27T11:17:05.373507+0000","last_unstale":"2023-08-27T11:17:06.369084+0000","last_undegraded":"2023-08-27T11:17:06.369084+0000","last_fullsized":"2023-08-27T11:17:06.369084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:45.136551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190029+0000","last_change":"2023-08-27T11:17:05.378874+0000","last_active":"2023-08-27T11:17:07.190029+0000","last_peered":"2023-08-27T11:17:07.190029+0000","last_clean":"2023-08-27T11:17:07.190029+0000","last_became_active":"2023-08-27T11:17:05.378585+0000","last_became_peered":"2023-08-27T11:17:05.378585+0000","last_unstale":"2023-08-27T11:17:07.190029+0000","last_undegraded":"2023-08-27T11:17:07.190029+0000","last_fullsized":"2023-08-27T11:17:07.190029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:19:23.730989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368860+0000","last_change":"2023-08-27T11:17:05.377656+0000","last_active":"2023-08-27T11:17:06.368860+0000","last_peered":"2023-08-27T11:17:06.368860+0000","last_clean":"2023-08-27T11:17:06.368860+0000","last_became_active":"2023-08-27T11:17:05.377366+0000","last_became_peered":"2023-08-27T11:17:05.377366+0000","last_unstale":"2023-08-27T11:17:06.368860+0000","last_undegraded":"2023-08-27T11:17:06.368860+0000","last_fullsized":"2023-08-27T11:17:06.368860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202861+0000","last_change":"2023-08-27T11:17:05.379425+0000","last_active":"2023-08-27T11:17:07.202861+0000","last_peered":"2023-08-27T11:17:07.202861+0000","last_clean":"2023-08-27T11:17:07.202861+0000","last_became_active":"2023-08-27T11:17:05.379084+0000","last_became_peered":"2023-08-27T11:17:05.379084+0000","last_unstale":"2023-08-27T11:17:07.202861+0000","last_undegraded":"2023-08-27T11:17:07.202861+0000","last_fullsized":"2023-08-27T11:17:07.202861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607602+0000","last_change":"2023-08-27T11:17:05.381421+0000","last_active":"2023-08-27T11:17:06.607602+0000","last_peered":"2023-08-27T11:17:06.607602+0000","last_clean":"2023-08-27T11:17:06.607602+0000","last_became_active":"2023-08-27T11:17:05.380452+0000","last_became_peered":"2023-08-27T11:17:05.380452+0000","last_unstale":"2023-08-27T11:17:06.607602+0000","last_undegraded":"2023-08-27T11:17:06.607602+0000","last_fullsized":"2023-08-27T11:17:06.607602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202657+0000","last_change":"2023-08-27T11:17:05.378132+0000","last_active":"2023-08-27T11:17:07.202657+0000","last_peered":"2023-08-27T11:17:07.202657+0000","last_clean":"2023-08-27T11:17:07.202657+0000","last_became_active":"2023-08-27T11:17:05.376190+0000","last_became_peered":"2023-08-27T11:17:05.376190+0000","last_unstale":"2023-08-27T11:17:07.202657+0000","last_undegraded":"2023-08-27T11:17:07.202657+0000","last_fullsized":"2023-08-27T11:17:07.202657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203004+0000","last_change":"2023-08-27T11:17:05.378821+0000","last_active":"2023-08-27T11:17:07.203004+0000","last_peered":"2023-08-27T11:17:07.203004+0000","last_clean":"2023-08-27T11:17:07.203004+0000","last_became_active":"2023-08-27T11:17:05.378520+0000","last_became_peered":"2023-08-27T11:17:05.378520+0000","last_unstale":"2023-08-27T11:17:07.203004+0000","last_undegraded":"2023-08-27T11:17:07.203004+0000","last_fullsized":"2023-08-27T11:17:07.203004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:54.962968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367698+0000","last_change":"2023-08-27T11:17:05.379672+0000","last_active":"2023-08-27T11:17:06.367698+0000","last_peered":"2023-08-27T11:17:06.367698+0000","last_clean":"2023-08-27T11:17:06.367698+0000","last_became_active":"2023-08-27T11:17:05.379399+0000","last_became_peered":"2023-08-27T11:17:05.379399+0000","last_unstale":"2023-08-27T11:17:06.367698+0000","last_undegraded":"2023-08-27T11:17:06.367698+0000","last_fullsized":"2023-08-27T11:17:06.367698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:15.192349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367765+0000","last_change":"2023-08-27T11:17:05.376638+0000","last_active":"2023-08-27T11:17:06.367765+0000","last_peered":"2023-08-27T11:17:06.367765+0000","last_clean":"2023-08-27T11:17:06.367765+0000","last_became_active":"2023-08-27T11:17:05.376274+0000","last_became_peered":"2023-08-27T11:17:05.376274+0000","last_unstale":"2023-08-27T11:17:06.367765+0000","last_undegraded":"2023-08-27T11:17:06.367765+0000","last_fullsized":"2023-08-27T11:17:06.367765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202811+0000","last_change":"2023-08-27T11:17:05.382137+0000","last_active":"2023-08-27T11:17:07.202811+0000","last_peered":"2023-08-27T11:17:07.202811+0000","last_clean":"2023-08-27T11:17:07.202811+0000","last_became_active":"2023-08-27T11:17:05.381664+0000","last_became_peered":"2023-08-27T11:17:05.381664+0000","last_unstale":"2023-08-27T11:17:07.202811+0000","last_undegraded":"2023-08-27T11:17:07.202811+0000","last_fullsized":"2023-08-27T11:17:07.202811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369199+0000","last_change":"2023-08-27T11:17:04.376604+0000","last_active":"2023-08-27T11:17:06.369199+0000","last_peered":"2023-08-27T11:17:06.369199+0000","last_clean":"2023-08-27T11:17:06.369199+0000","last_became_active":"2023-08-27T11:17:04.376288+0000","last_became_peered":"2023-08-27T11:17:04.376288+0000","last_unstale":"2023-08-27T11:17:06.369199+0000","last_undegraded":"2023-08-27T11:17:06.369199+0000","last_fullsized":"2023-08-27T11:17:06.369199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:08:24.626675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203128+0000","last_change":"2023-08-27T11:17:05.382097+0000","last_active":"2023-08-27T11:17:07.203128+0000","last_peered":"2023-08-27T11:17:07.203128+0000","last_clean":"2023-08-27T11:17:07.203128+0000","last_became_active":"2023-08-27T11:17:05.381829+0000","last_became_peered":"2023-08-27T11:17:05.381829+0000","last_unstale":"2023-08-27T11:17:07.203128+0000","last_undegraded":"2023-08-27T11:17:07.203128+0000","last_fullsized":"2023-08-27T11:17:07.203128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:45.136551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367598+0000","last_change":"2023-08-27T11:17:04.379398+0000","last_active":"2023-08-27T11:17:06.367598+0000","last_peered":"2023-08-27T11:17:06.367598+0000","last_clean":"2023-08-27T11:17:06.367598+0000","last_became_active":"2023-08-27T11:17:04.379109+0000","last_became_peered":"2023-08-27T11:17:04.379109+0000","last_unstale":"2023-08-27T11:17:06.367598+0000","last_undegraded":"2023-08-27T11:17:06.367598+0000","last_fullsized":"2023-08-27T11:17:06.367598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:45.809057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607933+0000","last_change":"2023-08-27T11:17:05.379671+0000","last_active":"2023-08-27T11:17:06.607933+0000","last_peered":"2023-08-27T11:17:06.607933+0000","last_clean":"2023-08-27T11:17:06.607933+0000","last_became_active":"2023-08-27T11:17:05.379358+0000","last_became_peered":"2023-08-27T11:17:05.379358+0000","last_unstale":"2023-08-27T11:17:06.607933+0000","last_undegraded":"2023-08-27T11:17:06.607933+0000","last_fullsized":"2023-08-27T11:17:06.607933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:54.962968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202719+0000","last_change":"2023-08-27T11:17:04.377528+0000","last_active":"2023-08-27T11:17:07.202719+0000","last_peered":"2023-08-27T11:17:07.202719+0000","last_clean":"2023-08-27T11:17:07.202719+0000","last_became_active":"2023-08-27T11:17:04.377219+0000","last_became_peered":"2023-08-27T11:17:04.377219+0000","last_unstale":"2023-08-27T11:17:07.202719+0000","last_undegraded":"2023-08-27T11:17:07.202719+0000","last_fullsized":"2023-08-27T11:17:07.202719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:38.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369530+0000","last_change":"2023-08-27T11:17:05.372053+0000","last_active":"2023-08-27T11:17:06.369530+0000","last_peered":"2023-08-27T11:17:06.369530+0000","last_clean":"2023-08-27T11:17:06.369530+0000","last_became_active":"2023-08-27T11:17:05.371769+0000","last_became_peered":"2023-08-27T11:17:05.371769+0000","last_unstale":"2023-08-27T11:17:06.369530+0000","last_undegraded":"2023-08-27T11:17:06.369530+0000","last_fullsized":"2023-08-27T11:17:06.369530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607988+0000","last_change":"2023-08-27T11:17:04.378264+0000","last_active":"2023-08-27T11:17:06.607988+0000","last_peered":"2023-08-27T11:17:06.607988+0000","last_clean":"2023-08-27T11:17:06.607988+0000","last_became_active":"2023-08-27T11:17:04.377919+0000","last_became_peered":"2023-08-27T11:17:04.377919+0000","last_unstale":"2023-08-27T11:17:06.607988+0000","last_undegraded":"2023-08-27T11:17:06.607988+0000","last_fullsized":"2023-08-27T11:17:06.607988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:53.143119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202657+0000","last_change":"2023-08-27T11:17:05.383385+0000","last_active":"2023-08-27T11:17:07.202657+0000","last_peered":"2023-08-27T11:17:07.202657+0000","last_clean":"2023-08-27T11:17:07.202657+0000","last_became_active":"2023-08-27T11:17:05.382932+0000","last_became_peered":"2023-08-27T11:17:05.382932+0000","last_unstale":"2023-08-27T11:17:07.202657+0000","last_undegraded":"2023-08-27T11:17:07.202657+0000","last_fullsized":"2023-08-27T11:17:07.202657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202798+0000","last_change":"2023-08-27T11:17:04.373528+0000","last_active":"2023-08-27T11:17:07.202798+0000","last_peered":"2023-08-27T11:17:07.202798+0000","last_clean":"2023-08-27T11:17:07.202798+0000","last_became_active":"2023-08-27T11:17:04.373266+0000","last_became_peered":"2023-08-27T11:17:04.373266+0000","last_unstale":"2023-08-27T11:17:07.202798+0000","last_undegraded":"2023-08-27T11:17:07.202798+0000","last_fullsized":"2023-08-27T11:17:07.202798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:14.186836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202805+0000","last_change":"2023-08-27T11:17:05.383919+0000","last_active":"2023-08-27T11:17:07.202805+0000","last_peered":"2023-08-27T11:17:07.202805+0000","last_clean":"2023-08-27T11:17:07.202805+0000","last_became_active":"2023-08-27T11:17:05.383319+0000","last_became_peered":"2023-08-27T11:17:05.383319+0000","last_unstale":"2023-08-27T11:17:07.202805+0000","last_undegraded":"2023-08-27T11:17:07.202805+0000","last_fullsized":"2023-08-27T11:17:07.202805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:42.812467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203294+0000","last_change":"2023-08-27T11:17:04.379878+0000","last_active":"2023-08-27T11:17:07.203294+0000","last_peered":"2023-08-27T11:17:07.203294+0000","last_clean":"2023-08-27T11:17:07.203294+0000","last_became_active":"2023-08-27T11:17:04.379586+0000","last_became_peered":"2023-08-27T11:17:04.379586+0000","last_unstale":"2023-08-27T11:17:07.203294+0000","last_undegraded":"2023-08-27T11:17:07.203294+0000","last_fullsized":"2023-08-27T11:17:07.203294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:02.601133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367611+0000","last_change":"2023-08-27T11:17:05.385687+0000","last_active":"2023-08-27T11:17:06.367611+0000","last_peered":"2023-08-27T11:17:06.367611+0000","last_clean":"2023-08-27T11:17:06.367611+0000","last_became_active":"2023-08-27T11:17:05.385422+0000","last_became_peered":"2023-08-27T11:17:05.385422+0000","last_unstale":"2023-08-27T11:17:06.367611+0000","last_undegraded":"2023-08-27T11:17:06.367611+0000","last_fullsized":"2023-08-27T11:17:06.367611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:31.508513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369381+0000","last_change":"2023-08-27T11:17:04.372137+0000","last_active":"2023-08-27T11:17:06.369381+0000","last_peered":"2023-08-27T11:17:06.369381+0000","last_clean":"2023-08-27T11:17:06.369381+0000","last_became_active":"2023-08-27T11:17:04.371246+0000","last_became_peered":"2023-08-27T11:17:04.371246+0000","last_unstale":"2023-08-27T11:17:06.369381+0000","last_undegraded":"2023-08-27T11:17:06.369381+0000","last_fullsized":"2023-08-27T11:17:06.369381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:27:15.596770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369513+0000","last_change":"2023-08-27T11:17:05.375967+0000","last_active":"2023-08-27T11:17:06.369513+0000","last_peered":"2023-08-27T11:17:06.369513+0000","last_clean":"2023-08-27T11:17:06.369513+0000","last_became_active":"2023-08-27T11:17:05.375704+0000","last_became_peered":"2023-08-27T11:17:05.375704+0000","last_unstale":"2023-08-27T11:17:06.369513+0000","last_undegraded":"2023-08-27T11:17:06.369513+0000","last_fullsized":"2023-08-27T11:17:06.369513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:55:02.014817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202803+0000","last_change":"2023-08-27T11:17:04.374228+0000","last_active":"2023-08-27T11:17:07.202803+0000","last_peered":"2023-08-27T11:17:07.202803+0000","last_clean":"2023-08-27T11:17:07.202803+0000","last_became_active":"2023-08-27T11:17:04.373855+0000","last_became_peered":"2023-08-27T11:17:04.373855+0000","last_unstale":"2023-08-27T11:17:07.202803+0000","last_undegraded":"2023-08-27T11:17:07.202803+0000","last_fullsized":"2023-08-27T11:17:07.202803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:38.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202908+0000","last_change":"2023-08-27T11:17:05.382048+0000","last_active":"2023-08-27T11:17:07.202908+0000","last_peered":"2023-08-27T11:17:07.202908+0000","last_clean":"2023-08-27T11:17:07.202908+0000","last_became_active":"2023-08-27T11:17:05.381804+0000","last_became_peered":"2023-08-27T11:17:05.381804+0000","last_unstale":"2023-08-27T11:17:07.202908+0000","last_undegraded":"2023-08-27T11:17:07.202908+0000","last_fullsized":"2023-08-27T11:17:07.202908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:55.731983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607767+0000","last_change":"2023-08-27T11:17:04.376663+0000","last_active":"2023-08-27T11:17:06.607767+0000","last_peered":"2023-08-27T11:17:06.607767+0000","last_clean":"2023-08-27T11:17:06.607767+0000","last_became_active":"2023-08-27T11:17:04.376347+0000","last_became_peered":"2023-08-27T11:17:04.376347+0000","last_unstale":"2023-08-27T11:17:06.607767+0000","last_undegraded":"2023-08-27T11:17:06.607767+0000","last_fullsized":"2023-08-27T11:17:06.607767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:14.186836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203533+0000","last_change":"2023-08-27T11:17:05.382733+0000","last_active":"2023-08-27T11:17:07.203533+0000","last_peered":"2023-08-27T11:17:07.203533+0000","last_clean":"2023-08-27T11:17:07.203533+0000","last_became_active":"2023-08-27T11:17:05.382358+0000","last_became_peered":"2023-08-27T11:17:05.382358+0000","last_unstale":"2023-08-27T11:17:07.203533+0000","last_undegraded":"2023-08-27T11:17:07.203533+0000","last_fullsized":"2023-08-27T11:17:07.203533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:42.812467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203143+0000","last_change":"2023-08-27T11:17:04.372997+0000","last_active":"2023-08-27T11:17:07.203143+0000","last_peered":"2023-08-27T11:17:07.203143+0000","last_clean":"2023-08-27T11:17:07.203143+0000","last_became_active":"2023-08-27T11:17:04.372719+0000","last_became_peered":"2023-08-27T11:17:04.372719+0000","last_unstale":"2023-08-27T11:17:07.203143+0000","last_undegraded":"2023-08-27T11:17:07.203143+0000","last_fullsized":"2023-08-27T11:17:07.203143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:54.726470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607805+0000","last_change":"2023-08-27T11:17:05.379397+0000","last_active":"2023-08-27T11:17:06.607805+0000","last_peered":"2023-08-27T11:17:06.607805+0000","last_clean":"2023-08-27T11:17:06.607805+0000","last_became_active":"2023-08-27T11:17:05.377874+0000","last_became_peered":"2023-08-27T11:17:05.377874+0000","last_unstale":"2023-08-27T11:17:06.607805+0000","last_undegraded":"2023-08-27T11:17:06.607805+0000","last_fullsized":"2023-08-27T11:17:06.607805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368437+0000","last_change":"2023-08-27T11:17:04.376949+0000","last_active":"2023-08-27T11:17:06.368437+0000","last_peered":"2023-08-27T11:17:06.368437+0000","last_clean":"2023-08-27T11:17:06.368437+0000","last_became_active":"2023-08-27T11:17:04.376645+0000","last_became_peered":"2023-08-27T11:17:04.376645+0000","last_unstale":"2023-08-27T11:17:06.368437+0000","last_undegraded":"2023-08-27T11:17:06.368437+0000","last_fullsized":"2023-08-27T11:17:06.368437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:30.937446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203216+0000","last_change":"2023-08-27T11:17:05.378949+0000","last_active":"2023-08-27T11:17:07.203216+0000","last_peered":"2023-08-27T11:17:07.203216+0000","last_clean":"2023-08-27T11:17:07.203216+0000","last_became_active":"2023-08-27T11:17:05.378649+0000","last_became_peered":"2023-08-27T11:17:05.378649+0000","last_unstale":"2023-08-27T11:17:07.203216+0000","last_undegraded":"2023-08-27T11:17:07.203216+0000","last_fullsized":"2023-08-27T11:17:07.203216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:43:40.338423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203379+0000","last_change":"2023-08-27T11:17:04.374131+0000","last_active":"2023-08-27T11:17:07.203379+0000","last_peered":"2023-08-27T11:17:07.203379+0000","last_clean":"2023-08-27T11:17:07.203379+0000","last_became_active":"2023-08-27T11:17:04.373809+0000","last_became_peered":"2023-08-27T11:17:04.373809+0000","last_unstale":"2023-08-27T11:17:07.203379+0000","last_undegraded":"2023-08-27T11:17:07.203379+0000","last_fullsized":"2023-08-27T11:17:07.203379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:43:39.332910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202567+0000","last_change":"2023-08-27T11:17:05.379390+0000","last_active":"2023-08-27T11:17:07.202567+0000","last_peered":"2023-08-27T11:17:07.202567+0000","last_clean":"2023-08-27T11:17:07.202567+0000","last_became_active":"2023-08-27T11:17:05.379131+0000","last_became_peered":"2023-08-27T11:17:05.379131+0000","last_unstale":"2023-08-27T11:17:07.202567+0000","last_undegraded":"2023-08-27T11:17:07.202567+0000","last_fullsized":"2023-08-27T11:17:07.202567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:28.711930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203264+0000","last_change":"2023-08-27T11:17:04.374694+0000","last_active":"2023-08-27T11:17:07.203264+0000","last_peered":"2023-08-27T11:17:07.203264+0000","last_clean":"2023-08-27T11:17:07.203264+0000","last_became_active":"2023-08-27T11:17:04.373836+0000","last_became_peered":"2023-08-27T11:17:04.373836+0000","last_unstale":"2023-08-27T11:17:07.203264+0000","last_undegraded":"2023-08-27T11:17:07.203264+0000","last_fullsized":"2023-08-27T11:17:07.203264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:44.131038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368972+0000","last_change":"2023-08-27T11:17:05.380858+0000","last_active":"2023-08-27T11:17:06.368972+0000","last_peered":"2023-08-27T11:17:06.368972+0000","last_clean":"2023-08-27T11:17:06.368972+0000","last_became_active":"2023-08-27T11:17:05.380573+0000","last_became_peered":"2023-08-27T11:17:05.380573+0000","last_unstale":"2023-08-27T11:17:06.368972+0000","last_undegraded":"2023-08-27T11:17:06.368972+0000","last_fullsized":"2023-08-27T11:17:06.368972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:42.999614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203215+0000","last_change":"2023-08-27T11:17:04.374901+0000","last_active":"2023-08-27T11:17:07.203215+0000","last_peered":"2023-08-27T11:17:07.203215+0000","last_clean":"2023-08-27T11:17:07.203215+0000","last_became_active":"2023-08-27T11:17:04.374186+0000","last_became_peered":"2023-08-27T11:17:04.374186+0000","last_unstale":"2023-08-27T11:17:07.203215+0000","last_undegraded":"2023-08-27T11:17:07.203215+0000","last_fullsized":"2023-08-27T11:17:07.203215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:47.391583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369664+0000","last_change":"2023-08-27T11:17:05.380802+0000","last_active":"2023-08-27T11:17:06.369664+0000","last_peered":"2023-08-27T11:17:06.369664+0000","last_clean":"2023-08-27T11:17:06.369664+0000","last_became_active":"2023-08-27T11:17:05.380537+0000","last_became_peered":"2023-08-27T11:17:05.380537+0000","last_unstale":"2023-08-27T11:17:06.369664+0000","last_undegraded":"2023-08-27T11:17:06.369664+0000","last_fullsized":"2023-08-27T11:17:06.369664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203202+0000","last_change":"2023-08-27T11:17:04.379623+0000","last_active":"2023-08-27T11:17:07.203202+0000","last_peered":"2023-08-27T11:17:07.203202+0000","last_clean":"2023-08-27T11:17:07.203202+0000","last_became_active":"2023-08-27T11:17:04.379342+0000","last_became_peered":"2023-08-27T11:17:04.379342+0000","last_unstale":"2023-08-27T11:17:07.203202+0000","last_undegraded":"2023-08-27T11:17:07.203202+0000","last_fullsized":"2023-08-27T11:17:07.203202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:38.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607824+0000","last_change":"2023-08-27T11:17:05.375956+0000","last_active":"2023-08-27T11:17:06.607824+0000","last_peered":"2023-08-27T11:17:06.607824+0000","last_clean":"2023-08-27T11:17:06.607824+0000","last_became_active":"2023-08-27T11:17:05.375040+0000","last_became_peered":"2023-08-27T11:17:05.375040+0000","last_unstale":"2023-08-27T11:17:06.607824+0000","last_undegraded":"2023-08-27T11:17:06.607824+0000","last_fullsized":"2023-08-27T11:17:06.607824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202873+0000","last_change":"2023-08-27T11:17:04.379633+0000","last_active":"2023-08-27T11:17:07.202873+0000","last_peered":"2023-08-27T11:17:07.202873+0000","last_clean":"2023-08-27T11:17:07.202873+0000","last_became_active":"2023-08-27T11:17:04.379306+0000","last_became_peered":"2023-08-27T11:17:04.379306+0000","last_unstale":"2023-08-27T11:17:07.202873+0000","last_undegraded":"2023-08-27T11:17:07.202873+0000","last_fullsized":"2023-08-27T11:17:07.202873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:53.143119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369174+0000","last_change":"2023-08-27T11:17:05.374205+0000","last_active":"2023-08-27T11:17:06.369174+0000","last_peered":"2023-08-27T11:17:06.369174+0000","last_clean":"2023-08-27T11:17:06.369174+0000","last_became_active":"2023-08-27T11:17:05.373675+0000","last_became_peered":"2023-08-27T11:17:05.373675+0000","last_unstale":"2023-08-27T11:17:06.369174+0000","last_undegraded":"2023-08-27T11:17:06.369174+0000","last_fullsized":"2023-08-27T11:17:06.369174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:43:40.338423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369599+0000","last_change":"2023-08-27T11:17:04.371348+0000","last_active":"2023-08-27T11:17:06.369599+0000","last_peered":"2023-08-27T11:17:06.369599+0000","last_clean":"2023-08-27T11:17:06.369599+0000","last_became_active":"2023-08-27T11:17:04.370929+0000","last_became_peered":"2023-08-27T11:17:04.370929+0000","last_unstale":"2023-08-27T11:17:06.369599+0000","last_undegraded":"2023-08-27T11:17:06.369599+0000","last_fullsized":"2023-08-27T11:17:06.369599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:27.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369296+0000","last_change":"2023-08-27T11:17:05.373711+0000","last_active":"2023-08-27T11:17:06.369296+0000","last_peered":"2023-08-27T11:17:06.369296+0000","last_clean":"2023-08-27T11:17:06.369296+0000","last_became_active":"2023-08-27T11:17:05.373392+0000","last_became_peered":"2023-08-27T11:17:05.373392+0000","last_unstale":"2023-08-27T11:17:06.369296+0000","last_undegraded":"2023-08-27T11:17:06.369296+0000","last_fullsized":"2023-08-27T11:17:06.369296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:35.735781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369479+0000","last_change":"2023-08-27T11:17:04.373259+0000","last_active":"2023-08-27T11:17:06.369479+0000","last_peered":"2023-08-27T11:17:06.369479+0000","last_clean":"2023-08-27T11:17:06.369479+0000","last_became_active":"2023-08-27T11:17:04.372983+0000","last_became_peered":"2023-08-27T11:17:04.372983+0000","last_unstale":"2023-08-27T11:17:06.369479+0000","last_undegraded":"2023-08-27T11:17:06.369479+0000","last_fullsized":"2023-08-27T11:17:06.369479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:36.570705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369051+0000","last_change":"2023-08-27T11:17:05.376729+0000","last_active":"2023-08-27T11:17:06.369051+0000","last_peered":"2023-08-27T11:17:06.369051+0000","last_clean":"2023-08-27T11:17:06.369051+0000","last_became_active":"2023-08-27T11:17:05.376443+0000","last_became_peered":"2023-08-27T11:17:05.376443+0000","last_unstale":"2023-08-27T11:17:06.369051+0000","last_undegraded":"2023-08-27T11:17:06.369051+0000","last_fullsized":"2023-08-27T11:17:06.369051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:48.397096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202559+0000","last_change":"2023-08-27T11:17:04.377051+0000","last_active":"2023-08-27T11:17:07.202559+0000","last_peered":"2023-08-27T11:17:07.202559+0000","last_clean":"2023-08-27T11:17:07.202559+0000","last_became_active":"2023-08-27T11:17:04.376700+0000","last_became_peered":"2023-08-27T11:17:04.376700+0000","last_unstale":"2023-08-27T11:17:07.202559+0000","last_undegraded":"2023-08-27T11:17:07.202559+0000","last_fullsized":"2023-08-27T11:17:07.202559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:42.307697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367824+0000","last_change":"2023-08-27T11:17:05.383341+0000","last_active":"2023-08-27T11:17:06.367824+0000","last_peered":"2023-08-27T11:17:06.367824+0000","last_clean":"2023-08-27T11:17:06.367824+0000","last_became_active":"2023-08-27T11:17:05.379929+0000","last_became_peered":"2023-08-27T11:17:05.379929+0000","last_unstale":"2023-08-27T11:17:06.367824+0000","last_undegraded":"2023-08-27T11:17:06.367824+0000","last_fullsized":"2023-08-27T11:17:06.367824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203147+0000","last_change":"2023-08-27T11:17:04.378623+0000","last_active":"2023-08-27T11:17:07.203147+0000","last_peered":"2023-08-27T11:17:07.203147+0000","last_clean":"2023-08-27T11:17:07.203147+0000","last_became_active":"2023-08-27T11:17:04.378327+0000","last_became_peered":"2023-08-27T11:17:04.378327+0000","last_unstale":"2023-08-27T11:17:07.203147+0000","last_undegraded":"2023-08-27T11:17:07.203147+0000","last_fullsized":"2023-08-27T11:17:07.203147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:02.601133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368983+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:17:06.368983+0000","last_peered":"2023-08-27T11:17:06.368983+0000","last_clean":"2023-08-27T11:17:06.368983+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:17:06.368983+0000","last_undegraded":"2023-08-27T11:17:06.368983+0000","last_fullsized":"2023-08-27T11:17:06.368983+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:07.481452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202508+0000","last_change":"2023-08-27T11:17:05.383215+0000","last_active":"2023-08-27T11:17:07.202508+0000","last_peered":"2023-08-27T11:17:07.202508+0000","last_clean":"2023-08-27T11:17:07.202508+0000","last_became_active":"2023-08-27T11:17:05.382502+0000","last_became_peered":"2023-08-27T11:17:05.382502+0000","last_unstale":"2023-08-27T11:17:07.202508+0000","last_undegraded":"2023-08-27T11:17:07.202508+0000","last_fullsized":"2023-08-27T11:17:07.202508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203019+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:17:07.203019+0000","last_peered":"2023-08-27T11:17:07.203019+0000","last_clean":"2023-08-27T11:17:07.203019+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:17:07.203019+0000","last_undegraded":"2023-08-27T11:17:07.203019+0000","last_fullsized":"2023-08-27T11:17:07.203019+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:42.350470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607950+0000","last_change":"2023-08-27T11:17:04.380486+0000","last_active":"2023-08-27T11:17:06.607950+0000","last_peered":"2023-08-27T11:17:06.607950+0000","last_clean":"2023-08-27T11:17:06.607950+0000","last_became_active":"2023-08-27T11:17:04.376679+0000","last_became_peered":"2023-08-27T11:17:04.376679+0000","last_unstale":"2023-08-27T11:17:06.607950+0000","last_undegraded":"2023-08-27T11:17:06.607950+0000","last_fullsized":"2023-08-27T11:17:06.607950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:41.806953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369152+0000","last_change":"2023-08-27T11:17:05.370457+0000","last_active":"2023-08-27T11:17:06.369152+0000","last_peered":"2023-08-27T11:17:06.369152+0000","last_clean":"2023-08-27T11:17:06.369152+0000","last_became_active":"2023-08-27T11:17:05.370040+0000","last_became_peered":"2023-08-27T11:17:05.370040+0000","last_unstale":"2023-08-27T11:17:06.369152+0000","last_undegraded":"2023-08-27T11:17:06.369152+0000","last_fullsized":"2023-08-27T11:17:06.369152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:03.606646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367481+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:17:06.367481+0000","last_peered":"2023-08-27T11:17:06.367481+0000","last_clean":"2023-08-27T11:17:06.367481+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:17:06.367481+0000","last_undegraded":"2023-08-27T11:17:06.367481+0000","last_fullsized":"2023-08-27T11:17:06.367481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:07:15.067358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203119+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:17:07.203119+0000","last_peered":"2023-08-27T11:17:07.203119+0000","last_clean":"2023-08-27T11:17:07.203119+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:17:07.203119+0000","last_undegraded":"2023-08-27T11:17:07.203119+0000","last_fullsized":"2023-08-27T11:17:07.203119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:36.952988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607599+0000","last_change":"2023-08-27T11:17:04.373430+0000","last_active":"2023-08-27T11:17:06.607599+0000","last_peered":"2023-08-27T11:17:06.607599+0000","last_clean":"2023-08-27T11:17:06.607599+0000","last_became_active":"2023-08-27T11:17:04.373098+0000","last_became_peered":"2023-08-27T11:17:04.373098+0000","last_unstale":"2023-08-27T11:17:06.607599+0000","last_undegraded":"2023-08-27T11:17:06.607599+0000","last_fullsized":"2023-08-27T11:17:06.607599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:07:20.108470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607829+0000","last_change":"2023-08-27T11:17:05.380096+0000","last_active":"2023-08-27T11:17:06.607829+0000","last_peered":"2023-08-27T11:17:06.607829+0000","last_clean":"2023-08-27T11:17:06.607829+0000","last_became_active":"2023-08-27T11:17:05.379846+0000","last_became_peered":"2023-08-27T11:17:05.379846+0000","last_unstale":"2023-08-27T11:17:06.607829+0000","last_undegraded":"2023-08-27T11:17:06.607829+0000","last_fullsized":"2023-08-27T11:17:06.607829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202806+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:17:07.202806+0000","last_peered":"2023-08-27T11:17:07.202806+0000","last_clean":"2023-08-27T11:17:07.202806+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:17:07.202806+0000","last_undegraded":"2023-08-27T11:17:07.202806+0000","last_fullsized":"2023-08-27T11:17:07.202806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:48.102006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367401+0000","last_change":"2023-08-27T11:17:04.373552+0000","last_active":"2023-08-27T11:17:06.367401+0000","last_peered":"2023-08-27T11:17:06.367401+0000","last_clean":"2023-08-27T11:17:06.367401+0000","last_became_active":"2023-08-27T11:17:04.373258+0000","last_became_peered":"2023-08-27T11:17:04.373258+0000","last_unstale":"2023-08-27T11:17:06.367401+0000","last_undegraded":"2023-08-27T11:17:06.367401+0000","last_fullsized":"2023-08-27T11:17:06.367401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:30.937446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367737+0000","last_change":"2023-08-27T11:17:05.383636+0000","last_active":"2023-08-27T11:17:06.367737+0000","last_peered":"2023-08-27T11:17:06.367737+0000","last_clean":"2023-08-27T11:17:06.367737+0000","last_became_active":"2023-08-27T11:17:05.383410+0000","last_became_peered":"2023-08-27T11:17:05.383410+0000","last_unstale":"2023-08-27T11:17:06.367737+0000","last_undegraded":"2023-08-27T11:17:06.367737+0000","last_fullsized":"2023-08-27T11:17:06.367737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:43.313210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202834+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:17:07.202834+0000","last_peered":"2023-08-27T11:17:07.202834+0000","last_clean":"2023-08-27T11:17:07.202834+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:17:07.202834+0000","last_undegraded":"2023-08-27T11:17:07.202834+0000","last_fullsized":"2023-08-27T11:17:07.202834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:36.765841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202793+0000","last_change":"2023-08-27T11:17:04.375838+0000","last_active":"2023-08-27T11:17:07.202793+0000","last_peered":"2023-08-27T11:17:07.202793+0000","last_clean":"2023-08-27T11:17:07.202793+0000","last_became_active":"2023-08-27T11:17:04.375536+0000","last_became_peered":"2023-08-27T11:17:04.375536+0000","last_unstale":"2023-08-27T11:17:07.202793+0000","last_undegraded":"2023-08-27T11:17:07.202793+0000","last_fullsized":"2023-08-27T11:17:07.202793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:53.957455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369664+0000","last_change":"2023-08-27T11:17:05.375110+0000","last_active":"2023-08-27T11:17:06.369664+0000","last_peered":"2023-08-27T11:17:06.369664+0000","last_clean":"2023-08-27T11:17:06.369664+0000","last_became_active":"2023-08-27T11:17:05.374844+0000","last_became_peered":"2023-08-27T11:17:05.374844+0000","last_unstale":"2023-08-27T11:17:06.369664+0000","last_undegraded":"2023-08-27T11:17:06.369664+0000","last_fullsized":"2023-08-27T11:17:06.369664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:15.192349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202453+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:17:07.202453+0000","last_peered":"2023-08-27T11:17:07.202453+0000","last_clean":"2023-08-27T11:17:07.202453+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:17:07.202453+0000","last_undegraded":"2023-08-27T11:17:07.202453+0000","last_fullsized":"2023-08-27T11:17:07.202453+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:43.315205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368949+0000","last_change":"2023-08-27T11:17:04.379053+0000","last_active":"2023-08-27T11:17:06.368949+0000","last_peered":"2023-08-27T11:17:06.368949+0000","last_clean":"2023-08-27T11:17:06.368949+0000","last_became_active":"2023-08-27T11:17:04.378731+0000","last_became_peered":"2023-08-27T11:17:04.378731+0000","last_unstale":"2023-08-27T11:17:06.368949+0000","last_undegraded":"2023-08-27T11:17:06.368949+0000","last_fullsized":"2023-08-27T11:17:06.368949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:53.143119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367664+0000","last_change":"2023-08-27T11:17:05.373510+0000","last_active":"2023-08-27T11:17:06.367664+0000","last_peered":"2023-08-27T11:17:06.367664+0000","last_clean":"2023-08-27T11:17:06.367664+0000","last_became_active":"2023-08-27T11:17:05.373229+0000","last_became_peered":"2023-08-27T11:17:05.373229+0000","last_unstale":"2023-08-27T11:17:06.367664+0000","last_undegraded":"2023-08-27T11:17:06.367664+0000","last_fullsized":"2023-08-27T11:17:06.367664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369394+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:17:06.369394+0000","last_peered":"2023-08-27T11:17:06.369394+0000","last_clean":"2023-08-27T11:17:06.369394+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:17:06.369394+0000","last_undegraded":"2023-08-27T11:17:06.369394+0000","last_fullsized":"2023-08-27T11:17:06.369394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:07:15.067358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367592+0000","last_change":"2023-08-27T11:17:04.374506+0000","last_active":"2023-08-27T11:17:06.367592+0000","last_peered":"2023-08-27T11:17:06.367592+0000","last_clean":"2023-08-27T11:17:06.367592+0000","last_became_active":"2023-08-27T11:17:04.374209+0000","last_became_peered":"2023-08-27T11:17:04.374209+0000","last_unstale":"2023-08-27T11:17:06.367592+0000","last_undegraded":"2023-08-27T11:17:06.367592+0000","last_fullsized":"2023-08-27T11:17:06.367592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:27.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607676+0000","last_change":"2023-08-27T11:17:05.369565+0000","last_active":"2023-08-27T11:17:06.607676+0000","last_peered":"2023-08-27T11:17:06.607676+0000","last_clean":"2023-08-27T11:17:06.607676+0000","last_became_active":"2023-08-27T11:17:05.369334+0000","last_became_peered":"2023-08-27T11:17:05.369334+0000","last_unstale":"2023-08-27T11:17:06.607676+0000","last_undegraded":"2023-08-27T11:17:06.607676+0000","last_fullsized":"2023-08-27T11:17:06.607676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369316+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:17:06.369316+0000","last_peered":"2023-08-27T11:17:06.369316+0000","last_clean":"2023-08-27T11:17:06.369316+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:17:06.369316+0000","last_undegraded":"2023-08-27T11:17:06.369316+0000","last_fullsized":"2023-08-27T11:17:06.369316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:25.461887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203500+0000","last_change":"2023-08-27T11:17:04.380000+0000","last_active":"2023-08-27T11:17:07.203500+0000","last_peered":"2023-08-27T11:17:07.203500+0000","last_clean":"2023-08-27T11:17:07.203500+0000","last_became_active":"2023-08-27T11:17:04.379666+0000","last_became_peered":"2023-08-27T11:17:04.379666+0000","last_unstale":"2023-08-27T11:17:07.203500+0000","last_undegraded":"2023-08-27T11:17:07.203500+0000","last_fullsized":"2023-08-27T11:17:07.203500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:53.957455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367721+0000","last_change":"2023-08-27T11:17:05.380929+0000","last_active":"2023-08-27T11:17:06.367721+0000","last_peered":"2023-08-27T11:17:06.367721+0000","last_clean":"2023-08-27T11:17:06.367721+0000","last_became_active":"2023-08-27T11:17:05.380573+0000","last_became_peered":"2023-08-27T11:17:05.380573+0000","last_unstale":"2023-08-27T11:17:06.367721+0000","last_undegraded":"2023-08-27T11:17:06.367721+0000","last_fullsized":"2023-08-27T11:17:06.367721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607806+0000","last_change":"2023-08-27T11:17:04.377322+0000","last_active":"2023-08-27T11:17:06.607806+0000","last_peered":"2023-08-27T11:17:06.607806+0000","last_clean":"2023-08-27T11:17:06.607806+0000","last_became_active":"2023-08-27T11:17:04.376967+0000","last_became_peered":"2023-08-27T11:17:04.376967+0000","last_unstale":"2023-08-27T11:17:06.607806+0000","last_undegraded":"2023-08-27T11:17:06.607806+0000","last_fullsized":"2023-08-27T11:17:06.607806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:42.307697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190107+0000","last_change":"2023-08-27T11:17:05.382640+0000","last_active":"2023-08-27T11:17:07.190107+0000","last_peered":"2023-08-27T11:17:07.190107+0000","last_clean":"2023-08-27T11:17:07.190107+0000","last_became_active":"2023-08-27T11:17:05.382377+0000","last_became_peered":"2023-08-27T11:17:05.382377+0000","last_unstale":"2023-08-27T11:17:07.190107+0000","last_undegraded":"2023-08-27T11:17:07.190107+0000","last_fullsized":"2023-08-27T11:17:07.190107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:27:57.016677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203339+0000","last_change":"2023-08-27T11:17:04.372997+0000","last_active":"2023-08-27T11:17:07.203339+0000","last_peered":"2023-08-27T11:17:07.203339+0000","last_clean":"2023-08-27T11:17:07.203339+0000","last_became_active":"2023-08-27T11:17:04.372719+0000","last_became_peered":"2023-08-27T11:17:04.372719+0000","last_unstale":"2023-08-27T11:17:07.203339+0000","last_undegraded":"2023-08-27T11:17:07.203339+0000","last_fullsized":"2023-08-27T11:17:07.203339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:51.710890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368989+0000","last_change":"2023-08-27T11:17:05.374254+0000","last_active":"2023-08-27T11:17:06.368989+0000","last_peered":"2023-08-27T11:17:06.368989+0000","last_clean":"2023-08-27T11:17:06.368989+0000","last_became_active":"2023-08-27T11:17:05.373877+0000","last_became_peered":"2023-08-27T11:17:05.373877+0000","last_unstale":"2023-08-27T11:17:06.368989+0000","last_undegraded":"2023-08-27T11:17:06.368989+0000","last_fullsized":"2023-08-27T11:17:06.368989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:42.812467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203587+0000","last_change":"2023-08-27T11:17:04.372930+0000","last_active":"2023-08-27T11:17:07.203587+0000","last_peered":"2023-08-27T11:17:07.203587+0000","last_clean":"2023-08-27T11:17:07.203587+0000","last_became_active":"2023-08-27T11:17:04.372626+0000","last_became_peered":"2023-08-27T11:17:04.372626+0000","last_unstale":"2023-08-27T11:17:07.203587+0000","last_undegraded":"2023-08-27T11:17:07.203587+0000","last_fullsized":"2023-08-27T11:17:07.203587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:45.738091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369549+0000","last_change":"2023-08-27T11:17:05.374125+0000","last_active":"2023-08-27T11:17:06.369549+0000","last_peered":"2023-08-27T11:17:06.369549+0000","last_clean":"2023-08-27T11:17:06.369549+0000","last_became_active":"2023-08-27T11:17:05.373856+0000","last_became_peered":"2023-08-27T11:17:05.373856+0000","last_unstale":"2023-08-27T11:17:06.369549+0000","last_undegraded":"2023-08-27T11:17:06.369549+0000","last_fullsized":"2023-08-27T11:17:06.369549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:17:32.516797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203038+0000","last_change":"2023-08-27T11:17:04.375965+0000","last_active":"2023-08-27T11:17:07.203038+0000","last_peered":"2023-08-27T11:17:07.203038+0000","last_clean":"2023-08-27T11:17:07.203038+0000","last_became_active":"2023-08-27T11:17:04.375687+0000","last_became_peered":"2023-08-27T11:17:04.375687+0000","last_unstale":"2023-08-27T11:17:07.203038+0000","last_undegraded":"2023-08-27T11:17:07.203038+0000","last_fullsized":"2023-08-27T11:17:07.203038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:12.522565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202911+0000","last_change":"2023-08-27T11:17:05.372749+0000","last_active":"2023-08-27T11:17:07.202911+0000","last_peered":"2023-08-27T11:17:07.202911+0000","last_clean":"2023-08-27T11:17:07.202911+0000","last_became_active":"2023-08-27T11:17:05.372494+0000","last_became_peered":"2023-08-27T11:17:05.372494+0000","last_unstale":"2023-08-27T11:17:07.202911+0000","last_undegraded":"2023-08-27T11:17:07.202911+0000","last_fullsized":"2023-08-27T11:17:07.202911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:42.999614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607966+0000","last_change":"2023-08-27T11:17:04.374287+0000","last_active":"2023-08-27T11:17:06.607966+0000","last_peered":"2023-08-27T11:17:06.607966+0000","last_clean":"2023-08-27T11:17:06.607966+0000","last_became_active":"2023-08-27T11:17:04.374009+0000","last_became_peered":"2023-08-27T11:17:04.374009+0000","last_unstale":"2023-08-27T11:17:06.607966+0000","last_undegraded":"2023-08-27T11:17:06.607966+0000","last_fullsized":"2023-08-27T11:17:06.607966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:45.738091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368606+0000","last_change":"2023-08-27T11:17:05.369490+0000","last_active":"2023-08-27T11:17:06.368606+0000","last_peered":"2023-08-27T11:17:06.368606+0000","last_clean":"2023-08-27T11:17:06.368606+0000","last_became_active":"2023-08-27T11:17:05.369193+0000","last_became_peered":"2023-08-27T11:17:05.369193+0000","last_unstale":"2023-08-27T11:17:06.368606+0000","last_undegraded":"2023-08-27T11:17:06.368606+0000","last_fullsized":"2023-08-27T11:17:06.368606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368508+0000","last_change":"2023-08-27T11:17:05.376280+0000","last_active":"2023-08-27T11:17:06.368508+0000","last_peered":"2023-08-27T11:17:06.368508+0000","last_clean":"2023-08-27T11:17:06.368508+0000","last_became_active":"2023-08-27T11:17:05.375763+0000","last_became_peered":"2023-08-27T11:17:05.375763+0000","last_unstale":"2023-08-27T11:17:06.368508+0000","last_undegraded":"2023-08-27T11:17:06.368508+0000","last_fullsized":"2023-08-27T11:17:06.368508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:46.814570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202731+0000","last_change":"2023-08-27T11:17:05.384097+0000","last_active":"2023-08-27T11:17:07.202731+0000","last_peered":"2023-08-27T11:17:07.202731+0000","last_clean":"2023-08-27T11:17:07.202731+0000","last_became_active":"2023-08-27T11:17:05.383673+0000","last_became_peered":"2023-08-27T11:17:05.383673+0000","last_unstale":"2023-08-27T11:17:07.202731+0000","last_undegraded":"2023-08-27T11:17:07.202731+0000","last_fullsized":"2023-08-27T11:17:07.202731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368848+0000","last_change":"2023-08-27T11:17:05.369440+0000","last_active":"2023-08-27T11:17:06.368848+0000","last_peered":"2023-08-27T11:17:06.368848+0000","last_clean":"2023-08-27T11:17:06.368848+0000","last_became_active":"2023-08-27T11:17:05.369113+0000","last_became_peered":"2023-08-27T11:17:05.369113+0000","last_unstale":"2023-08-27T11:17:06.368848+0000","last_undegraded":"2023-08-27T11:17:06.368848+0000","last_fullsized":"2023-08-27T11:17:06.368848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:39.209301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202857+0000","last_change":"2023-08-27T11:17:05.381671+0000","last_active":"2023-08-27T11:17:07.202857+0000","last_peered":"2023-08-27T11:17:07.202857+0000","last_clean":"2023-08-27T11:17:07.202857+0000","last_became_active":"2023-08-27T11:17:05.380933+0000","last_became_peered":"2023-08-27T11:17:05.380933+0000","last_unstale":"2023-08-27T11:17:07.202857+0000","last_undegraded":"2023-08-27T11:17:07.202857+0000","last_fullsized":"2023-08-27T11:17:07.202857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:46.743604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367534+0000","last_change":"2023-08-27T11:17:05.379944+0000","last_active":"2023-08-27T11:17:06.367534+0000","last_peered":"2023-08-27T11:17:06.367534+0000","last_clean":"2023-08-27T11:17:06.367534+0000","last_became_active":"2023-08-27T11:17:05.379633+0000","last_became_peered":"2023-08-27T11:17:05.379633+0000","last_unstale":"2023-08-27T11:17:06.367534+0000","last_undegraded":"2023-08-27T11:17:06.367534+0000","last_fullsized":"2023-08-27T11:17:06.367534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":659971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":74428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":786432,"data_stored":658143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":73649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":67775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":76096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":200704,"data_stored":68427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:17:09.462 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:17:09.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.610+0000 7f11a4fe8700 1 Processor -- start 2023-08-27T11:17:09.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.610+0000 7f11a4fe8700 1 -- start start 2023-08-27T11:17:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.611+0000 7f11a4fe8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f11a00b1400 0x7f11a00cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.611+0000 7f11a4fe8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11a00b17e0 0x7f11a006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.611+0000 7f11a4fe8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f11a0161ce0 0x7f11a006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.611+0000 7f11a4fe8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a0161850 con 0x7f11a00b1400 2023-08-27T11:17:09.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.611+0000 7f11a4fe8700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a006e7d0 con 0x7f11a0161ce0 2023-08-27T11:17:09.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.611+0000 7f11a4fe8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00cfc10 con 0x7f11a00b17e0 2023-08-27T11:17:09.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119e59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f11a00b1400 0x7f11a00cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119e59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f11a00b1400 0x7f11a00cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54744/0 (socket says 172.21.15.112:54744) 2023-08-27T11:17:09.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119e59c700 1 -- 172.21.15.112:0/2425083247 learned_addr learned my addr 172.21.15.112:0/2425083247 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:09.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119dd9b700 1 --2- 172.21.15.112:0/2425083247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11a00b17e0 0x7f11a006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119ed9d700 1 --2- 172.21.15.112:0/2425083247 >> v2:172.21.15.138:3300/0 conn(0x7f11a0161ce0 0x7f11a006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119e59c700 1 -- 172.21.15.112:0/2425083247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11a00b17e0 msgr2=0x7f11a006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:09.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119e59c700 1 --2- 172.21.15.112:0/2425083247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11a00b17e0 0x7f11a006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119e59c700 1 -- 172.21.15.112:0/2425083247 >> v2:172.21.15.138:3300/0 conn(0x7f11a0161ce0 msgr2=0x7f11a006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:09.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119e59c700 1 --2- 172.21.15.112:0/2425083247 >> v2:172.21.15.138:3300/0 conn(0x7f11a0161ce0 0x7f11a006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119e59c700 1 -- 172.21.15.112:0/2425083247 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a0161200 con 0x7f11a00b1400 2023-08-27T11:17:09.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119e59c700 1 --2- 172.21.15.112:0/2425083247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a00b1400 0x7f11a00cf5a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f11a00ca550 tx=0x7f1194009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9aca7d0e60359b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:09.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119d59a700 1 -- 172.21.15.112:0/2425083247 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119400b870 con 0x7f11a00b1400 2023-08-27T11:17:09.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119d59a700 1 -- 172.21.15.112:0/2425083247 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119400b9d0 con 0x7f11a00b1400 2023-08-27T11:17:09.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.612+0000 7f119d59a700 1 -- 172.21.15.112:0/2425083247 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119400bd60 con 0x7f11a00b1400 2023-08-27T11:17:09.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.613+0000 7f11a4fe8700 1 -- 172.21.15.112:0/2425083247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a00b1400 msgr2=0x7f11a00cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.613+0000 7f11a4fe8700 1 --2- 172.21.15.112:0/2425083247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a00b1400 0x7f11a00cf5a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f11a00ca550 tx=0x7f1194009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.613+0000 7f11a4fe8700 1 -- 172.21.15.112:0/2425083247 shutdown_connections 2023-08-27T11:17:09.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.613+0000 7f11a4fe8700 1 --2- 172.21.15.112:0/2425083247 >> v2:172.21.15.138:3300/0 conn(0x7f11a0161ce0 0x7f11a006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.613+0000 7f11a4fe8700 1 --2- 172.21.15.112:0/2425083247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a00b1400 0x7f11a00cf5a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.613+0000 7f11a4fe8700 1 --2- 172.21.15.112:0/2425083247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11a00b17e0 0x7f11a006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.613+0000 7f11a4fe8700 1 -- 172.21.15.112:0/2425083247 >> 172.21.15.112:0/2425083247 conn(0x7f11a00d0000 msgr2=0x7f11a00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:09.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.613+0000 7f11a4fe8700 1 -- 172.21.15.112:0/2425083247 shutdown_connections 2023-08-27T11:17:09.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.613+0000 7f11a4fe8700 1 -- 172.21.15.112:0/2425083247 wait complete. 2023-08-27T11:17:09.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.614+0000 7f11a4fe8700 1 Processor -- start 2023-08-27T11:17:09.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.614+0000 7f11a4fe8700 1 -- start start 2023-08-27T11:17:09.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f11a4fe8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a00b1400 0x7f11a00c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f11a4fe8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a00b17e0 0x7f11a00c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f11a4fe8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11a0161ce0 0x7f11a00c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f11a4fe8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00c73c0 con 0x7f11a00b17e0 2023-08-27T11:17:09.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f11a4fe8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00ae0b0 con 0x7f11a00b1400 2023-08-27T11:17:09.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f11a4fe8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00ca000 con 0x7f11a0161ce0 2023-08-27T11:17:09.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f119dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a00b17e0 0x7f11a00c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f119dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a00b17e0 0x7f11a00c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54750/0 (socket says 172.21.15.112:54750) 2023-08-27T11:17:09.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f119dd9b700 1 -- 172.21.15.112:0/3576806703 learned_addr learned my addr 172.21.15.112:0/3576806703 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:09.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f119e59c700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a00b1400 0x7f11a00c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f119ed9d700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11a0161ce0 0x7f11a00c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f119dd9b700 1 -- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11a0161ce0 msgr2=0x7f11a00c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f119dd9b700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11a0161ce0 0x7f11a00c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f119dd9b700 1 -- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a00b1400 msgr2=0x7f11a00c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f119dd9b700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a00b1400 0x7f11a00c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f119dd9b700 1 -- 172.21.15.112:0/3576806703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1194009550 con 0x7f11a00b17e0 2023-08-27T11:17:09.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.615+0000 7f119dd9b700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a00b17e0 0x7f11a00c88d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f119000e810 tx=0x7f119000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:09.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.616+0000 7f118effd700 1 -- 172.21.15.112:0/3576806703 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119000c070 con 0x7f11a00b17e0 2023-08-27T11:17:09.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.616+0000 7f118effd700 1 -- 172.21.15.112:0/3576806703 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1190013610 con 0x7f11a00b17e0 2023-08-27T11:17:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.616+0000 7f11a4fe8700 1 -- 172.21.15.112:0/3576806703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a00c75d0 con 0x7f11a00b17e0 2023-08-27T11:17:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.616+0000 7f118effd700 1 -- 172.21.15.112:0/3576806703 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11900139a0 con 0x7f11a00b17e0 2023-08-27T11:17:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.616+0000 7f11a4fe8700 1 -- 172.21.15.112:0/3576806703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a00c7b90 con 0x7f11a00b17e0 2023-08-27T11:17:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.617+0000 7f11a4fe8700 1 -- 172.21.15.112:0/3576806703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a0067a20 con 0x7f11a00b17e0 2023-08-27T11:17:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.617+0000 7f118effd700 1 -- 172.21.15.112:0/3576806703 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1190013b00 con 0x7f11a00b17e0 2023-08-27T11:17:09.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.617+0000 7f118effd700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f118006a0e0 0x7f118006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.617+0000 7f118effd700 1 -- 172.21.15.112:0/3576806703 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1190088200 con 0x7f11a00b17e0 2023-08-27T11:17:09.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.618+0000 7f119e59c700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f118006a0e0 0x7f118006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.618+0000 7f119e59c700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f118006a0e0 0x7f118006c5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1194000c00 tx=0x7f1194008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:09.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.622+0000 7f118effd700 1 -- 172.21.15.112:0/3576806703 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1190053980 con 0x7f11a00b17e0 2023-08-27T11:17:09.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.805+0000 7f11a4fe8700 1 -- 172.21.15.112:0/3576806703 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f11a01614e0 con 0x7f118006a0e0 2023-08-27T11:17:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.814+0000 7f118effd700 1 -- 172.21.15.112:0/3576806703 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287020 (secure 0 0 0) 0x7f119400bb40 con 0x7f118006a0e0 2023-08-27T11:17:09.818 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.817+0000 7f11a4fe8700 1 -- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f118006a0e0 msgr2=0x7f118006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.817+0000 7f11a4fe8700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f118006a0e0 0x7f118006c5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1194000c00 tx=0x7f1194008030 comp rx=0 tx=0).stop 2023-08-27T11:17:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.817+0000 7f11a4fe8700 1 -- 172.21.15.112:0/3576806703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a00b17e0 msgr2=0x7f11a00c88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.817+0000 7f11a4fe8700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a00b17e0 0x7f11a00c88d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f119000e810 tx=0x7f119000eb20 comp rx=0 tx=0).stop 2023-08-27T11:17:09.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.817+0000 7f11a4fe8700 1 -- 172.21.15.112:0/3576806703 shutdown_connections 2023-08-27T11:17:09.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.817+0000 7f11a4fe8700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f118006a0e0 0x7f118006c5a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.817+0000 7f11a4fe8700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11a0161ce0 0x7f11a00c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.817+0000 7f11a4fe8700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a00b1400 0x7f11a00c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.817+0000 7f11a4fe8700 1 --2- 172.21.15.112:0/3576806703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a00b17e0 0x7f11a00c88d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.817+0000 7f11a4fe8700 1 -- 172.21.15.112:0/3576806703 >> 172.21.15.112:0/3576806703 conn(0x7f11a00d0000 msgr2=0x7f11a00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:09.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.817+0000 7f11a4fe8700 1 -- 172.21.15.112:0/3576806703 shutdown_connections 2023-08-27T11:17:09.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.817+0000 7f11a4fe8700 1 -- 172.21.15.112:0/3576806703 wait complete. 2023-08-27T11:17:09.823 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:17:09.830 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-08-27T11:17:08.224670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":198,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218980,"kb_used_data":2772,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755740,"statfs":{"total":773094113280,"available":772869877760,"internally_reserved":0,"allocated":2838528,"data_stored":1753007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.872765"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190109+0000","last_change":"2023-08-27T11:17:05.373064+0000","last_active":"2023-08-27T11:17:07.190109+0000","last_peered":"2023-08-27T11:17:07.190109+0000","last_clean":"2023-08-27T11:17:07.190109+0000","last_became_active":"2023-08-27T11:17:05.372771+0000","last_became_peered":"2023-08-27T11:17:05.372771+0000","last_unstale":"2023-08-27T11:17:07.190109+0000","last_undegraded":"2023-08-27T11:17:07.190109+0000","last_fullsized":"2023-08-27T11:17:07.190109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:12:06.488460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202664+0000","last_change":"2023-08-27T11:17:05.380926+0000","last_active":"2023-08-27T11:17:07.202664+0000","last_peered":"2023-08-27T11:17:07.202664+0000","last_clean":"2023-08-27T11:17:07.202664+0000","last_became_active":"2023-08-27T11:17:05.380623+0000","last_became_peered":"2023-08-27T11:17:05.380623+0000","last_unstale":"2023-08-27T11:17:07.202664+0000","last_undegraded":"2023-08-27T11:17:07.202664+0000","last_fullsized":"2023-08-27T11:17:07.202664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368869+0000","last_change":"2023-08-27T11:17:05.377335+0000","last_active":"2023-08-27T11:17:06.368869+0000","last_peered":"2023-08-27T11:17:06.368869+0000","last_clean":"2023-08-27T11:17:06.368869+0000","last_became_active":"2023-08-27T11:17:05.376963+0000","last_became_peered":"2023-08-27T11:17:05.376963+0000","last_unstale":"2023-08-27T11:17:06.368869+0000","last_undegraded":"2023-08-27T11:17:06.368869+0000","last_fullsized":"2023-08-27T11:17:06.368869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:54.962968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369836+0000","last_change":"2023-08-27T11:17:05.371708+0000","last_active":"2023-08-27T11:17:06.369836+0000","last_peered":"2023-08-27T11:17:06.369836+0000","last_clean":"2023-08-27T11:17:06.369836+0000","last_became_active":"2023-08-27T11:17:05.371385+0000","last_became_peered":"2023-08-27T11:17:05.371385+0000","last_unstale":"2023-08-27T11:17:06.369836+0000","last_undegraded":"2023-08-27T11:17:06.369836+0000","last_fullsized":"2023-08-27T11:17:06.369836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607644+0000","last_change":"2023-08-27T11:17:05.377806+0000","last_active":"2023-08-27T11:17:06.607644+0000","last_peered":"2023-08-27T11:17:06.607644+0000","last_clean":"2023-08-27T11:17:06.607644+0000","last_became_active":"2023-08-27T11:17:05.377546+0000","last_became_peered":"2023-08-27T11:17:05.377546+0000","last_unstale":"2023-08-27T11:17:06.607644+0000","last_undegraded":"2023-08-27T11:17:06.607644+0000","last_fullsized":"2023-08-27T11:17:06.607644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:28.711930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369347+0000","last_change":"2023-08-27T11:17:05.371766+0000","last_active":"2023-08-27T11:17:06.369347+0000","last_peered":"2023-08-27T11:17:06.369347+0000","last_clean":"2023-08-27T11:17:06.369347+0000","last_became_active":"2023-08-27T11:17:05.370938+0000","last_became_peered":"2023-08-27T11:17:05.370938+0000","last_unstale":"2023-08-27T11:17:06.369347+0000","last_undegraded":"2023-08-27T11:17:06.369347+0000","last_fullsized":"2023-08-27T11:17:06.369347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:55.731983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203009+0000","last_change":"2023-08-27T11:17:05.383079+0000","last_active":"2023-08-27T11:17:07.203009+0000","last_peered":"2023-08-27T11:17:07.203009+0000","last_clean":"2023-08-27T11:17:07.203009+0000","last_became_active":"2023-08-27T11:17:05.382810+0000","last_became_peered":"2023-08-27T11:17:05.382810+0000","last_unstale":"2023-08-27T11:17:07.203009+0000","last_undegraded":"2023-08-27T11:17:07.203009+0000","last_fullsized":"2023-08-27T11:17:07.203009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:46.743604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202512+0000","last_change":"2023-08-27T11:17:05.376722+0000","last_active":"2023-08-27T11:17:07.202512+0000","last_peered":"2023-08-27T11:17:07.202512+0000","last_clean":"2023-08-27T11:17:07.202512+0000","last_became_active":"2023-08-27T11:17:05.376399+0000","last_became_peered":"2023-08-27T11:17:05.376399+0000","last_unstale":"2023-08-27T11:17:07.202512+0000","last_undegraded":"2023-08-27T11:17:07.202512+0000","last_fullsized":"2023-08-27T11:17:07.202512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607858+0000","last_change":"2023-08-27T11:17:05.371306+0000","last_active":"2023-08-27T11:17:06.607858+0000","last_peered":"2023-08-27T11:17:06.607858+0000","last_clean":"2023-08-27T11:17:06.607858+0000","last_became_active":"2023-08-27T11:17:05.370962+0000","last_became_peered":"2023-08-27T11:17:05.370962+0000","last_unstale":"2023-08-27T11:17:06.607858+0000","last_undegraded":"2023-08-27T11:17:06.607858+0000","last_fullsized":"2023-08-27T11:17:06.607858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:39.209301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202986+0000","last_change":"2023-08-27T11:17:05.372994+0000","last_active":"2023-08-27T11:17:07.202986+0000","last_peered":"2023-08-27T11:17:07.202986+0000","last_clean":"2023-08-27T11:17:07.202986+0000","last_became_active":"2023-08-27T11:17:05.372278+0000","last_became_peered":"2023-08-27T11:17:05.372278+0000","last_unstale":"2023-08-27T11:17:07.202986+0000","last_undegraded":"2023-08-27T11:17:07.202986+0000","last_fullsized":"2023-08-27T11:17:07.202986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369607+0000","last_change":"2023-08-27T11:17:05.376929+0000","last_active":"2023-08-27T11:17:06.369607+0000","last_peered":"2023-08-27T11:17:06.369607+0000","last_clean":"2023-08-27T11:17:06.369607+0000","last_became_active":"2023-08-27T11:17:05.376592+0000","last_became_peered":"2023-08-27T11:17:05.376592+0000","last_unstale":"2023-08-27T11:17:06.369607+0000","last_undegraded":"2023-08-27T11:17:06.369607+0000","last_fullsized":"2023-08-27T11:17:06.369607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:24:05.167634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190152+0000","last_change":"2023-08-27T11:17:05.377608+0000","last_active":"2023-08-27T11:17:07.190152+0000","last_peered":"2023-08-27T11:17:07.190152+0000","last_clean":"2023-08-27T11:17:07.190152+0000","last_became_active":"2023-08-27T11:17:05.376987+0000","last_became_peered":"2023-08-27T11:17:05.376987+0000","last_unstale":"2023-08-27T11:17:07.190152+0000","last_undegraded":"2023-08-27T11:17:07.190152+0000","last_fullsized":"2023-08-27T11:17:07.190152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:30:54.430091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.189991+0000","last_change":"2023-08-27T11:17:05.377602+0000","last_active":"2023-08-27T11:17:07.189991+0000","last_peered":"2023-08-27T11:17:07.189991+0000","last_clean":"2023-08-27T11:17:07.189991+0000","last_became_active":"2023-08-27T11:17:05.377023+0000","last_became_peered":"2023-08-27T11:17:05.377023+0000","last_unstale":"2023-08-27T11:17:07.189991+0000","last_undegraded":"2023-08-27T11:17:07.189991+0000","last_fullsized":"2023-08-27T11:17:07.189991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:18:00.607553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202965+0000","last_change":"2023-08-27T11:17:05.376491+0000","last_active":"2023-08-27T11:17:07.202965+0000","last_peered":"2023-08-27T11:17:07.202965+0000","last_clean":"2023-08-27T11:17:07.202965+0000","last_became_active":"2023-08-27T11:17:05.376149+0000","last_became_peered":"2023-08-27T11:17:05.376149+0000","last_unstale":"2023-08-27T11:17:07.202965+0000","last_undegraded":"2023-08-27T11:17:07.202965+0000","last_fullsized":"2023-08-27T11:17:07.202965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:43.313210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203191+0000","last_change":"2023-08-27T11:17:05.387318+0000","last_active":"2023-08-27T11:17:07.203191+0000","last_peered":"2023-08-27T11:17:07.203191+0000","last_clean":"2023-08-27T11:17:07.203191+0000","last_became_active":"2023-08-27T11:17:05.387002+0000","last_became_peered":"2023-08-27T11:17:05.387002+0000","last_unstale":"2023-08-27T11:17:07.203191+0000","last_undegraded":"2023-08-27T11:17:07.203191+0000","last_fullsized":"2023-08-27T11:17:07.203191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:13.528078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368880+0000","last_change":"2023-08-27T11:17:05.372919+0000","last_active":"2023-08-27T11:17:06.368880+0000","last_peered":"2023-08-27T11:17:06.368880+0000","last_clean":"2023-08-27T11:17:06.368880+0000","last_became_active":"2023-08-27T11:17:05.371149+0000","last_became_peered":"2023-08-27T11:17:05.371149+0000","last_unstale":"2023-08-27T11:17:06.368880+0000","last_undegraded":"2023-08-27T11:17:06.368880+0000","last_fullsized":"2023-08-27T11:17:06.368880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:46.743604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607880+0000","last_change":"2023-08-27T11:17:05.371456+0000","last_active":"2023-08-27T11:17:06.607880+0000","last_peered":"2023-08-27T11:17:06.607880+0000","last_clean":"2023-08-27T11:17:06.607880+0000","last_became_active":"2023-08-27T11:17:05.371193+0000","last_became_peered":"2023-08-27T11:17:05.371193+0000","last_unstale":"2023-08-27T11:17:06.607880+0000","last_undegraded":"2023-08-27T11:17:06.607880+0000","last_fullsized":"2023-08-27T11:17:06.607880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369084+0000","last_change":"2023-08-27T11:17:05.373957+0000","last_active":"2023-08-27T11:17:06.369084+0000","last_peered":"2023-08-27T11:17:06.369084+0000","last_clean":"2023-08-27T11:17:06.369084+0000","last_became_active":"2023-08-27T11:17:05.373507+0000","last_became_peered":"2023-08-27T11:17:05.373507+0000","last_unstale":"2023-08-27T11:17:06.369084+0000","last_undegraded":"2023-08-27T11:17:06.369084+0000","last_fullsized":"2023-08-27T11:17:06.369084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:45.136551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190029+0000","last_change":"2023-08-27T11:17:05.378874+0000","last_active":"2023-08-27T11:17:07.190029+0000","last_peered":"2023-08-27T11:17:07.190029+0000","last_clean":"2023-08-27T11:17:07.190029+0000","last_became_active":"2023-08-27T11:17:05.378585+0000","last_became_peered":"2023-08-27T11:17:05.378585+0000","last_unstale":"2023-08-27T11:17:07.190029+0000","last_undegraded":"2023-08-27T11:17:07.190029+0000","last_fullsized":"2023-08-27T11:17:07.190029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:19:23.730989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368860+0000","last_change":"2023-08-27T11:17:05.377656+0000","last_active":"2023-08-27T11:17:06.368860+0000","last_peered":"2023-08-27T11:17:06.368860+0000","last_clean":"2023-08-27T11:17:06.368860+0000","last_became_active":"2023-08-27T11:17:05.377366+0000","last_became_peered":"2023-08-27T11:17:05.377366+0000","last_unstale":"2023-08-27T11:17:06.368860+0000","last_undegraded":"2023-08-27T11:17:06.368860+0000","last_fullsized":"2023-08-27T11:17:06.368860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202861+0000","last_change":"2023-08-27T11:17:05.379425+0000","last_active":"2023-08-27T11:17:07.202861+0000","last_peered":"2023-08-27T11:17:07.202861+0000","last_clean":"2023-08-27T11:17:07.202861+0000","last_became_active":"2023-08-27T11:17:05.379084+0000","last_became_peered":"2023-08-27T11:17:05.379084+0000","last_unstale":"2023-08-27T11:17:07.202861+0000","last_undegraded":"2023-08-27T11:17:07.202861+0000","last_fullsized":"2023-08-27T11:17:07.202861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607602+0000","last_change":"2023-08-27T11:17:05.381421+0000","last_active":"2023-08-27T11:17:06.607602+0000","last_peered":"2023-08-27T11:17:06.607602+0000","last_clean":"2023-08-27T11:17:06.607602+0000","last_became_active":"2023-08-27T11:17:05.380452+0000","last_became_peered":"2023-08-27T11:17:05.380452+0000","last_unstale":"2023-08-27T11:17:06.607602+0000","last_undegraded":"2023-08-27T11:17:06.607602+0000","last_fullsized":"2023-08-27T11:17:06.607602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202657+0000","last_change":"2023-08-27T11:17:05.378132+0000","last_active":"2023-08-27T11:17:07.202657+0000","last_peered":"2023-08-27T11:17:07.202657+0000","last_clean":"2023-08-27T11:17:07.202657+0000","last_became_active":"2023-08-27T11:17:05.376190+0000","last_became_peered":"2023-08-27T11:17:05.376190+0000","last_unstale":"2023-08-27T11:17:07.202657+0000","last_undegraded":"2023-08-27T11:17:07.202657+0000","last_fullsized":"2023-08-27T11:17:07.202657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203004+0000","last_change":"2023-08-27T11:17:05.378821+0000","last_active":"2023-08-27T11:17:07.203004+0000","last_peered":"2023-08-27T11:17:07.203004+0000","last_clean":"2023-08-27T11:17:07.203004+0000","last_became_active":"2023-08-27T11:17:05.378520+0000","last_became_peered":"2023-08-27T11:17:05.378520+0000","last_unstale":"2023-08-27T11:17:07.203004+0000","last_undegraded":"2023-08-27T11:17:07.203004+0000","last_fullsized":"2023-08-27T11:17:07.203004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:54.962968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367698+0000","last_change":"2023-08-27T11:17:05.379672+0000","last_active":"2023-08-27T11:17:06.367698+0000","last_peered":"2023-08-27T11:17:06.367698+0000","last_clean":"2023-08-27T11:17:06.367698+0000","last_became_active":"2023-08-27T11:17:05.379399+0000","last_became_peered":"2023-08-27T11:17:05.379399+0000","last_unstale":"2023-08-27T11:17:06.367698+0000","last_undegraded":"2023-08-27T11:17:06.367698+0000","last_fullsized":"2023-08-27T11:17:06.367698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:15.192349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367765+0000","last_change":"2023-08-27T11:17:05.376638+0000","last_active":"2023-08-27T11:17:06.367765+0000","last_peered":"2023-08-27T11:17:06.367765+0000","last_clean":"2023-08-27T11:17:06.367765+0000","last_became_active":"2023-08-27T11:17:05.376274+0000","last_became_peered":"2023-08-27T11:17:05.376274+0000","last_unstale":"2023-08-27T11:17:06.367765+0000","last_undegraded":"2023-08-27T11:17:06.367765+0000","last_fullsized":"2023-08-27T11:17:06.367765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202811+0000","last_change":"2023-08-27T11:17:05.382137+0000","last_active":"2023-08-27T11:17:07.202811+0000","last_peered":"2023-08-27T11:17:07.202811+0000","last_clean":"2023-08-27T11:17:07.202811+0000","last_became_active":"2023-08-27T11:17:05.381664+0000","last_became_peered":"2023-08-27T11:17:05.381664+0000","last_unstale":"2023-08-27T11:17:07.202811+0000","last_undegraded":"2023-08-27T11:17:07.202811+0000","last_fullsized":"2023-08-27T11:17:07.202811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369199+0000","last_change":"2023-08-27T11:17:04.376604+0000","last_active":"2023-08-27T11:17:06.369199+0000","last_peered":"2023-08-27T11:17:06.369199+0000","last_clean":"2023-08-27T11:17:06.369199+0000","last_became_active":"2023-08-27T11:17:04.376288+0000","last_became_peered":"2023-08-27T11:17:04.376288+0000","last_unstale":"2023-08-27T11:17:06.369199+0000","last_undegraded":"2023-08-27T11:17:06.369199+0000","last_fullsized":"2023-08-27T11:17:06.369199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:08:24.626675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203128+0000","last_change":"2023-08-27T11:17:05.382097+0000","last_active":"2023-08-27T11:17:07.203128+0000","last_peered":"2023-08-27T11:17:07.203128+0000","last_clean":"2023-08-27T11:17:07.203128+0000","last_became_active":"2023-08-27T11:17:05.381829+0000","last_became_peered":"2023-08-27T11:17:05.381829+0000","last_unstale":"2023-08-27T11:17:07.203128+0000","last_undegraded":"2023-08-27T11:17:07.203128+0000","last_fullsized":"2023-08-27T11:17:07.203128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:45.136551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367598+0000","last_change":"2023-08-27T11:17:04.379398+0000","last_active":"2023-08-27T11:17:06.367598+0000","last_peered":"2023-08-27T11:17:06.367598+0000","last_clean":"2023-08-27T11:17:06.367598+0000","last_became_active":"2023-08-27T11:17:04.379109+0000","last_became_peered":"2023-08-27T11:17:04.379109+0000","last_unstale":"2023-08-27T11:17:06.367598+0000","last_undegraded":"2023-08-27T11:17:06.367598+0000","last_fullsized":"2023-08-27T11:17:06.367598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:45.809057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607933+0000","last_change":"2023-08-27T11:17:05.379671+0000","last_active":"2023-08-27T11:17:06.607933+0000","last_peered":"2023-08-27T11:17:06.607933+0000","last_clean":"2023-08-27T11:17:06.607933+0000","last_became_active":"2023-08-27T11:17:05.379358+0000","last_became_peered":"2023-08-27T11:17:05.379358+0000","last_unstale":"2023-08-27T11:17:06.607933+0000","last_undegraded":"2023-08-27T11:17:06.607933+0000","last_fullsized":"2023-08-27T11:17:06.607933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:54.962968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202719+0000","last_change":"2023-08-27T11:17:04.377528+0000","last_active":"2023-08-27T11:17:07.202719+0000","last_peered":"2023-08-27T11:17:07.202719+0000","last_clean":"2023-08-27T11:17:07.202719+0000","last_became_active":"2023-08-27T11:17:04.377219+0000","last_became_peered":"2023-08-27T11:17:04.377219+0000","last_unstale":"2023-08-27T11:17:07.202719+0000","last_undegraded":"2023-08-27T11:17:07.202719+0000","last_fullsized":"2023-08-27T11:17:07.202719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:38.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369530+0000","last_change":"2023-08-27T11:17:05.372053+0000","last_active":"2023-08-27T11:17:06.369530+0000","last_peered":"2023-08-27T11:17:06.369530+0000","last_clean":"2023-08-27T11:17:06.369530+0000","last_became_active":"2023-08-27T11:17:05.371769+0000","last_became_peered":"2023-08-27T11:17:05.371769+0000","last_unstale":"2023-08-27T11:17:06.369530+0000","last_undegraded":"2023-08-27T11:17:06.369530+0000","last_fullsized":"2023-08-27T11:17:06.369530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607988+0000","last_change":"2023-08-27T11:17:04.378264+0000","last_active":"2023-08-27T11:17:06.607988+0000","last_peered":"2023-08-27T11:17:06.607988+0000","last_clean":"2023-08-27T11:17:06.607988+0000","last_became_active":"2023-08-27T11:17:04.377919+0000","last_became_peered":"2023-08-27T11:17:04.377919+0000","last_unstale":"2023-08-27T11:17:06.607988+0000","last_undegraded":"2023-08-27T11:17:06.607988+0000","last_fullsized":"2023-08-27T11:17:06.607988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:53.143119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202657+0000","last_change":"2023-08-27T11:17:05.383385+0000","last_active":"2023-08-27T11:17:07.202657+0000","last_peered":"2023-08-27T11:17:07.202657+0000","last_clean":"2023-08-27T11:17:07.202657+0000","last_became_active":"2023-08-27T11:17:05.382932+0000","last_became_peered":"2023-08-27T11:17:05.382932+0000","last_unstale":"2023-08-27T11:17:07.202657+0000","last_undegraded":"2023-08-27T11:17:07.202657+0000","last_fullsized":"2023-08-27T11:17:07.202657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202798+0000","last_change":"2023-08-27T11:17:04.373528+0000","last_active":"2023-08-27T11:17:07.202798+0000","last_peered":"2023-08-27T11:17:07.202798+0000","last_clean":"2023-08-27T11:17:07.202798+0000","last_became_active":"2023-08-27T11:17:04.373266+0000","last_became_peered":"2023-08-27T11:17:04.373266+0000","last_unstale":"2023-08-27T11:17:07.202798+0000","last_undegraded":"2023-08-27T11:17:07.202798+0000","last_fullsized":"2023-08-27T11:17:07.202798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:14.186836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202805+0000","last_change":"2023-08-27T11:17:05.383919+0000","last_active":"2023-08-27T11:17:07.202805+0000","last_peered":"2023-08-27T11:17:07.202805+0000","last_clean":"2023-08-27T11:17:07.202805+0000","last_became_active":"2023-08-27T11:17:05.383319+0000","last_became_peered":"2023-08-27T11:17:05.383319+0000","last_unstale":"2023-08-27T11:17:07.202805+0000","last_undegraded":"2023-08-27T11:17:07.202805+0000","last_fullsized":"2023-08-27T11:17:07.202805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:42.812467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203294+0000","last_change":"2023-08-27T11:17:04.379878+0000","last_active":"2023-08-27T11:17:07.203294+0000","last_peered":"2023-08-27T11:17:07.203294+0000","last_clean":"2023-08-27T11:17:07.203294+0000","last_became_active":"2023-08-27T11:17:04.379586+0000","last_became_peered":"2023-08-27T11:17:04.379586+0000","last_unstale":"2023-08-27T11:17:07.203294+0000","last_undegraded":"2023-08-27T11:17:07.203294+0000","last_fullsized":"2023-08-27T11:17:07.203294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:02.601133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367611+0000","last_change":"2023-08-27T11:17:05.385687+0000","last_active":"2023-08-27T11:17:06.367611+0000","last_peered":"2023-08-27T11:17:06.367611+0000","last_clean":"2023-08-27T11:17:06.367611+0000","last_became_active":"2023-08-27T11:17:05.385422+0000","last_became_peered":"2023-08-27T11:17:05.385422+0000","last_unstale":"2023-08-27T11:17:06.367611+0000","last_undegraded":"2023-08-27T11:17:06.367611+0000","last_fullsized":"2023-08-27T11:17:06.367611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:31.508513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369381+0000","last_change":"2023-08-27T11:17:04.372137+0000","last_active":"2023-08-27T11:17:06.369381+0000","last_peered":"2023-08-27T11:17:06.369381+0000","last_clean":"2023-08-27T11:17:06.369381+0000","last_became_active":"2023-08-27T11:17:04.371246+0000","last_became_peered":"2023-08-27T11:17:04.371246+0000","last_unstale":"2023-08-27T11:17:06.369381+0000","last_undegraded":"2023-08-27T11:17:06.369381+0000","last_fullsized":"2023-08-27T11:17:06.369381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:27:15.596770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369513+0000","last_change":"2023-08-27T11:17:05.375967+0000","last_active":"2023-08-27T11:17:06.369513+0000","last_peered":"2023-08-27T11:17:06.369513+0000","last_clean":"2023-08-27T11:17:06.369513+0000","last_became_active":"2023-08-27T11:17:05.375704+0000","last_became_peered":"2023-08-27T11:17:05.375704+0000","last_unstale":"2023-08-27T11:17:06.369513+0000","last_undegraded":"2023-08-27T11:17:06.369513+0000","last_fullsized":"2023-08-27T11:17:06.369513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:55:02.014817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202803+0000","last_change":"2023-08-27T11:17:04.374228+0000","last_active":"2023-08-27T11:17:07.202803+0000","last_peered":"2023-08-27T11:17:07.202803+0000","last_clean":"2023-08-27T11:17:07.202803+0000","last_became_active":"2023-08-27T11:17:04.373855+0000","last_became_peered":"2023-08-27T11:17:04.373855+0000","last_unstale":"2023-08-27T11:17:07.202803+0000","last_undegraded":"2023-08-27T11:17:07.202803+0000","last_fullsized":"2023-08-27T11:17:07.202803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:38.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202908+0000","last_change":"2023-08-27T11:17:05.382048+0000","last_active":"2023-08-27T11:17:07.202908+0000","last_peered":"2023-08-27T11:17:07.202908+0000","last_clean":"2023-08-27T11:17:07.202908+0000","last_became_active":"2023-08-27T11:17:05.381804+0000","last_became_peered":"2023-08-27T11:17:05.381804+0000","last_unstale":"2023-08-27T11:17:07.202908+0000","last_undegraded":"2023-08-27T11:17:07.202908+0000","last_fullsized":"2023-08-27T11:17:07.202908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:55.731983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607767+0000","last_change":"2023-08-27T11:17:04.376663+0000","last_active":"2023-08-27T11:17:06.607767+0000","last_peered":"2023-08-27T11:17:06.607767+0000","last_clean":"2023-08-27T11:17:06.607767+0000","last_became_active":"2023-08-27T11:17:04.376347+0000","last_became_peered":"2023-08-27T11:17:04.376347+0000","last_unstale":"2023-08-27T11:17:06.607767+0000","last_undegraded":"2023-08-27T11:17:06.607767+0000","last_fullsized":"2023-08-27T11:17:06.607767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:14.186836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203533+0000","last_change":"2023-08-27T11:17:05.382733+0000","last_active":"2023-08-27T11:17:07.203533+0000","last_peered":"2023-08-27T11:17:07.203533+0000","last_clean":"2023-08-27T11:17:07.203533+0000","last_became_active":"2023-08-27T11:17:05.382358+0000","last_became_peered":"2023-08-27T11:17:05.382358+0000","last_unstale":"2023-08-27T11:17:07.203533+0000","last_undegraded":"2023-08-27T11:17:07.203533+0000","last_fullsized":"2023-08-27T11:17:07.203533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:42.812467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203143+0000","last_change":"2023-08-27T11:17:04.372997+0000","last_active":"2023-08-27T11:17:07.203143+0000","last_peered":"2023-08-27T11:17:07.203143+0000","last_clean":"2023-08-27T11:17:07.203143+0000","last_became_active":"2023-08-27T11:17:04.372719+0000","last_became_peered":"2023-08-27T11:17:04.372719+0000","last_unstale":"2023-08-27T11:17:07.203143+0000","last_undegraded":"2023-08-27T11:17:07.203143+0000","last_fullsized":"2023-08-27T11:17:07.203143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:54.726470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607805+0000","last_change":"2023-08-27T11:17:05.379397+0000","last_active":"2023-08-27T11:17:06.607805+0000","last_peered":"2023-08-27T11:17:06.607805+0000","last_clean":"2023-08-27T11:17:06.607805+0000","last_became_active":"2023-08-27T11:17:05.377874+0000","last_became_peered":"2023-08-27T11:17:05.377874+0000","last_unstale":"2023-08-27T11:17:06.607805+0000","last_undegraded":"2023-08-27T11:17:06.607805+0000","last_fullsized":"2023-08-27T11:17:06.607805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368437+0000","last_change":"2023-08-27T11:17:04.376949+0000","last_active":"2023-08-27T11:17:06.368437+0000","last_peered":"2023-08-27T11:17:06.368437+0000","last_clean":"2023-08-27T11:17:06.368437+0000","last_became_active":"2023-08-27T11:17:04.376645+0000","last_became_peered":"2023-08-27T11:17:04.376645+0000","last_unstale":"2023-08-27T11:17:06.368437+0000","last_undegraded":"2023-08-27T11:17:06.368437+0000","last_fullsized":"2023-08-27T11:17:06.368437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:30.937446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203216+0000","last_change":"2023-08-27T11:17:05.378949+0000","last_active":"2023-08-27T11:17:07.203216+0000","last_peered":"2023-08-27T11:17:07.203216+0000","last_clean":"2023-08-27T11:17:07.203216+0000","last_became_active":"2023-08-27T11:17:05.378649+0000","last_became_peered":"2023-08-27T11:17:05.378649+0000","last_unstale":"2023-08-27T11:17:07.203216+0000","last_undegraded":"2023-08-27T11:17:07.203216+0000","last_fullsized":"2023-08-27T11:17:07.203216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:43:40.338423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203379+0000","last_change":"2023-08-27T11:17:04.374131+0000","last_active":"2023-08-27T11:17:07.203379+0000","last_peered":"2023-08-27T11:17:07.203379+0000","last_clean":"2023-08-27T11:17:07.203379+0000","last_became_active":"2023-08-27T11:17:04.373809+0000","last_became_peered":"2023-08-27T11:17:04.373809+0000","last_unstale":"2023-08-27T11:17:07.203379+0000","last_undegraded":"2023-08-27T11:17:07.203379+0000","last_fullsized":"2023-08-27T11:17:07.203379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:43:39.332910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202567+0000","last_change":"2023-08-27T11:17:05.379390+0000","last_active":"2023-08-27T11:17:07.202567+0000","last_peered":"2023-08-27T11:17:07.202567+0000","last_clean":"2023-08-27T11:17:07.202567+0000","last_became_active":"2023-08-27T11:17:05.379131+0000","last_became_peered":"2023-08-27T11:17:05.379131+0000","last_unstale":"2023-08-27T11:17:07.202567+0000","last_undegraded":"2023-08-27T11:17:07.202567+0000","last_fullsized":"2023-08-27T11:17:07.202567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:28.711930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203264+0000","last_change":"2023-08-27T11:17:04.374694+0000","last_active":"2023-08-27T11:17:07.203264+0000","last_peered":"2023-08-27T11:17:07.203264+0000","last_clean":"2023-08-27T11:17:07.203264+0000","last_became_active":"2023-08-27T11:17:04.373836+0000","last_became_peered":"2023-08-27T11:17:04.373836+0000","last_unstale":"2023-08-27T11:17:07.203264+0000","last_undegraded":"2023-08-27T11:17:07.203264+0000","last_fullsized":"2023-08-27T11:17:07.203264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:44.131038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368972+0000","last_change":"2023-08-27T11:17:05.380858+0000","last_active":"2023-08-27T11:17:06.368972+0000","last_peered":"2023-08-27T11:17:06.368972+0000","last_clean":"2023-08-27T11:17:06.368972+0000","last_became_active":"2023-08-27T11:17:05.380573+0000","last_became_peered":"2023-08-27T11:17:05.380573+0000","last_unstale":"2023-08-27T11:17:06.368972+0000","last_undegraded":"2023-08-27T11:17:06.368972+0000","last_fullsized":"2023-08-27T11:17:06.368972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:42.999614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203215+0000","last_change":"2023-08-27T11:17:04.374901+0000","last_active":"2023-08-27T11:17:07.203215+0000","last_peered":"2023-08-27T11:17:07.203215+0000","last_clean":"2023-08-27T11:17:07.203215+0000","last_became_active":"2023-08-27T11:17:04.374186+0000","last_became_peered":"2023-08-27T11:17:04.374186+0000","last_unstale":"2023-08-27T11:17:07.203215+0000","last_undegraded":"2023-08-27T11:17:07.203215+0000","last_fullsized":"2023-08-27T11:17:07.203215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:47.391583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369664+0000","last_change":"2023-08-27T11:17:05.380802+0000","last_active":"2023-08-27T11:17:06.369664+0000","last_peered":"2023-08-27T11:17:06.369664+0000","last_clean":"2023-08-27T11:17:06.369664+0000","last_became_active":"2023-08-27T11:17:05.380537+0000","last_became_peered":"2023-08-27T11:17:05.380537+0000","last_unstale":"2023-08-27T11:17:06.369664+0000","last_undegraded":"2023-08-27T11:17:06.369664+0000","last_fullsized":"2023-08-27T11:17:06.369664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203202+0000","last_change":"2023-08-27T11:17:04.379623+0000","last_active":"2023-08-27T11:17:07.203202+0000","last_peered":"2023-08-27T11:17:07.203202+0000","last_clean":"2023-08-27T11:17:07.203202+0000","last_became_active":"2023-08-27T11:17:04.379342+0000","last_became_peered":"2023-08-27T11:17:04.379342+0000","last_unstale":"2023-08-27T11:17:07.203202+0000","last_undegraded":"2023-08-27T11:17:07.203202+0000","last_fullsized":"2023-08-27T11:17:07.203202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:38.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607824+0000","last_change":"2023-08-27T11:17:05.375956+0000","last_active":"2023-08-27T11:17:06.607824+0000","last_peered":"2023-08-27T11:17:06.607824+0000","last_clean":"2023-08-27T11:17:06.607824+0000","last_became_active":"2023-08-27T11:17:05.375040+0000","last_became_peered":"2023-08-27T11:17:05.375040+0000","last_unstale":"2023-08-27T11:17:06.607824+0000","last_undegraded":"2023-08-27T11:17:06.607824+0000","last_fullsized":"2023-08-27T11:17:06.607824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202873+0000","last_change":"2023-08-27T11:17:04.379633+0000","last_active":"2023-08-27T11:17:07.202873+0000","last_peered":"2023-08-27T11:17:07.202873+0000","last_clean":"2023-08-27T11:17:07.202873+0000","last_became_active":"2023-08-27T11:17:04.379306+0000","last_became_peered":"2023-08-27T11:17:04.379306+0000","last_unstale":"2023-08-27T11:17:07.202873+0000","last_undegraded":"2023-08-27T11:17:07.202873+0000","last_fullsized":"2023-08-27T11:17:07.202873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:53.143119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369174+0000","last_change":"2023-08-27T11:17:05.374205+0000","last_active":"2023-08-27T11:17:06.369174+0000","last_peered":"2023-08-27T11:17:06.369174+0000","last_clean":"2023-08-27T11:17:06.369174+0000","last_became_active":"2023-08-27T11:17:05.373675+0000","last_became_peered":"2023-08-27T11:17:05.373675+0000","last_unstale":"2023-08-27T11:17:06.369174+0000","last_undegraded":"2023-08-27T11:17:06.369174+0000","last_fullsized":"2023-08-27T11:17:06.369174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:43:40.338423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369599+0000","last_change":"2023-08-27T11:17:04.371348+0000","last_active":"2023-08-27T11:17:06.369599+0000","last_peered":"2023-08-27T11:17:06.369599+0000","last_clean":"2023-08-27T11:17:06.369599+0000","last_became_active":"2023-08-27T11:17:04.370929+0000","last_became_peered":"2023-08-27T11:17:04.370929+0000","last_unstale":"2023-08-27T11:17:06.369599+0000","last_undegraded":"2023-08-27T11:17:06.369599+0000","last_fullsized":"2023-08-27T11:17:06.369599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:27.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369296+0000","last_change":"2023-08-27T11:17:05.373711+0000","last_active":"2023-08-27T11:17:06.369296+0000","last_peered":"2023-08-27T11:17:06.369296+0000","last_clean":"2023-08-27T11:17:06.369296+0000","last_became_active":"2023-08-27T11:17:05.373392+0000","last_became_peered":"2023-08-27T11:17:05.373392+0000","last_unstale":"2023-08-27T11:17:06.369296+0000","last_undegraded":"2023-08-27T11:17:06.369296+0000","last_fullsized":"2023-08-27T11:17:06.369296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:35.735781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369479+0000","last_change":"2023-08-27T11:17:04.373259+0000","last_active":"2023-08-27T11:17:06.369479+0000","last_peered":"2023-08-27T11:17:06.369479+0000","last_clean":"2023-08-27T11:17:06.369479+0000","last_became_active":"2023-08-27T11:17:04.372983+0000","last_became_peered":"2023-08-27T11:17:04.372983+0000","last_unstale":"2023-08-27T11:17:06.369479+0000","last_undegraded":"2023-08-27T11:17:06.369479+0000","last_fullsized":"2023-08-27T11:17:06.369479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:36.570705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369051+0000","last_change":"2023-08-27T11:17:05.376729+0000","last_active":"2023-08-27T11:17:06.369051+0000","last_peered":"2023-08-27T11:17:06.369051+0000","last_clean":"2023-08-27T11:17:06.369051+0000","last_became_active":"2023-08-27T11:17:05.376443+0000","last_became_peered":"2023-08-27T11:17:05.376443+0000","last_unstale":"2023-08-27T11:17:06.369051+0000","last_undegraded":"2023-08-27T11:17:06.369051+0000","last_fullsized":"2023-08-27T11:17:06.369051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:48.397096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202559+0000","last_change":"2023-08-27T11:17:04.377051+0000","last_active":"2023-08-27T11:17:07.202559+0000","last_peered":"2023-08-27T11:17:07.202559+0000","last_clean":"2023-08-27T11:17:07.202559+0000","last_became_active":"2023-08-27T11:17:04.376700+0000","last_became_peered":"2023-08-27T11:17:04.376700+0000","last_unstale":"2023-08-27T11:17:07.202559+0000","last_undegraded":"2023-08-27T11:17:07.202559+0000","last_fullsized":"2023-08-27T11:17:07.202559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:42.307697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367824+0000","last_change":"2023-08-27T11:17:05.383341+0000","last_active":"2023-08-27T11:17:06.367824+0000","last_peered":"2023-08-27T11:17:06.367824+0000","last_clean":"2023-08-27T11:17:06.367824+0000","last_became_active":"2023-08-27T11:17:05.379929+0000","last_became_peered":"2023-08-27T11:17:05.379929+0000","last_unstale":"2023-08-27T11:17:06.367824+0000","last_undegraded":"2023-08-27T11:17:06.367824+0000","last_fullsized":"2023-08-27T11:17:06.367824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203147+0000","last_change":"2023-08-27T11:17:04.378623+0000","last_active":"2023-08-27T11:17:07.203147+0000","last_peered":"2023-08-27T11:17:07.203147+0000","last_clean":"2023-08-27T11:17:07.203147+0000","last_became_active":"2023-08-27T11:17:04.378327+0000","last_became_peered":"2023-08-27T11:17:04.378327+0000","last_unstale":"2023-08-27T11:17:07.203147+0000","last_undegraded":"2023-08-27T11:17:07.203147+0000","last_fullsized":"2023-08-27T11:17:07.203147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:02.601133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368983+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:17:06.368983+0000","last_peered":"2023-08-27T11:17:06.368983+0000","last_clean":"2023-08-27T11:17:06.368983+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:17:06.368983+0000","last_undegraded":"2023-08-27T11:17:06.368983+0000","last_fullsized":"2023-08-27T11:17:06.368983+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:07.481452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202508+0000","last_change":"2023-08-27T11:17:05.383215+0000","last_active":"2023-08-27T11:17:07.202508+0000","last_peered":"2023-08-27T11:17:07.202508+0000","last_clean":"2023-08-27T11:17:07.202508+0000","last_became_active":"2023-08-27T11:17:05.382502+0000","last_became_peered":"2023-08-27T11:17:05.382502+0000","last_unstale":"2023-08-27T11:17:07.202508+0000","last_undegraded":"2023-08-27T11:17:07.202508+0000","last_fullsized":"2023-08-27T11:17:07.202508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203019+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:17:07.203019+0000","last_peered":"2023-08-27T11:17:07.203019+0000","last_clean":"2023-08-27T11:17:07.203019+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:17:07.203019+0000","last_undegraded":"2023-08-27T11:17:07.203019+0000","last_fullsized":"2023-08-27T11:17:07.203019+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:42.350470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607950+0000","last_change":"2023-08-27T11:17:04.380486+0000","last_active":"2023-08-27T11:17:06.607950+0000","last_peered":"2023-08-27T11:17:06.607950+0000","last_clean":"2023-08-27T11:17:06.607950+0000","last_became_active":"2023-08-27T11:17:04.376679+0000","last_became_peered":"2023-08-27T11:17:04.376679+0000","last_unstale":"2023-08-27T11:17:06.607950+0000","last_undegraded":"2023-08-27T11:17:06.607950+0000","last_fullsized":"2023-08-27T11:17:06.607950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:41.806953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369152+0000","last_change":"2023-08-27T11:17:05.370457+0000","last_active":"2023-08-27T11:17:06.369152+0000","last_peered":"2023-08-27T11:17:06.369152+0000","last_clean":"2023-08-27T11:17:06.369152+0000","last_became_active":"2023-08-27T11:17:05.370040+0000","last_became_peered":"2023-08-27T11:17:05.370040+0000","last_unstale":"2023-08-27T11:17:06.369152+0000","last_undegraded":"2023-08-27T11:17:06.369152+0000","last_fullsized":"2023-08-27T11:17:06.369152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:03.606646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367481+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:17:06.367481+0000","last_peered":"2023-08-27T11:17:06.367481+0000","last_clean":"2023-08-27T11:17:06.367481+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:17:06.367481+0000","last_undegraded":"2023-08-27T11:17:06.367481+0000","last_fullsized":"2023-08-27T11:17:06.367481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:07:15.067358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203119+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:17:07.203119+0000","last_peered":"2023-08-27T11:17:07.203119+0000","last_clean":"2023-08-27T11:17:07.203119+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:17:07.203119+0000","last_undegraded":"2023-08-27T11:17:07.203119+0000","last_fullsized":"2023-08-27T11:17:07.203119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:36.952988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607599+0000","last_change":"2023-08-27T11:17:04.373430+0000","last_active":"2023-08-27T11:17:06.607599+0000","last_peered":"2023-08-27T11:17:06.607599+0000","last_clean":"2023-08-27T11:17:06.607599+0000","last_became_active":"2023-08-27T11:17:04.373098+0000","last_became_peered":"2023-08-27T11:17:04.373098+0000","last_unstale":"2023-08-27T11:17:06.607599+0000","last_undegraded":"2023-08-27T11:17:06.607599+0000","last_fullsized":"2023-08-27T11:17:06.607599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:07:20.108470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607829+0000","last_change":"2023-08-27T11:17:05.380096+0000","last_active":"2023-08-27T11:17:06.607829+0000","last_peered":"2023-08-27T11:17:06.607829+0000","last_clean":"2023-08-27T11:17:06.607829+0000","last_became_active":"2023-08-27T11:17:05.379846+0000","last_became_peered":"2023-08-27T11:17:05.379846+0000","last_unstale":"2023-08-27T11:17:06.607829+0000","last_undegraded":"2023-08-27T11:17:06.607829+0000","last_fullsized":"2023-08-27T11:17:06.607829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202806+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:17:07.202806+0000","last_peered":"2023-08-27T11:17:07.202806+0000","last_clean":"2023-08-27T11:17:07.202806+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:17:07.202806+0000","last_undegraded":"2023-08-27T11:17:07.202806+0000","last_fullsized":"2023-08-27T11:17:07.202806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:48.102006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367401+0000","last_change":"2023-08-27T11:17:04.373552+0000","last_active":"2023-08-27T11:17:06.367401+0000","last_peered":"2023-08-27T11:17:06.367401+0000","last_clean":"2023-08-27T11:17:06.367401+0000","last_became_active":"2023-08-27T11:17:04.373258+0000","last_became_peered":"2023-08-27T11:17:04.373258+0000","last_unstale":"2023-08-27T11:17:06.367401+0000","last_undegraded":"2023-08-27T11:17:06.367401+0000","last_fullsized":"2023-08-27T11:17:06.367401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:30.937446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367737+0000","last_change":"2023-08-27T11:17:05.383636+0000","last_active":"2023-08-27T11:17:06.367737+0000","last_peered":"2023-08-27T11:17:06.367737+0000","last_clean":"2023-08-27T11:17:06.367737+0000","last_became_active":"2023-08-27T11:17:05.383410+0000","last_became_peered":"2023-08-27T11:17:05.383410+0000","last_unstale":"2023-08-27T11:17:06.367737+0000","last_undegraded":"2023-08-27T11:17:06.367737+0000","last_fullsized":"2023-08-27T11:17:06.367737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:43.313210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202834+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:17:07.202834+0000","last_peered":"2023-08-27T11:17:07.202834+0000","last_clean":"2023-08-27T11:17:07.202834+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:17:07.202834+0000","last_undegraded":"2023-08-27T11:17:07.202834+0000","last_fullsized":"2023-08-27T11:17:07.202834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:36.765841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202793+0000","last_change":"2023-08-27T11:17:04.375838+0000","last_active":"2023-08-27T11:17:07.202793+0000","last_peered":"2023-08-27T11:17:07.202793+0000","last_clean":"2023-08-27T11:17:07.202793+0000","last_became_active":"2023-08-27T11:17:04.375536+0000","last_became_peered":"2023-08-27T11:17:04.375536+0000","last_unstale":"2023-08-27T11:17:07.202793+0000","last_undegraded":"2023-08-27T11:17:07.202793+0000","last_fullsized":"2023-08-27T11:17:07.202793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:53.957455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369664+0000","last_change":"2023-08-27T11:17:05.375110+0000","last_active":"2023-08-27T11:17:06.369664+0000","last_peered":"2023-08-27T11:17:06.369664+0000","last_clean":"2023-08-27T11:17:06.369664+0000","last_became_active":"2023-08-27T11:17:05.374844+0000","last_became_peered":"2023-08-27T11:17:05.374844+0000","last_unstale":"2023-08-27T11:17:06.369664+0000","last_undegraded":"2023-08-27T11:17:06.369664+0000","last_fullsized":"2023-08-27T11:17:06.369664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:15.192349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202453+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:17:07.202453+0000","last_peered":"2023-08-27T11:17:07.202453+0000","last_clean":"2023-08-27T11:17:07.202453+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:17:07.202453+0000","last_undegraded":"2023-08-27T11:17:07.202453+0000","last_fullsized":"2023-08-27T11:17:07.202453+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:43.315205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368949+0000","last_change":"2023-08-27T11:17:04.379053+0000","last_active":"2023-08-27T11:17:06.368949+0000","last_peered":"2023-08-27T11:17:06.368949+0000","last_clean":"2023-08-27T11:17:06.368949+0000","last_became_active":"2023-08-27T11:17:04.378731+0000","last_became_peered":"2023-08-27T11:17:04.378731+0000","last_unstale":"2023-08-27T11:17:06.368949+0000","last_undegraded":"2023-08-27T11:17:06.368949+0000","last_fullsized":"2023-08-27T11:17:06.368949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:53.143119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367664+0000","last_change":"2023-08-27T11:17:05.373510+0000","last_active":"2023-08-27T11:17:06.367664+0000","last_peered":"2023-08-27T11:17:06.367664+0000","last_clean":"2023-08-27T11:17:06.367664+0000","last_became_active":"2023-08-27T11:17:05.373229+0000","last_became_peered":"2023-08-27T11:17:05.373229+0000","last_unstale":"2023-08-27T11:17:06.367664+0000","last_undegraded":"2023-08-27T11:17:06.367664+0000","last_fullsized":"2023-08-27T11:17:06.367664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369394+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:17:06.369394+0000","last_peered":"2023-08-27T11:17:06.369394+0000","last_clean":"2023-08-27T11:17:06.369394+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:17:06.369394+0000","last_undegraded":"2023-08-27T11:17:06.369394+0000","last_fullsized":"2023-08-27T11:17:06.369394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:07:15.067358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367592+0000","last_change":"2023-08-27T11:17:04.374506+0000","last_active":"2023-08-27T11:17:06.367592+0000","last_peered":"2023-08-27T11:17:06.367592+0000","last_clean":"2023-08-27T11:17:06.367592+0000","last_became_active":"2023-08-27T11:17:04.374209+0000","last_became_peered":"2023-08-27T11:17:04.374209+0000","last_unstale":"2023-08-27T11:17:06.367592+0000","last_undegraded":"2023-08-27T11:17:06.367592+0000","last_fullsized":"2023-08-27T11:17:06.367592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:27.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607676+0000","last_change":"2023-08-27T11:17:05.369565+0000","last_active":"2023-08-27T11:17:06.607676+0000","last_peered":"2023-08-27T11:17:06.607676+0000","last_clean":"2023-08-27T11:17:06.607676+0000","last_became_active":"2023-08-27T11:17:05.369334+0000","last_became_peered":"2023-08-27T11:17:05.369334+0000","last_unstale":"2023-08-27T11:17:06.607676+0000","last_undegraded":"2023-08-27T11:17:06.607676+0000","last_fullsized":"2023-08-27T11:17:06.607676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369316+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:17:06.369316+0000","last_peered":"2023-08-27T11:17:06.369316+0000","last_clean":"2023-08-27T11:17:06.369316+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:17:06.369316+0000","last_undegraded":"2023-08-27T11:17:06.369316+0000","last_fullsized":"2023-08-27T11:17:06.369316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:25.461887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203500+0000","last_change":"2023-08-27T11:17:04.380000+0000","last_active":"2023-08-27T11:17:07.203500+0000","last_peered":"2023-08-27T11:17:07.203500+0000","last_clean":"2023-08-27T11:17:07.203500+0000","last_became_active":"2023-08-27T11:17:04.379666+0000","last_became_peered":"2023-08-27T11:17:04.379666+0000","last_unstale":"2023-08-27T11:17:07.203500+0000","last_undegraded":"2023-08-27T11:17:07.203500+0000","last_fullsized":"2023-08-27T11:17:07.203500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:53.957455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367721+0000","last_change":"2023-08-27T11:17:05.380929+0000","last_active":"2023-08-27T11:17:06.367721+0000","last_peered":"2023-08-27T11:17:06.367721+0000","last_clean":"2023-08-27T11:17:06.367721+0000","last_became_active":"2023-08-27T11:17:05.380573+0000","last_became_peered":"2023-08-27T11:17:05.380573+0000","last_unstale":"2023-08-27T11:17:06.367721+0000","last_undegraded":"2023-08-27T11:17:06.367721+0000","last_fullsized":"2023-08-27T11:17:06.367721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607806+0000","last_change":"2023-08-27T11:17:04.377322+0000","last_active":"2023-08-27T11:17:06.607806+0000","last_peered":"2023-08-27T11:17:06.607806+0000","last_clean":"2023-08-27T11:17:06.607806+0000","last_became_active":"2023-08-27T11:17:04.376967+0000","last_became_peered":"2023-08-27T11:17:04.376967+0000","last_unstale":"2023-08-27T11:17:06.607806+0000","last_undegraded":"2023-08-27T11:17:06.607806+0000","last_fullsized":"2023-08-27T11:17:06.607806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:42.307697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190107+0000","last_change":"2023-08-27T11:17:05.382640+0000","last_active":"2023-08-27T11:17:07.190107+0000","last_peered":"2023-08-27T11:17:07.190107+0000","last_clean":"2023-08-27T11:17:07.190107+0000","last_became_active":"2023-08-27T11:17:05.382377+0000","last_became_peered":"2023-08-27T11:17:05.382377+0000","last_unstale":"2023-08-27T11:17:07.190107+0000","last_undegraded":"2023-08-27T11:17:07.190107+0000","last_fullsized":"2023-08-27T11:17:07.190107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:27:57.016677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203339+0000","last_change":"2023-08-27T11:17:04.372997+0000","last_active":"2023-08-27T11:17:07.203339+0000","last_peered":"2023-08-27T11:17:07.203339+0000","last_clean":"2023-08-27T11:17:07.203339+0000","last_became_active":"2023-08-27T11:17:04.372719+0000","last_became_peered":"2023-08-27T11:17:04.372719+0000","last_unstale":"2023-08-27T11:17:07.203339+0000","last_undegraded":"2023-08-27T11:17:07.203339+0000","last_fullsized":"2023-08-27T11:17:07.203339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:51.710890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368989+0000","last_change":"2023-08-27T11:17:05.374254+0000","last_active":"2023-08-27T11:17:06.368989+0000","last_peered":"2023-08-27T11:17:06.368989+0000","last_clean":"2023-08-27T11:17:06.368989+0000","last_became_active":"2023-08-27T11:17:05.373877+0000","last_became_peered":"2023-08-27T11:17:05.373877+0000","last_unstale":"2023-08-27T11:17:06.368989+0000","last_undegraded":"2023-08-27T11:17:06.368989+0000","last_fullsized":"2023-08-27T11:17:06.368989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:42.812467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203587+0000","last_change":"2023-08-27T11:17:04.372930+0000","last_active":"2023-08-27T11:17:07.203587+0000","last_peered":"2023-08-27T11:17:07.203587+0000","last_clean":"2023-08-27T11:17:07.203587+0000","last_became_active":"2023-08-27T11:17:04.372626+0000","last_became_peered":"2023-08-27T11:17:04.372626+0000","last_unstale":"2023-08-27T11:17:07.203587+0000","last_undegraded":"2023-08-27T11:17:07.203587+0000","last_fullsized":"2023-08-27T11:17:07.203587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:45.738091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369549+0000","last_change":"2023-08-27T11:17:05.374125+0000","last_active":"2023-08-27T11:17:06.369549+0000","last_peered":"2023-08-27T11:17:06.369549+0000","last_clean":"2023-08-27T11:17:06.369549+0000","last_became_active":"2023-08-27T11:17:05.373856+0000","last_became_peered":"2023-08-27T11:17:05.373856+0000","last_unstale":"2023-08-27T11:17:06.369549+0000","last_undegraded":"2023-08-27T11:17:06.369549+0000","last_fullsized":"2023-08-27T11:17:06.369549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:17:32.516797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203038+0000","last_change":"2023-08-27T11:17:04.375965+0000","last_active":"2023-08-27T11:17:07.203038+0000","last_peered":"2023-08-27T11:17:07.203038+0000","last_clean":"2023-08-27T11:17:07.203038+0000","last_became_active":"2023-08-27T11:17:04.375687+0000","last_became_peered":"2023-08-27T11:17:04.375687+0000","last_unstale":"2023-08-27T11:17:07.203038+0000","last_undegraded":"2023-08-27T11:17:07.203038+0000","last_fullsized":"2023-08-27T11:17:07.203038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:12.522565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202911+0000","last_change":"2023-08-27T11:17:05.372749+0000","last_active":"2023-08-27T11:17:07.202911+0000","last_peered":"2023-08-27T11:17:07.202911+0000","last_clean":"2023-08-27T11:17:07.202911+0000","last_became_active":"2023-08-27T11:17:05.372494+0000","last_became_peered":"2023-08-27T11:17:05.372494+0000","last_unstale":"2023-08-27T11:17:07.202911+0000","last_undegraded":"2023-08-27T11:17:07.202911+0000","last_fullsized":"2023-08-27T11:17:07.202911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:42.999614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607966+0000","last_change":"2023-08-27T11:17:04.374287+0000","last_active":"2023-08-27T11:17:06.607966+0000","last_peered":"2023-08-27T11:17:06.607966+0000","last_clean":"2023-08-27T11:17:06.607966+0000","last_became_active":"2023-08-27T11:17:04.374009+0000","last_became_peered":"2023-08-27T11:17:04.374009+0000","last_unstale":"2023-08-27T11:17:06.607966+0000","last_undegraded":"2023-08-27T11:17:06.607966+0000","last_fullsized":"2023-08-27T11:17:06.607966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:45.738091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368606+0000","last_change":"2023-08-27T11:17:05.369490+0000","last_active":"2023-08-27T11:17:06.368606+0000","last_peered":"2023-08-27T11:17:06.368606+0000","last_clean":"2023-08-27T11:17:06.368606+0000","last_became_active":"2023-08-27T11:17:05.369193+0000","last_became_peered":"2023-08-27T11:17:05.369193+0000","last_unstale":"2023-08-27T11:17:06.368606+0000","last_undegraded":"2023-08-27T11:17:06.368606+0000","last_fullsized":"2023-08-27T11:17:06.368606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368508+0000","last_change":"2023-08-27T11:17:05.376280+0000","last_active":"2023-08-27T11:17:06.368508+0000","last_peered":"2023-08-27T11:17:06.368508+0000","last_clean":"2023-08-27T11:17:06.368508+0000","last_became_active":"2023-08-27T11:17:05.375763+0000","last_became_peered":"2023-08-27T11:17:05.375763+0000","last_unstale":"2023-08-27T11:17:06.368508+0000","last_undegraded":"2023-08-27T11:17:06.368508+0000","last_fullsized":"2023-08-27T11:17:06.368508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:46.814570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202731+0000","last_change":"2023-08-27T11:17:05.384097+0000","last_active":"2023-08-27T11:17:07.202731+0000","last_peered":"2023-08-27T11:17:07.202731+0000","last_clean":"2023-08-27T11:17:07.202731+0000","last_became_active":"2023-08-27T11:17:05.383673+0000","last_became_peered":"2023-08-27T11:17:05.383673+0000","last_unstale":"2023-08-27T11:17:07.202731+0000","last_undegraded":"2023-08-27T11:17:07.202731+0000","last_fullsized":"2023-08-27T11:17:07.202731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368848+0000","last_change":"2023-08-27T11:17:05.369440+0000","last_active":"2023-08-27T11:17:06.368848+0000","last_peered":"2023-08-27T11:17:06.368848+0000","last_clean":"2023-08-27T11:17:06.368848+0000","last_became_active":"2023-08-27T11:17:05.369113+0000","last_became_peered":"2023-08-27T11:17:05.369113+0000","last_unstale":"2023-08-27T11:17:06.368848+0000","last_undegraded":"2023-08-27T11:17:06.368848+0000","last_fullsized":"2023-08-27T11:17:06.368848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:39.209301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202857+0000","last_change":"2023-08-27T11:17:05.381671+0000","last_active":"2023-08-27T11:17:07.202857+0000","last_peered":"2023-08-27T11:17:07.202857+0000","last_clean":"2023-08-27T11:17:07.202857+0000","last_became_active":"2023-08-27T11:17:05.380933+0000","last_became_peered":"2023-08-27T11:17:05.380933+0000","last_unstale":"2023-08-27T11:17:07.202857+0000","last_undegraded":"2023-08-27T11:17:07.202857+0000","last_fullsized":"2023-08-27T11:17:07.202857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:46.743604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367534+0000","last_change":"2023-08-27T11:17:05.379944+0000","last_active":"2023-08-27T11:17:06.367534+0000","last_peered":"2023-08-27T11:17:06.367534+0000","last_clean":"2023-08-27T11:17:06.367534+0000","last_became_active":"2023-08-27T11:17:05.379633+0000","last_became_peered":"2023-08-27T11:17:05.379633+0000","last_unstale":"2023-08-27T11:17:06.367534+0000","last_undegraded":"2023-08-27T11:17:06.367534+0000","last_fullsized":"2023-08-27T11:17:06.367534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":659971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":74428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":786432,"data_stored":658143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":73649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":67775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":76096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":200704,"data_stored":68427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:17:09.835 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:17:09.836 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-08-27T11:17:09.836 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-08-27T11:17:09.837 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-08-27T11:17:09.838 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-08-27T11:17:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.988+0000 7f2f23d01700 1 Processor -- start 2023-08-27T11:17:09.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.988+0000 7f2f23d01700 1 -- start start 2023-08-27T11:17:09.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.989+0000 7f2f23d01700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1c06b230 0x7f2f1c0ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.989+0000 7f2f23d01700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c15eb60 con 0x7f2f1c151060 2023-08-27T11:17:09.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.989+0000 7f2f23d01700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c06b6c0 con 0x7f2f1c06adb0 2023-08-27T11:17:09.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.989+0000 7f2f23d01700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0b02f0 con 0x7f2f1c06b230 2023-08-27T11:17:09.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.989+0000 7f2f21a9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2f1c151060 0x7f2f1c15e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49138/0 (socket says 172.21.15.112:49138) 2023-08-27T11:17:09.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.989+0000 7f2f21a9d700 1 -- 172.21.15.112:0/1254043246 learned_addr learned my addr 172.21.15.112:0/1254043246 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:09.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.989+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398404535 0 0) 0x7f2f1c15eb60 con 0x7f2f1c151060 2023-08-27T11:17:09.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.989+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f080035b0 con 0x7f2f1c151060 2023-08-27T11:17:09.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.989+0000 7f2f2229e700 1 --2- 172.21.15.112:0/1254043246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1c06b230 0x7f2f1c0ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 542280134 0 0) 0x7f2f080035b0 con 0x7f2f1c151060 2023-08-27T11:17:09.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1c06b230 msgr2=0x7f2f1c0ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:09.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 --2- 172.21.15.112:0/1254043246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1c06b230 0x7f2f1c0ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 >> v1:172.21.15.138:6789/0 conn(0x7f2f1c06adb0 legacy=0x7f2f1c0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c0b05d0 con 0x7f2f1c151060 2023-08-27T11:17:09.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f2f0c001670 con 0x7f2f1c151060 2023-08-27T11:17:09.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 >> v1:172.21.15.112:6789/0 conn(0x7f2f1c151060 legacy=0x7f2f1c15e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:09.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 --2- 172.21.15.112:0/1254043246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f08003a30 0x7f2f08003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 --2- 172.21.15.112:0/1254043246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f08004850 0x7f2f08004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 --2- 172.21.15.112:0/1254043246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1c06b230 0x7f2f08005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:09.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0b02f0 con 0x7f2f08003a30 2023-08-27T11:17:09.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0800bac0 con 0x7f2f08004850 2023-08-27T11:17:09.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0800bc80 con 0x7f2f1c06b230 2023-08-27T11:17:09.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f21a9d700 1 --2- 172.21.15.112:0/1254043246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f08003a30 0x7f2f08003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f2129c700 1 --2- 172.21.15.112:0/1254043246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f08004850 0x7f2f08004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f2229e700 1 --2- 172.21.15.112:0/1254043246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1c06b230 0x7f2f08005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:09.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f21a9d700 1 -- 172.21.15.112:0/1254043246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1c06b230 msgr2=0x7f2f08005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:09.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f21a9d700 1 --2- 172.21.15.112:0/1254043246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1c06b230 0x7f2f08005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f21a9d700 1 -- 172.21.15.112:0/1254043246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f08004850 msgr2=0x7f2f08004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:09.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f21a9d700 1 --2- 172.21.15.112:0/1254043246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f08004850 0x7f2f08004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:09.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f21a9d700 1 -- 172.21.15.112:0/1254043246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f080035b0 con 0x7f2f08003a30 2023-08-27T11:17:09.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f2129c700 1 --2- 172.21.15.112:0/1254043246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f08004850 0x7f2f08004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:09.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f21a9d700 1 --2- 172.21.15.112:0/1254043246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f08003a30 0x7f2f08003e50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c001eb0 tx=0x7f2f08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5103608427b6de server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:10.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c00cd50 con 0x7f2f08003a30 2023-08-27T11:17:10.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0c005960 con 0x7f2f08003a30 2023-08-27T11:17:10.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.990+0000 7f2f20a9b700 1 -- 172.21.15.112:0/1254043246 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c005d10 con 0x7f2f08003a30 2023-08-27T11:17:10.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.991+0000 7f2f23d01700 1 -- 172.21.15.112:0/1254043246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f08003a30 msgr2=0x7f2f08003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.991+0000 7f2f23d01700 1 --2- 172.21.15.112:0/1254043246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f08003a30 0x7f2f08003e50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c001eb0 tx=0x7f2f08005880 comp rx=0 tx=0).stop 2023-08-27T11:17:10.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.991+0000 7f2f2229e700 1 -- 172.21.15.112:0/1254043246 reap_dead start 2023-08-27T11:17:10.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.991+0000 7f2f23d01700 1 -- 172.21.15.112:0/1254043246 shutdown_connections 2023-08-27T11:17:10.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.991+0000 7f2f23d01700 1 -- 172.21.15.112:0/1254043246 >> 172.21.15.112:0/1254043246 conn(0x7f2f1c0d0000 msgr2=0x7f2f1c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:10.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.991+0000 7f2f23d01700 1 -- 172.21.15.112:0/1254043246 shutdown_connections 2023-08-27T11:17:10.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.991+0000 7f2f23d01700 1 -- 172.21.15.112:0/1254043246 wait complete. 2023-08-27T11:17:10.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.992+0000 7f2f23d01700 1 Processor -- start 2023-08-27T11:17:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.992+0000 7f2f23d01700 1 -- start start 2023-08-27T11:17:10.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f23d01700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f08003a30 0x7f2f1c15bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f23d01700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1c072270 0x7f2f1c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f23d01700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1c070010 0x7f2f1c0716c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f23d01700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c152e20 con 0x7f2f08003a30 2023-08-27T11:17:10.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f23d01700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c070580 con 0x7f2f1c070010 2023-08-27T11:17:10.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f23d01700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c070860 con 0x7f2f1c072270 2023-08-27T11:17:10.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f21a9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f08003a30 0x7f2f1c15bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f21a9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f08003a30 0x7f2f1c15bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54768/0 (socket says 172.21.15.112:54768) 2023-08-27T11:17:10.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f21a9d700 1 -- 172.21.15.112:0/873388860 learned_addr learned my addr 172.21.15.112:0/873388860 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:10.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f2229e700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1c070010 0x7f2f1c0716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f2129c700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1c072270 0x7f2f1c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f21a9d700 1 -- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1c072270 msgr2=0x7f2f1c071180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f21a9d700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1c072270 0x7f2f1c071180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f21a9d700 1 -- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1c070010 msgr2=0x7f2f1c0716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f21a9d700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1c070010 0x7f2f1c0716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f21a9d700 1 -- 172.21.15.112:0/873388860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c1527d0 con 0x7f2f08003a30 2023-08-27T11:17:10.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f2229e700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1c070010 0x7f2f1c0716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:10.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.993+0000 7f2f21a9d700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f08003a30 0x7f2f1c15bb00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c001050 tx=0x7f2f0c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:10.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.994+0000 7f2f127fc700 1 -- 172.21.15.112:0/873388860 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c004870 con 0x7f2f08003a30 2023-08-27T11:17:10.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.994+0000 7f2f127fc700 1 -- 172.21.15.112:0/873388860 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0c0049f0 con 0x7f2f08003a30 2023-08-27T11:17:10.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.994+0000 7f2f23d01700 1 -- 172.21.15.112:0/873388860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c152ab0 con 0x7f2f08003a30 2023-08-27T11:17:10.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.994+0000 7f2f127fc700 1 -- 172.21.15.112:0/873388860 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c004da0 con 0x7f2f08003a30 2023-08-27T11:17:10.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.994+0000 7f2f23d01700 1 -- 172.21.15.112:0/873388860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c15c220 con 0x7f2f08003a30 2023-08-27T11:17:10.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.994+0000 7f2f23d01700 1 -- 172.21.15.112:0/873388860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c152630 con 0x7f2f08003a30 2023-08-27T11:17:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.995+0000 7f2f127fc700 1 -- 172.21.15.112:0/873388860 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2f0c019070 con 0x7f2f08003a30 2023-08-27T11:17:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.995+0000 7f2f127fc700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2f0006a0e0 0x7f2f0006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.995+0000 7f2f127fc700 1 -- 172.21.15.112:0/873388860 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2f0c089120 con 0x7f2f08003a30 2023-08-27T11:17:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.996+0000 7f2f2129c700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2f0006a0e0 0x7f2f0006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:09.996+0000 7f2f2129c700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2f0006a0e0 0x7f2f0006c5a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c167e50 tx=0x7f2f18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:10.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.000+0000 7f2f127fc700 1 -- 172.21.15.112:0/873388860 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2f1c152630 con 0x7f2f08003a30 2023-08-27T11:17:10.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.243+0000 7f2f23d01700 1 -- 172.21.15.112:0/873388860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f2f1c153100 con 0x7f2f08003a30 2023-08-27T11:17:10.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.245+0000 7f2f127fc700 1 -- 172.21.15.112:0/873388860 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+169495 (secure 0 0 0) 0x7f2f0c054820 con 0x7f2f08003a30 2023-08-27T11:17:10.249 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:10.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.248+0000 7f2f23d01700 1 -- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2f0006a0e0 msgr2=0x7f2f0006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.248+0000 7f2f23d01700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2f0006a0e0 0x7f2f0006c5a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c167e50 tx=0x7f2f18009040 comp rx=0 tx=0).stop 2023-08-27T11:17:10.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.248+0000 7f2f23d01700 1 -- 172.21.15.112:0/873388860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f08003a30 msgr2=0x7f2f1c15bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.248+0000 7f2f23d01700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f08003a30 0x7f2f1c15bb00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c001050 tx=0x7f2f0c000fe0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.248+0000 7f2f23d01700 1 -- 172.21.15.112:0/873388860 shutdown_connections 2023-08-27T11:17:10.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.248+0000 7f2f23d01700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2f0006a0e0 0x7f2f0006c5a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.248+0000 7f2f23d01700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1c070010 0x7f2f1c0716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.248+0000 7f2f23d01700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f08003a30 0x7f2f1c15bb00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.248+0000 7f2f23d01700 1 --2- 172.21.15.112:0/873388860 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1c072270 0x7f2f1c071180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.248+0000 7f2f23d01700 1 -- 172.21.15.112:0/873388860 >> 172.21.15.112:0/873388860 conn(0x7f2f1c0d0000 msgr2=0x7f2f1c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:10.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.248+0000 7f2f23d01700 1 -- 172.21.15.112:0/873388860 shutdown_connections 2023-08-27T11:17:10.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.248+0000 7f2f23d01700 1 -- 172.21.15.112:0/873388860 wait complete. 2023-08-27T11:17:10.261 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":5,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":702326761},{"type":"v1","addr":"172.21.15.138:6833","nonce":702326761}]},"active_addr":"172.21.15.138:6833/702326761","active_change":"2023-08-27T11:16:52.217185+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":4280155522}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":4223373079}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":1694700858}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":795634812}]}]} 2023-08-27T11:17:10.265 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-08-27T11:17:10.266 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-08-27T11:17:10.266 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:10.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.385+0000 7faf90d53700 1 Processor -- start 2023-08-27T11:17:10.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.385+0000 7faf90d53700 1 -- start start 2023-08-27T11:17:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.386+0000 7faf90d53700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faf8c158e20 0x7faf8c0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.386+0000 7faf90d53700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf8c06e3a0 0x7faf8c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.386+0000 7faf90d53700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf8c15c190 con 0x7faf8c06fcd0 2023-08-27T11:17:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.386+0000 7faf90d53700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf8c0b1730 con 0x7faf8c158e20 2023-08-27T11:17:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.386+0000 7faf90d53700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c070160 con 0x7faf8c06e3a0 2023-08-27T11:17:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.386+0000 7faf8ad9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7faf8c06fcd0 0x7faf8c15ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49158/0 (socket says 172.21.15.112:49158) 2023-08-27T11:17:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.386+0000 7faf8ad9d700 1 -- 172.21.15.112:0/2854988205 learned_addr learned my addr 172.21.15.112:0/2854988205 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:10.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.386+0000 7faf8a59c700 1 --2- 172.21.15.112:0/2854988205 >> v2:172.21.15.138:3300/0 conn(0x7faf8c158e20 0x7faf8c0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.386+0000 7faf89d9b700 1 --2- 172.21.15.112:0/2854988205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf8c06e3a0 0x7faf8c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.386+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990160652 0 0) 0x7faf8c15c190 con 0x7faf8c06fcd0 2023-08-27T11:17:10.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.386+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf740035b0 con 0x7faf8c06fcd0 2023-08-27T11:17:10.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.386+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1699839782 0 0) 0x7faf740035b0 con 0x7faf8c06fcd0 2023-08-27T11:17:10.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf8c06e3a0 msgr2=0x7faf8c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:10.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 --2- 172.21.15.112:0/2854988205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf8c06e3a0 0x7faf8c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 >> v2:172.21.15.138:3300/0 conn(0x7faf8c158e20 msgr2=0x7faf8c0b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:10.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 --2- 172.21.15.112:0/2854988205 >> v2:172.21.15.138:3300/0 conn(0x7faf8c158e20 0x7faf8c0b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c0b09d0 con 0x7faf8c06fcd0 2023-08-27T11:17:10.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7faf840019f0 con 0x7faf8c06fcd0 2023-08-27T11:17:10.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 >> v1:172.21.15.112:6789/0 conn(0x7faf8c06fcd0 legacy=0x7faf8c15ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 --2- 172.21.15.112:0/2854988205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf74003a00 0x7faf74003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 --2- 172.21.15.112:0/2854988205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf74004850 0x7faf74004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 --2- 172.21.15.112:0/2854988205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf8c06e3a0 0x7faf74005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c0b1730 con 0x7faf74003a00 2023-08-27T11:17:10.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c070160 con 0x7faf74004850 2023-08-27T11:17:10.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7400ba10 con 0x7faf8c06e3a0 2023-08-27T11:17:10.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8a59c700 1 --2- 172.21.15.112:0/2854988205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf74003a00 0x7faf74003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf89d9b700 1 --2- 172.21.15.112:0/2854988205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf74004850 0x7faf74004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8ad9d700 1 --2- 172.21.15.112:0/2854988205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf8c06e3a0 0x7faf74005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8a59c700 1 -- 172.21.15.112:0/2854988205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf8c06e3a0 msgr2=0x7faf74005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:10.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8a59c700 1 --2- 172.21.15.112:0/2854988205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf8c06e3a0 0x7faf74005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8a59c700 1 -- 172.21.15.112:0/2854988205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf74004850 msgr2=0x7faf74004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:10.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8a59c700 1 --2- 172.21.15.112:0/2854988205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf74004850 0x7faf74004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8a59c700 1 -- 172.21.15.112:0/2854988205 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf740035b0 con 0x7faf74003a00 2023-08-27T11:17:10.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8a59c700 1 --2- 172.21.15.112:0/2854988205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf74003a00 0x7faf74003e20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009b60 tx=0x7faf74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eea905b97ad405f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:10.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00e6c0 con 0x7faf74003a00 2023-08-27T11:17:10.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c00e820 con 0x7faf74003a00 2023-08-27T11:17:10.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.387+0000 7faf8959a700 1 -- 172.21.15.112:0/2854988205 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00eb90 con 0x7faf74003a00 2023-08-27T11:17:10.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.388+0000 7faf90d53700 1 -- 172.21.15.112:0/2854988205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf74003a00 msgr2=0x7faf74003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.388+0000 7faf90d53700 1 --2- 172.21.15.112:0/2854988205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf74003a00 0x7faf74003e20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009b60 tx=0x7faf74005880 comp rx=0 tx=0).stop 2023-08-27T11:17:10.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.388+0000 7faf8ad9d700 1 -- 172.21.15.112:0/2854988205 reap_dead start 2023-08-27T11:17:10.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.388+0000 7faf90d53700 1 -- 172.21.15.112:0/2854988205 shutdown_connections 2023-08-27T11:17:10.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.388+0000 7faf90d53700 1 -- 172.21.15.112:0/2854988205 >> 172.21.15.112:0/2854988205 conn(0x7faf8c0d0000 msgr2=0x7faf8c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:10.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.388+0000 7faf90d53700 1 -- 172.21.15.112:0/2854988205 shutdown_connections 2023-08-27T11:17:10.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.388+0000 7faf90d53700 1 -- 172.21.15.112:0/2854988205 wait complete. 2023-08-27T11:17:10.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.389+0000 7faf90d53700 1 Processor -- start 2023-08-27T11:17:10.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.389+0000 7faf90d53700 1 -- start start 2023-08-27T11:17:10.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf90d53700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf74003a00 0x7faf8c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf90d53700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf8c1f3d40 0x7faf8c1f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf90d53700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf8c1f3600 0x7faf8c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf90d53700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c0c56b0 con 0x7faf74003a00 2023-08-27T11:17:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf90d53700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c0cfb70 con 0x7faf8c1f3600 2023-08-27T11:17:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf90d53700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c0c4d80 con 0x7faf8c1f3d40 2023-08-27T11:17:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf8a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf74003a00 0x7faf8c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf8a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf74003a00 0x7faf8c0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54786/0 (socket says 172.21.15.112:54786) 2023-08-27T11:17:10.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf8a59c700 1 -- 172.21.15.112:0/2508185186 learned_addr learned my addr 172.21.15.112:0/2508185186 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:10.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf89d9b700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf8c1f3d40 0x7faf8c1f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf8ad9d700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf8c1f3600 0x7faf8c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf8a59c700 1 -- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf8c1f3d40 msgr2=0x7faf8c1f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf8a59c700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf8c1f3d40 0x7faf8c1f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf8a59c700 1 -- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf8c1f3600 msgr2=0x7faf8c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf8a59c700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf8c1f3600 0x7faf8c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf8a59c700 1 -- 172.21.15.112:0/2508185186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c0c5060 con 0x7faf74003a00 2023-08-27T11:17:10.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf8a59c700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf74003a00 0x7faf8c0cf590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0090b0 tx=0x7faf7c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:10.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.390+0000 7faf7affd700 1 -- 172.21.15.112:0/2508185186 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00e6c0 con 0x7faf74003a00 2023-08-27T11:17:10.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.391+0000 7faf7affd700 1 -- 172.21.15.112:0/2508185186 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c003ad0 con 0x7faf74003a00 2023-08-27T11:17:10.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.391+0000 7faf7affd700 1 -- 172.21.15.112:0/2508185186 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c003e80 con 0x7faf74003a00 2023-08-27T11:17:10.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.391+0000 7faf90d53700 1 -- 172.21.15.112:0/2508185186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf8c0c5340 con 0x7faf74003a00 2023-08-27T11:17:10.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.391+0000 7faf90d53700 1 -- 172.21.15.112:0/2508185186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8c15da00 con 0x7faf74003a00 2023-08-27T11:17:10.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.391+0000 7faf90d53700 1 -- 172.21.15.112:0/2508185186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8c0c5990 con 0x7faf74003a00 2023-08-27T11:17:10.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.392+0000 7faf7affd700 1 -- 172.21.15.112:0/2508185186 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faf7c013470 con 0x7faf74003a00 2023-08-27T11:17:10.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.392+0000 7faf7affd700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf6c06a0e0 0x7faf6c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.392+0000 7faf7affd700 1 -- 172.21.15.112:0/2508185186 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf7c089ec0 con 0x7faf74003a00 2023-08-27T11:17:10.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.393+0000 7faf89d9b700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf6c06a0e0 0x7faf6c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.393+0000 7faf89d9b700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf6c06a0e0 0x7faf6c06c5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf8c1f0770 tx=0x7faf80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:10.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.397+0000 7faf7affd700 1 -- 172.21.15.112:0/2508185186 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7faf7c055760 con 0x7faf74003a00 2023-08-27T11:17:10.453 INFO:tasks.ceph.mds.b.smithi138.stderr:2023-08-27T11:17:10.452+0000 7f869ef00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:17:10.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.585+0000 7faf90d53700 1 -- 172.21.15.112:0/2508185186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf8c067a20 con 0x7faf74003a00 2023-08-27T11:17:10.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.587+0000 7faf7affd700 1 -- 172.21.15.112:0/2508185186 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7faf7c058d80 con 0x7faf74003a00 2023-08-27T11:17:10.588 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:10.589 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:10.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.589+0000 7faf90d53700 1 -- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf6c06a0e0 msgr2=0x7faf6c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.589+0000 7faf90d53700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf6c06a0e0 0x7faf6c06c5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf8c1f0770 tx=0x7faf80008040 comp rx=0 tx=0).stop 2023-08-27T11:17:10.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.589+0000 7faf90d53700 1 -- 172.21.15.112:0/2508185186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf74003a00 msgr2=0x7faf8c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.589+0000 7faf90d53700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf74003a00 0x7faf8c0cf590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0090b0 tx=0x7faf7c0047c0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.589+0000 7faf90d53700 1 -- 172.21.15.112:0/2508185186 shutdown_connections 2023-08-27T11:17:10.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.589+0000 7faf90d53700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf6c06a0e0 0x7faf6c06c5a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.589+0000 7faf90d53700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf8c1f3600 0x7faf8c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.589+0000 7faf90d53700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf74003a00 0x7faf8c0cf590 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.589+0000 7faf90d53700 1 --2- 172.21.15.112:0/2508185186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf8c1f3d40 0x7faf8c1f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.589+0000 7faf90d53700 1 -- 172.21.15.112:0/2508185186 >> 172.21.15.112:0/2508185186 conn(0x7faf8c0d0000 msgr2=0x7faf8c071600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:10.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.589+0000 7faf90d53700 1 -- 172.21.15.112:0/2508185186 shutdown_connections 2023-08-27T11:17:10.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.589+0000 7faf90d53700 1 -- 172.21.15.112:0/2508185186 wait complete. 2023-08-27T11:17:10.603 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-08-27T11:17:10.604 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:10.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.722+0000 7fe9e1a4a700 1 Processor -- start 2023-08-27T11:17:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.722+0000 7fe9e1a4a700 1 -- start start 2023-08-27T11:17:10.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.723+0000 7fe9e1a4a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe9dc146be0 0x7fe9dc0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.723+0000 7fe9e1a4a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9dc1619b0 0x7fe9dc0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.723+0000 7fe9e1a4a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9dc161e10 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.723+0000 7fe9e1a4a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9dc0b3980 con 0x7fe9dc146be0 2023-08-27T11:17:10.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.723+0000 7fe9e1a4a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0b5730 con 0x7fe9dc1619b0 2023-08-27T11:17:10.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.723+0000 7fe9db7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe9dc0b52a0 0x7fe9dc15d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49178/0 (socket says 172.21.15.112:49178) 2023-08-27T11:17:10.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.723+0000 7fe9db7fe700 1 -- 172.21.15.112:0/4294119710 learned_addr learned my addr 172.21.15.112:0/4294119710 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:10.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.723+0000 7fe9daffd700 1 --2- 172.21.15.112:0/4294119710 >> v2:172.21.15.138:3300/0 conn(0x7fe9dc146be0 0x7fe9dc0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.723+0000 7fe9da7fc700 1 --2- 172.21.15.112:0/4294119710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9dc1619b0 0x7fe9dc0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.723+0000 7fe9d9ffb700 1 -- 172.21.15.112:0/4294119710 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187991205 0 0) 0x7fe9dc161e10 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.723+0000 7fe9d9ffb700 1 -- 172.21.15.112:0/4294119710 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9c40035b0 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.724+0000 7fe9daffd700 1 -- 172.21.15.112:0/4294119710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9dc1619b0 msgr2=0x7fe9dc0b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:10.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.724+0000 7fe9daffd700 1 --2- 172.21.15.112:0/4294119710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9dc1619b0 0x7fe9dc0b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.724+0000 7fe9daffd700 1 -- 172.21.15.112:0/4294119710 >> v1:172.21.15.112:6789/0 conn(0x7fe9dc0b52a0 legacy=0x7fe9dc15d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.724+0000 7fe9daffd700 1 -- 172.21.15.112:0/4294119710 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc0b3b70 con 0x7fe9dc146be0 2023-08-27T11:17:10.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.724+0000 7fe9da7fc700 1 --2- 172.21.15.112:0/4294119710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9dc1619b0 0x7fe9dc0b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:10.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.724+0000 7fe9daffd700 1 --2- 172.21.15.112:0/4294119710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9dc146be0 0x7fe9dc0b3440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009870 tx=0x7fe9d0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cde6d0faa1b15e4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:10.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.724+0000 7fe9d9ffb700 1 -- 172.21.15.112:0/4294119710 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0011830 con 0x7fe9dc146be0 2023-08-27T11:17:10.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.724+0000 7fe9d9ffb700 1 -- 172.21.15.112:0/4294119710 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d0011990 con 0x7fe9dc146be0 2023-08-27T11:17:10.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.724+0000 7fe9d9ffb700 1 -- 172.21.15.112:0/4294119710 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0011d20 con 0x7fe9dc146be0 2023-08-27T11:17:10.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.724+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/4294119710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9dc146be0 msgr2=0x7fe9dc0b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.724+0000 7fe9e1a4a700 1 --2- 172.21.15.112:0/4294119710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9dc146be0 0x7fe9dc0b3440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009870 tx=0x7fe9d0009b80 comp rx=0 tx=0).stop 2023-08-27T11:17:10.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.725+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/4294119710 shutdown_connections 2023-08-27T11:17:10.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.725+0000 7fe9e1a4a700 1 --2- 172.21.15.112:0/4294119710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9dc146be0 0x7fe9dc0b3440 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.725+0000 7fe9e1a4a700 1 --2- 172.21.15.112:0/4294119710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9dc1619b0 0x7fe9dc0b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.725+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/4294119710 >> 172.21.15.112:0/4294119710 conn(0x7fe9dc06c880 msgr2=0x7fe9dc0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:10.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.725+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/4294119710 shutdown_connections 2023-08-27T11:17:10.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.725+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/4294119710 wait complete. 2023-08-27T11:17:10.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.726+0000 7fe9e1a4a700 1 Processor -- start 2023-08-27T11:17:10.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.726+0000 7fe9e1a4a700 1 -- start start 2023-08-27T11:17:10.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9e1a4a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9dc0b52a0 0x7fe9dc1593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9e1a4a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9dc146be0 0x7fe9dc144710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9e1a4a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9dc1619b0 0x7fe9dc144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9e1a4a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc143330 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9e1a4a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc143610 con 0x7fe9dc1619b0 2023-08-27T11:17:10.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9e1a4a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc1438f0 con 0x7fe9dc146be0 2023-08-27T11:17:10.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9daffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9dc0b52a0 0x7fe9dc1593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9daffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9dc0b52a0 0x7fe9dc1593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54798/0 (socket says 172.21.15.112:54798) 2023-08-27T11:17:10.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9daffd700 1 -- 172.21.15.112:0/1469980076 learned_addr learned my addr 172.21.15.112:0/1469980076 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:10.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9da7fc700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9dc146be0 0x7fe9dc144710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9db7fe700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9dc1619b0 0x7fe9dc144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9daffd700 1 -- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9dc146be0 msgr2=0x7fe9dc144710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9daffd700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9dc146be0 0x7fe9dc144710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9daffd700 1 -- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9dc1619b0 msgr2=0x7fe9dc144c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9daffd700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9dc1619b0 0x7fe9dc144c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9daffd700 1 -- 172.21.15.112:0/1469980076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc0094a0 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9da7fc700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9dc146be0 0x7fe9dc144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:10.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9db7fe700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9dc1619b0 0x7fe9dc144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:10.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9daffd700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9dc0b52a0 0x7fe9dc1593d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0014040 tx=0x7fe9d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:10.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9c3fff700 1 -- 172.21.15.112:0/1469980076 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00036a0 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9c3fff700 1 -- 172.21.15.112:0/1469980076 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d001e070 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9c3fff700 1 -- 172.21.15.112:0/1469980076 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0003a20 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/1469980076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d00094a0 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.727+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/1469980076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc1425c0 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.728+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/1469980076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc067a20 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.728+0000 7fe9c3fff700 1 -- 172.21.15.112:0/1469980076 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe9d0003b80 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.729+0000 7fe9c3fff700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe9c806a0e0 0x7fe9c806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:10.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.729+0000 7fe9c3fff700 1 -- 172.21.15.112:0/1469980076 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9d00894f0 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.729+0000 7fe9da7fc700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe9c806a0e0 0x7fe9c806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:10.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.729+0000 7fe9da7fc700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe9c806a0e0 0x7fe9c806c5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc13b660 tx=0x7fe9cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:10.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.733+0000 7fe9c3fff700 1 -- 172.21.15.112:0/1469980076 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe9d0054c70 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.923+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/1469980076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9dc0d23f0 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.924+0000 7fe9c3fff700 1 -- 172.21.15.112:0/1469980076 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7fe9d0058290 con 0x7fe9dc0b52a0 2023-08-27T11:17:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:10.926 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:10.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.926+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe9c806a0e0 msgr2=0x7fe9c806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.926+0000 7fe9e1a4a700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe9c806a0e0 0x7fe9c806c5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc13b660 tx=0x7fe9cc008040 comp rx=0 tx=0).stop 2023-08-27T11:17:10.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.926+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/1469980076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9dc0b52a0 msgr2=0x7fe9dc1593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.926+0000 7fe9e1a4a700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9dc0b52a0 0x7fe9dc1593d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0014040 tx=0x7fe9d0004750 comp rx=0 tx=0).stop 2023-08-27T11:17:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.926+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/1469980076 shutdown_connections 2023-08-27T11:17:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.926+0000 7fe9e1a4a700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe9c806a0e0 0x7fe9c806c5a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.926+0000 7fe9e1a4a700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9dc1619b0 0x7fe9dc144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.926+0000 7fe9e1a4a700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9dc0b52a0 0x7fe9dc1593d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.926+0000 7fe9e1a4a700 1 --2- 172.21.15.112:0/1469980076 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9dc146be0 0x7fe9dc144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.926+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/1469980076 >> 172.21.15.112:0/1469980076 conn(0x7fe9dc06c880 msgr2=0x7fe9dc160e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.926+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/1469980076 shutdown_connections 2023-08-27T11:17:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:10.926+0000 7fe9e1a4a700 1 -- 172.21.15.112:0/1469980076 wait complete. 2023-08-27T11:17:10.940 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-08-27T11:17:10.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-08-27T11:17:10.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-08-27T11:17:10.942 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-08-27T11:17:10.943 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-08-27T11:17:10.943 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-08-27T11:17:10.944 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-08-27T11:17:10.944 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-08-27T11:17:11.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.150+0000 7f39b1ca7700 1 Processor -- start 2023-08-27T11:17:11.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.150+0000 7f39b1ca7700 1 -- start start 2023-08-27T11:17:11.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.151+0000 7fd5c7631700 1 Processor -- start 2023-08-27T11:17:11.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.151+0000 7f39b1ca7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39ac146be0 0x7f39ac0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.151+0000 7f39b1ca7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39ac1619b0 0x7f39ac0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.151+0000 7f39b1ca7700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ac161e10 con 0x7f39ac0b52a0 2023-08-27T11:17:11.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.151+0000 7f39b1ca7700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ac0b3980 con 0x7f39ac146be0 2023-08-27T11:17:11.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.151+0000 7f39b1ca7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0b5730 con 0x7f39ac1619b0 2023-08-27T11:17:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.151+0000 7fd5c7631700 1 -- start start 2023-08-27T11:17:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.152+0000 7f39aaffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39ac1619b0 0x7f39ac0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.152+0000 7f39aaffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39ac1619b0 0x7f39ac0b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38232/0 (socket says 172.21.15.112:38232) 2023-08-27T11:17:11.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.152+0000 7f39aaffd700 1 -- 172.21.15.112:0/4173296138 learned_addr learned my addr 172.21.15.112:0/4173296138 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.152+0000 7f39ab7fe700 1 --2- 172.21.15.112:0/4173296138 >> v2:172.21.15.138:3300/0 conn(0x7f39ac146be0 0x7f39ac0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.152+0000 7f1fc88c8700 1 Processor -- start 2023-08-27T11:17:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7fd5c7631700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c013bef0 0x7fd5c015d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7fd5c7631700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5c015db40 0x7fd5c00cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7fd5c7631700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c015ce90 con 0x7fd5c00b1500 2023-08-27T11:17:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7fd5c7631700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c0153de0 con 0x7fd5c015db40 2023-08-27T11:17:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7fd5c7631700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c013f920 con 0x7fd5c013bef0 2023-08-27T11:17:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7fd5c53cd700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd5c00b1500 0x7fd5c01598e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:56718/0 (socket says 172.21.15.112:56718) 2023-08-27T11:17:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7fd5c53cd700 1 -- 172.21.15.112:0/2486807158 learned_addr learned my addr 172.21.15.112:0/2486807158 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7fd5c4bcc700 1 --2- 172.21.15.112:0/2486807158 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c013bef0 0x7fd5c015d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.152+0000 7f1fc88c8700 1 -- start start 2023-08-27T11:17:11.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7f39ab7fe700 1 -- 172.21.15.112:0/4173296138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39ac1619b0 msgr2=0x7f39ac0b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7f39ab7fe700 1 --2- 172.21.15.112:0/4173296138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39ac1619b0 0x7f39ac0b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7f39ab7fe700 1 -- 172.21.15.112:0/4173296138 >> v1:172.21.15.138:6789/0 conn(0x7f39ac0b52a0 legacy=0x7f39ac15d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7f39ab7fe700 1 -- 172.21.15.112:0/4173296138 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac0b3b70 con 0x7f39ac146be0 2023-08-27T11:17:11.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7f39ab7fe700 1 --2- 172.21.15.112:0/4173296138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ac146be0 0x7f39ac0b3440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f399c009870 tx=0x7f399c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecec722e7aca921a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7f39aa7fc700 1 -- 172.21.15.112:0/4173296138 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c004230 con 0x7f39ac146be0 2023-08-27T11:17:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7fd5c5bce700 1 --2- 172.21.15.112:0/2486807158 >> v2:172.21.15.112:3300/0 conn(0x7fd5c015db40 0x7fd5c00cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7fd5c4bcc700 1 -- 172.21.15.112:0/2486807158 >> v2:172.21.15.112:3300/0 conn(0x7fd5c015db40 msgr2=0x7fd5c00cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7fd5c4bcc700 1 --2- 172.21.15.112:0/2486807158 >> v2:172.21.15.112:3300/0 conn(0x7fd5c015db40 0x7fd5c00cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7f1fc88c8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1fc406adb0 0x7f1fc406b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7f1fc88c8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc415eb60 con 0x7f1fc4151060 2023-08-27T11:17:11.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7f1fc88c8700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc40b02f0 con 0x7f1fc40cdbd0 2023-08-27T11:17:11.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.153+0000 7f1fc88c8700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc406b770 con 0x7f1fc406adb0 2023-08-27T11:17:11.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7f39aa7fc700 1 -- 172.21.15.112:0/4173296138 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c004390 con 0x7f39ac146be0 2023-08-27T11:17:11.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7f39aa7fc700 1 -- 172.21.15.112:0/4173296138 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c0036a0 con 0x7f39ac146be0 2023-08-27T11:17:11.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7fd5c4bcc700 1 -- 172.21.15.112:0/2486807158 >> v1:172.21.15.138:6789/0 conn(0x7fd5c00b1500 legacy=0x7fd5c01598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7fd5c4bcc700 1 -- 172.21.15.112:0/2486807158 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c015a3e0 con 0x7fd5c013bef0 2023-08-27T11:17:11.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7fd5c4bcc700 1 --2- 172.21.15.112:0/2486807158 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c013bef0 0x7fd5c015d600 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009840 tx=0x7fd5bc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ab3252d83cfb6c0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7f1fc259c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1fc4151060 0x7f1fc415e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49200/0 (socket says 172.21.15.112:49200) 2023-08-27T11:17:11.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7f1fc259c700 1 -- 172.21.15.112:0/2496755001 learned_addr learned my addr 172.21.15.112:0/2496755001 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:11.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7f1fc1d9b700 1 --2- 172.21.15.112:0/2496755001 >> v2:172.21.15.138:3300/0 conn(0x7f1fc406adb0 0x7f1fc406b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7fd5b7fff700 1 -- 172.21.15.112:0/2486807158 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc004230 con 0x7fd5c013bef0 2023-08-27T11:17:11.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7f1fc159a700 1 -- 172.21.15.112:0/2496755001 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046925260 0 0) 0x7f1fc415eb60 con 0x7f1fc4151060 2023-08-27T11:17:11.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7f1fc159a700 1 -- 172.21.15.112:0/2496755001 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fac0035b0 con 0x7f1fc4151060 2023-08-27T11:17:11.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7f1fc159a700 1 -- 172.21.15.112:0/2496755001 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254352780 0 0) 0x7f1fc40b02f0 con 0x7f1fc40cdbd0 2023-08-27T11:17:11.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7f1fc159a700 1 -- 172.21.15.112:0/2496755001 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc415eb60 con 0x7f1fc40cdbd0 2023-08-27T11:17:11.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7f39b1ca7700 1 -- 172.21.15.112:0/4173296138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ac146be0 msgr2=0x7f39ac0b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7f39b1ca7700 1 --2- 172.21.15.112:0/4173296138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ac146be0 0x7f39ac0b3440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f399c009870 tx=0x7f399c009b80 comp rx=0 tx=0).stop 2023-08-27T11:17:11.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7f39b1ca7700 1 -- 172.21.15.112:0/4173296138 shutdown_connections 2023-08-27T11:17:11.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7f39b1ca7700 1 --2- 172.21.15.112:0/4173296138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ac146be0 0x7f39ac0b3440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7f39b1ca7700 1 --2- 172.21.15.112:0/4173296138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39ac1619b0 0x7f39ac0b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7f39b1ca7700 1 -- 172.21.15.112:0/4173296138 >> 172.21.15.112:0/4173296138 conn(0x7f39ac06c880 msgr2=0x7f39ac0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7f1fc1d9b700 1 -- 172.21.15.112:0/2496755001 >> v1:172.21.15.138:6789/0 conn(0x7f1fc40cdbd0 legacy=0x7f1fc40ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7f1fc1d9b700 1 -- 172.21.15.112:0/2496755001 >> v1:172.21.15.112:6789/0 conn(0x7f1fc4151060 legacy=0x7f1fc415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7f1fc1d9b700 1 -- 172.21.15.112:0/2496755001 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc40b05d0 con 0x7f1fc406adb0 2023-08-27T11:17:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7f39b1ca7700 1 -- 172.21.15.112:0/4173296138 shutdown_connections 2023-08-27T11:17:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7f39b1ca7700 1 -- 172.21.15.112:0/4173296138 wait complete. 2023-08-27T11:17:11.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7f7263020700 1 Processor -- start 2023-08-27T11:17:11.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.154+0000 7f7263020700 1 -- start start 2023-08-27T11:17:11.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7f1fc1d9b700 1 --2- 172.21.15.112:0/2496755001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc406adb0 0x7f1fc406b230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400bae0 tx=0x7f1fb400bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95db40d0838dd307 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.156+0000 7f1fc159a700 1 -- 172.21.15.112:0/2496755001 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400dc70 con 0x7f1fc406adb0 2023-08-27T11:17:11.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.156+0000 7f1fc159a700 1 -- 172.21.15.112:0/2496755001 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb400ddd0 con 0x7f1fc406adb0 2023-08-27T11:17:11.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.156+0000 7f1fc159a700 1 -- 172.21.15.112:0/2496755001 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4010690 con 0x7f1fc406adb0 2023-08-27T11:17:11.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7fd5b7fff700 1 -- 172.21.15.112:0/2486807158 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5bc004390 con 0x7fd5c013bef0 2023-08-27T11:17:11.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.155+0000 7fd5b7fff700 1 -- 172.21.15.112:0/2486807158 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0036a0 con 0x7fd5c013bef0 2023-08-27T11:17:11.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f1fc88c8700 1 -- 172.21.15.112:0/2496755001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc406adb0 msgr2=0x7f1fc406b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f1fc88c8700 1 --2- 172.21.15.112:0/2496755001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc406adb0 0x7f1fc406b230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400bae0 tx=0x7f1fb400bdf0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f1fc88c8700 1 -- 172.21.15.112:0/2496755001 shutdown_connections 2023-08-27T11:17:11.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f1fc88c8700 1 --2- 172.21.15.112:0/2496755001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc406adb0 0x7f1fc406b230 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f1fc88c8700 1 -- 172.21.15.112:0/2496755001 >> 172.21.15.112:0/2496755001 conn(0x7f1fc40d0000 msgr2=0x7f1fc40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.156+0000 7f7263020700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f725c161c90 0x7f725c0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.156+0000 7f7263020700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c1652d0 con 0x7f725c147c70 2023-08-27T11:17:11.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.156+0000 7f7263020700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c162210 con 0x7f725c147890 2023-08-27T11:17:11.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.156+0000 7f7263020700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c0708a0 con 0x7f725c161c90 2023-08-27T11:17:11.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.156+0000 7f72615bd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f725c161c90 0x7f725c0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.156+0000 7f72615bd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f725c161c90 0x7f725c0b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54826/0 (socket says 172.21.15.112:54826) 2023-08-27T11:17:11.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.156+0000 7f72615bd700 1 -- 172.21.15.112:0/3130931666 learned_addr learned my addr 172.21.15.112:0/3130931666 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.156+0000 7f72615bd700 1 -- 172.21.15.112:0/3130931666 >> v1:172.21.15.138:6789/0 conn(0x7f725c147890 legacy=0x7f725c06c650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:17:11.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f72615bd700 1 -- 172.21.15.112:0/3130931666 >> v1:172.21.15.112:6789/0 conn(0x7f725c147c70 legacy=0x7f725c070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f72615bd700 1 -- 172.21.15.112:0/3130931666 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c1624f0 con 0x7f725c161c90 2023-08-27T11:17:11.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f72615bd700 1 --2- 172.21.15.112:0/3130931666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f725c161c90 0x7f725c0b2050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f72500098f0 tx=0x7f7250009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb4c9cd2f889821 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f39b1ca7700 1 Processor -- start 2023-08-27T11:17:11.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f1fc88c8700 1 -- 172.21.15.112:0/2496755001 shutdown_connections 2023-08-27T11:17:11.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f39b1ca7700 1 -- start start 2023-08-27T11:17:11.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7fd5c7631700 1 -- 172.21.15.112:0/2486807158 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c013bef0 msgr2=0x7fd5c015d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7fd5c7631700 1 --2- 172.21.15.112:0/2486807158 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c013bef0 0x7fd5c015d600 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009840 tx=0x7fd5bc009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:11.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7fd5c7631700 1 -- 172.21.15.112:0/2486807158 shutdown_connections 2023-08-27T11:17:11.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7fd5c7631700 1 --2- 172.21.15.112:0/2486807158 >> v2:172.21.15.112:3300/0 conn(0x7fd5c015db40 0x7fd5c00cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7fd5c7631700 1 --2- 172.21.15.112:0/2486807158 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c013bef0 0x7fd5c015d600 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7fd5c7631700 1 -- 172.21.15.112:0/2486807158 >> 172.21.15.112:0/2486807158 conn(0x7fd5c00b0a10 msgr2=0x7fd5c00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f1fc88c8700 1 -- 172.21.15.112:0/2496755001 wait complete. 2023-08-27T11:17:11.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7fd5c7631700 1 -- 172.21.15.112:0/2486807158 shutdown_connections 2023-08-27T11:17:11.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f39b1ca7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ac0b52a0 0x7f39ac0d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f39b1ca7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39ac146be0 0x7f39ac0b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f39b1ca7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39ac1619b0 0x7f39ac0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f39b1ca7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0b2740 con 0x7f39ac1619b0 2023-08-27T11:17:11.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f39b1ca7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac08afb0 con 0x7f39ac0b52a0 2023-08-27T11:17:11.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f39b1ca7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac08b200 con 0x7f39ac146be0 2023-08-27T11:17:11.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f9d063ee700 1 Processor -- start 2023-08-27T11:17:11.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f9d063ee700 1 -- start start 2023-08-27T11:17:11.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f39abfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39ac1619b0 0x7f39ac0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f39abfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39ac1619b0 0x7f39ac0b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54838/0 (socket says 172.21.15.112:54838) 2023-08-27T11:17:11.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f39abfff700 1 -- 172.21.15.112:0/1363003614 learned_addr learned my addr 172.21.15.112:0/1363003614 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f39aaffd700 1 --2- 172.21.15.112:0/1363003614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39ac146be0 0x7f39ac0b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.156+0000 7f8bdeda3700 1 Processor -- start 2023-08-27T11:17:11.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.156+0000 7f8bdeda3700 1 -- start start 2023-08-27T11:17:11.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f8bdeda3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8147890 0x7f8bd806c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f8bdeda3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8bd8161c90 0x7f8bd80b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f8bdeda3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd81652d0 con 0x7f8bd8147c70 2023-08-27T11:17:11.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f8bdeda3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd80708a0 con 0x7f8bd8161c90 2023-08-27T11:17:11.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f8bdeda3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8162210 con 0x7f8bd8147890 2023-08-27T11:17:11.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f8bdd340700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8bd8161c90 0x7f8bd80b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f8bdd340700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8bd8161c90 0x7f8bd80b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54832/0 (socket says 172.21.15.112:54832) 2023-08-27T11:17:11.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.157+0000 7f8bdd340700 1 -- 172.21.15.112:0/1200362328 learned_addr learned my addr 172.21.15.112:0/1200362328 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7f8bdcb3f700 1 --2- 172.21.15.112:0/1200362328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8147890 0x7f8bd806c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7f8bdd340700 1 -- 172.21.15.112:0/1200362328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8147890 msgr2=0x7f8bd806c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7f8bdd340700 1 --2- 172.21.15.112:0/1200362328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8147890 0x7f8bd806c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7f8bdd340700 1 -- 172.21.15.112:0/1200362328 >> v1:172.21.15.112:6789/0 conn(0x7f8bd8147c70 legacy=0x7f8bd8070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7f8bdd340700 1 -- 172.21.15.112:0/1200362328 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd81624f0 con 0x7f8bd8161c90 2023-08-27T11:17:11.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7f8bdcb3f700 1 --2- 172.21.15.112:0/1200362328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8147890 0x7f8bd806c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:11.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7f8bdd340700 1 --2- 172.21.15.112:0/1200362328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd8161c90 0x7f8bd80b2050 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009e80 tx=0x7f8bd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f3b792d764bcc5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7f8bd77fe700 1 -- 172.21.15.112:0/1200362328 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd000e070 con 0x7f8bd8161c90 2023-08-27T11:17:11.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7f8bd77fe700 1 -- 172.21.15.112:0/1200362328 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd0003a20 con 0x7f8bd8161c90 2023-08-27T11:17:11.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.158+0000 7f8bd77fe700 1 -- 172.21.15.112:0/1200362328 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0003dd0 con 0x7f8bd8161c90 2023-08-27T11:17:11.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f1fc88c8700 1 Processor -- start 2023-08-27T11:17:11.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f39abfff700 1 -- 172.21.15.112:0/1363003614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39ac146be0 msgr2=0x7f39ac0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f39abfff700 1 --2- 172.21.15.112:0/1363003614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39ac146be0 0x7f39ac0b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f39abfff700 1 -- 172.21.15.112:0/1363003614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ac0b52a0 msgr2=0x7f39ac0d0c70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-27T11:17:11.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f39abfff700 1 --2- 172.21.15.112:0/1363003614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ac0b52a0 0x7f39ac0d0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f39abfff700 1 -- 172.21.15.112:0/1363003614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c0094a0 con 0x7f39ac1619b0 2023-08-27T11:17:11.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f725b7fe700 1 -- 172.21.15.112:0/3130931666 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7250004230 con 0x7f725c161c90 2023-08-27T11:17:11.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f725b7fe700 1 -- 172.21.15.112:0/3130931666 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7250004390 con 0x7f725c161c90 2023-08-27T11:17:11.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f725b7fe700 1 -- 172.21.15.112:0/3130931666 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72500036a0 con 0x7f725c161c90 2023-08-27T11:17:11.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f39abfff700 1 --2- 172.21.15.112:0/1363003614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39ac1619b0 0x7f39ac0b2200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c870 tx=0x7f39a400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f8bdeda3700 1 -- 172.21.15.112:0/1200362328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd8161c90 msgr2=0x7f8bd80b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f8bdeda3700 1 --2- 172.21.15.112:0/1200362328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd8161c90 0x7f8bd80b2050 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009e80 tx=0x7f8bd000b330 comp rx=0 tx=0).stop 2023-08-27T11:17:11.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f8bdeda3700 1 -- 172.21.15.112:0/1200362328 shutdown_connections 2023-08-27T11:17:11.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f8bdeda3700 1 --2- 172.21.15.112:0/1200362328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd8161c90 0x7f8bd80b2050 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f8bdeda3700 1 --2- 172.21.15.112:0/1200362328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8147890 0x7f8bd806c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f8bdeda3700 1 -- 172.21.15.112:0/1200362328 >> 172.21.15.112:0/1200362328 conn(0x7f8bd80b0a10 msgr2=0x7f8bd80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f8bdeda3700 1 -- 172.21.15.112:0/1200362328 shutdown_connections 2023-08-27T11:17:11.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f8bdeda3700 1 -- 172.21.15.112:0/1200362328 wait complete. 2023-08-27T11:17:11.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f3993fff700 1 -- 172.21.15.112:0/1363003614 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4003e30 con 0x7f39ac1619b0 2023-08-27T11:17:11.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f3993fff700 1 -- 172.21.15.112:0/1363003614 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a400ce70 con 0x7f39ac1619b0 2023-08-27T11:17:11.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f3993fff700 1 -- 172.21.15.112:0/1363003614 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4005270 con 0x7f39ac1619b0 2023-08-27T11:17:11.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f8bdeda3700 1 Processor -- start 2023-08-27T11:17:11.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f8bdeda3700 1 -- start start 2023-08-27T11:17:11.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7fd5c7631700 1 -- 172.21.15.112:0/2486807158 wait complete. 2023-08-27T11:17:11.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7fd5c7631700 1 Processor -- start 2023-08-27T11:17:11.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.159+0000 7f1fc88c8700 1 -- start start 2023-08-27T11:17:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f1fc88c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc406adb0 0x7f1fc4151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc88c8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fc40cdbd0 0x7f1fc41f9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc88c8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fc4151060 0x7f1fc41fa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc88c8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc40c7a30 con 0x7f1fc4151060 2023-08-27T11:17:11.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc88c8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc4154650 con 0x7f1fc406adb0 2023-08-27T11:17:11.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc88c8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc4154930 con 0x7f1fc40cdbd0 2023-08-27T11:17:11.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc259c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc406adb0 0x7f1fc4151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc259c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc406adb0 0x7f1fc4151990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55268/0 (socket says 172.21.15.112:55268) 2023-08-27T11:17:11.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc259c700 1 -- 172.21.15.112:0/942817423 learned_addr learned my addr 172.21.15.112:0/942817423 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc2d9d700 1 --2- 172.21.15.112:0/942817423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fc4151060 0x7f1fc41fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc259c700 1 -- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fc40cdbd0 msgr2=0x7f1fc41f9c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-27T11:17:11.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc259c700 1 --2- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fc40cdbd0 0x7f1fc41f9c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc259c700 1 -- 172.21.15.112:0/942817423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fc4151060 msgr2=0x7f1fc41fa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc259c700 1 --2- 172.21.15.112:0/942817423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fc4151060 0x7f1fc41fa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc259c700 1 -- 172.21.15.112:0/942817423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb400b740 con 0x7f1fc406adb0 2023-08-27T11:17:11.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f1fc259c700 1 --2- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc406adb0 0x7f1fc4151990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00cbc0 tx=0x7f1fbc00ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f7263020700 1 -- 172.21.15.112:0/3130931666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f725c161c90 msgr2=0x7f725c0b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f7263020700 1 --2- 172.21.15.112:0/3130931666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f725c161c90 0x7f725c0b2050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f72500098f0 tx=0x7f7250009c00 comp rx=0 tx=0).stop 2023-08-27T11:17:11.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f7263020700 1 -- 172.21.15.112:0/3130931666 shutdown_connections 2023-08-27T11:17:11.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f7263020700 1 --2- 172.21.15.112:0/3130931666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f725c161c90 0x7f725c0b2050 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f7263020700 1 -- 172.21.15.112:0/3130931666 >> 172.21.15.112:0/3130931666 conn(0x7f725c0b0a10 msgr2=0x7f725c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f39b1ca7700 1 -- 172.21.15.112:0/1363003614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac08b4e0 con 0x7f39ac1619b0 2023-08-27T11:17:11.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.160+0000 7f39b1ca7700 1 -- 172.21.15.112:0/1363003614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac08baa0 con 0x7f39ac1619b0 2023-08-27T11:17:11.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7f39b1ca7700 1 -- 172.21.15.112:0/1363003614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f39ac0b4130 con 0x7f39ac1619b0 2023-08-27T11:17:11.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.161+0000 7fd5c7631700 1 -- start start 2023-08-27T11:17:11.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f7263020700 1 -- 172.21.15.112:0/3130931666 shutdown_connections 2023-08-27T11:17:11.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f7263020700 1 -- 172.21.15.112:0/3130931666 wait complete. 2023-08-27T11:17:11.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7fd5c7631700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c00b1500 0x7fd5c0154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7fd5c7631700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c013bef0 0x7fd5c00aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7fd5c7631700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5c013b7d0 0x7fd5c015a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7fd5c7631700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00af460 con 0x7fd5c00b1500 2023-08-27T11:17:11.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7fd5c7631700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c013d840 con 0x7fd5c013b7d0 2023-08-27T11:17:11.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7fd5c7631700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0155080 con 0x7fd5c013bef0 2023-08-27T11:17:11.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7fd5c53cd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c00b1500 0x7fd5c0154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7fd5c53cd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c00b1500 0x7fd5c0154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54850/0 (socket says 172.21.15.112:54850) 2023-08-27T11:17:11.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7fd5c53cd700 1 -- 172.21.15.112:0/2842995552 learned_addr learned my addr 172.21.15.112:0/2842995552 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f1fb2ffd700 1 -- 172.21.15.112:0/942817423 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc00ab00 con 0x7f1fc406adb0 2023-08-27T11:17:11.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f1fb2ffd700 1 -- 172.21.15.112:0/942817423 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc00ac80 con 0x7f1fc406adb0 2023-08-27T11:17:11.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f1fb2ffd700 1 -- 172.21.15.112:0/942817423 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc012400 con 0x7f1fc406adb0 2023-08-27T11:17:11.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f1fc88c8700 1 -- 172.21.15.112:0/942817423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc40c7c80 con 0x7f1fc406adb0 2023-08-27T11:17:11.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7fd5c5bce700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5c013b7d0 0x7fd5c015a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7fd5c4bcc700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c013bef0 0x7fd5c00aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7fd5c53cd700 1 -- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c013bef0 msgr2=0x7fd5c00aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7fd5c53cd700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c013bef0 0x7fd5c00aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7fd5c53cd700 1 -- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5c013b7d0 msgr2=0x7fd5c015a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7fd5c53cd700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5c013b7d0 0x7fd5c015a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7fd5c53cd700 1 -- 172.21.15.112:0/2842995552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc0094a0 con 0x7fd5c00b1500 2023-08-27T11:17:11.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f8bdeda3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd8147890 0x7f8bd8158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f8bdeda3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8147c70 0x7f8bd8159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f8bdeda3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bd8161c90 0x7f8bd8157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f8bdeda3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd81593f0 con 0x7f8bd8147890 2023-08-27T11:17:11.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f8bdeda3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8140560 con 0x7f8bd8161c90 2023-08-27T11:17:11.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f8bdeda3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd81409f0 con 0x7f8bd8147c70 2023-08-27T11:17:11.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7fd5c53cd700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c00b1500 0x7fd5c0154a30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8002080 tx=0x7fd5b800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f8bdcb3f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd8147890 0x7f8bd8158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f8bdcb3f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd8147890 0x7f8bd8158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54860/0 (socket says 172.21.15.112:54860) 2023-08-27T11:17:11.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f8bdcb3f700 1 -- 172.21.15.112:0/2336482638 learned_addr learned my addr 172.21.15.112:0/2336482638 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7fd5b5ffb700 1 -- 172.21.15.112:0/2842995552 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b800ad40 con 0x7fd5c00b1500 2023-08-27T11:17:11.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7fd5b5ffb700 1 -- 172.21.15.112:0/2842995552 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b80070d0 con 0x7fd5c00b1500 2023-08-27T11:17:11.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7fd5b5ffb700 1 -- 172.21.15.112:0/2842995552 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b800aea0 con 0x7fd5c00b1500 2023-08-27T11:17:11.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7fd5c7631700 1 -- 172.21.15.112:0/2842995552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c0155360 con 0x7fd5c00b1500 2023-08-27T11:17:11.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f8bdcb3f700 1 -- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8147c70 msgr2=0x7f8bd8159860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-27T11:17:11.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f8bd7fff700 1 --2- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8147c70 0x7f8bd8159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f1fc88c8700 1 -- 172.21.15.112:0/942817423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc414cb80 con 0x7f1fc406adb0 2023-08-27T11:17:11.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f1fc88c8700 1 -- 172.21.15.112:0/942817423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1fa8000fc0 con 0x7f1fc406adb0 2023-08-27T11:17:11.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f9d063ee700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9d00146c00 0x7f9d0015e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f9d063ee700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d00066720 0x7f9d000708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f9d063ee700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d00072250 con 0x7f9d000cec70 2023-08-27T11:17:11.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f9d063ee700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d000d0b60 con 0x7f9d00146c00 2023-08-27T11:17:11.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.162+0000 7f9d063ee700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d000724a0 con 0x7f9d00066720 2023-08-27T11:17:11.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f9cff7fe700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9d000cec70 0x7f9d00066010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:56764/0 (socket says 172.21.15.112:56764) 2023-08-27T11:17:11.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f9cff7fe700 1 -- 172.21.15.112:0/1420414807 learned_addr learned my addr 172.21.15.112:0/1420414807 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:11.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865276932 0 0) 0x7f9d00072250 con 0x7f9d000cec70 2023-08-27T11:17:11.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce80035b0 con 0x7f9d000cec70 2023-08-27T11:17:11.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1414505804 0 0) 0x7f9ce80035b0 con 0x7f9d000cec70 2023-08-27T11:17:11.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d00066720 msgr2=0x7f9d000708a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-27T11:17:11.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f9cfeffd700 1 --2- 172.21.15.112:0/1420414807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d00066720 0x7f9d000708a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 >> v2:172.21.15.112:3300/0 conn(0x7f9d00146c00 msgr2=0x7f9d0015e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cfeffd700 1 --2- 172.21.15.112:0/1420414807 >> v2:172.21.15.112:3300/0 conn(0x7f9d00146c00 0x7f9d0015e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d00072780 con 0x7f9d000cec70 2023-08-27T11:17:11.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f9cf80019f0 con 0x7f9d000cec70 2023-08-27T11:17:11.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 >> v1:172.21.15.138:6789/0 conn(0x7f9d000cec70 legacy=0x7f9d00066010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cfeffd700 1 --2- 172.21.15.112:0/1420414807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ce8003a00 0x7f9ce8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cfeffd700 1 --2- 172.21.15.112:0/1420414807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ce8004850 0x7f9ce8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cfeffd700 1 --2- 172.21.15.112:0/1420414807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ce8005410 0x7f9ce801ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d000d0b60 con 0x7f9ce8003a00 2023-08-27T11:17:11.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d000724a0 con 0x7f9ce8005410 2023-08-27T11:17:11.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce801d3a0 con 0x7f9ce8004850 2023-08-27T11:17:11.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cff7fe700 1 --2- 172.21.15.112:0/1420414807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ce8004850 0x7f9ce8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f8bdcb3f700 1 --2- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bd8147c70 0x7f8bd8159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f8bdcb3f700 1 -- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bd8161c90 msgr2=0x7f8bd8157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f8bdcb3f700 1 --2- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bd8161c90 0x7f8bd8157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7f8bdcb3f700 1 -- 172.21.15.112:0/2336482638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd0009b30 con 0x7f8bd8147890 2023-08-27T11:17:11.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f8bdcb3f700 1 --2- 172.21.15.112:0/2336482638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd8147890 0x7f8bd8158b30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800c750 tx=0x7f8bc800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f8bd57fa700 1 -- 172.21.15.112:0/2336482638 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8011800 con 0x7f8bd8147890 2023-08-27T11:17:11.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f8bd57fa700 1 -- 172.21.15.112:0/2336482638 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc8011960 con 0x7f8bd8147890 2023-08-27T11:17:11.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f8bd57fa700 1 -- 172.21.15.112:0/2336482638 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8011cd0 con 0x7f8bd8147890 2023-08-27T11:17:11.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f7263020700 1 Processor -- start 2023-08-27T11:17:11.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.163+0000 7fd5c7631700 1 -- 172.21.15.112:0/2842995552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c013bc80 con 0x7fd5c00b1500 2023-08-27T11:17:11.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cfffff700 1 --2- 172.21.15.112:0/1420414807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ce8003a00 0x7f9ce8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cff7fe700 1 -- 172.21.15.112:0/1420414807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ce8005410 msgr2=0x7f9ce801ca40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-27T11:17:11.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cff7fe700 1 --2- 172.21.15.112:0/1420414807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ce8005410 0x7f9ce801ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cff7fe700 1 -- 172.21.15.112:0/1420414807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ce8003a00 msgr2=0x7f9ce8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cff7fe700 1 --2- 172.21.15.112:0/1420414807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ce8003a00 0x7f9ce8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cff7fe700 1 -- 172.21.15.112:0/1420414807 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce80035b0 con 0x7f9ce8004850 2023-08-27T11:17:11.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f9cff7fe700 1 --2- 172.21.15.112:0/1420414807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ce8004850 0x7f9ce8004cd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80097b0 tx=0x7f9ce8005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd2dfad936b20e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf800e6f0 con 0x7f9ce8004850 2023-08-27T11:17:11.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf800e850 con 0x7f9ce8004850 2023-08-27T11:17:11.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7f9cfeffd700 1 -- 172.21.15.112:0/1420414807 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf800ec00 con 0x7f9ce8004850 2023-08-27T11:17:11.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7fb50312d700 1 Processor -- start 2023-08-27T11:17:11.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7f3993fff700 1 -- 172.21.15.112:0/1363003614 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f39a40053d0 con 0x7f39ac1619b0 2023-08-27T11:17:11.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7fb50312d700 1 -- start start 2023-08-27T11:17:11.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.164+0000 7f7263020700 1 -- start start 2023-08-27T11:17:11.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7f7263020700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f725c147890 0x7f725c0aa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7f7263020700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f725c147c70 0x7f725c0ac410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7f7263020700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f725c161c90 0x7f725c0a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7f7263020700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c1647a0 con 0x7f725c147c70 2023-08-27T11:17:11.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7f7263020700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0cc0a0 con 0x7f725c161c90 2023-08-27T11:17:11.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7f7263020700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0cc380 con 0x7f725c147890 2023-08-27T11:17:11.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7f7260dbc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f725c147890 0x7f725c0aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.165+0000 7f72615bd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f725c161c90 0x7f725c0a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f725bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f725c147c70 0x7f725c0ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f72615bd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f725c161c90 0x7f725c0a8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55298/0 (socket says 172.21.15.112:55298) 2023-08-27T11:17:11.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f72615bd700 1 -- 172.21.15.112:0/3752408818 learned_addr learned my addr 172.21.15.112:0/3752408818 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f3993fff700 1 --2- 172.21.15.112:0/1363003614 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f399806a0e0 0x7f399806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f72615bd700 1 -- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f725c147890 msgr2=0x7f725c0aa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f72615bd700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f725c147890 0x7f725c0aa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f72615bd700 1 -- 172.21.15.112:0/3752408818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f725c147c70 msgr2=0x7f725c0ac410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f72615bd700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f725c147c70 0x7f725c0ac410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f72615bd700 1 -- 172.21.15.112:0/3752408818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7250009550 con 0x7f725c161c90 2023-08-27T11:17:11.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f3993fff700 1 -- 172.21.15.112:0/1363003614 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39a4087c30 con 0x7f39ac1619b0 2023-08-27T11:17:11.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f725bfff700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f725c147c70 0x7f725c0ac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:11.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f7260dbc700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f725c147890 0x7f725c0aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:11.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f3993fff700 1 --2- 172.21.15.112:0/1363003614 >> [v2:172.21.15.112:6816/3936386115,v1:172.21.15.112:6817/3936386115] conn(0x7f3998072920 0x7f3998074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f3993fff700 1 -- 172.21.15.112:0/1363003614 --> [v2:172.21.15.112:6816/3936386115,v1:172.21.15.112:6817/3936386115] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39980753f0 con 0x7f3998072920 2023-08-27T11:17:11.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f3993fff700 1 -- 172.21.15.112:0/1363003614 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39a4087fa0 con 0x7f39ac1619b0 2023-08-27T11:17:11.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f72615bd700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f725c161c90 0x7f725c0a8e90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7250004000 tx=0x7f7250004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f39aaffd700 1 --2- 172.21.15.112:0/1363003614 >> [v2:172.21.15.112:6816/3936386115,v1:172.21.15.112:6817/3936386115] conn(0x7f3998072920 0x7f3998074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f72597fa700 1 -- 172.21.15.112:0/3752408818 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7250003e60 con 0x7f725c161c90 2023-08-27T11:17:11.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f72597fa700 1 -- 172.21.15.112:0/3752408818 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7250011630 con 0x7f725c161c90 2023-08-27T11:17:11.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f7263020700 1 -- 172.21.15.112:0/3752408818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c0cc660 con 0x7f725c161c90 2023-08-27T11:17:11.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7f72597fa700 1 -- 172.21.15.112:0/3752408818 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72500119c0 con 0x7f725c161c90 2023-08-27T11:17:11.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7f39ab7fe700 1 --2- 172.21.15.112:0/1363003614 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f399806a0e0 0x7f399806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fb50312d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb4fc0ccc00 0x7fb4fc071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fb50312d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4fc0b2910 0x7fb4fc0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fb50312d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4fc15e690 con 0x7fb4fc0cbf30 2023-08-27T11:17:11.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fb50312d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4fc0ccfe0 con 0x7fb4fc0ccc00 2023-08-27T11:17:11.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fb50312d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc15daa0 con 0x7fb4fc0b2910 2023-08-27T11:17:11.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fb500ec9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb4fc0ccc00 0x7fb4fc071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7f9d063ee700 1 -- 172.21.15.112:0/1420414807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ce8004850 msgr2=0x7f9ce8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7f9d063ee700 1 --2- 172.21.15.112:0/1420414807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ce8004850 0x7f9ce8004cd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80097b0 tx=0x7f9ce8005880 comp rx=0 tx=0).stop 2023-08-27T11:17:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7f9d0498b700 1 -- 172.21.15.112:0/1420414807 reap_dead start 2023-08-27T11:17:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fb500ec9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb4fc0ccc00 0x7fb4fc071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55306/0 (socket says 172.21.15.112:55306) 2023-08-27T11:17:11.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fb500ec9700 1 -- 172.21.15.112:0/3014470614 learned_addr learned my addr 172.21.15.112:0/3014470614 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fb5016ca700 1 --2- 172.21.15.112:0/3014470614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4fc0b2910 0x7fb4fc0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7f7263020700 1 -- 172.21.15.112:0/3752408818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c0ccc20 con 0x7f725c161c90 2023-08-27T11:17:11.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f8bdeda3700 1 -- 172.21.15.112:0/2336482638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd813b650 con 0x7f8bd8147890 2023-08-27T11:17:11.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.166+0000 7f8bdeda3700 1 -- 172.21.15.112:0/2336482638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd813ff10 con 0x7f8bd8147890 2023-08-27T11:17:11.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fd5c7631700 1 -- 172.21.15.112:0/2842995552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5a0000fc0 con 0x7fd5c00b1500 2023-08-27T11:17:11.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.168+0000 7f1fb2ffd700 1 -- 172.21.15.112:0/942817423 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1fbc01b3b0 con 0x7f1fc406adb0 2023-08-27T11:17:11.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7f39ab7fe700 1 --2- 172.21.15.112:0/1363003614 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f399806a0e0 0x7f399806c5a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f399c000c00 tx=0x7f399c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.168+0000 7f39aaffd700 1 --2- 172.21.15.112:0/1363003614 >> [v2:172.21.15.112:6816/3936386115,v1:172.21.15.112:6817/3936386115] conn(0x7f3998072920 0x7f3998074d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f3993fff700 1 -- 172.21.15.112:0/1363003614 <== osd.0 v2:172.21.15.112:6816/3936386115 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f39980753f0 con 0x7f3998072920 2023-08-27T11:17:11.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fb5016ca700 1 -- 172.21.15.112:0/3014470614 >> v2:172.21.15.138:3300/0 conn(0x7fb4fc0ccc00 msgr2=0x7fb4fc071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fb5016ca700 1 --2- 172.21.15.112:0/3014470614 >> v2:172.21.15.138:3300/0 conn(0x7fb4fc0ccc00 0x7fb4fc071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fb5016ca700 1 -- 172.21.15.112:0/3014470614 >> v1:172.21.15.112:6789/0 conn(0x7fb4fc0cbf30 legacy=0x7fb4fc0b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.167+0000 7fb5016ca700 1 -- 172.21.15.112:0/3014470614 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc15dd60 con 0x7fb4fc0b2910 2023-08-27T11:17:11.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7fb5016ca700 1 --2- 172.21.15.112:0/3014470614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4fc0b2910 0x7fb4fc0ce3e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009d90 tx=0x7fb4f400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=532e4858ba6a613 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7fb4fb7fe700 1 -- 172.21.15.112:0/3014470614 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f400bce0 con 0x7fb4fc0b2910 2023-08-27T11:17:11.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7fb4fb7fe700 1 -- 172.21.15.112:0/3014470614 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f400be40 con 0x7fb4fc0b2910 2023-08-27T11:17:11.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.168+0000 7f8bdeda3700 1 -- 172.21.15.112:0/2336482638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8bd8162ab0 con 0x7f8bd8147890 2023-08-27T11:17:11.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7fd5b5ffb700 1 -- 172.21.15.112:0/2842995552 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd5b8007480 con 0x7fd5c00b1500 2023-08-27T11:17:11.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.168+0000 7f1fb2ffd700 1 --2- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1fa406a1b0 0x7f1fa406c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.168+0000 7f1fb2ffd700 1 -- 172.21.15.112:0/942817423 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1fbc088a50 con 0x7f1fc406adb0 2023-08-27T11:17:11.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.168+0000 7f1fb2ffd700 1 --2- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:6808/570347350,v1:172.21.15.138:6809/570347350] conn(0x7f1fa4072a70 0x7f1fa4074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.168+0000 7f1fb2ffd700 1 -- 172.21.15.112:0/942817423 --> [v2:172.21.15.138:6808/570347350,v1:172.21.15.138:6809/570347350] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1fa4075540 con 0x7f1fa4072a70 2023-08-27T11:17:11.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.168+0000 7f1fb2ffd700 1 -- 172.21.15.112:0/942817423 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fbc088e00 con 0x7f1fc406adb0 2023-08-27T11:17:11.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f1fc1d9b700 1 --2- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1fa406a1b0 0x7f1fa406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f1fc2d9d700 1 --2- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:6808/570347350,v1:172.21.15.138:6809/570347350] conn(0x7f1fa4072a70 0x7f1fa4074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.168+0000 7f7263020700 1 -- 172.21.15.112:0/3752408818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f723c000fc0 con 0x7f725c161c90 2023-08-27T11:17:11.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f72597fa700 1 -- 172.21.15.112:0/3752408818 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7250021070 con 0x7f725c161c90 2023-08-27T11:17:11.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f9d063ee700 1 -- 172.21.15.112:0/1420414807 shutdown_connections 2023-08-27T11:17:11.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f9d063ee700 1 -- 172.21.15.112:0/1420414807 >> 172.21.15.112:0/1420414807 conn(0x7f9d0006f970 msgr2=0x7f9d0006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f1fc2d9d700 1 --2- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:6808/570347350,v1:172.21.15.138:6809/570347350] conn(0x7f1fa4072a70 0x7f1fa4074e90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7fd5b5ffb700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd5ac06a1f0 0x7fd5ac06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7fd5b5ffb700 1 -- 172.21.15.112:0/2842995552 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd5b8088840 con 0x7fd5c00b1500 2023-08-27T11:17:11.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f9d063ee700 1 -- 172.21.15.112:0/1420414807 shutdown_connections 2023-08-27T11:17:11.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f9d063ee700 1 -- 172.21.15.112:0/1420414807 wait complete. 2023-08-27T11:17:11.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7fd5b5ffb700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:6800/2981217086,v1:172.21.15.138:6801/2981217086] conn(0x7fd5ac072ab0 0x7fd5ac074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7fd5b5ffb700 1 -- 172.21.15.112:0/2842995552 --> [v2:172.21.15.138:6800/2981217086,v1:172.21.15.138:6801/2981217086] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5ac075580 con 0x7fd5ac072ab0 2023-08-27T11:17:11.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7fd5b5ffb700 1 -- 172.21.15.112:0/2842995552 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5b8088bf0 con 0x7fd5c00b1500 2023-08-27T11:17:11.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f72597fa700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f724406a290 0x7f724406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f72597fa700 1 -- 172.21.15.112:0/3752408818 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7250089be0 con 0x7f725c161c90 2023-08-27T11:17:11.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f72597fa700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:6816/2837763972,v1:172.21.15.138:6817/2837763972] conn(0x7f7244072b50 0x7f7244074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f72597fa700 1 -- 172.21.15.112:0/3752408818 --> [v2:172.21.15.138:6816/2837763972,v1:172.21.15.138:6817/2837763972] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7244075620 con 0x7f7244072b50 2023-08-27T11:17:11.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7f72597fa700 1 -- 172.21.15.112:0/3752408818 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f725008e2a0 con 0x7f725c161c90 2023-08-27T11:17:11.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7f7260dbc700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f724406a290 0x7f724406c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7fd5c5bce700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:6800/2981217086,v1:172.21.15.138:6801/2981217086] conn(0x7fd5ac072ab0 0x7fd5ac074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7f725bfff700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:6816/2837763972,v1:172.21.15.138:6817/2837763972] conn(0x7f7244072b50 0x7f7244074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7fd5c4bcc700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd5ac06a1f0 0x7fd5ac06c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.169+0000 7fb4fb7fe700 1 -- 172.21.15.112:0/3014470614 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4003c60 con 0x7fb4fc0b2910 2023-08-27T11:17:11.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7fb50312d700 1 -- 172.21.15.112:0/3014470614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4fc0b2910 msgr2=0x7fb4fc0ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7fb50312d700 1 --2- 172.21.15.112:0/3014470614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4fc0b2910 0x7fb4fc0ce3e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009d90 tx=0x7fb4f400b330 comp rx=0 tx=0).stop 2023-08-27T11:17:11.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7f7260dbc700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f724406a290 0x7f724406c750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f725400a7f0 tx=0x7f725400d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7f1fb2ffd700 1 -- 172.21.15.112:0/942817423 <== osd.6 v2:172.21.15.138:6808/570347350 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1fa4075540 con 0x7f1fa4072a70 2023-08-27T11:17:11.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7f1fc1d9b700 1 --2- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1fa406a1b0 0x7f1fa406c670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4000c00 tx=0x7f1fb4013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7f725bfff700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:6816/2837763972,v1:172.21.15.138:6817/2837763972] conn(0x7f7244072b50 0x7f7244074f70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7fb50312d700 1 -- 172.21.15.112:0/3014470614 shutdown_connections 2023-08-27T11:17:11.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7fb50312d700 1 --2- 172.21.15.112:0/3014470614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4fc0b2910 0x7fb4fc0ce3e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7fb50312d700 1 --2- 172.21.15.112:0/3014470614 >> v2:172.21.15.138:3300/0 conn(0x7fb4fc0ccc00 0x7fb4fc071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7fb50312d700 1 -- 172.21.15.112:0/3014470614 >> 172.21.15.112:0/3014470614 conn(0x7fb4fc0b0a10 msgr2=0x7fb4fc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7fb50312d700 1 -- 172.21.15.112:0/3014470614 shutdown_connections 2023-08-27T11:17:11.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7fb50312d700 1 -- 172.21.15.112:0/3014470614 wait complete. 2023-08-27T11:17:11.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7f8bd57fa700 1 -- 172.21.15.112:0/2336482638 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8bc80223e0 con 0x7f8bd8147890 2023-08-27T11:17:11.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.170+0000 7f8bd57fa700 1 --2- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8bc406a0e0 0x7f8bc406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7f8bd57fa700 1 -- 172.21.15.112:0/2336482638 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8bc8088b00 con 0x7f8bd8147890 2023-08-27T11:17:11.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7f8bd7fff700 1 --2- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8bc406a0e0 0x7f8bc406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7f8bd57fa700 1 --2- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:6824/547050660,v1:172.21.15.138:6825/547050660] conn(0x7f8bc40729a0 0x7f8bc4074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7f8bd57fa700 1 -- 172.21.15.112:0/2336482638 --> [v2:172.21.15.138:6824/547050660,v1:172.21.15.138:6825/547050660] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8bc4075470 con 0x7f8bc40729a0 2023-08-27T11:17:11.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7f8bd57fa700 1 -- 172.21.15.112:0/2336482638 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bc8088eb0 con 0x7f8bd8147890 2023-08-27T11:17:11.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7fd5c5bce700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:6800/2981217086,v1:172.21.15.138:6801/2981217086] conn(0x7fd5ac072ab0 0x7fd5ac074ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7fd5c4bcc700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd5ac06a1f0 0x7fd5ac06c6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0090b0 tx=0x7fd5bc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7f9d063ee700 1 Processor -- start 2023-08-27T11:17:11.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7f8bdd340700 1 --2- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:6824/547050660,v1:172.21.15.138:6825/547050660] conn(0x7f8bc40729a0 0x7f8bc4074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7f8bd7fff700 1 --2- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8bc406a0e0 0x7f8bc406c5a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc003e60 tx=0x7f8bcc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7f72597fa700 1 -- 172.21.15.112:0/3752408818 <== osd.7 v2:172.21.15.138:6816/2837763972 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f7244075620 con 0x7f7244072b50 2023-08-27T11:17:11.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.171+0000 7f8bdd340700 1 --2- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:6824/547050660,v1:172.21.15.138:6825/547050660] conn(0x7f8bc40729a0 0x7f8bc4074dc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.172+0000 7f9d063ee700 1 -- start start 2023-08-27T11:17:11.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.172+0000 7fd5b5ffb700 1 -- 172.21.15.112:0/2842995552 <== osd.5 v2:172.21.15.138:6800/2981217086 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd5ac075580 con 0x7fd5ac072ab0 2023-08-27T11:17:11.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.172+0000 7fb50312d700 1 Processor -- start 2023-08-27T11:17:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.172+0000 7f8bd57fa700 1 -- 172.21.15.112:0/2336482638 <== osd.4 v2:172.21.15.138:6824/547050660 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f8bc4075470 con 0x7f8bc40729a0 2023-08-27T11:17:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.172+0000 7fb50312d700 1 -- start start 2023-08-27T11:17:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.173+0000 7f9d063ee700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ce8004850 0x7f9d00144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.173+0000 7f9d063ee700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d00143b00 0x7f9d000cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb50312d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4fc0b2910 0x7fb4fc1f7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb50312d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4fc0cbf30 0x7fb4fc14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb50312d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4fc0ccc00 0x7fb4fc1492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb50312d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc1f80d0 con 0x7fb4fc0ccc00 2023-08-27T11:17:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb50312d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc14f390 con 0x7fb4fc0cbf30 2023-08-27T11:17:11.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb50312d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc1f7830 con 0x7fb4fc0b2910 2023-08-27T11:17:11.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb500ec9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4fc0b2910 0x7fb4fc1f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb500ec9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4fc0b2910 0x7fb4fc1f7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38326/0 (socket says 172.21.15.112:38326) 2023-08-27T11:17:11.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb500ec9700 1 -- 172.21.15.112:0/3772419506 learned_addr learned my addr 172.21.15.112:0/3772419506 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb500ec9700 1 -- 172.21.15.112:0/3772419506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4fc0cbf30 msgr2=0x7fb4fc14ee50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:17:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb500ec9700 1 --2- 172.21.15.112:0/3772419506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4fc0cbf30 0x7fb4fc14ee50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb500ec9700 1 -- 172.21.15.112:0/3772419506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4fc0ccc00 msgr2=0x7fb4fc1492d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:17:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb500ec9700 1 --2- 172.21.15.112:0/3772419506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4fc0ccc00 0x7fb4fc1492d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb500ec9700 1 -- 172.21.15.112:0/3772419506 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4009a70 con 0x7fb4fc0b2910 2023-08-27T11:17:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.174+0000 7fb500ec9700 1 --2- 172.21.15.112:0/3772419506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4fc0b2910 0x7fb4fc1f7130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000f9e0 tx=0x7fb4f000fda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.175+0000 7fb4f97fa700 1 -- 172.21.15.112:0/3772419506 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000c070 con 0x7fb4fc0b2910 2023-08-27T11:17:11.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.175+0000 7fb4f97fa700 1 -- 172.21.15.112:0/3772419506 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f000da60 con 0x7fb4fc0b2910 2023-08-27T11:17:11.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.175+0000 7fb50312d700 1 -- 172.21.15.112:0/3772419506 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4fc1f7b10 con 0x7fb4fc0b2910 2023-08-27T11:17:11.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.175+0000 7fb4f97fa700 1 -- 172.21.15.112:0/3772419506 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000ddf0 con 0x7fb4fc0b2910 2023-08-27T11:17:11.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.175+0000 7fb50312d700 1 -- 172.21.15.112:0/3772419506 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc1564d0 con 0x7fb4fc0b2910 2023-08-27T11:17:11.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.175+0000 7fb50312d700 1 -- 172.21.15.112:0/3772419506 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb4fc15e320 con 0x7fb4fc0b2910 2023-08-27T11:17:11.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.176+0000 7fb4f97fa700 1 -- 172.21.15.112:0/3772419506 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb4f001a480 con 0x7fb4fc0b2910 2023-08-27T11:17:11.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.177+0000 7fb4f97fa700 1 --2- 172.21.15.112:0/3772419506 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb4e806a0e0 0x7fb4e806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.177+0000 7fb4f97fa700 1 -- 172.21.15.112:0/3772419506 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4f0015070 con 0x7fb4fc0b2910 2023-08-27T11:17:11.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.177+0000 7fb4f97fa700 1 --2- 172.21.15.112:0/3772419506 >> [v2:172.21.15.112:6808/742922092,v1:172.21.15.112:6809/742922092] conn(0x7fb4e8072920 0x7fb4e8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.177+0000 7fb4f97fa700 1 -- 172.21.15.112:0/3772419506 --> [v2:172.21.15.112:6808/742922092,v1:172.21.15.112:6809/742922092] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4e80753f0 con 0x7fb4e8072920 2023-08-27T11:17:11.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.177+0000 7fb4f97fa700 1 -- 172.21.15.112:0/3772419506 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4f0089040 con 0x7fb4fc0b2910 2023-08-27T11:17:11.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9cff7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d00143b00 0x7f9d000cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9cff7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d00143b00 0x7f9d000cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55326/0 (socket says 172.21.15.112:55326) 2023-08-27T11:17:11.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9cfffff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ce8004850 0x7f9d00144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9d063ee700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d001391f0 0x7f9d001396a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9d063ee700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d00145980 con 0x7f9d001391f0 2023-08-27T11:17:11.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9d063ee700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d00139be0 con 0x7f9d00143b00 2023-08-27T11:17:11.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9d063ee700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d00144a60 con 0x7f9ce8004850 2023-08-27T11:17:11.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9cff7fe700 1 -- 172.21.15.112:0/809724070 learned_addr learned my addr 172.21.15.112:0/809724070 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9cff7fe700 1 -- 172.21.15.112:0/809724070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ce8004850 msgr2=0x7f9d00144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9cff7fe700 1 --2- 172.21.15.112:0/809724070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ce8004850 0x7f9d00144520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9cff7fe700 1 -- 172.21.15.112:0/809724070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d001391f0 msgr2=0x7f9d001396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9cff7fe700 1 --2- 172.21.15.112:0/809724070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d001391f0 0x7f9d001396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9cff7fe700 1 -- 172.21.15.112:0/809724070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d00144d00 con 0x7f9d00143b00 2023-08-27T11:17:11.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.180+0000 7f9cff7fe700 1 --2- 172.21.15.112:0/809724070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d00143b00 0x7f9d000cb940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9ce801d530 tx=0x7f9d0015ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.181+0000 7f9cfcff9700 1 -- 172.21.15.112:0/809724070 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf801a070 con 0x7f9d00143b00 2023-08-27T11:17:11.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.181+0000 7f9cfcff9700 1 -- 172.21.15.112:0/809724070 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf800e6e0 con 0x7f9d00143b00 2023-08-27T11:17:11.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.181+0000 7f9cfcff9700 1 -- 172.21.15.112:0/809724070 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf800ee20 con 0x7f9d00143b00 2023-08-27T11:17:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.182+0000 7f9d063ee700 1 -- 172.21.15.112:0/809724070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d00144fe0 con 0x7f9d00143b00 2023-08-27T11:17:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.182+0000 7f9d063ee700 1 -- 172.21.15.112:0/809724070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d001455a0 con 0x7f9d00143b00 2023-08-27T11:17:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.182+0000 7f9d063ee700 1 -- 172.21.15.112:0/809724070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d00066d70 con 0x7f9d00143b00 2023-08-27T11:17:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.183+0000 7fb4fbfff700 1 --2- 172.21.15.112:0/3772419506 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb4e806a0e0 0x7fb4e806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.183+0000 7fb5016ca700 1 --2- 172.21.15.112:0/3772419506 >> [v2:172.21.15.112:6808/742922092,v1:172.21.15.112:6809/742922092] conn(0x7fb4e8072920 0x7fb4e8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.183+0000 7fb5016ca700 1 --2- 172.21.15.112:0/3772419506 >> [v2:172.21.15.112:6808/742922092,v1:172.21.15.112:6809/742922092] conn(0x7fb4e8072920 0x7fb4e8074d40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.183+0000 7fb4fbfff700 1 --2- 172.21.15.112:0/3772419506 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb4e806a0e0 0x7fb4e806c5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0cc350 tx=0x7fb4ec007a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.184+0000 7fb4f97fa700 1 -- 172.21.15.112:0/3772419506 <== osd.2 v2:172.21.15.112:6808/742922092 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb4e80753f0 con 0x7fb4e8072920 2023-08-27T11:17:11.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.184+0000 7f9cfcff9700 1 -- 172.21.15.112:0/809724070 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9cf8016610 con 0x7f9d00143b00 2023-08-27T11:17:11.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.185+0000 7f9cfcff9700 1 --2- 172.21.15.112:0/809724070 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9cec06a0e0 0x7f9cec06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.185+0000 7f9cfcff9700 1 -- 172.21.15.112:0/809724070 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9cf8089f40 con 0x7f9d00143b00 2023-08-27T11:17:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.185+0000 7f9cfffff700 1 --2- 172.21.15.112:0/809724070 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9cec06a0e0 0x7f9cec06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.185+0000 7f9cfcff9700 1 --2- 172.21.15.112:0/809724070 >> [v2:172.21.15.112:6800/3136809237,v1:172.21.15.112:6801/3136809237] conn(0x7f9cec072920 0x7f9cec074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.185+0000 7f9cfcff9700 1 -- 172.21.15.112:0/809724070 --> [v2:172.21.15.112:6800/3136809237,v1:172.21.15.112:6801/3136809237] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9cec0753f0 con 0x7f9cec072920 2023-08-27T11:17:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.185+0000 7f9cfcff9700 1 -- 172.21.15.112:0/809724070 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cf808a2f0 con 0x7f9d00143b00 2023-08-27T11:17:11.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.185+0000 7f9d0498b700 1 --2- 172.21.15.112:0/809724070 >> [v2:172.21.15.112:6800/3136809237,v1:172.21.15.112:6801/3136809237] conn(0x7f9cec072920 0x7f9cec074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.186+0000 7f9cfffff700 1 --2- 172.21.15.112:0/809724070 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9cec06a0e0 0x7f9cec06c5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000ba70 tx=0x7f9cf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.186+0000 7f9d0498b700 1 --2- 172.21.15.112:0/809724070 >> [v2:172.21.15.112:6800/3136809237,v1:172.21.15.112:6801/3136809237] conn(0x7f9cec072920 0x7f9cec074d40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.191+0000 7f9cfcff9700 1 -- 172.21.15.112:0/809724070 <== osd.1 v2:172.21.15.112:6800/3136809237 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9cec0753f0 con 0x7f9cec072920 2023-08-27T11:17:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.195+0000 7f65eaa1b700 1 Processor -- start 2023-08-27T11:17:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.196+0000 7f65eaa1b700 1 -- start start 2023-08-27T11:17:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.197+0000 7f65eaa1b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65e413c160 0x7f65e41598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.197+0000 7f65eaa1b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f65e415d600 0x7f65e40cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.197+0000 7f65eaa1b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f65e415da80 0x7f65e40cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.197+0000 7f65eaa1b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e4153de0 con 0x7f65e415da80 2023-08-27T11:17:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.197+0000 7f65eaa1b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e415ce90 con 0x7f65e415d600 2023-08-27T11:17:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.197+0000 7f65eaa1b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e415a100 con 0x7f65e413c160 2023-08-27T11:17:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.197+0000 7f65e8fb8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f65e415da80 0x7f65e40cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.197+0000 7f65e8fb8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f65e415da80 0x7f65e40cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54916/0 (socket says 172.21.15.112:54916) 2023-08-27T11:17:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.197+0000 7f65e8fb8700 1 -- 172.21.15.112:0/380866874 learned_addr learned my addr 172.21.15.112:0/380866874 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.198+0000 7f65e8fb8700 1 -- 172.21.15.112:0/380866874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65e413c160 msgr2=0x7f65e41598e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:17:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.198+0000 7f65e8fb8700 1 --2- 172.21.15.112:0/380866874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65e413c160 0x7f65e41598e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.198+0000 7f65e8fb8700 1 -- 172.21.15.112:0/380866874 >> v2:172.21.15.138:3300/0 conn(0x7f65e415d600 msgr2=0x7f65e40cb7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:17:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.198+0000 7f65e8fb8700 1 --2- 172.21.15.112:0/380866874 >> v2:172.21.15.138:3300/0 conn(0x7f65e415d600 0x7f65e40cb7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.198+0000 7f65e8fb8700 1 -- 172.21.15.112:0/380866874 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e415a3e0 con 0x7f65e415da80 2023-08-27T11:17:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.198+0000 7f65e8fb8700 1 --2- 172.21.15.112:0/380866874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e415da80 0x7f65e40cbd30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f65e40cfeb0 tx=0x7f65dc00d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63520d1132692809 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.199+0000 7f65e2ffd700 1 -- 172.21.15.112:0/380866874 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc007d30 con 0x7f65e415da80 2023-08-27T11:17:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.199+0000 7f65e2ffd700 1 -- 172.21.15.112:0/380866874 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc007e90 con 0x7f65e415da80 2023-08-27T11:17:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.199+0000 7f65e2ffd700 1 -- 172.21.15.112:0/380866874 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc012710 con 0x7f65e415da80 2023-08-27T11:17:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.200+0000 7f65eaa1b700 1 -- 172.21.15.112:0/380866874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e415da80 msgr2=0x7f65e40cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.200+0000 7f65eaa1b700 1 --2- 172.21.15.112:0/380866874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e415da80 0x7f65e40cbd30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f65e40cfeb0 tx=0x7f65dc00d650 comp rx=0 tx=0).stop 2023-08-27T11:17:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.200+0000 7f39b1ca7700 1 -- 172.21.15.112:0/1363003614 --> [v2:172.21.15.112:6816/3936386115,v1:172.21.15.112:6817/3936386115] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f39ac0b3e50 con 0x7f3998072920 2023-08-27T11:17:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.200+0000 7f1fc88c8700 1 -- 172.21.15.112:0/942817423 --> [v2:172.21.15.138:6808/570347350,v1:172.21.15.138:6809/570347350] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1fa8002d40 con 0x7f1fa4072a70 2023-08-27T11:17:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.201+0000 7f3993fff700 1 -- 172.21.15.112:0/1363003614 <== osd.0 v2:172.21.15.112:6816/3936386115 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f39a0009890 con 0x7f3998072920 2023-08-27T11:17:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.201+0000 7f1fb2ffd700 1 -- 172.21.15.112:0/942817423 <== osd.6 v2:172.21.15.138:6808/570347350 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1fa8002d40 con 0x7f1fa4072a70 2023-08-27T11:17:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.201+0000 7f1fc88c8700 1 -- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:6808/570347350,v1:172.21.15.138:6809/570347350] conn(0x7f1fa4072a70 msgr2=0x7f1fa4074e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.201+0000 7f1fc88c8700 1 --2- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:6808/570347350,v1:172.21.15.138:6809/570347350] conn(0x7f1fa4072a70 0x7f1fa4074e90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.202+0000 7f39b1ca7700 1 -- 172.21.15.112:0/1363003614 >> [v2:172.21.15.112:6816/3936386115,v1:172.21.15.112:6817/3936386115] conn(0x7f3998072920 msgr2=0x7f3998074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.202+0000 7f39b1ca7700 1 --2- 172.21.15.112:0/1363003614 >> [v2:172.21.15.112:6816/3936386115,v1:172.21.15.112:6817/3936386115] conn(0x7f3998072920 0x7f3998074d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.203+0000 7f8bdeda3700 1 -- 172.21.15.112:0/2336482638 --> [v2:172.21.15.138:6824/547050660,v1:172.21.15.138:6825/547050660] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8bd81627d0 con 0x7f8bc40729a0 2023-08-27T11:17:11.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f1fc88c8700 1 -- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1fa406a1b0 msgr2=0x7f1fa406c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f1fc88c8700 1 --2- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1fa406a1b0 0x7f1fa406c670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4000c00 tx=0x7f1fb4013000 comp rx=0 tx=0).stop 2023-08-27T11:17:11.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f1fc88c8700 1 -- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc406adb0 msgr2=0x7f1fc4151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f1fc88c8700 1 --2- 172.21.15.112:0/942817423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc406adb0 0x7f1fc4151990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00cbc0 tx=0x7f1fbc00ced0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f65eaa1b700 1 -- 172.21.15.112:0/380866874 shutdown_connections 2023-08-27T11:17:11.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f65eaa1b700 1 --2- 172.21.15.112:0/380866874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e415da80 0x7f65e40cbd30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f65eaa1b700 1 --2- 172.21.15.112:0/380866874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65e413c160 0x7f65e41598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f65eaa1b700 1 --2- 172.21.15.112:0/380866874 >> v2:172.21.15.138:3300/0 conn(0x7f65e415d600 0x7f65e40cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f65eaa1b700 1 -- 172.21.15.112:0/380866874 >> 172.21.15.112:0/380866874 conn(0x7f65e40b0a10 msgr2=0x7f65e40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f1fc2d9d700 1 -- 172.21.15.112:0/942817423 reap_dead start 2023-08-27T11:17:11.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f1fc88c8700 1 -- 172.21.15.112:0/942817423 shutdown_connections 2023-08-27T11:17:11.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f1fc88c8700 1 -- 172.21.15.112:0/942817423 >> 172.21.15.112:0/942817423 conn(0x7f1fc40d0000 msgr2=0x7f1fc415a750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f65eaa1b700 1 -- 172.21.15.112:0/380866874 shutdown_connections 2023-08-27T11:17:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f65eaa1b700 1 -- 172.21.15.112:0/380866874 wait complete. 2023-08-27T11:17:11.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f39b1ca7700 1 -- 172.21.15.112:0/1363003614 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f399806a0e0 msgr2=0x7f399806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f39b1ca7700 1 --2- 172.21.15.112:0/1363003614 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f399806a0e0 0x7f399806c5a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f399c000c00 tx=0x7f399c008030 comp rx=0 tx=0).stop 2023-08-27T11:17:11.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f39b1ca7700 1 -- 172.21.15.112:0/1363003614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39ac1619b0 msgr2=0x7f39ac0b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f39b1ca7700 1 --2- 172.21.15.112:0/1363003614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39ac1619b0 0x7f39ac0b2200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c870 tx=0x7f39a400cc30 comp rx=0 tx=0).stop 2023-08-27T11:17:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f39abfff700 1 -- 172.21.15.112:0/1363003614 reap_dead start 2023-08-27T11:17:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f39b1ca7700 1 -- 172.21.15.112:0/1363003614 shutdown_connections 2023-08-27T11:17:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f39b1ca7700 1 -- 172.21.15.112:0/1363003614 >> 172.21.15.112:0/1363003614 conn(0x7f39ac06c880 msgr2=0x7f39ac0b7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f39b1ca7700 1 -- 172.21.15.112:0/1363003614 shutdown_connections 2023-08-27T11:17:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.204+0000 7f39b1ca7700 1 -- 172.21.15.112:0/1363003614 wait complete. 2023-08-27T11:17:11.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.205+0000 7f8bd57fa700 1 -- 172.21.15.112:0/2336482638 <== osd.4 v2:172.21.15.138:6824/547050660 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8bd0003e30 con 0x7f8bc40729a0 2023-08-27T11:17:11.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.205+0000 7f7263020700 1 -- 172.21.15.112:0/3752408818 --> [v2:172.21.15.138:6816/2837763972,v1:172.21.15.138:6817/2837763972] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f723c002d40 con 0x7f7244072b50 2023-08-27T11:17:11.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.205+0000 7f65eaa1b700 1 Processor -- start 2023-08-27T11:17:11.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.205+0000 7f1fc88c8700 1 -- 172.21.15.112:0/942817423 shutdown_connections 2023-08-27T11:17:11.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.205+0000 7fd5c7631700 1 -- 172.21.15.112:0/2842995552 --> [v2:172.21.15.138:6800/2981217086,v1:172.21.15.138:6801/2981217086] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd5a0002d40 con 0x7fd5ac072ab0 2023-08-27T11:17:11.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f65eaa1b700 1 -- start start 2023-08-27T11:17:11.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f1fc88c8700 1 -- 172.21.15.112:0/942817423 wait complete. 2023-08-27T11:17:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f8bdeda3700 1 -- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:6824/547050660,v1:172.21.15.138:6825/547050660] conn(0x7f8bc40729a0 msgr2=0x7f8bc4074dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f8bdeda3700 1 --2- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:6824/547050660,v1:172.21.15.138:6825/547050660] conn(0x7f8bc40729a0 0x7f8bc4074dc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f8bdeda3700 1 -- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8bc406a0e0 msgr2=0x7f8bc406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f8bdeda3700 1 --2- 172.21.15.112:0/2336482638 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8bc406a0e0 0x7f8bc406c5a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc003e60 tx=0x7f8bcc009040 comp rx=0 tx=0).stop 2023-08-27T11:17:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f8bdeda3700 1 -- 172.21.15.112:0/2336482638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd8147890 msgr2=0x7f8bd8158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f8bdeda3700 1 --2- 172.21.15.112:0/2336482638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd8147890 0x7f8bd8158b30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800c750 tx=0x7f8bc800cb10 comp rx=0 tx=0).stop 2023-08-27T11:17:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f8bdd340700 1 -- 172.21.15.112:0/2336482638 reap_dead start 2023-08-27T11:17:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f8bdeda3700 1 -- 172.21.15.112:0/2336482638 shutdown_connections 2023-08-27T11:17:11.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f8bdeda3700 1 -- 172.21.15.112:0/2336482638 >> 172.21.15.112:0/2336482638 conn(0x7f8bd80b0a10 msgr2=0x7f8bd813e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7fd5b5ffb700 1 -- 172.21.15.112:0/2842995552 <== osd.5 v2:172.21.15.138:6800/2981217086 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd5a0002d40 con 0x7fd5ac072ab0 2023-08-27T11:17:11.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f72597fa700 1 -- 172.21.15.112:0/3752408818 <== osd.7 v2:172.21.15.138:6816/2837763972 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f723c002d40 con 0x7f7244072b50 2023-08-27T11:17:11.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f65eaa1b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65e413c160 0x7f65e413b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f65eaa1b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65e415d600 0x7f65e406c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f65eaa1b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e415da80 0x7f65e406ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f65eaa1b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e41f30c0 con 0x7f65e415da80 2023-08-27T11:17:11.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f65eaa1b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e415ae90 con 0x7f65e413c160 2023-08-27T11:17:11.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.206+0000 7f65eaa1b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e415b170 con 0x7f65e415d600 2023-08-27T11:17:11.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.207+0000 7f65e3fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65e413c160 0x7f65e413b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.207+0000 7f65e3fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65e413c160 0x7f65e413b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55348/0 (socket says 172.21.15.112:55348) 2023-08-27T11:17:11.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.207+0000 7f65e3fff700 1 -- 172.21.15.112:0/470977398 learned_addr learned my addr 172.21.15.112:0/470977398 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.207+0000 7f65e8fb8700 1 --2- 172.21.15.112:0/470977398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e415da80 0x7f65e406ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.207+0000 7f7263020700 1 -- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:6816/2837763972,v1:172.21.15.138:6817/2837763972] conn(0x7f7244072b50 msgr2=0x7f7244074f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.207+0000 7f7263020700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:6816/2837763972,v1:172.21.15.138:6817/2837763972] conn(0x7f7244072b50 0x7f7244074f70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.207+0000 7f65e3fff700 1 -- 172.21.15.112:0/470977398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65e415d600 msgr2=0x7f65e406c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.207+0000 7f65e3fff700 1 --2- 172.21.15.112:0/470977398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65e415d600 0x7f65e406c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.207+0000 7f65e3fff700 1 -- 172.21.15.112:0/470977398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e415da80 msgr2=0x7f65e406ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.207+0000 7f65e3fff700 1 --2- 172.21.15.112:0/470977398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e415da80 0x7f65e406ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.207+0000 7f65e3fff700 1 -- 172.21.15.112:0/470977398 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc00d050 con 0x7f65e413c160 2023-08-27T11:17:11.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.207+0000 7f65e3fff700 1 --2- 172.21.15.112:0/470977398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65e413c160 0x7f65e413b660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f65d800b550 tx=0x7f65d800b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.208+0000 7f8bdeda3700 1 -- 172.21.15.112:0/2336482638 shutdown_connections 2023-08-27T11:17:11.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.208+0000 7f7263020700 1 -- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f724406a290 msgr2=0x7f724406c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.208+0000 7f7263020700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f724406a290 0x7f724406c750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f725400a7f0 tx=0x7f725400d040 comp rx=0 tx=0).stop 2023-08-27T11:17:11.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.208+0000 7f7263020700 1 -- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f725c161c90 msgr2=0x7f725c0a8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.208+0000 7f7263020700 1 --2- 172.21.15.112:0/3752408818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f725c161c90 0x7f725c0a8e90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7250004000 tx=0x7f7250004750 comp rx=0 tx=0).stop 2023-08-27T11:17:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.208+0000 7f72615bd700 1 -- 172.21.15.112:0/3752408818 reap_dead start 2023-08-27T11:17:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.208+0000 7f7263020700 1 -- 172.21.15.112:0/3752408818 shutdown_connections 2023-08-27T11:17:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.208+0000 7f7263020700 1 -- 172.21.15.112:0/3752408818 >> 172.21.15.112:0/3752408818 conn(0x7f725c0b0a10 msgr2=0x7f725c06ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.208+0000 7f8bdeda3700 1 -- 172.21.15.112:0/2336482638 wait complete. 2023-08-27T11:17:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.207+0000 7f65e0ff9700 1 -- 172.21.15.112:0/470977398 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8011830 con 0x7f65e413c160 2023-08-27T11:17:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.208+0000 7f65e0ff9700 1 -- 172.21.15.112:0/470977398 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d80119b0 con 0x7f65e413c160 2023-08-27T11:17:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.208+0000 7f65e0ff9700 1 -- 172.21.15.112:0/470977398 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8011d60 con 0x7f65e413c160 2023-08-27T11:17:11.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.209+0000 7f65eaa1b700 1 -- 172.21.15.112:0/470977398 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e415b450 con 0x7f65e413c160 2023-08-27T11:17:11.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.209+0000 7f65eaa1b700 1 -- 172.21.15.112:0/470977398 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e41f2400 con 0x7f65e413c160 2023-08-27T11:17:11.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.210+0000 7f7263020700 1 -- 172.21.15.112:0/3752408818 shutdown_connections 2023-08-27T11:17:11.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.210+0000 7f7263020700 1 -- 172.21.15.112:0/3752408818 wait complete. 2023-08-27T11:17:11.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.211+0000 7f65e0ff9700 1 -- 172.21.15.112:0/470977398 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f65d801a400 con 0x7f65e413c160 2023-08-27T11:17:11.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.211+0000 7f65e0ff9700 1 --2- 172.21.15.112:0/470977398 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65c806a0b0 0x7f65c806c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.211+0000 7f65e0ff9700 1 -- 172.21.15.112:0/470977398 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f65d8088450 con 0x7f65e413c160 2023-08-27T11:17:11.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.213+0000 7f65e37fe700 1 --2- 172.21.15.112:0/470977398 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65c806a0b0 0x7f65c806c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.213+0000 7fb50312d700 1 -- 172.21.15.112:0/3772419506 --> [v2:172.21.15.112:6808/742922092,v1:172.21.15.112:6809/742922092] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb4fc15e040 con 0x7fb4e8072920 2023-08-27T11:17:11.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.218+0000 7fb4f97fa700 1 -- 172.21.15.112:0/3772419506 <== osd.2 v2:172.21.15.112:6808/742922092 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb4f40098f0 con 0x7fb4e8072920 2023-08-27T11:17:11.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.218+0000 7f9d063ee700 1 -- 172.21.15.112:0/809724070 --> [v2:172.21.15.112:6800/3136809237,v1:172.21.15.112:6801/3136809237] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9d00139be0 con 0x7f9cec072920 2023-08-27T11:17:11.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.218+0000 7f65eaa1b700 1 --2- 172.21.15.112:0/470977398 >> [v2:172.21.15.112:6824/3108899647,v1:172.21.15.112:6825/3108899647] conn(0x7f65e41f2680 0x7f65e41f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.218+0000 7f65eaa1b700 1 -- 172.21.15.112:0/470977398 --> [v2:172.21.15.112:6824/3108899647,v1:172.21.15.112:6825/3108899647] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65e415d0e0 con 0x7f65e41f2680 2023-08-27T11:17:11.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.218+0000 7f65e8fb8700 1 --2- 172.21.15.112:0/470977398 >> [v2:172.21.15.112:6824/3108899647,v1:172.21.15.112:6825/3108899647] conn(0x7f65e41f2680 0x7f65e41f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.218+0000 7f65e37fe700 1 --2- 172.21.15.112:0/470977398 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65c806a0b0 0x7f65c806c570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65e41f2360 tx=0x7f65d4006cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.219+0000 7f65e8fb8700 1 --2- 172.21.15.112:0/470977398 >> [v2:172.21.15.112:6824/3108899647,v1:172.21.15.112:6825/3108899647] conn(0x7f65e41f2680 0x7f65e41f2a60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.219+0000 7fb50312d700 1 -- 172.21.15.112:0/3772419506 >> [v2:172.21.15.112:6808/742922092,v1:172.21.15.112:6809/742922092] conn(0x7fb4e8072920 msgr2=0x7fb4e8074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.219+0000 7fb50312d700 1 --2- 172.21.15.112:0/3772419506 >> [v2:172.21.15.112:6808/742922092,v1:172.21.15.112:6809/742922092] conn(0x7fb4e8072920 0x7fb4e8074d40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.220+0000 7fd5c7631700 1 -- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:6800/2981217086,v1:172.21.15.138:6801/2981217086] conn(0x7fd5ac072ab0 msgr2=0x7fd5ac074ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.220+0000 7fd5c7631700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:6800/2981217086,v1:172.21.15.138:6801/2981217086] conn(0x7fd5ac072ab0 0x7fd5ac074ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.220+0000 7f65e0ff9700 1 -- 172.21.15.112:0/470977398 <== osd.3 v2:172.21.15.112:6824/3108899647 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f65dc007210 con 0x7f65e41f2680 2023-08-27T11:17:11.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.220+0000 7fd5c7631700 1 -- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd5ac06a1f0 msgr2=0x7fd5ac06c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.220+0000 7fd5c7631700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd5ac06a1f0 0x7fd5ac06c6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0090b0 tx=0x7fd5bc008030 comp rx=0 tx=0).stop 2023-08-27T11:17:11.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.220+0000 7fd5c7631700 1 -- 172.21.15.112:0/2842995552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c00b1500 msgr2=0x7fd5c0154a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.220+0000 7fd5c7631700 1 --2- 172.21.15.112:0/2842995552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c00b1500 0x7fd5c0154a30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8002080 tx=0x7fd5b800a340 comp rx=0 tx=0).stop 2023-08-27T11:17:11.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.220+0000 7fd5c5bce700 1 -- 172.21.15.112:0/2842995552 reap_dead start 2023-08-27T11:17:11.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.221+0000 7fd5c7631700 1 -- 172.21.15.112:0/2842995552 shutdown_connections 2023-08-27T11:17:11.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.221+0000 7fd5c7631700 1 -- 172.21.15.112:0/2842995552 >> 172.21.15.112:0/2842995552 conn(0x7fd5c00b0a10 msgr2=0x7fd5c00af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.223+0000 7fd5c7631700 1 -- 172.21.15.112:0/2842995552 shutdown_connections 2023-08-27T11:17:11.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.223+0000 7f9cfcff9700 1 -- 172.21.15.112:0/809724070 <== osd.1 v2:172.21.15.112:6800/3136809237 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9d00139be0 con 0x7f9cec072920 2023-08-27T11:17:11.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.224+0000 7fd5c7631700 1 -- 172.21.15.112:0/2842995552 wait complete. 2023-08-27T11:17:11.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.225+0000 7fb50312d700 1 -- 172.21.15.112:0/3772419506 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb4e806a0e0 msgr2=0x7fb4e806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.225+0000 7fb50312d700 1 --2- 172.21.15.112:0/3772419506 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb4e806a0e0 0x7fb4e806c5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0cc350 tx=0x7fb4ec007a20 comp rx=0 tx=0).stop 2023-08-27T11:17:11.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.225+0000 7fb50312d700 1 -- 172.21.15.112:0/3772419506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4fc0b2910 msgr2=0x7fb4fc1f7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.225+0000 7fb50312d700 1 --2- 172.21.15.112:0/3772419506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4fc0b2910 0x7fb4fc1f7130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000f9e0 tx=0x7fb4f000fda0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.335 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2023-08-27T11:17:11.335 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-08-27T11:17:11.336 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2023-08-27T11:17:11.337 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-08-27T11:17:11.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.228+0000 7fb5016ca700 1 -- 172.21.15.112:0/3772419506 reap_dead start 2023-08-27T11:17:11.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.228+0000 7fb50312d700 1 -- 172.21.15.112:0/3772419506 shutdown_connections 2023-08-27T11:17:11.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.228+0000 7fb50312d700 1 -- 172.21.15.112:0/3772419506 >> 172.21.15.112:0/3772419506 conn(0x7fb4fc0b0a10 msgr2=0x7fb4fc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.228+0000 7fb50312d700 1 -- 172.21.15.112:0/3772419506 shutdown_connections 2023-08-27T11:17:11.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.228+0000 7fb50312d700 1 -- 172.21.15.112:0/3772419506 wait complete. 2023-08-27T11:17:11.339 INFO:teuthology.orchestra.run.smithi112.stdout:55834574854 2023-08-27T11:17:11.339 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-08-27T11:17:11.340 INFO:teuthology.orchestra.run.smithi112.stdout:55834574854 2023-08-27T11:17:11.341 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-08-27T11:17:11.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.230+0000 7f9d063ee700 1 -- 172.21.15.112:0/809724070 >> [v2:172.21.15.112:6800/3136809237,v1:172.21.15.112:6801/3136809237] conn(0x7f9cec072920 msgr2=0x7f9cec074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.230+0000 7f9d063ee700 1 --2- 172.21.15.112:0/809724070 >> [v2:172.21.15.112:6800/3136809237,v1:172.21.15.112:6801/3136809237] conn(0x7f9cec072920 0x7f9cec074d40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.230+0000 7f9d063ee700 1 -- 172.21.15.112:0/809724070 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9cec06a0e0 msgr2=0x7f9cec06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.230+0000 7f9d063ee700 1 --2- 172.21.15.112:0/809724070 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9cec06a0e0 0x7f9cec06c5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000ba70 tx=0x7f9cf0008040 comp rx=0 tx=0).stop 2023-08-27T11:17:11.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.230+0000 7f9d063ee700 1 -- 172.21.15.112:0/809724070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d00143b00 msgr2=0x7f9d000cb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.230+0000 7f9d063ee700 1 --2- 172.21.15.112:0/809724070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d00143b00 0x7f9d000cb940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9ce801d530 tx=0x7f9d0015ece0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.230+0000 7f9d0498b700 1 -- 172.21.15.112:0/809724070 reap_dead start 2023-08-27T11:17:11.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.230+0000 7f9d063ee700 1 -- 172.21.15.112:0/809724070 shutdown_connections 2023-08-27T11:17:11.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.230+0000 7f9d063ee700 1 -- 172.21.15.112:0/809724070 >> 172.21.15.112:0/809724070 conn(0x7f9d0006f970 msgr2=0x7f9d0015b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.230+0000 7f9d063ee700 1 -- 172.21.15.112:0/809724070 shutdown_connections 2023-08-27T11:17:11.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.230+0000 7f9d063ee700 1 -- 172.21.15.112:0/809724070 wait complete. 2023-08-27T11:17:11.344 INFO:teuthology.orchestra.run.smithi112.stdout:55834574854 2023-08-27T11:17:11.345 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-08-27T11:17:11.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.240+0000 7f65eaa1b700 1 -- 172.21.15.112:0/470977398 --> [v2:172.21.15.112:6824/3108899647,v1:172.21.15.112:6825/3108899647] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f65e415a6c0 con 0x7f65e41f2680 2023-08-27T11:17:11.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.241+0000 7f65e0ff9700 1 -- 172.21.15.112:0/470977398 <== osd.3 v2:172.21.15.112:6824/3108899647 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f65dc003a20 con 0x7f65e41f2680 2023-08-27T11:17:11.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.241+0000 7f65eaa1b700 1 -- 172.21.15.112:0/470977398 >> [v2:172.21.15.112:6824/3108899647,v1:172.21.15.112:6825/3108899647] conn(0x7f65e41f2680 msgr2=0x7f65e41f2a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.241+0000 7f65eaa1b700 1 --2- 172.21.15.112:0/470977398 >> [v2:172.21.15.112:6824/3108899647,v1:172.21.15.112:6825/3108899647] conn(0x7f65e41f2680 0x7f65e41f2a60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.241+0000 7f65eaa1b700 1 -- 172.21.15.112:0/470977398 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65c806a0b0 msgr2=0x7f65c806c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.241+0000 7f65eaa1b700 1 --2- 172.21.15.112:0/470977398 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65c806a0b0 0x7f65c806c570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65e41f2360 tx=0x7f65d4006cd0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.241+0000 7f65eaa1b700 1 -- 172.21.15.112:0/470977398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65e413c160 msgr2=0x7f65e413b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.241+0000 7f65eaa1b700 1 --2- 172.21.15.112:0/470977398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65e413c160 0x7f65e413b660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f65d800b550 tx=0x7f65d800b860 comp rx=0 tx=0).stop 2023-08-27T11:17:11.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.241+0000 7f65e8fb8700 1 -- 172.21.15.112:0/470977398 reap_dead start 2023-08-27T11:17:11.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.241+0000 7f65eaa1b700 1 -- 172.21.15.112:0/470977398 shutdown_connections 2023-08-27T11:17:11.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.241+0000 7f65eaa1b700 1 -- 172.21.15.112:0/470977398 >> 172.21.15.112:0/470977398 conn(0x7f65e40b0a10 msgr2=0x7f65e40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.241+0000 7f65eaa1b700 1 -- 172.21.15.112:0/470977398 shutdown_connections 2023-08-27T11:17:11.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.241+0000 7f65eaa1b700 1 -- 172.21.15.112:0/470977398 wait complete. 2023-08-27T11:17:11.350 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2023-08-27T11:17:11.350 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-08-27T11:17:11.351 INFO:teuthology.orchestra.run.smithi112.stdout:55834574854 2023-08-27T11:17:11.351 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-08-27T11:17:11.352 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2023-08-27T11:17:11.353 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-08-27T11:17:11.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.583+0000 7f9b32e9d700 1 Processor -- start 2023-08-27T11:17:11.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.584+0000 7fc7e818f700 1 Processor -- start 2023-08-27T11:17:11.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.584+0000 7fc7e818f700 1 -- start start 2023-08-27T11:17:11.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.583+0000 7f9b32e9d700 1 -- start start 2023-08-27T11:17:11.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.584+0000 7f9b32e9d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9b2c13c160 0x7f9b2c1598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.584+0000 7f9b32e9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9b2c15d600 0x7f9b2c0cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.584+0000 7f9b32e9d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b2c15ce90 con 0x7f9b2c15da90 2023-08-27T11:17:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.584+0000 7f9b32e9d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2c153de0 con 0x7f9b2c15d600 2023-08-27T11:17:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.584+0000 7f9b32e9d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2c13f920 con 0x7f9b2c13c160 2023-08-27T11:17:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7f9b3143a700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9b2c15da90 0x7f9b2c0cd660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49318/0 (socket says 172.21.15.112:49318) 2023-08-27T11:17:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7f9b2bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9b2c15d600 0x7f9b2c0cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7f9b3143a700 1 -- 172.21.15.112:0/1852592095 learned_addr learned my addr 172.21.15.112:0/1852592095 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:11.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7f9b30c39700 1 --2- 172.21.15.112:0/1852592095 >> v2:172.21.15.138:3300/0 conn(0x7f9b2c13c160 0x7f9b2c1598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7f9b30c39700 1 -- 172.21.15.112:0/1852592095 >> v2:172.21.15.112:3300/0 conn(0x7f9b2c15d600 msgr2=0x7f9b2c0cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7f9b30c39700 1 --2- 172.21.15.112:0/1852592095 >> v2:172.21.15.112:3300/0 conn(0x7f9b2c15d600 0x7f9b2c0cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7f9b30c39700 1 -- 172.21.15.112:0/1852592095 >> v1:172.21.15.112:6789/0 conn(0x7f9b2c15da90 legacy=0x7f9b2c0cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7f9b30c39700 1 -- 172.21.15.112:0/1852592095 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c15a100 con 0x7f9b2c13c160 2023-08-27T11:17:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7f9b30c39700 1 --2- 172.21.15.112:0/1852592095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b2c13c160 0x7f9b2c1598e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c070ac0 tx=0x7f9b2000df40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbbfe53f77cb9739 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7fc7e818f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc7e0143220 0x7fc7e00ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7fc7e818f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc7e006e860 0x7fc7e00a8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7fc7e818f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e015e2e0 con 0x7fc7e0159dd0 2023-08-27T11:17:11.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7fc7e818f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e00ce900 con 0x7fc7e006e860 2023-08-27T11:17:11.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.585+0000 7fc7e818f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e00cdc80 con 0x7fc7e0143220 2023-08-27T11:17:11.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e5f2b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc7e0143220 0x7fc7e00ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e5f2b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc7e0143220 0x7fc7e00ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55364/0 (socket says 172.21.15.112:55364) 2023-08-27T11:17:11.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e5f2b700 1 -- 172.21.15.112:0/1996542752 learned_addr learned my addr 172.21.15.112:0/1996542752 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309433566 0 0) 0x7fc7e015e2e0 con 0x7fc7e0159dd0 2023-08-27T11:17:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7cc0035b0 con 0x7fc7e0159dd0 2023-08-27T11:17:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 512265230 0 0) 0x7fc7cc0035b0 con 0x7fc7e0159dd0 2023-08-27T11:17:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 >> v2:172.21.15.138:3300/0 conn(0x7fc7e0143220 msgr2=0x7fc7e00ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 --2- 172.21.15.112:0/1996542752 >> v2:172.21.15.138:3300/0 conn(0x7fc7e0143220 0x7fc7e00ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 >> v2:172.21.15.112:3300/0 conn(0x7fc7e006e860 msgr2=0x7fc7e00a8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 --2- 172.21.15.112:0/1996542752 >> v2:172.21.15.112:3300/0 conn(0x7fc7e006e860 0x7fc7e00a8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e00cdf60 con 0x7fc7e0159dd0 2023-08-27T11:17:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fc7d0001670 con 0x7fc7e0159dd0 2023-08-27T11:17:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 >> v1:172.21.15.138:6789/0 conn(0x7fc7e0159dd0 legacy=0x7fc7e006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 --2- 172.21.15.112:0/1996542752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7cc003a00 0x7fc7cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 --2- 172.21.15.112:0/1996542752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7cc004850 0x7fc7cc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 --2- 172.21.15.112:0/1996542752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc005210 0x7fc7cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e00ce900 con 0x7fc7cc003a00 2023-08-27T11:17:11.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e00cdc80 con 0x7fc7cc005210 2023-08-27T11:17:11.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc00c130 con 0x7fc7cc004850 2023-08-27T11:17:11.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7fc7e572a700 1 --2- 172.21.15.112:0/1996542752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7cc004850 0x7fc7cc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7f9b2b7fe700 1 -- 172.21.15.112:0/1852592095 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20012070 con 0x7f9b2c13c160 2023-08-27T11:17:11.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7f9b2b7fe700 1 -- 172.21.15.112:0/1852592095 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2000d2e0 con 0x7f9b2c13c160 2023-08-27T11:17:11.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.586+0000 7f9b2b7fe700 1 -- 172.21.15.112:0/1852592095 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20004230 con 0x7f9b2c13c160 2023-08-27T11:17:11.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7f9b32e9d700 1 -- 172.21.15.112:0/1852592095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b2c13c160 msgr2=0x7f9b2c1598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7f9b32e9d700 1 --2- 172.21.15.112:0/1852592095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b2c13c160 0x7f9b2c1598e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c070ac0 tx=0x7f9b2000df40 comp rx=0 tx=0).stop 2023-08-27T11:17:11.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7fc7e572a700 1 -- 172.21.15.112:0/1996542752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc005210 msgr2=0x7fc7cc00ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:17:11.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7fc7e572a700 1 --2- 172.21.15.112:0/1996542752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc005210 0x7fc7cc00ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7fc7e572a700 1 -- 172.21.15.112:0/1996542752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7cc003a00 msgr2=0x7fc7cc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7fc7e572a700 1 --2- 172.21.15.112:0/1996542752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7cc003a00 0x7fc7cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7fc7e572a700 1 -- 172.21.15.112:0/1996542752 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e00cdf60 con 0x7fc7cc004850 2023-08-27T11:17:11.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7fc7e672c700 1 -- 172.21.15.112:0/1996542752 reap_dead start 2023-08-27T11:17:11.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7fa9e055f700 1 Processor -- start 2023-08-27T11:17:11.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7fc7e572a700 1 --2- 172.21.15.112:0/1996542752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7cc004850 0x7fc7cc004cd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000d370 tx=0x7fc7cc005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57045d17e2e790cb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.588+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d000de90 con 0x7fc7cc004850 2023-08-27T11:17:11.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7f9b32e9d700 1 -- 172.21.15.112:0/1852592095 shutdown_connections 2023-08-27T11:17:11.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7f9b32e9d700 1 --2- 172.21.15.112:0/1852592095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b2c13c160 0x7f9b2c1598e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.588+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d00113f0 con 0x7fc7cc004850 2023-08-27T11:17:11.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.588+0000 7fc7e4f29700 1 -- 172.21.15.112:0/1996542752 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0011780 con 0x7fc7cc004850 2023-08-27T11:17:11.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7f9b32e9d700 1 --2- 172.21.15.112:0/1852592095 >> v2:172.21.15.112:3300/0 conn(0x7f9b2c15d600 0x7f9b2c0cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7f9b32e9d700 1 -- 172.21.15.112:0/1852592095 >> 172.21.15.112:0/1852592095 conn(0x7f9b2c0b0a10 msgr2=0x7f9b2c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.587+0000 7fa9e055f700 1 -- start start 2023-08-27T11:17:11.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fc7e818f700 1 -- 172.21.15.112:0/1996542752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7cc004850 msgr2=0x7fc7cc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fc7e818f700 1 --2- 172.21.15.112:0/1996542752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7cc004850 0x7fc7cc004cd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000d370 tx=0x7fc7cc005880 comp rx=0 tx=0).stop 2023-08-27T11:17:11.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fc7e818f700 1 -- 172.21.15.112:0/1996542752 shutdown_connections 2023-08-27T11:17:11.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fc7e818f700 1 --2- 172.21.15.112:0/1996542752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7cc004850 0x7fc7cc004cd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fc7e818f700 1 -- 172.21.15.112:0/1996542752 >> 172.21.15.112:0/1996542752 conn(0x7fc7e006cd30 msgr2=0x7fc7e00717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7f9b32e9d700 1 -- 172.21.15.112:0/1852592095 shutdown_connections 2023-08-27T11:17:11.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fc7e818f700 1 -- 172.21.15.112:0/1996542752 shutdown_connections 2023-08-27T11:17:11.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fc7e818f700 1 -- 172.21.15.112:0/1996542752 wait complete. 2023-08-27T11:17:11.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fa9e055f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9d80cb7f0 0x7fa9d80cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fa9e055f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d80cc180 con 0x7fa9d80b1500 2023-08-27T11:17:11.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fa9e055f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d813f920 con 0x7fa9d813c160 2023-08-27T11:17:11.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fa9e055f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d8153de0 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fa9deafc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9d80cb7f0 0x7fa9d80cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fa9deafc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9d80cb7f0 0x7fa9d80cbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54954/0 (socket says 172.21.15.112:54954) 2023-08-27T11:17:11.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7fa9deafc700 1 -- 172.21.15.112:0/1948891345 learned_addr learned my addr 172.21.15.112:0/1948891345 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.590+0000 7fa9deafc700 1 -- 172.21.15.112:0/1948891345 >> v1:172.21.15.138:6789/0 conn(0x7fa9d813c160 legacy=0x7fa9d815d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.590+0000 7fa9deafc700 1 -- 172.21.15.112:0/1948891345 >> v1:172.21.15.112:6789/0 conn(0x7fa9d80b1500 legacy=0x7fa9d81598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.590+0000 7fa9deafc700 1 -- 172.21.15.112:0/1948891345 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d815ce90 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.590+0000 7fa9deafc700 1 --2- 172.21.15.112:0/1948891345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d80cb7f0 0x7fa9d80cbc40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009870 tx=0x7fa9d4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2457bfff68d5096 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.590+0000 7fa9dd2f9700 1 -- 172.21.15.112:0/1948891345 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4004230 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.590+0000 7fc7e818f700 1 Processor -- start 2023-08-27T11:17:11.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.590+0000 7fa9dd2f9700 1 -- 172.21.15.112:0/1948891345 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4004390 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.590+0000 7fa9dd2f9700 1 -- 172.21.15.112:0/1948891345 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40036a0 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.589+0000 7f9b32e9d700 1 -- 172.21.15.112:0/1852592095 wait complete. 2023-08-27T11:17:11.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.591+0000 7f9b32e9d700 1 Processor -- start 2023-08-27T11:17:11.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.591+0000 7fa9e055f700 1 -- 172.21.15.112:0/1948891345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d80cb7f0 msgr2=0x7fa9d80cbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.591+0000 7fa9e055f700 1 --2- 172.21.15.112:0/1948891345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d80cb7f0 0x7fa9d80cbc40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009870 tx=0x7fa9d4009b80 comp rx=0 tx=0).stop 2023-08-27T11:17:11.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.591+0000 7f9b32e9d700 1 -- start start 2023-08-27T11:17:11.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fa9e055f700 1 -- 172.21.15.112:0/1948891345 shutdown_connections 2023-08-27T11:17:11.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fa9e055f700 1 --2- 172.21.15.112:0/1948891345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d80cb7f0 0x7fa9d80cbc40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fa9e055f700 1 -- 172.21.15.112:0/1948891345 >> 172.21.15.112:0/1948891345 conn(0x7fa9d80b0a10 msgr2=0x7fa9d80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fa9e055f700 1 -- 172.21.15.112:0/1948891345 shutdown_connections 2023-08-27T11:17:11.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.591+0000 7fc7e818f700 1 -- start start 2023-08-27T11:17:11.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e818f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7cc004850 0x7fc7e00ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e818f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e01f6880 0x7fc7e01f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fa9e055f700 1 -- 172.21.15.112:0/1948891345 wait complete. 2023-08-27T11:17:11.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e818f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e013bbc0 0x7fc7e013c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e818f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e013df90 con 0x7fc7e01f6880 2023-08-27T11:17:11.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e818f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e013e270 con 0x7fc7e013bbc0 2023-08-27T11:17:11.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e818f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e013e550 con 0x7fc7cc004850 2023-08-27T11:17:11.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7f9b32e9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b2c13c160 0x7f9b2c2001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7f9b32e9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b2c15d600 0x7f9b2c1f3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7f9b32e9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b2c15da90 0x7f9b2c1f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7f9b32e9d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c157ad0 con 0x7f9b2c15da90 2023-08-27T11:17:11.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7f9b32e9d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c1574a0 con 0x7f9b2c15d600 2023-08-27T11:17:11.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7f9b32e9d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c1f4f50 con 0x7f9b2c13c160 2023-08-27T11:17:11.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e5f2b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7cc004850 0x7fc7e00ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7f9b2bfff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b2c15d600 0x7f9b2c1f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7f9b2bfff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b2c15d600 0x7f9b2c1f3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55384/0 (socket says 172.21.15.112:55384) 2023-08-27T11:17:11.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7f9b2bfff700 1 -- 172.21.15.112:0/606435181 learned_addr learned my addr 172.21.15.112:0/606435181 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e5f2b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7cc004850 0x7fc7e00ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38378/0 (socket says 172.21.15.112:38378) 2023-08-27T11:17:11.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e5f2b700 1 -- 172.21.15.112:0/2339597067 learned_addr learned my addr 172.21.15.112:0/2339597067 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e672c700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e013bbc0 0x7fc7e013c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7f9b3143a700 1 --2- 172.21.15.112:0/606435181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b2c15da90 0x7f9b2c1f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e572a700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e01f6880 0x7fc7e01f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e5f2b700 1 -- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e013bbc0 msgr2=0x7fc7e013c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e5f2b700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e013bbc0 0x7fc7e013c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e5f2b700 1 -- 172.21.15.112:0/2339597067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e01f6880 msgr2=0x7fc7e01f6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e5f2b700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e01f6880 0x7fc7e01f6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.592+0000 7fc7e5f2b700 1 -- 172.21.15.112:0/2339597067 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e013e830 con 0x7fc7cc004850 2023-08-27T11:17:11.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7f9b3143a700 1 -- 172.21.15.112:0/606435181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b2c13c160 msgr2=0x7f9b2c2001d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7f9b3143a700 1 --2- 172.21.15.112:0/606435181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b2c13c160 0x7f9b2c2001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7f9b3143a700 1 -- 172.21.15.112:0/606435181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b2c15d600 msgr2=0x7f9b2c1f3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7f9b3143a700 1 --2- 172.21.15.112:0/606435181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b2c15d600 0x7f9b2c1f3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7f9b3143a700 1 -- 172.21.15.112:0/606435181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2000d940 con 0x7f9b2c15da90 2023-08-27T11:17:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7fc7e5f2b700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7cc004850 0x7fc7e00ca930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc00e830 tx=0x7fc7dc00ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7f9b3143a700 1 --2- 172.21.15.112:0/606435181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b2c15da90 0x7f9b2c1f3940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00c550 tx=0x7f9b1c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7fc7d67fc700 1 -- 172.21.15.112:0/2339597067 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc00c690 con 0x7fc7cc004850 2023-08-27T11:17:11.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7fc7d67fc700 1 -- 172.21.15.112:0/2339597067 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7dc00c7f0 con 0x7fc7cc004850 2023-08-27T11:17:11.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7fc7d67fc700 1 -- 172.21.15.112:0/2339597067 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc00cb60 con 0x7fc7cc004850 2023-08-27T11:17:11.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7f7c63f29700 1 Processor -- start 2023-08-27T11:17:11.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7fc7e818f700 1 -- 172.21.15.112:0/2339597067 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e01f7240 con 0x7fc7cc004850 2023-08-27T11:17:11.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7fc7e818f700 1 -- 172.21.15.112:0/2339597067 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e013c580 con 0x7fc7cc004850 2023-08-27T11:17:11.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.594+0000 7f7215b55700 1 Processor -- start 2023-08-27T11:17:11.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.594+0000 7f7215b55700 1 -- start start 2023-08-27T11:17:11.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7f7c63f29700 1 -- start start 2023-08-27T11:17:11.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.593+0000 7f9b297fa700 1 -- 172.21.15.112:0/606435181 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c005dd0 con 0x7f9b2c15da90 2023-08-27T11:17:11.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.594+0000 7f9b297fa700 1 -- 172.21.15.112:0/606435181 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c0070d0 con 0x7f9b2c15da90 2023-08-27T11:17:11.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.594+0000 7f9b297fa700 1 -- 172.21.15.112:0/606435181 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c007460 con 0x7f9b2c15da90 2023-08-27T11:17:11.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.594+0000 7f9b32e9d700 1 -- 172.21.15.112:0/606435181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2c1f5230 con 0x7f9b2c15da90 2023-08-27T11:17:11.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.594+0000 7f9b32e9d700 1 -- 172.21.15.112:0/606435181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c200710 con 0x7f9b2c15da90 2023-08-27T11:17:11.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.594+0000 7fa9e055f700 1 Processor -- start 2023-08-27T11:17:11.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f7c63f29700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7c5c0b4bf0 0x7f7c5c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f7c63f29700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c5c0b3440 con 0x7f7c5c071ac0 2023-08-27T11:17:11.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f7c63f29700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c5c0b3720 con 0x7f7c5c0716e0 2023-08-27T11:17:11.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f7c63f29700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c5c0b3a00 con 0x7f7c5c0b4bf0 2023-08-27T11:17:11.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f7215b55700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f721013c380 0x7f72100cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f7215b55700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721013f920 con 0x7f72100b1500 2023-08-27T11:17:11.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f7215b55700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721015ce90 con 0x7f721013bef0 2023-08-27T11:17:11.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f7215b55700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7210153de0 con 0x7f721013c380 2023-08-27T11:17:11.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f96f10f2700 1 Processor -- start 2023-08-27T11:17:11.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f720ffff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f721013c380 0x7f72100cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f720ffff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f721013c380 0x7f72100cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55398/0 (socket says 172.21.15.112:55398) 2023-08-27T11:17:11.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f720ffff700 1 -- 172.21.15.112:0/2143844850 learned_addr learned my addr 172.21.15.112:0/2143844850 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f1774d68700 1 Processor -- start 2023-08-27T11:17:11.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f1774d68700 1 -- start start 2023-08-27T11:17:11.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7f7c614c4700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7c5c071ac0 0x7f7c5c0b44e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49350/0 (socket says 172.21.15.112:49350) 2023-08-27T11:17:11.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7f7c614c4700 1 -- 172.21.15.112:0/3756924566 learned_addr learned my addr 172.21.15.112:0/3756924566 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:11.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7f7c624c6700 1 --2- 172.21.15.112:0/3756924566 >> v2:172.21.15.112:3300/0 conn(0x7f7c5c0b4bf0 0x7f7c5c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7f96f10f2700 1 -- start start 2023-08-27T11:17:11.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.595+0000 7fa9e055f700 1 -- start start 2023-08-27T11:17:11.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7fa9e055f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d80b1500 0x7fa9d806d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7fa9e055f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9d80cb7f0 0x7fa9d815c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7fa9e055f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d813c160 0x7fa9d8167750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7fa9e055f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d815be00 con 0x7fa9d813c160 2023-08-27T11:17:11.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7fa9e055f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d815b4d0 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7fa9e055f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d815b7b0 con 0x7fa9d80b1500 2023-08-27T11:17:11.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7f7c624c6700 1 -- 172.21.15.112:0/3756924566 >> v1:172.21.15.138:6789/0 conn(0x7f7c5c0716e0 legacy=0x7f7c5c0d0cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:17:11.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7f7c624c6700 1 -- 172.21.15.112:0/3756924566 >> v1:172.21.15.112:6789/0 conn(0x7f7c5c071ac0 legacy=0x7f7c5c0b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7f7c624c6700 1 -- 172.21.15.112:0/3756924566 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c0b3ce0 con 0x7f7c5c0b4bf0 2023-08-27T11:17:11.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7fa9ddafa700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9d80cb7f0 0x7fa9d815c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7fa9ddafa700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9d80cb7f0 0x7fa9d815c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55406/0 (socket says 172.21.15.112:55406) 2023-08-27T11:17:11.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7fa9ddafa700 1 -- 172.21.15.112:0/179018862 learned_addr learned my addr 172.21.15.112:0/179018862 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f96f10f2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f96ec1598e0 0x7f96ec15d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f96f10f2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ec15db40 0x7f96ec0cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f96f10f2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ec15ce90 con 0x7f96ec0b1500 2023-08-27T11:17:11.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f96f10f2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec153de0 con 0x7f96ec1598e0 2023-08-27T11:17:11.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f96f10f2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec13f920 con 0x7f96ec15db40 2023-08-27T11:17:11.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7fa9de2fb700 1 --2- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d80b1500 0x7fa9d806d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.596+0000 7f7c624c6700 1 --2- 172.21.15.112:0/3756924566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c5c0b4bf0 0x7f7c5c0701a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009840 tx=0x7f7c4c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3382564615d37a5e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f96ead9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f96ec0b1500 0x7f96ec13c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49370/0 (socket says 172.21.15.112:49370) 2023-08-27T11:17:11.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f96ead9d700 1 -- 172.21.15.112:0/2004167327 learned_addr learned my addr 172.21.15.112:0/2004167327 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:11.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f96eb59e700 1 --2- 172.21.15.112:0/2004167327 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ec15db40 0x7f96ec0cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f9b32e9d700 1 -- 172.21.15.112:0/606435181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b0c0052f0 con 0x7f9b2c15da90 2023-08-27T11:17:11.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f1774d68700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17700b2860 0x7f1770160f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f1774d68700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1770161cb0 con 0x7f1770161480 2023-08-27T11:17:11.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f1774d68700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f177015dbc0 con 0x7f17701403d0 2023-08-27T11:17:11.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f1774d68700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f177006cee0 con 0x7f17700b2860 2023-08-27T11:17:11.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7fa9ddafa700 1 -- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d80b1500 msgr2=0x7fa9d806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7fa9ddafa700 1 --2- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d80b1500 0x7fa9d806d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7fa9ddafa700 1 -- 172.21.15.112:0/179018862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d813c160 msgr2=0x7fa9d8167750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7fa9ddafa700 1 --2- 172.21.15.112:0/179018862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d813c160 0x7fa9d8167750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7fa9ddafa700 1 -- 172.21.15.112:0/179018862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d4009550 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f176e59c700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f17701403d0 0x7f17700b2150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:56898/0 (socket says 172.21.15.112:56898) 2023-08-27T11:17:11.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f176e59c700 1 -- 172.21.15.112:0/3154813994 learned_addr learned my addr 172.21.15.112:0/3154813994 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:11.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f96ea59c700 1 --2- 172.21.15.112:0/2004167327 >> v2:172.21.15.138:3300/0 conn(0x7f96ec1598e0 0x7f96ec15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7fc7d67fc700 1 -- 172.21.15.112:0/2339597067 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc7dc00ccc0 con 0x7fc7cc004850 2023-08-27T11:17:11.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7fa9ddafa700 1 --2- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9d80cb7f0 0x7fa9d815c1e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000caa0 tx=0x7fa9d000cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f96eb59e700 1 -- 172.21.15.112:0/2004167327 >> v2:172.21.15.138:3300/0 conn(0x7f96ec1598e0 msgr2=0x7f96ec15d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f96eb59e700 1 --2- 172.21.15.112:0/2004167327 >> v2:172.21.15.138:3300/0 conn(0x7f96ec1598e0 0x7f96ec15d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f96eb59e700 1 -- 172.21.15.112:0/2004167327 >> v1:172.21.15.112:6789/0 conn(0x7f96ec0b1500 legacy=0x7f96ec13c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f96eb59e700 1 -- 172.21.15.112:0/2004167327 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec15a100 con 0x7f96ec15db40 2023-08-27T11:17:11.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f176dd9b700 1 --2- 172.21.15.112:0/3154813994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17700b2860 0x7f1770160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f176d59a700 1 -- 172.21.15.112:0/3154813994 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573187807 0 0) 0x7f177015dbc0 con 0x7f17701403d0 2023-08-27T11:17:11.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f176d59a700 1 -- 172.21.15.112:0/3154813994 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f175c0035b0 con 0x7f17701403d0 2023-08-27T11:17:11.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7f176dd9b700 1 -- 172.21.15.112:0/3154813994 >> v1:172.21.15.138:6789/0 conn(0x7f17701403d0 legacy=0x7f17700b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7f176dd9b700 1 -- 172.21.15.112:0/3154813994 >> v1:172.21.15.112:6789/0 conn(0x7f1770161480 legacy=0x7f17700ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7f176dd9b700 1 -- 172.21.15.112:0/3154813994 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177015dea0 con 0x7f17700b2860 2023-08-27T11:17:11.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7f96eb59e700 1 --2- 172.21.15.112:0/2004167327 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ec15db40 0x7f96ec0cb7f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009840 tx=0x7f96e0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad25ebfa74f76eb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7fc7d67fc700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7cc06bae0 0x7fc7cc06dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7f96e9d9b700 1 -- 172.21.15.112:0/2004167327 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0011830 con 0x7f96ec15db40 2023-08-27T11:17:11.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7f96e9d9b700 1 -- 172.21.15.112:0/2004167327 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e0011990 con 0x7f96ec15db40 2023-08-27T11:17:11.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7f96e9d9b700 1 -- 172.21.15.112:0/2004167327 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0011d20 con 0x7f96ec15db40 2023-08-27T11:17:11.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f720ffff700 1 -- 172.21.15.112:0/2143844850 >> v1:172.21.15.138:6789/0 conn(0x7f721013bef0 legacy=0x7f721015d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f720ffff700 1 -- 172.21.15.112:0/2143844850 >> v1:172.21.15.112:6789/0 conn(0x7f72100b1500 legacy=0x7f72101598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.597+0000 7f720ffff700 1 -- 172.21.15.112:0/2143844850 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7210161220 con 0x7f721013c380 2023-08-27T11:17:11.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7f720ffff700 1 --2- 172.21.15.112:0/2143844850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721013c380 0x7f72100cb7f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7208009e50 tx=0x7f720800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff3aa0b84734b6c5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7f720e7fc700 1 -- 172.21.15.112:0/2143844850 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720800e070 con 0x7f721013c380 2023-08-27T11:17:11.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7f720e7fc700 1 -- 172.21.15.112:0/2143844850 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7208003a20 con 0x7f721013c380 2023-08-27T11:17:11.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7f720e7fc700 1 -- 172.21.15.112:0/2143844850 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7208003dd0 con 0x7f721013c380 2023-08-27T11:17:11.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7fc7d67fc700 1 -- 172.21.15.112:0/2339597067 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc7dc088590 con 0x7fc7cc004850 2023-08-27T11:17:11.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7fc7e572a700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7cc06bae0 0x7fc7cc06dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7fa9ceffd700 1 -- 172.21.15.112:0/179018862 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0003e30 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7fa9ceffd700 1 -- 172.21.15.112:0/179018862 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d0005070 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7fa9ceffd700 1 -- 172.21.15.112:0/179018862 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00053e0 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7fa9e055f700 1 -- 172.21.15.112:0/179018862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d815ba90 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7fa9e055f700 1 -- 172.21.15.112:0/179018862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d81687d0 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7f96f10f2700 1 -- 172.21.15.112:0/2004167327 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ec15db40 msgr2=0x7f96ec0cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7f96f10f2700 1 --2- 172.21.15.112:0/2004167327 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ec15db40 0x7f96ec0cb7f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009840 tx=0x7f96e0009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7f96f10f2700 1 -- 172.21.15.112:0/2004167327 shutdown_connections 2023-08-27T11:17:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7f96f10f2700 1 --2- 172.21.15.112:0/2004167327 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ec15db40 0x7f96ec0cb7f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7f96f10f2700 1 --2- 172.21.15.112:0/2004167327 >> v2:172.21.15.138:3300/0 conn(0x7f96ec1598e0 0x7f96ec15d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7f96f10f2700 1 -- 172.21.15.112:0/2004167327 >> 172.21.15.112:0/2004167327 conn(0x7f96ec0b0a10 msgr2=0x7f96ec0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7f96f10f2700 1 -- 172.21.15.112:0/2004167327 shutdown_connections 2023-08-27T11:17:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7f96f10f2700 1 -- 172.21.15.112:0/2004167327 wait complete. 2023-08-27T11:17:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7fa9e055f700 1 -- 172.21.15.112:0/179018862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d8067a20 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7f7215b55700 1 -- 172.21.15.112:0/2143844850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721013c380 msgr2=0x7f72100cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7f7215b55700 1 --2- 172.21.15.112:0/2143844850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721013c380 0x7f72100cb7f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7208009e50 tx=0x7f720800b330 comp rx=0 tx=0).stop 2023-08-27T11:17:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7f7215b55700 1 -- 172.21.15.112:0/2143844850 shutdown_connections 2023-08-27T11:17:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7f7215b55700 1 --2- 172.21.15.112:0/2143844850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721013c380 0x7f72100cb7f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.599+0000 7f7215b55700 1 -- 172.21.15.112:0/2143844850 >> 172.21.15.112:0/2143844850 conn(0x7f72100b0a10 msgr2=0x7f72100b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.598+0000 7fc7e818f700 1 -- 172.21.15.112:0/2339597067 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c80052f0 con 0x7fc7cc004850 2023-08-27T11:17:11.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.600+0000 7fc7e572a700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7cc06bae0 0x7fc7cc06dfa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00711d0 tx=0x7fc7d0009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.600+0000 7f176dd9b700 1 --2- 172.21.15.112:0/3154813994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17700b2860 0x7f1770160f40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1760009dc0 tx=0x7f176000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0dc99872257cc57 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.600+0000 7f7215b55700 1 -- 172.21.15.112:0/2143844850 shutdown_connections 2023-08-27T11:17:11.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.600+0000 7f7c60cc3700 1 -- 172.21.15.112:0/3756924566 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c004230 con 0x7f7c5c0b4bf0 2023-08-27T11:17:11.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.600+0000 7f7c60cc3700 1 -- 172.21.15.112:0/3756924566 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4c004390 con 0x7f7c5c0b4bf0 2023-08-27T11:17:11.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.600+0000 7f7c60cc3700 1 -- 172.21.15.112:0/3756924566 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c0036a0 con 0x7f7c5c0b4bf0 2023-08-27T11:17:11.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.601+0000 7f7c63f29700 1 -- 172.21.15.112:0/3756924566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c5c0b4bf0 msgr2=0x7f7c5c0701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.601+0000 7f7c63f29700 1 --2- 172.21.15.112:0/3756924566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c5c0b4bf0 0x7f7c5c0701a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009840 tx=0x7f7c4c009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:11.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.601+0000 7f7c63f29700 1 -- 172.21.15.112:0/3756924566 shutdown_connections 2023-08-27T11:17:11.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.601+0000 7f7c63f29700 1 --2- 172.21.15.112:0/3756924566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c5c0b4bf0 0x7f7c5c0701a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.601+0000 7f7c63f29700 1 -- 172.21.15.112:0/3756924566 >> 172.21.15.112:0/3756924566 conn(0x7f7c5c06c920 msgr2=0x7f7c5c06cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.601+0000 7fa9ceffd700 1 -- 172.21.15.112:0/179018862 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa9d0006bc0 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.601+0000 7fa9ceffd700 1 --2- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa9bc06a0e0 0x7fa9bc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.601+0000 7fa9ceffd700 1 -- 172.21.15.112:0/179018862 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa9d0087c00 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.601+0000 7f96f10f2700 1 Processor -- start 2023-08-27T11:17:11.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.601+0000 7f7215b55700 1 -- 172.21.15.112:0/2143844850 wait complete. 2023-08-27T11:17:11.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.601+0000 7f7c63f29700 1 -- 172.21.15.112:0/3756924566 shutdown_connections 2023-08-27T11:17:11.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.601+0000 7f7c63f29700 1 -- 172.21.15.112:0/3756924566 wait complete. 2023-08-27T11:17:11.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.601+0000 7f9b297fa700 1 -- 172.21.15.112:0/606435181 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9b1c0075c0 con 0x7f9b2c15da90 2023-08-27T11:17:11.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.602+0000 7f9b297fa700 1 --2- 172.21.15.112:0/606435181 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b1806a130 0x7f9b1806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.602+0000 7f9b297fa700 1 -- 172.21.15.112:0/606435181 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9b1c087cb0 con 0x7f9b2c15da90 2023-08-27T11:17:11.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.602+0000 7f9b30c39700 1 --2- 172.21.15.112:0/606435181 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b1806a130 0x7f9b1806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.602+0000 7fa9de2fb700 1 --2- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa9bc06a0e0 0x7fa9bc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.602+0000 7f96f10f2700 1 -- start start 2023-08-27T11:17:11.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96f10f2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ec0b1500 0x7f96ec13b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96f10f2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ec1598e0 0x7f96ec152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96f10f2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ec153000 0x7f96ec15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96f10f2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec150ac0 con 0x7f96ec0b1500 2023-08-27T11:17:11.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96f10f2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec150da0 con 0x7f96ec153000 2023-08-27T11:17:11.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96f10f2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec151080 con 0x7f96ec1598e0 2023-08-27T11:17:11.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f7215b55700 1 Processor -- start 2023-08-27T11:17:11.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7fa9de2fb700 1 --2- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa9bc06a0e0 0x7fa9bc06c5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8007fd0 tx=0x7fa9c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96ead9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ec0b1500 0x7f96ec13b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96ead9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ec0b1500 0x7f96ec13b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55004/0 (socket says 172.21.15.112:55004) 2023-08-27T11:17:11.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96ead9d700 1 -- 172.21.15.112:0/1157354051 learned_addr learned my addr 172.21.15.112:0/1157354051 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96ea59c700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ec1598e0 0x7f96ec152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f7215b55700 1 -- start start 2023-08-27T11:17:11.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f7c63f29700 1 Processor -- start 2023-08-27T11:17:11.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96ead9d700 1 -- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ec1598e0 msgr2=0x7f96ec152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96ead9d700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ec1598e0 0x7f96ec152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96ead9d700 1 -- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ec153000 msgr2=0x7f96ec15b950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-27T11:17:11.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96ead9d700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ec153000 0x7f96ec15b950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96ead9d700 1 -- 172.21.15.112:0/1157354051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e00094a0 con 0x7f96ec0b1500 2023-08-27T11:17:11.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f96ea59c700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ec1598e0 0x7f96ec152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:11.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f7215b55700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72100b1500 0x7f721014e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f7215b55700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721013c380 0x7f7210139740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f7215b55700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f721013a860 0x7f72100cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f7215b55700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721014f3e0 con 0x7f72100b1500 2023-08-27T11:17:11.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f7215b55700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721014ee80 con 0x7f721013c380 2023-08-27T11:17:11.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f7215b55700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721014f100 con 0x7f721013a860 2023-08-27T11:17:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f720effd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721013c380 0x7f7210139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f720ffff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f721013a860 0x7f72100cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f720effd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721013c380 0x7f7210139740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55438/0 (socket says 172.21.15.112:55438) 2023-08-27T11:17:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f720effd700 1 -- 172.21.15.112:0/711193814 learned_addr learned my addr 172.21.15.112:0/711193814 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.603+0000 7f7c63f29700 1 -- start start 2023-08-27T11:17:11.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f720ffff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f721013a860 0x7f72100cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38436/0 (socket says 172.21.15.112:38436) 2023-08-27T11:17:11.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f96ead9d700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ec0b1500 0x7f96ec13b8d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00c520 tx=0x7f96dc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f720f7fe700 1 --2- 172.21.15.112:0/711193814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72100b1500 0x7f721014e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f7c63f29700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c5c0716e0 0x7f7c5c158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.604+0000 7f7c63f29700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c5c071ac0 0x7f7c5c1fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c61cc5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c5c0716e0 0x7f7c5c158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c61cc5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c5c0716e0 0x7f7c5c158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38440/0 (socket says 172.21.15.112:38440) 2023-08-27T11:17:11.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c63f29700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c5c0b4bf0 0x7f7c5c0b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c63f29700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1fab30 con 0x7f7c5c0b4bf0 2023-08-27T11:17:11.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c63f29700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c157940 con 0x7f7c5c071ac0 2023-08-27T11:17:11.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c63f29700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1fad60 con 0x7f7c5c0716e0 2023-08-27T11:17:11.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f720f7fe700 1 -- 172.21.15.112:0/711193814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f721013a860 msgr2=0x7f72100cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f720f7fe700 1 --2- 172.21.15.112:0/711193814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f721013a860 0x7f72100cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f720f7fe700 1 -- 172.21.15.112:0/711193814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721013c380 msgr2=0x7f7210139740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f720f7fe700 1 --2- 172.21.15.112:0/711193814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721013c380 0x7f7210139740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f720f7fe700 1 -- 172.21.15.112:0/711193814 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7208009b30 con 0x7f72100b1500 2023-08-27T11:17:11.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c61cc5700 1 -- 172.21.15.112:0/3910107179 learned_addr learned my addr 172.21.15.112:0/3910107179 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f720f7fe700 1 --2- 172.21.15.112:0/711193814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72100b1500 0x7f721014e940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f720000c450 tx=0x7f720000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c624c6700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c5c0b4bf0 0x7f7c5c0b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c614c4700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c5c071ac0 0x7f7c5c1fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c61cc5700 1 -- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c5c071ac0 msgr2=0x7f7c5c1fa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c61cc5700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c5c071ac0 0x7f7c5c1fa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c61cc5700 1 -- 172.21.15.112:0/3910107179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c5c0b4bf0 msgr2=0x7f7c5c0b73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c61cc5700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c5c0b4bf0 0x7f7c5c0b73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c61cc5700 1 -- 172.21.15.112:0/3910107179 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4c0094a0 con 0x7f7c5c0716e0 2023-08-27T11:17:11.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f9b30c39700 1 --2- 172.21.15.112:0/606435181 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b1806a130 0x7f9b1806c5f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000dc30 tx=0x7f9b20014000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f96d37fe700 1 -- 172.21.15.112:0/1157354051 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc005d70 con 0x7f96ec0b1500 2023-08-27T11:17:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f71f7fff700 1 -- 172.21.15.112:0/711193814 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72000052d0 con 0x7f72100b1500 2023-08-27T11:17:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7c61cc5700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c5c0716e0 0x7f7c5c158dd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400b840 tx=0x7f7c5400bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f7215b55700 1 -- 172.21.15.112:0/711193814 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7210150530 con 0x7f72100b1500 2023-08-27T11:17:11.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.606+0000 7f71f7fff700 1 -- 172.21.15.112:0/711193814 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7200005430 con 0x7f72100b1500 2023-08-27T11:17:11.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.606+0000 7f71f7fff700 1 -- 172.21.15.112:0/711193814 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72000038c0 con 0x7f72100b1500 2023-08-27T11:17:11.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.606+0000 7f7215b55700 1 -- 172.21.15.112:0/711193814 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7210150af0 con 0x7f72100b1500 2023-08-27T11:17:11.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.606+0000 7f9b297fa700 1 -- 172.21.15.112:0/606435181 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9b1c053510 con 0x7f9b2c15da90 2023-08-27T11:17:11.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.605+0000 7f96d37fe700 1 -- 172.21.15.112:0/1157354051 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96dc0070d0 con 0x7f96ec0b1500 2023-08-27T11:17:11.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.606+0000 7f96f10f2700 1 -- 172.21.15.112:0/1157354051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ec151360 con 0x7f96ec0b1500 2023-08-27T11:17:11.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.606+0000 7f96f10f2700 1 -- 172.21.15.112:0/1157354051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec151920 con 0x7f96ec0b1500 2023-08-27T11:17:11.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.606+0000 7f96d37fe700 1 -- 172.21.15.112:0/1157354051 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc007440 con 0x7f96ec0b1500 2023-08-27T11:17:11.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.607+0000 7f96f10f2700 1 -- 172.21.15.112:0/1157354051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec067a20 con 0x7f96ec0b1500 2023-08-27T11:17:11.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.607+0000 7f176d59a700 1 -- 172.21.15.112:0/3154813994 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176000e070 con 0x7f17700b2860 2023-08-27T11:17:11.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.607+0000 7f176d59a700 1 -- 172.21.15.112:0/3154813994 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176000bb20 con 0x7f17700b2860 2023-08-27T11:17:11.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.608+0000 7fc7d67fc700 1 -- 172.21.15.112:0/2339597067 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc7dc053c60 con 0x7fc7cc004850 2023-08-27T11:17:11.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.608+0000 7f7c527fc700 1 -- 172.21.15.112:0/3910107179 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5400f840 con 0x7f7c5c0716e0 2023-08-27T11:17:11.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.608+0000 7f7c527fc700 1 -- 172.21.15.112:0/3910107179 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5400f9a0 con 0x7f7c5c0716e0 2023-08-27T11:17:11.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.608+0000 7f7c527fc700 1 -- 172.21.15.112:0/3910107179 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5400fd50 con 0x7f7c5c0716e0 2023-08-27T11:17:11.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.609+0000 7f96d37fe700 1 -- 172.21.15.112:0/1157354051 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f96dc0075a0 con 0x7f96ec0b1500 2023-08-27T11:17:11.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.609+0000 7f96d37fe700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f96d806a0e0 0x7f96d806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.609+0000 7f96d37fe700 1 -- 172.21.15.112:0/1157354051 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f96dc087fe0 con 0x7f96ec0b1500 2023-08-27T11:17:11.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.609+0000 7f96ea59c700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f96d806a0e0 0x7f96d806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.610+0000 7f96ea59c700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f96d806a0e0 0x7f96d806c5a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f96ec1661a0 tx=0x7f96e400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.607+0000 7f176d59a700 1 -- 172.21.15.112:0/3154813994 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760012430 con 0x7f17700b2860 2023-08-27T11:17:11.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.608+0000 7f1774d68700 1 -- 172.21.15.112:0/3154813994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17700b2860 msgr2=0x7f1770160f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.608+0000 7f1774d68700 1 --2- 172.21.15.112:0/3154813994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17700b2860 0x7f1770160f40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1760009dc0 tx=0x7f176000b330 comp rx=0 tx=0).stop 2023-08-27T11:17:11.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.610+0000 7f1774d68700 1 -- 172.21.15.112:0/3154813994 shutdown_connections 2023-08-27T11:17:11.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.610+0000 7f1774d68700 1 --2- 172.21.15.112:0/3154813994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17700b2860 0x7f1770160f40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.610+0000 7f1774d68700 1 -- 172.21.15.112:0/3154813994 >> 172.21.15.112:0/3154813994 conn(0x7f17700b0a10 msgr2=0x7f17700b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.610+0000 7f1774d68700 1 -- 172.21.15.112:0/3154813994 shutdown_connections 2023-08-27T11:17:11.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.610+0000 7f1774d68700 1 -- 172.21.15.112:0/3154813994 wait complete. 2023-08-27T11:17:11.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.612+0000 7f1774d68700 1 Processor -- start 2023-08-27T11:17:11.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.612+0000 7f1774d68700 1 -- start start 2023-08-27T11:17:11.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.613+0000 7f1774d68700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17700b2860 0x7f17701539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.613+0000 7f1774d68700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17701403d0 0x7f17700afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.613+0000 7f1774d68700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1770161480 0x7f1770202b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.613+0000 7f1774d68700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17702002b0 con 0x7f17700b2860 2023-08-27T11:17:11.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.613+0000 7f1774d68700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17701ff6a0 con 0x7f17701403d0 2023-08-27T11:17:11.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.613+0000 7f1774d68700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17701ff980 con 0x7f1770161480 2023-08-27T11:17:11.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.608+0000 7f71f7fff700 1 -- 172.21.15.112:0/711193814 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7200003a20 con 0x7f72100b1500 2023-08-27T11:17:11.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.609+0000 7f71f7fff700 1 --2- 172.21.15.112:0/711193814 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f71fc06a0e0 0x7f71fc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.609+0000 7f71f7fff700 1 -- 172.21.15.112:0/711193814 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7200087960 con 0x7f72100b1500 2023-08-27T11:17:11.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.609+0000 7f720effd700 1 --2- 172.21.15.112:0/711193814 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f71fc06a0e0 0x7f71fc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.609+0000 7f720effd700 1 --2- 172.21.15.112:0/711193814 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f71fc06a0e0 0x7f71fc06c5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7210161130 tx=0x7f7204007f50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.610+0000 7f7215b55700 1 -- 172.21.15.112:0/711193814 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7210150390 con 0x7f72100b1500 2023-08-27T11:17:11.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.610+0000 7fa9ceffd700 1 -- 172.21.15.112:0/179018862 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa9d0053340 con 0x7fa9d80cb7f0 2023-08-27T11:17:11.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.610+0000 7f7c63f29700 1 -- 172.21.15.112:0/3910107179 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5c1fb040 con 0x7f7c5c0716e0 2023-08-27T11:17:11.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.610+0000 7f7c63f29700 1 -- 172.21.15.112:0/3910107179 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5c152c10 con 0x7f7c5c0716e0 2023-08-27T11:17:11.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.613+0000 7f7c527fc700 1 -- 172.21.15.112:0/3910107179 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7c540203f0 con 0x7f7c5c0716e0 2023-08-27T11:17:11.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.613+0000 7f7c527fc700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c4006a130 0x7f7c4006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.613+0000 7f7c527fc700 1 -- 172.21.15.112:0/3910107179 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7c54088920 con 0x7f7c5c0716e0 2023-08-27T11:17:11.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.614+0000 7f7c63f29700 1 -- 172.21.15.112:0/3910107179 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5c1fb560 con 0x7f7c5c0716e0 2023-08-27T11:17:11.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.614+0000 7f96d37fe700 1 -- 172.21.15.112:0/1157354051 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f96dc0538f0 con 0x7f96ec0b1500 2023-08-27T11:17:11.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.617+0000 7f7c614c4700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c4006a130 0x7f7c4006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.617+0000 7f7c614c4700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c4006a130 0x7f7c4006c5f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c08f260 tx=0x7f7c58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.619+0000 7f71f7fff700 1 -- 172.21.15.112:0/711193814 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7210150390 con 0x7f72100b1500 2023-08-27T11:17:11.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.620+0000 7f7c527fc700 1 -- 172.21.15.112:0/3910107179 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7c54053ff0 con 0x7f7c5c0716e0 2023-08-27T11:17:11.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.614+0000 7f176ed9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1770161480 0x7f1770202b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.614+0000 7f176ed9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1770161480 0x7f1770202b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38454/0 (socket says 172.21.15.112:38454) 2023-08-27T11:17:11.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.614+0000 7f176ed9d700 1 -- 172.21.15.112:0/1864311500 learned_addr learned my addr 172.21.15.112:0/1864311500 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.614+0000 7f176ed9d700 1 -- 172.21.15.112:0/1864311500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17701403d0 msgr2=0x7f17700afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.614+0000 7f176ed9d700 1 --2- 172.21.15.112:0/1864311500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17701403d0 0x7f17700afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.614+0000 7f176ed9d700 1 -- 172.21.15.112:0/1864311500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17700b2860 msgr2=0x7f17701539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.614+0000 7f176ed9d700 1 --2- 172.21.15.112:0/1864311500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17700b2860 0x7f17701539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.614+0000 7f176ed9d700 1 -- 172.21.15.112:0/1864311500 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1760009a70 con 0x7f1770161480 2023-08-27T11:17:11.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.614+0000 7f176ed9d700 1 --2- 172.21.15.112:0/1864311500 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1770161480 0x7f1770202b00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f175800c500 tx=0x7f175800c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.618+0000 7f1766ffd700 1 -- 172.21.15.112:0/1864311500 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17580052d0 con 0x7f1770161480 2023-08-27T11:17:11.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.618+0000 7f1766ffd700 1 -- 172.21.15.112:0/1864311500 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1758005430 con 0x7f1770161480 2023-08-27T11:17:11.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.618+0000 7f1774d68700 1 -- 172.21.15.112:0/1864311500 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17701ffc60 con 0x7f1770161480 2023-08-27T11:17:11.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.619+0000 7f1766ffd700 1 -- 172.21.15.112:0/1864311500 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17580038a0 con 0x7f1770161480 2023-08-27T11:17:11.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.619+0000 7f1774d68700 1 -- 172.21.15.112:0/1864311500 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17700b0180 con 0x7f1770161480 2023-08-27T11:17:11.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.620+0000 7f1766ffd700 1 -- 172.21.15.112:0/1864311500 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1758003a00 con 0x7f1770161480 2023-08-27T11:17:11.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.620+0000 7f1766ffd700 1 --2- 172.21.15.112:0/1864311500 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f175006a0e0 0x7f175006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.621+0000 7f1766ffd700 1 -- 172.21.15.112:0/1864311500 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1758087840 con 0x7f1770161480 2023-08-27T11:17:11.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.621+0000 7f1774d68700 1 -- 172.21.15.112:0/1864311500 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17540052f0 con 0x7f1770161480 2023-08-27T11:17:11.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.621+0000 7f176e59c700 1 --2- 172.21.15.112:0/1864311500 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f175006a0e0 0x7f175006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.622+0000 7f0207d66700 1 Processor -- start 2023-08-27T11:17:11.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.622+0000 7f0207d66700 1 -- start start 2023-08-27T11:17:11.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.624+0000 7f176e59c700 1 --2- 172.21.15.112:0/1864311500 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f175006a0e0 0x7f175006c5a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f17701576f0 tx=0x7f17680079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.624+0000 7f0207d66700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f020015dc80 0x7f020006e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.624+0000 7f0207d66700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020015e3f0 con 0x7f02000a9e60 2023-08-27T11:17:11.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.624+0000 7f0207d66700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020015e6d0 con 0x7f02000ce920 2023-08-27T11:17:11.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.624+0000 7f0207d66700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020015a690 con 0x7f020015dc80 2023-08-27T11:17:11.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.624+0000 7f0205b02700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f02000a9e60 0x7f02000cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49418/0 (socket says 172.21.15.112:49418) 2023-08-27T11:17:11.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.624+0000 7f0205b02700 1 -- 172.21.15.112:0/2357445609 learned_addr learned my addr 172.21.15.112:0/2357445609 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:11.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.624+0000 7f0206303700 1 --2- 172.21.15.112:0/2357445609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f020015dc80 0x7f020006e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.624+0000 7f0206303700 1 -- 172.21.15.112:0/2357445609 >> v1:172.21.15.138:6789/0 conn(0x7f02000ce920 legacy=0x7f020015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:11.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.624+0000 7f0206303700 1 -- 172.21.15.112:0/2357445609 >> v1:172.21.15.112:6789/0 conn(0x7f02000a9e60 legacy=0x7f02000cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.624+0000 7f0206303700 1 -- 172.21.15.112:0/2357445609 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020015e9b0 con 0x7f020015dc80 2023-08-27T11:17:11.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.626+0000 7f1766ffd700 1 -- 172.21.15.112:0/1864311500 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1758052fe0 con 0x7f1770161480 2023-08-27T11:17:11.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.632+0000 7f0206303700 1 --2- 172.21.15.112:0/2357445609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f020015dc80 0x7f020006e150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009dc0 tx=0x7f01f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fd4913419c14ffc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.632+0000 7f0204b00700 1 -- 172.21.15.112:0/2357445609 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f800e070 con 0x7f020015dc80 2023-08-27T11:17:11.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.634+0000 7f0204b00700 1 -- 172.21.15.112:0/2357445609 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f800bad0 con 0x7f020015dc80 2023-08-27T11:17:11.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.634+0000 7f0204b00700 1 -- 172.21.15.112:0/2357445609 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f800be60 con 0x7f020015dc80 2023-08-27T11:17:11.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.634+0000 7f0207d66700 1 -- 172.21.15.112:0/2357445609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f020015dc80 msgr2=0x7f020006e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.634+0000 7f0207d66700 1 --2- 172.21.15.112:0/2357445609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f020015dc80 0x7f020006e150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009dc0 tx=0x7f01f800b330 comp rx=0 tx=0).stop 2023-08-27T11:17:11.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.645+0000 7f0207d66700 1 -- 172.21.15.112:0/2357445609 shutdown_connections 2023-08-27T11:17:11.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.645+0000 7f0207d66700 1 --2- 172.21.15.112:0/2357445609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f020015dc80 0x7f020006e150 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.645+0000 7f0207d66700 1 -- 172.21.15.112:0/2357445609 >> 172.21.15.112:0/2357445609 conn(0x7f020006cd30 msgr2=0x7f02000717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:11.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.646+0000 7f0207d66700 1 -- 172.21.15.112:0/2357445609 shutdown_connections 2023-08-27T11:17:11.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.649+0000 7f0207d66700 1 -- 172.21.15.112:0/2357445609 wait complete. 2023-08-27T11:17:11.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.650+0000 7f0207d66700 1 Processor -- start 2023-08-27T11:17:11.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.651+0000 7f0207d66700 1 -- start start 2023-08-27T11:17:11.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.652+0000 7f0207d66700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02000a9e60 0x7f0200149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.652+0000 7f0207d66700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02000ce920 0x7f0200149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.652+0000 7f0207d66700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020015dc80 0x7f0200202bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.652+0000 7f0207d66700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020015c910 con 0x7f020015dc80 2023-08-27T11:17:11.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.652+0000 7f0207d66700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020015c630 con 0x7f02000a9e60 2023-08-27T11:17:11.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.652+0000 7f0207d66700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020006f6a0 con 0x7f02000ce920 2023-08-27T11:17:11.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.653+0000 7f0205b02700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02000a9e60 0x7f0200149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.654+0000 7f0206303700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020015dc80 0x7f0200202bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.654+0000 7f0206303700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020015dc80 0x7f0200202bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55046/0 (socket says 172.21.15.112:55046) 2023-08-27T11:17:11.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.654+0000 7f0206303700 1 -- 172.21.15.112:0/1561238215 learned_addr learned my addr 172.21.15.112:0/1561238215 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:11.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.655+0000 7f0206303700 1 -- 172.21.15.112:0/1561238215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02000ce920 msgr2=0x7f0200149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.655+0000 7f0206303700 1 --2- 172.21.15.112:0/1561238215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02000ce920 0x7f0200149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.655+0000 7f0206303700 1 -- 172.21.15.112:0/1561238215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02000a9e60 msgr2=0x7f0200149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:11.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.655+0000 7f0206303700 1 --2- 172.21.15.112:0/1561238215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02000a9e60 0x7f0200149310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:11.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.655+0000 7f0206303700 1 -- 172.21.15.112:0/1561238215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f8009a70 con 0x7f020015dc80 2023-08-27T11:17:11.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.655+0000 7f0206303700 1 --2- 172.21.15.112:0/1561238215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020015dc80 0x7f0200202bc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f01f800b370 tx=0x7f01f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:11.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.655+0000 7f01f67fc700 1 -- 172.21.15.112:0/1561238215 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f80047a0 con 0x7f020015dc80 2023-08-27T11:17:11.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.656+0000 7f01f67fc700 1 -- 172.21.15.112:0/1561238215 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f8004900 con 0x7f020015dc80 2023-08-27T11:17:11.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.656+0000 7f01f67fc700 1 -- 172.21.15.112:0/1561238215 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f8012690 con 0x7f020015dc80 2023-08-27T11:17:11.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.657+0000 7f0207d66700 1 -- 172.21.15.112:0/1561238215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020006f980 con 0x7f020015dc80 2023-08-27T11:17:11.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.657+0000 7f0207d66700 1 -- 172.21.15.112:0/1561238215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0200203100 con 0x7f020015dc80 2023-08-27T11:17:11.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.660+0000 7f01f67fc700 1 -- 172.21.15.112:0/1561238215 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f01f8019030 con 0x7f020015dc80 2023-08-27T11:17:11.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.660+0000 7f01f67fc700 1 --2- 172.21.15.112:0/1561238215 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f01ec06a2c0 0x7f01ec06c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:11.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.660+0000 7f01f67fc700 1 -- 172.21.15.112:0/1561238215 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f01f808a5a0 con 0x7f020015dc80 2023-08-27T11:17:11.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.667+0000 7f0205b02700 1 --2- 172.21.15.112:0/1561238215 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f01ec06a2c0 0x7f01ec06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:11.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.667+0000 7f0207d66700 1 -- 172.21.15.112:0/1561238215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e80052f0 con 0x7f020015dc80 2023-08-27T11:17:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.677+0000 7f01f67fc700 1 -- 172.21.15.112:0/1561238215 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f01f8055d00 con 0x7f020015dc80 2023-08-27T11:17:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.705+0000 7f0205b02700 1 --2- 172.21.15.112:0/1561238215 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f01ec06a2c0 0x7f01ec06c780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f01fc001810 tx=0x7f01fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:12.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.998+0000 7f7c63f29700 1 -- 172.21.15.112:0/3910107179 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7c5c067a20 con 0x7f7c5c0716e0 2023-08-27T11:17:12.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:11.999+0000 7f7c527fc700 1 -- 172.21.15.112:0/3910107179 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7c54057610 con 0x7f7c5c0716e0 2023-08-27T11:17:12.003 INFO:teuthology.orchestra.run.smithi112.stdout:55834574854 2023-08-27T11:17:12.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.002+0000 7fc7e818f700 1 -- 172.21.15.112:0/2339597067 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc7c80055a0 con 0x7fc7cc004850 2023-08-27T11:17:12.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.003+0000 7f7c63f29700 1 -- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c4006a130 msgr2=0x7f7c4006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.003+0000 7f7c63f29700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c4006a130 0x7f7c4006c5f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c08f260 tx=0x7f7c58009040 comp rx=0 tx=0).stop 2023-08-27T11:17:12.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.003+0000 7f7c63f29700 1 -- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c5c0716e0 msgr2=0x7f7c5c158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.003+0000 7f7c63f29700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c5c0716e0 0x7f7c5c158dd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400b840 tx=0x7f7c5400bb50 comp rx=0 tx=0).stop 2023-08-27T11:17:12.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.003+0000 7fc7d67fc700 1 -- 172.21.15.112:0/2339597067 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc7dc021020 con 0x7fc7cc004850 2023-08-27T11:17:12.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.003+0000 7f9b32e9d700 1 -- 172.21.15.112:0/606435181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9b0c005870 con 0x7f9b2c15da90 2023-08-27T11:17:12.006 INFO:teuthology.orchestra.run.smithi112.stdout:55834574853 2023-08-27T11:17:12.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.003+0000 7f7c63f29700 1 -- 172.21.15.112:0/3910107179 shutdown_connections 2023-08-27T11:17:12.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.003+0000 7f7c63f29700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c4006a130 0x7f7c4006c5f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.003+0000 7f7c63f29700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c5c0b4bf0 0x7f7c5c0b73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.003+0000 7f7c63f29700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c5c0716e0 0x7f7c5c158dd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.003+0000 7f7c63f29700 1 --2- 172.21.15.112:0/3910107179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c5c071ac0 0x7f7c5c1fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.003+0000 7f7c63f29700 1 -- 172.21.15.112:0/3910107179 >> 172.21.15.112:0/3910107179 conn(0x7f7c5c06c920 msgr2=0x7f7c5c15d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:12.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.004+0000 7f9b297fa700 1 -- 172.21.15.112:0/606435181 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9b1c056b30 con 0x7f9b2c15da90 2023-08-27T11:17:12.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.004+0000 7f7c63f29700 1 -- 172.21.15.112:0/3910107179 shutdown_connections 2023-08-27T11:17:12.009 INFO:teuthology.orchestra.run.smithi112.stdout:55834574854 2023-08-27T11:17:12.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.005+0000 7f7c63f29700 1 -- 172.21.15.112:0/3910107179 wait complete. 2023-08-27T11:17:12.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.007+0000 7fc7e818f700 1 -- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7cc06bae0 msgr2=0x7fc7cc06dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.007+0000 7fc7e818f700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7cc06bae0 0x7fc7cc06dfa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00711d0 tx=0x7fc7d0009030 comp rx=0 tx=0).stop 2023-08-27T11:17:12.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.007+0000 7fc7e818f700 1 -- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7cc004850 msgr2=0x7fc7e00ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.007+0000 7fc7e818f700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7cc004850 0x7fc7e00ca930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc00e830 tx=0x7fc7dc00ebf0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.007+0000 7fc7e818f700 1 -- 172.21.15.112:0/2339597067 shutdown_connections 2023-08-27T11:17:12.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.007+0000 7fc7e818f700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7cc06bae0 0x7fc7cc06dfa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.007+0000 7fc7e818f700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e013bbc0 0x7fc7e013c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.007+0000 7fc7e818f700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7cc004850 0x7fc7e00ca930 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.007+0000 7fc7e818f700 1 --2- 172.21.15.112:0/2339597067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e01f6880 0x7fc7e01f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.007+0000 7fc7e818f700 1 -- 172.21.15.112:0/2339597067 >> 172.21.15.112:0/2339597067 conn(0x7fc7e006cd30 msgr2=0x7fc7e00a8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:12.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.008+0000 7f9b32e9d700 1 -- 172.21.15.112:0/606435181 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b1806a130 msgr2=0x7f9b1806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.008+0000 7f9b32e9d700 1 --2- 172.21.15.112:0/606435181 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b1806a130 0x7f9b1806c5f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000dc30 tx=0x7f9b20014000 comp rx=0 tx=0).stop 2023-08-27T11:17:12.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.008+0000 7f9b32e9d700 1 -- 172.21.15.112:0/606435181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b2c15da90 msgr2=0x7f9b2c1f3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.008+0000 7f9b32e9d700 1 --2- 172.21.15.112:0/606435181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b2c15da90 0x7f9b2c1f3940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00c550 tx=0x7f9b1c00c860 comp rx=0 tx=0).stop 2023-08-27T11:17:12.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.011+0000 7f9b32e9d700 1 -- 172.21.15.112:0/606435181 shutdown_connections 2023-08-27T11:17:12.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.011+0000 7f9b32e9d700 1 --2- 172.21.15.112:0/606435181 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b1806a130 0x7f9b1806c5f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.011+0000 7f9b32e9d700 1 --2- 172.21.15.112:0/606435181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b2c15da90 0x7f9b2c1f3940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.011+0000 7f9b32e9d700 1 --2- 172.21.15.112:0/606435181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b2c13c160 0x7f9b2c2001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.011+0000 7f9b32e9d700 1 --2- 172.21.15.112:0/606435181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b2c15d600 0x7f9b2c1f3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.011+0000 7f9b32e9d700 1 -- 172.21.15.112:0/606435181 >> 172.21.15.112:0/606435181 conn(0x7f9b2c0b0a10 msgr2=0x7f9b2c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.011+0000 7fc7e818f700 1 -- 172.21.15.112:0/2339597067 shutdown_connections 2023-08-27T11:17:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.011+0000 7fc7e818f700 1 -- 172.21.15.112:0/2339597067 wait complete. 2023-08-27T11:17:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.014+0000 7fa9e055f700 1 -- 172.21.15.112:0/179018862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa9d815b610 con 0x7fa9d80cb7f0 2023-08-27T11:17:12.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.015+0000 7f9b32e9d700 1 -- 172.21.15.112:0/606435181 shutdown_connections 2023-08-27T11:17:12.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.015+0000 7f9b32e9d700 1 -- 172.21.15.112:0/606435181 wait complete. 2023-08-27T11:17:12.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.016+0000 7fa9ceffd700 1 -- 172.21.15.112:0/179018862 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa9d815b610 con 0x7fa9d80cb7f0 2023-08-27T11:17:12.021 INFO:teuthology.orchestra.run.smithi112.stdout:55834574853 2023-08-27T11:17:12.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.022+0000 7fa9e055f700 1 -- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa9bc06a0e0 msgr2=0x7fa9bc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.022+0000 7fa9e055f700 1 --2- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa9bc06a0e0 0x7fa9bc06c5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8007fd0 tx=0x7fa9c8009040 comp rx=0 tx=0).stop 2023-08-27T11:17:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.022+0000 7fa9e055f700 1 -- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9d80cb7f0 msgr2=0x7fa9d815c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.022+0000 7fa9e055f700 1 --2- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9d80cb7f0 0x7fa9d815c1e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000caa0 tx=0x7fa9d000cdb0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.023+0000 7f96f10f2700 1 -- 172.21.15.112:0/1157354051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f96ec071460 con 0x7f96ec0b1500 2023-08-27T11:17:12.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.023+0000 7f7215b55700 1 -- 172.21.15.112:0/711193814 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7210150390 con 0x7f72100b1500 2023-08-27T11:17:12.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.023+0000 7f1774d68700 1 -- 172.21.15.112:0/1864311500 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1754005870 con 0x7f1770161480 2023-08-27T11:17:12.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.024+0000 7fa9e055f700 1 -- 172.21.15.112:0/179018862 shutdown_connections 2023-08-27T11:17:12.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.024+0000 7fa9e055f700 1 --2- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa9bc06a0e0 0x7fa9bc06c5a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.024+0000 7fa9e055f700 1 --2- 172.21.15.112:0/179018862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d813c160 0x7fa9d8167750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.024+0000 7fa9e055f700 1 --2- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d80b1500 0x7fa9d806d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.024+0000 7fa9e055f700 1 --2- 172.21.15.112:0/179018862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9d80cb7f0 0x7fa9d815c1e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.024+0000 7fa9e055f700 1 -- 172.21.15.112:0/179018862 >> 172.21.15.112:0/179018862 conn(0x7fa9d80b0a10 msgr2=0x7fa9d813b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.024+0000 7f96d37fe700 1 -- 172.21.15.112:0/1157354051 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f96dc017020 con 0x7f96ec0b1500 2023-08-27T11:17:12.027 INFO:teuthology.orchestra.run.smithi112.stdout:55834574854 2023-08-27T11:17:12.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.024+0000 7f71f7fff700 1 -- 172.21.15.112:0/711193814 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7210150390 con 0x7f72100b1500 2023-08-27T11:17:12.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.024+0000 7fa9e055f700 1 -- 172.21.15.112:0/179018862 shutdown_connections 2023-08-27T11:17:12.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.024+0000 7fa9e055f700 1 -- 172.21.15.112:0/179018862 wait complete. 2023-08-27T11:17:12.029 INFO:teuthology.orchestra.run.smithi112.stdout:55834574854 2023-08-27T11:17:12.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.027+0000 7f1766ffd700 1 -- 172.21.15.112:0/1864311500 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1758017020 con 0x7f1770161480 2023-08-27T11:17:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.028+0000 7f96f10f2700 1 -- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f96d806a0e0 msgr2=0x7f96d806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.028+0000 7f96f10f2700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f96d806a0e0 0x7f96d806c5a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f96ec1661a0 tx=0x7f96e400b040 comp rx=0 tx=0).stop 2023-08-27T11:17:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.028+0000 7f96f10f2700 1 -- 172.21.15.112:0/1157354051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ec0b1500 msgr2=0x7f96ec13b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.028+0000 7f96f10f2700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ec0b1500 0x7f96ec13b8d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00c520 tx=0x7f96dc00c830 comp rx=0 tx=0).stop 2023-08-27T11:17:12.031 INFO:teuthology.orchestra.run.smithi112.stdout:55834574853 2023-08-27T11:17:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.028+0000 7f7215b55700 1 -- 172.21.15.112:0/711193814 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f71fc06a0e0 msgr2=0x7f71fc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.028+0000 7f7215b55700 1 --2- 172.21.15.112:0/711193814 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f71fc06a0e0 0x7f71fc06c5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7210161130 tx=0x7f7204007f50 comp rx=0 tx=0).stop 2023-08-27T11:17:12.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.028+0000 7f7215b55700 1 -- 172.21.15.112:0/711193814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72100b1500 msgr2=0x7f721014e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.028+0000 7f7215b55700 1 --2- 172.21.15.112:0/711193814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72100b1500 0x7f721014e940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f720000c450 tx=0x7f720000c760 comp rx=0 tx=0).stop 2023-08-27T11:17:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.029+0000 7f96f10f2700 1 -- 172.21.15.112:0/1157354051 shutdown_connections 2023-08-27T11:17:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.029+0000 7f96f10f2700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f96d806a0e0 0x7f96d806c5a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.029+0000 7f96f10f2700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ec153000 0x7f96ec15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.029+0000 7f96f10f2700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ec0b1500 0x7f96ec13b8d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.029+0000 7f96f10f2700 1 --2- 172.21.15.112:0/1157354051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ec1598e0 0x7f96ec152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.029+0000 7f96f10f2700 1 -- 172.21.15.112:0/1157354051 >> 172.21.15.112:0/1157354051 conn(0x7f96ec0b0a10 msgr2=0x7f96ec06d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:12.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.029+0000 7f96f10f2700 1 -- 172.21.15.112:0/1157354051 shutdown_connections 2023-08-27T11:17:12.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.029+0000 7f96f10f2700 1 -- 172.21.15.112:0/1157354051 wait complete. 2023-08-27T11:17:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f1774d68700 1 -- 172.21.15.112:0/1864311500 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f175006a0e0 msgr2=0x7f175006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f1774d68700 1 --2- 172.21.15.112:0/1864311500 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f175006a0e0 0x7f175006c5a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f17701576f0 tx=0x7f17680079c0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f1774d68700 1 -- 172.21.15.112:0/1864311500 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1770161480 msgr2=0x7f1770202b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f1774d68700 1 --2- 172.21.15.112:0/1864311500 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1770161480 0x7f1770202b00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f175800c500 tx=0x7f175800c8c0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f1774d68700 1 -- 172.21.15.112:0/1864311500 shutdown_connections 2023-08-27T11:17:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f1774d68700 1 --2- 172.21.15.112:0/1864311500 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f175006a0e0 0x7f175006c5a0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f17701576f0 tx=0x7f17680079c0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f1774d68700 1 --2- 172.21.15.112:0/1864311500 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1770161480 0x7f1770202b00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f1774d68700 1 --2- 172.21.15.112:0/1864311500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17700b2860 0x7f17701539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f1774d68700 1 --2- 172.21.15.112:0/1864311500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17701403d0 0x7f17700afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f1774d68700 1 -- 172.21.15.112:0/1864311500 >> 172.21.15.112:0/1864311500 conn(0x7f17700b0a10 msgr2=0x7f177006f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f7215b55700 1 -- 172.21.15.112:0/711193814 shutdown_connections 2023-08-27T11:17:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f7215b55700 1 --2- 172.21.15.112:0/711193814 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f71fc06a0e0 0x7f71fc06c5a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f7215b55700 1 --2- 172.21.15.112:0/711193814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f721013a860 0x7f72100cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f7215b55700 1 --2- 172.21.15.112:0/711193814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72100b1500 0x7f721014e940 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f7215b55700 1 --2- 172.21.15.112:0/711193814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721013c380 0x7f7210139740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.030+0000 7f7215b55700 1 -- 172.21.15.112:0/711193814 >> 172.21.15.112:0/711193814 conn(0x7f72100b0a10 msgr2=0x7f7210152900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:12.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.031+0000 7f1774d68700 1 -- 172.21.15.112:0/1864311500 shutdown_connections 2023-08-27T11:17:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.031+0000 7f7215b55700 1 -- 172.21.15.112:0/711193814 shutdown_connections 2023-08-27T11:17:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.031+0000 7f7215b55700 1 -- 172.21.15.112:0/711193814 wait complete. 2023-08-27T11:17:12.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.031+0000 7f1774d68700 1 -- 172.21.15.112:0/1864311500 wait complete. 2023-08-27T11:17:12.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-08-27T11:17:12.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2023-08-27T11:17:12.042 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-08-27T11:17:12.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-08-27T11:17:12.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-08-27T11:17:12.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-08-27T11:17:12.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-08-27T11:17:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.059+0000 7f0207d66700 1 -- 172.21.15.112:0/1561238215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f01e80055a0 con 0x7f020015dc80 2023-08-27T11:17:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.060+0000 7f01f67fc700 1 -- 172.21.15.112:0/1561238215 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01f8017070 con 0x7f020015dc80 2023-08-27T11:17:12.061 INFO:teuthology.orchestra.run.smithi112.stdout:55834574853 2023-08-27T11:17:12.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.062+0000 7f0207d66700 1 -- 172.21.15.112:0/1561238215 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f01ec06a2c0 msgr2=0x7f01ec06c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.062+0000 7f0207d66700 1 --2- 172.21.15.112:0/1561238215 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f01ec06a2c0 0x7f01ec06c780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f01fc001810 tx=0x7f01fc009040 comp rx=0 tx=0).stop 2023-08-27T11:17:12.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.062+0000 7f0207d66700 1 -- 172.21.15.112:0/1561238215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020015dc80 msgr2=0x7f0200202bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:12.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.062+0000 7f0207d66700 1 --2- 172.21.15.112:0/1561238215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020015dc80 0x7f0200202bc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f01f800b370 tx=0x7f01f8004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.062+0000 7f0207d66700 1 -- 172.21.15.112:0/1561238215 shutdown_connections 2023-08-27T11:17:12.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.062+0000 7f0207d66700 1 --2- 172.21.15.112:0/1561238215 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f01ec06a2c0 0x7f01ec06c780 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.062+0000 7f0207d66700 1 --2- 172.21.15.112:0/1561238215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020015dc80 0x7f0200202bc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.062+0000 7f0207d66700 1 --2- 172.21.15.112:0/1561238215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02000a9e60 0x7f0200149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.062+0000 7f0207d66700 1 --2- 172.21.15.112:0/1561238215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02000ce920 0x7f0200149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.062+0000 7f0207d66700 1 -- 172.21.15.112:0/1561238215 >> 172.21.15.112:0/1561238215 conn(0x7f020006cd30 msgr2=0x7f02000cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.062+0000 7f0207d66700 1 -- 172.21.15.112:0/1561238215 shutdown_connections 2023-08-27T11:17:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:12.062+0000 7f0207d66700 1 -- 172.21.15.112:0/1561238215 wait complete. 2023-08-27T11:17:12.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2023-08-27T11:17:13.042 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-08-27T11:17:13.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-08-27T11:17:13.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-08-27T11:17:13.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-08-27T11:17:13.052 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-08-27T11:17:13.052 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-08-27T11:17:13.054 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-08-27T11:17:13.076 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-08-27T11:17:13.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.237+0000 7fa76c361700 1 Processor -- start 2023-08-27T11:17:13.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.237+0000 7fa76c361700 1 -- start start 2023-08-27T11:17:13.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.239+0000 7fa76c361700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa76413c160 0x7fa7641598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.239+0000 7fa76c361700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa76415d600 0x7fa7640cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.239+0000 7fa76c361700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76413f920 con 0x7fa7640b1500 2023-08-27T11:17:13.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.239+0000 7fa76c361700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa764153de0 con 0x7fa76413c160 2023-08-27T11:17:13.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.239+0000 7fa76c361700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76415ce90 con 0x7fa76415d600 2023-08-27T11:17:13.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.239+0000 7fa76a0fd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa76413c160 0x7fa7641598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.239+0000 7fa76a0fd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa76413c160 0x7fa7641598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55050/0 (socket says 172.21.15.112:55050) 2023-08-27T11:17:13.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.239+0000 7fa76a0fd700 1 -- 172.21.15.112:0/3239678466 learned_addr learned my addr 172.21.15.112:0/3239678466 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.240+0000 7fa76a0fd700 1 -- 172.21.15.112:0/3239678466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa76415d600 msgr2=0x7fa7640cb7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:17:13.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.240+0000 7fa76a0fd700 1 --2- 172.21.15.112:0/3239678466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa76415d600 0x7fa7640cb7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.240+0000 7fa76a0fd700 1 -- 172.21.15.112:0/3239678466 >> v1:172.21.15.112:6789/0 conn(0x7fa7640b1500 legacy=0x7fa76406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:13.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.240+0000 7fa76a0fd700 1 -- 172.21.15.112:0/3239678466 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76415a100 con 0x7fa76413c160 2023-08-27T11:17:13.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.240+0000 7fa76a0fd700 1 --2- 172.21.15.112:0/3239678466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76413c160 0x7fa7641598e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa7540098f0 tx=0x7fa754009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270a480e0bd8a964 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.241+0000 7fa7690fb700 1 -- 172.21.15.112:0/3239678466 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa754004230 con 0x7fa76413c160 2023-08-27T11:17:13.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.241+0000 7fa7690fb700 1 -- 172.21.15.112:0/3239678466 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa754004390 con 0x7fa76413c160 2023-08-27T11:17:13.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.241+0000 7fa7690fb700 1 -- 172.21.15.112:0/3239678466 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7540036a0 con 0x7fa76413c160 2023-08-27T11:17:13.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.242+0000 7fa76c361700 1 -- 172.21.15.112:0/3239678466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76413c160 msgr2=0x7fa7641598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.242+0000 7fa76c361700 1 --2- 172.21.15.112:0/3239678466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76413c160 0x7fa7641598e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa7540098f0 tx=0x7fa754009c00 comp rx=0 tx=0).stop 2023-08-27T11:17:13.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.242+0000 7fa76c361700 1 -- 172.21.15.112:0/3239678466 shutdown_connections 2023-08-27T11:17:13.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.242+0000 7fa76c361700 1 --2- 172.21.15.112:0/3239678466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76413c160 0x7fa7641598e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.242+0000 7fa76c361700 1 --2- 172.21.15.112:0/3239678466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa76415d600 0x7fa7640cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.242+0000 7fa76c361700 1 -- 172.21.15.112:0/3239678466 >> 172.21.15.112:0/3239678466 conn(0x7fa7640b0a10 msgr2=0x7fa7640b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.242+0000 7fa76c361700 1 -- 172.21.15.112:0/3239678466 shutdown_connections 2023-08-27T11:17:13.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.244+0000 7ff79304b700 1 Processor -- start 2023-08-27T11:17:13.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.242+0000 7fa76c361700 1 -- 172.21.15.112:0/3239678466 wait complete. 2023-08-27T11:17:13.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.244+0000 7fa76c361700 1 Processor -- start 2023-08-27T11:17:13.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.244+0000 7fa76c361700 1 -- start start 2023-08-27T11:17:13.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.245+0000 7fa76c361700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7640b1500 0x7fa764152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7fa76c361700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa76413c160 0x7fa76415bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7fa76c361700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa76415d600 0x7fa76415c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7fa76c361700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76406d4c0 con 0x7fa7640b1500 2023-08-27T11:17:13.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7fa76c361700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76413edf0 con 0x7fa76415d600 2023-08-27T11:17:13.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7fa76c361700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76413f0d0 con 0x7fa76413c160 2023-08-27T11:17:13.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.244+0000 7ff79304b700 1 -- start start 2023-08-27T11:17:13.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7fa76a0fd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7640b1500 0x7fa764152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7fa76a0fd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7640b1500 0x7fa764152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55064/0 (socket says 172.21.15.112:55064) 2023-08-27T11:17:13.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7fa76a0fd700 1 -- 172.21.15.112:0/2336075047 learned_addr learned my addr 172.21.15.112:0/2336075047 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7ff79304b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff78c15d600 0x7ff78c0cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7ff79304b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c13f920 con 0x7ff78c13c160 2023-08-27T11:17:13.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7ff79304b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c15ce90 con 0x7ff78c15da90 2023-08-27T11:17:13.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7ff79304b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78c153de0 con 0x7ff78c15d600 2023-08-27T11:17:13.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7ff783fff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff78c15d600 0x7ff78c0cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7ff783fff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff78c15d600 0x7ff78c0cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55484/0 (socket says 172.21.15.112:55484) 2023-08-27T11:17:13.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7ff783fff700 1 -- 172.21.15.112:0/322776770 learned_addr learned my addr 172.21.15.112:0/322776770 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7fa76a0fd700 1 -- 172.21.15.112:0/2336075047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa76413c160 msgr2=0x7fa76415bef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-27T11:17:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7fa76a0fd700 1 --2- 172.21.15.112:0/2336075047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa76413c160 0x7fa76415bef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7fa76a0fd700 1 -- 172.21.15.112:0/2336075047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa76415d600 msgr2=0x7fa76415c430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-27T11:17:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7fa76a0fd700 1 --2- 172.21.15.112:0/2336075047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa76415d600 0x7fa76415c430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.246+0000 7fa76a0fd700 1 -- 172.21.15.112:0/2336075047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa754009550 con 0x7fa7640b1500 2023-08-27T11:17:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.247+0000 7fa76a0fd700 1 --2- 172.21.15.112:0/2336075047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7640b1500 0x7fa764152290 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa754009c40 tx=0x7fa754004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.247+0000 7fa75affd700 1 -- 172.21.15.112:0/2336075047 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75400b860 con 0x7fa7640b1500 2023-08-27T11:17:13.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.247+0000 7fa75affd700 1 -- 172.21.15.112:0/2336075047 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75400b9c0 con 0x7fa7640b1500 2023-08-27T11:17:13.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.247+0000 7fa75affd700 1 -- 172.21.15.112:0/2336075047 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75400bd30 con 0x7fa7640b1500 2023-08-27T11:17:13.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.247+0000 7fa76c361700 1 -- 172.21.15.112:0/2336075047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7640ce9e0 con 0x7fa7640b1500 2023-08-27T11:17:13.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.247+0000 7fa76c361700 1 -- 172.21.15.112:0/2336075047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7640cf880 con 0x7fa7640b1500 2023-08-27T11:17:13.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.249+0000 7fa76c361700 1 -- 172.21.15.112:0/2336075047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76413d6c0 con 0x7fa7640b1500 2023-08-27T11:17:13.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.248+0000 7ff7837fe700 1 -- 172.21.15.112:0/322776770 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176429627 0 0) 0x7ff78c13f920 con 0x7ff78c13c160 2023-08-27T11:17:13.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.248+0000 7ff7837fe700 1 -- 172.21.15.112:0/322776770 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7740035b0 con 0x7ff78c13c160 2023-08-27T11:17:13.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.249+0000 7ff783fff700 1 -- 172.21.15.112:0/322776770 >> v1:172.21.15.138:6789/0 conn(0x7ff78c15da90 legacy=0x7ff78c0cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.249+0000 7ff783fff700 1 -- 172.21.15.112:0/322776770 >> v1:172.21.15.112:6789/0 conn(0x7ff78c13c160 legacy=0x7ff78c1598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.249+0000 7ff783fff700 1 -- 172.21.15.112:0/322776770 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c15a100 con 0x7ff78c15d600 2023-08-27T11:17:13.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.249+0000 7ff783fff700 1 --2- 172.21.15.112:0/322776770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78c15d600 0x7ff78c0cb7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff7880088f0 tx=0x7ff788008c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff5103d842782e4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.250+0000 7ff7837fe700 1 -- 172.21.15.112:0/322776770 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800b590 con 0x7ff78c15d600 2023-08-27T11:17:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.250+0000 7ff7837fe700 1 -- 172.21.15.112:0/322776770 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78800b6f0 con 0x7ff78c15d600 2023-08-27T11:17:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.250+0000 7ff7837fe700 1 -- 172.21.15.112:0/322776770 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800ba80 con 0x7ff78c15d600 2023-08-27T11:17:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.251+0000 7ff79304b700 1 -- 172.21.15.112:0/322776770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78c15d600 msgr2=0x7ff78c0cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.251+0000 7ff79304b700 1 --2- 172.21.15.112:0/322776770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78c15d600 0x7ff78c0cb7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff7880088f0 tx=0x7ff788008c00 comp rx=0 tx=0).stop 2023-08-27T11:17:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.250+0000 7fa75affd700 1 -- 172.21.15.112:0/2336075047 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa754021070 con 0x7fa7640b1500 2023-08-27T11:17:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.251+0000 7fa75affd700 1 --2- 172.21.15.112:0/2336075047 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa74806a0e0 0x7fa74806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.251+0000 7fa75affd700 1 -- 172.21.15.112:0/2336075047 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa754089fc0 con 0x7fa7640b1500 2023-08-27T11:17:13.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.251+0000 7ff79304b700 1 -- 172.21.15.112:0/322776770 shutdown_connections 2023-08-27T11:17:13.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.251+0000 7ff79304b700 1 --2- 172.21.15.112:0/322776770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78c15d600 0x7ff78c0cb7f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.251+0000 7ff79304b700 1 -- 172.21.15.112:0/322776770 >> 172.21.15.112:0/322776770 conn(0x7ff78c0b0a10 msgr2=0x7ff78c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.251+0000 7ff79304b700 1 -- 172.21.15.112:0/322776770 shutdown_connections 2023-08-27T11:17:13.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.251+0000 7fa7698fc700 1 --2- 172.21.15.112:0/2336075047 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa74806a0e0 0x7fa74806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.251+0000 7ff79304b700 1 -- 172.21.15.112:0/322776770 wait complete. 2023-08-27T11:17:13.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.252+0000 7fa7698fc700 1 --2- 172.21.15.112:0/2336075047 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa74806a0e0 0x7fa74806c5a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa760006fd0 tx=0x7fa760006bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.253+0000 7ff79304b700 1 Processor -- start 2023-08-27T11:17:13.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.253+0000 7ff79304b700 1 -- start start 2023-08-27T11:17:13.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff79304b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78c13c160 0x7ff78c1f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff79304b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff78c15d600 0x7ff78c155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff79304b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c15da90 0x7ff78c156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff79304b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c154930 con 0x7ff78c15d600 2023-08-27T11:17:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff79304b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c155190 con 0x7ff78c13c160 2023-08-27T11:17:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff79304b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c1f6b20 con 0x7ff78c15da90 2023-08-27T11:17:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff783fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff78c15d600 0x7ff78c155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff783fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff78c15d600 0x7ff78c155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55074/0 (socket says 172.21.15.112:55074) 2023-08-27T11:17:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff783fff700 1 -- 172.21.15.112:0/1153879083 learned_addr learned my addr 172.21.15.112:0/1153879083 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff783fff700 1 -- 172.21.15.112:0/1153879083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c15da90 msgr2=0x7ff78c156300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-27T11:17:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff783fff700 1 --2- 172.21.15.112:0/1153879083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c15da90 0x7ff78c156300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff783fff700 1 -- 172.21.15.112:0/1153879083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78c13c160 msgr2=0x7ff78c1f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff783fff700 1 --2- 172.21.15.112:0/1153879083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78c13c160 0x7ff78c1f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.255+0000 7ff783fff700 1 -- 172.21.15.112:0/1153879083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7880085d0 con 0x7ff78c15d600 2023-08-27T11:17:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.256+0000 7ff783fff700 1 --2- 172.21.15.112:0/1153879083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff78c15d600 0x7ff78c155dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff788008c40 tx=0x7ff788004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.258+0000 7ff7817fa700 1 -- 172.21.15.112:0/1153879083 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800b860 con 0x7ff78c15d600 2023-08-27T11:17:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.258+0000 7ff7817fa700 1 -- 172.21.15.112:0/1153879083 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78800b9c0 con 0x7ff78c15d600 2023-08-27T11:17:13.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.258+0000 7ff7817fa700 1 -- 172.21.15.112:0/1153879083 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788004270 con 0x7ff78c15d600 2023-08-27T11:17:13.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.258+0000 7ff79304b700 1 -- 172.21.15.112:0/1153879083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c15ca40 con 0x7ff78c15d600 2023-08-27T11:17:13.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.258+0000 7ff79304b700 1 -- 172.21.15.112:0/1153879083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c154e20 con 0x7ff78c15d600 2023-08-27T11:17:13.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.258+0000 7fa75affd700 1 -- 172.21.15.112:0/2336075047 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa7540556a0 con 0x7fa7640b1500 2023-08-27T11:17:13.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.260+0000 7ff79304b700 1 -- 172.21.15.112:0/1153879083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76c0052f0 con 0x7ff78c15d600 2023-08-27T11:17:13.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.268+0000 7faea9444700 1 Processor -- start 2023-08-27T11:17:13.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.268+0000 7faea9444700 1 -- start start 2023-08-27T11:17:13.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.269+0000 7faea9444700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea415c030 0x7faea406d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.269+0000 7faea9444700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea415c410 con 0x7faea4162780 2023-08-27T11:17:13.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.269+0000 7faea9444700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea4162da0 con 0x7faea4166270 2023-08-27T11:17:13.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.269+0000 7faea9444700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea40b4be0 con 0x7faea415c030 2023-08-27T11:17:13.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.270+0000 7faea27fc700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7faea4166270 0x7faea4162070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:56984/0 (socket says 172.21.15.112:56984) 2023-08-27T11:17:13.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.270+0000 7faea27fc700 1 -- 172.21.15.112:0/3910346364 learned_addr learned my addr 172.21.15.112:0/3910346364 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:13.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.270+0000 7faea2ffd700 1 --2- 172.21.15.112:0/3910346364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea415c030 0x7faea406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.270+0000 7faea1ffb700 1 -- 172.21.15.112:0/3910346364 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804880173 0 0) 0x7faea4162da0 con 0x7faea4166270 2023-08-27T11:17:13.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.270+0000 7faea1ffb700 1 -- 172.21.15.112:0/3910346364 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae8c0035b0 con 0x7faea4166270 2023-08-27T11:17:13.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.270+0000 7faea2ffd700 1 -- 172.21.15.112:0/3910346364 >> v1:172.21.15.138:6789/0 conn(0x7faea4166270 legacy=0x7faea4162070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.270+0000 7faea2ffd700 1 -- 172.21.15.112:0/3910346364 >> v1:172.21.15.112:6789/0 conn(0x7faea4162780 legacy=0x7faea4070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:13.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.270+0000 7faea2ffd700 1 -- 172.21.15.112:0/3910346364 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea4165790 con 0x7faea415c030 2023-08-27T11:17:13.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.271+0000 7faea2ffd700 1 --2- 172.21.15.112:0/3910346364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea415c030 0x7faea406d510 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009dc0 tx=0x7fae9c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0fd9627ad2704c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.271+0000 7faea1ffb700 1 -- 172.21.15.112:0/3910346364 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00e070 con 0x7faea415c030 2023-08-27T11:17:13.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.271+0000 7faea1ffb700 1 -- 172.21.15.112:0/3910346364 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c00bae0 con 0x7faea415c030 2023-08-27T11:17:13.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.271+0000 7faea1ffb700 1 -- 172.21.15.112:0/3910346364 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00be50 con 0x7faea415c030 2023-08-27T11:17:13.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.272+0000 7ff7817fa700 1 -- 172.21.15.112:0/1153879083 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff7880043d0 con 0x7ff78c15d600 2023-08-27T11:17:13.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.272+0000 7ff7817fa700 1 --2- 172.21.15.112:0/1153879083 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff77406a590 0x7ff77406ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.272+0000 7ff7817fa700 1 -- 172.21.15.112:0/1153879083 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff78808a2f0 con 0x7ff78c15d600 2023-08-27T11:17:13.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.272+0000 7faea9444700 1 -- 172.21.15.112:0/3910346364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea415c030 msgr2=0x7faea406d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.272+0000 7faea9444700 1 --2- 172.21.15.112:0/3910346364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea415c030 0x7faea406d510 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009dc0 tx=0x7fae9c00b330 comp rx=0 tx=0).stop 2023-08-27T11:17:13.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.273+0000 7ff790de7700 1 --2- 172.21.15.112:0/1153879083 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff77406a590 0x7ff77406ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.273+0000 7faea9444700 1 -- 172.21.15.112:0/3910346364 shutdown_connections 2023-08-27T11:17:13.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.273+0000 7faea9444700 1 --2- 172.21.15.112:0/3910346364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea415c030 0x7faea406d510 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.273+0000 7faea9444700 1 -- 172.21.15.112:0/3910346364 >> 172.21.15.112:0/3910346364 conn(0x7faea40cfe50 msgr2=0x7faea40d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.273+0000 7faea9444700 1 -- 172.21.15.112:0/3910346364 shutdown_connections 2023-08-27T11:17:13.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.273+0000 7ff790de7700 1 --2- 172.21.15.112:0/1153879083 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff77406a590 0x7ff77406ca50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0010e0 tx=0x7ff77c0079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.273+0000 7faea9444700 1 -- 172.21.15.112:0/3910346364 wait complete. 2023-08-27T11:17:13.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.274+0000 7ff7817fa700 1 -- 172.21.15.112:0/1153879083 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff7880559d0 con 0x7ff78c15d600 2023-08-27T11:17:13.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.275+0000 7faea9444700 1 Processor -- start 2023-08-27T11:17:13.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.275+0000 7faea9444700 1 -- start start 2023-08-27T11:17:13.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.276+0000 7faea9444700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea415c030 0x7faea41574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.276+0000 7faea9444700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea4162780 0x7faea415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.276+0000 7faea9444700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea4166270 0x7faea415ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.276+0000 7faea9444700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea4157e70 con 0x7faea415c030 2023-08-27T11:17:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.276+0000 7faea9444700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea415d3b0 con 0x7faea4166270 2023-08-27T11:17:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.276+0000 7faea9444700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea415d690 con 0x7faea4162780 2023-08-27T11:17:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.277+0000 7faea2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea415c030 0x7faea41574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.277+0000 7faea2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea415c030 0x7faea41574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55084/0 (socket says 172.21.15.112:55084) 2023-08-27T11:17:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.277+0000 7faea2ffd700 1 -- 172.21.15.112:0/2094138918 learned_addr learned my addr 172.21.15.112:0/2094138918 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.277+0000 7faea37fe700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea4166270 0x7faea415ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.277+0000 7faea27fc700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea4162780 0x7faea415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.277+0000 7faea2ffd700 1 -- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea4162780 msgr2=0x7faea415ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.277+0000 7faea2ffd700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea4162780 0x7faea415ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.277+0000 7faea2ffd700 1 -- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea4166270 msgr2=0x7faea415ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.277+0000 7faea2ffd700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea4166270 0x7faea415ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.277+0000 7faea2ffd700 1 -- 172.21.15.112:0/2094138918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c009a70 con 0x7faea415c030 2023-08-27T11:17:13.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.278+0000 7faea2ffd700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea415c030 0x7faea41574c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00d010 tx=0x7fae9c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.281+0000 7fae8b7fe700 1 -- 172.21.15.112:0/2094138918 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c009410 con 0x7faea415c030 2023-08-27T11:17:13.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.281+0000 7fae8b7fe700 1 -- 172.21.15.112:0/2094138918 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c012430 con 0x7faea415c030 2023-08-27T11:17:13.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.281+0000 7fae8b7fe700 1 -- 172.21.15.112:0/2094138918 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0127e0 con 0x7faea415c030 2023-08-27T11:17:13.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.281+0000 7faea9444700 1 -- 172.21.15.112:0/2094138918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea415d970 con 0x7faea415c030 2023-08-27T11:17:13.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.282+0000 7faea9444700 1 -- 172.21.15.112:0/2094138918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea4157a00 con 0x7faea415c030 2023-08-27T11:17:13.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.285+0000 7fae8b7fe700 1 -- 172.21.15.112:0/2094138918 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fae9c012940 con 0x7faea415c030 2023-08-27T11:17:13.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.286+0000 7fae8b7fe700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fae9006a1b0 0x7fae9006c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.286+0000 7fae8b7fe700 1 -- 172.21.15.112:0/2094138918 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fae9c08a5e0 con 0x7faea415c030 2023-08-27T11:17:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.288+0000 7faea9444700 1 -- 172.21.15.112:0/2094138918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae840052f0 con 0x7faea415c030 2023-08-27T11:17:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.288+0000 7faea27fc700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fae9006a1b0 0x7fae9006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.289+0000 7faea27fc700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fae9006a1b0 0x7fae9006c670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faea40d0770 tx=0x7fae98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.297+0000 7fae8b7fe700 1 -- 172.21.15.112:0/2094138918 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fae9c055e60 con 0x7faea415c030 2023-08-27T11:17:13.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.317+0000 7fadd6343700 1 Processor -- start 2023-08-27T11:17:13.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.317+0000 7fadd6343700 1 -- start start 2023-08-27T11:17:13.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.316+0000 7fa3b87ec700 1 Processor -- start 2023-08-27T11:17:13.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.316+0000 7fa3b87ec700 1 -- start start 2023-08-27T11:17:13.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.317+0000 7f74aed34700 1 Processor -- start 2023-08-27T11:17:13.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fa3b87ec700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3b00b2480 0x7fa3b00cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fa3b87ec700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b00cd770 con 0x7fa3b006e260 2023-08-27T11:17:13.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fa3b87ec700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b016ddb0 con 0x7fa3b01678a0 2023-08-27T11:17:13.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fa3b87ec700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b016e930 con 0x7fa3b00b2480 2023-08-27T11:17:13.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fadd6343700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fadd015f050 0x7fadd0161c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fadd6343700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadd006c450 0x7fadd00ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fadd6343700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd00b1e10 con 0x7fadd015f430 2023-08-27T11:17:13.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fadd6343700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd00b2cb0 con 0x7fadd015f050 2023-08-27T11:17:13.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fadd6343700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd0162530 con 0x7fadd006c450 2023-08-27T11:17:13.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fa3b6d89700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa3b01678a0 0x7fa3b006c810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:56998/0 (socket says 172.21.15.112:56998) 2023-08-27T11:17:13.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fa3b6d89700 1 -- 172.21.15.112:0/762952982 learned_addr learned my addr 172.21.15.112:0/762952982 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:13.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fa3b5d87700 1 --2- 172.21.15.112:0/762952982 >> v2:172.21.15.138:3300/0 conn(0x7fa3b00b2480 0x7fa3b00cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fadcf7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fadd015f430 0x7fadd00b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49482/0 (socket says 172.21.15.112:49482) 2023-08-27T11:17:13.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7fadcf7fe700 1 -- 172.21.15.112:0/2678106589 learned_addr learned my addr 172.21.15.112:0/2678106589 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:13.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fadcffff700 1 --2- 172.21.15.112:0/2678106589 >> v2:172.21.15.138:3300/0 conn(0x7fadd015f050 0x7fadd0161c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fadd48e0700 1 --2- 172.21.15.112:0/2678106589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadd006c450 0x7fadd00ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.318+0000 7f74aed34700 1 -- start start 2023-08-27T11:17:13.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fa3b5586700 1 -- 172.21.15.112:0/762952982 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81975456 0 0) 0x7fa3b016ddb0 con 0x7fa3b01678a0 2023-08-27T11:17:13.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fadcffff700 1 -- 172.21.15.112:0/2678106589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadd006c450 msgr2=0x7fadd00ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:13.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fadcffff700 1 --2- 172.21.15.112:0/2678106589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadd006c450 0x7fadd00ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fadcffff700 1 -- 172.21.15.112:0/2678106589 >> v1:172.21.15.112:6789/0 conn(0x7fadd015f430 legacy=0x7fadd00b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fadcffff700 1 -- 172.21.15.112:0/2678106589 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd0162810 con 0x7fadd015f050 2023-08-27T11:17:13.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fa3b5586700 1 -- 172.21.15.112:0/762952982 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa39c0035b0 con 0x7fa3b01678a0 2023-08-27T11:17:13.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fa3b5d87700 1 -- 172.21.15.112:0/762952982 >> v1:172.21.15.138:6789/0 conn(0x7fa3b01678a0 legacy=0x7fa3b006c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fa3b5d87700 1 -- 172.21.15.112:0/762952982 >> v1:172.21.15.112:6789/0 conn(0x7fa3b006e260 legacy=0x7fa3b00b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fa3b5d87700 1 -- 172.21.15.112:0/762952982 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b016e090 con 0x7fa3b00b2480 2023-08-27T11:17:13.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7f74aed34700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a80cddb0 0x7f74a815f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7f74aed34700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f74a815bf90 0x7f74a80cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7f74aed34700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a815eed0 con 0x7f74a80ce190 2023-08-27T11:17:13.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7f74aed34700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a815f500 con 0x7f74a815bf90 2023-08-27T11:17:13.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7f74aed34700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a815fe60 con 0x7f74a80cddb0 2023-08-27T11:17:13.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fadd48e0700 1 --2- 172.21.15.112:0/2678106589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadd006c450 0x7fadd00ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:13.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fadcffff700 1 --2- 172.21.15.112:0/2678106589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd015f050 0x7fadd0161c90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009840 tx=0x7fadc0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1240a9605d98a6f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7f74ad2d1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f74a815bf90 0x7f74a80cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.319+0000 7fa3b5d87700 1 --2- 172.21.15.112:0/762952982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b00b2480 0x7fa3b00cd230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009840 tx=0x7fa3ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90d2958d9f69037f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7f74acad0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a80cddb0 0x7f74a815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7f74ad2d1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f74a815bf90 0x7f74a80cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55532/0 (socket says 172.21.15.112:55532) 2023-08-27T11:17:13.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7f74ad2d1700 1 -- 172.21.15.112:0/1563604744 learned_addr learned my addr 172.21.15.112:0/1563604744 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7f46f00f3700 1 Processor -- start 2023-08-27T11:17:13.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7f74acad0700 1 -- 172.21.15.112:0/1563604744 >> v2:172.21.15.138:3300/0 conn(0x7f74a815bf90 msgr2=0x7f74a80cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:13.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7f74acad0700 1 --2- 172.21.15.112:0/1563604744 >> v2:172.21.15.138:3300/0 conn(0x7f74a815bf90 0x7f74a80cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7f74acad0700 1 -- 172.21.15.112:0/1563604744 >> v1:172.21.15.138:6789/0 conn(0x7f74a80ce190 legacy=0x7f74a815b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7f74acad0700 1 -- 172.21.15.112:0/1563604744 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a815c450 con 0x7f74a80cddb0 2023-08-27T11:17:13.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7f74acad0700 1 --2- 172.21.15.112:0/1563604744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a80cddb0 0x7f74a815f920 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f749c009840 tx=0x7f749c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=614a03c3ff540eb2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7f74a77fe700 1 -- 172.21.15.112:0/1563604744 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c011830 con 0x7f74a80cddb0 2023-08-27T11:17:13.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7fadceffd700 1 -- 172.21.15.112:0/2678106589 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0011830 con 0x7fadd015f050 2023-08-27T11:17:13.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7fadceffd700 1 -- 172.21.15.112:0/2678106589 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc0011990 con 0x7fadd015f050 2023-08-27T11:17:13.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7fadceffd700 1 -- 172.21.15.112:0/2678106589 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0011d20 con 0x7fadd015f050 2023-08-27T11:17:13.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7fadd6343700 1 -- 172.21.15.112:0/2678106589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd015f050 msgr2=0x7fadd0161c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7fadd6343700 1 --2- 172.21.15.112:0/2678106589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd015f050 0x7fadd0161c90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009840 tx=0x7fadc0009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:13.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.320+0000 7f46f00f3700 1 -- start start 2023-08-27T11:17:13.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.321+0000 7f74a77fe700 1 -- 172.21.15.112:0/1563604744 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c011990 con 0x7f74a80cddb0 2023-08-27T11:17:13.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.321+0000 7f74a77fe700 1 -- 172.21.15.112:0/1563604744 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c011d00 con 0x7f74a80cddb0 2023-08-27T11:17:13.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.321+0000 7fa3b5586700 1 -- 172.21.15.112:0/762952982 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac011830 con 0x7fa3b00b2480 2023-08-27T11:17:13.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.321+0000 7fa3b5586700 1 -- 172.21.15.112:0/762952982 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac011990 con 0x7fa3b00b2480 2023-08-27T11:17:13.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.321+0000 7fa3b5586700 1 -- 172.21.15.112:0/762952982 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac011d20 con 0x7fa3b00b2480 2023-08-27T11:17:13.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.321+0000 7fa3b87ec700 1 -- 172.21.15.112:0/762952982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b00b2480 msgr2=0x7fa3b00cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.321+0000 7fa3b87ec700 1 --2- 172.21.15.112:0/762952982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b00b2480 0x7fa3b00cd230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009840 tx=0x7fa3ac009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:13.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f74aed34700 1 -- 172.21.15.112:0/1563604744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a80cddb0 msgr2=0x7f74a815f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f74aed34700 1 --2- 172.21.15.112:0/1563604744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a80cddb0 0x7f74a815f920 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f749c009840 tx=0x7f749c009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:13.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46f00f3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46e815f050 0x7f46e8161c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46f00f3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f46e815f430 0x7f46e80b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46f00f3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f46e806c450 0x7f46e80ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46f00f3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e80b1e10 con 0x7f46e806c450 2023-08-27T11:17:13.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46f00f3700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e8162530 con 0x7f46e815f430 2023-08-27T11:17:13.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46f00f3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e8162810 con 0x7f46e815f050 2023-08-27T11:17:13.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f74aed34700 1 -- 172.21.15.112:0/1563604744 shutdown_connections 2023-08-27T11:17:13.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f74aed34700 1 --2- 172.21.15.112:0/1563604744 >> v2:172.21.15.138:3300/0 conn(0x7f74a815bf90 0x7f74a80cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f74aed34700 1 --2- 172.21.15.112:0/1563604744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a80cddb0 0x7f74a815f920 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f74aed34700 1 -- 172.21.15.112:0/1563604744 >> 172.21.15.112:0/1563604744 conn(0x7f74a806f970 msgr2=0x7f74a806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7fa3b87ec700 1 -- 172.21.15.112:0/762952982 shutdown_connections 2023-08-27T11:17:13.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7fa3b87ec700 1 --2- 172.21.15.112:0/762952982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b00b2480 0x7fa3b00cd230 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009840 tx=0x7fa3ac009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:13.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7fa3b87ec700 1 -- 172.21.15.112:0/762952982 >> 172.21.15.112:0/762952982 conn(0x7fa3b00b0a10 msgr2=0x7fa3b00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f74aed34700 1 -- 172.21.15.112:0/1563604744 shutdown_connections 2023-08-27T11:17:13.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f74aed34700 1 -- 172.21.15.112:0/1563604744 wait complete. 2023-08-27T11:17:13.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46ed68e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f46e815f430 0x7f46e80b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46ee690700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f46e806c450 0x7f46e80ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46ede8f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46e815f050 0x7f46e8161c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46ed68e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f46e815f430 0x7f46e80b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55536/0 (socket says 172.21.15.112:55536) 2023-08-27T11:17:13.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46ed68e700 1 -- 172.21.15.112:0/756516221 learned_addr learned my addr 172.21.15.112:0/756516221 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7fa3b87ec700 1 -- 172.21.15.112:0/762952982 shutdown_connections 2023-08-27T11:17:13.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46ede8f700 1 -- 172.21.15.112:0/756516221 >> v2:172.21.15.138:3300/0 conn(0x7f46e815f430 msgr2=0x7f46e80b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:13.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46ede8f700 1 --2- 172.21.15.112:0/756516221 >> v2:172.21.15.138:3300/0 conn(0x7f46e815f430 0x7f46e80b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46ede8f700 1 -- 172.21.15.112:0/756516221 >> v2:172.21.15.112:3300/0 conn(0x7f46e806c450 msgr2=0x7f46e80ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:13.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46ede8f700 1 --2- 172.21.15.112:0/756516221 >> v2:172.21.15.112:3300/0 conn(0x7f46e806c450 0x7f46e80ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46ede8f700 1 -- 172.21.15.112:0/756516221 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e80b2080 con 0x7f46e815f050 2023-08-27T11:17:13.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7f46ed68e700 1 --2- 172.21.15.112:0/756516221 >> v2:172.21.15.138:3300/0 conn(0x7f46e815f430 0x7f46e80b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:13.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.323+0000 7f46ede8f700 1 --2- 172.21.15.112:0/756516221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46e815f050 0x7f46e8161c90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46e80b2f40 tx=0x7f46e000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac86c97fd8157d29 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.322+0000 7fa3b87ec700 1 -- 172.21.15.112:0/762952982 wait complete. 2023-08-27T11:17:13.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.324+0000 7fa3b87ec700 1 Processor -- start 2023-08-27T11:17:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.324+0000 7f74aed34700 1 Processor -- start 2023-08-27T11:17:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.324+0000 7fadd6343700 1 -- 172.21.15.112:0/2678106589 shutdown_connections 2023-08-27T11:17:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.324+0000 7fadd6343700 1 --2- 172.21.15.112:0/2678106589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadd006c450 0x7fadd00ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.324+0000 7fadd6343700 1 --2- 172.21.15.112:0/2678106589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd015f050 0x7fadd0161c90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.324+0000 7fadd6343700 1 -- 172.21.15.112:0/2678106589 >> 172.21.15.112:0/2678106589 conn(0x7fadd00b0bf0 msgr2=0x7fadd00b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.324+0000 7fadd6343700 1 -- 172.21.15.112:0/2678106589 shutdown_connections 2023-08-27T11:17:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.324+0000 7fa3b87ec700 1 -- start start 2023-08-27T11:17:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.324+0000 7f74aed34700 1 -- start start 2023-08-27T11:17:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b87ec700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b006e260 0x7fa3b02069d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b87ec700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3b01678a0 0x7fa3b0088c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b87ec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3b00894e0 0x7fa3b0166c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b87ec700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b02065b0 con 0x7fa3b00894e0 2023-08-27T11:17:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b87ec700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b0205c10 con 0x7fa3b006e260 2023-08-27T11:17:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b87ec700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b0205ef0 con 0x7fa3b01678a0 2023-08-27T11:17:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b6588700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b006e260 0x7fa3b02069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b5d87700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3b01678a0 0x7fa3b0088c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b6d89700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3b00894e0 0x7fa3b0166c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b6d89700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3b00894e0 0x7fa3b0166c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55120/0 (socket says 172.21.15.112:55120) 2023-08-27T11:17:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b6d89700 1 -- 172.21.15.112:0/2732803386 learned_addr learned my addr 172.21.15.112:0/2732803386 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b6d89700 1 -- 172.21.15.112:0/2732803386 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3b01678a0 msgr2=0x7fa3b0088c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b6d89700 1 --2- 172.21.15.112:0/2732803386 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3b01678a0 0x7fa3b0088c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b6d89700 1 -- 172.21.15.112:0/2732803386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b006e260 msgr2=0x7fa3b02069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b6d89700 1 --2- 172.21.15.112:0/2732803386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b006e260 0x7fa3b02069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fa3b6d89700 1 -- 172.21.15.112:0/2732803386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac0094a0 con 0x7fa3b00894e0 2023-08-27T11:17:13.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7fa3b6d89700 1 --2- 172.21.15.112:0/2732803386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3b00894e0 0x7fa3b0166c40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800cb40 tx=0x7fa3a800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7f74aed34700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a80cddb0 0x7f74a81f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7f74aed34700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a80ce190 0x7f74a8155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7f74aed34700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a815bf90 0x7f74a80ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7f74aed34700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80c8400 con 0x7f74a80cddb0 2023-08-27T11:17:13.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7f74aed34700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a8155560 con 0x7f74a815bf90 2023-08-27T11:17:13.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7f74aed34700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a81f32a0 con 0x7f74a80ce190 2023-08-27T11:17:13.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7f74ad2d1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a815bf90 0x7f74a80ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7f74ad2d1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a815bf90 0x7f74a80ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55550/0 (socket says 172.21.15.112:55550) 2023-08-27T11:17:13.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7f74ad2d1700 1 -- 172.21.15.112:0/3907881711 learned_addr learned my addr 172.21.15.112:0/3907881711 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.325+0000 7fadd6343700 1 -- 172.21.15.112:0/2678106589 wait complete. 2023-08-27T11:17:13.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7f74acad0700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a80cddb0 0x7f74a81f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7f74a7fff700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a80ce190 0x7f74a8155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7fadd6343700 1 Processor -- start 2023-08-27T11:17:13.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7f74ad2d1700 1 -- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a80ce190 msgr2=0x7f74a8155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.326+0000 7f74ad2d1700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a80ce190 0x7f74a8155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7f74ad2d1700 1 -- 172.21.15.112:0/3907881711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a80cddb0 msgr2=0x7f74a81f2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7f74ad2d1700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a80cddb0 0x7f74a81f2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7f74ad2d1700 1 -- 172.21.15.112:0/3907881711 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c0094a0 con 0x7f74a815bf90 2023-08-27T11:17:13.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7fa3a6ffd700 1 -- 172.21.15.112:0/2732803386 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a800aaa0 con 0x7fa3b00894e0 2023-08-27T11:17:13.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7fa3a6ffd700 1 -- 172.21.15.112:0/2732803386 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a800ac00 con 0x7fa3b00894e0 2023-08-27T11:17:13.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7fa3a6ffd700 1 -- 172.21.15.112:0/2732803386 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8012400 con 0x7fa3b00894e0 2023-08-27T11:17:13.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7f74ad2d1700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a815bf90 0x7f74a80ca450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f74a000efc0 tx=0x7f74a000c0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7fa3b87ec700 1 -- 172.21.15.112:0/2732803386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b02061d0 con 0x7fa3b00894e0 2023-08-27T11:17:13.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7fa3b87ec700 1 -- 172.21.15.112:0/2732803386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b0206f10 con 0x7fa3b00894e0 2023-08-27T11:17:13.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7f74a57fa700 1 -- 172.21.15.112:0/3907881711 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0003c20 con 0x7f74a815bf90 2023-08-27T11:17:13.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7f74a57fa700 1 -- 172.21.15.112:0/3907881711 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a000f040 con 0x7f74a815bf90 2023-08-27T11:17:13.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7f74aed34700 1 -- 172.21.15.112:0/3907881711 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a81f3580 con 0x7f74a815bf90 2023-08-27T11:17:13.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7f74aed34700 1 -- 172.21.15.112:0/3907881711 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a80ca990 con 0x7f74a815bf90 2023-08-27T11:17:13.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.328+0000 7f74a57fa700 1 -- 172.21.15.112:0/3907881711 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a000c710 con 0x7f74a815bf90 2023-08-27T11:17:13.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.327+0000 7fadd6343700 1 -- start start 2023-08-27T11:17:13.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.328+0000 7fadd6343700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadd006c450 0x7fadd0202d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.328+0000 7fadd6343700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadd015f050 0x7fadd01f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.328+0000 7fadd6343700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd015bc80 0x7fadd0200080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.328+0000 7fadd6343700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd0203660 con 0x7fadd006c450 2023-08-27T11:17:13.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.328+0000 7fadd6343700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd0203940 con 0x7fadd015bc80 2023-08-27T11:17:13.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.328+0000 7fadd6343700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd0203c20 con 0x7fadd015f050 2023-08-27T11:17:13.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.328+0000 7fadcf7fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadd015f050 0x7fadd01f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.328+0000 7fadcf7fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadd015f050 0x7fadd01f95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38552/0 (socket says 172.21.15.112:38552) 2023-08-27T11:17:13.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.328+0000 7fadcf7fe700 1 -- 172.21.15.112:0/1117452780 learned_addr learned my addr 172.21.15.112:0/1117452780 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.328+0000 7fadd48e0700 1 --2- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd015bc80 0x7fadd0200080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.329+0000 7f46ece8d700 1 -- 172.21.15.112:0/756516221 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e000e070 con 0x7f46e815f050 2023-08-27T11:17:13.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.329+0000 7f46ece8d700 1 -- 172.21.15.112:0/756516221 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e000bb20 con 0x7f46e815f050 2023-08-27T11:17:13.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.329+0000 7f46ece8d700 1 -- 172.21.15.112:0/756516221 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0012430 con 0x7f46e815f050 2023-08-27T11:17:13.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.329+0000 7fadd48e0700 1 -- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadd015f050 msgr2=0x7fadd01f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.329+0000 7fadd48e0700 1 --2- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadd015f050 0x7fadd01f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.329+0000 7fadd48e0700 1 -- 172.21.15.112:0/1117452780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadd006c450 msgr2=0x7fadd0202d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.329+0000 7fadd48e0700 1 --2- 172.21.15.112:0/1117452780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadd006c450 0x7fadd0202d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.329+0000 7fadd48e0700 1 -- 172.21.15.112:0/1117452780 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc8009a70 con 0x7fadd015bc80 2023-08-27T11:17:13.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.329+0000 7fadd48e0700 1 --2- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd015bc80 0x7fadd0200080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fadc800ef90 tx=0x7fadc800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.330+0000 7f46f00f3700 1 -- 172.21.15.112:0/756516221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46e815f050 msgr2=0x7f46e8161c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.330+0000 7f46f00f3700 1 --2- 172.21.15.112:0/756516221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46e815f050 0x7f46e8161c90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46e80b2f40 tx=0x7f46e000b330 comp rx=0 tx=0).stop 2023-08-27T11:17:13.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.330+0000 7f74aed34700 1 -- 172.21.15.112:0/3907881711 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74880052f0 con 0x7f74a815bf90 2023-08-27T11:17:13.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.331+0000 7fa3a6ffd700 1 -- 172.21.15.112:0/2732803386 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa3a801b3b0 con 0x7fa3b00894e0 2023-08-27T11:17:13.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.331+0000 7fa3a6ffd700 1 --2- 172.21.15.112:0/2732803386 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa39406a1b0 0x7fa39406c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.331+0000 7fa3a6ffd700 1 -- 172.21.15.112:0/2732803386 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3a8088100 con 0x7fa3b00894e0 2023-08-27T11:17:13.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.331+0000 7fadccff9700 1 -- 172.21.15.112:0/1117452780 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800cd40 con 0x7fadd015bc80 2023-08-27T11:17:13.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.331+0000 7fadccff9700 1 -- 172.21.15.112:0/1117452780 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc800f040 con 0x7fadd015bc80 2023-08-27T11:17:13.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.331+0000 7fadccff9700 1 -- 172.21.15.112:0/1117452780 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc80146c0 con 0x7fadd015bc80 2023-08-27T11:17:13.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.332+0000 7fadd6343700 1 -- 172.21.15.112:0/1117452780 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc00094a0 con 0x7fadd015bc80 2023-08-27T11:17:13.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.332+0000 7fa3b87ec700 1 -- 172.21.15.112:0/2732803386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3980052f0 con 0x7fa3b00894e0 2023-08-27T11:17:13.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.331+0000 7f74a57fa700 1 -- 172.21.15.112:0/3907881711 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f74a000c870 con 0x7f74a815bf90 2023-08-27T11:17:13.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.332+0000 7f74a57fa700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f749406a130 0x7f749406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.332+0000 7f74a57fa700 1 -- 172.21.15.112:0/3907881711 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f74a0088840 con 0x7f74a815bf90 2023-08-27T11:17:13.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.332+0000 7f46f00f3700 1 -- 172.21.15.112:0/756516221 shutdown_connections 2023-08-27T11:17:13.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.332+0000 7f46f00f3700 1 --2- 172.21.15.112:0/756516221 >> v2:172.21.15.112:3300/0 conn(0x7f46e806c450 0x7f46e80ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.332+0000 7f46f00f3700 1 --2- 172.21.15.112:0/756516221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46e815f050 0x7f46e8161c90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.332+0000 7f46f00f3700 1 --2- 172.21.15.112:0/756516221 >> v2:172.21.15.138:3300/0 conn(0x7f46e815f430 0x7f46e80b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.332+0000 7f46f00f3700 1 -- 172.21.15.112:0/756516221 >> 172.21.15.112:0/756516221 conn(0x7f46e80b0bf0 msgr2=0x7f46e80b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.332+0000 7f74acad0700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f749406a130 0x7f749406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.332+0000 7f46f00f3700 1 -- 172.21.15.112:0/756516221 shutdown_connections 2023-08-27T11:17:13.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.333+0000 7fa3b6588700 1 --2- 172.21.15.112:0/2732803386 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa39406a1b0 0x7fa39406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.332+0000 7fadd6343700 1 -- 172.21.15.112:0/1117452780 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd01ffb60 con 0x7fadd015bc80 2023-08-27T11:17:13.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.332+0000 7f46f00f3700 1 -- 172.21.15.112:0/756516221 wait complete. 2023-08-27T11:17:13.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.334+0000 7f46f00f3700 1 Processor -- start 2023-08-27T11:17:13.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.336+0000 7fadd6343700 1 -- 172.21.15.112:0/1117452780 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadac0052f0 con 0x7fadd015bc80 2023-08-27T11:17:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.335+0000 7f46f00f3700 1 -- start start 2023-08-27T11:17:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.336+0000 7f46f00f3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46e806c450 0x7f46e81f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.336+0000 7f46f00f3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46e815f050 0x7f46e8140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.336+0000 7f46f00f3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e813fcb0 0x7f46e8142b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.336+0000 7f46f00f3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e8140530 con 0x7f46e813fcb0 2023-08-27T11:17:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.336+0000 7f46f00f3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e8142690 con 0x7f46e806c450 2023-08-27T11:17:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.336+0000 7f46f00f3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e81f7a50 con 0x7f46e815f050 2023-08-27T11:17:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.336+0000 7f46ed68e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46e815f050 0x7f46e8140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.336+0000 7f46ede8f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46e806c450 0x7f46e81f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.336+0000 7f46ed68e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46e815f050 0x7f46e8140910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38562/0 (socket says 172.21.15.112:38562) 2023-08-27T11:17:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.336+0000 7f46ed68e700 1 -- 172.21.15.112:0/196719758 learned_addr learned my addr 172.21.15.112:0/196719758 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.337+0000 7f74a57fa700 1 -- 172.21.15.112:0/3907881711 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f74a0053e90 con 0x7f74a815bf90 2023-08-27T11:17:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.338+0000 7fa3a6ffd700 1 -- 172.21.15.112:0/2732803386 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa3a8053910 con 0x7fa3b00894e0 2023-08-27T11:17:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.338+0000 7fadccff9700 1 -- 172.21.15.112:0/1117452780 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fadc8014820 con 0x7fadd015bc80 2023-08-27T11:17:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.338+0000 7fadccff9700 1 --2- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fadb806a310 0x7fadb806c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.338+0000 7fadccff9700 1 -- 172.21.15.112:0/1117452780 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fadc8089760 con 0x7fadd015bc80 2023-08-27T11:17:13.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.338+0000 7fadcffff700 1 --2- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fadb806a310 0x7fadb806c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.337+0000 7f46ee690700 1 --2- 172.21.15.112:0/196719758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e813fcb0 0x7f46e8142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.337+0000 7f46ee690700 1 -- 172.21.15.112:0/196719758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46e815f050 msgr2=0x7f46e8140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.337+0000 7f46ee690700 1 --2- 172.21.15.112:0/196719758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46e815f050 0x7f46e8140910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.337+0000 7f46ee690700 1 -- 172.21.15.112:0/196719758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46e806c450 msgr2=0x7f46e81f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.337+0000 7f46ee690700 1 --2- 172.21.15.112:0/196719758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46e806c450 0x7f46e81f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.337+0000 7f46ee690700 1 -- 172.21.15.112:0/196719758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4009550 con 0x7f46e813fcb0 2023-08-27T11:17:13.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.338+0000 7f46ee690700 1 --2- 172.21.15.112:0/196719758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e813fcb0 0x7f46e8142b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46d800d5a0 tx=0x7f46d800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.338+0000 7f46de7fc700 1 -- 172.21.15.112:0/196719758 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d80125f0 con 0x7f46e813fcb0 2023-08-27T11:17:13.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.338+0000 7f46de7fc700 1 -- 172.21.15.112:0/196719758 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d8012750 con 0x7f46e813fcb0 2023-08-27T11:17:13.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.338+0000 7f46de7fc700 1 -- 172.21.15.112:0/196719758 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8012ac0 con 0x7f46e813fcb0 2023-08-27T11:17:13.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.338+0000 7f46f00f3700 1 -- 172.21.15.112:0/196719758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e0009a70 con 0x7f46e813fcb0 2023-08-27T11:17:13.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.338+0000 7f46f00f3700 1 -- 172.21.15.112:0/196719758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e8142040 con 0x7f46e813fcb0 2023-08-27T11:17:13.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.341+0000 7f46de7fc700 1 -- 172.21.15.112:0/196719758 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f46d8012c20 con 0x7f46e813fcb0 2023-08-27T11:17:13.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.341+0000 7f46de7fc700 1 --2- 172.21.15.112:0/196719758 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f46cc06a130 0x7f46cc06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.341+0000 7f46de7fc700 1 -- 172.21.15.112:0/196719758 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f46d8088530 con 0x7f46e813fcb0 2023-08-27T11:17:13.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.341+0000 7f46ede8f700 1 --2- 172.21.15.112:0/196719758 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f46cc06a130 0x7f46cc06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.349+0000 7f46f00f3700 1 -- 172.21.15.112:0/196719758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d00052f0 con 0x7f46e813fcb0 2023-08-27T11:17:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.345+0000 7f1412ace700 1 Processor -- start 2023-08-27T11:17:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.345+0000 7f1412ace700 1 -- start start 2023-08-27T11:17:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.346+0000 7f1412ace700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f140c161860 0x7f140c06e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.347+0000 7f1412ace700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140c0c3320 con 0x7f140c06fea0 2023-08-27T11:17:13.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.347+0000 7f1412ace700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140c0cdd00 con 0x7f140c0c35b0 2023-08-27T11:17:13.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.347+0000 7f1412ace700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140c0cdfe0 con 0x7f140c161860 2023-08-27T11:17:13.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.347+0000 7f141086a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f140c161860 0x7f140c06e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.347+0000 7f141086a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f140c161860 0x7f140c06e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55574/0 (socket says 172.21.15.112:55574) 2023-08-27T11:17:13.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.347+0000 7f141086a700 1 -- 172.21.15.112:0/1624534575 learned_addr learned my addr 172.21.15.112:0/1624534575 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.347+0000 7f141086a700 1 -- 172.21.15.112:0/1624534575 >> v1:172.21.15.138:6789/0 conn(0x7f140c0c35b0 legacy=0x7f140c06f790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:17:13.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.347+0000 7f141086a700 1 -- 172.21.15.112:0/1624534575 >> v1:172.21.15.112:6789/0 conn(0x7f140c06fea0 legacy=0x7f140c165660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:13.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.347+0000 7f141086a700 1 -- 172.21.15.112:0/1624534575 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c0cf9a0 con 0x7f140c161860 2023-08-27T11:17:13.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.347+0000 7f141086a700 1 --2- 172.21.15.112:0/1624534575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f140c161860 0x7f140c06e300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f14000098f0 tx=0x7f1400009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3bf34bcad446924 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.347+0000 7f140b7fe700 1 -- 172.21.15.112:0/1624534575 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1400004230 con 0x7f140c161860 2023-08-27T11:17:13.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.348+0000 7f140b7fe700 1 -- 172.21.15.112:0/1624534575 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1400004390 con 0x7f140c161860 2023-08-27T11:17:13.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.348+0000 7f140b7fe700 1 -- 172.21.15.112:0/1624534575 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14000036a0 con 0x7f140c161860 2023-08-27T11:17:13.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.348+0000 7f1412ace700 1 -- 172.21.15.112:0/1624534575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f140c161860 msgr2=0x7f140c06e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.348+0000 7f1412ace700 1 --2- 172.21.15.112:0/1624534575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f140c161860 0x7f140c06e300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f14000098f0 tx=0x7f1400009c00 comp rx=0 tx=0).stop 2023-08-27T11:17:13.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.348+0000 7f1412ace700 1 -- 172.21.15.112:0/1624534575 shutdown_connections 2023-08-27T11:17:13.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.348+0000 7f1412ace700 1 --2- 172.21.15.112:0/1624534575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f140c161860 0x7f140c06e300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.348+0000 7f1412ace700 1 -- 172.21.15.112:0/1624534575 >> 172.21.15.112:0/1624534575 conn(0x7f140c0d0000 msgr2=0x7f140c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.349+0000 7f1412ace700 1 -- 172.21.15.112:0/1624534575 shutdown_connections 2023-08-27T11:17:13.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.349+0000 7f1412ace700 1 -- 172.21.15.112:0/1624534575 wait complete. 2023-08-27T11:17:13.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.339+0000 7fadcffff700 1 --2- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fadb806a310 0x7fadb806c7d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fadd015c320 tx=0x7fadc0010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.349+0000 7fadccff9700 1 -- 172.21.15.112:0/1117452780 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fadc8054e30 con 0x7fadd015bc80 2023-08-27T11:17:13.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.339+0000 7fa3b6588700 1 --2- 172.21.15.112:0/2732803386 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa39406a1b0 0x7fa39406c670 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0007fd0 tx=0x7fa3a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.350+0000 7f46ede8f700 1 --2- 172.21.15.112:0/196719758 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f46cc06a130 0x7f46cc06c5f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46e0014040 tx=0x7f46e0011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.351+0000 7f74acad0700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f749406a130 0x7f749406c5f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f749c000c00 tx=0x7f749c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.351+0000 7f1412ace700 1 Processor -- start 2023-08-27T11:17:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.351+0000 7f1412ace700 1 -- start start 2023-08-27T11:17:13.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.352+0000 7f1412ace700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f140c06fea0 0x7f140c14fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.352+0000 7f1412ace700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f140c161860 0x7f140c0c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.352+0000 7f1412ace700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140c0c5110 0x7f140c0c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.352+0000 7f1412ace700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0c3b30 con 0x7f140c06fea0 2023-08-27T11:17:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.352+0000 7f1412ace700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0c3e10 con 0x7f140c161860 2023-08-27T11:17:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.352+0000 7f1412ace700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0c40f0 con 0x7f140c0c5110 2023-08-27T11:17:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.355+0000 7f141106b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140c0c5110 0x7f140c0c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.355+0000 7f141106b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140c0c5110 0x7f140c0c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38580/0 (socket says 172.21.15.112:38580) 2023-08-27T11:17:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.355+0000 7f141106b700 1 -- 172.21.15.112:0/3635755118 learned_addr learned my addr 172.21.15.112:0/3635755118 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.355+0000 7f141106b700 1 -- 172.21.15.112:0/3635755118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f140c161860 msgr2=0x7f140c0c4bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-27T11:17:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.355+0000 7f141106b700 1 --2- 172.21.15.112:0/3635755118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f140c161860 0x7f140c0c4bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.355+0000 7f141106b700 1 -- 172.21.15.112:0/3635755118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f140c06fea0 msgr2=0x7f140c14fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.355+0000 7f141106b700 1 --2- 172.21.15.112:0/3635755118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f140c06fea0 0x7f140c14fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.355+0000 7f141106b700 1 -- 172.21.15.112:0/3635755118 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1400009550 con 0x7f140c0c5110 2023-08-27T11:17:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.355+0000 7f141106b700 1 --2- 172.21.15.112:0/3635755118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140c0c5110 0x7f140c0c60b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f140400c780 tx=0x7f140400cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.356+0000 7f14097fa700 1 -- 172.21.15.112:0/3635755118 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404003e10 con 0x7f140c0c5110 2023-08-27T11:17:13.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.356+0000 7f14097fa700 1 -- 172.21.15.112:0/3635755118 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1404005070 con 0x7f140c0c5110 2023-08-27T11:17:13.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.356+0000 7f1412ace700 1 -- 172.21.15.112:0/3635755118 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140c0c43d0 con 0x7f140c0c5110 2023-08-27T11:17:13.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.356+0000 7f14097fa700 1 -- 172.21.15.112:0/3635755118 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404005420 con 0x7f140c0c5110 2023-08-27T11:17:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.357+0000 7f1412ace700 1 -- 172.21.15.112:0/3635755118 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140c153bb0 con 0x7f140c0c5110 2023-08-27T11:17:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.358+0000 7f14097fa700 1 -- 172.21.15.112:0/3635755118 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1404006ba0 con 0x7f140c0c5110 2023-08-27T11:17:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.358+0000 7f14097fa700 1 --2- 172.21.15.112:0/3635755118 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13f006a0e0 0x7f13f006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.358+0000 7f14097fa700 1 -- 172.21.15.112:0/3635755118 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1404087ab0 con 0x7f140c0c5110 2023-08-27T11:17:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.358+0000 7f141086a700 1 --2- 172.21.15.112:0/3635755118 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13f006a0e0 0x7f13f006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.359+0000 7f141086a700 1 --2- 172.21.15.112:0/3635755118 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13f006a0e0 0x7f13f006c5a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1400000c00 tx=0x7f1400010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.362+0000 7f46de7fc700 1 -- 172.21.15.112:0/196719758 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f46d8053d00 con 0x7f46e813fcb0 2023-08-27T11:17:13.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.363+0000 7f1412ace700 1 -- 172.21.15.112:0/3635755118 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140c0c48f0 con 0x7f140c0c5110 2023-08-27T11:17:13.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.371+0000 7f14097fa700 1 -- 172.21.15.112:0/3635755118 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f140c0c48f0 con 0x7f140c0c5110 2023-08-27T11:17:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.651+0000 7ff79304b700 1 -- 172.21.15.112:0/1153879083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff76c005870 con 0x7ff78c15d600 2023-08-27T11:17:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.652+0000 7ff7817fa700 1 -- 172.21.15.112:0/1153879083 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff788058ff0 con 0x7ff78c15d600 2023-08-27T11:17:13.656 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2023-08-27T11:17:13.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.655+0000 7ff79304b700 1 -- 172.21.15.112:0/1153879083 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff77406a590 msgr2=0x7ff77406ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.655+0000 7ff79304b700 1 --2- 172.21.15.112:0/1153879083 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff77406a590 0x7ff77406ca50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0010e0 tx=0x7ff77c0079e0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.655+0000 7ff79304b700 1 -- 172.21.15.112:0/1153879083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff78c15d600 msgr2=0x7ff78c155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.655+0000 7ff79304b700 1 --2- 172.21.15.112:0/1153879083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff78c15d600 0x7ff78c155dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff788008c40 tx=0x7ff788004750 comp rx=0 tx=0).stop 2023-08-27T11:17:13.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.656+0000 7ff79304b700 1 -- 172.21.15.112:0/1153879083 shutdown_connections 2023-08-27T11:17:13.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.656+0000 7ff79304b700 1 --2- 172.21.15.112:0/1153879083 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff77406a590 0x7ff77406ca50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.656+0000 7ff79304b700 1 --2- 172.21.15.112:0/1153879083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c15da90 0x7ff78c156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.656+0000 7ff79304b700 1 --2- 172.21.15.112:0/1153879083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78c13c160 0x7ff78c1f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.656+0000 7ff79304b700 1 --2- 172.21.15.112:0/1153879083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff78c15d600 0x7ff78c155dc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.656+0000 7ff79304b700 1 -- 172.21.15.112:0/1153879083 >> 172.21.15.112:0/1153879083 conn(0x7ff78c0b0a10 msgr2=0x7ff78c13b760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.656+0000 7ff79304b700 1 -- 172.21.15.112:0/1153879083 shutdown_connections 2023-08-27T11:17:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.656+0000 7ff79304b700 1 -- 172.21.15.112:0/1153879083 wait complete. 2023-08-27T11:17:13.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.672+0000 7fa76c361700 1 -- 172.21.15.112:0/2336075047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa76413f2f0 con 0x7fa7640b1500 2023-08-27T11:17:13.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-08-27T11:17:13.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.674+0000 7fa75affd700 1 -- 172.21.15.112:0/2336075047 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa754029020 con 0x7fa7640b1500 2023-08-27T11:17:13.676 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2023-08-27T11:17:13.676 DEBUG:teuthology.parallel:result is None 2023-08-27T11:17:13.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.678+0000 7fa76c361700 1 -- 172.21.15.112:0/2336075047 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa74806a0e0 msgr2=0x7fa74806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.678+0000 7fa76c361700 1 --2- 172.21.15.112:0/2336075047 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa74806a0e0 0x7fa74806c5a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa760006fd0 tx=0x7fa760006bc0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.678+0000 7fa76c361700 1 -- 172.21.15.112:0/2336075047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7640b1500 msgr2=0x7fa764152290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.678+0000 7fa76c361700 1 --2- 172.21.15.112:0/2336075047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7640b1500 0x7fa764152290 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa754009c40 tx=0x7fa754004750 comp rx=0 tx=0).stop 2023-08-27T11:17:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.678+0000 7fa76c361700 1 -- 172.21.15.112:0/2336075047 shutdown_connections 2023-08-27T11:17:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.678+0000 7fa76c361700 1 --2- 172.21.15.112:0/2336075047 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa74806a0e0 0x7fa74806c5a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.678+0000 7fa76c361700 1 --2- 172.21.15.112:0/2336075047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa76415d600 0x7fa76415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.678+0000 7fa76c361700 1 --2- 172.21.15.112:0/2336075047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7640b1500 0x7fa764152290 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.678+0000 7fa76c361700 1 --2- 172.21.15.112:0/2336075047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa76413c160 0x7fa76415bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.678+0000 7fa76c361700 1 -- 172.21.15.112:0/2336075047 >> 172.21.15.112:0/2336075047 conn(0x7fa7640b0a10 msgr2=0x7fa7641361c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.678+0000 7fa76c361700 1 -- 172.21.15.112:0/2336075047 shutdown_connections 2023-08-27T11:17:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.678+0000 7fa76c361700 1 -- 172.21.15.112:0/2336075047 wait complete. 2023-08-27T11:17:13.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-08-27T11:17:13.697 DEBUG:teuthology.parallel:result is None 2023-08-27T11:17:13.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.713+0000 7faea9444700 1 -- 172.21.15.112:0/2094138918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fae840055a0 con 0x7faea415c030 2023-08-27T11:17:13.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.713+0000 7fae8b7fe700 1 -- 172.21.15.112:0/2094138918 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae9c059480 con 0x7faea415c030 2023-08-27T11:17:13.715 INFO:teuthology.orchestra.run.smithi112.stdout:55834574854 2023-08-27T11:17:13.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.714+0000 7fa3b87ec700 1 -- 172.21.15.112:0/2732803386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa3980055a0 con 0x7fa3b00894e0 2023-08-27T11:17:13.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.714+0000 7fa3a6ffd700 1 -- 172.21.15.112:0/2732803386 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa3a8017090 con 0x7fa3b00894e0 2023-08-27T11:17:13.716 INFO:teuthology.orchestra.run.smithi112.stdout:55834574854 2023-08-27T11:17:13.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7faea9444700 1 -- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fae9006a1b0 msgr2=0x7fae9006c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7faea9444700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fae9006a1b0 0x7fae9006c670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faea40d0770 tx=0x7fae98009040 comp rx=0 tx=0).stop 2023-08-27T11:17:13.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7faea9444700 1 -- 172.21.15.112:0/2094138918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea415c030 msgr2=0x7faea41574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7faea9444700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea415c030 0x7faea41574c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00d010 tx=0x7fae9c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7fa3b87ec700 1 -- 172.21.15.112:0/2732803386 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa39406a1b0 msgr2=0x7fa39406c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7fa3b87ec700 1 --2- 172.21.15.112:0/2732803386 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa39406a1b0 0x7fa39406c670 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0007fd0 tx=0x7fa3a0009040 comp rx=0 tx=0).stop 2023-08-27T11:17:13.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7fa3b87ec700 1 -- 172.21.15.112:0/2732803386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3b00894e0 msgr2=0x7fa3b0166c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7fa3b87ec700 1 --2- 172.21.15.112:0/2732803386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3b00894e0 0x7fa3b0166c40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800cb40 tx=0x7fa3a800ce50 comp rx=0 tx=0).stop 2023-08-27T11:17:13.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7faea9444700 1 -- 172.21.15.112:0/2094138918 shutdown_connections 2023-08-27T11:17:13.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7faea9444700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fae9006a1b0 0x7fae9006c670 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7faea9444700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea4166270 0x7faea415ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7faea9444700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea415c030 0x7faea41574c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7faea9444700 1 --2- 172.21.15.112:0/2094138918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faea4162780 0x7faea415ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.717+0000 7faea9444700 1 -- 172.21.15.112:0/2094138918 >> 172.21.15.112:0/2094138918 conn(0x7faea40cfe50 msgr2=0x7faea415b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.718+0000 7faea9444700 1 -- 172.21.15.112:0/2094138918 shutdown_connections 2023-08-27T11:17:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.718+0000 7fa3b87ec700 1 -- 172.21.15.112:0/2732803386 shutdown_connections 2023-08-27T11:17:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.718+0000 7fa3b87ec700 1 --2- 172.21.15.112:0/2732803386 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa39406a1b0 0x7fa39406c670 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.718+0000 7fa3b87ec700 1 --2- 172.21.15.112:0/2732803386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3b00894e0 0x7fa3b0166c40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.718+0000 7fa3b87ec700 1 --2- 172.21.15.112:0/2732803386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b006e260 0x7fa3b02069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.718+0000 7fa3b87ec700 1 --2- 172.21.15.112:0/2732803386 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3b01678a0 0x7fa3b0088c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.718+0000 7fa3b87ec700 1 -- 172.21.15.112:0/2732803386 >> 172.21.15.112:0/2732803386 conn(0x7fa3b00b0a10 msgr2=0x7fa3b00b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.718+0000 7faea9444700 1 -- 172.21.15.112:0/2094138918 wait complete. 2023-08-27T11:17:13.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.723+0000 7f46f00f3700 1 -- 172.21.15.112:0/196719758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f46d0005870 con 0x7f46e813fcb0 2023-08-27T11:17:13.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.723+0000 7f46de7fc700 1 -- 172.21.15.112:0/196719758 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f46d8006da0 con 0x7f46e813fcb0 2023-08-27T11:17:13.725 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2023-08-27T11:17:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.725+0000 7f46f00f3700 1 -- 172.21.15.112:0/196719758 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f46cc06a130 msgr2=0x7f46cc06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.725+0000 7f46f00f3700 1 --2- 172.21.15.112:0/196719758 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f46cc06a130 0x7f46cc06c5f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46e0014040 tx=0x7f46e0011000 comp rx=0 tx=0).stop 2023-08-27T11:17:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.725+0000 7f46f00f3700 1 -- 172.21.15.112:0/196719758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e813fcb0 msgr2=0x7f46e8142b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.725+0000 7f46f00f3700 1 --2- 172.21.15.112:0/196719758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e813fcb0 0x7f46e8142b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46d800d5a0 tx=0x7f46d800d960 comp rx=0 tx=0).stop 2023-08-27T11:17:13.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.725+0000 7f46f00f3700 1 -- 172.21.15.112:0/196719758 shutdown_connections 2023-08-27T11:17:13.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.725+0000 7f46f00f3700 1 --2- 172.21.15.112:0/196719758 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f46cc06a130 0x7f46cc06c5f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.725+0000 7f46f00f3700 1 --2- 172.21.15.112:0/196719758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e813fcb0 0x7f46e8142b20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.725+0000 7f46f00f3700 1 --2- 172.21.15.112:0/196719758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46e806c450 0x7f46e81f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.725+0000 7f46f00f3700 1 --2- 172.21.15.112:0/196719758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46e815f050 0x7f46e8140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.725+0000 7f46f00f3700 1 -- 172.21.15.112:0/196719758 >> 172.21.15.112:0/196719758 conn(0x7f46e80b0bf0 msgr2=0x7f46e8161ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.726+0000 7f46f00f3700 1 -- 172.21.15.112:0/196719758 shutdown_connections 2023-08-27T11:17:13.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.729+0000 7fa3b87ec700 1 -- 172.21.15.112:0/2732803386 shutdown_connections 2023-08-27T11:17:13.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.729+0000 7fa3b87ec700 1 -- 172.21.15.112:0/2732803386 wait complete. 2023-08-27T11:17:13.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.732+0000 7f46f00f3700 1 -- 172.21.15.112:0/196719758 wait complete. 2023-08-27T11:17:13.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-08-27T11:17:13.738 DEBUG:teuthology.parallel:result is None 2023-08-27T11:17:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.739+0000 7f74aed34700 1 -- 172.21.15.112:0/3907881711 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f74880055a0 con 0x7f74a815bf90 2023-08-27T11:17:13.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.740+0000 7f74a57fa700 1 -- 172.21.15.112:0/3907881711 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f74a00574b0 con 0x7f74a815bf90 2023-08-27T11:17:13.741 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2023-08-27T11:17:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.740+0000 7fadd6343700 1 -- 172.21.15.112:0/1117452780 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fadac005870 con 0x7fadd015bc80 2023-08-27T11:17:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.740+0000 7fadccff9700 1 -- 172.21.15.112:0/1117452780 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fadc8058450 con 0x7fadd015bc80 2023-08-27T11:17:13.742 INFO:teuthology.orchestra.run.smithi112.stdout:55834574854 2023-08-27T11:17:13.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-08-27T11:17:13.745 DEBUG:teuthology.parallel:result is None 2023-08-27T11:17:13.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7f74aed34700 1 -- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f749406a130 msgr2=0x7f749406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7f74aed34700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f749406a130 0x7f749406c5f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f749c000c00 tx=0x7f749c008030 comp rx=0 tx=0).stop 2023-08-27T11:17:13.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7f74aed34700 1 -- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a815bf90 msgr2=0x7f74a80ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7f74aed34700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a815bf90 0x7f74a80ca450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f74a000efc0 tx=0x7f74a000c0c0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7fadd6343700 1 -- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fadb806a310 msgr2=0x7fadb806c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7fadd6343700 1 --2- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fadb806a310 0x7fadb806c7d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fadd015c320 tx=0x7fadc0010000 comp rx=0 tx=0).stop 2023-08-27T11:17:13.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7fadd6343700 1 -- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd015bc80 msgr2=0x7fadd0200080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7fadd6343700 1 --2- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd015bc80 0x7fadd0200080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fadc800ef90 tx=0x7fadc800c3a0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7f74aed34700 1 -- 172.21.15.112:0/3907881711 shutdown_connections 2023-08-27T11:17:13.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7f74aed34700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f749406a130 0x7f749406c5f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7f74aed34700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a815bf90 0x7f74a80ca450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7f74aed34700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a80cddb0 0x7f74a81f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7f74aed34700 1 --2- 172.21.15.112:0/3907881711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a80ce190 0x7f74a8155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7f74aed34700 1 -- 172.21.15.112:0/3907881711 >> 172.21.15.112:0/3907881711 conn(0x7f74a806f970 msgr2=0x7f74a80b5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7f74aed34700 1 -- 172.21.15.112:0/3907881711 shutdown_connections 2023-08-27T11:17:13.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.744+0000 7f74aed34700 1 -- 172.21.15.112:0/3907881711 wait complete. 2023-08-27T11:17:13.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.746+0000 7fadd6343700 1 -- 172.21.15.112:0/1117452780 shutdown_connections 2023-08-27T11:17:13.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.746+0000 7fadd6343700 1 --2- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fadb806a310 0x7fadb806c7d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.746+0000 7fadd6343700 1 --2- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd015bc80 0x7fadd0200080 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.746+0000 7fadd6343700 1 --2- 172.21.15.112:0/1117452780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadd006c450 0x7fadd0202d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.746+0000 7fadd6343700 1 --2- 172.21.15.112:0/1117452780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadd015f050 0x7fadd01f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.746+0000 7fadd6343700 1 -- 172.21.15.112:0/1117452780 >> 172.21.15.112:0/1117452780 conn(0x7fadd00b0bf0 msgr2=0x7fadd015e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.747+0000 7fadd6343700 1 -- 172.21.15.112:0/1117452780 shutdown_connections 2023-08-27T11:17:13.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.747+0000 7fadd6343700 1 -- 172.21.15.112:0/1117452780 wait complete. 2023-08-27T11:17:13.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-08-27T11:17:13.753 DEBUG:teuthology.parallel:result is None 2023-08-27T11:17:13.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-08-27T11:17:13.760 DEBUG:teuthology.parallel:result is None 2023-08-27T11:17:13.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-08-27T11:17:13.762 DEBUG:teuthology.parallel:result is None 2023-08-27T11:17:13.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.766+0000 7f1412ace700 1 -- 172.21.15.112:0/3635755118 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f140c0c48f0 con 0x7f140c0c5110 2023-08-27T11:17:13.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.767+0000 7f14097fa700 1 -- 172.21.15.112:0/3635755118 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f140c0c48f0 con 0x7f140c0c5110 2023-08-27T11:17:13.768 INFO:teuthology.orchestra.run.smithi112.stdout:55834574854 2023-08-27T11:17:13.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.769+0000 7f1412ace700 1 -- 172.21.15.112:0/3635755118 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13f006a0e0 msgr2=0x7f13f006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.769+0000 7f1412ace700 1 --2- 172.21.15.112:0/3635755118 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13f006a0e0 0x7f13f006c5a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1400000c00 tx=0x7f1400010000 comp rx=0 tx=0).stop 2023-08-27T11:17:13.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.769+0000 7f1412ace700 1 -- 172.21.15.112:0/3635755118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140c0c5110 msgr2=0x7f140c0c60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.769+0000 7f1412ace700 1 --2- 172.21.15.112:0/3635755118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140c0c5110 0x7f140c0c60b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f140400c780 tx=0x7f140400cb40 comp rx=0 tx=0).stop 2023-08-27T11:17:13.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.769+0000 7f1412ace700 1 -- 172.21.15.112:0/3635755118 shutdown_connections 2023-08-27T11:17:13.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.769+0000 7f1412ace700 1 --2- 172.21.15.112:0/3635755118 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13f006a0e0 0x7f13f006c5a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.769+0000 7f1412ace700 1 --2- 172.21.15.112:0/3635755118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f140c0c5110 0x7f140c0c60b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.769+0000 7f1412ace700 1 --2- 172.21.15.112:0/3635755118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f140c06fea0 0x7f140c14fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.769+0000 7f1412ace700 1 --2- 172.21.15.112:0/3635755118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f140c161860 0x7f140c0c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.769+0000 7f1412ace700 1 -- 172.21.15.112:0/3635755118 >> 172.21.15.112:0/3635755118 conn(0x7f140c0d0000 msgr2=0x7f140c0cb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.769+0000 7f1412ace700 1 -- 172.21.15.112:0/3635755118 shutdown_connections 2023-08-27T11:17:13.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.769+0000 7f1412ace700 1 -- 172.21.15.112:0/3635755118 wait complete. 2023-08-27T11:17:13.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-08-27T11:17:13.783 DEBUG:teuthology.parallel:result is None 2023-08-27T11:17:13.783 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-08-27T11:17:13.783 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:17:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.941+0000 7fe787018700 1 Processor -- start 2023-08-27T11:17:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.942+0000 7fe787018700 1 -- start start 2023-08-27T11:17:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.942+0000 7fe787018700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe78006fea0 0x7fe7800b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.942+0000 7fe787018700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe78006e3a0 0x7fe7800af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.942+0000 7fe787018700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe78015c190 con 0x7fe780158e20 2023-08-27T11:17:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.942+0000 7fe787018700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7800b1730 con 0x7fe78006e3a0 2023-08-27T11:17:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.942+0000 7fe787018700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe780159200 con 0x7fe78006fea0 2023-08-27T11:17:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.943+0000 7fe77ffff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe78006fea0 0x7fe7800b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.943+0000 7fe77ffff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe78006fea0 0x7fe7800b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38584/0 (socket says 172.21.15.112:38584) 2023-08-27T11:17:13.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.943+0000 7fe7855b5700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe78006e3a0 0x7fe7800af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.943+0000 7fe77ffff700 1 -- 172.21.15.112:0/1298266416 learned_addr learned my addr 172.21.15.112:0/1298266416 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.943+0000 7fe7855b5700 1 -- 172.21.15.112:0/1298266416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe78006fea0 msgr2=0x7fe7800b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:13.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.943+0000 7fe7855b5700 1 --2- 172.21.15.112:0/1298266416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe78006fea0 0x7fe7800b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.943+0000 7fe7855b5700 1 -- 172.21.15.112:0/1298266416 >> v1:172.21.15.138:6789/0 conn(0x7fe780158e20 legacy=0x7fe78006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.943+0000 7fe77f7fe700 1 -- 172.21.15.112:0/1298266416 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693882700 0 0) 0x7fe78015c190 con 0x7fe780158e20 2023-08-27T11:17:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.943+0000 7fe7855b5700 1 -- 172.21.15.112:0/1298266416 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7800b1910 con 0x7fe78006e3a0 2023-08-27T11:17:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.943+0000 7fe7855b5700 1 --2- 172.21.15.112:0/1298266416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78006e3a0 0x7fe7800af9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe778009d90 tx=0x7fe77800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4bd4a7efbda427a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.943+0000 7fe77f7fe700 1 -- 172.21.15.112:0/1298266416 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77800e070 con 0x7fe78006e3a0 2023-08-27T11:17:13.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.943+0000 7fe77f7fe700 1 -- 172.21.15.112:0/1298266416 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77800bb20 con 0x7fe78006e3a0 2023-08-27T11:17:13.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.943+0000 7fe77f7fe700 1 -- 172.21.15.112:0/1298266416 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778012430 con 0x7fe78006e3a0 2023-08-27T11:17:13.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.944+0000 7fe787018700 1 -- 172.21.15.112:0/1298266416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78006e3a0 msgr2=0x7fe7800af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.944+0000 7fe787018700 1 --2- 172.21.15.112:0/1298266416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78006e3a0 0x7fe7800af9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe778009d90 tx=0x7fe77800b330 comp rx=0 tx=0).stop 2023-08-27T11:17:13.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.944+0000 7fe787018700 1 -- 172.21.15.112:0/1298266416 shutdown_connections 2023-08-27T11:17:13.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.944+0000 7fe787018700 1 --2- 172.21.15.112:0/1298266416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78006e3a0 0x7fe7800af9c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.944+0000 7fe787018700 1 --2- 172.21.15.112:0/1298266416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe78006fea0 0x7fe7800b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.944+0000 7fe787018700 1 -- 172.21.15.112:0/1298266416 >> 172.21.15.112:0/1298266416 conn(0x7fe7800d0000 msgr2=0x7fe7800d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:13.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.944+0000 7fe787018700 1 -- 172.21.15.112:0/1298266416 shutdown_connections 2023-08-27T11:17:13.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.944+0000 7fe787018700 1 -- 172.21.15.112:0/1298266416 wait complete. 2023-08-27T11:17:13.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.945+0000 7fe787018700 1 Processor -- start 2023-08-27T11:17:13.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.945+0000 7fe787018700 1 -- start start 2023-08-27T11:17:13.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe787018700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78006e3a0 0x7fe7801519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe787018700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe78006fea0 0x7fe78015c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe787018700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe780158e20 0x7fe7800c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe787018700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800c7ce0 con 0x7fe780158e20 2023-08-27T11:17:13.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe787018700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78015d2f0 con 0x7fe78006e3a0 2023-08-27T11:17:13.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe787018700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800c7f30 con 0x7fe78006fea0 2023-08-27T11:17:13.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe7855b5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe780158e20 0x7fe7800c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe7855b5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe780158e20 0x7fe7800c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55172/0 (socket says 172.21.15.112:55172) 2023-08-27T11:17:13.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe7855b5700 1 -- 172.21.15.112:0/2100661063 learned_addr learned my addr 172.21.15.112:0/2100661063 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:13.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe784db4700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78006e3a0 0x7fe7801519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe77ffff700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe78006fea0 0x7fe78015c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe7855b5700 1 -- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe78006fea0 msgr2=0x7fe78015c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe7855b5700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe78006fea0 0x7fe78015c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe7855b5700 1 -- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78006e3a0 msgr2=0x7fe7801519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:13.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe7855b5700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78006e3a0 0x7fe7801519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:13.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.946+0000 7fe7855b5700 1 -- 172.21.15.112:0/2100661063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe778009a70 con 0x7fe780158e20 2023-08-27T11:17:13.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.947+0000 7fe7855b5700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe780158e20 0x7fe7800c8a10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe778014040 tx=0x7fe778004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.947+0000 7fe77d7fa700 1 -- 172.21.15.112:0/2100661063 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778012430 con 0x7fe780158e20 2023-08-27T11:17:13.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.947+0000 7fe77d7fa700 1 -- 172.21.15.112:0/2100661063 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe778020070 con 0x7fe780158e20 2023-08-27T11:17:13.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.947+0000 7fe787018700 1 -- 172.21.15.112:0/2100661063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7800c8210 con 0x7fe780158e20 2023-08-27T11:17:13.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.947+0000 7fe77d7fa700 1 -- 172.21.15.112:0/2100661063 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778004280 con 0x7fe780158e20 2023-08-27T11:17:13.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.947+0000 7fe787018700 1 -- 172.21.15.112:0/2100661063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78015cec0 con 0x7fe780158e20 2023-08-27T11:17:13.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.948+0000 7fe787018700 1 -- 172.21.15.112:0/2100661063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7800c8730 con 0x7fe780158e20 2023-08-27T11:17:13.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.948+0000 7fe77d7fa700 1 -- 172.21.15.112:0/2100661063 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe7780043e0 con 0x7fe780158e20 2023-08-27T11:17:13.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.948+0000 7fe77d7fa700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe76406a0e0 0x7fe76406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:13.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.949+0000 7fe77d7fa700 1 -- 172.21.15.112:0/2100661063 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe778022070 con 0x7fe780158e20 2023-08-27T11:17:13.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.949+0000 7fe784db4700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe76406a0e0 0x7fe76406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:13.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.949+0000 7fe784db4700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe76406a0e0 0x7fe76406c5a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe774000f60 tx=0x7fe774009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:13.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:13.953+0000 7fe77d7fa700 1 -- 172.21.15.112:0/2100661063 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe778055e50 con 0x7fe780158e20 2023-08-27T11:17:14.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.136+0000 7fe787018700 1 -- 172.21.15.112:0/2100661063 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe78015be20 con 0x7fe76406a0e0 2023-08-27T11:17:14.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.145+0000 7fe77d7fa700 1 -- 172.21.15.112:0/2100661063 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286911 (secure 0 0 0) 0x7fe7740050d0 con 0x7fe76406a0e0 2023-08-27T11:17:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.148+0000 7fe787018700 1 -- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe76406a0e0 msgr2=0x7fe76406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.148+0000 7fe787018700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe76406a0e0 0x7fe76406c5a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe774000f60 tx=0x7fe774009040 comp rx=0 tx=0).stop 2023-08-27T11:17:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.148+0000 7fe787018700 1 -- 172.21.15.112:0/2100661063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe780158e20 msgr2=0x7fe7800c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.148+0000 7fe787018700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe780158e20 0x7fe7800c8a10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe778014040 tx=0x7fe778004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.148+0000 7fe787018700 1 -- 172.21.15.112:0/2100661063 shutdown_connections 2023-08-27T11:17:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.148+0000 7fe787018700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe76406a0e0 0x7fe76406c5a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.148+0000 7fe787018700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe780158e20 0x7fe7800c8a10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.148+0000 7fe787018700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78006e3a0 0x7fe7801519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.148+0000 7fe787018700 1 --2- 172.21.15.112:0/2100661063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe78006fea0 0x7fe78015c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.148+0000 7fe787018700 1 -- 172.21.15.112:0/2100661063 >> 172.21.15.112:0/2100661063 conn(0x7fe7800d0000 msgr2=0x7fe780152670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.148+0000 7fe787018700 1 -- 172.21.15.112:0/2100661063 shutdown_connections 2023-08-27T11:17:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.148+0000 7fe787018700 1 -- 172.21.15.112:0/2100661063 wait complete. 2023-08-27T11:17:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:17:14.161 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-08-27T11:17:12.225551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219044,"kb_used_data":2820,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755676,"statfs":{"total":773094113280,"available":772869812224,"internally_reserved":0,"allocated":2887680,"data_stored":1779619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.868296"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190109+0000","last_change":"2023-08-27T11:17:05.373064+0000","last_active":"2023-08-27T11:17:07.190109+0000","last_peered":"2023-08-27T11:17:07.190109+0000","last_clean":"2023-08-27T11:17:07.190109+0000","last_became_active":"2023-08-27T11:17:05.372771+0000","last_became_peered":"2023-08-27T11:17:05.372771+0000","last_unstale":"2023-08-27T11:17:07.190109+0000","last_undegraded":"2023-08-27T11:17:07.190109+0000","last_fullsized":"2023-08-27T11:17:07.190109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:12:06.488460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202664+0000","last_change":"2023-08-27T11:17:05.380926+0000","last_active":"2023-08-27T11:17:07.202664+0000","last_peered":"2023-08-27T11:17:07.202664+0000","last_clean":"2023-08-27T11:17:07.202664+0000","last_became_active":"2023-08-27T11:17:05.380623+0000","last_became_peered":"2023-08-27T11:17:05.380623+0000","last_unstale":"2023-08-27T11:17:07.202664+0000","last_undegraded":"2023-08-27T11:17:07.202664+0000","last_fullsized":"2023-08-27T11:17:07.202664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368869+0000","last_change":"2023-08-27T11:17:05.377335+0000","last_active":"2023-08-27T11:17:06.368869+0000","last_peered":"2023-08-27T11:17:06.368869+0000","last_clean":"2023-08-27T11:17:06.368869+0000","last_became_active":"2023-08-27T11:17:05.376963+0000","last_became_peered":"2023-08-27T11:17:05.376963+0000","last_unstale":"2023-08-27T11:17:06.368869+0000","last_undegraded":"2023-08-27T11:17:06.368869+0000","last_fullsized":"2023-08-27T11:17:06.368869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:54.962968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369836+0000","last_change":"2023-08-27T11:17:05.371708+0000","last_active":"2023-08-27T11:17:06.369836+0000","last_peered":"2023-08-27T11:17:06.369836+0000","last_clean":"2023-08-27T11:17:06.369836+0000","last_became_active":"2023-08-27T11:17:05.371385+0000","last_became_peered":"2023-08-27T11:17:05.371385+0000","last_unstale":"2023-08-27T11:17:06.369836+0000","last_undegraded":"2023-08-27T11:17:06.369836+0000","last_fullsized":"2023-08-27T11:17:06.369836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607644+0000","last_change":"2023-08-27T11:17:05.377806+0000","last_active":"2023-08-27T11:17:06.607644+0000","last_peered":"2023-08-27T11:17:06.607644+0000","last_clean":"2023-08-27T11:17:06.607644+0000","last_became_active":"2023-08-27T11:17:05.377546+0000","last_became_peered":"2023-08-27T11:17:05.377546+0000","last_unstale":"2023-08-27T11:17:06.607644+0000","last_undegraded":"2023-08-27T11:17:06.607644+0000","last_fullsized":"2023-08-27T11:17:06.607644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:28.711930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369347+0000","last_change":"2023-08-27T11:17:05.371766+0000","last_active":"2023-08-27T11:17:06.369347+0000","last_peered":"2023-08-27T11:17:06.369347+0000","last_clean":"2023-08-27T11:17:06.369347+0000","last_became_active":"2023-08-27T11:17:05.370938+0000","last_became_peered":"2023-08-27T11:17:05.370938+0000","last_unstale":"2023-08-27T11:17:06.369347+0000","last_undegraded":"2023-08-27T11:17:06.369347+0000","last_fullsized":"2023-08-27T11:17:06.369347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:55.731983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203009+0000","last_change":"2023-08-27T11:17:05.383079+0000","last_active":"2023-08-27T11:17:07.203009+0000","last_peered":"2023-08-27T11:17:07.203009+0000","last_clean":"2023-08-27T11:17:07.203009+0000","last_became_active":"2023-08-27T11:17:05.382810+0000","last_became_peered":"2023-08-27T11:17:05.382810+0000","last_unstale":"2023-08-27T11:17:07.203009+0000","last_undegraded":"2023-08-27T11:17:07.203009+0000","last_fullsized":"2023-08-27T11:17:07.203009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:46.743604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202512+0000","last_change":"2023-08-27T11:17:05.376722+0000","last_active":"2023-08-27T11:17:07.202512+0000","last_peered":"2023-08-27T11:17:07.202512+0000","last_clean":"2023-08-27T11:17:07.202512+0000","last_became_active":"2023-08-27T11:17:05.376399+0000","last_became_peered":"2023-08-27T11:17:05.376399+0000","last_unstale":"2023-08-27T11:17:07.202512+0000","last_undegraded":"2023-08-27T11:17:07.202512+0000","last_fullsized":"2023-08-27T11:17:07.202512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607858+0000","last_change":"2023-08-27T11:17:05.371306+0000","last_active":"2023-08-27T11:17:06.607858+0000","last_peered":"2023-08-27T11:17:06.607858+0000","last_clean":"2023-08-27T11:17:06.607858+0000","last_became_active":"2023-08-27T11:17:05.370962+0000","last_became_peered":"2023-08-27T11:17:05.370962+0000","last_unstale":"2023-08-27T11:17:06.607858+0000","last_undegraded":"2023-08-27T11:17:06.607858+0000","last_fullsized":"2023-08-27T11:17:06.607858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:39.209301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202986+0000","last_change":"2023-08-27T11:17:05.372994+0000","last_active":"2023-08-27T11:17:07.202986+0000","last_peered":"2023-08-27T11:17:07.202986+0000","last_clean":"2023-08-27T11:17:07.202986+0000","last_became_active":"2023-08-27T11:17:05.372278+0000","last_became_peered":"2023-08-27T11:17:05.372278+0000","last_unstale":"2023-08-27T11:17:07.202986+0000","last_undegraded":"2023-08-27T11:17:07.202986+0000","last_fullsized":"2023-08-27T11:17:07.202986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369607+0000","last_change":"2023-08-27T11:17:05.376929+0000","last_active":"2023-08-27T11:17:06.369607+0000","last_peered":"2023-08-27T11:17:06.369607+0000","last_clean":"2023-08-27T11:17:06.369607+0000","last_became_active":"2023-08-27T11:17:05.376592+0000","last_became_peered":"2023-08-27T11:17:05.376592+0000","last_unstale":"2023-08-27T11:17:06.369607+0000","last_undegraded":"2023-08-27T11:17:06.369607+0000","last_fullsized":"2023-08-27T11:17:06.369607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:24:05.167634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190152+0000","last_change":"2023-08-27T11:17:05.377608+0000","last_active":"2023-08-27T11:17:07.190152+0000","last_peered":"2023-08-27T11:17:07.190152+0000","last_clean":"2023-08-27T11:17:07.190152+0000","last_became_active":"2023-08-27T11:17:05.376987+0000","last_became_peered":"2023-08-27T11:17:05.376987+0000","last_unstale":"2023-08-27T11:17:07.190152+0000","last_undegraded":"2023-08-27T11:17:07.190152+0000","last_fullsized":"2023-08-27T11:17:07.190152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:30:54.430091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.189991+0000","last_change":"2023-08-27T11:17:05.377602+0000","last_active":"2023-08-27T11:17:07.189991+0000","last_peered":"2023-08-27T11:17:07.189991+0000","last_clean":"2023-08-27T11:17:07.189991+0000","last_became_active":"2023-08-27T11:17:05.377023+0000","last_became_peered":"2023-08-27T11:17:05.377023+0000","last_unstale":"2023-08-27T11:17:07.189991+0000","last_undegraded":"2023-08-27T11:17:07.189991+0000","last_fullsized":"2023-08-27T11:17:07.189991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:18:00.607553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202965+0000","last_change":"2023-08-27T11:17:05.376491+0000","last_active":"2023-08-27T11:17:07.202965+0000","last_peered":"2023-08-27T11:17:07.202965+0000","last_clean":"2023-08-27T11:17:07.202965+0000","last_became_active":"2023-08-27T11:17:05.376149+0000","last_became_peered":"2023-08-27T11:17:05.376149+0000","last_unstale":"2023-08-27T11:17:07.202965+0000","last_undegraded":"2023-08-27T11:17:07.202965+0000","last_fullsized":"2023-08-27T11:17:07.202965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:43.313210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203191+0000","last_change":"2023-08-27T11:17:05.387318+0000","last_active":"2023-08-27T11:17:07.203191+0000","last_peered":"2023-08-27T11:17:07.203191+0000","last_clean":"2023-08-27T11:17:07.203191+0000","last_became_active":"2023-08-27T11:17:05.387002+0000","last_became_peered":"2023-08-27T11:17:05.387002+0000","last_unstale":"2023-08-27T11:17:07.203191+0000","last_undegraded":"2023-08-27T11:17:07.203191+0000","last_fullsized":"2023-08-27T11:17:07.203191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:13.528078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368880+0000","last_change":"2023-08-27T11:17:05.372919+0000","last_active":"2023-08-27T11:17:06.368880+0000","last_peered":"2023-08-27T11:17:06.368880+0000","last_clean":"2023-08-27T11:17:06.368880+0000","last_became_active":"2023-08-27T11:17:05.371149+0000","last_became_peered":"2023-08-27T11:17:05.371149+0000","last_unstale":"2023-08-27T11:17:06.368880+0000","last_undegraded":"2023-08-27T11:17:06.368880+0000","last_fullsized":"2023-08-27T11:17:06.368880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:46.743604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607880+0000","last_change":"2023-08-27T11:17:05.371456+0000","last_active":"2023-08-27T11:17:06.607880+0000","last_peered":"2023-08-27T11:17:06.607880+0000","last_clean":"2023-08-27T11:17:06.607880+0000","last_became_active":"2023-08-27T11:17:05.371193+0000","last_became_peered":"2023-08-27T11:17:05.371193+0000","last_unstale":"2023-08-27T11:17:06.607880+0000","last_undegraded":"2023-08-27T11:17:06.607880+0000","last_fullsized":"2023-08-27T11:17:06.607880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369084+0000","last_change":"2023-08-27T11:17:05.373957+0000","last_active":"2023-08-27T11:17:06.369084+0000","last_peered":"2023-08-27T11:17:06.369084+0000","last_clean":"2023-08-27T11:17:06.369084+0000","last_became_active":"2023-08-27T11:17:05.373507+0000","last_became_peered":"2023-08-27T11:17:05.373507+0000","last_unstale":"2023-08-27T11:17:06.369084+0000","last_undegraded":"2023-08-27T11:17:06.369084+0000","last_fullsized":"2023-08-27T11:17:06.369084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:45.136551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190029+0000","last_change":"2023-08-27T11:17:05.378874+0000","last_active":"2023-08-27T11:17:07.190029+0000","last_peered":"2023-08-27T11:17:07.190029+0000","last_clean":"2023-08-27T11:17:07.190029+0000","last_became_active":"2023-08-27T11:17:05.378585+0000","last_became_peered":"2023-08-27T11:17:05.378585+0000","last_unstale":"2023-08-27T11:17:07.190029+0000","last_undegraded":"2023-08-27T11:17:07.190029+0000","last_fullsized":"2023-08-27T11:17:07.190029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:19:23.730989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368860+0000","last_change":"2023-08-27T11:17:05.377656+0000","last_active":"2023-08-27T11:17:06.368860+0000","last_peered":"2023-08-27T11:17:06.368860+0000","last_clean":"2023-08-27T11:17:06.368860+0000","last_became_active":"2023-08-27T11:17:05.377366+0000","last_became_peered":"2023-08-27T11:17:05.377366+0000","last_unstale":"2023-08-27T11:17:06.368860+0000","last_undegraded":"2023-08-27T11:17:06.368860+0000","last_fullsized":"2023-08-27T11:17:06.368860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202861+0000","last_change":"2023-08-27T11:17:05.379425+0000","last_active":"2023-08-27T11:17:07.202861+0000","last_peered":"2023-08-27T11:17:07.202861+0000","last_clean":"2023-08-27T11:17:07.202861+0000","last_became_active":"2023-08-27T11:17:05.379084+0000","last_became_peered":"2023-08-27T11:17:05.379084+0000","last_unstale":"2023-08-27T11:17:07.202861+0000","last_undegraded":"2023-08-27T11:17:07.202861+0000","last_fullsized":"2023-08-27T11:17:07.202861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607602+0000","last_change":"2023-08-27T11:17:05.381421+0000","last_active":"2023-08-27T11:17:06.607602+0000","last_peered":"2023-08-27T11:17:06.607602+0000","last_clean":"2023-08-27T11:17:06.607602+0000","last_became_active":"2023-08-27T11:17:05.380452+0000","last_became_peered":"2023-08-27T11:17:05.380452+0000","last_unstale":"2023-08-27T11:17:06.607602+0000","last_undegraded":"2023-08-27T11:17:06.607602+0000","last_fullsized":"2023-08-27T11:17:06.607602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202657+0000","last_change":"2023-08-27T11:17:05.378132+0000","last_active":"2023-08-27T11:17:07.202657+0000","last_peered":"2023-08-27T11:17:07.202657+0000","last_clean":"2023-08-27T11:17:07.202657+0000","last_became_active":"2023-08-27T11:17:05.376190+0000","last_became_peered":"2023-08-27T11:17:05.376190+0000","last_unstale":"2023-08-27T11:17:07.202657+0000","last_undegraded":"2023-08-27T11:17:07.202657+0000","last_fullsized":"2023-08-27T11:17:07.202657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203004+0000","last_change":"2023-08-27T11:17:05.378821+0000","last_active":"2023-08-27T11:17:07.203004+0000","last_peered":"2023-08-27T11:17:07.203004+0000","last_clean":"2023-08-27T11:17:07.203004+0000","last_became_active":"2023-08-27T11:17:05.378520+0000","last_became_peered":"2023-08-27T11:17:05.378520+0000","last_unstale":"2023-08-27T11:17:07.203004+0000","last_undegraded":"2023-08-27T11:17:07.203004+0000","last_fullsized":"2023-08-27T11:17:07.203004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:54.962968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367698+0000","last_change":"2023-08-27T11:17:05.379672+0000","last_active":"2023-08-27T11:17:06.367698+0000","last_peered":"2023-08-27T11:17:06.367698+0000","last_clean":"2023-08-27T11:17:06.367698+0000","last_became_active":"2023-08-27T11:17:05.379399+0000","last_became_peered":"2023-08-27T11:17:05.379399+0000","last_unstale":"2023-08-27T11:17:06.367698+0000","last_undegraded":"2023-08-27T11:17:06.367698+0000","last_fullsized":"2023-08-27T11:17:06.367698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:15.192349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367765+0000","last_change":"2023-08-27T11:17:05.376638+0000","last_active":"2023-08-27T11:17:06.367765+0000","last_peered":"2023-08-27T11:17:06.367765+0000","last_clean":"2023-08-27T11:17:06.367765+0000","last_became_active":"2023-08-27T11:17:05.376274+0000","last_became_peered":"2023-08-27T11:17:05.376274+0000","last_unstale":"2023-08-27T11:17:06.367765+0000","last_undegraded":"2023-08-27T11:17:06.367765+0000","last_fullsized":"2023-08-27T11:17:06.367765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202811+0000","last_change":"2023-08-27T11:17:05.382137+0000","last_active":"2023-08-27T11:17:07.202811+0000","last_peered":"2023-08-27T11:17:07.202811+0000","last_clean":"2023-08-27T11:17:07.202811+0000","last_became_active":"2023-08-27T11:17:05.381664+0000","last_became_peered":"2023-08-27T11:17:05.381664+0000","last_unstale":"2023-08-27T11:17:07.202811+0000","last_undegraded":"2023-08-27T11:17:07.202811+0000","last_fullsized":"2023-08-27T11:17:07.202811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369199+0000","last_change":"2023-08-27T11:17:04.376604+0000","last_active":"2023-08-27T11:17:06.369199+0000","last_peered":"2023-08-27T11:17:06.369199+0000","last_clean":"2023-08-27T11:17:06.369199+0000","last_became_active":"2023-08-27T11:17:04.376288+0000","last_became_peered":"2023-08-27T11:17:04.376288+0000","last_unstale":"2023-08-27T11:17:06.369199+0000","last_undegraded":"2023-08-27T11:17:06.369199+0000","last_fullsized":"2023-08-27T11:17:06.369199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:08:24.626675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203128+0000","last_change":"2023-08-27T11:17:05.382097+0000","last_active":"2023-08-27T11:17:07.203128+0000","last_peered":"2023-08-27T11:17:07.203128+0000","last_clean":"2023-08-27T11:17:07.203128+0000","last_became_active":"2023-08-27T11:17:05.381829+0000","last_became_peered":"2023-08-27T11:17:05.381829+0000","last_unstale":"2023-08-27T11:17:07.203128+0000","last_undegraded":"2023-08-27T11:17:07.203128+0000","last_fullsized":"2023-08-27T11:17:07.203128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:45.136551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367598+0000","last_change":"2023-08-27T11:17:04.379398+0000","last_active":"2023-08-27T11:17:06.367598+0000","last_peered":"2023-08-27T11:17:06.367598+0000","last_clean":"2023-08-27T11:17:06.367598+0000","last_became_active":"2023-08-27T11:17:04.379109+0000","last_became_peered":"2023-08-27T11:17:04.379109+0000","last_unstale":"2023-08-27T11:17:06.367598+0000","last_undegraded":"2023-08-27T11:17:06.367598+0000","last_fullsized":"2023-08-27T11:17:06.367598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:45.809057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607933+0000","last_change":"2023-08-27T11:17:05.379671+0000","last_active":"2023-08-27T11:17:06.607933+0000","last_peered":"2023-08-27T11:17:06.607933+0000","last_clean":"2023-08-27T11:17:06.607933+0000","last_became_active":"2023-08-27T11:17:05.379358+0000","last_became_peered":"2023-08-27T11:17:05.379358+0000","last_unstale":"2023-08-27T11:17:06.607933+0000","last_undegraded":"2023-08-27T11:17:06.607933+0000","last_fullsized":"2023-08-27T11:17:06.607933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:54.962968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202719+0000","last_change":"2023-08-27T11:17:04.377528+0000","last_active":"2023-08-27T11:17:07.202719+0000","last_peered":"2023-08-27T11:17:07.202719+0000","last_clean":"2023-08-27T11:17:07.202719+0000","last_became_active":"2023-08-27T11:17:04.377219+0000","last_became_peered":"2023-08-27T11:17:04.377219+0000","last_unstale":"2023-08-27T11:17:07.202719+0000","last_undegraded":"2023-08-27T11:17:07.202719+0000","last_fullsized":"2023-08-27T11:17:07.202719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:38.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369530+0000","last_change":"2023-08-27T11:17:05.372053+0000","last_active":"2023-08-27T11:17:06.369530+0000","last_peered":"2023-08-27T11:17:06.369530+0000","last_clean":"2023-08-27T11:17:06.369530+0000","last_became_active":"2023-08-27T11:17:05.371769+0000","last_became_peered":"2023-08-27T11:17:05.371769+0000","last_unstale":"2023-08-27T11:17:06.369530+0000","last_undegraded":"2023-08-27T11:17:06.369530+0000","last_fullsized":"2023-08-27T11:17:06.369530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607988+0000","last_change":"2023-08-27T11:17:04.378264+0000","last_active":"2023-08-27T11:17:06.607988+0000","last_peered":"2023-08-27T11:17:06.607988+0000","last_clean":"2023-08-27T11:17:06.607988+0000","last_became_active":"2023-08-27T11:17:04.377919+0000","last_became_peered":"2023-08-27T11:17:04.377919+0000","last_unstale":"2023-08-27T11:17:06.607988+0000","last_undegraded":"2023-08-27T11:17:06.607988+0000","last_fullsized":"2023-08-27T11:17:06.607988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:53.143119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202657+0000","last_change":"2023-08-27T11:17:05.383385+0000","last_active":"2023-08-27T11:17:07.202657+0000","last_peered":"2023-08-27T11:17:07.202657+0000","last_clean":"2023-08-27T11:17:07.202657+0000","last_became_active":"2023-08-27T11:17:05.382932+0000","last_became_peered":"2023-08-27T11:17:05.382932+0000","last_unstale":"2023-08-27T11:17:07.202657+0000","last_undegraded":"2023-08-27T11:17:07.202657+0000","last_fullsized":"2023-08-27T11:17:07.202657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202798+0000","last_change":"2023-08-27T11:17:04.373528+0000","last_active":"2023-08-27T11:17:07.202798+0000","last_peered":"2023-08-27T11:17:07.202798+0000","last_clean":"2023-08-27T11:17:07.202798+0000","last_became_active":"2023-08-27T11:17:04.373266+0000","last_became_peered":"2023-08-27T11:17:04.373266+0000","last_unstale":"2023-08-27T11:17:07.202798+0000","last_undegraded":"2023-08-27T11:17:07.202798+0000","last_fullsized":"2023-08-27T11:17:07.202798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:14.186836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202805+0000","last_change":"2023-08-27T11:17:05.383919+0000","last_active":"2023-08-27T11:17:07.202805+0000","last_peered":"2023-08-27T11:17:07.202805+0000","last_clean":"2023-08-27T11:17:07.202805+0000","last_became_active":"2023-08-27T11:17:05.383319+0000","last_became_peered":"2023-08-27T11:17:05.383319+0000","last_unstale":"2023-08-27T11:17:07.202805+0000","last_undegraded":"2023-08-27T11:17:07.202805+0000","last_fullsized":"2023-08-27T11:17:07.202805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:42.812467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203294+0000","last_change":"2023-08-27T11:17:04.379878+0000","last_active":"2023-08-27T11:17:07.203294+0000","last_peered":"2023-08-27T11:17:07.203294+0000","last_clean":"2023-08-27T11:17:07.203294+0000","last_became_active":"2023-08-27T11:17:04.379586+0000","last_became_peered":"2023-08-27T11:17:04.379586+0000","last_unstale":"2023-08-27T11:17:07.203294+0000","last_undegraded":"2023-08-27T11:17:07.203294+0000","last_fullsized":"2023-08-27T11:17:07.203294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:02.601133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367611+0000","last_change":"2023-08-27T11:17:05.385687+0000","last_active":"2023-08-27T11:17:06.367611+0000","last_peered":"2023-08-27T11:17:06.367611+0000","last_clean":"2023-08-27T11:17:06.367611+0000","last_became_active":"2023-08-27T11:17:05.385422+0000","last_became_peered":"2023-08-27T11:17:05.385422+0000","last_unstale":"2023-08-27T11:17:06.367611+0000","last_undegraded":"2023-08-27T11:17:06.367611+0000","last_fullsized":"2023-08-27T11:17:06.367611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:31.508513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369381+0000","last_change":"2023-08-27T11:17:04.372137+0000","last_active":"2023-08-27T11:17:06.369381+0000","last_peered":"2023-08-27T11:17:06.369381+0000","last_clean":"2023-08-27T11:17:06.369381+0000","last_became_active":"2023-08-27T11:17:04.371246+0000","last_became_peered":"2023-08-27T11:17:04.371246+0000","last_unstale":"2023-08-27T11:17:06.369381+0000","last_undegraded":"2023-08-27T11:17:06.369381+0000","last_fullsized":"2023-08-27T11:17:06.369381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:27:15.596770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369513+0000","last_change":"2023-08-27T11:17:05.375967+0000","last_active":"2023-08-27T11:17:06.369513+0000","last_peered":"2023-08-27T11:17:06.369513+0000","last_clean":"2023-08-27T11:17:06.369513+0000","last_became_active":"2023-08-27T11:17:05.375704+0000","last_became_peered":"2023-08-27T11:17:05.375704+0000","last_unstale":"2023-08-27T11:17:06.369513+0000","last_undegraded":"2023-08-27T11:17:06.369513+0000","last_fullsized":"2023-08-27T11:17:06.369513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:55:02.014817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202803+0000","last_change":"2023-08-27T11:17:04.374228+0000","last_active":"2023-08-27T11:17:07.202803+0000","last_peered":"2023-08-27T11:17:07.202803+0000","last_clean":"2023-08-27T11:17:07.202803+0000","last_became_active":"2023-08-27T11:17:04.373855+0000","last_became_peered":"2023-08-27T11:17:04.373855+0000","last_unstale":"2023-08-27T11:17:07.202803+0000","last_undegraded":"2023-08-27T11:17:07.202803+0000","last_fullsized":"2023-08-27T11:17:07.202803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:38.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202908+0000","last_change":"2023-08-27T11:17:05.382048+0000","last_active":"2023-08-27T11:17:07.202908+0000","last_peered":"2023-08-27T11:17:07.202908+0000","last_clean":"2023-08-27T11:17:07.202908+0000","last_became_active":"2023-08-27T11:17:05.381804+0000","last_became_peered":"2023-08-27T11:17:05.381804+0000","last_unstale":"2023-08-27T11:17:07.202908+0000","last_undegraded":"2023-08-27T11:17:07.202908+0000","last_fullsized":"2023-08-27T11:17:07.202908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:55.731983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607767+0000","last_change":"2023-08-27T11:17:04.376663+0000","last_active":"2023-08-27T11:17:06.607767+0000","last_peered":"2023-08-27T11:17:06.607767+0000","last_clean":"2023-08-27T11:17:06.607767+0000","last_became_active":"2023-08-27T11:17:04.376347+0000","last_became_peered":"2023-08-27T11:17:04.376347+0000","last_unstale":"2023-08-27T11:17:06.607767+0000","last_undegraded":"2023-08-27T11:17:06.607767+0000","last_fullsized":"2023-08-27T11:17:06.607767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:14.186836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203533+0000","last_change":"2023-08-27T11:17:05.382733+0000","last_active":"2023-08-27T11:17:07.203533+0000","last_peered":"2023-08-27T11:17:07.203533+0000","last_clean":"2023-08-27T11:17:07.203533+0000","last_became_active":"2023-08-27T11:17:05.382358+0000","last_became_peered":"2023-08-27T11:17:05.382358+0000","last_unstale":"2023-08-27T11:17:07.203533+0000","last_undegraded":"2023-08-27T11:17:07.203533+0000","last_fullsized":"2023-08-27T11:17:07.203533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:42.812467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203143+0000","last_change":"2023-08-27T11:17:04.372997+0000","last_active":"2023-08-27T11:17:07.203143+0000","last_peered":"2023-08-27T11:17:07.203143+0000","last_clean":"2023-08-27T11:17:07.203143+0000","last_became_active":"2023-08-27T11:17:04.372719+0000","last_became_peered":"2023-08-27T11:17:04.372719+0000","last_unstale":"2023-08-27T11:17:07.203143+0000","last_undegraded":"2023-08-27T11:17:07.203143+0000","last_fullsized":"2023-08-27T11:17:07.203143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:54.726470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607805+0000","last_change":"2023-08-27T11:17:05.379397+0000","last_active":"2023-08-27T11:17:06.607805+0000","last_peered":"2023-08-27T11:17:06.607805+0000","last_clean":"2023-08-27T11:17:06.607805+0000","last_became_active":"2023-08-27T11:17:05.377874+0000","last_became_peered":"2023-08-27T11:17:05.377874+0000","last_unstale":"2023-08-27T11:17:06.607805+0000","last_undegraded":"2023-08-27T11:17:06.607805+0000","last_fullsized":"2023-08-27T11:17:06.607805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368437+0000","last_change":"2023-08-27T11:17:04.376949+0000","last_active":"2023-08-27T11:17:06.368437+0000","last_peered":"2023-08-27T11:17:06.368437+0000","last_clean":"2023-08-27T11:17:06.368437+0000","last_became_active":"2023-08-27T11:17:04.376645+0000","last_became_peered":"2023-08-27T11:17:04.376645+0000","last_unstale":"2023-08-27T11:17:06.368437+0000","last_undegraded":"2023-08-27T11:17:06.368437+0000","last_fullsized":"2023-08-27T11:17:06.368437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:30.937446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203216+0000","last_change":"2023-08-27T11:17:05.378949+0000","last_active":"2023-08-27T11:17:07.203216+0000","last_peered":"2023-08-27T11:17:07.203216+0000","last_clean":"2023-08-27T11:17:07.203216+0000","last_became_active":"2023-08-27T11:17:05.378649+0000","last_became_peered":"2023-08-27T11:17:05.378649+0000","last_unstale":"2023-08-27T11:17:07.203216+0000","last_undegraded":"2023-08-27T11:17:07.203216+0000","last_fullsized":"2023-08-27T11:17:07.203216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:43:40.338423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203379+0000","last_change":"2023-08-27T11:17:04.374131+0000","last_active":"2023-08-27T11:17:07.203379+0000","last_peered":"2023-08-27T11:17:07.203379+0000","last_clean":"2023-08-27T11:17:07.203379+0000","last_became_active":"2023-08-27T11:17:04.373809+0000","last_became_peered":"2023-08-27T11:17:04.373809+0000","last_unstale":"2023-08-27T11:17:07.203379+0000","last_undegraded":"2023-08-27T11:17:07.203379+0000","last_fullsized":"2023-08-27T11:17:07.203379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:43:39.332910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202567+0000","last_change":"2023-08-27T11:17:05.379390+0000","last_active":"2023-08-27T11:17:07.202567+0000","last_peered":"2023-08-27T11:17:07.202567+0000","last_clean":"2023-08-27T11:17:07.202567+0000","last_became_active":"2023-08-27T11:17:05.379131+0000","last_became_peered":"2023-08-27T11:17:05.379131+0000","last_unstale":"2023-08-27T11:17:07.202567+0000","last_undegraded":"2023-08-27T11:17:07.202567+0000","last_fullsized":"2023-08-27T11:17:07.202567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:28.711930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203264+0000","last_change":"2023-08-27T11:17:04.374694+0000","last_active":"2023-08-27T11:17:07.203264+0000","last_peered":"2023-08-27T11:17:07.203264+0000","last_clean":"2023-08-27T11:17:07.203264+0000","last_became_active":"2023-08-27T11:17:04.373836+0000","last_became_peered":"2023-08-27T11:17:04.373836+0000","last_unstale":"2023-08-27T11:17:07.203264+0000","last_undegraded":"2023-08-27T11:17:07.203264+0000","last_fullsized":"2023-08-27T11:17:07.203264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:44.131038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368972+0000","last_change":"2023-08-27T11:17:05.380858+0000","last_active":"2023-08-27T11:17:06.368972+0000","last_peered":"2023-08-27T11:17:06.368972+0000","last_clean":"2023-08-27T11:17:06.368972+0000","last_became_active":"2023-08-27T11:17:05.380573+0000","last_became_peered":"2023-08-27T11:17:05.380573+0000","last_unstale":"2023-08-27T11:17:06.368972+0000","last_undegraded":"2023-08-27T11:17:06.368972+0000","last_fullsized":"2023-08-27T11:17:06.368972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:42.999614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203215+0000","last_change":"2023-08-27T11:17:04.374901+0000","last_active":"2023-08-27T11:17:07.203215+0000","last_peered":"2023-08-27T11:17:07.203215+0000","last_clean":"2023-08-27T11:17:07.203215+0000","last_became_active":"2023-08-27T11:17:04.374186+0000","last_became_peered":"2023-08-27T11:17:04.374186+0000","last_unstale":"2023-08-27T11:17:07.203215+0000","last_undegraded":"2023-08-27T11:17:07.203215+0000","last_fullsized":"2023-08-27T11:17:07.203215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:47.391583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369664+0000","last_change":"2023-08-27T11:17:05.380802+0000","last_active":"2023-08-27T11:17:06.369664+0000","last_peered":"2023-08-27T11:17:06.369664+0000","last_clean":"2023-08-27T11:17:06.369664+0000","last_became_active":"2023-08-27T11:17:05.380537+0000","last_became_peered":"2023-08-27T11:17:05.380537+0000","last_unstale":"2023-08-27T11:17:06.369664+0000","last_undegraded":"2023-08-27T11:17:06.369664+0000","last_fullsized":"2023-08-27T11:17:06.369664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203202+0000","last_change":"2023-08-27T11:17:04.379623+0000","last_active":"2023-08-27T11:17:07.203202+0000","last_peered":"2023-08-27T11:17:07.203202+0000","last_clean":"2023-08-27T11:17:07.203202+0000","last_became_active":"2023-08-27T11:17:04.379342+0000","last_became_peered":"2023-08-27T11:17:04.379342+0000","last_unstale":"2023-08-27T11:17:07.203202+0000","last_undegraded":"2023-08-27T11:17:07.203202+0000","last_fullsized":"2023-08-27T11:17:07.203202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:38.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607824+0000","last_change":"2023-08-27T11:17:05.375956+0000","last_active":"2023-08-27T11:17:06.607824+0000","last_peered":"2023-08-27T11:17:06.607824+0000","last_clean":"2023-08-27T11:17:06.607824+0000","last_became_active":"2023-08-27T11:17:05.375040+0000","last_became_peered":"2023-08-27T11:17:05.375040+0000","last_unstale":"2023-08-27T11:17:06.607824+0000","last_undegraded":"2023-08-27T11:17:06.607824+0000","last_fullsized":"2023-08-27T11:17:06.607824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202873+0000","last_change":"2023-08-27T11:17:04.379633+0000","last_active":"2023-08-27T11:17:07.202873+0000","last_peered":"2023-08-27T11:17:07.202873+0000","last_clean":"2023-08-27T11:17:07.202873+0000","last_became_active":"2023-08-27T11:17:04.379306+0000","last_became_peered":"2023-08-27T11:17:04.379306+0000","last_unstale":"2023-08-27T11:17:07.202873+0000","last_undegraded":"2023-08-27T11:17:07.202873+0000","last_fullsized":"2023-08-27T11:17:07.202873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:53.143119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369174+0000","last_change":"2023-08-27T11:17:05.374205+0000","last_active":"2023-08-27T11:17:06.369174+0000","last_peered":"2023-08-27T11:17:06.369174+0000","last_clean":"2023-08-27T11:17:06.369174+0000","last_became_active":"2023-08-27T11:17:05.373675+0000","last_became_peered":"2023-08-27T11:17:05.373675+0000","last_unstale":"2023-08-27T11:17:06.369174+0000","last_undegraded":"2023-08-27T11:17:06.369174+0000","last_fullsized":"2023-08-27T11:17:06.369174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:43:40.338423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369599+0000","last_change":"2023-08-27T11:17:04.371348+0000","last_active":"2023-08-27T11:17:06.369599+0000","last_peered":"2023-08-27T11:17:06.369599+0000","last_clean":"2023-08-27T11:17:06.369599+0000","last_became_active":"2023-08-27T11:17:04.370929+0000","last_became_peered":"2023-08-27T11:17:04.370929+0000","last_unstale":"2023-08-27T11:17:06.369599+0000","last_undegraded":"2023-08-27T11:17:06.369599+0000","last_fullsized":"2023-08-27T11:17:06.369599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:27.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369296+0000","last_change":"2023-08-27T11:17:05.373711+0000","last_active":"2023-08-27T11:17:06.369296+0000","last_peered":"2023-08-27T11:17:06.369296+0000","last_clean":"2023-08-27T11:17:06.369296+0000","last_became_active":"2023-08-27T11:17:05.373392+0000","last_became_peered":"2023-08-27T11:17:05.373392+0000","last_unstale":"2023-08-27T11:17:06.369296+0000","last_undegraded":"2023-08-27T11:17:06.369296+0000","last_fullsized":"2023-08-27T11:17:06.369296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:35.735781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369479+0000","last_change":"2023-08-27T11:17:04.373259+0000","last_active":"2023-08-27T11:17:06.369479+0000","last_peered":"2023-08-27T11:17:06.369479+0000","last_clean":"2023-08-27T11:17:06.369479+0000","last_became_active":"2023-08-27T11:17:04.372983+0000","last_became_peered":"2023-08-27T11:17:04.372983+0000","last_unstale":"2023-08-27T11:17:06.369479+0000","last_undegraded":"2023-08-27T11:17:06.369479+0000","last_fullsized":"2023-08-27T11:17:06.369479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:36.570705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369051+0000","last_change":"2023-08-27T11:17:05.376729+0000","last_active":"2023-08-27T11:17:06.369051+0000","last_peered":"2023-08-27T11:17:06.369051+0000","last_clean":"2023-08-27T11:17:06.369051+0000","last_became_active":"2023-08-27T11:17:05.376443+0000","last_became_peered":"2023-08-27T11:17:05.376443+0000","last_unstale":"2023-08-27T11:17:06.369051+0000","last_undegraded":"2023-08-27T11:17:06.369051+0000","last_fullsized":"2023-08-27T11:17:06.369051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:48.397096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202559+0000","last_change":"2023-08-27T11:17:04.377051+0000","last_active":"2023-08-27T11:17:07.202559+0000","last_peered":"2023-08-27T11:17:07.202559+0000","last_clean":"2023-08-27T11:17:07.202559+0000","last_became_active":"2023-08-27T11:17:04.376700+0000","last_became_peered":"2023-08-27T11:17:04.376700+0000","last_unstale":"2023-08-27T11:17:07.202559+0000","last_undegraded":"2023-08-27T11:17:07.202559+0000","last_fullsized":"2023-08-27T11:17:07.202559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:42.307697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367824+0000","last_change":"2023-08-27T11:17:05.383341+0000","last_active":"2023-08-27T11:17:06.367824+0000","last_peered":"2023-08-27T11:17:06.367824+0000","last_clean":"2023-08-27T11:17:06.367824+0000","last_became_active":"2023-08-27T11:17:05.379929+0000","last_became_peered":"2023-08-27T11:17:05.379929+0000","last_unstale":"2023-08-27T11:17:06.367824+0000","last_undegraded":"2023-08-27T11:17:06.367824+0000","last_fullsized":"2023-08-27T11:17:06.367824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203147+0000","last_change":"2023-08-27T11:17:04.378623+0000","last_active":"2023-08-27T11:17:07.203147+0000","last_peered":"2023-08-27T11:17:07.203147+0000","last_clean":"2023-08-27T11:17:07.203147+0000","last_became_active":"2023-08-27T11:17:04.378327+0000","last_became_peered":"2023-08-27T11:17:04.378327+0000","last_unstale":"2023-08-27T11:17:07.203147+0000","last_undegraded":"2023-08-27T11:17:07.203147+0000","last_fullsized":"2023-08-27T11:17:07.203147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:02.601133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368983+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:17:06.368983+0000","last_peered":"2023-08-27T11:17:06.368983+0000","last_clean":"2023-08-27T11:17:06.368983+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:17:06.368983+0000","last_undegraded":"2023-08-27T11:17:06.368983+0000","last_fullsized":"2023-08-27T11:17:06.368983+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:07.481452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202508+0000","last_change":"2023-08-27T11:17:05.383215+0000","last_active":"2023-08-27T11:17:07.202508+0000","last_peered":"2023-08-27T11:17:07.202508+0000","last_clean":"2023-08-27T11:17:07.202508+0000","last_became_active":"2023-08-27T11:17:05.382502+0000","last_became_peered":"2023-08-27T11:17:05.382502+0000","last_unstale":"2023-08-27T11:17:07.202508+0000","last_undegraded":"2023-08-27T11:17:07.202508+0000","last_fullsized":"2023-08-27T11:17:07.202508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203019+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:17:07.203019+0000","last_peered":"2023-08-27T11:17:07.203019+0000","last_clean":"2023-08-27T11:17:07.203019+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:17:07.203019+0000","last_undegraded":"2023-08-27T11:17:07.203019+0000","last_fullsized":"2023-08-27T11:17:07.203019+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:42.350470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.290563+0000","last_change":"2023-08-27T11:17:04.380486+0000","last_active":"2023-08-27T11:17:07.290563+0000","last_peered":"2023-08-27T11:17:07.290563+0000","last_clean":"2023-08-27T11:17:07.290563+0000","last_became_active":"2023-08-27T11:17:04.376679+0000","last_became_peered":"2023-08-27T11:17:04.376679+0000","last_unstale":"2023-08-27T11:17:07.290563+0000","last_undegraded":"2023-08-27T11:17:07.290563+0000","last_fullsized":"2023-08-27T11:17:07.290563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:41.806953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369152+0000","last_change":"2023-08-27T11:17:05.370457+0000","last_active":"2023-08-27T11:17:06.369152+0000","last_peered":"2023-08-27T11:17:06.369152+0000","last_clean":"2023-08-27T11:17:06.369152+0000","last_became_active":"2023-08-27T11:17:05.370040+0000","last_became_peered":"2023-08-27T11:17:05.370040+0000","last_unstale":"2023-08-27T11:17:06.369152+0000","last_undegraded":"2023-08-27T11:17:06.369152+0000","last_fullsized":"2023-08-27T11:17:06.369152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:03.606646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367481+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:17:06.367481+0000","last_peered":"2023-08-27T11:17:06.367481+0000","last_clean":"2023-08-27T11:17:06.367481+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:17:06.367481+0000","last_undegraded":"2023-08-27T11:17:06.367481+0000","last_fullsized":"2023-08-27T11:17:06.367481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:07:15.067358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203119+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:17:07.203119+0000","last_peered":"2023-08-27T11:17:07.203119+0000","last_clean":"2023-08-27T11:17:07.203119+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:17:07.203119+0000","last_undegraded":"2023-08-27T11:17:07.203119+0000","last_fullsized":"2023-08-27T11:17:07.203119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:36.952988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607599+0000","last_change":"2023-08-27T11:17:04.373430+0000","last_active":"2023-08-27T11:17:06.607599+0000","last_peered":"2023-08-27T11:17:06.607599+0000","last_clean":"2023-08-27T11:17:06.607599+0000","last_became_active":"2023-08-27T11:17:04.373098+0000","last_became_peered":"2023-08-27T11:17:04.373098+0000","last_unstale":"2023-08-27T11:17:06.607599+0000","last_undegraded":"2023-08-27T11:17:06.607599+0000","last_fullsized":"2023-08-27T11:17:06.607599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:07:20.108470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607829+0000","last_change":"2023-08-27T11:17:05.380096+0000","last_active":"2023-08-27T11:17:06.607829+0000","last_peered":"2023-08-27T11:17:06.607829+0000","last_clean":"2023-08-27T11:17:06.607829+0000","last_became_active":"2023-08-27T11:17:05.379846+0000","last_became_peered":"2023-08-27T11:17:05.379846+0000","last_unstale":"2023-08-27T11:17:06.607829+0000","last_undegraded":"2023-08-27T11:17:06.607829+0000","last_fullsized":"2023-08-27T11:17:06.607829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202806+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:17:07.202806+0000","last_peered":"2023-08-27T11:17:07.202806+0000","last_clean":"2023-08-27T11:17:07.202806+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:17:07.202806+0000","last_undegraded":"2023-08-27T11:17:07.202806+0000","last_fullsized":"2023-08-27T11:17:07.202806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:48.102006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367401+0000","last_change":"2023-08-27T11:17:04.373552+0000","last_active":"2023-08-27T11:17:06.367401+0000","last_peered":"2023-08-27T11:17:06.367401+0000","last_clean":"2023-08-27T11:17:06.367401+0000","last_became_active":"2023-08-27T11:17:04.373258+0000","last_became_peered":"2023-08-27T11:17:04.373258+0000","last_unstale":"2023-08-27T11:17:06.367401+0000","last_undegraded":"2023-08-27T11:17:06.367401+0000","last_fullsized":"2023-08-27T11:17:06.367401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:30.937446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367737+0000","last_change":"2023-08-27T11:17:05.383636+0000","last_active":"2023-08-27T11:17:06.367737+0000","last_peered":"2023-08-27T11:17:06.367737+0000","last_clean":"2023-08-27T11:17:06.367737+0000","last_became_active":"2023-08-27T11:17:05.383410+0000","last_became_peered":"2023-08-27T11:17:05.383410+0000","last_unstale":"2023-08-27T11:17:06.367737+0000","last_undegraded":"2023-08-27T11:17:06.367737+0000","last_fullsized":"2023-08-27T11:17:06.367737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:43.313210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202834+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:17:07.202834+0000","last_peered":"2023-08-27T11:17:07.202834+0000","last_clean":"2023-08-27T11:17:07.202834+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:17:07.202834+0000","last_undegraded":"2023-08-27T11:17:07.202834+0000","last_fullsized":"2023-08-27T11:17:07.202834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:36.765841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202793+0000","last_change":"2023-08-27T11:17:04.375838+0000","last_active":"2023-08-27T11:17:07.202793+0000","last_peered":"2023-08-27T11:17:07.202793+0000","last_clean":"2023-08-27T11:17:07.202793+0000","last_became_active":"2023-08-27T11:17:04.375536+0000","last_became_peered":"2023-08-27T11:17:04.375536+0000","last_unstale":"2023-08-27T11:17:07.202793+0000","last_undegraded":"2023-08-27T11:17:07.202793+0000","last_fullsized":"2023-08-27T11:17:07.202793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:53.957455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369664+0000","last_change":"2023-08-27T11:17:05.375110+0000","last_active":"2023-08-27T11:17:06.369664+0000","last_peered":"2023-08-27T11:17:06.369664+0000","last_clean":"2023-08-27T11:17:06.369664+0000","last_became_active":"2023-08-27T11:17:05.374844+0000","last_became_peered":"2023-08-27T11:17:05.374844+0000","last_unstale":"2023-08-27T11:17:06.369664+0000","last_undegraded":"2023-08-27T11:17:06.369664+0000","last_fullsized":"2023-08-27T11:17:06.369664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:15.192349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202453+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:17:07.202453+0000","last_peered":"2023-08-27T11:17:07.202453+0000","last_clean":"2023-08-27T11:17:07.202453+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:17:07.202453+0000","last_undegraded":"2023-08-27T11:17:07.202453+0000","last_fullsized":"2023-08-27T11:17:07.202453+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:43.315205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368949+0000","last_change":"2023-08-27T11:17:04.379053+0000","last_active":"2023-08-27T11:17:06.368949+0000","last_peered":"2023-08-27T11:17:06.368949+0000","last_clean":"2023-08-27T11:17:06.368949+0000","last_became_active":"2023-08-27T11:17:04.378731+0000","last_became_peered":"2023-08-27T11:17:04.378731+0000","last_unstale":"2023-08-27T11:17:06.368949+0000","last_undegraded":"2023-08-27T11:17:06.368949+0000","last_fullsized":"2023-08-27T11:17:06.368949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:53.143119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367664+0000","last_change":"2023-08-27T11:17:05.373510+0000","last_active":"2023-08-27T11:17:06.367664+0000","last_peered":"2023-08-27T11:17:06.367664+0000","last_clean":"2023-08-27T11:17:06.367664+0000","last_became_active":"2023-08-27T11:17:05.373229+0000","last_became_peered":"2023-08-27T11:17:05.373229+0000","last_unstale":"2023-08-27T11:17:06.367664+0000","last_undegraded":"2023-08-27T11:17:06.367664+0000","last_fullsized":"2023-08-27T11:17:06.367664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369394+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:17:06.369394+0000","last_peered":"2023-08-27T11:17:06.369394+0000","last_clean":"2023-08-27T11:17:06.369394+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:17:06.369394+0000","last_undegraded":"2023-08-27T11:17:06.369394+0000","last_fullsized":"2023-08-27T11:17:06.369394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:07:15.067358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367592+0000","last_change":"2023-08-27T11:17:04.374506+0000","last_active":"2023-08-27T11:17:06.367592+0000","last_peered":"2023-08-27T11:17:06.367592+0000","last_clean":"2023-08-27T11:17:06.367592+0000","last_became_active":"2023-08-27T11:17:04.374209+0000","last_became_peered":"2023-08-27T11:17:04.374209+0000","last_unstale":"2023-08-27T11:17:06.367592+0000","last_undegraded":"2023-08-27T11:17:06.367592+0000","last_fullsized":"2023-08-27T11:17:06.367592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:27.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607676+0000","last_change":"2023-08-27T11:17:05.369565+0000","last_active":"2023-08-27T11:17:06.607676+0000","last_peered":"2023-08-27T11:17:06.607676+0000","last_clean":"2023-08-27T11:17:06.607676+0000","last_became_active":"2023-08-27T11:17:05.369334+0000","last_became_peered":"2023-08-27T11:17:05.369334+0000","last_unstale":"2023-08-27T11:17:06.607676+0000","last_undegraded":"2023-08-27T11:17:06.607676+0000","last_fullsized":"2023-08-27T11:17:06.607676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369316+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:17:06.369316+0000","last_peered":"2023-08-27T11:17:06.369316+0000","last_clean":"2023-08-27T11:17:06.369316+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:17:06.369316+0000","last_undegraded":"2023-08-27T11:17:06.369316+0000","last_fullsized":"2023-08-27T11:17:06.369316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:25.461887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203500+0000","last_change":"2023-08-27T11:17:04.380000+0000","last_active":"2023-08-27T11:17:07.203500+0000","last_peered":"2023-08-27T11:17:07.203500+0000","last_clean":"2023-08-27T11:17:07.203500+0000","last_became_active":"2023-08-27T11:17:04.379666+0000","last_became_peered":"2023-08-27T11:17:04.379666+0000","last_unstale":"2023-08-27T11:17:07.203500+0000","last_undegraded":"2023-08-27T11:17:07.203500+0000","last_fullsized":"2023-08-27T11:17:07.203500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:53.957455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367721+0000","last_change":"2023-08-27T11:17:05.380929+0000","last_active":"2023-08-27T11:17:06.367721+0000","last_peered":"2023-08-27T11:17:06.367721+0000","last_clean":"2023-08-27T11:17:06.367721+0000","last_became_active":"2023-08-27T11:17:05.380573+0000","last_became_peered":"2023-08-27T11:17:05.380573+0000","last_unstale":"2023-08-27T11:17:06.367721+0000","last_undegraded":"2023-08-27T11:17:06.367721+0000","last_fullsized":"2023-08-27T11:17:06.367721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607806+0000","last_change":"2023-08-27T11:17:04.377322+0000","last_active":"2023-08-27T11:17:06.607806+0000","last_peered":"2023-08-27T11:17:06.607806+0000","last_clean":"2023-08-27T11:17:06.607806+0000","last_became_active":"2023-08-27T11:17:04.376967+0000","last_became_peered":"2023-08-27T11:17:04.376967+0000","last_unstale":"2023-08-27T11:17:06.607806+0000","last_undegraded":"2023-08-27T11:17:06.607806+0000","last_fullsized":"2023-08-27T11:17:06.607806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:42.307697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190107+0000","last_change":"2023-08-27T11:17:05.382640+0000","last_active":"2023-08-27T11:17:07.190107+0000","last_peered":"2023-08-27T11:17:07.190107+0000","last_clean":"2023-08-27T11:17:07.190107+0000","last_became_active":"2023-08-27T11:17:05.382377+0000","last_became_peered":"2023-08-27T11:17:05.382377+0000","last_unstale":"2023-08-27T11:17:07.190107+0000","last_undegraded":"2023-08-27T11:17:07.190107+0000","last_fullsized":"2023-08-27T11:17:07.190107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:27:57.016677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203339+0000","last_change":"2023-08-27T11:17:04.372997+0000","last_active":"2023-08-27T11:17:07.203339+0000","last_peered":"2023-08-27T11:17:07.203339+0000","last_clean":"2023-08-27T11:17:07.203339+0000","last_became_active":"2023-08-27T11:17:04.372719+0000","last_became_peered":"2023-08-27T11:17:04.372719+0000","last_unstale":"2023-08-27T11:17:07.203339+0000","last_undegraded":"2023-08-27T11:17:07.203339+0000","last_fullsized":"2023-08-27T11:17:07.203339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:51.710890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368989+0000","last_change":"2023-08-27T11:17:05.374254+0000","last_active":"2023-08-27T11:17:06.368989+0000","last_peered":"2023-08-27T11:17:06.368989+0000","last_clean":"2023-08-27T11:17:06.368989+0000","last_became_active":"2023-08-27T11:17:05.373877+0000","last_became_peered":"2023-08-27T11:17:05.373877+0000","last_unstale":"2023-08-27T11:17:06.368989+0000","last_undegraded":"2023-08-27T11:17:06.368989+0000","last_fullsized":"2023-08-27T11:17:06.368989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:42.812467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203587+0000","last_change":"2023-08-27T11:17:04.372930+0000","last_active":"2023-08-27T11:17:07.203587+0000","last_peered":"2023-08-27T11:17:07.203587+0000","last_clean":"2023-08-27T11:17:07.203587+0000","last_became_active":"2023-08-27T11:17:04.372626+0000","last_became_peered":"2023-08-27T11:17:04.372626+0000","last_unstale":"2023-08-27T11:17:07.203587+0000","last_undegraded":"2023-08-27T11:17:07.203587+0000","last_fullsized":"2023-08-27T11:17:07.203587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:45.738091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369549+0000","last_change":"2023-08-27T11:17:05.374125+0000","last_active":"2023-08-27T11:17:06.369549+0000","last_peered":"2023-08-27T11:17:06.369549+0000","last_clean":"2023-08-27T11:17:06.369549+0000","last_became_active":"2023-08-27T11:17:05.373856+0000","last_became_peered":"2023-08-27T11:17:05.373856+0000","last_unstale":"2023-08-27T11:17:06.369549+0000","last_undegraded":"2023-08-27T11:17:06.369549+0000","last_fullsized":"2023-08-27T11:17:06.369549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:17:32.516797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203038+0000","last_change":"2023-08-27T11:17:04.375965+0000","last_active":"2023-08-27T11:17:07.203038+0000","last_peered":"2023-08-27T11:17:07.203038+0000","last_clean":"2023-08-27T11:17:07.203038+0000","last_became_active":"2023-08-27T11:17:04.375687+0000","last_became_peered":"2023-08-27T11:17:04.375687+0000","last_unstale":"2023-08-27T11:17:07.203038+0000","last_undegraded":"2023-08-27T11:17:07.203038+0000","last_fullsized":"2023-08-27T11:17:07.203038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:12.522565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202911+0000","last_change":"2023-08-27T11:17:05.372749+0000","last_active":"2023-08-27T11:17:07.202911+0000","last_peered":"2023-08-27T11:17:07.202911+0000","last_clean":"2023-08-27T11:17:07.202911+0000","last_became_active":"2023-08-27T11:17:05.372494+0000","last_became_peered":"2023-08-27T11:17:05.372494+0000","last_unstale":"2023-08-27T11:17:07.202911+0000","last_undegraded":"2023-08-27T11:17:07.202911+0000","last_fullsized":"2023-08-27T11:17:07.202911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:42.999614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607966+0000","last_change":"2023-08-27T11:17:04.374287+0000","last_active":"2023-08-27T11:17:06.607966+0000","last_peered":"2023-08-27T11:17:06.607966+0000","last_clean":"2023-08-27T11:17:06.607966+0000","last_became_active":"2023-08-27T11:17:04.374009+0000","last_became_peered":"2023-08-27T11:17:04.374009+0000","last_unstale":"2023-08-27T11:17:06.607966+0000","last_undegraded":"2023-08-27T11:17:06.607966+0000","last_fullsized":"2023-08-27T11:17:06.607966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:45.738091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368606+0000","last_change":"2023-08-27T11:17:05.369490+0000","last_active":"2023-08-27T11:17:06.368606+0000","last_peered":"2023-08-27T11:17:06.368606+0000","last_clean":"2023-08-27T11:17:06.368606+0000","last_became_active":"2023-08-27T11:17:05.369193+0000","last_became_peered":"2023-08-27T11:17:05.369193+0000","last_unstale":"2023-08-27T11:17:06.368606+0000","last_undegraded":"2023-08-27T11:17:06.368606+0000","last_fullsized":"2023-08-27T11:17:06.368606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368508+0000","last_change":"2023-08-27T11:17:05.376280+0000","last_active":"2023-08-27T11:17:06.368508+0000","last_peered":"2023-08-27T11:17:06.368508+0000","last_clean":"2023-08-27T11:17:06.368508+0000","last_became_active":"2023-08-27T11:17:05.375763+0000","last_became_peered":"2023-08-27T11:17:05.375763+0000","last_unstale":"2023-08-27T11:17:06.368508+0000","last_undegraded":"2023-08-27T11:17:06.368508+0000","last_fullsized":"2023-08-27T11:17:06.368508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:46.814570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202731+0000","last_change":"2023-08-27T11:17:05.384097+0000","last_active":"2023-08-27T11:17:07.202731+0000","last_peered":"2023-08-27T11:17:07.202731+0000","last_clean":"2023-08-27T11:17:07.202731+0000","last_became_active":"2023-08-27T11:17:05.383673+0000","last_became_peered":"2023-08-27T11:17:05.383673+0000","last_unstale":"2023-08-27T11:17:07.202731+0000","last_undegraded":"2023-08-27T11:17:07.202731+0000","last_fullsized":"2023-08-27T11:17:07.202731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368848+0000","last_change":"2023-08-27T11:17:05.369440+0000","last_active":"2023-08-27T11:17:06.368848+0000","last_peered":"2023-08-27T11:17:06.368848+0000","last_clean":"2023-08-27T11:17:06.368848+0000","last_became_active":"2023-08-27T11:17:05.369113+0000","last_became_peered":"2023-08-27T11:17:05.369113+0000","last_unstale":"2023-08-27T11:17:06.368848+0000","last_undegraded":"2023-08-27T11:17:06.368848+0000","last_fullsized":"2023-08-27T11:17:06.368848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:39.209301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202857+0000","last_change":"2023-08-27T11:17:05.381671+0000","last_active":"2023-08-27T11:17:07.202857+0000","last_peered":"2023-08-27T11:17:07.202857+0000","last_clean":"2023-08-27T11:17:07.202857+0000","last_became_active":"2023-08-27T11:17:05.380933+0000","last_became_peered":"2023-08-27T11:17:05.380933+0000","last_unstale":"2023-08-27T11:17:07.202857+0000","last_undegraded":"2023-08-27T11:17:07.202857+0000","last_fullsized":"2023-08-27T11:17:07.202857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:46.743604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367534+0000","last_change":"2023-08-27T11:17:05.379944+0000","last_active":"2023-08-27T11:17:06.367534+0000","last_peered":"2023-08-27T11:17:06.367534+0000","last_clean":"2023-08-27T11:17:06.367534+0000","last_became_active":"2023-08-27T11:17:05.379633+0000","last_became_peered":"2023-08-27T11:17:05.379633+0000","last_unstale":"2023-08-27T11:17:06.367534+0000","last_undegraded":"2023-08-27T11:17:06.367534+0000","last_fullsized":"2023-08-27T11:17:06.367534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":666624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":74428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":664796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":73649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":74428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":76096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":75080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:17:14.168 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:17:14.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.319+0000 7f044074a700 1 Processor -- start 2023-08-27T11:17:14.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.319+0000 7f044074a700 1 -- start start 2023-08-27T11:17:14.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.320+0000 7f044074a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f04380b17e0 0x7f04380cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:14.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.320+0000 7f044074a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0438161ce0 0x7f043806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:14.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.320+0000 7f044074a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0438161850 con 0x7f04380b1400 2023-08-27T11:17:14.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.320+0000 7f044074a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04380cfc10 con 0x7f04380b17e0 2023-08-27T11:17:14.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.320+0000 7f044074a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0438161200 con 0x7f0438161ce0 2023-08-27T11:17:14.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.320+0000 7f043dce5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f04380b17e0 0x7f04380cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:14.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.320+0000 7f043dce5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f04380b17e0 0x7f04380cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55178/0 (socket says 172.21.15.112:55178) 2023-08-27T11:17:14.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.320+0000 7f043dce5700 1 -- 172.21.15.112:0/3920053497 learned_addr learned my addr 172.21.15.112:0/3920053497 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:14.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.320+0000 7f043ece7700 1 --2- 172.21.15.112:0/3920053497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0438161ce0 0x7f043806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:14.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f043dce5700 1 -- 172.21.15.112:0/3920053497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0438161ce0 msgr2=0x7f043806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:14.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f043dce5700 1 --2- 172.21.15.112:0/3920053497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0438161ce0 0x7f043806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f043dce5700 1 -- 172.21.15.112:0/3920053497 >> v1:172.21.15.138:6789/0 conn(0x7f04380b1400 legacy=0x7f043806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f043dce5700 1 -- 172.21.15.112:0/3920053497 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04381614e0 con 0x7f04380b17e0 2023-08-27T11:17:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f043dce5700 1 --2- 172.21.15.112:0/3920053497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04380b17e0 0x7f04380cf5a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f04280098f0 tx=0x7f0428009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6cf019345c90e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f043d4e4700 1 -- 172.21.15.112:0/3920053497 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042800b8a0 con 0x7f04380b17e0 2023-08-27T11:17:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f043d4e4700 1 -- 172.21.15.112:0/3920053497 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042800ba00 con 0x7f04380b17e0 2023-08-27T11:17:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f043d4e4700 1 -- 172.21.15.112:0/3920053497 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042800bd90 con 0x7f04380b17e0 2023-08-27T11:17:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f044074a700 1 -- 172.21.15.112:0/3920053497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04380b17e0 msgr2=0x7f04380cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f044074a700 1 --2- 172.21.15.112:0/3920053497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04380b17e0 0x7f04380cf5a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f04280098f0 tx=0x7f0428009c00 comp rx=0 tx=0).stop 2023-08-27T11:17:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f044074a700 1 -- 172.21.15.112:0/3920053497 shutdown_connections 2023-08-27T11:17:14.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f044074a700 1 --2- 172.21.15.112:0/3920053497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0438161ce0 0x7f043806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f044074a700 1 --2- 172.21.15.112:0/3920053497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04380b17e0 0x7f04380cf5a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f044074a700 1 -- 172.21.15.112:0/3920053497 >> 172.21.15.112:0/3920053497 conn(0x7f04380d0000 msgr2=0x7f04380d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:14.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.321+0000 7f044074a700 1 -- 172.21.15.112:0/3920053497 shutdown_connections 2023-08-27T11:17:14.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.322+0000 7f044074a700 1 -- 172.21.15.112:0/3920053497 wait complete. 2023-08-27T11:17:14.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.323+0000 7f044074a700 1 Processor -- start 2023-08-27T11:17:14.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.323+0000 7f044074a700 1 -- start start 2023-08-27T11:17:14.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.323+0000 7f044074a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04380b1400 0x7f04381474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:14.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.323+0000 7f044074a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04380b17e0 0x7f04380c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:14.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.323+0000 7f044074a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0438161ce0 0x7f0438147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:14.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.323+0000 7f044074a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04380c7270 con 0x7f04380b1400 2023-08-27T11:17:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.323+0000 7f044074a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04380c7550 con 0x7f04380b17e0 2023-08-27T11:17:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.323+0000 7f044074a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04380c7830 con 0x7f0438161ce0 2023-08-27T11:17:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f043e4e6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04380b1400 0x7f04381474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f043e4e6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04380b1400 0x7f04381474a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55182/0 (socket says 172.21.15.112:55182) 2023-08-27T11:17:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f043e4e6700 1 -- 172.21.15.112:0/3436039189 learned_addr learned my addr 172.21.15.112:0/3436039189 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:14.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f043dce5700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04380b17e0 0x7f04380c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:14.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f043ece7700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0438161ce0 0x7f0438147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:14.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f043e4e6700 1 -- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0438161ce0 msgr2=0x7f0438147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f043e4e6700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0438161ce0 0x7f0438147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f043e4e6700 1 -- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04380b17e0 msgr2=0x7f04380c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f043e4e6700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04380b17e0 0x7f04380c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f043e4e6700 1 -- 172.21.15.112:0/3436039189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0428009550 con 0x7f04380b1400 2023-08-27T11:17:14.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f043ece7700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0438161ce0 0x7f0438147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f043e4e6700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04380b1400 0x7f04381474a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f043400c4e0 tx=0x7f043400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f042effd700 1 -- 172.21.15.112:0/3436039189 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04340052d0 con 0x7f04380b1400 2023-08-27T11:17:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f042effd700 1 -- 172.21.15.112:0/3436039189 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0434005430 con 0x7f04380b1400 2023-08-27T11:17:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f044074a700 1 -- 172.21.15.112:0/3436039189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04380c7b10 con 0x7f04380b1400 2023-08-27T11:17:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f042effd700 1 -- 172.21.15.112:0/3436039189 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04340038c0 con 0x7f04380b1400 2023-08-27T11:17:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.324+0000 7f044074a700 1 -- 172.21.15.112:0/3436039189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04381479e0 con 0x7f04380b1400 2023-08-27T11:17:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.325+0000 7f044074a700 1 -- 172.21.15.112:0/3436039189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0438067a20 con 0x7f04380b1400 2023-08-27T11:17:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.325+0000 7f042effd700 1 -- 172.21.15.112:0/3436039189 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0434003a20 con 0x7f04380b1400 2023-08-27T11:17:14.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.326+0000 7f042effd700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f041c06a0e0 0x7f041c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:14.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.326+0000 7f042effd700 1 -- 172.21.15.112:0/3436039189 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f04340879e0 con 0x7f04380b1400 2023-08-27T11:17:14.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.326+0000 7f043dce5700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f041c06a0e0 0x7f041c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:14.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.326+0000 7f043dce5700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f041c06a0e0 0x7f041c06c5a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f04380b1bf0 tx=0x7f0428008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:14.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.330+0000 7f042effd700 1 -- 172.21.15.112:0/3436039189 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f04340530d0 con 0x7f04380b1400 2023-08-27T11:17:14.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.516+0000 7f044074a700 1 -- 172.21.15.112:0/3436039189 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04380cb3c0 con 0x7f041c06a0e0 2023-08-27T11:17:14.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.524+0000 7f042effd700 1 -- 172.21.15.112:0/3436039189 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286852 (secure 0 0 0) 0x7f04380cb3c0 con 0x7f041c06a0e0 2023-08-27T11:17:14.529 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:14.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.527+0000 7f044074a700 1 -- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f041c06a0e0 msgr2=0x7f041c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.527+0000 7f044074a700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f041c06a0e0 0x7f041c06c5a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f04380b1bf0 tx=0x7f0428008030 comp rx=0 tx=0).stop 2023-08-27T11:17:14.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.527+0000 7f044074a700 1 -- 172.21.15.112:0/3436039189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04380b1400 msgr2=0x7f04381474a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.527+0000 7f044074a700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04380b1400 0x7f04381474a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f043400c4e0 tx=0x7f043400c7f0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.527+0000 7f044074a700 1 -- 172.21.15.112:0/3436039189 shutdown_connections 2023-08-27T11:17:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.527+0000 7f044074a700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f041c06a0e0 0x7f041c06c5a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.527+0000 7f044074a700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0438161ce0 0x7f0438147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.527+0000 7f044074a700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04380b1400 0x7f04381474a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.527+0000 7f044074a700 1 --2- 172.21.15.112:0/3436039189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04380b17e0 0x7f04380c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.527+0000 7f044074a700 1 -- 172.21.15.112:0/3436039189 >> 172.21.15.112:0/3436039189 conn(0x7f04380d0000 msgr2=0x7f043815ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.527+0000 7f044074a700 1 -- 172.21.15.112:0/3436039189 shutdown_connections 2023-08-27T11:17:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.527+0000 7f044074a700 1 -- 172.21.15.112:0/3436039189 wait complete. 2023-08-27T11:17:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:17:14.540 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-08-27T11:17:14.226121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219044,"kb_used_data":2820,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755676,"statfs":{"total":773094113280,"available":772869812224,"internally_reserved":0,"allocated":2887680,"data_stored":1779619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.865249"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190109+0000","last_change":"2023-08-27T11:17:05.373064+0000","last_active":"2023-08-27T11:17:07.190109+0000","last_peered":"2023-08-27T11:17:07.190109+0000","last_clean":"2023-08-27T11:17:07.190109+0000","last_became_active":"2023-08-27T11:17:05.372771+0000","last_became_peered":"2023-08-27T11:17:05.372771+0000","last_unstale":"2023-08-27T11:17:07.190109+0000","last_undegraded":"2023-08-27T11:17:07.190109+0000","last_fullsized":"2023-08-27T11:17:07.190109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:12:06.488460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202664+0000","last_change":"2023-08-27T11:17:05.380926+0000","last_active":"2023-08-27T11:17:07.202664+0000","last_peered":"2023-08-27T11:17:07.202664+0000","last_clean":"2023-08-27T11:17:07.202664+0000","last_became_active":"2023-08-27T11:17:05.380623+0000","last_became_peered":"2023-08-27T11:17:05.380623+0000","last_unstale":"2023-08-27T11:17:07.202664+0000","last_undegraded":"2023-08-27T11:17:07.202664+0000","last_fullsized":"2023-08-27T11:17:07.202664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368869+0000","last_change":"2023-08-27T11:17:05.377335+0000","last_active":"2023-08-27T11:17:06.368869+0000","last_peered":"2023-08-27T11:17:06.368869+0000","last_clean":"2023-08-27T11:17:06.368869+0000","last_became_active":"2023-08-27T11:17:05.376963+0000","last_became_peered":"2023-08-27T11:17:05.376963+0000","last_unstale":"2023-08-27T11:17:06.368869+0000","last_undegraded":"2023-08-27T11:17:06.368869+0000","last_fullsized":"2023-08-27T11:17:06.368869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:54.962968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369836+0000","last_change":"2023-08-27T11:17:05.371708+0000","last_active":"2023-08-27T11:17:06.369836+0000","last_peered":"2023-08-27T11:17:06.369836+0000","last_clean":"2023-08-27T11:17:06.369836+0000","last_became_active":"2023-08-27T11:17:05.371385+0000","last_became_peered":"2023-08-27T11:17:05.371385+0000","last_unstale":"2023-08-27T11:17:06.369836+0000","last_undegraded":"2023-08-27T11:17:06.369836+0000","last_fullsized":"2023-08-27T11:17:06.369836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607644+0000","last_change":"2023-08-27T11:17:05.377806+0000","last_active":"2023-08-27T11:17:06.607644+0000","last_peered":"2023-08-27T11:17:06.607644+0000","last_clean":"2023-08-27T11:17:06.607644+0000","last_became_active":"2023-08-27T11:17:05.377546+0000","last_became_peered":"2023-08-27T11:17:05.377546+0000","last_unstale":"2023-08-27T11:17:06.607644+0000","last_undegraded":"2023-08-27T11:17:06.607644+0000","last_fullsized":"2023-08-27T11:17:06.607644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:28.711930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369347+0000","last_change":"2023-08-27T11:17:05.371766+0000","last_active":"2023-08-27T11:17:06.369347+0000","last_peered":"2023-08-27T11:17:06.369347+0000","last_clean":"2023-08-27T11:17:06.369347+0000","last_became_active":"2023-08-27T11:17:05.370938+0000","last_became_peered":"2023-08-27T11:17:05.370938+0000","last_unstale":"2023-08-27T11:17:06.369347+0000","last_undegraded":"2023-08-27T11:17:06.369347+0000","last_fullsized":"2023-08-27T11:17:06.369347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:55.731983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203009+0000","last_change":"2023-08-27T11:17:05.383079+0000","last_active":"2023-08-27T11:17:07.203009+0000","last_peered":"2023-08-27T11:17:07.203009+0000","last_clean":"2023-08-27T11:17:07.203009+0000","last_became_active":"2023-08-27T11:17:05.382810+0000","last_became_peered":"2023-08-27T11:17:05.382810+0000","last_unstale":"2023-08-27T11:17:07.203009+0000","last_undegraded":"2023-08-27T11:17:07.203009+0000","last_fullsized":"2023-08-27T11:17:07.203009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:46.743604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202512+0000","last_change":"2023-08-27T11:17:05.376722+0000","last_active":"2023-08-27T11:17:07.202512+0000","last_peered":"2023-08-27T11:17:07.202512+0000","last_clean":"2023-08-27T11:17:07.202512+0000","last_became_active":"2023-08-27T11:17:05.376399+0000","last_became_peered":"2023-08-27T11:17:05.376399+0000","last_unstale":"2023-08-27T11:17:07.202512+0000","last_undegraded":"2023-08-27T11:17:07.202512+0000","last_fullsized":"2023-08-27T11:17:07.202512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607858+0000","last_change":"2023-08-27T11:17:05.371306+0000","last_active":"2023-08-27T11:17:06.607858+0000","last_peered":"2023-08-27T11:17:06.607858+0000","last_clean":"2023-08-27T11:17:06.607858+0000","last_became_active":"2023-08-27T11:17:05.370962+0000","last_became_peered":"2023-08-27T11:17:05.370962+0000","last_unstale":"2023-08-27T11:17:06.607858+0000","last_undegraded":"2023-08-27T11:17:06.607858+0000","last_fullsized":"2023-08-27T11:17:06.607858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:39.209301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202986+0000","last_change":"2023-08-27T11:17:05.372994+0000","last_active":"2023-08-27T11:17:07.202986+0000","last_peered":"2023-08-27T11:17:07.202986+0000","last_clean":"2023-08-27T11:17:07.202986+0000","last_became_active":"2023-08-27T11:17:05.372278+0000","last_became_peered":"2023-08-27T11:17:05.372278+0000","last_unstale":"2023-08-27T11:17:07.202986+0000","last_undegraded":"2023-08-27T11:17:07.202986+0000","last_fullsized":"2023-08-27T11:17:07.202986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369607+0000","last_change":"2023-08-27T11:17:05.376929+0000","last_active":"2023-08-27T11:17:06.369607+0000","last_peered":"2023-08-27T11:17:06.369607+0000","last_clean":"2023-08-27T11:17:06.369607+0000","last_became_active":"2023-08-27T11:17:05.376592+0000","last_became_peered":"2023-08-27T11:17:05.376592+0000","last_unstale":"2023-08-27T11:17:06.369607+0000","last_undegraded":"2023-08-27T11:17:06.369607+0000","last_fullsized":"2023-08-27T11:17:06.369607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:24:05.167634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190152+0000","last_change":"2023-08-27T11:17:05.377608+0000","last_active":"2023-08-27T11:17:07.190152+0000","last_peered":"2023-08-27T11:17:07.190152+0000","last_clean":"2023-08-27T11:17:07.190152+0000","last_became_active":"2023-08-27T11:17:05.376987+0000","last_became_peered":"2023-08-27T11:17:05.376987+0000","last_unstale":"2023-08-27T11:17:07.190152+0000","last_undegraded":"2023-08-27T11:17:07.190152+0000","last_fullsized":"2023-08-27T11:17:07.190152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:30:54.430091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.189991+0000","last_change":"2023-08-27T11:17:05.377602+0000","last_active":"2023-08-27T11:17:07.189991+0000","last_peered":"2023-08-27T11:17:07.189991+0000","last_clean":"2023-08-27T11:17:07.189991+0000","last_became_active":"2023-08-27T11:17:05.377023+0000","last_became_peered":"2023-08-27T11:17:05.377023+0000","last_unstale":"2023-08-27T11:17:07.189991+0000","last_undegraded":"2023-08-27T11:17:07.189991+0000","last_fullsized":"2023-08-27T11:17:07.189991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:18:00.607553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202965+0000","last_change":"2023-08-27T11:17:05.376491+0000","last_active":"2023-08-27T11:17:07.202965+0000","last_peered":"2023-08-27T11:17:07.202965+0000","last_clean":"2023-08-27T11:17:07.202965+0000","last_became_active":"2023-08-27T11:17:05.376149+0000","last_became_peered":"2023-08-27T11:17:05.376149+0000","last_unstale":"2023-08-27T11:17:07.202965+0000","last_undegraded":"2023-08-27T11:17:07.202965+0000","last_fullsized":"2023-08-27T11:17:07.202965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:43.313210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203191+0000","last_change":"2023-08-27T11:17:05.387318+0000","last_active":"2023-08-27T11:17:07.203191+0000","last_peered":"2023-08-27T11:17:07.203191+0000","last_clean":"2023-08-27T11:17:07.203191+0000","last_became_active":"2023-08-27T11:17:05.387002+0000","last_became_peered":"2023-08-27T11:17:05.387002+0000","last_unstale":"2023-08-27T11:17:07.203191+0000","last_undegraded":"2023-08-27T11:17:07.203191+0000","last_fullsized":"2023-08-27T11:17:07.203191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:13.528078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368880+0000","last_change":"2023-08-27T11:17:05.372919+0000","last_active":"2023-08-27T11:17:06.368880+0000","last_peered":"2023-08-27T11:17:06.368880+0000","last_clean":"2023-08-27T11:17:06.368880+0000","last_became_active":"2023-08-27T11:17:05.371149+0000","last_became_peered":"2023-08-27T11:17:05.371149+0000","last_unstale":"2023-08-27T11:17:06.368880+0000","last_undegraded":"2023-08-27T11:17:06.368880+0000","last_fullsized":"2023-08-27T11:17:06.368880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:46.743604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607880+0000","last_change":"2023-08-27T11:17:05.371456+0000","last_active":"2023-08-27T11:17:06.607880+0000","last_peered":"2023-08-27T11:17:06.607880+0000","last_clean":"2023-08-27T11:17:06.607880+0000","last_became_active":"2023-08-27T11:17:05.371193+0000","last_became_peered":"2023-08-27T11:17:05.371193+0000","last_unstale":"2023-08-27T11:17:06.607880+0000","last_undegraded":"2023-08-27T11:17:06.607880+0000","last_fullsized":"2023-08-27T11:17:06.607880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369084+0000","last_change":"2023-08-27T11:17:05.373957+0000","last_active":"2023-08-27T11:17:06.369084+0000","last_peered":"2023-08-27T11:17:06.369084+0000","last_clean":"2023-08-27T11:17:06.369084+0000","last_became_active":"2023-08-27T11:17:05.373507+0000","last_became_peered":"2023-08-27T11:17:05.373507+0000","last_unstale":"2023-08-27T11:17:06.369084+0000","last_undegraded":"2023-08-27T11:17:06.369084+0000","last_fullsized":"2023-08-27T11:17:06.369084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:45.136551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190029+0000","last_change":"2023-08-27T11:17:05.378874+0000","last_active":"2023-08-27T11:17:07.190029+0000","last_peered":"2023-08-27T11:17:07.190029+0000","last_clean":"2023-08-27T11:17:07.190029+0000","last_became_active":"2023-08-27T11:17:05.378585+0000","last_became_peered":"2023-08-27T11:17:05.378585+0000","last_unstale":"2023-08-27T11:17:07.190029+0000","last_undegraded":"2023-08-27T11:17:07.190029+0000","last_fullsized":"2023-08-27T11:17:07.190029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:19:23.730989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368860+0000","last_change":"2023-08-27T11:17:05.377656+0000","last_active":"2023-08-27T11:17:06.368860+0000","last_peered":"2023-08-27T11:17:06.368860+0000","last_clean":"2023-08-27T11:17:06.368860+0000","last_became_active":"2023-08-27T11:17:05.377366+0000","last_became_peered":"2023-08-27T11:17:05.377366+0000","last_unstale":"2023-08-27T11:17:06.368860+0000","last_undegraded":"2023-08-27T11:17:06.368860+0000","last_fullsized":"2023-08-27T11:17:06.368860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202861+0000","last_change":"2023-08-27T11:17:05.379425+0000","last_active":"2023-08-27T11:17:07.202861+0000","last_peered":"2023-08-27T11:17:07.202861+0000","last_clean":"2023-08-27T11:17:07.202861+0000","last_became_active":"2023-08-27T11:17:05.379084+0000","last_became_peered":"2023-08-27T11:17:05.379084+0000","last_unstale":"2023-08-27T11:17:07.202861+0000","last_undegraded":"2023-08-27T11:17:07.202861+0000","last_fullsized":"2023-08-27T11:17:07.202861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607602+0000","last_change":"2023-08-27T11:17:05.381421+0000","last_active":"2023-08-27T11:17:06.607602+0000","last_peered":"2023-08-27T11:17:06.607602+0000","last_clean":"2023-08-27T11:17:06.607602+0000","last_became_active":"2023-08-27T11:17:05.380452+0000","last_became_peered":"2023-08-27T11:17:05.380452+0000","last_unstale":"2023-08-27T11:17:06.607602+0000","last_undegraded":"2023-08-27T11:17:06.607602+0000","last_fullsized":"2023-08-27T11:17:06.607602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202657+0000","last_change":"2023-08-27T11:17:05.378132+0000","last_active":"2023-08-27T11:17:07.202657+0000","last_peered":"2023-08-27T11:17:07.202657+0000","last_clean":"2023-08-27T11:17:07.202657+0000","last_became_active":"2023-08-27T11:17:05.376190+0000","last_became_peered":"2023-08-27T11:17:05.376190+0000","last_unstale":"2023-08-27T11:17:07.202657+0000","last_undegraded":"2023-08-27T11:17:07.202657+0000","last_fullsized":"2023-08-27T11:17:07.202657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203004+0000","last_change":"2023-08-27T11:17:05.378821+0000","last_active":"2023-08-27T11:17:07.203004+0000","last_peered":"2023-08-27T11:17:07.203004+0000","last_clean":"2023-08-27T11:17:07.203004+0000","last_became_active":"2023-08-27T11:17:05.378520+0000","last_became_peered":"2023-08-27T11:17:05.378520+0000","last_unstale":"2023-08-27T11:17:07.203004+0000","last_undegraded":"2023-08-27T11:17:07.203004+0000","last_fullsized":"2023-08-27T11:17:07.203004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:54.962968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367698+0000","last_change":"2023-08-27T11:17:05.379672+0000","last_active":"2023-08-27T11:17:06.367698+0000","last_peered":"2023-08-27T11:17:06.367698+0000","last_clean":"2023-08-27T11:17:06.367698+0000","last_became_active":"2023-08-27T11:17:05.379399+0000","last_became_peered":"2023-08-27T11:17:05.379399+0000","last_unstale":"2023-08-27T11:17:06.367698+0000","last_undegraded":"2023-08-27T11:17:06.367698+0000","last_fullsized":"2023-08-27T11:17:06.367698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:15.192349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367765+0000","last_change":"2023-08-27T11:17:05.376638+0000","last_active":"2023-08-27T11:17:06.367765+0000","last_peered":"2023-08-27T11:17:06.367765+0000","last_clean":"2023-08-27T11:17:06.367765+0000","last_became_active":"2023-08-27T11:17:05.376274+0000","last_became_peered":"2023-08-27T11:17:05.376274+0000","last_unstale":"2023-08-27T11:17:06.367765+0000","last_undegraded":"2023-08-27T11:17:06.367765+0000","last_fullsized":"2023-08-27T11:17:06.367765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202811+0000","last_change":"2023-08-27T11:17:05.382137+0000","last_active":"2023-08-27T11:17:07.202811+0000","last_peered":"2023-08-27T11:17:07.202811+0000","last_clean":"2023-08-27T11:17:07.202811+0000","last_became_active":"2023-08-27T11:17:05.381664+0000","last_became_peered":"2023-08-27T11:17:05.381664+0000","last_unstale":"2023-08-27T11:17:07.202811+0000","last_undegraded":"2023-08-27T11:17:07.202811+0000","last_fullsized":"2023-08-27T11:17:07.202811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369199+0000","last_change":"2023-08-27T11:17:04.376604+0000","last_active":"2023-08-27T11:17:06.369199+0000","last_peered":"2023-08-27T11:17:06.369199+0000","last_clean":"2023-08-27T11:17:06.369199+0000","last_became_active":"2023-08-27T11:17:04.376288+0000","last_became_peered":"2023-08-27T11:17:04.376288+0000","last_unstale":"2023-08-27T11:17:06.369199+0000","last_undegraded":"2023-08-27T11:17:06.369199+0000","last_fullsized":"2023-08-27T11:17:06.369199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:08:24.626675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203128+0000","last_change":"2023-08-27T11:17:05.382097+0000","last_active":"2023-08-27T11:17:07.203128+0000","last_peered":"2023-08-27T11:17:07.203128+0000","last_clean":"2023-08-27T11:17:07.203128+0000","last_became_active":"2023-08-27T11:17:05.381829+0000","last_became_peered":"2023-08-27T11:17:05.381829+0000","last_unstale":"2023-08-27T11:17:07.203128+0000","last_undegraded":"2023-08-27T11:17:07.203128+0000","last_fullsized":"2023-08-27T11:17:07.203128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:45.136551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367598+0000","last_change":"2023-08-27T11:17:04.379398+0000","last_active":"2023-08-27T11:17:06.367598+0000","last_peered":"2023-08-27T11:17:06.367598+0000","last_clean":"2023-08-27T11:17:06.367598+0000","last_became_active":"2023-08-27T11:17:04.379109+0000","last_became_peered":"2023-08-27T11:17:04.379109+0000","last_unstale":"2023-08-27T11:17:06.367598+0000","last_undegraded":"2023-08-27T11:17:06.367598+0000","last_fullsized":"2023-08-27T11:17:06.367598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:45.809057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607933+0000","last_change":"2023-08-27T11:17:05.379671+0000","last_active":"2023-08-27T11:17:06.607933+0000","last_peered":"2023-08-27T11:17:06.607933+0000","last_clean":"2023-08-27T11:17:06.607933+0000","last_became_active":"2023-08-27T11:17:05.379358+0000","last_became_peered":"2023-08-27T11:17:05.379358+0000","last_unstale":"2023-08-27T11:17:06.607933+0000","last_undegraded":"2023-08-27T11:17:06.607933+0000","last_fullsized":"2023-08-27T11:17:06.607933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:54.962968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202719+0000","last_change":"2023-08-27T11:17:04.377528+0000","last_active":"2023-08-27T11:17:07.202719+0000","last_peered":"2023-08-27T11:17:07.202719+0000","last_clean":"2023-08-27T11:17:07.202719+0000","last_became_active":"2023-08-27T11:17:04.377219+0000","last_became_peered":"2023-08-27T11:17:04.377219+0000","last_unstale":"2023-08-27T11:17:07.202719+0000","last_undegraded":"2023-08-27T11:17:07.202719+0000","last_fullsized":"2023-08-27T11:17:07.202719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:38.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369530+0000","last_change":"2023-08-27T11:17:05.372053+0000","last_active":"2023-08-27T11:17:06.369530+0000","last_peered":"2023-08-27T11:17:06.369530+0000","last_clean":"2023-08-27T11:17:06.369530+0000","last_became_active":"2023-08-27T11:17:05.371769+0000","last_became_peered":"2023-08-27T11:17:05.371769+0000","last_unstale":"2023-08-27T11:17:06.369530+0000","last_undegraded":"2023-08-27T11:17:06.369530+0000","last_fullsized":"2023-08-27T11:17:06.369530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607988+0000","last_change":"2023-08-27T11:17:04.378264+0000","last_active":"2023-08-27T11:17:06.607988+0000","last_peered":"2023-08-27T11:17:06.607988+0000","last_clean":"2023-08-27T11:17:06.607988+0000","last_became_active":"2023-08-27T11:17:04.377919+0000","last_became_peered":"2023-08-27T11:17:04.377919+0000","last_unstale":"2023-08-27T11:17:06.607988+0000","last_undegraded":"2023-08-27T11:17:06.607988+0000","last_fullsized":"2023-08-27T11:17:06.607988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:53.143119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202657+0000","last_change":"2023-08-27T11:17:05.383385+0000","last_active":"2023-08-27T11:17:07.202657+0000","last_peered":"2023-08-27T11:17:07.202657+0000","last_clean":"2023-08-27T11:17:07.202657+0000","last_became_active":"2023-08-27T11:17:05.382932+0000","last_became_peered":"2023-08-27T11:17:05.382932+0000","last_unstale":"2023-08-27T11:17:07.202657+0000","last_undegraded":"2023-08-27T11:17:07.202657+0000","last_fullsized":"2023-08-27T11:17:07.202657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202798+0000","last_change":"2023-08-27T11:17:04.373528+0000","last_active":"2023-08-27T11:17:07.202798+0000","last_peered":"2023-08-27T11:17:07.202798+0000","last_clean":"2023-08-27T11:17:07.202798+0000","last_became_active":"2023-08-27T11:17:04.373266+0000","last_became_peered":"2023-08-27T11:17:04.373266+0000","last_unstale":"2023-08-27T11:17:07.202798+0000","last_undegraded":"2023-08-27T11:17:07.202798+0000","last_fullsized":"2023-08-27T11:17:07.202798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:14.186836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202805+0000","last_change":"2023-08-27T11:17:05.383919+0000","last_active":"2023-08-27T11:17:07.202805+0000","last_peered":"2023-08-27T11:17:07.202805+0000","last_clean":"2023-08-27T11:17:07.202805+0000","last_became_active":"2023-08-27T11:17:05.383319+0000","last_became_peered":"2023-08-27T11:17:05.383319+0000","last_unstale":"2023-08-27T11:17:07.202805+0000","last_undegraded":"2023-08-27T11:17:07.202805+0000","last_fullsized":"2023-08-27T11:17:07.202805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:42.812467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203294+0000","last_change":"2023-08-27T11:17:04.379878+0000","last_active":"2023-08-27T11:17:07.203294+0000","last_peered":"2023-08-27T11:17:07.203294+0000","last_clean":"2023-08-27T11:17:07.203294+0000","last_became_active":"2023-08-27T11:17:04.379586+0000","last_became_peered":"2023-08-27T11:17:04.379586+0000","last_unstale":"2023-08-27T11:17:07.203294+0000","last_undegraded":"2023-08-27T11:17:07.203294+0000","last_fullsized":"2023-08-27T11:17:07.203294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:02.601133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367611+0000","last_change":"2023-08-27T11:17:05.385687+0000","last_active":"2023-08-27T11:17:06.367611+0000","last_peered":"2023-08-27T11:17:06.367611+0000","last_clean":"2023-08-27T11:17:06.367611+0000","last_became_active":"2023-08-27T11:17:05.385422+0000","last_became_peered":"2023-08-27T11:17:05.385422+0000","last_unstale":"2023-08-27T11:17:06.367611+0000","last_undegraded":"2023-08-27T11:17:06.367611+0000","last_fullsized":"2023-08-27T11:17:06.367611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:31.508513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369381+0000","last_change":"2023-08-27T11:17:04.372137+0000","last_active":"2023-08-27T11:17:06.369381+0000","last_peered":"2023-08-27T11:17:06.369381+0000","last_clean":"2023-08-27T11:17:06.369381+0000","last_became_active":"2023-08-27T11:17:04.371246+0000","last_became_peered":"2023-08-27T11:17:04.371246+0000","last_unstale":"2023-08-27T11:17:06.369381+0000","last_undegraded":"2023-08-27T11:17:06.369381+0000","last_fullsized":"2023-08-27T11:17:06.369381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:27:15.596770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369513+0000","last_change":"2023-08-27T11:17:05.375967+0000","last_active":"2023-08-27T11:17:06.369513+0000","last_peered":"2023-08-27T11:17:06.369513+0000","last_clean":"2023-08-27T11:17:06.369513+0000","last_became_active":"2023-08-27T11:17:05.375704+0000","last_became_peered":"2023-08-27T11:17:05.375704+0000","last_unstale":"2023-08-27T11:17:06.369513+0000","last_undegraded":"2023-08-27T11:17:06.369513+0000","last_fullsized":"2023-08-27T11:17:06.369513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:55:02.014817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202803+0000","last_change":"2023-08-27T11:17:04.374228+0000","last_active":"2023-08-27T11:17:07.202803+0000","last_peered":"2023-08-27T11:17:07.202803+0000","last_clean":"2023-08-27T11:17:07.202803+0000","last_became_active":"2023-08-27T11:17:04.373855+0000","last_became_peered":"2023-08-27T11:17:04.373855+0000","last_unstale":"2023-08-27T11:17:07.202803+0000","last_undegraded":"2023-08-27T11:17:07.202803+0000","last_fullsized":"2023-08-27T11:17:07.202803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:38.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202908+0000","last_change":"2023-08-27T11:17:05.382048+0000","last_active":"2023-08-27T11:17:07.202908+0000","last_peered":"2023-08-27T11:17:07.202908+0000","last_clean":"2023-08-27T11:17:07.202908+0000","last_became_active":"2023-08-27T11:17:05.381804+0000","last_became_peered":"2023-08-27T11:17:05.381804+0000","last_unstale":"2023-08-27T11:17:07.202908+0000","last_undegraded":"2023-08-27T11:17:07.202908+0000","last_fullsized":"2023-08-27T11:17:07.202908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:55.731983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607767+0000","last_change":"2023-08-27T11:17:04.376663+0000","last_active":"2023-08-27T11:17:06.607767+0000","last_peered":"2023-08-27T11:17:06.607767+0000","last_clean":"2023-08-27T11:17:06.607767+0000","last_became_active":"2023-08-27T11:17:04.376347+0000","last_became_peered":"2023-08-27T11:17:04.376347+0000","last_unstale":"2023-08-27T11:17:06.607767+0000","last_undegraded":"2023-08-27T11:17:06.607767+0000","last_fullsized":"2023-08-27T11:17:06.607767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:14.186836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203533+0000","last_change":"2023-08-27T11:17:05.382733+0000","last_active":"2023-08-27T11:17:07.203533+0000","last_peered":"2023-08-27T11:17:07.203533+0000","last_clean":"2023-08-27T11:17:07.203533+0000","last_became_active":"2023-08-27T11:17:05.382358+0000","last_became_peered":"2023-08-27T11:17:05.382358+0000","last_unstale":"2023-08-27T11:17:07.203533+0000","last_undegraded":"2023-08-27T11:17:07.203533+0000","last_fullsized":"2023-08-27T11:17:07.203533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:42.812467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203143+0000","last_change":"2023-08-27T11:17:04.372997+0000","last_active":"2023-08-27T11:17:07.203143+0000","last_peered":"2023-08-27T11:17:07.203143+0000","last_clean":"2023-08-27T11:17:07.203143+0000","last_became_active":"2023-08-27T11:17:04.372719+0000","last_became_peered":"2023-08-27T11:17:04.372719+0000","last_unstale":"2023-08-27T11:17:07.203143+0000","last_undegraded":"2023-08-27T11:17:07.203143+0000","last_fullsized":"2023-08-27T11:17:07.203143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:54.726470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607805+0000","last_change":"2023-08-27T11:17:05.379397+0000","last_active":"2023-08-27T11:17:06.607805+0000","last_peered":"2023-08-27T11:17:06.607805+0000","last_clean":"2023-08-27T11:17:06.607805+0000","last_became_active":"2023-08-27T11:17:05.377874+0000","last_became_peered":"2023-08-27T11:17:05.377874+0000","last_unstale":"2023-08-27T11:17:06.607805+0000","last_undegraded":"2023-08-27T11:17:06.607805+0000","last_fullsized":"2023-08-27T11:17:06.607805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368437+0000","last_change":"2023-08-27T11:17:04.376949+0000","last_active":"2023-08-27T11:17:06.368437+0000","last_peered":"2023-08-27T11:17:06.368437+0000","last_clean":"2023-08-27T11:17:06.368437+0000","last_became_active":"2023-08-27T11:17:04.376645+0000","last_became_peered":"2023-08-27T11:17:04.376645+0000","last_unstale":"2023-08-27T11:17:06.368437+0000","last_undegraded":"2023-08-27T11:17:06.368437+0000","last_fullsized":"2023-08-27T11:17:06.368437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:30.937446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203216+0000","last_change":"2023-08-27T11:17:05.378949+0000","last_active":"2023-08-27T11:17:07.203216+0000","last_peered":"2023-08-27T11:17:07.203216+0000","last_clean":"2023-08-27T11:17:07.203216+0000","last_became_active":"2023-08-27T11:17:05.378649+0000","last_became_peered":"2023-08-27T11:17:05.378649+0000","last_unstale":"2023-08-27T11:17:07.203216+0000","last_undegraded":"2023-08-27T11:17:07.203216+0000","last_fullsized":"2023-08-27T11:17:07.203216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:43:40.338423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203379+0000","last_change":"2023-08-27T11:17:04.374131+0000","last_active":"2023-08-27T11:17:07.203379+0000","last_peered":"2023-08-27T11:17:07.203379+0000","last_clean":"2023-08-27T11:17:07.203379+0000","last_became_active":"2023-08-27T11:17:04.373809+0000","last_became_peered":"2023-08-27T11:17:04.373809+0000","last_unstale":"2023-08-27T11:17:07.203379+0000","last_undegraded":"2023-08-27T11:17:07.203379+0000","last_fullsized":"2023-08-27T11:17:07.203379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:43:39.332910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202567+0000","last_change":"2023-08-27T11:17:05.379390+0000","last_active":"2023-08-27T11:17:07.202567+0000","last_peered":"2023-08-27T11:17:07.202567+0000","last_clean":"2023-08-27T11:17:07.202567+0000","last_became_active":"2023-08-27T11:17:05.379131+0000","last_became_peered":"2023-08-27T11:17:05.379131+0000","last_unstale":"2023-08-27T11:17:07.202567+0000","last_undegraded":"2023-08-27T11:17:07.202567+0000","last_fullsized":"2023-08-27T11:17:07.202567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:28.711930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203264+0000","last_change":"2023-08-27T11:17:04.374694+0000","last_active":"2023-08-27T11:17:07.203264+0000","last_peered":"2023-08-27T11:17:07.203264+0000","last_clean":"2023-08-27T11:17:07.203264+0000","last_became_active":"2023-08-27T11:17:04.373836+0000","last_became_peered":"2023-08-27T11:17:04.373836+0000","last_unstale":"2023-08-27T11:17:07.203264+0000","last_undegraded":"2023-08-27T11:17:07.203264+0000","last_fullsized":"2023-08-27T11:17:07.203264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:44.131038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368972+0000","last_change":"2023-08-27T11:17:05.380858+0000","last_active":"2023-08-27T11:17:06.368972+0000","last_peered":"2023-08-27T11:17:06.368972+0000","last_clean":"2023-08-27T11:17:06.368972+0000","last_became_active":"2023-08-27T11:17:05.380573+0000","last_became_peered":"2023-08-27T11:17:05.380573+0000","last_unstale":"2023-08-27T11:17:06.368972+0000","last_undegraded":"2023-08-27T11:17:06.368972+0000","last_fullsized":"2023-08-27T11:17:06.368972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:42.999614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203215+0000","last_change":"2023-08-27T11:17:04.374901+0000","last_active":"2023-08-27T11:17:07.203215+0000","last_peered":"2023-08-27T11:17:07.203215+0000","last_clean":"2023-08-27T11:17:07.203215+0000","last_became_active":"2023-08-27T11:17:04.374186+0000","last_became_peered":"2023-08-27T11:17:04.374186+0000","last_unstale":"2023-08-27T11:17:07.203215+0000","last_undegraded":"2023-08-27T11:17:07.203215+0000","last_fullsized":"2023-08-27T11:17:07.203215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:47.391583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369664+0000","last_change":"2023-08-27T11:17:05.380802+0000","last_active":"2023-08-27T11:17:06.369664+0000","last_peered":"2023-08-27T11:17:06.369664+0000","last_clean":"2023-08-27T11:17:06.369664+0000","last_became_active":"2023-08-27T11:17:05.380537+0000","last_became_peered":"2023-08-27T11:17:05.380537+0000","last_unstale":"2023-08-27T11:17:06.369664+0000","last_undegraded":"2023-08-27T11:17:06.369664+0000","last_fullsized":"2023-08-27T11:17:06.369664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203202+0000","last_change":"2023-08-27T11:17:04.379623+0000","last_active":"2023-08-27T11:17:07.203202+0000","last_peered":"2023-08-27T11:17:07.203202+0000","last_clean":"2023-08-27T11:17:07.203202+0000","last_became_active":"2023-08-27T11:17:04.379342+0000","last_became_peered":"2023-08-27T11:17:04.379342+0000","last_unstale":"2023-08-27T11:17:07.203202+0000","last_undegraded":"2023-08-27T11:17:07.203202+0000","last_fullsized":"2023-08-27T11:17:07.203202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:38.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607824+0000","last_change":"2023-08-27T11:17:05.375956+0000","last_active":"2023-08-27T11:17:06.607824+0000","last_peered":"2023-08-27T11:17:06.607824+0000","last_clean":"2023-08-27T11:17:06.607824+0000","last_became_active":"2023-08-27T11:17:05.375040+0000","last_became_peered":"2023-08-27T11:17:05.375040+0000","last_unstale":"2023-08-27T11:17:06.607824+0000","last_undegraded":"2023-08-27T11:17:06.607824+0000","last_fullsized":"2023-08-27T11:17:06.607824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:33.634108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202873+0000","last_change":"2023-08-27T11:17:04.379633+0000","last_active":"2023-08-27T11:17:07.202873+0000","last_peered":"2023-08-27T11:17:07.202873+0000","last_clean":"2023-08-27T11:17:07.202873+0000","last_became_active":"2023-08-27T11:17:04.379306+0000","last_became_peered":"2023-08-27T11:17:04.379306+0000","last_unstale":"2023-08-27T11:17:07.202873+0000","last_undegraded":"2023-08-27T11:17:07.202873+0000","last_fullsized":"2023-08-27T11:17:07.202873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:53.143119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369174+0000","last_change":"2023-08-27T11:17:05.374205+0000","last_active":"2023-08-27T11:17:06.369174+0000","last_peered":"2023-08-27T11:17:06.369174+0000","last_clean":"2023-08-27T11:17:06.369174+0000","last_became_active":"2023-08-27T11:17:05.373675+0000","last_became_peered":"2023-08-27T11:17:05.373675+0000","last_unstale":"2023-08-27T11:17:06.369174+0000","last_undegraded":"2023-08-27T11:17:06.369174+0000","last_fullsized":"2023-08-27T11:17:06.369174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:43:40.338423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369599+0000","last_change":"2023-08-27T11:17:04.371348+0000","last_active":"2023-08-27T11:17:06.369599+0000","last_peered":"2023-08-27T11:17:06.369599+0000","last_clean":"2023-08-27T11:17:06.369599+0000","last_became_active":"2023-08-27T11:17:04.370929+0000","last_became_peered":"2023-08-27T11:17:04.370929+0000","last_unstale":"2023-08-27T11:17:06.369599+0000","last_undegraded":"2023-08-27T11:17:06.369599+0000","last_fullsized":"2023-08-27T11:17:06.369599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:27.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369296+0000","last_change":"2023-08-27T11:17:05.373711+0000","last_active":"2023-08-27T11:17:06.369296+0000","last_peered":"2023-08-27T11:17:06.369296+0000","last_clean":"2023-08-27T11:17:06.369296+0000","last_became_active":"2023-08-27T11:17:05.373392+0000","last_became_peered":"2023-08-27T11:17:05.373392+0000","last_unstale":"2023-08-27T11:17:06.369296+0000","last_undegraded":"2023-08-27T11:17:06.369296+0000","last_fullsized":"2023-08-27T11:17:06.369296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:35.735781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369479+0000","last_change":"2023-08-27T11:17:04.373259+0000","last_active":"2023-08-27T11:17:06.369479+0000","last_peered":"2023-08-27T11:17:06.369479+0000","last_clean":"2023-08-27T11:17:06.369479+0000","last_became_active":"2023-08-27T11:17:04.372983+0000","last_became_peered":"2023-08-27T11:17:04.372983+0000","last_unstale":"2023-08-27T11:17:06.369479+0000","last_undegraded":"2023-08-27T11:17:06.369479+0000","last_fullsized":"2023-08-27T11:17:06.369479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:36.570705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369051+0000","last_change":"2023-08-27T11:17:05.376729+0000","last_active":"2023-08-27T11:17:06.369051+0000","last_peered":"2023-08-27T11:17:06.369051+0000","last_clean":"2023-08-27T11:17:06.369051+0000","last_became_active":"2023-08-27T11:17:05.376443+0000","last_became_peered":"2023-08-27T11:17:05.376443+0000","last_unstale":"2023-08-27T11:17:06.369051+0000","last_undegraded":"2023-08-27T11:17:06.369051+0000","last_fullsized":"2023-08-27T11:17:06.369051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:48.397096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202559+0000","last_change":"2023-08-27T11:17:04.377051+0000","last_active":"2023-08-27T11:17:07.202559+0000","last_peered":"2023-08-27T11:17:07.202559+0000","last_clean":"2023-08-27T11:17:07.202559+0000","last_became_active":"2023-08-27T11:17:04.376700+0000","last_became_peered":"2023-08-27T11:17:04.376700+0000","last_unstale":"2023-08-27T11:17:07.202559+0000","last_undegraded":"2023-08-27T11:17:07.202559+0000","last_fullsized":"2023-08-27T11:17:07.202559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:42.307697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367824+0000","last_change":"2023-08-27T11:17:05.383341+0000","last_active":"2023-08-27T11:17:06.367824+0000","last_peered":"2023-08-27T11:17:06.367824+0000","last_clean":"2023-08-27T11:17:06.367824+0000","last_became_active":"2023-08-27T11:17:05.379929+0000","last_became_peered":"2023-08-27T11:17:05.379929+0000","last_unstale":"2023-08-27T11:17:06.367824+0000","last_undegraded":"2023-08-27T11:17:06.367824+0000","last_fullsized":"2023-08-27T11:17:06.367824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203147+0000","last_change":"2023-08-27T11:17:04.378623+0000","last_active":"2023-08-27T11:17:07.203147+0000","last_peered":"2023-08-27T11:17:07.203147+0000","last_clean":"2023-08-27T11:17:07.203147+0000","last_became_active":"2023-08-27T11:17:04.378327+0000","last_became_peered":"2023-08-27T11:17:04.378327+0000","last_unstale":"2023-08-27T11:17:07.203147+0000","last_undegraded":"2023-08-27T11:17:07.203147+0000","last_fullsized":"2023-08-27T11:17:07.203147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:02.601133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368983+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:17:06.368983+0000","last_peered":"2023-08-27T11:17:06.368983+0000","last_clean":"2023-08-27T11:17:06.368983+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:17:06.368983+0000","last_undegraded":"2023-08-27T11:17:06.368983+0000","last_fullsized":"2023-08-27T11:17:06.368983+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:07.481452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202508+0000","last_change":"2023-08-27T11:17:05.383215+0000","last_active":"2023-08-27T11:17:07.202508+0000","last_peered":"2023-08-27T11:17:07.202508+0000","last_clean":"2023-08-27T11:17:07.202508+0000","last_became_active":"2023-08-27T11:17:05.382502+0000","last_became_peered":"2023-08-27T11:17:05.382502+0000","last_unstale":"2023-08-27T11:17:07.202508+0000","last_undegraded":"2023-08-27T11:17:07.202508+0000","last_fullsized":"2023-08-27T11:17:07.202508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203019+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:17:07.203019+0000","last_peered":"2023-08-27T11:17:07.203019+0000","last_clean":"2023-08-27T11:17:07.203019+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:17:07.203019+0000","last_undegraded":"2023-08-27T11:17:07.203019+0000","last_fullsized":"2023-08-27T11:17:07.203019+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:42.350470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.290563+0000","last_change":"2023-08-27T11:17:04.380486+0000","last_active":"2023-08-27T11:17:07.290563+0000","last_peered":"2023-08-27T11:17:07.290563+0000","last_clean":"2023-08-27T11:17:07.290563+0000","last_became_active":"2023-08-27T11:17:04.376679+0000","last_became_peered":"2023-08-27T11:17:04.376679+0000","last_unstale":"2023-08-27T11:17:07.290563+0000","last_undegraded":"2023-08-27T11:17:07.290563+0000","last_fullsized":"2023-08-27T11:17:07.290563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:41.806953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369152+0000","last_change":"2023-08-27T11:17:05.370457+0000","last_active":"2023-08-27T11:17:06.369152+0000","last_peered":"2023-08-27T11:17:06.369152+0000","last_clean":"2023-08-27T11:17:06.369152+0000","last_became_active":"2023-08-27T11:17:05.370040+0000","last_became_peered":"2023-08-27T11:17:05.370040+0000","last_unstale":"2023-08-27T11:17:06.369152+0000","last_undegraded":"2023-08-27T11:17:06.369152+0000","last_fullsized":"2023-08-27T11:17:06.369152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:03.606646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367481+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:17:06.367481+0000","last_peered":"2023-08-27T11:17:06.367481+0000","last_clean":"2023-08-27T11:17:06.367481+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:17:06.367481+0000","last_undegraded":"2023-08-27T11:17:06.367481+0000","last_fullsized":"2023-08-27T11:17:06.367481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:07:15.067358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203119+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:17:07.203119+0000","last_peered":"2023-08-27T11:17:07.203119+0000","last_clean":"2023-08-27T11:17:07.203119+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:17:07.203119+0000","last_undegraded":"2023-08-27T11:17:07.203119+0000","last_fullsized":"2023-08-27T11:17:07.203119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:36.952988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607599+0000","last_change":"2023-08-27T11:17:04.373430+0000","last_active":"2023-08-27T11:17:06.607599+0000","last_peered":"2023-08-27T11:17:06.607599+0000","last_clean":"2023-08-27T11:17:06.607599+0000","last_became_active":"2023-08-27T11:17:04.373098+0000","last_became_peered":"2023-08-27T11:17:04.373098+0000","last_unstale":"2023-08-27T11:17:06.607599+0000","last_undegraded":"2023-08-27T11:17:06.607599+0000","last_fullsized":"2023-08-27T11:17:06.607599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:07:20.108470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607829+0000","last_change":"2023-08-27T11:17:05.380096+0000","last_active":"2023-08-27T11:17:06.607829+0000","last_peered":"2023-08-27T11:17:06.607829+0000","last_clean":"2023-08-27T11:17:06.607829+0000","last_became_active":"2023-08-27T11:17:05.379846+0000","last_became_peered":"2023-08-27T11:17:05.379846+0000","last_unstale":"2023-08-27T11:17:06.607829+0000","last_undegraded":"2023-08-27T11:17:06.607829+0000","last_fullsized":"2023-08-27T11:17:06.607829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202806+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:17:07.202806+0000","last_peered":"2023-08-27T11:17:07.202806+0000","last_clean":"2023-08-27T11:17:07.202806+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:17:07.202806+0000","last_undegraded":"2023-08-27T11:17:07.202806+0000","last_fullsized":"2023-08-27T11:17:07.202806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:48.102006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367401+0000","last_change":"2023-08-27T11:17:04.373552+0000","last_active":"2023-08-27T11:17:06.367401+0000","last_peered":"2023-08-27T11:17:06.367401+0000","last_clean":"2023-08-27T11:17:06.367401+0000","last_became_active":"2023-08-27T11:17:04.373258+0000","last_became_peered":"2023-08-27T11:17:04.373258+0000","last_unstale":"2023-08-27T11:17:06.367401+0000","last_undegraded":"2023-08-27T11:17:06.367401+0000","last_fullsized":"2023-08-27T11:17:06.367401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:30.937446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367737+0000","last_change":"2023-08-27T11:17:05.383636+0000","last_active":"2023-08-27T11:17:06.367737+0000","last_peered":"2023-08-27T11:17:06.367737+0000","last_clean":"2023-08-27T11:17:06.367737+0000","last_became_active":"2023-08-27T11:17:05.383410+0000","last_became_peered":"2023-08-27T11:17:05.383410+0000","last_unstale":"2023-08-27T11:17:06.367737+0000","last_undegraded":"2023-08-27T11:17:06.367737+0000","last_fullsized":"2023-08-27T11:17:06.367737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:43.313210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202834+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:17:07.202834+0000","last_peered":"2023-08-27T11:17:07.202834+0000","last_clean":"2023-08-27T11:17:07.202834+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:17:07.202834+0000","last_undegraded":"2023-08-27T11:17:07.202834+0000","last_fullsized":"2023-08-27T11:17:07.202834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:36.765841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202793+0000","last_change":"2023-08-27T11:17:04.375838+0000","last_active":"2023-08-27T11:17:07.202793+0000","last_peered":"2023-08-27T11:17:07.202793+0000","last_clean":"2023-08-27T11:17:07.202793+0000","last_became_active":"2023-08-27T11:17:04.375536+0000","last_became_peered":"2023-08-27T11:17:04.375536+0000","last_unstale":"2023-08-27T11:17:07.202793+0000","last_undegraded":"2023-08-27T11:17:07.202793+0000","last_fullsized":"2023-08-27T11:17:07.202793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:53.957455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369664+0000","last_change":"2023-08-27T11:17:05.375110+0000","last_active":"2023-08-27T11:17:06.369664+0000","last_peered":"2023-08-27T11:17:06.369664+0000","last_clean":"2023-08-27T11:17:06.369664+0000","last_became_active":"2023-08-27T11:17:05.374844+0000","last_became_peered":"2023-08-27T11:17:05.374844+0000","last_unstale":"2023-08-27T11:17:06.369664+0000","last_undegraded":"2023-08-27T11:17:06.369664+0000","last_fullsized":"2023-08-27T11:17:06.369664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:15.192349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202453+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:17:07.202453+0000","last_peered":"2023-08-27T11:17:07.202453+0000","last_clean":"2023-08-27T11:17:07.202453+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:17:07.202453+0000","last_undegraded":"2023-08-27T11:17:07.202453+0000","last_fullsized":"2023-08-27T11:17:07.202453+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:43.315205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368949+0000","last_change":"2023-08-27T11:17:04.379053+0000","last_active":"2023-08-27T11:17:06.368949+0000","last_peered":"2023-08-27T11:17:06.368949+0000","last_clean":"2023-08-27T11:17:06.368949+0000","last_became_active":"2023-08-27T11:17:04.378731+0000","last_became_peered":"2023-08-27T11:17:04.378731+0000","last_unstale":"2023-08-27T11:17:06.368949+0000","last_undegraded":"2023-08-27T11:17:06.368949+0000","last_fullsized":"2023-08-27T11:17:06.368949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:53.143119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367664+0000","last_change":"2023-08-27T11:17:05.373510+0000","last_active":"2023-08-27T11:17:06.367664+0000","last_peered":"2023-08-27T11:17:06.367664+0000","last_clean":"2023-08-27T11:17:06.367664+0000","last_became_active":"2023-08-27T11:17:05.373229+0000","last_became_peered":"2023-08-27T11:17:05.373229+0000","last_unstale":"2023-08-27T11:17:06.367664+0000","last_undegraded":"2023-08-27T11:17:06.367664+0000","last_fullsized":"2023-08-27T11:17:06.367664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369394+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:17:06.369394+0000","last_peered":"2023-08-27T11:17:06.369394+0000","last_clean":"2023-08-27T11:17:06.369394+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:17:06.369394+0000","last_undegraded":"2023-08-27T11:17:06.369394+0000","last_fullsized":"2023-08-27T11:17:06.369394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:07:15.067358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367592+0000","last_change":"2023-08-27T11:17:04.374506+0000","last_active":"2023-08-27T11:17:06.367592+0000","last_peered":"2023-08-27T11:17:06.367592+0000","last_clean":"2023-08-27T11:17:06.367592+0000","last_became_active":"2023-08-27T11:17:04.374209+0000","last_became_peered":"2023-08-27T11:17:04.374209+0000","last_unstale":"2023-08-27T11:17:06.367592+0000","last_undegraded":"2023-08-27T11:17:06.367592+0000","last_fullsized":"2023-08-27T11:17:06.367592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:27.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607676+0000","last_change":"2023-08-27T11:17:05.369565+0000","last_active":"2023-08-27T11:17:06.607676+0000","last_peered":"2023-08-27T11:17:06.607676+0000","last_clean":"2023-08-27T11:17:06.607676+0000","last_became_active":"2023-08-27T11:17:05.369334+0000","last_became_peered":"2023-08-27T11:17:05.369334+0000","last_unstale":"2023-08-27T11:17:06.607676+0000","last_undegraded":"2023-08-27T11:17:06.607676+0000","last_fullsized":"2023-08-27T11:17:06.607676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:46.825053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369316+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:17:06.369316+0000","last_peered":"2023-08-27T11:17:06.369316+0000","last_clean":"2023-08-27T11:17:06.369316+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:17:06.369316+0000","last_undegraded":"2023-08-27T11:17:06.369316+0000","last_fullsized":"2023-08-27T11:17:06.369316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:25.461887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203500+0000","last_change":"2023-08-27T11:17:04.380000+0000","last_active":"2023-08-27T11:17:07.203500+0000","last_peered":"2023-08-27T11:17:07.203500+0000","last_clean":"2023-08-27T11:17:07.203500+0000","last_became_active":"2023-08-27T11:17:04.379666+0000","last_became_peered":"2023-08-27T11:17:04.379666+0000","last_unstale":"2023-08-27T11:17:07.203500+0000","last_undegraded":"2023-08-27T11:17:07.203500+0000","last_fullsized":"2023-08-27T11:17:07.203500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:53.957455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367721+0000","last_change":"2023-08-27T11:17:05.380929+0000","last_active":"2023-08-27T11:17:06.367721+0000","last_peered":"2023-08-27T11:17:06.367721+0000","last_clean":"2023-08-27T11:17:06.367721+0000","last_became_active":"2023-08-27T11:17:05.380573+0000","last_became_peered":"2023-08-27T11:17:05.380573+0000","last_unstale":"2023-08-27T11:17:06.367721+0000","last_undegraded":"2023-08-27T11:17:06.367721+0000","last_fullsized":"2023-08-27T11:17:06.367721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:15.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607806+0000","last_change":"2023-08-27T11:17:04.377322+0000","last_active":"2023-08-27T11:17:06.607806+0000","last_peered":"2023-08-27T11:17:06.607806+0000","last_clean":"2023-08-27T11:17:06.607806+0000","last_became_active":"2023-08-27T11:17:04.376967+0000","last_became_peered":"2023-08-27T11:17:04.376967+0000","last_unstale":"2023-08-27T11:17:06.607806+0000","last_undegraded":"2023-08-27T11:17:06.607806+0000","last_fullsized":"2023-08-27T11:17:06.607806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:42.307697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.190107+0000","last_change":"2023-08-27T11:17:05.382640+0000","last_active":"2023-08-27T11:17:07.190107+0000","last_peered":"2023-08-27T11:17:07.190107+0000","last_clean":"2023-08-27T11:17:07.190107+0000","last_became_active":"2023-08-27T11:17:05.382377+0000","last_became_peered":"2023-08-27T11:17:05.382377+0000","last_unstale":"2023-08-27T11:17:07.190107+0000","last_undegraded":"2023-08-27T11:17:07.190107+0000","last_fullsized":"2023-08-27T11:17:07.190107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:27:57.016677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203339+0000","last_change":"2023-08-27T11:17:04.372997+0000","last_active":"2023-08-27T11:17:07.203339+0000","last_peered":"2023-08-27T11:17:07.203339+0000","last_clean":"2023-08-27T11:17:07.203339+0000","last_became_active":"2023-08-27T11:17:04.372719+0000","last_became_peered":"2023-08-27T11:17:04.372719+0000","last_unstale":"2023-08-27T11:17:07.203339+0000","last_undegraded":"2023-08-27T11:17:07.203339+0000","last_fullsized":"2023-08-27T11:17:07.203339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:51.710890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368989+0000","last_change":"2023-08-27T11:17:05.374254+0000","last_active":"2023-08-27T11:17:06.368989+0000","last_peered":"2023-08-27T11:17:06.368989+0000","last_clean":"2023-08-27T11:17:06.368989+0000","last_became_active":"2023-08-27T11:17:05.373877+0000","last_became_peered":"2023-08-27T11:17:05.373877+0000","last_unstale":"2023-08-27T11:17:06.368989+0000","last_undegraded":"2023-08-27T11:17:06.368989+0000","last_fullsized":"2023-08-27T11:17:06.368989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:42.812467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203587+0000","last_change":"2023-08-27T11:17:04.372930+0000","last_active":"2023-08-27T11:17:07.203587+0000","last_peered":"2023-08-27T11:17:07.203587+0000","last_clean":"2023-08-27T11:17:07.203587+0000","last_became_active":"2023-08-27T11:17:04.372626+0000","last_became_peered":"2023-08-27T11:17:04.372626+0000","last_unstale":"2023-08-27T11:17:07.203587+0000","last_undegraded":"2023-08-27T11:17:07.203587+0000","last_fullsized":"2023-08-27T11:17:07.203587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:45.738091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.369549+0000","last_change":"2023-08-27T11:17:05.374125+0000","last_active":"2023-08-27T11:17:06.369549+0000","last_peered":"2023-08-27T11:17:06.369549+0000","last_clean":"2023-08-27T11:17:06.369549+0000","last_became_active":"2023-08-27T11:17:05.373856+0000","last_became_peered":"2023-08-27T11:17:05.373856+0000","last_unstale":"2023-08-27T11:17:06.369549+0000","last_undegraded":"2023-08-27T11:17:06.369549+0000","last_fullsized":"2023-08-27T11:17:06.369549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:17:32.516797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.203038+0000","last_change":"2023-08-27T11:17:04.375965+0000","last_active":"2023-08-27T11:17:07.203038+0000","last_peered":"2023-08-27T11:17:07.203038+0000","last_clean":"2023-08-27T11:17:07.203038+0000","last_became_active":"2023-08-27T11:17:04.375687+0000","last_became_peered":"2023-08-27T11:17:04.375687+0000","last_unstale":"2023-08-27T11:17:07.203038+0000","last_undegraded":"2023-08-27T11:17:07.203038+0000","last_fullsized":"2023-08-27T11:17:07.203038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:12.522565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202911+0000","last_change":"2023-08-27T11:17:05.372749+0000","last_active":"2023-08-27T11:17:07.202911+0000","last_peered":"2023-08-27T11:17:07.202911+0000","last_clean":"2023-08-27T11:17:07.202911+0000","last_became_active":"2023-08-27T11:17:05.372494+0000","last_became_peered":"2023-08-27T11:17:05.372494+0000","last_unstale":"2023-08-27T11:17:07.202911+0000","last_undegraded":"2023-08-27T11:17:07.202911+0000","last_fullsized":"2023-08-27T11:17:07.202911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:42.999614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.607966+0000","last_change":"2023-08-27T11:17:04.374287+0000","last_active":"2023-08-27T11:17:06.607966+0000","last_peered":"2023-08-27T11:17:06.607966+0000","last_clean":"2023-08-27T11:17:06.607966+0000","last_became_active":"2023-08-27T11:17:04.374009+0000","last_became_peered":"2023-08-27T11:17:04.374009+0000","last_unstale":"2023-08-27T11:17:06.607966+0000","last_undegraded":"2023-08-27T11:17:06.607966+0000","last_fullsized":"2023-08-27T11:17:06.607966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:03.346030+0000","last_clean_scrub_stamp":"2023-08-27T11:17:03.346030+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:45.738091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368606+0000","last_change":"2023-08-27T11:17:05.369490+0000","last_active":"2023-08-27T11:17:06.368606+0000","last_peered":"2023-08-27T11:17:06.368606+0000","last_clean":"2023-08-27T11:17:06.368606+0000","last_became_active":"2023-08-27T11:17:05.369193+0000","last_became_peered":"2023-08-27T11:17:05.369193+0000","last_unstale":"2023-08-27T11:17:06.368606+0000","last_undegraded":"2023-08-27T11:17:06.368606+0000","last_fullsized":"2023-08-27T11:17:06.368606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:49.361831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368508+0000","last_change":"2023-08-27T11:17:05.376280+0000","last_active":"2023-08-27T11:17:06.368508+0000","last_peered":"2023-08-27T11:17:06.368508+0000","last_clean":"2023-08-27T11:17:06.368508+0000","last_became_active":"2023-08-27T11:17:05.375763+0000","last_became_peered":"2023-08-27T11:17:05.375763+0000","last_unstale":"2023-08-27T11:17:06.368508+0000","last_undegraded":"2023-08-27T11:17:06.368508+0000","last_fullsized":"2023-08-27T11:17:06.368508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:46.814570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202731+0000","last_change":"2023-08-27T11:17:05.384097+0000","last_active":"2023-08-27T11:17:07.202731+0000","last_peered":"2023-08-27T11:17:07.202731+0000","last_clean":"2023-08-27T11:17:07.202731+0000","last_became_active":"2023-08-27T11:17:05.383673+0000","last_became_peered":"2023-08-27T11:17:05.383673+0000","last_unstale":"2023-08-27T11:17:07.202731+0000","last_undegraded":"2023-08-27T11:17:07.202731+0000","last_fullsized":"2023-08-27T11:17:07.202731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:52.716404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.368848+0000","last_change":"2023-08-27T11:17:05.369440+0000","last_active":"2023-08-27T11:17:06.368848+0000","last_peered":"2023-08-27T11:17:06.368848+0000","last_clean":"2023-08-27T11:17:06.368848+0000","last_became_active":"2023-08-27T11:17:05.369113+0000","last_became_peered":"2023-08-27T11:17:05.369113+0000","last_unstale":"2023-08-27T11:17:06.368848+0000","last_undegraded":"2023-08-27T11:17:06.368848+0000","last_fullsized":"2023-08-27T11:17:06.368848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:16:39.209301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:07.202857+0000","last_change":"2023-08-27T11:17:05.381671+0000","last_active":"2023-08-27T11:17:07.202857+0000","last_peered":"2023-08-27T11:17:07.202857+0000","last_clean":"2023-08-27T11:17:07.202857+0000","last_became_active":"2023-08-27T11:17:05.380933+0000","last_became_peered":"2023-08-27T11:17:05.380933+0000","last_unstale":"2023-08-27T11:17:07.202857+0000","last_undegraded":"2023-08-27T11:17:07.202857+0000","last_fullsized":"2023-08-27T11:17:07.202857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:46.743604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-08-27T11:17:06.367534+0000","last_change":"2023-08-27T11:17:05.379944+0000","last_active":"2023-08-27T11:17:06.367534+0000","last_peered":"2023-08-27T11:17:06.367534+0000","last_clean":"2023-08-27T11:17:06.367534+0000","last_became_active":"2023-08-27T11:17:05.379633+0000","last_became_peered":"2023-08-27T11:17:05.379633+0000","last_unstale":"2023-08-27T11:17:06.367534+0000","last_undegraded":"2023-08-27T11:17:06.367534+0000","last_fullsized":"2023-08-27T11:17:06.367534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:17:04.351543+0000","last_clean_scrub_stamp":"2023-08-27T11:17:04.351543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:21.933173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":666624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":74428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":664796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.971}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.928000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.99}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.016999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.925000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.988}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":73649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":74428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":76096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":75080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:17:14.547 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-08-27T11:17:14.547 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-08-27T11:17:14.547 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-08-27T11:17:14.548 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-08-27T11:17:14.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.698+0000 7f632d110700 1 Processor -- start 2023-08-27T11:17:14.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.698+0000 7f632d110700 1 -- start start 2023-08-27T11:17:14.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f632d110700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f63280b1400 0x7f63280cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:14.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f632d110700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6328161ce0 0x7f632806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:14.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f632d110700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6328161850 con 0x7f63280b17e0 2023-08-27T11:17:14.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f632d110700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63280cfc10 con 0x7f63280b1400 2023-08-27T11:17:14.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f632d110700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6328161200 con 0x7f6328161ce0 2023-08-27T11:17:14.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f6326d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f63280b1400 0x7f63280cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:14.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f632659c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f63280b17e0 0x7f632806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49572/0 (socket says 172.21.15.112:49572) 2023-08-27T11:17:14.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f632659c700 1 -- 172.21.15.112:0/1454905062 learned_addr learned my addr 172.21.15.112:0/1454905062 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:14.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f632759e700 1 --2- 172.21.15.112:0/1454905062 >> v2:172.21.15.138:3300/0 conn(0x7f6328161ce0 0x7f632806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:14.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f6326d9d700 1 -- 172.21.15.112:0/1454905062 >> v2:172.21.15.138:3300/0 conn(0x7f6328161ce0 msgr2=0x7f632806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:14.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f6326d9d700 1 --2- 172.21.15.112:0/1454905062 >> v2:172.21.15.138:3300/0 conn(0x7f6328161ce0 0x7f632806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f6326d9d700 1 -- 172.21.15.112:0/1454905062 >> v1:172.21.15.112:6789/0 conn(0x7f63280b17e0 legacy=0x7f632806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f6325d9b700 1 -- 172.21.15.112:0/1454905062 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943262039 0 0) 0x7f6328161850 con 0x7f63280b17e0 2023-08-27T11:17:14.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f6326d9d700 1 -- 172.21.15.112:0/1454905062 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63281614e0 con 0x7f63280b1400 2023-08-27T11:17:14.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f6326d9d700 1 --2- 172.21.15.112:0/1454905062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63280b1400 0x7f63280cf5a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f632806df10 tx=0x7f631c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2349c38af2630d07 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:14.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.699+0000 7f6325d9b700 1 -- 172.21.15.112:0/1454905062 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c011830 con 0x7f63280b1400 2023-08-27T11:17:14.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.700+0000 7f6325d9b700 1 -- 172.21.15.112:0/1454905062 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631c011990 con 0x7f63280b1400 2023-08-27T11:17:14.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.700+0000 7f6325d9b700 1 -- 172.21.15.112:0/1454905062 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c011d20 con 0x7f63280b1400 2023-08-27T11:17:14.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.700+0000 7f632d110700 1 -- 172.21.15.112:0/1454905062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63280b1400 msgr2=0x7f63280cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.700+0000 7f632d110700 1 --2- 172.21.15.112:0/1454905062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63280b1400 0x7f63280cf5a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f632806df10 tx=0x7f631c009b20 comp rx=0 tx=0).stop 2023-08-27T11:17:14.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.700+0000 7f632d110700 1 -- 172.21.15.112:0/1454905062 shutdown_connections 2023-08-27T11:17:14.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.700+0000 7f632d110700 1 --2- 172.21.15.112:0/1454905062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63280b1400 0x7f63280cf5a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.700+0000 7f632d110700 1 --2- 172.21.15.112:0/1454905062 >> v2:172.21.15.138:3300/0 conn(0x7f6328161ce0 0x7f632806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.700+0000 7f632d110700 1 -- 172.21.15.112:0/1454905062 >> 172.21.15.112:0/1454905062 conn(0x7f63280d0000 msgr2=0x7f63280d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:14.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.700+0000 7f632d110700 1 -- 172.21.15.112:0/1454905062 shutdown_connections 2023-08-27T11:17:14.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.700+0000 7f632d110700 1 -- 172.21.15.112:0/1454905062 wait complete. 2023-08-27T11:17:14.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.702+0000 7f632d110700 1 Processor -- start 2023-08-27T11:17:14.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.702+0000 7f632d110700 1 -- start start 2023-08-27T11:17:14.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632d110700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63280b1400 0x7f63280c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:14.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632d110700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63280b17e0 0x7f63281f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:14.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632d110700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6328161ce0 0x7f6328156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:14.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632d110700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6328144390 con 0x7f6328161ce0 2023-08-27T11:17:14.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632d110700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63280c7610 con 0x7f63280b17e0 2023-08-27T11:17:14.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632d110700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63281f7510 con 0x7f63280b1400 2023-08-27T11:17:14.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6328161ce0 0x7f6328156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:14.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6328161ce0 0x7f6328156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55200/0 (socket says 172.21.15.112:55200) 2023-08-27T11:17:14.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632759e700 1 -- 172.21.15.112:0/4013386723 learned_addr learned my addr 172.21.15.112:0/4013386723 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:14.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632659c700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63280b17e0 0x7f63281f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:14.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f6326d9d700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63280b1400 0x7f63280c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:14.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632759e700 1 -- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63280b1400 msgr2=0x7f63280c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632759e700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63280b1400 0x7f63280c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632759e700 1 -- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63280b17e0 msgr2=0x7f63281f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632759e700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63280b17e0 0x7f63281f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632759e700 1 -- 172.21.15.112:0/4013386723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c0094a0 con 0x7f6328161ce0 2023-08-27T11:17:14.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f6326d9d700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63280b1400 0x7f63280c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:14.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.703+0000 7f632759e700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6328161ce0 0x7f6328156160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6320007e40 tx=0x7f632000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:14.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.704+0000 7f630f7fe700 1 -- 172.21.15.112:0/4013386723 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632000ede0 con 0x7f6328161ce0 2023-08-27T11:17:14.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.704+0000 7f630f7fe700 1 -- 172.21.15.112:0/4013386723 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6320014420 con 0x7f6328161ce0 2023-08-27T11:17:14.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.704+0000 7f630f7fe700 1 -- 172.21.15.112:0/4013386723 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63200147d0 con 0x7f6328161ce0 2023-08-27T11:17:14.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.704+0000 7f632d110700 1 -- 172.21.15.112:0/4013386723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63281445e0 con 0x7f6328161ce0 2023-08-27T11:17:14.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.704+0000 7f632d110700 1 -- 172.21.15.112:0/4013386723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6328144ba0 con 0x7f6328161ce0 2023-08-27T11:17:14.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.705+0000 7f632d110700 1 -- 172.21.15.112:0/4013386723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6328067a20 con 0x7f6328161ce0 2023-08-27T11:17:14.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.705+0000 7f630f7fe700 1 -- 172.21.15.112:0/4013386723 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6320014930 con 0x7f6328161ce0 2023-08-27T11:17:14.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.705+0000 7f630f7fe700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f631406a0e0 0x7f631406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:14.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.705+0000 7f630f7fe700 1 -- 172.21.15.112:0/4013386723 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6320088ca0 con 0x7f6328161ce0 2023-08-27T11:17:14.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.706+0000 7f6326d9d700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f631406a0e0 0x7f631406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:14.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.706+0000 7f6326d9d700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f631406a0e0 0x7f631406c5a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f631c0090b0 tx=0x7f631c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:14.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.710+0000 7f630f7fe700 1 -- 172.21.15.112:0/4013386723 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6320057e20 con 0x7f6328161ce0 2023-08-27T11:17:14.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.941+0000 7f632d110700 1 -- 172.21.15.112:0/4013386723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f632806db60 con 0x7f6328161ce0 2023-08-27T11:17:14.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.941+0000 7f630f7fe700 1 -- 172.21.15.112:0/4013386723 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f63200579b0 con 0x7f6328161ce0 2023-08-27T11:17:14.943 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:14.943 INFO:teuthology.orchestra.run.smithi112.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-08-27T11:17:14.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.943+0000 7f632d110700 1 -- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f631406a0e0 msgr2=0x7f631406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.943+0000 7f632d110700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f631406a0e0 0x7f631406c5a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f631c0090b0 tx=0x7f631c01c040 comp rx=0 tx=0).stop 2023-08-27T11:17:14.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.943+0000 7f632d110700 1 -- 172.21.15.112:0/4013386723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6328161ce0 msgr2=0x7f6328156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:14.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.943+0000 7f632d110700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6328161ce0 0x7f6328156160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6320007e40 tx=0x7f632000e3c0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.943+0000 7f632d110700 1 -- 172.21.15.112:0/4013386723 shutdown_connections 2023-08-27T11:17:14.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.943+0000 7f632d110700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f631406a0e0 0x7f631406c5a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.943+0000 7f632d110700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6328161ce0 0x7f6328156160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.943+0000 7f632d110700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63280b1400 0x7f63280c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.943+0000 7f632d110700 1 --2- 172.21.15.112:0/4013386723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63280b17e0 0x7f63281f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:14.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.943+0000 7f632d110700 1 -- 172.21.15.112:0/4013386723 >> 172.21.15.112:0/4013386723 conn(0x7f63280d0000 msgr2=0x7f63280cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:14.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.943+0000 7f632d110700 1 -- 172.21.15.112:0/4013386723 shutdown_connections 2023-08-27T11:17:14.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:14.943+0000 7f632d110700 1 -- 172.21.15.112:0/4013386723 wait complete. 2023-08-27T11:17:14.956 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-08-27T11:17:14.957 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.074+0000 7efc99e8b700 1 Processor -- start 2023-08-27T11:17:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.074+0000 7efc99e8b700 1 -- start start 2023-08-27T11:17:15.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.075+0000 7efc99e8b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efc940b1400 0x7efc940cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.075+0000 7efc99e8b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc940b17e0 0x7efc9406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.075+0000 7efc99e8b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efc94161ce0 0x7efc9406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.075+0000 7efc99e8b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc94161850 con 0x7efc940b1400 2023-08-27T11:17:15.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.075+0000 7efc99e8b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc9406e7d0 con 0x7efc94161ce0 2023-08-27T11:17:15.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.075+0000 7efc99e8b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc940cfc10 con 0x7efc940b17e0 2023-08-27T11:17:15.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.075+0000 7efc937fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efc940b1400 0x7efc940cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.075+0000 7efc937fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efc940b1400 0x7efc940cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55204/0 (socket says 172.21.15.112:55204) 2023-08-27T11:17:15.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.075+0000 7efc937fe700 1 -- 172.21.15.112:0/725754574 learned_addr learned my addr 172.21.15.112:0/725754574 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:15.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.075+0000 7efc93fff700 1 --2- 172.21.15.112:0/725754574 >> v2:172.21.15.138:3300/0 conn(0x7efc94161ce0 0x7efc9406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.075+0000 7efc92ffd700 1 --2- 172.21.15.112:0/725754574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc940b17e0 0x7efc9406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc937fe700 1 -- 172.21.15.112:0/725754574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc940b17e0 msgr2=0x7efc9406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc937fe700 1 --2- 172.21.15.112:0/725754574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc940b17e0 0x7efc9406e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc937fe700 1 -- 172.21.15.112:0/725754574 >> v2:172.21.15.138:3300/0 conn(0x7efc94161ce0 msgr2=0x7efc9406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc937fe700 1 --2- 172.21.15.112:0/725754574 >> v2:172.21.15.138:3300/0 conn(0x7efc94161ce0 0x7efc9406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc937fe700 1 -- 172.21.15.112:0/725754574 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc94161200 con 0x7efc940b1400 2023-08-27T11:17:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc93fff700 1 --2- 172.21.15.112:0/725754574 >> v2:172.21.15.138:3300/0 conn(0x7efc94161ce0 0x7efc9406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc92ffd700 1 --2- 172.21.15.112:0/725754574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc940b17e0 0x7efc9406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc937fe700 1 --2- 172.21.15.112:0/725754574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc940b1400 0x7efc940cf5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc940ca550 tx=0x7efc88009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e103e4e9894760f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc927fc700 1 -- 172.21.15.112:0/725754574 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8800b870 con 0x7efc940b1400 2023-08-27T11:17:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc927fc700 1 -- 172.21.15.112:0/725754574 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8800b9d0 con 0x7efc940b1400 2023-08-27T11:17:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc927fc700 1 -- 172.21.15.112:0/725754574 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8800bd60 con 0x7efc940b1400 2023-08-27T11:17:15.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc99e8b700 1 -- 172.21.15.112:0/725754574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc940b1400 msgr2=0x7efc940cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc99e8b700 1 --2- 172.21.15.112:0/725754574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc940b1400 0x7efc940cf5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc940ca550 tx=0x7efc88009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc99e8b700 1 -- 172.21.15.112:0/725754574 shutdown_connections 2023-08-27T11:17:15.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc99e8b700 1 --2- 172.21.15.112:0/725754574 >> v2:172.21.15.138:3300/0 conn(0x7efc94161ce0 0x7efc9406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc99e8b700 1 --2- 172.21.15.112:0/725754574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc940b1400 0x7efc940cf5a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc99e8b700 1 --2- 172.21.15.112:0/725754574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc940b17e0 0x7efc9406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.076+0000 7efc99e8b700 1 -- 172.21.15.112:0/725754574 >> 172.21.15.112:0/725754574 conn(0x7efc940d0000 msgr2=0x7efc940d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:15.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.077+0000 7efc99e8b700 1 -- 172.21.15.112:0/725754574 shutdown_connections 2023-08-27T11:17:15.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.077+0000 7efc99e8b700 1 -- 172.21.15.112:0/725754574 wait complete. 2023-08-27T11:17:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.078+0000 7efc99e8b700 1 Processor -- start 2023-08-27T11:17:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.078+0000 7efc99e8b700 1 -- start start 2023-08-27T11:17:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc99e8b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc940b1400 0x7efc941f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc99e8b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc940b17e0 0x7efc9415acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc99e8b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc94161ce0 0x7efc940c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc99e8b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9414cfd0 con 0x7efc940b17e0 2023-08-27T11:17:15.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc99e8b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9415b7b0 con 0x7efc940b1400 2023-08-27T11:17:15.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc99e8b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9415ba90 con 0x7efc94161ce0 2023-08-27T11:17:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc92ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc940b17e0 0x7efc9415acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc92ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc940b17e0 0x7efc9415acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55212/0 (socket says 172.21.15.112:55212) 2023-08-27T11:17:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc92ffd700 1 -- 172.21.15.112:0/4174198138 learned_addr learned my addr 172.21.15.112:0/4174198138 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc937fe700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc940b1400 0x7efc941f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc93fff700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc94161ce0 0x7efc940c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc92ffd700 1 -- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc94161ce0 msgr2=0x7efc940c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc92ffd700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc94161ce0 0x7efc940c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc92ffd700 1 -- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc940b1400 msgr2=0x7efc941f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc92ffd700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc940b1400 0x7efc941f27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc92ffd700 1 -- 172.21.15.112:0/4174198138 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc88009550 con 0x7efc940b17e0 2023-08-27T11:17:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc93fff700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc94161ce0 0x7efc940c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc92ffd700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc940b17e0 0x7efc9415acf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efc8400d800 tx=0x7efc8400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc7bfff700 1 -- 172.21.15.112:0/4174198138 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84004030 con 0x7efc940b17e0 2023-08-27T11:17:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc7bfff700 1 -- 172.21.15.112:0/4174198138 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc84004190 con 0x7efc940b17e0 2023-08-27T11:17:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc7bfff700 1 -- 172.21.15.112:0/4174198138 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84004520 con 0x7efc940b17e0 2023-08-27T11:17:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc99e8b700 1 -- 172.21.15.112:0/4174198138 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc9415bd70 con 0x7efc940b17e0 2023-08-27T11:17:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.079+0000 7efc99e8b700 1 -- 172.21.15.112:0/4174198138 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9415c330 con 0x7efc940b17e0 2023-08-27T11:17:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.080+0000 7efc99e8b700 1 -- 172.21.15.112:0/4174198138 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9415dda0 con 0x7efc940b17e0 2023-08-27T11:17:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.081+0000 7efc7bfff700 1 -- 172.21.15.112:0/4174198138 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efc840036a0 con 0x7efc940b17e0 2023-08-27T11:17:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.081+0000 7efc7bfff700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc8006a0e0 0x7efc8006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.081+0000 7efc7bfff700 1 -- 172.21.15.112:0/4174198138 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efc84088a50 con 0x7efc940b17e0 2023-08-27T11:17:15.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.081+0000 7efc937fe700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc8006a0e0 0x7efc8006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.082+0000 7efc937fe700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc8006a0e0 0x7efc8006c5a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc88000c00 tx=0x7efc88008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:15.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.085+0000 7efc7bfff700 1 -- 172.21.15.112:0/4174198138 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efc84057bd0 con 0x7efc940b17e0 2023-08-27T11:17:15.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.273+0000 7efc99e8b700 1 -- 172.21.15.112:0/4174198138 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc94067a20 con 0x7efc940b17e0 2023-08-27T11:17:15.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.275+0000 7efc7bfff700 1 -- 172.21.15.112:0/4174198138 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7efc84057760 con 0x7efc940b17e0 2023-08-27T11:17:15.276 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:15.276 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:15.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.277+0000 7efc99e8b700 1 -- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc8006a0e0 msgr2=0x7efc8006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.277+0000 7efc99e8b700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc8006a0e0 0x7efc8006c5a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc88000c00 tx=0x7efc88008030 comp rx=0 tx=0).stop 2023-08-27T11:17:15.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.277+0000 7efc99e8b700 1 -- 172.21.15.112:0/4174198138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc940b17e0 msgr2=0x7efc9415acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.277+0000 7efc99e8b700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc940b17e0 0x7efc9415acf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efc8400d800 tx=0x7efc8400db10 comp rx=0 tx=0).stop 2023-08-27T11:17:15.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.277+0000 7efc99e8b700 1 -- 172.21.15.112:0/4174198138 shutdown_connections 2023-08-27T11:17:15.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.277+0000 7efc99e8b700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc8006a0e0 0x7efc8006c5a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.277+0000 7efc99e8b700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc94161ce0 0x7efc940c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.277+0000 7efc99e8b700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc940b1400 0x7efc941f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.277+0000 7efc99e8b700 1 --2- 172.21.15.112:0/4174198138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc940b17e0 0x7efc9415acf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.277+0000 7efc99e8b700 1 -- 172.21.15.112:0/4174198138 >> 172.21.15.112:0/4174198138 conn(0x7efc940d0000 msgr2=0x7efc940cc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:15.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.277+0000 7efc99e8b700 1 -- 172.21.15.112:0/4174198138 shutdown_connections 2023-08-27T11:17:15.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.277+0000 7efc99e8b700 1 -- 172.21.15.112:0/4174198138 wait complete. 2023-08-27T11:17:15.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:17:02.484063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:17:03.722976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:17:15.291 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:17:15.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.412+0000 7f0a9ab99700 1 Processor -- start 2023-08-27T11:17:15.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.412+0000 7f0a9ab99700 1 -- start start 2023-08-27T11:17:15.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a9ab99700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0a94143ba0 0x7f0a940b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a9ab99700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a940cfd50 con 0x7f0a941437c0 2023-08-27T11:17:15.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a9ab99700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a940cf140 con 0x7f0a9406d5b0 2023-08-27T11:17:15.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a9ab99700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a940cf420 con 0x7f0a94143ba0 2023-08-27T11:17:15.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a93fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0a94143ba0 0x7f0a940b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a98935700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0a941437c0 0x7f0a940ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49600/0 (socket says 172.21.15.112:49600) 2023-08-27T11:17:15.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a98935700 1 -- 172.21.15.112:0/3561884537 learned_addr learned my addr 172.21.15.112:0/3561884537 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a93fff700 1 -- 172.21.15.112:0/3561884537 >> v1:172.21.15.138:6789/0 conn(0x7f0a9406d5b0 legacy=0x7f0a9415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a93fff700 1 -- 172.21.15.112:0/3561884537 >> v1:172.21.15.112:6789/0 conn(0x7f0a941437c0 legacy=0x7f0a940ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a93fff700 1 -- 172.21.15.112:0/3561884537 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a940cf700 con 0x7f0a94143ba0 2023-08-27T11:17:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a93fff700 1 --2- 172.21.15.112:0/3561884537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a94143ba0 0x7f0a940b4670 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009840 tx=0x7f0a80009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef6f576d2ff3ad0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a937fe700 1 -- 172.21.15.112:0/3561884537 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80004230 con 0x7f0a94143ba0 2023-08-27T11:17:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a937fe700 1 -- 172.21.15.112:0/3561884537 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a80004390 con 0x7f0a94143ba0 2023-08-27T11:17:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.414+0000 7f0a937fe700 1 -- 172.21.15.112:0/3561884537 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a800036a0 con 0x7f0a94143ba0 2023-08-27T11:17:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.415+0000 7f0a9ab99700 1 -- 172.21.15.112:0/3561884537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a94143ba0 msgr2=0x7f0a940b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.415+0000 7f0a9ab99700 1 --2- 172.21.15.112:0/3561884537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a94143ba0 0x7f0a940b4670 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009840 tx=0x7f0a80009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.415+0000 7f0a9ab99700 1 -- 172.21.15.112:0/3561884537 shutdown_connections 2023-08-27T11:17:15.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.415+0000 7f0a9ab99700 1 --2- 172.21.15.112:0/3561884537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a94143ba0 0x7f0a940b4670 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.415+0000 7f0a9ab99700 1 -- 172.21.15.112:0/3561884537 >> 172.21.15.112:0/3561884537 conn(0x7f0a940b1e90 msgr2=0x7f0a940b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:15.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.415+0000 7f0a9ab99700 1 -- 172.21.15.112:0/3561884537 shutdown_connections 2023-08-27T11:17:15.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.415+0000 7f0a9ab99700 1 -- 172.21.15.112:0/3561884537 wait complete. 2023-08-27T11:17:15.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.416+0000 7f0a9ab99700 1 Processor -- start 2023-08-27T11:17:15.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.416+0000 7f0a9ab99700 1 -- start start 2023-08-27T11:17:15.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a9ab99700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a9406d5b0 0x7f0a941394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a9ab99700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a941437c0 0x7f0a941383a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a9ab99700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a94143ba0 0x7f0a9413cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a9ab99700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a941381e0 con 0x7f0a94143ba0 2023-08-27T11:17:15.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a9ab99700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a941ef990 con 0x7f0a9406d5b0 2023-08-27T11:17:15.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a9ab99700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a941f33b0 con 0x7f0a941437c0 2023-08-27T11:17:15.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a99136700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a94143ba0 0x7f0a9413cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a99136700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a94143ba0 0x7f0a9413cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55228/0 (socket says 172.21.15.112:55228) 2023-08-27T11:17:15.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a99136700 1 -- 172.21.15.112:0/2210282043 learned_addr learned my addr 172.21.15.112:0/2210282043 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:15.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a98935700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a9406d5b0 0x7f0a941394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a99136700 1 -- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a941437c0 msgr2=0x7f0a941383a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a93fff700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a941437c0 0x7f0a941383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a99136700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a941437c0 0x7f0a941383a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a99136700 1 -- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a9406d5b0 msgr2=0x7f0a941394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a99136700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a9406d5b0 0x7f0a941394f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a99136700 1 -- 172.21.15.112:0/2210282043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a800094a0 con 0x7f0a94143ba0 2023-08-27T11:17:15.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a99136700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a94143ba0 0x7f0a9413cd20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00eeb0 tx=0x7f0a8c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:15.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a917fa700 1 -- 172.21.15.112:0/2210282043 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c011670 con 0x7f0a94143ba0 2023-08-27T11:17:15.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a917fa700 1 -- 172.21.15.112:0/2210282043 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c0117d0 con 0x7f0a94143ba0 2023-08-27T11:17:15.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a9ab99700 1 -- 172.21.15.112:0/2210282043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a941f3690 con 0x7f0a94143ba0 2023-08-27T11:17:15.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a917fa700 1 -- 172.21.15.112:0/2210282043 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c011b80 con 0x7f0a94143ba0 2023-08-27T11:17:15.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.418+0000 7f0a9ab99700 1 -- 172.21.15.112:0/2210282043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a941f3c50 con 0x7f0a94143ba0 2023-08-27T11:17:15.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.419+0000 7f0a9ab99700 1 -- 172.21.15.112:0/2210282043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a94067e90 con 0x7f0a94143ba0 2023-08-27T11:17:15.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.420+0000 7f0a917fa700 1 -- 172.21.15.112:0/2210282043 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0a8c011ce0 con 0x7f0a94143ba0 2023-08-27T11:17:15.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.420+0000 7f0a917fa700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a7806a0e0 0x7f0a7806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.420+0000 7f0a917fa700 1 -- 172.21.15.112:0/2210282043 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0a8c08b130 con 0x7f0a94143ba0 2023-08-27T11:17:15.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.420+0000 7f0a98935700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a7806a0e0 0x7f0a7806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.420+0000 7f0a98935700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a7806a0e0 0x7f0a7806c5a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a9413c9a0 tx=0x7f0a88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:15.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.424+0000 7f0a917fa700 1 -- 172.21.15.112:0/2210282043 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0a8c056820 con 0x7f0a94143ba0 2023-08-27T11:17:15.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.612+0000 7f0a9ab99700 1 -- 172.21.15.112:0/2210282043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0a94067a20 con 0x7f0a94143ba0 2023-08-27T11:17:15.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.612+0000 7f0a917fa700 1 -- 172.21.15.112:0/2210282043 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0a8c00ce60 con 0x7f0a94143ba0 2023-08-27T11:17:15.614 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:15.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.614+0000 7f0a9ab99700 1 -- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a7806a0e0 msgr2=0x7f0a7806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.614+0000 7f0a9ab99700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a7806a0e0 0x7f0a7806c5a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a9413c9a0 tx=0x7f0a88009040 comp rx=0 tx=0).stop 2023-08-27T11:17:15.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.614+0000 7f0a9ab99700 1 -- 172.21.15.112:0/2210282043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a94143ba0 msgr2=0x7f0a9413cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.614+0000 7f0a9ab99700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a94143ba0 0x7f0a9413cd20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00eeb0 tx=0x7f0a8c00ef90 comp rx=0 tx=0).stop 2023-08-27T11:17:15.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.614+0000 7f0a9ab99700 1 -- 172.21.15.112:0/2210282043 shutdown_connections 2023-08-27T11:17:15.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.614+0000 7f0a9ab99700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a7806a0e0 0x7f0a7806c5a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.614+0000 7f0a9ab99700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a94143ba0 0x7f0a9413cd20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.614+0000 7f0a9ab99700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a9406d5b0 0x7f0a941394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.614+0000 7f0a9ab99700 1 --2- 172.21.15.112:0/2210282043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a941437c0 0x7f0a941383a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.614+0000 7f0a9ab99700 1 -- 172.21.15.112:0/2210282043 >> 172.21.15.112:0/2210282043 conn(0x7f0a940b1e90 msgr2=0x7f0a940708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:15.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.614+0000 7f0a9ab99700 1 -- 172.21.15.112:0/2210282043 shutdown_connections 2023-08-27T11:17:15.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.614+0000 7f0a9ab99700 1 -- 172.21.15.112:0/2210282043 wait complete. 2023-08-27T11:17:15.627 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:15.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.750+0000 7f8ce303f700 1 Processor -- start 2023-08-27T11:17:15.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.750+0000 7f8ce303f700 1 -- start start 2023-08-27T11:17:15.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.750+0000 7f8ce303f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8cdc144b80 0x7f8cdc0b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.750+0000 7f8ce303f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cdc06c6e0 0x7f8cdc0cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.750+0000 7f8ce303f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cdc0cef20 con 0x7f8cdc0b3de0 2023-08-27T11:17:15.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.750+0000 7f8ce303f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cdc0b4b40 con 0x7f8cdc144b80 2023-08-27T11:17:15.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.750+0000 7f8ce303f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0b42a0 con 0x7f8cdc06c6e0 2023-08-27T11:17:15.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.751+0000 7f8ce15dc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cdc06c6e0 0x7f8cdc0cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.751+0000 7f8ce0ddb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8cdc144b80 0x7f8cdc0b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.751+0000 7f8cdbfff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8cdc0b3de0 0x7f8cdc06bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:57138/0 (socket says 172.21.15.112:57138) 2023-08-27T11:17:15.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.751+0000 7f8ce15dc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cdc06c6e0 0x7f8cdc0cf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38656/0 (socket says 172.21.15.112:38656) 2023-08-27T11:17:15.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.751+0000 7f8cdbfff700 1 -- 172.21.15.112:0/2905814319 learned_addr learned my addr 172.21.15.112:0/2905814319 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:15.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.751+0000 7f8ce15dc700 1 -- 172.21.15.112:0/2905814319 >> v2:172.21.15.138:3300/0 conn(0x7f8cdc144b80 msgr2=0x7f8cdc0b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:15.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.751+0000 7f8ce15dc700 1 --2- 172.21.15.112:0/2905814319 >> v2:172.21.15.138:3300/0 conn(0x7f8cdc144b80 0x7f8cdc0b6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.751+0000 7f8ce15dc700 1 -- 172.21.15.112:0/2905814319 >> v1:172.21.15.138:6789/0 conn(0x7f8cdc0b3de0 legacy=0x7f8cdc06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.751+0000 7f8ce15dc700 1 -- 172.21.15.112:0/2905814319 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc0b4580 con 0x7f8cdc06c6e0 2023-08-27T11:17:15.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.751+0000 7f8ce0ddb700 1 --2- 172.21.15.112:0/2905814319 >> v2:172.21.15.138:3300/0 conn(0x7f8cdc144b80 0x7f8cdc0b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:15.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.751+0000 7f8ce15dc700 1 --2- 172.21.15.112:0/2905814319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cdc06c6e0 0x7f8cdc0cf3b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009d90 tx=0x7f8cd400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f366d711ff8b84 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:15.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.751+0000 7f8cdb7fe700 1 -- 172.21.15.112:0/2905814319 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd400e070 con 0x7f8cdc06c6e0 2023-08-27T11:17:15.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.752+0000 7f8cdb7fe700 1 -- 172.21.15.112:0/2905814319 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd400bb20 con 0x7f8cdc06c6e0 2023-08-27T11:17:15.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.752+0000 7f8cdb7fe700 1 -- 172.21.15.112:0/2905814319 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4012430 con 0x7f8cdc06c6e0 2023-08-27T11:17:15.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.752+0000 7f8ce303f700 1 -- 172.21.15.112:0/2905814319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cdc06c6e0 msgr2=0x7f8cdc0cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.752+0000 7f8ce303f700 1 --2- 172.21.15.112:0/2905814319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cdc06c6e0 0x7f8cdc0cf3b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009d90 tx=0x7f8cd400b330 comp rx=0 tx=0).stop 2023-08-27T11:17:15.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.752+0000 7f8ce303f700 1 -- 172.21.15.112:0/2905814319 shutdown_connections 2023-08-27T11:17:15.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.752+0000 7f8ce303f700 1 --2- 172.21.15.112:0/2905814319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cdc06c6e0 0x7f8cdc0cf3b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.752+0000 7f8ce303f700 1 --2- 172.21.15.112:0/2905814319 >> v2:172.21.15.138:3300/0 conn(0x7f8cdc144b80 0x7f8cdc0b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.752+0000 7f8ce303f700 1 -- 172.21.15.112:0/2905814319 >> 172.21.15.112:0/2905814319 conn(0x7f8cdc0cdc50 msgr2=0x7f8cdc0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:15.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.752+0000 7f8ce303f700 1 -- 172.21.15.112:0/2905814319 shutdown_connections 2023-08-27T11:17:15.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.752+0000 7f8ce303f700 1 -- 172.21.15.112:0/2905814319 wait complete. 2023-08-27T11:17:15.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.753+0000 7f8ce303f700 1 Processor -- start 2023-08-27T11:17:15.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.753+0000 7f8ce303f700 1 -- start start 2023-08-27T11:17:15.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.754+0000 7f8ce303f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc06c6e0 0x7f8cdc157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.754+0000 7f8ce303f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cdc0b3de0 0x7f8cdc143580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.754+0000 7f8ce303f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc144b80 0x7f8cdc15b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.754+0000 7f8ce303f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc1570f0 con 0x7f8cdc144b80 2023-08-27T11:17:15.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.754+0000 7f8ce303f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc157a50 con 0x7f8cdc06c6e0 2023-08-27T11:17:15.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.754+0000 7f8ce303f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc156470 con 0x7f8cdc0b3de0 2023-08-27T11:17:15.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.754+0000 7f8ce15dc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc144b80 0x7f8cdc15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.754+0000 7f8ce15dc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc144b80 0x7f8cdc15b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55240/0 (socket says 172.21.15.112:55240) 2023-08-27T11:17:15.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.754+0000 7f8ce15dc700 1 -- 172.21.15.112:0/886602016 learned_addr learned my addr 172.21.15.112:0/886602016 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:15.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.754+0000 7f8cdbfff700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cdc0b3de0 0x7f8cdc143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.754+0000 7f8ce0ddb700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc06c6e0 0x7f8cdc157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.755+0000 7f8ce15dc700 1 -- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cdc0b3de0 msgr2=0x7f8cdc143580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.755+0000 7f8ce15dc700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cdc0b3de0 0x7f8cdc143580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.755+0000 7f8ce15dc700 1 -- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc06c6e0 msgr2=0x7f8cdc157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.755+0000 7f8ce15dc700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc06c6e0 0x7f8cdc157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.755+0000 7f8ce15dc700 1 -- 172.21.15.112:0/886602016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd00094a0 con 0x7f8cdc144b80 2023-08-27T11:17:15.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.755+0000 7f8ce15dc700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc144b80 0x7f8cdc15b150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4013000 tx=0x7f8cd4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:15.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.755+0000 7f8cd97fa700 1 -- 172.21.15.112:0/886602016 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4012430 con 0x7f8cdc144b80 2023-08-27T11:17:15.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.755+0000 7f8cd97fa700 1 -- 172.21.15.112:0/886602016 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd4020070 con 0x7f8cdc144b80 2023-08-27T11:17:15.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.755+0000 7f8ce303f700 1 -- 172.21.15.112:0/886602016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd4009a70 con 0x7f8cdc144b80 2023-08-27T11:17:15.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.755+0000 7f8cd97fa700 1 -- 172.21.15.112:0/886602016 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4003ea0 con 0x7f8cdc144b80 2023-08-27T11:17:15.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.755+0000 7f8ce303f700 1 -- 172.21.15.112:0/886602016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cdc156a30 con 0x7f8cdc144b80 2023-08-27T11:17:15.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.756+0000 7f8ce303f700 1 -- 172.21.15.112:0/886602016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cdc157230 con 0x7f8cdc144b80 2023-08-27T11:17:15.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.756+0000 7f8cd97fa700 1 -- 172.21.15.112:0/886602016 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8cd4004280 con 0x7f8cdc144b80 2023-08-27T11:17:15.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.756+0000 7f8cd97fa700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cc006a0e0 0x7f8cc006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:15.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.757+0000 7f8cd97fa700 1 -- 172.21.15.112:0/886602016 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8cd4022070 con 0x7f8cdc144b80 2023-08-27T11:17:15.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.757+0000 7f8ce0ddb700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cc006a0e0 0x7f8cc006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:15.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.757+0000 7f8ce0ddb700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cc006a0e0 0x7f8cc006c5a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009c60 tx=0x7f8cd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:15.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.761+0000 7f8cd97fa700 1 -- 172.21.15.112:0/886602016 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8cdc157230 con 0x7f8cdc144b80 2023-08-27T11:17:15.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.950+0000 7f8ce303f700 1 -- 172.21.15.112:0/886602016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8cdc157230 con 0x7f8cdc144b80 2023-08-27T11:17:15.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.950+0000 7f8cd97fa700 1 -- 172.21.15.112:0/886602016 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8cdc157230 con 0x7f8cdc144b80 2023-08-27T11:17:15.952 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:15.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.952+0000 7f8ce303f700 1 -- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cc006a0e0 msgr2=0x7f8cc006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.952+0000 7f8ce303f700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cc006a0e0 0x7f8cc006c5a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009c60 tx=0x7f8cd0008040 comp rx=0 tx=0).stop 2023-08-27T11:17:15.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.952+0000 7f8ce303f700 1 -- 172.21.15.112:0/886602016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc144b80 msgr2=0x7f8cdc15b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:15.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.952+0000 7f8ce303f700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc144b80 0x7f8cdc15b150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4013000 tx=0x7f8cd4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.952+0000 7f8ce303f700 1 -- 172.21.15.112:0/886602016 shutdown_connections 2023-08-27T11:17:15.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.952+0000 7f8ce303f700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cc006a0e0 0x7f8cc006c5a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.952+0000 7f8ce303f700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cdc144b80 0x7f8cdc15b150 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.952+0000 7f8ce303f700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc06c6e0 0x7f8cdc157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.952+0000 7f8ce303f700 1 --2- 172.21.15.112:0/886602016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cdc0b3de0 0x7f8cdc143580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:15.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.952+0000 7f8ce303f700 1 -- 172.21.15.112:0/886602016 >> 172.21.15.112:0/886602016 conn(0x7f8cdc0cdc50 msgr2=0x7f8cdc0c62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:15.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.952+0000 7f8ce303f700 1 -- 172.21.15.112:0/886602016 shutdown_connections 2023-08-27T11:17:15.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:15.952+0000 7f8ce303f700 1 -- 172.21.15.112:0/886602016 wait complete. 2023-08-27T11:17:15.965 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:17:16.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.086+0000 7fa34594e700 1 Processor -- start 2023-08-27T11:17:16.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.086+0000 7fa34594e700 1 -- start start 2023-08-27T11:17:16.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.087+0000 7fa34594e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa34006f790 0x7fa34006fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.087+0000 7fa34594e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3400b5070 con 0x7fa340146c20 2023-08-27T11:17:16.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.087+0000 7fa34594e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa340147000 con 0x7fa3400b3440 2023-08-27T11:17:16.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.087+0000 7fa34594e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa340070160 con 0x7fa34006f790 2023-08-27T11:17:16.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.087+0000 7fa33f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa34006f790 0x7fa34006fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.087+0000 7fa33effd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa340146c20 0x7fa340160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49626/0 (socket says 172.21.15.112:49626) 2023-08-27T11:17:16.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.087+0000 7fa33effd700 1 -- 172.21.15.112:0/2277174551 learned_addr learned my addr 172.21.15.112:0/2277174551 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:16.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa33f7fe700 1 -- 172.21.15.112:0/2277174551 >> v1:172.21.15.138:6789/0 conn(0x7fa3400b3440 legacy=0x7fa34015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa33f7fe700 1 -- 172.21.15.112:0/2277174551 >> v1:172.21.15.112:6789/0 conn(0x7fa340146c20 legacy=0x7fa340160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa33dffb700 1 -- 172.21.15.112:0/2277174551 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559553203 0 0) 0x7fa334001110 con 0x7fa340146c20 2023-08-27T11:17:16.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa33f7fe700 1 -- 172.21.15.112:0/2277174551 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa340161740 con 0x7fa34006f790 2023-08-27T11:17:16.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa33f7fe700 1 --2- 172.21.15.112:0/2277174551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa34006f790 0x7fa34006fc20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa338009d90 tx=0x7fa33800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04b96b803fc6acd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:16.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa33dffb700 1 -- 172.21.15.112:0/2277174551 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33800e070 con 0x7fa34006f790 2023-08-27T11:17:16.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa33dffb700 1 -- 172.21.15.112:0/2277174551 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33800bbd0 con 0x7fa34006f790 2023-08-27T11:17:16.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa33dffb700 1 -- 172.21.15.112:0/2277174551 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338012430 con 0x7fa34006f790 2023-08-27T11:17:16.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa34594e700 1 -- 172.21.15.112:0/2277174551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa34006f790 msgr2=0x7fa34006fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa34594e700 1 --2- 172.21.15.112:0/2277174551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa34006f790 0x7fa34006fc20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa338009d90 tx=0x7fa33800b330 comp rx=0 tx=0).stop 2023-08-27T11:17:16.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa34594e700 1 -- 172.21.15.112:0/2277174551 shutdown_connections 2023-08-27T11:17:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa34594e700 1 --2- 172.21.15.112:0/2277174551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa34006f790 0x7fa34006fc20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa34594e700 1 -- 172.21.15.112:0/2277174551 >> 172.21.15.112:0/2277174551 conn(0x7fa3400b1e90 msgr2=0x7fa3400b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa34594e700 1 -- 172.21.15.112:0/2277174551 shutdown_connections 2023-08-27T11:17:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.088+0000 7fa34594e700 1 -- 172.21.15.112:0/2277174551 wait complete. 2023-08-27T11:17:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.090+0000 7fa34594e700 1 Processor -- start 2023-08-27T11:17:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.090+0000 7fa34594e700 1 -- start start 2023-08-27T11:17:16.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.090+0000 7fa34594e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa34006f790 0x7fa34015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa34594e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3400b3440 0x7fa3401404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa34594e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa340146c20 0x7fa3400707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa34594e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34015f9b0 con 0x7fa3400b3440 2023-08-27T11:17:16.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa34594e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34015cc30 con 0x7fa340146c20 2023-08-27T11:17:16.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa34594e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34015ceb0 con 0x7fa34006f790 2023-08-27T11:17:16.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa33e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3400b3440 0x7fa3401404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa33e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3400b3440 0x7fa3401404e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55252/0 (socket says 172.21.15.112:55252) 2023-08-27T11:17:16.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa33e7fc700 1 -- 172.21.15.112:0/3543473188 learned_addr learned my addr 172.21.15.112:0/3543473188 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:16.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa33effd700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa34006f790 0x7fa34015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa33f7fe700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa340146c20 0x7fa3400707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa33e7fc700 1 -- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa34006f790 msgr2=0x7fa34015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa33e7fc700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa34006f790 0x7fa34015c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa33e7fc700 1 -- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa340146c20 msgr2=0x7fa3400707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa33e7fc700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa340146c20 0x7fa3400707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa33e7fc700 1 -- 172.21.15.112:0/3543473188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa338009a70 con 0x7fa3400b3440 2023-08-27T11:17:16.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa33f7fe700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa340146c20 0x7fa3400707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:16.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa33e7fc700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3400b3440 0x7fa3401404e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa33000c510 tx=0x7fa33000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:16.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa327fff700 1 -- 172.21.15.112:0/3543473188 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3300052d0 con 0x7fa3400b3440 2023-08-27T11:17:16.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa327fff700 1 -- 172.21.15.112:0/3543473188 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa330005430 con 0x7fa3400b3440 2023-08-27T11:17:16.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa34594e700 1 -- 172.21.15.112:0/3543473188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34015f080 con 0x7fa3400b3440 2023-08-27T11:17:16.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa327fff700 1 -- 172.21.15.112:0/3543473188 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3300038c0 con 0x7fa3400b3440 2023-08-27T11:17:16.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.091+0000 7fa34594e700 1 -- 172.21.15.112:0/3543473188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34015f640 con 0x7fa3400b3440 2023-08-27T11:17:16.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.092+0000 7fa34594e700 1 -- 172.21.15.112:0/3543473188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34015fb60 con 0x7fa3400b3440 2023-08-27T11:17:16.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.093+0000 7fa327fff700 1 -- 172.21.15.112:0/3543473188 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa330003a20 con 0x7fa3400b3440 2023-08-27T11:17:16.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.093+0000 7fa327fff700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa32c06a0e0 0x7fa32c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.093+0000 7fa327fff700 1 -- 172.21.15.112:0/3543473188 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3300879d0 con 0x7fa3400b3440 2023-08-27T11:17:16.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.093+0000 7fa33effd700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa32c06a0e0 0x7fa32c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.094+0000 7fa33effd700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa32c06a0e0 0x7fa32c06c5a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa334000f60 tx=0x7fa334004a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:16.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.097+0000 7fa327fff700 1 -- 172.21.15.112:0/3543473188 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa34015fb60 con 0x7fa3400b3440 2023-08-27T11:17:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.285+0000 7fa34594e700 1 -- 172.21.15.112:0/3543473188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa340067e90 con 0x7fa3400b3440 2023-08-27T11:17:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.285+0000 7fa327fff700 1 -- 172.21.15.112:0/3543473188 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa3300170a0 con 0x7fa3400b3440 2023-08-27T11:17:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:17:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.287+0000 7fa34594e700 1 -- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa32c06a0e0 msgr2=0x7fa32c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.287+0000 7fa34594e700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa32c06a0e0 0x7fa32c06c5a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa334000f60 tx=0x7fa334004a50 comp rx=0 tx=0).stop 2023-08-27T11:17:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.287+0000 7fa34594e700 1 -- 172.21.15.112:0/3543473188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3400b3440 msgr2=0x7fa3401404e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.287+0000 7fa34594e700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3400b3440 0x7fa3401404e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa33000c510 tx=0x7fa33000c820 comp rx=0 tx=0).stop 2023-08-27T11:17:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.287+0000 7fa34594e700 1 -- 172.21.15.112:0/3543473188 shutdown_connections 2023-08-27T11:17:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.287+0000 7fa34594e700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa32c06a0e0 0x7fa32c06c5a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.287+0000 7fa34594e700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa340146c20 0x7fa3400707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.287+0000 7fa34594e700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa34006f790 0x7fa34015c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.287+0000 7fa34594e700 1 --2- 172.21.15.112:0/3543473188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3400b3440 0x7fa3401404e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.287+0000 7fa34594e700 1 -- 172.21.15.112:0/3543473188 >> 172.21.15.112:0/3543473188 conn(0x7fa3400b1e90 msgr2=0x7fa3401412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.287+0000 7fa34594e700 1 -- 172.21.15.112:0/3543473188 shutdown_connections 2023-08-27T11:17:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.287+0000 7fa34594e700 1 -- 172.21.15.112:0/3543473188 wait complete. 2023-08-27T11:17:16.300 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:17:16.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.422+0000 7f4490294700 1 Processor -- start 2023-08-27T11:17:16.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.422+0000 7f4490294700 1 -- start start 2023-08-27T11:17:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.423+0000 7f4490294700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4488144900 0x7f44880b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.423+0000 7f4490294700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44880cca90 0x7f448815f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.423+0000 7f4490294700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448815ee90 con 0x7f448806f790 2023-08-27T11:17:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.423+0000 7f4490294700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44880b11d0 con 0x7f4488144900 2023-08-27T11:17:16.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.423+0000 7f4490294700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44880b14b0 con 0x7f44880cca90 2023-08-27T11:17:16.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.423+0000 7f448e030700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4488144900 0x7f44880b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.423+0000 7f448e831700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f448806f790 0x7f448815b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49644/0 (socket says 172.21.15.112:49644) 2023-08-27T11:17:16.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.423+0000 7f448e831700 1 -- 172.21.15.112:0/2807510194 learned_addr learned my addr 172.21.15.112:0/2807510194 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:16.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.423+0000 7f448d82f700 1 --2- 172.21.15.112:0/2807510194 >> v2:172.21.15.138:3300/0 conn(0x7f44880cca90 0x7f448815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f448e030700 1 -- 172.21.15.112:0/2807510194 >> v2:172.21.15.138:3300/0 conn(0x7f44880cca90 msgr2=0x7f448815f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:16.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f448e030700 1 --2- 172.21.15.112:0/2807510194 >> v2:172.21.15.138:3300/0 conn(0x7f44880cca90 0x7f448815f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f448e030700 1 -- 172.21.15.112:0/2807510194 >> v1:172.21.15.112:6789/0 conn(0x7f448806f790 legacy=0x7f448815b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f448d02e700 1 -- 172.21.15.112:0/2807510194 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542985798 0 0) 0x7f448815ee90 con 0x7f448806f790 2023-08-27T11:17:16.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f448e030700 1 -- 172.21.15.112:0/2807510194 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448815f130 con 0x7f4488144900 2023-08-27T11:17:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f448d82f700 1 --2- 172.21.15.112:0/2807510194 >> v2:172.21.15.138:3300/0 conn(0x7f44880cca90 0x7f448815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f448e030700 1 --2- 172.21.15.112:0/2807510194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4488144900 0x7f44880b09a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f44880703e0 tx=0x7f4478009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82923f5094e3209 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f448d02e700 1 -- 172.21.15.112:0/2807510194 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478011830 con 0x7f4488144900 2023-08-27T11:17:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f448d02e700 1 -- 172.21.15.112:0/2807510194 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4478011990 con 0x7f4488144900 2023-08-27T11:17:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f448d02e700 1 -- 172.21.15.112:0/2807510194 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478011d20 con 0x7f4488144900 2023-08-27T11:17:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f4490294700 1 -- 172.21.15.112:0/2807510194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4488144900 msgr2=0x7f44880b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f4490294700 1 --2- 172.21.15.112:0/2807510194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4488144900 0x7f44880b09a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f44880703e0 tx=0x7f4478009b20 comp rx=0 tx=0).stop 2023-08-27T11:17:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f4490294700 1 -- 172.21.15.112:0/2807510194 shutdown_connections 2023-08-27T11:17:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f4490294700 1 --2- 172.21.15.112:0/2807510194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4488144900 0x7f44880b09a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f4490294700 1 --2- 172.21.15.112:0/2807510194 >> v2:172.21.15.138:3300/0 conn(0x7f44880cca90 0x7f448815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f4490294700 1 -- 172.21.15.112:0/2807510194 >> 172.21.15.112:0/2807510194 conn(0x7f44880b1e90 msgr2=0x7f44880b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:16.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f4490294700 1 -- 172.21.15.112:0/2807510194 shutdown_connections 2023-08-27T11:17:16.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.424+0000 7f4490294700 1 -- 172.21.15.112:0/2807510194 wait complete. 2023-08-27T11:17:16.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.426+0000 7f4490294700 1 Processor -- start 2023-08-27T11:17:16.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.426+0000 7f4490294700 1 -- start start 2023-08-27T11:17:16.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.426+0000 7f4490294700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f448806f790 0x7f4488135980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.426+0000 7f4490294700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44880cca90 0x7f44880a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.426+0000 7f4490294700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4488144900 0x7f44880a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.426+0000 7f4490294700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4488135ec0 con 0x7f44880cca90 2023-08-27T11:17:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.426+0000 7f4490294700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4488136380 con 0x7f4488144900 2023-08-27T11:17:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.426+0000 7f4490294700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44880aaa60 con 0x7f448806f790 2023-08-27T11:17:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.426+0000 7f448d82f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44880cca90 0x7f44880a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.426+0000 7f448d82f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44880cca90 0x7f44880a8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55266/0 (socket says 172.21.15.112:55266) 2023-08-27T11:17:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.426+0000 7f448d82f700 1 -- 172.21.15.112:0/412316000 learned_addr learned my addr 172.21.15.112:0/412316000 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:16.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.427+0000 7f448e831700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4488144900 0x7f44880a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.427+0000 7f448e030700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f448806f790 0x7f4488135980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.427+0000 7f448d82f700 1 -- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f448806f790 msgr2=0x7f4488135980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.427+0000 7f448d82f700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f448806f790 0x7f4488135980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.427+0000 7f448d82f700 1 -- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4488144900 msgr2=0x7f44880a93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.427+0000 7f448d82f700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4488144900 0x7f44880a93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.427+0000 7f448d82f700 1 -- 172.21.15.112:0/412316000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780094a0 con 0x7f44880cca90 2023-08-27T11:17:16.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.427+0000 7f448d82f700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44880cca90 0x7f44880a8e60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f448400e810 tx=0x7f448400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:16.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.427+0000 7f447effd700 1 -- 172.21.15.112:0/412316000 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448400c070 con 0x7f44880cca90 2023-08-27T11:17:16.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.427+0000 7f447effd700 1 -- 172.21.15.112:0/412316000 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4484004230 con 0x7f44880cca90 2023-08-27T11:17:16.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.427+0000 7f4490294700 1 -- 172.21.15.112:0/412316000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44880a9a70 con 0x7f44880cca90 2023-08-27T11:17:16.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.427+0000 7f447effd700 1 -- 172.21.15.112:0/412316000 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44840036a0 con 0x7f44880cca90 2023-08-27T11:17:16.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.427+0000 7f4490294700 1 -- 172.21.15.112:0/412316000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44880aaeb0 con 0x7f44880cca90 2023-08-27T11:17:16.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.428+0000 7f4490294700 1 -- 172.21.15.112:0/412316000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4488067e90 con 0x7f44880cca90 2023-08-27T11:17:16.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.430+0000 7f447effd700 1 -- 172.21.15.112:0/412316000 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4484003800 con 0x7f44880cca90 2023-08-27T11:17:16.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.431+0000 7f447effd700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f446c06a0e0 0x7f446c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.431+0000 7f447effd700 1 -- 172.21.15.112:0/412316000 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f44840884c0 con 0x7f44880cca90 2023-08-27T11:17:16.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.431+0000 7f448e030700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f446c06a0e0 0x7f446c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.431+0000 7f448e030700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f446c06a0e0 0x7f446c06c5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f44780090b0 tx=0x7f447801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:16.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.434+0000 7f447effd700 1 -- 172.21.15.112:0/412316000 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4484053d20 con 0x7f44880cca90 2023-08-27T11:17:16.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.622+0000 7f4490294700 1 -- 172.21.15.112:0/412316000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4488067a20 con 0x7f44880cca90 2023-08-27T11:17:16.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.623+0000 7f447effd700 1 -- 172.21.15.112:0/412316000 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f44840094c0 con 0x7f44880cca90 2023-08-27T11:17:16.624 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:17:16.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.625+0000 7f4490294700 1 -- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f446c06a0e0 msgr2=0x7f446c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.625+0000 7f4490294700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f446c06a0e0 0x7f446c06c5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f44780090b0 tx=0x7f447801c040 comp rx=0 tx=0).stop 2023-08-27T11:17:16.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.625+0000 7f4490294700 1 -- 172.21.15.112:0/412316000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44880cca90 msgr2=0x7f44880a8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.625+0000 7f4490294700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44880cca90 0x7f44880a8e60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f448400e810 tx=0x7f448400eb20 comp rx=0 tx=0).stop 2023-08-27T11:17:16.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.625+0000 7f4490294700 1 -- 172.21.15.112:0/412316000 shutdown_connections 2023-08-27T11:17:16.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.625+0000 7f4490294700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f446c06a0e0 0x7f446c06c5a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.625+0000 7f4490294700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4488144900 0x7f44880a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.625+0000 7f4490294700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f448806f790 0x7f4488135980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.625+0000 7f4490294700 1 --2- 172.21.15.112:0/412316000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44880cca90 0x7f44880a8e60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.625+0000 7f4490294700 1 -- 172.21.15.112:0/412316000 >> 172.21.15.112:0/412316000 conn(0x7f44880b1e90 msgr2=0x7f44880cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:16.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.625+0000 7f4490294700 1 -- 172.21.15.112:0/412316000 shutdown_connections 2023-08-27T11:17:16.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.625+0000 7f4490294700 1 -- 172.21.15.112:0/412316000 wait complete. 2023-08-27T11:17:16.638 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:17:16.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.756+0000 7fe185574700 1 Processor -- start 2023-08-27T11:17:16.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.756+0000 7fe185574700 1 -- start start 2023-08-27T11:17:16.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe185574700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe1800b3440 0x7fe1801619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe185574700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1800b3b70 con 0x7fe180146be0 2023-08-27T11:17:16.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe185574700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1800b3e50 con 0x7fe1800b5470 2023-08-27T11:17:16.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe185574700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe180161ef0 con 0x7fe1800b3440 2023-08-27T11:17:16.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17effd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe180146be0 0x7fe1800b4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49654/0 (socket says 172.21.15.112:49654) 2023-08-27T11:17:16.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17effd700 1 -- 172.21.15.112:0/327748852 learned_addr learned my addr 172.21.15.112:0/327748852 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:16.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568788744 0 0) 0x7fe1800b3b70 con 0x7fe180146be0 2023-08-27T11:17:16.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17f7fe700 1 --2- 172.21.15.112:0/327748852 >> v2:172.21.15.138:3300/0 conn(0x7fe1800b3440 0x7fe1801619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1680035b0 con 0x7fe180146be0 2023-08-27T11:17:16.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2689128326 0 0) 0x7fe1680035b0 con 0x7fe180146be0 2023-08-27T11:17:16.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 >> v2:172.21.15.138:3300/0 conn(0x7fe1800b3440 msgr2=0x7fe1801619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:16.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17dffb700 1 --2- 172.21.15.112:0/327748852 >> v2:172.21.15.138:3300/0 conn(0x7fe1800b3440 0x7fe1801619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 >> v1:172.21.15.138:6789/0 conn(0x7fe1800b5470 legacy=0x7fe1800cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1800b4130 con 0x7fe180146be0 2023-08-27T11:17:16.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fe174001670 con 0x7fe180146be0 2023-08-27T11:17:16.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 >> v1:172.21.15.112:6789/0 conn(0x7fe180146be0 legacy=0x7fe1800b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17dffb700 1 --2- 172.21.15.112:0/327748852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe168003a00 0x7fe168003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17dffb700 1 --2- 172.21.15.112:0/327748852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe168004850 0x7fe168004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.757+0000 7fe17dffb700 1 --2- 172.21.15.112:0/327748852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe168005210 0x7fe16800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16800c080 con 0x7fe168003a00 2023-08-27T11:17:16.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16800c1f0 con 0x7fe168004850 2023-08-27T11:17:16.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16800c3b0 con 0x7fe168005210 2023-08-27T11:17:16.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17effd700 1 --2- 172.21.15.112:0/327748852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe168003a00 0x7fe168003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17e7fc700 1 --2- 172.21.15.112:0/327748852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe168004850 0x7fe168004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17f7fe700 1 --2- 172.21.15.112:0/327748852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe168005210 0x7fe16800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17effd700 1 -- 172.21.15.112:0/327748852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe168005210 msgr2=0x7fe16800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17effd700 1 --2- 172.21.15.112:0/327748852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe168005210 0x7fe16800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17effd700 1 -- 172.21.15.112:0/327748852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe168004850 msgr2=0x7fe168004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17effd700 1 --2- 172.21.15.112:0/327748852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe168004850 0x7fe168004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17effd700 1 -- 172.21.15.112:0/327748852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680035b0 con 0x7fe168003a00 2023-08-27T11:17:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17f7fe700 1 -- 172.21.15.112:0/327748852 reap_dead start 2023-08-27T11:17:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17effd700 1 --2- 172.21.15.112:0/327748852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe168003a00 0x7fe168003e20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe17400c240 tx=0x7fe168005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cdc6e37844508b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174007c10 con 0x7fe168003a00 2023-08-27T11:17:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17400cd60 con 0x7fe168003a00 2023-08-27T11:17:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.758+0000 7fe17dffb700 1 -- 172.21.15.112:0/327748852 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174005ba0 con 0x7fe168003a00 2023-08-27T11:17:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.759+0000 7fe185574700 1 -- 172.21.15.112:0/327748852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe168003a00 msgr2=0x7fe168003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.759+0000 7fe185574700 1 --2- 172.21.15.112:0/327748852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe168003a00 0x7fe168003e20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe17400c240 tx=0x7fe168005880 comp rx=0 tx=0).stop 2023-08-27T11:17:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.759+0000 7fe185574700 1 -- 172.21.15.112:0/327748852 shutdown_connections 2023-08-27T11:17:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.759+0000 7fe185574700 1 --2- 172.21.15.112:0/327748852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe168003a00 0x7fe168003e20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.759+0000 7fe185574700 1 -- 172.21.15.112:0/327748852 >> 172.21.15.112:0/327748852 conn(0x7fe18006c880 msgr2=0x7fe1800717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.759+0000 7fe185574700 1 -- 172.21.15.112:0/327748852 shutdown_connections 2023-08-27T11:17:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.759+0000 7fe185574700 1 -- 172.21.15.112:0/327748852 wait complete. 2023-08-27T11:17:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.760+0000 7fe185574700 1 Processor -- start 2023-08-27T11:17:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.760+0000 7fe185574700 1 -- start start 2023-08-27T11:17:16.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe185574700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe168003a00 0x7fe18015e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe185574700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe18013daf0 0x7fe18015feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe185574700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1801603f0 0x7fe18013b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe185574700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe180160880 con 0x7fe1801603f0 2023-08-27T11:17:16.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe185574700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18015ea20 con 0x7fe18013daf0 2023-08-27T11:17:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe185574700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18015ed00 con 0x7fe168003a00 2023-08-27T11:17:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe17f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1801603f0 0x7fe18013b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe17effd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe168003a00 0x7fe18015e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe17effd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe168003a00 0x7fe18015e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38706/0 (socket says 172.21.15.112:38706) 2023-08-27T11:17:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe17f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1801603f0 0x7fe18013b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55288/0 (socket says 172.21.15.112:55288) 2023-08-27T11:17:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe17effd700 1 -- 172.21.15.112:0/2869119229 learned_addr learned my addr 172.21.15.112:0/2869119229 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe17e7fc700 1 --2- 172.21.15.112:0/2869119229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe18013daf0 0x7fe18015feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe17f7fe700 1 -- 172.21.15.112:0/2869119229 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe168003a00 msgr2=0x7fe18015e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe17f7fe700 1 --2- 172.21.15.112:0/2869119229 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe168003a00 0x7fe18015e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe17f7fe700 1 -- 172.21.15.112:0/2869119229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe18013daf0 msgr2=0x7fe18015feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe17f7fe700 1 --2- 172.21.15.112:0/2869119229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe18013daf0 0x7fe18015feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe17f7fe700 1 -- 172.21.15.112:0/2869119229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18015efe0 con 0x7fe1801603f0 2023-08-27T11:17:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe17f7fe700 1 --2- 172.21.15.112:0/2869119229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1801603f0 0x7fe18013b820 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe1800d3290 tx=0x7fe178009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe1677fe700 1 -- 172.21.15.112:0/2869119229 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17800e070 con 0x7fe1801603f0 2023-08-27T11:17:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe1677fe700 1 -- 172.21.15.112:0/2869119229 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe178003cf0 con 0x7fe1801603f0 2023-08-27T11:17:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe185574700 1 -- 172.21.15.112:0/2869119229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18015f2c0 con 0x7fe1801603f0 2023-08-27T11:17:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.761+0000 7fe1677fe700 1 -- 172.21.15.112:0/2869119229 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178011730 con 0x7fe1801603f0 2023-08-27T11:17:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.762+0000 7fe185574700 1 -- 172.21.15.112:0/2869119229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18013df70 con 0x7fe1801603f0 2023-08-27T11:17:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.762+0000 7fe185574700 1 -- 172.21.15.112:0/2869119229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18013f500 con 0x7fe1801603f0 2023-08-27T11:17:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.763+0000 7fe1677fe700 1 -- 172.21.15.112:0/2869119229 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe178011890 con 0x7fe1801603f0 2023-08-27T11:17:16.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.763+0000 7fe1677fe700 1 --2- 172.21.15.112:0/2869119229 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe16c06a0e0 0x7fe16c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:16.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.764+0000 7fe1677fe700 1 -- 172.21.15.112:0/2869119229 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe1780887b0 con 0x7fe1801603f0 2023-08-27T11:17:16.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.764+0000 7fe17effd700 1 --2- 172.21.15.112:0/2869119229 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe16c06a0e0 0x7fe16c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:16.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.764+0000 7fe17effd700 1 --2- 172.21.15.112:0/2869119229 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe16c06a0e0 0x7fe16c06c5a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe174015040 tx=0x7fe174007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:16.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.768+0000 7fe1677fe700 1 -- 172.21.15.112:0/2869119229 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe18013f500 con 0x7fe1801603f0 2023-08-27T11:17:17.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.998+0000 7fe185574700 1 -- 172.21.15.112:0/2869119229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe18013f8e0 con 0x7fe1801603f0 2023-08-27T11:17:17.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:16.999+0000 7fe1677fe700 1 -- 172.21.15.112:0/2869119229 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4881 (secure 0 0 0) 0x7fe18013f8e0 con 0x7fe1801603f0 2023-08-27T11:17:17.001 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:17.001 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3289488908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3289488908},{"type":"v1","addr":"172.21.15.112:6835","nonce":3289488908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3991653721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3991653721},{"type":"v1","addr":"172.21.15.112:6834","nonce":3991653721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1923303457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1923303457},{"type":"v1","addr":"172.21.15.138:6836","nonce":1923303457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:17:04.761658+0000","modified":"2023-08-27T11:17:08.372245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/1827312637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1827312637},{"type":"v1","addr":"172.21.15.138:6837","nonce":1827312637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:17:17.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.001+0000 7fe185574700 1 -- 172.21.15.112:0/2869119229 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe16c06a0e0 msgr2=0x7fe16c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.001+0000 7fe185574700 1 --2- 172.21.15.112:0/2869119229 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe16c06a0e0 0x7fe16c06c5a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe174015040 tx=0x7fe174007000 comp rx=0 tx=0).stop 2023-08-27T11:17:17.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.001+0000 7fe185574700 1 -- 172.21.15.112:0/2869119229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1801603f0 msgr2=0x7fe18013b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.001+0000 7fe185574700 1 --2- 172.21.15.112:0/2869119229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1801603f0 0x7fe18013b820 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe1800d3290 tx=0x7fe178009f70 comp rx=0 tx=0).stop 2023-08-27T11:17:17.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.001+0000 7fe185574700 1 -- 172.21.15.112:0/2869119229 shutdown_connections 2023-08-27T11:17:17.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.001+0000 7fe185574700 1 --2- 172.21.15.112:0/2869119229 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe16c06a0e0 0x7fe16c06c5a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.001+0000 7fe185574700 1 --2- 172.21.15.112:0/2869119229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1801603f0 0x7fe18013b820 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.001+0000 7fe185574700 1 --2- 172.21.15.112:0/2869119229 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe168003a00 0x7fe18015e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.001+0000 7fe185574700 1 --2- 172.21.15.112:0/2869119229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe18013daf0 0x7fe18015feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.001+0000 7fe185574700 1 -- 172.21.15.112:0/2869119229 >> 172.21.15.112:0/2869119229 conn(0x7fe18006c880 msgr2=0x7fe1800b4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:17.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.001+0000 7fe185574700 1 -- 172.21.15.112:0/2869119229 shutdown_connections 2023-08-27T11:17:17.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.001+0000 7fe185574700 1 -- 172.21.15.112:0/2869119229 wait complete. 2023-08-27T11:17:17.005 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2023-08-27T11:17:17.015 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:17.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.131+0000 7f2282ee4700 1 Processor -- start 2023-08-27T11:17:17.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.131+0000 7f2282ee4700 1 -- start start 2023-08-27T11:17:17.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.132+0000 7f2282ee4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f227c13f990 0x7f227c0cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.132+0000 7f2282ee4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f227c0cd270 0x7f227c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.132+0000 7f2282ee4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227c13ff50 con 0x7f227c06e150 2023-08-27T11:17:17.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.132+0000 7f2282ee4700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227c0cecb0 con 0x7f227c0cd270 2023-08-27T11:17:17.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.132+0000 7f2282ee4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227c0cf570 con 0x7f227c13f990 2023-08-27T11:17:17.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.132+0000 7f2281481700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f227c0cd270 0x7f227c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.132+0000 7f2280c80700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f227c06e150 0x7f227c06e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49674/0 (socket says 172.21.15.112:49674) 2023-08-27T11:17:17.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.132+0000 7f2280c80700 1 -- 172.21.15.112:0/1334546704 learned_addr learned my addr 172.21.15.112:0/1334546704 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:17.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f227bfff700 1 --2- 172.21.15.112:0/1334546704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f227c13f990 0x7f227c0cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f2281481700 1 -- 172.21.15.112:0/1334546704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f227c13f990 msgr2=0x7f227c0cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:17.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f2281481700 1 --2- 172.21.15.112:0/1334546704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f227c13f990 0x7f227c0cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f2281481700 1 -- 172.21.15.112:0/1334546704 >> v1:172.21.15.112:6789/0 conn(0x7f227c06e150 legacy=0x7f227c06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f2281481700 1 -- 172.21.15.112:0/1334546704 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c140230 con 0x7f227c0cd270 2023-08-27T11:17:17.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f227b7fe700 1 -- 172.21.15.112:0/1334546704 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947154658 0 0) 0x7f227c13ff50 con 0x7f227c06e150 2023-08-27T11:17:17.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f2281481700 1 --2- 172.21.15.112:0/1334546704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f227c0cd270 0x7f227c15ad20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2274009d90 tx=0x7f227400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d58847ac79c55d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:17.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f227b7fe700 1 -- 172.21.15.112:0/1334546704 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227400e070 con 0x7f227c0cd270 2023-08-27T11:17:17.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f227b7fe700 1 -- 172.21.15.112:0/1334546704 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227400bb20 con 0x7f227c0cd270 2023-08-27T11:17:17.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f227b7fe700 1 -- 172.21.15.112:0/1334546704 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274012430 con 0x7f227c0cd270 2023-08-27T11:17:17.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f2282ee4700 1 -- 172.21.15.112:0/1334546704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f227c0cd270 msgr2=0x7f227c15ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f2282ee4700 1 --2- 172.21.15.112:0/1334546704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f227c0cd270 0x7f227c15ad20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2274009d90 tx=0x7f227400b330 comp rx=0 tx=0).stop 2023-08-27T11:17:17.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f2282ee4700 1 -- 172.21.15.112:0/1334546704 shutdown_connections 2023-08-27T11:17:17.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f2282ee4700 1 --2- 172.21.15.112:0/1334546704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f227c0cd270 0x7f227c15ad20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.133+0000 7f2282ee4700 1 --2- 172.21.15.112:0/1334546704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f227c13f990 0x7f227c0cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.134+0000 7f2282ee4700 1 -- 172.21.15.112:0/1334546704 >> 172.21.15.112:0/1334546704 conn(0x7f227c06cd30 msgr2=0x7f227c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:17.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.134+0000 7f2282ee4700 1 -- 172.21.15.112:0/1334546704 shutdown_connections 2023-08-27T11:17:17.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.134+0000 7f2282ee4700 1 -- 172.21.15.112:0/1334546704 wait complete. 2023-08-27T11:17:17.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.135+0000 7f2282ee4700 1 Processor -- start 2023-08-27T11:17:17.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.135+0000 7f2282ee4700 1 -- start start 2023-08-27T11:17:17.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2282ee4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227c06e150 0x7f227c1f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2282ee4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f227c0cd270 0x7f227c1e4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2282ee4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f227c13f990 0x7f227c1e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2282ee4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c13d710 con 0x7f227c13f990 2023-08-27T11:17:17.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2282ee4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c0cbda0 con 0x7f227c06e150 2023-08-27T11:17:17.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2282ee4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227c1e54c0 con 0x7f227c0cd270 2023-08-27T11:17:17.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2281481700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f227c13f990 0x7f227c1e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2281481700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f227c13f990 0x7f227c1e4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55302/0 (socket says 172.21.15.112:55302) 2023-08-27T11:17:17.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2281481700 1 -- 172.21.15.112:0/1083693947 learned_addr learned my addr 172.21.15.112:0/1083693947 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:17.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f227bfff700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f227c0cd270 0x7f227c1e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2280c80700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227c06e150 0x7f227c1f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2281481700 1 -- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f227c0cd270 msgr2=0x7f227c1e4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2281481700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f227c0cd270 0x7f227c1e4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2281481700 1 -- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227c06e150 msgr2=0x7f227c1f25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2281481700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227c06e150 0x7f227c1f25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2281481700 1 -- 172.21.15.112:0/1083693947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2274009a70 con 0x7f227c13f990 2023-08-27T11:17:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f227bfff700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f227c0cd270 0x7f227c1e4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2281481700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f227c13f990 0x7f227c1e4e40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2274014040 tx=0x7f2274004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:17.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f22797fa700 1 -- 172.21.15.112:0/1083693947 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274012430 con 0x7f227c13f990 2023-08-27T11:17:17.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f22797fa700 1 -- 172.21.15.112:0/1083693947 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2274020070 con 0x7f227c13f990 2023-08-27T11:17:17.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f22797fa700 1 -- 172.21.15.112:0/1083693947 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274004280 con 0x7f227c13f990 2023-08-27T11:17:17.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2282ee4700 1 -- 172.21.15.112:0/1083693947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227c1e57a0 con 0x7f227c13f990 2023-08-27T11:17:17.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.136+0000 7f2282ee4700 1 -- 172.21.15.112:0/1083693947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227c13dc00 con 0x7f227c13f990 2023-08-27T11:17:17.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.137+0000 7f2282ee4700 1 -- 172.21.15.112:0/1083693947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227c067a20 con 0x7f227c13f990 2023-08-27T11:17:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.138+0000 7f22797fa700 1 -- 172.21.15.112:0/1083693947 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f22740043e0 con 0x7f227c13f990 2023-08-27T11:17:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.138+0000 7f22797fa700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f226006a0e0 0x7f226006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.138+0000 7f22797fa700 1 -- 172.21.15.112:0/1083693947 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2274022070 con 0x7f227c13f990 2023-08-27T11:17:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.138+0000 7f2280c80700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f226006a0e0 0x7f226006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.139+0000 7f2280c80700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f226006a0e0 0x7f226006c5a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2270000f60 tx=0x7f2270007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:17.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.143+0000 7f22797fa700 1 -- 172.21.15.112:0/1083693947 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f22740559e0 con 0x7f227c13f990 2023-08-27T11:17:17.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.334+0000 7f2282ee4700 1 -- 172.21.15.112:0/1083693947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f227c13aca0 con 0x7f227c13f990 2023-08-27T11:17:17.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.335+0000 7f22797fa700 1 -- 172.21.15.112:0/1083693947 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7f2274059000 con 0x7f227c13f990 2023-08-27T11:17:17.337 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:17.337 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:17.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.337+0000 7f2282ee4700 1 -- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f226006a0e0 msgr2=0x7f226006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.337+0000 7f2282ee4700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f226006a0e0 0x7f226006c5a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2270000f60 tx=0x7f2270007e70 comp rx=0 tx=0).stop 2023-08-27T11:17:17.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.337+0000 7f2282ee4700 1 -- 172.21.15.112:0/1083693947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f227c13f990 msgr2=0x7f227c1e4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.337+0000 7f2282ee4700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f227c13f990 0x7f227c1e4e40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2274014040 tx=0x7f2274004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.337+0000 7f2282ee4700 1 -- 172.21.15.112:0/1083693947 shutdown_connections 2023-08-27T11:17:17.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.337+0000 7f2282ee4700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f226006a0e0 0x7f226006c5a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.337+0000 7f2282ee4700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f227c13f990 0x7f227c1e4e40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.337+0000 7f2282ee4700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227c06e150 0x7f227c1f25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.337+0000 7f2282ee4700 1 --2- 172.21.15.112:0/1083693947 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f227c0cd270 0x7f227c1e4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.337+0000 7f2282ee4700 1 -- 172.21.15.112:0/1083693947 >> 172.21.15.112:0/1083693947 conn(0x7f227c06cd30 msgr2=0x7f227c13bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:17.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.337+0000 7f2282ee4700 1 -- 172.21.15.112:0/1083693947 shutdown_connections 2023-08-27T11:17:17.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.337+0000 7f2282ee4700 1 -- 172.21.15.112:0/1083693947 wait complete. 2023-08-27T11:17:17.351 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:17:02.484063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:17:03.722976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:17:17.352 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:17:17.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.474+0000 7f49a332b700 1 Processor -- start 2023-08-27T11:17:17.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.474+0000 7f49a332b700 1 -- start start 2023-08-27T11:17:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.475+0000 7f49a332b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f499c146c20 0x7f499c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.475+0000 7f49a332b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f499c06f790 0x7f499c06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.475+0000 7f49a332b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c0b5070 con 0x7f499c0b3440 2023-08-27T11:17:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.475+0000 7f49a332b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499c070160 con 0x7f499c06f790 2023-08-27T11:17:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.475+0000 7f49a332b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c147000 con 0x7f499c146c20 2023-08-27T11:17:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.475+0000 7f49a18c8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f499c06f790 0x7f499c06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.475+0000 7f49a08c6700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f499c0b3440 0x7f499c15e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49688/0 (socket says 172.21.15.112:49688) 2023-08-27T11:17:17.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.475+0000 7f49a08c6700 1 -- 172.21.15.112:0/86009189 learned_addr learned my addr 172.21.15.112:0/86009189 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:17.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f49a10c7700 1 --2- 172.21.15.112:0/86009189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f499c146c20 0x7f499c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f49a18c8700 1 -- 172.21.15.112:0/86009189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f499c146c20 msgr2=0x7f499c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:17.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f49a18c8700 1 --2- 172.21.15.112:0/86009189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f499c146c20 0x7f499c160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f49a18c8700 1 -- 172.21.15.112:0/86009189 >> v1:172.21.15.112:6789/0 conn(0x7f499c0b3440 legacy=0x7f499c15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f49a18c8700 1 -- 172.21.15.112:0/86009189 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c1614a0 con 0x7f499c06f790 2023-08-27T11:17:17.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f4993fff700 1 -- 172.21.15.112:0/86009189 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306796644 0 0) 0x7f4998001110 con 0x7f499c0b3440 2023-08-27T11:17:17.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f49a18c8700 1 --2- 172.21.15.112:0/86009189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f499c06f790 0x7f499c06fc20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4994009d90 tx=0x7f499400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c5102c2a3f8994 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:17.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f4993fff700 1 -- 172.21.15.112:0/86009189 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499400e070 con 0x7f499c06f790 2023-08-27T11:17:17.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f4993fff700 1 -- 172.21.15.112:0/86009189 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499400bbd0 con 0x7f499c06f790 2023-08-27T11:17:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f4993fff700 1 -- 172.21.15.112:0/86009189 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994012430 con 0x7f499c06f790 2023-08-27T11:17:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f49a332b700 1 -- 172.21.15.112:0/86009189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f499c06f790 msgr2=0x7f499c06fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f49a332b700 1 --2- 172.21.15.112:0/86009189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f499c06f790 0x7f499c06fc20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4994009d90 tx=0x7f499400b330 comp rx=0 tx=0).stop 2023-08-27T11:17:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f49a332b700 1 -- 172.21.15.112:0/86009189 shutdown_connections 2023-08-27T11:17:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f49a332b700 1 --2- 172.21.15.112:0/86009189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f499c06f790 0x7f499c06fc20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f49a332b700 1 --2- 172.21.15.112:0/86009189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f499c146c20 0x7f499c160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.476+0000 7f49a332b700 1 -- 172.21.15.112:0/86009189 >> 172.21.15.112:0/86009189 conn(0x7f499c0b1e90 msgr2=0x7f499c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.477+0000 7f49a332b700 1 -- 172.21.15.112:0/86009189 shutdown_connections 2023-08-27T11:17:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.477+0000 7f49a332b700 1 -- 172.21.15.112:0/86009189 wait complete. 2023-08-27T11:17:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.478+0000 7f49a332b700 1 Processor -- start 2023-08-27T11:17:17.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.478+0000 7f49a332b700 1 -- start start 2023-08-27T11:17:17.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.478+0000 7f49a332b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f499c06f790 0x7f499c15f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.478+0000 7f49a332b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f499c0b3440 0x7f499c0adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.478+0000 7f49a332b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f499c146c20 0x7f499c145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.478+0000 7f49a332b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c140160 con 0x7f499c06f790 2023-08-27T11:17:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.478+0000 7f49a332b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c1403b0 con 0x7f499c146c20 2023-08-27T11:17:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.478+0000 7f49a332b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c140690 con 0x7f499c0b3440 2023-08-27T11:17:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f49a10c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f499c06f790 0x7f499c15f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f49a10c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f499c06f790 0x7f499c15f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55310/0 (socket says 172.21.15.112:55310) 2023-08-27T11:17:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f49a10c7700 1 -- 172.21.15.112:0/2455023329 learned_addr learned my addr 172.21.15.112:0/2455023329 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f49a10c7700 1 -- 172.21.15.112:0/2455023329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f499c0b3440 msgr2=0x7f499c0adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f49a18c8700 1 --2- 172.21.15.112:0/2455023329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f499c146c20 0x7f499c145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f49a10c7700 1 --2- 172.21.15.112:0/2455023329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f499c0b3440 0x7f499c0adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f49a10c7700 1 -- 172.21.15.112:0/2455023329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f499c146c20 msgr2=0x7f499c145390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f49a10c7700 1 --2- 172.21.15.112:0/2455023329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f499c146c20 0x7f499c145390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f49a10c7700 1 -- 172.21.15.112:0/2455023329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994009a70 con 0x7f499c06f790 2023-08-27T11:17:17.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f49a10c7700 1 --2- 172.21.15.112:0/2455023329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f499c06f790 0x7f499c15f8c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f498c00c7f0 tx=0x7f498c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:17.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f4991ffb700 1 -- 172.21.15.112:0/2455023329 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c011800 con 0x7f499c06f790 2023-08-27T11:17:17.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f4991ffb700 1 -- 172.21.15.112:0/2455023329 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498c011960 con 0x7f499c06f790 2023-08-27T11:17:17.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f49a332b700 1 -- 172.21.15.112:0/2455023329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c140970 con 0x7f499c06f790 2023-08-27T11:17:17.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f4991ffb700 1 -- 172.21.15.112:0/2455023329 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c011cf0 con 0x7f499c06f790 2023-08-27T11:17:17.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.479+0000 7f49a332b700 1 -- 172.21.15.112:0/2455023329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c140f30 con 0x7f499c06f790 2023-08-27T11:17:17.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.480+0000 7f49a332b700 1 -- 172.21.15.112:0/2455023329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c067e90 con 0x7f499c06f790 2023-08-27T11:17:17.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.481+0000 7f4991ffb700 1 -- 172.21.15.112:0/2455023329 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f498c018360 con 0x7f499c06f790 2023-08-27T11:17:17.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.481+0000 7f4991ffb700 1 --2- 172.21.15.112:0/2455023329 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f498006a0e0 0x7f498006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.481+0000 7f4991ffb700 1 -- 172.21.15.112:0/2455023329 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f498c088460 con 0x7f499c06f790 2023-08-27T11:17:17.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.481+0000 7f49a08c6700 1 --2- 172.21.15.112:0/2455023329 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f498006a0e0 0x7f498006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.482+0000 7f49a08c6700 1 --2- 172.21.15.112:0/2455023329 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f498006a0e0 0x7f498006c5a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f499c160970 tx=0x7f4998001310 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:17.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.485+0000 7f4991ffb700 1 -- 172.21.15.112:0/2455023329 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f498c053b50 con 0x7f499c06f790 2023-08-27T11:17:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.671+0000 7f49a332b700 1 -- 172.21.15.112:0/2455023329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f499c141120 con 0x7f499c06f790 2023-08-27T11:17:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.672+0000 7f4991ffb700 1 -- 172.21.15.112:0/2455023329 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f498c057170 con 0x7f499c06f790 2023-08-27T11:17:17.673 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.674+0000 7f49a332b700 1 -- 172.21.15.112:0/2455023329 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f498006a0e0 msgr2=0x7f498006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.674+0000 7f49a332b700 1 --2- 172.21.15.112:0/2455023329 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f498006a0e0 0x7f498006c5a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f499c160970 tx=0x7f4998001310 comp rx=0 tx=0).stop 2023-08-27T11:17:17.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.674+0000 7f49a332b700 1 -- 172.21.15.112:0/2455023329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f499c06f790 msgr2=0x7f499c15f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.674+0000 7f49a332b700 1 --2- 172.21.15.112:0/2455023329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f499c06f790 0x7f499c15f8c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f498c00c7f0 tx=0x7f498c00cbb0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.674+0000 7f49a332b700 1 -- 172.21.15.112:0/2455023329 shutdown_connections 2023-08-27T11:17:17.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.674+0000 7f49a332b700 1 --2- 172.21.15.112:0/2455023329 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f498006a0e0 0x7f498006c5a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.674+0000 7f49a332b700 1 --2- 172.21.15.112:0/2455023329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f499c146c20 0x7f499c145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.674+0000 7f49a332b700 1 --2- 172.21.15.112:0/2455023329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f499c06f790 0x7f499c15f8c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.674+0000 7f49a332b700 1 --2- 172.21.15.112:0/2455023329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f499c0b3440 0x7f499c0adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.674+0000 7f49a332b700 1 -- 172.21.15.112:0/2455023329 >> 172.21.15.112:0/2455023329 conn(0x7f499c0b1e90 msgr2=0x7f499c071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.674+0000 7f49a332b700 1 -- 172.21.15.112:0/2455023329 shutdown_connections 2023-08-27T11:17:17.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.674+0000 7f49a332b700 1 -- 172.21.15.112:0/2455023329 wait complete. 2023-08-27T11:17:17.687 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:17.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.808+0000 7f764691f700 1 Processor -- start 2023-08-27T11:17:17.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.808+0000 7f764691f700 1 -- start start 2023-08-27T11:17:17.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f764691f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7640146c20 0x7f7640160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f764691f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f76400b3440 0x7f764015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f764691f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76400b5070 con 0x7f764015e6e0 2023-08-27T11:17:17.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f764691f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7640147000 con 0x7f76400b3440 2023-08-27T11:17:17.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f764691f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76401614a0 con 0x7f7640146c20 2023-08-27T11:17:17.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f763ffff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7640146c20 0x7f7640160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f763ffff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7640146c20 0x7f7640160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38740/0 (socket says 172.21.15.112:38740) 2023-08-27T11:17:17.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f763f7fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f76400b3440 0x7f764015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f763ffff700 1 -- 172.21.15.112:0/3577475435 learned_addr learned my addr 172.21.15.112:0/3577475435 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:17.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f763f7fe700 1 -- 172.21.15.112:0/3577475435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7640146c20 msgr2=0x7f7640160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:17.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f763f7fe700 1 --2- 172.21.15.112:0/3577475435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7640146c20 0x7f7640160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f763f7fe700 1 -- 172.21.15.112:0/3577475435 >> v1:172.21.15.138:6789/0 conn(0x7f764015e6e0 legacy=0x7f764006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f763f7fe700 1 -- 172.21.15.112:0/3577475435 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7640161740 con 0x7f76400b3440 2023-08-27T11:17:17.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.809+0000 7f763ffff700 1 --2- 172.21.15.112:0/3577475435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7640146c20 0x7f7640160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:17.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.810+0000 7f763f7fe700 1 --2- 172.21.15.112:0/3577475435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76400b3440 0x7f764015e1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7630009840 tx=0x7f7630009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=331da2e1f0eb730f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:17.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.810+0000 7f763effd700 1 -- 172.21.15.112:0/3577475435 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630011830 con 0x7f76400b3440 2023-08-27T11:17:17.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.810+0000 7f763effd700 1 -- 172.21.15.112:0/3577475435 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7630011990 con 0x7f76400b3440 2023-08-27T11:17:17.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.810+0000 7f763effd700 1 -- 172.21.15.112:0/3577475435 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630011d20 con 0x7f76400b3440 2023-08-27T11:17:17.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.810+0000 7f764691f700 1 -- 172.21.15.112:0/3577475435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76400b3440 msgr2=0x7f764015e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.810+0000 7f764691f700 1 --2- 172.21.15.112:0/3577475435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76400b3440 0x7f764015e1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7630009840 tx=0x7f7630009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:17.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.810+0000 7f764691f700 1 -- 172.21.15.112:0/3577475435 shutdown_connections 2023-08-27T11:17:17.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.810+0000 7f764691f700 1 --2- 172.21.15.112:0/3577475435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7640146c20 0x7f7640160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.810+0000 7f764691f700 1 --2- 172.21.15.112:0/3577475435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76400b3440 0x7f764015e1a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.810+0000 7f764691f700 1 -- 172.21.15.112:0/3577475435 >> 172.21.15.112:0/3577475435 conn(0x7f76400b1e90 msgr2=0x7f76400b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:17.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.810+0000 7f764691f700 1 -- 172.21.15.112:0/3577475435 shutdown_connections 2023-08-27T11:17:17.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.810+0000 7f764691f700 1 -- 172.21.15.112:0/3577475435 wait complete. 2023-08-27T11:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.811+0000 7f764691f700 1 Processor -- start 2023-08-27T11:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.812+0000 7f764691f700 1 -- start start 2023-08-27T11:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.812+0000 7f764691f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76400b3440 0x7f764013d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.812+0000 7f764691f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7640146c20 0x7f764015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.812+0000 7f764691f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764015e6e0 0x7f764015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.812+0000 7f764691f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764013ef70 con 0x7f764015e6e0 2023-08-27T11:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.812+0000 7f764691f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764015d6b0 con 0x7f7640146c20 2023-08-27T11:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.812+0000 7f764691f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764013e9f0 con 0x7f76400b3440 2023-08-27T11:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f7644ebc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764015e6e0 0x7f764015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f7644ebc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764015e6e0 0x7f764015bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55328/0 (socket says 172.21.15.112:55328) 2023-08-27T11:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f7644ebc700 1 -- 172.21.15.112:0/2473768938 learned_addr learned my addr 172.21.15.112:0/2473768938 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f763ffff700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76400b3440 0x7f764013d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f763f7fe700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7640146c20 0x7f764015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f7644ebc700 1 -- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76400b3440 msgr2=0x7f764013d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f7644ebc700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76400b3440 0x7f764013d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f7644ebc700 1 -- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7640146c20 msgr2=0x7f764015b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f7644ebc700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7640146c20 0x7f764015b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f7644ebc700 1 -- 172.21.15.112:0/2473768938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7634009550 con 0x7f764015e6e0 2023-08-27T11:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f7644ebc700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764015e6e0 0x7f764015bab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f763800cb80 tx=0x7f763800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f763cff9700 1 -- 172.21.15.112:0/2473768938 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638003c00 con 0x7f764015e6e0 2023-08-27T11:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f763cff9700 1 -- 172.21.15.112:0/2473768938 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7638003d60 con 0x7f764015e6e0 2023-08-27T11:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f764691f700 1 -- 172.21.15.112:0/2473768938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76300094a0 con 0x7f764015e6e0 2023-08-27T11:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f763cff9700 1 -- 172.21.15.112:0/2473768938 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763800abb0 con 0x7f764015e6e0 2023-08-27T11:17:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.813+0000 7f764691f700 1 -- 172.21.15.112:0/2473768938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764015d340 con 0x7f764015e6e0 2023-08-27T11:17:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.814+0000 7f764691f700 1 -- 172.21.15.112:0/2473768938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7640067e90 con 0x7f764015e6e0 2023-08-27T11:17:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.815+0000 7f763cff9700 1 -- 172.21.15.112:0/2473768938 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f763800ad10 con 0x7f764015e6e0 2023-08-27T11:17:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.815+0000 7f763cff9700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f762c06a0e0 0x7f762c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.815+0000 7f763cff9700 1 -- 172.21.15.112:0/2473768938 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f76380883a0 con 0x7f764015e6e0 2023-08-27T11:17:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.815+0000 7f763ffff700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f762c06a0e0 0x7f762c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.815+0000 7f763ffff700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f762c06a0e0 0x7f762c06c5a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7634009cf0 tx=0x7f7634008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:17.819+0000 7f763cff9700 1 -- 172.21.15.112:0/2473768938 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7638053b20 con 0x7f764015e6e0 2023-08-27T11:17:18.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.006+0000 7f764691f700 1 -- 172.21.15.112:0/2473768938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7640161cd0 con 0x7f764015e6e0 2023-08-27T11:17:18.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.007+0000 7f763cff9700 1 -- 172.21.15.112:0/2473768938 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7638057140 con 0x7f764015e6e0 2023-08-27T11:17:18.009 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:18.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.008+0000 7f764691f700 1 -- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f762c06a0e0 msgr2=0x7f762c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.008+0000 7f764691f700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f762c06a0e0 0x7f762c06c5a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7634009cf0 tx=0x7f7634008040 comp rx=0 tx=0).stop 2023-08-27T11:17:18.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.008+0000 7f764691f700 1 -- 172.21.15.112:0/2473768938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764015e6e0 msgr2=0x7f764015bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.008+0000 7f764691f700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764015e6e0 0x7f764015bab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f763800cb80 tx=0x7f763800cf40 comp rx=0 tx=0).stop 2023-08-27T11:17:18.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.009+0000 7f764691f700 1 -- 172.21.15.112:0/2473768938 shutdown_connections 2023-08-27T11:17:18.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.009+0000 7f764691f700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f762c06a0e0 0x7f762c06c5a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.009+0000 7f764691f700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764015e6e0 0x7f764015bab0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.009+0000 7f764691f700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76400b3440 0x7f764013d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.009+0000 7f764691f700 1 --2- 172.21.15.112:0/2473768938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7640146c20 0x7f764015b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.009+0000 7f764691f700 1 -- 172.21.15.112:0/2473768938 >> 172.21.15.112:0/2473768938 conn(0x7f76400b1e90 msgr2=0x7f76400b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:18.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.009+0000 7f764691f700 1 -- 172.21.15.112:0/2473768938 shutdown_connections 2023-08-27T11:17:18.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.009+0000 7f764691f700 1 -- 172.21.15.112:0/2473768938 wait complete. 2023-08-27T11:17:18.022 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:17:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.144+0000 7f063af79700 1 Processor -- start 2023-08-27T11:17:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.144+0000 7f063af79700 1 -- start start 2023-08-27T11:17:18.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f063af79700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0634146c20 0x7f0634160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f063af79700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06340b5070 con 0x7f063406f790 2023-08-27T11:17:18.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f063af79700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0634147000 con 0x7f06340b3440 2023-08-27T11:17:18.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f063af79700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06341614a0 con 0x7f0634146c20 2023-08-27T11:17:18.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f0639516700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f063406f790 0x7f06340b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49718/0 (socket says 172.21.15.112:49718) 2023-08-27T11:17:18.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f0639516700 1 -- 172.21.15.112:0/2554216692 learned_addr learned my addr 172.21.15.112:0/2554216692 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:18.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175499916 0 0) 0x7f062c001490 con 0x7f063406f790 2023-08-27T11:17:18.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06240035b0 con 0x7f063406f790 2023-08-27T11:17:18.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f0638d15700 1 --2- 172.21.15.112:0/2554216692 >> v2:172.21.15.138:3300/0 conn(0x7f0634146c20 0x7f0634160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1424597385 0 0) 0x7f06240035b0 con 0x7f063406f790 2023-08-27T11:17:18.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 >> v2:172.21.15.138:3300/0 conn(0x7f0634146c20 msgr2=0x7f0634160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:18.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f06337fe700 1 --2- 172.21.15.112:0/2554216692 >> v2:172.21.15.138:3300/0 conn(0x7f0634146c20 0x7f0634160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 >> v1:172.21.15.138:6789/0 conn(0x7f06340b3440 legacy=0x7f063415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0634161740 con 0x7f063406f790 2023-08-27T11:17:18.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f062c001b80 con 0x7f063406f790 2023-08-27T11:17:18.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 >> v1:172.21.15.112:6789/0 conn(0x7f063406f790 legacy=0x7f06340b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.145+0000 7f06337fe700 1 --2- 172.21.15.112:0/2554216692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0624003a00 0x7f0624003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f06337fe700 1 --2- 172.21.15.112:0/2554216692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0624004850 0x7f0624004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f06337fe700 1 --2- 172.21.15.112:0/2554216692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0624005210 0x7f062400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06341614a0 con 0x7f0624003a00 2023-08-27T11:17:18.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062400c1b0 con 0x7f0624005210 2023-08-27T11:17:18.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062400c370 con 0x7f0624004850 2023-08-27T11:17:18.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f0638d15700 1 --2- 172.21.15.112:0/2554216692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0624003a00 0x7f0624003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f0639516700 1 --2- 172.21.15.112:0/2554216692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0624005210 0x7f062400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f0633fff700 1 --2- 172.21.15.112:0/2554216692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0624004850 0x7f0624004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f0638d15700 1 -- 172.21.15.112:0/2554216692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0624004850 msgr2=0x7f0624004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f0638d15700 1 --2- 172.21.15.112:0/2554216692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0624004850 0x7f0624004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f0638d15700 1 -- 172.21.15.112:0/2554216692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0624005210 msgr2=0x7f062400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f0638d15700 1 --2- 172.21.15.112:0/2554216692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0624005210 0x7f062400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f0639516700 1 -- 172.21.15.112:0/2554216692 reap_dead start 2023-08-27T11:17:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f0638d15700 1 -- 172.21.15.112:0/2554216692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06240035b0 con 0x7f0624003a00 2023-08-27T11:17:18.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f0638d15700 1 --2- 172.21.15.112:0/2554216692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0624003a00 0x7f0624003e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0628009b60 tx=0x7f0624005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b97b287ba35a1ae server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:18.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062800e6c0 con 0x7f0624003a00 2023-08-27T11:17:18.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062800e820 con 0x7f0624003a00 2023-08-27T11:17:18.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.146+0000 7f06337fe700 1 -- 172.21.15.112:0/2554216692 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062800eb90 con 0x7f0624003a00 2023-08-27T11:17:18.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.147+0000 7f063af79700 1 -- 172.21.15.112:0/2554216692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0624003a00 msgr2=0x7f0624003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.147+0000 7f063af79700 1 --2- 172.21.15.112:0/2554216692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0624003a00 0x7f0624003e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0628009b60 tx=0x7f0624005880 comp rx=0 tx=0).stop 2023-08-27T11:17:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.147+0000 7f063af79700 1 -- 172.21.15.112:0/2554216692 shutdown_connections 2023-08-27T11:17:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.147+0000 7f063af79700 1 --2- 172.21.15.112:0/2554216692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0624003a00 0x7f0624003e20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.147+0000 7f063af79700 1 -- 172.21.15.112:0/2554216692 >> 172.21.15.112:0/2554216692 conn(0x7f06340b1e90 msgr2=0x7f06340b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.147+0000 7f063af79700 1 -- 172.21.15.112:0/2554216692 shutdown_connections 2023-08-27T11:17:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.147+0000 7f063af79700 1 -- 172.21.15.112:0/2554216692 wait complete. 2023-08-27T11:17:18.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.148+0000 7f063af79700 1 Processor -- start 2023-08-27T11:17:18.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.148+0000 7f063af79700 1 -- start start 2023-08-27T11:17:18.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f063af79700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0624003a00 0x7f06340af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f063af79700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06340ad4c0 0x7f063415f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f063af79700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f063415a710 0x7f063415abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f063af79700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06340ae500 con 0x7f0624003a00 2023-08-27T11:17:18.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f063af79700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06341332a0 con 0x7f06340ad4c0 2023-08-27T11:17:18.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f063af79700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063415b100 con 0x7f063415a710 2023-08-27T11:17:18.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f0638d15700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0624003a00 0x7f06340af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f0638d15700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0624003a00 0x7f06340af430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55344/0 (socket says 172.21.15.112:55344) 2023-08-27T11:17:18.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f0638d15700 1 -- 172.21.15.112:0/2631429177 learned_addr learned my addr 172.21.15.112:0/2631429177 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:18.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f0633fff700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06340ad4c0 0x7f063415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f0639516700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f063415a710 0x7f063415abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f0638d15700 1 -- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f063415a710 msgr2=0x7f063415abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f0638d15700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f063415a710 0x7f063415abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f0638d15700 1 -- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06340ad4c0 msgr2=0x7f063415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f0638d15700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06340ad4c0 0x7f063415f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f0638d15700 1 -- 172.21.15.112:0/2631429177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06340adc60 con 0x7f0624003a00 2023-08-27T11:17:18.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f0639516700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f063415a710 0x7f063415abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:18.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f0638d15700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0624003a00 0x7f06340af430 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f062800e4b0 tx=0x7f06280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:18.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f06317fa700 1 -- 172.21.15.112:0/2631429177 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062800e6c0 con 0x7f0624003a00 2023-08-27T11:17:18.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f06317fa700 1 -- 172.21.15.112:0/2631429177 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0628003ad0 con 0x7f0624003a00 2023-08-27T11:17:18.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f06317fa700 1 -- 172.21.15.112:0/2631429177 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628003e80 con 0x7f0624003a00 2023-08-27T11:17:18.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.149+0000 7f063af79700 1 -- 172.21.15.112:0/2631429177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06340adf40 con 0x7f0624003a00 2023-08-27T11:17:18.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.150+0000 7f063af79700 1 -- 172.21.15.112:0/2631429177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0634142b30 con 0x7f0624003a00 2023-08-27T11:17:18.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.150+0000 7f063af79700 1 -- 172.21.15.112:0/2631429177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0634142990 con 0x7f0624003a00 2023-08-27T11:17:18.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.151+0000 7f06317fa700 1 -- 172.21.15.112:0/2631429177 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0628013470 con 0x7f0624003a00 2023-08-27T11:17:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.151+0000 7f06317fa700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f061806a0e0 0x7f061806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.151+0000 7f06317fa700 1 -- 172.21.15.112:0/2631429177 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0628089c50 con 0x7f0624003a00 2023-08-27T11:17:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.151+0000 7f0633fff700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f061806a0e0 0x7f061806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.152+0000 7f0633fff700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f061806a0e0 0x7f061806c5a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0634144680 tx=0x7f0620009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.155+0000 7f06317fa700 1 -- 172.21.15.112:0/2631429177 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0634142990 con 0x7f0624003a00 2023-08-27T11:17:18.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.343+0000 7f063af79700 1 -- 172.21.15.112:0/2631429177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0634142990 con 0x7f0624003a00 2023-08-27T11:17:18.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.343+0000 7f06317fa700 1 -- 172.21.15.112:0/2631429177 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0634142990 con 0x7f0624003a00 2023-08-27T11:17:18.345 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:17:18.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.345+0000 7f063af79700 1 -- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f061806a0e0 msgr2=0x7f061806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.345+0000 7f063af79700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f061806a0e0 0x7f061806c5a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0634144680 tx=0x7f0620009040 comp rx=0 tx=0).stop 2023-08-27T11:17:18.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.345+0000 7f063af79700 1 -- 172.21.15.112:0/2631429177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0624003a00 msgr2=0x7f06340af430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.345+0000 7f063af79700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0624003a00 0x7f06340af430 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f062800e4b0 tx=0x7f06280047c0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.346+0000 7f063af79700 1 -- 172.21.15.112:0/2631429177 shutdown_connections 2023-08-27T11:17:18.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.346+0000 7f063af79700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f061806a0e0 0x7f061806c5a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.346+0000 7f063af79700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f063415a710 0x7f063415abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.346+0000 7f063af79700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0624003a00 0x7f06340af430 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.346+0000 7f063af79700 1 --2- 172.21.15.112:0/2631429177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06340ad4c0 0x7f063415f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.346+0000 7f063af79700 1 -- 172.21.15.112:0/2631429177 >> 172.21.15.112:0/2631429177 conn(0x7f06340b1e90 msgr2=0x7f06340b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:18.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.346+0000 7f063af79700 1 -- 172.21.15.112:0/2631429177 shutdown_connections 2023-08-27T11:17:18.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.346+0000 7f063af79700 1 -- 172.21.15.112:0/2631429177 wait complete. 2023-08-27T11:17:18.359 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:17:18.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.481+0000 7f08bf271700 1 Processor -- start 2023-08-27T11:17:18.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.482+0000 7f08bf271700 1 -- start start 2023-08-27T11:17:18.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.482+0000 7f08bf271700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f08b806dcc0 0x7f08b80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.482+0000 7f08bf271700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b80cfd50 con 0x7f08b8143ba0 2023-08-27T11:17:18.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.482+0000 7f08bf271700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b80cf140 con 0x7f08b81437c0 2023-08-27T11:17:18.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.482+0000 7f08bf271700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b80cf420 con 0x7f08b806dcc0 2023-08-27T11:17:18.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.482+0000 7f08bc80c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f08b8143ba0 0x7f08b806d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49736/0 (socket says 172.21.15.112:49736) 2023-08-27T11:17:18.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.482+0000 7f08bc80c700 1 -- 172.21.15.112:0/4041144519 learned_addr learned my addr 172.21.15.112:0/4041144519 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:18.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08bd80e700 1 --2- 172.21.15.112:0/4041144519 >> v2:172.21.15.138:3300/0 conn(0x7f08b806dcc0 0x7f08b80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189586755 0 0) 0x7f08b80cfd50 con 0x7f08b8143ba0 2023-08-27T11:17:18.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08a40035b0 con 0x7f08b8143ba0 2023-08-27T11:17:18.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2808772905 0 0) 0x7f08a40035b0 con 0x7f08b8143ba0 2023-08-27T11:17:18.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 >> v2:172.21.15.138:3300/0 conn(0x7f08b806dcc0 msgr2=0x7f08b80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:18.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 --2- 172.21.15.112:0/4041144519 >> v2:172.21.15.138:3300/0 conn(0x7f08b806dcc0 0x7f08b80b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 >> v1:172.21.15.138:6789/0 conn(0x7f08b81437c0 legacy=0x7f08b80ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b80cf700 con 0x7f08b8143ba0 2023-08-27T11:17:18.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f08b4001670 con 0x7f08b8143ba0 2023-08-27T11:17:18.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 >> v1:172.21.15.112:6789/0 conn(0x7f08b8143ba0 legacy=0x7f08b806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 --2- 172.21.15.112:0/4041144519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a4003a00 0x7f08a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 --2- 172.21.15.112:0/4041144519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08a4004850 0x7f08a4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 --2- 172.21.15.112:0/4041144519 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08a4005210 0x7f08a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80cf420 con 0x7f08a4003a00 2023-08-27T11:17:18.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a400c0b0 con 0x7f08a4004850 2023-08-27T11:17:18.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a400c270 con 0x7f08a4005210 2023-08-27T11:17:18.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08bd00d700 1 --2- 172.21.15.112:0/4041144519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a4003a00 0x7f08a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08bc80c700 1 --2- 172.21.15.112:0/4041144519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08a4004850 0x7f08a4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08bd80e700 1 --2- 172.21.15.112:0/4041144519 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08a4005210 0x7f08a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08bd00d700 1 -- 172.21.15.112:0/4041144519 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08a4005210 msgr2=0x7f08a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:18.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08bd00d700 1 --2- 172.21.15.112:0/4041144519 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08a4005210 0x7f08a400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08bd00d700 1 -- 172.21.15.112:0/4041144519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08a4004850 msgr2=0x7f08a4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:18.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08bd00d700 1 --2- 172.21.15.112:0/4041144519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08a4004850 0x7f08a4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08bd80e700 1 -- 172.21.15.112:0/4041144519 reap_dead start 2023-08-27T11:17:18.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.483+0000 7f08bd00d700 1 -- 172.21.15.112:0/4041144519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a40035b0 con 0x7f08a4003a00 2023-08-27T11:17:18.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.484+0000 7f08bd00d700 1 --2- 172.21.15.112:0/4041144519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a4003a00 0x7f08a4003e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f08a80017c0 tx=0x7f08a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15d1c5c2a70add6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:18.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.484+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a80052d0 con 0x7f08a4003a00 2023-08-27T11:17:18.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.484+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a8005450 con 0x7f08a4003a00 2023-08-27T11:17:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.484+0000 7f08affff700 1 -- 172.21.15.112:0/4041144519 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a80038c0 con 0x7f08a4003a00 2023-08-27T11:17:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.484+0000 7f08bf271700 1 -- 172.21.15.112:0/4041144519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a4003a00 msgr2=0x7f08a4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.484+0000 7f08bf271700 1 --2- 172.21.15.112:0/4041144519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a4003a00 0x7f08a4003e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f08a80017c0 tx=0x7f08a4005880 comp rx=0 tx=0).stop 2023-08-27T11:17:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.484+0000 7f08bf271700 1 -- 172.21.15.112:0/4041144519 shutdown_connections 2023-08-27T11:17:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.484+0000 7f08bf271700 1 --2- 172.21.15.112:0/4041144519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a4003a00 0x7f08a4003e20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.484+0000 7f08bf271700 1 -- 172.21.15.112:0/4041144519 >> 172.21.15.112:0/4041144519 conn(0x7f08b80b1e90 msgr2=0x7f08b80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.484+0000 7f08bf271700 1 -- 172.21.15.112:0/4041144519 shutdown_connections 2023-08-27T11:17:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.484+0000 7f08bf271700 1 -- 172.21.15.112:0/4041144519 wait complete. 2023-08-27T11:17:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.486+0000 7f08bf271700 1 Processor -- start 2023-08-27T11:17:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.486+0000 7f08bf271700 1 -- start start 2023-08-27T11:17:18.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.486+0000 7f08bf271700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a4003a00 0x7f08b8159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.486+0000 7f08bf271700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08b806f790 0x7f08b806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.486+0000 7f08bf271700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08b8139a00 0x7f08b8139e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.486+0000 7f08bf271700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80caa20 con 0x7f08a4003a00 2023-08-27T11:17:18.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.486+0000 7f08bf271700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b8070150 con 0x7f08b806f790 2023-08-27T11:17:18.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.486+0000 7f08bf271700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b813afd0 con 0x7f08b8139a00 2023-08-27T11:17:18.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.486+0000 7f08bd00d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a4003a00 0x7f08b8159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.486+0000 7f08bd00d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a4003a00 0x7f08b8159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55364/0 (socket says 172.21.15.112:55364) 2023-08-27T11:17:18.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.486+0000 7f08bd00d700 1 -- 172.21.15.112:0/1693620831 learned_addr learned my addr 172.21.15.112:0/1693620831 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:18.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.487+0000 7f08bd80e700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08b8139a00 0x7f08b8139e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.487+0000 7f08bc80c700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08b806f790 0x7f08b806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.487+0000 7f08bd00d700 1 -- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08b8139a00 msgr2=0x7f08b8139e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.487+0000 7f08bd00d700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08b8139a00 0x7f08b8139e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.487+0000 7f08bd00d700 1 -- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08b806f790 msgr2=0x7f08b806fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.487+0000 7f08bd00d700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08b806f790 0x7f08b806fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.487+0000 7f08bd00d700 1 -- 172.21.15.112:0/1693620831 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b813a730 con 0x7f08a4003a00 2023-08-27T11:17:18.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.487+0000 7f08bd00d700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a4003a00 0x7f08b8159890 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f08a8000c00 tx=0x7f08a8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:18.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.487+0000 7f08adffb700 1 -- 172.21.15.112:0/1693620831 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8001290 con 0x7f08a4003a00 2023-08-27T11:17:18.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.487+0000 7f08adffb700 1 -- 172.21.15.112:0/1693620831 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a8007040 con 0x7f08a4003a00 2023-08-27T11:17:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.487+0000 7f08adffb700 1 -- 172.21.15.112:0/1693620831 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a80073f0 con 0x7f08a4003a00 2023-08-27T11:17:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.487+0000 7f08bf271700 1 -- 172.21.15.112:0/1693620831 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b813aa10 con 0x7f08a4003a00 2023-08-27T11:17:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.487+0000 7f08bf271700 1 -- 172.21.15.112:0/1693620831 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b813a3d0 con 0x7f08a4003a00 2023-08-27T11:17:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.488+0000 7f08bf271700 1 -- 172.21.15.112:0/1693620831 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b8067e90 con 0x7f08a4003a00 2023-08-27T11:17:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.489+0000 7f08adffb700 1 -- 172.21.15.112:0/1693620831 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f08a8007550 con 0x7f08a4003a00 2023-08-27T11:17:18.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.489+0000 7f08adffb700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f089c06a0e0 0x7f089c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.489+0000 7f08adffb700 1 -- 172.21.15.112:0/1693620831 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f08a8088e50 con 0x7f08a4003a00 2023-08-27T11:17:18.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.489+0000 7f08bc80c700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f089c06a0e0 0x7f089c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.490+0000 7f08bc80c700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f089c06a0e0 0x7f089c06c5a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f08b4000c00 tx=0x7f08b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:18.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.493+0000 7f08adffb700 1 -- 172.21.15.112:0/1693620831 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f08a80545d0 con 0x7f08a4003a00 2023-08-27T11:17:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.679+0000 7f08bf271700 1 -- 172.21.15.112:0/1693620831 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08b8067a20 con 0x7f08a4003a00 2023-08-27T11:17:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.679+0000 7f08adffb700 1 -- 172.21.15.112:0/1693620831 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f08a80170a0 con 0x7f08a4003a00 2023-08-27T11:17:18.681 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:17:18.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.681+0000 7f08bf271700 1 -- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f089c06a0e0 msgr2=0x7f089c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.681+0000 7f08bf271700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f089c06a0e0 0x7f089c06c5a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f08b4000c00 tx=0x7f08b4009040 comp rx=0 tx=0).stop 2023-08-27T11:17:18.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.681+0000 7f08bf271700 1 -- 172.21.15.112:0/1693620831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a4003a00 msgr2=0x7f08b8159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.681+0000 7f08bf271700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a4003a00 0x7f08b8159890 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f08a8000c00 tx=0x7f08a8001120 comp rx=0 tx=0).stop 2023-08-27T11:17:18.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.681+0000 7f08bf271700 1 -- 172.21.15.112:0/1693620831 shutdown_connections 2023-08-27T11:17:18.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.681+0000 7f08bf271700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f089c06a0e0 0x7f089c06c5a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.681+0000 7f08bf271700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08b8139a00 0x7f08b8139e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.681+0000 7f08bf271700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a4003a00 0x7f08b8159890 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.681+0000 7f08bf271700 1 --2- 172.21.15.112:0/1693620831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08b806f790 0x7f08b806fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.681+0000 7f08bf271700 1 -- 172.21.15.112:0/1693620831 >> 172.21.15.112:0/1693620831 conn(0x7f08b80b1e90 msgr2=0x7f08b815a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:18.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.682+0000 7f08bf271700 1 -- 172.21.15.112:0/1693620831 shutdown_connections 2023-08-27T11:17:18.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.682+0000 7f08bf271700 1 -- 172.21.15.112:0/1693620831 wait complete. 2023-08-27T11:17:18.695 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:17:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.814+0000 7fe1136cc700 1 Processor -- start 2023-08-27T11:17:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.814+0000 7fe1136cc700 1 -- start start 2023-08-27T11:17:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe1136cc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe10c0b1400 0x7fe10c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe1136cc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe10c0b17e0 0x7fe10c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe1136cc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe10c161ce0 0x7fe10c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe1136cc700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10c161850 con 0x7fe10c0b17e0 2023-08-27T11:17:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe1136cc700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10c06e7d0 con 0x7fe10c0b1400 2023-08-27T11:17:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe1136cc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c0cfc10 con 0x7fe10c161ce0 2023-08-27T11:17:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe110c67700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe10c0b17e0 0x7fe10c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe110c67700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe10c0b17e0 0x7fe10c06e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55374/0 (socket says 172.21.15.112:55374) 2023-08-27T11:17:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe110c67700 1 -- 172.21.15.112:0/484823046 learned_addr learned my addr 172.21.15.112:0/484823046 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe111c69700 1 --2- 172.21.15.112:0/484823046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe10c161ce0 0x7fe10c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe111468700 1 --2- 172.21.15.112:0/484823046 >> v2:172.21.15.138:3300/0 conn(0x7fe10c0b1400 0x7fe10c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe110c67700 1 -- 172.21.15.112:0/484823046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe10c161ce0 msgr2=0x7fe10c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe110c67700 1 --2- 172.21.15.112:0/484823046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe10c161ce0 0x7fe10c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe110c67700 1 -- 172.21.15.112:0/484823046 >> v2:172.21.15.138:3300/0 conn(0x7fe10c0b1400 msgr2=0x7fe10c0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:18.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe110c67700 1 --2- 172.21.15.112:0/484823046 >> v2:172.21.15.138:3300/0 conn(0x7fe10c0b1400 0x7fe10c0cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.815+0000 7fe110c67700 1 -- 172.21.15.112:0/484823046 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c161200 con 0x7fe10c0b17e0 2023-08-27T11:17:18.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.816+0000 7fe110c67700 1 --2- 172.21.15.112:0/484823046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c0b17e0 0x7fe10c06e290 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe10c070370 tx=0x7fe108009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981c91cf96e1bc0d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:18.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.816+0000 7fe103fff700 1 -- 172.21.15.112:0/484823046 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10800b870 con 0x7fe10c0b17e0 2023-08-27T11:17:18.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.816+0000 7fe103fff700 1 -- 172.21.15.112:0/484823046 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10800b9d0 con 0x7fe10c0b17e0 2023-08-27T11:17:18.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.816+0000 7fe103fff700 1 -- 172.21.15.112:0/484823046 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10800bd60 con 0x7fe10c0b17e0 2023-08-27T11:17:18.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.816+0000 7fe1136cc700 1 -- 172.21.15.112:0/484823046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c0b17e0 msgr2=0x7fe10c06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.816+0000 7fe1136cc700 1 --2- 172.21.15.112:0/484823046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c0b17e0 0x7fe10c06e290 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe10c070370 tx=0x7fe108009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.816+0000 7fe1136cc700 1 -- 172.21.15.112:0/484823046 shutdown_connections 2023-08-27T11:17:18.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.816+0000 7fe1136cc700 1 --2- 172.21.15.112:0/484823046 >> v2:172.21.15.138:3300/0 conn(0x7fe10c0b1400 0x7fe10c0cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.816+0000 7fe1136cc700 1 --2- 172.21.15.112:0/484823046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe10c161ce0 0x7fe10c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.816+0000 7fe1136cc700 1 --2- 172.21.15.112:0/484823046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c0b17e0 0x7fe10c06e290 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.816+0000 7fe1136cc700 1 -- 172.21.15.112:0/484823046 >> 172.21.15.112:0/484823046 conn(0x7fe10c0d0000 msgr2=0x7fe10c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:18.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.816+0000 7fe1136cc700 1 -- 172.21.15.112:0/484823046 shutdown_connections 2023-08-27T11:17:18.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.816+0000 7fe1136cc700 1 -- 172.21.15.112:0/484823046 wait complete. 2023-08-27T11:17:18.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.817+0000 7fe1136cc700 1 Processor -- start 2023-08-27T11:17:18.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.817+0000 7fe1136cc700 1 -- start start 2023-08-27T11:17:18.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe1136cc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe10c0b1400 0x7fe10c0c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe1136cc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe10c0b17e0 0x7fe10c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe1136cc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c161ce0 0x7fe10c1525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe1136cc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c0ca120 con 0x7fe10c161ce0 2023-08-27T11:17:18.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe1136cc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c160260 con 0x7fe10c0b17e0 2023-08-27T11:17:18.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe1136cc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c15f650 con 0x7fe10c0b1400 2023-08-27T11:17:18.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe111c69700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c161ce0 0x7fe10c1525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe111c69700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c161ce0 0x7fe10c1525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55382/0 (socket says 172.21.15.112:55382) 2023-08-27T11:17:18.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe111c69700 1 -- 172.21.15.112:0/842740694 learned_addr learned my addr 172.21.15.112:0/842740694 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:18.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe110c67700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe10c0b17e0 0x7fe10c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe111c69700 1 -- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe10c0b1400 msgr2=0x7fe10c0c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe111c69700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe10c0b1400 0x7fe10c0c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe111c69700 1 -- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe10c0b17e0 msgr2=0x7fe10c15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:18.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.818+0000 7fe111c69700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe10c0b17e0 0x7fe10c15ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:18.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.819+0000 7fe111c69700 1 -- 172.21.15.112:0/842740694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe108009550 con 0x7fe10c161ce0 2023-08-27T11:17:18.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.819+0000 7fe110c67700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe10c0b17e0 0x7fe10c15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:18.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.819+0000 7fe111c69700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c161ce0 0x7fe10c1525e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe104007e60 tx=0x7fe10400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:18.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.819+0000 7fe101ffb700 1 -- 172.21.15.112:0/842740694 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10400ede0 con 0x7fe10c161ce0 2023-08-27T11:17:18.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.819+0000 7fe101ffb700 1 -- 172.21.15.112:0/842740694 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe104014420 con 0x7fe10c161ce0 2023-08-27T11:17:18.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.819+0000 7fe1136cc700 1 -- 172.21.15.112:0/842740694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10c15f930 con 0x7fe10c161ce0 2023-08-27T11:17:18.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.819+0000 7fe101ffb700 1 -- 172.21.15.112:0/842740694 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1040147d0 con 0x7fe10c161ce0 2023-08-27T11:17:18.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.819+0000 7fe1136cc700 1 -- 172.21.15.112:0/842740694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10c15fef0 con 0x7fe10c161ce0 2023-08-27T11:17:18.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.820+0000 7fe1136cc700 1 -- 172.21.15.112:0/842740694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10c067a20 con 0x7fe10c161ce0 2023-08-27T11:17:18.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.820+0000 7fe101ffb700 1 -- 172.21.15.112:0/842740694 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe104014930 con 0x7fe10c161ce0 2023-08-27T11:17:18.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.821+0000 7fe101ffb700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0f006a0e0 0x7fe0f006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:18.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.821+0000 7fe101ffb700 1 -- 172.21.15.112:0/842740694 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe104088c50 con 0x7fe10c161ce0 2023-08-27T11:17:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.821+0000 7fe111468700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0f006a0e0 0x7fe0f006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.821+0000 7fe111468700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0f006a0e0 0x7fe0f006c5a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009770 tx=0x7fe0fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:18.825+0000 7fe101ffb700 1 -- 172.21.15.112:0/842740694 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe10408e020 con 0x7fe10c161ce0 2023-08-27T11:17:19.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.057+0000 7fe1136cc700 1 -- 172.21.15.112:0/842740694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe10c1603d0 con 0x7fe10c161ce0 2023-08-27T11:17:19.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.058+0000 7fe101ffb700 1 -- 172.21.15.112:0/842740694 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4881 (secure 0 0 0) 0x7fe104057e60 con 0x7fe10c161ce0 2023-08-27T11:17:19.060 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:19.060 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3289488908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3289488908},{"type":"v1","addr":"172.21.15.112:6835","nonce":3289488908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3991653721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3991653721},{"type":"v1","addr":"172.21.15.112:6834","nonce":3991653721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1923303457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1923303457},{"type":"v1","addr":"172.21.15.138:6836","nonce":1923303457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:17:04.761658+0000","modified":"2023-08-27T11:17:08.372245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/1827312637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1827312637},{"type":"v1","addr":"172.21.15.138:6837","nonce":1827312637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:17:19.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.060+0000 7fe1136cc700 1 -- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0f006a0e0 msgr2=0x7fe0f006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.060+0000 7fe1136cc700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0f006a0e0 0x7fe0f006c5a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009770 tx=0x7fe0fc008040 comp rx=0 tx=0).stop 2023-08-27T11:17:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.060+0000 7fe1136cc700 1 -- 172.21.15.112:0/842740694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c161ce0 msgr2=0x7fe10c1525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.060+0000 7fe1136cc700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c161ce0 0x7fe10c1525e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe104007e60 tx=0x7fe10400e3c0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.060+0000 7fe1136cc700 1 -- 172.21.15.112:0/842740694 shutdown_connections 2023-08-27T11:17:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.060+0000 7fe1136cc700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0f006a0e0 0x7fe0f006c5a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.060+0000 7fe1136cc700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c161ce0 0x7fe10c1525e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.060+0000 7fe1136cc700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe10c0b1400 0x7fe10c0c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.060+0000 7fe1136cc700 1 --2- 172.21.15.112:0/842740694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe10c0b17e0 0x7fe10c15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.060+0000 7fe1136cc700 1 -- 172.21.15.112:0/842740694 >> 172.21.15.112:0/842740694 conn(0x7fe10c0d0000 msgr2=0x7fe10c06ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.060+0000 7fe1136cc700 1 -- 172.21.15.112:0/842740694 shutdown_connections 2023-08-27T11:17:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.060+0000 7fe1136cc700 1 -- 172.21.15.112:0/842740694 wait complete. 2023-08-27T11:17:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2023-08-27T11:17:19.074 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:19.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.192+0000 7f1f08c62700 1 Processor -- start 2023-08-27T11:17:19.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.192+0000 7f1f08c62700 1 -- start start 2023-08-27T11:17:19.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.193+0000 7f1f08c62700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f0415a120 0x7f1f040b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:19.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.193+0000 7f1f08c62700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0406deb0 con 0x7f1f0406d5b0 2023-08-27T11:17:19.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.193+0000 7f1f08c62700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0415b540 con 0x7f1f0406da40 2023-08-27T11:17:19.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.193+0000 7f1f08c62700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f040af830 con 0x7f1f0415a120 2023-08-27T11:17:19.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.193+0000 7f1f0259c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f0415a120 0x7f1f040b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:19.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.193+0000 7f1f01d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1f0406d5b0 0x7f1f040b3440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49768/0 (socket says 172.21.15.112:49768) 2023-08-27T11:17:19.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.193+0000 7f1f01d9b700 1 -- 172.21.15.112:0/79678315 learned_addr learned my addr 172.21.15.112:0/79678315 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:19.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.194+0000 7f1f0259c700 1 -- 172.21.15.112:0/79678315 >> v1:172.21.15.138:6789/0 conn(0x7f1f0406da40 legacy=0x7f1f040b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.194+0000 7f1f0259c700 1 -- 172.21.15.112:0/79678315 >> v1:172.21.15.112:6789/0 conn(0x7f1f0406d5b0 legacy=0x7f1f040b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.194+0000 7f1f0259c700 1 -- 172.21.15.112:0/79678315 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f040afb10 con 0x7f1f0415a120 2023-08-27T11:17:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.194+0000 7f1f0159a700 1 -- 172.21.15.112:0/79678315 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164261031 0 0) 0x7f1f0406deb0 con 0x7f1f0406d5b0 2023-08-27T11:17:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.194+0000 7f1f0259c700 1 --2- 172.21.15.112:0/79678315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f0415a120 0x7f1f040b05f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009840 tx=0x7f1ef4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e185e2f8b42761 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.194+0000 7f1f0159a700 1 -- 172.21.15.112:0/79678315 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4011830 con 0x7f1f0415a120 2023-08-27T11:17:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.194+0000 7f1f0159a700 1 -- 172.21.15.112:0/79678315 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef4011990 con 0x7f1f0415a120 2023-08-27T11:17:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.194+0000 7f1f0159a700 1 -- 172.21.15.112:0/79678315 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4011d20 con 0x7f1f0415a120 2023-08-27T11:17:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.194+0000 7f1f08c62700 1 -- 172.21.15.112:0/79678315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f0415a120 msgr2=0x7f1f040b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.194+0000 7f1f08c62700 1 --2- 172.21.15.112:0/79678315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f0415a120 0x7f1f040b05f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009840 tx=0x7f1ef4009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.195+0000 7f1f08c62700 1 -- 172.21.15.112:0/79678315 shutdown_connections 2023-08-27T11:17:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.195+0000 7f1f08c62700 1 --2- 172.21.15.112:0/79678315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f0415a120 0x7f1f040b05f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.195+0000 7f1f08c62700 1 -- 172.21.15.112:0/79678315 >> 172.21.15.112:0/79678315 conn(0x7f1f040cff80 msgr2=0x7f1f040d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.195+0000 7f1f08c62700 1 -- 172.21.15.112:0/79678315 shutdown_connections 2023-08-27T11:17:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.195+0000 7f1f08c62700 1 -- 172.21.15.112:0/79678315 wait complete. 2023-08-27T11:17:19.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.196+0000 7f1f08c62700 1 Processor -- start 2023-08-27T11:17:19.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.196+0000 7f1f08c62700 1 -- start start 2023-08-27T11:17:19.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f08c62700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f0406d5b0 0x7f1f041465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:19.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f08c62700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0406da40 0x7f1f040718f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:19.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f08c62700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f0415a120 0x7f1f04071e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:19.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f08c62700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f04146ec0 con 0x7f1f0415a120 2023-08-27T11:17:19.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f08c62700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f040cbb80 con 0x7f1f0406da40 2023-08-27T11:17:19.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f08c62700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f04147410 con 0x7f1f0406d5b0 2023-08-27T11:17:19.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f02d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f0415a120 0x7f1f04071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:19.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f02d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f0415a120 0x7f1f04071e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55396/0 (socket says 172.21.15.112:55396) 2023-08-27T11:17:19.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f02d9d700 1 -- 172.21.15.112:0/213686950 learned_addr learned my addr 172.21.15.112:0/213686950 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:19.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f01d9b700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0406da40 0x7f1f040718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:19.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f0259c700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f0406d5b0 0x7f1f041465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:19.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f02d9d700 1 -- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f0406d5b0 msgr2=0x7f1f041465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f02d9d700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f0406d5b0 0x7f1f041465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f02d9d700 1 -- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0406da40 msgr2=0x7f1f040718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f02d9d700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0406da40 0x7f1f040718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f02d9d700 1 -- 172.21.15.112:0/213686950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef40094a0 con 0x7f1f0415a120 2023-08-27T11:17:19.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f01d9b700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0406da40 0x7f1f040718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:19.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f02d9d700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f0415a120 0x7f1f04071e30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0021c0 tx=0x7f1efc0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:19.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1efaffd700 1 -- 172.21.15.112:0/213686950 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00cea0 con 0x7f1f0415a120 2023-08-27T11:17:19.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1efaffd700 1 -- 172.21.15.112:0/213686950 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc003c00 con 0x7f1f0415a120 2023-08-27T11:17:19.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1efaffd700 1 -- 172.21.15.112:0/213686950 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0050d0 con 0x7f1f0415a120 2023-08-27T11:17:19.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f08c62700 1 -- 172.21.15.112:0/213686950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f040c5d40 con 0x7f1f0415a120 2023-08-27T11:17:19.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.197+0000 7f1f08c62700 1 -- 172.21.15.112:0/213686950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f041470a0 con 0x7f1f0415a120 2023-08-27T11:17:19.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.198+0000 7f1f08c62700 1 -- 172.21.15.112:0/213686950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f040c6260 con 0x7f1f0415a120 2023-08-27T11:17:19.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.199+0000 7f1efaffd700 1 -- 172.21.15.112:0/213686950 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1efc005230 con 0x7f1f0415a120 2023-08-27T11:17:19.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.199+0000 7f1efaffd700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ee406a0e0 0x7f1ee406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:19.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.199+0000 7f1efaffd700 1 -- 172.21.15.112:0/213686950 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1efc08ad90 con 0x7f1f0415a120 2023-08-27T11:17:19.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.199+0000 7f1f0259c700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ee406a0e0 0x7f1ee406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:19.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.200+0000 7f1f0259c700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ee406a0e0 0x7f1ee406c5a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4000c00 tx=0x7f1ef401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:19.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.203+0000 7f1efaffd700 1 -- 172.21.15.112:0/213686950 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1efc056480 con 0x7f1f0415a120 2023-08-27T11:17:19.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.392+0000 7f1f08c62700 1 -- 172.21.15.112:0/213686950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f04067a20 con 0x7f1f0415a120 2023-08-27T11:17:19.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.393+0000 7f1efaffd700 1 -- 172.21.15.112:0/213686950 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7f1efc01f070 con 0x7f1f0415a120 2023-08-27T11:17:19.395 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:19.395 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:19.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.395+0000 7f1f08c62700 1 -- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ee406a0e0 msgr2=0x7f1ee406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.395+0000 7f1f08c62700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ee406a0e0 0x7f1ee406c5a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4000c00 tx=0x7f1ef401c040 comp rx=0 tx=0).stop 2023-08-27T11:17:19.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.395+0000 7f1f08c62700 1 -- 172.21.15.112:0/213686950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f0415a120 msgr2=0x7f1f04071e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.395+0000 7f1f08c62700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f0415a120 0x7f1f04071e30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0021c0 tx=0x7f1efc0021f0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.395+0000 7f1f08c62700 1 -- 172.21.15.112:0/213686950 shutdown_connections 2023-08-27T11:17:19.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.395+0000 7f1f08c62700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ee406a0e0 0x7f1ee406c5a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.395+0000 7f1f08c62700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f0415a120 0x7f1f04071e30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.395+0000 7f1f08c62700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f0406d5b0 0x7f1f041465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.395+0000 7f1f08c62700 1 --2- 172.21.15.112:0/213686950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0406da40 0x7f1f040718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.396+0000 7f1f08c62700 1 -- 172.21.15.112:0/213686950 >> 172.21.15.112:0/213686950 conn(0x7f1f040cff80 msgr2=0x7f1f0415a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.396+0000 7f1f08c62700 1 -- 172.21.15.112:0/213686950 shutdown_connections 2023-08-27T11:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.396+0000 7f1f08c62700 1 -- 172.21.15.112:0/213686950 wait complete. 2023-08-27T11:17:19.409 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:17:19.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.528+0000 7f48e8a4c700 1 Processor -- start 2023-08-27T11:17:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.528+0000 7f48e8a4c700 1 -- start start 2023-08-27T11:17:19.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e8a4c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e40b1400 0x7f48e40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:19.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e8a4c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f48e40b17e0 0x7f48e406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:19.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e8a4c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48e4161ce0 0x7f48e406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:19.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e8a4c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e4161850 con 0x7f48e4161ce0 2023-08-27T11:17:19.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e8a4c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e406e7d0 con 0x7f48e40b17e0 2023-08-27T11:17:19.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e8a4c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40cfc10 con 0x7f48e40b1400 2023-08-27T11:17:19.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e2d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48e4161ce0 0x7f48e406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e2d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48e4161ce0 0x7f48e406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55402/0 (socket says 172.21.15.112:55402) 2023-08-27T11:17:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e2d9d700 1 -- 172.21.15.112:0/939503575 learned_addr learned my addr 172.21.15.112:0/939503575 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e1d9b700 1 --2- 172.21.15.112:0/939503575 >> v2:172.21.15.138:3300/0 conn(0x7f48e40b17e0 0x7f48e406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e259c700 1 --2- 172.21.15.112:0/939503575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e40b1400 0x7f48e40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e2d9d700 1 -- 172.21.15.112:0/939503575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e40b1400 msgr2=0x7f48e40cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e2d9d700 1 --2- 172.21.15.112:0/939503575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e40b1400 0x7f48e40cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e2d9d700 1 -- 172.21.15.112:0/939503575 >> v2:172.21.15.138:3300/0 conn(0x7f48e40b17e0 msgr2=0x7f48e406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e2d9d700 1 --2- 172.21.15.112:0/939503575 >> v2:172.21.15.138:3300/0 conn(0x7f48e40b17e0 0x7f48e406e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e2d9d700 1 -- 172.21.15.112:0/939503575 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e4161200 con 0x7f48e4161ce0 2023-08-27T11:17:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.529+0000 7f48e259c700 1 --2- 172.21.15.112:0/939503575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e40b1400 0x7f48e40cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.530+0000 7f48e2d9d700 1 --2- 172.21.15.112:0/939503575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4161ce0 0x7f48e406f790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f48e40d0db0 tx=0x7f48dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71787f9df6247a4f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.530+0000 7f48e159a700 1 -- 172.21.15.112:0/939503575 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00e070 con 0x7f48e4161ce0 2023-08-27T11:17:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.530+0000 7f48e159a700 1 -- 172.21.15.112:0/939503575 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc00b9c0 con 0x7f48e4161ce0 2023-08-27T11:17:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.530+0000 7f48e159a700 1 -- 172.21.15.112:0/939503575 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00bd70 con 0x7f48e4161ce0 2023-08-27T11:17:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.530+0000 7f48e8a4c700 1 -- 172.21.15.112:0/939503575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4161ce0 msgr2=0x7f48e406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.530+0000 7f48e8a4c700 1 --2- 172.21.15.112:0/939503575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4161ce0 0x7f48e406f790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f48e40d0db0 tx=0x7f48dc00b330 comp rx=0 tx=0).stop 2023-08-27T11:17:19.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.530+0000 7f48e8a4c700 1 -- 172.21.15.112:0/939503575 shutdown_connections 2023-08-27T11:17:19.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.530+0000 7f48e8a4c700 1 --2- 172.21.15.112:0/939503575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4161ce0 0x7f48e406f790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.530+0000 7f48e8a4c700 1 --2- 172.21.15.112:0/939503575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e40b1400 0x7f48e40cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.530+0000 7f48e8a4c700 1 --2- 172.21.15.112:0/939503575 >> v2:172.21.15.138:3300/0 conn(0x7f48e40b17e0 0x7f48e406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.530+0000 7f48e8a4c700 1 -- 172.21.15.112:0/939503575 >> 172.21.15.112:0/939503575 conn(0x7f48e40d0000 msgr2=0x7f48e40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:19.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.530+0000 7f48e8a4c700 1 -- 172.21.15.112:0/939503575 shutdown_connections 2023-08-27T11:17:19.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.530+0000 7f48e8a4c700 1 -- 172.21.15.112:0/939503575 wait complete. 2023-08-27T11:17:19.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.532+0000 7f48e8a4c700 1 Processor -- start 2023-08-27T11:17:19.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.532+0000 7f48e8a4c700 1 -- start start 2023-08-27T11:17:19.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.532+0000 7f48e8a4c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e40b1400 0x7f48e40c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:19.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.532+0000 7f48e8a4c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e40b17e0 0x7f48e415e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:19.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.532+0000 7f48e8a4c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e4161ce0 0x7f48e415ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:19.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.532+0000 7f48e8a4c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e415ef20 con 0x7f48e40b17e0 2023-08-27T11:17:19.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.532+0000 7f48e8a4c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e415f200 con 0x7f48e4161ce0 2023-08-27T11:17:19.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.532+0000 7f48e8a4c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e415f4e0 con 0x7f48e40b1400 2023-08-27T11:17:19.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.532+0000 7f48e1d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e40b17e0 0x7f48e415e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:19.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.532+0000 7f48e1d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e40b17e0 0x7f48e415e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55406/0 (socket says 172.21.15.112:55406) 2023-08-27T11:17:19.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.532+0000 7f48e1d9b700 1 -- 172.21.15.112:0/4155826136 learned_addr learned my addr 172.21.15.112:0/4155826136 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:19.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48e2d9d700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e4161ce0 0x7f48e415ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:19.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48e259c700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e40b1400 0x7f48e40c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:19.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48e1d9b700 1 -- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e40b1400 msgr2=0x7f48e40c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48e1d9b700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e40b1400 0x7f48e40c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48e1d9b700 1 -- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e4161ce0 msgr2=0x7f48e415ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48e1d9b700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e4161ce0 0x7f48e415ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48e1d9b700 1 -- 172.21.15.112:0/4155826136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc009b30 con 0x7f48e40b17e0 2023-08-27T11:17:19.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48e259c700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e40b1400 0x7f48e40c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:19.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48e1d9b700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e40b17e0 0x7f48e415e890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f48d400e810 tx=0x7f48d400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:19.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48d2ffd700 1 -- 172.21.15.112:0/4155826136 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4013610 con 0x7f48e40b17e0 2023-08-27T11:17:19.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48d2ffd700 1 -- 172.21.15.112:0/4155826136 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d4013770 con 0x7f48e40b17e0 2023-08-27T11:17:19.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48e8a4c700 1 -- 172.21.15.112:0/4155826136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e415f7c0 con 0x7f48e40b17e0 2023-08-27T11:17:19.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48d2ffd700 1 -- 172.21.15.112:0/4155826136 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4013b20 con 0x7f48e40b17e0 2023-08-27T11:17:19.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.533+0000 7f48e8a4c700 1 -- 172.21.15.112:0/4155826136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e40c9de0 con 0x7f48e40b17e0 2023-08-27T11:17:19.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.534+0000 7f48e8a4c700 1 -- 172.21.15.112:0/4155826136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e4067a20 con 0x7f48e40b17e0 2023-08-27T11:17:19.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.534+0000 7f48d2ffd700 1 -- 172.21.15.112:0/4155826136 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f48d4013c80 con 0x7f48e40b17e0 2023-08-27T11:17:19.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.535+0000 7f48d2ffd700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48c406a0e0 0x7f48c406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:19.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.535+0000 7f48d2ffd700 1 -- 172.21.15.112:0/4155826136 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f48d4088170 con 0x7f48e40b17e0 2023-08-27T11:17:19.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.535+0000 7f48e259c700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48c406a0e0 0x7f48c406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:19.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.535+0000 7f48e259c700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48c406a0e0 0x7f48c406c5a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006ba0 tx=0x7f48d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:19.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.539+0000 7f48d2ffd700 1 -- 172.21.15.112:0/4155826136 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f48d4057300 con 0x7f48e40b17e0 2023-08-27T11:17:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.768+0000 7f48e8a4c700 1 -- 172.21.15.112:0/4155826136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48e406db60 con 0x7f48e40b17e0 2023-08-27T11:17:19.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.769+0000 7f48d2ffd700 1 -- 172.21.15.112:0/4155826136 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4881 (secure 0 0 0) 0x7f48d4056e90 con 0x7f48e40b17e0 2023-08-27T11:17:19.770 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:19.771 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3289488908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3289488908},{"type":"v1","addr":"172.21.15.112:6835","nonce":3289488908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3991653721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3991653721},{"type":"v1","addr":"172.21.15.112:6834","nonce":3991653721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1923303457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1923303457},{"type":"v1","addr":"172.21.15.138:6836","nonce":1923303457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:17:04.761658+0000","modified":"2023-08-27T11:17:08.372245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/1827312637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1827312637},{"type":"v1","addr":"172.21.15.138:6837","nonce":1827312637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:17:19.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.771+0000 7f48e8a4c700 1 -- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48c406a0e0 msgr2=0x7f48c406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.771+0000 7f48e8a4c700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48c406a0e0 0x7f48c406c5a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006ba0 tx=0x7f48d8008040 comp rx=0 tx=0).stop 2023-08-27T11:17:19.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.771+0000 7f48e8a4c700 1 -- 172.21.15.112:0/4155826136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e40b17e0 msgr2=0x7f48e415e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:19.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.771+0000 7f48e8a4c700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e40b17e0 0x7f48e415e890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f48d400e810 tx=0x7f48d400eb20 comp rx=0 tx=0).stop 2023-08-27T11:17:19.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.771+0000 7f48e8a4c700 1 -- 172.21.15.112:0/4155826136 shutdown_connections 2023-08-27T11:17:19.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.771+0000 7f48e8a4c700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48c406a0e0 0x7f48c406c5a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.771+0000 7f48e8a4c700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e4161ce0 0x7f48e415ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.771+0000 7f48e8a4c700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e40b1400 0x7f48e40c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.771+0000 7f48e8a4c700 1 --2- 172.21.15.112:0/4155826136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e40b17e0 0x7f48e415e890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:19.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.771+0000 7f48e8a4c700 1 -- 172.21.15.112:0/4155826136 >> 172.21.15.112:0/4155826136 conn(0x7f48e40d0000 msgr2=0x7f48e4071830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:19.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.771+0000 7f48e8a4c700 1 -- 172.21.15.112:0/4155826136 shutdown_connections 2023-08-27T11:17:19.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:19.771+0000 7f48e8a4c700 1 -- 172.21.15.112:0/4155826136 wait complete. 2023-08-27T11:17:19.775 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2023-08-27T11:17:20.186 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:17:04.761658+0000', 'modified': '2023-08-27T11:17:08.372245+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6837/1827312637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6835', 'nonce': 1827312637}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 1827312637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:17:20.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:17:20.187 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-27T11:17:20.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.310+0000 7fcbca05c700 1 Processor -- start 2023-08-27T11:17:20.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.310+0000 7fcbca05c700 1 -- start start 2023-08-27T11:17:20.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.310+0000 7fcbca05c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbc4160a00 0x7fcbc40b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.310+0000 7fcbca05c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcbc4146990 0x7fcbc415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbca05c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc415e140 con 0x7fcbc406d510 2023-08-27T11:17:20.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbca05c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc406f930 con 0x7fcbc4146990 2023-08-27T11:17:20.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbca05c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc406fc10 con 0x7fcbc4160a00 2023-08-27T11:17:20.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc3fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fcbc406d510 0x7fcbc4161400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49794/0 (socket says 172.21.15.112:49794) 2023-08-27T11:17:20.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc3fff700 1 -- 172.21.15.112:0/1128075811 learned_addr learned my addr 172.21.15.112:0/1128075811 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:20.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc2ffd700 1 --2- 172.21.15.112:0/1128075811 >> v2:172.21.15.138:3300/0 conn(0x7fcbc4146990 0x7fcbc415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc37fe700 1 --2- 172.21.15.112:0/1128075811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbc4160a00 0x7fcbc40b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc27fc700 1 -- 172.21.15.112:0/1128075811 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054127546 0 0) 0x7fcbc415e140 con 0x7fcbc406d510 2023-08-27T11:17:20.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc27fc700 1 -- 172.21.15.112:0/1128075811 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbb00035b0 con 0x7fcbc406d510 2023-08-27T11:17:20.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc2ffd700 1 -- 172.21.15.112:0/1128075811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbc4160a00 msgr2=0x7fcbc40b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:20.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc2ffd700 1 --2- 172.21.15.112:0/1128075811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbc4160a00 0x7fcbc40b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc2ffd700 1 -- 172.21.15.112:0/1128075811 >> v1:172.21.15.112:6789/0 conn(0x7fcbc406d510 legacy=0x7fcbc4161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc2ffd700 1 -- 172.21.15.112:0/1128075811 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc406fef0 con 0x7fcbc4146990 2023-08-27T11:17:20.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc2ffd700 1 --2- 172.21.15.112:0/1128075811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc4146990 0x7fcbc415dc00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009840 tx=0x7fcbac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=200209f24bb0eaa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:20.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc27fc700 1 -- 172.21.15.112:0/1128075811 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac011830 con 0x7fcbc4146990 2023-08-27T11:17:20.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc27fc700 1 -- 172.21.15.112:0/1128075811 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac011990 con 0x7fcbc4146990 2023-08-27T11:17:20.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.311+0000 7fcbc27fc700 1 -- 172.21.15.112:0/1128075811 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac011d20 con 0x7fcbc4146990 2023-08-27T11:17:20.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.312+0000 7fcbca05c700 1 -- 172.21.15.112:0/1128075811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc4146990 msgr2=0x7fcbc415dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.312+0000 7fcbca05c700 1 --2- 172.21.15.112:0/1128075811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc4146990 0x7fcbc415dc00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009840 tx=0x7fcbac009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:20.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.312+0000 7fcbca05c700 1 -- 172.21.15.112:0/1128075811 shutdown_connections 2023-08-27T11:17:20.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.312+0000 7fcbca05c700 1 --2- 172.21.15.112:0/1128075811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbc4160a00 0x7fcbc40b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.312+0000 7fcbca05c700 1 --2- 172.21.15.112:0/1128075811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc4146990 0x7fcbc415dc00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.312+0000 7fcbca05c700 1 -- 172.21.15.112:0/1128075811 >> 172.21.15.112:0/1128075811 conn(0x7fcbc40cfe50 msgr2=0x7fcbc40d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:20.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.312+0000 7fcbca05c700 1 -- 172.21.15.112:0/1128075811 shutdown_connections 2023-08-27T11:17:20.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.312+0000 7fcbca05c700 1 -- 172.21.15.112:0/1128075811 wait complete. 2023-08-27T11:17:20.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.313+0000 7fcbca05c700 1 Processor -- start 2023-08-27T11:17:20.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.313+0000 7fcbca05c700 1 -- start start 2023-08-27T11:17:20.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.314+0000 7fcbca05c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbc406d510 0x7fcbc41f6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.314+0000 7fcbca05c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbc4146990 0x7fcbc41f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.314+0000 7fcbca05c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc4160a00 0x7fcbc40b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.314+0000 7fcbca05c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc4138a70 con 0x7fcbc406d510 2023-08-27T11:17:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.314+0000 7fcbca05c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc41f6e50 con 0x7fcbc4160a00 2023-08-27T11:17:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.314+0000 7fcbca05c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc4137df0 con 0x7fcbc4146990 2023-08-27T11:17:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.314+0000 7fcbc37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbc406d510 0x7fcbc41f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.314+0000 7fcbc37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbc406d510 0x7fcbc41f6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55416/0 (socket says 172.21.15.112:55416) 2023-08-27T11:17:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.314+0000 7fcbc37fe700 1 -- 172.21.15.112:0/2341379868 learned_addr learned my addr 172.21.15.112:0/2341379868 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.314+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc4160a00 0x7fcbc40b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbc2ffd700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbc4146990 0x7fcbc41f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbc37fe700 1 -- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbc4146990 msgr2=0x7fcbc41f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbc37fe700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbc4146990 0x7fcbc41f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbc37fe700 1 -- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc4160a00 msgr2=0x7fcbc40b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbc37fe700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc4160a00 0x7fcbc40b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbc37fe700 1 -- 172.21.15.112:0/2341379868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0094a0 con 0x7fcbc406d510 2023-08-27T11:17:20.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbc2ffd700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbc4146990 0x7fcbc41f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:20.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc4160a00 0x7fcbc40b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:20.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbc37fe700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbc406d510 0x7fcbc41f6910 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400efc0 tx=0x7fcbb400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:20.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbbbfff700 1 -- 172.21.15.112:0/2341379868 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4009b80 con 0x7fcbc406d510 2023-08-27T11:17:20.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbbbfff700 1 -- 172.21.15.112:0/2341379868 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb400f040 con 0x7fcbc406d510 2023-08-27T11:17:20.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbca05c700 1 -- 172.21.15.112:0/2341379868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc41380d0 con 0x7fcbc406d510 2023-08-27T11:17:20.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbbbfff700 1 -- 172.21.15.112:0/2341379868 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4014480 con 0x7fcbc406d510 2023-08-27T11:17:20.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.315+0000 7fcbca05c700 1 -- 172.21.15.112:0/2341379868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc4138690 con 0x7fcbc406d510 2023-08-27T11:17:20.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.316+0000 7fcbbbfff700 1 -- 172.21.15.112:0/2341379868 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcbb40145e0 con 0x7fcbc406d510 2023-08-27T11:17:20.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.317+0000 7fcbbbfff700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcba406a0e0 0x7fcba406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.317+0000 7fcbbbfff700 1 -- 172.21.15.112:0/2341379868 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbb4088de0 con 0x7fcbc406d510 2023-08-27T11:17:20.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.317+0000 7fcbc2ffd700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcba406a0e0 0x7fcba406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.317+0000 7fcbc2ffd700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcba406a0e0 0x7fcba406c5a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcbc413ac50 tx=0x7fcbac008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:20.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.319+0000 7fcbd621eb80 1 Processor -- start 2023-08-27T11:17:20.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.319+0000 7fcbd621eb80 1 -- start start 2023-08-27T11:17:20.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbd621eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e77cf40 0x55568e77d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbd621eb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e78e960 0x55568e779b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbd621eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e7503d0 0x55568e77a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbd621eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568e6ab730 con 0x55568e7503d0 2023-08-27T11:17:20.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbd621eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568e6a9050 con 0x55568e77cf40 2023-08-27T11:17:20.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbd621eb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55568e6a4940 con 0x55568e78e960 2023-08-27T11:17:20.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e7503d0 0x55568e77a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e7503d0 0x55568e77a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55428/0 (socket says 172.21.15.112:55428) 2023-08-27T11:17:20.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbc3fff700 1 -- 172.21.15.112:0/3668283423 learned_addr learned my addr 172.21.15.112:0/3668283423 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:20.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbc37fe700 1 --2- 172.21.15.112:0/3668283423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e77cf40 0x55568e77d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbc2ffd700 1 --2- 172.21.15.112:0/3668283423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e78e960 0x55568e779b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbc3fff700 1 -- 172.21.15.112:0/3668283423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e78e960 msgr2=0x55568e779b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:20.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/3668283423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e78e960 0x55568e779b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbc3fff700 1 -- 172.21.15.112:0/3668283423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e77cf40 msgr2=0x55568e77d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:20.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/3668283423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e77cf40 0x55568e77d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbc3fff700 1 -- 172.21.15.112:0/3668283423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55568e5245e0 con 0x55568e7503d0 2023-08-27T11:17:20.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/3668283423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e7503d0 0x55568e77a0b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc001730 tx=0x7fcbbc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b4e9d77bf63a3a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:20.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbb97fa700 1 -- 172.21.15.112:0/3668283423 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc005b50 con 0x55568e7503d0 2023-08-27T11:17:20.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbb97fa700 1 -- 172.21.15.112:0/3668283423 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc005cb0 con 0x55568e7503d0 2023-08-27T11:17:20.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.320+0000 7fcbb97fa700 1 -- 172.21.15.112:0/3668283423 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc0072d0 con 0x55568e7503d0 2023-08-27T11:17:20.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.321+0000 7fcbd621eb80 1 -- 172.21.15.112:0/3668283423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e7503d0 msgr2=0x55568e77a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.321+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/3668283423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e7503d0 0x55568e77a0b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc001730 tx=0x7fcbbc001a40 comp rx=0 tx=0).stop 2023-08-27T11:17:20.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.321+0000 7fcbd621eb80 1 -- 172.21.15.112:0/3668283423 shutdown_connections 2023-08-27T11:17:20.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.321+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/3668283423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e7503d0 0x55568e77a0b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.321+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/3668283423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e77cf40 0x55568e77d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.321+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/3668283423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e78e960 0x55568e779b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.321+0000 7fcbd621eb80 1 -- 172.21.15.112:0/3668283423 >> 172.21.15.112:0/3668283423 conn(0x55568e6b0c30 msgr2=0x55568e797d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:20.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.321+0000 7fcbd621eb80 1 -- 172.21.15.112:0/3668283423 shutdown_connections 2023-08-27T11:17:20.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.321+0000 7fcbd621eb80 1 -- 172.21.15.112:0/3668283423 wait complete. 2023-08-27T11:17:20.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.322+0000 7fcbd621eb80 1 -- start start 2023-08-27T11:17:20.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.322+0000 7fcbd621eb80 1 Processor -- start 2023-08-27T11:17:20.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.322+0000 7fcbd621eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e7503d0 0x55568e796ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.322+0000 7fcbd621eb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e77cf40 0x55568e778f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.322+0000 7fcbd621eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e79a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.322+0000 7fcbd621eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568e642b60 con 0x55568e78e960 2023-08-27T11:17:20.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.322+0000 7fcbd621eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568e642860 con 0x55568e7503d0 2023-08-27T11:17:20.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.322+0000 7fcbd621eb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55568e5268d0 con 0x55568e77cf40 2023-08-27T11:17:20.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbd621eb80 10 client.0 _ll_register_callbacks cb 0x55568e640630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:17:20.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e79a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e79a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55434/0 (socket says 172.21.15.112:55434) 2023-08-27T11:17:20.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbc3fff700 1 -- 172.21.15.112:0/916880635 learned_addr learned my addr 172.21.15.112:0/916880635 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:20.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbc37fe700 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e7503d0 0x55568e796ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbc2ffd700 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e77cf40 0x55568e778f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbc3fff700 1 -- 172.21.15.112:0/916880635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e77cf40 msgr2=0x55568e778f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:20.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e77cf40 0x55568e778f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbc3fff700 1 -- 172.21.15.112:0/916880635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e7503d0 msgr2=0x55568e796ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:20.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e7503d0 0x55568e796ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbc3fff700 1 -- 172.21.15.112:0/916880635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc00f040 con 0x55568e78e960 2023-08-27T11:17:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbc37fe700 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e7503d0 0x55568e796ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e79a460 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0016d0 tx=0x7fcbbc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646db687ac1fc8ed server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcba27fc700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:17:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcba27fc700 1 -- 172.21.15.112:0/916880635 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc005b50 con 0x55568e78e960 2023-08-27T11:17:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcba27fc700 1 -- 172.21.15.112:0/916880635 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbbc0075f0 con 0x55568e78e960 2023-08-27T11:17:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbd621eb80 1 -- 172.21.15.112:0/916880635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55568e7454f0 con 0x55568e78e960 2023-08-27T11:17:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcba27fc700 1 -- 172.21.15.112:0/916880635 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbbc007c00 con 0x55568e78e960 2023-08-27T11:17:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbd621eb80 10 client.4656 fetch_fsmap learned FSMap version 7 2023-08-27T11:17:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbd621eb80 1 -- 172.21.15.112:0/916880635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55568e6bc750 con 0x55568e78e960 2023-08-27T11:17:20.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcba27fc700 1 -- 172.21.15.112:0/916880635 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55568e7454f0 con 0x55568e78e960 2023-08-27T11:17:20.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcba27fc700 1 -- 172.21.15.112:0/916880635 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcbbc014480 con 0x55568e78e960 2023-08-27T11:17:20.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.323+0000 7fcbd621eb80 10 client.4656 fetch_fsmap finished waiting for FSMap version 7 2023-08-27T11:17:20.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.324+0000 7fcbd621eb80 10 client.4656 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-27T11:17:20.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.324+0000 7fcbd621eb80 20 client.4656 populate_metadata read hostname 'smithi112' 2023-08-27T11:17:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.324+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.138:6835/1827312637,v1:172.21.15.138:6837/1827312637] conn(0x55568e7a7170 0x55568e7a7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.324+0000 7fcbd621eb80 4 client.4656 mds_command: new command op to 4285 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:17:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.324+0000 7fcbd621eb80 1 -- 172.21.15.112:0/916880635 --> [v2:172.21.15.138:6835/1827312637,v1:172.21.15.138:6837/1827312637] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55568e69d920 con 0x55568e7a7170 2023-08-27T11:17:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.324+0000 7fcbc37fe700 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.138:6835/1827312637,v1:172.21.15.138:6837/1827312637] conn(0x55568e7a7170 0x55568e7a7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.324+0000 7fcba27fc700 10 client.4656 ms_handle_connect on v2:172.21.15.138:6835/1827312637 2023-08-27T11:17:20.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.324+0000 7fcbc37fe700 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.138:6835/1827312637,v1:172.21.15.138:6837/1827312637] conn(0x55568e7a7170 0x55568e7a7550 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35784cf484281652 server_cookie=120acafcbcfe784b in_seq=0 out_seq=0 2023-08-27T11:17:20.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcba27fc700 1 -- 172.21.15.112:0/916880635 <== mds.0 v2:172.21.15.138:6835/1827312637 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55568e69d920 con 0x55568e7a7170 2023-08-27T11:17:20.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcba27fc700 10 client.4656 handle_command_reply: tid=0 2023-08-27T11:17:20.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcbd621eb80 1 client.4656 shutdown 2023-08-27T11:17:20.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcbd621eb80 1 -- 172.21.15.112:0/916880635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 msgr2=0x55568e79a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e79a460 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0016d0 tx=0x7fcbbc000ed0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcbd621eb80 1 -- 172.21.15.112:0/916880635 shutdown_connections 2023-08-27T11:17:20.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.138:6835/1827312637,v1:172.21.15.138:6837/1827312637] conn(0x55568e7a7170 0x55568e7a7550 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e79a460 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e7503d0 0x55568e796ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/916880635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e77cf40 0x55568e778f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcba27fc700 0 client.4656 ms_handle_reset on v2:172.21.15.138:6835/1827312637 2023-08-27T11:17:20.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcbd621eb80 1 -- 172.21.15.112:0/916880635 >> 172.21.15.112:0/916880635 conn(0x55568e6b0c30 msgr2=0x55568e776910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:20.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcbd621eb80 1 -- 172.21.15.112:0/916880635 shutdown_connections 2023-08-27T11:17:20.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcbd621eb80 1 -- 172.21.15.112:0/916880635 wait complete. 2023-08-27T11:17:20.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.325+0000 7fcbd621eb80 20 client.4656 trim_cache size 0 max 16384 2023-08-27T11:17:20.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.343+0000 7fcbd621eb80 1 Processor -- start 2023-08-27T11:17:20.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.343+0000 7fcbd621eb80 1 -- start start 2023-08-27T11:17:20.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.343+0000 7fcbd621eb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e7503d0 0x55568e776280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.343+0000 7fcbd621eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e77cf40 0x55568e7507b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbd621eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e74f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbd621eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568e7a79e0 con 0x55568e78e960 2023-08-27T11:17:20.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbd621eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568e7965e0 con 0x55568e77cf40 2023-08-27T11:17:20.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbd621eb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55568e74eb50 con 0x55568e7503d0 2023-08-27T11:17:20.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e74f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e74f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55442/0 (socket says 172.21.15.112:55442) 2023-08-27T11:17:20.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbc3fff700 1 -- 172.21.15.112:0/3208297843 learned_addr learned my addr 172.21.15.112:0/3208297843 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:20.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbc3fff700 1 -- 172.21.15.112:0/3208297843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e7503d0 msgr2=0x55568e776280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:20.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbc2ffd700 1 --2- 172.21.15.112:0/3208297843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e77cf40 0x55568e7507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/3208297843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e7503d0 0x55568e776280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbc3fff700 1 -- 172.21.15.112:0/3208297843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e77cf40 msgr2=0x55568e7507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/3208297843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e77cf40 0x55568e7507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbc3fff700 1 -- 172.21.15.112:0/3208297843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55568e4d0d90 con 0x55568e78e960 2023-08-27T11:17:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/3208297843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e74f320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc007af0 tx=0x7fcbbc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e561a97d4bb98d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbb9ffb700 1 -- 172.21.15.112:0/3208297843 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc005b50 con 0x55568e78e960 2023-08-27T11:17:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbb9ffb700 1 -- 172.21.15.112:0/3208297843 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc007be0 con 0x55568e78e960 2023-08-27T11:17:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.344+0000 7fcbb9ffb700 1 -- 172.21.15.112:0/3208297843 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc014480 con 0x55568e78e960 2023-08-27T11:17:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.345+0000 7fcbd621eb80 1 -- 172.21.15.112:0/3208297843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 msgr2=0x55568e74f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.345+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/3208297843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e74f320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc007af0 tx=0x7fcbbc01c000 comp rx=0 tx=0).stop 2023-08-27T11:17:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.345+0000 7fcbd621eb80 1 -- 172.21.15.112:0/3208297843 shutdown_connections 2023-08-27T11:17:20.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.345+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/3208297843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e74f320 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.345+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/3208297843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e7503d0 0x55568e776280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.345+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/3208297843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e77cf40 0x55568e7507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.345+0000 7fcbd621eb80 1 -- 172.21.15.112:0/3208297843 >> 172.21.15.112:0/3208297843 conn(0x55568e6b0c30 msgr2=0x55568e796cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:20.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.345+0000 7fcbd621eb80 1 -- 172.21.15.112:0/3208297843 shutdown_connections 2023-08-27T11:17:20.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.345+0000 7fcbd621eb80 1 -- 172.21.15.112:0/3208297843 wait complete. 2023-08-27T11:17:20.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.345+0000 7fcbd621eb80 1 -- start start 2023-08-27T11:17:20.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.345+0000 7fcbd621eb80 1 Processor -- start 2023-08-27T11:17:20.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbd621eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e7503d0 0x55568e78d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbd621eb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e77cf40 0x55568e74e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbd621eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e7947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbd621eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568e78dcb0 con 0x55568e78e960 2023-08-27T11:17:20.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbd621eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568e779a70 con 0x55568e7503d0 2023-08-27T11:17:20.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbd621eb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55568e779d30 con 0x55568e77cf40 2023-08-27T11:17:20.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbd621eb80 10 client.0 _ll_register_callbacks cb 0x55568e640630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:17:20.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e7947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e7947f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55448/0 (socket says 172.21.15.112:55448) 2023-08-27T11:17:20.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbc3fff700 1 -- 172.21.15.112:0/2342537789 learned_addr learned my addr 172.21.15.112:0/2342537789 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:20.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbc2ffd700 1 --2- 172.21.15.112:0/2342537789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e77cf40 0x55568e74e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbc37fe700 1 --2- 172.21.15.112:0/2342537789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e7503d0 0x55568e78d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbc3fff700 1 -- 172.21.15.112:0/2342537789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e77cf40 msgr2=0x55568e74e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:20.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/2342537789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e77cf40 0x55568e74e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbc3fff700 1 -- 172.21.15.112:0/2342537789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e7503d0 msgr2=0x55568e78d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:20.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.346+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/2342537789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e7503d0 0x55568e78d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcbc3fff700 1 -- 172.21.15.112:0/2342537789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55568e6bc750 con 0x55568e78e960 2023-08-27T11:17:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcbc3fff700 1 --2- 172.21.15.112:0/2342537789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e7947f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc001a80 tx=0x7fcbbc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b40a97ff7aa200 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcba27fc700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:17:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcba27fc700 1 -- 172.21.15.112:0/2342537789 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc014480 con 0x55568e78e960 2023-08-27T11:17:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcba27fc700 1 -- 172.21.15.112:0/2342537789 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbbc0075f0 con 0x55568e78e960 2023-08-27T11:17:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcbd621eb80 1 -- 172.21.15.112:0/2342537789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55568e65e000 con 0x55568e78e960 2023-08-27T11:17:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcba27fc700 1 -- 172.21.15.112:0/2342537789 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbbc0145e0 con 0x55568e78e960 2023-08-27T11:17:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcbd621eb80 10 client.4662 fetch_fsmap learned FSMap version 7 2023-08-27T11:17:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcbd621eb80 1 -- 172.21.15.112:0/2342537789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55568e4b9c10 con 0x55568e78e960 2023-08-27T11:17:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcba27fc700 1 -- 172.21.15.112:0/2342537789 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55568e65e000 con 0x55568e78e960 2023-08-27T11:17:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcba27fc700 1 -- 172.21.15.112:0/2342537789 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcbbc014bf0 con 0x55568e78e960 2023-08-27T11:17:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcbd621eb80 10 client.4662 fetch_fsmap finished waiting for FSMap version 7 2023-08-27T11:17:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcbd621eb80 10 client.4662 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-27T11:17:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcbd621eb80 20 client.4662 populate_metadata read hostname 'smithi112' 2023-08-27T11:17:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/2342537789 >> [v2:172.21.15.138:6835/1827312637,v1:172.21.15.138:6837/1827312637] conn(0x55568e794f40 0x55568e799590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcbd621eb80 4 client.4662 mds_command: new command op to 4285 tid=0[{"prefix": "status"}] 2023-08-27T11:17:20.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcbd621eb80 1 -- 172.21.15.112:0/2342537789 --> [v2:172.21.15.138:6835/1827312637,v1:172.21.15.138:6837/1827312637] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55568e4f1f20 con 0x55568e794f40 2023-08-27T11:17:20.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.347+0000 7fcbc37fe700 1 --2- 172.21.15.112:0/2342537789 >> [v2:172.21.15.138:6835/1827312637,v1:172.21.15.138:6837/1827312637] conn(0x55568e794f40 0x55568e799590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcba27fc700 10 client.4662 ms_handle_connect on v2:172.21.15.138:6835/1827312637 2023-08-27T11:17:20.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcbc37fe700 1 --2- 172.21.15.112:0/2342537789 >> [v2:172.21.15.138:6835/1827312637,v1:172.21.15.138:6837/1827312637] conn(0x55568e794f40 0x55568e799590 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf8fe2b0a0c4875 server_cookie=1e80acf529b40068 in_seq=0 out_seq=0 2023-08-27T11:17:20.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcba27fc700 1 -- 172.21.15.112:0/2342537789 <== mds.0 v2:172.21.15.138:6835/1827312637 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55568e4f1f20 con 0x55568e794f40 2023-08-27T11:17:20.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcba27fc700 10 client.4662 handle_command_reply: tid=0 2023-08-27T11:17:20.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcbd621eb80 1 client.4662 shutdown 2023-08-27T11:17:20.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcbd621eb80 1 -- 172.21.15.112:0/2342537789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 msgr2=0x55568e7947f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/2342537789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e7947f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc001a80 tx=0x7fcbbc009040 comp rx=0 tx=0).stop 2023-08-27T11:17:20.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcbd621eb80 1 -- 172.21.15.112:0/2342537789 shutdown_connections 2023-08-27T11:17:20.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/2342537789 >> [v2:172.21.15.138:6835/1827312637,v1:172.21.15.138:6837/1827312637] conn(0x55568e794f40 0x55568e799590 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/2342537789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55568e78e960 0x55568e7947f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/2342537789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55568e7503d0 0x55568e78d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcba27fc700 0 client.4662 ms_handle_reset on v2:172.21.15.138:6835/1827312637 2023-08-27T11:17:20.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcbd621eb80 1 --2- 172.21.15.112:0/2342537789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55568e77cf40 0x55568e74e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcbd621eb80 1 -- 172.21.15.112:0/2342537789 >> 172.21.15.112:0/2342537789 conn(0x55568e6b0c30 msgr2=0x55568e74f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcbd621eb80 1 -- 172.21.15.112:0/2342537789 shutdown_connections 2023-08-27T11:17:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcbd621eb80 1 -- 172.21.15.112:0/2342537789 wait complete. 2023-08-27T11:17:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.348+0000 7fcbd621eb80 20 client.4662 trim_cache size 0 max 16384 2023-08-27T11:17:20.371 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:17:20.372 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e099a24f-c1cd-4f11-ac4f-ecacc10e5e98", 2023-08-27T11:17:20.372 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2023-08-27T11:17:20.372 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4285, 2023-08-27T11:17:20.372 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2023-08-27T11:17:20.372 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2023-08-27T11:17:20.373 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2023-08-27T11:17:20.373 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 14.967860482000001, 2023-08-27T11:17:20.373 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 7, 2023-08-27T11:17:20.373 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 20, 2023-08-27T11:17:20.373 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 20, 2023-08-27T11:17:20.374 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 20.905508008000002 2023-08-27T11:17:20.374 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:17:20.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.349+0000 7fcbca05c700 1 -- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcba406a0e0 msgr2=0x7fcba406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.349+0000 7fcbca05c700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcba406a0e0 0x7fcba406c5a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcbc413ac50 tx=0x7fcbac008030 comp rx=0 tx=0).stop 2023-08-27T11:17:20.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.349+0000 7fcbca05c700 1 -- 172.21.15.112:0/2341379868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbc406d510 msgr2=0x7fcbc41f6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.349+0000 7fcbca05c700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbc406d510 0x7fcbc41f6910 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400efc0 tx=0x7fcbb400c3a0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.349+0000 7fcbca05c700 1 -- 172.21.15.112:0/2341379868 shutdown_connections 2023-08-27T11:17:20.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.349+0000 7fcbca05c700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcba406a0e0 0x7fcba406c5a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.349+0000 7fcbca05c700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc4160a00 0x7fcbc40b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.349+0000 7fcbca05c700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbc406d510 0x7fcbc41f6910 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.349+0000 7fcbca05c700 1 --2- 172.21.15.112:0/2341379868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbc4146990 0x7fcbc41f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.349+0000 7fcbca05c700 1 -- 172.21.15.112:0/2341379868 >> 172.21.15.112:0/2341379868 conn(0x7fcbc40cfe50 msgr2=0x7fcbc4073410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:20.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.349+0000 7fcbca05c700 1 -- 172.21.15.112:0/2341379868 shutdown_connections 2023-08-27T11:17:20.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.349+0000 7fcbca05c700 1 -- 172.21.15.112:0/2341379868 wait complete. 2023-08-27T11:17:20.378 INFO:teuthology.run_tasks:Running task kclient... 2023-08-27T11:17:20.400 INFO:tasks.kclient:Mounting kernel clients... 2023-08-27T11:17:20.401 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-08-27T11:17:20.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:20.402 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:17:20.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:20.419 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2023-08-27T11:17:20.488 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2023-08-27T11:17:20.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:17:20.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:20.490 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:17:20.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:20.546 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2023-08-27T11:17:20.613 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2023-08-27T11:17:20.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:17:20.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:20.615 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:17:20.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:20.636 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2023-08-27T11:17:20.711 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2023-08-27T11:17:20.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:17:20.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:20.713 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:17:20.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:20.773 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2023-08-27T11:17:20.842 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2023-08-27T11:17:20.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:17:20.843 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-08-27T11:17:20.843 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-27T11:17:20.844 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:17:20.844 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:17:20.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.962+0000 7fa7d9db2700 1 Processor -- start 2023-08-27T11:17:20.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.963+0000 7fa7d9db2700 1 -- start start 2023-08-27T11:17:20.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.963+0000 7fa7d9db2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7d40b17e0 0x7fa7d40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.963+0000 7fa7d9db2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d4161ce0 0x7fa7d406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.963+0000 7fa7d9db2700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d4161850 con 0x7fa7d40b1400 2023-08-27T11:17:20.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.963+0000 7fa7d9db2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d40cfc10 con 0x7fa7d40b17e0 2023-08-27T11:17:20.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.963+0000 7fa7d9db2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4161200 con 0x7fa7d4161ce0 2023-08-27T11:17:20.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d3fff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d4161ce0 0x7fa7d406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d2ffd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7d40b17e0 0x7fa7d40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d37fe700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa7d40b1400 0x7fa7d406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:57358/0 (socket says 172.21.15.112:57358) 2023-08-27T11:17:20.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d2ffd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7d40b17e0 0x7fa7d40cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:55880/0 (socket says 172.21.15.112:55880) 2023-08-27T11:17:20.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d37fe700 1 -- 172.21.15.112:0/2051101910 learned_addr learned my addr 172.21.15.112:0/2051101910 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:20.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d3fff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d4161ce0 0x7fa7d406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38878/0 (socket says 172.21.15.112:38878) 2023-08-27T11:17:20.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d2ffd700 1 -- 172.21.15.112:0/2051101910 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d4161ce0 msgr2=0x7fa7d406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:20.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d2ffd700 1 --2- 172.21.15.112:0/2051101910 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d4161ce0 0x7fa7d406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d2ffd700 1 -- 172.21.15.112:0/2051101910 >> v1:172.21.15.138:6789/0 conn(0x7fa7d40b1400 legacy=0x7fa7d406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d2ffd700 1 -- 172.21.15.112:0/2051101910 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d41614e0 con 0x7fa7d40b17e0 2023-08-27T11:17:20.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d3fff700 1 --2- 172.21.15.112:0/2051101910 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d4161ce0 0x7fa7d406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:20.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d2ffd700 1 --2- 172.21.15.112:0/2051101910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d40b17e0 0x7fa7d40cf5a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009840 tx=0x7fa7bc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b55ae705f09b45d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:20.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d27fc700 1 -- 172.21.15.112:0/2051101910 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc011830 con 0x7fa7d40b17e0 2023-08-27T11:17:20.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d27fc700 1 -- 172.21.15.112:0/2051101910 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc011990 con 0x7fa7d40b17e0 2023-08-27T11:17:20.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.964+0000 7fa7d27fc700 1 -- 172.21.15.112:0/2051101910 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc011d20 con 0x7fa7d40b17e0 2023-08-27T11:17:20.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.965+0000 7fa7d9db2700 1 -- 172.21.15.112:0/2051101910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d40b17e0 msgr2=0x7fa7d40cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.965+0000 7fa7d9db2700 1 --2- 172.21.15.112:0/2051101910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d40b17e0 0x7fa7d40cf5a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009840 tx=0x7fa7bc009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:20.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.965+0000 7fa7d9db2700 1 -- 172.21.15.112:0/2051101910 shutdown_connections 2023-08-27T11:17:20.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.965+0000 7fa7d9db2700 1 --2- 172.21.15.112:0/2051101910 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d4161ce0 0x7fa7d406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.965+0000 7fa7d9db2700 1 --2- 172.21.15.112:0/2051101910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d40b17e0 0x7fa7d40cf5a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.965+0000 7fa7d9db2700 1 -- 172.21.15.112:0/2051101910 >> 172.21.15.112:0/2051101910 conn(0x7fa7d40d0000 msgr2=0x7fa7d40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:20.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.965+0000 7fa7d9db2700 1 -- 172.21.15.112:0/2051101910 shutdown_connections 2023-08-27T11:17:20.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.965+0000 7fa7d9db2700 1 -- 172.21.15.112:0/2051101910 wait complete. 2023-08-27T11:17:20.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.966+0000 7fa7d9db2700 1 Processor -- start 2023-08-27T11:17:20.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.966+0000 7fa7d9db2700 1 -- start start 2023-08-27T11:17:20.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d9db2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d40b1400 0x7fa7d41f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d9db2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d40b17e0 0x7fa7d4141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d9db2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d4161ce0 0x7fa7d41f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d9db2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4142120 con 0x7fa7d40b1400 2023-08-27T11:17:20.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d9db2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4141670 con 0x7fa7d4161ce0 2023-08-27T11:17:20.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d9db2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d40c88b0 con 0x7fa7d40b17e0 2023-08-27T11:17:20.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d40b1400 0x7fa7d41f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d40b1400 0x7fa7d41f6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55458/0 (socket says 172.21.15.112:55458) 2023-08-27T11:17:20.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d37fe700 1 -- 172.21.15.112:0/57479800 learned_addr learned my addr 172.21.15.112:0/57479800 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:20.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d2ffd700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d40b17e0 0x7fa7d4141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d3fff700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d4161ce0 0x7fa7d41f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d37fe700 1 -- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d40b17e0 msgr2=0x7fa7d4141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d37fe700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d40b17e0 0x7fa7d4141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d37fe700 1 -- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d4161ce0 msgr2=0x7fa7d41f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:20.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d37fe700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d4161ce0 0x7fa7d41f26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:20.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d37fe700 1 -- 172.21.15.112:0/57479800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc009a70 con 0x7fa7d40b1400 2023-08-27T11:17:20.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d3fff700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d4161ce0 0x7fa7d41f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:20.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.967+0000 7fa7d37fe700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d40b1400 0x7fa7d41f6da0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400c520 tx=0x7fa7c400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:20.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.968+0000 7fa7cbfff700 1 -- 172.21.15.112:0/57479800 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4005d70 con 0x7fa7d40b1400 2023-08-27T11:17:20.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.968+0000 7fa7cbfff700 1 -- 172.21.15.112:0/57479800 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c40070d0 con 0x7fa7d40b1400 2023-08-27T11:17:20.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.968+0000 7fa7d9db2700 1 -- 172.21.15.112:0/57479800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc0094a0 con 0x7fa7d40b1400 2023-08-27T11:17:20.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.968+0000 7fa7cbfff700 1 -- 172.21.15.112:0/57479800 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4007440 con 0x7fa7d40b1400 2023-08-27T11:17:20.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.968+0000 7fa7d9db2700 1 -- 172.21.15.112:0/57479800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d4141b60 con 0x7fa7d40b1400 2023-08-27T11:17:20.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.969+0000 7fa7d9db2700 1 -- 172.21.15.112:0/57479800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d4067a20 con 0x7fa7d40b1400 2023-08-27T11:17:20.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.969+0000 7fa7cbfff700 1 -- 172.21.15.112:0/57479800 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa7c40075a0 con 0x7fa7d40b1400 2023-08-27T11:17:20.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.969+0000 7fa7cbfff700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa7b406a0e0 0x7fa7b406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:20.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.970+0000 7fa7cbfff700 1 -- 172.21.15.112:0/57479800 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7c4087960 con 0x7fa7d40b1400 2023-08-27T11:17:20.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.970+0000 7fa7d2ffd700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa7b406a0e0 0x7fa7b406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:20.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.970+0000 7fa7d2ffd700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa7b406a0e0 0x7fa7b406c5a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40b1bf0 tx=0x7fa7bc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:20.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:20.974+0000 7fa7cbfff700 1 -- 172.21.15.112:0/57479800 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa7c40530f0 con 0x7fa7d40b1400 2023-08-27T11:17:21.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.205+0000 7fa7d9db2700 1 -- 172.21.15.112:0/57479800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa7d406db60 con 0x7fa7d40b1400 2023-08-27T11:17:21.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.205+0000 7fa7cbfff700 1 -- 172.21.15.112:0/57479800 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa7c4017060 con 0x7fa7d40b1400 2023-08-27T11:17:21.207 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:17:21.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.207+0000 7fa7d9db2700 1 -- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa7b406a0e0 msgr2=0x7fa7b406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:21.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.207+0000 7fa7d9db2700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa7b406a0e0 0x7fa7b406c5a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40b1bf0 tx=0x7fa7bc008030 comp rx=0 tx=0).stop 2023-08-27T11:17:21.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.207+0000 7fa7d9db2700 1 -- 172.21.15.112:0/57479800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d40b1400 msgr2=0x7fa7d41f6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:21.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.207+0000 7fa7d9db2700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d40b1400 0x7fa7d41f6da0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400c520 tx=0x7fa7c400c8e0 comp rx=0 tx=0).stop 2023-08-27T11:17:21.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.207+0000 7fa7d9db2700 1 -- 172.21.15.112:0/57479800 shutdown_connections 2023-08-27T11:17:21.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.207+0000 7fa7d9db2700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa7b406a0e0 0x7fa7b406c5a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:21.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.207+0000 7fa7d9db2700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d4161ce0 0x7fa7d41f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:21.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.207+0000 7fa7d9db2700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d40b1400 0x7fa7d41f6da0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:21.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.207+0000 7fa7d9db2700 1 --2- 172.21.15.112:0/57479800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d40b17e0 0x7fa7d4141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:21.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.207+0000 7fa7d9db2700 1 -- 172.21.15.112:0/57479800 >> 172.21.15.112:0/57479800 conn(0x7fa7d40d0000 msgr2=0x7fa7d4157970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:21.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.207+0000 7fa7d9db2700 1 -- 172.21.15.112:0/57479800 shutdown_connections 2023-08-27T11:17:21.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:21.207+0000 7fa7d9db2700 1 -- 172.21.15.112:0/57479800 wait complete. 2023-08-27T11:17:21.221 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-27T11:17:21.221 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:17:21.222 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-08-27T11:17:21.222 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:17:21.222 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:17:21.222 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-27T11:17:21.223 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-27T11:17:21.223 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:17:21.223 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:17:21.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:21.224 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-08-27T11:17:21.240 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:17:21.241 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:17:21.241 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:17:21.241 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:21.241 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:17:21.242 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:21.242 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:17:21.242 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:21.242 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:17:21.242 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:21.243 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:17:21.243 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:21.243 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:17:21.243 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:21.243 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:17:21.244 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42401sec preferred_lft 42401sec 2023-08-27T11:17:21.244 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8d6c:4fe6:6a84:8222/64 scope link noprefixroute 2023-08-27T11:17:21.244 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:21.244 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-27T11:17:21.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:21.245 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:17:21.245 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:17:21.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2023-08-27T11:17:21.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2023-08-27T11:17:21.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2023-08-27T11:17:21.247 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-27T11:17:21.247 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:17:21.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:21.417 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:17:21.445 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-08-27T11:17:21.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:21.447 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-08-27T11:17:21.542 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-08-27T11:17:21.543 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:17:21.543 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:17:21.543 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:17:21.543 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:17:21.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:21.544 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:17:21.545 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:17:21.545 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:17:21.546 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:17:21.546 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:17:21.547 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:17:21.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:21.752 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:17:21.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:21.770 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2023-08-27T11:17:21.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:21.826 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:17:21.826 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:17:21.826 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:17:21.827 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-08-27T11:17:21.827 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:17:21.920 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-08-27T11:17:21.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:21.921 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:17:21.921 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:17:21.921 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-08-27T11:17:21.921 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-27T11:17:21.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-08-27T11:17:21.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-08-27T11:17:21.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-08-27T11:17:21.923 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:17:22.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:22.141 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:17:22.141 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:17:22.142 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-08-27T11:17:22.142 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2023-08-27T11:17:22.142 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:17:22.241 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:17:22.242 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:17:22.262 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:17:22.262 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:17:22.263 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:17:22.318 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:17:22.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:22.319 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:17:22.409 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:17:22.410 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:17:22.410 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:17:22.504 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:17:22.505 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:17:22.505 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:17:22.506 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:17:22.506 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:17:22.582 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:17:22.583 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:17:22.583 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:17:22.583 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:17:22.583 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:17:22.584 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:17:22.584 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:17:22.584 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:17:22.586 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:17:22.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:22.586 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:17:22.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:22.614 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:17:22.701 INFO:teuthology.orchestra.run.smithi112.stdout:client4642 2023-08-27T11:17:22.701 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:17:22.702 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:22.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:22.719 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:17:22.807 INFO:teuthology.orchestra.run.smithi112.stdout:client4642 2023-08-27T11:17:22.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:22.808 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:17:22.855 INFO:teuthology.orchestra.run.smithi112.stdout:client4642 2023-08-27T11:17:22.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:22.856 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e099a24f-c1cd-4f11-ac4f-ecacc10e5e98.client4642/status') 2023-08-27T11:17:22.907 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.4642 (3)172.21.15.112:0/85844508 2023-08-27T11:17:22.907 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2023-08-27T11:17:22.908 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2023-08-27T11:17:22.908 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2023-08-27T11:17:22.909 INFO:teuthology.orchestra.run.smithi112.stderr:69 bytes copied, 2.2572e-05 s, 3.1 MB/s 2023-08-27T11:17:22.909 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-08-27T11:17:22.909 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-27T11:17:22.910 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-08-27T11:17:22.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:17:23.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.030+0000 7f44ebdbe700 1 Processor -- start 2023-08-27T11:17:23.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.030+0000 7f44ebdbe700 1 -- start start 2023-08-27T11:17:23.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44ebdbe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44e406dc20 0x7f44e40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:23.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44ebdbe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f44e406cf80 0x7f44e415e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:23.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44ebdbe700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e415b3e0 con 0x7f44e40708a0 2023-08-27T11:17:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44ebdbe700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e415a5c0 con 0x7f44e406cf80 2023-08-27T11:17:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44ebdbe700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e415eb00 con 0x7f44e406dc20 2023-08-27T11:17:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e9359700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f44e406cf80 0x7f44e415e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e9359700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f44e406cf80 0x7f44e415e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55474/0 (socket says 172.21.15.112:55474) 2023-08-27T11:17:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e9359700 1 -- 172.21.15.112:0/1814072465 learned_addr learned my addr 172.21.15.112:0/1814072465 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:23.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e9b5a700 1 --2- 172.21.15.112:0/1814072465 >> v2:172.21.15.138:3300/0 conn(0x7f44e406dc20 0x7f44e40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:23.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e9359700 1 -- 172.21.15.112:0/1814072465 >> v2:172.21.15.138:3300/0 conn(0x7f44e406dc20 msgr2=0x7f44e40ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:23.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e9359700 1 --2- 172.21.15.112:0/1814072465 >> v2:172.21.15.138:3300/0 conn(0x7f44e406dc20 0x7f44e40ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:23.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e9359700 1 -- 172.21.15.112:0/1814072465 >> v1:172.21.15.112:6789/0 conn(0x7f44e40708a0 legacy=0x7f44e4070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:23.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e9359700 1 -- 172.21.15.112:0/1814072465 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e415a7d0 con 0x7f44e406cf80 2023-08-27T11:17:23.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e9b5a700 1 --2- 172.21.15.112:0/1814072465 >> v2:172.21.15.138:3300/0 conn(0x7f44e406dc20 0x7f44e40ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:23.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e8b58700 1 -- 172.21.15.112:0/1814072465 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741330025 0 0) 0x7f44e415b3e0 con 0x7f44e40708a0 2023-08-27T11:17:23.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e9359700 1 --2- 172.21.15.112:0/1814072465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e406cf80 0x7f44e415e5c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f44e40cfee0 tx=0x7f44d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd579a192f474a7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:23.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e8b58700 1 -- 172.21.15.112:0/1814072465 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d400b870 con 0x7f44e406cf80 2023-08-27T11:17:23.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e8b58700 1 -- 172.21.15.112:0/1814072465 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d400b9d0 con 0x7f44e406cf80 2023-08-27T11:17:23.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.031+0000 7f44e8b58700 1 -- 172.21.15.112:0/1814072465 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d400bd60 con 0x7f44e406cf80 2023-08-27T11:17:23.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.032+0000 7f44ebdbe700 1 -- 172.21.15.112:0/1814072465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e406cf80 msgr2=0x7f44e415e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:23.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.032+0000 7f44ebdbe700 1 --2- 172.21.15.112:0/1814072465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e406cf80 0x7f44e415e5c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f44e40cfee0 tx=0x7f44d4009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:23.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.032+0000 7f44ebdbe700 1 -- 172.21.15.112:0/1814072465 shutdown_connections 2023-08-27T11:17:23.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.032+0000 7f44ebdbe700 1 --2- 172.21.15.112:0/1814072465 >> v2:172.21.15.138:3300/0 conn(0x7f44e406dc20 0x7f44e40ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:23.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.032+0000 7f44ebdbe700 1 --2- 172.21.15.112:0/1814072465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e406cf80 0x7f44e415e5c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:23.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.032+0000 7f44ebdbe700 1 -- 172.21.15.112:0/1814072465 >> 172.21.15.112:0/1814072465 conn(0x7f44e40b0bf0 msgr2=0x7f44e40b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:23.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.032+0000 7f44ebdbe700 1 -- 172.21.15.112:0/1814072465 shutdown_connections 2023-08-27T11:17:23.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.032+0000 7f44ebdbe700 1 -- 172.21.15.112:0/1814072465 wait complete. 2023-08-27T11:17:23.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.034+0000 7f44ebdbe700 1 Processor -- start 2023-08-27T11:17:23.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.034+0000 7f44ebdbe700 1 -- start start 2023-08-27T11:17:23.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44ebdbe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e406cf80 0x7f44e414b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44ebdbe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44e406dc20 0x7f44e41575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44ebdbe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44e40708a0 0x7f44e4155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44ebdbe700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40701a0 con 0x7f44e406cf80 2023-08-27T11:17:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44ebdbe700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40706d0 con 0x7f44e40708a0 2023-08-27T11:17:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44ebdbe700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e415c8e0 con 0x7f44e406dc20 2023-08-27T11:17:23.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44e9b5a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e406cf80 0x7f44e414b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:23.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44e9b5a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e406cf80 0x7f44e414b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55478/0 (socket says 172.21.15.112:55478) 2023-08-27T11:17:23.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44e9b5a700 1 -- 172.21.15.112:0/3914838271 learned_addr learned my addr 172.21.15.112:0/3914838271 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:23.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44ea35b700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44e40708a0 0x7f44e4155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:23.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44e9359700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44e406dc20 0x7f44e41575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:23.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44e9b5a700 1 -- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44e406dc20 msgr2=0x7f44e41575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:23.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44e9b5a700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44e406dc20 0x7f44e41575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:23.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44e9b5a700 1 -- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44e40708a0 msgr2=0x7f44e4155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:23.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44e9b5a700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44e40708a0 0x7f44e4155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:23.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44e9b5a700 1 -- 172.21.15.112:0/3914838271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d4009550 con 0x7f44e406cf80 2023-08-27T11:17:23.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44ea35b700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44e40708a0 0x7f44e4155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44e9b5a700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e406cf80 0x7f44e414b7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f44e000c850 tx=0x7f44e000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44da7fc700 1 -- 172.21.15.112:0/3914838271 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0007730 con 0x7f44e406cf80 2023-08-27T11:17:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44da7fc700 1 -- 172.21.15.112:0/3914838271 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e0007890 con 0x7f44e406cf80 2023-08-27T11:17:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44ebdbe700 1 -- 172.21.15.112:0/3914838271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e40703f0 con 0x7f44e406cf80 2023-08-27T11:17:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44da7fc700 1 -- 172.21.15.112:0/3914838271 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0007c20 con 0x7f44e406cf80 2023-08-27T11:17:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.035+0000 7f44ebdbe700 1 -- 172.21.15.112:0/3914838271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e4154530 con 0x7f44e406cf80 2023-08-27T11:17:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.036+0000 7f44ebdbe700 1 -- 172.21.15.112:0/3914838271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e4071890 con 0x7f44e406cf80 2023-08-27T11:17:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.037+0000 7f44da7fc700 1 -- 172.21.15.112:0/3914838271 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f44e00113f0 con 0x7f44e406cf80 2023-08-27T11:17:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.037+0000 7f44da7fc700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f44c806a0e0 0x7f44c806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.037+0000 7f44da7fc700 1 -- 172.21.15.112:0/3914838271 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f44e0088480 con 0x7f44e406cf80 2023-08-27T11:17:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.037+0000 7f44e9359700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f44c806a0e0 0x7f44c806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.038+0000 7f44e9359700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f44c806a0e0 0x7f44c806c5a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f44d4014040 tx=0x7f44d4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.042+0000 7f44da7fc700 1 -- 172.21.15.112:0/3914838271 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f44e0053c00 con 0x7f44e406cf80 2023-08-27T11:17:23.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.271+0000 7f44ebdbe700 1 -- 172.21.15.112:0/3914838271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f44e4067a20 con 0x7f44e406cf80 2023-08-27T11:17:23.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.272+0000 7f44da7fc700 1 -- 172.21.15.112:0/3914838271 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f44e0057220 con 0x7f44e406cf80 2023-08-27T11:17:23.273 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:17:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.273+0000 7f44ebdbe700 1 -- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f44c806a0e0 msgr2=0x7f44c806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.273+0000 7f44ebdbe700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f44c806a0e0 0x7f44c806c5a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f44d4014040 tx=0x7f44d4008030 comp rx=0 tx=0).stop 2023-08-27T11:17:23.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.273+0000 7f44ebdbe700 1 -- 172.21.15.112:0/3914838271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e406cf80 msgr2=0x7f44e414b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:23.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.273+0000 7f44ebdbe700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e406cf80 0x7f44e414b7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f44e000c850 tx=0x7f44e000cc10 comp rx=0 tx=0).stop 2023-08-27T11:17:23.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.274+0000 7f44ebdbe700 1 -- 172.21.15.112:0/3914838271 shutdown_connections 2023-08-27T11:17:23.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.274+0000 7f44ebdbe700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f44c806a0e0 0x7f44c806c5a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:23.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.274+0000 7f44ebdbe700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44e40708a0 0x7f44e4155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:23.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.274+0000 7f44ebdbe700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e406cf80 0x7f44e414b7a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:23.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.274+0000 7f44ebdbe700 1 --2- 172.21.15.112:0/3914838271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44e406dc20 0x7f44e41575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:23.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.274+0000 7f44ebdbe700 1 -- 172.21.15.112:0/3914838271 >> 172.21.15.112:0/3914838271 conn(0x7f44e40b0bf0 msgr2=0x7f44e40cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:23.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.274+0000 7f44ebdbe700 1 -- 172.21.15.112:0/3914838271 shutdown_connections 2023-08-27T11:17:23.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:23.274+0000 7f44ebdbe700 1 -- 172.21.15.112:0/3914838271 wait complete. 2023-08-27T11:17:23.287 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-27T11:17:23.287 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:17:23.288 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-08-27T11:17:23.288 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-08-27T11:17:23.288 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-08-27T11:17:23.288 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-27T11:17:23.289 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-27T11:17:23.289 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:17:23.289 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-08-27T11:17:23.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:23.290 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-08-27T11:17:23.307 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:17:23.307 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:17:23.307 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:17:23.308 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.308 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:17:23.308 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.308 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:17:23.308 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:23.309 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:17:23.309 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:23.309 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:17:23.309 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:23.309 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:17:23.310 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:23.310 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:17:23.310 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42399sec preferred_lft 42399sec 2023-08-27T11:17:23.310 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8d6c:4fe6:6a84:8222/64 scope link noprefixroute 2023-08-27T11:17:23.310 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.310 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:17:23.311 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:23.311 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:17:23.311 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.311 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::d87c:a1ff:fe1d:b635/64 scope link 2023-08-27T11:17:23.311 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.312 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:17:23.312 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:17:23.312 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::cc14:79ff:feb3:452f/64 scope link tentative 2023-08-27T11:17:23.312 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:23.313 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:17:23.372 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-08-27T11:17:23.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:23.376 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-08-27T11:17:23.431 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-08-27T11:17:23.431 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:17:23.432 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:17:23.432 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:17:23.432 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:17:23.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:23.433 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:17:23.434 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:17:23.434 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:17:23.435 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:17:23.435 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:17:23.435 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:17:23.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:23.537 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:17:23.554 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:17:23.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:23.555 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2023-08-27T11:17:23.611 INFO:teuthology.orchestra.run.smithi112.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-08-27T11:17:23.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:23.612 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:17:23.612 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:17:23.613 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:17:23.613 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-08-27T11:17:23.613 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:17:23.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:23.704 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-08-27T11:17:23.743 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:17:23.743 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:17:23.744 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:17:23.744 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.744 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:17:23.744 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.744 INFO:teuthology.orchestra.run.smithi112.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:17:23.745 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 12:61:20:6b:32:e9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-27T11:17:23.745 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-27T11:17:23.745 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.745 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1061:20ff:fe6b:32e9/64 scope link 2023-08-27T11:17:23.745 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:23.760 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-08-27T11:17:23.834 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:17:23.834 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:17:23.835 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:17:23.835 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.835 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:17:23.835 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.835 INFO:teuthology.orchestra.run.smithi112.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:17:23.836 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 12:61:20:6b:32:e9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-27T11:17:23.836 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-27T11:17:23.836 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.836 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1061:20ff:fe6b:32e9/64 scope link 2023-08-27T11:17:23.836 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:23.847 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-08-27T11:17:23.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:23.848 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:17:23.848 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:17:23.849 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-08-27T11:17:23.849 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-08-27T11:17:23.849 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-08-27T11:17:23.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-08-27T11:17:23.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-08-27T11:17:23.850 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:17:24.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:24.094 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:17:24.095 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:17:24.095 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.1 up 2023-08-27T11:17:24.095 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.1 master ceph-brx 2023-08-27T11:17:24.096 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:17:24.203 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:17:24.204 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-08-27T11:17:24.223 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-08-27T11:17:24.224 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:17:24.224 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-08-27T11:17:24.281 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:17:24.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:24.281 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-08-27T11:17:24.366 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-08-27T11:17:24.366 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-08-27T11:17:24.366 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:17:24.374 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-08-27T11:17:24.375 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-08-27T11:17:24.375 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:17:24.375 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:17:24.376 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:17:24.402 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-08-27T11:17:24.402 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-08-27T11:17:24.403 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:17:24.403 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:17:24.403 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:17:24.403 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:17:24.404 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:17:24.404 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:17:24.407 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:17:24.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:24.408 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-08-27T11:17:24.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:24.442 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:17:24.514 INFO:teuthology.orchestra.run.smithi112.stdout:client4677 2023-08-27T11:17:24.515 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:17:24.515 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:24.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:24.573 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:17:24.645 INFO:teuthology.orchestra.run.smithi112.stdout:client4677 2023-08-27T11:17:24.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:24.646 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:17:24.721 INFO:teuthology.orchestra.run.smithi112.stdout:client4677 2023-08-27T11:17:24.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:24.722 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e099a24f-c1cd-4f11-ac4f-ecacc10e5e98.client4677/status') 2023-08-27T11:17:24.801 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.4677 (3)192.168.0.2:0/2332143688 2023-08-27T11:17:24.802 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2023-08-27T11:17:24.802 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2023-08-27T11:17:24.803 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2023-08-27T11:17:24.803 INFO:teuthology.orchestra.run.smithi112.stderr:69 bytes copied, 2.1385e-05 s, 3.2 MB/s 2023-08-27T11:17:24.803 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-08-27T11:17:24.804 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-27T11:17:24.804 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:17:24.804 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:17:24.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.962+0000 7feb16cf3700 1 Processor -- start 2023-08-27T11:17:24.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.962+0000 7feb16cf3700 1 -- start start 2023-08-27T11:17:24.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.962+0000 7feb16cf3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb10158e20 0x7feb100b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.962+0000 7feb16cf3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feb1006e3a0 0x7feb1006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.962+0000 7feb16cf3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb1015c190 con 0x7feb1006fcd0 2023-08-27T11:17:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.962+0000 7feb16cf3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb100b1730 con 0x7feb1006e3a0 2023-08-27T11:17:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.962+0000 7feb16cf3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb10070160 con 0x7feb10158e20 2023-08-27T11:17:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.963+0000 7feb0ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feb1006e3a0 0x7feb1006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.963+0000 7feb0ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feb1006e3a0 0x7feb1006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55502/0 (socket says 172.21.15.112:55502) 2023-08-27T11:17:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.963+0000 7feb0ffff700 1 -- 172.21.15.112:0/3537450509 learned_addr learned my addr 172.21.15.112:0/3537450509 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.963+0000 7feb0ffff700 1 -- 172.21.15.112:0/3537450509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb10158e20 msgr2=0x7feb100b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.963+0000 7feb0ffff700 1 --2- 172.21.15.112:0/3537450509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb10158e20 0x7feb100b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.963+0000 7feb0ffff700 1 -- 172.21.15.112:0/3537450509 >> v1:172.21.15.112:6789/0 conn(0x7feb1006fcd0 legacy=0x7feb1015ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.963+0000 7feb0ffff700 1 -- 172.21.15.112:0/3537450509 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb100b09d0 con 0x7feb1006e3a0 2023-08-27T11:17:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.963+0000 7feb0f7fe700 1 -- 172.21.15.112:0/3537450509 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 940131013 0 0) 0x7feb1015c190 con 0x7feb1006fcd0 2023-08-27T11:17:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.963+0000 7feb0ffff700 1 --2- 172.21.15.112:0/3537450509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb1006e3a0 0x7feb1006f790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feafc0098f0 tx=0x7feafc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d57e36174d6ff1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.963+0000 7feb0f7fe700 1 -- 172.21.15.112:0/3537450509 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00b8a0 con 0x7feb1006e3a0 2023-08-27T11:17:24.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.963+0000 7feb0f7fe700 1 -- 172.21.15.112:0/3537450509 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc00ba00 con 0x7feb1006e3a0 2023-08-27T11:17:24.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.963+0000 7feb0f7fe700 1 -- 172.21.15.112:0/3537450509 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00bd90 con 0x7feb1006e3a0 2023-08-27T11:17:24.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.964+0000 7feb16cf3700 1 -- 172.21.15.112:0/3537450509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb1006e3a0 msgr2=0x7feb1006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:24.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.964+0000 7feb16cf3700 1 --2- 172.21.15.112:0/3537450509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb1006e3a0 0x7feb1006f790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feafc0098f0 tx=0x7feafc009c00 comp rx=0 tx=0).stop 2023-08-27T11:17:24.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.964+0000 7feb16cf3700 1 -- 172.21.15.112:0/3537450509 shutdown_connections 2023-08-27T11:17:24.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.964+0000 7feb16cf3700 1 --2- 172.21.15.112:0/3537450509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb10158e20 0x7feb100b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:24.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.964+0000 7feb16cf3700 1 --2- 172.21.15.112:0/3537450509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb1006e3a0 0x7feb1006f790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:24.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.964+0000 7feb16cf3700 1 -- 172.21.15.112:0/3537450509 >> 172.21.15.112:0/3537450509 conn(0x7feb100d0000 msgr2=0x7feb100d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:24.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.964+0000 7feb16cf3700 1 -- 172.21.15.112:0/3537450509 shutdown_connections 2023-08-27T11:17:24.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.964+0000 7feb16cf3700 1 -- 172.21.15.112:0/3537450509 wait complete. 2023-08-27T11:17:24.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.965+0000 7feb16cf3700 1 Processor -- start 2023-08-27T11:17:24.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.965+0000 7feb16cf3700 1 -- start start 2023-08-27T11:17:24.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb16cf3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb1006e3a0 0x7feb100c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:24.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb16cf3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb1006fcd0 0x7feb101533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:24.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb16cf3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10158e20 0x7feb100c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:24.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb16cf3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb100c84a0 con 0x7feb10158e20 2023-08-27T11:17:24.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb16cf3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb100c7610 con 0x7feb1006fcd0 2023-08-27T11:17:24.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb16cf3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb101538e0 con 0x7feb1006e3a0 2023-08-27T11:17:24.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb15290700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10158e20 0x7feb100c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:24.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb15290700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10158e20 0x7feb100c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55508/0 (socket says 172.21.15.112:55508) 2023-08-27T11:17:24.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb15290700 1 -- 172.21.15.112:0/2867134032 learned_addr learned my addr 172.21.15.112:0/2867134032 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:24.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb0ffff700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb1006fcd0 0x7feb101533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:24.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb14a8f700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb1006e3a0 0x7feb100c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:24.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb15290700 1 -- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb1006e3a0 msgr2=0x7feb100c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:24.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb15290700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb1006e3a0 0x7feb100c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb15290700 1 -- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb1006fcd0 msgr2=0x7feb101533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb15290700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb1006fcd0 0x7feb101533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb15290700 1 -- 172.21.15.112:0/2867134032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc009550 con 0x7feb10158e20 2023-08-27T11:17:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb15290700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10158e20 0x7feb100c87b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feb0800c870 tx=0x7feb0800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb0d7fa700 1 -- 172.21.15.112:0/2867134032 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb08003e30 con 0x7feb10158e20 2023-08-27T11:17:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.966+0000 7feb0d7fa700 1 -- 172.21.15.112:0/2867134032 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb08005070 con 0x7feb10158e20 2023-08-27T11:17:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.967+0000 7feb16cf3700 1 -- 172.21.15.112:0/2867134032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb100c7890 con 0x7feb10158e20 2023-08-27T11:17:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.967+0000 7feb0d7fa700 1 -- 172.21.15.112:0/2867134032 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb08005420 con 0x7feb10158e20 2023-08-27T11:17:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.967+0000 7feb16cf3700 1 -- 172.21.15.112:0/2867134032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb100c7e50 con 0x7feb10158e20 2023-08-27T11:17:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.967+0000 7feb16cf3700 1 -- 172.21.15.112:0/2867134032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb10067a20 con 0x7feb10158e20 2023-08-27T11:17:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.968+0000 7feb0d7fa700 1 -- 172.21.15.112:0/2867134032 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7feb08005580 con 0x7feb10158e20 2023-08-27T11:17:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.968+0000 7feb0d7fa700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaf406a0e0 0x7feaf406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:24.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.968+0000 7feb0d7fa700 1 -- 172.21.15.112:0/2867134032 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feb08087b10 con 0x7feb10158e20 2023-08-27T11:17:24.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.968+0000 7feb14a8f700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaf406a0e0 0x7feaf406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:24.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.969+0000 7feb14a8f700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaf406a0e0 0x7feaf406c5a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feb04009970 tx=0x7feb04009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:24.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:24.972+0000 7feb0d7fa700 1 -- 172.21.15.112:0/2867134032 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7feb08053290 con 0x7feb10158e20 2023-08-27T11:17:25.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.204+0000 7feb16cf3700 1 -- 172.21.15.112:0/2867134032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feb100722b0 con 0x7feb10158e20 2023-08-27T11:17:25.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.204+0000 7feb0d7fa700 1 -- 172.21.15.112:0/2867134032 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7feb08017020 con 0x7feb10158e20 2023-08-27T11:17:25.206 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:17:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.206+0000 7feb16cf3700 1 -- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaf406a0e0 msgr2=0x7feaf406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.206+0000 7feb16cf3700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaf406a0e0 0x7feaf406c5a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feb04009970 tx=0x7feb04009110 comp rx=0 tx=0).stop 2023-08-27T11:17:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.206+0000 7feb16cf3700 1 -- 172.21.15.112:0/2867134032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10158e20 msgr2=0x7feb100c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.206+0000 7feb16cf3700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10158e20 0x7feb100c87b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feb0800c870 tx=0x7feb0800cb80 comp rx=0 tx=0).stop 2023-08-27T11:17:25.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.206+0000 7feb16cf3700 1 -- 172.21.15.112:0/2867134032 shutdown_connections 2023-08-27T11:17:25.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.206+0000 7feb16cf3700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaf406a0e0 0x7feaf406c5a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:25.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.206+0000 7feb16cf3700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10158e20 0x7feb100c87b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.206+0000 7feb16cf3700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb1006e3a0 0x7feb100c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.206+0000 7feb16cf3700 1 --2- 172.21.15.112:0/2867134032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb1006fcd0 0x7feb101533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.206+0000 7feb16cf3700 1 -- 172.21.15.112:0/2867134032 >> 172.21.15.112:0/2867134032 conn(0x7feb100d0000 msgr2=0x7feb1015f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.207+0000 7feb16cf3700 1 -- 172.21.15.112:0/2867134032 shutdown_connections 2023-08-27T11:17:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:25.207+0000 7feb16cf3700 1 -- 172.21.15.112:0/2867134032 wait complete. 2023-08-27T11:17:25.220 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-27T11:17:25.221 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:17:25.221 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2023-08-27T11:17:25.221 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-27T11:17:25.222 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:17:25.223 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-27T11:17:25.223 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-27T11:17:25.224 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:17:25.224 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:17:25.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:25.225 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2023-08-27T11:17:25.246 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:17:25.247 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:17:25.247 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:17:25.247 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:25.247 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:17:25.248 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:25.248 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:17:25.248 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:25.248 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:17:25.249 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:25.249 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:17:25.249 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:25.249 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:17:25.250 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:25.250 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:17:25.250 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42413sec preferred_lft 42413sec 2023-08-27T11:17:25.250 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ac17:49c5:cb39:801a/64 scope link noprefixroute 2023-08-27T11:17:25.250 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:25.251 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-27T11:17:25.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:25.252 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:17:25.252 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:17:25.253 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2023-08-27T11:17:25.253 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2023-08-27T11:17:25.254 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2023-08-27T11:17:25.254 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-27T11:17:25.254 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:17:25.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:25.435 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:17:25.463 INFO:teuthology.orchestra.run.smithi138.stdout:1 2023-08-27T11:17:25.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:25.465 DEBUG:teuthology.orchestra.run.smithi138:> route 2023-08-27T11:17:25.535 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2023-08-27T11:17:25.535 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:17:25.535 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:17:25.536 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:17:25.536 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:17:25.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:25.537 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:17:25.537 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:17:25.538 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:17:25.538 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:17:25.538 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:17:25.538 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:17:25.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:25.890 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:17:25.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:25.908 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2023-08-27T11:17:25.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:25.965 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:17:25.965 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:17:25.966 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:17:25.966 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-08-27T11:17:25.966 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:17:26.058 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-08-27T11:17:26.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:26.059 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:17:26.060 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:17:26.060 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-08-27T11:17:26.060 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-27T11:17:26.060 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-08-27T11:17:26.061 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-08-27T11:17:26.061 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-08-27T11:17:26.061 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:17:26.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:26.280 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:17:26.281 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:17:26.281 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2023-08-27T11:17:26.281 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2023-08-27T11:17:26.282 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:17:26.383 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:17:26.384 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-27T11:17:26.403 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:17:26.403 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:17:26.404 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:17:26.460 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:17:26.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:26.460 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:17:26.559 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:17:26.560 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:17:26.560 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:17:26.679 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:17:26.680 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:17:26.681 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:17:26.681 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:17:26.681 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:17:26.705 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:17:26.705 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:17:26.706 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:17:26.706 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:17:26.706 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:17:26.706 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:17:26.706 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:17:26.707 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:17:26.723 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:17:26.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:26.724 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:17:26.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:26.750 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:17:26.821 INFO:teuthology.orchestra.run.smithi138.stdout:client4496 2023-08-27T11:17:26.821 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:17:26.822 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:26.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:26.878 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:17:26.950 INFO:teuthology.orchestra.run.smithi138.stdout:client4496 2023-08-27T11:17:26.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:26.950 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:17:27.022 INFO:teuthology.orchestra.run.smithi138.stdout:client4496 2023-08-27T11:17:27.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:27.023 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e099a24f-c1cd-4f11-ac4f-ecacc10e5e98.client4496/status') 2023-08-27T11:17:27.102 INFO:teuthology.orchestra.run.smithi138.stdout:instance: client.4496 (3)192.168.0.1:0/3182345033 2023-08-27T11:17:27.103 INFO:teuthology.orchestra.run.smithi138.stdout:blocklisted: false 2023-08-27T11:17:27.103 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2023-08-27T11:17:27.104 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2023-08-27T11:17:27.104 INFO:teuthology.orchestra.run.smithi138.stderr:69 bytes copied, 2.2825e-05 s, 3.0 MB/s 2023-08-27T11:17:27.104 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-08-27T11:17:27.105 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-27T11:17:27.105 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-08-27T11:17:27.105 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:17:27.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.239+0000 7f33e4083700 1 Processor -- start 2023-08-27T11:17:27.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.239+0000 7f33e4083700 1 -- start start 2023-08-27T11:17:27.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.240+0000 7f33e4083700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33dc06adb0 0x7f33dc06b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:27.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.240+0000 7f33e4083700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f33dc0cdbd0 0x7f33dc0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:27.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.240+0000 7f33e4083700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33dc15eb60 con 0x7f33dc151060 2023-08-27T11:17:27.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.240+0000 7f33e4083700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33dc06b770 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.240+0000 7f33e4083700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0b02f0 con 0x7f33dc06adb0 2023-08-27T11:17:27.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.240+0000 7f33e161e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33dc06adb0 0x7f33dc06b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:27.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.240+0000 7f33e161e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33dc06adb0 0x7f33dc06b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:38936/0 (socket says 172.21.15.112:38936) 2023-08-27T11:17:27.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.240+0000 7f33e2620700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f33dc0cdbd0 0x7f33dc0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:27.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.240+0000 7f33e161e700 1 -- 172.21.15.112:0/2946789396 learned_addr learned my addr 172.21.15.112:0/2946789396 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:27.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.241+0000 7f33e161e700 1 -- 172.21.15.112:0/2946789396 >> v2:172.21.15.138:3300/0 conn(0x7f33dc0cdbd0 msgr2=0x7f33dc0ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:27.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.241+0000 7f33e161e700 1 --2- 172.21.15.112:0/2946789396 >> v2:172.21.15.138:3300/0 conn(0x7f33dc0cdbd0 0x7f33dc0ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:27.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.241+0000 7f33e161e700 1 -- 172.21.15.112:0/2946789396 >> v1:172.21.15.138:6789/0 conn(0x7f33dc151060 legacy=0x7f33dc15e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:27.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.241+0000 7f33e161e700 1 -- 172.21.15.112:0/2946789396 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc0b05d0 con 0x7f33dc06adb0 2023-08-27T11:17:27.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.241+0000 7f33e2620700 1 --2- 172.21.15.112:0/2946789396 >> v2:172.21.15.138:3300/0 conn(0x7f33dc0cdbd0 0x7f33dc0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:27.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.241+0000 7f33e161e700 1 --2- 172.21.15.112:0/2946789396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33dc06adb0 0x7f33dc06b230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f33d800bb90 tx=0x7f33d800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec6c8865e16a1da server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:27.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.241+0000 7f33e0e1d700 1 -- 172.21.15.112:0/2946789396 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d800dd30 con 0x7f33dc06adb0 2023-08-27T11:17:27.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.241+0000 7f33e0e1d700 1 -- 172.21.15.112:0/2946789396 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d800de90 con 0x7f33dc06adb0 2023-08-27T11:17:27.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.241+0000 7f33e0e1d700 1 -- 172.21.15.112:0/2946789396 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8010690 con 0x7f33dc06adb0 2023-08-27T11:17:27.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.242+0000 7f33e4083700 1 -- 172.21.15.112:0/2946789396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33dc06adb0 msgr2=0x7f33dc06b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:27.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.242+0000 7f33e4083700 1 --2- 172.21.15.112:0/2946789396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33dc06adb0 0x7f33dc06b230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f33d800bb90 tx=0x7f33d800bea0 comp rx=0 tx=0).stop 2023-08-27T11:17:27.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.242+0000 7f33e4083700 1 -- 172.21.15.112:0/2946789396 shutdown_connections 2023-08-27T11:17:27.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.242+0000 7f33e4083700 1 --2- 172.21.15.112:0/2946789396 >> v2:172.21.15.138:3300/0 conn(0x7f33dc0cdbd0 0x7f33dc0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:27.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.242+0000 7f33e4083700 1 --2- 172.21.15.112:0/2946789396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33dc06adb0 0x7f33dc06b230 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:27.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.242+0000 7f33e4083700 1 -- 172.21.15.112:0/2946789396 >> 172.21.15.112:0/2946789396 conn(0x7f33dc0d0000 msgr2=0x7f33dc0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:27.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.242+0000 7f33e4083700 1 -- 172.21.15.112:0/2946789396 shutdown_connections 2023-08-27T11:17:27.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.242+0000 7f33e4083700 1 -- 172.21.15.112:0/2946789396 wait complete. 2023-08-27T11:17:27.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.243+0000 7f33e4083700 1 Processor -- start 2023-08-27T11:17:27.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.243+0000 7f33e4083700 1 -- start start 2023-08-27T11:17:27.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.244+0000 7f33e4083700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33dc06adb0 0x7f33dc0c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:27.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.244+0000 7f33e4083700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33dc0cdbd0 0x7f33dc1f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:27.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.244+0000 7f33e4083700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33dc151060 0x7f33dc1f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:27.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.244+0000 7f33e4083700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc14e970 con 0x7f33dc06adb0 2023-08-27T11:17:27.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.244+0000 7f33e4083700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0c4890 con 0x7f33dc151060 2023-08-27T11:17:27.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.244+0000 7f33e4083700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0c4b70 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.244+0000 7f33e1e1f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33dc06adb0 0x7f33dc0c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:27.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.244+0000 7f33e161e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33dc0cdbd0 0x7f33dc1f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:27.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.244+0000 7f33e1e1f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33dc06adb0 0x7f33dc0c6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55518/0 (socket says 172.21.15.112:55518) 2023-08-27T11:17:27.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.244+0000 7f33e1e1f700 1 -- 172.21.15.112:0/4258601961 learned_addr learned my addr 172.21.15.112:0/4258601961 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:27.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.244+0000 7f33e2620700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33dc151060 0x7f33dc1f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:27.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.245+0000 7f33e161e700 1 -- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33dc151060 msgr2=0x7f33dc1f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:27.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.245+0000 7f33e161e700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33dc151060 0x7f33dc1f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:27.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.245+0000 7f33e161e700 1 -- 172.21.15.112:0/4258601961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33dc06adb0 msgr2=0x7f33dc0c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:27.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.245+0000 7f33e161e700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33dc06adb0 0x7f33dc0c6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:27.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.245+0000 7f33e161e700 1 -- 172.21.15.112:0/4258601961 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d4009a70 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.245+0000 7f33e1e1f700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33dc06adb0 0x7f33dc0c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:27.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.245+0000 7f33e161e700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33dc0cdbd0 0x7f33dc1f1d50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f33d8018000 tx=0x7f33d8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:27.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.245+0000 7f33d27fc700 1 -- 172.21.15.112:0/4258601961 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d801f070 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.245+0000 7f33d27fc700 1 -- 172.21.15.112:0/4258601961 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d80038b0 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.245+0000 7f33e4083700 1 -- 172.21.15.112:0/4258601961 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d800b7f0 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.245+0000 7f33d27fc700 1 -- 172.21.15.112:0/4258601961 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8003c40 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.245+0000 7f33e4083700 1 -- 172.21.15.112:0/4258601961 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc0c5130 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.246+0000 7f33e4083700 1 -- 172.21.15.112:0/4258601961 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33dc067a20 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.246+0000 7f33d27fc700 1 -- 172.21.15.112:0/4258601961 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f33d802d440 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.247+0000 7f33d27fc700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f33c006a0e0 0x7f33c006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:27.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.247+0000 7f33d27fc700 1 -- 172.21.15.112:0/4258601961 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f33d808bdc0 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.247+0000 7f33e1e1f700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f33c006a0e0 0x7f33c006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:27.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.247+0000 7f33e1e1f700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f33c006a0e0 0x7f33c006c5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f33cc001c70 tx=0x7f33cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:27.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.252+0000 7f33d27fc700 1 -- 172.21.15.112:0/4258601961 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f33d80574a0 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.482+0000 7f33e4083700 1 -- 172.21.15.112:0/4258601961 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f33dc0c6940 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.483+0000 7f33d27fc700 1 -- 172.21.15.112:0/4258601961 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f33d805aac0 con 0x7f33dc0cdbd0 2023-08-27T11:17:27.484 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:17:27.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.484+0000 7f33e4083700 1 -- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f33c006a0e0 msgr2=0x7f33c006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:27.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.484+0000 7f33e4083700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f33c006a0e0 0x7f33c006c5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f33cc001c70 tx=0x7f33cc009040 comp rx=0 tx=0).stop 2023-08-27T11:17:27.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.484+0000 7f33e4083700 1 -- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33dc0cdbd0 msgr2=0x7f33dc1f1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:27.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.484+0000 7f33e4083700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33dc0cdbd0 0x7f33dc1f1d50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f33d8018000 tx=0x7f33d8004750 comp rx=0 tx=0).stop 2023-08-27T11:17:27.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.485+0000 7f33e4083700 1 -- 172.21.15.112:0/4258601961 shutdown_connections 2023-08-27T11:17:27.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.485+0000 7f33e4083700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f33c006a0e0 0x7f33c006c5a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:27.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.485+0000 7f33e4083700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33dc151060 0x7f33dc1f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:27.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.485+0000 7f33e4083700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33dc06adb0 0x7f33dc0c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:27.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.485+0000 7f33e4083700 1 --2- 172.21.15.112:0/4258601961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33dc0cdbd0 0x7f33dc1f1d50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:27.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.485+0000 7f33e4083700 1 -- 172.21.15.112:0/4258601961 >> 172.21.15.112:0/4258601961 conn(0x7f33dc0d0000 msgr2=0x7f33dc150390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:27.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.485+0000 7f33e4083700 1 -- 172.21.15.112:0/4258601961 shutdown_connections 2023-08-27T11:17:27.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:27.485+0000 7f33e4083700 1 -- 172.21.15.112:0/4258601961 wait complete. 2023-08-27T11:17:27.498 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-27T11:17:27.498 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:17:27.499 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2023-08-27T11:17:27.499 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-08-27T11:17:27.499 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-08-27T11:17:27.499 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-27T11:17:27.500 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-27T11:17:27.500 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:17:27.500 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-08-27T11:17:27.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:27.501 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2023-08-27T11:17:27.518 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:17:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:17:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:17:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:17:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:27.520 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:17:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:27.520 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:17:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:27.520 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:17:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:27.521 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:17:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:17:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42411sec preferred_lft 42411sec 2023-08-27T11:17:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ac17:49c5:cb39:801a/64 scope link noprefixroute 2023-08-27T11:17:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:27.522 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:17:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:17:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:17:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c4b4:94ff:fe58:4e35/64 scope link 2023-08-27T11:17:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:27.523 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:17:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:17:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::5806:6aff:fe04:ef32/64 scope link tentative 2023-08-27T11:17:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:27.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:27.524 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:17:27.584 INFO:teuthology.orchestra.run.smithi138.stdout:1 2023-08-27T11:17:27.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:27.585 DEBUG:teuthology.orchestra.run.smithi138:> route 2023-08-27T11:17:27.644 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2023-08-27T11:17:27.645 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:17:27.645 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:17:27.645 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:17:27.645 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:17:27.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:27.646 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:17:27.646 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:17:27.647 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:17:27.647 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:17:27.647 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:17:27.648 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:17:27.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:27.750 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:17:27.772 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-27T11:17:27.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:27.773 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2023-08-27T11:17:27.839 INFO:teuthology.orchestra.run.smithi138.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-08-27T11:17:27.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:27.840 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:17:27.840 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:17:27.840 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-27T11:17:27.840 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-08-27T11:17:27.841 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:17:27.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:27.939 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-08-27T11:17:27.982 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:17:27.982 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:17:27.983 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:17:27.983 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:27.983 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:17:27.983 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:27.983 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:17:27.984 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether d2:b8:6a:73:c5:54 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-27T11:17:27.984 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-27T11:17:27.984 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:27.984 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::d0b8:6aff:fe73:c554/64 scope link 2023-08-27T11:17:27.984 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:27.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:27.997 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-08-27T11:17:28.078 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:17:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:17:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:17:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:17:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:28.081 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:17:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether d2:b8:6a:73:c5:54 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-27T11:17:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-27T11:17:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:28.083 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::d0b8:6aff:fe73:c554/64 scope link 2023-08-27T11:17:28.083 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:17:28.097 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-08-27T11:17:28.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:28.101 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:17:28.101 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:17:28.102 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-08-27T11:17:28.103 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-08-27T11:17:28.104 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-08-27T11:17:28.105 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-08-27T11:17:28.105 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-08-27T11:17:28.105 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:17:28.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:28.352 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:17:28.352 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:17:28.353 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.1 up 2023-08-27T11:17:28.353 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.1 master ceph-brx 2023-08-27T11:17:28.353 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:17:28.460 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:17:28.460 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-08-27T11:17:28.485 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-08-27T11:17:28.486 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:17:28.486 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-08-27T11:17:28.545 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:17:28.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:28.546 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-08-27T11:17:28.631 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-08-27T11:17:28.631 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-08-27T11:17:28.631 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:17:28.639 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-08-27T11:17:28.640 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-08-27T11:17:28.640 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:17:28.640 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:17:28.640 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:17:28.713 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-08-27T11:17:28.714 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-08-27T11:17:28.714 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:17:28.715 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:17:28.715 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:17:28.715 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:17:28.716 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:17:28.716 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:17:28.717 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:17:28.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:28.718 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-08-27T11:17:28.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:28.745 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:17:28.818 INFO:teuthology.orchestra.run.smithi138.stdout:client4692 2023-08-27T11:17:28.819 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:17:28.819 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:28.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:28.876 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:17:28.949 INFO:teuthology.orchestra.run.smithi138.stdout:client4692 2023-08-27T11:17:28.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:28.950 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:17:29.021 INFO:teuthology.orchestra.run.smithi138.stdout:client4692 2023-08-27T11:17:29.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:29.022 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e099a24f-c1cd-4f11-ac4f-ecacc10e5e98.client4692/status') 2023-08-27T11:17:29.102 INFO:teuthology.orchestra.run.smithi138.stdout:instance: client.4692 (3)172.21.15.138:0/2408191746 2023-08-27T11:17:29.102 INFO:teuthology.orchestra.run.smithi138.stdout:blocklisted: false 2023-08-27T11:17:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2023-08-27T11:17:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2023-08-27T11:17:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:71 bytes copied, 2.1502e-05 s, 3.3 MB/s 2023-08-27T11:17:29.104 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-08-27T11:17:29.127 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:29.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.261+0000 7f8fdd277700 1 Processor -- start 2023-08-27T11:17:29.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.261+0000 7f8fdd277700 1 -- start start 2023-08-27T11:17:29.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.261+0000 7f8fdd277700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8fd8157880 0x7f8fd8160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fdd277700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8fd80d0e30 0x7f8fd815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fdd277700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd815f110 con 0x7f8fd8161430 2023-08-27T11:17:29.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fdd277700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd81618c0 con 0x7f8fd8157880 2023-08-27T11:17:29.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fdd277700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd80d1280 con 0x7f8fd80d0e30 2023-08-27T11:17:29.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd6ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8fd8157880 0x7f8fd8160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd67fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8fd8161430 0x7f8fd80d0720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49910/0 (socket says 172.21.15.112:49910) 2023-08-27T11:17:29.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd6ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8fd8157880 0x7f8fd8160ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55528/0 (socket says 172.21.15.112:55528) 2023-08-27T11:17:29.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd67fc700 1 -- 172.21.15.112:0/2386055202 learned_addr learned my addr 172.21.15.112:0/2386055202 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:29.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd77fe700 1 --2- 172.21.15.112:0/2386055202 >> v2:172.21.15.138:3300/0 conn(0x7f8fd80d0e30 0x7f8fd815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd6ffd700 1 -- 172.21.15.112:0/2386055202 >> v2:172.21.15.138:3300/0 conn(0x7f8fd80d0e30 msgr2=0x7f8fd815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:29.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd6ffd700 1 --2- 172.21.15.112:0/2386055202 >> v2:172.21.15.138:3300/0 conn(0x7f8fd80d0e30 0x7f8fd815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd6ffd700 1 -- 172.21.15.112:0/2386055202 >> v1:172.21.15.112:6789/0 conn(0x7f8fd8161430 legacy=0x7f8fd80d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd6ffd700 1 -- 172.21.15.112:0/2386055202 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd80d1490 con 0x7f8fd8157880 2023-08-27T11:17:29.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd5ffb700 1 -- 172.21.15.112:0/2386055202 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133512499 0 0) 0x7f8fd815f110 con 0x7f8fd8161430 2023-08-27T11:17:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd6ffd700 1 --2- 172.21.15.112:0/2386055202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd8157880 0x7f8fd8160ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009840 tx=0x7f8fc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa53826617e1f80a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd5ffb700 1 -- 172.21.15.112:0/2386055202 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8011830 con 0x7f8fd8157880 2023-08-27T11:17:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd5ffb700 1 -- 172.21.15.112:0/2386055202 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc8011990 con 0x7f8fd8157880 2023-08-27T11:17:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.262+0000 7f8fd5ffb700 1 -- 172.21.15.112:0/2386055202 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8011d20 con 0x7f8fd8157880 2023-08-27T11:17:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.263+0000 7f8fdd277700 1 -- 172.21.15.112:0/2386055202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd8157880 msgr2=0x7f8fd8160ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.263+0000 7f8fdd277700 1 --2- 172.21.15.112:0/2386055202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd8157880 0x7f8fd8160ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009840 tx=0x7f8fc8009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.263+0000 7f8fdd277700 1 -- 172.21.15.112:0/2386055202 shutdown_connections 2023-08-27T11:17:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.263+0000 7f8fdd277700 1 --2- 172.21.15.112:0/2386055202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd8157880 0x7f8fd8160ef0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.263+0000 7f8fdd277700 1 --2- 172.21.15.112:0/2386055202 >> v2:172.21.15.138:3300/0 conn(0x7f8fd80d0e30 0x7f8fd815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.263+0000 7f8fdd277700 1 -- 172.21.15.112:0/2386055202 >> 172.21.15.112:0/2386055202 conn(0x7f8fd806c880 msgr2=0x7f8fd80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:29.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.263+0000 7f8fdd277700 1 -- 172.21.15.112:0/2386055202 shutdown_connections 2023-08-27T11:17:29.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.263+0000 7f8fdd277700 1 -- 172.21.15.112:0/2386055202 wait complete. 2023-08-27T11:17:29.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.264+0000 7f8fdd277700 1 Processor -- start 2023-08-27T11:17:29.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.264+0000 7f8fdd277700 1 -- start start 2023-08-27T11:17:29.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fdd277700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80d0e30 0x7f8fd814ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fdd277700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd8157880 0x7f8fd815ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fdd277700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fd8161430 0x7f8fd80b8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fdd277700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd81f5eb0 con 0x7f8fd8157880 2023-08-27T11:17:29.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fdd277700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd814f110 con 0x7f8fd80d0e30 2023-08-27T11:17:29.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fdd277700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd81f6100 con 0x7f8fd8161430 2023-08-27T11:17:29.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fd67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd8157880 0x7f8fd815ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fd67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd8157880 0x7f8fd815ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55536/0 (socket says 172.21.15.112:55536) 2023-08-27T11:17:29.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fd67fc700 1 -- 172.21.15.112:0/184408644 learned_addr learned my addr 172.21.15.112:0/184408644 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:29.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fd77fe700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fd8161430 0x7f8fd80b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fd6ffd700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80d0e30 0x7f8fd814ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fd67fc700 1 -- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fd8161430 msgr2=0x7f8fd80b8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fd67fc700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fd8161430 0x7f8fd80b8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fd67fc700 1 -- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80d0e30 msgr2=0x7f8fd814ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fd67fc700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80d0e30 0x7f8fd814ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.265+0000 7f8fd67fc700 1 -- 172.21.15.112:0/184408644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc80094a0 con 0x7f8fd8157880 2023-08-27T11:17:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.266+0000 7f8fd67fc700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd8157880 0x7f8fd815ca50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00c550 tx=0x7f8fcc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.266+0000 7f8fbf7fe700 1 -- 172.21.15.112:0/184408644 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc004c80 con 0x7f8fd8157880 2023-08-27T11:17:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.266+0000 7f8fbf7fe700 1 -- 172.21.15.112:0/184408644 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fcc004e00 con 0x7f8fd8157880 2023-08-27T11:17:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.266+0000 7f8fdd277700 1 -- 172.21.15.112:0/184408644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd81f63e0 con 0x7f8fd8157880 2023-08-27T11:17:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.266+0000 7f8fbf7fe700 1 -- 172.21.15.112:0/184408644 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc007310 con 0x7f8fd8157880 2023-08-27T11:17:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.266+0000 7f8fdd277700 1 -- 172.21.15.112:0/184408644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd81f69a0 con 0x7f8fd8157880 2023-08-27T11:17:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.267+0000 7f8fdd277700 1 -- 172.21.15.112:0/184408644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd8067a20 con 0x7f8fd8157880 2023-08-27T11:17:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.267+0000 7f8fbf7fe700 1 -- 172.21.15.112:0/184408644 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8fcc007470 con 0x7f8fd8157880 2023-08-27T11:17:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.268+0000 7f8fbf7fe700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8fc406a0e0 0x7f8fc406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.268+0000 7f8fbf7fe700 1 -- 172.21.15.112:0/184408644 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8fcc088180 con 0x7f8fd8157880 2023-08-27T11:17:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.268+0000 7f8fd6ffd700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8fc406a0e0 0x7f8fc406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.268+0000 7f8fd6ffd700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8fc406a0e0 0x7f8fc406c5a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80090b0 tx=0x7f8fc8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.272+0000 7f8fbf7fe700 1 -- 172.21.15.112:0/184408644 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8fcc053870 con 0x7f8fd8157880 2023-08-27T11:17:29.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.466+0000 7f8fdd277700 1 -- 172.21.15.112:0/184408644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fd815d340 con 0x7f8fd8157880 2023-08-27T11:17:29.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.467+0000 7f8fbf7fe700 1 -- 172.21.15.112:0/184408644 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7f8fcc056e90 con 0x7f8fd8157880 2023-08-27T11:17:29.469 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:29.469 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:29.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.470+0000 7f8fdd277700 1 -- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8fc406a0e0 msgr2=0x7f8fc406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.470+0000 7f8fdd277700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8fc406a0e0 0x7f8fc406c5a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80090b0 tx=0x7f8fc8008030 comp rx=0 tx=0).stop 2023-08-27T11:17:29.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.470+0000 7f8fdd277700 1 -- 172.21.15.112:0/184408644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd8157880 msgr2=0x7f8fd815ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.470+0000 7f8fdd277700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd8157880 0x7f8fd815ca50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00c550 tx=0x7f8fcc00c860 comp rx=0 tx=0).stop 2023-08-27T11:17:29.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.470+0000 7f8fdd277700 1 -- 172.21.15.112:0/184408644 shutdown_connections 2023-08-27T11:17:29.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.470+0000 7f8fdd277700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8fc406a0e0 0x7f8fc406c5a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.470+0000 7f8fdd277700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fd8161430 0x7f8fd80b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.470+0000 7f8fdd277700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd80d0e30 0x7f8fd814ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.470+0000 7f8fdd277700 1 --2- 172.21.15.112:0/184408644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd8157880 0x7f8fd815ca50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.470+0000 7f8fdd277700 1 -- 172.21.15.112:0/184408644 >> 172.21.15.112:0/184408644 conn(0x7f8fd806c880 msgr2=0x7f8fd8152490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:29.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.470+0000 7f8fdd277700 1 -- 172.21.15.112:0/184408644 shutdown_connections 2023-08-27T11:17:29.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.470+0000 7f8fdd277700 1 -- 172.21.15.112:0/184408644 wait complete. 2023-08-27T11:17:29.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:17:02.484063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:17:03.722976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:17:29.484 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:17:29.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.605+0000 7f6de7e22700 1 Processor -- start 2023-08-27T11:17:29.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.605+0000 7f6de7e22700 1 -- start start 2023-08-27T11:17:29.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.606+0000 7f6de7e22700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6de01437c0 0x7f6de00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.606+0000 7f6de7e22700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6de006d5b0 0x7f6de006da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.606+0000 7f6de7e22700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de00cfd50 con 0x7f6de0143ba0 2023-08-27T11:17:29.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.606+0000 7f6de7e22700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de006df80 con 0x7f6de01437c0 2023-08-27T11:17:29.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.606+0000 7f6de7e22700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de00cf140 con 0x7f6de006d5b0 2023-08-27T11:17:29.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.606+0000 7f6de5bbe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6de01437c0 0x7f6de00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.606+0000 7f6de53bd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6de0143ba0 0x7f6de00ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49930/0 (socket says 172.21.15.112:49930) 2023-08-27T11:17:29.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.606+0000 7f6de53bd700 1 -- 172.21.15.112:0/1823098694 learned_addr learned my addr 172.21.15.112:0/1823098694 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:29.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de63bf700 1 --2- 172.21.15.112:0/1823098694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6de006d5b0 0x7f6de006da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de5bbe700 1 -- 172.21.15.112:0/1823098694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6de006d5b0 msgr2=0x7f6de006da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:29.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de5bbe700 1 --2- 172.21.15.112:0/1823098694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6de006d5b0 0x7f6de006da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de5bbe700 1 -- 172.21.15.112:0/1823098694 >> v1:172.21.15.112:6789/0 conn(0x7f6de0143ba0 legacy=0x7f6de00ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de5bbe700 1 -- 172.21.15.112:0/1823098694 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de00cf420 con 0x7f6de01437c0 2023-08-27T11:17:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de4bbc700 1 -- 172.21.15.112:0/1823098694 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663410239 0 0) 0x7f6de00cfd50 con 0x7f6de0143ba0 2023-08-27T11:17:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de63bf700 1 --2- 172.21.15.112:0/1823098694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6de006d5b0 0x7f6de006da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de5bbe700 1 --2- 172.21.15.112:0/1823098694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de01437c0 0x7f6de00b4670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009840 tx=0x7f6dd0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeaf8a3a527dab61 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de4bbc700 1 -- 172.21.15.112:0/1823098694 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0011830 con 0x7f6de01437c0 2023-08-27T11:17:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de4bbc700 1 -- 172.21.15.112:0/1823098694 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd0011990 con 0x7f6de01437c0 2023-08-27T11:17:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de4bbc700 1 -- 172.21.15.112:0/1823098694 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0011d20 con 0x7f6de01437c0 2023-08-27T11:17:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de7e22700 1 -- 172.21.15.112:0/1823098694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de01437c0 msgr2=0x7f6de00b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de7e22700 1 --2- 172.21.15.112:0/1823098694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de01437c0 0x7f6de00b4670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009840 tx=0x7f6dd0009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de7e22700 1 -- 172.21.15.112:0/1823098694 shutdown_connections 2023-08-27T11:17:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de7e22700 1 --2- 172.21.15.112:0/1823098694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6de006d5b0 0x7f6de006da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.607+0000 7f6de7e22700 1 --2- 172.21.15.112:0/1823098694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de01437c0 0x7f6de00b4670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.608+0000 7f6de7e22700 1 -- 172.21.15.112:0/1823098694 >> 172.21.15.112:0/1823098694 conn(0x7f6de00b1e90 msgr2=0x7f6de00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:29.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.608+0000 7f6de7e22700 1 -- 172.21.15.112:0/1823098694 shutdown_connections 2023-08-27T11:17:29.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.608+0000 7f6de7e22700 1 -- 172.21.15.112:0/1823098694 wait complete. 2023-08-27T11:17:29.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.609+0000 7f6de7e22700 1 Processor -- start 2023-08-27T11:17:29.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.609+0000 7f6de7e22700 1 -- start start 2023-08-27T11:17:29.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.609+0000 7f6de7e22700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6de006d5b0 0x7f6de0157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.609+0000 7f6de7e22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6de01437c0 0x7f6de015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.609+0000 7f6de7e22700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de0143ba0 0x7f6de015c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.609+0000 7f6de7e22700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de0134870 con 0x7f6de0143ba0 2023-08-27T11:17:29.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.609+0000 7f6de7e22700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de015c8f0 con 0x7f6de01437c0 2023-08-27T11:17:29.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.609+0000 7f6de7e22700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de015cbd0 con 0x7f6de006d5b0 2023-08-27T11:17:29.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6de63bf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de0143ba0 0x7f6de015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6de63bf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de0143ba0 0x7f6de015c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55556/0 (socket says 172.21.15.112:55556) 2023-08-27T11:17:29.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6de63bf700 1 -- 172.21.15.112:0/2829543715 learned_addr learned my addr 172.21.15.112:0/2829543715 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:29.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6de5bbe700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6de006d5b0 0x7f6de0157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6de53bd700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6de01437c0 0x7f6de015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6de63bf700 1 -- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6de006d5b0 msgr2=0x7f6de0157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6de63bf700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6de006d5b0 0x7f6de0157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6de63bf700 1 -- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6de01437c0 msgr2=0x7f6de015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6de63bf700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6de01437c0 0x7f6de015bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6de63bf700 1 -- 172.21.15.112:0/2829543715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd00094a0 con 0x7f6de0143ba0 2023-08-27T11:17:29.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6de63bf700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de0143ba0 0x7f6de015c250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8007ef0 tx=0x7f6dd800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:29.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6dd67fc700 1 -- 172.21.15.112:0/2829543715 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800dd80 con 0x7f6de0143ba0 2023-08-27T11:17:29.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6dd67fc700 1 -- 172.21.15.112:0/2829543715 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd800f040 con 0x7f6de0143ba0 2023-08-27T11:17:29.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6de7e22700 1 -- 172.21.15.112:0/2829543715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de015ceb0 con 0x7f6de0143ba0 2023-08-27T11:17:29.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6dd67fc700 1 -- 172.21.15.112:0/2829543715 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8003c60 con 0x7f6de0143ba0 2023-08-27T11:17:29.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.610+0000 7f6de7e22700 1 -- 172.21.15.112:0/2829543715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de015d470 con 0x7f6de0143ba0 2023-08-27T11:17:29.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.611+0000 7f6de7e22700 1 -- 172.21.15.112:0/2829543715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de0067e90 con 0x7f6de0143ba0 2023-08-27T11:17:29.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.612+0000 7f6dd67fc700 1 -- 172.21.15.112:0/2829543715 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6dd8004030 con 0x7f6de0143ba0 2023-08-27T11:17:29.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.612+0000 7f6dd67fc700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6dc406a0e0 0x7f6dc406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.612+0000 7f6dd67fc700 1 -- 172.21.15.112:0/2829543715 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6dd8088570 con 0x7f6de0143ba0 2023-08-27T11:17:29.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.612+0000 7f6de5bbe700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6dc406a0e0 0x7f6dc406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.612+0000 7f6de5bbe700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6dc406a0e0 0x7f6dc406c5a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00090b0 tx=0x7f6dd0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:29.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.617+0000 7f6dd67fc700 1 -- 172.21.15.112:0/2829543715 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6dd8053cf0 con 0x7f6de0143ba0 2023-08-27T11:17:29.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.803+0000 7f6de7e22700 1 -- 172.21.15.112:0/2829543715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6de0067a20 con 0x7f6de0143ba0 2023-08-27T11:17:29.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.804+0000 7f6dd67fc700 1 -- 172.21.15.112:0/2829543715 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6dd8057310 con 0x7f6de0143ba0 2023-08-27T11:17:29.807 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:29.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.805+0000 7f6de7e22700 1 -- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6dc406a0e0 msgr2=0x7f6dc406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.805+0000 7f6de7e22700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6dc406a0e0 0x7f6dc406c5a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00090b0 tx=0x7f6dd0008030 comp rx=0 tx=0).stop 2023-08-27T11:17:29.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.805+0000 7f6de7e22700 1 -- 172.21.15.112:0/2829543715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de0143ba0 msgr2=0x7f6de015c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.805+0000 7f6de7e22700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de0143ba0 0x7f6de015c250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8007ef0 tx=0x7f6dd800d3b0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.805+0000 7f6de7e22700 1 -- 172.21.15.112:0/2829543715 shutdown_connections 2023-08-27T11:17:29.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.805+0000 7f6de7e22700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6dc406a0e0 0x7f6dc406c5a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.805+0000 7f6de7e22700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de0143ba0 0x7f6de015c250 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.805+0000 7f6de7e22700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6de006d5b0 0x7f6de0157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.805+0000 7f6de7e22700 1 --2- 172.21.15.112:0/2829543715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6de01437c0 0x7f6de015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.805+0000 7f6de7e22700 1 -- 172.21.15.112:0/2829543715 >> 172.21.15.112:0/2829543715 conn(0x7f6de00b1e90 msgr2=0x7f6de00b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:29.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.806+0000 7f6de7e22700 1 -- 172.21.15.112:0/2829543715 shutdown_connections 2023-08-27T11:17:29.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.806+0000 7f6de7e22700 1 -- 172.21.15.112:0/2829543715 wait complete. 2023-08-27T11:17:29.819 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:29.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.941+0000 7f9d24aea700 1 Processor -- start 2023-08-27T11:17:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.941+0000 7f9d24aea700 1 -- start start 2023-08-27T11:17:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d24aea700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9d20152130 0x7f9d2015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d24aea700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d2015cd60 0x7f9d2015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d24aea700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2015f750 con 0x7f9d2015f240 2023-08-27T11:17:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d24aea700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2015fa30 con 0x7f9d20152130 2023-08-27T11:17:29.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d24aea700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d200b60a0 con 0x7f9d2015cd60 2023-08-27T11:17:29.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d1e59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9d20152130 0x7f9d2015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d1e59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9d20152130 0x7f9d2015c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55560/0 (socket says 172.21.15.112:55560) 2023-08-27T11:17:29.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d1ed9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9d2015f240 0x7f9d200b3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49942/0 (socket says 172.21.15.112:49942) 2023-08-27T11:17:29.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d1e59c700 1 -- 172.21.15.112:0/1094279471 learned_addr learned my addr 172.21.15.112:0/1094279471 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:29.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d1dd9b700 1 --2- 172.21.15.112:0/1094279471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d2015cd60 0x7f9d2015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d1e59c700 1 -- 172.21.15.112:0/1094279471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d2015cd60 msgr2=0x7f9d2015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:29.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d1e59c700 1 --2- 172.21.15.112:0/1094279471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d2015cd60 0x7f9d2015ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d1e59c700 1 -- 172.21.15.112:0/1094279471 >> v1:172.21.15.112:6789/0 conn(0x7f9d2015f240 legacy=0x7f9d200b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d1d59a700 1 -- 172.21.15.112:0/1094279471 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818144380 0 0) 0x7f9d2015f750 con 0x7f9d2015f240 2023-08-27T11:17:29.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.942+0000 7f9d1e59c700 1 -- 172.21.15.112:0/1094279471 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d200b6380 con 0x7f9d20152130 2023-08-27T11:17:29.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.943+0000 7f9d1e59c700 1 --2- 172.21.15.112:0/1094279471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d20152130 0x7f9d2015c820 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9d100098f0 tx=0x7f9d10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416dcb9c98ef3a73 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:29.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.943+0000 7f9d1d59a700 1 -- 172.21.15.112:0/1094279471 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10011830 con 0x7f9d20152130 2023-08-27T11:17:29.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.943+0000 7f9d1d59a700 1 -- 172.21.15.112:0/1094279471 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d10011990 con 0x7f9d20152130 2023-08-27T11:17:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.943+0000 7f9d1d59a700 1 -- 172.21.15.112:0/1094279471 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10011d20 con 0x7f9d20152130 2023-08-27T11:17:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.943+0000 7f9d24aea700 1 -- 172.21.15.112:0/1094279471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d20152130 msgr2=0x7f9d2015c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.943+0000 7f9d24aea700 1 --2- 172.21.15.112:0/1094279471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d20152130 0x7f9d2015c820 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9d100098f0 tx=0x7f9d10009c00 comp rx=0 tx=0).stop 2023-08-27T11:17:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.943+0000 7f9d24aea700 1 -- 172.21.15.112:0/1094279471 shutdown_connections 2023-08-27T11:17:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.943+0000 7f9d24aea700 1 --2- 172.21.15.112:0/1094279471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d20152130 0x7f9d2015c820 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.943+0000 7f9d24aea700 1 --2- 172.21.15.112:0/1094279471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d2015cd60 0x7f9d2015ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.943+0000 7f9d24aea700 1 -- 172.21.15.112:0/1094279471 >> 172.21.15.112:0/1094279471 conn(0x7f9d20066010 msgr2=0x7f9d200663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.943+0000 7f9d24aea700 1 -- 172.21.15.112:0/1094279471 shutdown_connections 2023-08-27T11:17:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.943+0000 7f9d24aea700 1 -- 172.21.15.112:0/1094279471 wait complete. 2023-08-27T11:17:29.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.945+0000 7f9d24aea700 1 Processor -- start 2023-08-27T11:17:29.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.945+0000 7f9d24aea700 1 -- start start 2023-08-27T11:17:29.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.946+0000 7f9d24aea700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d20152130 0x7f9d20154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.946+0000 7f9d24aea700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d2015cd60 0x7f9d20151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.946+0000 7f9d24aea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2015f240 0x7f9d201f0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.946+0000 7f9d24aea700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2014b560 con 0x7f9d2015f240 2023-08-27T11:17:29.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.946+0000 7f9d24aea700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d20154b60 con 0x7f9d20152130 2023-08-27T11:17:29.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.946+0000 7f9d24aea700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d20154d70 con 0x7f9d2015cd60 2023-08-27T11:17:29.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.946+0000 7f9d1ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2015f240 0x7f9d201f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.946+0000 7f9d1ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2015f240 0x7f9d201f0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55568/0 (socket says 172.21.15.112:55568) 2023-08-27T11:17:29.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.946+0000 7f9d1ed9d700 1 -- 172.21.15.112:0/3651392859 learned_addr learned my addr 172.21.15.112:0/3651392859 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.946+0000 7f9d1e59c700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d20152130 0x7f9d20154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.946+0000 7f9d1dd9b700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d2015cd60 0x7f9d20151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.947+0000 7f9d1ed9d700 1 -- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d2015cd60 msgr2=0x7f9d20151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.947+0000 7f9d1ed9d700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d2015cd60 0x7f9d20151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.947+0000 7f9d1ed9d700 1 -- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d20152130 msgr2=0x7f9d20154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:29.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.947+0000 7f9d1ed9d700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d20152130 0x7f9d20154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:29.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.947+0000 7f9d1ed9d700 1 -- 172.21.15.112:0/3651392859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d10009550 con 0x7f9d2015f240 2023-08-27T11:17:29.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.947+0000 7f9d1e59c700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d20152130 0x7f9d20154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:29.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.947+0000 7f9d1ed9d700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2015f240 0x7f9d201f0030 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800cc10 tx=0x7f9d1800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:29.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.947+0000 7f9d0effd700 1 -- 172.21.15.112:0/3651392859 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1800aaa0 con 0x7f9d2015f240 2023-08-27T11:17:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.947+0000 7f9d0effd700 1 -- 172.21.15.112:0/3651392859 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1800ac20 con 0x7f9d2015f240 2023-08-27T11:17:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.947+0000 7f9d24aea700 1 -- 172.21.15.112:0/3651392859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d20153480 con 0x7f9d2015f240 2023-08-27T11:17:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.947+0000 7f9d0effd700 1 -- 172.21.15.112:0/3651392859 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d180070d0 con 0x7f9d2015f240 2023-08-27T11:17:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.947+0000 7f9d24aea700 1 -- 172.21.15.112:0/3651392859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d201531a0 con 0x7f9d2015f240 2023-08-27T11:17:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.948+0000 7f9d24aea700 1 -- 172.21.15.112:0/3651392859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d200b0920 con 0x7f9d2015f240 2023-08-27T11:17:29.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.948+0000 7f9d0effd700 1 -- 172.21.15.112:0/3651392859 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9d18007230 con 0x7f9d2015f240 2023-08-27T11:17:29.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.949+0000 7f9d0effd700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d0006a0e0 0x7f9d0006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:29.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.949+0000 7f9d0effd700 1 -- 172.21.15.112:0/3651392859 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9d18088210 con 0x7f9d2015f240 2023-08-27T11:17:29.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.949+0000 7f9d1e59c700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d0006a0e0 0x7f9d0006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:29.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.949+0000 7f9d1e59c700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d0006a0e0 0x7f9d0006c5a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009c40 tx=0x7f9d10008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:29.954+0000 7f9d0effd700 1 -- 172.21.15.112:0/3651392859 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9d18053900 con 0x7f9d2015f240 2023-08-27T11:17:30.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.142+0000 7f9d24aea700 1 -- 172.21.15.112:0/3651392859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d20067a20 con 0x7f9d2015f240 2023-08-27T11:17:30.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.143+0000 7f9d0effd700 1 -- 172.21.15.112:0/3651392859 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d18056f20 con 0x7f9d2015f240 2023-08-27T11:17:30.145 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:30.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.145+0000 7f9d24aea700 1 -- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d0006a0e0 msgr2=0x7f9d0006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.145+0000 7f9d24aea700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d0006a0e0 0x7f9d0006c5a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009c40 tx=0x7f9d10008030 comp rx=0 tx=0).stop 2023-08-27T11:17:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.145+0000 7f9d24aea700 1 -- 172.21.15.112:0/3651392859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2015f240 msgr2=0x7f9d201f0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.145+0000 7f9d24aea700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2015f240 0x7f9d201f0030 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800cc10 tx=0x7f9d1800cf20 comp rx=0 tx=0).stop 2023-08-27T11:17:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.145+0000 7f9d24aea700 1 -- 172.21.15.112:0/3651392859 shutdown_connections 2023-08-27T11:17:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.145+0000 7f9d24aea700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d0006a0e0 0x7f9d0006c5a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.145+0000 7f9d24aea700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2015f240 0x7f9d201f0030 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.145+0000 7f9d24aea700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d20152130 0x7f9d20154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.145+0000 7f9d24aea700 1 --2- 172.21.15.112:0/3651392859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d2015cd60 0x7f9d20151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.145+0000 7f9d24aea700 1 -- 172.21.15.112:0/3651392859 >> 172.21.15.112:0/3651392859 conn(0x7f9d20066010 msgr2=0x7f9d2015dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:30.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.145+0000 7f9d24aea700 1 -- 172.21.15.112:0/3651392859 shutdown_connections 2023-08-27T11:17:30.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.145+0000 7f9d24aea700 1 -- 172.21.15.112:0/3651392859 wait complete. 2023-08-27T11:17:30.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:17:30.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.281+0000 7f0a5a87c700 1 Processor -- start 2023-08-27T11:17:30.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.281+0000 7f0a5a87c700 1 -- start start 2023-08-27T11:17:30.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.281+0000 7f0a5a87c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0a540b3de0 0x7f0a5406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.281+0000 7f0a5a87c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a540cef20 con 0x7f0a5406c510 2023-08-27T11:17:30.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.281+0000 7f0a5a87c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a540b7310 con 0x7f0a54144b80 2023-08-27T11:17:30.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.281+0000 7f0a5a87c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a540b4b40 con 0x7f0a540b3de0 2023-08-27T11:17:30.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.281+0000 7f0a537fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0a540b3de0 0x7f0a5406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.281+0000 7f0a58e19700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0a5406c510 0x7f0a540cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49958/0 (socket says 172.21.15.112:49958) 2023-08-27T11:17:30.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.281+0000 7f0a58e19700 1 -- 172.21.15.112:0/4043811271 learned_addr learned my addr 172.21.15.112:0/4043811271 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:30.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.281+0000 7f0a537fe700 1 -- 172.21.15.112:0/4043811271 >> v1:172.21.15.138:6789/0 conn(0x7f0a54144b80 legacy=0x7f0a540b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.281+0000 7f0a537fe700 1 -- 172.21.15.112:0/4043811271 >> v1:172.21.15.112:6789/0 conn(0x7f0a5406c510 legacy=0x7f0a540cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.281+0000 7f0a537fe700 1 -- 172.21.15.112:0/4043811271 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5406c910 con 0x7f0a540b3de0 2023-08-27T11:17:30.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.281+0000 7f0a537fe700 1 --2- 172.21.15.112:0/4043811271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a540b3de0 0x7f0a5406bfd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009840 tx=0x7f0a44009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2985de5b817bc6d7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:30.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.283+0000 7f0a52ffd700 1 -- 172.21.15.112:0/4043811271 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44011830 con 0x7f0a540b3de0 2023-08-27T11:17:30.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.283+0000 7f0a52ffd700 1 -- 172.21.15.112:0/4043811271 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a44011990 con 0x7f0a540b3de0 2023-08-27T11:17:30.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.283+0000 7f0a52ffd700 1 -- 172.21.15.112:0/4043811271 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44011d20 con 0x7f0a540b3de0 2023-08-27T11:17:30.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.283+0000 7f0a5a87c700 1 -- 172.21.15.112:0/4043811271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a540b3de0 msgr2=0x7f0a5406bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.283+0000 7f0a5a87c700 1 --2- 172.21.15.112:0/4043811271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a540b3de0 0x7f0a5406bfd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009840 tx=0x7f0a44009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:30.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.283+0000 7f0a5a87c700 1 -- 172.21.15.112:0/4043811271 shutdown_connections 2023-08-27T11:17:30.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.283+0000 7f0a5a87c700 1 --2- 172.21.15.112:0/4043811271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a540b3de0 0x7f0a5406bfd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.283+0000 7f0a5a87c700 1 -- 172.21.15.112:0/4043811271 >> 172.21.15.112:0/4043811271 conn(0x7f0a540cdc50 msgr2=0x7f0a540ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:30.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.283+0000 7f0a5a87c700 1 -- 172.21.15.112:0/4043811271 shutdown_connections 2023-08-27T11:17:30.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.284+0000 7f0a5a87c700 1 -- 172.21.15.112:0/4043811271 wait complete. 2023-08-27T11:17:30.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.285+0000 7f0a5a87c700 1 Processor -- start 2023-08-27T11:17:30.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.285+0000 7f0a5a87c700 1 -- start start 2023-08-27T11:17:30.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.286+0000 7f0a5a87c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5406c510 0x7f0a54159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.286+0000 7f0a5a87c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a540b3de0 0x7f0a540cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.286+0000 7f0a5a87c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a54144b80 0x7f0a541f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.286+0000 7f0a5a87c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54158e50 con 0x7f0a540b3de0 2023-08-27T11:17:30.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.286+0000 7f0a5a87c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54158470 con 0x7f0a5406c510 2023-08-27T11:17:30.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.286+0000 7f0a5a87c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54136990 con 0x7f0a54144b80 2023-08-27T11:17:30.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.286+0000 7f0a537fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a540b3de0 0x7f0a540cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.286+0000 7f0a537fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a540b3de0 0x7f0a540cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55580/0 (socket says 172.21.15.112:55580) 2023-08-27T11:17:30.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.286+0000 7f0a537fe700 1 -- 172.21.15.112:0/113915122 learned_addr learned my addr 172.21.15.112:0/113915122 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:30.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.286+0000 7f0a537fe700 1 -- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a54144b80 msgr2=0x7f0a541f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.286+0000 7f0a537fe700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a54144b80 0x7f0a541f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.286+0000 7f0a537fe700 1 -- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5406c510 msgr2=0x7f0a54159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.286+0000 7f0a53fff700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5406c510 0x7f0a54159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.287+0000 7f0a537fe700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5406c510 0x7f0a54159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.287+0000 7f0a537fe700 1 -- 172.21.15.112:0/113915122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a440094a0 con 0x7f0a540b3de0 2023-08-27T11:17:30.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.287+0000 7f0a53fff700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5406c510 0x7f0a54159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:30.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.287+0000 7f0a537fe700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a540b3de0 0x7f0a540cf500 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0a440090b0 tx=0x7f0a44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:30.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.287+0000 7f0a50ff9700 1 -- 172.21.15.112:0/113915122 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a440038a0 con 0x7f0a540b3de0 2023-08-27T11:17:30.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.287+0000 7f0a50ff9700 1 -- 172.21.15.112:0/113915122 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4401f070 con 0x7f0a540b3de0 2023-08-27T11:17:30.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.287+0000 7f0a5a87c700 1 -- 172.21.15.112:0/113915122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a541f5050 con 0x7f0a540b3de0 2023-08-27T11:17:30.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.287+0000 7f0a50ff9700 1 -- 172.21.15.112:0/113915122 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44003c60 con 0x7f0a540b3de0 2023-08-27T11:17:30.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.287+0000 7f0a5a87c700 1 -- 172.21.15.112:0/113915122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a54158790 con 0x7f0a540b3de0 2023-08-27T11:17:30.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.288+0000 7f0a5a87c700 1 -- 172.21.15.112:0/113915122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a54158f90 con 0x7f0a540b3de0 2023-08-27T11:17:30.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.288+0000 7f0a50ff9700 1 -- 172.21.15.112:0/113915122 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0a4401a430 con 0x7f0a540b3de0 2023-08-27T11:17:30.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.288+0000 7f0a50ff9700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a4006a0e0 0x7f0a4006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.289+0000 7f0a50ff9700 1 -- 172.21.15.112:0/113915122 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0a44021070 con 0x7f0a540b3de0 2023-08-27T11:17:30.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.289+0000 7f0a53fff700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a4006a0e0 0x7f0a4006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.289+0000 7f0a53fff700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a4006a0e0 0x7f0a4006c5a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0a48000f60 tx=0x7f0a48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:30.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.294+0000 7f0a50ff9700 1 -- 172.21.15.112:0/113915122 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0a54158f90 con 0x7f0a540b3de0 2023-08-27T11:17:30.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.481+0000 7f0a5a87c700 1 -- 172.21.15.112:0/113915122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a54158f90 con 0x7f0a540b3de0 2023-08-27T11:17:30.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.482+0000 7f0a50ff9700 1 -- 172.21.15.112:0/113915122 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a44015020 con 0x7f0a540b3de0 2023-08-27T11:17:30.483 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:17:30.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.483+0000 7f0a5a87c700 1 -- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a4006a0e0 msgr2=0x7f0a4006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.483+0000 7f0a5a87c700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a4006a0e0 0x7f0a4006c5a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0a48000f60 tx=0x7f0a48009040 comp rx=0 tx=0).stop 2023-08-27T11:17:30.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.484+0000 7f0a5a87c700 1 -- 172.21.15.112:0/113915122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a540b3de0 msgr2=0x7f0a540cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.484+0000 7f0a5a87c700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a540b3de0 0x7f0a540cf500 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0a440090b0 tx=0x7f0a44004750 comp rx=0 tx=0).stop 2023-08-27T11:17:30.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.484+0000 7f0a5a87c700 1 -- 172.21.15.112:0/113915122 shutdown_connections 2023-08-27T11:17:30.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.484+0000 7f0a5a87c700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a4006a0e0 0x7f0a4006c5a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.484+0000 7f0a5a87c700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a54144b80 0x7f0a541f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.484+0000 7f0a5a87c700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5406c510 0x7f0a54159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.484+0000 7f0a5a87c700 1 --2- 172.21.15.112:0/113915122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a540b3de0 0x7f0a540cf500 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.484+0000 7f0a5a87c700 1 -- 172.21.15.112:0/113915122 >> 172.21.15.112:0/113915122 conn(0x7f0a540cdc50 msgr2=0x7f0a5415e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:30.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.484+0000 7f0a5a87c700 1 -- 172.21.15.112:0/113915122 shutdown_connections 2023-08-27T11:17:30.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.484+0000 7f0a5a87c700 1 -- 172.21.15.112:0/113915122 wait complete. 2023-08-27T11:17:30.497 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:17:30.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.620+0000 7fbfe2658700 1 Processor -- start 2023-08-27T11:17:30.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.620+0000 7fbfe2658700 1 -- start start 2023-08-27T11:17:30.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe2658700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfdc146bf0 0x7fbfdc0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe2658700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbfdc146fd0 0x7fbfdc0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe2658700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfdc0d0290 0x7fbfdc070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe2658700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfdc0d1500 con 0x7fbfdc0d0290 2023-08-27T11:17:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe2658700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfdc0d0710 con 0x7fbfdc146fd0 2023-08-27T11:17:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe2658700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc0d09f0 con 0x7fbfdc146bf0 2023-08-27T11:17:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe0bf5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfdc0d0290 0x7fbfdc070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe0bf5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfdc0d0290 0x7fbfdc070eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55590/0 (socket says 172.21.15.112:55590) 2023-08-27T11:17:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe0bf5700 1 -- 172.21.15.112:0/3944186390 learned_addr learned my addr 172.21.15.112:0/3944186390 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfdb7fe700 1 --2- 172.21.15.112:0/3944186390 >> v2:172.21.15.138:3300/0 conn(0x7fbfdc146fd0 0x7fbfdc0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfdbfff700 1 --2- 172.21.15.112:0/3944186390 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfdc146bf0 0x7fbfdc0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe0bf5700 1 -- 172.21.15.112:0/3944186390 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfdc146bf0 msgr2=0x7fbfdc0b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe0bf5700 1 --2- 172.21.15.112:0/3944186390 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfdc146bf0 0x7fbfdc0b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe0bf5700 1 -- 172.21.15.112:0/3944186390 >> v2:172.21.15.138:3300/0 conn(0x7fbfdc146fd0 msgr2=0x7fbfdc0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe0bf5700 1 --2- 172.21.15.112:0/3944186390 >> v2:172.21.15.138:3300/0 conn(0x7fbfdc146fd0 0x7fbfdc0cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.621+0000 7fbfe0bf5700 1 -- 172.21.15.112:0/3944186390 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc0d0cd0 con 0x7fbfdc0d0290 2023-08-27T11:17:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.622+0000 7fbfe0bf5700 1 --2- 172.21.15.112:0/3944186390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfdc0d0290 0x7fbfdc070eb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc06e920 tx=0x7fbfd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff61b31950e7a9f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.622+0000 7fbfdaffd700 1 -- 172.21.15.112:0/3944186390 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd400e070 con 0x7fbfdc0d0290 2023-08-27T11:17:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.622+0000 7fbfdaffd700 1 -- 172.21.15.112:0/3944186390 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd400b9c0 con 0x7fbfdc0d0290 2023-08-27T11:17:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.622+0000 7fbfdaffd700 1 -- 172.21.15.112:0/3944186390 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd400bd70 con 0x7fbfdc0d0290 2023-08-27T11:17:30.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.622+0000 7fbfe2658700 1 -- 172.21.15.112:0/3944186390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfdc0d0290 msgr2=0x7fbfdc070eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.622+0000 7fbfe2658700 1 --2- 172.21.15.112:0/3944186390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfdc0d0290 0x7fbfdc070eb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc06e920 tx=0x7fbfd400b330 comp rx=0 tx=0).stop 2023-08-27T11:17:30.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.622+0000 7fbfe2658700 1 -- 172.21.15.112:0/3944186390 shutdown_connections 2023-08-27T11:17:30.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.622+0000 7fbfe2658700 1 --2- 172.21.15.112:0/3944186390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfdc0d0290 0x7fbfdc070eb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.622+0000 7fbfe2658700 1 --2- 172.21.15.112:0/3944186390 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfdc146bf0 0x7fbfdc0b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.622+0000 7fbfe2658700 1 --2- 172.21.15.112:0/3944186390 >> v2:172.21.15.138:3300/0 conn(0x7fbfdc146fd0 0x7fbfdc0cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.622+0000 7fbfe2658700 1 -- 172.21.15.112:0/3944186390 >> 172.21.15.112:0/3944186390 conn(0x7fbfdc06c450 msgr2=0x7fbfdc06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.622+0000 7fbfe2658700 1 -- 172.21.15.112:0/3944186390 shutdown_connections 2023-08-27T11:17:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.622+0000 7fbfe2658700 1 -- 172.21.15.112:0/3944186390 wait complete. 2023-08-27T11:17:30.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.623+0000 7fbfe2658700 1 Processor -- start 2023-08-27T11:17:30.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.624+0000 7fbfe2658700 1 -- start start 2023-08-27T11:17:30.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.624+0000 7fbfe2658700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfdc0d0290 0x7fbfdc1f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.624+0000 7fbfe2658700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfdc146bf0 0x7fbfdc0b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.624+0000 7fbfe2658700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfdc146fd0 0x7fbfdc0b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.624+0000 7fbfe2658700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc143070 con 0x7fbfdc0d0290 2023-08-27T11:17:30.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.624+0000 7fbfe2658700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc13f420 con 0x7fbfdc146bf0 2023-08-27T11:17:30.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.624+0000 7fbfe2658700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc13f700 con 0x7fbfdc146fd0 2023-08-27T11:17:30.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.624+0000 7fbfdbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfdc0d0290 0x7fbfdc1f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfdbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfdc0d0290 0x7fbfdc1f58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55592/0 (socket says 172.21.15.112:55592) 2023-08-27T11:17:30.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfdbfff700 1 -- 172.21.15.112:0/1554730327 learned_addr learned my addr 172.21.15.112:0/1554730327 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:30.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfe0bf5700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfdc146fd0 0x7fbfdc0b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfdb7fe700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfdc146bf0 0x7fbfdc0b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfdbfff700 1 -- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfdc146fd0 msgr2=0x7fbfdc0b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfdbfff700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfdc146fd0 0x7fbfdc0b8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfdbfff700 1 -- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfdc146bf0 msgr2=0x7fbfdc0b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfdbfff700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfdc146bf0 0x7fbfdc0b7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfdbfff700 1 -- 172.21.15.112:0/1554730327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd4009b30 con 0x7fbfdc0d0290 2023-08-27T11:17:30.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfe0bf5700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfdc146fd0 0x7fbfdc0b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:30.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfdb7fe700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfdc146bf0 0x7fbfdc0b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:30.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfdbfff700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfdc0d0290 0x7fbfdc1f58f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e810 tx=0x7fbfd000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:30.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfd8ff9700 1 -- 172.21.15.112:0/1554730327 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0013610 con 0x7fbfdc0d0290 2023-08-27T11:17:30.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfd8ff9700 1 -- 172.21.15.112:0/1554730327 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd0013770 con 0x7fbfdc0d0290 2023-08-27T11:17:30.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfe2658700 1 -- 172.21.15.112:0/1554730327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfdc13f9e0 con 0x7fbfdc0d0290 2023-08-27T11:17:30.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfd8ff9700 1 -- 172.21.15.112:0/1554730327 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0013b00 con 0x7fbfdc0d0290 2023-08-27T11:17:30.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.625+0000 7fbfe2658700 1 -- 172.21.15.112:0/1554730327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfdc1f5e30 con 0x7fbfdc0d0290 2023-08-27T11:17:30.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.626+0000 7fbfe2658700 1 -- 172.21.15.112:0/1554730327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfdc067e90 con 0x7fbfdc0d0290 2023-08-27T11:17:30.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.627+0000 7fbfd8ff9700 1 -- 172.21.15.112:0/1554730327 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbfd0013c60 con 0x7fbfdc0d0290 2023-08-27T11:17:30.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.627+0000 7fbfd8ff9700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfc806a0e0 0x7fbfc806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.627+0000 7fbfd8ff9700 1 -- 172.21.15.112:0/1554730327 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbfd0088240 con 0x7fbfdc0d0290 2023-08-27T11:17:30.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.627+0000 7fbfdb7fe700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfc806a0e0 0x7fbfc806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.628+0000 7fbfdb7fe700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfc806a0e0 0x7fbfc806c5a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc13ffc0 tx=0x7fbfcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:30.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.631+0000 7fbfd8ff9700 1 -- 172.21.15.112:0/1554730327 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbfd0053930 con 0x7fbfdc0d0290 2023-08-27T11:17:30.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.818+0000 7fbfe2658700 1 -- 172.21.15.112:0/1554730327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbfdc067a20 con 0x7fbfdc0d0290 2023-08-27T11:17:30.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.819+0000 7fbfd8ff9700 1 -- 172.21.15.112:0/1554730327 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbfd0056f50 con 0x7fbfdc0d0290 2023-08-27T11:17:30.820 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:17:30.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.821+0000 7fbfe2658700 1 -- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfc806a0e0 msgr2=0x7fbfc806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.821+0000 7fbfe2658700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfc806a0e0 0x7fbfc806c5a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc13ffc0 tx=0x7fbfcc008040 comp rx=0 tx=0).stop 2023-08-27T11:17:30.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.821+0000 7fbfe2658700 1 -- 172.21.15.112:0/1554730327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfdc0d0290 msgr2=0x7fbfdc1f58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.821+0000 7fbfe2658700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfdc0d0290 0x7fbfdc1f58f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e810 tx=0x7fbfd000ebd0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.821+0000 7fbfe2658700 1 -- 172.21.15.112:0/1554730327 shutdown_connections 2023-08-27T11:17:30.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.821+0000 7fbfe2658700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfc806a0e0 0x7fbfc806c5a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.821+0000 7fbfe2658700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfdc146fd0 0x7fbfdc0b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.821+0000 7fbfe2658700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfdc0d0290 0x7fbfdc1f58f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.821+0000 7fbfe2658700 1 --2- 172.21.15.112:0/1554730327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfdc146bf0 0x7fbfdc0b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.821+0000 7fbfe2658700 1 -- 172.21.15.112:0/1554730327 >> 172.21.15.112:0/1554730327 conn(0x7fbfdc06c450 msgr2=0x7fbfdc15fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:30.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.821+0000 7fbfe2658700 1 -- 172.21.15.112:0/1554730327 shutdown_connections 2023-08-27T11:17:30.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.821+0000 7fbfe2658700 1 -- 172.21.15.112:0/1554730327 wait complete. 2023-08-27T11:17:30.833 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:30.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.952+0000 7f75b693f700 1 Processor -- start 2023-08-27T11:17:30.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.953+0000 7f75b693f700 1 -- start start 2023-08-27T11:17:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.953+0000 7f75b693f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f75b00b05f0 0x7f75b00b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.953+0000 7f75b693f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b00b0a60 con 0x7f75b006dcc0 2023-08-27T11:17:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.953+0000 7f75b693f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b015b540 con 0x7f75b015a120 2023-08-27T11:17:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.953+0000 7f75b693f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b00af830 con 0x7f75b00b05f0 2023-08-27T11:17:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.953+0000 7f75af7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f75b006dcc0 0x7f75b00b3440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49982/0 (socket says 172.21.15.112:49982) 2023-08-27T11:17:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75af7fe700 1 -- 172.21.15.112:0/1395238806 learned_addr learned my addr 172.21.15.112:0/1395238806 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75b4edc700 1 --2- 172.21.15.112:0/1395238806 >> v2:172.21.15.138:3300/0 conn(0x7f75b00b05f0 0x7f75b00b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76209704 0 0) 0x7f75b00b0a60 con 0x7f75b006dcc0 2023-08-27T11:17:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75980035b0 con 0x7f75b006dcc0 2023-08-27T11:17:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1100011815 0 0) 0x7f75980035b0 con 0x7f75b006dcc0 2023-08-27T11:17:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 >> v2:172.21.15.138:3300/0 conn(0x7f75b00b05f0 msgr2=0x7f75b00b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 --2- 172.21.15.112:0/1395238806 >> v2:172.21.15.138:3300/0 conn(0x7f75b00b05f0 0x7f75b00b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 >> v1:172.21.15.138:6789/0 conn(0x7f75b015a120 legacy=0x7f75b006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b00afb10 con 0x7f75b006dcc0 2023-08-27T11:17:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f75a0001670 con 0x7f75b006dcc0 2023-08-27T11:17:30.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 >> v1:172.21.15.112:6789/0 conn(0x7f75b006dcc0 legacy=0x7f75b00b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 --2- 172.21.15.112:0/1395238806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7598003a00 0x7f7598003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 --2- 172.21.15.112:0/1395238806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7598004850 0x7f7598004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 --2- 172.21.15.112:0/1395238806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7598005210 0x7f759800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00af830 con 0x7f7598004850 2023-08-27T11:17:30.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759800c0b0 con 0x7f7598005210 2023-08-27T11:17:30.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759800c270 con 0x7f7598003a00 2023-08-27T11:17:30.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75af7fe700 1 --2- 172.21.15.112:0/1395238806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7598004850 0x7f7598004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75b4edc700 1 --2- 172.21.15.112:0/1395238806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7598005210 0x7f759800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.954+0000 7f75affff700 1 --2- 172.21.15.112:0/1395238806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7598003a00 0x7f7598003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75af7fe700 1 -- 172.21.15.112:0/1395238806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7598003a00 msgr2=0x7f7598003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:30.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75af7fe700 1 --2- 172.21.15.112:0/1395238806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7598003a00 0x7f7598003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75af7fe700 1 -- 172.21.15.112:0/1395238806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7598005210 msgr2=0x7f759800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:30.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75af7fe700 1 --2- 172.21.15.112:0/1395238806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7598005210 0x7f759800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75af7fe700 1 -- 172.21.15.112:0/1395238806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75980035b0 con 0x7f7598004850 2023-08-27T11:17:30.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75b4edc700 1 -- 172.21.15.112:0/1395238806 reap_dead start 2023-08-27T11:17:30.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75af7fe700 1 --2- 172.21.15.112:0/1395238806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7598004850 0x7f7598004cd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f75a000c240 tx=0x7f7598005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633a1ba26af190da server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:30.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0007c10 con 0x7f7598004850 2023-08-27T11:17:30.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a000cd80 con 0x7f7598004850 2023-08-27T11:17:30.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75aeffd700 1 -- 172.21.15.112:0/1395238806 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0005ba0 con 0x7f7598004850 2023-08-27T11:17:30.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75b693f700 1 -- 172.21.15.112:0/1395238806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7598004850 msgr2=0x7f7598004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75b693f700 1 --2- 172.21.15.112:0/1395238806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7598004850 0x7f7598004cd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f75a000c240 tx=0x7f7598005880 comp rx=0 tx=0).stop 2023-08-27T11:17:30.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75b693f700 1 -- 172.21.15.112:0/1395238806 shutdown_connections 2023-08-27T11:17:30.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75b693f700 1 --2- 172.21.15.112:0/1395238806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7598004850 0x7f7598004cd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75b693f700 1 -- 172.21.15.112:0/1395238806 >> 172.21.15.112:0/1395238806 conn(0x7f75b00cff80 msgr2=0x7f75b00d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:30.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75b693f700 1 -- 172.21.15.112:0/1395238806 shutdown_connections 2023-08-27T11:17:30.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.955+0000 7f75b693f700 1 -- 172.21.15.112:0/1395238806 wait complete. 2023-08-27T11:17:30.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.956+0000 7f75b693f700 1 Processor -- start 2023-08-27T11:17:30.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.957+0000 7f75b693f700 1 -- start start 2023-08-27T11:17:30.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.957+0000 7f75b693f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7598004850 0x7f75b0071b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.957+0000 7f75b693f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75b00c7a50 0x7f75b00c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.957+0000 7f75b693f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75b0157150 0x7f75b01575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.957+0000 7f75b693f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00c8410 con 0x7f75b00c7a50 2023-08-27T11:17:30.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.957+0000 7f75b693f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b0145010 con 0x7f7598004850 2023-08-27T11:17:30.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.957+0000 7f75b693f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b0157b10 con 0x7f75b0157150 2023-08-27T11:17:30.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75af7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75b00c7a50 0x7f75b00c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75af7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75b00c7a50 0x7f75b00c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55614/0 (socket says 172.21.15.112:55614) 2023-08-27T11:17:30.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75af7fe700 1 -- 172.21.15.112:0/2175425689 learned_addr learned my addr 172.21.15.112:0/2175425689 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:30.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75affff700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7598004850 0x7f75b0071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75b4edc700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75b0157150 0x7f75b01575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75af7fe700 1 -- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75b0157150 msgr2=0x7f75b01575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75af7fe700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75b0157150 0x7f75b01575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75af7fe700 1 -- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7598004850 msgr2=0x7f75b0071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75af7fe700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7598004850 0x7f75b0071b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75af7fe700 1 -- 172.21.15.112:0/2175425689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b0157dd0 con 0x7f75b00c7a50 2023-08-27T11:17:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75af7fe700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75b00c7a50 0x7f75b00c7ed0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f75a000c830 tx=0x7f75a0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75acff9700 1 -- 172.21.15.112:0/2175425689 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0020070 con 0x7f75b00c7a50 2023-08-27T11:17:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75acff9700 1 -- 172.21.15.112:0/2175425689 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a0001430 con 0x7f75b00c7a50 2023-08-27T11:17:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75b693f700 1 -- 172.21.15.112:0/2175425689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b01580b0 con 0x7f75b00c7a50 2023-08-27T11:17:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75acff9700 1 -- 172.21.15.112:0/2175425689 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a00070d0 con 0x7f75b00c7a50 2023-08-27T11:17:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.958+0000 7f75b693f700 1 -- 172.21.15.112:0/2175425689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b0145540 con 0x7f75b00c7a50 2023-08-27T11:17:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.959+0000 7f75b693f700 1 -- 172.21.15.112:0/2175425689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b01456d0 con 0x7f75b00c7a50 2023-08-27T11:17:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.960+0000 7f75acff9700 1 -- 172.21.15.112:0/2175425689 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f75a0007230 con 0x7f75b00c7a50 2023-08-27T11:17:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.960+0000 7f75acff9700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f759c06a0e0 0x7f759c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.960+0000 7f75acff9700 1 -- 172.21.15.112:0/2175425689 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f75a0088700 con 0x7f75b00c7a50 2023-08-27T11:17:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.960+0000 7f75affff700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f759c06a0e0 0x7f759c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.961+0000 7f75affff700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f759c06a0e0 0x7f759c06c5a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f75a4000c00 tx=0x7f75a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:30.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:30.964+0000 7f75acff9700 1 -- 172.21.15.112:0/2175425689 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f75a0053e80 con 0x7f75b00c7a50 2023-08-27T11:17:31.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.154+0000 7f75b693f700 1 -- 172.21.15.112:0/2175425689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75b00afdc0 con 0x7f75b00c7a50 2023-08-27T11:17:31.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.155+0000 7f75acff9700 1 -- 172.21.15.112:0/2175425689 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7f75a00574a0 con 0x7f75b00c7a50 2023-08-27T11:17:31.157 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:31.157 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:31.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.157+0000 7f75b693f700 1 -- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f759c06a0e0 msgr2=0x7f759c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.157+0000 7f75b693f700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f759c06a0e0 0x7f759c06c5a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f75a4000c00 tx=0x7f75a4009040 comp rx=0 tx=0).stop 2023-08-27T11:17:31.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.157+0000 7f75b693f700 1 -- 172.21.15.112:0/2175425689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75b00c7a50 msgr2=0x7f75b00c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.157+0000 7f75b693f700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75b00c7a50 0x7f75b00c7ed0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f75a000c830 tx=0x7f75a0001a00 comp rx=0 tx=0).stop 2023-08-27T11:17:31.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.157+0000 7f75b693f700 1 -- 172.21.15.112:0/2175425689 shutdown_connections 2023-08-27T11:17:31.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.157+0000 7f75b693f700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f759c06a0e0 0x7f759c06c5a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.157+0000 7f75b693f700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75b0157150 0x7f75b01575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.157+0000 7f75b693f700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7598004850 0x7f75b0071b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.157+0000 7f75b693f700 1 --2- 172.21.15.112:0/2175425689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75b00c7a50 0x7f75b00c7ed0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.157+0000 7f75b693f700 1 -- 172.21.15.112:0/2175425689 >> 172.21.15.112:0/2175425689 conn(0x7f75b00cff80 msgr2=0x7f75b00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.158+0000 7f75b693f700 1 -- 172.21.15.112:0/2175425689 shutdown_connections 2023-08-27T11:17:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.158+0000 7f75b693f700 1 -- 172.21.15.112:0/2175425689 wait complete. 2023-08-27T11:17:31.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:17:02.484063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:17:03.722976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:17:31.171 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:17:31.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.295+0000 7f0eb865a700 1 Processor -- start 2023-08-27T11:17:31.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.295+0000 7f0eb865a700 1 -- start start 2023-08-27T11:17:31.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb865a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0eb0146fd0 0x7f0eb00b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb865a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb00d1500 con 0x7f0eb0070eb0 2023-08-27T11:17:31.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb865a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb00d0460 con 0x7f0eb0146bf0 2023-08-27T11:17:31.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb865a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00d0710 con 0x7f0eb0146fd0 2023-08-27T11:17:31.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb6bf7700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0eb0070eb0 0x7f0eb00d1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50006/0 (socket says 172.21.15.112:50006) 2023-08-27T11:17:31.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb6bf7700 1 -- 172.21.15.112:0/568226586 learned_addr learned my addr 172.21.15.112:0/568226586 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:31.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb5bf5700 1 --2- 172.21.15.112:0/568226586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0eb0146fd0 0x7f0eb00b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050820086 0 0) 0x7f0eb00d1500 con 0x7f0eb0070eb0 2023-08-27T11:17:31.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e9c0035b0 con 0x7f0eb0070eb0 2023-08-27T11:17:31.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198964035 0 0) 0x7f0eb00d0460 con 0x7f0eb0146bf0 2023-08-27T11:17:31.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eb00d1500 con 0x7f0eb0146bf0 2023-08-27T11:17:31.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2043428842 0 0) 0x7f0e9c0035b0 con 0x7f0eb0070eb0 2023-08-27T11:17:31.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0eb0146fd0 msgr2=0x7f0eb00b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:31.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.296+0000 7f0eb53f4700 1 --2- 172.21.15.112:0/568226586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0eb0146fd0 0x7f0eb00b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 >> v1:172.21.15.138:6789/0 conn(0x7f0eb0146bf0 legacy=0x7f0eb00cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb00d09f0 con 0x7f0eb0070eb0 2023-08-27T11:17:31.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f0ea80019f0 con 0x7f0eb0070eb0 2023-08-27T11:17:31.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 >> v1:172.21.15.112:6789/0 conn(0x7f0eb0070eb0 legacy=0x7f0eb00d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb53f4700 1 --2- 172.21.15.112:0/568226586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0eb0146fd0 0x7f0e9c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb53f4700 1 --2- 172.21.15.112:0/568226586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e9c004170 0x7f0e9c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb53f4700 1 --2- 172.21.15.112:0/568226586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e9c004d90 0x7f0e9c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00d0710 con 0x7f0e9c004d90 2023-08-27T11:17:31.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c00bbd0 con 0x7f0e9c004170 2023-08-27T11:17:31.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c00bd90 con 0x7f0eb0146fd0 2023-08-27T11:17:31.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb6bf7700 1 --2- 172.21.15.112:0/568226586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e9c004d90 0x7f0e9c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb63f6700 1 --2- 172.21.15.112:0/568226586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0eb0146fd0 0x7f0e9c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb5bf5700 1 --2- 172.21.15.112:0/568226586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e9c004170 0x7f0e9c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb6bf7700 1 -- 172.21.15.112:0/568226586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0eb0146fd0 msgr2=0x7f0e9c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:31.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb6bf7700 1 --2- 172.21.15.112:0/568226586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0eb0146fd0 0x7f0e9c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb6bf7700 1 -- 172.21.15.112:0/568226586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e9c004170 msgr2=0x7f0e9c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:31.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb6bf7700 1 --2- 172.21.15.112:0/568226586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e9c004170 0x7f0e9c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb6bf7700 1 -- 172.21.15.112:0/568226586 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c0035b0 con 0x7f0e9c004d90 2023-08-27T11:17:31.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb6bf7700 1 --2- 172.21.15.112:0/568226586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e9c004d90 0x7f0e9c005210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800a700 tx=0x7f0e9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3fa36d5b0a0a9f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:31.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80072d0 con 0x7f0e9c004d90 2023-08-27T11:17:31.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea8007430 con 0x7f0e9c004d90 2023-08-27T11:17:31.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.297+0000 7f0eb53f4700 1 -- 172.21.15.112:0/568226586 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80077e0 con 0x7f0e9c004d90 2023-08-27T11:17:31.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.298+0000 7f0eb865a700 1 -- 172.21.15.112:0/568226586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e9c004d90 msgr2=0x7f0e9c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.298+0000 7f0eb865a700 1 --2- 172.21.15.112:0/568226586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e9c004d90 0x7f0e9c005210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800a700 tx=0x7f0e9c005880 comp rx=0 tx=0).stop 2023-08-27T11:17:31.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.298+0000 7f0eb6bf7700 1 -- 172.21.15.112:0/568226586 reap_dead start 2023-08-27T11:17:31.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.298+0000 7f0eb865a700 1 -- 172.21.15.112:0/568226586 shutdown_connections 2023-08-27T11:17:31.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.298+0000 7f0eb865a700 1 -- 172.21.15.112:0/568226586 >> 172.21.15.112:0/568226586 conn(0x7f0eb006c450 msgr2=0x7f0eb006f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:31.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.298+0000 7f0eb865a700 1 -- 172.21.15.112:0/568226586 shutdown_connections 2023-08-27T11:17:31.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.298+0000 7f0eb865a700 1 -- 172.21.15.112:0/568226586 wait complete. 2023-08-27T11:17:31.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.299+0000 7f0eb865a700 1 Processor -- start 2023-08-27T11:17:31.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.299+0000 7f0eb865a700 1 -- start start 2023-08-27T11:17:31.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb865a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e9c004d90 0x7f0eb0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb865a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb01421a0 0x7f0eb01400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb865a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eb0142620 0x7f0eb0140620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb865a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00b6ef0 con 0x7f0eb0142620 2023-08-27T11:17:31.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb865a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb015a420 con 0x7f0eb01421a0 2023-08-27T11:17:31.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb865a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb013eba0 con 0x7f0e9c004d90 2023-08-27T11:17:31.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb6bf7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eb0142620 0x7f0eb0140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb6bf7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eb0142620 0x7f0eb0140620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55636/0 (socket says 172.21.15.112:55636) 2023-08-27T11:17:31.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb6bf7700 1 -- 172.21.15.112:0/3948596298 learned_addr learned my addr 172.21.15.112:0/3948596298 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:31.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb63f6700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e9c004d90 0x7f0eb0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb5bf5700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb01421a0 0x7f0eb01400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb6bf7700 1 -- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e9c004d90 msgr2=0x7f0eb0159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb6bf7700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e9c004d90 0x7f0eb0159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb6bf7700 1 -- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb01421a0 msgr2=0x7f0eb01400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb6bf7700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb01421a0 0x7f0eb01400e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb6bf7700 1 -- 172.21.15.112:0/3948596298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb013ee80 con 0x7f0eb0142620 2023-08-27T11:17:31.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.300+0000 7f0eb6bf7700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eb0142620 0x7f0eb0140620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8011000 tx=0x7f0ea8001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:31.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.301+0000 7f0ea6ffd700 1 -- 172.21.15.112:0/3948596298 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80077b0 con 0x7f0eb0142620 2023-08-27T11:17:31.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.301+0000 7f0ea6ffd700 1 -- 172.21.15.112:0/3948596298 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea8007930 con 0x7f0eb0142620 2023-08-27T11:17:31.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.301+0000 7f0eb865a700 1 -- 172.21.15.112:0/3948596298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb013f160 con 0x7f0eb0142620 2023-08-27T11:17:31.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.301+0000 7f0ea6ffd700 1 -- 172.21.15.112:0/3948596298 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8012400 con 0x7f0eb0142620 2023-08-27T11:17:31.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.301+0000 7f0eb865a700 1 -- 172.21.15.112:0/3948596298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb013f720 con 0x7f0eb0142620 2023-08-27T11:17:31.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.302+0000 7f0eb865a700 1 -- 172.21.15.112:0/3948596298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb0067e90 con 0x7f0eb0142620 2023-08-27T11:17:31.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.302+0000 7f0ea6ffd700 1 -- 172.21.15.112:0/3948596298 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0ea801a070 con 0x7f0eb0142620 2023-08-27T11:17:31.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.302+0000 7f0ea6ffd700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e9406a0e0 0x7f0e9406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.302+0000 7f0ea6ffd700 1 -- 172.21.15.112:0/3948596298 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0ea8089f80 con 0x7f0eb0142620 2023-08-27T11:17:31.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.303+0000 7f0eb63f6700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e9406a0e0 0x7f0e9406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.303+0000 7f0eb63f6700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e9406a0e0 0x7f0e9406c5a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0007fd0 tx=0x7f0ea0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:31.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.307+0000 7f0ea6ffd700 1 -- 172.21.15.112:0/3948596298 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0ea8055830 con 0x7f0eb0142620 2023-08-27T11:17:31.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.499+0000 7f0eb865a700 1 -- 172.21.15.112:0/3948596298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0eb0067a20 con 0x7f0eb0142620 2023-08-27T11:17:31.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.499+0000 7f0ea6ffd700 1 -- 172.21.15.112:0/3948596298 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0ea8018020 con 0x7f0eb0142620 2023-08-27T11:17:31.500 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:31.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.501+0000 7f0eb865a700 1 -- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e9406a0e0 msgr2=0x7f0e9406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.501+0000 7f0eb865a700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e9406a0e0 0x7f0e9406c5a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0007fd0 tx=0x7f0ea0009040 comp rx=0 tx=0).stop 2023-08-27T11:17:31.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.501+0000 7f0eb865a700 1 -- 172.21.15.112:0/3948596298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eb0142620 msgr2=0x7f0eb0140620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.501+0000 7f0eb865a700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eb0142620 0x7f0eb0140620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8011000 tx=0x7f0ea8001cb0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.501+0000 7f0eb865a700 1 -- 172.21.15.112:0/3948596298 shutdown_connections 2023-08-27T11:17:31.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.501+0000 7f0eb865a700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e9406a0e0 0x7f0e9406c5a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.501+0000 7f0eb865a700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eb0142620 0x7f0eb0140620 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.501+0000 7f0eb865a700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e9c004d90 0x7f0eb0159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.501+0000 7f0eb865a700 1 --2- 172.21.15.112:0/3948596298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb01421a0 0x7f0eb01400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.501+0000 7f0eb865a700 1 -- 172.21.15.112:0/3948596298 >> 172.21.15.112:0/3948596298 conn(0x7f0eb006c450 msgr2=0x7f0eb0142fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.502+0000 7f0eb865a700 1 -- 172.21.15.112:0/3948596298 shutdown_connections 2023-08-27T11:17:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.502+0000 7f0eb865a700 1 -- 172.21.15.112:0/3948596298 wait complete. 2023-08-27T11:17:31.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:31.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.636+0000 7f8d65334700 1 Processor -- start 2023-08-27T11:17:31.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.636+0000 7f8d65334700 1 -- start start 2023-08-27T11:17:31.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d65334700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8d60146b60 0x7f8d6006ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d65334700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d6006e2f0 0x7f8d600acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d65334700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8d600ad000 0x7f8d60143f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d65334700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d60161b70 con 0x7f8d60146b60 2023-08-27T11:17:31.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d65334700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d60143ad0 con 0x7f8d600ad000 2023-08-27T11:17:31.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d65334700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60160fa0 con 0x7f8d6006e2f0 2023-08-27T11:17:31.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d5effd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8d60146b60 0x7f8d6006ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d5effd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8d60146b60 0x7f8d6006ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55640/0 (socket says 172.21.15.112:55640) 2023-08-27T11:17:31.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d5effd700 1 -- 172.21.15.112:0/3232131451 learned_addr learned my addr 172.21.15.112:0/3232131451 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:31.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d5e7fc700 1 --2- 172.21.15.112:0/3232131451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d6006e2f0 0x7f8d600acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d5f7fe700 1 --2- 172.21.15.112:0/3232131451 >> v2:172.21.15.138:3300/0 conn(0x7f8d600ad000 0x7f8d60143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d5effd700 1 -- 172.21.15.112:0/3232131451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d6006e2f0 msgr2=0x7f8d600acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:31.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d5effd700 1 --2- 172.21.15.112:0/3232131451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d6006e2f0 0x7f8d600acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d5effd700 1 -- 172.21.15.112:0/3232131451 >> v2:172.21.15.138:3300/0 conn(0x7f8d600ad000 msgr2=0x7f8d60143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:31.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d5effd700 1 --2- 172.21.15.112:0/3232131451 >> v2:172.21.15.138:3300/0 conn(0x7f8d600ad000 0x7f8d60143f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.637+0000 7f8d5effd700 1 -- 172.21.15.112:0/3232131451 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6006e750 con 0x7f8d60146b60 2023-08-27T11:17:31.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d5f7fe700 1 --2- 172.21.15.112:0/3232131451 >> v2:172.21.15.138:3300/0 conn(0x7f8d600ad000 0x7f8d60143f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:31.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d5effd700 1 --2- 172.21.15.112:0/3232131451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d60146b60 0x7f8d6006ddb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8d6015b5a0 tx=0x7f8d54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de17b74ad5a9d0fb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:31.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d5dffb700 1 -- 172.21.15.112:0/3232131451 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54011830 con 0x7f8d60146b60 2023-08-27T11:17:31.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d5dffb700 1 -- 172.21.15.112:0/3232131451 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d54011990 con 0x7f8d60146b60 2023-08-27T11:17:31.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d5dffb700 1 -- 172.21.15.112:0/3232131451 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54011d20 con 0x7f8d60146b60 2023-08-27T11:17:31.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d65334700 1 -- 172.21.15.112:0/3232131451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d60146b60 msgr2=0x7f8d6006ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d65334700 1 --2- 172.21.15.112:0/3232131451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d60146b60 0x7f8d6006ddb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8d6015b5a0 tx=0x7f8d54009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d65334700 1 -- 172.21.15.112:0/3232131451 shutdown_connections 2023-08-27T11:17:31.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d65334700 1 --2- 172.21.15.112:0/3232131451 >> v2:172.21.15.138:3300/0 conn(0x7f8d600ad000 0x7f8d60143f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d65334700 1 --2- 172.21.15.112:0/3232131451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d60146b60 0x7f8d6006ddb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d65334700 1 --2- 172.21.15.112:0/3232131451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d6006e2f0 0x7f8d600acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d65334700 1 -- 172.21.15.112:0/3232131451 >> 172.21.15.112:0/3232131451 conn(0x7f8d600cc240 msgr2=0x7f8d600cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:31.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d65334700 1 -- 172.21.15.112:0/3232131451 shutdown_connections 2023-08-27T11:17:31.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.638+0000 7f8d65334700 1 -- 172.21.15.112:0/3232131451 wait complete. 2023-08-27T11:17:31.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.640+0000 7f8d65334700 1 Processor -- start 2023-08-27T11:17:31.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.640+0000 7f8d65334700 1 -- start start 2023-08-27T11:17:31.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d65334700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d6006e2f0 0x7f8d601f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d65334700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d600ad000 0x7f8d601f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d65334700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d60146b60 0x7f8d601419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d65334700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d601f70b0 con 0x7f8d6006e2f0 2023-08-27T11:17:31.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d65334700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d601f2a90 con 0x7f8d60146b60 2023-08-27T11:17:31.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d65334700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6013b9d0 con 0x7f8d600ad000 2023-08-27T11:17:31.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d5effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d6006e2f0 0x7f8d601f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d5effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d6006e2f0 0x7f8d601f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55646/0 (socket says 172.21.15.112:55646) 2023-08-27T11:17:31.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d5effd700 1 -- 172.21.15.112:0/783214512 learned_addr learned my addr 172.21.15.112:0/783214512 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:31.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d5effd700 1 -- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d600ad000 msgr2=0x7f8d601f2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d5e7fc700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d600ad000 0x7f8d601f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d5effd700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d600ad000 0x7f8d601f2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d5effd700 1 -- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d60146b60 msgr2=0x7f8d601419f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d5f7fe700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d60146b60 0x7f8d601419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d5effd700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d60146b60 0x7f8d601419f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d5effd700 1 -- 172.21.15.112:0/783214512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d54009550 con 0x7f8d6006e2f0 2023-08-27T11:17:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d5effd700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d6006e2f0 0x7f8d601f6b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009c10 tx=0x7f8d54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d477fe700 1 -- 172.21.15.112:0/783214512 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d540036a0 con 0x7f8d6006e2f0 2023-08-27T11:17:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d477fe700 1 -- 172.21.15.112:0/783214512 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5401e070 con 0x7f8d6006e2f0 2023-08-27T11:17:31.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d65334700 1 -- 172.21.15.112:0/783214512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6013bcb0 con 0x7f8d6006e2f0 2023-08-27T11:17:31.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d477fe700 1 -- 172.21.15.112:0/783214512 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54003a60 con 0x7f8d6006e2f0 2023-08-27T11:17:31.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.641+0000 7f8d65334700 1 -- 172.21.15.112:0/783214512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6013c270 con 0x7f8d6006e2f0 2023-08-27T11:17:31.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.642+0000 7f8d65334700 1 -- 172.21.15.112:0/783214512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d60070820 con 0x7f8d6006e2f0 2023-08-27T11:17:31.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.643+0000 7f8d477fe700 1 -- 172.21.15.112:0/783214512 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8d54003bc0 con 0x7f8d6006e2f0 2023-08-27T11:17:31.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.643+0000 7f8d477fe700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8d4c06a0e0 0x7f8d4c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.643+0000 7f8d477fe700 1 -- 172.21.15.112:0/783214512 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8d540893b0 con 0x7f8d6006e2f0 2023-08-27T11:17:31.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.643+0000 7f8d5e7fc700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8d4c06a0e0 0x7f8d4c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.644+0000 7f8d5e7fc700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8d4c06a0e0 0x7f8d4c06c5a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8d6013d900 tx=0x7f8d50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:31.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.647+0000 7f8d477fe700 1 -- 172.21.15.112:0/783214512 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8d54054b30 con 0x7f8d6006e2f0 2023-08-27T11:17:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.836+0000 7f8d65334700 1 -- 172.21.15.112:0/783214512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d60161480 con 0x7f8d6006e2f0 2023-08-27T11:17:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.837+0000 7f8d477fe700 1 -- 172.21.15.112:0/783214512 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d54058150 con 0x7f8d6006e2f0 2023-08-27T11:17:31.837 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.838+0000 7f8d65334700 1 -- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8d4c06a0e0 msgr2=0x7f8d4c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.838+0000 7f8d65334700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8d4c06a0e0 0x7f8d4c06c5a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8d6013d900 tx=0x7f8d50008040 comp rx=0 tx=0).stop 2023-08-27T11:17:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.838+0000 7f8d65334700 1 -- 172.21.15.112:0/783214512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d6006e2f0 msgr2=0x7f8d601f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.839+0000 7f8d65334700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d6006e2f0 0x7f8d601f6b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009c10 tx=0x7f8d54004750 comp rx=0 tx=0).stop 2023-08-27T11:17:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.839+0000 7f8d65334700 1 -- 172.21.15.112:0/783214512 shutdown_connections 2023-08-27T11:17:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.839+0000 7f8d65334700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8d4c06a0e0 0x7f8d4c06c5a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.839+0000 7f8d65334700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d60146b60 0x7f8d601419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.839+0000 7f8d65334700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d6006e2f0 0x7f8d601f6b70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.839+0000 7f8d65334700 1 --2- 172.21.15.112:0/783214512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d600ad000 0x7f8d601f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.839+0000 7f8d65334700 1 -- 172.21.15.112:0/783214512 >> 172.21.15.112:0/783214512 conn(0x7f8d600cc240 msgr2=0x7f8d600ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.839+0000 7f8d65334700 1 -- 172.21.15.112:0/783214512 shutdown_connections 2023-08-27T11:17:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.839+0000 7f8d65334700 1 -- 172.21.15.112:0/783214512 wait complete. 2023-08-27T11:17:31.851 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:17:31.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.973+0000 7fdb5d075700 1 Processor -- start 2023-08-27T11:17:31.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.973+0000 7fdb5d075700 1 -- start start 2023-08-27T11:17:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.974+0000 7fdb5d075700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdb58146c20 0x7fdb58160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.974+0000 7fdb5d075700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb580b5070 con 0x7fdb580b3440 2023-08-27T11:17:31.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.974+0000 7fdb5d075700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb58147000 con 0x7fdb5806f790 2023-08-27T11:17:31.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.974+0000 7fdb5d075700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb581614a0 con 0x7fdb58146c20 2023-08-27T11:17:31.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.974+0000 7fdb5659c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fdb580b3440 0x7fdb5815e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50036/0 (socket says 172.21.15.112:50036) 2023-08-27T11:17:31.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.974+0000 7fdb5659c700 1 -- 172.21.15.112:0/764436831 learned_addr learned my addr 172.21.15.112:0/764436831 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:31.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb56d9d700 1 --2- 172.21.15.112:0/764436831 >> v2:172.21.15.138:3300/0 conn(0x7fdb58146c20 0x7fdb58160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132989355 0 0) 0x7fdb48001110 con 0x7fdb580b3440 2023-08-27T11:17:31.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb400035b0 con 0x7fdb580b3440 2023-08-27T11:17:31.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2705576451 0 0) 0x7fdb400035b0 con 0x7fdb580b3440 2023-08-27T11:17:31.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 >> v2:172.21.15.138:3300/0 conn(0x7fdb58146c20 msgr2=0x7fdb58160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:31.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 --2- 172.21.15.112:0/764436831 >> v2:172.21.15.138:3300/0 conn(0x7fdb58146c20 0x7fdb58160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 >> v1:172.21.15.138:6789/0 conn(0x7fdb5806f790 legacy=0x7fdb580b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb58161740 con 0x7fdb580b3440 2023-08-27T11:17:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fdb48001800 con 0x7fdb580b3440 2023-08-27T11:17:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 >> v1:172.21.15.112:6789/0 conn(0x7fdb580b3440 legacy=0x7fdb5815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 --2- 172.21.15.112:0/764436831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb40003a00 0x7fdb40003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 --2- 172.21.15.112:0/764436831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb40004850 0x7fdb40004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 --2- 172.21.15.112:0/764436831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb40005210 0x7fdb4000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb581614a0 con 0x7fdb40003a00 2023-08-27T11:17:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4000c1b0 con 0x7fdb40005210 2023-08-27T11:17:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4000c370 con 0x7fdb40004850 2023-08-27T11:17:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb56d9d700 1 --2- 172.21.15.112:0/764436831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb40003a00 0x7fdb40003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb5659c700 1 --2- 172.21.15.112:0/764436831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb40004850 0x7fdb40004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb5759e700 1 --2- 172.21.15.112:0/764436831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb40005210 0x7fdb4000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb56d9d700 1 -- 172.21.15.112:0/764436831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb40004850 msgr2=0x7fdb40004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb56d9d700 1 --2- 172.21.15.112:0/764436831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb40004850 0x7fdb40004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb56d9d700 1 -- 172.21.15.112:0/764436831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb40005210 msgr2=0x7fdb4000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb56d9d700 1 --2- 172.21.15.112:0/764436831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb40005210 0x7fdb4000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb56d9d700 1 -- 172.21.15.112:0/764436831 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb400035b0 con 0x7fdb40003a00 2023-08-27T11:17:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb5759e700 1 --2- 172.21.15.112:0/764436831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb40005210 0x7fdb4000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb5659c700 1 --2- 172.21.15.112:0/764436831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb40004850 0x7fdb40004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.975+0000 7fdb5759e700 1 -- 172.21.15.112:0/764436831 reap_dead start 2023-08-27T11:17:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.976+0000 7fdb56d9d700 1 --2- 172.21.15.112:0/764436831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb40003a00 0x7fdb40003e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009b60 tx=0x7fdb40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7df30d8fc86873c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.976+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00e6c0 con 0x7fdb40003a00 2023-08-27T11:17:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.976+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4c00e820 con 0x7fdb40003a00 2023-08-27T11:17:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.976+0000 7fdb55d9b700 1 -- 172.21.15.112:0/764436831 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00eb90 con 0x7fdb40003a00 2023-08-27T11:17:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.976+0000 7fdb5d075700 1 -- 172.21.15.112:0/764436831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb40003a00 msgr2=0x7fdb40003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.976+0000 7fdb5d075700 1 --2- 172.21.15.112:0/764436831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb40003a00 0x7fdb40003e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009b60 tx=0x7fdb40005880 comp rx=0 tx=0).stop 2023-08-27T11:17:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.976+0000 7fdb5d075700 1 -- 172.21.15.112:0/764436831 shutdown_connections 2023-08-27T11:17:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.976+0000 7fdb5d075700 1 --2- 172.21.15.112:0/764436831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb40003a00 0x7fdb40003e20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.976+0000 7fdb5d075700 1 -- 172.21.15.112:0/764436831 >> 172.21.15.112:0/764436831 conn(0x7fdb580b1e90 msgr2=0x7fdb580b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.976+0000 7fdb5d075700 1 -- 172.21.15.112:0/764436831 shutdown_connections 2023-08-27T11:17:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.976+0000 7fdb5d075700 1 -- 172.21.15.112:0/764436831 wait complete. 2023-08-27T11:17:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.978+0000 7fdb5d075700 1 Processor -- start 2023-08-27T11:17:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.978+0000 7fdb5d075700 1 -- start start 2023-08-27T11:17:31.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.978+0000 7fdb5d075700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb40003a00 0x7fdb580c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.978+0000 7fdb5d075700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb581408a0 0x7fdb581f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.978+0000 7fdb5d075700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb58140d20 0x7fdb581f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.978+0000 7fdb5d075700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5815ed90 con 0x7fdb581408a0 2023-08-27T11:17:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.978+0000 7fdb5d075700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5815f070 con 0x7fdb58140d20 2023-08-27T11:17:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.978+0000 7fdb5d075700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5815f350 con 0x7fdb40003a00 2023-08-27T11:17:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb5659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb581408a0 0x7fdb581f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb5659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb581408a0 0x7fdb581f6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55668/0 (socket says 172.21.15.112:55668) 2023-08-27T11:17:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb5659c700 1 -- 172.21.15.112:0/63740694 learned_addr learned my addr 172.21.15.112:0/63740694 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb56d9d700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb40003a00 0x7fdb580c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb5759e700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb58140d20 0x7fdb581f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb5659c700 1 -- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb40003a00 msgr2=0x7fdb580c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb5659c700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb40003a00 0x7fdb580c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb5659c700 1 -- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb58140d20 msgr2=0x7fdb581f6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:31.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb5659c700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb58140d20 0x7fdb581f6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:31.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb5659c700 1 -- 172.21.15.112:0/63740694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5815f630 con 0x7fdb581408a0 2023-08-27T11:17:31.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb56d9d700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb40003a00 0x7fdb580c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:31.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb5759e700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb58140d20 0x7fdb581f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:31.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb5659c700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb581408a0 0x7fdb581f6520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdb48000fb0 tx=0x7fdb48007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:31.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb3f7fe700 1 -- 172.21.15.112:0/63740694 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48010510 con 0x7fdb581408a0 2023-08-27T11:17:31.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb3f7fe700 1 -- 172.21.15.112:0/63740694 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb48010670 con 0x7fdb581408a0 2023-08-27T11:17:31.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb5d075700 1 -- 172.21.15.112:0/63740694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5815f910 con 0x7fdb581408a0 2023-08-27T11:17:31.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb3f7fe700 1 -- 172.21.15.112:0/63740694 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48010a00 con 0x7fdb581408a0 2023-08-27T11:17:31.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.979+0000 7fdb5d075700 1 -- 172.21.15.112:0/63740694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb58146390 con 0x7fdb581408a0 2023-08-27T11:17:31.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.980+0000 7fdb5d075700 1 -- 172.21.15.112:0/63740694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb581437b0 con 0x7fdb581408a0 2023-08-27T11:17:31.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.981+0000 7fdb3f7fe700 1 -- 172.21.15.112:0/63740694 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdb48010b60 con 0x7fdb581408a0 2023-08-27T11:17:31.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.981+0000 7fdb3f7fe700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdb4406a0e0 0x7fdb4406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:31.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.981+0000 7fdb3f7fe700 1 -- 172.21.15.112:0/63740694 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdb48087d40 con 0x7fdb581408a0 2023-08-27T11:17:31.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.981+0000 7fdb56d9d700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdb4406a0e0 0x7fdb4406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:31.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.982+0000 7fdb56d9d700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdb4406a0e0 0x7fdb4406c5a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0090b0 tx=0x7fdb4c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:31.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:31.985+0000 7fdb3f7fe700 1 -- 172.21.15.112:0/63740694 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdb480534e0 con 0x7fdb581408a0 2023-08-27T11:17:32.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.174+0000 7fdb5d075700 1 -- 172.21.15.112:0/63740694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb58067e90 con 0x7fdb581408a0 2023-08-27T11:17:32.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.174+0000 7fdb3f7fe700 1 -- 172.21.15.112:0/63740694 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb48019020 con 0x7fdb581408a0 2023-08-27T11:17:32.175 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:17:32.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.176+0000 7fdb5d075700 1 -- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdb4406a0e0 msgr2=0x7fdb4406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.176+0000 7fdb5d075700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdb4406a0e0 0x7fdb4406c5a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0090b0 tx=0x7fdb4c008030 comp rx=0 tx=0).stop 2023-08-27T11:17:32.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.176+0000 7fdb5d075700 1 -- 172.21.15.112:0/63740694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb581408a0 msgr2=0x7fdb581f6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.176+0000 7fdb5d075700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb581408a0 0x7fdb581f6520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdb48000fb0 tx=0x7fdb48007330 comp rx=0 tx=0).stop 2023-08-27T11:17:32.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.176+0000 7fdb5d075700 1 -- 172.21.15.112:0/63740694 shutdown_connections 2023-08-27T11:17:32.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.176+0000 7fdb5d075700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdb4406a0e0 0x7fdb4406c5a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.176+0000 7fdb5d075700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb58140d20 0x7fdb581f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.176+0000 7fdb5d075700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb40003a00 0x7fdb580c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.176+0000 7fdb5d075700 1 --2- 172.21.15.112:0/63740694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb581408a0 0x7fdb581f6520 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.176+0000 7fdb5d075700 1 -- 172.21.15.112:0/63740694 >> 172.21.15.112:0/63740694 conn(0x7fdb580b1e90 msgr2=0x7fdb580b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:32.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.176+0000 7fdb5d075700 1 -- 172.21.15.112:0/63740694 shutdown_connections 2023-08-27T11:17:32.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.176+0000 7fdb5d075700 1 -- 172.21.15.112:0/63740694 wait complete. 2023-08-27T11:17:32.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:17:32.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.313+0000 7f21372cc700 1 Processor -- start 2023-08-27T11:17:32.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.313+0000 7f21372cc700 1 -- start start 2023-08-27T11:17:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f21372cc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2130146c20 0x7f2130160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f21372cc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21300b3440 0x7f213015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f21372cc700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21300b5070 con 0x7f213015e6e0 2023-08-27T11:17:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f21372cc700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2130147000 con 0x7f2130146c20 2023-08-27T11:17:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f21372cc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21301614a0 con 0x7f21300b3440 2023-08-27T11:17:32.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f2135068700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2130146c20 0x7f2130160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f2134867700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21300b3440 0x7f213015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f2135869700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f213015e6e0 0x7f213006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:57584/0 (socket says 172.21.15.112:57584) 2023-08-27T11:17:32.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f2134867700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21300b3440 0x7f213015e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:39096/0 (socket says 172.21.15.112:39096) 2023-08-27T11:17:32.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f2135869700 1 -- 172.21.15.112:0/3374606127 learned_addr learned my addr 172.21.15.112:0/3374606127 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:32.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f2135068700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2130146c20 0x7f2130160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:56100/0 (socket says 172.21.15.112:56100) 2023-08-27T11:17:32.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f2135068700 1 -- 172.21.15.112:0/3374606127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21300b3440 msgr2=0x7f213015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:32.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f2135068700 1 --2- 172.21.15.112:0/3374606127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21300b3440 0x7f213015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f2135068700 1 -- 172.21.15.112:0/3374606127 >> v1:172.21.15.138:6789/0 conn(0x7f213015e6e0 legacy=0x7f213006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f2135068700 1 -- 172.21.15.112:0/3374606127 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2130161740 con 0x7f2130146c20 2023-08-27T11:17:32.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.314+0000 7f2127fff700 1 -- 172.21.15.112:0/3374606127 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902541114 0 0) 0x7f2128001490 con 0x7f213015e6e0 2023-08-27T11:17:32.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.315+0000 7f2135068700 1 --2- 172.21.15.112:0/3374606127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2130146c20 0x7f2130160f60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2120009840 tx=0x7f2120009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54da4d78bcc0ad9d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:32.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.315+0000 7f2127fff700 1 -- 172.21.15.112:0/3374606127 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120011830 con 0x7f2130146c20 2023-08-27T11:17:32.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.315+0000 7f2127fff700 1 -- 172.21.15.112:0/3374606127 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2120011990 con 0x7f2130146c20 2023-08-27T11:17:32.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.315+0000 7f2127fff700 1 -- 172.21.15.112:0/3374606127 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120011d20 con 0x7f2130146c20 2023-08-27T11:17:32.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.315+0000 7f21372cc700 1 -- 172.21.15.112:0/3374606127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2130146c20 msgr2=0x7f2130160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.315+0000 7f21372cc700 1 --2- 172.21.15.112:0/3374606127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2130146c20 0x7f2130160f60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2120009840 tx=0x7f2120009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:32.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.315+0000 7f21372cc700 1 -- 172.21.15.112:0/3374606127 shutdown_connections 2023-08-27T11:17:32.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.315+0000 7f21372cc700 1 --2- 172.21.15.112:0/3374606127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2130146c20 0x7f2130160f60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.315+0000 7f21372cc700 1 --2- 172.21.15.112:0/3374606127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21300b3440 0x7f213015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.315+0000 7f21372cc700 1 -- 172.21.15.112:0/3374606127 >> 172.21.15.112:0/3374606127 conn(0x7f21300b1e90 msgr2=0x7f21300b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:32.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.315+0000 7f21372cc700 1 -- 172.21.15.112:0/3374606127 shutdown_connections 2023-08-27T11:17:32.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.315+0000 7f21372cc700 1 -- 172.21.15.112:0/3374606127 wait complete. 2023-08-27T11:17:32.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.317+0000 7f21372cc700 1 Processor -- start 2023-08-27T11:17:32.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.317+0000 7f21372cc700 1 -- start start 2023-08-27T11:17:32.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.317+0000 7f21372cc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21300b3440 0x7f213015a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.317+0000 7f21372cc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2130146c20 0x7f2130141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.317+0000 7f21372cc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f213015e6e0 0x7f2130141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.317+0000 7f21372cc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213015d930 con 0x7f2130146c20 2023-08-27T11:17:32.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.317+0000 7f21372cc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213013ee10 con 0x7f213015e6e0 2023-08-27T11:17:32.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.317+0000 7f21372cc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213013f0f0 con 0x7f21300b3440 2023-08-27T11:17:32.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.317+0000 7f2134867700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2130146c20 0x7f2130141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.317+0000 7f2134867700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2130146c20 0x7f2130141130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55682/0 (socket says 172.21.15.112:55682) 2023-08-27T11:17:32.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.317+0000 7f2134867700 1 -- 172.21.15.112:0/4094390642 learned_addr learned my addr 172.21.15.112:0/4094390642 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:32.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f2135068700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21300b3440 0x7f213015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f2135869700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f213015e6e0 0x7f2130141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f2134867700 1 -- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21300b3440 msgr2=0x7f213015a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f2134867700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21300b3440 0x7f213015a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f2134867700 1 -- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f213015e6e0 msgr2=0x7f2130141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f2134867700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f213015e6e0 0x7f2130141670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f2134867700 1 -- 172.21.15.112:0/4094390642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21200094a0 con 0x7f2130146c20 2023-08-27T11:17:32.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f2135068700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21300b3440 0x7f213015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:32.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f2135869700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f213015e6e0 0x7f2130141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:32.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f2134867700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2130146c20 0x7f2130141130 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f212c00e9b0 tx=0x7f212c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:32.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f2125ffb700 1 -- 172.21.15.112:0/4094390642 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c00c850 con 0x7f2130146c20 2023-08-27T11:17:32.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f2125ffb700 1 -- 172.21.15.112:0/4094390642 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212c00c9b0 con 0x7f2130146c20 2023-08-27T11:17:32.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f2125ffb700 1 -- 172.21.15.112:0/4094390642 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c00cd20 con 0x7f2130146c20 2023-08-27T11:17:32.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f21372cc700 1 -- 172.21.15.112:0/4094390642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213013f3d0 con 0x7f2130146c20 2023-08-27T11:17:32.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.318+0000 7f21372cc700 1 -- 172.21.15.112:0/4094390642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213015ada0 con 0x7f2130146c20 2023-08-27T11:17:32.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.319+0000 7f21372cc700 1 -- 172.21.15.112:0/4094390642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2130067e90 con 0x7f2130146c20 2023-08-27T11:17:32.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.320+0000 7f2125ffb700 1 -- 172.21.15.112:0/4094390642 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f212c013470 con 0x7f2130146c20 2023-08-27T11:17:32.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.320+0000 7f2125ffb700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f211406a0e0 0x7f211406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.320+0000 7f2125ffb700 1 -- 172.21.15.112:0/4094390642 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f212c088700 con 0x7f2130146c20 2023-08-27T11:17:32.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.320+0000 7f2135068700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f211406a0e0 0x7f211406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.321+0000 7f2135068700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f211406a0e0 0x7f211406c5a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2120000c00 tx=0x7f2120008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:32.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.324+0000 7f2125ffb700 1 -- 172.21.15.112:0/4094390642 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f212c053df0 con 0x7f2130146c20 2023-08-27T11:17:32.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.514+0000 7f21372cc700 1 -- 172.21.15.112:0/4094390642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2130067a20 con 0x7f2130146c20 2023-08-27T11:17:32.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.514+0000 7f2125ffb700 1 -- 172.21.15.112:0/4094390642 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f212c057410 con 0x7f2130146c20 2023-08-27T11:17:32.515 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:17:32.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.516+0000 7f21372cc700 1 -- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f211406a0e0 msgr2=0x7f211406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.516+0000 7f21372cc700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f211406a0e0 0x7f211406c5a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2120000c00 tx=0x7f2120008030 comp rx=0 tx=0).stop 2023-08-27T11:17:32.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.516+0000 7f21372cc700 1 -- 172.21.15.112:0/4094390642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2130146c20 msgr2=0x7f2130141130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.516+0000 7f21372cc700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2130146c20 0x7f2130141130 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f212c00e9b0 tx=0x7f212c00ed70 comp rx=0 tx=0).stop 2023-08-27T11:17:32.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.516+0000 7f21372cc700 1 -- 172.21.15.112:0/4094390642 shutdown_connections 2023-08-27T11:17:32.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.516+0000 7f21372cc700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f211406a0e0 0x7f211406c5a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.517+0000 7f21372cc700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f213015e6e0 0x7f2130141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.517+0000 7f21372cc700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21300b3440 0x7f213015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.517+0000 7f21372cc700 1 --2- 172.21.15.112:0/4094390642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2130146c20 0x7f2130141130 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.517+0000 7f21372cc700 1 -- 172.21.15.112:0/4094390642 >> 172.21.15.112:0/4094390642 conn(0x7f21300b1e90 msgr2=0x7f2130070780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:32.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.517+0000 7f21372cc700 1 -- 172.21.15.112:0/4094390642 shutdown_connections 2023-08-27T11:17:32.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.517+0000 7f21372cc700 1 -- 172.21.15.112:0/4094390642 wait complete. 2023-08-27T11:17:32.529 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:32.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.648+0000 7ff945620700 1 Processor -- start 2023-08-27T11:17:32.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.648+0000 7ff945620700 1 -- start start 2023-08-27T11:17:32.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.649+0000 7ff945620700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9400b1400 0x7ff9400cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.649+0000 7ff945620700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff9400b17e0 0x7ff94006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.649+0000 7ff945620700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff940161ce0 0x7ff94006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.649+0000 7ff945620700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff940161850 con 0x7ff9400b17e0 2023-08-27T11:17:32.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.649+0000 7ff945620700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94006e7d0 con 0x7ff940161ce0 2023-08-27T11:17:32.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.649+0000 7ff945620700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9400cfc10 con 0x7ff9400b1400 2023-08-27T11:17:32.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.649+0000 7ff93e7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff9400b17e0 0x7ff94006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.649+0000 7ff93e7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff9400b17e0 0x7ff94006e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55688/0 (socket says 172.21.15.112:55688) 2023-08-27T11:17:32.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.649+0000 7ff93e7fc700 1 -- 172.21.15.112:0/3109012406 learned_addr learned my addr 172.21.15.112:0/3109012406 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:32.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.649+0000 7ff93effd700 1 --2- 172.21.15.112:0/3109012406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9400b1400 0x7ff9400cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.649+0000 7ff93f7fe700 1 --2- 172.21.15.112:0/3109012406 >> v2:172.21.15.138:3300/0 conn(0x7ff940161ce0 0x7ff94006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.650+0000 7ff93e7fc700 1 -- 172.21.15.112:0/3109012406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9400b1400 msgr2=0x7ff9400cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:32.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.650+0000 7ff93e7fc700 1 --2- 172.21.15.112:0/3109012406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9400b1400 0x7ff9400cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.650+0000 7ff93e7fc700 1 -- 172.21.15.112:0/3109012406 >> v2:172.21.15.138:3300/0 conn(0x7ff940161ce0 msgr2=0x7ff94006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:32.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.650+0000 7ff93e7fc700 1 --2- 172.21.15.112:0/3109012406 >> v2:172.21.15.138:3300/0 conn(0x7ff940161ce0 0x7ff94006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.650+0000 7ff93e7fc700 1 -- 172.21.15.112:0/3109012406 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff940161200 con 0x7ff9400b17e0 2023-08-27T11:17:32.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.650+0000 7ff93e7fc700 1 --2- 172.21.15.112:0/3109012406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9400b17e0 0x7ff94006e290 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9400ca550 tx=0x7ff930009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e956c388d9336f9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:32.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.650+0000 7ff93dffb700 1 -- 172.21.15.112:0/3109012406 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930011830 con 0x7ff9400b17e0 2023-08-27T11:17:32.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.650+0000 7ff93dffb700 1 -- 172.21.15.112:0/3109012406 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff930011990 con 0x7ff9400b17e0 2023-08-27T11:17:32.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.650+0000 7ff93dffb700 1 -- 172.21.15.112:0/3109012406 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930011d20 con 0x7ff9400b17e0 2023-08-27T11:17:32.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.651+0000 7ff945620700 1 -- 172.21.15.112:0/3109012406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9400b17e0 msgr2=0x7ff94006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.651+0000 7ff945620700 1 --2- 172.21.15.112:0/3109012406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9400b17e0 0x7ff94006e290 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9400ca550 tx=0x7ff930009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.651+0000 7ff945620700 1 -- 172.21.15.112:0/3109012406 shutdown_connections 2023-08-27T11:17:32.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.651+0000 7ff945620700 1 --2- 172.21.15.112:0/3109012406 >> v2:172.21.15.138:3300/0 conn(0x7ff940161ce0 0x7ff94006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.651+0000 7ff945620700 1 --2- 172.21.15.112:0/3109012406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9400b1400 0x7ff9400cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.651+0000 7ff945620700 1 --2- 172.21.15.112:0/3109012406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9400b17e0 0x7ff94006e290 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.651+0000 7ff945620700 1 -- 172.21.15.112:0/3109012406 >> 172.21.15.112:0/3109012406 conn(0x7ff9400d0000 msgr2=0x7ff9400d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:32.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.651+0000 7ff945620700 1 -- 172.21.15.112:0/3109012406 shutdown_connections 2023-08-27T11:17:32.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.651+0000 7ff945620700 1 -- 172.21.15.112:0/3109012406 wait complete. 2023-08-27T11:17:32.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.652+0000 7ff945620700 1 Processor -- start 2023-08-27T11:17:32.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.652+0000 7ff945620700 1 -- start start 2023-08-27T11:17:32.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff945620700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9400b1400 0x7ff9400c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff945620700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9400b17e0 0x7ff9400c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff945620700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff940161ce0 0x7ff9400c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff945620700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9400c4ce0 con 0x7ff9400b1400 2023-08-27T11:17:32.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff945620700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9400c40d0 con 0x7ff940161ce0 2023-08-27T11:17:32.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff945620700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9400c43b0 con 0x7ff9400b17e0 2023-08-27T11:17:32.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff93effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9400b1400 0x7ff9400c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff93effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9400b1400 0x7ff9400c2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55692/0 (socket says 172.21.15.112:55692) 2023-08-27T11:17:32.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff93effd700 1 -- 172.21.15.112:0/451730750 learned_addr learned my addr 172.21.15.112:0/451730750 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:32.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff93e7fc700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9400b17e0 0x7ff9400c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff93f7fe700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff940161ce0 0x7ff9400c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff93effd700 1 -- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9400b17e0 msgr2=0x7ff9400c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff93effd700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9400b17e0 0x7ff9400c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff93effd700 1 -- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff940161ce0 msgr2=0x7ff9400c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff93effd700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff940161ce0 0x7ff9400c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff93effd700 1 -- 172.21.15.112:0/451730750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff930009550 con 0x7ff9400b1400 2023-08-27T11:17:32.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff93effd700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9400b1400 0x7ff9400c2e70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff93400e930 tx=0x7ff93400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:32.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff9277fe700 1 -- 172.21.15.112:0/451730750 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934013680 con 0x7ff9400b1400 2023-08-27T11:17:32.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff9277fe700 1 -- 172.21.15.112:0/451730750 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9340137e0 con 0x7ff9400b1400 2023-08-27T11:17:32.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff945620700 1 -- 172.21.15.112:0/451730750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9400c4690 con 0x7ff9400b1400 2023-08-27T11:17:32.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff9277fe700 1 -- 172.21.15.112:0/451730750 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934013b70 con 0x7ff9400b1400 2023-08-27T11:17:32.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.653+0000 7ff945620700 1 -- 172.21.15.112:0/451730750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9400c33b0 con 0x7ff9400b1400 2023-08-27T11:17:32.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.654+0000 7ff945620700 1 -- 172.21.15.112:0/451730750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff940067a20 con 0x7ff9400b1400 2023-08-27T11:17:32.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.655+0000 7ff9277fe700 1 -- 172.21.15.112:0/451730750 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff934013cd0 con 0x7ff9400b1400 2023-08-27T11:17:32.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.655+0000 7ff9277fe700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff92c06a0e0 0x7ff92c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.655+0000 7ff9277fe700 1 -- 172.21.15.112:0/451730750 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff934088530 con 0x7ff9400b1400 2023-08-27T11:17:32.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.655+0000 7ff93e7fc700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff92c06a0e0 0x7ff92c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.656+0000 7ff93e7fc700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff92c06a0e0 0x7ff92c06c5a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff930009c10 tx=0x7ff930008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:32.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.660+0000 7ff9277fe700 1 -- 172.21.15.112:0/451730750 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff934053b90 con 0x7ff9400b1400 2023-08-27T11:17:32.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.847+0000 7ff945620700 1 -- 172.21.15.112:0/451730750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff94006db60 con 0x7ff9400b1400 2023-08-27T11:17:32.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.849+0000 7ff9277fe700 1 -- 172.21.15.112:0/451730750 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7ff9340571b0 con 0x7ff9400b1400 2023-08-27T11:17:32.850 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:32.850 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:32.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.851+0000 7ff945620700 1 -- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff92c06a0e0 msgr2=0x7ff92c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.851+0000 7ff945620700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff92c06a0e0 0x7ff92c06c5a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff930009c10 tx=0x7ff930008030 comp rx=0 tx=0).stop 2023-08-27T11:17:32.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.851+0000 7ff945620700 1 -- 172.21.15.112:0/451730750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9400b1400 msgr2=0x7ff9400c2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.851+0000 7ff945620700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9400b1400 0x7ff9400c2e70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff93400e930 tx=0x7ff93400ec40 comp rx=0 tx=0).stop 2023-08-27T11:17:32.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.851+0000 7ff945620700 1 -- 172.21.15.112:0/451730750 shutdown_connections 2023-08-27T11:17:32.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.851+0000 7ff945620700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff92c06a0e0 0x7ff92c06c5a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.851+0000 7ff945620700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff940161ce0 0x7ff9400c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.851+0000 7ff945620700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9400b1400 0x7ff9400c2e70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.851+0000 7ff945620700 1 --2- 172.21.15.112:0/451730750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9400b17e0 0x7ff9400c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.851+0000 7ff945620700 1 -- 172.21.15.112:0/451730750 >> 172.21.15.112:0/451730750 conn(0x7ff9400d0000 msgr2=0x7ff9400ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:32.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.851+0000 7ff945620700 1 -- 172.21.15.112:0/451730750 shutdown_connections 2023-08-27T11:17:32.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.851+0000 7ff945620700 1 -- 172.21.15.112:0/451730750 wait complete. 2023-08-27T11:17:32.863 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:17:02.484063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:17:03.722976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:17:32.865 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:17:32.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.987+0000 7fa5cebf1700 1 Processor -- start 2023-08-27T11:17:32.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.987+0000 7fa5cebf1700 1 -- start start 2023-08-27T11:17:32.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.987+0000 7fa5cebf1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5c806e4c0 0x7fa5c80acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.987+0000 7fa5cebf1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5c80ad000 0x7fa5c815d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.987+0000 7fa5cebf1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c8160fa0 con 0x7fa5c8146b60 2023-08-27T11:17:32.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.987+0000 7fa5cebf1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c8161b70 con 0x7fa5c806e4c0 2023-08-27T11:17:32.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.987+0000 7fa5cebf1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8143ad0 con 0x7fa5c80ad000 2023-08-27T11:17:32.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5cc98d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa5c8146b60 0x7fa5c806ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50082/0 (socket says 172.21.15.112:50082) 2023-08-27T11:17:32.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5cc98d700 1 -- 172.21.15.112:0/1585594705 learned_addr learned my addr 172.21.15.112:0/1585594705 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:32.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5cd18e700 1 --2- 172.21.15.112:0/1585594705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5c80ad000 0x7fa5c815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c7fff700 1 --2- 172.21.15.112:0/1585594705 >> v2:172.21.15.138:3300/0 conn(0x7fa5c806e4c0 0x7fa5c80acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192298973 0 0) 0x7fa5c8160fa0 con 0x7fa5c8146b60 2023-08-27T11:17:32.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b80035b0 con 0x7fa5c8146b60 2023-08-27T11:17:32.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1645384998 0 0) 0x7fa5b80035b0 con 0x7fa5c8146b60 2023-08-27T11:17:32.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5c80ad000 msgr2=0x7fa5c815d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:32.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 --2- 172.21.15.112:0/1585594705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5c80ad000 0x7fa5c815d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 >> v2:172.21.15.138:3300/0 conn(0x7fa5c806e4c0 msgr2=0x7fa5c80acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:32.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 --2- 172.21.15.112:0/1585594705 >> v2:172.21.15.138:3300/0 conn(0x7fa5c806e4c0 0x7fa5c80acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:32.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c8146f40 con 0x7fa5c8146b60 2023-08-27T11:17:32.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fa5bc001670 con 0x7fa5c8146b60 2023-08-27T11:17:32.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c7fff700 1 --2- 172.21.15.112:0/1585594705 >> v2:172.21.15.138:3300/0 conn(0x7fa5c806e4c0 0x7fa5c80acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 >> v1:172.21.15.112:6789/0 conn(0x7fa5c8146b60 legacy=0x7fa5c806ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 --2- 172.21.15.112:0/1585594705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b8003a00 0x7fa5b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 --2- 172.21.15.112:0/1585594705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5c80ad000 0x7fa5b8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 --2- 172.21.15.112:0/1585594705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b8004d90 0x7fa5b8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8161b70 con 0x7fa5b8004d90 2023-08-27T11:17:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8143ad0 con 0x7fa5b8003a00 2023-08-27T11:17:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b800ba10 con 0x7fa5c80ad000 2023-08-27T11:17:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5cd18e700 1 --2- 172.21.15.112:0/1585594705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b8004d90 0x7fa5b8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5c7fff700 1 --2- 172.21.15.112:0/1585594705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5c80ad000 0x7fa5b8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.988+0000 7fa5cc98d700 1 --2- 172.21.15.112:0/1585594705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b8003a00 0x7fa5b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5cd18e700 1 -- 172.21.15.112:0/1585594705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5c80ad000 msgr2=0x7fa5b8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5cd18e700 1 --2- 172.21.15.112:0/1585594705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5c80ad000 0x7fa5b8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5cd18e700 1 -- 172.21.15.112:0/1585594705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b8003a00 msgr2=0x7fa5b8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5cd18e700 1 --2- 172.21.15.112:0/1585594705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b8003a00 0x7fa5b8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5cd18e700 1 -- 172.21.15.112:0/1585594705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b80035b0 con 0x7fa5b8004d90 2023-08-27T11:17:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5cd18e700 1 --2- 172.21.15.112:0/1585594705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b8004d90 0x7fa5b8005210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000e170 tx=0x7fa5b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dbdabad27b4321 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c000ec90 con 0x7fa5b8004d90 2023-08-27T11:17:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c000f040 con 0x7fa5b8004d90 2023-08-27T11:17:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5c77fe700 1 -- 172.21.15.112:0/1585594705 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0014480 con 0x7fa5b8004d90 2023-08-27T11:17:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1585594705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b8004d90 msgr2=0x7fa5b8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5cebf1700 1 --2- 172.21.15.112:0/1585594705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b8004d90 0x7fa5b8005210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000e170 tx=0x7fa5b8005880 comp rx=0 tx=0).stop 2023-08-27T11:17:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5cd18e700 1 -- 172.21.15.112:0/1585594705 reap_dead start 2023-08-27T11:17:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1585594705 shutdown_connections 2023-08-27T11:17:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1585594705 >> 172.21.15.112:0/1585594705 conn(0x7fa5c80cc240 msgr2=0x7fa5c80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1585594705 shutdown_connections 2023-08-27T11:17:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.989+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1585594705 wait complete. 2023-08-27T11:17:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.991+0000 7fa5cebf1700 1 Processor -- start 2023-08-27T11:17:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.991+0000 7fa5cebf1700 1 -- start start 2023-08-27T11:17:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.991+0000 7fa5cebf1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5b8004d90 0x7fa5c8071110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.991+0000 7fa5cebf1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c80ce7a0 0x7fa5c80cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.991+0000 7fa5cebf1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5c806faa0 0x7fa5c806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.991+0000 7fa5cebf1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c813bc50 con 0x7fa5c806faa0 2023-08-27T11:17:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cebf1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c81f28a0 con 0x7fa5c80ce7a0 2023-08-27T11:17:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cebf1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80cf160 con 0x7fa5b8004d90 2023-08-27T11:17:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cd18e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5c806faa0 0x7fa5c806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cd18e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5c806faa0 0x7fa5c806ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55716/0 (socket says 172.21.15.112:55716) 2023-08-27T11:17:33.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cd18e700 1 -- 172.21.15.112:0/1306064830 learned_addr learned my addr 172.21.15.112:0/1306064830 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:33.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5c7fff700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c80ce7a0 0x7fa5c80cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cc98d700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5b8004d90 0x7fa5c8071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cd18e700 1 -- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5b8004d90 msgr2=0x7fa5c8071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cd18e700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5b8004d90 0x7fa5c8071110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cd18e700 1 -- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c80ce7a0 msgr2=0x7fa5c80cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cd18e700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c80ce7a0 0x7fa5c80cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cd18e700 1 -- 172.21.15.112:0/1306064830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c8071650 con 0x7fa5c806faa0 2023-08-27T11:17:33.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5c7fff700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c80ce7a0 0x7fa5c80cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:33.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cd18e700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5c806faa0 0x7fa5c806ff50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0016000 tx=0x7fa5c0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:33.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5c57fa700 1 -- 172.21.15.112:0/1306064830 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0007000 con 0x7fa5c806faa0 2023-08-27T11:17:33.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5c57fa700 1 -- 172.21.15.112:0/1306064830 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c000ee80 con 0x7fa5c806faa0 2023-08-27T11:17:33.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1306064830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c80718f0 con 0x7fa5c806faa0 2023-08-27T11:17:33.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5c57fa700 1 -- 172.21.15.112:0/1306064830 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0003c60 con 0x7fa5c806faa0 2023-08-27T11:17:33.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.992+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1306064830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c8071eb0 con 0x7fa5c806faa0 2023-08-27T11:17:33.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.993+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1306064830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c813bd90 con 0x7fa5c806faa0 2023-08-27T11:17:33.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.994+0000 7fa5c57fa700 1 -- 172.21.15.112:0/1306064830 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa5c0024030 con 0x7fa5c806faa0 2023-08-27T11:17:33.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.994+0000 7fa5c57fa700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa5ac06a0e0 0x7fa5ac06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.994+0000 7fa5c57fa700 1 -- 172.21.15.112:0/1306064830 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa5c0089de0 con 0x7fa5c806faa0 2023-08-27T11:17:33.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.994+0000 7fa5cc98d700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa5ac06a0e0 0x7fa5ac06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.995+0000 7fa5cc98d700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa5ac06a0e0 0x7fa5ac06c5a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc000c00 tx=0x7fa5bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:33.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:32.998+0000 7fa5c57fa700 1 -- 172.21.15.112:0/1306064830 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa5c813bd90 con 0x7fa5c806faa0 2023-08-27T11:17:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.188+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1306064830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa5c813bd90 con 0x7fa5c806faa0 2023-08-27T11:17:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.189+0000 7fa5c57fa700 1 -- 172.21.15.112:0/1306064830 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa5c0018020 con 0x7fa5c806faa0 2023-08-27T11:17:33.190 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:33.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.190+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa5ac06a0e0 msgr2=0x7fa5ac06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.190+0000 7fa5cebf1700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa5ac06a0e0 0x7fa5ac06c5a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc000c00 tx=0x7fa5bc009040 comp rx=0 tx=0).stop 2023-08-27T11:17:33.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.190+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1306064830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5c806faa0 msgr2=0x7fa5c806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.190+0000 7fa5cebf1700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5c806faa0 0x7fa5c806ff50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0016000 tx=0x7fa5c0004b40 comp rx=0 tx=0).stop 2023-08-27T11:17:33.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.191+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1306064830 shutdown_connections 2023-08-27T11:17:33.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.191+0000 7fa5cebf1700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa5ac06a0e0 0x7fa5ac06c5a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.191+0000 7fa5cebf1700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5c806faa0 0x7fa5c806ff50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.191+0000 7fa5cebf1700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5b8004d90 0x7fa5c8071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.191+0000 7fa5cebf1700 1 --2- 172.21.15.112:0/1306064830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c80ce7a0 0x7fa5c80cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.191+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1306064830 >> 172.21.15.112:0/1306064830 conn(0x7fa5c80cc240 msgr2=0x7fa5c8145c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:33.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.191+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1306064830 shutdown_connections 2023-08-27T11:17:33.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.191+0000 7fa5cebf1700 1 -- 172.21.15.112:0/1306064830 wait complete. 2023-08-27T11:17:33.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:33.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.327+0000 7f951ffc4700 1 Processor -- start 2023-08-27T11:17:33.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.327+0000 7f951ffc4700 1 -- start start 2023-08-27T11:17:33.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.328+0000 7f951ffc4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f95180b3440 0x7f951815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.328+0000 7f951ffc4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f951815e6e0 0x7f951806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.328+0000 7f951ffc4700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95180b5070 con 0x7f9518146c20 2023-08-27T11:17:33.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.328+0000 7f951ffc4700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9518147000 con 0x7f95180b3440 2023-08-27T11:17:33.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.328+0000 7f951ffc4700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9518161740 con 0x7f951815e6e0 2023-08-27T11:17:33.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.328+0000 7f951d55f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f95180b3440 0x7f951815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.328+0000 7f951d55f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f95180b3440 0x7f951815e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55722/0 (socket says 172.21.15.112:55722) 2023-08-27T11:17:33.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.328+0000 7f951d55f700 1 -- 172.21.15.112:0/3828309978 learned_addr learned my addr 172.21.15.112:0/3828309978 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:33.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.328+0000 7f951e561700 1 --2- 172.21.15.112:0/3828309978 >> v2:172.21.15.138:3300/0 conn(0x7f951815e6e0 0x7f951806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.328+0000 7f951d55f700 1 -- 172.21.15.112:0/3828309978 >> v2:172.21.15.138:3300/0 conn(0x7f951815e6e0 msgr2=0x7f951806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:33.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.328+0000 7f951d55f700 1 --2- 172.21.15.112:0/3828309978 >> v2:172.21.15.138:3300/0 conn(0x7f951815e6e0 0x7f951806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.328+0000 7f951d55f700 1 -- 172.21.15.112:0/3828309978 >> v1:172.21.15.138:6789/0 conn(0x7f9518146c20 legacy=0x7f9518160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:33.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.328+0000 7f951d55f700 1 -- 172.21.15.112:0/3828309978 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9518161a20 con 0x7f95180b3440 2023-08-27T11:17:33.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.329+0000 7f951d55f700 1 --2- 172.21.15.112:0/3828309978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95180b3440 0x7f951815e1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f95180b38c0 tx=0x7f9514009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9e83aae1344b2b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:33.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.329+0000 7f951cd5e700 1 -- 172.21.15.112:0/3828309978 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514004230 con 0x7f95180b3440 2023-08-27T11:17:33.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.329+0000 7f951cd5e700 1 -- 172.21.15.112:0/3828309978 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9514004390 con 0x7f95180b3440 2023-08-27T11:17:33.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.329+0000 7f951cd5e700 1 -- 172.21.15.112:0/3828309978 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140036a0 con 0x7f95180b3440 2023-08-27T11:17:33.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.329+0000 7f951ffc4700 1 -- 172.21.15.112:0/3828309978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95180b3440 msgr2=0x7f951815e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.329+0000 7f951ffc4700 1 --2- 172.21.15.112:0/3828309978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95180b3440 0x7f951815e1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f95180b38c0 tx=0x7f9514009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.329+0000 7f951ffc4700 1 -- 172.21.15.112:0/3828309978 shutdown_connections 2023-08-27T11:17:33.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.329+0000 7f951ffc4700 1 --2- 172.21.15.112:0/3828309978 >> v2:172.21.15.138:3300/0 conn(0x7f951815e6e0 0x7f951806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.329+0000 7f951ffc4700 1 --2- 172.21.15.112:0/3828309978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95180b3440 0x7f951815e1a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.329+0000 7f951ffc4700 1 -- 172.21.15.112:0/3828309978 >> 172.21.15.112:0/3828309978 conn(0x7f95180b1e90 msgr2=0x7f95180b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:33.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.329+0000 7f951ffc4700 1 -- 172.21.15.112:0/3828309978 shutdown_connections 2023-08-27T11:17:33.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.329+0000 7f951ffc4700 1 -- 172.21.15.112:0/3828309978 wait complete. 2023-08-27T11:17:33.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.331+0000 7f951ffc4700 1 Processor -- start 2023-08-27T11:17:33.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.331+0000 7f951ffc4700 1 -- start start 2023-08-27T11:17:33.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.331+0000 7f951ffc4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95180b3440 0x7f951815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.331+0000 7f951ffc4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9518146c20 0x7f9518142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951ffc4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f951815e6e0 0x7f9518143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951ffc4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95180aef80 con 0x7f95180b3440 2023-08-27T11:17:33.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951ffc4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95181f33f0 con 0x7f9518146c20 2023-08-27T11:17:33.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951ffc4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95181f36d0 con 0x7f951815e6e0 2023-08-27T11:17:33.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951dd60700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95180b3440 0x7f951815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951dd60700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95180b3440 0x7f951815aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55726/0 (socket says 172.21.15.112:55726) 2023-08-27T11:17:33.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951dd60700 1 -- 172.21.15.112:0/118263508 learned_addr learned my addr 172.21.15.112:0/118263508 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:33.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951d55f700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9518146c20 0x7f9518142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951e561700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f951815e6e0 0x7f9518143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951dd60700 1 -- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f951815e6e0 msgr2=0x7f9518143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951dd60700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f951815e6e0 0x7f9518143070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951dd60700 1 -- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9518146c20 msgr2=0x7f9518142b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951dd60700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9518146c20 0x7f9518142b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951dd60700 1 -- 172.21.15.112:0/118263508 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9514009550 con 0x7f95180b3440 2023-08-27T11:17:33.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951d55f700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9518146c20 0x7f9518142b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:33.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951dd60700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95180b3440 0x7f951815aad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f950800c4d0 tx=0x7f950800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:33.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f950e7fc700 1 -- 172.21.15.112:0/118263508 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508011790 con 0x7f95180b3440 2023-08-27T11:17:33.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f950e7fc700 1 -- 172.21.15.112:0/118263508 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95080118f0 con 0x7f95180b3440 2023-08-27T11:17:33.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951ffc4700 1 -- 172.21.15.112:0/118263508 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9518070780 con 0x7f95180b3440 2023-08-27T11:17:33.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f950e7fc700 1 -- 172.21.15.112:0/118263508 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508011ca0 con 0x7f95180b3440 2023-08-27T11:17:33.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.332+0000 7f951ffc4700 1 -- 172.21.15.112:0/118263508 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9518070d40 con 0x7f95180b3440 2023-08-27T11:17:33.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.333+0000 7f951ffc4700 1 -- 172.21.15.112:0/118263508 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9518067e90 con 0x7f95180b3440 2023-08-27T11:17:33.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.334+0000 7f950e7fc700 1 -- 172.21.15.112:0/118263508 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f950801c460 con 0x7f95180b3440 2023-08-27T11:17:33.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.334+0000 7f950e7fc700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f94fc06a0e0 0x7f94fc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.334+0000 7f950e7fc700 1 -- 172.21.15.112:0/118263508 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9508088340 con 0x7f95180b3440 2023-08-27T11:17:33.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.334+0000 7f951d55f700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f94fc06a0e0 0x7f94fc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.335+0000 7f951d55f700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f94fc06a0e0 0x7f94fc06c5a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9514009130 tx=0x7f9514008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:33.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.338+0000 7f950e7fc700 1 -- 172.21.15.112:0/118263508 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9508053ac0 con 0x7f95180b3440 2023-08-27T11:17:33.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.526+0000 7f951ffc4700 1 -- 172.21.15.112:0/118263508 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9518067a20 con 0x7f95180b3440 2023-08-27T11:17:33.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.526+0000 7f950e7fc700 1 -- 172.21.15.112:0/118263508 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95080570e0 con 0x7f95180b3440 2023-08-27T11:17:33.527 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:33.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.528+0000 7f951ffc4700 1 -- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f94fc06a0e0 msgr2=0x7f94fc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.528+0000 7f951ffc4700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f94fc06a0e0 0x7f94fc06c5a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9514009130 tx=0x7f9514008030 comp rx=0 tx=0).stop 2023-08-27T11:17:33.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.528+0000 7f951ffc4700 1 -- 172.21.15.112:0/118263508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95180b3440 msgr2=0x7f951815aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.528+0000 7f951ffc4700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95180b3440 0x7f951815aad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f950800c4d0 tx=0x7f950800c890 comp rx=0 tx=0).stop 2023-08-27T11:17:33.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.528+0000 7f951ffc4700 1 -- 172.21.15.112:0/118263508 shutdown_connections 2023-08-27T11:17:33.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.528+0000 7f951ffc4700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f94fc06a0e0 0x7f94fc06c5a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.528+0000 7f951ffc4700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f951815e6e0 0x7f9518143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.528+0000 7f951ffc4700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95180b3440 0x7f951815aad0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.528+0000 7f951ffc4700 1 --2- 172.21.15.112:0/118263508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9518146c20 0x7f9518142b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.528+0000 7f951ffc4700 1 -- 172.21.15.112:0/118263508 >> 172.21.15.112:0/118263508 conn(0x7f95180b1e90 msgr2=0x7f9518140f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:33.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.529+0000 7f951ffc4700 1 -- 172.21.15.112:0/118263508 shutdown_connections 2023-08-27T11:17:33.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.529+0000 7f951ffc4700 1 -- 172.21.15.112:0/118263508 wait complete. 2023-08-27T11:17:33.541 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:17:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.663+0000 7f01291f5700 1 Processor -- start 2023-08-27T11:17:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.663+0000 7f01291f5700 1 -- start start 2023-08-27T11:17:33.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.664+0000 7f01291f5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01240b3440 0x7f012415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.664+0000 7f01291f5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f012415e6e0 0x7f012406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.664+0000 7f01291f5700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01240b5070 con 0x7f0124146c20 2023-08-27T11:17:33.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.664+0000 7f01291f5700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0124147000 con 0x7f012415e6e0 2023-08-27T11:17:33.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.664+0000 7f01291f5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0124161740 con 0x7f01240b3440 2023-08-27T11:17:33.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.664+0000 7f012359e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f012415e6e0 0x7f012406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.664+0000 7f0122d9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0124146c20 0x7f0124160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50116/0 (socket says 172.21.15.112:50116) 2023-08-27T11:17:33.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.664+0000 7f0122d9d700 1 -- 172.21.15.112:0/1873551978 learned_addr learned my addr 172.21.15.112:0/1873551978 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:33.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.664+0000 7f012259c700 1 --2- 172.21.15.112:0/1873551978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01240b3440 0x7f012415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f012359e700 1 -- 172.21.15.112:0/1873551978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01240b3440 msgr2=0x7f012415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:33.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f012359e700 1 --2- 172.21.15.112:0/1873551978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01240b3440 0x7f012415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f012359e700 1 -- 172.21.15.112:0/1873551978 >> v1:172.21.15.112:6789/0 conn(0x7f0124146c20 legacy=0x7f0124160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f012359e700 1 -- 172.21.15.112:0/1873551978 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0124161a20 con 0x7f012415e6e0 2023-08-27T11:17:33.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f0121d9b700 1 -- 172.21.15.112:0/1873551978 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817406451 0 0) 0x7f0118001110 con 0x7f0124146c20 2023-08-27T11:17:33.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f012359e700 1 --2- 172.21.15.112:0/1873551978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012415e6e0 0x7f012406f790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f011c009d90 tx=0x7f011c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc303b75948588d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:33.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f0121d9b700 1 -- 172.21.15.112:0/1873551978 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c00e070 con 0x7f012415e6e0 2023-08-27T11:17:33.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f0121d9b700 1 -- 172.21.15.112:0/1873551978 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011c00bbd0 con 0x7f012415e6e0 2023-08-27T11:17:33.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f0121d9b700 1 -- 172.21.15.112:0/1873551978 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c012430 con 0x7f012415e6e0 2023-08-27T11:17:33.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f01291f5700 1 -- 172.21.15.112:0/1873551978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012415e6e0 msgr2=0x7f012406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f01291f5700 1 --2- 172.21.15.112:0/1873551978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012415e6e0 0x7f012406f790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f011c009d90 tx=0x7f011c00b330 comp rx=0 tx=0).stop 2023-08-27T11:17:33.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f01291f5700 1 -- 172.21.15.112:0/1873551978 shutdown_connections 2023-08-27T11:17:33.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f01291f5700 1 --2- 172.21.15.112:0/1873551978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012415e6e0 0x7f012406f790 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f01291f5700 1 --2- 172.21.15.112:0/1873551978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01240b3440 0x7f012415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f01291f5700 1 -- 172.21.15.112:0/1873551978 >> 172.21.15.112:0/1873551978 conn(0x7f01240b1e90 msgr2=0x7f01240b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:33.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f01291f5700 1 -- 172.21.15.112:0/1873551978 shutdown_connections 2023-08-27T11:17:33.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.665+0000 7f01291f5700 1 -- 172.21.15.112:0/1873551978 wait complete. 2023-08-27T11:17:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.667+0000 7f01291f5700 1 Processor -- start 2023-08-27T11:17:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.667+0000 7f01291f5700 1 -- start start 2023-08-27T11:17:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f01291f5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01240b3440 0x7f01241603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f01291f5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0124146c20 0x7f01241f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f01291f5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012415e6e0 0x7f012415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f01291f5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012415f9a0 con 0x7f012415e6e0 2023-08-27T11:17:33.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f01291f5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124160920 con 0x7f0124146c20 2023-08-27T11:17:33.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f01291f5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0124146260 con 0x7f01240b3440 2023-08-27T11:17:33.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f012359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012415e6e0 0x7f012415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f012359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012415e6e0 0x7f012415a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55742/0 (socket says 172.21.15.112:55742) 2023-08-27T11:17:33.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f012359e700 1 -- 172.21.15.112:0/3166724837 learned_addr learned my addr 172.21.15.112:0/3166724837 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:33.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f0122d9d700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01240b3440 0x7f01241603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f012259c700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0124146c20 0x7f01241f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f012359e700 1 -- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01240b3440 msgr2=0x7f01241603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f012359e700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01240b3440 0x7f01241603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f012359e700 1 -- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0124146c20 msgr2=0x7f01241f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f012359e700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0124146c20 0x7f01241f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f012359e700 1 -- 172.21.15.112:0/3166724837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c009a70 con 0x7f012415e6e0 2023-08-27T11:17:33.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f0122d9d700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01240b3440 0x7f01241603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:33.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f012259c700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0124146c20 0x7f01241f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:33.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f012359e700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012415e6e0 0x7f012415a2c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f012406e8e0 tx=0x7f011c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:33.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f010b7fe700 1 -- 172.21.15.112:0/3166724837 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c012430 con 0x7f012415e6e0 2023-08-27T11:17:33.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f010b7fe700 1 -- 172.21.15.112:0/3166724837 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011c020070 con 0x7f012415e6e0 2023-08-27T11:17:33.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f01291f5700 1 -- 172.21.15.112:0/3166724837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012415f3e0 con 0x7f012415e6e0 2023-08-27T11:17:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.668+0000 7f010b7fe700 1 -- 172.21.15.112:0/3166724837 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c004280 con 0x7f012415e6e0 2023-08-27T11:17:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.669+0000 7f01291f5700 1 -- 172.21.15.112:0/3166724837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01241f6780 con 0x7f012415e6e0 2023-08-27T11:17:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.669+0000 7f01291f5700 1 -- 172.21.15.112:0/3166724837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012413eb50 con 0x7f012415e6e0 2023-08-27T11:17:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.670+0000 7f010b7fe700 1 -- 172.21.15.112:0/3166724837 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f011c0043e0 con 0x7f012415e6e0 2023-08-27T11:17:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.670+0000 7f010b7fe700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f011006a0e0 0x7f011006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.670+0000 7f010b7fe700 1 -- 172.21.15.112:0/3166724837 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f011c022070 con 0x7f012415e6e0 2023-08-27T11:17:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.670+0000 7f0122d9d700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f011006a0e0 0x7f011006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.671+0000 7f0122d9d700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f011006a0e0 0x7f011006c5a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0118000f60 tx=0x7f0118007ae0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.675+0000 7f010b7fe700 1 -- 172.21.15.112:0/3166724837 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f012413eb50 con 0x7f012415e6e0 2023-08-27T11:17:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.860+0000 7f01291f5700 1 -- 172.21.15.112:0/3166724837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f012413eb50 con 0x7f012415e6e0 2023-08-27T11:17:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.860+0000 7f010b7fe700 1 -- 172.21.15.112:0/3166724837 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f012413eb50 con 0x7f012415e6e0 2023-08-27T11:17:33.862 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:17:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.862+0000 7f01291f5700 1 -- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f011006a0e0 msgr2=0x7f011006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.862+0000 7f01291f5700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f011006a0e0 0x7f011006c5a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0118000f60 tx=0x7f0118007ae0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.862+0000 7f01291f5700 1 -- 172.21.15.112:0/3166724837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012415e6e0 msgr2=0x7f012415a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.862+0000 7f01291f5700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012415e6e0 0x7f012415a2c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f012406e8e0 tx=0x7f011c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.862+0000 7f01291f5700 1 -- 172.21.15.112:0/3166724837 shutdown_connections 2023-08-27T11:17:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.862+0000 7f01291f5700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f011006a0e0 0x7f011006c5a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.862+0000 7f01291f5700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012415e6e0 0x7f012415a2c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.862+0000 7f01291f5700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01240b3440 0x7f01241603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.862+0000 7f01291f5700 1 --2- 172.21.15.112:0/3166724837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0124146c20 0x7f01241f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:33.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.862+0000 7f01291f5700 1 -- 172.21.15.112:0/3166724837 >> 172.21.15.112:0/3166724837 conn(0x7f01240b1e90 msgr2=0x7f012406d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:33.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.863+0000 7f01291f5700 1 -- 172.21.15.112:0/3166724837 shutdown_connections 2023-08-27T11:17:33.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.863+0000 7f01291f5700 1 -- 172.21.15.112:0/3166724837 wait complete. 2023-08-27T11:17:33.875 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:17:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.996+0000 7ffb864a1700 1 Processor -- start 2023-08-27T11:17:33.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.996+0000 7ffb864a1700 1 -- start start 2023-08-27T11:17:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.997+0000 7ffb864a1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb800b3440 0x7ffb8015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.997+0000 7ffb864a1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb800b5070 con 0x7ffb8015e6e0 2023-08-27T11:17:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.997+0000 7ffb864a1700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb80147000 con 0x7ffb80146c20 2023-08-27T11:17:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.997+0000 7ffb864a1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb80161740 con 0x7ffb800b3440 2023-08-27T11:17:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.997+0000 7ffb7f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb800b3440 0x7ffb8015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.997+0000 7ffb84a3e700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ffb8015e6e0 0x7ffb8006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50132/0 (socket says 172.21.15.112:50132) 2023-08-27T11:17:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.997+0000 7ffb84a3e700 1 -- 172.21.15.112:0/2222910241 learned_addr learned my addr 172.21.15.112:0/2222910241 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.997+0000 7ffb7f7fe700 1 --2- 172.21.15.112:0/2222910241 >> v2:172.21.15.112:3300/0 conn(0x7ffb800b3440 0x7ffb8015e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55748/0 (socket says 172.21.15.112:55748) 2023-08-27T11:17:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.997+0000 7ffb7f7fe700 1 -- 172.21.15.112:0/2222910241 >> v1:172.21.15.138:6789/0 conn(0x7ffb80146c20 legacy=0x7ffb80160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.997+0000 7ffb7f7fe700 1 -- 172.21.15.112:0/2222910241 >> v1:172.21.15.112:6789/0 conn(0x7ffb8015e6e0 legacy=0x7ffb8006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.997+0000 7ffb7f7fe700 1 -- 172.21.15.112:0/2222910241 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb80161a20 con 0x7ffb800b3440 2023-08-27T11:17:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.997+0000 7ffb7effd700 1 -- 172.21.15.112:0/2222910241 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408523636 0 0) 0x7ffb78001490 con 0x7ffb8015e6e0 2023-08-27T11:17:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.997+0000 7ffb7f7fe700 1 --2- 172.21.15.112:0/2222910241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb800b3440 0x7ffb8015e1a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009840 tx=0x7ffb70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3519cb7d0c9682e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.998+0000 7ffb7effd700 1 -- 172.21.15.112:0/2222910241 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb70004230 con 0x7ffb800b3440 2023-08-27T11:17:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.998+0000 7ffb7effd700 1 -- 172.21.15.112:0/2222910241 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb70004390 con 0x7ffb800b3440 2023-08-27T11:17:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.998+0000 7ffb7effd700 1 -- 172.21.15.112:0/2222910241 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb700036a0 con 0x7ffb800b3440 2023-08-27T11:17:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.998+0000 7ffb864a1700 1 -- 172.21.15.112:0/2222910241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb800b3440 msgr2=0x7ffb8015e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.998+0000 7ffb864a1700 1 --2- 172.21.15.112:0/2222910241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb800b3440 0x7ffb8015e1a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009840 tx=0x7ffb70009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.998+0000 7ffb864a1700 1 -- 172.21.15.112:0/2222910241 shutdown_connections 2023-08-27T11:17:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.998+0000 7ffb864a1700 1 --2- 172.21.15.112:0/2222910241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb800b3440 0x7ffb8015e1a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.998+0000 7ffb864a1700 1 -- 172.21.15.112:0/2222910241 >> 172.21.15.112:0/2222910241 conn(0x7ffb800b1e90 msgr2=0x7ffb800b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:34.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.998+0000 7ffb864a1700 1 -- 172.21.15.112:0/2222910241 shutdown_connections 2023-08-27T11:17:34.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.998+0000 7ffb864a1700 1 -- 172.21.15.112:0/2222910241 wait complete. 2023-08-27T11:17:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:33.999+0000 7ffb864a1700 1 Processor -- start 2023-08-27T11:17:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.000+0000 7ffb864a1700 1 -- start start 2023-08-27T11:17:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.000+0000 7ffb864a1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb800b3440 0x7ffb8014cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.000+0000 7ffb864a1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb80146c20 0x7ffb800aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.000+0000 7ffb864a1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8015e6e0 0x7ffb801f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.000+0000 7ffb864a1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8015f380 con 0x7ffb8015e6e0 2023-08-27T11:17:34.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.000+0000 7ffb864a1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8015fc20 con 0x7ffb80146c20 2023-08-27T11:17:34.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.000+0000 7ffb864a1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800af4b0 con 0x7ffb800b3440 2023-08-27T11:17:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.000+0000 7ffb84a3e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8015e6e0 0x7ffb801f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.000+0000 7ffb84a3e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8015e6e0 0x7ffb801f23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55754/0 (socket says 172.21.15.112:55754) 2023-08-27T11:17:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.000+0000 7ffb84a3e700 1 -- 172.21.15.112:0/2970868914 learned_addr learned my addr 172.21.15.112:0/2970868914 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb7f7fe700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb80146c20 0x7ffb800aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb7ffff700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb800b3440 0x7ffb8014cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb84a3e700 1 -- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb800b3440 msgr2=0x7ffb8014cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb84a3e700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb800b3440 0x7ffb8014cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb84a3e700 1 -- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb80146c20 msgr2=0x7ffb800aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb84a3e700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb80146c20 0x7ffb800aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb84a3e700 1 -- 172.21.15.112:0/2970868914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb700094a0 con 0x7ffb8015e6e0 2023-08-27T11:17:34.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb7ffff700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb800b3440 0x7ffb8014cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:34.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb84a3e700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8015e6e0 0x7ffb801f23f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800cb30 tx=0x7ffb7800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:34.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb7cff9700 1 -- 172.21.15.112:0/2970868914 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7800ab50 con 0x7ffb8015e6e0 2023-08-27T11:17:34.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb7cff9700 1 -- 172.21.15.112:0/2970868914 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7800acb0 con 0x7ffb8015e6e0 2023-08-27T11:17:34.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb864a1700 1 -- 172.21.15.112:0/2970868914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb800af750 con 0x7ffb8015e6e0 2023-08-27T11:17:34.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb7cff9700 1 -- 172.21.15.112:0/2970868914 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78007310 con 0x7ffb8015e6e0 2023-08-27T11:17:34.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.001+0000 7ffb864a1700 1 -- 172.21.15.112:0/2970868914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8015f8b0 con 0x7ffb8015e6e0 2023-08-27T11:17:34.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.002+0000 7ffb864a1700 1 -- 172.21.15.112:0/2970868914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb80067e90 con 0x7ffb8015e6e0 2023-08-27T11:17:34.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.002+0000 7ffb7cff9700 1 -- 172.21.15.112:0/2970868914 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffb78007470 con 0x7ffb8015e6e0 2023-08-27T11:17:34.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.003+0000 7ffb7cff9700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb6c06a0e0 0x7ffb6c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.003+0000 7ffb7cff9700 1 -- 172.21.15.112:0/2970868914 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb78088160 con 0x7ffb8015e6e0 2023-08-27T11:17:34.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.003+0000 7ffb7ffff700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb6c06a0e0 0x7ffb6c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.003+0000 7ffb7ffff700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb6c06a0e0 0x7ffb6c06c5a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffb74003f40 tx=0x7ffb74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:34.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.007+0000 7ffb7cff9700 1 -- 172.21.15.112:0/2970868914 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffb78053850 con 0x7ffb8015e6e0 2023-08-27T11:17:34.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.195+0000 7ffb864a1700 1 -- 172.21.15.112:0/2970868914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb800b5070 con 0x7ffb8015e6e0 2023-08-27T11:17:34.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.196+0000 7ffb7cff9700 1 -- 172.21.15.112:0/2970868914 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb800b5070 con 0x7ffb8015e6e0 2023-08-27T11:17:34.197 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:17:34.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.198+0000 7ffb864a1700 1 -- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb6c06a0e0 msgr2=0x7ffb6c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.198+0000 7ffb864a1700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb6c06a0e0 0x7ffb6c06c5a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffb74003f40 tx=0x7ffb74009040 comp rx=0 tx=0).stop 2023-08-27T11:17:34.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.198+0000 7ffb864a1700 1 -- 172.21.15.112:0/2970868914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8015e6e0 msgr2=0x7ffb801f23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.198+0000 7ffb864a1700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8015e6e0 0x7ffb801f23f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800cb30 tx=0x7ffb7800cef0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.198+0000 7ffb864a1700 1 -- 172.21.15.112:0/2970868914 shutdown_connections 2023-08-27T11:17:34.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.198+0000 7ffb864a1700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb6c06a0e0 0x7ffb6c06c5a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.198+0000 7ffb864a1700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8015e6e0 0x7ffb801f23f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.198+0000 7ffb864a1700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb800b3440 0x7ffb8014cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.198+0000 7ffb864a1700 1 --2- 172.21.15.112:0/2970868914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb80146c20 0x7ffb800aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.198+0000 7ffb864a1700 1 -- 172.21.15.112:0/2970868914 >> 172.21.15.112:0/2970868914 conn(0x7ffb800b1e90 msgr2=0x7ffb8006daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:34.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.198+0000 7ffb864a1700 1 -- 172.21.15.112:0/2970868914 shutdown_connections 2023-08-27T11:17:34.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.198+0000 7ffb864a1700 1 -- 172.21.15.112:0/2970868914 wait complete. 2023-08-27T11:17:34.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:34.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.332+0000 7faf6e42e700 1 Processor -- start 2023-08-27T11:17:34.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.332+0000 7faf6e42e700 1 -- start start 2023-08-27T11:17:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf6e42e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faf68151060 0x7faf6815e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf6e42e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faf6815e990 0x7faf6806adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf6e42e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf6806b710 con 0x7faf6806b2f0 2023-08-27T11:17:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf6e42e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf680b02f0 con 0x7faf68151060 2023-08-27T11:17:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf6e42e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf680b05d0 con 0x7faf6815e990 2023-08-27T11:17:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf67fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faf68151060 0x7faf6815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf67fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faf68151060 0x7faf6815e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55758/0 (socket says 172.21.15.112:55758) 2023-08-27T11:17:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf67fff700 1 -- 172.21.15.112:0/2677477266 learned_addr learned my addr 172.21.15.112:0/2677477266 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf677fe700 1 --2- 172.21.15.112:0/2677477266 >> v2:172.21.15.138:3300/0 conn(0x7faf6815e990 0x7faf6806adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf67fff700 1 -- 172.21.15.112:0/2677477266 >> v2:172.21.15.138:3300/0 conn(0x7faf6815e990 msgr2=0x7faf6806adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf67fff700 1 --2- 172.21.15.112:0/2677477266 >> v2:172.21.15.138:3300/0 conn(0x7faf6815e990 0x7faf6806adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf67fff700 1 -- 172.21.15.112:0/2677477266 >> v1:172.21.15.138:6789/0 conn(0x7faf6806b2f0 legacy=0x7faf6806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf67fff700 1 -- 172.21.15.112:0/2677477266 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf680b08b0 con 0x7faf68151060 2023-08-27T11:17:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.333+0000 7faf67fff700 1 --2- 172.21.15.112:0/2677477266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf68151060 0x7faf6815e450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf6806ccb0 tx=0x7faf5c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655bab50b52b66fb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.334+0000 7faf66ffd700 1 -- 172.21.15.112:0/2677477266 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00b870 con 0x7faf68151060 2023-08-27T11:17:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.334+0000 7faf66ffd700 1 -- 172.21.15.112:0/2677477266 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c00b9d0 con 0x7faf68151060 2023-08-27T11:17:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.334+0000 7faf66ffd700 1 -- 172.21.15.112:0/2677477266 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00bd60 con 0x7faf68151060 2023-08-27T11:17:34.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.334+0000 7faf6e42e700 1 -- 172.21.15.112:0/2677477266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf68151060 msgr2=0x7faf6815e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.334+0000 7faf6e42e700 1 --2- 172.21.15.112:0/2677477266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf68151060 0x7faf6815e450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf6806ccb0 tx=0x7faf5c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.334+0000 7faf6e42e700 1 -- 172.21.15.112:0/2677477266 shutdown_connections 2023-08-27T11:17:34.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.334+0000 7faf6e42e700 1 --2- 172.21.15.112:0/2677477266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf68151060 0x7faf6815e450 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.334+0000 7faf6e42e700 1 --2- 172.21.15.112:0/2677477266 >> v2:172.21.15.138:3300/0 conn(0x7faf6815e990 0x7faf6806adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.334+0000 7faf6e42e700 1 -- 172.21.15.112:0/2677477266 >> 172.21.15.112:0/2677477266 conn(0x7faf680d0000 msgr2=0x7faf680d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:34.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.334+0000 7faf6e42e700 1 -- 172.21.15.112:0/2677477266 shutdown_connections 2023-08-27T11:17:34.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.334+0000 7faf6e42e700 1 -- 172.21.15.112:0/2677477266 wait complete. 2023-08-27T11:17:34.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.335+0000 7faf6e42e700 1 Processor -- start 2023-08-27T11:17:34.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.336+0000 7faf6e42e700 1 -- start start 2023-08-27T11:17:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.336+0000 7faf6e42e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf6806b2f0 0x7faf680c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.336+0000 7faf6e42e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf68151060 0x7faf681f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.336+0000 7faf6e42e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf6815e990 0x7faf6815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.336+0000 7faf6e42e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681f4210 con 0x7faf68151060 2023-08-27T11:17:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.336+0000 7faf6e42e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf680c33b0 con 0x7faf6806b2f0 2023-08-27T11:17:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.336+0000 7faf6e42e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf681f3700 con 0x7faf6815e990 2023-08-27T11:17:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.336+0000 7faf677fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf68151060 0x7faf681f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.336+0000 7faf677fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf68151060 0x7faf681f31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55766/0 (socket says 172.21.15.112:55766) 2023-08-27T11:17:34.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.336+0000 7faf677fe700 1 -- 172.21.15.112:0/4153551504 learned_addr learned my addr 172.21.15.112:0/4153551504 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:34.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf6c9cb700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf6815e990 0x7faf6815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf67fff700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf6806b2f0 0x7faf680c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf677fe700 1 -- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf6815e990 msgr2=0x7faf6815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf677fe700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf6815e990 0x7faf6815b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf677fe700 1 -- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf6806b2f0 msgr2=0x7faf680c2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf677fe700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf6806b2f0 0x7faf680c2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf677fe700 1 -- 172.21.15.112:0/4153551504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c009550 con 0x7faf68151060 2023-08-27T11:17:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf67fff700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf6806b2f0 0x7faf680c2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf677fe700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf68151060 0x7faf681f31c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf5800e810 tx=0x7faf5800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf64ff9700 1 -- 172.21.15.112:0/4153551504 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58013610 con 0x7faf68151060 2023-08-27T11:17:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf64ff9700 1 -- 172.21.15.112:0/4153551504 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf58013770 con 0x7faf68151060 2023-08-27T11:17:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf6e42e700 1 -- 172.21.15.112:0/4153551504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6815d800 con 0x7faf68151060 2023-08-27T11:17:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf64ff9700 1 -- 172.21.15.112:0/4153551504 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58013b20 con 0x7faf68151060 2023-08-27T11:17:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.337+0000 7faf6e42e700 1 -- 172.21.15.112:0/4153551504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf681f3c50 con 0x7faf68151060 2023-08-27T11:17:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.338+0000 7faf6e42e700 1 -- 172.21.15.112:0/4153551504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf68067a20 con 0x7faf68151060 2023-08-27T11:17:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.338+0000 7faf64ff9700 1 -- 172.21.15.112:0/4153551504 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faf58013c80 con 0x7faf68151060 2023-08-27T11:17:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.339+0000 7faf64ff9700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf5406a0e0 0x7faf5406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.339+0000 7faf64ff9700 1 -- 172.21.15.112:0/4153551504 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf58088280 con 0x7faf68151060 2023-08-27T11:17:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.339+0000 7faf67fff700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf5406a0e0 0x7faf5406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.339+0000 7faf67fff700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf5406a0e0 0x7faf5406c5a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009130 tx=0x7faf5c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.343+0000 7faf64ff9700 1 -- 172.21.15.112:0/4153551504 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7faf58057400 con 0x7faf68151060 2023-08-27T11:17:34.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.533+0000 7faf6e42e700 1 -- 172.21.15.112:0/4153551504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf6815bd30 con 0x7faf68151060 2023-08-27T11:17:34.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.535+0000 7faf64ff9700 1 -- 172.21.15.112:0/4153551504 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7faf58056f90 con 0x7faf68151060 2023-08-27T11:17:34.536 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:34.537 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:34.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.537+0000 7faf6e42e700 1 -- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf5406a0e0 msgr2=0x7faf5406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.537+0000 7faf6e42e700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf5406a0e0 0x7faf5406c5a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009130 tx=0x7faf5c008030 comp rx=0 tx=0).stop 2023-08-27T11:17:34.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.537+0000 7faf6e42e700 1 -- 172.21.15.112:0/4153551504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf68151060 msgr2=0x7faf681f31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.537+0000 7faf6e42e700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf68151060 0x7faf681f31c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf5800e810 tx=0x7faf5800ebd0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.537+0000 7faf6e42e700 1 -- 172.21.15.112:0/4153551504 shutdown_connections 2023-08-27T11:17:34.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.537+0000 7faf6e42e700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf5406a0e0 0x7faf5406c5a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.537+0000 7faf6e42e700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf6815e990 0x7faf6815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.537+0000 7faf6e42e700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf6806b2f0 0x7faf680c2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.537+0000 7faf6e42e700 1 --2- 172.21.15.112:0/4153551504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf68151060 0x7faf681f31c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.537+0000 7faf6e42e700 1 -- 172.21.15.112:0/4153551504 >> 172.21.15.112:0/4153551504 conn(0x7faf680d0000 msgr2=0x7faf68071f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:34.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.537+0000 7faf6e42e700 1 -- 172.21.15.112:0/4153551504 shutdown_connections 2023-08-27T11:17:34.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.537+0000 7faf6e42e700 1 -- 172.21.15.112:0/4153551504 wait complete. 2023-08-27T11:17:34.550 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:17:02.484063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:17:03.722976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:17:34.551 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:17:34.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.672+0000 7f238ee51700 1 Processor -- start 2023-08-27T11:17:34.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.672+0000 7f238ee51700 1 -- start start 2023-08-27T11:17:34.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.673+0000 7f238ee51700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23880b3440 0x7f238815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.673+0000 7f238ee51700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23880b5070 con 0x7f238815e6e0 2023-08-27T11:17:34.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.673+0000 7f238ee51700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2388147000 con 0x7f2388146c20 2023-08-27T11:17:34.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.673+0000 7f238ee51700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2388161740 con 0x7f23880b3440 2023-08-27T11:17:34.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.673+0000 7f238d3ee700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f238815e6e0 0x7f238806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50154/0 (socket says 172.21.15.112:50154) 2023-08-27T11:17:34.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.673+0000 7f238d3ee700 1 -- 172.21.15.112:0/3174479081 learned_addr learned my addr 172.21.15.112:0/3174479081 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:34.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.674+0000 7f2387fff700 1 --2- 172.21.15.112:0/3174479081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23880b3440 0x7f238815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.674+0000 7f23877fe700 1 -- 172.21.15.112:0/3174479081 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703703085 0 0) 0x7f2380001490 con 0x7f238815e6e0 2023-08-27T11:17:34.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.674+0000 7f23877fe700 1 -- 172.21.15.112:0/3174479081 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23740035b0 con 0x7f238815e6e0 2023-08-27T11:17:34.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.674+0000 7f23877fe700 1 -- 172.21.15.112:0/3174479081 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313672082 0 0) 0x7f2388147000 con 0x7f2388146c20 2023-08-27T11:17:34.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.674+0000 7f23877fe700 1 -- 172.21.15.112:0/3174479081 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2380001490 con 0x7f2388146c20 2023-08-27T11:17:34.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.674+0000 7f2387fff700 1 -- 172.21.15.112:0/3174479081 >> v1:172.21.15.138:6789/0 conn(0x7f2388146c20 legacy=0x7f2388160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.674+0000 7f2387fff700 1 -- 172.21.15.112:0/3174479081 >> v1:172.21.15.112:6789/0 conn(0x7f238815e6e0 legacy=0x7f238806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.674+0000 7f2387fff700 1 -- 172.21.15.112:0/3174479081 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2388161a20 con 0x7f23880b3440 2023-08-27T11:17:34.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.674+0000 7f2387fff700 1 --2- 172.21.15.112:0/3174479081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23880b3440 0x7f238815e1a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2378009870 tx=0x7f2378009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa30f077bd3564ce server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:34.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.674+0000 7f23877fe700 1 -- 172.21.15.112:0/3174479081 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378011830 con 0x7f23880b3440 2023-08-27T11:17:34.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.674+0000 7f23877fe700 1 -- 172.21.15.112:0/3174479081 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2378011990 con 0x7f23880b3440 2023-08-27T11:17:34.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.674+0000 7f23877fe700 1 -- 172.21.15.112:0/3174479081 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378011d00 con 0x7f23880b3440 2023-08-27T11:17:34.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.675+0000 7f238ee51700 1 -- 172.21.15.112:0/3174479081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23880b3440 msgr2=0x7f238815e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.675+0000 7f238ee51700 1 --2- 172.21.15.112:0/3174479081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23880b3440 0x7f238815e1a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2378009870 tx=0x7f2378009b80 comp rx=0 tx=0).stop 2023-08-27T11:17:34.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.675+0000 7f238ee51700 1 -- 172.21.15.112:0/3174479081 shutdown_connections 2023-08-27T11:17:34.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.675+0000 7f238ee51700 1 --2- 172.21.15.112:0/3174479081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23880b3440 0x7f238815e1a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.675+0000 7f238ee51700 1 -- 172.21.15.112:0/3174479081 >> 172.21.15.112:0/3174479081 conn(0x7f23880b1e90 msgr2=0x7f23880b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:34.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.675+0000 7f238ee51700 1 -- 172.21.15.112:0/3174479081 shutdown_connections 2023-08-27T11:17:34.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.675+0000 7f238ee51700 1 -- 172.21.15.112:0/3174479081 wait complete. 2023-08-27T11:17:34.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.676+0000 7f238ee51700 1 Processor -- start 2023-08-27T11:17:34.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.676+0000 7f238ee51700 1 -- start start 2023-08-27T11:17:34.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238ee51700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23880b3440 0x7f23880af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238ee51700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2388146c20 0x7f238813ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238ee51700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f238815e6e0 0x7f23880add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238ee51700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2388140540 con 0x7f23880b3440 2023-08-27T11:17:34.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238ee51700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238813ff80 con 0x7f2388146c20 2023-08-27T11:17:34.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238ee51700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2388140260 con 0x7f238815e6e0 2023-08-27T11:17:34.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238cbed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23880b3440 0x7f23880af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238cbed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23880b3440 0x7f23880af070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55776/0 (socket says 172.21.15.112:55776) 2023-08-27T11:17:34.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238cbed700 1 -- 172.21.15.112:0/3293342310 learned_addr learned my addr 172.21.15.112:0/3293342310 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:34.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f2387fff700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2388146c20 0x7f238813ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238d3ee700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f238815e6e0 0x7f23880add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238cbed700 1 -- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f238815e6e0 msgr2=0x7f23880add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238cbed700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f238815e6e0 0x7f23880add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238cbed700 1 -- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2388146c20 msgr2=0x7f238813ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238cbed700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2388146c20 0x7f238813ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f238cbed700 1 -- 172.21.15.112:0/3293342310 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23780094a0 con 0x7f23880b3440 2023-08-27T11:17:34.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.677+0000 7f2387fff700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2388146c20 0x7f238813ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:34.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.678+0000 7f238cbed700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23880b3440 0x7f23880af070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f237000c500 tx=0x7f237000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:34.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.678+0000 7f23857fa700 1 -- 172.21.15.112:0/3293342310 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23700052d0 con 0x7f23880b3440 2023-08-27T11:17:34.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.678+0000 7f23857fa700 1 -- 172.21.15.112:0/3293342310 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2370005430 con 0x7f23880b3440 2023-08-27T11:17:34.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.678+0000 7f238ee51700 1 -- 172.21.15.112:0/3293342310 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23880ad560 con 0x7f23880b3440 2023-08-27T11:17:34.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.678+0000 7f23857fa700 1 -- 172.21.15.112:0/3293342310 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23700038c0 con 0x7f23880b3440 2023-08-27T11:17:34.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.678+0000 7f238ee51700 1 -- 172.21.15.112:0/3293342310 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238813b1a0 con 0x7f23880b3440 2023-08-27T11:17:34.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.679+0000 7f238ee51700 1 -- 172.21.15.112:0/3293342310 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2388067e90 con 0x7f23880b3440 2023-08-27T11:17:34.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.679+0000 7f23857fa700 1 -- 172.21.15.112:0/3293342310 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2370003a20 con 0x7f23880b3440 2023-08-27T11:17:34.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.679+0000 7f23857fa700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f236806a0e0 0x7f236806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:34.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.680+0000 7f23857fa700 1 -- 172.21.15.112:0/3293342310 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2370087b40 con 0x7f23880b3440 2023-08-27T11:17:34.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.680+0000 7f2387fff700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f236806a0e0 0x7f236806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:34.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.680+0000 7f2387fff700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f236806a0e0 0x7f236806c5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2388141a00 tx=0x7f2378008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:34.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.684+0000 7f23857fa700 1 -- 172.21.15.112:0/3293342310 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f23700531a0 con 0x7f23880b3440 2023-08-27T11:17:34.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.872+0000 7f238ee51700 1 -- 172.21.15.112:0/3293342310 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2388067a20 con 0x7f23880b3440 2023-08-27T11:17:34.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.873+0000 7f23857fa700 1 -- 172.21.15.112:0/3293342310 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2370017020 con 0x7f23880b3440 2023-08-27T11:17:34.874 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:34.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.874+0000 7f238ee51700 1 -- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f236806a0e0 msgr2=0x7f236806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.874+0000 7f238ee51700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f236806a0e0 0x7f236806c5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2388141a00 tx=0x7f2378008030 comp rx=0 tx=0).stop 2023-08-27T11:17:34.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.874+0000 7f238ee51700 1 -- 172.21.15.112:0/3293342310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23880b3440 msgr2=0x7f23880af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:34.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.874+0000 7f238ee51700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23880b3440 0x7f23880af070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f237000c500 tx=0x7f237000c8c0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.874+0000 7f238ee51700 1 -- 172.21.15.112:0/3293342310 shutdown_connections 2023-08-27T11:17:34.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.874+0000 7f238ee51700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f236806a0e0 0x7f236806c5a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.874+0000 7f238ee51700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f238815e6e0 0x7f23880add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.875+0000 7f238ee51700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23880b3440 0x7f23880af070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.875+0000 7f238ee51700 1 --2- 172.21.15.112:0/3293342310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2388146c20 0x7f238813ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:34.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.875+0000 7f238ee51700 1 -- 172.21.15.112:0/3293342310 >> 172.21.15.112:0/3293342310 conn(0x7f23880b1e90 msgr2=0x7f23880b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:34.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.875+0000 7f238ee51700 1 -- 172.21.15.112:0/3293342310 shutdown_connections 2023-08-27T11:17:34.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:34.875+0000 7f238ee51700 1 -- 172.21.15.112:0/3293342310 wait complete. 2023-08-27T11:17:34.888 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.010+0000 7fde3a2ea700 1 Processor -- start 2023-08-27T11:17:35.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.010+0000 7fde3a2ea700 1 -- start start 2023-08-27T11:17:35.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde3a2ea700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde340b3de0 0x7fde3406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde3a2ea700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fde3406c510 0x7fde340cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde3a2ea700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde340cef20 con 0x7fde34144b80 2023-08-27T11:17:35.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde3a2ea700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde340b4b40 con 0x7fde3406c510 2023-08-27T11:17:35.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde3a2ea700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3406c940 con 0x7fde340b3de0 2023-08-27T11:17:35.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde33fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fde34144b80 0x7fde340b6c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50166/0 (socket says 172.21.15.112:50166) 2023-08-27T11:17:35.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde33fff700 1 -- 172.21.15.112:0/2951121703 learned_addr learned my addr 172.21.15.112:0/2951121703 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:35.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde337fe700 1 --2- 172.21.15.112:0/2951121703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde340b3de0 0x7fde3406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde38887700 1 --2- 172.21.15.112:0/2951121703 >> v2:172.21.15.138:3300/0 conn(0x7fde3406c510 0x7fde340cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde32ffd700 1 -- 172.21.15.112:0/2951121703 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055206801 0 0) 0x7fde340cef20 con 0x7fde34144b80 2023-08-27T11:17:35.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde32ffd700 1 -- 172.21.15.112:0/2951121703 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde1c0035b0 con 0x7fde34144b80 2023-08-27T11:17:35.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde38887700 1 -- 172.21.15.112:0/2951121703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde340b3de0 msgr2=0x7fde3406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:35.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde32ffd700 1 -- 172.21.15.112:0/2951121703 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1743697099 0 0) 0x7fde1c0035b0 con 0x7fde34144b80 2023-08-27T11:17:35.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde38887700 1 --2- 172.21.15.112:0/2951121703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde340b3de0 0x7fde3406bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde38887700 1 -- 172.21.15.112:0/2951121703 >> v1:172.21.15.112:6789/0 conn(0x7fde34144b80 legacy=0x7fde340b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.011+0000 7fde38887700 1 -- 172.21.15.112:0/2951121703 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde340b7310 con 0x7fde3406c510 2023-08-27T11:17:35.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.012+0000 7fde38887700 1 --2- 172.21.15.112:0/2951121703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde3406c510 0x7fde340cf3b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009d90 tx=0x7fde2c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe824ef0c013fbdd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:35.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.012+0000 7fde32ffd700 1 -- 172.21.15.112:0/2951121703 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00e070 con 0x7fde3406c510 2023-08-27T11:17:35.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.012+0000 7fde32ffd700 1 -- 172.21.15.112:0/2951121703 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c003a20 con 0x7fde3406c510 2023-08-27T11:17:35.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.012+0000 7fde32ffd700 1 -- 172.21.15.112:0/2951121703 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28001670 con 0x7fde3406c510 2023-08-27T11:17:35.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.012+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2951121703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde3406c510 msgr2=0x7fde340cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.012+0000 7fde3a2ea700 1 --2- 172.21.15.112:0/2951121703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde3406c510 0x7fde340cf3b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009d90 tx=0x7fde2c00b330 comp rx=0 tx=0).stop 2023-08-27T11:17:35.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.012+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2951121703 shutdown_connections 2023-08-27T11:17:35.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.012+0000 7fde3a2ea700 1 --2- 172.21.15.112:0/2951121703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde3406c510 0x7fde340cf3b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.012+0000 7fde3a2ea700 1 --2- 172.21.15.112:0/2951121703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde340b3de0 0x7fde3406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.012+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2951121703 >> 172.21.15.112:0/2951121703 conn(0x7fde340cdc50 msgr2=0x7fde340ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:35.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.012+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2951121703 shutdown_connections 2023-08-27T11:17:35.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.012+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2951121703 wait complete. 2023-08-27T11:17:35.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.013+0000 7fde3a2ea700 1 Processor -- start 2023-08-27T11:17:35.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.014+0000 7fde3a2ea700 1 -- start start 2023-08-27T11:17:35.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.014+0000 7fde3a2ea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3406c510 0x7fde340abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.014+0000 7fde3a2ea700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde340b3de0 0x7fde340a8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.014+0000 7fde3a2ea700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde34144b80 0x7fde34143ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.014+0000 7fde3a2ea700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde340a93c0 con 0x7fde3406c510 2023-08-27T11:17:35.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.014+0000 7fde3a2ea700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde340a96a0 con 0x7fde340b3de0 2023-08-27T11:17:35.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.014+0000 7fde3a2ea700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde340a9980 con 0x7fde34144b80 2023-08-27T11:17:35.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.014+0000 7fde33fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3406c510 0x7fde340abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.014+0000 7fde33fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3406c510 0x7fde340abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55790/0 (socket says 172.21.15.112:55790) 2023-08-27T11:17:35.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.014+0000 7fde33fff700 1 -- 172.21.15.112:0/2072867596 learned_addr learned my addr 172.21.15.112:0/2072867596 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:35.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.015+0000 7fde38887700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde34144b80 0x7fde34143ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.015+0000 7fde337fe700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde340b3de0 0x7fde340a8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.015+0000 7fde33fff700 1 -- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde34144b80 msgr2=0x7fde34143ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.015+0000 7fde33fff700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde34144b80 0x7fde34143ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.015+0000 7fde33fff700 1 -- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde340b3de0 msgr2=0x7fde340a8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.015+0000 7fde33fff700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde340b3de0 0x7fde340a8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.015+0000 7fde33fff700 1 -- 172.21.15.112:0/2072867596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c009a70 con 0x7fde3406c510 2023-08-27T11:17:35.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.015+0000 7fde33fff700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3406c510 0x7fde340abcc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fde2800c520 tx=0x7fde2800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:35.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.015+0000 7fde30ff9700 1 -- 172.21.15.112:0/2072867596 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28004c80 con 0x7fde3406c510 2023-08-27T11:17:35.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.015+0000 7fde30ff9700 1 -- 172.21.15.112:0/2072867596 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde28004e20 con 0x7fde3406c510 2023-08-27T11:17:35.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.015+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2072867596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde340a9c60 con 0x7fde3406c510 2023-08-27T11:17:35.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.015+0000 7fde30ff9700 1 -- 172.21.15.112:0/2072867596 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28007310 con 0x7fde3406c510 2023-08-27T11:17:35.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.015+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2072867596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde340a90d0 con 0x7fde3406c510 2023-08-27T11:17:35.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.016+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2072867596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde340aa180 con 0x7fde3406c510 2023-08-27T11:17:35.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.016+0000 7fde30ff9700 1 -- 172.21.15.112:0/2072867596 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fde28007470 con 0x7fde3406c510 2023-08-27T11:17:35.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.017+0000 7fde30ff9700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fde2006a0e0 0x7fde2006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.017+0000 7fde30ff9700 1 -- 172.21.15.112:0/2072867596 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fde280882d0 con 0x7fde3406c510 2023-08-27T11:17:35.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.017+0000 7fde337fe700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fde2006a0e0 0x7fde2006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.017+0000 7fde337fe700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fde2006a0e0 0x7fde2006c5a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fde340ccc70 tx=0x7fde24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:35.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.021+0000 7fde30ff9700 1 -- 172.21.15.112:0/2072867596 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fde340aa180 con 0x7fde3406c510 2023-08-27T11:17:35.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.210+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2072867596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde340aa180 con 0x7fde3406c510 2023-08-27T11:17:35.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.211+0000 7fde30ff9700 1 -- 172.21.15.112:0/2072867596 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde340aa180 con 0x7fde3406c510 2023-08-27T11:17:35.212 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:35.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.213+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fde2006a0e0 msgr2=0x7fde2006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.213+0000 7fde3a2ea700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fde2006a0e0 0x7fde2006c5a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fde340ccc70 tx=0x7fde24008040 comp rx=0 tx=0).stop 2023-08-27T11:17:35.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.213+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2072867596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3406c510 msgr2=0x7fde340abcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.213+0000 7fde3a2ea700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3406c510 0x7fde340abcc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fde2800c520 tx=0x7fde2800c8e0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.213+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2072867596 shutdown_connections 2023-08-27T11:17:35.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.213+0000 7fde3a2ea700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fde2006a0e0 0x7fde2006c5a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.213+0000 7fde3a2ea700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde34144b80 0x7fde34143ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.213+0000 7fde3a2ea700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3406c510 0x7fde340abcc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.213+0000 7fde3a2ea700 1 --2- 172.21.15.112:0/2072867596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde340b3de0 0x7fde340a8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.213+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2072867596 >> 172.21.15.112:0/2072867596 conn(0x7fde340cdc50 msgr2=0x7fde340b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:35.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.213+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2072867596 shutdown_connections 2023-08-27T11:17:35.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.213+0000 7fde3a2ea700 1 -- 172.21.15.112:0/2072867596 wait complete. 2023-08-27T11:17:35.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:17:35.297 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:17:35.300 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:17:35.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.351+0000 7f9d9efdd700 1 Processor -- start 2023-08-27T11:17:35.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.351+0000 7f9d9efdd700 1 -- start start 2023-08-27T11:17:35.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d9efdd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d980b3440 0x7f9d9815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d9efdd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9d9815e6e0 0x7f9d9806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d9efdd700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d980b5070 con 0x7f9d98146c20 2023-08-27T11:17:35.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d9efdd700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d98147000 con 0x7f9d9815e6e0 2023-08-27T11:17:35.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d9efdd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d98161740 con 0x7f9d980b3440 2023-08-27T11:17:35.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d9d57a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9d9815e6e0 0x7f9d9806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d9cd79700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9d98146c20 0x7f9d98160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:57704/0 (socket says 172.21.15.112:57704) 2023-08-27T11:17:35.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d97fff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d980b3440 0x7f9d9815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d9cd79700 1 -- 172.21.15.112:0/690753006 learned_addr learned my addr 172.21.15.112:0/690753006 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:35.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d9d57a700 1 -- 172.21.15.112:0/690753006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d980b3440 msgr2=0x7f9d9815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:35.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d9d57a700 1 --2- 172.21.15.112:0/690753006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d980b3440 0x7f9d9815e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d9d57a700 1 -- 172.21.15.112:0/690753006 >> v1:172.21.15.138:6789/0 conn(0x7f9d98146c20 legacy=0x7f9d98160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d9d57a700 1 -- 172.21.15.112:0/690753006 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d98161a20 con 0x7f9d9815e6e0 2023-08-27T11:17:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d977fe700 1 -- 172.21.15.112:0/690753006 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82326180 0 0) 0x7f9d8c001110 con 0x7f9d98146c20 2023-08-27T11:17:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.352+0000 7f9d97fff700 1 --2- 172.21.15.112:0/690753006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d980b3440 0x7f9d9815e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.353+0000 7f9d9d57a700 1 --2- 172.21.15.112:0/690753006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d9815e6e0 0x7f9d9806f790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009d90 tx=0x7f9d9000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=693127228f3dab9e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.353+0000 7f9d977fe700 1 -- 172.21.15.112:0/690753006 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000e070 con 0x7f9d9815e6e0 2023-08-27T11:17:35.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.353+0000 7f9d977fe700 1 -- 172.21.15.112:0/690753006 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9000bb20 con 0x7f9d9815e6e0 2023-08-27T11:17:35.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.353+0000 7f9d977fe700 1 -- 172.21.15.112:0/690753006 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90012430 con 0x7f9d9815e6e0 2023-08-27T11:17:35.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.353+0000 7f9d9efdd700 1 -- 172.21.15.112:0/690753006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d9815e6e0 msgr2=0x7f9d9806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.353+0000 7f9d9efdd700 1 --2- 172.21.15.112:0/690753006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d9815e6e0 0x7f9d9806f790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009d90 tx=0x7f9d9000b330 comp rx=0 tx=0).stop 2023-08-27T11:17:35.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.353+0000 7f9d9efdd700 1 -- 172.21.15.112:0/690753006 shutdown_connections 2023-08-27T11:17:35.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.353+0000 7f9d9efdd700 1 --2- 172.21.15.112:0/690753006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d9815e6e0 0x7f9d9806f790 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.353+0000 7f9d9efdd700 1 --2- 172.21.15.112:0/690753006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d980b3440 0x7f9d9815e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.353+0000 7f9d9efdd700 1 -- 172.21.15.112:0/690753006 >> 172.21.15.112:0/690753006 conn(0x7f9d980b1e90 msgr2=0x7f9d980b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:35.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.353+0000 7f9d9efdd700 1 -- 172.21.15.112:0/690753006 shutdown_connections 2023-08-27T11:17:35.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.353+0000 7f9d9efdd700 1 -- 172.21.15.112:0/690753006 wait complete. 2023-08-27T11:17:35.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.355+0000 7f9d9efdd700 1 Processor -- start 2023-08-27T11:17:35.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.355+0000 7f9d9efdd700 1 -- start start 2023-08-27T11:17:35.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.355+0000 7f9d9efdd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d980b3440 0x7f9d98160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.355+0000 7f9d9efdd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d98146c20 0x7f9d98145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.355+0000 7f9d9efdd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d9815e6e0 0x7f9d9813eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.355+0000 7f9d9efdd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9815f4a0 con 0x7f9d9815e6e0 2023-08-27T11:17:35.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.355+0000 7f9d9efdd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9815f780 con 0x7f9d98146c20 2023-08-27T11:17:35.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.355+0000 7f9d9efdd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9815fa60 con 0x7f9d980b3440 2023-08-27T11:17:35.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d9d57a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d9815e6e0 0x7f9d9813eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d9d57a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d9815e6e0 0x7f9d9813eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55806/0 (socket says 172.21.15.112:55806) 2023-08-27T11:17:35.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d9d57a700 1 -- 172.21.15.112:0/1788743356 learned_addr learned my addr 172.21.15.112:0/1788743356 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:35.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d9cd79700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d980b3440 0x7f9d98160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d97fff700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d98146c20 0x7f9d98145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d9d57a700 1 -- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d980b3440 msgr2=0x7f9d98160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d9d57a700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d980b3440 0x7f9d98160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d9d57a700 1 -- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d98146c20 msgr2=0x7f9d98145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d9d57a700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d98146c20 0x7f9d98145ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d9d57a700 1 -- 172.21.15.112:0/1788743356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d840094a0 con 0x7f9d9815e6e0 2023-08-27T11:17:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d9d57a700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d9815e6e0 0x7f9d9813eb30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d9806e8e0 tx=0x7f9d90004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d957fa700 1 -- 172.21.15.112:0/1788743356 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90012430 con 0x7f9d9815e6e0 2023-08-27T11:17:35.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d957fa700 1 -- 172.21.15.112:0/1788743356 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d90020070 con 0x7f9d9815e6e0 2023-08-27T11:17:35.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d9efdd700 1 -- 172.21.15.112:0/1788743356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d90009a70 con 0x7f9d9815e6e0 2023-08-27T11:17:35.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d957fa700 1 -- 172.21.15.112:0/1788743356 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90004280 con 0x7f9d9815e6e0 2023-08-27T11:17:35.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.356+0000 7f9d9efdd700 1 -- 172.21.15.112:0/1788743356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d98160020 con 0x7f9d9815e6e0 2023-08-27T11:17:35.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.357+0000 7f9d9efdd700 1 -- 172.21.15.112:0/1788743356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d98067e90 con 0x7f9d9815e6e0 2023-08-27T11:17:35.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.358+0000 7f9d957fa700 1 -- 172.21.15.112:0/1788743356 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9d900043e0 con 0x7f9d9815e6e0 2023-08-27T11:17:35.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.358+0000 7f9d957fa700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d7c06a0e0 0x7f9d7c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.358+0000 7f9d957fa700 1 -- 172.21.15.112:0/1788743356 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9d90022070 con 0x7f9d9815e6e0 2023-08-27T11:17:35.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.359+0000 7f9d9cd79700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d7c06a0e0 0x7f9d7c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.359+0000 7f9d9cd79700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d7c06a0e0 0x7f9d7c06c5a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c000f60 tx=0x7f9d8c007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:35.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.362+0000 7f9d957fa700 1 -- 172.21.15.112:0/1788743356 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9d90056070 con 0x7f9d9815e6e0 2023-08-27T11:17:35.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.552+0000 7f9d9efdd700 1 -- 172.21.15.112:0/1788743356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d9815e500 con 0x7f9d9815e6e0 2023-08-27T11:17:35.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.552+0000 7f9d957fa700 1 -- 172.21.15.112:0/1788743356 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d90059690 con 0x7f9d9815e6e0 2023-08-27T11:17:35.554 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:17:35.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.554+0000 7f9d9efdd700 1 -- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d7c06a0e0 msgr2=0x7f9d7c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.554+0000 7f9d9efdd700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d7c06a0e0 0x7f9d7c06c5a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c000f60 tx=0x7f9d8c007a40 comp rx=0 tx=0).stop 2023-08-27T11:17:35.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.554+0000 7f9d9efdd700 1 -- 172.21.15.112:0/1788743356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d9815e6e0 msgr2=0x7f9d9813eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.554+0000 7f9d9efdd700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d9815e6e0 0x7f9d9813eb30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d9806e8e0 tx=0x7f9d90004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.554+0000 7f9d9efdd700 1 -- 172.21.15.112:0/1788743356 shutdown_connections 2023-08-27T11:17:35.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.554+0000 7f9d9efdd700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d7c06a0e0 0x7f9d7c06c5a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.554+0000 7f9d9efdd700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d9815e6e0 0x7f9d9813eb30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.554+0000 7f9d9efdd700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d980b3440 0x7f9d98160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.554+0000 7f9d9efdd700 1 --2- 172.21.15.112:0/1788743356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d98146c20 0x7f9d98145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.554+0000 7f9d9efdd700 1 -- 172.21.15.112:0/1788743356 >> 172.21.15.112:0/1788743356 conn(0x7f9d980b1e90 msgr2=0x7f9d9806ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:35.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.555+0000 7f9d9efdd700 1 -- 172.21.15.112:0/1788743356 shutdown_connections 2023-08-27T11:17:35.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.555+0000 7f9d9efdd700 1 -- 172.21.15.112:0/1788743356 wait complete. 2023-08-27T11:17:35.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:17:35.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.729+0000 7f1d4ab6d700 1 Processor -- start 2023-08-27T11:17:35.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.730+0000 7f1d4ab6d700 1 -- start start 2023-08-27T11:17:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.730+0000 7f1d4ab6d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1d440b3440 0x7f1d4415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.730+0000 7f1d4ab6d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4415e6e0 0x7f1d4406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.730+0000 7f1d4ab6d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d440b5070 con 0x7f1d44146c20 2023-08-27T11:17:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.730+0000 7f1d4ab6d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d44147000 con 0x7f1d440b3440 2023-08-27T11:17:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.730+0000 7f1d4ab6d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d44161740 con 0x7f1d4415e6e0 2023-08-27T11:17:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d48909700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1d44146c20 0x7f1d44160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50192/0 (socket says 172.21.15.112:50192) 2023-08-27T11:17:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d48909700 1 -- 172.21.15.112:0/3329507996 learned_addr learned my addr 172.21.15.112:0/3329507996 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736849495 0 0) 0x7f1d38001110 con 0x7f1d44146c20 2023-08-27T11:17:35.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d340035b0 con 0x7f1d44146c20 2023-08-27T11:17:35.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d4910a700 1 --2- 172.21.15.112:0/3329507996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4415e6e0 0x7f1d4406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d43fff700 1 --2- 172.21.15.112:0/3329507996 >> v2:172.21.15.138:3300/0 conn(0x7f1d440b3440 0x7f1d4415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3391489856 0 0) 0x7f1d340035b0 con 0x7f1d44146c20 2023-08-27T11:17:35.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4415e6e0 msgr2=0x7f1d4406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:35.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 --2- 172.21.15.112:0/3329507996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4415e6e0 0x7f1d4406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 >> v2:172.21.15.138:3300/0 conn(0x7f1d440b3440 msgr2=0x7f1d4415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:35.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 --2- 172.21.15.112:0/3329507996 >> v2:172.21.15.138:3300/0 conn(0x7f1d440b3440 0x7f1d4415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44161a20 con 0x7f1d44146c20 2023-08-27T11:17:35.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f1d38001800 con 0x7f1d44146c20 2023-08-27T11:17:35.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 >> v1:172.21.15.112:6789/0 conn(0x7f1d44146c20 legacy=0x7f1d44160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 --2- 172.21.15.112:0/3329507996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d34003a00 0x7f1d34003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 --2- 172.21.15.112:0/3329507996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d34004850 0x7f1d34004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 --2- 172.21.15.112:0/3329507996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4415e6e0 0x7f1d34005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d44147000 con 0x7f1d34003a00 2023-08-27T11:17:35.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d44161740 con 0x7f1d34004850 2023-08-27T11:17:35.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3400ba10 con 0x7f1d4415e6e0 2023-08-27T11:17:35.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d48909700 1 --2- 172.21.15.112:0/3329507996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d34003a00 0x7f1d34003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d4910a700 1 --2- 172.21.15.112:0/3329507996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4415e6e0 0x7f1d34005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d43fff700 1 --2- 172.21.15.112:0/3329507996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d34004850 0x7f1d34004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d48909700 1 -- 172.21.15.112:0/3329507996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4415e6e0 msgr2=0x7f1d34005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:35.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d48909700 1 --2- 172.21.15.112:0/3329507996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4415e6e0 0x7f1d34005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d48909700 1 -- 172.21.15.112:0/3329507996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d34004850 msgr2=0x7f1d34004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:35.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d48909700 1 --2- 172.21.15.112:0/3329507996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d34004850 0x7f1d34004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.731+0000 7f1d48909700 1 -- 172.21.15.112:0/3329507996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d340035b0 con 0x7f1d34003a00 2023-08-27T11:17:35.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.732+0000 7f1d48909700 1 --2- 172.21.15.112:0/3329507996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d34003a00 0x7f1d34003e20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800c320 tx=0x7f1d34005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439dd3accef1c151 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:35.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.732+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800ce20 con 0x7f1d34003a00 2023-08-27T11:17:35.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.732+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d38005960 con 0x7f1d34003a00 2023-08-27T11:17:35.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.732+0000 7f1d437fe700 1 -- 172.21.15.112:0/3329507996 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38005d10 con 0x7f1d34003a00 2023-08-27T11:17:35.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.732+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/3329507996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d34003a00 msgr2=0x7f1d34003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.732+0000 7f1d4ab6d700 1 --2- 172.21.15.112:0/3329507996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d34003a00 0x7f1d34003e20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800c320 tx=0x7f1d34005880 comp rx=0 tx=0).stop 2023-08-27T11:17:35.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.732+0000 7f1d4910a700 1 -- 172.21.15.112:0/3329507996 reap_dead start 2023-08-27T11:17:35.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.732+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/3329507996 shutdown_connections 2023-08-27T11:17:35.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.732+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/3329507996 >> 172.21.15.112:0/3329507996 conn(0x7f1d440b1e90 msgr2=0x7f1d440b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:35.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.732+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/3329507996 shutdown_connections 2023-08-27T11:17:35.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.732+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/3329507996 wait complete. 2023-08-27T11:17:35.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.733+0000 7f1d4ab6d700 1 Processor -- start 2023-08-27T11:17:35.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.734+0000 7f1d4ab6d700 1 -- start start 2023-08-27T11:17:35.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.734+0000 7f1d4ab6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d34003a00 0x7f1d44143d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.734+0000 7f1d4ab6d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4415f890 0x7f1d4415fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.734+0000 7f1d4ab6d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d441f61b0 0x7f1d441f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.734+0000 7f1d4ab6d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d44160250 con 0x7f1d34003a00 2023-08-27T11:17:35.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.734+0000 7f1d4ab6d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d44132070 con 0x7f1d441f61b0 2023-08-27T11:17:35.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.734+0000 7f1d4ab6d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d44132350 con 0x7f1d4415f890 2023-08-27T11:17:35.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d48909700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d34003a00 0x7f1d44143d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d48909700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d34003a00 0x7f1d44143d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55822/0 (socket says 172.21.15.112:55822) 2023-08-27T11:17:35.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d48909700 1 -- 172.21.15.112:0/1673495671 learned_addr learned my addr 172.21.15.112:0/1673495671 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:35.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d4910a700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d441f61b0 0x7f1d441f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d43fff700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4415f890 0x7f1d4415fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d48909700 1 -- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4415f890 msgr2=0x7f1d4415fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d48909700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4415f890 0x7f1d4415fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d48909700 1 -- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d441f61b0 msgr2=0x7f1d441f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d48909700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d441f61b0 0x7f1d441f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d48909700 1 -- 172.21.15.112:0/1673495671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44132630 con 0x7f1d34003a00 2023-08-27T11:17:35.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d48909700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d34003a00 0x7f1d44143d60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1d38000fb0 tx=0x7f1d38000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:35.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d417fa700 1 -- 172.21.15.112:0/1673495671 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38004870 con 0x7f1d34003a00 2023-08-27T11:17:35.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d417fa700 1 -- 172.21.15.112:0/1673495671 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d380049d0 con 0x7f1d34003a00 2023-08-27T11:17:35.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/1673495671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d44132910 con 0x7f1d34003a00 2023-08-27T11:17:35.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d417fa700 1 -- 172.21.15.112:0/1673495671 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38004d60 con 0x7f1d34003a00 2023-08-27T11:17:35.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.735+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/1673495671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d44132e30 con 0x7f1d34003a00 2023-08-27T11:17:35.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.736+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/1673495671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d44067e90 con 0x7f1d34003a00 2023-08-27T11:17:35.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.737+0000 7f1d417fa700 1 -- 172.21.15.112:0/1673495671 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1d38005040 con 0x7f1d34003a00 2023-08-27T11:17:35.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.737+0000 7f1d417fa700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d2806a0e0 0x7f1d2806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:35.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.737+0000 7f1d417fa700 1 -- 172.21.15.112:0/1673495671 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1d38088ed0 con 0x7f1d34003a00 2023-08-27T11:17:35.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.737+0000 7f1d43fff700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d2806a0e0 0x7f1d2806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:35.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.738+0000 7f1d43fff700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d2806a0e0 0x7f1d2806c5a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1d440b0350 tx=0x7f1d30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:35.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.741+0000 7f1d417fa700 1 -- 172.21.15.112:0/1673495671 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1d38054650 con 0x7f1d34003a00 2023-08-27T11:17:35.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.927+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/1673495671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d44132c80 con 0x7f1d34003a00 2023-08-27T11:17:35.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.927+0000 7f1d417fa700 1 -- 172.21.15.112:0/1673495671 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d38057c70 con 0x7f1d34003a00 2023-08-27T11:17:35.929 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:17:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.929+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d2806a0e0 msgr2=0x7f1d2806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.929+0000 7f1d4ab6d700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d2806a0e0 0x7f1d2806c5a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1d440b0350 tx=0x7f1d30008040 comp rx=0 tx=0).stop 2023-08-27T11:17:35.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.929+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/1673495671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d34003a00 msgr2=0x7f1d44143d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:35.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.929+0000 7f1d4ab6d700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d34003a00 0x7f1d44143d60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1d38000fb0 tx=0x7f1d38000fe0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.930+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/1673495671 shutdown_connections 2023-08-27T11:17:35.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.930+0000 7f1d4ab6d700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d2806a0e0 0x7f1d2806c5a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.930+0000 7f1d4ab6d700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d441f61b0 0x7f1d441f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.930+0000 7f1d4ab6d700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d34003a00 0x7f1d44143d60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.930+0000 7f1d4ab6d700 1 --2- 172.21.15.112:0/1673495671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4415f890 0x7f1d4415fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.930+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/1673495671 >> 172.21.15.112:0/1673495671 conn(0x7f1d440b1e90 msgr2=0x7f1d4415ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.930+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/1673495671 shutdown_connections 2023-08-27T11:17:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:35.930+0000 7f1d4ab6d700 1 -- 172.21.15.112:0/1673495671 wait complete. 2023-08-27T11:17:36.100 INFO:tasks.cephfs_test_runner:Starting test: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) That the volume can be created and then cleans up 2023-08-27T11:17:36.101 INFO:tasks.cephfs_test_runner:test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2023-08-27T11:17:36.102 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2023-08-27T11:17:36.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.222+0000 7fa4a8767700 1 Processor -- start 2023-08-27T11:17:36.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.222+0000 7fa4a8767700 1 -- start start 2023-08-27T11:17:36.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a8767700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa4a00b1400 0x7fa4a00cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:36.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a8767700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4a00b17e0 0x7fa4a006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:36.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a8767700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa4a0161ce0 0x7fa4a006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:36.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a8767700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a0161850 con 0x7fa4a0161ce0 2023-08-27T11:17:36.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a8767700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a006e7d0 con 0x7fa4a00b1400 2023-08-27T11:17:36.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a8767700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a00cfc10 con 0x7fa4a00b17e0 2023-08-27T11:17:36.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a6d04700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa4a0161ce0 0x7fa4a006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:36.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a6d04700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa4a0161ce0 0x7fa4a006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55834/0 (socket says 172.21.15.112:55834) 2023-08-27T11:17:36.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a6d04700 1 -- 172.21.15.112:0/777301040 learned_addr learned my addr 172.21.15.112:0/777301040 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:36.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a6503700 1 --2- 172.21.15.112:0/777301040 >> v2:172.21.15.138:3300/0 conn(0x7fa4a00b1400 0x7fa4a00cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:36.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a5d02700 1 --2- 172.21.15.112:0/777301040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4a00b17e0 0x7fa4a006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:36.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a6d04700 1 -- 172.21.15.112:0/777301040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4a00b17e0 msgr2=0x7fa4a006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:36.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a6d04700 1 --2- 172.21.15.112:0/777301040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4a00b17e0 0x7fa4a006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:36.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a6d04700 1 -- 172.21.15.112:0/777301040 >> v2:172.21.15.138:3300/0 conn(0x7fa4a00b1400 msgr2=0x7fa4a00cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a6d04700 1 --2- 172.21.15.112:0/777301040 >> v2:172.21.15.138:3300/0 conn(0x7fa4a00b1400 0x7fa4a00cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.223+0000 7fa4a6d04700 1 -- 172.21.15.112:0/777301040 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a0161200 con 0x7fa4a0161ce0 2023-08-27T11:17:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.224+0000 7fa4a6d04700 1 --2- 172.21.15.112:0/777301040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0161ce0 0x7fa4a006f790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0070370 tx=0x7fa49800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f514fb8d14cdab49 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.224+0000 7fa4a5501700 1 -- 172.21.15.112:0/777301040 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49800e070 con 0x7fa4a0161ce0 2023-08-27T11:17:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.224+0000 7fa4a5501700 1 -- 172.21.15.112:0/777301040 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49800b910 con 0x7fa4a0161ce0 2023-08-27T11:17:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.224+0000 7fa4a5501700 1 -- 172.21.15.112:0/777301040 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49800bcc0 con 0x7fa4a0161ce0 2023-08-27T11:17:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.224+0000 7fa4a8767700 1 -- 172.21.15.112:0/777301040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0161ce0 msgr2=0x7fa4a006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:36.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.224+0000 7fa4a8767700 1 --2- 172.21.15.112:0/777301040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0161ce0 0x7fa4a006f790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0070370 tx=0x7fa49800b330 comp rx=0 tx=0).stop 2023-08-27T11:17:36.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.224+0000 7fa4a8767700 1 -- 172.21.15.112:0/777301040 shutdown_connections 2023-08-27T11:17:36.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.224+0000 7fa4a8767700 1 --2- 172.21.15.112:0/777301040 >> v2:172.21.15.138:3300/0 conn(0x7fa4a00b1400 0x7fa4a00cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:36.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.224+0000 7fa4a8767700 1 --2- 172.21.15.112:0/777301040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0161ce0 0x7fa4a006f790 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:36.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.224+0000 7fa4a8767700 1 --2- 172.21.15.112:0/777301040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4a00b17e0 0x7fa4a006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:36.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.224+0000 7fa4a8767700 1 -- 172.21.15.112:0/777301040 >> 172.21.15.112:0/777301040 conn(0x7fa4a00d0000 msgr2=0x7fa4a00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:36.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.224+0000 7fa4a8767700 1 -- 172.21.15.112:0/777301040 shutdown_connections 2023-08-27T11:17:36.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.225+0000 7fa4a8767700 1 -- 172.21.15.112:0/777301040 wait complete. 2023-08-27T11:17:36.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.226+0000 7fa4a8767700 1 Processor -- start 2023-08-27T11:17:36.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.226+0000 7fa4a8767700 1 -- start start 2023-08-27T11:17:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.226+0000 7fa4a8767700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4a00b1400 0x7fa4a00c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.226+0000 7fa4a8767700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a00b17e0 0x7fa4a0152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.226+0000 7fa4a8767700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0161ce0 0x7fa4a01f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.226+0000 7fa4a8767700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01528d0 con 0x7fa4a0161ce0 2023-08-27T11:17:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.226+0000 7fa4a8767700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01f6fd0 con 0x7fa4a00b17e0 2023-08-27T11:17:36.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.226+0000 7fa4a8767700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a00c9e10 con 0x7fa4a00b1400 2023-08-27T11:17:36.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a6d04700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0161ce0 0x7fa4a01f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:36.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a6d04700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0161ce0 0x7fa4a01f6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55840/0 (socket says 172.21.15.112:55840) 2023-08-27T11:17:36.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a6d04700 1 -- 172.21.15.112:0/2182120615 learned_addr learned my addr 172.21.15.112:0/2182120615 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:36.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a5d02700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a00b17e0 0x7fa4a0152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:36.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a6d04700 1 -- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4a00b1400 msgr2=0x7fa4a00c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:36.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a6503700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4a00b1400 0x7fa4a00c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:36.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a6d04700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4a00b1400 0x7fa4a00c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:36.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a6d04700 1 -- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a00b17e0 msgr2=0x7fa4a0152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:36.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a6d04700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a00b17e0 0x7fa4a0152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:36.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a6d04700 1 -- 172.21.15.112:0/2182120615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa498009b30 con 0x7fa4a0161ce0 2023-08-27T11:17:36.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a5d02700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a00b17e0 0x7fa4a0152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:36.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a6d04700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0161ce0 0x7fa4a01f6a90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00d1100 tx=0x7fa498004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:36.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa496ffd700 1 -- 172.21.15.112:0/2182120615 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49800ce80 con 0x7fa4a0161ce0 2023-08-27T11:17:36.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa496ffd700 1 -- 172.21.15.112:0/2182120615 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa498004250 con 0x7fa4a0161ce0 2023-08-27T11:17:36.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a8767700 1 -- 172.21.15.112:0/2182120615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a00ca0f0 con 0x7fa4a0161ce0 2023-08-27T11:17:36.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa496ffd700 1 -- 172.21.15.112:0/2182120615 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498004600 con 0x7fa4a0161ce0 2023-08-27T11:17:36.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.227+0000 7fa4a8767700 1 -- 172.21.15.112:0/2182120615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a01f6570 con 0x7fa4a0161ce0 2023-08-27T11:17:36.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.228+0000 7fa4a8767700 1 -- 172.21.15.112:0/2182120615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a00ca330 con 0x7fa4a0161ce0 2023-08-27T11:17:36.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.228+0000 7fa496ffd700 1 -- 172.21.15.112:0/2182120615 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa498004760 con 0x7fa4a0161ce0 2023-08-27T11:17:36.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.229+0000 7fa496ffd700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa48406a0e0 0x7fa48406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:36.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.229+0000 7fa496ffd700 1 -- 172.21.15.112:0/2182120615 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa498089930 con 0x7fa4a0161ce0 2023-08-27T11:17:36.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.229+0000 7fa4a6503700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa48406a0e0 0x7fa48406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:36.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.229+0000 7fa4a6503700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa48406a0e0 0x7fa48406c5a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa490009a30 tx=0x7fa490008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:36.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.233+0000 7fa496ffd700 1 -- 172.21.15.112:0/2182120615 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa4a00ca330 con 0x7fa4a0161ce0 2023-08-27T11:17:36.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:36.507+0000 7fa4a8767700 1 -- 172.21.15.112:0/2182120615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7fa4a01f60f0 con 0x7fa4a0161ce0 2023-08-27T11:17:37.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.300+0000 7fa496ffd700 1 -- 172.21.15.112:0/2182120615 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa4a01f60f0 con 0x7fa4a0161ce0 2023-08-27T11:17:37.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.303+0000 7fa4a8767700 1 -- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa48406a0e0 msgr2=0x7fa48406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.303+0000 7fa4a8767700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa48406a0e0 0x7fa48406c5a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa490009a30 tx=0x7fa490008040 comp rx=0 tx=0).stop 2023-08-27T11:17:37.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.303+0000 7fa4a8767700 1 -- 172.21.15.112:0/2182120615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0161ce0 msgr2=0x7fa4a01f6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.303+0000 7fa4a8767700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0161ce0 0x7fa4a01f6a90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00d1100 tx=0x7fa498004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.303+0000 7fa4a8767700 1 -- 172.21.15.112:0/2182120615 shutdown_connections 2023-08-27T11:17:37.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.303+0000 7fa4a8767700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa48406a0e0 0x7fa48406c5a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.303+0000 7fa4a8767700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0161ce0 0x7fa4a01f6a90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.303+0000 7fa4a8767700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4a00b1400 0x7fa4a00c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.303+0000 7fa4a8767700 1 --2- 172.21.15.112:0/2182120615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a00b17e0 0x7fa4a0152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.303+0000 7fa4a8767700 1 -- 172.21.15.112:0/2182120615 >> 172.21.15.112:0/2182120615 conn(0x7fa4a00d0000 msgr2=0x7fa4a00703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:37.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.303+0000 7fa4a8767700 1 -- 172.21.15.112:0/2182120615 shutdown_connections 2023-08-27T11:17:37.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.303+0000 7fa4a8767700 1 -- 172.21.15.112:0/2182120615 wait complete. 2023-08-27T11:17:37.319 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:37.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.446+0000 7f307f9c9700 1 Processor -- start 2023-08-27T11:17:37.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.446+0000 7f307f9c9700 1 -- start start 2023-08-27T11:17:37.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.447+0000 7f307f9c9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3078160a00 0x7f30780b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.447+0000 7f307f9c9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f307806d510 0x7f307806d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.447+0000 7f307f9c9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307806dea0 con 0x7f3078146990 2023-08-27T11:17:37.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.447+0000 7f307f9c9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307806f930 con 0x7f3078160a00 2023-08-27T11:17:37.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.447+0000 7f307f9c9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307806fc10 con 0x7f307806d510 2023-08-27T11:17:37.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.447+0000 7f307cf64700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3078146990 0x7f307815dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50228/0 (socket says 172.21.15.112:50228) 2023-08-27T11:17:37.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.447+0000 7f307cf64700 1 -- 172.21.15.112:0/3200478678 learned_addr learned my addr 172.21.15.112:0/3200478678 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:37.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.447+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277338952 0 0) 0x7f3074001110 con 0x7f3078146990 2023-08-27T11:17:37.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.447+0000 7f307df66700 1 --2- 172.21.15.112:0/3200478678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f307806d510 0x7f307806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.447+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30640035b0 con 0x7f3078146990 2023-08-27T11:17:37.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.447+0000 7f307d765700 1 --2- 172.21.15.112:0/3200478678 >> v2:172.21.15.138:3300/0 conn(0x7f3078160a00 0x7f30780b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 329426927 0 0) 0x7f30640035b0 con 0x7f3078146990 2023-08-27T11:17:37.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f307806d510 msgr2=0x7f307806d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:37.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 --2- 172.21.15.112:0/3200478678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f307806d510 0x7f307806d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 >> v2:172.21.15.138:3300/0 conn(0x7f3078160a00 msgr2=0x7f30780b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:37.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 --2- 172.21.15.112:0/3200478678 >> v2:172.21.15.138:3300/0 conn(0x7f3078160a00 0x7f30780b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307806fef0 con 0x7f3078146990 2023-08-27T11:17:37.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f3074001800 con 0x7f3078146990 2023-08-27T11:17:37.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 >> v1:172.21.15.112:6789/0 conn(0x7f3078146990 legacy=0x7f307815dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 --2- 172.21.15.112:0/3200478678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3064003a00 0x7f3064003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 --2- 172.21.15.112:0/3200478678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064004850 0x7f3064004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 --2- 172.21.15.112:0/3200478678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f307806d510 0x7f3064005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307806f930 con 0x7f3064003a00 2023-08-27T11:17:37.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307806fc10 con 0x7f3064004850 2023-08-27T11:17:37.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306400ba10 con 0x7f307806d510 2023-08-27T11:17:37.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f307d765700 1 --2- 172.21.15.112:0/3200478678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3064003a00 0x7f3064003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f307df66700 1 --2- 172.21.15.112:0/3200478678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f307806d510 0x7f3064005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f307cf64700 1 --2- 172.21.15.112:0/3200478678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064004850 0x7f3064004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f307d765700 1 -- 172.21.15.112:0/3200478678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f307806d510 msgr2=0x7f3064005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:37.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f307d765700 1 --2- 172.21.15.112:0/3200478678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f307806d510 0x7f3064005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f307d765700 1 -- 172.21.15.112:0/3200478678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064004850 msgr2=0x7f3064004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:37.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f307d765700 1 --2- 172.21.15.112:0/3200478678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064004850 0x7f3064004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f307d765700 1 -- 172.21.15.112:0/3200478678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30640035b0 con 0x7f3064003a00 2023-08-27T11:17:37.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.448+0000 7f307d765700 1 --2- 172.21.15.112:0/3200478678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3064003a00 0x7f3064003e20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3068009b60 tx=0x7f3064005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5dd9871971445c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:37.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.449+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306800e6c0 con 0x7f3064003a00 2023-08-27T11:17:37.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.449+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306800e820 con 0x7f3064003a00 2023-08-27T11:17:37.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.449+0000 7f306ffff700 1 -- 172.21.15.112:0/3200478678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306800eb90 con 0x7f3064003a00 2023-08-27T11:17:37.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.449+0000 7f307f9c9700 1 -- 172.21.15.112:0/3200478678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3064003a00 msgr2=0x7f3064003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.449+0000 7f307f9c9700 1 --2- 172.21.15.112:0/3200478678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3064003a00 0x7f3064003e20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3068009b60 tx=0x7f3064005880 comp rx=0 tx=0).stop 2023-08-27T11:17:37.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.449+0000 7f307df66700 1 -- 172.21.15.112:0/3200478678 reap_dead start 2023-08-27T11:17:37.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.449+0000 7f307f9c9700 1 -- 172.21.15.112:0/3200478678 shutdown_connections 2023-08-27T11:17:37.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.449+0000 7f307f9c9700 1 -- 172.21.15.112:0/3200478678 >> 172.21.15.112:0/3200478678 conn(0x7f30780cfe50 msgr2=0x7f30780d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:37.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.449+0000 7f307f9c9700 1 -- 172.21.15.112:0/3200478678 shutdown_connections 2023-08-27T11:17:37.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.449+0000 7f307f9c9700 1 -- 172.21.15.112:0/3200478678 wait complete. 2023-08-27T11:17:37.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.450+0000 7f307f9c9700 1 Processor -- start 2023-08-27T11:17:37.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.450+0000 7f307f9c9700 1 -- start start 2023-08-27T11:17:37.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.451+0000 7f307f9c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064003a00 0x7f307813dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.451+0000 7f307f9c9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307813ccb0 0x7f30780b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.451+0000 7f307f9c9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30781411a0 0x7f3078141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.451+0000 7f307f9c9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30780b7ea0 con 0x7f307813ccb0 2023-08-27T11:17:37.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.451+0000 7f307f9c9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30780b6fd0 con 0x7f3064003a00 2023-08-27T11:17:37.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.451+0000 7f307f9c9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307813dff0 con 0x7f30781411a0 2023-08-27T11:17:37.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.451+0000 7f307cf64700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307813ccb0 0x7f30780b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.451+0000 7f307cf64700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307813ccb0 0x7f30780b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55858/0 (socket says 172.21.15.112:55858) 2023-08-27T11:17:37.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.451+0000 7f307cf64700 1 -- 172.21.15.112:0/3010145096 learned_addr learned my addr 172.21.15.112:0/3010145096 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:37.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.451+0000 7f307df66700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30781411a0 0x7f3078141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.451+0000 7f307d765700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064003a00 0x7f307813dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.452+0000 7f307cf64700 1 -- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30781411a0 msgr2=0x7f3078141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.452+0000 7f307cf64700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30781411a0 0x7f3078141630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.452+0000 7f307cf64700 1 -- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064003a00 msgr2=0x7f307813dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.452+0000 7f307cf64700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064003a00 0x7f307813dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.452+0000 7f307cf64700 1 -- 172.21.15.112:0/3010145096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30780b7220 con 0x7f307813ccb0 2023-08-27T11:17:37.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.452+0000 7f307cf64700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307813ccb0 0x7f30780b82c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3074001c30 tx=0x7f30740075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:37.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.452+0000 7f306dffb700 1 -- 172.21.15.112:0/3010145096 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400f5e0 con 0x7f307813ccb0 2023-08-27T11:17:37.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.452+0000 7f306dffb700 1 -- 172.21.15.112:0/3010145096 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307400f780 con 0x7f307813ccb0 2023-08-27T11:17:37.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.452+0000 7f306dffb700 1 -- 172.21.15.112:0/3010145096 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400fb30 con 0x7f307813ccb0 2023-08-27T11:17:37.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.452+0000 7f307f9c9700 1 -- 172.21.15.112:0/3010145096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30780b7500 con 0x7f307813ccb0 2023-08-27T11:17:37.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.452+0000 7f307f9c9700 1 -- 172.21.15.112:0/3010145096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30780b7ac0 con 0x7f307813ccb0 2023-08-27T11:17:37.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.453+0000 7f307f9c9700 1 -- 172.21.15.112:0/3010145096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30780b7fe0 con 0x7f307813ccb0 2023-08-27T11:17:37.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.453+0000 7f306dffb700 1 -- 172.21.15.112:0/3010145096 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f307400fc90 con 0x7f307813ccb0 2023-08-27T11:17:37.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.454+0000 7f306dffb700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f305c06a0e0 0x7f305c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.454+0000 7f306dffb700 1 -- 172.21.15.112:0/3010145096 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3074088290 con 0x7f307813ccb0 2023-08-27T11:17:37.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.454+0000 7f307d765700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f305c06a0e0 0x7f305c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.454+0000 7f307d765700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f305c06a0e0 0x7f305c06c5a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f30680090b0 tx=0x7f306801f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:37.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.458+0000 7f306dffb700 1 -- 172.21.15.112:0/3010145096 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f30780b7fe0 con 0x7f307813ccb0 2023-08-27T11:17:37.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.650+0000 7f307f9c9700 1 -- 172.21.15.112:0/3010145096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f307806dea0 con 0x7f307813ccb0 2023-08-27T11:17:37.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.651+0000 7f306dffb700 1 -- 172.21.15.112:0/3010145096 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7f307806dea0 con 0x7f307813ccb0 2023-08-27T11:17:37.654 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:37.654 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:37.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.653+0000 7f307f9c9700 1 -- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f305c06a0e0 msgr2=0x7f305c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.653+0000 7f307f9c9700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f305c06a0e0 0x7f305c06c5a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f30680090b0 tx=0x7f306801f040 comp rx=0 tx=0).stop 2023-08-27T11:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.653+0000 7f307f9c9700 1 -- 172.21.15.112:0/3010145096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307813ccb0 msgr2=0x7f30780b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.653+0000 7f307f9c9700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307813ccb0 0x7f30780b82c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3074001c30 tx=0x7f30740075f0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.653+0000 7f307f9c9700 1 -- 172.21.15.112:0/3010145096 shutdown_connections 2023-08-27T11:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.653+0000 7f307f9c9700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f305c06a0e0 0x7f305c06c5a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.653+0000 7f307f9c9700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30781411a0 0x7f3078141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.653+0000 7f307f9c9700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064003a00 0x7f307813dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.653+0000 7f307f9c9700 1 --2- 172.21.15.112:0/3010145096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307813ccb0 0x7f30780b82c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.653+0000 7f307f9c9700 1 -- 172.21.15.112:0/3010145096 >> 172.21.15.112:0/3010145096 conn(0x7f30780cfe50 msgr2=0x7f307815f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.653+0000 7f307f9c9700 1 -- 172.21.15.112:0/3010145096 shutdown_connections 2023-08-27T11:17:37.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.653+0000 7f307f9c9700 1 -- 172.21.15.112:0/3010145096 wait complete. 2023-08-27T11:17:37.667 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:17:02.484063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:17:03.722976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:17:37.668 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:17:37.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.790+0000 7f87e6860700 1 Processor -- start 2023-08-27T11:17:37.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.790+0000 7f87e6860700 1 -- start start 2023-08-27T11:17:37.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87e6860700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87e0146bf0 0x7f87e00b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87e6860700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f87e0146fd0 0x7f87e00cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87e6860700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e00d1500 con 0x7f87e00d0290 2023-08-27T11:17:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87e6860700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e00d0710 con 0x7f87e0146bf0 2023-08-27T11:17:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87e6860700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e00d09f0 con 0x7f87e0146fd0 2023-08-27T11:17:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87dffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87e0146bf0 0x7f87e00b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87e4dfd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f87e00d0290 0x7f87e0070eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50250/0 (socket says 172.21.15.112:50250) 2023-08-27T11:17:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87e4dfd700 1 -- 172.21.15.112:0/364927369 learned_addr learned my addr 172.21.15.112:0/364927369 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:37.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87df7fe700 1 --2- 172.21.15.112:0/364927369 >> v2:172.21.15.138:3300/0 conn(0x7f87e0146fd0 0x7f87e00cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87dffff700 1 -- 172.21.15.112:0/364927369 >> v2:172.21.15.138:3300/0 conn(0x7f87e0146fd0 msgr2=0x7f87e00cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:37.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87dffff700 1 --2- 172.21.15.112:0/364927369 >> v2:172.21.15.138:3300/0 conn(0x7f87e0146fd0 0x7f87e00cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87dffff700 1 -- 172.21.15.112:0/364927369 >> v1:172.21.15.112:6789/0 conn(0x7f87e00d0290 legacy=0x7f87e0070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87dffff700 1 -- 172.21.15.112:0/364927369 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e00d0cd0 con 0x7f87e0146bf0 2023-08-27T11:17:37.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87deffd700 1 -- 172.21.15.112:0/364927369 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513415687 0 0) 0x7f87e00d1500 con 0x7f87e00d0290 2023-08-27T11:17:37.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87dffff700 1 --2- 172.21.15.112:0/364927369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e0146bf0 0x7f87e00b5db0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f87e0071d70 tx=0x7f87d0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1427df9e3b26d58 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:37.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87deffd700 1 -- 172.21.15.112:0/364927369 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0011830 con 0x7f87e0146bf0 2023-08-27T11:17:37.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87deffd700 1 -- 172.21.15.112:0/364927369 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d0011990 con 0x7f87e0146bf0 2023-08-27T11:17:37.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.791+0000 7f87deffd700 1 -- 172.21.15.112:0/364927369 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0011d20 con 0x7f87e0146bf0 2023-08-27T11:17:37.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.792+0000 7f87e6860700 1 -- 172.21.15.112:0/364927369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e0146bf0 msgr2=0x7f87e00b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.792+0000 7f87e6860700 1 --2- 172.21.15.112:0/364927369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e0146bf0 0x7f87e00b5db0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f87e0071d70 tx=0x7f87d0009b20 comp rx=0 tx=0).stop 2023-08-27T11:17:37.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.792+0000 7f87e6860700 1 -- 172.21.15.112:0/364927369 shutdown_connections 2023-08-27T11:17:37.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.792+0000 7f87e6860700 1 --2- 172.21.15.112:0/364927369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e0146bf0 0x7f87e00b5db0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.792+0000 7f87e6860700 1 --2- 172.21.15.112:0/364927369 >> v2:172.21.15.138:3300/0 conn(0x7f87e0146fd0 0x7f87e00cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.792+0000 7f87e6860700 1 -- 172.21.15.112:0/364927369 >> 172.21.15.112:0/364927369 conn(0x7f87e006c450 msgr2=0x7f87e006f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:37.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.792+0000 7f87e6860700 1 -- 172.21.15.112:0/364927369 shutdown_connections 2023-08-27T11:17:37.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.792+0000 7f87e6860700 1 -- 172.21.15.112:0/364927369 wait complete. 2023-08-27T11:17:37.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.794+0000 7f87e6860700 1 Processor -- start 2023-08-27T11:17:37.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.794+0000 7f87e6860700 1 -- start start 2023-08-27T11:17:37.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87e6860700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e00d0290 0x7f87e01420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87e6860700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87e0146bf0 0x7f87e013fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87e6860700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87e0146fd0 0x7f87e015c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87e6860700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e013fff0 con 0x7f87e00d0290 2023-08-27T11:17:37.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87e6860700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01401f0 con 0x7f87e0146bf0 2023-08-27T11:17:37.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87e6860700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01404d0 con 0x7f87e0146fd0 2023-08-27T11:17:37.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87dffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e00d0290 0x7f87e01420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87dffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e00d0290 0x7f87e01420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55870/0 (socket says 172.21.15.112:55870) 2023-08-27T11:17:37.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87dffff700 1 -- 172.21.15.112:0/1497652634 learned_addr learned my addr 172.21.15.112:0/1497652634 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:37.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87dffff700 1 -- 172.21.15.112:0/1497652634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87e0146fd0 msgr2=0x7f87e015c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87dffff700 1 --2- 172.21.15.112:0/1497652634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87e0146fd0 0x7f87e015c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87df7fe700 1 --2- 172.21.15.112:0/1497652634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87e0146bf0 0x7f87e013fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87dffff700 1 -- 172.21.15.112:0/1497652634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87e0146bf0 msgr2=0x7f87e013fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87dffff700 1 --2- 172.21.15.112:0/1497652634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87e0146bf0 0x7f87e013fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87dffff700 1 -- 172.21.15.112:0/1497652634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d00094a0 con 0x7f87e00d0290 2023-08-27T11:17:37.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87dffff700 1 --2- 172.21.15.112:0/1497652634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e00d0290 0x7f87e01420b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f87d00090b0 tx=0x7f87d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:37.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87dcff9700 1 -- 172.21.15.112:0/1497652634 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d00038a0 con 0x7f87e00d0290 2023-08-27T11:17:37.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87dcff9700 1 -- 172.21.15.112:0/1497652634 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d001f070 con 0x7f87e00d0290 2023-08-27T11:17:37.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87e6860700 1 -- 172.21.15.112:0/1497652634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e01407b0 con 0x7f87e00d0290 2023-08-27T11:17:37.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87dcff9700 1 -- 172.21.15.112:0/1497652634 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0003c20 con 0x7f87e00d0290 2023-08-27T11:17:37.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.795+0000 7f87e6860700 1 -- 172.21.15.112:0/1497652634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e01f29f0 con 0x7f87e00d0290 2023-08-27T11:17:37.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.796+0000 7f87e6860700 1 -- 172.21.15.112:0/1497652634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e0067e90 con 0x7f87e00d0290 2023-08-27T11:17:37.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.797+0000 7f87dcff9700 1 -- 172.21.15.112:0/1497652634 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f87d001a430 con 0x7f87e00d0290 2023-08-27T11:17:37.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.797+0000 7f87dcff9700 1 --2- 172.21.15.112:0/1497652634 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87cc06a0e0 0x7f87cc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:37.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.797+0000 7f87dcff9700 1 -- 172.21.15.112:0/1497652634 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f87d0021070 con 0x7f87e00d0290 2023-08-27T11:17:37.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.797+0000 7f87df7fe700 1 --2- 172.21.15.112:0/1497652634 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87cc06a0e0 0x7f87cc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:37.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.798+0000 7f87df7fe700 1 --2- 172.21.15.112:0/1497652634 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87cc06a0e0 0x7f87cc06c5a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f87e00b32e0 tx=0x7f87d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:37.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.801+0000 7f87dcff9700 1 -- 172.21.15.112:0/1497652634 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f87d0055680 con 0x7f87e00d0290 2023-08-27T11:17:37.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.989+0000 7f87e6860700 1 -- 172.21.15.112:0/1497652634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f87e0067a20 con 0x7f87e00d0290 2023-08-27T11:17:37.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.989+0000 7f87dcff9700 1 -- 172.21.15.112:0/1497652634 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f87d0015020 con 0x7f87e00d0290 2023-08-27T11:17:37.992 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.991+0000 7f87e6860700 1 -- 172.21.15.112:0/1497652634 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87cc06a0e0 msgr2=0x7f87cc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.991+0000 7f87e6860700 1 --2- 172.21.15.112:0/1497652634 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87cc06a0e0 0x7f87cc06c5a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f87e00b32e0 tx=0x7f87d4008040 comp rx=0 tx=0).stop 2023-08-27T11:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.991+0000 7f87e6860700 1 -- 172.21.15.112:0/1497652634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e00d0290 msgr2=0x7f87e01420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:37.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.991+0000 7f87e6860700 1 --2- 172.21.15.112:0/1497652634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e00d0290 0x7f87e01420b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f87d00090b0 tx=0x7f87d0004750 comp rx=0 tx=0).stop 2023-08-27T11:17:37.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.991+0000 7f87e6860700 1 -- 172.21.15.112:0/1497652634 shutdown_connections 2023-08-27T11:17:37.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.992+0000 7f87e6860700 1 --2- 172.21.15.112:0/1497652634 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87cc06a0e0 0x7f87cc06c5a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.992+0000 7f87e6860700 1 --2- 172.21.15.112:0/1497652634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87e0146fd0 0x7f87e015c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.992+0000 7f87e6860700 1 --2- 172.21.15.112:0/1497652634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e00d0290 0x7f87e01420b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.992+0000 7f87e6860700 1 --2- 172.21.15.112:0/1497652634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87e0146bf0 0x7f87e013fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:37.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.992+0000 7f87e6860700 1 -- 172.21.15.112:0/1497652634 >> 172.21.15.112:0/1497652634 conn(0x7f87e006c450 msgr2=0x7f87e0145140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:37.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.992+0000 7f87e6860700 1 -- 172.21.15.112:0/1497652634 shutdown_connections 2023-08-27T11:17:37.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:37.992+0000 7f87e6860700 1 -- 172.21.15.112:0/1497652634 wait complete. 2023-08-27T11:17:38.005 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:38.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.126+0000 7ff234722700 1 Processor -- start 2023-08-27T11:17:38.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.126+0000 7ff234722700 1 -- start start 2023-08-27T11:17:38.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff234722700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff22c143390 0x7ff22c0723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff234722700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22c0b16e0 con 0x7ff22c06f820 2023-08-27T11:17:38.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff234722700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22c143770 con 0x7ff22c06e4c0 2023-08-27T11:17:38.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff234722700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22c070000 con 0x7ff22c143390 2023-08-27T11:17:38.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff2324be700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff22c143390 0x7ff22c0723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff231cbd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff22c06f820 0x7ff22c06ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50262/0 (socket says 172.21.15.112:50262) 2023-08-27T11:17:38.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff231cbd700 1 -- 172.21.15.112:0/100316039 learned_addr learned my addr 172.21.15.112:0/100316039 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:38.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff2324be700 1 -- 172.21.15.112:0/100316039 >> v1:172.21.15.138:6789/0 conn(0x7ff22c06e4c0 legacy=0x7ff22c0af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff2324be700 1 -- 172.21.15.112:0/100316039 >> v1:172.21.15.112:6789/0 conn(0x7ff22c06f820 legacy=0x7ff22c06ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff2314bc700 1 -- 172.21.15.112:0/100316039 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302554978 0 0) 0x7ff22c0b16e0 con 0x7ff22c06f820 2023-08-27T11:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff2324be700 1 -- 172.21.15.112:0/100316039 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c070f40 con 0x7ff22c143390 2023-08-27T11:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff2324be700 1 --2- 172.21.15.112:0/100316039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff22c143390 0x7ff22c0723f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff228009840 tx=0x7ff228009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164b4cbffa0ac805 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff2314bc700 1 -- 172.21.15.112:0/100316039 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228004230 con 0x7ff22c143390 2023-08-27T11:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff2314bc700 1 -- 172.21.15.112:0/100316039 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff228004390 con 0x7ff22c143390 2023-08-27T11:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.127+0000 7ff2314bc700 1 -- 172.21.15.112:0/100316039 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2280036a0 con 0x7ff22c143390 2023-08-27T11:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.128+0000 7ff234722700 1 -- 172.21.15.112:0/100316039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff22c143390 msgr2=0x7ff22c0723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.128+0000 7ff234722700 1 --2- 172.21.15.112:0/100316039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff22c143390 0x7ff22c0723f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff228009840 tx=0x7ff228009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.128+0000 7ff234722700 1 -- 172.21.15.112:0/100316039 shutdown_connections 2023-08-27T11:17:38.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.128+0000 7ff234722700 1 --2- 172.21.15.112:0/100316039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff22c143390 0x7ff22c0723f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.128+0000 7ff234722700 1 -- 172.21.15.112:0/100316039 >> 172.21.15.112:0/100316039 conn(0x7ff22c0cc240 msgr2=0x7ff22c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:38.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.128+0000 7ff234722700 1 -- 172.21.15.112:0/100316039 shutdown_connections 2023-08-27T11:17:38.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.128+0000 7ff234722700 1 -- 172.21.15.112:0/100316039 wait complete. 2023-08-27T11:17:38.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.129+0000 7ff234722700 1 Processor -- start 2023-08-27T11:17:38.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.129+0000 7ff234722700 1 -- start start 2023-08-27T11:17:38.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff234722700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff22c06e4c0 0x7ff22c154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff234722700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff22c06f820 0x7ff22c155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff234722700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff22c143390 0x7ff22c135f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff234722700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c152380 con 0x7ff22c06e4c0 2023-08-27T11:17:38.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff234722700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0cdab0 con 0x7ff22c143390 2023-08-27T11:17:38.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff234722700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c154880 con 0x7ff22c06f820 2023-08-27T11:17:38.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff2324be700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff22c06e4c0 0x7ff22c154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff2324be700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff22c06e4c0 0x7ff22c154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55886/0 (socket says 172.21.15.112:55886) 2023-08-27T11:17:38.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff2324be700 1 -- 172.21.15.112:0/2511431926 learned_addr learned my addr 172.21.15.112:0/2511431926 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:38.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff231cbd700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff22c06f820 0x7ff22c155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff232cbf700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff22c143390 0x7ff22c135f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff2324be700 1 -- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff22c06f820 msgr2=0x7ff22c155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff2324be700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff22c06f820 0x7ff22c155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff2324be700 1 -- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff22c143390 msgr2=0x7ff22c135f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff2324be700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff22c143390 0x7ff22c135f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff2324be700 1 -- 172.21.15.112:0/2511431926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2280094a0 con 0x7ff22c06e4c0 2023-08-27T11:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff232cbf700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff22c143390 0x7ff22c135f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff2324be700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff22c06e4c0 0x7ff22c154ff0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff22c06ac50 tx=0x7ff2280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.130+0000 7ff222ffd700 1 -- 172.21.15.112:0/2511431926 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228011630 con 0x7ff22c06e4c0 2023-08-27T11:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.131+0000 7ff222ffd700 1 -- 172.21.15.112:0/2511431926 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff228011790 con 0x7ff22c06e4c0 2023-08-27T11:17:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.131+0000 7ff234722700 1 -- 172.21.15.112:0/2511431926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22c154b60 con 0x7ff22c06e4c0 2023-08-27T11:17:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.131+0000 7ff222ffd700 1 -- 172.21.15.112:0/2511431926 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228011b40 con 0x7ff22c06e4c0 2023-08-27T11:17:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.131+0000 7ff234722700 1 -- 172.21.15.112:0/2511431926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22c0cdd00 con 0x7ff22c06e4c0 2023-08-27T11:17:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.131+0000 7ff234722700 1 -- 172.21.15.112:0/2511431926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22c067e90 con 0x7ff22c06e4c0 2023-08-27T11:17:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.132+0000 7ff222ffd700 1 -- 172.21.15.112:0/2511431926 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff228011ca0 con 0x7ff22c06e4c0 2023-08-27T11:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.132+0000 7ff222ffd700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff21006a0e0 0x7ff21006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.132+0000 7ff222ffd700 1 -- 172.21.15.112:0/2511431926 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff228089d90 con 0x7ff22c06e4c0 2023-08-27T11:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.132+0000 7ff231cbd700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff21006a0e0 0x7ff21006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.133+0000 7ff231cbd700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff21006a0e0 0x7ff21006c5a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff22c152090 tx=0x7ff21c007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.136+0000 7ff222ffd700 1 -- 172.21.15.112:0/2511431926 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff228055510 con 0x7ff22c06e4c0 2023-08-27T11:17:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.326+0000 7ff234722700 1 -- 172.21.15.112:0/2511431926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff22c067a20 con 0x7ff22c06e4c0 2023-08-27T11:17:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.326+0000 7ff222ffd700 1 -- 172.21.15.112:0/2511431926 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff228058b30 con 0x7ff22c06e4c0 2023-08-27T11:17:38.328 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.328+0000 7ff234722700 1 -- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff21006a0e0 msgr2=0x7ff21006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.328+0000 7ff234722700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff21006a0e0 0x7ff21006c5a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff22c152090 tx=0x7ff21c007de0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.328+0000 7ff234722700 1 -- 172.21.15.112:0/2511431926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff22c06e4c0 msgr2=0x7ff22c154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.328+0000 7ff234722700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff22c06e4c0 0x7ff22c154ff0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff22c06ac50 tx=0x7ff2280047c0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.328+0000 7ff234722700 1 -- 172.21.15.112:0/2511431926 shutdown_connections 2023-08-27T11:17:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.328+0000 7ff234722700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff21006a0e0 0x7ff21006c5a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.328+0000 7ff234722700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff22c143390 0x7ff22c135f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.328+0000 7ff234722700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff22c06e4c0 0x7ff22c154ff0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.328+0000 7ff234722700 1 --2- 172.21.15.112:0/2511431926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff22c06f820 0x7ff22c155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.328+0000 7ff234722700 1 -- 172.21.15.112:0/2511431926 >> 172.21.15.112:0/2511431926 conn(0x7ff22c0cc240 msgr2=0x7ff22c0af500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:38.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.328+0000 7ff234722700 1 -- 172.21.15.112:0/2511431926 shutdown_connections 2023-08-27T11:17:38.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.328+0000 7ff234722700 1 -- 172.21.15.112:0/2511431926 wait complete. 2023-08-27T11:17:38.342 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:17:38.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.464+0000 7f625116c700 1 Processor -- start 2023-08-27T11:17:38.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.464+0000 7f625116c700 1 -- start start 2023-08-27T11:17:38.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.464+0000 7f625116c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f624c0b3440 0x7f624c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.464+0000 7f625116c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f624c15e6e0 0x7f624c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.464+0000 7f625116c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f624c0b5070 con 0x7f624c146c20 2023-08-27T11:17:38.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.464+0000 7f625116c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624c147000 con 0x7f624c0b3440 2023-08-27T11:17:38.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.464+0000 7f625116c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624c161740 con 0x7f624c15e6e0 2023-08-27T11:17:38.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.465+0000 7f624a59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f624c0b3440 0x7f624c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.465+0000 7f624a59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f624c0b3440 0x7f624c15e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55894/0 (socket says 172.21.15.112:55894) 2023-08-27T11:17:38.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.465+0000 7f624a59c700 1 -- 172.21.15.112:0/2212747432 learned_addr learned my addr 172.21.15.112:0/2212747432 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:38.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.465+0000 7f624b59e700 1 --2- 172.21.15.112:0/2212747432 >> v2:172.21.15.138:3300/0 conn(0x7f624c15e6e0 0x7f624c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.465+0000 7f624a59c700 1 -- 172.21.15.112:0/2212747432 >> v2:172.21.15.138:3300/0 conn(0x7f624c15e6e0 msgr2=0x7f624c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:38.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.465+0000 7f624a59c700 1 --2- 172.21.15.112:0/2212747432 >> v2:172.21.15.138:3300/0 conn(0x7f624c15e6e0 0x7f624c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.465+0000 7f624a59c700 1 -- 172.21.15.112:0/2212747432 >> v1:172.21.15.138:6789/0 conn(0x7f624c146c20 legacy=0x7f624c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:38.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.465+0000 7f624a59c700 1 -- 172.21.15.112:0/2212747432 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624c161a20 con 0x7f624c0b3440 2023-08-27T11:17:38.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.465+0000 7f624a59c700 1 --2- 172.21.15.112:0/2212747432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f624c0b3440 0x7f624c15e1a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f624c0b38c0 tx=0x7f623c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee10af5b5ab8c09 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:38.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.465+0000 7f6249d9b700 1 -- 172.21.15.112:0/2212747432 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c004230 con 0x7f624c0b3440 2023-08-27T11:17:38.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.465+0000 7f6249d9b700 1 -- 172.21.15.112:0/2212747432 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c004390 con 0x7f624c0b3440 2023-08-27T11:17:38.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.465+0000 7f6249d9b700 1 -- 172.21.15.112:0/2212747432 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c0036a0 con 0x7f624c0b3440 2023-08-27T11:17:38.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.466+0000 7f625116c700 1 -- 172.21.15.112:0/2212747432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f624c0b3440 msgr2=0x7f624c15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.466+0000 7f625116c700 1 --2- 172.21.15.112:0/2212747432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f624c0b3440 0x7f624c15e1a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f624c0b38c0 tx=0x7f623c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.466+0000 7f625116c700 1 -- 172.21.15.112:0/2212747432 shutdown_connections 2023-08-27T11:17:38.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.466+0000 7f625116c700 1 --2- 172.21.15.112:0/2212747432 >> v2:172.21.15.138:3300/0 conn(0x7f624c15e6e0 0x7f624c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.466+0000 7f625116c700 1 --2- 172.21.15.112:0/2212747432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f624c0b3440 0x7f624c15e1a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.466+0000 7f625116c700 1 -- 172.21.15.112:0/2212747432 >> 172.21.15.112:0/2212747432 conn(0x7f624c0b1e90 msgr2=0x7f624c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:38.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.466+0000 7f625116c700 1 -- 172.21.15.112:0/2212747432 shutdown_connections 2023-08-27T11:17:38.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.466+0000 7f625116c700 1 -- 172.21.15.112:0/2212747432 wait complete. 2023-08-27T11:17:38.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.467+0000 7f625116c700 1 Processor -- start 2023-08-27T11:17:38.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.467+0000 7f625116c700 1 -- start start 2023-08-27T11:17:38.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f625116c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f624c0b3440 0x7f624c15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f625116c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f624c146c20 0x7f624c142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f625116c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f624c15e6e0 0x7f624c143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f625116c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624c0aef80 con 0x7f624c0b3440 2023-08-27T11:17:38.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f625116c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624c1f33f0 con 0x7f624c15e6e0 2023-08-27T11:17:38.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f625116c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624c1f36d0 con 0x7f624c146c20 2023-08-27T11:17:38.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f624ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f624c0b3440 0x7f624c15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f624ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f624c0b3440 0x7f624c15aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55898/0 (socket says 172.21.15.112:55898) 2023-08-27T11:17:38.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f624ad9d700 1 -- 172.21.15.112:0/4254023022 learned_addr learned my addr 172.21.15.112:0/4254023022 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:38.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f624a59c700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f624c146c20 0x7f624c142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f624b59e700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f624c15e6e0 0x7f624c143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f624ad9d700 1 -- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f624c146c20 msgr2=0x7f624c142b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f624ad9d700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f624c146c20 0x7f624c142b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f624ad9d700 1 -- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f624c15e6e0 msgr2=0x7f624c143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f624ad9d700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f624c15e6e0 0x7f624c143070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f624ad9d700 1 -- 172.21.15.112:0/4254023022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c009550 con 0x7f624c0b3440 2023-08-27T11:17:38.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.468+0000 7f624ad9d700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f624c0b3440 0x7f624c15aad0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f624000c4d0 tx=0x7f624000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:38.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.469+0000 7f62337fe700 1 -- 172.21.15.112:0/4254023022 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240011790 con 0x7f624c0b3440 2023-08-27T11:17:38.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.469+0000 7f62337fe700 1 -- 172.21.15.112:0/4254023022 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62400118f0 con 0x7f624c0b3440 2023-08-27T11:17:38.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.469+0000 7f625116c700 1 -- 172.21.15.112:0/4254023022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624c070780 con 0x7f624c0b3440 2023-08-27T11:17:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.469+0000 7f62337fe700 1 -- 172.21.15.112:0/4254023022 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240011ca0 con 0x7f624c0b3440 2023-08-27T11:17:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.469+0000 7f625116c700 1 -- 172.21.15.112:0/4254023022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624c070d40 con 0x7f624c0b3440 2023-08-27T11:17:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.469+0000 7f625116c700 1 -- 172.21.15.112:0/4254023022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624c067e90 con 0x7f624c0b3440 2023-08-27T11:17:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.470+0000 7f62337fe700 1 -- 172.21.15.112:0/4254023022 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f624001c460 con 0x7f624c0b3440 2023-08-27T11:17:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.470+0000 7f62337fe700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f623806a0e0 0x7f623806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.470+0000 7f62337fe700 1 -- 172.21.15.112:0/4254023022 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6240088340 con 0x7f624c0b3440 2023-08-27T11:17:38.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.471+0000 7f624a59c700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f623806a0e0 0x7f623806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.471+0000 7f624a59c700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f623806a0e0 0x7f623806c5a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f623c009130 tx=0x7f623c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:38.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.475+0000 7f62337fe700 1 -- 172.21.15.112:0/4254023022 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6240053ac0 con 0x7f624c0b3440 2023-08-27T11:17:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.662+0000 7f625116c700 1 -- 172.21.15.112:0/4254023022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f624c067a20 con 0x7f624c0b3440 2023-08-27T11:17:38.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.662+0000 7f62337fe700 1 -- 172.21.15.112:0/4254023022 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62400570e0 con 0x7f624c0b3440 2023-08-27T11:17:38.664 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:17:38.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.664+0000 7f625116c700 1 -- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f623806a0e0 msgr2=0x7f623806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.664+0000 7f625116c700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f623806a0e0 0x7f623806c5a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f623c009130 tx=0x7f623c008030 comp rx=0 tx=0).stop 2023-08-27T11:17:38.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.664+0000 7f625116c700 1 -- 172.21.15.112:0/4254023022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f624c0b3440 msgr2=0x7f624c15aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.664+0000 7f625116c700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f624c0b3440 0x7f624c15aad0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f624000c4d0 tx=0x7f624000c890 comp rx=0 tx=0).stop 2023-08-27T11:17:38.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.664+0000 7f625116c700 1 -- 172.21.15.112:0/4254023022 shutdown_connections 2023-08-27T11:17:38.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.664+0000 7f625116c700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f623806a0e0 0x7f623806c5a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.664+0000 7f625116c700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f624c15e6e0 0x7f624c143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.664+0000 7f625116c700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f624c0b3440 0x7f624c15aad0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.664+0000 7f625116c700 1 --2- 172.21.15.112:0/4254023022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f624c146c20 0x7f624c142b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.664+0000 7f625116c700 1 -- 172.21.15.112:0/4254023022 >> 172.21.15.112:0/4254023022 conn(0x7f624c0b1e90 msgr2=0x7f624c140f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:38.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.665+0000 7f625116c700 1 -- 172.21.15.112:0/4254023022 shutdown_connections 2023-08-27T11:17:38.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.665+0000 7f625116c700 1 -- 172.21.15.112:0/4254023022 wait complete. 2023-08-27T11:17:38.678 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:17:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.800+0000 7f57696e0700 1 Processor -- start 2023-08-27T11:17:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.800+0000 7f57696e0700 1 -- start start 2023-08-27T11:17:38.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.801+0000 7f57696e0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5764146bf0 0x7f57640b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.801+0000 7f57696e0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5764146fd0 0x7f57640cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.801+0000 7f57696e0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57640d1500 con 0x7f57640d0290 2023-08-27T11:17:38.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.801+0000 7f57696e0700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57640d0710 con 0x7f5764146fd0 2023-08-27T11:17:38.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.801+0000 7f57696e0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57640d09f0 con 0x7f5764146bf0 2023-08-27T11:17:38.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.801+0000 7f57637fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f57640d0290 0x7f5764070eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50292/0 (socket says 172.21.15.112:50292) 2023-08-27T11:17:38.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.801+0000 7f57637fe700 1 -- 172.21.15.112:0/61468241 learned_addr learned my addr 172.21.15.112:0/61468241 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:38.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.801+0000 7f5762ffd700 1 --2- 172.21.15.112:0/61468241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5764146bf0 0x7f57640b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.801+0000 7f57627fc700 1 --2- 172.21.15.112:0/61468241 >> v2:172.21.15.138:3300/0 conn(0x7f5764146fd0 0x7f57640cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.801+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110779120 0 0) 0x7f57640d1500 con 0x7f57640d0290 2023-08-27T11:17:38.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.801+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f574c0035b0 con 0x7f57640d0290 2023-08-27T11:17:38.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 62245127 0 0) 0x7f574c0035b0 con 0x7f57640d0290 2023-08-27T11:17:38.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5764146bf0 msgr2=0x7f57640b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:38.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 --2- 172.21.15.112:0/61468241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5764146bf0 0x7f57640b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 >> v2:172.21.15.138:3300/0 conn(0x7f5764146fd0 msgr2=0x7f57640cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:38.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 --2- 172.21.15.112:0/61468241 >> v2:172.21.15.138:3300/0 conn(0x7f5764146fd0 0x7f57640cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57640d0cd0 con 0x7f57640d0290 2023-08-27T11:17:38.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f575c0019f0 con 0x7f57640d0290 2023-08-27T11:17:38.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 >> v1:172.21.15.112:6789/0 conn(0x7f57640d0290 legacy=0x7f5764070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 --2- 172.21.15.112:0/61468241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f574c003a00 0x7f574c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 --2- 172.21.15.112:0/61468241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5764146bf0 0x7f574c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 --2- 172.21.15.112:0/61468241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f574c004d90 0x7f574c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640d0710 con 0x7f574c003a00 2023-08-27T11:17:38.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640d09f0 con 0x7f574c004d90 2023-08-27T11:17:38.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c00ba10 con 0x7f5764146bf0 2023-08-27T11:17:38.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5762ffd700 1 --2- 172.21.15.112:0/61468241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f574c003a00 0x7f574c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f57627fc700 1 --2- 172.21.15.112:0/61468241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5764146bf0 0x7f574c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f57637fe700 1 --2- 172.21.15.112:0/61468241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f574c004d90 0x7f574c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5762ffd700 1 -- 172.21.15.112:0/61468241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5764146bf0 msgr2=0x7f574c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:38.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5762ffd700 1 --2- 172.21.15.112:0/61468241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5764146bf0 0x7f574c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5762ffd700 1 -- 172.21.15.112:0/61468241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f574c004d90 msgr2=0x7f574c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:38.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5762ffd700 1 --2- 172.21.15.112:0/61468241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f574c004d90 0x7f574c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5762ffd700 1 -- 172.21.15.112:0/61468241 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c0035b0 con 0x7f574c003a00 2023-08-27T11:17:38.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5762ffd700 1 --2- 172.21.15.112:0/61468241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f574c003a00 0x7f574c003e20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5758009b60 tx=0x7f574c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cdb1bcdd6ce0c6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:38.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575800e6c0 con 0x7f574c003a00 2023-08-27T11:17:38.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575800e820 con 0x7f574c003a00 2023-08-27T11:17:38.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.802+0000 7f5761ffb700 1 -- 172.21.15.112:0/61468241 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575800eb90 con 0x7f574c003a00 2023-08-27T11:17:38.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.803+0000 7f57696e0700 1 -- 172.21.15.112:0/61468241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f574c003a00 msgr2=0x7f574c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.803+0000 7f57696e0700 1 --2- 172.21.15.112:0/61468241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f574c003a00 0x7f574c003e20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5758009b60 tx=0x7f574c005880 comp rx=0 tx=0).stop 2023-08-27T11:17:38.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.803+0000 7f57637fe700 1 -- 172.21.15.112:0/61468241 reap_dead start 2023-08-27T11:17:38.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.803+0000 7f57696e0700 1 -- 172.21.15.112:0/61468241 shutdown_connections 2023-08-27T11:17:38.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.803+0000 7f57696e0700 1 -- 172.21.15.112:0/61468241 >> 172.21.15.112:0/61468241 conn(0x7f576406c450 msgr2=0x7f576406f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:38.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.803+0000 7f57696e0700 1 -- 172.21.15.112:0/61468241 shutdown_connections 2023-08-27T11:17:38.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.803+0000 7f57696e0700 1 -- 172.21.15.112:0/61468241 wait complete. 2023-08-27T11:17:38.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.804+0000 7f57696e0700 1 Processor -- start 2023-08-27T11:17:38.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.804+0000 7f57696e0700 1 -- start start 2023-08-27T11:17:38.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57696e0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f574c003a00 0x7f57641f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57696e0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f576413f3a0 0x7f576413bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57696e0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f576413f820 0x7f57641f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57696e0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57641f4040 con 0x7f576413f3a0 2023-08-27T11:17:38.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57696e0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640b62b0 con 0x7f574c003a00 2023-08-27T11:17:38.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57696e0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57640b6520 con 0x7f576413f820 2023-08-27T11:17:38.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57627fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f576413f3a0 0x7f576413bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57627fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f576413f3a0 0x7f576413bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55920/0 (socket says 172.21.15.112:55920) 2023-08-27T11:17:38.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57627fc700 1 -- 172.21.15.112:0/3198706391 learned_addr learned my addr 172.21.15.112:0/3198706391 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:38.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f5762ffd700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f574c003a00 0x7f57641f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57637fe700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f576413f820 0x7f57641f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57627fc700 1 -- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f576413f820 msgr2=0x7f57641f5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57627fc700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f576413f820 0x7f57641f5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57627fc700 1 -- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f574c003a00 msgr2=0x7f57641f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:38.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57627fc700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f574c003a00 0x7f57641f3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:38.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.805+0000 7f57627fc700 1 -- 172.21.15.112:0/3198706391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57640b6800 con 0x7f576413f3a0 2023-08-27T11:17:38.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.806+0000 7f57627fc700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f576413f3a0 0x7f576413bb70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f57540116d0 tx=0x7f57540119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:38.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.806+0000 7f574b7fe700 1 -- 172.21.15.112:0/3198706391 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5754015870 con 0x7f576413f3a0 2023-08-27T11:17:38.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.806+0000 7f574b7fe700 1 -- 172.21.15.112:0/3198706391 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57540159d0 con 0x7f576413f3a0 2023-08-27T11:17:38.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.806+0000 7f574b7fe700 1 -- 172.21.15.112:0/3198706391 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5754015d60 con 0x7f576413f3a0 2023-08-27T11:17:38.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.806+0000 7f57696e0700 1 -- 172.21.15.112:0/3198706391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57640b6ae0 con 0x7f576413f3a0 2023-08-27T11:17:38.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.806+0000 7f57696e0700 1 -- 172.21.15.112:0/3198706391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57641f6cd0 con 0x7f576413f3a0 2023-08-27T11:17:38.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.807+0000 7f57696e0700 1 -- 172.21.15.112:0/3198706391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57640b6e50 con 0x7f576413f3a0 2023-08-27T11:17:38.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.807+0000 7f574b7fe700 1 -- 172.21.15.112:0/3198706391 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5754013400 con 0x7f576413f3a0 2023-08-27T11:17:38.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.807+0000 7f574b7fe700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f575006a0e0 0x7f575006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:38.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.808+0000 7f574b7fe700 1 -- 172.21.15.112:0/3198706391 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57540887b0 con 0x7f576413f3a0 2023-08-27T11:17:38.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.808+0000 7f5762ffd700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f575006a0e0 0x7f575006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:38.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.808+0000 7f5762ffd700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f575006a0e0 0x7f575006c5a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f57580090b0 tx=0x7f5758008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:38.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.812+0000 7f574b7fe700 1 -- 172.21.15.112:0/3198706391 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5754053ea0 con 0x7f576413f3a0 2023-08-27T11:17:39.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:38.999+0000 7f57696e0700 1 -- 172.21.15.112:0/3198706391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5764067e90 con 0x7f576413f3a0 2023-08-27T11:17:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.000+0000 7f574b7fe700 1 -- 172.21.15.112:0/3198706391 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57540574c0 con 0x7f576413f3a0 2023-08-27T11:17:39.002 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:17:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.001+0000 7f57696e0700 1 -- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f575006a0e0 msgr2=0x7f575006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.001+0000 7f57696e0700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f575006a0e0 0x7f575006c5a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f57580090b0 tx=0x7f5758008030 comp rx=0 tx=0).stop 2023-08-27T11:17:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.002+0000 7f57696e0700 1 -- 172.21.15.112:0/3198706391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f576413f3a0 msgr2=0x7f576413bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.002+0000 7f57696e0700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f576413f3a0 0x7f576413bb70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f57540116d0 tx=0x7f57540119e0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.002+0000 7f57696e0700 1 -- 172.21.15.112:0/3198706391 shutdown_connections 2023-08-27T11:17:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.002+0000 7f57696e0700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f575006a0e0 0x7f575006c5a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.002+0000 7f57696e0700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f576413f820 0x7f57641f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.002+0000 7f57696e0700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f574c003a00 0x7f57641f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.002+0000 7f57696e0700 1 --2- 172.21.15.112:0/3198706391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f576413f3a0 0x7f576413bb70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.002+0000 7f57696e0700 1 -- 172.21.15.112:0/3198706391 >> 172.21.15.112:0/3198706391 conn(0x7f576406c450 msgr2=0x7f576415fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.002+0000 7f57696e0700 1 -- 172.21.15.112:0/3198706391 shutdown_connections 2023-08-27T11:17:39.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.002+0000 7f57696e0700 1 -- 172.21.15.112:0/3198706391 wait complete. 2023-08-27T11:17:39.015 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:17:39.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.137+0000 7f889eaa4700 1 Processor -- start 2023-08-27T11:17:39.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.137+0000 7f889eaa4700 1 -- start start 2023-08-27T11:17:39.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f889eaa4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f88980ce5f0 0x7f889815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:39.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f889eaa4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88980cb7b0 con 0x7f889806c470 2023-08-27T11:17:39.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f889eaa4700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889815d530 con 0x7f88980ce210 2023-08-27T11:17:39.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f889eaa4700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889815ded0 con 0x7f88980ce5f0 2023-08-27T11:17:39.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f8897fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f88980ce5f0 0x7f889815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:39.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f889d041700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f889806c470 0x7f88980ccb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50312/0 (socket says 172.21.15.112:50312) 2023-08-27T11:17:39.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f889d041700 1 -- 172.21.15.112:0/137326581 learned_addr learned my addr 172.21.15.112:0/137326581 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:39.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f8897fff700 1 -- 172.21.15.112:0/137326581 >> v1:172.21.15.138:6789/0 conn(0x7f88980ce210 legacy=0x7f88980cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:39.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f8897fff700 1 -- 172.21.15.112:0/137326581 >> v1:172.21.15.112:6789/0 conn(0x7f889806c470 legacy=0x7f88980ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f8897fff700 1 -- 172.21.15.112:0/137326581 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889815d810 con 0x7f88980ce5f0 2023-08-27T11:17:39.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f88977fe700 1 -- 172.21.15.112:0/137326581 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274075510 0 0) 0x7f88980cb7b0 con 0x7f889806c470 2023-08-27T11:17:39.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f8897fff700 1 --2- 172.21.15.112:0/137326581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980ce5f0 0x7f889815aab0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8884009840 tx=0x7f8884009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efe6497430412be server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:39.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f88977fe700 1 -- 172.21.15.112:0/137326581 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884004230 con 0x7f88980ce5f0 2023-08-27T11:17:39.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f88977fe700 1 -- 172.21.15.112:0/137326581 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8884004390 con 0x7f88980ce5f0 2023-08-27T11:17:39.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.138+0000 7f88977fe700 1 -- 172.21.15.112:0/137326581 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88840036a0 con 0x7f88980ce5f0 2023-08-27T11:17:39.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.139+0000 7f889eaa4700 1 -- 172.21.15.112:0/137326581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980ce5f0 msgr2=0x7f889815aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.139+0000 7f889eaa4700 1 --2- 172.21.15.112:0/137326581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980ce5f0 0x7f889815aab0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8884009840 tx=0x7f8884009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:39.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.139+0000 7f889eaa4700 1 -- 172.21.15.112:0/137326581 shutdown_connections 2023-08-27T11:17:39.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.139+0000 7f889eaa4700 1 --2- 172.21.15.112:0/137326581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980ce5f0 0x7f889815aab0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.139+0000 7f889eaa4700 1 -- 172.21.15.112:0/137326581 >> 172.21.15.112:0/137326581 conn(0x7f88980cef70 msgr2=0x7f88980cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.139+0000 7f889eaa4700 1 -- 172.21.15.112:0/137326581 shutdown_connections 2023-08-27T11:17:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.139+0000 7f889eaa4700 1 -- 172.21.15.112:0/137326581 wait complete. 2023-08-27T11:17:39.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.140+0000 7f889eaa4700 1 Processor -- start 2023-08-27T11:17:39.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.140+0000 7f889eaa4700 1 -- start start 2023-08-27T11:17:39.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889eaa4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f889806c470 0x7f8898071350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:39.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889eaa4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88980ce210 0x7f889814d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:39.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889eaa4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980ce5f0 0x7f8898152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:39.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889eaa4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88981e5650 con 0x7f88980ce5f0 2023-08-27T11:17:39.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889eaa4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8898071890 con 0x7f889806c470 2023-08-27T11:17:39.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889eaa4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88981549b0 con 0x7f88980ce210 2023-08-27T11:17:39.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889d041700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980ce5f0 0x7f8898152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:39.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889d041700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980ce5f0 0x7f8898152cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55936/0 (socket says 172.21.15.112:55936) 2023-08-27T11:17:39.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889d041700 1 -- 172.21.15.112:0/3660215633 learned_addr learned my addr 172.21.15.112:0/3660215633 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:39.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f8897fff700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88980ce210 0x7f889814d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:39.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889c840700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f889806c470 0x7f8898071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:39.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889d041700 1 -- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88980ce210 msgr2=0x7f889814d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889d041700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88980ce210 0x7f889814d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889d041700 1 -- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f889806c470 msgr2=0x7f8898071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889d041700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f889806c470 0x7f8898071350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.141+0000 7f889d041700 1 -- 172.21.15.112:0/3660215633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840094a0 con 0x7f88980ce5f0 2023-08-27T11:17:39.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.142+0000 7f889d041700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980ce5f0 0x7f8898152cd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f889000ee70 tx=0x7f889000c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:39.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.142+0000 7f88957fa700 1 -- 172.21.15.112:0/3660215633 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889000cd00 con 0x7f88980ce5f0 2023-08-27T11:17:39.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.142+0000 7f88957fa700 1 -- 172.21.15.112:0/3660215633 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889000ce60 con 0x7f88980ce5f0 2023-08-27T11:17:39.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.142+0000 7f889eaa4700 1 -- 172.21.15.112:0/3660215633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88981e5090 con 0x7f88980ce5f0 2023-08-27T11:17:39.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.142+0000 7f88957fa700 1 -- 172.21.15.112:0/3660215633 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88900116b0 con 0x7f88980ce5f0 2023-08-27T11:17:39.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.142+0000 7f889eaa4700 1 -- 172.21.15.112:0/3660215633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8898154640 con 0x7f88980ce5f0 2023-08-27T11:17:39.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.143+0000 7f889eaa4700 1 -- 172.21.15.112:0/3660215633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f889815e2d0 con 0x7f88980ce5f0 2023-08-27T11:17:39.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.143+0000 7f88957fa700 1 -- 172.21.15.112:0/3660215633 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8890011810 con 0x7f88980ce5f0 2023-08-27T11:17:39.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.143+0000 7f88957fa700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f887c06a0e0 0x7f887c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:39.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.144+0000 7f88957fa700 1 -- 172.21.15.112:0/3660215633 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f889008b430 con 0x7f88980ce5f0 2023-08-27T11:17:39.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.144+0000 7f889c840700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f887c06a0e0 0x7f887c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:39.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.144+0000 7f889c840700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f887c06a0e0 0x7f887c06c5a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f88981f2b90 tx=0x7f888c007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:39.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.148+0000 7f88957fa700 1 -- 172.21.15.112:0/3660215633 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f889815e2d0 con 0x7f88980ce5f0 2023-08-27T11:17:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.338+0000 7f889eaa4700 1 -- 172.21.15.112:0/3660215633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f889815e2d0 con 0x7f88980ce5f0 2023-08-27T11:17:39.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.341+0000 7f88957fa700 1 -- 172.21.15.112:0/3660215633 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f889815e2d0 con 0x7f88980ce5f0 2023-08-27T11:17:39.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.343+0000 7f889eaa4700 1 -- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f887c06a0e0 msgr2=0x7f887c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.343+0000 7f889eaa4700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f887c06a0e0 0x7f887c06c5a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f88981f2b90 tx=0x7f888c007ec0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.343+0000 7f889eaa4700 1 -- 172.21.15.112:0/3660215633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980ce5f0 msgr2=0x7f8898152cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.343+0000 7f889eaa4700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980ce5f0 0x7f8898152cd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f889000ee70 tx=0x7f889000c550 comp rx=0 tx=0).stop 2023-08-27T11:17:39.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.343+0000 7f889eaa4700 1 -- 172.21.15.112:0/3660215633 shutdown_connections 2023-08-27T11:17:39.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.343+0000 7f889eaa4700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f887c06a0e0 0x7f887c06c5a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.343+0000 7f889eaa4700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980ce5f0 0x7f8898152cd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.343+0000 7f889eaa4700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f889806c470 0x7f8898071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.343+0000 7f889eaa4700 1 --2- 172.21.15.112:0/3660215633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88980ce210 0x7f889814d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.343+0000 7f889eaa4700 1 -- 172.21.15.112:0/3660215633 >> 172.21.15.112:0/3660215633 conn(0x7f88980cef70 msgr2=0x7f88980c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:39.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.343+0000 7f889eaa4700 1 -- 172.21.15.112:0/3660215633 shutdown_connections 2023-08-27T11:17:39.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.343+0000 7f889eaa4700 1 -- 172.21.15.112:0/3660215633 wait complete. 2023-08-27T11:17:39.357 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:17:39.358 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:17:39.392 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:17:39.448 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:17:39.449 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:17:39.449 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:39.466 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:17:39.467 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:39.523 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:17:39.523 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:39.580 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:39.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.739+0000 7f887b5d1700 1 Processor -- start 2023-08-27T11:17:39.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.739+0000 7f887b5d1700 1 -- start start 2023-08-27T11:17:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.740+0000 7f887b5d1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88741578e0 0x7f88740b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.740+0000 7f887b5d1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88740cfb40 con 0x7f8874160f90 2023-08-27T11:17:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.740+0000 7f887b5d1700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88740cfde0 con 0x7f88740cf170 2023-08-27T11:17:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.740+0000 7f887b5d1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88740d00c0 con 0x7f88741578e0 2023-08-27T11:17:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.740+0000 7f8879b6e700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8874160f90 0x7f88740708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50326/0 (socket says 172.21.15.112:50326) 2023-08-27T11:17:39.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.740+0000 7f8879b6e700 1 -- 172.21.15.112:0/4203244191 learned_addr learned my addr 172.21.15.112:0/4203244191 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:39.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f887936d700 1 --2- 172.21.15.112:0/4203244191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88741578e0 0x7f88740b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:39.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128570759 0 0) 0x7f88740cfb40 con 0x7f8874160f90 2023-08-27T11:17:39.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88600035b0 con 0x7f8874160f90 2023-08-27T11:17:39.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3088487112 0 0) 0x7f88600035b0 con 0x7f8874160f90 2023-08-27T11:17:39.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88741578e0 msgr2=0x7f88740b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:39.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 --2- 172.21.15.112:0/4203244191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88741578e0 0x7f88740b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 >> v1:172.21.15.138:6789/0 conn(0x7f88740cf170 legacy=0x7f88740d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88740d03a0 con 0x7f8874160f90 2023-08-27T11:17:39.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f886c0019f0 con 0x7f8874160f90 2023-08-27T11:17:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 >> v1:172.21.15.112:6789/0 conn(0x7f8874160f90 legacy=0x7f88740708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 --2- 172.21.15.112:0/4203244191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8860003a30 0x7f8860003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 --2- 172.21.15.112:0/4203244191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88741578e0 0x7f8860004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 --2- 172.21.15.112:0/4203244191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8860004d90 0x7f8860005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88740d00c0 con 0x7f8860003a30 2023-08-27T11:17:39.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886000bb40 con 0x7f8860004d90 2023-08-27T11:17:39.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886000bd00 con 0x7f88741578e0 2023-08-27T11:17:39.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f887936d700 1 --2- 172.21.15.112:0/4203244191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8860003a30 0x7f8860003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:39.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f8878b6c700 1 --2- 172.21.15.112:0/4203244191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88741578e0 0x7f8860004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:39.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f8879b6e700 1 --2- 172.21.15.112:0/4203244191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8860004d90 0x7f8860005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:39.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f887936d700 1 -- 172.21.15.112:0/4203244191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88741578e0 msgr2=0x7f8860004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:39.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f887936d700 1 --2- 172.21.15.112:0/4203244191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88741578e0 0x7f8860004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f887936d700 1 -- 172.21.15.112:0/4203244191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8860004d90 msgr2=0x7f8860005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:39.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f887936d700 1 --2- 172.21.15.112:0/4203244191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8860004d90 0x7f8860005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.741+0000 7f887936d700 1 -- 172.21.15.112:0/4203244191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88600035b0 con 0x7f8860003a30 2023-08-27T11:17:39.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.742+0000 7f887936d700 1 --2- 172.21.15.112:0/4203244191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8860003a30 0x7f8860003e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8864009b60 tx=0x7f8860005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bba866376ef36e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:39.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.742+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886400e6c0 con 0x7f8860003a30 2023-08-27T11:17:39.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.742+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886400e820 con 0x7f8860003a30 2023-08-27T11:17:39.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.742+0000 7f886bfff700 1 -- 172.21.15.112:0/4203244191 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886400eb90 con 0x7f8860003a30 2023-08-27T11:17:39.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.742+0000 7f887b5d1700 1 -- 172.21.15.112:0/4203244191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8860003a30 msgr2=0x7f8860003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.742+0000 7f887b5d1700 1 --2- 172.21.15.112:0/4203244191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8860003a30 0x7f8860003e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8864009b60 tx=0x7f8860005880 comp rx=0 tx=0).stop 2023-08-27T11:17:39.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.742+0000 7f8879b6e700 1 -- 172.21.15.112:0/4203244191 reap_dead start 2023-08-27T11:17:39.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.742+0000 7f887b5d1700 1 -- 172.21.15.112:0/4203244191 shutdown_connections 2023-08-27T11:17:39.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.742+0000 7f887b5d1700 1 -- 172.21.15.112:0/4203244191 >> 172.21.15.112:0/4203244191 conn(0x7f887406f970 msgr2=0x7f887406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:39.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.742+0000 7f887b5d1700 1 -- 172.21.15.112:0/4203244191 shutdown_connections 2023-08-27T11:17:39.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.742+0000 7f887b5d1700 1 -- 172.21.15.112:0/4203244191 wait complete. 2023-08-27T11:17:39.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.744+0000 7f887b5d1700 1 Processor -- start 2023-08-27T11:17:39.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.744+0000 7f887b5d1700 1 -- start start 2023-08-27T11:17:39.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887b5d1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8860003a30 0x7f887414b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:39.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887b5d1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88741529d0 0x7f8874159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:39.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887b5d1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f887415d0f0 0x7f88741541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:39.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887b5d1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887415ae50 con 0x7f8860003a30 2023-08-27T11:17:39.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887b5d1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887415b530 con 0x7f88741529d0 2023-08-27T11:17:39.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887b5d1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8874152ed0 con 0x7f887415d0f0 2023-08-27T11:17:39.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887936d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8860003a30 0x7f887414b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:39.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887936d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8860003a30 0x7f887414b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55952/0 (socket says 172.21.15.112:55952) 2023-08-27T11:17:39.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887936d700 1 -- 172.21.15.112:0/4113976370 learned_addr learned my addr 172.21.15.112:0/4113976370 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:39.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f8878b6c700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88741529d0 0x7f8874159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:39.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f8879b6e700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f887415d0f0 0x7f88741541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:39.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887936d700 1 -- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f887415d0f0 msgr2=0x7f88741541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887936d700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f887415d0f0 0x7f88741541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887936d700 1 -- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88741529d0 msgr2=0x7f8874159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887936d700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88741529d0 0x7f8874159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887936d700 1 -- 172.21.15.112:0/4113976370 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8874157e40 con 0x7f8860003a30 2023-08-27T11:17:39.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f8879b6e700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f887415d0f0 0x7f88741541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:39.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f8878b6c700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88741529d0 0x7f8874159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:39.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.745+0000 7f887936d700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8860003a30 0x7f887414b690 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f88640090b0 tx=0x7f88640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:39.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.746+0000 7f8869ffb700 1 -- 172.21.15.112:0/4113976370 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886400e6c0 con 0x7f8860003a30 2023-08-27T11:17:39.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.746+0000 7f8869ffb700 1 -- 172.21.15.112:0/4113976370 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8864009ed0 con 0x7f8860003a30 2023-08-27T11:17:39.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.746+0000 7f887b5d1700 1 -- 172.21.15.112:0/4113976370 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887415ae50 con 0x7f8860003a30 2023-08-27T11:17:39.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.746+0000 7f8869ffb700 1 -- 172.21.15.112:0/4113976370 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864003cf0 con 0x7f8860003a30 2023-08-27T11:17:39.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.746+0000 7f887b5d1700 1 -- 172.21.15.112:0/4113976370 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8874153490 con 0x7f8860003a30 2023-08-27T11:17:39.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.747+0000 7f887b5d1700 1 -- 172.21.15.112:0/4113976370 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887415b6e0 con 0x7f8860003a30 2023-08-27T11:17:39.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.747+0000 7f8869ffb700 1 -- 172.21.15.112:0/4113976370 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8864013470 con 0x7f8860003a30 2023-08-27T11:17:39.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.747+0000 7f8869ffb700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f885806a0e0 0x7f885806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:39.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.748+0000 7f8869ffb700 1 -- 172.21.15.112:0/4113976370 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8864089ed0 con 0x7f8860003a30 2023-08-27T11:17:39.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.748+0000 7f8878b6c700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f885806a0e0 0x7f885806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:39.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.748+0000 7f8878b6c700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f885806a0e0 0x7f885806c5a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8870000c00 tx=0x7f8870009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:39.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.752+0000 7f8869ffb700 1 -- 172.21.15.112:0/4113976370 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f887415b6e0 con 0x7f8860003a30 2023-08-27T11:17:39.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.941+0000 7f887b5d1700 1 -- 172.21.15.112:0/4113976370 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f887415b6a0 con 0x7f8860003a30 2023-08-27T11:17:39.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.942+0000 7f8869ffb700 1 -- 172.21.15.112:0/4113976370 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7f8864017020 con 0x7f8860003a30 2023-08-27T11:17:39.944 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:39.945 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:39.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.944+0000 7f887b5d1700 1 -- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f885806a0e0 msgr2=0x7f885806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.944+0000 7f887b5d1700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f885806a0e0 0x7f885806c5a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8870000c00 tx=0x7f8870009040 comp rx=0 tx=0).stop 2023-08-27T11:17:39.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.944+0000 7f887b5d1700 1 -- 172.21.15.112:0/4113976370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8860003a30 msgr2=0x7f887414b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:39.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.944+0000 7f887b5d1700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8860003a30 0x7f887414b690 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f88640090b0 tx=0x7f88640047c0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.944+0000 7f887b5d1700 1 -- 172.21.15.112:0/4113976370 shutdown_connections 2023-08-27T11:17:39.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.944+0000 7f887b5d1700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f885806a0e0 0x7f885806c5a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.944+0000 7f887b5d1700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f887415d0f0 0x7f88741541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.944+0000 7f887b5d1700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8860003a30 0x7f887414b690 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.944+0000 7f887b5d1700 1 --2- 172.21.15.112:0/4113976370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88741529d0 0x7f8874159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:39.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.944+0000 7f887b5d1700 1 -- 172.21.15.112:0/4113976370 >> 172.21.15.112:0/4113976370 conn(0x7f887406f970 msgr2=0x7f88740667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:39.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.945+0000 7f887b5d1700 1 -- 172.21.15.112:0/4113976370 shutdown_connections 2023-08-27T11:17:39.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:39.945+0000 7f887b5d1700 1 -- 172.21.15.112:0/4113976370 wait complete. 2023-08-27T11:17:39.958 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:17:02.484063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:17:03.722976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:17:39.959 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:17:40.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.080+0000 7f52a1a24700 1 Processor -- start 2023-08-27T11:17:40.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.081+0000 7f52a1a24700 1 -- start start 2023-08-27T11:17:40.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.081+0000 7f52a1a24700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f529c0715c0 0x7f529c0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.081+0000 7f52a1a24700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529c0d1500 con 0x7f529c146bf0 2023-08-27T11:17:40.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.081+0000 7f52a1a24700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529c0d0460 con 0x7f529c146fd0 2023-08-27T11:17:40.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.081+0000 7f52a1a24700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c0d0710 con 0x7f529c0715c0 2023-08-27T11:17:40.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.081+0000 7f529affd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f529c146bf0 0x7f529c0cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50342/0 (socket says 172.21.15.112:50342) 2023-08-27T11:17:40.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.081+0000 7f529b7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f529c0715c0 0x7f529c0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.081+0000 7f529affd700 1 -- 172.21.15.112:0/4022956849 learned_addr learned my addr 172.21.15.112:0/4022956849 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:40.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.082+0000 7f529b7fe700 1 -- 172.21.15.112:0/4022956849 >> v1:172.21.15.138:6789/0 conn(0x7f529c146fd0 legacy=0x7f529c070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:40.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.082+0000 7f529b7fe700 1 -- 172.21.15.112:0/4022956849 >> v1:172.21.15.112:6789/0 conn(0x7f529c146bf0 legacy=0x7f529c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.082+0000 7f529b7fe700 1 -- 172.21.15.112:0/4022956849 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c0d09f0 con 0x7f529c0715c0 2023-08-27T11:17:40.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.082+0000 7f5299ffb700 1 -- 172.21.15.112:0/4022956849 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280320285 0 0) 0x7f529c0d1500 con 0x7f529c146bf0 2023-08-27T11:17:40.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.082+0000 7f529b7fe700 1 --2- 172.21.15.112:0/4022956849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f529c0715c0 0x7f529c0b5db0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5294009d90 tx=0x7f529400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da13bd8923b5966e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:40.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.082+0000 7f5299ffb700 1 -- 172.21.15.112:0/4022956849 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400e070 con 0x7f529c0715c0 2023-08-27T11:17:40.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.082+0000 7f5299ffb700 1 -- 172.21.15.112:0/4022956849 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529400bb90 con 0x7f529c0715c0 2023-08-27T11:17:40.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.082+0000 7f5299ffb700 1 -- 172.21.15.112:0/4022956849 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294003a20 con 0x7f529c0715c0 2023-08-27T11:17:40.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.082+0000 7f52a1a24700 1 -- 172.21.15.112:0/4022956849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f529c0715c0 msgr2=0x7f529c0b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.082+0000 7f52a1a24700 1 --2- 172.21.15.112:0/4022956849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f529c0715c0 0x7f529c0b5db0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5294009d90 tx=0x7f529400b330 comp rx=0 tx=0).stop 2023-08-27T11:17:40.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.083+0000 7f52a1a24700 1 -- 172.21.15.112:0/4022956849 shutdown_connections 2023-08-27T11:17:40.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.083+0000 7f52a1a24700 1 --2- 172.21.15.112:0/4022956849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f529c0715c0 0x7f529c0b5db0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.083+0000 7f52a1a24700 1 -- 172.21.15.112:0/4022956849 >> 172.21.15.112:0/4022956849 conn(0x7f529c06c450 msgr2=0x7f529c06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:40.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.083+0000 7f52a1a24700 1 -- 172.21.15.112:0/4022956849 shutdown_connections 2023-08-27T11:17:40.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.083+0000 7f52a1a24700 1 -- 172.21.15.112:0/4022956849 wait complete. 2023-08-27T11:17:40.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.084+0000 7f52a1a24700 1 Processor -- start 2023-08-27T11:17:40.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.084+0000 7f52a1a24700 1 -- start start 2023-08-27T11:17:40.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.085+0000 7f52a1a24700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f529c0715c0 0x7f529c1f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.085+0000 7f52a1a24700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f529c146bf0 0x7f529c13cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.085+0000 7f52a1a24700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f529c146fd0 0x7f529c1f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.085+0000 7f52a1a24700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0b2380 con 0x7f529c0715c0 2023-08-27T11:17:40.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.085+0000 7f52a1a24700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1f6f50 con 0x7f529c146bf0 2023-08-27T11:17:40.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.085+0000 7f52a1a24700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1f7230 con 0x7f529c146fd0 2023-08-27T11:17:40.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.085+0000 7f529affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f529c0715c0 0x7f529c1f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.085+0000 7f529affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f529c0715c0 0x7f529c1f3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55966/0 (socket says 172.21.15.112:55966) 2023-08-27T11:17:40.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.085+0000 7f529affd700 1 -- 172.21.15.112:0/3814195924 learned_addr learned my addr 172.21.15.112:0/3814195924 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:40.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.085+0000 7f529b7fe700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f529c146fd0 0x7f529c1f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.085+0000 7f529a7fc700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f529c146bf0 0x7f529c13cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.086+0000 7f529affd700 1 -- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f529c146fd0 msgr2=0x7f529c1f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.086+0000 7f529affd700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f529c146fd0 0x7f529c1f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.086+0000 7f529affd700 1 -- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f529c146bf0 msgr2=0x7f529c13cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.086+0000 7f529affd700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f529c146bf0 0x7f529c13cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.086+0000 7f529affd700 1 -- 172.21.15.112:0/3814195924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5294009a70 con 0x7f529c0715c0 2023-08-27T11:17:40.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.086+0000 7f529affd700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f529c0715c0 0x7f529c1f3940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f52900011e0 tx=0x7f529000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:40.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.086+0000 7f5283fff700 1 -- 172.21.15.112:0/3814195924 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290004c80 con 0x7f529c0715c0 2023-08-27T11:17:40.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.086+0000 7f5283fff700 1 -- 172.21.15.112:0/3814195924 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5290004e00 con 0x7f529c0715c0 2023-08-27T11:17:40.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.086+0000 7f52a1a24700 1 -- 172.21.15.112:0/3814195924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c1f3140 con 0x7f529c0715c0 2023-08-27T11:17:40.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.086+0000 7f5283fff700 1 -- 172.21.15.112:0/3814195924 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290007310 con 0x7f529c0715c0 2023-08-27T11:17:40.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.086+0000 7f52a1a24700 1 -- 172.21.15.112:0/3814195924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c1f2dc0 con 0x7f529c0715c0 2023-08-27T11:17:40.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.087+0000 7f52a1a24700 1 -- 172.21.15.112:0/3814195924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c1f2fb0 con 0x7f529c0715c0 2023-08-27T11:17:40.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.087+0000 7f5283fff700 1 -- 172.21.15.112:0/3814195924 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5290007470 con 0x7f529c0715c0 2023-08-27T11:17:40.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.088+0000 7f5283fff700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f528806a0e0 0x7f528806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.088+0000 7f5283fff700 1 -- 172.21.15.112:0/3814195924 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5290088120 con 0x7f529c0715c0 2023-08-27T11:17:40.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.088+0000 7f529a7fc700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f528806a0e0 0x7f528806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.088+0000 7f529a7fc700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f528806a0e0 0x7f528806c5a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f529c14d7a0 tx=0x7f528c0090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:40.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.092+0000 7f5283fff700 1 -- 172.21.15.112:0/3814195924 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f52900538a0 con 0x7f529c0715c0 2023-08-27T11:17:40.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.279+0000 7f52a1a24700 1 -- 172.21.15.112:0/3814195924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f529c067e90 con 0x7f529c0715c0 2023-08-27T11:17:40.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.280+0000 7f5283fff700 1 -- 172.21.15.112:0/3814195924 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5290056ec0 con 0x7f529c0715c0 2023-08-27T11:17:40.281 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:40.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.281+0000 7f52a1a24700 1 -- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f528806a0e0 msgr2=0x7f528806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.281+0000 7f52a1a24700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f528806a0e0 0x7f528806c5a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f529c14d7a0 tx=0x7f528c0090e0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.281+0000 7f52a1a24700 1 -- 172.21.15.112:0/3814195924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f529c0715c0 msgr2=0x7f529c1f3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.281+0000 7f52a1a24700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f529c0715c0 0x7f529c1f3940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f52900011e0 tx=0x7f529000ca30 comp rx=0 tx=0).stop 2023-08-27T11:17:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.282+0000 7f52a1a24700 1 -- 172.21.15.112:0/3814195924 shutdown_connections 2023-08-27T11:17:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.282+0000 7f52a1a24700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f528806a0e0 0x7f528806c5a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.282+0000 7f52a1a24700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f529c146fd0 0x7f529c1f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.282+0000 7f52a1a24700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f529c0715c0 0x7f529c1f3940 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.282+0000 7f52a1a24700 1 --2- 172.21.15.112:0/3814195924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f529c146bf0 0x7f529c13cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.282+0000 7f52a1a24700 1 -- 172.21.15.112:0/3814195924 >> 172.21.15.112:0/3814195924 conn(0x7f529c06c450 msgr2=0x7f529c15fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.282+0000 7f52a1a24700 1 -- 172.21.15.112:0/3814195924 shutdown_connections 2023-08-27T11:17:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.282+0000 7f52a1a24700 1 -- 172.21.15.112:0/3814195924 wait complete. 2023-08-27T11:17:40.295 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:40.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.417+0000 7f38e36df700 1 Processor -- start 2023-08-27T11:17:40.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.417+0000 7f38e36df700 1 -- start start 2023-08-27T11:17:40.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e36df700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f38dc1437c0 0x7f38dc0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e36df700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f38dc143ba0 0x7f38dc0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e36df700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38dc0cfd50 con 0x7f38dc06d5b0 2023-08-27T11:17:40.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e36df700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38dc0cf140 con 0x7f38dc1437c0 2023-08-27T11:17:40.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e36df700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38dc0cf420 con 0x7f38dc143ba0 2023-08-27T11:17:40.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e147b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f38dc1437c0 0x7f38dc0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e147b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f38dc1437c0 0x7f38dc0b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55974/0 (socket says 172.21.15.112:55974) 2023-08-27T11:17:40.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e147b700 1 -- 172.21.15.112:0/3750892664 learned_addr learned my addr 172.21.15.112:0/3750892664 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:40.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e0c7a700 1 --2- 172.21.15.112:0/3750892664 >> v2:172.21.15.138:3300/0 conn(0x7f38dc143ba0 0x7f38dc0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e147b700 1 -- 172.21.15.112:0/3750892664 >> v2:172.21.15.138:3300/0 conn(0x7f38dc143ba0 msgr2=0x7f38dc0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e147b700 1 --2- 172.21.15.112:0/3750892664 >> v2:172.21.15.138:3300/0 conn(0x7f38dc143ba0 0x7f38dc0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e147b700 1 -- 172.21.15.112:0/3750892664 >> v1:172.21.15.138:6789/0 conn(0x7f38dc06d5b0 legacy=0x7f38dc15dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e147b700 1 -- 172.21.15.112:0/3750892664 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc0cf700 con 0x7f38dc1437c0 2023-08-27T11:17:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38e147b700 1 --2- 172.21.15.112:0/3750892664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc1437c0 0x7f38dc0b4670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f38dc06e1a0 tx=0x7f38d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64568ba54a7b7bdc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38d3fff700 1 -- 172.21.15.112:0/3750892664 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8004230 con 0x7f38dc1437c0 2023-08-27T11:17:40.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38d3fff700 1 -- 172.21.15.112:0/3750892664 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8004390 con 0x7f38dc1437c0 2023-08-27T11:17:40.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.418+0000 7f38d3fff700 1 -- 172.21.15.112:0/3750892664 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d80036a0 con 0x7f38dc1437c0 2023-08-27T11:17:40.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.419+0000 7f38e36df700 1 -- 172.21.15.112:0/3750892664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc1437c0 msgr2=0x7f38dc0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.419+0000 7f38e36df700 1 --2- 172.21.15.112:0/3750892664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc1437c0 0x7f38dc0b4670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f38dc06e1a0 tx=0x7f38d8009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.419+0000 7f38e36df700 1 -- 172.21.15.112:0/3750892664 shutdown_connections 2023-08-27T11:17:40.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.419+0000 7f38e36df700 1 --2- 172.21.15.112:0/3750892664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc1437c0 0x7f38dc0b4670 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.419+0000 7f38e36df700 1 --2- 172.21.15.112:0/3750892664 >> v2:172.21.15.138:3300/0 conn(0x7f38dc143ba0 0x7f38dc0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.419+0000 7f38e36df700 1 -- 172.21.15.112:0/3750892664 >> 172.21.15.112:0/3750892664 conn(0x7f38dc0b1e90 msgr2=0x7f38dc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:40.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.419+0000 7f38e36df700 1 -- 172.21.15.112:0/3750892664 shutdown_connections 2023-08-27T11:17:40.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.419+0000 7f38e36df700 1 -- 172.21.15.112:0/3750892664 wait complete. 2023-08-27T11:17:40.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.420+0000 7f38e36df700 1 Processor -- start 2023-08-27T11:17:40.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.420+0000 7f38e36df700 1 -- start start 2023-08-27T11:17:40.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e36df700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38dc06d5b0 0x7f38dc0aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e36df700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38dc1437c0 0x7f38dc0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e36df700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc143ba0 0x7f38dc0cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e36df700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc1f3780 con 0x7f38dc143ba0 2023-08-27T11:17:40.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e36df700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0a8520 con 0x7f38dc1437c0 2023-08-27T11:17:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e36df700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0a87c0 con 0x7f38dc06d5b0 2023-08-27T11:17:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e1c7c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc143ba0 0x7f38dc0cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e1c7c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc143ba0 0x7f38dc0cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55984/0 (socket says 172.21.15.112:55984) 2023-08-27T11:17:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e1c7c700 1 -- 172.21.15.112:0/691987958 learned_addr learned my addr 172.21.15.112:0/691987958 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e0c7a700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38dc1437c0 0x7f38dc0c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e147b700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38dc06d5b0 0x7f38dc0aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e1c7c700 1 -- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38dc06d5b0 msgr2=0x7f38dc0aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e1c7c700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38dc06d5b0 0x7f38dc0aa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e1c7c700 1 -- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38dc1437c0 msgr2=0x7f38dc0c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e1c7c700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38dc1437c0 0x7f38dc0c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.421+0000 7f38e1c7c700 1 -- 172.21.15.112:0/691987958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d8009550 con 0x7f38dc143ba0 2023-08-27T11:17:40.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.422+0000 7f38e1c7c700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc143ba0 0x7f38dc0cbd40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f38d400cc10 tx=0x7f38d400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:40.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.422+0000 7f38d1ffb700 1 -- 172.21.15.112:0/691987958 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d400aa60 con 0x7f38dc143ba0 2023-08-27T11:17:40.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.422+0000 7f38d1ffb700 1 -- 172.21.15.112:0/691987958 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d400abc0 con 0x7f38dc143ba0 2023-08-27T11:17:40.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.422+0000 7f38d1ffb700 1 -- 172.21.15.112:0/691987958 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4012420 con 0x7f38dc143ba0 2023-08-27T11:17:40.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.422+0000 7f38e36df700 1 -- 172.21.15.112:0/691987958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38dc0a8aa0 con 0x7f38dc143ba0 2023-08-27T11:17:40.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.422+0000 7f38e36df700 1 -- 172.21.15.112:0/691987958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38dc0cc920 con 0x7f38dc143ba0 2023-08-27T11:17:40.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.423+0000 7f38e36df700 1 -- 172.21.15.112:0/691987958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38dc0a8fc0 con 0x7f38dc143ba0 2023-08-27T11:17:40.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.423+0000 7f38d1ffb700 1 -- 172.21.15.112:0/691987958 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f38d4012580 con 0x7f38dc143ba0 2023-08-27T11:17:40.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.423+0000 7f38d1ffb700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f38c006a0e0 0x7f38c006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.423+0000 7f38d1ffb700 1 -- 172.21.15.112:0/691987958 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38d4088800 con 0x7f38dc143ba0 2023-08-27T11:17:40.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.424+0000 7f38e147b700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f38c006a0e0 0x7f38c006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.424+0000 7f38e147b700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f38c006a0e0 0x7f38c006c5a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009130 tx=0x7f38d8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:40.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.428+0000 7f38d1ffb700 1 -- 172.21.15.112:0/691987958 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f38dc0a8fc0 con 0x7f38dc143ba0 2023-08-27T11:17:40.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.617+0000 7f38e36df700 1 -- 172.21.15.112:0/691987958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f38dc0a8fc0 con 0x7f38dc143ba0 2023-08-27T11:17:40.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.617+0000 7f38d1ffb700 1 -- 172.21.15.112:0/691987958 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38dc0a8fc0 con 0x7f38dc143ba0 2023-08-27T11:17:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:40.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.619+0000 7f38e36df700 1 -- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f38c006a0e0 msgr2=0x7f38c006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.619+0000 7f38e36df700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f38c006a0e0 0x7f38c006c5a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009130 tx=0x7f38d8008030 comp rx=0 tx=0).stop 2023-08-27T11:17:40.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.619+0000 7f38e36df700 1 -- 172.21.15.112:0/691987958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc143ba0 msgr2=0x7f38dc0cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.619+0000 7f38e36df700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc143ba0 0x7f38dc0cbd40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f38d400cc10 tx=0x7f38d400cf20 comp rx=0 tx=0).stop 2023-08-27T11:17:40.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.619+0000 7f38e36df700 1 -- 172.21.15.112:0/691987958 shutdown_connections 2023-08-27T11:17:40.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.619+0000 7f38e36df700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f38c006a0e0 0x7f38c006c5a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.619+0000 7f38e36df700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc143ba0 0x7f38dc0cbd40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.619+0000 7f38e36df700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38dc06d5b0 0x7f38dc0aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.619+0000 7f38e36df700 1 --2- 172.21.15.112:0/691987958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38dc1437c0 0x7f38dc0c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.619+0000 7f38e36df700 1 -- 172.21.15.112:0/691987958 >> 172.21.15.112:0/691987958 conn(0x7f38dc0b1e90 msgr2=0x7f38dc15e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.619+0000 7f38e36df700 1 -- 172.21.15.112:0/691987958 shutdown_connections 2023-08-27T11:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.619+0000 7f38e36df700 1 -- 172.21.15.112:0/691987958 wait complete. 2023-08-27T11:17:40.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:17:40.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.754+0000 7f1a159c9700 1 Processor -- start 2023-08-27T11:17:40.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.754+0000 7f1a159c9700 1 -- start start 2023-08-27T11:17:40.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.755+0000 7f1a159c9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1a1015af90 0x7f1a1015e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.755+0000 7f1a159c9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1015b370 con 0x7f1a100d1a60 2023-08-27T11:17:40.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.755+0000 7f1a159c9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a100b6e70 con 0x7f1a10071360 2023-08-27T11:17:40.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.755+0000 7f1a159c9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a100b7150 con 0x7f1a1015af90 2023-08-27T11:17:40.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.755+0000 7f1a0effd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1a1015af90 0x7f1a1015e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.755+0000 7f1a0e7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1a100d1a60 0x7f1a100b6680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50372/0 (socket says 172.21.15.112:50372) 2023-08-27T11:17:40.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.755+0000 7f1a0e7fc700 1 -- 172.21.15.112:0/2343303606 learned_addr learned my addr 172.21.15.112:0/2343303606 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:40.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.756+0000 7f1a0effd700 1 -- 172.21.15.112:0/2343303606 >> v1:172.21.15.138:6789/0 conn(0x7f1a10071360 legacy=0x7f1a10071760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.756+0000 7f1a0effd700 1 -- 172.21.15.112:0/2343303606 >> v1:172.21.15.112:6789/0 conn(0x7f1a100d1a60 legacy=0x7f1a100b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.756+0000 7f1a0effd700 1 -- 172.21.15.112:0/2343303606 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1015dc70 con 0x7f1a1015af90 2023-08-27T11:17:40.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.756+0000 7f1a0effd700 1 --2- 172.21.15.112:0/2343303606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a1015af90 0x7f1a1015e470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009840 tx=0x7f1a04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691b8707f80fd45f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:40.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.756+0000 7f1a0dffb700 1 -- 172.21.15.112:0/2343303606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a04011830 con 0x7f1a1015af90 2023-08-27T11:17:40.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.756+0000 7f1a0dffb700 1 -- 172.21.15.112:0/2343303606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a04011990 con 0x7f1a1015af90 2023-08-27T11:17:40.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.756+0000 7f1a0dffb700 1 -- 172.21.15.112:0/2343303606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a04011d20 con 0x7f1a1015af90 2023-08-27T11:17:40.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.756+0000 7f1a159c9700 1 -- 172.21.15.112:0/2343303606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a1015af90 msgr2=0x7f1a1015e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.756+0000 7f1a159c9700 1 --2- 172.21.15.112:0/2343303606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a1015af90 0x7f1a1015e470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009840 tx=0x7f1a04009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:40.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.757+0000 7f1a159c9700 1 -- 172.21.15.112:0/2343303606 shutdown_connections 2023-08-27T11:17:40.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.757+0000 7f1a159c9700 1 --2- 172.21.15.112:0/2343303606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a1015af90 0x7f1a1015e470 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.757+0000 7f1a159c9700 1 -- 172.21.15.112:0/2343303606 >> 172.21.15.112:0/2343303606 conn(0x7f1a10066010 msgr2=0x7f1a100663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:40.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.757+0000 7f1a159c9700 1 -- 172.21.15.112:0/2343303606 shutdown_connections 2023-08-27T11:17:40.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.757+0000 7f1a159c9700 1 -- 172.21.15.112:0/2343303606 wait complete. 2023-08-27T11:17:40.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.758+0000 7f1a159c9700 1 Processor -- start 2023-08-27T11:17:40.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.758+0000 7f1a159c9700 1 -- start start 2023-08-27T11:17:40.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a159c9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a10071360 0x7f1a101f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a159c9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a100d1a60 0x7f1a101f4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a159c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a1015af90 0x7f1a10144b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a159c9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a101f5810 con 0x7f1a100d1a60 2023-08-27T11:17:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a159c9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a101f4960 con 0x7f1a1015af90 2023-08-27T11:17:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a159c9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a101f4c00 con 0x7f1a10071360 2023-08-27T11:17:40.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a0e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a100d1a60 0x7f1a101f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a0e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a100d1a60 0x7f1a101f4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55996/0 (socket says 172.21.15.112:55996) 2023-08-27T11:17:40.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a0e7fc700 1 -- 172.21.15.112:0/3653304668 learned_addr learned my addr 172.21.15.112:0/3653304668 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:40.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a0effd700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a10071360 0x7f1a101f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a0f7fe700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a1015af90 0x7f1a10144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a0e7fc700 1 -- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a10071360 msgr2=0x7f1a101f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a0e7fc700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a10071360 0x7f1a101f33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a0e7fc700 1 -- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a1015af90 msgr2=0x7f1a10144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a0e7fc700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a1015af90 0x7f1a10144b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a0e7fc700 1 -- 172.21.15.112:0/3653304668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a040094a0 con 0x7f1a100d1a60 2023-08-27T11:17:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a0f7fe700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a1015af90 0x7f1a10144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a0e7fc700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a100d1a60 0x7f1a101f4420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000c520 tx=0x7f1a0000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:40.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f19f7fff700 1 -- 172.21.15.112:0/3653304668 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00005d70 con 0x7f1a100d1a60 2023-08-27T11:17:40.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f19f7fff700 1 -- 172.21.15.112:0/3653304668 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a000070d0 con 0x7f1a100d1a60 2023-08-27T11:17:40.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a159c9700 1 -- 172.21.15.112:0/3653304668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a101f4ee0 con 0x7f1a100d1a60 2023-08-27T11:17:40.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f19f7fff700 1 -- 172.21.15.112:0/3653304668 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00007440 con 0x7f1a100d1a60 2023-08-27T11:17:40.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.759+0000 7f1a159c9700 1 -- 172.21.15.112:0/3653304668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a101f54a0 con 0x7f1a100d1a60 2023-08-27T11:17:40.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.760+0000 7f1a159c9700 1 -- 172.21.15.112:0/3653304668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a101f5980 con 0x7f1a100d1a60 2023-08-27T11:17:40.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.761+0000 7f19f7fff700 1 -- 172.21.15.112:0/3653304668 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1a000075a0 con 0x7f1a100d1a60 2023-08-27T11:17:40.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.761+0000 7f19f7fff700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f19fc06a0e0 0x7f19fc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:40.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.761+0000 7f19f7fff700 1 -- 172.21.15.112:0/3653304668 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a000878b0 con 0x7f1a100d1a60 2023-08-27T11:17:40.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.761+0000 7f1a0effd700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f19fc06a0e0 0x7f19fc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:40.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.762+0000 7f1a0effd700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f19fc06a0e0 0x7f19fc06c5a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1a040090b0 tx=0x7f1a0401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:40.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.765+0000 7f19f7fff700 1 -- 172.21.15.112:0/3653304668 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1a00053060 con 0x7f1a100d1a60 2023-08-27T11:17:40.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.952+0000 7f1a159c9700 1 -- 172.21.15.112:0/3653304668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a10067a20 con 0x7f1a100d1a60 2023-08-27T11:17:40.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.952+0000 7f19f7fff700 1 -- 172.21.15.112:0/3653304668 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1a00056680 con 0x7f1a100d1a60 2023-08-27T11:17:40.954 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:17:40.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.955+0000 7f1a159c9700 1 -- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f19fc06a0e0 msgr2=0x7f19fc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.955+0000 7f1a159c9700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f19fc06a0e0 0x7f19fc06c5a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1a040090b0 tx=0x7f1a0401c040 comp rx=0 tx=0).stop 2023-08-27T11:17:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.955+0000 7f1a159c9700 1 -- 172.21.15.112:0/3653304668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a100d1a60 msgr2=0x7f1a101f4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.955+0000 7f1a159c9700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a100d1a60 0x7f1a101f4420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000c520 tx=0x7f1a0000c830 comp rx=0 tx=0).stop 2023-08-27T11:17:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.955+0000 7f1a159c9700 1 -- 172.21.15.112:0/3653304668 shutdown_connections 2023-08-27T11:17:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.955+0000 7f1a159c9700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f19fc06a0e0 0x7f19fc06c5a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.955+0000 7f1a159c9700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a1015af90 0x7f1a10144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.955+0000 7f1a159c9700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a10071360 0x7f1a101f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.955+0000 7f1a159c9700 1 --2- 172.21.15.112:0/3653304668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a100d1a60 0x7f1a101f4420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:40.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.955+0000 7f1a159c9700 1 -- 172.21.15.112:0/3653304668 >> 172.21.15.112:0/3653304668 conn(0x7f1a10066010 msgr2=0x7f1a100ce2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:40.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.955+0000 7f1a159c9700 1 -- 172.21.15.112:0/3653304668 shutdown_connections 2023-08-27T11:17:40.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:40.955+0000 7f1a159c9700 1 -- 172.21.15.112:0/3653304668 wait complete. 2023-08-27T11:17:40.968 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:17:41.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.090+0000 7fb1822fa700 1 Processor -- start 2023-08-27T11:17:41.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.090+0000 7fb1822fa700 1 -- start start 2023-08-27T11:17:41.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.091+0000 7fb1822fa700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb17c06fea0 0x7fb17c0b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:41.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.091+0000 7fb1822fa700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17c0b0c30 con 0x7fb17c15d3f0 2023-08-27T11:17:41.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.091+0000 7fb1822fa700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17c0add00 con 0x7fb17c0b1350 2023-08-27T11:17:41.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.091+0000 7fb1822fa700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb17c0b3a10 con 0x7fb17c06fea0 2023-08-27T11:17:41.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.091+0000 7fb180897700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb17c06fea0 0x7fb17c0b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:41.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.091+0000 7fb180897700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb17c06fea0 0x7fb17c0b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56006/0 (socket says 172.21.15.112:56006) 2023-08-27T11:17:41.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.091+0000 7fb17bfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb17c15d3f0 0x7fb17c0ad5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50384/0 (socket says 172.21.15.112:50384) 2023-08-27T11:17:41.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.091+0000 7fb180897700 1 -- 172.21.15.112:0/2648359404 learned_addr learned my addr 172.21.15.112:0/2648359404 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:41.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.091+0000 7fb180897700 1 -- 172.21.15.112:0/2648359404 >> v1:172.21.15.138:6789/0 conn(0x7fb17c0b1350 legacy=0x7fb17c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:41.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.091+0000 7fb180897700 1 -- 172.21.15.112:0/2648359404 >> v1:172.21.15.112:6789/0 conn(0x7fb17c15d3f0 legacy=0x7fb17c0ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:41.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.091+0000 7fb180897700 1 -- 172.21.15.112:0/2648359404 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c161d60 con 0x7fb17c06fea0 2023-08-27T11:17:41.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.091+0000 7fb17affd700 1 -- 172.21.15.112:0/2648359404 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736303768 0 0) 0x7fb17c0b0c30 con 0x7fb17c15d3f0 2023-08-27T11:17:41.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.091+0000 7fb180897700 1 --2- 172.21.15.112:0/2648359404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb17c06fea0 0x7fb17c0b44b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb174009e80 tx=0x7fb17400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a80985e174b4b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:41.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.092+0000 7fb17affd700 1 -- 172.21.15.112:0/2648359404 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17400e070 con 0x7fb17c06fea0 2023-08-27T11:17:41.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.092+0000 7fb17affd700 1 -- 172.21.15.112:0/2648359404 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb174003a20 con 0x7fb17c06fea0 2023-08-27T11:17:41.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.092+0000 7fb17affd700 1 -- 172.21.15.112:0/2648359404 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174003dd0 con 0x7fb17c06fea0 2023-08-27T11:17:41.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.092+0000 7fb1822fa700 1 -- 172.21.15.112:0/2648359404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb17c06fea0 msgr2=0x7fb17c0b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:41.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.092+0000 7fb1822fa700 1 --2- 172.21.15.112:0/2648359404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb17c06fea0 0x7fb17c0b44b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb174009e80 tx=0x7fb17400b330 comp rx=0 tx=0).stop 2023-08-27T11:17:41.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.092+0000 7fb1822fa700 1 -- 172.21.15.112:0/2648359404 shutdown_connections 2023-08-27T11:17:41.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.092+0000 7fb1822fa700 1 --2- 172.21.15.112:0/2648359404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb17c06fea0 0x7fb17c0b44b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.092+0000 7fb1822fa700 1 -- 172.21.15.112:0/2648359404 >> 172.21.15.112:0/2648359404 conn(0x7fb17c0b1e90 msgr2=0x7fb17c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:41.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.092+0000 7fb1822fa700 1 -- 172.21.15.112:0/2648359404 shutdown_connections 2023-08-27T11:17:41.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.092+0000 7fb1822fa700 1 -- 172.21.15.112:0/2648359404 wait complete. 2023-08-27T11:17:41.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.093+0000 7fb1822fa700 1 Processor -- start 2023-08-27T11:17:41.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.094+0000 7fb1822fa700 1 -- start start 2023-08-27T11:17:41.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.094+0000 7fb1822fa700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb17c06fea0 0x7fb17c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:41.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.094+0000 7fb1822fa700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb17c0b1350 0x7fb17c153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:41.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.094+0000 7fb1822fa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb17c15d3f0 0x7fb17c14fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:41.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.094+0000 7fb1822fa700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c156790 con 0x7fb17c15d3f0 2023-08-27T11:17:41.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.094+0000 7fb1822fa700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c150080 con 0x7fb17c06fea0 2023-08-27T11:17:41.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.094+0000 7fb1822fa700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c160640 con 0x7fb17c0b1350 2023-08-27T11:17:41.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.094+0000 7fb180897700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb17c15d3f0 0x7fb17c14fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:41.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.094+0000 7fb180897700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb17c15d3f0 0x7fb17c14fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56012/0 (socket says 172.21.15.112:56012) 2023-08-27T11:17:41.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.094+0000 7fb180897700 1 -- 172.21.15.112:0/984468598 learned_addr learned my addr 172.21.15.112:0/984468598 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:41.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.094+0000 7fb17b7fe700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb17c0b1350 0x7fb17c153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:41.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.094+0000 7fb17bfff700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb17c06fea0 0x7fb17c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:41.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.095+0000 7fb180897700 1 -- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb17c0b1350 msgr2=0x7fb17c153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:41.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.095+0000 7fb180897700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb17c0b1350 0x7fb17c153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.095+0000 7fb180897700 1 -- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb17c06fea0 msgr2=0x7fb17c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:41.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.095+0000 7fb180897700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb17c06fea0 0x7fb17c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.095+0000 7fb180897700 1 -- 172.21.15.112:0/984468598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb174009b30 con 0x7fb17c15d3f0 2023-08-27T11:17:41.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.095+0000 7fb180897700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb17c15d3f0 0x7fb17c14fb40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb174004000 tx=0x7fb174004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:41.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.095+0000 7fb178ff9700 1 -- 172.21.15.112:0/984468598 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1740094d0 con 0x7fb17c15d3f0 2023-08-27T11:17:41.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.095+0000 7fb178ff9700 1 -- 172.21.15.112:0/984468598 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb174004240 con 0x7fb17c15d3f0 2023-08-27T11:17:41.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.095+0000 7fb1822fa700 1 -- 172.21.15.112:0/984468598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17c160920 con 0x7fb17c15d3f0 2023-08-27T11:17:41.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.095+0000 7fb178ff9700 1 -- 172.21.15.112:0/984468598 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1740045f0 con 0x7fb17c15d3f0 2023-08-27T11:17:41.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.095+0000 7fb1822fa700 1 -- 172.21.15.112:0/984468598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c156c80 con 0x7fb17c15d3f0 2023-08-27T11:17:41.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.096+0000 7fb1822fa700 1 -- 172.21.15.112:0/984468598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17c160b30 con 0x7fb17c15d3f0 2023-08-27T11:17:41.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.096+0000 7fb178ff9700 1 -- 172.21.15.112:0/984468598 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb174019070 con 0x7fb17c15d3f0 2023-08-27T11:17:41.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.097+0000 7fb178ff9700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb16806a0e0 0x7fb16806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.097+0000 7fb178ff9700 1 -- 172.21.15.112:0/984468598 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb1740893d0 con 0x7fb17c15d3f0 2023-08-27T11:17:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.097+0000 7fb17bfff700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb16806a0e0 0x7fb16806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.097+0000 7fb17bfff700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb16806a0e0 0x7fb16806c5a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb170007fd0 tx=0x7fb170007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.101+0000 7fb178ff9700 1 -- 172.21.15.112:0/984468598 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb174054b50 con 0x7fb17c15d3f0 2023-08-27T11:17:41.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.291+0000 7fb1822fa700 1 -- 172.21.15.112:0/984468598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb17c0b0dc0 con 0x7fb17c15d3f0 2023-08-27T11:17:41.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.291+0000 7fb178ff9700 1 -- 172.21.15.112:0/984468598 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb174058170 con 0x7fb17c15d3f0 2023-08-27T11:17:41.293 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:17:41.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.293+0000 7fb1822fa700 1 -- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb16806a0e0 msgr2=0x7fb16806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:41.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.293+0000 7fb1822fa700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb16806a0e0 0x7fb16806c5a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb170007fd0 tx=0x7fb170007de0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.293+0000 7fb1822fa700 1 -- 172.21.15.112:0/984468598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb17c15d3f0 msgr2=0x7fb17c14fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:41.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.293+0000 7fb1822fa700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb17c15d3f0 0x7fb17c14fb40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb174004000 tx=0x7fb174004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.293+0000 7fb1822fa700 1 -- 172.21.15.112:0/984468598 shutdown_connections 2023-08-27T11:17:41.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.293+0000 7fb1822fa700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb16806a0e0 0x7fb16806c5a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.293+0000 7fb1822fa700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb17c15d3f0 0x7fb17c14fb40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.293+0000 7fb1822fa700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb17c06fea0 0x7fb17c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.293+0000 7fb1822fa700 1 --2- 172.21.15.112:0/984468598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb17c0b1350 0x7fb17c153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.293+0000 7fb1822fa700 1 -- 172.21.15.112:0/984468598 >> 172.21.15.112:0/984468598 conn(0x7fb17c0b1e90 msgr2=0x7fb17c0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:41.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.293+0000 7fb1822fa700 1 -- 172.21.15.112:0/984468598 shutdown_connections 2023-08-27T11:17:41.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.293+0000 7fb1822fa700 1 -- 172.21.15.112:0/984468598 wait complete. 2023-08-27T11:17:41.307 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:17:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.428+0000 7f8c459d2700 1 Processor -- start 2023-08-27T11:17:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.428+0000 7f8c459d2700 1 -- start start 2023-08-27T11:17:41.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.429+0000 7f8c459d2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c40141420 0x7f8c400aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:41.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.429+0000 7f8c459d2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8c4006e150 0x7f8c4006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:41.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.429+0000 7f8c459d2700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c400ab440 con 0x7f8c400cc070 2023-08-27T11:17:41.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.429+0000 7f8c459d2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4006eb10 con 0x7f8c4006e150 2023-08-27T11:17:41.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.429+0000 7f8c459d2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4015eed0 con 0x7f8c40141420 2023-08-27T11:17:41.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.429+0000 7f8c3e7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8c4006e150 0x7f8c4006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:41.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.429+0000 7f8c3e7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8c4006e150 0x7f8c4006e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56016/0 (socket says 172.21.15.112:56016) 2023-08-27T11:17:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.429+0000 7f8c3e7fc700 1 -- 172.21.15.112:0/1191368150 learned_addr learned my addr 172.21.15.112:0/1191368150 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.429+0000 7f8c3effd700 1 --2- 172.21.15.112:0/1191368150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c40141420 0x7f8c400aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c3e7fc700 1 -- 172.21.15.112:0/1191368150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c40141420 msgr2=0x7f8c400aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c3e7fc700 1 --2- 172.21.15.112:0/1191368150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c40141420 0x7f8c400aaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c3e7fc700 1 -- 172.21.15.112:0/1191368150 >> v1:172.21.15.138:6789/0 conn(0x7f8c400cc070 legacy=0x7f8c400cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:41.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c3e7fc700 1 -- 172.21.15.112:0/1191368150 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4015f1b0 con 0x7f8c4006e150 2023-08-27T11:17:41.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c3effd700 1 --2- 172.21.15.112:0/1191368150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c40141420 0x7f8c400aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:41.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c3e7fc700 1 --2- 172.21.15.112:0/1191368150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4006e150 0x7f8c4006e5d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8c30006fd0 tx=0x7f8c30009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652110ac271885b2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:41.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c3dffb700 1 -- 172.21.15.112:0/1191368150 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30005e30 con 0x7f8c4006e150 2023-08-27T11:17:41.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c3dffb700 1 -- 172.21.15.112:0/1191368150 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c30004030 con 0x7f8c4006e150 2023-08-27T11:17:41.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c3dffb700 1 -- 172.21.15.112:0/1191368150 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300043e0 con 0x7f8c4006e150 2023-08-27T11:17:41.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c459d2700 1 -- 172.21.15.112:0/1191368150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4006e150 msgr2=0x7f8c4006e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:41.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c459d2700 1 --2- 172.21.15.112:0/1191368150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4006e150 0x7f8c4006e5d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8c30006fd0 tx=0x7f8c30009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c459d2700 1 -- 172.21.15.112:0/1191368150 shutdown_connections 2023-08-27T11:17:41.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c459d2700 1 --2- 172.21.15.112:0/1191368150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c40141420 0x7f8c400aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c459d2700 1 --2- 172.21.15.112:0/1191368150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4006e150 0x7f8c4006e5d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c459d2700 1 -- 172.21.15.112:0/1191368150 >> 172.21.15.112:0/1191368150 conn(0x7f8c4006cd30 msgr2=0x7f8c400717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:41.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c459d2700 1 -- 172.21.15.112:0/1191368150 shutdown_connections 2023-08-27T11:17:41.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.430+0000 7f8c459d2700 1 -- 172.21.15.112:0/1191368150 wait complete. 2023-08-27T11:17:41.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.432+0000 7f8c459d2700 1 Processor -- start 2023-08-27T11:17:41.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.432+0000 7f8c459d2700 1 -- start start 2023-08-27T11:17:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.432+0000 7f8c459d2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c4006e150 0x7f8c4015ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.432+0000 7f8c459d2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c400cc070 0x7f8c40136570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.432+0000 7f8c459d2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c40141420 0x7f8c401398d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.432+0000 7f8c459d2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40136ee0 con 0x7f8c400cc070 2023-08-27T11:17:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.432+0000 7f8c459d2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40138e10 con 0x7f8c4006e150 2023-08-27T11:17:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.432+0000 7f8c459d2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c401390f0 con 0x7f8c40141420 2023-08-27T11:17:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c3e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c400cc070 0x7f8c40136570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c3e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c400cc070 0x7f8c40136570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56024/0 (socket says 172.21.15.112:56024) 2023-08-27T11:17:41.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c3e7fc700 1 -- 172.21.15.112:0/2839918354 learned_addr learned my addr 172.21.15.112:0/2839918354 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:41.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c3f7fe700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c40141420 0x7f8c401398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:41.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c3effd700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c4006e150 0x7f8c4015ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:41.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c3e7fc700 1 -- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c40141420 msgr2=0x7f8c401398d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:41.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c3e7fc700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c40141420 0x7f8c401398d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c3e7fc700 1 -- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c4006e150 msgr2=0x7f8c4015ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c3e7fc700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c4006e150 0x7f8c4015ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c3e7fc700 1 -- 172.21.15.112:0/2839918354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30009500 con 0x7f8c400cc070 2023-08-27T11:17:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c3f7fe700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c40141420 0x7f8c401398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c3e7fc700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c400cc070 0x7f8c40136570 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009c10 tx=0x7f8c30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:41.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c27fff700 1 -- 172.21.15.112:0/2839918354 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30003d90 con 0x7f8c400cc070 2023-08-27T11:17:41.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c27fff700 1 -- 172.21.15.112:0/2839918354 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c30011420 con 0x7f8c400cc070 2023-08-27T11:17:41.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c459d2700 1 -- 172.21.15.112:0/2839918354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c401393d0 con 0x7f8c400cc070 2023-08-27T11:17:41.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c27fff700 1 -- 172.21.15.112:0/2839918354 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300117d0 con 0x7f8c400cc070 2023-08-27T11:17:41.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.433+0000 7f8c459d2700 1 -- 172.21.15.112:0/2839918354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4015d390 con 0x7f8c400cc070 2023-08-27T11:17:41.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.434+0000 7f8c459d2700 1 -- 172.21.15.112:0/2839918354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4015d580 con 0x7f8c400cc070 2023-08-27T11:17:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.435+0000 7f8c27fff700 1 -- 172.21.15.112:0/2839918354 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8c30011930 con 0x7f8c400cc070 2023-08-27T11:17:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.435+0000 7f8c27fff700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8c2c06a0e0 0x7f8c2c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.435+0000 7f8c27fff700 1 -- 172.21.15.112:0/2839918354 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8c30089800 con 0x7f8c400cc070 2023-08-27T11:17:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.435+0000 7f8c3effd700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8c2c06a0e0 0x7f8c2c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.436+0000 7f8c3effd700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8c2c06a0e0 0x7f8c2c06c5a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009990 tx=0x7f8c34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:41.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.439+0000 7f8c27fff700 1 -- 172.21.15.112:0/2839918354 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8c30054f80 con 0x7f8c400cc070 2023-08-27T11:17:41.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.643+0000 7f8c459d2700 1 -- 172.21.15.112:0/2839918354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8c40067a20 con 0x7f8c400cc070 2023-08-27T11:17:41.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.644+0000 7f8c27fff700 1 -- 172.21.15.112:0/2839918354 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8c3002a080 con 0x7f8c400cc070 2023-08-27T11:17:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.646+0000 7f8c459d2700 1 -- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8c2c06a0e0 msgr2=0x7f8c2c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.646+0000 7f8c459d2700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8c2c06a0e0 0x7f8c2c06c5a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009990 tx=0x7f8c34008040 comp rx=0 tx=0).stop 2023-08-27T11:17:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.646+0000 7f8c459d2700 1 -- 172.21.15.112:0/2839918354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c400cc070 msgr2=0x7f8c40136570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.646+0000 7f8c459d2700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c400cc070 0x7f8c40136570 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009c10 tx=0x7f8c30004750 comp rx=0 tx=0).stop 2023-08-27T11:17:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.646+0000 7f8c459d2700 1 -- 172.21.15.112:0/2839918354 shutdown_connections 2023-08-27T11:17:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.646+0000 7f8c459d2700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8c2c06a0e0 0x7f8c2c06c5a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.646+0000 7f8c459d2700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c40141420 0x7f8c401398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.646+0000 7f8c459d2700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c4006e150 0x7f8c4015ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.646+0000 7f8c459d2700 1 --2- 172.21.15.112:0/2839918354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c400cc070 0x7f8c40136570 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:41.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.646+0000 7f8c459d2700 1 -- 172.21.15.112:0/2839918354 >> 172.21.15.112:0/2839918354 conn(0x7f8c4006cd30 msgr2=0x7f8c4013e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:41.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.646+0000 7f8c459d2700 1 -- 172.21.15.112:0/2839918354 shutdown_connections 2023-08-27T11:17:41.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.646+0000 7f8c459d2700 1 -- 172.21.15.112:0/2839918354 wait complete. 2023-08-27T11:17:41.650 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:17:41.659 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:17:41.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:41.660 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-08-27T11:17:41.711 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:17:41.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:41.712 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:17:41.738 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:17:41.739 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:41.760 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:17:41.760 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:41.821 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:17:41.821 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:41.881 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:42.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.999+0000 7f74602f2700 1 Processor -- start 2023-08-27T11:17:42.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:41.999+0000 7f74602f2700 1 -- start start 2023-08-27T11:17:42.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f74602f2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7458151060 0x7f745815e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f74602f2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f745806b4c0 0x7f74580cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f74602f2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74580b02f0 con 0x7f745815e990 2023-08-27T11:17:42.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f74602f2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74580b05d0 con 0x7f745806b4c0 2023-08-27T11:17:42.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f74602f2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74580b08b0 con 0x7f7458151060 2023-08-27T11:17:42.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f745e88f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f745806b4c0 0x7f74580cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f745d88d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f745815e990 0x7f745806adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50414/0 (socket says 172.21.15.112:50414) 2023-08-27T11:17:42.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f745d88d700 1 -- 172.21.15.112:0/3911818744 learned_addr learned my addr 172.21.15.112:0/3911818744 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:42.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f745e08e700 1 --2- 172.21.15.112:0/3911818744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7458151060 0x7f745815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f745e88f700 1 -- 172.21.15.112:0/3911818744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7458151060 msgr2=0x7f745815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:42.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f745e88f700 1 --2- 172.21.15.112:0/3911818744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7458151060 0x7f745815e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f745e88f700 1 -- 172.21.15.112:0/3911818744 >> v1:172.21.15.112:6789/0 conn(0x7f745815e990 legacy=0x7f745806adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f745e88f700 1 -- 172.21.15.112:0/3911818744 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580b0b90 con 0x7f745806b4c0 2023-08-27T11:17:42.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f745d08c700 1 -- 172.21.15.112:0/3911818744 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612595056 0 0) 0x7f74580b02f0 con 0x7f745815e990 2023-08-27T11:17:42.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f745e88f700 1 --2- 172.21.15.112:0/3911818744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745806b4c0 0x7f74580cdbd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7450009d90 tx=0x7f745000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fba367215c3c14 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:42.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f745d08c700 1 -- 172.21.15.112:0/3911818744 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745000e070 con 0x7f745806b4c0 2023-08-27T11:17:42.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f745d08c700 1 -- 172.21.15.112:0/3911818744 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745000bbd0 con 0x7f745806b4c0 2023-08-27T11:17:42.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.001+0000 7f745d08c700 1 -- 172.21.15.112:0/3911818744 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450012430 con 0x7f745806b4c0 2023-08-27T11:17:42.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.002+0000 7f74602f2700 1 -- 172.21.15.112:0/3911818744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745806b4c0 msgr2=0x7f74580cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.002+0000 7f74602f2700 1 --2- 172.21.15.112:0/3911818744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745806b4c0 0x7f74580cdbd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7450009d90 tx=0x7f745000b330 comp rx=0 tx=0).stop 2023-08-27T11:17:42.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.002+0000 7f74602f2700 1 -- 172.21.15.112:0/3911818744 shutdown_connections 2023-08-27T11:17:42.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.002+0000 7f74602f2700 1 --2- 172.21.15.112:0/3911818744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745806b4c0 0x7f74580cdbd0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.002+0000 7f74602f2700 1 --2- 172.21.15.112:0/3911818744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7458151060 0x7f745815e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.002+0000 7f74602f2700 1 -- 172.21.15.112:0/3911818744 >> 172.21.15.112:0/3911818744 conn(0x7f74580d0000 msgr2=0x7f74580d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:42.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.002+0000 7f74602f2700 1 -- 172.21.15.112:0/3911818744 shutdown_connections 2023-08-27T11:17:42.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.002+0000 7f74602f2700 1 -- 172.21.15.112:0/3911818744 wait complete. 2023-08-27T11:17:42.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.003+0000 7f74602f2700 1 Processor -- start 2023-08-27T11:17:42.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.003+0000 7f74602f2700 1 -- start start 2023-08-27T11:17:42.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f74602f2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745806b4c0 0x7f74580c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f74602f2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7458151060 0x7f7458150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f74602f2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f745815e990 0x7f745814ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f74602f2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74581529d0 con 0x7f745806b4c0 2023-08-27T11:17:42.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f74602f2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580c82f0 con 0x7f745815e990 2023-08-27T11:17:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f74602f2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74580c7a50 con 0x7f7458151060 2023-08-27T11:17:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f745e08e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745806b4c0 0x7f74580c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f745e08e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745806b4c0 0x7f74580c39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56036/0 (socket says 172.21.15.112:56036) 2023-08-27T11:17:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f745e08e700 1 -- 172.21.15.112:0/2495105926 learned_addr learned my addr 172.21.15.112:0/2495105926 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f745e88f700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f745815e990 0x7f745814ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f745d88d700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7458151060 0x7f7458150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f745e08e700 1 -- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7458151060 msgr2=0x7f7458150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f745e08e700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7458151060 0x7f7458150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f745e08e700 1 -- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f745815e990 msgr2=0x7f745814ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f745e08e700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f745815e990 0x7f745814ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f745e08e700 1 -- 172.21.15.112:0/2495105926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7450009a70 con 0x7f745806b4c0 2023-08-27T11:17:42.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f745e88f700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f745815e990 0x7f745814ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:42.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.004+0000 7f745d88d700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7458151060 0x7f7458150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:42.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.005+0000 7f745e08e700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745806b4c0 0x7f74580c39f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f744800e810 tx=0x7f744800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:42.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.005+0000 7f744effd700 1 -- 172.21.15.112:0/2495105926 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448013610 con 0x7f745806b4c0 2023-08-27T11:17:42.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.005+0000 7f744effd700 1 -- 172.21.15.112:0/2495105926 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7448013770 con 0x7f745806b4c0 2023-08-27T11:17:42.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.005+0000 7f74602f2700 1 -- 172.21.15.112:0/2495105926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74580c7d30 con 0x7f745806b4c0 2023-08-27T11:17:42.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.005+0000 7f744effd700 1 -- 172.21.15.112:0/2495105926 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448013b20 con 0x7f745806b4c0 2023-08-27T11:17:42.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.005+0000 7f74602f2700 1 -- 172.21.15.112:0/2495105926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74581525f0 con 0x7f745806b4c0 2023-08-27T11:17:42.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.006+0000 7f74602f2700 1 -- 172.21.15.112:0/2495105926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7458152c40 con 0x7f745806b4c0 2023-08-27T11:17:42.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.006+0000 7f744effd700 1 -- 172.21.15.112:0/2495105926 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7448013c80 con 0x7f745806b4c0 2023-08-27T11:17:42.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.007+0000 7f744effd700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f743c06a0e0 0x7f743c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.007+0000 7f744effd700 1 -- 172.21.15.112:0/2495105926 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7448088700 con 0x7f745806b4c0 2023-08-27T11:17:42.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.007+0000 7f745d88d700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f743c06a0e0 0x7f743c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.008+0000 7f745d88d700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f743c06a0e0 0x7f743c06c5a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7458153da0 tx=0x7f7454009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:42.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.011+0000 7f744effd700 1 -- 172.21.15.112:0/2495105926 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7448053e80 con 0x7f745806b4c0 2023-08-27T11:17:42.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.198+0000 7f74602f2700 1 -- 172.21.15.112:0/2495105926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7458067a20 con 0x7f745806b4c0 2023-08-27T11:17:42.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.199+0000 7f744effd700 1 -- 172.21.15.112:0/2495105926 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7f74480574a0 con 0x7f745806b4c0 2023-08-27T11:17:42.201 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:42.201 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:42.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.201+0000 7f74602f2700 1 -- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f743c06a0e0 msgr2=0x7f743c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.201+0000 7f74602f2700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f743c06a0e0 0x7f743c06c5a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7458153da0 tx=0x7f7454009040 comp rx=0 tx=0).stop 2023-08-27T11:17:42.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.201+0000 7f74602f2700 1 -- 172.21.15.112:0/2495105926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745806b4c0 msgr2=0x7f74580c39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.201+0000 7f74602f2700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745806b4c0 0x7f74580c39f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f744800e810 tx=0x7f744800ebd0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.201+0000 7f74602f2700 1 -- 172.21.15.112:0/2495105926 shutdown_connections 2023-08-27T11:17:42.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.201+0000 7f74602f2700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f743c06a0e0 0x7f743c06c5a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.201+0000 7f74602f2700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f745815e990 0x7f745814ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.201+0000 7f74602f2700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745806b4c0 0x7f74580c39f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.201+0000 7f74602f2700 1 --2- 172.21.15.112:0/2495105926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7458151060 0x7f7458150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.201+0000 7f74602f2700 1 -- 172.21.15.112:0/2495105926 >> 172.21.15.112:0/2495105926 conn(0x7f74580d0000 msgr2=0x7f745815a750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:42.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.202+0000 7f74602f2700 1 -- 172.21.15.112:0/2495105926 shutdown_connections 2023-08-27T11:17:42.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.202+0000 7f74602f2700 1 -- 172.21.15.112:0/2495105926 wait complete. 2023-08-27T11:17:42.215 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:17:02.484063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:17:03.722976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:17:42.216 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:17:42.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.339+0000 7fedd5a64700 1 Processor -- start 2023-08-27T11:17:42.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.339+0000 7fedd5a64700 1 -- start start 2023-08-27T11:17:42.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedd5a64700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedd015aeb0 0x7fedd006b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedd5a64700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fedd006b7f0 0x7fedd00aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedd5a64700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fedd00af340 0x7fedd006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedd5a64700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd00af7f0 con 0x7fedd00af340 2023-08-27T11:17:42.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedd5a64700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd0161aa0 con 0x7fedd006b7f0 2023-08-27T11:17:42.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedd5a64700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd0161170 con 0x7fedd015aeb0 2023-08-27T11:17:42.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedcffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fedd00af340 0x7fedd006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedcffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fedd00af340 0x7fedd006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56048/0 (socket says 172.21.15.112:56048) 2023-08-27T11:17:42.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedcffff700 1 -- 172.21.15.112:0/2180657861 learned_addr learned my addr 172.21.15.112:0/2180657861 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:42.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedceffd700 1 --2- 172.21.15.112:0/2180657861 >> v2:172.21.15.138:3300/0 conn(0x7fedd006b7f0 0x7fedd00aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedcf7fe700 1 --2- 172.21.15.112:0/2180657861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedd015aeb0 0x7fedd006b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedcffff700 1 -- 172.21.15.112:0/2180657861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedd015aeb0 msgr2=0x7fedd006b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:42.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedcffff700 1 --2- 172.21.15.112:0/2180657861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedd015aeb0 0x7fedd006b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedcffff700 1 -- 172.21.15.112:0/2180657861 >> v2:172.21.15.138:3300/0 conn(0x7fedd006b7f0 msgr2=0x7fedd00aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:42.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedcffff700 1 --2- 172.21.15.112:0/2180657861 >> v2:172.21.15.138:3300/0 conn(0x7fedd006b7f0 0x7fedd00aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedcffff700 1 -- 172.21.15.112:0/2180657861 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd0161450 con 0x7fedd00af340 2023-08-27T11:17:42.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.340+0000 7fedcffff700 1 --2- 172.21.15.112:0/2180657861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedd00af340 0x7fedd006f790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fedd00707f0 tx=0x7fedc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed843d13d37ea98d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:42.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.341+0000 7fedce7fc700 1 -- 172.21.15.112:0/2180657861 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc800e070 con 0x7fedd00af340 2023-08-27T11:17:42.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.341+0000 7fedce7fc700 1 -- 172.21.15.112:0/2180657861 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc800b910 con 0x7fedd00af340 2023-08-27T11:17:42.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.341+0000 7fedce7fc700 1 -- 172.21.15.112:0/2180657861 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc800bcc0 con 0x7fedd00af340 2023-08-27T11:17:42.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.341+0000 7fedd5a64700 1 -- 172.21.15.112:0/2180657861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedd00af340 msgr2=0x7fedd006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.341+0000 7fedd5a64700 1 --2- 172.21.15.112:0/2180657861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedd00af340 0x7fedd006f790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fedd00707f0 tx=0x7fedc800b330 comp rx=0 tx=0).stop 2023-08-27T11:17:42.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.341+0000 7fedd5a64700 1 -- 172.21.15.112:0/2180657861 shutdown_connections 2023-08-27T11:17:42.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.341+0000 7fedd5a64700 1 --2- 172.21.15.112:0/2180657861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedd00af340 0x7fedd006f790 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.341+0000 7fedd5a64700 1 --2- 172.21.15.112:0/2180657861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedd015aeb0 0x7fedd006b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.341+0000 7fedd5a64700 1 --2- 172.21.15.112:0/2180657861 >> v2:172.21.15.138:3300/0 conn(0x7fedd006b7f0 0x7fedd00aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.341+0000 7fedd5a64700 1 -- 172.21.15.112:0/2180657861 >> 172.21.15.112:0/2180657861 conn(0x7fedd00d03e0 msgr2=0x7fedd00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:42.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.341+0000 7fedd5a64700 1 -- 172.21.15.112:0/2180657861 shutdown_connections 2023-08-27T11:17:42.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.341+0000 7fedd5a64700 1 -- 172.21.15.112:0/2180657861 wait complete. 2023-08-27T11:17:42.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.343+0000 7fedd5a64700 1 Processor -- start 2023-08-27T11:17:42.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.343+0000 7fedd5a64700 1 -- start start 2023-08-27T11:17:42.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedd5a64700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedd006b7f0 0x7fedd015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedd5a64700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedd00af340 0x7fedd0152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedd5a64700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedd015aeb0 0x7fedd014a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedd5a64700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd015c600 con 0x7fedd00af340 2023-08-27T11:17:42.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedd5a64700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd015c8e0 con 0x7fedd006b7f0 2023-08-27T11:17:42.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedd5a64700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd015cbc0 con 0x7fedd015aeb0 2023-08-27T11:17:42.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedceffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedd00af340 0x7fedd0152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedceffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedd00af340 0x7fedd0152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56052/0 (socket says 172.21.15.112:56052) 2023-08-27T11:17:42.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedceffd700 1 -- 172.21.15.112:0/1324416415 learned_addr learned my addr 172.21.15.112:0/1324416415 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:42.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedcf7fe700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedd006b7f0 0x7fedd015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedcffff700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedd015aeb0 0x7fedd014a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedceffd700 1 -- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedd015aeb0 msgr2=0x7fedd014a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedceffd700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedd015aeb0 0x7fedd014a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedceffd700 1 -- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedd006b7f0 msgr2=0x7fedd015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedceffd700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedd006b7f0 0x7fedd015bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedceffd700 1 -- 172.21.15.112:0/1324416415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc8009b30 con 0x7fedd00af340 2023-08-27T11:17:42.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.344+0000 7fedceffd700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedd00af340 0x7fedd0152240 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fedc000e930 tx=0x7fedc000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:42.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.345+0000 7fedb7fff700 1 -- 172.21.15.112:0/1324416415 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0013680 con 0x7fedd00af340 2023-08-27T11:17:42.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.345+0000 7fedb7fff700 1 -- 172.21.15.112:0/1324416415 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc00137e0 con 0x7fedd00af340 2023-08-27T11:17:42.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.345+0000 7fedd5a64700 1 -- 172.21.15.112:0/1324416415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd015cea0 con 0x7fedd00af340 2023-08-27T11:17:42.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.345+0000 7fedb7fff700 1 -- 172.21.15.112:0/1324416415 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0013b70 con 0x7fedd00af340 2023-08-27T11:17:42.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.345+0000 7fedd5a64700 1 -- 172.21.15.112:0/1324416415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd015c270 con 0x7fedd00af340 2023-08-27T11:17:42.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.346+0000 7fedd5a64700 1 -- 172.21.15.112:0/1324416415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd0067e90 con 0x7fedd00af340 2023-08-27T11:17:42.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.346+0000 7fedb7fff700 1 -- 172.21.15.112:0/1324416415 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fedc0013cd0 con 0x7fedd00af340 2023-08-27T11:17:42.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.346+0000 7fedb7fff700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedbc06a0e0 0x7fedbc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.346+0000 7fedb7fff700 1 -- 172.21.15.112:0/1324416415 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fedc00884c0 con 0x7fedd00af340 2023-08-27T11:17:42.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.347+0000 7fedcf7fe700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedbc06a0e0 0x7fedbc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.347+0000 7fedcf7fe700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedbc06a0e0 0x7fedbc06c5a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009b40 tx=0x7fedc4009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:42.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.352+0000 7fedb7fff700 1 -- 172.21.15.112:0/1324416415 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fedc0053bb0 con 0x7fedd00af340 2023-08-27T11:17:42.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.543+0000 7fedd5a64700 1 -- 172.21.15.112:0/1324416415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fedd00aff90 con 0x7fedd00af340 2023-08-27T11:17:42.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.544+0000 7fedb7fff700 1 -- 172.21.15.112:0/1324416415 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fedc00571d0 con 0x7fedd00af340 2023-08-27T11:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:42.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.546+0000 7fedd5a64700 1 -- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedbc06a0e0 msgr2=0x7fedbc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.546+0000 7fedd5a64700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedbc06a0e0 0x7fedbc06c5a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009b40 tx=0x7fedc4009210 comp rx=0 tx=0).stop 2023-08-27T11:17:42.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.546+0000 7fedd5a64700 1 -- 172.21.15.112:0/1324416415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedd00af340 msgr2=0x7fedd0152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.546+0000 7fedd5a64700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedd00af340 0x7fedd0152240 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fedc000e930 tx=0x7fedc000ec40 comp rx=0 tx=0).stop 2023-08-27T11:17:42.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.546+0000 7fedd5a64700 1 -- 172.21.15.112:0/1324416415 shutdown_connections 2023-08-27T11:17:42.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.546+0000 7fedd5a64700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedbc06a0e0 0x7fedbc06c5a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.546+0000 7fedd5a64700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedd015aeb0 0x7fedd014a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.546+0000 7fedd5a64700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedd006b7f0 0x7fedd015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.546+0000 7fedd5a64700 1 --2- 172.21.15.112:0/1324416415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedd00af340 0x7fedd0152240 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.546+0000 7fedd5a64700 1 -- 172.21.15.112:0/1324416415 >> 172.21.15.112:0/1324416415 conn(0x7fedd00d03e0 msgr2=0x7fedd00b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:42.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.546+0000 7fedd5a64700 1 -- 172.21.15.112:0/1324416415 shutdown_connections 2023-08-27T11:17:42.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.546+0000 7fedd5a64700 1 -- 172.21.15.112:0/1324416415 wait complete. 2023-08-27T11:17:42.559 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.680+0000 7f071ed74700 1 Processor -- start 2023-08-27T11:17:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.680+0000 7f071ed74700 1 -- start start 2023-08-27T11:17:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f071ed74700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0718143390 0x7f07180723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f071ed74700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f071806f820 0x7f071806ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f071ed74700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07180b16e0 con 0x7f071806e2f0 2023-08-27T11:17:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f071ed74700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0718070000 con 0x7f071806f820 2023-08-27T11:17:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f071ed74700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071806e780 con 0x7f0718143390 2023-08-27T11:17:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f0717fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f071806f820 0x7f071806ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f071d311700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f071806e2f0 0x7f071815d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50442/0 (socket says 172.21.15.112:50442) 2023-08-27T11:17:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f0717fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f071806f820 0x7f071806ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56062/0 (socket says 172.21.15.112:56062) 2023-08-27T11:17:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f071d311700 1 -- 172.21.15.112:0/3703620731 learned_addr learned my addr 172.21.15.112:0/3703620731 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f071cb10700 1 --2- 172.21.15.112:0/3703620731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0718143390 0x7f07180723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f0717fff700 1 -- 172.21.15.112:0/3703620731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0718143390 msgr2=0x7f07180723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f0717fff700 1 --2- 172.21.15.112:0/3703620731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0718143390 0x7f07180723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f0717fff700 1 -- 172.21.15.112:0/3703620731 >> v1:172.21.15.112:6789/0 conn(0x7f071806e2f0 legacy=0x7f071815d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f0717fff700 1 -- 172.21.15.112:0/3703620731 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07180b0010 con 0x7f071806f820 2023-08-27T11:17:42.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.681+0000 7f07177fe700 1 -- 172.21.15.112:0/3703620731 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683638698 0 0) 0x7f07180b16e0 con 0x7f071806e2f0 2023-08-27T11:17:42.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.682+0000 7f0717fff700 1 --2- 172.21.15.112:0/3703620731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071806f820 0x7f071806ddb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f070400a9e0 tx=0x7f070400acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3fe979f2f4528a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:42.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.682+0000 7f07177fe700 1 -- 172.21.15.112:0/3703620731 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070400cb70 con 0x7f071806f820 2023-08-27T11:17:42.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.682+0000 7f07177fe700 1 -- 172.21.15.112:0/3703620731 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070400ccd0 con 0x7f071806f820 2023-08-27T11:17:42.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.682+0000 7f07177fe700 1 -- 172.21.15.112:0/3703620731 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704018620 con 0x7f071806f820 2023-08-27T11:17:42.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.682+0000 7f071ed74700 1 -- 172.21.15.112:0/3703620731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071806f820 msgr2=0x7f071806ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.682+0000 7f071ed74700 1 --2- 172.21.15.112:0/3703620731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071806f820 0x7f071806ddb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f070400a9e0 tx=0x7f070400acf0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.682+0000 7f071ed74700 1 -- 172.21.15.112:0/3703620731 shutdown_connections 2023-08-27T11:17:42.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.682+0000 7f071ed74700 1 --2- 172.21.15.112:0/3703620731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0718143390 0x7f07180723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.682+0000 7f071ed74700 1 --2- 172.21.15.112:0/3703620731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071806f820 0x7f071806ddb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.682+0000 7f071ed74700 1 -- 172.21.15.112:0/3703620731 >> 172.21.15.112:0/3703620731 conn(0x7f07180cc240 msgr2=0x7f07180cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:42.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.682+0000 7f071ed74700 1 -- 172.21.15.112:0/3703620731 shutdown_connections 2023-08-27T11:17:42.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.682+0000 7f071ed74700 1 -- 172.21.15.112:0/3703620731 wait complete. 2023-08-27T11:17:42.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.683+0000 7f071ed74700 1 Processor -- start 2023-08-27T11:17:42.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.684+0000 7f071ed74700 1 -- start start 2023-08-27T11:17:42.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.684+0000 7f071ed74700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f071806e2f0 0x7f07180a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.684+0000 7f071ed74700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f071806f820 0x7f07180ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.684+0000 7f071ed74700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0718143390 0x7f07181412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.684+0000 7f071ed74700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0718142c30 con 0x7f0718143390 2023-08-27T11:17:42.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.684+0000 7f071ed74700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07181427a0 con 0x7f071806e2f0 2023-08-27T11:17:42.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.684+0000 7f071ed74700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07180ae660 con 0x7f071806f820 2023-08-27T11:17:42.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f071d311700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0718143390 0x7f07181412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f071d311700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0718143390 0x7f07181412d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56066/0 (socket says 172.21.15.112:56066) 2023-08-27T11:17:42.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f071d311700 1 -- 172.21.15.112:0/1176688026 learned_addr learned my addr 172.21.15.112:0/1176688026 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:42.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f071d311700 1 -- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f071806f820 msgr2=0x7f07180ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f0717fff700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f071806f820 0x7f07180ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f071cb10700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f071806e2f0 0x7f07180a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f071d311700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f071806f820 0x7f07180ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f071d311700 1 -- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f071806e2f0 msgr2=0x7f07180a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f071d311700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f071806e2f0 0x7f07180a8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f071d311700 1 -- 172.21.15.112:0/1176688026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070400a640 con 0x7f0718143390 2023-08-27T11:17:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f071cb10700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f071806e2f0 0x7f07180a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f071d311700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0718143390 0x7f07181412d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f071000cc70 tx=0x7f071000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f07157fa700 1 -- 172.21.15.112:0/1176688026 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071000ada0 con 0x7f0718143390 2023-08-27T11:17:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f07157fa700 1 -- 172.21.15.112:0/1176688026 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071000ce70 con 0x7f0718143390 2023-08-27T11:17:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f071ed74700 1 -- 172.21.15.112:0/1176688026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0718141b90 con 0x7f0718143390 2023-08-27T11:17:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f07157fa700 1 -- 172.21.15.112:0/1176688026 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07100072f0 con 0x7f0718143390 2023-08-27T11:17:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.685+0000 7f071ed74700 1 -- 172.21.15.112:0/1176688026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0718142150 con 0x7f0718143390 2023-08-27T11:17:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.686+0000 7f071ed74700 1 -- 172.21.15.112:0/1176688026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0718142950 con 0x7f0718143390 2023-08-27T11:17:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.687+0000 7f07157fa700 1 -- 172.21.15.112:0/1176688026 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0710007450 con 0x7f0718143390 2023-08-27T11:17:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.687+0000 7f07157fa700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f06fc06a0e0 0x7f06fc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.687+0000 7f07157fa700 1 -- 172.21.15.112:0/1176688026 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0710088440 con 0x7f0718143390 2023-08-27T11:17:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.687+0000 7f071cb10700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f06fc06a0e0 0x7f06fc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.688+0000 7f071cb10700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f06fc06a0e0 0x7f06fc06c5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f070c006fd0 tx=0x7f070c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.691+0000 7f07157fa700 1 -- 172.21.15.112:0/1176688026 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0710053b30 con 0x7f0718143390 2023-08-27T11:17:42.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.877+0000 7f071ed74700 1 -- 172.21.15.112:0/1176688026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0718067e90 con 0x7f0718143390 2023-08-27T11:17:42.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.877+0000 7f07157fa700 1 -- 172.21.15.112:0/1176688026 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0710057150 con 0x7f0718143390 2023-08-27T11:17:42.879 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:42.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.879+0000 7f071ed74700 1 -- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f06fc06a0e0 msgr2=0x7f06fc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.879+0000 7f071ed74700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f06fc06a0e0 0x7f06fc06c5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f070c006fd0 tx=0x7f070c008040 comp rx=0 tx=0).stop 2023-08-27T11:17:42.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.879+0000 7f071ed74700 1 -- 172.21.15.112:0/1176688026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0718143390 msgr2=0x7f07181412d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:42.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.879+0000 7f071ed74700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0718143390 0x7f07181412d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f071000cc70 tx=0x7f071000a340 comp rx=0 tx=0).stop 2023-08-27T11:17:42.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.880+0000 7f071ed74700 1 -- 172.21.15.112:0/1176688026 shutdown_connections 2023-08-27T11:17:42.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.880+0000 7f071ed74700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f06fc06a0e0 0x7f06fc06c5a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.880+0000 7f071ed74700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0718143390 0x7f07181412d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.880+0000 7f071ed74700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f071806e2f0 0x7f07180a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.880+0000 7f071ed74700 1 --2- 172.21.15.112:0/1176688026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f071806f820 0x7f07180ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:42.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.880+0000 7f071ed74700 1 -- 172.21.15.112:0/1176688026 >> 172.21.15.112:0/1176688026 conn(0x7f07180cc240 msgr2=0x7f07180cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:42.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.880+0000 7f071ed74700 1 -- 172.21.15.112:0/1176688026 shutdown_connections 2023-08-27T11:17:42.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:42.880+0000 7f071ed74700 1 -- 172.21.15.112:0/1176688026 wait complete. 2023-08-27T11:17:42.893 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:17:43.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.015+0000 7fa83375c700 1 Processor -- start 2023-08-27T11:17:43.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.015+0000 7fa83375c700 1 -- start start 2023-08-27T11:17:43.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa83375c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa82c144b80 0x7fa82c0b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa83375c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa82c06c6e0 0x7fa82c0cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa83375c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c0cef20 con 0x7fa82c0b3de0 2023-08-27T11:17:43.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa83375c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c0b4b40 con 0x7fa82c144b80 2023-08-27T11:17:43.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa83375c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c0b42a0 con 0x7fa82c06c6e0 2023-08-27T11:17:43.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa8314f8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa82c144b80 0x7fa82c0b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa830cf7700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa82c0b3de0 0x7fa82c06bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50454/0 (socket says 172.21.15.112:50454) 2023-08-27T11:17:43.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa830cf7700 1 -- 172.21.15.112:0/1567229454 learned_addr learned my addr 172.21.15.112:0/1567229454 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:43.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa831cf9700 1 --2- 172.21.15.112:0/1567229454 >> v2:172.21.15.138:3300/0 conn(0x7fa82c06c6e0 0x7fa82c0cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa8314f8700 1 -- 172.21.15.112:0/1567229454 >> v2:172.21.15.138:3300/0 conn(0x7fa82c06c6e0 msgr2=0x7fa82c0cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:43.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa8314f8700 1 --2- 172.21.15.112:0/1567229454 >> v2:172.21.15.138:3300/0 conn(0x7fa82c06c6e0 0x7fa82c0cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa8314f8700 1 -- 172.21.15.112:0/1567229454 >> v1:172.21.15.112:6789/0 conn(0x7fa82c0b3de0 legacy=0x7fa82c06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa8314f8700 1 -- 172.21.15.112:0/1567229454 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c0b4580 con 0x7fa82c144b80 2023-08-27T11:17:43.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa823fff700 1 -- 172.21.15.112:0/1567229454 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715011754 0 0) 0x7fa82c0cef20 con 0x7fa82c0b3de0 2023-08-27T11:17:43.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa8314f8700 1 --2- 172.21.15.112:0/1567229454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa82c144b80 0x7fa82c0b6c00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa82c074170 tx=0x7fa828009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d6ab3e89b1d6fd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:43.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.016+0000 7fa823fff700 1 -- 172.21.15.112:0/1567229454 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828011830 con 0x7fa82c144b80 2023-08-27T11:17:43.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.017+0000 7fa823fff700 1 -- 172.21.15.112:0/1567229454 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa828011990 con 0x7fa82c144b80 2023-08-27T11:17:43.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.017+0000 7fa823fff700 1 -- 172.21.15.112:0/1567229454 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828011d20 con 0x7fa82c144b80 2023-08-27T11:17:43.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.017+0000 7fa83375c700 1 -- 172.21.15.112:0/1567229454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa82c144b80 msgr2=0x7fa82c0b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.017+0000 7fa83375c700 1 --2- 172.21.15.112:0/1567229454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa82c144b80 0x7fa82c0b6c00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa82c074170 tx=0x7fa828009b20 comp rx=0 tx=0).stop 2023-08-27T11:17:43.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.017+0000 7fa83375c700 1 -- 172.21.15.112:0/1567229454 shutdown_connections 2023-08-27T11:17:43.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.017+0000 7fa83375c700 1 --2- 172.21.15.112:0/1567229454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa82c144b80 0x7fa82c0b6c00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.017+0000 7fa83375c700 1 --2- 172.21.15.112:0/1567229454 >> v2:172.21.15.138:3300/0 conn(0x7fa82c06c6e0 0x7fa82c0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.017+0000 7fa83375c700 1 -- 172.21.15.112:0/1567229454 >> 172.21.15.112:0/1567229454 conn(0x7fa82c0cdc50 msgr2=0x7fa82c0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:43.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.017+0000 7fa83375c700 1 -- 172.21.15.112:0/1567229454 shutdown_connections 2023-08-27T11:17:43.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.017+0000 7fa83375c700 1 -- 172.21.15.112:0/1567229454 wait complete. 2023-08-27T11:17:43.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.018+0000 7fa83375c700 1 Processor -- start 2023-08-27T11:17:43.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.018+0000 7fa83375c700 1 -- start start 2023-08-27T11:17:43.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa83375c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c06c6e0 0x7fa82c143d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa83375c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa82c0b3de0 0x7fa82c0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa83375c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa82c144b80 0x7fa82c0a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa83375c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0c9980 con 0x7fa82c0b3de0 2023-08-27T11:17:43.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa83375c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0cd680 con 0x7fa82c06c6e0 2023-08-27T11:17:43.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa83375c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0c8d70 con 0x7fa82c144b80 2023-08-27T11:17:43.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa830cf7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa82c0b3de0 0x7fa82c0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa830cf7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa82c0b3de0 0x7fa82c0c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56080/0 (socket says 172.21.15.112:56080) 2023-08-27T11:17:43.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa830cf7700 1 -- 172.21.15.112:0/1144668626 learned_addr learned my addr 172.21.15.112:0/1144668626 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:43.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa8314f8700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c06c6e0 0x7fa82c143d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa830cf7700 1 -- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa82c144b80 msgr2=0x7fa82c0a8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa830cf7700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa82c144b80 0x7fa82c0a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa830cf7700 1 -- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c06c6e0 msgr2=0x7fa82c143d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa830cf7700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c06c6e0 0x7fa82c143d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa830cf7700 1 -- 172.21.15.112:0/1144668626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8280094a0 con 0x7fa82c0b3de0 2023-08-27T11:17:43.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.019+0000 7fa8314f8700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c06c6e0 0x7fa82c143d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:43.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.020+0000 7fa830cf7700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa82c0b3de0 0x7fa82c0c9e10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0011e0 tx=0x7fa81c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:43.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.020+0000 7fa821ffb700 1 -- 172.21.15.112:0/1144668626 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c004c80 con 0x7fa82c0b3de0 2023-08-27T11:17:43.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.020+0000 7fa821ffb700 1 -- 172.21.15.112:0/1144668626 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81c004e00 con 0x7fa82c0b3de0 2023-08-27T11:17:43.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.020+0000 7fa821ffb700 1 -- 172.21.15.112:0/1144668626 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c007310 con 0x7fa82c0b3de0 2023-08-27T11:17:43.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.020+0000 7fa83375c700 1 -- 172.21.15.112:0/1144668626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c0c9050 con 0x7fa82c0b3de0 2023-08-27T11:17:43.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.020+0000 7fa83375c700 1 -- 172.21.15.112:0/1144668626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c0c9610 con 0x7fa82c0b3de0 2023-08-27T11:17:43.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.021+0000 7fa83375c700 1 -- 172.21.15.112:0/1144668626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c0c9b30 con 0x7fa82c0b3de0 2023-08-27T11:17:43.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.021+0000 7fa821ffb700 1 -- 172.21.15.112:0/1144668626 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa81c007470 con 0x7fa82c0b3de0 2023-08-27T11:17:43.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.021+0000 7fa821ffb700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa81006a0e0 0x7fa81006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.022+0000 7fa821ffb700 1 -- 172.21.15.112:0/1144668626 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa81c088330 con 0x7fa82c0b3de0 2023-08-27T11:17:43.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.022+0000 7fa8314f8700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa81006a0e0 0x7fa81006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.022+0000 7fa8314f8700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa81006a0e0 0x7fa81006c5a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa8280090b0 tx=0x7fa828008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:43.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.026+0000 7fa821ffb700 1 -- 172.21.15.112:0/1144668626 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa82c0c9b30 con 0x7fa82c0b3de0 2023-08-27T11:17:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.214+0000 7fa83375c700 1 -- 172.21.15.112:0/1144668626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa82c067a20 con 0x7fa82c0b3de0 2023-08-27T11:17:43.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.214+0000 7fa821ffb700 1 -- 172.21.15.112:0/1144668626 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa81c053990 con 0x7fa82c0b3de0 2023-08-27T11:17:43.217 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:17:43.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.216+0000 7fa83375c700 1 -- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa81006a0e0 msgr2=0x7fa81006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.216+0000 7fa83375c700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa81006a0e0 0x7fa81006c5a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa8280090b0 tx=0x7fa828008030 comp rx=0 tx=0).stop 2023-08-27T11:17:43.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.216+0000 7fa83375c700 1 -- 172.21.15.112:0/1144668626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa82c0b3de0 msgr2=0x7fa82c0c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.216+0000 7fa83375c700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa82c0b3de0 0x7fa82c0c9e10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0011e0 tx=0x7fa81c00cae0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.216+0000 7fa83375c700 1 -- 172.21.15.112:0/1144668626 shutdown_connections 2023-08-27T11:17:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.216+0000 7fa83375c700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa81006a0e0 0x7fa81006c5a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.216+0000 7fa83375c700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa82c144b80 0x7fa82c0a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.216+0000 7fa83375c700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c06c6e0 0x7fa82c143d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.216+0000 7fa83375c700 1 --2- 172.21.15.112:0/1144668626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa82c0b3de0 0x7fa82c0c9e10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.216+0000 7fa83375c700 1 -- 172.21.15.112:0/1144668626 >> 172.21.15.112:0/1144668626 conn(0x7fa82c0cdc50 msgr2=0x7fa82c06bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:43.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.216+0000 7fa83375c700 1 -- 172.21.15.112:0/1144668626 shutdown_connections 2023-08-27T11:17:43.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.216+0000 7fa83375c700 1 -- 172.21.15.112:0/1144668626 wait complete. 2023-08-27T11:17:43.229 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:17:43.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.351+0000 7f2548027700 1 Processor -- start 2023-08-27T11:17:43.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.351+0000 7f2548027700 1 -- start start 2023-08-27T11:17:43.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f2548027700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f25401437c0 0x7f25400b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f2548027700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2540143ba0 0x7f25400ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f2548027700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25400cfd50 con 0x7f254006d5b0 2023-08-27T11:17:43.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f2548027700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25400cf140 con 0x7f2540143ba0 2023-08-27T11:17:43.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f2548027700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25400cf420 con 0x7f25401437c0 2023-08-27T11:17:43.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f25455c2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2540143ba0 0x7f25400ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f25465c4700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f254006d5b0 0x7f254015dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50472/0 (socket says 172.21.15.112:50472) 2023-08-27T11:17:43.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f25465c4700 1 -- 172.21.15.112:0/3765312127 learned_addr learned my addr 172.21.15.112:0/3765312127 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:43.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f2545dc3700 1 --2- 172.21.15.112:0/3765312127 >> v2:172.21.15.138:3300/0 conn(0x7f25401437c0 0x7f25400b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f25455c2700 1 -- 172.21.15.112:0/3765312127 >> v2:172.21.15.138:3300/0 conn(0x7f25401437c0 msgr2=0x7f25400b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:43.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f25455c2700 1 --2- 172.21.15.112:0/3765312127 >> v2:172.21.15.138:3300/0 conn(0x7f25401437c0 0x7f25400b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f25455c2700 1 -- 172.21.15.112:0/3765312127 >> v1:172.21.15.112:6789/0 conn(0x7f254006d5b0 legacy=0x7f254015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f25455c2700 1 -- 172.21.15.112:0/3765312127 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25400cf700 con 0x7f2540143ba0 2023-08-27T11:17:43.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f2544dc1700 1 -- 172.21.15.112:0/3765312127 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201853060 0 0) 0x7f25400cfd50 con 0x7f254006d5b0 2023-08-27T11:17:43.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f25455c2700 1 --2- 172.21.15.112:0/3765312127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540143ba0 0x7f25400ce860 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f254006e3d0 tx=0x7f253c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cd0f159c01b819 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:43.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f2544dc1700 1 -- 172.21.15.112:0/3765312127 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c011830 con 0x7f2540143ba0 2023-08-27T11:17:43.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f2544dc1700 1 -- 172.21.15.112:0/3765312127 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253c011990 con 0x7f2540143ba0 2023-08-27T11:17:43.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.352+0000 7f2544dc1700 1 -- 172.21.15.112:0/3765312127 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c011d20 con 0x7f2540143ba0 2023-08-27T11:17:43.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.353+0000 7f2548027700 1 -- 172.21.15.112:0/3765312127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540143ba0 msgr2=0x7f25400ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.353+0000 7f2548027700 1 --2- 172.21.15.112:0/3765312127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540143ba0 0x7f25400ce860 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f254006e3d0 tx=0x7f253c009b20 comp rx=0 tx=0).stop 2023-08-27T11:17:43.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.353+0000 7f2548027700 1 -- 172.21.15.112:0/3765312127 shutdown_connections 2023-08-27T11:17:43.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.353+0000 7f2548027700 1 --2- 172.21.15.112:0/3765312127 >> v2:172.21.15.138:3300/0 conn(0x7f25401437c0 0x7f25400b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.353+0000 7f2548027700 1 --2- 172.21.15.112:0/3765312127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540143ba0 0x7f25400ce860 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.353+0000 7f2548027700 1 -- 172.21.15.112:0/3765312127 >> 172.21.15.112:0/3765312127 conn(0x7f25400b1e90 msgr2=0x7f25400b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:43.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.353+0000 7f2548027700 1 -- 172.21.15.112:0/3765312127 shutdown_connections 2023-08-27T11:17:43.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.353+0000 7f2548027700 1 -- 172.21.15.112:0/3765312127 wait complete. 2023-08-27T11:17:43.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.354+0000 7f2548027700 1 Processor -- start 2023-08-27T11:17:43.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.354+0000 7f2548027700 1 -- start start 2023-08-27T11:17:43.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.355+0000 7f2548027700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f254006d5b0 0x7f25400cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.355+0000 7f2548027700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25401437c0 0x7f25401517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.355+0000 7f2548027700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540143ba0 0x7f254013e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.355+0000 7f2548027700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254013e8c0 con 0x7f2540143ba0 2023-08-27T11:17:43.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.355+0000 7f2548027700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254013b920 con 0x7f25401437c0 2023-08-27T11:17:43.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.355+0000 7f2548027700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25400cbf50 con 0x7f254006d5b0 2023-08-27T11:17:43.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.355+0000 7f25465c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540143ba0 0x7f254013e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.355+0000 7f25465c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540143ba0 0x7f254013e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56096/0 (socket says 172.21.15.112:56096) 2023-08-27T11:17:43.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.355+0000 7f25465c4700 1 -- 172.21.15.112:0/3382377653 learned_addr learned my addr 172.21.15.112:0/3382377653 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:43.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.355+0000 7f25455c2700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25401437c0 0x7f25401517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.355+0000 7f2545dc3700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f254006d5b0 0x7f25400cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.356+0000 7f25465c4700 1 -- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f254006d5b0 msgr2=0x7f25400cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.356+0000 7f25465c4700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f254006d5b0 0x7f25400cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.356+0000 7f25465c4700 1 -- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25401437c0 msgr2=0x7f25401517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.356+0000 7f25465c4700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25401437c0 0x7f25401517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.356+0000 7f25465c4700 1 -- 172.21.15.112:0/3382377653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c0094a0 con 0x7f2540143ba0 2023-08-27T11:17:43.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.356+0000 7f2545dc3700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f254006d5b0 0x7f25400cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:43.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.356+0000 7f25465c4700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540143ba0 0x7f254013e380 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2538002230 tx=0x7f253800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:43.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.356+0000 7f25367fc700 1 -- 172.21.15.112:0/3382377653 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253800ab70 con 0x7f2540143ba0 2023-08-27T11:17:43.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.356+0000 7f25367fc700 1 -- 172.21.15.112:0/3382377653 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253800acf0 con 0x7f2540143ba0 2023-08-27T11:17:43.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.356+0000 7f25367fc700 1 -- 172.21.15.112:0/3382377653 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538007310 con 0x7f2540143ba0 2023-08-27T11:17:43.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.356+0000 7f2548027700 1 -- 172.21.15.112:0/3382377653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25400cc230 con 0x7f2540143ba0 2023-08-27T11:17:43.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.356+0000 7f2548027700 1 -- 172.21.15.112:0/3382377653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25400cc510 con 0x7f2540143ba0 2023-08-27T11:17:43.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.357+0000 7f2548027700 1 -- 172.21.15.112:0/3382377653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254013bdd0 con 0x7f2540143ba0 2023-08-27T11:17:43.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.357+0000 7f25367fc700 1 -- 172.21.15.112:0/3382377653 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2538007470 con 0x7f2540143ba0 2023-08-27T11:17:43.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.358+0000 7f25367fc700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f252406a0e0 0x7f252406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.358+0000 7f25367fc700 1 -- 172.21.15.112:0/3382377653 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2538088000 con 0x7f2540143ba0 2023-08-27T11:17:43.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.358+0000 7f2545dc3700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f252406a0e0 0x7f252406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.358+0000 7f2545dc3700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f252406a0e0 0x7f252406c5a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f25300099f0 tx=0x7f2530008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:43.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.362+0000 7f25367fc700 1 -- 172.21.15.112:0/3382377653 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f254013bdd0 con 0x7f2540143ba0 2023-08-27T11:17:43.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.549+0000 7f2548027700 1 -- 172.21.15.112:0/3382377653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f254013bdd0 con 0x7f2540143ba0 2023-08-27T11:17:43.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.549+0000 7f25367fc700 1 -- 172.21.15.112:0/3382377653 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f254013bdd0 con 0x7f2540143ba0 2023-08-27T11:17:43.551 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:17:43.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.551+0000 7f2548027700 1 -- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f252406a0e0 msgr2=0x7f252406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.551+0000 7f2548027700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f252406a0e0 0x7f252406c5a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f25300099f0 tx=0x7f2530008040 comp rx=0 tx=0).stop 2023-08-27T11:17:43.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.551+0000 7f2548027700 1 -- 172.21.15.112:0/3382377653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540143ba0 msgr2=0x7f254013e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.551+0000 7f2548027700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540143ba0 0x7f254013e380 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2538002230 tx=0x7f253800cf50 comp rx=0 tx=0).stop 2023-08-27T11:17:43.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.551+0000 7f2548027700 1 -- 172.21.15.112:0/3382377653 shutdown_connections 2023-08-27T11:17:43.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.551+0000 7f2548027700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f252406a0e0 0x7f252406c5a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.551+0000 7f2548027700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540143ba0 0x7f254013e380 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.551+0000 7f2548027700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f254006d5b0 0x7f25400cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.551+0000 7f2548027700 1 --2- 172.21.15.112:0/3382377653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25401437c0 0x7f25401517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.551+0000 7f2548027700 1 -- 172.21.15.112:0/3382377653 >> 172.21.15.112:0/3382377653 conn(0x7f25400b1e90 msgr2=0x7f254015db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:43.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.551+0000 7f2548027700 1 -- 172.21.15.112:0/3382377653 shutdown_connections 2023-08-27T11:17:43.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.551+0000 7f2548027700 1 -- 172.21.15.112:0/3382377653 wait complete. 2023-08-27T11:17:43.566 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:17:43.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.688+0000 7f5125388700 1 Processor -- start 2023-08-27T11:17:43.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.688+0000 7f5125388700 1 -- start start 2023-08-27T11:17:43.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.689+0000 7f5125388700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5120163f90 0x7f5120166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.689+0000 7f5125388700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51200b5ae0 con 0x7f51200b3e50 2023-08-27T11:17:43.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.689+0000 7f5125388700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51200b45f0 con 0x7f5120144570 2023-08-27T11:17:43.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.689+0000 7f5125388700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51200cf7a0 con 0x7f5120163f90 2023-08-27T11:17:43.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.689+0000 7f511f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5120163f90 0x7f5120166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.689+0000 7f511e7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f51200b3e50 0x7f512015f530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50484/0 (socket says 172.21.15.112:50484) 2023-08-27T11:17:43.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.689+0000 7f511e7fc700 1 -- 172.21.15.112:0/537328358 learned_addr learned my addr 172.21.15.112:0/537328358 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:43.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.690+0000 7f511f7fe700 1 -- 172.21.15.112:0/537328358 >> v1:172.21.15.138:6789/0 conn(0x7f5120144570 legacy=0x7f512015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.690+0000 7f511f7fe700 1 -- 172.21.15.112:0/537328358 >> v1:172.21.15.112:6789/0 conn(0x7f51200b3e50 legacy=0x7f512015f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.690+0000 7f511dffb700 1 -- 172.21.15.112:0/537328358 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3754545402 0 0) 0x7f5110001110 con 0x7f51200b3e50 2023-08-27T11:17:43.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.690+0000 7f511f7fe700 1 -- 172.21.15.112:0/537328358 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512015fc40 con 0x7f5120163f90 2023-08-27T11:17:43.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.690+0000 7f511f7fe700 1 --2- 172.21.15.112:0/537328358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5120163f90 0x7f5120166380 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5118009d90 tx=0x7f511800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c67cbcd734cc958 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:43.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.690+0000 7f511dffb700 1 -- 172.21.15.112:0/537328358 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511800e070 con 0x7f5120163f90 2023-08-27T11:17:43.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.690+0000 7f511dffb700 1 -- 172.21.15.112:0/537328358 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511800bbd0 con 0x7f5120163f90 2023-08-27T11:17:43.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.690+0000 7f511dffb700 1 -- 172.21.15.112:0/537328358 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118012430 con 0x7f5120163f90 2023-08-27T11:17:43.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.690+0000 7f5125388700 1 -- 172.21.15.112:0/537328358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5120163f90 msgr2=0x7f5120166380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.690+0000 7f5125388700 1 --2- 172.21.15.112:0/537328358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5120163f90 0x7f5120166380 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5118009d90 tx=0x7f511800b330 comp rx=0 tx=0).stop 2023-08-27T11:17:43.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.690+0000 7f5125388700 1 -- 172.21.15.112:0/537328358 shutdown_connections 2023-08-27T11:17:43.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.690+0000 7f5125388700 1 --2- 172.21.15.112:0/537328358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5120163f90 0x7f5120166380 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.690+0000 7f5125388700 1 -- 172.21.15.112:0/537328358 >> 172.21.15.112:0/537328358 conn(0x7f512006f970 msgr2=0x7f512006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:43.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.691+0000 7f5125388700 1 -- 172.21.15.112:0/537328358 shutdown_connections 2023-08-27T11:17:43.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.691+0000 7f5125388700 1 -- 172.21.15.112:0/537328358 wait complete. 2023-08-27T11:17:43.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.692+0000 7f5125388700 1 Processor -- start 2023-08-27T11:17:43.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.692+0000 7f5125388700 1 -- start start 2023-08-27T11:17:43.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f5125388700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51200b3e50 0x7f5120143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f5125388700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5120144570 0x7f51201435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f5125388700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5120163f90 0x7f51200708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f5125388700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51200721e0 con 0x7f5120144570 2023-08-27T11:17:43.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f5125388700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51200715d0 con 0x7f5120163f90 2023-08-27T11:17:43.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f5125388700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51200718b0 con 0x7f51200b3e50 2023-08-27T11:17:43.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f511e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5120144570 0x7f51201435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f511e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5120144570 0x7f51201435a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56108/0 (socket says 172.21.15.112:56108) 2023-08-27T11:17:43.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f511e7fc700 1 -- 172.21.15.112:0/3937065006 learned_addr learned my addr 172.21.15.112:0/3937065006 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:43.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f511f7fe700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5120163f90 0x7f51200708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f511effd700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51200b3e50 0x7f5120143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f511e7fc700 1 -- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51200b3e50 msgr2=0x7f5120143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f511e7fc700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51200b3e50 0x7f5120143060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f511e7fc700 1 -- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5120163f90 msgr2=0x7f51200708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f511e7fc700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5120163f90 0x7f51200708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f511e7fc700 1 -- 172.21.15.112:0/3937065006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118009a70 con 0x7f5120144570 2023-08-27T11:17:43.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f511e7fc700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5120144570 0x7f51201435a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f511000c520 tx=0x7f511000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:43.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f51077fe700 1 -- 172.21.15.112:0/3937065006 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110004c80 con 0x7f5120144570 2023-08-27T11:17:43.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f51077fe700 1 -- 172.21.15.112:0/3937065006 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5110004e20 con 0x7f5120144570 2023-08-27T11:17:43.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f51077fe700 1 -- 172.21.15.112:0/3937065006 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110007310 con 0x7f5120144570 2023-08-27T11:17:43.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f5125388700 1 -- 172.21.15.112:0/3937065006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5120071b90 con 0x7f5120144570 2023-08-27T11:17:43.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.693+0000 7f5125388700 1 -- 172.21.15.112:0/3937065006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512015a730 con 0x7f5120144570 2023-08-27T11:17:43.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.694+0000 7f5125388700 1 -- 172.21.15.112:0/3937065006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5120072350 con 0x7f5120144570 2023-08-27T11:17:43.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.695+0000 7f51077fe700 1 -- 172.21.15.112:0/3937065006 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5110007470 con 0x7f5120144570 2023-08-27T11:17:43.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.695+0000 7f51077fe700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f510c06a0e0 0x7f510c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:43.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.695+0000 7f51077fe700 1 -- 172.21.15.112:0/3937065006 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f51100881d0 con 0x7f5120144570 2023-08-27T11:17:43.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.695+0000 7f511effd700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f510c06a0e0 0x7f510c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:43.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.696+0000 7f511effd700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f510c06a0e0 0x7f510c06c5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f512013fde0 tx=0x7f5114009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:43.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.699+0000 7f51077fe700 1 -- 172.21.15.112:0/3937065006 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f51100572c0 con 0x7f5120144570 2023-08-27T11:17:43.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.902+0000 7f5125388700 1 -- 172.21.15.112:0/3937065006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f51200b5ae0 con 0x7f5120144570 2023-08-27T11:17:43.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.903+0000 7f51077fe700 1 -- 172.21.15.112:0/3937065006 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f51200b5ae0 con 0x7f5120144570 2023-08-27T11:17:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.905+0000 7f5125388700 1 -- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f510c06a0e0 msgr2=0x7f510c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.905+0000 7f5125388700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f510c06a0e0 0x7f510c06c5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f512013fde0 tx=0x7f5114009040 comp rx=0 tx=0).stop 2023-08-27T11:17:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.905+0000 7f5125388700 1 -- 172.21.15.112:0/3937065006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5120144570 msgr2=0x7f51201435a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.905+0000 7f5125388700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5120144570 0x7f51201435a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f511000c520 tx=0x7f511000c8e0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.905+0000 7f5125388700 1 -- 172.21.15.112:0/3937065006 shutdown_connections 2023-08-27T11:17:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.905+0000 7f5125388700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f510c06a0e0 0x7f510c06c5a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.905+0000 7f5125388700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5120163f90 0x7f51200708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.905+0000 7f5125388700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51200b3e50 0x7f5120143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.905+0000 7f5125388700 1 --2- 172.21.15.112:0/3937065006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5120144570 0x7f51201435a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.905+0000 7f5125388700 1 -- 172.21.15.112:0/3937065006 >> 172.21.15.112:0/3937065006 conn(0x7f512006f970 msgr2=0x7f51201412b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.905+0000 7f5125388700 1 -- 172.21.15.112:0/3937065006 shutdown_connections 2023-08-27T11:17:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:43.905+0000 7f5125388700 1 -- 172.21.15.112:0/3937065006 wait complete. 2023-08-27T11:17:43.910 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:17:43.921 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-27T11:17:43.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:43.922 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-08-27T11:17:43.967 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:17:43.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:43.968 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-27T11:17:44.041 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:17:44.042 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:44.059 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:17:44.061 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:44.118 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:17:44.119 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:44.179 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:44.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.338+0000 7fba6fff6700 1 Processor -- start 2023-08-27T11:17:44.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.338+0000 7fba6fff6700 1 -- start start 2023-08-27T11:17:44.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.339+0000 7fba6fff6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba680cddb0 0x7fba6815f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.339+0000 7fba6fff6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fba680ce190 0x7fba6815b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.339+0000 7fba6fff6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba6815bdc0 0x7fba680cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.339+0000 7fba6fff6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6815f500 con 0x7fba6815bdc0 2023-08-27T11:17:44.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.339+0000 7fba6fff6700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6815eed0 con 0x7fba680ce190 2023-08-27T11:17:44.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.339+0000 7fba6fff6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6815c200 con 0x7fba680cddb0 2023-08-27T11:17:44.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.339+0000 7fba6e593700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba6815bdc0 0x7fba680cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:44.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.339+0000 7fba6e593700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba6815bdc0 0x7fba680cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56116/0 (socket says 172.21.15.112:56116) 2023-08-27T11:17:44.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.339+0000 7fba6e593700 1 -- 172.21.15.112:0/198708403 learned_addr learned my addr 172.21.15.112:0/198708403 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:44.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.340+0000 7fba6e593700 1 -- 172.21.15.112:0/198708403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba680cddb0 msgr2=0x7fba6815f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:44.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.340+0000 7fba6e593700 1 --2- 172.21.15.112:0/198708403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba680cddb0 0x7fba6815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.340+0000 7fba6e593700 1 -- 172.21.15.112:0/198708403 >> v2:172.21.15.138:3300/0 conn(0x7fba680ce190 msgr2=0x7fba6815b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:44.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.340+0000 7fba6e593700 1 --2- 172.21.15.112:0/198708403 >> v2:172.21.15.138:3300/0 conn(0x7fba680ce190 0x7fba6815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.340+0000 7fba6e593700 1 -- 172.21.15.112:0/198708403 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6815fe60 con 0x7fba6815bdc0 2023-08-27T11:17:44.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.340+0000 7fba6e593700 1 --2- 172.21.15.112:0/198708403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6815bdc0 0x7fba680cf790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fba680ce760 tx=0x7fba6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf29a2c03057410 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:44.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.340+0000 7fba6cd90700 1 -- 172.21.15.112:0/198708403 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6400e070 con 0x7fba6815bdc0 2023-08-27T11:17:44.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.340+0000 7fba6cd90700 1 -- 172.21.15.112:0/198708403 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6400b9c0 con 0x7fba6815bdc0 2023-08-27T11:17:44.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.340+0000 7fba6cd90700 1 -- 172.21.15.112:0/198708403 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6400bd70 con 0x7fba6815bdc0 2023-08-27T11:17:44.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.341+0000 7fba6fff6700 1 -- 172.21.15.112:0/198708403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6815bdc0 msgr2=0x7fba680cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:44.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.341+0000 7fba6fff6700 1 --2- 172.21.15.112:0/198708403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6815bdc0 0x7fba680cf790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fba680ce760 tx=0x7fba6400b330 comp rx=0 tx=0).stop 2023-08-27T11:17:44.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.341+0000 7fba6fff6700 1 -- 172.21.15.112:0/198708403 shutdown_connections 2023-08-27T11:17:44.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.341+0000 7fba6fff6700 1 --2- 172.21.15.112:0/198708403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6815bdc0 0x7fba680cf790 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.341+0000 7fba6fff6700 1 --2- 172.21.15.112:0/198708403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba680cddb0 0x7fba6815f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.341+0000 7fba6fff6700 1 --2- 172.21.15.112:0/198708403 >> v2:172.21.15.138:3300/0 conn(0x7fba680ce190 0x7fba6815b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.341+0000 7fba6fff6700 1 -- 172.21.15.112:0/198708403 >> 172.21.15.112:0/198708403 conn(0x7fba6806f970 msgr2=0x7fba6806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.341+0000 7fba6fff6700 1 -- 172.21.15.112:0/198708403 shutdown_connections 2023-08-27T11:17:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.341+0000 7fba6fff6700 1 -- 172.21.15.112:0/198708403 wait complete. 2023-08-27T11:17:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.342+0000 7fba6fff6700 1 Processor -- start 2023-08-27T11:17:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.342+0000 7fba6fff6700 1 -- start start 2023-08-27T11:17:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.342+0000 7fba6fff6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba680cddb0 0x7fba68156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6fff6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba680ce190 0x7fba68152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6fff6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6815bdc0 0x7fba681f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6fff6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba68159400 con 0x7fba6815bdc0 2023-08-27T11:17:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6fff6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba681595e0 con 0x7fba680ce190 2023-08-27T11:17:44.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6fff6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba681598c0 con 0x7fba680cddb0 2023-08-27T11:17:44.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6e593700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6815bdc0 0x7fba681f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:44.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6e593700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6815bdc0 0x7fba681f43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56122/0 (socket says 172.21.15.112:56122) 2023-08-27T11:17:44.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6e593700 1 -- 172.21.15.112:0/104810780 learned_addr learned my addr 172.21.15.112:0/104810780 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:44.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6d591700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba680ce190 0x7fba68152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:44.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6dd92700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba680cddb0 0x7fba68156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:44.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6e593700 1 -- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba680cddb0 msgr2=0x7fba68156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:44.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6e593700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba680cddb0 0x7fba68156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6e593700 1 -- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba680ce190 msgr2=0x7fba68152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:44.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6e593700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba680ce190 0x7fba68152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6e593700 1 -- 172.21.15.112:0/104810780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba64009b30 con 0x7fba6815bdc0 2023-08-27T11:17:44.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6d591700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba680ce190 0x7fba68152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:44.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6dd92700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba680cddb0 0x7fba68156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:44.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6e593700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6815bdc0 0x7fba681f43a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fba6400b370 tx=0x7fba64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:44.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba5e7fc700 1 -- 172.21.15.112:0/104810780 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64004510 con 0x7fba6815bdc0 2023-08-27T11:17:44.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba5e7fc700 1 -- 172.21.15.112:0/104810780 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba64004690 con 0x7fba6815bdc0 2023-08-27T11:17:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6fff6700 1 -- 172.21.15.112:0/104810780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba68159ba0 con 0x7fba6815bdc0 2023-08-27T11:17:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba5e7fc700 1 -- 172.21.15.112:0/104810780 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64025460 con 0x7fba6815bdc0 2023-08-27T11:17:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.343+0000 7fba6fff6700 1 -- 172.21.15.112:0/104810780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba68157530 con 0x7fba6815bdc0 2023-08-27T11:17:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.344+0000 7fba6fff6700 1 -- 172.21.15.112:0/104810780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba6815a0c0 con 0x7fba6815bdc0 2023-08-27T11:17:44.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.345+0000 7fba5e7fc700 1 -- 172.21.15.112:0/104810780 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fba6401c410 con 0x7fba6815bdc0 2023-08-27T11:17:44.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.345+0000 7fba5e7fc700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fba4c06a0e0 0x7fba4c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.345+0000 7fba5e7fc700 1 -- 172.21.15.112:0/104810780 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba6408a3d0 con 0x7fba6815bdc0 2023-08-27T11:17:44.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.345+0000 7fba6dd92700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fba4c06a0e0 0x7fba4c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:44.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.346+0000 7fba6dd92700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fba4c06a0e0 0x7fba4c06c5a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba60007900 tx=0x7fba60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:44.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.349+0000 7fba5e7fc700 1 -- 172.21.15.112:0/104810780 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fba6815a0c0 con 0x7fba6815bdc0 2023-08-27T11:17:44.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.543+0000 7fba6fff6700 1 -- 172.21.15.112:0/104810780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba6815a0c0 con 0x7fba6815bdc0 2023-08-27T11:17:44.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.544+0000 7fba5e7fc700 1 -- 172.21.15.112:0/104810780 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7fba6815a0c0 con 0x7fba6815bdc0 2023-08-27T11:17:44.546 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:44.546 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:44.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.546+0000 7fba6fff6700 1 -- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fba4c06a0e0 msgr2=0x7fba4c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:44.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.546+0000 7fba6fff6700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fba4c06a0e0 0x7fba4c06c5a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba60007900 tx=0x7fba60008040 comp rx=0 tx=0).stop 2023-08-27T11:17:44.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.546+0000 7fba6fff6700 1 -- 172.21.15.112:0/104810780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6815bdc0 msgr2=0x7fba681f43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:44.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.546+0000 7fba6fff6700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6815bdc0 0x7fba681f43a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fba6400b370 tx=0x7fba64004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.546+0000 7fba6fff6700 1 -- 172.21.15.112:0/104810780 shutdown_connections 2023-08-27T11:17:44.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.546+0000 7fba6fff6700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fba4c06a0e0 0x7fba4c06c5a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.546+0000 7fba6fff6700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6815bdc0 0x7fba681f43a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.546+0000 7fba6fff6700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba680cddb0 0x7fba68156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.546+0000 7fba6fff6700 1 --2- 172.21.15.112:0/104810780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba680ce190 0x7fba68152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.546+0000 7fba6fff6700 1 -- 172.21.15.112:0/104810780 >> 172.21.15.112:0/104810780 conn(0x7fba6806f970 msgr2=0x7fba680cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:44.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.546+0000 7fba6fff6700 1 -- 172.21.15.112:0/104810780 shutdown_connections 2023-08-27T11:17:44.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.546+0000 7fba6fff6700 1 -- 172.21.15.112:0/104810780 wait complete. 2023-08-27T11:17:44.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:17:02.484063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:17:03.722976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:17:44.561 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:17:44.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.684+0000 7feef242d700 1 Processor -- start 2023-08-27T11:17:44.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.684+0000 7feef242d700 1 -- start start 2023-08-27T11:17:44.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.684+0000 7feef242d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feeec158ed0 0x7feeec15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.684+0000 7feef242d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeec15ca50 con 0x7feeec06e4c0 2023-08-27T11:17:44.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.684+0000 7feef242d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeec072490 con 0x7feeec072000 2023-08-27T11:17:44.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.684+0000 7feef242d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec15be20 con 0x7feeec158ed0 2023-08-27T11:17:44.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feef09ca700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7feeec06e4c0 0x7feeec15cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50508/0 (socket says 172.21.15.112:50508) 2023-08-27T11:17:44.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feef09ca700 1 -- 172.21.15.112:0/1280811776 learned_addr learned my addr 172.21.15.112:0/1280811776 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeebfff700 1 --2- 172.21.15.112:0/1280811776 >> v2:172.21.15.138:3300/0 conn(0x7feeec158ed0 0x7feeec15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955594871 0 0) 0x7feeec15ca50 con 0x7feeec06e4c0 2023-08-27T11:17:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feed40035b0 con 0x7feeec06e4c0 2023-08-27T11:17:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 100271993 0 0) 0x7feed40035b0 con 0x7feeec06e4c0 2023-08-27T11:17:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 >> v2:172.21.15.138:3300/0 conn(0x7feeec158ed0 msgr2=0x7feeec15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:44.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 --2- 172.21.15.112:0/1280811776 >> v2:172.21.15.138:3300/0 conn(0x7feeec158ed0 0x7feeec15b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 >> v1:172.21.15.138:6789/0 conn(0x7feeec072000 legacy=0x7feeec06ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:44.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec15c100 con 0x7feeec06e4c0 2023-08-27T11:17:44.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7feee40019f0 con 0x7feeec06e4c0 2023-08-27T11:17:44.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 >> v1:172.21.15.112:6789/0 conn(0x7feeec06e4c0 legacy=0x7feeec15cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:44.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 --2- 172.21.15.112:0/1280811776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed4003a00 0x7feed4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 --2- 172.21.15.112:0/1280811776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feed4004850 0x7feed4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 --2- 172.21.15.112:0/1280811776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed4005210 0x7feed400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec15be20 con 0x7feed4004850 2023-08-27T11:17:44.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed400c0b0 con 0x7feed4003a00 2023-08-27T11:17:44.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed400c270 con 0x7feed4005210 2023-08-27T11:17:44.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeeb7fe700 1 --2- 172.21.15.112:0/1280811776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feed4004850 0x7feed4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:44.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feeebfff700 1 --2- 172.21.15.112:0/1280811776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed4003a00 0x7feed4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:44.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.685+0000 7feef09ca700 1 --2- 172.21.15.112:0/1280811776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed4005210 0x7feed400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:44.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feeeb7fe700 1 -- 172.21.15.112:0/1280811776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed4005210 msgr2=0x7feed400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:44.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feeeb7fe700 1 --2- 172.21.15.112:0/1280811776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed4005210 0x7feed400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feeeb7fe700 1 -- 172.21.15.112:0/1280811776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed4003a00 msgr2=0x7feed4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:44.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feeeb7fe700 1 --2- 172.21.15.112:0/1280811776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed4003a00 0x7feed4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feeeb7fe700 1 -- 172.21.15.112:0/1280811776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed40035b0 con 0x7feed4004850 2023-08-27T11:17:44.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feef09ca700 1 --2- 172.21.15.112:0/1280811776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed4005210 0x7feed400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:44.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feef09ca700 1 -- 172.21.15.112:0/1280811776 reap_dead start 2023-08-27T11:17:44.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feeeb7fe700 1 --2- 172.21.15.112:0/1280811776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feed4004850 0x7feed4004cd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feee00017c0 tx=0x7feed4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a735dee0e70232 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:44.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee00052d0 con 0x7feed4004850 2023-08-27T11:17:44.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee0005430 con 0x7feed4004850 2023-08-27T11:17:44.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feeeaffd700 1 -- 172.21.15.112:0/1280811776 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee00038c0 con 0x7feed4004850 2023-08-27T11:17:44.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feef242d700 1 -- 172.21.15.112:0/1280811776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feed4004850 msgr2=0x7feed4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:44.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feef242d700 1 --2- 172.21.15.112:0/1280811776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feed4004850 0x7feed4004cd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feee00017c0 tx=0x7feed4005880 comp rx=0 tx=0).stop 2023-08-27T11:17:44.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feef242d700 1 -- 172.21.15.112:0/1280811776 shutdown_connections 2023-08-27T11:17:44.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feef242d700 1 --2- 172.21.15.112:0/1280811776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feed4004850 0x7feed4004cd0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feef242d700 1 -- 172.21.15.112:0/1280811776 >> 172.21.15.112:0/1280811776 conn(0x7feeec0cc240 msgr2=0x7feeec0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:44.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feef242d700 1 -- 172.21.15.112:0/1280811776 shutdown_connections 2023-08-27T11:17:44.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.686+0000 7feef242d700 1 -- 172.21.15.112:0/1280811776 wait complete. 2023-08-27T11:17:44.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.688+0000 7feef242d700 1 Processor -- start 2023-08-27T11:17:44.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.688+0000 7feef242d700 1 -- start start 2023-08-27T11:17:44.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.688+0000 7feef242d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed4004850 0x7feeec141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.688+0000 7feef242d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeec13e7e0 0x7feeec157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.688+0000 7feef242d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeec1433f0 0x7feeec143880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.688+0000 7feef242d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec142fd0 con 0x7feeec1433f0 2023-08-27T11:17:44.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.688+0000 7feef242d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec137ba0 con 0x7feed4004850 2023-08-27T11:17:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.688+0000 7feef242d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec142350 con 0x7feeec13e7e0 2023-08-27T11:17:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feef09ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeec1433f0 0x7feeec143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feef09ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeec1433f0 0x7feeec143880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56138/0 (socket says 172.21.15.112:56138) 2023-08-27T11:17:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feef09ca700 1 -- 172.21.15.112:0/2500663280 learned_addr learned my addr 172.21.15.112:0/2500663280 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feeebfff700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed4004850 0x7feeec141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feeeb7fe700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeec13e7e0 0x7feeec157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feef09ca700 1 -- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeec13e7e0 msgr2=0x7feeec157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feef09ca700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeec13e7e0 0x7feeec157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feef09ca700 1 -- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed4004850 msgr2=0x7feeec141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feef09ca700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed4004850 0x7feeec141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feef09ca700 1 -- 172.21.15.112:0/2500663280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec142630 con 0x7feeec1433f0 2023-08-27T11:17:44.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feef09ca700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeec1433f0 0x7feeec143880 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feeec0661e0 tx=0x7feee4007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:44.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feee8ff9700 1 -- 172.21.15.112:0/2500663280 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee400eba0 con 0x7feeec1433f0 2023-08-27T11:17:44.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feee8ff9700 1 -- 172.21.15.112:0/2500663280 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee400ed00 con 0x7feeec1433f0 2023-08-27T11:17:44.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feef242d700 1 -- 172.21.15.112:0/2500663280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec142910 con 0x7feeec1433f0 2023-08-27T11:17:44.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feee8ff9700 1 -- 172.21.15.112:0/2500663280 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4013620 con 0x7feeec1433f0 2023-08-27T11:17:44.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.689+0000 7feef242d700 1 -- 172.21.15.112:0/2500663280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec157970 con 0x7feeec1433f0 2023-08-27T11:17:44.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.690+0000 7feef242d700 1 -- 172.21.15.112:0/2500663280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeec143110 con 0x7feeec1433f0 2023-08-27T11:17:44.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.691+0000 7feee8ff9700 1 -- 172.21.15.112:0/2500663280 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7feee4013780 con 0x7feeec1433f0 2023-08-27T11:17:44.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.691+0000 7feee8ff9700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feed806a0e0 0x7feed806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:44.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.691+0000 7feee8ff9700 1 -- 172.21.15.112:0/2500663280 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feee4088990 con 0x7feeec1433f0 2023-08-27T11:17:44.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.691+0000 7feeebfff700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feed806a0e0 0x7feed806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:44.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.692+0000 7feeebfff700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feed806a0e0 0x7feed806c5a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7feedc009ce0 tx=0x7feedc009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:44.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.696+0000 7feee8ff9700 1 -- 172.21.15.112:0/2500663280 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7feeec143110 con 0x7feeec1433f0 2023-08-27T11:17:44.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.883+0000 7feef242d700 1 -- 172.21.15.112:0/2500663280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feeec143110 con 0x7feeec1433f0 2023-08-27T11:17:44.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.884+0000 7feee8ff9700 1 -- 172.21.15.112:0/2500663280 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7feeec143110 con 0x7feeec1433f0 2023-08-27T11:17:44.886 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:44.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.886+0000 7feef242d700 1 -- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feed806a0e0 msgr2=0x7feed806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:44.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.886+0000 7feef242d700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feed806a0e0 0x7feed806c5a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7feedc009ce0 tx=0x7feedc009210 comp rx=0 tx=0).stop 2023-08-27T11:17:44.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.886+0000 7feef242d700 1 -- 172.21.15.112:0/2500663280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeec1433f0 msgr2=0x7feeec143880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:44.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.886+0000 7feef242d700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeec1433f0 0x7feeec143880 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feeec0661e0 tx=0x7feee4007c90 comp rx=0 tx=0).stop 2023-08-27T11:17:44.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.886+0000 7feef242d700 1 -- 172.21.15.112:0/2500663280 shutdown_connections 2023-08-27T11:17:44.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.886+0000 7feef242d700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feed806a0e0 0x7feed806c5a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.886+0000 7feef242d700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeec1433f0 0x7feeec143880 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.886+0000 7feef242d700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed4004850 0x7feeec141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.886+0000 7feef242d700 1 --2- 172.21.15.112:0/2500663280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeec13e7e0 0x7feeec157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:44.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.886+0000 7feef242d700 1 -- 172.21.15.112:0/2500663280 >> 172.21.15.112:0/2500663280 conn(0x7feeec0cc240 msgr2=0x7feeec13c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:44.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.886+0000 7feef242d700 1 -- 172.21.15.112:0/2500663280 shutdown_connections 2023-08-27T11:17:44.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:44.886+0000 7feef242d700 1 -- 172.21.15.112:0/2500663280 wait complete. 2023-08-27T11:17:44.899 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:45.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.021+0000 7f3a0c878700 1 Processor -- start 2023-08-27T11:17:45.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.021+0000 7f3a0c878700 1 -- start start 2023-08-27T11:17:45.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.022+0000 7f3a0c878700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3a08146c20 0x7f3a08160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.022+0000 7f3a0c878700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a080b5070 con 0x7f3a080b3440 2023-08-27T11:17:45.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.022+0000 7f3a0c878700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a08147000 con 0x7f3a0806f790 2023-08-27T11:17:45.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.022+0000 7f3a0c878700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a081614a0 con 0x7f3a08146c20 2023-08-27T11:17:45.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.022+0000 7f3a0659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3a08146c20 0x7f3a08160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.022+0000 7f3a05d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3a080b3440 0x7f3a0815e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50526/0 (socket says 172.21.15.112:50526) 2023-08-27T11:17:45.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.022+0000 7f3a05d9b700 1 -- 172.21.15.112:0/2195116881 learned_addr learned my addr 172.21.15.112:0/2195116881 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:45.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.022+0000 7f3a0659c700 1 -- 172.21.15.112:0/2195116881 >> v1:172.21.15.138:6789/0 conn(0x7f3a0806f790 legacy=0x7f3a080b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:45.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.022+0000 7f3a0659c700 1 -- 172.21.15.112:0/2195116881 >> v1:172.21.15.112:6789/0 conn(0x7f3a080b3440 legacy=0x7f3a0815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.023+0000 7f3a0659c700 1 -- 172.21.15.112:0/2195116881 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a08161740 con 0x7f3a08146c20 2023-08-27T11:17:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.023+0000 7f3a0659c700 1 --2- 172.21.15.112:0/2195116881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a08146c20 0x7f3a08160f60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009840 tx=0x7f39fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4392fc0bbfa04c9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.023+0000 7f3a0559a700 1 -- 172.21.15.112:0/2195116881 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc004230 con 0x7f3a08146c20 2023-08-27T11:17:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.023+0000 7f3a0559a700 1 -- 172.21.15.112:0/2195116881 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc004390 con 0x7f3a08146c20 2023-08-27T11:17:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.023+0000 7f3a0559a700 1 -- 172.21.15.112:0/2195116881 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc0036a0 con 0x7f3a08146c20 2023-08-27T11:17:45.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.023+0000 7f3a0c878700 1 -- 172.21.15.112:0/2195116881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a08146c20 msgr2=0x7f3a08160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.023+0000 7f3a0c878700 1 --2- 172.21.15.112:0/2195116881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a08146c20 0x7f3a08160f60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009840 tx=0x7f39fc009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:45.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.023+0000 7f3a0c878700 1 -- 172.21.15.112:0/2195116881 shutdown_connections 2023-08-27T11:17:45.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.023+0000 7f3a0c878700 1 --2- 172.21.15.112:0/2195116881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a08146c20 0x7f3a08160f60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.023+0000 7f3a0c878700 1 -- 172.21.15.112:0/2195116881 >> 172.21.15.112:0/2195116881 conn(0x7f3a080b1e90 msgr2=0x7f3a080b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:45.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.023+0000 7f3a0c878700 1 -- 172.21.15.112:0/2195116881 shutdown_connections 2023-08-27T11:17:45.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.023+0000 7f3a0c878700 1 -- 172.21.15.112:0/2195116881 wait complete. 2023-08-27T11:17:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.025+0000 7f3a0c878700 1 Processor -- start 2023-08-27T11:17:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.025+0000 7f3a0c878700 1 -- start start 2023-08-27T11:17:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.025+0000 7f3a0c878700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a0806f790 0x7f3a081408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.025+0000 7f3a0c878700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a080b3440 0x7f3a081428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0c878700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a08146c20 0x7f3a08144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0c878700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a081f3370 con 0x7f3a0806f790 2023-08-27T11:17:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0c878700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a08145f30 con 0x7f3a080b3440 2023-08-27T11:17:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0c878700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a080c5220 con 0x7f3a08146c20 2023-08-27T11:17:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a0806f790 0x7f3a081408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a0806f790 0x7f3a081408a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56148/0 (socket says 172.21.15.112:56148) 2023-08-27T11:17:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0659c700 1 -- 172.21.15.112:0/3462926404 learned_addr learned my addr 172.21.15.112:0/3462926404 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a05d9b700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a080b3440 0x7f3a081428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a06d9d700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a08146c20 0x7f3a08144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0659c700 1 -- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a08146c20 msgr2=0x7f3a08144d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0659c700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a08146c20 0x7f3a08144d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0659c700 1 -- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a080b3440 msgr2=0x7f3a081428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0659c700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a080b3440 0x7f3a081428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0659c700 1 -- 172.21.15.112:0/3462926404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0094a0 con 0x7f3a0806f790 2023-08-27T11:17:45.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a05d9b700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a080b3440 0x7f3a081428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:45.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0659c700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a0806f790 0x7f3a081408a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0090b0 tx=0x7f39fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:45.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f39f6ffd700 1 -- 172.21.15.112:0/3462926404 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc003e30 con 0x7f3a0806f790 2023-08-27T11:17:45.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f39f6ffd700 1 -- 172.21.15.112:0/3462926404 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc01f070 con 0x7f3a0806f790 2023-08-27T11:17:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0c878700 1 -- 172.21.15.112:0/3462926404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a08145970 con 0x7f3a0806f790 2023-08-27T11:17:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f39f6ffd700 1 -- 172.21.15.112:0/3462926404 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc011850 con 0x7f3a0806f790 2023-08-27T11:17:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.026+0000 7f3a0c878700 1 -- 172.21.15.112:0/3462926404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a080c54c0 con 0x7f3a0806f790 2023-08-27T11:17:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.027+0000 7f3a0c878700 1 -- 172.21.15.112:0/3462926404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a080af990 con 0x7f3a0806f790 2023-08-27T11:17:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.028+0000 7f39f6ffd700 1 -- 172.21.15.112:0/3462926404 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f39fc021030 con 0x7f3a0806f790 2023-08-27T11:17:45.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.028+0000 7f39f6ffd700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39e806a0e0 0x7f39e806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.028+0000 7f39f6ffd700 1 -- 172.21.15.112:0/3462926404 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39fc0899f0 con 0x7f3a0806f790 2023-08-27T11:17:45.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.028+0000 7f3a05d9b700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39e806a0e0 0x7f39e806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.029+0000 7f3a05d9b700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39e806a0e0 0x7f39e806c5a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3a080cedb0 tx=0x7f39f8007e30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:45.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.032+0000 7f39f6ffd700 1 -- 172.21.15.112:0/3462926404 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f39fc055170 con 0x7f3a0806f790 2023-08-27T11:17:45.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.221+0000 7f3a0c878700 1 -- 172.21.15.112:0/3462926404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a080af990 con 0x7f3a0806f790 2023-08-27T11:17:45.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.222+0000 7f39f6ffd700 1 -- 172.21.15.112:0/3462926404 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39fc058790 con 0x7f3a0806f790 2023-08-27T11:17:45.223 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:45.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.224+0000 7f3a0c878700 1 -- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39e806a0e0 msgr2=0x7f39e806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.224+0000 7f3a0c878700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39e806a0e0 0x7f39e806c5a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3a080cedb0 tx=0x7f39f8007e30 comp rx=0 tx=0).stop 2023-08-27T11:17:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.224+0000 7f3a0c878700 1 -- 172.21.15.112:0/3462926404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a0806f790 msgr2=0x7f3a081408a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.224+0000 7f3a0c878700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a0806f790 0x7f3a081408a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0090b0 tx=0x7f39fc004750 comp rx=0 tx=0).stop 2023-08-27T11:17:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.224+0000 7f3a0c878700 1 -- 172.21.15.112:0/3462926404 shutdown_connections 2023-08-27T11:17:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.224+0000 7f3a0c878700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39e806a0e0 0x7f39e806c5a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.224+0000 7f3a0c878700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a08146c20 0x7f3a08144d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.224+0000 7f3a0c878700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a0806f790 0x7f3a081408a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.224+0000 7f3a0c878700 1 --2- 172.21.15.112:0/3462926404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a080b3440 0x7f3a081428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.224+0000 7f3a0c878700 1 -- 172.21.15.112:0/3462926404 >> 172.21.15.112:0/3462926404 conn(0x7f3a080b1e90 msgr2=0x7f3a081600b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.224+0000 7f3a0c878700 1 -- 172.21.15.112:0/3462926404 shutdown_connections 2023-08-27T11:17:45.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.224+0000 7f3a0c878700 1 -- 172.21.15.112:0/3462926404 wait complete. 2023-08-27T11:17:45.237 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:17:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.359+0000 7f2ea1b1c700 1 Processor -- start 2023-08-27T11:17:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.359+0000 7f2ea1b1c700 1 -- start start 2023-08-27T11:17:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.359+0000 7f2ea1b1c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e9c06f790 0x7f2e9c06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.359+0000 7f2ea1b1c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c0b5070 con 0x7f2e9c146c20 2023-08-27T11:17:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.359+0000 7f2ea1b1c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c147000 con 0x7f2e9c0b3440 2023-08-27T11:17:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.359+0000 7f2ea1b1c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c070160 con 0x7f2e9c06f790 2023-08-27T11:17:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2e9c146c20 0x7f2e9c160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50538/0 (socket says 172.21.15.112:50538) 2023-08-27T11:17:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9b7fe700 1 -- 172.21.15.112:0/168211715 learned_addr learned my addr 172.21.15.112:0/168211715 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9bfff700 1 --2- 172.21.15.112:0/168211715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e9c06f790 0x7f2e9c06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 461680610 0 0) 0x7f2e8c001110 con 0x7f2e9c146c20 2023-08-27T11:17:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e880035b0 con 0x7f2e9c146c20 2023-08-27T11:17:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3625091770 0 0) 0x7f2e880035b0 con 0x7f2e9c146c20 2023-08-27T11:17:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e9c06f790 msgr2=0x7f2e9c06fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 --2- 172.21.15.112:0/168211715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e9c06f790 0x7f2e9c06fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 >> v1:172.21.15.138:6789/0 conn(0x7f2e9c0b3440 legacy=0x7f2e9c15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c161740 con 0x7f2e9c146c20 2023-08-27T11:17:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f2e8c001800 con 0x7f2e9c146c20 2023-08-27T11:17:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 >> v1:172.21.15.112:6789/0 conn(0x7f2e9c146c20 legacy=0x7f2e9c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 --2- 172.21.15.112:0/168211715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e88003a30 0x7f2e88003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 --2- 172.21.15.112:0/168211715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e88004850 0x7f2e88004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 --2- 172.21.15.112:0/168211715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e9c06f790 0x7f2e88005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c070160 con 0x7f2e88003a30 2023-08-27T11:17:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8800bb70 con 0x7f2e88004850 2023-08-27T11:17:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8800bd30 con 0x7f2e9c06f790 2023-08-27T11:17:45.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9b7fe700 1 --2- 172.21.15.112:0/168211715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e88003a30 0x7f2e88003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9bfff700 1 --2- 172.21.15.112:0/168211715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e9c06f790 0x7f2e88005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.360+0000 7f2e9affd700 1 --2- 172.21.15.112:0/168211715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e88004850 0x7f2e88004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.361+0000 7f2e9b7fe700 1 -- 172.21.15.112:0/168211715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e9c06f790 msgr2=0x7f2e88005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:45.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.361+0000 7f2e9b7fe700 1 --2- 172.21.15.112:0/168211715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e9c06f790 0x7f2e88005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.361+0000 7f2e9b7fe700 1 -- 172.21.15.112:0/168211715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e88004850 msgr2=0x7f2e88004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:45.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.361+0000 7f2e9b7fe700 1 --2- 172.21.15.112:0/168211715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e88004850 0x7f2e88004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.361+0000 7f2e9b7fe700 1 -- 172.21.15.112:0/168211715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e880035b0 con 0x7f2e88003a30 2023-08-27T11:17:45.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.361+0000 7f2e9b7fe700 1 --2- 172.21.15.112:0/168211715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e88003a30 0x7f2e88003e50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00c240 tx=0x7f2e88005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e775711200ee987 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:45.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.362+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c007db0 con 0x7f2e88003a30 2023-08-27T11:17:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.362+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c00cde0 con 0x7f2e88003a30 2023-08-27T11:17:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.362+0000 7f2e9a7fc700 1 -- 172.21.15.112:0/168211715 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c005b80 con 0x7f2e88003a30 2023-08-27T11:17:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.363+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/168211715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e88003a30 msgr2=0x7f2e88003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.363+0000 7f2ea1b1c700 1 --2- 172.21.15.112:0/168211715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e88003a30 0x7f2e88003e50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00c240 tx=0x7f2e88005880 comp rx=0 tx=0).stop 2023-08-27T11:17:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.363+0000 7f2e9bfff700 1 -- 172.21.15.112:0/168211715 reap_dead start 2023-08-27T11:17:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.363+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/168211715 shutdown_connections 2023-08-27T11:17:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.363+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/168211715 >> 172.21.15.112:0/168211715 conn(0x7f2e9c0b1e90 msgr2=0x7f2e9c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.363+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/168211715 shutdown_connections 2023-08-27T11:17:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.363+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/168211715 wait complete. 2023-08-27T11:17:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.364+0000 7f2ea1b1c700 1 Processor -- start 2023-08-27T11:17:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.365+0000 7f2ea1b1c700 1 -- start start 2023-08-27T11:17:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.365+0000 7f2ea1b1c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e88003a30 0x7f2e9c071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.365+0000 7f2ea1b1c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e9c144dc0 0x7f2e9c1f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.365+0000 7f2ea1b1c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e9c1f39a0 0x7f2e9c15c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.365+0000 7f2ea1b1c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c1f2860 con 0x7f2e88003a30 2023-08-27T11:17:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.365+0000 7f2ea1b1c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c145760 con 0x7f2e9c1f39a0 2023-08-27T11:17:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.365+0000 7f2ea1b1c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c145a40 con 0x7f2e9c144dc0 2023-08-27T11:17:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.365+0000 7f2e9b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e88003a30 0x7f2e9c071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.365+0000 7f2e9b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e88003a30 0x7f2e9c071ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56168/0 (socket says 172.21.15.112:56168) 2023-08-27T11:17:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.365+0000 7f2e9b7fe700 1 -- 172.21.15.112:0/3336117746 learned_addr learned my addr 172.21.15.112:0/3336117746 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.365+0000 7f2e9bfff700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e9c1f39a0 0x7f2e9c15c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.365+0000 7f2e9affd700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e9c144dc0 0x7f2e9c1f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.366+0000 7f2e9b7fe700 1 -- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e9c144dc0 msgr2=0x7f2e9c1f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.366+0000 7f2e9b7fe700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e9c144dc0 0x7f2e9c1f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.366+0000 7f2e9b7fe700 1 -- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e9c1f39a0 msgr2=0x7f2e9c15c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.366+0000 7f2e9b7fe700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e9c1f39a0 0x7f2e9c15c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.366+0000 7f2e9b7fe700 1 -- 172.21.15.112:0/3336117746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c145d20 con 0x7f2e88003a30 2023-08-27T11:17:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.366+0000 7f2e9b7fe700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e88003a30 0x7f2e9c071ef0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800bb40 tx=0x7f2e8c001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.366+0000 7f2e93fff700 1 -- 172.21.15.112:0/3336117746 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c004c80 con 0x7f2e88003a30 2023-08-27T11:17:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.366+0000 7f2e93fff700 1 -- 172.21.15.112:0/3336117746 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c004de0 con 0x7f2e88003a30 2023-08-27T11:17:45.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.366+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/3336117746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c1f3e80 con 0x7f2e88003a30 2023-08-27T11:17:45.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.366+0000 7f2e93fff700 1 -- 172.21.15.112:0/3336117746 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c007310 con 0x7f2e88003a30 2023-08-27T11:17:45.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.366+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/3336117746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c15cc40 con 0x7f2e88003a30 2023-08-27T11:17:45.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.367+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/3336117746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c1f41f0 con 0x7f2e88003a30 2023-08-27T11:17:45.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.367+0000 7f2e93fff700 1 -- 172.21.15.112:0/3336117746 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2e8c019070 con 0x7f2e88003a30 2023-08-27T11:17:45.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.368+0000 7f2e93fff700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2e7c06a0e0 0x7f2e7c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.368+0000 7f2e93fff700 1 -- 172.21.15.112:0/3336117746 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2e8c088780 con 0x7f2e88003a30 2023-08-27T11:17:45.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.368+0000 7f2e9affd700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2e7c06a0e0 0x7f2e7c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.368+0000 7f2e9affd700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2e7c06a0e0 0x7f2e7c06c5a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c1f5f30 tx=0x7f2e84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:45.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.372+0000 7f2e93fff700 1 -- 172.21.15.112:0/3336117746 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2e8c053fb0 con 0x7f2e88003a30 2023-08-27T11:17:45.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.563+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/3336117746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2e9c0b5070 con 0x7f2e88003a30 2023-08-27T11:17:45.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.564+0000 7f2e93fff700 1 -- 172.21.15.112:0/3336117746 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2e9c0b5070 con 0x7f2e88003a30 2023-08-27T11:17:45.566 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:17:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.565+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2e7c06a0e0 msgr2=0x7f2e7c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.565+0000 7f2ea1b1c700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2e7c06a0e0 0x7f2e7c06c5a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c1f5f30 tx=0x7f2e84009040 comp rx=0 tx=0).stop 2023-08-27T11:17:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.565+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/3336117746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e88003a30 msgr2=0x7f2e9c071ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.565+0000 7f2ea1b1c700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e88003a30 0x7f2e9c071ef0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800bb40 tx=0x7f2e8c001b90 comp rx=0 tx=0).stop 2023-08-27T11:17:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.566+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/3336117746 shutdown_connections 2023-08-27T11:17:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.566+0000 7f2ea1b1c700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2e7c06a0e0 0x7f2e7c06c5a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.566+0000 7f2ea1b1c700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e9c1f39a0 0x7f2e9c15c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.566+0000 7f2ea1b1c700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e88003a30 0x7f2e9c071ef0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.566+0000 7f2ea1b1c700 1 --2- 172.21.15.112:0/3336117746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e9c144dc0 0x7f2e9c1f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.566+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/3336117746 >> 172.21.15.112:0/3336117746 conn(0x7f2e9c0b1e90 msgr2=0x7f2e9c0b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.566+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/3336117746 shutdown_connections 2023-08-27T11:17:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.566+0000 7f2ea1b1c700 1 -- 172.21.15.112:0/3336117746 wait complete. 2023-08-27T11:17:45.579 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:17:45.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.700+0000 7f829d4f2700 1 Processor -- start 2023-08-27T11:17:45.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.700+0000 7f829d4f2700 1 -- start start 2023-08-27T11:17:45.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.701+0000 7f829d4f2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8298144900 0x7f82980b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.701+0000 7f829d4f2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f82980cca90 0x7f829815f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.701+0000 7f829d4f2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f829806f790 0x7f829806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.701+0000 7f829d4f2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8298070140 con 0x7f8298144900 2023-08-27T11:17:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.701+0000 7f829d4f2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f829815ee90 con 0x7f82980cca90 2023-08-27T11:17:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.701+0000 7f829d4f2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82980b11d0 con 0x7f829806f790 2023-08-27T11:17:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.701+0000 7f8296ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8298144900 0x7f82980b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.701+0000 7f8296ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8298144900 0x7f82980b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56176/0 (socket says 172.21.15.112:56176) 2023-08-27T11:17:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.701+0000 7f8296ffd700 1 -- 172.21.15.112:0/1989597340 learned_addr learned my addr 172.21.15.112:0/1989597340 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.701+0000 7f82967fc700 1 --2- 172.21.15.112:0/1989597340 >> v2:172.21.15.138:3300/0 conn(0x7f82980cca90 0x7f829815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.701+0000 7f82977fe700 1 --2- 172.21.15.112:0/1989597340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f829806f790 0x7f829806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f8296ffd700 1 -- 172.21.15.112:0/1989597340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f829806f790 msgr2=0x7f829806fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f8296ffd700 1 --2- 172.21.15.112:0/1989597340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f829806f790 0x7f829806fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f8296ffd700 1 -- 172.21.15.112:0/1989597340 >> v2:172.21.15.138:3300/0 conn(0x7f82980cca90 msgr2=0x7f829815f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:45.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f8296ffd700 1 --2- 172.21.15.112:0/1989597340 >> v2:172.21.15.138:3300/0 conn(0x7f82980cca90 0x7f829815f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f8296ffd700 1 -- 172.21.15.112:0/1989597340 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82980b14b0 con 0x7f8298144900 2023-08-27T11:17:45.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f82977fe700 1 --2- 172.21.15.112:0/1989597340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f829806f790 0x7f829806fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:45.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f82967fc700 1 --2- 172.21.15.112:0/1989597340 >> v2:172.21.15.138:3300/0 conn(0x7f82980cca90 0x7f829815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f8296ffd700 1 --2- 172.21.15.112:0/1989597340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8298144900 0x7f82980b09a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f82980703e0 tx=0x7f828c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970d7c379078b507 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f8295ffb700 1 -- 172.21.15.112:0/1989597340 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c00b870 con 0x7f8298144900 2023-08-27T11:17:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f8295ffb700 1 -- 172.21.15.112:0/1989597340 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c00b9d0 con 0x7f8298144900 2023-08-27T11:17:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f8295ffb700 1 -- 172.21.15.112:0/1989597340 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c00bd60 con 0x7f8298144900 2023-08-27T11:17:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f829d4f2700 1 -- 172.21.15.112:0/1989597340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8298144900 msgr2=0x7f82980b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f829d4f2700 1 --2- 172.21.15.112:0/1989597340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8298144900 0x7f82980b09a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f82980703e0 tx=0x7f828c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f829d4f2700 1 -- 172.21.15.112:0/1989597340 shutdown_connections 2023-08-27T11:17:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f829d4f2700 1 --2- 172.21.15.112:0/1989597340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f829806f790 0x7f829806fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f829d4f2700 1 --2- 172.21.15.112:0/1989597340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8298144900 0x7f82980b09a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f829d4f2700 1 --2- 172.21.15.112:0/1989597340 >> v2:172.21.15.138:3300/0 conn(0x7f82980cca90 0x7f829815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f829d4f2700 1 -- 172.21.15.112:0/1989597340 >> 172.21.15.112:0/1989597340 conn(0x7f82980b1e90 msgr2=0x7f82980b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f829d4f2700 1 -- 172.21.15.112:0/1989597340 shutdown_connections 2023-08-27T11:17:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.702+0000 7f829d4f2700 1 -- 172.21.15.112:0/1989597340 wait complete. 2023-08-27T11:17:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.704+0000 7f829d4f2700 1 Processor -- start 2023-08-27T11:17:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.704+0000 7f829d4f2700 1 -- start start 2023-08-27T11:17:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.704+0000 7f829d4f2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f829806f790 0x7f829815a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.704+0000 7f829d4f2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82980cca90 0x7f82980712f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.704+0000 7f829d4f2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8298144900 0x7f829813fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.704+0000 7f829d4f2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82981f2af0 con 0x7f82980cca90 2023-08-27T11:17:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.704+0000 7f829d4f2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82980ae1b0 con 0x7f8298144900 2023-08-27T11:17:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.704+0000 7f829d4f2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829815abb0 con 0x7f829806f790 2023-08-27T11:17:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f82967fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82980cca90 0x7f82980712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f82967fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82980cca90 0x7f82980712f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56182/0 (socket says 172.21.15.112:56182) 2023-08-27T11:17:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f82967fc700 1 -- 172.21.15.112:0/1146671996 learned_addr learned my addr 172.21.15.112:0/1146671996 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f8296ffd700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f829806f790 0x7f829815a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f82977fe700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8298144900 0x7f829813fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f82967fc700 1 -- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f829806f790 msgr2=0x7f829815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f82967fc700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f829806f790 0x7f829815a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f82967fc700 1 -- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8298144900 msgr2=0x7f829813fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f82967fc700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8298144900 0x7f829813fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f82967fc700 1 -- 172.21.15.112:0/1146671996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c009550 con 0x7f82980cca90 2023-08-27T11:17:45.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f8296ffd700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f829806f790 0x7f829815a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:45.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f82967fc700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82980cca90 0x7f82980712f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f828800e840 tx=0x7f828800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:45.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f827f7fe700 1 -- 172.21.15.112:0/1146671996 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288004230 con 0x7f82980cca90 2023-08-27T11:17:45.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f829d4f2700 1 -- 172.21.15.112:0/1146671996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8298071c40 con 0x7f82980cca90 2023-08-27T11:17:45.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f829d4f2700 1 -- 172.21.15.112:0/1146671996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82980ade40 con 0x7f82980cca90 2023-08-27T11:17:45.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f827f7fe700 1 -- 172.21.15.112:0/1146671996 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8288004390 con 0x7f82980cca90 2023-08-27T11:17:45.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.705+0000 7f827f7fe700 1 -- 172.21.15.112:0/1146671996 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82880036a0 con 0x7f82980cca90 2023-08-27T11:17:45.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.706+0000 7f829d4f2700 1 -- 172.21.15.112:0/1146671996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82980ad9c0 con 0x7f82980cca90 2023-08-27T11:17:45.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.707+0000 7f827f7fe700 1 -- 172.21.15.112:0/1146671996 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8288003800 con 0x7f82980cca90 2023-08-27T11:17:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.707+0000 7f827f7fe700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f828406a0e0 0x7f828406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.707+0000 7f827f7fe700 1 -- 172.21.15.112:0/1146671996 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8288088450 con 0x7f82980cca90 2023-08-27T11:17:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.707+0000 7f8296ffd700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f828406a0e0 0x7f828406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.708+0000 7f8296ffd700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f828406a0e0 0x7f828406c5a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f828c009130 tx=0x7f828c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.711+0000 7f827f7fe700 1 -- 172.21.15.112:0/1146671996 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f82980ad9c0 con 0x7f82980cca90 2023-08-27T11:17:45.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.900+0000 7f829d4f2700 1 -- 172.21.15.112:0/1146671996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82980ad9c0 con 0x7f82980cca90 2023-08-27T11:17:45.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.900+0000 7f827f7fe700 1 -- 172.21.15.112:0/1146671996 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f82980ad9c0 con 0x7f82980cca90 2023-08-27T11:17:45.903 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:17:45.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.902+0000 7f829d4f2700 1 -- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f828406a0e0 msgr2=0x7f828406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.902+0000 7f829d4f2700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f828406a0e0 0x7f828406c5a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f828c009130 tx=0x7f828c008030 comp rx=0 tx=0).stop 2023-08-27T11:17:45.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.902+0000 7f829d4f2700 1 -- 172.21.15.112:0/1146671996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82980cca90 msgr2=0x7f82980712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:45.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.902+0000 7f829d4f2700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82980cca90 0x7f82980712f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f828800e840 tx=0x7f828800ec00 comp rx=0 tx=0).stop 2023-08-27T11:17:45.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.902+0000 7f829d4f2700 1 -- 172.21.15.112:0/1146671996 shutdown_connections 2023-08-27T11:17:45.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.902+0000 7f829d4f2700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f828406a0e0 0x7f828406c5a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.902+0000 7f829d4f2700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8298144900 0x7f829813fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.902+0000 7f829d4f2700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f829806f790 0x7f829815a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.902+0000 7f829d4f2700 1 --2- 172.21.15.112:0/1146671996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82980cca90 0x7f82980712f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:45.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.902+0000 7f829d4f2700 1 -- 172.21.15.112:0/1146671996 >> 172.21.15.112:0/1146671996 conn(0x7f82980b1e90 msgr2=0x7f829813e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:45.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.902+0000 7f829d4f2700 1 -- 172.21.15.112:0/1146671996 shutdown_connections 2023-08-27T11:17:45.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:45.902+0000 7f829d4f2700 1 -- 172.21.15.112:0/1146671996 wait complete. 2023-08-27T11:17:45.916 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:17:46.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.035+0000 7f776fd72700 1 Processor -- start 2023-08-27T11:17:46.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.035+0000 7f776fd72700 1 -- start start 2023-08-27T11:17:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776fd72700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f77680b1400 0x7f77680cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776fd72700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f77680b17e0 0x7f776806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776fd72700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7768161850 con 0x7f7768161ce0 2023-08-27T11:17:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776fd72700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776806e7d0 con 0x7f77680b1400 2023-08-27T11:17:46.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776fd72700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77680cfc10 con 0x7f77680b17e0 2023-08-27T11:17:46.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776db0e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f77680b1400 0x7f77680cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776e30f700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7768161ce0 0x7f776806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50574/0 (socket says 172.21.15.112:50574) 2023-08-27T11:17:46.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776e30f700 1 -- 172.21.15.112:0/1286469284 learned_addr learned my addr 172.21.15.112:0/1286469284 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:46.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776d30d700 1 --2- 172.21.15.112:0/1286469284 >> v2:172.21.15.138:3300/0 conn(0x7f77680b17e0 0x7f776806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776db0e700 1 -- 172.21.15.112:0/1286469284 >> v2:172.21.15.138:3300/0 conn(0x7f77680b17e0 msgr2=0x7f776806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:46.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776db0e700 1 --2- 172.21.15.112:0/1286469284 >> v2:172.21.15.138:3300/0 conn(0x7f77680b17e0 0x7f776806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776db0e700 1 -- 172.21.15.112:0/1286469284 >> v1:172.21.15.112:6789/0 conn(0x7f7768161ce0 legacy=0x7f776806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776db0e700 1 -- 172.21.15.112:0/1286469284 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7768161200 con 0x7f77680b1400 2023-08-27T11:17:46.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776cb0c700 1 -- 172.21.15.112:0/1286469284 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113874871 0 0) 0x7f7768161850 con 0x7f7768161ce0 2023-08-27T11:17:46.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.036+0000 7f776d30d700 1 --2- 172.21.15.112:0/1286469284 >> v2:172.21.15.138:3300/0 conn(0x7f77680b17e0 0x7f776806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:46.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.037+0000 7f776db0e700 1 --2- 172.21.15.112:0/1286469284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77680b1400 0x7f77680cf5a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f77680cc420 tx=0x7f7758009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ba9dbd28ab0d8f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:46.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.037+0000 7f776cb0c700 1 -- 172.21.15.112:0/1286469284 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758011830 con 0x7f77680b1400 2023-08-27T11:17:46.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.037+0000 7f776cb0c700 1 -- 172.21.15.112:0/1286469284 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7758011990 con 0x7f77680b1400 2023-08-27T11:17:46.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.037+0000 7f776cb0c700 1 -- 172.21.15.112:0/1286469284 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758011d20 con 0x7f77680b1400 2023-08-27T11:17:46.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.037+0000 7f776fd72700 1 -- 172.21.15.112:0/1286469284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77680b1400 msgr2=0x7f77680cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.037+0000 7f776fd72700 1 --2- 172.21.15.112:0/1286469284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77680b1400 0x7f77680cf5a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f77680cc420 tx=0x7f7758009b20 comp rx=0 tx=0).stop 2023-08-27T11:17:46.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.037+0000 7f776fd72700 1 -- 172.21.15.112:0/1286469284 shutdown_connections 2023-08-27T11:17:46.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.037+0000 7f776fd72700 1 --2- 172.21.15.112:0/1286469284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77680b1400 0x7f77680cf5a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.037+0000 7f776fd72700 1 --2- 172.21.15.112:0/1286469284 >> v2:172.21.15.138:3300/0 conn(0x7f77680b17e0 0x7f776806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.037+0000 7f776fd72700 1 -- 172.21.15.112:0/1286469284 >> 172.21.15.112:0/1286469284 conn(0x7f77680d0000 msgr2=0x7f77680d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:46.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.037+0000 7f776fd72700 1 -- 172.21.15.112:0/1286469284 shutdown_connections 2023-08-27T11:17:46.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.037+0000 7f776fd72700 1 -- 172.21.15.112:0/1286469284 wait complete. 2023-08-27T11:17:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.039+0000 7f776fd72700 1 Processor -- start 2023-08-27T11:17:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.039+0000 7f776fd72700 1 -- start start 2023-08-27T11:17:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.039+0000 7f776fd72700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77680b1400 0x7f77680c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.039+0000 7f776fd72700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77680b17e0 0x7f77681f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.039+0000 7f776fd72700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7768161ce0 0x7f776806ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.039+0000 7f776fd72700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77680c4c90 con 0x7f77680b1400 2023-08-27T11:17:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.039+0000 7f776fd72700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77680c4080 con 0x7f77680b17e0 2023-08-27T11:17:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.039+0000 7f776fd72700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77680c4360 con 0x7f7768161ce0 2023-08-27T11:17:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.039+0000 7f776db0e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77680b1400 0x7f77680c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.039+0000 7f776db0e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77680b1400 0x7f77680c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56194/0 (socket says 172.21.15.112:56194) 2023-08-27T11:17:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.039+0000 7f776db0e700 1 -- 172.21.15.112:0/1163069526 learned_addr learned my addr 172.21.15.112:0/1163069526 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f776d30d700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77680b17e0 0x7f77681f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f776e30f700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7768161ce0 0x7f776806ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f776db0e700 1 -- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7768161ce0 msgr2=0x7f776806ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f776db0e700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7768161ce0 0x7f776806ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f776db0e700 1 -- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77680b17e0 msgr2=0x7f77681f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f776db0e700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77680b17e0 0x7f77681f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f776db0e700 1 -- 172.21.15.112:0/1163069526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580094a0 con 0x7f77680b1400 2023-08-27T11:17:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f776e30f700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7768161ce0 0x7f776806ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f776db0e700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77680b1400 0x7f77680c5950 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7758000c00 tx=0x7f7758004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f775e7fc700 1 -- 172.21.15.112:0/1163069526 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77580038a0 con 0x7f77680b1400 2023-08-27T11:17:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f775e7fc700 1 -- 172.21.15.112:0/1163069526 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775801f070 con 0x7f77680b1400 2023-08-27T11:17:46.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f776fd72700 1 -- 172.21.15.112:0/1163069526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77680c4640 con 0x7f77680b1400 2023-08-27T11:17:46.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f775e7fc700 1 -- 172.21.15.112:0/1163069526 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758003c40 con 0x7f77680b1400 2023-08-27T11:17:46.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.040+0000 7f776fd72700 1 -- 172.21.15.112:0/1163069526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77680c5e90 con 0x7f77680b1400 2023-08-27T11:17:46.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.041+0000 7f776fd72700 1 -- 172.21.15.112:0/1163069526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7768067a20 con 0x7f77680b1400 2023-08-27T11:17:46.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.042+0000 7f775e7fc700 1 -- 172.21.15.112:0/1163069526 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7758026480 con 0x7f77680b1400 2023-08-27T11:17:46.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.042+0000 7f775e7fc700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f774c06a0e0 0x7f774c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.042+0000 7f775e7fc700 1 -- 172.21.15.112:0/1163069526 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7758021070 con 0x7f77680b1400 2023-08-27T11:17:46.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.042+0000 7f776d30d700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f774c06a0e0 0x7f774c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.042+0000 7f776d30d700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f774c06a0e0 0x7f774c06c5a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7768155a90 tx=0x7f7764008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:46.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.046+0000 7f775e7fc700 1 -- 172.21.15.112:0/1163069526 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f77580556e0 con 0x7f77680b1400 2023-08-27T11:17:46.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.253+0000 7f776fd72700 1 -- 172.21.15.112:0/1163069526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f776806db60 con 0x7f77680b1400 2023-08-27T11:17:46.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.253+0000 7f775e7fc700 1 -- 172.21.15.112:0/1163069526 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7758015070 con 0x7f77680b1400 2023-08-27T11:17:46.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.255+0000 7f776fd72700 1 -- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f774c06a0e0 msgr2=0x7f774c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.255+0000 7f776fd72700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f774c06a0e0 0x7f774c06c5a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7768155a90 tx=0x7f7764008040 comp rx=0 tx=0).stop 2023-08-27T11:17:46.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.255+0000 7f776fd72700 1 -- 172.21.15.112:0/1163069526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77680b1400 msgr2=0x7f77680c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.255+0000 7f776fd72700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77680b1400 0x7f77680c5950 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7758000c00 tx=0x7f7758004750 comp rx=0 tx=0).stop 2023-08-27T11:17:46.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.255+0000 7f776fd72700 1 -- 172.21.15.112:0/1163069526 shutdown_connections 2023-08-27T11:17:46.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.256+0000 7f776fd72700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f774c06a0e0 0x7f774c06c5a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.256+0000 7f776fd72700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7768161ce0 0x7f776806ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.256+0000 7f776fd72700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77680b1400 0x7f77680c5950 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.256+0000 7f776fd72700 1 --2- 172.21.15.112:0/1163069526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77680b17e0 0x7f77681f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.256+0000 7f776fd72700 1 -- 172.21.15.112:0/1163069526 >> 172.21.15.112:0/1163069526 conn(0x7f77680d0000 msgr2=0x7f77680cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:46.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.256+0000 7f776fd72700 1 -- 172.21.15.112:0/1163069526 shutdown_connections 2023-08-27T11:17:46.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.256+0000 7f776fd72700 1 -- 172.21.15.112:0/1163069526 wait complete. 2023-08-27T11:17:46.260 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:17:46.269 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-08-27T11:17:46.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:46.270 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-08-27T11:17:46.308 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:17:46.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:46.309 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-27T11:17:46.365 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:17:46.365 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:46.386 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:17:46.387 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:46.446 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:17:46.447 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:17:46.506 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:46.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.625+0000 7fd715b8e700 1 Processor -- start 2023-08-27T11:17:46.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.625+0000 7fd715b8e700 1 -- start start 2023-08-27T11:17:46.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.626+0000 7fd715b8e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd710159070 0x7fd71006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.626+0000 7fd715b8e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd7100b1270 0x7fd71006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.626+0000 7fd715b8e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd71015f5d0 con 0x7fd710158c90 2023-08-27T11:17:46.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.626+0000 7fd715b8e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7100ca990 con 0x7fd710159070 2023-08-27T11:17:46.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.626+0000 7fd715b8e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7100b1a10 con 0x7fd7100b1270 2023-08-27T11:17:46.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.626+0000 7fd70effd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd710159070 0x7fd71006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.626+0000 7fd70effd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd710159070 0x7fd71006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56204/0 (socket says 172.21.15.112:56204) 2023-08-27T11:17:46.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.626+0000 7fd70effd700 1 -- 172.21.15.112:0/4010978577 learned_addr learned my addr 172.21.15.112:0/4010978577 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:46.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.627+0000 7fd70ffff700 1 --2- 172.21.15.112:0/4010978577 >> v2:172.21.15.138:3300/0 conn(0x7fd7100b1270 0x7fd71006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.627+0000 7fd70effd700 1 -- 172.21.15.112:0/4010978577 >> v2:172.21.15.138:3300/0 conn(0x7fd7100b1270 msgr2=0x7fd71006fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:46.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.627+0000 7fd70effd700 1 --2- 172.21.15.112:0/4010978577 >> v2:172.21.15.138:3300/0 conn(0x7fd7100b1270 0x7fd71006fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.627+0000 7fd70effd700 1 -- 172.21.15.112:0/4010978577 >> v1:172.21.15.138:6789/0 conn(0x7fd710158c90 legacy=0x7fd71006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:46.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.627+0000 7fd70effd700 1 -- 172.21.15.112:0/4010978577 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7100b1730 con 0x7fd710159070 2023-08-27T11:17:46.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.627+0000 7fd70effd700 1 --2- 172.21.15.112:0/4010978577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd710159070 0x7fd71006f790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd7100ca5a0 tx=0x7fd704009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf1486815cd2985 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:46.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.627+0000 7fd70e7fc700 1 -- 172.21.15.112:0/4010978577 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704004230 con 0x7fd710159070 2023-08-27T11:17:46.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.627+0000 7fd70e7fc700 1 -- 172.21.15.112:0/4010978577 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd704004390 con 0x7fd710159070 2023-08-27T11:17:46.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.627+0000 7fd70e7fc700 1 -- 172.21.15.112:0/4010978577 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7040036a0 con 0x7fd710159070 2023-08-27T11:17:46.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.627+0000 7fd715b8e700 1 -- 172.21.15.112:0/4010978577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd710159070 msgr2=0x7fd71006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.627+0000 7fd715b8e700 1 --2- 172.21.15.112:0/4010978577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd710159070 0x7fd71006f790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd7100ca5a0 tx=0x7fd704009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.628+0000 7fd715b8e700 1 -- 172.21.15.112:0/4010978577 shutdown_connections 2023-08-27T11:17:46.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.628+0000 7fd715b8e700 1 --2- 172.21.15.112:0/4010978577 >> v2:172.21.15.138:3300/0 conn(0x7fd7100b1270 0x7fd71006fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.628+0000 7fd715b8e700 1 --2- 172.21.15.112:0/4010978577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd710159070 0x7fd71006f790 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.628+0000 7fd715b8e700 1 -- 172.21.15.112:0/4010978577 >> 172.21.15.112:0/4010978577 conn(0x7fd7100d0000 msgr2=0x7fd7100d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:46.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.628+0000 7fd715b8e700 1 -- 172.21.15.112:0/4010978577 shutdown_connections 2023-08-27T11:17:46.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.628+0000 7fd715b8e700 1 -- 172.21.15.112:0/4010978577 wait complete. 2023-08-27T11:17:46.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.629+0000 7fd715b8e700 1 Processor -- start 2023-08-27T11:17:46.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.629+0000 7fd715b8e700 1 -- start start 2023-08-27T11:17:46.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd715b8e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7100b1270 0x7fd710151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd715b8e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd710158c90 0x7fd71006f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd715b8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd710159070 0x7fd7100c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd715b8e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100c8820 con 0x7fd710159070 2023-08-27T11:17:46.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd715b8e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd710151da0 con 0x7fd7100b1270 2023-08-27T11:17:46.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd715b8e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100c8210 con 0x7fd710158c90 2023-08-27T11:17:46.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd70ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd710159070 0x7fd7100c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd70ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd710159070 0x7fd7100c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56212/0 (socket says 172.21.15.112:56212) 2023-08-27T11:17:46.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd70ffff700 1 -- 172.21.15.112:0/3559198725 learned_addr learned my addr 172.21.15.112:0/3559198725 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:46.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd70f7fe700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7100b1270 0x7fd710151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd70effd700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd710158c90 0x7fd71006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd70ffff700 1 -- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd710158c90 msgr2=0x7fd71006f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd70ffff700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd710158c90 0x7fd71006f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd70ffff700 1 -- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7100b1270 msgr2=0x7fd710151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd70ffff700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7100b1270 0x7fd710151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd70ffff700 1 -- 172.21.15.112:0/3559198725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd704009550 con 0x7fd710159070 2023-08-27T11:17:46.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd70effd700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd710158c90 0x7fd71006f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:46.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd70ffff700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd710159070 0x7fd7100c8cb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd70800cd70 tx=0x7fd70800a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:46.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd6f7fff700 1 -- 172.21.15.112:0/3559198725 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708003a20 con 0x7fd710159070 2023-08-27T11:17:46.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd6f7fff700 1 -- 172.21.15.112:0/3559198725 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd708003b80 con 0x7fd710159070 2023-08-27T11:17:46.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd715b8e700 1 -- 172.21.15.112:0/3559198725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd710155c90 con 0x7fd710159070 2023-08-27T11:17:46.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd6f7fff700 1 -- 172.21.15.112:0/3559198725 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708004020 con 0x7fd710159070 2023-08-27T11:17:46.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.630+0000 7fd715b8e700 1 -- 172.21.15.112:0/3559198725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd710155f70 con 0x7fd710159070 2023-08-27T11:17:46.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.631+0000 7fd715b8e700 1 -- 172.21.15.112:0/3559198725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71015fa40 con 0x7fd710159070 2023-08-27T11:17:46.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.632+0000 7fd6f7fff700 1 -- 172.21.15.112:0/3559198725 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd708004180 con 0x7fd710159070 2023-08-27T11:17:46.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.632+0000 7fd6f7fff700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6fc06a0e0 0x7fd6fc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.632+0000 7fd6f7fff700 1 -- 172.21.15.112:0/3559198725 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd708012070 con 0x7fd710159070 2023-08-27T11:17:46.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.632+0000 7fd70f7fe700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6fc06a0e0 0x7fd6fc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.633+0000 7fd70f7fe700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6fc06a0e0 0x7fd6fc06c5a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd700007fd0 tx=0x7fd700007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:46.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.637+0000 7fd6f7fff700 1 -- 172.21.15.112:0/3559198725 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd71015fa40 con 0x7fd710159070 2023-08-27T11:17:46.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.826+0000 7fd715b8e700 1 -- 172.21.15.112:0/3559198725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd71015fa40 con 0x7fd710159070 2023-08-27T11:17:46.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.827+0000 7fd6f7fff700 1 -- 172.21.15.112:0/3559198725 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7fd71015fa40 con 0x7fd710159070 2023-08-27T11:17:46.829 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:46.829 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:46.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.829+0000 7fd715b8e700 1 -- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6fc06a0e0 msgr2=0x7fd6fc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.829+0000 7fd715b8e700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6fc06a0e0 0x7fd6fc06c5a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd700007fd0 tx=0x7fd700007ec0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.829+0000 7fd715b8e700 1 -- 172.21.15.112:0/3559198725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd710159070 msgr2=0x7fd7100c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.829+0000 7fd715b8e700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd710159070 0x7fd7100c8cb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd70800cd70 tx=0x7fd70800a570 comp rx=0 tx=0).stop 2023-08-27T11:17:46.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.829+0000 7fd715b8e700 1 -- 172.21.15.112:0/3559198725 shutdown_connections 2023-08-27T11:17:46.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.829+0000 7fd715b8e700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6fc06a0e0 0x7fd6fc06c5a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.829+0000 7fd715b8e700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd710159070 0x7fd7100c8cb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.829+0000 7fd715b8e700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7100b1270 0x7fd710151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.829+0000 7fd715b8e700 1 --2- 172.21.15.112:0/3559198725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd710158c90 0x7fd71006f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.829+0000 7fd715b8e700 1 -- 172.21.15.112:0/3559198725 >> 172.21.15.112:0/3559198725 conn(0x7fd7100d0000 msgr2=0x7fd7100c3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:46.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.829+0000 7fd715b8e700 1 -- 172.21.15.112:0/3559198725 shutdown_connections 2023-08-27T11:17:46.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.829+0000 7fd715b8e700 1 -- 172.21.15.112:0/3559198725 wait complete. 2023-08-27T11:17:46.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:17:02.484063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:17:03.722976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:17:46.844 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:17:46.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.965+0000 7f4e3819b700 1 Processor -- start 2023-08-27T11:17:46.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.965+0000 7f4e3819b700 1 -- start start 2023-08-27T11:17:46.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e3819b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4e301437c0 0x7f4e300b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e3819b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e3006d5b0 0x7f4e3006da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e3819b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e300cfd50 con 0x7f4e30143ba0 2023-08-27T11:17:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e3819b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e3006df80 con 0x7f4e301437c0 2023-08-27T11:17:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e3819b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300cf140 con 0x7f4e3006d5b0 2023-08-27T11:17:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e36738700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e3006d5b0 0x7f4e3006da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e35f37700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4e301437c0 0x7f4e300b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e35736700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4e30143ba0 0x7f4e300ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:58122/0 (socket says 172.21.15.112:58122) 2023-08-27T11:17:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e36738700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e3006d5b0 0x7f4e3006da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:39640/0 (socket says 172.21.15.112:39640) 2023-08-27T11:17:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e35736700 1 -- 172.21.15.112:0/2227090035 learned_addr learned my addr 172.21.15.112:0/2227090035 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e36738700 1 -- 172.21.15.112:0/2227090035 >> v2:172.21.15.138:3300/0 conn(0x7f4e301437c0 msgr2=0x7f4e300b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e36738700 1 --2- 172.21.15.112:0/2227090035 >> v2:172.21.15.138:3300/0 conn(0x7f4e301437c0 0x7f4e300b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e36738700 1 -- 172.21.15.112:0/2227090035 >> v1:172.21.15.138:6789/0 conn(0x7f4e30143ba0 legacy=0x7f4e300ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e36738700 1 -- 172.21.15.112:0/2227090035 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e300cf420 con 0x7f4e3006d5b0 2023-08-27T11:17:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.966+0000 7f4e35f37700 1 --2- 172.21.15.112:0/2227090035 >> v2:172.21.15.138:3300/0 conn(0x7f4e301437c0 0x7f4e300b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:17:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.967+0000 7f4e36738700 1 --2- 172.21.15.112:0/2227090035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e3006d5b0 0x7f4e3006da40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009d90 tx=0x7f4e2800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21ef8faf65c6bbb1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.967+0000 7f4e34f35700 1 -- 172.21.15.112:0/2227090035 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800e070 con 0x7f4e3006d5b0 2023-08-27T11:17:46.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.967+0000 7f4e34f35700 1 -- 172.21.15.112:0/2227090035 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2800bb20 con 0x7f4e3006d5b0 2023-08-27T11:17:46.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.967+0000 7f4e34f35700 1 -- 172.21.15.112:0/2227090035 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28012430 con 0x7f4e3006d5b0 2023-08-27T11:17:46.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.967+0000 7f4e3819b700 1 -- 172.21.15.112:0/2227090035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e3006d5b0 msgr2=0x7f4e3006da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.967+0000 7f4e3819b700 1 --2- 172.21.15.112:0/2227090035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e3006d5b0 0x7f4e3006da40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009d90 tx=0x7f4e2800b330 comp rx=0 tx=0).stop 2023-08-27T11:17:46.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.967+0000 7f4e3819b700 1 -- 172.21.15.112:0/2227090035 shutdown_connections 2023-08-27T11:17:46.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.967+0000 7f4e3819b700 1 --2- 172.21.15.112:0/2227090035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e3006d5b0 0x7f4e3006da40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.967+0000 7f4e3819b700 1 --2- 172.21.15.112:0/2227090035 >> v2:172.21.15.138:3300/0 conn(0x7f4e301437c0 0x7f4e300b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.967+0000 7f4e3819b700 1 -- 172.21.15.112:0/2227090035 >> 172.21.15.112:0/2227090035 conn(0x7f4e300b1e90 msgr2=0x7f4e300b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:46.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.967+0000 7f4e3819b700 1 -- 172.21.15.112:0/2227090035 shutdown_connections 2023-08-27T11:17:46.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.968+0000 7f4e3819b700 1 -- 172.21.15.112:0/2227090035 wait complete. 2023-08-27T11:17:46.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.969+0000 7f4e3819b700 1 Processor -- start 2023-08-27T11:17:46.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.969+0000 7f4e3819b700 1 -- start start 2023-08-27T11:17:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.970+0000 7f4e3819b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e3006d5b0 0x7f4e30155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.970+0000 7f4e3819b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e301437c0 0x7f4e3015dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.970+0000 7f4e3819b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e30143ba0 0x7f4e30158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.970+0000 7f4e3819b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e30157e30 con 0x7f4e30143ba0 2023-08-27T11:17:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.970+0000 7f4e3819b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e30157870 con 0x7f4e3006d5b0 2023-08-27T11:17:46.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.970+0000 7f4e3819b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e30157b50 con 0x7f4e301437c0 2023-08-27T11:17:46.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.970+0000 7f4e36738700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e30143ba0 0x7f4e30158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.970+0000 7f4e36738700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e30143ba0 0x7f4e30158b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56224/0 (socket says 172.21.15.112:56224) 2023-08-27T11:17:46.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.970+0000 7f4e36738700 1 -- 172.21.15.112:0/4122934314 learned_addr learned my addr 172.21.15.112:0/4122934314 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:46.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.970+0000 7f4e35f37700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e3006d5b0 0x7f4e30155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.970+0000 7f4e35736700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e301437c0 0x7f4e3015dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.970+0000 7f4e36738700 1 -- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e301437c0 msgr2=0x7f4e3015dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.971+0000 7f4e36738700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e301437c0 0x7f4e3015dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.971+0000 7f4e36738700 1 -- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e3006d5b0 msgr2=0x7f4e30155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:46.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.971+0000 7f4e36738700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e3006d5b0 0x7f4e30155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.971+0000 7f4e36738700 1 -- 172.21.15.112:0/4122934314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c0094a0 con 0x7f4e30143ba0 2023-08-27T11:17:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.971+0000 7f4e36738700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e30143ba0 0x7f4e30158b50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800b370 tx=0x7f4e28004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.971+0000 7f4e267fc700 1 -- 172.21.15.112:0/4122934314 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28012430 con 0x7f4e30143ba0 2023-08-27T11:17:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.971+0000 7f4e267fc700 1 -- 172.21.15.112:0/4122934314 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e28020070 con 0x7f4e30143ba0 2023-08-27T11:17:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.971+0000 7f4e3819b700 1 -- 172.21.15.112:0/4122934314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e28009a70 con 0x7f4e30143ba0 2023-08-27T11:17:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.971+0000 7f4e267fc700 1 -- 172.21.15.112:0/4122934314 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e280098f0 con 0x7f4e30143ba0 2023-08-27T11:17:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.971+0000 7f4e3819b700 1 -- 172.21.15.112:0/4122934314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e30158630 con 0x7f4e30143ba0 2023-08-27T11:17:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.972+0000 7f4e3819b700 1 -- 172.21.15.112:0/4122934314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e30067e90 con 0x7f4e30143ba0 2023-08-27T11:17:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.972+0000 7f4e267fc700 1 -- 172.21.15.112:0/4122934314 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4e28004040 con 0x7f4e30143ba0 2023-08-27T11:17:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.973+0000 7f4e267fc700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e1406a0e0 0x7f4e1406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:46.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.973+0000 7f4e267fc700 1 -- 172.21.15.112:0/4122934314 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e28022070 con 0x7f4e30143ba0 2023-08-27T11:17:46.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.973+0000 7f4e35f37700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e1406a0e0 0x7f4e1406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:46.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.973+0000 7f4e35f37700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e1406a0e0 0x7f4e1406c5a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009d70 tx=0x7f4e2c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:46.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:46.977+0000 7f4e267fc700 1 -- 172.21.15.112:0/4122934314 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4e280bc990 con 0x7f4e30143ba0 2023-08-27T11:17:47.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.164+0000 7f4e3819b700 1 -- 172.21.15.112:0/4122934314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4e30067a20 con 0x7f4e30143ba0 2023-08-27T11:17:47.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.164+0000 7f4e267fc700 1 -- 172.21.15.112:0/4122934314 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4e28055870 con 0x7f4e30143ba0 2023-08-27T11:17:47.166 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:47.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.166+0000 7f4e3819b700 1 -- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e1406a0e0 msgr2=0x7f4e1406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.166+0000 7f4e3819b700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e1406a0e0 0x7f4e1406c5a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009d70 tx=0x7f4e2c009210 comp rx=0 tx=0).stop 2023-08-27T11:17:47.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.166+0000 7f4e3819b700 1 -- 172.21.15.112:0/4122934314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e30143ba0 msgr2=0x7f4e30158b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.166+0000 7f4e3819b700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e30143ba0 0x7f4e30158b50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800b370 tx=0x7f4e28004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.166+0000 7f4e3819b700 1 -- 172.21.15.112:0/4122934314 shutdown_connections 2023-08-27T11:17:47.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.166+0000 7f4e3819b700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e1406a0e0 0x7f4e1406c5a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.166+0000 7f4e3819b700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e30143ba0 0x7f4e30158b50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.166+0000 7f4e3819b700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e3006d5b0 0x7f4e30155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.166+0000 7f4e3819b700 1 --2- 172.21.15.112:0/4122934314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e301437c0 0x7f4e3015dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.166+0000 7f4e3819b700 1 -- 172.21.15.112:0/4122934314 >> 172.21.15.112:0/4122934314 conn(0x7f4e300b1e90 msgr2=0x7f4e30071660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:47.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.166+0000 7f4e3819b700 1 -- 172.21.15.112:0/4122934314 shutdown_connections 2023-08-27T11:17:47.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.166+0000 7f4e3819b700 1 -- 172.21.15.112:0/4122934314 wait complete. 2023-08-27T11:17:47.180 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:47.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.301+0000 7f3941ad7700 1 Processor -- start 2023-08-27T11:17:47.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.301+0000 7f3941ad7700 1 -- start start 2023-08-27T11:17:47.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.302+0000 7f3941ad7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f393c06b0d0 0x7f393c06b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.302+0000 7f3941ad7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f393c0cd9e0 0x7f393c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.302+0000 7f3941ad7700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f393c0b19b0 con 0x7f393c0cf910 2023-08-27T11:17:47.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.302+0000 7f3941ad7700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f393c06bac0 con 0x7f393c06b0d0 2023-08-27T11:17:47.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.302+0000 7f3941ad7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393c143eb0 con 0x7f393c0cd9e0 2023-08-27T11:17:47.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.302+0000 7f393affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f393c06b0d0 0x7f393c06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.302+0000 7f393affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f393c06b0d0 0x7f393c06b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56230/0 (socket says 172.21.15.112:56230) 2023-08-27T11:17:47.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.302+0000 7f393affd700 1 -- 172.21.15.112:0/1879378078 learned_addr learned my addr 172.21.15.112:0/1879378078 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:47.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.303+0000 7f393bfff700 1 --2- 172.21.15.112:0/1879378078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f393c0cd9e0 0x7f393c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.303+0000 7f393affd700 1 -- 172.21.15.112:0/1879378078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f393c0cd9e0 msgr2=0x7f393c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:47.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.303+0000 7f393affd700 1 --2- 172.21.15.112:0/1879378078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f393c0cd9e0 0x7f393c0cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.303+0000 7f393affd700 1 -- 172.21.15.112:0/1879378078 >> v1:172.21.15.138:6789/0 conn(0x7f393c0cf910 legacy=0x7f393c1437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.303+0000 7f393affd700 1 -- 172.21.15.112:0/1879378078 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c0b0da0 con 0x7f393c06b0d0 2023-08-27T11:17:47.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.303+0000 7f393affd700 1 --2- 172.21.15.112:0/1879378078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c06b0d0 0x7f393c06b580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f392c0098f0 tx=0x7f392c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75be8bf20ab30af9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:47.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.303+0000 7f393a7fc700 1 -- 172.21.15.112:0/1879378078 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c00b8a0 con 0x7f393c06b0d0 2023-08-27T11:17:47.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.303+0000 7f393a7fc700 1 -- 172.21.15.112:0/1879378078 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392c00ba00 con 0x7f393c06b0d0 2023-08-27T11:17:47.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.303+0000 7f393a7fc700 1 -- 172.21.15.112:0/1879378078 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c00bd90 con 0x7f393c06b0d0 2023-08-27T11:17:47.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.303+0000 7f3941ad7700 1 -- 172.21.15.112:0/1879378078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c06b0d0 msgr2=0x7f393c06b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.303+0000 7f3941ad7700 1 --2- 172.21.15.112:0/1879378078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c06b0d0 0x7f393c06b580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f392c0098f0 tx=0x7f392c009c00 comp rx=0 tx=0).stop 2023-08-27T11:17:47.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.304+0000 7f3941ad7700 1 -- 172.21.15.112:0/1879378078 shutdown_connections 2023-08-27T11:17:47.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.304+0000 7f3941ad7700 1 --2- 172.21.15.112:0/1879378078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f393c0cd9e0 0x7f393c0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.304+0000 7f3941ad7700 1 --2- 172.21.15.112:0/1879378078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c06b0d0 0x7f393c06b580 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.304+0000 7f3941ad7700 1 -- 172.21.15.112:0/1879378078 >> 172.21.15.112:0/1879378078 conn(0x7f393c0d01b0 msgr2=0x7f393c0d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:47.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.304+0000 7f3941ad7700 1 -- 172.21.15.112:0/1879378078 shutdown_connections 2023-08-27T11:17:47.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.304+0000 7f3941ad7700 1 -- 172.21.15.112:0/1879378078 wait complete. 2023-08-27T11:17:47.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.305+0000 7f3941ad7700 1 Processor -- start 2023-08-27T11:17:47.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.305+0000 7f3941ad7700 1 -- start start 2023-08-27T11:17:47.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f3941ad7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c06b0d0 0x7f393c0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f3941ad7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f393c0cd9e0 0x7f393c159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f3941ad7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393c0cf910 0x7f393c15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f3941ad7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c15a020 con 0x7f393c06b0d0 2023-08-27T11:17:47.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f3941ad7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c15d6b0 con 0x7f393c0cf910 2023-08-27T11:17:47.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f3941ad7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393c0ae250 con 0x7f393c0cd9e0 2023-08-27T11:17:47.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f393b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c06b0d0 0x7f393c0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f393b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c06b0d0 0x7f393c0ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56234/0 (socket says 172.21.15.112:56234) 2023-08-27T11:17:47.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f393b7fe700 1 -- 172.21.15.112:0/1331097040 learned_addr learned my addr 172.21.15.112:0/1331097040 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:47.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f393affd700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f393c0cd9e0 0x7f393c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f393bfff700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393c0cf910 0x7f393c15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f393b7fe700 1 -- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f393c0cd9e0 msgr2=0x7f393c159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f393b7fe700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f393c0cd9e0 0x7f393c159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f393b7fe700 1 -- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393c0cf910 msgr2=0x7f393c15a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f393b7fe700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393c0cf910 0x7f393c15a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f393b7fe700 1 -- 172.21.15.112:0/1331097040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c009550 con 0x7f393c06b0d0 2023-08-27T11:17:47.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f393b7fe700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c06b0d0 0x7f393c0ae6e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f393000c480 tx=0x7f393000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:47.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.306+0000 7f3923fff700 1 -- 172.21.15.112:0/1331097040 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39300052d0 con 0x7f393c06b0d0 2023-08-27T11:17:47.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.307+0000 7f3923fff700 1 -- 172.21.15.112:0/1331097040 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3930005430 con 0x7f393c06b0d0 2023-08-27T11:17:47.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.307+0000 7f3941ad7700 1 -- 172.21.15.112:0/1331097040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393c0aec20 con 0x7f393c06b0d0 2023-08-27T11:17:47.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.307+0000 7f3923fff700 1 -- 172.21.15.112:0/1331097040 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39300038c0 con 0x7f393c06b0d0 2023-08-27T11:17:47.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.307+0000 7f3941ad7700 1 -- 172.21.15.112:0/1331097040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393c159d40 con 0x7f393c06b0d0 2023-08-27T11:17:47.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.307+0000 7f3941ad7700 1 -- 172.21.15.112:0/1331097040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393c0ae530 con 0x7f393c06b0d0 2023-08-27T11:17:47.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.308+0000 7f3923fff700 1 -- 172.21.15.112:0/1331097040 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3930003a20 con 0x7f393c06b0d0 2023-08-27T11:17:47.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.308+0000 7f3923fff700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f392806a0e0 0x7f392806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.308+0000 7f3923fff700 1 -- 172.21.15.112:0/1331097040 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3930087880 con 0x7f393c06b0d0 2023-08-27T11:17:47.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.308+0000 7f393affd700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f392806a0e0 0x7f392806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.309+0000 7f393affd700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f392806a0e0 0x7f392806c5a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f392c009c40 tx=0x7f392c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:47.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.313+0000 7f3923fff700 1 -- 172.21.15.112:0/1331097040 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3930052f60 con 0x7f393c06b0d0 2023-08-27T11:17:47.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.503+0000 7f3941ad7700 1 -- 172.21.15.112:0/1331097040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f393c067a20 con 0x7f393c06b0d0 2023-08-27T11:17:47.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.503+0000 7f3923fff700 1 -- 172.21.15.112:0/1331097040 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3930056580 con 0x7f393c06b0d0 2023-08-27T11:17:47.505 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:47.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.505+0000 7f3941ad7700 1 -- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f392806a0e0 msgr2=0x7f392806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.505+0000 7f3941ad7700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f392806a0e0 0x7f392806c5a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f392c009c40 tx=0x7f392c008030 comp rx=0 tx=0).stop 2023-08-27T11:17:47.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.505+0000 7f3941ad7700 1 -- 172.21.15.112:0/1331097040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c06b0d0 msgr2=0x7f393c0ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.505+0000 7f3941ad7700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c06b0d0 0x7f393c0ae6e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f393000c480 tx=0x7f393000c790 comp rx=0 tx=0).stop 2023-08-27T11:17:47.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.505+0000 7f3941ad7700 1 -- 172.21.15.112:0/1331097040 shutdown_connections 2023-08-27T11:17:47.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.505+0000 7f3941ad7700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f392806a0e0 0x7f392806c5a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.505+0000 7f3941ad7700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393c0cf910 0x7f393c15a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.505+0000 7f3941ad7700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c06b0d0 0x7f393c0ae6e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.505+0000 7f3941ad7700 1 --2- 172.21.15.112:0/1331097040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f393c0cd9e0 0x7f393c159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.505+0000 7f3941ad7700 1 -- 172.21.15.112:0/1331097040 >> 172.21.15.112:0/1331097040 conn(0x7f393c0d01b0 msgr2=0x7f393c0d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:47.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.505+0000 7f3941ad7700 1 -- 172.21.15.112:0/1331097040 shutdown_connections 2023-08-27T11:17:47.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.505+0000 7f3941ad7700 1 -- 172.21.15.112:0/1331097040 wait complete. 2023-08-27T11:17:47.519 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:17:47.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.640+0000 7fe3746eb700 1 Processor -- start 2023-08-27T11:17:47.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.641+0000 7fe3746eb700 1 -- start start 2023-08-27T11:17:47.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.641+0000 7fe3746eb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c146c20 0x7fe36c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.641+0000 7fe3746eb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe36c0b3440 0x7fe36c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.641+0000 7fe3746eb700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c0b5070 con 0x7fe36c15e6e0 2023-08-27T11:17:47.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe3746eb700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c147000 con 0x7fe36c0b3440 2023-08-27T11:17:47.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe3746eb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1614a0 con 0x7fe36c146c20 2023-08-27T11:17:47.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe371c86700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe36c0b3440 0x7fe36c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe372c88700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe36c15e6e0 0x7fe36c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50628/0 (socket says 172.21.15.112:50628) 2023-08-27T11:17:47.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe371c86700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe36c0b3440 0x7fe36c15e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56244/0 (socket says 172.21.15.112:56244) 2023-08-27T11:17:47.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe372c88700 1 -- 172.21.15.112:0/3467163278 learned_addr learned my addr 172.21.15.112:0/3467163278 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:47.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe372487700 1 --2- 172.21.15.112:0/3467163278 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c146c20 0x7fe36c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe371c86700 1 -- 172.21.15.112:0/3467163278 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c146c20 msgr2=0x7fe36c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:47.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe371c86700 1 --2- 172.21.15.112:0/3467163278 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c146c20 0x7fe36c160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe371c86700 1 -- 172.21.15.112:0/3467163278 >> v1:172.21.15.112:6789/0 conn(0x7fe36c15e6e0 legacy=0x7fe36c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe371c86700 1 -- 172.21.15.112:0/3467163278 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c161740 con 0x7fe36c0b3440 2023-08-27T11:17:47.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe372487700 1 --2- 172.21.15.112:0/3467163278 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c146c20 0x7fe36c160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe371c86700 1 --2- 172.21.15.112:0/3467163278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe36c0b3440 0x7fe36c15e1a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe368009840 tx=0x7fe368009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f027eb3bdc7883 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe371485700 1 -- 172.21.15.112:0/3467163278 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368011830 con 0x7fe36c0b3440 2023-08-27T11:17:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe371485700 1 -- 172.21.15.112:0/3467163278 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe368011990 con 0x7fe36c0b3440 2023-08-27T11:17:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.642+0000 7fe371485700 1 -- 172.21.15.112:0/3467163278 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368011d20 con 0x7fe36c0b3440 2023-08-27T11:17:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.643+0000 7fe3746eb700 1 -- 172.21.15.112:0/3467163278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe36c0b3440 msgr2=0x7fe36c15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.643+0000 7fe3746eb700 1 --2- 172.21.15.112:0/3467163278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe36c0b3440 0x7fe36c15e1a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe368009840 tx=0x7fe368009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.643+0000 7fe3746eb700 1 -- 172.21.15.112:0/3467163278 shutdown_connections 2023-08-27T11:17:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.643+0000 7fe3746eb700 1 --2- 172.21.15.112:0/3467163278 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c146c20 0x7fe36c160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.643+0000 7fe3746eb700 1 --2- 172.21.15.112:0/3467163278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe36c0b3440 0x7fe36c15e1a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.643+0000 7fe3746eb700 1 -- 172.21.15.112:0/3467163278 >> 172.21.15.112:0/3467163278 conn(0x7fe36c0b1e90 msgr2=0x7fe36c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.643+0000 7fe3746eb700 1 -- 172.21.15.112:0/3467163278 shutdown_connections 2023-08-27T11:17:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.643+0000 7fe3746eb700 1 -- 172.21.15.112:0/3467163278 wait complete. 2023-08-27T11:17:47.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.644+0000 7fe3746eb700 1 Processor -- start 2023-08-27T11:17:47.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.644+0000 7fe3746eb700 1 -- start start 2023-08-27T11:17:47.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe3746eb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c0b3440 0x7fe36c0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe3746eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe36c146c20 0x7fe36c13f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe3746eb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c15e6e0 0x7fe36c0ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe3746eb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1f2640 con 0x7fe36c146c20 2023-08-27T11:17:47.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe3746eb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c15cf20 con 0x7fe36c15e6e0 2023-08-27T11:17:47.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe3746eb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c15f2e0 con 0x7fe36c0b3440 2023-08-27T11:17:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe371c86700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe36c146c20 0x7fe36c13f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe371c86700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe36c146c20 0x7fe36c13f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56250/0 (socket says 172.21.15.112:56250) 2023-08-27T11:17:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe371c86700 1 -- 172.21.15.112:0/3765733423 learned_addr learned my addr 172.21.15.112:0/3765733423 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe372487700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c0b3440 0x7fe36c0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe372c88700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c15e6e0 0x7fe36c0ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe371c86700 1 -- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c0b3440 msgr2=0x7fe36c0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe371c86700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c0b3440 0x7fe36c0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe371c86700 1 -- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c15e6e0 msgr2=0x7fe36c0ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe371c86700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c15e6e0 0x7fe36c0ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe371c86700 1 -- 172.21.15.112:0/3765733423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3680094a0 con 0x7fe36c146c20 2023-08-27T11:17:47.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe371c86700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe36c146c20 0x7fe36c13f660 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe3680090b0 tx=0x7fe368004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:47.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe362ffd700 1 -- 172.21.15.112:0/3765733423 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3680038a0 con 0x7fe36c146c20 2023-08-27T11:17:47.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe362ffd700 1 -- 172.21.15.112:0/3765733423 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36801f070 con 0x7fe36c146c20 2023-08-27T11:17:47.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.645+0000 7fe362ffd700 1 -- 172.21.15.112:0/3765733423 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368003c60 con 0x7fe36c146c20 2023-08-27T11:17:47.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.646+0000 7fe3746eb700 1 -- 172.21.15.112:0/3765733423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c13fba0 con 0x7fe36c146c20 2023-08-27T11:17:47.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.646+0000 7fe3746eb700 1 -- 172.21.15.112:0/3765733423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c15ec90 con 0x7fe36c146c20 2023-08-27T11:17:47.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.646+0000 7fe3746eb700 1 -- 172.21.15.112:0/3765733423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c15eaf0 con 0x7fe36c146c20 2023-08-27T11:17:47.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.647+0000 7fe362ffd700 1 -- 172.21.15.112:0/3765733423 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe36801a430 con 0x7fe36c146c20 2023-08-27T11:17:47.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.647+0000 7fe362ffd700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe35006a0e0 0x7fe35006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.648+0000 7fe362ffd700 1 -- 172.21.15.112:0/3765733423 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe368021070 con 0x7fe36c146c20 2023-08-27T11:17:47.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.648+0000 7fe372487700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe35006a0e0 0x7fe35006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.648+0000 7fe372487700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe35006a0e0 0x7fe35006c5a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe35c006fd0 tx=0x7fe35c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:47.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.652+0000 7fe362ffd700 1 -- 172.21.15.112:0/3765733423 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe36c15eaf0 con 0x7fe36c146c20 2023-08-27T11:17:47.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.839+0000 7fe3746eb700 1 -- 172.21.15.112:0/3765733423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe36c15eaf0 con 0x7fe36c146c20 2023-08-27T11:17:47.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.839+0000 7fe362ffd700 1 -- 172.21.15.112:0/3765733423 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe368015020 con 0x7fe36c146c20 2023-08-27T11:17:47.842 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:17:47.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.842+0000 7fe3746eb700 1 -- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe35006a0e0 msgr2=0x7fe35006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.842+0000 7fe3746eb700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe35006a0e0 0x7fe35006c5a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe35c006fd0 tx=0x7fe35c008040 comp rx=0 tx=0).stop 2023-08-27T11:17:47.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.842+0000 7fe3746eb700 1 -- 172.21.15.112:0/3765733423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe36c146c20 msgr2=0x7fe36c13f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.842+0000 7fe3746eb700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe36c146c20 0x7fe36c13f660 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe3680090b0 tx=0x7fe368004750 comp rx=0 tx=0).stop 2023-08-27T11:17:47.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.842+0000 7fe3746eb700 1 -- 172.21.15.112:0/3765733423 shutdown_connections 2023-08-27T11:17:47.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.842+0000 7fe3746eb700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe35006a0e0 0x7fe35006c5a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.842+0000 7fe3746eb700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c15e6e0 0x7fe36c0ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.842+0000 7fe3746eb700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c0b3440 0x7fe36c0722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.842+0000 7fe3746eb700 1 --2- 172.21.15.112:0/3765733423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe36c146c20 0x7fe36c13f660 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.842+0000 7fe3746eb700 1 -- 172.21.15.112:0/3765733423 >> 172.21.15.112:0/3765733423 conn(0x7fe36c0b1e90 msgr2=0x7fe36c06dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:47.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.842+0000 7fe3746eb700 1 -- 172.21.15.112:0/3765733423 shutdown_connections 2023-08-27T11:17:47.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.842+0000 7fe3746eb700 1 -- 172.21.15.112:0/3765733423 wait complete. 2023-08-27T11:17:47.855 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:17:47.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.979+0000 7fe0673cc700 1 Processor -- start 2023-08-27T11:17:47.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.979+0000 7fe0673cc700 1 -- start start 2023-08-27T11:17:47.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe0673cc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe060146bf0 0x7fe0600b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe0673cc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe060146fd0 0x7fe0600cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe0673cc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0600d0290 0x7fe060070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe0673cc700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0600d1500 con 0x7fe060146bf0 2023-08-27T11:17:47.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe0673cc700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0600d0710 con 0x7fe060146fd0 2023-08-27T11:17:47.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe0673cc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0600d09f0 con 0x7fe0600d0290 2023-08-27T11:17:47.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe065168700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe060146bf0 0x7fe0600b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe065168700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe060146bf0 0x7fe0600b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56256/0 (socket says 172.21.15.112:56256) 2023-08-27T11:17:47.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe065168700 1 -- 172.21.15.112:0/2727547228 learned_addr learned my addr 172.21.15.112:0/2727547228 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:47.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe064967700 1 --2- 172.21.15.112:0/2727547228 >> v2:172.21.15.138:3300/0 conn(0x7fe060146fd0 0x7fe0600cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe065969700 1 --2- 172.21.15.112:0/2727547228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0600d0290 0x7fe060070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe065168700 1 -- 172.21.15.112:0/2727547228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0600d0290 msgr2=0x7fe060070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:47.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe065168700 1 --2- 172.21.15.112:0/2727547228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0600d0290 0x7fe060070eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe065168700 1 -- 172.21.15.112:0/2727547228 >> v2:172.21.15.138:3300/0 conn(0x7fe060146fd0 msgr2=0x7fe0600cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:47.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe065168700 1 --2- 172.21.15.112:0/2727547228 >> v2:172.21.15.138:3300/0 conn(0x7fe060146fd0 0x7fe0600cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe065168700 1 -- 172.21.15.112:0/2727547228 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0600d0cd0 con 0x7fe060146bf0 2023-08-27T11:17:47.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe064967700 1 --2- 172.21.15.112:0/2727547228 >> v2:172.21.15.138:3300/0 conn(0x7fe060146fd0 0x7fe0600cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:47.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe065168700 1 --2- 172.21.15.112:0/2727547228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe060146bf0 0x7fe0600b5db0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe060071d70 tx=0x7fe05c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7704ac05cb99b08 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:47.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.980+0000 7fe057fff700 1 -- 172.21.15.112:0/2727547228 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00b870 con 0x7fe060146bf0 2023-08-27T11:17:47.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.981+0000 7fe057fff700 1 -- 172.21.15.112:0/2727547228 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c00b9d0 con 0x7fe060146bf0 2023-08-27T11:17:47.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.981+0000 7fe057fff700 1 -- 172.21.15.112:0/2727547228 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00bd60 con 0x7fe060146bf0 2023-08-27T11:17:47.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.981+0000 7fe0673cc700 1 -- 172.21.15.112:0/2727547228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe060146bf0 msgr2=0x7fe0600b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.981+0000 7fe0673cc700 1 --2- 172.21.15.112:0/2727547228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe060146bf0 0x7fe0600b5db0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe060071d70 tx=0x7fe05c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.981+0000 7fe0673cc700 1 -- 172.21.15.112:0/2727547228 shutdown_connections 2023-08-27T11:17:47.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.981+0000 7fe0673cc700 1 --2- 172.21.15.112:0/2727547228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0600d0290 0x7fe060070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.981+0000 7fe0673cc700 1 --2- 172.21.15.112:0/2727547228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe060146bf0 0x7fe0600b5db0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.981+0000 7fe0673cc700 1 --2- 172.21.15.112:0/2727547228 >> v2:172.21.15.138:3300/0 conn(0x7fe060146fd0 0x7fe0600cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.981+0000 7fe0673cc700 1 -- 172.21.15.112:0/2727547228 >> 172.21.15.112:0/2727547228 conn(0x7fe06006c450 msgr2=0x7fe06006f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:47.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.981+0000 7fe0673cc700 1 -- 172.21.15.112:0/2727547228 shutdown_connections 2023-08-27T11:17:47.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.981+0000 7fe0673cc700 1 -- 172.21.15.112:0/2727547228 wait complete. 2023-08-27T11:17:47.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.982+0000 7fe0673cc700 1 Processor -- start 2023-08-27T11:17:47.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.982+0000 7fe0673cc700 1 -- start start 2023-08-27T11:17:47.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe0673cc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0600d0290 0x7fe060159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe0673cc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe060146bf0 0x7fe06015e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe0673cc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe060146fd0 0x7fe06015e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe0673cc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0600b1060 con 0x7fe060146fd0 2023-08-27T11:17:47.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe0673cc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0600ad5c0 con 0x7fe0600d0290 2023-08-27T11:17:47.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe0673cc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0600ad8a0 con 0x7fe060146bf0 2023-08-27T11:17:47.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe065969700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe060146fd0 0x7fe06015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe065969700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe060146fd0 0x7fe06015e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56266/0 (socket says 172.21.15.112:56266) 2023-08-27T11:17:47.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe065969700 1 -- 172.21.15.112:0/1987334200 learned_addr learned my addr 172.21.15.112:0/1987334200 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:47.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe065168700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0600d0290 0x7fe060159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe064967700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe060146bf0 0x7fe06015e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe065969700 1 -- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe060146bf0 msgr2=0x7fe06015e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe065969700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe060146bf0 0x7fe06015e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe065969700 1 -- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0600d0290 msgr2=0x7fe060159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe065969700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0600d0290 0x7fe060159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.983+0000 7fe065969700 1 -- 172.21.15.112:0/1987334200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c009550 con 0x7fe060146fd0 2023-08-27T11:17:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.984+0000 7fe065969700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe060146fd0 0x7fe06015e750 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe05800ccf0 tx=0x7fe05800a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:47.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.984+0000 7fe055ffb700 1 -- 172.21.15.112:0/1987334200 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058003a20 con 0x7fe060146fd0 2023-08-27T11:17:47.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.984+0000 7fe055ffb700 1 -- 172.21.15.112:0/1987334200 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe058003b80 con 0x7fe060146fd0 2023-08-27T11:17:47.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.984+0000 7fe0673cc700 1 -- 172.21.15.112:0/1987334200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0600adb80 con 0x7fe060146fd0 2023-08-27T11:17:47.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.984+0000 7fe055ffb700 1 -- 172.21.15.112:0/1987334200 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058004020 con 0x7fe060146fd0 2023-08-27T11:17:47.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.984+0000 7fe0673cc700 1 -- 172.21.15.112:0/1987334200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0600ac890 con 0x7fe060146fd0 2023-08-27T11:17:47.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.985+0000 7fe0673cc700 1 -- 172.21.15.112:0/1987334200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe060067e90 con 0x7fe060146fd0 2023-08-27T11:17:47.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.985+0000 7fe055ffb700 1 -- 172.21.15.112:0/1987334200 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe058004180 con 0x7fe060146fd0 2023-08-27T11:17:47.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.985+0000 7fe055ffb700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe04406a0e0 0x7fe04406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:47.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.986+0000 7fe055ffb700 1 -- 172.21.15.112:0/1987334200 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe058012070 con 0x7fe060146fd0 2023-08-27T11:17:47.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.986+0000 7fe065168700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe04406a0e0 0x7fe04406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:47.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.986+0000 7fe065168700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe04406a0e0 0x7fe04406c5a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe05c000c00 tx=0x7fe05c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:47.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:47.990+0000 7fe055ffb700 1 -- 172.21.15.112:0/1987334200 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe0580543e0 con 0x7fe060146fd0 2023-08-27T11:17:48.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.176+0000 7fe0673cc700 1 -- 172.21.15.112:0/1987334200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe060067a20 con 0x7fe060146fd0 2023-08-27T11:17:48.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.177+0000 7fe055ffb700 1 -- 172.21.15.112:0/1987334200 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe058057a00 con 0x7fe060146fd0 2023-08-27T11:17:48.179 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:17:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.179+0000 7fe0673cc700 1 -- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe04406a0e0 msgr2=0x7fe04406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.179+0000 7fe0673cc700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe04406a0e0 0x7fe04406c5a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe05c000c00 tx=0x7fe05c008030 comp rx=0 tx=0).stop 2023-08-27T11:17:48.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.179+0000 7fe0673cc700 1 -- 172.21.15.112:0/1987334200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe060146fd0 msgr2=0x7fe06015e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:48.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.179+0000 7fe0673cc700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe060146fd0 0x7fe06015e750 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe05800ccf0 tx=0x7fe05800a570 comp rx=0 tx=0).stop 2023-08-27T11:17:48.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.179+0000 7fe0673cc700 1 -- 172.21.15.112:0/1987334200 shutdown_connections 2023-08-27T11:17:48.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.179+0000 7fe0673cc700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe04406a0e0 0x7fe04406c5a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.179+0000 7fe0673cc700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe060146fd0 0x7fe06015e750 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.179+0000 7fe0673cc700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0600d0290 0x7fe060159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.179+0000 7fe0673cc700 1 --2- 172.21.15.112:0/1987334200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe060146bf0 0x7fe06015e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.179+0000 7fe0673cc700 1 -- 172.21.15.112:0/1987334200 >> 172.21.15.112:0/1987334200 conn(0x7fe06006c450 msgr2=0x7fe06006fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:48.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.179+0000 7fe0673cc700 1 -- 172.21.15.112:0/1987334200 shutdown_connections 2023-08-27T11:17:48.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.179+0000 7fe0673cc700 1 -- 172.21.15.112:0/1987334200 wait complete. 2023-08-27T11:17:48.192 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:17:48.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.312+0000 7f99d40c9700 1 Processor -- start 2023-08-27T11:17:48.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.312+0000 7f99d40c9700 1 -- start start 2023-08-27T11:17:48.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d40c9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f99cc151060 0x7f99cc15e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d40c9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f99cc15e990 0x7f99cc06adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d40c9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99cc06b710 con 0x7f99cc06b2f0 2023-08-27T11:17:48.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d40c9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99cc0b02f0 con 0x7f99cc15e990 2023-08-27T11:17:48.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d40c9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99cc0b05d0 con 0x7f99cc151060 2023-08-27T11:17:48.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d1664700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f99cc15e990 0x7f99cc06adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d2666700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f99cc06b2f0 0x7f99cc06d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50656/0 (socket says 172.21.15.112:50656) 2023-08-27T11:17:48.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d2666700 1 -- 172.21.15.112:0/2985988678 learned_addr learned my addr 172.21.15.112:0/2985988678 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:48.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d1e65700 1 --2- 172.21.15.112:0/2985988678 >> v2:172.21.15.138:3300/0 conn(0x7f99cc151060 0x7f99cc15e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d1664700 1 -- 172.21.15.112:0/2985988678 >> v2:172.21.15.138:3300/0 conn(0x7f99cc151060 msgr2=0x7f99cc15e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:48.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d1664700 1 --2- 172.21.15.112:0/2985988678 >> v2:172.21.15.138:3300/0 conn(0x7f99cc151060 0x7f99cc15e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d1664700 1 -- 172.21.15.112:0/2985988678 >> v1:172.21.15.112:6789/0 conn(0x7f99cc06b2f0 legacy=0x7f99cc06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:48.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d0e63700 1 -- 172.21.15.112:0/2985988678 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240577198 0 0) 0x7f99cc06b710 con 0x7f99cc06b2f0 2023-08-27T11:17:48.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d1664700 1 -- 172.21.15.112:0/2985988678 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc0b08b0 con 0x7f99cc15e990 2023-08-27T11:17:48.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d1664700 1 --2- 172.21.15.112:0/2985988678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99cc15e990 0x7f99cc06adb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f99cc06e8c0 tx=0x7f99bc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd179397795aa391 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:48.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d0e63700 1 -- 172.21.15.112:0/2985988678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc011830 con 0x7f99cc15e990 2023-08-27T11:17:48.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d0e63700 1 -- 172.21.15.112:0/2985988678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99bc011990 con 0x7f99cc15e990 2023-08-27T11:17:48.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.313+0000 7f99d0e63700 1 -- 172.21.15.112:0/2985988678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc011d20 con 0x7f99cc15e990 2023-08-27T11:17:48.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.314+0000 7f99d40c9700 1 -- 172.21.15.112:0/2985988678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99cc15e990 msgr2=0x7f99cc06adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:48.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.314+0000 7f99d40c9700 1 --2- 172.21.15.112:0/2985988678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99cc15e990 0x7f99cc06adb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f99cc06e8c0 tx=0x7f99bc009b20 comp rx=0 tx=0).stop 2023-08-27T11:17:48.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.314+0000 7f99d40c9700 1 -- 172.21.15.112:0/2985988678 shutdown_connections 2023-08-27T11:17:48.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.314+0000 7f99d40c9700 1 --2- 172.21.15.112:0/2985988678 >> v2:172.21.15.138:3300/0 conn(0x7f99cc151060 0x7f99cc15e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.314+0000 7f99d40c9700 1 --2- 172.21.15.112:0/2985988678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99cc15e990 0x7f99cc06adb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.314+0000 7f99d40c9700 1 -- 172.21.15.112:0/2985988678 >> 172.21.15.112:0/2985988678 conn(0x7f99cc0d0000 msgr2=0x7f99cc0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:48.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.314+0000 7f99d40c9700 1 -- 172.21.15.112:0/2985988678 shutdown_connections 2023-08-27T11:17:48.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.314+0000 7f99d40c9700 1 -- 172.21.15.112:0/2985988678 wait complete. 2023-08-27T11:17:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.315+0000 7f99d40c9700 1 Processor -- start 2023-08-27T11:17:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.315+0000 7f99d40c9700 1 -- start start 2023-08-27T11:17:48.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d40c9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99cc06b2f0 0x7f99cc071f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d40c9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99cc151060 0x7f99cc14a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d40c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99cc15e990 0x7f99cc1f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d40c9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0ca290 con 0x7f99cc151060 2023-08-27T11:17:48.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d40c9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0c4100 con 0x7f99cc15e990 2023-08-27T11:17:48.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d40c9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc15d8e0 con 0x7f99cc06b2f0 2023-08-27T11:17:48.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d1664700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99cc151060 0x7f99cc14a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d1664700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99cc151060 0x7f99cc14a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56278/0 (socket says 172.21.15.112:56278) 2023-08-27T11:17:48.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d1664700 1 -- 172.21.15.112:0/2337514554 learned_addr learned my addr 172.21.15.112:0/2337514554 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:48.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d1664700 1 -- 172.21.15.112:0/2337514554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99cc06b2f0 msgr2=0x7f99cc071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:48.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d1e65700 1 --2- 172.21.15.112:0/2337514554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99cc06b2f0 0x7f99cc071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d1664700 1 --2- 172.21.15.112:0/2337514554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99cc06b2f0 0x7f99cc071f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d1664700 1 -- 172.21.15.112:0/2337514554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99cc15e990 msgr2=0x7f99cc1f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:48.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d1664700 1 --2- 172.21.15.112:0/2337514554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99cc15e990 0x7f99cc1f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.316+0000 7f99d1664700 1 -- 172.21.15.112:0/2337514554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc0094a0 con 0x7f99cc151060 2023-08-27T11:17:48.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.317+0000 7f99d1664700 1 --2- 172.21.15.112:0/2337514554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99cc151060 0x7f99cc14a3a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0090b0 tx=0x7f99bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:48.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.317+0000 7f99c27fc700 1 -- 172.21.15.112:0/2337514554 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0038a0 con 0x7f99cc151060 2023-08-27T11:17:48.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.317+0000 7f99c27fc700 1 -- 172.21.15.112:0/2337514554 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99bc01f070 con 0x7f99cc151060 2023-08-27T11:17:48.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.317+0000 7f99d40c9700 1 -- 172.21.15.112:0/2337514554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc0c9fb0 con 0x7f99cc151060 2023-08-27T11:17:48.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.317+0000 7f99c27fc700 1 -- 172.21.15.112:0/2337514554 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc003c20 con 0x7f99cc151060 2023-08-27T11:17:48.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.317+0000 7f99d40c9700 1 -- 172.21.15.112:0/2337514554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc0c45f0 con 0x7f99cc151060 2023-08-27T11:17:48.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.318+0000 7f99d40c9700 1 -- 172.21.15.112:0/2337514554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99cc067a20 con 0x7f99cc151060 2023-08-27T11:17:48.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.318+0000 7f99c27fc700 1 -- 172.21.15.112:0/2337514554 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f99bc01a430 con 0x7f99cc151060 2023-08-27T11:17:48.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.319+0000 7f99c27fc700 1 --2- 172.21.15.112:0/2337514554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f99b006a0e0 0x7f99b006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.319+0000 7f99c27fc700 1 -- 172.21.15.112:0/2337514554 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99bc021070 con 0x7f99cc151060 2023-08-27T11:17:48.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.319+0000 7f99d1e65700 1 --2- 172.21.15.112:0/2337514554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f99b006a0e0 0x7f99b006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.319+0000 7f99d1e65700 1 --2- 172.21.15.112:0/2337514554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f99b006a0e0 0x7f99b006c5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f99c8009950 tx=0x7f99c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:48.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.323+0000 7f99c27fc700 1 -- 172.21.15.112:0/2337514554 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f99bc0556d0 con 0x7f99cc151060 2023-08-27T11:17:48.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.529+0000 7f99d40c9700 1 -- 172.21.15.112:0/2337514554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f99cc0c4780 con 0x7f99cc151060 2023-08-27T11:17:48.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.529+0000 7f99c27fc700 1 -- 172.21.15.112:0/2337514554 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f99bc015020 con 0x7f99cc151060 2023-08-27T11:17:48.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.531+0000 7f99d40c9700 1 -- 172.21.15.112:0/2337514554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f99b006a0e0 msgr2=0x7f99b006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.531+0000 7f99d40c9700 1 --2- 172.21.15.112:0/2337514554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f99b006a0e0 0x7f99b006c5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f99c8009950 tx=0x7f99c8008040 comp rx=0 tx=0).stop 2023-08-27T11:17:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.531+0000 7f99d40c9700 1 -- 172.21.15.112:0/2337514554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99cc151060 msgr2=0x7f99cc14a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.531+0000 7f99d40c9700 1 --2- 172.21.15.112:0/2337514554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99cc151060 0x7f99cc14a3a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0090b0 tx=0x7f99bc004750 comp rx=0 tx=0).stop 2023-08-27T11:17:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.531+0000 7f99d40c9700 1 -- 172.21.15.112:0/2337514554 shutdown_connections 2023-08-27T11:17:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.531+0000 7f99d40c9700 1 --2- 172.21.15.112:0/2337514554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f99b006a0e0 0x7f99b006c5a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.531+0000 7f99d40c9700 1 --2- 172.21.15.112:0/2337514554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99cc15e990 0x7f99cc1f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.531+0000 7f99d40c9700 1 --2- 172.21.15.112:0/2337514554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99cc06b2f0 0x7f99cc071f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.531+0000 7f99d40c9700 1 --2- 172.21.15.112:0/2337514554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99cc151060 0x7f99cc14a3a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.531+0000 7f99d40c9700 1 -- 172.21.15.112:0/2337514554 >> 172.21.15.112:0/2337514554 conn(0x7f99cc0d0000 msgr2=0x7f99cc150690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.531+0000 7f99d40c9700 1 -- 172.21.15.112:0/2337514554 shutdown_connections 2023-08-27T11:17:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.531+0000 7f99d40c9700 1 -- 172.21.15.112:0/2337514554 wait complete. 2023-08-27T11:17:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:17:48.544 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-08-27T11:17:48.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:48.545 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-08-27T11:17:48.589 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:17:48.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:17:48.590 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-27T11:17:48.650 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:17:48.650 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:17:48.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.769+0000 7f35c1d9d700 1 Processor -- start 2023-08-27T11:17:48.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.770+0000 7f35c1d9d700 1 -- start start 2023-08-27T11:17:48.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.770+0000 7f35c1d9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35bc15d490 0x7f35bc15d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.770+0000 7f35c1d9d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f35bc0708b0 0x7f35bc15ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.770+0000 7f35c1d9d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35bc0ad580 con 0x7f35bc066450 2023-08-27T11:17:48.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.770+0000 7f35c1d9d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35bc0ad260 con 0x7f35bc0708b0 2023-08-27T11:17:48.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.770+0000 7f35c1d9d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc06f790 con 0x7f35bc15d490 2023-08-27T11:17:48.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.770+0000 7f35b2ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f35bc066450 0x7f35bc0701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50666/0 (socket says 172.21.15.112:50666) 2023-08-27T11:17:48.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.770+0000 7f35b2ffd700 1 -- 172.21.15.112:0/1171072111 learned_addr learned my addr 172.21.15.112:0/1171072111 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:48.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35bbfff700 1 --2- 172.21.15.112:0/1171072111 >> v2:172.21.15.138:3300/0 conn(0x7f35bc0708b0 0x7f35bc15ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35bb7fe700 1 --2- 172.21.15.112:0/1171072111 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35bc15d490 0x7f35bc15d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823403370 0 0) 0x7f35bc0ad580 con 0x7f35bc066450 2023-08-27T11:17:48.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35a80035b0 con 0x7f35bc066450 2023-08-27T11:17:48.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2874250254 0 0) 0x7f35a80035b0 con 0x7f35bc066450 2023-08-27T11:17:48.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35bc15d490 msgr2=0x7f35bc15d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:48.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 --2- 172.21.15.112:0/1171072111 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35bc15d490 0x7f35bc15d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 >> v2:172.21.15.138:3300/0 conn(0x7f35bc0708b0 msgr2=0x7f35bc15ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:48.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 --2- 172.21.15.112:0/1171072111 >> v2:172.21.15.138:3300/0 conn(0x7f35bc0708b0 0x7f35bc15ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc06fa70 con 0x7f35bc066450 2023-08-27T11:17:48.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f35a4001670 con 0x7f35bc066450 2023-08-27T11:17:48.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 >> v1:172.21.15.112:6789/0 conn(0x7f35bc066450 legacy=0x7f35bc0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:48.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 --2- 172.21.15.112:0/1171072111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35a8003a00 0x7f35a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 --2- 172.21.15.112:0/1171072111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35a8004850 0x7f35a8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 --2- 172.21.15.112:0/1171072111 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35bc15d490 0x7f35a8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0ad260 con 0x7f35a8003a00 2023-08-27T11:17:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc06f790 con 0x7f35a8004850 2023-08-27T11:17:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a800ba10 con 0x7f35bc15d490 2023-08-27T11:17:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35bb7fe700 1 --2- 172.21.15.112:0/1171072111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35a8003a00 0x7f35a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35b2ffd700 1 --2- 172.21.15.112:0/1171072111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35a8004850 0x7f35a8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35bbfff700 1 --2- 172.21.15.112:0/1171072111 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35bc15d490 0x7f35a8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35bb7fe700 1 -- 172.21.15.112:0/1171072111 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35bc15d490 msgr2=0x7f35a8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35bb7fe700 1 --2- 172.21.15.112:0/1171072111 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35bc15d490 0x7f35a8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35bb7fe700 1 -- 172.21.15.112:0/1171072111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35a8004850 msgr2=0x7f35a8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35bb7fe700 1 --2- 172.21.15.112:0/1171072111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35a8004850 0x7f35a8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35bb7fe700 1 -- 172.21.15.112:0/1171072111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a80035b0 con 0x7f35a8003a00 2023-08-27T11:17:48.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.771+0000 7f35bbfff700 1 --2- 172.21.15.112:0/1171072111 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35bc15d490 0x7f35a8005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:48.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.772+0000 7f35bb7fe700 1 --2- 172.21.15.112:0/1171072111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35a8003a00 0x7f35a8003e20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009b90 tx=0x7f35a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de17d0c462d547b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:48.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.772+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac00e6c0 con 0x7f35a8003a00 2023-08-27T11:17:48.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.772+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ac00e820 con 0x7f35a8003a00 2023-08-27T11:17:48.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.772+0000 7f35baffd700 1 -- 172.21.15.112:0/1171072111 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac00eb90 con 0x7f35a8003a00 2023-08-27T11:17:48.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.772+0000 7f35c1d9d700 1 -- 172.21.15.112:0/1171072111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35a8003a00 msgr2=0x7f35a8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:48.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.772+0000 7f35c1d9d700 1 --2- 172.21.15.112:0/1171072111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35a8003a00 0x7f35a8003e20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009b90 tx=0x7f35a8005880 comp rx=0 tx=0).stop 2023-08-27T11:17:48.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.772+0000 7f35bbfff700 1 -- 172.21.15.112:0/1171072111 reap_dead start 2023-08-27T11:17:48.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.772+0000 7f35c1d9d700 1 -- 172.21.15.112:0/1171072111 shutdown_connections 2023-08-27T11:17:48.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.772+0000 7f35c1d9d700 1 -- 172.21.15.112:0/1171072111 >> 172.21.15.112:0/1171072111 conn(0x7f35bc0b1380 msgr2=0x7f35bc0b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:48.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.772+0000 7f35c1d9d700 1 -- 172.21.15.112:0/1171072111 shutdown_connections 2023-08-27T11:17:48.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.772+0000 7f35c1d9d700 1 -- 172.21.15.112:0/1171072111 wait complete. 2023-08-27T11:17:48.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.774+0000 7f35c1d9d700 1 Processor -- start 2023-08-27T11:17:48.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.774+0000 7f35c1d9d700 1 -- start start 2023-08-27T11:17:48.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35c1d9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35a8003a00 0x7f35bc145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35c1d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35bc156680 0x7f35bc156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35c1d9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc159290 0x7f35bc159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35c1d9d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc157040 con 0x7f35bc156680 2023-08-27T11:17:48.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35c1d9d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc145b70 con 0x7f35bc159290 2023-08-27T11:17:48.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35c1d9d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1439a0 con 0x7f35a8003a00 2023-08-27T11:17:48.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35b2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35bc156680 0x7f35bc156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35b2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35bc156680 0x7f35bc156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56296/0 (socket says 172.21.15.112:56296) 2023-08-27T11:17:48.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35b2ffd700 1 -- 172.21.15.112:0/2554903160 learned_addr learned my addr 172.21.15.112:0/2554903160 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:48.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35bbfff700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc159290 0x7f35bc159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35bb7fe700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35a8003a00 0x7f35bc145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35b2ffd700 1 -- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35a8003a00 msgr2=0x7f35bc145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:48.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35b2ffd700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35a8003a00 0x7f35bc145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35b2ffd700 1 -- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc159290 msgr2=0x7f35bc159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:48.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35b2ffd700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc159290 0x7f35bc159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:48.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35b2ffd700 1 -- 172.21.15.112:0/2554903160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc159c80 con 0x7f35bc156680 2023-08-27T11:17:48.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.775+0000 7f35b2ffd700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35bc156680 0x7f35bc156b00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f35a4007310 tx=0x7f35a4007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:48.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.776+0000 7f35b8ff9700 1 -- 172.21.15.112:0/2554903160 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4007800 con 0x7f35bc156680 2023-08-27T11:17:48.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.776+0000 7f35b8ff9700 1 -- 172.21.15.112:0/2554903160 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a400f8d0 con 0x7f35bc156680 2023-08-27T11:17:48.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.776+0000 7f35b8ff9700 1 -- 172.21.15.112:0/2554903160 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a400fc80 con 0x7f35bc156680 2023-08-27T11:17:48.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.776+0000 7f35c1d9d700 1 -- 172.21.15.112:0/2554903160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc15b3f0 con 0x7f35bc156680 2023-08-27T11:17:48.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.776+0000 7f35c1d9d700 1 -- 172.21.15.112:0/2554903160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc15b9b0 con 0x7f35bc156680 2023-08-27T11:17:48.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.777+0000 7f35c1d9d700 1 -- 172.21.15.112:0/2554903160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc067a20 con 0x7f35bc156680 2023-08-27T11:17:48.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.777+0000 7f35b8ff9700 1 -- 172.21.15.112:0/2554903160 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f35a401d460 con 0x7f35bc156680 2023-08-27T11:17:48.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.777+0000 7f35b8ff9700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f359c06a110 0x7f359c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:48.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.777+0000 7f35b8ff9700 1 -- 172.21.15.112:0/2554903160 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f35a4088680 con 0x7f35bc156680 2023-08-27T11:17:48.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.778+0000 7f35bb7fe700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f359c06a110 0x7f359c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:48.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.778+0000 7f35bb7fe700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f359c06a110 0x7f359c06c5d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0090b0 tx=0x7f35ac008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:48.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:48.782+0000 7f35b8ff9700 1 -- 172.21.15.112:0/2554903160 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f35a4053d60 con 0x7f35bc156680 2023-08-27T11:17:49.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.012+0000 7f35c1d9d700 1 -- 172.21.15.112:0/2554903160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35bc156440 con 0x7f35bc156680 2023-08-27T11:17:49.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.013+0000 7f35b8ff9700 1 -- 172.21.15.112:0/2554903160 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4881 (secure 0 0 0) 0x7f35a4057380 con 0x7f35bc156680 2023-08-27T11:17:49.015 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:49.015 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3289488908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3289488908},{"type":"v1","addr":"172.21.15.112:6835","nonce":3289488908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3991653721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3991653721},{"type":"v1","addr":"172.21.15.112:6834","nonce":3991653721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1923303457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1923303457},{"type":"v1","addr":"172.21.15.138:6836","nonce":1923303457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:17:04.761658+0000","modified":"2023-08-27T11:17:08.372245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/1827312637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1827312637},{"type":"v1","addr":"172.21.15.138:6837","nonce":1827312637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:17:49.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.015+0000 7f35c1d9d700 1 -- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f359c06a110 msgr2=0x7f359c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.015+0000 7f35c1d9d700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f359c06a110 0x7f359c06c5d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0090b0 tx=0x7f35ac008030 comp rx=0 tx=0).stop 2023-08-27T11:17:49.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.015+0000 7f35c1d9d700 1 -- 172.21.15.112:0/2554903160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35bc156680 msgr2=0x7f35bc156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.015+0000 7f35c1d9d700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35bc156680 0x7f35bc156b00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f35a4007310 tx=0x7f35a4007620 comp rx=0 tx=0).stop 2023-08-27T11:17:49.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.015+0000 7f35c1d9d700 1 -- 172.21.15.112:0/2554903160 shutdown_connections 2023-08-27T11:17:49.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.015+0000 7f35c1d9d700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f359c06a110 0x7f359c06c5d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.015+0000 7f35c1d9d700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc159290 0x7f35bc159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.015+0000 7f35c1d9d700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35a8003a00 0x7f35bc145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.015+0000 7f35c1d9d700 1 --2- 172.21.15.112:0/2554903160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35bc156680 0x7f35bc156b00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.015+0000 7f35c1d9d700 1 -- 172.21.15.112:0/2554903160 >> 172.21.15.112:0/2554903160 conn(0x7f35bc0b1380 msgr2=0x7f35bc15e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:49.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.015+0000 7f35c1d9d700 1 -- 172.21.15.112:0/2554903160 shutdown_connections 2023-08-27T11:17:49.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.015+0000 7f35c1d9d700 1 -- 172.21.15.112:0/2554903160 wait complete. 2023-08-27T11:17:49.019 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2023-08-27T11:17:49.028 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:49.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.147+0000 7f5e55b8e700 1 Processor -- start 2023-08-27T11:17:49.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.147+0000 7f5e55b8e700 1 -- start start 2023-08-27T11:17:49.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.147+0000 7f5e55b8e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e500b1400 0x7f5e500cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.147+0000 7f5e55b8e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5e50161ce0 0x7f5e5006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.147+0000 7f5e55b8e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e50161850 con 0x7f5e500b17e0 2023-08-27T11:17:49.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.147+0000 7f5e55b8e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e500cfc10 con 0x7f5e50161ce0 2023-08-27T11:17:49.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.147+0000 7f5e55b8e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50161200 con 0x7f5e500b1400 2023-08-27T11:17:49.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.148+0000 7f5e4ffff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5e50161ce0 0x7f5e5006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.148+0000 7f5e4f7fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e500b1400 0x7f5e500cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.148+0000 7f5e4ffff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5e50161ce0 0x7f5e5006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:56732/0 (socket says 172.21.15.112:56732) 2023-08-27T11:17:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.148+0000 7f5e4ffff700 1 -- 172.21.15.112:0/2758598938 learned_addr learned my addr 172.21.15.112:0/2758598938 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.148+0000 7f5e4ffff700 1 -- 172.21.15.112:0/2758598938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e500b1400 msgr2=0x7f5e500cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.148+0000 7f5e4ffff700 1 --2- 172.21.15.112:0/2758598938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e500b1400 0x7f5e500cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.148+0000 7f5e4ffff700 1 -- 172.21.15.112:0/2758598938 >> v1:172.21.15.138:6789/0 conn(0x7f5e500b17e0 legacy=0x7f5e5006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.148+0000 7f5e4ffff700 1 -- 172.21.15.112:0/2758598938 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e501614e0 con 0x7f5e50161ce0 2023-08-27T11:17:49.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.148+0000 7f5e4ffff700 1 --2- 172.21.15.112:0/2758598938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e50161ce0 0x7f5e5006f790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009e50 tx=0x7f5e4800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c76aa809b8d4cc49 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:49.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.148+0000 7f5e4e7fc700 1 -- 172.21.15.112:0/2758598938 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4800e070 con 0x7f5e50161ce0 2023-08-27T11:17:49.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.149+0000 7f5e4e7fc700 1 -- 172.21.15.112:0/2758598938 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4800b910 con 0x7f5e50161ce0 2023-08-27T11:17:49.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.149+0000 7f5e4e7fc700 1 -- 172.21.15.112:0/2758598938 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4800bcc0 con 0x7f5e50161ce0 2023-08-27T11:17:49.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.149+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2758598938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e50161ce0 msgr2=0x7f5e5006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.149+0000 7f5e55b8e700 1 --2- 172.21.15.112:0/2758598938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e50161ce0 0x7f5e5006f790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009e50 tx=0x7f5e4800b330 comp rx=0 tx=0).stop 2023-08-27T11:17:49.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.149+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2758598938 shutdown_connections 2023-08-27T11:17:49.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.149+0000 7f5e55b8e700 1 --2- 172.21.15.112:0/2758598938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e50161ce0 0x7f5e5006f790 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.149+0000 7f5e55b8e700 1 --2- 172.21.15.112:0/2758598938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e500b1400 0x7f5e500cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.149+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2758598938 >> 172.21.15.112:0/2758598938 conn(0x7f5e500d0000 msgr2=0x7f5e500d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:49.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.149+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2758598938 shutdown_connections 2023-08-27T11:17:49.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.149+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2758598938 wait complete. 2023-08-27T11:17:49.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.150+0000 7f5e55b8e700 1 Processor -- start 2023-08-27T11:17:49.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.150+0000 7f5e55b8e700 1 -- start start 2023-08-27T11:17:49.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.151+0000 7f5e55b8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e500b1400 0x7f5e500704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.151+0000 7f5e55b8e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e500b17e0 0x7f5e500c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.151+0000 7f5e55b8e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e50161ce0 0x7f5e50144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.151+0000 7f5e55b8e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50070d10 con 0x7f5e500b1400 2023-08-27T11:17:49.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.151+0000 7f5e55b8e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50070ff0 con 0x7f5e500b17e0 2023-08-27T11:17:49.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.151+0000 7f5e55b8e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e500712d0 con 0x7f5e50161ce0 2023-08-27T11:17:49.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.151+0000 7f5e4f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e500b1400 0x7f5e500704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.151+0000 7f5e4f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e500b1400 0x7f5e500704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56308/0 (socket says 172.21.15.112:56308) 2023-08-27T11:17:49.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.151+0000 7f5e4f7fe700 1 -- 172.21.15.112:0/2285493797 learned_addr learned my addr 172.21.15.112:0/2285493797 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:49.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.151+0000 7f5e4effd700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e500b17e0 0x7f5e500c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.151+0000 7f5e4ffff700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e50161ce0 0x7f5e50144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.152+0000 7f5e4f7fe700 1 -- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e50161ce0 msgr2=0x7f5e50144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.152+0000 7f5e4f7fe700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e50161ce0 0x7f5e50144550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.152+0000 7f5e4f7fe700 1 -- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e500b17e0 msgr2=0x7f5e500c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.152+0000 7f5e4f7fe700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e500b17e0 0x7f5e500c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.152+0000 7f5e4f7fe700 1 -- 172.21.15.112:0/2285493797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e48009b30 con 0x7f5e500b1400 2023-08-27T11:17:49.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.152+0000 7f5e4effd700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e500b17e0 0x7f5e500c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:49.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.152+0000 7f5e4f7fe700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e500b1400 0x7f5e500704d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e9b0 tx=0x7f5e4000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:49.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.152+0000 7f5e47fff700 1 -- 172.21.15.112:0/2285493797 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40009710 con 0x7f5e500b1400 2023-08-27T11:17:49.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.152+0000 7f5e47fff700 1 -- 172.21.15.112:0/2285493797 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4000c850 con 0x7f5e500b1400 2023-08-27T11:17:49.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.152+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2285493797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e500715b0 con 0x7f5e500b1400 2023-08-27T11:17:49.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.152+0000 7f5e47fff700 1 -- 172.21.15.112:0/2285493797 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000cbc0 con 0x7f5e500b1400 2023-08-27T11:17:49.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.152+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2285493797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e500c9e20 con 0x7f5e500b1400 2023-08-27T11:17:49.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.153+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2285493797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e50071740 con 0x7f5e500b1400 2023-08-27T11:17:49.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.153+0000 7f5e47fff700 1 -- 172.21.15.112:0/2285493797 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5e4000cd20 con 0x7f5e500b1400 2023-08-27T11:17:49.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.153+0000 7f5e47fff700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5e3006a0e0 0x7f5e3006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.154+0000 7f5e47fff700 1 -- 172.21.15.112:0/2285493797 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5e400886d0 con 0x7f5e500b1400 2023-08-27T11:17:49.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.154+0000 7f5e4effd700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5e3006a0e0 0x7f5e3006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.154+0000 7f5e4effd700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5e3006a0e0 0x7f5e3006c5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5e38000f60 tx=0x7f5e38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:49.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.158+0000 7f5e47fff700 1 -- 172.21.15.112:0/2285493797 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5e40057850 con 0x7f5e500b1400 2023-08-27T11:17:49.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.348+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2285493797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e50070e50 con 0x7f5e500b1400 2023-08-27T11:17:49.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.349+0000 7f5e47fff700 1 -- 172.21.15.112:0/2285493797 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7f5e50070e50 con 0x7f5e500b1400 2023-08-27T11:17:49.351 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:49.351 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:49.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.352+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5e3006a0e0 msgr2=0x7f5e3006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.352+0000 7f5e55b8e700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5e3006a0e0 0x7f5e3006c5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5e38000f60 tx=0x7f5e38009040 comp rx=0 tx=0).stop 2023-08-27T11:17:49.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.352+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2285493797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e500b1400 msgr2=0x7f5e500704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.352+0000 7f5e55b8e700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e500b1400 0x7f5e500704d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e9b0 tx=0x7f5e4000ed70 comp rx=0 tx=0).stop 2023-08-27T11:17:49.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.352+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2285493797 shutdown_connections 2023-08-27T11:17:49.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.352+0000 7f5e55b8e700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5e3006a0e0 0x7f5e3006c5a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.352+0000 7f5e55b8e700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e50161ce0 0x7f5e50144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.352+0000 7f5e55b8e700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e500b1400 0x7f5e500704d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.352+0000 7f5e55b8e700 1 --2- 172.21.15.112:0/2285493797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e500b17e0 0x7f5e500c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.352+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2285493797 >> 172.21.15.112:0/2285493797 conn(0x7f5e500d0000 msgr2=0x7f5e501578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:49.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.352+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2285493797 shutdown_connections 2023-08-27T11:17:49.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.352+0000 7f5e55b8e700 1 -- 172.21.15.112:0/2285493797 wait complete. 2023-08-27T11:17:49.365 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:17:02.484063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:17:03.722976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:17:49.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:17:49.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.487+0000 7fd4f9474700 1 Processor -- start 2023-08-27T11:17:49.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.488+0000 7fd4f9474700 1 -- start start 2023-08-27T11:17:49.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.488+0000 7fd4f9474700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd4f415b160 0x7fd4f40ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.488+0000 7fd4f9474700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f40cdb10 con 0x7fd4f406db60 2023-08-27T11:17:49.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.488+0000 7fd4f9474700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f406c610 con 0x7fd4f40cd080 2023-08-27T11:17:49.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.488+0000 7fd4f9474700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f406c9b0 con 0x7fd4f415b160 2023-08-27T11:17:49.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.488+0000 7fd4f37fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd4f406db60 0x7fd4f406cdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50700/0 (socket says 172.21.15.112:50700) 2023-08-27T11:17:49.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.488+0000 7fd4f37fe700 1 -- 172.21.15.112:0/4005080423 learned_addr learned my addr 172.21.15.112:0/4005080423 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:49.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123718616 0 0) 0x7fd4ec0014e0 con 0x7fd4f406db60 2023-08-27T11:17:49.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4dc0035b0 con 0x7fd4f406db60 2023-08-27T11:17:49.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f2ffd700 1 --2- 172.21.15.112:0/4005080423 >> v2:172.21.15.138:3300/0 conn(0x7fd4f415b160 0x7fd4f40ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1157391374 0 0) 0x7fd4dc0035b0 con 0x7fd4f406db60 2023-08-27T11:17:49.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 >> v2:172.21.15.138:3300/0 conn(0x7fd4f415b160 msgr2=0x7fd4f40ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:49.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 --2- 172.21.15.112:0/4005080423 >> v2:172.21.15.138:3300/0 conn(0x7fd4f415b160 0x7fd4f40ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 >> v1:172.21.15.138:6789/0 conn(0x7fd4f40cd080 legacy=0x7fd4f4070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f40cd550 con 0x7fd4f406db60 2023-08-27T11:17:49.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fd4ec001bd0 con 0x7fd4f406db60 2023-08-27T11:17:49.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 >> v1:172.21.15.112:6789/0 conn(0x7fd4f406db60 legacy=0x7fd4f406cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 --2- 172.21.15.112:0/4005080423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4dc003a00 0x7fd4dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 --2- 172.21.15.112:0/4005080423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc004850 0x7fd4dc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 --2- 172.21.15.112:0/4005080423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4dc005210 0x7fd4dc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f406c9b0 con 0x7fd4dc004850 2023-08-27T11:17:49.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc00c1b0 con 0x7fd4dc005210 2023-08-27T11:17:49.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc00c370 con 0x7fd4dc003a00 2023-08-27T11:17:49.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f27fc700 1 --2- 172.21.15.112:0/4005080423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc004850 0x7fd4dc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f37fe700 1 --2- 172.21.15.112:0/4005080423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4dc005210 0x7fd4dc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f2ffd700 1 --2- 172.21.15.112:0/4005080423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4dc003a00 0x7fd4dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f27fc700 1 -- 172.21.15.112:0/4005080423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4dc003a00 msgr2=0x7fd4dc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:49.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f27fc700 1 --2- 172.21.15.112:0/4005080423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4dc003a00 0x7fd4dc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f27fc700 1 -- 172.21.15.112:0/4005080423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4dc005210 msgr2=0x7fd4dc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:49.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f27fc700 1 --2- 172.21.15.112:0/4005080423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4dc005210 0x7fd4dc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f37fe700 1 -- 172.21.15.112:0/4005080423 reap_dead start 2023-08-27T11:17:49.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.489+0000 7fd4f27fc700 1 -- 172.21.15.112:0/4005080423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc0035b0 con 0x7fd4dc004850 2023-08-27T11:17:49.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.490+0000 7fd4f27fc700 1 --2- 172.21.15.112:0/4005080423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc004850 0x7fd4dc004cd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40cd480 tx=0x7fd4dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a63948935ccbff9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:49.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.490+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80052d0 con 0x7fd4dc004850 2023-08-27T11:17:49.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.490+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e8005450 con 0x7fd4dc004850 2023-08-27T11:17:49.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.490+0000 7fd4f1ffb700 1 -- 172.21.15.112:0/4005080423 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80038c0 con 0x7fd4dc004850 2023-08-27T11:17:49.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.490+0000 7fd4f9474700 1 -- 172.21.15.112:0/4005080423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc004850 msgr2=0x7fd4dc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.490+0000 7fd4f9474700 1 --2- 172.21.15.112:0/4005080423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc004850 0x7fd4dc004cd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40cd480 tx=0x7fd4dc005880 comp rx=0 tx=0).stop 2023-08-27T11:17:49.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.490+0000 7fd4f9474700 1 -- 172.21.15.112:0/4005080423 shutdown_connections 2023-08-27T11:17:49.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.490+0000 7fd4f9474700 1 --2- 172.21.15.112:0/4005080423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc004850 0x7fd4dc004cd0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.490+0000 7fd4f9474700 1 -- 172.21.15.112:0/4005080423 >> 172.21.15.112:0/4005080423 conn(0x7fd4f40cef70 msgr2=0x7fd4f40cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:49.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.490+0000 7fd4f9474700 1 -- 172.21.15.112:0/4005080423 shutdown_connections 2023-08-27T11:17:49.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.490+0000 7fd4f9474700 1 -- 172.21.15.112:0/4005080423 wait complete. 2023-08-27T11:17:49.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.492+0000 7fd4f9474700 1 Processor -- start 2023-08-27T11:17:49.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.492+0000 7fd4f9474700 1 -- start start 2023-08-27T11:17:49.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.492+0000 7fd4f9474700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc004850 0x7fd4f4154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.492+0000 7fd4f9474700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f41f0800 0x7fd4f40ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.492+0000 7fd4f9474700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4f41f0c80 0x7fd4f40adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.492+0000 7fd4f9474700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f4155240 con 0x7fd4dc004850 2023-08-27T11:17:49.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.492+0000 7fd4f9474700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40ae230 con 0x7fd4f41f0800 2023-08-27T11:17:49.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.492+0000 7fd4f9474700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40a8eb0 con 0x7fd4f41f0c80 2023-08-27T11:17:49.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.492+0000 7fd4f2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc004850 0x7fd4f4154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4f2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc004850 0x7fd4f4154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56328/0 (socket says 172.21.15.112:56328) 2023-08-27T11:17:49.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4f2ffd700 1 -- 172.21.15.112:0/4060238693 learned_addr learned my addr 172.21.15.112:0/4060238693 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:49.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4f2ffd700 1 -- 172.21.15.112:0/4060238693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4f41f0c80 msgr2=0x7fd4f40adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4f2ffd700 1 --2- 172.21.15.112:0/4060238693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4f41f0c80 0x7fd4f40adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4f2ffd700 1 -- 172.21.15.112:0/4060238693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f41f0800 msgr2=0x7fd4f40ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4f27fc700 1 --2- 172.21.15.112:0/4060238693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f41f0800 0x7fd4f40ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4f2ffd700 1 --2- 172.21.15.112:0/4060238693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f41f0800 0x7fd4f40ad7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4f2ffd700 1 -- 172.21.15.112:0/4060238693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f40a9190 con 0x7fd4dc004850 2023-08-27T11:17:49.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4f2ffd700 1 --2- 172.21.15.112:0/4060238693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc004850 0x7fd4f4154d00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4011a20 tx=0x7fd4e4011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:49.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4db7fe700 1 -- 172.21.15.112:0/4060238693 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400fa00 con 0x7fd4dc004850 2023-08-27T11:17:49.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4db7fe700 1 -- 172.21.15.112:0/4060238693 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e400fb60 con 0x7fd4dc004850 2023-08-27T11:17:49.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4db7fe700 1 -- 172.21.15.112:0/4060238693 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4014440 con 0x7fd4dc004850 2023-08-27T11:17:49.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4f9474700 1 -- 172.21.15.112:0/4060238693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f40a9470 con 0x7fd4dc004850 2023-08-27T11:17:49.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.493+0000 7fd4f9474700 1 -- 172.21.15.112:0/4060238693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f40a9a30 con 0x7fd4dc004850 2023-08-27T11:17:49.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.494+0000 7fd4f9474700 1 -- 172.21.15.112:0/4060238693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f4067a20 con 0x7fd4dc004850 2023-08-27T11:17:49.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.495+0000 7fd4db7fe700 1 -- 172.21.15.112:0/4060238693 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd4e40145a0 con 0x7fd4dc004850 2023-08-27T11:17:49.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.495+0000 7fd4db7fe700 1 --2- 172.21.15.112:0/4060238693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd4e006a0e0 0x7fd4e006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.495+0000 7fd4db7fe700 1 -- 172.21.15.112:0/4060238693 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd4e4088d40 con 0x7fd4dc004850 2023-08-27T11:17:49.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.496+0000 7fd4f27fc700 1 --2- 172.21.15.112:0/4060238693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd4e006a0e0 0x7fd4e006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.496+0000 7fd4f27fc700 1 --2- 172.21.15.112:0/4060238693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd4e006a0e0 0x7fd4e006c5a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40c9500 tx=0x7fd4e8009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:49.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.500+0000 7fd4db7fe700 1 -- 172.21.15.112:0/4060238693 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd4e40544e0 con 0x7fd4dc004850 2023-08-27T11:17:49.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.686+0000 7fd4f9474700 1 -- 172.21.15.112:0/4060238693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd4f415c390 con 0x7fd4dc004850 2023-08-27T11:17:49.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.686+0000 7fd4db7fe700 1 -- 172.21.15.112:0/4060238693 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd4e4022020 con 0x7fd4dc004850 2023-08-27T11:17:49.688 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:17:49.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.688+0000 7fd4f9474700 1 -- 172.21.15.112:0/4060238693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd4e006a0e0 msgr2=0x7fd4e006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.688+0000 7fd4f9474700 1 --2- 172.21.15.112:0/4060238693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd4e006a0e0 0x7fd4e006c5a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40c9500 tx=0x7fd4e8009030 comp rx=0 tx=0).stop 2023-08-27T11:17:49.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.688+0000 7fd4f9474700 1 -- 172.21.15.112:0/4060238693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc004850 msgr2=0x7fd4f4154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.688+0000 7fd4f9474700 1 --2- 172.21.15.112:0/4060238693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc004850 0x7fd4f4154d00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4011a20 tx=0x7fd4e4011de0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.689+0000 7fd4f9474700 1 -- 172.21.15.112:0/4060238693 shutdown_connections 2023-08-27T11:17:49.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.689+0000 7fd4f9474700 1 --2- 172.21.15.112:0/4060238693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd4e006a0e0 0x7fd4e006c5a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.689+0000 7fd4f9474700 1 --2- 172.21.15.112:0/4060238693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4f41f0c80 0x7fd4f40adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.689+0000 7fd4f9474700 1 --2- 172.21.15.112:0/4060238693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4dc004850 0x7fd4f4154d00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.689+0000 7fd4f9474700 1 --2- 172.21.15.112:0/4060238693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f41f0800 0x7fd4f40ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.689+0000 7fd4f9474700 1 -- 172.21.15.112:0/4060238693 >> 172.21.15.112:0/4060238693 conn(0x7fd4f40cef70 msgr2=0x7fd4f4070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.689+0000 7fd4f9474700 1 -- 172.21.15.112:0/4060238693 shutdown_connections 2023-08-27T11:17:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.689+0000 7fd4f9474700 1 -- 172.21.15.112:0/4060238693 wait complete. 2023-08-27T11:17:49.702 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:17:49.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.826+0000 7fdbc68ed700 1 Processor -- start 2023-08-27T11:17:49.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.827+0000 7fdbc68ed700 1 -- start start 2023-08-27T11:17:49.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.827+0000 7fdbc68ed700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdbc0146c20 0x7fdbc0160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.827+0000 7fdbc68ed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbc006f790 0x7fdbc006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.827+0000 7fdbc68ed700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc00b5070 con 0x7fdbc00b3440 2023-08-27T11:17:49.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.827+0000 7fdbc68ed700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc0070180 con 0x7fdbc0146c20 2023-08-27T11:17:49.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.827+0000 7fdbc68ed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc0147000 con 0x7fdbc006f790 2023-08-27T11:17:49.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.827+0000 7fdbbffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdbc0146c20 0x7fdbc0160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.827+0000 7fdbbf7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fdbc00b3440 0x7fdbc015e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50720/0 (socket says 172.21.15.112:50720) 2023-08-27T11:17:49.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.827+0000 7fdbbf7fe700 1 -- 172.21.15.112:0/1176163956 learned_addr learned my addr 172.21.15.112:0/1176163956 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:49.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.828+0000 7fdbc4e8a700 1 --2- 172.21.15.112:0/1176163956 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbc006f790 0x7fdbc006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.828+0000 7fdbbffff700 1 -- 172.21.15.112:0/1176163956 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbc006f790 msgr2=0x7fdbc006fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:49.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.828+0000 7fdbbffff700 1 --2- 172.21.15.112:0/1176163956 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbc006f790 0x7fdbc006fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.828+0000 7fdbbffff700 1 -- 172.21.15.112:0/1176163956 >> v1:172.21.15.112:6789/0 conn(0x7fdbc00b3440 legacy=0x7fdbc015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.828+0000 7fdbbeffd700 1 -- 172.21.15.112:0/1176163956 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553745414 0 0) 0x7fdba8001110 con 0x7fdbc00b3440 2023-08-27T11:17:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.828+0000 7fdbbffff700 1 -- 172.21.15.112:0/1176163956 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc01614a0 con 0x7fdbc0146c20 2023-08-27T11:17:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.828+0000 7fdbbffff700 1 --2- 172.21.15.112:0/1176163956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbc0146c20 0x7fdbc0160f60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009840 tx=0x7fdbb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1072146221685cd9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.828+0000 7fdbbeffd700 1 -- 172.21.15.112:0/1176163956 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0011830 con 0x7fdbc0146c20 2023-08-27T11:17:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.828+0000 7fdbbeffd700 1 -- 172.21.15.112:0/1176163956 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb0011990 con 0x7fdbc0146c20 2023-08-27T11:17:49.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.828+0000 7fdbbeffd700 1 -- 172.21.15.112:0/1176163956 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0011d20 con 0x7fdbc0146c20 2023-08-27T11:17:49.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.829+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1176163956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbc0146c20 msgr2=0x7fdbc0160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.829+0000 7fdbc68ed700 1 --2- 172.21.15.112:0/1176163956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbc0146c20 0x7fdbc0160f60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009840 tx=0x7fdbb0009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:49.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.829+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1176163956 shutdown_connections 2023-08-27T11:17:49.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.829+0000 7fdbc68ed700 1 --2- 172.21.15.112:0/1176163956 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbc006f790 0x7fdbc006fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.829+0000 7fdbc68ed700 1 --2- 172.21.15.112:0/1176163956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbc0146c20 0x7fdbc0160f60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.829+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1176163956 >> 172.21.15.112:0/1176163956 conn(0x7fdbc00b1e90 msgr2=0x7fdbc00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:49.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.829+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1176163956 shutdown_connections 2023-08-27T11:17:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.829+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1176163956 wait complete. 2023-08-27T11:17:49.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.830+0000 7fdbc68ed700 1 Processor -- start 2023-08-27T11:17:49.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.830+0000 7fdbc68ed700 1 -- start start 2023-08-27T11:17:49.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc68ed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc006f790 0x7fdbc0144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc68ed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbc00b3440 0x7fdbc015c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc68ed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbc0146c20 0x7fdbc015cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc68ed700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc01442e0 con 0x7fdbc0146c20 2023-08-27T11:17:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc68ed700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc015d1b0 con 0x7fdbc006f790 2023-08-27T11:17:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc68ed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc013f360 con 0x7fdbc00b3440 2023-08-27T11:17:49.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc4e8a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbc0146c20 0x7fdbc015cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc4e8a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbc0146c20 0x7fdbc015cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56344/0 (socket says 172.21.15.112:56344) 2023-08-27T11:17:49.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc4e8a700 1 -- 172.21.15.112:0/1322648804 learned_addr learned my addr 172.21.15.112:0/1322648804 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:49.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbbf7fe700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbc00b3440 0x7fdbc015c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbbffff700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc006f790 0x7fdbc0144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc4e8a700 1 -- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbc00b3440 msgr2=0x7fdbc015c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc4e8a700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbc00b3440 0x7fdbc015c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc4e8a700 1 -- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc006f790 msgr2=0x7fdbc0144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc4e8a700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc006f790 0x7fdbc0144830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc4e8a700 1 -- 172.21.15.112:0/1322648804 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb00094a0 con 0x7fdbc0146c20 2023-08-27T11:17:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.831+0000 7fdbc4e8a700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbc0146c20 0x7fdbc015cc70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800efc0 tx=0x7fdbb800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.832+0000 7fdbbcff9700 1 -- 172.21.15.112:0/1322648804 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb800cd40 con 0x7fdbc0146c20 2023-08-27T11:17:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.832+0000 7fdbbcff9700 1 -- 172.21.15.112:0/1322648804 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb800f040 con 0x7fdbc0146c20 2023-08-27T11:17:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.832+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1322648804 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc013f640 con 0x7fdbc0146c20 2023-08-27T11:17:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.832+0000 7fdbbcff9700 1 -- 172.21.15.112:0/1322648804 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80146c0 con 0x7fdbc0146c20 2023-08-27T11:17:49.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.832+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1322648804 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc013fc00 con 0x7fdbc0146c20 2023-08-27T11:17:49.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.832+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1322648804 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc0067e90 con 0x7fdbc0146c20 2023-08-27T11:17:49.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.833+0000 7fdbbcff9700 1 -- 172.21.15.112:0/1322648804 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdbb8014820 con 0x7fdbc0146c20 2023-08-27T11:17:49.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.833+0000 7fdbbcff9700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdba006a0e0 0x7fdba006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.833+0000 7fdbbcff9700 1 -- 172.21.15.112:0/1322648804 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdbb8088e50 con 0x7fdbc0146c20 2023-08-27T11:17:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.834+0000 7fdbbffff700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdba006a0e0 0x7fdba006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.834+0000 7fdbbffff700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdba006a0e0 0x7fdba006c5a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0000c00 tx=0x7fdbb001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:49.838+0000 7fdbbcff9700 1 -- 172.21.15.112:0/1322648804 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdbb8058060 con 0x7fdbc0146c20 2023-08-27T11:17:50.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.028+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1322648804 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdbc013fdf0 con 0x7fdbc0146c20 2023-08-27T11:17:50.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.028+0000 7fdbbcff9700 1 -- 172.21.15.112:0/1322648804 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdbb80180a0 con 0x7fdbc0146c20 2023-08-27T11:17:50.030 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:17:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.030+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdba006a0e0 msgr2=0x7fdba006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.030+0000 7fdbc68ed700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdba006a0e0 0x7fdba006c5a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0000c00 tx=0x7fdbb001c040 comp rx=0 tx=0).stop 2023-08-27T11:17:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.030+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1322648804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbc0146c20 msgr2=0x7fdbc015cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.030+0000 7fdbc68ed700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbc0146c20 0x7fdbc015cc70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800efc0 tx=0x7fdbb800c3a0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.030+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1322648804 shutdown_connections 2023-08-27T11:17:50.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.030+0000 7fdbc68ed700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdba006a0e0 0x7fdba006c5a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.030+0000 7fdbc68ed700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbc0146c20 0x7fdbc015cc70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.030+0000 7fdbc68ed700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc006f790 0x7fdbc0144830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.030+0000 7fdbc68ed700 1 --2- 172.21.15.112:0/1322648804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbc00b3440 0x7fdbc015c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.030+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1322648804 >> 172.21.15.112:0/1322648804 conn(0x7fdbc00b1e90 msgr2=0x7fdbc00720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:50.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.030+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1322648804 shutdown_connections 2023-08-27T11:17:50.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.030+0000 7fdbc68ed700 1 -- 172.21.15.112:0/1322648804 wait complete. 2023-08-27T11:17:50.044 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:17:50.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.167+0000 7f9ba0104700 1 Processor -- start 2023-08-27T11:17:50.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.168+0000 7f9ba0104700 1 -- start start 2023-08-27T11:17:50.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.168+0000 7f9ba0104700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9b98146c20 0x7f9b98160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.168+0000 7f9ba0104700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9b980b3440 0x7f9b9815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.168+0000 7f9ba0104700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b980b5070 con 0x7f9b9815e6e0 2023-08-27T11:17:50.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.168+0000 7f9ba0104700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b98147000 con 0x7f9b98146c20 2023-08-27T11:17:50.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.168+0000 7f9ba0104700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b981614a0 con 0x7f9b980b3440 2023-08-27T11:17:50.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.169+0000 7f9b9dea0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9b98146c20 0x7f9b98160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.169+0000 7f9b9dea0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9b98146c20 0x7f9b98160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56350/0 (socket says 172.21.15.112:56350) 2023-08-27T11:17:50.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.169+0000 7f9b9dea0700 1 -- 172.21.15.112:0/1126759345 learned_addr learned my addr 172.21.15.112:0/1126759345 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:50.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.169+0000 7f9b9dea0700 1 -- 172.21.15.112:0/1126759345 >> v2:172.21.15.138:3300/0 conn(0x7f9b980b3440 msgr2=0x7f9b9815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:50.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.169+0000 7f9b9d69f700 1 --2- 172.21.15.112:0/1126759345 >> v2:172.21.15.138:3300/0 conn(0x7f9b980b3440 0x7f9b9815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.169+0000 7f9b9dea0700 1 --2- 172.21.15.112:0/1126759345 >> v2:172.21.15.138:3300/0 conn(0x7f9b980b3440 0x7f9b9815e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.169+0000 7f9b9dea0700 1 -- 172.21.15.112:0/1126759345 >> v1:172.21.15.138:6789/0 conn(0x7f9b9815e6e0 legacy=0x7f9b9806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:50.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.169+0000 7f9b9dea0700 1 -- 172.21.15.112:0/1126759345 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b98161740 con 0x7f9b98146c20 2023-08-27T11:17:50.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.169+0000 7f9b9dea0700 1 --2- 172.21.15.112:0/1126759345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b98146c20 0x7f9b98160f60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9b980b2d80 tx=0x7f9b88009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9db7cbd6a4ac94b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:50.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.169+0000 7f9b9ce9e700 1 -- 172.21.15.112:0/1126759345 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88004230 con 0x7f9b98146c20 2023-08-27T11:17:50.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.169+0000 7f9b9ce9e700 1 -- 172.21.15.112:0/1126759345 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b88004390 con 0x7f9b98146c20 2023-08-27T11:17:50.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.169+0000 7f9b9ce9e700 1 -- 172.21.15.112:0/1126759345 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b880036a0 con 0x7f9b98146c20 2023-08-27T11:17:50.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.170+0000 7f9ba0104700 1 -- 172.21.15.112:0/1126759345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b98146c20 msgr2=0x7f9b98160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.170+0000 7f9ba0104700 1 --2- 172.21.15.112:0/1126759345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b98146c20 0x7f9b98160f60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9b980b2d80 tx=0x7f9b88009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.170+0000 7f9ba0104700 1 -- 172.21.15.112:0/1126759345 shutdown_connections 2023-08-27T11:17:50.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.170+0000 7f9ba0104700 1 --2- 172.21.15.112:0/1126759345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b98146c20 0x7f9b98160f60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.170+0000 7f9ba0104700 1 --2- 172.21.15.112:0/1126759345 >> v2:172.21.15.138:3300/0 conn(0x7f9b980b3440 0x7f9b9815e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.170+0000 7f9ba0104700 1 -- 172.21.15.112:0/1126759345 >> 172.21.15.112:0/1126759345 conn(0x7f9b980b1e90 msgr2=0x7f9b980b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:50.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.170+0000 7f9ba0104700 1 -- 172.21.15.112:0/1126759345 shutdown_connections 2023-08-27T11:17:50.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.170+0000 7f9ba0104700 1 -- 172.21.15.112:0/1126759345 wait complete. 2023-08-27T11:17:50.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.171+0000 7f9ba0104700 1 Processor -- start 2023-08-27T11:17:50.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.171+0000 7f9ba0104700 1 -- start start 2023-08-27T11:17:50.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9ba0104700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b980b3440 0x7f9b980c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9ba0104700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b98146c20 0x7f9b980c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9ba0104700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b9815e6e0 0x7f9b98144da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9ba0104700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980c5b90 con 0x7f9b9815e6e0 2023-08-27T11:17:50.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9ba0104700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980c5e70 con 0x7f9b980b3440 2023-08-27T11:17:50.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9ba0104700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980c6150 con 0x7f9b98146c20 2023-08-27T11:17:50.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9b9e6a1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b9815e6e0 0x7f9b98144da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9b9e6a1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b9815e6e0 0x7f9b98144da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56358/0 (socket says 172.21.15.112:56358) 2023-08-27T11:17:50.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9b9e6a1700 1 -- 172.21.15.112:0/294125796 learned_addr learned my addr 172.21.15.112:0/294125796 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9b9dea0700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b980b3440 0x7f9b980c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9b9d69f700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b98146c20 0x7f9b980c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9b9e6a1700 1 -- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b98146c20 msgr2=0x7f9b980c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9b9e6a1700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b98146c20 0x7f9b980c50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9b9e6a1700 1 -- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b980b3440 msgr2=0x7f9b980c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9b9e6a1700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b980b3440 0x7f9b980c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9b9e6a1700 1 -- 172.21.15.112:0/294125796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b88009550 con 0x7f9b9815e6e0 2023-08-27T11:17:50.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9b9dea0700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b980b3440 0x7f9b980c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:50.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.172+0000 7f9b9e6a1700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b9815e6e0 0x7f9b98144da0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000b9e0 tx=0x7f9b9000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:50.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.173+0000 7f9b8e7fc700 1 -- 172.21.15.112:0/294125796 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9000c9b0 con 0x7f9b9815e6e0 2023-08-27T11:17:50.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.173+0000 7f9b8e7fc700 1 -- 172.21.15.112:0/294125796 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9000cb50 con 0x7f9b9815e6e0 2023-08-27T11:17:50.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.173+0000 7f9ba0104700 1 -- 172.21.15.112:0/294125796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b980c6430 con 0x7f9b9815e6e0 2023-08-27T11:17:50.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.173+0000 7f9b8e7fc700 1 -- 172.21.15.112:0/294125796 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90014440 con 0x7f9b9815e6e0 2023-08-27T11:17:50.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.173+0000 7f9ba0104700 1 -- 172.21.15.112:0/294125796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b98137110 con 0x7f9b9815e6e0 2023-08-27T11:17:50.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.173+0000 7f9ba0104700 1 -- 172.21.15.112:0/294125796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b98067e90 con 0x7f9b9815e6e0 2023-08-27T11:17:50.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.174+0000 7f9b8e7fc700 1 -- 172.21.15.112:0/294125796 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9b900145a0 con 0x7f9b9815e6e0 2023-08-27T11:17:50.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.174+0000 7f9b8e7fc700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b7c06a0e0 0x7f9b7c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.175+0000 7f9b8e7fc700 1 -- 172.21.15.112:0/294125796 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9b90088050 con 0x7f9b9815e6e0 2023-08-27T11:17:50.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.175+0000 7f9b9dea0700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b7c06a0e0 0x7f9b7c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.175+0000 7f9b9dea0700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b7c06a0e0 0x7f9b7c06c5a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009c10 tx=0x7f9b88008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:50.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.179+0000 7f9b8e7fc700 1 -- 172.21.15.112:0/294125796 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9b90053880 con 0x7f9b9815e6e0 2023-08-27T11:17:50.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.368+0000 7f9ba0104700 1 -- 172.21.15.112:0/294125796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b981426d0 con 0x7f9b9815e6e0 2023-08-27T11:17:50.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.368+0000 7f9b8e7fc700 1 -- 172.21.15.112:0/294125796 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b90056ea0 con 0x7f9b9815e6e0 2023-08-27T11:17:50.370 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:17:50.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.370+0000 7f9ba0104700 1 -- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b7c06a0e0 msgr2=0x7f9b7c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.370+0000 7f9ba0104700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b7c06a0e0 0x7f9b7c06c5a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009c10 tx=0x7f9b88008030 comp rx=0 tx=0).stop 2023-08-27T11:17:50.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.370+0000 7f9ba0104700 1 -- 172.21.15.112:0/294125796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b9815e6e0 msgr2=0x7f9b98144da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.370+0000 7f9ba0104700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b9815e6e0 0x7f9b98144da0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000b9e0 tx=0x7f9b9000bda0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.370+0000 7f9ba0104700 1 -- 172.21.15.112:0/294125796 shutdown_connections 2023-08-27T11:17:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.370+0000 7f9ba0104700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b7c06a0e0 0x7f9b7c06c5a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.370+0000 7f9ba0104700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b9815e6e0 0x7f9b98144da0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.370+0000 7f9ba0104700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b980b3440 0x7f9b980c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.370+0000 7f9ba0104700 1 --2- 172.21.15.112:0/294125796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b98146c20 0x7f9b980c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.370+0000 7f9ba0104700 1 -- 172.21.15.112:0/294125796 >> 172.21.15.112:0/294125796 conn(0x7f9b980b1e90 msgr2=0x7f9b980b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.371+0000 7f9ba0104700 1 -- 172.21.15.112:0/294125796 shutdown_connections 2023-08-27T11:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.371+0000 7f9ba0104700 1 -- 172.21.15.112:0/294125796 wait complete. 2023-08-27T11:17:50.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:17:50.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.508+0000 7f3971712700 1 Processor -- start 2023-08-27T11:17:50.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.508+0000 7f3971712700 1 -- start start 2023-08-27T11:17:50.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f3971712700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f396c146bf0 0x7f396c0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f3971712700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f396c0d1500 con 0x7f396c070eb0 2023-08-27T11:17:50.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f3971712700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f396c0d0460 con 0x7f396c146fd0 2023-08-27T11:17:50.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f3971712700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396c0d0710 con 0x7f396c146bf0 2023-08-27T11:17:50.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f396b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f396c070eb0 0x7f396c0d1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50748/0 (socket says 172.21.15.112:50748) 2023-08-27T11:17:50.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f396b7fe700 1 -- 172.21.15.112:0/770400902 learned_addr learned my addr 172.21.15.112:0/770400902 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:50.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f396affd700 1 --2- 172.21.15.112:0/770400902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f396c146bf0 0x7f396c0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f3969ffb700 1 -- 172.21.15.112:0/770400902 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543714238 0 0) 0x7f396c0d1500 con 0x7f396c070eb0 2023-08-27T11:17:50.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f3969ffb700 1 -- 172.21.15.112:0/770400902 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39540035b0 con 0x7f396c070eb0 2023-08-27T11:17:50.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f3969ffb700 1 -- 172.21.15.112:0/770400902 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379713323 0 0) 0x7f396c0d0460 con 0x7f396c146fd0 2023-08-27T11:17:50.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f3969ffb700 1 -- 172.21.15.112:0/770400902 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f396c0d1500 con 0x7f396c146fd0 2023-08-27T11:17:50.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f396affd700 1 -- 172.21.15.112:0/770400902 >> v1:172.21.15.138:6789/0 conn(0x7f396c146fd0 legacy=0x7f396c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f396affd700 1 -- 172.21.15.112:0/770400902 >> v1:172.21.15.112:6789/0 conn(0x7f396c070eb0 legacy=0x7f396c0d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.509+0000 7f396affd700 1 -- 172.21.15.112:0/770400902 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396c0d09f0 con 0x7f396c146bf0 2023-08-27T11:17:50.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.510+0000 7f396affd700 1 --2- 172.21.15.112:0/770400902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f396c146bf0 0x7f396c0b5db0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f395c009870 tx=0x7f395c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66e6d984e00b5f98 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:50.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.510+0000 7f3969ffb700 1 -- 172.21.15.112:0/770400902 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c011830 con 0x7f396c146bf0 2023-08-27T11:17:50.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.510+0000 7f3969ffb700 1 -- 172.21.15.112:0/770400902 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c011990 con 0x7f396c146bf0 2023-08-27T11:17:50.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.510+0000 7f3969ffb700 1 -- 172.21.15.112:0/770400902 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c011d00 con 0x7f396c146bf0 2023-08-27T11:17:50.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.510+0000 7f3971712700 1 -- 172.21.15.112:0/770400902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f396c146bf0 msgr2=0x7f396c0b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.510+0000 7f3971712700 1 --2- 172.21.15.112:0/770400902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f396c146bf0 0x7f396c0b5db0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f395c009870 tx=0x7f395c009b80 comp rx=0 tx=0).stop 2023-08-27T11:17:50.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.510+0000 7f3971712700 1 -- 172.21.15.112:0/770400902 shutdown_connections 2023-08-27T11:17:50.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.510+0000 7f3971712700 1 --2- 172.21.15.112:0/770400902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f396c146bf0 0x7f396c0b5db0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.510+0000 7f3971712700 1 -- 172.21.15.112:0/770400902 >> 172.21.15.112:0/770400902 conn(0x7f396c06c450 msgr2=0x7f396c06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:50.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.510+0000 7f3971712700 1 -- 172.21.15.112:0/770400902 shutdown_connections 2023-08-27T11:17:50.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.510+0000 7f3971712700 1 -- 172.21.15.112:0/770400902 wait complete. 2023-08-27T11:17:50.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.512+0000 7f3971712700 1 Processor -- start 2023-08-27T11:17:50.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.512+0000 7f3971712700 1 -- start start 2023-08-27T11:17:50.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.512+0000 7f3971712700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f396c070eb0 0x7f396c13f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f3971712700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f396c146bf0 0x7f396c1f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f3971712700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f396c146fd0 0x7f396c1f3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f3971712700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c0b5fd0 con 0x7f396c146fd0 2023-08-27T11:17:50.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f3971712700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c13d710 con 0x7f396c146bf0 2023-08-27T11:17:50.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f3971712700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396c13d9f0 con 0x7f396c070eb0 2023-08-27T11:17:50.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f396b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f396c146fd0 0x7f396c1f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f396b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f396c146fd0 0x7f396c1f3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56372/0 (socket says 172.21.15.112:56372) 2023-08-27T11:17:50.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f396b7fe700 1 -- 172.21.15.112:0/2569635365 learned_addr learned my addr 172.21.15.112:0/2569635365 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:50.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f396a7fc700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f396c146bf0 0x7f396c1f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f396affd700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f396c070eb0 0x7f396c13f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f396b7fe700 1 -- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f396c070eb0 msgr2=0x7f396c13f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f396b7fe700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f396c070eb0 0x7f396c13f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f396b7fe700 1 -- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f396c146bf0 msgr2=0x7f396c1f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f396b7fe700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f396c146bf0 0x7f396c1f2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f396b7fe700 1 -- 172.21.15.112:0/2569635365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395c0094a0 con 0x7f396c146fd0 2023-08-27T11:17:50.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f396b7fe700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f396c146fd0 0x7f396c1f3360 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f396400cba0 tx=0x7f396400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:50.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f39537fe700 1 -- 172.21.15.112:0/2569635365 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396400aa40 con 0x7f396c146fd0 2023-08-27T11:17:50.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f3971712700 1 -- 172.21.15.112:0/2569635365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396c13dcd0 con 0x7f396c146fd0 2023-08-27T11:17:50.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f39537fe700 1 -- 172.21.15.112:0/2569635365 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396400aba0 con 0x7f396c146fd0 2023-08-27T11:17:50.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f39537fe700 1 -- 172.21.15.112:0/2569635365 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964012400 con 0x7f396c146fd0 2023-08-27T11:17:50.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.513+0000 7f3971712700 1 -- 172.21.15.112:0/2569635365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396c142ee0 con 0x7f396c146fd0 2023-08-27T11:17:50.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.514+0000 7f3971712700 1 -- 172.21.15.112:0/2569635365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396c067e90 con 0x7f396c146fd0 2023-08-27T11:17:50.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.515+0000 7f39537fe700 1 -- 172.21.15.112:0/2569635365 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f396401b3b0 con 0x7f396c146fd0 2023-08-27T11:17:50.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.515+0000 7f39537fe700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f395806a0e0 0x7f395806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.515+0000 7f39537fe700 1 -- 172.21.15.112:0/2569635365 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39640881b0 con 0x7f396c146fd0 2023-08-27T11:17:50.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.515+0000 7f396affd700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f395806a0e0 0x7f395806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.516+0000 7f396affd700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f395806a0e0 0x7f395806c5a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f395c0090b0 tx=0x7f395c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:50.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.519+0000 7f39537fe700 1 -- 172.21.15.112:0/2569635365 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f39640538b0 con 0x7f396c146fd0 2023-08-27T11:17:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.708+0000 7f3971712700 1 -- 172.21.15.112:0/2569635365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f396c067a20 con 0x7f396c146fd0 2023-08-27T11:17:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.708+0000 7f39537fe700 1 -- 172.21.15.112:0/2569635365 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3964056ed0 con 0x7f396c146fd0 2023-08-27T11:17:50.710 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:17:50.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.710+0000 7f3971712700 1 -- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f395806a0e0 msgr2=0x7f395806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.710+0000 7f3971712700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f395806a0e0 0x7f395806c5a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f395c0090b0 tx=0x7f395c008030 comp rx=0 tx=0).stop 2023-08-27T11:17:50.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.710+0000 7f3971712700 1 -- 172.21.15.112:0/2569635365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f396c146fd0 msgr2=0x7f396c1f3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.710+0000 7f3971712700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f396c146fd0 0x7f396c1f3360 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f396400cba0 tx=0x7f396400ceb0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.710+0000 7f3971712700 1 -- 172.21.15.112:0/2569635365 shutdown_connections 2023-08-27T11:17:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.710+0000 7f3971712700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f395806a0e0 0x7f395806c5a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.710+0000 7f3971712700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f396c146fd0 0x7f396c1f3360 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.710+0000 7f3971712700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f396c070eb0 0x7f396c13f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.710+0000 7f3971712700 1 --2- 172.21.15.112:0/2569635365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f396c146bf0 0x7f396c1f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.710+0000 7f3971712700 1 -- 172.21.15.112:0/2569635365 >> 172.21.15.112:0/2569635365 conn(0x7f396c06c450 msgr2=0x7f396c15f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:50.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.710+0000 7f3971712700 1 -- 172.21.15.112:0/2569635365 shutdown_connections 2023-08-27T11:17:50.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.710+0000 7f3971712700 1 -- 172.21.15.112:0/2569635365 wait complete. 2023-08-27T11:17:50.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:17:50.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.842+0000 7f769dee7700 1 Processor -- start 2023-08-27T11:17:50.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.842+0000 7f769dee7700 1 -- start start 2023-08-27T11:17:50.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.843+0000 7f769dee7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7698151060 0x7f769815e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f769dee7700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76980b02f0 con 0x7f769815e990 2023-08-27T11:17:50.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f769dee7700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76980b05d0 con 0x7f769806b4c0 2023-08-27T11:17:50.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f769dee7700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76980b08b0 con 0x7f7698151060 2023-08-27T11:17:50.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f7696ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f769815e990 0x7f769806adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50760/0 (socket says 172.21.15.112:50760) 2023-08-27T11:17:50.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f7696ffd700 1 -- 172.21.15.112:0/4218860691 learned_addr learned my addr 172.21.15.112:0/4218860691 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:50.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f76977fe700 1 --2- 172.21.15.112:0/4218860691 >> v2:172.21.15.138:3300/0 conn(0x7f7698151060 0x7f769815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f76967fc700 1 -- 172.21.15.112:0/4218860691 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801008682 0 0) 0x7f76980b02f0 con 0x7f769815e990 2023-08-27T11:17:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f76967fc700 1 -- 172.21.15.112:0/4218860691 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76800035b0 con 0x7f769815e990 2023-08-27T11:17:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f76967fc700 1 -- 172.21.15.112:0/4218860691 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589232744 0 0) 0x7f76980b05d0 con 0x7f769806b4c0 2023-08-27T11:17:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f76967fc700 1 -- 172.21.15.112:0/4218860691 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76980b02f0 con 0x7f769806b4c0 2023-08-27T11:17:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f76977fe700 1 -- 172.21.15.112:0/4218860691 >> v1:172.21.15.138:6789/0 conn(0x7f769806b4c0 legacy=0x7f76980cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f76967fc700 1 -- 172.21.15.112:0/4218860691 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2159007560 0 0) 0x7f76800035b0 con 0x7f769815e990 2023-08-27T11:17:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f76977fe700 1 -- 172.21.15.112:0/4218860691 >> v1:172.21.15.112:6789/0 conn(0x7f769815e990 legacy=0x7f769806adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f76977fe700 1 -- 172.21.15.112:0/4218860691 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76980b0b90 con 0x7f7698151060 2023-08-27T11:17:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.844+0000 7f76977fe700 1 --2- 172.21.15.112:0/4218860691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7698151060 0x7f769815e450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f768c009840 tx=0x7f768c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=584432e169c17ad2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.845+0000 7f76967fc700 1 -- 172.21.15.112:0/4218860691 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688001670 con 0x7f7698151060 2023-08-27T11:17:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.845+0000 7f76967fc700 1 -- 172.21.15.112:0/4218860691 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c004030 con 0x7f7698151060 2023-08-27T11:17:50.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.845+0000 7f76967fc700 1 -- 172.21.15.112:0/4218860691 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0043c0 con 0x7f7698151060 2023-08-27T11:17:50.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.845+0000 7f769dee7700 1 -- 172.21.15.112:0/4218860691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7698151060 msgr2=0x7f769815e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.845+0000 7f769dee7700 1 --2- 172.21.15.112:0/4218860691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7698151060 0x7f769815e450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f768c009840 tx=0x7f768c009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:50.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.845+0000 7f769dee7700 1 -- 172.21.15.112:0/4218860691 shutdown_connections 2023-08-27T11:17:50.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.845+0000 7f769dee7700 1 --2- 172.21.15.112:0/4218860691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7698151060 0x7f769815e450 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.845+0000 7f769dee7700 1 -- 172.21.15.112:0/4218860691 >> 172.21.15.112:0/4218860691 conn(0x7f76980d0000 msgr2=0x7f76980d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:50.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.845+0000 7f769dee7700 1 -- 172.21.15.112:0/4218860691 shutdown_connections 2023-08-27T11:17:50.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.845+0000 7f769dee7700 1 -- 172.21.15.112:0/4218860691 wait complete. 2023-08-27T11:17:50.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.847+0000 7f769dee7700 1 Processor -- start 2023-08-27T11:17:50.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.847+0000 7f769dee7700 1 -- start start 2023-08-27T11:17:50.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f769dee7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769806b4c0 0x7f7698150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f769dee7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7698151060 0x7f76980718c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f769dee7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769815e990 0x7f7698071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f769dee7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76981f2eb0 con 0x7f7698151060 2023-08-27T11:17:50.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f769dee7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76981f3190 con 0x7f769806b4c0 2023-08-27T11:17:50.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f769dee7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76981f3470 con 0x7f769815e990 2023-08-27T11:17:50.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f7696ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7698151060 0x7f76980718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f7696ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7698151060 0x7f76980718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56384/0 (socket says 172.21.15.112:56384) 2023-08-27T11:17:50.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f7696ffd700 1 -- 172.21.15.112:0/3233984422 learned_addr learned my addr 172.21.15.112:0/3233984422 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:50.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f76977fe700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769806b4c0 0x7f7698150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f7697fff700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769815e990 0x7f7698071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f7696ffd700 1 -- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769815e990 msgr2=0x7f7698071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f7696ffd700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769815e990 0x7f7698071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f7696ffd700 1 -- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769806b4c0 msgr2=0x7f7698150270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:50.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f7696ffd700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769806b4c0 0x7f7698150270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:50.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f7696ffd700 1 -- 172.21.15.112:0/3233984422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c0094a0 con 0x7f7698151060 2023-08-27T11:17:50.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f7696ffd700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7698151060 0x7f76980718c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f768800e8e0 tx=0x7f768800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:50.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f767ffff700 1 -- 172.21.15.112:0/3233984422 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688004a80 con 0x7f7698151060 2023-08-27T11:17:50.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.848+0000 7f767ffff700 1 -- 172.21.15.112:0/3233984422 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7688004be0 con 0x7f7698151060 2023-08-27T11:17:50.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.849+0000 7f769dee7700 1 -- 172.21.15.112:0/3233984422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76981f3750 con 0x7f7698151060 2023-08-27T11:17:50.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.849+0000 7f767ffff700 1 -- 172.21.15.112:0/3233984422 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76880070d0 con 0x7f7698151060 2023-08-27T11:17:50.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.849+0000 7f769dee7700 1 -- 172.21.15.112:0/3233984422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7698154020 con 0x7f7698151060 2023-08-27T11:17:50.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.849+0000 7f769dee7700 1 -- 172.21.15.112:0/3233984422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76981f3c70 con 0x7f7698151060 2023-08-27T11:17:50.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.850+0000 7f767ffff700 1 -- 172.21.15.112:0/3233984422 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7688007230 con 0x7f7698151060 2023-08-27T11:17:50.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.850+0000 7f767ffff700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f768406a0e0 0x7f768406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:50.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.850+0000 7f767ffff700 1 -- 172.21.15.112:0/3233984422 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f768808aa20 con 0x7f7698151060 2023-08-27T11:17:50.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.851+0000 7f76977fe700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f768406a0e0 0x7f768406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:50.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.851+0000 7f76977fe700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f768406a0e0 0x7f768406c5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f768c0090b0 tx=0x7f768c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:50.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:50.855+0000 7f767ffff700 1 -- 172.21.15.112:0/3233984422 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f76981f3c70 con 0x7f7698151060 2023-08-27T11:17:51.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.084+0000 7f769dee7700 1 -- 172.21.15.112:0/3233984422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76981f3c70 con 0x7f7698151060 2023-08-27T11:17:51.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.084+0000 7f767ffff700 1 -- 172.21.15.112:0/3233984422 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4881 (secure 0 0 0) 0x7f76981f3c70 con 0x7f7698151060 2023-08-27T11:17:51.086 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:51.086 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3289488908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3289488908},{"type":"v1","addr":"172.21.15.112:6835","nonce":3289488908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3991653721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3991653721},{"type":"v1","addr":"172.21.15.112:6834","nonce":3991653721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1923303457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1923303457},{"type":"v1","addr":"172.21.15.138:6836","nonce":1923303457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:17:04.761658+0000","modified":"2023-08-27T11:17:08.372245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/1827312637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1827312637},{"type":"v1","addr":"172.21.15.138:6837","nonce":1827312637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:17:51.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.086+0000 7f769dee7700 1 -- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f768406a0e0 msgr2=0x7f768406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.086+0000 7f769dee7700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f768406a0e0 0x7f768406c5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f768c0090b0 tx=0x7f768c008030 comp rx=0 tx=0).stop 2023-08-27T11:17:51.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.086+0000 7f769dee7700 1 -- 172.21.15.112:0/3233984422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7698151060 msgr2=0x7f76980718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.086+0000 7f769dee7700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7698151060 0x7f76980718c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f768800e8e0 tx=0x7f768800ebf0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.086+0000 7f769dee7700 1 -- 172.21.15.112:0/3233984422 shutdown_connections 2023-08-27T11:17:51.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.086+0000 7f769dee7700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f768406a0e0 0x7f768406c5a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.086+0000 7f769dee7700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769815e990 0x7f7698071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.086+0000 7f769dee7700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769806b4c0 0x7f7698150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.086+0000 7f769dee7700 1 --2- 172.21.15.112:0/3233984422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7698151060 0x7f76980718c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.086+0000 7f769dee7700 1 -- 172.21.15.112:0/3233984422 >> 172.21.15.112:0/3233984422 conn(0x7f76980d0000 msgr2=0x7f76980708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.086+0000 7f769dee7700 1 -- 172.21.15.112:0/3233984422 shutdown_connections 2023-08-27T11:17:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.086+0000 7f769dee7700 1 -- 172.21.15.112:0/3233984422 wait complete. 2023-08-27T11:17:51.091 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2023-08-27T11:17:51.100 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.218+0000 7ff69f94d700 1 Processor -- start 2023-08-27T11:17:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.219+0000 7ff69f94d700 1 -- start start 2023-08-27T11:17:51.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.219+0000 7ff69f94d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff69806adb0 0x7ff69806b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.219+0000 7ff69f94d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69815eb60 con 0x7ff698151060 2023-08-27T11:17:51.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.219+0000 7ff69f94d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6980b02f0 con 0x7ff6980cdbd0 2023-08-27T11:17:51.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.219+0000 7ff69f94d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69806b770 con 0x7ff69806adb0 2023-08-27T11:17:51.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.219+0000 7ff69d6e9700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff698151060 0x7ff69815e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50772/0 (socket says 172.21.15.112:50772) 2023-08-27T11:17:51.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.219+0000 7ff69d6e9700 1 -- 172.21.15.112:0/591923526 learned_addr learned my addr 172.21.15.112:0/591923526 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:51.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343346519 0 0) 0x7ff69815eb60 con 0x7ff698151060 2023-08-27T11:17:51.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6840035b0 con 0x7ff698151060 2023-08-27T11:17:51.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff69cee8700 1 --2- 172.21.15.112:0/591923526 >> v2:172.21.15.138:3300/0 conn(0x7ff69806adb0 0x7ff69806b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 478979084 0 0) 0x7ff6840035b0 con 0x7ff698151060 2023-08-27T11:17:51.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 >> v2:172.21.15.138:3300/0 conn(0x7ff69806adb0 msgr2=0x7ff69806b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:51.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 --2- 172.21.15.112:0/591923526 >> v2:172.21.15.138:3300/0 conn(0x7ff69806adb0 0x7ff69806b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 >> v1:172.21.15.138:6789/0 conn(0x7ff6980cdbd0 legacy=0x7ff6980ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6980b05d0 con 0x7ff698151060 2023-08-27T11:17:51.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7ff688001670 con 0x7ff698151060 2023-08-27T11:17:51.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 >> v1:172.21.15.112:6789/0 conn(0x7ff698151060 legacy=0x7ff69815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 --2- 172.21.15.112:0/591923526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff684003a00 0x7ff684003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 --2- 172.21.15.112:0/591923526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684004850 0x7ff684004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 --2- 172.21.15.112:0/591923526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff684005210 0x7ff68400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69806b770 con 0x7ff684004850 2023-08-27T11:17:51.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68400c160 con 0x7ff684003a00 2023-08-27T11:17:51.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68400c320 con 0x7ff684005210 2023-08-27T11:17:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff69cee8700 1 --2- 172.21.15.112:0/591923526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684004850 0x7ff684004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff69deea700 1 --2- 172.21.15.112:0/591923526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff684005210 0x7ff68400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.220+0000 7ff69d6e9700 1 --2- 172.21.15.112:0/591923526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff684003a00 0x7ff684003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.221+0000 7ff69cee8700 1 -- 172.21.15.112:0/591923526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff684005210 msgr2=0x7ff68400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.221+0000 7ff69cee8700 1 --2- 172.21.15.112:0/591923526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff684005210 0x7ff68400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.221+0000 7ff69cee8700 1 -- 172.21.15.112:0/591923526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff684003a00 msgr2=0x7ff684003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:51.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.221+0000 7ff69cee8700 1 --2- 172.21.15.112:0/591923526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff684003a00 0x7ff684003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.221+0000 7ff69cee8700 1 -- 172.21.15.112:0/591923526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6840035b0 con 0x7ff684004850 2023-08-27T11:17:51.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.221+0000 7ff69deea700 1 -- 172.21.15.112:0/591923526 reap_dead start 2023-08-27T11:17:51.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.221+0000 7ff69d6e9700 1 --2- 172.21.15.112:0/591923526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff684003a00 0x7ff684003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:51.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.221+0000 7ff69cee8700 1 --2- 172.21.15.112:0/591923526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684004850 0x7ff684004cd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff694009890 tx=0x7ff684005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99cfcd75cc617f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:51.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.221+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69400c070 con 0x7ff684004850 2023-08-27T11:17:51.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.222+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff694013610 con 0x7ff684004850 2023-08-27T11:17:51.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.222+0000 7ff68ffff700 1 -- 172.21.15.112:0/591923526 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6940139a0 con 0x7ff684004850 2023-08-27T11:17:51.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.222+0000 7ff69f94d700 1 -- 172.21.15.112:0/591923526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684004850 msgr2=0x7ff684004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.222+0000 7ff69f94d700 1 --2- 172.21.15.112:0/591923526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684004850 0x7ff684004cd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff694009890 tx=0x7ff684005880 comp rx=0 tx=0).stop 2023-08-27T11:17:51.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.222+0000 7ff69f94d700 1 -- 172.21.15.112:0/591923526 shutdown_connections 2023-08-27T11:17:51.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.222+0000 7ff69f94d700 1 --2- 172.21.15.112:0/591923526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684004850 0x7ff684004cd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.222+0000 7ff69f94d700 1 -- 172.21.15.112:0/591923526 >> 172.21.15.112:0/591923526 conn(0x7ff6980d0000 msgr2=0x7ff6980d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.223+0000 7ff69f94d700 1 -- 172.21.15.112:0/591923526 shutdown_connections 2023-08-27T11:17:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.223+0000 7ff69f94d700 1 -- 172.21.15.112:0/591923526 wait complete. 2023-08-27T11:17:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.224+0000 7ff69f94d700 1 Processor -- start 2023-08-27T11:17:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.224+0000 7ff69f94d700 1 -- start start 2023-08-27T11:17:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69f94d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684004850 0x7ff6980c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69f94d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff698153650 0x7ff6980c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69f94d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff698153ad0 0x7ff6980c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69f94d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6980c9b10 con 0x7ff684004850 2023-08-27T11:17:51.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69f94d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6981540a0 con 0x7ff698153650 2023-08-27T11:17:51.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69f94d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff698154380 con 0x7ff698153ad0 2023-08-27T11:17:51.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69d6e9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684004850 0x7ff6980c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69d6e9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684004850 0x7ff6980c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56402/0 (socket says 172.21.15.112:56402) 2023-08-27T11:17:51.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69d6e9700 1 -- 172.21.15.112:0/2719555220 learned_addr learned my addr 172.21.15.112:0/2719555220 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:51.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69cee8700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff698153650 0x7ff6980c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69deea700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff698153ad0 0x7ff6980c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69d6e9700 1 -- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff698153ad0 msgr2=0x7ff6980c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69d6e9700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff698153ad0 0x7ff6980c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69d6e9700 1 -- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff698153650 msgr2=0x7ff6980c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69d6e9700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff698153650 0x7ff6980c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.225+0000 7ff69d6e9700 1 -- 172.21.15.112:0/2719555220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698154660 con 0x7ff684004850 2023-08-27T11:17:51.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.226+0000 7ff69d6e9700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684004850 0x7ff6980c95d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff688001bb0 tx=0x7ff68800ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:51.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.226+0000 7ff68dffb700 1 -- 172.21.15.112:0/2719555220 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688010510 con 0x7ff684004850 2023-08-27T11:17:51.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.226+0000 7ff68dffb700 1 -- 172.21.15.112:0/2719555220 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff688010670 con 0x7ff684004850 2023-08-27T11:17:51.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.226+0000 7ff69f94d700 1 -- 172.21.15.112:0/2719555220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6980c73d0 con 0x7ff684004850 2023-08-27T11:17:51.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.226+0000 7ff68dffb700 1 -- 172.21.15.112:0/2719555220 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688010a20 con 0x7ff684004850 2023-08-27T11:17:51.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.226+0000 7ff69f94d700 1 -- 172.21.15.112:0/2719555220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6980ca220 con 0x7ff684004850 2023-08-27T11:17:51.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.227+0000 7ff69f94d700 1 -- 172.21.15.112:0/2719555220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6981544c0 con 0x7ff684004850 2023-08-27T11:17:51.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.227+0000 7ff68dffb700 1 -- 172.21.15.112:0/2719555220 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff688010b80 con 0x7ff684004850 2023-08-27T11:17:51.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.227+0000 7ff68dffb700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff67c06a0e0 0x7ff67c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.228+0000 7ff68dffb700 1 -- 172.21.15.112:0/2719555220 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff688087bd0 con 0x7ff684004850 2023-08-27T11:17:51.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.228+0000 7ff69cee8700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff67c06a0e0 0x7ff67c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.228+0000 7ff69cee8700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff67c06a0e0 0x7ff67c06c5a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff694016040 tx=0x7ff69401e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:51.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.232+0000 7ff68dffb700 1 -- 172.21.15.112:0/2719555220 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff6981544c0 con 0x7ff684004850 2023-08-27T11:17:51.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.421+0000 7ff69f94d700 1 -- 172.21.15.112:0/2719555220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6980c78f0 con 0x7ff684004850 2023-08-27T11:17:51.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.422+0000 7ff68dffb700 1 -- 172.21.15.112:0/2719555220 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18245 (secure 0 0 0) 0x7ff6980c78f0 con 0x7ff684004850 2023-08-27T11:17:51.424 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:51.424 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:06.357745+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:51.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.424+0000 7ff69f94d700 1 -- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff67c06a0e0 msgr2=0x7ff67c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.424+0000 7ff69f94d700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff67c06a0e0 0x7ff67c06c5a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff694016040 tx=0x7ff69401e040 comp rx=0 tx=0).stop 2023-08-27T11:17:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.424+0000 7ff69f94d700 1 -- 172.21.15.112:0/2719555220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684004850 msgr2=0x7ff6980c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.424+0000 7ff69f94d700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684004850 0x7ff6980c95d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff688001bb0 tx=0x7ff68800ea60 comp rx=0 tx=0).stop 2023-08-27T11:17:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.424+0000 7ff69f94d700 1 -- 172.21.15.112:0/2719555220 shutdown_connections 2023-08-27T11:17:51.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.424+0000 7ff69f94d700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff67c06a0e0 0x7ff67c06c5a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.424+0000 7ff69f94d700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff698153ad0 0x7ff6980c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.424+0000 7ff69f94d700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684004850 0x7ff6980c95d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.424+0000 7ff69f94d700 1 --2- 172.21.15.112:0/2719555220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff698153650 0x7ff6980c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.424+0000 7ff69f94d700 1 -- 172.21.15.112:0/2719555220 >> 172.21.15.112:0/2719555220 conn(0x7ff6980d0000 msgr2=0x7ff6980720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:51.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.424+0000 7ff69f94d700 1 -- 172.21.15.112:0/2719555220 shutdown_connections 2023-08-27T11:17:51.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.424+0000 7ff69f94d700 1 -- 172.21.15.112:0/2719555220 wait complete. 2023-08-27T11:17:51.438 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:17:51.438 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:17:51.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.556+0000 7f1527bed700 1 Processor -- start 2023-08-27T11:17:51.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.557+0000 7f1527bed700 1 -- start start 2023-08-27T11:17:51.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.557+0000 7f1527bed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152006e150 0x7f15200cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.557+0000 7f1527bed700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f152013f990 0x7f152006e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.557+0000 7f1527bed700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152006ea70 con 0x7f15200cd270 2023-08-27T11:17:51.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.557+0000 7f1527bed700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15200cecb0 con 0x7f152013f990 2023-08-27T11:17:51.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.557+0000 7f1527bed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15200cf570 con 0x7f152006e150 2023-08-27T11:17:51.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.557+0000 7f152618a700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f15200cd270 0x7f152015ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50796/0 (socket says 172.21.15.112:50796) 2023-08-27T11:17:51.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f152618a700 1 -- 172.21.15.112:0/2738653326 learned_addr learned my addr 172.21.15.112:0/2738653326 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:51.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f1525989700 1 --2- 172.21.15.112:0/2738653326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152006e150 0x7f15200cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f1525188700 1 --2- 172.21.15.112:0/2738653326 >> v2:172.21.15.138:3300/0 conn(0x7f152013f990 0x7f152006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f1524987700 1 -- 172.21.15.112:0/2738653326 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243509618 0 0) 0x7f152006ea70 con 0x7f15200cd270 2023-08-27T11:17:51.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f1524987700 1 -- 172.21.15.112:0/2738653326 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f150c0035b0 con 0x7f15200cd270 2023-08-27T11:17:51.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f1524987700 1 -- 172.21.15.112:0/2738653326 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2703273760 0 0) 0x7f150c0035b0 con 0x7f15200cd270 2023-08-27T11:17:51.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f1525989700 1 -- 172.21.15.112:0/2738653326 >> v2:172.21.15.138:3300/0 conn(0x7f152013f990 msgr2=0x7f152006e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:51.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f1525989700 1 --2- 172.21.15.112:0/2738653326 >> v2:172.21.15.138:3300/0 conn(0x7f152013f990 0x7f152006e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f1525989700 1 -- 172.21.15.112:0/2738653326 >> v1:172.21.15.112:6789/0 conn(0x7f15200cd270 legacy=0x7f152015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f1525989700 1 -- 172.21.15.112:0/2738653326 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152013ff50 con 0x7f152006e150 2023-08-27T11:17:51.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f1525989700 1 --2- 172.21.15.112:0/2738653326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152006e150 0x7f15200cf030 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1510009840 tx=0x7f1510009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f297473e09331ce5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:51.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f1524987700 1 -- 172.21.15.112:0/2738653326 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15180019f0 con 0x7f152006e150 2023-08-27T11:17:51.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f1524987700 1 -- 172.21.15.112:0/2738653326 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1510004030 con 0x7f152006e150 2023-08-27T11:17:51.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.558+0000 7f1524987700 1 -- 172.21.15.112:0/2738653326 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15100043c0 con 0x7f152006e150 2023-08-27T11:17:51.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.559+0000 7f1527bed700 1 -- 172.21.15.112:0/2738653326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152006e150 msgr2=0x7f15200cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.559+0000 7f1527bed700 1 --2- 172.21.15.112:0/2738653326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152006e150 0x7f15200cf030 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1510009840 tx=0x7f1510009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:51.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.559+0000 7f1527bed700 1 -- 172.21.15.112:0/2738653326 shutdown_connections 2023-08-27T11:17:51.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.559+0000 7f1527bed700 1 --2- 172.21.15.112:0/2738653326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152006e150 0x7f15200cf030 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.559+0000 7f1527bed700 1 --2- 172.21.15.112:0/2738653326 >> v2:172.21.15.138:3300/0 conn(0x7f152013f990 0x7f152006e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.559+0000 7f1527bed700 1 -- 172.21.15.112:0/2738653326 >> 172.21.15.112:0/2738653326 conn(0x7f152006cd30 msgr2=0x7f15200717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:51.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.559+0000 7f1527bed700 1 -- 172.21.15.112:0/2738653326 shutdown_connections 2023-08-27T11:17:51.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.559+0000 7f1527bed700 1 -- 172.21.15.112:0/2738653326 wait complete. 2023-08-27T11:17:51.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.560+0000 7f1527bed700 1 Processor -- start 2023-08-27T11:17:51.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.560+0000 7f1527bed700 1 -- start start 2023-08-27T11:17:51.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1527bed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f152006e150 0x7f15200b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1527bed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15200cd270 0x7f15200b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1527bed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152013f990 0x7f152013bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1527bed700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15201f2a20 con 0x7f15200cd270 2023-08-27T11:17:51.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1527bed700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15200b08e0 con 0x7f152006e150 2023-08-27T11:17:51.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1527bed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15201f3840 con 0x7f152013f990 2023-08-27T11:17:51.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1525188700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15200cd270 0x7f15200b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1525188700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15200cd270 0x7f15200b03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56418/0 (socket says 172.21.15.112:56418) 2023-08-27T11:17:51.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1525188700 1 -- 172.21.15.112:0/1611274346 learned_addr learned my addr 172.21.15.112:0/1611274346 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:51.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f152618a700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152013f990 0x7f152013bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1525989700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f152006e150 0x7f15200b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1525188700 1 -- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152013f990 msgr2=0x7f152013bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1525188700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152013f990 0x7f152013bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1525188700 1 -- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f152006e150 msgr2=0x7f15200b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1525188700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f152006e150 0x7f15200b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1525188700 1 -- 172.21.15.112:0/1611274346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15100094a0 con 0x7f15200cd270 2023-08-27T11:17:51.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1525989700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f152006e150 0x7f15200b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:51.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.561+0000 7f1525188700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15200cd270 0x7f15200b03a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f151c00e900 tx=0x7f151c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:51.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.562+0000 7f15167fc700 1 -- 172.21.15.112:0/1611274346 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c009650 con 0x7f15200cd270 2023-08-27T11:17:51.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.562+0000 7f15167fc700 1 -- 172.21.15.112:0/1611274346 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151c00ee50 con 0x7f15200cd270 2023-08-27T11:17:51.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.562+0000 7f1527bed700 1 -- 172.21.15.112:0/1611274346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15201f2c30 con 0x7f15200cd270 2023-08-27T11:17:51.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.562+0000 7f15167fc700 1 -- 172.21.15.112:0/1611274346 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c00ca50 con 0x7f15200cd270 2023-08-27T11:17:51.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.562+0000 7f1527bed700 1 -- 172.21.15.112:0/1611274346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15201f31f0 con 0x7f15200cd270 2023-08-27T11:17:51.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.562+0000 7f1527bed700 1 -- 172.21.15.112:0/1611274346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15201f6010 con 0x7f15200cd270 2023-08-27T11:17:51.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.563+0000 7f15167fc700 1 -- 172.21.15.112:0/1611274346 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f151c00cbb0 con 0x7f15200cd270 2023-08-27T11:17:51.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.563+0000 7f15167fc700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f150406a0e0 0x7f150406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.563+0000 7f15167fc700 1 -- 172.21.15.112:0/1611274346 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f151c088900 con 0x7f15200cd270 2023-08-27T11:17:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.564+0000 7f1525989700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f150406a0e0 0x7f150406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.564+0000 7f1525989700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f150406a0e0 0x7f150406c5a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15100090b0 tx=0x7f1510008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.568+0000 7f15167fc700 1 -- 172.21.15.112:0/1611274346 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f151c057a80 con 0x7f15200cd270 2023-08-27T11:17:51.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.798+0000 7f1527bed700 1 -- 172.21.15.112:0/1611274346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1520067a20 con 0x7f15200cd270 2023-08-27T11:17:51.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.798+0000 7f15167fc700 1 -- 172.21.15.112:0/1611274346 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4881 (secure 0 0 0) 0x7f151c017090 con 0x7f15200cd270 2023-08-27T11:17:51.800 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:51.800 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3289488908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3289488908},{"type":"v1","addr":"172.21.15.112:6835","nonce":3289488908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3991653721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3991653721},{"type":"v1","addr":"172.21.15.112:6834","nonce":3991653721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1923303457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1923303457},{"type":"v1","addr":"172.21.15.138:6836","nonce":1923303457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:17:04.761658+0000","modified":"2023-08-27T11:17:08.372245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/1827312637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1827312637},{"type":"v1","addr":"172.21.15.138:6837","nonce":1827312637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:17:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.800+0000 7f1527bed700 1 -- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f150406a0e0 msgr2=0x7f150406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.800+0000 7f1527bed700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f150406a0e0 0x7f150406c5a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15100090b0 tx=0x7f1510008030 comp rx=0 tx=0).stop 2023-08-27T11:17:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.800+0000 7f1527bed700 1 -- 172.21.15.112:0/1611274346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15200cd270 msgr2=0x7f15200b03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.800+0000 7f1527bed700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15200cd270 0x7f15200b03a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f151c00e900 tx=0x7f151c00ec10 comp rx=0 tx=0).stop 2023-08-27T11:17:51.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.800+0000 7f1527bed700 1 -- 172.21.15.112:0/1611274346 shutdown_connections 2023-08-27T11:17:51.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.800+0000 7f1527bed700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f150406a0e0 0x7f150406c5a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.800+0000 7f1527bed700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152013f990 0x7f152013bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.800+0000 7f1527bed700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f152006e150 0x7f15200b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.800+0000 7f1527bed700 1 --2- 172.21.15.112:0/1611274346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15200cd270 0x7f15200b03a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.800+0000 7f1527bed700 1 -- 172.21.15.112:0/1611274346 >> 172.21.15.112:0/1611274346 conn(0x7f152006cd30 msgr2=0x7f152013f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:51.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.801+0000 7f1527bed700 1 -- 172.21.15.112:0/1611274346 shutdown_connections 2023-08-27T11:17:51.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.801+0000 7f1527bed700 1 -- 172.21.15.112:0/1611274346 wait complete. 2023-08-27T11:17:51.805 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2023-08-27T11:17:51.814 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:17:51.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.932+0000 7ffad916b700 1 Processor -- start 2023-08-27T11:17:51.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.932+0000 7ffad916b700 1 -- start start 2023-08-27T11:17:51.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.933+0000 7ffad916b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffad40b1400 0x7ffad40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.933+0000 7ffad916b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffad40b17e0 0x7ffad406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.933+0000 7ffad916b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad4161850 con 0x7ffad4161ce0 2023-08-27T11:17:51.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.933+0000 7ffad916b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad406e7d0 con 0x7ffad40b17e0 2023-08-27T11:17:51.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.933+0000 7ffad916b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40cfc10 con 0x7ffad40b1400 2023-08-27T11:17:51.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.933+0000 7ffad2d9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffad40b1400 0x7ffad40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.933+0000 7ffad2d9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffad40b1400 0x7ffad40cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:39846/0 (socket says 172.21.15.112:39846) 2023-08-27T11:17:51.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.933+0000 7ffad2d9d700 1 -- 172.21.15.112:0/1343750453 learned_addr learned my addr 172.21.15.112:0/1343750453 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:51.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.933+0000 7ffad259c700 1 --2- 172.21.15.112:0/1343750453 >> v2:172.21.15.138:3300/0 conn(0x7ffad40b17e0 0x7ffad406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.933+0000 7ffad2d9d700 1 -- 172.21.15.112:0/1343750453 >> v2:172.21.15.138:3300/0 conn(0x7ffad40b17e0 msgr2=0x7ffad406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:51.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.933+0000 7ffad2d9d700 1 --2- 172.21.15.112:0/1343750453 >> v2:172.21.15.138:3300/0 conn(0x7ffad40b17e0 0x7ffad406e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.933+0000 7ffad2d9d700 1 -- 172.21.15.112:0/1343750453 >> v1:172.21.15.138:6789/0 conn(0x7ffad4161ce0 legacy=0x7ffad406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.933+0000 7ffad2d9d700 1 -- 172.21.15.112:0/1343750453 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad4161200 con 0x7ffad40b1400 2023-08-27T11:17:51.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.934+0000 7ffad2d9d700 1 --2- 172.21.15.112:0/1343750453 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffad40b1400 0x7ffad40cf5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffac40098f0 tx=0x7ffac4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30f86ed54888c4c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:51.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.934+0000 7ffad1d9b700 1 -- 172.21.15.112:0/1343750453 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4011830 con 0x7ffad40b1400 2023-08-27T11:17:51.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.934+0000 7ffad1d9b700 1 -- 172.21.15.112:0/1343750453 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac4011990 con 0x7ffad40b1400 2023-08-27T11:17:51.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.934+0000 7ffad1d9b700 1 -- 172.21.15.112:0/1343750453 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4011d20 con 0x7ffad40b1400 2023-08-27T11:17:51.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.934+0000 7ffad916b700 1 -- 172.21.15.112:0/1343750453 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffad40b1400 msgr2=0x7ffad40cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.934+0000 7ffad916b700 1 --2- 172.21.15.112:0/1343750453 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffad40b1400 0x7ffad40cf5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffac40098f0 tx=0x7ffac4009c00 comp rx=0 tx=0).stop 2023-08-27T11:17:51.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.934+0000 7ffad916b700 1 -- 172.21.15.112:0/1343750453 shutdown_connections 2023-08-27T11:17:51.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.934+0000 7ffad916b700 1 --2- 172.21.15.112:0/1343750453 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffad40b1400 0x7ffad40cf5a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.934+0000 7ffad916b700 1 --2- 172.21.15.112:0/1343750453 >> v2:172.21.15.138:3300/0 conn(0x7ffad40b17e0 0x7ffad406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.934+0000 7ffad916b700 1 -- 172.21.15.112:0/1343750453 >> 172.21.15.112:0/1343750453 conn(0x7ffad40d0000 msgr2=0x7ffad40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:51.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.934+0000 7ffad916b700 1 -- 172.21.15.112:0/1343750453 shutdown_connections 2023-08-27T11:17:51.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.934+0000 7ffad916b700 1 -- 172.21.15.112:0/1343750453 wait complete. 2023-08-27T11:17:51.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.936+0000 7ffad916b700 1 Processor -- start 2023-08-27T11:17:51.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.936+0000 7ffad916b700 1 -- start start 2023-08-27T11:17:51.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.936+0000 7ffad916b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad40b1400 0x7ffad41560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.936+0000 7ffad916b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad40b17e0 0x7ffad41f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.936+0000 7ffad916b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffad4161ce0 0x7ffad41f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.936+0000 7ffad916b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad41f6290 con 0x7ffad40b17e0 2023-08-27T11:17:51.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.936+0000 7ffad916b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40c7be0 con 0x7ffad40b1400 2023-08-27T11:17:51.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad916b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40c8480 con 0x7ffad4161ce0 2023-08-27T11:17:51.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad40b17e0 0x7ffad41f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad40b17e0 0x7ffad41f6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56432/0 (socket says 172.21.15.112:56432) 2023-08-27T11:17:51.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad259c700 1 -- 172.21.15.112:0/3173070756 learned_addr learned my addr 172.21.15.112:0/3173070756 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:51.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad2d9d700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad40b1400 0x7ffad41560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad359e700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffad4161ce0 0x7ffad41f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad259c700 1 -- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffad4161ce0 msgr2=0x7ffad41f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad259c700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffad4161ce0 0x7ffad41f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad259c700 1 -- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad40b1400 msgr2=0x7ffad41560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:51.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad259c700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad40b1400 0x7ffad41560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:51.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad259c700 1 -- 172.21.15.112:0/3173070756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac4009550 con 0x7ffad40b17e0 2023-08-27T11:17:51.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad259c700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad40b17e0 0x7ffad41f6450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e9b0 tx=0x7ffac800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:51.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffabb7fe700 1 -- 172.21.15.112:0/3173070756 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8009710 con 0x7ffad40b17e0 2023-08-27T11:17:51.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffabb7fe700 1 -- 172.21.15.112:0/3173070756 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac800ee50 con 0x7ffad40b17e0 2023-08-27T11:17:51.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad916b700 1 -- 172.21.15.112:0/3173070756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad41f7360 con 0x7ffad40b17e0 2023-08-27T11:17:51.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffabb7fe700 1 -- 172.21.15.112:0/3173070756 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac800ca50 con 0x7ffad40b17e0 2023-08-27T11:17:51.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.937+0000 7ffad916b700 1 -- 172.21.15.112:0/3173070756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad40ae940 con 0x7ffad40b17e0 2023-08-27T11:17:51.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.938+0000 7ffad916b700 1 -- 172.21.15.112:0/3173070756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad4067a20 con 0x7ffad40b17e0 2023-08-27T11:17:51.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.939+0000 7ffabb7fe700 1 -- 172.21.15.112:0/3173070756 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffac800cbb0 con 0x7ffad40b17e0 2023-08-27T11:17:51.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.939+0000 7ffabb7fe700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffac006a0e0 0x7ffac006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:51.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.939+0000 7ffabb7fe700 1 -- 172.21.15.112:0/3173070756 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffac80884c0 con 0x7ffad40b17e0 2023-08-27T11:17:51.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.939+0000 7ffad2d9d700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffac006a0e0 0x7ffac006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:51.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.940+0000 7ffad2d9d700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffac006a0e0 0x7ffac006c5a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009c40 tx=0x7ffac4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:51.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:51.943+0000 7ffabb7fe700 1 -- 172.21.15.112:0/3173070756 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffac80576d0 con 0x7ffad40b17e0 2023-08-27T11:17:52.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.173+0000 7ffad916b700 1 -- 172.21.15.112:0/3173070756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffad40d0470 con 0x7ffad40b17e0 2023-08-27T11:17:52.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.174+0000 7ffabb7fe700 1 -- 172.21.15.112:0/3173070756 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4881 (secure 0 0 0) 0x7ffac800ce60 con 0x7ffad40b17e0 2023-08-27T11:17:52.175 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:52.176 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3289488908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3289488908},{"type":"v1","addr":"172.21.15.112:6835","nonce":3289488908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3991653721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3991653721},{"type":"v1","addr":"172.21.15.112:6834","nonce":3991653721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1923303457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1923303457},{"type":"v1","addr":"172.21.15.138:6836","nonce":1923303457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:17:04.761658+0000","modified":"2023-08-27T11:17:08.372245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/1827312637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1827312637},{"type":"v1","addr":"172.21.15.138:6837","nonce":1827312637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:17:52.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.176+0000 7ffad916b700 1 -- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffac006a0e0 msgr2=0x7ffac006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:52.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.176+0000 7ffad916b700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffac006a0e0 0x7ffac006c5a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009c40 tx=0x7ffac4008030 comp rx=0 tx=0).stop 2023-08-27T11:17:52.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.176+0000 7ffad916b700 1 -- 172.21.15.112:0/3173070756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad40b17e0 msgr2=0x7ffad41f6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:52.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.176+0000 7ffad916b700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad40b17e0 0x7ffad41f6450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e9b0 tx=0x7ffac800ecc0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.176+0000 7ffad916b700 1 -- 172.21.15.112:0/3173070756 shutdown_connections 2023-08-27T11:17:52.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.176+0000 7ffad916b700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffac006a0e0 0x7ffac006c5a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.176+0000 7ffad916b700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffad4161ce0 0x7ffad41f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.176+0000 7ffad916b700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad40b1400 0x7ffad41560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.176+0000 7ffad916b700 1 --2- 172.21.15.112:0/3173070756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad40b17e0 0x7ffad41f6450 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.176+0000 7ffad916b700 1 -- 172.21.15.112:0/3173070756 >> 172.21.15.112:0/3173070756 conn(0x7ffad40d0000 msgr2=0x7ffad415ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:52.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.176+0000 7ffad916b700 1 -- 172.21.15.112:0/3173070756 shutdown_connections 2023-08-27T11:17:52.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.176+0000 7ffad916b700 1 -- 172.21.15.112:0/3173070756 wait complete. 2023-08-27T11:17:52.180 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2023-08-27T11:17:52.190 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:17:52.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.309+0000 7fc65bda6700 1 Processor -- start 2023-08-27T11:17:52.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.309+0000 7fc65bda6700 1 -- start start 2023-08-27T11:17:52.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc65bda6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc6541578e0 0x7fc6540b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:52.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc65bda6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc654160f90 0x7fc654161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:52.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc65bda6700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6540cfb40 con 0x7fc6540cf170 2023-08-27T11:17:52.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc65bda6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc654161980 con 0x7fc6541578e0 2023-08-27T11:17:52.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc65bda6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540cfde0 con 0x7fc654160f90 2023-08-27T11:17:52.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc659b42700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc6541578e0 0x7fc6540b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:52.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc659341700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc6540cf170 0x7fc6540d08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50820/0 (socket says 172.21.15.112:50820) 2023-08-27T11:17:52.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc659341700 1 -- 172.21.15.112:0/60205325 learned_addr learned my addr 172.21.15.112:0/60205325 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:52.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc65a343700 1 --2- 172.21.15.112:0/60205325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc654160f90 0x7fc654161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:52.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc659b42700 1 -- 172.21.15.112:0/60205325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc654160f90 msgr2=0x7fc654161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:52.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc659b42700 1 --2- 172.21.15.112:0/60205325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc654160f90 0x7fc654161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc659b42700 1 -- 172.21.15.112:0/60205325 >> v1:172.21.15.112:6789/0 conn(0x7fc6540cf170 legacy=0x7fc6540d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:52.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc658b40700 1 -- 172.21.15.112:0/60205325 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720825949 0 0) 0x7fc6540cfb40 con 0x7fc6540cf170 2023-08-27T11:17:52.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc659b42700 1 -- 172.21.15.112:0/60205325 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540d00c0 con 0x7fc6541578e0 2023-08-27T11:17:52.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.310+0000 7fc659b42700 1 --2- 172.21.15.112:0/60205325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6541578e0 0x7fc6540b0710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc650009840 tx=0x7fc650009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc82268737599a49 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:52.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.311+0000 7fc658b40700 1 -- 172.21.15.112:0/60205325 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650011830 con 0x7fc6541578e0 2023-08-27T11:17:52.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.311+0000 7fc658b40700 1 -- 172.21.15.112:0/60205325 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc650011990 con 0x7fc6541578e0 2023-08-27T11:17:52.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.311+0000 7fc658b40700 1 -- 172.21.15.112:0/60205325 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650011d20 con 0x7fc6541578e0 2023-08-27T11:17:52.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.311+0000 7fc65bda6700 1 -- 172.21.15.112:0/60205325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6541578e0 msgr2=0x7fc6540b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:52.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.311+0000 7fc65bda6700 1 --2- 172.21.15.112:0/60205325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6541578e0 0x7fc6540b0710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc650009840 tx=0x7fc650009b50 comp rx=0 tx=0).stop 2023-08-27T11:17:52.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.311+0000 7fc65bda6700 1 -- 172.21.15.112:0/60205325 shutdown_connections 2023-08-27T11:17:52.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.311+0000 7fc65bda6700 1 --2- 172.21.15.112:0/60205325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc654160f90 0x7fc654161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.311+0000 7fc65bda6700 1 --2- 172.21.15.112:0/60205325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6541578e0 0x7fc6540b0710 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.311+0000 7fc65bda6700 1 -- 172.21.15.112:0/60205325 >> 172.21.15.112:0/60205325 conn(0x7fc65406f970 msgr2=0x7fc65406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:52.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.311+0000 7fc65bda6700 1 -- 172.21.15.112:0/60205325 shutdown_connections 2023-08-27T11:17:52.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.311+0000 7fc65bda6700 1 -- 172.21.15.112:0/60205325 wait complete. 2023-08-27T11:17:52.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.312+0000 7fc65bda6700 1 Processor -- start 2023-08-27T11:17:52.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.312+0000 7fc65bda6700 1 -- start start 2023-08-27T11:17:52.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc65bda6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6540cf170 0x7fc6541f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:52.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc65bda6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6541578e0 0x7fc654160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:52.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc65bda6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc654160f90 0x7fc6540b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:52.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc65bda6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654148580 con 0x7fc6540cf170 2023-08-27T11:17:52.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc65bda6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654160c40 con 0x7fc654160f90 2023-08-27T11:17:52.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc65bda6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6541f5d20 con 0x7fc6541578e0 2023-08-27T11:17:52.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc659b42700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6540cf170 0x7fc6541f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:52.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc659b42700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6540cf170 0x7fc6541f2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56444/0 (socket says 172.21.15.112:56444) 2023-08-27T11:17:52.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc659b42700 1 -- 172.21.15.112:0/1037816671 learned_addr learned my addr 172.21.15.112:0/1037816671 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:52.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc659b42700 1 -- 172.21.15.112:0/1037816671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6541578e0 msgr2=0x7fc654160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:52.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc659b42700 1 --2- 172.21.15.112:0/1037816671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6541578e0 0x7fc654160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc659b42700 1 -- 172.21.15.112:0/1037816671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc654160f90 msgr2=0x7fc6540b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:52.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc659b42700 1 --2- 172.21.15.112:0/1037816671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc654160f90 0x7fc6540b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.313+0000 7fc659b42700 1 -- 172.21.15.112:0/1037816671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6500094a0 con 0x7fc6540cf170 2023-08-27T11:17:52.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.314+0000 7fc659b42700 1 --2- 172.21.15.112:0/1037816671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6540cf170 0x7fc6541f2250 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc6500090b0 tx=0x7fc650004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:52.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.314+0000 7fc64a7fc700 1 -- 172.21.15.112:0/1037816671 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6500036a0 con 0x7fc6540cf170 2023-08-27T11:17:52.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.314+0000 7fc64a7fc700 1 -- 172.21.15.112:0/1037816671 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65001f070 con 0x7fc6540cf170 2023-08-27T11:17:52.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.314+0000 7fc64a7fc700 1 -- 172.21.15.112:0/1037816671 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650003a60 con 0x7fc6540cf170 2023-08-27T11:17:52.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.314+0000 7fc65bda6700 1 -- 172.21.15.112:0/1037816671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6541f6000 con 0x7fc6540cf170 2023-08-27T11:17:52.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.314+0000 7fc65bda6700 1 -- 172.21.15.112:0/1037816671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6541f65c0 con 0x7fc6540cf170 2023-08-27T11:17:52.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.315+0000 7fc65bda6700 1 -- 172.21.15.112:0/1037816671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc654067a20 con 0x7fc6540cf170 2023-08-27T11:17:52.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.315+0000 7fc64a7fc700 1 -- 172.21.15.112:0/1037816671 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc650003bc0 con 0x7fc6540cf170 2023-08-27T11:17:52.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.315+0000 7fc64a7fc700 1 --2- 172.21.15.112:0/1037816671 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc63806a0e0 0x7fc63806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:52.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.315+0000 7fc64a7fc700 1 -- 172.21.15.112:0/1037816671 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc650021070 con 0x7fc6540cf170 2023-08-27T11:17:52.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.316+0000 7fc659341700 1 --2- 172.21.15.112:0/1037816671 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc63806a0e0 0x7fc63806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:52.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.316+0000 7fc659341700 1 --2- 172.21.15.112:0/1037816671 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc63806a0e0 0x7fc63806c5a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc6440011e0 tx=0x7fc6440079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:52.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.320+0000 7fc64a7fc700 1 -- 172.21.15.112:0/1037816671 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc6500555a0 con 0x7fc6540cf170 2023-08-27T11:17:52.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.518+0000 7fc65bda6700 1 -- 172.21.15.112:0/1037816671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc654072150 con 0x7fc6540cf170 2023-08-27T11:17:52.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.519+0000 7fc64a7fc700 1 -- 172.21.15.112:0/1037816671 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18341 (secure 0 0 0) 0x7fc65002b090 con 0x7fc6540cf170 2023-08-27T11:17:52.521 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:52.521 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:52.375443+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:17:02.484063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:17:03.722976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.21","mappings":[{"from":3,"to":2}]},{"pgid":"4.39","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:17:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.521+0000 7fc65bda6700 1 -- 172.21.15.112:0/1037816671 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc63806a0e0 msgr2=0x7fc63806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.521+0000 7fc65bda6700 1 --2- 172.21.15.112:0/1037816671 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc63806a0e0 0x7fc63806c5a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc6440011e0 tx=0x7fc6440079b0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.521+0000 7fc65bda6700 1 -- 172.21.15.112:0/1037816671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6540cf170 msgr2=0x7fc6541f2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.521+0000 7fc65bda6700 1 --2- 172.21.15.112:0/1037816671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6540cf170 0x7fc6541f2250 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc6500090b0 tx=0x7fc650004750 comp rx=0 tx=0).stop 2023-08-27T11:17:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.521+0000 7fc65bda6700 1 -- 172.21.15.112:0/1037816671 shutdown_connections 2023-08-27T11:17:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.521+0000 7fc65bda6700 1 --2- 172.21.15.112:0/1037816671 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc63806a0e0 0x7fc63806c5a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.521+0000 7fc65bda6700 1 --2- 172.21.15.112:0/1037816671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc654160f90 0x7fc6540b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.521+0000 7fc65bda6700 1 --2- 172.21.15.112:0/1037816671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6540cf170 0x7fc6541f2250 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.521+0000 7fc65bda6700 1 --2- 172.21.15.112:0/1037816671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6541578e0 0x7fc654160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.521+0000 7fc65bda6700 1 -- 172.21.15.112:0/1037816671 >> 172.21.15.112:0/1037816671 conn(0x7fc65406f970 msgr2=0x7fc6540cbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.521+0000 7fc65bda6700 1 -- 172.21.15.112:0/1037816671 shutdown_connections 2023-08-27T11:17:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.521+0000 7fc65bda6700 1 -- 172.21.15.112:0/1037816671 wait complete. 2023-08-27T11:17:52.535 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:17:52.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.655+0000 7f443d9c7700 1 Processor -- start 2023-08-27T11:17:52.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.655+0000 7f443d9c7700 1 -- start start 2023-08-27T11:17:52.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.656+0000 7f443d9c7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f44380b3e50 0x7f443815f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:52.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.656+0000 7f443d9c7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44380b4ad0 0x7f44380b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:52.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.656+0000 7f443d9c7700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44380b45f0 con 0x7f4438144570 2023-08-27T11:17:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.656+0000 7f443d9c7700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44380cf7a0 con 0x7f44380b3e50 2023-08-27T11:17:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.656+0000 7f443d9c7700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4438144950 con 0x7f44380b4ad0 2023-08-27T11:17:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.656+0000 7f44367fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f44380b3e50 0x7f443815f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.656+0000 7f44367fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f44380b3e50 0x7f443815f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56454/0 (socket says 172.21.15.112:56454) 2023-08-27T11:17:52.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.656+0000 7f44367fc700 1 -- 172.21.15.112:0/4273552304 learned_addr learned my addr 172.21.15.112:0/4273552304 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:52.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.657+0000 7f44377fe700 1 --2- 172.21.15.112:0/4273552304 >> v2:172.21.15.138:3300/0 conn(0x7f44380b4ad0 0x7f44380b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:52.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.657+0000 7f44367fc700 1 -- 172.21.15.112:0/4273552304 >> v2:172.21.15.138:3300/0 conn(0x7f44380b4ad0 msgr2=0x7f44380b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:52.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.657+0000 7f44367fc700 1 --2- 172.21.15.112:0/4273552304 >> v2:172.21.15.138:3300/0 conn(0x7f44380b4ad0 0x7f44380b4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.657+0000 7f44367fc700 1 -- 172.21.15.112:0/4273552304 >> v1:172.21.15.138:6789/0 conn(0x7f4438144570 legacy=0x7f443815ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:52.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.657+0000 7f44367fc700 1 -- 172.21.15.112:0/4273552304 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44380b4310 con 0x7f44380b3e50 2023-08-27T11:17:52.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.657+0000 7f44367fc700 1 --2- 172.21.15.112:0/4273552304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380b3e50 0x7f443815f530 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f443806b760 tx=0x7f4428009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6b1f1a9be43cd9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:52.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.657+0000 7f4435ffb700 1 -- 172.21.15.112:0/4273552304 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428004230 con 0x7f44380b3e50 2023-08-27T11:17:52.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.657+0000 7f4435ffb700 1 -- 172.21.15.112:0/4273552304 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4428004390 con 0x7f44380b3e50 2023-08-27T11:17:52.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.657+0000 7f4435ffb700 1 -- 172.21.15.112:0/4273552304 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44280036a0 con 0x7f44380b3e50 2023-08-27T11:17:52.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.658+0000 7f443d9c7700 1 -- 172.21.15.112:0/4273552304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380b3e50 msgr2=0x7f443815f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:52.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.658+0000 7f443d9c7700 1 --2- 172.21.15.112:0/4273552304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380b3e50 0x7f443815f530 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f443806b760 tx=0x7f4428009c00 comp rx=0 tx=0).stop 2023-08-27T11:17:52.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.658+0000 7f443d9c7700 1 -- 172.21.15.112:0/4273552304 shutdown_connections 2023-08-27T11:17:52.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.658+0000 7f443d9c7700 1 --2- 172.21.15.112:0/4273552304 >> v2:172.21.15.138:3300/0 conn(0x7f44380b4ad0 0x7f44380b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.658+0000 7f443d9c7700 1 --2- 172.21.15.112:0/4273552304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380b3e50 0x7f443815f530 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.658+0000 7f443d9c7700 1 -- 172.21.15.112:0/4273552304 >> 172.21.15.112:0/4273552304 conn(0x7f443806f970 msgr2=0x7f443806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:52.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.658+0000 7f443d9c7700 1 -- 172.21.15.112:0/4273552304 shutdown_connections 2023-08-27T11:17:52.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.658+0000 7f443d9c7700 1 -- 172.21.15.112:0/4273552304 wait complete. 2023-08-27T11:17:52.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.659+0000 7f443d9c7700 1 Processor -- start 2023-08-27T11:17:52.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.659+0000 7f443d9c7700 1 -- start start 2023-08-27T11:17:52.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f443d9c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380b3e50 0x7f443813d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:52.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f443d9c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44380b4ad0 0x7f44381408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:52.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f443d9c7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438144570 0x7f443815d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:52.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f443d9c7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44380c8de0 con 0x7f44380b3e50 2023-08-27T11:17:52.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f443d9c7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44380c81d0 con 0x7f44380b4ad0 2023-08-27T11:17:52.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f443d9c7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44380c84b0 con 0x7f4438144570 2023-08-27T11:17:52.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f4436ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380b3e50 0x7f443813d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:52.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f4436ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380b3e50 0x7f443813d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56458/0 (socket says 172.21.15.112:56458) 2023-08-27T11:17:52.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f4436ffd700 1 -- 172.21.15.112:0/530786561 learned_addr learned my addr 172.21.15.112:0/530786561 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f44367fc700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44380b4ad0 0x7f44381408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f44377fe700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438144570 0x7f443815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f4436ffd700 1 -- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438144570 msgr2=0x7f443815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f4436ffd700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438144570 0x7f443815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f4436ffd700 1 -- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44380b4ad0 msgr2=0x7f44381408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f4436ffd700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44380b4ad0 0x7f44381408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f4436ffd700 1 -- 172.21.15.112:0/530786561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4428009550 con 0x7f44380b3e50 2023-08-27T11:17:52.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f4436ffd700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380b3e50 0x7f443813d380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f442c00c4d0 tx=0x7f442c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:52.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f441ffff700 1 -- 172.21.15.112:0/530786561 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c011790 con 0x7f44380b3e50 2023-08-27T11:17:52.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f441ffff700 1 -- 172.21.15.112:0/530786561 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442c0118f0 con 0x7f44380b3e50 2023-08-27T11:17:52.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f443d9c7700 1 -- 172.21.15.112:0/530786561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44380c8790 con 0x7f44380b3e50 2023-08-27T11:17:52.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f441ffff700 1 -- 172.21.15.112:0/530786561 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c011ca0 con 0x7f44380b3e50 2023-08-27T11:17:52.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.660+0000 7f443d9c7700 1 -- 172.21.15.112:0/530786561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443813d8c0 con 0x7f44380b3e50 2023-08-27T11:17:52.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.661+0000 7f443d9c7700 1 -- 172.21.15.112:0/530786561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44380c8f50 con 0x7f44380b3e50 2023-08-27T11:17:52.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.662+0000 7f441ffff700 1 -- 172.21.15.112:0/530786561 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f442c01c460 con 0x7f44380b3e50 2023-08-27T11:17:52.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.662+0000 7f441ffff700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f442406a0e0 0x7f442406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:52.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.662+0000 7f441ffff700 1 -- 172.21.15.112:0/530786561 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f442c088410 con 0x7f44380b3e50 2023-08-27T11:17:52.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.662+0000 7f44367fc700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f442406a0e0 0x7f442406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:52.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.663+0000 7f44367fc700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f442406a0e0 0x7f442406c5a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4428009130 tx=0x7f4428008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:52.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.667+0000 7f441ffff700 1 -- 172.21.15.112:0/530786561 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f442c053ac0 con 0x7f44380b3e50 2023-08-27T11:17:52.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.900+0000 7f443d9c7700 1 -- 172.21.15.112:0/530786561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4438067a20 con 0x7f44380b3e50 2023-08-27T11:17:52.902 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:17:52.901+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:17:52.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.905+0000 7f441ffff700 1 -- 172.21.15.112:0/530786561 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f442c0570e0 con 0x7f44380b3e50 2023-08-27T11:17:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.907+0000 7f443d9c7700 1 -- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f442406a0e0 msgr2=0x7f442406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.907+0000 7f443d9c7700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f442406a0e0 0x7f442406c5a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4428009130 tx=0x7f4428008030 comp rx=0 tx=0).stop 2023-08-27T11:17:52.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.907+0000 7f443d9c7700 1 -- 172.21.15.112:0/530786561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380b3e50 msgr2=0x7f443813d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:52.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.907+0000 7f443d9c7700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380b3e50 0x7f443813d380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f442c00c4d0 tx=0x7f442c00c890 comp rx=0 tx=0).stop 2023-08-27T11:17:52.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.907+0000 7f443d9c7700 1 -- 172.21.15.112:0/530786561 shutdown_connections 2023-08-27T11:17:52.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.907+0000 7f443d9c7700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f442406a0e0 0x7f442406c5a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.907+0000 7f443d9c7700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438144570 0x7f443815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.907+0000 7f443d9c7700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380b3e50 0x7f443813d380 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.907+0000 7f443d9c7700 1 --2- 172.21.15.112:0/530786561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44380b4ad0 0x7f44381408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:52.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.907+0000 7f443d9c7700 1 -- 172.21.15.112:0/530786561 >> 172.21.15.112:0/530786561 conn(0x7f443806f970 msgr2=0x7f443806ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:52.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.907+0000 7f443d9c7700 1 -- 172.21.15.112:0/530786561 shutdown_connections 2023-08-27T11:17:52.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:52.907+0000 7f443d9c7700 1 -- 172.21.15.112:0/530786561 wait complete. 2023-08-27T11:17:52.912 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:17:52.920 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:17:52.946 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:17:53.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.042+0000 7f5ec1eaf700 1 Processor -- start 2023-08-27T11:17:53.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.042+0000 7f5ec1eaf700 1 -- start start 2023-08-27T11:17:53.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ec1eaf700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5ebc06fea0 0x7f5ebc0b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:53.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ec1eaf700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ebc0b51d0 0x7f5ebc0ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:53.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ec1eaf700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ebc0b59e0 con 0x7f5ebc143f20 2023-08-27T11:17:53.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ec1eaf700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ebc0b5cc0 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ec1eaf700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ebc0b5fa0 con 0x7f5ebc06fea0 2023-08-27T11:17:53.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ebb7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5ebc143f20 0x7f5ebc06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50848/0 (socket says 172.21.15.112:50848) 2023-08-27T11:17:53.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ebb7fe700 1 -- 172.21.15.112:0/2034094987 learned_addr learned my addr 172.21.15.112:0/2034094987 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:53.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ebbfff700 1 --2- 172.21.15.112:0/2034094987 >> v2:172.21.15.112:3300/0 conn(0x7f5ebc0b51d0 0x7f5ebc0ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:53.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ebaffd700 1 --2- 172.21.15.112:0/2034094987 >> v2:172.21.15.138:3300/0 conn(0x7f5ebc06fea0 0x7f5ebc0b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:53.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5eba7fc700 1 -- 172.21.15.112:0/2034094987 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100725518 0 0) 0x7f5ebc0b59e0 con 0x7f5ebc143f20 2023-08-27T11:17:53.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5eba7fc700 1 -- 172.21.15.112:0/2034094987 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ea80035b0 con 0x7f5ebc143f20 2023-08-27T11:17:53.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ebbfff700 1 -- 172.21.15.112:0/2034094987 >> v2:172.21.15.138:3300/0 conn(0x7f5ebc06fea0 msgr2=0x7f5ebc0b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:53.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ebbfff700 1 --2- 172.21.15.112:0/2034094987 >> v2:172.21.15.138:3300/0 conn(0x7f5ebc06fea0 0x7f5ebc0b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:53.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ebbfff700 1 -- 172.21.15.112:0/2034094987 >> v1:172.21.15.112:6789/0 conn(0x7f5ebc143f20 legacy=0x7f5ebc06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:53.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ebbfff700 1 -- 172.21.15.112:0/2034094987 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ebc0cdf10 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.043+0000 7f5ebbfff700 1 --2- 172.21.15.112:0/2034094987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ebc0b51d0 0x7f5ebc0ce710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc145180 tx=0x7f5eb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1a1bb8d1bc1c57 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:53.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.044+0000 7f5eba7fc700 1 -- 172.21.15.112:0/2034094987 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb400e070 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.044+0000 7f5eba7fc700 1 -- 172.21.15.112:0/2034094987 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb400bb20 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.044+0000 7f5eba7fc700 1 -- 172.21.15.112:0/2034094987 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4012430 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.044+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/2034094987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ebc0b51d0 msgr2=0x7f5ebc0ce710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:53.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.044+0000 7f5ec1eaf700 1 --2- 172.21.15.112:0/2034094987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ebc0b51d0 0x7f5ebc0ce710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc145180 tx=0x7f5eb400b330 comp rx=0 tx=0).stop 2023-08-27T11:17:53.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.044+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/2034094987 shutdown_connections 2023-08-27T11:17:53.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.044+0000 7f5ec1eaf700 1 --2- 172.21.15.112:0/2034094987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ebc0b51d0 0x7f5ebc0ce710 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:53.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.044+0000 7f5ec1eaf700 1 --2- 172.21.15.112:0/2034094987 >> v2:172.21.15.138:3300/0 conn(0x7f5ebc06fea0 0x7f5ebc0b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:53.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.044+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/2034094987 >> 172.21.15.112:0/2034094987 conn(0x7f5ebc06bbf0 msgr2=0x7f5ebc0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:53.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.044+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/2034094987 shutdown_connections 2023-08-27T11:17:53.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.044+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/2034094987 wait complete. 2023-08-27T11:17:53.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.045+0000 7f5ec1eaf700 1 Processor -- start 2023-08-27T11:17:53.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.046+0000 7f5ec1eaf700 1 -- start start 2023-08-27T11:17:53.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.046+0000 7f5ec1eaf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ebc06fea0 0x7f5ebc15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:53.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.046+0000 7f5ec1eaf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ebc0b51d0 0x7f5ebc13cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:53.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.046+0000 7f5ec1eaf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc143f20 0x7f5ebc1e1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:53.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.046+0000 7f5ec1eaf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ebc15b260 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.046+0000 7f5ec1eaf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ebc0cb5a0 con 0x7f5ebc06fea0 2023-08-27T11:17:53.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.046+0000 7f5ec1eaf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ebc0cb880 con 0x7f5ebc143f20 2023-08-27T11:17:53.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.046+0000 7f5ebaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ebc0b51d0 0x7f5ebc13cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:53.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.046+0000 7f5ebaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ebc0b51d0 0x7f5ebc13cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56472/0 (socket says 172.21.15.112:56472) 2023-08-27T11:17:53.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.046+0000 7f5ebaffd700 1 -- 172.21.15.112:0/1537122371 learned_addr learned my addr 172.21.15.112:0/1537122371 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.047+0000 7f5ebb7fe700 1 --2- 172.21.15.112:0/1537122371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ebc06fea0 0x7f5ebc15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.047+0000 7f5ebaffd700 1 -- 172.21.15.112:0/1537122371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc143f20 msgr2=0x7f5ebc1e1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.047+0000 7f5ebaffd700 1 --2- 172.21.15.112:0/1537122371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc143f20 0x7f5ebc1e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.047+0000 7f5ebaffd700 1 -- 172.21.15.112:0/1537122371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ebc06fea0 msgr2=0x7f5ebc15ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.047+0000 7f5ebaffd700 1 --2- 172.21.15.112:0/1537122371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ebc06fea0 0x7f5ebc15ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.047+0000 7f5ebaffd700 1 -- 172.21.15.112:0/1537122371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb4009a70 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.047+0000 7f5ebaffd700 1 --2- 172.21.15.112:0/1537122371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ebc0b51d0 0x7f5ebc13cf60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00e810 tx=0x7f5eac00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.047+0000 7f5eb3fff700 1 -- 172.21.15.112:0/1537122371 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac013610 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.047+0000 7f5eb3fff700 1 -- 172.21.15.112:0/1537122371 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eac013770 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.047+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/1537122371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ebc0cbb60 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.047+0000 7f5eb3fff700 1 -- 172.21.15.112:0/1537122371 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac013b00 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.047+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/1537122371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ebc1f3070 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.048+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/1537122371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ebc066fa0 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.048+0000 7f5eb3fff700 1 -- 172.21.15.112:0/1537122371 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5eac013c60 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.049+0000 7f5eb3fff700 1 --2- 172.21.15.112:0/1537122371 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5e9c06a0e0 0x7f5e9c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.049+0000 7f5eb3fff700 1 -- 172.21.15.112:0/1537122371 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f5eac0883b0 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.049+0000 7f5ebb7fe700 1 --2- 172.21.15.112:0/1537122371 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5e9c06a0e0 0x7f5e9c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.049+0000 7f5ebb7fe700 1 --2- 172.21.15.112:0/1537122371 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5e9c06a0e0 0x7f5e9c06c5a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40011e0 tx=0x7f5ea4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.053+0000 7f5eb3fff700 1 -- 172.21.15.112:0/1537122371 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5eac053980 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.290+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/1537122371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5ebc067a20 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.905+0000 7f5eb3fff700 1 -- 172.21.15.112:0/1537122371 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5eac056fa0 con 0x7f5ebc0b51d0 2023-08-27T11:17:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.908+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/1537122371 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5e9c06a0e0 msgr2=0x7f5e9c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.908+0000 7f5ec1eaf700 1 --2- 172.21.15.112:0/1537122371 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5e9c06a0e0 0x7f5e9c06c5a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40011e0 tx=0x7f5ea4009040 comp rx=0 tx=0).stop 2023-08-27T11:17:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.908+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/1537122371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ebc0b51d0 msgr2=0x7f5ebc13cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.908+0000 7f5ec1eaf700 1 --2- 172.21.15.112:0/1537122371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ebc0b51d0 0x7f5ebc13cf60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00e810 tx=0x7f5eac00ebd0 comp rx=0 tx=0).stop 2023-08-27T11:17:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.908+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/1537122371 shutdown_connections 2023-08-27T11:17:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.908+0000 7f5ec1eaf700 1 --2- 172.21.15.112:0/1537122371 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5e9c06a0e0 0x7f5e9c06c5a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.908+0000 7f5ec1eaf700 1 --2- 172.21.15.112:0/1537122371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc143f20 0x7f5ebc1e1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.908+0000 7f5ec1eaf700 1 --2- 172.21.15.112:0/1537122371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ebc06fea0 0x7f5ebc15ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.908+0000 7f5ec1eaf700 1 --2- 172.21.15.112:0/1537122371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ebc0b51d0 0x7f5ebc13cf60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.908+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/1537122371 >> 172.21.15.112:0/1537122371 conn(0x7f5ebc06bbf0 msgr2=0x7f5ebc0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.908+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/1537122371 shutdown_connections 2023-08-27T11:17:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:53.908+0000 7f5ec1eaf700 1 -- 172.21.15.112:0/1537122371 wait complete. 2023-08-27T11:17:53.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:17:54.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.047+0000 7f5a6b95d700 1 Processor -- start 2023-08-27T11:17:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.047+0000 7f5a6b95d700 1 -- start start 2023-08-27T11:17:54.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.048+0000 7f5a6b95d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5a64157ae0 0x7f5a6415e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:54.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.048+0000 7f5a6b95d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a64161390 con 0x7f5a640b3de0 2023-08-27T11:17:54.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.048+0000 7f5a6b95d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a640b35f0 con 0x7f5a6406f790 2023-08-27T11:17:54.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.048+0000 7f5a6b95d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640b1aa0 con 0x7f5a64157ae0 2023-08-27T11:17:54.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.048+0000 7f5a68ef8700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5a640b3de0 0x7f5a64161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50862/0 (socket says 172.21.15.112:50862) 2023-08-27T11:17:54.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.048+0000 7f5a68ef8700 1 -- 172.21.15.112:0/1864280235 learned_addr learned my addr 172.21.15.112:0/1864280235 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:54.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.048+0000 7f5a696f9700 1 --2- 172.21.15.112:0/1864280235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5a64157ae0 0x7f5a6415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:54.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.048+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648648971 0 0) 0x7f5a64161390 con 0x7f5a640b3de0 2023-08-27T11:17:54.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.048+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a500035b0 con 0x7f5a640b3de0 2023-08-27T11:17:54.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.048+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1548262708 0 0) 0x7f5a500035b0 con 0x7f5a640b3de0 2023-08-27T11:17:54.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.048+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5a64157ae0 msgr2=0x7f5a6415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:54.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.048+0000 7f5a5bfff700 1 --2- 172.21.15.112:0/1864280235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5a64157ae0 0x7f5a6415e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:54.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.048+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 >> v1:172.21.15.138:6789/0 conn(0x7f5a6406f790 legacy=0x7f5a640b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a640b4330 con 0x7f5a640b3de0 2023-08-27T11:17:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f5a54001670 con 0x7f5a640b3de0 2023-08-27T11:17:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 >> v1:172.21.15.112:6789/0 conn(0x7f5a640b3de0 legacy=0x7f5a64161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a5bfff700 1 --2- 172.21.15.112:0/1864280235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a50003a30 0x7f5a50003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a5bfff700 1 --2- 172.21.15.112:0/1864280235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5a64157ae0 0x7f5a50004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a5bfff700 1 --2- 172.21.15.112:0/1864280235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a50004d90 0x7f5a50005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640b1aa0 con 0x7f5a50003a30 2023-08-27T11:17:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5000ba10 con 0x7f5a50004d90 2023-08-27T11:17:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5000bbd0 con 0x7f5a64157ae0 2023-08-27T11:17:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a696f9700 1 --2- 172.21.15.112:0/1864280235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a50003a30 0x7f5a50003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a68ef8700 1 --2- 172.21.15.112:0/1864280235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5a64157ae0 0x7f5a50004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a69efa700 1 --2- 172.21.15.112:0/1864280235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a50004d90 0x7f5a50005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a696f9700 1 -- 172.21.15.112:0/1864280235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5a64157ae0 msgr2=0x7f5a50004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a696f9700 1 --2- 172.21.15.112:0/1864280235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5a64157ae0 0x7f5a50004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a696f9700 1 -- 172.21.15.112:0/1864280235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a50004d90 msgr2=0x7f5a50005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:54.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a696f9700 1 --2- 172.21.15.112:0/1864280235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a50004d90 0x7f5a50005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:54.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a696f9700 1 -- 172.21.15.112:0/1864280235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a500035b0 con 0x7f5a50003a30 2023-08-27T11:17:54.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a69efa700 1 --2- 172.21.15.112:0/1864280235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a50004d90 0x7f5a50005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:54.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a696f9700 1 --2- 172.21.15.112:0/1864280235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a50003a30 0x7f5a50003e10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009b60 tx=0x7f5a50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54589b13b930d10 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:54.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000e6c0 con 0x7f5a50003a30 2023-08-27T11:17:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6000e820 con 0x7f5a50003a30 2023-08-27T11:17:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.049+0000 7f5a5bfff700 1 -- 172.21.15.112:0/1864280235 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000eb90 con 0x7f5a50003a30 2023-08-27T11:17:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.050+0000 7f5a6b95d700 1 -- 172.21.15.112:0/1864280235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a50003a30 msgr2=0x7f5a50003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.050+0000 7f5a6b95d700 1 --2- 172.21.15.112:0/1864280235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a50003a30 0x7f5a50003e10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009b60 tx=0x7f5a50005880 comp rx=0 tx=0).stop 2023-08-27T11:17:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.050+0000 7f5a69efa700 1 -- 172.21.15.112:0/1864280235 reap_dead start 2023-08-27T11:17:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.050+0000 7f5a6b95d700 1 -- 172.21.15.112:0/1864280235 shutdown_connections 2023-08-27T11:17:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.050+0000 7f5a6b95d700 1 -- 172.21.15.112:0/1864280235 >> 172.21.15.112:0/1864280235 conn(0x7f5a640b1be0 msgr2=0x7f5a640b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.050+0000 7f5a6b95d700 1 -- 172.21.15.112:0/1864280235 shutdown_connections 2023-08-27T11:17:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.050+0000 7f5a6b95d700 1 -- 172.21.15.112:0/1864280235 wait complete. 2023-08-27T11:17:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.052+0000 7f5a6b95d700 1 Processor -- start 2023-08-27T11:17:54.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.052+0000 7f5a6b95d700 1 -- start start 2023-08-27T11:17:54.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.052+0000 7f5a6b95d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a50003a30 0x7f5a640cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:54.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.052+0000 7f5a6b95d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5a6415d830 0x7f5a64141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:54.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.052+0000 7f5a6b95d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a640ad540 0x7f5a640add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:54.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a6b95d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6415e9e0 con 0x7f5a50003a30 2023-08-27T11:17:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a6b95d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a64141f40 con 0x7f5a640ad540 2023-08-27T11:17:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a6b95d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6415ddd0 con 0x7f5a6415d830 2023-08-27T11:17:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a696f9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a50003a30 0x7f5a640cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a696f9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a50003a30 0x7f5a640cce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56490/0 (socket says 172.21.15.112:56490) 2023-08-27T11:17:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a696f9700 1 -- 172.21.15.112:0/2934442359 learned_addr learned my addr 172.21.15.112:0/2934442359 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a68ef8700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5a6415d830 0x7f5a64141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a69efa700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a640ad540 0x7f5a640add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a696f9700 1 -- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5a6415d830 msgr2=0x7f5a64141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a696f9700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5a6415d830 0x7f5a64141a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a696f9700 1 -- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a640ad540 msgr2=0x7f5a640add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:54.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a696f9700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a640ad540 0x7f5a640add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:54.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a696f9700 1 -- 172.21.15.112:0/2934442359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6415e0b0 con 0x7f5a50003a30 2023-08-27T11:17:54.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a696f9700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a50003a30 0x7f5a640cce20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5a50005830 tx=0x7f5a600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:54.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a59ffb700 1 -- 172.21.15.112:0/2934442359 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60021070 con 0x7f5a50003a30 2023-08-27T11:17:54.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a59ffb700 1 -- 172.21.15.112:0/2934442359 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a60009ed0 con 0x7f5a50003a30 2023-08-27T11:17:54.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a6b95d700 1 -- 172.21.15.112:0/2934442359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6415e390 con 0x7f5a50003a30 2023-08-27T11:17:54.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a59ffb700 1 -- 172.21.15.112:0/2934442359 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000e6c0 con 0x7f5a50003a30 2023-08-27T11:17:54.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.053+0000 7f5a6b95d700 1 -- 172.21.15.112:0/2934442359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a640ad9c0 con 0x7f5a50003a30 2023-08-27T11:17:54.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.054+0000 7f5a6b95d700 1 -- 172.21.15.112:0/2934442359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a640adb80 con 0x7f5a50003a30 2023-08-27T11:17:54.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.055+0000 7f5a59ffb700 1 -- 172.21.15.112:0/2934442359 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5a60025070 con 0x7f5a50003a30 2023-08-27T11:17:54.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.055+0000 7f5a59ffb700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5a4806a0e0 0x7f5a4806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:54.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.055+0000 7f5a59ffb700 1 -- 172.21.15.112:0/2934442359 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f5a60089670 con 0x7f5a50003a30 2023-08-27T11:17:54.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.055+0000 7f5a68ef8700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5a4806a0e0 0x7f5a4806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:54.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.056+0000 7f5a68ef8700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5a4806a0e0 0x7f5a4806c5a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a54001050 tx=0x7f5a54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:54.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.059+0000 7f5a59ffb700 1 -- 172.21.15.112:0/2934442359 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5a60054e80 con 0x7f5a50003a30 2023-08-27T11:17:54.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.249+0000 7f5a6b95d700 1 -- 172.21.15.112:0/2934442359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5a6415ecc0 con 0x7f5a50003a30 2023-08-27T11:17:54.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.910+0000 7f5a59ffb700 1 -- 172.21.15.112:0/2934442359 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5a600584a0 con 0x7f5a50003a30 2023-08-27T11:17:54.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.913+0000 7f5a6b95d700 1 -- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5a4806a0e0 msgr2=0x7f5a4806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:54.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.913+0000 7f5a6b95d700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5a4806a0e0 0x7f5a4806c5a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a54001050 tx=0x7f5a54009040 comp rx=0 tx=0).stop 2023-08-27T11:17:54.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.913+0000 7f5a6b95d700 1 -- 172.21.15.112:0/2934442359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a50003a30 msgr2=0x7f5a640cce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:54.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.913+0000 7f5a6b95d700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a50003a30 0x7f5a640cce20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5a50005830 tx=0x7f5a600047c0 comp rx=0 tx=0).stop 2023-08-27T11:17:54.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.913+0000 7f5a6b95d700 1 -- 172.21.15.112:0/2934442359 shutdown_connections 2023-08-27T11:17:54.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.913+0000 7f5a6b95d700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5a4806a0e0 0x7f5a4806c5a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:54.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.913+0000 7f5a6b95d700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a640ad540 0x7f5a640add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:54.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.913+0000 7f5a6b95d700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a50003a30 0x7f5a640cce20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:54.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.913+0000 7f5a6b95d700 1 --2- 172.21.15.112:0/2934442359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5a6415d830 0x7f5a64141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:54.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.913+0000 7f5a6b95d700 1 -- 172.21.15.112:0/2934442359 >> 172.21.15.112:0/2934442359 conn(0x7f5a640b1be0 msgr2=0x7f5a640b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:54.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.913+0000 7f5a6b95d700 1 -- 172.21.15.112:0/2934442359 shutdown_connections 2023-08-27T11:17:54.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:54.913+0000 7f5a6b95d700 1 -- 172.21.15.112:0/2934442359 wait complete. 2023-08-27T11:17:54.918 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:17:54.928 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:17:55.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.053+0000 7fbdad039700 1 Processor -- start 2023-08-27T11:17:55.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.053+0000 7fbdad039700 1 -- start start 2023-08-27T11:17:55.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.054+0000 7fbdad039700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbda80b3de0 0x7fbda815e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:55.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.054+0000 7fbdad039700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbda806f790 0x7fbda806fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:55.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.054+0000 7fbdad039700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda8161390 con 0x7fbda8157ae0 2023-08-27T11:17:55.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.054+0000 7fbdad039700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda80b1aa0 con 0x7fbda80b3de0 2023-08-27T11:17:55.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.054+0000 7fbdad039700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda8070180 con 0x7fbda806f790 2023-08-27T11:17:55.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.054+0000 7fbda659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbda80b3de0 0x7fbda815e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:55.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.054+0000 7fbda659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbda80b3de0 0x7fbda815e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56500/0 (socket says 172.21.15.112:56500) 2023-08-27T11:17:55.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.054+0000 7fbda659c700 1 -- 172.21.15.112:0/1394900564 learned_addr learned my addr 172.21.15.112:0/1394900564 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:55.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.055+0000 7fbda659c700 1 -- 172.21.15.112:0/1394900564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbda806f790 msgr2=0x7fbda806fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:55.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.055+0000 7fbda659c700 1 --2- 172.21.15.112:0/1394900564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbda806f790 0x7fbda806fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:55.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.055+0000 7fbda659c700 1 -- 172.21.15.112:0/1394900564 >> v1:172.21.15.138:6789/0 conn(0x7fbda8157ae0 legacy=0x7fbda8161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:55.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.055+0000 7fbda659c700 1 -- 172.21.15.112:0/1394900564 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda80b35f0 con 0x7fbda80b3de0 2023-08-27T11:17:55.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.055+0000 7fbda659c700 1 --2- 172.21.15.112:0/1394900564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbda80b3de0 0x7fbda815e580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbd900098f0 tx=0x7fbd90009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76686b77a92f3a64 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:55.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.055+0000 7fbda5d9b700 1 -- 172.21.15.112:0/1394900564 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90004230 con 0x7fbda80b3de0 2023-08-27T11:17:55.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.055+0000 7fbda5d9b700 1 -- 172.21.15.112:0/1394900564 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd90004390 con 0x7fbda80b3de0 2023-08-27T11:17:55.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.055+0000 7fbda5d9b700 1 -- 172.21.15.112:0/1394900564 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd900036a0 con 0x7fbda80b3de0 2023-08-27T11:17:55.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.055+0000 7fbdad039700 1 -- 172.21.15.112:0/1394900564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbda80b3de0 msgr2=0x7fbda815e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:55.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.055+0000 7fbdad039700 1 --2- 172.21.15.112:0/1394900564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbda80b3de0 0x7fbda815e580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbd900098f0 tx=0x7fbd90009c00 comp rx=0 tx=0).stop 2023-08-27T11:17:55.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.055+0000 7fbdad039700 1 -- 172.21.15.112:0/1394900564 shutdown_connections 2023-08-27T11:17:55.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.055+0000 7fbdad039700 1 --2- 172.21.15.112:0/1394900564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbda806f790 0x7fbda806fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:55.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.055+0000 7fbdad039700 1 --2- 172.21.15.112:0/1394900564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbda80b3de0 0x7fbda815e580 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:55.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.056+0000 7fbdad039700 1 -- 172.21.15.112:0/1394900564 >> 172.21.15.112:0/1394900564 conn(0x7fbda80b1be0 msgr2=0x7fbda80b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:55.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.056+0000 7fbdad039700 1 -- 172.21.15.112:0/1394900564 shutdown_connections 2023-08-27T11:17:55.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.056+0000 7fbdad039700 1 -- 172.21.15.112:0/1394900564 wait complete. 2023-08-27T11:17:55.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.057+0000 7fbdad039700 1 Processor -- start 2023-08-27T11:17:55.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.057+0000 7fbdad039700 1 -- start start 2023-08-27T11:17:55.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbdad039700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbda806f790 0x7fbda8160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:55.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbdad039700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda80b3de0 0x7fbda80adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:55.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbdad039700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbda8157ae0 0x7fbda8158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:55.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbdad039700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda8160010 con 0x7fbda806f790 2023-08-27T11:17:55.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbdad039700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda8140300 con 0x7fbda80b3de0 2023-08-27T11:17:55.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbdad039700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda815f390 con 0x7fbda8157ae0 2023-08-27T11:17:55.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbda6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbda806f790 0x7fbda8160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:55.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbda6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbda806f790 0x7fbda8160430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56504/0 (socket says 172.21.15.112:56504) 2023-08-27T11:17:55.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbda6d9d700 1 -- 172.21.15.112:0/4199293007 learned_addr learned my addr 172.21.15.112:0/4199293007 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:55.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbda759e700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbda8157ae0 0x7fbda8158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:55.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbda659c700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda80b3de0 0x7fbda80adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:55.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbda6d9d700 1 -- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbda8157ae0 msgr2=0x7fbda8158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:55.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbda6d9d700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbda8157ae0 0x7fbda8158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:55.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbda6d9d700 1 -- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda80b3de0 msgr2=0x7fbda80adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:55.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbda6d9d700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda80b3de0 0x7fbda80adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:55.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbda6d9d700 1 -- 172.21.15.112:0/4199293007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd90009550 con 0x7fbda806f790 2023-08-27T11:17:55.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbda759e700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbda8157ae0 0x7fbda8158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:55.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbda659c700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda80b3de0 0x7fbda80adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:55.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbda6d9d700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbda806f790 0x7fbda8160430 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800b470 tx=0x7fbd9800b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:55.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.058+0000 7fbd9f7fe700 1 -- 172.21.15.112:0/4199293007 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98011830 con 0x7fbda806f790 2023-08-27T11:17:55.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.059+0000 7fbd9f7fe700 1 -- 172.21.15.112:0/4199293007 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd98011990 con 0x7fbda806f790 2023-08-27T11:17:55.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.059+0000 7fbd9f7fe700 1 -- 172.21.15.112:0/4199293007 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98011d40 con 0x7fbda806f790 2023-08-27T11:17:55.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.059+0000 7fbdad039700 1 -- 172.21.15.112:0/4199293007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda815f670 con 0x7fbda806f790 2023-08-27T11:17:55.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.059+0000 7fbdad039700 1 -- 172.21.15.112:0/4199293007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda815fc30 con 0x7fbda806f790 2023-08-27T11:17:55.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.059+0000 7fbdad039700 1 -- 172.21.15.112:0/4199293007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda8160150 con 0x7fbda806f790 2023-08-27T11:17:55.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.060+0000 7fbd9f7fe700 1 -- 172.21.15.112:0/4199293007 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbd9801a400 con 0x7fbda806f790 2023-08-27T11:17:55.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.060+0000 7fbd9f7fe700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd8806a0e0 0x7fbd8806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:55.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.060+0000 7fbd9f7fe700 1 -- 172.21.15.112:0/4199293007 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5768+0+0 (secure 0 0 0) 0x7fbd98088470 con 0x7fbda806f790 2023-08-27T11:17:55.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.061+0000 7fbda659c700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd8806a0e0 0x7fbd8806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:55.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.061+0000 7fbda659c700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd8806a0e0 0x7fbd8806c5a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbda8159970 tx=0x7fbd90008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:55.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.065+0000 7fbd9f7fe700 1 -- 172.21.15.112:0/4199293007 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbda8160150 con 0x7fbda806f790 2023-08-27T11:17:55.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.255+0000 7fbdad039700 1 -- 172.21.15.112:0/4199293007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbda8160150 con 0x7fbda806f790 2023-08-27T11:17:55.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.912+0000 7fbd9f7fe700 1 -- 172.21.15.112:0/4199293007 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbd98053d10 con 0x7fbda806f790 2023-08-27T11:17:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.915+0000 7fbdad039700 1 -- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd8806a0e0 msgr2=0x7fbd8806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.915+0000 7fbdad039700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd8806a0e0 0x7fbd8806c5a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbda8159970 tx=0x7fbd90008030 comp rx=0 tx=0).stop 2023-08-27T11:17:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.915+0000 7fbdad039700 1 -- 172.21.15.112:0/4199293007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbda806f790 msgr2=0x7fbda8160430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.915+0000 7fbdad039700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbda806f790 0x7fbda8160430 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800b470 tx=0x7fbd9800b830 comp rx=0 tx=0).stop 2023-08-27T11:17:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.915+0000 7fbdad039700 1 -- 172.21.15.112:0/4199293007 shutdown_connections 2023-08-27T11:17:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.915+0000 7fbdad039700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd8806a0e0 0x7fbd8806c5a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:55.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.915+0000 7fbdad039700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbda8157ae0 0x7fbda8158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:55.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.915+0000 7fbdad039700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbda806f790 0x7fbda8160430 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:55.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.915+0000 7fbdad039700 1 --2- 172.21.15.112:0/4199293007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda80b3de0 0x7fbda80adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:55.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.915+0000 7fbdad039700 1 -- 172.21.15.112:0/4199293007 >> 172.21.15.112:0/4199293007 conn(0x7fbda80b1be0 msgr2=0x7fbda815b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:55.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.915+0000 7fbdad039700 1 -- 172.21.15.112:0/4199293007 shutdown_connections 2023-08-27T11:17:55.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:55.915+0000 7fbdad039700 1 -- 172.21.15.112:0/4199293007 wait complete. 2023-08-27T11:17:55.919 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2023-08-27T11:17:55.930 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:17:55.930 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:17:55.931 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:17:55.931 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:17:55.931 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:17:55.932 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:17:55.932 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:17:55.932 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:17:55.933 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:17:55.933 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:17:55.934 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:17:55.934 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:17:55.935 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:17:55.935 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:17:55.935 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:17:55.936 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:17:55.940 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:17:55.940 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:17:55.942 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:17:55.942 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:17:55.948 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:17:55.948 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:17:55.949 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:17:55.950 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:17:55.984 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:17:55.985 DEBUG:teuthology.parallel:result is None 2023-08-27T11:17:55.986 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:17:55.987 DEBUG:teuthology.parallel:result is None 2023-08-27T11:17:55.990 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:17:55.991 DEBUG:teuthology.parallel:result is None 2023-08-27T11:17:55.991 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:17:55.993 DEBUG:teuthology.parallel:result is None 2023-08-27T11:17:55.993 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:17:56.063 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:17:56.063 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:17:56.075 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:17:56.075 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2023-08-27T11:17:56.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.119+0000 7f3c5f19e700 1 Processor -- start 2023-08-27T11:17:56.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.119+0000 7f3c5f19e700 1 -- start start 2023-08-27T11:17:56.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.120+0000 7f3c5f19e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3c580b1400 0x7f3c580cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:56.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.120+0000 7f3c5f19e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3c580b17e0 0x7f3c5806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:56.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.120+0000 7f3c5f19e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c58161850 con 0x7f3c58161ce0 2023-08-27T11:17:56.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.120+0000 7f3c5f19e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c5806e7d0 con 0x7f3c580b1400 2023-08-27T11:17:56.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.120+0000 7f3c5f19e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c580cfc10 con 0x7f3c580b17e0 2023-08-27T11:17:56.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.120+0000 7f3c5cf3a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3c580b1400 0x7f3c580cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:56.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.120+0000 7f3c5d73b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3c58161ce0 0x7f3c5806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50918/0 (socket says 172.21.15.112:50918) 2023-08-27T11:17:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.120+0000 7f3c5d73b700 1 -- 172.21.15.112:0/2952425259 learned_addr learned my addr 172.21.15.112:0/2952425259 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.120+0000 7f3c57fff700 1 --2- 172.21.15.112:0/2952425259 >> v2:172.21.15.138:3300/0 conn(0x7f3c580b17e0 0x7f3c5806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c5cf3a700 1 -- 172.21.15.112:0/2952425259 >> v2:172.21.15.138:3300/0 conn(0x7f3c580b17e0 msgr2=0x7f3c5806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c5cf3a700 1 --2- 172.21.15.112:0/2952425259 >> v2:172.21.15.138:3300/0 conn(0x7f3c580b17e0 0x7f3c5806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c5cf3a700 1 -- 172.21.15.112:0/2952425259 >> v1:172.21.15.112:6789/0 conn(0x7f3c58161ce0 legacy=0x7f3c5806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c5cf3a700 1 -- 172.21.15.112:0/2952425259 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c58161200 con 0x7f3c580b1400 2023-08-27T11:17:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c577fe700 1 -- 172.21.15.112:0/2952425259 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874447447 0 0) 0x7f3c58161850 con 0x7f3c58161ce0 2023-08-27T11:17:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c57fff700 1 --2- 172.21.15.112:0/2952425259 >> v2:172.21.15.138:3300/0 conn(0x7f3c580b17e0 0x7f3c5806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c5cf3a700 1 --2- 172.21.15.112:0/2952425259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c580b1400 0x7f3c580cf5a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3c580cc420 tx=0x7f3c5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8b3948929051ae server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c577fe700 1 -- 172.21.15.112:0/2952425259 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5000e070 con 0x7f3c580b1400 2023-08-27T11:17:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c577fe700 1 -- 172.21.15.112:0/2952425259 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5000bbd0 con 0x7f3c580b1400 2023-08-27T11:17:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c577fe700 1 -- 172.21.15.112:0/2952425259 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50012430 con 0x7f3c580b1400 2023-08-27T11:17:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c5f19e700 1 -- 172.21.15.112:0/2952425259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c580b1400 msgr2=0x7f3c580cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c5f19e700 1 --2- 172.21.15.112:0/2952425259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c580b1400 0x7f3c580cf5a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3c580cc420 tx=0x7f3c5000b330 comp rx=0 tx=0).stop 2023-08-27T11:17:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c5f19e700 1 -- 172.21.15.112:0/2952425259 shutdown_connections 2023-08-27T11:17:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c5f19e700 1 --2- 172.21.15.112:0/2952425259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c580b1400 0x7f3c580cf5a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c5f19e700 1 --2- 172.21.15.112:0/2952425259 >> v2:172.21.15.138:3300/0 conn(0x7f3c580b17e0 0x7f3c5806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.121+0000 7f3c5f19e700 1 -- 172.21.15.112:0/2952425259 >> 172.21.15.112:0/2952425259 conn(0x7f3c580d0000 msgr2=0x7f3c580d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.122+0000 7f3c5f19e700 1 -- 172.21.15.112:0/2952425259 shutdown_connections 2023-08-27T11:17:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.122+0000 7f3c5f19e700 1 -- 172.21.15.112:0/2952425259 wait complete. 2023-08-27T11:17:56.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.123+0000 7f3c5f19e700 1 Processor -- start 2023-08-27T11:17:56.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.123+0000 7f3c5f19e700 1 -- start start 2023-08-27T11:17:56.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.124+0000 7f3c5f19e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c580b1400 0x7f3c580c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:56.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.124+0000 7f3c5f19e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c580b17e0 0x7f3c58153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:56.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.124+0000 7f3c5f19e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c58161ce0 0x7f3c5806ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:56.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.124+0000 7f3c5f19e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580c6090 con 0x7f3c580b17e0 2023-08-27T11:17:56.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.124+0000 7f3c5f19e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580c7b60 con 0x7f3c58161ce0 2023-08-27T11:17:56.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.124+0000 7f3c5f19e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580c7270 con 0x7f3c580b1400 2023-08-27T11:17:56.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.124+0000 7f3c57fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c580b17e0 0x7f3c58153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:56.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.124+0000 7f3c57fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c580b17e0 0x7f3c58153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56542/0 (socket says 172.21.15.112:56542) 2023-08-27T11:17:56.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.124+0000 7f3c57fff700 1 -- 172.21.15.112:0/3853041308 learned_addr learned my addr 172.21.15.112:0/3853041308 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:56.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.124+0000 7f3c5cf3a700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c580b1400 0x7f3c580c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:56.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.124+0000 7f3c5d73b700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c58161ce0 0x7f3c5806ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:56.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.125+0000 7f3c57fff700 1 -- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c580b1400 msgr2=0x7f3c580c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:56.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.125+0000 7f3c57fff700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c580b1400 0x7f3c580c5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:56.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.125+0000 7f3c57fff700 1 -- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c58161ce0 msgr2=0x7f3c5806ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:56.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.125+0000 7f3c57fff700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c58161ce0 0x7f3c5806ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:56.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.125+0000 7f3c57fff700 1 -- 172.21.15.112:0/3853041308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c50009a70 con 0x7f3c580b17e0 2023-08-27T11:17:56.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.125+0000 7f3c57fff700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c580b17e0 0x7f3c58153e60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000e7e0 tx=0x7f3c4000eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:56.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.125+0000 7f3c557fa700 1 -- 172.21.15.112:0/3853041308 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4000c070 con 0x7f3c580b17e0 2023-08-27T11:17:56.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.125+0000 7f3c557fa700 1 -- 172.21.15.112:0/3853041308 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c40004230 con 0x7f3c580b17e0 2023-08-27T11:17:56.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.125+0000 7f3c5f19e700 1 -- 172.21.15.112:0/3853041308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c580c7510 con 0x7f3c580b17e0 2023-08-27T11:17:56.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.125+0000 7f3c557fa700 1 -- 172.21.15.112:0/3853041308 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c400045a0 con 0x7f3c580b17e0 2023-08-27T11:17:56.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.125+0000 7f3c5f19e700 1 -- 172.21.15.112:0/3853041308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c580c5d80 con 0x7f3c580b17e0 2023-08-27T11:17:56.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.126+0000 7f3c5f19e700 1 -- 172.21.15.112:0/3853041308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c580c7cd0 con 0x7f3c580b17e0 2023-08-27T11:17:56.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.126+0000 7f3c557fa700 1 -- 172.21.15.112:0/3853041308 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3c400036a0 con 0x7f3c580b17e0 2023-08-27T11:17:56.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.127+0000 7f3c557fa700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3c3806a0e0 0x7f3c3806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:56.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.127+0000 7f3c557fa700 1 -- 172.21.15.112:0/3853041308 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3c40087f90 con 0x7f3c580b17e0 2023-08-27T11:17:56.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.127+0000 7f3c5cf3a700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3c3806a0e0 0x7f3c3806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:56.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.127+0000 7f3c5cf3a700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3c3806a0e0 0x7f3c3806c5a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3c50014040 tx=0x7f3c50011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:56.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.131+0000 7f3c557fa700 1 -- 172.21.15.112:0/3853041308 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3c40053910 con 0x7f3c580b17e0 2023-08-27T11:17:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:56.347+0000 7f3c5f19e700 1 -- 172.21.15.112:0/3853041308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3c58067a20 con 0x7f3c580b17e0 2023-08-27T11:17:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.245+0000 7f3c557fa700 1 -- 172.21.15.112:0/3853041308 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3c40056f30 con 0x7f3c580b17e0 2023-08-27T11:17:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.248+0000 7f3c5f19e700 1 -- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3c3806a0e0 msgr2=0x7f3c3806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.248+0000 7f3c5f19e700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3c3806a0e0 0x7f3c3806c5a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3c50014040 tx=0x7f3c50011000 comp rx=0 tx=0).stop 2023-08-27T11:17:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.248+0000 7f3c5f19e700 1 -- 172.21.15.112:0/3853041308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c580b17e0 msgr2=0x7f3c58153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.248+0000 7f3c5f19e700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c580b17e0 0x7f3c58153e60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000e7e0 tx=0x7f3c4000eaf0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.248+0000 7f3c5f19e700 1 -- 172.21.15.112:0/3853041308 shutdown_connections 2023-08-27T11:17:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.248+0000 7f3c5f19e700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3c3806a0e0 0x7f3c3806c5a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.248+0000 7f3c5f19e700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c58161ce0 0x7f3c5806ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.248+0000 7f3c5f19e700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c580b1400 0x7f3c580c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.248+0000 7f3c5f19e700 1 --2- 172.21.15.112:0/3853041308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c580b17e0 0x7f3c58153e60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.248+0000 7f3c5f19e700 1 -- 172.21.15.112:0/3853041308 >> 172.21.15.112:0/3853041308 conn(0x7f3c580d0000 msgr2=0x7f3c581556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.248+0000 7f3c5f19e700 1 -- 172.21.15.112:0/3853041308 shutdown_connections 2023-08-27T11:17:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.248+0000 7f3c5f19e700 1 -- 172.21.15.112:0/3853041308 wait complete. 2023-08-27T11:17:57.252 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2023-08-27T11:17:57.264 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:17:57.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.387+0000 7f9afcd34700 1 Processor -- start 2023-08-27T11:17:57.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.387+0000 7f9afcd34700 1 -- start start 2023-08-27T11:17:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.388+0000 7f9afcd34700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9af806adb0 0x7f9af806b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.388+0000 7f9afcd34700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9af80cdbd0 0x7f9af80ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.388+0000 7f9afcd34700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af815eb60 con 0x7f9af8151060 2023-08-27T11:17:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.388+0000 7f9afcd34700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af806b770 con 0x7f9af80cdbd0 2023-08-27T11:17:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.388+0000 7f9afcd34700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80b02f0 con 0x7f9af806adb0 2023-08-27T11:17:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.388+0000 7f9af6d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9af80cdbd0 0x7f9af80ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.388+0000 7f9af6d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9af80cdbd0 0x7f9af80ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56556/0 (socket says 172.21.15.112:56556) 2023-08-27T11:17:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.388+0000 7f9af6d9d700 1 -- 172.21.15.112:0/4238307554 learned_addr learned my addr 172.21.15.112:0/4238307554 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.388+0000 7f9af5d9b700 1 --2- 172.21.15.112:0/4238307554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9af806adb0 0x7f9af806b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.388+0000 7f9af6d9d700 1 -- 172.21.15.112:0/4238307554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9af806adb0 msgr2=0x7f9af806b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.388+0000 7f9af6d9d700 1 --2- 172.21.15.112:0/4238307554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9af806adb0 0x7f9af806b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.388+0000 7f9af6d9d700 1 -- 172.21.15.112:0/4238307554 >> v1:172.21.15.138:6789/0 conn(0x7f9af8151060 legacy=0x7f9af815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.388+0000 7f9af6d9d700 1 -- 172.21.15.112:0/4238307554 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80b05d0 con 0x7f9af80cdbd0 2023-08-27T11:17:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.389+0000 7f9af6d9d700 1 --2- 172.21.15.112:0/4238307554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af80cdbd0 0x7f9af80ce060 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9af000d150 tx=0x7f9af000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7090be5a9b41115e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.389+0000 7f9af559a700 1 -- 172.21.15.112:0/4238307554 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0010070 con 0x7f9af80cdbd0 2023-08-27T11:17:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.389+0000 7f9af559a700 1 -- 172.21.15.112:0/4238307554 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af000dd80 con 0x7f9af80cdbd0 2023-08-27T11:17:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.389+0000 7f9af559a700 1 -- 172.21.15.112:0/4238307554 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af000b310 con 0x7f9af80cdbd0 2023-08-27T11:17:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.389+0000 7f9afcd34700 1 -- 172.21.15.112:0/4238307554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af80cdbd0 msgr2=0x7f9af80ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.389+0000 7f9afcd34700 1 --2- 172.21.15.112:0/4238307554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af80cdbd0 0x7f9af80ce060 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9af000d150 tx=0x7f9af000d460 comp rx=0 tx=0).stop 2023-08-27T11:17:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.389+0000 7f9afcd34700 1 -- 172.21.15.112:0/4238307554 shutdown_connections 2023-08-27T11:17:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.389+0000 7f9afcd34700 1 --2- 172.21.15.112:0/4238307554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af80cdbd0 0x7f9af80ce060 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.389+0000 7f9afcd34700 1 --2- 172.21.15.112:0/4238307554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9af806adb0 0x7f9af806b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.389+0000 7f9afcd34700 1 -- 172.21.15.112:0/4238307554 >> 172.21.15.112:0/4238307554 conn(0x7f9af80d0000 msgr2=0x7f9af80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.389+0000 7f9afcd34700 1 -- 172.21.15.112:0/4238307554 shutdown_connections 2023-08-27T11:17:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.389+0000 7f9afcd34700 1 -- 172.21.15.112:0/4238307554 wait complete. 2023-08-27T11:17:57.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.391+0000 7f9afcd34700 1 Processor -- start 2023-08-27T11:17:57.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.391+0000 7f9afcd34700 1 -- start start 2023-08-27T11:17:57.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.391+0000 7f9afcd34700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9af806adb0 0x7f9af80c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:57.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.391+0000 7f9afcd34700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af80cdbd0 0x7f9af80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:57.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.391+0000 7f9afcd34700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af8151060 0x7f9af81f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:57.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.391+0000 7f9afcd34700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af81f3950 con 0x7f9af8151060 2023-08-27T11:17:57.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.391+0000 7f9afcd34700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80aca20 con 0x7f9af80cdbd0 2023-08-27T11:17:57.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.391+0000 7f9afcd34700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af81f3390 con 0x7f9af806adb0 2023-08-27T11:17:57.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9af6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af8151060 0x7f9af81f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:57.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9af6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af8151060 0x7f9af81f2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56562/0 (socket says 172.21.15.112:56562) 2023-08-27T11:17:57.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9af6d9d700 1 -- 172.21.15.112:0/1792181027 learned_addr learned my addr 172.21.15.112:0/1792181027 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:57.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9af5d9b700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af80cdbd0 0x7f9af80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:57.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9af659c700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9af806adb0 0x7f9af80c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:57.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9af6d9d700 1 -- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9af806adb0 msgr2=0x7f9af80c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:57.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9af6d9d700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9af806adb0 0x7f9af80c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9af6d9d700 1 -- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af80cdbd0 msgr2=0x7f9af80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:57.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9af6d9d700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af80cdbd0 0x7f9af80ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9af6d9d700 1 -- 172.21.15.112:0/1792181027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af000bd40 con 0x7f9af8151060 2023-08-27T11:17:57.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9af5d9b700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af80cdbd0 0x7f9af80ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:57.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9af6d9d700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af8151060 0x7f9af81f2930 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9af0017040 tx=0x7f9af0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:57.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9ae6ffd700 1 -- 172.21.15.112:0/1792181027 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0009ad0 con 0x7f9af8151060 2023-08-27T11:17:57.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9ae6ffd700 1 -- 172.21.15.112:0/1792181027 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af0009c50 con 0x7f9af8151060 2023-08-27T11:17:57.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9afcd34700 1 -- 172.21.15.112:0/1792181027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af81f3670 con 0x7f9af8151060 2023-08-27T11:17:57.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9ae6ffd700 1 -- 172.21.15.112:0/1792181027 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0004030 con 0x7f9af8151060 2023-08-27T11:17:57.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.392+0000 7f9afcd34700 1 -- 172.21.15.112:0/1792181027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af80ca040 con 0x7f9af8151060 2023-08-27T11:17:57.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.393+0000 7f9afcd34700 1 -- 172.21.15.112:0/1792181027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af80ca200 con 0x7f9af8151060 2023-08-27T11:17:57.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.394+0000 7f9ae6ffd700 1 -- 172.21.15.112:0/1792181027 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9af0009050 con 0x7f9af8151060 2023-08-27T11:17:57.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.394+0000 7f9ae6ffd700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9ad806a0e0 0x7f9ad806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:57.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.394+0000 7f9ae6ffd700 1 -- 172.21.15.112:0/1792181027 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9af0089b80 con 0x7f9af8151060 2023-08-27T11:17:57.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.394+0000 7f9af659c700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9ad806a0e0 0x7f9ad806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:57.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.395+0000 7f9af659c700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9ad806a0e0 0x7f9ad806c5a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00a7d0 tx=0x7f9aec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:57.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.398+0000 7f9ae6ffd700 1 -- 172.21.15.112:0/1792181027 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9af0055650 con 0x7f9af8151060 2023-08-27T11:17:57.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.662+0000 7f9afcd34700 1 -- 172.21.15.112:0/1792181027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9af80acec0 con 0x7f9af8151060 2023-08-27T11:17:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.662+0000 7f9ae6ffd700 1 -- 172.21.15.112:0/1792181027 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f9af0019090 con 0x7f9af8151060 2023-08-27T11:17:57.664 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:17:57.664 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:17:57.665 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2023-08-27T11:17:57.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.665 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2023-08-27T11:17:57.666 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk2bnkLBAA+vxcTeJzucKk9iSF/Zw3Lg==", 2023-08-27T11:17:57.666 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.666 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:17:57.666 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:17:57.667 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:17:57.667 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.667 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.668 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.668 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2023-08-27T11:17:57.668 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk3zD5OBAASVnwG6yn/YQzfYyeMakFpA==", 2023-08-27T11:17:57.668 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.669 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:17:57.669 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:17:57.669 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:17:57.670 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.670 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.670 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.671 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2023-08-27T11:17:57.671 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtki7wJNBAAsIGgDLM82sWNcKnCDlSBMw==", 2023-08-27T11:17:57.671 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.671 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:17:57.671 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:17:57.672 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:17:57.672 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.672 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.674 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2023-08-27T11:17:57.674 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkCsW/BBAAFTPiTOpBo1L+rLo9px+AjQ==", 2023-08-27T11:17:57.674 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.675 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:17:57.675 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:17:57.675 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:17:57.675 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.675 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.677 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2023-08-27T11:17:57.677 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB7MOtkR/cSHhAAR8e8VRd+e0ESJkqQDYWbEw==", 2023-08-27T11:17:57.677 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.677 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:17:57.678 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:17:57.678 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.678 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.679 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2023-08-27T11:17:57.679 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB+MOtksBF3HRAA1OIXiIkhgPDYpQyYPaWOXg==", 2023-08-27T11:17:57.679 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.679 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:17:57.679 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:17:57.680 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.680 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.680 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2023-08-27T11:17:57.681 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCBMOtkWsrdHBAAkfWbiE6wAFzpt9+BxRh8fw==", 2023-08-27T11:17:57.681 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.681 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:17:57.681 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:17:57.681 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.682 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.682 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.682 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2023-08-27T11:17:57.682 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCEMOtkIkiKHRAARDoh/FH9ZdqUlZZ29Mc/Dw==", 2023-08-27T11:17:57.683 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.683 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:17:57.683 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:17:57.683 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.683 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.684 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2023-08-27T11:17:57.684 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCKMOtkPgDzGxAA+U4kbwD0d4lS8aNPG2CgHw==", 2023-08-27T11:17:57.684 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.684 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:17:57.685 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:17:57.685 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.685 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.685 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.686 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2023-08-27T11:17:57.686 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCNMOtkKomSGxAAGAQUcIH2H/uyZIU8vlq1pw==", 2023-08-27T11:17:57.686 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.686 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:17:57.686 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:17:57.687 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.687 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.687 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.687 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2023-08-27T11:17:57.688 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCQMOtkHI2MGhAA7sgbi8JXO8P9wnVvMy3PqA==", 2023-08-27T11:17:57.689 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.689 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:17:57.690 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:17:57.690 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.690 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.692 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2023-08-27T11:17:57.692 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCTMOtkVHpiGhAA4C4YJMWbQ0mmGgB4i8G/AA==", 2023-08-27T11:17:57.692 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.692 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:17:57.693 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:17:57.694 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.694 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.695 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.695 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2023-08-27T11:17:57.695 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA==", 2023-08-27T11:17:57.696 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.696 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:17:57.696 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:17:57.697 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:17:57.697 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:17:57.697 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.698 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2023-08-27T11:17:57.699 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtk66DbDxAAWZ5StELux0lriJ2gEN9icg==", 2023-08-27T11:17:57.699 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.699 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:17:57.700 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:17:57.700 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:17:57.700 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:17:57.701 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.701 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.702 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2023-08-27T11:17:57.702 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkViKJDBAAF2ofwO+78er9OyFfn2bO+g==", 2023-08-27T11:17:57.702 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.703 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:17:57.703 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:17:57.704 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:17:57.704 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:17:57.705 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.705 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.705 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2023-08-27T11:17:57.706 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkwt+QFRAAowWE69VqeRhQz0896rxx9Q==", 2023-08-27T11:17:57.706 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.706 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:17:57.706 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:17:57.706 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:17:57.707 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:17:57.707 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.708 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2023-08-27T11:17:57.708 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkjDA+DxAAjmOn82AC2OMpKlAVCD6hlg==", 2023-08-27T11:17:57.708 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.708 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:17:57.709 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:17:57.709 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:17:57.709 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.710 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.710 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-mds", 2023-08-27T11:17:57.710 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCeMOtkp0cVHhAAkklcIM0CXFSn6aQ880rEbA==", 2023-08-27T11:17:57.711 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.711 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-mds" 2023-08-27T11:17:57.711 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.712 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-mgr", 2023-08-27T11:17:57.713 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCeMOtkM1kVHhAARJTsKU56o2MrTScRWdopkw==", 2023-08-27T11:17:57.713 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.713 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-mgr" 2023-08-27T11:17:57.713 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.714 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-osd", 2023-08-27T11:17:57.714 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCeMOtk22UVHhAAFDJoSCVC97uV2A3pP4C7dQ==", 2023-08-27T11:17:57.714 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.714 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-osd" 2023-08-27T11:17:57.715 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.715 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-rbd", 2023-08-27T11:17:57.715 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCeMOtkE3AVHhAAf8x4LGKGMG3S25ru3hi8YQ==", 2023-08-27T11:17:57.716 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.716 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-rbd" 2023-08-27T11:17:57.716 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.716 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-08-27T11:17:57.717 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCeMOtkongVHhAAptkqWfgstpwYprreb6LXLQ==", 2023-08-27T11:17:57.717 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.717 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-08-27T11:17:57.717 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.718 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-rgw", 2023-08-27T11:17:57.718 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCeMOtkhoAVHhAAHdXhlGdo17XKkSeYqXJmYg==", 2023-08-27T11:17:57.718 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.718 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-rgw" 2023-08-27T11:17:57.719 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.719 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2023-08-27T11:17:57.719 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkXguHKRAALHwzk8diVXqsMvTKr+AzLQ==", 2023-08-27T11:17:57.720 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.720 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:17:57.720 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:17:57.720 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.720 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:17:57.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:17:57.721 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2023-08-27T11:17:57.721 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkrRjBJRAAq3ftmIh81LKadpliVUiJ6g==", 2023-08-27T11:17:57.721 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:17:57.722 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:17:57.722 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:17:57.722 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:17:57.722 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.723 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:17:57.723 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:17:57.723 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:17:57.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.664+0000 7f9afcd34700 1 -- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9ad806a0e0 msgr2=0x7f9ad806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:57.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.664+0000 7f9afcd34700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9ad806a0e0 0x7f9ad806c5a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00a7d0 tx=0x7f9aec008040 comp rx=0 tx=0).stop 2023-08-27T11:17:57.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.664+0000 7f9afcd34700 1 -- 172.21.15.112:0/1792181027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af8151060 msgr2=0x7f9af81f2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:57.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.664+0000 7f9afcd34700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af8151060 0x7f9af81f2930 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9af0017040 tx=0x7f9af0004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.664+0000 7f9afcd34700 1 -- 172.21.15.112:0/1792181027 shutdown_connections 2023-08-27T11:17:57.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.664+0000 7f9afcd34700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9ad806a0e0 0x7f9ad806c5a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.664+0000 7f9afcd34700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af8151060 0x7f9af81f2930 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.664+0000 7f9afcd34700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9af806adb0 0x7f9af80c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.664+0000 7f9afcd34700 1 --2- 172.21.15.112:0/1792181027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af80cdbd0 0x7f9af80ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.664+0000 7f9afcd34700 1 -- 172.21.15.112:0/1792181027 >> 172.21.15.112:0/1792181027 conn(0x7f9af80d0000 msgr2=0x7f9af815d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.664+0000 7f9afcd34700 1 -- 172.21.15.112:0/1792181027 shutdown_connections 2023-08-27T11:17:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.664+0000 7f9afcd34700 1 -- 172.21.15.112:0/1792181027 wait complete. 2023-08-27T11:17:57.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-08-27T11:17:57.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.850+0000 7f87c150c700 1 Processor -- start 2023-08-27T11:17:57.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.850+0000 7f87c150c700 1 -- start start 2023-08-27T11:17:57.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.851+0000 7f87c150c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f87bc0b5470 0x7f87bc0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:57.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.851+0000 7f87c150c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87bc1619b0 0x7f87bc0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:57.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.851+0000 7f87c150c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87bc0b3b70 con 0x7f87bc146be0 2023-08-27T11:17:57.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.851+0000 7f87c150c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87bc0b3980 con 0x7f87bc1619b0 2023-08-27T11:17:57.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.851+0000 7f87c150c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87bc0b3e50 con 0x7f87bc0b5470 2023-08-27T11:17:57.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.851+0000 7f87bb7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87bc1619b0 0x7f87bc0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:57.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.851+0000 7f87bb7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87bc1619b0 0x7f87bc0cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56570/0 (socket says 172.21.15.112:56570) 2023-08-27T11:17:57.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.851+0000 7f87bb7fe700 1 -- 172.21.15.112:0/1104555479 learned_addr learned my addr 172.21.15.112:0/1104555479 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:57.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.851+0000 7f87ba7fc700 1 --2- 172.21.15.112:0/1104555479 >> v2:172.21.15.138:3300/0 conn(0x7f87bc0b5470 0x7f87bc0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:57.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87bb7fe700 1 -- 172.21.15.112:0/1104555479 >> v2:172.21.15.138:3300/0 conn(0x7f87bc0b5470 msgr2=0x7f87bc0b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:57.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87bb7fe700 1 --2- 172.21.15.112:0/1104555479 >> v2:172.21.15.138:3300/0 conn(0x7f87bc0b5470 0x7f87bc0b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87bb7fe700 1 -- 172.21.15.112:0/1104555479 >> v1:172.21.15.138:6789/0 conn(0x7f87bc146be0 legacy=0x7f87bc0b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:57.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87bb7fe700 1 -- 172.21.15.112:0/1104555479 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc0b4130 con 0x7f87bc1619b0 2023-08-27T11:17:57.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87bb7fe700 1 --2- 172.21.15.112:0/1104555479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87bc1619b0 0x7f87bc0cfa40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0d2120 tx=0x7f87b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fc7bdf3a5fe5bb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:57.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87b9ffb700 1 -- 172.21.15.112:0/1104555479 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400e070 con 0x7f87bc1619b0 2023-08-27T11:17:57.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87b9ffb700 1 -- 172.21.15.112:0/1104555479 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b400b9c0 con 0x7f87bc1619b0 2023-08-27T11:17:57.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87b9ffb700 1 -- 172.21.15.112:0/1104555479 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400bd70 con 0x7f87bc1619b0 2023-08-27T11:17:57.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87c150c700 1 -- 172.21.15.112:0/1104555479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87bc1619b0 msgr2=0x7f87bc0cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:57.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87c150c700 1 --2- 172.21.15.112:0/1104555479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87bc1619b0 0x7f87bc0cfa40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0d2120 tx=0x7f87b400b330 comp rx=0 tx=0).stop 2023-08-27T11:17:57.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87c150c700 1 -- 172.21.15.112:0/1104555479 shutdown_connections 2023-08-27T11:17:57.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87c150c700 1 --2- 172.21.15.112:0/1104555479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87bc1619b0 0x7f87bc0cfa40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87c150c700 1 --2- 172.21.15.112:0/1104555479 >> v2:172.21.15.138:3300/0 conn(0x7f87bc0b5470 0x7f87bc0b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.852+0000 7f87c150c700 1 -- 172.21.15.112:0/1104555479 >> 172.21.15.112:0/1104555479 conn(0x7f87bc06c880 msgr2=0x7f87bc0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:57.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.853+0000 7f87c150c700 1 -- 172.21.15.112:0/1104555479 shutdown_connections 2023-08-27T11:17:57.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.853+0000 7f87c150c700 1 -- 172.21.15.112:0/1104555479 wait complete. 2023-08-27T11:17:57.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.854+0000 7f87c150c700 1 Processor -- start 2023-08-27T11:17:57.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.854+0000 7f87c150c700 1 -- start start 2023-08-27T11:17:57.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87c150c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87bc0b5470 0x7f87bc0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:57.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87c150c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87bc146be0 0x7f87bc142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:57.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87c150c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87bc1619b0 0x7f87bc0713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:57.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87c150c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc14e650 con 0x7f87bc0b5470 2023-08-27T11:17:57.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87c150c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc070910 con 0x7f87bc146be0 2023-08-27T11:17:57.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87c150c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc1f3610 con 0x7f87bc1619b0 2023-08-27T11:17:57.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87baffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87bc0b5470 0x7f87bc0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:57.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87baffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87bc0b5470 0x7f87bc0b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56572/0 (socket says 172.21.15.112:56572) 2023-08-27T11:17:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87baffd700 1 -- 172.21.15.112:0/1409960392 learned_addr learned my addr 172.21.15.112:0/1409960392 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87ba7fc700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87bc146be0 0x7f87bc142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87bb7fe700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87bc1619b0 0x7f87bc0713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87baffd700 1 -- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87bc1619b0 msgr2=0x7f87bc0713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87baffd700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87bc1619b0 0x7f87bc0713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87baffd700 1 -- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87bc146be0 msgr2=0x7f87bc142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:57.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.855+0000 7f87baffd700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87bc146be0 0x7f87bc142740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:57.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.856+0000 7f87baffd700 1 -- 172.21.15.112:0/1409960392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b4009b30 con 0x7f87bc0b5470 2023-08-27T11:17:57.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.856+0000 7f87baffd700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87bc0b5470 0x7f87bc0b2040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87b000c4e0 tx=0x7f87b000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:57.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.856+0000 7f87a37fe700 1 -- 172.21.15.112:0/1409960392 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b00052d0 con 0x7f87bc0b5470 2023-08-27T11:17:57.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.856+0000 7f87a37fe700 1 -- 172.21.15.112:0/1409960392 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b0005430 con 0x7f87bc0b5470 2023-08-27T11:17:57.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.856+0000 7f87a37fe700 1 -- 172.21.15.112:0/1409960392 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b00038c0 con 0x7f87bc0b5470 2023-08-27T11:17:57.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.856+0000 7f87c150c700 1 -- 172.21.15.112:0/1409960392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87bc1f38f0 con 0x7f87bc0b5470 2023-08-27T11:17:57.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.856+0000 7f87c150c700 1 -- 172.21.15.112:0/1409960392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc070eb0 con 0x7f87bc0b5470 2023-08-27T11:17:57.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.857+0000 7f87c150c700 1 -- 172.21.15.112:0/1409960392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87bc067a20 con 0x7f87bc0b5470 2023-08-27T11:17:57.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.857+0000 7f87a37fe700 1 -- 172.21.15.112:0/1409960392 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f87b0003a20 con 0x7f87bc0b5470 2023-08-27T11:17:57.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.857+0000 7f87a37fe700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87a806a0e0 0x7f87a806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:57.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.858+0000 7f87a37fe700 1 -- 172.21.15.112:0/1409960392 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f87b0087660 con 0x7f87bc0b5470 2023-08-27T11:17:57.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.858+0000 7f87ba7fc700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87a806a0e0 0x7f87a806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:57.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.858+0000 7f87ba7fc700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87a806a0e0 0x7f87a806c5a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f87bc144500 tx=0x7f87ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:57.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:57.862+0000 7f87a37fe700 1 -- 172.21.15.112:0/1409960392 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f87b0053130 con 0x7f87bc0b5470 2023-08-27T11:17:58.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.129+0000 7f87c150c700 1 -- 172.21.15.112:0/1409960392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f87bc0d23f0 con 0x7f87bc0b5470 2023-08-27T11:17:58.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.134+0000 7f87a37fe700 1 -- 172.21.15.112:0/1409960392 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f87b0056750 con 0x7f87bc0b5470 2023-08-27T11:17:58.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.137+0000 7f87c150c700 1 -- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87a806a0e0 msgr2=0x7f87a806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:58.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.137+0000 7f87c150c700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87a806a0e0 0x7f87a806c5a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f87bc144500 tx=0x7f87ac008040 comp rx=0 tx=0).stop 2023-08-27T11:17:58.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.137+0000 7f87c150c700 1 -- 172.21.15.112:0/1409960392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87bc0b5470 msgr2=0x7f87bc0b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:58.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.137+0000 7f87c150c700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87bc0b5470 0x7f87bc0b2040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87b000c4e0 tx=0x7f87b000c8a0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.137+0000 7f87c150c700 1 -- 172.21.15.112:0/1409960392 shutdown_connections 2023-08-27T11:17:58.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.137+0000 7f87c150c700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87a806a0e0 0x7f87a806c5a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.137+0000 7f87c150c700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87bc1619b0 0x7f87bc0713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.137+0000 7f87c150c700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87bc0b5470 0x7f87bc0b2040 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.137+0000 7f87c150c700 1 --2- 172.21.15.112:0/1409960392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87bc146be0 0x7f87bc142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.137+0000 7f87c150c700 1 -- 172.21.15.112:0/1409960392 >> 172.21.15.112:0/1409960392 conn(0x7f87bc06c880 msgr2=0x7f87bc0b4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:58.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.137+0000 7f87c150c700 1 -- 172.21.15.112:0/1409960392 shutdown_connections 2023-08-27T11:17:58.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.137+0000 7f87c150c700 1 -- 172.21.15.112:0/1409960392 wait complete. 2023-08-27T11:17:58.150 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-08-27T11:17:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.270+0000 7f0b0f72b700 1 Processor -- start 2023-08-27T11:17:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.270+0000 7f0b0f72b700 1 -- start start 2023-08-27T11:17:58.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.271+0000 7f0b0f72b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0b080ad210 0x7f0b0815e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:58.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.271+0000 7f0b0f72b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b0806e150 0x7f0b0806e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:58.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.271+0000 7f0b0f72b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0b08161050 0x7f0b081614e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:58.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.271+0000 7f0b0f72b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b08143bf0 con 0x7f0b08161050 2023-08-27T11:17:58.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.271+0000 7f0b0f72b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b080add70 con 0x7f0b080ad210 2023-08-27T11:17:58.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.271+0000 7f0b0f72b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0806ead0 con 0x7f0b0806e150 2023-08-27T11:17:58.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.271+0000 7f0b0dcc8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0b08161050 0x7f0b081614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:58.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.271+0000 7f0b0dcc8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0b08161050 0x7f0b081614e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56584/0 (socket says 172.21.15.112:56584) 2023-08-27T11:17:58.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.271+0000 7f0b0dcc8700 1 -- 172.21.15.112:0/1734625024 learned_addr learned my addr 172.21.15.112:0/1734625024 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:58.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.272+0000 7f0b0ccc6700 1 --2- 172.21.15.112:0/1734625024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b0806e150 0x7f0b0806e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.272+0000 7f0b0d4c7700 1 --2- 172.21.15.112:0/1734625024 >> v2:172.21.15.138:3300/0 conn(0x7f0b080ad210 0x7f0b0815e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.272+0000 7f0b0dcc8700 1 -- 172.21.15.112:0/1734625024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b0806e150 msgr2=0x7f0b0806e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.272+0000 7f0b0dcc8700 1 --2- 172.21.15.112:0/1734625024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b0806e150 0x7f0b0806e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.272+0000 7f0b0dcc8700 1 -- 172.21.15.112:0/1734625024 >> v2:172.21.15.138:3300/0 conn(0x7f0b080ad210 msgr2=0x7f0b0815e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.272+0000 7f0b0dcc8700 1 --2- 172.21.15.112:0/1734625024 >> v2:172.21.15.138:3300/0 conn(0x7f0b080ad210 0x7f0b0815e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.272+0000 7f0b0dcc8700 1 -- 172.21.15.112:0/1734625024 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b08142fe0 con 0x7f0b08161050 2023-08-27T11:17:58.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.272+0000 7f0b0d4c7700 1 --2- 172.21.15.112:0/1734625024 >> v2:172.21.15.138:3300/0 conn(0x7f0b080ad210 0x7f0b0815e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:58.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.272+0000 7f0b0dcc8700 1 --2- 172.21.15.112:0/1734625024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b08161050 0x7f0b081614e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0b08072230 tx=0x7f0b0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b162517e997d62 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:58.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.272+0000 7f0afffff700 1 -- 172.21.15.112:0/1734625024 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000e070 con 0x7f0b08161050 2023-08-27T11:17:58.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.272+0000 7f0afffff700 1 -- 172.21.15.112:0/1734625024 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0000b910 con 0x7f0b08161050 2023-08-27T11:17:58.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.272+0000 7f0afffff700 1 -- 172.21.15.112:0/1734625024 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000bcc0 con 0x7f0b08161050 2023-08-27T11:17:58.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.273+0000 7f0b0f72b700 1 -- 172.21.15.112:0/1734625024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b08161050 msgr2=0x7f0b081614e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:58.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.273+0000 7f0b0f72b700 1 --2- 172.21.15.112:0/1734625024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b08161050 0x7f0b081614e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0b08072230 tx=0x7f0b0000b330 comp rx=0 tx=0).stop 2023-08-27T11:17:58.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.273+0000 7f0b0f72b700 1 -- 172.21.15.112:0/1734625024 shutdown_connections 2023-08-27T11:17:58.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.273+0000 7f0b0f72b700 1 --2- 172.21.15.112:0/1734625024 >> v2:172.21.15.138:3300/0 conn(0x7f0b080ad210 0x7f0b0815e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.273+0000 7f0b0f72b700 1 --2- 172.21.15.112:0/1734625024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b08161050 0x7f0b081614e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.273+0000 7f0b0f72b700 1 --2- 172.21.15.112:0/1734625024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b0806e150 0x7f0b0806e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.273+0000 7f0b0f72b700 1 -- 172.21.15.112:0/1734625024 >> 172.21.15.112:0/1734625024 conn(0x7f0b0806cd30 msgr2=0x7f0b080717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:58.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.273+0000 7f0b0f72b700 1 -- 172.21.15.112:0/1734625024 shutdown_connections 2023-08-27T11:17:58.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.273+0000 7f0b0f72b700 1 -- 172.21.15.112:0/1734625024 wait complete. 2023-08-27T11:17:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.274+0000 7f0b0f72b700 1 Processor -- start 2023-08-27T11:17:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.274+0000 7f0b0f72b700 1 -- start start 2023-08-27T11:17:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0f72b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b0806e150 0x7f0b080b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0f72b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b080ad210 0x7f0b08138de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0f72b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b08161050 0x7f0b08070650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0f72b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b081f6fc0 con 0x7f0b0806e150 2023-08-27T11:17:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0f72b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b08139320 con 0x7f0b08161050 2023-08-27T11:17:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0f72b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b081f63b0 con 0x7f0b080ad210 2023-08-27T11:17:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0d4c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b0806e150 0x7f0b080b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0d4c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b0806e150 0x7f0b080b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56586/0 (socket says 172.21.15.112:56586) 2023-08-27T11:17:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0d4c7700 1 -- 172.21.15.112:0/3941553126 learned_addr learned my addr 172.21.15.112:0/3941553126 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0ccc6700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b080ad210 0x7f0b08138de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0dcc8700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b08161050 0x7f0b08070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:58.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0d4c7700 1 -- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b080ad210 msgr2=0x7f0b08138de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:58.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0d4c7700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b080ad210 0x7f0b08138de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0d4c7700 1 -- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b08161050 msgr2=0x7f0b08070650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:58.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0d4c7700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b08161050 0x7f0b08070650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0d4c7700 1 -- 172.21.15.112:0/3941553126 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b00009b30 con 0x7f0b0806e150 2023-08-27T11:17:58.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0d4c7700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b0806e150 0x7f0b080b16d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0af800e810 tx=0x7f0af800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:58.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0afdffb700 1 -- 172.21.15.112:0/3941553126 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af800c070 con 0x7f0b0806e150 2023-08-27T11:17:58.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0afdffb700 1 -- 172.21.15.112:0/3941553126 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af8004230 con 0x7f0b0806e150 2023-08-27T11:17:58.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0b0f72b700 1 -- 172.21.15.112:0/3941553126 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b081f6690 con 0x7f0b0806e150 2023-08-27T11:17:58.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.275+0000 7f0afdffb700 1 -- 172.21.15.112:0/3941553126 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af80036a0 con 0x7f0b0806e150 2023-08-27T11:17:58.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.276+0000 7f0b0f72b700 1 -- 172.21.15.112:0/3941553126 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b081f6c50 con 0x7f0b0806e150 2023-08-27T11:17:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.277+0000 7f0b0f72b700 1 -- 172.21.15.112:0/3941553126 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b08067a20 con 0x7f0b0806e150 2023-08-27T11:17:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.277+0000 7f0afdffb700 1 -- 172.21.15.112:0/3941553126 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0af8003800 con 0x7f0b0806e150 2023-08-27T11:17:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.277+0000 7f0afdffb700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0aec06a0e0 0x7f0aec06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.277+0000 7f0afdffb700 1 -- 172.21.15.112:0/3941553126 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0af8087f30 con 0x7f0b0806e150 2023-08-27T11:17:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.277+0000 7f0b0ccc6700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0aec06a0e0 0x7f0aec06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:58.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.278+0000 7f0b0ccc6700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0aec06a0e0 0x7f0aec06c5a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0b080b1c10 tx=0x7f0b040091c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:58.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.282+0000 7f0afdffb700 1 -- 172.21.15.112:0/3941553126 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0af8053970 con 0x7f0b0806e150 2023-08-27T11:17:58.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.544+0000 7f0b0f72b700 1 -- 172.21.15.112:0/3941553126 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f0b08072500 con 0x7f0b0806e150 2023-08-27T11:17:58.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.550+0000 7f0afdffb700 1 -- 172.21.15.112:0/3941553126 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0af80094c0 con 0x7f0b0806e150 2023-08-27T11:17:58.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.552+0000 7f0b0f72b700 1 -- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0aec06a0e0 msgr2=0x7f0aec06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:58.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.552+0000 7f0b0f72b700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0aec06a0e0 0x7f0aec06c5a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0b080b1c10 tx=0x7f0b040091c0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.552+0000 7f0b0f72b700 1 -- 172.21.15.112:0/3941553126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b0806e150 msgr2=0x7f0b080b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:58.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.552+0000 7f0b0f72b700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b0806e150 0x7f0b080b16d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0af800e810 tx=0x7f0af800eb20 comp rx=0 tx=0).stop 2023-08-27T11:17:58.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.552+0000 7f0b0f72b700 1 -- 172.21.15.112:0/3941553126 shutdown_connections 2023-08-27T11:17:58.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.552+0000 7f0b0f72b700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0aec06a0e0 0x7f0aec06c5a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.552+0000 7f0b0f72b700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b08161050 0x7f0b08070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.552+0000 7f0b0f72b700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b0806e150 0x7f0b080b16d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.552+0000 7f0b0f72b700 1 --2- 172.21.15.112:0/3941553126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b080ad210 0x7f0b08138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.552+0000 7f0b0f72b700 1 -- 172.21.15.112:0/3941553126 >> 172.21.15.112:0/3941553126 conn(0x7f0b0806cd30 msgr2=0x7f0b0815fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:58.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.552+0000 7f0b0f72b700 1 -- 172.21.15.112:0/3941553126 shutdown_connections 2023-08-27T11:17:58.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.552+0000 7f0b0f72b700 1 -- 172.21.15.112:0/3941553126 wait complete. 2023-08-27T11:17:58.567 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-08-27T11:17:58.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.686+0000 7fcca7300700 1 Processor -- start 2023-08-27T11:17:58.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.686+0000 7fcca7300700 1 -- start start 2023-08-27T11:17:58.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.687+0000 7fcca7300700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcca00b1400 0x7fcca00cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:58.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.687+0000 7fcca7300700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcca00b17e0 0x7fcca006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:58.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.687+0000 7fcca7300700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca0161850 con 0x7fcca0161ce0 2023-08-27T11:17:58.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.687+0000 7fcca7300700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca006e7d0 con 0x7fcca00b17e0 2023-08-27T11:17:58.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.687+0000 7fcca7300700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca00cfc10 con 0x7fcca00b1400 2023-08-27T11:17:58.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.687+0000 7fcca489b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcca00b17e0 0x7fcca006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:58.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.687+0000 7fcca489b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcca00b17e0 0x7fcca006e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56596/0 (socket says 172.21.15.112:56596) 2023-08-27T11:17:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.687+0000 7fcca489b700 1 -- 172.21.15.112:0/1424238846 learned_addr learned my addr 172.21.15.112:0/1424238846 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.687+0000 7fcca509c700 1 --2- 172.21.15.112:0/1424238846 >> v2:172.21.15.138:3300/0 conn(0x7fcca00b1400 0x7fcca00cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.687+0000 7fcca489b700 1 -- 172.21.15.112:0/1424238846 >> v2:172.21.15.138:3300/0 conn(0x7fcca00b1400 msgr2=0x7fcca00cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.687+0000 7fcca489b700 1 --2- 172.21.15.112:0/1424238846 >> v2:172.21.15.138:3300/0 conn(0x7fcca00b1400 0x7fcca00cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.687+0000 7fcca489b700 1 -- 172.21.15.112:0/1424238846 >> v1:172.21.15.138:6789/0 conn(0x7fcca0161ce0 legacy=0x7fcca006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.687+0000 7fcca489b700 1 -- 172.21.15.112:0/1424238846 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca0161200 con 0x7fcca00b17e0 2023-08-27T11:17:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.688+0000 7fcca489b700 1 --2- 172.21.15.112:0/1424238846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca00b17e0 0x7fcca006e290 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcca0070370 tx=0x7fcc9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e31a0d5dfe40cf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.688+0000 7fcc97fff700 1 -- 172.21.15.112:0/1424238846 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c004230 con 0x7fcca00b17e0 2023-08-27T11:17:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.688+0000 7fcc97fff700 1 -- 172.21.15.112:0/1424238846 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c004390 con 0x7fcca00b17e0 2023-08-27T11:17:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.688+0000 7fcc97fff700 1 -- 172.21.15.112:0/1424238846 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0036a0 con 0x7fcca00b17e0 2023-08-27T11:17:58.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.688+0000 7fcca7300700 1 -- 172.21.15.112:0/1424238846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca00b17e0 msgr2=0x7fcca006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:58.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.688+0000 7fcca7300700 1 --2- 172.21.15.112:0/1424238846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca00b17e0 0x7fcca006e290 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcca0070370 tx=0x7fcc9c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.688+0000 7fcca7300700 1 -- 172.21.15.112:0/1424238846 shutdown_connections 2023-08-27T11:17:58.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.688+0000 7fcca7300700 1 --2- 172.21.15.112:0/1424238846 >> v2:172.21.15.138:3300/0 conn(0x7fcca00b1400 0x7fcca00cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.688+0000 7fcca7300700 1 --2- 172.21.15.112:0/1424238846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca00b17e0 0x7fcca006e290 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.688+0000 7fcca7300700 1 -- 172.21.15.112:0/1424238846 >> 172.21.15.112:0/1424238846 conn(0x7fcca00d0000 msgr2=0x7fcca00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:58.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.688+0000 7fcca7300700 1 -- 172.21.15.112:0/1424238846 shutdown_connections 2023-08-27T11:17:58.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.688+0000 7fcca7300700 1 -- 172.21.15.112:0/1424238846 wait complete. 2023-08-27T11:17:58.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.690+0000 7fcca7300700 1 Processor -- start 2023-08-27T11:17:58.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.690+0000 7fcca7300700 1 -- start start 2023-08-27T11:17:58.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca7300700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcca00b1400 0x7fcca01450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:58.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca7300700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca00b17e0 0x7fcca00c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:58.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca7300700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcca0161ce0 0x7fcca00c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:58.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca7300700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca00c7920 con 0x7fcca00b17e0 2023-08-27T11:17:58.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca7300700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca01f73f0 con 0x7fcca00b1400 2023-08-27T11:17:58.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca7300700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca01f6f60 con 0x7fcca0161ce0 2023-08-27T11:17:58.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca489b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca00b17e0 0x7fcca00c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:58.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca489b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca00b17e0 0x7fcca00c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56602/0 (socket says 172.21.15.112:56602) 2023-08-27T11:17:58.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca489b700 1 -- 172.21.15.112:0/3298616071 learned_addr learned my addr 172.21.15.112:0/3298616071 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:58.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca509c700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcca00b1400 0x7fcca01450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:58.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca589d700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcca0161ce0 0x7fcca00c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:58.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca489b700 1 -- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcca0161ce0 msgr2=0x7fcca00c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:58.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca489b700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcca0161ce0 0x7fcca00c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca489b700 1 -- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcca00b1400 msgr2=0x7fcca01450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:58.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca489b700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcca00b1400 0x7fcca01450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca489b700 1 -- 172.21.15.112:0/3298616071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9c009550 con 0x7fcca00b17e0 2023-08-27T11:17:58.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.691+0000 7fcca489b700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca00b17e0 0x7fcca00c70d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c004000 tx=0x7fcc9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:58.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.692+0000 7fcc95ffb700 1 -- 172.21.15.112:0/3298616071 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c003d40 con 0x7fcca00b17e0 2023-08-27T11:17:58.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.692+0000 7fcc95ffb700 1 -- 172.21.15.112:0/3298616071 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c01f070 con 0x7fcca00b17e0 2023-08-27T11:17:58.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.692+0000 7fcca7300700 1 -- 172.21.15.112:0/3298616071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca01f6630 con 0x7fcca00b17e0 2023-08-27T11:17:58.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.692+0000 7fcc95ffb700 1 -- 172.21.15.112:0/3298616071 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c011870 con 0x7fcca00b17e0 2023-08-27T11:17:58.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.692+0000 7fcca7300700 1 -- 172.21.15.112:0/3298616071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca01f6bf0 con 0x7fcca00b17e0 2023-08-27T11:17:58.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.693+0000 7fcca7300700 1 -- 172.21.15.112:0/3298616071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca0067a20 con 0x7fcca00b17e0 2023-08-27T11:17:58.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.693+0000 7fcc95ffb700 1 -- 172.21.15.112:0/3298616071 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcc9c021030 con 0x7fcca00b17e0 2023-08-27T11:17:58.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.694+0000 7fcc95ffb700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcc8406a0e0 0x7fcc8406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:58.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.694+0000 7fcc95ffb700 1 -- 172.21.15.112:0/3298616071 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcc9c0896a0 con 0x7fcca00b17e0 2023-08-27T11:17:58.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.694+0000 7fcca509c700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcc8406a0e0 0x7fcc8406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:58.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.694+0000 7fcca509c700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcc8406a0e0 0x7fcc8406c5a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009930 tx=0x7fcc90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:58.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.698+0000 7fcc95ffb700 1 -- 172.21.15.112:0/3298616071 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcc9c055170 con 0x7fcca00b17e0 2023-08-27T11:17:58.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.962+0000 7fcca7300700 1 -- 172.21.15.112:0/3298616071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fcca01f70a0 con 0x7fcca00b17e0 2023-08-27T11:17:58.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.965+0000 7fcc95ffb700 1 -- 172.21.15.112:0/3298616071 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcc9c058790 con 0x7fcca00b17e0 2023-08-27T11:17:58.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.967+0000 7fcca7300700 1 -- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcc8406a0e0 msgr2=0x7fcc8406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:58.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.967+0000 7fcca7300700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcc8406a0e0 0x7fcc8406c5a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009930 tx=0x7fcc90008040 comp rx=0 tx=0).stop 2023-08-27T11:17:58.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.967+0000 7fcca7300700 1 -- 172.21.15.112:0/3298616071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca00b17e0 msgr2=0x7fcca00c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:58.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.967+0000 7fcca7300700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca00b17e0 0x7fcca00c70d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c004000 tx=0x7fcc9c004750 comp rx=0 tx=0).stop 2023-08-27T11:17:58.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.967+0000 7fcca7300700 1 -- 172.21.15.112:0/3298616071 shutdown_connections 2023-08-27T11:17:58.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.967+0000 7fcca7300700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcc8406a0e0 0x7fcc8406c5a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.967+0000 7fcca7300700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcca0161ce0 0x7fcca00c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.967+0000 7fcca7300700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcca00b1400 0x7fcca01450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.967+0000 7fcca7300700 1 --2- 172.21.15.112:0/3298616071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca00b17e0 0x7fcca00c70d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.967+0000 7fcca7300700 1 -- 172.21.15.112:0/3298616071 >> 172.21.15.112:0/3298616071 conn(0x7fcca00d0000 msgr2=0x7fcca00cb980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:58.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.967+0000 7fcca7300700 1 -- 172.21.15.112:0/3298616071 shutdown_connections 2023-08-27T11:17:58.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:58.967+0000 7fcca7300700 1 -- 172.21.15.112:0/3298616071 wait complete. 2023-08-27T11:17:58.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-08-27T11:17:59.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.109+0000 7f336d12f700 1 Processor -- start 2023-08-27T11:17:59.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.109+0000 7f336d12f700 1 -- start start 2023-08-27T11:17:59.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.110+0000 7f336d12f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f336815b770 0x7f336815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.110+0000 7f336d12f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33680af110 con 0x7f33680aed30 2023-08-27T11:17:59.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.110+0000 7f336d12f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33680721f0 con 0x7f336806de20 2023-08-27T11:17:59.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.110+0000 7f336d12f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336815c100 con 0x7f336815b770 2023-08-27T11:17:59.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.110+0000 7f3366d9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f33680aed30 0x7f336815b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50990/0 (socket says 172.21.15.112:50990) 2023-08-27T11:17:59.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.110+0000 7f3366d9d700 1 -- 172.21.15.112:0/3320100283 learned_addr learned my addr 172.21.15.112:0/3320100283 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:17:59.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.110+0000 7f336759e700 1 --2- 172.21.15.112:0/3320100283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f336815b770 0x7f336815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.111+0000 7f3365d9b700 1 -- 172.21.15.112:0/3320100283 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561458049 0 0) 0x7f33680af110 con 0x7f33680aed30 2023-08-27T11:17:59.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.111+0000 7f3365d9b700 1 -- 172.21.15.112:0/3320100283 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33500035b0 con 0x7f33680aed30 2023-08-27T11:17:59.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.111+0000 7f3365d9b700 1 -- 172.21.15.112:0/3320100283 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402591148 0 0) 0x7f33680721f0 con 0x7f336806de20 2023-08-27T11:17:59.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.111+0000 7f3365d9b700 1 -- 172.21.15.112:0/3320100283 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33680af110 con 0x7f336806de20 2023-08-27T11:17:59.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.111+0000 7f3365d9b700 1 -- 172.21.15.112:0/3320100283 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2203854608 0 0) 0x7f33500035b0 con 0x7f33680aed30 2023-08-27T11:17:59.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.111+0000 7f336759e700 1 -- 172.21.15.112:0/3320100283 >> v1:172.21.15.138:6789/0 conn(0x7f336806de20 legacy=0x7f3368071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.111+0000 7f336759e700 1 -- 172.21.15.112:0/3320100283 >> v1:172.21.15.112:6789/0 conn(0x7f33680aed30 legacy=0x7f336815b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.111+0000 7f336759e700 1 -- 172.21.15.112:0/3320100283 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3368070a40 con 0x7f336815b770 2023-08-27T11:17:59.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.111+0000 7f336759e700 1 --2- 172.21.15.112:0/3320100283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f336815b770 0x7f336815bbc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3360009dc0 tx=0x7f336000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d9b97768d82664 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:59.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.111+0000 7f3365d9b700 1 -- 172.21.15.112:0/3320100283 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336000e070 con 0x7f336815b770 2023-08-27T11:17:59.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.111+0000 7f3365d9b700 1 -- 172.21.15.112:0/3320100283 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3360003a20 con 0x7f336815b770 2023-08-27T11:17:59.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.111+0000 7f3365d9b700 1 -- 172.21.15.112:0/3320100283 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c001670 con 0x7f336815b770 2023-08-27T11:17:59.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.112+0000 7f336d12f700 1 -- 172.21.15.112:0/3320100283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f336815b770 msgr2=0x7f336815bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.112+0000 7f336d12f700 1 --2- 172.21.15.112:0/3320100283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f336815b770 0x7f336815bbc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3360009dc0 tx=0x7f336000b330 comp rx=0 tx=0).stop 2023-08-27T11:17:59.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.112+0000 7f336d12f700 1 -- 172.21.15.112:0/3320100283 shutdown_connections 2023-08-27T11:17:59.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.112+0000 7f336d12f700 1 --2- 172.21.15.112:0/3320100283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f336815b770 0x7f336815bbc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.112+0000 7f336d12f700 1 -- 172.21.15.112:0/3320100283 >> 172.21.15.112:0/3320100283 conn(0x7f33680b1380 msgr2=0x7f33680b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:59.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.112+0000 7f336d12f700 1 -- 172.21.15.112:0/3320100283 shutdown_connections 2023-08-27T11:17:59.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.112+0000 7f336d12f700 1 -- 172.21.15.112:0/3320100283 wait complete. 2023-08-27T11:17:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.113+0000 7f336d12f700 1 Processor -- start 2023-08-27T11:17:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.113+0000 7f336d12f700 1 -- start start 2023-08-27T11:17:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336d12f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f336806de20 0x7f336814f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336d12f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33680aed30 0x7f33680a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336d12f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f336815b770 0x7f33680a9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336d12f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33680ac600 con 0x7f336815b770 2023-08-27T11:17:59.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336d12f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33680ac8c0 con 0x7f33680aed30 2023-08-27T11:17:59.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336d12f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33680aa550 con 0x7f336806de20 2023-08-27T11:17:59.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f336815b770 0x7f33680a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f336815b770 0x7f33680a9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56618/0 (socket says 172.21.15.112:56618) 2023-08-27T11:17:59.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336759e700 1 -- 172.21.15.112:0/72187046 learned_addr learned my addr 172.21.15.112:0/72187046 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:59.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f3366d9d700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f336806de20 0x7f336814f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336659c700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33680aed30 0x7f33680a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336759e700 1 -- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f336806de20 msgr2=0x7f336814f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336759e700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f336806de20 0x7f336814f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336759e700 1 -- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33680aed30 msgr2=0x7f33680a91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336759e700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33680aed30 0x7f33680a91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336759e700 1 -- 172.21.15.112:0/72187046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3360009a70 con 0x7f336815b770 2023-08-27T11:17:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f3366d9d700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f336806de20 0x7f336814f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.114+0000 7f336659c700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33680aed30 0x7f33680a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.115+0000 7f336759e700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f336815b770 0x7f33680a9720 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3360009d90 tx=0x7f3360004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:59.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.115+0000 7f334f7fe700 1 -- 172.21.15.112:0/72187046 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33600043c0 con 0x7f336815b770 2023-08-27T11:17:59.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.115+0000 7f336d12f700 1 -- 172.21.15.112:0/72187046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33680aa830 con 0x7f336815b770 2023-08-27T11:17:59.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.115+0000 7f334f7fe700 1 -- 172.21.15.112:0/72187046 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3360004540 con 0x7f336815b770 2023-08-27T11:17:59.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.115+0000 7f334f7fe700 1 -- 172.21.15.112:0/72187046 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33600048f0 con 0x7f336815b770 2023-08-27T11:17:59.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.115+0000 7f336d12f700 1 -- 172.21.15.112:0/72187046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33680aadf0 con 0x7f336815b770 2023-08-27T11:17:59.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.116+0000 7f336d12f700 1 -- 172.21.15.112:0/72187046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33680aca50 con 0x7f336815b770 2023-08-27T11:17:59.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.116+0000 7f334f7fe700 1 -- 172.21.15.112:0/72187046 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f336001c460 con 0x7f336815b770 2023-08-27T11:17:59.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.117+0000 7f334f7fe700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f335406a0e0 0x7f335406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.117+0000 7f334f7fe700 1 -- 172.21.15.112:0/72187046 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3360089980 con 0x7f336815b770 2023-08-27T11:17:59.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.117+0000 7f3366d9d700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f335406a0e0 0x7f335406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.117+0000 7f3366d9d700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f335406a0e0 0x7f335406c5a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f335c001eb0 tx=0x7f335c001ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:59.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.121+0000 7f334f7fe700 1 -- 172.21.15.112:0/72187046 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3360058f10 con 0x7f336815b770 2023-08-27T11:17:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.387+0000 7f336d12f700 1 -- 172.21.15.112:0/72187046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f3368067a20 con 0x7f336815b770 2023-08-27T11:17:59.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.390+0000 7f334f7fe700 1 -- 172.21.15.112:0/72187046 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3360058aa0 con 0x7f336815b770 2023-08-27T11:17:59.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.392+0000 7f336d12f700 1 -- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f335406a0e0 msgr2=0x7f335406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.392+0000 7f336d12f700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f335406a0e0 0x7f335406c5a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f335c001eb0 tx=0x7f335c001ca0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.392+0000 7f336d12f700 1 -- 172.21.15.112:0/72187046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f336815b770 msgr2=0x7f33680a9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.392+0000 7f336d12f700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f336815b770 0x7f33680a9720 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3360009d90 tx=0x7f3360004ad0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.392+0000 7f336d12f700 1 -- 172.21.15.112:0/72187046 shutdown_connections 2023-08-27T11:17:59.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.392+0000 7f336d12f700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f335406a0e0 0x7f335406c5a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.392+0000 7f336d12f700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f336815b770 0x7f33680a9720 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.392+0000 7f336d12f700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f336806de20 0x7f336814f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.392+0000 7f336d12f700 1 --2- 172.21.15.112:0/72187046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33680aed30 0x7f33680a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.392+0000 7f336d12f700 1 -- 172.21.15.112:0/72187046 >> 172.21.15.112:0/72187046 conn(0x7f33680b1380 msgr2=0x7f3368150e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:59.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.393+0000 7f336d12f700 1 -- 172.21.15.112:0/72187046 shutdown_connections 2023-08-27T11:17:59.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.393+0000 7f336d12f700 1 -- 172.21.15.112:0/72187046 wait complete. 2023-08-27T11:17:59.407 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-08-27T11:17:59.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.528+0000 7f1f0e051700 1 Processor -- start 2023-08-27T11:17:59.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.528+0000 7f1f0e051700 1 -- start start 2023-08-27T11:17:59.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f0e051700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f080b1400 0x7f1f080cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f0e051700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1f080b17e0 0x7f1f0806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f0e051700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f08161850 con 0x7f1f08161ce0 2023-08-27T11:17:59.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f0e051700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0806e7d0 con 0x7f1f080b1400 2023-08-27T11:17:59.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f0e051700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f080cfc10 con 0x7f1f080b17e0 2023-08-27T11:17:59.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f077fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f080b1400 0x7f1f080cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f077fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f080b1400 0x7f1f080cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56622/0 (socket says 172.21.15.112:56622) 2023-08-27T11:17:59.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f077fe700 1 -- 172.21.15.112:0/587078252 learned_addr learned my addr 172.21.15.112:0/587078252 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:59.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f06ffd700 1 --2- 172.21.15.112:0/587078252 >> v2:172.21.15.138:3300/0 conn(0x7f1f080b17e0 0x7f1f0806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f077fe700 1 -- 172.21.15.112:0/587078252 >> v2:172.21.15.138:3300/0 conn(0x7f1f080b17e0 msgr2=0x7f1f0806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:59.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f077fe700 1 --2- 172.21.15.112:0/587078252 >> v2:172.21.15.138:3300/0 conn(0x7f1f080b17e0 0x7f1f0806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f077fe700 1 -- 172.21.15.112:0/587078252 >> v1:172.21.15.112:6789/0 conn(0x7f1f08161ce0 legacy=0x7f1f0806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f077fe700 1 -- 172.21.15.112:0/587078252 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f08161200 con 0x7f1f080b1400 2023-08-27T11:17:59.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f067fc700 1 -- 172.21.15.112:0/587078252 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140694596 0 0) 0x7f1f08161850 con 0x7f1f08161ce0 2023-08-27T11:17:59.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f06ffd700 1 --2- 172.21.15.112:0/587078252 >> v2:172.21.15.138:3300/0 conn(0x7f1f080b17e0 0x7f1f0806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:17:59.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f077fe700 1 --2- 172.21.15.112:0/587078252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f080b1400 0x7f1f080cf5a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1f080cc420 tx=0x7f1efc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84e4861f5ae7048 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:59.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f067fc700 1 -- 172.21.15.112:0/587078252 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00b870 con 0x7f1f080b1400 2023-08-27T11:17:59.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f067fc700 1 -- 172.21.15.112:0/587078252 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc00b9d0 con 0x7f1f080b1400 2023-08-27T11:17:59.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.529+0000 7f1f067fc700 1 -- 172.21.15.112:0/587078252 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00bd60 con 0x7f1f080b1400 2023-08-27T11:17:59.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.530+0000 7f1f0e051700 1 -- 172.21.15.112:0/587078252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f080b1400 msgr2=0x7f1f080cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.530+0000 7f1f0e051700 1 --2- 172.21.15.112:0/587078252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f080b1400 0x7f1f080cf5a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1f080cc420 tx=0x7f1efc009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.530+0000 7f1f0e051700 1 -- 172.21.15.112:0/587078252 shutdown_connections 2023-08-27T11:17:59.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.530+0000 7f1f0e051700 1 --2- 172.21.15.112:0/587078252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f080b1400 0x7f1f080cf5a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.530+0000 7f1f0e051700 1 --2- 172.21.15.112:0/587078252 >> v2:172.21.15.138:3300/0 conn(0x7f1f080b17e0 0x7f1f0806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.530+0000 7f1f0e051700 1 -- 172.21.15.112:0/587078252 >> 172.21.15.112:0/587078252 conn(0x7f1f080d0000 msgr2=0x7f1f080d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:59.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.530+0000 7f1f0e051700 1 -- 172.21.15.112:0/587078252 shutdown_connections 2023-08-27T11:17:59.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.530+0000 7f1f0e051700 1 -- 172.21.15.112:0/587078252 wait complete. 2023-08-27T11:17:59.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.531+0000 7f1f0e051700 1 Processor -- start 2023-08-27T11:17:59.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.531+0000 7f1f0e051700 1 -- start start 2023-08-27T11:17:59.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.532+0000 7f1f0e051700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f080b1400 0x7f1f080c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.532+0000 7f1f0e051700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f080b17e0 0x7f1f081f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.532+0000 7f1f0e051700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08161ce0 0x7f1f0806ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.532+0000 7f1f0e051700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080c4c90 con 0x7f1f080b1400 2023-08-27T11:17:59.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.532+0000 7f1f0e051700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080c4080 con 0x7f1f08161ce0 2023-08-27T11:17:59.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.532+0000 7f1f0e051700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080c4360 con 0x7f1f080b17e0 2023-08-27T11:17:59.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.532+0000 7f1f077fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f080b1400 0x7f1f080c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.532+0000 7f1f077fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f080b1400 0x7f1f080c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56628/0 (socket says 172.21.15.112:56628) 2023-08-27T11:17:59.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.532+0000 7f1f077fe700 1 -- 172.21.15.112:0/1554561401 learned_addr learned my addr 172.21.15.112:0/1554561401 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:59.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.532+0000 7f1f07fff700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08161ce0 0x7f1f0806ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.533+0000 7f1f06ffd700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f080b17e0 0x7f1f081f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.533+0000 7f1f077fe700 1 -- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f080b17e0 msgr2=0x7f1f081f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.533+0000 7f1f077fe700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f080b17e0 0x7f1f081f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.533+0000 7f1f077fe700 1 -- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08161ce0 msgr2=0x7f1f0806ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.533+0000 7f1f077fe700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08161ce0 0x7f1f0806ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.533+0000 7f1f077fe700 1 -- 172.21.15.112:0/1554561401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc009550 con 0x7f1f080b1400 2023-08-27T11:17:59.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.533+0000 7f1f077fe700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f080b1400 0x7f1f080c5950 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009c10 tx=0x7f1efc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:59.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.533+0000 7f1eeffff700 1 -- 172.21.15.112:0/1554561401 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc004310 con 0x7f1f080b1400 2023-08-27T11:17:59.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.533+0000 7f1eeffff700 1 -- 172.21.15.112:0/1554561401 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc01f070 con 0x7f1f080b1400 2023-08-27T11:17:59.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.533+0000 7f1eeffff700 1 -- 172.21.15.112:0/1554561401 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0036a0 con 0x7f1f080b1400 2023-08-27T11:17:59.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.533+0000 7f1f0e051700 1 -- 172.21.15.112:0/1554561401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f080c4640 con 0x7f1f080b1400 2023-08-27T11:17:59.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.533+0000 7f1f0e051700 1 -- 172.21.15.112:0/1554561401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f080c5e90 con 0x7f1f080b1400 2023-08-27T11:17:59.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.534+0000 7f1f0e051700 1 -- 172.21.15.112:0/1554561401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f08067a20 con 0x7f1f080b1400 2023-08-27T11:17:59.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.534+0000 7f1eeffff700 1 -- 172.21.15.112:0/1554561401 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1efc021030 con 0x7f1f080b1400 2023-08-27T11:17:59.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.535+0000 7f1eeffff700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ef406a0e0 0x7f1ef406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.535+0000 7f1eeffff700 1 -- 172.21.15.112:0/1554561401 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1efc0896c0 con 0x7f1f080b1400 2023-08-27T11:17:59.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.535+0000 7f1f06ffd700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ef406a0e0 0x7f1ef406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.535+0000 7f1f06ffd700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ef406a0e0 0x7f1ef406c5a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f08155a90 tx=0x7f1ef8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:59.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.539+0000 7f1eeffff700 1 -- 172.21.15.112:0/1554561401 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1efc055190 con 0x7f1f080b1400 2023-08-27T11:17:59.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.803+0000 7f1f0e051700 1 -- 172.21.15.112:0/1554561401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f1f0806db60 con 0x7f1f080b1400 2023-08-27T11:17:59.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.807+0000 7f1eeffff700 1 -- 172.21.15.112:0/1554561401 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1efc0587b0 con 0x7f1f080b1400 2023-08-27T11:17:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.809+0000 7f1f0e051700 1 -- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ef406a0e0 msgr2=0x7f1ef406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.809+0000 7f1f0e051700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ef406a0e0 0x7f1ef406c5a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f08155a90 tx=0x7f1ef8008040 comp rx=0 tx=0).stop 2023-08-27T11:17:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.809+0000 7f1f0e051700 1 -- 172.21.15.112:0/1554561401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f080b1400 msgr2=0x7f1f080c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.809+0000 7f1f0e051700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f080b1400 0x7f1f080c5950 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009c10 tx=0x7f1efc004750 comp rx=0 tx=0).stop 2023-08-27T11:17:59.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.809+0000 7f1f0e051700 1 -- 172.21.15.112:0/1554561401 shutdown_connections 2023-08-27T11:17:59.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.809+0000 7f1f0e051700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ef406a0e0 0x7f1ef406c5a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.809+0000 7f1f0e051700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08161ce0 0x7f1f0806ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.809+0000 7f1f0e051700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f080b1400 0x7f1f080c5950 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.809+0000 7f1f0e051700 1 --2- 172.21.15.112:0/1554561401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f080b17e0 0x7f1f081f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.809+0000 7f1f0e051700 1 -- 172.21.15.112:0/1554561401 >> 172.21.15.112:0/1554561401 conn(0x7f1f080d0000 msgr2=0x7f1f080cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:59.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.809+0000 7f1f0e051700 1 -- 172.21.15.112:0/1554561401 shutdown_connections 2023-08-27T11:17:59.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.809+0000 7f1f0e051700 1 -- 172.21.15.112:0/1554561401 wait complete. 2023-08-27T11:17:59.822 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-08-27T11:17:59.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.943+0000 7ffbc0cb8700 1 Processor -- start 2023-08-27T11:17:59.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.943+0000 7ffbc0cb8700 1 -- start start 2023-08-27T11:17:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.944+0000 7ffbc0cb8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffbbc06e150 0x7ffbbc06e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.944+0000 7ffbc0cb8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffbbc0a8960 0x7ffbbc0a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.944+0000 7ffbc0cb8700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbbc15eed0 con 0x7ffbbc141420 2023-08-27T11:17:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.944+0000 7ffbc0cb8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbbc06eb40 con 0x7ffbbc06e150 2023-08-27T11:17:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.944+0000 7ffbc0cb8700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbbc15f1b0 con 0x7ffbbc0a8960 2023-08-27T11:17:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.945+0000 7ffbb9d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffbbc06e150 0x7ffbbc06e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.945+0000 7ffbb9d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffbbc06e150 0x7ffbbc06e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56636/0 (socket says 172.21.15.112:56636) 2023-08-27T11:17:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.945+0000 7ffbb9d9b700 1 -- 172.21.15.112:0/1845624810 learned_addr learned my addr 172.21.15.112:0/1845624810 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.945+0000 7ffbbad9d700 1 --2- 172.21.15.112:0/1845624810 >> v2:172.21.15.138:3300/0 conn(0x7ffbbc0a8960 0x7ffbbc0a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.945+0000 7ffbb9d9b700 1 -- 172.21.15.112:0/1845624810 >> v2:172.21.15.138:3300/0 conn(0x7ffbbc0a8960 msgr2=0x7ffbbc0a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:17:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.945+0000 7ffbb9d9b700 1 --2- 172.21.15.112:0/1845624810 >> v2:172.21.15.138:3300/0 conn(0x7ffbbc0a8960 0x7ffbbc0a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.945+0000 7ffbb9d9b700 1 -- 172.21.15.112:0/1845624810 >> v1:172.21.15.138:6789/0 conn(0x7ffbbc141420 legacy=0x7ffbbc0aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.945+0000 7ffbb9d9b700 1 -- 172.21.15.112:0/1845624810 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc15f490 con 0x7ffbbc06e150 2023-08-27T11:17:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.945+0000 7ffbb9d9b700 1 --2- 172.21.15.112:0/1845624810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbbc06e150 0x7ffbbc06e600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc067060 tx=0x7ffbac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1173ca274c11f41 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.945+0000 7ffbb959a700 1 -- 172.21.15.112:0/1845624810 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac005e30 con 0x7ffbbc06e150 2023-08-27T11:17:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.945+0000 7ffbb959a700 1 -- 172.21.15.112:0/1845624810 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac004030 con 0x7ffbbc06e150 2023-08-27T11:17:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.945+0000 7ffbb959a700 1 -- 172.21.15.112:0/1845624810 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac0043e0 con 0x7ffbbc06e150 2023-08-27T11:17:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.946+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1845624810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbbc06e150 msgr2=0x7ffbbc06e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.946+0000 7ffbc0cb8700 1 --2- 172.21.15.112:0/1845624810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbbc06e150 0x7ffbbc06e600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc067060 tx=0x7ffbac009bd0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.946+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1845624810 shutdown_connections 2023-08-27T11:17:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.946+0000 7ffbc0cb8700 1 --2- 172.21.15.112:0/1845624810 >> v2:172.21.15.138:3300/0 conn(0x7ffbbc0a8960 0x7ffbbc0a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.946+0000 7ffbc0cb8700 1 --2- 172.21.15.112:0/1845624810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbbc06e150 0x7ffbbc06e600 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.946+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1845624810 >> 172.21.15.112:0/1845624810 conn(0x7ffbbc06cd30 msgr2=0x7ffbbc0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:17:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.946+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1845624810 shutdown_connections 2023-08-27T11:17:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.946+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1845624810 wait complete. 2023-08-27T11:17:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.947+0000 7ffbc0cb8700 1 Processor -- start 2023-08-27T11:17:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.947+0000 7ffbc0cb8700 1 -- start start 2023-08-27T11:17:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbc0cb8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbbc06e150 0x7ffbbc14c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbc0cb8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbbc0a8960 0x7ffbbc13b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbc0cb8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbbc141420 0x7ffbbc1495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbc0cb8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc13e150 con 0x7ffbbc141420 2023-08-27T11:17:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbc0cb8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc13e6d0 con 0x7ffbbc06e150 2023-08-27T11:17:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbc0cb8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc13e3f0 con 0x7ffbbc0a8960 2023-08-27T11:17:59.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbbad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbbc141420 0x7ffbbc1495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbbad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbbc141420 0x7ffbbc1495e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56644/0 (socket says 172.21.15.112:56644) 2023-08-27T11:17:59.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbbad9d700 1 -- 172.21.15.112:0/1455059879 learned_addr learned my addr 172.21.15.112:0/1455059879 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:17:59.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbba59c700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbbc06e150 0x7ffbbc14c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbb9d9b700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbbc0a8960 0x7ffbbc13b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbbad9d700 1 -- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbbc0a8960 msgr2=0x7ffbbc13b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbbad9d700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbbc0a8960 0x7ffbbc13b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbbad9d700 1 -- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbbc06e150 msgr2=0x7ffbbc14c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:17:59.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbbad9d700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbbc06e150 0x7ffbbc14c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:17:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbbad9d700 1 -- 172.21.15.112:0/1455059879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac009500 con 0x7ffbbc141420 2023-08-27T11:17:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.948+0000 7ffbb9d9b700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbbc0a8960 0x7ffbbc13b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:17:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.949+0000 7ffbbad9d700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbbc141420 0x7ffbbc1495e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400efc0 tx=0x7ffbb400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.949+0000 7ffbaaffd700 1 -- 172.21.15.112:0/1455059879 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4009ea0 con 0x7ffbbc141420 2023-08-27T11:17:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.949+0000 7ffbaaffd700 1 -- 172.21.15.112:0/1455059879 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb400f040 con 0x7ffbbc141420 2023-08-27T11:17:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.949+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1455059879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbbc136490 con 0x7ffbbc141420 2023-08-27T11:17:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.949+0000 7ffbaaffd700 1 -- 172.21.15.112:0/1455059879 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4014480 con 0x7ffbbc141420 2023-08-27T11:17:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.949+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1455059879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbbc13b9d0 con 0x7ffbbc141420 2023-08-27T11:17:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.950+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1455059879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbbc1369b0 con 0x7ffbbc141420 2023-08-27T11:17:59.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.950+0000 7ffbaaffd700 1 -- 172.21.15.112:0/1455059879 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffbb40145e0 con 0x7ffbbc141420 2023-08-27T11:17:59.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.951+0000 7ffbaaffd700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb9c06a0e0 0x7ffb9c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:17:59.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.951+0000 7ffbaaffd700 1 -- 172.21.15.112:0/1455059879 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ffbb4088a90 con 0x7ffbbc141420 2023-08-27T11:17:59.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.951+0000 7ffbba59c700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb9c06a0e0 0x7ffb9c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:17:59.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.951+0000 7ffbba59c700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb9c06a0e0 0x7ffb9c06c5a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0007fd0 tx=0x7ffbb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:17:59.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:17:59.955+0000 7ffbaaffd700 1 -- 172.21.15.112:0/1455059879 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffbbc1369b0 con 0x7ffbbc141420 2023-08-27T11:18:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.221+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1455059879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7ffbbc1369b0 con 0x7ffbbc141420 2023-08-27T11:18:00.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.224+0000 7ffbaaffd700 1 -- 172.21.15.112:0/1455059879 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7ffbbc1369b0 con 0x7ffbbc141420 2023-08-27T11:18:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.226+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb9c06a0e0 msgr2=0x7ffb9c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.226+0000 7ffbc0cb8700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb9c06a0e0 0x7ffb9c06c5a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0007fd0 tx=0x7ffbb0009040 comp rx=0 tx=0).stop 2023-08-27T11:18:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.226+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1455059879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbbc141420 msgr2=0x7ffbbc1495e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.226+0000 7ffbc0cb8700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbbc141420 0x7ffbbc1495e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400efc0 tx=0x7ffbb400c3a0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.227+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1455059879 shutdown_connections 2023-08-27T11:18:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.227+0000 7ffbc0cb8700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb9c06a0e0 0x7ffb9c06c5a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.227+0000 7ffbc0cb8700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbbc141420 0x7ffbbc1495e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.227+0000 7ffbc0cb8700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbbc06e150 0x7ffbbc14c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.227+0000 7ffbc0cb8700 1 --2- 172.21.15.112:0/1455059879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbbc0a8960 0x7ffbbc13b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.227+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1455059879 >> 172.21.15.112:0/1455059879 conn(0x7ffbbc06cd30 msgr2=0x7ffbbc0b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.227+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1455059879 shutdown_connections 2023-08-27T11:18:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.227+0000 7ffbc0cb8700 1 -- 172.21.15.112:0/1455059879 wait complete. 2023-08-27T11:18:00.240 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.359+0000 7f2bc8fe5700 1 Processor -- start 2023-08-27T11:18:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.359+0000 7f2bc8fe5700 1 -- start start 2023-08-27T11:18:00.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.360+0000 7f2bc8fe5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2bc4151060 0x7f2bc415e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:00.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.360+0000 7f2bc8fe5700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc40b02f0 con 0x7f2bc415e990 2023-08-27T11:18:00.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.360+0000 7f2bc8fe5700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc40b05d0 con 0x7f2bc406b4c0 2023-08-27T11:18:00.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.360+0000 7f2bc8fe5700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc40b08b0 con 0x7f2bc4151060 2023-08-27T11:18:00.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.360+0000 7f2bc259c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2bc4151060 0x7f2bc415e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:00.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.360+0000 7f2bc1d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2bc415e990 0x7f2bc406adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51030/0 (socket says 172.21.15.112:51030) 2023-08-27T11:18:00.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.360+0000 7f2bc259c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2bc4151060 0x7f2bc415e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56648/0 (socket says 172.21.15.112:56648) 2023-08-27T11:18:00.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.360+0000 7f2bc1d9b700 1 -- 172.21.15.112:0/467674871 learned_addr learned my addr 172.21.15.112:0/467674871 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:00.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.360+0000 7f2bc259c700 1 -- 172.21.15.112:0/467674871 >> v1:172.21.15.138:6789/0 conn(0x7f2bc406b4c0 legacy=0x7f2bc40cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:00.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.360+0000 7f2bc259c700 1 -- 172.21.15.112:0/467674871 >> v1:172.21.15.112:6789/0 conn(0x7f2bc415e990 legacy=0x7f2bc406adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.360+0000 7f2bc259c700 1 -- 172.21.15.112:0/467674871 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc40b0b90 con 0x7f2bc4151060 2023-08-27T11:18:00.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.360+0000 7f2bc159a700 1 -- 172.21.15.112:0/467674871 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377277232 0 0) 0x7f2bc40b02f0 con 0x7f2bc415e990 2023-08-27T11:18:00.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.360+0000 7f2bc259c700 1 --2- 172.21.15.112:0/467674871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc4151060 0x7f2bc415e450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009840 tx=0x7f2bb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d016c0e7cf2825 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:00.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.361+0000 7f2bc159a700 1 -- 172.21.15.112:0/467674871 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8004230 con 0x7f2bc4151060 2023-08-27T11:18:00.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.361+0000 7f2bc159a700 1 -- 172.21.15.112:0/467674871 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb8004390 con 0x7f2bc4151060 2023-08-27T11:18:00.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.361+0000 7f2bc159a700 1 -- 172.21.15.112:0/467674871 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80036a0 con 0x7f2bc4151060 2023-08-27T11:18:00.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.361+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/467674871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc4151060 msgr2=0x7f2bc415e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.361+0000 7f2bc8fe5700 1 --2- 172.21.15.112:0/467674871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc4151060 0x7f2bc415e450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009840 tx=0x7f2bb8009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.361+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/467674871 shutdown_connections 2023-08-27T11:18:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.361+0000 7f2bc8fe5700 1 --2- 172.21.15.112:0/467674871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc4151060 0x7f2bc415e450 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.361+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/467674871 >> 172.21.15.112:0/467674871 conn(0x7f2bc40d0000 msgr2=0x7f2bc40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:00.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.361+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/467674871 shutdown_connections 2023-08-27T11:18:00.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.361+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/467674871 wait complete. 2023-08-27T11:18:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.363+0000 7f2bc8fe5700 1 Processor -- start 2023-08-27T11:18:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.363+0000 7f2bc8fe5700 1 -- start start 2023-08-27T11:18:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.364+0000 7f2bc8fe5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bc406b4c0 0x7f2bc4151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.364+0000 7f2bc8fe5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2bc4151060 0x7f2bc414fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.364+0000 7f2bc8fe5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc415e990 0x7f2bc40c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.364+0000 7f2bc8fe5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc41537f0 con 0x7f2bc415e990 2023-08-27T11:18:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.364+0000 7f2bc8fe5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4152960 con 0x7f2bc406b4c0 2023-08-27T11:18:00.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.364+0000 7f2bc8fe5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4152510 con 0x7f2bc4151060 2023-08-27T11:18:00.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.364+0000 7f2bc2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc415e990 0x7f2bc40c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:00.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.364+0000 7f2bc2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc415e990 0x7f2bc40c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56658/0 (socket says 172.21.15.112:56658) 2023-08-27T11:18:00.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.364+0000 7f2bc2d9d700 1 -- 172.21.15.112:0/109185952 learned_addr learned my addr 172.21.15.112:0/109185952 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:00.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.364+0000 7f2bc259c700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bc406b4c0 0x7f2bc4151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:00.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.364+0000 7f2bc1d9b700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2bc4151060 0x7f2bc414fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:00.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.364+0000 7f2bc2d9d700 1 -- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2bc4151060 msgr2=0x7f2bc414fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.364+0000 7f2bc2d9d700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2bc4151060 0x7f2bc414fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.365+0000 7f2bc2d9d700 1 -- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bc406b4c0 msgr2=0x7f2bc4151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.365+0000 7f2bc2d9d700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bc406b4c0 0x7f2bc4151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.365+0000 7f2bc2d9d700 1 -- 172.21.15.112:0/109185952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb80094a0 con 0x7f2bc415e990 2023-08-27T11:18:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.365+0000 7f2bc2d9d700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc415e990 0x7f2bc40c70d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00b970 tx=0x7f2bbc00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.365+0000 7f2bb2ffd700 1 -- 172.21.15.112:0/109185952 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc00c9b0 con 0x7f2bc415e990 2023-08-27T11:18:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.365+0000 7f2bb2ffd700 1 -- 172.21.15.112:0/109185952 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bbc00cb10 con 0x7f2bc415e990 2023-08-27T11:18:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.365+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/109185952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc4152e50 con 0x7f2bc415e990 2023-08-27T11:18:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.365+0000 7f2bb2ffd700 1 -- 172.21.15.112:0/109185952 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc014440 con 0x7f2bc415e990 2023-08-27T11:18:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.365+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/109185952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc4153410 con 0x7f2bc415e990 2023-08-27T11:18:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.366+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/109185952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc4067a20 con 0x7f2bc415e990 2023-08-27T11:18:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.366+0000 7f2bb2ffd700 1 -- 172.21.15.112:0/109185952 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2bbc01d3b0 con 0x7f2bc415e990 2023-08-27T11:18:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.367+0000 7f2bb2ffd700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ba406a0e0 0x7f2ba406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.367+0000 7f2bb2ffd700 1 -- 172.21.15.112:0/109185952 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2bbc087e20 con 0x7f2bc415e990 2023-08-27T11:18:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.367+0000 7f2bc259c700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ba406a0e0 0x7f2ba406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.367+0000 7f2bc259c700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ba406a0e0 0x7f2ba406c5a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8000c00 tx=0x7f2bb801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.371+0000 7f2bb2ffd700 1 -- 172.21.15.112:0/109185952 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2bbc0538f0 con 0x7f2bc415e990 2023-08-27T11:18:00.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.560+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/109185952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bc4073700 con 0x7f2bc415e990 2023-08-27T11:18:00.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.561+0000 7f2bb2ffd700 1 -- 172.21.15.112:0/109185952 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+14605 (secure 0 0 0) 0x7f2bbc056f10 con 0x7f2bc415e990 2023-08-27T11:18:00.563 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:00.563 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:17:57.240877+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:00.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.563+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ba406a0e0 msgr2=0x7f2ba406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.563+0000 7f2bc8fe5700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ba406a0e0 0x7f2ba406c5a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8000c00 tx=0x7f2bb801c040 comp rx=0 tx=0).stop 2023-08-27T11:18:00.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.563+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/109185952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc415e990 msgr2=0x7f2bc40c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.563+0000 7f2bc8fe5700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc415e990 0x7f2bc40c70d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00b970 tx=0x7f2bbc00bd30 comp rx=0 tx=0).stop 2023-08-27T11:18:00.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.563+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/109185952 shutdown_connections 2023-08-27T11:18:00.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.563+0000 7f2bc8fe5700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ba406a0e0 0x7f2ba406c5a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.563+0000 7f2bc8fe5700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc415e990 0x7f2bc40c70d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.563+0000 7f2bc8fe5700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bc406b4c0 0x7f2bc4151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.563+0000 7f2bc8fe5700 1 --2- 172.21.15.112:0/109185952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2bc4151060 0x7f2bc414fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.563+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/109185952 >> 172.21.15.112:0/109185952 conn(0x7f2bc40d0000 msgr2=0x7f2bc40ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:00.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.563+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/109185952 shutdown_connections 2023-08-27T11:18:00.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.563+0000 7f2bc8fe5700 1 -- 172.21.15.112:0/109185952 wait complete. 2023-08-27T11:18:00.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:18:00.578 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:18:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.700+0000 7f67f3f6c700 1 Processor -- start 2023-08-27T11:18:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.700+0000 7f67f3f6c700 1 -- start start 2023-08-27T11:18:00.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f3f6c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67ec15b7f0 0x7f67ec072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:00.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f3f6c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec15ca50 con 0x7f67ec06e4c0 2023-08-27T11:18:00.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f3f6c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec15be20 con 0x7f67ec158ed0 2023-08-27T11:18:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f3f6c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec072540 con 0x7f67ec15b7f0 2023-08-27T11:18:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f1507700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f67ec06e4c0 0x7f67ec15cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51046/0 (socket says 172.21.15.112:51046) 2023-08-27T11:18:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f1507700 1 -- 172.21.15.112:0/3572958842 learned_addr learned my addr 172.21.15.112:0/3572958842 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777426292 0 0) 0x7f67ec15ca50 con 0x7f67ec06e4c0 2023-08-27T11:18:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d80035b0 con 0x7f67ec06e4c0 2023-08-27T11:18:00.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f2509700 1 --2- 172.21.15.112:0/3572958842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67ec15b7f0 0x7f67ec072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:00.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3825290167 0 0) 0x7f67d80035b0 con 0x7f67ec06e4c0 2023-08-27T11:18:00.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67ec15b7f0 msgr2=0x7f67ec072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:00.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 --2- 172.21.15.112:0/3572958842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67ec15b7f0 0x7f67ec072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 >> v1:172.21.15.138:6789/0 conn(0x7f67ec158ed0 legacy=0x7f67ec06ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec15c100 con 0x7f67ec06e4c0 2023-08-27T11:18:00.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f67e8001670 con 0x7f67ec06e4c0 2023-08-27T11:18:00.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 >> v1:172.21.15.112:6789/0 conn(0x7f67ec06e4c0 legacy=0x7f67ec15cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 --2- 172.21.15.112:0/3572958842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67ec15b7f0 0x7f67d8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:00.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 --2- 172.21.15.112:0/3572958842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d8003fb0 0x7f67d8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 --2- 172.21.15.112:0/3572958842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d8004d90 0x7f67d8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec072540 con 0x7f67d8004d90 2023-08-27T11:18:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d800baf0 con 0x7f67d8003fb0 2023-08-27T11:18:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d800bcb0 con 0x7f67ec15b7f0 2023-08-27T11:18:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.701+0000 7f67f2509700 1 --2- 172.21.15.112:0/3572958842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d8004d90 0x7f67d8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:00.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f1d08700 1 --2- 172.21.15.112:0/3572958842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67ec15b7f0 0x7f67d8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:00.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f1507700 1 --2- 172.21.15.112:0/3572958842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d8003fb0 0x7f67d8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:00.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f2509700 1 -- 172.21.15.112:0/3572958842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67ec15b7f0 msgr2=0x7f67d8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:00.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f2509700 1 --2- 172.21.15.112:0/3572958842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67ec15b7f0 0x7f67d8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f2509700 1 -- 172.21.15.112:0/3572958842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d8003fb0 msgr2=0x7f67d8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:00.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f2509700 1 --2- 172.21.15.112:0/3572958842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d8003fb0 0x7f67d8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f2509700 1 -- 172.21.15.112:0/3572958842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d80035b0 con 0x7f67d8004d90 2023-08-27T11:18:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f1507700 1 --2- 172.21.15.112:0/3572958842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d8003fb0 0x7f67d8004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f2509700 1 --2- 172.21.15.112:0/3572958842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d8004d90 0x7f67d8005210 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009fc0 tx=0x7f67d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e718b3ee714e4af server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e400ec30 con 0x7f67d8004d90 2023-08-27T11:18:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e400f040 con 0x7f67d8004d90 2023-08-27T11:18:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f0d06700 1 -- 172.21.15.112:0/3572958842 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e4014480 con 0x7f67d8004d90 2023-08-27T11:18:00.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f3f6c700 1 -- 172.21.15.112:0/3572958842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d8004d90 msgr2=0x7f67d8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f3f6c700 1 --2- 172.21.15.112:0/3572958842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d8004d90 0x7f67d8005210 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009fc0 tx=0x7f67d8005880 comp rx=0 tx=0).stop 2023-08-27T11:18:00.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f2509700 1 -- 172.21.15.112:0/3572958842 reap_dead start 2023-08-27T11:18:00.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.702+0000 7f67f3f6c700 1 -- 172.21.15.112:0/3572958842 shutdown_connections 2023-08-27T11:18:00.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.703+0000 7f67f3f6c700 1 -- 172.21.15.112:0/3572958842 >> 172.21.15.112:0/3572958842 conn(0x7f67ec0cc240 msgr2=0x7f67ec0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:00.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.703+0000 7f67f3f6c700 1 -- 172.21.15.112:0/3572958842 shutdown_connections 2023-08-27T11:18:00.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.703+0000 7f67f3f6c700 1 -- 172.21.15.112:0/3572958842 wait complete. 2023-08-27T11:18:00.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.704+0000 7f67f3f6c700 1 Processor -- start 2023-08-27T11:18:00.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.704+0000 7f67f3f6c700 1 -- start start 2023-08-27T11:18:00.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f3f6c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d8004d90 0x7f67ec0b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:00.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f3f6c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ec06f790 0x7f67ec071070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:00.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f3f6c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67ec0adcb0 0x7f67ec0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:00.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f3f6c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec0b0590 con 0x7f67ec06f790 2023-08-27T11:18:00.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f3f6c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec13a9d0 con 0x7f67ec0adcb0 2023-08-27T11:18:00.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f3f6c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec06fd00 con 0x7f67d8004d90 2023-08-27T11:18:00.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f1507700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ec06f790 0x7f67ec071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:00.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f1507700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ec06f790 0x7f67ec071070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56676/0 (socket says 172.21.15.112:56676) 2023-08-27T11:18:00.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f1507700 1 -- 172.21.15.112:0/1524209945 learned_addr learned my addr 172.21.15.112:0/1524209945 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:00.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f1d08700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d8004d90 0x7f67ec0b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f2509700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67ec0adcb0 0x7f67ec0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f1507700 1 -- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d8004d90 msgr2=0x7f67ec0b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f1507700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d8004d90 0x7f67ec0b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f1507700 1 -- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67ec0adcb0 msgr2=0x7f67ec0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f1507700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67ec0adcb0 0x7f67ec0ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f1507700 1 -- 172.21.15.112:0/1524209945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec06ffe0 con 0x7f67ec06f790 2023-08-27T11:18:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f2509700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67ec0adcb0 0x7f67ec0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f1507700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ec06f790 0x7f67ec071070 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f67e800e8f0 tx=0x7f67e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67e27fc700 1 -- 172.21.15.112:0/1524209945 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e800c780 con 0x7f67ec06f790 2023-08-27T11:18:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67e27fc700 1 -- 172.21.15.112:0/1524209945 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e800c8e0 con 0x7f67ec06f790 2023-08-27T11:18:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f3f6c700 1 -- 172.21.15.112:0/1524209945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ec0702c0 con 0x7f67ec06f790 2023-08-27T11:18:00.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67e27fc700 1 -- 172.21.15.112:0/1524209945 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e800cc50 con 0x7f67ec06f790 2023-08-27T11:18:00.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.705+0000 7f67f3f6c700 1 -- 172.21.15.112:0/1524209945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ec0715b0 con 0x7f67ec06f790 2023-08-27T11:18:00.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.706+0000 7f67f3f6c700 1 -- 172.21.15.112:0/1524209945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ec071770 con 0x7f67ec06f790 2023-08-27T11:18:00.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.707+0000 7f67e27fc700 1 -- 172.21.15.112:0/1524209945 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f67e801d460 con 0x7f67ec06f790 2023-08-27T11:18:00.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.707+0000 7f67e27fc700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f67d006a0e0 0x7f67d006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:00.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.707+0000 7f67e27fc700 1 -- 172.21.15.112:0/1524209945 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f67e8088110 con 0x7f67ec06f790 2023-08-27T11:18:00.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.707+0000 7f67f1d08700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f67d006a0e0 0x7f67d006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:00.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.708+0000 7f67f1d08700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f67d006a0e0 0x7f67d006c5a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67dc007fd0 tx=0x7f67dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:00.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.712+0000 7f67e27fc700 1 -- 172.21.15.112:0/1524209945 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f67e8053b50 con 0x7f67ec06f790 2023-08-27T11:18:00.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.899+0000 7f67f3f6c700 1 -- 172.21.15.112:0/1524209945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f67ec067e90 con 0x7f67ec06f790 2023-08-27T11:18:00.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.899+0000 7f67e27fc700 1 -- 172.21.15.112:0/1524209945 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f67e800cdb0 con 0x7f67ec06f790 2023-08-27T11:18:00.901 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:18:00.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.901+0000 7f67f3f6c700 1 -- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f67d006a0e0 msgr2=0x7f67d006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.901+0000 7f67f3f6c700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f67d006a0e0 0x7f67d006c5a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67dc007fd0 tx=0x7f67dc009040 comp rx=0 tx=0).stop 2023-08-27T11:18:00.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.901+0000 7f67f3f6c700 1 -- 172.21.15.112:0/1524209945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ec06f790 msgr2=0x7f67ec071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:00.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.901+0000 7f67f3f6c700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ec06f790 0x7f67ec071070 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f67e800e8f0 tx=0x7f67e800ecb0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.901+0000 7f67f3f6c700 1 -- 172.21.15.112:0/1524209945 shutdown_connections 2023-08-27T11:18:00.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.901+0000 7f67f3f6c700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f67d006a0e0 0x7f67d006c5a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.901+0000 7f67f3f6c700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67ec0adcb0 0x7f67ec0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.901+0000 7f67f3f6c700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d8004d90 0x7f67ec0b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.901+0000 7f67f3f6c700 1 --2- 172.21.15.112:0/1524209945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ec06f790 0x7f67ec071070 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:00.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.901+0000 7f67f3f6c700 1 -- 172.21.15.112:0/1524209945 >> 172.21.15.112:0/1524209945 conn(0x7f67ec0cc240 msgr2=0x7f67ec06d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:00.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.901+0000 7f67f3f6c700 1 -- 172.21.15.112:0/1524209945 shutdown_connections 2023-08-27T11:18:00.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:00.901+0000 7f67f3f6c700 1 -- 172.21.15.112:0/1524209945 wait complete. 2023-08-27T11:18:00.914 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:18:01.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.036+0000 7fa703106700 1 Processor -- start 2023-08-27T11:18:01.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.036+0000 7fa703106700 1 -- start start 2023-08-27T11:18:01.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.037+0000 7fa703106700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6fc144b80 0x7fa6fc0b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.037+0000 7fa703106700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6fc06c6e0 0x7fa6fc0cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.037+0000 7fa703106700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc0cef20 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.037+0000 7fa703106700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6fc0b4b40 con 0x7fa6fc06c6e0 2023-08-27T11:18:01.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.037+0000 7fa703106700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0b42a0 con 0x7fa6fc144b80 2023-08-27T11:18:01.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.037+0000 7fa700ea2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6fc144b80 0x7fa6fc0b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.037+0000 7fa7016a3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6fc06c6e0 0x7fa6fc0cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.037+0000 7fa6fbfff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa6fc0b3de0 0x7fa6fc06bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:58588/0 (socket says 172.21.15.112:58588) 2023-08-27T11:18:01.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.037+0000 7fa6fbfff700 1 -- 172.21.15.112:0/1072156650 learned_addr learned my addr 172.21.15.112:0/1072156650 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:01.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.038+0000 7fa700ea2700 1 -- 172.21.15.112:0/1072156650 >> v2:172.21.15.138:3300/0 conn(0x7fa6fc06c6e0 msgr2=0x7fa6fc0cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:01.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.038+0000 7fa700ea2700 1 --2- 172.21.15.112:0/1072156650 >> v2:172.21.15.138:3300/0 conn(0x7fa6fc06c6e0 0x7fa6fc0cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.038+0000 7fa700ea2700 1 -- 172.21.15.112:0/1072156650 >> v1:172.21.15.138:6789/0 conn(0x7fa6fc0b3de0 legacy=0x7fa6fc06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.038+0000 7fa700ea2700 1 -- 172.21.15.112:0/1072156650 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc0b4580 con 0x7fa6fc144b80 2023-08-27T11:18:01.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.038+0000 7fa6fb7fe700 1 -- 172.21.15.112:0/1072156650 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964368597 0 0) 0x7fa6fc0cef20 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.038+0000 7fa7016a3700 1 --2- 172.21.15.112:0/1072156650 >> v2:172.21.15.138:3300/0 conn(0x7fa6fc06c6e0 0x7fa6fc0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:01.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.038+0000 7fa700ea2700 1 --2- 172.21.15.112:0/1072156650 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6fc144b80 0x7fa6fc0b6c00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009840 tx=0x7fa6f0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0ba20b954902b2f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:01.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.038+0000 7fa6fb7fe700 1 -- 172.21.15.112:0/1072156650 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0011830 con 0x7fa6fc144b80 2023-08-27T11:18:01.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.038+0000 7fa6fb7fe700 1 -- 172.21.15.112:0/1072156650 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f0011990 con 0x7fa6fc144b80 2023-08-27T11:18:01.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.038+0000 7fa6fb7fe700 1 -- 172.21.15.112:0/1072156650 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0011d20 con 0x7fa6fc144b80 2023-08-27T11:18:01.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.039+0000 7fa703106700 1 -- 172.21.15.112:0/1072156650 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6fc144b80 msgr2=0x7fa6fc0b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.039+0000 7fa703106700 1 --2- 172.21.15.112:0/1072156650 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6fc144b80 0x7fa6fc0b6c00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009840 tx=0x7fa6f0009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:01.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.039+0000 7fa703106700 1 -- 172.21.15.112:0/1072156650 shutdown_connections 2023-08-27T11:18:01.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.039+0000 7fa703106700 1 --2- 172.21.15.112:0/1072156650 >> v2:172.21.15.138:3300/0 conn(0x7fa6fc06c6e0 0x7fa6fc0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.039+0000 7fa703106700 1 --2- 172.21.15.112:0/1072156650 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6fc144b80 0x7fa6fc0b6c00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.039+0000 7fa703106700 1 -- 172.21.15.112:0/1072156650 >> 172.21.15.112:0/1072156650 conn(0x7fa6fc0cdc50 msgr2=0x7fa6fc0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:01.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.039+0000 7fa703106700 1 -- 172.21.15.112:0/1072156650 shutdown_connections 2023-08-27T11:18:01.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.039+0000 7fa703106700 1 -- 172.21.15.112:0/1072156650 wait complete. 2023-08-27T11:18:01.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.040+0000 7fa703106700 1 Processor -- start 2023-08-27T11:18:01.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.040+0000 7fa703106700 1 -- start start 2023-08-27T11:18:01.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa703106700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6fc06c6e0 0x7fa6fc1f0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa703106700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6fc0b3de0 0x7fa6fc1f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa703106700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6fc144b80 0x7fa6fc156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa703106700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0cd5d0 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa703106700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0c9390 con 0x7fa6fc144b80 2023-08-27T11:18:01.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa703106700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0c9670 con 0x7fa6fc06c6e0 2023-08-27T11:18:01.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa6fbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6fc0b3de0 0x7fa6fc1f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa6fbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6fc0b3de0 0x7fa6fc1f39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56688/0 (socket says 172.21.15.112:56688) 2023-08-27T11:18:01.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa6fbfff700 1 -- 172.21.15.112:0/930215920 learned_addr learned my addr 172.21.15.112:0/930215920 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:01.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa7016a3700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6fc144b80 0x7fa6fc156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa700ea2700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6fc06c6e0 0x7fa6fc1f0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa6fbfff700 1 -- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6fc06c6e0 msgr2=0x7fa6fc1f0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa6fbfff700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6fc06c6e0 0x7fa6fc1f0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa6fbfff700 1 -- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6fc144b80 msgr2=0x7fa6fc156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa6fbfff700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6fc144b80 0x7fa6fc156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa6fbfff700 1 -- 172.21.15.112:0/930215920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f4009a70 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.041+0000 7fa6fbfff700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6fc0b3de0 0x7fa6fc1f39a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80011e0 tx=0x7fa6e800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.042+0000 7fa6f97fa700 1 -- 172.21.15.112:0/930215920 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8004c80 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.042+0000 7fa6f97fa700 1 -- 172.21.15.112:0/930215920 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8004e00 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.042+0000 7fa6f97fa700 1 -- 172.21.15.112:0/930215920 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8007310 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.042+0000 7fa703106700 1 -- 172.21.15.112:0/930215920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f00094a0 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.042+0000 7fa703106700 1 -- 172.21.15.112:0/930215920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc0c9c30 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.043+0000 7fa703106700 1 -- 172.21.15.112:0/930215920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6fc0c9e50 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.043+0000 7fa6f97fa700 1 -- 172.21.15.112:0/930215920 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa6e8007470 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.043+0000 7fa6f97fa700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa6e006a0e0 0x7fa6e006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.044+0000 7fa6f97fa700 1 -- 172.21.15.112:0/930215920 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa6e8087dd0 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.044+0000 7fa700ea2700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa6e006a0e0 0x7fa6e006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.044+0000 7fa700ea2700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa6e006a0e0 0x7fa6e006c5a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00090b0 tx=0x7fa6f001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:01.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.048+0000 7fa6f97fa700 1 -- 172.21.15.112:0/930215920 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa6e80538a0 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.237+0000 7fa703106700 1 -- 172.21.15.112:0/930215920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6fc067a20 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.237+0000 7fa6f97fa700 1 -- 172.21.15.112:0/930215920 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6e8056ec0 con 0x7fa6fc0b3de0 2023-08-27T11:18:01.239 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:18:01.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.239+0000 7fa703106700 1 -- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa6e006a0e0 msgr2=0x7fa6e006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.239+0000 7fa703106700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa6e006a0e0 0x7fa6e006c5a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00090b0 tx=0x7fa6f001c040 comp rx=0 tx=0).stop 2023-08-27T11:18:01.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.239+0000 7fa703106700 1 -- 172.21.15.112:0/930215920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6fc0b3de0 msgr2=0x7fa6fc1f39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.239+0000 7fa703106700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6fc0b3de0 0x7fa6fc1f39a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80011e0 tx=0x7fa6e800ca30 comp rx=0 tx=0).stop 2023-08-27T11:18:01.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.239+0000 7fa703106700 1 -- 172.21.15.112:0/930215920 shutdown_connections 2023-08-27T11:18:01.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.239+0000 7fa703106700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa6e006a0e0 0x7fa6e006c5a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.239+0000 7fa703106700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6fc144b80 0x7fa6fc156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.239+0000 7fa703106700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6fc06c6e0 0x7fa6fc1f0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.239+0000 7fa703106700 1 --2- 172.21.15.112:0/930215920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6fc0b3de0 0x7fa6fc1f39a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.239+0000 7fa703106700 1 -- 172.21.15.112:0/930215920 >> 172.21.15.112:0/930215920 conn(0x7fa6fc0cdc50 msgr2=0x7fa6fc0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:01.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.239+0000 7fa703106700 1 -- 172.21.15.112:0/930215920 shutdown_connections 2023-08-27T11:18:01.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.239+0000 7fa703106700 1 -- 172.21.15.112:0/930215920 wait complete. 2023-08-27T11:18:01.253 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:18:01.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.371+0000 7f258b7d3700 1 Processor -- start 2023-08-27T11:18:01.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.371+0000 7f258b7d3700 1 -- start start 2023-08-27T11:18:01.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f258b7d3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f25840b1400 0x7f25840cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f258b7d3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f25840b17e0 0x7f258406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f258b7d3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2584161850 con 0x7f2584161ce0 2023-08-27T11:18:01.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f258b7d3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258406e7d0 con 0x7f25840b17e0 2023-08-27T11:18:01.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f258b7d3700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25840cfc10 con 0x7f25840b1400 2023-08-27T11:18:01.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f2588d6e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f25840b17e0 0x7f258406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f2588d6e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f25840b17e0 0x7f258406e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56696/0 (socket says 172.21.15.112:56696) 2023-08-27T11:18:01.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f2589d70700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2584161ce0 0x7f258406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51080/0 (socket says 172.21.15.112:51080) 2023-08-27T11:18:01.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f2588d6e700 1 -- 172.21.15.112:0/744269694 learned_addr learned my addr 172.21.15.112:0/744269694 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:01.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f258956f700 1 --2- 172.21.15.112:0/744269694 >> v2:172.21.15.138:3300/0 conn(0x7f25840b1400 0x7f25840cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f2588d6e700 1 -- 172.21.15.112:0/744269694 >> v2:172.21.15.138:3300/0 conn(0x7f25840b1400 msgr2=0x7f25840cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:01.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f2588d6e700 1 --2- 172.21.15.112:0/744269694 >> v2:172.21.15.138:3300/0 conn(0x7f25840b1400 0x7f25840cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f2588d6e700 1 -- 172.21.15.112:0/744269694 >> v1:172.21.15.112:6789/0 conn(0x7f2584161ce0 legacy=0x7f258406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f257bfff700 1 -- 172.21.15.112:0/744269694 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699032183 0 0) 0x7f2584161850 con 0x7f2584161ce0 2023-08-27T11:18:01.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f2588d6e700 1 -- 172.21.15.112:0/744269694 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2584161200 con 0x7f25840b17e0 2023-08-27T11:18:01.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f2588d6e700 1 --2- 172.21.15.112:0/744269694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b17e0 0x7f258406e290 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2584070370 tx=0x7f2574009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f37788448ee24b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:01.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f257bfff700 1 -- 172.21.15.112:0/744269694 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574011830 con 0x7f25840b17e0 2023-08-27T11:18:01.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.372+0000 7f257bfff700 1 -- 172.21.15.112:0/744269694 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2574011990 con 0x7f25840b17e0 2023-08-27T11:18:01.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.373+0000 7f257bfff700 1 -- 172.21.15.112:0/744269694 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574011d20 con 0x7f25840b17e0 2023-08-27T11:18:01.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.373+0000 7f258b7d3700 1 -- 172.21.15.112:0/744269694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b17e0 msgr2=0x7f258406e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.373+0000 7f258b7d3700 1 --2- 172.21.15.112:0/744269694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b17e0 0x7f258406e290 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2584070370 tx=0x7f2574009bd0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.373+0000 7f258b7d3700 1 -- 172.21.15.112:0/744269694 shutdown_connections 2023-08-27T11:18:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.373+0000 7f258b7d3700 1 --2- 172.21.15.112:0/744269694 >> v2:172.21.15.138:3300/0 conn(0x7f25840b1400 0x7f25840cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.373+0000 7f258b7d3700 1 --2- 172.21.15.112:0/744269694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b17e0 0x7f258406e290 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.373+0000 7f258b7d3700 1 -- 172.21.15.112:0/744269694 >> 172.21.15.112:0/744269694 conn(0x7f25840d0000 msgr2=0x7f25840d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:01.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.373+0000 7f258b7d3700 1 -- 172.21.15.112:0/744269694 shutdown_connections 2023-08-27T11:18:01.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.373+0000 7f258b7d3700 1 -- 172.21.15.112:0/744269694 wait complete. 2023-08-27T11:18:01.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.374+0000 7f258b7d3700 1 Processor -- start 2023-08-27T11:18:01.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.374+0000 7f258b7d3700 1 -- start start 2023-08-27T11:18:01.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f258b7d3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25840b1400 0x7f25841f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f258b7d3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b17e0 0x7f2584152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f258b7d3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2584161ce0 0x7f25840c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f258b7d3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25840c5490 con 0x7f25840b17e0 2023-08-27T11:18:01.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f258b7d3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841f3e80 con 0x7f2584161ce0 2023-08-27T11:18:01.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f258b7d3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25840c56c0 con 0x7f25840b1400 2023-08-27T11:18:01.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f2588d6e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b17e0 0x7f2584152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f2588d6e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b17e0 0x7f2584152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56702/0 (socket says 172.21.15.112:56702) 2023-08-27T11:18:01.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f2588d6e700 1 -- 172.21.15.112:0/861649587 learned_addr learned my addr 172.21.15.112:0/861649587 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:01.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f258956f700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25840b1400 0x7f25841f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f2589d70700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2584161ce0 0x7f25840c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f2588d6e700 1 -- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25840b1400 msgr2=0x7f25841f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f2588d6e700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25840b1400 0x7f25841f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f2588d6e700 1 -- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2584161ce0 msgr2=0x7f25840c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f2588d6e700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2584161ce0 0x7f25840c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.375+0000 7f2588d6e700 1 -- 172.21.15.112:0/861649587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2574009550 con 0x7f25840b17e0 2023-08-27T11:18:01.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.376+0000 7f2589d70700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2584161ce0 0x7f25840c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:01.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.376+0000 7f2588d6e700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b17e0 0x7f2584152650 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2574009c10 tx=0x7f2574004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:01.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.376+0000 7f2579ffb700 1 -- 172.21.15.112:0/861649587 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25740036a0 con 0x7f25840b17e0 2023-08-27T11:18:01.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.376+0000 7f2579ffb700 1 -- 172.21.15.112:0/861649587 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2574003800 con 0x7f25840b17e0 2023-08-27T11:18:01.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.376+0000 7f258b7d3700 1 -- 172.21.15.112:0/861649587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25840c59a0 con 0x7f25840b17e0 2023-08-27T11:18:01.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.376+0000 7f2579ffb700 1 -- 172.21.15.112:0/861649587 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574003b70 con 0x7f25840b17e0 2023-08-27T11:18:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.376+0000 7f258b7d3700 1 -- 172.21.15.112:0/861649587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25840c5f60 con 0x7f25840b17e0 2023-08-27T11:18:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.377+0000 7f258b7d3700 1 -- 172.21.15.112:0/861649587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2584067a20 con 0x7f25840b17e0 2023-08-27T11:18:01.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.377+0000 7f2579ffb700 1 -- 172.21.15.112:0/861649587 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2574003cd0 con 0x7f25840b17e0 2023-08-27T11:18:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.378+0000 7f2579ffb700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f256806a0e0 0x7f256806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.378+0000 7f2579ffb700 1 -- 172.21.15.112:0/861649587 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2574021030 con 0x7f25840b17e0 2023-08-27T11:18:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.378+0000 7f258956f700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f256806a0e0 0x7f256806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.378+0000 7f258956f700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f256806a0e0 0x7f256806c5a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2580009b70 tx=0x7f2580008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:01.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.382+0000 7f2579ffb700 1 -- 172.21.15.112:0/861649587 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2574055790 con 0x7f25840b17e0 2023-08-27T11:18:01.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.576+0000 7f258b7d3700 1 -- 172.21.15.112:0/861649587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f258406db60 con 0x7f25840b17e0 2023-08-27T11:18:01.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.576+0000 7f2579ffb700 1 -- 172.21.15.112:0/861649587 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+126 (secure 0 0 0) 0x7f257402b090 con 0x7f25840b17e0 2023-08-27T11:18:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:01.578 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:18:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2023-08-27T11:18:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "rbd" 2023-08-27T11:18:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 2, 2023-08-27T11:18:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2023-08-27T11:18:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:01.580 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:18:01.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.578+0000 7f258b7d3700 1 -- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f256806a0e0 msgr2=0x7f256806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.578+0000 7f258b7d3700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f256806a0e0 0x7f256806c5a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2580009b70 tx=0x7f2580008040 comp rx=0 tx=0).stop 2023-08-27T11:18:01.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.578+0000 7f258b7d3700 1 -- 172.21.15.112:0/861649587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b17e0 msgr2=0x7f2584152650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.578+0000 7f258b7d3700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b17e0 0x7f2584152650 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2574009c10 tx=0x7f2574004750 comp rx=0 tx=0).stop 2023-08-27T11:18:01.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.578+0000 7f258b7d3700 1 -- 172.21.15.112:0/861649587 shutdown_connections 2023-08-27T11:18:01.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.578+0000 7f258b7d3700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f256806a0e0 0x7f256806c5a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.578+0000 7f258b7d3700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2584161ce0 0x7f25840c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.578+0000 7f258b7d3700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25840b1400 0x7f25841f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.578+0000 7f258b7d3700 1 --2- 172.21.15.112:0/861649587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b17e0 0x7f2584152650 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.579+0000 7f258b7d3700 1 -- 172.21.15.112:0/861649587 >> 172.21.15.112:0/861649587 conn(0x7f25840d0000 msgr2=0x7f258406b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:01.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.579+0000 7f258b7d3700 1 -- 172.21.15.112:0/861649587 shutdown_connections 2023-08-27T11:18:01.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.579+0000 7f258b7d3700 1 -- 172.21.15.112:0/861649587 wait complete. 2023-08-27T11:18:01.592 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:18:01.593 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:18:01.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.715+0000 7f79b3c5e700 1 Processor -- start 2023-08-27T11:18:01.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.715+0000 7f79b3c5e700 1 -- start start 2023-08-27T11:18:01.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.716+0000 7f79b3c5e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79ac146d50 0x7f79ac0b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.716+0000 7f79b3c5e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f79ac0b5750 0x7f79ac0d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.716+0000 7f79b3c5e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ac0b7f60 con 0x7f79ac0b69b0 2023-08-27T11:18:01.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.716+0000 7f79b3c5e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ac0b6ec0 con 0x7f79ac0b5750 2023-08-27T11:18:01.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.716+0000 7f79b3c5e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac0b71a0 con 0x7f79ac146d50 2023-08-27T11:18:01.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.716+0000 7f79b21fb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f79ac0b5750 0x7f79ac0d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.716+0000 7f79b11f9700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f79ac0b69b0 0x7f79ac0b5040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51090/0 (socket says 172.21.15.112:51090) 2023-08-27T11:18:01.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.716+0000 7f79b11f9700 1 -- 172.21.15.112:0/3497508217 learned_addr learned my addr 172.21.15.112:0/3497508217 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:01.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.717+0000 7f79b21fb700 1 -- 172.21.15.112:0/3497508217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79ac146d50 msgr2=0x7f79ac0b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:01.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.717+0000 7f79b19fa700 1 --2- 172.21.15.112:0/3497508217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79ac146d50 0x7f79ac0b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.717+0000 7f79b21fb700 1 --2- 172.21.15.112:0/3497508217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79ac146d50 0x7f79ac0b6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.717+0000 7f79b09f8700 1 -- 172.21.15.112:0/3497508217 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3796091708 0 0) 0x7f79ac0b7f60 con 0x7f79ac0b69b0 2023-08-27T11:18:01.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.717+0000 7f79b21fb700 1 -- 172.21.15.112:0/3497508217 >> v1:172.21.15.112:6789/0 conn(0x7f79ac0b69b0 legacy=0x7f79ac0b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.717+0000 7f79b21fb700 1 -- 172.21.15.112:0/3497508217 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac0b7480 con 0x7f79ac0b5750 2023-08-27T11:18:01.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.717+0000 7f79b21fb700 1 --2- 172.21.15.112:0/3497508217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac0b5750 0x7f79ac0d0cb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009d90 tx=0x7f79a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831701daaa9baaab server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:01.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.717+0000 7f79b09f8700 1 -- 172.21.15.112:0/3497508217 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a400e070 con 0x7f79ac0b5750 2023-08-27T11:18:01.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.717+0000 7f79b09f8700 1 -- 172.21.15.112:0/3497508217 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a400bbd0 con 0x7f79ac0b5750 2023-08-27T11:18:01.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.717+0000 7f79b09f8700 1 -- 172.21.15.112:0/3497508217 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4012430 con 0x7f79ac0b5750 2023-08-27T11:18:01.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.718+0000 7f79b3c5e700 1 -- 172.21.15.112:0/3497508217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac0b5750 msgr2=0x7f79ac0d0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.718+0000 7f79b3c5e700 1 --2- 172.21.15.112:0/3497508217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac0b5750 0x7f79ac0d0cb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009d90 tx=0x7f79a400b330 comp rx=0 tx=0).stop 2023-08-27T11:18:01.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.718+0000 7f79b3c5e700 1 -- 172.21.15.112:0/3497508217 shutdown_connections 2023-08-27T11:18:01.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.718+0000 7f79b3c5e700 1 --2- 172.21.15.112:0/3497508217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac0b5750 0x7f79ac0d0cb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.718+0000 7f79b3c5e700 1 --2- 172.21.15.112:0/3497508217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79ac146d50 0x7f79ac0b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.718+0000 7f79b3c5e700 1 -- 172.21.15.112:0/3497508217 >> 172.21.15.112:0/3497508217 conn(0x7f79ac06d330 msgr2=0x7f79ac0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:01.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.718+0000 7f79b3c5e700 1 -- 172.21.15.112:0/3497508217 shutdown_connections 2023-08-27T11:18:01.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.718+0000 7f79b3c5e700 1 -- 172.21.15.112:0/3497508217 wait complete. 2023-08-27T11:18:01.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.719+0000 7f79b3c5e700 1 Processor -- start 2023-08-27T11:18:01.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.719+0000 7f79b3c5e700 1 -- start start 2023-08-27T11:18:01.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b3c5e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac0b5750 0x7f79ac15e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b3c5e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79ac0b69b0 0x7f79ac15ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b3c5e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac146d50 0x7f79ac15dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b3c5e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac070bf0 con 0x7f79ac0b5750 2023-08-27T11:18:01.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b3c5e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac070ed0 con 0x7f79ac146d50 2023-08-27T11:18:01.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b3c5e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac0d1250 con 0x7f79ac0b69b0 2023-08-27T11:18:01.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b19fa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac0b5750 0x7f79ac15e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b19fa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac0b5750 0x7f79ac15e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56714/0 (socket says 172.21.15.112:56714) 2023-08-27T11:18:01.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b19fa700 1 -- 172.21.15.112:0/4073398390 learned_addr learned my addr 172.21.15.112:0/4073398390 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:01.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b19fa700 1 -- 172.21.15.112:0/4073398390 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79ac0b69b0 msgr2=0x7f79ac15ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b19fa700 1 --2- 172.21.15.112:0/4073398390 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79ac0b69b0 0x7f79ac15ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b19fa700 1 -- 172.21.15.112:0/4073398390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac146d50 msgr2=0x7f79ac15dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:01.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b19fa700 1 --2- 172.21.15.112:0/4073398390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac146d50 0x7f79ac15dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:01.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b19fa700 1 -- 172.21.15.112:0/4073398390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a4009a70 con 0x7f79ac0b5750 2023-08-27T11:18:01.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b19fa700 1 --2- 172.21.15.112:0/4073398390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac0b5750 0x7f79ac15e7e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f799c00c7f0 tx=0x7f799c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:01.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79a27fc700 1 -- 172.21.15.112:0/4073398390 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c011800 con 0x7f79ac0b5750 2023-08-27T11:18:01.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79a27fc700 1 -- 172.21.15.112:0/4073398390 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c011960 con 0x7f79ac0b5750 2023-08-27T11:18:01.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79a27fc700 1 -- 172.21.15.112:0/4073398390 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c011cf0 con 0x7f79ac0b5750 2023-08-27T11:18:01.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b3c5e700 1 -- 172.21.15.112:0/4073398390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ac0d1530 con 0x7f79ac0b5750 2023-08-27T11:18:01.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.720+0000 7f79b3c5e700 1 -- 172.21.15.112:0/4073398390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ac15ed20 con 0x7f79ac0b5750 2023-08-27T11:18:01.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.721+0000 7f79b3c5e700 1 -- 172.21.15.112:0/4073398390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ac067a20 con 0x7f79ac0b5750 2023-08-27T11:18:01.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.722+0000 7f79a27fc700 1 -- 172.21.15.112:0/4073398390 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f799c018360 con 0x7f79ac0b5750 2023-08-27T11:18:01.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.722+0000 7f79a27fc700 1 --2- 172.21.15.112:0/4073398390 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f799006a0e0 0x7f799006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:01.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.722+0000 7f79a27fc700 1 -- 172.21.15.112:0/4073398390 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f799c088110 con 0x7f79ac0b5750 2023-08-27T11:18:01.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.722+0000 7f79b11f9700 1 --2- 172.21.15.112:0/4073398390 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f799006a0e0 0x7f799006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:01.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.723+0000 7f79b11f9700 1 --2- 172.21.15.112:0/4073398390 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f799006a0e0 0x7f799006c5a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f79a80011e0 tx=0x7f79a80079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:01.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.726+0000 7f79a27fc700 1 -- 172.21.15.112:0/4073398390 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f799c053b50 con 0x7f79ac0b5750 2023-08-27T11:18:01.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:01.916+0000 7f79b3c5e700 1 -- 172.21.15.112:0/4073398390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f79ac0b54d0 con 0x7f79ac0b5750 2023-08-27T11:18:02.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.235+0000 7f79a27fc700 1 -- 172.21.15.112:0/4073398390 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f799c057170 con 0x7f79ac0b5750 2023-08-27T11:18:02.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.238+0000 7f79b3c5e700 1 -- 172.21.15.112:0/4073398390 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f799006a0e0 msgr2=0x7f799006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:02.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.238+0000 7f79b3c5e700 1 --2- 172.21.15.112:0/4073398390 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f799006a0e0 0x7f799006c5a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f79a80011e0 tx=0x7f79a80079b0 comp rx=0 tx=0).stop 2023-08-27T11:18:02.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.238+0000 7f79b3c5e700 1 -- 172.21.15.112:0/4073398390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac0b5750 msgr2=0x7f79ac15e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:02.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.238+0000 7f79b3c5e700 1 --2- 172.21.15.112:0/4073398390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac0b5750 0x7f79ac15e7e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f799c00c7f0 tx=0x7f799c00cbb0 comp rx=0 tx=0).stop 2023-08-27T11:18:02.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.238+0000 7f79b3c5e700 1 -- 172.21.15.112:0/4073398390 shutdown_connections 2023-08-27T11:18:02.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.238+0000 7f79b3c5e700 1 --2- 172.21.15.112:0/4073398390 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f799006a0e0 0x7f799006c5a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:02.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.238+0000 7f79b3c5e700 1 --2- 172.21.15.112:0/4073398390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac146d50 0x7f79ac15dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:02.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.238+0000 7f79b3c5e700 1 --2- 172.21.15.112:0/4073398390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac0b5750 0x7f79ac15e7e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:02.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.238+0000 7f79b3c5e700 1 --2- 172.21.15.112:0/4073398390 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79ac0b69b0 0x7f79ac15ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:02.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.238+0000 7f79b3c5e700 1 -- 172.21.15.112:0/4073398390 >> 172.21.15.112:0/4073398390 conn(0x7f79ac06d330 msgr2=0x7f79ac1457a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:02.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.238+0000 7f79b3c5e700 1 -- 172.21.15.112:0/4073398390 shutdown_connections 2023-08-27T11:18:02.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.238+0000 7f79b3c5e700 1 -- 172.21.15.112:0/4073398390 wait complete. 2023-08-27T11:18:02.242 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2023-08-27T11:18:02.255 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:18:02.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.382+0000 7fe24a85b700 1 Processor -- start 2023-08-27T11:18:02.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.382+0000 7fe24a85b700 1 -- start start 2023-08-27T11:18:02.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe24a85b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe244146fd0 0x7fe2440b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:02.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe24a85b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe244070eb0 0x7fe244071320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:02.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe24a85b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2440d1500 con 0x7fe244146bf0 2023-08-27T11:18:02.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe24a85b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe244071860 con 0x7fe244146fd0 2023-08-27T11:18:02.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe24a85b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2440d0460 con 0x7fe244070eb0 2023-08-27T11:18:02.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe2437fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe244146fd0 0x7fe2440b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:02.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe2437fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe244146fd0 0x7fe2440b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56726/0 (socket says 172.21.15.112:56726) 2023-08-27T11:18:02.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe2437fe700 1 -- 172.21.15.112:0/3043984889 learned_addr learned my addr 172.21.15.112:0/3043984889 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:02.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe248df8700 1 --2- 172.21.15.112:0/3043984889 >> v2:172.21.15.138:3300/0 conn(0x7fe244070eb0 0x7fe244071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:02.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe2437fe700 1 -- 172.21.15.112:0/3043984889 >> v2:172.21.15.138:3300/0 conn(0x7fe244070eb0 msgr2=0x7fe244071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:02.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe2437fe700 1 --2- 172.21.15.112:0/3043984889 >> v2:172.21.15.138:3300/0 conn(0x7fe244070eb0 0x7fe244071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:02.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe2437fe700 1 -- 172.21.15.112:0/3043984889 >> v1:172.21.15.138:6789/0 conn(0x7fe244146bf0 legacy=0x7fe2440cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:02.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe2437fe700 1 -- 172.21.15.112:0/3043984889 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2440d0710 con 0x7fe244146fd0 2023-08-27T11:18:02.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.383+0000 7fe248df8700 1 --2- 172.21.15.112:0/3043984889 >> v2:172.21.15.138:3300/0 conn(0x7fe244070eb0 0x7fe244071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:02.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.384+0000 7fe2437fe700 1 --2- 172.21.15.112:0/3043984889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe244146fd0 0x7fe2440b5db0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe2440b58f0 tx=0x7fe234009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b747fd52d1737c2f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:02.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.384+0000 7fe242ffd700 1 -- 172.21.15.112:0/3043984889 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234004230 con 0x7fe244146fd0 2023-08-27T11:18:02.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.384+0000 7fe242ffd700 1 -- 172.21.15.112:0/3043984889 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234004390 con 0x7fe244146fd0 2023-08-27T11:18:02.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.384+0000 7fe242ffd700 1 -- 172.21.15.112:0/3043984889 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340036a0 con 0x7fe244146fd0 2023-08-27T11:18:02.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.384+0000 7fe24a85b700 1 -- 172.21.15.112:0/3043984889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe244146fd0 msgr2=0x7fe2440b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:02.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.384+0000 7fe24a85b700 1 --2- 172.21.15.112:0/3043984889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe244146fd0 0x7fe2440b5db0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe2440b58f0 tx=0x7fe234009bd0 comp rx=0 tx=0).stop 2023-08-27T11:18:02.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.384+0000 7fe24a85b700 1 -- 172.21.15.112:0/3043984889 shutdown_connections 2023-08-27T11:18:02.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.384+0000 7fe24a85b700 1 --2- 172.21.15.112:0/3043984889 >> v2:172.21.15.138:3300/0 conn(0x7fe244070eb0 0x7fe244071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:02.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.384+0000 7fe24a85b700 1 --2- 172.21.15.112:0/3043984889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe244146fd0 0x7fe2440b5db0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:02.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.384+0000 7fe24a85b700 1 -- 172.21.15.112:0/3043984889 >> 172.21.15.112:0/3043984889 conn(0x7fe24406c450 msgr2=0x7fe24406f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:02.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.384+0000 7fe24a85b700 1 -- 172.21.15.112:0/3043984889 shutdown_connections 2023-08-27T11:18:02.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.384+0000 7fe24a85b700 1 -- 172.21.15.112:0/3043984889 wait complete. 2023-08-27T11:18:02.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.385+0000 7fe24a85b700 1 Processor -- start 2023-08-27T11:18:02.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.386+0000 7fe24a85b700 1 -- start start 2023-08-27T11:18:02.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.386+0000 7fe24a85b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe244070eb0 0x7fe24413cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:02.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.386+0000 7fe24a85b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe244146bf0 0x7fe24413f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:02.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.386+0000 7fe24a85b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe244146fd0 0x7fe24415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:02.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.386+0000 7fe24a85b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24413e220 con 0x7fe244146fd0 2023-08-27T11:18:02.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.386+0000 7fe24a85b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24413e4c0 con 0x7fe244070eb0 2023-08-27T11:18:02.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.386+0000 7fe24a85b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24413e7a0 con 0x7fe244146bf0 2023-08-27T11:18:02.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe248df8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe244146fd0 0x7fe24415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:02.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe248df8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe244146fd0 0x7fe24415a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56734/0 (socket says 172.21.15.112:56734) 2023-08-27T11:18:02.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe248df8700 1 -- 172.21.15.112:0/2740243946 learned_addr learned my addr 172.21.15.112:0/2740243946 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:02.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe243fff700 1 --2- 172.21.15.112:0/2740243946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe244070eb0 0x7fe24413cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:02.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe248df8700 1 -- 172.21.15.112:0/2740243946 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe244146bf0 msgr2=0x7fe24413f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:02.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe248df8700 1 --2- 172.21.15.112:0/2740243946 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe244146bf0 0x7fe24413f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:02.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe248df8700 1 -- 172.21.15.112:0/2740243946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe244070eb0 msgr2=0x7fe24413cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:02.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe248df8700 1 --2- 172.21.15.112:0/2740243946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe244070eb0 0x7fe24413cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:02.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe248df8700 1 -- 172.21.15.112:0/2740243946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe234009550 con 0x7fe244146fd0 2023-08-27T11:18:02.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe248df8700 1 --2- 172.21.15.112:0/2740243946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe244146fd0 0x7fe24415a020 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00cef0 tx=0x7fe23c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:02.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe240ff9700 1 -- 172.21.15.112:0/2740243946 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00ae50 con 0x7fe244146fd0 2023-08-27T11:18:02.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe240ff9700 1 -- 172.21.15.112:0/2740243946 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c015440 con 0x7fe244146fd0 2023-08-27T11:18:02.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe24a85b700 1 -- 172.21.15.112:0/2740243946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24413ea80 con 0x7fe244146fd0 2023-08-27T11:18:02.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe240ff9700 1 -- 172.21.15.112:0/2740243946 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0157f0 con 0x7fe244146fd0 2023-08-27T11:18:02.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.387+0000 7fe24a85b700 1 -- 172.21.15.112:0/2740243946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24413d170 con 0x7fe244146fd0 2023-08-27T11:18:02.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.388+0000 7fe24a85b700 1 -- 172.21.15.112:0/2740243946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24413efa0 con 0x7fe244146fd0 2023-08-27T11:18:02.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.389+0000 7fe240ff9700 1 -- 172.21.15.112:0/2740243946 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe23c00d430 con 0x7fe244146fd0 2023-08-27T11:18:02.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.389+0000 7fe240ff9700 1 --2- 172.21.15.112:0/2740243946 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe23006a0e0 0x7fe23006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:02.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.389+0000 7fe240ff9700 1 -- 172.21.15.112:0/2740243946 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fe23c088970 con 0x7fe244146fd0 2023-08-27T11:18:02.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.389+0000 7fe243fff700 1 --2- 172.21.15.112:0/2740243946 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe23006a0e0 0x7fe23006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:02.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.390+0000 7fe243fff700 1 --2- 172.21.15.112:0/2740243946 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe23006a0e0 0x7fe23006c5a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe238007fd0 tx=0x7fe238009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:02.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.394+0000 7fe240ff9700 1 -- 172.21.15.112:0/2740243946 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe24413efa0 con 0x7fe244146fd0 2023-08-27T11:18:02.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:02.584+0000 7fe24a85b700 1 -- 172.21.15.112:0/2740243946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe24413efa0 con 0x7fe244146fd0 2023-08-27T11:18:03.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.240+0000 7fe240ff9700 1 -- 172.21.15.112:0/2740243946 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe24413efa0 con 0x7fe244146fd0 2023-08-27T11:18:03.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.245+0000 7fe24a85b700 1 -- 172.21.15.112:0/2740243946 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe23006a0e0 msgr2=0x7fe23006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:03.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.245+0000 7fe24a85b700 1 --2- 172.21.15.112:0/2740243946 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe23006a0e0 0x7fe23006c5a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe238007fd0 tx=0x7fe238009040 comp rx=0 tx=0).stop 2023-08-27T11:18:03.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.245+0000 7fe24a85b700 1 -- 172.21.15.112:0/2740243946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe244146fd0 msgr2=0x7fe24415a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:03.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.245+0000 7fe24a85b700 1 --2- 172.21.15.112:0/2740243946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe244146fd0 0x7fe24415a020 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00cef0 tx=0x7fe23c00a340 comp rx=0 tx=0).stop 2023-08-27T11:18:03.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.246+0000 7fe24a85b700 1 -- 172.21.15.112:0/2740243946 shutdown_connections 2023-08-27T11:18:03.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.246+0000 7fe24a85b700 1 --2- 172.21.15.112:0/2740243946 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe23006a0e0 0x7fe23006c5a0 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe238007fd0 tx=0x7fe238009040 comp rx=0 tx=0).stop 2023-08-27T11:18:03.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.246+0000 7fe24a85b700 1 --2- 172.21.15.112:0/2740243946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe244146fd0 0x7fe24415a020 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:03.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.246+0000 7fe24a85b700 1 --2- 172.21.15.112:0/2740243946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe244070eb0 0x7fe24413cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:03.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.246+0000 7fe24a85b700 1 --2- 172.21.15.112:0/2740243946 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe244146bf0 0x7fe24413f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:03.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.246+0000 7fe24a85b700 1 -- 172.21.15.112:0/2740243946 >> 172.21.15.112:0/2740243946 conn(0x7fe24406c450 msgr2=0x7fe2440d1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:03.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.247+0000 7fe24a85b700 1 -- 172.21.15.112:0/2740243946 shutdown_connections 2023-08-27T11:18:03.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.247+0000 7fe24a85b700 1 -- 172.21.15.112:0/2740243946 wait complete. 2023-08-27T11:18:03.253 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2023-08-27T11:18:03.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:18:03.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.392+0000 7f20fd222700 1 Processor -- start 2023-08-27T11:18:03.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.392+0000 7f20fd222700 1 -- start start 2023-08-27T11:18:03.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.393+0000 7f20fd222700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20f81437c0 0x7f20f80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.393+0000 7f20fd222700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f20f8143ba0 0x7f20f80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.393+0000 7f20fd222700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f20f806d5b0 0x7f20f806da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.393+0000 7f20fd222700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f80cfd50 con 0x7f20f8143ba0 2023-08-27T11:18:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.393+0000 7f20fd222700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f806df40 con 0x7f20f806d5b0 2023-08-27T11:18:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.393+0000 7f20fd222700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80cf140 con 0x7f20f81437c0 2023-08-27T11:18:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.393+0000 7f20f659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f20f8143ba0 0x7f20f80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.393+0000 7f20f659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f20f8143ba0 0x7f20f80ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56740/0 (socket says 172.21.15.112:56740) 2023-08-27T11:18:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.393+0000 7f20f659c700 1 -- 172.21.15.112:0/1742383109 learned_addr learned my addr 172.21.15.112:0/1742383109 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.394+0000 7f20f6d9d700 1 --2- 172.21.15.112:0/1742383109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20f81437c0 0x7f20f80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.394+0000 7f20f759e700 1 --2- 172.21.15.112:0/1742383109 >> v2:172.21.15.138:3300/0 conn(0x7f20f806d5b0 0x7f20f806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.394+0000 7f20f659c700 1 -- 172.21.15.112:0/1742383109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20f81437c0 msgr2=0x7f20f80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.394+0000 7f20f659c700 1 --2- 172.21.15.112:0/1742383109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20f81437c0 0x7f20f80b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.394+0000 7f20f659c700 1 -- 172.21.15.112:0/1742383109 >> v2:172.21.15.138:3300/0 conn(0x7f20f806d5b0 msgr2=0x7f20f806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.394+0000 7f20f659c700 1 --2- 172.21.15.112:0/1742383109 >> v2:172.21.15.138:3300/0 conn(0x7f20f806d5b0 0x7f20f806da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.394+0000 7f20f659c700 1 -- 172.21.15.112:0/1742383109 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f80cf420 con 0x7f20f8143ba0 2023-08-27T11:18:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.394+0000 7f20f6d9d700 1 --2- 172.21.15.112:0/1742383109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20f81437c0 0x7f20f80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.394+0000 7f20f659c700 1 --2- 172.21.15.112:0/1742383109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f8143ba0 0x7f20f80ce860 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f20f80b3af0 tx=0x7f20ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3147b1766e89a59a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.394+0000 7f20f5d9b700 1 -- 172.21.15.112:0/1742383109 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00b870 con 0x7f20f8143ba0 2023-08-27T11:18:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.394+0000 7f20f5d9b700 1 -- 172.21.15.112:0/1742383109 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec00b9d0 con 0x7f20f8143ba0 2023-08-27T11:18:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.394+0000 7f20f5d9b700 1 -- 172.21.15.112:0/1742383109 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00bd60 con 0x7f20f8143ba0 2023-08-27T11:18:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.394+0000 7f20fd222700 1 -- 172.21.15.112:0/1742383109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f8143ba0 msgr2=0x7f20f80ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.395+0000 7f20fd222700 1 --2- 172.21.15.112:0/1742383109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f8143ba0 0x7f20f80ce860 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f20f80b3af0 tx=0x7f20ec009bd0 comp rx=0 tx=0).stop 2023-08-27T11:18:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.395+0000 7f20fd222700 1 -- 172.21.15.112:0/1742383109 shutdown_connections 2023-08-27T11:18:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.395+0000 7f20fd222700 1 --2- 172.21.15.112:0/1742383109 >> v2:172.21.15.138:3300/0 conn(0x7f20f806d5b0 0x7f20f806da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.395+0000 7f20fd222700 1 --2- 172.21.15.112:0/1742383109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20f81437c0 0x7f20f80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.395+0000 7f20fd222700 1 --2- 172.21.15.112:0/1742383109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f8143ba0 0x7f20f80ce860 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.395+0000 7f20fd222700 1 -- 172.21.15.112:0/1742383109 >> 172.21.15.112:0/1742383109 conn(0x7f20f80b1e90 msgr2=0x7f20f80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.395+0000 7f20fd222700 1 -- 172.21.15.112:0/1742383109 shutdown_connections 2023-08-27T11:18:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.395+0000 7f20fd222700 1 -- 172.21.15.112:0/1742383109 wait complete. 2023-08-27T11:18:03.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.396+0000 7f20fd222700 1 Processor -- start 2023-08-27T11:18:03.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.396+0000 7f20fd222700 1 -- start start 2023-08-27T11:18:03.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20fd222700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f806d5b0 0x7f20f81585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:03.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20fd222700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20f81437c0 0x7f20f8158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:03.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20fd222700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20f8143ba0 0x7f20f815dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:03.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20fd222700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f81f06e0 con 0x7f20f806d5b0 2023-08-27T11:18:03.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20fd222700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f8158b30 con 0x7f20f8143ba0 2023-08-27T11:18:03.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20fd222700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f8139800 con 0x7f20f81437c0 2023-08-27T11:18:03.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20f6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f806d5b0 0x7f20f81585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:03.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20f6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f806d5b0 0x7f20f81585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56744/0 (socket says 172.21.15.112:56744) 2023-08-27T11:18:03.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20f6d9d700 1 -- 172.21.15.112:0/2424613991 learned_addr learned my addr 172.21.15.112:0/2424613991 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:03.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20f759e700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20f8143ba0 0x7f20f815dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:03.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20f659c700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20f81437c0 0x7f20f8158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:03.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20f6d9d700 1 -- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20f81437c0 msgr2=0x7f20f8158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20f6d9d700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20f81437c0 0x7f20f8158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20f6d9d700 1 -- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20f8143ba0 msgr2=0x7f20f815dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20f6d9d700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20f8143ba0 0x7f20f815dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20f6d9d700 1 -- 172.21.15.112:0/2424613991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec009550 con 0x7f20f806d5b0 2023-08-27T11:18:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.397+0000 7f20f6d9d700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f806d5b0 0x7f20f81585f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f20e800e810 tx=0x7f20e800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.398+0000 7f20df7fe700 1 -- 172.21.15.112:0/2424613991 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e800c070 con 0x7f20f806d5b0 2023-08-27T11:18:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.398+0000 7f20df7fe700 1 -- 172.21.15.112:0/2424613991 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e8004230 con 0x7f20f806d5b0 2023-08-27T11:18:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.398+0000 7f20fd222700 1 -- 172.21.15.112:0/2424613991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f813bb40 con 0x7f20f806d5b0 2023-08-27T11:18:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.398+0000 7f20df7fe700 1 -- 172.21.15.112:0/2424613991 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80036a0 con 0x7f20f806d5b0 2023-08-27T11:18:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.398+0000 7f20fd222700 1 -- 172.21.15.112:0/2424613991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f813c100 con 0x7f20f806d5b0 2023-08-27T11:18:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.399+0000 7f20fd222700 1 -- 172.21.15.112:0/2424613991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f8067e90 con 0x7f20f806d5b0 2023-08-27T11:18:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.400+0000 7f20df7fe700 1 -- 172.21.15.112:0/2424613991 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f20e8003800 con 0x7f20f806d5b0 2023-08-27T11:18:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.400+0000 7f20df7fe700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f20e406a0e0 0x7f20e406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.401+0000 7f20df7fe700 1 -- 172.21.15.112:0/2424613991 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f20e80885f0 con 0x7f20f806d5b0 2023-08-27T11:18:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.401+0000 7f20f659c700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f20e406a0e0 0x7f20e406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.401+0000 7f20f659c700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f20e406a0e0 0x7f20e406c5a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f20ec014040 tx=0x7f20ec008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.404+0000 7f20df7fe700 1 -- 172.21.15.112:0/2424613991 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f20e8053d20 con 0x7f20f806d5b0 2023-08-27T11:18:03.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:03.636+0000 7f20fd222700 1 -- 172.21.15.112:0/2424613991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f20f8067a20 con 0x7f20f806d5b0 2023-08-27T11:18:04.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.242+0000 7f20df7fe700 1 -- 172.21.15.112:0/2424613991 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f20e80094c0 con 0x7f20f806d5b0 2023-08-27T11:18:04.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.247+0000 7f20fd222700 1 -- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f20e406a0e0 msgr2=0x7f20e406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.247+0000 7f20fd222700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f20e406a0e0 0x7f20e406c5a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f20ec014040 tx=0x7f20ec008030 comp rx=0 tx=0).stop 2023-08-27T11:18:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.247+0000 7f20fd222700 1 -- 172.21.15.112:0/2424613991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f806d5b0 msgr2=0x7f20f81585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.247+0000 7f20fd222700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f806d5b0 0x7f20f81585f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f20e800e810 tx=0x7f20e800eb20 comp rx=0 tx=0).stop 2023-08-27T11:18:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.247+0000 7f20fd222700 1 -- 172.21.15.112:0/2424613991 shutdown_connections 2023-08-27T11:18:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.247+0000 7f20fd222700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f20e406a0e0 0x7f20e406c5a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.247+0000 7f20fd222700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20f8143ba0 0x7f20f815dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.247+0000 7f20fd222700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f806d5b0 0x7f20f81585f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.248+0000 7f20fd222700 1 --2- 172.21.15.112:0/2424613991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20f81437c0 0x7f20f8158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.248+0000 7f20fd222700 1 -- 172.21.15.112:0/2424613991 >> 172.21.15.112:0/2424613991 conn(0x7f20f80b1e90 msgr2=0x7f20f80708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:04.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.248+0000 7f20fd222700 1 -- 172.21.15.112:0/2424613991 shutdown_connections 2023-08-27T11:18:04.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.249+0000 7f20fd222700 1 -- 172.21.15.112:0/2424613991 wait complete. 2023-08-27T11:18:04.257 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 5 and data pool 6 2023-08-27T11:18:04.275 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:04.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.394+0000 7f4792c9d700 1 Processor -- start 2023-08-27T11:18:04.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.394+0000 7f4792c9d700 1 -- start start 2023-08-27T11:18:04.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.395+0000 7f4792c9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f478c163f90 0x7f478c166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.395+0000 7f4792c9d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478c0b5ae0 con 0x7f478c0b3e50 2023-08-27T11:18:04.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.395+0000 7f4792c9d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478c0b45f0 con 0x7f478c144570 2023-08-27T11:18:04.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.395+0000 7f4792c9d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0cf7a0 con 0x7f478c163f90 2023-08-27T11:18:04.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.395+0000 7f478bfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f478c0b3e50 0x7f478c15f530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51134/0 (socket says 172.21.15.112:51134) 2023-08-27T11:18:04.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.395+0000 7f478bfff700 1 -- 172.21.15.112:0/3432785088 learned_addr learned my addr 172.21.15.112:0/3432785088 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:04.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f479123a700 1 --2- 172.21.15.112:0/3432785088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f478c163f90 0x7f478c166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614991353 0 0) 0x7f4778001110 con 0x7f478c0b3e50 2023-08-27T11:18:04.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f477c0035b0 con 0x7f478c0b3e50 2023-08-27T11:18:04.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078978669 0 0) 0x7f478c0b45f0 con 0x7f478c144570 2023-08-27T11:18:04.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4778001110 con 0x7f478c144570 2023-08-27T11:18:04.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 448581920 0 0) 0x7f477c0035b0 con 0x7f478c0b3e50 2023-08-27T11:18:04.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f478c163f90 msgr2=0x7f478c166380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:04.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 --2- 172.21.15.112:0/3432785088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f478c163f90 0x7f478c166380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 >> v1:172.21.15.138:6789/0 conn(0x7f478c144570 legacy=0x7f478c15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c15fc40 con 0x7f478c0b3e50 2023-08-27T11:18:04.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f4778001800 con 0x7f478c0b3e50 2023-08-27T11:18:04.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 >> v1:172.21.15.112:6789/0 conn(0x7f478c0b3e50 legacy=0x7f478c15f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 --2- 172.21.15.112:0/3432785088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f477c003bc0 0x7f477c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 --2- 172.21.15.112:0/3432785088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f478c163f90 0x7f477c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 --2- 172.21.15.112:0/3432785088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c004d90 0x7f477c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0cf7a0 con 0x7f477c003bc0 2023-08-27T11:18:04.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c00bb10 con 0x7f477c004d90 2023-08-27T11:18:04.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c00bcd0 con 0x7f478c163f90 2023-08-27T11:18:04.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f4790a39700 1 --2- 172.21.15.112:0/3432785088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f477c003bc0 0x7f477c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f478bfff700 1 --2- 172.21.15.112:0/3432785088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f478c163f90 0x7f477c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f479123a700 1 --2- 172.21.15.112:0/3432785088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c004d90 0x7f477c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f4790a39700 1 -- 172.21.15.112:0/3432785088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f478c163f90 msgr2=0x7f477c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:04.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f4790a39700 1 --2- 172.21.15.112:0/3432785088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f478c163f90 0x7f477c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f4790a39700 1 -- 172.21.15.112:0/3432785088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c004d90 msgr2=0x7f477c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:04.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f4790a39700 1 --2- 172.21.15.112:0/3432785088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c004d90 0x7f477c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.396+0000 7f4790a39700 1 -- 172.21.15.112:0/3432785088 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0035b0 con 0x7f477c003bc0 2023-08-27T11:18:04.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.397+0000 7f4790a39700 1 --2- 172.21.15.112:0/3432785088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f477c003bc0 0x7f477c003fc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4780001940 tx=0x7f477c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35336cdf319490fa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:04.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.397+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780001430 con 0x7f477c003bc0 2023-08-27T11:18:04.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.397+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4780001d70 con 0x7f477c003bc0 2023-08-27T11:18:04.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.397+0000 7f478b7fe700 1 -- 172.21.15.112:0/3432785088 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780005510 con 0x7f477c003bc0 2023-08-27T11:18:04.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.397+0000 7f4792c9d700 1 -- 172.21.15.112:0/3432785088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f477c003bc0 msgr2=0x7f477c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.397+0000 7f4792c9d700 1 --2- 172.21.15.112:0/3432785088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f477c003bc0 0x7f477c003fc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4780001940 tx=0x7f477c005880 comp rx=0 tx=0).stop 2023-08-27T11:18:04.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.397+0000 7f479123a700 1 -- 172.21.15.112:0/3432785088 reap_dead start 2023-08-27T11:18:04.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.397+0000 7f4792c9d700 1 -- 172.21.15.112:0/3432785088 shutdown_connections 2023-08-27T11:18:04.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.397+0000 7f4792c9d700 1 -- 172.21.15.112:0/3432785088 >> 172.21.15.112:0/3432785088 conn(0x7f478c06f970 msgr2=0x7f478c06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:04.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.397+0000 7f4792c9d700 1 -- 172.21.15.112:0/3432785088 shutdown_connections 2023-08-27T11:18:04.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.397+0000 7f4792c9d700 1 -- 172.21.15.112:0/3432785088 wait complete. 2023-08-27T11:18:04.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.399+0000 7f4792c9d700 1 Processor -- start 2023-08-27T11:18:04.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.399+0000 7f4792c9d700 1 -- start start 2023-08-27T11:18:04.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f4792c9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c003bc0 0x7f478c15fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f4792c9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f478c13e800 0x7f478c0ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f4792c9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f478c13ec80 0x7f478c0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f4792c9d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c161de0 con 0x7f478c13ec80 2023-08-27T11:18:04.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f4792c9d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c160370 con 0x7f477c003bc0 2023-08-27T11:18:04.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f4792c9d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478c160610 con 0x7f478c13e800 2023-08-27T11:18:04.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f479123a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f478c13ec80 0x7f478c0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f479123a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f478c13ec80 0x7f478c0cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56768/0 (socket says 172.21.15.112:56768) 2023-08-27T11:18:04.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f479123a700 1 -- 172.21.15.112:0/4007356914 learned_addr learned my addr 172.21.15.112:0/4007356914 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:04.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f4790a39700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c003bc0 0x7f478c15fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f478bfff700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f478c13e800 0x7f478c0ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f479123a700 1 -- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f478c13e800 msgr2=0x7f478c0ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f479123a700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f478c13e800 0x7f478c0ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f479123a700 1 -- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c003bc0 msgr2=0x7f478c15fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f479123a700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c003bc0 0x7f478c15fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.400+0000 7f479123a700 1 -- 172.21.15.112:0/4007356914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c161ff0 con 0x7f478c13ec80 2023-08-27T11:18:04.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.401+0000 7f479123a700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f478c13ec80 0x7f478c0cf090 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f478c06b780 tx=0x7f4784011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:04.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.401+0000 7f47897fa700 1 -- 172.21.15.112:0/4007356914 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478400e040 con 0x7f478c13ec80 2023-08-27T11:18:04.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.401+0000 7f47897fa700 1 -- 172.21.15.112:0/4007356914 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478400f040 con 0x7f478c13ec80 2023-08-27T11:18:04.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.401+0000 7f4792c9d700 1 -- 172.21.15.112:0/4007356914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478c1622d0 con 0x7f478c13ec80 2023-08-27T11:18:04.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.401+0000 7f47897fa700 1 -- 172.21.15.112:0/4007356914 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478400fa00 con 0x7f478c13ec80 2023-08-27T11:18:04.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.401+0000 7f4792c9d700 1 -- 172.21.15.112:0/4007356914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478c1608f0 con 0x7f478c13ec80 2023-08-27T11:18:04.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.402+0000 7f4792c9d700 1 -- 172.21.15.112:0/4007356914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478c1627f0 con 0x7f478c13ec80 2023-08-27T11:18:04.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.402+0000 7f47897fa700 1 -- 172.21.15.112:0/4007356914 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f478400fb60 con 0x7f478c13ec80 2023-08-27T11:18:04.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.402+0000 7f47897fa700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f477006a0e0 0x7f477006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.402+0000 7f47897fa700 1 -- 172.21.15.112:0/4007356914 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4784088b80 con 0x7f478c13ec80 2023-08-27T11:18:04.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.403+0000 7f4790a39700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f477006a0e0 0x7f477006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.403+0000 7f4790a39700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f477006a0e0 0x7f477006c5a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4780015040 tx=0x7f478000a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:04.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.407+0000 7f47897fa700 1 -- 172.21.15.112:0/4007356914 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f478c1627f0 con 0x7f478c13ec80 2023-08-27T11:18:04.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.599+0000 7f4792c9d700 1 -- 172.21.15.112:0/4007356914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f478c1627f0 con 0x7f478c13ec80 2023-08-27T11:18:04.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.600+0000 7f47897fa700 1 -- 172.21.15.112:0/4007356914 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+18330 (secure 0 0 0) 0x7f478c1627f0 con 0x7f478c13ec80 2023-08-27T11:18:04.602 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:04.602 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":32,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:03.637541+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:01.935213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:02.602607+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.602+0000 7f4792c9d700 1 -- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f477006a0e0 msgr2=0x7f477006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.602+0000 7f4792c9d700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f477006a0e0 0x7f477006c5a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4780015040 tx=0x7f478000a000 comp rx=0 tx=0).stop 2023-08-27T11:18:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.602+0000 7f4792c9d700 1 -- 172.21.15.112:0/4007356914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f478c13ec80 msgr2=0x7f478c0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.602+0000 7f4792c9d700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f478c13ec80 0x7f478c0cf090 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f478c06b780 tx=0x7f4784011f80 comp rx=0 tx=0).stop 2023-08-27T11:18:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.602+0000 7f4792c9d700 1 -- 172.21.15.112:0/4007356914 shutdown_connections 2023-08-27T11:18:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.602+0000 7f4792c9d700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f477006a0e0 0x7f477006c5a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.603+0000 7f4792c9d700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f478c13ec80 0x7f478c0cf090 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.603+0000 7f4792c9d700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c003bc0 0x7f478c15fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.603+0000 7f4792c9d700 1 --2- 172.21.15.112:0/4007356914 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f478c13e800 0x7f478c0ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.603+0000 7f4792c9d700 1 -- 172.21.15.112:0/4007356914 >> 172.21.15.112:0/4007356914 conn(0x7f478c06f970 msgr2=0x7f478c15a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:04.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.603+0000 7f4792c9d700 1 -- 172.21.15.112:0/4007356914 shutdown_connections 2023-08-27T11:18:04.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.603+0000 7f4792c9d700 1 -- 172.21.15.112:0/4007356914 wait complete. 2023-08-27T11:18:04.617 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.737+0000 7f5b6cd02700 1 Processor -- start 2023-08-27T11:18:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.737+0000 7f5b6cd02700 1 -- start start 2023-08-27T11:18:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.738+0000 7f5b6cd02700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5b6815d490 0x7f5b6815d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.738+0000 7f5b6cd02700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b68066450 0x7f5b6815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.738+0000 7f5b6cd02700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b680ad580 con 0x7f5b6806e150 2023-08-27T11:18:04.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.738+0000 7f5b6cd02700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b680ad260 con 0x7f5b6815d490 2023-08-27T11:18:04.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.738+0000 7f5b6cd02700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6806f790 con 0x7f5b68066450 2023-08-27T11:18:04.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.738+0000 7f5b6659c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5b6815d490 0x7f5b6815d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.738+0000 7f5b65d9b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b68066450 0x7f5b6815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.738+0000 7f5b66d9d700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5b6806e150 0x7f5b680701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:58680/0 (socket says 172.21.15.112:58680) 2023-08-27T11:18:04.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.738+0000 7f5b66d9d700 1 -- 172.21.15.112:0/260880100 learned_addr learned my addr 172.21.15.112:0/260880100 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.738+0000 7f5b6659c700 1 -- 172.21.15.112:0/260880100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b68066450 msgr2=0x7f5b6815ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.738+0000 7f5b6659c700 1 --2- 172.21.15.112:0/260880100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b68066450 0x7f5b6815ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.738+0000 7f5b6659c700 1 -- 172.21.15.112:0/260880100 >> v1:172.21.15.138:6789/0 conn(0x7f5b6806e150 legacy=0x7f5b680701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.738+0000 7f5b6659c700 1 -- 172.21.15.112:0/260880100 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6806fa70 con 0x7f5b6815d490 2023-08-27T11:18:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.739+0000 7f5b6659c700 1 --2- 172.21.15.112:0/260880100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b6815d490 0x7f5b6815d870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009840 tx=0x7f5b58009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80f8c97317026414 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.739+0000 7f5b6559a700 1 -- 172.21.15.112:0/260880100 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b58011830 con 0x7f5b6815d490 2023-08-27T11:18:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.739+0000 7f5b6559a700 1 -- 172.21.15.112:0/260880100 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b58011990 con 0x7f5b6815d490 2023-08-27T11:18:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.739+0000 7f5b6559a700 1 -- 172.21.15.112:0/260880100 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b58011d20 con 0x7f5b6815d490 2023-08-27T11:18:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.739+0000 7f5b6cd02700 1 -- 172.21.15.112:0/260880100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b6815d490 msgr2=0x7f5b6815d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.739+0000 7f5b6cd02700 1 --2- 172.21.15.112:0/260880100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b6815d490 0x7f5b6815d870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009840 tx=0x7f5b58009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.739+0000 7f5b6cd02700 1 -- 172.21.15.112:0/260880100 shutdown_connections 2023-08-27T11:18:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.739+0000 7f5b6cd02700 1 --2- 172.21.15.112:0/260880100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b6815d490 0x7f5b6815d870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.739+0000 7f5b6cd02700 1 --2- 172.21.15.112:0/260880100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b68066450 0x7f5b6815ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.739+0000 7f5b6cd02700 1 -- 172.21.15.112:0/260880100 >> 172.21.15.112:0/260880100 conn(0x7f5b680b1380 msgr2=0x7f5b680b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.739+0000 7f5b6cd02700 1 -- 172.21.15.112:0/260880100 shutdown_connections 2023-08-27T11:18:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.739+0000 7f5b6cd02700 1 -- 172.21.15.112:0/260880100 wait complete. 2023-08-27T11:18:04.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.741+0000 7f5b6cd02700 1 Processor -- start 2023-08-27T11:18:04.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.741+0000 7f5b6cd02700 1 -- start start 2023-08-27T11:18:04.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.741+0000 7f5b6cd02700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b68066450 0x7f5b681f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b6cd02700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b6806e150 0x7f5b6815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b6cd02700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b6815d490 0x7f5b680adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b6cd02700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b68152060 con 0x7f5b6806e150 2023-08-27T11:18:04.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b6cd02700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b68152ec0 con 0x7f5b6815d490 2023-08-27T11:18:04.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b6cd02700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b681f7320 con 0x7f5b68066450 2023-08-27T11:18:04.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b65d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b6806e150 0x7f5b6815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b65d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b6806e150 0x7f5b6815dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56780/0 (socket says 172.21.15.112:56780) 2023-08-27T11:18:04.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b65d9b700 1 -- 172.21.15.112:0/1814920198 learned_addr learned my addr 172.21.15.112:0/1814920198 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:04.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b6659c700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b68066450 0x7f5b681f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b66d9d700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b6815d490 0x7f5b680adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b65d9b700 1 -- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b68066450 msgr2=0x7f5b681f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b65d9b700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b68066450 0x7f5b681f6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b65d9b700 1 -- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b6815d490 msgr2=0x7f5b680adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b65d9b700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b6815d490 0x7f5b680adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b65d9b700 1 -- 172.21.15.112:0/1814920198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b580094a0 con 0x7f5b6806e150 2023-08-27T11:18:04.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b65d9b700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b6806e150 0x7f5b6815dc90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00e9e0 tx=0x7f5b5c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:04.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b56ffd700 1 -- 172.21.15.112:0/1814920198 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c009710 con 0x7f5b6806e150 2023-08-27T11:18:04.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b56ffd700 1 -- 172.21.15.112:0/1814920198 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c00ee80 con 0x7f5b6806e150 2023-08-27T11:18:04.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b6cd02700 1 -- 172.21.15.112:0/1814920198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b681522b0 con 0x7f5b6806e150 2023-08-27T11:18:04.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b56ffd700 1 -- 172.21.15.112:0/1814920198 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c00ca90 con 0x7f5b6806e150 2023-08-27T11:18:04.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.742+0000 7f5b6cd02700 1 -- 172.21.15.112:0/1814920198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b68152870 con 0x7f5b6806e150 2023-08-27T11:18:04.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.743+0000 7f5b6cd02700 1 -- 172.21.15.112:0/1814920198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b68153050 con 0x7f5b6806e150 2023-08-27T11:18:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.744+0000 7f5b56ffd700 1 -- 172.21.15.112:0/1814920198 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5b5c00cbf0 con 0x7f5b6806e150 2023-08-27T11:18:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.744+0000 7f5b56ffd700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b4806a0e0 0x7f5b4806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.745+0000 7f5b56ffd700 1 -- 172.21.15.112:0/1814920198 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5b5c088c50 con 0x7f5b6806e150 2023-08-27T11:18:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.745+0000 7f5b6659c700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b4806a0e0 0x7f5b4806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.745+0000 7f5b6659c700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b4806a0e0 0x7f5b4806c5a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5b58000c00 tx=0x7f5b5801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.748+0000 7f5b56ffd700 1 -- 172.21.15.112:0/1814920198 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5b5c057e40 con 0x7f5b6806e150 2023-08-27T11:18:04.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.940+0000 7f5b6cd02700 1 -- 172.21.15.112:0/1814920198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b68067a20 con 0x7f5b6806e150 2023-08-27T11:18:04.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.941+0000 7f5b56ffd700 1 -- 172.21.15.112:0/1814920198 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+18330 (secure 0 0 0) 0x7f5b5c017090 con 0x7f5b6806e150 2023-08-27T11:18:04.943 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:04.943 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":32,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:03.637541+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:01.935213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:02.602607+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:04.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.943+0000 7f5b6cd02700 1 -- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b4806a0e0 msgr2=0x7f5b4806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.943+0000 7f5b6cd02700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b4806a0e0 0x7f5b4806c5a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5b58000c00 tx=0x7f5b5801c040 comp rx=0 tx=0).stop 2023-08-27T11:18:04.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.943+0000 7f5b6cd02700 1 -- 172.21.15.112:0/1814920198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b6806e150 msgr2=0x7f5b6815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:04.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.943+0000 7f5b6cd02700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b6806e150 0x7f5b6815dc90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00e9e0 tx=0x7f5b5c00ecf0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.943+0000 7f5b6cd02700 1 -- 172.21.15.112:0/1814920198 shutdown_connections 2023-08-27T11:18:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.943+0000 7f5b6cd02700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b4806a0e0 0x7f5b4806c5a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.943+0000 7f5b6cd02700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b6815d490 0x7f5b680adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.943+0000 7f5b6cd02700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b68066450 0x7f5b681f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.943+0000 7f5b6cd02700 1 --2- 172.21.15.112:0/1814920198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b6806e150 0x7f5b6815dc90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.943+0000 7f5b6cd02700 1 -- 172.21.15.112:0/1814920198 >> 172.21.15.112:0/1814920198 conn(0x7f5b680b1380 msgr2=0x7f5b68159940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.943+0000 7f5b6cd02700 1 -- 172.21.15.112:0/1814920198 shutdown_connections 2023-08-27T11:18:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:04.943+0000 7f5b6cd02700 1 -- 172.21.15.112:0/1814920198 wait complete. 2023-08-27T11:18:04.957 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:18:05.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.079+0000 7fe24caa4700 1 Processor -- start 2023-08-27T11:18:05.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.079+0000 7fe24caa4700 1 -- start start 2023-08-27T11:18:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.080+0000 7fe24caa4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe24813f470 0x7fe24815b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.080+0000 7fe24caa4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe248071510 0x7fe24813f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:05.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.080+0000 7fe24caa4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24815abd0 con 0x7fe24813fd90 2023-08-27T11:18:05.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.080+0000 7fe24caa4700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2481408f0 con 0x7fe24813f470 2023-08-27T11:18:05.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.080+0000 7fe24caa4700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24815a650 con 0x7fe248071510 2023-08-27T11:18:05.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.080+0000 7fe24659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe24813f470 0x7fe24815b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:05.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.081+0000 7fe246d9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe24813fd90 0x7fe24806ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51172/0 (socket says 172.21.15.112:51172) 2023-08-27T11:18:05.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.081+0000 7fe246d9d700 1 -- 172.21.15.112:0/86998945 learned_addr learned my addr 172.21.15.112:0/86998945 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:05.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.081+0000 7fe24659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe24813f470 0x7fe24815b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56786/0 (socket says 172.21.15.112:56786) 2023-08-27T11:18:05.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.081+0000 7fe24659c700 1 -- 172.21.15.112:0/86998945 >> v2:172.21.15.138:3300/0 conn(0x7fe248071510 msgr2=0x7fe24813f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:05.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.081+0000 7fe24659c700 1 --2- 172.21.15.112:0/86998945 >> v2:172.21.15.138:3300/0 conn(0x7fe248071510 0x7fe24813f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:05.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.081+0000 7fe24659c700 1 -- 172.21.15.112:0/86998945 >> v1:172.21.15.112:6789/0 conn(0x7fe24813fd90 legacy=0x7fe24806ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:05.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.081+0000 7fe24659c700 1 -- 172.21.15.112:0/86998945 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe248140330 con 0x7fe24813f470 2023-08-27T11:18:05.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.081+0000 7fe24559a700 1 -- 172.21.15.112:0/86998945 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1064754337 0 0) 0x7fe24815abd0 con 0x7fe24813fd90 2023-08-27T11:18:05.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.081+0000 7fe24659c700 1 --2- 172.21.15.112:0/86998945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24813f470 0x7fe24815b060 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe248143f30 tx=0x7fe23800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8690bbbaf679ab server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:05.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.081+0000 7fe24559a700 1 -- 172.21.15.112:0/86998945 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23800e070 con 0x7fe24813f470 2023-08-27T11:18:05.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.081+0000 7fe24559a700 1 -- 172.21.15.112:0/86998945 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23800bbd0 con 0x7fe24813f470 2023-08-27T11:18:05.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.081+0000 7fe24559a700 1 -- 172.21.15.112:0/86998945 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238012430 con 0x7fe24813f470 2023-08-27T11:18:05.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.082+0000 7fe24caa4700 1 -- 172.21.15.112:0/86998945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24813f470 msgr2=0x7fe24815b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:05.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.082+0000 7fe24caa4700 1 --2- 172.21.15.112:0/86998945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24813f470 0x7fe24815b060 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe248143f30 tx=0x7fe23800b330 comp rx=0 tx=0).stop 2023-08-27T11:18:05.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.082+0000 7fe24caa4700 1 -- 172.21.15.112:0/86998945 shutdown_connections 2023-08-27T11:18:05.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.082+0000 7fe24caa4700 1 --2- 172.21.15.112:0/86998945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24813f470 0x7fe24815b060 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:05.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.082+0000 7fe24caa4700 1 --2- 172.21.15.112:0/86998945 >> v2:172.21.15.138:3300/0 conn(0x7fe248071510 0x7fe24813f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:05.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.082+0000 7fe24caa4700 1 -- 172.21.15.112:0/86998945 >> 172.21.15.112:0/86998945 conn(0x7fe2480cc240 msgr2=0x7fe2480cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:05.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.082+0000 7fe24caa4700 1 -- 172.21.15.112:0/86998945 shutdown_connections 2023-08-27T11:18:05.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.082+0000 7fe24caa4700 1 -- 172.21.15.112:0/86998945 wait complete. 2023-08-27T11:18:05.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.083+0000 7fe24caa4700 1 Processor -- start 2023-08-27T11:18:05.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.083+0000 7fe24caa4700 1 -- start start 2023-08-27T11:18:05.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe24caa4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe248071510 0x7fe24813d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:05.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe24caa4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe24813f470 0x7fe248158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:05.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe24caa4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24813fd90 0x7fe2480b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:05.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe24caa4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2480ad920 con 0x7fe24813fd90 2023-08-27T11:18:05.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe24caa4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24813dd70 con 0x7fe24813f470 2023-08-27T11:18:05.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe24caa4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2480acd10 con 0x7fe248071510 2023-08-27T11:18:05.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe246d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24813fd90 0x7fe2480b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:05.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe246d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24813fd90 0x7fe2480b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56796/0 (socket says 172.21.15.112:56796) 2023-08-27T11:18:05.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe246d9d700 1 -- 172.21.15.112:0/2339310555 learned_addr learned my addr 172.21.15.112:0/2339310555 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:05.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe245d9b700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe24813f470 0x7fe248158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:05.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe24659c700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe248071510 0x7fe24813d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:05.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe246d9d700 1 -- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe248071510 msgr2=0x7fe24813d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe246d9d700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe248071510 0x7fe24813d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe246d9d700 1 -- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe24813f470 msgr2=0x7fe248158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe246d9d700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe24813f470 0x7fe248158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe246d9d700 1 -- 172.21.15.112:0/2339310555 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe238009a70 con 0x7fe24813fd90 2023-08-27T11:18:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe24659c700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe248071510 0x7fe24813d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:05.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.084+0000 7fe246d9d700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24813fd90 0x7fe2480b0560 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe2400011e0 tx=0x7fe24000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:05.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.085+0000 7fe23effd700 1 -- 172.21.15.112:0/2339310555 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240004c80 con 0x7fe24813fd90 2023-08-27T11:18:05.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.085+0000 7fe23effd700 1 -- 172.21.15.112:0/2339310555 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe240004de0 con 0x7fe24813fd90 2023-08-27T11:18:05.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.085+0000 7fe24caa4700 1 -- 172.21.15.112:0/2339310555 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2480acff0 con 0x7fe24813fd90 2023-08-27T11:18:05.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.085+0000 7fe23effd700 1 -- 172.21.15.112:0/2339310555 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2400072d0 con 0x7fe24813fd90 2023-08-27T11:18:05.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.085+0000 7fe24caa4700 1 -- 172.21.15.112:0/2339310555 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2480ad5b0 con 0x7fe24813fd90 2023-08-27T11:18:05.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.086+0000 7fe24caa4700 1 -- 172.21.15.112:0/2339310555 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe248067e90 con 0x7fe24813fd90 2023-08-27T11:18:05.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.086+0000 7fe23effd700 1 -- 172.21.15.112:0/2339310555 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe240007430 con 0x7fe24813fd90 2023-08-27T11:18:05.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.086+0000 7fe23effd700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe22806a0e0 0x7fe22806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:05.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.086+0000 7fe23effd700 1 -- 172.21.15.112:0/2339310555 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe240088150 con 0x7fe24813fd90 2023-08-27T11:18:05.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.087+0000 7fe24659c700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe22806a0e0 0x7fe22806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:05.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.087+0000 7fe24659c700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe22806a0e0 0x7fe22806c5a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe238014040 tx=0x7fe238011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:05.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.091+0000 7fe23effd700 1 -- 172.21.15.112:0/2339310555 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe2400538d0 con 0x7fe24813fd90 2023-08-27T11:18:05.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:05.326+0000 7fe24caa4700 1 -- 172.21.15.112:0/2339310555 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe24815b570 con 0x7fe24813fd90 2023-08-27T11:18:05.565 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:18:05.569 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:18:06.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.257+0000 7fe23effd700 1 -- 172.21.15.112:0/2339310555 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe240056ef0 con 0x7fe24813fd90 2023-08-27T11:18:06.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.260+0000 7fe24caa4700 1 -- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe22806a0e0 msgr2=0x7fe22806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:06.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.260+0000 7fe24caa4700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe22806a0e0 0x7fe22806c5a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe238014040 tx=0x7fe238011000 comp rx=0 tx=0).stop 2023-08-27T11:18:06.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.260+0000 7fe24caa4700 1 -- 172.21.15.112:0/2339310555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24813fd90 msgr2=0x7fe2480b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:06.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.260+0000 7fe24caa4700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24813fd90 0x7fe2480b0560 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe2400011e0 tx=0x7fe24000c800 comp rx=0 tx=0).stop 2023-08-27T11:18:06.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.260+0000 7fe24caa4700 1 -- 172.21.15.112:0/2339310555 shutdown_connections 2023-08-27T11:18:06.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.260+0000 7fe24caa4700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe22806a0e0 0x7fe22806c5a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:06.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.260+0000 7fe24caa4700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24813fd90 0x7fe2480b0560 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:06.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.260+0000 7fe24caa4700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe248071510 0x7fe24813d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:06.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.260+0000 7fe24caa4700 1 --2- 172.21.15.112:0/2339310555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe24813f470 0x7fe248158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:06.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.260+0000 7fe24caa4700 1 -- 172.21.15.112:0/2339310555 >> 172.21.15.112:0/2339310555 conn(0x7fe2480cc240 msgr2=0x7fe2480cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:06.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.260+0000 7fe24caa4700 1 -- 172.21.15.112:0/2339310555 shutdown_connections 2023-08-27T11:18:06.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.260+0000 7fe24caa4700 1 -- 172.21.15.112:0/2339310555 wait complete. 2023-08-27T11:18:06.276 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:18:06.277 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:18:06.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.405+0000 7ff098558700 1 Processor -- start 2023-08-27T11:18:06.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.405+0000 7ff098558700 1 -- start start 2023-08-27T11:18:06.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.406+0000 7ff098558700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09006dcc0 0x7ff0900b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:06.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.406+0000 7ff098558700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0900cfd50 con 0x7ff0901437c0 2023-08-27T11:18:06.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.406+0000 7ff098558700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0900cf140 con 0x7ff090143ba0 2023-08-27T11:18:06.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.406+0000 7ff098558700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900cf420 con 0x7ff09006dcc0 2023-08-27T11:18:06.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.406+0000 7ff0962f4700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff0901437c0 0x7ff0900ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51182/0 (socket says 172.21.15.112:51182) 2023-08-27T11:18:06.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.406+0000 7ff0962f4700 1 -- 172.21.15.112:0/4111722749 learned_addr learned my addr 172.21.15.112:0/4111722749 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:06.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff096af5700 1 --2- 172.21.15.112:0/4111722749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09006dcc0 0x7ff0900b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:06.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496091598 0 0) 0x7ff0900cfd50 con 0x7ff0901437c0 2023-08-27T11:18:06.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff07c0035b0 con 0x7ff0901437c0 2023-08-27T11:18:06.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2795137842 0 0) 0x7ff07c0035b0 con 0x7ff0901437c0 2023-08-27T11:18:06.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09006dcc0 msgr2=0x7ff0900b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:06.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 --2- 172.21.15.112:0/4111722749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09006dcc0 0x7ff0900b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:06.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 >> v1:172.21.15.138:6789/0 conn(0x7ff090143ba0 legacy=0x7ff09006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:06.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0900cf700 con 0x7ff0901437c0 2023-08-27T11:18:06.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7ff080001670 con 0x7ff0901437c0 2023-08-27T11:18:06.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 >> v1:172.21.15.112:6789/0 conn(0x7ff0901437c0 legacy=0x7ff0900ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:06.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 --2- 172.21.15.112:0/4111722749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09006dcc0 0x7ff07c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:06.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 --2- 172.21.15.112:0/4111722749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff07c003fb0 0x7ff07c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:06.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 --2- 172.21.15.112:0/4111722749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c004d90 0x7ff07c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:06.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900cf420 con 0x7ff07c003fb0 2023-08-27T11:18:06.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c00baf0 con 0x7ff07c004d90 2023-08-27T11:18:06.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c00bcb0 con 0x7ff09006dcc0 2023-08-27T11:18:06.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff095af3700 1 --2- 172.21.15.112:0/4111722749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff07c003fb0 0x7ff07c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:06.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff0962f4700 1 --2- 172.21.15.112:0/4111722749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09006dcc0 0x7ff07c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:06.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff096af5700 1 --2- 172.21.15.112:0/4111722749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c004d90 0x7ff07c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:06.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff095af3700 1 -- 172.21.15.112:0/4111722749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09006dcc0 msgr2=0x7ff07c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:06.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff095af3700 1 --2- 172.21.15.112:0/4111722749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09006dcc0 0x7ff07c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:06.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff095af3700 1 -- 172.21.15.112:0/4111722749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c004d90 msgr2=0x7ff07c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:06.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff095af3700 1 --2- 172.21.15.112:0/4111722749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c004d90 0x7ff07c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:06.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff095af3700 1 -- 172.21.15.112:0/4111722749 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c0035b0 con 0x7ff07c003fb0 2023-08-27T11:18:06.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.407+0000 7ff095af3700 1 --2- 172.21.15.112:0/4111722749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff07c003fb0 0x7ff07c004850 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0017c0 tx=0x7ff07c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d406fb625cab7bf3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:06.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.408+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0052d0 con 0x7ff07c003fb0 2023-08-27T11:18:06.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.408+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c005430 con 0x7ff07c003fb0 2023-08-27T11:18:06.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.408+0000 7ff0952f2700 1 -- 172.21.15.112:0/4111722749 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0038c0 con 0x7ff07c003fb0 2023-08-27T11:18:06.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.408+0000 7ff098558700 1 -- 172.21.15.112:0/4111722749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff07c003fb0 msgr2=0x7ff07c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:06.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.408+0000 7ff098558700 1 --2- 172.21.15.112:0/4111722749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff07c003fb0 0x7ff07c004850 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0017c0 tx=0x7ff07c005880 comp rx=0 tx=0).stop 2023-08-27T11:18:06.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.408+0000 7ff096af5700 1 -- 172.21.15.112:0/4111722749 reap_dead start 2023-08-27T11:18:06.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.408+0000 7ff098558700 1 -- 172.21.15.112:0/4111722749 shutdown_connections 2023-08-27T11:18:06.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.408+0000 7ff098558700 1 -- 172.21.15.112:0/4111722749 >> 172.21.15.112:0/4111722749 conn(0x7ff0900b1e90 msgr2=0x7ff0900b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:06.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.408+0000 7ff098558700 1 -- 172.21.15.112:0/4111722749 shutdown_connections 2023-08-27T11:18:06.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.408+0000 7ff098558700 1 -- 172.21.15.112:0/4111722749 wait complete. 2023-08-27T11:18:06.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.409+0000 7ff098558700 1 Processor -- start 2023-08-27T11:18:06.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.410+0000 7ff098558700 1 -- start start 2023-08-27T11:18:06.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.410+0000 7ff098558700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c003fb0 0x7ff090153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:06.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.410+0000 7ff098558700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0901539c0 0x7ff0901317d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:06.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.410+0000 7ff098558700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff090131d10 0x7ff0900b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:06.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.410+0000 7ff098558700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff090157320 con 0x7ff090131d10 2023-08-27T11:18:06.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.410+0000 7ff098558700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff090132320 con 0x7ff07c003fb0 2023-08-27T11:18:06.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.410+0000 7ff098558700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff090132600 con 0x7ff0901539c0 2023-08-27T11:18:06.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.410+0000 7ff096af5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff090131d10 0x7ff0900b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:06.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.410+0000 7ff096af5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff090131d10 0x7ff0900b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56816/0 (socket says 172.21.15.112:56816) 2023-08-27T11:18:06.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.410+0000 7ff096af5700 1 -- 172.21.15.112:0/527326378 learned_addr learned my addr 172.21.15.112:0/527326378 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:06.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.410+0000 7ff095af3700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0901539c0 0x7ff0901317d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:06.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.410+0000 7ff0962f4700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c003fb0 0x7ff090153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:06.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.411+0000 7ff096af5700 1 -- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0901539c0 msgr2=0x7ff0901317d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:06.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.411+0000 7ff096af5700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0901539c0 0x7ff0901317d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:06.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.411+0000 7ff096af5700 1 -- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c003fb0 msgr2=0x7ff090153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:06.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.411+0000 7ff096af5700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c003fb0 0x7ff090153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:06.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.411+0000 7ff096af5700 1 -- 172.21.15.112:0/527326378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0901328e0 con 0x7ff090131d10 2023-08-27T11:18:06.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.411+0000 7ff096af5700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff090131d10 0x7ff0900b3b20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff0900b2d80 tx=0x7ff088011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:06.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.411+0000 7ff086ffd700 1 -- 172.21.15.112:0/527326378 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08800e040 con 0x7ff090131d10 2023-08-27T11:18:06.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.411+0000 7ff086ffd700 1 -- 172.21.15.112:0/527326378 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08800f040 con 0x7ff090131d10 2023-08-27T11:18:06.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.411+0000 7ff086ffd700 1 -- 172.21.15.112:0/527326378 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08800f9a0 con 0x7ff090131d10 2023-08-27T11:18:06.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.411+0000 7ff098558700 1 -- 172.21.15.112:0/527326378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff090132bc0 con 0x7ff090131d10 2023-08-27T11:18:06.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.411+0000 7ff098558700 1 -- 172.21.15.112:0/527326378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0900b4060 con 0x7ff090131d10 2023-08-27T11:18:06.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.412+0000 7ff098558700 1 -- 172.21.15.112:0/527326378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0901574f0 con 0x7ff090131d10 2023-08-27T11:18:06.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.412+0000 7ff086ffd700 1 -- 172.21.15.112:0/527326378 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff08800fb00 con 0x7ff090131d10 2023-08-27T11:18:06.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.413+0000 7ff086ffd700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff07406a0e0 0x7ff07406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:06.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.413+0000 7ff086ffd700 1 -- 172.21.15.112:0/527326378 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff088088c70 con 0x7ff090131d10 2023-08-27T11:18:06.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.413+0000 7ff0962f4700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff07406a0e0 0x7ff07406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:06.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.413+0000 7ff0962f4700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff07406a0e0 0x7ff07406c5a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff0900b3760 tx=0x7ff080009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:06.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.417+0000 7ff086ffd700 1 -- 172.21.15.112:0/527326378 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff0901574f0 con 0x7ff090131d10 2023-08-27T11:18:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:06.651+0000 7ff098558700 1 -- 172.21.15.112:0/527326378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff090067e90 con 0x7ff090131d10 2023-08-27T11:18:06.920 INFO:tasks.ceph.mds.b.smithi138.stderr:2023-08-27T11:18:06.919+0000 7fb7a80b6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:18:07.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.266+0000 7ff086ffd700 1 -- 172.21.15.112:0/527326378 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff0880543f0 con 0x7ff090131d10 2023-08-27T11:18:07.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.268+0000 7ff098558700 1 -- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff07406a0e0 msgr2=0x7ff07406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:07.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.268+0000 7ff098558700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff07406a0e0 0x7ff07406c5a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff0900b3760 tx=0x7ff080009040 comp rx=0 tx=0).stop 2023-08-27T11:18:07.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.268+0000 7ff098558700 1 -- 172.21.15.112:0/527326378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff090131d10 msgr2=0x7ff0900b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:07.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.268+0000 7ff098558700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff090131d10 0x7ff0900b3b20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff0900b2d80 tx=0x7ff088011f80 comp rx=0 tx=0).stop 2023-08-27T11:18:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.268+0000 7ff098558700 1 -- 172.21.15.112:0/527326378 shutdown_connections 2023-08-27T11:18:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.268+0000 7ff098558700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff07406a0e0 0x7ff07406c5a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.268+0000 7ff098558700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff090131d10 0x7ff0900b3b20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.268+0000 7ff098558700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c003fb0 0x7ff090153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.269+0000 7ff098558700 1 --2- 172.21.15.112:0/527326378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0901539c0 0x7ff0901317d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.269+0000 7ff098558700 1 -- 172.21.15.112:0/527326378 >> 172.21.15.112:0/527326378 conn(0x7ff0900b1e90 msgr2=0x7ff09013b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:07.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.269+0000 7ff098558700 1 -- 172.21.15.112:0/527326378 shutdown_connections 2023-08-27T11:18:07.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.269+0000 7ff098558700 1 -- 172.21.15.112:0/527326378 wait complete. 2023-08-27T11:18:07.283 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:18:07.283 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:18:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.406+0000 7f8a5c838700 1 Processor -- start 2023-08-27T11:18:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.406+0000 7f8a5c838700 1 -- start start 2023-08-27T11:18:07.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a5c838700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8a58158c90 0x7f8a5806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:07.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a5c838700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8a58159070 0x7f8a5806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:07.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a5c838700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a580b1270 0x7f8a5806fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:07.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a5c838700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5815f5d0 con 0x7f8a58159070 2023-08-27T11:18:07.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a5c838700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a580ca990 con 0x7f8a58158c90 2023-08-27T11:18:07.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a5c838700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5806e870 con 0x7f8a580b1270 2023-08-27T11:18:07.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a55d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8a58159070 0x7f8a5806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:07.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a55d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8a58159070 0x7f8a5806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56822/0 (socket says 172.21.15.112:56822) 2023-08-27T11:18:07.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a55d9b700 1 -- 172.21.15.112:0/3349982504 learned_addr learned my addr 172.21.15.112:0/3349982504 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:07.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a5659c700 1 --2- 172.21.15.112:0/3349982504 >> v2:172.21.15.138:3300/0 conn(0x7f8a58158c90 0x7f8a5806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:07.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a56d9d700 1 --2- 172.21.15.112:0/3349982504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a580b1270 0x7f8a5806fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:07.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a55d9b700 1 -- 172.21.15.112:0/3349982504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a580b1270 msgr2=0x7f8a5806fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:07.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a55d9b700 1 --2- 172.21.15.112:0/3349982504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a580b1270 0x7f8a5806fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a55d9b700 1 -- 172.21.15.112:0/3349982504 >> v2:172.21.15.138:3300/0 conn(0x7f8a58158c90 msgr2=0x7f8a5806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:07.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a55d9b700 1 --2- 172.21.15.112:0/3349982504 >> v2:172.21.15.138:3300/0 conn(0x7f8a58158c90 0x7f8a5806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a55d9b700 1 -- 172.21.15.112:0/3349982504 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580b1a10 con 0x7f8a58159070 2023-08-27T11:18:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a56d9d700 1 --2- 172.21.15.112:0/3349982504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a580b1270 0x7f8a5806fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a55d9b700 1 --2- 172.21.15.112:0/3349982504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a58159070 0x7f8a5806f790 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8a580cc9e0 tx=0x7f8a4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ff4f3103554914 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a5559a700 1 -- 172.21.15.112:0/3349982504 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c00b870 con 0x7f8a58159070 2023-08-27T11:18:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a5559a700 1 -- 172.21.15.112:0/3349982504 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4c00b9d0 con 0x7f8a58159070 2023-08-27T11:18:07.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.407+0000 7f8a5559a700 1 -- 172.21.15.112:0/3349982504 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c00bd60 con 0x7f8a58159070 2023-08-27T11:18:07.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.408+0000 7f8a5c838700 1 -- 172.21.15.112:0/3349982504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a58159070 msgr2=0x7f8a5806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:07.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.408+0000 7f8a5c838700 1 --2- 172.21.15.112:0/3349982504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a58159070 0x7f8a5806f790 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8a580cc9e0 tx=0x7f8a4c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.408+0000 7f8a5c838700 1 -- 172.21.15.112:0/3349982504 shutdown_connections 2023-08-27T11:18:07.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.408+0000 7f8a5c838700 1 --2- 172.21.15.112:0/3349982504 >> v2:172.21.15.138:3300/0 conn(0x7f8a58158c90 0x7f8a5806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.408+0000 7f8a5c838700 1 --2- 172.21.15.112:0/3349982504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a580b1270 0x7f8a5806fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.408+0000 7f8a5c838700 1 --2- 172.21.15.112:0/3349982504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a58159070 0x7f8a5806f790 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.408+0000 7f8a5c838700 1 -- 172.21.15.112:0/3349982504 >> 172.21.15.112:0/3349982504 conn(0x7f8a580d0000 msgr2=0x7f8a580d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.408+0000 7f8a5c838700 1 -- 172.21.15.112:0/3349982504 shutdown_connections 2023-08-27T11:18:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.408+0000 7f8a5c838700 1 -- 172.21.15.112:0/3349982504 wait complete. 2023-08-27T11:18:07.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.409+0000 7f8a5c838700 1 Processor -- start 2023-08-27T11:18:07.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.409+0000 7f8a5c838700 1 -- start start 2023-08-27T11:18:07.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5c838700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a580b1270 0x7f8a581f1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:07.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5c838700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a58158c90 0x7f8a580c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:07.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5c838700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a58159070 0x7f8a581f1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:07.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5c838700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a581f15a0 con 0x7f8a580b1270 2023-08-27T11:18:07.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5c838700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580c3b10 con 0x7f8a58158c90 2023-08-27T11:18:07.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5c838700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580c3df0 con 0x7f8a58159070 2023-08-27T11:18:07.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a580b1270 0x7f8a581f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:07.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a580b1270 0x7f8a581f1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56826/0 (socket says 172.21.15.112:56826) 2023-08-27T11:18:07.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5659c700 1 -- 172.21.15.112:0/3807581346 learned_addr learned my addr 172.21.15.112:0/3807581346 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:07.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5659c700 1 -- 172.21.15.112:0/3807581346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a58159070 msgr2=0x7f8a581f1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:07.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5659c700 1 --2- 172.21.15.112:0/3807581346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a58159070 0x7f8a581f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5659c700 1 -- 172.21.15.112:0/3807581346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a58158c90 msgr2=0x7f8a580c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:07.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5659c700 1 --2- 172.21.15.112:0/3807581346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a58158c90 0x7f8a580c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.410+0000 7f8a5659c700 1 -- 172.21.15.112:0/3807581346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c009550 con 0x7f8a580b1270 2023-08-27T11:18:07.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.411+0000 7f8a5659c700 1 --2- 172.21.15.112:0/3807581346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a580b1270 0x7f8a581f1060 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800e810 tx=0x7f8a4800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:07.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.411+0000 7f8a46ffd700 1 -- 172.21.15.112:0/3807581346 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48013610 con 0x7f8a580b1270 2023-08-27T11:18:07.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.411+0000 7f8a46ffd700 1 -- 172.21.15.112:0/3807581346 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a48013770 con 0x7f8a580b1270 2023-08-27T11:18:07.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.411+0000 7f8a5c838700 1 -- 172.21.15.112:0/3807581346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a580c40d0 con 0x7f8a580b1270 2023-08-27T11:18:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.411+0000 7f8a46ffd700 1 -- 172.21.15.112:0/3807581346 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48013b20 con 0x7f8a580b1270 2023-08-27T11:18:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.411+0000 7f8a5c838700 1 -- 172.21.15.112:0/3807581346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a580c4690 con 0x7f8a580b1270 2023-08-27T11:18:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.412+0000 7f8a5c838700 1 -- 172.21.15.112:0/3807581346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a58067a20 con 0x7f8a580b1270 2023-08-27T11:18:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.412+0000 7f8a46ffd700 1 -- 172.21.15.112:0/3807581346 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8a48013c80 con 0x7f8a580b1270 2023-08-27T11:18:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.412+0000 7f8a46ffd700 1 --2- 172.21.15.112:0/3807581346 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a3806a0e0 0x7f8a3806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:07.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.413+0000 7f8a46ffd700 1 -- 172.21.15.112:0/3807581346 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8a48088210 con 0x7f8a580b1270 2023-08-27T11:18:07.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.413+0000 7f8a55d9b700 1 --2- 172.21.15.112:0/3807581346 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a3806a0e0 0x7f8a3806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:07.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.413+0000 7f8a55d9b700 1 --2- 172.21.15.112:0/3807581346 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a3806a0e0 0x7f8a3806c5a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8a58159450 tx=0x7f8a4c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:07.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.417+0000 7f8a46ffd700 1 -- 172.21.15.112:0/3807581346 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8a480573a0 con 0x7f8a580b1270 2023-08-27T11:18:07.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.647+0000 7f8a5c838700 1 -- 172.21.15.112:0/3807581346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a580c6960 con 0x7f8a580b1270 2023-08-27T11:18:07.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.648+0000 7f8a46ffd700 1 -- 172.21.15.112:0/3807581346 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4887 (secure 0 0 0) 0x7f8a48056f30 con 0x7f8a580b1270 2023-08-27T11:18:07.649 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:07.650 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1941625698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1941625698},{"type":"v1","addr":"172.21.15.138:6836","nonce":1941625698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1530886230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1530886230},{"type":"v1","addr":"172.21.15.112:6834","nonce":1530886230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1024768929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1024768929},{"type":"v1","addr":"172.21.15.112:6835","nonce":1024768929}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:18:03.637863+0000","modified":"2023-08-27T11:18:07.262281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5175":{"gid":5175,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6837/3409848660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3409848660},{"type":"v1","addr":"172.21.15.138:6837","nonce":3409848660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-27T11:18:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.649+0000 7f8a5c838700 1 -- 172.21.15.112:0/3807581346 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a3806a0e0 msgr2=0x7f8a3806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.649+0000 7f8a5c838700 1 --2- 172.21.15.112:0/3807581346 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a3806a0e0 0x7f8a3806c5a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8a58159450 tx=0x7f8a4c010000 comp rx=0 tx=0).stop 2023-08-27T11:18:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.649+0000 7f8a5c838700 1 -- 172.21.15.112:0/3807581346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a580b1270 msgr2=0x7f8a581f1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.649+0000 7f8a5c838700 1 --2- 172.21.15.112:0/3807581346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a580b1270 0x7f8a581f1060 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800e810 tx=0x7f8a4800ebd0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.650+0000 7f8a5c838700 1 -- 172.21.15.112:0/3807581346 shutdown_connections 2023-08-27T11:18:07.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.650+0000 7f8a5c838700 1 --2- 172.21.15.112:0/3807581346 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a3806a0e0 0x7f8a3806c5a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.650+0000 7f8a5c838700 1 --2- 172.21.15.112:0/3807581346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a58159070 0x7f8a581f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.650+0000 7f8a5c838700 1 --2- 172.21.15.112:0/3807581346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a580b1270 0x7f8a581f1060 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.650+0000 7f8a5c838700 1 --2- 172.21.15.112:0/3807581346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a58158c90 0x7f8a580c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.650+0000 7f8a5c838700 1 -- 172.21.15.112:0/3807581346 >> 172.21.15.112:0/3807581346 conn(0x7f8a580d0000 msgr2=0x7f8a58159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:07.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.650+0000 7f8a5c838700 1 -- 172.21.15.112:0/3807581346 shutdown_connections 2023-08-27T11:18:07.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.650+0000 7f8a5c838700 1 -- 172.21.15.112:0/3807581346 wait complete. 2023-08-27T11:18:07.655 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 15 2023-08-27T11:18:07.663 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:07.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.781+0000 7f7e1fe12700 1 Processor -- start 2023-08-27T11:18:07.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.782+0000 7f7e1fe12700 1 -- start start 2023-08-27T11:18:07.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.782+0000 7f7e1fe12700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7e1806fea0 0x7f7e180cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:07.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.782+0000 7f7e1fe12700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e18161850 con 0x7f7e180b1400 2023-08-27T11:18:07.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.782+0000 7f7e1fe12700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e180cfc10 con 0x7f7e180b17e0 2023-08-27T11:18:07.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.782+0000 7f7e1fe12700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e18161200 con 0x7f7e1806fea0 2023-08-27T11:18:07.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.782+0000 7f7e1e3af700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7e1806fea0 0x7f7e180cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:07.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.783+0000 7f7e1e3af700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7e1806fea0 0x7f7e180cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56838/0 (socket says 172.21.15.112:56838) 2023-08-27T11:18:07.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.783+0000 7f7e1e3af700 1 -- 172.21.15.112:0/2767252659 learned_addr learned my addr 172.21.15.112:0/2767252659 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:07.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.783+0000 7f7e1dbae700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7e180b1400 0x7f7e1806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51216/0 (socket says 172.21.15.112:51216) 2023-08-27T11:18:07.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.783+0000 7f7e1e3af700 1 -- 172.21.15.112:0/2767252659 >> v1:172.21.15.138:6789/0 conn(0x7f7e180b17e0 legacy=0x7f7e1806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:07.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.783+0000 7f7e1e3af700 1 -- 172.21.15.112:0/2767252659 >> v1:172.21.15.112:6789/0 conn(0x7f7e180b1400 legacy=0x7f7e1806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:07.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.783+0000 7f7e1e3af700 1 -- 172.21.15.112:0/2767252659 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e181614e0 con 0x7f7e1806fea0 2023-08-27T11:18:07.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.783+0000 7f7e1cbac700 1 -- 172.21.15.112:0/2767252659 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922678153 0 0) 0x7f7e18161850 con 0x7f7e180b1400 2023-08-27T11:18:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.783+0000 7f7e1e3af700 1 --2- 172.21.15.112:0/2767252659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e1806fea0 0x7f7e180cf5a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009e50 tx=0x7f7e1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72af7f28f4ce7abc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.783+0000 7f7e1cbac700 1 -- 172.21.15.112:0/2767252659 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1000e070 con 0x7f7e1806fea0 2023-08-27T11:18:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.783+0000 7f7e1cbac700 1 -- 172.21.15.112:0/2767252659 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e10003a20 con 0x7f7e1806fea0 2023-08-27T11:18:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.783+0000 7f7e1cbac700 1 -- 172.21.15.112:0/2767252659 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10003dd0 con 0x7f7e1806fea0 2023-08-27T11:18:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.784+0000 7f7e1fe12700 1 -- 172.21.15.112:0/2767252659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e1806fea0 msgr2=0x7f7e180cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.784+0000 7f7e1fe12700 1 --2- 172.21.15.112:0/2767252659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e1806fea0 0x7f7e180cf5a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009e50 tx=0x7f7e1000b330 comp rx=0 tx=0).stop 2023-08-27T11:18:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.784+0000 7f7e1fe12700 1 -- 172.21.15.112:0/2767252659 shutdown_connections 2023-08-27T11:18:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.784+0000 7f7e1fe12700 1 --2- 172.21.15.112:0/2767252659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e1806fea0 0x7f7e180cf5a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.784+0000 7f7e1fe12700 1 -- 172.21.15.112:0/2767252659 >> 172.21.15.112:0/2767252659 conn(0x7f7e180d0000 msgr2=0x7f7e180d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.784+0000 7f7e1fe12700 1 -- 172.21.15.112:0/2767252659 shutdown_connections 2023-08-27T11:18:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.784+0000 7f7e1fe12700 1 -- 172.21.15.112:0/2767252659 wait complete. 2023-08-27T11:18:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.785+0000 7f7e1fe12700 1 Processor -- start 2023-08-27T11:18:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.785+0000 7f7e1fe12700 1 -- start start 2023-08-27T11:18:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.786+0000 7f7e1fe12700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e1806fea0 0x7f7e180c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.786+0000 7f7e1fe12700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e180b1400 0x7f7e180c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.786+0000 7f7e1fe12700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e180b17e0 0x7f7e181526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.786+0000 7f7e1fe12700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180c9e90 con 0x7f7e180b17e0 2023-08-27T11:18:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.786+0000 7f7e1fe12700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180ca170 con 0x7f7e1806fea0 2023-08-27T11:18:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.786+0000 7f7e1fe12700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180c3e40 con 0x7f7e180b1400 2023-08-27T11:18:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.786+0000 7f7e1e3af700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e180b17e0 0x7f7e181526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.786+0000 7f7e1e3af700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e180b17e0 0x7f7e181526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56844/0 (socket says 172.21.15.112:56844) 2023-08-27T11:18:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.786+0000 7f7e1e3af700 1 -- 172.21.15.112:0/625723228 learned_addr learned my addr 172.21.15.112:0/625723228 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.787+0000 7f7e1d3ad700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e180b1400 0x7f7e180c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.787+0000 7f7e1dbae700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e1806fea0 0x7f7e180c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.787+0000 7f7e1e3af700 1 -- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e180b1400 msgr2=0x7f7e180c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.787+0000 7f7e1e3af700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e180b1400 0x7f7e180c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.787+0000 7f7e1e3af700 1 -- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e1806fea0 msgr2=0x7f7e180c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.787+0000 7f7e1e3af700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e1806fea0 0x7f7e180c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.787+0000 7f7e1e3af700 1 -- 172.21.15.112:0/625723228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e10009b30 con 0x7f7e180b17e0 2023-08-27T11:18:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.787+0000 7f7e1e3af700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e180b17e0 0x7f7e181526e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7e180d1100 tx=0x7f7e10004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.787+0000 7f7e0e7fc700 1 -- 172.21.15.112:0/625723228 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10004240 con 0x7f7e180b17e0 2023-08-27T11:18:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.787+0000 7f7e0e7fc700 1 -- 172.21.15.112:0/625723228 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e100043a0 con 0x7f7e180b17e0 2023-08-27T11:18:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.787+0000 7f7e1fe12700 1 -- 172.21.15.112:0/625723228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e180c4120 con 0x7f7e180b17e0 2023-08-27T11:18:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.787+0000 7f7e0e7fc700 1 -- 172.21.15.112:0/625723228 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10004750 con 0x7f7e180b17e0 2023-08-27T11:18:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.787+0000 7f7e1fe12700 1 -- 172.21.15.112:0/625723228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e18152c20 con 0x7f7e180b17e0 2023-08-27T11:18:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.788+0000 7f7e1fe12700 1 -- 172.21.15.112:0/625723228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e180ca390 con 0x7f7e180b17e0 2023-08-27T11:18:07.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.788+0000 7f7e0e7fc700 1 -- 172.21.15.112:0/625723228 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7e10019070 con 0x7f7e180b17e0 2023-08-27T11:18:07.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.789+0000 7f7e0e7fc700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7dfc06a0e0 0x7f7dfc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:07.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.789+0000 7f7e0e7fc700 1 -- 172.21.15.112:0/625723228 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e10089570 con 0x7f7e180b17e0 2023-08-27T11:18:07.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.789+0000 7f7e1dbae700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7dfc06a0e0 0x7f7dfc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:07.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.789+0000 7f7e1dbae700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7dfc06a0e0 0x7f7dfc06c5a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e08000f60 tx=0x7f7e08007be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:07.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.793+0000 7f7e0e7fc700 1 -- 172.21.15.112:0/625723228 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7e10054c70 con 0x7f7e180b17e0 2023-08-27T11:18:07.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.983+0000 7f7e1fe12700 1 -- 172.21.15.112:0/625723228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e180c4310 con 0x7f7e180b17e0 2023-08-27T11:18:07.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.985+0000 7f7e0e7fc700 1 -- 172.21.15.112:0/625723228 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+18317 (secure 0 0 0) 0x7f7e10058290 con 0x7f7e180b17e0 2023-08-27T11:18:07.986 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:07.986 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":33,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:05.246738+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:01.935213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:02.602607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:07.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.986+0000 7f7e1fe12700 1 -- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7dfc06a0e0 msgr2=0x7f7dfc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:07.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.986+0000 7f7e1fe12700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7dfc06a0e0 0x7f7dfc06c5a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e08000f60 tx=0x7f7e08007be0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.986+0000 7f7e1fe12700 1 -- 172.21.15.112:0/625723228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e180b17e0 msgr2=0x7f7e181526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:07.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.986+0000 7f7e1fe12700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e180b17e0 0x7f7e181526e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7e180d1100 tx=0x7f7e10004ad0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.987+0000 7f7e1fe12700 1 -- 172.21.15.112:0/625723228 shutdown_connections 2023-08-27T11:18:07.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.987+0000 7f7e1fe12700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7dfc06a0e0 0x7f7dfc06c5a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.987+0000 7f7e1fe12700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e180b17e0 0x7f7e181526e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.987+0000 7f7e1fe12700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e1806fea0 0x7f7e180c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.987+0000 7f7e1fe12700 1 --2- 172.21.15.112:0/625723228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e180b1400 0x7f7e180c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:07.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.987+0000 7f7e1fe12700 1 -- 172.21.15.112:0/625723228 >> 172.21.15.112:0/625723228 conn(0x7f7e180d0000 msgr2=0x7f7e1806ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:07.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.987+0000 7f7e1fe12700 1 -- 172.21.15.112:0/625723228 shutdown_connections 2023-08-27T11:18:07.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:07.987+0000 7f7e1fe12700 1 -- 172.21.15.112:0/625723228 wait complete. 2023-08-27T11:18:08.000 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:18:08.001 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:18:08.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.120+0000 7f8df709d700 1 Processor -- start 2023-08-27T11:18:08.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.120+0000 7f8df709d700 1 -- start start 2023-08-27T11:18:08.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8df709d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df013f990 0x7f8df00cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8df709d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df013ff50 con 0x7f8df00cd270 2023-08-27T11:18:08.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8df709d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df0140230 con 0x7f8df006e150 2023-08-27T11:18:08.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8df709d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df00cecb0 con 0x7f8df013f990 2023-08-27T11:18:08.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8df563a700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8df00cd270 0x7f8df015ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51234/0 (socket says 172.21.15.112:51234) 2023-08-27T11:18:08.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8df563a700 1 -- 172.21.15.112:0/1562532843 learned_addr learned my addr 172.21.15.112:0/1562532843 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:08.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8deffff700 1 --2- 172.21.15.112:0/1562532843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df013f990 0x7f8df00cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587774065 0 0) 0x7f8df013ff50 con 0x7f8df00cd270 2023-08-27T11:18:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8de00035b0 con 0x7f8df00cd270 2023-08-27T11:18:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2356979114 0 0) 0x7f8de00035b0 con 0x7f8df00cd270 2023-08-27T11:18:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df013f990 msgr2=0x7f8df00cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 --2- 172.21.15.112:0/1562532843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df013f990 0x7f8df00cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 >> v1:172.21.15.138:6789/0 conn(0x7f8df006e150 legacy=0x7f8df006e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df0140510 con 0x7f8df00cd270 2023-08-27T11:18:08.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f8de80019f0 con 0x7f8df00cd270 2023-08-27T11:18:08.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 >> v1:172.21.15.112:6789/0 conn(0x7f8df00cd270 legacy=0x7f8df015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 --2- 172.21.15.112:0/1562532843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de0003a30 0x7f8de0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 --2- 172.21.15.112:0/1562532843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df013f990 0x7f8de0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 --2- 172.21.15.112:0/1562532843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de0004d90 0x7f8de0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de000bac0 con 0x7f8de0003a30 2023-08-27T11:18:08.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de000bc30 con 0x7f8de0004d90 2023-08-27T11:18:08.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.121+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de000e000 con 0x7f8df013f990 2023-08-27T11:18:08.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.122+0000 7f8df4e39700 1 --2- 172.21.15.112:0/1562532843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de0003a30 0x7f8de0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.122+0000 7f8df563a700 1 --2- 172.21.15.112:0/1562532843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de0004d90 0x7f8de0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.122+0000 7f8deffff700 1 --2- 172.21.15.112:0/1562532843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df013f990 0x7f8de0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.122+0000 7f8df4e39700 1 -- 172.21.15.112:0/1562532843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df013f990 msgr2=0x7f8de0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:08.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.122+0000 7f8df4e39700 1 --2- 172.21.15.112:0/1562532843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df013f990 0x7f8de0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.122+0000 7f8df4e39700 1 -- 172.21.15.112:0/1562532843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de0004d90 msgr2=0x7f8de0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:08.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.122+0000 7f8df4e39700 1 --2- 172.21.15.112:0/1562532843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de0004d90 0x7f8de0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.122+0000 7f8df4e39700 1 -- 172.21.15.112:0/1562532843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de00035b0 con 0x7f8de0003a30 2023-08-27T11:18:08.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.122+0000 7f8df4e39700 1 --2- 172.21.15.112:0/1562532843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de0003a30 0x7f8de0003e50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8de4007fd0 tx=0x7f8de0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c335d51cfce4c9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:08.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.122+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de40052d0 con 0x7f8de0003a30 2023-08-27T11:18:08.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.122+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de4005470 con 0x7f8de0003a30 2023-08-27T11:18:08.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.122+0000 7f8def7fe700 1 -- 172.21.15.112:0/1562532843 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de40038c0 con 0x7f8de0003a30 2023-08-27T11:18:08.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.123+0000 7f8df709d700 1 -- 172.21.15.112:0/1562532843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de0003a30 msgr2=0x7f8de0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.123+0000 7f8df709d700 1 --2- 172.21.15.112:0/1562532843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de0003a30 0x7f8de0003e50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8de4007fd0 tx=0x7f8de0005880 comp rx=0 tx=0).stop 2023-08-27T11:18:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.123+0000 7f8df563a700 1 -- 172.21.15.112:0/1562532843 reap_dead start 2023-08-27T11:18:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.123+0000 7f8df709d700 1 -- 172.21.15.112:0/1562532843 shutdown_connections 2023-08-27T11:18:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.123+0000 7f8df709d700 1 -- 172.21.15.112:0/1562532843 >> 172.21.15.112:0/1562532843 conn(0x7f8df006cd30 msgr2=0x7f8df00717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.123+0000 7f8df709d700 1 -- 172.21.15.112:0/1562532843 shutdown_connections 2023-08-27T11:18:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.123+0000 7f8df709d700 1 -- 172.21.15.112:0/1562532843 wait complete. 2023-08-27T11:18:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.124+0000 7f8df709d700 1 Processor -- start 2023-08-27T11:18:08.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.124+0000 7f8df709d700 1 -- start start 2023-08-27T11:18:08.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.125+0000 7f8df709d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de0003a30 0x7f8df013c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.125+0000 7f8df709d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df006fd60 0x7f8df00701e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.125+0000 7f8df709d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df0159600 0x7f8df0159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.125+0000 7f8df709d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df0070720 con 0x7f8de0003a30 2023-08-27T11:18:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.125+0000 7f8df709d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df0159fc0 con 0x7f8df006fd60 2023-08-27T11:18:08.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.125+0000 7f8df709d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df00ac3f0 con 0x7f8df0159600 2023-08-27T11:18:08.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.125+0000 7f8df4e39700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de0003a30 0x7f8df013c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.125+0000 7f8df4e39700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de0003a30 0x7f8df013c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56862/0 (socket says 172.21.15.112:56862) 2023-08-27T11:18:08.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.125+0000 7f8df4e39700 1 -- 172.21.15.112:0/3215448021 learned_addr learned my addr 172.21.15.112:0/3215448021 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:08.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.126+0000 7f8df563a700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df0159600 0x7f8df0159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.126+0000 7f8deffff700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df006fd60 0x7f8df00701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.126+0000 7f8df4e39700 1 -- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df0159600 msgr2=0x7f8df0159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.126+0000 7f8df4e39700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df0159600 0x7f8df0159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.126+0000 7f8df4e39700 1 -- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df006fd60 msgr2=0x7f8df00701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.126+0000 7f8df4e39700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df006fd60 0x7f8df00701e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.126+0000 7f8df4e39700 1 -- 172.21.15.112:0/3215448021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df013cd30 con 0x7f8de0003a30 2023-08-27T11:18:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.126+0000 7f8df4e39700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de0003a30 0x7f8df013c7f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8de000bc00 tx=0x7f8de4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.126+0000 7f8ded7fa700 1 -- 172.21.15.112:0/3215448021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4007450 con 0x7f8de0003a30 2023-08-27T11:18:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.126+0000 7f8ded7fa700 1 -- 172.21.15.112:0/3215448021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de40075f0 con 0x7f8de0003a30 2023-08-27T11:18:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.126+0000 7f8ded7fa700 1 -- 172.21.15.112:0/3215448021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de40038c0 con 0x7f8de0003a30 2023-08-27T11:18:08.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.126+0000 7f8df709d700 1 -- 172.21.15.112:0/3215448021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df0071530 con 0x7f8de0003a30 2023-08-27T11:18:08.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.126+0000 7f8df709d700 1 -- 172.21.15.112:0/3215448021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df0070c00 con 0x7f8de0003a30 2023-08-27T11:18:08.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.127+0000 7f8df709d700 1 -- 172.21.15.112:0/3215448021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df0067a20 con 0x7f8de0003a30 2023-08-27T11:18:08.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.127+0000 7f8ded7fa700 1 -- 172.21.15.112:0/3215448021 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8de4003a20 con 0x7f8de0003a30 2023-08-27T11:18:08.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.128+0000 7f8ded7fa700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8dd406a0e0 0x7f8dd406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.128+0000 7f8ded7fa700 1 -- 172.21.15.112:0/3215448021 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8de4089090 con 0x7f8de0003a30 2023-08-27T11:18:08.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.128+0000 7f8deffff700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8dd406a0e0 0x7f8dd406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.128+0000 7f8deffff700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8dd406a0e0 0x7f8dd406c5a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8df0158ac0 tx=0x7f8ddc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:08.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.132+0000 7f8ded7fa700 1 -- 172.21.15.112:0/3215448021 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8de4054770 con 0x7f8de0003a30 2023-08-27T11:18:08.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.318+0000 7f8df709d700 1 -- 172.21.15.112:0/3215448021 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8df01407f0 con 0x7f8dd406a0e0 2023-08-27T11:18:08.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.328+0000 7f8ded7fa700 1 -- 172.21.15.112:0/3215448021 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311631 (secure 0 0 0) 0x7f8ddc006c00 con 0x7f8dd406a0e0 2023-08-27T11:18:08.333 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:08.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.331+0000 7f8df709d700 1 -- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8dd406a0e0 msgr2=0x7f8dd406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.331+0000 7f8df709d700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8dd406a0e0 0x7f8dd406c5a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8df0158ac0 tx=0x7f8ddc008040 comp rx=0 tx=0).stop 2023-08-27T11:18:08.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.331+0000 7f8df709d700 1 -- 172.21.15.112:0/3215448021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de0003a30 msgr2=0x7f8df013c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.331+0000 7f8df709d700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de0003a30 0x7f8df013c7f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8de000bc00 tx=0x7f8de4001120 comp rx=0 tx=0).stop 2023-08-27T11:18:08.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.331+0000 7f8df709d700 1 -- 172.21.15.112:0/3215448021 shutdown_connections 2023-08-27T11:18:08.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.331+0000 7f8df709d700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8dd406a0e0 0x7f8dd406c5a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.331+0000 7f8df709d700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df0159600 0x7f8df0159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.331+0000 7f8df709d700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de0003a30 0x7f8df013c7f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.331+0000 7f8df709d700 1 --2- 172.21.15.112:0/3215448021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df006fd60 0x7f8df00701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.331+0000 7f8df709d700 1 -- 172.21.15.112:0/3215448021 >> 172.21.15.112:0/3215448021 conn(0x7f8df006cd30 msgr2=0x7f8df0157e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.331+0000 7f8df709d700 1 -- 172.21.15.112:0/3215448021 shutdown_connections 2023-08-27T11:18:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.332+0000 7f8df709d700 1 -- 172.21.15.112:0/3215448021 wait complete. 2023-08-27T11:18:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:18:08.345 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2023-08-27T11:18:08.239379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220280,"kb_used_data":4096,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754754440,"statfs":{"total":773094113280,"available":772868546560,"internally_reserved":0,"allocated":4194304,"data_stored":2394948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001561"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796682+0000","last_change":"2023-08-27T11:18:04.257125+0000","last_active":"2023-08-27T11:18:06.796682+0000","last_peered":"2023-08-27T11:18:06.796682+0000","last_clean":"2023-08-27T11:18:06.796682+0000","last_became_active":"2023-08-27T11:18:04.256863+0000","last_became_peered":"2023-08-27T11:18:04.256863+0000","last_unstale":"2023-08-27T11:18:06.796682+0000","last_undegraded":"2023-08-27T11:18:06.796682+0000","last_fullsized":"2023-08-27T11:18:06.796682+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:50:15.147923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796139+0000","last_change":"2023-08-27T11:18:04.254854+0000","last_active":"2023-08-27T11:18:06.796139+0000","last_peered":"2023-08-27T11:18:06.796139+0000","last_clean":"2023-08-27T11:18:06.796139+0000","last_became_active":"2023-08-27T11:18:04.254554+0000","last_became_peered":"2023-08-27T11:18:04.254554+0000","last_unstale":"2023-08-27T11:18:06.796139+0000","last_undegraded":"2023-08-27T11:18:06.796139+0000","last_fullsized":"2023-08-27T11:18:06.796139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:38:49.728424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257571+0000","last_change":"2023-08-27T11:18:04.273065+0000","last_active":"2023-08-27T11:18:05.257571+0000","last_peered":"2023-08-27T11:18:05.257571+0000","last_clean":"2023-08-27T11:18:05.257571+0000","last_became_active":"2023-08-27T11:18:04.272786+0000","last_became_peered":"2023-08-27T11:18:04.272786+0000","last_unstale":"2023-08-27T11:18:05.257571+0000","last_undegraded":"2023-08-27T11:18:05.257571+0000","last_fullsized":"2023-08-27T11:18:05.257571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:18:51.736233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081577+0000","last_change":"2023-08-27T11:18:04.263371+0000","last_active":"2023-08-27T11:18:06.081577+0000","last_peered":"2023-08-27T11:18:06.081577+0000","last_clean":"2023-08-27T11:18:06.081577+0000","last_became_active":"2023-08-27T11:18:04.263114+0000","last_became_peered":"2023-08-27T11:18:04.263114+0000","last_unstale":"2023-08-27T11:18:06.081577+0000","last_undegraded":"2023-08-27T11:18:06.081577+0000","last_fullsized":"2023-08-27T11:18:06.081577+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:23:41.688216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081389+0000","last_change":"2023-08-27T11:18:04.262970+0000","last_active":"2023-08-27T11:18:06.081389+0000","last_peered":"2023-08-27T11:18:06.081389+0000","last_clean":"2023-08-27T11:18:06.081389+0000","last_became_active":"2023-08-27T11:18:04.262688+0000","last_became_peered":"2023-08-27T11:18:04.262688+0000","last_unstale":"2023-08-27T11:18:06.081389+0000","last_undegraded":"2023-08-27T11:18:06.081389+0000","last_fullsized":"2023-08-27T11:18:06.081389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:07.829879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796109+0000","last_change":"2023-08-27T11:18:04.260800+0000","last_active":"2023-08-27T11:18:06.796109+0000","last_peered":"2023-08-27T11:18:06.796109+0000","last_clean":"2023-08-27T11:18:06.796109+0000","last_became_active":"2023-08-27T11:18:04.260538+0000","last_became_peered":"2023-08-27T11:18:04.260538+0000","last_unstale":"2023-08-27T11:18:06.796109+0000","last_undegraded":"2023-08-27T11:18:06.796109+0000","last_fullsized":"2023-08-27T11:18:06.796109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:03.931204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084711+0000","last_change":"2023-08-27T11:18:04.268721+0000","last_active":"2023-08-27T11:18:06.084711+0000","last_peered":"2023-08-27T11:18:06.084711+0000","last_clean":"2023-08-27T11:18:06.084711+0000","last_became_active":"2023-08-27T11:18:04.268092+0000","last_became_peered":"2023-08-27T11:18:04.268092+0000","last_unstale":"2023-08-27T11:18:06.084711+0000","last_undegraded":"2023-08-27T11:18:06.084711+0000","last_fullsized":"2023-08-27T11:18:06.084711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:58.144442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.795936+0000","last_change":"2023-08-27T11:18:04.259189+0000","last_active":"2023-08-27T11:18:06.795936+0000","last_peered":"2023-08-27T11:18:06.795936+0000","last_clean":"2023-08-27T11:18:06.795936+0000","last_became_active":"2023-08-27T11:18:04.258850+0000","last_became_peered":"2023-08-27T11:18:04.258850+0000","last_unstale":"2023-08-27T11:18:06.795936+0000","last_undegraded":"2023-08-27T11:18:06.795936+0000","last_fullsized":"2023-08-27T11:18:06.795936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:02:56.610036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257680+0000","last_change":"2023-08-27T11:18:04.270302+0000","last_active":"2023-08-27T11:18:05.257680+0000","last_peered":"2023-08-27T11:18:05.257680+0000","last_clean":"2023-08-27T11:18:05.257680+0000","last_became_active":"2023-08-27T11:18:04.270032+0000","last_became_peered":"2023-08-27T11:18:04.270032+0000","last_unstale":"2023-08-27T11:18:05.257680+0000","last_undegraded":"2023-08-27T11:18:05.257680+0000","last_fullsized":"2023-08-27T11:18:05.257680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:29.089882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084727+0000","last_change":"2023-08-27T11:18:04.267180+0000","last_active":"2023-08-27T11:18:06.084727+0000","last_peered":"2023-08-27T11:18:06.084727+0000","last_clean":"2023-08-27T11:18:06.084727+0000","last_became_active":"2023-08-27T11:18:04.266796+0000","last_became_peered":"2023-08-27T11:18:04.266796+0000","last_unstale":"2023-08-27T11:18:06.084727+0000","last_undegraded":"2023-08-27T11:18:06.084727+0000","last_fullsized":"2023-08-27T11:18:06.084727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:20:45.867963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084939+0000","last_change":"2023-08-27T11:18:04.264324+0000","last_active":"2023-08-27T11:18:06.084939+0000","last_peered":"2023-08-27T11:18:06.084939+0000","last_clean":"2023-08-27T11:18:06.084939+0000","last_became_active":"2023-08-27T11:18:04.264025+0000","last_became_peered":"2023-08-27T11:18:04.264025+0000","last_unstale":"2023-08-27T11:18:06.084939+0000","last_undegraded":"2023-08-27T11:18:06.084939+0000","last_fullsized":"2023-08-27T11:18:06.084939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:42:46.084990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257721+0000","last_change":"2023-08-27T11:18:04.256478+0000","last_active":"2023-08-27T11:18:05.257721+0000","last_peered":"2023-08-27T11:18:05.257721+0000","last_clean":"2023-08-27T11:18:05.257721+0000","last_became_active":"2023-08-27T11:18:04.256232+0000","last_became_peered":"2023-08-27T11:18:04.256232+0000","last_unstale":"2023-08-27T11:18:05.257721+0000","last_undegraded":"2023-08-27T11:18:05.257721+0000","last_fullsized":"2023-08-27T11:18:05.257721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:11:07.765705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084498+0000","last_change":"2023-08-27T11:18:04.268507+0000","last_active":"2023-08-27T11:18:06.084498+0000","last_peered":"2023-08-27T11:18:06.084498+0000","last_clean":"2023-08-27T11:18:06.084498+0000","last_became_active":"2023-08-27T11:18:04.268056+0000","last_became_peered":"2023-08-27T11:18:04.268056+0000","last_unstale":"2023-08-27T11:18:06.084498+0000","last_undegraded":"2023-08-27T11:18:06.084498+0000","last_fullsized":"2023-08-27T11:18:06.084498+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:11:07.765705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256982+0000","last_change":"2023-08-27T11:18:04.260786+0000","last_active":"2023-08-27T11:18:05.256982+0000","last_peered":"2023-08-27T11:18:05.256982+0000","last_clean":"2023-08-27T11:18:05.256982+0000","last_became_active":"2023-08-27T11:18:04.260562+0000","last_became_peered":"2023-08-27T11:18:04.260562+0000","last_unstale":"2023-08-27T11:18:05.256982+0000","last_undegraded":"2023-08-27T11:18:05.256982+0000","last_fullsized":"2023-08-27T11:18:05.256982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:44.523518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084280+0000","last_change":"2023-08-27T11:18:04.269789+0000","last_active":"2023-08-27T11:18:06.084280+0000","last_peered":"2023-08-27T11:18:06.084280+0000","last_clean":"2023-08-27T11:18:06.084280+0000","last_became_active":"2023-08-27T11:18:04.269552+0000","last_became_peered":"2023-08-27T11:18:04.269552+0000","last_unstale":"2023-08-27T11:18:06.084280+0000","last_undegraded":"2023-08-27T11:18:06.084280+0000","last_fullsized":"2023-08-27T11:18:06.084280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:12.709794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796696+0000","last_change":"2023-08-27T11:18:04.271085+0000","last_active":"2023-08-27T11:18:06.796696+0000","last_peered":"2023-08-27T11:18:06.796696+0000","last_clean":"2023-08-27T11:18:06.796696+0000","last_became_active":"2023-08-27T11:18:04.270178+0000","last_became_peered":"2023-08-27T11:18:04.270178+0000","last_unstale":"2023-08-27T11:18:06.796696+0000","last_undegraded":"2023-08-27T11:18:06.796696+0000","last_fullsized":"2023-08-27T11:18:06.796696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:02:56.610036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797318+0000","last_change":"2023-08-27T11:18:04.273533+0000","last_active":"2023-08-27T11:18:06.797318+0000","last_peered":"2023-08-27T11:18:06.797318+0000","last_clean":"2023-08-27T11:18:06.797318+0000","last_became_active":"2023-08-27T11:18:04.273222+0000","last_became_peered":"2023-08-27T11:18:04.273222+0000","last_unstale":"2023-08-27T11:18:06.797318+0000","last_undegraded":"2023-08-27T11:18:06.797318+0000","last_fullsized":"2023-08-27T11:18:06.797318+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:50:15.147923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256986+0000","last_change":"2023-08-27T11:18:04.261729+0000","last_active":"2023-08-27T11:18:05.256986+0000","last_peered":"2023-08-27T11:18:05.256986+0000","last_clean":"2023-08-27T11:18:05.256986+0000","last_became_active":"2023-08-27T11:18:04.261316+0000","last_became_peered":"2023-08-27T11:18:04.261316+0000","last_unstale":"2023-08-27T11:18:05.256986+0000","last_undegraded":"2023-08-27T11:18:05.256986+0000","last_fullsized":"2023-08-27T11:18:05.256986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:03.931204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081526+0000","last_change":"2023-08-27T11:18:04.260253+0000","last_active":"2023-08-27T11:18:06.081526+0000","last_peered":"2023-08-27T11:18:06.081526+0000","last_clean":"2023-08-27T11:18:06.081526+0000","last_became_active":"2023-08-27T11:18:04.259930+0000","last_became_peered":"2023-08-27T11:18:04.259930+0000","last_unstale":"2023-08-27T11:18:06.081526+0000","last_undegraded":"2023-08-27T11:18:06.081526+0000","last_fullsized":"2023-08-27T11:18:06.081526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:11:37.956860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257412+0000","last_change":"2023-08-27T11:18:04.261550+0000","last_active":"2023-08-27T11:18:05.257412+0000","last_peered":"2023-08-27T11:18:05.257412+0000","last_clean":"2023-08-27T11:18:05.257412+0000","last_became_active":"2023-08-27T11:18:04.261249+0000","last_became_peered":"2023-08-27T11:18:04.261249+0000","last_unstale":"2023-08-27T11:18:05.257412+0000","last_undegraded":"2023-08-27T11:18:05.257412+0000","last_fullsized":"2023-08-27T11:18:05.257412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:04:35.973096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084693+0000","last_change":"2023-08-27T11:18:04.268997+0000","last_active":"2023-08-27T11:18:06.084693+0000","last_peered":"2023-08-27T11:18:06.084693+0000","last_clean":"2023-08-27T11:18:06.084693+0000","last_became_active":"2023-08-27T11:18:04.268733+0000","last_became_peered":"2023-08-27T11:18:04.268733+0000","last_unstale":"2023-08-27T11:18:06.084693+0000","last_undegraded":"2023-08-27T11:18:06.084693+0000","last_fullsized":"2023-08-27T11:18:06.084693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:19.211820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256429+0000","last_change":"2023-08-27T11:18:04.257131+0000","last_active":"2023-08-27T11:18:05.256429+0000","last_peered":"2023-08-27T11:18:05.256429+0000","last_clean":"2023-08-27T11:18:05.256429+0000","last_became_active":"2023-08-27T11:18:04.256859+0000","last_became_peered":"2023-08-27T11:18:04.256859+0000","last_unstale":"2023-08-27T11:18:05.256429+0000","last_undegraded":"2023-08-27T11:18:05.256429+0000","last_fullsized":"2023-08-27T11:18:05.256429+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:31:19.039678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796832+0000","last_change":"2023-08-27T11:18:04.266733+0000","last_active":"2023-08-27T11:18:06.796832+0000","last_peered":"2023-08-27T11:18:06.796832+0000","last_clean":"2023-08-27T11:18:06.796832+0000","last_became_active":"2023-08-27T11:18:04.266350+0000","last_became_peered":"2023-08-27T11:18:04.266350+0000","last_unstale":"2023-08-27T11:18:06.796832+0000","last_undegraded":"2023-08-27T11:18:06.796832+0000","last_fullsized":"2023-08-27T11:18:06.796832+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:38:49.728424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081308+0000","last_change":"2023-08-27T11:18:04.269130+0000","last_active":"2023-08-27T11:18:06.081308+0000","last_peered":"2023-08-27T11:18:06.081308+0000","last_clean":"2023-08-27T11:18:06.081308+0000","last_became_active":"2023-08-27T11:18:04.268587+0000","last_became_peered":"2023-08-27T11:18:04.268587+0000","last_unstale":"2023-08-27T11:18:06.081308+0000","last_undegraded":"2023-08-27T11:18:06.081308+0000","last_fullsized":"2023-08-27T11:18:06.081308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:16:52.344515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257116+0000","last_change":"2023-08-27T11:18:04.257945+0000","last_active":"2023-08-27T11:18:05.257116+0000","last_peered":"2023-08-27T11:18:05.257116+0000","last_clean":"2023-08-27T11:18:05.257116+0000","last_became_active":"2023-08-27T11:18:04.257616+0000","last_became_peered":"2023-08-27T11:18:04.257616+0000","last_unstale":"2023-08-27T11:18:05.257116+0000","last_undegraded":"2023-08-27T11:18:05.257116+0000","last_fullsized":"2023-08-27T11:18:05.257116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:04:48.125632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796683+0000","last_change":"2023-08-27T11:18:04.273603+0000","last_active":"2023-08-27T11:18:06.796683+0000","last_peered":"2023-08-27T11:18:06.796683+0000","last_clean":"2023-08-27T11:18:06.796683+0000","last_became_active":"2023-08-27T11:18:04.273326+0000","last_became_peered":"2023-08-27T11:18:04.273326+0000","last_unstale":"2023-08-27T11:18:06.796683+0000","last_undegraded":"2023-08-27T11:18:06.796683+0000","last_fullsized":"2023-08-27T11:18:06.796683+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:12.709794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257400+0000","last_change":"2023-08-27T11:18:04.258196+0000","last_active":"2023-08-27T11:18:05.257400+0000","last_peered":"2023-08-27T11:18:05.257400+0000","last_clean":"2023-08-27T11:18:05.257400+0000","last_became_active":"2023-08-27T11:18:04.257902+0000","last_became_peered":"2023-08-27T11:18:04.257902+0000","last_unstale":"2023-08-27T11:18:05.257400+0000","last_undegraded":"2023-08-27T11:18:05.257400+0000","last_fullsized":"2023-08-27T11:18:05.257400+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:01:04.142517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084607+0000","last_change":"2023-08-27T11:18:04.268371+0000","last_active":"2023-08-27T11:18:06.084607+0000","last_peered":"2023-08-27T11:18:06.084607+0000","last_clean":"2023-08-27T11:18:06.084607+0000","last_became_active":"2023-08-27T11:18:04.268082+0000","last_became_peered":"2023-08-27T11:18:04.268082+0000","last_unstale":"2023-08-27T11:18:06.084607+0000","last_undegraded":"2023-08-27T11:18:06.084607+0000","last_fullsized":"2023-08-27T11:18:06.084607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:16:32.261875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256778+0000","last_change":"2023-08-27T11:18:04.259863+0000","last_active":"2023-08-27T11:18:05.256778+0000","last_peered":"2023-08-27T11:18:05.256778+0000","last_clean":"2023-08-27T11:18:05.256778+0000","last_became_active":"2023-08-27T11:18:04.259019+0000","last_became_peered":"2023-08-27T11:18:04.259019+0000","last_unstale":"2023-08-27T11:18:05.256778+0000","last_undegraded":"2023-08-27T11:18:05.256778+0000","last_fullsized":"2023-08-27T11:18:05.256778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:20:58.467861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256578+0000","last_change":"2023-08-27T11:18:04.257359+0000","last_active":"2023-08-27T11:18:05.256578+0000","last_peered":"2023-08-27T11:18:05.256578+0000","last_clean":"2023-08-27T11:18:05.256578+0000","last_became_active":"2023-08-27T11:18:04.256844+0000","last_became_peered":"2023-08-27T11:18:04.256844+0000","last_unstale":"2023-08-27T11:18:05.256578+0000","last_undegraded":"2023-08-27T11:18:05.256578+0000","last_fullsized":"2023-08-27T11:18:05.256578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:44:59.715657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796771+0000","last_change":"2023-08-27T11:18:04.266573+0000","last_active":"2023-08-27T11:18:06.796771+0000","last_peered":"2023-08-27T11:18:06.796771+0000","last_clean":"2023-08-27T11:18:06.796771+0000","last_became_active":"2023-08-27T11:18:04.266052+0000","last_became_peered":"2023-08-27T11:18:04.266052+0000","last_unstale":"2023-08-27T11:18:06.796771+0000","last_undegraded":"2023-08-27T11:18:06.796771+0000","last_fullsized":"2023-08-27T11:18:06.796771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:05.181763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796004+0000","last_change":"2023-08-27T11:18:04.258125+0000","last_active":"2023-08-27T11:18:06.796004+0000","last_peered":"2023-08-27T11:18:06.796004+0000","last_clean":"2023-08-27T11:18:06.796004+0000","last_became_active":"2023-08-27T11:18:04.257848+0000","last_became_peered":"2023-08-27T11:18:04.257848+0000","last_unstale":"2023-08-27T11:18:06.796004+0000","last_undegraded":"2023-08-27T11:18:06.796004+0000","last_fullsized":"2023-08-27T11:18:06.796004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:20:58.467861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797110+0000","last_change":"2023-08-27T11:18:04.272780+0000","last_active":"2023-08-27T11:18:06.797110+0000","last_peered":"2023-08-27T11:18:06.797110+0000","last_clean":"2023-08-27T11:18:06.797110+0000","last_became_active":"2023-08-27T11:18:04.272509+0000","last_became_peered":"2023-08-27T11:18:04.272509+0000","last_unstale":"2023-08-27T11:18:06.797110+0000","last_undegraded":"2023-08-27T11:18:06.797110+0000","last_fullsized":"2023-08-27T11:18:06.797110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:28:29.852805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"32'4","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257215+0000","last_change":"2023-08-27T11:18:03.254346+0000","last_active":"2023-08-27T11:18:05.257215+0000","last_peered":"2023-08-27T11:18:05.257215+0000","last_clean":"2023-08-27T11:18:05.257215+0000","last_became_active":"2023-08-27T11:18:03.254053+0000","last_became_peered":"2023-08-27T11:18:03.254053+0000","last_unstale":"2023-08-27T11:18:05.257215+0000","last_undegraded":"2023-08-27T11:18:05.257215+0000","last_fullsized":"2023-08-27T11:18:05.257215+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:07:07.127876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796584+0000","last_change":"2023-08-27T11:18:04.273641+0000","last_active":"2023-08-27T11:18:06.796584+0000","last_peered":"2023-08-27T11:18:06.796584+0000","last_clean":"2023-08-27T11:18:06.796584+0000","last_became_active":"2023-08-27T11:18:04.273374+0000","last_became_peered":"2023-08-27T11:18:04.273374+0000","last_unstale":"2023-08-27T11:18:06.796584+0000","last_undegraded":"2023-08-27T11:18:06.796584+0000","last_fullsized":"2023-08-27T11:18:06.796584+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:39:40.502028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257640+0000","last_change":"2023-08-27T11:18:03.253693+0000","last_active":"2023-08-27T11:18:05.257640+0000","last_peered":"2023-08-27T11:18:05.257640+0000","last_clean":"2023-08-27T11:18:05.257640+0000","last_became_active":"2023-08-27T11:18:03.253433+0000","last_became_peered":"2023-08-27T11:18:03.253433+0000","last_unstale":"2023-08-27T11:18:05.257640+0000","last_undegraded":"2023-08-27T11:18:05.257640+0000","last_fullsized":"2023-08-27T11:18:05.257640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:24.361949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257821+0000","last_change":"2023-08-27T11:18:04.260748+0000","last_active":"2023-08-27T11:18:05.257821+0000","last_peered":"2023-08-27T11:18:05.257821+0000","last_clean":"2023-08-27T11:18:05.257821+0000","last_became_active":"2023-08-27T11:18:04.260442+0000","last_became_peered":"2023-08-27T11:18:04.260442+0000","last_unstale":"2023-08-27T11:18:05.257821+0000","last_undegraded":"2023-08-27T11:18:05.257821+0000","last_fullsized":"2023-08-27T11:18:05.257821+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:43:37.898998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796557+0000","last_change":"2023-08-27T11:18:03.252549+0000","last_active":"2023-08-27T11:18:06.796557+0000","last_peered":"2023-08-27T11:18:06.796557+0000","last_clean":"2023-08-27T11:18:06.796557+0000","last_became_active":"2023-08-27T11:18:03.252302+0000","last_became_peered":"2023-08-27T11:18:03.252302+0000","last_unstale":"2023-08-27T11:18:06.796557+0000","last_undegraded":"2023-08-27T11:18:06.796557+0000","last_fullsized":"2023-08-27T11:18:06.796557+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:20:38.147319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796452+0000","last_change":"2023-08-27T11:18:04.259866+0000","last_active":"2023-08-27T11:18:06.796452+0000","last_peered":"2023-08-27T11:18:06.796452+0000","last_clean":"2023-08-27T11:18:06.796452+0000","last_became_active":"2023-08-27T11:18:04.258750+0000","last_became_peered":"2023-08-27T11:18:04.258750+0000","last_unstale":"2023-08-27T11:18:06.796452+0000","last_undegraded":"2023-08-27T11:18:06.796452+0000","last_fullsized":"2023-08-27T11:18:06.796452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:28:29.852805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257520+0000","last_change":"2023-08-27T11:18:03.258266+0000","last_active":"2023-08-27T11:18:05.257520+0000","last_peered":"2023-08-27T11:18:05.257520+0000","last_clean":"2023-08-27T11:18:05.257520+0000","last_became_active":"2023-08-27T11:18:03.257980+0000","last_became_peered":"2023-08-27T11:18:03.257980+0000","last_unstale":"2023-08-27T11:18:05.257520+0000","last_undegraded":"2023-08-27T11:18:05.257520+0000","last_fullsized":"2023-08-27T11:18:05.257520+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:14:28.905888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257301+0000","last_change":"2023-08-27T11:18:04.258361+0000","last_active":"2023-08-27T11:18:05.257301+0000","last_peered":"2023-08-27T11:18:05.257301+0000","last_clean":"2023-08-27T11:18:05.257301+0000","last_became_active":"2023-08-27T11:18:04.258084+0000","last_became_peered":"2023-08-27T11:18:04.258084+0000","last_unstale":"2023-08-27T11:18:05.257301+0000","last_undegraded":"2023-08-27T11:18:05.257301+0000","last_fullsized":"2023-08-27T11:18:05.257301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:02:14.628161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084433+0000","last_change":"2023-08-27T11:18:03.261880+0000","last_active":"2023-08-27T11:18:06.084433+0000","last_peered":"2023-08-27T11:18:06.084433+0000","last_clean":"2023-08-27T11:18:06.084433+0000","last_became_active":"2023-08-27T11:18:03.261672+0000","last_became_peered":"2023-08-27T11:18:03.261672+0000","last_unstale":"2023-08-27T11:18:06.084433+0000","last_undegraded":"2023-08-27T11:18:06.084433+0000","last_fullsized":"2023-08-27T11:18:06.084433+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:24.361949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084326+0000","last_change":"2023-08-27T11:18:04.268908+0000","last_active":"2023-08-27T11:18:06.084326+0000","last_peered":"2023-08-27T11:18:06.084326+0000","last_clean":"2023-08-27T11:18:06.084326+0000","last_became_active":"2023-08-27T11:18:04.268433+0000","last_became_peered":"2023-08-27T11:18:04.268433+0000","last_unstale":"2023-08-27T11:18:06.084326+0000","last_undegraded":"2023-08-27T11:18:06.084326+0000","last_fullsized":"2023-08-27T11:18:06.084326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:18:51.736233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797122+0000","last_change":"2023-08-27T11:18:03.260690+0000","last_active":"2023-08-27T11:18:06.797122+0000","last_peered":"2023-08-27T11:18:06.797122+0000","last_clean":"2023-08-27T11:18:06.797122+0000","last_became_active":"2023-08-27T11:18:03.260428+0000","last_became_peered":"2023-08-27T11:18:03.260428+0000","last_unstale":"2023-08-27T11:18:06.797122+0000","last_undegraded":"2023-08-27T11:18:06.797122+0000","last_fullsized":"2023-08-27T11:18:06.797122+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:34:17.508519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256580+0000","last_change":"2023-08-27T11:18:04.268936+0000","last_active":"2023-08-27T11:18:05.256580+0000","last_peered":"2023-08-27T11:18:05.256580+0000","last_clean":"2023-08-27T11:18:05.256580+0000","last_became_active":"2023-08-27T11:18:04.268677+0000","last_became_peered":"2023-08-27T11:18:04.268677+0000","last_unstale":"2023-08-27T11:18:05.256580+0000","last_undegraded":"2023-08-27T11:18:05.256580+0000","last_fullsized":"2023-08-27T11:18:05.256580+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:34:01.673085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796274+0000","last_change":"2023-08-27T11:18:03.261094+0000","last_active":"2023-08-27T11:18:06.796274+0000","last_peered":"2023-08-27T11:18:06.796274+0000","last_clean":"2023-08-27T11:18:06.796274+0000","last_became_active":"2023-08-27T11:18:03.260838+0000","last_became_peered":"2023-08-27T11:18:03.260838+0000","last_unstale":"2023-08-27T11:18:06.796274+0000","last_undegraded":"2023-08-27T11:18:06.796274+0000","last_fullsized":"2023-08-27T11:18:06.796274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:01:03.137176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796461+0000","last_change":"2023-08-27T11:18:04.263784+0000","last_active":"2023-08-27T11:18:06.796461+0000","last_peered":"2023-08-27T11:18:06.796461+0000","last_clean":"2023-08-27T11:18:06.796461+0000","last_became_active":"2023-08-27T11:18:04.263342+0000","last_became_peered":"2023-08-27T11:18:04.263342+0000","last_unstale":"2023-08-27T11:18:06.796461+0000","last_undegraded":"2023-08-27T11:18:06.796461+0000","last_fullsized":"2023-08-27T11:18:06.796461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:39.229819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256514+0000","last_change":"2023-08-27T11:18:03.257108+0000","last_active":"2023-08-27T11:18:05.256514+0000","last_peered":"2023-08-27T11:18:05.256514+0000","last_clean":"2023-08-27T11:18:05.256514+0000","last_became_active":"2023-08-27T11:18:03.256821+0000","last_became_peered":"2023-08-27T11:18:03.256821+0000","last_unstale":"2023-08-27T11:18:05.256514+0000","last_undegraded":"2023-08-27T11:18:05.256514+0000","last_fullsized":"2023-08-27T11:18:05.256514+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:56.352101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081616+0000","last_change":"2023-08-27T11:18:04.262997+0000","last_active":"2023-08-27T11:18:06.081616+0000","last_peered":"2023-08-27T11:18:06.081616+0000","last_clean":"2023-08-27T11:18:06.081616+0000","last_became_active":"2023-08-27T11:18:04.262756+0000","last_became_peered":"2023-08-27T11:18:04.262756+0000","last_unstale":"2023-08-27T11:18:06.081616+0000","last_undegraded":"2023-08-27T11:18:06.081616+0000","last_fullsized":"2023-08-27T11:18:06.081616+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:16:13.542596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796531+0000","last_change":"2023-08-27T11:18:03.256396+0000","last_active":"2023-08-27T11:18:06.796531+0000","last_peered":"2023-08-27T11:18:06.796531+0000","last_clean":"2023-08-27T11:18:06.796531+0000","last_became_active":"2023-08-27T11:18:03.256122+0000","last_became_peered":"2023-08-27T11:18:03.256122+0000","last_unstale":"2023-08-27T11:18:06.796531+0000","last_undegraded":"2023-08-27T11:18:06.796531+0000","last_fullsized":"2023-08-27T11:18:06.796531+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:00:35.000631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797212+0000","last_change":"2023-08-27T11:18:04.265022+0000","last_active":"2023-08-27T11:18:06.797212+0000","last_peered":"2023-08-27T11:18:06.797212+0000","last_clean":"2023-08-27T11:18:06.797212+0000","last_became_active":"2023-08-27T11:18:04.264804+0000","last_became_peered":"2023-08-27T11:18:04.264804+0000","last_unstale":"2023-08-27T11:18:06.797212+0000","last_undegraded":"2023-08-27T11:18:06.797212+0000","last_fullsized":"2023-08-27T11:18:06.797212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:39.229819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084861+0000","last_change":"2023-08-27T11:18:03.260612+0000","last_active":"2023-08-27T11:18:06.084861+0000","last_peered":"2023-08-27T11:18:06.084861+0000","last_clean":"2023-08-27T11:18:06.084861+0000","last_became_active":"2023-08-27T11:18:03.260171+0000","last_became_peered":"2023-08-27T11:18:03.260171+0000","last_unstale":"2023-08-27T11:18:06.084861+0000","last_undegraded":"2023-08-27T11:18:06.084861+0000","last_fullsized":"2023-08-27T11:18:06.084861+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:38.565900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257551+0000","last_change":"2023-08-27T11:18:04.264989+0000","last_active":"2023-08-27T11:18:05.257551+0000","last_peered":"2023-08-27T11:18:05.257551+0000","last_clean":"2023-08-27T11:18:05.257551+0000","last_became_active":"2023-08-27T11:18:04.263821+0000","last_became_peered":"2023-08-27T11:18:04.263821+0000","last_unstale":"2023-08-27T11:18:05.257551+0000","last_undegraded":"2023-08-27T11:18:05.257551+0000","last_fullsized":"2023-08-27T11:18:05.257551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:39:40.502028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"32'3","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084883+0000","last_change":"2023-08-27T11:18:03.259559+0000","last_active":"2023-08-27T11:18:06.084883+0000","last_peered":"2023-08-27T11:18:06.084883+0000","last_clean":"2023-08-27T11:18:06.084883+0000","last_became_active":"2023-08-27T11:18:03.256952+0000","last_became_peered":"2023-08-27T11:18:03.256952+0000","last_unstale":"2023-08-27T11:18:06.084883+0000","last_undegraded":"2023-08-27T11:18:06.084883+0000","last_fullsized":"2023-08-27T11:18:06.084883+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:08:58.446358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257294+0000","last_change":"2023-08-27T11:18:04.256712+0000","last_active":"2023-08-27T11:18:05.257294+0000","last_peered":"2023-08-27T11:18:05.257294+0000","last_clean":"2023-08-27T11:18:05.257294+0000","last_became_active":"2023-08-27T11:18:04.256427+0000","last_became_peered":"2023-08-27T11:18:04.256427+0000","last_unstale":"2023-08-27T11:18:05.257294+0000","last_undegraded":"2023-08-27T11:18:05.257294+0000","last_fullsized":"2023-08-27T11:18:05.257294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:42:59.274843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"32'6","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796298+0000","last_change":"2023-08-27T11:18:03.254733+0000","last_active":"2023-08-27T11:18:06.796298+0000","last_peered":"2023-08-27T11:18:06.796298+0000","last_clean":"2023-08-27T11:18:06.796298+0000","last_became_active":"2023-08-27T11:18:03.254497+0000","last_became_peered":"2023-08-27T11:18:03.254497+0000","last_unstale":"2023-08-27T11:18:06.796298+0000","last_undegraded":"2023-08-27T11:18:06.796298+0000","last_fullsized":"2023-08-27T11:18:06.796298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:41:43.956388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257446+0000","last_change":"2023-08-27T11:18:04.268368+0000","last_active":"2023-08-27T11:18:05.257446+0000","last_peered":"2023-08-27T11:18:05.257446+0000","last_clean":"2023-08-27T11:18:05.257446+0000","last_became_active":"2023-08-27T11:18:04.268082+0000","last_became_peered":"2023-08-27T11:18:04.268082+0000","last_unstale":"2023-08-27T11:18:05.257446+0000","last_undegraded":"2023-08-27T11:18:05.257446+0000","last_fullsized":"2023-08-27T11:18:05.257446+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:20:39.152661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796478+0000","last_change":"2023-08-27T11:18:03.255944+0000","last_active":"2023-08-27T11:18:06.796478+0000","last_peered":"2023-08-27T11:18:06.796478+0000","last_clean":"2023-08-27T11:18:06.796478+0000","last_became_active":"2023-08-27T11:18:03.255681+0000","last_became_peered":"2023-08-27T11:18:03.255681+0000","last_unstale":"2023-08-27T11:18:06.796478+0000","last_undegraded":"2023-08-27T11:18:06.796478+0000","last_fullsized":"2023-08-27T11:18:06.796478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:12:40.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796097+0000","last_change":"2023-08-27T11:18:04.256175+0000","last_active":"2023-08-27T11:18:06.796097+0000","last_peered":"2023-08-27T11:18:06.796097+0000","last_clean":"2023-08-27T11:18:06.796097+0000","last_became_active":"2023-08-27T11:18:04.254794+0000","last_became_peered":"2023-08-27T11:18:04.254794+0000","last_unstale":"2023-08-27T11:18:06.796097+0000","last_undegraded":"2023-08-27T11:18:06.796097+0000","last_fullsized":"2023-08-27T11:18:06.796097+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:00:23.976483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796192+0000","last_change":"2023-08-27T11:18:03.262020+0000","last_active":"2023-08-27T11:18:06.796192+0000","last_peered":"2023-08-27T11:18:06.796192+0000","last_clean":"2023-08-27T11:18:06.796192+0000","last_became_active":"2023-08-27T11:18:03.261770+0000","last_became_peered":"2023-08-27T11:18:03.261770+0000","last_unstale":"2023-08-27T11:18:06.796192+0000","last_undegraded":"2023-08-27T11:18:06.796192+0000","last_fullsized":"2023-08-27T11:18:06.796192+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:04:47.120291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797000+0000","last_change":"2023-08-27T11:18:04.263516+0000","last_active":"2023-08-27T11:18:06.797000+0000","last_peered":"2023-08-27T11:18:06.797000+0000","last_clean":"2023-08-27T11:18:06.797000+0000","last_became_active":"2023-08-27T11:18:04.263215+0000","last_became_peered":"2023-08-27T11:18:04.263215+0000","last_unstale":"2023-08-27T11:18:06.797000+0000","last_undegraded":"2023-08-27T11:18:06.797000+0000","last_fullsized":"2023-08-27T11:18:06.797000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:34.331158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256567+0000","last_change":"2023-08-27T11:18:03.258848+0000","last_active":"2023-08-27T11:18:05.256567+0000","last_peered":"2023-08-27T11:18:05.256567+0000","last_clean":"2023-08-27T11:18:05.256567+0000","last_became_active":"2023-08-27T11:18:03.256331+0000","last_became_peered":"2023-08-27T11:18:03.256331+0000","last_unstale":"2023-08-27T11:18:05.256567+0000","last_undegraded":"2023-08-27T11:18:05.256567+0000","last_fullsized":"2023-08-27T11:18:05.256567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:41.301776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796249+0000","last_change":"2023-08-27T11:18:04.255391+0000","last_active":"2023-08-27T11:18:06.796249+0000","last_peered":"2023-08-27T11:18:06.796249+0000","last_clean":"2023-08-27T11:18:06.796249+0000","last_became_active":"2023-08-27T11:18:04.255076+0000","last_became_peered":"2023-08-27T11:18:04.255076+0000","last_unstale":"2023-08-27T11:18:06.796249+0000","last_undegraded":"2023-08-27T11:18:06.796249+0000","last_fullsized":"2023-08-27T11:18:06.796249+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:38:03.049024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"32'8","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796870+0000","last_change":"2023-08-27T11:18:03.260359+0000","last_active":"2023-08-27T11:18:06.796870+0000","last_peered":"2023-08-27T11:18:06.796870+0000","last_clean":"2023-08-27T11:18:06.796870+0000","last_became_active":"2023-08-27T11:18:03.260073+0000","last_became_peered":"2023-08-27T11:18:03.260073+0000","last_unstale":"2023-08-27T11:18:06.796870+0000","last_undegraded":"2023-08-27T11:18:06.796870+0000","last_fullsized":"2023-08-27T11:18:06.796870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:00.657502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796212+0000","last_change":"2023-08-27T11:18:04.258973+0000","last_active":"2023-08-27T11:18:06.796212+0000","last_peered":"2023-08-27T11:18:06.796212+0000","last_clean":"2023-08-27T11:18:06.796212+0000","last_became_active":"2023-08-27T11:18:04.258629+0000","last_became_peered":"2023-08-27T11:18:04.258629+0000","last_unstale":"2023-08-27T11:18:06.796212+0000","last_undegraded":"2023-08-27T11:18:06.796212+0000","last_fullsized":"2023-08-27T11:18:06.796212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:01.662843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084971+0000","last_change":"2023-08-27T11:18:03.259635+0000","last_active":"2023-08-27T11:18:06.084971+0000","last_peered":"2023-08-27T11:18:06.084971+0000","last_clean":"2023-08-27T11:18:06.084971+0000","last_became_active":"2023-08-27T11:18:03.259370+0000","last_became_peered":"2023-08-27T11:18:03.259370+0000","last_unstale":"2023-08-27T11:18:06.084971+0000","last_undegraded":"2023-08-27T11:18:06.084971+0000","last_fullsized":"2023-08-27T11:18:06.084971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:02:55.604695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084750+0000","last_change":"2023-08-27T11:18:04.272101+0000","last_active":"2023-08-27T11:18:06.084750+0000","last_peered":"2023-08-27T11:18:06.084750+0000","last_clean":"2023-08-27T11:18:06.084750+0000","last_became_active":"2023-08-27T11:18:04.271813+0000","last_became_peered":"2023-08-27T11:18:04.271813+0000","last_unstale":"2023-08-27T11:18:06.084750+0000","last_undegraded":"2023-08-27T11:18:06.084750+0000","last_fullsized":"2023-08-27T11:18:06.084750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:41:02.824652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.258023+0000","last_change":"2023-08-27T11:18:03.260695+0000","last_active":"2023-08-27T11:18:05.258023+0000","last_peered":"2023-08-27T11:18:05.258023+0000","last_clean":"2023-08-27T11:18:05.258023+0000","last_became_active":"2023-08-27T11:18:03.260434+0000","last_became_peered":"2023-08-27T11:18:03.260434+0000","last_unstale":"2023-08-27T11:18:05.258023+0000","last_undegraded":"2023-08-27T11:18:05.258023+0000","last_fullsized":"2023-08-27T11:18:05.258023+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:49.647795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257947+0000","last_change":"2023-08-27T11:18:04.260156+0000","last_active":"2023-08-27T11:18:05.257947+0000","last_peered":"2023-08-27T11:18:05.257947+0000","last_clean":"2023-08-27T11:18:05.257947+0000","last_became_active":"2023-08-27T11:18:04.259770+0000","last_became_peered":"2023-08-27T11:18:04.259770+0000","last_unstale":"2023-08-27T11:18:05.257947+0000","last_undegraded":"2023-08-27T11:18:05.257947+0000","last_fullsized":"2023-08-27T11:18:05.257947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:01.222049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081565+0000","last_change":"2023-08-27T11:18:03.254682+0000","last_active":"2023-08-27T11:18:06.081565+0000","last_peered":"2023-08-27T11:18:06.081565+0000","last_clean":"2023-08-27T11:18:06.081565+0000","last_became_active":"2023-08-27T11:18:03.254368+0000","last_became_peered":"2023-08-27T11:18:03.254368+0000","last_unstale":"2023-08-27T11:18:06.081565+0000","last_undegraded":"2023-08-27T11:18:06.081565+0000","last_fullsized":"2023-08-27T11:18:06.081565+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:38:27.916096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081671+0000","last_change":"2023-08-27T11:18:04.254883+0000","last_active":"2023-08-27T11:18:06.081671+0000","last_peered":"2023-08-27T11:18:06.081671+0000","last_clean":"2023-08-27T11:18:06.081671+0000","last_became_active":"2023-08-27T11:18:04.254616+0000","last_became_peered":"2023-08-27T11:18:04.254616+0000","last_unstale":"2023-08-27T11:18:06.081671+0000","last_undegraded":"2023-08-27T11:18:06.081671+0000","last_fullsized":"2023-08-27T11:18:06.081671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:33:45.827898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796460+0000","last_change":"2023-08-27T11:18:03.251651+0000","last_active":"2023-08-27T11:18:06.796460+0000","last_peered":"2023-08-27T11:18:06.796460+0000","last_clean":"2023-08-27T11:18:06.796460+0000","last_became_active":"2023-08-27T11:18:03.251415+0000","last_became_peered":"2023-08-27T11:18:03.251415+0000","last_unstale":"2023-08-27T11:18:06.796460+0000","last_undegraded":"2023-08-27T11:18:06.796460+0000","last_fullsized":"2023-08-27T11:18:06.796460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:02:13.622820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796771+0000","last_change":"2023-08-27T11:18:04.267587+0000","last_active":"2023-08-27T11:18:06.796771+0000","last_peered":"2023-08-27T11:18:06.796771+0000","last_clean":"2023-08-27T11:18:06.796771+0000","last_became_active":"2023-08-27T11:18:04.267338+0000","last_became_peered":"2023-08-27T11:18:04.267338+0000","last_unstale":"2023-08-27T11:18:06.796771+0000","last_undegraded":"2023-08-27T11:18:06.796771+0000","last_fullsized":"2023-08-27T11:18:06.796771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:00:23.976483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081377+0000","last_change":"2023-08-27T11:18:03.250378+0000","last_active":"2023-08-27T11:18:06.081377+0000","last_peered":"2023-08-27T11:18:06.081377+0000","last_clean":"2023-08-27T11:18:06.081377+0000","last_became_active":"2023-08-27T11:18:03.250134+0000","last_became_peered":"2023-08-27T11:18:03.250134+0000","last_unstale":"2023-08-27T11:18:06.081377+0000","last_undegraded":"2023-08-27T11:18:06.081377+0000","last_fullsized":"2023-08-27T11:18:06.081377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:35:48.840154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257729+0000","last_change":"2023-08-27T11:18:04.264156+0000","last_active":"2023-08-27T11:18:05.257729+0000","last_peered":"2023-08-27T11:18:05.257729+0000","last_clean":"2023-08-27T11:18:05.257729+0000","last_became_active":"2023-08-27T11:18:04.263798+0000","last_became_peered":"2023-08-27T11:18:04.263798+0000","last_unstale":"2023-08-27T11:18:05.257729+0000","last_undegraded":"2023-08-27T11:18:05.257729+0000","last_fullsized":"2023-08-27T11:18:05.257729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:12.709794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081327+0000","last_change":"2023-08-27T11:18:03.250737+0000","last_active":"2023-08-27T11:18:06.081327+0000","last_peered":"2023-08-27T11:18:06.081327+0000","last_clean":"2023-08-27T11:18:06.081327+0000","last_became_active":"2023-08-27T11:18:03.250454+0000","last_became_peered":"2023-08-27T11:18:03.250454+0000","last_unstale":"2023-08-27T11:18:06.081327+0000","last_undegraded":"2023-08-27T11:18:06.081327+0000","last_fullsized":"2023-08-27T11:18:06.081327+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:49:51.997537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084537+0000","last_change":"2023-08-27T11:18:04.259741+0000","last_active":"2023-08-27T11:18:06.084537+0000","last_peered":"2023-08-27T11:18:06.084537+0000","last_clean":"2023-08-27T11:18:06.084537+0000","last_became_active":"2023-08-27T11:18:04.259463+0000","last_became_peered":"2023-08-27T11:18:04.259463+0000","last_unstale":"2023-08-27T11:18:06.084537+0000","last_undegraded":"2023-08-27T11:18:06.084537+0000","last_fullsized":"2023-08-27T11:18:06.084537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:29.089882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796887+0000","last_change":"2023-08-27T11:18:03.261229+0000","last_active":"2023-08-27T11:18:06.796887+0000","last_peered":"2023-08-27T11:18:06.796887+0000","last_clean":"2023-08-27T11:18:06.796887+0000","last_became_active":"2023-08-27T11:18:03.260955+0000","last_became_peered":"2023-08-27T11:18:03.260955+0000","last_unstale":"2023-08-27T11:18:06.796887+0000","last_undegraded":"2023-08-27T11:18:06.796887+0000","last_fullsized":"2023-08-27T11:18:06.796887+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:01.354170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796379+0000","last_change":"2023-08-27T11:18:04.263733+0000","last_active":"2023-08-27T11:18:06.796379+0000","last_peered":"2023-08-27T11:18:06.796379+0000","last_clean":"2023-08-27T11:18:06.796379+0000","last_became_active":"2023-08-27T11:18:04.263379+0000","last_became_peered":"2023-08-27T11:18:04.263379+0000","last_unstale":"2023-08-27T11:18:06.796379+0000","last_undegraded":"2023-08-27T11:18:06.796379+0000","last_fullsized":"2023-08-27T11:18:06.796379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:34.331158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081232+0000","last_change":"2023-08-27T11:18:03.250425+0000","last_active":"2023-08-27T11:18:06.081232+0000","last_peered":"2023-08-27T11:18:06.081232+0000","last_clean":"2023-08-27T11:18:06.081232+0000","last_became_active":"2023-08-27T11:18:03.250161+0000","last_became_peered":"2023-08-27T11:18:03.250161+0000","last_unstale":"2023-08-27T11:18:06.081232+0000","last_undegraded":"2023-08-27T11:18:06.081232+0000","last_fullsized":"2023-08-27T11:18:06.081232+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:46.837627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256716+0000","last_change":"2023-08-27T11:18:04.256639+0000","last_active":"2023-08-27T11:18:05.256716+0000","last_peered":"2023-08-27T11:18:05.256716+0000","last_clean":"2023-08-27T11:18:05.256716+0000","last_became_active":"2023-08-27T11:18:04.256360+0000","last_became_peered":"2023-08-27T11:18:04.256360+0000","last_unstale":"2023-08-27T11:18:05.256716+0000","last_undegraded":"2023-08-27T11:18:05.256716+0000","last_fullsized":"2023-08-27T11:18:05.256716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:53:41.349653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256497+0000","last_change":"2023-08-27T11:18:03.256597+0000","last_active":"2023-08-27T11:18:05.256497+0000","last_peered":"2023-08-27T11:18:05.256497+0000","last_clean":"2023-08-27T11:18:05.256497+0000","last_became_active":"2023-08-27T11:18:03.256301+0000","last_became_peered":"2023-08-27T11:18:03.256301+0000","last_unstale":"2023-08-27T11:18:05.256497+0000","last_undegraded":"2023-08-27T11:18:05.256497+0000","last_fullsized":"2023-08-27T11:18:05.256497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:40.754237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256830+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:18:05.256830+0000","last_peered":"2023-08-27T11:18:05.256830+0000","last_clean":"2023-08-27T11:18:05.256830+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:18:05.256830+0000","last_undegraded":"2023-08-27T11:18:05.256830+0000","last_fullsized":"2023-08-27T11:18:05.256830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:22:28.764468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797106+0000","last_change":"2023-08-27T11:18:04.266366+0000","last_active":"2023-08-27T11:18:06.797106+0000","last_peered":"2023-08-27T11:18:06.797106+0000","last_clean":"2023-08-27T11:18:06.797106+0000","last_became_active":"2023-08-27T11:18:04.265953+0000","last_became_peered":"2023-08-27T11:18:04.265953+0000","last_unstale":"2023-08-27T11:18:06.797106+0000","last_undegraded":"2023-08-27T11:18:06.797106+0000","last_fullsized":"2023-08-27T11:18:06.797106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:03.267453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.795921+0000","last_change":"2023-08-27T11:18:03.257780+0000","last_active":"2023-08-27T11:18:06.795921+0000","last_peered":"2023-08-27T11:18:06.795921+0000","last_clean":"2023-08-27T11:18:06.795921+0000","last_became_active":"2023-08-27T11:18:03.255666+0000","last_became_peered":"2023-08-27T11:18:03.255666+0000","last_unstale":"2023-08-27T11:18:06.795921+0000","last_undegraded":"2023-08-27T11:18:06.795921+0000","last_fullsized":"2023-08-27T11:18:06.795921+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:04.176421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796319+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:18:06.796319+0000","last_peered":"2023-08-27T11:18:06.796319+0000","last_clean":"2023-08-27T11:18:06.796319+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:18:06.796319+0000","last_undegraded":"2023-08-27T11:18:06.796319+0000","last_fullsized":"2023-08-27T11:18:06.796319+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:57.429873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796501+0000","last_change":"2023-08-27T11:18:04.273615+0000","last_active":"2023-08-27T11:18:06.796501+0000","last_peered":"2023-08-27T11:18:06.796501+0000","last_clean":"2023-08-27T11:18:06.796501+0000","last_became_active":"2023-08-27T11:18:04.273250+0000","last_became_peered":"2023-08-27T11:18:04.273250+0000","last_unstale":"2023-08-27T11:18:06.796501+0000","last_undegraded":"2023-08-27T11:18:06.796501+0000","last_fullsized":"2023-08-27T11:18:06.796501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:04:35.973096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.085014+0000","last_change":"2023-08-27T11:18:03.259045+0000","last_active":"2023-08-27T11:18:06.085014+0000","last_peered":"2023-08-27T11:18:06.085014+0000","last_clean":"2023-08-27T11:18:06.085014+0000","last_became_active":"2023-08-27T11:18:03.256833+0000","last_became_peered":"2023-08-27T11:18:03.256833+0000","last_unstale":"2023-08-27T11:18:06.085014+0000","last_undegraded":"2023-08-27T11:18:06.085014+0000","last_fullsized":"2023-08-27T11:18:06.085014+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:04.176421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796279+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:18:06.796279+0000","last_peered":"2023-08-27T11:18:06.796279+0000","last_clean":"2023-08-27T11:18:06.796279+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:18:06.796279+0000","last_undegraded":"2023-08-27T11:18:06.796279+0000","last_fullsized":"2023-08-27T11:18:06.796279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:06:03.203579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":189,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797321+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:18:06.797321+0000","last_peered":"2023-08-27T11:18:06.797321+0000","last_clean":"2023-08-27T11:18:06.797321+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:18:06.797321+0000","last_undegraded":"2023-08-27T11:18:06.797321+0000","last_fullsized":"2023-08-27T11:18:06.797321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:38:40.551571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797232+0000","last_change":"2023-08-27T11:18:04.265567+0000","last_active":"2023-08-27T11:18:06.797232+0000","last_peered":"2023-08-27T11:18:06.797232+0000","last_clean":"2023-08-27T11:18:06.797232+0000","last_became_active":"2023-08-27T11:18:04.265192+0000","last_became_peered":"2023-08-27T11:18:04.265192+0000","last_unstale":"2023-08-27T11:18:06.797232+0000","last_undegraded":"2023-08-27T11:18:06.797232+0000","last_fullsized":"2023-08-27T11:18:06.797232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:00:36.005972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081661+0000","last_change":"2023-08-27T11:18:03.263247+0000","last_active":"2023-08-27T11:18:06.081661+0000","last_peered":"2023-08-27T11:18:06.081661+0000","last_clean":"2023-08-27T11:18:06.081661+0000","last_became_active":"2023-08-27T11:18:03.262935+0000","last_became_peered":"2023-08-27T11:18:03.262935+0000","last_unstale":"2023-08-27T11:18:06.081661+0000","last_undegraded":"2023-08-27T11:18:06.081661+0000","last_fullsized":"2023-08-27T11:18:06.081661+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:30:06.873560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796787+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:18:06.796787+0000","last_peered":"2023-08-27T11:18:06.796787+0000","last_clean":"2023-08-27T11:18:06.796787+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:18:06.796787+0000","last_undegraded":"2023-08-27T11:18:06.796787+0000","last_fullsized":"2023-08-27T11:18:06.796787+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:58.119385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084941+0000","last_change":"2023-08-27T11:18:04.269410+0000","last_active":"2023-08-27T11:18:06.084941+0000","last_peered":"2023-08-27T11:18:06.084941+0000","last_clean":"2023-08-27T11:18:06.084941+0000","last_became_active":"2023-08-27T11:18:04.269152+0000","last_became_peered":"2023-08-27T11:18:04.269152+0000","last_unstale":"2023-08-27T11:18:06.084941+0000","last_undegraded":"2023-08-27T11:18:06.084941+0000","last_fullsized":"2023-08-27T11:18:06.084941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:07:21.159766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797143+0000","last_change":"2023-08-27T11:18:03.261146+0000","last_active":"2023-08-27T11:18:06.797143+0000","last_peered":"2023-08-27T11:18:06.797143+0000","last_clean":"2023-08-27T11:18:06.797143+0000","last_became_active":"2023-08-27T11:18:03.260863+0000","last_became_peered":"2023-08-27T11:18:03.260863+0000","last_unstale":"2023-08-27T11:18:06.797143+0000","last_undegraded":"2023-08-27T11:18:06.797143+0000","last_fullsized":"2023-08-27T11:18:06.797143+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:12:40.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796874+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:18:06.796874+0000","last_peered":"2023-08-27T11:18:06.796874+0000","last_clean":"2023-08-27T11:18:06.796874+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:18:06.796874+0000","last_undegraded":"2023-08-27T11:18:06.796874+0000","last_fullsized":"2023-08-27T11:18:06.796874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:40:40.032091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796452+0000","last_change":"2023-08-27T11:18:04.263009+0000","last_active":"2023-08-27T11:18:06.796452+0000","last_peered":"2023-08-27T11:18:06.796452+0000","last_clean":"2023-08-27T11:18:06.796452+0000","last_became_active":"2023-08-27T11:18:04.262652+0000","last_became_peered":"2023-08-27T11:18:04.262652+0000","last_unstale":"2023-08-27T11:18:06.796452+0000","last_undegraded":"2023-08-27T11:18:06.796452+0000","last_fullsized":"2023-08-27T11:18:06.796452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:03.267453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257731+0000","last_change":"2023-08-27T11:18:03.252729+0000","last_active":"2023-08-27T11:18:05.257731+0000","last_peered":"2023-08-27T11:18:05.257731+0000","last_clean":"2023-08-27T11:18:05.257731+0000","last_became_active":"2023-08-27T11:18:03.252459+0000","last_became_peered":"2023-08-27T11:18:03.252459+0000","last_unstale":"2023-08-27T11:18:05.257731+0000","last_undegraded":"2023-08-27T11:18:05.257731+0000","last_fullsized":"2023-08-27T11:18:05.257731+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:05:20.307605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084518+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:18:06.084518+0000","last_peered":"2023-08-27T11:18:06.084518+0000","last_clean":"2023-08-27T11:18:06.084518+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:18:06.084518+0000","last_undegraded":"2023-08-27T11:18:06.084518+0000","last_fullsized":"2023-08-27T11:18:06.084518+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:04:16.383307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796187+0000","last_change":"2023-08-27T11:18:04.264759+0000","last_active":"2023-08-27T11:18:06.796187+0000","last_peered":"2023-08-27T11:18:06.796187+0000","last_clean":"2023-08-27T11:18:06.796187+0000","last_became_active":"2023-08-27T11:18:04.264416+0000","last_became_peered":"2023-08-27T11:18:04.264416+0000","last_unstale":"2023-08-27T11:18:06.796187+0000","last_undegraded":"2023-08-27T11:18:06.796187+0000","last_fullsized":"2023-08-27T11:18:06.796187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:38:34.516670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257453+0000","last_change":"2023-08-27T11:18:03.255301+0000","last_active":"2023-08-27T11:18:05.257453+0000","last_peered":"2023-08-27T11:18:05.257453+0000","last_clean":"2023-08-27T11:18:05.257453+0000","last_became_active":"2023-08-27T11:18:03.255034+0000","last_became_peered":"2023-08-27T11:18:03.255034+0000","last_unstale":"2023-08-27T11:18:05.257453+0000","last_undegraded":"2023-08-27T11:18:05.257453+0000","last_fullsized":"2023-08-27T11:18:05.257453+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:20:38.147319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257052+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:18:05.257052+0000","last_peered":"2023-08-27T11:18:05.257052+0000","last_clean":"2023-08-27T11:18:05.257052+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:18:05.257052+0000","last_undegraded":"2023-08-27T11:18:05.257052+0000","last_fullsized":"2023-08-27T11:18:05.257052+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:37:29.587031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257388+0000","last_change":"2023-08-27T11:18:04.254633+0000","last_active":"2023-08-27T11:18:05.257388+0000","last_peered":"2023-08-27T11:18:05.257388+0000","last_clean":"2023-08-27T11:18:05.257388+0000","last_became_active":"2023-08-27T11:18:04.254350+0000","last_became_peered":"2023-08-27T11:18:04.254350+0000","last_unstale":"2023-08-27T11:18:05.257388+0000","last_undegraded":"2023-08-27T11:18:05.257388+0000","last_fullsized":"2023-08-27T11:18:05.257388+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:04:35.973096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257722+0000","last_change":"2023-08-27T11:18:03.256558+0000","last_active":"2023-08-27T11:18:05.257722+0000","last_peered":"2023-08-27T11:18:05.257722+0000","last_clean":"2023-08-27T11:18:05.257722+0000","last_became_active":"2023-08-27T11:18:03.256290+0000","last_became_peered":"2023-08-27T11:18:03.256290+0000","last_unstale":"2023-08-27T11:18:05.257722+0000","last_undegraded":"2023-08-27T11:18:05.257722+0000","last_fullsized":"2023-08-27T11:18:05.257722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:16:31.256534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257349+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:18:05.257349+0000","last_peered":"2023-08-27T11:18:05.257349+0000","last_clean":"2023-08-27T11:18:05.257349+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:18:05.257349+0000","last_undegraded":"2023-08-27T11:18:05.257349+0000","last_fullsized":"2023-08-27T11:18:05.257349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:09.698522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":739236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.502}]},{"osd":1,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.45900000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.54500000000000004}]},{"osd":3,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.433}]},{"osd":4,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.56299999999999994}]},{"osd":5,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.67900000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.621}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":154421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.443},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.51600000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58299999999999996}]},{"osd":2,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.626}]},{"osd":3,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54700000000000004}]},{"osd":4,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.5},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.61299999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.66600000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.65200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":958464,"data_stored":737644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":11.971,"5min":11.971,"15min":11.971},"last":0.34100000000000003},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":23.045999999999999,"5min":23.045999999999999,"15min":23.045999999999999},"last":0.254}]},{"osd":2,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.928000000000001,"5min":11.928000000000001,"15min":11.928000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.371}]},{"osd":3,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":10.99,"5min":10.99,"15min":10.99},"last":0.31900000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":22.893999999999998,"5min":22.893999999999998,"15min":22.893999999999998},"last":0.28699999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":10.957000000000001,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.40899999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.75,"15min":1.75},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":23.016999999999999,"5min":23.016999999999999,"15min":23.016999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.41999999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":10.925000000000001,"5min":10.925000000000001,"15min":10.925000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":22.952000000000002,"5min":22.952000000000002,"15min":22.952000000000002},"last":0.57699999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":22.988,"5min":22.988,"15min":22.988},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.46400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":152512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.28299999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.32500000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.308},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.42399999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.63},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.69799999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.77500000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.78800000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.80400000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":153909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":0.29099999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.255}]},{"osd":3,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.36399999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.64600000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52400000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.495},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56399999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.63100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":148832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.224},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.248}]},{"osd":1,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.28599999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.26500000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42199999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.377},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45800000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.40899999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.48999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":154443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.495}]},{"osd":1,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.57999999999999996}]},{"osd":2,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.54500000000000004}]},{"osd":3,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.56200000000000006}]},{"osd":5,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.375},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.52900000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47599999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.50700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":153951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45200000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.53900000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.56799999999999995}]},{"osd":3,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.59699999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.627},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.745}]},{"osd":6,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:18:08.353 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:18:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.503+0000 7f538d72e700 1 Processor -- start 2023-08-27T11:18:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.503+0000 7f538d72e700 1 -- start start 2023-08-27T11:18:08.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.504+0000 7f538d72e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f53880b4960 0x7f538815db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.504+0000 7f538d72e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53880b4dc0 0x7f538815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.504+0000 7f538d72e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53880cfe70 con 0x7f5388143a60 2023-08-27T11:18:08.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.504+0000 7f538d72e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5388143e40 con 0x7f53880b4960 2023-08-27T11:18:08.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.504+0000 7f538d72e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53880b5340 con 0x7f53880b4dc0 2023-08-27T11:18:08.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.504+0000 7f53867fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f53880b4960 0x7f538815db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.504+0000 7f5386ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5388143a60 0x7f53880cf760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51250/0 (socket says 172.21.15.112:51250) 2023-08-27T11:18:08.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.504+0000 7f5386ffd700 1 -- 172.21.15.112:0/3236443385 learned_addr learned my addr 172.21.15.112:0/3236443385 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:08.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.504+0000 7f53877fe700 1 --2- 172.21.15.112:0/3236443385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53880b4dc0 0x7f538815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f53867fc700 1 -- 172.21.15.112:0/3236443385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53880b4dc0 msgr2=0x7f538815e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:08.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f53867fc700 1 --2- 172.21.15.112:0/3236443385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53880b4dc0 0x7f538815e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f53867fc700 1 -- 172.21.15.112:0/3236443385 >> v1:172.21.15.112:6789/0 conn(0x7f5388143a60 legacy=0x7f53880cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f53867fc700 1 -- 172.21.15.112:0/3236443385 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53880ce9a0 con 0x7f53880b4960 2023-08-27T11:18:08.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f5385ffb700 1 -- 172.21.15.112:0/3236443385 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106200661 0 0) 0x7f53880cfe70 con 0x7f5388143a60 2023-08-27T11:18:08.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f53867fc700 1 --2- 172.21.15.112:0/3236443385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53880b4960 0x7f538815db80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f537800bac0 tx=0x7f537800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f58d3ed642229bc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:08.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f5385ffb700 1 -- 172.21.15.112:0/3236443385 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378005e30 con 0x7f53880b4960 2023-08-27T11:18:08.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f5385ffb700 1 -- 172.21.15.112:0/3236443385 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537800d860 con 0x7f53880b4960 2023-08-27T11:18:08.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f5385ffb700 1 -- 172.21.15.112:0/3236443385 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537800dbf0 con 0x7f53880b4960 2023-08-27T11:18:08.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f538d72e700 1 -- 172.21.15.112:0/3236443385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53880b4960 msgr2=0x7f538815db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f538d72e700 1 --2- 172.21.15.112:0/3236443385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53880b4960 0x7f538815db80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f537800bac0 tx=0x7f537800bdd0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f538d72e700 1 -- 172.21.15.112:0/3236443385 shutdown_connections 2023-08-27T11:18:08.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f538d72e700 1 --2- 172.21.15.112:0/3236443385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53880b4dc0 0x7f538815e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f538d72e700 1 --2- 172.21.15.112:0/3236443385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53880b4960 0x7f538815db80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f538d72e700 1 -- 172.21.15.112:0/3236443385 >> 172.21.15.112:0/3236443385 conn(0x7f538806adb0 msgr2=0x7f538806b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:08.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.505+0000 7f538d72e700 1 -- 172.21.15.112:0/3236443385 shutdown_connections 2023-08-27T11:18:08.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.506+0000 7f538d72e700 1 -- 172.21.15.112:0/3236443385 wait complete. 2023-08-27T11:18:08.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.507+0000 7f538d72e700 1 Processor -- start 2023-08-27T11:18:08.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.507+0000 7f538d72e700 1 -- start start 2023-08-27T11:18:08.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f538d72e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53880b4960 0x7f538812c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f538d72e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53880b4dc0 0x7f5388154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f538d72e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5388143a60 0x7f53881553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f538d72e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538812e830 con 0x7f53880b4960 2023-08-27T11:18:08.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f538d72e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53881f2310 con 0x7f53880b4dc0 2023-08-27T11:18:08.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f538d72e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53881f25f0 con 0x7f5388143a60 2023-08-27T11:18:08.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f5386ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53880b4960 0x7f538812c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f5386ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53880b4960 0x7f538812c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56874/0 (socket says 172.21.15.112:56874) 2023-08-27T11:18:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f5386ffd700 1 -- 172.21.15.112:0/3578755046 learned_addr learned my addr 172.21.15.112:0/3578755046 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f5386ffd700 1 -- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5388143a60 msgr2=0x7f53881553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f53877fe700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5388143a60 0x7f53881553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f5386ffd700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5388143a60 0x7f53881553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f53867fc700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53880b4dc0 0x7f5388154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f5386ffd700 1 -- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53880b4dc0 msgr2=0x7f5388154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f5386ffd700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53880b4dc0 0x7f5388154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f5386ffd700 1 -- 172.21.15.112:0/3578755046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537800b6d0 con 0x7f53880b4960 2023-08-27T11:18:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f53867fc700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53880b4dc0 0x7f5388154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f5386ffd700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53880b4960 0x7f538812c710 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f537c00c4a0 tx=0x7f537c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f536f7fe700 1 -- 172.21.15.112:0/3578755046 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c0074e0 con 0x7f53880b4960 2023-08-27T11:18:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f536f7fe700 1 -- 172.21.15.112:0/3578755046 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537c007640 con 0x7f53880b4960 2023-08-27T11:18:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f538d72e700 1 -- 172.21.15.112:0/3578755046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53881f28d0 con 0x7f53880b4960 2023-08-27T11:18:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f536f7fe700 1 -- 172.21.15.112:0/3578755046 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c005a70 con 0x7f53880b4960 2023-08-27T11:18:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.508+0000 7f538d72e700 1 -- 172.21.15.112:0/3578755046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538812cc50 con 0x7f53880b4960 2023-08-27T11:18:08.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.509+0000 7f538d72e700 1 -- 172.21.15.112:0/3578755046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53880ce180 con 0x7f53880b4960 2023-08-27T11:18:08.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.510+0000 7f536f7fe700 1 -- 172.21.15.112:0/3578755046 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f537c005bd0 con 0x7f53880b4960 2023-08-27T11:18:08.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.510+0000 7f536f7fe700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f537406a0e0 0x7f537406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.510+0000 7f536f7fe700 1 -- 172.21.15.112:0/3578755046 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f537c087a30 con 0x7f53880b4960 2023-08-27T11:18:08.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.510+0000 7f53867fc700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f537406a0e0 0x7f537406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.511+0000 7f53867fc700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f537406a0e0 0x7f537406c5a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5388154ab0 tx=0x7f537801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:08.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.515+0000 7f536f7fe700 1 -- 172.21.15.112:0/3578755046 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f537c053120 con 0x7f53880b4960 2023-08-27T11:18:08.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.698+0000 7f538d72e700 1 -- 172.21.15.112:0/3578755046 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53880cef60 con 0x7f537406a0e0 2023-08-27T11:18:08.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.708+0000 7f536f7fe700 1 -- 172.21.15.112:0/3578755046 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311631 (secure 0 0 0) 0x7f537800b550 con 0x7f537406a0e0 2023-08-27T11:18:08.713 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.711+0000 7f538d72e700 1 -- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f537406a0e0 msgr2=0x7f537406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.711+0000 7f538d72e700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f537406a0e0 0x7f537406c5a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5388154ab0 tx=0x7f537801c040 comp rx=0 tx=0).stop 2023-08-27T11:18:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.711+0000 7f538d72e700 1 -- 172.21.15.112:0/3578755046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53880b4960 msgr2=0x7f538812c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.711+0000 7f538d72e700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53880b4960 0x7f538812c710 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f537c00c4a0 tx=0x7f537c00c860 comp rx=0 tx=0).stop 2023-08-27T11:18:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.711+0000 7f538d72e700 1 -- 172.21.15.112:0/3578755046 shutdown_connections 2023-08-27T11:18:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.711+0000 7f538d72e700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f537406a0e0 0x7f537406c5a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.711+0000 7f538d72e700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5388143a60 0x7f53881553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.711+0000 7f538d72e700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53880b4960 0x7f538812c710 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.711+0000 7f538d72e700 1 --2- 172.21.15.112:0/3578755046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53880b4dc0 0x7f5388154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.711+0000 7f538d72e700 1 -- 172.21.15.112:0/3578755046 >> 172.21.15.112:0/3578755046 conn(0x7f538806adb0 msgr2=0x7f5388071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:08.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.711+0000 7f538d72e700 1 -- 172.21.15.112:0/3578755046 shutdown_connections 2023-08-27T11:18:08.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.711+0000 7f538d72e700 1 -- 172.21.15.112:0/3578755046 wait complete. 2023-08-27T11:18:08.718 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:18:08.724 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2023-08-27T11:18:08.239379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220280,"kb_used_data":4096,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754754440,"statfs":{"total":773094113280,"available":772868546560,"internally_reserved":0,"allocated":4194304,"data_stored":2394948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001561"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796682+0000","last_change":"2023-08-27T11:18:04.257125+0000","last_active":"2023-08-27T11:18:06.796682+0000","last_peered":"2023-08-27T11:18:06.796682+0000","last_clean":"2023-08-27T11:18:06.796682+0000","last_became_active":"2023-08-27T11:18:04.256863+0000","last_became_peered":"2023-08-27T11:18:04.256863+0000","last_unstale":"2023-08-27T11:18:06.796682+0000","last_undegraded":"2023-08-27T11:18:06.796682+0000","last_fullsized":"2023-08-27T11:18:06.796682+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:50:15.147923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796139+0000","last_change":"2023-08-27T11:18:04.254854+0000","last_active":"2023-08-27T11:18:06.796139+0000","last_peered":"2023-08-27T11:18:06.796139+0000","last_clean":"2023-08-27T11:18:06.796139+0000","last_became_active":"2023-08-27T11:18:04.254554+0000","last_became_peered":"2023-08-27T11:18:04.254554+0000","last_unstale":"2023-08-27T11:18:06.796139+0000","last_undegraded":"2023-08-27T11:18:06.796139+0000","last_fullsized":"2023-08-27T11:18:06.796139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:38:49.728424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257571+0000","last_change":"2023-08-27T11:18:04.273065+0000","last_active":"2023-08-27T11:18:05.257571+0000","last_peered":"2023-08-27T11:18:05.257571+0000","last_clean":"2023-08-27T11:18:05.257571+0000","last_became_active":"2023-08-27T11:18:04.272786+0000","last_became_peered":"2023-08-27T11:18:04.272786+0000","last_unstale":"2023-08-27T11:18:05.257571+0000","last_undegraded":"2023-08-27T11:18:05.257571+0000","last_fullsized":"2023-08-27T11:18:05.257571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:18:51.736233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081577+0000","last_change":"2023-08-27T11:18:04.263371+0000","last_active":"2023-08-27T11:18:06.081577+0000","last_peered":"2023-08-27T11:18:06.081577+0000","last_clean":"2023-08-27T11:18:06.081577+0000","last_became_active":"2023-08-27T11:18:04.263114+0000","last_became_peered":"2023-08-27T11:18:04.263114+0000","last_unstale":"2023-08-27T11:18:06.081577+0000","last_undegraded":"2023-08-27T11:18:06.081577+0000","last_fullsized":"2023-08-27T11:18:06.081577+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:23:41.688216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081389+0000","last_change":"2023-08-27T11:18:04.262970+0000","last_active":"2023-08-27T11:18:06.081389+0000","last_peered":"2023-08-27T11:18:06.081389+0000","last_clean":"2023-08-27T11:18:06.081389+0000","last_became_active":"2023-08-27T11:18:04.262688+0000","last_became_peered":"2023-08-27T11:18:04.262688+0000","last_unstale":"2023-08-27T11:18:06.081389+0000","last_undegraded":"2023-08-27T11:18:06.081389+0000","last_fullsized":"2023-08-27T11:18:06.081389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:07.829879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796109+0000","last_change":"2023-08-27T11:18:04.260800+0000","last_active":"2023-08-27T11:18:06.796109+0000","last_peered":"2023-08-27T11:18:06.796109+0000","last_clean":"2023-08-27T11:18:06.796109+0000","last_became_active":"2023-08-27T11:18:04.260538+0000","last_became_peered":"2023-08-27T11:18:04.260538+0000","last_unstale":"2023-08-27T11:18:06.796109+0000","last_undegraded":"2023-08-27T11:18:06.796109+0000","last_fullsized":"2023-08-27T11:18:06.796109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:03.931204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084711+0000","last_change":"2023-08-27T11:18:04.268721+0000","last_active":"2023-08-27T11:18:06.084711+0000","last_peered":"2023-08-27T11:18:06.084711+0000","last_clean":"2023-08-27T11:18:06.084711+0000","last_became_active":"2023-08-27T11:18:04.268092+0000","last_became_peered":"2023-08-27T11:18:04.268092+0000","last_unstale":"2023-08-27T11:18:06.084711+0000","last_undegraded":"2023-08-27T11:18:06.084711+0000","last_fullsized":"2023-08-27T11:18:06.084711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:58.144442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.795936+0000","last_change":"2023-08-27T11:18:04.259189+0000","last_active":"2023-08-27T11:18:06.795936+0000","last_peered":"2023-08-27T11:18:06.795936+0000","last_clean":"2023-08-27T11:18:06.795936+0000","last_became_active":"2023-08-27T11:18:04.258850+0000","last_became_peered":"2023-08-27T11:18:04.258850+0000","last_unstale":"2023-08-27T11:18:06.795936+0000","last_undegraded":"2023-08-27T11:18:06.795936+0000","last_fullsized":"2023-08-27T11:18:06.795936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:02:56.610036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257680+0000","last_change":"2023-08-27T11:18:04.270302+0000","last_active":"2023-08-27T11:18:05.257680+0000","last_peered":"2023-08-27T11:18:05.257680+0000","last_clean":"2023-08-27T11:18:05.257680+0000","last_became_active":"2023-08-27T11:18:04.270032+0000","last_became_peered":"2023-08-27T11:18:04.270032+0000","last_unstale":"2023-08-27T11:18:05.257680+0000","last_undegraded":"2023-08-27T11:18:05.257680+0000","last_fullsized":"2023-08-27T11:18:05.257680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:29.089882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084727+0000","last_change":"2023-08-27T11:18:04.267180+0000","last_active":"2023-08-27T11:18:06.084727+0000","last_peered":"2023-08-27T11:18:06.084727+0000","last_clean":"2023-08-27T11:18:06.084727+0000","last_became_active":"2023-08-27T11:18:04.266796+0000","last_became_peered":"2023-08-27T11:18:04.266796+0000","last_unstale":"2023-08-27T11:18:06.084727+0000","last_undegraded":"2023-08-27T11:18:06.084727+0000","last_fullsized":"2023-08-27T11:18:06.084727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:20:45.867963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084939+0000","last_change":"2023-08-27T11:18:04.264324+0000","last_active":"2023-08-27T11:18:06.084939+0000","last_peered":"2023-08-27T11:18:06.084939+0000","last_clean":"2023-08-27T11:18:06.084939+0000","last_became_active":"2023-08-27T11:18:04.264025+0000","last_became_peered":"2023-08-27T11:18:04.264025+0000","last_unstale":"2023-08-27T11:18:06.084939+0000","last_undegraded":"2023-08-27T11:18:06.084939+0000","last_fullsized":"2023-08-27T11:18:06.084939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:42:46.084990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257721+0000","last_change":"2023-08-27T11:18:04.256478+0000","last_active":"2023-08-27T11:18:05.257721+0000","last_peered":"2023-08-27T11:18:05.257721+0000","last_clean":"2023-08-27T11:18:05.257721+0000","last_became_active":"2023-08-27T11:18:04.256232+0000","last_became_peered":"2023-08-27T11:18:04.256232+0000","last_unstale":"2023-08-27T11:18:05.257721+0000","last_undegraded":"2023-08-27T11:18:05.257721+0000","last_fullsized":"2023-08-27T11:18:05.257721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:11:07.765705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084498+0000","last_change":"2023-08-27T11:18:04.268507+0000","last_active":"2023-08-27T11:18:06.084498+0000","last_peered":"2023-08-27T11:18:06.084498+0000","last_clean":"2023-08-27T11:18:06.084498+0000","last_became_active":"2023-08-27T11:18:04.268056+0000","last_became_peered":"2023-08-27T11:18:04.268056+0000","last_unstale":"2023-08-27T11:18:06.084498+0000","last_undegraded":"2023-08-27T11:18:06.084498+0000","last_fullsized":"2023-08-27T11:18:06.084498+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:11:07.765705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256982+0000","last_change":"2023-08-27T11:18:04.260786+0000","last_active":"2023-08-27T11:18:05.256982+0000","last_peered":"2023-08-27T11:18:05.256982+0000","last_clean":"2023-08-27T11:18:05.256982+0000","last_became_active":"2023-08-27T11:18:04.260562+0000","last_became_peered":"2023-08-27T11:18:04.260562+0000","last_unstale":"2023-08-27T11:18:05.256982+0000","last_undegraded":"2023-08-27T11:18:05.256982+0000","last_fullsized":"2023-08-27T11:18:05.256982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:44.523518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084280+0000","last_change":"2023-08-27T11:18:04.269789+0000","last_active":"2023-08-27T11:18:06.084280+0000","last_peered":"2023-08-27T11:18:06.084280+0000","last_clean":"2023-08-27T11:18:06.084280+0000","last_became_active":"2023-08-27T11:18:04.269552+0000","last_became_peered":"2023-08-27T11:18:04.269552+0000","last_unstale":"2023-08-27T11:18:06.084280+0000","last_undegraded":"2023-08-27T11:18:06.084280+0000","last_fullsized":"2023-08-27T11:18:06.084280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:12.709794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796696+0000","last_change":"2023-08-27T11:18:04.271085+0000","last_active":"2023-08-27T11:18:06.796696+0000","last_peered":"2023-08-27T11:18:06.796696+0000","last_clean":"2023-08-27T11:18:06.796696+0000","last_became_active":"2023-08-27T11:18:04.270178+0000","last_became_peered":"2023-08-27T11:18:04.270178+0000","last_unstale":"2023-08-27T11:18:06.796696+0000","last_undegraded":"2023-08-27T11:18:06.796696+0000","last_fullsized":"2023-08-27T11:18:06.796696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:02:56.610036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797318+0000","last_change":"2023-08-27T11:18:04.273533+0000","last_active":"2023-08-27T11:18:06.797318+0000","last_peered":"2023-08-27T11:18:06.797318+0000","last_clean":"2023-08-27T11:18:06.797318+0000","last_became_active":"2023-08-27T11:18:04.273222+0000","last_became_peered":"2023-08-27T11:18:04.273222+0000","last_unstale":"2023-08-27T11:18:06.797318+0000","last_undegraded":"2023-08-27T11:18:06.797318+0000","last_fullsized":"2023-08-27T11:18:06.797318+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:50:15.147923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256986+0000","last_change":"2023-08-27T11:18:04.261729+0000","last_active":"2023-08-27T11:18:05.256986+0000","last_peered":"2023-08-27T11:18:05.256986+0000","last_clean":"2023-08-27T11:18:05.256986+0000","last_became_active":"2023-08-27T11:18:04.261316+0000","last_became_peered":"2023-08-27T11:18:04.261316+0000","last_unstale":"2023-08-27T11:18:05.256986+0000","last_undegraded":"2023-08-27T11:18:05.256986+0000","last_fullsized":"2023-08-27T11:18:05.256986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:03.931204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081526+0000","last_change":"2023-08-27T11:18:04.260253+0000","last_active":"2023-08-27T11:18:06.081526+0000","last_peered":"2023-08-27T11:18:06.081526+0000","last_clean":"2023-08-27T11:18:06.081526+0000","last_became_active":"2023-08-27T11:18:04.259930+0000","last_became_peered":"2023-08-27T11:18:04.259930+0000","last_unstale":"2023-08-27T11:18:06.081526+0000","last_undegraded":"2023-08-27T11:18:06.081526+0000","last_fullsized":"2023-08-27T11:18:06.081526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:11:37.956860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257412+0000","last_change":"2023-08-27T11:18:04.261550+0000","last_active":"2023-08-27T11:18:05.257412+0000","last_peered":"2023-08-27T11:18:05.257412+0000","last_clean":"2023-08-27T11:18:05.257412+0000","last_became_active":"2023-08-27T11:18:04.261249+0000","last_became_peered":"2023-08-27T11:18:04.261249+0000","last_unstale":"2023-08-27T11:18:05.257412+0000","last_undegraded":"2023-08-27T11:18:05.257412+0000","last_fullsized":"2023-08-27T11:18:05.257412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:04:35.973096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084693+0000","last_change":"2023-08-27T11:18:04.268997+0000","last_active":"2023-08-27T11:18:06.084693+0000","last_peered":"2023-08-27T11:18:06.084693+0000","last_clean":"2023-08-27T11:18:06.084693+0000","last_became_active":"2023-08-27T11:18:04.268733+0000","last_became_peered":"2023-08-27T11:18:04.268733+0000","last_unstale":"2023-08-27T11:18:06.084693+0000","last_undegraded":"2023-08-27T11:18:06.084693+0000","last_fullsized":"2023-08-27T11:18:06.084693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:19.211820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256429+0000","last_change":"2023-08-27T11:18:04.257131+0000","last_active":"2023-08-27T11:18:05.256429+0000","last_peered":"2023-08-27T11:18:05.256429+0000","last_clean":"2023-08-27T11:18:05.256429+0000","last_became_active":"2023-08-27T11:18:04.256859+0000","last_became_peered":"2023-08-27T11:18:04.256859+0000","last_unstale":"2023-08-27T11:18:05.256429+0000","last_undegraded":"2023-08-27T11:18:05.256429+0000","last_fullsized":"2023-08-27T11:18:05.256429+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:31:19.039678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796832+0000","last_change":"2023-08-27T11:18:04.266733+0000","last_active":"2023-08-27T11:18:06.796832+0000","last_peered":"2023-08-27T11:18:06.796832+0000","last_clean":"2023-08-27T11:18:06.796832+0000","last_became_active":"2023-08-27T11:18:04.266350+0000","last_became_peered":"2023-08-27T11:18:04.266350+0000","last_unstale":"2023-08-27T11:18:06.796832+0000","last_undegraded":"2023-08-27T11:18:06.796832+0000","last_fullsized":"2023-08-27T11:18:06.796832+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:38:49.728424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081308+0000","last_change":"2023-08-27T11:18:04.269130+0000","last_active":"2023-08-27T11:18:06.081308+0000","last_peered":"2023-08-27T11:18:06.081308+0000","last_clean":"2023-08-27T11:18:06.081308+0000","last_became_active":"2023-08-27T11:18:04.268587+0000","last_became_peered":"2023-08-27T11:18:04.268587+0000","last_unstale":"2023-08-27T11:18:06.081308+0000","last_undegraded":"2023-08-27T11:18:06.081308+0000","last_fullsized":"2023-08-27T11:18:06.081308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:16:52.344515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257116+0000","last_change":"2023-08-27T11:18:04.257945+0000","last_active":"2023-08-27T11:18:05.257116+0000","last_peered":"2023-08-27T11:18:05.257116+0000","last_clean":"2023-08-27T11:18:05.257116+0000","last_became_active":"2023-08-27T11:18:04.257616+0000","last_became_peered":"2023-08-27T11:18:04.257616+0000","last_unstale":"2023-08-27T11:18:05.257116+0000","last_undegraded":"2023-08-27T11:18:05.257116+0000","last_fullsized":"2023-08-27T11:18:05.257116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:04:48.125632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796683+0000","last_change":"2023-08-27T11:18:04.273603+0000","last_active":"2023-08-27T11:18:06.796683+0000","last_peered":"2023-08-27T11:18:06.796683+0000","last_clean":"2023-08-27T11:18:06.796683+0000","last_became_active":"2023-08-27T11:18:04.273326+0000","last_became_peered":"2023-08-27T11:18:04.273326+0000","last_unstale":"2023-08-27T11:18:06.796683+0000","last_undegraded":"2023-08-27T11:18:06.796683+0000","last_fullsized":"2023-08-27T11:18:06.796683+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:12.709794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257400+0000","last_change":"2023-08-27T11:18:04.258196+0000","last_active":"2023-08-27T11:18:05.257400+0000","last_peered":"2023-08-27T11:18:05.257400+0000","last_clean":"2023-08-27T11:18:05.257400+0000","last_became_active":"2023-08-27T11:18:04.257902+0000","last_became_peered":"2023-08-27T11:18:04.257902+0000","last_unstale":"2023-08-27T11:18:05.257400+0000","last_undegraded":"2023-08-27T11:18:05.257400+0000","last_fullsized":"2023-08-27T11:18:05.257400+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:01:04.142517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084607+0000","last_change":"2023-08-27T11:18:04.268371+0000","last_active":"2023-08-27T11:18:06.084607+0000","last_peered":"2023-08-27T11:18:06.084607+0000","last_clean":"2023-08-27T11:18:06.084607+0000","last_became_active":"2023-08-27T11:18:04.268082+0000","last_became_peered":"2023-08-27T11:18:04.268082+0000","last_unstale":"2023-08-27T11:18:06.084607+0000","last_undegraded":"2023-08-27T11:18:06.084607+0000","last_fullsized":"2023-08-27T11:18:06.084607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:16:32.261875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256778+0000","last_change":"2023-08-27T11:18:04.259863+0000","last_active":"2023-08-27T11:18:05.256778+0000","last_peered":"2023-08-27T11:18:05.256778+0000","last_clean":"2023-08-27T11:18:05.256778+0000","last_became_active":"2023-08-27T11:18:04.259019+0000","last_became_peered":"2023-08-27T11:18:04.259019+0000","last_unstale":"2023-08-27T11:18:05.256778+0000","last_undegraded":"2023-08-27T11:18:05.256778+0000","last_fullsized":"2023-08-27T11:18:05.256778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:20:58.467861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256578+0000","last_change":"2023-08-27T11:18:04.257359+0000","last_active":"2023-08-27T11:18:05.256578+0000","last_peered":"2023-08-27T11:18:05.256578+0000","last_clean":"2023-08-27T11:18:05.256578+0000","last_became_active":"2023-08-27T11:18:04.256844+0000","last_became_peered":"2023-08-27T11:18:04.256844+0000","last_unstale":"2023-08-27T11:18:05.256578+0000","last_undegraded":"2023-08-27T11:18:05.256578+0000","last_fullsized":"2023-08-27T11:18:05.256578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:44:59.715657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796771+0000","last_change":"2023-08-27T11:18:04.266573+0000","last_active":"2023-08-27T11:18:06.796771+0000","last_peered":"2023-08-27T11:18:06.796771+0000","last_clean":"2023-08-27T11:18:06.796771+0000","last_became_active":"2023-08-27T11:18:04.266052+0000","last_became_peered":"2023-08-27T11:18:04.266052+0000","last_unstale":"2023-08-27T11:18:06.796771+0000","last_undegraded":"2023-08-27T11:18:06.796771+0000","last_fullsized":"2023-08-27T11:18:06.796771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:05.181763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796004+0000","last_change":"2023-08-27T11:18:04.258125+0000","last_active":"2023-08-27T11:18:06.796004+0000","last_peered":"2023-08-27T11:18:06.796004+0000","last_clean":"2023-08-27T11:18:06.796004+0000","last_became_active":"2023-08-27T11:18:04.257848+0000","last_became_peered":"2023-08-27T11:18:04.257848+0000","last_unstale":"2023-08-27T11:18:06.796004+0000","last_undegraded":"2023-08-27T11:18:06.796004+0000","last_fullsized":"2023-08-27T11:18:06.796004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:20:58.467861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797110+0000","last_change":"2023-08-27T11:18:04.272780+0000","last_active":"2023-08-27T11:18:06.797110+0000","last_peered":"2023-08-27T11:18:06.797110+0000","last_clean":"2023-08-27T11:18:06.797110+0000","last_became_active":"2023-08-27T11:18:04.272509+0000","last_became_peered":"2023-08-27T11:18:04.272509+0000","last_unstale":"2023-08-27T11:18:06.797110+0000","last_undegraded":"2023-08-27T11:18:06.797110+0000","last_fullsized":"2023-08-27T11:18:06.797110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:28:29.852805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"32'4","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257215+0000","last_change":"2023-08-27T11:18:03.254346+0000","last_active":"2023-08-27T11:18:05.257215+0000","last_peered":"2023-08-27T11:18:05.257215+0000","last_clean":"2023-08-27T11:18:05.257215+0000","last_became_active":"2023-08-27T11:18:03.254053+0000","last_became_peered":"2023-08-27T11:18:03.254053+0000","last_unstale":"2023-08-27T11:18:05.257215+0000","last_undegraded":"2023-08-27T11:18:05.257215+0000","last_fullsized":"2023-08-27T11:18:05.257215+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:07:07.127876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796584+0000","last_change":"2023-08-27T11:18:04.273641+0000","last_active":"2023-08-27T11:18:06.796584+0000","last_peered":"2023-08-27T11:18:06.796584+0000","last_clean":"2023-08-27T11:18:06.796584+0000","last_became_active":"2023-08-27T11:18:04.273374+0000","last_became_peered":"2023-08-27T11:18:04.273374+0000","last_unstale":"2023-08-27T11:18:06.796584+0000","last_undegraded":"2023-08-27T11:18:06.796584+0000","last_fullsized":"2023-08-27T11:18:06.796584+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:39:40.502028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257640+0000","last_change":"2023-08-27T11:18:03.253693+0000","last_active":"2023-08-27T11:18:05.257640+0000","last_peered":"2023-08-27T11:18:05.257640+0000","last_clean":"2023-08-27T11:18:05.257640+0000","last_became_active":"2023-08-27T11:18:03.253433+0000","last_became_peered":"2023-08-27T11:18:03.253433+0000","last_unstale":"2023-08-27T11:18:05.257640+0000","last_undegraded":"2023-08-27T11:18:05.257640+0000","last_fullsized":"2023-08-27T11:18:05.257640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:24.361949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257821+0000","last_change":"2023-08-27T11:18:04.260748+0000","last_active":"2023-08-27T11:18:05.257821+0000","last_peered":"2023-08-27T11:18:05.257821+0000","last_clean":"2023-08-27T11:18:05.257821+0000","last_became_active":"2023-08-27T11:18:04.260442+0000","last_became_peered":"2023-08-27T11:18:04.260442+0000","last_unstale":"2023-08-27T11:18:05.257821+0000","last_undegraded":"2023-08-27T11:18:05.257821+0000","last_fullsized":"2023-08-27T11:18:05.257821+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:43:37.898998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796557+0000","last_change":"2023-08-27T11:18:03.252549+0000","last_active":"2023-08-27T11:18:06.796557+0000","last_peered":"2023-08-27T11:18:06.796557+0000","last_clean":"2023-08-27T11:18:06.796557+0000","last_became_active":"2023-08-27T11:18:03.252302+0000","last_became_peered":"2023-08-27T11:18:03.252302+0000","last_unstale":"2023-08-27T11:18:06.796557+0000","last_undegraded":"2023-08-27T11:18:06.796557+0000","last_fullsized":"2023-08-27T11:18:06.796557+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:20:38.147319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796452+0000","last_change":"2023-08-27T11:18:04.259866+0000","last_active":"2023-08-27T11:18:06.796452+0000","last_peered":"2023-08-27T11:18:06.796452+0000","last_clean":"2023-08-27T11:18:06.796452+0000","last_became_active":"2023-08-27T11:18:04.258750+0000","last_became_peered":"2023-08-27T11:18:04.258750+0000","last_unstale":"2023-08-27T11:18:06.796452+0000","last_undegraded":"2023-08-27T11:18:06.796452+0000","last_fullsized":"2023-08-27T11:18:06.796452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:28:29.852805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257520+0000","last_change":"2023-08-27T11:18:03.258266+0000","last_active":"2023-08-27T11:18:05.257520+0000","last_peered":"2023-08-27T11:18:05.257520+0000","last_clean":"2023-08-27T11:18:05.257520+0000","last_became_active":"2023-08-27T11:18:03.257980+0000","last_became_peered":"2023-08-27T11:18:03.257980+0000","last_unstale":"2023-08-27T11:18:05.257520+0000","last_undegraded":"2023-08-27T11:18:05.257520+0000","last_fullsized":"2023-08-27T11:18:05.257520+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:14:28.905888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257301+0000","last_change":"2023-08-27T11:18:04.258361+0000","last_active":"2023-08-27T11:18:05.257301+0000","last_peered":"2023-08-27T11:18:05.257301+0000","last_clean":"2023-08-27T11:18:05.257301+0000","last_became_active":"2023-08-27T11:18:04.258084+0000","last_became_peered":"2023-08-27T11:18:04.258084+0000","last_unstale":"2023-08-27T11:18:05.257301+0000","last_undegraded":"2023-08-27T11:18:05.257301+0000","last_fullsized":"2023-08-27T11:18:05.257301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:02:14.628161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084433+0000","last_change":"2023-08-27T11:18:03.261880+0000","last_active":"2023-08-27T11:18:06.084433+0000","last_peered":"2023-08-27T11:18:06.084433+0000","last_clean":"2023-08-27T11:18:06.084433+0000","last_became_active":"2023-08-27T11:18:03.261672+0000","last_became_peered":"2023-08-27T11:18:03.261672+0000","last_unstale":"2023-08-27T11:18:06.084433+0000","last_undegraded":"2023-08-27T11:18:06.084433+0000","last_fullsized":"2023-08-27T11:18:06.084433+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:24.361949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084326+0000","last_change":"2023-08-27T11:18:04.268908+0000","last_active":"2023-08-27T11:18:06.084326+0000","last_peered":"2023-08-27T11:18:06.084326+0000","last_clean":"2023-08-27T11:18:06.084326+0000","last_became_active":"2023-08-27T11:18:04.268433+0000","last_became_peered":"2023-08-27T11:18:04.268433+0000","last_unstale":"2023-08-27T11:18:06.084326+0000","last_undegraded":"2023-08-27T11:18:06.084326+0000","last_fullsized":"2023-08-27T11:18:06.084326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:18:51.736233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797122+0000","last_change":"2023-08-27T11:18:03.260690+0000","last_active":"2023-08-27T11:18:06.797122+0000","last_peered":"2023-08-27T11:18:06.797122+0000","last_clean":"2023-08-27T11:18:06.797122+0000","last_became_active":"2023-08-27T11:18:03.260428+0000","last_became_peered":"2023-08-27T11:18:03.260428+0000","last_unstale":"2023-08-27T11:18:06.797122+0000","last_undegraded":"2023-08-27T11:18:06.797122+0000","last_fullsized":"2023-08-27T11:18:06.797122+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:34:17.508519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256580+0000","last_change":"2023-08-27T11:18:04.268936+0000","last_active":"2023-08-27T11:18:05.256580+0000","last_peered":"2023-08-27T11:18:05.256580+0000","last_clean":"2023-08-27T11:18:05.256580+0000","last_became_active":"2023-08-27T11:18:04.268677+0000","last_became_peered":"2023-08-27T11:18:04.268677+0000","last_unstale":"2023-08-27T11:18:05.256580+0000","last_undegraded":"2023-08-27T11:18:05.256580+0000","last_fullsized":"2023-08-27T11:18:05.256580+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:34:01.673085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796274+0000","last_change":"2023-08-27T11:18:03.261094+0000","last_active":"2023-08-27T11:18:06.796274+0000","last_peered":"2023-08-27T11:18:06.796274+0000","last_clean":"2023-08-27T11:18:06.796274+0000","last_became_active":"2023-08-27T11:18:03.260838+0000","last_became_peered":"2023-08-27T11:18:03.260838+0000","last_unstale":"2023-08-27T11:18:06.796274+0000","last_undegraded":"2023-08-27T11:18:06.796274+0000","last_fullsized":"2023-08-27T11:18:06.796274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:01:03.137176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796461+0000","last_change":"2023-08-27T11:18:04.263784+0000","last_active":"2023-08-27T11:18:06.796461+0000","last_peered":"2023-08-27T11:18:06.796461+0000","last_clean":"2023-08-27T11:18:06.796461+0000","last_became_active":"2023-08-27T11:18:04.263342+0000","last_became_peered":"2023-08-27T11:18:04.263342+0000","last_unstale":"2023-08-27T11:18:06.796461+0000","last_undegraded":"2023-08-27T11:18:06.796461+0000","last_fullsized":"2023-08-27T11:18:06.796461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:39.229819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256514+0000","last_change":"2023-08-27T11:18:03.257108+0000","last_active":"2023-08-27T11:18:05.256514+0000","last_peered":"2023-08-27T11:18:05.256514+0000","last_clean":"2023-08-27T11:18:05.256514+0000","last_became_active":"2023-08-27T11:18:03.256821+0000","last_became_peered":"2023-08-27T11:18:03.256821+0000","last_unstale":"2023-08-27T11:18:05.256514+0000","last_undegraded":"2023-08-27T11:18:05.256514+0000","last_fullsized":"2023-08-27T11:18:05.256514+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:56.352101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081616+0000","last_change":"2023-08-27T11:18:04.262997+0000","last_active":"2023-08-27T11:18:06.081616+0000","last_peered":"2023-08-27T11:18:06.081616+0000","last_clean":"2023-08-27T11:18:06.081616+0000","last_became_active":"2023-08-27T11:18:04.262756+0000","last_became_peered":"2023-08-27T11:18:04.262756+0000","last_unstale":"2023-08-27T11:18:06.081616+0000","last_undegraded":"2023-08-27T11:18:06.081616+0000","last_fullsized":"2023-08-27T11:18:06.081616+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:16:13.542596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796531+0000","last_change":"2023-08-27T11:18:03.256396+0000","last_active":"2023-08-27T11:18:06.796531+0000","last_peered":"2023-08-27T11:18:06.796531+0000","last_clean":"2023-08-27T11:18:06.796531+0000","last_became_active":"2023-08-27T11:18:03.256122+0000","last_became_peered":"2023-08-27T11:18:03.256122+0000","last_unstale":"2023-08-27T11:18:06.796531+0000","last_undegraded":"2023-08-27T11:18:06.796531+0000","last_fullsized":"2023-08-27T11:18:06.796531+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:00:35.000631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797212+0000","last_change":"2023-08-27T11:18:04.265022+0000","last_active":"2023-08-27T11:18:06.797212+0000","last_peered":"2023-08-27T11:18:06.797212+0000","last_clean":"2023-08-27T11:18:06.797212+0000","last_became_active":"2023-08-27T11:18:04.264804+0000","last_became_peered":"2023-08-27T11:18:04.264804+0000","last_unstale":"2023-08-27T11:18:06.797212+0000","last_undegraded":"2023-08-27T11:18:06.797212+0000","last_fullsized":"2023-08-27T11:18:06.797212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:39.229819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084861+0000","last_change":"2023-08-27T11:18:03.260612+0000","last_active":"2023-08-27T11:18:06.084861+0000","last_peered":"2023-08-27T11:18:06.084861+0000","last_clean":"2023-08-27T11:18:06.084861+0000","last_became_active":"2023-08-27T11:18:03.260171+0000","last_became_peered":"2023-08-27T11:18:03.260171+0000","last_unstale":"2023-08-27T11:18:06.084861+0000","last_undegraded":"2023-08-27T11:18:06.084861+0000","last_fullsized":"2023-08-27T11:18:06.084861+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:38.565900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257551+0000","last_change":"2023-08-27T11:18:04.264989+0000","last_active":"2023-08-27T11:18:05.257551+0000","last_peered":"2023-08-27T11:18:05.257551+0000","last_clean":"2023-08-27T11:18:05.257551+0000","last_became_active":"2023-08-27T11:18:04.263821+0000","last_became_peered":"2023-08-27T11:18:04.263821+0000","last_unstale":"2023-08-27T11:18:05.257551+0000","last_undegraded":"2023-08-27T11:18:05.257551+0000","last_fullsized":"2023-08-27T11:18:05.257551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:39:40.502028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"32'3","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084883+0000","last_change":"2023-08-27T11:18:03.259559+0000","last_active":"2023-08-27T11:18:06.084883+0000","last_peered":"2023-08-27T11:18:06.084883+0000","last_clean":"2023-08-27T11:18:06.084883+0000","last_became_active":"2023-08-27T11:18:03.256952+0000","last_became_peered":"2023-08-27T11:18:03.256952+0000","last_unstale":"2023-08-27T11:18:06.084883+0000","last_undegraded":"2023-08-27T11:18:06.084883+0000","last_fullsized":"2023-08-27T11:18:06.084883+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:08:58.446358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257294+0000","last_change":"2023-08-27T11:18:04.256712+0000","last_active":"2023-08-27T11:18:05.257294+0000","last_peered":"2023-08-27T11:18:05.257294+0000","last_clean":"2023-08-27T11:18:05.257294+0000","last_became_active":"2023-08-27T11:18:04.256427+0000","last_became_peered":"2023-08-27T11:18:04.256427+0000","last_unstale":"2023-08-27T11:18:05.257294+0000","last_undegraded":"2023-08-27T11:18:05.257294+0000","last_fullsized":"2023-08-27T11:18:05.257294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:42:59.274843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"32'6","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796298+0000","last_change":"2023-08-27T11:18:03.254733+0000","last_active":"2023-08-27T11:18:06.796298+0000","last_peered":"2023-08-27T11:18:06.796298+0000","last_clean":"2023-08-27T11:18:06.796298+0000","last_became_active":"2023-08-27T11:18:03.254497+0000","last_became_peered":"2023-08-27T11:18:03.254497+0000","last_unstale":"2023-08-27T11:18:06.796298+0000","last_undegraded":"2023-08-27T11:18:06.796298+0000","last_fullsized":"2023-08-27T11:18:06.796298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:41:43.956388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257446+0000","last_change":"2023-08-27T11:18:04.268368+0000","last_active":"2023-08-27T11:18:05.257446+0000","last_peered":"2023-08-27T11:18:05.257446+0000","last_clean":"2023-08-27T11:18:05.257446+0000","last_became_active":"2023-08-27T11:18:04.268082+0000","last_became_peered":"2023-08-27T11:18:04.268082+0000","last_unstale":"2023-08-27T11:18:05.257446+0000","last_undegraded":"2023-08-27T11:18:05.257446+0000","last_fullsized":"2023-08-27T11:18:05.257446+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:20:39.152661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796478+0000","last_change":"2023-08-27T11:18:03.255944+0000","last_active":"2023-08-27T11:18:06.796478+0000","last_peered":"2023-08-27T11:18:06.796478+0000","last_clean":"2023-08-27T11:18:06.796478+0000","last_became_active":"2023-08-27T11:18:03.255681+0000","last_became_peered":"2023-08-27T11:18:03.255681+0000","last_unstale":"2023-08-27T11:18:06.796478+0000","last_undegraded":"2023-08-27T11:18:06.796478+0000","last_fullsized":"2023-08-27T11:18:06.796478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:12:40.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796097+0000","last_change":"2023-08-27T11:18:04.256175+0000","last_active":"2023-08-27T11:18:06.796097+0000","last_peered":"2023-08-27T11:18:06.796097+0000","last_clean":"2023-08-27T11:18:06.796097+0000","last_became_active":"2023-08-27T11:18:04.254794+0000","last_became_peered":"2023-08-27T11:18:04.254794+0000","last_unstale":"2023-08-27T11:18:06.796097+0000","last_undegraded":"2023-08-27T11:18:06.796097+0000","last_fullsized":"2023-08-27T11:18:06.796097+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:00:23.976483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796192+0000","last_change":"2023-08-27T11:18:03.262020+0000","last_active":"2023-08-27T11:18:06.796192+0000","last_peered":"2023-08-27T11:18:06.796192+0000","last_clean":"2023-08-27T11:18:06.796192+0000","last_became_active":"2023-08-27T11:18:03.261770+0000","last_became_peered":"2023-08-27T11:18:03.261770+0000","last_unstale":"2023-08-27T11:18:06.796192+0000","last_undegraded":"2023-08-27T11:18:06.796192+0000","last_fullsized":"2023-08-27T11:18:06.796192+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:04:47.120291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797000+0000","last_change":"2023-08-27T11:18:04.263516+0000","last_active":"2023-08-27T11:18:06.797000+0000","last_peered":"2023-08-27T11:18:06.797000+0000","last_clean":"2023-08-27T11:18:06.797000+0000","last_became_active":"2023-08-27T11:18:04.263215+0000","last_became_peered":"2023-08-27T11:18:04.263215+0000","last_unstale":"2023-08-27T11:18:06.797000+0000","last_undegraded":"2023-08-27T11:18:06.797000+0000","last_fullsized":"2023-08-27T11:18:06.797000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:34.331158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256567+0000","last_change":"2023-08-27T11:18:03.258848+0000","last_active":"2023-08-27T11:18:05.256567+0000","last_peered":"2023-08-27T11:18:05.256567+0000","last_clean":"2023-08-27T11:18:05.256567+0000","last_became_active":"2023-08-27T11:18:03.256331+0000","last_became_peered":"2023-08-27T11:18:03.256331+0000","last_unstale":"2023-08-27T11:18:05.256567+0000","last_undegraded":"2023-08-27T11:18:05.256567+0000","last_fullsized":"2023-08-27T11:18:05.256567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:41.301776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796249+0000","last_change":"2023-08-27T11:18:04.255391+0000","last_active":"2023-08-27T11:18:06.796249+0000","last_peered":"2023-08-27T11:18:06.796249+0000","last_clean":"2023-08-27T11:18:06.796249+0000","last_became_active":"2023-08-27T11:18:04.255076+0000","last_became_peered":"2023-08-27T11:18:04.255076+0000","last_unstale":"2023-08-27T11:18:06.796249+0000","last_undegraded":"2023-08-27T11:18:06.796249+0000","last_fullsized":"2023-08-27T11:18:06.796249+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:38:03.049024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"32'8","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796870+0000","last_change":"2023-08-27T11:18:03.260359+0000","last_active":"2023-08-27T11:18:06.796870+0000","last_peered":"2023-08-27T11:18:06.796870+0000","last_clean":"2023-08-27T11:18:06.796870+0000","last_became_active":"2023-08-27T11:18:03.260073+0000","last_became_peered":"2023-08-27T11:18:03.260073+0000","last_unstale":"2023-08-27T11:18:06.796870+0000","last_undegraded":"2023-08-27T11:18:06.796870+0000","last_fullsized":"2023-08-27T11:18:06.796870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:00.657502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796212+0000","last_change":"2023-08-27T11:18:04.258973+0000","last_active":"2023-08-27T11:18:06.796212+0000","last_peered":"2023-08-27T11:18:06.796212+0000","last_clean":"2023-08-27T11:18:06.796212+0000","last_became_active":"2023-08-27T11:18:04.258629+0000","last_became_peered":"2023-08-27T11:18:04.258629+0000","last_unstale":"2023-08-27T11:18:06.796212+0000","last_undegraded":"2023-08-27T11:18:06.796212+0000","last_fullsized":"2023-08-27T11:18:06.796212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:01.662843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084971+0000","last_change":"2023-08-27T11:18:03.259635+0000","last_active":"2023-08-27T11:18:06.084971+0000","last_peered":"2023-08-27T11:18:06.084971+0000","last_clean":"2023-08-27T11:18:06.084971+0000","last_became_active":"2023-08-27T11:18:03.259370+0000","last_became_peered":"2023-08-27T11:18:03.259370+0000","last_unstale":"2023-08-27T11:18:06.084971+0000","last_undegraded":"2023-08-27T11:18:06.084971+0000","last_fullsized":"2023-08-27T11:18:06.084971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:02:55.604695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084750+0000","last_change":"2023-08-27T11:18:04.272101+0000","last_active":"2023-08-27T11:18:06.084750+0000","last_peered":"2023-08-27T11:18:06.084750+0000","last_clean":"2023-08-27T11:18:06.084750+0000","last_became_active":"2023-08-27T11:18:04.271813+0000","last_became_peered":"2023-08-27T11:18:04.271813+0000","last_unstale":"2023-08-27T11:18:06.084750+0000","last_undegraded":"2023-08-27T11:18:06.084750+0000","last_fullsized":"2023-08-27T11:18:06.084750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:41:02.824652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.258023+0000","last_change":"2023-08-27T11:18:03.260695+0000","last_active":"2023-08-27T11:18:05.258023+0000","last_peered":"2023-08-27T11:18:05.258023+0000","last_clean":"2023-08-27T11:18:05.258023+0000","last_became_active":"2023-08-27T11:18:03.260434+0000","last_became_peered":"2023-08-27T11:18:03.260434+0000","last_unstale":"2023-08-27T11:18:05.258023+0000","last_undegraded":"2023-08-27T11:18:05.258023+0000","last_fullsized":"2023-08-27T11:18:05.258023+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:49.647795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257947+0000","last_change":"2023-08-27T11:18:04.260156+0000","last_active":"2023-08-27T11:18:05.257947+0000","last_peered":"2023-08-27T11:18:05.257947+0000","last_clean":"2023-08-27T11:18:05.257947+0000","last_became_active":"2023-08-27T11:18:04.259770+0000","last_became_peered":"2023-08-27T11:18:04.259770+0000","last_unstale":"2023-08-27T11:18:05.257947+0000","last_undegraded":"2023-08-27T11:18:05.257947+0000","last_fullsized":"2023-08-27T11:18:05.257947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:01.222049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081565+0000","last_change":"2023-08-27T11:18:03.254682+0000","last_active":"2023-08-27T11:18:06.081565+0000","last_peered":"2023-08-27T11:18:06.081565+0000","last_clean":"2023-08-27T11:18:06.081565+0000","last_became_active":"2023-08-27T11:18:03.254368+0000","last_became_peered":"2023-08-27T11:18:03.254368+0000","last_unstale":"2023-08-27T11:18:06.081565+0000","last_undegraded":"2023-08-27T11:18:06.081565+0000","last_fullsized":"2023-08-27T11:18:06.081565+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:38:27.916096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081671+0000","last_change":"2023-08-27T11:18:04.254883+0000","last_active":"2023-08-27T11:18:06.081671+0000","last_peered":"2023-08-27T11:18:06.081671+0000","last_clean":"2023-08-27T11:18:06.081671+0000","last_became_active":"2023-08-27T11:18:04.254616+0000","last_became_peered":"2023-08-27T11:18:04.254616+0000","last_unstale":"2023-08-27T11:18:06.081671+0000","last_undegraded":"2023-08-27T11:18:06.081671+0000","last_fullsized":"2023-08-27T11:18:06.081671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:33:45.827898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796460+0000","last_change":"2023-08-27T11:18:03.251651+0000","last_active":"2023-08-27T11:18:06.796460+0000","last_peered":"2023-08-27T11:18:06.796460+0000","last_clean":"2023-08-27T11:18:06.796460+0000","last_became_active":"2023-08-27T11:18:03.251415+0000","last_became_peered":"2023-08-27T11:18:03.251415+0000","last_unstale":"2023-08-27T11:18:06.796460+0000","last_undegraded":"2023-08-27T11:18:06.796460+0000","last_fullsized":"2023-08-27T11:18:06.796460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:02:13.622820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796771+0000","last_change":"2023-08-27T11:18:04.267587+0000","last_active":"2023-08-27T11:18:06.796771+0000","last_peered":"2023-08-27T11:18:06.796771+0000","last_clean":"2023-08-27T11:18:06.796771+0000","last_became_active":"2023-08-27T11:18:04.267338+0000","last_became_peered":"2023-08-27T11:18:04.267338+0000","last_unstale":"2023-08-27T11:18:06.796771+0000","last_undegraded":"2023-08-27T11:18:06.796771+0000","last_fullsized":"2023-08-27T11:18:06.796771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:00:23.976483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081377+0000","last_change":"2023-08-27T11:18:03.250378+0000","last_active":"2023-08-27T11:18:06.081377+0000","last_peered":"2023-08-27T11:18:06.081377+0000","last_clean":"2023-08-27T11:18:06.081377+0000","last_became_active":"2023-08-27T11:18:03.250134+0000","last_became_peered":"2023-08-27T11:18:03.250134+0000","last_unstale":"2023-08-27T11:18:06.081377+0000","last_undegraded":"2023-08-27T11:18:06.081377+0000","last_fullsized":"2023-08-27T11:18:06.081377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:35:48.840154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257729+0000","last_change":"2023-08-27T11:18:04.264156+0000","last_active":"2023-08-27T11:18:05.257729+0000","last_peered":"2023-08-27T11:18:05.257729+0000","last_clean":"2023-08-27T11:18:05.257729+0000","last_became_active":"2023-08-27T11:18:04.263798+0000","last_became_peered":"2023-08-27T11:18:04.263798+0000","last_unstale":"2023-08-27T11:18:05.257729+0000","last_undegraded":"2023-08-27T11:18:05.257729+0000","last_fullsized":"2023-08-27T11:18:05.257729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:12.709794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081327+0000","last_change":"2023-08-27T11:18:03.250737+0000","last_active":"2023-08-27T11:18:06.081327+0000","last_peered":"2023-08-27T11:18:06.081327+0000","last_clean":"2023-08-27T11:18:06.081327+0000","last_became_active":"2023-08-27T11:18:03.250454+0000","last_became_peered":"2023-08-27T11:18:03.250454+0000","last_unstale":"2023-08-27T11:18:06.081327+0000","last_undegraded":"2023-08-27T11:18:06.081327+0000","last_fullsized":"2023-08-27T11:18:06.081327+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:49:51.997537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084537+0000","last_change":"2023-08-27T11:18:04.259741+0000","last_active":"2023-08-27T11:18:06.084537+0000","last_peered":"2023-08-27T11:18:06.084537+0000","last_clean":"2023-08-27T11:18:06.084537+0000","last_became_active":"2023-08-27T11:18:04.259463+0000","last_became_peered":"2023-08-27T11:18:04.259463+0000","last_unstale":"2023-08-27T11:18:06.084537+0000","last_undegraded":"2023-08-27T11:18:06.084537+0000","last_fullsized":"2023-08-27T11:18:06.084537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:29.089882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796887+0000","last_change":"2023-08-27T11:18:03.261229+0000","last_active":"2023-08-27T11:18:06.796887+0000","last_peered":"2023-08-27T11:18:06.796887+0000","last_clean":"2023-08-27T11:18:06.796887+0000","last_became_active":"2023-08-27T11:18:03.260955+0000","last_became_peered":"2023-08-27T11:18:03.260955+0000","last_unstale":"2023-08-27T11:18:06.796887+0000","last_undegraded":"2023-08-27T11:18:06.796887+0000","last_fullsized":"2023-08-27T11:18:06.796887+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:01.354170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796379+0000","last_change":"2023-08-27T11:18:04.263733+0000","last_active":"2023-08-27T11:18:06.796379+0000","last_peered":"2023-08-27T11:18:06.796379+0000","last_clean":"2023-08-27T11:18:06.796379+0000","last_became_active":"2023-08-27T11:18:04.263379+0000","last_became_peered":"2023-08-27T11:18:04.263379+0000","last_unstale":"2023-08-27T11:18:06.796379+0000","last_undegraded":"2023-08-27T11:18:06.796379+0000","last_fullsized":"2023-08-27T11:18:06.796379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:34.331158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081232+0000","last_change":"2023-08-27T11:18:03.250425+0000","last_active":"2023-08-27T11:18:06.081232+0000","last_peered":"2023-08-27T11:18:06.081232+0000","last_clean":"2023-08-27T11:18:06.081232+0000","last_became_active":"2023-08-27T11:18:03.250161+0000","last_became_peered":"2023-08-27T11:18:03.250161+0000","last_unstale":"2023-08-27T11:18:06.081232+0000","last_undegraded":"2023-08-27T11:18:06.081232+0000","last_fullsized":"2023-08-27T11:18:06.081232+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:46.837627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256716+0000","last_change":"2023-08-27T11:18:04.256639+0000","last_active":"2023-08-27T11:18:05.256716+0000","last_peered":"2023-08-27T11:18:05.256716+0000","last_clean":"2023-08-27T11:18:05.256716+0000","last_became_active":"2023-08-27T11:18:04.256360+0000","last_became_peered":"2023-08-27T11:18:04.256360+0000","last_unstale":"2023-08-27T11:18:05.256716+0000","last_undegraded":"2023-08-27T11:18:05.256716+0000","last_fullsized":"2023-08-27T11:18:05.256716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:53:41.349653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256497+0000","last_change":"2023-08-27T11:18:03.256597+0000","last_active":"2023-08-27T11:18:05.256497+0000","last_peered":"2023-08-27T11:18:05.256497+0000","last_clean":"2023-08-27T11:18:05.256497+0000","last_became_active":"2023-08-27T11:18:03.256301+0000","last_became_peered":"2023-08-27T11:18:03.256301+0000","last_unstale":"2023-08-27T11:18:05.256497+0000","last_undegraded":"2023-08-27T11:18:05.256497+0000","last_fullsized":"2023-08-27T11:18:05.256497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:40.754237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.256830+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:18:05.256830+0000","last_peered":"2023-08-27T11:18:05.256830+0000","last_clean":"2023-08-27T11:18:05.256830+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:18:05.256830+0000","last_undegraded":"2023-08-27T11:18:05.256830+0000","last_fullsized":"2023-08-27T11:18:05.256830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:22:28.764468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797106+0000","last_change":"2023-08-27T11:18:04.266366+0000","last_active":"2023-08-27T11:18:06.797106+0000","last_peered":"2023-08-27T11:18:06.797106+0000","last_clean":"2023-08-27T11:18:06.797106+0000","last_became_active":"2023-08-27T11:18:04.265953+0000","last_became_peered":"2023-08-27T11:18:04.265953+0000","last_unstale":"2023-08-27T11:18:06.797106+0000","last_undegraded":"2023-08-27T11:18:06.797106+0000","last_fullsized":"2023-08-27T11:18:06.797106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:03.267453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.795921+0000","last_change":"2023-08-27T11:18:03.257780+0000","last_active":"2023-08-27T11:18:06.795921+0000","last_peered":"2023-08-27T11:18:06.795921+0000","last_clean":"2023-08-27T11:18:06.795921+0000","last_became_active":"2023-08-27T11:18:03.255666+0000","last_became_peered":"2023-08-27T11:18:03.255666+0000","last_unstale":"2023-08-27T11:18:06.795921+0000","last_undegraded":"2023-08-27T11:18:06.795921+0000","last_fullsized":"2023-08-27T11:18:06.795921+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:04.176421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796319+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:18:06.796319+0000","last_peered":"2023-08-27T11:18:06.796319+0000","last_clean":"2023-08-27T11:18:06.796319+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:18:06.796319+0000","last_undegraded":"2023-08-27T11:18:06.796319+0000","last_fullsized":"2023-08-27T11:18:06.796319+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:57.429873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796501+0000","last_change":"2023-08-27T11:18:04.273615+0000","last_active":"2023-08-27T11:18:06.796501+0000","last_peered":"2023-08-27T11:18:06.796501+0000","last_clean":"2023-08-27T11:18:06.796501+0000","last_became_active":"2023-08-27T11:18:04.273250+0000","last_became_peered":"2023-08-27T11:18:04.273250+0000","last_unstale":"2023-08-27T11:18:06.796501+0000","last_undegraded":"2023-08-27T11:18:06.796501+0000","last_fullsized":"2023-08-27T11:18:06.796501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:04:35.973096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.085014+0000","last_change":"2023-08-27T11:18:03.259045+0000","last_active":"2023-08-27T11:18:06.085014+0000","last_peered":"2023-08-27T11:18:06.085014+0000","last_clean":"2023-08-27T11:18:06.085014+0000","last_became_active":"2023-08-27T11:18:03.256833+0000","last_became_peered":"2023-08-27T11:18:03.256833+0000","last_unstale":"2023-08-27T11:18:06.085014+0000","last_undegraded":"2023-08-27T11:18:06.085014+0000","last_fullsized":"2023-08-27T11:18:06.085014+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:04.176421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796279+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:18:06.796279+0000","last_peered":"2023-08-27T11:18:06.796279+0000","last_clean":"2023-08-27T11:18:06.796279+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:18:06.796279+0000","last_undegraded":"2023-08-27T11:18:06.796279+0000","last_fullsized":"2023-08-27T11:18:06.796279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:06:03.203579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":189,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797321+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:18:06.797321+0000","last_peered":"2023-08-27T11:18:06.797321+0000","last_clean":"2023-08-27T11:18:06.797321+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:18:06.797321+0000","last_undegraded":"2023-08-27T11:18:06.797321+0000","last_fullsized":"2023-08-27T11:18:06.797321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:38:40.551571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797232+0000","last_change":"2023-08-27T11:18:04.265567+0000","last_active":"2023-08-27T11:18:06.797232+0000","last_peered":"2023-08-27T11:18:06.797232+0000","last_clean":"2023-08-27T11:18:06.797232+0000","last_became_active":"2023-08-27T11:18:04.265192+0000","last_became_peered":"2023-08-27T11:18:04.265192+0000","last_unstale":"2023-08-27T11:18:06.797232+0000","last_undegraded":"2023-08-27T11:18:06.797232+0000","last_fullsized":"2023-08-27T11:18:06.797232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:00:36.005972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.081661+0000","last_change":"2023-08-27T11:18:03.263247+0000","last_active":"2023-08-27T11:18:06.081661+0000","last_peered":"2023-08-27T11:18:06.081661+0000","last_clean":"2023-08-27T11:18:06.081661+0000","last_became_active":"2023-08-27T11:18:03.262935+0000","last_became_peered":"2023-08-27T11:18:03.262935+0000","last_unstale":"2023-08-27T11:18:06.081661+0000","last_undegraded":"2023-08-27T11:18:06.081661+0000","last_fullsized":"2023-08-27T11:18:06.081661+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:30:06.873560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796787+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:18:06.796787+0000","last_peered":"2023-08-27T11:18:06.796787+0000","last_clean":"2023-08-27T11:18:06.796787+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:18:06.796787+0000","last_undegraded":"2023-08-27T11:18:06.796787+0000","last_fullsized":"2023-08-27T11:18:06.796787+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:58.119385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084941+0000","last_change":"2023-08-27T11:18:04.269410+0000","last_active":"2023-08-27T11:18:06.084941+0000","last_peered":"2023-08-27T11:18:06.084941+0000","last_clean":"2023-08-27T11:18:06.084941+0000","last_became_active":"2023-08-27T11:18:04.269152+0000","last_became_peered":"2023-08-27T11:18:04.269152+0000","last_unstale":"2023-08-27T11:18:06.084941+0000","last_undegraded":"2023-08-27T11:18:06.084941+0000","last_fullsized":"2023-08-27T11:18:06.084941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:07:21.159766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.797143+0000","last_change":"2023-08-27T11:18:03.261146+0000","last_active":"2023-08-27T11:18:06.797143+0000","last_peered":"2023-08-27T11:18:06.797143+0000","last_clean":"2023-08-27T11:18:06.797143+0000","last_became_active":"2023-08-27T11:18:03.260863+0000","last_became_peered":"2023-08-27T11:18:03.260863+0000","last_unstale":"2023-08-27T11:18:06.797143+0000","last_undegraded":"2023-08-27T11:18:06.797143+0000","last_fullsized":"2023-08-27T11:18:06.797143+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:12:40.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796874+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:18:06.796874+0000","last_peered":"2023-08-27T11:18:06.796874+0000","last_clean":"2023-08-27T11:18:06.796874+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:18:06.796874+0000","last_undegraded":"2023-08-27T11:18:06.796874+0000","last_fullsized":"2023-08-27T11:18:06.796874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:40:40.032091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796452+0000","last_change":"2023-08-27T11:18:04.263009+0000","last_active":"2023-08-27T11:18:06.796452+0000","last_peered":"2023-08-27T11:18:06.796452+0000","last_clean":"2023-08-27T11:18:06.796452+0000","last_became_active":"2023-08-27T11:18:04.262652+0000","last_became_peered":"2023-08-27T11:18:04.262652+0000","last_unstale":"2023-08-27T11:18:06.796452+0000","last_undegraded":"2023-08-27T11:18:06.796452+0000","last_fullsized":"2023-08-27T11:18:06.796452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:03.267453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257731+0000","last_change":"2023-08-27T11:18:03.252729+0000","last_active":"2023-08-27T11:18:05.257731+0000","last_peered":"2023-08-27T11:18:05.257731+0000","last_clean":"2023-08-27T11:18:05.257731+0000","last_became_active":"2023-08-27T11:18:03.252459+0000","last_became_peered":"2023-08-27T11:18:03.252459+0000","last_unstale":"2023-08-27T11:18:05.257731+0000","last_undegraded":"2023-08-27T11:18:05.257731+0000","last_fullsized":"2023-08-27T11:18:05.257731+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:05:20.307605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.084518+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:18:06.084518+0000","last_peered":"2023-08-27T11:18:06.084518+0000","last_clean":"2023-08-27T11:18:06.084518+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:18:06.084518+0000","last_undegraded":"2023-08-27T11:18:06.084518+0000","last_fullsized":"2023-08-27T11:18:06.084518+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:04:16.383307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:06.796187+0000","last_change":"2023-08-27T11:18:04.264759+0000","last_active":"2023-08-27T11:18:06.796187+0000","last_peered":"2023-08-27T11:18:06.796187+0000","last_clean":"2023-08-27T11:18:06.796187+0000","last_became_active":"2023-08-27T11:18:04.264416+0000","last_became_peered":"2023-08-27T11:18:04.264416+0000","last_unstale":"2023-08-27T11:18:06.796187+0000","last_undegraded":"2023-08-27T11:18:06.796187+0000","last_fullsized":"2023-08-27T11:18:06.796187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:38:34.516670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257453+0000","last_change":"2023-08-27T11:18:03.255301+0000","last_active":"2023-08-27T11:18:05.257453+0000","last_peered":"2023-08-27T11:18:05.257453+0000","last_clean":"2023-08-27T11:18:05.257453+0000","last_became_active":"2023-08-27T11:18:03.255034+0000","last_became_peered":"2023-08-27T11:18:03.255034+0000","last_unstale":"2023-08-27T11:18:05.257453+0000","last_undegraded":"2023-08-27T11:18:05.257453+0000","last_fullsized":"2023-08-27T11:18:05.257453+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:20:38.147319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257052+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:18:05.257052+0000","last_peered":"2023-08-27T11:18:05.257052+0000","last_clean":"2023-08-27T11:18:05.257052+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:18:05.257052+0000","last_undegraded":"2023-08-27T11:18:05.257052+0000","last_fullsized":"2023-08-27T11:18:05.257052+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:37:29.587031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257388+0000","last_change":"2023-08-27T11:18:04.254633+0000","last_active":"2023-08-27T11:18:05.257388+0000","last_peered":"2023-08-27T11:18:05.257388+0000","last_clean":"2023-08-27T11:18:05.257388+0000","last_became_active":"2023-08-27T11:18:04.254350+0000","last_became_peered":"2023-08-27T11:18:04.254350+0000","last_unstale":"2023-08-27T11:18:05.257388+0000","last_undegraded":"2023-08-27T11:18:05.257388+0000","last_fullsized":"2023-08-27T11:18:05.257388+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:03.234556+0000","last_clean_scrub_stamp":"2023-08-27T11:18:03.234556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:04:35.973096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257722+0000","last_change":"2023-08-27T11:18:03.256558+0000","last_active":"2023-08-27T11:18:05.257722+0000","last_peered":"2023-08-27T11:18:05.257722+0000","last_clean":"2023-08-27T11:18:05.257722+0000","last_became_active":"2023-08-27T11:18:03.256290+0000","last_became_peered":"2023-08-27T11:18:03.256290+0000","last_unstale":"2023-08-27T11:18:05.257722+0000","last_undegraded":"2023-08-27T11:18:05.257722+0000","last_fullsized":"2023-08-27T11:18:05.257722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:02.229215+0000","last_clean_scrub_stamp":"2023-08-27T11:18:02.229215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:16:31.256534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-08-27T11:18:05.257349+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:18:05.257349+0000","last_peered":"2023-08-27T11:18:05.257349+0000","last_clean":"2023-08-27T11:18:05.257349+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:18:05.257349+0000","last_undegraded":"2023-08-27T11:18:05.257349+0000","last_fullsized":"2023-08-27T11:18:05.257349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:09.698522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":739236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.502}]},{"osd":1,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.45900000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.54500000000000004}]},{"osd":3,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.433}]},{"osd":4,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.56299999999999994}]},{"osd":5,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.67900000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.621}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":154421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.443},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.51600000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58299999999999996}]},{"osd":2,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.626}]},{"osd":3,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54700000000000004}]},{"osd":4,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.5},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.61299999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.66600000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.65200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":958464,"data_stored":737644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":11.971,"5min":11.971,"15min":11.971},"last":0.34100000000000003},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":23.045999999999999,"5min":23.045999999999999,"15min":23.045999999999999},"last":0.254}]},{"osd":2,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.928000000000001,"5min":11.928000000000001,"15min":11.928000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.371}]},{"osd":3,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":10.99,"5min":10.99,"15min":10.99},"last":0.31900000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":22.893999999999998,"5min":22.893999999999998,"15min":22.893999999999998},"last":0.28699999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":10.957000000000001,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.40899999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.75,"15min":1.75},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":23.016999999999999,"5min":23.016999999999999,"15min":23.016999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.41999999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":10.925000000000001,"5min":10.925000000000001,"15min":10.925000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":22.952000000000002,"5min":22.952000000000002,"15min":22.952000000000002},"last":0.57699999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":22.988,"5min":22.988,"15min":22.988},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.46400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":152512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.28299999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.32500000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.308},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.42399999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.63},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.69799999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.77500000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.78800000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.80400000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":153909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":0.29099999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.255}]},{"osd":3,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.36399999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.64600000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52400000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.495},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56399999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.63100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":148832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.224},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.248}]},{"osd":1,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.28599999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.26500000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42199999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.377},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45800000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.40899999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.48999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":154443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.495}]},{"osd":1,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.57999999999999996}]},{"osd":2,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.54500000000000004}]},{"osd":3,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.56200000000000006}]},{"osd":5,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.375},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.52900000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47599999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.50700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":153951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45200000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.53900000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.56799999999999995}]},{"osd":3,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.59699999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.627},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.745}]},{"osd":6,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:18:08.731 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:18:08.731 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:18:08.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.891+0000 7f9fb75b4700 1 Processor -- start 2023-08-27T11:18:08.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.891+0000 7f9fb75b4700 1 -- start start 2023-08-27T11:18:08.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.892+0000 7f9fb75b4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fb0146c20 0x7f9fb0160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.892+0000 7f9fb75b4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb00b5070 con 0x7f9fb006f790 2023-08-27T11:18:08.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.892+0000 7f9fb75b4700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb0147000 con 0x7f9fb00b3440 2023-08-27T11:18:08.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.892+0000 7f9fb75b4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb01614a0 con 0x7f9fb0146c20 2023-08-27T11:18:08.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.892+0000 7f9fb5b51700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9fb006f790 0x7f9fb00b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51268/0 (socket says 172.21.15.112:51268) 2023-08-27T11:18:08.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.892+0000 7f9fb5b51700 1 -- 172.21.15.112:0/2972994586 learned_addr learned my addr 172.21.15.112:0/2972994586 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:08.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.892+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249698995 0 0) 0x7f9fa8001490 con 0x7f9fb006f790 2023-08-27T11:18:08.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f9c0035b0 con 0x7f9fb006f790 2023-08-27T11:18:08.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fb5350700 1 --2- 172.21.15.112:0/2972994586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fb0146c20 0x7f9fb0160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 495407308 0 0) 0x7f9f9c0035b0 con 0x7f9fb006f790 2023-08-27T11:18:08.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fb0146c20 msgr2=0x7f9fb0160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:08.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 --2- 172.21.15.112:0/2972994586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fb0146c20 0x7f9fb0160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 >> v1:172.21.15.138:6789/0 conn(0x7f9fb00b3440 legacy=0x7f9fb015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb0161740 con 0x7f9fb006f790 2023-08-27T11:18:08.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f9fa8001b80 con 0x7f9fb006f790 2023-08-27T11:18:08.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 >> v1:172.21.15.112:6789/0 conn(0x7f9fb006f790 legacy=0x7f9fb00b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 --2- 172.21.15.112:0/2972994586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9c003a30 0x7f9f9c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 --2- 172.21.15.112:0/2972994586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fb0146c20 0x7f9f9c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 --2- 172.21.15.112:0/2972994586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f9c004d90 0x7f9f9c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb01614a0 con 0x7f9f9c004d90 2023-08-27T11:18:08.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c00bb40 con 0x7f9f9c003a30 2023-08-27T11:18:08.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c00bd00 con 0x7f9fb0146c20 2023-08-27T11:18:08.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fb5b51700 1 --2- 172.21.15.112:0/2972994586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f9c004d90 0x7f9f9c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fb4b4f700 1 --2- 172.21.15.112:0/2972994586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fb0146c20 0x7f9f9c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fb5350700 1 --2- 172.21.15.112:0/2972994586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9c003a30 0x7f9f9c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fb5b51700 1 -- 172.21.15.112:0/2972994586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fb0146c20 msgr2=0x7f9f9c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:08.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fb5b51700 1 --2- 172.21.15.112:0/2972994586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fb0146c20 0x7f9f9c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fb5b51700 1 -- 172.21.15.112:0/2972994586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9c003a30 msgr2=0x7f9f9c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:08.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fb5b51700 1 --2- 172.21.15.112:0/2972994586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9c003a30 0x7f9f9c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fb5b51700 1 -- 172.21.15.112:0/2972994586 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c0035b0 con 0x7f9f9c004d90 2023-08-27T11:18:08.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fb5b51700 1 --2- 172.21.15.112:0/2972994586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f9c004d90 0x7f9f9c005210 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800a830 tx=0x7f9f9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efa60be53e23677 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:08.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.893+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa800a490 con 0x7f9f9c004d90 2023-08-27T11:18:08.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.894+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa8012600 con 0x7f9f9c004d90 2023-08-27T11:18:08.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.894+0000 7f9fa7fff700 1 -- 172.21.15.112:0/2972994586 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8012990 con 0x7f9f9c004d90 2023-08-27T11:18:08.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.894+0000 7f9fb75b4700 1 -- 172.21.15.112:0/2972994586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f9c004d90 msgr2=0x7f9f9c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.894+0000 7f9fb75b4700 1 --2- 172.21.15.112:0/2972994586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f9c004d90 0x7f9f9c005210 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800a830 tx=0x7f9f9c005880 comp rx=0 tx=0).stop 2023-08-27T11:18:08.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.894+0000 7f9fb5b51700 1 -- 172.21.15.112:0/2972994586 reap_dead start 2023-08-27T11:18:08.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.894+0000 7f9fb75b4700 1 -- 172.21.15.112:0/2972994586 shutdown_connections 2023-08-27T11:18:08.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.894+0000 7f9fb75b4700 1 -- 172.21.15.112:0/2972994586 >> 172.21.15.112:0/2972994586 conn(0x7f9fb00b1e90 msgr2=0x7f9fb00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:08.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.894+0000 7f9fb75b4700 1 -- 172.21.15.112:0/2972994586 shutdown_connections 2023-08-27T11:18:08.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.894+0000 7f9fb75b4700 1 -- 172.21.15.112:0/2972994586 wait complete. 2023-08-27T11:18:08.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.895+0000 7f9fb75b4700 1 Processor -- start 2023-08-27T11:18:08.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.895+0000 7f9fb75b4700 1 -- start start 2023-08-27T11:18:08.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb75b4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9c004d90 0x7f9fb0140770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb75b4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fb01f6bb0 0x7f9fb0140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb75b4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fb01414e0 0x7f9fb0071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb75b4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb0141cb0 con 0x7f9fb01414e0 2023-08-27T11:18:08.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb75b4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb01f5ec0 con 0x7f9f9c004d90 2023-08-27T11:18:08.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb75b4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb0140cb0 con 0x7f9fb01f6bb0 2023-08-27T11:18:08.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb5b51700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fb01414e0 0x7f9fb0071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb5b51700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fb01414e0 0x7f9fb0071a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56898/0 (socket says 172.21.15.112:56898) 2023-08-27T11:18:08.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb5b51700 1 -- 172.21.15.112:0/3804918954 learned_addr learned my addr 172.21.15.112:0/3804918954 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:08.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb5350700 1 --2- 172.21.15.112:0/3804918954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9c004d90 0x7f9fb0140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb5b51700 1 -- 172.21.15.112:0/3804918954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fb01f6bb0 msgr2=0x7f9fb0140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb5b51700 1 --2- 172.21.15.112:0/3804918954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fb01f6bb0 0x7f9fb0140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb5b51700 1 -- 172.21.15.112:0/3804918954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9c004d90 msgr2=0x7f9fb0140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:08.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb5b51700 1 --2- 172.21.15.112:0/3804918954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9c004d90 0x7f9fb0140770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:08.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.896+0000 7f9fb5b51700 1 -- 172.21.15.112:0/3804918954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb01f6720 con 0x7f9fb01414e0 2023-08-27T11:18:08.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.897+0000 7f9fb5b51700 1 --2- 172.21.15.112:0/3804918954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fb01414e0 0x7f9fb0071a30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8012000 tx=0x7f9fa8001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:08.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.897+0000 7f9fa5ffb700 1 -- 172.21.15.112:0/3804918954 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8001490 con 0x7f9fb01414e0 2023-08-27T11:18:08.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.897+0000 7f9fa5ffb700 1 -- 172.21.15.112:0/3804918954 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa8001b50 con 0x7f9fb01414e0 2023-08-27T11:18:08.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.897+0000 7f9fb75b4700 1 -- 172.21.15.112:0/3804918954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb01419d0 con 0x7f9fb01414e0 2023-08-27T11:18:08.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.897+0000 7f9fa5ffb700 1 -- 172.21.15.112:0/3804918954 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa801e430 con 0x7f9fb01414e0 2023-08-27T11:18:08.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.897+0000 7f9fb75b4700 1 -- 172.21.15.112:0/3804918954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb01f63b0 con 0x7f9fb01414e0 2023-08-27T11:18:08.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.898+0000 7f9fb75b4700 1 -- 172.21.15.112:0/3804918954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb01f68b0 con 0x7f9fb01414e0 2023-08-27T11:18:08.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.898+0000 7f9fa5ffb700 1 -- 172.21.15.112:0/3804918954 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9fa801e590 con 0x7f9fb01414e0 2023-08-27T11:18:08.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.899+0000 7f9fa5ffb700 1 --2- 172.21.15.112:0/3804918954 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9f9406a0e0 0x7f9f9406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:08.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.899+0000 7f9fa5ffb700 1 -- 172.21.15.112:0/3804918954 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9fa808a2b0 con 0x7f9fb01414e0 2023-08-27T11:18:08.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.899+0000 7f9fb5350700 1 --2- 172.21.15.112:0/3804918954 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9f9406a0e0 0x7f9f9406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:08.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.899+0000 7f9fb5350700 1 --2- 172.21.15.112:0/3804918954 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9f9406a0e0 0x7f9f9406c5a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00099f0 tx=0x7f9fa0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:08.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:08.904+0000 7f9fa5ffb700 1 -- 172.21.15.112:0/3804918954 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9fa8055b50 con 0x7f9fb01414e0 2023-08-27T11:18:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.166+0000 7f9fb75b4700 1 -- 172.21.15.112:0/3804918954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9fb0067e90 con 0x7f9fb01414e0 2023-08-27T11:18:09.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.170+0000 7f9fa5ffb700 1 -- 172.21.15.112:0/3804918954 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f9fa8059170 con 0x7f9fb01414e0 2023-08-27T11:18:09.172 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-08-27T11:18:09.172 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA== 2023-08-27T11:18:09.172 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2023-08-27T11:18:09.172 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2023-08-27T11:18:09.172 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:18:09.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.173+0000 7f9fb75b4700 1 -- 172.21.15.112:0/3804918954 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9f9406a0e0 msgr2=0x7f9f9406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.173+0000 7f9fb75b4700 1 --2- 172.21.15.112:0/3804918954 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9f9406a0e0 0x7f9f9406c5a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00099f0 tx=0x7f9fa0008040 comp rx=0 tx=0).stop 2023-08-27T11:18:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.173+0000 7f9fb75b4700 1 -- 172.21.15.112:0/3804918954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fb01414e0 msgr2=0x7f9fb0071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.173+0000 7f9fb75b4700 1 --2- 172.21.15.112:0/3804918954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fb01414e0 0x7f9fb0071a30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8012000 tx=0x7f9fa8001f10 comp rx=0 tx=0).stop 2023-08-27T11:18:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.173+0000 7f9fb75b4700 1 -- 172.21.15.112:0/3804918954 shutdown_connections 2023-08-27T11:18:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.173+0000 7f9fb75b4700 1 --2- 172.21.15.112:0/3804918954 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9f9406a0e0 0x7f9f9406c5a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.173+0000 7f9fb75b4700 1 --2- 172.21.15.112:0/3804918954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fb01414e0 0x7f9fb0071a30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.173+0000 7f9fb75b4700 1 --2- 172.21.15.112:0/3804918954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9c004d90 0x7f9fb0140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.173+0000 7f9fb75b4700 1 --2- 172.21.15.112:0/3804918954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fb01f6bb0 0x7f9fb0140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.173+0000 7f9fb75b4700 1 -- 172.21.15.112:0/3804918954 >> 172.21.15.112:0/3804918954 conn(0x7f9fb00b1e90 msgr2=0x7f9fb00b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.173+0000 7f9fb75b4700 1 -- 172.21.15.112:0/3804918954 shutdown_connections 2023-08-27T11:18:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.173+0000 7f9fb75b4700 1 -- 172.21.15.112:0/3804918954 wait complete. 2023-08-27T11:18:09.178 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2023-08-27T11:18:09.186 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:18:09.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18d4a97700 1 Processor -- start 2023-08-27T11:18:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18d4a97700 1 -- start start 2023-08-27T11:18:09.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18d4a97700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f18d0143190 0x7f18d00ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:09.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18d4a97700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00cf1e0 0x7f18d006e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:09.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18d4a97700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d00a9b70 con 0x7f18d006e690 2023-08-27T11:18:09.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18d4a97700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d006eb20 con 0x7f18d0143190 2023-08-27T11:18:09.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18d4a97700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00cc200 con 0x7f18d00cf1e0 2023-08-27T11:18:09.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18ced9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f18d006e690 0x7f18d00cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51286/0 (socket says 172.21.15.112:51286) 2023-08-27T11:18:09.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18ced9d700 1 -- 172.21.15.112:0/3902496671 learned_addr learned my addr 172.21.15.112:0/3902496671 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:09.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074427916 0 0) 0x7f18d00a9b70 con 0x7f18d006e690 2023-08-27T11:18:09.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18b80035b0 con 0x7f18d006e690 2023-08-27T11:18:09.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18ce59c700 1 --2- 172.21.15.112:0/3902496671 >> v2:172.21.15.138:3300/0 conn(0x7f18d0143190 0x7f18d00ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:09.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18cdd9b700 1 --2- 172.21.15.112:0/3902496671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00cf1e0 0x7f18d006e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:09.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1063657225 0 0) 0x7f18b80035b0 con 0x7f18d006e690 2023-08-27T11:18:09.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00cf1e0 msgr2=0x7f18d006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:09.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18cd59a700 1 --2- 172.21.15.112:0/3902496671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00cf1e0 0x7f18d006e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 >> v2:172.21.15.138:3300/0 conn(0x7f18d0143190 msgr2=0x7f18d00ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:09.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18cd59a700 1 --2- 172.21.15.112:0/3902496671 >> v2:172.21.15.138:3300/0 conn(0x7f18d0143190 0x7f18d00ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d00cc4e0 con 0x7f18d006e690 2023-08-27T11:18:09.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f18c80019f0 con 0x7f18d006e690 2023-08-27T11:18:09.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.306+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 >> v1:172.21.15.112:6789/0 conn(0x7f18d006e690 legacy=0x7f18d00cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:09.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18cd59a700 1 --2- 172.21.15.112:0/3902496671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8003a00 0x7f18b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:09.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18cd59a700 1 --2- 172.21.15.112:0/3902496671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18b8004850 0x7f18b8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:09.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18cd59a700 1 --2- 172.21.15.112:0/3902496671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00cf1e0 0x7f18b8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:09.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d006eb20 con 0x7f18b8003a00 2023-08-27T11:18:09.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00cc200 con 0x7f18b8004850 2023-08-27T11:18:09.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b800ba10 con 0x7f18d00cf1e0 2023-08-27T11:18:09.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18ce59c700 1 --2- 172.21.15.112:0/3902496671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8003a00 0x7f18b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:09.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18cdd9b700 1 --2- 172.21.15.112:0/3902496671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18b8004850 0x7f18b8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:09.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18ced9d700 1 --2- 172.21.15.112:0/3902496671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00cf1e0 0x7f18b8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:09.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18ce59c700 1 -- 172.21.15.112:0/3902496671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00cf1e0 msgr2=0x7f18b8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:09.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18ce59c700 1 --2- 172.21.15.112:0/3902496671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00cf1e0 0x7f18b8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18ce59c700 1 -- 172.21.15.112:0/3902496671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18b8004850 msgr2=0x7f18b8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:09.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18ce59c700 1 --2- 172.21.15.112:0/3902496671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18b8004850 0x7f18b8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18ce59c700 1 -- 172.21.15.112:0/3902496671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b80035b0 con 0x7f18b8003a00 2023-08-27T11:18:09.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18ce59c700 1 --2- 172.21.15.112:0/3902496671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8003a00 0x7f18b8003e20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009b60 tx=0x7f18b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f930730fd2fe1db server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:09.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000e6c0 con 0x7f18b8003a00 2023-08-27T11:18:09.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c000e820 con 0x7f18b8003a00 2023-08-27T11:18:09.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.307+0000 7f18cd59a700 1 -- 172.21.15.112:0/3902496671 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000eb90 con 0x7f18b8003a00 2023-08-27T11:18:09.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.308+0000 7f18d4a97700 1 -- 172.21.15.112:0/3902496671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8003a00 msgr2=0x7f18b8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:09.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.308+0000 7f18d4a97700 1 --2- 172.21.15.112:0/3902496671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8003a00 0x7f18b8003e20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009b60 tx=0x7f18b8005880 comp rx=0 tx=0).stop 2023-08-27T11:18:09.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.308+0000 7f18ced9d700 1 -- 172.21.15.112:0/3902496671 reap_dead start 2023-08-27T11:18:09.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.308+0000 7f18d4a97700 1 -- 172.21.15.112:0/3902496671 shutdown_connections 2023-08-27T11:18:09.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.308+0000 7f18d4a97700 1 -- 172.21.15.112:0/3902496671 >> 172.21.15.112:0/3902496671 conn(0x7f18d006cd30 msgr2=0x7f18d00717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:09.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.308+0000 7f18d4a97700 1 -- 172.21.15.112:0/3902496671 shutdown_connections 2023-08-27T11:18:09.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.308+0000 7f18d4a97700 1 -- 172.21.15.112:0/3902496671 wait complete. 2023-08-27T11:18:09.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.309+0000 7f18d4a97700 1 Processor -- start 2023-08-27T11:18:09.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.309+0000 7f18d4a97700 1 -- start start 2023-08-27T11:18:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18d4a97700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8003a00 0x7f18d013d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18d4a97700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d0152980 0x7f18d0153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:09.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18d4a97700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d006dae0 0x7f18d006df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:09.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18d4a97700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d006e4b0 con 0x7f18b8003a00 2023-08-27T11:18:09.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18d4a97700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d01530f0 con 0x7f18d006dae0 2023-08-27T11:18:09.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18d4a97700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d0152e10 con 0x7f18d0152980 2023-08-27T11:18:09.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18ce59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8003a00 0x7f18d013d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:09.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18ce59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8003a00 0x7f18d013d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56912/0 (socket says 172.21.15.112:56912) 2023-08-27T11:18:09.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18ce59c700 1 -- 172.21.15.112:0/786654255 learned_addr learned my addr 172.21.15.112:0/786654255 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:09.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18ced9d700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d006dae0 0x7f18d006df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:09.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18cdd9b700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d0152980 0x7f18d0153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:09.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18ce59c700 1 -- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d0152980 msgr2=0x7f18d0153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:09.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18ce59c700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d0152980 0x7f18d0153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18ce59c700 1 -- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d006dae0 msgr2=0x7f18d006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:09.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18ce59c700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d006dae0 0x7f18d006df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18ce59c700 1 -- 172.21.15.112:0/786654255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d01595b0 con 0x7f18b8003a00 2023-08-27T11:18:09.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.310+0000 7f18ce59c700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8003a00 0x7f18d013d9e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f18c00090b0 tx=0x7f18c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:09.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.311+0000 7f18beffd700 1 -- 172.21.15.112:0/786654255 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000e6c0 con 0x7f18b8003a00 2023-08-27T11:18:09.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.311+0000 7f18beffd700 1 -- 172.21.15.112:0/786654255 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c0003ad0 con 0x7f18b8003a00 2023-08-27T11:18:09.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.311+0000 7f18d4a97700 1 -- 172.21.15.112:0/786654255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d0159890 con 0x7f18b8003a00 2023-08-27T11:18:09.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.311+0000 7f18beffd700 1 -- 172.21.15.112:0/786654255 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0003e80 con 0x7f18b8003a00 2023-08-27T11:18:09.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.311+0000 7f18d4a97700 1 -- 172.21.15.112:0/786654255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d013df20 con 0x7f18b8003a00 2023-08-27T11:18:09.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.312+0000 7f18d4a97700 1 -- 172.21.15.112:0/786654255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d0159db0 con 0x7f18b8003a00 2023-08-27T11:18:09.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.312+0000 7f18beffd700 1 -- 172.21.15.112:0/786654255 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f18c0013470 con 0x7f18b8003a00 2023-08-27T11:18:09.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.312+0000 7f18beffd700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18b006a0e0 0x7f18b006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:09.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.312+0000 7f18beffd700 1 -- 172.21.15.112:0/786654255 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f18c0089d80 con 0x7f18b8003a00 2023-08-27T11:18:09.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.313+0000 7f18cdd9b700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18b006a0e0 0x7f18b006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:09.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.313+0000 7f18cdd9b700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18b006a0e0 0x7f18b006c5a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18d015b790 tx=0x7f18c40091c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:09.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.317+0000 7f18beffd700 1 -- 172.21.15.112:0/786654255 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f18d0159db0 con 0x7f18b8003a00 2023-08-27T11:18:09.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.546+0000 7f18d4a97700 1 -- 172.21.15.112:0/786654255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18d0159db0 con 0x7f18b8003a00 2023-08-27T11:18:09.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.547+0000 7f18beffd700 1 -- 172.21.15.112:0/786654255 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4887 (secure 0 0 0) 0x7f18d0159db0 con 0x7f18b8003a00 2023-08-27T11:18:09.549 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:09.550 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1941625698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1941625698},{"type":"v1","addr":"172.21.15.138:6836","nonce":1941625698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1530886230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1530886230},{"type":"v1","addr":"172.21.15.112:6834","nonce":1530886230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1024768929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1024768929},{"type":"v1","addr":"172.21.15.112:6835","nonce":1024768929}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:18:03.637863+0000","modified":"2023-08-27T11:18:07.262281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5175":{"gid":5175,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6837/3409848660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3409848660},{"type":"v1","addr":"172.21.15.138:6837","nonce":3409848660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-27T11:18:09.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.549+0000 7f18d4a97700 1 -- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18b006a0e0 msgr2=0x7f18b006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.549+0000 7f18d4a97700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18b006a0e0 0x7f18b006c5a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18d015b790 tx=0x7f18c40091c0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.549+0000 7f18d4a97700 1 -- 172.21.15.112:0/786654255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8003a00 msgr2=0x7f18d013d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.549+0000 7f18d4a97700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8003a00 0x7f18d013d9e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f18c00090b0 tx=0x7f18c00047c0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.549+0000 7f18d4a97700 1 -- 172.21.15.112:0/786654255 shutdown_connections 2023-08-27T11:18:09.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.549+0000 7f18d4a97700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18b006a0e0 0x7f18b006c5a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.549+0000 7f18d4a97700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d006dae0 0x7f18d006df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.549+0000 7f18d4a97700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8003a00 0x7f18d013d9e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.549+0000 7f18d4a97700 1 --2- 172.21.15.112:0/786654255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d0152980 0x7f18d0153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:09.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.549+0000 7f18d4a97700 1 -- 172.21.15.112:0/786654255 >> 172.21.15.112:0/786654255 conn(0x7f18d006cd30 msgr2=0x7f18d013f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:09.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.549+0000 7f18d4a97700 1 -- 172.21.15.112:0/786654255 shutdown_connections 2023-08-27T11:18:09.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:09.549+0000 7f18d4a97700 1 -- 172.21.15.112:0/786654255 wait complete. 2023-08-27T11:18:09.555 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 15 2023-08-27T11:18:09.965 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:18:03.637863+0000', 'modified': '2023-08-27T11:18:07.262281+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5175}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5175': {'gid': 5175, 'name': 'b', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.138:6837/3409848660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6835', 'nonce': 3409848660}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 3409848660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:18:09.966 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:18:09.966 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-27T11:18:10.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.090+0000 7fd1f7603700 1 Processor -- start 2023-08-27T11:18:10.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.090+0000 7fd1f7603700 1 -- start start 2023-08-27T11:18:10.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.090+0000 7fd1f7603700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1f0151060 0x7fd1f015e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1f7603700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f00b02f0 con 0x7fd1f015e990 2023-08-27T11:18:10.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1f7603700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f00b05d0 con 0x7fd1f006b4c0 2023-08-27T11:18:10.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1f7603700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f00b08b0 con 0x7fd1f0151060 2023-08-27T11:18:10.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1f4b9e700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd1f015e990 0x7fd1f006adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51302/0 (socket says 172.21.15.112:51302) 2023-08-27T11:18:10.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/2281826775 learned_addr learned my addr 172.21.15.112:0/2281826775 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:10.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1f539f700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1f0151060 0x7fd1f015e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791921769 0 0) 0x7fd1f00b02f0 con 0x7fd1f015e990 2023-08-27T11:18:10.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1dc0035b0 con 0x7fd1f015e990 2023-08-27T11:18:10.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 838835998 0 0) 0x7fd1dc0035b0 con 0x7fd1f015e990 2023-08-27T11:18:10.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1f0151060 msgr2=0x7fd1f015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:10.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1f0151060 0x7fd1f015e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 >> v1:172.21.15.138:6789/0 conn(0x7fd1f006b4c0 legacy=0x7fd1f00cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f00b0b90 con 0x7fd1f015e990 2023-08-27T11:18:10.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fd1ec001670 con 0x7fd1f015e990 2023-08-27T11:18:10.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1f539f700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1f0151060 0x7fd1f015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:10.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 >> v1:172.21.15.112:6789/0 conn(0x7fd1f015e990 legacy=0x7fd1f006adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1dc003a30 0x7fd1dc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1dc0043e0 0x7fd1dc0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1f0151060 0x7fd1dc0050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f00b08b0 con 0x7fd1dc0043e0 2023-08-27T11:18:10.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc00ba10 con 0x7fd1dc003a30 2023-08-27T11:18:10.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc00bb80 con 0x7fd1f0151060 2023-08-27T11:18:10.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1dc0043e0 0x7fd1dc0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1f539f700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1dc003a30 0x7fd1dc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.091+0000 7fd1f5ba0700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1f0151060 0x7fd1dc0050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/2281826775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1f0151060 msgr2=0x7fd1dc0050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:10.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1f0151060 0x7fd1dc0050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/2281826775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1dc003a30 msgr2=0x7fd1dc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:10.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1dc003a30 0x7fd1dc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/2281826775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e00094a0 con 0x7fd1dc0043e0 2023-08-27T11:18:10.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f5ba0700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1f0151060 0x7fd1dc0050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:10.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1dc0043e0 0x7fd1dc0048b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c240 tx=0x7fd1dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c650ae0c9c654c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:10.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec00cd80 con 0x7fd1dc0043e0 2023-08-27T11:18:10.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec005960 con 0x7fd1dc0043e0 2023-08-27T11:18:10.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1e7fff700 1 -- 172.21.15.112:0/2281826775 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec005cf0 con 0x7fd1dc0043e0 2023-08-27T11:18:10.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f7603700 1 -- 172.21.15.112:0/2281826775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1dc0043e0 msgr2=0x7fd1dc0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f7603700 1 --2- 172.21.15.112:0/2281826775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1dc0043e0 0x7fd1dc0048b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c240 tx=0x7fd1dc005880 comp rx=0 tx=0).stop 2023-08-27T11:18:10.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f5ba0700 1 -- 172.21.15.112:0/2281826775 reap_dead start 2023-08-27T11:18:10.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f7603700 1 -- 172.21.15.112:0/2281826775 shutdown_connections 2023-08-27T11:18:10.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f7603700 1 -- 172.21.15.112:0/2281826775 >> 172.21.15.112:0/2281826775 conn(0x7fd1f00d0000 msgr2=0x7fd1f00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:10.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f7603700 1 -- 172.21.15.112:0/2281826775 shutdown_connections 2023-08-27T11:18:10.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.092+0000 7fd1f7603700 1 -- 172.21.15.112:0/2281826775 wait complete. 2023-08-27T11:18:10.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.094+0000 7fd1f7603700 1 Processor -- start 2023-08-27T11:18:10.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.094+0000 7fd1f7603700 1 -- start start 2023-08-27T11:18:10.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.094+0000 7fd1f7603700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1dc0043e0 0x7fd1f01453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.094+0000 7fd1f7603700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f00c6720 0x7fd1f00c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.094+0000 7fd1f7603700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f00c5170 0x7fd1f00cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.094+0000 7fd1f7603700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0145930 con 0x7fd1f00c6720 2023-08-27T11:18:10.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.094+0000 7fd1f7603700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f00c5680 con 0x7fd1f00c5170 2023-08-27T11:18:10.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.094+0000 7fd1f7603700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f00c5960 con 0x7fd1dc0043e0 2023-08-27T11:18:10.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.094+0000 7fd1f4b9e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f00c6720 0x7fd1f00c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.094+0000 7fd1f4b9e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f00c6720 0x7fd1f00c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56934/0 (socket says 172.21.15.112:56934) 2023-08-27T11:18:10.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.094+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/393094213 learned_addr learned my addr 172.21.15.112:0/393094213 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:10.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.095+0000 7fd1f539f700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1dc0043e0 0x7fd1f01453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.095+0000 7fd1f5ba0700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f00c5170 0x7fd1f00cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.095+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1dc0043e0 msgr2=0x7fd1f01453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.095+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1dc0043e0 0x7fd1f01453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.095+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f00c5170 msgr2=0x7fd1f00cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.095+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f00c5170 0x7fd1f00cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.095+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/393094213 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f00c5c40 con 0x7fd1f00c6720 2023-08-27T11:18:10.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.095+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f00c6720 0x7fd1f00c4c30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0057e0 tx=0x7fd1ec001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:10.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.095+0000 7fd1e5ffb700 1 -- 172.21.15.112:0/393094213 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec004870 con 0x7fd1f00c6720 2023-08-27T11:18:10.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.095+0000 7fd1e5ffb700 1 -- 172.21.15.112:0/393094213 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec0049d0 con 0x7fd1f00c6720 2023-08-27T11:18:10.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.095+0000 7fd1f7603700 1 -- 172.21.15.112:0/393094213 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f00c5f20 con 0x7fd1f00c6720 2023-08-27T11:18:10.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.095+0000 7fd1e5ffb700 1 -- 172.21.15.112:0/393094213 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec004d80 con 0x7fd1f00c6720 2023-08-27T11:18:10.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.095+0000 7fd1f7603700 1 -- 172.21.15.112:0/393094213 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f00c9ed0 con 0x7fd1f00c6720 2023-08-27T11:18:10.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.097+0000 7fd1e5ffb700 1 -- 172.21.15.112:0/393094213 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd1ec0070d0 con 0x7fd1f00c6720 2023-08-27T11:18:10.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.097+0000 7fd1e5ffb700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd1d406a0e0 0x7fd1d406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.097+0000 7fd1e5ffb700 1 -- 172.21.15.112:0/393094213 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd1ec089c10 con 0x7fd1f00c6720 2023-08-27T11:18:10.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.097+0000 7fd1f539f700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd1d406a0e0 0x7fd1d406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.098+0000 7fd1f539f700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd1d406a0e0 0x7fd1d406c5a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00c77b0 tx=0x7fd1e0013040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:10.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.099+0000 7fd2037c5b80 1 Processor -- start 2023-08-27T11:18:10.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.099+0000 7fd2037c5b80 1 -- start start 2023-08-27T11:18:10.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd2037c5b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22221b0 0x5568f2222590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd2037c5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22691d0 0x5568f221e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd2037c5b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22203d0 0x5568f2249520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd2037c5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f217b730 con 0x5568f22691d0 2023-08-27T11:18:10.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd2037c5b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f2179050 con 0x5568f22203d0 2023-08-27T11:18:10.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd2037c5b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568f2174940 con 0x5568f22221b0 2023-08-27T11:18:10.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1f4b9e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22691d0 0x5568f221e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1f4b9e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22691d0 0x5568f221e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56942/0 (socket says 172.21.15.112:56942) 2023-08-27T11:18:10.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/335378988 learned_addr learned my addr 172.21.15.112:0/335378988 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:10.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1f539f700 1 --2- 172.21.15.112:0/335378988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22221b0 0x5568f2222590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1f5ba0700 1 --2- 172.21.15.112:0/335378988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22203d0 0x5568f2249520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/335378988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22221b0 msgr2=0x5568f2222590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:10.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/335378988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22221b0 0x5568f2222590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/335378988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22203d0 msgr2=0x5568f2249520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:10.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/335378988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22203d0 0x5568f2249520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/335378988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568f1ff45e0 con 0x5568f22691d0 2023-08-27T11:18:10.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1f5ba0700 1 --2- 172.21.15.112:0/335378988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22203d0 0x5568f2249520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:10.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/335378988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22691d0 0x5568f221e060 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec015040 tx=0x7fd1ec0555e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b9a91a0933e9ec server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:10.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1daffd700 1 -- 172.21.15.112:0/335378988 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec060e80 con 0x5568f22691d0 2023-08-27T11:18:10.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1daffd700 1 -- 172.21.15.112:0/335378988 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec057510 con 0x5568f22691d0 2023-08-27T11:18:10.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.100+0000 7fd1daffd700 1 -- 172.21.15.112:0/335378988 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec007cf0 con 0x5568f22691d0 2023-08-27T11:18:10.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.101+0000 7fd2037c5b80 1 -- 172.21.15.112:0/335378988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22691d0 msgr2=0x5568f221e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.101+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/335378988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22691d0 0x5568f221e060 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec015040 tx=0x7fd1ec0555e0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.101+0000 7fd2037c5b80 1 -- 172.21.15.112:0/335378988 shutdown_connections 2023-08-27T11:18:10.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.101+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/335378988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22203d0 0x5568f2249520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.101+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/335378988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22221b0 0x5568f2222590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.101+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/335378988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22691d0 0x5568f221e060 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.101+0000 7fd2037c5b80 1 -- 172.21.15.112:0/335378988 >> 172.21.15.112:0/335378988 conn(0x5568f2180c30 msgr2=0x5568f2267d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:10.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.101+0000 7fd2037c5b80 1 -- 172.21.15.112:0/335378988 shutdown_connections 2023-08-27T11:18:10.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.101+0000 7fd2037c5b80 1 -- 172.21.15.112:0/335378988 wait complete. 2023-08-27T11:18:10.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.102+0000 7fd2037c5b80 1 -- start start 2023-08-27T11:18:10.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.102+0000 7fd2037c5b80 1 Processor -- start 2023-08-27T11:18:10.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd2037c5b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22203d0 0x5568f224f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd2037c5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22221b0 0x5568f2261100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd2037c5b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22691d0 0x5568f2250790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd2037c5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f2112b60 con 0x5568f22221b0 2023-08-27T11:18:10.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd2037c5b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f2112860 con 0x5568f22691d0 2023-08-27T11:18:10.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd2037c5b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568f1ff68d0 con 0x5568f22203d0 2023-08-27T11:18:10.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd2037c5b80 10 client.0 _ll_register_callbacks cb 0x5568f2110630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:18:10.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1f4b9e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22221b0 0x5568f2261100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1f4b9e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22221b0 0x5568f2261100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56948/0 (socket says 172.21.15.112:56948) 2023-08-27T11:18:10.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/759052166 learned_addr learned my addr 172.21.15.112:0/759052166 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:10.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1f5ba0700 1 --2- 172.21.15.112:0/759052166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22691d0 0x5568f2250790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1f539f700 1 --2- 172.21.15.112:0/759052166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22203d0 0x5568f224f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/759052166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22203d0 msgr2=0x5568f224f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:10.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/759052166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22203d0 0x5568f224f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/759052166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22691d0 msgr2=0x5568f2250790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:10.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/759052166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22691d0 0x5568f2250790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1f4b9e700 1 -- 172.21.15.112:0/759052166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f00c9ed0 con 0x5568f22221b0 2023-08-27T11:18:10.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/759052166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22221b0 0x5568f2261100 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec015040 tx=0x7fd1ec001760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c79bc1fe679ef1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:10.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1cffff700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:18:10.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1cffff700 1 -- 172.21.15.112:0/759052166 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec055880 con 0x5568f22221b0 2023-08-27T11:18:10.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1cffff700 1 -- 172.21.15.112:0/759052166 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd1ec05e040 con 0x5568f22221b0 2023-08-27T11:18:10.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd2037c5b80 1 -- 172.21.15.112:0/759052166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568f22154f0 con 0x5568f22221b0 2023-08-27T11:18:10.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1cffff700 1 -- 172.21.15.112:0/759052166 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1ec0559e0 con 0x5568f22221b0 2023-08-27T11:18:10.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd2037c5b80 10 client.5358 fetch_fsmap learned FSMap version 15 2023-08-27T11:18:10.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd2037c5b80 1 -- 172.21.15.112:0/759052166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5568f218c750 con 0x5568f22221b0 2023-08-27T11:18:10.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.103+0000 7fd1cffff700 1 -- 172.21.15.112:0/759052166 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5568f22154f0 con 0x5568f22221b0 2023-08-27T11:18:10.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.104+0000 7fd1cffff700 1 -- 172.21.15.112:0/759052166 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd1ec085020 con 0x5568f22221b0 2023-08-27T11:18:10.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.104+0000 7fd2037c5b80 10 client.5358 fetch_fsmap finished waiting for FSMap version 15 2023-08-27T11:18:10.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.104+0000 7fd2037c5b80 10 client.5358 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-27T11:18:10.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.104+0000 7fd2037c5b80 20 client.5358 populate_metadata read hostname 'smithi112' 2023-08-27T11:18:10.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.104+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/759052166 >> [v2:172.21.15.138:6835/3409848660,v1:172.21.15.138:6837/3409848660] conn(0x5568f22479c0 0x5568f2247da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.104+0000 7fd2037c5b80 4 client.5358 mds_command: new command op to 5175 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:18:10.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.104+0000 7fd2037c5b80 1 -- 172.21.15.112:0/759052166 --> [v2:172.21.15.138:6835/3409848660,v1:172.21.15.138:6837/3409848660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568f216d920 con 0x5568f22479c0 2023-08-27T11:18:10.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.104+0000 7fd1f539f700 1 --2- 172.21.15.112:0/759052166 >> [v2:172.21.15.138:6835/3409848660,v1:172.21.15.138:6837/3409848660] conn(0x5568f22479c0 0x5568f2247da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.104+0000 7fd1cffff700 10 client.5358 ms_handle_connect on v2:172.21.15.138:6835/3409848660 2023-08-27T11:18:10.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.104+0000 7fd1f539f700 1 --2- 172.21.15.112:0/759052166 >> [v2:172.21.15.138:6835/3409848660,v1:172.21.15.138:6837/3409848660] conn(0x5568f22479c0 0x5568f2247da0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3da43ac8dbc9138 server_cookie=d394da89d4df3369 in_seq=0 out_seq=0 2023-08-27T11:18:10.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd1cffff700 1 -- 172.21.15.112:0/759052166 <== mds.0 v2:172.21.15.138:6835/3409848660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5568f216d920 con 0x5568f22479c0 2023-08-27T11:18:10.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd1cffff700 10 client.5358 handle_command_reply: tid=0 2023-08-27T11:18:10.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd2037c5b80 1 client.5358 shutdown 2023-08-27T11:18:10.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd2037c5b80 1 -- 172.21.15.112:0/759052166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22221b0 msgr2=0x5568f2261100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/759052166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22221b0 0x5568f2261100 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec015040 tx=0x7fd1ec001760 comp rx=0 tx=0).stop 2023-08-27T11:18:10.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd2037c5b80 1 -- 172.21.15.112:0/759052166 shutdown_connections 2023-08-27T11:18:10.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/759052166 >> [v2:172.21.15.138:6835/3409848660,v1:172.21.15.138:6837/3409848660] conn(0x5568f22479c0 0x5568f2247da0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/759052166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22691d0 0x5568f2250790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/759052166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22203d0 0x5568f224f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd1cffff700 0 client.5358 ms_handle_reset on v2:172.21.15.138:6835/3409848660 2023-08-27T11:18:10.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/759052166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22221b0 0x5568f2261100 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd2037c5b80 1 -- 172.21.15.112:0/759052166 >> 172.21.15.112:0/759052166 conn(0x5568f2180c30 msgr2=0x5568f22229e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:10.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd2037c5b80 1 -- 172.21.15.112:0/759052166 shutdown_connections 2023-08-27T11:18:10.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd2037c5b80 1 -- 172.21.15.112:0/759052166 wait complete. 2023-08-27T11:18:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.105+0000 7fd2037c5b80 20 client.5358 trim_cache size 0 max 16384 2023-08-27T11:18:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.123+0000 7fd2037c5b80 1 Processor -- start 2023-08-27T11:18:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.123+0000 7fd2037c5b80 1 -- start start 2023-08-27T11:18:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.123+0000 7fd2037c5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 0x5568f224fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.123+0000 7fd2037c5b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22221b0 0x5568f2246b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.123+0000 7fd2037c5b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22691d0 0x5568f2247050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.123+0000 7fd2037c5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f224e680 con 0x5568f22203d0 2023-08-27T11:18:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.123+0000 7fd2037c5b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f2247820 con 0x5568f22691d0 2023-08-27T11:18:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.123+0000 7fd2037c5b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568f2267fe0 con 0x5568f22221b0 2023-08-27T11:18:10.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1f539f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 0x5568f224fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1f539f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 0x5568f224fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56956/0 (socket says 172.21.15.112:56956) 2023-08-27T11:18:10.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1f539f700 1 -- 172.21.15.112:0/2403437957 learned_addr learned my addr 172.21.15.112:0/2403437957 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:10.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1f539f700 1 -- 172.21.15.112:0/2403437957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22221b0 msgr2=0x5568f2246b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:10.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1f5ba0700 1 --2- 172.21.15.112:0/2403437957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22691d0 0x5568f2247050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/2403437957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22221b0 0x5568f2246b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1f539f700 1 --2- 172.21.15.112:0/2403437957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22221b0 0x5568f2246b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1f539f700 1 -- 172.21.15.112:0/2403437957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22691d0 msgr2=0x5568f2247050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:10.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1f539f700 1 --2- 172.21.15.112:0/2403437957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22691d0 0x5568f2247050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1f539f700 1 -- 172.21.15.112:0/2403437957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568f1fa0d90 con 0x5568f22203d0 2023-08-27T11:18:10.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1f5ba0700 1 --2- 172.21.15.112:0/2403437957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22691d0 0x5568f2247050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:10.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/2403437957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22221b0 0x5568f2246b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:10.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1f539f700 1 --2- 172.21.15.112:0/2403437957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 0x5568f224fcc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x5568f2249410 tx=0x7fd1e0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03a698734e64e29 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:10.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1db7fe700 1 -- 172.21.15.112:0/2403437957 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0011c50 con 0x5568f22203d0 2023-08-27T11:18:10.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1db7fe700 1 -- 172.21.15.112:0/2403437957 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e0024070 con 0x5568f22203d0 2023-08-27T11:18:10.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.124+0000 7fd1db7fe700 1 -- 172.21.15.112:0/2403437957 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0004420 con 0x5568f22203d0 2023-08-27T11:18:10.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.125+0000 7fd2037c5b80 1 -- 172.21.15.112:0/2403437957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 msgr2=0x5568f224fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.125+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/2403437957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 0x5568f224fcc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x5568f2249410 tx=0x7fd1e0019000 comp rx=0 tx=0).stop 2023-08-27T11:18:10.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.125+0000 7fd2037c5b80 1 -- 172.21.15.112:0/2403437957 shutdown_connections 2023-08-27T11:18:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.125+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/2403437957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22691d0 0x5568f2247050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.125+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/2403437957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 0x5568f224fcc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.125+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/2403437957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22221b0 0x5568f2246b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.125+0000 7fd2037c5b80 1 -- 172.21.15.112:0/2403437957 >> 172.21.15.112:0/2403437957 conn(0x5568f2180c30 msgr2=0x5568f224ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.125+0000 7fd2037c5b80 1 -- 172.21.15.112:0/2403437957 shutdown_connections 2023-08-27T11:18:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.125+0000 7fd2037c5b80 1 -- 172.21.15.112:0/2403437957 wait complete. 2023-08-27T11:18:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.125+0000 7fd2037c5b80 1 -- start start 2023-08-27T11:18:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.125+0000 7fd2037c5b80 1 Processor -- start 2023-08-27T11:18:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.126+0000 7fd2037c5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 0x5568f2223a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.126+0000 7fd2037c5b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22221b0 0x5568f2223fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.126+0000 7fd2037c5b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22691d0 0x5568f224cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.126+0000 7fd2037c5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f227d0c0 con 0x5568f22203d0 2023-08-27T11:18:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.126+0000 7fd2037c5b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f2279e30 con 0x5568f22221b0 2023-08-27T11:18:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.126+0000 7fd2037c5b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568f227d350 con 0x5568f22691d0 2023-08-27T11:18:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.126+0000 7fd2037c5b80 10 client.0 _ll_register_callbacks cb 0x5568f2224900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:18:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.126+0000 7fd1f539f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 0x5568f2223a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.126+0000 7fd1f539f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 0x5568f2223a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56960/0 (socket says 172.21.15.112:56960) 2023-08-27T11:18:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.126+0000 7fd1f539f700 1 -- 172.21.15.112:0/3386374451 learned_addr learned my addr 172.21.15.112:0/3386374451 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.126+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/3386374451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22221b0 0x5568f2223fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.126+0000 7fd1f5ba0700 1 --2- 172.21.15.112:0/3386374451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22691d0 0x5568f224cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd1f539f700 1 -- 172.21.15.112:0/3386374451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22691d0 msgr2=0x5568f224cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:10.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd1f539f700 1 --2- 172.21.15.112:0/3386374451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22691d0 0x5568f224cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd1f539f700 1 -- 172.21.15.112:0/3386374451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22221b0 msgr2=0x5568f2223fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:10.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd1f539f700 1 --2- 172.21.15.112:0/3386374451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22221b0 0x5568f2223fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd1f539f700 1 -- 172.21.15.112:0/3386374451 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e002da70 con 0x5568f22203d0 2023-08-27T11:18:10.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd1f539f700 1 --2- 172.21.15.112:0/3386374451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 0x5568f2223a60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0011420 tx=0x7fd1e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4f91bfd297cb54 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:10.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd1cffff700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:18:10.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd1cffff700 1 -- 172.21.15.112:0/3386374451 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e002d2f0 con 0x5568f22203d0 2023-08-27T11:18:10.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd1cffff700 1 -- 172.21.15.112:0/3386374451 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd1e002d450 con 0x5568f22203d0 2023-08-27T11:18:10.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd2037c5b80 1 -- 172.21.15.112:0/3386374451 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1ec057510 con 0x5568f22203d0 2023-08-27T11:18:10.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd1cffff700 1 -- 172.21.15.112:0/3386374451 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1e00036a0 con 0x5568f22203d0 2023-08-27T11:18:10.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd2037c5b80 10 client.5364 fetch_fsmap learned FSMap version 15 2023-08-27T11:18:10.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd2037c5b80 1 -- 172.21.15.112:0/3386374451 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5568f1f89c10 con 0x5568f22203d0 2023-08-27T11:18:10.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd1cffff700 1 -- 172.21.15.112:0/3386374451 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec057510 con 0x5568f22203d0 2023-08-27T11:18:10.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd1cffff700 1 -- 172.21.15.112:0/3386374451 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd1e0034030 con 0x5568f22203d0 2023-08-27T11:18:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd2037c5b80 10 client.5364 fetch_fsmap finished waiting for FSMap version 15 2023-08-27T11:18:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd2037c5b80 10 client.5364 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-27T11:18:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd2037c5b80 20 client.5364 populate_metadata read hostname 'smithi112' 2023-08-27T11:18:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/3386374451 >> [v2:172.21.15.138:6835/3409848660,v1:172.21.15.138:6837/3409848660] conn(0x5568f2284f80 0x5568f224d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd2037c5b80 4 client.5364 mds_command: new command op to 5175 tid=0[{"prefix": "status"}] 2023-08-27T11:18:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd2037c5b80 1 -- 172.21.15.112:0/3386374451 --> [v2:172.21.15.138:6835/3409848660,v1:172.21.15.138:6837/3409848660] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5568f212e000 con 0x5568f2284f80 2023-08-27T11:18:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.127+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/3386374451 >> [v2:172.21.15.138:6835/3409848660,v1:172.21.15.138:6837/3409848660] conn(0x5568f2284f80 0x5568f224d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd1f4b9e700 1 --2- 172.21.15.112:0/3386374451 >> [v2:172.21.15.138:6835/3409848660,v1:172.21.15.138:6837/3409848660] conn(0x5568f2284f80 0x5568f224d0b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c41f69e0c44d45e server_cookie=d81d97a41d2a2f93 in_seq=0 out_seq=0 2023-08-27T11:18:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd1cffff700 10 client.5364 ms_handle_connect on v2:172.21.15.138:6835/3409848660 2023-08-27T11:18:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd1cffff700 1 -- 172.21.15.112:0/3386374451 <== mds.0 v2:172.21.15.138:6835/3409848660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5568f212e000 con 0x5568f2284f80 2023-08-27T11:18:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd1cffff700 10 client.5364 handle_command_reply: tid=0 2023-08-27T11:18:10.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd2037c5b80 1 client.5364 shutdown 2023-08-27T11:18:10.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd2037c5b80 1 -- 172.21.15.112:0/3386374451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 msgr2=0x5568f2223a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/3386374451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 0x5568f2223a60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0011420 tx=0x7fd1e00047c0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd2037c5b80 1 -- 172.21.15.112:0/3386374451 shutdown_connections 2023-08-27T11:18:10.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/3386374451 >> [v2:172.21.15.138:6835/3409848660,v1:172.21.15.138:6837/3409848660] conn(0x5568f2284f80 0x5568f224d0b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/3386374451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568f22691d0 0x5568f224cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/3386374451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568f22203d0 0x5568f2223a60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd2037c5b80 1 --2- 172.21.15.112:0/3386374451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568f22221b0 0x5568f2223fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd1cffff700 0 client.5364 ms_handle_reset on v2:172.21.15.138:6835/3409848660 2023-08-27T11:18:10.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd2037c5b80 1 -- 172.21.15.112:0/3386374451 >> 172.21.15.112:0/3386374451 conn(0x5568f2180c30 msgr2=0x5568f2222870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:10.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd2037c5b80 1 -- 172.21.15.112:0/3386374451 shutdown_connections 2023-08-27T11:18:10.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd2037c5b80 1 -- 172.21.15.112:0/3386374451 wait complete. 2023-08-27T11:18:10.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.128+0000 7fd2037c5b80 20 client.5364 trim_cache size 0 max 16384 2023-08-27T11:18:10.159 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:18:10.160 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e099a24f-c1cd-4f11-ac4f-ecacc10e5e98", 2023-08-27T11:18:10.160 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2023-08-27T11:18:10.160 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 5175, 2023-08-27T11:18:10.160 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2023-08-27T11:18:10.161 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2023-08-27T11:18:10.161 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2023-08-27T11:18:10.161 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 5.8877783990000001, 2023-08-27T11:18:10.161 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 15, 2023-08-27T11:18:10.161 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 33, 2023-08-27T11:18:10.162 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 33, 2023-08-27T11:18:10.162 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 14.051807923 2023-08-27T11:18:10.162 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:18:10.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.129+0000 7fd1f7603700 1 -- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd1d406a0e0 msgr2=0x7fd1d406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.129+0000 7fd1f7603700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd1d406a0e0 0x7fd1d406c5a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00c77b0 tx=0x7fd1e0013040 comp rx=0 tx=0).stop 2023-08-27T11:18:10.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.129+0000 7fd1f7603700 1 -- 172.21.15.112:0/393094213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f00c6720 msgr2=0x7fd1f00c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.129+0000 7fd1f7603700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f00c6720 0x7fd1f00c4c30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0057e0 tx=0x7fd1ec001980 comp rx=0 tx=0).stop 2023-08-27T11:18:10.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.129+0000 7fd1f7603700 1 -- 172.21.15.112:0/393094213 shutdown_connections 2023-08-27T11:18:10.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.129+0000 7fd1f7603700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd1d406a0e0 0x7fd1d406c5a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.129+0000 7fd1f7603700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f00c5170 0x7fd1f00cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.129+0000 7fd1f7603700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1dc0043e0 0x7fd1f01453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.129+0000 7fd1f7603700 1 --2- 172.21.15.112:0/393094213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f00c6720 0x7fd1f00c4c30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.129+0000 7fd1f7603700 1 -- 172.21.15.112:0/393094213 >> 172.21.15.112:0/393094213 conn(0x7fd1f00d0000 msgr2=0x7fd1f00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:10.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.129+0000 7fd1f7603700 1 -- 172.21.15.112:0/393094213 shutdown_connections 2023-08-27T11:18:10.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.129+0000 7fd1f7603700 1 -- 172.21.15.112:0/393094213 wait complete. 2023-08-27T11:18:10.167 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:18:10.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.288+0000 7fee4f71e700 1 Processor -- start 2023-08-27T11:18:10.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.288+0000 7fee4f71e700 1 -- start start 2023-08-27T11:18:10.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4f71e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee480b1400 0x7fee480cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4f71e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fee48161ce0 0x7fee4806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4f71e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee48161850 con 0x7fee480b17e0 2023-08-27T11:18:10.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4f71e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee480cfc10 con 0x7fee48161ce0 2023-08-27T11:18:10.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4f71e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee48161200 con 0x7fee480b1400 2023-08-27T11:18:10.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4dcbb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fee48161ce0 0x7fee4806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4d4ba700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee480b1400 0x7fee480cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4d4ba700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee480b1400 0x7fee480cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:40390/0 (socket says 172.21.15.112:40390) 2023-08-27T11:18:10.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4ccb9700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fee480b17e0 0x7fee4806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:58874/0 (socket says 172.21.15.112:58874) 2023-08-27T11:18:10.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4d4ba700 1 -- 172.21.15.112:0/668802295 learned_addr learned my addr 172.21.15.112:0/668802295 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:10.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4dcbb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fee48161ce0 0x7fee4806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:57396/0 (socket says 172.21.15.112:57396) 2023-08-27T11:18:10.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4dcbb700 1 -- 172.21.15.112:0/668802295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee480b1400 msgr2=0x7fee480cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:10.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4dcbb700 1 --2- 172.21.15.112:0/668802295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee480b1400 0x7fee480cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4dcbb700 1 -- 172.21.15.112:0/668802295 >> v1:172.21.15.138:6789/0 conn(0x7fee480b17e0 legacy=0x7fee4806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4dcbb700 1 -- 172.21.15.112:0/668802295 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee481614e0 con 0x7fee48161ce0 2023-08-27T11:18:10.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.289+0000 7fee4d4ba700 1 --2- 172.21.15.112:0/668802295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee480b1400 0x7fee480cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:10.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.290+0000 7fee4dcbb700 1 --2- 172.21.15.112:0/668802295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee48161ce0 0x7fee4806f790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fee40009d90 tx=0x7fee4000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e99e557f8dab141 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:10.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.290+0000 7fee3ffff700 1 -- 172.21.15.112:0/668802295 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4000e070 con 0x7fee48161ce0 2023-08-27T11:18:10.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.290+0000 7fee3ffff700 1 -- 172.21.15.112:0/668802295 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4000bb20 con 0x7fee48161ce0 2023-08-27T11:18:10.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.290+0000 7fee3ffff700 1 -- 172.21.15.112:0/668802295 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee40012430 con 0x7fee48161ce0 2023-08-27T11:18:10.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.290+0000 7fee4f71e700 1 -- 172.21.15.112:0/668802295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee48161ce0 msgr2=0x7fee4806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.290+0000 7fee4f71e700 1 --2- 172.21.15.112:0/668802295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee48161ce0 0x7fee4806f790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fee40009d90 tx=0x7fee4000b330 comp rx=0 tx=0).stop 2023-08-27T11:18:10.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.290+0000 7fee4f71e700 1 -- 172.21.15.112:0/668802295 shutdown_connections 2023-08-27T11:18:10.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.290+0000 7fee4f71e700 1 --2- 172.21.15.112:0/668802295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee48161ce0 0x7fee4806f790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.290+0000 7fee4f71e700 1 --2- 172.21.15.112:0/668802295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee480b1400 0x7fee480cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.290+0000 7fee4f71e700 1 -- 172.21.15.112:0/668802295 >> 172.21.15.112:0/668802295 conn(0x7fee480d0000 msgr2=0x7fee480d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:10.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.290+0000 7fee4f71e700 1 -- 172.21.15.112:0/668802295 shutdown_connections 2023-08-27T11:18:10.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.290+0000 7fee4f71e700 1 -- 172.21.15.112:0/668802295 wait complete. 2023-08-27T11:18:10.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.292+0000 7fee4f71e700 1 Processor -- start 2023-08-27T11:18:10.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.292+0000 7fee4f71e700 1 -- start start 2023-08-27T11:18:10.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.292+0000 7fee4f71e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee480b1400 0x7fee480c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.292+0000 7fee4f71e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee480b17e0 0x7fee481507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.292+0000 7fee4f71e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee48161ce0 0x7fee4815e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.292+0000 7fee4f71e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee480c83d0 con 0x7fee480b17e0 2023-08-27T11:18:10.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.292+0000 7fee4f71e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee480c7e10 con 0x7fee480b1400 2023-08-27T11:18:10.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.292+0000 7fee4f71e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee480c80f0 con 0x7fee48161ce0 2023-08-27T11:18:10.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.292+0000 7fee4ccb9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee480b17e0 0x7fee481507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee4ccb9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee480b17e0 0x7fee481507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56974/0 (socket says 172.21.15.112:56974) 2023-08-27T11:18:10.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee4ccb9700 1 -- 172.21.15.112:0/186921319 learned_addr learned my addr 172.21.15.112:0/186921319 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:10.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee4d4ba700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee480b1400 0x7fee480c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee4dcbb700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee48161ce0 0x7fee4815e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee4ccb9700 1 -- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee48161ce0 msgr2=0x7fee4815e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee4ccb9700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee48161ce0 0x7fee4815e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee4ccb9700 1 -- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee480b1400 msgr2=0x7fee480c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee4ccb9700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee480b1400 0x7fee480c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee4ccb9700 1 -- 172.21.15.112:0/186921319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee44009550 con 0x7fee480b17e0 2023-08-27T11:18:10.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee4ccb9700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee480b17e0 0x7fee481507c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fee3800c520 tx=0x7fee3800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:10.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee3dffb700 1 -- 172.21.15.112:0/186921319 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38005cb0 con 0x7fee480b17e0 2023-08-27T11:18:10.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee3dffb700 1 -- 172.21.15.112:0/186921319 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee38005e10 con 0x7fee480b17e0 2023-08-27T11:18:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee4f71e700 1 -- 172.21.15.112:0/186921319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee40009a70 con 0x7fee480b17e0 2023-08-27T11:18:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee3dffb700 1 -- 172.21.15.112:0/186921319 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee380072d0 con 0x7fee480b17e0 2023-08-27T11:18:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.293+0000 7fee4f71e700 1 -- 172.21.15.112:0/186921319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee480706e0 con 0x7fee480b17e0 2023-08-27T11:18:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.294+0000 7fee4f71e700 1 -- 172.21.15.112:0/186921319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee48070260 con 0x7fee480b17e0 2023-08-27T11:18:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.295+0000 7fee3dffb700 1 -- 172.21.15.112:0/186921319 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fee38007430 con 0x7fee480b17e0 2023-08-27T11:18:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.295+0000 7fee3dffb700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fee2c06a0e0 0x7fee2c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.295+0000 7fee3dffb700 1 -- 172.21.15.112:0/186921319 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fee38087750 con 0x7fee480b17e0 2023-08-27T11:18:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.295+0000 7fee4d4ba700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fee2c06a0e0 0x7fee2c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.296+0000 7fee4d4ba700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fee2c06a0e0 0x7fee2c06c5a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fee44006d30 tx=0x7fee440092c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.300+0000 7fee3dffb700 1 -- 172.21.15.112:0/186921319 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fee48070260 con 0x7fee480b17e0 2023-08-27T11:18:10.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.532+0000 7fee4f71e700 1 -- 172.21.15.112:0/186921319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fee48070260 con 0x7fee480b17e0 2023-08-27T11:18:10.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.532+0000 7fee3dffb700 1 -- 172.21.15.112:0/186921319 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7fee48070260 con 0x7fee480b17e0 2023-08-27T11:18:10.533 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:18:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.534+0000 7fee4f71e700 1 -- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fee2c06a0e0 msgr2=0x7fee2c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.534+0000 7fee4f71e700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fee2c06a0e0 0x7fee2c06c5a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fee44006d30 tx=0x7fee440092c0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.534+0000 7fee4f71e700 1 -- 172.21.15.112:0/186921319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee480b17e0 msgr2=0x7fee481507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.534+0000 7fee4f71e700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee480b17e0 0x7fee481507c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fee3800c520 tx=0x7fee3800c830 comp rx=0 tx=0).stop 2023-08-27T11:18:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.534+0000 7fee4f71e700 1 -- 172.21.15.112:0/186921319 shutdown_connections 2023-08-27T11:18:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.534+0000 7fee4f71e700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fee2c06a0e0 0x7fee2c06c5a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.534+0000 7fee4f71e700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee48161ce0 0x7fee4815e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.534+0000 7fee4f71e700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee480b1400 0x7fee480c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.534+0000 7fee4f71e700 1 --2- 172.21.15.112:0/186921319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee480b17e0 0x7fee481507c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:10.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.534+0000 7fee4f71e700 1 -- 172.21.15.112:0/186921319 >> 172.21.15.112:0/186921319 conn(0x7fee480d0000 msgr2=0x7fee4815eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:10.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.535+0000 7fee4f71e700 1 -- 172.21.15.112:0/186921319 shutdown_connections 2023-08-27T11:18:10.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:10.535+0000 7fee4f71e700 1 -- 172.21.15.112:0/186921319 wait complete. 2023-08-27T11:18:10.547 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:18:10.547 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:18:10.548 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-08-27T11:18:10.548 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:18:10.549 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:18:10.549 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:18:10.549 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:18:10.550 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:18:10.550 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:18:10.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:10.551 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-08-27T11:18:10.569 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:18:10.569 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:18:10.569 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:18:10.570 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:10.570 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:18:10.570 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:10.570 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:18:10.570 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:10.571 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:18:10.571 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:10.571 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:18:10.571 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:10.571 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:18:10.572 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:10.572 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:18:10.572 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42352sec preferred_lft 42352sec 2023-08-27T11:18:10.572 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8d6c:4fe6:6a84:8222/64 scope link noprefixroute 2023-08-27T11:18:10.572 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:10.572 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:18:10.573 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:10.573 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:18:10.573 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:10.573 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::d87c:a1ff:fe1d:b635/64 scope link 2023-08-27T11:18:10.574 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:10.574 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:18:10.574 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:18:10.574 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::cc14:79ff:feb3:452f/64 scope link 2023-08-27T11:18:10.574 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:10.575 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:18:10.575 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether e6:51:f0:78:04:02 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:18:10.575 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::e451:f0ff:fe78:402/64 scope link 2023-08-27T11:18:10.575 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:10.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:10.576 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:18:10.634 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-08-27T11:18:10.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:10.636 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-08-27T11:18:10.694 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-08-27T11:18:10.695 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:18:10.695 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:18:10.695 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:18:10.695 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:18:10.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:10.696 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:18:10.696 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:18:10.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:18:10.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:18:10.698 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:18:10.698 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:18:10.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:10.799 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:18:10.816 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:18:10.816 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:18:10.817 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:18:10.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:10.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-08-27T11:18:10.884 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:18:10.885 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:18:10.941 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:18:10.941 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:18:10.942 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:18:10.997 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:18:10.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:10.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:18:11.082 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:18:11.082 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:18:11.082 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:18:11.090 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:18:11.091 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:18:11.091 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:18:11.092 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:18:11.092 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:18:11.133 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:18:11.134 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:18:11.134 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:18:11.134 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:18:11.135 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:18:11.135 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:18:11.135 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:18:11.135 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:18:11.148 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:18:11.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:11.149 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:18:11.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:11.177 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:18:11.264 INFO:teuthology.orchestra.run.smithi112.stdout:client5370 2023-08-27T11:18:11.264 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:18:11.265 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:11.282 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:18:11.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.443+0000 7f1f8e01a700 1 Processor -- start 2023-08-27T11:18:11.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.443+0000 7f1f8e01a700 1 -- start start 2023-08-27T11:18:11.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.443+0000 7f1f8e01a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1f8815a120 0x7f1f880b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:11.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.443+0000 7f1f8e01a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f8806da40 0x7f1f880b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:11.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.443+0000 7f1f8e01a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8806deb0 con 0x7f1f8806d5b0 2023-08-27T11:18:11.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.443+0000 7f1f8e01a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8815b540 con 0x7f1f8815a120 2023-08-27T11:18:11.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.443+0000 7f1f8e01a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f880af830 con 0x7f1f8806da40 2023-08-27T11:18:11.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f86ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1f8806d5b0 0x7f1f880b3440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51368/0 (socket says 172.21.15.112:51368) 2023-08-27T11:18:11.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f86ffd700 1 -- 172.21.15.112:0/477103081 learned_addr learned my addr 172.21.15.112:0/477103081 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:11.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f877fe700 1 --2- 172.21.15.112:0/477103081 >> v2:172.21.15.138:3300/0 conn(0x7f1f8815a120 0x7f1f880b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:11.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f87fff700 1 --2- 172.21.15.112:0/477103081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f8806da40 0x7f1f880b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:11.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125643226 0 0) 0x7f1f8806deb0 con 0x7f1f8806d5b0 2023-08-27T11:18:11.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f740035b0 con 0x7f1f8806d5b0 2023-08-27T11:18:11.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2332098281 0 0) 0x7f1f740035b0 con 0x7f1f8806d5b0 2023-08-27T11:18:11.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f8806da40 msgr2=0x7f1f880b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:11.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 --2- 172.21.15.112:0/477103081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f8806da40 0x7f1f880b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:11.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 >> v2:172.21.15.138:3300/0 conn(0x7f1f8815a120 msgr2=0x7f1f880b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:11.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 --2- 172.21.15.112:0/477103081 >> v2:172.21.15.138:3300/0 conn(0x7f1f8815a120 0x7f1f880b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:11.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f880afb10 con 0x7f1f8806d5b0 2023-08-27T11:18:11.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f1f70001670 con 0x7f1f8806d5b0 2023-08-27T11:18:11.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 >> v1:172.21.15.112:6789/0 conn(0x7f1f8806d5b0 legacy=0x7f1f880b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:11.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 --2- 172.21.15.112:0/477103081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f74003a00 0x7f1f74003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:11.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 --2- 172.21.15.112:0/477103081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f8806da40 0x7f1f74004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:11.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 --2- 172.21.15.112:0/477103081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74004d90 0x7f1f74005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:11.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8815b540 con 0x7f1f74003a00 2023-08-27T11:18:11.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f880af830 con 0x7f1f74004d90 2023-08-27T11:18:11.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7400ba10 con 0x7f1f8806da40 2023-08-27T11:18:11.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f877fe700 1 --2- 172.21.15.112:0/477103081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f74003a00 0x7f1f74003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:11.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f87fff700 1 --2- 172.21.15.112:0/477103081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74004d90 0x7f1f74005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:11.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f86ffd700 1 --2- 172.21.15.112:0/477103081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f8806da40 0x7f1f74004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:11.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f877fe700 1 -- 172.21.15.112:0/477103081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f8806da40 msgr2=0x7f1f74004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:11.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.444+0000 7f1f877fe700 1 --2- 172.21.15.112:0/477103081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f8806da40 0x7f1f74004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:11.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f877fe700 1 -- 172.21.15.112:0/477103081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74004d90 msgr2=0x7f1f74005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:11.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f877fe700 1 --2- 172.21.15.112:0/477103081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74004d90 0x7f1f74005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:11.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f877fe700 1 -- 172.21.15.112:0/477103081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740035b0 con 0x7f1f74003a00 2023-08-27T11:18:11.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f87fff700 1 --2- 172.21.15.112:0/477103081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74004d90 0x7f1f74005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:11.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f877fe700 1 --2- 172.21.15.112:0/477103081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f74003a00 0x7f1f74003e20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009b60 tx=0x7f1f74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3269fb414e35dc72 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:11.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7800e6c0 con 0x7f1f74003a00 2023-08-27T11:18:11.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7800e820 con 0x7f1f74003a00 2023-08-27T11:18:11.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f867fc700 1 -- 172.21.15.112:0/477103081 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7800eb90 con 0x7f1f74003a00 2023-08-27T11:18:11.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f8e01a700 1 -- 172.21.15.112:0/477103081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f74003a00 msgr2=0x7f1f74003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:11.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f8e01a700 1 --2- 172.21.15.112:0/477103081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f74003a00 0x7f1f74003e20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009b60 tx=0x7f1f74005880 comp rx=0 tx=0).stop 2023-08-27T11:18:11.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f87fff700 1 -- 172.21.15.112:0/477103081 reap_dead start 2023-08-27T11:18:11.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f8e01a700 1 -- 172.21.15.112:0/477103081 shutdown_connections 2023-08-27T11:18:11.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f8e01a700 1 -- 172.21.15.112:0/477103081 >> 172.21.15.112:0/477103081 conn(0x7f1f880cff80 msgr2=0x7f1f880d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:11.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f8e01a700 1 -- 172.21.15.112:0/477103081 shutdown_connections 2023-08-27T11:18:11.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.445+0000 7f1f8e01a700 1 -- 172.21.15.112:0/477103081 wait complete. 2023-08-27T11:18:11.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.447+0000 7f1f8e01a700 1 Processor -- start 2023-08-27T11:18:11.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.447+0000 7f1f8e01a700 1 -- start start 2023-08-27T11:18:11.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.447+0000 7f1f8e01a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f74003a00 0x7f1f881420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:11.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.447+0000 7f1f8e01a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f881f4730 0x7f1f880706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:11.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.447+0000 7f1f8e01a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f88070bf0 0x7f1f881448b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:11.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.447+0000 7f1f8e01a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f881458b0 con 0x7f1f881f4730 2023-08-27T11:18:11.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.447+0000 7f1f8e01a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f881f4310 con 0x7f1f88070bf0 2023-08-27T11:18:11.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.447+0000 7f1f8e01a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f88071070 con 0x7f1f74003a00 2023-08-27T11:18:11.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f86ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f881f4730 0x7f1f880706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:11.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f86ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f881f4730 0x7f1f880706b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56998/0 (socket says 172.21.15.112:56998) 2023-08-27T11:18:11.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f86ffd700 1 -- 172.21.15.112:0/3501041022 learned_addr learned my addr 172.21.15.112:0/3501041022 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:11.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f877fe700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f74003a00 0x7f1f881420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:11.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f87fff700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f88070bf0 0x7f1f881448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:11.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f86ffd700 1 -- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f74003a00 msgr2=0x7f1f881420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:11.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f86ffd700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f74003a00 0x7f1f881420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:11.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f86ffd700 1 -- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f88070bf0 msgr2=0x7f1f881448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:11.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f86ffd700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f88070bf0 0x7f1f881448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:11.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f86ffd700 1 -- 172.21.15.112:0/3501041022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f881425e0 con 0x7f1f881f4730 2023-08-27T11:18:11.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f87fff700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f88070bf0 0x7f1f881448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:11.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f86ffd700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f881f4730 0x7f1f880706b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000e8c0 tx=0x7f1f7000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:11.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f7ffff700 1 -- 172.21.15.112:0/3501041022 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70012590 con 0x7f1f881f4730 2023-08-27T11:18:11.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f7ffff700 1 -- 172.21.15.112:0/3501041022 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f700126f0 con 0x7f1f881f4730 2023-08-27T11:18:11.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f8e01a700 1 -- 172.21.15.112:0/3501041022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f881455d0 con 0x7f1f881f4730 2023-08-27T11:18:11.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f7ffff700 1 -- 172.21.15.112:0/3501041022 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70012aa0 con 0x7f1f881f4730 2023-08-27T11:18:11.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.448+0000 7f1f8e01a700 1 -- 172.21.15.112:0/3501041022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f881f3f30 con 0x7f1f881f4730 2023-08-27T11:18:11.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.449+0000 7f1f8e01a700 1 -- 172.21.15.112:0/3501041022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f881f3ab0 con 0x7f1f881f4730 2023-08-27T11:18:11.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.449+0000 7f1f7ffff700 1 -- 172.21.15.112:0/3501041022 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1f70012c00 con 0x7f1f881f4730 2023-08-27T11:18:11.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.450+0000 7f1f7ffff700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1f6806a0e0 0x7f1f6806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:11.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.450+0000 7f1f7ffff700 1 -- 172.21.15.112:0/3501041022 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f700880b0 con 0x7f1f881f4730 2023-08-27T11:18:11.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.450+0000 7f1f877fe700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1f6806a0e0 0x7f1f6806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:11.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.451+0000 7f1f877fe700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1f6806a0e0 0x7f1f6806c5a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1f780090b0 tx=0x7f1f7801f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:11.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.454+0000 7f1f7ffff700 1 -- 172.21.15.112:0/3501041022 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1f881f3ab0 con 0x7f1f881f4730 2023-08-27T11:18:11.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.688+0000 7f1f8e01a700 1 -- 172.21.15.112:0/3501041022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1f881f4450 con 0x7f1f881f4730 2023-08-27T11:18:11.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.688+0000 7f1f7ffff700 1 -- 172.21.15.112:0/3501041022 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1f881f4450 con 0x7f1f881f4730 2023-08-27T11:18:11.689 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:18:11.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.690+0000 7f1f8e01a700 1 -- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1f6806a0e0 msgr2=0x7f1f6806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:11.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.690+0000 7f1f8e01a700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1f6806a0e0 0x7f1f6806c5a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1f780090b0 tx=0x7f1f7801f040 comp rx=0 tx=0).stop 2023-08-27T11:18:11.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.690+0000 7f1f8e01a700 1 -- 172.21.15.112:0/3501041022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f881f4730 msgr2=0x7f1f880706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:11.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.690+0000 7f1f8e01a700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f881f4730 0x7f1f880706b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000e8c0 tx=0x7f1f7000ebd0 comp rx=0 tx=0).stop 2023-08-27T11:18:11.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.690+0000 7f1f8e01a700 1 -- 172.21.15.112:0/3501041022 shutdown_connections 2023-08-27T11:18:11.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.690+0000 7f1f8e01a700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1f6806a0e0 0x7f1f6806c5a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:11.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.690+0000 7f1f8e01a700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f88070bf0 0x7f1f881448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:11.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.690+0000 7f1f8e01a700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f74003a00 0x7f1f881420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:11.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.690+0000 7f1f8e01a700 1 --2- 172.21.15.112:0/3501041022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f881f4730 0x7f1f880706b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:11.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.690+0000 7f1f8e01a700 1 -- 172.21.15.112:0/3501041022 >> 172.21.15.112:0/3501041022 conn(0x7f1f880cff80 msgr2=0x7f1f88159320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:11.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.690+0000 7f1f8e01a700 1 -- 172.21.15.112:0/3501041022 shutdown_connections 2023-08-27T11:18:11.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:11.690+0000 7f1f8e01a700 1 -- 172.21.15.112:0/3501041022 wait complete. 2023-08-27T11:18:11.703 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:18:11.703 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:18:11.703 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2023-08-27T11:18:11.704 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-27T11:18:11.704 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:18:11.704 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:18:11.705 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:18:11.705 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:18:11.705 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:18:11.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:11.706 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2023-08-27T11:18:11.728 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:18:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:18:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:18:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:18:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:11.730 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:18:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:11.730 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:18:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:11.731 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:18:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:11.731 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:18:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:18:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42367sec preferred_lft 42367sec 2023-08-27T11:18:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ac17:49c5:cb39:801a/64 scope link noprefixroute 2023-08-27T11:18:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:11.732 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:18:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:18:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c4b4:94ff:fe58:4e35/64 scope link 2023-08-27T11:18:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:11.734 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:18:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:18:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::5806:6aff:fe04:ef32/64 scope link 2023-08-27T11:18:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:11.734 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:18:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 8a:21:ba:15:69:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-27T11:18:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::8821:baff:fe15:6919/64 scope link 2023-08-27T11:18:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:11.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:11.736 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:18:11.800 INFO:teuthology.orchestra.run.smithi138.stdout:1 2023-08-27T11:18:11.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:11.802 DEBUG:teuthology.orchestra.run.smithi138:> route 2023-08-27T11:18:11.862 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2023-08-27T11:18:11.863 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:18:11.863 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:18:11.863 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:18:11.863 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:18:11.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:11.864 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:18:11.865 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:18:11.865 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:18:11.865 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:18:11.866 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:18:11.866 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:18:11.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:11.977 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:18:11.995 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-27T11:18:11.995 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-27T11:18:11.996 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:18:11.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:11.997 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2023-08-27T11:18:12.064 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:18:12.065 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-27T11:18:12.120 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:18:12.121 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:18:12.121 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:18:12.178 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:18:12.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:12.179 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:18:12.266 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:18:12.266 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:18:12.266 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:18:12.275 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:18:12.276 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:18:12.276 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:18:12.276 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:18:12.276 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:18:12.302 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:18:12.303 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:18:12.303 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:18:12.303 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:18:12.303 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:18:12.304 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:18:12.304 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:18:12.304 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:18:12.306 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:18:12.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:12.306 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:18:12.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:12.338 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:18:12.409 INFO:teuthology.orchestra.run.smithi138.stdout:client5128 2023-08-27T11:18:12.409 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:18:12.410 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:12.468 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-08-27T11:18:12.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.596+0000 7f306241c700 1 Processor -- start 2023-08-27T11:18:12.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.596+0000 7f306241c700 1 -- start start 2023-08-27T11:18:12.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f306241c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f305c06fea0 0x7f305c15e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:12.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f306241c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305c161390 con 0x7f305c0b3de0 2023-08-27T11:18:12.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f306241c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305c0b35f0 con 0x7f305c157ae0 2023-08-27T11:18:12.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f306241c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c0b1aa0 con 0x7f305c06fea0 2023-08-27T11:18:12.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f305b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f305c0b3de0 0x7f305c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51388/0 (socket says 172.21.15.112:51388) 2023-08-27T11:18:12.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f305b7fe700 1 -- 172.21.15.112:0/4050767450 learned_addr learned my addr 172.21.15.112:0/4050767450 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:12.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f30609b9700 1 --2- 172.21.15.112:0/4050767450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f305c06fea0 0x7f305c15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:12.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f305affd700 1 -- 172.21.15.112:0/4050767450 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473951338 0 0) 0x7f305c161390 con 0x7f305c0b3de0 2023-08-27T11:18:12.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f305affd700 1 -- 172.21.15.112:0/4050767450 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30440035b0 con 0x7f305c0b3de0 2023-08-27T11:18:12.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f305affd700 1 -- 172.21.15.112:0/4050767450 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 162417298 0 0) 0x7f305c0b35f0 con 0x7f305c157ae0 2023-08-27T11:18:12.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f305affd700 1 -- 172.21.15.112:0/4050767450 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f305c161390 con 0x7f305c157ae0 2023-08-27T11:18:12.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f30609b9700 1 -- 172.21.15.112:0/4050767450 >> v1:172.21.15.138:6789/0 conn(0x7f305c157ae0 legacy=0x7f305c161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:12.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f30609b9700 1 -- 172.21.15.112:0/4050767450 >> v1:172.21.15.112:6789/0 conn(0x7f305c0b3de0 legacy=0x7f305c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:12.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.597+0000 7f30609b9700 1 -- 172.21.15.112:0/4050767450 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c0b4330 con 0x7f305c06fea0 2023-08-27T11:18:12.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.598+0000 7f30609b9700 1 --2- 172.21.15.112:0/4050767450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f305c06fea0 0x7f305c15e580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3054009dc0 tx=0x7f305400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe09ab91619ae346 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:12.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.598+0000 7f305affd700 1 -- 172.21.15.112:0/4050767450 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305400e070 con 0x7f305c06fea0 2023-08-27T11:18:12.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.598+0000 7f305affd700 1 -- 172.21.15.112:0/4050767450 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305400bb20 con 0x7f305c06fea0 2023-08-27T11:18:12.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.598+0000 7f305affd700 1 -- 172.21.15.112:0/4050767450 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054012430 con 0x7f305c06fea0 2023-08-27T11:18:12.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.598+0000 7f306241c700 1 -- 172.21.15.112:0/4050767450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f305c06fea0 msgr2=0x7f305c15e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:12.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.598+0000 7f306241c700 1 --2- 172.21.15.112:0/4050767450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f305c06fea0 0x7f305c15e580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3054009dc0 tx=0x7f305400b330 comp rx=0 tx=0).stop 2023-08-27T11:18:12.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.598+0000 7f306241c700 1 -- 172.21.15.112:0/4050767450 shutdown_connections 2023-08-27T11:18:12.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.598+0000 7f306241c700 1 --2- 172.21.15.112:0/4050767450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f305c06fea0 0x7f305c15e580 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:12.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.598+0000 7f306241c700 1 -- 172.21.15.112:0/4050767450 >> 172.21.15.112:0/4050767450 conn(0x7f305c0b1be0 msgr2=0x7f305c0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:12.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.598+0000 7f306241c700 1 -- 172.21.15.112:0/4050767450 shutdown_connections 2023-08-27T11:18:12.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.598+0000 7f306241c700 1 -- 172.21.15.112:0/4050767450 wait complete. 2023-08-27T11:18:12.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.599+0000 7f306241c700 1 Processor -- start 2023-08-27T11:18:12.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.600+0000 7f306241c700 1 -- start start 2023-08-27T11:18:12.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.600+0000 7f306241c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f305c06fea0 0x7f305c0ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:12.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.600+0000 7f306241c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f305c0b3de0 0x7f305c0708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:12.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.600+0000 7f306241c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c157ae0 0x7f305c070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:12.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.600+0000 7f306241c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c0ae800 con 0x7f305c157ae0 2023-08-27T11:18:12.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.600+0000 7f306241c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c157650 con 0x7f305c0b3de0 2023-08-27T11:18:12.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.600+0000 7f306241c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c0ae9f0 con 0x7f305c06fea0 2023-08-27T11:18:12.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.600+0000 7f30609b9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c157ae0 0x7f305c070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:12.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f30609b9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c157ae0 0x7f305c070de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57012/0 (socket says 172.21.15.112:57012) 2023-08-27T11:18:12.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f30609b9700 1 -- 172.21.15.112:0/3715297702 learned_addr learned my addr 172.21.15.112:0/3715297702 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:12.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f305b7fe700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f305c0b3de0 0x7f305c0708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:12.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f305bfff700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f305c06fea0 0x7f305c0ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:12.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f30609b9700 1 -- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f305c06fea0 msgr2=0x7f305c0ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:12.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f30609b9700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f305c06fea0 0x7f305c0ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:12.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f30609b9700 1 -- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f305c0b3de0 msgr2=0x7f305c0708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:12.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f30609b9700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f305c0b3de0 0x7f305c0708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:12.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f30609b9700 1 -- 172.21.15.112:0/3715297702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3054009a70 con 0x7f305c157ae0 2023-08-27T11:18:12.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f30609b9700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c157ae0 0x7f305c070de0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f305400b370 tx=0x7f3054004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:12.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f3058ff9700 1 -- 172.21.15.112:0/3715297702 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054012430 con 0x7f305c157ae0 2023-08-27T11:18:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f306241c700 1 -- 172.21.15.112:0/3715297702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305c0aecd0 con 0x7f305c157ae0 2023-08-27T11:18:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f3058ff9700 1 -- 172.21.15.112:0/3715297702 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3054020070 con 0x7f305c157ae0 2023-08-27T11:18:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f3058ff9700 1 -- 172.21.15.112:0/3715297702 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305401ba90 con 0x7f305c157ae0 2023-08-27T11:18:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.601+0000 7f306241c700 1 -- 172.21.15.112:0/3715297702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c0af290 con 0x7f305c157ae0 2023-08-27T11:18:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.602+0000 7f306241c700 1 -- 172.21.15.112:0/3715297702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f305c067a20 con 0x7f305c157ae0 2023-08-27T11:18:12.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.603+0000 7f3058ff9700 1 -- 172.21.15.112:0/3715297702 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f305401bbf0 con 0x7f305c157ae0 2023-08-27T11:18:12.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.603+0000 7f3058ff9700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f304806a0e0 0x7f304806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:12.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.603+0000 7f3058ff9700 1 -- 172.21.15.112:0/3715297702 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3054022070 con 0x7f305c157ae0 2023-08-27T11:18:12.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.603+0000 7f305bfff700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f304806a0e0 0x7f304806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:12.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.604+0000 7f305bfff700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f304806a0e0 0x7f304806c5a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3050001220 tx=0x7f3050009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:12.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.608+0000 7f3058ff9700 1 -- 172.21.15.112:0/3715297702 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3054055f60 con 0x7f305c157ae0 2023-08-27T11:18:12.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:12.840+0000 7f306241c700 1 -- 172.21.15.112:0/3715297702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f305c160e60 con 0x7f305c157ae0 2023-08-27T11:18:13.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.287+0000 7f3058ff9700 1 -- 172.21.15.112:0/3715297702 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 136+0+0 (secure 0 0 0) 0x7f3054059580 con 0x7f305c157ae0 2023-08-27T11:18:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.289+0000 7f306241c700 1 -- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f304806a0e0 msgr2=0x7f304806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.289+0000 7f306241c700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f304806a0e0 0x7f304806c5a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3050001220 tx=0x7f3050009040 comp rx=0 tx=0).stop 2023-08-27T11:18:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.289+0000 7f306241c700 1 -- 172.21.15.112:0/3715297702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c157ae0 msgr2=0x7f305c070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.289+0000 7f306241c700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c157ae0 0x7f305c070de0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f305400b370 tx=0x7f3054004ad0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.289+0000 7f306241c700 1 -- 172.21.15.112:0/3715297702 shutdown_connections 2023-08-27T11:18:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.289+0000 7f306241c700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f304806a0e0 0x7f304806c5a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.289+0000 7f306241c700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c157ae0 0x7f305c070de0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.289+0000 7f306241c700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f305c06fea0 0x7f305c0ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.289+0000 7f306241c700 1 --2- 172.21.15.112:0/3715297702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f305c0b3de0 0x7f305c0708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.289+0000 7f306241c700 1 -- 172.21.15.112:0/3715297702 >> 172.21.15.112:0/3715297702 conn(0x7f305c0b1be0 msgr2=0x7f305c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.289+0000 7f306241c700 1 -- 172.21.15.112:0/3715297702 shutdown_connections 2023-08-27T11:18:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.289+0000 7f306241c700 1 -- 172.21.15.112:0/3715297702 wait complete. 2023-08-27T11:18:13.304 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:18:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.426+0000 7fd2521f2700 1 Processor -- start 2023-08-27T11:18:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.427+0000 7fd2521f2700 1 -- start start 2023-08-27T11:18:13.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.427+0000 7fd2521f2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd24c06e150 0x7fd24c0cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:13.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.427+0000 7fd2521f2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd24c13f990 0x7fd24c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:13.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.427+0000 7fd2521f2700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24c06ea70 con 0x7fd24c0cd270 2023-08-27T11:18:13.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.427+0000 7fd2521f2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c0cecb0 con 0x7fd24c13f990 2023-08-27T11:18:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.427+0000 7fd2521f2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0cf570 con 0x7fd24c06e150 2023-08-27T11:18:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.428+0000 7fd24b7fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd24c06e150 0x7fd24c0cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.428+0000 7fd24affd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd24c13f990 0x7fd24c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:13.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.428+0000 7fd24bfff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd24c0cd270 0x7fd24c15ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:58928/0 (socket says 172.21.15.112:58928) 2023-08-27T11:18:13.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.428+0000 7fd24bfff700 1 -- 172.21.15.112:0/1811475486 learned_addr learned my addr 172.21.15.112:0/1811475486 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:13.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.428+0000 7fd24a7fc700 1 -- 172.21.15.112:0/1811475486 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061793155 0 0) 0x7fd24c06ea70 con 0x7fd24c0cd270 2023-08-27T11:18:13.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.428+0000 7fd24a7fc700 1 -- 172.21.15.112:0/1811475486 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2340035b0 con 0x7fd24c0cd270 2023-08-27T11:18:13.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.428+0000 7fd24affd700 1 -- 172.21.15.112:0/1811475486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd24c06e150 msgr2=0x7fd24c0cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:13.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.428+0000 7fd24affd700 1 --2- 172.21.15.112:0/1811475486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd24c06e150 0x7fd24c0cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.428+0000 7fd24affd700 1 -- 172.21.15.112:0/1811475486 >> v1:172.21.15.138:6789/0 conn(0x7fd24c0cd270 legacy=0x7fd24c15ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:13.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.428+0000 7fd24affd700 1 -- 172.21.15.112:0/1811475486 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c13ff50 con 0x7fd24c13f990 2023-08-27T11:18:13.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.428+0000 7fd24b7fe700 1 --2- 172.21.15.112:0/1811475486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd24c06e150 0x7fd24c0cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.428+0000 7fd24affd700 1 --2- 172.21.15.112:0/1811475486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c13f990 0x7fd24c06e530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd240009820 tx=0x7fd240009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d90f2959008deddd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.429+0000 7fd24a7fc700 1 -- 172.21.15.112:0/1811475486 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240005e30 con 0x7fd24c13f990 2023-08-27T11:18:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.429+0000 7fd24a7fc700 1 -- 172.21.15.112:0/1811475486 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd240006030 con 0x7fd24c13f990 2023-08-27T11:18:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.429+0000 7fd24a7fc700 1 -- 172.21.15.112:0/1811475486 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2400063c0 con 0x7fd24c13f990 2023-08-27T11:18:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.429+0000 7fd2521f2700 1 -- 172.21.15.112:0/1811475486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c13f990 msgr2=0x7fd24c06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.429+0000 7fd2521f2700 1 --2- 172.21.15.112:0/1811475486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c13f990 0x7fd24c06e530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd240009820 tx=0x7fd240009b30 comp rx=0 tx=0).stop 2023-08-27T11:18:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.429+0000 7fd2521f2700 1 -- 172.21.15.112:0/1811475486 shutdown_connections 2023-08-27T11:18:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.429+0000 7fd2521f2700 1 --2- 172.21.15.112:0/1811475486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd24c06e150 0x7fd24c0cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.429+0000 7fd2521f2700 1 --2- 172.21.15.112:0/1811475486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c13f990 0x7fd24c06e530 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.429+0000 7fd2521f2700 1 -- 172.21.15.112:0/1811475486 >> 172.21.15.112:0/1811475486 conn(0x7fd24c06cd30 msgr2=0x7fd24c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.429+0000 7fd2521f2700 1 -- 172.21.15.112:0/1811475486 shutdown_connections 2023-08-27T11:18:13.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.429+0000 7fd2521f2700 1 -- 172.21.15.112:0/1811475486 wait complete. 2023-08-27T11:18:13.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.430+0000 7fd2521f2700 1 Processor -- start 2023-08-27T11:18:13.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.431+0000 7fd2521f2700 1 -- start start 2023-08-27T11:18:13.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.431+0000 7fd2521f2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd24c06e150 0x7fd24c154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:13.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.431+0000 7fd2521f2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd24c0cd270 0x7fd24c0ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:13.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.431+0000 7fd2521f2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c13f990 0x7fd24c15bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:13.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.431+0000 7fd2521f2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c1546b0 con 0x7fd24c06e150 2023-08-27T11:18:13.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.431+0000 7fd2521f2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0ac200 con 0x7fd24c13f990 2023-08-27T11:18:13.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.431+0000 7fd2521f2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0ac4e0 con 0x7fd24c0cd270 2023-08-27T11:18:13.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.431+0000 7fd24b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd24c06e150 0x7fd24c154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:13.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.431+0000 7fd24b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd24c06e150 0x7fd24c154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57024/0 (socket says 172.21.15.112:57024) 2023-08-27T11:18:13.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.431+0000 7fd24b7fe700 1 -- 172.21.15.112:0/4108897663 learned_addr learned my addr 172.21.15.112:0/4108897663 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:13.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd24bfff700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c13f990 0x7fd24c15bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:13.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd24affd700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd24c0cd270 0x7fd24c0ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:13.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd24b7fe700 1 -- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd24c0cd270 msgr2=0x7fd24c0ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:13.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd24b7fe700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd24c0cd270 0x7fd24c0ab890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd24b7fe700 1 -- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c13f990 msgr2=0x7fd24c15bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:13.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd24b7fe700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c13f990 0x7fd24c15bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd24b7fe700 1 -- 172.21.15.112:0/4108897663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c0094a0 con 0x7fd24c06e150 2023-08-27T11:18:13.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd24bfff700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c13f990 0x7fd24c15bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:13.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd24affd700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd24c0cd270 0x7fd24c0ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:13.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd24b7fe700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd24c06e150 0x7fd24c154170 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00ebe0 tx=0x7fd23c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:13.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd233fff700 1 -- 172.21.15.112:0/4108897663 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c00c910 con 0x7fd24c06e150 2023-08-27T11:18:13.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd233fff700 1 -- 172.21.15.112:0/4108897663 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c00ca70 con 0x7fd24c06e150 2023-08-27T11:18:13.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd2521f2700 1 -- 172.21.15.112:0/4108897663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd240009430 con 0x7fd24c06e150 2023-08-27T11:18:13.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd233fff700 1 -- 172.21.15.112:0/4108897663 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c00cde0 con 0x7fd24c06e150 2023-08-27T11:18:13.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.432+0000 7fd2521f2700 1 -- 172.21.15.112:0/4108897663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c0acaa0 con 0x7fd24c06e150 2023-08-27T11:18:13.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.433+0000 7fd2521f2700 1 -- 172.21.15.112:0/4108897663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24c067a20 con 0x7fd24c06e150 2023-08-27T11:18:13.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.433+0000 7fd233fff700 1 -- 172.21.15.112:0/4108897663 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd23c013470 con 0x7fd24c06e150 2023-08-27T11:18:13.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.434+0000 7fd233fff700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd23806a0e0 0x7fd23806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:13.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.434+0000 7fd233fff700 1 -- 172.21.15.112:0/4108897663 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd23c0886c0 con 0x7fd24c06e150 2023-08-27T11:18:13.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.434+0000 7fd24affd700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd23806a0e0 0x7fd23806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:13.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.435+0000 7fd24affd700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd23806a0e0 0x7fd23806c5a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd240013040 tx=0x7fd240008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:13.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.438+0000 7fd233fff700 1 -- 172.21.15.112:0/4108897663 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd23c053e40 con 0x7fd24c06e150 2023-08-27T11:18:13.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.654+0000 7fd2521f2700 1 -- 172.21.15.112:0/4108897663 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd24c140510 con 0x7fd23806a0e0 2023-08-27T11:18:13.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.657+0000 7fd233fff700 1 -- 172.21.15.112:0/4108897663 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd240021070 con 0x7fd23806a0e0 2023-08-27T11:18:13.658 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:18:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs" 2023-08-27T11:18:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:13.659 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:18:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.659+0000 7fd2521f2700 1 -- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd23806a0e0 msgr2=0x7fd23806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.659+0000 7fd2521f2700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd23806a0e0 0x7fd23806c5a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd240013040 tx=0x7fd240008070 comp rx=0 tx=0).stop 2023-08-27T11:18:13.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.659+0000 7fd2521f2700 1 -- 172.21.15.112:0/4108897663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd24c06e150 msgr2=0x7fd24c154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:13.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.659+0000 7fd2521f2700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd24c06e150 0x7fd24c154170 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00ebe0 tx=0x7fd23c009ec0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.659+0000 7fd2521f2700 1 -- 172.21.15.112:0/4108897663 shutdown_connections 2023-08-27T11:18:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.659+0000 7fd2521f2700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd23806a0e0 0x7fd23806c5a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.659+0000 7fd2521f2700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c13f990 0x7fd24c15bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.659+0000 7fd2521f2700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd24c06e150 0x7fd24c154170 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.659+0000 7fd2521f2700 1 --2- 172.21.15.112:0/4108897663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd24c0cd270 0x7fd24c0ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.659+0000 7fd2521f2700 1 -- 172.21.15.112:0/4108897663 >> 172.21.15.112:0/4108897663 conn(0x7fd24c06cd30 msgr2=0x7fd24c0cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.659+0000 7fd2521f2700 1 -- 172.21.15.112:0/4108897663 shutdown_connections 2023-08-27T11:18:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.659+0000 7fd2521f2700 1 -- 172.21.15.112:0/4108897663 wait complete. 2023-08-27T11:18:13.672 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:18:13.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.794+0000 7f9767b7c700 1 Processor -- start 2023-08-27T11:18:13.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.794+0000 7f9767b7c700 1 -- start start 2023-08-27T11:18:13.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.795+0000 7f9767b7c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9760146bf0 0x7f97600b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:13.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.795+0000 7f9767b7c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9760070eb0 0x7f9760071320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:13.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.795+0000 7f9767b7c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97600d1500 con 0x7f9760146fd0 2023-08-27T11:18:13.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.795+0000 7f9767b7c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9760071860 con 0x7f9760146bf0 2023-08-27T11:18:13.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.795+0000 7f9767b7c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97600d0460 con 0x7f9760070eb0 2023-08-27T11:18:13.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.795+0000 7f9765918700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9760146bf0 0x7f97600b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:13.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.795+0000 7f9765117700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9760146fd0 0x7f97600cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51416/0 (socket says 172.21.15.112:51416) 2023-08-27T11:18:13.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.795+0000 7f9765117700 1 -- 172.21.15.112:0/77182946 learned_addr learned my addr 172.21.15.112:0/77182946 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:13.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9766119700 1 --2- 172.21.15.112:0/77182946 >> v2:172.21.15.138:3300/0 conn(0x7f9760070eb0 0x7f9760071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:13.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9765918700 1 -- 172.21.15.112:0/77182946 >> v2:172.21.15.138:3300/0 conn(0x7f9760070eb0 msgr2=0x7f9760071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:13.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9765918700 1 --2- 172.21.15.112:0/77182946 >> v2:172.21.15.138:3300/0 conn(0x7f9760070eb0 0x7f9760071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9765918700 1 -- 172.21.15.112:0/77182946 >> v1:172.21.15.112:6789/0 conn(0x7f9760146fd0 legacy=0x7f97600cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:13.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9765918700 1 -- 172.21.15.112:0/77182946 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97600d0710 con 0x7f9760146bf0 2023-08-27T11:18:13.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9764916700 1 -- 172.21.15.112:0/77182946 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530602257 0 0) 0x7f97600d1500 con 0x7f9760146fd0 2023-08-27T11:18:13.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9765918700 1 --2- 172.21.15.112:0/77182946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760146bf0 0x7f97600b5db0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f97600b58f0 tx=0x7f975c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd89ceec4ecbcd6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:13.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9764916700 1 -- 172.21.15.112:0/77182946 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c011830 con 0x7f9760146bf0 2023-08-27T11:18:13.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9764916700 1 -- 172.21.15.112:0/77182946 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c011990 con 0x7f9760146bf0 2023-08-27T11:18:13.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9764916700 1 -- 172.21.15.112:0/77182946 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c011d20 con 0x7f9760146bf0 2023-08-27T11:18:13.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9767b7c700 1 -- 172.21.15.112:0/77182946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760146bf0 msgr2=0x7f97600b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:13.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9767b7c700 1 --2- 172.21.15.112:0/77182946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760146bf0 0x7f97600b5db0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f97600b58f0 tx=0x7f975c009b20 comp rx=0 tx=0).stop 2023-08-27T11:18:13.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9767b7c700 1 -- 172.21.15.112:0/77182946 shutdown_connections 2023-08-27T11:18:13.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.796+0000 7f9767b7c700 1 --2- 172.21.15.112:0/77182946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760146bf0 0x7f97600b5db0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.797+0000 7f9767b7c700 1 --2- 172.21.15.112:0/77182946 >> v2:172.21.15.138:3300/0 conn(0x7f9760070eb0 0x7f9760071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.797+0000 7f9767b7c700 1 -- 172.21.15.112:0/77182946 >> 172.21.15.112:0/77182946 conn(0x7f976006c450 msgr2=0x7f976006f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:13.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.797+0000 7f9767b7c700 1 -- 172.21.15.112:0/77182946 shutdown_connections 2023-08-27T11:18:13.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.797+0000 7f9767b7c700 1 -- 172.21.15.112:0/77182946 wait complete. 2023-08-27T11:18:13.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.798+0000 7f9767b7c700 1 Processor -- start 2023-08-27T11:18:13.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.798+0000 7f9767b7c700 1 -- start start 2023-08-27T11:18:13.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.798+0000 7f9767b7c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9760070eb0 0x7f976013cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:13.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.798+0000 7f9767b7c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760146bf0 0x7f9760159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:13.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.798+0000 7f9767b7c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9760146fd0 0x7f9760142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:13.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.798+0000 7f9767b7c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97601595a0 con 0x7f9760146bf0 2023-08-27T11:18:13.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.798+0000 7f9767b7c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97600b85b0 con 0x7f9760146fd0 2023-08-27T11:18:13.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.798+0000 7f9767b7c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97600b8890 con 0x7f9760070eb0 2023-08-27T11:18:13.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9765117700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760146bf0 0x7f9760159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:13.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9765117700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760146bf0 0x7f9760159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57038/0 (socket says 172.21.15.112:57038) 2023-08-27T11:18:13.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9765117700 1 -- 172.21.15.112:0/53013919 learned_addr learned my addr 172.21.15.112:0/53013919 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:13.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9766119700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9760146fd0 0x7f9760142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:13.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9765918700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9760070eb0 0x7f976013cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:13.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9765117700 1 -- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9760070eb0 msgr2=0x7f976013cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:13.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9765117700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9760070eb0 0x7f976013cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9765117700 1 -- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9760146fd0 msgr2=0x7f9760142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:13.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9765117700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9760146fd0 0x7f9760142570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:13.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9765117700 1 -- 172.21.15.112:0/53013919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c0094a0 con 0x7f9760146bf0 2023-08-27T11:18:13.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9765918700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9760070eb0 0x7f976013cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:13.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9765117700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760146bf0 0x7f9760159060 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f975000c550 tx=0x7f975000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:13.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f97567fc700 1 -- 172.21.15.112:0/53013919 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750004c80 con 0x7f9760146bf0 2023-08-27T11:18:13.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f97567fc700 1 -- 172.21.15.112:0/53013919 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9750004e00 con 0x7f9760146bf0 2023-08-27T11:18:13.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9767b7c700 1 -- 172.21.15.112:0/53013919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97600b8b70 con 0x7f9760146bf0 2023-08-27T11:18:13.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f97567fc700 1 -- 172.21.15.112:0/53013919 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750007310 con 0x7f9760146bf0 2023-08-27T11:18:13.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.799+0000 7f9767b7c700 1 -- 172.21.15.112:0/53013919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976013dee0 con 0x7f9760146bf0 2023-08-27T11:18:13.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.800+0000 7f9767b7c700 1 -- 172.21.15.112:0/53013919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9760067e90 con 0x7f9760146bf0 2023-08-27T11:18:13.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.801+0000 7f97567fc700 1 -- 172.21.15.112:0/53013919 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9750007470 con 0x7f9760146bf0 2023-08-27T11:18:13.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.801+0000 7f97567fc700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f974406a0e0 0x7f974406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:13.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.801+0000 7f97567fc700 1 -- 172.21.15.112:0/53013919 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9750088270 con 0x7f9760146bf0 2023-08-27T11:18:13.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.801+0000 7f9765918700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f974406a0e0 0x7f974406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:13.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.802+0000 7f9765918700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f974406a0e0 0x7f974406c5a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f975c0090b0 tx=0x7f975c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:13.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.805+0000 7f97567fc700 1 -- 172.21.15.112:0/53013919 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9750053960 con 0x7f9760146bf0 2023-08-27T11:18:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.997+0000 7f9767b7c700 1 -- 172.21.15.112:0/53013919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f976013e0d0 con 0x7f9760146bf0 2023-08-27T11:18:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:13.997+0000 7f97567fc700 1 -- 172.21.15.112:0/53013919 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9750056f80 con 0x7f9760146bf0 2023-08-27T11:18:14.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.000+0000 7f9767b7c700 1 -- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f974406a0e0 msgr2=0x7f974406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:14.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.000+0000 7f9767b7c700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f974406a0e0 0x7f974406c5a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f975c0090b0 tx=0x7f975c008030 comp rx=0 tx=0).stop 2023-08-27T11:18:14.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.000+0000 7f9767b7c700 1 -- 172.21.15.112:0/53013919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760146bf0 msgr2=0x7f9760159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:14.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.000+0000 7f9767b7c700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760146bf0 0x7f9760159060 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f975000c550 tx=0x7f975000c910 comp rx=0 tx=0).stop 2023-08-27T11:18:14.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.000+0000 7f9767b7c700 1 -- 172.21.15.112:0/53013919 shutdown_connections 2023-08-27T11:18:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.000+0000 7f9767b7c700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f974406a0e0 0x7f974406c5a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.000+0000 7f9767b7c700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9760146fd0 0x7f9760142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.000+0000 7f9767b7c700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9760070eb0 0x7f976013cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.000+0000 7f9767b7c700 1 --2- 172.21.15.112:0/53013919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760146bf0 0x7f9760159060 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.000+0000 7f9767b7c700 1 -- 172.21.15.112:0/53013919 >> 172.21.15.112:0/53013919 conn(0x7f976006c450 msgr2=0x7f9760143bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.000+0000 7f9767b7c700 1 -- 172.21.15.112:0/53013919 shutdown_connections 2023-08-27T11:18:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.000+0000 7f9767b7c700 1 -- 172.21.15.112:0/53013919 wait complete. 2023-08-27T11:18:14.014 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000766396 2023-08-27T11:18:14.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.135+0000 7f5d86ccf700 1 Processor -- start 2023-08-27T11:18:14.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.135+0000 7f5d86ccf700 1 -- start start 2023-08-27T11:18:14.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d86ccf700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5d8015e190 0x7f5d8015e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:14.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d86ccf700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d800b46c0 con 0x7f5d800cc980 2023-08-27T11:18:14.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d86ccf700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d800b49a0 con 0x7f5d80161dd0 2023-08-27T11:18:14.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d86ccf700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d800b44d0 con 0x7f5d8015e190 2023-08-27T11:18:14.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d8526c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5d800cc980 0x7f5d800ce680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51432/0 (socket says 172.21.15.112:51432) 2023-08-27T11:18:14.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d8526c700 1 -- 172.21.15.112:0/84872217 learned_addr learned my addr 172.21.15.112:0/84872217 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:14.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d84a6b700 1 --2- 172.21.15.112:0/84872217 >> v2:172.21.15.138:3300/0 conn(0x7f5d8015e190 0x7f5d8015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:14.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634762759 0 0) 0x7f5d800b46c0 con 0x7f5d800cc980 2023-08-27T11:18:14.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d700035b0 con 0x7f5d800cc980 2023-08-27T11:18:14.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2523558159 0 0) 0x7f5d700035b0 con 0x7f5d800cc980 2023-08-27T11:18:14.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 >> v2:172.21.15.138:3300/0 conn(0x7f5d8015e190 msgr2=0x7f5d8015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:14.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 --2- 172.21.15.112:0/84872217 >> v2:172.21.15.138:3300/0 conn(0x7f5d8015e190 0x7f5d8015e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:14.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 >> v1:172.21.15.138:6789/0 conn(0x7f5d80161dd0 legacy=0x7f5d800b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:14.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d801612f0 con 0x7f5d800cc980 2023-08-27T11:18:14.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f5d780019f0 con 0x7f5d800cc980 2023-08-27T11:18:14.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 >> v1:172.21.15.112:6789/0 conn(0x7f5d800cc980 legacy=0x7f5d800ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:14.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 --2- 172.21.15.112:0/84872217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d70003a00 0x7f5d70003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:14.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 --2- 172.21.15.112:0/84872217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d70004400 0x7f5d700048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:14.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 --2- 172.21.15.112:0/84872217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d700050d0 0x7f5d7000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:14.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7000c000 con 0x7f5d70004400 2023-08-27T11:18:14.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7000c170 con 0x7f5d70003a00 2023-08-27T11:18:14.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7000c330 con 0x7f5d700050d0 2023-08-27T11:18:14.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.136+0000 7f5d7ffff700 1 --2- 172.21.15.112:0/84872217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d70004400 0x7f5d700048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:14.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d84a6b700 1 --2- 172.21.15.112:0/84872217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d70003a00 0x7f5d70003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:14.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d8526c700 1 --2- 172.21.15.112:0/84872217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d700050d0 0x7f5d7000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:14.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d7ffff700 1 -- 172.21.15.112:0/84872217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d700050d0 msgr2=0x7f5d7000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:14.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d7ffff700 1 --2- 172.21.15.112:0/84872217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d700050d0 0x7f5d7000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:14.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d7ffff700 1 -- 172.21.15.112:0/84872217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d70003a00 msgr2=0x7f5d70003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:14.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d7ffff700 1 --2- 172.21.15.112:0/84872217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d70003a00 0x7f5d70003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:14.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d7ffff700 1 -- 172.21.15.112:0/84872217 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d700035b0 con 0x7f5d70004400 2023-08-27T11:18:14.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d8526c700 1 --2- 172.21.15.112:0/84872217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d700050d0 0x7f5d7000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:14.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d8526c700 1 -- 172.21.15.112:0/84872217 reap_dead start 2023-08-27T11:18:14.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d7ffff700 1 --2- 172.21.15.112:0/84872217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d70004400 0x7f5d700048d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c0018d0 tx=0x7f5d70005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9641029cb4144a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:14.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c0052d0 con 0x7f5d70004400 2023-08-27T11:18:14.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6c005430 con 0x7f5d70004400 2023-08-27T11:18:14.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d7f7fe700 1 -- 172.21.15.112:0/84872217 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c0038a0 con 0x7f5d70004400 2023-08-27T11:18:14.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d86ccf700 1 -- 172.21.15.112:0/84872217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d70004400 msgr2=0x7f5d700048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:14.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.137+0000 7f5d86ccf700 1 --2- 172.21.15.112:0/84872217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d70004400 0x7f5d700048d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c0018d0 tx=0x7f5d70005880 comp rx=0 tx=0).stop 2023-08-27T11:18:14.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.138+0000 7f5d86ccf700 1 -- 172.21.15.112:0/84872217 shutdown_connections 2023-08-27T11:18:14.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.138+0000 7f5d86ccf700 1 --2- 172.21.15.112:0/84872217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d70004400 0x7f5d700048d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:14.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.138+0000 7f5d86ccf700 1 -- 172.21.15.112:0/84872217 >> 172.21.15.112:0/84872217 conn(0x7f5d8006bbf0 msgr2=0x7f5d800cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:14.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.138+0000 7f5d86ccf700 1 -- 172.21.15.112:0/84872217 shutdown_connections 2023-08-27T11:18:14.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.138+0000 7f5d86ccf700 1 -- 172.21.15.112:0/84872217 wait complete. 2023-08-27T11:18:14.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.139+0000 7f5d86ccf700 1 Processor -- start 2023-08-27T11:18:14.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.139+0000 7f5d86ccf700 1 -- start start 2023-08-27T11:18:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d86ccf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d70004400 0x7f5d80141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d86ccf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d801430b0 0x7f5d801f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d86ccf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d80072120 0x7f5d80156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d86ccf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d801440e0 con 0x7f5d801430b0 2023-08-27T11:18:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d86ccf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d800ae8f0 con 0x7f5d70004400 2023-08-27T11:18:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d86ccf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d800ae570 con 0x7f5d80072120 2023-08-27T11:18:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d7ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d801430b0 0x7f5d801f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d7ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d801430b0 0x7f5d801f68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57060/0 (socket says 172.21.15.112:57060) 2023-08-27T11:18:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d7ffff700 1 -- 172.21.15.112:0/2698235407 learned_addr learned my addr 172.21.15.112:0/2698235407 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d8526c700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d80072120 0x7f5d80156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d84a6b700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d70004400 0x7f5d80141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d7ffff700 1 -- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d80072120 msgr2=0x7f5d80156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d7ffff700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d80072120 0x7f5d80156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d7ffff700 1 -- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d70004400 msgr2=0x7f5d80141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:14.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d7ffff700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d70004400 0x7f5d80141060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:14.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d7ffff700 1 -- 172.21.15.112:0/2698235407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d801f6e20 con 0x7f5d801430b0 2023-08-27T11:18:14.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d8526c700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d80072120 0x7f5d80156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:14.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d84a6b700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d70004400 0x7f5d80141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:14.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d7ffff700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d801430b0 0x7f5d801f68e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c003f70 tx=0x7f5d6c001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:14.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d7d7fa700 1 -- 172.21.15.112:0/2698235407 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c00c690 con 0x7f5d801430b0 2023-08-27T11:18:14.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d7d7fa700 1 -- 172.21.15.112:0/2698235407 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6c00c810 con 0x7f5d801430b0 2023-08-27T11:18:14.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d7d7fa700 1 -- 172.21.15.112:0/2698235407 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c00cbc0 con 0x7f5d801430b0 2023-08-27T11:18:14.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d86ccf700 1 -- 172.21.15.112:0/2698235407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d80143840 con 0x7f5d801430b0 2023-08-27T11:18:14.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.140+0000 7f5d86ccf700 1 -- 172.21.15.112:0/2698235407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d80143e00 con 0x7f5d801430b0 2023-08-27T11:18:14.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.141+0000 7f5d86ccf700 1 -- 172.21.15.112:0/2698235407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d80066fa0 con 0x7f5d801430b0 2023-08-27T11:18:14.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.142+0000 7f5d7d7fa700 1 -- 172.21.15.112:0/2698235407 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5d6c00cd20 con 0x7f5d801430b0 2023-08-27T11:18:14.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.142+0000 7f5d7d7fa700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5d6406a0e0 0x7f5d6406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:14.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.142+0000 7f5d7d7fa700 1 -- 172.21.15.112:0/2698235407 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d6c089370 con 0x7f5d801430b0 2023-08-27T11:18:14.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.142+0000 7f5d84a6b700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5d6406a0e0 0x7f5d6406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:14.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.143+0000 7f5d84a6b700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5d6406a0e0 0x7f5d6406c5a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009ce0 tx=0x7f5d74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:14.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.147+0000 7f5d7d7fa700 1 -- 172.21.15.112:0/2698235407 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5d6c054af0 con 0x7f5d801430b0 2023-08-27T11:18:14.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:14.364+0000 7f5d86ccf700 1 -- 172.21.15.112:0/2698235407 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000766396", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d801618b0 con 0x7f5d6406a0e0 2023-08-27T11:18:17.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.333+0000 7f5d7d7fa700 1 -- 172.21.15.112:0/2698235407 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f5d74009e20 con 0x7f5d6406a0e0 2023-08-27T11:18:17.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.336+0000 7f5d86ccf700 1 -- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5d6406a0e0 msgr2=0x7f5d6406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:17.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.336+0000 7f5d86ccf700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5d6406a0e0 0x7f5d6406c5a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009ce0 tx=0x7f5d74008040 comp rx=0 tx=0).stop 2023-08-27T11:18:17.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.336+0000 7f5d86ccf700 1 -- 172.21.15.112:0/2698235407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d801430b0 msgr2=0x7f5d801f68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:17.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.336+0000 7f5d86ccf700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d801430b0 0x7f5d801f68e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c003f70 tx=0x7f5d6c001230 comp rx=0 tx=0).stop 2023-08-27T11:18:17.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.336+0000 7f5d86ccf700 1 -- 172.21.15.112:0/2698235407 shutdown_connections 2023-08-27T11:18:17.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.336+0000 7f5d86ccf700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5d6406a0e0 0x7f5d6406c5a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.336+0000 7f5d86ccf700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d80072120 0x7f5d80156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.336+0000 7f5d86ccf700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d70004400 0x7f5d80141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.336+0000 7f5d86ccf700 1 --2- 172.21.15.112:0/2698235407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d801430b0 0x7f5d801f68e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.336+0000 7f5d86ccf700 1 -- 172.21.15.112:0/2698235407 >> 172.21.15.112:0/2698235407 conn(0x7f5d8006bbf0 msgr2=0x7f5d80152b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:17.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.336+0000 7f5d86ccf700 1 -- 172.21.15.112:0/2698235407 shutdown_connections 2023-08-27T11:18:17.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.336+0000 7f5d86ccf700 1 -- 172.21.15.112:0/2698235407 wait complete. 2023-08-27T11:18:17.342 INFO:teuthology.orchestra.run.smithi112.stderr:Volume created successfully (no MDS daemons created) 2023-08-27T11:18:17.351 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:18:17.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.474+0000 7f515f24b700 1 Processor -- start 2023-08-27T11:18:17.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.474+0000 7f515f24b700 1 -- start start 2023-08-27T11:18:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f515f24b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f51580b1400 0x7f51580cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f515f24b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5158161ce0 0x7f515806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f515f24b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5158161850 con 0x7f51580b17e0 2023-08-27T11:18:17.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f515f24b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51580cfc10 con 0x7f51580b1400 2023-08-27T11:18:17.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f515f24b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5158161200 con 0x7f5158161ce0 2023-08-27T11:18:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f515cfe7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f51580b1400 0x7f51580cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f515d7e8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5158161ce0 0x7f515806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f5157fff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f51580b17e0 0x7f515806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:58990/0 (socket says 172.21.15.112:58990) 2023-08-27T11:18:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f5157fff700 1 -- 172.21.15.112:0/2842178280 learned_addr learned my addr 172.21.15.112:0/2842178280 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f515cfe7700 1 -- 172.21.15.112:0/2842178280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5158161ce0 msgr2=0x7f515806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:17.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f515cfe7700 1 --2- 172.21.15.112:0/2842178280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5158161ce0 0x7f515806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f515cfe7700 1 -- 172.21.15.112:0/2842178280 >> v1:172.21.15.138:6789/0 conn(0x7f51580b17e0 legacy=0x7f515806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:17.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f515cfe7700 1 -- 172.21.15.112:0/2842178280 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51581614e0 con 0x7f51580b1400 2023-08-27T11:18:17.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.475+0000 7f515d7e8700 1 --2- 172.21.15.112:0/2842178280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5158161ce0 0x7f515806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:17.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.476+0000 7f515cfe7700 1 --2- 172.21.15.112:0/2842178280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51580b1400 0x7f51580cf5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f514c009840 tx=0x7f514c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffb589894fdef6ad server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:17.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.476+0000 7f51577fe700 1 -- 172.21.15.112:0/2842178280 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c011830 con 0x7f51580b1400 2023-08-27T11:18:17.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.476+0000 7f51577fe700 1 -- 172.21.15.112:0/2842178280 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c011990 con 0x7f51580b1400 2023-08-27T11:18:17.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.476+0000 7f51577fe700 1 -- 172.21.15.112:0/2842178280 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c011d20 con 0x7f51580b1400 2023-08-27T11:18:17.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.476+0000 7f515f24b700 1 -- 172.21.15.112:0/2842178280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51580b1400 msgr2=0x7f51580cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:17.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.476+0000 7f515f24b700 1 --2- 172.21.15.112:0/2842178280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51580b1400 0x7f51580cf5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f514c009840 tx=0x7f514c009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:17.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.476+0000 7f515f24b700 1 -- 172.21.15.112:0/2842178280 shutdown_connections 2023-08-27T11:18:17.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.476+0000 7f515f24b700 1 --2- 172.21.15.112:0/2842178280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5158161ce0 0x7f515806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.476+0000 7f515f24b700 1 --2- 172.21.15.112:0/2842178280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51580b1400 0x7f51580cf5a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.476+0000 7f515f24b700 1 -- 172.21.15.112:0/2842178280 >> 172.21.15.112:0/2842178280 conn(0x7f51580d0000 msgr2=0x7f51580d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:17.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.476+0000 7f515f24b700 1 -- 172.21.15.112:0/2842178280 shutdown_connections 2023-08-27T11:18:17.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.476+0000 7f515f24b700 1 -- 172.21.15.112:0/2842178280 wait complete. 2023-08-27T11:18:17.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.478+0000 7f515f24b700 1 Processor -- start 2023-08-27T11:18:17.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.478+0000 7f515f24b700 1 -- start start 2023-08-27T11:18:17.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.478+0000 7f515f24b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51580b1400 0x7f5158142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.478+0000 7f515f24b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51580b17e0 0x7f51581f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515f24b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5158161ce0 0x7f5158152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515f24b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5158070880 con 0x7f5158161ce0 2023-08-27T11:18:17.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515f24b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5158070530 con 0x7f51580b17e0 2023-08-27T11:18:17.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515f24b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51581f66f0 con 0x7f51580b1400 2023-08-27T11:18:17.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515d7e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5158161ce0 0x7f5158152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515d7e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5158161ce0 0x7f5158152950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57092/0 (socket says 172.21.15.112:57092) 2023-08-27T11:18:17.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515d7e8700 1 -- 172.21.15.112:0/4016157476 learned_addr learned my addr 172.21.15.112:0/4016157476 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:17.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f5157fff700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51580b17e0 0x7f51581f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515cfe7700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51580b1400 0x7f5158142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515d7e8700 1 -- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51580b1400 msgr2=0x7f5158142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:17.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515d7e8700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51580b1400 0x7f5158142810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515d7e8700 1 -- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51580b17e0 msgr2=0x7f51581f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:17.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515d7e8700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51580b17e0 0x7f51581f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515d7e8700 1 -- 172.21.15.112:0/4016157476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5150009a70 con 0x7f5158161ce0 2023-08-27T11:18:17.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515d7e8700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5158161ce0 0x7f5158152950 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f515000efc0 tx=0x7f515000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:17.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f51557fa700 1 -- 172.21.15.112:0/4016157476 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515000ca90 con 0x7f5158161ce0 2023-08-27T11:18:17.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f51557fa700 1 -- 172.21.15.112:0/4016157476 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515000f040 con 0x7f5158161ce0 2023-08-27T11:18:17.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f51557fa700 1 -- 172.21.15.112:0/4016157476 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515000ce30 con 0x7f5158161ce0 2023-08-27T11:18:17.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515f24b700 1 -- 172.21.15.112:0/4016157476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c0094a0 con 0x7f5158161ce0 2023-08-27T11:18:17.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.479+0000 7f515f24b700 1 -- 172.21.15.112:0/4016157476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51581f6c80 con 0x7f5158161ce0 2023-08-27T11:18:17.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.480+0000 7f515f24b700 1 -- 172.21.15.112:0/4016157476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51580709c0 con 0x7f5158161ce0 2023-08-27T11:18:17.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.481+0000 7f51557fa700 1 -- 172.21.15.112:0/4016157476 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5150014480 con 0x7f5158161ce0 2023-08-27T11:18:17.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.481+0000 7f51557fa700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f513c06a0e0 0x7f513c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.481+0000 7f51557fa700 1 -- 172.21.15.112:0/4016157476 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6960+0+0 (secure 0 0 0) 0x7f5150088f50 con 0x7f5158161ce0 2023-08-27T11:18:17.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.481+0000 7f515cfe7700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f513c06a0e0 0x7f513c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.482+0000 7f515cfe7700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f513c06a0e0 0x7f513c06c5a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f514c000c00 tx=0x7f514c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:17.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.486+0000 7f51557fa700 1 -- 172.21.15.112:0/4016157476 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f51580709c0 con 0x7f5158161ce0 2023-08-27T11:18:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.700+0000 7f515f24b700 1 -- 172.21.15.112:0/4016157476 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f51580709c0 con 0x7f513c06a0e0 2023-08-27T11:18:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.702+0000 7f51557fa700 1 -- 172.21.15.112:0/4016157476 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7f51580709c0 con 0x7f513c06a0e0 2023-08-27T11:18:17.704 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:18:17.704 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:17.704 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs" 2023-08-27T11:18:17.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:17.705 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:17.705 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volume_0000000000766396" 2023-08-27T11:18:17.705 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:17.705 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:18:17.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.704+0000 7f515f24b700 1 -- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f513c06a0e0 msgr2=0x7f513c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:17.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.704+0000 7f515f24b700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f513c06a0e0 0x7f513c06c5a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f514c000c00 tx=0x7f514c008030 comp rx=0 tx=0).stop 2023-08-27T11:18:17.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.704+0000 7f515f24b700 1 -- 172.21.15.112:0/4016157476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5158161ce0 msgr2=0x7f5158152950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:17.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.704+0000 7f515f24b700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5158161ce0 0x7f5158152950 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f515000efc0 tx=0x7f515000c3a0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.704+0000 7f515f24b700 1 -- 172.21.15.112:0/4016157476 shutdown_connections 2023-08-27T11:18:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.704+0000 7f515f24b700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f513c06a0e0 0x7f513c06c5a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.704+0000 7f515f24b700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5158161ce0 0x7f5158152950 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.704+0000 7f515f24b700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51580b1400 0x7f5158142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.704+0000 7f515f24b700 1 --2- 172.21.15.112:0/4016157476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51580b17e0 0x7f51581f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.704+0000 7f515f24b700 1 -- 172.21.15.112:0/4016157476 >> 172.21.15.112:0/4016157476 conn(0x7f51580d0000 msgr2=0x7f515815eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.704+0000 7f515f24b700 1 -- 172.21.15.112:0/4016157476 shutdown_connections 2023-08-27T11:18:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.704+0000 7f515f24b700 1 -- 172.21.15.112:0/4016157476 wait complete. 2023-08-27T11:18:17.717 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool ls detail --format=json 2023-08-27T11:18:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.838+0000 7f587df88700 1 Processor -- start 2023-08-27T11:18:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.838+0000 7f587df88700 1 -- start start 2023-08-27T11:18:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f587df88700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587806e1d0 0x7f58780b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f587df88700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587806e770 con 0x7f587806f790 2023-08-27T11:18:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f587df88700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587815b910 con 0x7f58780cfa90 2023-08-27T11:18:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f587df88700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58781444b0 con 0x7f587806e1d0 2023-08-27T11:18:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f5877fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f587806f790 0x7f587815c0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51482/0 (socket says 172.21.15.112:51482) 2023-08-27T11:18:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f5877fff700 1 -- 172.21.15.112:0/2354748837 learned_addr learned my addr 172.21.15.112:0/2354748837 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1479724210 0 0) 0x7f587806e770 con 0x7f587806f790 2023-08-27T11:18:17.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f5876ffd700 1 --2- 172.21.15.112:0/2354748837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587806e1d0 0x7f58780b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58600035b0 con 0x7f587806f790 2023-08-27T11:18:17.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1712078751 0 0) 0x7f58600035b0 con 0x7f587806f790 2023-08-27T11:18:17.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587806e1d0 msgr2=0x7f58780b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:17.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f58767fc700 1 --2- 172.21.15.112:0/2354748837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587806e1d0 0x7f58780b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 >> v1:172.21.15.138:6789/0 conn(0x7f58780cfa90 legacy=0x7f58780b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:17.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5878144e10 con 0x7f587806f790 2023-08-27T11:18:17.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.839+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f58700019f0 con 0x7f587806f790 2023-08-27T11:18:17.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 >> v1:172.21.15.112:6789/0 conn(0x7f587806f790 legacy=0x7f587815c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f58767fc700 1 --2- 172.21.15.112:0/2354748837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587806e1d0 0x7f5860003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f58767fc700 1 --2- 172.21.15.112:0/2354748837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5860004060 0x7f5860004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f58767fc700 1 --2- 172.21.15.112:0/2354748837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5860004d90 0x7f5860005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58781444b0 con 0x7f5860004060 2023-08-27T11:18:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586000bbd0 con 0x7f5860004d90 2023-08-27T11:18:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586000bd90 con 0x7f587806e1d0 2023-08-27T11:18:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f5876ffd700 1 --2- 172.21.15.112:0/2354748837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5860004060 0x7f5860004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f5877fff700 1 --2- 172.21.15.112:0/2354748837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5860004d90 0x7f5860005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f58777fe700 1 --2- 172.21.15.112:0/2354748837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587806e1d0 0x7f5860003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f5876ffd700 1 -- 172.21.15.112:0/2354748837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587806e1d0 msgr2=0x7f5860003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:17.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f5876ffd700 1 --2- 172.21.15.112:0/2354748837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587806e1d0 0x7f5860003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f5876ffd700 1 -- 172.21.15.112:0/2354748837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5860004d90 msgr2=0x7f5860005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:17.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f5876ffd700 1 --2- 172.21.15.112:0/2354748837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5860004d90 0x7f5860005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f5876ffd700 1 -- 172.21.15.112:0/2354748837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58600035b0 con 0x7f5860004060 2023-08-27T11:18:17.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f58777fe700 1 --2- 172.21.15.112:0/2354748837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587806e1d0 0x7f5860003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:17.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f5876ffd700 1 --2- 172.21.15.112:0/2354748837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5860004060 0x7f5860004850 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5868009890 tx=0x7f5860005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1b2396c3115cca server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:17.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586800c070 con 0x7f5860004060 2023-08-27T11:18:17.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5868013610 con 0x7f5860004060 2023-08-27T11:18:17.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.840+0000 7f58767fc700 1 -- 172.21.15.112:0/2354748837 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58680139a0 con 0x7f5860004060 2023-08-27T11:18:17.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.841+0000 7f587df88700 1 -- 172.21.15.112:0/2354748837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5860004060 msgr2=0x7f5860004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.841+0000 7f587df88700 1 --2- 172.21.15.112:0/2354748837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5860004060 0x7f5860004850 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5868009890 tx=0x7f5860005880 comp rx=0 tx=0).stop 2023-08-27T11:18:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.841+0000 7f5877fff700 1 -- 172.21.15.112:0/2354748837 reap_dead start 2023-08-27T11:18:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.841+0000 7f587df88700 1 -- 172.21.15.112:0/2354748837 shutdown_connections 2023-08-27T11:18:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.841+0000 7f587df88700 1 -- 172.21.15.112:0/2354748837 >> 172.21.15.112:0/2354748837 conn(0x7f587806bbf0 msgr2=0x7f58780cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.841+0000 7f587df88700 1 -- 172.21.15.112:0/2354748837 shutdown_connections 2023-08-27T11:18:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.841+0000 7f587df88700 1 -- 172.21.15.112:0/2354748837 wait complete. 2023-08-27T11:18:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.842+0000 7f587df88700 1 Processor -- start 2023-08-27T11:18:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.842+0000 7f587df88700 1 -- start start 2023-08-27T11:18:17.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f587df88700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5860004060 0x7f587814ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f587df88700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587814d660 0x7f587815f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f587df88700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5878159640 0x7f5878149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f587df88700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587815fe30 con 0x7f5878159640 2023-08-27T11:18:17.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f587df88700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58781585a0 con 0x7f5860004060 2023-08-27T11:18:17.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f587df88700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5878158880 con 0x7f587814d660 2023-08-27T11:18:17.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f5877fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5878159640 0x7f5878149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f5877fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5878159640 0x7f5878149500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57112/0 (socket says 172.21.15.112:57112) 2023-08-27T11:18:17.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f5877fff700 1 -- 172.21.15.112:0/2391677359 learned_addr learned my addr 172.21.15.112:0/2391677359 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:17.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f58777fe700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5860004060 0x7f587814ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f5876ffd700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587814d660 0x7f587815f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f5877fff700 1 -- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587814d660 msgr2=0x7f587815f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:17.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f5877fff700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587814d660 0x7f587815f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f5877fff700 1 -- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5860004060 msgr2=0x7f587814ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:17.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f5877fff700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5860004060 0x7f587814ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:17.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f5877fff700 1 -- 172.21.15.112:0/2391677359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5878158b60 con 0x7f5878159640 2023-08-27T11:18:17.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f5876ffd700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587814d660 0x7f587815f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:17.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f5877fff700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5878159640 0x7f5878149500 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f58780cda60 tx=0x7f587000a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:17.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f585ffff700 1 -- 172.21.15.112:0/2391677359 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587000ad60 con 0x7f5878159640 2023-08-27T11:18:17.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.843+0000 7f585ffff700 1 -- 172.21.15.112:0/2391677359 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58700133f0 con 0x7f5878159640 2023-08-27T11:18:17.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.844+0000 7f587df88700 1 -- 172.21.15.112:0/2391677359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5878158e40 con 0x7f5878159640 2023-08-27T11:18:17.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.844+0000 7f585ffff700 1 -- 172.21.15.112:0/2391677359 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58700137a0 con 0x7f5878159640 2023-08-27T11:18:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.844+0000 7f587df88700 1 -- 172.21.15.112:0/2391677359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5878158250 con 0x7f5878159640 2023-08-27T11:18:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.845+0000 7f587df88700 1 -- 172.21.15.112:0/2391677359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5878066fa0 con 0x7f5878159640 2023-08-27T11:18:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.845+0000 7f585ffff700 1 -- 172.21.15.112:0/2391677359 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5870013900 con 0x7f5878159640 2023-08-27T11:18:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.845+0000 7f585ffff700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f586406a0e0 0x7f586406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.846+0000 7f585ffff700 1 -- 172.21.15.112:0/2391677359 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6960+0+0 (secure 0 0 0) 0x7f5870089190 con 0x7f5878159640 2023-08-27T11:18:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.846+0000 7f58777fe700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f586406a0e0 0x7f586406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.846+0000 7f58777fe700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f586406a0e0 0x7f586406c5a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f586c000c00 tx=0x7f586c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:17.850+0000 7f585ffff700 1 -- 172.21.15.112:0/2391677359 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5870054500 con 0x7f5878159640 2023-08-27T11:18:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.036+0000 7f587df88700 1 -- 172.21.15.112:0/2391677359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool ls", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f5878067a20 con 0x7f5878159640 2023-08-27T11:18:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.038+0000 7f585ffff700 1 -- 172.21.15.112:0/2391677359 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls", "detail": "detail", "format": "json"}]=0 v36) v1 ==== 97+0+10920 (secure 0 0 0) 0x7f5870057b20 con 0x7f5878159640 2023-08-27T11:18:18.039 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:18.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.039+0000 7f587df88700 1 -- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f586406a0e0 msgr2=0x7f586406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.039+0000 7f587df88700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f586406a0e0 0x7f586406c5a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f586c000c00 tx=0x7f586c009040 comp rx=0 tx=0).stop 2023-08-27T11:18:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.040+0000 7f587df88700 1 -- 172.21.15.112:0/2391677359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5878159640 msgr2=0x7f5878149500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.040+0000 7f587df88700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5878159640 0x7f5878149500 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f58780cda60 tx=0x7f587000a5f0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.040+0000 7f587df88700 1 -- 172.21.15.112:0/2391677359 shutdown_connections 2023-08-27T11:18:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.040+0000 7f587df88700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f586406a0e0 0x7f586406c5a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.040+0000 7f587df88700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5878159640 0x7f5878149500 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.040+0000 7f587df88700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5860004060 0x7f587814ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.040+0000 7f587df88700 1 --2- 172.21.15.112:0/2391677359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f587814d660 0x7f587815f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.040+0000 7f587df88700 1 -- 172.21.15.112:0/2391677359 >> 172.21.15.112:0/2391677359 conn(0x7f587806bbf0 msgr2=0x7f58780b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:18.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.040+0000 7f587df88700 1 -- 172.21.15.112:0/2391677359 shutdown_connections 2023-08-27T11:18:18.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.040+0000 7f587df88700 1 -- 172.21.15.112:0/2391677359 wait complete. 2023-08-27T11:18:18.053 INFO:teuthology.orchestra.run.smithi112.stdout:[{"pool_id":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool_id":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool_id":5,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:01.935213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool_id":6,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:02.602607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool_id":7,"pool_name":"cephfs.volume_0000000000766396.meta","create_time":"2023-08-27T11:18:14.385464+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000766396"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool_id":8,"pool_name":"cephfs.volume_0000000000766396.data","create_time":"2023-08-27T11:18:15.332020+0000","flags":163841,"flags_names":"hashpspool,creating,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000766396"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}] 2023-08-27T11:18:18.054 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs get volume_0000000000766396 --format=json 2023-08-27T11:18:18.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.174+0000 7fee15728700 1 Processor -- start 2023-08-27T11:18:18.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.174+0000 7fee15728700 1 -- start start 2023-08-27T11:18:18.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee15728700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006dc20 0x7fee100b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee15728700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1006f930 con 0x7fee10160a00 2023-08-27T11:18:18.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee15728700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1006fc10 con 0x7fee10146990 2023-08-27T11:18:18.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee15728700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1006fef0 con 0x7fee1006dc20 2023-08-27T11:18:18.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0effd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fee10160a00 0x7fee1015dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51498/0 (socket says 172.21.15.112:51498) 2023-08-27T11:18:18.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0effd700 1 -- 172.21.15.112:0/1143369491 learned_addr learned my addr 172.21.15.112:0/1143369491 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:18.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0f7fe700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006dc20 0x7fee100b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984475695 0 0) 0x7fee1006f930 con 0x7fee10160a00 2023-08-27T11:18:18.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedf80035b0 con 0x7fee10160a00 2023-08-27T11:18:18.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916325170 0 0) 0x7fee1006fc10 con 0x7fee10146990 2023-08-27T11:18:18.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee1006f930 con 0x7fee10146990 2023-08-27T11:18:18.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1599840201 0 0) 0x7fedf80035b0 con 0x7fee10160a00 2023-08-27T11:18:18.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006dc20 msgr2=0x7fee100b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:18.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006dc20 0x7fee100b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 >> v1:172.21.15.138:6789/0 conn(0x7fee10146990 legacy=0x7fee1006d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:18.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee100701d0 con 0x7fee10160a00 2023-08-27T11:18:18.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fee00001670 con 0x7fee10160a00 2023-08-27T11:18:18.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 >> v1:172.21.15.112:6789/0 conn(0x7fee10160a00 legacy=0x7fee1015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:18.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf8003bc0 0x7fedf8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006dc20 0x7fedf8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedf8004d90 0x7fedf8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1006fef0 con 0x7fedf8003bc0 2023-08-27T11:18:18.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf800bb50 con 0x7fedf8004d90 2023-08-27T11:18:18.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf800bd10 con 0x7fee1006dc20 2023-08-27T11:18:18.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.175+0000 7fee0effd700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf8003bc0 0x7fedf8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.176+0000 7fee0f7fe700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedf8004d90 0x7fedf8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.176+0000 7fee0e7fc700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006dc20 0x7fedf8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.176+0000 7fee0effd700 1 -- 172.21.15.112:0/1143369491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006dc20 msgr2=0x7fedf8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:18.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.176+0000 7fee0effd700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006dc20 0x7fedf8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.176+0000 7fee0effd700 1 -- 172.21.15.112:0/1143369491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedf8004d90 msgr2=0x7fedf8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:18.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.176+0000 7fee0effd700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedf8004d90 0x7fedf8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.176+0000 7fee0effd700 1 -- 172.21.15.112:0/1143369491 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf80035b0 con 0x7fedf8003bc0 2023-08-27T11:18:18.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.176+0000 7fee0f7fe700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedf8004d90 0x7fedf8005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:18.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.176+0000 7fee0e7fc700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006dc20 0x7fedf8004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:18.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.176+0000 7fee0effd700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf8003bc0 0x7fedf8003fc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fee0000c160 tx=0x7fedf8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a3d16cd5881804 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:18.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.176+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0000cbe0 con 0x7fedf8003bc0 2023-08-27T11:18:18.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.176+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0000cd60 con 0x7fedf8003bc0 2023-08-27T11:18:18.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.176+0000 7fee0dffb700 1 -- 172.21.15.112:0/1143369491 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00005ba0 con 0x7fedf8003bc0 2023-08-27T11:18:18.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.177+0000 7fee15728700 1 -- 172.21.15.112:0/1143369491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf8003bc0 msgr2=0x7fedf8003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:18.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.177+0000 7fee15728700 1 --2- 172.21.15.112:0/1143369491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf8003bc0 0x7fedf8003fc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fee0000c160 tx=0x7fedf8005880 comp rx=0 tx=0).stop 2023-08-27T11:18:18.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.177+0000 7fee0f7fe700 1 -- 172.21.15.112:0/1143369491 reap_dead start 2023-08-27T11:18:18.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.177+0000 7fee15728700 1 -- 172.21.15.112:0/1143369491 shutdown_connections 2023-08-27T11:18:18.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.177+0000 7fee15728700 1 -- 172.21.15.112:0/1143369491 >> 172.21.15.112:0/1143369491 conn(0x7fee100cfe50 msgr2=0x7fee100d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:18.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.177+0000 7fee15728700 1 -- 172.21.15.112:0/1143369491 shutdown_connections 2023-08-27T11:18:18.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.177+0000 7fee15728700 1 -- 172.21.15.112:0/1143369491 wait complete. 2023-08-27T11:18:18.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.178+0000 7fee15728700 1 Processor -- start 2023-08-27T11:18:18.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.178+0000 7fee15728700 1 -- start start 2023-08-27T11:18:18.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee15728700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf8003bc0 0x7fee101f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee15728700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006cbd0 0x7fee1006d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee15728700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee1013eee0 0x7fee1013f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee15728700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1006d590 con 0x7fedf8003bc0 2023-08-27T11:18:18.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee15728700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1013f8a0 con 0x7fee1013eee0 2023-08-27T11:18:18.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee15728700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1013d070 con 0x7fee1006cbd0 2023-08-27T11:18:18.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee0effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf8003bc0 0x7fee101f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee0effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf8003bc0 0x7fee101f28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57128/0 (socket says 172.21.15.112:57128) 2023-08-27T11:18:18.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee0effd700 1 -- 172.21.15.112:0/217742411 learned_addr learned my addr 172.21.15.112:0/217742411 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:18.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee0f7fe700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee1013eee0 0x7fee1013f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee0e7fc700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006cbd0 0x7fee1006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee0effd700 1 -- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006cbd0 msgr2=0x7fee1006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:18.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee0effd700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006cbd0 0x7fee1006d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee0effd700 1 -- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee1013eee0 msgr2=0x7fee1013f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:18.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee0effd700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee1013eee0 0x7fee1013f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee0effd700 1 -- 172.21.15.112:0/217742411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1013d2c0 con 0x7fedf8003bc0 2023-08-27T11:18:18.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee0f7fe700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee1013eee0 0x7fee1013f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:18.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee0e7fc700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006cbd0 0x7fee1006d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:18.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee0effd700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf8003bc0 0x7fee101f28c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fee000017a0 tx=0x7fee00001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:18.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fedf77fe700 1 -- 172.21.15.112:0/217742411 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0000ced0 con 0x7fedf8003bc0 2023-08-27T11:18:18.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fedf77fe700 1 -- 172.21.15.112:0/217742411 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee000070d0 con 0x7fedf8003bc0 2023-08-27T11:18:18.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fee15728700 1 -- 172.21.15.112:0/217742411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1013d5a0 con 0x7fedf8003bc0 2023-08-27T11:18:18.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.179+0000 7fedf77fe700 1 -- 172.21.15.112:0/217742411 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00007480 con 0x7fedf8003bc0 2023-08-27T11:18:18.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.180+0000 7fee15728700 1 -- 172.21.15.112:0/217742411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1013db60 con 0x7fedf8003bc0 2023-08-27T11:18:18.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.180+0000 7fee15728700 1 -- 172.21.15.112:0/217742411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee100d0260 con 0x7fedf8003bc0 2023-08-27T11:18:18.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.181+0000 7fedf77fe700 1 -- 172.21.15.112:0/217742411 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fee00019030 con 0x7fedf8003bc0 2023-08-27T11:18:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.181+0000 7fedf77fe700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedfc06a0e0 0x7fedfc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.181+0000 7fedf77fe700 1 -- 172.21.15.112:0/217742411 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6960+0+0 (secure 0 0 0) 0x7fee00089970 con 0x7fedf8003bc0 2023-08-27T11:18:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.182+0000 7fee0e7fc700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedfc06a0e0 0x7fedfc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.182+0000 7fee0e7fc700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedfc06a0e0 0x7fedfc06c5a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fee1013e610 tx=0x7fee04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.185+0000 7fedf77fe700 1 -- 172.21.15.112:0/217742411 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fee00054d70 con 0x7fedf8003bc0 2023-08-27T11:18:18.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.426+0000 7fee15728700 1 -- 172.21.15.112:0/217742411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs get", "fs_name": "volume_0000000000766396", "format": "json"} v 0) v1 -- 0x7fee10067a20 con 0x7fedf8003bc0 2023-08-27T11:18:18.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.427+0000 7fedf77fe700 1 -- 172.21.15.112:0/217742411 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs get", "fs_name": "volume_0000000000766396", "format": "json"}]=0 v19) v1 ==== 110+0+1989 (secure 0 0 0) 0x7fee0002b7e0 con 0x7fedf8003bc0 2023-08-27T11:18:18.428 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:18.428 INFO:teuthology.orchestra.run.smithi112.stdout:{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:18:16.332428+0000","modified":"2023-08-27T11:18:18.327167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5163":{"gid":5163,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.112:6835/1024768929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1024768929},{"type":"v1","addr":"172.21.15.112:6835","nonce":1024768929}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"volume_0000000000766396","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":3} 2023-08-27T11:18:18.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.429+0000 7fee15728700 1 -- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedfc06a0e0 msgr2=0x7fedfc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:18.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.429+0000 7fee15728700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedfc06a0e0 0x7fedfc06c5a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fee1013e610 tx=0x7fee04009040 comp rx=0 tx=0).stop 2023-08-27T11:18:18.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.429+0000 7fee15728700 1 -- 172.21.15.112:0/217742411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf8003bc0 msgr2=0x7fee101f28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:18.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.429+0000 7fee15728700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf8003bc0 0x7fee101f28c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fee000017a0 tx=0x7fee00001930 comp rx=0 tx=0).stop 2023-08-27T11:18:18.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.429+0000 7fee15728700 1 -- 172.21.15.112:0/217742411 shutdown_connections 2023-08-27T11:18:18.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.429+0000 7fee15728700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedfc06a0e0 0x7fedfc06c5a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.429+0000 7fee15728700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee1013eee0 0x7fee1013f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.429+0000 7fee15728700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf8003bc0 0x7fee101f28c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.429+0000 7fee15728700 1 --2- 172.21.15.112:0/217742411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee1006cbd0 0x7fee1006d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.429+0000 7fee15728700 1 -- 172.21.15.112:0/217742411 >> 172.21.15.112:0/217742411 conn(0x7fee100cfe50 msgr2=0x7fee1015f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.429+0000 7fee15728700 1 -- 172.21.15.112:0/217742411 shutdown_connections 2023-08-27T11:18:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.429+0000 7fee15728700 1 -- 172.21.15.112:0/217742411 wait complete. 2023-08-27T11:18:18.442 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000766396 --yes-i-really-mean-it 2023-08-27T11:18:18.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.564+0000 7f7f30647700 1 Processor -- start 2023-08-27T11:18:18.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.564+0000 7f7f30647700 1 -- start start 2023-08-27T11:18:18.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f30647700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7f28158da0 0x7f7f2815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f30647700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7f280b6930 0x7f7f280b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f30647700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f280b64a0 con 0x7f7f2815f2a0 2023-08-27T11:18:18.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f30647700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f28159180 con 0x7f7f280b6930 2023-08-27T11:18:18.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f30647700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f2815f820 con 0x7f7f28158da0 2023-08-27T11:18:18.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f2dbe2700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7f2815f2a0 0x7f7f2815c930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51518/0 (socket says 172.21.15.112:51518) 2023-08-27T11:18:18.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f2ebe4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7f280b6930 0x7f7f280b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f2dbe2700 1 -- 172.21.15.112:0/2665965268 learned_addr learned my addr 172.21.15.112:0/2665965268 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:18.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f2e3e3700 1 --2- 172.21.15.112:0/2665965268 >> v2:172.21.15.138:3300/0 conn(0x7f7f28158da0 0x7f7f2815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f2ebe4700 1 -- 172.21.15.112:0/2665965268 >> v2:172.21.15.138:3300/0 conn(0x7f7f28158da0 msgr2=0x7f7f2815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:18.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f2ebe4700 1 --2- 172.21.15.112:0/2665965268 >> v2:172.21.15.138:3300/0 conn(0x7f7f28158da0 0x7f7f2815ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f2ebe4700 1 -- 172.21.15.112:0/2665965268 >> v1:172.21.15.112:6789/0 conn(0x7f7f2815f2a0 legacy=0x7f7f2815c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:18.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f2ebe4700 1 -- 172.21.15.112:0/2665965268 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2815fb00 con 0x7f7f280b6930 2023-08-27T11:18:18.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f2d3e1700 1 -- 172.21.15.112:0/2665965268 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071132543 0 0) 0x7f7f280b64a0 con 0x7f7f2815f2a0 2023-08-27T11:18:18.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.565+0000 7f7f2ebe4700 1 --2- 172.21.15.112:0/2665965268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f280b6930 0x7f7f280b6d80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7f28068000 tx=0x7f7f2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7d52eb1dac4fb4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:18.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.566+0000 7f7f2d3e1700 1 -- 172.21.15.112:0/2665965268 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2000e070 con 0x7f7f280b6930 2023-08-27T11:18:18.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.566+0000 7f7f2d3e1700 1 -- 172.21.15.112:0/2665965268 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2000bbd0 con 0x7f7f280b6930 2023-08-27T11:18:18.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.566+0000 7f7f2d3e1700 1 -- 172.21.15.112:0/2665965268 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20012430 con 0x7f7f280b6930 2023-08-27T11:18:18.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.566+0000 7f7f30647700 1 -- 172.21.15.112:0/2665965268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f280b6930 msgr2=0x7f7f280b6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:18.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.566+0000 7f7f30647700 1 --2- 172.21.15.112:0/2665965268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f280b6930 0x7f7f280b6d80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7f28068000 tx=0x7f7f2000b330 comp rx=0 tx=0).stop 2023-08-27T11:18:18.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.566+0000 7f7f30647700 1 -- 172.21.15.112:0/2665965268 shutdown_connections 2023-08-27T11:18:18.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.566+0000 7f7f30647700 1 --2- 172.21.15.112:0/2665965268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f280b6930 0x7f7f280b6d80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.566+0000 7f7f30647700 1 --2- 172.21.15.112:0/2665965268 >> v2:172.21.15.138:3300/0 conn(0x7f7f28158da0 0x7f7f2815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.566+0000 7f7f30647700 1 -- 172.21.15.112:0/2665965268 >> 172.21.15.112:0/2665965268 conn(0x7f7f28066010 msgr2=0x7f7f280663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:18.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.566+0000 7f7f30647700 1 -- 172.21.15.112:0/2665965268 shutdown_connections 2023-08-27T11:18:18.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.566+0000 7f7f30647700 1 -- 172.21.15.112:0/2665965268 wait complete. 2023-08-27T11:18:18.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.568+0000 7f7f30647700 1 Processor -- start 2023-08-27T11:18:18.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.568+0000 7f7f30647700 1 -- start start 2023-08-27T11:18:18.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f30647700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f280b6930 0x7f7f280c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f30647700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f28158da0 0x7f7f28148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f30647700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f2815f2a0 0x7f7f280c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f30647700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f280c6640 con 0x7f7f2815f2a0 2023-08-27T11:18:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f30647700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f28154150 con 0x7f7f280b6930 2023-08-27T11:18:18.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f30647700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f28155880 con 0x7f7f28158da0 2023-08-27T11:18:18.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f2ebe4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f2815f2a0 0x7f7f280c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f2ebe4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f2815f2a0 0x7f7f280c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57144/0 (socket says 172.21.15.112:57144) 2023-08-27T11:18:18.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f2ebe4700 1 -- 172.21.15.112:0/3767279405 learned_addr learned my addr 172.21.15.112:0/3767279405 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:18.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f2e3e3700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f280b6930 0x7f7f280c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f2dbe2700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f28158da0 0x7f7f28148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f2ebe4700 1 -- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f28158da0 msgr2=0x7f7f28148cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:18.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f2ebe4700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f28158da0 0x7f7f28148cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f2ebe4700 1 -- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f280b6930 msgr2=0x7f7f280c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:18.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f2ebe4700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f280b6930 0x7f7f280c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:18.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f2ebe4700 1 -- 172.21.15.112:0/3767279405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f20009a70 con 0x7f7f2815f2a0 2023-08-27T11:18:18.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.569+0000 7f7f2ebe4700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f2815f2a0 0x7f7f280c6db0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7f2806d1c0 tx=0x7f7f20004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:18.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.570+0000 7f7f1effd700 1 -- 172.21.15.112:0/3767279405 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20012430 con 0x7f7f2815f2a0 2023-08-27T11:18:18.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.570+0000 7f7f1effd700 1 -- 172.21.15.112:0/3767279405 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f20020070 con 0x7f7f2815f2a0 2023-08-27T11:18:18.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.570+0000 7f7f30647700 1 -- 172.21.15.112:0/3767279405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f28155b60 con 0x7f7f2815f2a0 2023-08-27T11:18:18.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.570+0000 7f7f1effd700 1 -- 172.21.15.112:0/3767279405 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20004280 con 0x7f7f2815f2a0 2023-08-27T11:18:18.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.570+0000 7f7f30647700 1 -- 172.21.15.112:0/3767279405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f28156120 con 0x7f7f2815f2a0 2023-08-27T11:18:18.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.571+0000 7f7f30647700 1 -- 172.21.15.112:0/3767279405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f28067a20 con 0x7f7f2815f2a0 2023-08-27T11:18:18.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.571+0000 7f7f1effd700 1 -- 172.21.15.112:0/3767279405 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7f200043e0 con 0x7f7f2815f2a0 2023-08-27T11:18:18.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.571+0000 7f7f1effd700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f0c06a0e0 0x7f7f0c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:18.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.572+0000 7f7f1effd700 1 -- 172.21.15.112:0/3767279405 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6960+0+0 (secure 0 0 0) 0x7f7f20022070 con 0x7f7f2815f2a0 2023-08-27T11:18:18.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.572+0000 7f7f2e3e3700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f0c06a0e0 0x7f7f0c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:18.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.572+0000 7f7f2e3e3700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f0c06a0e0 0x7f7f0c06c5a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009930 tx=0x7f7f24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:18.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.576+0000 7f7f1effd700 1 -- 172.21.15.112:0/3767279405 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7f20055ac0 con 0x7f7f2815f2a0 2023-08-27T11:18:18.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:18.794+0000 7f7f30647700 1 -- 172.21.15.112:0/3767279405 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000766396", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f280cda40 con 0x7f7f0c06a0e0 2023-08-27T11:18:18.800 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:18:18.799+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:18:18.827 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:18:21.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:21.813+0000 7f7f1effd700 1 -- 172.21.15.112:0/3767279405 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7f7f280cda40 con 0x7f7f0c06a0e0 2023-08-27T11:18:21.815 INFO:teuthology.orchestra.run.smithi112.stdout:metadata pool: cephfs.volume_0000000000766396.meta data pool: ['cephfs.volume_0000000000766396.data'] removed 2023-08-27T11:18:21.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:21.816+0000 7f7f30647700 1 -- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f0c06a0e0 msgr2=0x7f7f0c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:21.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:21.816+0000 7f7f30647700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f0c06a0e0 0x7f7f0c06c5a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009930 tx=0x7f7f24008040 comp rx=0 tx=0).stop 2023-08-27T11:18:21.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:21.816+0000 7f7f30647700 1 -- 172.21.15.112:0/3767279405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f2815f2a0 msgr2=0x7f7f280c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:21.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:21.816+0000 7f7f30647700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f2815f2a0 0x7f7f280c6db0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7f2806d1c0 tx=0x7f7f20004ad0 comp rx=0 tx=0).stop 2023-08-27T11:18:21.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:21.816+0000 7f7f30647700 1 -- 172.21.15.112:0/3767279405 shutdown_connections 2023-08-27T11:18:21.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:21.816+0000 7f7f30647700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f0c06a0e0 0x7f7f0c06c5a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:21.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:21.816+0000 7f7f30647700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f2815f2a0 0x7f7f280c6db0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:21.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:21.816+0000 7f7f30647700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f280b6930 0x7f7f280c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:21.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:21.816+0000 7f7f30647700 1 --2- 172.21.15.112:0/3767279405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f28158da0 0x7f7f28148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:21.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:21.816+0000 7f7f30647700 1 -- 172.21.15.112:0/3767279405 >> 172.21.15.112:0/3767279405 conn(0x7f7f28066010 msgr2=0x7f7f280b6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:21.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:21.816+0000 7f7f30647700 1 -- 172.21.15.112:0/3767279405 shutdown_connections 2023-08-27T11:18:21.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:21.816+0000 7f7f30647700 1 -- 172.21.15.112:0/3767279405 wait complete. 2023-08-27T11:18:21.833 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:18:21.833 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:18:21.880 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:18:21.928 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:18:21.928 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:18:21.929 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:21.949 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:18:21.950 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:22.008 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:22.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.172+0000 7f2de38a3700 1 Processor -- start 2023-08-27T11:18:22.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.172+0000 7f2de38a3700 1 -- start start 2023-08-27T11:18:22.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.173+0000 7f2de38a3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2ddc141420 0x7f2ddc0aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.173+0000 7f2de38a3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ddc0ab440 con 0x7f2ddc06e150 2023-08-27T11:18:22.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.173+0000 7f2de38a3700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ddc15eed0 con 0x7f2ddc0a9970 2023-08-27T11:18:22.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.173+0000 7f2de38a3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ddc15f1b0 con 0x7f2ddc141420 2023-08-27T11:18:22.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.173+0000 7f2de163f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2ddc141420 0x7f2ddc0aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.173+0000 7f2de163f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2ddc141420 0x7f2ddc0aaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57162/0 (socket says 172.21.15.112:57162) 2023-08-27T11:18:22.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.173+0000 7f2de163f700 1 -- 172.21.15.112:0/3369747338 learned_addr learned my addr 172.21.15.112:0/3369747338 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:22.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.173+0000 7f2de163f700 1 -- 172.21.15.112:0/3369747338 >> v1:172.21.15.138:6789/0 conn(0x7f2ddc0a9970 legacy=0x7f2ddc0cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:22.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.173+0000 7f2de163f700 1 -- 172.21.15.112:0/3369747338 >> v1:172.21.15.112:6789/0 conn(0x7f2ddc06e150 legacy=0x7f2ddc0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.173+0000 7f2de163f700 1 -- 172.21.15.112:0/3369747338 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc15f490 con 0x7f2ddc141420 2023-08-27T11:18:22.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.173+0000 7f2de163f700 1 --2- 172.21.15.112:0/3369747338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ddc141420 0x7f2ddc0aaf00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0098f0 tx=0x7f2dcc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e843920db75bdad server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:22.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.173+0000 7f2dd3fff700 1 -- 172.21.15.112:0/3369747338 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc004230 con 0x7f2ddc141420 2023-08-27T11:18:22.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.174+0000 7f2dd3fff700 1 -- 172.21.15.112:0/3369747338 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc004390 con 0x7f2ddc141420 2023-08-27T11:18:22.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.174+0000 7f2dd3fff700 1 -- 172.21.15.112:0/3369747338 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc0036a0 con 0x7f2ddc141420 2023-08-27T11:18:22.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.174+0000 7f2de38a3700 1 -- 172.21.15.112:0/3369747338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ddc141420 msgr2=0x7f2ddc0aaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.174+0000 7f2de38a3700 1 --2- 172.21.15.112:0/3369747338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ddc141420 0x7f2ddc0aaf00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0098f0 tx=0x7f2dcc009c00 comp rx=0 tx=0).stop 2023-08-27T11:18:22.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.174+0000 7f2de38a3700 1 -- 172.21.15.112:0/3369747338 shutdown_connections 2023-08-27T11:18:22.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.174+0000 7f2de38a3700 1 --2- 172.21.15.112:0/3369747338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ddc141420 0x7f2ddc0aaf00 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.174+0000 7f2de38a3700 1 -- 172.21.15.112:0/3369747338 >> 172.21.15.112:0/3369747338 conn(0x7f2ddc06cd30 msgr2=0x7f2ddc0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:22.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.174+0000 7f2de38a3700 1 -- 172.21.15.112:0/3369747338 shutdown_connections 2023-08-27T11:18:22.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.174+0000 7f2de38a3700 1 -- 172.21.15.112:0/3369747338 wait complete. 2023-08-27T11:18:22.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.175+0000 7f2de38a3700 1 Processor -- start 2023-08-27T11:18:22.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.175+0000 7f2de38a3700 1 -- start start 2023-08-27T11:18:22.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.176+0000 7f2de38a3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ddc06e150 0x7f2ddc0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.176+0000 7f2de38a3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ddc0a9970 0x7f2ddc1377a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.176+0000 7f2de38a3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ddc141420 0x7f2ddc13c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.176+0000 7f2de38a3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc1581e0 con 0x7f2ddc141420 2023-08-27T11:18:22.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.176+0000 7f2de38a3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc137ce0 con 0x7f2ddc06e150 2023-08-27T11:18:22.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.176+0000 7f2de38a3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc0cd8e0 con 0x7f2ddc0a9970 2023-08-27T11:18:22.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.176+0000 7f2de1e40700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ddc141420 0x7f2ddc13c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.176+0000 7f2de1e40700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ddc141420 0x7f2ddc13c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57172/0 (socket says 172.21.15.112:57172) 2023-08-27T11:18:22.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.176+0000 7f2de1e40700 1 -- 172.21.15.112:0/1396002655 learned_addr learned my addr 172.21.15.112:0/1396002655 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:22.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.176+0000 7f2de163f700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ddc06e150 0x7f2ddc0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.176+0000 7f2de0e3e700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ddc0a9970 0x7f2ddc1377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.176+0000 7f2de1e40700 1 -- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ddc0a9970 msgr2=0x7f2ddc1377a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.176+0000 7f2de1e40700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ddc0a9970 0x7f2ddc1377a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.177+0000 7f2de1e40700 1 -- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ddc06e150 msgr2=0x7f2ddc0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.177+0000 7f2de1e40700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ddc06e150 0x7f2ddc0cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.177+0000 7f2de1e40700 1 -- 172.21.15.112:0/1396002655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc009550 con 0x7f2ddc141420 2023-08-27T11:18:22.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.177+0000 7f2de1e40700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ddc141420 0x7f2ddc13c320 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400cc10 tx=0x7f2dd400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:22.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.177+0000 7f2dd1ffb700 1 -- 172.21.15.112:0/1396002655 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd400abb0 con 0x7f2ddc141420 2023-08-27T11:18:22.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.177+0000 7f2dd1ffb700 1 -- 172.21.15.112:0/1396002655 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd400ad10 con 0x7f2ddc141420 2023-08-27T11:18:22.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.177+0000 7f2de38a3700 1 -- 172.21.15.112:0/1396002655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ddc1443b0 con 0x7f2ddc141420 2023-08-27T11:18:22.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.177+0000 7f2dd1ffb700 1 -- 172.21.15.112:0/1396002655 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4012660 con 0x7f2ddc141420 2023-08-27T11:18:22.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.177+0000 7f2de38a3700 1 -- 172.21.15.112:0/1396002655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ddc140730 con 0x7f2ddc141420 2023-08-27T11:18:22.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.178+0000 7f2de38a3700 1 -- 172.21.15.112:0/1396002655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ddc15f750 con 0x7f2ddc141420 2023-08-27T11:18:22.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.178+0000 7f2dd1ffb700 1 -- 172.21.15.112:0/1396002655 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2dd40127c0 con 0x7f2ddc141420 2023-08-27T11:18:22.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.179+0000 7f2dd1ffb700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2dc006a0e0 0x7f2dc006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.179+0000 7f2dd1ffb700 1 -- 172.21.15.112:0/1396002655 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f2dd4088cc0 con 0x7f2ddc141420 2023-08-27T11:18:22.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.179+0000 7f2de163f700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2dc006a0e0 0x7f2dc006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.179+0000 7f2de163f700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2dc006a0e0 0x7f2dc006c5a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009130 tx=0x7f2dcc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:22.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.183+0000 7f2dd1ffb700 1 -- 172.21.15.112:0/1396002655 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2ddc15f750 con 0x7f2ddc141420 2023-08-27T11:18:22.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.374+0000 7f2de38a3700 1 -- 172.21.15.112:0/1396002655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ddc158320 con 0x7f2ddc141420 2023-08-27T11:18:22.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.376+0000 7f2dd1ffb700 1 -- 172.21.15.112:0/1396002655 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18448 (secure 0 0 0) 0x7f2ddc158320 con 0x7f2ddc141420 2023-08-27T11:18:22.377 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:22.378 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":40,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:21.807644+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:01.935213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:02.602607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/1024768929":"2023-08-28T11:18:18.799945+0000","172.21.15.112:6833/1024768929":"2023-08-28T11:18:18.799945+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:22.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.378+0000 7f2de38a3700 1 -- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2dc006a0e0 msgr2=0x7f2dc006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.378+0000 7f2de38a3700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2dc006a0e0 0x7f2dc006c5a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009130 tx=0x7f2dcc008030 comp rx=0 tx=0).stop 2023-08-27T11:18:22.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.378+0000 7f2de38a3700 1 -- 172.21.15.112:0/1396002655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ddc141420 msgr2=0x7f2ddc13c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.378+0000 7f2de38a3700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ddc141420 0x7f2ddc13c320 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400cc10 tx=0x7f2dd400a060 comp rx=0 tx=0).stop 2023-08-27T11:18:22.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.378+0000 7f2de38a3700 1 -- 172.21.15.112:0/1396002655 shutdown_connections 2023-08-27T11:18:22.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.378+0000 7f2de38a3700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2dc006a0e0 0x7f2dc006c5a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.378+0000 7f2de38a3700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ddc141420 0x7f2ddc13c320 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.378+0000 7f2de38a3700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ddc06e150 0x7f2ddc0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.378+0000 7f2de38a3700 1 --2- 172.21.15.112:0/1396002655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ddc0a9970 0x7f2ddc1377a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.378+0000 7f2de38a3700 1 -- 172.21.15.112:0/1396002655 >> 172.21.15.112:0/1396002655 conn(0x7f2ddc06cd30 msgr2=0x7f2ddc0aa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.378+0000 7f2de38a3700 1 -- 172.21.15.112:0/1396002655 shutdown_connections 2023-08-27T11:18:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.378+0000 7f2de38a3700 1 -- 172.21.15.112:0/1396002655 wait complete. 2023-08-27T11:18:22.392 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:18:01.935213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:18:02.602607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:18:22.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:18:22.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.516+0000 7f3ffff4e700 1 Processor -- start 2023-08-27T11:18:22.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.516+0000 7f3ffff4e700 1 -- start start 2023-08-27T11:18:22.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.517+0000 7f3ffff4e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3ff8071e70 0x7f3ff815f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.517+0000 7f3ffff4e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff815c560 0x7f3ff80ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.517+0000 7f3ffff4e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff815fdb0 con 0x7f3ff8072250 2023-08-27T11:18:22.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.517+0000 7f3ffff4e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff806e610 con 0x7f3ff8071e70 2023-08-27T11:18:22.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.517+0000 7f3ffff4e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff815b870 con 0x7f3ff815c560 2023-08-27T11:18:22.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.517+0000 7f3ffe4eb700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3ff8072250 0x7f3ff813e9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51562/0 (socket says 172.21.15.112:51562) 2023-08-27T11:18:22.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.517+0000 7f3ffe4eb700 1 -- 172.21.15.112:0/2603930347 learned_addr learned my addr 172.21.15.112:0/2603930347 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:22.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.517+0000 7f3ffdcea700 1 --2- 172.21.15.112:0/2603930347 >> v2:172.21.15.138:3300/0 conn(0x7f3ff8071e70 0x7f3ff815f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.517+0000 7f3ffd4e9700 1 --2- 172.21.15.112:0/2603930347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff815c560 0x7f3ff80ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388972313 0 0) 0x7f3ff815fdb0 con 0x7f3ff8072250 2023-08-27T11:18:22.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fe40035b0 con 0x7f3ff8072250 2023-08-27T11:18:22.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1442145652 0 0) 0x7f3fe40035b0 con 0x7f3ff8072250 2023-08-27T11:18:22.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff815c560 msgr2=0x7f3ff80ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:22.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 --2- 172.21.15.112:0/2603930347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff815c560 0x7f3ff80ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 >> v2:172.21.15.138:3300/0 conn(0x7f3ff8071e70 msgr2=0x7f3ff815f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:22.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 --2- 172.21.15.112:0/2603930347 >> v2:172.21.15.138:3300/0 conn(0x7f3ff8071e70 0x7f3ff815f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff806e330 con 0x7f3ff8072250 2023-08-27T11:18:22.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f3ff00019f0 con 0x7f3ff8072250 2023-08-27T11:18:22.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 >> v1:172.21.15.112:6789/0 conn(0x7f3ff8072250 legacy=0x7f3ff813e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 --2- 172.21.15.112:0/2603930347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff815c560 0x7f3fe4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 --2- 172.21.15.112:0/2603930347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe4003f80 0x7f3fe4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 --2- 172.21.15.112:0/2603930347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe4004d90 0x7f3fe4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff806e610 con 0x7f3fe4004d90 2023-08-27T11:18:22.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff815b870 con 0x7f3fe4003f80 2023-08-27T11:18:22.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe400ba10 con 0x7f3ff815c560 2023-08-27T11:18:22.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffe4eb700 1 --2- 172.21.15.112:0/2603930347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe4004d90 0x7f3fe4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffdcea700 1 --2- 172.21.15.112:0/2603930347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff815c560 0x7f3fe4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffd4e9700 1 --2- 172.21.15.112:0/2603930347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe4003f80 0x7f3fe4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffe4eb700 1 -- 172.21.15.112:0/2603930347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff815c560 msgr2=0x7f3fe4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:22.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffe4eb700 1 --2- 172.21.15.112:0/2603930347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff815c560 0x7f3fe4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffe4eb700 1 -- 172.21.15.112:0/2603930347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe4003f80 msgr2=0x7f3fe4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:22.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffe4eb700 1 --2- 172.21.15.112:0/2603930347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe4003f80 0x7f3fe4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffe4eb700 1 -- 172.21.15.112:0/2603930347 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe40035b0 con 0x7f3fe4004d90 2023-08-27T11:18:22.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.518+0000 7f3ffe4eb700 1 --2- 172.21.15.112:0/2603930347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe4004d90 0x7f3fe4005210 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000a8d0 tx=0x7f3fe4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342132fc8eb0ab42 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:22.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.519+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000a340 con 0x7f3fe4004d90 2023-08-27T11:18:22.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.519+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff0012600 con 0x7f3fe4004d90 2023-08-27T11:18:22.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.519+0000 7f3ffcce8700 1 -- 172.21.15.112:0/2603930347 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00129b0 con 0x7f3fe4004d90 2023-08-27T11:18:22.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.519+0000 7f3ffff4e700 1 -- 172.21.15.112:0/2603930347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe4004d90 msgr2=0x7f3fe4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.519+0000 7f3ffff4e700 1 --2- 172.21.15.112:0/2603930347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe4004d90 0x7f3fe4005210 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000a8d0 tx=0x7f3fe4005880 comp rx=0 tx=0).stop 2023-08-27T11:18:22.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.519+0000 7f3ffe4eb700 1 -- 172.21.15.112:0/2603930347 reap_dead start 2023-08-27T11:18:22.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.519+0000 7f3ffff4e700 1 -- 172.21.15.112:0/2603930347 shutdown_connections 2023-08-27T11:18:22.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.519+0000 7f3ffff4e700 1 -- 172.21.15.112:0/2603930347 >> 172.21.15.112:0/2603930347 conn(0x7f3ff80cf7b0 msgr2=0x7f3ff80cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:22.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.519+0000 7f3ffff4e700 1 -- 172.21.15.112:0/2603930347 shutdown_connections 2023-08-27T11:18:22.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.519+0000 7f3ffff4e700 1 -- 172.21.15.112:0/2603930347 wait complete. 2023-08-27T11:18:22.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.520+0000 7f3ffff4e700 1 Processor -- start 2023-08-27T11:18:22.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.520+0000 7f3ffff4e700 1 -- start start 2023-08-27T11:18:22.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffff4e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff80adcb0 0x7f3ff8155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffff4e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff80ae090 0x7f3ff81f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffff4e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff8142ad0 0x7f3ff81f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffff4e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff81f7240 con 0x7f3ff8142ad0 2023-08-27T11:18:22.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffff4e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff813e740 con 0x7f3ff80adcb0 2023-08-27T11:18:22.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffff4e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff8156610 con 0x7f3ff80ae090 2023-08-27T11:18:22.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffe4eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff8142ad0 0x7f3ff81f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffe4eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff8142ad0 0x7f3ff81f2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57192/0 (socket says 172.21.15.112:57192) 2023-08-27T11:18:22.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffe4eb700 1 -- 172.21.15.112:0/1039983042 learned_addr learned my addr 172.21.15.112:0/1039983042 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:22.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffdcea700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff80adcb0 0x7f3ff8155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffd4e9700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff80ae090 0x7f3ff81f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffe4eb700 1 -- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff80ae090 msgr2=0x7f3ff81f6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffe4eb700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff80ae090 0x7f3ff81f6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffe4eb700 1 -- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff80adcb0 msgr2=0x7f3ff8155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.521+0000 7f3ffe4eb700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff80adcb0 0x7f3ff8155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.522+0000 7f3ffe4eb700 1 -- 172.21.15.112:0/1039983042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff81568f0 con 0x7f3ff8142ad0 2023-08-27T11:18:22.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.522+0000 7f3ffe4eb700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff8142ad0 0x7f3ff81f2150 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0015040 tx=0x7f3ff0001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:22.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.522+0000 7f3fee7fc700 1 -- 172.21.15.112:0/1039983042 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00019c0 con 0x7f3ff8142ad0 2023-08-27T11:18:22.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.522+0000 7f3fee7fc700 1 -- 172.21.15.112:0/1039983042 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff00011c0 con 0x7f3ff8142ad0 2023-08-27T11:18:22.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.522+0000 7f3ffff4e700 1 -- 172.21.15.112:0/1039983042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff813e9e0 con 0x7f3ff8142ad0 2023-08-27T11:18:22.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.522+0000 7f3fee7fc700 1 -- 172.21.15.112:0/1039983042 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0027410 con 0x7f3ff8142ad0 2023-08-27T11:18:22.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.522+0000 7f3ffff4e700 1 -- 172.21.15.112:0/1039983042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff8142f30 con 0x7f3ff8142ad0 2023-08-27T11:18:22.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.523+0000 7f3ffff4e700 1 -- 172.21.15.112:0/1039983042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff8067e90 con 0x7f3ff8142ad0 2023-08-27T11:18:22.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.523+0000 7f3fee7fc700 1 -- 172.21.15.112:0/1039983042 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3ff0019070 con 0x7f3ff8142ad0 2023-08-27T11:18:22.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.523+0000 7f3fee7fc700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fdc06a0e0 0x7f3fdc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.524+0000 7f3fee7fc700 1 -- 172.21.15.112:0/1039983042 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f3ff0089c30 con 0x7f3ff8142ad0 2023-08-27T11:18:22.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.524+0000 7f3ffdcea700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fdc06a0e0 0x7f3fdc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.524+0000 7f3ffdcea700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fdc06a0e0 0x7f3fdc06c5a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009c20 tx=0x7f3fe8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:22.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.528+0000 7f3fee7fc700 1 -- 172.21.15.112:0/1039983042 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3ff0055350 con 0x7f3ff8142ad0 2023-08-27T11:18:22.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.716+0000 7f3ffff4e700 1 -- 172.21.15.112:0/1039983042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3ff80ae8b0 con 0x7f3ff8142ad0 2023-08-27T11:18:22.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.716+0000 7f3fee7fc700 1 -- 172.21.15.112:0/1039983042 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3ff002c020 con 0x7f3ff8142ad0 2023-08-27T11:18:22.718 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:18:22.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.719+0000 7f3ffff4e700 1 -- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fdc06a0e0 msgr2=0x7f3fdc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.719+0000 7f3ffff4e700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fdc06a0e0 0x7f3fdc06c5a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009c20 tx=0x7f3fe8008040 comp rx=0 tx=0).stop 2023-08-27T11:18:22.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.719+0000 7f3ffff4e700 1 -- 172.21.15.112:0/1039983042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff8142ad0 msgr2=0x7f3ff81f2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.719+0000 7f3ffff4e700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff8142ad0 0x7f3ff81f2150 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0015040 tx=0x7f3ff0001360 comp rx=0 tx=0).stop 2023-08-27T11:18:22.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.719+0000 7f3ffff4e700 1 -- 172.21.15.112:0/1039983042 shutdown_connections 2023-08-27T11:18:22.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.719+0000 7f3ffff4e700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fdc06a0e0 0x7f3fdc06c5a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.719+0000 7f3ffff4e700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff8142ad0 0x7f3ff81f2150 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.719+0000 7f3ffff4e700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff80adcb0 0x7f3ff8155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.719+0000 7f3ffff4e700 1 --2- 172.21.15.112:0/1039983042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff80ae090 0x7f3ff81f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.719+0000 7f3ffff4e700 1 -- 172.21.15.112:0/1039983042 >> 172.21.15.112:0/1039983042 conn(0x7f3ff80cf7b0 msgr2=0x7f3ff8155370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:22.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.719+0000 7f3ffff4e700 1 -- 172.21.15.112:0/1039983042 shutdown_connections 2023-08-27T11:18:22.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.719+0000 7f3ffff4e700 1 -- 172.21.15.112:0/1039983042 wait complete. 2023-08-27T11:18:22.732 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:18:22.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.856+0000 7f832d7dc700 1 Processor -- start 2023-08-27T11:18:22.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.856+0000 7f832d7dc700 1 -- start start 2023-08-27T11:18:22.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.856+0000 7f832d7dc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f83280ce210 0x7f832815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.856+0000 7f832d7dc700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83280cb7b0 con 0x7f83280ce5f0 2023-08-27T11:18:22.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.857+0000 7f832d7dc700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832815d530 con 0x7f832806c470 2023-08-27T11:18:22.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.857+0000 7f832d7dc700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832815ded0 con 0x7f83280ce210 2023-08-27T11:18:22.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.857+0000 7f8326ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f83280ce210 0x7f832815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.857+0000 7f8326ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f83280ce210 0x7f832815aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57196/0 (socket says 172.21.15.112:57196) 2023-08-27T11:18:22.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.857+0000 7f8326ffd700 1 -- 172.21.15.112:0/4103364638 learned_addr learned my addr 172.21.15.112:0/4103364638 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:22.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.857+0000 7f8326ffd700 1 -- 172.21.15.112:0/4103364638 >> v1:172.21.15.138:6789/0 conn(0x7f832806c470 legacy=0x7f83280ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:22.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.857+0000 7f8326ffd700 1 -- 172.21.15.112:0/4103364638 >> v1:172.21.15.112:6789/0 conn(0x7f83280ce5f0 legacy=0x7f83280cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.857+0000 7f8326ffd700 1 -- 172.21.15.112:0/4103364638 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832815d810 con 0x7f83280ce210 2023-08-27T11:18:22.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.857+0000 7f8326ffd700 1 --2- 172.21.15.112:0/4103364638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280ce210 0x7f832815aab0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f83180098f0 tx=0x7f8318009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03756d2e1702600 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:22.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.857+0000 7f8325ffb700 1 -- 172.21.15.112:0/4103364638 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831800b8a0 con 0x7f83280ce210 2023-08-27T11:18:22.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.857+0000 7f8325ffb700 1 -- 172.21.15.112:0/4103364638 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831800ba00 con 0x7f83280ce210 2023-08-27T11:18:22.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.857+0000 7f8325ffb700 1 -- 172.21.15.112:0/4103364638 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831800bd90 con 0x7f83280ce210 2023-08-27T11:18:22.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.858+0000 7f832d7dc700 1 -- 172.21.15.112:0/4103364638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280ce210 msgr2=0x7f832815aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.858+0000 7f832d7dc700 1 --2- 172.21.15.112:0/4103364638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280ce210 0x7f832815aab0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f83180098f0 tx=0x7f8318009c00 comp rx=0 tx=0).stop 2023-08-27T11:18:22.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.858+0000 7f832d7dc700 1 -- 172.21.15.112:0/4103364638 shutdown_connections 2023-08-27T11:18:22.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.858+0000 7f832d7dc700 1 --2- 172.21.15.112:0/4103364638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280ce210 0x7f832815aab0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.858+0000 7f832d7dc700 1 -- 172.21.15.112:0/4103364638 >> 172.21.15.112:0/4103364638 conn(0x7f83280cef70 msgr2=0x7f83280cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:22.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.858+0000 7f832d7dc700 1 -- 172.21.15.112:0/4103364638 shutdown_connections 2023-08-27T11:18:22.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.858+0000 7f832d7dc700 1 -- 172.21.15.112:0/4103364638 wait complete. 2023-08-27T11:18:22.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.859+0000 7f832d7dc700 1 Processor -- start 2023-08-27T11:18:22.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.859+0000 7f832d7dc700 1 -- start start 2023-08-27T11:18:22.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f832d7dc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832806c470 0x7f8328147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f832d7dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83280ce210 0x7f8328140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f832d7dc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83280ce5f0 0x7f832814cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f832d7dc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8328140b90 con 0x7f832806c470 2023-08-27T11:18:22.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f832d7dc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83280ccde0 con 0x7f83280ce210 2023-08-27T11:18:22.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f832d7dc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83280cbd40 con 0x7f83280ce5f0 2023-08-27T11:18:22.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f8326ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832806c470 0x7f8328147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f8326ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832806c470 0x7f8328147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57202/0 (socket says 172.21.15.112:57202) 2023-08-27T11:18:22.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f8326ffd700 1 -- 172.21.15.112:0/1206260275 learned_addr learned my addr 172.21.15.112:0/1206260275 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:22.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f83267fc700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83280ce210 0x7f8328140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f8326ffd700 1 -- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83280ce5f0 msgr2=0x7f832814cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f83277fe700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83280ce5f0 0x7f832814cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f8326ffd700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83280ce5f0 0x7f832814cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f8326ffd700 1 -- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83280ce210 msgr2=0x7f8328140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:22.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f8326ffd700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83280ce210 0x7f8328140650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:22.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f8326ffd700 1 -- 172.21.15.112:0/1206260275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8318009550 con 0x7f832806c470 2023-08-27T11:18:22.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.860+0000 7f83277fe700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83280ce5f0 0x7f832814cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:22.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.861+0000 7f8326ffd700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832806c470 0x7f8328147020 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8318000c00 tx=0x7f8318004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:22.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.861+0000 7f830f7fe700 1 -- 172.21.15.112:0/1206260275 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8318004570 con 0x7f832806c470 2023-08-27T11:18:22.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.861+0000 7f830f7fe700 1 -- 172.21.15.112:0/1206260275 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831801f070 con 0x7f832806c470 2023-08-27T11:18:22.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.861+0000 7f832d7dc700 1 -- 172.21.15.112:0/1206260275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83280cc020 con 0x7f832806c470 2023-08-27T11:18:22.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.861+0000 7f830f7fe700 1 -- 172.21.15.112:0/1206260275 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83180038e0 con 0x7f832806c470 2023-08-27T11:18:22.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.861+0000 7f832d7dc700 1 -- 172.21.15.112:0/1206260275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83280cc5e0 con 0x7f832806c470 2023-08-27T11:18:22.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.862+0000 7f832d7dc700 1 -- 172.21.15.112:0/1206260275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8328067a20 con 0x7f832806c470 2023-08-27T11:18:22.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.862+0000 7f830f7fe700 1 -- 172.21.15.112:0/1206260275 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8318021030 con 0x7f832806c470 2023-08-27T11:18:22.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.863+0000 7f830f7fe700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f831406a0e0 0x7f831406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:22.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.863+0000 7f830f7fe700 1 -- 172.21.15.112:0/1206260275 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f8318089ac0 con 0x7f832806c470 2023-08-27T11:18:22.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.863+0000 7f83267fc700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f831406a0e0 0x7f831406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:22.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.863+0000 7f83267fc700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f831406a0e0 0x7f831406c5a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f832815c2e0 tx=0x7f831c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:22.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:22.867+0000 7f830f7fe700 1 -- 172.21.15.112:0/1206260275 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8318055140 con 0x7f832806c470 2023-08-27T11:18:23.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.053+0000 7f832d7dc700 1 -- 172.21.15.112:0/1206260275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83280b2d10 con 0x7f832806c470 2023-08-27T11:18:23.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.053+0000 7f830f7fe700 1 -- 172.21.15.112:0/1206260275 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8318015020 con 0x7f832806c470 2023-08-27T11:18:23.056 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:18:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.055+0000 7f832d7dc700 1 -- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f831406a0e0 msgr2=0x7f831406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.055+0000 7f832d7dc700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f831406a0e0 0x7f831406c5a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f832815c2e0 tx=0x7f831c009040 comp rx=0 tx=0).stop 2023-08-27T11:18:23.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.055+0000 7f832d7dc700 1 -- 172.21.15.112:0/1206260275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832806c470 msgr2=0x7f8328147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.055+0000 7f832d7dc700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832806c470 0x7f8328147020 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8318000c00 tx=0x7f8318004750 comp rx=0 tx=0).stop 2023-08-27T11:18:23.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.055+0000 7f832d7dc700 1 -- 172.21.15.112:0/1206260275 shutdown_connections 2023-08-27T11:18:23.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.055+0000 7f832d7dc700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f831406a0e0 0x7f831406c5a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.055+0000 7f832d7dc700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83280ce5f0 0x7f832814cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.055+0000 7f832d7dc700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832806c470 0x7f8328147020 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.055+0000 7f832d7dc700 1 --2- 172.21.15.112:0/1206260275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83280ce210 0x7f8328140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.055+0000 7f832d7dc700 1 -- 172.21.15.112:0/1206260275 >> 172.21.15.112:0/1206260275 conn(0x7f83280cef70 msgr2=0x7f832815c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:23.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.055+0000 7f832d7dc700 1 -- 172.21.15.112:0/1206260275 shutdown_connections 2023-08-27T11:18:23.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.055+0000 7f832d7dc700 1 -- 172.21.15.112:0/1206260275 wait complete. 2023-08-27T11:18:23.070 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:18:23.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.193+0000 7f4c3915c700 1 Processor -- start 2023-08-27T11:18:23.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.193+0000 7f4c3915c700 1 -- start start 2023-08-27T11:18:23.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3915c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4c340b3de0 0x7f4c3406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3915c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4c3406c510 0x7f4c340cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3915c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c340cef20 con 0x7f4c34144b80 2023-08-27T11:18:23.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3915c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c340b4b40 con 0x7f4c340b3de0 2023-08-27T11:18:23.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3915c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c340b7310 con 0x7f4c3406c510 2023-08-27T11:18:23.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3259c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4c340b3de0 0x7f4c3406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3259c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4c340b3de0 0x7f4c3406bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57212/0 (socket says 172.21.15.112:57212) 2023-08-27T11:18:23.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3259c700 1 -- 172.21.15.112:0/2106757193 learned_addr learned my addr 172.21.15.112:0/2106757193 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:23.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3259c700 1 -- 172.21.15.112:0/2106757193 >> v2:172.21.15.138:3300/0 conn(0x7f4c3406c510 msgr2=0x7f4c340cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:23.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3359e700 1 --2- 172.21.15.112:0/2106757193 >> v2:172.21.15.138:3300/0 conn(0x7f4c3406c510 0x7f4c340cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3259c700 1 --2- 172.21.15.112:0/2106757193 >> v2:172.21.15.138:3300/0 conn(0x7f4c3406c510 0x7f4c340cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3259c700 1 -- 172.21.15.112:0/2106757193 >> v1:172.21.15.112:6789/0 conn(0x7f4c34144b80 legacy=0x7f4c340b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3259c700 1 -- 172.21.15.112:0/2106757193 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3406c910 con 0x7f4c340b3de0 2023-08-27T11:18:23.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3359e700 1 --2- 172.21.15.112:0/2106757193 >> v2:172.21.15.138:3300/0 conn(0x7f4c3406c510 0x7f4c340cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:23.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c3259c700 1 --2- 172.21.15.112:0/2106757193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c340b3de0 0x7f4c3406bfd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4c34074170 tx=0x7f4c28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f4e6a428d144b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:23.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c31d9b700 1 -- 172.21.15.112:0/2106757193 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2800b870 con 0x7f4c340b3de0 2023-08-27T11:18:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c31d9b700 1 -- 172.21.15.112:0/2106757193 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2800b9d0 con 0x7f4c340b3de0 2023-08-27T11:18:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.194+0000 7f4c31d9b700 1 -- 172.21.15.112:0/2106757193 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2800bd60 con 0x7f4c340b3de0 2023-08-27T11:18:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.195+0000 7f4c3915c700 1 -- 172.21.15.112:0/2106757193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c340b3de0 msgr2=0x7f4c3406bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.195+0000 7f4c3915c700 1 --2- 172.21.15.112:0/2106757193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c340b3de0 0x7f4c3406bfd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4c34074170 tx=0x7f4c28009bd0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.195+0000 7f4c3915c700 1 -- 172.21.15.112:0/2106757193 shutdown_connections 2023-08-27T11:18:23.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.195+0000 7f4c3915c700 1 --2- 172.21.15.112:0/2106757193 >> v2:172.21.15.138:3300/0 conn(0x7f4c3406c510 0x7f4c340cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.195+0000 7f4c3915c700 1 --2- 172.21.15.112:0/2106757193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c340b3de0 0x7f4c3406bfd0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.195+0000 7f4c3915c700 1 -- 172.21.15.112:0/2106757193 >> 172.21.15.112:0/2106757193 conn(0x7f4c340cdc50 msgr2=0x7f4c340ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:23.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.195+0000 7f4c3915c700 1 -- 172.21.15.112:0/2106757193 shutdown_connections 2023-08-27T11:18:23.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.195+0000 7f4c3915c700 1 -- 172.21.15.112:0/2106757193 wait complete. 2023-08-27T11:18:23.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.196+0000 7f4c3915c700 1 Processor -- start 2023-08-27T11:18:23.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.197+0000 7f4c3915c700 1 -- start start 2023-08-27T11:18:23.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.197+0000 7f4c3915c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c3406c510 0x7f4c34071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.197+0000 7f4c3915c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c340b3de0 0x7f4c34142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.197+0000 7f4c3915c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c34144b80 0x7f4c341430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.197+0000 7f4c3915c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34158dd0 con 0x7f4c3406c510 2023-08-27T11:18:23.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.197+0000 7f4c3915c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34143620 con 0x7f4c34144b80 2023-08-27T11:18:23.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.197+0000 7f4c3915c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34158450 con 0x7f4c340b3de0 2023-08-27T11:18:23.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.197+0000 7f4c32d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c3406c510 0x7f4c34071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.197+0000 7f4c32d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c3406c510 0x7f4c34071ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57216/0 (socket says 172.21.15.112:57216) 2023-08-27T11:18:23.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.197+0000 7f4c32d9d700 1 -- 172.21.15.112:0/2822290762 learned_addr learned my addr 172.21.15.112:0/2822290762 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:23.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.198+0000 7f4c3359e700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c34144b80 0x7f4c341430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.198+0000 7f4c3259c700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c340b3de0 0x7f4c34142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.198+0000 7f4c32d9d700 1 -- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c340b3de0 msgr2=0x7f4c34142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.198+0000 7f4c32d9d700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c340b3de0 0x7f4c34142ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.198+0000 7f4c32d9d700 1 -- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c34144b80 msgr2=0x7f4c341430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.198+0000 7f4c32d9d700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c34144b80 0x7f4c341430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.198+0000 7f4c32d9d700 1 -- 172.21.15.112:0/2822290762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c28009550 con 0x7f4c3406c510 2023-08-27T11:18:23.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.198+0000 7f4c32d9d700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c3406c510 0x7f4c34071ad0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c450 tx=0x7f4c2400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:23.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.198+0000 7f4c1b7fe700 1 -- 172.21.15.112:0/2822290762 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c240052d0 con 0x7f4c3406c510 2023-08-27T11:18:23.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.198+0000 7f4c1b7fe700 1 -- 172.21.15.112:0/2822290762 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c24005430 con 0x7f4c3406c510 2023-08-27T11:18:23.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.198+0000 7f4c3915c700 1 -- 172.21.15.112:0/2822290762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c340cc9d0 con 0x7f4c3406c510 2023-08-27T11:18:23.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.198+0000 7f4c1b7fe700 1 -- 172.21.15.112:0/2822290762 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c240038c0 con 0x7f4c3406c510 2023-08-27T11:18:23.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.198+0000 7f4c3915c700 1 -- 172.21.15.112:0/2822290762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c341589f0 con 0x7f4c3406c510 2023-08-27T11:18:23.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.199+0000 7f4c3915c700 1 -- 172.21.15.112:0/2822290762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c34158f10 con 0x7f4c3406c510 2023-08-27T11:18:23.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.199+0000 7f4c1b7fe700 1 -- 172.21.15.112:0/2822290762 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4c24003a20 con 0x7f4c3406c510 2023-08-27T11:18:23.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.200+0000 7f4c1b7fe700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4c2006a0e0 0x7f4c2006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.200+0000 7f4c1b7fe700 1 -- 172.21.15.112:0/2822290762 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4c240878c0 con 0x7f4c3406c510 2023-08-27T11:18:23.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.200+0000 7f4c3259c700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4c2006a0e0 0x7f4c2006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.200+0000 7f4c3259c700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4c2006a0e0 0x7f4c2006c5a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4c28000c00 tx=0x7f4c28008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:23.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.204+0000 7f4c1b7fe700 1 -- 172.21.15.112:0/2822290762 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4c34158f10 con 0x7f4c3406c510 2023-08-27T11:18:23.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.395+0000 7f4c3915c700 1 -- 172.21.15.112:0/2822290762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4c34158f10 con 0x7f4c3406c510 2023-08-27T11:18:23.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.395+0000 7f4c1b7fe700 1 -- 172.21.15.112:0/2822290762 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4c34158f10 con 0x7f4c3406c510 2023-08-27T11:18:23.397 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:18:23.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.397+0000 7f4c3915c700 1 -- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4c2006a0e0 msgr2=0x7f4c2006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.397+0000 7f4c3915c700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4c2006a0e0 0x7f4c2006c5a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4c28000c00 tx=0x7f4c28008030 comp rx=0 tx=0).stop 2023-08-27T11:18:23.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.397+0000 7f4c3915c700 1 -- 172.21.15.112:0/2822290762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c3406c510 msgr2=0x7f4c34071ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.397+0000 7f4c3915c700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c3406c510 0x7f4c34071ad0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c450 tx=0x7f4c2400c760 comp rx=0 tx=0).stop 2023-08-27T11:18:23.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.397+0000 7f4c3915c700 1 -- 172.21.15.112:0/2822290762 shutdown_connections 2023-08-27T11:18:23.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.397+0000 7f4c3915c700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4c2006a0e0 0x7f4c2006c5a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.397+0000 7f4c3915c700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c34144b80 0x7f4c341430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.397+0000 7f4c3915c700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c3406c510 0x7f4c34071ad0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.397+0000 7f4c3915c700 1 --2- 172.21.15.112:0/2822290762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c340b3de0 0x7f4c34142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.397+0000 7f4c3915c700 1 -- 172.21.15.112:0/2822290762 >> 172.21.15.112:0/2822290762 conn(0x7f4c340cdc50 msgr2=0x7f4c340c6200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:23.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.397+0000 7f4c3915c700 1 -- 172.21.15.112:0/2822290762 shutdown_connections 2023-08-27T11:18:23.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.397+0000 7f4c3915c700 1 -- 172.21.15.112:0/2822290762 wait complete. 2023-08-27T11:18:23.410 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:18:23.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.533+0000 7f4726b1f700 1 Processor -- start 2023-08-27T11:18:23.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.533+0000 7f4726b1f700 1 -- start start 2023-08-27T11:18:23.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.533+0000 7f4726b1f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4720146b60 0x7f472006ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.533+0000 7f4726b1f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f47200ad1d0 0x7f472015d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.533+0000 7f4726b1f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4720160fa0 con 0x7f472006e2f0 2023-08-27T11:18:23.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.533+0000 7f4726b1f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4720161b70 con 0x7f4720146b60 2023-08-27T11:18:23.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.533+0000 7f4726b1f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4720143ad0 con 0x7f47200ad1d0 2023-08-27T11:18:23.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.533+0000 7f47248bb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4720146b60 0x7f472006ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.534+0000 7f47248bb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4720146b60 0x7f472006ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57224/0 (socket says 172.21.15.112:57224) 2023-08-27T11:18:23.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.534+0000 7f47248bb700 1 -- 172.21.15.112:0/2716363313 learned_addr learned my addr 172.21.15.112:0/2716363313 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:23.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.534+0000 7f47250bc700 1 --2- 172.21.15.112:0/2716363313 >> v2:172.21.15.138:3300/0 conn(0x7f47200ad1d0 0x7f472015d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.534+0000 7f47248bb700 1 -- 172.21.15.112:0/2716363313 >> v2:172.21.15.138:3300/0 conn(0x7f47200ad1d0 msgr2=0x7f472015d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:23.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.534+0000 7f47248bb700 1 --2- 172.21.15.112:0/2716363313 >> v2:172.21.15.138:3300/0 conn(0x7f47200ad1d0 0x7f472015d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.534+0000 7f47248bb700 1 -- 172.21.15.112:0/2716363313 >> v1:172.21.15.138:6789/0 conn(0x7f472006e2f0 legacy=0x7f47200acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.534+0000 7f47248bb700 1 -- 172.21.15.112:0/2716363313 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472006e750 con 0x7f4720146b60 2023-08-27T11:18:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.534+0000 7f47248bb700 1 --2- 172.21.15.112:0/2716363313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4720146b60 0x7f472006ddb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f47200cd100 tx=0x7f4714009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe1c7f160961b51 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.534+0000 7f471f7fe700 1 -- 172.21.15.112:0/2716363313 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714004230 con 0x7f4720146b60 2023-08-27T11:18:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.534+0000 7f471f7fe700 1 -- 172.21.15.112:0/2716363313 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4714004390 con 0x7f4720146b60 2023-08-27T11:18:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.534+0000 7f471f7fe700 1 -- 172.21.15.112:0/2716363313 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140036a0 con 0x7f4720146b60 2023-08-27T11:18:23.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.535+0000 7f4726b1f700 1 -- 172.21.15.112:0/2716363313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4720146b60 msgr2=0x7f472006ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.535+0000 7f4726b1f700 1 --2- 172.21.15.112:0/2716363313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4720146b60 0x7f472006ddb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f47200cd100 tx=0x7f4714009bd0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.535+0000 7f4726b1f700 1 -- 172.21.15.112:0/2716363313 shutdown_connections 2023-08-27T11:18:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.535+0000 7f4726b1f700 1 --2- 172.21.15.112:0/2716363313 >> v2:172.21.15.138:3300/0 conn(0x7f47200ad1d0 0x7f472015d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.535+0000 7f4726b1f700 1 --2- 172.21.15.112:0/2716363313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4720146b60 0x7f472006ddb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.535+0000 7f4726b1f700 1 -- 172.21.15.112:0/2716363313 >> 172.21.15.112:0/2716363313 conn(0x7f47200cc240 msgr2=0x7f47200cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.535+0000 7f4726b1f700 1 -- 172.21.15.112:0/2716363313 shutdown_connections 2023-08-27T11:18:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.535+0000 7f4726b1f700 1 -- 172.21.15.112:0/2716363313 wait complete. 2023-08-27T11:18:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.536+0000 7f4726b1f700 1 Processor -- start 2023-08-27T11:18:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.536+0000 7f4726b1f700 1 -- start start 2023-08-27T11:18:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f4726b1f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f472006e2f0 0x7f47201586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f4726b1f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47200ad1d0 0x7f472006f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f4726b1f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4720146b60 0x7f47201f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f4726b1f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472006fb40 con 0x7f4720146b60 2023-08-27T11:18:23.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f4726b1f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47200705f0 con 0x7f47200ad1d0 2023-08-27T11:18:23.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f4726b1f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472006fd50 con 0x7f472006e2f0 2023-08-27T11:18:23.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f47250bc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4720146b60 0x7f47201f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f47250bc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4720146b60 0x7f47201f2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57234/0 (socket says 172.21.15.112:57234) 2023-08-27T11:18:23.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f47250bc700 1 -- 172.21.15.112:0/4260721501 learned_addr learned my addr 172.21.15.112:0/4260721501 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:23.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f471ffff700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47200ad1d0 0x7f472006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f47248bb700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f472006e2f0 0x7f47201586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f47250bc700 1 -- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f472006e2f0 msgr2=0x7f47201586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f47250bc700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f472006e2f0 0x7f47201586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f47250bc700 1 -- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47200ad1d0 msgr2=0x7f472006f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f47250bc700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47200ad1d0 0x7f472006f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f47250bc700 1 -- 172.21.15.112:0/4260721501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4714009550 con 0x7f4720146b60 2023-08-27T11:18:23.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f47250bc700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4720146b60 0x7f47201f2530 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4718007e40 tx=0x7f471800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:23.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f471d7fa700 1 -- 172.21.15.112:0/4260721501 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471800eea0 con 0x7f4720146b60 2023-08-27T11:18:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f471d7fa700 1 -- 172.21.15.112:0/4260721501 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4718014420 con 0x7f4720146b60 2023-08-27T11:18:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f471d7fa700 1 -- 172.21.15.112:0/4260721501 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180147b0 con 0x7f4720146b60 2023-08-27T11:18:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f4726b1f700 1 -- 172.21.15.112:0/4260721501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4720070030 con 0x7f4720146b60 2023-08-27T11:18:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.537+0000 7f4726b1f700 1 -- 172.21.15.112:0/4260721501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4720070b10 con 0x7f4720146b60 2023-08-27T11:18:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.538+0000 7f4726b1f700 1 -- 172.21.15.112:0/4260721501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4720067e90 con 0x7f4720146b60 2023-08-27T11:18:23.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.539+0000 7f471d7fa700 1 -- 172.21.15.112:0/4260721501 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4718014910 con 0x7f4720146b60 2023-08-27T11:18:23.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.539+0000 7f471d7fa700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f470406a0e0 0x7f470406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.539+0000 7f471d7fa700 1 -- 172.21.15.112:0/4260721501 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4718088d20 con 0x7f4720146b60 2023-08-27T11:18:23.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.539+0000 7f47248bb700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f470406a0e0 0x7f470406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.540+0000 7f47248bb700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f470406a0e0 0x7f470406c5a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4714009130 tx=0x7f4714008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:23.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.543+0000 7f471d7fa700 1 -- 172.21.15.112:0/4260721501 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4718054440 con 0x7f4720146b60 2023-08-27T11:18:23.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.733+0000 7f4726b1f700 1 -- 172.21.15.112:0/4260721501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4720067a20 con 0x7f4720146b60 2023-08-27T11:18:23.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.734+0000 7f471d7fa700 1 -- 172.21.15.112:0/4260721501 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4718057a60 con 0x7f4720146b60 2023-08-27T11:18:23.736 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:18:23.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.736+0000 7f4726b1f700 1 -- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f470406a0e0 msgr2=0x7f470406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.736+0000 7f4726b1f700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f470406a0e0 0x7f470406c5a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4714009130 tx=0x7f4714008030 comp rx=0 tx=0).stop 2023-08-27T11:18:23.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.736+0000 7f4726b1f700 1 -- 172.21.15.112:0/4260721501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4720146b60 msgr2=0x7f47201f2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.736+0000 7f4726b1f700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4720146b60 0x7f47201f2530 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4718007e40 tx=0x7f471800e3c0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.736+0000 7f4726b1f700 1 -- 172.21.15.112:0/4260721501 shutdown_connections 2023-08-27T11:18:23.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.736+0000 7f4726b1f700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f470406a0e0 0x7f470406c5a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.736+0000 7f4726b1f700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4720146b60 0x7f47201f2530 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.736+0000 7f4726b1f700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f472006e2f0 0x7f47201586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.736+0000 7f4726b1f700 1 --2- 172.21.15.112:0/4260721501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47200ad1d0 0x7f472006f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.736+0000 7f4726b1f700 1 -- 172.21.15.112:0/4260721501 >> 172.21.15.112:0/4260721501 conn(0x7f47200cc240 msgr2=0x7f4720145c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:23.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.736+0000 7f4726b1f700 1 -- 172.21.15.112:0/4260721501 shutdown_connections 2023-08-27T11:18:23.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.736+0000 7f4726b1f700 1 -- 172.21.15.112:0/4260721501 wait complete. 2023-08-27T11:18:23.750 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:18:23.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.871+0000 7fc3ec089700 1 Processor -- start 2023-08-27T11:18:23.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.871+0000 7fc3ec089700 1 -- start start 2023-08-27T11:18:23.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3ec089700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3e406e3c0 0x7fc3e40ad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3ec089700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3e413ff80 0x7fc3e41403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3ec089700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e4140bb0 con 0x7fc3e406e7a0 2023-08-27T11:18:23.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3ec089700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e4140930 con 0x7fc3e413ff80 2023-08-27T11:18:23.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3ec089700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e4143470 con 0x7fc3e406e3c0 2023-08-27T11:18:23.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3e9624700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc3e406e7a0 0x7fc3e4142bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51620/0 (socket says 172.21.15.112:51620) 2023-08-27T11:18:23.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3e9624700 1 -- 172.21.15.112:0/3613863388 learned_addr learned my addr 172.21.15.112:0/3613863388 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:23.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3e9e25700 1 --2- 172.21.15.112:0/3613863388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3e406e3c0 0x7fc3e40ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3ea626700 1 --2- 172.21.15.112:0/3613863388 >> v2:172.21.15.138:3300/0 conn(0x7fc3e413ff80 0x7fc3e41403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023754540 0 0) 0x7fc3e4140bb0 con 0x7fc3e406e7a0 2023-08-27T11:18:23.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d00035b0 con 0x7fc3e406e7a0 2023-08-27T11:18:23.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 196140995 0 0) 0x7fc3d00035b0 con 0x7fc3e406e7a0 2023-08-27T11:18:23.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3e406e3c0 msgr2=0x7fc3e40ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:23.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3e8e23700 1 --2- 172.21.15.112:0/3613863388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3e406e3c0 0x7fc3e40ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 >> v2:172.21.15.138:3300/0 conn(0x7fc3e413ff80 msgr2=0x7fc3e41403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:23.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3e8e23700 1 --2- 172.21.15.112:0/3613863388 >> v2:172.21.15.138:3300/0 conn(0x7fc3e413ff80 0x7fc3e41403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e4143750 con 0x7fc3e406e7a0 2023-08-27T11:18:23.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.872+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fc3d4001670 con 0x7fc3e406e7a0 2023-08-27T11:18:23.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 >> v1:172.21.15.112:6789/0 conn(0x7fc3e406e7a0 legacy=0x7fc3e4142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3e8e23700 1 --2- 172.21.15.112:0/3613863388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3e406e3c0 0x7fc3d0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3e8e23700 1 --2- 172.21.15.112:0/3613863388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3d0003f80 0x7fc3d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3e8e23700 1 --2- 172.21.15.112:0/3613863388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d0004d90 0x7fc3d0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e4140930 con 0x7fc3d0004d90 2023-08-27T11:18:23.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e4143470 con 0x7fc3d0003f80 2023-08-27T11:18:23.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d000ba10 con 0x7fc3e406e3c0 2023-08-27T11:18:23.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3ea626700 1 --2- 172.21.15.112:0/3613863388 >> v2:172.21.15.138:3300/0 conn(0x7fc3e413ff80 0x7fc3e41403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:23.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3ea626700 1 --2- 172.21.15.112:0/3613863388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d0004d90 0x7fc3d0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3e9e25700 1 --2- 172.21.15.112:0/3613863388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3e406e3c0 0x7fc3d0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3e9624700 1 --2- 172.21.15.112:0/3613863388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3d0003f80 0x7fc3d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3ea626700 1 -- 172.21.15.112:0/3613863388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3e406e3c0 msgr2=0x7fc3d0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:23.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3ea626700 1 --2- 172.21.15.112:0/3613863388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3e406e3c0 0x7fc3d0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3ea626700 1 -- 172.21.15.112:0/3613863388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3d0003f80 msgr2=0x7fc3d0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:23.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3ea626700 1 --2- 172.21.15.112:0/3613863388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3d0003f80 0x7fc3d0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3ea626700 1 -- 172.21.15.112:0/3613863388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d00035b0 con 0x7fc3d0004d90 2023-08-27T11:18:23.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3e9624700 1 --2- 172.21.15.112:0/3613863388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3d0003f80 0x7fc3d0004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:23.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3ea626700 1 --2- 172.21.15.112:0/3613863388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d0004d90 0x7fc3d0005210 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00e350 tx=0x7fc3d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932fbf10fa95506f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:23.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc00edb0 con 0x7fc3d0004d90 2023-08-27T11:18:23.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc00f040 con 0x7fc3d0004d90 2023-08-27T11:18:23.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.873+0000 7fc3e8e23700 1 -- 172.21.15.112:0/3613863388 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc003c40 con 0x7fc3d0004d90 2023-08-27T11:18:23.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.874+0000 7fc3ec089700 1 -- 172.21.15.112:0/3613863388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d0004d90 msgr2=0x7fc3d0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.874+0000 7fc3ec089700 1 --2- 172.21.15.112:0/3613863388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d0004d90 0x7fc3d0005210 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00e350 tx=0x7fc3d0005880 comp rx=0 tx=0).stop 2023-08-27T11:18:23.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.874+0000 7fc3ea626700 1 -- 172.21.15.112:0/3613863388 reap_dead start 2023-08-27T11:18:23.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.874+0000 7fc3ec089700 1 -- 172.21.15.112:0/3613863388 shutdown_connections 2023-08-27T11:18:23.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.874+0000 7fc3ec089700 1 -- 172.21.15.112:0/3613863388 >> 172.21.15.112:0/3613863388 conn(0x7fc3e406cd30 msgr2=0x7fc3e40717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:23.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.874+0000 7fc3ec089700 1 -- 172.21.15.112:0/3613863388 shutdown_connections 2023-08-27T11:18:23.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.874+0000 7fc3ec089700 1 -- 172.21.15.112:0/3613863388 wait complete. 2023-08-27T11:18:23.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.875+0000 7fc3ec089700 1 Processor -- start 2023-08-27T11:18:23.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.875+0000 7fc3ec089700 1 -- start start 2023-08-27T11:18:23.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ec089700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3e41f2490 0x7fc3e415e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ec089700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e415f2f0 0x7fc3e41f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ec089700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e413c7f0 0x7fc3e413cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ec089700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e4135490 con 0x7fc3e413c7f0 2023-08-27T11:18:23.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ec089700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e413d1b0 con 0x7fc3e415f2f0 2023-08-27T11:18:23.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ec089700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e415fd40 con 0x7fc3e41f2490 2023-08-27T11:18:23.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ea626700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e413c7f0 0x7fc3e413cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ea626700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e413c7f0 0x7fc3e413cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57252/0 (socket says 172.21.15.112:57252) 2023-08-27T11:18:23.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ea626700 1 -- 172.21.15.112:0/3787486078 learned_addr learned my addr 172.21.15.112:0/3787486078 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:23.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3e9e25700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3e41f2490 0x7fc3e415e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3e9624700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e415f2f0 0x7fc3e41f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ea626700 1 -- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3e41f2490 msgr2=0x7fc3e415e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ea626700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3e41f2490 0x7fc3e415e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ea626700 1 -- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e415f2f0 msgr2=0x7fc3e41f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:23.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ea626700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e415f2f0 0x7fc3e41f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:23.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ea626700 1 -- 172.21.15.112:0/3787486078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e415fa60 con 0x7fc3e413c7f0 2023-08-27T11:18:23.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.876+0000 7fc3ea626700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e413c7f0 0x7fc3e413cc70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc004000 tx=0x7fc3dc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:23.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.877+0000 7fc3da7fc700 1 -- 172.21.15.112:0/3787486078 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc004440 con 0x7fc3e413c7f0 2023-08-27T11:18:23.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.877+0000 7fc3da7fc700 1 -- 172.21.15.112:0/3787486078 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc022070 con 0x7fc3e413c7f0 2023-08-27T11:18:23.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.877+0000 7fc3ec089700 1 -- 172.21.15.112:0/3787486078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e4134dd0 con 0x7fc3e413c7f0 2023-08-27T11:18:23.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.877+0000 7fc3da7fc700 1 -- 172.21.15.112:0/3787486078 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc004800 con 0x7fc3e413c7f0 2023-08-27T11:18:23.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.877+0000 7fc3ec089700 1 -- 172.21.15.112:0/3787486078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e415f770 con 0x7fc3e413c7f0 2023-08-27T11:18:23.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.877+0000 7fc3ec089700 1 -- 172.21.15.112:0/3787486078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e4067a20 con 0x7fc3e413c7f0 2023-08-27T11:18:23.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.878+0000 7fc3da7fc700 1 -- 172.21.15.112:0/3787486078 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc3dc014480 con 0x7fc3e413c7f0 2023-08-27T11:18:23.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.878+0000 7fc3da7fc700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc3c806a0e0 0x7fc3c806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:23.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.878+0000 7fc3da7fc700 1 -- 172.21.15.112:0/3787486078 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fc3dc024070 con 0x7fc3e413c7f0 2023-08-27T11:18:23.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.879+0000 7fc3e9e25700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc3c806a0e0 0x7fc3c806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:23.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.879+0000 7fc3e9e25700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc3c806a0e0 0x7fc3c806c5a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009c60 tx=0x7fc3e0009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:23.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:23.883+0000 7fc3da7fc700 1 -- 172.21.15.112:0/3787486078 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc3dc056060 con 0x7fc3e413c7f0 2023-08-27T11:18:24.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.087+0000 7fc3ec089700 1 -- 172.21.15.112:0/3787486078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc3e4067a20 con 0x7fc3e413c7f0 2023-08-27T11:18:24.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.088+0000 7fc3da7fc700 1 -- 172.21.15.112:0/3787486078 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v40) v1 ==== 80+0+124 (secure 0 0 0) 0x7fc3dc059680 con 0x7fc3e413c7f0 2023-08-27T11:18:24.089 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6835/1024768929 2023-08-28T11:18:18.799945+0000 2023-08-27T11:18:24.090 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6833/1024768929 2023-08-28T11:18:18.799945+0000 2023-08-27T11:18:24.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.090+0000 7fc3ec089700 1 -- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc3c806a0e0 msgr2=0x7fc3c806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.090+0000 7fc3ec089700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc3c806a0e0 0x7fc3c806c5a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009c60 tx=0x7fc3e0009210 comp rx=0 tx=0).stop 2023-08-27T11:18:24.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.090+0000 7fc3ec089700 1 -- 172.21.15.112:0/3787486078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e413c7f0 msgr2=0x7fc3e413cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.090+0000 7fc3ec089700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e413c7f0 0x7fc3e413cc70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc004000 tx=0x7fc3dc009750 comp rx=0 tx=0).stop 2023-08-27T11:18:24.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.090+0000 7fc3ec089700 1 -- 172.21.15.112:0/3787486078 shutdown_connections 2023-08-27T11:18:24.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.090+0000 7fc3ec089700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc3c806a0e0 0x7fc3c806c5a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.090+0000 7fc3ec089700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e413c7f0 0x7fc3e413cc70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.090+0000 7fc3ec089700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3e41f2490 0x7fc3e415e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.090+0000 7fc3ec089700 1 --2- 172.21.15.112:0/3787486078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e415f2f0 0x7fc3e41f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.090+0000 7fc3ec089700 1 -- 172.21.15.112:0/3787486078 >> 172.21.15.112:0/3787486078 conn(0x7fc3e406cd30 msgr2=0x7fc3e4145ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:24.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.090+0000 7fc3ec089700 1 -- 172.21.15.112:0/3787486078 shutdown_connections 2023-08-27T11:18:24.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.090+0000 7fc3ec089700 1 -- 172.21.15.112:0/3787486078 wait complete. 2023-08-27T11:18:24.094 INFO:teuthology.orchestra.run.smithi112.stderr:listed 2 entries 2023-08-27T11:18:24.103 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:18:24.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:24.104 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:18:24.143 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:18:24.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:24.144 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:18:24.201 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:18:24.202 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:24.223 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:18:24.224 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:24.283 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.402+0000 7f7cb5167700 1 Processor -- start 2023-08-27T11:18:24.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.402+0000 7f7cb5167700 1 -- start start 2023-08-27T11:18:24.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cb5167700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb006adb0 0x7f7cb006b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cb5167700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7cb00cdbd0 0x7f7cb00ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cb5167700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb015eb60 con 0x7f7cb0151060 2023-08-27T11:18:24.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cb5167700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb006b770 con 0x7f7cb00cdbd0 2023-08-27T11:18:24.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cb5167700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00b02f0 con 0x7f7cb006adb0 2023-08-27T11:18:24.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7caed9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7cb0151060 0x7f7cb015e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51638/0 (socket says 172.21.15.112:51638) 2023-08-27T11:18:24.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7caed9d700 1 -- 172.21.15.112:0/1382051838 learned_addr learned my addr 172.21.15.112:0/1382051838 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:24.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206847549 0 0) 0x7f7cb015eb60 con 0x7f7cb0151060 2023-08-27T11:18:24.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c980035b0 con 0x7f7cb0151060 2023-08-27T11:18:24.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7caf59e700 1 --2- 172.21.15.112:0/1382051838 >> v2:172.21.15.138:3300/0 conn(0x7f7cb00cdbd0 0x7f7cb00ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cae59c700 1 --2- 172.21.15.112:0/1382051838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb006adb0 0x7f7cb006b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3717100161 0 0) 0x7f7c980035b0 con 0x7f7cb0151060 2023-08-27T11:18:24.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb006adb0 msgr2=0x7f7cb006b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:24.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 --2- 172.21.15.112:0/1382051838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb006adb0 0x7f7cb006b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 >> v2:172.21.15.138:3300/0 conn(0x7f7cb00cdbd0 msgr2=0x7f7cb00ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 --2- 172.21.15.112:0/1382051838 >> v2:172.21.15.138:3300/0 conn(0x7f7cb00cdbd0 0x7f7cb00ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb00b05d0 con 0x7f7cb0151060 2023-08-27T11:18:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f7ca0001670 con 0x7f7cb0151060 2023-08-27T11:18:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cae59c700 1 --2- 172.21.15.112:0/1382051838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb006adb0 0x7f7cb006b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 >> v1:172.21.15.112:6789/0 conn(0x7f7cb0151060 legacy=0x7f7cb015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 --2- 172.21.15.112:0/1382051838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c98003a00 0x7f7c98003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 --2- 172.21.15.112:0/1382051838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb006adb0 0x7f7c98004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 --2- 172.21.15.112:0/1382051838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c98004d90 0x7f7c98005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb006b770 con 0x7f7c98003a00 2023-08-27T11:18:24.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00b02f0 con 0x7f7c98004d90 2023-08-27T11:18:24.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9800dca0 con 0x7f7cb006adb0 2023-08-27T11:18:24.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.403+0000 7f7caed9d700 1 --2- 172.21.15.112:0/1382051838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c98003a00 0x7f7c98003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7cae59c700 1 --2- 172.21.15.112:0/1382051838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb006adb0 0x7f7c98004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7caf59e700 1 --2- 172.21.15.112:0/1382051838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c98004d90 0x7f7c98005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7caed9d700 1 -- 172.21.15.112:0/1382051838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb006adb0 msgr2=0x7f7c98004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:24.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7caed9d700 1 --2- 172.21.15.112:0/1382051838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb006adb0 0x7f7c98004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7caed9d700 1 -- 172.21.15.112:0/1382051838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c98004d90 msgr2=0x7f7c98005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:24.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7caed9d700 1 --2- 172.21.15.112:0/1382051838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c98004d90 0x7f7c98005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7caed9d700 1 -- 172.21.15.112:0/1382051838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c980035b0 con 0x7f7c98003a00 2023-08-27T11:18:24.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7caed9d700 1 --2- 172.21.15.112:0/1382051838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c98003a00 0x7f7c98003e20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000c320 tx=0x7f7c98005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a7165950bfa8e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:24.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca000ce60 con 0x7f7c98003a00 2023-08-27T11:18:24.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca0005960 con 0x7f7c98003a00 2023-08-27T11:18:24.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7cadd9b700 1 -- 172.21.15.112:0/1382051838 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0005cf0 con 0x7f7c98003a00 2023-08-27T11:18:24.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7cb5167700 1 -- 172.21.15.112:0/1382051838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c98003a00 msgr2=0x7f7c98003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7cb5167700 1 --2- 172.21.15.112:0/1382051838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c98003a00 0x7f7c98003e20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000c320 tx=0x7f7c98005880 comp rx=0 tx=0).stop 2023-08-27T11:18:24.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7caf59e700 1 -- 172.21.15.112:0/1382051838 reap_dead start 2023-08-27T11:18:24.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7cb5167700 1 -- 172.21.15.112:0/1382051838 shutdown_connections 2023-08-27T11:18:24.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.404+0000 7f7cb5167700 1 -- 172.21.15.112:0/1382051838 >> 172.21.15.112:0/1382051838 conn(0x7f7cb00d0000 msgr2=0x7f7cb00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.405+0000 7f7cb5167700 1 -- 172.21.15.112:0/1382051838 shutdown_connections 2023-08-27T11:18:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.405+0000 7f7cb5167700 1 -- 172.21.15.112:0/1382051838 wait complete. 2023-08-27T11:18:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.406+0000 7f7cb5167700 1 Processor -- start 2023-08-27T11:18:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.406+0000 7f7cb5167700 1 -- start start 2023-08-27T11:18:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7cb5167700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c98003a00 0x7f7cb00c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7cb5167700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cb00c2f90 0x7f7cb00c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7cb5167700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb0070470 0x7f7cb00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7cb5167700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00c3410 con 0x7f7c98003a00 2023-08-27T11:18:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7cb5167700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00c8810 con 0x7f7cb00c2f90 2023-08-27T11:18:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7cb5167700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00c8af0 con 0x7f7cb0070470 2023-08-27T11:18:24.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7caed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c98003a00 0x7f7cb00c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7caed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c98003a00 0x7f7cb00c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57268/0 (socket says 172.21.15.112:57268) 2023-08-27T11:18:24.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7caed9d700 1 -- 172.21.15.112:0/341242891 learned_addr learned my addr 172.21.15.112:0/341242891 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:24.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7cae59c700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cb00c2f90 0x7f7cb00c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7caf59e700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb0070470 0x7f7cb00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7caed9d700 1 -- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb0070470 msgr2=0x7f7cb00cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7caed9d700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb0070470 0x7f7cb00cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7caed9d700 1 -- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cb00c2f90 msgr2=0x7f7cb00c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7caed9d700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cb00c2f90 0x7f7cb00c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7caed9d700 1 -- 172.21.15.112:0/341242891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb00c8dd0 con 0x7f7c98003a00 2023-08-27T11:18:24.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7caed9d700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c98003a00 0x7f7cb00c8200 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005810 tx=0x7f7ca0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:24.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7c977fe700 1 -- 172.21.15.112:0/341242891 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0000ed0 con 0x7f7c98003a00 2023-08-27T11:18:24.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.407+0000 7f7c977fe700 1 -- 172.21.15.112:0/341242891 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca0001680 con 0x7f7c98003a00 2023-08-27T11:18:24.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.408+0000 7f7cb5167700 1 -- 172.21.15.112:0/341242891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb00c90b0 con 0x7f7c98003a00 2023-08-27T11:18:24.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.408+0000 7f7c977fe700 1 -- 172.21.15.112:0/341242891 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0004cc0 con 0x7f7c98003a00 2023-08-27T11:18:24.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.408+0000 7f7cb5167700 1 -- 172.21.15.112:0/341242891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb00708f0 con 0x7f7c98003a00 2023-08-27T11:18:24.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.408+0000 7f7cb5167700 1 -- 172.21.15.112:0/341242891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb00c9700 con 0x7f7c98003a00 2023-08-27T11:18:24.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.409+0000 7f7c977fe700 1 -- 172.21.15.112:0/341242891 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7ca00070d0 con 0x7f7c98003a00 2023-08-27T11:18:24.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.409+0000 7f7c977fe700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c9c06a0e0 0x7f7c9c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.409+0000 7f7c977fe700 1 -- 172.21.15.112:0/341242891 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f7ca0089d50 con 0x7f7c98003a00 2023-08-27T11:18:24.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.409+0000 7f7cae59c700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c9c06a0e0 0x7f7c9c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.410+0000 7f7cae59c700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c9c06a0e0 0x7f7c9c06c5a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7cb014d940 tx=0x7f7ca4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:24.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.413+0000 7f7c977fe700 1 -- 172.21.15.112:0/341242891 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7ca008d030 con 0x7f7c98003a00 2023-08-27T11:18:24.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.604+0000 7f7cb5167700 1 -- 172.21.15.112:0/341242891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cb0067a20 con 0x7f7c98003a00 2023-08-27T11:18:24.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.605+0000 7f7c977fe700 1 -- 172.21.15.112:0/341242891 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18448 (secure 0 0 0) 0x7f7ca0055530 con 0x7f7c98003a00 2023-08-27T11:18:24.607 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:24.607 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":40,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:21.807644+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:01.935213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:02.602607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/1024768929":"2023-08-28T11:18:18.799945+0000","172.21.15.112:6833/1024768929":"2023-08-28T11:18:18.799945+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:24.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.607+0000 7f7cb5167700 1 -- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c9c06a0e0 msgr2=0x7f7c9c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.607+0000 7f7cb5167700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c9c06a0e0 0x7f7c9c06c5a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7cb014d940 tx=0x7f7ca4008040 comp rx=0 tx=0).stop 2023-08-27T11:18:24.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.607+0000 7f7cb5167700 1 -- 172.21.15.112:0/341242891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c98003a00 msgr2=0x7f7cb00c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.607+0000 7f7cb5167700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c98003a00 0x7f7cb00c8200 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005810 tx=0x7f7ca0001a00 comp rx=0 tx=0).stop 2023-08-27T11:18:24.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.607+0000 7f7cb5167700 1 -- 172.21.15.112:0/341242891 shutdown_connections 2023-08-27T11:18:24.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.607+0000 7f7cb5167700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c9c06a0e0 0x7f7c9c06c5a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.607+0000 7f7cb5167700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cb0070470 0x7f7cb00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.607+0000 7f7cb5167700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c98003a00 0x7f7cb00c8200 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.607+0000 7f7cb5167700 1 --2- 172.21.15.112:0/341242891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cb00c2f90 0x7f7cb00c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.607+0000 7f7cb5167700 1 -- 172.21.15.112:0/341242891 >> 172.21.15.112:0/341242891 conn(0x7f7cb00d0000 msgr2=0x7f7cb006de90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:24.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.607+0000 7f7cb5167700 1 -- 172.21.15.112:0/341242891 shutdown_connections 2023-08-27T11:18:24.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.607+0000 7f7cb5167700 1 -- 172.21.15.112:0/341242891 wait complete. 2023-08-27T11:18:24.621 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:18:01.935213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:18:02.602607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:18:24.621 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:18:24.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.748+0000 7f5ae9fc0700 1 Processor -- start 2023-08-27T11:18:24.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.749+0000 7f5ae9fc0700 1 -- start start 2023-08-27T11:18:24.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.749+0000 7f5ae9fc0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5ae415d300 0x7f5ae406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.749+0000 7f5ae9fc0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ae40b52f0 0x7f5ae406da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.749+0000 7f5ae9fc0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae40ad8e0 con 0x7f5ae406d580 2023-08-27T11:18:24.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.749+0000 7f5ae9fc0700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae40ae4f0 con 0x7f5ae415d300 2023-08-27T11:18:24.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.749+0000 7f5ae9fc0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae406df40 con 0x7f5ae40b52f0 2023-08-27T11:18:24.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae2ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5ae406d580 0x7f5ae40b4be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51660/0 (socket says 172.21.15.112:51660) 2023-08-27T11:18:24.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae2ffd700 1 -- 172.21.15.112:0/2546375993 learned_addr learned my addr 172.21.15.112:0/2546375993 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:24.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae37fe700 1 --2- 172.21.15.112:0/2546375993 >> v2:172.21.15.138:3300/0 conn(0x7f5ae415d300 0x7f5ae406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae3fff700 1 --2- 172.21.15.112:0/2546375993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ae40b52f0 0x7f5ae406da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156790734 0 0) 0x7f5ae40ad8e0 con 0x7f5ae406d580 2023-08-27T11:18:24.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5acc0035b0 con 0x7f5ae406d580 2023-08-27T11:18:24.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1567078596 0 0) 0x7f5acc0035b0 con 0x7f5ae406d580 2023-08-27T11:18:24.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ae40b52f0 msgr2=0x7f5ae406da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:24.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 --2- 172.21.15.112:0/2546375993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ae40b52f0 0x7f5ae406da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 >> v2:172.21.15.138:3300/0 conn(0x7f5ae415d300 msgr2=0x7f5ae406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:24.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 --2- 172.21.15.112:0/2546375993 >> v2:172.21.15.138:3300/0 conn(0x7f5ae415d300 0x7f5ae406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae40adbc0 con 0x7f5ae406d580 2023-08-27T11:18:24.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f5ad4001670 con 0x7f5ae406d580 2023-08-27T11:18:24.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 >> v1:172.21.15.112:6789/0 conn(0x7f5ae406d580 legacy=0x7f5ae40b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 --2- 172.21.15.112:0/2546375993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ae40b52f0 0x7f5acc003a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 --2- 172.21.15.112:0/2546375993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc003ff0 0x7f5acc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 --2- 172.21.15.112:0/2546375993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5acc004d90 0x7f5acc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae40ae4f0 con 0x7f5acc003ff0 2023-08-27T11:18:24.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae406df40 con 0x7f5acc004d90 2023-08-27T11:18:24.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc00ba40 con 0x7f5ae40b52f0 2023-08-27T11:18:24.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae2ffd700 1 --2- 172.21.15.112:0/2546375993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc003ff0 0x7f5acc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae37fe700 1 --2- 172.21.15.112:0/2546375993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ae40b52f0 0x7f5acc003a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae3fff700 1 --2- 172.21.15.112:0/2546375993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5acc004d90 0x7f5acc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae2ffd700 1 -- 172.21.15.112:0/2546375993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ae40b52f0 msgr2=0x7f5acc003a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:24.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae2ffd700 1 --2- 172.21.15.112:0/2546375993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ae40b52f0 0x7f5acc003a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae2ffd700 1 -- 172.21.15.112:0/2546375993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5acc004d90 msgr2=0x7f5acc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:24.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.750+0000 7f5ae2ffd700 1 --2- 172.21.15.112:0/2546375993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5acc004d90 0x7f5acc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.751+0000 7f5ae2ffd700 1 -- 172.21.15.112:0/2546375993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc0035b0 con 0x7f5acc003ff0 2023-08-27T11:18:24.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.751+0000 7f5ae2ffd700 1 --2- 172.21.15.112:0/2546375993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc003ff0 0x7f5acc004850 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c370 tx=0x7f5acc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0d57ed05db0517 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:24.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.751+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4007c60 con 0x7f5acc003ff0 2023-08-27T11:18:24.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.751+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad4005960 con 0x7f5acc003ff0 2023-08-27T11:18:24.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.751+0000 7f5ae27fc700 1 -- 172.21.15.112:0/2546375993 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4005d10 con 0x7f5acc003ff0 2023-08-27T11:18:24.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.751+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/2546375993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc003ff0 msgr2=0x7f5acc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.751+0000 7f5ae9fc0700 1 --2- 172.21.15.112:0/2546375993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc003ff0 0x7f5acc004850 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c370 tx=0x7f5acc005880 comp rx=0 tx=0).stop 2023-08-27T11:18:24.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.751+0000 7f5ae3fff700 1 -- 172.21.15.112:0/2546375993 reap_dead start 2023-08-27T11:18:24.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.751+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/2546375993 shutdown_connections 2023-08-27T11:18:24.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.751+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/2546375993 >> 172.21.15.112:0/2546375993 conn(0x7f5ae4066400 msgr2=0x7f5ae40cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:24.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.751+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/2546375993 shutdown_connections 2023-08-27T11:18:24.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.751+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/2546375993 wait complete. 2023-08-27T11:18:24.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.753+0000 7f5ae9fc0700 1 Processor -- start 2023-08-27T11:18:24.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.753+0000 7f5ae9fc0700 1 -- start start 2023-08-27T11:18:24.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.754+0000 7f5ae9fc0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc003ff0 0x7f5ae4159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.754+0000 7f5ae9fc0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ae415bbb0 0x7f5ae4150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.754+0000 7f5ae9fc0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ae4141540 0x7f5ae415f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.754+0000 7f5ae9fc0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae4151e10 con 0x7f5acc003ff0 2023-08-27T11:18:24.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.754+0000 7f5ae9fc0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae41508a0 con 0x7f5ae415bbb0 2023-08-27T11:18:24.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.754+0000 7f5ae9fc0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae41520b0 con 0x7f5ae4141540 2023-08-27T11:18:24.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.754+0000 7f5ae37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc003ff0 0x7f5ae4159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.754+0000 7f5ae37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc003ff0 0x7f5ae4159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57288/0 (socket says 172.21.15.112:57288) 2023-08-27T11:18:24.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.754+0000 7f5ae37fe700 1 -- 172.21.15.112:0/759450530 learned_addr learned my addr 172.21.15.112:0/759450530 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:24.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.754+0000 7f5ae37fe700 1 -- 172.21.15.112:0/759450530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ae4141540 msgr2=0x7f5ae415f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.754+0000 7f5ae37fe700 1 --2- 172.21.15.112:0/759450530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ae4141540 0x7f5ae415f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.754+0000 7f5ae37fe700 1 -- 172.21.15.112:0/759450530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ae415bbb0 msgr2=0x7f5ae4150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.755+0000 7f5ae37fe700 1 --2- 172.21.15.112:0/759450530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ae415bbb0 0x7f5ae4150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.755+0000 7f5ae37fe700 1 -- 172.21.15.112:0/759450530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae4152390 con 0x7f5acc003ff0 2023-08-27T11:18:24.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.755+0000 7f5ae37fe700 1 --2- 172.21.15.112:0/759450530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc003ff0 0x7f5ae4159af0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8011a20 tx=0x7f5ad8011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:24.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.755+0000 7f5acbfff700 1 -- 172.21.15.112:0/759450530 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad800f940 con 0x7f5acc003ff0 2023-08-27T11:18:24.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.755+0000 7f5acbfff700 1 -- 172.21.15.112:0/759450530 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad800fac0 con 0x7f5acc003ff0 2023-08-27T11:18:24.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.755+0000 7f5acbfff700 1 -- 172.21.15.112:0/759450530 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad800fe70 con 0x7f5acc003ff0 2023-08-27T11:18:24.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.755+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/759450530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae4152670 con 0x7f5acc003ff0 2023-08-27T11:18:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.755+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/759450530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae41419c0 con 0x7f5acc003ff0 2023-08-27T11:18:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.756+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/759450530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae4152cc0 con 0x7f5acc003ff0 2023-08-27T11:18:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.756+0000 7f5acbfff700 1 -- 172.21.15.112:0/759450530 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5ad8014440 con 0x7f5acc003ff0 2023-08-27T11:18:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.757+0000 7f5acbfff700 1 --2- 172.21.15.112:0/759450530 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ad006a0e0 0x7f5ad006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.757+0000 7f5acbfff700 1 -- 172.21.15.112:0/759450530 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f5ad8088ca0 con 0x7f5acc003ff0 2023-08-27T11:18:24.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.757+0000 7f5ae2ffd700 1 --2- 172.21.15.112:0/759450530 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ad006a0e0 0x7f5ad006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:24.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.758+0000 7f5ae2ffd700 1 --2- 172.21.15.112:0/759450530 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ad006a0e0 0x7f5ad006c5a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40b0780 tx=0x7f5ad400f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:24.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.761+0000 7f5acbfff700 1 -- 172.21.15.112:0/759450530 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5ad8057f30 con 0x7f5acc003ff0 2023-08-27T11:18:24.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.953+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/759450530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5ae4061f50 con 0x7f5acc003ff0 2023-08-27T11:18:24.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.953+0000 7f5acbfff700 1 -- 172.21.15.112:0/759450530 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5ad8057ac0 con 0x7f5acc003ff0 2023-08-27T11:18:24.955 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:18:24.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.955+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/759450530 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ad006a0e0 msgr2=0x7f5ad006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.955+0000 7f5ae9fc0700 1 --2- 172.21.15.112:0/759450530 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ad006a0e0 0x7f5ad006c5a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40b0780 tx=0x7f5ad400f000 comp rx=0 tx=0).stop 2023-08-27T11:18:24.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.955+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/759450530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc003ff0 msgr2=0x7f5ae4159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.955+0000 7f5ae9fc0700 1 --2- 172.21.15.112:0/759450530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc003ff0 0x7f5ae4159af0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8011a20 tx=0x7f5ad8011de0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.955+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/759450530 shutdown_connections 2023-08-27T11:18:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.955+0000 7f5ae9fc0700 1 --2- 172.21.15.112:0/759450530 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ad006a0e0 0x7f5ad006c5a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.955+0000 7f5ae9fc0700 1 --2- 172.21.15.112:0/759450530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ae4141540 0x7f5ae415f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.955+0000 7f5ae9fc0700 1 --2- 172.21.15.112:0/759450530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc003ff0 0x7f5ae4159af0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.955+0000 7f5ae9fc0700 1 --2- 172.21.15.112:0/759450530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ae415bbb0 0x7f5ae4150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:24.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.955+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/759450530 >> 172.21.15.112:0/759450530 conn(0x7f5ae4066400 msgr2=0x7f5ae40afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:24.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.955+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/759450530 shutdown_connections 2023-08-27T11:18:24.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:24.955+0000 7f5ae9fc0700 1 -- 172.21.15.112:0/759450530 wait complete. 2023-08-27T11:18:24.969 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:18:25.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.093+0000 7f647fc36700 1 Processor -- start 2023-08-27T11:18:25.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.093+0000 7f647fc36700 1 -- start start 2023-08-27T11:18:25.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647fc36700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64781437c0 0x7f64780cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647fc36700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6478143ba0 0x7f647806d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647fc36700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f647806dac0 0x7f64780afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647fc36700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647806df70 con 0x7f64781437c0 2023-08-27T11:18:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647fc36700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64780cfde0 con 0x7f647806dac0 2023-08-27T11:18:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647fc36700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64780cf410 con 0x7f6478143ba0 2023-08-27T11:18:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647d9d2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64781437c0 0x7f64780cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647d9d2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64781437c0 0x7f64780cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57296/0 (socket says 172.21.15.112:57296) 2023-08-27T11:18:25.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647d9d2700 1 -- 172.21.15.112:0/422264097 learned_addr learned my addr 172.21.15.112:0/422264097 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:25.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647d1d1700 1 --2- 172.21.15.112:0/422264097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6478143ba0 0x7f647806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647e1d3700 1 --2- 172.21.15.112:0/422264097 >> v2:172.21.15.138:3300/0 conn(0x7f647806dac0 0x7f64780afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647d9d2700 1 -- 172.21.15.112:0/422264097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6478143ba0 msgr2=0x7f647806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:25.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647d9d2700 1 --2- 172.21.15.112:0/422264097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6478143ba0 0x7f647806d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647d9d2700 1 -- 172.21.15.112:0/422264097 >> v2:172.21.15.138:3300/0 conn(0x7f647806dac0 msgr2=0x7f64780afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:25.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647d9d2700 1 --2- 172.21.15.112:0/422264097 >> v2:172.21.15.138:3300/0 conn(0x7f647806dac0 0x7f64780afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647d9d2700 1 -- 172.21.15.112:0/422264097 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64780b09e0 con 0x7f64781437c0 2023-08-27T11:18:25.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647e1d3700 1 --2- 172.21.15.112:0/422264097 >> v2:172.21.15.138:3300/0 conn(0x7f647806dac0 0x7f64780afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:25.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647d1d1700 1 --2- 172.21.15.112:0/422264097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6478143ba0 0x7f647806d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:25.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647d9d2700 1 --2- 172.21.15.112:0/422264097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64781437c0 0x7f64780cf8a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f64780b4f80 tx=0x7f6474009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=affdff5a5919e160 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:25.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.094+0000 7f647c9d0700 1 -- 172.21.15.112:0/422264097 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647400b870 con 0x7f64781437c0 2023-08-27T11:18:25.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.095+0000 7f647c9d0700 1 -- 172.21.15.112:0/422264097 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647400b9d0 con 0x7f64781437c0 2023-08-27T11:18:25.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.095+0000 7f647c9d0700 1 -- 172.21.15.112:0/422264097 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647400bd60 con 0x7f64781437c0 2023-08-27T11:18:25.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.095+0000 7f647fc36700 1 -- 172.21.15.112:0/422264097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64781437c0 msgr2=0x7f64780cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.095+0000 7f647fc36700 1 --2- 172.21.15.112:0/422264097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64781437c0 0x7f64780cf8a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f64780b4f80 tx=0x7f6474009bd0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.095+0000 7f647fc36700 1 -- 172.21.15.112:0/422264097 shutdown_connections 2023-08-27T11:18:25.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.095+0000 7f647fc36700 1 --2- 172.21.15.112:0/422264097 >> v2:172.21.15.138:3300/0 conn(0x7f647806dac0 0x7f64780afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.095+0000 7f647fc36700 1 --2- 172.21.15.112:0/422264097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64781437c0 0x7f64780cf8a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.095+0000 7f647fc36700 1 --2- 172.21.15.112:0/422264097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6478143ba0 0x7f647806d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.095+0000 7f647fc36700 1 -- 172.21.15.112:0/422264097 >> 172.21.15.112:0/422264097 conn(0x7f64780b1e90 msgr2=0x7f64780b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:25.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.095+0000 7f647fc36700 1 -- 172.21.15.112:0/422264097 shutdown_connections 2023-08-27T11:18:25.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.095+0000 7f647fc36700 1 -- 172.21.15.112:0/422264097 wait complete. 2023-08-27T11:18:25.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.096+0000 7f647fc36700 1 Processor -- start 2023-08-27T11:18:25.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.096+0000 7f647fc36700 1 -- start start 2023-08-27T11:18:25.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647fc36700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f647806dac0 0x7f6478155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647fc36700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64781437c0 0x7f6478136c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647fc36700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6478143ba0 0x7f6478137190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647fc36700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647813ca30 con 0x7f647806dac0 2023-08-27T11:18:25.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647fc36700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64781558d0 con 0x7f6478143ba0 2023-08-27T11:18:25.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647fc36700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6478070160 con 0x7f64781437c0 2023-08-27T11:18:25.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647d9d2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f647806dac0 0x7f6478155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647d9d2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f647806dac0 0x7f6478155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57302/0 (socket says 172.21.15.112:57302) 2023-08-27T11:18:25.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647d9d2700 1 -- 172.21.15.112:0/1569810400 learned_addr learned my addr 172.21.15.112:0/1569810400 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:25.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647d1d1700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64781437c0 0x7f6478136c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647e1d3700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6478143ba0 0x7f6478137190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647d9d2700 1 -- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64781437c0 msgr2=0x7f6478136c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647d9d2700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64781437c0 0x7f6478136c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647d9d2700 1 -- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6478143ba0 msgr2=0x7f6478137190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647d9d2700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6478143ba0 0x7f6478137190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647d9d2700 1 -- 172.21.15.112:0/1569810400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6474009550 con 0x7f647806dac0 2023-08-27T11:18:25.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.097+0000 7f647e1d3700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6478143ba0 0x7f6478137190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:25.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.098+0000 7f647d9d2700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f647806dac0 0x7f6478155390 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f647806e3d0 tx=0x7f6474004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:25.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.098+0000 7f646e7fc700 1 -- 172.21.15.112:0/1569810400 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6474004310 con 0x7f647806dac0 2023-08-27T11:18:25.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.098+0000 7f646e7fc700 1 -- 172.21.15.112:0/1569810400 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647401f070 con 0x7f647806dac0 2023-08-27T11:18:25.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.098+0000 7f647fc36700 1 -- 172.21.15.112:0/1569810400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647813bdb0 con 0x7f647806dac0 2023-08-27T11:18:25.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.098+0000 7f646e7fc700 1 -- 172.21.15.112:0/1569810400 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64740036a0 con 0x7f647806dac0 2023-08-27T11:18:25.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.098+0000 7f647fc36700 1 -- 172.21.15.112:0/1569810400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647813c370 con 0x7f647806dac0 2023-08-27T11:18:25.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.099+0000 7f647fc36700 1 -- 172.21.15.112:0/1569810400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647813cca0 con 0x7f647806dac0 2023-08-27T11:18:25.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.099+0000 7f646e7fc700 1 -- 172.21.15.112:0/1569810400 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6474021030 con 0x7f647806dac0 2023-08-27T11:18:25.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.099+0000 7f646e7fc700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f645c06a0e0 0x7f645c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.100+0000 7f646e7fc700 1 -- 172.21.15.112:0/1569810400 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f64740899a0 con 0x7f647806dac0 2023-08-27T11:18:25.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.100+0000 7f647d1d1700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f645c06a0e0 0x7f645c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.100+0000 7f647d1d1700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f645c06a0e0 0x7f645c06c5a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6468006fd0 tx=0x7f6468008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:25.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.104+0000 7f646e7fc700 1 -- 172.21.15.112:0/1569810400 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f64740550c0 con 0x7f647806dac0 2023-08-27T11:18:25.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.290+0000 7f647fc36700 1 -- 172.21.15.112:0/1569810400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6478067a20 con 0x7f647806dac0 2023-08-27T11:18:25.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.291+0000 7f646e7fc700 1 -- 172.21.15.112:0/1569810400 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6474015020 con 0x7f647806dac0 2023-08-27T11:18:25.292 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:18:25.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.293+0000 7f647fc36700 1 -- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f645c06a0e0 msgr2=0x7f645c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.293+0000 7f647fc36700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f645c06a0e0 0x7f645c06c5a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6468006fd0 tx=0x7f6468008040 comp rx=0 tx=0).stop 2023-08-27T11:18:25.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.293+0000 7f647fc36700 1 -- 172.21.15.112:0/1569810400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f647806dac0 msgr2=0x7f6478155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.293+0000 7f647fc36700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f647806dac0 0x7f6478155390 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f647806e3d0 tx=0x7f6474004750 comp rx=0 tx=0).stop 2023-08-27T11:18:25.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.293+0000 7f647fc36700 1 -- 172.21.15.112:0/1569810400 shutdown_connections 2023-08-27T11:18:25.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.293+0000 7f647fc36700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f645c06a0e0 0x7f645c06c5a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.293+0000 7f647fc36700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6478143ba0 0x7f6478137190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.293+0000 7f647fc36700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f647806dac0 0x7f6478155390 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.293+0000 7f647fc36700 1 --2- 172.21.15.112:0/1569810400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64781437c0 0x7f6478136c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.293+0000 7f647fc36700 1 -- 172.21.15.112:0/1569810400 >> 172.21.15.112:0/1569810400 conn(0x7f64780b1e90 msgr2=0x7f647813db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:25.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.293+0000 7f647fc36700 1 -- 172.21.15.112:0/1569810400 shutdown_connections 2023-08-27T11:18:25.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.293+0000 7f647fc36700 1 -- 172.21.15.112:0/1569810400 wait complete. 2023-08-27T11:18:25.309 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:18:25.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.430+0000 7efd43b13700 1 Processor -- start 2023-08-27T11:18:25.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.431+0000 7efd43b13700 1 -- start start 2023-08-27T11:18:25.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.431+0000 7efd43b13700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c15ed00 0x7efd3c0b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.431+0000 7efd43b13700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3c15f750 con 0x7efd3c15f180 2023-08-27T11:18:25.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.431+0000 7efd43b13700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3c15fa30 con 0x7efd3c152130 2023-08-27T11:18:25.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.431+0000 7efd43b13700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0b60a0 con 0x7efd3c15ed00 2023-08-27T11:18:25.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd420b0700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7efd3c15f180 0x7efd3c0b68a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51696/0 (socket says 172.21.15.112:51696) 2023-08-27T11:18:25.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd420b0700 1 -- 172.21.15.112:0/44923533 learned_addr learned my addr 172.21.15.112:0/44923533 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:25.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd410ae700 1 --2- 172.21.15.112:0/44923533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c15ed00 0x7efd3c0b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd408ad700 1 -- 172.21.15.112:0/44923533 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368107323 0 0) 0x7efd3c15f750 con 0x7efd3c15f180 2023-08-27T11:18:25.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd408ad700 1 -- 172.21.15.112:0/44923533 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd280035b0 con 0x7efd3c15f180 2023-08-27T11:18:25.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd408ad700 1 -- 172.21.15.112:0/44923533 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742736242 0 0) 0x7efd3c15fa30 con 0x7efd3c152130 2023-08-27T11:18:25.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd408ad700 1 -- 172.21.15.112:0/44923533 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd3c15f750 con 0x7efd3c152130 2023-08-27T11:18:25.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd408ad700 1 -- 172.21.15.112:0/44923533 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4291716314 0 0) 0x7efd280035b0 con 0x7efd3c15f180 2023-08-27T11:18:25.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd410ae700 1 -- 172.21.15.112:0/44923533 >> v1:172.21.15.138:6789/0 conn(0x7efd3c152130 legacy=0x7efd3c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd410ae700 1 -- 172.21.15.112:0/44923533 >> v1:172.21.15.112:6789/0 conn(0x7efd3c15f180 legacy=0x7efd3c0b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd410ae700 1 -- 172.21.15.112:0/44923533 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c0b6380 con 0x7efd3c15ed00 2023-08-27T11:18:25.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd410ae700 1 --2- 172.21.15.112:0/44923533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c15ed00 0x7efd3c0b3cf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009870 tx=0x7efd2c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f845d18cfe6d8a8f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:25.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd408ad700 1 -- 172.21.15.112:0/44923533 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340019f0 con 0x7efd3c15ed00 2023-08-27T11:18:25.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd408ad700 1 -- 172.21.15.112:0/44923533 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c004030 con 0x7efd3c15ed00 2023-08-27T11:18:25.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.432+0000 7efd408ad700 1 -- 172.21.15.112:0/44923533 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0043a0 con 0x7efd3c15ed00 2023-08-27T11:18:25.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.433+0000 7efd43b13700 1 -- 172.21.15.112:0/44923533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c15ed00 msgr2=0x7efd3c0b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.433+0000 7efd43b13700 1 --2- 172.21.15.112:0/44923533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c15ed00 0x7efd3c0b3cf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009870 tx=0x7efd2c009b80 comp rx=0 tx=0).stop 2023-08-27T11:18:25.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.433+0000 7efd43b13700 1 -- 172.21.15.112:0/44923533 shutdown_connections 2023-08-27T11:18:25.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.433+0000 7efd43b13700 1 --2- 172.21.15.112:0/44923533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c15ed00 0x7efd3c0b3cf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.433+0000 7efd43b13700 1 -- 172.21.15.112:0/44923533 >> 172.21.15.112:0/44923533 conn(0x7efd3c066010 msgr2=0x7efd3c0663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:25.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.433+0000 7efd43b13700 1 -- 172.21.15.112:0/44923533 shutdown_connections 2023-08-27T11:18:25.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.433+0000 7efd43b13700 1 -- 172.21.15.112:0/44923533 wait complete. 2023-08-27T11:18:25.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.434+0000 7efd43b13700 1 Processor -- start 2023-08-27T11:18:25.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.434+0000 7efd43b13700 1 -- start start 2023-08-27T11:18:25.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd43b13700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c152130 0x7efd3c0708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd43b13700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c15ed00 0x7efd3c072240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd43b13700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd3c15f180 0x7efd3c150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd43b13700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c071db0 con 0x7efd3c15f180 2023-08-27T11:18:25.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd43b13700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0711a0 con 0x7efd3c152130 2023-08-27T11:18:25.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd43b13700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c071480 con 0x7efd3c15ed00 2023-08-27T11:18:25.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd420b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd3c15f180 0x7efd3c150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd420b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd3c15f180 0x7efd3c150d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57320/0 (socket says 172.21.15.112:57320) 2023-08-27T11:18:25.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd420b0700 1 -- 172.21.15.112:0/103357738 learned_addr learned my addr 172.21.15.112:0/103357738 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:25.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd410ae700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c15ed00 0x7efd3c072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd418af700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c152130 0x7efd3c0708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd420b0700 1 -- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c15ed00 msgr2=0x7efd3c072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd420b0700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c15ed00 0x7efd3c072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd420b0700 1 -- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c152130 msgr2=0x7efd3c0708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd420b0700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c152130 0x7efd3c0708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.435+0000 7efd420b0700 1 -- 172.21.15.112:0/103357738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c0094a0 con 0x7efd3c15f180 2023-08-27T11:18:25.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.436+0000 7efd420b0700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd3c15f180 0x7efd3c150d00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7efd3400cba0 tx=0x7efd3400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:25.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.436+0000 7efd327fc700 1 -- 172.21.15.112:0/103357738 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3400aaa0 con 0x7efd3c15f180 2023-08-27T11:18:25.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.436+0000 7efd327fc700 1 -- 172.21.15.112:0/103357738 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3400ac20 con 0x7efd3c15f180 2023-08-27T11:18:25.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.436+0000 7efd327fc700 1 -- 172.21.15.112:0/103357738 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340070d0 con 0x7efd3c15f180 2023-08-27T11:18:25.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.436+0000 7efd43b13700 1 -- 172.21.15.112:0/103357738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c071760 con 0x7efd3c15f180 2023-08-27T11:18:25.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.436+0000 7efd43b13700 1 -- 172.21.15.112:0/103357738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c070de0 con 0x7efd3c15f180 2023-08-27T11:18:25.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.437+0000 7efd43b13700 1 -- 172.21.15.112:0/103357738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c071f20 con 0x7efd3c15f180 2023-08-27T11:18:25.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.437+0000 7efd327fc700 1 -- 172.21.15.112:0/103357738 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efd34007230 con 0x7efd3c15f180 2023-08-27T11:18:25.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.437+0000 7efd327fc700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efd2006a0e0 0x7efd2006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.438+0000 7efd327fc700 1 -- 172.21.15.112:0/103357738 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7efd340880f0 con 0x7efd3c15f180 2023-08-27T11:18:25.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.438+0000 7efd418af700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efd2006a0e0 0x7efd2006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.438+0000 7efd418af700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efd2006a0e0 0x7efd2006c5a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efd38001220 tx=0x7efd38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:25.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.442+0000 7efd327fc700 1 -- 172.21.15.112:0/103357738 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efd34053780 con 0x7efd3c15f180 2023-08-27T11:18:25.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.629+0000 7efd43b13700 1 -- 172.21.15.112:0/103357738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd3c067a20 con 0x7efd3c15f180 2023-08-27T11:18:25.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.630+0000 7efd327fc700 1 -- 172.21.15.112:0/103357738 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd34056da0 con 0x7efd3c15f180 2023-08-27T11:18:25.631 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:18:25.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.631+0000 7efd43b13700 1 -- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efd2006a0e0 msgr2=0x7efd2006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.631+0000 7efd43b13700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efd2006a0e0 0x7efd2006c5a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efd38001220 tx=0x7efd38009040 comp rx=0 tx=0).stop 2023-08-27T11:18:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.631+0000 7efd43b13700 1 -- 172.21.15.112:0/103357738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd3c15f180 msgr2=0x7efd3c150d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.631+0000 7efd43b13700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd3c15f180 0x7efd3c150d00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7efd3400cba0 tx=0x7efd3400ceb0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.631+0000 7efd43b13700 1 -- 172.21.15.112:0/103357738 shutdown_connections 2023-08-27T11:18:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.631+0000 7efd43b13700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efd2006a0e0 0x7efd2006c5a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.631+0000 7efd43b13700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd3c15f180 0x7efd3c150d00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.631+0000 7efd43b13700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c152130 0x7efd3c0708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.631+0000 7efd43b13700 1 --2- 172.21.15.112:0/103357738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c15ed00 0x7efd3c072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.631+0000 7efd43b13700 1 -- 172.21.15.112:0/103357738 >> 172.21.15.112:0/103357738 conn(0x7efd3c066010 msgr2=0x7efd3c15d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.632+0000 7efd43b13700 1 -- 172.21.15.112:0/103357738 shutdown_connections 2023-08-27T11:18:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.632+0000 7efd43b13700 1 -- 172.21.15.112:0/103357738 wait complete. 2023-08-27T11:18:25.645 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:18:25.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.766+0000 7f646022e700 1 Processor -- start 2023-08-27T11:18:25.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.766+0000 7f646022e700 1 -- start start 2023-08-27T11:18:25.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f646022e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64581437c0 0x7f64580b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f646022e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f645806d5b0 0x7f645806da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f646022e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64580cfd50 con 0x7f6458143ba0 2023-08-27T11:18:25.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f646022e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645806df80 con 0x7f64581437c0 2023-08-27T11:18:25.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f646022e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64580cf140 con 0x7f645806d5b0 2023-08-27T11:18:25.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f645dfca700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64581437c0 0x7f64580b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f645dfca700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64581437c0 0x7f64580b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57324/0 (socket says 172.21.15.112:57324) 2023-08-27T11:18:25.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f645dfca700 1 -- 172.21.15.112:0/644505165 learned_addr learned my addr 172.21.15.112:0/644505165 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:25.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f645e7cb700 1 --2- 172.21.15.112:0/644505165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f645806d5b0 0x7f645806da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f645dfca700 1 -- 172.21.15.112:0/644505165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f645806d5b0 msgr2=0x7f645806da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:25.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f645dfca700 1 --2- 172.21.15.112:0/644505165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f645806d5b0 0x7f645806da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f645dfca700 1 -- 172.21.15.112:0/644505165 >> v1:172.21.15.138:6789/0 conn(0x7f6458143ba0 legacy=0x7f64580ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f645dfca700 1 -- 172.21.15.112:0/644505165 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64580cf420 con 0x7f64581437c0 2023-08-27T11:18:25.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.767+0000 7f645e7cb700 1 --2- 172.21.15.112:0/644505165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f645806d5b0 0x7f645806da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:25.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.768+0000 7f645dfca700 1 --2- 172.21.15.112:0/644505165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64581437c0 0x7f64580b4670 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f64480098f0 tx=0x7f6448009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b492cc4050f6d28f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:25.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.768+0000 7f645cfc8700 1 -- 172.21.15.112:0/644505165 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644800b8a0 con 0x7f64581437c0 2023-08-27T11:18:25.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.768+0000 7f645cfc8700 1 -- 172.21.15.112:0/644505165 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644800ba00 con 0x7f64581437c0 2023-08-27T11:18:25.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.768+0000 7f645cfc8700 1 -- 172.21.15.112:0/644505165 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644800bd90 con 0x7f64581437c0 2023-08-27T11:18:25.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.768+0000 7f646022e700 1 -- 172.21.15.112:0/644505165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64581437c0 msgr2=0x7f64580b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.768+0000 7f646022e700 1 --2- 172.21.15.112:0/644505165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64581437c0 0x7f64580b4670 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f64480098f0 tx=0x7f6448009c00 comp rx=0 tx=0).stop 2023-08-27T11:18:25.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.768+0000 7f646022e700 1 -- 172.21.15.112:0/644505165 shutdown_connections 2023-08-27T11:18:25.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.768+0000 7f646022e700 1 --2- 172.21.15.112:0/644505165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f645806d5b0 0x7f645806da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.768+0000 7f646022e700 1 --2- 172.21.15.112:0/644505165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64581437c0 0x7f64580b4670 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.768+0000 7f646022e700 1 -- 172.21.15.112:0/644505165 >> 172.21.15.112:0/644505165 conn(0x7f64580b1e90 msgr2=0x7f64580b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:25.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.768+0000 7f646022e700 1 -- 172.21.15.112:0/644505165 shutdown_connections 2023-08-27T11:18:25.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.768+0000 7f646022e700 1 -- 172.21.15.112:0/644505165 wait complete. 2023-08-27T11:18:25.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.770+0000 7f646022e700 1 Processor -- start 2023-08-27T11:18:25.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.770+0000 7f646022e700 1 -- start start 2023-08-27T11:18:25.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.770+0000 7f646022e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f645806d5b0 0x7f6458141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.770+0000 7f646022e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64581437c0 0x7f6458155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.770+0000 7f646022e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6458143ba0 0x7f645813b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.770+0000 7f646022e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645813cfd0 con 0x7f645806d5b0 2023-08-27T11:18:25.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.770+0000 7f646022e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6458142440 con 0x7f6458143ba0 2023-08-27T11:18:25.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.770+0000 7f646022e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645815c080 con 0x7f64581437c0 2023-08-27T11:18:25.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.770+0000 7f645dfca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f645806d5b0 0x7f6458141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.770+0000 7f645dfca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f645806d5b0 0x7f6458141d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57330/0 (socket says 172.21.15.112:57330) 2023-08-27T11:18:25.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.770+0000 7f645dfca700 1 -- 172.21.15.112:0/596385303 learned_addr learned my addr 172.21.15.112:0/596385303 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:25.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.771+0000 7f645dfca700 1 -- 172.21.15.112:0/596385303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64581437c0 msgr2=0x7f6458155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.771+0000 7f645dfca700 1 --2- 172.21.15.112:0/596385303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64581437c0 0x7f6458155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.771+0000 7f645dfca700 1 -- 172.21.15.112:0/596385303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6458143ba0 msgr2=0x7f645813b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.771+0000 7f645dfca700 1 --2- 172.21.15.112:0/596385303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6458143ba0 0x7f645813b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.771+0000 7f645dfca700 1 -- 172.21.15.112:0/596385303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6448009550 con 0x7f645806d5b0 2023-08-27T11:18:25.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.771+0000 7f645dfca700 1 --2- 172.21.15.112:0/596385303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f645806d5b0 0x7f6458141d40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6448009c40 tx=0x7f6448004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:25.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.771+0000 7f644e7fc700 1 -- 172.21.15.112:0/596385303 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448004400 con 0x7f645806d5b0 2023-08-27T11:18:25.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.771+0000 7f644e7fc700 1 -- 172.21.15.112:0/596385303 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644801f070 con 0x7f645806d5b0 2023-08-27T11:18:25.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.771+0000 7f646022e700 1 -- 172.21.15.112:0/596385303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645815c360 con 0x7f645806d5b0 2023-08-27T11:18:25.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.771+0000 7f644e7fc700 1 -- 172.21.15.112:0/596385303 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64480038c0 con 0x7f645806d5b0 2023-08-27T11:18:25.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.771+0000 7f646022e700 1 -- 172.21.15.112:0/596385303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645815c920 con 0x7f645806d5b0 2023-08-27T11:18:25.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.772+0000 7f646022e700 1 -- 172.21.15.112:0/596385303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645806f750 con 0x7f645806d5b0 2023-08-27T11:18:25.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.772+0000 7f644e7fc700 1 -- 172.21.15.112:0/596385303 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6448021030 con 0x7f645806d5b0 2023-08-27T11:18:25.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.773+0000 7f644e7fc700 1 --2- 172.21.15.112:0/596385303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f643c06a0e0 0x7f643c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:25.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.773+0000 7f644e7fc700 1 -- 172.21.15.112:0/596385303 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f6448089d50 con 0x7f645806d5b0 2023-08-27T11:18:25.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.773+0000 7f645d7c9700 1 --2- 172.21.15.112:0/596385303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f643c06a0e0 0x7f643c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:25.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.773+0000 7f645d7c9700 1 --2- 172.21.15.112:0/596385303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f643c06a0e0 0x7f643c06c5a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6454000f60 tx=0x7f6454007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:25.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.777+0000 7f644e7fc700 1 -- 172.21.15.112:0/596385303 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f644808d030 con 0x7f645806d5b0 2023-08-27T11:18:25.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.969+0000 7f646022e700 1 -- 172.21.15.112:0/596385303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6458067e90 con 0x7f645806d5b0 2023-08-27T11:18:25.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.969+0000 7f644e7fc700 1 -- 172.21.15.112:0/596385303 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6448055590 con 0x7f645806d5b0 2023-08-27T11:18:25.972 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:18:25.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.971+0000 7f646022e700 1 -- 172.21.15.112:0/596385303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f643c06a0e0 msgr2=0x7f643c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.971+0000 7f646022e700 1 --2- 172.21.15.112:0/596385303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f643c06a0e0 0x7f643c06c5a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6454000f60 tx=0x7f6454007a40 comp rx=0 tx=0).stop 2023-08-27T11:18:25.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.971+0000 7f646022e700 1 -- 172.21.15.112:0/596385303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f645806d5b0 msgr2=0x7f6458141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:25.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.971+0000 7f646022e700 1 --2- 172.21.15.112:0/596385303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f645806d5b0 0x7f6458141d40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6448009c40 tx=0x7f6448004750 comp rx=0 tx=0).stop 2023-08-27T11:18:25.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.971+0000 7f646022e700 1 -- 172.21.15.112:0/596385303 shutdown_connections 2023-08-27T11:18:25.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.971+0000 7f646022e700 1 --2- 172.21.15.112:0/596385303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f643c06a0e0 0x7f643c06c5a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.971+0000 7f646022e700 1 --2- 172.21.15.112:0/596385303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6458143ba0 0x7f645813b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.971+0000 7f646022e700 1 --2- 172.21.15.112:0/596385303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f645806d5b0 0x7f6458141d40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.971+0000 7f646022e700 1 --2- 172.21.15.112:0/596385303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64581437c0 0x7f6458155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:25.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.971+0000 7f646022e700 1 -- 172.21.15.112:0/596385303 >> 172.21.15.112:0/596385303 conn(0x7f64580b1e90 msgr2=0x7f64580b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:25.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.971+0000 7f646022e700 1 -- 172.21.15.112:0/596385303 shutdown_connections 2023-08-27T11:18:25.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:25.971+0000 7f646022e700 1 -- 172.21.15.112:0/596385303 wait complete. 2023-08-27T11:18:25.985 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:18:26.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.104+0000 7fc53ae2e700 1 Processor -- start 2023-08-27T11:18:26.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.104+0000 7fc53ae2e700 1 -- start start 2023-08-27T11:18:26.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc53ae2e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc534144570 0x7fc53415ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc53ae2e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc5340b4ad0 0x7fc5340b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc53ae2e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5340b45f0 con 0x7fc5340b3e50 2023-08-27T11:18:26.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc53ae2e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5340cf7a0 con 0x7fc5340b4ad0 2023-08-27T11:18:26.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc53ae2e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc534144950 con 0x7fc534144570 2023-08-27T11:18:26.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc533fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc5340b3e50 0x7fc53415f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51720/0 (socket says 172.21.15.112:51720) 2023-08-27T11:18:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc533fff700 1 -- 172.21.15.112:0/591158718 learned_addr learned my addr 172.21.15.112:0/591158718 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5393cb700 1 --2- 172.21.15.112:0/591158718 >> v2:172.21.15.138:3300/0 conn(0x7fc5340b4ad0 0x7fc5340b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc538bca700 1 --2- 172.21.15.112:0/591158718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc534144570 0x7fc53415ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216444115 0 0) 0x7fc5340b45f0 con 0x7fc5340b3e50 2023-08-27T11:18:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5240035b0 con 0x7fc5340b3e50 2023-08-27T11:18:26.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3349113323 0 0) 0x7fc5240035b0 con 0x7fc5340b3e50 2023-08-27T11:18:26.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc534144570 msgr2=0x7fc53415ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:26.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5337fe700 1 --2- 172.21.15.112:0/591158718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc534144570 0x7fc53415ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 >> v2:172.21.15.138:3300/0 conn(0x7fc5340b4ad0 msgr2=0x7fc5340b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:26.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5337fe700 1 --2- 172.21.15.112:0/591158718 >> v2:172.21.15.138:3300/0 conn(0x7fc5340b4ad0 0x7fc5340b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5340b4310 con 0x7fc5340b3e50 2023-08-27T11:18:26.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fc520001670 con 0x7fc5340b3e50 2023-08-27T11:18:26.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 >> v1:172.21.15.112:6789/0 conn(0x7fc5340b3e50 legacy=0x7fc53415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5337fe700 1 --2- 172.21.15.112:0/591158718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524003a30 0x7fc524003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5337fe700 1 --2- 172.21.15.112:0/591158718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc524004850 0x7fc524004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.105+0000 7fc5337fe700 1 --2- 172.21.15.112:0/591158718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc534144570 0x7fc524005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340cf7a0 con 0x7fc524003a30 2023-08-27T11:18:26.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc534144950 con 0x7fc524004850 2023-08-27T11:18:26.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52400ba40 con 0x7fc534144570 2023-08-27T11:18:26.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc538bca700 1 --2- 172.21.15.112:0/591158718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524003a30 0x7fc524003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc533fff700 1 --2- 172.21.15.112:0/591158718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc524004850 0x7fc524004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc5393cb700 1 --2- 172.21.15.112:0/591158718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc534144570 0x7fc524005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc538bca700 1 -- 172.21.15.112:0/591158718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc534144570 msgr2=0x7fc524005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:26.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc538bca700 1 --2- 172.21.15.112:0/591158718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc534144570 0x7fc524005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc538bca700 1 -- 172.21.15.112:0/591158718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc524004850 msgr2=0x7fc524004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:26.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc538bca700 1 --2- 172.21.15.112:0/591158718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc524004850 0x7fc524004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc538bca700 1 -- 172.21.15.112:0/591158718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240035b0 con 0x7fc524003a30 2023-08-27T11:18:26.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc538bca700 1 --2- 172.21.15.112:0/591158718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524003a30 0x7fc524003e50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc528009b60 tx=0x7fc524005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e8e9a130e4f983 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:26.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52800e660 con 0x7fc524003a30 2023-08-27T11:18:26.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52800e7c0 con 0x7fc524003a30 2023-08-27T11:18:26.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.106+0000 7fc5337fe700 1 -- 172.21.15.112:0/591158718 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52800eb30 con 0x7fc524003a30 2023-08-27T11:18:26.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.107+0000 7fc53ae2e700 1 -- 172.21.15.112:0/591158718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524003a30 msgr2=0x7fc524003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.107+0000 7fc53ae2e700 1 --2- 172.21.15.112:0/591158718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524003a30 0x7fc524003e50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc528009b60 tx=0x7fc524005880 comp rx=0 tx=0).stop 2023-08-27T11:18:26.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.107+0000 7fc5393cb700 1 -- 172.21.15.112:0/591158718 reap_dead start 2023-08-27T11:18:26.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.107+0000 7fc53ae2e700 1 -- 172.21.15.112:0/591158718 shutdown_connections 2023-08-27T11:18:26.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.107+0000 7fc53ae2e700 1 -- 172.21.15.112:0/591158718 >> 172.21.15.112:0/591158718 conn(0x7fc53406f970 msgr2=0x7fc53406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:26.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.107+0000 7fc53ae2e700 1 -- 172.21.15.112:0/591158718 shutdown_connections 2023-08-27T11:18:26.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.107+0000 7fc53ae2e700 1 -- 172.21.15.112:0/591158718 wait complete. 2023-08-27T11:18:26.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.108+0000 7fc53ae2e700 1 Processor -- start 2023-08-27T11:18:26.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.108+0000 7fc53ae2e700 1 -- start start 2023-08-27T11:18:26.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc53ae2e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524003a30 0x7fc5341576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc53ae2e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc534157c20 0x7fc534140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc53ae2e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc534130f10 0x7fc534141390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc53ae2e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5341e4c80 con 0x7fc524003a30 2023-08-27T11:18:26.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc53ae2e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5341e4f60 con 0x7fc534157c20 2023-08-27T11:18:26.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc53ae2e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5341e5240 con 0x7fc534130f10 2023-08-27T11:18:26.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc538bca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524003a30 0x7fc5341576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc538bca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524003a30 0x7fc5341576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57350/0 (socket says 172.21.15.112:57350) 2023-08-27T11:18:26.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc538bca700 1 -- 172.21.15.112:0/1680072864 learned_addr learned my addr 172.21.15.112:0/1680072864 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:26.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc5393cb700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc534130f10 0x7fc534141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc533fff700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc534157c20 0x7fc534140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc538bca700 1 -- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc534130f10 msgr2=0x7fc534141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc538bca700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc534130f10 0x7fc534141390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc538bca700 1 -- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc534157c20 msgr2=0x7fc534140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc538bca700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc534157c20 0x7fc534140e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc538bca700 1 -- 172.21.15.112:0/1680072864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5341e5520 con 0x7fc524003a30 2023-08-27T11:18:26.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.109+0000 7fc538bca700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524003a30 0x7fc5341576e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc524004770 tx=0x7fc5280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:26.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.110+0000 7fc5317fa700 1 -- 172.21.15.112:0/1680072864 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52800e660 con 0x7fc524003a30 2023-08-27T11:18:26.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.110+0000 7fc5317fa700 1 -- 172.21.15.112:0/1680072864 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc528003ab0 con 0x7fc524003a30 2023-08-27T11:18:26.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.110+0000 7fc53ae2e700 1 -- 172.21.15.112:0/1680072864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5341e5800 con 0x7fc524003a30 2023-08-27T11:18:26.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.110+0000 7fc5317fa700 1 -- 172.21.15.112:0/1680072864 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528003e60 con 0x7fc524003a30 2023-08-27T11:18:26.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.110+0000 7fc53ae2e700 1 -- 172.21.15.112:0/1680072864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53415b8a0 con 0x7fc524003a30 2023-08-27T11:18:26.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.111+0000 7fc53ae2e700 1 -- 172.21.15.112:0/1680072864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc534067a20 con 0x7fc524003a30 2023-08-27T11:18:26.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.111+0000 7fc5317fa700 1 -- 172.21.15.112:0/1680072864 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc528013470 con 0x7fc524003a30 2023-08-27T11:18:26.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.111+0000 7fc5317fa700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc51806a110 0x7fc51806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.111+0000 7fc5317fa700 1 -- 172.21.15.112:0/1680072864 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fc528089f00 con 0x7fc524003a30 2023-08-27T11:18:26.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.112+0000 7fc533fff700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc51806a110 0x7fc51806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.112+0000 7fc533fff700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc51806a110 0x7fc51806c5d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc520001050 tx=0x7fc520009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:26.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.116+0000 7fc5317fa700 1 -- 172.21.15.112:0/1680072864 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc528055740 con 0x7fc524003a30 2023-08-27T11:18:26.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.318+0000 7fc53ae2e700 1 -- 172.21.15.112:0/1680072864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc534140150 con 0x7fc524003a30 2023-08-27T11:18:26.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.319+0000 7fc5317fa700 1 -- 172.21.15.112:0/1680072864 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v40) v1 ==== 80+0+124 (secure 0 0 0) 0x7fc528058d60 con 0x7fc524003a30 2023-08-27T11:18:26.322 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6835/1024768929 2023-08-28T11:18:18.799945+0000 2023-08-27T11:18:26.322 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6833/1024768929 2023-08-28T11:18:18.799945+0000 2023-08-27T11:18:26.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.321+0000 7fc53ae2e700 1 -- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc51806a110 msgr2=0x7fc51806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.321+0000 7fc53ae2e700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc51806a110 0x7fc51806c5d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc520001050 tx=0x7fc520009040 comp rx=0 tx=0).stop 2023-08-27T11:18:26.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.321+0000 7fc53ae2e700 1 -- 172.21.15.112:0/1680072864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524003a30 msgr2=0x7fc5341576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.321+0000 7fc53ae2e700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524003a30 0x7fc5341576e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc524004770 tx=0x7fc5280047c0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.321+0000 7fc53ae2e700 1 -- 172.21.15.112:0/1680072864 shutdown_connections 2023-08-27T11:18:26.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.321+0000 7fc53ae2e700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc51806a110 0x7fc51806c5d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.321+0000 7fc53ae2e700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc534130f10 0x7fc534141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.321+0000 7fc53ae2e700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524003a30 0x7fc5341576e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.321+0000 7fc53ae2e700 1 --2- 172.21.15.112:0/1680072864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc534157c20 0x7fc534140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.321+0000 7fc53ae2e700 1 -- 172.21.15.112:0/1680072864 >> 172.21.15.112:0/1680072864 conn(0x7fc53406f970 msgr2=0x7fc5341433f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:26.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.321+0000 7fc53ae2e700 1 -- 172.21.15.112:0/1680072864 shutdown_connections 2023-08-27T11:18:26.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.321+0000 7fc53ae2e700 1 -- 172.21.15.112:0/1680072864 wait complete. 2023-08-27T11:18:26.326 INFO:teuthology.orchestra.run.smithi112.stderr:listed 2 entries 2023-08-27T11:18:26.334 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-27T11:18:26.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:26.336 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-08-27T11:18:26.391 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:18:26.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:26.392 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-27T11:18:26.411 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:18:26.412 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:26.430 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:18:26.430 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:18:26.431 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:26.470 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:18:26.470 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:18:26.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.588+0000 7fc3779ec700 1 Processor -- start 2023-08-27T11:18:26.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.588+0000 7fc3779ec700 1 -- start start 2023-08-27T11:18:26.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.589+0000 7fc3779ec700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc37015cd10 0x7fc37015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.589+0000 7fc3779ec700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37015bc70 con 0x7fc3701574e0 2023-08-27T11:18:26.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.589+0000 7fc3779ec700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37015bf50 con 0x7fc37015d860 2023-08-27T11:18:26.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.589+0000 7fc3779ec700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3700cf460 con 0x7fc37015cd10 2023-08-27T11:18:26.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.589+0000 7fc375788700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc3701574e0 0x7fc3700ced50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51740/0 (socket says 172.21.15.112:51740) 2023-08-27T11:18:26.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.589+0000 7fc375788700 1 -- 172.21.15.112:0/486407215 learned_addr learned my addr 172.21.15.112:0/486407215 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:26.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc374f87700 1 --2- 172.21.15.112:0/486407215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc37015cd10 0x7fc37015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142766696 0 0) 0x7fc37015bc70 con 0x7fc3701574e0 2023-08-27T11:18:26.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc35c0035b0 con 0x7fc3701574e0 2023-08-27T11:18:26.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147704103 0 0) 0x7fc37015bf50 con 0x7fc37015d860 2023-08-27T11:18:26.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc37015bc70 con 0x7fc37015d860 2023-08-27T11:18:26.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 27648814 0 0) 0x7fc35c0035b0 con 0x7fc3701574e0 2023-08-27T11:18:26.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc37015cd10 msgr2=0x7fc37015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:26.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 --2- 172.21.15.112:0/486407215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc37015cd10 0x7fc37015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 >> v1:172.21.15.138:6789/0 conn(0x7fc37015d860 legacy=0x7fc370157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37015c230 con 0x7fc3701574e0 2023-08-27T11:18:26.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fc360001670 con 0x7fc3701574e0 2023-08-27T11:18:26.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 >> v1:172.21.15.112:6789/0 conn(0x7fc3701574e0 legacy=0x7fc3700ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 --2- 172.21.15.112:0/486407215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc37015cd10 0x7fc35c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 --2- 172.21.15.112:0/486407215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc35c004170 0x7fc35c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 --2- 172.21.15.112:0/486407215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc35c004d90 0x7fc35c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3700cf460 con 0x7fc35c004170 2023-08-27T11:18:26.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c00bbd0 con 0x7fc35c004d90 2023-08-27T11:18:26.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c00bd90 con 0x7fc37015cd10 2023-08-27T11:18:26.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc374f87700 1 --2- 172.21.15.112:0/486407215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc35c004170 0x7fc35c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc375788700 1 --2- 172.21.15.112:0/486407215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc37015cd10 0x7fc35c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc375f89700 1 --2- 172.21.15.112:0/486407215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc35c004d90 0x7fc35c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc374f87700 1 -- 172.21.15.112:0/486407215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc37015cd10 msgr2=0x7fc35c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:26.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc374f87700 1 --2- 172.21.15.112:0/486407215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc37015cd10 0x7fc35c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc374f87700 1 -- 172.21.15.112:0/486407215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc35c004d90 msgr2=0x7fc35c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:26.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc374f87700 1 --2- 172.21.15.112:0/486407215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc35c004d90 0x7fc35c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.590+0000 7fc374f87700 1 -- 172.21.15.112:0/486407215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c0035b0 con 0x7fc35c004170 2023-08-27T11:18:26.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.591+0000 7fc374f87700 1 --2- 172.21.15.112:0/486407215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc35c004170 0x7fc35c004850 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0099c0 tx=0x7fc35c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbeee16610676918 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:26.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.591+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0106f0 con 0x7fc35c004170 2023-08-27T11:18:26.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.591+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c010850 con 0x7fc35c004170 2023-08-27T11:18:26.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.591+0000 7fc367fff700 1 -- 172.21.15.112:0/486407215 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c010bc0 con 0x7fc35c004170 2023-08-27T11:18:26.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.591+0000 7fc3779ec700 1 -- 172.21.15.112:0/486407215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc35c004170 msgr2=0x7fc35c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.591+0000 7fc3779ec700 1 --2- 172.21.15.112:0/486407215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc35c004170 0x7fc35c004850 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0099c0 tx=0x7fc35c005880 comp rx=0 tx=0).stop 2023-08-27T11:18:26.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.591+0000 7fc375f89700 1 -- 172.21.15.112:0/486407215 reap_dead start 2023-08-27T11:18:26.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.591+0000 7fc3779ec700 1 -- 172.21.15.112:0/486407215 shutdown_connections 2023-08-27T11:18:26.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.591+0000 7fc3779ec700 1 -- 172.21.15.112:0/486407215 >> 172.21.15.112:0/486407215 conn(0x7fc3700b1380 msgr2=0x7fc3700b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:26.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.591+0000 7fc3779ec700 1 -- 172.21.15.112:0/486407215 shutdown_connections 2023-08-27T11:18:26.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.591+0000 7fc3779ec700 1 -- 172.21.15.112:0/486407215 wait complete. 2023-08-27T11:18:26.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.593+0000 7fc3779ec700 1 Processor -- start 2023-08-27T11:18:26.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.593+0000 7fc3779ec700 1 -- start start 2023-08-27T11:18:26.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.593+0000 7fc3779ec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc35c004170 0x7fc3701e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.593+0000 7fc3779ec700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3701f6dc0 0x7fc3700b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.593+0000 7fc3779ec700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3701f3560 0x7fc3701f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.593+0000 7fc3779ec700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37015d170 con 0x7fc35c004170 2023-08-27T11:18:26.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.593+0000 7fc3779ec700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc370150b30 con 0x7fc3701f6dc0 2023-08-27T11:18:26.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.593+0000 7fc3779ec700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3701e8100 con 0x7fc3701f3560 2023-08-27T11:18:26.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.593+0000 7fc375788700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc35c004170 0x7fc3701e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.593+0000 7fc375788700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc35c004170 0x7fc3701e7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57374/0 (socket says 172.21.15.112:57374) 2023-08-27T11:18:26.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.593+0000 7fc375788700 1 -- 172.21.15.112:0/1256695866 learned_addr learned my addr 172.21.15.112:0/1256695866 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:26.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.594+0000 7fc374f87700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3701f6dc0 0x7fc3700b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.594+0000 7fc375f89700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3701f3560 0x7fc3701f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.594+0000 7fc375788700 1 -- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3701f3560 msgr2=0x7fc3701f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.594+0000 7fc375788700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3701f3560 0x7fc3701f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.594+0000 7fc375788700 1 -- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3701f6dc0 msgr2=0x7fc3700b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.594+0000 7fc375788700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3701f6dc0 0x7fc3700b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.594+0000 7fc375788700 1 -- 172.21.15.112:0/1256695866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3701f2dc0 con 0x7fc35c004170 2023-08-27T11:18:26.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.594+0000 7fc375788700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc35c004170 0x7fc3701e7bc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc360004f70 tx=0x7fc3600073f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:26.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.594+0000 7fc365ffb700 1 -- 172.21.15.112:0/1256695866 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360011580 con 0x7fc35c004170 2023-08-27T11:18:26.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.594+0000 7fc365ffb700 1 -- 172.21.15.112:0/1256695866 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc360011720 con 0x7fc35c004170 2023-08-27T11:18:26.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.594+0000 7fc3779ec700 1 -- 172.21.15.112:0/1256695866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3701f7250 con 0x7fc35c004170 2023-08-27T11:18:26.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.594+0000 7fc365ffb700 1 -- 172.21.15.112:0/1256695866 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360011ad0 con 0x7fc35c004170 2023-08-27T11:18:26.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.594+0000 7fc3779ec700 1 -- 172.21.15.112:0/1256695866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3701f3040 con 0x7fc35c004170 2023-08-27T11:18:26.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.595+0000 7fc3779ec700 1 -- 172.21.15.112:0/1256695866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc370067a20 con 0x7fc35c004170 2023-08-27T11:18:26.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.595+0000 7fc365ffb700 1 -- 172.21.15.112:0/1256695866 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc360011c30 con 0x7fc35c004170 2023-08-27T11:18:26.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.596+0000 7fc365ffb700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc35406a0e0 0x7fc35406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.596+0000 7fc365ffb700 1 -- 172.21.15.112:0/1256695866 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fc3600885b0 con 0x7fc35c004170 2023-08-27T11:18:26.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.596+0000 7fc374f87700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc35406a0e0 0x7fc35406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.596+0000 7fc374f87700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc35406a0e0 0x7fc35406c5a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc36c000c00 tx=0x7fc36c00e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:26.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.600+0000 7fc365ffb700 1 -- 172.21.15.112:0/1256695866 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc360053c40 con 0x7fc35c004170 2023-08-27T11:18:26.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.832+0000 7fc3779ec700 1 -- 172.21.15.112:0/1256695866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc370072290 con 0x7fc35c004170 2023-08-27T11:18:26.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.833+0000 7fc365ffb700 1 -- 172.21.15.112:0/1256695866 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4887 (secure 0 0 0) 0x7fc360057260 con 0x7fc35c004170 2023-08-27T11:18:26.834 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:26.835 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1941625698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1941625698},{"type":"v1","addr":"172.21.15.138:6836","nonce":1941625698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4904,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1317948937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1317948937},{"type":"v1","addr":"172.21.15.112:6835","nonce":1317948937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1530886230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1530886230},{"type":"v1","addr":"172.21.15.112:6834","nonce":1530886230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:18:03.637863+0000","modified":"2023-08-27T11:18:07.262281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5175":{"gid":5175,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6837/3409848660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3409848660},{"type":"v1","addr":"172.21.15.138:6837","nonce":3409848660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-27T11:18:26.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.835+0000 7fc3779ec700 1 -- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc35406a0e0 msgr2=0x7fc35406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.835+0000 7fc3779ec700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc35406a0e0 0x7fc35406c5a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc36c000c00 tx=0x7fc36c00e000 comp rx=0 tx=0).stop 2023-08-27T11:18:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.835+0000 7fc3779ec700 1 -- 172.21.15.112:0/1256695866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc35c004170 msgr2=0x7fc3701e7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.835+0000 7fc3779ec700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc35c004170 0x7fc3701e7bc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc360004f70 tx=0x7fc3600073f0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.835+0000 7fc3779ec700 1 -- 172.21.15.112:0/1256695866 shutdown_connections 2023-08-27T11:18:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.835+0000 7fc3779ec700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc35406a0e0 0x7fc35406c5a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.835+0000 7fc3779ec700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3701f3560 0x7fc3701f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.835+0000 7fc3779ec700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc35c004170 0x7fc3701e7bc0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.835+0000 7fc3779ec700 1 --2- 172.21.15.112:0/1256695866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3701f6dc0 0x7fc3700b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.835+0000 7fc3779ec700 1 -- 172.21.15.112:0/1256695866 >> 172.21.15.112:0/1256695866 conn(0x7fc3700b1380 msgr2=0x7fc370070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.835+0000 7fc3779ec700 1 -- 172.21.15.112:0/1256695866 shutdown_connections 2023-08-27T11:18:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.835+0000 7fc3779ec700 1 -- 172.21.15.112:0/1256695866 wait complete. 2023-08-27T11:18:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2023-08-27T11:18:26.848 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:26.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.967+0000 7f98c8571700 1 Processor -- start 2023-08-27T11:18:26.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.967+0000 7f98c8571700 1 -- start start 2023-08-27T11:18:26.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.968+0000 7f98c8571700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f98c015bf90 0x7f98c015f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.968+0000 7f98c8571700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c015eed0 con 0x7f98c00ce190 2023-08-27T11:18:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.968+0000 7f98c8571700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c015fe60 con 0x7f98c00cddb0 2023-08-27T11:18:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.968+0000 7f98c8571700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c015f500 con 0x7f98c015bf90 2023-08-27T11:18:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.969+0000 7f98c5b0c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f98c015bf90 0x7f98c015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.969+0000 7f98c6b0e700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f98c00ce190 0x7f98c00cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51764/0 (socket says 172.21.15.112:51764) 2023-08-27T11:18:26.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.969+0000 7f98c6b0e700 1 -- 172.21.15.112:0/1778346101 learned_addr learned my addr 172.21.15.112:0/1778346101 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:26.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.969+0000 7f98c5b0c700 1 -- 172.21.15.112:0/1778346101 >> v1:172.21.15.138:6789/0 conn(0x7f98c00cddb0 legacy=0x7f98c015b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:26.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.969+0000 7f98c5b0c700 1 -- 172.21.15.112:0/1778346101 >> v1:172.21.15.112:6789/0 conn(0x7f98c00ce190 legacy=0x7f98c00cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.969+0000 7f98c5b0c700 1 -- 172.21.15.112:0/1778346101 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c015c450 con 0x7f98c015bf90 2023-08-27T11:18:26.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.969+0000 7f98c530b700 1 -- 172.21.15.112:0/1778346101 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507598972 0 0) 0x7f98c015eed0 con 0x7f98c00ce190 2023-08-27T11:18:26.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.969+0000 7f98c5b0c700 1 --2- 172.21.15.112:0/1778346101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c015bf90 0x7f98c015f920 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009840 tx=0x7f98bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76166ce39c35a5a2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.969+0000 7f98c530b700 1 -- 172.21.15.112:0/1778346101 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc004230 con 0x7f98c015bf90 2023-08-27T11:18:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.969+0000 7f98c530b700 1 -- 172.21.15.112:0/1778346101 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc004390 con 0x7f98c015bf90 2023-08-27T11:18:26.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.969+0000 7f98c530b700 1 -- 172.21.15.112:0/1778346101 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0036a0 con 0x7f98c015bf90 2023-08-27T11:18:26.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.970+0000 7f98c8571700 1 -- 172.21.15.112:0/1778346101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c015bf90 msgr2=0x7f98c015f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.970+0000 7f98c8571700 1 --2- 172.21.15.112:0/1778346101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c015bf90 0x7f98c015f920 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009840 tx=0x7f98bc009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:26.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.970+0000 7f98c8571700 1 -- 172.21.15.112:0/1778346101 shutdown_connections 2023-08-27T11:18:26.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.970+0000 7f98c8571700 1 --2- 172.21.15.112:0/1778346101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c015bf90 0x7f98c015f920 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.970+0000 7f98c8571700 1 -- 172.21.15.112:0/1778346101 >> 172.21.15.112:0/1778346101 conn(0x7f98c006f970 msgr2=0x7f98c006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:26.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.970+0000 7f98c8571700 1 -- 172.21.15.112:0/1778346101 shutdown_connections 2023-08-27T11:18:26.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.970+0000 7f98c8571700 1 -- 172.21.15.112:0/1778346101 wait complete. 2023-08-27T11:18:26.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.971+0000 7f98c8571700 1 Processor -- start 2023-08-27T11:18:26.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.971+0000 7f98c8571700 1 -- start start 2023-08-27T11:18:26.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c8571700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c00cddb0 0x7f98c0145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c8571700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c00ce190 0x7f98c00ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c8571700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c015bf90 0x7f98c00b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c8571700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c00c6230 con 0x7f98c00ce190 2023-08-27T11:18:26.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c8571700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c00c64d0 con 0x7f98c015bf90 2023-08-27T11:18:26.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c8571700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c00c67b0 con 0x7f98c00cddb0 2023-08-27T11:18:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c5b0c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c00ce190 0x7f98c00ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c5b0c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c00ce190 0x7f98c00ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57386/0 (socket says 172.21.15.112:57386) 2023-08-27T11:18:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c5b0c700 1 -- 172.21.15.112:0/2237488114 learned_addr learned my addr 172.21.15.112:0/2237488114 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c630d700 1 --2- 172.21.15.112:0/2237488114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c00cddb0 0x7f98c0145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c5b0c700 1 -- 172.21.15.112:0/2237488114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c00cddb0 msgr2=0x7f98c0145e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c5b0c700 1 --2- 172.21.15.112:0/2237488114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c00cddb0 0x7f98c0145e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c5b0c700 1 -- 172.21.15.112:0/2237488114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c015bf90 msgr2=0x7f98c00b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c5b0c700 1 --2- 172.21.15.112:0/2237488114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c015bf90 0x7f98c00b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c5b0c700 1 -- 172.21.15.112:0/2237488114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc0094a0 con 0x7f98c00ce190 2023-08-27T11:18:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c5b0c700 1 --2- 172.21.15.112:0/2237488114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c00ce190 0x7f98c00ca2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0090b0 tx=0x7f98bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98b6ffd700 1 -- 172.21.15.112:0/2237488114 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc011830 con 0x7f98c00ce190 2023-08-27T11:18:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98b6ffd700 1 -- 172.21.15.112:0/2237488114 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc011990 con 0x7f98c00ce190 2023-08-27T11:18:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c8571700 1 -- 172.21.15.112:0/2237488114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c00c6a90 con 0x7f98c00ce190 2023-08-27T11:18:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98b6ffd700 1 -- 172.21.15.112:0/2237488114 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc011d40 con 0x7f98c00ce190 2023-08-27T11:18:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.972+0000 7f98c8571700 1 -- 172.21.15.112:0/2237488114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c0146390 con 0x7f98c00ce190 2023-08-27T11:18:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.973+0000 7f98c8571700 1 -- 172.21.15.112:0/2237488114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c0067a20 con 0x7f98c00ce190 2023-08-27T11:18:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.974+0000 7f98b6ffd700 1 -- 172.21.15.112:0/2237488114 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f98bc01b440 con 0x7f98c00ce190 2023-08-27T11:18:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.974+0000 7f98b6ffd700 1 --2- 172.21.15.112:0/2237488114 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f98a406a0e0 0x7f98a406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.974+0000 7f98b6ffd700 1 -- 172.21.15.112:0/2237488114 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f98bc089e80 con 0x7f98c00ce190 2023-08-27T11:18:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.975+0000 7f98c630d700 1 --2- 172.21.15.112:0/2237488114 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f98a406a0e0 0x7f98a406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.975+0000 7f98c630d700 1 --2- 172.21.15.112:0/2237488114 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f98a406a0e0 0x7f98a406c5a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f98b0001670 tx=0x7f98b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:26.979+0000 7f98b6ffd700 1 -- 172.21.15.112:0/2237488114 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f98bc0556c0 con 0x7f98c00ce190 2023-08-27T11:18:27.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.166+0000 7f98c8571700 1 -- 172.21.15.112:0/2237488114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98c00cf1c0 con 0x7f98c00ce190 2023-08-27T11:18:27.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.168+0000 7f98b6ffd700 1 -- 172.21.15.112:0/2237488114 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18448 (secure 0 0 0) 0x7f98bc058ce0 con 0x7f98c00ce190 2023-08-27T11:18:27.170 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:27.170 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":40,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:21.807644+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:01.935213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:02.602607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/1024768929":"2023-08-28T11:18:18.799945+0000","172.21.15.112:6833/1024768929":"2023-08-28T11:18:18.799945+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:27.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.170+0000 7f98c8571700 1 -- 172.21.15.112:0/2237488114 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f98a406a0e0 msgr2=0x7f98a406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.170+0000 7f98c8571700 1 --2- 172.21.15.112:0/2237488114 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f98a406a0e0 0x7f98a406c5a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f98b0001670 tx=0x7f98b0009040 comp rx=0 tx=0).stop 2023-08-27T11:18:27.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.170+0000 7f98c8571700 1 -- 172.21.15.112:0/2237488114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c00ce190 msgr2=0x7f98c00ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.170+0000 7f98c8571700 1 --2- 172.21.15.112:0/2237488114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c00ce190 0x7f98c00ca2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0090b0 tx=0x7f98bc0047c0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.170+0000 7f98c8571700 1 -- 172.21.15.112:0/2237488114 shutdown_connections 2023-08-27T11:18:27.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.170+0000 7f98c8571700 1 --2- 172.21.15.112:0/2237488114 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f98a406a0e0 0x7f98a406c5a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.170+0000 7f98c8571700 1 --2- 172.21.15.112:0/2237488114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c015bf90 0x7f98c00b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.170+0000 7f98c8571700 1 --2- 172.21.15.112:0/2237488114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c00cddb0 0x7f98c0145e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.170+0000 7f98c8571700 1 --2- 172.21.15.112:0/2237488114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c00ce190 0x7f98c00ca2f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.170+0000 7f98c8571700 1 -- 172.21.15.112:0/2237488114 >> 172.21.15.112:0/2237488114 conn(0x7f98c006f970 msgr2=0x7f98c00d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:27.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.170+0000 7f98c8571700 1 -- 172.21.15.112:0/2237488114 shutdown_connections 2023-08-27T11:18:27.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.170+0000 7f98c8571700 1 -- 172.21.15.112:0/2237488114 wait complete. 2023-08-27T11:18:27.183 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:18:01.935213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:18:02.602607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:18:27.184 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:18:27.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.309+0000 7ff431676700 1 Processor -- start 2023-08-27T11:18:27.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.309+0000 7ff431676700 1 -- start start 2023-08-27T11:18:27.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.309+0000 7ff431676700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff42c146bf0 0x7ff42c0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.309+0000 7ff431676700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff42c146fd0 0x7ff42c0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.309+0000 7ff431676700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff42c0d0290 0x7ff42c070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.309+0000 7ff431676700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff42c0d1500 con 0x7ff42c146bf0 2023-08-27T11:18:27.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.309+0000 7ff431676700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff42c0d0710 con 0x7ff42c146fd0 2023-08-27T11:18:27.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.309+0000 7ff431676700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0d09f0 con 0x7ff42c0d0290 2023-08-27T11:18:27.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff42affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff42c146bf0 0x7ff42c0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff42affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff42c146bf0 0x7ff42c0b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57392/0 (socket says 172.21.15.112:57392) 2023-08-27T11:18:27.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff42affd700 1 -- 172.21.15.112:0/2614733773 learned_addr learned my addr 172.21.15.112:0/2614733773 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:27.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff42a7fc700 1 --2- 172.21.15.112:0/2614733773 >> v2:172.21.15.138:3300/0 conn(0x7ff42c146fd0 0x7ff42c0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff42b7fe700 1 --2- 172.21.15.112:0/2614733773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff42c0d0290 0x7ff42c070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff42affd700 1 -- 172.21.15.112:0/2614733773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff42c0d0290 msgr2=0x7ff42c070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:27.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff42affd700 1 --2- 172.21.15.112:0/2614733773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff42c0d0290 0x7ff42c070eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff42affd700 1 -- 172.21.15.112:0/2614733773 >> v2:172.21.15.138:3300/0 conn(0x7ff42c146fd0 msgr2=0x7ff42c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:27.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff42affd700 1 --2- 172.21.15.112:0/2614733773 >> v2:172.21.15.138:3300/0 conn(0x7ff42c146fd0 0x7ff42c0cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff42affd700 1 -- 172.21.15.112:0/2614733773 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c0d0cd0 con 0x7ff42c146bf0 2023-08-27T11:18:27.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff42b7fe700 1 --2- 172.21.15.112:0/2614733773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff42c0d0290 0x7ff42c070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:27.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff42affd700 1 --2- 172.21.15.112:0/2614733773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c146bf0 0x7ff42c0b5db0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff42c071d70 tx=0x7ff41c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91aea04c78aff82 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:27.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff429ffb700 1 -- 172.21.15.112:0/2614733773 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00b870 con 0x7ff42c146bf0 2023-08-27T11:18:27.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff429ffb700 1 -- 172.21.15.112:0/2614733773 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41c00b9d0 con 0x7ff42c146bf0 2023-08-27T11:18:27.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.310+0000 7ff429ffb700 1 -- 172.21.15.112:0/2614733773 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00bd60 con 0x7ff42c146bf0 2023-08-27T11:18:27.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.311+0000 7ff431676700 1 -- 172.21.15.112:0/2614733773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c146bf0 msgr2=0x7ff42c0b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.311+0000 7ff431676700 1 --2- 172.21.15.112:0/2614733773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c146bf0 0x7ff42c0b5db0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff42c071d70 tx=0x7ff41c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.311+0000 7ff431676700 1 -- 172.21.15.112:0/2614733773 shutdown_connections 2023-08-27T11:18:27.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.311+0000 7ff431676700 1 --2- 172.21.15.112:0/2614733773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff42c0d0290 0x7ff42c070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.311+0000 7ff431676700 1 --2- 172.21.15.112:0/2614733773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c146bf0 0x7ff42c0b5db0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.311+0000 7ff431676700 1 --2- 172.21.15.112:0/2614733773 >> v2:172.21.15.138:3300/0 conn(0x7ff42c146fd0 0x7ff42c0cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.311+0000 7ff431676700 1 -- 172.21.15.112:0/2614733773 >> 172.21.15.112:0/2614733773 conn(0x7ff42c06c450 msgr2=0x7ff42c06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.311+0000 7ff431676700 1 -- 172.21.15.112:0/2614733773 shutdown_connections 2023-08-27T11:18:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.311+0000 7ff431676700 1 -- 172.21.15.112:0/2614733773 wait complete. 2023-08-27T11:18:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.312+0000 7ff431676700 1 Processor -- start 2023-08-27T11:18:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.312+0000 7ff431676700 1 -- start start 2023-08-27T11:18:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff431676700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff42c0d0290 0x7ff42c0b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff431676700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c146bf0 0x7ff42c0adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff431676700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff42c146fd0 0x7ff42c15ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff431676700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0af340 con 0x7ff42c146bf0 2023-08-27T11:18:27.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff431676700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0ae730 con 0x7ff42c0d0290 2023-08-27T11:18:27.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff431676700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0aea10 con 0x7ff42c146fd0 2023-08-27T11:18:27.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff42a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c146bf0 0x7ff42c0adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff42a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c146bf0 0x7ff42c0adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57398/0 (socket says 172.21.15.112:57398) 2023-08-27T11:18:27.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff42a7fc700 1 -- 172.21.15.112:0/3726064317 learned_addr learned my addr 172.21.15.112:0/3726064317 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:27.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff42b7fe700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff42c146fd0 0x7ff42c15ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff42affd700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff42c0d0290 0x7ff42c0b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff42a7fc700 1 -- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff42c146fd0 msgr2=0x7ff42c15ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff42a7fc700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff42c146fd0 0x7ff42c15ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff42a7fc700 1 -- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff42c0d0290 msgr2=0x7ff42c0b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff42a7fc700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff42c0d0290 0x7ff42c0b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff42a7fc700 1 -- 172.21.15.112:0/3726064317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c009550 con 0x7ff42c146bf0 2023-08-27T11:18:27.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.313+0000 7ff42a7fc700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c146bf0 0x7ff42c0adf00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff42000d800 tx=0x7ff42000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:27.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.314+0000 7ff4137fe700 1 -- 172.21.15.112:0/3726064317 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4200125f0 con 0x7ff42c146bf0 2023-08-27T11:18:27.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.314+0000 7ff4137fe700 1 -- 172.21.15.112:0/3726064317 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff420012750 con 0x7ff42c146bf0 2023-08-27T11:18:27.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.314+0000 7ff431676700 1 -- 172.21.15.112:0/3726064317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42c0aecf0 con 0x7ff42c146bf0 2023-08-27T11:18:27.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.314+0000 7ff4137fe700 1 -- 172.21.15.112:0/3726064317 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff420012b00 con 0x7ff42c146bf0 2023-08-27T11:18:27.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.314+0000 7ff431676700 1 -- 172.21.15.112:0/3726064317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42c0ae440 con 0x7ff42c146bf0 2023-08-27T11:18:27.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.315+0000 7ff431676700 1 -- 172.21.15.112:0/3726064317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42c0af4f0 con 0x7ff42c146bf0 2023-08-27T11:18:27.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.315+0000 7ff4137fe700 1 -- 172.21.15.112:0/3726064317 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff420012c60 con 0x7ff42c146bf0 2023-08-27T11:18:27.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.315+0000 7ff4137fe700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff41806a0e0 0x7ff41806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.315+0000 7ff4137fe700 1 -- 172.21.15.112:0/3726064317 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7ff420088610 con 0x7ff42c146bf0 2023-08-27T11:18:27.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.316+0000 7ff42affd700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff41806a0e0 0x7ff41806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.316+0000 7ff42affd700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff41806a0e0 0x7ff41806c5a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff41c000c00 tx=0x7ff41c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:27.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.320+0000 7ff4137fe700 1 -- 172.21.15.112:0/3726064317 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff42c0af4f0 con 0x7ff42c146bf0 2023-08-27T11:18:27.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.509+0000 7ff431676700 1 -- 172.21.15.112:0/3726064317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff42c067e90 con 0x7ff42c146bf0 2023-08-27T11:18:27.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.510+0000 7ff4137fe700 1 -- 172.21.15.112:0/3726064317 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff420053ca0 con 0x7ff42c146bf0 2023-08-27T11:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:18:27.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.511+0000 7ff431676700 1 -- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff41806a0e0 msgr2=0x7ff41806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.511+0000 7ff431676700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff41806a0e0 0x7ff41806c5a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff41c000c00 tx=0x7ff41c008030 comp rx=0 tx=0).stop 2023-08-27T11:18:27.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.511+0000 7ff431676700 1 -- 172.21.15.112:0/3726064317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c146bf0 msgr2=0x7ff42c0adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.511+0000 7ff431676700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c146bf0 0x7ff42c0adf00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff42000d800 tx=0x7ff42000db10 comp rx=0 tx=0).stop 2023-08-27T11:18:27.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.511+0000 7ff431676700 1 -- 172.21.15.112:0/3726064317 shutdown_connections 2023-08-27T11:18:27.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.511+0000 7ff431676700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff41806a0e0 0x7ff41806c5a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.511+0000 7ff431676700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff42c146fd0 0x7ff42c15ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.511+0000 7ff431676700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff42c0d0290 0x7ff42c0b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.511+0000 7ff431676700 1 --2- 172.21.15.112:0/3726064317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c146bf0 0x7ff42c0adf00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.511+0000 7ff431676700 1 -- 172.21.15.112:0/3726064317 >> 172.21.15.112:0/3726064317 conn(0x7ff42c06c450 msgr2=0x7ff42c15da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:27.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.511+0000 7ff431676700 1 -- 172.21.15.112:0/3726064317 shutdown_connections 2023-08-27T11:18:27.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.512+0000 7ff431676700 1 -- 172.21.15.112:0/3726064317 wait complete. 2023-08-27T11:18:27.525 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:18:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.647+0000 7fc7e2058700 1 Processor -- start 2023-08-27T11:18:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.647+0000 7fc7e2058700 1 -- start start 2023-08-27T11:18:27.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7e2058700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7dc146bf0 0x7fc7dc0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7e2058700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc7dc146fd0 0x7fc7dc0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7e2058700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7dc0d1500 con 0x7fc7dc0d0290 2023-08-27T11:18:27.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7e2058700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7dc0d0710 con 0x7fc7dc146fd0 2023-08-27T11:18:27.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7e2058700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc0d09f0 con 0x7fc7dc146bf0 2023-08-27T11:18:27.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7dbfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc7dc0d0290 0x7fc7dc070eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51792/0 (socket says 172.21.15.112:51792) 2023-08-27T11:18:27.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7dbfff700 1 -- 172.21.15.112:0/4115427108 learned_addr learned my addr 172.21.15.112:0/4115427108 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:27.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7db7fe700 1 --2- 172.21.15.112:0/4115427108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7dc146bf0 0x7fc7dc0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7daffd700 1 --2- 172.21.15.112:0/4115427108 >> v2:172.21.15.138:3300/0 conn(0x7fc7dc146fd0 0x7fc7dc0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7da7fc700 1 -- 172.21.15.112:0/4115427108 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367999241 0 0) 0x7fc7dc0d1500 con 0x7fc7dc0d0290 2023-08-27T11:18:27.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7da7fc700 1 -- 172.21.15.112:0/4115427108 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7c80035b0 con 0x7fc7dc0d0290 2023-08-27T11:18:27.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7da7fc700 1 -- 172.21.15.112:0/4115427108 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2884220907 0 0) 0x7fc7c80035b0 con 0x7fc7dc0d0290 2023-08-27T11:18:27.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7daffd700 1 -- 172.21.15.112:0/4115427108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7dc146bf0 msgr2=0x7fc7dc0b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:27.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7daffd700 1 --2- 172.21.15.112:0/4115427108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7dc146bf0 0x7fc7dc0b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7daffd700 1 -- 172.21.15.112:0/4115427108 >> v1:172.21.15.112:6789/0 conn(0x7fc7dc0d0290 legacy=0x7fc7dc070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.648+0000 7fc7daffd700 1 -- 172.21.15.112:0/4115427108 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc0d0cd0 con 0x7fc7dc146fd0 2023-08-27T11:18:27.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.649+0000 7fc7daffd700 1 --2- 172.21.15.112:0/4115427108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc146fd0 0x7fc7dc0cfd50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009840 tx=0x7fc7cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6e21a1c10bbaf78 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:27.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.649+0000 7fc7da7fc700 1 -- 172.21.15.112:0/4115427108 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40019f0 con 0x7fc7dc146fd0 2023-08-27T11:18:27.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.649+0000 7fc7da7fc700 1 -- 172.21.15.112:0/4115427108 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc004030 con 0x7fc7dc146fd0 2023-08-27T11:18:27.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.649+0000 7fc7da7fc700 1 -- 172.21.15.112:0/4115427108 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0043c0 con 0x7fc7dc146fd0 2023-08-27T11:18:27.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.649+0000 7fc7e2058700 1 -- 172.21.15.112:0/4115427108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc146fd0 msgr2=0x7fc7dc0cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.649+0000 7fc7e2058700 1 --2- 172.21.15.112:0/4115427108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc146fd0 0x7fc7dc0cfd50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009840 tx=0x7fc7cc009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:27.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.649+0000 7fc7e2058700 1 -- 172.21.15.112:0/4115427108 shutdown_connections 2023-08-27T11:18:27.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.649+0000 7fc7e2058700 1 --2- 172.21.15.112:0/4115427108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7dc146bf0 0x7fc7dc0b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.649+0000 7fc7e2058700 1 --2- 172.21.15.112:0/4115427108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc146fd0 0x7fc7dc0cfd50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.649+0000 7fc7e2058700 1 -- 172.21.15.112:0/4115427108 >> 172.21.15.112:0/4115427108 conn(0x7fc7dc06c450 msgr2=0x7fc7dc06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:27.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.649+0000 7fc7e2058700 1 -- 172.21.15.112:0/4115427108 shutdown_connections 2023-08-27T11:18:27.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.649+0000 7fc7e2058700 1 -- 172.21.15.112:0/4115427108 wait complete. 2023-08-27T11:18:27.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.651+0000 7fc7e2058700 1 Processor -- start 2023-08-27T11:18:27.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.651+0000 7fc7e2058700 1 -- start start 2023-08-27T11:18:27.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.651+0000 7fc7e2058700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc0d0290 0x7fc7dc13e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.651+0000 7fc7e2058700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7dc146bf0 0x7fc7dc1f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.651+0000 7fc7e2058700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7dc146fd0 0x7fc7dc06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.651+0000 7fc7e2058700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc0b7380 con 0x7fc7dc146bf0 2023-08-27T11:18:27.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.651+0000 7fc7e2058700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc1f7170 con 0x7fc7dc0d0290 2023-08-27T11:18:27.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7e2058700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc13f890 con 0x7fc7dc146fd0 2023-08-27T11:18:27.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7daffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7dc146bf0 0x7fc7dc1f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7daffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7dc146bf0 0x7fc7dc1f6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57412/0 (socket says 172.21.15.112:57412) 2023-08-27T11:18:27.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7daffd700 1 -- 172.21.15.112:0/3481664887 learned_addr learned my addr 172.21.15.112:0/3481664887 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:27.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7db7fe700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc0d0290 0x7fc7dc13e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7dbfff700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7dc146fd0 0x7fc7dc06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7daffd700 1 -- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7dc146fd0 msgr2=0x7fc7dc06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7daffd700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7dc146fd0 0x7fc7dc06ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7daffd700 1 -- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc0d0290 msgr2=0x7fc7dc13e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7daffd700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc0d0290 0x7fc7dc13e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7daffd700 1 -- 172.21.15.112:0/3481664887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc0094a0 con 0x7fc7dc146bf0 2023-08-27T11:18:27.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7db7fe700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc0d0290 0x7fc7dc13e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:27.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7dbfff700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7dc146fd0 0x7fc7dc06ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:27.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7daffd700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7dc146bf0 0x7fc7dc1f6c30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00a010 tx=0x7fc7cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:27.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7d3fff700 1 -- 172.21.15.112:0/3481664887 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc003c00 con 0x7fc7dc146bf0 2023-08-27T11:18:27.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7d3fff700 1 -- 172.21.15.112:0/3481664887 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc01f070 con 0x7fc7dc146bf0 2023-08-27T11:18:27.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7e2058700 1 -- 172.21.15.112:0/3481664887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7dc13ed10 con 0x7fc7dc146bf0 2023-08-27T11:18:27.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7d3fff700 1 -- 172.21.15.112:0/3481664887 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc011630 con 0x7fc7dc146bf0 2023-08-27T11:18:27.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.652+0000 7fc7e2058700 1 -- 172.21.15.112:0/3481664887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7dc13f2d0 con 0x7fc7dc146bf0 2023-08-27T11:18:27.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.653+0000 7fc7e2058700 1 -- 172.21.15.112:0/3481664887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7dc0b7510 con 0x7fc7dc146bf0 2023-08-27T11:18:27.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.654+0000 7fc7d3fff700 1 -- 172.21.15.112:0/3481664887 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc7cc021030 con 0x7fc7dc146bf0 2023-08-27T11:18:27.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.654+0000 7fc7d3fff700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7bc06a0e0 0x7fc7bc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.654+0000 7fc7d3fff700 1 -- 172.21.15.112:0/3481664887 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fc7cc089850 con 0x7fc7dc146bf0 2023-08-27T11:18:27.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.654+0000 7fc7db7fe700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7bc06a0e0 0x7fc7bc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.655+0000 7fc7db7fe700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7bc06a0e0 0x7fc7bc06c5a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009c30 tx=0x7fc7c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:27.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.658+0000 7fc7d3fff700 1 -- 172.21.15.112:0/3481664887 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc7dc0b7510 con 0x7fc7dc146bf0 2023-08-27T11:18:27.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.847+0000 7fc7e2058700 1 -- 172.21.15.112:0/3481664887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7dc067e90 con 0x7fc7dc146bf0 2023-08-27T11:18:27.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.848+0000 7fc7d3fff700 1 -- 172.21.15.112:0/3481664887 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7cc058b20 con 0x7fc7dc146bf0 2023-08-27T11:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:18:27.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.850+0000 7fc7e2058700 1 -- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7bc06a0e0 msgr2=0x7fc7bc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.850+0000 7fc7e2058700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7bc06a0e0 0x7fc7bc06c5a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009c30 tx=0x7fc7c4008040 comp rx=0 tx=0).stop 2023-08-27T11:18:27.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.850+0000 7fc7e2058700 1 -- 172.21.15.112:0/3481664887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7dc146bf0 msgr2=0x7fc7dc1f6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.850+0000 7fc7e2058700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7dc146bf0 0x7fc7dc1f6c30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00a010 tx=0x7fc7cc004750 comp rx=0 tx=0).stop 2023-08-27T11:18:27.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.850+0000 7fc7e2058700 1 -- 172.21.15.112:0/3481664887 shutdown_connections 2023-08-27T11:18:27.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.850+0000 7fc7e2058700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7bc06a0e0 0x7fc7bc06c5a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.850+0000 7fc7e2058700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7dc146fd0 0x7fc7dc06ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.850+0000 7fc7e2058700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc0d0290 0x7fc7dc13e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.850+0000 7fc7e2058700 1 --2- 172.21.15.112:0/3481664887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7dc146bf0 0x7fc7dc1f6c30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.850+0000 7fc7e2058700 1 -- 172.21.15.112:0/3481664887 >> 172.21.15.112:0/3481664887 conn(0x7fc7dc06c450 msgr2=0x7fc7dc0d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:27.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.850+0000 7fc7e2058700 1 -- 172.21.15.112:0/3481664887 shutdown_connections 2023-08-27T11:18:27.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.850+0000 7fc7e2058700 1 -- 172.21.15.112:0/3481664887 wait complete. 2023-08-27T11:18:27.863 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:18:27.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.988+0000 7ff392994700 1 Processor -- start 2023-08-27T11:18:27.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.988+0000 7ff392994700 1 -- start start 2023-08-27T11:18:27.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.988+0000 7ff392994700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff38c0cd980 0x7ff38c15b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff392994700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38c0718b0 con 0x7ff38c0cdd60 2023-08-27T11:18:27.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff392994700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38c0cc370 con 0x7ff38c13efd0 2023-08-27T11:18:27.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff392994700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38c0cc160 con 0x7ff38c0cd980 2023-08-27T11:18:27.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff38c0cdd60 0x7ff38c06ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51804/0 (socket says 172.21.15.112:51804) 2023-08-27T11:18:27.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38b7fe700 1 -- 172.21.15.112:0/1316158901 learned_addr learned my addr 172.21.15.112:0/1316158901 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:27.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129113046 0 0) 0x7ff38c0718b0 con 0x7ff38c0cdd60 2023-08-27T11:18:27.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38bfff700 1 --2- 172.21.15.112:0/1316158901 >> v2:172.21.15.138:3300/0 conn(0x7ff38c0cd980 0x7ff38c15b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3740035b0 con 0x7ff38c0cdd60 2023-08-27T11:18:27.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2858496273 0 0) 0x7ff3740035b0 con 0x7ff38c0cdd60 2023-08-27T11:18:27.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 >> v2:172.21.15.138:3300/0 conn(0x7ff38c0cd980 msgr2=0x7ff38c15b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:27.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 --2- 172.21.15.112:0/1316158901 >> v2:172.21.15.138:3300/0 conn(0x7ff38c0cd980 0x7ff38c15b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 >> v1:172.21.15.138:6789/0 conn(0x7ff38c13efd0 legacy=0x7ff38c13f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c0cc650 con 0x7ff38c0cdd60 2023-08-27T11:18:27.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7ff380001670 con 0x7ff38c0cdd60 2023-08-27T11:18:27.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38bfff700 1 --2- 172.21.15.112:0/1316158901 >> v2:172.21.15.138:3300/0 conn(0x7ff38c0cd980 0x7ff38c15b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:27.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 >> v1:172.21.15.112:6789/0 conn(0x7ff38c0cdd60 legacy=0x7ff38c06ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:27.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 --2- 172.21.15.112:0/1316158901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff374003a00 0x7ff374003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 --2- 172.21.15.112:0/1316158901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff374004850 0x7ff374004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 --2- 172.21.15.112:0/1316158901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff374005210 0x7ff37400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:27.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0cc160 con 0x7ff374005210 2023-08-27T11:18:27.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37400c140 con 0x7ff374003a00 2023-08-27T11:18:27.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37400c300 con 0x7ff374004850 2023-08-27T11:18:27.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.989+0000 7ff390f31700 1 --2- 172.21.15.112:0/1316158901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff374005210 0x7ff37400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff38bfff700 1 --2- 172.21.15.112:0/1316158901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff374003a00 0x7ff374003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff38b7fe700 1 --2- 172.21.15.112:0/1316158901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff374004850 0x7ff374004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:27.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff390f31700 1 -- 172.21.15.112:0/1316158901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff374004850 msgr2=0x7ff374004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:27.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff390f31700 1 --2- 172.21.15.112:0/1316158901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff374004850 0x7ff374004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff390f31700 1 -- 172.21.15.112:0/1316158901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff374003a00 msgr2=0x7ff374003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:27.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff390f31700 1 --2- 172.21.15.112:0/1316158901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff374003a00 0x7ff374003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:27.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff390f31700 1 -- 172.21.15.112:0/1316158901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3740035b0 con 0x7ff374005210 2023-08-27T11:18:28.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff390f31700 1 -- 172.21.15.112:0/1316158901 reap_dead start 2023-08-27T11:18:28.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff390f31700 1 --2- 172.21.15.112:0/1316158901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff374005210 0x7ff37400ba10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff38400c190 tx=0x7ff374005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6050dc093833939 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:28.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff384001db0 con 0x7ff374005210 2023-08-27T11:18:28.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38400ccd0 con 0x7ff374005210 2023-08-27T11:18:28.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff38affd700 1 -- 172.21.15.112:0/1316158901 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff384003c40 con 0x7ff374005210 2023-08-27T11:18:28.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff392994700 1 -- 172.21.15.112:0/1316158901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff374005210 msgr2=0x7ff37400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff392994700 1 --2- 172.21.15.112:0/1316158901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff374005210 0x7ff37400ba10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff38400c190 tx=0x7ff374005880 comp rx=0 tx=0).stop 2023-08-27T11:18:28.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff392994700 1 -- 172.21.15.112:0/1316158901 shutdown_connections 2023-08-27T11:18:28.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff392994700 1 --2- 172.21.15.112:0/1316158901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff374005210 0x7ff37400ba10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.990+0000 7ff392994700 1 -- 172.21.15.112:0/1316158901 >> 172.21.15.112:0/1316158901 conn(0x7ff38c0cf7b0 msgr2=0x7ff38c0cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:28.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.991+0000 7ff392994700 1 -- 172.21.15.112:0/1316158901 shutdown_connections 2023-08-27T11:18:28.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.991+0000 7ff392994700 1 -- 172.21.15.112:0/1316158901 wait complete. 2023-08-27T11:18:28.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.992+0000 7ff392994700 1 Processor -- start 2023-08-27T11:18:28.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.992+0000 7ff392994700 1 -- start start 2023-08-27T11:18:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff392994700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff374005210 0x7ff38c1f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff392994700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff38c15ce10 0x7ff38c140780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff392994700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff38c1f00a0 0x7ff38c06cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff392994700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c1efc10 con 0x7ff38c1f00a0 2023-08-27T11:18:28.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff392994700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c1f3680 con 0x7ff38c15ce10 2023-08-27T11:18:28.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff392994700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c1f3960 con 0x7ff374005210 2023-08-27T11:18:28.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff390f31700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff38c1f00a0 0x7ff38c06cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff390f31700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff38c1f00a0 0x7ff38c06cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57436/0 (socket says 172.21.15.112:57436) 2023-08-27T11:18:28.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff390f31700 1 -- 172.21.15.112:0/248863307 learned_addr learned my addr 172.21.15.112:0/248863307 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:28.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff38bfff700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff374005210 0x7ff38c1f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff38b7fe700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff38c15ce10 0x7ff38c140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff390f31700 1 -- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff374005210 msgr2=0x7ff38c1f2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff390f31700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff374005210 0x7ff38c1f2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff390f31700 1 -- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff38c15ce10 msgr2=0x7ff38c140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff390f31700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff38c15ce10 0x7ff38c140780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff390f31700 1 -- 172.21.15.112:0/248863307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c1f3c40 con 0x7ff38c1f00a0 2023-08-27T11:18:28.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff390f31700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff38c1f00a0 0x7ff38c06cda0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff38c072740 tx=0x7ff3840014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:28.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff388ff9700 1 -- 172.21.15.112:0/248863307 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff384003db0 con 0x7ff38c1f00a0 2023-08-27T11:18:28.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff388ff9700 1 -- 172.21.15.112:0/248863307 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff384005070 con 0x7ff38c1f00a0 2023-08-27T11:18:28.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff388ff9700 1 -- 172.21.15.112:0/248863307 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff384005420 con 0x7ff38c1f00a0 2023-08-27T11:18:28.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff392994700 1 -- 172.21.15.112:0/248863307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38c1ef8a0 con 0x7ff38c1f00a0 2023-08-27T11:18:28.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.993+0000 7ff392994700 1 -- 172.21.15.112:0/248863307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38c140cc0 con 0x7ff38c1f00a0 2023-08-27T11:18:28.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.994+0000 7ff392994700 1 -- 172.21.15.112:0/248863307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38c067e90 con 0x7ff38c1f00a0 2023-08-27T11:18:28.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.995+0000 7ff388ff9700 1 -- 172.21.15.112:0/248863307 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff384005580 con 0x7ff38c1f00a0 2023-08-27T11:18:28.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.995+0000 7ff388ff9700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff37806a0e0 0x7ff37806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.995+0000 7ff388ff9700 1 -- 172.21.15.112:0/248863307 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7ff384088f70 con 0x7ff38c1f00a0 2023-08-27T11:18:28.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.995+0000 7ff38bfff700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff37806a0e0 0x7ff37806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:27.996+0000 7ff38bfff700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff37806a0e0 0x7ff37806c5a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff37c004520 tx=0x7ff37c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:28.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.000+0000 7ff388ff9700 1 -- 172.21.15.112:0/248863307 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff384054690 con 0x7ff38c1f00a0 2023-08-27T11:18:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.186+0000 7ff392994700 1 -- 172.21.15.112:0/248863307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff38c067a20 con 0x7ff38c1f00a0 2023-08-27T11:18:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.186+0000 7ff388ff9700 1 -- 172.21.15.112:0/248863307 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff384057cb0 con 0x7ff38c1f00a0 2023-08-27T11:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:18:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.188+0000 7ff392994700 1 -- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff37806a0e0 msgr2=0x7ff37806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.188+0000 7ff392994700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff37806a0e0 0x7ff37806c5a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff37c004520 tx=0x7ff37c008040 comp rx=0 tx=0).stop 2023-08-27T11:18:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.188+0000 7ff392994700 1 -- 172.21.15.112:0/248863307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff38c1f00a0 msgr2=0x7ff38c06cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.188+0000 7ff392994700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff38c1f00a0 0x7ff38c06cda0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff38c072740 tx=0x7ff3840014a0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.188+0000 7ff392994700 1 -- 172.21.15.112:0/248863307 shutdown_connections 2023-08-27T11:18:28.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.188+0000 7ff392994700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff37806a0e0 0x7ff37806c5a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.188+0000 7ff392994700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff38c1f00a0 0x7ff38c06cda0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.188+0000 7ff392994700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff374005210 0x7ff38c1f2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.188+0000 7ff392994700 1 --2- 172.21.15.112:0/248863307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff38c15ce10 0x7ff38c140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.188+0000 7ff392994700 1 -- 172.21.15.112:0/248863307 >> 172.21.15.112:0/248863307 conn(0x7ff38c0cf7b0 msgr2=0x7ff38c0af310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:28.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.188+0000 7ff392994700 1 -- 172.21.15.112:0/248863307 shutdown_connections 2023-08-27T11:18:28.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.188+0000 7ff392994700 1 -- 172.21.15.112:0/248863307 wait complete. 2023-08-27T11:18:28.201 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:18:28.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.326+0000 7f8341205700 1 Processor -- start 2023-08-27T11:18:28.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.326+0000 7f8341205700 1 -- start start 2023-08-27T11:18:28.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.327+0000 7f8341205700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f833c146b60 0x7f833c06ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.327+0000 7f8341205700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c160fa0 con 0x7f833c06e2f0 2023-08-27T11:18:28.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.327+0000 7f8341205700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c06e750 con 0x7f833c0ad1d0 2023-08-27T11:18:28.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.327+0000 7f8341205700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833c161b70 con 0x7f833c146b60 2023-08-27T11:18:28.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.327+0000 7f833a59c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f833c06e2f0 0x7f833c0acac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51824/0 (socket says 172.21.15.112:51824) 2023-08-27T11:18:28.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.327+0000 7f833a59c700 1 -- 172.21.15.112:0/3561333426 learned_addr learned my addr 172.21.15.112:0/3561333426 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:28.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.327+0000 7f833ad9d700 1 --2- 172.21.15.112:0/3561333426 >> v2:172.21.15.138:3300/0 conn(0x7f833c146b60 0x7f833c06ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.327+0000 7f8339d9b700 1 -- 172.21.15.112:0/3561333426 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 222623018 0 0) 0x7f833c160fa0 con 0x7f833c06e2f0 2023-08-27T11:18:28.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.327+0000 7f8339d9b700 1 -- 172.21.15.112:0/3561333426 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83240035b0 con 0x7f833c06e2f0 2023-08-27T11:18:28.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.328+0000 7f833ad9d700 1 -- 172.21.15.112:0/3561333426 >> v1:172.21.15.138:6789/0 conn(0x7f833c0ad1d0 legacy=0x7f833c15d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.328+0000 7f8339d9b700 1 -- 172.21.15.112:0/3561333426 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368260538 0 0) 0x7f833c06e750 con 0x7f833c0ad1d0 2023-08-27T11:18:28.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.328+0000 7f833ad9d700 1 -- 172.21.15.112:0/3561333426 >> v1:172.21.15.112:6789/0 conn(0x7f833c06e2f0 legacy=0x7f833c0acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.328+0000 7f833ad9d700 1 -- 172.21.15.112:0/3561333426 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c146f40 con 0x7f833c146b60 2023-08-27T11:18:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.328+0000 7f833ad9d700 1 --2- 172.21.15.112:0/3561333426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f833c146b60 0x7f833c06ddb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8330009840 tx=0x7f8330009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce5d3b02dc28fc09 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.328+0000 7f8339d9b700 1 -- 172.21.15.112:0/3561333426 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330004230 con 0x7f833c146b60 2023-08-27T11:18:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.328+0000 7f8339d9b700 1 -- 172.21.15.112:0/3561333426 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8330004390 con 0x7f833c146b60 2023-08-27T11:18:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.328+0000 7f8339d9b700 1 -- 172.21.15.112:0/3561333426 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83300036a0 con 0x7f833c146b60 2023-08-27T11:18:28.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.329+0000 7f8341205700 1 -- 172.21.15.112:0/3561333426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f833c146b60 msgr2=0x7f833c06ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.329+0000 7f8341205700 1 --2- 172.21.15.112:0/3561333426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f833c146b60 0x7f833c06ddb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8330009840 tx=0x7f8330009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:28.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.329+0000 7f8341205700 1 -- 172.21.15.112:0/3561333426 shutdown_connections 2023-08-27T11:18:28.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.329+0000 7f8341205700 1 --2- 172.21.15.112:0/3561333426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f833c146b60 0x7f833c06ddb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.329+0000 7f8341205700 1 -- 172.21.15.112:0/3561333426 >> 172.21.15.112:0/3561333426 conn(0x7f833c0cc240 msgr2=0x7f833c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:28.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.329+0000 7f8341205700 1 -- 172.21.15.112:0/3561333426 shutdown_connections 2023-08-27T11:18:28.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.329+0000 7f8341205700 1 -- 172.21.15.112:0/3561333426 wait complete. 2023-08-27T11:18:28.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.330+0000 7f8341205700 1 Processor -- start 2023-08-27T11:18:28.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.330+0000 7f8341205700 1 -- start start 2023-08-27T11:18:28.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f8341205700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f833c06e2f0 0x7f833c1f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f8341205700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f833c0ad1d0 0x7f833c1f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f8341205700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f833c146b60 0x7f833c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f8341205700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c1f2a90 con 0x7f833c146b60 2023-08-27T11:18:28.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f8341205700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c13f190 con 0x7f833c06e2f0 2023-08-27T11:18:28.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f8341205700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833c13f430 con 0x7f833c0ad1d0 2023-08-27T11:18:28.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f833b59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f833c146b60 0x7f833c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f833b59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f833c146b60 0x7f833c06f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57450/0 (socket says 172.21.15.112:57450) 2023-08-27T11:18:28.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f833b59e700 1 -- 172.21.15.112:0/516723869 learned_addr learned my addr 172.21.15.112:0/516723869 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:28.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f833ad9d700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f833c06e2f0 0x7f833c1f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f833a59c700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f833c0ad1d0 0x7f833c1f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f833b59e700 1 -- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f833c0ad1d0 msgr2=0x7f833c1f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f833b59e700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f833c0ad1d0 0x7f833c1f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f833b59e700 1 -- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f833c06e2f0 msgr2=0x7f833c1f2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f833b59e700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f833c06e2f0 0x7f833c1f2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f833b59e700 1 -- 172.21.15.112:0/516723869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83300094a0 con 0x7f833c146b60 2023-08-27T11:18:28.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f833a59c700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f833c0ad1d0 0x7f833c1f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:28.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.331+0000 7f833b59e700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f833c146b60 0x7f833c06f790 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f833400c810 tx=0x7f833400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:28.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.332+0000 7f83237fe700 1 -- 172.21.15.112:0/516723869 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334003e30 con 0x7f833c146b60 2023-08-27T11:18:28.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.332+0000 7f83237fe700 1 -- 172.21.15.112:0/516723869 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334005070 con 0x7f833c146b60 2023-08-27T11:18:28.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.332+0000 7f8341205700 1 -- 172.21.15.112:0/516723869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833c13f710 con 0x7f833c146b60 2023-08-27T11:18:28.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.332+0000 7f83237fe700 1 -- 172.21.15.112:0/516723869 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334005420 con 0x7f833c146b60 2023-08-27T11:18:28.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.332+0000 7f8341205700 1 -- 172.21.15.112:0/516723869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833c070210 con 0x7f833c146b60 2023-08-27T11:18:28.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.333+0000 7f8341205700 1 -- 172.21.15.112:0/516723869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833c067e90 con 0x7f833c146b60 2023-08-27T11:18:28.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.333+0000 7f83237fe700 1 -- 172.21.15.112:0/516723869 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8334005580 con 0x7f833c146b60 2023-08-27T11:18:28.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.333+0000 7f83237fe700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f832806a0e0 0x7f832806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.334+0000 7f83237fe700 1 -- 172.21.15.112:0/516723869 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f83340882e0 con 0x7f833c146b60 2023-08-27T11:18:28.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.334+0000 7f833ad9d700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f832806a0e0 0x7f832806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.334+0000 7f833ad9d700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f832806a0e0 0x7f832806c5a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f833000a010 tx=0x7f8330013040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:28.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.338+0000 7f83237fe700 1 -- 172.21.15.112:0/516723869 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8334053a00 con 0x7f833c146b60 2023-08-27T11:18:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.526+0000 7f8341205700 1 -- 172.21.15.112:0/516723869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f833c067a20 con 0x7f833c146b60 2023-08-27T11:18:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.526+0000 7f83237fe700 1 -- 172.21.15.112:0/516723869 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8334057020 con 0x7f833c146b60 2023-08-27T11:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:18:28.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.528+0000 7f8341205700 1 -- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f832806a0e0 msgr2=0x7f832806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.528+0000 7f8341205700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f832806a0e0 0x7f832806c5a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f833000a010 tx=0x7f8330013040 comp rx=0 tx=0).stop 2023-08-27T11:18:28.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.528+0000 7f8341205700 1 -- 172.21.15.112:0/516723869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f833c146b60 msgr2=0x7f833c06f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.528+0000 7f8341205700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f833c146b60 0x7f833c06f790 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f833400c810 tx=0x7f833400cbd0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.528+0000 7f8341205700 1 -- 172.21.15.112:0/516723869 shutdown_connections 2023-08-27T11:18:28.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.528+0000 7f8341205700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f832806a0e0 0x7f832806c5a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.528+0000 7f8341205700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f833c146b60 0x7f833c06f790 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.528+0000 7f8341205700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f833c06e2f0 0x7f833c1f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.528+0000 7f8341205700 1 --2- 172.21.15.112:0/516723869 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f833c0ad1d0 0x7f833c1f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.528+0000 7f8341205700 1 -- 172.21.15.112:0/516723869 >> 172.21.15.112:0/516723869 conn(0x7f833c0cc240 msgr2=0x7f833c142bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:28.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.528+0000 7f8341205700 1 -- 172.21.15.112:0/516723869 shutdown_connections 2023-08-27T11:18:28.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.528+0000 7f8341205700 1 -- 172.21.15.112:0/516723869 wait complete. 2023-08-27T11:18:28.542 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:18:28.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.661+0000 7ff79161c700 1 Processor -- start 2023-08-27T11:18:28.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.661+0000 7ff79161c700 1 -- start start 2023-08-27T11:18:28.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.662+0000 7ff79161c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c160a00 0x7ff78c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.662+0000 7ff79161c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff78c06d510 0x7ff78c06d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.662+0000 7ff79161c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c06f930 con 0x7ff78c146990 2023-08-27T11:18:28.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.662+0000 7ff79161c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78c06dec0 con 0x7ff78c06d510 2023-08-27T11:18:28.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.662+0000 7ff79161c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c06fc10 con 0x7ff78c160a00 2023-08-27T11:18:28.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.662+0000 7ff78a7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff78c146990 0x7ff78c15dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51836/0 (socket says 172.21.15.112:51836) 2023-08-27T11:18:28.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.662+0000 7ff78a7fc700 1 -- 172.21.15.112:0/708974021 learned_addr learned my addr 172.21.15.112:0/708974021 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:28.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.662+0000 7ff78affd700 1 --2- 172.21.15.112:0/708974021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c160a00 0x7ff78c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.662+0000 7ff78b7fe700 1 --2- 172.21.15.112:0/708974021 >> v2:172.21.15.138:3300/0 conn(0x7ff78c06d510 0x7ff78c06d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.662+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940215607 0 0) 0x7ff78c06f930 con 0x7ff78c146990 2023-08-27T11:18:28.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.662+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7740035b0 con 0x7ff78c146990 2023-08-27T11:18:28.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4019218156 0 0) 0x7ff7740035b0 con 0x7ff78c146990 2023-08-27T11:18:28.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c160a00 msgr2=0x7ff78c0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:28.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 --2- 172.21.15.112:0/708974021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c160a00 0x7ff78c0b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 >> v2:172.21.15.138:3300/0 conn(0x7ff78c06d510 msgr2=0x7ff78c06d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:28.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 --2- 172.21.15.112:0/708974021 >> v2:172.21.15.138:3300/0 conn(0x7ff78c06d510 0x7ff78c06d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c06fef0 con 0x7ff78c146990 2023-08-27T11:18:28.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7ff77c001670 con 0x7ff78c146990 2023-08-27T11:18:28.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff78b7fe700 1 --2- 172.21.15.112:0/708974021 >> v2:172.21.15.138:3300/0 conn(0x7ff78c06d510 0x7ff78c06d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:28.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 >> v1:172.21.15.112:6789/0 conn(0x7ff78c146990 legacy=0x7ff78c15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 --2- 172.21.15.112:0/708974021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff774003a00 0x7ff774003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 --2- 172.21.15.112:0/708974021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff774004850 0x7ff774004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 --2- 172.21.15.112:0/708974021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c160a00 0x7ff774005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c06dec0 con 0x7ff774003a00 2023-08-27T11:18:28.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c06fc10 con 0x7ff774004850 2023-08-27T11:18:28.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77400ba10 con 0x7ff78c160a00 2023-08-27T11:18:28.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff78affd700 1 --2- 172.21.15.112:0/708974021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff774003a00 0x7ff774003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff78a7fc700 1 --2- 172.21.15.112:0/708974021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff774004850 0x7ff774004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff78b7fe700 1 --2- 172.21.15.112:0/708974021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c160a00 0x7ff774005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff78affd700 1 -- 172.21.15.112:0/708974021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c160a00 msgr2=0x7ff774005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:28.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff78affd700 1 --2- 172.21.15.112:0/708974021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c160a00 0x7ff774005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff78affd700 1 -- 172.21.15.112:0/708974021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff774004850 msgr2=0x7ff774004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:28.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff78affd700 1 --2- 172.21.15.112:0/708974021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff774004850 0x7ff774004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff78affd700 1 -- 172.21.15.112:0/708974021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7740035b0 con 0x7ff774003a00 2023-08-27T11:18:28.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff78b7fe700 1 --2- 172.21.15.112:0/708974021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c160a00 0x7ff774005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:28.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff78affd700 1 --2- 172.21.15.112:0/708974021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff774003a00 0x7ff774003e20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff780009b60 tx=0x7ff774005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b22cf86c8438684 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000e6c0 con 0x7ff774003a00 2023-08-27T11:18:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78000e820 con 0x7ff774003a00 2023-08-27T11:18:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.663+0000 7ff789ffb700 1 -- 172.21.15.112:0/708974021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000eb90 con 0x7ff774003a00 2023-08-27T11:18:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.664+0000 7ff79161c700 1 -- 172.21.15.112:0/708974021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff774003a00 msgr2=0x7ff774003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.664+0000 7ff79161c700 1 --2- 172.21.15.112:0/708974021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff774003a00 0x7ff774003e20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff780009b60 tx=0x7ff774005880 comp rx=0 tx=0).stop 2023-08-27T11:18:28.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.664+0000 7ff78b7fe700 1 -- 172.21.15.112:0/708974021 reap_dead start 2023-08-27T11:18:28.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.664+0000 7ff79161c700 1 -- 172.21.15.112:0/708974021 shutdown_connections 2023-08-27T11:18:28.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.664+0000 7ff79161c700 1 -- 172.21.15.112:0/708974021 >> 172.21.15.112:0/708974021 conn(0x7ff78c0cfe50 msgr2=0x7ff78c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:28.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.664+0000 7ff79161c700 1 -- 172.21.15.112:0/708974021 shutdown_connections 2023-08-27T11:18:28.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.664+0000 7ff79161c700 1 -- 172.21.15.112:0/708974021 wait complete. 2023-08-27T11:18:28.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.666+0000 7ff79161c700 1 Processor -- start 2023-08-27T11:18:28.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.666+0000 7ff79161c700 1 -- start start 2023-08-27T11:18:28.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff79161c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff774003a00 0x7ff78c0b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff79161c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c0b84a0 0x7ff78c13d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff79161c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78c13dc60 0x7ff78c13e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff79161c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c1508a0 con 0x7ff774003a00 2023-08-27T11:18:28.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff79161c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0b8c90 con 0x7ff78c13dc60 2023-08-27T11:18:28.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff79161c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0b8f70 con 0x7ff78c0b84a0 2023-08-27T11:18:28.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff78affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff774003a00 0x7ff78c0b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff78affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff774003a00 0x7ff78c0b7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57466/0 (socket says 172.21.15.112:57466) 2023-08-27T11:18:28.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff78affd700 1 -- 172.21.15.112:0/2216163016 learned_addr learned my addr 172.21.15.112:0/2216163016 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:28.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff78affd700 1 -- 172.21.15.112:0/2216163016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c0b84a0 msgr2=0x7ff78c13d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff78affd700 1 --2- 172.21.15.112:0/2216163016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c0b84a0 0x7ff78c13d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff78affd700 1 -- 172.21.15.112:0/2216163016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78c13dc60 msgr2=0x7ff78c13e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff78affd700 1 --2- 172.21.15.112:0/2216163016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78c13dc60 0x7ff78c13e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff78affd700 1 -- 172.21.15.112:0/2216163016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c150b10 con 0x7ff774003a00 2023-08-27T11:18:28.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff78affd700 1 --2- 172.21.15.112:0/2216163016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff774003a00 0x7ff78c0b7f60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff7740046a0 tx=0x7ff7800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:28.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff7737fe700 1 -- 172.21.15.112:0/2216163016 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780021070 con 0x7ff774003a00 2023-08-27T11:18:28.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff7737fe700 1 -- 172.21.15.112:0/2216163016 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff780004550 con 0x7ff774003a00 2023-08-27T11:18:28.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff79161c700 1 -- 172.21.15.112:0/2216163016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c0b9250 con 0x7ff774003a00 2023-08-27T11:18:28.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff7737fe700 1 -- 172.21.15.112:0/2216163016 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000e6c0 con 0x7ff774003a00 2023-08-27T11:18:28.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.667+0000 7ff79161c700 1 -- 172.21.15.112:0/2216163016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c15d2f0 con 0x7ff774003a00 2023-08-27T11:18:28.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.668+0000 7ff79161c700 1 -- 172.21.15.112:0/2216163016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c150e80 con 0x7ff774003a00 2023-08-27T11:18:28.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.669+0000 7ff7737fe700 1 -- 172.21.15.112:0/2216163016 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff780025070 con 0x7ff774003a00 2023-08-27T11:18:28.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.669+0000 7ff7737fe700 1 --2- 172.21.15.112:0/2216163016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff77806a0e0 0x7ff77806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:28.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.669+0000 7ff7737fe700 1 -- 172.21.15.112:0/2216163016 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7ff7800896f0 con 0x7ff774003a00 2023-08-27T11:18:28.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.670+0000 7ff78a7fc700 1 --2- 172.21.15.112:0/2216163016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff77806a0e0 0x7ff77806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:28.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.670+0000 7ff78a7fc700 1 --2- 172.21.15.112:0/2216163016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff77806a0e0 0x7ff77806c5a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0d3260 tx=0x7ff77c000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:28.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.674+0000 7ff7737fe700 1 -- 172.21.15.112:0/2216163016 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff780054e90 con 0x7ff774003a00 2023-08-27T11:18:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.905+0000 7ff79161c700 1 -- 172.21.15.112:0/2216163016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff78c150ca0 con 0x7ff774003a00 2023-08-27T11:18:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.906+0000 7ff7737fe700 1 -- 172.21.15.112:0/2216163016 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4887 (secure 0 0 0) 0x7ff7800584b0 con 0x7ff774003a00 2023-08-27T11:18:28.908 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:28.908 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1941625698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1941625698},{"type":"v1","addr":"172.21.15.138:6836","nonce":1941625698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4904,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1317948937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1317948937},{"type":"v1","addr":"172.21.15.112:6835","nonce":1317948937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1530886230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1530886230},{"type":"v1","addr":"172.21.15.112:6834","nonce":1530886230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:18:03.637863+0000","modified":"2023-08-27T11:18:07.262281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5175":{"gid":5175,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6837/3409848660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3409848660},{"type":"v1","addr":"172.21.15.138:6837","nonce":3409848660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-27T11:18:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.908+0000 7ff79161c700 1 -- 172.21.15.112:0/2216163016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff77806a0e0 msgr2=0x7ff77806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.908+0000 7ff79161c700 1 --2- 172.21.15.112:0/2216163016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff77806a0e0 0x7ff77806c5a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0d3260 tx=0x7ff77c000bc0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.908+0000 7ff79161c700 1 -- 172.21.15.112:0/2216163016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff774003a00 msgr2=0x7ff78c0b7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.908+0000 7ff79161c700 1 --2- 172.21.15.112:0/2216163016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff774003a00 0x7ff78c0b7f60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff7740046a0 tx=0x7ff7800047c0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.908+0000 7ff79161c700 1 -- 172.21.15.112:0/2216163016 shutdown_connections 2023-08-27T11:18:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.908+0000 7ff79161c700 1 --2- 172.21.15.112:0/2216163016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff77806a0e0 0x7ff77806c5a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.908+0000 7ff79161c700 1 --2- 172.21.15.112:0/2216163016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78c13dc60 0x7ff78c13e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.908+0000 7ff79161c700 1 --2- 172.21.15.112:0/2216163016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff774003a00 0x7ff78c0b7f60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.908+0000 7ff79161c700 1 --2- 172.21.15.112:0/2216163016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff78c0b84a0 0x7ff78c13d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.908+0000 7ff79161c700 1 -- 172.21.15.112:0/2216163016 >> 172.21.15.112:0/2216163016 conn(0x7ff78c0cfe50 msgr2=0x7ff78c15e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.908+0000 7ff79161c700 1 -- 172.21.15.112:0/2216163016 shutdown_connections 2023-08-27T11:18:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:28.908+0000 7ff79161c700 1 -- 172.21.15.112:0/2216163016 wait complete. 2023-08-27T11:18:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2023-08-27T11:18:28.923 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:29.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.041+0000 7fd1c0777700 1 Processor -- start 2023-08-27T11:18:29.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.041+0000 7fd1c0777700 1 -- start start 2023-08-27T11:18:29.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.042+0000 7fd1c0777700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b80b05f0 0x7fd1b80b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.042+0000 7fd1c0777700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b80b0a60 con 0x7fd1b806dcc0 2023-08-27T11:18:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.042+0000 7fd1c0777700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b815b540 con 0x7fd1b815a120 2023-08-27T11:18:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.042+0000 7fd1c0777700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b80af830 con 0x7fd1b80b05f0 2023-08-27T11:18:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.042+0000 7fd1bdd12700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd1b806dcc0 0x7fd1b80b3440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51860/0 (socket says 172.21.15.112:51860) 2023-08-27T11:18:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.042+0000 7fd1bdd12700 1 -- 172.21.15.112:0/617976161 learned_addr learned my addr 172.21.15.112:0/617976161 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:29.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.042+0000 7fd1bed14700 1 --2- 172.21.15.112:0/617976161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b80b05f0 0x7fd1b80b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252592111 0 0) 0x7fd1b80b0a60 con 0x7fd1b806dcc0 2023-08-27T11:18:29.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1a40035b0 con 0x7fd1b806dcc0 2023-08-27T11:18:29.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110771855 0 0) 0x7fd1b815b540 con 0x7fd1b815a120 2023-08-27T11:18:29.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1b80b0a60 con 0x7fd1b815a120 2023-08-27T11:18:29.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2261192434 0 0) 0x7fd1a40035b0 con 0x7fd1b806dcc0 2023-08-27T11:18:29.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b80b05f0 msgr2=0x7fd1b80b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:29.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 --2- 172.21.15.112:0/617976161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b80b05f0 0x7fd1b80b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 >> v1:172.21.15.138:6789/0 conn(0x7fd1b815a120 legacy=0x7fd1b806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b80afb10 con 0x7fd1b806dcc0 2023-08-27T11:18:29.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fd1b4001670 con 0x7fd1b806dcc0 2023-08-27T11:18:29.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 >> v1:172.21.15.112:6789/0 conn(0x7fd1b806dcc0 legacy=0x7fd1b80b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 --2- 172.21.15.112:0/617976161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a4003bc0 0x7fd1a4003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 --2- 172.21.15.112:0/617976161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1a4004850 0x7fd1a4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 --2- 172.21.15.112:0/617976161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b80b05f0 0x7fd1a4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b80af830 con 0x7fd1a4004850 2023-08-27T11:18:29.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a400bac0 con 0x7fd1a4003bc0 2023-08-27T11:18:29.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a400bc50 con 0x7fd1b80b05f0 2023-08-27T11:18:29.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bdd12700 1 --2- 172.21.15.112:0/617976161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1a4004850 0x7fd1a4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1be513700 1 --2- 172.21.15.112:0/617976161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a4003bc0 0x7fd1a4003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bed14700 1 --2- 172.21.15.112:0/617976161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b80b05f0 0x7fd1a4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bdd12700 1 -- 172.21.15.112:0/617976161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b80b05f0 msgr2=0x7fd1a4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:29.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bdd12700 1 --2- 172.21.15.112:0/617976161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b80b05f0 0x7fd1a4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bdd12700 1 -- 172.21.15.112:0/617976161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a4003bc0 msgr2=0x7fd1a4003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:29.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bdd12700 1 --2- 172.21.15.112:0/617976161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a4003bc0 0x7fd1a4003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bdd12700 1 -- 172.21.15.112:0/617976161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a40035b0 con 0x7fd1a4004850 2023-08-27T11:18:29.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bed14700 1 --2- 172.21.15.112:0/617976161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b80b05f0 0x7fd1a4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:29.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.043+0000 7fd1bdd12700 1 --2- 172.21.15.112:0/617976161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1a4004850 0x7fd1a4004cd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400c210 tx=0x7fd1a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297eacd3d10edd7f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:29.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.044+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b400cd70 con 0x7fd1a4004850 2023-08-27T11:18:29.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.044+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b4001af0 con 0x7fd1a4004850 2023-08-27T11:18:29.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.044+0000 7fd1bd511700 1 -- 172.21.15.112:0/617976161 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4005ba0 con 0x7fd1a4004850 2023-08-27T11:18:29.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.044+0000 7fd1c0777700 1 -- 172.21.15.112:0/617976161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1a4004850 msgr2=0x7fd1a4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.044+0000 7fd1c0777700 1 --2- 172.21.15.112:0/617976161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1a4004850 0x7fd1a4004cd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400c210 tx=0x7fd1a4005880 comp rx=0 tx=0).stop 2023-08-27T11:18:29.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.044+0000 7fd1bed14700 1 -- 172.21.15.112:0/617976161 reap_dead start 2023-08-27T11:18:29.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.044+0000 7fd1c0777700 1 -- 172.21.15.112:0/617976161 shutdown_connections 2023-08-27T11:18:29.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.044+0000 7fd1c0777700 1 -- 172.21.15.112:0/617976161 >> 172.21.15.112:0/617976161 conn(0x7fd1b80cff80 msgr2=0x7fd1b80d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:29.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.044+0000 7fd1c0777700 1 -- 172.21.15.112:0/617976161 shutdown_connections 2023-08-27T11:18:29.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.044+0000 7fd1c0777700 1 -- 172.21.15.112:0/617976161 wait complete. 2023-08-27T11:18:29.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.046+0000 7fd1c0777700 1 Processor -- start 2023-08-27T11:18:29.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.046+0000 7fd1c0777700 1 -- start start 2023-08-27T11:18:29.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1c0777700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a4004850 0x7fd1b81595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1c0777700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b80c69e0 0x7fd1b80c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1c0777700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b8147970 0x7fd1b80c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1c0777700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b80c5940 con 0x7fd1b80c69e0 2023-08-27T11:18:29.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1c0777700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b80c5c20 con 0x7fd1a4004850 2023-08-27T11:18:29.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1c0777700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b80c5f00 con 0x7fd1b8147970 2023-08-27T11:18:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1bdd12700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b80c69e0 0x7fd1b80c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1bdd12700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b80c69e0 0x7fd1b80c50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57488/0 (socket says 172.21.15.112:57488) 2023-08-27T11:18:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1bdd12700 1 -- 172.21.15.112:0/3570793334 learned_addr learned my addr 172.21.15.112:0/3570793334 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1be513700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a4004850 0x7fd1b81595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1bed14700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b8147970 0x7fd1b80c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1bdd12700 1 -- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b8147970 msgr2=0x7fd1b80c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1bdd12700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b8147970 0x7fd1b80c74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1bdd12700 1 -- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a4004850 msgr2=0x7fd1b81595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1bdd12700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a4004850 0x7fd1b81595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1bdd12700 1 -- 172.21.15.112:0/3570793334 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b80c61e0 con 0x7fd1b80c69e0 2023-08-27T11:18:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1bed14700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b8147970 0x7fd1b80c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1bdd12700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b80c69e0 0x7fd1b80c50f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4004790 tx=0x7fd1b4001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1aeffd700 1 -- 172.21.15.112:0/3570793334 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4020070 con 0x7fd1b80c69e0 2023-08-27T11:18:29.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1aeffd700 1 -- 172.21.15.112:0/3570793334 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b40070d0 con 0x7fd1b80c69e0 2023-08-27T11:18:29.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1c0777700 1 -- 172.21.15.112:0/3570793334 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b80c64c0 con 0x7fd1b80c69e0 2023-08-27T11:18:29.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1aeffd700 1 -- 172.21.15.112:0/3570793334 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4007480 con 0x7fd1b80c69e0 2023-08-27T11:18:29.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.047+0000 7fd1c0777700 1 -- 172.21.15.112:0/3570793334 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b8147df0 con 0x7fd1b80c69e0 2023-08-27T11:18:29.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.048+0000 7fd1c0777700 1 -- 172.21.15.112:0/3570793334 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b80c6830 con 0x7fd1b80c69e0 2023-08-27T11:18:29.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.049+0000 7fd1aeffd700 1 -- 172.21.15.112:0/3570793334 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd1b40075e0 con 0x7fd1b80c69e0 2023-08-27T11:18:29.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.049+0000 7fd1aeffd700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd19c06a0e0 0x7fd19c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.049+0000 7fd1aeffd700 1 -- 172.21.15.112:0/3570793334 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fd1b408a6c0 con 0x7fd1b80c69e0 2023-08-27T11:18:29.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.049+0000 7fd1be513700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd19c06a0e0 0x7fd19c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.050+0000 7fd1be513700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd19c06a0e0 0x7fd19c06c5a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8007fd0 tx=0x7fd1a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.053+0000 7fd1aeffd700 1 -- 172.21.15.112:0/3570793334 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd1b4055f00 con 0x7fd1b80c69e0 2023-08-27T11:18:29.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.244+0000 7fd1c0777700 1 -- 172.21.15.112:0/3570793334 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1b8067a20 con 0x7fd1b80c69e0 2023-08-27T11:18:29.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.245+0000 7fd1aeffd700 1 -- 172.21.15.112:0/3570793334 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18448 (secure 0 0 0) 0x7fd1b4017030 con 0x7fd1b80c69e0 2023-08-27T11:18:29.247 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:29.247 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":40,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:21.807644+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:01.935213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:02.602607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/1024768929":"2023-08-28T11:18:18.799945+0000","172.21.15.112:6833/1024768929":"2023-08-28T11:18:18.799945+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:29.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.247+0000 7fd1c0777700 1 -- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd19c06a0e0 msgr2=0x7fd19c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.247+0000 7fd1c0777700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd19c06a0e0 0x7fd19c06c5a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8007fd0 tx=0x7fd1a8009040 comp rx=0 tx=0).stop 2023-08-27T11:18:29.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.247+0000 7fd1c0777700 1 -- 172.21.15.112:0/3570793334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b80c69e0 msgr2=0x7fd1b80c50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.247+0000 7fd1c0777700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b80c69e0 0x7fd1b80c50f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4004790 tx=0x7fd1b4001980 comp rx=0 tx=0).stop 2023-08-27T11:18:29.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.247+0000 7fd1c0777700 1 -- 172.21.15.112:0/3570793334 shutdown_connections 2023-08-27T11:18:29.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.247+0000 7fd1c0777700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd19c06a0e0 0x7fd19c06c5a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.247+0000 7fd1c0777700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1b8147970 0x7fd1b80c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.247+0000 7fd1c0777700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a4004850 0x7fd1b81595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.247+0000 7fd1c0777700 1 --2- 172.21.15.112:0/3570793334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b80c69e0 0x7fd1b80c50f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.247+0000 7fd1c0777700 1 -- 172.21.15.112:0/3570793334 >> 172.21.15.112:0/3570793334 conn(0x7fd1b80cff80 msgr2=0x7fd1b806e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:29.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.247+0000 7fd1c0777700 1 -- 172.21.15.112:0/3570793334 shutdown_connections 2023-08-27T11:18:29.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.247+0000 7fd1c0777700 1 -- 172.21.15.112:0/3570793334 wait complete. 2023-08-27T11:18:29.261 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:18:29.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:18:29.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.380+0000 7fb5a091e700 1 Processor -- start 2023-08-27T11:18:29.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.380+0000 7fb5a091e700 1 -- start start 2023-08-27T11:18:29.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.381+0000 7fb5a091e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb59c06e860 0x7fb59c0acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.381+0000 7fb5a091e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb59c0ad150 0x7fb59c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.381+0000 7fb5a091e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59c140810 con 0x7fb59c140430 2023-08-27T11:18:29.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.381+0000 7fb5a091e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59c15db70 con 0x7fb59c0ad150 2023-08-27T11:18:29.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.381+0000 7fb5a091e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59c15de50 con 0x7fb59c06e860 2023-08-27T11:18:29.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.381+0000 7fb59ad9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb59c0ad150 0x7fb59c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.381+0000 7fb59ad9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb59c0ad150 0x7fb59c0cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57498/0 (socket says 172.21.15.112:57498) 2023-08-27T11:18:29.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.381+0000 7fb59ad9d700 1 -- 172.21.15.112:0/1286818547 learned_addr learned my addr 172.21.15.112:0/1286818547 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:29.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.382+0000 7fb59ad9d700 1 -- 172.21.15.112:0/1286818547 >> v2:172.21.15.138:3300/0 conn(0x7fb59c06e860 msgr2=0x7fb59c0acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:29.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.382+0000 7fb599d9b700 1 --2- 172.21.15.112:0/1286818547 >> v2:172.21.15.138:3300/0 conn(0x7fb59c06e860 0x7fb59c0acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.382+0000 7fb59ad9d700 1 --2- 172.21.15.112:0/1286818547 >> v2:172.21.15.138:3300/0 conn(0x7fb59c06e860 0x7fb59c0acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.382+0000 7fb59ad9d700 1 -- 172.21.15.112:0/1286818547 >> v1:172.21.15.138:6789/0 conn(0x7fb59c140430 legacy=0x7fb59c06e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:29.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.382+0000 7fb59ad9d700 1 -- 172.21.15.112:0/1286818547 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c15e130 con 0x7fb59c0ad150 2023-08-27T11:18:29.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.382+0000 7fb599d9b700 1 --2- 172.21.15.112:0/1286818547 >> v2:172.21.15.138:3300/0 conn(0x7fb59c06e860 0x7fb59c0acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:29.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.382+0000 7fb59ad9d700 1 --2- 172.21.15.112:0/1286818547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb59c0ad150 0x7fb59c0cd740 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb59c147190 tx=0x7fb59400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b52fd465d98a78f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:29.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.382+0000 7fb59959a700 1 -- 172.21.15.112:0/1286818547 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59400e070 con 0x7fb59c0ad150 2023-08-27T11:18:29.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.382+0000 7fb59959a700 1 -- 172.21.15.112:0/1286818547 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb594003a20 con 0x7fb59c0ad150 2023-08-27T11:18:29.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.382+0000 7fb59959a700 1 -- 172.21.15.112:0/1286818547 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594003dd0 con 0x7fb59c0ad150 2023-08-27T11:18:29.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.382+0000 7fb5a091e700 1 -- 172.21.15.112:0/1286818547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb59c0ad150 msgr2=0x7fb59c0cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.382+0000 7fb5a091e700 1 --2- 172.21.15.112:0/1286818547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb59c0ad150 0x7fb59c0cd740 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb59c147190 tx=0x7fb59400b330 comp rx=0 tx=0).stop 2023-08-27T11:18:29.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.383+0000 7fb5a091e700 1 -- 172.21.15.112:0/1286818547 shutdown_connections 2023-08-27T11:18:29.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.383+0000 7fb5a091e700 1 --2- 172.21.15.112:0/1286818547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb59c0ad150 0x7fb59c0cd740 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.383+0000 7fb5a091e700 1 --2- 172.21.15.112:0/1286818547 >> v2:172.21.15.138:3300/0 conn(0x7fb59c06e860 0x7fb59c0acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.383+0000 7fb5a091e700 1 -- 172.21.15.112:0/1286818547 >> 172.21.15.112:0/1286818547 conn(0x7fb59c06cd30 msgr2=0x7fb59c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:29.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.383+0000 7fb5a091e700 1 -- 172.21.15.112:0/1286818547 shutdown_connections 2023-08-27T11:18:29.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.383+0000 7fb5a091e700 1 -- 172.21.15.112:0/1286818547 wait complete. 2023-08-27T11:18:29.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.384+0000 7fb5a091e700 1 Processor -- start 2023-08-27T11:18:29.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.384+0000 7fb5a091e700 1 -- start start 2023-08-27T11:18:29.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb5a091e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb59c06e860 0x7fb59c156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb5a091e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb59c0ad150 0x7fb59c160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb5a091e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb59c140430 0x7fb59c0cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb5a091e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c158ba0 con 0x7fb59c140430 2023-08-27T11:18:29.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb5a091e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c158e80 con 0x7fb59c0ad150 2023-08-27T11:18:29.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb5a091e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c159160 con 0x7fb59c06e860 2023-08-27T11:18:29.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb59ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb59c140430 0x7fb59c0cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb59ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb59c140430 0x7fb59c0cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57504/0 (socket says 172.21.15.112:57504) 2023-08-27T11:18:29.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb59ad9d700 1 -- 172.21.15.112:0/1123654183 learned_addr learned my addr 172.21.15.112:0/1123654183 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:29.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb599d9b700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb59c0ad150 0x7fb59c160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb59a59c700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb59c06e860 0x7fb59c156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb59ad9d700 1 -- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb59c06e860 msgr2=0x7fb59c156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb59ad9d700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb59c06e860 0x7fb59c156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb59ad9d700 1 -- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb59c0ad150 msgr2=0x7fb59c160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb59ad9d700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb59c0ad150 0x7fb59c160530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb59ad9d700 1 -- 172.21.15.112:0/1123654183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb594009b30 con 0x7fb59c140430 2023-08-27T11:18:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb59ad9d700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb59c140430 0x7fb59c0cd640 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb594015040 tx=0x7fb594004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb58affd700 1 -- 172.21.15.112:0/1123654183 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594004440 con 0x7fb59c140430 2023-08-27T11:18:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb58affd700 1 -- 172.21.15.112:0/1123654183 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5940092e0 con 0x7fb59c140430 2023-08-27T11:18:29.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb5a091e700 1 -- 172.21.15.112:0/1123654183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c159440 con 0x7fb59c140430 2023-08-27T11:18:29.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.385+0000 7fb58affd700 1 -- 172.21.15.112:0/1123654183 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5940047e0 con 0x7fb59c140430 2023-08-27T11:18:29.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.386+0000 7fb5a091e700 1 -- 172.21.15.112:0/1123654183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c157270 con 0x7fb59c140430 2023-08-27T11:18:29.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.386+0000 7fb5a091e700 1 -- 172.21.15.112:0/1123654183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59c0cd4b0 con 0x7fb59c140430 2023-08-27T11:18:29.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.387+0000 7fb58affd700 1 -- 172.21.15.112:0/1123654183 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb594012430 con 0x7fb59c140430 2023-08-27T11:18:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.387+0000 7fb58affd700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb57c06a0e0 0x7fb57c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.387+0000 7fb58affd700 1 -- 172.21.15.112:0/1123654183 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fb594089910 con 0x7fb59c140430 2023-08-27T11:18:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.387+0000 7fb59a59c700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb57c06a0e0 0x7fb57c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.388+0000 7fb59a59c700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb57c06a0e0 0x7fb57c06c5a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb590009960 tx=0x7fb5900090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.392+0000 7fb58affd700 1 -- 172.21.15.112:0/1123654183 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb594058ac0 con 0x7fb59c140430 2023-08-27T11:18:29.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.628+0000 7fb5a091e700 1 -- 172.21.15.112:0/1123654183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb59c15e3e0 con 0x7fb59c140430 2023-08-27T11:18:29.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.629+0000 7fb58affd700 1 -- 172.21.15.112:0/1123654183 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4887 (secure 0 0 0) 0x7fb594058650 con 0x7fb59c140430 2023-08-27T11:18:29.631 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:29.631 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1941625698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1941625698},{"type":"v1","addr":"172.21.15.138:6836","nonce":1941625698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4904,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1317948937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1317948937},{"type":"v1","addr":"172.21.15.112:6835","nonce":1317948937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1530886230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1530886230},{"type":"v1","addr":"172.21.15.112:6834","nonce":1530886230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:18:03.637863+0000","modified":"2023-08-27T11:18:07.262281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5175":{"gid":5175,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6837/3409848660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3409848660},{"type":"v1","addr":"172.21.15.138:6837","nonce":3409848660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-27T11:18:29.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.631+0000 7fb5a091e700 1 -- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb57c06a0e0 msgr2=0x7fb57c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.631+0000 7fb5a091e700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb57c06a0e0 0x7fb57c06c5a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb590009960 tx=0x7fb5900090e0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.631+0000 7fb5a091e700 1 -- 172.21.15.112:0/1123654183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb59c140430 msgr2=0x7fb59c0cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.631+0000 7fb5a091e700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb59c140430 0x7fb59c0cd640 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb594015040 tx=0x7fb594004ad0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.631+0000 7fb5a091e700 1 -- 172.21.15.112:0/1123654183 shutdown_connections 2023-08-27T11:18:29.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.631+0000 7fb5a091e700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb57c06a0e0 0x7fb57c06c5a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.631+0000 7fb5a091e700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb59c140430 0x7fb59c0cd640 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.631+0000 7fb5a091e700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb59c06e860 0x7fb59c156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.631+0000 7fb5a091e700 1 --2- 172.21.15.112:0/1123654183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb59c0ad150 0x7fb59c160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.631+0000 7fb5a091e700 1 -- 172.21.15.112:0/1123654183 >> 172.21.15.112:0/1123654183 conn(0x7fb59c06cd30 msgr2=0x7fb59c15b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:29.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.631+0000 7fb5a091e700 1 -- 172.21.15.112:0/1123654183 shutdown_connections 2023-08-27T11:18:29.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.631+0000 7fb5a091e700 1 -- 172.21.15.112:0/1123654183 wait complete. 2023-08-27T11:18:29.635 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2023-08-27T11:18:29.646 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:18:29.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.765+0000 7f95f644b700 1 Processor -- start 2023-08-27T11:18:29.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.765+0000 7f95f644b700 1 -- start start 2023-08-27T11:18:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95f644b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f95f00b1400 0x7f95f00cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95f644b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f0161ce0 0x7f95f006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95f644b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f0161850 con 0x7f95f00b17e0 2023-08-27T11:18:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95f644b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f00cfc10 con 0x7f95f00b1400 2023-08-27T11:18:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95f644b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f0161200 con 0x7f95f0161ce0 2023-08-27T11:18:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95ef7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f95f00b17e0 0x7f95f006e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51892/0 (socket says 172.21.15.112:51892) 2023-08-27T11:18:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95ef7fe700 1 -- 172.21.15.112:0/1534518132 learned_addr learned my addr 172.21.15.112:0/1534518132 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95effff700 1 --2- 172.21.15.112:0/1534518132 >> v2:172.21.15.138:3300/0 conn(0x7f95f00b1400 0x7f95f00cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95f49e8700 1 --2- 172.21.15.112:0/1534518132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f0161ce0 0x7f95f006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95eeffd700 1 -- 172.21.15.112:0/1534518132 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032908646 0 0) 0x7f95f0161850 con 0x7f95f00b17e0 2023-08-27T11:18:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95eeffd700 1 -- 172.21.15.112:0/1534518132 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d80035b0 con 0x7f95f00b17e0 2023-08-27T11:18:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95f49e8700 1 -- 172.21.15.112:0/1534518132 >> v2:172.21.15.138:3300/0 conn(0x7f95f00b1400 msgr2=0x7f95f00cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95f49e8700 1 --2- 172.21.15.112:0/1534518132 >> v2:172.21.15.138:3300/0 conn(0x7f95f00b1400 0x7f95f00cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95f49e8700 1 -- 172.21.15.112:0/1534518132 >> v1:172.21.15.112:6789/0 conn(0x7f95f00b17e0 legacy=0x7f95f006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95f49e8700 1 -- 172.21.15.112:0/1534518132 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f01614e0 con 0x7f95f0161ce0 2023-08-27T11:18:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.766+0000 7f95f49e8700 1 --2- 172.21.15.112:0/1534518132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f0161ce0 0x7f95f006f790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009d90 tx=0x7f95e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f36951e1c15c1d25 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.767+0000 7f95eeffd700 1 -- 172.21.15.112:0/1534518132 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e800e070 con 0x7f95f0161ce0 2023-08-27T11:18:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.767+0000 7f95eeffd700 1 -- 172.21.15.112:0/1534518132 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e800bb20 con 0x7f95f0161ce0 2023-08-27T11:18:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.767+0000 7f95eeffd700 1 -- 172.21.15.112:0/1534518132 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8012430 con 0x7f95f0161ce0 2023-08-27T11:18:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.767+0000 7f95f644b700 1 -- 172.21.15.112:0/1534518132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f0161ce0 msgr2=0x7f95f006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.767+0000 7f95f644b700 1 --2- 172.21.15.112:0/1534518132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f0161ce0 0x7f95f006f790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009d90 tx=0x7f95e800b330 comp rx=0 tx=0).stop 2023-08-27T11:18:29.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.767+0000 7f95f644b700 1 -- 172.21.15.112:0/1534518132 shutdown_connections 2023-08-27T11:18:29.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.767+0000 7f95f644b700 1 --2- 172.21.15.112:0/1534518132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f0161ce0 0x7f95f006f790 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.767+0000 7f95f644b700 1 --2- 172.21.15.112:0/1534518132 >> v2:172.21.15.138:3300/0 conn(0x7f95f00b1400 0x7f95f00cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.767+0000 7f95f644b700 1 -- 172.21.15.112:0/1534518132 >> 172.21.15.112:0/1534518132 conn(0x7f95f00d0000 msgr2=0x7f95f00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.767+0000 7f95f644b700 1 -- 172.21.15.112:0/1534518132 shutdown_connections 2023-08-27T11:18:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.767+0000 7f95f644b700 1 -- 172.21.15.112:0/1534518132 wait complete. 2023-08-27T11:18:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.768+0000 7f95f644b700 1 Processor -- start 2023-08-27T11:18:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.769+0000 7f95f644b700 1 -- start start 2023-08-27T11:18:29.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.769+0000 7f95f644b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f00b1400 0x7f95f00affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.769+0000 7f95f644b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00b17e0 0x7f95f00cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.769+0000 7f95f644b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f0161ce0 0x7f95f00cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.769+0000 7f95f644b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f01f3c50 con 0x7f95f0161ce0 2023-08-27T11:18:29.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.769+0000 7f95f644b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f00c4a90 con 0x7f95f00b1400 2023-08-27T11:18:29.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.769+0000 7f95f644b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f00c4600 con 0x7f95f00b17e0 2023-08-27T11:18:29.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.769+0000 7f95f49e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f0161ce0 0x7f95f00cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.769+0000 7f95f49e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f0161ce0 0x7f95f00cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57518/0 (socket says 172.21.15.112:57518) 2023-08-27T11:18:29.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.769+0000 7f95f49e8700 1 -- 172.21.15.112:0/547057025 learned_addr learned my addr 172.21.15.112:0/547057025 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:29.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.770+0000 7f95effff700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f00b1400 0x7f95f00affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.770+0000 7f95ef7fe700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00b17e0 0x7f95f00cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.770+0000 7f95f49e8700 1 -- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00b17e0 msgr2=0x7f95f00cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.770+0000 7f95f49e8700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00b17e0 0x7f95f00cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.770+0000 7f95f49e8700 1 -- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f00b1400 msgr2=0x7f95f00affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:29.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.770+0000 7f95f49e8700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f00b1400 0x7f95f00affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:29.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.770+0000 7f95f49e8700 1 -- 172.21.15.112:0/547057025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e8009a70 con 0x7f95f0161ce0 2023-08-27T11:18:29.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.770+0000 7f95f49e8700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f0161ce0 0x7f95f00cbd00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f95e8014040 tx=0x7f95e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:29.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.770+0000 7f95ecff9700 1 -- 172.21.15.112:0/547057025 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e80129f0 con 0x7f95f0161ce0 2023-08-27T11:18:29.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.770+0000 7f95ecff9700 1 -- 172.21.15.112:0/547057025 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e8020070 con 0x7f95f0161ce0 2023-08-27T11:18:29.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.770+0000 7f95f644b700 1 -- 172.21.15.112:0/547057025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f00c3cd0 con 0x7f95f0161ce0 2023-08-27T11:18:29.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.770+0000 7f95ecff9700 1 -- 172.21.15.112:0/547057025 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8012db0 con 0x7f95f0161ce0 2023-08-27T11:18:29.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.770+0000 7f95f644b700 1 -- 172.21.15.112:0/547057025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f00c4290 con 0x7f95f0161ce0 2023-08-27T11:18:29.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.771+0000 7f95f644b700 1 -- 172.21.15.112:0/547057025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f00c47b0 con 0x7f95f0161ce0 2023-08-27T11:18:29.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.771+0000 7f95ecff9700 1 -- 172.21.15.112:0/547057025 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f95e8004040 con 0x7f95f0161ce0 2023-08-27T11:18:29.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.772+0000 7f95ecff9700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95dc06a0e0 0x7f95dc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:29.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.772+0000 7f95ecff9700 1 -- 172.21.15.112:0/547057025 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f95e8022070 con 0x7f95f0161ce0 2023-08-27T11:18:29.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.772+0000 7f95effff700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95dc06a0e0 0x7f95dc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:29.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.772+0000 7f95effff700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95dc06a0e0 0x7f95dc06c5a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009b70 tx=0x7f95e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:29.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:29.776+0000 7f95ecff9700 1 -- 172.21.15.112:0/547057025 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f95f00c47b0 con 0x7f95f0161ce0 2023-08-27T11:18:30.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.007+0000 7f95f644b700 1 -- 172.21.15.112:0/547057025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95f00c4770 con 0x7f95f0161ce0 2023-08-27T11:18:30.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.007+0000 7f95ecff9700 1 -- 172.21.15.112:0/547057025 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4887 (secure 0 0 0) 0x7f95e8059780 con 0x7f95f0161ce0 2023-08-27T11:18:30.009 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:30.009 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1941625698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1941625698},{"type":"v1","addr":"172.21.15.138:6836","nonce":1941625698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4904,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1317948937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1317948937},{"type":"v1","addr":"172.21.15.112:6835","nonce":1317948937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1530886230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1530886230},{"type":"v1","addr":"172.21.15.112:6834","nonce":1530886230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:18:03.637863+0000","modified":"2023-08-27T11:18:07.262281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5175":{"gid":5175,"name":"b","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6837/3409848660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3409848660},{"type":"v1","addr":"172.21.15.138:6837","nonce":3409848660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-27T11:18:30.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.010+0000 7f95f644b700 1 -- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95dc06a0e0 msgr2=0x7f95dc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.010+0000 7f95f644b700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95dc06a0e0 0x7f95dc06c5a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009b70 tx=0x7f95e0008040 comp rx=0 tx=0).stop 2023-08-27T11:18:30.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.010+0000 7f95f644b700 1 -- 172.21.15.112:0/547057025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f0161ce0 msgr2=0x7f95f00cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.010+0000 7f95f644b700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f0161ce0 0x7f95f00cbd00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f95e8014040 tx=0x7f95e8004ad0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.010+0000 7f95f644b700 1 -- 172.21.15.112:0/547057025 shutdown_connections 2023-08-27T11:18:30.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.010+0000 7f95f644b700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95dc06a0e0 0x7f95dc06c5a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.010+0000 7f95f644b700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f0161ce0 0x7f95f00cbd00 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.010+0000 7f95f644b700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f00b1400 0x7f95f00affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.010+0000 7f95f644b700 1 --2- 172.21.15.112:0/547057025 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00b17e0 0x7f95f00cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.010+0000 7f95f644b700 1 -- 172.21.15.112:0/547057025 >> 172.21.15.112:0/547057025 conn(0x7f95f00d0000 msgr2=0x7f95f00cada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:30.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.010+0000 7f95f644b700 1 -- 172.21.15.112:0/547057025 shutdown_connections 2023-08-27T11:18:30.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.010+0000 7f95f644b700 1 -- 172.21.15.112:0/547057025 wait complete. 2023-08-27T11:18:30.018 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2023-08-27T11:18:30.024 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:30.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.144+0000 7fce61a11700 1 Processor -- start 2023-08-27T11:18:30.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.144+0000 7fce61a11700 1 -- start start 2023-08-27T11:18:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.145+0000 7fce61a11700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c06dc20 0x7fce5c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.145+0000 7fce61a11700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5c06f930 con 0x7fce5c146990 2023-08-27T11:18:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.145+0000 7fce61a11700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5c06fc10 con 0x7fce5c160a00 2023-08-27T11:18:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.145+0000 7fce61a11700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c06fef0 con 0x7fce5c06dc20 2023-08-27T11:18:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.145+0000 7fce5a7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fce5c146990 0x7fce5c06d510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51906/0 (socket says 172.21.15.112:51906) 2023-08-27T11:18:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.145+0000 7fce5a7fc700 1 -- 172.21.15.112:0/3165507282 learned_addr learned my addr 172.21.15.112:0/3165507282 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.145+0000 7fce5b7fe700 1 --2- 172.21.15.112:0/3165507282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c06dc20 0x7fce5c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.145+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891392363 0 0) 0x7fce5c06f930 con 0x7fce5c146990 2023-08-27T11:18:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.145+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce440035b0 con 0x7fce5c146990 2023-08-27T11:18:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1835308095 0 0) 0x7fce440035b0 con 0x7fce5c146990 2023-08-27T11:18:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c06dc20 msgr2=0x7fce5c0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 --2- 172.21.15.112:0/3165507282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c06dc20 0x7fce5c0b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 >> v1:172.21.15.138:6789/0 conn(0x7fce5c160a00 legacy=0x7fce5c15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c0701d0 con 0x7fce5c146990 2023-08-27T11:18:30.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fce4c001670 con 0x7fce5c146990 2023-08-27T11:18:30.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 >> v1:172.21.15.112:6789/0 conn(0x7fce5c146990 legacy=0x7fce5c06d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 --2- 172.21.15.112:0/3165507282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce44003a30 0x7fce44003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 --2- 172.21.15.112:0/3165507282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c06dc20 0x7fce44004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 --2- 172.21.15.112:0/3165507282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44004d90 0x7fce44005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c06fef0 con 0x7fce44004d90 2023-08-27T11:18:30.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4400bbd0 con 0x7fce44003a30 2023-08-27T11:18:30.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4400bd90 con 0x7fce5c06dc20 2023-08-27T11:18:30.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce5b7fe700 1 --2- 172.21.15.112:0/3165507282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44004d90 0x7fce44005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce5affd700 1 --2- 172.21.15.112:0/3165507282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce44003a30 0x7fce44003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce5a7fc700 1 --2- 172.21.15.112:0/3165507282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c06dc20 0x7fce44004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce5b7fe700 1 -- 172.21.15.112:0/3165507282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c06dc20 msgr2=0x7fce44004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce5b7fe700 1 --2- 172.21.15.112:0/3165507282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c06dc20 0x7fce44004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce5b7fe700 1 -- 172.21.15.112:0/3165507282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce44003a30 msgr2=0x7fce44003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce5b7fe700 1 --2- 172.21.15.112:0/3165507282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce44003a30 0x7fce44003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce5b7fe700 1 -- 172.21.15.112:0/3165507282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce440035b0 con 0x7fce44004d90 2023-08-27T11:18:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce5b7fe700 1 --2- 172.21.15.112:0/3165507282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44004d90 0x7fce44005210 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fce54007330 tx=0x7fce44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22a9b282dba0bb0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54007e50 con 0x7fce44004d90 2023-08-27T11:18:30.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce54014400 con 0x7fce44004d90 2023-08-27T11:18:30.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.146+0000 7fce59ffb700 1 -- 172.21.15.112:0/3165507282 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce540147b0 con 0x7fce44004d90 2023-08-27T11:18:30.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.147+0000 7fce61a11700 1 -- 172.21.15.112:0/3165507282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44004d90 msgr2=0x7fce44005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.147+0000 7fce61a11700 1 --2- 172.21.15.112:0/3165507282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44004d90 0x7fce44005210 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fce54007330 tx=0x7fce44005880 comp rx=0 tx=0).stop 2023-08-27T11:18:30.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.147+0000 7fce5b7fe700 1 -- 172.21.15.112:0/3165507282 reap_dead start 2023-08-27T11:18:30.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.147+0000 7fce61a11700 1 -- 172.21.15.112:0/3165507282 shutdown_connections 2023-08-27T11:18:30.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.147+0000 7fce61a11700 1 -- 172.21.15.112:0/3165507282 >> 172.21.15.112:0/3165507282 conn(0x7fce5c0cfe50 msgr2=0x7fce5c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:30.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.147+0000 7fce61a11700 1 -- 172.21.15.112:0/3165507282 shutdown_connections 2023-08-27T11:18:30.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.147+0000 7fce61a11700 1 -- 172.21.15.112:0/3165507282 wait complete. 2023-08-27T11:18:30.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.148+0000 7fce61a11700 1 Processor -- start 2023-08-27T11:18:30.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.148+0000 7fce61a11700 1 -- start start 2023-08-27T11:18:30.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce61a11700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce5c0aff40 0x7fce5c0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce61a11700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce5c0b1eb0 0x7fce5c143ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce61a11700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c144020 0x7fce5c1f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce61a11700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c1444a0 con 0x7fce5c0b1eb0 2023-08-27T11:18:30.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce61a11700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c0ad080 con 0x7fce5c0aff40 2023-08-27T11:18:30.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce61a11700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c0b24c0 con 0x7fce5c144020 2023-08-27T11:18:30.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce5a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce5c0b1eb0 0x7fce5c143ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce5a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce5c0b1eb0 0x7fce5c143ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57536/0 (socket says 172.21.15.112:57536) 2023-08-27T11:18:30.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce5a7fc700 1 -- 172.21.15.112:0/461973693 learned_addr learned my addr 172.21.15.112:0/461973693 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce5affd700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce5c0aff40 0x7fce5c0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce5b7fe700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c144020 0x7fce5c1f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce5a7fc700 1 -- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c144020 msgr2=0x7fce5c1f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce5a7fc700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c144020 0x7fce5c1f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce5a7fc700 1 -- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce5c0aff40 msgr2=0x7fce5c0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce5a7fc700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce5c0aff40 0x7fce5c0b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce5a7fc700 1 -- 172.21.15.112:0/461973693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c0aca30 con 0x7fce5c0b1eb0 2023-08-27T11:18:30.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce5affd700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce5c0aff40 0x7fce5c0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:30.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.149+0000 7fce5b7fe700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c144020 0x7fce5c1f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:30.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.150+0000 7fce5a7fc700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce5c0b1eb0 0x7fce5c143ae0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fce4c007290 tx=0x7fce4c007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:30.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.150+0000 7fce43fff700 1 -- 172.21.15.112:0/461973693 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c007830 con 0x7fce5c0b1eb0 2023-08-27T11:18:30.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.150+0000 7fce43fff700 1 -- 172.21.15.112:0/461973693 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c00f870 con 0x7fce5c0b1eb0 2023-08-27T11:18:30.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.150+0000 7fce43fff700 1 -- 172.21.15.112:0/461973693 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c00fc20 con 0x7fce5c0b1eb0 2023-08-27T11:18:30.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.150+0000 7fce61a11700 1 -- 172.21.15.112:0/461973693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5c0acd10 con 0x7fce5c0b1eb0 2023-08-27T11:18:30.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.150+0000 7fce61a11700 1 -- 172.21.15.112:0/461973693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c0b0320 con 0x7fce5c0b1eb0 2023-08-27T11:18:30.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.151+0000 7fce61a11700 1 -- 172.21.15.112:0/461973693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5c0ac890 con 0x7fce5c0b1eb0 2023-08-27T11:18:30.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.151+0000 7fce43fff700 1 -- 172.21.15.112:0/461973693 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fce4c00fd80 con 0x7fce5c0b1eb0 2023-08-27T11:18:30.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.151+0000 7fce43fff700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fce4806a0e0 0x7fce4806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.151+0000 7fce43fff700 1 -- 172.21.15.112:0/461973693 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fce4c0886a0 con 0x7fce5c0b1eb0 2023-08-27T11:18:30.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.152+0000 7fce5affd700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fce4806a0e0 0x7fce4806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.152+0000 7fce5affd700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fce4806a0e0 0x7fce4806c5a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fce50007fd0 tx=0x7fce50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:30.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.156+0000 7fce43fff700 1 -- 172.21.15.112:0/461973693 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fce5c0ac890 con 0x7fce5c0b1eb0 2023-08-27T11:18:30.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.344+0000 7fce61a11700 1 -- 172.21.15.112:0/461973693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce5c0ac890 con 0x7fce5c0b1eb0 2023-08-27T11:18:30.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.345+0000 7fce43fff700 1 -- 172.21.15.112:0/461973693 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18448 (secure 0 0 0) 0x7fce5c0ac890 con 0x7fce5c0b1eb0 2023-08-27T11:18:30.347 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:30.348 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":40,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:21.807644+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:01.935213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:02.602607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/1024768929":"2023-08-28T11:18:18.799945+0000","172.21.15.112:6833/1024768929":"2023-08-28T11:18:18.799945+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.347+0000 7fce61a11700 1 -- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fce4806a0e0 msgr2=0x7fce4806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.347+0000 7fce61a11700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fce4806a0e0 0x7fce4806c5a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fce50007fd0 tx=0x7fce50009040 comp rx=0 tx=0).stop 2023-08-27T11:18:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.347+0000 7fce61a11700 1 -- 172.21.15.112:0/461973693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce5c0b1eb0 msgr2=0x7fce5c143ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.347+0000 7fce61a11700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce5c0b1eb0 0x7fce5c143ae0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fce4c007290 tx=0x7fce4c007650 comp rx=0 tx=0).stop 2023-08-27T11:18:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.347+0000 7fce61a11700 1 -- 172.21.15.112:0/461973693 shutdown_connections 2023-08-27T11:18:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.347+0000 7fce61a11700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fce4806a0e0 0x7fce4806c5a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.347+0000 7fce61a11700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce5c144020 0x7fce5c1f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.347+0000 7fce61a11700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce5c0aff40 0x7fce5c0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.347+0000 7fce61a11700 1 --2- 172.21.15.112:0/461973693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce5c0b1eb0 0x7fce5c143ae0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.347+0000 7fce61a11700 1 -- 172.21.15.112:0/461973693 >> 172.21.15.112:0/461973693 conn(0x7fce5c0cfe50 msgr2=0x7fce5c0b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:30.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.347+0000 7fce61a11700 1 -- 172.21.15.112:0/461973693 shutdown_connections 2023-08-27T11:18:30.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.347+0000 7fce61a11700 1 -- 172.21.15.112:0/461973693 wait complete. 2023-08-27T11:18:30.361 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:18:30.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.487+0000 7ff65c9dc700 1 Processor -- start 2023-08-27T11:18:30.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.487+0000 7ff65c9dc700 1 -- start start 2023-08-27T11:18:30.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.488+0000 7ff65c9dc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65806adb0 0x7ff65806b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.488+0000 7ff65c9dc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6580cdbd0 0x7ff6580ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.488+0000 7ff65c9dc700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65815eb60 con 0x7ff658151060 2023-08-27T11:18:30.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.488+0000 7ff65c9dc700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65806b770 con 0x7ff6580cdbd0 2023-08-27T11:18:30.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.488+0000 7ff65c9dc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580b02f0 con 0x7ff65806adb0 2023-08-27T11:18:30.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.488+0000 7ff656d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6580cdbd0 0x7ff6580ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.488+0000 7ff656d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6580cdbd0 0x7ff6580ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57546/0 (socket says 172.21.15.112:57546) 2023-08-27T11:18:30.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.488+0000 7ff656d9d700 1 -- 172.21.15.112:0/1364150366 learned_addr learned my addr 172.21.15.112:0/1364150366 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:30.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.488+0000 7ff655d9b700 1 --2- 172.21.15.112:0/1364150366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65806adb0 0x7ff65806b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff656d9d700 1 -- 172.21.15.112:0/1364150366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65806adb0 msgr2=0x7ff65806b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:30.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff656d9d700 1 --2- 172.21.15.112:0/1364150366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65806adb0 0x7ff65806b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff656d9d700 1 -- 172.21.15.112:0/1364150366 >> v1:172.21.15.138:6789/0 conn(0x7ff658151060 legacy=0x7ff65815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff656d9d700 1 -- 172.21.15.112:0/1364150366 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6580b05d0 con 0x7ff6580cdbd0 2023-08-27T11:18:30.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff656d9d700 1 --2- 172.21.15.112:0/1364150366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6580cdbd0 0x7ff6580ce060 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff650009e50 tx=0x7ff65000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561737d0c4477452 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:30.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff65559a700 1 -- 172.21.15.112:0/1364150366 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000e070 con 0x7ff6580cdbd0 2023-08-27T11:18:30.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff65559a700 1 -- 172.21.15.112:0/1364150366 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65000b910 con 0x7ff6580cdbd0 2023-08-27T11:18:30.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff65559a700 1 -- 172.21.15.112:0/1364150366 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000bcc0 con 0x7ff6580cdbd0 2023-08-27T11:18:30.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff65c9dc700 1 -- 172.21.15.112:0/1364150366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6580cdbd0 msgr2=0x7ff6580ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff65c9dc700 1 --2- 172.21.15.112:0/1364150366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6580cdbd0 0x7ff6580ce060 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff650009e50 tx=0x7ff65000b330 comp rx=0 tx=0).stop 2023-08-27T11:18:30.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff65c9dc700 1 -- 172.21.15.112:0/1364150366 shutdown_connections 2023-08-27T11:18:30.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff65c9dc700 1 --2- 172.21.15.112:0/1364150366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6580cdbd0 0x7ff6580ce060 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff65c9dc700 1 --2- 172.21.15.112:0/1364150366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65806adb0 0x7ff65806b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff65c9dc700 1 -- 172.21.15.112:0/1364150366 >> 172.21.15.112:0/1364150366 conn(0x7ff6580d0000 msgr2=0x7ff6580d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:30.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.489+0000 7ff65c9dc700 1 -- 172.21.15.112:0/1364150366 shutdown_connections 2023-08-27T11:18:30.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.490+0000 7ff65c9dc700 1 -- 172.21.15.112:0/1364150366 wait complete. 2023-08-27T11:18:30.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.491+0000 7ff65c9dc700 1 Processor -- start 2023-08-27T11:18:30.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.491+0000 7ff65c9dc700 1 -- start start 2023-08-27T11:18:30.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.492+0000 7ff65c9dc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65806adb0 0x7ff658070870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.492+0000 7ff65c9dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6580cdbd0 0x7ff658071d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.492+0000 7ff65c9dc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff658151060 0x7ff6580c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.492+0000 7ff65c9dc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580718f0 con 0x7ff658151060 2023-08-27T11:18:30.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.492+0000 7ff65c9dc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff658070db0 con 0x7ff6580cdbd0 2023-08-27T11:18:30.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.492+0000 7ff65c9dc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6581519a0 con 0x7ff65806adb0 2023-08-27T11:18:30.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.492+0000 7ff656d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff658151060 0x7ff6580c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.492+0000 7ff656d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff658151060 0x7ff6580c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57552/0 (socket says 172.21.15.112:57552) 2023-08-27T11:18:30.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.492+0000 7ff656d9d700 1 -- 172.21.15.112:0/575201950 learned_addr learned my addr 172.21.15.112:0/575201950 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:30.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.492+0000 7ff65659c700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65806adb0 0x7ff658070870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.493+0000 7ff655d9b700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6580cdbd0 0x7ff658071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.493+0000 7ff656d9d700 1 -- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65806adb0 msgr2=0x7ff658070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.493+0000 7ff656d9d700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65806adb0 0x7ff658070870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.493+0000 7ff656d9d700 1 -- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6580cdbd0 msgr2=0x7ff658071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.493+0000 7ff656d9d700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6580cdbd0 0x7ff658071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.493+0000 7ff656d9d700 1 -- 172.21.15.112:0/575201950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff650009b30 con 0x7ff658151060 2023-08-27T11:18:30.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.493+0000 7ff656d9d700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff658151060 0x7ff6580c70d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff65806e590 tx=0x7ff650004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:30.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.493+0000 7ff646ffd700 1 -- 172.21.15.112:0/575201950 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6500099b0 con 0x7ff658151060 2023-08-27T11:18:30.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.493+0000 7ff646ffd700 1 -- 172.21.15.112:0/575201950 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6500092e0 con 0x7ff658151060 2023-08-27T11:18:30.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.493+0000 7ff65c9dc700 1 -- 172.21.15.112:0/575201950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff658151c80 con 0x7ff658151060 2023-08-27T11:18:30.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.493+0000 7ff646ffd700 1 -- 172.21.15.112:0/575201950 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65001c640 con 0x7ff658151060 2023-08-27T11:18:30.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.493+0000 7ff65c9dc700 1 -- 172.21.15.112:0/575201950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff658071580 con 0x7ff658151060 2023-08-27T11:18:30.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.494+0000 7ff65c9dc700 1 -- 172.21.15.112:0/575201950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff658071740 con 0x7ff658151060 2023-08-27T11:18:30.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.494+0000 7ff646ffd700 1 -- 172.21.15.112:0/575201950 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff65001c7a0 con 0x7ff658151060 2023-08-27T11:18:30.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.495+0000 7ff646ffd700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff63806a0e0 0x7ff63806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.495+0000 7ff646ffd700 1 -- 172.21.15.112:0/575201950 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7ff65008a270 con 0x7ff658151060 2023-08-27T11:18:30.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.495+0000 7ff65659c700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff63806a0e0 0x7ff63806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.495+0000 7ff65659c700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff63806a0e0 0x7ff63806c5a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff6581f1950 tx=0x7ff64c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:30.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.499+0000 7ff646ffd700 1 -- 172.21.15.112:0/575201950 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff650055910 con 0x7ff658151060 2023-08-27T11:18:30.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.730+0000 7ff65c9dc700 1 -- 172.21.15.112:0/575201950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff6580710f0 con 0x7ff658151060 2023-08-27T11:18:30.733 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:18:30.731+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:18:30.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.735+0000 7ff646ffd700 1 -- 172.21.15.112:0/575201950 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff650058f30 con 0x7ff658151060 2023-08-27T11:18:30.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.737+0000 7ff65c9dc700 1 -- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff63806a0e0 msgr2=0x7ff63806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.737+0000 7ff65c9dc700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff63806a0e0 0x7ff63806c5a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff6581f1950 tx=0x7ff64c009040 comp rx=0 tx=0).stop 2023-08-27T11:18:30.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.737+0000 7ff65c9dc700 1 -- 172.21.15.112:0/575201950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff658151060 msgr2=0x7ff6580c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.737+0000 7ff65c9dc700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff658151060 0x7ff6580c70d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff65806e590 tx=0x7ff650004ad0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.737+0000 7ff65c9dc700 1 -- 172.21.15.112:0/575201950 shutdown_connections 2023-08-27T11:18:30.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.737+0000 7ff65c9dc700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff63806a0e0 0x7ff63806c5a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.737+0000 7ff65c9dc700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff658151060 0x7ff6580c70d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.737+0000 7ff65c9dc700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65806adb0 0x7ff658070870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.737+0000 7ff65c9dc700 1 --2- 172.21.15.112:0/575201950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6580cdbd0 0x7ff658071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.737+0000 7ff65c9dc700 1 -- 172.21.15.112:0/575201950 >> 172.21.15.112:0/575201950 conn(0x7ff6580d0000 msgr2=0x7ff6580d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:30.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.737+0000 7ff65c9dc700 1 -- 172.21.15.112:0/575201950 shutdown_connections 2023-08-27T11:18:30.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.737+0000 7ff65c9dc700 1 -- 172.21.15.112:0/575201950 wait complete. 2023-08-27T11:18:30.743 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:18:30.744 INFO:tasks.ceph.mds.b.smithi138.stderr: -3677> 2023-08-27T11:18:06.919+0000 7fb7a80b6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:18:30.751 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:18:30.771 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:18:30.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.879+0000 7f8e50c66700 1 Processor -- start 2023-08-27T11:18:30.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.879+0000 7f8e50c66700 1 -- start start 2023-08-27T11:18:30.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.880+0000 7f8e50c66700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8e4c15e190 0x7f8e4c15e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.880+0000 7f8e50c66700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8e4c0cc980 0x7f8e4c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.880+0000 7f8e50c66700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4c0b46c0 con 0x7f8e4c161dd0 2023-08-27T11:18:30.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.880+0000 7f8e50c66700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4c0cd370 con 0x7f8e4c15e190 2023-08-27T11:18:30.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.880+0000 7f8e50c66700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4c0b44d0 con 0x7f8e4c0cc980 2023-08-27T11:18:30.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.880+0000 7f8e4a59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8e4c15e190 0x7f8e4c15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.880+0000 7f8e4a59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8e4c15e190 0x7f8e4c15e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57556/0 (socket says 172.21.15.112:57556) 2023-08-27T11:18:30.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.880+0000 7f8e4a59c700 1 -- 172.21.15.112:0/2195284836 learned_addr learned my addr 172.21.15.112:0/2195284836 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:30.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.880+0000 7f8e4ad9d700 1 --2- 172.21.15.112:0/2195284836 >> v2:172.21.15.138:3300/0 conn(0x7f8e4c0cc980 0x7f8e4c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e4a59c700 1 -- 172.21.15.112:0/2195284836 >> v2:172.21.15.138:3300/0 conn(0x7f8e4c0cc980 msgr2=0x7f8e4c0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:30.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e4a59c700 1 --2- 172.21.15.112:0/2195284836 >> v2:172.21.15.138:3300/0 conn(0x7f8e4c0cc980 0x7f8e4c0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e4a59c700 1 -- 172.21.15.112:0/2195284836 >> v1:172.21.15.138:6789/0 conn(0x7f8e4c161dd0 legacy=0x7f8e4c0b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e4a59c700 1 -- 172.21.15.112:0/2195284836 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c0b49a0 con 0x7f8e4c15e190 2023-08-27T11:18:30.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e4a59c700 1 --2- 172.21.15.112:0/2195284836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e4c15e190 0x7f8e4c15e570 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c1581b0 tx=0x7f8e40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6f758e9762a3bb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:30.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e4959a700 1 -- 172.21.15.112:0/2195284836 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40011830 con 0x7f8e4c15e190 2023-08-27T11:18:30.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e4959a700 1 -- 172.21.15.112:0/2195284836 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e40011990 con 0x7f8e4c15e190 2023-08-27T11:18:30.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e4959a700 1 -- 172.21.15.112:0/2195284836 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40011d20 con 0x7f8e4c15e190 2023-08-27T11:18:30.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e50c66700 1 -- 172.21.15.112:0/2195284836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e4c15e190 msgr2=0x7f8e4c15e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e50c66700 1 --2- 172.21.15.112:0/2195284836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e4c15e190 0x7f8e4c15e570 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c1581b0 tx=0x7f8e40009bd0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e50c66700 1 -- 172.21.15.112:0/2195284836 shutdown_connections 2023-08-27T11:18:30.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e50c66700 1 --2- 172.21.15.112:0/2195284836 >> v2:172.21.15.138:3300/0 conn(0x7f8e4c0cc980 0x7f8e4c0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e50c66700 1 --2- 172.21.15.112:0/2195284836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e4c15e190 0x7f8e4c15e570 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e50c66700 1 -- 172.21.15.112:0/2195284836 >> 172.21.15.112:0/2195284836 conn(0x7f8e4c06bbf0 msgr2=0x7f8e4c0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:30.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e50c66700 1 -- 172.21.15.112:0/2195284836 shutdown_connections 2023-08-27T11:18:30.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.881+0000 7f8e50c66700 1 -- 172.21.15.112:0/2195284836 wait complete. 2023-08-27T11:18:30.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.883+0000 7f8e50c66700 1 Processor -- start 2023-08-27T11:18:30.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.883+0000 7f8e50c66700 1 -- start start 2023-08-27T11:18:30.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e50c66700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e4c0cc980 0x7f8e4c1f3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e50c66700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e4c15e190 0x7f8e4c154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e50c66700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e4c161dd0 0x7f8e4c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e50c66700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c071280 con 0x7f8e4c15e190 2023-08-27T11:18:30.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e50c66700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c071560 con 0x7f8e4c161dd0 2023-08-27T11:18:30.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e50c66700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c071840 con 0x7f8e4c0cc980 2023-08-27T11:18:30.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e49d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e4c15e190 0x7f8e4c154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e49d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e4c15e190 0x7f8e4c154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57564/0 (socket says 172.21.15.112:57564) 2023-08-27T11:18:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e49d9b700 1 -- 172.21.15.112:0/2656815594 learned_addr learned my addr 172.21.15.112:0/2656815594 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e4ad9d700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e4c161dd0 0x7f8e4c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e4a59c700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e4c0cc980 0x7f8e4c1f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e49d9b700 1 -- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e4c0cc980 msgr2=0x7f8e4c1f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e49d9b700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e4c0cc980 0x7f8e4c1f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e49d9b700 1 -- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e4c161dd0 msgr2=0x7f8e4c0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:30.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e49d9b700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e4c161dd0 0x7f8e4c0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:30.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e49d9b700 1 -- 172.21.15.112:0/2656815594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e40009550 con 0x7f8e4c15e190 2023-08-27T11:18:30.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e49d9b700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e4c15e190 0x7f8e4c154c80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00c450 tx=0x7f8e3c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:30.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e3affd700 1 -- 172.21.15.112:0/2656815594 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c0052d0 con 0x7f8e4c15e190 2023-08-27T11:18:30.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e3affd700 1 -- 172.21.15.112:0/2656815594 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c005430 con 0x7f8e4c15e190 2023-08-27T11:18:30.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e50c66700 1 -- 172.21.15.112:0/2656815594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c071b20 con 0x7f8e4c15e190 2023-08-27T11:18:30.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e3affd700 1 -- 172.21.15.112:0/2656815594 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c0038c0 con 0x7f8e4c15e190 2023-08-27T11:18:30.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.884+0000 7f8e50c66700 1 -- 172.21.15.112:0/2656815594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c072320 con 0x7f8e4c15e190 2023-08-27T11:18:30.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.885+0000 7f8e50c66700 1 -- 172.21.15.112:0/2656815594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c066fa0 con 0x7f8e4c15e190 2023-08-27T11:18:30.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.886+0000 7f8e3affd700 1 -- 172.21.15.112:0/2656815594 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8e3c003a20 con 0x7f8e4c15e190 2023-08-27T11:18:30.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.886+0000 7f8e3affd700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8e2c06a0e0 0x7f8e2c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:30.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.887+0000 7f8e3affd700 1 -- 172.21.15.112:0/2656815594 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f8e3c087930 con 0x7f8e4c15e190 2023-08-27T11:18:30.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.887+0000 7f8e4a59c700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8e2c06a0e0 0x7f8e2c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:30.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.887+0000 7f8e4a59c700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8e2c06a0e0 0x7f8e2c06c5a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009c10 tx=0x7f8e40008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:30.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:30.891+0000 7f8e3affd700 1 -- 172.21.15.112:0/2656815594 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8e3c052f60 con 0x7f8e4c15e190 2023-08-27T11:18:31.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.121+0000 7f8e50c66700 1 -- 172.21.15.112:0/2656815594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8e4c072040 con 0x7f8e4c15e190 2023-08-27T11:18:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.736+0000 7f8e3affd700 1 -- 172.21.15.112:0/2656815594 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8e4c072040 con 0x7f8e4c15e190 2023-08-27T11:18:31.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.738+0000 7f8e50c66700 1 -- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8e2c06a0e0 msgr2=0x7f8e2c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.738+0000 7f8e50c66700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8e2c06a0e0 0x7f8e2c06c5a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009c10 tx=0x7f8e40008030 comp rx=0 tx=0).stop 2023-08-27T11:18:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.738+0000 7f8e50c66700 1 -- 172.21.15.112:0/2656815594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e4c15e190 msgr2=0x7f8e4c154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.738+0000 7f8e50c66700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e4c15e190 0x7f8e4c154c80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00c450 tx=0x7f8e3c00c760 comp rx=0 tx=0).stop 2023-08-27T11:18:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.738+0000 7f8e50c66700 1 -- 172.21.15.112:0/2656815594 shutdown_connections 2023-08-27T11:18:31.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.738+0000 7f8e50c66700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8e2c06a0e0 0x7f8e2c06c5a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:31.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.738+0000 7f8e50c66700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e4c161dd0 0x7f8e4c0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:31.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.738+0000 7f8e50c66700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e4c0cc980 0x7f8e4c1f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:31.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.738+0000 7f8e50c66700 1 --2- 172.21.15.112:0/2656815594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e4c15e190 0x7f8e4c154c80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:31.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.738+0000 7f8e50c66700 1 -- 172.21.15.112:0/2656815594 >> 172.21.15.112:0/2656815594 conn(0x7f8e4c06bbf0 msgr2=0x7f8e4c0b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:31.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.738+0000 7f8e50c66700 1 -- 172.21.15.112:0/2656815594 shutdown_connections 2023-08-27T11:18:31.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.738+0000 7f8e50c66700 1 -- 172.21.15.112:0/2656815594 wait complete. 2023-08-27T11:18:31.754 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:18:31.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.882+0000 7f9b481d8700 1 Processor -- start 2023-08-27T11:18:31.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.882+0000 7f9b481d8700 1 -- start start 2023-08-27T11:18:31.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b481d8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9b40157ae0 0x7f9b4015e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:31.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b481d8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9b4006f790 0x7f9b4006fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:31.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b481d8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b40161390 con 0x7f9b400b3de0 2023-08-27T11:18:31.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b481d8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b400b1aa0 con 0x7f9b4006f790 2023-08-27T11:18:31.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b481d8700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b40070160 con 0x7f9b40157ae0 2023-08-27T11:18:31.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b46775700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9b4006f790 0x7f9b4006fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:31.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b45773700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9b400b3de0 0x7f9b40161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51954/0 (socket says 172.21.15.112:51954) 2023-08-27T11:18:31.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b45773700 1 -- 172.21.15.112:0/464196049 learned_addr learned my addr 172.21.15.112:0/464196049 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:31.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b46775700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9b4006f790 0x7f9b4006fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57574/0 (socket says 172.21.15.112:57574) 2023-08-27T11:18:31.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b46775700 1 -- 172.21.15.112:0/464196049 >> v2:172.21.15.138:3300/0 conn(0x7f9b40157ae0 msgr2=0x7f9b4015e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:31.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b46775700 1 --2- 172.21.15.112:0/464196049 >> v2:172.21.15.138:3300/0 conn(0x7f9b40157ae0 0x7f9b4015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:31.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b46775700 1 -- 172.21.15.112:0/464196049 >> v1:172.21.15.112:6789/0 conn(0x7f9b400b3de0 legacy=0x7f9b40161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:31.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b44f72700 1 -- 172.21.15.112:0/464196049 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240868999 0 0) 0x7f9b40161390 con 0x7f9b400b3de0 2023-08-27T11:18:31.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b46775700 1 -- 172.21.15.112:0/464196049 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b400b35f0 con 0x7f9b4006f790 2023-08-27T11:18:31.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.883+0000 7f9b46775700 1 --2- 172.21.15.112:0/464196049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b4006f790 0x7f9b4006fc20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9b4006e580 tx=0x7f9b3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c617ee6d81f04fd6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:31.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.884+0000 7f9b44f72700 1 -- 172.21.15.112:0/464196049 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3800e070 con 0x7f9b4006f790 2023-08-27T11:18:31.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.884+0000 7f9b44f72700 1 -- 172.21.15.112:0/464196049 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3800bbd0 con 0x7f9b4006f790 2023-08-27T11:18:31.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.884+0000 7f9b44f72700 1 -- 172.21.15.112:0/464196049 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38012430 con 0x7f9b4006f790 2023-08-27T11:18:31.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.884+0000 7f9b481d8700 1 -- 172.21.15.112:0/464196049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b4006f790 msgr2=0x7f9b4006fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:31.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.884+0000 7f9b481d8700 1 --2- 172.21.15.112:0/464196049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b4006f790 0x7f9b4006fc20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9b4006e580 tx=0x7f9b3800b330 comp rx=0 tx=0).stop 2023-08-27T11:18:31.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.884+0000 7f9b481d8700 1 -- 172.21.15.112:0/464196049 shutdown_connections 2023-08-27T11:18:31.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.884+0000 7f9b481d8700 1 --2- 172.21.15.112:0/464196049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b4006f790 0x7f9b4006fc20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:31.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.884+0000 7f9b481d8700 1 --2- 172.21.15.112:0/464196049 >> v2:172.21.15.138:3300/0 conn(0x7f9b40157ae0 0x7f9b4015e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:31.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.884+0000 7f9b481d8700 1 -- 172.21.15.112:0/464196049 >> 172.21.15.112:0/464196049 conn(0x7f9b400b1be0 msgr2=0x7f9b400b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:31.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.884+0000 7f9b481d8700 1 -- 172.21.15.112:0/464196049 shutdown_connections 2023-08-27T11:18:31.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.884+0000 7f9b481d8700 1 -- 172.21.15.112:0/464196049 wait complete. 2023-08-27T11:18:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.885+0000 7f9b481d8700 1 Processor -- start 2023-08-27T11:18:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.885+0000 7f9b481d8700 1 -- start start 2023-08-27T11:18:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b481d8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b4006f790 0x7f9b40157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b481d8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b400b3de0 0x7f9b4015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b481d8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b40157ae0 0x7f9b401f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b481d8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b40158400 con 0x7f9b4006f790 2023-08-27T11:18:31.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b481d8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b40158660 con 0x7f9b40157ae0 2023-08-27T11:18:31.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b481d8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b40158940 con 0x7f9b400b3de0 2023-08-27T11:18:31.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b45f74700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b4006f790 0x7f9b40157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:31.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b45f74700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b4006f790 0x7f9b40157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57576/0 (socket says 172.21.15.112:57576) 2023-08-27T11:18:31.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b45f74700 1 -- 172.21.15.112:0/1358658149 learned_addr learned my addr 172.21.15.112:0/1358658149 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:31.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b46775700 1 --2- 172.21.15.112:0/1358658149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b40157ae0 0x7f9b401f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b45f74700 1 -- 172.21.15.112:0/1358658149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b400b3de0 msgr2=0x7f9b4015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b45f74700 1 --2- 172.21.15.112:0/1358658149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b400b3de0 0x7f9b4015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b45f74700 1 -- 172.21.15.112:0/1358658149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b40157ae0 msgr2=0x7f9b401f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b45f74700 1 --2- 172.21.15.112:0/1358658149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b40157ae0 0x7f9b401f2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.886+0000 7f9b45f74700 1 -- 172.21.15.112:0/1358658149 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b38009a70 con 0x7f9b4006f790 2023-08-27T11:18:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.887+0000 7f9b45f74700 1 --2- 172.21.15.112:0/1358658149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b4006f790 0x7f9b40157ec0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000b4a0 tx=0x7f9b3000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.887+0000 7f9b367fc700 1 -- 172.21.15.112:0/1358658149 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b30011830 con 0x7f9b4006f790 2023-08-27T11:18:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.887+0000 7f9b367fc700 1 -- 172.21.15.112:0/1358658149 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b30011990 con 0x7f9b4006f790 2023-08-27T11:18:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.887+0000 7f9b481d8700 1 -- 172.21.15.112:0/1358658149 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b40158c20 con 0x7f9b4006f790 2023-08-27T11:18:31.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.887+0000 7f9b367fc700 1 -- 172.21.15.112:0/1358658149 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b30011d40 con 0x7f9b4006f790 2023-08-27T11:18:31.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.887+0000 7f9b481d8700 1 -- 172.21.15.112:0/1358658149 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b401591e0 con 0x7f9b4006f790 2023-08-27T11:18:31.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.888+0000 7f9b481d8700 1 -- 172.21.15.112:0/1358658149 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b40067a20 con 0x7f9b4006f790 2023-08-27T11:18:31.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.888+0000 7f9b367fc700 1 -- 172.21.15.112:0/1358658149 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9b3001a400 con 0x7f9b4006f790 2023-08-27T11:18:31.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.889+0000 7f9b367fc700 1 --2- 172.21.15.112:0/1358658149 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b2406a0e0 0x7f9b2406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:31.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.889+0000 7f9b367fc700 1 -- 172.21.15.112:0/1358658149 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f9b300885d0 con 0x7f9b4006f790 2023-08-27T11:18:31.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.889+0000 7f9b45773700 1 --2- 172.21.15.112:0/1358658149 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b2406a0e0 0x7f9b2406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:31.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.889+0000 7f9b45773700 1 --2- 172.21.15.112:0/1358658149 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b2406a0e0 0x7f9b2406c5a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0011e0 tx=0x7f9b3c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:31.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:31.893+0000 7f9b367fc700 1 -- 172.21.15.112:0/1358658149 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9b30053cd0 con 0x7f9b4006f790 2023-08-27T11:18:32.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.080+0000 7f9b481d8700 1 -- 172.21.15.112:0/1358658149 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9b40160e60 con 0x7f9b4006f790 2023-08-27T11:18:32.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.849+0000 7f9b367fc700 1 -- 172.21.15.112:0/1358658149 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9b300572f0 con 0x7f9b4006f790 2023-08-27T11:18:32.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.852+0000 7f9b481d8700 1 -- 172.21.15.112:0/1358658149 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b2406a0e0 msgr2=0x7f9b2406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:32.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.852+0000 7f9b481d8700 1 --2- 172.21.15.112:0/1358658149 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b2406a0e0 0x7f9b2406c5a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0011e0 tx=0x7f9b3c0079b0 comp rx=0 tx=0).stop 2023-08-27T11:18:32.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.852+0000 7f9b481d8700 1 -- 172.21.15.112:0/1358658149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b4006f790 msgr2=0x7f9b40157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:32.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.852+0000 7f9b481d8700 1 --2- 172.21.15.112:0/1358658149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b4006f790 0x7f9b40157ec0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000b4a0 tx=0x7f9b3000b860 comp rx=0 tx=0).stop 2023-08-27T11:18:32.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.852+0000 7f9b481d8700 1 -- 172.21.15.112:0/1358658149 shutdown_connections 2023-08-27T11:18:32.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.852+0000 7f9b481d8700 1 --2- 172.21.15.112:0/1358658149 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9b2406a0e0 0x7f9b2406c5a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:32.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.852+0000 7f9b481d8700 1 --2- 172.21.15.112:0/1358658149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b40157ae0 0x7f9b401f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:32.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.852+0000 7f9b481d8700 1 --2- 172.21.15.112:0/1358658149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b4006f790 0x7f9b40157ec0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:32.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.852+0000 7f9b481d8700 1 --2- 172.21.15.112:0/1358658149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b400b3de0 0x7f9b4015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:32.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.852+0000 7f9b481d8700 1 -- 172.21.15.112:0/1358658149 >> 172.21.15.112:0/1358658149 conn(0x7f9b400b1be0 msgr2=0x7f9b4015ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:32.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.852+0000 7f9b481d8700 1 -- 172.21.15.112:0/1358658149 shutdown_connections 2023-08-27T11:18:32.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.852+0000 7f9b481d8700 1 -- 172.21.15.112:0/1358658149 wait complete. 2023-08-27T11:18:32.860 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:18:32.867 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:18:32.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.992+0000 7f7ca013a700 1 Processor -- start 2023-08-27T11:18:32.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.993+0000 7f7ca013a700 1 -- start start 2023-08-27T11:18:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7ca013a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7c98146000 0x7f7c980d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7ca013a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c9815ed20 0x7f7c980d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7ca013a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c98070c10 con 0x7f7c98146760 2023-08-27T11:18:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7ca013a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c980d03d0 con 0x7f7c98146000 2023-08-27T11:18:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7ca013a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980d1790 con 0x7f7c9815ed20 2023-08-27T11:18:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9d6d5700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7c98146760 0x7f7c98161050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51966/0 (socket says 172.21.15.112:51966) 2023-08-27T11:18:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9d6d5700 1 -- 172.21.15.112:0/1739533107 learned_addr learned my addr 172.21.15.112:0/1739533107 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832381746 0 0) 0x7f7c98070c10 con 0x7f7c98146760 2023-08-27T11:18:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c840035b0 con 0x7f7c98146760 2023-08-27T11:18:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9e6d7700 1 --2- 172.21.15.112:0/1739533107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c9815ed20 0x7f7c980d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ded6700 1 --2- 172.21.15.112:0/1739533107 >> v2:172.21.15.138:3300/0 conn(0x7f7c98146000 0x7f7c980d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3611104041 0 0) 0x7f7c840035b0 con 0x7f7c98146760 2023-08-27T11:18:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c9815ed20 msgr2=0x7f7c980d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 --2- 172.21.15.112:0/1739533107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c9815ed20 0x7f7c980d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 >> v2:172.21.15.138:3300/0 conn(0x7f7c98146000 msgr2=0x7f7c980d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 --2- 172.21.15.112:0/1739533107 >> v2:172.21.15.138:3300/0 conn(0x7f7c98146000 0x7f7c980d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c98070ef0 con 0x7f7c98146760 2023-08-27T11:18:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f7c8c001670 con 0x7f7c98146760 2023-08-27T11:18:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 >> v1:172.21.15.112:6789/0 conn(0x7f7c98146760 legacy=0x7f7c98161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 --2- 172.21.15.112:0/1739533107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c84003a00 0x7f7c84003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 --2- 172.21.15.112:0/1739533107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c9815ed20 0x7f7c84004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 --2- 172.21.15.112:0/1739533107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c84004d90 0x7f7c84005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980d03d0 con 0x7f7c84004d90 2023-08-27T11:18:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8400bac0 con 0x7f7c84003a00 2023-08-27T11:18:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8400bc30 con 0x7f7c9815ed20 2023-08-27T11:18:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9e6d7700 1 --2- 172.21.15.112:0/1739533107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c84004d90 0x7f7c84005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.994+0000 7f7c9d6d5700 1 --2- 172.21.15.112:0/1739533107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c9815ed20 0x7f7c84004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7c9ded6700 1 --2- 172.21.15.112:0/1739533107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c84003a00 0x7f7c84003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7c9e6d7700 1 -- 172.21.15.112:0/1739533107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c9815ed20 msgr2=0x7f7c84004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7c9e6d7700 1 --2- 172.21.15.112:0/1739533107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c9815ed20 0x7f7c84004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7c9e6d7700 1 -- 172.21.15.112:0/1739533107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c84003a00 msgr2=0x7f7c84003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7c9e6d7700 1 --2- 172.21.15.112:0/1739533107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c84003a00 0x7f7c84003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7c9e6d7700 1 -- 172.21.15.112:0/1739533107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c840035b0 con 0x7f7c84004d90 2023-08-27T11:18:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7c9ded6700 1 --2- 172.21.15.112:0/1739533107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c84003a00 0x7f7c84003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7c9e6d7700 1 --2- 172.21.15.112:0/1739533107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c84004d90 0x7f7c84005210 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7c90007360 tx=0x7f7c84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639341bfa337ae3d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90007e20 con 0x7f7c84004d90 2023-08-27T11:18:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c90014400 con 0x7f7c84004d90 2023-08-27T11:18:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7c9ced4700 1 -- 172.21.15.112:0/1739533107 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c900147b0 con 0x7f7c84004d90 2023-08-27T11:18:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7ca013a700 1 -- 172.21.15.112:0/1739533107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c84004d90 msgr2=0x7f7c84005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:33.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7ca013a700 1 --2- 172.21.15.112:0/1739533107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c84004d90 0x7f7c84005210 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7c90007360 tx=0x7f7c84005880 comp rx=0 tx=0).stop 2023-08-27T11:18:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7c9e6d7700 1 -- 172.21.15.112:0/1739533107 reap_dead start 2023-08-27T11:18:33.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7ca013a700 1 -- 172.21.15.112:0/1739533107 shutdown_connections 2023-08-27T11:18:33.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.995+0000 7f7ca013a700 1 -- 172.21.15.112:0/1739533107 >> 172.21.15.112:0/1739533107 conn(0x7f7c9806c8d0 msgr2=0x7f7c9806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:33.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.996+0000 7f7ca013a700 1 -- 172.21.15.112:0/1739533107 shutdown_connections 2023-08-27T11:18:33.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.996+0000 7f7ca013a700 1 -- 172.21.15.112:0/1739533107 wait complete. 2023-08-27T11:18:33.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.997+0000 7f7ca013a700 1 Processor -- start 2023-08-27T11:18:33.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.997+0000 7f7ca013a700 1 -- start start 2023-08-27T11:18:33.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7ca013a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c84004d90 0x7f7c9815bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:33.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7ca013a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c9815dad0 0x7f7c9815df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:33.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7ca013a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c980b3420 0x7f7c980b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:33.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7ca013a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9815e490 con 0x7f7c980b3420 2023-08-27T11:18:33.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7ca013a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980b3df0 con 0x7f7c9815dad0 2023-08-27T11:18:33.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7ca013a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9813fc60 con 0x7f7c84004d90 2023-08-27T11:18:33.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c9e6d7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c980b3420 0x7f7c980b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:33.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c9e6d7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c980b3420 0x7f7c980b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57600/0 (socket says 172.21.15.112:57600) 2023-08-27T11:18:33.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c9e6d7700 1 -- 172.21.15.112:0/3209253159 learned_addr learned my addr 172.21.15.112:0/3209253159 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:33.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c9d6d5700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c9815dad0 0x7f7c9815df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:33.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c9ded6700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c84004d90 0x7f7c9815bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:33.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c9e6d7700 1 -- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c84004d90 msgr2=0x7f7c9815bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:33.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c9e6d7700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c84004d90 0x7f7c9815bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:33.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c9e6d7700 1 -- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c9815dad0 msgr2=0x7f7c9815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:33.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c9e6d7700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c9815dad0 0x7f7c9815df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:33.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c9e6d7700 1 -- 172.21.15.112:0/3209253159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9815c0c0 con 0x7f7c980b3420 2023-08-27T11:18:33.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c9d6d5700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c9815dad0 0x7f7c9815df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:33.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c9ded6700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c84004d90 0x7f7c9815bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:33.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c9e6d7700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c980b3420 0x7f7c980b38b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7c90014000 tx=0x7f7c90004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:33.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c8a7fc700 1 -- 172.21.15.112:0/3209253159 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90007000 con 0x7f7c980b3420 2023-08-27T11:18:33.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c8a7fc700 1 -- 172.21.15.112:0/3209253159 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9001e410 con 0x7f7c980b3420 2023-08-27T11:18:33.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7ca013a700 1 -- 172.21.15.112:0/3209253159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c980b4000 con 0x7f7c980b3420 2023-08-27T11:18:33.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7c8a7fc700 1 -- 172.21.15.112:0/3209253159 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9001e7c0 con 0x7f7c980b3420 2023-08-27T11:18:33.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.998+0000 7f7ca013a700 1 -- 172.21.15.112:0/3209253159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c980b45c0 con 0x7f7c980b3420 2023-08-27T11:18:33.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:32.999+0000 7f7ca013a700 1 -- 172.21.15.112:0/3209253159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c98067a20 con 0x7f7c980b3420 2023-08-27T11:18:33.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.000+0000 7f7c8a7fc700 1 -- 172.21.15.112:0/3209253159 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7c9001b050 con 0x7f7c980b3420 2023-08-27T11:18:33.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.000+0000 7f7c8a7fc700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c7c06a0e0 0x7f7c7c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:33.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.000+0000 7f7c8a7fc700 1 -- 172.21.15.112:0/3209253159 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f7c90089a10 con 0x7f7c980b3420 2023-08-27T11:18:33.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.000+0000 7f7c9ded6700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c7c06a0e0 0x7f7c7c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:33.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.001+0000 7f7c9ded6700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c7c06a0e0 0x7f7c7c06c5a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7c980b5cd0 tx=0x7f7c94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:33.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.004+0000 7f7c8a7fc700 1 -- 172.21.15.112:0/3209253159 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7c900553b0 con 0x7f7c980b3420 2023-08-27T11:18:33.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.192+0000 7f7ca013a700 1 -- 172.21.15.112:0/3209253159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7c9815e490 con 0x7f7c980b3420 2023-08-27T11:18:33.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.855+0000 7f7c8a7fc700 1 -- 172.21.15.112:0/3209253159 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7c9002e0d0 con 0x7f7c980b3420 2023-08-27T11:18:33.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.858+0000 7f7ca013a700 1 -- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c7c06a0e0 msgr2=0x7f7c7c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:33.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.858+0000 7f7ca013a700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c7c06a0e0 0x7f7c7c06c5a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7c980b5cd0 tx=0x7f7c94008040 comp rx=0 tx=0).stop 2023-08-27T11:18:33.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.858+0000 7f7ca013a700 1 -- 172.21.15.112:0/3209253159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c980b3420 msgr2=0x7f7c980b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.858+0000 7f7ca013a700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c980b3420 0x7f7c980b38b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7c90014000 tx=0x7f7c90004b40 comp rx=0 tx=0).stop 2023-08-27T11:18:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.858+0000 7f7ca013a700 1 -- 172.21.15.112:0/3209253159 shutdown_connections 2023-08-27T11:18:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.858+0000 7f7ca013a700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c7c06a0e0 0x7f7c7c06c5a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.858+0000 7f7ca013a700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c980b3420 0x7f7c980b38b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.858+0000 7f7ca013a700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c84004d90 0x7f7c9815bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.858+0000 7f7ca013a700 1 --2- 172.21.15.112:0/3209253159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c9815dad0 0x7f7c9815df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.858+0000 7f7ca013a700 1 -- 172.21.15.112:0/3209253159 >> 172.21.15.112:0/3209253159 conn(0x7f7c9806c8d0 msgr2=0x7f7c9815c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.858+0000 7f7ca013a700 1 -- 172.21.15.112:0/3209253159 shutdown_connections 2023-08-27T11:18:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.858+0000 7f7ca013a700 1 -- 172.21.15.112:0/3209253159 wait complete. 2023-08-27T11:18:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2023-08-27T11:18:33.874 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:18:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.996+0000 7ff3ae2a9700 1 Processor -- start 2023-08-27T11:18:33.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.996+0000 7ff3ae2a9700 1 -- start start 2023-08-27T11:18:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3ae2a9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff3a806fea0 0x7ff3a80b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3ae2a9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a80b51d0 0x7ff3a80ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3ae2a9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a80b59e0 con 0x7ff3a8143f20 2023-08-27T11:18:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3ae2a9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a80b5cc0 con 0x7ff3a806fea0 2023-08-27T11:18:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3ae2a9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80b5fa0 con 0x7ff3a80b51d0 2023-08-27T11:18:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3a77fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff3a806fea0 0x7ff3a80b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3a77fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff3a806fea0 0x7ff3a80b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:58032/0 (socket says 172.21.15.112:58032) 2023-08-27T11:18:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3a7fff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff3a8143f20 0x7ff3a806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:59510/0 (socket says 172.21.15.112:59510) 2023-08-27T11:18:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3ac846700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a80b51d0 0x7ff3a80ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3a77fe700 1 -- 172.21.15.112:0/1814440620 learned_addr learned my addr 172.21.15.112:0/1814440620 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3ac846700 1 -- 172.21.15.112:0/1814440620 >> v2:172.21.15.138:3300/0 conn(0x7ff3a806fea0 msgr2=0x7ff3a80b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3ac846700 1 --2- 172.21.15.112:0/1814440620 >> v2:172.21.15.138:3300/0 conn(0x7ff3a806fea0 0x7ff3a80b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3ac846700 1 -- 172.21.15.112:0/1814440620 >> v1:172.21.15.138:6789/0 conn(0x7ff3a8143f20 legacy=0x7ff3a806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3ac846700 1 -- 172.21.15.112:0/1814440620 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a80cdf10 con 0x7ff3a80b51d0 2023-08-27T11:18:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.997+0000 7ff3a77fe700 1 --2- 172.21.15.112:0/1814440620 >> v2:172.21.15.138:3300/0 conn(0x7ff3a806fea0 0x7ff3a80b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.998+0000 7ff3ac846700 1 --2- 172.21.15.112:0/1814440620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a80b51d0 0x7ff3a80ce710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0009d90 tx=0x7ff3a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ac323f597f763c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.998+0000 7ff3a6ffd700 1 -- 172.21.15.112:0/1814440620 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a000e070 con 0x7ff3a80b51d0 2023-08-27T11:18:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.998+0000 7ff3a6ffd700 1 -- 172.21.15.112:0/1814440620 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a000bb20 con 0x7ff3a80b51d0 2023-08-27T11:18:34.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.998+0000 7ff3a6ffd700 1 -- 172.21.15.112:0/1814440620 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0012430 con 0x7ff3a80b51d0 2023-08-27T11:18:34.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.999+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/1814440620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a80b51d0 msgr2=0x7ff3a80ce710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:34.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.999+0000 7ff3ae2a9700 1 --2- 172.21.15.112:0/1814440620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a80b51d0 0x7ff3a80ce710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0009d90 tx=0x7ff3a000b330 comp rx=0 tx=0).stop 2023-08-27T11:18:34.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.999+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/1814440620 shutdown_connections 2023-08-27T11:18:34.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.999+0000 7ff3ae2a9700 1 --2- 172.21.15.112:0/1814440620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a80b51d0 0x7ff3a80ce710 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.999+0000 7ff3ae2a9700 1 --2- 172.21.15.112:0/1814440620 >> v2:172.21.15.138:3300/0 conn(0x7ff3a806fea0 0x7ff3a80b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.999+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/1814440620 >> 172.21.15.112:0/1814440620 conn(0x7ff3a806bbf0 msgr2=0x7ff3a80cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:33.999+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/1814440620 shutdown_connections 2023-08-27T11:18:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.000+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/1814440620 wait complete. 2023-08-27T11:18:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.002+0000 7ff3ae2a9700 1 Processor -- start 2023-08-27T11:18:34.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.003+0000 7ff3ae2a9700 1 -- start start 2023-08-27T11:18:34.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.004+0000 7ff3ae2a9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3a806fea0 0x7ff3a8140a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.004+0000 7ff3ae2a9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3a80b51d0 0x7ff3a80cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.004+0000 7ff3ae2a9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8143f20 0x7ff3a80c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.004+0000 7ff3ae2a9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a8140f50 con 0x7ff3a80b51d0 2023-08-27T11:18:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.004+0000 7ff3ae2a9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80c60a0 con 0x7ff3a806fea0 2023-08-27T11:18:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.004+0000 7ff3ae2a9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80c6380 con 0x7ff3a8143f20 2023-08-27T11:18:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.004+0000 7ff3ac846700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8143f20 0x7ff3a80c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.004+0000 7ff3ac846700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8143f20 0x7ff3a80c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:41034/0 (socket says 172.21.15.112:41034) 2023-08-27T11:18:34.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.004+0000 7ff3ac846700 1 -- 172.21.15.112:0/4224524060 learned_addr learned my addr 172.21.15.112:0/4224524060 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:34.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.005+0000 7ff3a7fff700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3a806fea0 0x7ff3a8140a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.005+0000 7ff3ac846700 1 -- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3a806fea0 msgr2=0x7ff3a8140a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:34.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.005+0000 7ff3ac846700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3a806fea0 0x7ff3a8140a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.005+0000 7ff3ac846700 1 -- 172.21.15.112:0/4224524060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3a80b51d0 msgr2=0x7ff3a80cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:34.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.005+0000 7ff3ac846700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3a80b51d0 0x7ff3a80cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.005+0000 7ff3ac846700 1 -- 172.21.15.112:0/4224524060 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff398009550 con 0x7ff3a8143f20 2023-08-27T11:18:34.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.005+0000 7ff3ac846700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8143f20 0x7ff3a80c7140 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0014040 tx=0x7ff3a0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:34.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.005+0000 7ff3a7fff700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3a806fea0 0x7ff3a8140a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:34.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.005+0000 7ff3a4ff9700 1 -- 172.21.15.112:0/4224524060 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0012430 con 0x7ff3a8143f20 2023-08-27T11:18:34.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.005+0000 7ff3a4ff9700 1 -- 172.21.15.112:0/4224524060 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a0020070 con 0x7ff3a8143f20 2023-08-27T11:18:34.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.005+0000 7ff3a4ff9700 1 -- 172.21.15.112:0/4224524060 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0003ea0 con 0x7ff3a8143f20 2023-08-27T11:18:34.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.005+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/4224524060 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a0009a70 con 0x7ff3a8143f20 2023-08-27T11:18:34.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.006+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/4224524060 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a8144480 con 0x7ff3a8143f20 2023-08-27T11:18:34.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.007+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/4224524060 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a8066fa0 con 0x7ff3a8143f20 2023-08-27T11:18:34.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.009+0000 7ff3a4ff9700 1 -- 172.21.15.112:0/4224524060 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff3a0004260 con 0x7ff3a8143f20 2023-08-27T11:18:34.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.009+0000 7ff3a4ff9700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff39406a110 0x7ff39406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.009+0000 7ff3a4ff9700 1 -- 172.21.15.112:0/4224524060 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7ff3a0022070 con 0x7ff3a8143f20 2023-08-27T11:18:34.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.009+0000 7ff3a7fff700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff39406a110 0x7ff39406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.010+0000 7ff3a7fff700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff39406a110 0x7ff39406c5d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff39c000f60 tx=0x7ff39c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:34.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.014+0000 7ff3a4ff9700 1 -- 172.21.15.112:0/4224524060 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff3a00561e0 con 0x7ff3a8143f20 2023-08-27T11:18:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.209+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/4224524060 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff3a80c64c0 con 0x7ff3a8143f20 2023-08-27T11:18:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.220+0000 7ff3a4ff9700 1 -- 172.21.15.112:0/4224524060 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff3a80c64c0 con 0x7ff3a8143f20 2023-08-27T11:18:34.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.222+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff39406a110 msgr2=0x7ff39406c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:34.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.222+0000 7ff3ae2a9700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff39406a110 0x7ff39406c5d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff39c000f60 tx=0x7ff39c009040 comp rx=0 tx=0).stop 2023-08-27T11:18:34.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.222+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8143f20 msgr2=0x7ff3a80c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:34.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.222+0000 7ff3ae2a9700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8143f20 0x7ff3a80c7140 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0014040 tx=0x7ff3a0004ad0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.222+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/4224524060 shutdown_connections 2023-08-27T11:18:34.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.222+0000 7ff3ae2a9700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff39406a110 0x7ff39406c5d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.222+0000 7ff3ae2a9700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8143f20 0x7ff3a80c7140 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.222+0000 7ff3ae2a9700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3a806fea0 0x7ff3a8140a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.222+0000 7ff3ae2a9700 1 --2- 172.21.15.112:0/4224524060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3a80b51d0 0x7ff3a80cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.222+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/4224524060 >> 172.21.15.112:0/4224524060 conn(0x7ff3a806bbf0 msgr2=0x7ff3a80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:34.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.222+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/4224524060 shutdown_connections 2023-08-27T11:18:34.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.222+0000 7ff3ae2a9700 1 -- 172.21.15.112:0/4224524060 wait complete. 2023-08-27T11:18:34.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2023-08-27T11:18:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.355+0000 7f3a24309700 1 Processor -- start 2023-08-27T11:18:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.355+0000 7f3a24309700 1 -- start start 2023-08-27T11:18:34.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a24309700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a1c06adb0 0x7f3a1c06b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a24309700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3a1c0cdbd0 0x7f3a1c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a24309700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1c15eb60 con 0x7f3a1c151060 2023-08-27T11:18:34.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a24309700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1c06b770 con 0x7f3a1c0cdbd0 2023-08-27T11:18:34.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a24309700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0b02f0 con 0x7f3a1c06adb0 2023-08-27T11:18:34.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a220a5700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3a1c151060 0x7f3a1c15e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52000/0 (socket says 172.21.15.112:52000) 2023-08-27T11:18:34.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a220a5700 1 -- 172.21.15.112:0/1629331357 learned_addr learned my addr 172.21.15.112:0/1629331357 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:34.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a228a6700 1 --2- 172.21.15.112:0/1629331357 >> v2:172.21.15.138:3300/0 conn(0x7f3a1c0cdbd0 0x7f3a1c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a218a4700 1 --2- 172.21.15.112:0/1629331357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a1c06adb0 0x7f3a1c06b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660367137 0 0) 0x7f3a1c15eb60 con 0x7f3a1c151060 2023-08-27T11:18:34.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a080035b0 con 0x7f3a1c151060 2023-08-27T11:18:34.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2676333153 0 0) 0x7f3a080035b0 con 0x7f3a1c151060 2023-08-27T11:18:34.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a1c06adb0 msgr2=0x7f3a1c06b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 --2- 172.21.15.112:0/1629331357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a1c06adb0 0x7f3a1c06b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 >> v2:172.21.15.138:3300/0 conn(0x7f3a1c0cdbd0 msgr2=0x7f3a1c0ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 --2- 172.21.15.112:0/1629331357 >> v2:172.21.15.138:3300/0 conn(0x7f3a1c0cdbd0 0x7f3a1c0ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c0b05d0 con 0x7f3a1c151060 2023-08-27T11:18:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f3a18001670 con 0x7f3a1c151060 2023-08-27T11:18:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 >> v1:172.21.15.112:6789/0 conn(0x7f3a1c151060 legacy=0x7f3a1c15e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 --2- 172.21.15.112:0/1629331357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a08003a00 0x7f3a08003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 --2- 172.21.15.112:0/1629331357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a08004850 0x7f3a08004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 --2- 172.21.15.112:0/1629331357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a1c06adb0 0x7f3a08005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.356+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c06b770 con 0x7f3a08004850 2023-08-27T11:18:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0b02f0 con 0x7f3a08003a00 2023-08-27T11:18:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0800dc20 con 0x7f3a1c06adb0 2023-08-27T11:18:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a218a4700 1 --2- 172.21.15.112:0/1629331357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a08004850 0x7f3a08004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a220a5700 1 --2- 172.21.15.112:0/1629331357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a08003a00 0x7f3a08003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a228a6700 1 --2- 172.21.15.112:0/1629331357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a1c06adb0 0x7f3a08005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a218a4700 1 -- 172.21.15.112:0/1629331357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a1c06adb0 msgr2=0x7f3a08005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:34.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a218a4700 1 --2- 172.21.15.112:0/1629331357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a1c06adb0 0x7f3a08005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a218a4700 1 -- 172.21.15.112:0/1629331357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a08003a00 msgr2=0x7f3a08003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:34.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a218a4700 1 --2- 172.21.15.112:0/1629331357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a08003a00 0x7f3a08003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a218a4700 1 -- 172.21.15.112:0/1629331357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a080035b0 con 0x7f3a08004850 2023-08-27T11:18:34.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a220a5700 1 --2- 172.21.15.112:0/1629331357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a08003a00 0x7f3a08003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:34.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a218a4700 1 --2- 172.21.15.112:0/1629331357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a08004850 0x7f3a08004cd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009b60 tx=0x7f3a08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362c3256c689cd04 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:34.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00e6c0 con 0x7f3a08004850 2023-08-27T11:18:34.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c00e820 con 0x7f3a08004850 2023-08-27T11:18:34.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.357+0000 7f3a210a3700 1 -- 172.21.15.112:0/1629331357 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00eb90 con 0x7f3a08004850 2023-08-27T11:18:34.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.358+0000 7f3a24309700 1 -- 172.21.15.112:0/1629331357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a08004850 msgr2=0x7f3a08004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:34.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.358+0000 7f3a24309700 1 --2- 172.21.15.112:0/1629331357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a08004850 0x7f3a08004cd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009b60 tx=0x7f3a08005880 comp rx=0 tx=0).stop 2023-08-27T11:18:34.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.358+0000 7f3a228a6700 1 -- 172.21.15.112:0/1629331357 reap_dead start 2023-08-27T11:18:34.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.358+0000 7f3a24309700 1 -- 172.21.15.112:0/1629331357 shutdown_connections 2023-08-27T11:18:34.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.358+0000 7f3a24309700 1 -- 172.21.15.112:0/1629331357 >> 172.21.15.112:0/1629331357 conn(0x7f3a1c0d0000 msgr2=0x7f3a1c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:34.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.358+0000 7f3a24309700 1 -- 172.21.15.112:0/1629331357 shutdown_connections 2023-08-27T11:18:34.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.358+0000 7f3a24309700 1 -- 172.21.15.112:0/1629331357 wait complete. 2023-08-27T11:18:34.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.359+0000 7f3a24309700 1 Processor -- start 2023-08-27T11:18:34.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.359+0000 7f3a24309700 1 -- start start 2023-08-27T11:18:34.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a24309700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a08004850 0x7f3a1c1489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a24309700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1c0ca000 0x7f3a1c1476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a24309700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a1c150070 0x7f3a1c1504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a24309700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c150a30 con 0x7f3a1c0ca000 2023-08-27T11:18:34.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a24309700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c070660 con 0x7f3a08004850 2023-08-27T11:18:34.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a24309700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c1491f0 con 0x7f3a1c150070 2023-08-27T11:18:34.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a218a4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1c0ca000 0x7f3a1c1476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a218a4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1c0ca000 0x7f3a1c1476c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57632/0 (socket says 172.21.15.112:57632) 2023-08-27T11:18:34.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a218a4700 1 -- 172.21.15.112:0/981926304 learned_addr learned my addr 172.21.15.112:0/981926304 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:34.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a220a5700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a08004850 0x7f3a1c1489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a228a6700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a1c150070 0x7f3a1c1504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a218a4700 1 -- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a1c150070 msgr2=0x7f3a1c1504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:34.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a218a4700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a1c150070 0x7f3a1c1504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a218a4700 1 -- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a08004850 msgr2=0x7f3a1c1489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:34.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a218a4700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a08004850 0x7f3a1c1489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a218a4700 1 -- 172.21.15.112:0/981926304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c070900 con 0x7f3a1c0ca000 2023-08-27T11:18:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a218a4700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1c0ca000 0x7f3a1c1476c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3a080046a0 tx=0x7f3a0c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a12ffd700 1 -- 172.21.15.112:0/981926304 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c021070 con 0x7f3a1c0ca000 2023-08-27T11:18:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a12ffd700 1 -- 172.21.15.112:0/981926304 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c004470 con 0x7f3a1c0ca000 2023-08-27T11:18:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a24309700 1 -- 172.21.15.112:0/981926304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1c070be0 con 0x7f3a1c0ca000 2023-08-27T11:18:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a12ffd700 1 -- 172.21.15.112:0/981926304 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00e6c0 con 0x7f3a1c0ca000 2023-08-27T11:18:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.360+0000 7f3a24309700 1 -- 172.21.15.112:0/981926304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c148ee0 con 0x7f3a1c0ca000 2023-08-27T11:18:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.361+0000 7f3a24309700 1 -- 172.21.15.112:0/981926304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1c071230 con 0x7f3a1c0ca000 2023-08-27T11:18:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.362+0000 7f3a12ffd700 1 -- 172.21.15.112:0/981926304 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3a0c025070 con 0x7f3a1c0ca000 2023-08-27T11:18:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.362+0000 7f3a12ffd700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3a0006a0e0 0x7f3a0006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.362+0000 7f3a12ffd700 1 -- 172.21.15.112:0/981926304 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f3a0c0892c0 con 0x7f3a1c0ca000 2023-08-27T11:18:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.362+0000 7f3a220a5700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3a0006a0e0 0x7f3a0006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.363+0000 7f3a220a5700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3a0006a0e0 0x7f3a0006c5a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3a180010e0 tx=0x7f3a18007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.367+0000 7f3a12ffd700 1 -- 172.21.15.112:0/981926304 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3a0c054e00 con 0x7f3a1c0ca000 2023-08-27T11:18:34.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.636+0000 7f3a24309700 1 -- 172.21.15.112:0/981926304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7f3a1c067a20 con 0x7f3a1c0ca000 2023-08-27T11:18:34.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.857+0000 7f3a12ffd700 1 -- 172.21.15.112:0/981926304 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v104) v1 ==== 135+0+0 (secure 0 0 0) 0x7f3a0c058420 con 0x7f3a1c0ca000 2023-08-27T11:18:34.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.859+0000 7f3a24309700 1 -- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3a0006a0e0 msgr2=0x7f3a0006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:34.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.859+0000 7f3a24309700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3a0006a0e0 0x7f3a0006c5a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3a180010e0 tx=0x7f3a18007040 comp rx=0 tx=0).stop 2023-08-27T11:18:34.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.859+0000 7f3a24309700 1 -- 172.21.15.112:0/981926304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1c0ca000 msgr2=0x7f3a1c1476c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:34.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.859+0000 7f3a24309700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1c0ca000 0x7f3a1c1476c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3a080046a0 tx=0x7f3a0c0047c0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.859+0000 7f3a24309700 1 -- 172.21.15.112:0/981926304 shutdown_connections 2023-08-27T11:18:34.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.859+0000 7f3a24309700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3a0006a0e0 0x7f3a0006c5a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.859+0000 7f3a24309700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a1c150070 0x7f3a1c1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.859+0000 7f3a24309700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a08004850 0x7f3a1c1489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.859+0000 7f3a24309700 1 --2- 172.21.15.112:0/981926304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1c0ca000 0x7f3a1c1476c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:34.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.859+0000 7f3a24309700 1 -- 172.21.15.112:0/981926304 >> 172.21.15.112:0/981926304 conn(0x7f3a1c0d0000 msgr2=0x7f3a1c0c62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:34.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.859+0000 7f3a24309700 1 -- 172.21.15.112:0/981926304 shutdown_connections 2023-08-27T11:18:34.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.859+0000 7f3a24309700 1 -- 172.21.15.112:0/981926304 wait complete. 2023-08-27T11:18:34.873 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up ... ok 2023-08-27T11:18:34.874 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info' command 2023-08-27T11:18:34.874 INFO:tasks.cephfs_test_runner:test_volume_info (tasks.cephfs.test_volumes.TestVolumes) 2023-08-27T11:18:34.874 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2023-08-27T11:18:34.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.993+0000 7fa1e696d700 1 Processor -- start 2023-08-27T11:18:34.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.993+0000 7fa1e696d700 1 -- start start 2023-08-27T11:18:35.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1e696d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e006dcc0 0x7fa1e00b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:35.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1e696d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e015b540 con 0x7fa1e00b3440 2023-08-27T11:18:35.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1e696d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e00af830 con 0x7fa1e015a120 2023-08-27T11:18:35.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1e696d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00afb10 con 0x7fa1e006dcc0 2023-08-27T11:18:35.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1e4f0a700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa1e00b3440 0x7fa1e00b38b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52022/0 (socket says 172.21.15.112:52022) 2023-08-27T11:18:35.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1e4f0a700 1 -- 172.21.15.112:0/2962025829 learned_addr learned my addr 172.21.15.112:0/2962025829 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:35.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1df7fe700 1 --2- 172.21.15.112:0/2962025829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e006dcc0 0x7fa1e00b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:35.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567539540 0 0) 0x7fa1e015b540 con 0x7fa1e00b3440 2023-08-27T11:18:35.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1c80035b0 con 0x7fa1e00b3440 2023-08-27T11:18:35.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 550081426 0 0) 0x7fa1c80035b0 con 0x7fa1e00b3440 2023-08-27T11:18:35.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e006dcc0 msgr2=0x7fa1e00b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:35.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 --2- 172.21.15.112:0/2962025829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e006dcc0 0x7fa1e00b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:35.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 >> v1:172.21.15.138:6789/0 conn(0x7fa1e015a120 legacy=0x7fa1e006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e00afdf0 con 0x7fa1e00b3440 2023-08-27T11:18:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fa1d80019f0 con 0x7fa1e00b3440 2023-08-27T11:18:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 >> v1:172.21.15.112:6789/0 conn(0x7fa1e00b3440 legacy=0x7fa1e00b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 --2- 172.21.15.112:0/2962025829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c8003a30 0x7fa1c8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 --2- 172.21.15.112:0/2962025829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e006dcc0 0x7fa1c8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 --2- 172.21.15.112:0/2962025829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c8004d90 0x7fa1c8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00afb10 con 0x7fa1c8003a30 2023-08-27T11:18:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c800ba10 con 0x7fa1c8004d90 2023-08-27T11:18:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.994+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c800bbd0 con 0x7fa1e006dcc0 2023-08-27T11:18:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1dffff700 1 --2- 172.21.15.112:0/2962025829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c8003a30 0x7fa1c8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1df7fe700 1 --2- 172.21.15.112:0/2962025829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e006dcc0 0x7fa1c8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1e4f0a700 1 --2- 172.21.15.112:0/2962025829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c8004d90 0x7fa1c8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1dffff700 1 -- 172.21.15.112:0/2962025829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e006dcc0 msgr2=0x7fa1c8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1dffff700 1 --2- 172.21.15.112:0/2962025829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e006dcc0 0x7fa1c8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1dffff700 1 -- 172.21.15.112:0/2962025829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c8004d90 msgr2=0x7fa1c8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1dffff700 1 --2- 172.21.15.112:0/2962025829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c8004d90 0x7fa1c8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1dffff700 1 -- 172.21.15.112:0/2962025829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c80035b0 con 0x7fa1c8003a30 2023-08-27T11:18:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1dffff700 1 --2- 172.21.15.112:0/2962025829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c8003a30 0x7fa1c8003e10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40018d0 tx=0x7fa1c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6018b0dbe9a51cc3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4012620 con 0x7fa1c8003a30 2023-08-27T11:18:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d40127a0 con 0x7fa1c8003a30 2023-08-27T11:18:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1deffd700 1 -- 172.21.15.112:0/2962025829 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4012b50 con 0x7fa1c8003a30 2023-08-27T11:18:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1e696d700 1 -- 172.21.15.112:0/2962025829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c8003a30 msgr2=0x7fa1c8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.995+0000 7fa1e696d700 1 --2- 172.21.15.112:0/2962025829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c8003a30 0x7fa1c8003e10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40018d0 tx=0x7fa1c8005880 comp rx=0 tx=0).stop 2023-08-27T11:18:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.996+0000 7fa1e4f0a700 1 -- 172.21.15.112:0/2962025829 reap_dead start 2023-08-27T11:18:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.996+0000 7fa1e696d700 1 -- 172.21.15.112:0/2962025829 shutdown_connections 2023-08-27T11:18:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.996+0000 7fa1e696d700 1 -- 172.21.15.112:0/2962025829 >> 172.21.15.112:0/2962025829 conn(0x7fa1e00cff80 msgr2=0x7fa1e00d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.996+0000 7fa1e696d700 1 -- 172.21.15.112:0/2962025829 shutdown_connections 2023-08-27T11:18:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.996+0000 7fa1e696d700 1 -- 172.21.15.112:0/2962025829 wait complete. 2023-08-27T11:18:35.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.997+0000 7fa1e696d700 1 Processor -- start 2023-08-27T11:18:35.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.997+0000 7fa1e696d700 1 -- start start 2023-08-27T11:18:35.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e696d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c8003a30 0x7fa1e0157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:35.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e696d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e01f00f0 0x7fa1e0158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:35.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e696d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1e0151e70 0x7fa1e00c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e696d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e01597d0 con 0x7fa1e0151e70 2023-08-27T11:18:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e696d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00c5bc0 con 0x7fa1c8003a30 2023-08-27T11:18:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e696d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00c5290 con 0x7fa1e01f00f0 2023-08-27T11:18:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e4f0a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1e0151e70 0x7fa1e00c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e4f0a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1e0151e70 0x7fa1e00c6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57654/0 (socket says 172.21.15.112:57654) 2023-08-27T11:18:35.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e4f0a700 1 -- 172.21.15.112:0/677535769 learned_addr learned my addr 172.21.15.112:0/677535769 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:35.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1dffff700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c8003a30 0x7fa1e0157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:35.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1df7fe700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e01f00f0 0x7fa1e0158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:35.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e4f0a700 1 -- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e01f00f0 msgr2=0x7fa1e0158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:35.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e4f0a700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e01f00f0 0x7fa1e0158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:35.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e4f0a700 1 -- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c8003a30 msgr2=0x7fa1e0157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:35.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e4f0a700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c8003a30 0x7fa1e0157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:35.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e4f0a700 1 -- 172.21.15.112:0/677535769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e00c5570 con 0x7fa1e0151e70 2023-08-27T11:18:35.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1df7fe700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e01f00f0 0x7fa1e0158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:35.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e4f0a700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1e0151e70 0x7fa1e00c6d30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00cd010 tx=0x7fa1d800e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:35.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1dcff9700 1 -- 172.21.15.112:0/677535769 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8017070 con 0x7fa1e0151e70 2023-08-27T11:18:35.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1dcff9700 1 -- 172.21.15.112:0/677535769 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d800eca0 con 0x7fa1e0151e70 2023-08-27T11:18:35.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e696d700 1 -- 172.21.15.112:0/677535769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e00c5850 con 0x7fa1e0151e70 2023-08-27T11:18:35.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1dcff9700 1 -- 172.21.15.112:0/677535769 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800ee50 con 0x7fa1e0151e70 2023-08-27T11:18:35.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.998+0000 7fa1e696d700 1 -- 172.21.15.112:0/677535769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e01593c0 con 0x7fa1e0151e70 2023-08-27T11:18:35.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:34.999+0000 7fa1e696d700 1 -- 172.21.15.112:0/677535769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e0067a20 con 0x7fa1e0151e70 2023-08-27T11:18:35.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.000+0000 7fa1dcff9700 1 -- 172.21.15.112:0/677535769 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa1d8013420 con 0x7fa1e0151e70 2023-08-27T11:18:35.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.000+0000 7fa1dcff9700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1cc06a0e0 0x7fa1cc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:35.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.000+0000 7fa1dcff9700 1 -- 172.21.15.112:0/677535769 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fa1d8088020 con 0x7fa1e0151e70 2023-08-27T11:18:35.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.000+0000 7fa1dffff700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1cc06a0e0 0x7fa1cc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:35.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.001+0000 7fa1dffff700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1cc06a0e0 0x7fa1cc06c5a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4014040 tx=0x7fa1d4005040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:35.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.004+0000 7fa1dcff9700 1 -- 172.21.15.112:0/677535769 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa1d80539b0 con 0x7fa1e0151e70 2023-08-27T11:18:35.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.274+0000 7fa1e696d700 1 -- 172.21.15.112:0/677535769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) v1 -- 0x7fa1e00c5d00 con 0x7fa1e0151e70 2023-08-27T11:18:35.637 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:18:35.650 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:18:35.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.856+0000 7fa1dcff9700 1 -- 172.21.15.112:0/677535769 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v105) v1 ==== 136+0+0 (secure 0 0 0) 0x7fa1d8056fd0 con 0x7fa1e0151e70 2023-08-27T11:18:35.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.858+0000 7fa1e696d700 1 -- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1cc06a0e0 msgr2=0x7fa1cc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:35.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.858+0000 7fa1e696d700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1cc06a0e0 0x7fa1cc06c5a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4014040 tx=0x7fa1d4005040 comp rx=0 tx=0).stop 2023-08-27T11:18:35.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.858+0000 7fa1e696d700 1 -- 172.21.15.112:0/677535769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1e0151e70 msgr2=0x7fa1e00c6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:35.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.858+0000 7fa1e696d700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1e0151e70 0x7fa1e00c6d30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00cd010 tx=0x7fa1d800e440 comp rx=0 tx=0).stop 2023-08-27T11:18:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.858+0000 7fa1e696d700 1 -- 172.21.15.112:0/677535769 shutdown_connections 2023-08-27T11:18:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.858+0000 7fa1e696d700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1cc06a0e0 0x7fa1cc06c5a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.858+0000 7fa1e696d700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1e0151e70 0x7fa1e00c6d30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.858+0000 7fa1e696d700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1c8003a30 0x7fa1e0157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.858+0000 7fa1e696d700 1 --2- 172.21.15.112:0/677535769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1e01f00f0 0x7fa1e0158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.858+0000 7fa1e696d700 1 -- 172.21.15.112:0/677535769 >> 172.21.15.112:0/677535769 conn(0x7fa1e00cff80 msgr2=0x7fa1e00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.858+0000 7fa1e696d700 1 -- 172.21.15.112:0/677535769 shutdown_connections 2023-08-27T11:18:35.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.858+0000 7fa1e696d700 1 -- 172.21.15.112:0/677535769 wait complete. 2023-08-27T11:18:35.872 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:35.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.992+0000 7f7e4b5a3700 1 Processor -- start 2023-08-27T11:18:35.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.992+0000 7f7e4b5a3700 1 -- start start 2023-08-27T11:18:35.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.993+0000 7f7e4b5a3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e441578e0 0x7f7e440b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:35.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.993+0000 7f7e4b5a3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7e440cf170 0x7f7e440d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:35.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.993+0000 7f7e4b5a3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7e44160f90 0x7f7e44161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:35.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.993+0000 7f7e4b5a3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e44161980 con 0x7f7e44160f90 2023-08-27T11:18:35.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.993+0000 7f7e4b5a3700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e440cfb40 con 0x7f7e440cf170 2023-08-27T11:18:35.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.993+0000 7f7e4b5a3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e440cfde0 con 0x7f7e441578e0 2023-08-27T11:18:35.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.993+0000 7f7e49b40700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7e44160f90 0x7f7e44161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:35.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.993+0000 7f7e49b40700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7e44160f90 0x7f7e44161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57662/0 (socket says 172.21.15.112:57662) 2023-08-27T11:18:35.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.993+0000 7f7e49b40700 1 -- 172.21.15.112:0/3806048121 learned_addr learned my addr 172.21.15.112:0/3806048121 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:35.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.994+0000 7f7e48b3e700 1 --2- 172.21.15.112:0/3806048121 >> v2:172.21.15.138:3300/0 conn(0x7f7e440cf170 0x7f7e440d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:35.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.994+0000 7f7e4933f700 1 --2- 172.21.15.112:0/3806048121 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e441578e0 0x7f7e440b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:35.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.994+0000 7f7e49b40700 1 -- 172.21.15.112:0/3806048121 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e441578e0 msgr2=0x7f7e440b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:35.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.994+0000 7f7e49b40700 1 --2- 172.21.15.112:0/3806048121 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e441578e0 0x7f7e440b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:35.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.994+0000 7f7e49b40700 1 -- 172.21.15.112:0/3806048121 >> v2:172.21.15.138:3300/0 conn(0x7f7e440cf170 msgr2=0x7f7e440d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:36.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.994+0000 7f7e49b40700 1 --2- 172.21.15.112:0/3806048121 >> v2:172.21.15.138:3300/0 conn(0x7f7e440cf170 0x7f7e440d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.994+0000 7f7e49b40700 1 -- 172.21.15.112:0/3806048121 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e440d00c0 con 0x7f7e44160f90 2023-08-27T11:18:36.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.994+0000 7f7e48b3e700 1 --2- 172.21.15.112:0/3806048121 >> v2:172.21.15.138:3300/0 conn(0x7f7e440cf170 0x7f7e440d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:36.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.994+0000 7f7e49b40700 1 --2- 172.21.15.112:0/3806048121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e44160f90 0x7f7e44161440 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7e440724f0 tx=0x7f7e3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7865f2a4f8d0b3a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:36.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.994+0000 7f7e3bfff700 1 -- 172.21.15.112:0/3806048121 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c00e070 con 0x7f7e44160f90 2023-08-27T11:18:36.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.994+0000 7f7e3bfff700 1 -- 172.21.15.112:0/3806048121 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3c00b9c0 con 0x7f7e44160f90 2023-08-27T11:18:36.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.994+0000 7f7e3bfff700 1 -- 172.21.15.112:0/3806048121 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c00bd70 con 0x7f7e44160f90 2023-08-27T11:18:36.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.995+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3806048121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e44160f90 msgr2=0x7f7e44161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.995+0000 7f7e4b5a3700 1 --2- 172.21.15.112:0/3806048121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e44160f90 0x7f7e44161440 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7e440724f0 tx=0x7f7e3c00b330 comp rx=0 tx=0).stop 2023-08-27T11:18:36.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.995+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3806048121 shutdown_connections 2023-08-27T11:18:36.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.995+0000 7f7e4b5a3700 1 --2- 172.21.15.112:0/3806048121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e44160f90 0x7f7e44161440 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.995+0000 7f7e4b5a3700 1 --2- 172.21.15.112:0/3806048121 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e441578e0 0x7f7e440b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.995+0000 7f7e4b5a3700 1 --2- 172.21.15.112:0/3806048121 >> v2:172.21.15.138:3300/0 conn(0x7f7e440cf170 0x7f7e440d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.995+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3806048121 >> 172.21.15.112:0/3806048121 conn(0x7f7e4406f970 msgr2=0x7f7e4406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:36.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.995+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3806048121 shutdown_connections 2023-08-27T11:18:36.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.995+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3806048121 wait complete. 2023-08-27T11:18:36.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.996+0000 7f7e4b5a3700 1 Processor -- start 2023-08-27T11:18:36.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.996+0000 7f7e4b5a3700 1 -- start start 2023-08-27T11:18:36.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.997+0000 7f7e4b5a3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e440cf170 0x7f7e4415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.997+0000 7f7e4b5a3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e441578e0 0x7f7e44158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.997+0000 7f7e4b5a3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e44160f90 0x7f7e44152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.997+0000 7f7e4b5a3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4415f080 con 0x7f7e441578e0 2023-08-27T11:18:36.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.997+0000 7f7e4b5a3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e440b1b20 con 0x7f7e44160f90 2023-08-27T11:18:36.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.997+0000 7f7e4b5a3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e440b1e00 con 0x7f7e440cf170 2023-08-27T11:18:36.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.997+0000 7f7e48b3e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e441578e0 0x7f7e44158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.997+0000 7f7e48b3e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e441578e0 0x7f7e44158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57666/0 (socket says 172.21.15.112:57666) 2023-08-27T11:18:36.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.997+0000 7f7e48b3e700 1 -- 172.21.15.112:0/3093754735 learned_addr learned my addr 172.21.15.112:0/3093754735 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:36.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.997+0000 7f7e49b40700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e44160f90 0x7f7e44152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.997+0000 7f7e4933f700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e440cf170 0x7f7e4415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.998+0000 7f7e48b3e700 1 -- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e440cf170 msgr2=0x7f7e4415a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.998+0000 7f7e48b3e700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e440cf170 0x7f7e4415a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.998+0000 7f7e48b3e700 1 -- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e44160f90 msgr2=0x7f7e44152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.998+0000 7f7e48b3e700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e44160f90 0x7f7e44152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.998+0000 7f7e48b3e700 1 -- 172.21.15.112:0/3093754735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c009b30 con 0x7f7e441578e0 2023-08-27T11:18:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.998+0000 7f7e48b3e700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e441578e0 0x7f7e44158680 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c740 tx=0x7f7e4000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.998+0000 7f7e39ffb700 1 -- 172.21.15.112:0/3093754735 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e40011800 con 0x7f7e441578e0 2023-08-27T11:18:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.998+0000 7f7e39ffb700 1 -- 172.21.15.112:0/3093754735 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e40011960 con 0x7f7e441578e0 2023-08-27T11:18:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.998+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3093754735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e440b20e0 con 0x7f7e441578e0 2023-08-27T11:18:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.998+0000 7f7e39ffb700 1 -- 172.21.15.112:0/3093754735 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e40011cd0 con 0x7f7e441578e0 2023-08-27T11:18:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.998+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3093754735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e440b3340 con 0x7f7e441578e0 2023-08-27T11:18:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.999+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3093754735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4415f510 con 0x7f7e441578e0 2023-08-27T11:18:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:35.999+0000 7f7e39ffb700 1 -- 172.21.15.112:0/3093754735 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7e40021350 con 0x7f7e441578e0 2023-08-27T11:18:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.000+0000 7f7e39ffb700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7e2806a0e0 0x7f7e2806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.000+0000 7f7e39ffb700 1 -- 172.21.15.112:0/3093754735 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f7e40088000 con 0x7f7e441578e0 2023-08-27T11:18:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.000+0000 7f7e4933f700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7e2806a0e0 0x7f7e2806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.001+0000 7f7e4933f700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7e2806a0e0 0x7f7e2806c5a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400ab60 tx=0x7f7e34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.004+0000 7f7e39ffb700 1 -- 172.21.15.112:0/3093754735 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7e4415f510 con 0x7f7e441578e0 2023-08-27T11:18:36.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.195+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3093754735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e4415f510 con 0x7f7e441578e0 2023-08-27T11:18:36.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.196+0000 7f7e39ffb700 1 -- 172.21.15.112:0/3093754735 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+14868 (secure 0 0 0) 0x7f7e4415f510 con 0x7f7e441578e0 2023-08-27T11:18:36.198 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:36.198 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":43,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:33.848236+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6837/3409848660":"2023-08-28T11:18:30.732010+0000","172.21.15.138:6835/3409848660":"2023-08-28T11:18:30.732010+0000","172.21.15.112:6833/1024768929":"2023-08-28T11:18:18.799945+0000","172.21.15.112:6835/1024768929":"2023-08-28T11:18:18.799945+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:36.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.198+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7e2806a0e0 msgr2=0x7f7e2806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.198+0000 7f7e4b5a3700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7e2806a0e0 0x7f7e2806c5a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400ab60 tx=0x7f7e34008040 comp rx=0 tx=0).stop 2023-08-27T11:18:36.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.198+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3093754735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e441578e0 msgr2=0x7f7e44158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.198+0000 7f7e4b5a3700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e441578e0 0x7f7e44158680 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c740 tx=0x7f7e4000ca50 comp rx=0 tx=0).stop 2023-08-27T11:18:36.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.198+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3093754735 shutdown_connections 2023-08-27T11:18:36.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.198+0000 7f7e4b5a3700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7e2806a0e0 0x7f7e2806c5a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.198+0000 7f7e4b5a3700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e44160f90 0x7f7e44152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.198+0000 7f7e4b5a3700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e440cf170 0x7f7e4415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.198+0000 7f7e4b5a3700 1 --2- 172.21.15.112:0/3093754735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e441578e0 0x7f7e44158680 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.198+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3093754735 >> 172.21.15.112:0/3093754735 conn(0x7f7e4406f970 msgr2=0x7f7e44155fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:36.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.198+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3093754735 shutdown_connections 2023-08-27T11:18:36.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.198+0000 7f7e4b5a3700 1 -- 172.21.15.112:0/3093754735 wait complete. 2023-08-27T11:18:36.213 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:18:36.214 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:18:36.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.336+0000 7fb196fa9700 1 Processor -- start 2023-08-27T11:18:36.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.336+0000 7fb196fa9700 1 -- start start 2023-08-27T11:18:36.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.337+0000 7fb196fa9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb190146c20 0x7fb190160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.337+0000 7fb196fa9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1900b3440 0x7fb19015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.337+0000 7fb196fa9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19015e6e0 0x7fb19006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.337+0000 7fb196fa9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1900b5070 con 0x7fb190146c20 2023-08-27T11:18:36.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.337+0000 7fb196fa9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb190147000 con 0x7fb1900b3440 2023-08-27T11:18:36.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.337+0000 7fb196fa9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1901614a0 con 0x7fb19015e6e0 2023-08-27T11:18:36.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.337+0000 7fb194d45700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb190146c20 0x7fb190160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.337+0000 7fb194d45700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb190146c20 0x7fb190160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57672/0 (socket says 172.21.15.112:57672) 2023-08-27T11:18:36.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.337+0000 7fb194d45700 1 -- 172.21.15.112:0/335899634 learned_addr learned my addr 172.21.15.112:0/335899634 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:36.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb194d45700 1 -- 172.21.15.112:0/335899634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19015e6e0 msgr2=0x7fb19006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:36.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb18ffff700 1 --2- 172.21.15.112:0/335899634 >> v2:172.21.15.138:3300/0 conn(0x7fb1900b3440 0x7fb19015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb195546700 1 --2- 172.21.15.112:0/335899634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19015e6e0 0x7fb19006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb194d45700 1 --2- 172.21.15.112:0/335899634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19015e6e0 0x7fb19006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb194d45700 1 -- 172.21.15.112:0/335899634 >> v2:172.21.15.138:3300/0 conn(0x7fb1900b3440 msgr2=0x7fb19015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:36.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb194d45700 1 --2- 172.21.15.112:0/335899634 >> v2:172.21.15.138:3300/0 conn(0x7fb1900b3440 0x7fb19015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb194d45700 1 -- 172.21.15.112:0/335899634 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb190161740 con 0x7fb190146c20 2023-08-27T11:18:36.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb18ffff700 1 --2- 172.21.15.112:0/335899634 >> v2:172.21.15.138:3300/0 conn(0x7fb1900b3440 0x7fb19015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:36.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb195546700 1 --2- 172.21.15.112:0/335899634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19015e6e0 0x7fb19006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:36.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb194d45700 1 --2- 172.21.15.112:0/335899634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb190146c20 0x7fb190160f60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb1900b2d80 tx=0x7fb184009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145462ea8f02edca server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:36.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb18f7fe700 1 -- 172.21.15.112:0/335899634 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18400b870 con 0x7fb190146c20 2023-08-27T11:18:36.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb18f7fe700 1 -- 172.21.15.112:0/335899634 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18400b9d0 con 0x7fb190146c20 2023-08-27T11:18:36.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb18f7fe700 1 -- 172.21.15.112:0/335899634 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18400bd60 con 0x7fb190146c20 2023-08-27T11:18:36.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb196fa9700 1 -- 172.21.15.112:0/335899634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb190146c20 msgr2=0x7fb190160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.338+0000 7fb196fa9700 1 --2- 172.21.15.112:0/335899634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb190146c20 0x7fb190160f60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb1900b2d80 tx=0x7fb184009bd0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.339+0000 7fb196fa9700 1 -- 172.21.15.112:0/335899634 shutdown_connections 2023-08-27T11:18:36.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.339+0000 7fb196fa9700 1 --2- 172.21.15.112:0/335899634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19015e6e0 0x7fb19006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.339+0000 7fb196fa9700 1 --2- 172.21.15.112:0/335899634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb190146c20 0x7fb190160f60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.339+0000 7fb196fa9700 1 --2- 172.21.15.112:0/335899634 >> v2:172.21.15.138:3300/0 conn(0x7fb1900b3440 0x7fb19015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.339+0000 7fb196fa9700 1 -- 172.21.15.112:0/335899634 >> 172.21.15.112:0/335899634 conn(0x7fb1900b1e90 msgr2=0x7fb1900b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:36.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.339+0000 7fb196fa9700 1 -- 172.21.15.112:0/335899634 shutdown_connections 2023-08-27T11:18:36.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.339+0000 7fb196fa9700 1 -- 172.21.15.112:0/335899634 wait complete. 2023-08-27T11:18:36.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.340+0000 7fb196fa9700 1 Processor -- start 2023-08-27T11:18:36.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.340+0000 7fb196fa9700 1 -- start start 2023-08-27T11:18:36.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb196fa9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1900b3440 0x7fb1901f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb196fa9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb190146c20 0x7fb1900c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb196fa9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb19015e6e0 0x7fb1900c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb196fa9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1901f4e30 con 0x7fb1900b3440 2023-08-27T11:18:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb196fa9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1901f3f80 con 0x7fb19015e6e0 2023-08-27T11:18:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb196fa9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1901f5800 con 0x7fb190146c20 2023-08-27T11:18:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb194d45700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1900b3440 0x7fb1901f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb194d45700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1900b3440 0x7fb1901f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57678/0 (socket says 172.21.15.112:57678) 2023-08-27T11:18:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb194d45700 1 -- 172.21.15.112:0/2997322665 learned_addr learned my addr 172.21.15.112:0/2997322665 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:36.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb18ffff700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb190146c20 0x7fb1900c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb195546700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb19015e6e0 0x7fb1900c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb194d45700 1 -- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb190146c20 msgr2=0x7fb1900c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb194d45700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb190146c20 0x7fb1900c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb194d45700 1 -- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb19015e6e0 msgr2=0x7fb1900c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb194d45700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb19015e6e0 0x7fb1900c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb194d45700 1 -- 172.21.15.112:0/2997322665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb184009550 con 0x7fb1900b3440 2023-08-27T11:18:36.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb194d45700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1900b3440 0x7fb1901f52c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb184014040 tx=0x7fb184004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:36.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb18d7fa700 1 -- 172.21.15.112:0/2997322665 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184004250 con 0x7fb1900b3440 2023-08-27T11:18:36.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb18d7fa700 1 -- 172.21.15.112:0/2997322665 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18401e070 con 0x7fb1900b3440 2023-08-27T11:18:36.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb18d7fa700 1 -- 172.21.15.112:0/2997322665 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1840036a0 con 0x7fb1900b3440 2023-08-27T11:18:36.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb196fa9700 1 -- 172.21.15.112:0/2997322665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1901f4220 con 0x7fb1900b3440 2023-08-27T11:18:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.341+0000 7fb196fa9700 1 -- 172.21.15.112:0/2997322665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1901f47e0 con 0x7fb1900b3440 2023-08-27T11:18:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.342+0000 7fb196fa9700 1 -- 172.21.15.112:0/2997322665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1901f5110 con 0x7fb1900b3440 2023-08-27T11:18:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.343+0000 7fb18d7fa700 1 -- 172.21.15.112:0/2997322665 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb184020030 con 0x7fb1900b3440 2023-08-27T11:18:36.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.343+0000 7fb18d7fa700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb17406a0e0 0x7fb17406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.343+0000 7fb18d7fa700 1 -- 172.21.15.112:0/2997322665 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fb1840892f0 con 0x7fb1900b3440 2023-08-27T11:18:36.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.343+0000 7fb18ffff700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb17406a0e0 0x7fb17406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.344+0000 7fb18ffff700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb17406a0e0 0x7fb17406c5a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb1900b2900 tx=0x7fb17c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:36.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.347+0000 7fb18d7fa700 1 -- 172.21.15.112:0/2997322665 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb184054d10 con 0x7fb1900b3440 2023-08-27T11:18:36.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.536+0000 7fb196fa9700 1 -- 172.21.15.112:0/2997322665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb190067e90 con 0x7fb1900b3440 2023-08-27T11:18:36.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.536+0000 7fb18d7fa700 1 -- 172.21.15.112:0/2997322665 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb184058330 con 0x7fb1900b3440 2023-08-27T11:18:36.538 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:18:36.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.538+0000 7fb196fa9700 1 -- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb17406a0e0 msgr2=0x7fb17406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.538+0000 7fb196fa9700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb17406a0e0 0x7fb17406c5a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb1900b2900 tx=0x7fb17c008040 comp rx=0 tx=0).stop 2023-08-27T11:18:36.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.538+0000 7fb196fa9700 1 -- 172.21.15.112:0/2997322665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1900b3440 msgr2=0x7fb1901f52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.538+0000 7fb196fa9700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1900b3440 0x7fb1901f52c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb184014040 tx=0x7fb184004750 comp rx=0 tx=0).stop 2023-08-27T11:18:36.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.538+0000 7fb196fa9700 1 -- 172.21.15.112:0/2997322665 shutdown_connections 2023-08-27T11:18:36.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.538+0000 7fb196fa9700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb17406a0e0 0x7fb17406c5a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.538+0000 7fb196fa9700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb19015e6e0 0x7fb1900c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.538+0000 7fb196fa9700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1900b3440 0x7fb1901f52c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.538+0000 7fb196fa9700 1 --2- 172.21.15.112:0/2997322665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb190146c20 0x7fb1900c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.538+0000 7fb196fa9700 1 -- 172.21.15.112:0/2997322665 >> 172.21.15.112:0/2997322665 conn(0x7fb1900b1e90 msgr2=0x7fb19006d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:36.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.538+0000 7fb196fa9700 1 -- 172.21.15.112:0/2997322665 shutdown_connections 2023-08-27T11:18:36.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.538+0000 7fb196fa9700 1 -- 172.21.15.112:0/2997322665 wait complete. 2023-08-27T11:18:36.552 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:18:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.677+0000 7f648cbda700 1 Processor -- start 2023-08-27T11:18:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.677+0000 7f648cbda700 1 -- start start 2023-08-27T11:18:36.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f648cbda700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64881437c0 0x7f64880b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f648cbda700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f648806d5b0 0x7f648806da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f648cbda700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64880cfd50 con 0x7f6488143ba0 2023-08-27T11:18:36.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f648cbda700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f648806df80 con 0x7f64881437c0 2023-08-27T11:18:36.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f648cbda700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64880cf140 con 0x7f648806d5b0 2023-08-27T11:18:36.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f648659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64881437c0 0x7f64880b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f648659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64881437c0 0x7f64880b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57686/0 (socket says 172.21.15.112:57686) 2023-08-27T11:18:36.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f648659c700 1 -- 172.21.15.112:0/654781386 learned_addr learned my addr 172.21.15.112:0/654781386 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:36.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f6486d9d700 1 --2- 172.21.15.112:0/654781386 >> v2:172.21.15.138:3300/0 conn(0x7f648806d5b0 0x7f648806da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f648659c700 1 -- 172.21.15.112:0/654781386 >> v2:172.21.15.138:3300/0 conn(0x7f648806d5b0 msgr2=0x7f648806da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:36.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f648659c700 1 --2- 172.21.15.112:0/654781386 >> v2:172.21.15.138:3300/0 conn(0x7f648806d5b0 0x7f648806da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f648659c700 1 -- 172.21.15.112:0/654781386 >> v1:172.21.15.138:6789/0 conn(0x7f6488143ba0 legacy=0x7f64880ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f648659c700 1 -- 172.21.15.112:0/654781386 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880cf420 con 0x7f64881437c0 2023-08-27T11:18:36.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f6486d9d700 1 --2- 172.21.15.112:0/654781386 >> v2:172.21.15.138:3300/0 conn(0x7f648806d5b0 0x7f648806da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:36.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.678+0000 7f648659c700 1 --2- 172.21.15.112:0/654781386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64881437c0 0x7f64880b4670 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f64880cfcc0 tx=0x7f6478009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3534ff3a7fed7049 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:36.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.679+0000 7f648559a700 1 -- 172.21.15.112:0/654781386 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647800b870 con 0x7f64881437c0 2023-08-27T11:18:36.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.679+0000 7f648559a700 1 -- 172.21.15.112:0/654781386 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647800b9d0 con 0x7f64881437c0 2023-08-27T11:18:36.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.679+0000 7f648559a700 1 -- 172.21.15.112:0/654781386 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647800bd60 con 0x7f64881437c0 2023-08-27T11:18:36.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.679+0000 7f648cbda700 1 -- 172.21.15.112:0/654781386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64881437c0 msgr2=0x7f64880b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.679+0000 7f648cbda700 1 --2- 172.21.15.112:0/654781386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64881437c0 0x7f64880b4670 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f64880cfcc0 tx=0x7f6478009bd0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.679+0000 7f648cbda700 1 -- 172.21.15.112:0/654781386 shutdown_connections 2023-08-27T11:18:36.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.679+0000 7f648cbda700 1 --2- 172.21.15.112:0/654781386 >> v2:172.21.15.138:3300/0 conn(0x7f648806d5b0 0x7f648806da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.679+0000 7f648cbda700 1 --2- 172.21.15.112:0/654781386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64881437c0 0x7f64880b4670 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.679+0000 7f648cbda700 1 -- 172.21.15.112:0/654781386 >> 172.21.15.112:0/654781386 conn(0x7f64880b1e90 msgr2=0x7f64880b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:36.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.679+0000 7f648cbda700 1 -- 172.21.15.112:0/654781386 shutdown_connections 2023-08-27T11:18:36.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.679+0000 7f648cbda700 1 -- 172.21.15.112:0/654781386 wait complete. 2023-08-27T11:18:36.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.681+0000 7f648cbda700 1 Processor -- start 2023-08-27T11:18:36.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.681+0000 7f648cbda700 1 -- start start 2023-08-27T11:18:36.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f648cbda700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f648806d5b0 0x7f64880ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f648cbda700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64881437c0 0x7f64880708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f648cbda700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6488143ba0 0x7f6488070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f648cbda700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64880abdf0 con 0x7f6488143ba0 2023-08-27T11:18:36.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f648cbda700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64880ab4e0 con 0x7f648806d5b0 2023-08-27T11:18:36.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f648cbda700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64880aeba0 con 0x7f64881437c0 2023-08-27T11:18:36.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f6486d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6488143ba0 0x7f6488070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f6486d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6488143ba0 0x7f6488070de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57696/0 (socket says 172.21.15.112:57696) 2023-08-27T11:18:36.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f6486d9d700 1 -- 172.21.15.112:0/1196934180 learned_addr learned my addr 172.21.15.112:0/1196934180 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:36.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f648659c700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f648806d5b0 0x7f64880ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f6485d9b700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64881437c0 0x7f64880708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f6486d9d700 1 -- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64881437c0 msgr2=0x7f64880708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f6486d9d700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64881437c0 0x7f64880708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f6486d9d700 1 -- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f648806d5b0 msgr2=0x7f64880ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f6486d9d700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f648806d5b0 0x7f64880ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f6486d9d700 1 -- 172.21.15.112:0/1196934180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6478009550 con 0x7f6488143ba0 2023-08-27T11:18:36.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f648659c700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f648806d5b0 0x7f64880ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:36.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.682+0000 7f6486d9d700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6488143ba0 0x7f6488070de0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6480007f20 tx=0x7f648000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:36.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.683+0000 7f6476ffd700 1 -- 172.21.15.112:0/1196934180 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648000de90 con 0x7f6488143ba0 2023-08-27T11:18:36.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.683+0000 7f6476ffd700 1 -- 172.21.15.112:0/1196934180 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648000f040 con 0x7f6488143ba0 2023-08-27T11:18:36.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.683+0000 7f648cbda700 1 -- 172.21.15.112:0/1196934180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64880ab730 con 0x7f6488143ba0 2023-08-27T11:18:36.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.683+0000 7f6476ffd700 1 -- 172.21.15.112:0/1196934180 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480003c60 con 0x7f6488143ba0 2023-08-27T11:18:36.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.683+0000 7f648cbda700 1 -- 172.21.15.112:0/1196934180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648813ffa0 con 0x7f6488143ba0 2023-08-27T11:18:36.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.684+0000 7f648cbda700 1 -- 172.21.15.112:0/1196934180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648815a970 con 0x7f6488143ba0 2023-08-27T11:18:36.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.684+0000 7f6476ffd700 1 -- 172.21.15.112:0/1196934180 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6480004030 con 0x7f6488143ba0 2023-08-27T11:18:36.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.684+0000 7f6476ffd700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f646806a0e0 0x7f646806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:36.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.684+0000 7f6476ffd700 1 -- 172.21.15.112:0/1196934180 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f6480088490 con 0x7f6488143ba0 2023-08-27T11:18:36.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.685+0000 7f648659c700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f646806a0e0 0x7f646806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:36.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.685+0000 7f648659c700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f646806a0e0 0x7f646806c5a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6478000c00 tx=0x7f6478008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:36.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.689+0000 7f6476ffd700 1 -- 172.21.15.112:0/1196934180 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f648815a970 con 0x7f6488143ba0 2023-08-27T11:18:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.879+0000 7f648cbda700 1 -- 172.21.15.112:0/1196934180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f64880abf30 con 0x7f6488143ba0 2023-08-27T11:18:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.880+0000 7f6476ffd700 1 -- 172.21.15.112:0/1196934180 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64880abf30 con 0x7f6488143ba0 2023-08-27T11:18:36.882 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:18:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.882+0000 7f648cbda700 1 -- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f646806a0e0 msgr2=0x7f646806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.882+0000 7f648cbda700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f646806a0e0 0x7f646806c5a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6478000c00 tx=0x7f6478008030 comp rx=0 tx=0).stop 2023-08-27T11:18:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.882+0000 7f648cbda700 1 -- 172.21.15.112:0/1196934180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6488143ba0 msgr2=0x7f6488070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.882+0000 7f648cbda700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6488143ba0 0x7f6488070de0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6480007f20 tx=0x7f648000d3b0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.882+0000 7f648cbda700 1 -- 172.21.15.112:0/1196934180 shutdown_connections 2023-08-27T11:18:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.882+0000 7f648cbda700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f646806a0e0 0x7f646806c5a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.882+0000 7f648cbda700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6488143ba0 0x7f6488070de0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.882+0000 7f648cbda700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f648806d5b0 0x7f64880ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.882+0000 7f648cbda700 1 --2- 172.21.15.112:0/1196934180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64881437c0 0x7f64880708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.882+0000 7f648cbda700 1 -- 172.21.15.112:0/1196934180 >> 172.21.15.112:0/1196934180 conn(0x7f64880b1e90 msgr2=0x7f64880b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.882+0000 7f648cbda700 1 -- 172.21.15.112:0/1196934180 shutdown_connections 2023-08-27T11:18:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:36.882+0000 7f648cbda700 1 -- 172.21.15.112:0/1196934180 wait complete. 2023-08-27T11:18:36.896 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:18:37.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.018+0000 7ff24f1c9700 1 Processor -- start 2023-08-27T11:18:37.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.018+0000 7ff24f1c9700 1 -- start start 2023-08-27T11:18:37.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff24f1c9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff2480ce210 0x7ff24815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff24f1c9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff2480ce5f0 0x7ff2480cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff24f1c9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2480cb500 con 0x7ff2480b1ff0 2023-08-27T11:18:37.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff24f1c9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24815ded0 con 0x7ff2480ce5f0 2023-08-27T11:18:37.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff24f1c9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2480cb7b0 con 0x7ff2480ce210 2023-08-27T11:18:37.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff247fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff2480ce5f0 0x7ff2480cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff24d766700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff2480b1ff0 0x7ff24806c470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52086/0 (socket says 172.21.15.112:52086) 2023-08-27T11:18:37.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff247fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff2480ce5f0 0x7ff2480cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57702/0 (socket says 172.21.15.112:57702) 2023-08-27T11:18:37.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff24d766700 1 -- 172.21.15.112:0/2815919206 learned_addr learned my addr 172.21.15.112:0/2815919206 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:37.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff24cf65700 1 --2- 172.21.15.112:0/2815919206 >> v2:172.21.15.138:3300/0 conn(0x7ff2480ce210 0x7ff24815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff247fff700 1 -- 172.21.15.112:0/2815919206 >> v2:172.21.15.138:3300/0 conn(0x7ff2480ce210 msgr2=0x7ff24815aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:37.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff247fff700 1 --2- 172.21.15.112:0/2815919206 >> v2:172.21.15.138:3300/0 conn(0x7ff2480ce210 0x7ff24815aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff247fff700 1 -- 172.21.15.112:0/2815919206 >> v1:172.21.15.112:6789/0 conn(0x7ff2480b1ff0 legacy=0x7ff24806c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:37.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff247fff700 1 -- 172.21.15.112:0/2815919206 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24815d530 con 0x7ff2480ce5f0 2023-08-27T11:18:37.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff2477fe700 1 -- 172.21.15.112:0/2815919206 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3772927293 0 0) 0x7ff2480cb500 con 0x7ff2480b1ff0 2023-08-27T11:18:37.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff24cf65700 1 --2- 172.21.15.112:0/2815919206 >> v2:172.21.15.138:3300/0 conn(0x7ff2480ce210 0x7ff24815aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:37.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.019+0000 7ff247fff700 1 --2- 172.21.15.112:0/2815919206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2480ce5f0 0x7ff2480cafc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff2480cfa80 tx=0x7ff23400bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49043773439931bc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:37.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.020+0000 7ff2477fe700 1 -- 172.21.15.112:0/2815919206 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23400dc70 con 0x7ff2480ce5f0 2023-08-27T11:18:37.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.020+0000 7ff2477fe700 1 -- 172.21.15.112:0/2815919206 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23400ddd0 con 0x7ff2480ce5f0 2023-08-27T11:18:37.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.020+0000 7ff2477fe700 1 -- 172.21.15.112:0/2815919206 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234010690 con 0x7ff2480ce5f0 2023-08-27T11:18:37.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.020+0000 7ff24f1c9700 1 -- 172.21.15.112:0/2815919206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2480ce5f0 msgr2=0x7ff2480cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:37.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.020+0000 7ff24f1c9700 1 --2- 172.21.15.112:0/2815919206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2480ce5f0 0x7ff2480cafc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff2480cfa80 tx=0x7ff23400bdc0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.020+0000 7ff24f1c9700 1 -- 172.21.15.112:0/2815919206 shutdown_connections 2023-08-27T11:18:37.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.020+0000 7ff24f1c9700 1 --2- 172.21.15.112:0/2815919206 >> v2:172.21.15.138:3300/0 conn(0x7ff2480ce210 0x7ff24815aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.020+0000 7ff24f1c9700 1 --2- 172.21.15.112:0/2815919206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2480ce5f0 0x7ff2480cafc0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.020+0000 7ff24f1c9700 1 -- 172.21.15.112:0/2815919206 >> 172.21.15.112:0/2815919206 conn(0x7ff2480cef70 msgr2=0x7ff2480cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:37.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.020+0000 7ff24f1c9700 1 -- 172.21.15.112:0/2815919206 shutdown_connections 2023-08-27T11:18:37.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.020+0000 7ff24f1c9700 1 -- 172.21.15.112:0/2815919206 wait complete. 2023-08-27T11:18:37.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.021+0000 7ff24f1c9700 1 Processor -- start 2023-08-27T11:18:37.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.021+0000 7ff24f1c9700 1 -- start start 2023-08-27T11:18:37.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.022+0000 7ff24f1c9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2480b1ff0 0x7ff2481469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.022+0000 7ff24f1c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2480ce210 0x7ff24814f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.022+0000 7ff24f1c9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2480ce5f0 0x7ff2481479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.022+0000 7ff24f1c9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2481475b0 con 0x7ff2480ce5f0 2023-08-27T11:18:37.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.022+0000 7ff24f1c9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff248146f80 con 0x7ff2480ce210 2023-08-27T11:18:37.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.022+0000 7ff24f1c9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24814da40 con 0x7ff2480b1ff0 2023-08-27T11:18:37.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.022+0000 7ff24d766700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2480ce5f0 0x7ff2481479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.022+0000 7ff24d766700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2480ce5f0 0x7ff2481479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57710/0 (socket says 172.21.15.112:57710) 2023-08-27T11:18:37.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.022+0000 7ff24d766700 1 -- 172.21.15.112:0/354292896 learned_addr learned my addr 172.21.15.112:0/354292896 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:37.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff24cf65700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2480b1ff0 0x7ff2481469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff247fff700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2480ce210 0x7ff24814f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff24d766700 1 -- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2480b1ff0 msgr2=0x7ff2481469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:37.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff24d766700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2480b1ff0 0x7ff2481469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff24d766700 1 -- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2480ce210 msgr2=0x7ff24814f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:37.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff24d766700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2480ce210 0x7ff24814f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff24d766700 1 -- 172.21.15.112:0/354292896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23400b740 con 0x7ff2480ce5f0 2023-08-27T11:18:37.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff24cf65700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2480b1ff0 0x7ff2481469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:37.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff24d766700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2480ce5f0 0x7ff2481479d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff24000cbf0 tx=0x7ff24000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:37.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff2457fa700 1 -- 172.21.15.112:0/354292896 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24000abc0 con 0x7ff2480ce5f0 2023-08-27T11:18:37.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff2457fa700 1 -- 172.21.15.112:0/354292896 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24000ad40 con 0x7ff2480ce5f0 2023-08-27T11:18:37.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff24f1c9700 1 -- 172.21.15.112:0/354292896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24814dd20 con 0x7ff2480ce5f0 2023-08-27T11:18:37.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff2457fa700 1 -- 172.21.15.112:0/354292896 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240007310 con 0x7ff2480ce5f0 2023-08-27T11:18:37.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.023+0000 7ff24f1c9700 1 -- 172.21.15.112:0/354292896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24814fc50 con 0x7ff2480ce5f0 2023-08-27T11:18:37.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.024+0000 7ff24f1c9700 1 -- 172.21.15.112:0/354292896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2481476f0 con 0x7ff2480ce5f0 2023-08-27T11:18:37.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.024+0000 7ff2457fa700 1 -- 172.21.15.112:0/354292896 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff240007470 con 0x7ff2480ce5f0 2023-08-27T11:18:37.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.025+0000 7ff2457fa700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff22c06a0e0 0x7ff22c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.025+0000 7ff2457fa700 1 -- 172.21.15.112:0/354292896 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7ff240088070 con 0x7ff2480ce5f0 2023-08-27T11:18:37.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.025+0000 7ff24cf65700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff22c06a0e0 0x7ff22c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.025+0000 7ff24cf65700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff22c06a0e0 0x7ff22c06c5a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0089b0 tx=0x7ff23c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:37.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.029+0000 7ff2457fa700 1 -- 172.21.15.112:0/354292896 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff2481476f0 con 0x7ff2480ce5f0 2023-08-27T11:18:37.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.216+0000 7ff24f1c9700 1 -- 172.21.15.112:0/354292896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff2481476f0 con 0x7ff2480ce5f0 2023-08-27T11:18:37.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.220+0000 7ff2457fa700 1 -- 172.21.15.112:0/354292896 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff2481476f0 con 0x7ff2480ce5f0 2023-08-27T11:18:37.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.222+0000 7ff24f1c9700 1 -- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff22c06a0e0 msgr2=0x7ff22c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:37.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.222+0000 7ff24f1c9700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff22c06a0e0 0x7ff22c06c5a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0089b0 tx=0x7ff23c008040 comp rx=0 tx=0).stop 2023-08-27T11:18:37.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.222+0000 7ff24f1c9700 1 -- 172.21.15.112:0/354292896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2480ce5f0 msgr2=0x7ff2481479d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:37.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.222+0000 7ff24f1c9700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2480ce5f0 0x7ff2481479d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff24000cbf0 tx=0x7ff24000a060 comp rx=0 tx=0).stop 2023-08-27T11:18:37.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.222+0000 7ff24f1c9700 1 -- 172.21.15.112:0/354292896 shutdown_connections 2023-08-27T11:18:37.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.222+0000 7ff24f1c9700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff22c06a0e0 0x7ff22c06c5a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.222+0000 7ff24f1c9700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2480ce5f0 0x7ff2481479d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.222+0000 7ff24f1c9700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2480b1ff0 0x7ff2481469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.222+0000 7ff24f1c9700 1 --2- 172.21.15.112:0/354292896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2480ce210 0x7ff24814f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.222+0000 7ff24f1c9700 1 -- 172.21.15.112:0/354292896 >> 172.21.15.112:0/354292896 conn(0x7ff2480cef70 msgr2=0x7ff24815a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:37.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.222+0000 7ff24f1c9700 1 -- 172.21.15.112:0/354292896 shutdown_connections 2023-08-27T11:18:37.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.222+0000 7ff24f1c9700 1 -- 172.21.15.112:0/354292896 wait complete. 2023-08-27T11:18:37.236 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:18:37.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:18:37.282 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:18:37.330 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:18:37.331 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:18:37.331 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:37.348 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:18:37.349 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:18:37.349 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:37.369 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:18:37.369 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:18:37.370 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:37.406 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:18:37.407 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:18:37.408 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:37.428 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:18:37.428 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:18:37.429 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:18:37.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.565+0000 7fc184768700 1 Processor -- start 2023-08-27T11:18:37.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.565+0000 7fc184768700 1 -- start start 2023-08-27T11:18:37.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc184768700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0b1400 0x7fc17c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc184768700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc17c0b17e0 0x7fc17c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc184768700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17c161850 con 0x7fc17c161ce0 2023-08-27T11:18:37.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc184768700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17c06e7d0 con 0x7fc17c0b17e0 2023-08-27T11:18:37.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc184768700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0cfc10 con 0x7fc17c0b1400 2023-08-27T11:18:37.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc182d05700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc17c161ce0 0x7fc17c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52100/0 (socket says 172.21.15.112:52100) 2023-08-27T11:18:37.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc182d05700 1 -- 172.21.15.112:0/3135608961 learned_addr learned my addr 172.21.15.112:0/3135608961 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:37.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389347592 0 0) 0x7fc17c161850 con 0x7fc17c161ce0 2023-08-27T11:18:37.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181d03700 1 --2- 172.21.15.112:0/3135608961 >> v2:172.21.15.138:3300/0 conn(0x7fc17c0b17e0 0x7fc17c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1680035b0 con 0x7fc17c161ce0 2023-08-27T11:18:37.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc182504700 1 --2- 172.21.15.112:0/3135608961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0b1400 0x7fc17c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 571666472 0 0) 0x7fc1680035b0 con 0x7fc17c161ce0 2023-08-27T11:18:37.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0b1400 msgr2=0x7fc17c0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:37.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 --2- 172.21.15.112:0/3135608961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0b1400 0x7fc17c0cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 >> v2:172.21.15.138:3300/0 conn(0x7fc17c0b17e0 msgr2=0x7fc17c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:37.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 --2- 172.21.15.112:0/3135608961 >> v2:172.21.15.138:3300/0 conn(0x7fc17c0b17e0 0x7fc17c06e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c161200 con 0x7fc17c161ce0 2023-08-27T11:18:37.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fc1740019f0 con 0x7fc17c161ce0 2023-08-27T11:18:37.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 >> v1:172.21.15.112:6789/0 conn(0x7fc17c161ce0 legacy=0x7fc17c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:37.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 --2- 172.21.15.112:0/3135608961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc168003a00 0x7fc168003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 --2- 172.21.15.112:0/3135608961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc168004850 0x7fc168004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 --2- 172.21.15.112:0/3135608961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0b1400 0x7fc168005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c06e7d0 con 0x7fc168004850 2023-08-27T11:18:37.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0cfc10 con 0x7fc168003a00 2023-08-27T11:18:37.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16800ba90 con 0x7fc17c0b1400 2023-08-27T11:18:37.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.566+0000 7fc181d03700 1 --2- 172.21.15.112:0/3135608961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc168004850 0x7fc168004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc182504700 1 --2- 172.21.15.112:0/3135608961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc168003a00 0x7fc168003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc182d05700 1 --2- 172.21.15.112:0/3135608961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0b1400 0x7fc168005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc181d03700 1 -- 172.21.15.112:0/3135608961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0b1400 msgr2=0x7fc168005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:37.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc181d03700 1 --2- 172.21.15.112:0/3135608961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0b1400 0x7fc168005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc181d03700 1 -- 172.21.15.112:0/3135608961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc168003a00 msgr2=0x7fc168003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:37.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc181d03700 1 --2- 172.21.15.112:0/3135608961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc168003a00 0x7fc168003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc181d03700 1 -- 172.21.15.112:0/3135608961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1680035b0 con 0x7fc168004850 2023-08-27T11:18:37.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc181d03700 1 --2- 172.21.15.112:0/3135608961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc168004850 0x7fc168004cd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009b60 tx=0x7fc168005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2791adcafc0dd6d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c00e6c0 con 0x7fc168004850 2023-08-27T11:18:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c00e820 con 0x7fc168004850 2023-08-27T11:18:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc181502700 1 -- 172.21.15.112:0/3135608961 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c00eb90 con 0x7fc168004850 2023-08-27T11:18:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc184768700 1 -- 172.21.15.112:0/3135608961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc168004850 msgr2=0x7fc168004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc184768700 1 --2- 172.21.15.112:0/3135608961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc168004850 0x7fc168004cd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009b60 tx=0x7fc168005880 comp rx=0 tx=0).stop 2023-08-27T11:18:37.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.567+0000 7fc182d05700 1 -- 172.21.15.112:0/3135608961 reap_dead start 2023-08-27T11:18:37.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.568+0000 7fc184768700 1 -- 172.21.15.112:0/3135608961 shutdown_connections 2023-08-27T11:18:37.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.568+0000 7fc184768700 1 -- 172.21.15.112:0/3135608961 >> 172.21.15.112:0/3135608961 conn(0x7fc17c0d0000 msgr2=0x7fc17c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:37.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.568+0000 7fc184768700 1 -- 172.21.15.112:0/3135608961 shutdown_connections 2023-08-27T11:18:37.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.568+0000 7fc184768700 1 -- 172.21.15.112:0/3135608961 wait complete. 2023-08-27T11:18:37.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.569+0000 7fc184768700 1 Processor -- start 2023-08-27T11:18:37.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.569+0000 7fc184768700 1 -- start start 2023-08-27T11:18:37.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.569+0000 7fc184768700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc168004850 0x7fc17c1f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc184768700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0b00a0 0x7fc17c0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc184768700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c0ca130 0x7fc17c0c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc184768700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c1f3800 con 0x7fc168004850 2023-08-27T11:18:37.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc184768700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c1514a0 con 0x7fc17c0ca130 2023-08-27T11:18:37.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc184768700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c151780 con 0x7fc17c0b00a0 2023-08-27T11:18:37.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc182504700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc168004850 0x7fc17c1f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc182504700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc168004850 0x7fc17c1f32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57726/0 (socket says 172.21.15.112:57726) 2023-08-27T11:18:37.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc182504700 1 -- 172.21.15.112:0/2596728212 learned_addr learned my addr 172.21.15.112:0/2596728212 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:37.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc182d05700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c0ca130 0x7fc17c0c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc181d03700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0b00a0 0x7fc17c0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc182504700 1 -- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0b00a0 msgr2=0x7fc17c0c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:37.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc182504700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0b00a0 0x7fc17c0c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc182504700 1 -- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c0ca130 msgr2=0x7fc17c0c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:37.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc182504700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c0ca130 0x7fc17c0c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc182504700 1 -- 172.21.15.112:0/2596728212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c151a60 con 0x7fc168004850 2023-08-27T11:18:37.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc182504700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc168004850 0x7fc17c1f32c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc178011aa0 tx=0x7fc178011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:37.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc172ffd700 1 -- 172.21.15.112:0/2596728212 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178011ee0 con 0x7fc168004850 2023-08-27T11:18:37.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc172ffd700 1 -- 172.21.15.112:0/2596728212 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17800f940 con 0x7fc168004850 2023-08-27T11:18:37.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc172ffd700 1 -- 172.21.15.112:0/2596728212 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17800fcf0 con 0x7fc168004850 2023-08-27T11:18:37.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc184768700 1 -- 172.21.15.112:0/2596728212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17c151d40 con 0x7fc168004850 2023-08-27T11:18:37.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.570+0000 7fc184768700 1 -- 172.21.15.112:0/2596728212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17c0b0520 con 0x7fc168004850 2023-08-27T11:18:37.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.571+0000 7fc184768700 1 -- 172.21.15.112:0/2596728212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17c0b0710 con 0x7fc168004850 2023-08-27T11:18:37.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.572+0000 7fc172ffd700 1 -- 172.21.15.112:0/2596728212 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc178014440 con 0x7fc168004850 2023-08-27T11:18:37.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.572+0000 7fc172ffd700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16006a0e0 0x7fc16006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:37.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.572+0000 7fc172ffd700 1 -- 172.21.15.112:0/2596728212 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fc178088870 con 0x7fc168004850 2023-08-27T11:18:37.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.572+0000 7fc181d03700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16006a0e0 0x7fc16006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:37.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.573+0000 7fc181d03700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16006a0e0 0x7fc16006c5a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0090b0 tx=0x7fc16c01f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:37.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.576+0000 7fc172ffd700 1 -- 172.21.15.112:0/2596728212 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc178054230 con 0x7fc168004850 2023-08-27T11:18:37.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.808+0000 7fc184768700 1 -- 172.21.15.112:0/2596728212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc17c067a20 con 0x7fc168004850 2023-08-27T11:18:37.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.809+0000 7fc172ffd700 1 -- 172.21.15.112:0/2596728212 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+3711 (secure 0 0 0) 0x7fc178022070 con 0x7fc168004850 2023-08-27T11:18:37.811 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:37.811 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1941625698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1941625698},{"type":"v1","addr":"172.21.15.138:6836","nonce":1941625698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4904,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1317948937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1317948937},{"type":"v1","addr":"172.21.15.112:6835","nonce":1317948937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2155951977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2155951977},{"type":"v1","addr":"172.21.15.138:6837","nonce":2155951977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1530886230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1530886230},{"type":"v1","addr":"172.21.15.112:6834","nonce":1530886230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-08-27T11:18:37.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.811+0000 7fc184768700 1 -- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16006a0e0 msgr2=0x7fc16006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:37.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.811+0000 7fc184768700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16006a0e0 0x7fc16006c5a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0090b0 tx=0x7fc16c01f040 comp rx=0 tx=0).stop 2023-08-27T11:18:37.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.811+0000 7fc184768700 1 -- 172.21.15.112:0/2596728212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc168004850 msgr2=0x7fc17c1f32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:37.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.811+0000 7fc184768700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc168004850 0x7fc17c1f32c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc178011aa0 tx=0x7fc178011db0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.811+0000 7fc184768700 1 -- 172.21.15.112:0/2596728212 shutdown_connections 2023-08-27T11:18:37.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.811+0000 7fc184768700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16006a0e0 0x7fc16006c5a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.811+0000 7fc184768700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c0ca130 0x7fc17c0c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.811+0000 7fc184768700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc168004850 0x7fc17c1f32c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.811+0000 7fc184768700 1 --2- 172.21.15.112:0/2596728212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0b00a0 0x7fc17c0c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:37.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.811+0000 7fc184768700 1 -- 172.21.15.112:0/2596728212 >> 172.21.15.112:0/2596728212 conn(0x7fc17c0d0000 msgr2=0x7fc17c0cb570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:37.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.811+0000 7fc184768700 1 -- 172.21.15.112:0/2596728212 shutdown_connections 2023-08-27T11:18:37.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:37.811+0000 7fc184768700 1 -- 172.21.15.112:0/2596728212 wait complete. 2023-08-27T11:18:37.815 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 23 2023-08-27T11:18:37.824 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:18:37.825 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:18:37.825 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:18:37.825 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:18:37.826 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:18:37.826 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:18:37.826 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:18:37.826 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:18:37.827 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:18:37.827 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:18:37.827 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:18:37.827 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:18:37.828 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:18:37.828 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:18:37.828 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:18:37.829 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:18:37.834 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:18:37.834 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:18:37.835 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:18:37.836 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:18:37.839 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:18:37.839 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:18:37.840 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:18:37.841 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:18:37.876 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:18:37.877 DEBUG:teuthology.parallel:result is None 2023-08-27T11:18:37.878 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:18:37.878 DEBUG:teuthology.parallel:result is None 2023-08-27T11:18:37.881 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:18:37.882 DEBUG:teuthology.parallel:result is None 2023-08-27T11:18:37.882 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:18:37.883 DEBUG:teuthology.parallel:result is None 2023-08-27T11:18:37.883 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:18:37.955 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:18:37.955 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:18:37.967 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:18:37.968 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2023-08-27T11:18:38.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.009+0000 7feaed0cb700 1 Processor -- start 2023-08-27T11:18:38.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.009+0000 7feaed0cb700 1 -- start start 2023-08-27T11:18:38.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feaed0cb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feae815d490 0x7feae815d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:38.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feaed0cb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae8066450 0x7feae815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:38.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feaed0cb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feae806e150 0x7feae80701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:38.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feaed0cb700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae80ad580 con 0x7feae806e150 2023-08-27T11:18:38.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feaed0cb700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae80ad260 con 0x7feae815d490 2023-08-27T11:18:38.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feaed0cb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae806f790 con 0x7feae8066450 2023-08-27T11:18:38.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feae759e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feae806e150 0x7feae80701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:38.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feae759e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feae806e150 0x7feae80701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57762/0 (socket says 172.21.15.112:57762) 2023-08-27T11:18:38.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feae759e700 1 -- 172.21.15.112:0/714035950 learned_addr learned my addr 172.21.15.112:0/714035950 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:38.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feae6d9d700 1 --2- 172.21.15.112:0/714035950 >> v2:172.21.15.138:3300/0 conn(0x7feae815d490 0x7feae815d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:38.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feae659c700 1 --2- 172.21.15.112:0/714035950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae8066450 0x7feae815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:38.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feae759e700 1 -- 172.21.15.112:0/714035950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae8066450 msgr2=0x7feae815ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:38.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feae759e700 1 --2- 172.21.15.112:0/714035950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae8066450 0x7feae815ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:38.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feae759e700 1 -- 172.21.15.112:0/714035950 >> v2:172.21.15.138:3300/0 conn(0x7feae815d490 msgr2=0x7feae815d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:38.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feae759e700 1 --2- 172.21.15.112:0/714035950 >> v2:172.21.15.138:3300/0 conn(0x7feae815d490 0x7feae815d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:38.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feae759e700 1 -- 172.21.15.112:0/714035950 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae806fa70 con 0x7feae806e150 2023-08-27T11:18:38.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feae659c700 1 --2- 172.21.15.112:0/714035950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae8066450 0x7feae815ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:38.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feae759e700 1 --2- 172.21.15.112:0/714035950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae806e150 0x7feae80701a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7feae8067720 tx=0x7fead800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f28774db6fd957 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:38.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.010+0000 7feae5d9b700 1 -- 172.21.15.112:0/714035950 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead800e070 con 0x7feae806e150 2023-08-27T11:18:38.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.011+0000 7feae5d9b700 1 -- 172.21.15.112:0/714035950 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead800b9c0 con 0x7feae806e150 2023-08-27T11:18:38.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.011+0000 7feae5d9b700 1 -- 172.21.15.112:0/714035950 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead800bd70 con 0x7feae806e150 2023-08-27T11:18:38.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.011+0000 7feaed0cb700 1 -- 172.21.15.112:0/714035950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae806e150 msgr2=0x7feae80701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.011+0000 7feaed0cb700 1 --2- 172.21.15.112:0/714035950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae806e150 0x7feae80701a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7feae8067720 tx=0x7fead800b330 comp rx=0 tx=0).stop 2023-08-27T11:18:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.011+0000 7feaed0cb700 1 -- 172.21.15.112:0/714035950 shutdown_connections 2023-08-27T11:18:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.011+0000 7feaed0cb700 1 --2- 172.21.15.112:0/714035950 >> v2:172.21.15.138:3300/0 conn(0x7feae815d490 0x7feae815d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.011+0000 7feaed0cb700 1 --2- 172.21.15.112:0/714035950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae806e150 0x7feae80701a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.011+0000 7feaed0cb700 1 --2- 172.21.15.112:0/714035950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae8066450 0x7feae815ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.011+0000 7feaed0cb700 1 -- 172.21.15.112:0/714035950 >> 172.21.15.112:0/714035950 conn(0x7feae80b1380 msgr2=0x7feae80b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.011+0000 7feaed0cb700 1 -- 172.21.15.112:0/714035950 shutdown_connections 2023-08-27T11:18:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.011+0000 7feaed0cb700 1 -- 172.21.15.112:0/714035950 wait complete. 2023-08-27T11:18:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.013+0000 7feaed0cb700 1 Processor -- start 2023-08-27T11:18:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.013+0000 7feaed0cb700 1 -- start start 2023-08-27T11:18:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feaed0cb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8066450 0x7feae8151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feaed0cb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae806e150 0x7feae814e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feaed0cb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae815d490 0x7feae8160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feaed0cb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae81f39f0 con 0x7feae8066450 2023-08-27T11:18:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feaed0cb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae81f2e70 con 0x7feae806e150 2023-08-27T11:18:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feaed0cb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae81f3150 con 0x7feae815d490 2023-08-27T11:18:38.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feae6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8066450 0x7feae8151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:38.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feae6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8066450 0x7feae8151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57764/0 (socket says 172.21.15.112:57764) 2023-08-27T11:18:38.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feae6d9d700 1 -- 172.21.15.112:0/1704746651 learned_addr learned my addr 172.21.15.112:0/1704746651 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:38.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feae759e700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae815d490 0x7feae8160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:38.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feae659c700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae806e150 0x7feae814e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feae6d9d700 1 -- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae815d490 msgr2=0x7feae8160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feae6d9d700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae815d490 0x7feae8160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feae6d9d700 1 -- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae806e150 msgr2=0x7feae814e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feae6d9d700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae806e150 0x7feae814e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.014+0000 7feae6d9d700 1 -- 172.21.15.112:0/1704746651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead8009b30 con 0x7feae8066450 2023-08-27T11:18:38.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.015+0000 7feae6d9d700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8066450 0x7feae8151c80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7feae000d800 tx=0x7feae000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:38.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.015+0000 7feadf7fe700 1 -- 172.21.15.112:0/1704746651 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae00125f0 con 0x7feae8066450 2023-08-27T11:18:38.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.015+0000 7feadf7fe700 1 -- 172.21.15.112:0/1704746651 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae0012750 con 0x7feae8066450 2023-08-27T11:18:38.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.015+0000 7feaed0cb700 1 -- 172.21.15.112:0/1704746651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae81f3430 con 0x7feae8066450 2023-08-27T11:18:38.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.015+0000 7feadf7fe700 1 -- 172.21.15.112:0/1704746651 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0012b00 con 0x7feae8066450 2023-08-27T11:18:38.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.015+0000 7feaed0cb700 1 -- 172.21.15.112:0/1704746651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae81f29d0 con 0x7feae8066450 2023-08-27T11:18:38.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.016+0000 7feaed0cb700 1 -- 172.21.15.112:0/1704746651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae81f2bf0 con 0x7feae8066450 2023-08-27T11:18:38.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.016+0000 7feadf7fe700 1 -- 172.21.15.112:0/1704746651 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7feae0012c60 con 0x7feae8066450 2023-08-27T11:18:38.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.016+0000 7feadf7fe700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feac806a0e0 0x7feac806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:38.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.017+0000 7feadf7fe700 1 -- 172.21.15.112:0/1704746651 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7feae0088220 con 0x7feae8066450 2023-08-27T11:18:38.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.017+0000 7feae659c700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feac806a0e0 0x7feac806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:38.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.017+0000 7feae659c700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feac806a0e0 0x7feac806c5a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feae815c250 tx=0x7fead0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:38.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.021+0000 7feadf7fe700 1 -- 172.21.15.112:0/1704746651 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7feae0053c40 con 0x7feae8066450 2023-08-27T11:18:38.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.225+0000 7feaed0cb700 1 -- 172.21.15.112:0/1704746651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7feae806fd20 con 0x7feae8066450 2023-08-27T11:18:38.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.871+0000 7feadf7fe700 1 -- 172.21.15.112:0/1704746651 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v45) v1 ==== 97+0+0 (secure 0 0 0) 0x7feae0057260 con 0x7feae8066450 2023-08-27T11:18:38.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.874+0000 7feaed0cb700 1 -- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feac806a0e0 msgr2=0x7feac806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:38.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.874+0000 7feaed0cb700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feac806a0e0 0x7feac806c5a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feae815c250 tx=0x7fead0008040 comp rx=0 tx=0).stop 2023-08-27T11:18:38.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.874+0000 7feaed0cb700 1 -- 172.21.15.112:0/1704746651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8066450 msgr2=0x7feae8151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:38.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.874+0000 7feaed0cb700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8066450 0x7feae8151c80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7feae000d800 tx=0x7feae000db10 comp rx=0 tx=0).stop 2023-08-27T11:18:38.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.874+0000 7feaed0cb700 1 -- 172.21.15.112:0/1704746651 shutdown_connections 2023-08-27T11:18:38.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.874+0000 7feaed0cb700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feac806a0e0 0x7feac806c5a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:38.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.874+0000 7feaed0cb700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae815d490 0x7feae8160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:38.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.874+0000 7feaed0cb700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8066450 0x7feae8151c80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:38.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.874+0000 7feaed0cb700 1 --2- 172.21.15.112:0/1704746651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae806e150 0x7feae814e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:38.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.874+0000 7feaed0cb700 1 -- 172.21.15.112:0/1704746651 >> 172.21.15.112:0/1704746651 conn(0x7feae80b1380 msgr2=0x7feae80af540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:38.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.874+0000 7feaed0cb700 1 -- 172.21.15.112:0/1704746651 shutdown_connections 2023-08-27T11:18:38.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:38.874+0000 7feaed0cb700 1 -- 172.21.15.112:0/1704746651 wait complete. 2023-08-27T11:18:38.878 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2023-08-27T11:18:38.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:18:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.010+0000 7f217bbf9700 1 Processor -- start 2023-08-27T11:18:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.010+0000 7f217bbf9700 1 -- start start 2023-08-27T11:18:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f217bbf9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f21740b1400 0x7f21740cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f217bbf9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21740b17e0 0x7f217406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f217bbf9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2174161850 con 0x7f2174161ce0 2023-08-27T11:18:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f217bbf9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217406e7d0 con 0x7f21740b1400 2023-08-27T11:18:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f217bbf9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21740cfc10 con 0x7f21740b17e0 2023-08-27T11:18:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f2179995700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f21740b1400 0x7f21740cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f217a196700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2174161ce0 0x7f217406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52160/0 (socket says 172.21.15.112:52160) 2023-08-27T11:18:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f217a196700 1 -- 172.21.15.112:0/4107974194 learned_addr learned my addr 172.21.15.112:0/4107974194 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f2179995700 1 -- 172.21.15.112:0/4107974194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21740b17e0 msgr2=0x7f217406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f2179194700 1 --2- 172.21.15.112:0/4107974194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21740b17e0 0x7f217406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f2179995700 1 --2- 172.21.15.112:0/4107974194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21740b17e0 0x7f217406e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f2178993700 1 -- 172.21.15.112:0/4107974194 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476496150 0 0) 0x7f2174161850 con 0x7f2174161ce0 2023-08-27T11:18:39.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f2179995700 1 -- 172.21.15.112:0/4107974194 >> v1:172.21.15.112:6789/0 conn(0x7f2174161ce0 legacy=0x7f217406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f2179995700 1 -- 172.21.15.112:0/4107974194 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2174161200 con 0x7f21740b1400 2023-08-27T11:18:39.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f2179995700 1 --2- 172.21.15.112:0/4107974194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21740b1400 0x7f21740cf5a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2170009840 tx=0x7f2170009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106d8f91d02388b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.011+0000 7f2178993700 1 -- 172.21.15.112:0/4107974194 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170011830 con 0x7f21740b1400 2023-08-27T11:18:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.012+0000 7f2178993700 1 -- 172.21.15.112:0/4107974194 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2170011990 con 0x7f21740b1400 2023-08-27T11:18:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.012+0000 7f2178993700 1 -- 172.21.15.112:0/4107974194 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170011d20 con 0x7f21740b1400 2023-08-27T11:18:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.012+0000 7f217bbf9700 1 -- 172.21.15.112:0/4107974194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21740b1400 msgr2=0x7f21740cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.012+0000 7f217bbf9700 1 --2- 172.21.15.112:0/4107974194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21740b1400 0x7f21740cf5a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2170009840 tx=0x7f2170009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.012+0000 7f217bbf9700 1 -- 172.21.15.112:0/4107974194 shutdown_connections 2023-08-27T11:18:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.012+0000 7f217bbf9700 1 --2- 172.21.15.112:0/4107974194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21740b1400 0x7f21740cf5a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.012+0000 7f217bbf9700 1 --2- 172.21.15.112:0/4107974194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21740b17e0 0x7f217406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.012+0000 7f217bbf9700 1 -- 172.21.15.112:0/4107974194 >> 172.21.15.112:0/4107974194 conn(0x7f21740d0000 msgr2=0x7f21740d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:39.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.012+0000 7f217bbf9700 1 -- 172.21.15.112:0/4107974194 shutdown_connections 2023-08-27T11:18:39.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.012+0000 7f217bbf9700 1 -- 172.21.15.112:0/4107974194 wait complete. 2023-08-27T11:18:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.013+0000 7f217bbf9700 1 Processor -- start 2023-08-27T11:18:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.013+0000 7f217bbf9700 1 -- start start 2023-08-27T11:18:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.014+0000 7f217bbf9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21740b1400 0x7f21741545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.014+0000 7f217bbf9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21740b17e0 0x7f21740c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.014+0000 7f217bbf9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2174161ce0 0x7f217406ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.014+0000 7f217bbf9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21740c58f0 con 0x7f2174161ce0 2023-08-27T11:18:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.014+0000 7f217bbf9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21741f6100 con 0x7f21740b17e0 2023-08-27T11:18:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.014+0000 7f217bbf9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21741f6930 con 0x7f21740b1400 2023-08-27T11:18:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.014+0000 7f217a196700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2174161ce0 0x7f217406ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.014+0000 7f217a196700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2174161ce0 0x7f217406ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57784/0 (socket says 172.21.15.112:57784) 2023-08-27T11:18:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.014+0000 7f217a196700 1 -- 172.21.15.112:0/3364663535 learned_addr learned my addr 172.21.15.112:0/3364663535 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.014+0000 7f217a196700 1 -- 172.21.15.112:0/3364663535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21740b1400 msgr2=0x7f21741545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.014+0000 7f217a196700 1 --2- 172.21.15.112:0/3364663535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21740b1400 0x7f21741545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.014+0000 7f217a196700 1 -- 172.21.15.112:0/3364663535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21740b17e0 msgr2=0x7f21740c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.014+0000 7f217a196700 1 --2- 172.21.15.112:0/3364663535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21740b17e0 0x7f21740c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.015+0000 7f217a196700 1 -- 172.21.15.112:0/3364663535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21700094a0 con 0x7f2174161ce0 2023-08-27T11:18:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.015+0000 7f217a196700 1 --2- 172.21.15.112:0/3364663535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2174161ce0 0x7f217406ad20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f216c00cbd0 tx=0x7f216c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.015+0000 7f216a7fc700 1 -- 172.21.15.112:0/3364663535 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c00ab70 con 0x7f2174161ce0 2023-08-27T11:18:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.015+0000 7f216a7fc700 1 -- 172.21.15.112:0/3364663535 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c00acf0 con 0x7f2174161ce0 2023-08-27T11:18:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.015+0000 7f217bbf9700 1 -- 172.21.15.112:0/3364663535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21740c9330 con 0x7f2174161ce0 2023-08-27T11:18:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.015+0000 7f216a7fc700 1 -- 172.21.15.112:0/3364663535 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c007310 con 0x7f2174161ce0 2023-08-27T11:18:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.015+0000 7f217bbf9700 1 -- 172.21.15.112:0/3364663535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21741f65c0 con 0x7f2174161ce0 2023-08-27T11:18:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.016+0000 7f217bbf9700 1 -- 172.21.15.112:0/3364663535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2174067a20 con 0x7f2174161ce0 2023-08-27T11:18:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.016+0000 7f216a7fc700 1 -- 172.21.15.112:0/3364663535 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f216c007470 con 0x7f2174161ce0 2023-08-27T11:18:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.017+0000 7f216a7fc700 1 --2- 172.21.15.112:0/3364663535 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f215806a0e0 0x7f215806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.017+0000 7f216a7fc700 1 -- 172.21.15.112:0/3364663535 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f216c088090 con 0x7f2174161ce0 2023-08-27T11:18:39.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.017+0000 7f2179995700 1 --2- 172.21.15.112:0/3364663535 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f215806a0e0 0x7f215806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.017+0000 7f2179995700 1 --2- 172.21.15.112:0/3364663535 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f215806a0e0 0x7f215806c5a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f21700090b0 tx=0x7f2170010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:39.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.021+0000 7f216a7fc700 1 -- 172.21.15.112:0/3364663535 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f216c053a40 con 0x7f2174161ce0 2023-08-27T11:18:39.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.287+0000 7f217bbf9700 1 -- 172.21.15.112:0/3364663535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f217406db60 con 0x7f2174161ce0 2023-08-27T11:18:39.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.288+0000 7f216a7fc700 1 -- 172.21.15.112:0/3364663535 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f216c018090 con 0x7f2174161ce0 2023-08-27T11:18:39.290 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:39.290 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:18:39.290 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2023-08-27T11:18:39.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.291 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2023-08-27T11:18:39.291 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk2bnkLBAA+vxcTeJzucKk9iSF/Zw3Lg==", 2023-08-27T11:18:39.291 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.291 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:18:39.291 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:18:39.292 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:18:39.292 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.292 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.293 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2023-08-27T11:18:39.293 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk3zD5OBAASVnwG6yn/YQzfYyeMakFpA==", 2023-08-27T11:18:39.293 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.293 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:18:39.293 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:18:39.294 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:18:39.294 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.294 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.295 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2023-08-27T11:18:39.295 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtki7wJNBAAsIGgDLM82sWNcKnCDlSBMw==", 2023-08-27T11:18:39.295 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.295 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:18:39.295 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:18:39.296 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:18:39.296 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.296 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.297 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2023-08-27T11:18:39.297 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkCsW/BBAAFTPiTOpBo1L+rLo9px+AjQ==", 2023-08-27T11:18:39.297 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.297 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:18:39.297 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:18:39.298 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:18:39.298 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.298 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.298 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.299 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2023-08-27T11:18:39.299 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB7MOtkR/cSHhAAR8e8VRd+e0ESJkqQDYWbEw==", 2023-08-27T11:18:39.299 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.299 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:18:39.299 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:18:39.299 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.300 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.301 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.301 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2023-08-27T11:18:39.301 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB+MOtksBF3HRAA1OIXiIkhgPDYpQyYPaWOXg==", 2023-08-27T11:18:39.301 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.301 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:18:39.302 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:18:39.302 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.302 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.302 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.303 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2023-08-27T11:18:39.303 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCBMOtkWsrdHBAAkfWbiE6wAFzpt9+BxRh8fw==", 2023-08-27T11:18:39.303 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.303 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:18:39.303 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:18:39.304 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.304 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.304 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.304 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2023-08-27T11:18:39.304 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCEMOtkIkiKHRAARDoh/FH9ZdqUlZZ29Mc/Dw==", 2023-08-27T11:18:39.305 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.305 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:18:39.305 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:18:39.305 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.305 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.306 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2023-08-27T11:18:39.306 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCKMOtkPgDzGxAA+U4kbwD0d4lS8aNPG2CgHw==", 2023-08-27T11:18:39.306 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.307 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:18:39.307 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:18:39.307 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.307 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.308 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2023-08-27T11:18:39.308 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCNMOtkKomSGxAAGAQUcIH2H/uyZIU8vlq1pw==", 2023-08-27T11:18:39.308 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.308 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:18:39.309 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:18:39.309 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.309 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.309 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.310 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2023-08-27T11:18:39.310 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCQMOtkHI2MGhAA7sgbi8JXO8P9wnVvMy3PqA==", 2023-08-27T11:18:39.310 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.311 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:18:39.311 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:18:39.311 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.311 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.312 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.312 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2023-08-27T11:18:39.312 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCTMOtkVHpiGhAA4C4YJMWbQ0mmGgB4i8G/AA==", 2023-08-27T11:18:39.312 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.312 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:18:39.313 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:18:39.313 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.313 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.314 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2023-08-27T11:18:39.314 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA==", 2023-08-27T11:18:39.314 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.314 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:18:39.314 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2023-08-27T11:18:39.315 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:18:39.315 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.315 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.315 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2023-08-27T11:18:39.316 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtk66DbDxAAWZ5StELux0lriJ2gEN9icg==", 2023-08-27T11:18:39.316 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.316 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:18:39.316 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:18:39.316 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:18:39.316 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:18:39.317 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.317 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2023-08-27T11:18:39.317 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkViKJDBAAF2ofwO+78er9OyFfn2bO+g==", 2023-08-27T11:18:39.318 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.318 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:18:39.318 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:18:39.318 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:18:39.318 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:18:39.319 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.319 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.319 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2023-08-27T11:18:39.319 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkwt+QFRAAowWE69VqeRhQz0896rxx9Q==", 2023-08-27T11:18:39.320 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.320 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:18:39.320 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:18:39.320 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:18:39.320 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:18:39.321 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.321 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.322 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2023-08-27T11:18:39.322 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkjDA+DxAAjmOn82AC2OMpKlAVCD6hlg==", 2023-08-27T11:18:39.322 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.322 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:18:39.322 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:18:39.323 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:18:39.323 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.323 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.324 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2023-08-27T11:18:39.324 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkXguHKRAALHwzk8diVXqsMvTKr+AzLQ==", 2023-08-27T11:18:39.324 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.324 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:18:39.324 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:18:39.325 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.325 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:39.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:39.325 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2023-08-27T11:18:39.325 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkrRjBJRAAq3ftmIh81LKadpliVUiJ6g==", 2023-08-27T11:18:39.326 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:18:39.326 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:18:39.326 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:18:39.326 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:18:39.326 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.327 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:39.327 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:18:39.327 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:18:39.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.289+0000 7f217bbf9700 1 -- 172.21.15.112:0/3364663535 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f215806a0e0 msgr2=0x7f215806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.289+0000 7f217bbf9700 1 --2- 172.21.15.112:0/3364663535 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f215806a0e0 0x7f215806c5a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f21700090b0 tx=0x7f2170010000 comp rx=0 tx=0).stop 2023-08-27T11:18:39.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.289+0000 7f217bbf9700 1 -- 172.21.15.112:0/3364663535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2174161ce0 msgr2=0x7f217406ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.289+0000 7f217bbf9700 1 --2- 172.21.15.112:0/3364663535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2174161ce0 0x7f217406ad20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f216c00cbd0 tx=0x7f216c00cf90 comp rx=0 tx=0).stop 2023-08-27T11:18:39.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.289+0000 7f217bbf9700 1 -- 172.21.15.112:0/3364663535 shutdown_connections 2023-08-27T11:18:39.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.289+0000 7f217bbf9700 1 --2- 172.21.15.112:0/3364663535 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f215806a0e0 0x7f215806c5a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.289+0000 7f217bbf9700 1 --2- 172.21.15.112:0/3364663535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2174161ce0 0x7f217406ad20 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.289+0000 7f217bbf9700 1 --2- 172.21.15.112:0/3364663535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21740b1400 0x7f21741545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.290+0000 7f217bbf9700 1 --2- 172.21.15.112:0/3364663535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21740b17e0 0x7f21740c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.290+0000 7f217bbf9700 1 -- 172.21.15.112:0/3364663535 >> 172.21.15.112:0/3364663535 conn(0x7f21740d0000 msgr2=0x7f21740cb630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:39.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.290+0000 7f217bbf9700 1 -- 172.21.15.112:0/3364663535 shutdown_connections 2023-08-27T11:18:39.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.290+0000 7f217bbf9700 1 -- 172.21.15.112:0/3364663535 wait complete. 2023-08-27T11:18:39.332 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:39.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.452+0000 7fcc0c0ed700 1 Processor -- start 2023-08-27T11:18:39.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.452+0000 7fcc0c0ed700 1 -- start start 2023-08-27T11:18:39.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc0c0ed700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcc04143a60 0x7fcc040cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc0c0ed700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcc0415e290 0x7fcc0415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc0c0ed700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc04143e40 con 0x7fcc040b4960 2023-08-27T11:18:39.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc0c0ed700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc040b5340 con 0x7fcc0415e290 2023-08-27T11:18:39.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc0c0ed700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc040ce9a0 con 0x7fcc04143a60 2023-08-27T11:18:39.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc0a68a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcc0415e290 0x7fcc0415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc09688700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fcc040b4960 0x7fcc0415db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52170/0 (socket says 172.21.15.112:52170) 2023-08-27T11:18:39.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc09688700 1 -- 172.21.15.112:0/275190818 learned_addr learned my addr 172.21.15.112:0/275190818 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:39.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc09e89700 1 --2- 172.21.15.112:0/275190818 >> v2:172.21.15.138:3300/0 conn(0x7fcc04143a60 0x7fcc040cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc0a68a700 1 -- 172.21.15.112:0/275190818 >> v2:172.21.15.138:3300/0 conn(0x7fcc04143a60 msgr2=0x7fcc040cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:39.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc0a68a700 1 --2- 172.21.15.112:0/275190818 >> v2:172.21.15.138:3300/0 conn(0x7fcc04143a60 0x7fcc040cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc0a68a700 1 -- 172.21.15.112:0/275190818 >> v1:172.21.15.112:6789/0 conn(0x7fcc040b4960 legacy=0x7fcc0415db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc08e87700 1 -- 172.21.15.112:0/275190818 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528272831 0 0) 0x7fcc04143e40 con 0x7fcc040b4960 2023-08-27T11:18:39.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc0a68a700 1 -- 172.21.15.112:0/275190818 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040cec80 con 0x7fcc0415e290 2023-08-27T11:18:39.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.453+0000 7fcc0a68a700 1 --2- 172.21.15.112:0/275190818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc0415e290 0x7fcc0415b6a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcc040b1820 tx=0x7fcbfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73e6bb9092e2b57 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:39.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.454+0000 7fcc08e87700 1 -- 172.21.15.112:0/275190818 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc00e070 con 0x7fcc0415e290 2023-08-27T11:18:39.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.454+0000 7fcc08e87700 1 -- 172.21.15.112:0/275190818 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc00bbd0 con 0x7fcc0415e290 2023-08-27T11:18:39.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.454+0000 7fcc08e87700 1 -- 172.21.15.112:0/275190818 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc012430 con 0x7fcc0415e290 2023-08-27T11:18:39.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.454+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/275190818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc0415e290 msgr2=0x7fcc0415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.454+0000 7fcc0c0ed700 1 --2- 172.21.15.112:0/275190818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc0415e290 0x7fcc0415b6a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcc040b1820 tx=0x7fcbfc00b330 comp rx=0 tx=0).stop 2023-08-27T11:18:39.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.454+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/275190818 shutdown_connections 2023-08-27T11:18:39.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.454+0000 7fcc0c0ed700 1 --2- 172.21.15.112:0/275190818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc0415e290 0x7fcc0415b6a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.454+0000 7fcc0c0ed700 1 --2- 172.21.15.112:0/275190818 >> v2:172.21.15.138:3300/0 conn(0x7fcc04143a60 0x7fcc040cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.454+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/275190818 >> 172.21.15.112:0/275190818 conn(0x7fcc0406adb0 msgr2=0x7fcc0406b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:39.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.454+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/275190818 shutdown_connections 2023-08-27T11:18:39.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.454+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/275190818 wait complete. 2023-08-27T11:18:39.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.455+0000 7fcc0c0ed700 1 Processor -- start 2023-08-27T11:18:39.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.455+0000 7fcc0c0ed700 1 -- start start 2023-08-27T11:18:39.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.456+0000 7fcc0c0ed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc040b4960 0x7fcc0413e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.456+0000 7fcc0c0ed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc04143a60 0x7fcc040c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.456+0000 7fcc0c0ed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc0415e290 0x7fcc0415b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.456+0000 7fcc0c0ed700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040c57e0 con 0x7fcc0415e290 2023-08-27T11:18:39.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.456+0000 7fcc0c0ed700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc041f3e80 con 0x7fcc040b4960 2023-08-27T11:18:39.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.456+0000 7fcc0c0ed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040c5220 con 0x7fcc04143a60 2023-08-27T11:18:39.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.456+0000 7fcc0a68a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc0415e290 0x7fcc0415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.456+0000 7fcc0a68a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc0415e290 0x7fcc0415b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57798/0 (socket says 172.21.15.112:57798) 2023-08-27T11:18:39.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.456+0000 7fcc0a68a700 1 -- 172.21.15.112:0/2552915907 learned_addr learned my addr 172.21.15.112:0/2552915907 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:39.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.456+0000 7fcc09688700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc04143a60 0x7fcc040c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.456+0000 7fcc09e89700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc040b4960 0x7fcc0413e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.457+0000 7fcc0a68a700 1 -- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc04143a60 msgr2=0x7fcc040c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.457+0000 7fcc0a68a700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc04143a60 0x7fcc040c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.457+0000 7fcc0a68a700 1 -- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc040b4960 msgr2=0x7fcc0413e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.457+0000 7fcc0a68a700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc040b4960 0x7fcc0413e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.457+0000 7fcc0a68a700 1 -- 172.21.15.112:0/2552915907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc009a70 con 0x7fcc0415e290 2023-08-27T11:18:39.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.457+0000 7fcc09688700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc04143a60 0x7fcc040c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.457+0000 7fcc09e89700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc040b4960 0x7fcc0413e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.457+0000 7fcc0a68a700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc0415e290 0x7fcc0415b580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc013000 tx=0x7fcbfc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.457+0000 7fcbfa7fc700 1 -- 172.21.15.112:0/2552915907 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc012430 con 0x7fcc0415e290 2023-08-27T11:18:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.457+0000 7fcbfa7fc700 1 -- 172.21.15.112:0/2552915907 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc020070 con 0x7fcc0415e290 2023-08-27T11:18:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.457+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/2552915907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc040c5500 con 0x7fcc0415e290 2023-08-27T11:18:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.457+0000 7fcbfa7fc700 1 -- 172.21.15.112:0/2552915907 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc004280 con 0x7fcc0415e290 2023-08-27T11:18:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.457+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/2552915907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0413ee50 con 0x7fcc0415e290 2023-08-27T11:18:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.458+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/2552915907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc041f3970 con 0x7fcc0415e290 2023-08-27T11:18:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.458+0000 7fcbfa7fc700 1 -- 172.21.15.112:0/2552915907 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcbfc0043e0 con 0x7fcc0415e290 2023-08-27T11:18:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.459+0000 7fcbfa7fc700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcbe806a0e0 0x7fcbe806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.459+0000 7fcbfa7fc700 1 -- 172.21.15.112:0/2552915907 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcbfc022070 con 0x7fcc0415e290 2023-08-27T11:18:39.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.459+0000 7fcc09e89700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcbe806a0e0 0x7fcbe806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.459+0000 7fcc09e89700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcbe806a0e0 0x7fcbe806c5a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcc00006fd0 tx=0x7fcc00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:39.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.463+0000 7fcbfa7fc700 1 -- 172.21.15.112:0/2552915907 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcc041f3970 con 0x7fcc0415e290 2023-08-27T11:18:39.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.653+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/2552915907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc041f3970 con 0x7fcc0415e290 2023-08-27T11:18:39.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.654+0000 7fcbfa7fc700 1 -- 172.21.15.112:0/2552915907 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+14605 (secure 0 0 0) 0x7fcc041f3970 con 0x7fcc0415e290 2023-08-27T11:18:39.658 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:39.658 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":47,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:38.879086+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:39.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.657+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcbe806a0e0 msgr2=0x7fcbe806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.657+0000 7fcc0c0ed700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcbe806a0e0 0x7fcbe806c5a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcc00006fd0 tx=0x7fcc00008040 comp rx=0 tx=0).stop 2023-08-27T11:18:39.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.657+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/2552915907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc0415e290 msgr2=0x7fcc0415b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.657+0000 7fcc0c0ed700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc0415e290 0x7fcc0415b580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc013000 tx=0x7fcbfc004ad0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.657+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/2552915907 shutdown_connections 2023-08-27T11:18:39.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.657+0000 7fcc0c0ed700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcbe806a0e0 0x7fcbe806c5a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.657+0000 7fcc0c0ed700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc0415e290 0x7fcc0415b580 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.657+0000 7fcc0c0ed700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc040b4960 0x7fcc0413e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.657+0000 7fcc0c0ed700 1 --2- 172.21.15.112:0/2552915907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc04143a60 0x7fcc040c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.657+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/2552915907 >> 172.21.15.112:0/2552915907 conn(0x7fcc0406adb0 msgr2=0x7fcc04142d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:39.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.657+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/2552915907 shutdown_connections 2023-08-27T11:18:39.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.657+0000 7fcc0c0ed700 1 -- 172.21.15.112:0/2552915907 wait complete. 2023-08-27T11:18:39.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:18:39.671 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:18:39.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.793+0000 7f0439703700 1 Processor -- start 2023-08-27T11:18:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.793+0000 7f0439703700 1 -- start start 2023-08-27T11:18:39.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.794+0000 7f0439703700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0434157b80 0x7f04340d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.794+0000 7f0439703700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04340ce030 0x7f04340706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.794+0000 7f0439703700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04340d2850 con 0x7f04340d23f0 2023-08-27T11:18:39.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.794+0000 7f0439703700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f043415ddd0 con 0x7f0434157b80 2023-08-27T11:18:39.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.794+0000 7f0439703700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04340cf0f0 con 0x7f04340ce030 2023-08-27T11:18:39.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.794+0000 7f04337fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04340ce030 0x7f04340706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.794+0000 7f0432ffd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0434157b80 0x7f04340d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.794+0000 7f04337fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04340ce030 0x7f04340706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:41228/0 (socket says 172.21.15.112:41228) 2023-08-27T11:18:39.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.794+0000 7f04337fe700 1 -- 172.21.15.112:0/2416891252 learned_addr learned my addr 172.21.15.112:0/2416891252 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:39.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.795+0000 7f04337fe700 1 -- 172.21.15.112:0/2416891252 >> v2:172.21.15.138:3300/0 conn(0x7f0434157b80 msgr2=0x7f04340d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:39.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.795+0000 7f04337fe700 1 --2- 172.21.15.112:0/2416891252 >> v2:172.21.15.138:3300/0 conn(0x7f0434157b80 0x7f04340d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.795+0000 7f04337fe700 1 -- 172.21.15.112:0/2416891252 >> v1:172.21.15.138:6789/0 conn(0x7f04340d23f0 legacy=0x7f04340cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.795+0000 7f04337fe700 1 -- 172.21.15.112:0/2416891252 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04340ce570 con 0x7f04340ce030 2023-08-27T11:18:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.795+0000 7f0432ffd700 1 --2- 172.21.15.112:0/2416891252 >> v2:172.21.15.138:3300/0 conn(0x7f0434157b80 0x7f04340d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.795+0000 7f04337fe700 1 --2- 172.21.15.112:0/2416891252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04340ce030 0x7f04340706d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f042c009e50 tx=0x7f042c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49992b712782b3c1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.795+0000 7f0431ffb700 1 -- 172.21.15.112:0/2416891252 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c00e070 con 0x7f04340ce030 2023-08-27T11:18:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.795+0000 7f0431ffb700 1 -- 172.21.15.112:0/2416891252 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c00b910 con 0x7f04340ce030 2023-08-27T11:18:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.795+0000 7f0431ffb700 1 -- 172.21.15.112:0/2416891252 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c00bcc0 con 0x7f04340ce030 2023-08-27T11:18:39.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.796+0000 7f0439703700 1 -- 172.21.15.112:0/2416891252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04340ce030 msgr2=0x7f04340706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.796+0000 7f0439703700 1 --2- 172.21.15.112:0/2416891252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04340ce030 0x7f04340706d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f042c009e50 tx=0x7f042c00b330 comp rx=0 tx=0).stop 2023-08-27T11:18:39.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.796+0000 7f0439703700 1 -- 172.21.15.112:0/2416891252 shutdown_connections 2023-08-27T11:18:39.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.796+0000 7f0439703700 1 --2- 172.21.15.112:0/2416891252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04340ce030 0x7f04340706d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.796+0000 7f0439703700 1 --2- 172.21.15.112:0/2416891252 >> v2:172.21.15.138:3300/0 conn(0x7f0434157b80 0x7f04340d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.796+0000 7f0439703700 1 -- 172.21.15.112:0/2416891252 >> 172.21.15.112:0/2416891252 conn(0x7f04340cfd50 msgr2=0x7f04340d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:39.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.796+0000 7f0439703700 1 -- 172.21.15.112:0/2416891252 shutdown_connections 2023-08-27T11:18:39.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.796+0000 7f0439703700 1 -- 172.21.15.112:0/2416891252 wait complete. 2023-08-27T11:18:39.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.797+0000 7f0439703700 1 Processor -- start 2023-08-27T11:18:39.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.797+0000 7f0439703700 1 -- start start 2023-08-27T11:18:39.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f0439703700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04340ce030 0x7f043415c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f0439703700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04340d23f0 0x7f04340cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f0439703700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0434157b80 0x7f0434157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f0439703700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043415cda0 con 0x7f0434157b80 2023-08-27T11:18:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f0439703700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04340cbd60 con 0x7f04340d23f0 2023-08-27T11:18:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f0439703700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04340cc040 con 0x7f04340ce030 2023-08-27T11:18:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f04337fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0434157b80 0x7f0434157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f04337fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0434157b80 0x7f0434157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57812/0 (socket says 172.21.15.112:57812) 2023-08-27T11:18:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f04337fe700 1 -- 172.21.15.112:0/2699549057 learned_addr learned my addr 172.21.15.112:0/2699549057 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f0432ffd700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04340ce030 0x7f043415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f04327fc700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04340d23f0 0x7f04340cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f04337fe700 1 -- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04340ce030 msgr2=0x7f043415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f04337fe700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04340ce030 0x7f043415c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f04337fe700 1 -- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04340d23f0 msgr2=0x7f04340cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f04337fe700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04340d23f0 0x7f04340cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f04337fe700 1 -- 172.21.15.112:0/2699549057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04280094a0 con 0x7f0434157b80 2023-08-27T11:18:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.798+0000 7f04337fe700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0434157b80 0x7f0434157f60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f042c012000 tx=0x7f042c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.799+0000 7f041b7fe700 1 -- 172.21.15.112:0/2699549057 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c004800 con 0x7f0434157b80 2023-08-27T11:18:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.799+0000 7f041b7fe700 1 -- 172.21.15.112:0/2699549057 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c004960 con 0x7f0434157b80 2023-08-27T11:18:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.799+0000 7f0439703700 1 -- 172.21.15.112:0/2699549057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042c009b30 con 0x7f0434157b80 2023-08-27T11:18:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.799+0000 7f041b7fe700 1 -- 172.21.15.112:0/2699549057 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c01c680 con 0x7f0434157b80 2023-08-27T11:18:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.799+0000 7f0439703700 1 -- 172.21.15.112:0/2699549057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04340cc600 con 0x7f0434157b80 2023-08-27T11:18:39.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.800+0000 7f0439703700 1 -- 172.21.15.112:0/2699549057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0434067e90 con 0x7f0434157b80 2023-08-27T11:18:39.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.800+0000 7f041b7fe700 1 -- 172.21.15.112:0/2699549057 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f042c01c7e0 con 0x7f0434157b80 2023-08-27T11:18:39.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.800+0000 7f041b7fe700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f042006a0e0 0x7f042006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:39.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.801+0000 7f041b7fe700 1 -- 172.21.15.112:0/2699549057 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f042c0899a0 con 0x7f0434157b80 2023-08-27T11:18:39.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.801+0000 7f0432ffd700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f042006a0e0 0x7f042006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:39.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.801+0000 7f0432ffd700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f042006a0e0 0x7f042006c5a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0428009cd0 tx=0x7f0428009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:39.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.805+0000 7f041b7fe700 1 -- 172.21.15.112:0/2699549057 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f042c055590 con 0x7f0434157b80 2023-08-27T11:18:39.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.994+0000 7f0439703700 1 -- 172.21.15.112:0/2699549057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f04340cede0 con 0x7f0434157b80 2023-08-27T11:18:39.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.994+0000 7f041b7fe700 1 -- 172.21.15.112:0/2699549057 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v47) v1 ==== 92+0+10 (secure 0 0 0) 0x7f042c02a020 con 0x7f0434157b80 2023-08-27T11:18:39.996 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:18:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.996+0000 7f0439703700 1 -- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f042006a0e0 msgr2=0x7f042006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.996+0000 7f0439703700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f042006a0e0 0x7f042006c5a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0428009cd0 tx=0x7f0428009210 comp rx=0 tx=0).stop 2023-08-27T11:18:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.996+0000 7f0439703700 1 -- 172.21.15.112:0/2699549057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0434157b80 msgr2=0x7f0434157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.996+0000 7f0439703700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0434157b80 0x7f0434157f60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f042c012000 tx=0x7f042c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.996+0000 7f0439703700 1 -- 172.21.15.112:0/2699549057 shutdown_connections 2023-08-27T11:18:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.996+0000 7f0439703700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f042006a0e0 0x7f042006c5a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.996+0000 7f0439703700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0434157b80 0x7f0434157f60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.996+0000 7f0439703700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04340ce030 0x7f043415c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.996+0000 7f0439703700 1 --2- 172.21.15.112:0/2699549057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04340d23f0 0x7f04340cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.996+0000 7f0439703700 1 -- 172.21.15.112:0/2699549057 >> 172.21.15.112:0/2699549057 conn(0x7f04340cfd50 msgr2=0x7f04340c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.996+0000 7f0439703700 1 -- 172.21.15.112:0/2699549057 shutdown_connections 2023-08-27T11:18:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:39.996+0000 7f0439703700 1 -- 172.21.15.112:0/2699549057 wait complete. 2023-08-27T11:18:40.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:18:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.131+0000 7fdd65a16700 1 Processor -- start 2023-08-27T11:18:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.131+0000 7fdd65a16700 1 -- start start 2023-08-27T11:18:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.131+0000 7fdd65a16700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd600cf5a0 0x7fdd6006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.131+0000 7fdd65a16700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd6006fcd0 0x7fdd600ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.131+0000 7fdd65a16700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd600cfd10 con 0x7fdd6015d440 2023-08-27T11:18:40.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.131+0000 7fdd65a16700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd60070160 con 0x7fdd6006fcd0 2023-08-27T11:18:40.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.131+0000 7fdd65a16700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600b1ed0 con 0x7fdd600cf5a0 2023-08-27T11:18:40.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.132+0000 7fdd5e7fc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd600cf5a0 0x7fdd6006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.132+0000 7fdd5f7fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd6006fcd0 0x7fdd600ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.132+0000 7fdd5effd700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fdd6015d440 0x7fdd600cdbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:59724/0 (socket says 172.21.15.112:59724) 2023-08-27T11:18:40.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.132+0000 7fdd5effd700 1 -- 172.21.15.112:0/254717669 learned_addr learned my addr 172.21.15.112:0/254717669 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:40.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.132+0000 7fdd5f7fe700 1 -- 172.21.15.112:0/254717669 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd600cf5a0 msgr2=0x7fdd6006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:40.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.132+0000 7fdd5f7fe700 1 --2- 172.21.15.112:0/254717669 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd600cf5a0 0x7fdd6006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.132+0000 7fdd5f7fe700 1 -- 172.21.15.112:0/254717669 >> v1:172.21.15.138:6789/0 conn(0x7fdd6015d440 legacy=0x7fdd600cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.132+0000 7fdd5f7fe700 1 -- 172.21.15.112:0/254717669 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600b21b0 con 0x7fdd6006fcd0 2023-08-27T11:18:40.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.132+0000 7fdd5e7fc700 1 --2- 172.21.15.112:0/254717669 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd600cf5a0 0x7fdd6006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:40.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.132+0000 7fdd5f7fe700 1 --2- 172.21.15.112:0/254717669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6006fcd0 0x7fdd600ca6e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009d90 tx=0x7fdd5800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c9d9000663e4dd6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:40.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.133+0000 7fdd5dffb700 1 -- 172.21.15.112:0/254717669 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800e070 con 0x7fdd6006fcd0 2023-08-27T11:18:40.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.133+0000 7fdd5dffb700 1 -- 172.21.15.112:0/254717669 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5800bb20 con 0x7fdd6006fcd0 2023-08-27T11:18:40.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.133+0000 7fdd5dffb700 1 -- 172.21.15.112:0/254717669 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58012430 con 0x7fdd6006fcd0 2023-08-27T11:18:40.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.133+0000 7fdd65a16700 1 -- 172.21.15.112:0/254717669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6006fcd0 msgr2=0x7fdd600ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.133+0000 7fdd65a16700 1 --2- 172.21.15.112:0/254717669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6006fcd0 0x7fdd600ca6e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009d90 tx=0x7fdd5800b330 comp rx=0 tx=0).stop 2023-08-27T11:18:40.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.133+0000 7fdd65a16700 1 -- 172.21.15.112:0/254717669 shutdown_connections 2023-08-27T11:18:40.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.133+0000 7fdd65a16700 1 --2- 172.21.15.112:0/254717669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6006fcd0 0x7fdd600ca6e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.133+0000 7fdd65a16700 1 --2- 172.21.15.112:0/254717669 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd600cf5a0 0x7fdd6006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.133+0000 7fdd65a16700 1 -- 172.21.15.112:0/254717669 >> 172.21.15.112:0/254717669 conn(0x7fdd600cca80 msgr2=0x7fdd600cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:40.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.133+0000 7fdd65a16700 1 -- 172.21.15.112:0/254717669 shutdown_connections 2023-08-27T11:18:40.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.133+0000 7fdd65a16700 1 -- 172.21.15.112:0/254717669 wait complete. 2023-08-27T11:18:40.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.134+0000 7fdd65a16700 1 Processor -- start 2023-08-27T11:18:40.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.135+0000 7fdd65a16700 1 -- start start 2023-08-27T11:18:40.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.135+0000 7fdd65a16700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6006fcd0 0x7fdd60153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.135+0000 7fdd65a16700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd600cf5a0 0x7fdd600c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.135+0000 7fdd65a16700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd6015d440 0x7fdd600c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.135+0000 7fdd65a16700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60154520 con 0x7fdd600cf5a0 2023-08-27T11:18:40.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.135+0000 7fdd65a16700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6015f5f0 con 0x7fdd6006fcd0 2023-08-27T11:18:40.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.135+0000 7fdd65a16700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6015f8d0 con 0x7fdd6015d440 2023-08-27T11:18:40.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.135+0000 7fdd5e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd600cf5a0 0x7fdd600c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.135+0000 7fdd5e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd600cf5a0 0x7fdd600c5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57826/0 (socket says 172.21.15.112:57826) 2023-08-27T11:18:40.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.135+0000 7fdd5e7fc700 1 -- 172.21.15.112:0/1367075028 learned_addr learned my addr 172.21.15.112:0/1367075028 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:40.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd5effd700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6006fcd0 0x7fdd60153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd5f7fe700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd6015d440 0x7fdd600c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd5e7fc700 1 -- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd6015d440 msgr2=0x7fdd600c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd5e7fc700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd6015d440 0x7fdd600c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd5e7fc700 1 -- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6006fcd0 msgr2=0x7fdd60153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd5e7fc700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6006fcd0 0x7fdd60153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd5e7fc700 1 -- 172.21.15.112:0/1367075028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd540094a0 con 0x7fdd600cf5a0 2023-08-27T11:18:40.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd5f7fe700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd6015d440 0x7fdd600c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:40.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd5effd700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6006fcd0 0x7fdd60153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:40.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd5e7fc700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd600cf5a0 0x7fdd600c5e50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400ebe0 tx=0x7fdd54009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:40.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd47fff700 1 -- 172.21.15.112:0/1367075028 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400c910 con 0x7fdd600cf5a0 2023-08-27T11:18:40.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd47fff700 1 -- 172.21.15.112:0/1367075028 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5400ca70 con 0x7fdd600cf5a0 2023-08-27T11:18:40.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd65a16700 1 -- 172.21.15.112:0/1367075028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd58009a70 con 0x7fdd600cf5a0 2023-08-27T11:18:40.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd47fff700 1 -- 172.21.15.112:0/1367075028 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400cde0 con 0x7fdd600cf5a0 2023-08-27T11:18:40.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.136+0000 7fdd65a16700 1 -- 172.21.15.112:0/1367075028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6015fe90 con 0x7fdd600cf5a0 2023-08-27T11:18:40.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.137+0000 7fdd65a16700 1 -- 172.21.15.112:0/1367075028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd601604e0 con 0x7fdd600cf5a0 2023-08-27T11:18:40.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.137+0000 7fdd47fff700 1 -- 172.21.15.112:0/1367075028 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdd54013470 con 0x7fdd600cf5a0 2023-08-27T11:18:40.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.138+0000 7fdd47fff700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdd4c06a0e0 0x7fdd4c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.138+0000 7fdd47fff700 1 -- 172.21.15.112:0/1367075028 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdd54088460 con 0x7fdd600cf5a0 2023-08-27T11:18:40.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.138+0000 7fdd5effd700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdd4c06a0e0 0x7fdd4c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.138+0000 7fdd5effd700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdd4c06a0e0 0x7fdd4c06c5a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdd50007fd0 tx=0x7fdd50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:40.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.142+0000 7fdd47fff700 1 -- 172.21.15.112:0/1367075028 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdd54053e20 con 0x7fdd600cf5a0 2023-08-27T11:18:40.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.331+0000 7fdd65a16700 1 -- 172.21.15.112:0/1367075028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd60067a20 con 0x7fdd600cf5a0 2023-08-27T11:18:40.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.331+0000 7fdd47fff700 1 -- 172.21.15.112:0/1367075028 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd54057440 con 0x7fdd600cf5a0 2023-08-27T11:18:40.333 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:18:40.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.334+0000 7fdd65a16700 1 -- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdd4c06a0e0 msgr2=0x7fdd4c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.334+0000 7fdd65a16700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdd4c06a0e0 0x7fdd4c06c5a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdd50007fd0 tx=0x7fdd50009040 comp rx=0 tx=0).stop 2023-08-27T11:18:40.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.334+0000 7fdd65a16700 1 -- 172.21.15.112:0/1367075028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd600cf5a0 msgr2=0x7fdd600c5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.334+0000 7fdd65a16700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd600cf5a0 0x7fdd600c5e50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400ebe0 tx=0x7fdd54009ec0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.334+0000 7fdd65a16700 1 -- 172.21.15.112:0/1367075028 shutdown_connections 2023-08-27T11:18:40.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.334+0000 7fdd65a16700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdd4c06a0e0 0x7fdd4c06c5a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.334+0000 7fdd65a16700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd6015d440 0x7fdd600c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.334+0000 7fdd65a16700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6006fcd0 0x7fdd60153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.334+0000 7fdd65a16700 1 --2- 172.21.15.112:0/1367075028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd600cf5a0 0x7fdd600c5e50 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.334+0000 7fdd65a16700 1 -- 172.21.15.112:0/1367075028 >> 172.21.15.112:0/1367075028 conn(0x7fdd600cca80 msgr2=0x7fdd600cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:40.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.334+0000 7fdd65a16700 1 -- 172.21.15.112:0/1367075028 shutdown_connections 2023-08-27T11:18:40.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.334+0000 7fdd65a16700 1 -- 172.21.15.112:0/1367075028 wait complete. 2023-08-27T11:18:40.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:18:40.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.465+0000 7f61a667b700 1 Processor -- start 2023-08-27T11:18:40.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.465+0000 7f61a667b700 1 -- start start 2023-08-27T11:18:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.466+0000 7f61a667b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f61a00ce260 0x7f61a006fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.466+0000 7f61a667b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a006cb00 con 0x7f61a013f910 2023-08-27T11:18:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.466+0000 7f61a667b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a00707a0 con 0x7f61a006ff70 2023-08-27T11:18:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.466+0000 7f61a667b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a0070a80 con 0x7f61a00ce260 2023-08-27T11:18:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.466+0000 7f619f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f61a00ce260 0x7f61a006fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.466+0000 7f619ffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f61a013f910 0x7f61a00cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52214/0 (socket says 172.21.15.112:52214) 2023-08-27T11:18:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.466+0000 7f619f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f61a00ce260 0x7f61a006fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57834/0 (socket says 172.21.15.112:57834) 2023-08-27T11:18:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.466+0000 7f619ffff700 1 -- 172.21.15.112:0/3857783384 learned_addr learned my addr 172.21.15.112:0/3857783384 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:40.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.467+0000 7f619f7fe700 1 -- 172.21.15.112:0/3857783384 >> v1:172.21.15.138:6789/0 conn(0x7f61a006ff70 legacy=0x7f61a006cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.467+0000 7f619f7fe700 1 -- 172.21.15.112:0/3857783384 >> v1:172.21.15.112:6789/0 conn(0x7f61a013f910 legacy=0x7f61a00cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.467+0000 7f619f7fe700 1 -- 172.21.15.112:0/3857783384 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0070d60 con 0x7f61a00ce260 2023-08-27T11:18:40.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.467+0000 7f619effd700 1 -- 172.21.15.112:0/3857783384 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645733213 0 0) 0x7f61a006cb00 con 0x7f61a013f910 2023-08-27T11:18:40.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.467+0000 7f619f7fe700 1 --2- 172.21.15.112:0/3857783384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61a00ce260 0x7f61a006fa30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f619400a9e0 tx=0x7f619400acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8c45ba743af44b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:40.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.467+0000 7f619effd700 1 -- 172.21.15.112:0/3857783384 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619400cb70 con 0x7f61a00ce260 2023-08-27T11:18:40.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.467+0000 7f619effd700 1 -- 172.21.15.112:0/3857783384 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619400ccd0 con 0x7f61a00ce260 2023-08-27T11:18:40.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.467+0000 7f619effd700 1 -- 172.21.15.112:0/3857783384 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194018620 con 0x7f61a00ce260 2023-08-27T11:18:40.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.468+0000 7f61a667b700 1 -- 172.21.15.112:0/3857783384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61a00ce260 msgr2=0x7f61a006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.468+0000 7f61a667b700 1 --2- 172.21.15.112:0/3857783384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61a00ce260 0x7f61a006fa30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f619400a9e0 tx=0x7f619400acf0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.468+0000 7f61a667b700 1 -- 172.21.15.112:0/3857783384 shutdown_connections 2023-08-27T11:18:40.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.468+0000 7f61a667b700 1 --2- 172.21.15.112:0/3857783384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61a00ce260 0x7f61a006fa30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.468+0000 7f61a667b700 1 -- 172.21.15.112:0/3857783384 >> 172.21.15.112:0/3857783384 conn(0x7f61a00b0bf0 msgr2=0x7f61a00b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:40.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.468+0000 7f61a667b700 1 -- 172.21.15.112:0/3857783384 shutdown_connections 2023-08-27T11:18:40.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.468+0000 7f61a667b700 1 -- 172.21.15.112:0/3857783384 wait complete. 2023-08-27T11:18:40.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.469+0000 7f61a667b700 1 Processor -- start 2023-08-27T11:18:40.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.469+0000 7f61a667b700 1 -- start start 2023-08-27T11:18:40.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a667b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61a006ff70 0x7f61a0139700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a667b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a00ce260 0x7f61a0139c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a667b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61a013f910 0x7f61a01e3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a667b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01e48b0 con 0x7f61a013f910 2023-08-27T11:18:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a667b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01e4b90 con 0x7f61a00ce260 2023-08-27T11:18:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a667b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a015abc0 con 0x7f61a006ff70 2023-08-27T11:18:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a4c18700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61a013f910 0x7f61a01e3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a4c18700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61a013f910 0x7f61a01e3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57842/0 (socket says 172.21.15.112:57842) 2023-08-27T11:18:40.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a4c18700 1 -- 172.21.15.112:0/360369161 learned_addr learned my addr 172.21.15.112:0/360369161 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:40.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f619ffff700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61a006ff70 0x7f61a0139700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f619f7fe700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a00ce260 0x7f61a0139c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a4c18700 1 -- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61a006ff70 msgr2=0x7f61a0139700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a4c18700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61a006ff70 0x7f61a0139700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a4c18700 1 -- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a00ce260 msgr2=0x7f61a0139c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a4c18700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a00ce260 0x7f61a0139c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a4c18700 1 -- 172.21.15.112:0/360369161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619400a640 con 0x7f61a013f910 2023-08-27T11:18:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.470+0000 7f61a4c18700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61a013f910 0x7f61a01e3c30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f619800edf0 tx=0x7f619800c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:40.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.471+0000 7f619cff9700 1 -- 172.21.15.112:0/360369161 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198003a00 con 0x7f61a013f910 2023-08-27T11:18:40.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.471+0000 7f619cff9700 1 -- 172.21.15.112:0/360369161 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6198003b60 con 0x7f61a013f910 2023-08-27T11:18:40.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.471+0000 7f61a667b700 1 -- 172.21.15.112:0/360369161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a015aea0 con 0x7f61a013f910 2023-08-27T11:18:40.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.471+0000 7f619cff9700 1 -- 172.21.15.112:0/360369161 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61980050d0 con 0x7f61a013f910 2023-08-27T11:18:40.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.471+0000 7f61a667b700 1 -- 172.21.15.112:0/360369161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a0158b00 con 0x7f61a013f910 2023-08-27T11:18:40.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.472+0000 7f61a667b700 1 -- 172.21.15.112:0/360369161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a015b3c0 con 0x7f61a013f910 2023-08-27T11:18:40.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.472+0000 7f619cff9700 1 -- 172.21.15.112:0/360369161 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6198005230 con 0x7f61a013f910 2023-08-27T11:18:40.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.472+0000 7f619cff9700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f618c06a0e0 0x7f618c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.472+0000 7f619cff9700 1 -- 172.21.15.112:0/360369161 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f619808aa70 con 0x7f61a013f910 2023-08-27T11:18:40.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.472+0000 7f619ffff700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f618c06a0e0 0x7f618c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.473+0000 7f619ffff700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f618c06a0e0 0x7f618c06c5a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6190007fd0 tx=0x7f6190007c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:40.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.477+0000 7f619cff9700 1 -- 172.21.15.112:0/360369161 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f61a015b3c0 con 0x7f61a013f910 2023-08-27T11:18:40.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.667+0000 7f61a667b700 1 -- 172.21.15.112:0/360369161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f61a015b3c0 con 0x7f61a013f910 2023-08-27T11:18:40.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.667+0000 7f619cff9700 1 -- 172.21.15.112:0/360369161 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) v1 ==== 84+0+126 (secure 0 0 0) 0x7f61a015b3c0 con 0x7f61a013f910 2023-08-27T11:18:40.669 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:40.669 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:18:40.669 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:40.670 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2023-08-27T11:18:40.670 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "rbd" 2023-08-27T11:18:40.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:40.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:40.671 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 2, 2023-08-27T11:18:40.671 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2023-08-27T11:18:40.671 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:40.671 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:18:40.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.669+0000 7f61a667b700 1 -- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f618c06a0e0 msgr2=0x7f618c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.669+0000 7f61a667b700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f618c06a0e0 0x7f618c06c5a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6190007fd0 tx=0x7f6190007c20 comp rx=0 tx=0).stop 2023-08-27T11:18:40.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.669+0000 7f61a667b700 1 -- 172.21.15.112:0/360369161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61a013f910 msgr2=0x7f61a01e3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.669+0000 7f61a667b700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61a013f910 0x7f61a01e3c30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f619800edf0 tx=0x7f619800c550 comp rx=0 tx=0).stop 2023-08-27T11:18:40.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.669+0000 7f61a667b700 1 -- 172.21.15.112:0/360369161 shutdown_connections 2023-08-27T11:18:40.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.669+0000 7f61a667b700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f618c06a0e0 0x7f618c06c5a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.669+0000 7f61a667b700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61a013f910 0x7f61a01e3c30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.669+0000 7f61a667b700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61a006ff70 0x7f61a0139700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.669+0000 7f61a667b700 1 --2- 172.21.15.112:0/360369161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a00ce260 0x7f61a0139c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.669+0000 7f61a667b700 1 -- 172.21.15.112:0/360369161 >> 172.21.15.112:0/360369161 conn(0x7f61a00b0bf0 msgr2=0x7f61a00cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:40.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.669+0000 7f61a667b700 1 -- 172.21.15.112:0/360369161 shutdown_connections 2023-08-27T11:18:40.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.669+0000 7f61a667b700 1 -- 172.21.15.112:0/360369161 wait complete. 2023-08-27T11:18:40.682 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:18:40.683 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:18:40.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.810+0000 7f2f285fa700 1 Processor -- start 2023-08-27T11:18:40.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.810+0000 7f2f285fa700 1 -- start start 2023-08-27T11:18:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.811+0000 7f2f285fa700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f200725c0 0x7f2f20166190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.811+0000 7f2f285fa700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f200a9780 con 0x7f2f200b1e90 2023-08-27T11:18:40.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.811+0000 7f2f285fa700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f200aab00 con 0x7f2f200a8690 2023-08-27T11:18:40.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.811+0000 7f2f285fa700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f200a8b70 con 0x7f2f200725c0 2023-08-27T11:18:40.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.811+0000 7f2f26b97700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f200725c0 0x7f2f20166190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.811+0000 7f2f26396700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2f200b1e90 0x7f2f200a9c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52228/0 (socket says 172.21.15.112:52228) 2023-08-27T11:18:40.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.811+0000 7f2f26396700 1 -- 172.21.15.112:0/2602157616 learned_addr learned my addr 172.21.15.112:0/2602157616 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:40.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.811+0000 7f2f26b97700 1 -- 172.21.15.112:0/2602157616 >> v1:172.21.15.138:6789/0 conn(0x7f2f200a8690 legacy=0x7f2f20071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:40.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.811+0000 7f2f26b97700 1 -- 172.21.15.112:0/2602157616 >> v1:172.21.15.112:6789/0 conn(0x7f2f200b1e90 legacy=0x7f2f200a9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.811+0000 7f2f26b97700 1 -- 172.21.15.112:0/2602157616 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f200a8e50 con 0x7f2f200725c0 2023-08-27T11:18:40.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.811+0000 7f2f25394700 1 -- 172.21.15.112:0/2602157616 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532908031 0 0) 0x7f2f200a9780 con 0x7f2f200b1e90 2023-08-27T11:18:40.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.811+0000 7f2f26b97700 1 --2- 172.21.15.112:0/2602157616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f200725c0 0x7f2f20166190 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009d90 tx=0x7f2f1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6f1014f75616f9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:40.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.812+0000 7f2f25394700 1 -- 172.21.15.112:0/2602157616 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1800e070 con 0x7f2f200725c0 2023-08-27T11:18:40.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.812+0000 7f2f25394700 1 -- 172.21.15.112:0/2602157616 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1800bbd0 con 0x7f2f200725c0 2023-08-27T11:18:40.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.812+0000 7f2f25394700 1 -- 172.21.15.112:0/2602157616 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18003a20 con 0x7f2f200725c0 2023-08-27T11:18:40.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.812+0000 7f2f285fa700 1 -- 172.21.15.112:0/2602157616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f200725c0 msgr2=0x7f2f20166190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.812+0000 7f2f285fa700 1 --2- 172.21.15.112:0/2602157616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f200725c0 0x7f2f20166190 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009d90 tx=0x7f2f1800b330 comp rx=0 tx=0).stop 2023-08-27T11:18:40.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.812+0000 7f2f285fa700 1 -- 172.21.15.112:0/2602157616 shutdown_connections 2023-08-27T11:18:40.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.812+0000 7f2f285fa700 1 --2- 172.21.15.112:0/2602157616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f200725c0 0x7f2f20166190 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.812+0000 7f2f285fa700 1 -- 172.21.15.112:0/2602157616 >> 172.21.15.112:0/2602157616 conn(0x7f2f2006ccc0 msgr2=0x7f2f200717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:40.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.812+0000 7f2f285fa700 1 -- 172.21.15.112:0/2602157616 shutdown_connections 2023-08-27T11:18:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.812+0000 7f2f285fa700 1 -- 172.21.15.112:0/2602157616 wait complete. 2023-08-27T11:18:40.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.813+0000 7f2f285fa700 1 Processor -- start 2023-08-27T11:18:40.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.813+0000 7f2f285fa700 1 -- start start 2023-08-27T11:18:40.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.814+0000 7f2f285fa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f200725c0 0x7f2f201f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.814+0000 7f2f285fa700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f200a8690 0x7f2f201f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.814+0000 7f2f285fa700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f200b1e90 0x7f2f201f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.814+0000 7f2f285fa700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f201f6eb0 con 0x7f2f200725c0 2023-08-27T11:18:40.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.814+0000 7f2f285fa700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f201f2f60 con 0x7f2f200a8690 2023-08-27T11:18:40.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.814+0000 7f2f285fa700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f200ce1e0 con 0x7f2f200b1e90 2023-08-27T11:18:40.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.814+0000 7f2f26396700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f200725c0 0x7f2f201f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.814+0000 7f2f26396700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f200725c0 0x7f2f201f2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57852/0 (socket says 172.21.15.112:57852) 2023-08-27T11:18:40.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.814+0000 7f2f26396700 1 -- 172.21.15.112:0/609641259 learned_addr learned my addr 172.21.15.112:0/609641259 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:40.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.814+0000 7f2f25b95700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f200a8690 0x7f2f201f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.814+0000 7f2f26b97700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f200b1e90 0x7f2f201f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.815+0000 7f2f26396700 1 -- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f200b1e90 msgr2=0x7f2f201f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.815+0000 7f2f26396700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f200b1e90 0x7f2f201f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.815+0000 7f2f26396700 1 -- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f200a8690 msgr2=0x7f2f201f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.815+0000 7f2f26396700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f200a8690 0x7f2f201f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.815+0000 7f2f26396700 1 -- 172.21.15.112:0/609641259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f18009a70 con 0x7f2f200725c0 2023-08-27T11:18:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.815+0000 7f2f25b95700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f200a8690 0x7f2f201f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.815+0000 7f2f26b97700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f200b1e90 0x7f2f201f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.815+0000 7f2f26396700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f200725c0 0x7f2f201f2a20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00c750 tx=0x7f2f1c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.815+0000 7f2f16ffd700 1 -- 172.21.15.112:0/609641259 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c004c80 con 0x7f2f200725c0 2023-08-27T11:18:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.815+0000 7f2f16ffd700 1 -- 172.21.15.112:0/609641259 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c004e00 con 0x7f2f200725c0 2023-08-27T11:18:40.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.815+0000 7f2f285fa700 1 -- 172.21.15.112:0/609641259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f201f68f0 con 0x7f2f200725c0 2023-08-27T11:18:40.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.815+0000 7f2f16ffd700 1 -- 172.21.15.112:0/609641259 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c007310 con 0x7f2f200725c0 2023-08-27T11:18:40.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.815+0000 7f2f285fa700 1 -- 172.21.15.112:0/609641259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f200cde70 con 0x7f2f200725c0 2023-08-27T11:18:40.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.816+0000 7f2f285fa700 1 -- 172.21.15.112:0/609641259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f200cdca0 con 0x7f2f200725c0 2023-08-27T11:18:40.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.816+0000 7f2f16ffd700 1 -- 172.21.15.112:0/609641259 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2f1c007470 con 0x7f2f200725c0 2023-08-27T11:18:40.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.817+0000 7f2f16ffd700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2f0406a0e0 0x7f2f0406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:40.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.817+0000 7f2f16ffd700 1 -- 172.21.15.112:0/609641259 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2f1c088000 con 0x7f2f200725c0 2023-08-27T11:18:40.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.817+0000 7f2f25b95700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2f0406a0e0 0x7f2f0406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:40.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.817+0000 7f2f25b95700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2f0406a0e0 0x7f2f0406c5a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2f200a8aa0 tx=0x7f2f10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:40.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:40.821+0000 7f2f16ffd700 1 -- 172.21.15.112:0/609641259 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2f1c053a50 con 0x7f2f200725c0 2023-08-27T11:18:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.014+0000 7f2f285fa700 1 -- 172.21.15.112:0/609641259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2f20067a20 con 0x7f2f200725c0 2023-08-27T11:18:41.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.880+0000 7f2f16ffd700 1 -- 172.21.15.112:0/609641259 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2f1c057070 con 0x7f2f200725c0 2023-08-27T11:18:41.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.884+0000 7f2f285fa700 1 -- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2f0406a0e0 msgr2=0x7f2f0406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:41.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.884+0000 7f2f285fa700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2f0406a0e0 0x7f2f0406c5a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2f200a8aa0 tx=0x7f2f10009040 comp rx=0 tx=0).stop 2023-08-27T11:18:41.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.884+0000 7f2f285fa700 1 -- 172.21.15.112:0/609641259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f200725c0 msgr2=0x7f2f201f2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:41.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.884+0000 7f2f285fa700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f200725c0 0x7f2f201f2a20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00c750 tx=0x7f2f1c00cb10 comp rx=0 tx=0).stop 2023-08-27T11:18:41.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.884+0000 7f2f285fa700 1 -- 172.21.15.112:0/609641259 shutdown_connections 2023-08-27T11:18:41.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.884+0000 7f2f285fa700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2f0406a0e0 0x7f2f0406c5a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:41.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.884+0000 7f2f285fa700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f200b1e90 0x7f2f201f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:41.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.884+0000 7f2f285fa700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f200725c0 0x7f2f201f2a20 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:41.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.884+0000 7f2f285fa700 1 --2- 172.21.15.112:0/609641259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f200a8690 0x7f2f201f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:41.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.884+0000 7f2f285fa700 1 -- 172.21.15.112:0/609641259 >> 172.21.15.112:0/609641259 conn(0x7f2f2006ccc0 msgr2=0x7f2f2013f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:41.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.884+0000 7f2f285fa700 1 -- 172.21.15.112:0/609641259 shutdown_connections 2023-08-27T11:18:41.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:41.884+0000 7f2f285fa700 1 -- 172.21.15.112:0/609641259 wait complete. 2023-08-27T11:18:41.889 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2023-08-27T11:18:41.903 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:18:42.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.035+0000 7f4357a95700 1 Processor -- start 2023-08-27T11:18:42.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.035+0000 7f4357a95700 1 -- start start 2023-08-27T11:18:42.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.036+0000 7f4357a95700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4350146c20 0x7f4350160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:42.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.036+0000 7f4357a95700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43500b5070 con 0x7f43500b3440 2023-08-27T11:18:42.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.036+0000 7f4357a95700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4350147000 con 0x7f435006f790 2023-08-27T11:18:42.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.036+0000 7f4357a95700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43501614a0 con 0x7f4350146c20 2023-08-27T11:18:42.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.036+0000 7f4355030700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f43500b3440 0x7f435015e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52242/0 (socket says 172.21.15.112:52242) 2023-08-27T11:18:42.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.036+0000 7f4355030700 1 -- 172.21.15.112:0/571365896 learned_addr learned my addr 172.21.15.112:0/571365896 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:42.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.036+0000 7f4355831700 1 --2- 172.21.15.112:0/571365896 >> v2:172.21.15.138:3300/0 conn(0x7f4350146c20 0x7f4350160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:42.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.036+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832956662 0 0) 0x7f434c001110 con 0x7f43500b3440 2023-08-27T11:18:42.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.036+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f433c0035b0 con 0x7f43500b3440 2023-08-27T11:18:42.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1612394458 0 0) 0x7f433c0035b0 con 0x7f43500b3440 2023-08-27T11:18:42.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 >> v2:172.21.15.138:3300/0 conn(0x7f4350146c20 msgr2=0x7f4350160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:42.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f435482f700 1 --2- 172.21.15.112:0/571365896 >> v2:172.21.15.138:3300/0 conn(0x7f4350146c20 0x7f4350160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:42.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 >> v1:172.21.15.138:6789/0 conn(0x7f435006f790 legacy=0x7f43500b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:42.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4350161740 con 0x7f43500b3440 2023-08-27T11:18:42.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f434c001800 con 0x7f43500b3440 2023-08-27T11:18:42.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 >> v1:172.21.15.112:6789/0 conn(0x7f43500b3440 legacy=0x7f435015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:42.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f435482f700 1 --2- 172.21.15.112:0/571365896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f433c003a00 0x7f433c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:42.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f435482f700 1 --2- 172.21.15.112:0/571365896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f433c004850 0x7f433c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:42.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f435482f700 1 --2- 172.21.15.112:0/571365896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f433c005210 0x7f433c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:42.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43501614a0 con 0x7f433c005210 2023-08-27T11:18:42.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c00c1b0 con 0x7f433c004850 2023-08-27T11:18:42.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c00c370 con 0x7f433c003a00 2023-08-27T11:18:42.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f4356032700 1 --2- 172.21.15.112:0/571365896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f433c005210 0x7f433c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:42.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f4355030700 1 --2- 172.21.15.112:0/571365896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f433c004850 0x7f433c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:42.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f4355831700 1 --2- 172.21.15.112:0/571365896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f433c003a00 0x7f433c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:42.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f4356032700 1 -- 172.21.15.112:0/571365896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f433c003a00 msgr2=0x7f433c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:42.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f4356032700 1 --2- 172.21.15.112:0/571365896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f433c003a00 0x7f433c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:42.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f4356032700 1 -- 172.21.15.112:0/571365896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f433c004850 msgr2=0x7f433c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:42.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f4356032700 1 --2- 172.21.15.112:0/571365896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f433c004850 0x7f433c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:42.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f4356032700 1 -- 172.21.15.112:0/571365896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c0035b0 con 0x7f433c005210 2023-08-27T11:18:42.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f4355831700 1 --2- 172.21.15.112:0/571365896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f433c003a00 0x7f433c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:42.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f4356032700 1 -- 172.21.15.112:0/571365896 reap_dead start 2023-08-27T11:18:42.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.037+0000 7f4356032700 1 --2- 172.21.15.112:0/571365896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f433c005210 0x7f433c00ba10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f434800c190 tx=0x7f433c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7531aae24acf0d6e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:42.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.038+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348001db0 con 0x7f433c005210 2023-08-27T11:18:42.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.038+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434800ccf0 con 0x7f433c005210 2023-08-27T11:18:42.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.038+0000 7f435482f700 1 -- 172.21.15.112:0/571365896 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348003c40 con 0x7f433c005210 2023-08-27T11:18:42.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.038+0000 7f4357a95700 1 -- 172.21.15.112:0/571365896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f433c005210 msgr2=0x7f433c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:42.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.038+0000 7f4357a95700 1 --2- 172.21.15.112:0/571365896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f433c005210 0x7f433c00ba10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f434800c190 tx=0x7f433c005880 comp rx=0 tx=0).stop 2023-08-27T11:18:42.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.038+0000 7f4357a95700 1 -- 172.21.15.112:0/571365896 shutdown_connections 2023-08-27T11:18:42.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.038+0000 7f4357a95700 1 --2- 172.21.15.112:0/571365896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f433c005210 0x7f433c00ba10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:42.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.038+0000 7f4357a95700 1 -- 172.21.15.112:0/571365896 >> 172.21.15.112:0/571365896 conn(0x7f43500b1e90 msgr2=0x7f43500b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:42.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.038+0000 7f4357a95700 1 -- 172.21.15.112:0/571365896 shutdown_connections 2023-08-27T11:18:42.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.038+0000 7f4357a95700 1 -- 172.21.15.112:0/571365896 wait complete. 2023-08-27T11:18:42.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.040+0000 7f4357a95700 1 Processor -- start 2023-08-27T11:18:42.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.040+0000 7f4357a95700 1 -- start start 2023-08-27T11:18:42.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4357a95700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f433c005210 0x7f4350135d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:42.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4357a95700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43500b16d0 0x7f4350141660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:42.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4357a95700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4350141ba0 0x7f43501332a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:42.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4357a95700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4350142020 con 0x7f4350141ba0 2023-08-27T11:18:42.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4357a95700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4350136260 con 0x7f43500b16d0 2023-08-27T11:18:42.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4357a95700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4350142ab0 con 0x7f433c005210 2023-08-27T11:18:42.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4356032700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4350141ba0 0x7f43501332a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:42.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4356032700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4350141ba0 0x7f43501332a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57876/0 (socket says 172.21.15.112:57876) 2023-08-27T11:18:42.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4356032700 1 -- 172.21.15.112:0/2539890172 learned_addr learned my addr 172.21.15.112:0/2539890172 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:42.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4355831700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f433c005210 0x7f4350135d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:42.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4355030700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43500b16d0 0x7f4350141660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:42.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4356032700 1 -- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f433c005210 msgr2=0x7f4350135d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:42.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4356032700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f433c005210 0x7f4350135d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:42.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4356032700 1 -- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43500b16d0 msgr2=0x7f4350141660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:42.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4356032700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43500b16d0 0x7f4350141660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:42.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4356032700 1 -- 172.21.15.112:0/2539890172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4350142d90 con 0x7f4350141ba0 2023-08-27T11:18:42.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4355030700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43500b16d0 0x7f4350141660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:42.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4356032700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4350141ba0 0x7f43501332a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4348005040 tx=0x7f43480014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:42.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f43467fc700 1 -- 172.21.15.112:0/2539890172 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43480052d0 con 0x7f4350141ba0 2023-08-27T11:18:42.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f43467fc700 1 -- 172.21.15.112:0/2539890172 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4348005430 con 0x7f4350141ba0 2023-08-27T11:18:42.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4357a95700 1 -- 172.21.15.112:0/2539890172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4350143070 con 0x7f4350141ba0 2023-08-27T11:18:42.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f43467fc700 1 -- 172.21.15.112:0/2539890172 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43480057a0 con 0x7f4350141ba0 2023-08-27T11:18:42.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.041+0000 7f4357a95700 1 -- 172.21.15.112:0/2539890172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4350143630 con 0x7f4350141ba0 2023-08-27T11:18:42.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.042+0000 7f4357a95700 1 -- 172.21.15.112:0/2539890172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4350067e90 con 0x7f4350141ba0 2023-08-27T11:18:42.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.043+0000 7f43467fc700 1 -- 172.21.15.112:0/2539890172 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f434801d480 con 0x7f4350141ba0 2023-08-27T11:18:42.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.043+0000 7f43467fc700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f433406a0e0 0x7f433406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:42.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.043+0000 7f43467fc700 1 -- 172.21.15.112:0/2539890172 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f4348088f50 con 0x7f4350141ba0 2023-08-27T11:18:42.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.044+0000 7f4355831700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f433406a0e0 0x7f433406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:42.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.044+0000 7f4355831700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f433406a0e0 0x7f433406c5a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4340009b40 tx=0x7f4340008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:42.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.048+0000 7f43467fc700 1 -- 172.21.15.112:0/2539890172 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4348054960 con 0x7f4350141ba0 2023-08-27T11:18:42.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.237+0000 7f4357a95700 1 -- 172.21.15.112:0/2539890172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4350067a20 con 0x7f4350141ba0 2023-08-27T11:18:42.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.886+0000 7f43467fc700 1 -- 172.21.15.112:0/2539890172 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) v1 ==== 170+0+0 (secure 0 0 0) 0x7f43480160d0 con 0x7f4350141ba0 2023-08-27T11:18:42.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.889+0000 7f4357a95700 1 -- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f433406a0e0 msgr2=0x7f433406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:42.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.889+0000 7f4357a95700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f433406a0e0 0x7f433406c5a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4340009b40 tx=0x7f4340008040 comp rx=0 tx=0).stop 2023-08-27T11:18:42.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.889+0000 7f4357a95700 1 -- 172.21.15.112:0/2539890172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4350141ba0 msgr2=0x7f43501332a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:42.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.889+0000 7f4357a95700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4350141ba0 0x7f43501332a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4348005040 tx=0x7f43480014a0 comp rx=0 tx=0).stop 2023-08-27T11:18:42.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.889+0000 7f4357a95700 1 -- 172.21.15.112:0/2539890172 shutdown_connections 2023-08-27T11:18:42.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.889+0000 7f4357a95700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f433406a0e0 0x7f433406c5a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:42.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.889+0000 7f4357a95700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4350141ba0 0x7f43501332a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:42.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.889+0000 7f4357a95700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f433c005210 0x7f4350135d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:42.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.889+0000 7f4357a95700 1 --2- 172.21.15.112:0/2539890172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43500b16d0 0x7f4350141660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:42.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.889+0000 7f4357a95700 1 -- 172.21.15.112:0/2539890172 >> 172.21.15.112:0/2539890172 conn(0x7f43500b1e90 msgr2=0x7f4350071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:42.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.890+0000 7f4357a95700 1 -- 172.21.15.112:0/2539890172 shutdown_connections 2023-08-27T11:18:42.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:42.890+0000 7f4357a95700 1 -- 172.21.15.112:0/2539890172 wait complete. 2023-08-27T11:18:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2023-08-27T11:18:42.912 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:18:43.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.034+0000 7f6668c4d700 1 Processor -- start 2023-08-27T11:18:43.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.034+0000 7f6668c4d700 1 -- start start 2023-08-27T11:18:43.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.035+0000 7f6668c4d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66640b3440 0x7f666415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:43.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.035+0000 7f6668c4d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f666415e6e0 0x7f666406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:43.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.035+0000 7f6668c4d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66640b5070 con 0x7f6664146c20 2023-08-27T11:18:43.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.035+0000 7f6668c4d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6664147000 con 0x7f666415e6e0 2023-08-27T11:18:43.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.035+0000 7f6668c4d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6664161740 con 0x7f66640b3440 2023-08-27T11:18:43.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f6662d9d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f666415e6e0 0x7f666406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:43.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f6661d9b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66640b3440 0x7f666415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:43.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f6661d9b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66640b3440 0x7f666415e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:41304/0 (socket says 172.21.15.112:41304) 2023-08-27T11:18:43.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f6662d9d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f666415e6e0 0x7f666406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:58310/0 (socket says 172.21.15.112:58310) 2023-08-27T11:18:43.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f6661d9b700 1 -- 172.21.15.112:0/517039030 learned_addr learned my addr 172.21.15.112:0/517039030 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:43.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f666259c700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6664146c20 0x7f6664160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:59786/0 (socket says 172.21.15.112:59786) 2023-08-27T11:18:43.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f6661d9b700 1 -- 172.21.15.112:0/517039030 >> v2:172.21.15.138:3300/0 conn(0x7f666415e6e0 msgr2=0x7f666406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:43.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f6661d9b700 1 --2- 172.21.15.112:0/517039030 >> v2:172.21.15.138:3300/0 conn(0x7f666415e6e0 0x7f666406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:43.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f6661d9b700 1 -- 172.21.15.112:0/517039030 >> v1:172.21.15.138:6789/0 conn(0x7f6664146c20 legacy=0x7f6664160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:43.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f6661d9b700 1 -- 172.21.15.112:0/517039030 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6664161a20 con 0x7f66640b3440 2023-08-27T11:18:43.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f666159a700 1 -- 172.21.15.112:0/517039030 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997141364 0 0) 0x7f6658001110 con 0x7f6664146c20 2023-08-27T11:18:43.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f6662d9d700 1 --2- 172.21.15.112:0/517039030 >> v2:172.21.15.138:3300/0 conn(0x7f666415e6e0 0x7f666406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:43.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f6661d9b700 1 --2- 172.21.15.112:0/517039030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66640b3440 0x7f666415e1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66540098f0 tx=0x7f6654009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7f727eba6e2181d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:43.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.036+0000 7f666159a700 1 -- 172.21.15.112:0/517039030 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654011830 con 0x7f66640b3440 2023-08-27T11:18:43.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.037+0000 7f666159a700 1 -- 172.21.15.112:0/517039030 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6654011990 con 0x7f66640b3440 2023-08-27T11:18:43.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.037+0000 7f666159a700 1 -- 172.21.15.112:0/517039030 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654011d20 con 0x7f66640b3440 2023-08-27T11:18:43.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.037+0000 7f6668c4d700 1 -- 172.21.15.112:0/517039030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66640b3440 msgr2=0x7f666415e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:43.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.037+0000 7f6668c4d700 1 --2- 172.21.15.112:0/517039030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66640b3440 0x7f666415e1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66540098f0 tx=0x7f6654009c00 comp rx=0 tx=0).stop 2023-08-27T11:18:43.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.037+0000 7f6668c4d700 1 -- 172.21.15.112:0/517039030 shutdown_connections 2023-08-27T11:18:43.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.037+0000 7f6668c4d700 1 --2- 172.21.15.112:0/517039030 >> v2:172.21.15.138:3300/0 conn(0x7f666415e6e0 0x7f666406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:43.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.037+0000 7f6668c4d700 1 --2- 172.21.15.112:0/517039030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66640b3440 0x7f666415e1a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:43.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.037+0000 7f6668c4d700 1 -- 172.21.15.112:0/517039030 >> 172.21.15.112:0/517039030 conn(0x7f66640b1e90 msgr2=0x7f66640b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:43.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.037+0000 7f6668c4d700 1 -- 172.21.15.112:0/517039030 shutdown_connections 2023-08-27T11:18:43.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.037+0000 7f6668c4d700 1 -- 172.21.15.112:0/517039030 wait complete. 2023-08-27T11:18:43.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.039+0000 7f6668c4d700 1 Processor -- start 2023-08-27T11:18:43.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.039+0000 7f6668c4d700 1 -- start start 2023-08-27T11:18:43.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.039+0000 7f6668c4d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66640b3440 0x7f66640ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:43.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.039+0000 7f6668c4d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6664146c20 0x7f66641400f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:43.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.039+0000 7f6668c4d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f666415e6e0 0x7f6664143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:43.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.039+0000 7f6668c4d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66640af530 con 0x7f666415e6e0 2023-08-27T11:18:43.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.039+0000 7f6668c4d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66640ae920 con 0x7f6664146c20 2023-08-27T11:18:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.039+0000 7f6668c4d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66640aec00 con 0x7f66640b3440 2023-08-27T11:18:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.039+0000 7f6662d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f666415e6e0 0x7f6664143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.039+0000 7f6662d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f666415e6e0 0x7f6664143260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57890/0 (socket says 172.21.15.112:57890) 2023-08-27T11:18:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.039+0000 7f6662d9d700 1 -- 172.21.15.112:0/3352909598 learned_addr learned my addr 172.21.15.112:0/3352909598 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f6661d9b700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6664146c20 0x7f66641400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f666259c700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66640b3440 0x7f66640ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:43.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f6662d9d700 1 -- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66640b3440 msgr2=0x7f66640ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:43.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f6662d9d700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66640b3440 0x7f66640ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:43.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f6662d9d700 1 -- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6664146c20 msgr2=0x7f66641400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:43.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f6662d9d700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6664146c20 0x7f66641400f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:43.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f6662d9d700 1 -- 172.21.15.112:0/3352909598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c009a70 con 0x7f666415e6e0 2023-08-27T11:18:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f6661d9b700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6664146c20 0x7f66641400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f6662d9d700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f666415e6e0 0x7f6664143260 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f665c00efc0 tx=0x7f665c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f6652ffd700 1 -- 172.21.15.112:0/3352909598 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c009cd0 con 0x7f666415e6e0 2023-08-27T11:18:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f6652ffd700 1 -- 172.21.15.112:0/3352909598 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c00f040 con 0x7f666415e6e0 2023-08-27T11:18:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f6668c4d700 1 -- 172.21.15.112:0/3352909598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6654009550 con 0x7f666415e6e0 2023-08-27T11:18:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f6652ffd700 1 -- 172.21.15.112:0/3352909598 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c00cd60 con 0x7f666415e6e0 2023-08-27T11:18:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.040+0000 7f6668c4d700 1 -- 172.21.15.112:0/3352909598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66640af1c0 con 0x7f666415e6e0 2023-08-27T11:18:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.041+0000 7f6668c4d700 1 -- 172.21.15.112:0/3352909598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6664067e90 con 0x7f666415e6e0 2023-08-27T11:18:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.042+0000 7f6652ffd700 1 -- 172.21.15.112:0/3352909598 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f665c014480 con 0x7f666415e6e0 2023-08-27T11:18:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.042+0000 7f6652ffd700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664406a0e0 0x7f664406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.042+0000 7f6652ffd700 1 -- 172.21.15.112:0/3352909598 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f665c088c00 con 0x7f666415e6e0 2023-08-27T11:18:43.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.042+0000 7f666259c700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664406a0e0 0x7f664406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:43.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.043+0000 7f666259c700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664406a0e0 0x7f664406c5a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6658000f60 tx=0x7f6658007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:43.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.047+0000 7f6652ffd700 1 -- 172.21.15.112:0/3352909598 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f665c08f080 con 0x7f666415e6e0 2023-08-27T11:18:43.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.279+0000 7f6668c4d700 1 -- 172.21.15.112:0/3352909598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6664067a20 con 0x7f666415e6e0 2023-08-27T11:18:43.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.888+0000 7f6652ffd700 1 -- 172.21.15.112:0/3352909598 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v26) v1 ==== 173+0+0 (secure 0 0 0) 0x7f665c057de0 con 0x7f666415e6e0 2023-08-27T11:18:43.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.893+0000 7f6668c4d700 1 -- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664406a0e0 msgr2=0x7f664406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:43.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.893+0000 7f6668c4d700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664406a0e0 0x7f664406c5a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6658000f60 tx=0x7f6658007a40 comp rx=0 tx=0).stop 2023-08-27T11:18:43.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.893+0000 7f6668c4d700 1 -- 172.21.15.112:0/3352909598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f666415e6e0 msgr2=0x7f6664143260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:43.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.893+0000 7f6668c4d700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f666415e6e0 0x7f6664143260 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f665c00efc0 tx=0x7f665c00c3a0 comp rx=0 tx=0).stop 2023-08-27T11:18:43.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.895+0000 7f6668c4d700 1 -- 172.21.15.112:0/3352909598 shutdown_connections 2023-08-27T11:18:43.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.895+0000 7f6668c4d700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664406a0e0 0x7f664406c5a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:43.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.895+0000 7f6668c4d700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f666415e6e0 0x7f6664143260 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:43.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.895+0000 7f6668c4d700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66640b3440 0x7f66640ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:43.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.895+0000 7f6668c4d700 1 --2- 172.21.15.112:0/3352909598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6664146c20 0x7f66641400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:43.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.895+0000 7f6668c4d700 1 -- 172.21.15.112:0/3352909598 >> 172.21.15.112:0/3352909598 conn(0x7f66640b1e90 msgr2=0x7f666406dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:43.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.896+0000 7f6668c4d700 1 -- 172.21.15.112:0/3352909598 shutdown_connections 2023-08-27T11:18:43.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:43.898+0000 7f6668c4d700 1 -- 172.21.15.112:0/3352909598 wait complete. 2023-08-27T11:18:43.911 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 9 and data pool 10 2023-08-27T11:18:43.923 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:44.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.043+0000 7fc9c7296700 1 Processor -- start 2023-08-27T11:18:44.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.044+0000 7fc9c7296700 1 -- start start 2023-08-27T11:18:44.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.044+0000 7fc9c7296700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9c01578e0 0x7fc9c00b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.044+0000 7fc9c7296700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc9c00cf170 0x7fc9c00d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.044+0000 7fc9c7296700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c00cfb40 con 0x7fc9c0160f90 2023-08-27T11:18:44.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.044+0000 7fc9c7296700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c00cfde0 con 0x7fc9c00cf170 2023-08-27T11:18:44.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.044+0000 7fc9c7296700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00d00c0 con 0x7fc9c01578e0 2023-08-27T11:18:44.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.045+0000 7fc9c4831700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc9c00cf170 0x7fc9c00d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.045+0000 7fc9c4831700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc9c00cf170 0x7fc9c00d08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57908/0 (socket says 172.21.15.112:57908) 2023-08-27T11:18:44.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.045+0000 7fc9c4831700 1 -- 172.21.15.112:0/2287702209 learned_addr learned my addr 172.21.15.112:0/2287702209 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:44.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.045+0000 7fc9c5032700 1 --2- 172.21.15.112:0/2287702209 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9c01578e0 0x7fc9c00b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.045+0000 7fc9c4831700 1 -- 172.21.15.112:0/2287702209 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9c01578e0 msgr2=0x7fc9c00b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:44.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.045+0000 7fc9c4831700 1 --2- 172.21.15.112:0/2287702209 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9c01578e0 0x7fc9c00b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.045+0000 7fc9c4831700 1 -- 172.21.15.112:0/2287702209 >> v1:172.21.15.138:6789/0 conn(0x7fc9c0160f90 legacy=0x7fc9c00708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:44.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.045+0000 7fc9c4831700 1 -- 172.21.15.112:0/2287702209 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c00d03a0 con 0x7fc9c00cf170 2023-08-27T11:18:44.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.045+0000 7fc9c4831700 1 --2- 172.21.15.112:0/2287702209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c00cf170 0x7fc9c00d08c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0098f0 tx=0x7fc9bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ec3d338f9cbb27 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:44.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.045+0000 7fc9b7fff700 1 -- 172.21.15.112:0/2287702209 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc004230 con 0x7fc9c00cf170 2023-08-27T11:18:44.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.045+0000 7fc9b7fff700 1 -- 172.21.15.112:0/2287702209 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9bc004390 con 0x7fc9c00cf170 2023-08-27T11:18:44.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.045+0000 7fc9b7fff700 1 -- 172.21.15.112:0/2287702209 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc0036a0 con 0x7fc9c00cf170 2023-08-27T11:18:44.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.046+0000 7fc9c7296700 1 -- 172.21.15.112:0/2287702209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c00cf170 msgr2=0x7fc9c00d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.046+0000 7fc9c7296700 1 --2- 172.21.15.112:0/2287702209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c00cf170 0x7fc9c00d08c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0098f0 tx=0x7fc9bc009c00 comp rx=0 tx=0).stop 2023-08-27T11:18:44.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.046+0000 7fc9c7296700 1 -- 172.21.15.112:0/2287702209 shutdown_connections 2023-08-27T11:18:44.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.046+0000 7fc9c7296700 1 --2- 172.21.15.112:0/2287702209 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9c01578e0 0x7fc9c00b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.046+0000 7fc9c7296700 1 --2- 172.21.15.112:0/2287702209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c00cf170 0x7fc9c00d08c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.046+0000 7fc9c7296700 1 -- 172.21.15.112:0/2287702209 >> 172.21.15.112:0/2287702209 conn(0x7fc9c006f970 msgr2=0x7fc9c006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:44.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.046+0000 7fc9c7296700 1 -- 172.21.15.112:0/2287702209 shutdown_connections 2023-08-27T11:18:44.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.046+0000 7fc9c7296700 1 -- 172.21.15.112:0/2287702209 wait complete. 2023-08-27T11:18:44.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.047+0000 7fc9c7296700 1 Processor -- start 2023-08-27T11:18:44.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.047+0000 7fc9c7296700 1 -- start start 2023-08-27T11:18:44.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c7296700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9c00cf170 0x7fc9c0159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c7296700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c01578e0 0x7fc9c0158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c7296700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9c0160f90 0x7fc9c015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c7296700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00b15e0 con 0x7fc9c01578e0 2023-08-27T11:18:44.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c7296700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c015ba50 con 0x7fc9c0160f90 2023-08-27T11:18:44.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c7296700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00b0710 con 0x7fc9c00cf170 2023-08-27T11:18:44.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c4831700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c01578e0 0x7fc9c0158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c4831700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c01578e0 0x7fc9c0158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57914/0 (socket says 172.21.15.112:57914) 2023-08-27T11:18:44.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c4831700 1 -- 172.21.15.112:0/443506051 learned_addr learned my addr 172.21.15.112:0/443506051 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:44.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c5032700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9c00cf170 0x7fc9c0159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c5833700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9c0160f90 0x7fc9c015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c4831700 1 -- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9c00cf170 msgr2=0x7fc9c0159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c4831700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9c00cf170 0x7fc9c0159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c4831700 1 -- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9c0160f90 msgr2=0x7fc9c015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c4831700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9c0160f90 0x7fc9c015b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c4831700 1 -- 172.21.15.112:0/443506051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc009550 con 0x7fc9c01578e0 2023-08-27T11:18:44.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c5833700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9c0160f90 0x7fc9c015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:44.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c4831700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c01578e0 0x7fc9c0158d10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009130 tx=0x7fc9bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:44.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9b5ffb700 1 -- 172.21.15.112:0/443506051 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc003d80 con 0x7fc9c01578e0 2023-08-27T11:18:44.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9b5ffb700 1 -- 172.21.15.112:0/443506051 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9bc01f070 con 0x7fc9c01578e0 2023-08-27T11:18:44.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9c7296700 1 -- 172.21.15.112:0/443506051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c00b09d0 con 0x7fc9c01578e0 2023-08-27T11:18:44.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.048+0000 7fc9b5ffb700 1 -- 172.21.15.112:0/443506051 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc00b8a0 con 0x7fc9c01578e0 2023-08-27T11:18:44.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.049+0000 7fc9c7296700 1 -- 172.21.15.112:0/443506051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c00b0f90 con 0x7fc9c01578e0 2023-08-27T11:18:44.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.049+0000 7fc9c7296700 1 -- 172.21.15.112:0/443506051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c00b1750 con 0x7fc9c01578e0 2023-08-27T11:18:44.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.050+0000 7fc9b5ffb700 1 -- 172.21.15.112:0/443506051 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc9bc021030 con 0x7fc9c01578e0 2023-08-27T11:18:44.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.050+0000 7fc9b5ffb700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc9a406a0e0 0x7fc9a406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.051+0000 7fc9b5ffb700 1 -- 172.21.15.112:0/443506051 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc9bc089b60 con 0x7fc9c01578e0 2023-08-27T11:18:44.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.051+0000 7fc9c5032700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc9a406a0e0 0x7fc9a406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.051+0000 7fc9c5032700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc9a406a0e0 0x7fc9a406c5a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000ab60 tx=0x7fc9b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:44.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.055+0000 7fc9b5ffb700 1 -- 172.21.15.112:0/443506051 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc9bc055400 con 0x7fc9c01578e0 2023-08-27T11:18:44.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.247+0000 7fc9c7296700 1 -- 172.21.15.112:0/443506051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9c0067a20 con 0x7fc9c01578e0 2023-08-27T11:18:44.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.248+0000 7fc9b5ffb700 1 -- 172.21.15.112:0/443506051 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18272 (secure 0 0 0) 0x7fc9bc058a20 con 0x7fc9c01578e0 2023-08-27T11:18:44.250 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:44.250 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":50,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:43.281157+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:41.033680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:42.255772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:44.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.250+0000 7fc9c7296700 1 -- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc9a406a0e0 msgr2=0x7fc9a406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.250+0000 7fc9c7296700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc9a406a0e0 0x7fc9a406c5a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000ab60 tx=0x7fc9b0008040 comp rx=0 tx=0).stop 2023-08-27T11:18:44.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.250+0000 7fc9c7296700 1 -- 172.21.15.112:0/443506051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c01578e0 msgr2=0x7fc9c0158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.250+0000 7fc9c7296700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c01578e0 0x7fc9c0158d10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009130 tx=0x7fc9bc004750 comp rx=0 tx=0).stop 2023-08-27T11:18:44.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.250+0000 7fc9c7296700 1 -- 172.21.15.112:0/443506051 shutdown_connections 2023-08-27T11:18:44.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.250+0000 7fc9c7296700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc9a406a0e0 0x7fc9a406c5a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.250+0000 7fc9c7296700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9c0160f90 0x7fc9c015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.250+0000 7fc9c7296700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9c00cf170 0x7fc9c0159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.250+0000 7fc9c7296700 1 --2- 172.21.15.112:0/443506051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c01578e0 0x7fc9c0158d10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.250+0000 7fc9c7296700 1 -- 172.21.15.112:0/443506051 >> 172.21.15.112:0/443506051 conn(0x7fc9c006f970 msgr2=0x7fc9c006b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:44.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.250+0000 7fc9c7296700 1 -- 172.21.15.112:0/443506051 shutdown_connections 2023-08-27T11:18:44.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.250+0000 7fc9c7296700 1 -- 172.21.15.112:0/443506051 wait complete. 2023-08-27T11:18:44.265 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:44.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.384+0000 7f71b6f3f700 1 Processor -- start 2023-08-27T11:18:44.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.384+0000 7f71b6f3f700 1 -- start start 2023-08-27T11:18:44.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.384+0000 7f71b6f3f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71b00b17e0 0x7f71b00cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.384+0000 7f71b6f3f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b0161850 con 0x7f71b00b1400 2023-08-27T11:18:44.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.384+0000 7f71b6f3f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b00cfc10 con 0x7f71b0161ce0 2023-08-27T11:18:44.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.384+0000 7f71b6f3f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b0161200 con 0x7f71b00b17e0 2023-08-27T11:18:44.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.385+0000 7f71affff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71b00b17e0 0x7f71b00cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.385+0000 7f71b4cdb700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f71b00b1400 0x7f71b006e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52302/0 (socket says 172.21.15.112:52302) 2023-08-27T11:18:44.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.385+0000 7f71b4cdb700 1 -- 172.21.15.112:0/3470873225 learned_addr learned my addr 172.21.15.112:0/3470873225 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:44.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.385+0000 7f71affff700 1 -- 172.21.15.112:0/3470873225 >> v1:172.21.15.138:6789/0 conn(0x7f71b0161ce0 legacy=0x7f71b006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:44.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.385+0000 7f71affff700 1 -- 172.21.15.112:0/3470873225 >> v1:172.21.15.112:6789/0 conn(0x7f71b00b1400 legacy=0x7f71b006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.385+0000 7f71affff700 1 -- 172.21.15.112:0/3470873225 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b01614e0 con 0x7f71b00b17e0 2023-08-27T11:18:44.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.385+0000 7f71affff700 1 --2- 172.21.15.112:0/3470873225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71b00b17e0 0x7f71b00cf5a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f719c009840 tx=0x7f719c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1fc9cd9b76239e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.385+0000 7f71af7fe700 1 -- 172.21.15.112:0/3470873225 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c004230 con 0x7f71b00b17e0 2023-08-27T11:18:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.385+0000 7f71af7fe700 1 -- 172.21.15.112:0/3470873225 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c004390 con 0x7f71b00b17e0 2023-08-27T11:18:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.385+0000 7f71af7fe700 1 -- 172.21.15.112:0/3470873225 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c0036a0 con 0x7f71b00b17e0 2023-08-27T11:18:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.386+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3470873225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71b00b17e0 msgr2=0x7f71b00cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.386+0000 7f71b6f3f700 1 --2- 172.21.15.112:0/3470873225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71b00b17e0 0x7f71b00cf5a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f719c009840 tx=0x7f719c009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:44.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.386+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3470873225 shutdown_connections 2023-08-27T11:18:44.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.386+0000 7f71b6f3f700 1 --2- 172.21.15.112:0/3470873225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71b00b17e0 0x7f71b00cf5a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.386+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3470873225 >> 172.21.15.112:0/3470873225 conn(0x7f71b00d0000 msgr2=0x7f71b00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:44.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.386+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3470873225 shutdown_connections 2023-08-27T11:18:44.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.386+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3470873225 wait complete. 2023-08-27T11:18:44.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.387+0000 7f71b6f3f700 1 Processor -- start 2023-08-27T11:18:44.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.387+0000 7f71b6f3f700 1 -- start start 2023-08-27T11:18:44.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71b6f3f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71b00b1400 0x7f71b00c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71b6f3f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71b00b17e0 0x7f71b00afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71b6f3f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b0161ce0 0x7f71b01f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71b6f3f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b014eae0 con 0x7f71b00b17e0 2023-08-27T11:18:44.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71b6f3f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00ca2d0 con 0x7f71b0161ce0 2023-08-27T11:18:44.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71b6f3f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00c9750 con 0x7f71b00b1400 2023-08-27T11:18:44.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71affff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71b00b17e0 0x7f71b00afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71affff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71b00b17e0 0x7f71b00afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57928/0 (socket says 172.21.15.112:57928) 2023-08-27T11:18:44.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71affff700 1 -- 172.21.15.112:0/3021456220 learned_addr learned my addr 172.21.15.112:0/3021456220 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:44.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71b54dc700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b0161ce0 0x7f71b01f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71b4cdb700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71b00b1400 0x7f71b00c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71affff700 1 -- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71b00b1400 msgr2=0x7f71b00c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71affff700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71b00b1400 0x7f71b00c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71affff700 1 -- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b0161ce0 msgr2=0x7f71b01f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71affff700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b0161ce0 0x7f71b01f3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71affff700 1 -- 172.21.15.112:0/3021456220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c0094a0 con 0x7f71b00b17e0 2023-08-27T11:18:44.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71b54dc700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b0161ce0 0x7f71b01f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:44.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71affff700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71b00b17e0 0x7f71b00afd70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f719c0090b0 tx=0x7f719c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:44.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71ad7fa700 1 -- 172.21.15.112:0/3021456220 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c011830 con 0x7f71b00b17e0 2023-08-27T11:18:44.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71ad7fa700 1 -- 172.21.15.112:0/3021456220 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c01e070 con 0x7f71b00b17e0 2023-08-27T11:18:44.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3021456220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b00c9a30 con 0x7f71b00b17e0 2023-08-27T11:18:44.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71ad7fa700 1 -- 172.21.15.112:0/3021456220 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c011bd0 con 0x7f71b00b17e0 2023-08-27T11:18:44.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.388+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3021456220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b00c9ff0 con 0x7f71b00b17e0 2023-08-27T11:18:44.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.389+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3021456220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b0067a20 con 0x7f71b00b17e0 2023-08-27T11:18:44.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.390+0000 7f71ad7fa700 1 -- 172.21.15.112:0/3021456220 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f719c011d30 con 0x7f71b00b17e0 2023-08-27T11:18:44.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.390+0000 7f71ad7fa700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f719406a0e0 0x7f719406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.390+0000 7f71ad7fa700 1 -- 172.21.15.112:0/3021456220 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f719c020070 con 0x7f71b00b17e0 2023-08-27T11:18:44.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.391+0000 7f71b4cdb700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f719406a0e0 0x7f719406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.391+0000 7f71b4cdb700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f719406a0e0 0x7f719406c5a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f71a4000f60 tx=0x7f71a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:44.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.395+0000 7f71ad7fa700 1 -- 172.21.15.112:0/3021456220 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f719c055390 con 0x7f71b00b17e0 2023-08-27T11:18:44.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.585+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3021456220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71b0067a20 con 0x7f71b00b17e0 2023-08-27T11:18:44.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.586+0000 7f71ad7fa700 1 -- 172.21.15.112:0/3021456220 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18272 (secure 0 0 0) 0x7f719c0589b0 con 0x7f71b00b17e0 2023-08-27T11:18:44.588 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:44.588 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":50,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:43.281157+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:41.033680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:42.255772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:44.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.588+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f719406a0e0 msgr2=0x7f719406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.588+0000 7f71b6f3f700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f719406a0e0 0x7f719406c5a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f71a4000f60 tx=0x7f71a4009040 comp rx=0 tx=0).stop 2023-08-27T11:18:44.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.588+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3021456220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71b00b17e0 msgr2=0x7f71b00afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.588+0000 7f71b6f3f700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71b00b17e0 0x7f71b00afd70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f719c0090b0 tx=0x7f719c004750 comp rx=0 tx=0).stop 2023-08-27T11:18:44.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.588+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3021456220 shutdown_connections 2023-08-27T11:18:44.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.588+0000 7f71b6f3f700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f719406a0e0 0x7f719406c5a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.588+0000 7f71b6f3f700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b0161ce0 0x7f71b01f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.588+0000 7f71b6f3f700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71b00b1400 0x7f71b00c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.588+0000 7f71b6f3f700 1 --2- 172.21.15.112:0/3021456220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71b00b17e0 0x7f71b00afd70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.588+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3021456220 >> 172.21.15.112:0/3021456220 conn(0x7f71b00d0000 msgr2=0x7f71b00d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:44.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.588+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3021456220 shutdown_connections 2023-08-27T11:18:44.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.588+0000 7f71b6f3f700 1 -- 172.21.15.112:0/3021456220 wait complete. 2023-08-27T11:18:44.602 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:18:44.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.723+0000 7ff52c707700 1 Processor -- start 2023-08-27T11:18:44.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.723+0000 7ff52c707700 1 -- start start 2023-08-27T11:18:44.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.724+0000 7ff52c707700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff524146c20 0x7ff524160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.724+0000 7ff52c707700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff5240b3440 0x7ff52415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.724+0000 7ff52c707700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5240b5070 con 0x7ff52415e6e0 2023-08-27T11:18:44.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.724+0000 7ff52c707700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff524147000 con 0x7ff5240b3440 2023-08-27T11:18:44.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.724+0000 7ff52c707700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5241614a0 con 0x7ff524146c20 2023-08-27T11:18:44.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.724+0000 7ff529ca2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff5240b3440 0x7ff52415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.724+0000 7ff52aca4700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff52415e6e0 0x7ff52406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52316/0 (socket says 172.21.15.112:52316) 2023-08-27T11:18:44.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.724+0000 7ff52aca4700 1 -- 172.21.15.112:0/2838503074 learned_addr learned my addr 172.21.15.112:0/2838503074 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:44.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.725+0000 7ff52a4a3700 1 --2- 172.21.15.112:0/2838503074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff524146c20 0x7ff524160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.725+0000 7ff529ca2700 1 -- 172.21.15.112:0/2838503074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff524146c20 msgr2=0x7ff524160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:44.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.725+0000 7ff529ca2700 1 --2- 172.21.15.112:0/2838503074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff524146c20 0x7ff524160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.725+0000 7ff529ca2700 1 -- 172.21.15.112:0/2838503074 >> v1:172.21.15.112:6789/0 conn(0x7ff52415e6e0 legacy=0x7ff52406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.725+0000 7ff5294a1700 1 -- 172.21.15.112:0/2838503074 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380830755 0 0) 0x7ff51c001490 con 0x7ff52415e6e0 2023-08-27T11:18:44.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.725+0000 7ff529ca2700 1 -- 172.21.15.112:0/2838503074 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff524161740 con 0x7ff5240b3440 2023-08-27T11:18:44.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.725+0000 7ff52a4a3700 1 --2- 172.21.15.112:0/2838503074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff524146c20 0x7ff524160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:44.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.725+0000 7ff529ca2700 1 --2- 172.21.15.112:0/2838503074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5240b3440 0x7ff52415e1a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff520009840 tx=0x7ff520009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdf2832083beb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:44.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.725+0000 7ff5294a1700 1 -- 172.21.15.112:0/2838503074 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520011830 con 0x7ff5240b3440 2023-08-27T11:18:44.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.725+0000 7ff5294a1700 1 -- 172.21.15.112:0/2838503074 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff520011990 con 0x7ff5240b3440 2023-08-27T11:18:44.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.725+0000 7ff5294a1700 1 -- 172.21.15.112:0/2838503074 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520011d20 con 0x7ff5240b3440 2023-08-27T11:18:44.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.726+0000 7ff52c707700 1 -- 172.21.15.112:0/2838503074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5240b3440 msgr2=0x7ff52415e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.726+0000 7ff52c707700 1 --2- 172.21.15.112:0/2838503074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5240b3440 0x7ff52415e1a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff520009840 tx=0x7ff520009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:44.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.726+0000 7ff52c707700 1 -- 172.21.15.112:0/2838503074 shutdown_connections 2023-08-27T11:18:44.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.726+0000 7ff52c707700 1 --2- 172.21.15.112:0/2838503074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff524146c20 0x7ff524160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.726+0000 7ff52c707700 1 --2- 172.21.15.112:0/2838503074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5240b3440 0x7ff52415e1a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.726+0000 7ff52c707700 1 -- 172.21.15.112:0/2838503074 >> 172.21.15.112:0/2838503074 conn(0x7ff5240b1e90 msgr2=0x7ff5240b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:44.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.726+0000 7ff52c707700 1 -- 172.21.15.112:0/2838503074 shutdown_connections 2023-08-27T11:18:44.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.726+0000 7ff52c707700 1 -- 172.21.15.112:0/2838503074 wait complete. 2023-08-27T11:18:44.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.727+0000 7ff52c707700 1 Processor -- start 2023-08-27T11:18:44.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.727+0000 7ff52c707700 1 -- start start 2023-08-27T11:18:44.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.728+0000 7ff52c707700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5240b3440 0x7ff524140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.728+0000 7ff52c707700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff524146c20 0x7ff5241602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.728+0000 7ff52c707700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff52415e6e0 0x7ff5241f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.728+0000 7ff52c707700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52415d0a0 con 0x7ff524146c20 2023-08-27T11:18:44.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.728+0000 7ff52c707700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5240cef00 con 0x7ff5240b3440 2023-08-27T11:18:44.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.728+0000 7ff52c707700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52415c800 con 0x7ff52415e6e0 2023-08-27T11:18:44.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.728+0000 7ff529ca2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff524146c20 0x7ff5241602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.728+0000 7ff529ca2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff524146c20 0x7ff5241602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57940/0 (socket says 172.21.15.112:57940) 2023-08-27T11:18:44.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.728+0000 7ff529ca2700 1 -- 172.21.15.112:0/3252463365 learned_addr learned my addr 172.21.15.112:0/3252463365 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:44.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff52a4a3700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5240b3440 0x7ff524140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff52aca4700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff52415e6e0 0x7ff5241f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff529ca2700 1 -- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff52415e6e0 msgr2=0x7ff5241f2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff529ca2700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff52415e6e0 0x7ff5241f2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff529ca2700 1 -- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5240b3440 msgr2=0x7ff524140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:44.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff529ca2700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5240b3440 0x7ff524140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:44.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff529ca2700 1 -- 172.21.15.112:0/3252463365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5200094a0 con 0x7ff524146c20 2023-08-27T11:18:44.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff52a4a3700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5240b3440 0x7ff524140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:44.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff529ca2700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff524146c20 0x7ff5241602e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff520000c00 tx=0x7ff520004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:44.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff51affd700 1 -- 172.21.15.112:0/3252463365 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5200045f0 con 0x7ff524146c20 2023-08-27T11:18:44.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff51affd700 1 -- 172.21.15.112:0/3252463365 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52001f070 con 0x7ff524146c20 2023-08-27T11:18:44.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff52c707700 1 -- 172.21.15.112:0/3252463365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52415cae0 con 0x7ff524146c20 2023-08-27T11:18:44.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff51affd700 1 -- 172.21.15.112:0/3252463365 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520003ae0 con 0x7ff524146c20 2023-08-27T11:18:44.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.729+0000 7ff52c707700 1 -- 172.21.15.112:0/3252463365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5241453c0 con 0x7ff524146c20 2023-08-27T11:18:44.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.730+0000 7ff52c707700 1 -- 172.21.15.112:0/3252463365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff524067e90 con 0x7ff524146c20 2023-08-27T11:18:44.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.730+0000 7ff51affd700 1 -- 172.21.15.112:0/3252463365 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff520003c40 con 0x7ff524146c20 2023-08-27T11:18:44.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.731+0000 7ff51affd700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff50806a0e0 0x7ff50806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:44.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.731+0000 7ff51affd700 1 -- 172.21.15.112:0/3252463365 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff520021070 con 0x7ff524146c20 2023-08-27T11:18:44.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.731+0000 7ff52a4a3700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff50806a0e0 0x7ff50806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:44.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.731+0000 7ff52a4a3700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff50806a0e0 0x7ff50806c5a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff5140089b0 tx=0x7ff514008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:44.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.735+0000 7ff51affd700 1 -- 172.21.15.112:0/3252463365 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff520055760 con 0x7ff524146c20 2023-08-27T11:18:44.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:44.973+0000 7ff52c707700 1 -- 172.21.15.112:0/3252463365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff524067a20 con 0x7ff524146c20 2023-08-27T11:18:45.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:45.906+0000 7ff51affd700 1 -- 172.21.15.112:0/3252463365 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff520015020 con 0x7ff524146c20 2023-08-27T11:18:45.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:45.908+0000 7ff52c707700 1 -- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff50806a0e0 msgr2=0x7ff50806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:45.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:45.908+0000 7ff52c707700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff50806a0e0 0x7ff50806c5a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff5140089b0 tx=0x7ff514008040 comp rx=0 tx=0).stop 2023-08-27T11:18:45.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:45.908+0000 7ff52c707700 1 -- 172.21.15.112:0/3252463365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff524146c20 msgr2=0x7ff5241602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:45.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:45.908+0000 7ff52c707700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff524146c20 0x7ff5241602e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff520000c00 tx=0x7ff520004750 comp rx=0 tx=0).stop 2023-08-27T11:18:45.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:45.909+0000 7ff52c707700 1 -- 172.21.15.112:0/3252463365 shutdown_connections 2023-08-27T11:18:45.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:45.909+0000 7ff52c707700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff50806a0e0 0x7ff50806c5a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:45.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:45.909+0000 7ff52c707700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff52415e6e0 0x7ff5241f2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:45.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:45.909+0000 7ff52c707700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5240b3440 0x7ff524140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:45.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:45.909+0000 7ff52c707700 1 --2- 172.21.15.112:0/3252463365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff524146c20 0x7ff5241602e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:45.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:45.909+0000 7ff52c707700 1 -- 172.21.15.112:0/3252463365 >> 172.21.15.112:0/3252463365 conn(0x7ff5240b1e90 msgr2=0x7ff5240b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:45.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:45.909+0000 7ff52c707700 1 -- 172.21.15.112:0/3252463365 shutdown_connections 2023-08-27T11:18:45.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:45.909+0000 7ff52c707700 1 -- 172.21.15.112:0/3252463365 wait complete. 2023-08-27T11:18:45.924 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:18:45.925 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:18:46.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.052+0000 7f629221a700 1 Processor -- start 2023-08-27T11:18:46.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.053+0000 7f629221a700 1 -- start start 2023-08-27T11:18:46.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.053+0000 7f629221a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f628c0b3440 0x7f628c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:46.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.053+0000 7f629221a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c0b5070 con 0x7f628c146c20 2023-08-27T11:18:46.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.053+0000 7f629221a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c147000 con 0x7f628c15e6e0 2023-08-27T11:18:46.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.053+0000 7f629221a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c161740 con 0x7f628c0b3440 2023-08-27T11:18:46.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f628c146c20 0x7f628c160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52326/0 (socket says 172.21.15.112:52326) 2023-08-27T11:18:46.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628b7fe700 1 -- 172.21.15.112:0/1544510360 learned_addr learned my addr 172.21.15.112:0/1544510360 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:46.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715733253 0 0) 0x7f627c001110 con 0x7f628c146c20 2023-08-27T11:18:46.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62780035b0 con 0x7f628c146c20 2023-08-27T11:18:46.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628affd700 1 --2- 172.21.15.112:0/1544510360 >> v2:172.21.15.138:3300/0 conn(0x7f628c0b3440 0x7f628c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:46.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2354716886 0 0) 0x7f62780035b0 con 0x7f628c146c20 2023-08-27T11:18:46.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 >> v2:172.21.15.138:3300/0 conn(0x7f628c0b3440 msgr2=0x7f628c15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:46.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 --2- 172.21.15.112:0/1544510360 >> v2:172.21.15.138:3300/0 conn(0x7f628c0b3440 0x7f628c15e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:46.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 >> v1:172.21.15.138:6789/0 conn(0x7f628c15e6e0 legacy=0x7f628c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:46.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c161a20 con 0x7f628c146c20 2023-08-27T11:18:46.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f627c001800 con 0x7f628c146c20 2023-08-27T11:18:46.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 >> v1:172.21.15.112:6789/0 conn(0x7f628c146c20 legacy=0x7f628c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:46.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 --2- 172.21.15.112:0/1544510360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6278003a00 0x7f6278003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:46.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 --2- 172.21.15.112:0/1544510360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6278004850 0x7f6278004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:46.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 --2- 172.21.15.112:0/1544510360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6278005210 0x7f627800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c161740 con 0x7f6278005210 2023-08-27T11:18:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627800c1b0 con 0x7f6278004850 2023-08-27T11:18:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627800c370 con 0x7f6278003a00 2023-08-27T11:18:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628bfff700 1 --2- 172.21.15.112:0/1544510360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6278005210 0x7f627800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628affd700 1 --2- 172.21.15.112:0/1544510360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6278004850 0x7f6278004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.054+0000 7f628b7fe700 1 --2- 172.21.15.112:0/1544510360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6278003a00 0x7f6278003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f628bfff700 1 -- 172.21.15.112:0/1544510360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6278003a00 msgr2=0x7f6278003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f628bfff700 1 --2- 172.21.15.112:0/1544510360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6278003a00 0x7f6278003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f628bfff700 1 -- 172.21.15.112:0/1544510360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6278004850 msgr2=0x7f6278004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f628bfff700 1 --2- 172.21.15.112:0/1544510360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6278004850 0x7f6278004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:46.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f628bfff700 1 -- 172.21.15.112:0/1544510360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62780035b0 con 0x7f6278005210 2023-08-27T11:18:46.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f628bfff700 1 -- 172.21.15.112:0/1544510360 reap_dead start 2023-08-27T11:18:46.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f628bfff700 1 --2- 172.21.15.112:0/1544510360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6278005210 0x7f627800ba10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f628400c160 tx=0x7f6278005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c2d81d20eaaadf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:46.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284001db0 con 0x7f6278005210 2023-08-27T11:18:46.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628400cca0 con 0x7f6278005210 2023-08-27T11:18:46.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f628a7fc700 1 -- 172.21.15.112:0/1544510360 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284003c40 con 0x7f6278005210 2023-08-27T11:18:46.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f629221a700 1 -- 172.21.15.112:0/1544510360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6278005210 msgr2=0x7f627800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:46.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f629221a700 1 --2- 172.21.15.112:0/1544510360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6278005210 0x7f627800ba10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f628400c160 tx=0x7f6278005880 comp rx=0 tx=0).stop 2023-08-27T11:18:46.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f629221a700 1 -- 172.21.15.112:0/1544510360 shutdown_connections 2023-08-27T11:18:46.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f629221a700 1 --2- 172.21.15.112:0/1544510360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6278005210 0x7f627800ba10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:46.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.055+0000 7f629221a700 1 -- 172.21.15.112:0/1544510360 >> 172.21.15.112:0/1544510360 conn(0x7f628c0b1e90 msgr2=0x7f628c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:46.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.056+0000 7f629221a700 1 -- 172.21.15.112:0/1544510360 shutdown_connections 2023-08-27T11:18:46.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.056+0000 7f629221a700 1 -- 172.21.15.112:0/1544510360 wait complete. 2023-08-27T11:18:46.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.057+0000 7f629221a700 1 Processor -- start 2023-08-27T11:18:46.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.057+0000 7f629221a700 1 -- start start 2023-08-27T11:18:46.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f629221a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6278005210 0x7f628c1365f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:46.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f629221a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f628c0aec80 0x7f628c145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:46.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f629221a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f628c0af100 0x7f628c1458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:46.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f629221a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0c5440 con 0x7f6278005210 2023-08-27T11:18:46.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f629221a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0ae470 con 0x7f628c0af100 2023-08-27T11:18:46.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f629221a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0ad860 con 0x7f628c0aec80 2023-08-27T11:18:46.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f628b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6278005210 0x7f628c1365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:46.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f628b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6278005210 0x7f628c1365f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57956/0 (socket says 172.21.15.112:57956) 2023-08-27T11:18:46.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f628b7fe700 1 -- 172.21.15.112:0/1258727102 learned_addr learned my addr 172.21.15.112:0/1258727102 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:46.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f628bfff700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f628c0af100 0x7f628c1458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:46.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f628affd700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f628c0aec80 0x7f628c145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:46.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f628b7fe700 1 -- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f628c0aec80 msgr2=0x7f628c145390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:46.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f628b7fe700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f628c0aec80 0x7f628c145390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:46.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f628b7fe700 1 -- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f628c0af100 msgr2=0x7f628c1458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:46.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f628b7fe700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f628c0af100 0x7f628c1458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:46.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f628b7fe700 1 -- 172.21.15.112:0/1258727102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0adb40 con 0x7f6278005210 2023-08-27T11:18:46.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f628b7fe700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6278005210 0x7f628c1365f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f627c001930 tx=0x7f627c0075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:46.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f6283fff700 1 -- 172.21.15.112:0/1258727102 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c001240 con 0x7f6278005210 2023-08-27T11:18:46.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f6283fff700 1 -- 172.21.15.112:0/1258727102 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c00f6c0 con 0x7f6278005210 2023-08-27T11:18:46.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f629221a700 1 -- 172.21.15.112:0/1258727102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0ade20 con 0x7f6278005210 2023-08-27T11:18:46.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f6283fff700 1 -- 172.21.15.112:0/1258727102 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c00fa50 con 0x7f6278005210 2023-08-27T11:18:46.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.058+0000 7f629221a700 1 -- 172.21.15.112:0/1258727102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c136b30 con 0x7f6278005210 2023-08-27T11:18:46.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.059+0000 7f629221a700 1 -- 172.21.15.112:0/1258727102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c067e90 con 0x7f6278005210 2023-08-27T11:18:46.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.060+0000 7f6283fff700 1 -- 172.21.15.112:0/1258727102 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f627c00fbb0 con 0x7f6278005210 2023-08-27T11:18:46.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.060+0000 7f6283fff700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f626c06a0e0 0x7f626c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:46.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.060+0000 7f6283fff700 1 -- 172.21.15.112:0/1258727102 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f627c088340 con 0x7f6278005210 2023-08-27T11:18:46.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.060+0000 7f628affd700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f626c06a0e0 0x7f626c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:46.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.061+0000 7f628affd700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f626c06a0e0 0x7f626c06c5a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f628c1418b0 tx=0x7f6274009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:46.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.064+0000 7f6283fff700 1 -- 172.21.15.112:0/1258727102 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f627c0539a0 con 0x7f6278005210 2023-08-27T11:18:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:46.302+0000 7f629221a700 1 -- 172.21.15.112:0/1258727102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f628c0b5070 con 0x7f6278005210 2023-08-27T11:18:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.236+0000 7f6283fff700 1 -- 172.21.15.112:0/1258727102 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f628c0b5070 con 0x7f6278005210 2023-08-27T11:18:47.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.238+0000 7f629221a700 1 -- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f626c06a0e0 msgr2=0x7f626c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:47.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.239+0000 7f629221a700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f626c06a0e0 0x7f626c06c5a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f628c1418b0 tx=0x7f6274009210 comp rx=0 tx=0).stop 2023-08-27T11:18:47.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.239+0000 7f629221a700 1 -- 172.21.15.112:0/1258727102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6278005210 msgr2=0x7f628c1365f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:47.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.239+0000 7f629221a700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6278005210 0x7f628c1365f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f627c001930 tx=0x7f627c0075f0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.239+0000 7f629221a700 1 -- 172.21.15.112:0/1258727102 shutdown_connections 2023-08-27T11:18:47.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.239+0000 7f629221a700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f626c06a0e0 0x7f626c06c5a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.239+0000 7f629221a700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f628c0af100 0x7f628c1458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.239+0000 7f629221a700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6278005210 0x7f628c1365f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.239+0000 7f629221a700 1 --2- 172.21.15.112:0/1258727102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f628c0aec80 0x7f628c145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.239+0000 7f629221a700 1 -- 172.21.15.112:0/1258727102 >> 172.21.15.112:0/1258727102 conn(0x7f628c0b1e90 msgr2=0x7f628c06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.239+0000 7f629221a700 1 -- 172.21.15.112:0/1258727102 shutdown_connections 2023-08-27T11:18:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.239+0000 7f629221a700 1 -- 172.21.15.112:0/1258727102 wait complete. 2023-08-27T11:18:47.255 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:18:47.256 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:18:47.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.380+0000 7fe39bc8c700 1 Processor -- start 2023-08-27T11:18:47.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.380+0000 7fe39bc8c700 1 -- start start 2023-08-27T11:18:47.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.381+0000 7fe39bc8c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3940b1400 0x7fe3940cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:47.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.381+0000 7fe39bc8c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe394161ce0 0x7fe39406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:47.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.381+0000 7fe39bc8c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe394161850 con 0x7fe3940b17e0 2023-08-27T11:18:47.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.381+0000 7fe39bc8c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3940cfc10 con 0x7fe3940b1400 2023-08-27T11:18:47.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.381+0000 7fe39bc8c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe394161200 con 0x7fe394161ce0 2023-08-27T11:18:47.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.381+0000 7fe399a28700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3940b1400 0x7fe3940cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:47.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.381+0000 7fe399227700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe3940b17e0 0x7fe39406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52346/0 (socket says 172.21.15.112:52346) 2023-08-27T11:18:47.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.381+0000 7fe399227700 1 -- 172.21.15.112:0/3062912018 learned_addr learned my addr 172.21.15.112:0/3062912018 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:47.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.382+0000 7fe39a229700 1 --2- 172.21.15.112:0/3062912018 >> v2:172.21.15.138:3300/0 conn(0x7fe394161ce0 0x7fe39406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:47.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.382+0000 7fe399a28700 1 -- 172.21.15.112:0/3062912018 >> v2:172.21.15.138:3300/0 conn(0x7fe394161ce0 msgr2=0x7fe39406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:47.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.382+0000 7fe399a28700 1 --2- 172.21.15.112:0/3062912018 >> v2:172.21.15.138:3300/0 conn(0x7fe394161ce0 0x7fe39406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.382+0000 7fe399a28700 1 -- 172.21.15.112:0/3062912018 >> v1:172.21.15.112:6789/0 conn(0x7fe3940b17e0 legacy=0x7fe39406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:47.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.382+0000 7fe399a28700 1 -- 172.21.15.112:0/3062912018 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3941614e0 con 0x7fe3940b1400 2023-08-27T11:18:47.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.382+0000 7fe398a26700 1 -- 172.21.15.112:0/3062912018 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517503970 0 0) 0x7fe394161850 con 0x7fe3940b17e0 2023-08-27T11:18:47.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.382+0000 7fe39a229700 1 --2- 172.21.15.112:0/3062912018 >> v2:172.21.15.138:3300/0 conn(0x7fe394161ce0 0x7fe39406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:47.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.382+0000 7fe399a28700 1 --2- 172.21.15.112:0/3062912018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3940b1400 0x7fe3940cf5a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe39406df10 tx=0x7fe384009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b9afca62a38213 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:47.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.382+0000 7fe398a26700 1 -- 172.21.15.112:0/3062912018 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384011830 con 0x7fe3940b1400 2023-08-27T11:18:47.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.382+0000 7fe398a26700 1 -- 172.21.15.112:0/3062912018 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe384011990 con 0x7fe3940b1400 2023-08-27T11:18:47.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.382+0000 7fe398a26700 1 -- 172.21.15.112:0/3062912018 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384011d20 con 0x7fe3940b1400 2023-08-27T11:18:47.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.382+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3062912018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3940b1400 msgr2=0x7fe3940cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:47.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.382+0000 7fe39bc8c700 1 --2- 172.21.15.112:0/3062912018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3940b1400 0x7fe3940cf5a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe39406df10 tx=0x7fe384009b20 comp rx=0 tx=0).stop 2023-08-27T11:18:47.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.383+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3062912018 shutdown_connections 2023-08-27T11:18:47.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.383+0000 7fe39bc8c700 1 --2- 172.21.15.112:0/3062912018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3940b1400 0x7fe3940cf5a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.383+0000 7fe39bc8c700 1 --2- 172.21.15.112:0/3062912018 >> v2:172.21.15.138:3300/0 conn(0x7fe394161ce0 0x7fe39406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.383+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3062912018 >> 172.21.15.112:0/3062912018 conn(0x7fe3940d0000 msgr2=0x7fe3940d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:47.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.383+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3062912018 shutdown_connections 2023-08-27T11:18:47.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.383+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3062912018 wait complete. 2023-08-27T11:18:47.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.384+0000 7fe39bc8c700 1 Processor -- start 2023-08-27T11:18:47.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.384+0000 7fe39bc8c700 1 -- start start 2023-08-27T11:18:47.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39bc8c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3940b1400 0x7fe3941f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:47.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39bc8c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3940b17e0 0x7fe394153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:47.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39bc8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe394161ce0 0x7fe3941f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:47.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39bc8c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3941f3f00 con 0x7fe394161ce0 2023-08-27T11:18:47.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39bc8c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3941544a0 con 0x7fe3940b17e0 2023-08-27T11:18:47.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39bc8c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe394154780 con 0x7fe3940b1400 2023-08-27T11:18:47.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39a229700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe394161ce0 0x7fe3941f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:47.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39a229700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe394161ce0 0x7fe3941f39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57972/0 (socket says 172.21.15.112:57972) 2023-08-27T11:18:47.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39a229700 1 -- 172.21.15.112:0/3612706330 learned_addr learned my addr 172.21.15.112:0/3612706330 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:47.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe399227700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3940b17e0 0x7fe394153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:47.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe399a28700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3940b1400 0x7fe3941f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:47.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39a229700 1 -- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3940b1400 msgr2=0x7fe3941f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:47.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39a229700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3940b1400 0x7fe3941f2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39a229700 1 -- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3940b17e0 msgr2=0x7fe394153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:47.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39a229700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3940b17e0 0x7fe394153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39a229700 1 -- 172.21.15.112:0/3612706330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3840094a0 con 0x7fe394161ce0 2023-08-27T11:18:47.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe399a28700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3940b1400 0x7fe3941f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:47.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39a229700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe394161ce0 0x7fe3941f39c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe38c007ef0 tx=0x7fe38c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:47.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe38a7fc700 1 -- 172.21.15.112:0/3612706330 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c00de30 con 0x7fe394161ce0 2023-08-27T11:18:47.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe38a7fc700 1 -- 172.21.15.112:0/3612706330 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38c00f040 con 0x7fe394161ce0 2023-08-27T11:18:47.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3612706330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe394154a60 con 0x7fe394161ce0 2023-08-27T11:18:47.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe38a7fc700 1 -- 172.21.15.112:0/3612706330 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c003c60 con 0x7fe394161ce0 2023-08-27T11:18:47.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.385+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3612706330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe394155020 con 0x7fe394161ce0 2023-08-27T11:18:47.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.386+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3612706330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe394067a20 con 0x7fe394161ce0 2023-08-27T11:18:47.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.387+0000 7fe38a7fc700 1 -- 172.21.15.112:0/3612706330 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe38c004030 con 0x7fe394161ce0 2023-08-27T11:18:47.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.387+0000 7fe38a7fc700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe37806a0e0 0x7fe37806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:47.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.387+0000 7fe38a7fc700 1 -- 172.21.15.112:0/3612706330 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe38c0886c0 con 0x7fe394161ce0 2023-08-27T11:18:47.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.387+0000 7fe399a28700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe37806a0e0 0x7fe37806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:47.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.388+0000 7fe399a28700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe37806a0e0 0x7fe37806c5a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3840090b0 tx=0x7fe38401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:47.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.392+0000 7fe38a7fc700 1 -- 172.21.15.112:0/3612706330 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe38c053db0 con 0x7fe394161ce0 2023-08-27T11:18:47.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.620+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3612706330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe394160b50 con 0x7fe394161ce0 2023-08-27T11:18:47.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.621+0000 7fe38a7fc700 1 -- 172.21.15.112:0/3612706330 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4888 (secure 0 0 0) 0x7fe38c0573d0 con 0x7fe394161ce0 2023-08-27T11:18:47.627 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:47.628 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1692518554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1692518554},{"type":"v1","addr":"172.21.15.138:6837","nonce":1692518554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3294671617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3294671617},{"type":"v1","addr":"172.21.15.138:6836","nonce":3294671617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3012780135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3012780135},{"type":"v1","addr":"172.21.15.112:6834","nonce":3012780135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:18:43.281470+0000","modified":"2023-08-27T11:18:47.234075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5661":{"gid":5661,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3423469954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3423469954},{"type":"v1","addr":"172.21.15.112:6835","nonce":3423469954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-27T11:18:47.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.623+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe37806a0e0 msgr2=0x7fe37806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:47.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.623+0000 7fe39bc8c700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe37806a0e0 0x7fe37806c5a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3840090b0 tx=0x7fe38401c040 comp rx=0 tx=0).stop 2023-08-27T11:18:47.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.623+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3612706330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe394161ce0 msgr2=0x7fe3941f39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:47.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.623+0000 7fe39bc8c700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe394161ce0 0x7fe3941f39c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe38c007ef0 tx=0x7fe38c00d3b0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.623+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3612706330 shutdown_connections 2023-08-27T11:18:47.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.623+0000 7fe39bc8c700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe37806a0e0 0x7fe37806c5a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.623+0000 7fe39bc8c700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe394161ce0 0x7fe3941f39c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.623+0000 7fe39bc8c700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3940b1400 0x7fe3941f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.623+0000 7fe39bc8c700 1 --2- 172.21.15.112:0/3612706330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3940b17e0 0x7fe394153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.623+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3612706330 >> 172.21.15.112:0/3612706330 conn(0x7fe3940d0000 msgr2=0x7fe3940cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:47.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.623+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3612706330 shutdown_connections 2023-08-27T11:18:47.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.623+0000 7fe39bc8c700 1 -- 172.21.15.112:0/3612706330 wait complete. 2023-08-27T11:18:47.631 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2023-08-27T11:18:47.636 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:47.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.755+0000 7f18bb067700 1 Processor -- start 2023-08-27T11:18:47.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.755+0000 7f18bb067700 1 -- start start 2023-08-27T11:18:47.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.756+0000 7f18bb067700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f18b4158c90 0x7f18b406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:47.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.756+0000 7f18bb067700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f18b4159070 0x7f18b406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:47.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.756+0000 7f18bb067700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b40b1270 0x7f18b406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:47.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.756+0000 7f18bb067700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b415f5d0 con 0x7f18b4158c90 2023-08-27T11:18:47.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.756+0000 7f18bb067700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b40ca990 con 0x7f18b4159070 2023-08-27T11:18:47.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.756+0000 7f18bb067700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b406e870 con 0x7f18b40b1270 2023-08-27T11:18:47.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.756+0000 7f18b8e03700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f18b4158c90 0x7f18b406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:47.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.756+0000 7f18b8e03700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f18b4158c90 0x7f18b406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57976/0 (socket says 172.21.15.112:57976) 2023-08-27T11:18:47.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.756+0000 7f18b8e03700 1 -- 172.21.15.112:0/1549203606 learned_addr learned my addr 172.21.15.112:0/1549203606 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:47.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.756+0000 7f18b3fff700 1 --2- 172.21.15.112:0/1549203606 >> v2:172.21.15.138:3300/0 conn(0x7f18b4159070 0x7f18b406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:47.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.756+0000 7f18b9604700 1 --2- 172.21.15.112:0/1549203606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b40b1270 0x7f18b406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:47.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.757+0000 7f18b8e03700 1 -- 172.21.15.112:0/1549203606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b40b1270 msgr2=0x7f18b406fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:47.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.757+0000 7f18b8e03700 1 --2- 172.21.15.112:0/1549203606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b40b1270 0x7f18b406fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.757+0000 7f18b8e03700 1 -- 172.21.15.112:0/1549203606 >> v2:172.21.15.138:3300/0 conn(0x7f18b4159070 msgr2=0x7f18b406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:47.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.757+0000 7f18b8e03700 1 --2- 172.21.15.112:0/1549203606 >> v2:172.21.15.138:3300/0 conn(0x7f18b4159070 0x7f18b406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.757+0000 7f18b8e03700 1 -- 172.21.15.112:0/1549203606 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b40b1a10 con 0x7f18b4158c90 2023-08-27T11:18:47.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.757+0000 7f18b8e03700 1 --2- 172.21.15.112:0/1549203606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b4158c90 0x7f18b406e330 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f18b40b2ce0 tx=0x7f18a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4dbb921375faad server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.757+0000 7f18b37fe700 1 -- 172.21.15.112:0/1549203606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8011830 con 0x7f18b4158c90 2023-08-27T11:18:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.757+0000 7f18b37fe700 1 -- 172.21.15.112:0/1549203606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a8011990 con 0x7f18b4158c90 2023-08-27T11:18:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.757+0000 7f18b37fe700 1 -- 172.21.15.112:0/1549203606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8011d20 con 0x7f18b4158c90 2023-08-27T11:18:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.758+0000 7f18bb067700 1 -- 172.21.15.112:0/1549203606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b4158c90 msgr2=0x7f18b406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.758+0000 7f18bb067700 1 --2- 172.21.15.112:0/1549203606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b4158c90 0x7f18b406e330 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f18b40b2ce0 tx=0x7f18a8009bd0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.758+0000 7f18bb067700 1 -- 172.21.15.112:0/1549203606 shutdown_connections 2023-08-27T11:18:47.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.758+0000 7f18bb067700 1 --2- 172.21.15.112:0/1549203606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b40b1270 0x7f18b406fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.758+0000 7f18bb067700 1 --2- 172.21.15.112:0/1549203606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b4158c90 0x7f18b406e330 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.758+0000 7f18bb067700 1 --2- 172.21.15.112:0/1549203606 >> v2:172.21.15.138:3300/0 conn(0x7f18b4159070 0x7f18b406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.758+0000 7f18bb067700 1 -- 172.21.15.112:0/1549203606 >> 172.21.15.112:0/1549203606 conn(0x7f18b40d0000 msgr2=0x7f18b40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:47.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.758+0000 7f18bb067700 1 -- 172.21.15.112:0/1549203606 shutdown_connections 2023-08-27T11:18:47.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.758+0000 7f18bb067700 1 -- 172.21.15.112:0/1549203606 wait complete. 2023-08-27T11:18:47.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.759+0000 7f18bb067700 1 Processor -- start 2023-08-27T11:18:47.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.759+0000 7f18bb067700 1 -- start start 2023-08-27T11:18:47.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18bb067700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b40b1270 0x7f18b41f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:47.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18bb067700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b4158c90 0x7f18b40aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:47.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18bb067700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18b4159070 0x7f18b4071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:47.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18bb067700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b41f3bc0 con 0x7f18b40b1270 2023-08-27T11:18:47.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18bb067700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b40af220 con 0x7f18b4159070 2023-08-27T11:18:47.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18bb067700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b40ad9a0 con 0x7f18b4158c90 2023-08-27T11:18:47.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b8e03700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b40b1270 0x7f18b41f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:47.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b8e03700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b40b1270 0x7f18b41f3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57982/0 (socket says 172.21.15.112:57982) 2023-08-27T11:18:47.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b8e03700 1 -- 172.21.15.112:0/695152019 learned_addr learned my addr 172.21.15.112:0/695152019 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:47.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b9604700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18b4159070 0x7f18b4071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:47.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b3fff700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b4158c90 0x7f18b40aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:47.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b8e03700 1 -- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b4158c90 msgr2=0x7f18b40aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:47.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b8e03700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b4158c90 0x7f18b40aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b8e03700 1 -- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18b4159070 msgr2=0x7f18b4071ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:47.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b8e03700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18b4159070 0x7f18b4071ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b8e03700 1 -- 172.21.15.112:0/695152019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8009550 con 0x7f18b40b1270 2023-08-27T11:18:47.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b3fff700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b4158c90 0x7f18b40aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:47.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b8e03700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b40b1270 0x7f18b41f3680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f18a8000c00 tx=0x7f18a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:47.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b17fa700 1 -- 172.21.15.112:0/695152019 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8004300 con 0x7f18b40b1270 2023-08-27T11:18:47.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b17fa700 1 -- 172.21.15.112:0/695152019 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a801f070 con 0x7f18b40b1270 2023-08-27T11:18:47.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18bb067700 1 -- 172.21.15.112:0/695152019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b40adc40 con 0x7f18b40b1270 2023-08-27T11:18:47.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.760+0000 7f18b17fa700 1 -- 172.21.15.112:0/695152019 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a80036a0 con 0x7f18b40b1270 2023-08-27T11:18:47.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.761+0000 7f18bb067700 1 -- 172.21.15.112:0/695152019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b40ae4e0 con 0x7f18b40b1270 2023-08-27T11:18:47.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.762+0000 7f18bb067700 1 -- 172.21.15.112:0/695152019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b4067a20 con 0x7f18b40b1270 2023-08-27T11:18:47.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.762+0000 7f18b17fa700 1 -- 172.21.15.112:0/695152019 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f18a8021030 con 0x7f18b40b1270 2023-08-27T11:18:47.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.762+0000 7f18b17fa700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f189806a0e0 0x7f189806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:47.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.762+0000 7f18b17fa700 1 -- 172.21.15.112:0/695152019 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f18a8089720 con 0x7f18b40b1270 2023-08-27T11:18:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.763+0000 7f18b3fff700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f189806a0e0 0x7f189806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.763+0000 7f18b3fff700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f189806a0e0 0x7f189806c5a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f18b41f07f0 tx=0x7f18a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.768+0000 7f18b17fa700 1 -- 172.21.15.112:0/695152019 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f18a8054ea0 con 0x7f18b40b1270 2023-08-27T11:18:47.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.957+0000 7f18bb067700 1 -- 172.21.15.112:0/695152019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18b406d9b0 con 0x7f18b40b1270 2023-08-27T11:18:47.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.958+0000 7f18b17fa700 1 -- 172.21.15.112:0/695152019 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18259 (secure 0 0 0) 0x7f18a80584c0 con 0x7f18b40b1270 2023-08-27T11:18:47.959 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:47.959 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":51,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:44.885944+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:41.033680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:42.255772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:47.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.960+0000 7f18bb067700 1 -- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f189806a0e0 msgr2=0x7f189806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:47.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.960+0000 7f18bb067700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f189806a0e0 0x7f189806c5a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f18b41f07f0 tx=0x7f18a0008040 comp rx=0 tx=0).stop 2023-08-27T11:18:47.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.960+0000 7f18bb067700 1 -- 172.21.15.112:0/695152019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b40b1270 msgr2=0x7f18b41f3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:47.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.960+0000 7f18bb067700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b40b1270 0x7f18b41f3680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f18a8000c00 tx=0x7f18a8004750 comp rx=0 tx=0).stop 2023-08-27T11:18:47.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.961+0000 7f18bb067700 1 -- 172.21.15.112:0/695152019 shutdown_connections 2023-08-27T11:18:47.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.961+0000 7f18bb067700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f189806a0e0 0x7f189806c5a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.961+0000 7f18bb067700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18b4159070 0x7f18b4071ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.961+0000 7f18bb067700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b40b1270 0x7f18b41f3680 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.961+0000 7f18bb067700 1 --2- 172.21.15.112:0/695152019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b4158c90 0x7f18b40aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:47.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.961+0000 7f18bb067700 1 -- 172.21.15.112:0/695152019 >> 172.21.15.112:0/695152019 conn(0x7f18b40d0000 msgr2=0x7f18b40cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:47.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.961+0000 7f18bb067700 1 -- 172.21.15.112:0/695152019 shutdown_connections 2023-08-27T11:18:47.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:47.961+0000 7f18bb067700 1 -- 172.21.15.112:0/695152019 wait complete. 2023-08-27T11:18:47.976 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:18:47.977 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:18:48.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.097+0000 7f1d8d7a9700 1 Processor -- start 2023-08-27T11:18:48.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.097+0000 7f1d8d7a9700 1 -- start start 2023-08-27T11:18:48.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.098+0000 7f1d8d7a9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1d880b3440 0x7f1d881619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.098+0000 7f1d8d7a9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d880b3b70 con 0x7f1d88146be0 2023-08-27T11:18:48.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.098+0000 7f1d8d7a9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d880b3e50 con 0x7f1d880b5470 2023-08-27T11:18:48.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.098+0000 7f1d8d7a9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d88161ef0 con 0x7f1d880b3440 2023-08-27T11:18:48.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.098+0000 7f1d877fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1d880b3440 0x7f1d881619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.098+0000 7f1d86ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1d88146be0 0x7f1d880b4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52372/0 (socket says 172.21.15.112:52372) 2023-08-27T11:18:48.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.098+0000 7f1d877fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1d880b3440 0x7f1d881619b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57994/0 (socket says 172.21.15.112:57994) 2023-08-27T11:18:48.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.098+0000 7f1d86ffd700 1 -- 172.21.15.112:0/1911773946 learned_addr learned my addr 172.21.15.112:0/1911773946 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:48.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.098+0000 7f1d877fe700 1 -- 172.21.15.112:0/1911773946 >> v1:172.21.15.138:6789/0 conn(0x7f1d880b5470 legacy=0x7f1d880cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.098+0000 7f1d877fe700 1 -- 172.21.15.112:0/1911773946 >> v1:172.21.15.112:6789/0 conn(0x7f1d88146be0 legacy=0x7f1d880b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.098+0000 7f1d877fe700 1 -- 172.21.15.112:0/1911773946 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d880b4130 con 0x7f1d880b3440 2023-08-27T11:18:48.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.098+0000 7f1d85ffb700 1 -- 172.21.15.112:0/1911773946 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815893172 0 0) 0x7f1d880b3b70 con 0x7f1d88146be0 2023-08-27T11:18:48.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.099+0000 7f1d877fe700 1 --2- 172.21.15.112:0/1911773946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d880b3440 0x7f1d881619b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009d90 tx=0x7f1d8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac31b923b81f6c94 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:48.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.099+0000 7f1d85ffb700 1 -- 172.21.15.112:0/1911773946 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000e070 con 0x7f1d880b3440 2023-08-27T11:18:48.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.099+0000 7f1d85ffb700 1 -- 172.21.15.112:0/1911773946 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8000bbd0 con 0x7f1d880b3440 2023-08-27T11:18:48.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.099+0000 7f1d85ffb700 1 -- 172.21.15.112:0/1911773946 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80012430 con 0x7f1d880b3440 2023-08-27T11:18:48.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.099+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/1911773946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d880b3440 msgr2=0x7f1d881619b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.099+0000 7f1d8d7a9700 1 --2- 172.21.15.112:0/1911773946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d880b3440 0x7f1d881619b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009d90 tx=0x7f1d8000b330 comp rx=0 tx=0).stop 2023-08-27T11:18:48.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.099+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/1911773946 shutdown_connections 2023-08-27T11:18:48.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.099+0000 7f1d8d7a9700 1 --2- 172.21.15.112:0/1911773946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d880b3440 0x7f1d881619b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.099+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/1911773946 >> 172.21.15.112:0/1911773946 conn(0x7f1d8806c880 msgr2=0x7f1d880717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:48.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.099+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/1911773946 shutdown_connections 2023-08-27T11:18:48.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.099+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/1911773946 wait complete. 2023-08-27T11:18:48.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.101+0000 7f1d8d7a9700 1 Processor -- start 2023-08-27T11:18:48.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.101+0000 7f1d8d7a9700 1 -- start start 2023-08-27T11:18:48.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.101+0000 7f1d8d7a9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d880b3440 0x7f1d8815cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.101+0000 7f1d8d7a9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d880b5470 0x7f1d88143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.101+0000 7f1d8d7a9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d88146be0 0x7f1d88143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.101+0000 7f1d8d7a9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8815d2c0 con 0x7f1d880b5470 2023-08-27T11:18:48.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.101+0000 7f1d8d7a9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880b78b0 con 0x7f1d88146be0 2023-08-27T11:18:48.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.101+0000 7f1d8d7a9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880b93d0 con 0x7f1d880b3440 2023-08-27T11:18:48.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.101+0000 7f1d867fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d880b5470 0x7f1d88143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d867fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d880b5470 0x7f1d88143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57998/0 (socket says 172.21.15.112:57998) 2023-08-27T11:18:48.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d867fc700 1 -- 172.21.15.112:0/2500925294 learned_addr learned my addr 172.21.15.112:0/2500925294 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:48.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d877fe700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d88146be0 0x7f1d88143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d86ffd700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d880b3440 0x7f1d8815cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d867fc700 1 -- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d880b3440 msgr2=0x7f1d8815cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d867fc700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d880b3440 0x7f1d8815cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d867fc700 1 -- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d88146be0 msgr2=0x7f1d88143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d867fc700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d88146be0 0x7f1d88143660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d867fc700 1 -- 172.21.15.112:0/2500925294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d80009a70 con 0x7f1d880b5470 2023-08-27T11:18:48.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d877fe700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d88146be0 0x7f1d88143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:48.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d867fc700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d880b5470 0x7f1d88143120 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800c4e0 tx=0x7f1d7800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:48.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d6f7fe700 1 -- 172.21.15.112:0/2500925294 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d780052d0 con 0x7f1d880b5470 2023-08-27T11:18:48.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d6f7fe700 1 -- 172.21.15.112:0/2500925294 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78005430 con 0x7f1d880b5470 2023-08-27T11:18:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/2500925294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d880b6f80 con 0x7f1d880b5470 2023-08-27T11:18:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d6f7fe700 1 -- 172.21.15.112:0/2500925294 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d780038c0 con 0x7f1d880b5470 2023-08-27T11:18:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.102+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/2500925294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d880b7540 con 0x7f1d880b5470 2023-08-27T11:18:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.103+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/2500925294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d88067a20 con 0x7f1d880b5470 2023-08-27T11:18:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.104+0000 7f1d6f7fe700 1 -- 172.21.15.112:0/2500925294 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1d78003a20 con 0x7f1d880b5470 2023-08-27T11:18:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.104+0000 7f1d6f7fe700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d7406a0e0 0x7f1d7406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.104+0000 7f1d6f7fe700 1 -- 172.21.15.112:0/2500925294 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1d780879b0 con 0x7f1d880b5470 2023-08-27T11:18:48.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.104+0000 7f1d86ffd700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d7406a0e0 0x7f1d7406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.105+0000 7f1d86ffd700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d7406a0e0 0x7f1d7406c5a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c000f60 tx=0x7f1d7c007db0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:48.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.108+0000 7f1d6f7fe700 1 -- 172.21.15.112:0/2500925294 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1d78053130 con 0x7f1d880b5470 2023-08-27T11:18:48.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.294+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/2500925294 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d880b3880 con 0x7f1d7406a0e0 2023-08-27T11:18:48.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.303+0000 7f1d6f7fe700 1 -- 172.21.15.112:0/2500925294 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311736 (secure 0 0 0) 0x7f1d7c004870 con 0x7f1d7406a0e0 2023-08-27T11:18:48.307 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:48.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.306+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d7406a0e0 msgr2=0x7f1d7406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.306+0000 7f1d8d7a9700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d7406a0e0 0x7f1d7406c5a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c000f60 tx=0x7f1d7c007db0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.306+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/2500925294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d880b5470 msgr2=0x7f1d88143120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.306+0000 7f1d8d7a9700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d880b5470 0x7f1d88143120 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800c4e0 tx=0x7f1d7800c8a0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.306+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/2500925294 shutdown_connections 2023-08-27T11:18:48.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.306+0000 7f1d8d7a9700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d7406a0e0 0x7f1d7406c5a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.306+0000 7f1d8d7a9700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d88146be0 0x7f1d88143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.306+0000 7f1d8d7a9700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d880b3440 0x7f1d8815cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.306+0000 7f1d8d7a9700 1 --2- 172.21.15.112:0/2500925294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d880b5470 0x7f1d88143120 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.306+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/2500925294 >> 172.21.15.112:0/2500925294 conn(0x7f1d8806c880 msgr2=0x7f1d880d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:48.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.306+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/2500925294 shutdown_connections 2023-08-27T11:18:48.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.306+0000 7f1d8d7a9700 1 -- 172.21.15.112:0/2500925294 wait complete. 2023-08-27T11:18:48.312 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:18:48.319 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2023-08-27T11:18:48.248441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222180,"kb_used_data":5988,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754752540,"statfs":{"total":773094113280,"available":772866600960,"internally_reserved":0,"allocated":6131712,"data_stored":3369595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001226"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502767+0000","last_change":"2023-08-27T11:18:43.918721+0000","last_active":"2023-08-27T11:18:45.502767+0000","last_peered":"2023-08-27T11:18:45.502767+0000","last_clean":"2023-08-27T11:18:45.502767+0000","last_became_active":"2023-08-27T11:18:43.918486+0000","last_became_peered":"2023-08-27T11:18:43.918486+0000","last_unstale":"2023-08-27T11:18:45.502767+0000","last_undegraded":"2023-08-27T11:18:45.502767+0000","last_fullsized":"2023-08-27T11:18:45.502767+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:53.484375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502730+0000","last_change":"2023-08-27T11:18:43.912694+0000","last_active":"2023-08-27T11:18:45.502730+0000","last_peered":"2023-08-27T11:18:45.502730+0000","last_clean":"2023-08-27T11:18:45.502730+0000","last_became_active":"2023-08-27T11:18:43.912406+0000","last_became_peered":"2023-08-27T11:18:43.912406+0000","last_unstale":"2023-08-27T11:18:45.502730+0000","last_undegraded":"2023-08-27T11:18:45.502730+0000","last_fullsized":"2023-08-27T11:18:45.502730+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:33:08.360188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518952+0000","last_change":"2023-08-27T11:18:43.908731+0000","last_active":"2023-08-27T11:18:45.518952+0000","last_peered":"2023-08-27T11:18:45.518952+0000","last_clean":"2023-08-27T11:18:45.518952+0000","last_became_active":"2023-08-27T11:18:43.908451+0000","last_became_peered":"2023-08-27T11:18:43.908451+0000","last_unstale":"2023-08-27T11:18:45.518952+0000","last_undegraded":"2023-08-27T11:18:45.518952+0000","last_fullsized":"2023-08-27T11:18:45.518952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:36:39.533243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893020+0000","last_change":"2023-08-27T11:18:43.901847+0000","last_active":"2023-08-27T11:18:44.893020+0000","last_peered":"2023-08-27T11:18:44.893020+0000","last_clean":"2023-08-27T11:18:44.893020+0000","last_became_active":"2023-08-27T11:18:43.901203+0000","last_became_peered":"2023-08-27T11:18:43.901203+0000","last_unstale":"2023-08-27T11:18:44.893020+0000","last_undegraded":"2023-08-27T11:18:44.893020+0000","last_fullsized":"2023-08-27T11:18:44.893020+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:25:04.905222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893060+0000","last_change":"2023-08-27T11:18:43.902697+0000","last_active":"2023-08-27T11:18:44.893060+0000","last_peered":"2023-08-27T11:18:44.893060+0000","last_clean":"2023-08-27T11:18:44.893060+0000","last_became_active":"2023-08-27T11:18:43.902427+0000","last_became_peered":"2023-08-27T11:18:43.902427+0000","last_unstale":"2023-08-27T11:18:44.893060+0000","last_undegraded":"2023-08-27T11:18:44.893060+0000","last_fullsized":"2023-08-27T11:18:44.893060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:12:35.805369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517471+0000","last_change":"2023-08-27T11:18:43.915298+0000","last_active":"2023-08-27T11:18:45.517471+0000","last_peered":"2023-08-27T11:18:45.517471+0000","last_clean":"2023-08-27T11:18:45.517471+0000","last_became_active":"2023-08-27T11:18:43.915013+0000","last_became_peered":"2023-08-27T11:18:43.915013+0000","last_unstale":"2023-08-27T11:18:45.517471+0000","last_undegraded":"2023-08-27T11:18:45.517471+0000","last_fullsized":"2023-08-27T11:18:45.517471+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:27:13.341581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.889030+0000","last_change":"2023-08-27T11:18:43.914913+0000","last_active":"2023-08-27T11:18:44.889030+0000","last_peered":"2023-08-27T11:18:44.889030+0000","last_clean":"2023-08-27T11:18:44.889030+0000","last_became_active":"2023-08-27T11:18:43.914650+0000","last_became_peered":"2023-08-27T11:18:43.914650+0000","last_unstale":"2023-08-27T11:18:44.889030+0000","last_undegraded":"2023-08-27T11:18:44.889030+0000","last_fullsized":"2023-08-27T11:18:44.889030+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:13:42.585960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517946+0000","last_change":"2023-08-27T11:18:43.916491+0000","last_active":"2023-08-27T11:18:45.517946+0000","last_peered":"2023-08-27T11:18:45.517946+0000","last_clean":"2023-08-27T11:18:45.517946+0000","last_became_active":"2023-08-27T11:18:43.909631+0000","last_became_peered":"2023-08-27T11:18:43.909631+0000","last_unstale":"2023-08-27T11:18:45.517946+0000","last_undegraded":"2023-08-27T11:18:45.517946+0000","last_fullsized":"2023-08-27T11:18:45.517946+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:40:13.364620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518882+0000","last_change":"2023-08-27T11:18:43.912279+0000","last_active":"2023-08-27T11:18:45.518882+0000","last_peered":"2023-08-27T11:18:45.518882+0000","last_clean":"2023-08-27T11:18:45.518882+0000","last_became_active":"2023-08-27T11:18:43.911263+0000","last_became_peered":"2023-08-27T11:18:43.911263+0000","last_unstale":"2023-08-27T11:18:45.518882+0000","last_undegraded":"2023-08-27T11:18:45.518882+0000","last_fullsized":"2023-08-27T11:18:45.518882+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:02:30.793457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518853+0000","last_change":"2023-08-27T11:18:43.910985+0000","last_active":"2023-08-27T11:18:45.518853+0000","last_peered":"2023-08-27T11:18:45.518853+0000","last_clean":"2023-08-27T11:18:45.518853+0000","last_became_active":"2023-08-27T11:18:43.909150+0000","last_became_peered":"2023-08-27T11:18:43.909150+0000","last_unstale":"2023-08-27T11:18:45.518853+0000","last_undegraded":"2023-08-27T11:18:45.518853+0000","last_fullsized":"2023-08-27T11:18:45.518853+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:41.828758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892411+0000","last_change":"2023-08-27T11:18:43.913114+0000","last_active":"2023-08-27T11:18:44.892411+0000","last_peered":"2023-08-27T11:18:44.892411+0000","last_clean":"2023-08-27T11:18:44.892411+0000","last_became_active":"2023-08-27T11:18:43.912712+0000","last_became_peered":"2023-08-27T11:18:43.912712+0000","last_unstale":"2023-08-27T11:18:44.892411+0000","last_undegraded":"2023-08-27T11:18:44.892411+0000","last_fullsized":"2023-08-27T11:18:44.892411+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:57:21.671903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518035+0000","last_change":"2023-08-27T11:18:43.916509+0000","last_active":"2023-08-27T11:18:45.518035+0000","last_peered":"2023-08-27T11:18:45.518035+0000","last_clean":"2023-08-27T11:18:45.518035+0000","last_became_active":"2023-08-27T11:18:43.909640+0000","last_became_peered":"2023-08-27T11:18:43.909640+0000","last_unstale":"2023-08-27T11:18:45.518035+0000","last_undegraded":"2023-08-27T11:18:45.518035+0000","last_fullsized":"2023-08-27T11:18:45.518035+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:52:31.842605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518857+0000","last_change":"2023-08-27T11:18:43.908914+0000","last_active":"2023-08-27T11:18:45.518857+0000","last_peered":"2023-08-27T11:18:45.518857+0000","last_clean":"2023-08-27T11:18:45.518857+0000","last_became_active":"2023-08-27T11:18:43.908632+0000","last_became_peered":"2023-08-27T11:18:43.908632+0000","last_unstale":"2023-08-27T11:18:45.518857+0000","last_undegraded":"2023-08-27T11:18:45.518857+0000","last_fullsized":"2023-08-27T11:18:45.518857+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:19:43.663705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518742+0000","last_change":"2023-08-27T11:18:43.911227+0000","last_active":"2023-08-27T11:18:45.518742+0000","last_peered":"2023-08-27T11:18:45.518742+0000","last_clean":"2023-08-27T11:18:45.518742+0000","last_became_active":"2023-08-27T11:18:43.910659+0000","last_became_peered":"2023-08-27T11:18:43.910659+0000","last_unstale":"2023-08-27T11:18:45.518742+0000","last_undegraded":"2023-08-27T11:18:45.518742+0000","last_fullsized":"2023-08-27T11:18:45.518742+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:14:52.102360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518382+0000","last_change":"2023-08-27T11:18:43.911371+0000","last_active":"2023-08-27T11:18:45.518382+0000","last_peered":"2023-08-27T11:18:45.518382+0000","last_clean":"2023-08-27T11:18:45.518382+0000","last_became_active":"2023-08-27T11:18:43.909057+0000","last_became_peered":"2023-08-27T11:18:43.909057+0000","last_unstale":"2023-08-27T11:18:45.518382+0000","last_undegraded":"2023-08-27T11:18:45.518382+0000","last_fullsized":"2023-08-27T11:18:45.518382+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:31:27.268258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.888976+0000","last_change":"2023-08-27T11:18:43.909941+0000","last_active":"2023-08-27T11:18:44.888976+0000","last_peered":"2023-08-27T11:18:44.888976+0000","last_clean":"2023-08-27T11:18:44.888976+0000","last_became_active":"2023-08-27T11:18:43.909689+0000","last_became_peered":"2023-08-27T11:18:43.909689+0000","last_unstale":"2023-08-27T11:18:44.888976+0000","last_undegraded":"2023-08-27T11:18:44.888976+0000","last_fullsized":"2023-08-27T11:18:44.888976+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:03:31.839321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502875+0000","last_change":"2023-08-27T11:18:43.912926+0000","last_active":"2023-08-27T11:18:45.502875+0000","last_peered":"2023-08-27T11:18:45.502875+0000","last_clean":"2023-08-27T11:18:45.502875+0000","last_became_active":"2023-08-27T11:18:43.912648+0000","last_became_peered":"2023-08-27T11:18:43.912648+0000","last_unstale":"2023-08-27T11:18:45.502875+0000","last_undegraded":"2023-08-27T11:18:45.502875+0000","last_fullsized":"2023-08-27T11:18:45.502875+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:50.062489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893435+0000","last_change":"2023-08-27T11:18:43.904840+0000","last_active":"2023-08-27T11:18:44.893435+0000","last_peered":"2023-08-27T11:18:44.893435+0000","last_clean":"2023-08-27T11:18:44.893435+0000","last_became_active":"2023-08-27T11:18:43.904544+0000","last_became_peered":"2023-08-27T11:18:43.904544+0000","last_unstale":"2023-08-27T11:18:44.893435+0000","last_undegraded":"2023-08-27T11:18:44.893435+0000","last_fullsized":"2023-08-27T11:18:44.893435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:51:37.831765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893494+0000","last_change":"2023-08-27T11:18:43.899638+0000","last_active":"2023-08-27T11:18:44.893494+0000","last_peered":"2023-08-27T11:18:44.893494+0000","last_clean":"2023-08-27T11:18:44.893494+0000","last_became_active":"2023-08-27T11:18:43.899238+0000","last_became_peered":"2023-08-27T11:18:43.899238+0000","last_unstale":"2023-08-27T11:18:44.893494+0000","last_undegraded":"2023-08-27T11:18:44.893494+0000","last_fullsized":"2023-08-27T11:18:44.893494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:37:47.012098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928979+0000","last_change":"2023-08-27T11:18:43.893563+0000","last_active":"2023-08-27T11:18:44.928979+0000","last_peered":"2023-08-27T11:18:44.928979+0000","last_clean":"2023-08-27T11:18:44.928979+0000","last_became_active":"2023-08-27T11:18:43.893267+0000","last_became_peered":"2023-08-27T11:18:43.893267+0000","last_unstale":"2023-08-27T11:18:44.928979+0000","last_undegraded":"2023-08-27T11:18:44.928979+0000","last_fullsized":"2023-08-27T11:18:44.928979+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:27:32.810244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518997+0000","last_change":"2023-08-27T11:18:43.917971+0000","last_active":"2023-08-27T11:18:45.518997+0000","last_peered":"2023-08-27T11:18:45.518997+0000","last_clean":"2023-08-27T11:18:45.518997+0000","last_became_active":"2023-08-27T11:18:43.916988+0000","last_became_peered":"2023-08-27T11:18:43.916988+0000","last_unstale":"2023-08-27T11:18:45.518997+0000","last_undegraded":"2023-08-27T11:18:45.518997+0000","last_fullsized":"2023-08-27T11:18:45.518997+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:46:59.795902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928701+0000","last_change":"2023-08-27T11:18:43.909254+0000","last_active":"2023-08-27T11:18:44.928701+0000","last_peered":"2023-08-27T11:18:44.928701+0000","last_clean":"2023-08-27T11:18:44.928701+0000","last_became_active":"2023-08-27T11:18:43.908779+0000","last_became_peered":"2023-08-27T11:18:43.908779+0000","last_unstale":"2023-08-27T11:18:44.928701+0000","last_undegraded":"2023-08-27T11:18:44.928701+0000","last_fullsized":"2023-08-27T11:18:44.928701+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:33.720245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928631+0000","last_change":"2023-08-27T11:18:43.909526+0000","last_active":"2023-08-27T11:18:44.928631+0000","last_peered":"2023-08-27T11:18:44.928631+0000","last_clean":"2023-08-27T11:18:44.928631+0000","last_became_active":"2023-08-27T11:18:43.909243+0000","last_became_peered":"2023-08-27T11:18:43.909243+0000","last_unstale":"2023-08-27T11:18:44.928631+0000","last_undegraded":"2023-08-27T11:18:44.928631+0000","last_fullsized":"2023-08-27T11:18:44.928631+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:37:29.743299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518950+0000","last_change":"2023-08-27T11:18:43.906227+0000","last_active":"2023-08-27T11:18:45.518950+0000","last_peered":"2023-08-27T11:18:45.518950+0000","last_clean":"2023-08-27T11:18:45.518950+0000","last_became_active":"2023-08-27T11:18:43.904809+0000","last_became_peered":"2023-08-27T11:18:43.904809+0000","last_unstale":"2023-08-27T11:18:45.518950+0000","last_undegraded":"2023-08-27T11:18:45.518950+0000","last_fullsized":"2023-08-27T11:18:45.518950+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:45:00.093069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518214+0000","last_change":"2023-08-27T11:18:43.909466+0000","last_active":"2023-08-27T11:18:45.518214+0000","last_peered":"2023-08-27T11:18:45.518214+0000","last_clean":"2023-08-27T11:18:45.518214+0000","last_became_active":"2023-08-27T11:18:43.908920+0000","last_became_peered":"2023-08-27T11:18:43.908920+0000","last_unstale":"2023-08-27T11:18:45.518214+0000","last_undegraded":"2023-08-27T11:18:45.518214+0000","last_fullsized":"2023-08-27T11:18:45.518214+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:31:35.215080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518550+0000","last_change":"2023-08-27T11:18:43.917447+0000","last_active":"2023-08-27T11:18:45.518550+0000","last_peered":"2023-08-27T11:18:45.518550+0000","last_clean":"2023-08-27T11:18:45.518550+0000","last_became_active":"2023-08-27T11:18:43.916601+0000","last_became_peered":"2023-08-27T11:18:43.916601+0000","last_unstale":"2023-08-27T11:18:45.518550+0000","last_undegraded":"2023-08-27T11:18:45.518550+0000","last_fullsized":"2023-08-27T11:18:45.518550+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:31.376526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893670+0000","last_change":"2023-08-27T11:18:43.900651+0000","last_active":"2023-08-27T11:18:44.893670+0000","last_peered":"2023-08-27T11:18:44.893670+0000","last_clean":"2023-08-27T11:18:44.893670+0000","last_became_active":"2023-08-27T11:18:43.900317+0000","last_became_peered":"2023-08-27T11:18:43.900317+0000","last_unstale":"2023-08-27T11:18:44.893670+0000","last_undegraded":"2023-08-27T11:18:44.893670+0000","last_fullsized":"2023-08-27T11:18:44.893670+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:29.531830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502459+0000","last_change":"2023-08-27T11:18:43.917447+0000","last_active":"2023-08-27T11:18:45.502459+0000","last_peered":"2023-08-27T11:18:45.502459+0000","last_clean":"2023-08-27T11:18:45.502459+0000","last_became_active":"2023-08-27T11:18:43.917183+0000","last_became_peered":"2023-08-27T11:18:43.917183+0000","last_unstale":"2023-08-27T11:18:45.502459+0000","last_undegraded":"2023-08-27T11:18:45.502459+0000","last_fullsized":"2023-08-27T11:18:45.502459+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:36:11.845513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.519042+0000","last_change":"2023-08-27T11:18:43.905409+0000","last_active":"2023-08-27T11:18:45.519042+0000","last_peered":"2023-08-27T11:18:45.519042+0000","last_clean":"2023-08-27T11:18:45.519042+0000","last_became_active":"2023-08-27T11:18:43.905070+0000","last_became_peered":"2023-08-27T11:18:43.905070+0000","last_unstale":"2023-08-27T11:18:45.519042+0000","last_undegraded":"2023-08-27T11:18:45.519042+0000","last_fullsized":"2023-08-27T11:18:45.519042+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:15:23.130592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517862+0000","last_change":"2023-08-27T11:18:43.917600+0000","last_active":"2023-08-27T11:18:45.517862+0000","last_peered":"2023-08-27T11:18:45.517862+0000","last_clean":"2023-08-27T11:18:45.517862+0000","last_became_active":"2023-08-27T11:18:43.917331+0000","last_became_peered":"2023-08-27T11:18:43.917331+0000","last_unstale":"2023-08-27T11:18:45.517862+0000","last_undegraded":"2023-08-27T11:18:45.517862+0000","last_fullsized":"2023-08-27T11:18:45.517862+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:32:32.604323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892545+0000","last_change":"2023-08-27T11:18:43.902354+0000","last_active":"2023-08-27T11:18:44.892545+0000","last_peered":"2023-08-27T11:18:44.892545+0000","last_clean":"2023-08-27T11:18:44.892545+0000","last_became_active":"2023-08-27T11:18:43.900712+0000","last_became_peered":"2023-08-27T11:18:43.900712+0000","last_unstale":"2023-08-27T11:18:44.892545+0000","last_undegraded":"2023-08-27T11:18:44.892545+0000","last_fullsized":"2023-08-27T11:18:44.892545+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:10.877677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517192+0000","last_change":"2023-08-27T11:18:43.898862+0000","last_active":"2023-08-27T11:18:45.517192+0000","last_peered":"2023-08-27T11:18:45.517192+0000","last_clean":"2023-08-27T11:18:45.517192+0000","last_became_active":"2023-08-27T11:18:43.898600+0000","last_became_peered":"2023-08-27T11:18:43.898600+0000","last_unstale":"2023-08-27T11:18:45.517192+0000","last_undegraded":"2023-08-27T11:18:45.517192+0000","last_fullsized":"2023-08-27T11:18:45.517192+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:14:08.060916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893426+0000","last_change":"2023-08-27T11:18:43.907300+0000","last_active":"2023-08-27T11:18:44.893426+0000","last_peered":"2023-08-27T11:18:44.893426+0000","last_clean":"2023-08-27T11:18:44.893426+0000","last_became_active":"2023-08-27T11:18:43.907032+0000","last_became_peered":"2023-08-27T11:18:43.907032+0000","last_unstale":"2023-08-27T11:18:44.893426+0000","last_undegraded":"2023-08-27T11:18:44.893426+0000","last_fullsized":"2023-08-27T11:18:44.893426+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:17:46.002709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"50'4","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502760+0000","last_change":"2023-08-27T11:18:42.900464+0000","last_active":"2023-08-27T11:18:45.502760+0000","last_peered":"2023-08-27T11:18:45.502760+0000","last_clean":"2023-08-27T11:18:45.502760+0000","last_became_active":"2023-08-27T11:18:42.900160+0000","last_became_peered":"2023-08-27T11:18:42.900160+0000","last_unstale":"2023-08-27T11:18:45.502760+0000","last_undegraded":"2023-08-27T11:18:45.502760+0000","last_fullsized":"2023-08-27T11:18:45.502760+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:00:40.439886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892708+0000","last_change":"2023-08-27T11:18:43.912720+0000","last_active":"2023-08-27T11:18:44.892708+0000","last_peered":"2023-08-27T11:18:44.892708+0000","last_clean":"2023-08-27T11:18:44.892708+0000","last_became_active":"2023-08-27T11:18:43.912050+0000","last_became_peered":"2023-08-27T11:18:43.912050+0000","last_unstale":"2023-08-27T11:18:44.892708+0000","last_undegraded":"2023-08-27T11:18:44.892708+0000","last_fullsized":"2023-08-27T11:18:44.892708+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:34.441587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"50'1","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517611+0000","last_change":"2023-08-27T11:18:42.901531+0000","last_active":"2023-08-27T11:18:45.517611+0000","last_peered":"2023-08-27T11:18:45.517611+0000","last_clean":"2023-08-27T11:18:45.517611+0000","last_became_active":"2023-08-27T11:18:42.901259+0000","last_became_peered":"2023-08-27T11:18:42.901259+0000","last_unstale":"2023-08-27T11:18:45.517611+0000","last_undegraded":"2023-08-27T11:18:45.517611+0000","last_fullsized":"2023-08-27T11:18:45.517611+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:53:27.091530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517675+0000","last_change":"2023-08-27T11:18:43.904695+0000","last_active":"2023-08-27T11:18:45.517675+0000","last_peered":"2023-08-27T11:18:45.517675+0000","last_clean":"2023-08-27T11:18:45.517675+0000","last_became_active":"2023-08-27T11:18:43.898563+0000","last_became_peered":"2023-08-27T11:18:43.898563+0000","last_unstale":"2023-08-27T11:18:45.517675+0000","last_undegraded":"2023-08-27T11:18:45.517675+0000","last_fullsized":"2023-08-27T11:18:45.517675+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:28.772925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"50'2","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517543+0000","last_change":"2023-08-27T11:18:42.903728+0000","last_active":"2023-08-27T11:18:45.517543+0000","last_peered":"2023-08-27T11:18:45.517543+0000","last_clean":"2023-08-27T11:18:45.517543+0000","last_became_active":"2023-08-27T11:18:42.903514+0000","last_became_peered":"2023-08-27T11:18:42.903514+0000","last_unstale":"2023-08-27T11:18:45.517543+0000","last_undegraded":"2023-08-27T11:18:45.517543+0000","last_fullsized":"2023-08-27T11:18:45.517543+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:40:21.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.519139+0000","last_change":"2023-08-27T11:18:43.913720+0000","last_active":"2023-08-27T11:18:45.519139+0000","last_peered":"2023-08-27T11:18:45.519139+0000","last_clean":"2023-08-27T11:18:45.519139+0000","last_became_active":"2023-08-27T11:18:43.913365+0000","last_became_peered":"2023-08-27T11:18:43.913365+0000","last_unstale":"2023-08-27T11:18:45.519139+0000","last_undegraded":"2023-08-27T11:18:45.519139+0000","last_fullsized":"2023-08-27T11:18:45.519139+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:00:42.860280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"50'2","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517535+0000","last_change":"2023-08-27T11:18:42.896270+0000","last_active":"2023-08-27T11:18:45.517535+0000","last_peered":"2023-08-27T11:18:45.517535+0000","last_clean":"2023-08-27T11:18:45.517535+0000","last_became_active":"2023-08-27T11:18:42.895964+0000","last_became_peered":"2023-08-27T11:18:42.895964+0000","last_unstale":"2023-08-27T11:18:45.517535+0000","last_undegraded":"2023-08-27T11:18:45.517535+0000","last_fullsized":"2023-08-27T11:18:45.517535+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:43:26.881783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.889025+0000","last_change":"2023-08-27T11:18:43.906687+0000","last_active":"2023-08-27T11:18:44.889025+0000","last_peered":"2023-08-27T11:18:44.889025+0000","last_clean":"2023-08-27T11:18:44.889025+0000","last_became_active":"2023-08-27T11:18:43.906259+0000","last_became_peered":"2023-08-27T11:18:43.906259+0000","last_unstale":"2023-08-27T11:18:44.889025+0000","last_undegraded":"2023-08-27T11:18:44.889025+0000","last_fullsized":"2023-08-27T11:18:44.889025+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:22:21.995766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"50'1","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928888+0000","last_change":"2023-08-27T11:18:42.908241+0000","last_active":"2023-08-27T11:18:44.928888+0000","last_peered":"2023-08-27T11:18:44.928888+0000","last_clean":"2023-08-27T11:18:44.928888+0000","last_became_active":"2023-08-27T11:18:42.907925+0000","last_became_peered":"2023-08-27T11:18:42.907925+0000","last_unstale":"2023-08-27T11:18:44.928888+0000","last_undegraded":"2023-08-27T11:18:44.928888+0000","last_fullsized":"2023-08-27T11:18:44.928888+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:46:58.789360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502405+0000","last_change":"2023-08-27T11:18:43.908246+0000","last_active":"2023-08-27T11:18:45.502405+0000","last_peered":"2023-08-27T11:18:45.502405+0000","last_clean":"2023-08-27T11:18:45.502405+0000","last_became_active":"2023-08-27T11:18:43.907979+0000","last_became_peered":"2023-08-27T11:18:43.907979+0000","last_unstale":"2023-08-27T11:18:45.502405+0000","last_undegraded":"2023-08-27T11:18:45.502405+0000","last_fullsized":"2023-08-27T11:18:45.502405+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:21.921792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"50'1","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928647+0000","last_change":"2023-08-27T11:18:42.908733+0000","last_active":"2023-08-27T11:18:44.928647+0000","last_peered":"2023-08-27T11:18:44.928647+0000","last_clean":"2023-08-27T11:18:44.928647+0000","last_became_active":"2023-08-27T11:18:42.908491+0000","last_became_peered":"2023-08-27T11:18:42.908491+0000","last_unstale":"2023-08-27T11:18:44.928647+0000","last_undegraded":"2023-08-27T11:18:44.928647+0000","last_fullsized":"2023-08-27T11:18:44.928647+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:18:45.188825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893787+0000","last_change":"2023-08-27T11:18:43.903492+0000","last_active":"2023-08-27T11:18:44.893787+0000","last_peered":"2023-08-27T11:18:44.893787+0000","last_clean":"2023-08-27T11:18:44.893787+0000","last_became_active":"2023-08-27T11:18:43.898349+0000","last_became_peered":"2023-08-27T11:18:43.898349+0000","last_unstale":"2023-08-27T11:18:44.893787+0000","last_undegraded":"2023-08-27T11:18:44.893787+0000","last_fullsized":"2023-08-27T11:18:44.893787+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:57:17.078797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928640+0000","last_change":"2023-08-27T11:18:42.908350+0000","last_active":"2023-08-27T11:18:44.928640+0000","last_peered":"2023-08-27T11:18:44.928640+0000","last_clean":"2023-08-27T11:18:44.928640+0000","last_became_active":"2023-08-27T11:18:42.907970+0000","last_became_peered":"2023-08-27T11:18:42.907970+0000","last_unstale":"2023-08-27T11:18:44.928640+0000","last_undegraded":"2023-08-27T11:18:44.928640+0000","last_fullsized":"2023-08-27T11:18:44.928640+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:42:45.198442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928826+0000","last_change":"2023-08-27T11:18:43.904771+0000","last_active":"2023-08-27T11:18:44.928826+0000","last_peered":"2023-08-27T11:18:44.928826+0000","last_clean":"2023-08-27T11:18:44.928826+0000","last_became_active":"2023-08-27T11:18:43.904503+0000","last_became_peered":"2023-08-27T11:18:43.904503+0000","last_unstale":"2023-08-27T11:18:44.928826+0000","last_undegraded":"2023-08-27T11:18:44.928826+0000","last_fullsized":"2023-08-27T11:18:44.928826+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:19:43.663705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893161+0000","last_change":"2023-08-27T11:18:42.902151+0000","last_active":"2023-08-27T11:18:44.893161+0000","last_peered":"2023-08-27T11:18:44.893161+0000","last_clean":"2023-08-27T11:18:44.893161+0000","last_became_active":"2023-08-27T11:18:42.901900+0000","last_became_peered":"2023-08-27T11:18:42.901900+0000","last_unstale":"2023-08-27T11:18:44.893161+0000","last_undegraded":"2023-08-27T11:18:44.893161+0000","last_fullsized":"2023-08-27T11:18:44.893161+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:02:12.354166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502488+0000","last_change":"2023-08-27T11:18:43.905888+0000","last_active":"2023-08-27T11:18:45.502488+0000","last_peered":"2023-08-27T11:18:45.502488+0000","last_clean":"2023-08-27T11:18:45.502488+0000","last_became_active":"2023-08-27T11:18:43.905641+0000","last_became_peered":"2023-08-27T11:18:43.905641+0000","last_unstale":"2023-08-27T11:18:45.502488+0000","last_undegraded":"2023-08-27T11:18:45.502488+0000","last_fullsized":"2023-08-27T11:18:45.502488+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:25:01.340661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.519238+0000","last_change":"2023-08-27T11:18:42.906290+0000","last_active":"2023-08-27T11:18:45.519238+0000","last_peered":"2023-08-27T11:18:45.519238+0000","last_clean":"2023-08-27T11:18:45.519238+0000","last_became_active":"2023-08-27T11:18:42.905992+0000","last_became_peered":"2023-08-27T11:18:42.905992+0000","last_unstale":"2023-08-27T11:18:45.519238+0000","last_undegraded":"2023-08-27T11:18:45.519238+0000","last_fullsized":"2023-08-27T11:18:45.519238+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:22:36.744418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.929026+0000","last_change":"2023-08-27T11:18:43.909062+0000","last_active":"2023-08-27T11:18:44.929026+0000","last_peered":"2023-08-27T11:18:44.929026+0000","last_clean":"2023-08-27T11:18:44.929026+0000","last_became_active":"2023-08-27T11:18:43.908769+0000","last_became_peered":"2023-08-27T11:18:43.908769+0000","last_unstale":"2023-08-27T11:18:44.929026+0000","last_undegraded":"2023-08-27T11:18:44.929026+0000","last_fullsized":"2023-08-27T11:18:44.929026+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:52:38.617545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893250+0000","last_change":"2023-08-27T11:18:42.898836+0000","last_active":"2023-08-27T11:18:44.893250+0000","last_peered":"2023-08-27T11:18:44.893250+0000","last_clean":"2023-08-27T11:18:44.893250+0000","last_became_active":"2023-08-27T11:18:42.898515+0000","last_became_peered":"2023-08-27T11:18:42.898515+0000","last_unstale":"2023-08-27T11:18:44.893250+0000","last_undegraded":"2023-08-27T11:18:44.893250+0000","last_fullsized":"2023-08-27T11:18:44.893250+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:01:43.305122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893211+0000","last_change":"2023-08-27T11:18:43.919688+0000","last_active":"2023-08-27T11:18:44.893211+0000","last_peered":"2023-08-27T11:18:44.893211+0000","last_clean":"2023-08-27T11:18:44.893211+0000","last_became_active":"2023-08-27T11:18:43.919401+0000","last_became_peered":"2023-08-27T11:18:43.919401+0000","last_unstale":"2023-08-27T11:18:44.893211+0000","last_undegraded":"2023-08-27T11:18:44.893211+0000","last_fullsized":"2023-08-27T11:18:44.893211+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:16:13.567306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"50'3","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518629+0000","last_change":"2023-08-27T11:18:42.906276+0000","last_active":"2023-08-27T11:18:45.518629+0000","last_peered":"2023-08-27T11:18:45.518629+0000","last_clean":"2023-08-27T11:18:45.518629+0000","last_became_active":"2023-08-27T11:18:42.906009+0000","last_became_peered":"2023-08-27T11:18:42.906009+0000","last_unstale":"2023-08-27T11:18:45.518629+0000","last_undegraded":"2023-08-27T11:18:45.518629+0000","last_fullsized":"2023-08-27T11:18:45.518629+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:57.143682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893146+0000","last_change":"2023-08-27T11:18:43.903954+0000","last_active":"2023-08-27T11:18:44.893146+0000","last_peered":"2023-08-27T11:18:44.893146+0000","last_clean":"2023-08-27T11:18:44.893146+0000","last_became_active":"2023-08-27T11:18:43.903682+0000","last_became_peered":"2023-08-27T11:18:43.903682+0000","last_unstale":"2023-08-27T11:18:44.893146+0000","last_undegraded":"2023-08-27T11:18:44.893146+0000","last_fullsized":"2023-08-27T11:18:44.893146+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:52:46.854872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"50'6","reported_seq":19,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502636+0000","last_change":"2023-08-27T11:18:42.897689+0000","last_active":"2023-08-27T11:18:45.502636+0000","last_peered":"2023-08-27T11:18:45.502636+0000","last_clean":"2023-08-27T11:18:45.502636+0000","last_became_active":"2023-08-27T11:18:42.897445+0000","last_became_peered":"2023-08-27T11:18:42.897445+0000","last_unstale":"2023-08-27T11:18:45.502636+0000","last_undegraded":"2023-08-27T11:18:45.502636+0000","last_fullsized":"2023-08-27T11:18:45.502636+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:28:49.672644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892223+0000","last_change":"2023-08-27T11:18:43.915954+0000","last_active":"2023-08-27T11:18:44.892223+0000","last_peered":"2023-08-27T11:18:44.892223+0000","last_clean":"2023-08-27T11:18:44.892223+0000","last_became_active":"2023-08-27T11:18:43.915021+0000","last_became_peered":"2023-08-27T11:18:43.915021+0000","last_unstale":"2023-08-27T11:18:44.892223+0000","last_undegraded":"2023-08-27T11:18:44.892223+0000","last_fullsized":"2023-08-27T11:18:44.892223+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:46.250380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"50'2","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518674+0000","last_change":"2023-08-27T11:18:42.905644+0000","last_active":"2023-08-27T11:18:45.518674+0000","last_peered":"2023-08-27T11:18:45.518674+0000","last_clean":"2023-08-27T11:18:45.518674+0000","last_became_active":"2023-08-27T11:18:42.905180+0000","last_became_peered":"2023-08-27T11:18:42.905180+0000","last_unstale":"2023-08-27T11:18:45.518674+0000","last_undegraded":"2023-08-27T11:18:45.518674+0000","last_fullsized":"2023-08-27T11:18:45.518674+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:21:05.202954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518160+0000","last_change":"2023-08-27T11:18:43.908688+0000","last_active":"2023-08-27T11:18:45.518160+0000","last_peered":"2023-08-27T11:18:45.518160+0000","last_clean":"2023-08-27T11:18:45.518160+0000","last_became_active":"2023-08-27T11:18:43.908427+0000","last_became_peered":"2023-08-27T11:18:43.908427+0000","last_unstale":"2023-08-27T11:18:45.518160+0000","last_undegraded":"2023-08-27T11:18:45.518160+0000","last_fullsized":"2023-08-27T11:18:45.518160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:03:18.205212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517935+0000","last_change":"2023-08-27T11:18:42.907812+0000","last_active":"2023-08-27T11:18:45.517935+0000","last_peered":"2023-08-27T11:18:45.517935+0000","last_clean":"2023-08-27T11:18:45.517935+0000","last_became_active":"2023-08-27T11:18:42.907351+0000","last_became_peered":"2023-08-27T11:18:42.907351+0000","last_unstale":"2023-08-27T11:18:45.517935+0000","last_undegraded":"2023-08-27T11:18:45.517935+0000","last_fullsized":"2023-08-27T11:18:45.517935+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:22:52.323702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893070+0000","last_change":"2023-08-27T11:18:43.899866+0000","last_active":"2023-08-27T11:18:44.893070+0000","last_peered":"2023-08-27T11:18:44.893070+0000","last_clean":"2023-08-27T11:18:44.893070+0000","last_became_active":"2023-08-27T11:18:43.899580+0000","last_became_peered":"2023-08-27T11:18:43.899580+0000","last_unstale":"2023-08-27T11:18:44.893070+0000","last_undegraded":"2023-08-27T11:18:44.893070+0000","last_fullsized":"2023-08-27T11:18:44.893070+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:34:45.726494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.519058+0000","last_change":"2023-08-27T11:18:42.909897+0000","last_active":"2023-08-27T11:18:45.519058+0000","last_peered":"2023-08-27T11:18:45.519058+0000","last_clean":"2023-08-27T11:18:45.519058+0000","last_became_active":"2023-08-27T11:18:42.909605+0000","last_became_peered":"2023-08-27T11:18:42.909605+0000","last_unstale":"2023-08-27T11:18:45.519058+0000","last_undegraded":"2023-08-27T11:18:45.519058+0000","last_fullsized":"2023-08-27T11:18:45.519058+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:45.951900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502581+0000","last_change":"2023-08-27T11:18:43.909878+0000","last_active":"2023-08-27T11:18:45.502581+0000","last_peered":"2023-08-27T11:18:45.502581+0000","last_clean":"2023-08-27T11:18:45.502581+0000","last_became_active":"2023-08-27T11:18:43.909629+0000","last_became_peered":"2023-08-27T11:18:43.909629+0000","last_unstale":"2023-08-27T11:18:45.502581+0000","last_undegraded":"2023-08-27T11:18:45.502581+0000","last_fullsized":"2023-08-27T11:18:45.502581+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:25:48.807928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"50'8","reported_seq":21,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893559+0000","last_change":"2023-08-27T11:18:42.901981+0000","last_active":"2023-08-27T11:18:44.893559+0000","last_peered":"2023-08-27T11:18:44.893559+0000","last_clean":"2023-08-27T11:18:44.893559+0000","last_became_active":"2023-08-27T11:18:42.901728+0000","last_became_peered":"2023-08-27T11:18:42.901728+0000","last_unstale":"2023-08-27T11:18:44.893559+0000","last_undegraded":"2023-08-27T11:18:44.893559+0000","last_fullsized":"2023-08-27T11:18:44.893559+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:46:43.863658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517552+0000","last_change":"2023-08-27T11:18:43.901514+0000","last_active":"2023-08-27T11:18:45.517552+0000","last_peered":"2023-08-27T11:18:45.517552+0000","last_clean":"2023-08-27T11:18:45.517552+0000","last_became_active":"2023-08-27T11:18:43.901085+0000","last_became_peered":"2023-08-27T11:18:43.901085+0000","last_unstale":"2023-08-27T11:18:45.517552+0000","last_undegraded":"2023-08-27T11:18:45.517552+0000","last_fullsized":"2023-08-27T11:18:45.517552+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:35:59.130505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"50'2","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518373+0000","last_change":"2023-08-27T11:18:42.902004+0000","last_active":"2023-08-27T11:18:45.518373+0000","last_peered":"2023-08-27T11:18:45.518373+0000","last_clean":"2023-08-27T11:18:45.518373+0000","last_became_active":"2023-08-27T11:18:42.901718+0000","last_became_peered":"2023-08-27T11:18:42.901718+0000","last_unstale":"2023-08-27T11:18:45.518373+0000","last_undegraded":"2023-08-27T11:18:45.518373+0000","last_fullsized":"2023-08-27T11:18:45.518373+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:32.713703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.889225+0000","last_change":"2023-08-27T11:18:43.902858+0000","last_active":"2023-08-27T11:18:44.889225+0000","last_peered":"2023-08-27T11:18:44.889225+0000","last_clean":"2023-08-27T11:18:44.889225+0000","last_became_active":"2023-08-27T11:18:43.902603+0000","last_became_peered":"2023-08-27T11:18:43.902603+0000","last_unstale":"2023-08-27T11:18:44.889225+0000","last_undegraded":"2023-08-27T11:18:44.889225+0000","last_fullsized":"2023-08-27T11:18:44.889225+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:58.280505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518365+0000","last_change":"2023-08-27T11:18:42.902778+0000","last_active":"2023-08-27T11:18:45.518365+0000","last_peered":"2023-08-27T11:18:45.518365+0000","last_clean":"2023-08-27T11:18:45.518365+0000","last_became_active":"2023-08-27T11:18:42.902455+0000","last_became_peered":"2023-08-27T11:18:42.902455+0000","last_unstale":"2023-08-27T11:18:45.518365+0000","last_undegraded":"2023-08-27T11:18:45.518365+0000","last_fullsized":"2023-08-27T11:18:45.518365+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:18:45.188825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518257+0000","last_change":"2023-08-27T11:18:43.917693+0000","last_active":"2023-08-27T11:18:45.518257+0000","last_peered":"2023-08-27T11:18:45.518257+0000","last_clean":"2023-08-27T11:18:45.518257+0000","last_became_active":"2023-08-27T11:18:43.916404+0000","last_became_peered":"2023-08-27T11:18:43.916404+0000","last_unstale":"2023-08-27T11:18:45.518257+0000","last_undegraded":"2023-08-27T11:18:45.518257+0000","last_fullsized":"2023-08-27T11:18:45.518257+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:37:29.743299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517635+0000","last_change":"2023-08-27T11:18:42.903712+0000","last_active":"2023-08-27T11:18:45.517635+0000","last_peered":"2023-08-27T11:18:45.517635+0000","last_clean":"2023-08-27T11:18:45.517635+0000","last_became_active":"2023-08-27T11:18:42.903476+0000","last_became_peered":"2023-08-27T11:18:42.903476+0000","last_unstale":"2023-08-27T11:18:45.517635+0000","last_undegraded":"2023-08-27T11:18:45.517635+0000","last_fullsized":"2023-08-27T11:18:45.517635+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:49:45.344498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893474+0000","last_change":"2023-08-27T11:18:43.895176+0000","last_active":"2023-08-27T11:18:44.893474+0000","last_peered":"2023-08-27T11:18:44.893474+0000","last_clean":"2023-08-27T11:18:44.893474+0000","last_became_active":"2023-08-27T11:18:43.894779+0000","last_became_peered":"2023-08-27T11:18:43.894779+0000","last_unstale":"2023-08-27T11:18:44.893474+0000","last_undegraded":"2023-08-27T11:18:44.893474+0000","last_fullsized":"2023-08-27T11:18:44.893474+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:38:44.087181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502756+0000","last_change":"2023-08-27T11:18:42.897646+0000","last_active":"2023-08-27T11:18:45.502756+0000","last_peered":"2023-08-27T11:18:45.502756+0000","last_clean":"2023-08-27T11:18:45.502756+0000","last_became_active":"2023-08-27T11:18:42.897380+0000","last_became_peered":"2023-08-27T11:18:42.897380+0000","last_unstale":"2023-08-27T11:18:45.502756+0000","last_undegraded":"2023-08-27T11:18:45.502756+0000","last_fullsized":"2023-08-27T11:18:45.502756+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:34:05.074779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892913+0000","last_change":"2023-08-27T11:18:43.905727+0000","last_active":"2023-08-27T11:18:44.892913+0000","last_peered":"2023-08-27T11:18:44.892913+0000","last_clean":"2023-08-27T11:18:44.892913+0000","last_became_active":"2023-08-27T11:18:43.905459+0000","last_became_peered":"2023-08-27T11:18:43.905459+0000","last_unstale":"2023-08-27T11:18:44.892913+0000","last_undegraded":"2023-08-27T11:18:44.892913+0000","last_fullsized":"2023-08-27T11:18:44.892913+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:34:47.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518829+0000","last_change":"2023-08-27T11:18:42.909814+0000","last_active":"2023-08-27T11:18:45.518829+0000","last_peered":"2023-08-27T11:18:45.518829+0000","last_clean":"2023-08-27T11:18:45.518829+0000","last_became_active":"2023-08-27T11:18:42.909583+0000","last_became_peered":"2023-08-27T11:18:42.909583+0000","last_unstale":"2023-08-27T11:18:45.518829+0000","last_undegraded":"2023-08-27T11:18:45.518829+0000","last_fullsized":"2023-08-27T11:18:45.518829+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:04:51.015111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892714+0000","last_change":"2023-08-27T11:18:43.915183+0000","last_active":"2023-08-27T11:18:44.892714+0000","last_peered":"2023-08-27T11:18:44.892714+0000","last_clean":"2023-08-27T11:18:44.892714+0000","last_became_active":"2023-08-27T11:18:43.914883+0000","last_became_peered":"2023-08-27T11:18:43.914883+0000","last_unstale":"2023-08-27T11:18:44.892714+0000","last_undegraded":"2023-08-27T11:18:44.892714+0000","last_fullsized":"2023-08-27T11:18:44.892714+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:44:28.059085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928760+0000","last_change":"2023-08-27T11:18:42.908441+0000","last_active":"2023-08-27T11:18:44.928760+0000","last_peered":"2023-08-27T11:18:44.928760+0000","last_clean":"2023-08-27T11:18:44.928760+0000","last_became_active":"2023-08-27T11:18:42.908194+0000","last_became_peered":"2023-08-27T11:18:42.908194+0000","last_unstale":"2023-08-27T11:18:44.928760+0000","last_undegraded":"2023-08-27T11:18:44.928760+0000","last_fullsized":"2023-08-27T11:18:44.928760+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:30.369984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502434+0000","last_change":"2023-08-27T11:18:43.908300+0000","last_active":"2023-08-27T11:18:45.502434+0000","last_peered":"2023-08-27T11:18:45.502434+0000","last_clean":"2023-08-27T11:18:45.502434+0000","last_became_active":"2023-08-27T11:18:43.908019+0000","last_became_peered":"2023-08-27T11:18:43.908019+0000","last_unstale":"2023-08-27T11:18:45.502434+0000","last_undegraded":"2023-08-27T11:18:45.502434+0000","last_fullsized":"2023-08-27T11:18:45.502434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:52.379268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928718+0000","last_change":"2023-08-27T11:18:42.901243+0000","last_active":"2023-08-27T11:18:44.928718+0000","last_peered":"2023-08-27T11:18:44.928718+0000","last_clean":"2023-08-27T11:18:44.928718+0000","last_became_active":"2023-08-27T11:18:42.900946+0000","last_became_peered":"2023-08-27T11:18:42.900946+0000","last_unstale":"2023-08-27T11:18:44.928718+0000","last_undegraded":"2023-08-27T11:18:44.928718+0000","last_fullsized":"2023-08-27T11:18:44.928718+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:40:36.762083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.889126+0000","last_change":"2023-08-27T11:18:43.917816+0000","last_active":"2023-08-27T11:18:44.889126+0000","last_peered":"2023-08-27T11:18:44.889126+0000","last_clean":"2023-08-27T11:18:44.889126+0000","last_became_active":"2023-08-27T11:18:43.917601+0000","last_became_peered":"2023-08-27T11:18:43.917601+0000","last_unstale":"2023-08-27T11:18:44.889126+0000","last_undegraded":"2023-08-27T11:18:44.889126+0000","last_fullsized":"2023-08-27T11:18:44.889126+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:57:20.564110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502306+0000","last_change":"2023-08-27T11:18:42.902108+0000","last_active":"2023-08-27T11:18:45.502306+0000","last_peered":"2023-08-27T11:18:45.502306+0000","last_clean":"2023-08-27T11:18:45.502306+0000","last_became_active":"2023-08-27T11:18:42.901851+0000","last_became_peered":"2023-08-27T11:18:42.901851+0000","last_unstale":"2023-08-27T11:18:45.502306+0000","last_undegraded":"2023-08-27T11:18:45.502306+0000","last_fullsized":"2023-08-27T11:18:45.502306+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:57:23.923091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518813+0000","last_change":"2023-08-27T11:18:43.908784+0000","last_active":"2023-08-27T11:18:45.518813+0000","last_peered":"2023-08-27T11:18:45.518813+0000","last_clean":"2023-08-27T11:18:45.518813+0000","last_became_active":"2023-08-27T11:18:43.908491+0000","last_became_peered":"2023-08-27T11:18:43.908491+0000","last_unstale":"2023-08-27T11:18:45.518813+0000","last_undegraded":"2023-08-27T11:18:45.518813+0000","last_fullsized":"2023-08-27T11:18:45.518813+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:21.534978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"50'1","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518328+0000","last_change":"2023-08-27T11:18:42.907313+0000","last_active":"2023-08-27T11:18:45.518328+0000","last_peered":"2023-08-27T11:18:45.518328+0000","last_clean":"2023-08-27T11:18:45.518328+0000","last_became_active":"2023-08-27T11:18:42.907046+0000","last_became_peered":"2023-08-27T11:18:42.907046+0000","last_unstale":"2023-08-27T11:18:45.518328+0000","last_undegraded":"2023-08-27T11:18:45.518328+0000","last_fullsized":"2023-08-27T11:18:45.518328+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:42:45.198442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893493+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:18:44.893493+0000","last_peered":"2023-08-27T11:18:44.893493+0000","last_clean":"2023-08-27T11:18:44.893493+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:18:44.893493+0000","last_undegraded":"2023-08-27T11:18:44.893493+0000","last_fullsized":"2023-08-27T11:18:44.893493+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:12:23.487907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893503+0000","last_change":"2023-08-27T11:18:43.902461+0000","last_active":"2023-08-27T11:18:44.893503+0000","last_peered":"2023-08-27T11:18:44.893503+0000","last_clean":"2023-08-27T11:18:44.893503+0000","last_became_active":"2023-08-27T11:18:43.902182+0000","last_became_peered":"2023-08-27T11:18:43.902182+0000","last_unstale":"2023-08-27T11:18:44.893503+0000","last_undegraded":"2023-08-27T11:18:44.893503+0000","last_fullsized":"2023-08-27T11:18:44.893503+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:50.485759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"50'2","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.519080+0000","last_change":"2023-08-27T11:18:42.904757+0000","last_active":"2023-08-27T11:18:45.519080+0000","last_peered":"2023-08-27T11:18:45.519080+0000","last_clean":"2023-08-27T11:18:45.519080+0000","last_became_active":"2023-08-27T11:18:42.904421+0000","last_became_peered":"2023-08-27T11:18:42.904421+0000","last_unstale":"2023-08-27T11:18:45.519080+0000","last_undegraded":"2023-08-27T11:18:45.519080+0000","last_fullsized":"2023-08-27T11:18:45.519080+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:15:25.441471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893298+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:18:44.893298+0000","last_peered":"2023-08-27T11:18:44.893298+0000","last_clean":"2023-08-27T11:18:44.893298+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:18:44.893298+0000","last_undegraded":"2023-08-27T11:18:44.893298+0000","last_fullsized":"2023-08-27T11:18:44.893298+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:20:00.271710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893596+0000","last_change":"2023-08-27T11:18:43.898611+0000","last_active":"2023-08-27T11:18:44.893596+0000","last_peered":"2023-08-27T11:18:44.893596+0000","last_clean":"2023-08-27T11:18:44.893596+0000","last_became_active":"2023-08-27T11:18:43.897936+0000","last_became_peered":"2023-08-27T11:18:43.897936+0000","last_unstale":"2023-08-27T11:18:44.893596+0000","last_undegraded":"2023-08-27T11:18:44.893596+0000","last_fullsized":"2023-08-27T11:18:44.893596+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:37:06.441816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517217+0000","last_change":"2023-08-27T11:18:42.901517+0000","last_active":"2023-08-27T11:18:45.517217+0000","last_peered":"2023-08-27T11:18:45.517217+0000","last_clean":"2023-08-27T11:18:45.517217+0000","last_became_active":"2023-08-27T11:18:42.901219+0000","last_became_peered":"2023-08-27T11:18:42.901219+0000","last_unstale":"2023-08-27T11:18:45.517217+0000","last_undegraded":"2023-08-27T11:18:45.517217+0000","last_fullsized":"2023-08-27T11:18:45.517217+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:37:05.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518441+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:18:45.518441+0000","last_peered":"2023-08-27T11:18:45.518441+0000","last_clean":"2023-08-27T11:18:45.518441+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:18:45.518441+0000","last_undegraded":"2023-08-27T11:18:45.518441+0000","last_fullsized":"2023-08-27T11:18:45.518441+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:38:53.195617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893064+0000","last_change":"2023-08-27T11:18:43.907891+0000","last_active":"2023-08-27T11:18:44.893064+0000","last_peered":"2023-08-27T11:18:44.893064+0000","last_clean":"2023-08-27T11:18:44.893064+0000","last_became_active":"2023-08-27T11:18:43.907634+0000","last_became_peered":"2023-08-27T11:18:43.907634+0000","last_unstale":"2023-08-27T11:18:44.893064+0000","last_undegraded":"2023-08-27T11:18:44.893064+0000","last_fullsized":"2023-08-27T11:18:44.893064+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:34.428410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928569+0000","last_change":"2023-08-27T11:18:42.899247+0000","last_active":"2023-08-27T11:18:44.928569+0000","last_peered":"2023-08-27T11:18:44.928569+0000","last_clean":"2023-08-27T11:18:44.928569+0000","last_became_active":"2023-08-27T11:18:42.898971+0000","last_became_peered":"2023-08-27T11:18:42.898971+0000","last_unstale":"2023-08-27T11:18:44.928569+0000","last_undegraded":"2023-08-27T11:18:44.928569+0000","last_fullsized":"2023-08-27T11:18:44.928569+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:31:34.208538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.519110+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:18:45.519110+0000","last_peered":"2023-08-27T11:18:45.519110+0000","last_clean":"2023-08-27T11:18:45.519110+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:18:45.519110+0000","last_undegraded":"2023-08-27T11:18:45.519110+0000","last_fullsized":"2023-08-27T11:18:45.519110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:53.227753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518012+0000","last_change":"2023-08-27T11:18:43.906406+0000","last_active":"2023-08-27T11:18:45.518012+0000","last_peered":"2023-08-27T11:18:45.518012+0000","last_clean":"2023-08-27T11:18:45.518012+0000","last_became_active":"2023-08-27T11:18:43.906116+0000","last_became_peered":"2023-08-27T11:18:43.906116+0000","last_unstale":"2023-08-27T11:18:45.518012+0000","last_undegraded":"2023-08-27T11:18:45.518012+0000","last_fullsized":"2023-08-27T11:18:45.518012+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:30.422069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518004+0000","last_change":"2023-08-27T11:18:42.906745+0000","last_active":"2023-08-27T11:18:45.518004+0000","last_peered":"2023-08-27T11:18:45.518004+0000","last_clean":"2023-08-27T11:18:45.518004+0000","last_became_active":"2023-08-27T11:18:42.906486+0000","last_became_peered":"2023-08-27T11:18:42.906486+0000","last_unstale":"2023-08-27T11:18:45.518004+0000","last_undegraded":"2023-08-27T11:18:45.518004+0000","last_fullsized":"2023-08-27T11:18:45.518004+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:58:49.942800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":71,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518718+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:18:45.518718+0000","last_peered":"2023-08-27T11:18:45.518718+0000","last_clean":"2023-08-27T11:18:45.518718+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:18:45.518718+0000","last_undegraded":"2023-08-27T11:18:45.518718+0000","last_fullsized":"2023-08-27T11:18:45.518718+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:46:09.180631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517333+0000","last_change":"2023-08-27T11:18:43.898827+0000","last_active":"2023-08-27T11:18:45.517333+0000","last_peered":"2023-08-27T11:18:45.517333+0000","last_clean":"2023-08-27T11:18:45.517333+0000","last_became_active":"2023-08-27T11:18:43.898544+0000","last_became_peered":"2023-08-27T11:18:43.898544+0000","last_unstale":"2023-08-27T11:18:45.517333+0000","last_undegraded":"2023-08-27T11:18:45.517333+0000","last_fullsized":"2023-08-27T11:18:45.517333+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:57:17.078797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"50'2","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517391+0000","last_change":"2023-08-27T11:18:42.902418+0000","last_active":"2023-08-27T11:18:45.517391+0000","last_peered":"2023-08-27T11:18:45.517391+0000","last_clean":"2023-08-27T11:18:45.517391+0000","last_became_active":"2023-08-27T11:18:42.902127+0000","last_became_peered":"2023-08-27T11:18:42.902127+0000","last_unstale":"2023-08-27T11:18:45.517391+0000","last_undegraded":"2023-08-27T11:18:45.517391+0000","last_fullsized":"2023-08-27T11:18:45.517391+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:52:47.837898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":221,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518467+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:18:45.518467+0000","last_peered":"2023-08-27T11:18:45.518467+0000","last_clean":"2023-08-27T11:18:45.518467+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:18:45.518467+0000","last_undegraded":"2023-08-27T11:18:45.518467+0000","last_fullsized":"2023-08-27T11:18:45.518467+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:21:46.351857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":73,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502602+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:18:45.502602+0000","last_peered":"2023-08-27T11:18:45.502602+0000","last_clean":"2023-08-27T11:18:45.502602+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:18:45.502602+0000","last_undegraded":"2023-08-27T11:18:45.502602+0000","last_fullsized":"2023-08-27T11:18:45.502602+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:42:01.048075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893234+0000","last_change":"2023-08-27T11:18:43.895094+0000","last_active":"2023-08-27T11:18:44.893234+0000","last_peered":"2023-08-27T11:18:44.893234+0000","last_clean":"2023-08-27T11:18:44.893234+0000","last_became_active":"2023-08-27T11:18:43.894340+0000","last_became_peered":"2023-08-27T11:18:43.894340+0000","last_unstale":"2023-08-27T11:18:44.893234+0000","last_undegraded":"2023-08-27T11:18:44.893234+0000","last_fullsized":"2023-08-27T11:18:44.893234+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:18:57.183236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.929064+0000","last_change":"2023-08-27T11:18:42.908525+0000","last_active":"2023-08-27T11:18:44.929064+0000","last_peered":"2023-08-27T11:18:44.929064+0000","last_clean":"2023-08-27T11:18:44.929064+0000","last_became_active":"2023-08-27T11:18:42.908276+0000","last_became_peered":"2023-08-27T11:18:42.908276+0000","last_unstale":"2023-08-27T11:18:44.929064+0000","last_undegraded":"2023-08-27T11:18:44.929064+0000","last_fullsized":"2023-08-27T11:18:44.929064+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:45:18.457607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.929077+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:18:44.929077+0000","last_peered":"2023-08-27T11:18:44.929077+0000","last_clean":"2023-08-27T11:18:44.929077+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:18:44.929077+0000","last_undegraded":"2023-08-27T11:18:44.929077+0000","last_fullsized":"2023-08-27T11:18:44.929077+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:17.215188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892573+0000","last_change":"2023-08-27T11:18:43.912729+0000","last_active":"2023-08-27T11:18:44.892573+0000","last_peered":"2023-08-27T11:18:44.892573+0000","last_clean":"2023-08-27T11:18:44.892573+0000","last_became_active":"2023-08-27T11:18:43.912162+0000","last_became_peered":"2023-08-27T11:18:43.912162+0000","last_unstale":"2023-08-27T11:18:44.892573+0000","last_undegraded":"2023-08-27T11:18:44.892573+0000","last_fullsized":"2023-08-27T11:18:44.892573+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:53.858233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"50'1","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518818+0000","last_change":"2023-08-27T11:18:42.906418+0000","last_active":"2023-08-27T11:18:45.518818+0000","last_peered":"2023-08-27T11:18:45.518818+0000","last_clean":"2023-08-27T11:18:45.518818+0000","last_became_active":"2023-08-27T11:18:42.906122+0000","last_became_peered":"2023-08-27T11:18:42.906122+0000","last_unstale":"2023-08-27T11:18:45.518818+0000","last_undegraded":"2023-08-27T11:18:45.518818+0000","last_fullsized":"2023-08-27T11:18:45.518818+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:52:03.199395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892798+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:18:44.892798+0000","last_peered":"2023-08-27T11:18:44.892798+0000","last_clean":"2023-08-27T11:18:44.892798+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:18:44.892798+0000","last_undegraded":"2023-08-27T11:18:44.892798+0000","last_fullsized":"2023-08-27T11:18:44.892798+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:25:08.248378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":864319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65200000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.70699999999999996}]},{"osd":2,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52200000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.628},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60399999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.435}]},{"osd":6,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.46800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":614400,"data_stored":273209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58499999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.47599999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.64800000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.51100000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.44500000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.61299999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1232896,"data_stored":865919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":11.971,"5min":11.971,"15min":11.971},"last":0.28100000000000003},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":23.045999999999999,"5min":23.045999999999999,"15min":23.045999999999999},"last":0.30199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.928000000000001,"5min":11.928000000000001,"15min":11.928000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.38700000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":10.99,"5min":10.99,"15min":10.99},"last":0.42999999999999999},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":22.893999999999998,"5min":22.893999999999998,"15min":22.893999999999998},"last":0.35799999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":10.957000000000001,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.75,"15min":1.75},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":23.016999999999999,"5min":23.016999999999999,"15min":23.016999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":10.925000000000001,"5min":10.925000000000001,"15min":10.925000000000001},"last":0.751},{"interface":"front","average":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":22.952000000000002,"5min":22.952000000000002,"15min":22.952000000000002},"last":0.623}]},{"osd":7,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":22.988,"5min":22.988,"15min":22.988},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.67200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":606208,"data_stored":271814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.316}]},{"osd":2,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.247}]},{"osd":3,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.371}]},{"osd":4,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.438},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.47299999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.51600000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40799999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.42499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":273175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":0.35099999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.379},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.33600000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.36599999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.71499999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.72099999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.58699999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.79800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":273175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.35899999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.45200000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.503}]},{"osd":5,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.62}]},{"osd":7,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.52800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":622592,"data_stored":274787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.58099999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.46200000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.623}]},{"osd":3,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48099999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.501},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.59599999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":614400,"data_stored":273197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.503},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.54500000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.37},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.55800000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.433},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.48299999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.46700000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.44600000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.53100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:18:48.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:18:48.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.479+0000 7f6521182700 1 Processor -- start 2023-08-27T11:18:48.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.479+0000 7f6521182700 1 -- start start 2023-08-27T11:18:48.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.480+0000 7f6521182700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f651c15a120 0x7f651c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.480+0000 7f6521182700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c06d5b0 0x7f651c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.480+0000 7f6521182700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f651c0b3440 0x7f651c0b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.480+0000 7f6521182700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651c06df50 con 0x7f651c0b3440 2023-08-27T11:18:48.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.480+0000 7f6521182700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651c15b540 con 0x7f651c15a120 2023-08-27T11:18:48.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.480+0000 7f6521182700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0af830 con 0x7f651c06d5b0 2023-08-27T11:18:48.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.480+0000 7f651b59e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f651c0b3440 0x7f651c0b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.480+0000 7f651b59e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f651c0b3440 0x7f651c0b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58008/0 (socket says 172.21.15.112:58008) 2023-08-27T11:18:48.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.480+0000 7f651b59e700 1 -- 172.21.15.112:0/72974081 learned_addr learned my addr 172.21.15.112:0/72974081 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:48.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.480+0000 7f651ad9d700 1 --2- 172.21.15.112:0/72974081 >> v2:172.21.15.138:3300/0 conn(0x7f651c15a120 0x7f651c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.480+0000 7f651a59c700 1 --2- 172.21.15.112:0/72974081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c06d5b0 0x7f651c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f651b59e700 1 -- 172.21.15.112:0/72974081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c06d5b0 msgr2=0x7f651c06da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:48.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f651b59e700 1 --2- 172.21.15.112:0/72974081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c06d5b0 0x7f651c06da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f651b59e700 1 -- 172.21.15.112:0/72974081 >> v2:172.21.15.138:3300/0 conn(0x7f651c15a120 msgr2=0x7f651c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:48.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f651b59e700 1 --2- 172.21.15.112:0/72974081 >> v2:172.21.15.138:3300/0 conn(0x7f651c15a120 0x7f651c0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f651b59e700 1 -- 172.21.15.112:0/72974081 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c0afb10 con 0x7f651c0b3440 2023-08-27T11:18:48.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f651b59e700 1 --2- 172.21.15.112:0/72974081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c0b3440 0x7f651c0b38b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f651c0b6440 tx=0x7f651400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e45bae9ced0231f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:48.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f6519d9b700 1 -- 172.21.15.112:0/72974081 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400e070 con 0x7f651c0b3440 2023-08-27T11:18:48.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f6519d9b700 1 -- 172.21.15.112:0/72974081 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651400b910 con 0x7f651c0b3440 2023-08-27T11:18:48.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f6519d9b700 1 -- 172.21.15.112:0/72974081 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400bcc0 con 0x7f651c0b3440 2023-08-27T11:18:48.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f6521182700 1 -- 172.21.15.112:0/72974081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c0b3440 msgr2=0x7f651c0b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f6521182700 1 --2- 172.21.15.112:0/72974081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c0b3440 0x7f651c0b38b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f651c0b6440 tx=0x7f651400b330 comp rx=0 tx=0).stop 2023-08-27T11:18:48.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f6521182700 1 -- 172.21.15.112:0/72974081 shutdown_connections 2023-08-27T11:18:48.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f6521182700 1 --2- 172.21.15.112:0/72974081 >> v2:172.21.15.138:3300/0 conn(0x7f651c15a120 0x7f651c0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f6521182700 1 --2- 172.21.15.112:0/72974081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c0b3440 0x7f651c0b38b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f6521182700 1 --2- 172.21.15.112:0/72974081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c06d5b0 0x7f651c06da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f6521182700 1 -- 172.21.15.112:0/72974081 >> 172.21.15.112:0/72974081 conn(0x7f651c0cff80 msgr2=0x7f651c0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:48.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f6521182700 1 -- 172.21.15.112:0/72974081 shutdown_connections 2023-08-27T11:18:48.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.481+0000 7f6521182700 1 -- 172.21.15.112:0/72974081 wait complete. 2023-08-27T11:18:48.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.483+0000 7f6521182700 1 Processor -- start 2023-08-27T11:18:48.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.483+0000 7f6521182700 1 -- start start 2023-08-27T11:18:48.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f6521182700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c06d5b0 0x7f651c1403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f6521182700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c0b3440 0x7f651c0a8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f6521182700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c15a120 0x7f651c13fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f6521182700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c140a40 con 0x7f651c06d5b0 2023-08-27T11:18:48.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f6521182700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c140d20 con 0x7f651c15a120 2023-08-27T11:18:48.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f6521182700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c141000 con 0x7f651c0b3440 2023-08-27T11:18:48.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f651ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c06d5b0 0x7f651c1403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f651ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c06d5b0 0x7f651c1403f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58010/0 (socket says 172.21.15.112:58010) 2023-08-27T11:18:48.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f651ad9d700 1 -- 172.21.15.112:0/446386350 learned_addr learned my addr 172.21.15.112:0/446386350 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:48.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f651b59e700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c15a120 0x7f651c13fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f651a59c700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c0b3440 0x7f651c0a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f651ad9d700 1 -- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c0b3440 msgr2=0x7f651c0a8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f651ad9d700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c0b3440 0x7f651c0a8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f651ad9d700 1 -- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c15a120 msgr2=0x7f651c13fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f651ad9d700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c15a120 0x7f651c13fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f651ad9d700 1 -- 172.21.15.112:0/446386350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6514009b30 con 0x7f651c06d5b0 2023-08-27T11:18:48.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f651ad9d700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c06d5b0 0x7f651c1403f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f651000e870 tx=0x7f651000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:48.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f65037fe700 1 -- 172.21.15.112:0/446386350 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6510009590 con 0x7f651c06d5b0 2023-08-27T11:18:48.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f65037fe700 1 -- 172.21.15.112:0/446386350 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6510013680 con 0x7f651c06d5b0 2023-08-27T11:18:48.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.484+0000 7f6521182700 1 -- 172.21.15.112:0/446386350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c1412e0 con 0x7f651c06d5b0 2023-08-27T11:18:48.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.485+0000 7f65037fe700 1 -- 172.21.15.112:0/446386350 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65100139f0 con 0x7f651c06d5b0 2023-08-27T11:18:48.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.485+0000 7f6521182700 1 -- 172.21.15.112:0/446386350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c0a8d80 con 0x7f651c06d5b0 2023-08-27T11:18:48.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.485+0000 7f6521182700 1 -- 172.21.15.112:0/446386350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c141930 con 0x7f651c06d5b0 2023-08-27T11:18:48.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.486+0000 7f65037fe700 1 -- 172.21.15.112:0/446386350 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6510013b50 con 0x7f651c06d5b0 2023-08-27T11:18:48.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.486+0000 7f65037fe700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f650806a0e0 0x7f650806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.486+0000 7f65037fe700 1 -- 172.21.15.112:0/446386350 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6510088320 con 0x7f651c06d5b0 2023-08-27T11:18:48.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.487+0000 7f651a59c700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f650806a0e0 0x7f650806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.487+0000 7f651a59c700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f650806a0e0 0x7f650806c5a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f651c154440 tx=0x7f650c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:48.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.490+0000 7f65037fe700 1 -- 172.21.15.112:0/446386350 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f65100574a0 con 0x7f651c06d5b0 2023-08-27T11:18:48.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.677+0000 7f6521182700 1 -- 172.21.15.112:0/446386350 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f651c0b00d0 con 0x7f650806a0e0 2023-08-27T11:18:48.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.686+0000 7f65037fe700 1 -- 172.21.15.112:0/446386350 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311736 (secure 0 0 0) 0x7f650c011860 con 0x7f650806a0e0 2023-08-27T11:18:48.690 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:48.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.689+0000 7f6521182700 1 -- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f650806a0e0 msgr2=0x7f650806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.689+0000 7f6521182700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f650806a0e0 0x7f650806c5a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f651c154440 tx=0x7f650c009210 comp rx=0 tx=0).stop 2023-08-27T11:18:48.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.689+0000 7f6521182700 1 -- 172.21.15.112:0/446386350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c06d5b0 msgr2=0x7f651c1403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.689+0000 7f6521182700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c06d5b0 0x7f651c1403f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f651000e870 tx=0x7f651000eb80 comp rx=0 tx=0).stop 2023-08-27T11:18:48.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.689+0000 7f6521182700 1 -- 172.21.15.112:0/446386350 shutdown_connections 2023-08-27T11:18:48.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.689+0000 7f6521182700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f650806a0e0 0x7f650806c5a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.689+0000 7f6521182700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c15a120 0x7f651c13fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.689+0000 7f6521182700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c06d5b0 0x7f651c1403f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.689+0000 7f6521182700 1 --2- 172.21.15.112:0/446386350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c0b3440 0x7f651c0a8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.689+0000 7f6521182700 1 -- 172.21.15.112:0/446386350 >> 172.21.15.112:0/446386350 conn(0x7f651c0cff80 msgr2=0x7f651c0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:48.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.689+0000 7f6521182700 1 -- 172.21.15.112:0/446386350 shutdown_connections 2023-08-27T11:18:48.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.689+0000 7f6521182700 1 -- 172.21.15.112:0/446386350 wait complete. 2023-08-27T11:18:48.695 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:18:48.701 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2023-08-27T11:18:48.248441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222180,"kb_used_data":5988,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754752540,"statfs":{"total":773094113280,"available":772866600960,"internally_reserved":0,"allocated":6131712,"data_stored":3369595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001226"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502767+0000","last_change":"2023-08-27T11:18:43.918721+0000","last_active":"2023-08-27T11:18:45.502767+0000","last_peered":"2023-08-27T11:18:45.502767+0000","last_clean":"2023-08-27T11:18:45.502767+0000","last_became_active":"2023-08-27T11:18:43.918486+0000","last_became_peered":"2023-08-27T11:18:43.918486+0000","last_unstale":"2023-08-27T11:18:45.502767+0000","last_undegraded":"2023-08-27T11:18:45.502767+0000","last_fullsized":"2023-08-27T11:18:45.502767+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:53.484375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502730+0000","last_change":"2023-08-27T11:18:43.912694+0000","last_active":"2023-08-27T11:18:45.502730+0000","last_peered":"2023-08-27T11:18:45.502730+0000","last_clean":"2023-08-27T11:18:45.502730+0000","last_became_active":"2023-08-27T11:18:43.912406+0000","last_became_peered":"2023-08-27T11:18:43.912406+0000","last_unstale":"2023-08-27T11:18:45.502730+0000","last_undegraded":"2023-08-27T11:18:45.502730+0000","last_fullsized":"2023-08-27T11:18:45.502730+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:33:08.360188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518952+0000","last_change":"2023-08-27T11:18:43.908731+0000","last_active":"2023-08-27T11:18:45.518952+0000","last_peered":"2023-08-27T11:18:45.518952+0000","last_clean":"2023-08-27T11:18:45.518952+0000","last_became_active":"2023-08-27T11:18:43.908451+0000","last_became_peered":"2023-08-27T11:18:43.908451+0000","last_unstale":"2023-08-27T11:18:45.518952+0000","last_undegraded":"2023-08-27T11:18:45.518952+0000","last_fullsized":"2023-08-27T11:18:45.518952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:36:39.533243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893020+0000","last_change":"2023-08-27T11:18:43.901847+0000","last_active":"2023-08-27T11:18:44.893020+0000","last_peered":"2023-08-27T11:18:44.893020+0000","last_clean":"2023-08-27T11:18:44.893020+0000","last_became_active":"2023-08-27T11:18:43.901203+0000","last_became_peered":"2023-08-27T11:18:43.901203+0000","last_unstale":"2023-08-27T11:18:44.893020+0000","last_undegraded":"2023-08-27T11:18:44.893020+0000","last_fullsized":"2023-08-27T11:18:44.893020+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:25:04.905222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893060+0000","last_change":"2023-08-27T11:18:43.902697+0000","last_active":"2023-08-27T11:18:44.893060+0000","last_peered":"2023-08-27T11:18:44.893060+0000","last_clean":"2023-08-27T11:18:44.893060+0000","last_became_active":"2023-08-27T11:18:43.902427+0000","last_became_peered":"2023-08-27T11:18:43.902427+0000","last_unstale":"2023-08-27T11:18:44.893060+0000","last_undegraded":"2023-08-27T11:18:44.893060+0000","last_fullsized":"2023-08-27T11:18:44.893060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:12:35.805369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517471+0000","last_change":"2023-08-27T11:18:43.915298+0000","last_active":"2023-08-27T11:18:45.517471+0000","last_peered":"2023-08-27T11:18:45.517471+0000","last_clean":"2023-08-27T11:18:45.517471+0000","last_became_active":"2023-08-27T11:18:43.915013+0000","last_became_peered":"2023-08-27T11:18:43.915013+0000","last_unstale":"2023-08-27T11:18:45.517471+0000","last_undegraded":"2023-08-27T11:18:45.517471+0000","last_fullsized":"2023-08-27T11:18:45.517471+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:27:13.341581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.889030+0000","last_change":"2023-08-27T11:18:43.914913+0000","last_active":"2023-08-27T11:18:44.889030+0000","last_peered":"2023-08-27T11:18:44.889030+0000","last_clean":"2023-08-27T11:18:44.889030+0000","last_became_active":"2023-08-27T11:18:43.914650+0000","last_became_peered":"2023-08-27T11:18:43.914650+0000","last_unstale":"2023-08-27T11:18:44.889030+0000","last_undegraded":"2023-08-27T11:18:44.889030+0000","last_fullsized":"2023-08-27T11:18:44.889030+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:13:42.585960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517946+0000","last_change":"2023-08-27T11:18:43.916491+0000","last_active":"2023-08-27T11:18:45.517946+0000","last_peered":"2023-08-27T11:18:45.517946+0000","last_clean":"2023-08-27T11:18:45.517946+0000","last_became_active":"2023-08-27T11:18:43.909631+0000","last_became_peered":"2023-08-27T11:18:43.909631+0000","last_unstale":"2023-08-27T11:18:45.517946+0000","last_undegraded":"2023-08-27T11:18:45.517946+0000","last_fullsized":"2023-08-27T11:18:45.517946+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:40:13.364620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518882+0000","last_change":"2023-08-27T11:18:43.912279+0000","last_active":"2023-08-27T11:18:45.518882+0000","last_peered":"2023-08-27T11:18:45.518882+0000","last_clean":"2023-08-27T11:18:45.518882+0000","last_became_active":"2023-08-27T11:18:43.911263+0000","last_became_peered":"2023-08-27T11:18:43.911263+0000","last_unstale":"2023-08-27T11:18:45.518882+0000","last_undegraded":"2023-08-27T11:18:45.518882+0000","last_fullsized":"2023-08-27T11:18:45.518882+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:02:30.793457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518853+0000","last_change":"2023-08-27T11:18:43.910985+0000","last_active":"2023-08-27T11:18:45.518853+0000","last_peered":"2023-08-27T11:18:45.518853+0000","last_clean":"2023-08-27T11:18:45.518853+0000","last_became_active":"2023-08-27T11:18:43.909150+0000","last_became_peered":"2023-08-27T11:18:43.909150+0000","last_unstale":"2023-08-27T11:18:45.518853+0000","last_undegraded":"2023-08-27T11:18:45.518853+0000","last_fullsized":"2023-08-27T11:18:45.518853+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:41.828758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892411+0000","last_change":"2023-08-27T11:18:43.913114+0000","last_active":"2023-08-27T11:18:44.892411+0000","last_peered":"2023-08-27T11:18:44.892411+0000","last_clean":"2023-08-27T11:18:44.892411+0000","last_became_active":"2023-08-27T11:18:43.912712+0000","last_became_peered":"2023-08-27T11:18:43.912712+0000","last_unstale":"2023-08-27T11:18:44.892411+0000","last_undegraded":"2023-08-27T11:18:44.892411+0000","last_fullsized":"2023-08-27T11:18:44.892411+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:57:21.671903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518035+0000","last_change":"2023-08-27T11:18:43.916509+0000","last_active":"2023-08-27T11:18:45.518035+0000","last_peered":"2023-08-27T11:18:45.518035+0000","last_clean":"2023-08-27T11:18:45.518035+0000","last_became_active":"2023-08-27T11:18:43.909640+0000","last_became_peered":"2023-08-27T11:18:43.909640+0000","last_unstale":"2023-08-27T11:18:45.518035+0000","last_undegraded":"2023-08-27T11:18:45.518035+0000","last_fullsized":"2023-08-27T11:18:45.518035+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:52:31.842605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518857+0000","last_change":"2023-08-27T11:18:43.908914+0000","last_active":"2023-08-27T11:18:45.518857+0000","last_peered":"2023-08-27T11:18:45.518857+0000","last_clean":"2023-08-27T11:18:45.518857+0000","last_became_active":"2023-08-27T11:18:43.908632+0000","last_became_peered":"2023-08-27T11:18:43.908632+0000","last_unstale":"2023-08-27T11:18:45.518857+0000","last_undegraded":"2023-08-27T11:18:45.518857+0000","last_fullsized":"2023-08-27T11:18:45.518857+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:19:43.663705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518742+0000","last_change":"2023-08-27T11:18:43.911227+0000","last_active":"2023-08-27T11:18:45.518742+0000","last_peered":"2023-08-27T11:18:45.518742+0000","last_clean":"2023-08-27T11:18:45.518742+0000","last_became_active":"2023-08-27T11:18:43.910659+0000","last_became_peered":"2023-08-27T11:18:43.910659+0000","last_unstale":"2023-08-27T11:18:45.518742+0000","last_undegraded":"2023-08-27T11:18:45.518742+0000","last_fullsized":"2023-08-27T11:18:45.518742+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:14:52.102360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518382+0000","last_change":"2023-08-27T11:18:43.911371+0000","last_active":"2023-08-27T11:18:45.518382+0000","last_peered":"2023-08-27T11:18:45.518382+0000","last_clean":"2023-08-27T11:18:45.518382+0000","last_became_active":"2023-08-27T11:18:43.909057+0000","last_became_peered":"2023-08-27T11:18:43.909057+0000","last_unstale":"2023-08-27T11:18:45.518382+0000","last_undegraded":"2023-08-27T11:18:45.518382+0000","last_fullsized":"2023-08-27T11:18:45.518382+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:31:27.268258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.888976+0000","last_change":"2023-08-27T11:18:43.909941+0000","last_active":"2023-08-27T11:18:44.888976+0000","last_peered":"2023-08-27T11:18:44.888976+0000","last_clean":"2023-08-27T11:18:44.888976+0000","last_became_active":"2023-08-27T11:18:43.909689+0000","last_became_peered":"2023-08-27T11:18:43.909689+0000","last_unstale":"2023-08-27T11:18:44.888976+0000","last_undegraded":"2023-08-27T11:18:44.888976+0000","last_fullsized":"2023-08-27T11:18:44.888976+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:03:31.839321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502875+0000","last_change":"2023-08-27T11:18:43.912926+0000","last_active":"2023-08-27T11:18:45.502875+0000","last_peered":"2023-08-27T11:18:45.502875+0000","last_clean":"2023-08-27T11:18:45.502875+0000","last_became_active":"2023-08-27T11:18:43.912648+0000","last_became_peered":"2023-08-27T11:18:43.912648+0000","last_unstale":"2023-08-27T11:18:45.502875+0000","last_undegraded":"2023-08-27T11:18:45.502875+0000","last_fullsized":"2023-08-27T11:18:45.502875+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:50.062489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893435+0000","last_change":"2023-08-27T11:18:43.904840+0000","last_active":"2023-08-27T11:18:44.893435+0000","last_peered":"2023-08-27T11:18:44.893435+0000","last_clean":"2023-08-27T11:18:44.893435+0000","last_became_active":"2023-08-27T11:18:43.904544+0000","last_became_peered":"2023-08-27T11:18:43.904544+0000","last_unstale":"2023-08-27T11:18:44.893435+0000","last_undegraded":"2023-08-27T11:18:44.893435+0000","last_fullsized":"2023-08-27T11:18:44.893435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:51:37.831765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893494+0000","last_change":"2023-08-27T11:18:43.899638+0000","last_active":"2023-08-27T11:18:44.893494+0000","last_peered":"2023-08-27T11:18:44.893494+0000","last_clean":"2023-08-27T11:18:44.893494+0000","last_became_active":"2023-08-27T11:18:43.899238+0000","last_became_peered":"2023-08-27T11:18:43.899238+0000","last_unstale":"2023-08-27T11:18:44.893494+0000","last_undegraded":"2023-08-27T11:18:44.893494+0000","last_fullsized":"2023-08-27T11:18:44.893494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:37:47.012098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928979+0000","last_change":"2023-08-27T11:18:43.893563+0000","last_active":"2023-08-27T11:18:44.928979+0000","last_peered":"2023-08-27T11:18:44.928979+0000","last_clean":"2023-08-27T11:18:44.928979+0000","last_became_active":"2023-08-27T11:18:43.893267+0000","last_became_peered":"2023-08-27T11:18:43.893267+0000","last_unstale":"2023-08-27T11:18:44.928979+0000","last_undegraded":"2023-08-27T11:18:44.928979+0000","last_fullsized":"2023-08-27T11:18:44.928979+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:27:32.810244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518997+0000","last_change":"2023-08-27T11:18:43.917971+0000","last_active":"2023-08-27T11:18:45.518997+0000","last_peered":"2023-08-27T11:18:45.518997+0000","last_clean":"2023-08-27T11:18:45.518997+0000","last_became_active":"2023-08-27T11:18:43.916988+0000","last_became_peered":"2023-08-27T11:18:43.916988+0000","last_unstale":"2023-08-27T11:18:45.518997+0000","last_undegraded":"2023-08-27T11:18:45.518997+0000","last_fullsized":"2023-08-27T11:18:45.518997+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:46:59.795902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928701+0000","last_change":"2023-08-27T11:18:43.909254+0000","last_active":"2023-08-27T11:18:44.928701+0000","last_peered":"2023-08-27T11:18:44.928701+0000","last_clean":"2023-08-27T11:18:44.928701+0000","last_became_active":"2023-08-27T11:18:43.908779+0000","last_became_peered":"2023-08-27T11:18:43.908779+0000","last_unstale":"2023-08-27T11:18:44.928701+0000","last_undegraded":"2023-08-27T11:18:44.928701+0000","last_fullsized":"2023-08-27T11:18:44.928701+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:33.720245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928631+0000","last_change":"2023-08-27T11:18:43.909526+0000","last_active":"2023-08-27T11:18:44.928631+0000","last_peered":"2023-08-27T11:18:44.928631+0000","last_clean":"2023-08-27T11:18:44.928631+0000","last_became_active":"2023-08-27T11:18:43.909243+0000","last_became_peered":"2023-08-27T11:18:43.909243+0000","last_unstale":"2023-08-27T11:18:44.928631+0000","last_undegraded":"2023-08-27T11:18:44.928631+0000","last_fullsized":"2023-08-27T11:18:44.928631+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:37:29.743299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518950+0000","last_change":"2023-08-27T11:18:43.906227+0000","last_active":"2023-08-27T11:18:45.518950+0000","last_peered":"2023-08-27T11:18:45.518950+0000","last_clean":"2023-08-27T11:18:45.518950+0000","last_became_active":"2023-08-27T11:18:43.904809+0000","last_became_peered":"2023-08-27T11:18:43.904809+0000","last_unstale":"2023-08-27T11:18:45.518950+0000","last_undegraded":"2023-08-27T11:18:45.518950+0000","last_fullsized":"2023-08-27T11:18:45.518950+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:45:00.093069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518214+0000","last_change":"2023-08-27T11:18:43.909466+0000","last_active":"2023-08-27T11:18:45.518214+0000","last_peered":"2023-08-27T11:18:45.518214+0000","last_clean":"2023-08-27T11:18:45.518214+0000","last_became_active":"2023-08-27T11:18:43.908920+0000","last_became_peered":"2023-08-27T11:18:43.908920+0000","last_unstale":"2023-08-27T11:18:45.518214+0000","last_undegraded":"2023-08-27T11:18:45.518214+0000","last_fullsized":"2023-08-27T11:18:45.518214+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:31:35.215080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518550+0000","last_change":"2023-08-27T11:18:43.917447+0000","last_active":"2023-08-27T11:18:45.518550+0000","last_peered":"2023-08-27T11:18:45.518550+0000","last_clean":"2023-08-27T11:18:45.518550+0000","last_became_active":"2023-08-27T11:18:43.916601+0000","last_became_peered":"2023-08-27T11:18:43.916601+0000","last_unstale":"2023-08-27T11:18:45.518550+0000","last_undegraded":"2023-08-27T11:18:45.518550+0000","last_fullsized":"2023-08-27T11:18:45.518550+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:31.376526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893670+0000","last_change":"2023-08-27T11:18:43.900651+0000","last_active":"2023-08-27T11:18:44.893670+0000","last_peered":"2023-08-27T11:18:44.893670+0000","last_clean":"2023-08-27T11:18:44.893670+0000","last_became_active":"2023-08-27T11:18:43.900317+0000","last_became_peered":"2023-08-27T11:18:43.900317+0000","last_unstale":"2023-08-27T11:18:44.893670+0000","last_undegraded":"2023-08-27T11:18:44.893670+0000","last_fullsized":"2023-08-27T11:18:44.893670+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:29.531830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502459+0000","last_change":"2023-08-27T11:18:43.917447+0000","last_active":"2023-08-27T11:18:45.502459+0000","last_peered":"2023-08-27T11:18:45.502459+0000","last_clean":"2023-08-27T11:18:45.502459+0000","last_became_active":"2023-08-27T11:18:43.917183+0000","last_became_peered":"2023-08-27T11:18:43.917183+0000","last_unstale":"2023-08-27T11:18:45.502459+0000","last_undegraded":"2023-08-27T11:18:45.502459+0000","last_fullsized":"2023-08-27T11:18:45.502459+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:36:11.845513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.519042+0000","last_change":"2023-08-27T11:18:43.905409+0000","last_active":"2023-08-27T11:18:45.519042+0000","last_peered":"2023-08-27T11:18:45.519042+0000","last_clean":"2023-08-27T11:18:45.519042+0000","last_became_active":"2023-08-27T11:18:43.905070+0000","last_became_peered":"2023-08-27T11:18:43.905070+0000","last_unstale":"2023-08-27T11:18:45.519042+0000","last_undegraded":"2023-08-27T11:18:45.519042+0000","last_fullsized":"2023-08-27T11:18:45.519042+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:15:23.130592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517862+0000","last_change":"2023-08-27T11:18:43.917600+0000","last_active":"2023-08-27T11:18:45.517862+0000","last_peered":"2023-08-27T11:18:45.517862+0000","last_clean":"2023-08-27T11:18:45.517862+0000","last_became_active":"2023-08-27T11:18:43.917331+0000","last_became_peered":"2023-08-27T11:18:43.917331+0000","last_unstale":"2023-08-27T11:18:45.517862+0000","last_undegraded":"2023-08-27T11:18:45.517862+0000","last_fullsized":"2023-08-27T11:18:45.517862+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:32:32.604323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892545+0000","last_change":"2023-08-27T11:18:43.902354+0000","last_active":"2023-08-27T11:18:44.892545+0000","last_peered":"2023-08-27T11:18:44.892545+0000","last_clean":"2023-08-27T11:18:44.892545+0000","last_became_active":"2023-08-27T11:18:43.900712+0000","last_became_peered":"2023-08-27T11:18:43.900712+0000","last_unstale":"2023-08-27T11:18:44.892545+0000","last_undegraded":"2023-08-27T11:18:44.892545+0000","last_fullsized":"2023-08-27T11:18:44.892545+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:10.877677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517192+0000","last_change":"2023-08-27T11:18:43.898862+0000","last_active":"2023-08-27T11:18:45.517192+0000","last_peered":"2023-08-27T11:18:45.517192+0000","last_clean":"2023-08-27T11:18:45.517192+0000","last_became_active":"2023-08-27T11:18:43.898600+0000","last_became_peered":"2023-08-27T11:18:43.898600+0000","last_unstale":"2023-08-27T11:18:45.517192+0000","last_undegraded":"2023-08-27T11:18:45.517192+0000","last_fullsized":"2023-08-27T11:18:45.517192+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:14:08.060916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893426+0000","last_change":"2023-08-27T11:18:43.907300+0000","last_active":"2023-08-27T11:18:44.893426+0000","last_peered":"2023-08-27T11:18:44.893426+0000","last_clean":"2023-08-27T11:18:44.893426+0000","last_became_active":"2023-08-27T11:18:43.907032+0000","last_became_peered":"2023-08-27T11:18:43.907032+0000","last_unstale":"2023-08-27T11:18:44.893426+0000","last_undegraded":"2023-08-27T11:18:44.893426+0000","last_fullsized":"2023-08-27T11:18:44.893426+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:17:46.002709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"50'4","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502760+0000","last_change":"2023-08-27T11:18:42.900464+0000","last_active":"2023-08-27T11:18:45.502760+0000","last_peered":"2023-08-27T11:18:45.502760+0000","last_clean":"2023-08-27T11:18:45.502760+0000","last_became_active":"2023-08-27T11:18:42.900160+0000","last_became_peered":"2023-08-27T11:18:42.900160+0000","last_unstale":"2023-08-27T11:18:45.502760+0000","last_undegraded":"2023-08-27T11:18:45.502760+0000","last_fullsized":"2023-08-27T11:18:45.502760+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:00:40.439886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892708+0000","last_change":"2023-08-27T11:18:43.912720+0000","last_active":"2023-08-27T11:18:44.892708+0000","last_peered":"2023-08-27T11:18:44.892708+0000","last_clean":"2023-08-27T11:18:44.892708+0000","last_became_active":"2023-08-27T11:18:43.912050+0000","last_became_peered":"2023-08-27T11:18:43.912050+0000","last_unstale":"2023-08-27T11:18:44.892708+0000","last_undegraded":"2023-08-27T11:18:44.892708+0000","last_fullsized":"2023-08-27T11:18:44.892708+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:34.441587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"50'1","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517611+0000","last_change":"2023-08-27T11:18:42.901531+0000","last_active":"2023-08-27T11:18:45.517611+0000","last_peered":"2023-08-27T11:18:45.517611+0000","last_clean":"2023-08-27T11:18:45.517611+0000","last_became_active":"2023-08-27T11:18:42.901259+0000","last_became_peered":"2023-08-27T11:18:42.901259+0000","last_unstale":"2023-08-27T11:18:45.517611+0000","last_undegraded":"2023-08-27T11:18:45.517611+0000","last_fullsized":"2023-08-27T11:18:45.517611+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:53:27.091530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517675+0000","last_change":"2023-08-27T11:18:43.904695+0000","last_active":"2023-08-27T11:18:45.517675+0000","last_peered":"2023-08-27T11:18:45.517675+0000","last_clean":"2023-08-27T11:18:45.517675+0000","last_became_active":"2023-08-27T11:18:43.898563+0000","last_became_peered":"2023-08-27T11:18:43.898563+0000","last_unstale":"2023-08-27T11:18:45.517675+0000","last_undegraded":"2023-08-27T11:18:45.517675+0000","last_fullsized":"2023-08-27T11:18:45.517675+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:28.772925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"50'2","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517543+0000","last_change":"2023-08-27T11:18:42.903728+0000","last_active":"2023-08-27T11:18:45.517543+0000","last_peered":"2023-08-27T11:18:45.517543+0000","last_clean":"2023-08-27T11:18:45.517543+0000","last_became_active":"2023-08-27T11:18:42.903514+0000","last_became_peered":"2023-08-27T11:18:42.903514+0000","last_unstale":"2023-08-27T11:18:45.517543+0000","last_undegraded":"2023-08-27T11:18:45.517543+0000","last_fullsized":"2023-08-27T11:18:45.517543+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:40:21.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.519139+0000","last_change":"2023-08-27T11:18:43.913720+0000","last_active":"2023-08-27T11:18:45.519139+0000","last_peered":"2023-08-27T11:18:45.519139+0000","last_clean":"2023-08-27T11:18:45.519139+0000","last_became_active":"2023-08-27T11:18:43.913365+0000","last_became_peered":"2023-08-27T11:18:43.913365+0000","last_unstale":"2023-08-27T11:18:45.519139+0000","last_undegraded":"2023-08-27T11:18:45.519139+0000","last_fullsized":"2023-08-27T11:18:45.519139+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:00:42.860280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"50'2","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517535+0000","last_change":"2023-08-27T11:18:42.896270+0000","last_active":"2023-08-27T11:18:45.517535+0000","last_peered":"2023-08-27T11:18:45.517535+0000","last_clean":"2023-08-27T11:18:45.517535+0000","last_became_active":"2023-08-27T11:18:42.895964+0000","last_became_peered":"2023-08-27T11:18:42.895964+0000","last_unstale":"2023-08-27T11:18:45.517535+0000","last_undegraded":"2023-08-27T11:18:45.517535+0000","last_fullsized":"2023-08-27T11:18:45.517535+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:43:26.881783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.889025+0000","last_change":"2023-08-27T11:18:43.906687+0000","last_active":"2023-08-27T11:18:44.889025+0000","last_peered":"2023-08-27T11:18:44.889025+0000","last_clean":"2023-08-27T11:18:44.889025+0000","last_became_active":"2023-08-27T11:18:43.906259+0000","last_became_peered":"2023-08-27T11:18:43.906259+0000","last_unstale":"2023-08-27T11:18:44.889025+0000","last_undegraded":"2023-08-27T11:18:44.889025+0000","last_fullsized":"2023-08-27T11:18:44.889025+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:22:21.995766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"50'1","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928888+0000","last_change":"2023-08-27T11:18:42.908241+0000","last_active":"2023-08-27T11:18:44.928888+0000","last_peered":"2023-08-27T11:18:44.928888+0000","last_clean":"2023-08-27T11:18:44.928888+0000","last_became_active":"2023-08-27T11:18:42.907925+0000","last_became_peered":"2023-08-27T11:18:42.907925+0000","last_unstale":"2023-08-27T11:18:44.928888+0000","last_undegraded":"2023-08-27T11:18:44.928888+0000","last_fullsized":"2023-08-27T11:18:44.928888+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:46:58.789360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502405+0000","last_change":"2023-08-27T11:18:43.908246+0000","last_active":"2023-08-27T11:18:45.502405+0000","last_peered":"2023-08-27T11:18:45.502405+0000","last_clean":"2023-08-27T11:18:45.502405+0000","last_became_active":"2023-08-27T11:18:43.907979+0000","last_became_peered":"2023-08-27T11:18:43.907979+0000","last_unstale":"2023-08-27T11:18:45.502405+0000","last_undegraded":"2023-08-27T11:18:45.502405+0000","last_fullsized":"2023-08-27T11:18:45.502405+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:21.921792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"50'1","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928647+0000","last_change":"2023-08-27T11:18:42.908733+0000","last_active":"2023-08-27T11:18:44.928647+0000","last_peered":"2023-08-27T11:18:44.928647+0000","last_clean":"2023-08-27T11:18:44.928647+0000","last_became_active":"2023-08-27T11:18:42.908491+0000","last_became_peered":"2023-08-27T11:18:42.908491+0000","last_unstale":"2023-08-27T11:18:44.928647+0000","last_undegraded":"2023-08-27T11:18:44.928647+0000","last_fullsized":"2023-08-27T11:18:44.928647+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:18:45.188825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893787+0000","last_change":"2023-08-27T11:18:43.903492+0000","last_active":"2023-08-27T11:18:44.893787+0000","last_peered":"2023-08-27T11:18:44.893787+0000","last_clean":"2023-08-27T11:18:44.893787+0000","last_became_active":"2023-08-27T11:18:43.898349+0000","last_became_peered":"2023-08-27T11:18:43.898349+0000","last_unstale":"2023-08-27T11:18:44.893787+0000","last_undegraded":"2023-08-27T11:18:44.893787+0000","last_fullsized":"2023-08-27T11:18:44.893787+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:57:17.078797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928640+0000","last_change":"2023-08-27T11:18:42.908350+0000","last_active":"2023-08-27T11:18:44.928640+0000","last_peered":"2023-08-27T11:18:44.928640+0000","last_clean":"2023-08-27T11:18:44.928640+0000","last_became_active":"2023-08-27T11:18:42.907970+0000","last_became_peered":"2023-08-27T11:18:42.907970+0000","last_unstale":"2023-08-27T11:18:44.928640+0000","last_undegraded":"2023-08-27T11:18:44.928640+0000","last_fullsized":"2023-08-27T11:18:44.928640+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:42:45.198442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928826+0000","last_change":"2023-08-27T11:18:43.904771+0000","last_active":"2023-08-27T11:18:44.928826+0000","last_peered":"2023-08-27T11:18:44.928826+0000","last_clean":"2023-08-27T11:18:44.928826+0000","last_became_active":"2023-08-27T11:18:43.904503+0000","last_became_peered":"2023-08-27T11:18:43.904503+0000","last_unstale":"2023-08-27T11:18:44.928826+0000","last_undegraded":"2023-08-27T11:18:44.928826+0000","last_fullsized":"2023-08-27T11:18:44.928826+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:19:43.663705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893161+0000","last_change":"2023-08-27T11:18:42.902151+0000","last_active":"2023-08-27T11:18:44.893161+0000","last_peered":"2023-08-27T11:18:44.893161+0000","last_clean":"2023-08-27T11:18:44.893161+0000","last_became_active":"2023-08-27T11:18:42.901900+0000","last_became_peered":"2023-08-27T11:18:42.901900+0000","last_unstale":"2023-08-27T11:18:44.893161+0000","last_undegraded":"2023-08-27T11:18:44.893161+0000","last_fullsized":"2023-08-27T11:18:44.893161+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:02:12.354166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502488+0000","last_change":"2023-08-27T11:18:43.905888+0000","last_active":"2023-08-27T11:18:45.502488+0000","last_peered":"2023-08-27T11:18:45.502488+0000","last_clean":"2023-08-27T11:18:45.502488+0000","last_became_active":"2023-08-27T11:18:43.905641+0000","last_became_peered":"2023-08-27T11:18:43.905641+0000","last_unstale":"2023-08-27T11:18:45.502488+0000","last_undegraded":"2023-08-27T11:18:45.502488+0000","last_fullsized":"2023-08-27T11:18:45.502488+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:25:01.340661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.519238+0000","last_change":"2023-08-27T11:18:42.906290+0000","last_active":"2023-08-27T11:18:45.519238+0000","last_peered":"2023-08-27T11:18:45.519238+0000","last_clean":"2023-08-27T11:18:45.519238+0000","last_became_active":"2023-08-27T11:18:42.905992+0000","last_became_peered":"2023-08-27T11:18:42.905992+0000","last_unstale":"2023-08-27T11:18:45.519238+0000","last_undegraded":"2023-08-27T11:18:45.519238+0000","last_fullsized":"2023-08-27T11:18:45.519238+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:22:36.744418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.929026+0000","last_change":"2023-08-27T11:18:43.909062+0000","last_active":"2023-08-27T11:18:44.929026+0000","last_peered":"2023-08-27T11:18:44.929026+0000","last_clean":"2023-08-27T11:18:44.929026+0000","last_became_active":"2023-08-27T11:18:43.908769+0000","last_became_peered":"2023-08-27T11:18:43.908769+0000","last_unstale":"2023-08-27T11:18:44.929026+0000","last_undegraded":"2023-08-27T11:18:44.929026+0000","last_fullsized":"2023-08-27T11:18:44.929026+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:52:38.617545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893250+0000","last_change":"2023-08-27T11:18:42.898836+0000","last_active":"2023-08-27T11:18:44.893250+0000","last_peered":"2023-08-27T11:18:44.893250+0000","last_clean":"2023-08-27T11:18:44.893250+0000","last_became_active":"2023-08-27T11:18:42.898515+0000","last_became_peered":"2023-08-27T11:18:42.898515+0000","last_unstale":"2023-08-27T11:18:44.893250+0000","last_undegraded":"2023-08-27T11:18:44.893250+0000","last_fullsized":"2023-08-27T11:18:44.893250+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:01:43.305122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893211+0000","last_change":"2023-08-27T11:18:43.919688+0000","last_active":"2023-08-27T11:18:44.893211+0000","last_peered":"2023-08-27T11:18:44.893211+0000","last_clean":"2023-08-27T11:18:44.893211+0000","last_became_active":"2023-08-27T11:18:43.919401+0000","last_became_peered":"2023-08-27T11:18:43.919401+0000","last_unstale":"2023-08-27T11:18:44.893211+0000","last_undegraded":"2023-08-27T11:18:44.893211+0000","last_fullsized":"2023-08-27T11:18:44.893211+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:16:13.567306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"50'3","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518629+0000","last_change":"2023-08-27T11:18:42.906276+0000","last_active":"2023-08-27T11:18:45.518629+0000","last_peered":"2023-08-27T11:18:45.518629+0000","last_clean":"2023-08-27T11:18:45.518629+0000","last_became_active":"2023-08-27T11:18:42.906009+0000","last_became_peered":"2023-08-27T11:18:42.906009+0000","last_unstale":"2023-08-27T11:18:45.518629+0000","last_undegraded":"2023-08-27T11:18:45.518629+0000","last_fullsized":"2023-08-27T11:18:45.518629+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:57.143682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893146+0000","last_change":"2023-08-27T11:18:43.903954+0000","last_active":"2023-08-27T11:18:44.893146+0000","last_peered":"2023-08-27T11:18:44.893146+0000","last_clean":"2023-08-27T11:18:44.893146+0000","last_became_active":"2023-08-27T11:18:43.903682+0000","last_became_peered":"2023-08-27T11:18:43.903682+0000","last_unstale":"2023-08-27T11:18:44.893146+0000","last_undegraded":"2023-08-27T11:18:44.893146+0000","last_fullsized":"2023-08-27T11:18:44.893146+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:52:46.854872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"50'6","reported_seq":19,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502636+0000","last_change":"2023-08-27T11:18:42.897689+0000","last_active":"2023-08-27T11:18:45.502636+0000","last_peered":"2023-08-27T11:18:45.502636+0000","last_clean":"2023-08-27T11:18:45.502636+0000","last_became_active":"2023-08-27T11:18:42.897445+0000","last_became_peered":"2023-08-27T11:18:42.897445+0000","last_unstale":"2023-08-27T11:18:45.502636+0000","last_undegraded":"2023-08-27T11:18:45.502636+0000","last_fullsized":"2023-08-27T11:18:45.502636+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:28:49.672644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892223+0000","last_change":"2023-08-27T11:18:43.915954+0000","last_active":"2023-08-27T11:18:44.892223+0000","last_peered":"2023-08-27T11:18:44.892223+0000","last_clean":"2023-08-27T11:18:44.892223+0000","last_became_active":"2023-08-27T11:18:43.915021+0000","last_became_peered":"2023-08-27T11:18:43.915021+0000","last_unstale":"2023-08-27T11:18:44.892223+0000","last_undegraded":"2023-08-27T11:18:44.892223+0000","last_fullsized":"2023-08-27T11:18:44.892223+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:46.250380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"50'2","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518674+0000","last_change":"2023-08-27T11:18:42.905644+0000","last_active":"2023-08-27T11:18:45.518674+0000","last_peered":"2023-08-27T11:18:45.518674+0000","last_clean":"2023-08-27T11:18:45.518674+0000","last_became_active":"2023-08-27T11:18:42.905180+0000","last_became_peered":"2023-08-27T11:18:42.905180+0000","last_unstale":"2023-08-27T11:18:45.518674+0000","last_undegraded":"2023-08-27T11:18:45.518674+0000","last_fullsized":"2023-08-27T11:18:45.518674+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:21:05.202954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518160+0000","last_change":"2023-08-27T11:18:43.908688+0000","last_active":"2023-08-27T11:18:45.518160+0000","last_peered":"2023-08-27T11:18:45.518160+0000","last_clean":"2023-08-27T11:18:45.518160+0000","last_became_active":"2023-08-27T11:18:43.908427+0000","last_became_peered":"2023-08-27T11:18:43.908427+0000","last_unstale":"2023-08-27T11:18:45.518160+0000","last_undegraded":"2023-08-27T11:18:45.518160+0000","last_fullsized":"2023-08-27T11:18:45.518160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:03:18.205212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517935+0000","last_change":"2023-08-27T11:18:42.907812+0000","last_active":"2023-08-27T11:18:45.517935+0000","last_peered":"2023-08-27T11:18:45.517935+0000","last_clean":"2023-08-27T11:18:45.517935+0000","last_became_active":"2023-08-27T11:18:42.907351+0000","last_became_peered":"2023-08-27T11:18:42.907351+0000","last_unstale":"2023-08-27T11:18:45.517935+0000","last_undegraded":"2023-08-27T11:18:45.517935+0000","last_fullsized":"2023-08-27T11:18:45.517935+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:22:52.323702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893070+0000","last_change":"2023-08-27T11:18:43.899866+0000","last_active":"2023-08-27T11:18:44.893070+0000","last_peered":"2023-08-27T11:18:44.893070+0000","last_clean":"2023-08-27T11:18:44.893070+0000","last_became_active":"2023-08-27T11:18:43.899580+0000","last_became_peered":"2023-08-27T11:18:43.899580+0000","last_unstale":"2023-08-27T11:18:44.893070+0000","last_undegraded":"2023-08-27T11:18:44.893070+0000","last_fullsized":"2023-08-27T11:18:44.893070+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:34:45.726494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.519058+0000","last_change":"2023-08-27T11:18:42.909897+0000","last_active":"2023-08-27T11:18:45.519058+0000","last_peered":"2023-08-27T11:18:45.519058+0000","last_clean":"2023-08-27T11:18:45.519058+0000","last_became_active":"2023-08-27T11:18:42.909605+0000","last_became_peered":"2023-08-27T11:18:42.909605+0000","last_unstale":"2023-08-27T11:18:45.519058+0000","last_undegraded":"2023-08-27T11:18:45.519058+0000","last_fullsized":"2023-08-27T11:18:45.519058+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:45.951900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502581+0000","last_change":"2023-08-27T11:18:43.909878+0000","last_active":"2023-08-27T11:18:45.502581+0000","last_peered":"2023-08-27T11:18:45.502581+0000","last_clean":"2023-08-27T11:18:45.502581+0000","last_became_active":"2023-08-27T11:18:43.909629+0000","last_became_peered":"2023-08-27T11:18:43.909629+0000","last_unstale":"2023-08-27T11:18:45.502581+0000","last_undegraded":"2023-08-27T11:18:45.502581+0000","last_fullsized":"2023-08-27T11:18:45.502581+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:25:48.807928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"50'8","reported_seq":21,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893559+0000","last_change":"2023-08-27T11:18:42.901981+0000","last_active":"2023-08-27T11:18:44.893559+0000","last_peered":"2023-08-27T11:18:44.893559+0000","last_clean":"2023-08-27T11:18:44.893559+0000","last_became_active":"2023-08-27T11:18:42.901728+0000","last_became_peered":"2023-08-27T11:18:42.901728+0000","last_unstale":"2023-08-27T11:18:44.893559+0000","last_undegraded":"2023-08-27T11:18:44.893559+0000","last_fullsized":"2023-08-27T11:18:44.893559+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:46:43.863658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517552+0000","last_change":"2023-08-27T11:18:43.901514+0000","last_active":"2023-08-27T11:18:45.517552+0000","last_peered":"2023-08-27T11:18:45.517552+0000","last_clean":"2023-08-27T11:18:45.517552+0000","last_became_active":"2023-08-27T11:18:43.901085+0000","last_became_peered":"2023-08-27T11:18:43.901085+0000","last_unstale":"2023-08-27T11:18:45.517552+0000","last_undegraded":"2023-08-27T11:18:45.517552+0000","last_fullsized":"2023-08-27T11:18:45.517552+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:35:59.130505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"50'2","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518373+0000","last_change":"2023-08-27T11:18:42.902004+0000","last_active":"2023-08-27T11:18:45.518373+0000","last_peered":"2023-08-27T11:18:45.518373+0000","last_clean":"2023-08-27T11:18:45.518373+0000","last_became_active":"2023-08-27T11:18:42.901718+0000","last_became_peered":"2023-08-27T11:18:42.901718+0000","last_unstale":"2023-08-27T11:18:45.518373+0000","last_undegraded":"2023-08-27T11:18:45.518373+0000","last_fullsized":"2023-08-27T11:18:45.518373+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:32.713703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.889225+0000","last_change":"2023-08-27T11:18:43.902858+0000","last_active":"2023-08-27T11:18:44.889225+0000","last_peered":"2023-08-27T11:18:44.889225+0000","last_clean":"2023-08-27T11:18:44.889225+0000","last_became_active":"2023-08-27T11:18:43.902603+0000","last_became_peered":"2023-08-27T11:18:43.902603+0000","last_unstale":"2023-08-27T11:18:44.889225+0000","last_undegraded":"2023-08-27T11:18:44.889225+0000","last_fullsized":"2023-08-27T11:18:44.889225+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:58.280505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518365+0000","last_change":"2023-08-27T11:18:42.902778+0000","last_active":"2023-08-27T11:18:45.518365+0000","last_peered":"2023-08-27T11:18:45.518365+0000","last_clean":"2023-08-27T11:18:45.518365+0000","last_became_active":"2023-08-27T11:18:42.902455+0000","last_became_peered":"2023-08-27T11:18:42.902455+0000","last_unstale":"2023-08-27T11:18:45.518365+0000","last_undegraded":"2023-08-27T11:18:45.518365+0000","last_fullsized":"2023-08-27T11:18:45.518365+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:18:45.188825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518257+0000","last_change":"2023-08-27T11:18:43.917693+0000","last_active":"2023-08-27T11:18:45.518257+0000","last_peered":"2023-08-27T11:18:45.518257+0000","last_clean":"2023-08-27T11:18:45.518257+0000","last_became_active":"2023-08-27T11:18:43.916404+0000","last_became_peered":"2023-08-27T11:18:43.916404+0000","last_unstale":"2023-08-27T11:18:45.518257+0000","last_undegraded":"2023-08-27T11:18:45.518257+0000","last_fullsized":"2023-08-27T11:18:45.518257+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:37:29.743299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517635+0000","last_change":"2023-08-27T11:18:42.903712+0000","last_active":"2023-08-27T11:18:45.517635+0000","last_peered":"2023-08-27T11:18:45.517635+0000","last_clean":"2023-08-27T11:18:45.517635+0000","last_became_active":"2023-08-27T11:18:42.903476+0000","last_became_peered":"2023-08-27T11:18:42.903476+0000","last_unstale":"2023-08-27T11:18:45.517635+0000","last_undegraded":"2023-08-27T11:18:45.517635+0000","last_fullsized":"2023-08-27T11:18:45.517635+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:49:45.344498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893474+0000","last_change":"2023-08-27T11:18:43.895176+0000","last_active":"2023-08-27T11:18:44.893474+0000","last_peered":"2023-08-27T11:18:44.893474+0000","last_clean":"2023-08-27T11:18:44.893474+0000","last_became_active":"2023-08-27T11:18:43.894779+0000","last_became_peered":"2023-08-27T11:18:43.894779+0000","last_unstale":"2023-08-27T11:18:44.893474+0000","last_undegraded":"2023-08-27T11:18:44.893474+0000","last_fullsized":"2023-08-27T11:18:44.893474+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:38:44.087181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502756+0000","last_change":"2023-08-27T11:18:42.897646+0000","last_active":"2023-08-27T11:18:45.502756+0000","last_peered":"2023-08-27T11:18:45.502756+0000","last_clean":"2023-08-27T11:18:45.502756+0000","last_became_active":"2023-08-27T11:18:42.897380+0000","last_became_peered":"2023-08-27T11:18:42.897380+0000","last_unstale":"2023-08-27T11:18:45.502756+0000","last_undegraded":"2023-08-27T11:18:45.502756+0000","last_fullsized":"2023-08-27T11:18:45.502756+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:34:05.074779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892913+0000","last_change":"2023-08-27T11:18:43.905727+0000","last_active":"2023-08-27T11:18:44.892913+0000","last_peered":"2023-08-27T11:18:44.892913+0000","last_clean":"2023-08-27T11:18:44.892913+0000","last_became_active":"2023-08-27T11:18:43.905459+0000","last_became_peered":"2023-08-27T11:18:43.905459+0000","last_unstale":"2023-08-27T11:18:44.892913+0000","last_undegraded":"2023-08-27T11:18:44.892913+0000","last_fullsized":"2023-08-27T11:18:44.892913+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:34:47.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518829+0000","last_change":"2023-08-27T11:18:42.909814+0000","last_active":"2023-08-27T11:18:45.518829+0000","last_peered":"2023-08-27T11:18:45.518829+0000","last_clean":"2023-08-27T11:18:45.518829+0000","last_became_active":"2023-08-27T11:18:42.909583+0000","last_became_peered":"2023-08-27T11:18:42.909583+0000","last_unstale":"2023-08-27T11:18:45.518829+0000","last_undegraded":"2023-08-27T11:18:45.518829+0000","last_fullsized":"2023-08-27T11:18:45.518829+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:04:51.015111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892714+0000","last_change":"2023-08-27T11:18:43.915183+0000","last_active":"2023-08-27T11:18:44.892714+0000","last_peered":"2023-08-27T11:18:44.892714+0000","last_clean":"2023-08-27T11:18:44.892714+0000","last_became_active":"2023-08-27T11:18:43.914883+0000","last_became_peered":"2023-08-27T11:18:43.914883+0000","last_unstale":"2023-08-27T11:18:44.892714+0000","last_undegraded":"2023-08-27T11:18:44.892714+0000","last_fullsized":"2023-08-27T11:18:44.892714+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:44:28.059085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928760+0000","last_change":"2023-08-27T11:18:42.908441+0000","last_active":"2023-08-27T11:18:44.928760+0000","last_peered":"2023-08-27T11:18:44.928760+0000","last_clean":"2023-08-27T11:18:44.928760+0000","last_became_active":"2023-08-27T11:18:42.908194+0000","last_became_peered":"2023-08-27T11:18:42.908194+0000","last_unstale":"2023-08-27T11:18:44.928760+0000","last_undegraded":"2023-08-27T11:18:44.928760+0000","last_fullsized":"2023-08-27T11:18:44.928760+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:30.369984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502434+0000","last_change":"2023-08-27T11:18:43.908300+0000","last_active":"2023-08-27T11:18:45.502434+0000","last_peered":"2023-08-27T11:18:45.502434+0000","last_clean":"2023-08-27T11:18:45.502434+0000","last_became_active":"2023-08-27T11:18:43.908019+0000","last_became_peered":"2023-08-27T11:18:43.908019+0000","last_unstale":"2023-08-27T11:18:45.502434+0000","last_undegraded":"2023-08-27T11:18:45.502434+0000","last_fullsized":"2023-08-27T11:18:45.502434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:52.379268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928718+0000","last_change":"2023-08-27T11:18:42.901243+0000","last_active":"2023-08-27T11:18:44.928718+0000","last_peered":"2023-08-27T11:18:44.928718+0000","last_clean":"2023-08-27T11:18:44.928718+0000","last_became_active":"2023-08-27T11:18:42.900946+0000","last_became_peered":"2023-08-27T11:18:42.900946+0000","last_unstale":"2023-08-27T11:18:44.928718+0000","last_undegraded":"2023-08-27T11:18:44.928718+0000","last_fullsized":"2023-08-27T11:18:44.928718+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:40:36.762083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.889126+0000","last_change":"2023-08-27T11:18:43.917816+0000","last_active":"2023-08-27T11:18:44.889126+0000","last_peered":"2023-08-27T11:18:44.889126+0000","last_clean":"2023-08-27T11:18:44.889126+0000","last_became_active":"2023-08-27T11:18:43.917601+0000","last_became_peered":"2023-08-27T11:18:43.917601+0000","last_unstale":"2023-08-27T11:18:44.889126+0000","last_undegraded":"2023-08-27T11:18:44.889126+0000","last_fullsized":"2023-08-27T11:18:44.889126+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:57:20.564110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502306+0000","last_change":"2023-08-27T11:18:42.902108+0000","last_active":"2023-08-27T11:18:45.502306+0000","last_peered":"2023-08-27T11:18:45.502306+0000","last_clean":"2023-08-27T11:18:45.502306+0000","last_became_active":"2023-08-27T11:18:42.901851+0000","last_became_peered":"2023-08-27T11:18:42.901851+0000","last_unstale":"2023-08-27T11:18:45.502306+0000","last_undegraded":"2023-08-27T11:18:45.502306+0000","last_fullsized":"2023-08-27T11:18:45.502306+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:57:23.923091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518813+0000","last_change":"2023-08-27T11:18:43.908784+0000","last_active":"2023-08-27T11:18:45.518813+0000","last_peered":"2023-08-27T11:18:45.518813+0000","last_clean":"2023-08-27T11:18:45.518813+0000","last_became_active":"2023-08-27T11:18:43.908491+0000","last_became_peered":"2023-08-27T11:18:43.908491+0000","last_unstale":"2023-08-27T11:18:45.518813+0000","last_undegraded":"2023-08-27T11:18:45.518813+0000","last_fullsized":"2023-08-27T11:18:45.518813+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:21.534978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"50'1","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518328+0000","last_change":"2023-08-27T11:18:42.907313+0000","last_active":"2023-08-27T11:18:45.518328+0000","last_peered":"2023-08-27T11:18:45.518328+0000","last_clean":"2023-08-27T11:18:45.518328+0000","last_became_active":"2023-08-27T11:18:42.907046+0000","last_became_peered":"2023-08-27T11:18:42.907046+0000","last_unstale":"2023-08-27T11:18:45.518328+0000","last_undegraded":"2023-08-27T11:18:45.518328+0000","last_fullsized":"2023-08-27T11:18:45.518328+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:42:45.198442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893493+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:18:44.893493+0000","last_peered":"2023-08-27T11:18:44.893493+0000","last_clean":"2023-08-27T11:18:44.893493+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:18:44.893493+0000","last_undegraded":"2023-08-27T11:18:44.893493+0000","last_fullsized":"2023-08-27T11:18:44.893493+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:12:23.487907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893503+0000","last_change":"2023-08-27T11:18:43.902461+0000","last_active":"2023-08-27T11:18:44.893503+0000","last_peered":"2023-08-27T11:18:44.893503+0000","last_clean":"2023-08-27T11:18:44.893503+0000","last_became_active":"2023-08-27T11:18:43.902182+0000","last_became_peered":"2023-08-27T11:18:43.902182+0000","last_unstale":"2023-08-27T11:18:44.893503+0000","last_undegraded":"2023-08-27T11:18:44.893503+0000","last_fullsized":"2023-08-27T11:18:44.893503+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:50.485759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"50'2","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.519080+0000","last_change":"2023-08-27T11:18:42.904757+0000","last_active":"2023-08-27T11:18:45.519080+0000","last_peered":"2023-08-27T11:18:45.519080+0000","last_clean":"2023-08-27T11:18:45.519080+0000","last_became_active":"2023-08-27T11:18:42.904421+0000","last_became_peered":"2023-08-27T11:18:42.904421+0000","last_unstale":"2023-08-27T11:18:45.519080+0000","last_undegraded":"2023-08-27T11:18:45.519080+0000","last_fullsized":"2023-08-27T11:18:45.519080+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:15:25.441471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893298+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:18:44.893298+0000","last_peered":"2023-08-27T11:18:44.893298+0000","last_clean":"2023-08-27T11:18:44.893298+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:18:44.893298+0000","last_undegraded":"2023-08-27T11:18:44.893298+0000","last_fullsized":"2023-08-27T11:18:44.893298+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:20:00.271710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893596+0000","last_change":"2023-08-27T11:18:43.898611+0000","last_active":"2023-08-27T11:18:44.893596+0000","last_peered":"2023-08-27T11:18:44.893596+0000","last_clean":"2023-08-27T11:18:44.893596+0000","last_became_active":"2023-08-27T11:18:43.897936+0000","last_became_peered":"2023-08-27T11:18:43.897936+0000","last_unstale":"2023-08-27T11:18:44.893596+0000","last_undegraded":"2023-08-27T11:18:44.893596+0000","last_fullsized":"2023-08-27T11:18:44.893596+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:37:06.441816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517217+0000","last_change":"2023-08-27T11:18:42.901517+0000","last_active":"2023-08-27T11:18:45.517217+0000","last_peered":"2023-08-27T11:18:45.517217+0000","last_clean":"2023-08-27T11:18:45.517217+0000","last_became_active":"2023-08-27T11:18:42.901219+0000","last_became_peered":"2023-08-27T11:18:42.901219+0000","last_unstale":"2023-08-27T11:18:45.517217+0000","last_undegraded":"2023-08-27T11:18:45.517217+0000","last_fullsized":"2023-08-27T11:18:45.517217+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:37:05.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518441+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:18:45.518441+0000","last_peered":"2023-08-27T11:18:45.518441+0000","last_clean":"2023-08-27T11:18:45.518441+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:18:45.518441+0000","last_undegraded":"2023-08-27T11:18:45.518441+0000","last_fullsized":"2023-08-27T11:18:45.518441+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:38:53.195617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893064+0000","last_change":"2023-08-27T11:18:43.907891+0000","last_active":"2023-08-27T11:18:44.893064+0000","last_peered":"2023-08-27T11:18:44.893064+0000","last_clean":"2023-08-27T11:18:44.893064+0000","last_became_active":"2023-08-27T11:18:43.907634+0000","last_became_peered":"2023-08-27T11:18:43.907634+0000","last_unstale":"2023-08-27T11:18:44.893064+0000","last_undegraded":"2023-08-27T11:18:44.893064+0000","last_fullsized":"2023-08-27T11:18:44.893064+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:34.428410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.928569+0000","last_change":"2023-08-27T11:18:42.899247+0000","last_active":"2023-08-27T11:18:44.928569+0000","last_peered":"2023-08-27T11:18:44.928569+0000","last_clean":"2023-08-27T11:18:44.928569+0000","last_became_active":"2023-08-27T11:18:42.898971+0000","last_became_peered":"2023-08-27T11:18:42.898971+0000","last_unstale":"2023-08-27T11:18:44.928569+0000","last_undegraded":"2023-08-27T11:18:44.928569+0000","last_fullsized":"2023-08-27T11:18:44.928569+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:31:34.208538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.519110+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:18:45.519110+0000","last_peered":"2023-08-27T11:18:45.519110+0000","last_clean":"2023-08-27T11:18:45.519110+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:18:45.519110+0000","last_undegraded":"2023-08-27T11:18:45.519110+0000","last_fullsized":"2023-08-27T11:18:45.519110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:53.227753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518012+0000","last_change":"2023-08-27T11:18:43.906406+0000","last_active":"2023-08-27T11:18:45.518012+0000","last_peered":"2023-08-27T11:18:45.518012+0000","last_clean":"2023-08-27T11:18:45.518012+0000","last_became_active":"2023-08-27T11:18:43.906116+0000","last_became_peered":"2023-08-27T11:18:43.906116+0000","last_unstale":"2023-08-27T11:18:45.518012+0000","last_undegraded":"2023-08-27T11:18:45.518012+0000","last_fullsized":"2023-08-27T11:18:45.518012+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:30.422069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518004+0000","last_change":"2023-08-27T11:18:42.906745+0000","last_active":"2023-08-27T11:18:45.518004+0000","last_peered":"2023-08-27T11:18:45.518004+0000","last_clean":"2023-08-27T11:18:45.518004+0000","last_became_active":"2023-08-27T11:18:42.906486+0000","last_became_peered":"2023-08-27T11:18:42.906486+0000","last_unstale":"2023-08-27T11:18:45.518004+0000","last_undegraded":"2023-08-27T11:18:45.518004+0000","last_fullsized":"2023-08-27T11:18:45.518004+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:58:49.942800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":71,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518718+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:18:45.518718+0000","last_peered":"2023-08-27T11:18:45.518718+0000","last_clean":"2023-08-27T11:18:45.518718+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:18:45.518718+0000","last_undegraded":"2023-08-27T11:18:45.518718+0000","last_fullsized":"2023-08-27T11:18:45.518718+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:46:09.180631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517333+0000","last_change":"2023-08-27T11:18:43.898827+0000","last_active":"2023-08-27T11:18:45.517333+0000","last_peered":"2023-08-27T11:18:45.517333+0000","last_clean":"2023-08-27T11:18:45.517333+0000","last_became_active":"2023-08-27T11:18:43.898544+0000","last_became_peered":"2023-08-27T11:18:43.898544+0000","last_unstale":"2023-08-27T11:18:45.517333+0000","last_undegraded":"2023-08-27T11:18:45.517333+0000","last_fullsized":"2023-08-27T11:18:45.517333+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:57:17.078797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"50'2","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.517391+0000","last_change":"2023-08-27T11:18:42.902418+0000","last_active":"2023-08-27T11:18:45.517391+0000","last_peered":"2023-08-27T11:18:45.517391+0000","last_clean":"2023-08-27T11:18:45.517391+0000","last_became_active":"2023-08-27T11:18:42.902127+0000","last_became_peered":"2023-08-27T11:18:42.902127+0000","last_unstale":"2023-08-27T11:18:45.517391+0000","last_undegraded":"2023-08-27T11:18:45.517391+0000","last_fullsized":"2023-08-27T11:18:45.517391+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:52:47.837898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":221,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518467+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:18:45.518467+0000","last_peered":"2023-08-27T11:18:45.518467+0000","last_clean":"2023-08-27T11:18:45.518467+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:18:45.518467+0000","last_undegraded":"2023-08-27T11:18:45.518467+0000","last_fullsized":"2023-08-27T11:18:45.518467+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:21:46.351857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":73,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.502602+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:18:45.502602+0000","last_peered":"2023-08-27T11:18:45.502602+0000","last_clean":"2023-08-27T11:18:45.502602+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:18:45.502602+0000","last_undegraded":"2023-08-27T11:18:45.502602+0000","last_fullsized":"2023-08-27T11:18:45.502602+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:42:01.048075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.893234+0000","last_change":"2023-08-27T11:18:43.895094+0000","last_active":"2023-08-27T11:18:44.893234+0000","last_peered":"2023-08-27T11:18:44.893234+0000","last_clean":"2023-08-27T11:18:44.893234+0000","last_became_active":"2023-08-27T11:18:43.894340+0000","last_became_peered":"2023-08-27T11:18:43.894340+0000","last_unstale":"2023-08-27T11:18:44.893234+0000","last_undegraded":"2023-08-27T11:18:44.893234+0000","last_fullsized":"2023-08-27T11:18:44.893234+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:18:57.183236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.929064+0000","last_change":"2023-08-27T11:18:42.908525+0000","last_active":"2023-08-27T11:18:44.929064+0000","last_peered":"2023-08-27T11:18:44.929064+0000","last_clean":"2023-08-27T11:18:44.929064+0000","last_became_active":"2023-08-27T11:18:42.908276+0000","last_became_peered":"2023-08-27T11:18:42.908276+0000","last_unstale":"2023-08-27T11:18:44.929064+0000","last_undegraded":"2023-08-27T11:18:44.929064+0000","last_fullsized":"2023-08-27T11:18:44.929064+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:45:18.457607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.929077+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:18:44.929077+0000","last_peered":"2023-08-27T11:18:44.929077+0000","last_clean":"2023-08-27T11:18:44.929077+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:18:44.929077+0000","last_undegraded":"2023-08-27T11:18:44.929077+0000","last_fullsized":"2023-08-27T11:18:44.929077+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:17.215188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892573+0000","last_change":"2023-08-27T11:18:43.912729+0000","last_active":"2023-08-27T11:18:44.892573+0000","last_peered":"2023-08-27T11:18:44.892573+0000","last_clean":"2023-08-27T11:18:44.892573+0000","last_became_active":"2023-08-27T11:18:43.912162+0000","last_became_peered":"2023-08-27T11:18:43.912162+0000","last_unstale":"2023-08-27T11:18:44.892573+0000","last_undegraded":"2023-08-27T11:18:44.892573+0000","last_fullsized":"2023-08-27T11:18:44.892573+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:42.877926+0000","last_clean_scrub_stamp":"2023-08-27T11:18:42.877926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:53.858233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"50'1","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:45.518818+0000","last_change":"2023-08-27T11:18:42.906418+0000","last_active":"2023-08-27T11:18:45.518818+0000","last_peered":"2023-08-27T11:18:45.518818+0000","last_clean":"2023-08-27T11:18:45.518818+0000","last_became_active":"2023-08-27T11:18:42.906122+0000","last_became_peered":"2023-08-27T11:18:42.906122+0000","last_unstale":"2023-08-27T11:18:45.518818+0000","last_undegraded":"2023-08-27T11:18:45.518818+0000","last_fullsized":"2023-08-27T11:18:45.518818+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:18:41.871384+0000","last_clean_scrub_stamp":"2023-08-27T11:18:41.871384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:52:03.199395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-08-27T11:18:44.892798+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:18:44.892798+0000","last_peered":"2023-08-27T11:18:44.892798+0000","last_clean":"2023-08-27T11:18:44.892798+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:18:44.892798+0000","last_undegraded":"2023-08-27T11:18:44.892798+0000","last_fullsized":"2023-08-27T11:18:44.892798+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:25:08.248378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":864319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65200000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.70699999999999996}]},{"osd":2,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52200000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.628},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60399999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.435}]},{"osd":6,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.46800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":614400,"data_stored":273209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58499999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.47599999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.64800000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.51100000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.44500000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.61299999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1232896,"data_stored":865919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":11.971,"5min":11.971,"15min":11.971},"last":0.28100000000000003},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":23.045999999999999,"5min":23.045999999999999,"15min":23.045999999999999},"last":0.30199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.928000000000001,"5min":11.928000000000001,"15min":11.928000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.38700000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":10.99,"5min":10.99,"15min":10.99},"last":0.42999999999999999},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":22.893999999999998,"5min":22.893999999999998,"15min":22.893999999999998},"last":0.35799999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":10.957000000000001,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.75,"15min":1.75},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":23.016999999999999,"5min":23.016999999999999,"15min":23.016999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":10.925000000000001,"5min":10.925000000000001,"15min":10.925000000000001},"last":0.751},{"interface":"front","average":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":22.952000000000002,"5min":22.952000000000002,"15min":22.952000000000002},"last":0.623}]},{"osd":7,"last update":"Sun Aug 27 11:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":22.988,"5min":22.988,"15min":22.988},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.67200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":606208,"data_stored":271814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.316}]},{"osd":2,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.247}]},{"osd":3,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.371}]},{"osd":4,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.438},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.47299999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.51600000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40799999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.42499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":273175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":0.35099999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.379},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.33600000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.36599999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.71499999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.72099999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.58699999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.79800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":273175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.35899999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.45200000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.503}]},{"osd":5,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.62}]},{"osd":7,"last update":"Sun Aug 27 11:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.52800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":622592,"data_stored":274787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.58099999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.46200000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.623}]},{"osd":3,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48099999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.501},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.59599999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":614400,"data_stored":273197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.503},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.54500000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.37},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.55800000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.433},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.48299999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.46700000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.44600000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.53100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:18:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:18:48.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:18:48.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.868+0000 7faa039ef700 1 Processor -- start 2023-08-27T11:18:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.868+0000 7faa039ef700 1 -- start start 2023-08-27T11:18:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa039ef700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa9fc146c20 0x7fa9fc160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa039ef700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9fc0b3440 0x7fa9fc15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa039ef700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9fc15e6e0 0x7fa9fc06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa039ef700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9fc0b5070 con 0x7fa9fc0b3440 2023-08-27T11:18:48.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa039ef700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9fc147000 con 0x7fa9fc146c20 2023-08-27T11:18:48.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa039ef700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc1614a0 con 0x7fa9fc15e6e0 2023-08-27T11:18:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa00f8a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9fc0b3440 0x7fa9fc15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa00f8a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9fc0b3440 0x7fa9fc15e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58018/0 (socket says 172.21.15.112:58018) 2023-08-27T11:18:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa00f8a700 1 -- 172.21.15.112:0/1570666976 learned_addr learned my addr 172.21.15.112:0/1570666976 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa0178b700 1 --2- 172.21.15.112:0/1570666976 >> v2:172.21.15.138:3300/0 conn(0x7fa9fc146c20 0x7fa9fc160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa01f8c700 1 --2- 172.21.15.112:0/1570666976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9fc15e6e0 0x7fa9fc06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa00f8a700 1 -- 172.21.15.112:0/1570666976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9fc15e6e0 msgr2=0x7fa9fc06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa00f8a700 1 --2- 172.21.15.112:0/1570666976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9fc15e6e0 0x7fa9fc06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa00f8a700 1 -- 172.21.15.112:0/1570666976 >> v2:172.21.15.138:3300/0 conn(0x7fa9fc146c20 msgr2=0x7fa9fc160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa00f8a700 1 --2- 172.21.15.112:0/1570666976 >> v2:172.21.15.138:3300/0 conn(0x7fa9fc146c20 0x7fa9fc160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.869+0000 7faa00f8a700 1 -- 172.21.15.112:0/1570666976 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc161740 con 0x7fa9fc0b3440 2023-08-27T11:18:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.870+0000 7faa00f8a700 1 --2- 172.21.15.112:0/1570666976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9fc0b3440 0x7fa9fc15e1a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0b2d00 tx=0x7fa9f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf64c740051abdb9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.870+0000 7fa9f3fff700 1 -- 172.21.15.112:0/1570666976 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f800b870 con 0x7fa9fc0b3440 2023-08-27T11:18:48.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.870+0000 7fa9f3fff700 1 -- 172.21.15.112:0/1570666976 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f800b9d0 con 0x7fa9fc0b3440 2023-08-27T11:18:48.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.870+0000 7fa9f3fff700 1 -- 172.21.15.112:0/1570666976 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f800bd60 con 0x7fa9fc0b3440 2023-08-27T11:18:48.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.870+0000 7faa039ef700 1 -- 172.21.15.112:0/1570666976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9fc0b3440 msgr2=0x7fa9fc15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.870+0000 7faa039ef700 1 --2- 172.21.15.112:0/1570666976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9fc0b3440 0x7fa9fc15e1a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0b2d00 tx=0x7fa9f8009bd0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.870+0000 7faa039ef700 1 -- 172.21.15.112:0/1570666976 shutdown_connections 2023-08-27T11:18:48.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.870+0000 7faa039ef700 1 --2- 172.21.15.112:0/1570666976 >> v2:172.21.15.138:3300/0 conn(0x7fa9fc146c20 0x7fa9fc160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.870+0000 7faa039ef700 1 --2- 172.21.15.112:0/1570666976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9fc15e6e0 0x7fa9fc06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.870+0000 7faa039ef700 1 --2- 172.21.15.112:0/1570666976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9fc0b3440 0x7fa9fc15e1a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.870+0000 7faa039ef700 1 -- 172.21.15.112:0/1570666976 >> 172.21.15.112:0/1570666976 conn(0x7fa9fc0b1e90 msgr2=0x7fa9fc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.870+0000 7faa039ef700 1 -- 172.21.15.112:0/1570666976 shutdown_connections 2023-08-27T11:18:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.870+0000 7faa039ef700 1 -- 172.21.15.112:0/1570666976 wait complete. 2023-08-27T11:18:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.872+0000 7faa039ef700 1 Processor -- start 2023-08-27T11:18:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.872+0000 7faa039ef700 1 -- start start 2023-08-27T11:18:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.872+0000 7faa039ef700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9fc0b3440 0x7fa9fc13efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.872+0000 7faa039ef700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9fc146c20 0x7fa9fc145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.872+0000 7faa039ef700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9fc15e6e0 0x7fa9fc13e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.872+0000 7faa039ef700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc0ae1b0 con 0x7fa9fc0b3440 2023-08-27T11:18:48.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.872+0000 7faa039ef700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc0ae490 con 0x7fa9fc15e6e0 2023-08-27T11:18:48.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.872+0000 7faa039ef700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc0ae770 con 0x7fa9fc146c20 2023-08-27T11:18:48.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.872+0000 7faa0178b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9fc0b3440 0x7fa9fc13efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.872+0000 7faa0178b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9fc0b3440 0x7fa9fc13efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58024/0 (socket says 172.21.15.112:58024) 2023-08-27T11:18:48.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.872+0000 7faa0178b700 1 -- 172.21.15.112:0/1516065911 learned_addr learned my addr 172.21.15.112:0/1516065911 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:48.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7faa00f8a700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9fc146c20 0x7fa9fc145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7faa01f8c700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9fc15e6e0 0x7fa9fc13e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7faa0178b700 1 -- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9fc146c20 msgr2=0x7fa9fc145da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7faa0178b700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9fc146c20 0x7fa9fc145da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7faa0178b700 1 -- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9fc15e6e0 msgr2=0x7fa9fc13e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7faa0178b700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9fc15e6e0 0x7fa9fc13e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7faa0178b700 1 -- 172.21.15.112:0/1516065911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f8009550 con 0x7fa9fc0b3440 2023-08-27T11:18:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7faa00f8a700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9fc146c20 0x7fa9fc145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7faa01f8c700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9fc15e6e0 0x7fa9fc13e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7faa0178b700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9fc0b3440 0x7fa9fc13efb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00d800 tx=0x7fa9ec00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:48.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7fa9f1ffb700 1 -- 172.21.15.112:0/1516065911 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0125f0 con 0x7fa9fc0b3440 2023-08-27T11:18:48.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7fa9f1ffb700 1 -- 172.21.15.112:0/1516065911 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec012750 con 0x7fa9fc0b3440 2023-08-27T11:18:48.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7faa039ef700 1 -- 172.21.15.112:0/1516065911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9fc0aea50 con 0x7fa9fc0b3440 2023-08-27T11:18:48.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7fa9f1ffb700 1 -- 172.21.15.112:0/1516065911 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec012ae0 con 0x7fa9fc0b3440 2023-08-27T11:18:48.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.873+0000 7faa039ef700 1 -- 172.21.15.112:0/1516065911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9fc1462e0 con 0x7fa9fc0b3440 2023-08-27T11:18:48.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.874+0000 7faa039ef700 1 -- 172.21.15.112:0/1516065911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9fc067e90 con 0x7fa9fc0b3440 2023-08-27T11:18:48.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.874+0000 7fa9f1ffb700 1 -- 172.21.15.112:0/1516065911 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa9ec012c40 con 0x7fa9fc0b3440 2023-08-27T11:18:48.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.875+0000 7fa9f1ffb700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa9e006a0e0 0x7fa9e006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:48.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.875+0000 7fa9f1ffb700 1 -- 172.21.15.112:0/1516065911 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa9ec0887a0 con 0x7fa9fc0b3440 2023-08-27T11:18:48.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.875+0000 7faa00f8a700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa9e006a0e0 0x7fa9e006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:48.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.875+0000 7faa00f8a700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa9e006a0e0 0x7fa9e006c5a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0b2d00 tx=0x7fa9f8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:48.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:48.879+0000 7fa9f1ffb700 1 -- 172.21.15.112:0/1516065911 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa9ec053f20 con 0x7fa9fc0b3440 2023-08-27T11:18:49.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.146+0000 7faa039ef700 1 -- 172.21.15.112:0/1516065911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa9fc0ae2f0 con 0x7fa9fc0b3440 2023-08-27T11:18:49.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.150+0000 7fa9f1ffb700 1 -- 172.21.15.112:0/1516065911 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa9fc0ae2f0 con 0x7fa9fc0b3440 2023-08-27T11:18:49.150 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-08-27T11:18:49.150 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA== 2023-08-27T11:18:49.151 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2023-08-27T11:18:49.151 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2023-08-27T11:18:49.151 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:18:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.152+0000 7faa039ef700 1 -- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa9e006a0e0 msgr2=0x7fa9e006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:49.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.152+0000 7faa039ef700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa9e006a0e0 0x7fa9e006c5a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0b2d00 tx=0x7fa9f8008030 comp rx=0 tx=0).stop 2023-08-27T11:18:49.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.152+0000 7faa039ef700 1 -- 172.21.15.112:0/1516065911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9fc0b3440 msgr2=0x7fa9fc13efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:49.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.152+0000 7faa039ef700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9fc0b3440 0x7fa9fc13efb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00d800 tx=0x7fa9ec00dbc0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.152+0000 7faa039ef700 1 -- 172.21.15.112:0/1516065911 shutdown_connections 2023-08-27T11:18:49.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.152+0000 7faa039ef700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa9e006a0e0 0x7fa9e006c5a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.152+0000 7faa039ef700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9fc15e6e0 0x7fa9fc13e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.152+0000 7faa039ef700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9fc0b3440 0x7fa9fc13efb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.152+0000 7faa039ef700 1 --2- 172.21.15.112:0/1516065911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9fc146c20 0x7fa9fc145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.152+0000 7faa039ef700 1 -- 172.21.15.112:0/1516065911 >> 172.21.15.112:0/1516065911 conn(0x7fa9fc0b1e90 msgr2=0x7fa9fc0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:49.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.152+0000 7faa039ef700 1 -- 172.21.15.112:0/1516065911 shutdown_connections 2023-08-27T11:18:49.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.152+0000 7faa039ef700 1 -- 172.21.15.112:0/1516065911 wait complete. 2023-08-27T11:18:49.155 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2023-08-27T11:18:49.164 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:18:49.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.285+0000 7f558a72c700 1 Processor -- start 2023-08-27T11:18:49.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.285+0000 7f558a72c700 1 -- start start 2023-08-27T11:18:49.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.286+0000 7f558a72c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f55840b1400 0x7f55840cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:49.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.286+0000 7f558a72c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55840b17e0 0x7f558406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:49.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.286+0000 7f558a72c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5584161ce0 0x7f558406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:49.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.286+0000 7f558a72c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5584161850 con 0x7f5584161ce0 2023-08-27T11:18:49.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.286+0000 7f558a72c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558406e7d0 con 0x7f55840b1400 2023-08-27T11:18:49.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.286+0000 7f558a72c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55840cfc10 con 0x7f55840b17e0 2023-08-27T11:18:49.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.286+0000 7f5588cc9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5584161ce0 0x7f558406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:49.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.286+0000 7f5588cc9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5584161ce0 0x7f558406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58036/0 (socket says 172.21.15.112:58036) 2023-08-27T11:18:49.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.286+0000 7f5588cc9700 1 -- 172.21.15.112:0/371567689 learned_addr learned my addr 172.21.15.112:0/371567689 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:49.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.286+0000 7f5583fff700 1 --2- 172.21.15.112:0/371567689 >> v2:172.21.15.138:3300/0 conn(0x7f55840b1400 0x7f55840cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:49.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.286+0000 7f55837fe700 1 --2- 172.21.15.112:0/371567689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55840b17e0 0x7f558406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:49.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.286+0000 7f5588cc9700 1 -- 172.21.15.112:0/371567689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55840b17e0 msgr2=0x7f558406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:49.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.286+0000 7f5588cc9700 1 --2- 172.21.15.112:0/371567689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55840b17e0 0x7f558406e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f5588cc9700 1 -- 172.21.15.112:0/371567689 >> v2:172.21.15.138:3300/0 conn(0x7f55840b1400 msgr2=0x7f55840cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:49.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f5588cc9700 1 --2- 172.21.15.112:0/371567689 >> v2:172.21.15.138:3300/0 conn(0x7f55840b1400 0x7f55840cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f5588cc9700 1 -- 172.21.15.112:0/371567689 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5584161200 con 0x7f5584161ce0 2023-08-27T11:18:49.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f55837fe700 1 --2- 172.21.15.112:0/371567689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55840b17e0 0x7f558406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:49.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f5588cc9700 1 --2- 172.21.15.112:0/371567689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5584161ce0 0x7f558406f790 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5584070370 tx=0x7f557c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc45a3680ee6293 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:49.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f5582ffd700 1 -- 172.21.15.112:0/371567689 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c00e070 con 0x7f5584161ce0 2023-08-27T11:18:49.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f5582ffd700 1 -- 172.21.15.112:0/371567689 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c00b910 con 0x7f5584161ce0 2023-08-27T11:18:49.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f5582ffd700 1 -- 172.21.15.112:0/371567689 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c00bcc0 con 0x7f5584161ce0 2023-08-27T11:18:49.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f558a72c700 1 -- 172.21.15.112:0/371567689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5584161ce0 msgr2=0x7f558406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:49.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f558a72c700 1 --2- 172.21.15.112:0/371567689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5584161ce0 0x7f558406f790 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5584070370 tx=0x7f557c00b330 comp rx=0 tx=0).stop 2023-08-27T11:18:49.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f558a72c700 1 -- 172.21.15.112:0/371567689 shutdown_connections 2023-08-27T11:18:49.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f558a72c700 1 --2- 172.21.15.112:0/371567689 >> v2:172.21.15.138:3300/0 conn(0x7f55840b1400 0x7f55840cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f558a72c700 1 --2- 172.21.15.112:0/371567689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5584161ce0 0x7f558406f790 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f558a72c700 1 --2- 172.21.15.112:0/371567689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55840b17e0 0x7f558406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f558a72c700 1 -- 172.21.15.112:0/371567689 >> 172.21.15.112:0/371567689 conn(0x7f55840d0000 msgr2=0x7f55840d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:49.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f558a72c700 1 -- 172.21.15.112:0/371567689 shutdown_connections 2023-08-27T11:18:49.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.287+0000 7f558a72c700 1 -- 172.21.15.112:0/371567689 wait complete. 2023-08-27T11:18:49.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.289+0000 7f558a72c700 1 Processor -- start 2023-08-27T11:18:49.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.289+0000 7f558a72c700 1 -- start start 2023-08-27T11:18:49.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.289+0000 7f558a72c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55840b1400 0x7f55840c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:49.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f558a72c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55840b17e0 0x7f5584152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:49.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f558a72c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5584161ce0 0x7f55841604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:49.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f558a72c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55840c8780 con 0x7f55840b1400 2023-08-27T11:18:49.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f558a72c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5584160010 con 0x7f5584161ce0 2023-08-27T11:18:49.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f558a72c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55840c9f70 con 0x7f55840b17e0 2023-08-27T11:18:49.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f5583fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55840b1400 0x7f55840c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f5583fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55840b1400 0x7f55840c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58038/0 (socket says 172.21.15.112:58038) 2023-08-27T11:18:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f5583fff700 1 -- 172.21.15.112:0/2693330985 learned_addr learned my addr 172.21.15.112:0/2693330985 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f55837fe700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55840b17e0 0x7f5584152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f5588cc9700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5584161ce0 0x7f55841604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f5583fff700 1 -- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55840b17e0 msgr2=0x7f5584152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f5583fff700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55840b17e0 0x7f5584152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f5583fff700 1 -- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5584161ce0 msgr2=0x7f55841604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f5583fff700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5584161ce0 0x7f55841604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f5583fff700 1 -- 172.21.15.112:0/2693330985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c009b30 con 0x7f55840b1400 2023-08-27T11:18:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f5583fff700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55840b1400 0x7f55840c8240 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f557800e960 tx=0x7f557800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:49.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f5580ff9700 1 -- 172.21.15.112:0/2693330985 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578009690 con 0x7f55840b1400 2023-08-27T11:18:49.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f5580ff9700 1 -- 172.21.15.112:0/2693330985 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557800c850 con 0x7f55840b1400 2023-08-27T11:18:49.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f558a72c700 1 -- 172.21.15.112:0/2693330985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558415f400 con 0x7f55840b1400 2023-08-27T11:18:49.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f5580ff9700 1 -- 172.21.15.112:0/2693330985 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557800cbc0 con 0x7f55840b1400 2023-08-27T11:18:49.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.290+0000 7f558a72c700 1 -- 172.21.15.112:0/2693330985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558415f9c0 con 0x7f55840b1400 2023-08-27T11:18:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.291+0000 7f558a72c700 1 -- 172.21.15.112:0/2693330985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55841602f0 con 0x7f55840b1400 2023-08-27T11:18:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.292+0000 7f5580ff9700 1 -- 172.21.15.112:0/2693330985 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f557800cd20 con 0x7f55840b1400 2023-08-27T11:18:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.292+0000 7f5580ff9700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f557006a0e0 0x7f557006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.292+0000 7f5580ff9700 1 -- 172.21.15.112:0/2693330985 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5578088630 con 0x7f55840b1400 2023-08-27T11:18:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.292+0000 7f55837fe700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f557006a0e0 0x7f557006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.293+0000 7f55837fe700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f557006a0e0 0x7f557006c5a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f55840cb450 tx=0x7f5574008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.296+0000 7f5580ff9700 1 -- 172.21.15.112:0/2693330985 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5578057720 con 0x7f55840b1400 2023-08-27T11:18:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.525+0000 7f558a72c700 1 -- 172.21.15.112:0/2693330985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5584067a20 con 0x7f55840b1400 2023-08-27T11:18:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.526+0000 7f5580ff9700 1 -- 172.21.15.112:0/2693330985 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4888 (secure 0 0 0) 0x7f55780572b0 con 0x7f55840b1400 2023-08-27T11:18:49.527 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:49.527 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1692518554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1692518554},{"type":"v1","addr":"172.21.15.138:6837","nonce":1692518554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3294671617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3294671617},{"type":"v1","addr":"172.21.15.138:6836","nonce":3294671617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3012780135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3012780135},{"type":"v1","addr":"172.21.15.112:6834","nonce":3012780135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:18:43.281470+0000","modified":"2023-08-27T11:18:47.234075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5661":{"gid":5661,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3423469954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3423469954},{"type":"v1","addr":"172.21.15.112:6835","nonce":3423469954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-27T11:18:49.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.528+0000 7f558a72c700 1 -- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f557006a0e0 msgr2=0x7f557006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:49.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.528+0000 7f558a72c700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f557006a0e0 0x7f557006c5a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f55840cb450 tx=0x7f5574008040 comp rx=0 tx=0).stop 2023-08-27T11:18:49.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.528+0000 7f558a72c700 1 -- 172.21.15.112:0/2693330985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55840b1400 msgr2=0x7f55840c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:49.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.528+0000 7f558a72c700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55840b1400 0x7f55840c8240 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f557800e960 tx=0x7f557800ed20 comp rx=0 tx=0).stop 2023-08-27T11:18:49.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.528+0000 7f558a72c700 1 -- 172.21.15.112:0/2693330985 shutdown_connections 2023-08-27T11:18:49.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.528+0000 7f558a72c700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f557006a0e0 0x7f557006c5a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.528+0000 7f558a72c700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5584161ce0 0x7f55841604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.528+0000 7f558a72c700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55840b1400 0x7f55840c8240 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.528+0000 7f558a72c700 1 --2- 172.21.15.112:0/2693330985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55840b17e0 0x7f5584152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:49.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.528+0000 7f558a72c700 1 -- 172.21.15.112:0/2693330985 >> 172.21.15.112:0/2693330985 conn(0x7f55840d0000 msgr2=0x7f55840d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:49.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.528+0000 7f558a72c700 1 -- 172.21.15.112:0/2693330985 shutdown_connections 2023-08-27T11:18:49.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:49.528+0000 7f558a72c700 1 -- 172.21.15.112:0/2693330985 wait complete. 2023-08-27T11:18:49.532 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2023-08-27T11:18:49.945 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:18:43.281470+0000', 'modified': '2023-08-27T11:18:47.234075+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5661}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5661': {'gid': 5661, 'name': 'c', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/3423469954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 3423469954}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3423469954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:18:49.945 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:18:49.946 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-27T11:18:50.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.068+0000 7f42ad98c700 1 Processor -- start 2023-08-27T11:18:50.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.068+0000 7f42ad98c700 1 -- start start 2023-08-27T11:18:50.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.069+0000 7f42ad98c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a8146990 0x7f42a80b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.069+0000 7f42ad98c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f42a806d510 0x7f42a806d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.069+0000 7f42ad98c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a806f930 con 0x7f42a8160a00 2023-08-27T11:18:50.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.069+0000 7f42ad98c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a806dec0 con 0x7f42a806d510 2023-08-27T11:18:50.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.069+0000 7f42ad98c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a806fc10 con 0x7f42a8146990 2023-08-27T11:18:50.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.069+0000 7f42a67fc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a8146990 0x7f42a80b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.069+0000 7f42a77fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f42a806d510 0x7f42a806d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.069+0000 7f42a67fc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a8146990 0x7f42a80b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:41468/0 (socket says 172.21.15.112:41468) 2023-08-27T11:18:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.069+0000 7f42a6ffd700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f42a8160a00 0x7f42a815dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:59952/0 (socket says 172.21.15.112:59952) 2023-08-27T11:18:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.069+0000 7f42a67fc700 1 -- 172.21.15.112:0/858470210 learned_addr learned my addr 172.21.15.112:0/858470210 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.070+0000 7f42a77fe700 1 -- 172.21.15.112:0/858470210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a8146990 msgr2=0x7f42a80b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.070+0000 7f42a77fe700 1 --2- 172.21.15.112:0/858470210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a8146990 0x7f42a80b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.070+0000 7f42a77fe700 1 -- 172.21.15.112:0/858470210 >> v1:172.21.15.138:6789/0 conn(0x7f42a8160a00 legacy=0x7f42a815dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.070+0000 7f42a5ffb700 1 -- 172.21.15.112:0/858470210 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948612487 0 0) 0x7f42a806f930 con 0x7f42a8160a00 2023-08-27T11:18:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.070+0000 7f42a77fe700 1 -- 172.21.15.112:0/858470210 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a806fef0 con 0x7f42a806d510 2023-08-27T11:18:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.070+0000 7f42a77fe700 1 --2- 172.21.15.112:0/858470210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a806d510 0x7f42a806d980 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009d90 tx=0x7f42a000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c85ca7c454a528aa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.070+0000 7f42a5ffb700 1 -- 172.21.15.112:0/858470210 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a000e070 con 0x7f42a806d510 2023-08-27T11:18:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.070+0000 7f42a5ffb700 1 -- 172.21.15.112:0/858470210 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a000bb20 con 0x7f42a806d510 2023-08-27T11:18:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.070+0000 7f42a5ffb700 1 -- 172.21.15.112:0/858470210 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0012430 con 0x7f42a806d510 2023-08-27T11:18:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.070+0000 7f42ad98c700 1 -- 172.21.15.112:0/858470210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a806d510 msgr2=0x7f42a806d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.070+0000 7f42ad98c700 1 --2- 172.21.15.112:0/858470210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a806d510 0x7f42a806d980 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009d90 tx=0x7f42a000b330 comp rx=0 tx=0).stop 2023-08-27T11:18:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.071+0000 7f42ad98c700 1 -- 172.21.15.112:0/858470210 shutdown_connections 2023-08-27T11:18:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.071+0000 7f42ad98c700 1 --2- 172.21.15.112:0/858470210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a806d510 0x7f42a806d980 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.071+0000 7f42ad98c700 1 --2- 172.21.15.112:0/858470210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a8146990 0x7f42a80b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.071+0000 7f42ad98c700 1 -- 172.21.15.112:0/858470210 >> 172.21.15.112:0/858470210 conn(0x7f42a80cfe50 msgr2=0x7f42a80d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.071+0000 7f42ad98c700 1 -- 172.21.15.112:0/858470210 shutdown_connections 2023-08-27T11:18:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.071+0000 7f42ad98c700 1 -- 172.21.15.112:0/858470210 wait complete. 2023-08-27T11:18:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.072+0000 7f42ad98c700 1 Processor -- start 2023-08-27T11:18:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.072+0000 7f42ad98c700 1 -- start start 2023-08-27T11:18:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.073+0000 7f42ad98c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a806d510 0x7f42a815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.073+0000 7f42ad98c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42a8146990 0x7f42a80b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.073+0000 7f42ad98c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a8160a00 0x7f42a80d2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.073+0000 7f42ad98c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a8136bd0 con 0x7f42a8146990 2023-08-27T11:18:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.073+0000 7f42ad98c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a8136eb0 con 0x7f42a8160a00 2023-08-27T11:18:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.073+0000 7f42ad98c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a8137190 con 0x7f42a806d510 2023-08-27T11:18:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.073+0000 7f42a67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42a8146990 0x7f42a80b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.073+0000 7f42a67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42a8146990 0x7f42a80b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58054/0 (socket says 172.21.15.112:58054) 2023-08-27T11:18:50.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.073+0000 7f42a67fc700 1 -- 172.21.15.112:0/714361686 learned_addr learned my addr 172.21.15.112:0/714361686 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:50.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.073+0000 7f42a6ffd700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a806d510 0x7f42a815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.073+0000 7f42a77fe700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a8160a00 0x7f42a80d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.074+0000 7f42a67fc700 1 -- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a806d510 msgr2=0x7f42a815cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.074+0000 7f42a67fc700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a806d510 0x7f42a815cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.074+0000 7f42a67fc700 1 -- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a8160a00 msgr2=0x7f42a80d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.074+0000 7f42a67fc700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a8160a00 0x7f42a80d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.074+0000 7f42a67fc700 1 -- 172.21.15.112:0/714361686 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a0009a70 con 0x7f42a8146990 2023-08-27T11:18:50.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.074+0000 7f42a67fc700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42a8146990 0x7f42a80b25e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f429c00e910 tx=0x7f429c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:50.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.074+0000 7f428ffff700 1 -- 172.21.15.112:0/714361686 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c0096a0 con 0x7f42a8146990 2023-08-27T11:18:50.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.074+0000 7f428ffff700 1 -- 172.21.15.112:0/714361686 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c013680 con 0x7f42a8146990 2023-08-27T11:18:50.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.074+0000 7f42ad98c700 1 -- 172.21.15.112:0/714361686 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a8137470 con 0x7f42a8146990 2023-08-27T11:18:50.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.074+0000 7f428ffff700 1 -- 172.21.15.112:0/714361686 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c013a30 con 0x7f42a8146990 2023-08-27T11:18:50.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.074+0000 7f42ad98c700 1 -- 172.21.15.112:0/714361686 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a815d090 con 0x7f42a8146990 2023-08-27T11:18:50.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.075+0000 7f428ffff700 1 -- 172.21.15.112:0/714361686 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f429c013b90 con 0x7f42a8146990 2023-08-27T11:18:50.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.076+0000 7f428ffff700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f429406a0e0 0x7f429406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.076+0000 7f428ffff700 1 -- 172.21.15.112:0/714361686 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f429c088520 con 0x7f42a8146990 2023-08-27T11:18:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.076+0000 7f42a6ffd700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f429406a0e0 0x7f429406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.076+0000 7f42a6ffd700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f429406a0e0 0x7f429406c5a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4298007fd0 tx=0x7f4298007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.078+0000 7f42b9b4eb80 1 Processor -- start 2023-08-27T11:18:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.078+0000 7f42b9b4eb80 1 -- start start 2023-08-27T11:18:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42b9b4eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 0x55ffb9f31320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42b9b4eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f42960 0x55ffb9f2db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42b9b4eb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 0x55ffb9f2e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42b9b4eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffb9e5f730 con 0x55ffb9f30f40 2023-08-27T11:18:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42b9b4eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffb9e5d050 con 0x55ffb9f42960 2023-08-27T11:18:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42b9b4eb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffb9e58940 con 0x55ffb9f043d0 2023-08-27T11:18:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42a6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 0x55ffb9f31320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42a6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 0x55ffb9f31320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58060/0 (socket says 172.21.15.112:58060) 2023-08-27T11:18:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42a6ffd700 1 -- 172.21.15.112:0/1451742329 learned_addr learned my addr 172.21.15.112:0/1451742329 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42a67fc700 1 --2- 172.21.15.112:0/1451742329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f42960 0x55ffb9f2db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42a77fe700 1 --2- 172.21.15.112:0/1451742329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 0x55ffb9f2e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42a6ffd700 1 -- 172.21.15.112:0/1451742329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 msgr2=0x55ffb9f2e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42a6ffd700 1 --2- 172.21.15.112:0/1451742329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 0x55ffb9f2e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42a6ffd700 1 -- 172.21.15.112:0/1451742329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f42960 msgr2=0x55ffb9f2db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42a6ffd700 1 --2- 172.21.15.112:0/1451742329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f42960 0x55ffb9f2db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42a6ffd700 1 -- 172.21.15.112:0/1451742329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffb9cd85e0 con 0x55ffb9f30f40 2023-08-27T11:18:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42a77fe700 1 --2- 172.21.15.112:0/1451742329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 0x55ffb9f2e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f42a6ffd700 1 --2- 172.21.15.112:0/1451742329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 0x55ffb9f31320 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f42980017a0 tx=0x7f4298013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38676117b4921618 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f428d7fa700 1 -- 172.21.15.112:0/1451742329 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42980133f0 con 0x55ffb9f30f40 2023-08-27T11:18:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f428d7fa700 1 -- 172.21.15.112:0/1451742329 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4298013550 con 0x55ffb9f30f40 2023-08-27T11:18:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.079+0000 7f428d7fa700 1 -- 172.21.15.112:0/1451742329 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42980138c0 con 0x55ffb9f30f40 2023-08-27T11:18:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.080+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/1451742329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 msgr2=0x55ffb9f31320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.080+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/1451742329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 0x55ffb9f31320 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f42980017a0 tx=0x7f4298013000 comp rx=0 tx=0).stop 2023-08-27T11:18:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.080+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/1451742329 shutdown_connections 2023-08-27T11:18:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.080+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/1451742329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 0x55ffb9f2e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.080+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/1451742329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 0x55ffb9f31320 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.080+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/1451742329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f42960 0x55ffb9f2db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.080+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/1451742329 >> 172.21.15.112:0/1451742329 conn(0x55ffb9e64c30 msgr2=0x55ffb9f4bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:50.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.080+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/1451742329 shutdown_connections 2023-08-27T11:18:50.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.080+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/1451742329 wait complete. 2023-08-27T11:18:50.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.081+0000 7f42b9b4eb80 1 -- start start 2023-08-27T11:18:50.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.081+0000 7f42b9b4eb80 1 Processor -- start 2023-08-27T11:18:50.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.081+0000 7f42b9b4eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f043d0 0x55ffb9f34980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42b9b4eb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f30f40 0x55ffb9f31fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42b9b4eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 0x55ffb9f27590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42b9b4eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffb9df6b60 con 0x55ffb9f42960 2023-08-27T11:18:50.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42b9b4eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffb9df6860 con 0x55ffb9f043d0 2023-08-27T11:18:50.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42b9b4eb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffb9cda8d0 con 0x55ffb9f30f40 2023-08-27T11:18:50.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42b9b4eb80 10 client.0 _ll_register_callbacks cb 0x55ffb9df4630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:18:50.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42a77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 0x55ffb9f27590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42a77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 0x55ffb9f27590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58070/0 (socket says 172.21.15.112:58070) 2023-08-27T11:18:50.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42a77fe700 1 -- 172.21.15.112:0/1486401658 learned_addr learned my addr 172.21.15.112:0/1486401658 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:50.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42a6ffd700 1 --2- 172.21.15.112:0/1486401658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f043d0 0x55ffb9f34980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42a67fc700 1 --2- 172.21.15.112:0/1486401658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f30f40 0x55ffb9f31fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42a77fe700 1 -- 172.21.15.112:0/1486401658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f30f40 msgr2=0x55ffb9f31fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:50.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42a77fe700 1 --2- 172.21.15.112:0/1486401658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f30f40 0x55ffb9f31fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42a77fe700 1 -- 172.21.15.112:0/1486401658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f043d0 msgr2=0x55ffb9f34980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:50.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42a77fe700 1 --2- 172.21.15.112:0/1486401658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f043d0 0x55ffb9f34980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42a77fe700 1 -- 172.21.15.112:0/1486401658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4298001450 con 0x55ffb9f42960 2023-08-27T11:18:50.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42a77fe700 1 --2- 172.21.15.112:0/1486401658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 0x55ffb9f27590 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f42a80d09c0 tx=0x7f42a0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f3bb3227420cf7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:50.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42867fc700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:18:50.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42867fc700 1 -- 172.21.15.112:0/1486401658 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0021070 con 0x55ffb9f42960 2023-08-27T11:18:50.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42867fc700 1 -- 172.21.15.112:0/1486401658 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f42a0012ba0 con 0x55ffb9f42960 2023-08-27T11:18:50.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/1486401658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffb9ef94f0 con 0x55ffb9f42960 2023-08-27T11:18:50.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42867fc700 1 -- 172.21.15.112:0/1486401658 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42a0007000 con 0x55ffb9f42960 2023-08-27T11:18:50.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42b9b4eb80 10 client.5808 fetch_fsmap learned FSMap version 30 2023-08-27T11:18:50.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.082+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/1486401658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55ffb9e70750 con 0x55ffb9f42960 2023-08-27T11:18:50.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.083+0000 7f42867fc700 1 -- 172.21.15.112:0/1486401658 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ffb9ef94f0 con 0x55ffb9f42960 2023-08-27T11:18:50.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.083+0000 7f42867fc700 1 -- 172.21.15.112:0/1486401658 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f42a0004040 con 0x55ffb9f42960 2023-08-27T11:18:50.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.083+0000 7f42b9b4eb80 10 client.5808 fetch_fsmap finished waiting for FSMap version 30 2023-08-27T11:18:50.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.083+0000 7f42b9b4eb80 10 client.5808 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:18:50.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.083+0000 7f42b9b4eb80 20 client.5808 populate_metadata read hostname 'smithi112' 2023-08-27T11:18:50.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.083+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/1486401658 >> [v2:172.21.15.112:6833/3423469954,v1:172.21.15.112:6835/3423469954] conn(0x55ffb9f32bf0 0x55ffb9f28300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.083+0000 7f42b9b4eb80 4 client.5808 mds_command: new command op to 5661 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:18:50.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.083+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/1486401658 --> [v2:172.21.15.112:6833/3423469954,v1:172.21.15.112:6835/3423469954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffb9e51920 con 0x55ffb9f32bf0 2023-08-27T11:18:50.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.083+0000 7f42a6ffd700 1 --2- 172.21.15.112:0/1486401658 >> [v2:172.21.15.112:6833/3423469954,v1:172.21.15.112:6835/3423469954] conn(0x55ffb9f32bf0 0x55ffb9f28300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.083+0000 7f42867fc700 10 client.5808 ms_handle_connect on v2:172.21.15.112:6833/3423469954 2023-08-27T11:18:50.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.083+0000 7f42a6ffd700 1 --2- 172.21.15.112:0/1486401658 >> [v2:172.21.15.112:6833/3423469954,v1:172.21.15.112:6835/3423469954] conn(0x55ffb9f32bf0 0x55ffb9f28300 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a862c80138e90d33 server_cookie=2d81da219a28e429 in_seq=0 out_seq=0 2023-08-27T11:18:50.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42867fc700 1 -- 172.21.15.112:0/1486401658 <== mds.0 v2:172.21.15.112:6833/3423469954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ffb9e51920 con 0x55ffb9f32bf0 2023-08-27T11:18:50.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42867fc700 10 client.5808 handle_command_reply: tid=0 2023-08-27T11:18:50.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42b9b4eb80 1 client.5808 shutdown 2023-08-27T11:18:50.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/1486401658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 msgr2=0x55ffb9f27590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/1486401658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 0x55ffb9f27590 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f42a80d09c0 tx=0x7f42a0004b40 comp rx=0 tx=0).stop 2023-08-27T11:18:50.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/1486401658 shutdown_connections 2023-08-27T11:18:50.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/1486401658 >> [v2:172.21.15.112:6833/3423469954,v1:172.21.15.112:6835/3423469954] conn(0x55ffb9f32bf0 0x55ffb9f28300 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/1486401658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 0x55ffb9f27590 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42867fc700 0 client.5808 ms_handle_reset on v2:172.21.15.112:6833/3423469954 2023-08-27T11:18:50.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/1486401658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f043d0 0x55ffb9f34980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/1486401658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f30f40 0x55ffb9f31fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/1486401658 >> 172.21.15.112:0/1486401658 conn(0x55ffb9e64c30 msgr2=0x55ffb9f2a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:50.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/1486401658 shutdown_connections 2023-08-27T11:18:50.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/1486401658 wait complete. 2023-08-27T11:18:50.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.084+0000 7f42b9b4eb80 20 client.5808 trim_cache size 0 max 16384 2023-08-27T11:18:50.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.101+0000 7f42b9b4eb80 1 Processor -- start 2023-08-27T11:18:50.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.102+0000 7f42b9b4eb80 1 -- start start 2023-08-27T11:18:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.102+0000 7f42b9b4eb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 0x55ffb9f28ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.102+0000 7f42b9b4eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f30f40 0x55ffb9f01070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.102+0000 7f42b9b4eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 0x55ffb9f2bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.102+0000 7f42b9b4eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffb9f26d00 con 0x55ffb9f42960 2023-08-27T11:18:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.102+0000 7f42b9b4eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffb9f49430 con 0x55ffb9f30f40 2023-08-27T11:18:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.102+0000 7f42b9b4eb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffb9f03a40 con 0x55ffb9f043d0 2023-08-27T11:18:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.102+0000 7f42a77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 0x55ffb9f2bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.102+0000 7f42a77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 0x55ffb9f2bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58078/0 (socket says 172.21.15.112:58078) 2023-08-27T11:18:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.102+0000 7f42a77fe700 1 -- 172.21.15.112:0/759547298 learned_addr learned my addr 172.21.15.112:0/759547298 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.103+0000 7f42a67fc700 1 --2- 172.21.15.112:0/759547298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f30f40 0x55ffb9f01070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.103+0000 7f42a77fe700 1 -- 172.21.15.112:0/759547298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 msgr2=0x55ffb9f28ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:50.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.103+0000 7f42a6ffd700 1 --2- 172.21.15.112:0/759547298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 0x55ffb9f28ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.103+0000 7f42a77fe700 1 --2- 172.21.15.112:0/759547298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 0x55ffb9f28ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.103+0000 7f42a77fe700 1 -- 172.21.15.112:0/759547298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f30f40 msgr2=0x55ffb9f01070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:50.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.103+0000 7f42a77fe700 1 --2- 172.21.15.112:0/759547298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f30f40 0x55ffb9f01070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.103+0000 7f42a77fe700 1 -- 172.21.15.112:0/759547298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffb9c84d90 con 0x55ffb9f42960 2023-08-27T11:18:50.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.103+0000 7f42a77fe700 1 --2- 172.21.15.112:0/759547298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 0x55ffb9f2bad0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f42a0004bc0 tx=0x7f42a0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e40f5f83ca70d15 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:50.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.103+0000 7f428dffb700 1 -- 172.21.15.112:0/759547298 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0009840 con 0x55ffb9f42960 2023-08-27T11:18:50.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.103+0000 7f428dffb700 1 -- 172.21.15.112:0/759547298 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a00045d0 con 0x55ffb9f42960 2023-08-27T11:18:50.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.103+0000 7f428dffb700 1 -- 172.21.15.112:0/759547298 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0004040 con 0x55ffb9f42960 2023-08-27T11:18:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.104+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/759547298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 msgr2=0x55ffb9f2bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.104+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/759547298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 0x55ffb9f2bad0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f42a0004bc0 tx=0x7f42a0021000 comp rx=0 tx=0).stop 2023-08-27T11:18:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.104+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/759547298 shutdown_connections 2023-08-27T11:18:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.104+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/759547298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f42960 0x55ffb9f2bad0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.104+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/759547298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 0x55ffb9f28ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.104+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/759547298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f30f40 0x55ffb9f01070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.104+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/759547298 >> 172.21.15.112:0/759547298 conn(0x55ffb9e64c30 msgr2=0x55ffb9f338e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:50.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.104+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/759547298 shutdown_connections 2023-08-27T11:18:50.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.104+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/759547298 wait complete. 2023-08-27T11:18:50.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.104+0000 7f42b9b4eb80 1 -- start start 2023-08-27T11:18:50.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.104+0000 7f42b9b4eb80 1 Processor -- start 2023-08-27T11:18:50.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42b9b4eb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 0x55ffb9f98370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42b9b4eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 0x55ffb9f5d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42b9b4eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f42960 0x55ffb9f337b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42b9b4eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffb9fb3810 con 0x55ffb9f30f40 2023-08-27T11:18:50.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42b9b4eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffb9f5dee0 con 0x55ffb9f42960 2023-08-27T11:18:50.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42b9b4eb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffb9fb2650 con 0x55ffb9f043d0 2023-08-27T11:18:50.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42b9b4eb80 10 client.0 _ll_register_callbacks cb 0x55ffb9df4630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:18:50.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42a67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 0x55ffb9f5d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42a67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 0x55ffb9f5d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58082/0 (socket says 172.21.15.112:58082) 2023-08-27T11:18:50.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42a67fc700 1 -- 172.21.15.112:0/679563196 learned_addr learned my addr 172.21.15.112:0/679563196 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:50.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42a67fc700 1 -- 172.21.15.112:0/679563196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 msgr2=0x55ffb9f98370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:50.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42a67fc700 1 --2- 172.21.15.112:0/679563196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 0x55ffb9f98370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42a67fc700 1 -- 172.21.15.112:0/679563196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f42960 msgr2=0x55ffb9f337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:50.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42a67fc700 1 --2- 172.21.15.112:0/679563196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f42960 0x55ffb9f337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42a67fc700 1 -- 172.21.15.112:0/679563196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffb9e70750 con 0x55ffb9f30f40 2023-08-27T11:18:50.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42a67fc700 1 --2- 172.21.15.112:0/679563196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 0x55ffb9f5d400 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f429c0577d0 tx=0x7f429c05b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371aa3a20f6feea6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:50.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.105+0000 7f42867fc700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:18:50.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42867fc700 1 -- 172.21.15.112:0/679563196 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c084070 con 0x55ffb9f30f40 2023-08-27T11:18:50.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42867fc700 1 -- 172.21.15.112:0/679563196 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f429c013e40 con 0x55ffb9f30f40 2023-08-27T11:18:50.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/679563196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4298013550 con 0x55ffb9f30f40 2023-08-27T11:18:50.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42867fc700 1 -- 172.21.15.112:0/679563196 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f429c05f150 con 0x55ffb9f30f40 2023-08-27T11:18:50.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42b9b4eb80 10 client.5814 fetch_fsmap learned FSMap version 30 2023-08-27T11:18:50.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/679563196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55ffb9c6dc10 con 0x55ffb9f30f40 2023-08-27T11:18:50.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42867fc700 1 -- 172.21.15.112:0/679563196 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4298013550 con 0x55ffb9f30f40 2023-08-27T11:18:50.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42867fc700 1 -- 172.21.15.112:0/679563196 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f429c0880b0 con 0x55ffb9f30f40 2023-08-27T11:18:50.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42b9b4eb80 10 client.5814 fetch_fsmap finished waiting for FSMap version 30 2023-08-27T11:18:50.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42b9b4eb80 10 client.5814 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:18:50.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42b9b4eb80 20 client.5814 populate_metadata read hostname 'smithi112' 2023-08-27T11:18:50.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/679563196 >> [v2:172.21.15.112:6833/3423469954,v1:172.21.15.112:6835/3423469954] conn(0x55ffb9f04d10 0x55ffb9f050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42b9b4eb80 4 client.5814 mds_command: new command op to 5661 tid=0[{"prefix": "status"}] 2023-08-27T11:18:50.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/679563196 --> [v2:172.21.15.112:6833/3423469954,v1:172.21.15.112:6835/3423469954] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ffb9e12000 con 0x55ffb9f04d10 2023-08-27T11:18:50.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42a6ffd700 1 --2- 172.21.15.112:0/679563196 >> [v2:172.21.15.112:6833/3423469954,v1:172.21.15.112:6835/3423469954] conn(0x55ffb9f04d10 0x55ffb9f050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42867fc700 10 client.5814 ms_handle_connect on v2:172.21.15.112:6833/3423469954 2023-08-27T11:18:50.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42a6ffd700 1 --2- 172.21.15.112:0/679563196 >> [v2:172.21.15.112:6833/3423469954,v1:172.21.15.112:6835/3423469954] conn(0x55ffb9f04d10 0x55ffb9f050f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34ed9edc64fb7ad4 server_cookie=12ad45f190e1211e in_seq=0 out_seq=0 2023-08-27T11:18:50.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42867fc700 1 -- 172.21.15.112:0/679563196 <== mds.0 v2:172.21.15.112:6833/3423469954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55ffb9e12000 con 0x55ffb9f04d10 2023-08-27T11:18:50.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42867fc700 10 client.5814 handle_command_reply: tid=0 2023-08-27T11:18:50.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.106+0000 7f42b9b4eb80 1 client.5814 shutdown 2023-08-27T11:18:50.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/679563196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 msgr2=0x55ffb9f5d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/679563196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 0x55ffb9f5d400 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f429c0577d0 tx=0x7f429c05b6b0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/679563196 shutdown_connections 2023-08-27T11:18:50.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/679563196 >> [v2:172.21.15.112:6833/3423469954,v1:172.21.15.112:6835/3423469954] conn(0x55ffb9f04d10 0x55ffb9f050f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/679563196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffb9f42960 0x55ffb9f337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42867fc700 0 client.5814 ms_handle_reset on v2:172.21.15.112:6833/3423469954 2023-08-27T11:18:50.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/679563196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffb9f043d0 0x55ffb9f98370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42b9b4eb80 1 --2- 172.21.15.112:0/679563196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ffb9f30f40 0x55ffb9f5d400 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/679563196 >> 172.21.15.112:0/679563196 conn(0x55ffb9e64c30 msgr2=0x55ffb9f2bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:50.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/679563196 shutdown_connections 2023-08-27T11:18:50.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42b9b4eb80 1 -- 172.21.15.112:0/679563196 wait complete. 2023-08-27T11:18:50.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42b9b4eb80 20 client.5814 trim_cache size 0 max 16384 2023-08-27T11:18:50.129 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:18:50.130 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e099a24f-c1cd-4f11-ac4f-ecacc10e5e98", 2023-08-27T11:18:50.130 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2023-08-27T11:18:50.130 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 5661, 2023-08-27T11:18:50.130 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2023-08-27T11:18:50.130 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2023-08-27T11:18:50.131 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2023-08-27T11:18:50.131 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 6.2206455209999998, 2023-08-27T11:18:50.131 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 30, 2023-08-27T11:18:50.131 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 51, 2023-08-27T11:18:50.131 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 51, 2023-08-27T11:18:50.132 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 12.150251653 2023-08-27T11:18:50.132 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:18:50.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42ad98c700 1 -- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f429406a0e0 msgr2=0x7f429406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42ad98c700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f429406a0e0 0x7f429406c5a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4298007fd0 tx=0x7f4298007de0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42ad98c700 1 -- 172.21.15.112:0/714361686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42a8146990 msgr2=0x7f42a80b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42ad98c700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42a8146990 0x7f42a80b25e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f429c00e910 tx=0x7f429c00ec20 comp rx=0 tx=0).stop 2023-08-27T11:18:50.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42ad98c700 1 -- 172.21.15.112:0/714361686 shutdown_connections 2023-08-27T11:18:50.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42ad98c700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f429406a0e0 0x7f429406c5a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42ad98c700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a8160a00 0x7f42a80d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42ad98c700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a806d510 0x7f42a815cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42ad98c700 1 --2- 172.21.15.112:0/714361686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42a8146990 0x7f42a80b25e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42ad98c700 1 -- 172.21.15.112:0/714361686 >> 172.21.15.112:0/714361686 conn(0x7f42a80cfe50 msgr2=0x7f42a80d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:50.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42ad98c700 1 -- 172.21.15.112:0/714361686 shutdown_connections 2023-08-27T11:18:50.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.107+0000 7f42ad98c700 1 -- 172.21.15.112:0/714361686 wait complete. 2023-08-27T11:18:50.136 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:18:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.256+0000 7fe487f5c700 1 Processor -- start 2023-08-27T11:18:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.256+0000 7fe487f5c700 1 -- start start 2023-08-27T11:18:50.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.256+0000 7fe487f5c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe480158e20 0x7fe4800b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe487f5c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe48006fea0 0x7fe4800af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe487f5c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe48015c190 con 0x7fe48006e3a0 2023-08-27T11:18:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe487f5c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4800b1730 con 0x7fe480158e20 2023-08-27T11:18:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe487f5c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe480159200 con 0x7fe48006fea0 2023-08-27T11:18:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe485cf8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe480158e20 0x7fe4800b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe4854f7700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe48006e3a0 0x7fe48006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52472/0 (socket says 172.21.15.112:52472) 2023-08-27T11:18:50.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe4854f7700 1 -- 172.21.15.112:0/306055305 learned_addr learned my addr 172.21.15.112:0/306055305 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:50.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe4864f9700 1 --2- 172.21.15.112:0/306055305 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe48006fea0 0x7fe4800af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe485cf8700 1 -- 172.21.15.112:0/306055305 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe48006fea0 msgr2=0x7fe4800af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:50.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe485cf8700 1 --2- 172.21.15.112:0/306055305 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe48006fea0 0x7fe4800af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe485cf8700 1 -- 172.21.15.112:0/306055305 >> v1:172.21.15.112:6789/0 conn(0x7fe48006e3a0 legacy=0x7fe48006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe484cf6700 1 -- 172.21.15.112:0/306055305 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701662835 0 0) 0x7fe48015c190 con 0x7fe48006e3a0 2023-08-27T11:18:50.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe485cf8700 1 -- 172.21.15.112:0/306055305 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4800b1910 con 0x7fe480158e20 2023-08-27T11:18:50.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe485cf8700 1 --2- 172.21.15.112:0/306055305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480158e20 0x7fe4800b11f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe470009840 tx=0x7fe470009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00b589851a0b96a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:50.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe484cf6700 1 -- 172.21.15.112:0/306055305 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470011830 con 0x7fe480158e20 2023-08-27T11:18:50.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe484cf6700 1 -- 172.21.15.112:0/306055305 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe470011990 con 0x7fe480158e20 2023-08-27T11:18:50.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.257+0000 7fe484cf6700 1 -- 172.21.15.112:0/306055305 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470011d20 con 0x7fe480158e20 2023-08-27T11:18:50.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.258+0000 7fe487f5c700 1 -- 172.21.15.112:0/306055305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480158e20 msgr2=0x7fe4800b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.258+0000 7fe487f5c700 1 --2- 172.21.15.112:0/306055305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480158e20 0x7fe4800b11f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe470009840 tx=0x7fe470009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:50.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.258+0000 7fe487f5c700 1 -- 172.21.15.112:0/306055305 shutdown_connections 2023-08-27T11:18:50.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.258+0000 7fe487f5c700 1 --2- 172.21.15.112:0/306055305 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe48006fea0 0x7fe4800af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.258+0000 7fe487f5c700 1 --2- 172.21.15.112:0/306055305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480158e20 0x7fe4800b11f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.258+0000 7fe487f5c700 1 -- 172.21.15.112:0/306055305 >> 172.21.15.112:0/306055305 conn(0x7fe4800d0000 msgr2=0x7fe4800d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:50.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.258+0000 7fe487f5c700 1 -- 172.21.15.112:0/306055305 shutdown_connections 2023-08-27T11:18:50.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.258+0000 7fe487f5c700 1 -- 172.21.15.112:0/306055305 wait complete. 2023-08-27T11:18:50.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.259+0000 7fe487f5c700 1 Processor -- start 2023-08-27T11:18:50.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.259+0000 7fe487f5c700 1 -- start start 2023-08-27T11:18:50.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe487f5c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48006e3a0 0x7fe4801f3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe487f5c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe48006fea0 0x7fe480153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe487f5c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480158e20 0x7fe480157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe487f5c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4801f30e0 con 0x7fe480158e20 2023-08-27T11:18:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe487f5c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe480158640 con 0x7fe48006e3a0 2023-08-27T11:18:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe487f5c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe480158360 con 0x7fe48006fea0 2023-08-27T11:18:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe4864f9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480158e20 0x7fe480157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe4864f9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480158e20 0x7fe480157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58098/0 (socket says 172.21.15.112:58098) 2023-08-27T11:18:50.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe4864f9700 1 -- 172.21.15.112:0/1263619846 learned_addr learned my addr 172.21.15.112:0/1263619846 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:50.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe4854f7700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe48006fea0 0x7fe480153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe485cf8700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48006e3a0 0x7fe4801f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe4864f9700 1 -- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe48006fea0 msgr2=0x7fe480153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe4864f9700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe48006fea0 0x7fe480153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe4864f9700 1 -- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48006e3a0 msgr2=0x7fe4801f3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe4864f9700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48006e3a0 0x7fe4801f3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe4864f9700 1 -- 172.21.15.112:0/1263619846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4700094a0 con 0x7fe480158e20 2023-08-27T11:18:50.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe4864f9700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480158e20 0x7fe480157910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe478007de0 tx=0x7fe47800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:50.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe4767fc700 1 -- 172.21.15.112:0/1263619846 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47800ede0 con 0x7fe480158e20 2023-08-27T11:18:50.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe4767fc700 1 -- 172.21.15.112:0/1263619846 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe478014420 con 0x7fe480158e20 2023-08-27T11:18:50.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe487f5c700 1 -- 172.21.15.112:0/1263619846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4801f2a20 con 0x7fe480158e20 2023-08-27T11:18:50.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.260+0000 7fe4767fc700 1 -- 172.21.15.112:0/1263619846 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4780147d0 con 0x7fe480158e20 2023-08-27T11:18:50.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.261+0000 7fe487f5c700 1 -- 172.21.15.112:0/1263619846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe480153a80 con 0x7fe480158e20 2023-08-27T11:18:50.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.261+0000 7fe487f5c700 1 -- 172.21.15.112:0/1263619846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe480067a20 con 0x7fe480158e20 2023-08-27T11:18:50.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.262+0000 7fe4767fc700 1 -- 172.21.15.112:0/1263619846 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe478014930 con 0x7fe480158e20 2023-08-27T11:18:50.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.262+0000 7fe4767fc700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe46406a0e0 0x7fe46406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:50.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.262+0000 7fe4767fc700 1 -- 172.21.15.112:0/1263619846 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe478088c50 con 0x7fe480158e20 2023-08-27T11:18:50.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.263+0000 7fe485cf8700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe46406a0e0 0x7fe46406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:50.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.263+0000 7fe485cf8700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe46406a0e0 0x7fe46406c5a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe4800afdf0 tx=0x7fe470008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:50.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.267+0000 7fe4767fc700 1 -- 172.21.15.112:0/1263619846 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe47808e020 con 0x7fe480158e20 2023-08-27T11:18:50.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.497+0000 7fe487f5c700 1 -- 172.21.15.112:0/1263619846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe480159510 con 0x7fe480158e20 2023-08-27T11:18:50.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.498+0000 7fe4767fc700 1 -- 172.21.15.112:0/1263619846 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe4780543d0 con 0x7fe480158e20 2023-08-27T11:18:50.499 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:18:50.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.500+0000 7fe487f5c700 1 -- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe46406a0e0 msgr2=0x7fe46406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.500+0000 7fe487f5c700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe46406a0e0 0x7fe46406c5a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe4800afdf0 tx=0x7fe470008030 comp rx=0 tx=0).stop 2023-08-27T11:18:50.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.500+0000 7fe487f5c700 1 -- 172.21.15.112:0/1263619846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480158e20 msgr2=0x7fe480157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:50.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.500+0000 7fe487f5c700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480158e20 0x7fe480157910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe478007de0 tx=0x7fe47800e3c0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.500+0000 7fe487f5c700 1 -- 172.21.15.112:0/1263619846 shutdown_connections 2023-08-27T11:18:50.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.500+0000 7fe487f5c700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe46406a0e0 0x7fe46406c5a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.500+0000 7fe487f5c700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480158e20 0x7fe480157910 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.500+0000 7fe487f5c700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48006e3a0 0x7fe4801f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.500+0000 7fe487f5c700 1 --2- 172.21.15.112:0/1263619846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe48006fea0 0x7fe480153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:50.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.500+0000 7fe487f5c700 1 -- 172.21.15.112:0/1263619846 >> 172.21.15.112:0/1263619846 conn(0x7fe4800d0000 msgr2=0x7fe480070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:50.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.500+0000 7fe487f5c700 1 -- 172.21.15.112:0/1263619846 shutdown_connections 2023-08-27T11:18:50.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:50.500+0000 7fe487f5c700 1 -- 172.21.15.112:0/1263619846 wait complete. 2023-08-27T11:18:50.512 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:18:50.513 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:18:50.513 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-08-27T11:18:50.513 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:18:50.514 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:18:50.514 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:18:50.514 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:18:50.515 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:18:50.515 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:18:50.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:50.517 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-08-27T11:18:50.534 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:18:50.535 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:18:50.535 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:18:50.535 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:50.535 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:18:50.535 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:50.536 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:18:50.536 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:50.536 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:18:50.536 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:50.536 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:18:50.537 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:50.537 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:18:50.537 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:50.537 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:18:50.537 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42312sec preferred_lft 42312sec 2023-08-27T11:18:50.538 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8d6c:4fe6:6a84:8222/64 scope link noprefixroute 2023-08-27T11:18:50.538 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:50.538 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:18:50.538 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:50.538 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:18:50.538 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:50.539 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::d87c:a1ff:fe1d:b635/64 scope link 2023-08-27T11:18:50.539 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:50.539 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:18:50.539 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:18:50.539 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::cc14:79ff:feb3:452f/64 scope link 2023-08-27T11:18:50.540 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:50.540 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:18:50.540 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether e6:51:f0:78:04:02 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:18:50.540 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::e451:f0ff:fe78:402/64 scope link 2023-08-27T11:18:50.540 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:50.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:50.542 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:18:50.603 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-08-27T11:18:50.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:50.604 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-08-27T11:18:50.665 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-08-27T11:18:50.666 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:18:50.666 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:18:50.667 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:18:50.667 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:18:50.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:50.668 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:18:50.669 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:18:50.669 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:18:50.670 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:18:50.670 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:18:50.671 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:18:50.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:50.773 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:18:50.790 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:18:50.790 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:18:50.790 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:18:50.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:50.791 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-08-27T11:18:50.856 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:18:50.857 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:18:50.913 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:18:50.914 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:18:50.914 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:18:50.970 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:18:50.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:50.971 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:18:51.055 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:18:51.056 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:18:51.056 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:18:51.064 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:18:51.064 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:18:51.065 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:18:51.065 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:18:51.065 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:18:51.095 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:18:51.095 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:18:51.096 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:18:51.096 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:18:51.096 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:18:51.096 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:18:51.097 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:18:51.097 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:18:51.099 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:18:51.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:51.100 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:18:51.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:51.136 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:18:51.219 INFO:teuthology.orchestra.run.smithi112.stdout:client5826 2023-08-27T11:18:51.219 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:18:51.220 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:51.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:18:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.437+0000 7fc8394a0700 1 Processor -- start 2023-08-27T11:18:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.437+0000 7fc8394a0700 1 -- start start 2023-08-27T11:18:51.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc8394a0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc83413f910 0x7fc8340cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:51.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc8394a0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8340ce090 0x7fc83406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc8394a0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc83406cb00 con 0x7fc83406ff70 2023-08-27T11:18:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc8394a0700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8340b2580 con 0x7fc83413f910 2023-08-27T11:18:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc8394a0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8340707a0 con 0x7fc8340ce090 2023-08-27T11:18:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc832ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc83413f910 0x7fc8340cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc832ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc83413f910 0x7fc8340cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58106/0 (socket says 172.21.15.112:58106) 2023-08-27T11:18:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc832ffd700 1 -- 172.21.15.112:0/147991205 learned_addr learned my addr 172.21.15.112:0/147991205 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc8337fe700 1 --1- 172.21.15.112:0/147991205 >> v1:172.21.15.112:6789/0 conn(0x7fc83406ff70 0x7fc8340b1e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52492/0 (socket says 172.21.15.112:52492) 2023-08-27T11:18:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc8327fc700 1 --2- 172.21.15.112:0/147991205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8340ce090 0x7fc83406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc832ffd700 1 -- 172.21.15.112:0/147991205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8340ce090 msgr2=0x7fc83406fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc832ffd700 1 --2- 172.21.15.112:0/147991205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8340ce090 0x7fc83406fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:51.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc832ffd700 1 -- 172.21.15.112:0/147991205 >> v1:172.21.15.112:6789/0 conn(0x7fc83406ff70 legacy=0x7fc8340b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:51.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc832ffd700 1 -- 172.21.15.112:0/147991205 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc834070a80 con 0x7fc83413f910 2023-08-27T11:18:51.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc831ffb700 1 -- 172.21.15.112:0/147991205 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389354794 0 0) 0x7fc83406cb00 con 0x7fc83406ff70 2023-08-27T11:18:51.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc832ffd700 1 --2- 172.21.15.112:0/147991205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc83413f910 0x7fc8340cdb50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc82800bb90 tx=0x7fc82800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63162746faadb40 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:51.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.438+0000 7fc831ffb700 1 -- 172.21.15.112:0/147991205 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82800dd30 con 0x7fc83413f910 2023-08-27T11:18:51.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.439+0000 7fc831ffb700 1 -- 172.21.15.112:0/147991205 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82800de90 con 0x7fc83413f910 2023-08-27T11:18:51.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.439+0000 7fc831ffb700 1 -- 172.21.15.112:0/147991205 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828010690 con 0x7fc83413f910 2023-08-27T11:18:51.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.439+0000 7fc8394a0700 1 -- 172.21.15.112:0/147991205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc83413f910 msgr2=0x7fc8340cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:51.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.439+0000 7fc8394a0700 1 --2- 172.21.15.112:0/147991205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc83413f910 0x7fc8340cdb50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc82800bb90 tx=0x7fc82800bea0 comp rx=0 tx=0).stop 2023-08-27T11:18:51.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.439+0000 7fc8394a0700 1 -- 172.21.15.112:0/147991205 shutdown_connections 2023-08-27T11:18:51.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.439+0000 7fc8394a0700 1 --2- 172.21.15.112:0/147991205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc83413f910 0x7fc8340cdb50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:51.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.439+0000 7fc8394a0700 1 --2- 172.21.15.112:0/147991205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8340ce090 0x7fc83406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:51.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.439+0000 7fc8394a0700 1 -- 172.21.15.112:0/147991205 >> 172.21.15.112:0/147991205 conn(0x7fc8340b0bf0 msgr2=0x7fc8340b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:51.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.439+0000 7fc8394a0700 1 -- 172.21.15.112:0/147991205 shutdown_connections 2023-08-27T11:18:51.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.439+0000 7fc8394a0700 1 -- 172.21.15.112:0/147991205 wait complete. 2023-08-27T11:18:51.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.440+0000 7fc8394a0700 1 Processor -- start 2023-08-27T11:18:51.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.440+0000 7fc8394a0700 1 -- start start 2023-08-27T11:18:51.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.441+0000 7fc8394a0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc83406ff70 0x7fc834157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:51.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.441+0000 7fc8394a0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8340ce090 0x7fc8341381c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:51.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.441+0000 7fc8394a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc83413f910 0x7fc8340af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:51.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.441+0000 7fc8394a0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc834135ee0 con 0x7fc83413f910 2023-08-27T11:18:51.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.441+0000 7fc8394a0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc834157ec0 con 0x7fc83406ff70 2023-08-27T11:18:51.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.441+0000 7fc8394a0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc834158d20 con 0x7fc8340ce090 2023-08-27T11:18:51.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.441+0000 7fc8337fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc83413f910 0x7fc8340af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:51.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.441+0000 7fc8337fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc83413f910 0x7fc8340af9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58116/0 (socket says 172.21.15.112:58116) 2023-08-27T11:18:51.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.441+0000 7fc8337fe700 1 -- 172.21.15.112:0/3441179947 learned_addr learned my addr 172.21.15.112:0/3441179947 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:51.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.441+0000 7fc8327fc700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8340ce090 0x7fc8341381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:51.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.441+0000 7fc832ffd700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc83406ff70 0x7fc834157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:51.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.442+0000 7fc8337fe700 1 -- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8340ce090 msgr2=0x7fc8341381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:51.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.442+0000 7fc8337fe700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8340ce090 0x7fc8341381c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:51.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.442+0000 7fc8337fe700 1 -- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc83406ff70 msgr2=0x7fc834157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:51.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.442+0000 7fc8337fe700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc83406ff70 0x7fc834157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:51.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.442+0000 7fc8337fe700 1 -- 172.21.15.112:0/3441179947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82800b7f0 con 0x7fc83413f910 2023-08-27T11:18:51.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.442+0000 7fc832ffd700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc83406ff70 0x7fc834157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:51.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.442+0000 7fc8337fe700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc83413f910 0x7fc8340af9d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00cb60 tx=0x7fc82c00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:51.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.442+0000 7fc81b7fe700 1 -- 172.21.15.112:0/3441179947 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c00aaa0 con 0x7fc83413f910 2023-08-27T11:18:51.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.442+0000 7fc81b7fe700 1 -- 172.21.15.112:0/3441179947 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82c00ac00 con 0x7fc83413f910 2023-08-27T11:18:51.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.442+0000 7fc81b7fe700 1 -- 172.21.15.112:0/3441179947 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0070d0 con 0x7fc83413f910 2023-08-27T11:18:51.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.442+0000 7fc8394a0700 1 -- 172.21.15.112:0/3441179947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8341591b0 con 0x7fc83413f910 2023-08-27T11:18:51.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.442+0000 7fc8394a0700 1 -- 172.21.15.112:0/3441179947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8341583f0 con 0x7fc83413f910 2023-08-27T11:18:51.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.443+0000 7fc8394a0700 1 -- 172.21.15.112:0/3441179947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc834158eb0 con 0x7fc83413f910 2023-08-27T11:18:51.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.443+0000 7fc81b7fe700 1 -- 172.21.15.112:0/3441179947 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc82c007230 con 0x7fc83413f910 2023-08-27T11:18:51.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.444+0000 7fc81b7fe700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc82006a0e0 0x7fc82006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:51.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.444+0000 7fc81b7fe700 1 -- 172.21.15.112:0/3441179947 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc82c0880e0 con 0x7fc83413f910 2023-08-27T11:18:51.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.444+0000 7fc832ffd700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc82006a0e0 0x7fc82006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:51.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.444+0000 7fc832ffd700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc82006a0e0 0x7fc82006c5a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc82800bee0 tx=0x7fc828013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:51.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.448+0000 7fc81b7fe700 1 -- 172.21.15.112:0/3441179947 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc82c053730 con 0x7fc83413f910 2023-08-27T11:18:51.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.680+0000 7fc8394a0700 1 -- 172.21.15.112:0/3441179947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc834071010 con 0x7fc83413f910 2023-08-27T11:18:51.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.680+0000 7fc81b7fe700 1 -- 172.21.15.112:0/3441179947 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc82c056d50 con 0x7fc83413f910 2023-08-27T11:18:51.682 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:18:51.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.682+0000 7fc8394a0700 1 -- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc82006a0e0 msgr2=0x7fc82006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:51.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.682+0000 7fc8394a0700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc82006a0e0 0x7fc82006c5a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc82800bee0 tx=0x7fc828013000 comp rx=0 tx=0).stop 2023-08-27T11:18:51.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.682+0000 7fc8394a0700 1 -- 172.21.15.112:0/3441179947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc83413f910 msgr2=0x7fc8340af9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:51.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.682+0000 7fc8394a0700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc83413f910 0x7fc8340af9d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00cb60 tx=0x7fc82c00ce70 comp rx=0 tx=0).stop 2023-08-27T11:18:51.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.682+0000 7fc8394a0700 1 -- 172.21.15.112:0/3441179947 shutdown_connections 2023-08-27T11:18:51.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.682+0000 7fc8394a0700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc82006a0e0 0x7fc82006c5a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:51.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.682+0000 7fc8394a0700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc83413f910 0x7fc8340af9d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:51.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.682+0000 7fc8394a0700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc83406ff70 0x7fc834157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:51.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.682+0000 7fc8394a0700 1 --2- 172.21.15.112:0/3441179947 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8340ce090 0x7fc8341381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:51.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.682+0000 7fc8394a0700 1 -- 172.21.15.112:0/3441179947 >> 172.21.15.112:0/3441179947 conn(0x7fc8340b0bf0 msgr2=0x7fc834142720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:51.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.682+0000 7fc8394a0700 1 -- 172.21.15.112:0/3441179947 shutdown_connections 2023-08-27T11:18:51.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:51.682+0000 7fc8394a0700 1 -- 172.21.15.112:0/3441179947 wait complete. 2023-08-27T11:18:51.696 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:18:51.696 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:18:51.696 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2023-08-27T11:18:51.697 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-27T11:18:51.697 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:18:51.697 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:18:51.698 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:18:51.698 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:18:51.698 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:18:51.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:51.699 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2023-08-27T11:18:51.721 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:18:51.722 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:18:51.722 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:18:51.722 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:51.722 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:18:51.723 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:51.723 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:18:51.723 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:51.723 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:18:51.724 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:51.724 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:18:51.724 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:51.724 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:18:51.725 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:51.725 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:18:51.725 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42327sec preferred_lft 42327sec 2023-08-27T11:18:51.725 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ac17:49c5:cb39:801a/64 scope link noprefixroute 2023-08-27T11:18:51.726 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:51.726 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:18:51.726 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:18:51.727 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:18:51.727 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:51.727 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c4b4:94ff:fe58:4e35/64 scope link 2023-08-27T11:18:51.727 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:51.727 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:18:51.728 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:18:51.728 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::5806:6aff:fe04:ef32/64 scope link 2023-08-27T11:18:51.728 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:51.728 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:18:51.729 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 8a:21:ba:15:69:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-27T11:18:51.729 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::8821:baff:fe15:6919/64 scope link 2023-08-27T11:18:51.729 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:18:51.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:51.730 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:18:51.791 INFO:teuthology.orchestra.run.smithi138.stdout:1 2023-08-27T11:18:51.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:51.794 DEBUG:teuthology.orchestra.run.smithi138:> route 2023-08-27T11:18:51.855 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2023-08-27T11:18:51.856 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:18:51.856 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:18:51.856 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:18:51.857 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:18:51.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:51.858 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:18:51.858 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:18:51.858 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:18:51.859 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:18:51.859 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:18:51.859 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:18:51.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:51.966 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:18:51.984 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-27T11:18:51.984 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-27T11:18:51.984 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:18:51.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:51.985 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2023-08-27T11:18:52.051 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:18:52.052 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-27T11:18:52.108 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:18:52.109 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:18:52.109 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:18:52.164 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:18:52.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:52.165 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:18:52.251 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:18:52.251 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:18:52.252 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:18:52.262 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:18:52.267 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:18:52.267 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:18:52.268 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:18:52.268 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:18:52.287 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:18:52.288 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:18:52.288 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:18:52.288 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:18:52.288 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:18:52.289 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:18:52.289 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:18:52.289 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:18:52.294 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:18:52.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:52.295 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:18:52.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:52.325 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:18:52.397 INFO:teuthology.orchestra.run.smithi138.stdout:client5153 2023-08-27T11:18:52.398 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:18:52.398 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:52.456 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-08-27T11:18:52.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.593+0000 7fbbe4a0e700 1 Processor -- start 2023-08-27T11:18:52.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.593+0000 7fbbe4a0e700 1 -- start start 2023-08-27T11:18:52.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbe4a0e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbe0157ae0 0x7fbbe015e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:52.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbe4a0e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbbe006f790 0x7fbbe006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:52.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbe4a0e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe0161390 con 0x7fbbe00b3de0 2023-08-27T11:18:52.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbe4a0e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe00b1aa0 con 0x7fbbe006f790 2023-08-27T11:18:52.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbe4a0e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0070180 con 0x7fbbe0157ae0 2023-08-27T11:18:52.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbddd9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbbe00b3de0 0x7fbbe0161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52502/0 (socket says 172.21.15.112:52502) 2023-08-27T11:18:52.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbddd9b700 1 -- 172.21.15.112:0/1636944256 learned_addr learned my addr 172.21.15.112:0/1636944256 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:52.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbded9d700 1 --2- 172.21.15.112:0/1636944256 >> v2:172.21.15.138:3300/0 conn(0x7fbbe006f790 0x7fbbe006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:52.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbde59c700 1 --2- 172.21.15.112:0/1636944256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbe0157ae0 0x7fbbe015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:52.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbdd59a700 1 -- 172.21.15.112:0/1636944256 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019479594 0 0) 0x7fbbe0161390 con 0x7fbbe00b3de0 2023-08-27T11:18:52.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbdd59a700 1 -- 172.21.15.112:0/1636944256 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc80035b0 con 0x7fbbe00b3de0 2023-08-27T11:18:52.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbde59c700 1 -- 172.21.15.112:0/1636944256 >> v2:172.21.15.138:3300/0 conn(0x7fbbe006f790 msgr2=0x7fbbe006fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:52.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbde59c700 1 --2- 172.21.15.112:0/1636944256 >> v2:172.21.15.138:3300/0 conn(0x7fbbe006f790 0x7fbbe006fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:52.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbde59c700 1 -- 172.21.15.112:0/1636944256 >> v1:172.21.15.112:6789/0 conn(0x7fbbe00b3de0 legacy=0x7fbbe0161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:52.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbde59c700 1 -- 172.21.15.112:0/1636944256 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe00b35f0 con 0x7fbbe0157ae0 2023-08-27T11:18:52.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.594+0000 7fbbded9d700 1 --2- 172.21.15.112:0/1636944256 >> v2:172.21.15.138:3300/0 conn(0x7fbbe006f790 0x7fbbe006fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:52.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.595+0000 7fbbde59c700 1 --2- 172.21.15.112:0/1636944256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbe0157ae0 0x7fbbe015e580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009840 tx=0x7fbbd0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c7eaeccb012e425 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:52.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.595+0000 7fbbdd59a700 1 -- 172.21.15.112:0/1636944256 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0011830 con 0x7fbbe0157ae0 2023-08-27T11:18:52.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.595+0000 7fbbdd59a700 1 -- 172.21.15.112:0/1636944256 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd0011990 con 0x7fbbe0157ae0 2023-08-27T11:18:52.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.595+0000 7fbbdd59a700 1 -- 172.21.15.112:0/1636944256 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0011d20 con 0x7fbbe0157ae0 2023-08-27T11:18:52.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.595+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/1636944256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbe0157ae0 msgr2=0x7fbbe015e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:52.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.595+0000 7fbbe4a0e700 1 --2- 172.21.15.112:0/1636944256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbe0157ae0 0x7fbbe015e580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009840 tx=0x7fbbd0009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:52.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.595+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/1636944256 shutdown_connections 2023-08-27T11:18:52.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.595+0000 7fbbe4a0e700 1 --2- 172.21.15.112:0/1636944256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbe0157ae0 0x7fbbe015e580 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:52.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.595+0000 7fbbe4a0e700 1 --2- 172.21.15.112:0/1636944256 >> v2:172.21.15.138:3300/0 conn(0x7fbbe006f790 0x7fbbe006fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:52.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.595+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/1636944256 >> 172.21.15.112:0/1636944256 conn(0x7fbbe00b1be0 msgr2=0x7fbbe00b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:52.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.595+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/1636944256 shutdown_connections 2023-08-27T11:18:52.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.595+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/1636944256 wait complete. 2023-08-27T11:18:52.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.597+0000 7fbbe4a0e700 1 Processor -- start 2023-08-27T11:18:52.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.597+0000 7fbbe4a0e700 1 -- start start 2023-08-27T11:18:52.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.597+0000 7fbbe4a0e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbe006f790 0x7fbbe00ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:52.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.597+0000 7fbbe4a0e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe00b3de0 0x7fbbe0160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:52.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.597+0000 7fbbe4a0e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe0157ae0 0x7fbbe0144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:52.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.597+0000 7fbbe4a0e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0160a70 con 0x7fbbe00b3de0 2023-08-27T11:18:52.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.597+0000 7fbbe4a0e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00adb60 con 0x7fbbe0157ae0 2023-08-27T11:18:52.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.597+0000 7fbbe4a0e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00ad880 con 0x7fbbe006f790 2023-08-27T11:18:52.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbddd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe00b3de0 0x7fbbe0160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:52.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbddd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe00b3de0 0x7fbbe0160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58128/0 (socket says 172.21.15.112:58128) 2023-08-27T11:18:52.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbddd9b700 1 -- 172.21.15.112:0/3153659204 learned_addr learned my addr 172.21.15.112:0/3153659204 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:52.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbde59c700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbe006f790 0x7fbbe00ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:52.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbded9d700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe0157ae0 0x7fbbe0144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:52.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbddd9b700 1 -- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbe006f790 msgr2=0x7fbbe00ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:52.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbddd9b700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbe006f790 0x7fbbe00ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:52.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbddd9b700 1 -- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe0157ae0 msgr2=0x7fbbe0144370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:52.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbddd9b700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe0157ae0 0x7fbbe0144370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:52.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbddd9b700 1 -- 172.21.15.112:0/3153659204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8009a70 con 0x7fbbe00b3de0 2023-08-27T11:18:52.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbddd9b700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe00b3de0 0x7fbbe0160530 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4005910 tx=0x7fbbd400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:52.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbceffd700 1 -- 172.21.15.112:0/3153659204 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4005d70 con 0x7fbbe00b3de0 2023-08-27T11:18:52.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbceffd700 1 -- 172.21.15.112:0/3153659204 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd40070d0 con 0x7fbbe00b3de0 2023-08-27T11:18:52.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/3153659204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd00094a0 con 0x7fbbe00b3de0 2023-08-27T11:18:52.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbceffd700 1 -- 172.21.15.112:0/3153659204 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4007460 con 0x7fbbe00b3de0 2023-08-27T11:18:52.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.598+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/3153659204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe00ae340 con 0x7fbbe00b3de0 2023-08-27T11:18:52.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.599+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/3153659204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe0067a20 con 0x7fbbe00b3de0 2023-08-27T11:18:52.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.600+0000 7fbbceffd700 1 -- 172.21.15.112:0/3153659204 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbbd40075c0 con 0x7fbbe00b3de0 2023-08-27T11:18:52.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.600+0000 7fbbceffd700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbbc006a0e0 0x7fbbc006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:52.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.600+0000 7fbbceffd700 1 -- 172.21.15.112:0/3153659204 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbbd4087930 con 0x7fbbe00b3de0 2023-08-27T11:18:52.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.600+0000 7fbbde59c700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbbc006a0e0 0x7fbbc006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:52.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.601+0000 7fbbde59c700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbbc006a0e0 0x7fbbc006c5a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009fd0 tx=0x7fbbd0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:52.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.604+0000 7fbbceffd700 1 -- 172.21.15.112:0/3153659204 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbbd4053020 con 0x7fbbe00b3de0 2023-08-27T11:18:52.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:52.839+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/3153659204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbbe0157650 con 0x7fbbe00b3de0 2023-08-27T11:18:53.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.269+0000 7fbbceffd700 1 -- 172.21.15.112:0/3153659204 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 136+0+0 (secure 0 0 0) 0x7fbbd4017020 con 0x7fbbe00b3de0 2023-08-27T11:18:53.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.271+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbbc006a0e0 msgr2=0x7fbbc006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.271+0000 7fbbe4a0e700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbbc006a0e0 0x7fbbc006c5a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009fd0 tx=0x7fbbd0008030 comp rx=0 tx=0).stop 2023-08-27T11:18:53.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.271+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/3153659204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe00b3de0 msgr2=0x7fbbe0160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.271+0000 7fbbe4a0e700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe00b3de0 0x7fbbe0160530 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4005910 tx=0x7fbbd400c9e0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.272+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/3153659204 shutdown_connections 2023-08-27T11:18:53.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.272+0000 7fbbe4a0e700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbbc006a0e0 0x7fbbc006c5a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.272+0000 7fbbe4a0e700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe0157ae0 0x7fbbe0144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.272+0000 7fbbe4a0e700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbe006f790 0x7fbbe00ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.272+0000 7fbbe4a0e700 1 --2- 172.21.15.112:0/3153659204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe00b3de0 0x7fbbe0160530 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.272+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/3153659204 >> 172.21.15.112:0/3153659204 conn(0x7fbbe00b1be0 msgr2=0x7fbbe0145a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.272+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/3153659204 shutdown_connections 2023-08-27T11:18:53.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.272+0000 7fbbe4a0e700 1 -- 172.21.15.112:0/3153659204 wait complete. 2023-08-27T11:18:53.288 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:18:53.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.409+0000 7f73b7e94700 1 Processor -- start 2023-08-27T11:18:53.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.410+0000 7f73b7e94700 1 -- start start 2023-08-27T11:18:53.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b7e94700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73b006cf80 0x7f73b00ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:53.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b7e94700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f73b00708a0 0x7f73b0070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:53.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b7e94700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b0071230 con 0x7f73b006dc20 2023-08-27T11:18:53.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b7e94700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b015a5c0 con 0x7f73b00708a0 2023-08-27T11:18:53.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b7e94700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b015b3e0 con 0x7f73b006cf80 2023-08-27T11:18:53.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b6431700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f73b00708a0 0x7f73b0070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:53.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b542f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73b006cf80 0x7f73b00ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:53.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b542f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73b006cf80 0x7f73b00ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:41558/0 (socket says 172.21.15.112:41558) 2023-08-27T11:18:53.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b5c30700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f73b006dc20 0x7f73b015e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:60040/0 (socket says 172.21.15.112:60040) 2023-08-27T11:18:53.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b542f700 1 -- 172.21.15.112:0/1031193251 learned_addr learned my addr 172.21.15.112:0/1031193251 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:53.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b6431700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f73b00708a0 0x7f73b0070cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:58564/0 (socket says 172.21.15.112:58564) 2023-08-27T11:18:53.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b6431700 1 -- 172.21.15.112:0/1031193251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73b006cf80 msgr2=0x7f73b00ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:53.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b6431700 1 --2- 172.21.15.112:0/1031193251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73b006cf80 0x7f73b00ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b6431700 1 -- 172.21.15.112:0/1031193251 >> v1:172.21.15.138:6789/0 conn(0x7f73b006dc20 legacy=0x7f73b015e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b6431700 1 -- 172.21.15.112:0/1031193251 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b015a7d0 con 0x7f73b00708a0 2023-08-27T11:18:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b542f700 1 --2- 172.21.15.112:0/1031193251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73b006cf80 0x7f73b00ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.411+0000 7f73b6431700 1 --2- 172.21.15.112:0/1031193251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73b00708a0 0x7f73b0070cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009d90 tx=0x7f73a000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d5383acd68648cd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:53.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.412+0000 7f73b4c2e700 1 -- 172.21.15.112:0/1031193251 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000e070 con 0x7f73b00708a0 2023-08-27T11:18:53.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.412+0000 7f73b4c2e700 1 -- 172.21.15.112:0/1031193251 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a000bb20 con 0x7f73b00708a0 2023-08-27T11:18:53.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.412+0000 7f73b4c2e700 1 -- 172.21.15.112:0/1031193251 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0012430 con 0x7f73b00708a0 2023-08-27T11:18:53.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.412+0000 7f73b7e94700 1 -- 172.21.15.112:0/1031193251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73b00708a0 msgr2=0x7f73b0070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.412+0000 7f73b7e94700 1 --2- 172.21.15.112:0/1031193251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73b00708a0 0x7f73b0070cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009d90 tx=0x7f73a000b330 comp rx=0 tx=0).stop 2023-08-27T11:18:53.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.412+0000 7f73b7e94700 1 -- 172.21.15.112:0/1031193251 shutdown_connections 2023-08-27T11:18:53.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.412+0000 7f73b7e94700 1 --2- 172.21.15.112:0/1031193251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73b00708a0 0x7f73b0070cf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.412+0000 7f73b7e94700 1 --2- 172.21.15.112:0/1031193251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73b006cf80 0x7f73b00ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.412+0000 7f73b7e94700 1 -- 172.21.15.112:0/1031193251 >> 172.21.15.112:0/1031193251 conn(0x7f73b00b0bf0 msgr2=0x7f73b00b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:53.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.412+0000 7f73b7e94700 1 -- 172.21.15.112:0/1031193251 shutdown_connections 2023-08-27T11:18:53.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.412+0000 7f73b7e94700 1 -- 172.21.15.112:0/1031193251 wait complete. 2023-08-27T11:18:53.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.413+0000 7f73b7e94700 1 Processor -- start 2023-08-27T11:18:53.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.413+0000 7f73b7e94700 1 -- start start 2023-08-27T11:18:53.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.414+0000 7f73b7e94700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73b006cf80 0x7f73b014d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:53.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.414+0000 7f73b7e94700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73b006dc20 0x7f73b01520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:53.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.414+0000 7f73b7e94700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73b00708a0 0x7f73b0158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:53.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.414+0000 7f73b7e94700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00a9b70 con 0x7f73b006cf80 2023-08-27T11:18:53.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.414+0000 7f73b7e94700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0152620 con 0x7f73b006dc20 2023-08-27T11:18:53.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.414+0000 7f73b7e94700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0151360 con 0x7f73b00708a0 2023-08-27T11:18:53.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.414+0000 7f73b5c30700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73b006cf80 0x7f73b014d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:53.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.414+0000 7f73b5c30700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73b006cf80 0x7f73b014d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58140/0 (socket says 172.21.15.112:58140) 2023-08-27T11:18:53.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.414+0000 7f73b5c30700 1 -- 172.21.15.112:0/3007299176 learned_addr learned my addr 172.21.15.112:0/3007299176 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:53.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.414+0000 7f73b6431700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73b00708a0 0x7f73b0158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:53.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.414+0000 7f73b542f700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73b006dc20 0x7f73b01520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:53.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.415+0000 7f73b5c30700 1 -- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73b00708a0 msgr2=0x7f73b0158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.415+0000 7f73b5c30700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73b00708a0 0x7f73b0158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.415+0000 7f73b5c30700 1 -- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73b006dc20 msgr2=0x7f73b01520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.415+0000 7f73b5c30700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73b006dc20 0x7f73b01520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.415+0000 7f73b5c30700 1 -- 172.21.15.112:0/3007299176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a4009550 con 0x7f73b006cf80 2023-08-27T11:18:53.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.415+0000 7f73b542f700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73b006dc20 0x7f73b01520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:53.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.415+0000 7f73b5c30700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73b006cf80 0x7f73b014d6b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f73a800e940 tx=0x7f73a800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:53.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.415+0000 7f73ae7fc700 1 -- 172.21.15.112:0/3007299176 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8004a70 con 0x7f73b006cf80 2023-08-27T11:18:53.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.415+0000 7f73ae7fc700 1 -- 172.21.15.112:0/3007299176 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a8004bd0 con 0x7f73b006cf80 2023-08-27T11:18:53.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.415+0000 7f73ae7fc700 1 -- 172.21.15.112:0/3007299176 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8007070 con 0x7f73b006cf80 2023-08-27T11:18:53.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.415+0000 7f73b7e94700 1 -- 172.21.15.112:0/3007299176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a0009a70 con 0x7f73b006cf80 2023-08-27T11:18:53.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.415+0000 7f73b7e94700 1 -- 172.21.15.112:0/3007299176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b01518e0 con 0x7f73b006cf80 2023-08-27T11:18:53.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.416+0000 7f73b7e94700 1 -- 172.21.15.112:0/3007299176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b0071890 con 0x7f73b006cf80 2023-08-27T11:18:53.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.416+0000 7f73ae7fc700 1 -- 172.21.15.112:0/3007299176 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f73a80071d0 con 0x7f73b006cf80 2023-08-27T11:18:53.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.417+0000 7f73ae7fc700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f739c06a0e0 0x7f739c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:53.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.417+0000 7f73ae7fc700 1 -- 172.21.15.112:0/3007299176 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f73a808a590 con 0x7f73b006cf80 2023-08-27T11:18:53.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.417+0000 7f73b542f700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f739c06a0e0 0x7f739c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:53.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.417+0000 7f73b542f700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f739c06a0e0 0x7f739c06c5a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f73b01592f0 tx=0x7f73a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:53.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.421+0000 7f73ae7fc700 1 -- 172.21.15.112:0/3007299176 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f73a8055c90 con 0x7f73b006cf80 2023-08-27T11:18:53.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.638+0000 7f73b7e94700 1 -- 172.21.15.112:0/3007299176 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f73b015ad90 con 0x7f739c06a0e0 2023-08-27T11:18:53.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.640+0000 7f73ae7fc700 1 -- 172.21.15.112:0/3007299176 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f73a4009960 con 0x7f739c06a0e0 2023-08-27T11:18:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:18:53.642 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:53.643 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs" 2023-08-27T11:18:53.644 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:53.644 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:18:53.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.642+0000 7f73b7e94700 1 -- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f739c06a0e0 msgr2=0x7f739c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.642+0000 7f73b7e94700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f739c06a0e0 0x7f739c06c5a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f73b01592f0 tx=0x7f73a4008040 comp rx=0 tx=0).stop 2023-08-27T11:18:53.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.642+0000 7f73b7e94700 1 -- 172.21.15.112:0/3007299176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73b006cf80 msgr2=0x7f73b014d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.642+0000 7f73b7e94700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73b006cf80 0x7f73b014d6b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f73a800e940 tx=0x7f73a800ec50 comp rx=0 tx=0).stop 2023-08-27T11:18:53.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.642+0000 7f73b7e94700 1 -- 172.21.15.112:0/3007299176 shutdown_connections 2023-08-27T11:18:53.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.642+0000 7f73b7e94700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f739c06a0e0 0x7f739c06c5a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.642+0000 7f73b7e94700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73b00708a0 0x7f73b0158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.642+0000 7f73b7e94700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73b006cf80 0x7f73b014d6b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.642+0000 7f73b7e94700 1 --2- 172.21.15.112:0/3007299176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73b006dc20 0x7f73b01520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.642+0000 7f73b7e94700 1 -- 172.21.15.112:0/3007299176 >> 172.21.15.112:0/3007299176 conn(0x7f73b00b0bf0 msgr2=0x7f73b0154270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:53.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.642+0000 7f73b7e94700 1 -- 172.21.15.112:0/3007299176 shutdown_connections 2023-08-27T11:18:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.642+0000 7f73b7e94700 1 -- 172.21.15.112:0/3007299176 wait complete. 2023-08-27T11:18:53.656 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:18:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.778+0000 7fcdd128d700 1 Processor -- start 2023-08-27T11:18:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.778+0000 7fcdd128d700 1 -- start start 2023-08-27T11:18:53.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.779+0000 7fcdd128d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdcc143ba0 0x7fcdcc0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:53.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.779+0000 7fcdd128d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcdcc06d5b0 0x7fcdcc06da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:53.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.779+0000 7fcdd128d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdcc0cfd50 con 0x7fcdcc1437c0 2023-08-27T11:18:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.779+0000 7fcdd128d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdcc06df40 con 0x7fcdcc06d5b0 2023-08-27T11:18:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.779+0000 7fcdd128d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0cf140 con 0x7fcdcc143ba0 2023-08-27T11:18:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.779+0000 7fcdcaffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fcdcc1437c0 0x7fcdcc0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52530/0 (socket says 172.21.15.112:52530) 2023-08-27T11:18:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.779+0000 7fcdcaffd700 1 -- 172.21.15.112:0/3379866583 learned_addr learned my addr 172.21.15.112:0/3379866583 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.779+0000 7fcdcb7fe700 1 --2- 172.21.15.112:0/3379866583 >> v2:172.21.15.138:3300/0 conn(0x7fcdcc06d5b0 0x7fcdcc06da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:53.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.779+0000 7fcdca7fc700 1 --2- 172.21.15.112:0/3379866583 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdcc143ba0 0x7fcdcc0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:53.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.779+0000 7fcdc9ffb700 1 -- 172.21.15.112:0/3379866583 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567018942 0 0) 0x7fcdcc0cfd50 con 0x7fcdcc1437c0 2023-08-27T11:18:53.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.780+0000 7fcdc9ffb700 1 -- 172.21.15.112:0/3379866583 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdb40035b0 con 0x7fcdcc1437c0 2023-08-27T11:18:53.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.780+0000 7fcdc9ffb700 1 -- 172.21.15.112:0/3379866583 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1843622112 0 0) 0x7fcdb40035b0 con 0x7fcdcc1437c0 2023-08-27T11:18:53.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.780+0000 7fcdcb7fe700 1 -- 172.21.15.112:0/3379866583 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdcc143ba0 msgr2=0x7fcdcc0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:53.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.780+0000 7fcdcb7fe700 1 --2- 172.21.15.112:0/3379866583 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdcc143ba0 0x7fcdcc0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.780+0000 7fcdcb7fe700 1 -- 172.21.15.112:0/3379866583 >> v1:172.21.15.112:6789/0 conn(0x7fcdcc1437c0 legacy=0x7fcdcc0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.780+0000 7fcdcb7fe700 1 -- 172.21.15.112:0/3379866583 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc0cf420 con 0x7fcdcc06d5b0 2023-08-27T11:18:53.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.780+0000 7fcdcb7fe700 1 --2- 172.21.15.112:0/3379866583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdcc06d5b0 0x7fcdcc06da00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009d90 tx=0x7fcdc400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbdde6481aff1c7a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:53.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.780+0000 7fcdc9ffb700 1 -- 172.21.15.112:0/3379866583 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc400e070 con 0x7fcdcc06d5b0 2023-08-27T11:18:53.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.780+0000 7fcdc9ffb700 1 -- 172.21.15.112:0/3379866583 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc4003a20 con 0x7fcdcc06d5b0 2023-08-27T11:18:53.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.780+0000 7fcdc9ffb700 1 -- 172.21.15.112:0/3379866583 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0001670 con 0x7fcdcc06d5b0 2023-08-27T11:18:53.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.781+0000 7fcdd128d700 1 -- 172.21.15.112:0/3379866583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdcc06d5b0 msgr2=0x7fcdcc06da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.781+0000 7fcdd128d700 1 --2- 172.21.15.112:0/3379866583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdcc06d5b0 0x7fcdcc06da00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009d90 tx=0x7fcdc400b330 comp rx=0 tx=0).stop 2023-08-27T11:18:53.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.781+0000 7fcdd128d700 1 -- 172.21.15.112:0/3379866583 shutdown_connections 2023-08-27T11:18:53.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.781+0000 7fcdd128d700 1 --2- 172.21.15.112:0/3379866583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdcc06d5b0 0x7fcdcc06da00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.781+0000 7fcdd128d700 1 --2- 172.21.15.112:0/3379866583 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdcc143ba0 0x7fcdcc0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.781+0000 7fcdd128d700 1 -- 172.21.15.112:0/3379866583 >> 172.21.15.112:0/3379866583 conn(0x7fcdcc0b1e90 msgr2=0x7fcdcc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:53.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.781+0000 7fcdd128d700 1 -- 172.21.15.112:0/3379866583 shutdown_connections 2023-08-27T11:18:53.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.781+0000 7fcdd128d700 1 -- 172.21.15.112:0/3379866583 wait complete. 2023-08-27T11:18:53.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.782+0000 7fcdd128d700 1 Processor -- start 2023-08-27T11:18:53.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.782+0000 7fcdd128d700 1 -- start start 2023-08-27T11:18:53.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdd128d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdcc06d5b0 0x7fcdcc1f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:53.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdd128d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdcc1437c0 0x7fcdcc0714e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:53.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdd128d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdcc143ba0 0x7fcdcc071a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:53.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdd128d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc06f830 con 0x7fcdcc143ba0 2023-08-27T11:18:53.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdd128d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc06fb10 con 0x7fcdcc1437c0 2023-08-27T11:18:53.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdd128d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc06fdf0 con 0x7fcdcc06d5b0 2023-08-27T11:18:53.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdcb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdcc143ba0 0x7fcdcc071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:53.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdca7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdcc1437c0 0x7fcdcc0714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:53.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdcaffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdcc06d5b0 0x7fcdcc1f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:53.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdca7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdcc1437c0 0x7fcdcc0714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:58582/0 (socket says 172.21.15.112:58582) 2023-08-27T11:18:53.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdcb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdcc143ba0 0x7fcdcc071a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58158/0 (socket says 172.21.15.112:58158) 2023-08-27T11:18:53.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdca7fc700 1 -- 172.21.15.112:0/829312089 learned_addr learned my addr 172.21.15.112:0/829312089 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:53.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdcb7fe700 1 -- 172.21.15.112:0/829312089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdcc06d5b0 msgr2=0x7fcdcc1f00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdcb7fe700 1 --2- 172.21.15.112:0/829312089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdcc06d5b0 0x7fcdcc1f00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdcb7fe700 1 -- 172.21.15.112:0/829312089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdcc1437c0 msgr2=0x7fcdcc0714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdcb7fe700 1 --2- 172.21.15.112:0/829312089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdcc1437c0 0x7fcdcc0714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdcb7fe700 1 -- 172.21.15.112:0/829312089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc4009a70 con 0x7fcdcc143ba0 2023-08-27T11:18:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdca7fc700 1 --2- 172.21.15.112:0/829312089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdcc1437c0 0x7fcdcc0714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:53.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.783+0000 7fcdcb7fe700 1 --2- 172.21.15.112:0/829312089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdcc143ba0 0x7fcdcc071a20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4015000 tx=0x7fcdc4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:53.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.784+0000 7fcdb37fe700 1 -- 172.21.15.112:0/829312089 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc40044c0 con 0x7fcdcc143ba0 2023-08-27T11:18:53.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.784+0000 7fcdb37fe700 1 -- 172.21.15.112:0/829312089 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc4004620 con 0x7fcdcc143ba0 2023-08-27T11:18:53.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.784+0000 7fcdd128d700 1 -- 172.21.15.112:0/829312089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdcc0700d0 con 0x7fcdcc143ba0 2023-08-27T11:18:53.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.784+0000 7fcdb37fe700 1 -- 172.21.15.112:0/829312089 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4012430 con 0x7fcdcc143ba0 2023-08-27T11:18:53.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.784+0000 7fcdd128d700 1 -- 172.21.15.112:0/829312089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdcc1f0620 con 0x7fcdcc143ba0 2023-08-27T11:18:53.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.784+0000 7fcdd128d700 1 -- 172.21.15.112:0/829312089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdcc15a970 con 0x7fcdcc143ba0 2023-08-27T11:18:53.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.786+0000 7fcdb37fe700 1 -- 172.21.15.112:0/829312089 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcdc4012590 con 0x7fcdcc143ba0 2023-08-27T11:18:53.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.786+0000 7fcdb37fe700 1 --2- 172.21.15.112:0/829312089 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcdb806a0e0 0x7fcdb806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:53.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.786+0000 7fcdb37fe700 1 -- 172.21.15.112:0/829312089 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fcdc408a410 con 0x7fcdcc143ba0 2023-08-27T11:18:53.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.786+0000 7fcdcaffd700 1 --2- 172.21.15.112:0/829312089 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcdb806a0e0 0x7fcdb806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:53.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.787+0000 7fcdcaffd700 1 --2- 172.21.15.112:0/829312089 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcdb806a0e0 0x7fcdb806c5a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0001eb0 tx=0x7fcdc0001e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:53.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.790+0000 7fcdb37fe700 1 -- 172.21.15.112:0/829312089 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcdc4055bf0 con 0x7fcdcc143ba0 2023-08-27T11:18:53.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.979+0000 7fcdd128d700 1 -- 172.21.15.112:0/829312089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcdcc15a970 con 0x7fcdcc143ba0 2023-08-27T11:18:53.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.979+0000 7fcdb37fe700 1 -- 172.21.15.112:0/829312089 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcdcc15a970 con 0x7fcdcc143ba0 2023-08-27T11:18:53.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.981+0000 7fcdd128d700 1 -- 172.21.15.112:0/829312089 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcdb806a0e0 msgr2=0x7fcdb806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.981+0000 7fcdd128d700 1 --2- 172.21.15.112:0/829312089 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcdb806a0e0 0x7fcdb806c5a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0001eb0 tx=0x7fcdc0001e40 comp rx=0 tx=0).stop 2023-08-27T11:18:53.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.981+0000 7fcdd128d700 1 -- 172.21.15.112:0/829312089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdcc143ba0 msgr2=0x7fcdcc071a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:53.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.981+0000 7fcdd128d700 1 --2- 172.21.15.112:0/829312089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdcc143ba0 0x7fcdcc071a20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4015000 tx=0x7fcdc4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.981+0000 7fcdd128d700 1 -- 172.21.15.112:0/829312089 shutdown_connections 2023-08-27T11:18:53.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.981+0000 7fcdd128d700 1 --2- 172.21.15.112:0/829312089 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcdb806a0e0 0x7fcdb806c5a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.981+0000 7fcdd128d700 1 --2- 172.21.15.112:0/829312089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdcc143ba0 0x7fcdcc071a20 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.981+0000 7fcdd128d700 1 --2- 172.21.15.112:0/829312089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdcc06d5b0 0x7fcdcc1f00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.981+0000 7fcdd128d700 1 --2- 172.21.15.112:0/829312089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdcc1437c0 0x7fcdcc0714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:53.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.981+0000 7fcdd128d700 1 -- 172.21.15.112:0/829312089 >> 172.21.15.112:0/829312089 conn(0x7fcdcc0b1e90 msgr2=0x7fcdcc15ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:53.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.982+0000 7fcdd128d700 1 -- 172.21.15.112:0/829312089 shutdown_connections 2023-08-27T11:18:53.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:53.982+0000 7fcdd128d700 1 -- 172.21.15.112:0/829312089 wait complete. 2023-08-27T11:18:53.996 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000922281 2023-08-27T11:18:54.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.118+0000 7ff3e19bf700 1 Processor -- start 2023-08-27T11:18:54.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.118+0000 7ff3e19bf700 1 -- start start 2023-08-27T11:18:54.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.119+0000 7ff3e19bf700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff3dc146bf0 0x7ff3dc0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.119+0000 7ff3e19bf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3dc146fd0 0x7ff3dc0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.119+0000 7ff3e19bf700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc0d1500 con 0x7ff3dc0d0290 2023-08-27T11:18:54.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.119+0000 7ff3e19bf700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc0d0710 con 0x7ff3dc146bf0 2023-08-27T11:18:54.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.119+0000 7ff3e19bf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0d09f0 con 0x7ff3dc146fd0 2023-08-27T11:18:54.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.119+0000 7ff3db7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff3dc0d0290 0x7ff3dc070eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52548/0 (socket says 172.21.15.112:52548) 2023-08-27T11:18:54.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.119+0000 7ff3db7fe700 1 -- 172.21.15.112:0/2853626174 learned_addr learned my addr 172.21.15.112:0/2853626174 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:54.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.119+0000 7ff3daffd700 1 --2- 172.21.15.112:0/2853626174 >> v2:172.21.15.138:3300/0 conn(0x7ff3dc146bf0 0x7ff3dc0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3da7fc700 1 --2- 172.21.15.112:0/2853626174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3dc146fd0 0x7ff3dc0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761221113 0 0) 0x7ff3dc0d1500 con 0x7ff3dc0d0290 2023-08-27T11:18:54.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c40035b0 con 0x7ff3dc0d0290 2023-08-27T11:18:54.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3873390350 0 0) 0x7ff3c40035b0 con 0x7ff3dc0d0290 2023-08-27T11:18:54.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3dc146fd0 msgr2=0x7ff3dc0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:54.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 --2- 172.21.15.112:0/2853626174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3dc146fd0 0x7ff3dc0cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 >> v2:172.21.15.138:3300/0 conn(0x7ff3dc146bf0 msgr2=0x7ff3dc0b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:54.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 --2- 172.21.15.112:0/2853626174 >> v2:172.21.15.138:3300/0 conn(0x7ff3dc146bf0 0x7ff3dc0b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc0d0cd0 con 0x7ff3dc0d0290 2023-08-27T11:18:54.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7ff3d40019f0 con 0x7ff3dc0d0290 2023-08-27T11:18:54.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 >> v1:172.21.15.112:6789/0 conn(0x7ff3dc0d0290 legacy=0x7ff3dc070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:54.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 --2- 172.21.15.112:0/2853626174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3dc146fd0 0x7ff3c4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 --2- 172.21.15.112:0/2853626174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3c4003fd0 0x7ff3c4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 --2- 172.21.15.112:0/2853626174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c4004d90 0x7ff3c4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0d0710 con 0x7ff3c4004d90 2023-08-27T11:18:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0d09f0 con 0x7ff3c4003fd0 2023-08-27T11:18:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c400bac0 con 0x7ff3dc146fd0 2023-08-27T11:18:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3db7fe700 1 --2- 172.21.15.112:0/2853626174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c4004d90 0x7ff3c4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3da7fc700 1 --2- 172.21.15.112:0/2853626174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3c4003fd0 0x7ff3c4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3daffd700 1 --2- 172.21.15.112:0/2853626174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3dc146fd0 0x7ff3c4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3db7fe700 1 -- 172.21.15.112:0/2853626174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3dc146fd0 msgr2=0x7ff3c4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:54.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3db7fe700 1 --2- 172.21.15.112:0/2853626174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3dc146fd0 0x7ff3c4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3db7fe700 1 -- 172.21.15.112:0/2853626174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3c4003fd0 msgr2=0x7ff3c4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:54.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3db7fe700 1 --2- 172.21.15.112:0/2853626174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3c4003fd0 0x7ff3c4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3db7fe700 1 -- 172.21.15.112:0/2853626174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c40035b0 con 0x7ff3c4004d90 2023-08-27T11:18:54.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.120+0000 7ff3db7fe700 1 --2- 172.21.15.112:0/2853626174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c4004d90 0x7ff3c4005210 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0721b0 tx=0x7ff3c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcfee379901904ea server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:54.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.121+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40072d0 con 0x7ff3c4004d90 2023-08-27T11:18:54.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.121+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d4007430 con 0x7ff3c4004d90 2023-08-27T11:18:54.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.121+0000 7ff3d9ffb700 1 -- 172.21.15.112:0/2853626174 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40077e0 con 0x7ff3c4004d90 2023-08-27T11:18:54.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.121+0000 7ff3e19bf700 1 -- 172.21.15.112:0/2853626174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c4004d90 msgr2=0x7ff3c4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:54.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.121+0000 7ff3e19bf700 1 --2- 172.21.15.112:0/2853626174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c4004d90 0x7ff3c4005210 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0721b0 tx=0x7ff3c4005880 comp rx=0 tx=0).stop 2023-08-27T11:18:54.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.121+0000 7ff3db7fe700 1 -- 172.21.15.112:0/2853626174 reap_dead start 2023-08-27T11:18:54.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.121+0000 7ff3e19bf700 1 -- 172.21.15.112:0/2853626174 shutdown_connections 2023-08-27T11:18:54.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.121+0000 7ff3e19bf700 1 -- 172.21.15.112:0/2853626174 >> 172.21.15.112:0/2853626174 conn(0x7ff3dc06c450 msgr2=0x7ff3dc06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:54.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.121+0000 7ff3e19bf700 1 -- 172.21.15.112:0/2853626174 shutdown_connections 2023-08-27T11:18:54.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.121+0000 7ff3e19bf700 1 -- 172.21.15.112:0/2853626174 wait complete. 2023-08-27T11:18:54.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.122+0000 7ff3e19bf700 1 Processor -- start 2023-08-27T11:18:54.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.123+0000 7ff3e19bf700 1 -- start start 2023-08-27T11:18:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.123+0000 7ff3e19bf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3c4004d90 0x7ff3dc0b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.123+0000 7ff3e19bf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3dc15afc0 0x7ff3dc1582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.123+0000 7ff3e19bf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3dc0b1c40 0x7ff3dc0b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.123+0000 7ff3e19bf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0b2600 con 0x7ff3dc0b1c40 2023-08-27T11:18:54.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.123+0000 7ff3e19bf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0b8c50 con 0x7ff3c4004d90 2023-08-27T11:18:54.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.123+0000 7ff3e19bf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc13d940 con 0x7ff3dc15afc0 2023-08-27T11:18:54.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.123+0000 7ff3db7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3dc0b1c40 0x7ff3dc0b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.123+0000 7ff3db7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3dc0b1c40 0x7ff3dc0b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58178/0 (socket says 172.21.15.112:58178) 2023-08-27T11:18:54.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.123+0000 7ff3db7fe700 1 -- 172.21.15.112:0/157618283 learned_addr learned my addr 172.21.15.112:0/157618283 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:54.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.124+0000 7ff3da7fc700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3dc15afc0 0x7ff3dc1582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.124+0000 7ff3daffd700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3c4004d90 0x7ff3dc0b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.124+0000 7ff3db7fe700 1 -- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3dc15afc0 msgr2=0x7ff3dc1582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:54.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.124+0000 7ff3db7fe700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3dc15afc0 0x7ff3dc1582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.124+0000 7ff3db7fe700 1 -- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3c4004d90 msgr2=0x7ff3dc0b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:54.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.124+0000 7ff3db7fe700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3c4004d90 0x7ff3dc0b6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.124+0000 7ff3db7fe700 1 -- 172.21.15.112:0/157618283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc158830 con 0x7ff3dc0b1c40 2023-08-27T11:18:54.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.124+0000 7ff3db7fe700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3dc0b1c40 0x7ff3dc0b20c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4002230 tx=0x7ff3d4001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:54.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.124+0000 7ff3c3fff700 1 -- 172.21.15.112:0/157618283 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4025030 con 0x7ff3dc0b1c40 2023-08-27T11:18:54.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.124+0000 7ff3c3fff700 1 -- 172.21.15.112:0/157618283 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d4001a00 con 0x7ff3dc0b1c40 2023-08-27T11:18:54.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.124+0000 7ff3e19bf700 1 -- 172.21.15.112:0/157618283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3dc0b73f0 con 0x7ff3dc0b1c40 2023-08-27T11:18:54.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.124+0000 7ff3c3fff700 1 -- 172.21.15.112:0/157618283 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4012620 con 0x7ff3dc0b1c40 2023-08-27T11:18:54.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.124+0000 7ff3e19bf700 1 -- 172.21.15.112:0/157618283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3dc0b7110 con 0x7ff3dc0b1c40 2023-08-27T11:18:54.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.125+0000 7ff3e19bf700 1 -- 172.21.15.112:0/157618283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3dc13d7a0 con 0x7ff3dc0b1c40 2023-08-27T11:18:54.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.125+0000 7ff3c3fff700 1 -- 172.21.15.112:0/157618283 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff3d4028070 con 0x7ff3dc0b1c40 2023-08-27T11:18:54.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.126+0000 7ff3c3fff700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff3c806a0e0 0x7ff3c806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.126+0000 7ff3c3fff700 1 -- 172.21.15.112:0/157618283 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6178+0+0 (secure 0 0 0) 0x7ff3d4026070 con 0x7ff3dc0b1c40 2023-08-27T11:18:54.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.126+0000 7ff3daffd700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff3c806a0e0 0x7ff3c806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.126+0000 7ff3daffd700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff3c806a0e0 0x7ff3c806c5a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009b20 tx=0x7ff3d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:54.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.130+0000 7ff3c3fff700 1 -- 172.21.15.112:0/157618283 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff3dc13d7a0 con 0x7ff3dc0b1c40 2023-08-27T11:18:54.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.355+0000 7ff3e19bf700 1 -- 172.21.15.112:0/157618283 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group_0000000000922281", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3dc13d7a0 con 0x7ff3c806a0e0 2023-08-27T11:18:54.362 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:18:54.362+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:18:54.362 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:18:54.362+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:18:54.362 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:18:54.362+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:18:54.363 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:18:54.362+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:18:54.363 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:18:54.362+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:18:54.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.376+0000 7ff3c3fff700 1 -- 172.21.15.112:0/157618283 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff3dc13d7a0 con 0x7ff3c806a0e0 2023-08-27T11:18:54.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.378+0000 7ff3e19bf700 1 -- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff3c806a0e0 msgr2=0x7ff3c806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:54.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.378+0000 7ff3e19bf700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff3c806a0e0 0x7ff3c806c5a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009b20 tx=0x7ff3d0008040 comp rx=0 tx=0).stop 2023-08-27T11:18:54.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.378+0000 7ff3e19bf700 1 -- 172.21.15.112:0/157618283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3dc0b1c40 msgr2=0x7ff3dc0b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:54.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.378+0000 7ff3e19bf700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3dc0b1c40 0x7ff3dc0b20c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4002230 tx=0x7ff3d4001360 comp rx=0 tx=0).stop 2023-08-27T11:18:54.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.379+0000 7ff3e19bf700 1 -- 172.21.15.112:0/157618283 shutdown_connections 2023-08-27T11:18:54.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.379+0000 7ff3e19bf700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff3c806a0e0 0x7ff3c806c5a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.379+0000 7ff3e19bf700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3dc0b1c40 0x7ff3dc0b20c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.379+0000 7ff3e19bf700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3c4004d90 0x7ff3dc0b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.379+0000 7ff3e19bf700 1 --2- 172.21.15.112:0/157618283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3dc15afc0 0x7ff3dc1582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.379+0000 7ff3e19bf700 1 -- 172.21.15.112:0/157618283 >> 172.21.15.112:0/157618283 conn(0x7ff3dc06c450 msgr2=0x7ff3dc144920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:54.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.379+0000 7ff3e19bf700 1 -- 172.21.15.112:0/157618283 shutdown_connections 2023-08-27T11:18:54.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.379+0000 7ff3e19bf700 1 -- 172.21.15.112:0/157618283 wait complete. 2023-08-27T11:18:54.396 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-08-27T11:18:54.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.517+0000 7fd30f01b700 1 Processor -- start 2023-08-27T11:18:54.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.517+0000 7fd30f01b700 1 -- start start 2023-08-27T11:18:54.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.518+0000 7fd30f01b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd308071980 0x7fd3080cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.518+0000 7fd30f01b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3080cd2e0 con 0x7fd3080ccea0 2023-08-27T11:18:54.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.518+0000 7fd30f01b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30815b510 con 0x7fd30815ea00 2023-08-27T11:18:54.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.518+0000 7fd30f01b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd30815e5e0 con 0x7fd308071980 2023-08-27T11:18:54.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.518+0000 7fd30cdb7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd308071980 0x7fd3080cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.518+0000 7fd30d5b8700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd3080ccea0 0x7fd3080ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52564/0 (socket says 172.21.15.112:52564) 2023-08-27T11:18:54.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.518+0000 7fd30d5b8700 1 -- 172.21.15.112:0/3353993835 learned_addr learned my addr 172.21.15.112:0/3353993835 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:54.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.518+0000 7fd30cdb7700 1 -- 172.21.15.112:0/3353993835 >> v1:172.21.15.138:6789/0 conn(0x7fd30815ea00 legacy=0x7fd30815a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:54.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.518+0000 7fd30cdb7700 1 -- 172.21.15.112:0/3353993835 >> v1:172.21.15.112:6789/0 conn(0x7fd3080ccea0 legacy=0x7fd3080ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:54.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.518+0000 7fd3077fe700 1 -- 172.21.15.112:0/3353993835 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004999625 0 0) 0x7fd3080cd2e0 con 0x7fd3080ccea0 2023-08-27T11:18:54.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.518+0000 7fd30cdb7700 1 -- 172.21.15.112:0/3353993835 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30815df20 con 0x7fd308071980 2023-08-27T11:18:54.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.519+0000 7fd30cdb7700 1 --2- 172.21.15.112:0/3353993835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd308071980 0x7fd3080cc960 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009840 tx=0x7fd2fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1126fc7b68788172 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:54.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.519+0000 7fd3077fe700 1 -- 172.21.15.112:0/3353993835 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc004230 con 0x7fd308071980 2023-08-27T11:18:54.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.519+0000 7fd3077fe700 1 -- 172.21.15.112:0/3353993835 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc004390 con 0x7fd308071980 2023-08-27T11:18:54.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.519+0000 7fd3077fe700 1 -- 172.21.15.112:0/3353993835 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc0036a0 con 0x7fd308071980 2023-08-27T11:18:54.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.519+0000 7fd30f01b700 1 -- 172.21.15.112:0/3353993835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd308071980 msgr2=0x7fd3080cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:54.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.519+0000 7fd30f01b700 1 --2- 172.21.15.112:0/3353993835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd308071980 0x7fd3080cc960 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009840 tx=0x7fd2fc009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:54.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.519+0000 7fd30f01b700 1 -- 172.21.15.112:0/3353993835 shutdown_connections 2023-08-27T11:18:54.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.519+0000 7fd30f01b700 1 --2- 172.21.15.112:0/3353993835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd308071980 0x7fd3080cc960 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.519+0000 7fd30f01b700 1 -- 172.21.15.112:0/3353993835 >> 172.21.15.112:0/3353993835 conn(0x7fd30806bbf0 msgr2=0x7fd3080cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:54.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.519+0000 7fd30f01b700 1 -- 172.21.15.112:0/3353993835 shutdown_connections 2023-08-27T11:18:54.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.519+0000 7fd30f01b700 1 -- 172.21.15.112:0/3353993835 wait complete. 2023-08-27T11:18:54.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.520+0000 7fd30f01b700 1 Processor -- start 2023-08-27T11:18:54.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.521+0000 7fd30f01b700 1 -- start start 2023-08-27T11:18:54.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.521+0000 7fd30f01b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd308071980 0x7fd3081f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.521+0000 7fd30f01b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3080ccea0 0x7fd308153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.521+0000 7fd30f01b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30815ea00 0x7fd3080b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.521+0000 7fd30f01b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd308152690 con 0x7fd3080ccea0 2023-08-27T11:18:54.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.521+0000 7fd30f01b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3081520d0 con 0x7fd30815ea00 2023-08-27T11:18:54.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.521+0000 7fd30f01b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3081523b0 con 0x7fd308071980 2023-08-27T11:18:54.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.521+0000 7fd307fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3080ccea0 0x7fd308153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd307fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3080ccea0 0x7fd308153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58188/0 (socket says 172.21.15.112:58188) 2023-08-27T11:18:54.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd307fff700 1 -- 172.21.15.112:0/575667888 learned_addr learned my addr 172.21.15.112:0/575667888 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:54.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd30cdb7700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd308071980 0x7fd3081f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd30d5b8700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30815ea00 0x7fd3080b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd307fff700 1 -- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd308071980 msgr2=0x7fd3081f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:54.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd307fff700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd308071980 0x7fd3081f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd307fff700 1 -- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30815ea00 msgr2=0x7fd3080b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:54.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd307fff700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30815ea00 0x7fd3080b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd307fff700 1 -- 172.21.15.112:0/575667888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc0094a0 con 0x7fd3080ccea0 2023-08-27T11:18:54.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd307fff700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3080ccea0 0x7fd308153920 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400b4a0 tx=0x7fd2f400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:54.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd3057fa700 1 -- 172.21.15.112:0/575667888 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4011830 con 0x7fd3080ccea0 2023-08-27T11:18:54.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd3057fa700 1 -- 172.21.15.112:0/575667888 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f4011990 con 0x7fd3080ccea0 2023-08-27T11:18:54.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd30f01b700 1 -- 172.21.15.112:0/575667888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3081f2db0 con 0x7fd3080ccea0 2023-08-27T11:18:54.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd3057fa700 1 -- 172.21.15.112:0/575667888 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4011d40 con 0x7fd3080ccea0 2023-08-27T11:18:54.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.522+0000 7fd30f01b700 1 -- 172.21.15.112:0/575667888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3081f3af0 con 0x7fd3080ccea0 2023-08-27T11:18:54.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.523+0000 7fd30f01b700 1 -- 172.21.15.112:0/575667888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd308066fa0 con 0x7fd3080ccea0 2023-08-27T11:18:54.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.523+0000 7fd3057fa700 1 -- 172.21.15.112:0/575667888 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd2f401a400 con 0x7fd3080ccea0 2023-08-27T11:18:54.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.524+0000 7fd3057fa700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd2ec06a0e0 0x7fd2ec06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:54.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.524+0000 7fd3057fa700 1 -- 172.21.15.112:0/575667888 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fd2f4088300 con 0x7fd3080ccea0 2023-08-27T11:18:54.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.524+0000 7fd30cdb7700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd2ec06a0e0 0x7fd2ec06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:54.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.525+0000 7fd30cdb7700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd2ec06a0e0 0x7fd2ec06c5a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc000c00 tx=0x7fd2fc01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:54.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.528+0000 7fd3057fa700 1 -- 172.21.15.112:0/575667888 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd2f4053a20 con 0x7fd3080ccea0 2023-08-27T11:18:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.744+0000 7fd30f01b700 1 -- 172.21.15.112:0/575667888 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7fd30815e200 con 0x7fd2ec06a0e0 2023-08-27T11:18:54.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.748+0000 7fd3057fa700 1 -- 172.21.15.112:0/575667888 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+531 (secure 0 0 0) 0x7fd2fc0041b0 con 0x7fd2ec06a0e0 2023-08-27T11:18:54.749 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:18:54.750 INFO:teuthology.orchestra.run.smithi112.stdout: "mon_addrs": [ 2023-08-27T11:18:54.750 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.112:6789", 2023-08-27T11:18:54.750 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.138:6789", 2023-08-27T11:18:54.750 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.138:6790" 2023-08-27T11:18:54.751 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2023-08-27T11:18:54.751 INFO:teuthology.orchestra.run.smithi112.stdout: "pending_subvolume_deletions": 0, 2023-08-27T11:18:54.751 INFO:teuthology.orchestra.run.smithi112.stdout: "pools": { 2023-08-27T11:18:54.751 INFO:teuthology.orchestra.run.smithi112.stdout: "data": [ 2023-08-27T11:18:54.752 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:54.752 INFO:teuthology.orchestra.run.smithi112.stdout: "avail": 367104065536, 2023-08-27T11:18:54.752 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs_data", 2023-08-27T11:18:54.752 INFO:teuthology.orchestra.run.smithi112.stdout: "used": 0 2023-08-27T11:18:54.753 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:54.753 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2023-08-27T11:18:54.753 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": [ 2023-08-27T11:18:54.753 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:18:54.754 INFO:teuthology.orchestra.run.smithi112.stdout: "avail": 367104065536, 2023-08-27T11:18:54.754 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs_metadata", 2023-08-27T11:18:54.754 INFO:teuthology.orchestra.run.smithi112.stdout: "used": 65536 2023-08-27T11:18:54.754 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:18:54.755 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:18:54.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:18:54.755 INFO:teuthology.orchestra.run.smithi112.stdout: "used_size": 0 2023-08-27T11:18:54.755 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:18:54.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.750+0000 7fd30f01b700 1 -- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd2ec06a0e0 msgr2=0x7fd2ec06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:54.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.750+0000 7fd30f01b700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd2ec06a0e0 0x7fd2ec06c5a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc000c00 tx=0x7fd2fc01c040 comp rx=0 tx=0).stop 2023-08-27T11:18:54.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.750+0000 7fd30f01b700 1 -- 172.21.15.112:0/575667888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3080ccea0 msgr2=0x7fd308153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:54.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.750+0000 7fd30f01b700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3080ccea0 0x7fd308153920 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400b4a0 tx=0x7fd2f400b860 comp rx=0 tx=0).stop 2023-08-27T11:18:54.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.750+0000 7fd30f01b700 1 -- 172.21.15.112:0/575667888 shutdown_connections 2023-08-27T11:18:54.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.750+0000 7fd30f01b700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd2ec06a0e0 0x7fd2ec06c5a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.750+0000 7fd30f01b700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30815ea00 0x7fd3080b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.750+0000 7fd30f01b700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd308071980 0x7fd3081f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.750+0000 7fd30f01b700 1 --2- 172.21.15.112:0/575667888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3080ccea0 0x7fd308153920 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.750+0000 7fd30f01b700 1 -- 172.21.15.112:0/575667888 >> 172.21.15.112:0/575667888 conn(0x7fd30806bbf0 msgr2=0x7fd3080b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.750+0000 7fd30f01b700 1 -- 172.21.15.112:0/575667888 shutdown_connections 2023-08-27T11:18:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:54.750+0000 7fd30f01b700 1 -- 172.21.15.112:0/575667888 wait complete. 2023-08-27T11:18:54.763 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:18:54.764 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:18:54.803 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:18:54.856 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:18:54.856 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:18:54.857 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:54.875 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:18:54.876 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:54.932 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:55.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.091+0000 7f7410aa6700 1 Processor -- start 2023-08-27T11:18:55.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.091+0000 7f7410aa6700 1 -- start start 2023-08-27T11:18:55.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f7410aa6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c06b230 0x7f740c0ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f7410aa6700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740c15eb60 con 0x7f740c06adb0 2023-08-27T11:18:55.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f7410aa6700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740c06b6c0 con 0x7f740c151060 2023-08-27T11:18:55.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f7410aa6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0b02f0 con 0x7f740c06b230 2023-08-27T11:18:55.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f7409d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f740c06adb0 0x7f740c0cdbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52578/0 (socket says 172.21.15.112:52578) 2023-08-27T11:18:55.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f7409d9b700 1 -- 172.21.15.112:0/2320862904 learned_addr learned my addr 172.21.15.112:0/2320862904 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:55.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740ad9d700 1 --2- 172.21.15.112:0/2320862904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c06b230 0x7f740c0ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917635534 0 0) 0x7f740c15eb60 con 0x7f740c06adb0 2023-08-27T11:18:55.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73f40035b0 con 0x7f740c06adb0 2023-08-27T11:18:55.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2194703626 0 0) 0x7f73f40035b0 con 0x7f740c06adb0 2023-08-27T11:18:55.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c06b230 msgr2=0x7f740c0ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:55.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 --2- 172.21.15.112:0/2320862904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c06b230 0x7f740c0ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 >> v1:172.21.15.138:6789/0 conn(0x7f740c151060 legacy=0x7f740c15e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c0b05d0 con 0x7f740c06adb0 2023-08-27T11:18:55.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f7400001670 con 0x7f740c06adb0 2023-08-27T11:18:55.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 >> v1:172.21.15.112:6789/0 conn(0x7f740c06adb0 legacy=0x7f740c0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 --2- 172.21.15.112:0/2320862904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f4003a30 0x7f73f4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 --2- 172.21.15.112:0/2320862904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c06b230 0x7f73f4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 --2- 172.21.15.112:0/2320862904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f4004d90 0x7f73f4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0b02f0 con 0x7f73f4003a30 2023-08-27T11:18:55.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f400ba10 con 0x7f73f4004d90 2023-08-27T11:18:55.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f400bbd0 con 0x7f740c06b230 2023-08-27T11:18:55.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.092+0000 7f740a59c700 1 --2- 172.21.15.112:0/2320862904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f4003a30 0x7f73f4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f7409d9b700 1 --2- 172.21.15.112:0/2320862904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c06b230 0x7f73f4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f740ad9d700 1 --2- 172.21.15.112:0/2320862904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f4004d90 0x7f73f4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f740a59c700 1 -- 172.21.15.112:0/2320862904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c06b230 msgr2=0x7f73f4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:55.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f740a59c700 1 --2- 172.21.15.112:0/2320862904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c06b230 0x7f73f4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f740a59c700 1 -- 172.21.15.112:0/2320862904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f4004d90 msgr2=0x7f73f4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:55.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f740a59c700 1 --2- 172.21.15.112:0/2320862904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f4004d90 0x7f73f4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f740a59c700 1 -- 172.21.15.112:0/2320862904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f40035b0 con 0x7f73f4003a30 2023-08-27T11:18:55.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f740ad9d700 1 --2- 172.21.15.112:0/2320862904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f4004d90 0x7f73f4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:55.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f740a59c700 1 --2- 172.21.15.112:0/2320862904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f4003a30 0x7f73f4003e10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f73fc001980 tx=0x7f73f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d076e107182813 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:55.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc016070 con 0x7f73f4003a30 2023-08-27T11:18:55.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc001cf0 con 0x7f73f4003a30 2023-08-27T11:18:55.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f740959a700 1 -- 172.21.15.112:0/2320862904 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc001420 con 0x7f73f4003a30 2023-08-27T11:18:55.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f7410aa6700 1 -- 172.21.15.112:0/2320862904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f4003a30 msgr2=0x7f73f4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f7410aa6700 1 --2- 172.21.15.112:0/2320862904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f4003a30 0x7f73f4003e10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f73fc001980 tx=0x7f73f4005880 comp rx=0 tx=0).stop 2023-08-27T11:18:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.093+0000 7f740ad9d700 1 -- 172.21.15.112:0/2320862904 reap_dead start 2023-08-27T11:18:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.094+0000 7f7410aa6700 1 -- 172.21.15.112:0/2320862904 shutdown_connections 2023-08-27T11:18:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.094+0000 7f7410aa6700 1 -- 172.21.15.112:0/2320862904 >> 172.21.15.112:0/2320862904 conn(0x7f740c0d0000 msgr2=0x7f740c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.094+0000 7f7410aa6700 1 -- 172.21.15.112:0/2320862904 shutdown_connections 2023-08-27T11:18:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.094+0000 7f7410aa6700 1 -- 172.21.15.112:0/2320862904 wait complete. 2023-08-27T11:18:55.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.095+0000 7f7410aa6700 1 Processor -- start 2023-08-27T11:18:55.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.095+0000 7f7410aa6700 1 -- start start 2023-08-27T11:18:55.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7410aa6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f4003a30 0x7f740c15a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7410aa6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f740c1f3960 0x7f740c14f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7410aa6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f740c1f0aa0 0x7f740c1f0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7410aa6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c1f4900 con 0x7f740c1f3960 2023-08-27T11:18:55.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7410aa6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c1f0330 con 0x7f740c1f0aa0 2023-08-27T11:18:55.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7410aa6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c1f3de0 con 0x7f73f4003a30 2023-08-27T11:18:55.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7409d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f740c1f3960 0x7f740c14f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7409d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f740c1f3960 0x7f740c14f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58208/0 (socket says 172.21.15.112:58208) 2023-08-27T11:18:55.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7409d9b700 1 -- 172.21.15.112:0/173096867 learned_addr learned my addr 172.21.15.112:0/173096867 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:55.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f740a59c700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f4003a30 0x7f740c15a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f740ad9d700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f740c1f0aa0 0x7f740c1f0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7409d9b700 1 -- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f4003a30 msgr2=0x7f740c15a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7409d9b700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f4003a30 0x7f740c15a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7409d9b700 1 -- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f740c1f0aa0 msgr2=0x7f740c1f0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7409d9b700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f740c1f0aa0 0x7f740c1f0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7409d9b700 1 -- 172.21.15.112:0/173096867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c1f4060 con 0x7f740c1f3960 2023-08-27T11:18:55.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7409d9b700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f740c1f3960 0x7f740c14f0d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7400007290 tx=0x7f74000075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:55.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f73faffd700 1 -- 172.21.15.112:0/173096867 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400011580 con 0x7f740c1f3960 2023-08-27T11:18:55.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f73faffd700 1 -- 172.21.15.112:0/173096867 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74000116e0 con 0x7f740c1f3960 2023-08-27T11:18:55.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f7410aa6700 1 -- 172.21.15.112:0/173096867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c1f4340 con 0x7f740c1f3960 2023-08-27T11:18:55.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.096+0000 7f73faffd700 1 -- 172.21.15.112:0/173096867 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400011a70 con 0x7f740c1f3960 2023-08-27T11:18:55.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.097+0000 7f7410aa6700 1 -- 172.21.15.112:0/173096867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c1f0580 con 0x7f740c1f3960 2023-08-27T11:18:55.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.097+0000 7f7410aa6700 1 -- 172.21.15.112:0/173096867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c1f07a0 con 0x7f740c1f3960 2023-08-27T11:18:55.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.098+0000 7f73faffd700 1 -- 172.21.15.112:0/173096867 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7400011bd0 con 0x7f740c1f3960 2023-08-27T11:18:55.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.098+0000 7f73faffd700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73ec06a110 0x7f73ec06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.098+0000 7f73faffd700 1 -- 172.21.15.112:0/173096867 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f7400087b30 con 0x7f740c1f3960 2023-08-27T11:18:55.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.098+0000 7f740a59c700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73ec06a110 0x7f73ec06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.099+0000 7f740a59c700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73ec06a110 0x7f73ec06c5d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f73fc014040 tx=0x7f73fc009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:55.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.103+0000 7f73faffd700 1 -- 172.21.15.112:0/173096867 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f74000531a0 con 0x7f740c1f3960 2023-08-27T11:18:55.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.280+0000 7f73faffd700 1 -- 172.21.15.112:0/173096867 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f7400053570 con 0x7f740c1f3960 2023-08-27T11:18:55.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.297+0000 7f7410aa6700 1 -- 172.21.15.112:0/173096867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f740c067a20 con 0x7f740c1f3960 2023-08-27T11:18:55.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.298+0000 7f73faffd700 1 -- 172.21.15.112:0/173096867 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18452 (secure 0 0 0) 0x7f7400056c30 con 0x7f740c1f3960 2023-08-27T11:18:55.300 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:55.300 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":53,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:54.265785+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:41.033680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:42.255772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1e","mappings":[{"from":7,"to":2}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]},{"pgid":"10.34","mappings":[{"from":7,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:55.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.300+0000 7f7410aa6700 1 -- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73ec06a110 msgr2=0x7f73ec06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.300+0000 7f7410aa6700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73ec06a110 0x7f73ec06c5d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f73fc014040 tx=0x7f73fc009030 comp rx=0 tx=0).stop 2023-08-27T11:18:55.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.300+0000 7f7410aa6700 1 -- 172.21.15.112:0/173096867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f740c1f3960 msgr2=0x7f740c14f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.300+0000 7f7410aa6700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f740c1f3960 0x7f740c14f0d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7400007290 tx=0x7f74000075a0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.300+0000 7f7410aa6700 1 -- 172.21.15.112:0/173096867 shutdown_connections 2023-08-27T11:18:55.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.300+0000 7f7410aa6700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73ec06a110 0x7f73ec06c5d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.300+0000 7f7410aa6700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f740c1f0aa0 0x7f740c1f0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.300+0000 7f7410aa6700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f4003a30 0x7f740c15a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.300+0000 7f7410aa6700 1 --2- 172.21.15.112:0/173096867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f740c1f3960 0x7f740c14f0d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.300+0000 7f7410aa6700 1 -- 172.21.15.112:0/173096867 >> 172.21.15.112:0/173096867 conn(0x7f740c0d0000 msgr2=0x7f740c06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:55.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.300+0000 7f7410aa6700 1 -- 172.21.15.112:0/173096867 shutdown_connections 2023-08-27T11:18:55.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.300+0000 7f7410aa6700 1 -- 172.21.15.112:0/173096867 wait complete. 2023-08-27T11:18:55.315 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:18:41.033680+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:18:42.255772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:18:55.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:18:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.438+0000 7fc1b27bb700 1 Processor -- start 2023-08-27T11:18:55.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.438+0000 7fc1b27bb700 1 -- start start 2023-08-27T11:18:55.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.438+0000 7fc1b27bb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc1ac143390 0x7fc1ac0723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.438+0000 7fc1b27bb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc1ac06f820 0x7fc1ac06ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1b27bb700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1ac06e740 con 0x7fc1ac06e2f0 2023-08-27T11:18:55.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1b27bb700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1ac070000 con 0x7fc1ac143390 2023-08-27T11:18:55.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1b27bb700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1ac0b16e0 con 0x7fc1ac06f820 2023-08-27T11:18:55.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1abfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc1ac143390 0x7fc1ac0723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1abfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc1ac143390 0x7fc1ac0723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58214/0 (socket says 172.21.15.112:58214) 2023-08-27T11:18:55.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1abfff700 1 -- 172.21.15.112:0/2249940193 learned_addr learned my addr 172.21.15.112:0/2249940193 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:55.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1ab7fe700 1 --2- 172.21.15.112:0/2249940193 >> v2:172.21.15.138:3300/0 conn(0x7fc1ac06f820 0x7fc1ac06ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1abfff700 1 -- 172.21.15.112:0/2249940193 >> v2:172.21.15.138:3300/0 conn(0x7fc1ac06f820 msgr2=0x7fc1ac06ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:55.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1abfff700 1 --2- 172.21.15.112:0/2249940193 >> v2:172.21.15.138:3300/0 conn(0x7fc1ac06f820 0x7fc1ac06ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1abfff700 1 -- 172.21.15.112:0/2249940193 >> v1:172.21.15.138:6789/0 conn(0x7fc1ac06e2f0 legacy=0x7fc1ac15d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:55.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1abfff700 1 -- 172.21.15.112:0/2249940193 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac0b0010 con 0x7fc1ac143390 2023-08-27T11:18:55.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1abfff700 1 --2- 172.21.15.112:0/2249940193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1ac143390 0x7fc1ac0723f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0aec60 tx=0x7fc19c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d8b373aa7bc72c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:55.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1aaffd700 1 -- 172.21.15.112:0/2249940193 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00dbe0 con 0x7fc1ac143390 2023-08-27T11:18:55.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1aaffd700 1 -- 172.21.15.112:0/2249940193 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c00dd40 con 0x7fc1ac143390 2023-08-27T11:18:55.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.439+0000 7fc1aaffd700 1 -- 172.21.15.112:0/2249940193 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c004270 con 0x7fc1ac143390 2023-08-27T11:18:55.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.440+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2249940193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1ac143390 msgr2=0x7fc1ac0723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.440+0000 7fc1b27bb700 1 --2- 172.21.15.112:0/2249940193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1ac143390 0x7fc1ac0723f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0aec60 tx=0x7fc19c00be70 comp rx=0 tx=0).stop 2023-08-27T11:18:55.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.440+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2249940193 shutdown_connections 2023-08-27T11:18:55.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.440+0000 7fc1b27bb700 1 --2- 172.21.15.112:0/2249940193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1ac143390 0x7fc1ac0723f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.440+0000 7fc1b27bb700 1 --2- 172.21.15.112:0/2249940193 >> v2:172.21.15.138:3300/0 conn(0x7fc1ac06f820 0x7fc1ac06ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.440+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2249940193 >> 172.21.15.112:0/2249940193 conn(0x7fc1ac0cc240 msgr2=0x7fc1ac0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:55.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.440+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2249940193 shutdown_connections 2023-08-27T11:18:55.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.440+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2249940193 wait complete. 2023-08-27T11:18:55.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.441+0000 7fc1b27bb700 1 Processor -- start 2023-08-27T11:18:55.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.442+0000 7fc1b27bb700 1 -- start start 2023-08-27T11:18:55.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.442+0000 7fc1b27bb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1ac06e2f0 0x7fc1ac0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.442+0000 7fc1b27bb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1ac06f820 0x7fc1ac1576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.442+0000 7fc1b27bb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1ac143390 0x7fc1ac13b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.442+0000 7fc1b27bb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac13ed40 con 0x7fc1ac143390 2023-08-27T11:18:55.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.442+0000 7fc1b27bb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac155e20 con 0x7fc1ac06f820 2023-08-27T11:18:55.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.442+0000 7fc1b27bb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac155b40 con 0x7fc1ac06e2f0 2023-08-27T11:18:55.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.442+0000 7fc1b0d58700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1ac143390 0x7fc1ac13b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1b0d58700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1ac143390 0x7fc1ac13b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58224/0 (socket says 172.21.15.112:58224) 2023-08-27T11:18:55.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1b0d58700 1 -- 172.21.15.112:0/2495554218 learned_addr learned my addr 172.21.15.112:0/2495554218 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:55.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1abfff700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1ac06e2f0 0x7fc1ac0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1ab7fe700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1ac06f820 0x7fc1ac1576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1b0d58700 1 -- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1ac06e2f0 msgr2=0x7fc1ac0adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1b0d58700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1ac06e2f0 0x7fc1ac0adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1b0d58700 1 -- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1ac06f820 msgr2=0x7fc1ac1576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1b0d58700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1ac06f820 0x7fc1ac1576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1b0d58700 1 -- 172.21.15.112:0/2495554218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c00b7f0 con 0x7fc1ac143390 2023-08-27T11:18:55.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1b0d58700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1ac143390 0x7fc1ac13b8c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400cc10 tx=0x7fc1a400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:55.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1a8ff9700 1 -- 172.21.15.112:0/2495554218 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a400aa60 con 0x7fc1ac143390 2023-08-27T11:18:55.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1a8ff9700 1 -- 172.21.15.112:0/2495554218 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a400abc0 con 0x7fc1ac143390 2023-08-27T11:18:55.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2495554218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ac140170 con 0x7fc1ac143390 2023-08-27T11:18:55.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1a8ff9700 1 -- 172.21.15.112:0/2495554218 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4012420 con 0x7fc1ac143390 2023-08-27T11:18:55.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.443+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2495554218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac1557f0 con 0x7fc1ac143390 2023-08-27T11:18:55.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.444+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2495554218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1ac067e90 con 0x7fc1ac143390 2023-08-27T11:18:55.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.444+0000 7fc1a8ff9700 1 -- 172.21.15.112:0/2495554218 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fc1a4012580 con 0x7fc1ac143390 2023-08-27T11:18:55.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.445+0000 7fc1a8ff9700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc19806a1a0 0x7fc19806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.445+0000 7fc1a8ff9700 1 -- 172.21.15.112:0/2495554218 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fc1a40887c0 con 0x7fc1ac143390 2023-08-27T11:18:55.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.445+0000 7fc1abfff700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc19806a1a0 0x7fc19806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.445+0000 7fc1abfff700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc19806a1a0 0x7fc19806c660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc19c019040 tx=0x7fc19c00bf90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:55.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.449+0000 7fc1a8ff9700 1 -- 172.21.15.112:0/2495554218 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc1a4053ed0 con 0x7fc1ac143390 2023-08-27T11:18:55.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.640+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2495554218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc1ac0b07b0 con 0x7fc1ac143390 2023-08-27T11:18:55.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.640+0000 7fc1a8ff9700 1 -- 172.21.15.112:0/2495554218 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc1a40574f0 con 0x7fc1ac143390 2023-08-27T11:18:55.642 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:18:55.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.642+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc19806a1a0 msgr2=0x7fc19806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.642+0000 7fc1b27bb700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc19806a1a0 0x7fc19806c660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc19c019040 tx=0x7fc19c00bf90 comp rx=0 tx=0).stop 2023-08-27T11:18:55.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.642+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2495554218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1ac143390 msgr2=0x7fc1ac13b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.642+0000 7fc1b27bb700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1ac143390 0x7fc1ac13b8c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400cc10 tx=0x7fc1a400cf20 comp rx=0 tx=0).stop 2023-08-27T11:18:55.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.642+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2495554218 shutdown_connections 2023-08-27T11:18:55.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.642+0000 7fc1b27bb700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc19806a1a0 0x7fc19806c660 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.642+0000 7fc1b27bb700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1ac143390 0x7fc1ac13b8c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.642+0000 7fc1b27bb700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1ac06e2f0 0x7fc1ac0adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.642+0000 7fc1b27bb700 1 --2- 172.21.15.112:0/2495554218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1ac06f820 0x7fc1ac1576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.642+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2495554218 >> 172.21.15.112:0/2495554218 conn(0x7fc1ac0cc240 msgr2=0x7fc1ac0aec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:55.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.642+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2495554218 shutdown_connections 2023-08-27T11:18:55.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.642+0000 7fc1b27bb700 1 -- 172.21.15.112:0/2495554218 wait complete. 2023-08-27T11:18:55.657 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:18:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.778+0000 7f277533d700 1 Processor -- start 2023-08-27T11:18:55.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.778+0000 7f277533d700 1 -- start start 2023-08-27T11:18:55.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f277533d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2770152130 0x7f277015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f277533d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27700b3cf0 0x7f27700b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f277533d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277015f750 con 0x7f277015cd60 2023-08-27T11:18:55.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f277533d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27700b46a0 con 0x7f2770152130 2023-08-27T11:18:55.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f277533d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277015fa30 con 0x7f27700b3cf0 2023-08-27T11:18:55.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f276e7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f277015cd60 0x7f277015ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52610/0 (socket says 172.21.15.112:52610) 2023-08-27T11:18:55.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f276e7fc700 1 -- 172.21.15.112:0/3235099313 learned_addr learned my addr 172.21.15.112:0/3235099313 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:55.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f276effd700 1 --2- 172.21.15.112:0/3235099313 >> v2:172.21.15.138:3300/0 conn(0x7f2770152130 0x7f277015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f276f7fe700 1 --2- 172.21.15.112:0/3235099313 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27700b3cf0 0x7f27700b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f276dffb700 1 -- 172.21.15.112:0/3235099313 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160760156 0 0) 0x7f277015f750 con 0x7f277015cd60 2023-08-27T11:18:55.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f276dffb700 1 -- 172.21.15.112:0/3235099313 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27580035b0 con 0x7f277015cd60 2023-08-27T11:18:55.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f276effd700 1 -- 172.21.15.112:0/3235099313 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27700b3cf0 msgr2=0x7f27700b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:55.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f276effd700 1 --2- 172.21.15.112:0/3235099313 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27700b3cf0 0x7f27700b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f276effd700 1 -- 172.21.15.112:0/3235099313 >> v1:172.21.15.112:6789/0 conn(0x7f277015cd60 legacy=0x7f277015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.779+0000 7f276dffb700 1 -- 172.21.15.112:0/3235099313 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2852740279 0 0) 0x7f27580035b0 con 0x7f277015cd60 2023-08-27T11:18:55.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.780+0000 7f276effd700 1 -- 172.21.15.112:0/3235099313 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27700b60a0 con 0x7f2770152130 2023-08-27T11:18:55.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.780+0000 7f276effd700 1 --2- 172.21.15.112:0/3235099313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2770152130 0x7f277015c820 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2764009840 tx=0x7f2764009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb45dfdc83c3ec3c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:55.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.780+0000 7f276dffb700 1 -- 172.21.15.112:0/3235099313 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2760001670 con 0x7f2770152130 2023-08-27T11:18:55.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.780+0000 7f276dffb700 1 -- 172.21.15.112:0/3235099313 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2764004030 con 0x7f2770152130 2023-08-27T11:18:55.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.780+0000 7f276dffb700 1 -- 172.21.15.112:0/3235099313 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27640043c0 con 0x7f2770152130 2023-08-27T11:18:55.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.780+0000 7f277533d700 1 -- 172.21.15.112:0/3235099313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2770152130 msgr2=0x7f277015c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.780+0000 7f277533d700 1 --2- 172.21.15.112:0/3235099313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2770152130 0x7f277015c820 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2764009840 tx=0x7f2764009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:55.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.780+0000 7f277533d700 1 -- 172.21.15.112:0/3235099313 shutdown_connections 2023-08-27T11:18:55.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.780+0000 7f277533d700 1 --2- 172.21.15.112:0/3235099313 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27700b3cf0 0x7f27700b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.780+0000 7f277533d700 1 --2- 172.21.15.112:0/3235099313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2770152130 0x7f277015c820 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.780+0000 7f277533d700 1 -- 172.21.15.112:0/3235099313 >> 172.21.15.112:0/3235099313 conn(0x7f2770066010 msgr2=0x7f27700663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:55.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.781+0000 7f277533d700 1 -- 172.21.15.112:0/3235099313 shutdown_connections 2023-08-27T11:18:55.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.781+0000 7f277533d700 1 -- 172.21.15.112:0/3235099313 wait complete. 2023-08-27T11:18:55.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.782+0000 7f277533d700 1 Processor -- start 2023-08-27T11:18:55.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.782+0000 7f277533d700 1 -- start start 2023-08-27T11:18:55.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f277533d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27700b3cf0 0x7f2770154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f277533d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2770152130 0x7f277014aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f277533d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f277015cd60 0x7f2770153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f277533d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277014dca0 con 0x7f27700b3cf0 2023-08-27T11:18:55.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f277533d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2770147450 con 0x7f277015cd60 2023-08-27T11:18:55.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f277533d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277014b3f0 con 0x7f2770152130 2023-08-27T11:18:55.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f276effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27700b3cf0 0x7f2770154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f276effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27700b3cf0 0x7f2770154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58234/0 (socket says 172.21.15.112:58234) 2023-08-27T11:18:55.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f276effd700 1 -- 172.21.15.112:0/3147093648 learned_addr learned my addr 172.21.15.112:0/3147093648 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:55.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f276e7fc700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2770152130 0x7f277014aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f276f7fe700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f277015cd60 0x7f2770153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f276effd700 1 -- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2770152130 msgr2=0x7f277014aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f276effd700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2770152130 0x7f277014aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f276effd700 1 -- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f277015cd60 msgr2=0x7f2770153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f276effd700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f277015cd60 0x7f2770153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f276effd700 1 -- 172.21.15.112:0/3147093648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27640094a0 con 0x7f27700b3cf0 2023-08-27T11:18:55.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f276f7fe700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f277015cd60 0x7f2770153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:55.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f276effd700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27700b3cf0 0x7f2770154f00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f27640090b0 tx=0x7f2764004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:55.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f27577fe700 1 -- 172.21.15.112:0/3147093648 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764004590 con 0x7f27700b3cf0 2023-08-27T11:18:55.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f27577fe700 1 -- 172.21.15.112:0/3147093648 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276401f070 con 0x7f27700b3cf0 2023-08-27T11:18:55.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f27577fe700 1 -- 172.21.15.112:0/3147093648 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764011870 con 0x7f27700b3cf0 2023-08-27T11:18:55.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f277533d700 1 -- 172.21.15.112:0/3147093648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2770154340 con 0x7f27700b3cf0 2023-08-27T11:18:55.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.783+0000 7f277533d700 1 -- 172.21.15.112:0/3147093648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277014d300 con 0x7f27700b3cf0 2023-08-27T11:18:55.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.784+0000 7f277533d700 1 -- 172.21.15.112:0/3147093648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277014dde0 con 0x7f27700b3cf0 2023-08-27T11:18:55.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.785+0000 7f27577fe700 1 -- 172.21.15.112:0/3147093648 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f27640119d0 con 0x7f27700b3cf0 2023-08-27T11:18:55.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.785+0000 7f27577fe700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f275c06a170 0x7f275c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:55.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.785+0000 7f27577fe700 1 -- 172.21.15.112:0/3147093648 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f2764021070 con 0x7f27700b3cf0 2023-08-27T11:18:55.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.785+0000 7f276e7fc700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f275c06a170 0x7f275c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:55.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.786+0000 7f276e7fc700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f275c06a170 0x7f275c06c630 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f27701527c0 tx=0x7f2760001be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:55.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.789+0000 7f27577fe700 1 -- 172.21.15.112:0/3147093648 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f27640555f0 con 0x7f27700b3cf0 2023-08-27T11:18:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.978+0000 7f277533d700 1 -- 172.21.15.112:0/3147093648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2770067a20 con 0x7f27700b3cf0 2023-08-27T11:18:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.978+0000 7f27577fe700 1 -- 172.21.15.112:0/3147093648 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f276402b020 con 0x7f27700b3cf0 2023-08-27T11:18:55.980 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:18:55.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.980+0000 7f277533d700 1 -- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f275c06a170 msgr2=0x7f275c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.980+0000 7f277533d700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f275c06a170 0x7f275c06c630 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f27701527c0 tx=0x7f2760001be0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.980+0000 7f277533d700 1 -- 172.21.15.112:0/3147093648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27700b3cf0 msgr2=0x7f2770154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:55.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.980+0000 7f277533d700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27700b3cf0 0x7f2770154f00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f27640090b0 tx=0x7f2764004750 comp rx=0 tx=0).stop 2023-08-27T11:18:55.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.980+0000 7f277533d700 1 -- 172.21.15.112:0/3147093648 shutdown_connections 2023-08-27T11:18:55.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.980+0000 7f277533d700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f275c06a170 0x7f275c06c630 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.980+0000 7f277533d700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f277015cd60 0x7f2770153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.980+0000 7f277533d700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27700b3cf0 0x7f2770154f00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.980+0000 7f277533d700 1 --2- 172.21.15.112:0/3147093648 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2770152130 0x7f277014aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:55.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.980+0000 7f277533d700 1 -- 172.21.15.112:0/3147093648 >> 172.21.15.112:0/3147093648 conn(0x7f2770066010 msgr2=0x7f277015de20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:55.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.980+0000 7f277533d700 1 -- 172.21.15.112:0/3147093648 shutdown_connections 2023-08-27T11:18:55.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:55.980+0000 7f277533d700 1 -- 172.21.15.112:0/3147093648 wait complete. 2023-08-27T11:18:55.994 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:18:56.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.117+0000 7f0c8ba79700 1 Processor -- start 2023-08-27T11:18:56.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.117+0000 7f0c8ba79700 1 -- start start 2023-08-27T11:18:56.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.118+0000 7f0c8ba79700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0c84146c20 0x7f0c84160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.118+0000 7f0c8ba79700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c840b3440 0x7f0c8415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.118+0000 7f0c8ba79700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0c8415e6e0 0x7f0c8406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.118+0000 7f0c8ba79700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c840b5070 con 0x7f0c8415e6e0 2023-08-27T11:18:56.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.118+0000 7f0c8ba79700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c84147000 con 0x7f0c84146c20 2023-08-27T11:18:56.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.118+0000 7f0c8ba79700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c841614a0 con 0x7f0c840b3440 2023-08-27T11:18:56.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.118+0000 7f0c8a016700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0c8415e6e0 0x7f0c8406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.118+0000 7f0c8a016700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0c8415e6e0 0x7f0c8406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58246/0 (socket says 172.21.15.112:58246) 2023-08-27T11:18:56.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.118+0000 7f0c8a016700 1 -- 172.21.15.112:0/3537293998 learned_addr learned my addr 172.21.15.112:0/3537293998 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:56.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.118+0000 7f0c89014700 1 --2- 172.21.15.112:0/3537293998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c840b3440 0x7f0c8415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.118+0000 7f0c89815700 1 --2- 172.21.15.112:0/3537293998 >> v2:172.21.15.138:3300/0 conn(0x7f0c84146c20 0x7f0c84160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c8a016700 1 -- 172.21.15.112:0/3537293998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c840b3440 msgr2=0x7f0c8415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:56.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c8a016700 1 --2- 172.21.15.112:0/3537293998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c840b3440 0x7f0c8415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c8a016700 1 -- 172.21.15.112:0/3537293998 >> v2:172.21.15.138:3300/0 conn(0x7f0c84146c20 msgr2=0x7f0c84160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c8a016700 1 --2- 172.21.15.112:0/3537293998 >> v2:172.21.15.138:3300/0 conn(0x7f0c84146c20 0x7f0c84160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c8a016700 1 -- 172.21.15.112:0/3537293998 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c84161740 con 0x7f0c8415e6e0 2023-08-27T11:18:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c89815700 1 --2- 172.21.15.112:0/3537293998 >> v2:172.21.15.138:3300/0 conn(0x7f0c84146c20 0x7f0c84160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c8a016700 1 --2- 172.21.15.112:0/3537293998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c8415e6e0 0x7f0c8406f790 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0c840b2d00 tx=0x7f0c7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77fc002aa327768 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c88813700 1 -- 172.21.15.112:0/3537293998 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00e070 con 0x7f0c8415e6e0 2023-08-27T11:18:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c88813700 1 -- 172.21.15.112:0/3537293998 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c00b910 con 0x7f0c8415e6e0 2023-08-27T11:18:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c88813700 1 -- 172.21.15.112:0/3537293998 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00bcc0 con 0x7f0c8415e6e0 2023-08-27T11:18:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c8ba79700 1 -- 172.21.15.112:0/3537293998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c8415e6e0 msgr2=0x7f0c8406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c8ba79700 1 --2- 172.21.15.112:0/3537293998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c8415e6e0 0x7f0c8406f790 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0c840b2d00 tx=0x7f0c7c00b330 comp rx=0 tx=0).stop 2023-08-27T11:18:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c8ba79700 1 -- 172.21.15.112:0/3537293998 shutdown_connections 2023-08-27T11:18:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c8ba79700 1 --2- 172.21.15.112:0/3537293998 >> v2:172.21.15.138:3300/0 conn(0x7f0c84146c20 0x7f0c84160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c8ba79700 1 --2- 172.21.15.112:0/3537293998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c8415e6e0 0x7f0c8406f790 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c8ba79700 1 --2- 172.21.15.112:0/3537293998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c840b3440 0x7f0c8415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.119+0000 7f0c8ba79700 1 -- 172.21.15.112:0/3537293998 >> 172.21.15.112:0/3537293998 conn(0x7f0c840b1e90 msgr2=0x7f0c840b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.120+0000 7f0c8ba79700 1 -- 172.21.15.112:0/3537293998 shutdown_connections 2023-08-27T11:18:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.120+0000 7f0c8ba79700 1 -- 172.21.15.112:0/3537293998 wait complete. 2023-08-27T11:18:56.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.121+0000 7f0c8ba79700 1 Processor -- start 2023-08-27T11:18:56.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.121+0000 7f0c8ba79700 1 -- start start 2023-08-27T11:18:56.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.121+0000 7f0c8ba79700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840b3440 0x7f0c84141040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.121+0000 7f0c8ba79700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c84146c20 0x7f0c84133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.121+0000 7f0c8ba79700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8415e6e0 0x7f0c840aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.121+0000 7f0c8ba79700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84140ce0 con 0x7f0c840b3440 2023-08-27T11:18:56.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.121+0000 7f0c8ba79700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84141580 con 0x7f0c8415e6e0 2023-08-27T11:18:56.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.121+0000 7f0c8ba79700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c841344b0 con 0x7f0c84146c20 2023-08-27T11:18:56.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c89815700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840b3440 0x7f0c84141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c89815700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840b3440 0x7f0c84141040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58248/0 (socket says 172.21.15.112:58248) 2023-08-27T11:18:56.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c89815700 1 -- 172.21.15.112:0/1553204889 learned_addr learned my addr 172.21.15.112:0/1553204889 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:56.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c89014700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c84146c20 0x7f0c84133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c8a016700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8415e6e0 0x7f0c840aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c89815700 1 -- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c84146c20 msgr2=0x7f0c84133f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c89815700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c84146c20 0x7f0c84133f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c89815700 1 -- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8415e6e0 msgr2=0x7f0c840aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c89815700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8415e6e0 0x7f0c840aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c89815700 1 -- 172.21.15.112:0/1553204889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c009b30 con 0x7f0c840b3440 2023-08-27T11:18:56.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c89014700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c84146c20 0x7f0c84133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:56.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c8a016700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8415e6e0 0x7f0c840aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:56.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c89815700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840b3440 0x7f0c84141040 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400e810 tx=0x7f0c7400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:56.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c7a7fc700 1 -- 172.21.15.112:0/1553204889 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74004230 con 0x7f0c840b3440 2023-08-27T11:18:56.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c7a7fc700 1 -- 172.21.15.112:0/1553204889 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c74004390 con 0x7f0c840b3440 2023-08-27T11:18:56.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c7a7fc700 1 -- 172.21.15.112:0/1553204889 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c740036a0 con 0x7f0c840b3440 2023-08-27T11:18:56.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c8ba79700 1 -- 172.21.15.112:0/1553204889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c84140a00 con 0x7f0c840b3440 2023-08-27T11:18:56.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.122+0000 7f0c8ba79700 1 -- 172.21.15.112:0/1553204889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c84144620 con 0x7f0c840b3440 2023-08-27T11:18:56.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.123+0000 7f0c8ba79700 1 -- 172.21.15.112:0/1553204889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c84067e90 con 0x7f0c840b3440 2023-08-27T11:18:56.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.124+0000 7f0c7a7fc700 1 -- 172.21.15.112:0/1553204889 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f0c74003800 con 0x7f0c840b3440 2023-08-27T11:18:56.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.124+0000 7f0c7a7fc700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0c6806a170 0x7f0c6806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.124+0000 7f0c7a7fc700 1 -- 172.21.15.112:0/1553204889 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f0c74088160 con 0x7f0c840b3440 2023-08-27T11:18:56.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.124+0000 7f0c89014700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0c6806a170 0x7f0c6806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.125+0000 7f0c89014700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0c6806a170 0x7f0c6806c630 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c841347f0 tx=0x7f0c80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:56.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.128+0000 7f0c7a7fc700 1 -- 172.21.15.112:0/1553204889 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0c74053760 con 0x7f0c840b3440 2023-08-27T11:18:56.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.317+0000 7f0c8ba79700 1 -- 172.21.15.112:0/1553204889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c841619f0 con 0x7f0c840b3440 2023-08-27T11:18:56.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.318+0000 7f0c7a7fc700 1 -- 172.21.15.112:0/1553204889 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c74056d80 con 0x7f0c840b3440 2023-08-27T11:18:56.320 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:18:56.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.320+0000 7f0c8ba79700 1 -- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0c6806a170 msgr2=0x7f0c6806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.320+0000 7f0c8ba79700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0c6806a170 0x7f0c6806c630 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c841347f0 tx=0x7f0c80008040 comp rx=0 tx=0).stop 2023-08-27T11:18:56.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.320+0000 7f0c8ba79700 1 -- 172.21.15.112:0/1553204889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840b3440 msgr2=0x7f0c84141040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.320+0000 7f0c8ba79700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840b3440 0x7f0c84141040 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400e810 tx=0x7f0c7400ebd0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.320+0000 7f0c8ba79700 1 -- 172.21.15.112:0/1553204889 shutdown_connections 2023-08-27T11:18:56.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.320+0000 7f0c8ba79700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0c6806a170 0x7f0c6806c630 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.320+0000 7f0c8ba79700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8415e6e0 0x7f0c840aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.320+0000 7f0c8ba79700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840b3440 0x7f0c84141040 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.320+0000 7f0c8ba79700 1 --2- 172.21.15.112:0/1553204889 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c84146c20 0x7f0c84133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.320+0000 7f0c8ba79700 1 -- 172.21.15.112:0/1553204889 >> 172.21.15.112:0/1553204889 conn(0x7f0c840b1e90 msgr2=0x7f0c8406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:56.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.320+0000 7f0c8ba79700 1 -- 172.21.15.112:0/1553204889 shutdown_connections 2023-08-27T11:18:56.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.320+0000 7f0c8ba79700 1 -- 172.21.15.112:0/1553204889 wait complete. 2023-08-27T11:18:56.333 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:18:56.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.455+0000 7f80f2a9a700 1 Processor -- start 2023-08-27T11:18:56.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.455+0000 7f80f2a9a700 1 -- start start 2023-08-27T11:18:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.456+0000 7f80f2a9a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80ec146c20 0x7f80ec160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.456+0000 7f80f2a9a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f80ec06f790 0x7f80ec06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.456+0000 7f80f2a9a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ec0b5070 con 0x7f80ec0b3440 2023-08-27T11:18:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.456+0000 7f80f2a9a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80ec070160 con 0x7f80ec06f790 2023-08-27T11:18:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.456+0000 7f80f2a9a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec147000 con 0x7f80ec146c20 2023-08-27T11:18:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.456+0000 7f80f1037700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f80ec06f790 0x7f80ec06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.456+0000 7f80ebfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f80ec0b3440 0x7f80ec15e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52638/0 (socket says 172.21.15.112:52638) 2023-08-27T11:18:56.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.456+0000 7f80ebfff700 1 -- 172.21.15.112:0/724370710 learned_addr learned my addr 172.21.15.112:0/724370710 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:56.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.456+0000 7f80f0836700 1 --2- 172.21.15.112:0/724370710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80ec146c20 0x7f80ec160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80f1037700 1 -- 172.21.15.112:0/724370710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80ec146c20 msgr2=0x7f80ec160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:56.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80f1037700 1 --2- 172.21.15.112:0/724370710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80ec146c20 0x7f80ec160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80f1037700 1 -- 172.21.15.112:0/724370710 >> v1:172.21.15.112:6789/0 conn(0x7f80ec0b3440 legacy=0x7f80ec15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80eb7fe700 1 -- 172.21.15.112:0/724370710 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945938982 0 0) 0x7f80d8001110 con 0x7f80ec0b3440 2023-08-27T11:18:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80f1037700 1 -- 172.21.15.112:0/724370710 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ec1614a0 con 0x7f80ec06f790 2023-08-27T11:18:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80f1037700 1 --2- 172.21.15.112:0/724370710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80ec06f790 0x7f80ec06fc20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009d90 tx=0x7f80e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c767890ba27359 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80eb7fe700 1 -- 172.21.15.112:0/724370710 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e400e070 con 0x7f80ec06f790 2023-08-27T11:18:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80eb7fe700 1 -- 172.21.15.112:0/724370710 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e400bbd0 con 0x7f80ec06f790 2023-08-27T11:18:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80eb7fe700 1 -- 172.21.15.112:0/724370710 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4012430 con 0x7f80ec06f790 2023-08-27T11:18:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80f2a9a700 1 -- 172.21.15.112:0/724370710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80ec06f790 msgr2=0x7f80ec06fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80f2a9a700 1 --2- 172.21.15.112:0/724370710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80ec06f790 0x7f80ec06fc20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009d90 tx=0x7f80e400b330 comp rx=0 tx=0).stop 2023-08-27T11:18:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80f2a9a700 1 -- 172.21.15.112:0/724370710 shutdown_connections 2023-08-27T11:18:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80f2a9a700 1 --2- 172.21.15.112:0/724370710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80ec06f790 0x7f80ec06fc20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80f2a9a700 1 --2- 172.21.15.112:0/724370710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80ec146c20 0x7f80ec160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80f2a9a700 1 -- 172.21.15.112:0/724370710 >> 172.21.15.112:0/724370710 conn(0x7f80ec0b1e90 msgr2=0x7f80ec0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.457+0000 7f80f2a9a700 1 -- 172.21.15.112:0/724370710 shutdown_connections 2023-08-27T11:18:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.458+0000 7f80f2a9a700 1 -- 172.21.15.112:0/724370710 wait complete. 2023-08-27T11:18:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.459+0000 7f80f2a9a700 1 Processor -- start 2023-08-27T11:18:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.459+0000 7f80f2a9a700 1 -- start start 2023-08-27T11:18:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80f2a9a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80ec06f790 0x7f80ec15f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80f2a9a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80ec0b3440 0x7f80ec0adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80f2a9a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec146c20 0x7f80ec145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80f2a9a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec140160 con 0x7f80ec0b3440 2023-08-27T11:18:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80f2a9a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec1403b0 con 0x7f80ec146c20 2023-08-27T11:18:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80f2a9a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec140690 con 0x7f80ec06f790 2023-08-27T11:18:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80ebfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80ec0b3440 0x7f80ec0adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80ebfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80ec0b3440 0x7f80ec0adf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58264/0 (socket says 172.21.15.112:58264) 2023-08-27T11:18:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80ebfff700 1 -- 172.21.15.112:0/3020514000 learned_addr learned my addr 172.21.15.112:0/3020514000 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:56.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80f0836700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80ec06f790 0x7f80ec15f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80f1037700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec146c20 0x7f80ec145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80ebfff700 1 -- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80ec06f790 msgr2=0x7f80ec15f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80ebfff700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80ec06f790 0x7f80ec15f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80ebfff700 1 -- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec146c20 msgr2=0x7f80ec145390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80ebfff700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec146c20 0x7f80ec145390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80ebfff700 1 -- 172.21.15.112:0/3020514000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e4009a70 con 0x7f80ec0b3440 2023-08-27T11:18:56.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80f0836700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80ec06f790 0x7f80ec15f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:56.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80f1037700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec146c20 0x7f80ec145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:56.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80ebfff700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80ec0b3440 0x7f80ec0adf30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f80d800c730 tx=0x7f80d800caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80e97fa700 1 -- 172.21.15.112:0/3020514000 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8004c80 con 0x7f80ec0b3440 2023-08-27T11:18:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80e97fa700 1 -- 172.21.15.112:0/3020514000 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d8004e00 con 0x7f80ec0b3440 2023-08-27T11:18:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80f2a9a700 1 -- 172.21.15.112:0/3020514000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ec140970 con 0x7f80ec0b3440 2023-08-27T11:18:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80e97fa700 1 -- 172.21.15.112:0/3020514000 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8007310 con 0x7f80ec0b3440 2023-08-27T11:18:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.460+0000 7f80f2a9a700 1 -- 172.21.15.112:0/3020514000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ec140f30 con 0x7f80ec0b3440 2023-08-27T11:18:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.461+0000 7f80f2a9a700 1 -- 172.21.15.112:0/3020514000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ec067e90 con 0x7f80ec0b3440 2023-08-27T11:18:56.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.462+0000 7f80e97fa700 1 -- 172.21.15.112:0/3020514000 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f80d8007470 con 0x7f80ec0b3440 2023-08-27T11:18:56.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.462+0000 7f80e97fa700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f80d006a170 0x7f80d006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.462+0000 7f80e97fa700 1 -- 172.21.15.112:0/3020514000 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f80d80881d0 con 0x7f80ec0b3440 2023-08-27T11:18:56.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.462+0000 7f80f0836700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f80d006a170 0x7f80d006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.463+0000 7f80f0836700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f80d006a170 0x7f80d006c630 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009820 tx=0x7f80e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:56.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.467+0000 7f80e97fa700 1 -- 172.21.15.112:0/3020514000 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f80d8053850 con 0x7f80ec0b3440 2023-08-27T11:18:56.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.655+0000 7f80f2a9a700 1 -- 172.21.15.112:0/3020514000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80ec0b5070 con 0x7f80ec0b3440 2023-08-27T11:18:56.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.656+0000 7f80e97fa700 1 -- 172.21.15.112:0/3020514000 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80ec0b5070 con 0x7f80ec0b3440 2023-08-27T11:18:56.658 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:18:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.658+0000 7f80f2a9a700 1 -- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f80d006a170 msgr2=0x7f80d006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.658+0000 7f80f2a9a700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f80d006a170 0x7f80d006c630 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009820 tx=0x7f80e0008040 comp rx=0 tx=0).stop 2023-08-27T11:18:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.658+0000 7f80f2a9a700 1 -- 172.21.15.112:0/3020514000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80ec0b3440 msgr2=0x7f80ec0adf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.658+0000 7f80f2a9a700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80ec0b3440 0x7f80ec0adf30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f80d800c730 tx=0x7f80d800caf0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.658+0000 7f80f2a9a700 1 -- 172.21.15.112:0/3020514000 shutdown_connections 2023-08-27T11:18:56.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.658+0000 7f80f2a9a700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f80d006a170 0x7f80d006c630 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.658+0000 7f80f2a9a700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec146c20 0x7f80ec145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.658+0000 7f80f2a9a700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80ec06f790 0x7f80ec15f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.658+0000 7f80f2a9a700 1 --2- 172.21.15.112:0/3020514000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80ec0b3440 0x7f80ec0adf30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.658+0000 7f80f2a9a700 1 -- 172.21.15.112:0/3020514000 >> 172.21.15.112:0/3020514000 conn(0x7f80ec0b1e90 msgr2=0x7f80ec071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.658+0000 7f80f2a9a700 1 -- 172.21.15.112:0/3020514000 shutdown_connections 2023-08-27T11:18:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.658+0000 7f80f2a9a700 1 -- 172.21.15.112:0/3020514000 wait complete. 2023-08-27T11:18:56.672 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:18:56.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.792+0000 7f01b031a700 1 Processor -- start 2023-08-27T11:18:56.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.792+0000 7f01b031a700 1 -- start start 2023-08-27T11:18:56.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.793+0000 7f01b031a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01a80b1400 0x7f01a80cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.793+0000 7f01b031a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f01a80b17e0 0x7f01a806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.793+0000 7f01b031a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a8161850 con 0x7f01a8161ce0 2023-08-27T11:18:56.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.793+0000 7f01b031a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a806e7d0 con 0x7f01a80b17e0 2023-08-27T11:18:56.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.793+0000 7f01b031a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80cfc10 con 0x7f01a80b1400 2023-08-27T11:18:56.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ae8b7700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f01a8161ce0 0x7f01a806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52656/0 (socket says 172.21.15.112:52656) 2023-08-27T11:18:56.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ae8b7700 1 -- 172.21.15.112:0/3011578210 learned_addr learned my addr 172.21.15.112:0/3011578210 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:56.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ae0b6700 1 --2- 172.21.15.112:0/3011578210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01a80b1400 0x7f01a80cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad8b5700 1 --2- 172.21.15.112:0/3011578210 >> v2:172.21.15.138:3300/0 conn(0x7f01a80b17e0 0x7f01a806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022858808 0 0) 0x7f01a8161850 con 0x7f01a8161ce0 2023-08-27T11:18:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01940035b0 con 0x7f01a8161ce0 2023-08-27T11:18:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2930142220 0 0) 0x7f01940035b0 con 0x7f01a8161ce0 2023-08-27T11:18:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01a80b1400 msgr2=0x7f01a80cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:56.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 --2- 172.21.15.112:0/3011578210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01a80b1400 0x7f01a80cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 >> v2:172.21.15.138:3300/0 conn(0x7f01a80b17e0 msgr2=0x7f01a806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:56.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 --2- 172.21.15.112:0/3011578210 >> v2:172.21.15.138:3300/0 conn(0x7f01a80b17e0 0x7f01a806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a8161200 con 0x7f01a8161ce0 2023-08-27T11:18:56.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f01a00019f0 con 0x7f01a8161ce0 2023-08-27T11:18:56.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 >> v1:172.21.15.112:6789/0 conn(0x7f01a8161ce0 legacy=0x7f01a806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 --2- 172.21.15.112:0/3011578210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0194003a00 0x7f0194003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 --2- 172.21.15.112:0/3011578210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01a80b1400 0x7f0194004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 --2- 172.21.15.112:0/3011578210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0194004d90 0x7f0194005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a806e7d0 con 0x7f0194004d90 2023-08-27T11:18:56.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80cfc10 con 0x7f0194003a00 2023-08-27T11:18:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019400ba90 con 0x7f01a80b1400 2023-08-27T11:18:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ae8b7700 1 --2- 172.21.15.112:0/3011578210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0194004d90 0x7f0194005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ae0b6700 1 --2- 172.21.15.112:0/3011578210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0194003a00 0x7f0194003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.794+0000 7f01ad8b5700 1 --2- 172.21.15.112:0/3011578210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01a80b1400 0x7f0194004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01ae8b7700 1 -- 172.21.15.112:0/3011578210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01a80b1400 msgr2=0x7f0194004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01ae8b7700 1 --2- 172.21.15.112:0/3011578210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01a80b1400 0x7f0194004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01ae8b7700 1 -- 172.21.15.112:0/3011578210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0194003a00 msgr2=0x7f0194003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01ae8b7700 1 --2- 172.21.15.112:0/3011578210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0194003a00 0x7f0194003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01ae8b7700 1 -- 172.21.15.112:0/3011578210 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01940035b0 con 0x7f0194004d90 2023-08-27T11:18:56.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01ae8b7700 1 --2- 172.21.15.112:0/3011578210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0194004d90 0x7f0194005210 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f01a000a860 tx=0x7f0194005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c0555d1f48f586 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:56.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0012600 con 0x7f0194004d90 2023-08-27T11:18:56.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a0012760 con 0x7f0194004d90 2023-08-27T11:18:56.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01ad0b4700 1 -- 172.21.15.112:0/3011578210 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0012b10 con 0x7f0194004d90 2023-08-27T11:18:56.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01b031a700 1 -- 172.21.15.112:0/3011578210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0194004d90 msgr2=0x7f0194005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01b031a700 1 --2- 172.21.15.112:0/3011578210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0194004d90 0x7f0194005210 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f01a000a860 tx=0x7f0194005880 comp rx=0 tx=0).stop 2023-08-27T11:18:56.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01ae8b7700 1 -- 172.21.15.112:0/3011578210 reap_dead start 2023-08-27T11:18:56.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01b031a700 1 -- 172.21.15.112:0/3011578210 shutdown_connections 2023-08-27T11:18:56.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.795+0000 7f01b031a700 1 -- 172.21.15.112:0/3011578210 >> 172.21.15.112:0/3011578210 conn(0x7f01a80d0000 msgr2=0x7f01a80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:56.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.796+0000 7f01b031a700 1 -- 172.21.15.112:0/3011578210 shutdown_connections 2023-08-27T11:18:56.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.796+0000 7f01b031a700 1 -- 172.21.15.112:0/3011578210 wait complete. 2023-08-27T11:18:56.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.797+0000 7f01b031a700 1 Processor -- start 2023-08-27T11:18:56.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.797+0000 7f01b031a700 1 -- start start 2023-08-27T11:18:56.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.797+0000 7f01b031a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0194004d90 0x7f01a80c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.797+0000 7f01b031a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01a8144240 0x7f01a81446c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.797+0000 7f01b031a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01a815dbe0 0x7f01a815e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01b031a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81421b0 con 0x7f01a8144240 2023-08-27T11:18:56.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01b031a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a815e5a0 con 0x7f01a815dbe0 2023-08-27T11:18:56.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01b031a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80b0760 con 0x7f0194004d90 2023-08-27T11:18:56.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01ad8b5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01a8144240 0x7f01a81446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01ad8b5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01a8144240 0x7f01a81446c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58282/0 (socket says 172.21.15.112:58282) 2023-08-27T11:18:56.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01ad8b5700 1 -- 172.21.15.112:0/319888112 learned_addr learned my addr 172.21.15.112:0/319888112 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:56.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01ad8b5700 1 -- 172.21.15.112:0/319888112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0194004d90 msgr2=0x7f01a80c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01ad8b5700 1 --2- 172.21.15.112:0/319888112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0194004d90 0x7f01a80c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01ad8b5700 1 -- 172.21.15.112:0/319888112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01a815dbe0 msgr2=0x7f01a815e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:56.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01ad8b5700 1 --2- 172.21.15.112:0/319888112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01a815dbe0 0x7f01a815e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:56.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01ad8b5700 1 -- 172.21.15.112:0/319888112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a8142730 con 0x7f01a8144240 2023-08-27T11:18:56.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01ad8b5700 1 --2- 172.21.15.112:0/319888112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01a8144240 0x7f01a81446c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f01a4011aa0 tx=0x7f01a4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:56.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f019effd700 1 -- 172.21.15.112:0/319888112 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a400f960 con 0x7f01a8144240 2023-08-27T11:18:56.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f019effd700 1 -- 172.21.15.112:0/319888112 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a400fac0 con 0x7f01a8144240 2023-08-27T11:18:56.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01b031a700 1 -- 172.21.15.112:0/319888112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a8144d00 con 0x7f01a8144240 2023-08-27T11:18:56.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f019effd700 1 -- 172.21.15.112:0/319888112 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a400fe70 con 0x7f01a8144240 2023-08-27T11:18:56.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.798+0000 7f01b031a700 1 -- 172.21.15.112:0/319888112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a81423c0 con 0x7f01a8144240 2023-08-27T11:18:56.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.799+0000 7f01b031a700 1 -- 172.21.15.112:0/319888112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a8142a10 con 0x7f01a8144240 2023-08-27T11:18:56.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.799+0000 7f019effd700 1 -- 172.21.15.112:0/319888112 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f01a4014440 con 0x7f01a8144240 2023-08-27T11:18:56.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.800+0000 7f019effd700 1 --2- 172.21.15.112:0/319888112 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f018c06a170 0x7f018c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:56.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.800+0000 7f019effd700 1 -- 172.21.15.112:0/319888112 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f01a4088dd0 con 0x7f01a8144240 2023-08-27T11:18:56.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.800+0000 7f01ae0b6700 1 --2- 172.21.15.112:0/319888112 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f018c06a170 0x7f018c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:56.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.801+0000 7f01ae0b6700 1 --2- 172.21.15.112:0/319888112 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f018c06a170 0x7f018c06c630 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0198009b50 tx=0x7f0198009670 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:56.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:56.804+0000 7f019effd700 1 -- 172.21.15.112:0/319888112 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f01a4054450 con 0x7f01a8144240 2023-08-27T11:18:57.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.007+0000 7f01b031a700 1 -- 172.21.15.112:0/319888112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f01a815eb80 con 0x7f01a8144240 2023-08-27T11:18:57.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.007+0000 7f019effd700 1 -- 172.21.15.112:0/319888112 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) v1 ==== 80+0+0 (secure 0 0 0) 0x7f01a4057a70 con 0x7f01a8144240 2023-08-27T11:18:57.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.010+0000 7f01b031a700 1 -- 172.21.15.112:0/319888112 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f018c06a170 msgr2=0x7f018c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.010+0000 7f01b031a700 1 --2- 172.21.15.112:0/319888112 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f018c06a170 0x7f018c06c630 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0198009b50 tx=0x7f0198009670 comp rx=0 tx=0).stop 2023-08-27T11:18:57.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.010+0000 7f01b031a700 1 -- 172.21.15.112:0/319888112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01a8144240 msgr2=0x7f01a81446c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.010+0000 7f01b031a700 1 --2- 172.21.15.112:0/319888112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01a8144240 0x7f01a81446c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f01a4011aa0 tx=0x7f01a4011e60 comp rx=0 tx=0).stop 2023-08-27T11:18:57.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.010+0000 7f01b031a700 1 -- 172.21.15.112:0/319888112 shutdown_connections 2023-08-27T11:18:57.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.010+0000 7f01b031a700 1 --2- 172.21.15.112:0/319888112 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f018c06a170 0x7f018c06c630 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.010+0000 7f01b031a700 1 --2- 172.21.15.112:0/319888112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01a815dbe0 0x7f01a815e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.010+0000 7f01b031a700 1 --2- 172.21.15.112:0/319888112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0194004d90 0x7f01a80c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.010+0000 7f01b031a700 1 --2- 172.21.15.112:0/319888112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01a8144240 0x7f01a81446c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.010+0000 7f01b031a700 1 -- 172.21.15.112:0/319888112 >> 172.21.15.112:0/319888112 conn(0x7f01a80d0000 msgr2=0x7f01a80cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:57.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.010+0000 7f01b031a700 1 -- 172.21.15.112:0/319888112 shutdown_connections 2023-08-27T11:18:57.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.010+0000 7f01b031a700 1 -- 172.21.15.112:0/319888112 wait complete. 2023-08-27T11:18:57.017 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:18:57.023 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:18:57.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:57.024 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:18:57.076 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:18:57.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:57.077 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:18:57.094 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:18:57.095 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:57.118 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:18:57.118 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:57.177 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:57.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.299+0000 7fc392154700 1 Processor -- start 2023-08-27T11:18:57.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.299+0000 7fc392154700 1 -- start start 2023-08-27T11:18:57.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.299+0000 7fc392154700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc38c06adb0 0x7fc38c06b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.299+0000 7fc392154700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc38c0cdbd0 0x7fc38c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc392154700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38c15eb60 con 0x7fc38c151060 2023-08-27T11:18:57.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc392154700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc38c06b770 con 0x7fc38c06adb0 2023-08-27T11:18:57.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc392154700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c0b02f0 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc38affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc38c06adb0 0x7fc38c06b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc38affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc38c06adb0 0x7fc38c06b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58290/0 (socket says 172.21.15.112:58290) 2023-08-27T11:18:57.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc38affd700 1 -- 172.21.15.112:0/1306370957 learned_addr learned my addr 172.21.15.112:0/1306370957 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:57.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc38bfff700 1 --2- 172.21.15.112:0/1306370957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc38c0cdbd0 0x7fc38c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc38affd700 1 -- 172.21.15.112:0/1306370957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc38c0cdbd0 msgr2=0x7fc38c0ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:57.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc38affd700 1 --2- 172.21.15.112:0/1306370957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc38c0cdbd0 0x7fc38c0ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc38affd700 1 -- 172.21.15.112:0/1306370957 >> v1:172.21.15.138:6789/0 conn(0x7fc38c151060 legacy=0x7fc38c15e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc38affd700 1 -- 172.21.15.112:0/1306370957 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c0b05d0 con 0x7fc38c06adb0 2023-08-27T11:18:57.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc38affd700 1 --2- 172.21.15.112:0/1306370957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc38c06adb0 0x7fc38c06b230 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc38000bb90 tx=0x7fc38000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad71bcf756aee51 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:57.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc38a7fc700 1 -- 172.21.15.112:0/1306370957 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38000dd50 con 0x7fc38c06adb0 2023-08-27T11:18:57.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc38a7fc700 1 -- 172.21.15.112:0/1306370957 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc380010470 con 0x7fc38c06adb0 2023-08-27T11:18:57.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.300+0000 7fc38a7fc700 1 -- 172.21.15.112:0/1306370957 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380010800 con 0x7fc38c06adb0 2023-08-27T11:18:57.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.301+0000 7fc392154700 1 -- 172.21.15.112:0/1306370957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc38c06adb0 msgr2=0x7fc38c06b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.301+0000 7fc392154700 1 --2- 172.21.15.112:0/1306370957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc38c06adb0 0x7fc38c06b230 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc38000bb90 tx=0x7fc38000bea0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.301+0000 7fc392154700 1 -- 172.21.15.112:0/1306370957 shutdown_connections 2023-08-27T11:18:57.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.301+0000 7fc392154700 1 --2- 172.21.15.112:0/1306370957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc38c0cdbd0 0x7fc38c0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.301+0000 7fc392154700 1 --2- 172.21.15.112:0/1306370957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc38c06adb0 0x7fc38c06b230 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.301+0000 7fc392154700 1 -- 172.21.15.112:0/1306370957 >> 172.21.15.112:0/1306370957 conn(0x7fc38c0d0000 msgr2=0x7fc38c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:57.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.301+0000 7fc392154700 1 -- 172.21.15.112:0/1306370957 shutdown_connections 2023-08-27T11:18:57.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.301+0000 7fc392154700 1 -- 172.21.15.112:0/1306370957 wait complete. 2023-08-27T11:18:57.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.302+0000 7fc392154700 1 Processor -- start 2023-08-27T11:18:57.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.302+0000 7fc392154700 1 -- start start 2023-08-27T11:18:57.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc392154700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc38c06adb0 0x7fc38c154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc392154700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc38c0cdbd0 0x7fc38c14e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc392154700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc38c151060 0x7fc38c159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc392154700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c0ca110 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc392154700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c151c10 con 0x7fc38c151060 2023-08-27T11:18:57.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc392154700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c14e9c0 con 0x7fc38c06adb0 2023-08-27T11:18:57.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc38affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc38c0cdbd0 0x7fc38c14e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc38affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc38c0cdbd0 0x7fc38c14e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58296/0 (socket says 172.21.15.112:58296) 2023-08-27T11:18:57.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc38affd700 1 -- 172.21.15.112:0/1199499236 learned_addr learned my addr 172.21.15.112:0/1199499236 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:57.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc38b7fe700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc38c06adb0 0x7fc38c154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc38bfff700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc38c151060 0x7fc38c159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc38affd700 1 -- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc38c06adb0 msgr2=0x7fc38c154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc38affd700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc38c06adb0 0x7fc38c154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc38affd700 1 -- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc38c151060 msgr2=0x7fc38c159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.303+0000 7fc38affd700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc38c151060 0x7fc38c159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.304+0000 7fc38affd700 1 -- 172.21.15.112:0/1199499236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38000b7f0 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.304+0000 7fc38affd700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc38c0cdbd0 0x7fc38c14e480 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc38000bee0 tx=0x7fc380004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:57.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.304+0000 7fc373fff700 1 -- 172.21.15.112:0/1199499236 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380003ab0 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.304+0000 7fc373fff700 1 -- 172.21.15.112:0/1199499236 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc380003c30 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.304+0000 7fc373fff700 1 -- 172.21.15.112:0/1199499236 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3800105e0 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.304+0000 7fc392154700 1 -- 172.21.15.112:0/1199499236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38c0c9e30 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.304+0000 7fc392154700 1 -- 172.21.15.112:0/1199499236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38c1518a0 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.305+0000 7fc392154700 1 -- 172.21.15.112:0/1199499236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38c067a20 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.305+0000 7fc373fff700 1 -- 172.21.15.112:0/1199499236 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fc380019400 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.306+0000 7fc373fff700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc37806a170 0x7fc37806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.306+0000 7fc373fff700 1 -- 172.21.15.112:0/1199499236 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fc3800899d0 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.306+0000 7fc38b7fe700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc37806a170 0x7fc37806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.306+0000 7fc38b7fe700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc37806a170 0x7fc37806c630 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0af900 tx=0x7fc37c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:57.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.310+0000 7fc373fff700 1 -- 172.21.15.112:0/1199499236 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc380054fd0 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.499+0000 7fc392154700 1 -- 172.21.15.112:0/1199499236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc38c151d50 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.501+0000 7fc373fff700 1 -- 172.21.15.112:0/1199499236 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18452 (secure 0 0 0) 0x7fc3800585f0 con 0x7fc38c0cdbd0 2023-08-27T11:18:57.503 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:57.503 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":53,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:54.265785+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:41.033680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:42.255772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1e","mappings":[{"from":7,"to":2}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]},{"pgid":"10.34","mappings":[{"from":7,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:18:57.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.503+0000 7fc392154700 1 -- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc37806a170 msgr2=0x7fc37806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.503+0000 7fc392154700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc37806a170 0x7fc37806c630 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0af900 tx=0x7fc37c009040 comp rx=0 tx=0).stop 2023-08-27T11:18:57.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.503+0000 7fc392154700 1 -- 172.21.15.112:0/1199499236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc38c0cdbd0 msgr2=0x7fc38c14e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.503+0000 7fc392154700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc38c0cdbd0 0x7fc38c14e480 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc38000bee0 tx=0x7fc380004750 comp rx=0 tx=0).stop 2023-08-27T11:18:57.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.503+0000 7fc392154700 1 -- 172.21.15.112:0/1199499236 shutdown_connections 2023-08-27T11:18:57.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.503+0000 7fc392154700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc37806a170 0x7fc37806c630 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.503+0000 7fc392154700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc38c151060 0x7fc38c159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.503+0000 7fc392154700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc38c06adb0 0x7fc38c154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.503+0000 7fc392154700 1 --2- 172.21.15.112:0/1199499236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc38c0cdbd0 0x7fc38c14e480 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.503+0000 7fc392154700 1 -- 172.21.15.112:0/1199499236 >> 172.21.15.112:0/1199499236 conn(0x7fc38c0d0000 msgr2=0x7fc38c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:57.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.503+0000 7fc392154700 1 -- 172.21.15.112:0/1199499236 shutdown_connections 2023-08-27T11:18:57.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.503+0000 7fc392154700 1 -- 172.21.15.112:0/1199499236 wait complete. 2023-08-27T11:18:57.516 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:18:41.033680+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:18:42.255772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:18:57.517 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:18:57.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.639+0000 7f08dc3fd700 1 Processor -- start 2023-08-27T11:18:57.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.639+0000 7f08dc3fd700 1 -- start start 2023-08-27T11:18:57.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.640+0000 7f08dc3fd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f08d4146c20 0x7f08d4160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.640+0000 7f08dc3fd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08d406f790 0x7f08d406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.640+0000 7f08dc3fd700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d40b5070 con 0x7f08d40b3440 2023-08-27T11:18:57.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.640+0000 7f08dc3fd700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d4070180 con 0x7f08d4146c20 2023-08-27T11:18:57.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.640+0000 7f08dc3fd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d4147000 con 0x7f08d406f790 2023-08-27T11:18:57.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.640+0000 7f08d9998700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f08d40b3440 0x7f08d415e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52686/0 (socket says 172.21.15.112:52686) 2023-08-27T11:18:57.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.640+0000 7f08d9998700 1 -- 172.21.15.112:0/74642959 learned_addr learned my addr 172.21.15.112:0/74642959 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:57.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.640+0000 7f08da199700 1 --2- 172.21.15.112:0/74642959 >> v2:172.21.15.138:3300/0 conn(0x7f08d4146c20 0x7f08d4160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.640+0000 7f08da99a700 1 --2- 172.21.15.112:0/74642959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08d406f790 0x7f08d406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.640+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129611189 0 0) 0x7f08d0001110 con 0x7f08d40b3440 2023-08-27T11:18:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.640+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08c00035b0 con 0x7f08d40b3440 2023-08-27T11:18:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 564172554 0 0) 0x7f08c00035b0 con 0x7f08d40b3440 2023-08-27T11:18:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08d406f790 msgr2=0x7f08d406fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 --2- 172.21.15.112:0/74642959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08d406f790 0x7f08d406fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 >> v2:172.21.15.138:3300/0 conn(0x7f08d4146c20 msgr2=0x7f08d4160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 --2- 172.21.15.112:0/74642959 >> v2:172.21.15.138:3300/0 conn(0x7f08d4146c20 0x7f08d4160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d41614a0 con 0x7f08d40b3440 2023-08-27T11:18:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f08d0001800 con 0x7f08d40b3440 2023-08-27T11:18:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 >> v1:172.21.15.112:6789/0 conn(0x7f08d40b3440 legacy=0x7f08d415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 --2- 172.21.15.112:0/74642959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08d406f790 0x7f08c0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 --2- 172.21.15.112:0/74642959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c0003f80 0x7f08c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 --2- 172.21.15.112:0/74642959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c0004d90 0x7f08c0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d4070180 con 0x7f08c0004d90 2023-08-27T11:18:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d4147000 con 0x7f08c0003f80 2023-08-27T11:18:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c000ba60 con 0x7f08d406f790 2023-08-27T11:18:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9998700 1 --2- 172.21.15.112:0/74642959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c0003f80 0x7f08c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08da199700 1 --2- 172.21.15.112:0/74642959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08d406f790 0x7f08c0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08da99a700 1 --2- 172.21.15.112:0/74642959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c0004d90 0x7f08c0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9998700 1 -- 172.21.15.112:0/74642959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08d406f790 msgr2=0x7f08c0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9998700 1 --2- 172.21.15.112:0/74642959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08d406f790 0x7f08c0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9998700 1 -- 172.21.15.112:0/74642959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c0004d90 msgr2=0x7f08c0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9998700 1 --2- 172.21.15.112:0/74642959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c0004d90 0x7f08c0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9998700 1 -- 172.21.15.112:0/74642959 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00035b0 con 0x7f08c0003f80 2023-08-27T11:18:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08da99a700 1 --2- 172.21.15.112:0/74642959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c0004d90 0x7f08c0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:57.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.641+0000 7f08d9998700 1 --2- 172.21.15.112:0/74642959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c0003f80 0x7f08c0004850 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08d40b38f0 tx=0x7f08c0005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16da22963ac15c19 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:57.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.642+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d000ce00 con 0x7f08c0003f80 2023-08-27T11:18:57.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.642+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d0005960 con 0x7f08c0003f80 2023-08-27T11:18:57.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.642+0000 7f08d9197700 1 -- 172.21.15.112:0/74642959 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0005d10 con 0x7f08c0003f80 2023-08-27T11:18:57.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.642+0000 7f08dc3fd700 1 -- 172.21.15.112:0/74642959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c0003f80 msgr2=0x7f08c0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.642+0000 7f08dc3fd700 1 --2- 172.21.15.112:0/74642959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c0003f80 0x7f08c0004850 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08d40b38f0 tx=0x7f08c0005880 comp rx=0 tx=0).stop 2023-08-27T11:18:57.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.642+0000 7f08da99a700 1 -- 172.21.15.112:0/74642959 reap_dead start 2023-08-27T11:18:57.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.642+0000 7f08dc3fd700 1 -- 172.21.15.112:0/74642959 shutdown_connections 2023-08-27T11:18:57.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.642+0000 7f08dc3fd700 1 -- 172.21.15.112:0/74642959 >> 172.21.15.112:0/74642959 conn(0x7f08d40b1e90 msgr2=0x7f08d40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:57.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.642+0000 7f08dc3fd700 1 -- 172.21.15.112:0/74642959 shutdown_connections 2023-08-27T11:18:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.642+0000 7f08dc3fd700 1 -- 172.21.15.112:0/74642959 wait complete. 2023-08-27T11:18:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.643+0000 7f08dc3fd700 1 Processor -- start 2023-08-27T11:18:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.644+0000 7f08dc3fd700 1 -- start start 2023-08-27T11:18:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.644+0000 7f08dc3fd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08c0003f80 0x7f08d40c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.644+0000 7f08dc3fd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40ca260 0x7f08d40cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.644+0000 7f08dc3fd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08d40cb700 0x7f08d40c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.644+0000 7f08dc3fd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40cbb80 con 0x7f08d40ca260 2023-08-27T11:18:57.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.644+0000 7f08dc3fd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40c6550 con 0x7f08d40cb700 2023-08-27T11:18:57.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.644+0000 7f08dc3fd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40ca7c0 con 0x7f08c0003f80 2023-08-27T11:18:57.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.644+0000 7f08d9998700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40ca260 0x7f08d40cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.644+0000 7f08d9998700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40ca260 0x7f08d40cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58316/0 (socket says 172.21.15.112:58316) 2023-08-27T11:18:57.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.644+0000 7f08d9998700 1 -- 172.21.15.112:0/2709944978 learned_addr learned my addr 172.21.15.112:0/2709944978 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:57.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.645+0000 7f08da99a700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08d40cb700 0x7f08d40c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.645+0000 7f08d9998700 1 -- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08c0003f80 msgr2=0x7f08d40c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.645+0000 7f08d9998700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08c0003f80 0x7f08d40c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.645+0000 7f08d9998700 1 -- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08d40cb700 msgr2=0x7f08d40c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.645+0000 7f08d9998700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08d40cb700 0x7f08d40c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.645+0000 7f08d9998700 1 -- 172.21.15.112:0/2709944978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d40caaa0 con 0x7f08d40ca260 2023-08-27T11:18:57.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.645+0000 7f08da99a700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08d40cb700 0x7f08d40c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:57.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.645+0000 7f08d9998700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40ca260 0x7f08d40cb1c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f08d0001280 tx=0x7f08d0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:57.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.645+0000 7f08caffd700 1 -- 172.21.15.112:0/2709944978 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0004870 con 0x7f08d40ca260 2023-08-27T11:18:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.645+0000 7f08caffd700 1 -- 172.21.15.112:0/2709944978 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d00049d0 con 0x7f08d40ca260 2023-08-27T11:18:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.645+0000 7f08dc3fd700 1 -- 172.21.15.112:0/2709944978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d40cad80 con 0x7f08d40ca260 2023-08-27T11:18:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.645+0000 7f08caffd700 1 -- 172.21.15.112:0/2709944978 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0004d80 con 0x7f08d40ca260 2023-08-27T11:18:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.645+0000 7f08dc3fd700 1 -- 172.21.15.112:0/2709944978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d413b070 con 0x7f08d40ca260 2023-08-27T11:18:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.646+0000 7f08dc3fd700 1 -- 172.21.15.112:0/2709944978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d40c6830 con 0x7f08d40ca260 2023-08-27T11:18:57.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.646+0000 7f08caffd700 1 -- 172.21.15.112:0/2709944978 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f08d0005040 con 0x7f08d40ca260 2023-08-27T11:18:57.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.646+0000 7f08caffd700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f08b806a1a0 0x7f08b806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.647+0000 7f08caffd700 1 -- 172.21.15.112:0/2709944978 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f08d0089e10 con 0x7f08d40ca260 2023-08-27T11:18:57.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.647+0000 7f08da199700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f08b806a1a0 0x7f08b806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.647+0000 7f08da199700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f08b806a1a0 0x7f08b806c660 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f08c4006f10 tx=0x7f08c4006e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:57.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.651+0000 7f08caffd700 1 -- 172.21.15.112:0/2709944978 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f08d00555c0 con 0x7f08d40ca260 2023-08-27T11:18:57.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.837+0000 7f08dc3fd700 1 -- 172.21.15.112:0/2709944978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f08d40b5070 con 0x7f08d40ca260 2023-08-27T11:18:57.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.837+0000 7f08caffd700 1 -- 172.21.15.112:0/2709944978 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7f08d0017020 con 0x7f08d40ca260 2023-08-27T11:18:57.839 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:18:57.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.840+0000 7f08dc3fd700 1 -- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f08b806a1a0 msgr2=0x7f08b806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.840+0000 7f08dc3fd700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f08b806a1a0 0x7f08b806c660 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f08c4006f10 tx=0x7f08c4006e80 comp rx=0 tx=0).stop 2023-08-27T11:18:57.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.840+0000 7f08dc3fd700 1 -- 172.21.15.112:0/2709944978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40ca260 msgr2=0x7f08d40cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.840+0000 7f08dc3fd700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40ca260 0x7f08d40cb1c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f08d0001280 tx=0x7f08d0000fe0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.840+0000 7f08dc3fd700 1 -- 172.21.15.112:0/2709944978 shutdown_connections 2023-08-27T11:18:57.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.840+0000 7f08dc3fd700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f08b806a1a0 0x7f08b806c660 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.840+0000 7f08dc3fd700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08d40cb700 0x7f08d40c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.840+0000 7f08dc3fd700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08c0003f80 0x7f08d40c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.840+0000 7f08dc3fd700 1 --2- 172.21.15.112:0/2709944978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40ca260 0x7f08d40cb1c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.840+0000 7f08dc3fd700 1 -- 172.21.15.112:0/2709944978 >> 172.21.15.112:0/2709944978 conn(0x7f08d40b1e90 msgr2=0x7f08d4071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:57.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.840+0000 7f08dc3fd700 1 -- 172.21.15.112:0/2709944978 shutdown_connections 2023-08-27T11:18:57.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.840+0000 7f08dc3fd700 1 -- 172.21.15.112:0/2709944978 wait complete. 2023-08-27T11:18:57.853 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:18:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.975+0000 7f90f1708700 1 Processor -- start 2023-08-27T11:18:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.975+0000 7f90f1708700 1 -- start start 2023-08-27T11:18:57.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.976+0000 7f90f1708700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90ec0aee70 0x7f90ec0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.976+0000 7f90f1708700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90ec15d9b0 0x7f90ec15de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.976+0000 7f90f1708700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90ec0af7f0 con 0x7f90ec15af20 2023-08-27T11:18:57.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.976+0000 7f90f1708700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90ec06bce0 con 0x7f90ec15d9b0 2023-08-27T11:18:57.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.976+0000 7f90f1708700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec161190 con 0x7f90ec0aee70 2023-08-27T11:18:57.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.976+0000 7f90eaffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f90ec15af20 0x7f90ec06b1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52704/0 (socket says 172.21.15.112:52704) 2023-08-27T11:18:57.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.976+0000 7f90eaffd700 1 -- 172.21.15.112:0/2719754506 learned_addr learned my addr 172.21.15.112:0/2719754506 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:57.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.976+0000 7f90ea7fc700 1 --2- 172.21.15.112:0/2719754506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90ec0aee70 0x7f90ec0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.976+0000 7f90eb7fe700 1 --2- 172.21.15.112:0/2719754506 >> v2:172.21.15.138:3300/0 conn(0x7f90ec15d9b0 0x7f90ec15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.976+0000 7f90e9ffb700 1 -- 172.21.15.112:0/2719754506 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466110417 0 0) 0x7f90ec0af7f0 con 0x7f90ec15af20 2023-08-27T11:18:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.977+0000 7f90e9ffb700 1 -- 172.21.15.112:0/2719754506 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90d40035b0 con 0x7f90ec15af20 2023-08-27T11:18:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.977+0000 7f90ea7fc700 1 -- 172.21.15.112:0/2719754506 >> v2:172.21.15.138:3300/0 conn(0x7f90ec15d9b0 msgr2=0x7f90ec15de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.977+0000 7f90ea7fc700 1 --2- 172.21.15.112:0/2719754506 >> v2:172.21.15.138:3300/0 conn(0x7f90ec15d9b0 0x7f90ec15de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.977+0000 7f90ea7fc700 1 -- 172.21.15.112:0/2719754506 >> v1:172.21.15.112:6789/0 conn(0x7f90ec15af20 legacy=0x7f90ec06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.977+0000 7f90ea7fc700 1 -- 172.21.15.112:0/2719754506 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec161470 con 0x7f90ec0aee70 2023-08-27T11:18:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.977+0000 7f90ea7fc700 1 --2- 172.21.15.112:0/2719754506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90ec0aee70 0x7f90ec0af2b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009840 tx=0x7f90dc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca5fcec806857de3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.977+0000 7f90e9ffb700 1 -- 172.21.15.112:0/2719754506 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc011830 con 0x7f90ec0aee70 2023-08-27T11:18:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.977+0000 7f90e9ffb700 1 -- 172.21.15.112:0/2719754506 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc011990 con 0x7f90ec0aee70 2023-08-27T11:18:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.977+0000 7f90e9ffb700 1 -- 172.21.15.112:0/2719754506 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc011d20 con 0x7f90ec0aee70 2023-08-27T11:18:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.978+0000 7f90f1708700 1 -- 172.21.15.112:0/2719754506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90ec0aee70 msgr2=0x7f90ec0af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.978+0000 7f90f1708700 1 --2- 172.21.15.112:0/2719754506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90ec0aee70 0x7f90ec0af2b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009840 tx=0x7f90dc009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:57.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.978+0000 7f90f1708700 1 -- 172.21.15.112:0/2719754506 shutdown_connections 2023-08-27T11:18:57.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.978+0000 7f90f1708700 1 --2- 172.21.15.112:0/2719754506 >> v2:172.21.15.138:3300/0 conn(0x7f90ec15d9b0 0x7f90ec15de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.978+0000 7f90f1708700 1 --2- 172.21.15.112:0/2719754506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90ec0aee70 0x7f90ec0af2b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.978+0000 7f90f1708700 1 -- 172.21.15.112:0/2719754506 >> 172.21.15.112:0/2719754506 conn(0x7f90ec0d03e0 msgr2=0x7f90ec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:57.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.978+0000 7f90f1708700 1 -- 172.21.15.112:0/2719754506 shutdown_connections 2023-08-27T11:18:57.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.978+0000 7f90f1708700 1 -- 172.21.15.112:0/2719754506 wait complete. 2023-08-27T11:18:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.979+0000 7f90f1708700 1 Processor -- start 2023-08-27T11:18:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.979+0000 7f90f1708700 1 -- start start 2023-08-27T11:18:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90f1708700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ec0aee70 0x7f90ec1f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90f1708700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90ec15af20 0x7f90ec154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90f1708700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90ec15d9b0 0x7f90ec156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90f1708700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec0ae0d0 con 0x7f90ec15af20 2023-08-27T11:18:57.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90f1708700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec0ad7a0 con 0x7f90ec0aee70 2023-08-27T11:18:57.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90f1708700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec0ada80 con 0x7f90ec15d9b0 2023-08-27T11:18:57.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90ea7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90ec15af20 0x7f90ec154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90ea7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90ec15af20 0x7f90ec154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58330/0 (socket says 172.21.15.112:58330) 2023-08-27T11:18:57.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90ea7fc700 1 -- 172.21.15.112:0/3763853477 learned_addr learned my addr 172.21.15.112:0/3763853477 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:57.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90eaffd700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ec0aee70 0x7f90ec1f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90eb7fe700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90ec15d9b0 0x7f90ec156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90ea7fc700 1 -- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90ec15d9b0 msgr2=0x7f90ec156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90ea7fc700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90ec15d9b0 0x7f90ec156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90ea7fc700 1 -- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ec0aee70 msgr2=0x7f90ec1f2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:57.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90ea7fc700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ec0aee70 0x7f90ec1f2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:57.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90ea7fc700 1 -- 172.21.15.112:0/3763853477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90dc0094a0 con 0x7f90ec15af20 2023-08-27T11:18:57.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90ea7fc700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90ec15af20 0x7f90ec154970 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f90dc0090b0 tx=0x7f90dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:57.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.980+0000 7f90d37fe700 1 -- 172.21.15.112:0/3763853477 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc004590 con 0x7f90ec15af20 2023-08-27T11:18:57.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.981+0000 7f90d37fe700 1 -- 172.21.15.112:0/3763853477 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc01f070 con 0x7f90ec15af20 2023-08-27T11:18:57.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.981+0000 7f90f1708700 1 -- 172.21.15.112:0/3763853477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ec0add60 con 0x7f90ec15af20 2023-08-27T11:18:57.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.981+0000 7f90d37fe700 1 -- 172.21.15.112:0/3763853477 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0038e0 con 0x7f90ec15af20 2023-08-27T11:18:57.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.981+0000 7f90f1708700 1 -- 172.21.15.112:0/3763853477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ec154eb0 con 0x7f90ec15af20 2023-08-27T11:18:57.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.981+0000 7f90f1708700 1 -- 172.21.15.112:0/3763853477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ec06da50 con 0x7f90ec15af20 2023-08-27T11:18:57.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.982+0000 7f90d37fe700 1 -- 172.21.15.112:0/3763853477 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f90dc003a40 con 0x7f90ec15af20 2023-08-27T11:18:57.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.982+0000 7f90d37fe700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f90d806a170 0x7f90d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:57.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.982+0000 7f90d37fe700 1 -- 172.21.15.112:0/3763853477 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f90dc021070 con 0x7f90ec15af20 2023-08-27T11:18:57.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.982+0000 7f90eaffd700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f90d806a170 0x7f90d806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:57.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.983+0000 7f90eaffd700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f90d806a170 0x7f90d806c630 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f90e0007fd0 tx=0x7f90e0007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:57.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:57.986+0000 7f90d37fe700 1 -- 172.21.15.112:0/3763853477 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f90dc055340 con 0x7f90ec15af20 2023-08-27T11:18:58.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.172+0000 7f90f1708700 1 -- 172.21.15.112:0/3763853477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f90ec067e90 con 0x7f90ec15af20 2023-08-27T11:18:58.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.173+0000 7f90d37fe700 1 -- 172.21.15.112:0/3763853477 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90dc058960 con 0x7f90ec15af20 2023-08-27T11:18:58.174 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:18:58.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.175+0000 7f90f1708700 1 -- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f90d806a170 msgr2=0x7f90d806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.175+0000 7f90f1708700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f90d806a170 0x7f90d806c630 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f90e0007fd0 tx=0x7f90e0007dc0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.175+0000 7f90f1708700 1 -- 172.21.15.112:0/3763853477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90ec15af20 msgr2=0x7f90ec154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.175+0000 7f90f1708700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90ec15af20 0x7f90ec154970 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f90dc0090b0 tx=0x7f90dc004750 comp rx=0 tx=0).stop 2023-08-27T11:18:58.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.175+0000 7f90f1708700 1 -- 172.21.15.112:0/3763853477 shutdown_connections 2023-08-27T11:18:58.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.175+0000 7f90f1708700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f90d806a170 0x7f90d806c630 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.175+0000 7f90f1708700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90ec15d9b0 0x7f90ec156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.175+0000 7f90f1708700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ec0aee70 0x7f90ec1f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.175+0000 7f90f1708700 1 --2- 172.21.15.112:0/3763853477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90ec15af20 0x7f90ec154970 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.175+0000 7f90f1708700 1 -- 172.21.15.112:0/3763853477 >> 172.21.15.112:0/3763853477 conn(0x7f90ec0d03e0 msgr2=0x7f90ec155ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:58.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.175+0000 7f90f1708700 1 -- 172.21.15.112:0/3763853477 shutdown_connections 2023-08-27T11:18:58.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.175+0000 7f90f1708700 1 -- 172.21.15.112:0/3763853477 wait complete. 2023-08-27T11:18:58.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:18:58.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.313+0000 7f8cdd5ca700 1 Processor -- start 2023-08-27T11:18:58.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.313+0000 7f8cdd5ca700 1 -- start start 2023-08-27T11:18:58.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.314+0000 7f8cdd5ca700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8cd81437c0 0x7f8cd80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cdd5ca700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cd806d5b0 0x7f8cd806da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cdd5ca700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd80cfd50 con 0x7f8cd8143ba0 2023-08-27T11:18:58.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cdd5ca700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd806df80 con 0x7f8cd81437c0 2023-08-27T11:18:58.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cdd5ca700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80cf140 con 0x7f8cd806d5b0 2023-08-27T11:18:58.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cd77fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cd806d5b0 0x7f8cd806da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cd6ffd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8cd81437c0 0x7f8cd80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cd67fc700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8cd8143ba0 0x7f8cd80ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:60242/0 (socket says 172.21.15.112:60242) 2023-08-27T11:18:58.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cd67fc700 1 -- 172.21.15.112:0/3621133919 learned_addr learned my addr 172.21.15.112:0/3621133919 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:58.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cd6ffd700 1 -- 172.21.15.112:0/3621133919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cd806d5b0 msgr2=0x7f8cd806da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:58.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cd6ffd700 1 --2- 172.21.15.112:0/3621133919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cd806d5b0 0x7f8cd806da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cd6ffd700 1 -- 172.21.15.112:0/3621133919 >> v1:172.21.15.138:6789/0 conn(0x7f8cd8143ba0 legacy=0x7f8cd80ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cd6ffd700 1 -- 172.21.15.112:0/3621133919 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd80cf420 con 0x7f8cd81437c0 2023-08-27T11:18:58.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cd5ffb700 1 -- 172.21.15.112:0/3621133919 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4279072048 0 0) 0x7f8cd80cfd50 con 0x7f8cd8143ba0 2023-08-27T11:18:58.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.315+0000 7f8cd77fe700 1 --2- 172.21.15.112:0/3621133919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cd806d5b0 0x7f8cd806da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:58.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.316+0000 7f8cd6ffd700 1 --2- 172.21.15.112:0/3621133919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd81437c0 0x7f8cd80b4670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009840 tx=0x7f8ccc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c439d6c251c20a3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:58.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.316+0000 7f8cd5ffb700 1 -- 172.21.15.112:0/3621133919 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc011830 con 0x7f8cd81437c0 2023-08-27T11:18:58.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.316+0000 7f8cd5ffb700 1 -- 172.21.15.112:0/3621133919 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc011990 con 0x7f8cd81437c0 2023-08-27T11:18:58.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.316+0000 7f8cd5ffb700 1 -- 172.21.15.112:0/3621133919 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc011d20 con 0x7f8cd81437c0 2023-08-27T11:18:58.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.316+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/3621133919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd81437c0 msgr2=0x7f8cd80b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.316+0000 7f8cdd5ca700 1 --2- 172.21.15.112:0/3621133919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd81437c0 0x7f8cd80b4670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009840 tx=0x7f8ccc009b50 comp rx=0 tx=0).stop 2023-08-27T11:18:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.316+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/3621133919 shutdown_connections 2023-08-27T11:18:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.316+0000 7f8cdd5ca700 1 --2- 172.21.15.112:0/3621133919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cd806d5b0 0x7f8cd806da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.316+0000 7f8cdd5ca700 1 --2- 172.21.15.112:0/3621133919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd81437c0 0x7f8cd80b4670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.316+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/3621133919 >> 172.21.15.112:0/3621133919 conn(0x7f8cd80b1e90 msgr2=0x7f8cd80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:58.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.316+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/3621133919 shutdown_connections 2023-08-27T11:18:58.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.316+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/3621133919 wait complete. 2023-08-27T11:18:58.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.318+0000 7f8cdd5ca700 1 Processor -- start 2023-08-27T11:18:58.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.318+0000 7f8cdd5ca700 1 -- start start 2023-08-27T11:18:58.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.318+0000 7f8cdd5ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cd806d5b0 0x7f8cd81391b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.318+0000 7f8cdd5ca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd81437c0 0x7f8cd815ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.318+0000 7f8cdd5ca700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cd8143ba0 0x7f8cd81595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.318+0000 7f8cdd5ca700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd813e840 con 0x7f8cd806d5b0 2023-08-27T11:18:58.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.318+0000 7f8cdd5ca700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd81396f0 con 0x7f8cd81437c0 2023-08-27T11:18:58.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.318+0000 7f8cdd5ca700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd813d230 con 0x7f8cd8143ba0 2023-08-27T11:18:58.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.318+0000 7f8cd6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cd806d5b0 0x7f8cd81391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.318+0000 7f8cd6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cd806d5b0 0x7f8cd81391b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58342/0 (socket says 172.21.15.112:58342) 2023-08-27T11:18:58.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.318+0000 7f8cd6ffd700 1 -- 172.21.15.112:0/2718069654 learned_addr learned my addr 172.21.15.112:0/2718069654 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:58.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cd77fe700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cd8143ba0 0x7f8cd81595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cd67fc700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd81437c0 0x7f8cd815ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cd6ffd700 1 -- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cd8143ba0 msgr2=0x7f8cd81595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cd6ffd700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cd8143ba0 0x7f8cd81595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cd6ffd700 1 -- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd81437c0 msgr2=0x7f8cd815ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cd6ffd700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd81437c0 0x7f8cd815ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cd6ffd700 1 -- 172.21.15.112:0/2718069654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd0009a70 con 0x7f8cd806d5b0 2023-08-27T11:18:58.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cd77fe700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cd8143ba0 0x7f8cd81595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:58.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cd67fc700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd81437c0 0x7f8cd815ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:18:58.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cd6ffd700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cd806d5b0 0x7f8cd81391b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009fd0 tx=0x7f8ccc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:58.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cbf7fe700 1 -- 172.21.15.112:0/2718069654 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc0036a0 con 0x7f8cd806d5b0 2023-08-27T11:18:58.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cbf7fe700 1 -- 172.21.15.112:0/2718069654 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc003820 con 0x7f8cd806d5b0 2023-08-27T11:18:58.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/2718069654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc0094a0 con 0x7f8cd806d5b0 2023-08-27T11:18:58.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cbf7fe700 1 -- 172.21.15.112:0/2718069654 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc003bd0 con 0x7f8cd806d5b0 2023-08-27T11:18:58.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.319+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/2718069654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd813d7f0 con 0x7f8cd806d5b0 2023-08-27T11:18:58.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.320+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/2718069654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd815a970 con 0x7f8cd806d5b0 2023-08-27T11:18:58.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.321+0000 7f8cbf7fe700 1 -- 172.21.15.112:0/2718069654 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f8ccc003d30 con 0x7f8cd806d5b0 2023-08-27T11:18:58.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.321+0000 7f8cbf7fe700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cc406a170 0x7f8cc406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.321+0000 7f8cbf7fe700 1 -- 172.21.15.112:0/2718069654 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f8ccc08b950 con 0x7f8cd806d5b0 2023-08-27T11:18:58.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.321+0000 7f8cd67fc700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cc406a170 0x7f8cc406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.322+0000 7f8cd67fc700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cc406a170 0x7f8cc406c630 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8cd81382e0 tx=0x7f8cc8007e50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:58.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.325+0000 7f8cbf7fe700 1 -- 172.21.15.112:0/2718069654 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8cd815a970 con 0x7f8cd806d5b0 2023-08-27T11:18:58.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.511+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/2718069654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8cd815a970 con 0x7f8cd806d5b0 2023-08-27T11:18:58.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.512+0000 7f8cbf7fe700 1 -- 172.21.15.112:0/2718069654 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8cd815a970 con 0x7f8cd806d5b0 2023-08-27T11:18:58.514 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:18:58.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.514+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cc406a170 msgr2=0x7f8cc406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.514+0000 7f8cdd5ca700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cc406a170 0x7f8cc406c630 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8cd81382e0 tx=0x7f8cc8007e50 comp rx=0 tx=0).stop 2023-08-27T11:18:58.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.514+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/2718069654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cd806d5b0 msgr2=0x7f8cd81391b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.514+0000 7f8cdd5ca700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cd806d5b0 0x7f8cd81391b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009fd0 tx=0x7f8ccc004750 comp rx=0 tx=0).stop 2023-08-27T11:18:58.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.514+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/2718069654 shutdown_connections 2023-08-27T11:18:58.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.514+0000 7f8cdd5ca700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cc406a170 0x7f8cc406c630 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.514+0000 7f8cdd5ca700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cd8143ba0 0x7f8cd81595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.514+0000 7f8cdd5ca700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cd806d5b0 0x7f8cd81391b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.514+0000 7f8cdd5ca700 1 --2- 172.21.15.112:0/2718069654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd81437c0 0x7f8cd815ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.514+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/2718069654 >> 172.21.15.112:0/2718069654 conn(0x7f8cd80b1e90 msgr2=0x7f8cd80720a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:58.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.514+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/2718069654 shutdown_connections 2023-08-27T11:18:58.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.514+0000 7f8cdd5ca700 1 -- 172.21.15.112:0/2718069654 wait complete. 2023-08-27T11:18:58.528 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:18:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.650+0000 7f2f1135e700 1 Processor -- start 2023-08-27T11:18:58.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.650+0000 7f2f1135e700 1 -- start start 2023-08-27T11:18:58.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f1135e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f0c146bf0 0x7f2f0c0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f1135e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f0c070eb0 0x7f2f0c071320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f1135e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f0c0d1500 con 0x7f2f0c146fd0 2023-08-27T11:18:58.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f1135e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0c071860 con 0x7f2f0c070eb0 2023-08-27T11:18:58.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f1135e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0c0d0460 con 0x7f2f0c146bf0 2023-08-27T11:18:58.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f0b7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f0c070eb0 0x7f2f0c071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f0b7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f0c070eb0 0x7f2f0c071320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58354/0 (socket says 172.21.15.112:58354) 2023-08-27T11:18:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f0b7fe700 1 -- 172.21.15.112:0/3631706259 learned_addr learned my addr 172.21.15.112:0/3631706259 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f0affd700 1 --2- 172.21.15.112:0/3631706259 >> v2:172.21.15.138:3300/0 conn(0x7f2f0c146bf0 0x7f2f0c0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f0b7fe700 1 -- 172.21.15.112:0/3631706259 >> v2:172.21.15.138:3300/0 conn(0x7f2f0c146bf0 msgr2=0x7f2f0c0b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f0b7fe700 1 --2- 172.21.15.112:0/3631706259 >> v2:172.21.15.138:3300/0 conn(0x7f2f0c146bf0 0x7f2f0c0b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f0b7fe700 1 -- 172.21.15.112:0/3631706259 >> v1:172.21.15.138:6789/0 conn(0x7f2f0c146fd0 legacy=0x7f2f0c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f0b7fe700 1 -- 172.21.15.112:0/3631706259 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c0d0710 con 0x7f2f0c070eb0 2023-08-27T11:18:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f0b7fe700 1 --2- 172.21.15.112:0/3631706259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c070eb0 0x7f2f0c071320 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c06e8f0 tx=0x7f2f0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40fe3eb6e3cce96 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f09ffb700 1 -- 172.21.15.112:0/3631706259 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400e070 con 0x7f2f0c070eb0 2023-08-27T11:18:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f09ffb700 1 -- 172.21.15.112:0/3631706259 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0400b910 con 0x7f2f0c070eb0 2023-08-27T11:18:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.651+0000 7f2f09ffb700 1 -- 172.21.15.112:0/3631706259 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400bcc0 con 0x7f2f0c070eb0 2023-08-27T11:18:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.652+0000 7f2f1135e700 1 -- 172.21.15.112:0/3631706259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c070eb0 msgr2=0x7f2f0c071320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.652+0000 7f2f1135e700 1 --2- 172.21.15.112:0/3631706259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c070eb0 0x7f2f0c071320 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c06e8f0 tx=0x7f2f0400b330 comp rx=0 tx=0).stop 2023-08-27T11:18:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.652+0000 7f2f1135e700 1 -- 172.21.15.112:0/3631706259 shutdown_connections 2023-08-27T11:18:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.652+0000 7f2f1135e700 1 --2- 172.21.15.112:0/3631706259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c070eb0 0x7f2f0c071320 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.652+0000 7f2f1135e700 1 --2- 172.21.15.112:0/3631706259 >> v2:172.21.15.138:3300/0 conn(0x7f2f0c146bf0 0x7f2f0c0b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.652+0000 7f2f1135e700 1 -- 172.21.15.112:0/3631706259 >> 172.21.15.112:0/3631706259 conn(0x7f2f0c06c450 msgr2=0x7f2f0c06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.652+0000 7f2f1135e700 1 -- 172.21.15.112:0/3631706259 shutdown_connections 2023-08-27T11:18:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.652+0000 7f2f1135e700 1 -- 172.21.15.112:0/3631706259 wait complete. 2023-08-27T11:18:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.653+0000 7f2f1135e700 1 Processor -- start 2023-08-27T11:18:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.653+0000 7f2f1135e700 1 -- start start 2023-08-27T11:18:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f1135e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f0c070eb0 0x7f2f0c15c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f1135e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f0c146bf0 0x7f2f0c0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f1135e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c146fd0 0x7f2f0c0af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f1135e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c0acf60 con 0x7f2f0c146fd0 2023-08-27T11:18:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f1135e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c0b22c0 con 0x7f2f0c070eb0 2023-08-27T11:18:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f1135e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c0b25a0 con 0x7f2f0c146bf0 2023-08-27T11:18:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f0b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c146fd0 0x7f2f0c0af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f0b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c146fd0 0x7f2f0c0af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58360/0 (socket says 172.21.15.112:58360) 2023-08-27T11:18:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f0b7fe700 1 -- 172.21.15.112:0/4041886197 learned_addr learned my addr 172.21.15.112:0/4041886197 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f0affd700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f0c070eb0 0x7f2f0c15c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f0a7fc700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f0c146bf0 0x7f2f0c0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f0b7fe700 1 -- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f0c146bf0 msgr2=0x7f2f0c0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f0b7fe700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f0c146bf0 0x7f2f0c0b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f0b7fe700 1 -- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f0c070eb0 msgr2=0x7f2f0c15c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f0b7fe700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f0c070eb0 0x7f2f0c15c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f0b7fe700 1 -- 172.21.15.112:0/4041886197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f04009b30 con 0x7f2f0c146fd0 2023-08-27T11:18:58.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.654+0000 7f2f0a7fc700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f0c146bf0 0x7f2f0c0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:58.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.655+0000 7f2f0b7fe700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c146fd0 0x7f2f0c0af920 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c06aea0 tx=0x7f2f04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:58.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.655+0000 7f2ef37fe700 1 -- 172.21.15.112:0/4041886197 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f040099b0 con 0x7f2f0c146fd0 2023-08-27T11:18:58.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.655+0000 7f2ef37fe700 1 -- 172.21.15.112:0/4041886197 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f040092e0 con 0x7f2f0c146fd0 2023-08-27T11:18:58.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.655+0000 7f2f1135e700 1 -- 172.21.15.112:0/4041886197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0c0ad1b0 con 0x7f2f0c146fd0 2023-08-27T11:18:58.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.655+0000 7f2ef37fe700 1 -- 172.21.15.112:0/4041886197 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0401c640 con 0x7f2f0c146fd0 2023-08-27T11:18:58.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.655+0000 7f2f1135e700 1 -- 172.21.15.112:0/4041886197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0c15c970 con 0x7f2f0c146fd0 2023-08-27T11:18:58.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.656+0000 7f2f1135e700 1 -- 172.21.15.112:0/4041886197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0c067e90 con 0x7f2f0c146fd0 2023-08-27T11:18:58.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.656+0000 7f2ef37fe700 1 -- 172.21.15.112:0/4041886197 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f2f0401c7a0 con 0x7f2f0c146fd0 2023-08-27T11:18:58.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.656+0000 7f2ef37fe700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ef806a170 0x7f2ef806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.657+0000 7f2ef37fe700 1 -- 172.21.15.112:0/4041886197 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f2f0408a3e0 con 0x7f2f0c146fd0 2023-08-27T11:18:58.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.657+0000 7f2f0affd700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ef806a170 0x7f2ef806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.657+0000 7f2f0affd700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ef806a170 0x7f2ef806c630 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009b70 tx=0x7f2f00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:58.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.661+0000 7f2ef37fe700 1 -- 172.21.15.112:0/4041886197 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2f040559e0 con 0x7f2f0c146fd0 2023-08-27T11:18:58.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.849+0000 7f2f1135e700 1 -- 172.21.15.112:0/4041886197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f0c0ad670 con 0x7f2f0c146fd0 2023-08-27T11:18:58.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.850+0000 7f2ef37fe700 1 -- 172.21.15.112:0/4041886197 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f04059000 con 0x7f2f0c146fd0 2023-08-27T11:18:58.852 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:18:58.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.852+0000 7f2f1135e700 1 -- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ef806a170 msgr2=0x7f2ef806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.852+0000 7f2f1135e700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ef806a170 0x7f2ef806c630 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009b70 tx=0x7f2f00008040 comp rx=0 tx=0).stop 2023-08-27T11:18:58.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.852+0000 7f2f1135e700 1 -- 172.21.15.112:0/4041886197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c146fd0 msgr2=0x7f2f0c0af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.852+0000 7f2f1135e700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c146fd0 0x7f2f0c0af920 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c06aea0 tx=0x7f2f04004ad0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.852+0000 7f2f1135e700 1 -- 172.21.15.112:0/4041886197 shutdown_connections 2023-08-27T11:18:58.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.852+0000 7f2f1135e700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ef806a170 0x7f2ef806c630 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.852+0000 7f2f1135e700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c146fd0 0x7f2f0c0af920 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.852+0000 7f2f1135e700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f0c070eb0 0x7f2f0c15c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.852+0000 7f2f1135e700 1 --2- 172.21.15.112:0/4041886197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f0c146bf0 0x7f2f0c0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.852+0000 7f2f1135e700 1 -- 172.21.15.112:0/4041886197 >> 172.21.15.112:0/4041886197 conn(0x7f2f0c06c450 msgr2=0x7f2f0c140fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:58.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.852+0000 7f2f1135e700 1 -- 172.21.15.112:0/4041886197 shutdown_connections 2023-08-27T11:18:58.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.852+0000 7f2f1135e700 1 -- 172.21.15.112:0/4041886197 wait complete. 2023-08-27T11:18:58.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:18:58.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.985+0000 7f1884942700 1 Processor -- start 2023-08-27T11:18:58.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.986+0000 7f1884942700 1 -- start start 2023-08-27T11:18:58.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.986+0000 7f1884942700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f188006e150 0x7f188006e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.986+0000 7f1884942700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18800a8960 0x7f18800a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:58.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.986+0000 7f1884942700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188015eed0 con 0x7f1880141420 2023-08-27T11:18:58.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.986+0000 7f1884942700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188006eb40 con 0x7f188006e150 2023-08-27T11:18:58.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.986+0000 7f1884942700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188015f1b0 con 0x7f18800a8960 2023-08-27T11:18:58.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.987+0000 7f187ed9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18800a8960 0x7f18800a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.987+0000 7f187dd9b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f188006e150 0x7f188006e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:58.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.987+0000 7f187dd9b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f188006e150 0x7f188006e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:58792/0 (socket says 172.21.15.112:58792) 2023-08-27T11:18:58.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.987+0000 7f187e59c700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1880141420 0x7f18800aaf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:60270/0 (socket says 172.21.15.112:60270) 2023-08-27T11:18:58.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.987+0000 7f187dd9b700 1 -- 172.21.15.112:0/1977368459 learned_addr learned my addr 172.21.15.112:0/1977368459 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:58.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.987+0000 7f187d59a700 1 -- 172.21.15.112:0/1977368459 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813921437 0 0) 0x7f188015eed0 con 0x7f1880141420 2023-08-27T11:18:58.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.987+0000 7f187ed9d700 1 -- 172.21.15.112:0/1977368459 >> v2:172.21.15.138:3300/0 conn(0x7f188006e150 msgr2=0x7f188006e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:58.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.987+0000 7f187ed9d700 1 --2- 172.21.15.112:0/1977368459 >> v2:172.21.15.138:3300/0 conn(0x7f188006e150 0x7f188006e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.987+0000 7f187ed9d700 1 -- 172.21.15.112:0/1977368459 >> v1:172.21.15.138:6789/0 conn(0x7f1880141420 legacy=0x7f18800aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.987+0000 7f187ed9d700 1 -- 172.21.15.112:0/1977368459 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188015f490 con 0x7f18800a8960 2023-08-27T11:18:58.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.987+0000 7f187dd9b700 1 --2- 172.21.15.112:0/1977368459 >> v2:172.21.15.138:3300/0 conn(0x7f188006e150 0x7f188006e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:18:58.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.987+0000 7f187ed9d700 1 --2- 172.21.15.112:0/1977368459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18800a8960 0x7f18800a8d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1878009d90 tx=0x7f187800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae5ebe6162d1acbc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:58.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.987+0000 7f187d59a700 1 -- 172.21.15.112:0/1977368459 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187800e070 con 0x7f18800a8960 2023-08-27T11:18:58.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.988+0000 7f187d59a700 1 -- 172.21.15.112:0/1977368459 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187800bb20 con 0x7f18800a8960 2023-08-27T11:18:58.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.988+0000 7f187d59a700 1 -- 172.21.15.112:0/1977368459 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878012430 con 0x7f18800a8960 2023-08-27T11:18:58.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.988+0000 7f1884942700 1 -- 172.21.15.112:0/1977368459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18800a8960 msgr2=0x7f18800a8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:58.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.988+0000 7f1884942700 1 --2- 172.21.15.112:0/1977368459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18800a8960 0x7f18800a8d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1878009d90 tx=0x7f187800b330 comp rx=0 tx=0).stop 2023-08-27T11:18:58.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.988+0000 7f1884942700 1 -- 172.21.15.112:0/1977368459 shutdown_connections 2023-08-27T11:18:58.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.988+0000 7f1884942700 1 --2- 172.21.15.112:0/1977368459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18800a8960 0x7f18800a8d90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.988+0000 7f1884942700 1 --2- 172.21.15.112:0/1977368459 >> v2:172.21.15.138:3300/0 conn(0x7f188006e150 0x7f188006e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:58.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.988+0000 7f1884942700 1 -- 172.21.15.112:0/1977368459 >> 172.21.15.112:0/1977368459 conn(0x7f188006cd30 msgr2=0x7f18800717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:58.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.988+0000 7f1884942700 1 -- 172.21.15.112:0/1977368459 shutdown_connections 2023-08-27T11:18:58.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.988+0000 7f1884942700 1 -- 172.21.15.112:0/1977368459 wait complete. 2023-08-27T11:18:59.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.989+0000 7f1884942700 1 Processor -- start 2023-08-27T11:18:59.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.990+0000 7f1884942700 1 -- start start 2023-08-27T11:18:59.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.990+0000 7f1884942700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f188006e150 0x7f18801563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.990+0000 7f1884942700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18800a8960 0x7f1880143d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.990+0000 7f1884942700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1880141420 0x7f18801f0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.990+0000 7f1884942700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18801568f0 con 0x7f1880141420 2023-08-27T11:18:59.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.990+0000 7f1884942700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1880135880 con 0x7f188006e150 2023-08-27T11:18:59.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.990+0000 7f1884942700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1880135b60 con 0x7f18800a8960 2023-08-27T11:18:59.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.990+0000 7f187ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1880141420 0x7f18801f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.990+0000 7f187ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1880141420 0x7f18801f0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58374/0 (socket says 172.21.15.112:58374) 2023-08-27T11:18:59.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.990+0000 7f187ed9d700 1 -- 172.21.15.112:0/583587479 learned_addr learned my addr 172.21.15.112:0/583587479 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:59.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.991+0000 7f187dd9b700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18800a8960 0x7f1880143d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.991+0000 7f187e59c700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f188006e150 0x7f18801563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.991+0000 7f187ed9d700 1 -- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18800a8960 msgr2=0x7f1880143d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.991+0000 7f187ed9d700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18800a8960 0x7f1880143d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.991+0000 7f187ed9d700 1 -- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f188006e150 msgr2=0x7f18801563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.991+0000 7f187ed9d700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f188006e150 0x7f18801563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.991+0000 7f187ed9d700 1 -- 172.21.15.112:0/583587479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1870009500 con 0x7f1880141420 2023-08-27T11:18:59.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.991+0000 7f187ed9d700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1880141420 0x7f18801f0810 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f188006ed70 tx=0x7f1878004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:59.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.991+0000 7f186effd700 1 -- 172.21.15.112:0/583587479 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878012430 con 0x7f1880141420 2023-08-27T11:18:59.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.991+0000 7f186effd700 1 -- 172.21.15.112:0/583587479 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1878020070 con 0x7f1880141420 2023-08-27T11:18:59.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.991+0000 7f1884942700 1 -- 172.21.15.112:0/583587479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1878009a70 con 0x7f1880141420 2023-08-27T11:18:59.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.991+0000 7f186effd700 1 -- 172.21.15.112:0/583587479 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878004280 con 0x7f1880141420 2023-08-27T11:18:59.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.991+0000 7f1884942700 1 -- 172.21.15.112:0/583587479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1880136120 con 0x7f1880141420 2023-08-27T11:18:59.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.992+0000 7f1884942700 1 -- 172.21.15.112:0/583587479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1880067a20 con 0x7f1880141420 2023-08-27T11:18:59.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.992+0000 7f186effd700 1 -- 172.21.15.112:0/583587479 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f18780043e0 con 0x7f1880141420 2023-08-27T11:18:59.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.993+0000 7f186effd700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f186006a170 0x7f186006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.993+0000 7f186effd700 1 -- 172.21.15.112:0/583587479 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f1878022070 con 0x7f1880141420 2023-08-27T11:18:59.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.993+0000 7f187e59c700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f186006a170 0x7f186006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.993+0000 7f187e59c700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f186006a170 0x7f186006c630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1874000f60 tx=0x7f1874007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:59.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:58.997+0000 7f186effd700 1 -- 172.21.15.112:0/583587479 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1878055e50 con 0x7f1880141420 2023-08-27T11:18:59.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.202+0000 7f1884942700 1 -- 172.21.15.112:0/583587479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1880071f30 con 0x7f1880141420 2023-08-27T11:18:59.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.202+0000 7f186effd700 1 -- 172.21.15.112:0/583587479 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1878059470 con 0x7f1880141420 2023-08-27T11:18:59.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.204+0000 7f1884942700 1 -- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f186006a170 msgr2=0x7f186006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.204+0000 7f1884942700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f186006a170 0x7f186006c630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1874000f60 tx=0x7f1874007c90 comp rx=0 tx=0).stop 2023-08-27T11:18:59.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.204+0000 7f1884942700 1 -- 172.21.15.112:0/583587479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1880141420 msgr2=0x7f18801f0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.204+0000 7f1884942700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1880141420 0x7f18801f0810 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f188006ed70 tx=0x7f1878004ad0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.204+0000 7f1884942700 1 -- 172.21.15.112:0/583587479 shutdown_connections 2023-08-27T11:18:59.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.204+0000 7f1884942700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f186006a170 0x7f186006c630 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.204+0000 7f1884942700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1880141420 0x7f18801f0810 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.204+0000 7f1884942700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f188006e150 0x7f18801563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.204+0000 7f1884942700 1 --2- 172.21.15.112:0/583587479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18800a8960 0x7f1880143d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.204+0000 7f1884942700 1 -- 172.21.15.112:0/583587479 >> 172.21.15.112:0/583587479 conn(0x7f188006cd30 msgr2=0x7f1880071610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:59.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.204+0000 7f1884942700 1 -- 172.21.15.112:0/583587479 shutdown_connections 2023-08-27T11:18:59.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.204+0000 7f1884942700 1 -- 172.21.15.112:0/583587479 wait complete. 2023-08-27T11:18:59.209 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:18:59.218 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-27T11:18:59.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:59.219 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-08-27T11:18:59.268 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:18:59.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:18:59.269 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-27T11:18:59.289 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:18:59.290 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:59.309 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:18:59.309 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:18:59.310 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:18:59.348 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:18:59.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:18:59.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.467+0000 7f70afeec700 1 Processor -- start 2023-08-27T11:18:59.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.467+0000 7f70afeec700 1 -- start start 2023-08-27T11:18:59.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.468+0000 7f70afeec700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f70a80b3e50 0x7f70a815f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.468+0000 7f70afeec700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a80b45f0 con 0x7f70a8144570 2023-08-27T11:18:59.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.468+0000 7f70afeec700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a80b4310 con 0x7f70a80b4ad0 2023-08-27T11:18:59.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.468+0000 7f70afeec700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a80cf7a0 con 0x7f70a80b3e50 2023-08-27T11:18:59.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.468+0000 7f70adc88700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f70a8144570 0x7f70a815ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52760/0 (socket says 172.21.15.112:52760) 2023-08-27T11:18:59.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.468+0000 7f70adc88700 1 -- 172.21.15.112:0/3760327647 learned_addr learned my addr 172.21.15.112:0/3760327647 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:18:59.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.468+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813032277 0 0) 0x7f70a80b45f0 con 0x7f70a8144570 2023-08-27T11:18:59.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.468+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70940035b0 con 0x7f70a8144570 2023-08-27T11:18:59.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.468+0000 7f70ad487700 1 --2- 172.21.15.112:0/3760327647 >> v2:172.21.15.138:3300/0 conn(0x7f70a80b3e50 0x7f70a815f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3700352971 0 0) 0x7f70940035b0 con 0x7f70a8144570 2023-08-27T11:18:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 >> v2:172.21.15.138:3300/0 conn(0x7f70a80b3e50 msgr2=0x7f70a815f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 --2- 172.21.15.112:0/3760327647 >> v2:172.21.15.138:3300/0 conn(0x7f70a80b3e50 0x7f70a815f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 >> v1:172.21.15.138:6789/0 conn(0x7f70a80b4ad0 legacy=0x7f70a80b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a80cdcb0 con 0x7f70a8144570 2023-08-27T11:18:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f7098001670 con 0x7f70a8144570 2023-08-27T11:18:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 >> v1:172.21.15.112:6789/0 conn(0x7f70a8144570 legacy=0x7f70a815ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 --2- 172.21.15.112:0/3760327647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7094003a00 0x7f7094003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 --2- 172.21.15.112:0/3760327647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7094004850 0x7f7094004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 --2- 172.21.15.112:0/3760327647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7094005210 0x7f709400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a80cf7a0 con 0x7f7094004850 2023-08-27T11:18:59.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709400c1b0 con 0x7f7094003a00 2023-08-27T11:18:59.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709400c370 con 0x7f7094005210 2023-08-27T11:18:59.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70ad487700 1 --2- 172.21.15.112:0/3760327647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7094004850 0x7f7094004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70adc88700 1 --2- 172.21.15.112:0/3760327647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7094003a00 0x7f7094003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70ae489700 1 --2- 172.21.15.112:0/3760327647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7094005210 0x7f709400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70ad487700 1 -- 172.21.15.112:0/3760327647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7094005210 msgr2=0x7f709400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:59.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70ad487700 1 --2- 172.21.15.112:0/3760327647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7094005210 0x7f709400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70ad487700 1 -- 172.21.15.112:0/3760327647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7094003a00 msgr2=0x7f7094003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:59.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70ad487700 1 --2- 172.21.15.112:0/3760327647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7094003a00 0x7f7094003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70ae489700 1 -- 172.21.15.112:0/3760327647 reap_dead start 2023-08-27T11:18:59.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70ad487700 1 -- 172.21.15.112:0/3760327647 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70940035b0 con 0x7f7094004850 2023-08-27T11:18:59.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70ad487700 1 --2- 172.21.15.112:0/3760327647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7094004850 0x7f7094004cd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009b60 tx=0x7f7094005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2e113698223896 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:59.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400e6c0 con 0x7f7094004850 2023-08-27T11:18:59.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.469+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a400e820 con 0x7f7094004850 2023-08-27T11:18:59.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.470+0000 7f70acc86700 1 -- 172.21.15.112:0/3760327647 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400eb90 con 0x7f7094004850 2023-08-27T11:18:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.470+0000 7f70afeec700 1 -- 172.21.15.112:0/3760327647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7094004850 msgr2=0x7f7094004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.470+0000 7f70afeec700 1 --2- 172.21.15.112:0/3760327647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7094004850 0x7f7094004cd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009b60 tx=0x7f7094005880 comp rx=0 tx=0).stop 2023-08-27T11:18:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.470+0000 7f70afeec700 1 -- 172.21.15.112:0/3760327647 shutdown_connections 2023-08-27T11:18:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.470+0000 7f70afeec700 1 --2- 172.21.15.112:0/3760327647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7094004850 0x7f7094004cd0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.470+0000 7f70afeec700 1 -- 172.21.15.112:0/3760327647 >> 172.21.15.112:0/3760327647 conn(0x7f70a806f970 msgr2=0x7f70a806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.470+0000 7f70afeec700 1 -- 172.21.15.112:0/3760327647 shutdown_connections 2023-08-27T11:18:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.470+0000 7f70afeec700 1 -- 172.21.15.112:0/3760327647 wait complete. 2023-08-27T11:18:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.471+0000 7f70afeec700 1 Processor -- start 2023-08-27T11:18:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.471+0000 7f70afeec700 1 -- start start 2023-08-27T11:18:59.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70afeec700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7094004850 0x7f70a815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70afeec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70a81f33c0 0x7f70a80ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70afeec700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70a80caac0 0x7f70a8142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70afeec700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a8072240 con 0x7f70a81f33c0 2023-08-27T11:18:59.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70afeec700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a8072520 con 0x7f70a80caac0 2023-08-27T11:18:59.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70afeec700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a81f2bc0 con 0x7f7094004850 2023-08-27T11:18:59.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70ad487700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70a81f33c0 0x7f70a80ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70ad487700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70a81f33c0 0x7f70a80ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58392/0 (socket says 172.21.15.112:58392) 2023-08-27T11:18:59.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70ad487700 1 -- 172.21.15.112:0/2010573705 learned_addr learned my addr 172.21.15.112:0/2010573705 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:59.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70adc88700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7094004850 0x7f70a815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70ae489700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70a80caac0 0x7f70a8142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70ad487700 1 -- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7094004850 msgr2=0x7f70a815ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70ad487700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7094004850 0x7f70a815ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70ad487700 1 -- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70a80caac0 msgr2=0x7f70a8142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70ad487700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70a80caac0 0x7f70a8142a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.472+0000 7f70ad487700 1 -- 172.21.15.112:0/2010573705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a81f2ea0 con 0x7f70a81f33c0 2023-08-27T11:18:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.473+0000 7f70ad487700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70a81f33c0 0x7f70a80ca580 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f709400c180 tx=0x7f70a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.473+0000 7f709e7fc700 1 -- 172.21.15.112:0/2010573705 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400e6c0 con 0x7f70a81f33c0 2023-08-27T11:18:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.473+0000 7f709e7fc700 1 -- 172.21.15.112:0/2010573705 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a4003ad0 con 0x7f70a81f33c0 2023-08-27T11:18:59.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.473+0000 7f709e7fc700 1 -- 172.21.15.112:0/2010573705 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4003e80 con 0x7f70a81f33c0 2023-08-27T11:18:59.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.473+0000 7f70afeec700 1 -- 172.21.15.112:0/2010573705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a8072800 con 0x7f70a81f33c0 2023-08-27T11:18:59.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.473+0000 7f70afeec700 1 -- 172.21.15.112:0/2010573705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a81f3840 con 0x7f70a81f33c0 2023-08-27T11:18:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.474+0000 7f70afeec700 1 -- 172.21.15.112:0/2010573705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a81f30e0 con 0x7f70a81f33c0 2023-08-27T11:18:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.474+0000 7f709e7fc700 1 -- 172.21.15.112:0/2010573705 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f70a4013470 con 0x7f70a81f33c0 2023-08-27T11:18:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.474+0000 7f709e7fc700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f708c06a1a0 0x7f708c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.475+0000 7f709e7fc700 1 -- 172.21.15.112:0/2010573705 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f70a4089e80 con 0x7f70a81f33c0 2023-08-27T11:18:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.475+0000 7f70adc88700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f708c06a1a0 0x7f708c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.475+0000 7f70adc88700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f708c06a1a0 0x7f708c06c660 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7098001470 tx=0x7f7098009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.479+0000 7f709e7fc700 1 -- 172.21.15.112:0/2010573705 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f70a81f30e0 con 0x7f70a81f33c0 2023-08-27T11:18:59.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.709+0000 7f70afeec700 1 -- 172.21.15.112:0/2010573705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70a81f30e0 con 0x7f70a81f33c0 2023-08-27T11:18:59.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.709+0000 7f709e7fc700 1 -- 172.21.15.112:0/2010573705 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4888 (secure 0 0 0) 0x7f70a4017020 con 0x7f70a81f33c0 2023-08-27T11:18:59.711 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:18:59.712 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1692518554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1692518554},{"type":"v1","addr":"172.21.15.138:6837","nonce":1692518554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3294671617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3294671617},{"type":"v1","addr":"172.21.15.138:6836","nonce":3294671617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3012780135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3012780135},{"type":"v1","addr":"172.21.15.112:6834","nonce":3012780135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:18:43.281470+0000","modified":"2023-08-27T11:18:47.234075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5661":{"gid":5661,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3423469954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3423469954},{"type":"v1","addr":"172.21.15.112:6835","nonce":3423469954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-27T11:18:59.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.711+0000 7f70afeec700 1 -- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f708c06a1a0 msgr2=0x7f708c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.711+0000 7f70afeec700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f708c06a1a0 0x7f708c06c660 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7098001470 tx=0x7f7098009040 comp rx=0 tx=0).stop 2023-08-27T11:18:59.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.711+0000 7f70afeec700 1 -- 172.21.15.112:0/2010573705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70a81f33c0 msgr2=0x7f70a80ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.711+0000 7f70afeec700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70a81f33c0 0x7f70a80ca580 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f709400c180 tx=0x7f70a40047c0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.711+0000 7f70afeec700 1 -- 172.21.15.112:0/2010573705 shutdown_connections 2023-08-27T11:18:59.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.711+0000 7f70afeec700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f708c06a1a0 0x7f708c06c660 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.711+0000 7f70afeec700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70a80caac0 0x7f70a8142a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.711+0000 7f70afeec700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7094004850 0x7f70a815ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.711+0000 7f70afeec700 1 --2- 172.21.15.112:0/2010573705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70a81f33c0 0x7f70a80ca580 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.711+0000 7f70afeec700 1 -- 172.21.15.112:0/2010573705 >> 172.21.15.112:0/2010573705 conn(0x7f70a806f970 msgr2=0x7f70a81412e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:59.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.711+0000 7f70afeec700 1 -- 172.21.15.112:0/2010573705 shutdown_connections 2023-08-27T11:18:59.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.711+0000 7f70afeec700 1 -- 172.21.15.112:0/2010573705 wait complete. 2023-08-27T11:18:59.718 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 31 2023-08-27T11:18:59.725 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:18:59.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.844+0000 7f66636b9700 1 Processor -- start 2023-08-27T11:18:59.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.844+0000 7f66636b9700 1 -- start start 2023-08-27T11:18:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f66636b9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f665c15a120 0x7f665c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f66636b9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c06da40 0x7f665c0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f66636b9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c06deb0 con 0x7f665c06d5b0 2023-08-27T11:18:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f66636b9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f665c15b540 con 0x7f665c15a120 2023-08-27T11:18:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f66636b9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0af830 con 0x7f665c06da40 2023-08-27T11:18:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f6661455700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f665c15a120 0x7f665c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f6661455700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f665c15a120 0x7f665c0b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58398/0 (socket says 172.21.15.112:58398) 2023-08-27T11:18:59.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f6661455700 1 -- 172.21.15.112:0/4109109352 learned_addr learned my addr 172.21.15.112:0/4109109352 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:59.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f6661c56700 1 --2- 172.21.15.112:0/4109109352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c06da40 0x7f665c0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f6661455700 1 -- 172.21.15.112:0/4109109352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c06da40 msgr2=0x7f665c0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:59.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f6661455700 1 --2- 172.21.15.112:0/4109109352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c06da40 0x7f665c0b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f6661455700 1 -- 172.21.15.112:0/4109109352 >> v1:172.21.15.138:6789/0 conn(0x7f665c06d5b0 legacy=0x7f665c0b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:18:59.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f6661455700 1 -- 172.21.15.112:0/4109109352 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c0afb10 con 0x7f665c15a120 2023-08-27T11:18:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f6661c56700 1 --2- 172.21.15.112:0/4109109352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c06da40 0x7f665c0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f6661455700 1 --2- 172.21.15.112:0/4109109352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c15a120 0x7f665c0b05f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f66580098f0 tx=0x7f6658009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bc453d4bdb34a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f6653fff700 1 -- 172.21.15.112:0/4109109352 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658004230 con 0x7f665c15a120 2023-08-27T11:18:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f6653fff700 1 -- 172.21.15.112:0/4109109352 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6658004390 con 0x7f665c15a120 2023-08-27T11:18:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.845+0000 7f6653fff700 1 -- 172.21.15.112:0/4109109352 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66580036a0 con 0x7f665c15a120 2023-08-27T11:18:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.846+0000 7f66636b9700 1 -- 172.21.15.112:0/4109109352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c15a120 msgr2=0x7f665c0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.846+0000 7f66636b9700 1 --2- 172.21.15.112:0/4109109352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c15a120 0x7f665c0b05f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f66580098f0 tx=0x7f6658009c00 comp rx=0 tx=0).stop 2023-08-27T11:18:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.846+0000 7f66636b9700 1 -- 172.21.15.112:0/4109109352 shutdown_connections 2023-08-27T11:18:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.846+0000 7f66636b9700 1 --2- 172.21.15.112:0/4109109352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c06da40 0x7f665c0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.846+0000 7f66636b9700 1 --2- 172.21.15.112:0/4109109352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c15a120 0x7f665c0b05f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.846+0000 7f66636b9700 1 -- 172.21.15.112:0/4109109352 >> 172.21.15.112:0/4109109352 conn(0x7f665c0cff80 msgr2=0x7f665c0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:18:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.846+0000 7f66636b9700 1 -- 172.21.15.112:0/4109109352 shutdown_connections 2023-08-27T11:18:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.846+0000 7f66636b9700 1 -- 172.21.15.112:0/4109109352 wait complete. 2023-08-27T11:18:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.848+0000 7f66636b9700 1 Processor -- start 2023-08-27T11:18:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.848+0000 7f66636b9700 1 -- start start 2023-08-27T11:18:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f66636b9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c06d5b0 0x7f665c0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f66636b9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c06da40 0x7f665c151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f66636b9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c15a120 0x7f665c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f66636b9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0cf180 con 0x7f665c06da40 2023-08-27T11:18:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f66636b9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0706e0 con 0x7f665c15a120 2023-08-27T11:18:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f66636b9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0ca5a0 con 0x7f665c06d5b0 2023-08-27T11:18:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6660c54700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c06da40 0x7f665c151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6660c54700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c06da40 0x7f665c151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58406/0 (socket says 172.21.15.112:58406) 2023-08-27T11:18:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6660c54700 1 -- 172.21.15.112:0/1113479655 learned_addr learned my addr 172.21.15.112:0/1113479655 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:18:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6661455700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c06d5b0 0x7f665c0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6661c56700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c15a120 0x7f665c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6660c54700 1 -- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c06d5b0 msgr2=0x7f665c0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6660c54700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c06d5b0 0x7f665c0c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6660c54700 1 -- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c15a120 msgr2=0x7f665c0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:18:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6660c54700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c15a120 0x7f665c0701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:18:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6660c54700 1 -- 172.21.15.112:0/1113479655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6658009550 con 0x7f665c06da40 2023-08-27T11:18:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6661c56700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c15a120 0x7f665c0701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:18:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6660c54700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c06da40 0x7f665c151c30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c4d0 tx=0x7f664c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6651ffb700 1 -- 172.21.15.112:0/1113479655 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c011790 con 0x7f665c06da40 2023-08-27T11:18:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6651ffb700 1 -- 172.21.15.112:0/1113479655 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c0118f0 con 0x7f665c06da40 2023-08-27T11:18:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f66636b9700 1 -- 172.21.15.112:0/1113479655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c0ca880 con 0x7f665c06da40 2023-08-27T11:18:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.849+0000 7f6651ffb700 1 -- 172.21.15.112:0/1113479655 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c011ca0 con 0x7f665c06da40 2023-08-27T11:18:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.850+0000 7f66636b9700 1 -- 172.21.15.112:0/1113479655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c0cae40 con 0x7f665c06da40 2023-08-27T11:18:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.851+0000 7f66636b9700 1 -- 172.21.15.112:0/1113479655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665c067a20 con 0x7f665c06da40 2023-08-27T11:18:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.851+0000 7f6651ffb700 1 -- 172.21.15.112:0/1113479655 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f664c01c460 con 0x7f665c06da40 2023-08-27T11:18:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.851+0000 7f6651ffb700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664006a170 0x7f664006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:18:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.851+0000 7f6651ffb700 1 -- 172.21.15.112:0/1113479655 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f664c088470 con 0x7f665c06da40 2023-08-27T11:18:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.851+0000 7f6661455700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664006a170 0x7f664006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:18:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.852+0000 7f6661455700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664006a170 0x7f664006c630 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6658009130 tx=0x7f6658008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:18:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:18:59.855+0000 7f6651ffb700 1 -- 172.21.15.112:0/1113479655 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f664c053af0 con 0x7f665c06da40 2023-08-27T11:19:00.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.049+0000 7f66636b9700 1 -- 172.21.15.112:0/1113479655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f665c0cc630 con 0x7f665c06da40 2023-08-27T11:19:00.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.051+0000 7f6651ffb700 1 -- 172.21.15.112:0/1113479655 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18452 (secure 0 0 0) 0x7f664c057110 con 0x7f665c06da40 2023-08-27T11:19:00.052 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:00.053 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":53,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:54.265785+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:41.033680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:42.255772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1e","mappings":[{"from":7,"to":2}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]},{"pgid":"10.34","mappings":[{"from":7,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:19:00.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.053+0000 7f66636b9700 1 -- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664006a170 msgr2=0x7f664006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.053+0000 7f66636b9700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664006a170 0x7f664006c630 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6658009130 tx=0x7f6658008030 comp rx=0 tx=0).stop 2023-08-27T11:19:00.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.053+0000 7f66636b9700 1 -- 172.21.15.112:0/1113479655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c06da40 msgr2=0x7f665c151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.053+0000 7f66636b9700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c06da40 0x7f665c151c30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c4d0 tx=0x7f664c00c890 comp rx=0 tx=0).stop 2023-08-27T11:19:00.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.053+0000 7f66636b9700 1 -- 172.21.15.112:0/1113479655 shutdown_connections 2023-08-27T11:19:00.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.053+0000 7f66636b9700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664006a170 0x7f664006c630 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.053+0000 7f66636b9700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c15a120 0x7f665c0701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.053+0000 7f66636b9700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c06d5b0 0x7f665c0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.053+0000 7f66636b9700 1 --2- 172.21.15.112:0/1113479655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c06da40 0x7f665c151c30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.053+0000 7f66636b9700 1 -- 172.21.15.112:0/1113479655 >> 172.21.15.112:0/1113479655 conn(0x7f665c0cff80 msgr2=0x7f665c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.053+0000 7f66636b9700 1 -- 172.21.15.112:0/1113479655 shutdown_connections 2023-08-27T11:19:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.053+0000 7f66636b9700 1 -- 172.21.15.112:0/1113479655 wait complete. 2023-08-27T11:19:00.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:18:41.033680+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:18:42.255772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:19:00.068 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:19:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.189+0000 7f6cf7432700 1 Processor -- start 2023-08-27T11:19:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.190+0000 7f6cf7432700 1 -- start start 2023-08-27T11:19:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.190+0000 7f6cf7432700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cf00b3de0 0x7f6cf006bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.190+0000 7f6cf7432700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf00cef20 con 0x7f6cf0144b80 2023-08-27T11:19:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.190+0000 7f6cf7432700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf006c940 con 0x7f6cf006c510 2023-08-27T11:19:00.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.190+0000 7f6cf7432700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00b4b40 con 0x7f6cf00b3de0 2023-08-27T11:19:00.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.190+0000 7f6cf51ce700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6cf0144b80 0x7f6cf00b6c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52794/0 (socket says 172.21.15.112:52794) 2023-08-27T11:19:00.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6cf51ce700 1 -- 172.21.15.112:0/363449549 learned_addr learned my addr 172.21.15.112:0/363449549 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:00.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2459884158 0 0) 0x7f6cf00cef20 con 0x7f6cf0144b80 2023-08-27T11:19:00.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cdc0035b0 con 0x7f6cf0144b80 2023-08-27T11:19:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6cf49cd700 1 --2- 172.21.15.112:0/363449549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cf00b3de0 0x7f6cf006bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1507973936 0 0) 0x7f6cdc0035b0 con 0x7f6cf0144b80 2023-08-27T11:19:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cf00b3de0 msgr2=0x7f6cf006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 --2- 172.21.15.112:0/363449549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cf00b3de0 0x7f6cf006bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 >> v1:172.21.15.138:6789/0 conn(0x7f6cf006c510 legacy=0x7f6cf00cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf00b7310 con 0x7f6cf0144b80 2023-08-27T11:19:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f6ce0001670 con 0x7f6cf0144b80 2023-08-27T11:19:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 >> v1:172.21.15.112:6789/0 conn(0x7f6cf0144b80 legacy=0x7f6cf00b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 --2- 172.21.15.112:0/363449549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc003a30 0x7f6cdc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 --2- 172.21.15.112:0/363449549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc004850 0x7f6cdc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 --2- 172.21.15.112:0/363449549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cf00b3de0 0x7f6cdc005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00b4b40 con 0x7f6cdc004850 2023-08-27T11:19:00.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc00bb70 con 0x7f6cdc003a30 2023-08-27T11:19:00.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc00bd30 con 0x7f6cf00b3de0 2023-08-27T11:19:00.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6cf49cd700 1 --2- 172.21.15.112:0/363449549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc004850 0x7f6cdc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6cf51ce700 1 --2- 172.21.15.112:0/363449549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc003a30 0x7f6cdc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6cf59cf700 1 --2- 172.21.15.112:0/363449549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cf00b3de0 0x7f6cdc005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6cf49cd700 1 -- 172.21.15.112:0/363449549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cf00b3de0 msgr2=0x7f6cdc005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:00.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6cf49cd700 1 --2- 172.21.15.112:0/363449549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cf00b3de0 0x7f6cdc005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6cf49cd700 1 -- 172.21.15.112:0/363449549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc003a30 msgr2=0x7f6cdc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:00.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6cf49cd700 1 --2- 172.21.15.112:0/363449549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc003a30 0x7f6cdc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.191+0000 7f6cf49cd700 1 -- 172.21.15.112:0/363449549 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc0035b0 con 0x7f6cdc004850 2023-08-27T11:19:00.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.192+0000 7f6cf49cd700 1 --2- 172.21.15.112:0/363449549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc004850 0x7f6cdc004cd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009b60 tx=0x7f6cdc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118d745e75705279 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:00.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.192+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec00e6c0 con 0x7f6cdc004850 2023-08-27T11:19:00.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.192+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec00e820 con 0x7f6cdc004850 2023-08-27T11:19:00.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.192+0000 7f6ce7fff700 1 -- 172.21.15.112:0/363449549 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec00eb90 con 0x7f6cdc004850 2023-08-27T11:19:00.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.192+0000 7f6cf7432700 1 -- 172.21.15.112:0/363449549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc004850 msgr2=0x7f6cdc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.192+0000 7f6cf7432700 1 --2- 172.21.15.112:0/363449549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc004850 0x7f6cdc004cd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009b60 tx=0x7f6cdc005880 comp rx=0 tx=0).stop 2023-08-27T11:19:00.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.192+0000 7f6cf59cf700 1 -- 172.21.15.112:0/363449549 reap_dead start 2023-08-27T11:19:00.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.192+0000 7f6cf7432700 1 -- 172.21.15.112:0/363449549 shutdown_connections 2023-08-27T11:19:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.192+0000 7f6cf7432700 1 -- 172.21.15.112:0/363449549 >> 172.21.15.112:0/363449549 conn(0x7f6cf00cdc50 msgr2=0x7f6cf00ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.192+0000 7f6cf7432700 1 -- 172.21.15.112:0/363449549 shutdown_connections 2023-08-27T11:19:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.192+0000 7f6cf7432700 1 -- 172.21.15.112:0/363449549 wait complete. 2023-08-27T11:19:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.194+0000 7f6cf7432700 1 Processor -- start 2023-08-27T11:19:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.194+0000 7f6cf7432700 1 -- start start 2023-08-27T11:19:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.194+0000 7f6cf7432700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc004850 0x7f6cf00afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.194+0000 7f6cf7432700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cf01f0c60 0x7f6cf00cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.194+0000 7f6cf7432700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cf01f10e0 0x7f6cf00a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.194+0000 7f6cf7432700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00ca9d0 con 0x7f6cf01f0c60 2023-08-27T11:19:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.194+0000 7f6cf7432700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00c9de0 con 0x7f6cdc004850 2023-08-27T11:19:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.194+0000 7f6cf7432700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00ca030 con 0x7f6cf01f10e0 2023-08-27T11:19:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf49cd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cf01f0c60 0x7f6cf00cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf49cd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cf01f0c60 0x7f6cf00cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58426/0 (socket says 172.21.15.112:58426) 2023-08-27T11:19:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf49cd700 1 -- 172.21.15.112:0/3808211389 learned_addr learned my addr 172.21.15.112:0/3808211389 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf51ce700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc004850 0x7f6cf00afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf59cf700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cf01f10e0 0x7f6cf00a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf49cd700 1 -- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cf01f10e0 msgr2=0x7f6cf00a91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf49cd700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cf01f10e0 0x7f6cf00a91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf49cd700 1 -- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc004850 msgr2=0x7f6cf00afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf49cd700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc004850 0x7f6cf00afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf49cd700 1 -- 172.21.15.112:0/3808211389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf00ca310 con 0x7f6cf01f0c60 2023-08-27T11:19:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf51ce700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc004850 0x7f6cf00afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf49cd700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cf01f0c60 0x7f6cf00cc920 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00bb40 tx=0x7f6cec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6ce5ffb700 1 -- 172.21.15.112:0/3808211389 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec00e6c0 con 0x7f6cf01f0c60 2023-08-27T11:19:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6ce5ffb700 1 -- 172.21.15.112:0/3808211389 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec003ab0 con 0x7f6cf01f0c60 2023-08-27T11:19:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6ce5ffb700 1 -- 172.21.15.112:0/3808211389 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec003e60 con 0x7f6cf01f0c60 2023-08-27T11:19:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf7432700 1 -- 172.21.15.112:0/3808211389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf00ca5f0 con 0x7f6cf01f0c60 2023-08-27T11:19:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.195+0000 7f6cf7432700 1 -- 172.21.15.112:0/3808211389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf00cbc60 con 0x7f6cf01f0c60 2023-08-27T11:19:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.196+0000 7f6cf7432700 1 -- 172.21.15.112:0/3808211389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf0067a20 con 0x7f6cf01f0c60 2023-08-27T11:19:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.197+0000 7f6ce5ffb700 1 -- 172.21.15.112:0/3808211389 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f6cec013470 con 0x7f6cf01f0c60 2023-08-27T11:19:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.197+0000 7f6ce5ffb700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cd406a170 0x7f6cd406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.197+0000 7f6ce5ffb700 1 -- 172.21.15.112:0/3808211389 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f6cec08a830 con 0x7f6cf01f0c60 2023-08-27T11:19:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.198+0000 7f6cf51ce700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cd406a170 0x7f6cd406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.198+0000 7f6cf51ce700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cd406a170 0x7f6cd406c630 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0001050 tx=0x7f6ce0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.201+0000 7f6ce5ffb700 1 -- 172.21.15.112:0/3808211389 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6cec0560e0 con 0x7f6cf01f0c60 2023-08-27T11:19:00.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.392+0000 7f6cf7432700 1 -- 172.21.15.112:0/3808211389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6cf00b4830 con 0x7f6cf01f0c60 2023-08-27T11:19:00.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.392+0000 7f6ce5ffb700 1 -- 172.21.15.112:0/3808211389 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6cec059700 con 0x7f6cf01f0c60 2023-08-27T11:19:00.394 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:19:00.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.394+0000 7f6cf7432700 1 -- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cd406a170 msgr2=0x7f6cd406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.394+0000 7f6cf7432700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cd406a170 0x7f6cd406c630 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0001050 tx=0x7f6ce0009040 comp rx=0 tx=0).stop 2023-08-27T11:19:00.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.394+0000 7f6cf7432700 1 -- 172.21.15.112:0/3808211389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cf01f0c60 msgr2=0x7f6cf00cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.394+0000 7f6cf7432700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cf01f0c60 0x7f6cf00cc920 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00bb40 tx=0x7f6cec0047c0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.394+0000 7f6cf7432700 1 -- 172.21.15.112:0/3808211389 shutdown_connections 2023-08-27T11:19:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.394+0000 7f6cf7432700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cd406a170 0x7f6cd406c630 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.394+0000 7f6cf7432700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cf01f10e0 0x7f6cf00a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.394+0000 7f6cf7432700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc004850 0x7f6cf00afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.394+0000 7f6cf7432700 1 --2- 172.21.15.112:0/3808211389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cf01f0c60 0x7f6cf00cc920 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.394+0000 7f6cf7432700 1 -- 172.21.15.112:0/3808211389 >> 172.21.15.112:0/3808211389 conn(0x7f6cf00cdc50 msgr2=0x7f6cf00b6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.394+0000 7f6cf7432700 1 -- 172.21.15.112:0/3808211389 shutdown_connections 2023-08-27T11:19:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.394+0000 7f6cf7432700 1 -- 172.21.15.112:0/3808211389 wait complete. 2023-08-27T11:19:00.407 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:19:00.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.530+0000 7fc4aef32700 1 Processor -- start 2023-08-27T11:19:00.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.530+0000 7fc4aef32700 1 -- start start 2023-08-27T11:19:00.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.530+0000 7fc4aef32700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4a8144b80 0x7fc4a80b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.530+0000 7fc4aef32700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc4a806c6e0 0x7fc4a80cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.530+0000 7fc4aef32700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a80cef20 con 0x7fc4a80b3de0 2023-08-27T11:19:00.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.530+0000 7fc4aef32700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a80b4b40 con 0x7fc4a8144b80 2023-08-27T11:19:00.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.530+0000 7fc4aef32700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a80b42a0 con 0x7fc4a806c6e0 2023-08-27T11:19:00.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.531+0000 7fc4accce700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4a8144b80 0x7fc4a80b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.531+0000 7fc4accce700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4a8144b80 0x7fc4a80b6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58432/0 (socket says 172.21.15.112:58432) 2023-08-27T11:19:00.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.531+0000 7fc4accce700 1 -- 172.21.15.112:0/1292303804 learned_addr learned my addr 172.21.15.112:0/1292303804 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:00.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.531+0000 7fc4ad4cf700 1 --2- 172.21.15.112:0/1292303804 >> v2:172.21.15.138:3300/0 conn(0x7fc4a806c6e0 0x7fc4a80cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.531+0000 7fc4accce700 1 -- 172.21.15.112:0/1292303804 >> v2:172.21.15.138:3300/0 conn(0x7fc4a806c6e0 msgr2=0x7fc4a80cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:00.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.531+0000 7fc4accce700 1 --2- 172.21.15.112:0/1292303804 >> v2:172.21.15.138:3300/0 conn(0x7fc4a806c6e0 0x7fc4a80cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.531+0000 7fc4accce700 1 -- 172.21.15.112:0/1292303804 >> v1:172.21.15.138:6789/0 conn(0x7fc4a80b3de0 legacy=0x7fc4a806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.531+0000 7fc4accce700 1 -- 172.21.15.112:0/1292303804 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a80b4580 con 0x7fc4a8144b80 2023-08-27T11:19:00.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.531+0000 7fc4accce700 1 --2- 172.21.15.112:0/1292303804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8144b80 0x7fc4a80b6c00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8074170 tx=0x7fc49c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ff6be2f80e2b30 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:00.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.531+0000 7fc4a77fe700 1 -- 172.21.15.112:0/1292303804 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00b870 con 0x7fc4a8144b80 2023-08-27T11:19:00.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.531+0000 7fc4a77fe700 1 -- 172.21.15.112:0/1292303804 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c00b9d0 con 0x7fc4a8144b80 2023-08-27T11:19:00.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.531+0000 7fc4a77fe700 1 -- 172.21.15.112:0/1292303804 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00bd60 con 0x7fc4a8144b80 2023-08-27T11:19:00.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.532+0000 7fc4aef32700 1 -- 172.21.15.112:0/1292303804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8144b80 msgr2=0x7fc4a80b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.532+0000 7fc4aef32700 1 --2- 172.21.15.112:0/1292303804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8144b80 0x7fc4a80b6c00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8074170 tx=0x7fc49c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.532+0000 7fc4aef32700 1 -- 172.21.15.112:0/1292303804 shutdown_connections 2023-08-27T11:19:00.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.532+0000 7fc4aef32700 1 --2- 172.21.15.112:0/1292303804 >> v2:172.21.15.138:3300/0 conn(0x7fc4a806c6e0 0x7fc4a80cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.532+0000 7fc4aef32700 1 --2- 172.21.15.112:0/1292303804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8144b80 0x7fc4a80b6c00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.532+0000 7fc4aef32700 1 -- 172.21.15.112:0/1292303804 >> 172.21.15.112:0/1292303804 conn(0x7fc4a80cdc50 msgr2=0x7fc4a80ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:00.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.532+0000 7fc4aef32700 1 -- 172.21.15.112:0/1292303804 shutdown_connections 2023-08-27T11:19:00.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.532+0000 7fc4aef32700 1 -- 172.21.15.112:0/1292303804 wait complete. 2023-08-27T11:19:00.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.533+0000 7fc4aef32700 1 Processor -- start 2023-08-27T11:19:00.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.533+0000 7fc4aef32700 1 -- start start 2023-08-27T11:19:00.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4aef32700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4a806c6e0 0x7fc4a8140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4aef32700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4a80b3de0 0x7fc4a80c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4aef32700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8144b80 0x7fc4a81425d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4aef32700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a80c8250 con 0x7fc4a8144b80 2023-08-27T11:19:00.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4aef32700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a8159ba0 con 0x7fc4a80b3de0 2023-08-27T11:19:00.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4aef32700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a8159e80 con 0x7fc4a806c6e0 2023-08-27T11:19:00.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4ad4cf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8144b80 0x7fc4a81425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4ad4cf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8144b80 0x7fc4a81425d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58442/0 (socket says 172.21.15.112:58442) 2023-08-27T11:19:00.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4ad4cf700 1 -- 172.21.15.112:0/1993876524 learned_addr learned my addr 172.21.15.112:0/1993876524 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:00.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4accce700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4a806c6e0 0x7fc4a8140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4a7fff700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4a80b3de0 0x7fc4a80c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4ad4cf700 1 -- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4a806c6e0 msgr2=0x7fc4a8140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4ad4cf700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4a806c6e0 0x7fc4a8140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4ad4cf700 1 -- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4a80b3de0 msgr2=0x7fc4a80c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4ad4cf700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4a80b3de0 0x7fc4a80c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4ad4cf700 1 -- 172.21.15.112:0/1993876524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c009550 con 0x7fc4a8144b80 2023-08-27T11:19:00.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4a7fff700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4a80b3de0 0x7fc4a80c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:00.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.534+0000 7fc4ad4cf700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8144b80 0x7fc4a81425d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0007e40 tx=0x7fc4a000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:00.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.535+0000 7fc4a57fa700 1 -- 172.21.15.112:0/1993876524 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a000ede0 con 0x7fc4a8144b80 2023-08-27T11:19:00.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.535+0000 7fc4a57fa700 1 -- 172.21.15.112:0/1993876524 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a0014420 con 0x7fc4a8144b80 2023-08-27T11:19:00.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.535+0000 7fc4a57fa700 1 -- 172.21.15.112:0/1993876524 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a00147d0 con 0x7fc4a8144b80 2023-08-27T11:19:00.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.535+0000 7fc4aef32700 1 -- 172.21.15.112:0/1993876524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a815a160 con 0x7fc4a8144b80 2023-08-27T11:19:00.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.535+0000 7fc4aef32700 1 -- 172.21.15.112:0/1993876524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a8140e70 con 0x7fc4a8144b80 2023-08-27T11:19:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.535+0000 7fc4aef32700 1 -- 172.21.15.112:0/1993876524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a815a680 con 0x7fc4a8144b80 2023-08-27T11:19:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.536+0000 7fc4a57fa700 1 -- 172.21.15.112:0/1993876524 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fc4a0014930 con 0x7fc4a8144b80 2023-08-27T11:19:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.536+0000 7fc4a57fa700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc48c06a170 0x7fc48c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.536+0000 7fc4a57fa700 1 -- 172.21.15.112:0/1993876524 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fc4a0088d20 con 0x7fc4a8144b80 2023-08-27T11:19:00.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.536+0000 7fc4accce700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc48c06a170 0x7fc48c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.537+0000 7fc4accce700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc48c06a170 0x7fc48c06c630 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc49c014040 tx=0x7fc49c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:00.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.541+0000 7fc4a57fa700 1 -- 172.21.15.112:0/1993876524 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc4a815a680 con 0x7fc4a8144b80 2023-08-27T11:19:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.728+0000 7fc4aef32700 1 -- 172.21.15.112:0/1993876524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4a815a680 con 0x7fc4a8144b80 2023-08-27T11:19:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.729+0000 7fc4a57fa700 1 -- 172.21.15.112:0/1993876524 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4a815a680 con 0x7fc4a8144b80 2023-08-27T11:19:00.731 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:19:00.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.731+0000 7fc4aef32700 1 -- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc48c06a170 msgr2=0x7fc48c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.731+0000 7fc4aef32700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc48c06a170 0x7fc48c06c630 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc49c014040 tx=0x7fc49c008030 comp rx=0 tx=0).stop 2023-08-27T11:19:00.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.731+0000 7fc4aef32700 1 -- 172.21.15.112:0/1993876524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8144b80 msgr2=0x7fc4a81425d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.731+0000 7fc4aef32700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8144b80 0x7fc4a81425d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0007e40 tx=0x7fc4a000e3c0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.731+0000 7fc4aef32700 1 -- 172.21.15.112:0/1993876524 shutdown_connections 2023-08-27T11:19:00.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.731+0000 7fc4aef32700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc48c06a170 0x7fc48c06c630 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.731+0000 7fc4aef32700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8144b80 0x7fc4a81425d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.731+0000 7fc4aef32700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4a806c6e0 0x7fc4a8140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.731+0000 7fc4aef32700 1 --2- 172.21.15.112:0/1993876524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4a80b3de0 0x7fc4a80c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.731+0000 7fc4aef32700 1 -- 172.21.15.112:0/1993876524 >> 172.21.15.112:0/1993876524 conn(0x7fc4a80cdc50 msgr2=0x7fc4a80c6200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:00.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.731+0000 7fc4aef32700 1 -- 172.21.15.112:0/1993876524 shutdown_connections 2023-08-27T11:19:00.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.731+0000 7fc4aef32700 1 -- 172.21.15.112:0/1993876524 wait complete. 2023-08-27T11:19:00.744 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:19:00.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.868+0000 7fae50c0a700 1 Processor -- start 2023-08-27T11:19:00.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.868+0000 7fae50c0a700 1 -- start start 2023-08-27T11:19:00.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.869+0000 7fae50c0a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fae4c146c20 0x7fae4c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.869+0000 7fae50c0a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae4c0b3440 0x7fae4c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.869+0000 7fae50c0a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fae4c15e6e0 0x7fae4c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.869+0000 7fae50c0a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4c0b5070 con 0x7fae4c146c20 2023-08-27T11:19:00.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.869+0000 7fae50c0a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4c147000 con 0x7fae4c15e6e0 2023-08-27T11:19:00.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.869+0000 7fae50c0a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c1614a0 con 0x7fae4c0b3440 2023-08-27T11:19:00.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.869+0000 7fae4a59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fae4c146c20 0x7fae4c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae4a59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fae4c146c20 0x7fae4c160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58446/0 (socket says 172.21.15.112:58446) 2023-08-27T11:19:00.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae4a59c700 1 -- 172.21.15.112:0/2674820792 learned_addr learned my addr 172.21.15.112:0/2674820792 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:00.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae49d9b700 1 --2- 172.21.15.112:0/2674820792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae4c0b3440 0x7fae4c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae4ad9d700 1 --2- 172.21.15.112:0/2674820792 >> v2:172.21.15.138:3300/0 conn(0x7fae4c15e6e0 0x7fae4c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae4a59c700 1 -- 172.21.15.112:0/2674820792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae4c0b3440 msgr2=0x7fae4c15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:00.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae4a59c700 1 --2- 172.21.15.112:0/2674820792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae4c0b3440 0x7fae4c15e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae4a59c700 1 -- 172.21.15.112:0/2674820792 >> v2:172.21.15.138:3300/0 conn(0x7fae4c15e6e0 msgr2=0x7fae4c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:00.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae4a59c700 1 --2- 172.21.15.112:0/2674820792 >> v2:172.21.15.138:3300/0 conn(0x7fae4c15e6e0 0x7fae4c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae4a59c700 1 -- 172.21.15.112:0/2674820792 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c161740 con 0x7fae4c146c20 2023-08-27T11:19:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae49d9b700 1 --2- 172.21.15.112:0/2674820792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae4c0b3440 0x7fae4c15e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae4a59c700 1 --2- 172.21.15.112:0/2674820792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c146c20 0x7fae4c160f60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0b1c80 tx=0x7fae40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da954679924b2ad server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae4959a700 1 -- 172.21.15.112:0/2674820792 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4000b870 con 0x7fae4c146c20 2023-08-27T11:19:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae4959a700 1 -- 172.21.15.112:0/2674820792 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4000b9d0 con 0x7fae4c146c20 2023-08-27T11:19:00.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.870+0000 7fae4959a700 1 -- 172.21.15.112:0/2674820792 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4000bd60 con 0x7fae4c146c20 2023-08-27T11:19:00.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.871+0000 7fae50c0a700 1 -- 172.21.15.112:0/2674820792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c146c20 msgr2=0x7fae4c160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.871+0000 7fae50c0a700 1 --2- 172.21.15.112:0/2674820792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c146c20 0x7fae4c160f60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0b1c80 tx=0x7fae40009bd0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.871+0000 7fae50c0a700 1 -- 172.21.15.112:0/2674820792 shutdown_connections 2023-08-27T11:19:00.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.871+0000 7fae50c0a700 1 --2- 172.21.15.112:0/2674820792 >> v2:172.21.15.138:3300/0 conn(0x7fae4c15e6e0 0x7fae4c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.871+0000 7fae50c0a700 1 --2- 172.21.15.112:0/2674820792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c146c20 0x7fae4c160f60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.871+0000 7fae50c0a700 1 --2- 172.21.15.112:0/2674820792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae4c0b3440 0x7fae4c15e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.871+0000 7fae50c0a700 1 -- 172.21.15.112:0/2674820792 >> 172.21.15.112:0/2674820792 conn(0x7fae4c0b1e90 msgr2=0x7fae4c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:00.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.871+0000 7fae50c0a700 1 -- 172.21.15.112:0/2674820792 shutdown_connections 2023-08-27T11:19:00.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.871+0000 7fae50c0a700 1 -- 172.21.15.112:0/2674820792 wait complete. 2023-08-27T11:19:00.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.872+0000 7fae50c0a700 1 Processor -- start 2023-08-27T11:19:00.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.872+0000 7fae50c0a700 1 -- start start 2023-08-27T11:19:00.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae50c0a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae4c0b3440 0x7fae4c15b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae50c0a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c146c20 0x7fae4c15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae50c0a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae4c15e6e0 0x7fae4c13ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae50c0a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c13a0c0 con 0x7fae4c146c20 2023-08-27T11:19:00.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae50c0a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c1f69d0 con 0x7fae4c0b3440 2023-08-27T11:19:00.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae50c0a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c1f6cb0 con 0x7fae4c15e6e0 2023-08-27T11:19:00.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae49d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c146c20 0x7fae4c15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae49d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c146c20 0x7fae4c15a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58454/0 (socket says 172.21.15.112:58454) 2023-08-27T11:19:00.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae49d9b700 1 -- 172.21.15.112:0/3907350850 learned_addr learned my addr 172.21.15.112:0/3907350850 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:00.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae4a59c700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae4c0b3440 0x7fae4c15b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae4ad9d700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae4c15e6e0 0x7fae4c13ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae49d9b700 1 -- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae4c15e6e0 msgr2=0x7fae4c13ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae49d9b700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae4c15e6e0 0x7fae4c13ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae49d9b700 1 -- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae4c0b3440 msgr2=0x7fae4c15b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:00.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae49d9b700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae4c0b3440 0x7fae4c15b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:00.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae49d9b700 1 -- 172.21.15.112:0/3907350850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae40009550 con 0x7fae4c146c20 2023-08-27T11:19:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.873+0000 7fae49d9b700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c146c20 0x7fae4c15a7a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00d800 tx=0x7fae3c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.874+0000 7fae3affd700 1 -- 172.21.15.112:0/3907350850 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c004030 con 0x7fae4c146c20 2023-08-27T11:19:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.874+0000 7fae3affd700 1 -- 172.21.15.112:0/3907350850 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c004190 con 0x7fae4c146c20 2023-08-27T11:19:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.874+0000 7fae3affd700 1 -- 172.21.15.112:0/3907350850 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c004520 con 0x7fae4c146c20 2023-08-27T11:19:00.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.874+0000 7fae50c0a700 1 -- 172.21.15.112:0/3907350850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4c139d50 con 0x7fae4c146c20 2023-08-27T11:19:00.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.874+0000 7fae50c0a700 1 -- 172.21.15.112:0/3907350850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c1380e0 con 0x7fae4c146c20 2023-08-27T11:19:00.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.875+0000 7fae50c0a700 1 -- 172.21.15.112:0/3907350850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4c067e90 con 0x7fae4c146c20 2023-08-27T11:19:00.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.875+0000 7fae3affd700 1 -- 172.21.15.112:0/3907350850 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fae3c0036a0 con 0x7fae4c146c20 2023-08-27T11:19:00.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.875+0000 7fae3affd700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fae2c06a170 0x7fae2c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:00.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.875+0000 7fae3affd700 1 -- 172.21.15.112:0/3907350850 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fae3c087e50 con 0x7fae4c146c20 2023-08-27T11:19:00.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.876+0000 7fae4a59c700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fae2c06a170 0x7fae2c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:00.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.876+0000 7fae4a59c700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fae2c06a170 0x7fae2c06c630 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fae40000c00 tx=0x7fae40008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:00.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:00.880+0000 7fae3affd700 1 -- 172.21.15.112:0/3907350850 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fae3c0570a0 con 0x7fae4c146c20 2023-08-27T11:19:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.069+0000 7fae50c0a700 1 -- 172.21.15.112:0/3907350850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae4c139ee0 con 0x7fae4c146c20 2023-08-27T11:19:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.069+0000 7fae3affd700 1 -- 172.21.15.112:0/3907350850 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae3c019020 con 0x7fae4c146c20 2023-08-27T11:19:01.071 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:19:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.071+0000 7fae50c0a700 1 -- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fae2c06a170 msgr2=0x7fae2c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.071+0000 7fae50c0a700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fae2c06a170 0x7fae2c06c630 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fae40000c00 tx=0x7fae40008030 comp rx=0 tx=0).stop 2023-08-27T11:19:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.071+0000 7fae50c0a700 1 -- 172.21.15.112:0/3907350850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c146c20 msgr2=0x7fae4c15a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.071+0000 7fae50c0a700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c146c20 0x7fae4c15a7a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00d800 tx=0x7fae3c00dbc0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.071+0000 7fae50c0a700 1 -- 172.21.15.112:0/3907350850 shutdown_connections 2023-08-27T11:19:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.071+0000 7fae50c0a700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fae2c06a170 0x7fae2c06c630 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.071+0000 7fae50c0a700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae4c15e6e0 0x7fae4c13ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.071+0000 7fae50c0a700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae4c0b3440 0x7fae4c15b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.071+0000 7fae50c0a700 1 --2- 172.21.15.112:0/3907350850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c146c20 0x7fae4c15a7a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.071+0000 7fae50c0a700 1 -- 172.21.15.112:0/3907350850 >> 172.21.15.112:0/3907350850 conn(0x7fae4c0b1e90 msgr2=0x7fae4c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:01.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.071+0000 7fae50c0a700 1 -- 172.21.15.112:0/3907350850 shutdown_connections 2023-08-27T11:19:01.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.071+0000 7fae50c0a700 1 -- 172.21.15.112:0/3907350850 wait complete. 2023-08-27T11:19:01.085 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:19:01.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.207+0000 7fa8a867f700 1 Processor -- start 2023-08-27T11:19:01.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.207+0000 7fa8a867f700 1 -- start start 2023-08-27T11:19:01.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a867f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa8a0144900 0x7fa8a00b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a867f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa8a00cca90 0x7fa8a015f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a867f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a015ee90 con 0x7fa8a006f790 2023-08-27T11:19:01.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a867f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a00b11d0 con 0x7fa8a0144900 2023-08-27T11:19:01.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a867f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a00b14b0 con 0x7fa8a00cca90 2023-08-27T11:19:01.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a641b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa8a0144900 0x7fa8a00b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a641b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa8a0144900 0x7fa8a00b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58460/0 (socket says 172.21.15.112:58460) 2023-08-27T11:19:01.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a641b700 1 -- 172.21.15.112:0/2498497221 learned_addr learned my addr 172.21.15.112:0/2498497221 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:01.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a5c1a700 1 --2- 172.21.15.112:0/2498497221 >> v2:172.21.15.138:3300/0 conn(0x7fa8a00cca90 0x7fa8a015f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a641b700 1 -- 172.21.15.112:0/2498497221 >> v2:172.21.15.138:3300/0 conn(0x7fa8a00cca90 msgr2=0x7fa8a015f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a641b700 1 --2- 172.21.15.112:0/2498497221 >> v2:172.21.15.138:3300/0 conn(0x7fa8a00cca90 0x7fa8a015f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a641b700 1 -- 172.21.15.112:0/2498497221 >> v1:172.21.15.138:6789/0 conn(0x7fa8a006f790 legacy=0x7fa8a015b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a641b700 1 -- 172.21.15.112:0/2498497221 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a015f130 con 0x7fa8a0144900 2023-08-27T11:19:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a641b700 1 --2- 172.21.15.112:0/2498497221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a0144900 0x7fa8a00b09a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00703e0 tx=0x7fa89c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a158fb1f2c3eb73 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a5419700 1 -- 172.21.15.112:0/2498497221 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c011830 con 0x7fa8a0144900 2023-08-27T11:19:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a5419700 1 -- 172.21.15.112:0/2498497221 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c011990 con 0x7fa8a0144900 2023-08-27T11:19:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.208+0000 7fa8a5419700 1 -- 172.21.15.112:0/2498497221 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c011d20 con 0x7fa8a0144900 2023-08-27T11:19:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.209+0000 7fa8a867f700 1 -- 172.21.15.112:0/2498497221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a0144900 msgr2=0x7fa8a00b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.209+0000 7fa8a867f700 1 --2- 172.21.15.112:0/2498497221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a0144900 0x7fa8a00b09a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00703e0 tx=0x7fa89c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.209+0000 7fa8a867f700 1 -- 172.21.15.112:0/2498497221 shutdown_connections 2023-08-27T11:19:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.209+0000 7fa8a867f700 1 --2- 172.21.15.112:0/2498497221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a0144900 0x7fa8a00b09a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.209+0000 7fa8a867f700 1 --2- 172.21.15.112:0/2498497221 >> v2:172.21.15.138:3300/0 conn(0x7fa8a00cca90 0x7fa8a015f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.209+0000 7fa8a867f700 1 -- 172.21.15.112:0/2498497221 >> 172.21.15.112:0/2498497221 conn(0x7fa8a00b1e90 msgr2=0x7fa8a00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.209+0000 7fa8a867f700 1 -- 172.21.15.112:0/2498497221 shutdown_connections 2023-08-27T11:19:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.209+0000 7fa8a867f700 1 -- 172.21.15.112:0/2498497221 wait complete. 2023-08-27T11:19:01.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.210+0000 7fa8a867f700 1 Processor -- start 2023-08-27T11:19:01.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.210+0000 7fa8a867f700 1 -- start start 2023-08-27T11:19:01.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a867f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a006f790 0x7fa8a013a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a867f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8a00cca90 0x7fa8a015a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a867f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a0144900 0x7fa8a015d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a867f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a013a380 con 0x7fa8a006f790 2023-08-27T11:19:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a867f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0139800 con 0x7fa8a0144900 2023-08-27T11:19:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a867f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0139ae0 con 0x7fa8a00cca90 2023-08-27T11:19:01.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a641b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a006f790 0x7fa8a013a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a641b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a006f790 0x7fa8a013a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58466/0 (socket says 172.21.15.112:58466) 2023-08-27T11:19:01.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a641b700 1 -- 172.21.15.112:0/2359280579 learned_addr learned my addr 172.21.15.112:0/2359280579 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:01.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a5c1a700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8a00cca90 0x7fa8a015a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a6c1c700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a0144900 0x7fa8a015d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a641b700 1 -- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8a00cca90 msgr2=0x7fa8a015a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a641b700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8a00cca90 0x7fa8a015a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a641b700 1 -- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a0144900 msgr2=0x7fa8a015d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a641b700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a0144900 0x7fa8a015d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a641b700 1 -- 172.21.15.112:0/2359280579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c009550 con 0x7fa8a006f790 2023-08-27T11:19:01.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a5c1a700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8a00cca90 0x7fa8a015a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:01.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a6c1c700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a0144900 0x7fa8a015d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:01.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.211+0000 7fa8a641b700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a006f790 0x7fa8a013a530 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009c10 tx=0x7fa89c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:01.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.212+0000 7fa896ffd700 1 -- 172.21.15.112:0/2359280579 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c0036a0 con 0x7fa8a006f790 2023-08-27T11:19:01.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.212+0000 7fa896ffd700 1 -- 172.21.15.112:0/2359280579 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c01e070 con 0x7fa8a006f790 2023-08-27T11:19:01.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.212+0000 7fa8a867f700 1 -- 172.21.15.112:0/2359280579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a0139dc0 con 0x7fa8a006f790 2023-08-27T11:19:01.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.212+0000 7fa896ffd700 1 -- 172.21.15.112:0/2359280579 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c003a60 con 0x7fa8a006f790 2023-08-27T11:19:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.212+0000 7fa8a867f700 1 -- 172.21.15.112:0/2359280579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a015aa30 con 0x7fa8a006f790 2023-08-27T11:19:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.212+0000 7fa8a867f700 1 -- 172.21.15.112:0/2359280579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a0067e90 con 0x7fa8a006f790 2023-08-27T11:19:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.213+0000 7fa896ffd700 1 -- 172.21.15.112:0/2359280579 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fa89c003bc0 con 0x7fa8a006f790 2023-08-27T11:19:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.213+0000 7fa896ffd700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa88406a170 0x7fa88406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.213+0000 7fa896ffd700 1 -- 172.21.15.112:0/2359280579 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fa89c0894a0 con 0x7fa8a006f790 2023-08-27T11:19:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.213+0000 7fa8a5c1a700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa88406a170 0x7fa88406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.214+0000 7fa8a5c1a700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa88406a170 0x7fa88406c630 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00ce540 tx=0x7fa890008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.218+0000 7fa896ffd700 1 -- 172.21.15.112:0/2359280579 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa89c054bb0 con 0x7fa8a006f790 2023-08-27T11:19:01.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.406+0000 7fa8a867f700 1 -- 172.21.15.112:0/2359280579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa8a0067a20 con 0x7fa8a006f790 2023-08-27T11:19:01.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.407+0000 7fa896ffd700 1 -- 172.21.15.112:0/2359280579 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa89c0581d0 con 0x7fa8a006f790 2023-08-27T11:19:01.409 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:19:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.409+0000 7fa8a867f700 1 -- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa88406a170 msgr2=0x7fa88406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.409+0000 7fa8a867f700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa88406a170 0x7fa88406c630 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00ce540 tx=0x7fa890008040 comp rx=0 tx=0).stop 2023-08-27T11:19:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.409+0000 7fa8a867f700 1 -- 172.21.15.112:0/2359280579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a006f790 msgr2=0x7fa8a013a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.409+0000 7fa8a867f700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a006f790 0x7fa8a013a530 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009c10 tx=0x7fa89c004750 comp rx=0 tx=0).stop 2023-08-27T11:19:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.409+0000 7fa8a867f700 1 -- 172.21.15.112:0/2359280579 shutdown_connections 2023-08-27T11:19:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.409+0000 7fa8a867f700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa88406a170 0x7fa88406c630 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.409+0000 7fa8a867f700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a0144900 0x7fa8a015d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.409+0000 7fa8a867f700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a006f790 0x7fa8a013a530 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.409+0000 7fa8a867f700 1 --2- 172.21.15.112:0/2359280579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8a00cca90 0x7fa8a015a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.409+0000 7fa8a867f700 1 -- 172.21.15.112:0/2359280579 >> 172.21.15.112:0/2359280579 conn(0x7fa8a00b1e90 msgr2=0x7fa8a015b840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:01.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.409+0000 7fa8a867f700 1 -- 172.21.15.112:0/2359280579 shutdown_connections 2023-08-27T11:19:01.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.409+0000 7fa8a867f700 1 -- 172.21.15.112:0/2359280579 wait complete. 2023-08-27T11:19:01.426 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:19:01.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.551+0000 7f98e54e4700 1 Processor -- start 2023-08-27T11:19:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.552+0000 7f98e54e4700 1 -- start start 2023-08-27T11:19:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.552+0000 7f98e54e4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f98e0146be0 0x7f98e00b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.552+0000 7f98e54e4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98e00b5470 0x7f98e00cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.552+0000 7f98e54e4700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e0161e10 con 0x7f98e01619b0 2023-08-27T11:19:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.552+0000 7f98e54e4700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e00b3980 con 0x7f98e0146be0 2023-08-27T11:19:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.552+0000 7f98e54e4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00b3b70 con 0x7f98e00b5470 2023-08-27T11:19:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.552+0000 7f98deffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f98e0146be0 0x7f98e00b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.553+0000 7f98deffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f98e0146be0 0x7f98e00b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58476/0 (socket says 172.21.15.112:58476) 2023-08-27T11:19:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.553+0000 7f98deffd700 1 -- 172.21.15.112:0/870329138 learned_addr learned my addr 172.21.15.112:0/870329138 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.553+0000 7f98df7fe700 1 --2- 172.21.15.112:0/870329138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98e00b5470 0x7f98e00cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.553+0000 7f98deffd700 1 -- 172.21.15.112:0/870329138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98e00b5470 msgr2=0x7f98e00cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.553+0000 7f98deffd700 1 --2- 172.21.15.112:0/870329138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98e00b5470 0x7f98e00cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.553+0000 7f98deffd700 1 -- 172.21.15.112:0/870329138 >> v1:172.21.15.138:6789/0 conn(0x7f98e01619b0 legacy=0x7f98e00b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.553+0000 7f98deffd700 1 -- 172.21.15.112:0/870329138 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e00b3e50 con 0x7f98e0146be0 2023-08-27T11:19:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.553+0000 7f98df7fe700 1 --2- 172.21.15.112:0/870329138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98e00b5470 0x7f98e00cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.553+0000 7f98deffd700 1 --2- 172.21.15.112:0/870329138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e0146be0 0x7f98e00b3440 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f98d40098f0 tx=0x7f98d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ab99823129eb3f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:01.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.553+0000 7f98ddffb700 1 -- 172.21.15.112:0/870329138 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d400b8a0 con 0x7f98e0146be0 2023-08-27T11:19:01.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.553+0000 7f98ddffb700 1 -- 172.21.15.112:0/870329138 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d400ba00 con 0x7f98e0146be0 2023-08-27T11:19:01.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.553+0000 7f98ddffb700 1 -- 172.21.15.112:0/870329138 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d400bd90 con 0x7f98e0146be0 2023-08-27T11:19:01.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.554+0000 7f98e54e4700 1 -- 172.21.15.112:0/870329138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e0146be0 msgr2=0x7f98e00b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.554+0000 7f98e54e4700 1 --2- 172.21.15.112:0/870329138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e0146be0 0x7f98e00b3440 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f98d40098f0 tx=0x7f98d4009c00 comp rx=0 tx=0).stop 2023-08-27T11:19:01.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.554+0000 7f98e54e4700 1 -- 172.21.15.112:0/870329138 shutdown_connections 2023-08-27T11:19:01.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.554+0000 7f98e54e4700 1 --2- 172.21.15.112:0/870329138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98e00b5470 0x7f98e00cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.554+0000 7f98e54e4700 1 --2- 172.21.15.112:0/870329138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e0146be0 0x7f98e00b3440 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.554+0000 7f98e54e4700 1 -- 172.21.15.112:0/870329138 >> 172.21.15.112:0/870329138 conn(0x7f98e006c880 msgr2=0x7f98e00717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.554+0000 7f98e54e4700 1 -- 172.21.15.112:0/870329138 shutdown_connections 2023-08-27T11:19:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.554+0000 7f98e54e4700 1 -- 172.21.15.112:0/870329138 wait complete. 2023-08-27T11:19:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.555+0000 7f98e54e4700 1 Processor -- start 2023-08-27T11:19:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.555+0000 7f98e54e4700 1 -- start start 2023-08-27T11:19:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98e54e4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98e00b5470 0x7f98e0151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98e54e4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98e0146be0 0x7f98e0070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98e54e4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e01619b0 0x7f98e013caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98e54e4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e01521e0 con 0x7f98e01619b0 2023-08-27T11:19:01.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98e54e4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0151cc0 con 0x7f98e00b5470 2023-08-27T11:19:01.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98e54e4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e013ba00 con 0x7f98e0146be0 2023-08-27T11:19:01.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98df7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e01619b0 0x7f98e013caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98df7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e01619b0 0x7f98e013caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58484/0 (socket says 172.21.15.112:58484) 2023-08-27T11:19:01.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98df7fe700 1 -- 172.21.15.112:0/3062397827 learned_addr learned my addr 172.21.15.112:0/3062397827 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:01.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98deffd700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98e00b5470 0x7f98e0151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98de7fc700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98e0146be0 0x7f98e0070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98df7fe700 1 -- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98e0146be0 msgr2=0x7f98e0070fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98df7fe700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98e0146be0 0x7f98e0070fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98df7fe700 1 -- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98e00b5470 msgr2=0x7f98e0151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98df7fe700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98e00b5470 0x7f98e0151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98df7fe700 1 -- 172.21.15.112:0/3062397827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d4009550 con 0x7f98e01619b0 2023-08-27T11:19:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.556+0000 7f98df7fe700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e01619b0 0x7f98e013caa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f98d8007dc0 tx=0x7f98d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.557+0000 7f98c77fe700 1 -- 172.21.15.112:0/3062397827 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d800dd20 con 0x7f98e01619b0 2023-08-27T11:19:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.557+0000 7f98c77fe700 1 -- 172.21.15.112:0/3062397827 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d800f040 con 0x7f98e01619b0 2023-08-27T11:19:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.557+0000 7f98e54e4700 1 -- 172.21.15.112:0/3062397827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e013bce0 con 0x7f98e01619b0 2023-08-27T11:19:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.557+0000 7f98c77fe700 1 -- 172.21.15.112:0/3062397827 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8003c60 con 0x7f98e01619b0 2023-08-27T11:19:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.557+0000 7f98e54e4700 1 -- 172.21.15.112:0/3062397827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e013c2a0 con 0x7f98e01619b0 2023-08-27T11:19:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.557+0000 7f98e54e4700 1 -- 172.21.15.112:0/3062397827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e00b43f0 con 0x7f98e01619b0 2023-08-27T11:19:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.558+0000 7f98c77fe700 1 -- 172.21.15.112:0/3062397827 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f98d8004030 con 0x7f98e01619b0 2023-08-27T11:19:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.558+0000 7f98c77fe700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f98cc06a170 0x7f98cc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.558+0000 7f98c77fe700 1 -- 172.21.15.112:0/3062397827 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f98d80886e0 con 0x7f98e01619b0 2023-08-27T11:19:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.559+0000 7f98deffd700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f98cc06a170 0x7f98cc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.559+0000 7f98deffd700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f98cc06a170 0x7f98cc06c630 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009c40 tx=0x7f98d4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:01.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.563+0000 7f98c77fe700 1 -- 172.21.15.112:0/3062397827 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f98e00b43f0 con 0x7f98e01619b0 2023-08-27T11:19:01.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.793+0000 7f98e54e4700 1 -- 172.21.15.112:0/3062397827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98e00b43f0 con 0x7f98e01619b0 2023-08-27T11:19:01.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.794+0000 7f98c77fe700 1 -- 172.21.15.112:0/3062397827 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4888 (secure 0 0 0) 0x7f98e00b43f0 con 0x7f98e01619b0 2023-08-27T11:19:01.796 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:01.796 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1692518554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1692518554},{"type":"v1","addr":"172.21.15.138:6837","nonce":1692518554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3294671617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3294671617},{"type":"v1","addr":"172.21.15.138:6836","nonce":3294671617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3012780135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3012780135},{"type":"v1","addr":"172.21.15.112:6834","nonce":3012780135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:18:43.281470+0000","modified":"2023-08-27T11:18:47.234075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5661":{"gid":5661,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3423469954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3423469954},{"type":"v1","addr":"172.21.15.112:6835","nonce":3423469954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-27T11:19:01.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.796+0000 7f98e54e4700 1 -- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f98cc06a170 msgr2=0x7f98cc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.796+0000 7f98e54e4700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f98cc06a170 0x7f98cc06c630 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009c40 tx=0x7f98d4008030 comp rx=0 tx=0).stop 2023-08-27T11:19:01.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.796+0000 7f98e54e4700 1 -- 172.21.15.112:0/3062397827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e01619b0 msgr2=0x7f98e013caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.796+0000 7f98e54e4700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e01619b0 0x7f98e013caa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f98d8007dc0 tx=0x7f98d800d3b0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.796+0000 7f98e54e4700 1 -- 172.21.15.112:0/3062397827 shutdown_connections 2023-08-27T11:19:01.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.796+0000 7f98e54e4700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f98cc06a170 0x7f98cc06c630 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.796+0000 7f98e54e4700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e01619b0 0x7f98e013caa0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.796+0000 7f98e54e4700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98e00b5470 0x7f98e0151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.796+0000 7f98e54e4700 1 --2- 172.21.15.112:0/3062397827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98e0146be0 0x7f98e0070fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.796+0000 7f98e54e4700 1 -- 172.21.15.112:0/3062397827 >> 172.21.15.112:0/3062397827 conn(0x7f98e006c880 msgr2=0x7f98e00b8590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:01.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.796+0000 7f98e54e4700 1 -- 172.21.15.112:0/3062397827 shutdown_connections 2023-08-27T11:19:01.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.796+0000 7f98e54e4700 1 -- 172.21.15.112:0/3062397827 wait complete. 2023-08-27T11:19:01.806 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 31 2023-08-27T11:19:01.810 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:19:01.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.933+0000 7f517806d700 1 Processor -- start 2023-08-27T11:19:01.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.933+0000 7f517806d700 1 -- start start 2023-08-27T11:19:01.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f517806d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f51700b1400 0x7f51700cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f517806d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5170161ce0 0x7f517006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f517806d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5170161850 con 0x7f51700b17e0 2023-08-27T11:19:01.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f517806d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51700cfc10 con 0x7f5170161ce0 2023-08-27T11:19:01.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f517806d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5170161200 con 0x7f51700b1400 2023-08-27T11:19:01.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f5175608700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f51700b17e0 0x7f517006e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52870/0 (socket says 172.21.15.112:52870) 2023-08-27T11:19:01.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f5175608700 1 -- 172.21.15.112:0/3037921341 learned_addr learned my addr 172.21.15.112:0/3037921341 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:01.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f517660a700 1 --2- 172.21.15.112:0/3037921341 >> v2:172.21.15.112:3300/0 conn(0x7f5170161ce0 0x7f517006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f5175e09700 1 --2- 172.21.15.112:0/3037921341 >> v2:172.21.15.138:3300/0 conn(0x7f51700b1400 0x7f51700cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f5174e07700 1 -- 172.21.15.112:0/3037921341 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027806813 0 0) 0x7f5170161850 con 0x7f51700b17e0 2023-08-27T11:19:01.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f5174e07700 1 -- 172.21.15.112:0/3037921341 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f515c0035b0 con 0x7f51700b17e0 2023-08-27T11:19:01.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f517660a700 1 -- 172.21.15.112:0/3037921341 >> v2:172.21.15.138:3300/0 conn(0x7f51700b1400 msgr2=0x7f51700cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:01.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f517660a700 1 --2- 172.21.15.112:0/3037921341 >> v2:172.21.15.138:3300/0 conn(0x7f51700b1400 0x7f51700cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f517660a700 1 -- 172.21.15.112:0/3037921341 >> v1:172.21.15.112:6789/0 conn(0x7f51700b17e0 legacy=0x7f517006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f517660a700 1 -- 172.21.15.112:0/3037921341 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51701614e0 con 0x7f5170161ce0 2023-08-27T11:19:01.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f517660a700 1 --2- 172.21.15.112:0/3037921341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5170161ce0 0x7f517006f790 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5170070370 tx=0x7f516800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7123b5fb7a74097f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:01.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.934+0000 7f5174e07700 1 -- 172.21.15.112:0/3037921341 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516800e070 con 0x7f5170161ce0 2023-08-27T11:19:01.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.935+0000 7f5174e07700 1 -- 172.21.15.112:0/3037921341 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516800bb20 con 0x7f5170161ce0 2023-08-27T11:19:01.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.935+0000 7f5174e07700 1 -- 172.21.15.112:0/3037921341 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168012430 con 0x7f5170161ce0 2023-08-27T11:19:01.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.935+0000 7f517806d700 1 -- 172.21.15.112:0/3037921341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5170161ce0 msgr2=0x7f517006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.935+0000 7f517806d700 1 --2- 172.21.15.112:0/3037921341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5170161ce0 0x7f517006f790 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5170070370 tx=0x7f516800b330 comp rx=0 tx=0).stop 2023-08-27T11:19:01.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.935+0000 7f517806d700 1 -- 172.21.15.112:0/3037921341 shutdown_connections 2023-08-27T11:19:01.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.935+0000 7f517806d700 1 --2- 172.21.15.112:0/3037921341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5170161ce0 0x7f517006f790 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.935+0000 7f517806d700 1 --2- 172.21.15.112:0/3037921341 >> v2:172.21.15.138:3300/0 conn(0x7f51700b1400 0x7f51700cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.935+0000 7f517806d700 1 -- 172.21.15.112:0/3037921341 >> 172.21.15.112:0/3037921341 conn(0x7f51700d0000 msgr2=0x7f51700d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:01.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.935+0000 7f517806d700 1 -- 172.21.15.112:0/3037921341 shutdown_connections 2023-08-27T11:19:01.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.935+0000 7f517806d700 1 -- 172.21.15.112:0/3037921341 wait complete. 2023-08-27T11:19:01.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.937+0000 7f517806d700 1 Processor -- start 2023-08-27T11:19:01.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.937+0000 7f517806d700 1 -- start start 2023-08-27T11:19:01.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f517806d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51700b1400 0x7f517014a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f517806d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51700b17e0 0x7f51700c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f517806d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5170161ce0 0x7f51701f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f517806d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51700c48b0 con 0x7f51700b1400 2023-08-27T11:19:01.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f517806d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51700c4b90 con 0x7f51700b17e0 2023-08-27T11:19:01.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f517806d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51700c4e70 con 0x7f5170161ce0 2023-08-27T11:19:01.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f5175e09700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51700b1400 0x7f517014a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f5175e09700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51700b1400 0x7f517014a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58494/0 (socket says 172.21.15.112:58494) 2023-08-27T11:19:01.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f5175e09700 1 -- 172.21.15.112:0/2964032417 learned_addr learned my addr 172.21.15.112:0/2964032417 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:01.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f5175608700 1 --2- 172.21.15.112:0/2964032417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51700b17e0 0x7f51700c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f5175e09700 1 -- 172.21.15.112:0/2964032417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5170161ce0 msgr2=0x7f51701f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f5175e09700 1 --2- 172.21.15.112:0/2964032417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5170161ce0 0x7f51701f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f5175e09700 1 -- 172.21.15.112:0/2964032417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51700b17e0 msgr2=0x7f51700c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:01.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f5175e09700 1 --2- 172.21.15.112:0/2964032417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51700b17e0 0x7f51700c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:01.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f5175e09700 1 -- 172.21.15.112:0/2964032417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5168009a70 con 0x7f51700b1400 2023-08-27T11:19:01.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f5175e09700 1 --2- 172.21.15.112:0/2964032417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51700b1400 0x7f517014a3b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f516c00e870 tx=0x7f516c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:01.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f51667fc700 1 -- 172.21.15.112:0/2964032417 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c009590 con 0x7f51700b1400 2023-08-27T11:19:01.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f51667fc700 1 -- 172.21.15.112:0/2964032417 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c00ee70 con 0x7f51700b1400 2023-08-27T11:19:01.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f51667fc700 1 -- 172.21.15.112:0/2964032417 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c00ca50 con 0x7f51700b1400 2023-08-27T11:19:01.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f517806d700 1 -- 172.21.15.112:0/2964032417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51700c5150 con 0x7f51700b1400 2023-08-27T11:19:01.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.938+0000 7f517806d700 1 -- 172.21.15.112:0/2964032417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517014a8f0 con 0x7f51700b1400 2023-08-27T11:19:01.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.939+0000 7f517806d700 1 -- 172.21.15.112:0/2964032417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5170157f50 con 0x7f51700b1400 2023-08-27T11:19:01.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.940+0000 7f51667fc700 1 -- 172.21.15.112:0/2964032417 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f516c00cbb0 con 0x7f51700b1400 2023-08-27T11:19:01.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.940+0000 7f51667fc700 1 --2- 172.21.15.112:0/2964032417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f515406a170 0x7f515406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:01.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.940+0000 7f51667fc700 1 -- 172.21.15.112:0/2964032417 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f516c0886e0 con 0x7f51700b1400 2023-08-27T11:19:01.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.941+0000 7f5175608700 1 --2- 172.21.15.112:0/2964032417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f515406a170 0x7f515406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:01.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.941+0000 7f5175608700 1 --2- 172.21.15.112:0/2964032417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f515406a170 0x7f515406c630 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f51700d0710 tx=0x7f5160009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:01.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:01.945+0000 7f51667fc700 1 -- 172.21.15.112:0/2964032417 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f516c0576f0 con 0x7f51700b1400 2023-08-27T11:19:02.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.132+0000 7f517806d700 1 -- 172.21.15.112:0/2964032417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5170067a20 con 0x7f51700b1400 2023-08-27T11:19:02.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.133+0000 7f51667fc700 1 -- 172.21.15.112:0/2964032417 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18452 (secure 0 0 0) 0x7f516c00ce60 con 0x7f51700b1400 2023-08-27T11:19:02.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:02.136 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":53,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:54.265785+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:41.033680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:42.255772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1e","mappings":[{"from":7,"to":2}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]},{"pgid":"10.34","mappings":[{"from":7,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:19:02.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.135+0000 7f517806d700 1 -- 172.21.15.112:0/2964032417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f515406a170 msgr2=0x7f515406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:02.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.135+0000 7f517806d700 1 --2- 172.21.15.112:0/2964032417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f515406a170 0x7f515406c630 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f51700d0710 tx=0x7f5160009040 comp rx=0 tx=0).stop 2023-08-27T11:19:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.135+0000 7f517806d700 1 -- 172.21.15.112:0/2964032417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51700b1400 msgr2=0x7f517014a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.135+0000 7f517806d700 1 --2- 172.21.15.112:0/2964032417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51700b1400 0x7f517014a3b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f516c00e870 tx=0x7f516c00ec30 comp rx=0 tx=0).stop 2023-08-27T11:19:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.135+0000 7f517806d700 1 -- 172.21.15.112:0/2964032417 shutdown_connections 2023-08-27T11:19:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.135+0000 7f517806d700 1 --2- 172.21.15.112:0/2964032417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f515406a170 0x7f515406c630 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.135+0000 7f517806d700 1 --2- 172.21.15.112:0/2964032417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5170161ce0 0x7f51701f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.135+0000 7f517806d700 1 --2- 172.21.15.112:0/2964032417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51700b1400 0x7f517014a3b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.135+0000 7f517806d700 1 --2- 172.21.15.112:0/2964032417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51700b17e0 0x7f51700c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.135+0000 7f517806d700 1 -- 172.21.15.112:0/2964032417 >> 172.21.15.112:0/2964032417 conn(0x7f51700d0000 msgr2=0x7f51700cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.135+0000 7f517806d700 1 -- 172.21.15.112:0/2964032417 shutdown_connections 2023-08-27T11:19:02.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.135+0000 7f517806d700 1 -- 172.21.15.112:0/2964032417 wait complete. 2023-08-27T11:19:02.150 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:19:02.150 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:19:02.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.269+0000 7fd2abf99700 1 Processor -- start 2023-08-27T11:19:02.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.269+0000 7fd2abf99700 1 -- start start 2023-08-27T11:19:02.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2abf99700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd2a4160a00 0x7fd2a40b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:02.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2abf99700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a406d510 0x7fd2a406d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:02.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2abf99700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a406dea0 con 0x7fd2a4146990 2023-08-27T11:19:02.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2abf99700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a406f930 con 0x7fd2a4160a00 2023-08-27T11:19:02.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2abf99700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a406fc10 con 0x7fd2a406d510 2023-08-27T11:19:02.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2aa536700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a406d510 0x7fd2a406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:02.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2a9d35700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd2a4160a00 0x7fd2a40b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:02.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2aa536700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a406d510 0x7fd2a406d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:41928/0 (socket says 172.21.15.112:41928) 2023-08-27T11:19:02.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2a9d35700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd2a4160a00 0x7fd2a40b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:58928/0 (socket says 172.21.15.112:58928) 2023-08-27T11:19:02.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2aa536700 1 -- 172.21.15.112:0/2211205554 learned_addr learned my addr 172.21.15.112:0/2211205554 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:02.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2a9534700 1 --1- 172.21.15.112:0/2211205554 >> v1:172.21.15.138:6789/0 conn(0x7fd2a4146990 0x7fd2a415dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:60408/0 (socket says 172.21.15.112:60408) 2023-08-27T11:19:02.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2aa536700 1 -- 172.21.15.112:0/2211205554 >> v2:172.21.15.138:3300/0 conn(0x7fd2a4160a00 msgr2=0x7fd2a40b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:02.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2aa536700 1 --2- 172.21.15.112:0/2211205554 >> v2:172.21.15.138:3300/0 conn(0x7fd2a4160a00 0x7fd2a40b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2aa536700 1 -- 172.21.15.112:0/2211205554 >> v1:172.21.15.138:6789/0 conn(0x7fd2a4146990 legacy=0x7fd2a415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:02.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2aa536700 1 -- 172.21.15.112:0/2211205554 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a406fef0 con 0x7fd2a406d510 2023-08-27T11:19:02.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.270+0000 7fd2aa536700 1 --2- 172.21.15.112:0/2211205554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a406d510 0x7fd2a406d960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009e50 tx=0x7fd29c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dcdc07886f90535 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:02.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.271+0000 7fd2a8d33700 1 -- 172.21.15.112:0/2211205554 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c00e070 con 0x7fd2a406d510 2023-08-27T11:19:02.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.271+0000 7fd2a8d33700 1 -- 172.21.15.112:0/2211205554 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c00b910 con 0x7fd2a406d510 2023-08-27T11:19:02.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.271+0000 7fd2a8d33700 1 -- 172.21.15.112:0/2211205554 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c00bcc0 con 0x7fd2a406d510 2023-08-27T11:19:02.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.271+0000 7fd2abf99700 1 -- 172.21.15.112:0/2211205554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a406d510 msgr2=0x7fd2a406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:02.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.271+0000 7fd2abf99700 1 --2- 172.21.15.112:0/2211205554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a406d510 0x7fd2a406d960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009e50 tx=0x7fd29c00b330 comp rx=0 tx=0).stop 2023-08-27T11:19:02.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.271+0000 7fd2abf99700 1 -- 172.21.15.112:0/2211205554 shutdown_connections 2023-08-27T11:19:02.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.271+0000 7fd2abf99700 1 --2- 172.21.15.112:0/2211205554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a406d510 0x7fd2a406d960 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.271+0000 7fd2abf99700 1 --2- 172.21.15.112:0/2211205554 >> v2:172.21.15.138:3300/0 conn(0x7fd2a4160a00 0x7fd2a40b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.271+0000 7fd2abf99700 1 -- 172.21.15.112:0/2211205554 >> 172.21.15.112:0/2211205554 conn(0x7fd2a40cfe50 msgr2=0x7fd2a40d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:02.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.271+0000 7fd2abf99700 1 -- 172.21.15.112:0/2211205554 shutdown_connections 2023-08-27T11:19:02.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.271+0000 7fd2abf99700 1 -- 172.21.15.112:0/2211205554 wait complete. 2023-08-27T11:19:02.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.273+0000 7fd2abf99700 1 Processor -- start 2023-08-27T11:19:02.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.273+0000 7fd2abf99700 1 -- start start 2023-08-27T11:19:02.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2abf99700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a406d510 0x7fd2a415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:02.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2abf99700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a4146990 0x7fd2a40b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:02.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2abf99700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2a4160a00 0x7fd2a40b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:02.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2abf99700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a41365f0 con 0x7fd2a4146990 2023-08-27T11:19:02.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2abf99700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a4137450 con 0x7fd2a4160a00 2023-08-27T11:19:02.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2abf99700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a4136840 con 0x7fd2a406d510 2023-08-27T11:19:02.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2a9534700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a4146990 0x7fd2a40b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:02.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2a9534700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a4146990 0x7fd2a40b7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58510/0 (socket says 172.21.15.112:58510) 2023-08-27T11:19:02.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2a9d35700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a406d510 0x7fd2a415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:02.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2a9534700 1 -- 172.21.15.112:0/1102869698 learned_addr learned my addr 172.21.15.112:0/1102869698 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:02.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2aa536700 1 --2- 172.21.15.112:0/1102869698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2a4160a00 0x7fd2a40b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:02.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2a9534700 1 -- 172.21.15.112:0/1102869698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a406d510 msgr2=0x7fd2a415d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:02.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2a9534700 1 --2- 172.21.15.112:0/1102869698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a406d510 0x7fd2a415d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2a9534700 1 -- 172.21.15.112:0/1102869698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2a4160a00 msgr2=0x7fd2a40b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:02.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2a9534700 1 --2- 172.21.15.112:0/1102869698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2a4160a00 0x7fd2a40b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2a9534700 1 -- 172.21.15.112:0/1102869698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c009b30 con 0x7fd2a4146990 2023-08-27T11:19:02.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd2a9534700 1 --2- 172.21.15.112:0/1102869698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a4146990 0x7fd2a40b7f90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd29400c520 tx=0x7fd29400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:02.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd29a7fc700 1 -- 172.21.15.112:0/1102869698 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd294005cb0 con 0x7fd2a4146990 2023-08-27T11:19:02.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.274+0000 7fd29a7fc700 1 -- 172.21.15.112:0/1102869698 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd294005e10 con 0x7fd2a4146990 2023-08-27T11:19:02.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.275+0000 7fd2abf99700 1 -- 172.21.15.112:0/1102869698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a4136b20 con 0x7fd2a4146990 2023-08-27T11:19:02.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.275+0000 7fd29a7fc700 1 -- 172.21.15.112:0/1102869698 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2940072d0 con 0x7fd2a4146990 2023-08-27T11:19:02.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.275+0000 7fd2abf99700 1 -- 172.21.15.112:0/1102869698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a41370e0 con 0x7fd2a4146990 2023-08-27T11:19:02.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.275+0000 7fd2abf99700 1 -- 172.21.15.112:0/1102869698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a4067a20 con 0x7fd2a4146990 2023-08-27T11:19:02.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.276+0000 7fd29a7fc700 1 -- 172.21.15.112:0/1102869698 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fd294007430 con 0x7fd2a4146990 2023-08-27T11:19:02.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.276+0000 7fd29a7fc700 1 --2- 172.21.15.112:0/1102869698 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd28806a170 0x7fd28806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:02.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.276+0000 7fd29a7fc700 1 -- 172.21.15.112:0/1102869698 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fd294087700 con 0x7fd2a4146990 2023-08-27T11:19:02.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.276+0000 7fd2a9d35700 1 --2- 172.21.15.112:0/1102869698 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd28806a170 0x7fd28806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:02.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.277+0000 7fd2a9d35700 1 --2- 172.21.15.112:0/1102869698 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd28806a170 0x7fd28806c630 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009b40 tx=0x7fd2a0009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:02.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.281+0000 7fd29a7fc700 1 -- 172.21.15.112:0/1102869698 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd294052e30 con 0x7fd2a4146990 2023-08-27T11:19:02.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.513+0000 7fd2abf99700 1 -- 172.21.15.112:0/1102869698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2a406dea0 con 0x7fd2a4146990 2023-08-27T11:19:02.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.514+0000 7fd29a7fc700 1 -- 172.21.15.112:0/1102869698 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4888 (secure 0 0 0) 0x7fd2a406dea0 con 0x7fd2a4146990 2023-08-27T11:19:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:02.516 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1692518554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1692518554},{"type":"v1","addr":"172.21.15.138:6837","nonce":1692518554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3294671617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3294671617},{"type":"v1","addr":"172.21.15.138:6836","nonce":3294671617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3012780135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3012780135},{"type":"v1","addr":"172.21.15.112:6834","nonce":3012780135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:18:43.281470+0000","modified":"2023-08-27T11:18:47.234075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5661":{"gid":5661,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3423469954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3423469954},{"type":"v1","addr":"172.21.15.112:6835","nonce":3423469954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-27T11:19:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.516+0000 7fd2abf99700 1 -- 172.21.15.112:0/1102869698 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd28806a170 msgr2=0x7fd28806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.516+0000 7fd2abf99700 1 --2- 172.21.15.112:0/1102869698 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd28806a170 0x7fd28806c630 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009b40 tx=0x7fd2a0009210 comp rx=0 tx=0).stop 2023-08-27T11:19:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.516+0000 7fd2abf99700 1 -- 172.21.15.112:0/1102869698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a4146990 msgr2=0x7fd2a40b7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.516+0000 7fd2abf99700 1 --2- 172.21.15.112:0/1102869698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a4146990 0x7fd2a40b7f90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd29400c520 tx=0x7fd29400c830 comp rx=0 tx=0).stop 2023-08-27T11:19:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.516+0000 7fd2abf99700 1 -- 172.21.15.112:0/1102869698 shutdown_connections 2023-08-27T11:19:02.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.516+0000 7fd2abf99700 1 --2- 172.21.15.112:0/1102869698 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd28806a170 0x7fd28806c630 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.516+0000 7fd2abf99700 1 --2- 172.21.15.112:0/1102869698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2a4160a00 0x7fd2a40b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.516+0000 7fd2abf99700 1 --2- 172.21.15.112:0/1102869698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a406d510 0x7fd2a415d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.516+0000 7fd2abf99700 1 --2- 172.21.15.112:0/1102869698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a4146990 0x7fd2a40b7f90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.516+0000 7fd2abf99700 1 -- 172.21.15.112:0/1102869698 >> 172.21.15.112:0/1102869698 conn(0x7fd2a40cfe50 msgr2=0x7fd2a40d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.516+0000 7fd2abf99700 1 -- 172.21.15.112:0/1102869698 shutdown_connections 2023-08-27T11:19:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.516+0000 7fd2abf99700 1 -- 172.21.15.112:0/1102869698 wait complete. 2023-08-27T11:19:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 31 2023-08-27T11:19:02.529 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:19:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.651+0000 7fb95fef7700 1 Processor -- start 2023-08-27T11:19:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.652+0000 7fb95fef7700 1 -- start start 2023-08-27T11:19:02.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.652+0000 7fb95fef7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb9580b17e0 0x7fb9580cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:02.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.652+0000 7fb95fef7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb958161ce0 0x7fb95806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:02.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.652+0000 7fb95fef7700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb958161850 con 0x7fb9580b1400 2023-08-27T11:19:02.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.652+0000 7fb95fef7700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9580cfc10 con 0x7fb9580b17e0 2023-08-27T11:19:02.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.652+0000 7fb95fef7700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb958161200 con 0x7fb958161ce0 2023-08-27T11:19:02.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.652+0000 7fb95d492700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb9580b17e0 0x7fb9580cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:02.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.652+0000 7fb95d492700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb9580b17e0 0x7fb9580cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58518/0 (socket says 172.21.15.112:58518) 2023-08-27T11:19:02.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.652+0000 7fb95d492700 1 -- 172.21.15.112:0/3320725904 learned_addr learned my addr 172.21.15.112:0/3320725904 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:02.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.653+0000 7fb95d492700 1 -- 172.21.15.112:0/3320725904 >> v2:172.21.15.138:3300/0 conn(0x7fb958161ce0 msgr2=0x7fb95806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:02.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.653+0000 7fb95e494700 1 --2- 172.21.15.112:0/3320725904 >> v2:172.21.15.138:3300/0 conn(0x7fb958161ce0 0x7fb95806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:02.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.653+0000 7fb95d492700 1 --2- 172.21.15.112:0/3320725904 >> v2:172.21.15.138:3300/0 conn(0x7fb958161ce0 0x7fb95806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.653+0000 7fb95d492700 1 -- 172.21.15.112:0/3320725904 >> v1:172.21.15.138:6789/0 conn(0x7fb9580b1400 legacy=0x7fb95806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.653+0000 7fb95d492700 1 -- 172.21.15.112:0/3320725904 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9581614e0 con 0x7fb9580b17e0 2023-08-27T11:19:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.653+0000 7fb95d492700 1 --2- 172.21.15.112:0/3320725904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9580b17e0 0x7fb9580cf5a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb95806df10 tx=0x7fb954009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83760b0c67c8a7db server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.653+0000 7fb95cc91700 1 -- 172.21.15.112:0/3320725904 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954004230 con 0x7fb9580b17e0 2023-08-27T11:19:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.653+0000 7fb95cc91700 1 -- 172.21.15.112:0/3320725904 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb954004390 con 0x7fb9580b17e0 2023-08-27T11:19:02.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.653+0000 7fb95cc91700 1 -- 172.21.15.112:0/3320725904 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9540036a0 con 0x7fb9580b17e0 2023-08-27T11:19:02.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.654+0000 7fb95fef7700 1 -- 172.21.15.112:0/3320725904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9580b17e0 msgr2=0x7fb9580cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:02.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.654+0000 7fb95fef7700 1 --2- 172.21.15.112:0/3320725904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9580b17e0 0x7fb9580cf5a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb95806df10 tx=0x7fb954009bd0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.654+0000 7fb95fef7700 1 -- 172.21.15.112:0/3320725904 shutdown_connections 2023-08-27T11:19:02.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.654+0000 7fb95fef7700 1 --2- 172.21.15.112:0/3320725904 >> v2:172.21.15.138:3300/0 conn(0x7fb958161ce0 0x7fb95806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.654+0000 7fb95fef7700 1 --2- 172.21.15.112:0/3320725904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9580b17e0 0x7fb9580cf5a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.654+0000 7fb95fef7700 1 -- 172.21.15.112:0/3320725904 >> 172.21.15.112:0/3320725904 conn(0x7fb9580d0000 msgr2=0x7fb9580d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:02.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.654+0000 7fb95fef7700 1 -- 172.21.15.112:0/3320725904 shutdown_connections 2023-08-27T11:19:02.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.654+0000 7fb95fef7700 1 -- 172.21.15.112:0/3320725904 wait complete. 2023-08-27T11:19:02.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.655+0000 7fb95fef7700 1 Processor -- start 2023-08-27T11:19:02.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.655+0000 7fb95fef7700 1 -- start start 2023-08-27T11:19:02.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95fef7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9580b1400 0x7fb9581f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:02.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95fef7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9580b17e0 0x7fb95815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:02.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95fef7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb958161ce0 0x7fb95815de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:02.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95fef7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580c6af0 con 0x7fb9580b1400 2023-08-27T11:19:02.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95fef7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9581f28c0 con 0x7fb9580b17e0 2023-08-27T11:19:02.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95fef7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9581448c0 con 0x7fb958161ce0 2023-08-27T11:19:02.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95dc93700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9580b1400 0x7fb9581f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:02.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95dc93700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9580b1400 0x7fb9581f6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58522/0 (socket says 172.21.15.112:58522) 2023-08-27T11:19:02.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95dc93700 1 -- 172.21.15.112:0/2150733349 learned_addr learned my addr 172.21.15.112:0/2150733349 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:02.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95e494700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb958161ce0 0x7fb95815de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:02.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95d492700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9580b17e0 0x7fb95815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95dc93700 1 -- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb958161ce0 msgr2=0x7fb95815de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95dc93700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb958161ce0 0x7fb95815de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95dc93700 1 -- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9580b17e0 msgr2=0x7fb95815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95dc93700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9580b17e0 0x7fb95815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95dc93700 1 -- 172.21.15.112:0/2150733349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb954009550 con 0x7fb9580b1400 2023-08-27T11:19:02.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95e494700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb958161ce0 0x7fb95815de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:02.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95dc93700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9580b1400 0x7fb9581f6960 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb94800b470 tx=0x7fb94800b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:02.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb94e7fc700 1 -- 172.21.15.112:0/2150733349 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948011830 con 0x7fb9580b1400 2023-08-27T11:19:02.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb94e7fc700 1 -- 172.21.15.112:0/2150733349 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb948011990 con 0x7fb9580b1400 2023-08-27T11:19:02.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb94e7fc700 1 -- 172.21.15.112:0/2150733349 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948011d40 con 0x7fb9580b1400 2023-08-27T11:19:02.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95fef7700 1 -- 172.21.15.112:0/2150733349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9581f6ea0 con 0x7fb9580b1400 2023-08-27T11:19:02.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.656+0000 7fb95fef7700 1 -- 172.21.15.112:0/2150733349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9581483f0 con 0x7fb9580b1400 2023-08-27T11:19:02.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.657+0000 7fb95fef7700 1 -- 172.21.15.112:0/2150733349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb958067a20 con 0x7fb9580b1400 2023-08-27T11:19:02.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.658+0000 7fb94e7fc700 1 -- 172.21.15.112:0/2150733349 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fb94801a400 con 0x7fb9580b1400 2023-08-27T11:19:02.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.658+0000 7fb94e7fc700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb93c06a170 0x7fb93c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:02.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.658+0000 7fb94e7fc700 1 -- 172.21.15.112:0/2150733349 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fb948088650 con 0x7fb9580b1400 2023-08-27T11:19:02.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.658+0000 7fb95d492700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb93c06a170 0x7fb93c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:02.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.659+0000 7fb95d492700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb93c06a170 0x7fb93c06c630 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb9580b1bf0 tx=0x7fb954008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:02.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.662+0000 7fb94e7fc700 1 -- 172.21.15.112:0/2150733349 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb948053cd0 con 0x7fb9580b1400 2023-08-27T11:19:02.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.894+0000 7fb95fef7700 1 -- 172.21.15.112:0/2150733349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb95806db60 con 0x7fb9580b1400 2023-08-27T11:19:02.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.895+0000 7fb94e7fc700 1 -- 172.21.15.112:0/2150733349 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4888 (secure 0 0 0) 0x7fb9480572f0 con 0x7fb9580b1400 2023-08-27T11:19:02.897 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:02.897 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1692518554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1692518554},{"type":"v1","addr":"172.21.15.138:6837","nonce":1692518554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3294671617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3294671617},{"type":"v1","addr":"172.21.15.138:6836","nonce":3294671617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3012780135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3012780135},{"type":"v1","addr":"172.21.15.112:6834","nonce":3012780135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:18:43.281470+0000","modified":"2023-08-27T11:18:47.234075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5661":{"gid":5661,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3423469954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3423469954},{"type":"v1","addr":"172.21.15.112:6835","nonce":3423469954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-27T11:19:02.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.897+0000 7fb95fef7700 1 -- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb93c06a170 msgr2=0x7fb93c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:02.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.897+0000 7fb95fef7700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb93c06a170 0x7fb93c06c630 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb9580b1bf0 tx=0x7fb954008030 comp rx=0 tx=0).stop 2023-08-27T11:19:02.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.897+0000 7fb95fef7700 1 -- 172.21.15.112:0/2150733349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9580b1400 msgr2=0x7fb9581f6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:02.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.897+0000 7fb95fef7700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9580b1400 0x7fb9581f6960 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb94800b470 tx=0x7fb94800b830 comp rx=0 tx=0).stop 2023-08-27T11:19:02.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.897+0000 7fb95fef7700 1 -- 172.21.15.112:0/2150733349 shutdown_connections 2023-08-27T11:19:02.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.897+0000 7fb95fef7700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb93c06a170 0x7fb93c06c630 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.897+0000 7fb95fef7700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb958161ce0 0x7fb95815de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.897+0000 7fb95fef7700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9580b1400 0x7fb9581f6960 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.897+0000 7fb95fef7700 1 --2- 172.21.15.112:0/2150733349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9580b17e0 0x7fb95815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:02.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.897+0000 7fb95fef7700 1 -- 172.21.15.112:0/2150733349 >> 172.21.15.112:0/2150733349 conn(0x7fb9580d0000 msgr2=0x7fb9580cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:02.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.897+0000 7fb95fef7700 1 -- 172.21.15.112:0/2150733349 shutdown_connections 2023-08-27T11:19:02.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:02.897+0000 7fb95fef7700 1 -- 172.21.15.112:0/2150733349 wait complete. 2023-08-27T11:19:02.901 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 31 2023-08-27T11:19:02.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:19:03.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.029+0000 7f69a5098700 1 Processor -- start 2023-08-27T11:19:03.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.029+0000 7f69a5098700 1 -- start start 2023-08-27T11:19:03.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.030+0000 7f69a5098700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f69a006adb0 0x7f69a006b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.030+0000 7f69a5098700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f69a00cdbd0 0x7f69a00ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.030+0000 7f69a5098700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a015eb60 con 0x7f69a0151060 2023-08-27T11:19:03.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.030+0000 7f69a5098700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a006b770 con 0x7f69a006adb0 2023-08-27T11:19:03.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.030+0000 7f69a5098700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a00b02f0 con 0x7f69a00cdbd0 2023-08-27T11:19:03.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.030+0000 7f699e59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f69a006adb0 0x7f69a006b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.030+0000 7f699ed9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f69a0151060 0x7f69a015e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52912/0 (socket says 172.21.15.112:52912) 2023-08-27T11:19:03.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.030+0000 7f699ed9d700 1 -- 172.21.15.112:0/2516109367 learned_addr learned my addr 172.21.15.112:0/2516109367 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:03.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.031+0000 7f699f59e700 1 --2- 172.21.15.112:0/2516109367 >> v2:172.21.15.138:3300/0 conn(0x7f69a00cdbd0 0x7f69a00ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.031+0000 7f699e59c700 1 -- 172.21.15.112:0/2516109367 >> v2:172.21.15.138:3300/0 conn(0x7f69a00cdbd0 msgr2=0x7f69a00ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:03.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.031+0000 7f699e59c700 1 --2- 172.21.15.112:0/2516109367 >> v2:172.21.15.138:3300/0 conn(0x7f69a00cdbd0 0x7f69a00ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.031+0000 7f699e59c700 1 -- 172.21.15.112:0/2516109367 >> v1:172.21.15.112:6789/0 conn(0x7f69a0151060 legacy=0x7f69a015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.031+0000 7f699e59c700 1 -- 172.21.15.112:0/2516109367 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a00b05d0 con 0x7f69a006adb0 2023-08-27T11:19:03.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.031+0000 7f699dd9b700 1 -- 172.21.15.112:0/2516109367 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410946300 0 0) 0x7f69a015eb60 con 0x7f69a0151060 2023-08-27T11:19:03.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.031+0000 7f699f59e700 1 --2- 172.21.15.112:0/2516109367 >> v2:172.21.15.138:3300/0 conn(0x7f69a00cdbd0 0x7f69a00ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:03.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.031+0000 7f699e59c700 1 --2- 172.21.15.112:0/2516109367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69a006adb0 0x7f69a006b230 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f69a00b1960 tx=0x7f6990009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c6acc213d90765 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:03.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.031+0000 7f699dd9b700 1 -- 172.21.15.112:0/2516109367 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990011830 con 0x7f69a006adb0 2023-08-27T11:19:03.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.031+0000 7f699dd9b700 1 -- 172.21.15.112:0/2516109367 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6990011990 con 0x7f69a006adb0 2023-08-27T11:19:03.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.031+0000 7f699dd9b700 1 -- 172.21.15.112:0/2516109367 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990011d20 con 0x7f69a006adb0 2023-08-27T11:19:03.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.031+0000 7f69a5098700 1 -- 172.21.15.112:0/2516109367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69a006adb0 msgr2=0x7f69a006b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.031+0000 7f69a5098700 1 --2- 172.21.15.112:0/2516109367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69a006adb0 0x7f69a006b230 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f69a00b1960 tx=0x7f6990009b20 comp rx=0 tx=0).stop 2023-08-27T11:19:03.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.032+0000 7f69a5098700 1 -- 172.21.15.112:0/2516109367 shutdown_connections 2023-08-27T11:19:03.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.032+0000 7f69a5098700 1 --2- 172.21.15.112:0/2516109367 >> v2:172.21.15.138:3300/0 conn(0x7f69a00cdbd0 0x7f69a00ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.032+0000 7f69a5098700 1 --2- 172.21.15.112:0/2516109367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69a006adb0 0x7f69a006b230 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.032+0000 7f69a5098700 1 -- 172.21.15.112:0/2516109367 >> 172.21.15.112:0/2516109367 conn(0x7f69a00d0000 msgr2=0x7f69a00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:03.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.032+0000 7f69a5098700 1 -- 172.21.15.112:0/2516109367 shutdown_connections 2023-08-27T11:19:03.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.032+0000 7f69a5098700 1 -- 172.21.15.112:0/2516109367 wait complete. 2023-08-27T11:19:03.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.033+0000 7f69a5098700 1 Processor -- start 2023-08-27T11:19:03.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.033+0000 7f69a5098700 1 -- start start 2023-08-27T11:19:03.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f69a5098700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69a006adb0 0x7f69a00c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f69a5098700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69a00cdbd0 0x7f69a014ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f69a5098700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69a0151060 0x7f69a00c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f69a5098700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00c8b20 con 0x7f69a006adb0 2023-08-27T11:19:03.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f69a5098700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a014b140 con 0x7f69a00cdbd0 2023-08-27T11:19:03.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f69a5098700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00c7ea0 con 0x7f69a0151060 2023-08-27T11:19:03.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f699ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69a006adb0 0x7f69a00c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f699ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69a006adb0 0x7f69a00c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58536/0 (socket says 172.21.15.112:58536) 2023-08-27T11:19:03.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f699ed9d700 1 -- 172.21.15.112:0/2902327585 learned_addr learned my addr 172.21.15.112:0/2902327585 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:03.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f699e59c700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69a00cdbd0 0x7f69a014ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f699f59e700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69a0151060 0x7f69a00c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f699ed9d700 1 -- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69a0151060 msgr2=0x7f69a00c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f699ed9d700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69a0151060 0x7f69a00c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f699ed9d700 1 -- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69a00cdbd0 msgr2=0x7f69a014ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f699ed9d700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69a00cdbd0 0x7f69a014ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f699ed9d700 1 -- 172.21.15.112:0/2902327585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69900094a0 con 0x7f69a006adb0 2023-08-27T11:19:03.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f699ed9d700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69a006adb0 0x7f69a00c5c00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f69940011e0 tx=0x7f699400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:03.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f69877fe700 1 -- 172.21.15.112:0/2902327585 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6994004c80 con 0x7f69a006adb0 2023-08-27T11:19:03.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f69877fe700 1 -- 172.21.15.112:0/2902327585 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6994004e00 con 0x7f69a006adb0 2023-08-27T11:19:03.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f69877fe700 1 -- 172.21.15.112:0/2902327585 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6994007310 con 0x7f69a006adb0 2023-08-27T11:19:03.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.034+0000 7f69a5098700 1 -- 172.21.15.112:0/2902327585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a00c8180 con 0x7f69a006adb0 2023-08-27T11:19:03.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.035+0000 7f69a5098700 1 -- 172.21.15.112:0/2902327585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a00c8740 con 0x7f69a006adb0 2023-08-27T11:19:03.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.035+0000 7f69a5098700 1 -- 172.21.15.112:0/2902327585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a0067a20 con 0x7f69a006adb0 2023-08-27T11:19:03.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.036+0000 7f69877fe700 1 -- 172.21.15.112:0/2902327585 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f6994007470 con 0x7f69a006adb0 2023-08-27T11:19:03.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.036+0000 7f69877fe700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f698c06a170 0x7f698c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.036+0000 7f69877fe700 1 -- 172.21.15.112:0/2902327585 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f69940881f0 con 0x7f69a006adb0 2023-08-27T11:19:03.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.036+0000 7f699e59c700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f698c06a170 0x7f698c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.037+0000 7f699e59c700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f698c06a170 0x7f698c06c630 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f69900090b0 tx=0x7f699001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:03.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.041+0000 7f69877fe700 1 -- 172.21.15.112:0/2902327585 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6994053900 con 0x7f69a006adb0 2023-08-27T11:19:03.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.229+0000 7f69a5098700 1 -- 172.21.15.112:0/2902327585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69a00c8d90 con 0x7f69a006adb0 2023-08-27T11:19:03.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.230+0000 7f69877fe700 1 -- 172.21.15.112:0/2902327585 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18452 (secure 0 0 0) 0x7f6994056f20 con 0x7f69a006adb0 2023-08-27T11:19:03.232 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:03.232 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":53,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:18:54.265785+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:18:41.033680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-27T11:18:42.255772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1e","mappings":[{"from":7,"to":2}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]},{"pgid":"10.34","mappings":[{"from":7,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:19:03.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.232+0000 7f69a5098700 1 -- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f698c06a170 msgr2=0x7f698c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.232+0000 7f69a5098700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f698c06a170 0x7f698c06c630 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f69900090b0 tx=0x7f699001c040 comp rx=0 tx=0).stop 2023-08-27T11:19:03.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.232+0000 7f69a5098700 1 -- 172.21.15.112:0/2902327585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69a006adb0 msgr2=0x7f69a00c5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.232+0000 7f69a5098700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69a006adb0 0x7f69a00c5c00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f69940011e0 tx=0x7f699400ca30 comp rx=0 tx=0).stop 2023-08-27T11:19:03.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.232+0000 7f69a5098700 1 -- 172.21.15.112:0/2902327585 shutdown_connections 2023-08-27T11:19:03.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.232+0000 7f69a5098700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f698c06a170 0x7f698c06c630 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.232+0000 7f69a5098700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69a0151060 0x7f69a00c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.232+0000 7f69a5098700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69a006adb0 0x7f69a00c5c00 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.232+0000 7f69a5098700 1 --2- 172.21.15.112:0/2902327585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69a00cdbd0 0x7f69a014ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.232+0000 7f69a5098700 1 -- 172.21.15.112:0/2902327585 >> 172.21.15.112:0/2902327585 conn(0x7f69a00d0000 msgr2=0x7f69a015d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:03.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.232+0000 7f69a5098700 1 -- 172.21.15.112:0/2902327585 shutdown_connections 2023-08-27T11:19:03.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.232+0000 7f69a5098700 1 -- 172.21.15.112:0/2902327585 wait complete. 2023-08-27T11:19:03.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:19:03.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.365+0000 7f6bf0612700 1 Processor -- start 2023-08-27T11:19:03.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.365+0000 7f6bf0612700 1 -- start start 2023-08-27T11:19:03.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.366+0000 7f6bf0612700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be8146c00 0x7f6be815e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.366+0000 7f6bf0612700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6be8066720 0x7f6be80708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.366+0000 7f6bf0612700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be8072250 con 0x7f6be80cec70 2023-08-27T11:19:03.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.366+0000 7f6bf0612700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be80d0b60 con 0x7f6be8066720 2023-08-27T11:19:03.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.366+0000 7f6bf0612700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be80724a0 con 0x7f6be8146c00 2023-08-27T11:19:03.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.366+0000 7f6bedbad700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6be80cec70 0x7f6be8066010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52928/0 (socket says 172.21.15.112:52928) 2023-08-27T11:19:03.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.366+0000 7f6bedbad700 1 -- 172.21.15.112:0/1635851659 learned_addr learned my addr 172.21.15.112:0/1635851659 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:03.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6beebaf700 1 --2- 172.21.15.112:0/1635851659 >> v2:172.21.15.138:3300/0 conn(0x7f6be8066720 0x7f6be80708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bee3ae700 1 --2- 172.21.15.112:0/1635851659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be8146c00 0x7f6be815e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074329738 0 0) 0x7f6be8072250 con 0x7f6be80cec70 2023-08-27T11:19:03.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bd40035b0 con 0x7f6be80cec70 2023-08-27T11:19:03.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 545293626 0 0) 0x7f6bd40035b0 con 0x7f6be80cec70 2023-08-27T11:19:03.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be8146c00 msgr2=0x7f6be815e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:03.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 --2- 172.21.15.112:0/1635851659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be8146c00 0x7f6be815e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 >> v2:172.21.15.138:3300/0 conn(0x7f6be8066720 msgr2=0x7f6be80708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:03.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 --2- 172.21.15.112:0/1635851659 >> v2:172.21.15.138:3300/0 conn(0x7f6be8066720 0x7f6be80708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be8072780 con 0x7f6be80cec70 2023-08-27T11:19:03.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f6be4001670 con 0x7f6be80cec70 2023-08-27T11:19:03.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 >> v1:172.21.15.112:6789/0 conn(0x7f6be80cec70 legacy=0x7f6be8066010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 --2- 172.21.15.112:0/1635851659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bd4003a00 0x7f6bd4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 --2- 172.21.15.112:0/1635851659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd4004850 0x7f6bd4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 --2- 172.21.15.112:0/1635851659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be8146c00 0x7f6bd4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be80d0b60 con 0x7f6bd4003a00 2023-08-27T11:19:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be80724a0 con 0x7f6bd4004850 2023-08-27T11:19:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd400ba90 con 0x7f6be8146c00 2023-08-27T11:19:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bee3ae700 1 --2- 172.21.15.112:0/1635851659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bd4003a00 0x7f6bd4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bedbad700 1 --2- 172.21.15.112:0/1635851659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd4004850 0x7f6bd4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6beebaf700 1 --2- 172.21.15.112:0/1635851659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be8146c00 0x7f6bd4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bee3ae700 1 -- 172.21.15.112:0/1635851659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be8146c00 msgr2=0x7f6bd4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bee3ae700 1 --2- 172.21.15.112:0/1635851659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be8146c00 0x7f6bd4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bee3ae700 1 -- 172.21.15.112:0/1635851659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd4004850 msgr2=0x7f6bd4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:03.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bee3ae700 1 --2- 172.21.15.112:0/1635851659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd4004850 0x7f6bd4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bee3ae700 1 -- 172.21.15.112:0/1635851659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd40035b0 con 0x7f6bd4003a00 2023-08-27T11:19:03.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.367+0000 7f6bedbad700 1 --2- 172.21.15.112:0/1635851659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd4004850 0x7f6bd4004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:03.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.368+0000 7f6bee3ae700 1 --2- 172.21.15.112:0/1635851659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bd4003a00 0x7f6bd4003e20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009b60 tx=0x7f6bd4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c8484620e31750 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:03.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.368+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800e6c0 con 0x7f6bd4003a00 2023-08-27T11:19:03.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.368+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd800e820 con 0x7f6bd4003a00 2023-08-27T11:19:03.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.368+0000 7f6bed3ac700 1 -- 172.21.15.112:0/1635851659 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800eb90 con 0x7f6bd4003a00 2023-08-27T11:19:03.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.368+0000 7f6bf0612700 1 -- 172.21.15.112:0/1635851659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bd4003a00 msgr2=0x7f6bd4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.368+0000 7f6bf0612700 1 --2- 172.21.15.112:0/1635851659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bd4003a00 0x7f6bd4003e20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009b60 tx=0x7f6bd4005880 comp rx=0 tx=0).stop 2023-08-27T11:19:03.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.368+0000 7f6beebaf700 1 -- 172.21.15.112:0/1635851659 reap_dead start 2023-08-27T11:19:03.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.368+0000 7f6bf0612700 1 -- 172.21.15.112:0/1635851659 shutdown_connections 2023-08-27T11:19:03.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.368+0000 7f6bf0612700 1 -- 172.21.15.112:0/1635851659 >> 172.21.15.112:0/1635851659 conn(0x7f6be806f970 msgr2=0x7f6be806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:03.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.368+0000 7f6bf0612700 1 -- 172.21.15.112:0/1635851659 shutdown_connections 2023-08-27T11:19:03.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.368+0000 7f6bf0612700 1 -- 172.21.15.112:0/1635851659 wait complete. 2023-08-27T11:19:03.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.369+0000 7f6bf0612700 1 Processor -- start 2023-08-27T11:19:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.369+0000 7f6bf0612700 1 -- start start 2023-08-27T11:19:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.370+0000 7f6bf0612700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bd4003a00 0x7f6be81f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.370+0000 7f6bf0612700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be8144520 0x7f6be81f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.370+0000 7f6bf0612700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be81f6980 0x7f6be8142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.370+0000 7f6bf0612700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be8142070 con 0x7f6bd4003a00 2023-08-27T11:19:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.370+0000 7f6bf0612700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be81f2d00 con 0x7f6be8144520 2023-08-27T11:19:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.370+0000 7f6bf0612700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be81f6e90 con 0x7f6be81f6980 2023-08-27T11:19:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.370+0000 7f6bee3ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bd4003a00 0x7f6be81f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.370+0000 7f6bee3ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bd4003a00 0x7f6be81f27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58556/0 (socket says 172.21.15.112:58556) 2023-08-27T11:19:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.370+0000 7f6bee3ae700 1 -- 172.21.15.112:0/132127649 learned_addr learned my addr 172.21.15.112:0/132127649 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:03.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.370+0000 7f6bee3ae700 1 -- 172.21.15.112:0/132127649 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be81f6980 msgr2=0x7f6be8142500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.371+0000 7f6bee3ae700 1 --2- 172.21.15.112:0/132127649 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be81f6980 0x7f6be8142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.371+0000 7f6bee3ae700 1 -- 172.21.15.112:0/132127649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be8144520 msgr2=0x7f6be81f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.371+0000 7f6bee3ae700 1 --2- 172.21.15.112:0/132127649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be8144520 0x7f6be81f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.371+0000 7f6bee3ae700 1 -- 172.21.15.112:0/132127649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be81f7170 con 0x7f6bd4003a00 2023-08-27T11:19:03.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.371+0000 7f6bee3ae700 1 --2- 172.21.15.112:0/132127649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bd4003a00 0x7f6be81f27c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4005830 tx=0x7f6bd80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:03.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.371+0000 7f6bdeffd700 1 -- 172.21.15.112:0/132127649 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8021070 con 0x7f6bd4003a00 2023-08-27T11:19:03.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.371+0000 7f6bdeffd700 1 -- 172.21.15.112:0/132127649 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd8004550 con 0x7f6bd4003a00 2023-08-27T11:19:03.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.371+0000 7f6bf0612700 1 -- 172.21.15.112:0/132127649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be8141d00 con 0x7f6bd4003a00 2023-08-27T11:19:03.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.371+0000 7f6bdeffd700 1 -- 172.21.15.112:0/132127649 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800e6c0 con 0x7f6bd4003a00 2023-08-27T11:19:03.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.371+0000 7f6bf0612700 1 -- 172.21.15.112:0/132127649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be8142a40 con 0x7f6bd4003a00 2023-08-27T11:19:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.372+0000 7f6bf0612700 1 -- 172.21.15.112:0/132127649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be8141b40 con 0x7f6bd4003a00 2023-08-27T11:19:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.372+0000 7f6bdeffd700 1 -- 172.21.15.112:0/132127649 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f6bd8025070 con 0x7f6bd4003a00 2023-08-27T11:19:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.373+0000 7f6bdeffd700 1 --2- 172.21.15.112:0/132127649 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6bcc06a170 0x7f6bcc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.373+0000 7f6bdeffd700 1 -- 172.21.15.112:0/132127649 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f6bd80899e0 con 0x7f6bd4003a00 2023-08-27T11:19:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.373+0000 7f6bedbad700 1 --2- 172.21.15.112:0/132127649 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6bcc06a170 0x7f6bcc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.373+0000 7f6bedbad700 1 --2- 172.21.15.112:0/132127649 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6bcc06a170 0x7f6bcc06c630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6be4001ba0 tx=0x7f6be4001080 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:03.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.377+0000 7f6bdeffd700 1 -- 172.21.15.112:0/132127649 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6bd8055210 con 0x7f6bd4003a00 2023-08-27T11:19:03.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.609+0000 7f6bf0612700 1 -- 172.21.15.112:0/132127649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6be8142350 con 0x7f6bd4003a00 2023-08-27T11:19:03.619 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:19:03.610+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:19:03.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.616+0000 7f6bdeffd700 1 -- 172.21.15.112:0/132127649 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6bd8087e60 con 0x7f6bd4003a00 2023-08-27T11:19:03.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.618+0000 7f6bf0612700 1 -- 172.21.15.112:0/132127649 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6bcc06a170 msgr2=0x7f6bcc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.618+0000 7f6bf0612700 1 --2- 172.21.15.112:0/132127649 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6bcc06a170 0x7f6bcc06c630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6be4001ba0 tx=0x7f6be4001080 comp rx=0 tx=0).stop 2023-08-27T11:19:03.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.618+0000 7f6bf0612700 1 -- 172.21.15.112:0/132127649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bd4003a00 msgr2=0x7f6be81f27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.618+0000 7f6bf0612700 1 --2- 172.21.15.112:0/132127649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bd4003a00 0x7f6be81f27c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4005830 tx=0x7f6bd80047c0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.618+0000 7f6bf0612700 1 -- 172.21.15.112:0/132127649 shutdown_connections 2023-08-27T11:19:03.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.618+0000 7f6bf0612700 1 --2- 172.21.15.112:0/132127649 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6bcc06a170 0x7f6bcc06c630 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.618+0000 7f6bf0612700 1 --2- 172.21.15.112:0/132127649 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be81f6980 0x7f6be8142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.618+0000 7f6bf0612700 1 --2- 172.21.15.112:0/132127649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bd4003a00 0x7f6be81f27c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.618+0000 7f6bf0612700 1 --2- 172.21.15.112:0/132127649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be8144520 0x7f6be81f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.618+0000 7f6bf0612700 1 -- 172.21.15.112:0/132127649 >> 172.21.15.112:0/132127649 conn(0x7f6be806f970 msgr2=0x7f6be80d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:03.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.618+0000 7f6bf0612700 1 -- 172.21.15.112:0/132127649 shutdown_connections 2023-08-27T11:19:03.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.618+0000 7f6bf0612700 1 -- 172.21.15.112:0/132127649 wait complete. 2023-08-27T11:19:03.623 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:19:03.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:19:03.650 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:19:03.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.755+0000 7ffb11f94700 1 Processor -- start 2023-08-27T11:19:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.755+0000 7ffb11f94700 1 -- start start 2023-08-27T11:19:03.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb11f94700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0c15e190 0x7ffb0c15e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb11f94700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffb0c161dd0 0x7ffb0c0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb11f94700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb0c0cc980 0x7ffb0c0cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb11f94700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0c0cd350 con 0x7ffb0c0cc980 2023-08-27T11:19:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb11f94700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0c0b46c0 con 0x7ffb0c161dd0 2023-08-27T11:19:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb11f94700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0b49a0 con 0x7ffb0c15e190 2023-08-27T11:19:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb0bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb0c0cc980 0x7ffb0c0cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb0bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb0c0cc980 0x7ffb0c0cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58580/0 (socket says 172.21.15.112:58580) 2023-08-27T11:19:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb0bfff700 1 -- 172.21.15.112:0/14101212 learned_addr learned my addr 172.21.15.112:0/14101212 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb0b7fe700 1 --2- 172.21.15.112:0/14101212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0c15e190 0x7ffb0c15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb0affd700 1 --2- 172.21.15.112:0/14101212 >> v2:172.21.15.138:3300/0 conn(0x7ffb0c161dd0 0x7ffb0c0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb0bfff700 1 -- 172.21.15.112:0/14101212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0c15e190 msgr2=0x7ffb0c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb0bfff700 1 --2- 172.21.15.112:0/14101212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0c15e190 0x7ffb0c15e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb0bfff700 1 -- 172.21.15.112:0/14101212 >> v2:172.21.15.138:3300/0 conn(0x7ffb0c161dd0 msgr2=0x7ffb0c0b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb0bfff700 1 --2- 172.21.15.112:0/14101212 >> v2:172.21.15.138:3300/0 conn(0x7ffb0c161dd0 0x7ffb0c0b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb0bfff700 1 -- 172.21.15.112:0/14101212 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c1612f0 con 0x7ffb0c0cc980 2023-08-27T11:19:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb0b7fe700 1 --2- 172.21.15.112:0/14101212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0c15e190 0x7ffb0c15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.756+0000 7ffb0bfff700 1 --2- 172.21.15.112:0/14101212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c0cc980 0x7ffb0c0cce10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0d0c90 tx=0x7ffafc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd1ec039676c10 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.757+0000 7ffb0a7fc700 1 -- 172.21.15.112:0/14101212 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00b870 con 0x7ffb0c0cc980 2023-08-27T11:19:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.757+0000 7ffb0a7fc700 1 -- 172.21.15.112:0/14101212 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc00b9d0 con 0x7ffb0c0cc980 2023-08-27T11:19:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.757+0000 7ffb0a7fc700 1 -- 172.21.15.112:0/14101212 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00bd60 con 0x7ffb0c0cc980 2023-08-27T11:19:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.757+0000 7ffb11f94700 1 -- 172.21.15.112:0/14101212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c0cc980 msgr2=0x7ffb0c0cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.757+0000 7ffb11f94700 1 --2- 172.21.15.112:0/14101212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c0cc980 0x7ffb0c0cce10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0d0c90 tx=0x7ffafc009bd0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.757+0000 7ffb11f94700 1 -- 172.21.15.112:0/14101212 shutdown_connections 2023-08-27T11:19:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.757+0000 7ffb11f94700 1 --2- 172.21.15.112:0/14101212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c0cc980 0x7ffb0c0cce10 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.757+0000 7ffb11f94700 1 --2- 172.21.15.112:0/14101212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0c15e190 0x7ffb0c15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.757+0000 7ffb11f94700 1 --2- 172.21.15.112:0/14101212 >> v2:172.21.15.138:3300/0 conn(0x7ffb0c161dd0 0x7ffb0c0b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.757+0000 7ffb11f94700 1 -- 172.21.15.112:0/14101212 >> 172.21.15.112:0/14101212 conn(0x7ffb0c06bbf0 msgr2=0x7ffb0c0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.757+0000 7ffb11f94700 1 -- 172.21.15.112:0/14101212 shutdown_connections 2023-08-27T11:19:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.757+0000 7ffb11f94700 1 -- 172.21.15.112:0/14101212 wait complete. 2023-08-27T11:19:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.758+0000 7ffb11f94700 1 Processor -- start 2023-08-27T11:19:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.759+0000 7ffb11f94700 1 -- start start 2023-08-27T11:19:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.759+0000 7ffb11f94700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c0cc980 0x7ffb0c0b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.759+0000 7ffb11f94700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb0c15e190 0x7ffb0c0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.759+0000 7ffb11f94700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0c161dd0 0x7ffb0c1475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.759+0000 7ffb11f94700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0ad250 con 0x7ffb0c0cc980 2023-08-27T11:19:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.759+0000 7ffb11f94700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0ad530 con 0x7ffb0c15e190 2023-08-27T11:19:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.759+0000 7ffb11f94700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0ad810 con 0x7ffb0c161dd0 2023-08-27T11:19:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.759+0000 7ffb0b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c0cc980 0x7ffb0c0b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.759+0000 7ffb0b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c0cc980 0x7ffb0c0b14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58582/0 (socket says 172.21.15.112:58582) 2023-08-27T11:19:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.759+0000 7ffb0b7fe700 1 -- 172.21.15.112:0/125786407 learned_addr learned my addr 172.21.15.112:0/125786407 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb0affd700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb0c15e190 0x7ffb0c0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb0bfff700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0c161dd0 0x7ffb0c1475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb0b7fe700 1 -- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0c161dd0 msgr2=0x7ffb0c1475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb0b7fe700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0c161dd0 0x7ffb0c1475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb0b7fe700 1 -- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb0c15e190 msgr2=0x7ffb0c0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:03.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb0b7fe700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb0c15e190 0x7ffb0c0ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:03.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb0b7fe700 1 -- 172.21.15.112:0/125786407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc009550 con 0x7ffb0c0cc980 2023-08-27T11:19:03.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb0affd700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb0c15e190 0x7ffb0c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:03.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb0bfff700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0c161dd0 0x7ffb0c1475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:03.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb0b7fe700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c0cc980 0x7ffb0c0b14d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400cee0 tx=0x7ffb0400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:03.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb03fff700 1 -- 172.21.15.112:0/125786407 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400ae80 con 0x7ffb0c0cc980 2023-08-27T11:19:03.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb03fff700 1 -- 172.21.15.112:0/125786407 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb04015440 con 0x7ffb0c0cc980 2023-08-27T11:19:03.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb11f94700 1 -- 172.21.15.112:0/125786407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c0adaf0 con 0x7ffb0c0cc980 2023-08-27T11:19:03.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb03fff700 1 -- 172.21.15.112:0/125786407 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb040157b0 con 0x7ffb0c0cc980 2023-08-27T11:19:03.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.760+0000 7ffb11f94700 1 -- 172.21.15.112:0/125786407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c0b1a10 con 0x7ffb0c0cc980 2023-08-27T11:19:03.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.761+0000 7ffb11f94700 1 -- 172.21.15.112:0/125786407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c066fa0 con 0x7ffb0c0cc980 2023-08-27T11:19:03.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.761+0000 7ffb03fff700 1 -- 172.21.15.112:0/125786407 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7ffb04015910 con 0x7ffb0c0cc980 2023-08-27T11:19:03.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.762+0000 7ffb03fff700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffaec06a170 0x7ffaec06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:03.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.762+0000 7ffb03fff700 1 -- 172.21.15.112:0/125786407 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6264+0+0 (secure 0 0 0) 0x7ffb04088a40 con 0x7ffb0c0cc980 2023-08-27T11:19:03.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.762+0000 7ffb0affd700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffaec06a170 0x7ffaec06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.763+0000 7ffb0affd700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffaec06a170 0x7ffaec06c630 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c1530c0 tx=0x7ffaf4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.766+0000 7ffb03fff700 1 -- 172.21.15.112:0/125786407 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffb040541b0 con 0x7ffb0c0cc980 2023-08-27T11:19:03.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:03.996+0000 7ffb11f94700 1 -- 172.21.15.112:0/125786407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffb0c0b1c00 con 0x7ffb0c0cc980 2023-08-27T11:19:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.622+0000 7ffb03fff700 1 -- 172.21.15.112:0/125786407 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffb040577d0 con 0x7ffb0c0cc980 2023-08-27T11:19:04.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.624+0000 7ffb11f94700 1 -- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffaec06a170 msgr2=0x7ffaec06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:04.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.624+0000 7ffb11f94700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffaec06a170 0x7ffaec06c630 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c1530c0 tx=0x7ffaf4008040 comp rx=0 tx=0).stop 2023-08-27T11:19:04.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.624+0000 7ffb11f94700 1 -- 172.21.15.112:0/125786407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c0cc980 msgr2=0x7ffb0c0b14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.624+0000 7ffb11f94700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c0cc980 0x7ffb0c0b14d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400cee0 tx=0x7ffb0400a340 comp rx=0 tx=0).stop 2023-08-27T11:19:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.624+0000 7ffb11f94700 1 -- 172.21.15.112:0/125786407 shutdown_connections 2023-08-27T11:19:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.624+0000 7ffb11f94700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffaec06a170 0x7ffaec06c630 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.624+0000 7ffb11f94700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0c161dd0 0x7ffb0c1475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.624+0000 7ffb11f94700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0c0cc980 0x7ffb0c0b14d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.624+0000 7ffb11f94700 1 --2- 172.21.15.112:0/125786407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb0c15e190 0x7ffb0c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.624+0000 7ffb11f94700 1 -- 172.21.15.112:0/125786407 >> 172.21.15.112:0/125786407 conn(0x7ffb0c06bbf0 msgr2=0x7ffb0c0cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.625+0000 7ffb11f94700 1 -- 172.21.15.112:0/125786407 shutdown_connections 2023-08-27T11:19:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.625+0000 7ffb11f94700 1 -- 172.21.15.112:0/125786407 wait complete. 2023-08-27T11:19:04.640 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:19:04.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.807+0000 7fa4594b4700 1 Processor -- start 2023-08-27T11:19:04.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.807+0000 7fa4594b4700 1 -- start start 2023-08-27T11:19:04.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.808+0000 7fa4594b4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa454143dc0 0x7fa4540b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:04.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.808+0000 7fa4594b4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4540b1420 0x7fa45415a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:04.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.808+0000 7fa4594b4700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4540b0b10 con 0x7fa4541439e0 2023-08-27T11:19:04.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.808+0000 7fa4594b4700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45406d850 con 0x7fa454143dc0 2023-08-27T11:19:04.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.808+0000 7fa4594b4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45415b460 con 0x7fa4540b1420 2023-08-27T11:19:04.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.808+0000 7fa4537fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4540b1420 0x7fa45415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.808+0000 7fa4537fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4540b1420 0x7fa45415a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:42018/0 (socket says 172.21.15.112:42018) 2023-08-27T11:19:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.808+0000 7fa4537fe700 1 -- 172.21.15.112:0/44860447 learned_addr learned my addr 172.21.15.112:0/44860447 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.809+0000 7fa4527fc700 1 --2- 172.21.15.112:0/44860447 >> v2:172.21.15.138:3300/0 conn(0x7fa454143dc0 0x7fa4540b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.809+0000 7fa4537fe700 1 -- 172.21.15.112:0/44860447 >> v2:172.21.15.138:3300/0 conn(0x7fa454143dc0 msgr2=0x7fa4540b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.809+0000 7fa4537fe700 1 --2- 172.21.15.112:0/44860447 >> v2:172.21.15.138:3300/0 conn(0x7fa454143dc0 0x7fa4540b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:04.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.809+0000 7fa4537fe700 1 -- 172.21.15.112:0/44860447 >> v1:172.21.15.138:6789/0 conn(0x7fa4541439e0 legacy=0x7fa4540b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:04.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.809+0000 7fa4537fe700 1 -- 172.21.15.112:0/44860447 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45415b740 con 0x7fa4540b1420 2023-08-27T11:19:04.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.809+0000 7fa4537fe700 1 --2- 172.21.15.112:0/44860447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4540b1420 0x7fa45415a810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009e50 tx=0x7fa44c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a35289c87279856 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:04.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.809+0000 7fa451ffb700 1 -- 172.21.15.112:0/44860447 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00e070 con 0x7fa4540b1420 2023-08-27T11:19:04.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.809+0000 7fa451ffb700 1 -- 172.21.15.112:0/44860447 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c00b910 con 0x7fa4540b1420 2023-08-27T11:19:04.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.809+0000 7fa451ffb700 1 -- 172.21.15.112:0/44860447 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00bcc0 con 0x7fa4540b1420 2023-08-27T11:19:04.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.810+0000 7fa4594b4700 1 -- 172.21.15.112:0/44860447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4540b1420 msgr2=0x7fa45415a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:04.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.810+0000 7fa4594b4700 1 --2- 172.21.15.112:0/44860447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4540b1420 0x7fa45415a810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009e50 tx=0x7fa44c00b330 comp rx=0 tx=0).stop 2023-08-27T11:19:04.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.810+0000 7fa4594b4700 1 -- 172.21.15.112:0/44860447 shutdown_connections 2023-08-27T11:19:04.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.810+0000 7fa4594b4700 1 --2- 172.21.15.112:0/44860447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4540b1420 0x7fa45415a810 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:04.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.810+0000 7fa4594b4700 1 --2- 172.21.15.112:0/44860447 >> v2:172.21.15.138:3300/0 conn(0x7fa454143dc0 0x7fa4540b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:04.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.810+0000 7fa4594b4700 1 -- 172.21.15.112:0/44860447 >> 172.21.15.112:0/44860447 conn(0x7fa4540b1be0 msgr2=0x7fa4540b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:04.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.810+0000 7fa4594b4700 1 -- 172.21.15.112:0/44860447 shutdown_connections 2023-08-27T11:19:04.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.810+0000 7fa4594b4700 1 -- 172.21.15.112:0/44860447 wait complete. 2023-08-27T11:19:04.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.811+0000 7fa4594b4700 1 Processor -- start 2023-08-27T11:19:04.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.811+0000 7fa4594b4700 1 -- start start 2023-08-27T11:19:04.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4594b4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4540b1420 0x7fa454158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:04.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4594b4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4541439e0 0x7fa45406f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:04.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4594b4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45415ea20 0x7fa45406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:04.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4594b4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45415ca60 con 0x7fa45415ea20 2023-08-27T11:19:04.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4594b4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45413f300 con 0x7fa4541439e0 2023-08-27T11:19:04.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4594b4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa454159240 con 0x7fa4540b1420 2023-08-27T11:19:04.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4537fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45415ea20 0x7fa45406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:04.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4537fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45415ea20 0x7fa45406fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58602/0 (socket says 172.21.15.112:58602) 2023-08-27T11:19:04.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4537fe700 1 -- 172.21.15.112:0/2143269366 learned_addr learned my addr 172.21.15.112:0/2143269366 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:04.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa452ffd700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4540b1420 0x7fa454158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:04.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4527fc700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4541439e0 0x7fa45406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:04.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4537fe700 1 -- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4540b1420 msgr2=0x7fa454158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:04.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4537fe700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4540b1420 0x7fa454158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:04.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4537fe700 1 -- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4541439e0 msgr2=0x7fa45406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:04.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4537fe700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4541439e0 0x7fa45406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:04.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.812+0000 7fa4537fe700 1 -- 172.21.15.112:0/2143269366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c009b30 con 0x7fa45415ea20 2023-08-27T11:19:04.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.813+0000 7fa4537fe700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45415ea20 0x7fa45406fcd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa4540b0ca0 tx=0x7fa44c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:04.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.813+0000 7fa43b7fe700 1 -- 172.21.15.112:0/2143269366 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c004800 con 0x7fa45415ea20 2023-08-27T11:19:04.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.813+0000 7fa43b7fe700 1 -- 172.21.15.112:0/2143269366 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c004960 con 0x7fa45415ea20 2023-08-27T11:19:04.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.813+0000 7fa4594b4700 1 -- 172.21.15.112:0/2143269366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4541f0810 con 0x7fa45415ea20 2023-08-27T11:19:04.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.813+0000 7fa43b7fe700 1 -- 172.21.15.112:0/2143269366 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c01c680 con 0x7fa45415ea20 2023-08-27T11:19:04.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.813+0000 7fa4594b4700 1 -- 172.21.15.112:0/2143269366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45413ec40 con 0x7fa45415ea20 2023-08-27T11:19:04.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.814+0000 7fa4594b4700 1 -- 172.21.15.112:0/2143269366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45413eaa0 con 0x7fa45415ea20 2023-08-27T11:19:04.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.814+0000 7fa43b7fe700 1 -- 172.21.15.112:0/2143269366 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fa44c01c7e0 con 0x7fa45415ea20 2023-08-27T11:19:04.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.814+0000 7fa43b7fe700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa44006a170 0x7fa44006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:04.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.815+0000 7fa43b7fe700 1 -- 172.21.15.112:0/2143269366 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6264+0+0 (secure 0 0 0) 0x7fa44c08a2d0 con 0x7fa45415ea20 2023-08-27T11:19:04.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.815+0000 7fa452ffd700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa44006a170 0x7fa44006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:04.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.815+0000 7fa452ffd700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa44006a170 0x7fa44006c630 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa4540b3fc0 tx=0x7fa448009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:04.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:04.819+0000 7fa43b7fe700 1 -- 172.21.15.112:0/2143269366 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa45413eaa0 con 0x7fa45415ea20 2023-08-27T11:19:05.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.008+0000 7fa4594b4700 1 -- 172.21.15.112:0/2143269366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa45413f440 con 0x7fa45415ea20 2023-08-27T11:19:05.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.628+0000 7fa43b7fe700 1 -- 172.21.15.112:0/2143269366 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa45413f440 con 0x7fa45415ea20 2023-08-27T11:19:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.631+0000 7fa4594b4700 1 -- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa44006a170 msgr2=0x7fa44006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.631+0000 7fa4594b4700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa44006a170 0x7fa44006c630 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa4540b3fc0 tx=0x7fa448009040 comp rx=0 tx=0).stop 2023-08-27T11:19:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.631+0000 7fa4594b4700 1 -- 172.21.15.112:0/2143269366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45415ea20 msgr2=0x7fa45406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.631+0000 7fa4594b4700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45415ea20 0x7fa45406fcd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa4540b0ca0 tx=0x7fa44c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:19:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.631+0000 7fa4594b4700 1 -- 172.21.15.112:0/2143269366 shutdown_connections 2023-08-27T11:19:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.631+0000 7fa4594b4700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa44006a170 0x7fa44006c630 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.631+0000 7fa4594b4700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45415ea20 0x7fa45406fcd0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.631+0000 7fa4594b4700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4540b1420 0x7fa454158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.631+0000 7fa4594b4700 1 --2- 172.21.15.112:0/2143269366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4541439e0 0x7fa45406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.631+0000 7fa4594b4700 1 -- 172.21.15.112:0/2143269366 >> 172.21.15.112:0/2143269366 conn(0x7fa4540b1be0 msgr2=0x7fa454159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.632+0000 7fa4594b4700 1 -- 172.21.15.112:0/2143269366 shutdown_connections 2023-08-27T11:19:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.632+0000 7fa4594b4700 1 -- 172.21.15.112:0/2143269366 wait complete. 2023-08-27T11:19:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:19:05.652 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:19:05.719 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:19:05.722 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:19:05.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.782+0000 7f483d7e3700 1 Processor -- start 2023-08-27T11:19:05.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.782+0000 7f483d7e3700 1 -- start start 2023-08-27T11:19:05.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f483d7e3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4838157ae0 0x7f483815e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:05.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f483d7e3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48380b3de0 0x7f4838161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f483d7e3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4838161d60 0x7f483806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f483d7e3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48380b1aa0 con 0x7f4838157ae0 2023-08-27T11:19:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f483d7e3700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48380b4bd0 con 0x7f4838161d60 2023-08-27T11:19:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f483d7e3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48381610f0 con 0x7f48380b3de0 2023-08-27T11:19:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f4836ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4838157ae0 0x7f483815e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:05.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f4836ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4838157ae0 0x7f483815e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58606/0 (socket says 172.21.15.112:58606) 2023-08-27T11:19:05.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f4836ffd700 1 -- 172.21.15.112:0/2339350055 learned_addr learned my addr 172.21.15.112:0/2339350055 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:05.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f48377fe700 1 --2- 172.21.15.112:0/2339350055 >> v2:172.21.15.138:3300/0 conn(0x7f4838161d60 0x7f483806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:05.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f48367fc700 1 --2- 172.21.15.112:0/2339350055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48380b3de0 0x7f4838161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:05.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f4836ffd700 1 -- 172.21.15.112:0/2339350055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48380b3de0 msgr2=0x7f4838161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:05.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f4836ffd700 1 --2- 172.21.15.112:0/2339350055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48380b3de0 0x7f4838161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:05.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f4836ffd700 1 -- 172.21.15.112:0/2339350055 >> v2:172.21.15.138:3300/0 conn(0x7f4838161d60 msgr2=0x7f483806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:05.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f4836ffd700 1 --2- 172.21.15.112:0/2339350055 >> v2:172.21.15.138:3300/0 conn(0x7f4838161d60 0x7f483806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:05.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.783+0000 7f4836ffd700 1 -- 172.21.15.112:0/2339350055 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4838161390 con 0x7f4838157ae0 2023-08-27T11:19:05.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.784+0000 7f4836ffd700 1 --2- 172.21.15.112:0/2339350055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838157ae0 0x7f483815e580 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f48381472e0 tx=0x7f482c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f03440182d8a48 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:05.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.784+0000 7f4835ffb700 1 -- 172.21.15.112:0/2339350055 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c00b870 con 0x7f4838157ae0 2023-08-27T11:19:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.784+0000 7f4835ffb700 1 -- 172.21.15.112:0/2339350055 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c00b9d0 con 0x7f4838157ae0 2023-08-27T11:19:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.784+0000 7f4835ffb700 1 -- 172.21.15.112:0/2339350055 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c00bd60 con 0x7f4838157ae0 2023-08-27T11:19:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.784+0000 7f483d7e3700 1 -- 172.21.15.112:0/2339350055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838157ae0 msgr2=0x7f483815e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.784+0000 7f483d7e3700 1 --2- 172.21.15.112:0/2339350055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838157ae0 0x7f483815e580 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f48381472e0 tx=0x7f482c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:19:05.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.784+0000 7f483d7e3700 1 -- 172.21.15.112:0/2339350055 shutdown_connections 2023-08-27T11:19:05.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.784+0000 7f483d7e3700 1 --2- 172.21.15.112:0/2339350055 >> v2:172.21.15.138:3300/0 conn(0x7f4838161d60 0x7f483806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:05.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.784+0000 7f483d7e3700 1 --2- 172.21.15.112:0/2339350055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838157ae0 0x7f483815e580 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:05.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.784+0000 7f483d7e3700 1 --2- 172.21.15.112:0/2339350055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48380b3de0 0x7f4838161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:05.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.784+0000 7f483d7e3700 1 -- 172.21.15.112:0/2339350055 >> 172.21.15.112:0/2339350055 conn(0x7f48380b1be0 msgr2=0x7f48380b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.784+0000 7f483d7e3700 1 -- 172.21.15.112:0/2339350055 shutdown_connections 2023-08-27T11:19:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.784+0000 7f483d7e3700 1 -- 172.21.15.112:0/2339350055 wait complete. 2023-08-27T11:19:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.786+0000 7f483d7e3700 1 Processor -- start 2023-08-27T11:19:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.786+0000 7f483d7e3700 1 -- start start 2023-08-27T11:19:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.786+0000 7f483d7e3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48380b3de0 0x7f48380af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.786+0000 7f483d7e3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4838157ae0 0x7f48381602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.786+0000 7f483d7e3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4838161d60 0x7f4838159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.786+0000 7f483d7e3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48380ade60 con 0x7f48380b3de0 2023-08-27T11:19:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.786+0000 7f483d7e3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48380ae140 con 0x7f4838161d60 2023-08-27T11:19:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.786+0000 7f483d7e3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48380ae420 con 0x7f4838157ae0 2023-08-27T11:19:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f4836ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48380b3de0 0x7f48380af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f4836ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48380b3de0 0x7f48380af260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58612/0 (socket says 172.21.15.112:58612) 2023-08-27T11:19:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f4836ffd700 1 -- 172.21.15.112:0/4274353878 learned_addr learned my addr 172.21.15.112:0/4274353878 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f48367fc700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4838157ae0 0x7f48381602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f48377fe700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4838161d60 0x7f4838159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:05.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f4836ffd700 1 -- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4838157ae0 msgr2=0x7f48381602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:05.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f4836ffd700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4838157ae0 0x7f48381602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f4836ffd700 1 -- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4838161d60 msgr2=0x7f4838159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f4836ffd700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4838161d60 0x7f4838159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f4836ffd700 1 -- 172.21.15.112:0/4274353878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c009550 con 0x7f48380b3de0 2023-08-27T11:19:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f48377fe700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4838161d60 0x7f4838159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f4836ffd700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48380b3de0 0x7f48380af260 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f482c009c10 tx=0x7f482c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:05.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f481f7fe700 1 -- 172.21.15.112:0/4274353878 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c004250 con 0x7f48380b3de0 2023-08-27T11:19:05.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f481f7fe700 1 -- 172.21.15.112:0/4274353878 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c01f070 con 0x7f48380b3de0 2023-08-27T11:19:05.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f483d7e3700 1 -- 172.21.15.112:0/4274353878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48380ae700 con 0x7f48380b3de0 2023-08-27T11:19:05.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f481f7fe700 1 -- 172.21.15.112:0/4274353878 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c0045d0 con 0x7f48380b3de0 2023-08-27T11:19:05.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.787+0000 7f483d7e3700 1 -- 172.21.15.112:0/4274353878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48380ada80 con 0x7f48380b3de0 2023-08-27T11:19:05.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.788+0000 7f483d7e3700 1 -- 172.21.15.112:0/4274353878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48380adfa0 con 0x7f48380b3de0 2023-08-27T11:19:05.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.788+0000 7f481f7fe700 1 -- 172.21.15.112:0/4274353878 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f482c021030 con 0x7f48380b3de0 2023-08-27T11:19:05.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.789+0000 7f481f7fe700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f482406a170 0x7f482406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:05.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.789+0000 7f481f7fe700 1 -- 172.21.15.112:0/4274353878 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5797+0+0 (secure 0 0 0) 0x7f482c089860 con 0x7f48380b3de0 2023-08-27T11:19:05.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.789+0000 7f48367fc700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f482406a170 0x7f482406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:05.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.790+0000 7f48367fc700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f482406a170 0x7f482406c630 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f483813f590 tx=0x7f4828008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:05.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.793+0000 7f481f7fe700 1 -- 172.21.15.112:0/4274353878 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f48380adfa0 con 0x7f48380b3de0 2023-08-27T11:19:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:05.984+0000 7f483d7e3700 1 -- 172.21.15.112:0/4274353878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f48380adfa0 con 0x7f48380b3de0 2023-08-27T11:19:06.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.627+0000 7f481f7fe700 1 -- 172.21.15.112:0/4274353878 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7f48380adfa0 con 0x7f48380b3de0 2023-08-27T11:19:06.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.630+0000 7f483d7e3700 1 -- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f482406a170 msgr2=0x7f482406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:06.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.630+0000 7f483d7e3700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f482406a170 0x7f482406c630 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f483813f590 tx=0x7f4828008040 comp rx=0 tx=0).stop 2023-08-27T11:19:06.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.630+0000 7f483d7e3700 1 -- 172.21.15.112:0/4274353878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48380b3de0 msgr2=0x7f48380af260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:06.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.630+0000 7f483d7e3700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48380b3de0 0x7f48380af260 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f482c009c10 tx=0x7f482c004750 comp rx=0 tx=0).stop 2023-08-27T11:19:06.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.630+0000 7f483d7e3700 1 -- 172.21.15.112:0/4274353878 shutdown_connections 2023-08-27T11:19:06.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.630+0000 7f483d7e3700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f482406a170 0x7f482406c630 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:06.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.630+0000 7f483d7e3700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4838161d60 0x7f4838159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:06.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.630+0000 7f483d7e3700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48380b3de0 0x7f48380af260 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:06.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.630+0000 7f483d7e3700 1 --2- 172.21.15.112:0/4274353878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4838157ae0 0x7f48381602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:06.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.630+0000 7f483d7e3700 1 -- 172.21.15.112:0/4274353878 >> 172.21.15.112:0/4274353878 conn(0x7f48380b1be0 msgr2=0x7f48380d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:06.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.631+0000 7f483d7e3700 1 -- 172.21.15.112:0/4274353878 shutdown_connections 2023-08-27T11:19:06.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.631+0000 7f483d7e3700 1 -- 172.21.15.112:0/4274353878 wait complete. 2023-08-27T11:19:06.635 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2023-08-27T11:19:06.646 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:19:06.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.811+0000 7f0ea5858700 1 Processor -- start 2023-08-27T11:19:06.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.811+0000 7f0ea5858700 1 -- start start 2023-08-27T11:19:06.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0ea5858700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea0066720 0x7f0ea00ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:06.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0ea5858700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea00cead0 con 0x7f0ea015b250 2023-08-27T11:19:06.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0ea5858700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea015e000 con 0x7f0ea015db70 2023-08-27T11:19:06.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0ea5858700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea015b630 con 0x7f0ea0066720 2023-08-27T11:19:06.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9effd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0ea015b250 0x7f0ea015ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53002/0 (socket says 172.21.15.112:53002) 2023-08-27T11:19:06.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9effd700 1 -- 172.21.15.112:0/2482860477 learned_addr learned my addr 172.21.15.112:0/2482860477 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:06.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9f7fe700 1 --2- 172.21.15.112:0/2482860477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea0066720 0x7f0ea00ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:06.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189352474 0 0) 0x7f0ea00cead0 con 0x7f0ea015b250 2023-08-27T11:19:06.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e880035b0 con 0x7f0ea015b250 2023-08-27T11:19:06.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2448522475 0 0) 0x7f0e880035b0 con 0x7f0ea015b250 2023-08-27T11:19:06.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea0066720 msgr2=0x7f0ea00ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:06.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 --2- 172.21.15.112:0/2482860477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea0066720 0x7f0ea00ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:06.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 >> v1:172.21.15.138:6789/0 conn(0x7f0ea015db70 legacy=0x7f0ea0066010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:06.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea015f440 con 0x7f0ea015b250 2023-08-27T11:19:06.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f0e90001670 con 0x7f0ea015b250 2023-08-27T11:19:06.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 >> v1:172.21.15.112:6789/0 conn(0x7f0ea015b250 legacy=0x7f0ea015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:06.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 --2- 172.21.15.112:0/2482860477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea0066720 0x7f0e88003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:06.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 --2- 172.21.15.112:0/2482860477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e88003fb0 0x7f0e88004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:06.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 --2- 172.21.15.112:0/2482860477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e88004d90 0x7f0e88005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:06.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea015b630 con 0x7f0e88003fb0 2023-08-27T11:19:06.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8800bba0 con 0x7f0e88004d90 2023-08-27T11:19:06.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8800bd60 con 0x7f0ea0066720 2023-08-27T11:19:06.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.812+0000 7f0e9e7fc700 1 --2- 172.21.15.112:0/2482860477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e88003fb0 0x7f0e88004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:06.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.813+0000 7f0e9effd700 1 --2- 172.21.15.112:0/2482860477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea0066720 0x7f0e88003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:06.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.813+0000 7f0e9f7fe700 1 --2- 172.21.15.112:0/2482860477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e88004d90 0x7f0e88005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:06.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.813+0000 7f0e9e7fc700 1 -- 172.21.15.112:0/2482860477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea0066720 msgr2=0x7f0e88003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:06.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.813+0000 7f0e9e7fc700 1 --2- 172.21.15.112:0/2482860477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea0066720 0x7f0e88003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:06.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.813+0000 7f0e9e7fc700 1 -- 172.21.15.112:0/2482860477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e88004d90 msgr2=0x7f0e88005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:06.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.813+0000 7f0e9e7fc700 1 --2- 172.21.15.112:0/2482860477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e88004d90 0x7f0e88005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:06.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.813+0000 7f0e9e7fc700 1 -- 172.21.15.112:0/2482860477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e880035b0 con 0x7f0e88003fb0 2023-08-27T11:19:06.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.813+0000 7f0e9f7fe700 1 --2- 172.21.15.112:0/2482860477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e88004d90 0x7f0e88005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:06.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.813+0000 7f0e9e7fc700 1 --2- 172.21.15.112:0/2482860477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e88003fb0 0x7f0e88004850 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0e940017c0 tx=0x7f0e88005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9970e83f21a89b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:06.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.813+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e940052d0 con 0x7f0e88003fb0 2023-08-27T11:19:06.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.813+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e94005430 con 0x7f0e88003fb0 2023-08-27T11:19:06.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.813+0000 7f0e9dffb700 1 -- 172.21.15.112:0/2482860477 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e940038c0 con 0x7f0e88003fb0 2023-08-27T11:19:06.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.814+0000 7f0ea5858700 1 -- 172.21.15.112:0/2482860477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e88003fb0 msgr2=0x7f0e88004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:06.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.814+0000 7f0ea5858700 1 --2- 172.21.15.112:0/2482860477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e88003fb0 0x7f0e88004850 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0e940017c0 tx=0x7f0e88005880 comp rx=0 tx=0).stop 2023-08-27T11:19:06.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.814+0000 7f0e9f7fe700 1 -- 172.21.15.112:0/2482860477 reap_dead start 2023-08-27T11:19:06.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.814+0000 7f0ea5858700 1 -- 172.21.15.112:0/2482860477 shutdown_connections 2023-08-27T11:19:06.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.814+0000 7f0ea5858700 1 -- 172.21.15.112:0/2482860477 >> 172.21.15.112:0/2482860477 conn(0x7f0ea006f790 msgr2=0x7f0ea006fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:06.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.814+0000 7f0ea5858700 1 -- 172.21.15.112:0/2482860477 shutdown_connections 2023-08-27T11:19:06.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.814+0000 7f0ea5858700 1 -- 172.21.15.112:0/2482860477 wait complete. 2023-08-27T11:19:06.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.815+0000 7f0ea5858700 1 Processor -- start 2023-08-27T11:19:06.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.815+0000 7f0ea5858700 1 -- start start 2023-08-27T11:19:06.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0ea5858700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e88003fb0 0x7f0ea00cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:06.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0ea5858700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea01566a0 0x7f0ea0070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:06.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0ea5858700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea00cd680 0x7f0ea0071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:06.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0ea5858700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea00b3fe0 con 0x7f0ea00cd680 2023-08-27T11:19:06.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0ea5858700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea00cb800 con 0x7f0e88003fb0 2023-08-27T11:19:06.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0ea5858700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea00b4280 con 0x7f0ea01566a0 2023-08-27T11:19:06.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e9f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea00cd680 0x7f0ea0071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e9f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea00cd680 0x7f0ea0071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58636/0 (socket says 172.21.15.112:58636) 2023-08-27T11:19:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e9f7fe700 1 -- 172.21.15.112:0/2480916894 learned_addr learned my addr 172.21.15.112:0/2480916894 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e9effd700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e88003fb0 0x7f0ea00cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e9e7fc700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea01566a0 0x7f0ea0070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e9f7fe700 1 -- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea01566a0 msgr2=0x7f0ea0070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e9f7fe700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea01566a0 0x7f0ea0070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e9f7fe700 1 -- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e88003fb0 msgr2=0x7f0ea00cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e9f7fe700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e88003fb0 0x7f0ea00cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e9f7fe700 1 -- 172.21.15.112:0/2480916894 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea00ca8d0 con 0x7f0ea00cd680 2023-08-27T11:19:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e9f7fe700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea00cd680 0x7f0ea0071330 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0066e30 tx=0x7f0e980104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e87fff700 1 -- 172.21.15.112:0/2480916894 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9801b070 con 0x7f0ea00cd680 2023-08-27T11:19:06.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e87fff700 1 -- 172.21.15.112:0/2480916894 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9800f040 con 0x7f0ea00cd680 2023-08-27T11:19:06.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0ea5858700 1 -- 172.21.15.112:0/2480916894 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea00cabb0 con 0x7f0ea00cd680 2023-08-27T11:19:06.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0e87fff700 1 -- 172.21.15.112:0/2480916894 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9800e050 con 0x7f0ea00cd680 2023-08-27T11:19:06.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.816+0000 7f0ea5858700 1 -- 172.21.15.112:0/2480916894 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea015c7c0 con 0x7f0ea00cd680 2023-08-27T11:19:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.817+0000 7f0ea5858700 1 -- 172.21.15.112:0/2480916894 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea0070710 con 0x7f0ea00cd680 2023-08-27T11:19:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.818+0000 7f0e87fff700 1 -- 172.21.15.112:0/2480916894 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f0e98010d40 con 0x7f0ea00cd680 2023-08-27T11:19:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.818+0000 7f0e87fff700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e8c06a170 0x7f0e8c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.818+0000 7f0e87fff700 1 -- 172.21.15.112:0/2480916894 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0e98088640 con 0x7f0ea00cd680 2023-08-27T11:19:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.818+0000 7f0e9effd700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e8c06a170 0x7f0e8c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.819+0000 7f0e9effd700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e8c06a170 0x7f0e8c06c630 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0e90001050 tx=0x7f0e90000fe0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:06.823+0000 7f0e87fff700 1 -- 172.21.15.112:0/2480916894 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0e98053e50 con 0x7f0ea00cd680 2023-08-27T11:19:07.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.012+0000 7f0ea5858700 1 -- 172.21.15.112:0/2480916894 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0ea0074890 con 0x7f0ea00cd680 2023-08-27T11:19:07.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.016+0000 7f0e87fff700 1 -- 172.21.15.112:0/2480916894 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0e98057470 con 0x7f0ea00cd680 2023-08-27T11:19:07.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.018+0000 7f0ea5858700 1 -- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e8c06a170 msgr2=0x7f0e8c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:07.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.018+0000 7f0ea5858700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e8c06a170 0x7f0e8c06c630 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0e90001050 tx=0x7f0e90000fe0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.018+0000 7f0ea5858700 1 -- 172.21.15.112:0/2480916894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea00cd680 msgr2=0x7f0ea0071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:07.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.018+0000 7f0ea5858700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea00cd680 0x7f0ea0071330 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0066e30 tx=0x7f0e980104a0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.018+0000 7f0ea5858700 1 -- 172.21.15.112:0/2480916894 shutdown_connections 2023-08-27T11:19:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.018+0000 7f0ea5858700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e8c06a170 0x7f0e8c06c630 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.018+0000 7f0ea5858700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea00cd680 0x7f0ea0071330 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.018+0000 7f0ea5858700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e88003fb0 0x7f0ea00cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.018+0000 7f0ea5858700 1 --2- 172.21.15.112:0/2480916894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea01566a0 0x7f0ea0070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.018+0000 7f0ea5858700 1 -- 172.21.15.112:0/2480916894 >> 172.21.15.112:0/2480916894 conn(0x7f0ea006f790 msgr2=0x7f0ea00b6660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:07.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.018+0000 7f0ea5858700 1 -- 172.21.15.112:0/2480916894 shutdown_connections 2023-08-27T11:19:07.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.018+0000 7f0ea5858700 1 -- 172.21.15.112:0/2480916894 wait complete. 2023-08-27T11:19:07.031 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2023-08-27T11:19:07.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.150+0000 7ff7f0d8e700 1 Processor -- start 2023-08-27T11:19:07.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.150+0000 7ff7f0d8e700 1 -- start start 2023-08-27T11:19:07.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.151+0000 7ff7f0d8e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7ec0b17e0 0x7ff7ec0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.151+0000 7ff7f0d8e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7ec161ce0 0x7ff7ec06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.151+0000 7ff7f0d8e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ec161850 con 0x7ff7ec0b1400 2023-08-27T11:19:07.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.151+0000 7ff7f0d8e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ec0cfc10 con 0x7ff7ec161ce0 2023-08-27T11:19:07.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.151+0000 7ff7f0d8e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec161200 con 0x7ff7ec0b17e0 2023-08-27T11:19:07.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.152+0000 7ff7e9d9b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7ec0b17e0 0x7ff7ec0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.152+0000 7ff7ead9d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7ec161ce0 0x7ff7ec06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.152+0000 7ff7e9d9b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7ec0b17e0 0x7ff7ec0cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:42064/0 (socket says 172.21.15.112:42064) 2023-08-27T11:19:07.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.152+0000 7ff7e9d9b700 1 -- 172.21.15.112:0/699587398 learned_addr learned my addr 172.21.15.112:0/699587398 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:07.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.152+0000 7ff7ead9d700 1 -- 172.21.15.112:0/699587398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7ec0b17e0 msgr2=0x7ff7ec0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.152+0000 7ff7ead9d700 1 --2- 172.21.15.112:0/699587398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7ec0b17e0 0x7ff7ec0cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.152+0000 7ff7ead9d700 1 -- 172.21.15.112:0/699587398 >> v1:172.21.15.138:6789/0 conn(0x7ff7ec0b1400 legacy=0x7ff7ec06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.152+0000 7ff7ead9d700 1 -- 172.21.15.112:0/699587398 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec1614e0 con 0x7ff7ec161ce0 2023-08-27T11:19:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.152+0000 7ff7e9d9b700 1 --2- 172.21.15.112:0/699587398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7ec0b17e0 0x7ff7ec0cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:19:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.152+0000 7ff7ead9d700 1 --2- 172.21.15.112:0/699587398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec161ce0 0x7ff7ec06f790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009d90 tx=0x7ff7e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=355f8a8851829f01 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.152+0000 7ff7e959a700 1 -- 172.21.15.112:0/699587398 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400e070 con 0x7ff7ec161ce0 2023-08-27T11:19:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.152+0000 7ff7e959a700 1 -- 172.21.15.112:0/699587398 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e400bb20 con 0x7ff7ec161ce0 2023-08-27T11:19:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.152+0000 7ff7e959a700 1 -- 172.21.15.112:0/699587398 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4012430 con 0x7ff7ec161ce0 2023-08-27T11:19:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.153+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/699587398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec161ce0 msgr2=0x7ff7ec06f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.153+0000 7ff7f0d8e700 1 --2- 172.21.15.112:0/699587398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec161ce0 0x7ff7ec06f790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009d90 tx=0x7ff7e400b330 comp rx=0 tx=0).stop 2023-08-27T11:19:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.153+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/699587398 shutdown_connections 2023-08-27T11:19:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.153+0000 7ff7f0d8e700 1 --2- 172.21.15.112:0/699587398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec161ce0 0x7ff7ec06f790 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.153+0000 7ff7f0d8e700 1 --2- 172.21.15.112:0/699587398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7ec0b17e0 0x7ff7ec0cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.153+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/699587398 >> 172.21.15.112:0/699587398 conn(0x7ff7ec0d0000 msgr2=0x7ff7ec0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.153+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/699587398 shutdown_connections 2023-08-27T11:19:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.153+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/699587398 wait complete. 2023-08-27T11:19:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.154+0000 7ff7f0d8e700 1 Processor -- start 2023-08-27T11:19:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.154+0000 7ff7f0d8e700 1 -- start start 2023-08-27T11:19:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.155+0000 7ff7f0d8e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7ec0b1400 0x7ff7ec071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.155+0000 7ff7f0d8e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec0b17e0 0x7ff7ec0c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.155+0000 7ff7f0d8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec161ce0 0x7ff7ec0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.155+0000 7ff7f0d8e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec0710e0 con 0x7ff7ec161ce0 2023-08-27T11:19:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.155+0000 7ff7f0d8e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec0704d0 con 0x7ff7ec0b17e0 2023-08-27T11:19:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.155+0000 7ff7f0d8e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec0707b0 con 0x7ff7ec0b1400 2023-08-27T11:19:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.155+0000 7ff7ead9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec161ce0 0x7ff7ec0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.155+0000 7ff7ead9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec161ce0 0x7ff7ec0c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58650/0 (socket says 172.21.15.112:58650) 2023-08-27T11:19:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.155+0000 7ff7ead9d700 1 -- 172.21.15.112:0/3362793148 learned_addr learned my addr 172.21.15.112:0/3362793148 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.155+0000 7ff7e9d9b700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec0b17e0 0x7ff7ec0c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.155+0000 7ff7ea59c700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7ec0b1400 0x7ff7ec071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.156+0000 7ff7ead9d700 1 -- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7ec0b1400 msgr2=0x7ff7ec071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.156+0000 7ff7ead9d700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7ec0b1400 0x7ff7ec071570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.156+0000 7ff7ead9d700 1 -- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec0b17e0 msgr2=0x7ff7ec0c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.156+0000 7ff7ead9d700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec0b17e0 0x7ff7ec0c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.156+0000 7ff7ead9d700 1 -- 172.21.15.112:0/3362793148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc009550 con 0x7ff7ec161ce0 2023-08-27T11:19:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.156+0000 7ff7ead9d700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec161ce0 0x7ff7ec0c97e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4014040 tx=0x7ff7e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.156+0000 7ff7daffd700 1 -- 172.21.15.112:0/3362793148 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4012430 con 0x7ff7ec161ce0 2023-08-27T11:19:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.156+0000 7ff7daffd700 1 -- 172.21.15.112:0/3362793148 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e4020070 con 0x7ff7ec161ce0 2023-08-27T11:19:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.156+0000 7ff7daffd700 1 -- 172.21.15.112:0/3362793148 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4003ea0 con 0x7ff7ec161ce0 2023-08-27T11:19:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.156+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/3362793148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e4009a70 con 0x7ff7ec161ce0 2023-08-27T11:19:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.156+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/3362793148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ec070d70 con 0x7ff7ec161ce0 2023-08-27T11:19:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.157+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/3362793148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ec071290 con 0x7ff7ec161ce0 2023-08-27T11:19:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.157+0000 7ff7daffd700 1 -- 172.21.15.112:0/3362793148 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7ff7e4004280 con 0x7ff7ec161ce0 2023-08-27T11:19:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.158+0000 7ff7daffd700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff7cc06a170 0x7ff7cc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.158+0000 7ff7daffd700 1 -- 172.21.15.112:0/3362793148 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff7e4022070 con 0x7ff7ec161ce0 2023-08-27T11:19:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.158+0000 7ff7ea59c700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff7cc06a170 0x7ff7cc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.158+0000 7ff7ea59c700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff7cc06a170 0x7ff7cc06c630 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0000f60 tx=0x7ff7e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.162+0000 7ff7daffd700 1 -- 172.21.15.112:0/3362793148 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff7ec071290 con 0x7ff7ec161ce0 2023-08-27T11:19:07.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.441+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/3362793148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) v1 -- 0x7ff7ec067a20 con 0x7ff7ec161ce0 2023-08-27T11:19:07.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.635+0000 7ff7daffd700 1 -- 172.21.15.112:0/3362793148 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v136) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff7e402c0c0 con 0x7ff7ec161ce0 2023-08-27T11:19:07.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.638+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff7cc06a170 msgr2=0x7ff7cc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:07.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.638+0000 7ff7f0d8e700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff7cc06a170 0x7ff7cc06c630 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0000f60 tx=0x7ff7e0009040 comp rx=0 tx=0).stop 2023-08-27T11:19:07.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.638+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/3362793148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec161ce0 msgr2=0x7ff7ec0c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:07.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.638+0000 7ff7f0d8e700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec161ce0 0x7ff7ec0c97e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4014040 tx=0x7ff7e4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.638+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/3362793148 shutdown_connections 2023-08-27T11:19:07.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.638+0000 7ff7f0d8e700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff7cc06a170 0x7ff7cc06c630 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.638+0000 7ff7f0d8e700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec161ce0 0x7ff7ec0c97e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.638+0000 7ff7f0d8e700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7ec0b1400 0x7ff7ec071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.638+0000 7ff7f0d8e700 1 --2- 172.21.15.112:0/3362793148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec0b17e0 0x7ff7ec0c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.638+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/3362793148 >> 172.21.15.112:0/3362793148 conn(0x7ff7ec0d0000 msgr2=0x7ff7ec15eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:07.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.638+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/3362793148 shutdown_connections 2023-08-27T11:19:07.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.638+0000 7ff7f0d8e700 1 -- 172.21.15.112:0/3362793148 wait complete. 2023-08-27T11:19:07.654 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info' command ... ok 2023-08-27T11:19:07.655 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) Tests the pending_subvolume_deletions in 'fs volume info' command 2023-08-27T11:19:07.655 INFO:tasks.cephfs_test_runner:test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2023-08-27T11:19:07.657 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2023-08-27T11:19:07.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.777+0000 7fe354e17700 1 Processor -- start 2023-08-27T11:19:07.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.777+0000 7fe354e17700 1 -- start start 2023-08-27T11:19:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe354e17700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe35006dcc0 0x7fe3500b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe354e17700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3500b3440 0x7fe3500b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe354e17700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35015b540 con 0x7fe35015a120 2023-08-27T11:19:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe354e17700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3500af830 con 0x7fe35006dcc0 2023-08-27T11:19:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe354e17700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500afb10 con 0x7fe3500b3440 2023-08-27T11:19:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34e59c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe35015a120 0x7fe35006d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53036/0 (socket says 172.21.15.112:53036) 2023-08-27T11:19:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34e59c700 1 -- 172.21.15.112:0/3971318206 learned_addr learned my addr 172.21.15.112:0/3971318206 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34ed9d700 1 --2- 172.21.15.112:0/3971318206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3500b3440 0x7fe3500b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34dd9b700 1 --2- 172.21.15.112:0/3971318206 >> v2:172.21.15.138:3300/0 conn(0x7fe35006dcc0 0x7fe3500b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869998884 0 0) 0x7fe35015b540 con 0x7fe35015a120 2023-08-27T11:19:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3380057c0 con 0x7fe35015a120 2023-08-27T11:19:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 63702428 0 0) 0x7fe3380057c0 con 0x7fe35015a120 2023-08-27T11:19:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3500b3440 msgr2=0x7fe3500b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34d59a700 1 --2- 172.21.15.112:0/3971318206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3500b3440 0x7fe3500b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 >> v2:172.21.15.138:3300/0 conn(0x7fe35006dcc0 msgr2=0x7fe3500b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34d59a700 1 --2- 172.21.15.112:0/3971318206 >> v2:172.21.15.138:3300/0 conn(0x7fe35006dcc0 0x7fe3500b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3500afdf0 con 0x7fe35015a120 2023-08-27T11:19:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fe340001670 con 0x7fe35015a120 2023-08-27T11:19:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.778+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 >> v1:172.21.15.112:6789/0 conn(0x7fe35015a120 legacy=0x7fe35006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34d59a700 1 --2- 172.21.15.112:0/3971318206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe338005c10 0x7fe338006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34d59a700 1 --2- 172.21.15.112:0/3971318206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3500b3440 0x7fe338006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34d59a700 1 --2- 172.21.15.112:0/3971318206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe338006fa0 0x7fe338007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500af830 con 0x7fe338006fa0 2023-08-27T11:19:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500afb10 con 0x7fe338005c10 2023-08-27T11:19:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33800dc20 con 0x7fe3500b3440 2023-08-27T11:19:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34ed9d700 1 --2- 172.21.15.112:0/3971318206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe338006fa0 0x7fe338007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34e59c700 1 --2- 172.21.15.112:0/3971318206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe338005c10 0x7fe338006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34dd9b700 1 --2- 172.21.15.112:0/3971318206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3500b3440 0x7fe338006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34ed9d700 1 -- 172.21.15.112:0/3971318206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3500b3440 msgr2=0x7fe338006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34ed9d700 1 --2- 172.21.15.112:0/3971318206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3500b3440 0x7fe338006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34ed9d700 1 -- 172.21.15.112:0/3971318206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe338005c10 msgr2=0x7fe338006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34ed9d700 1 --2- 172.21.15.112:0/3971318206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe338005c10 0x7fe338006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34ed9d700 1 -- 172.21.15.112:0/3971318206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3380057c0 con 0x7fe338006fa0 2023-08-27T11:19:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34ed9d700 1 --2- 172.21.15.112:0/3971318206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe338006fa0 0x7fe338007420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe348009fc0 tx=0x7fe338007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bc955454eed4d4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34800ec30 con 0x7fe338006fa0 2023-08-27T11:19:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34800f040 con 0x7fe338006fa0 2023-08-27T11:19:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.779+0000 7fe34d59a700 1 -- 172.21.15.112:0/3971318206 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348014480 con 0x7fe338006fa0 2023-08-27T11:19:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.780+0000 7fe354e17700 1 -- 172.21.15.112:0/3971318206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe338006fa0 msgr2=0x7fe338007420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.780+0000 7fe354e17700 1 --2- 172.21.15.112:0/3971318206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe338006fa0 0x7fe338007420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe348009fc0 tx=0x7fe338007a90 comp rx=0 tx=0).stop 2023-08-27T11:19:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.780+0000 7fe34ed9d700 1 -- 172.21.15.112:0/3971318206 reap_dead start 2023-08-27T11:19:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.780+0000 7fe354e17700 1 -- 172.21.15.112:0/3971318206 shutdown_connections 2023-08-27T11:19:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.780+0000 7fe354e17700 1 -- 172.21.15.112:0/3971318206 >> 172.21.15.112:0/3971318206 conn(0x7fe3500cff80 msgr2=0x7fe3500d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.780+0000 7fe354e17700 1 -- 172.21.15.112:0/3971318206 shutdown_connections 2023-08-27T11:19:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.780+0000 7fe354e17700 1 -- 172.21.15.112:0/3971318206 wait complete. 2023-08-27T11:19:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.781+0000 7fe354e17700 1 Processor -- start 2023-08-27T11:19:07.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.781+0000 7fe354e17700 1 -- start start 2023-08-27T11:19:07.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe354e17700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe338006fa0 0x7fe350155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe354e17700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe35014c490 0x7fe35014c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe354e17700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe350157b20 0x7fe350157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe354e17700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35014e840 con 0x7fe350157b20 2023-08-27T11:19:07.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe354e17700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35014f850 con 0x7fe35014c490 2023-08-27T11:19:07.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe354e17700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35014ea90 con 0x7fe338006fa0 2023-08-27T11:19:07.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe34ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe350157b20 0x7fe350157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe34ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe350157b20 0x7fe350157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58670/0 (socket says 172.21.15.112:58670) 2023-08-27T11:19:07.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe34ed9d700 1 -- 172.21.15.112:0/2254856566 learned_addr learned my addr 172.21.15.112:0/2254856566 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:07.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe34e59c700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe338006fa0 0x7fe350155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe34dd9b700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe35014c490 0x7fe35014c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe34ed9d700 1 -- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe338006fa0 msgr2=0x7fe350155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:07.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe34ed9d700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe338006fa0 0x7fe350155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe34ed9d700 1 -- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe35014c490 msgr2=0x7fe35014c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:07.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe34ed9d700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe35014c490 0x7fe35014c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:07.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe34ed9d700 1 -- 172.21.15.112:0/2254856566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35014ed70 con 0x7fe350157b20 2023-08-27T11:19:07.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.782+0000 7fe34ed9d700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe350157b20 0x7fe350157fd0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe348017040 tx=0x7fe348009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:07.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.783+0000 7fe33effd700 1 -- 172.21.15.112:0/2254856566 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348007000 con 0x7fe350157b20 2023-08-27T11:19:07.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.783+0000 7fe33effd700 1 -- 172.21.15.112:0/2254856566 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe348003e50 con 0x7fe350157b20 2023-08-27T11:19:07.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.783+0000 7fe354e17700 1 -- 172.21.15.112:0/2254856566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35014f050 con 0x7fe350157b20 2023-08-27T11:19:07.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.783+0000 7fe33effd700 1 -- 172.21.15.112:0/2254856566 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348014b50 con 0x7fe350157b20 2023-08-27T11:19:07.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.783+0000 7fe354e17700 1 -- 172.21.15.112:0/2254856566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe350156460 con 0x7fe350157b20 2023-08-27T11:19:07.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.784+0000 7fe354e17700 1 -- 172.21.15.112:0/2254856566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35014f570 con 0x7fe350157b20 2023-08-27T11:19:07.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.784+0000 7fe33effd700 1 -- 172.21.15.112:0/2254856566 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fe348023020 con 0x7fe350157b20 2023-08-27T11:19:07.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.784+0000 7fe33effd700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe33006a170 0x7fe33006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:07.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.784+0000 7fe33effd700 1 -- 172.21.15.112:0/2254856566 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe348089730 con 0x7fe350157b20 2023-08-27T11:19:07.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.785+0000 7fe34e59c700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe33006a170 0x7fe33006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:07.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.785+0000 7fe34e59c700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe33006a170 0x7fe33006c630 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe340000c00 tx=0x7fe340009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:07.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:07.789+0000 7fe33effd700 1 -- 172.21.15.112:0/2254856566 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe35014f570 con 0x7fe350157b20 2023-08-27T11:19:08.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.055+0000 7fe354e17700 1 -- 172.21.15.112:0/2254856566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) v1 -- 0x7fe350158ce0 con 0x7fe350157b20 2023-08-27T11:19:08.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.644+0000 7fe33effd700 1 -- 172.21.15.112:0/2254856566 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v137) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe3480551e0 con 0x7fe350157b20 2023-08-27T11:19:08.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.646+0000 7fe354e17700 1 -- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe33006a170 msgr2=0x7fe33006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:08.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.646+0000 7fe354e17700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe33006a170 0x7fe33006c630 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe340000c00 tx=0x7fe340009040 comp rx=0 tx=0).stop 2023-08-27T11:19:08.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.646+0000 7fe354e17700 1 -- 172.21.15.112:0/2254856566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe350157b20 msgr2=0x7fe350157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:08.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.646+0000 7fe354e17700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe350157b20 0x7fe350157fd0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe348017040 tx=0x7fe348009750 comp rx=0 tx=0).stop 2023-08-27T11:19:08.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.646+0000 7fe354e17700 1 -- 172.21.15.112:0/2254856566 shutdown_connections 2023-08-27T11:19:08.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.646+0000 7fe354e17700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe33006a170 0x7fe33006c630 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.646+0000 7fe354e17700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe350157b20 0x7fe350157fd0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.646+0000 7fe354e17700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe338006fa0 0x7fe350155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.646+0000 7fe354e17700 1 --2- 172.21.15.112:0/2254856566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe35014c490 0x7fe35014c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.646+0000 7fe354e17700 1 -- 172.21.15.112:0/2254856566 >> 172.21.15.112:0/2254856566 conn(0x7fe3500cff80 msgr2=0x7fe3500b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:08.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.646+0000 7fe354e17700 1 -- 172.21.15.112:0/2254856566 shutdown_connections 2023-08-27T11:19:08.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.646+0000 7fe354e17700 1 -- 172.21.15.112:0/2254856566 wait complete. 2023-08-27T11:19:08.661 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:19:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.779+0000 7fd66fd86700 1 Processor -- start 2023-08-27T11:19:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.779+0000 7fd66fd86700 1 -- start start 2023-08-27T11:19:08.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.780+0000 7fd66fd86700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd668146f50 0x7fd6680cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:08.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.780+0000 7fd66fd86700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6680d0260 0x7fd668071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.780+0000 7fd66fd86700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6680719b0 con 0x7fd6680cb500 2023-08-27T11:19:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.780+0000 7fd66fd86700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd668147330 con 0x7fd668146f50 2023-08-27T11:19:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.780+0000 7fd66fd86700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd668070da0 con 0x7fd6680d0260 2023-08-27T11:19:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.780+0000 7fd66d321700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd6680cb500 0x7fd6680cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53058/0 (socket says 172.21.15.112:53058) 2023-08-27T11:19:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.780+0000 7fd66d321700 1 -- 172.21.15.112:0/1401276353 learned_addr learned my addr 172.21.15.112:0/1401276353 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447607215 0 0) 0x7fd6680719b0 con 0x7fd6680cb500 2023-08-27T11:19:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6540035b0 con 0x7fd6680cb500 2023-08-27T11:19:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66e323700 1 --2- 172.21.15.112:0/1401276353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6680d0260 0x7fd668071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66db22700 1 --2- 172.21.15.112:0/1401276353 >> v2:172.21.15.138:3300/0 conn(0x7fd668146f50 0x7fd6680cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3380520517 0 0) 0x7fd6540035b0 con 0x7fd6680cb500 2023-08-27T11:19:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6680d0260 msgr2=0x7fd668071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 --2- 172.21.15.112:0/1401276353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6680d0260 0x7fd668071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 >> v2:172.21.15.138:3300/0 conn(0x7fd668146f50 msgr2=0x7fd6680cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 --2- 172.21.15.112:0/1401276353 >> v2:172.21.15.138:3300/0 conn(0x7fd668146f50 0x7fd6680cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd668071080 con 0x7fd6680cb500 2023-08-27T11:19:08.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fd664001670 con 0x7fd6680cb500 2023-08-27T11:19:08.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66db22700 1 --2- 172.21.15.112:0/1401276353 >> v2:172.21.15.138:3300/0 conn(0x7fd668146f50 0x7fd6680cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:08.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 >> v1:172.21.15.112:6789/0 conn(0x7fd6680cb500 legacy=0x7fd6680cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:08.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 --2- 172.21.15.112:0/1401276353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6680d0260 0x7fd654003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:08.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 --2- 172.21.15.112:0/1401276353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd654003f80 0x7fd654004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 --2- 172.21.15.112:0/1401276353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd654004d90 0x7fd654005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd668147330 con 0x7fd654004d90 2023-08-27T11:19:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd668070da0 con 0x7fd654003f80 2023-08-27T11:19:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65400ba60 con 0x7fd6680d0260 2023-08-27T11:19:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66e323700 1 --2- 172.21.15.112:0/1401276353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd654004d90 0x7fd654005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:08.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66db22700 1 --2- 172.21.15.112:0/1401276353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6680d0260 0x7fd654003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:08.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66d321700 1 --2- 172.21.15.112:0/1401276353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd654003f80 0x7fd654004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:08.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66e323700 1 -- 172.21.15.112:0/1401276353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6680d0260 msgr2=0x7fd654003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:08.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66e323700 1 --2- 172.21.15.112:0/1401276353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6680d0260 0x7fd654003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66e323700 1 -- 172.21.15.112:0/1401276353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd654003f80 msgr2=0x7fd654004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:08.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66e323700 1 --2- 172.21.15.112:0/1401276353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd654003f80 0x7fd654004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.781+0000 7fd66e323700 1 -- 172.21.15.112:0/1401276353 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6540035b0 con 0x7fd654004d90 2023-08-27T11:19:08.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.782+0000 7fd66e323700 1 --2- 172.21.15.112:0/1401276353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd654004d90 0x7fd654005210 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd6680cf740 tx=0x7fd654005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83d25540bca970c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:08.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.782+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66000ebd0 con 0x7fd654004d90 2023-08-27T11:19:08.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.782+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66000f040 con 0x7fd654004d90 2023-08-27T11:19:08.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.782+0000 7fd66cb20700 1 -- 172.21.15.112:0/1401276353 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660014480 con 0x7fd654004d90 2023-08-27T11:19:08.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.782+0000 7fd66fd86700 1 -- 172.21.15.112:0/1401276353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd654004d90 msgr2=0x7fd654005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:08.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.782+0000 7fd66fd86700 1 --2- 172.21.15.112:0/1401276353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd654004d90 0x7fd654005210 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd6680cf740 tx=0x7fd654005880 comp rx=0 tx=0).stop 2023-08-27T11:19:08.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.782+0000 7fd66e323700 1 -- 172.21.15.112:0/1401276353 reap_dead start 2023-08-27T11:19:08.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.782+0000 7fd66fd86700 1 -- 172.21.15.112:0/1401276353 shutdown_connections 2023-08-27T11:19:08.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.782+0000 7fd66fd86700 1 -- 172.21.15.112:0/1401276353 >> 172.21.15.112:0/1401276353 conn(0x7fd6680cdc50 msgr2=0x7fd6680ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.782+0000 7fd66fd86700 1 -- 172.21.15.112:0/1401276353 shutdown_connections 2023-08-27T11:19:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.782+0000 7fd66fd86700 1 -- 172.21.15.112:0/1401276353 wait complete. 2023-08-27T11:19:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.784+0000 7fd66fd86700 1 Processor -- start 2023-08-27T11:19:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.784+0000 7fd66fd86700 1 -- start start 2023-08-27T11:19:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.784+0000 7fd66fd86700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd654004d90 0x7fd668144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.784+0000 7fd66fd86700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd668145cb0 0x7fd6681f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.784+0000 7fd66fd86700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6681f2690 0x7fd66815f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.784+0000 7fd66fd86700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66813b2b0 con 0x7fd668145cb0 2023-08-27T11:19:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.784+0000 7fd66fd86700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66813b590 con 0x7fd6681f2690 2023-08-27T11:19:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.784+0000 7fd66fd86700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66813b870 con 0x7fd654004d90 2023-08-27T11:19:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.784+0000 7fd66d321700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd668145cb0 0x7fd6681f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.784+0000 7fd66d321700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd668145cb0 0x7fd6681f2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58688/0 (socket says 172.21.15.112:58688) 2023-08-27T11:19:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.784+0000 7fd66d321700 1 -- 172.21.15.112:0/3466836631 learned_addr learned my addr 172.21.15.112:0/3466836631 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.785+0000 7fd66db22700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd654004d90 0x7fd668144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.785+0000 7fd66e323700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6681f2690 0x7fd66815f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.785+0000 7fd66d321700 1 -- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd654004d90 msgr2=0x7fd668144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.785+0000 7fd66d321700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd654004d90 0x7fd668144180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.785+0000 7fd66d321700 1 -- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6681f2690 msgr2=0x7fd66815f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:08.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.785+0000 7fd66d321700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6681f2690 0x7fd66815f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.785+0000 7fd66d321700 1 -- 172.21.15.112:0/3466836631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66813bb50 con 0x7fd668145cb0 2023-08-27T11:19:08.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.785+0000 7fd66d321700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd668145cb0 0x7fd6681f2150 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd6640017a0 tx=0x7fd6640075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:08.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.785+0000 7fd65e7fc700 1 -- 172.21.15.112:0/3466836631 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66400f610 con 0x7fd668145cb0 2023-08-27T11:19:08.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.785+0000 7fd65e7fc700 1 -- 172.21.15.112:0/3466836631 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66400f7b0 con 0x7fd668145cb0 2023-08-27T11:19:08.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.785+0000 7fd66fd86700 1 -- 172.21.15.112:0/3466836631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66813be30 con 0x7fd668145cb0 2023-08-27T11:19:08.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.785+0000 7fd65e7fc700 1 -- 172.21.15.112:0/3466836631 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66400fb60 con 0x7fd668145cb0 2023-08-27T11:19:08.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.785+0000 7fd66fd86700 1 -- 172.21.15.112:0/3466836631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66813c350 con 0x7fd668145cb0 2023-08-27T11:19:08.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.786+0000 7fd66fd86700 1 -- 172.21.15.112:0/3466836631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66813c570 con 0x7fd668145cb0 2023-08-27T11:19:08.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.786+0000 7fd65e7fc700 1 -- 172.21.15.112:0/3466836631 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fd66400fcc0 con 0x7fd668145cb0 2023-08-27T11:19:08.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.787+0000 7fd65e7fc700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd64c06a170 0x7fd64c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:08.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.787+0000 7fd65e7fc700 1 -- 172.21.15.112:0/3466836631 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd664088210 con 0x7fd668145cb0 2023-08-27T11:19:08.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.787+0000 7fd66db22700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd64c06a170 0x7fd64c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:08.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.788+0000 7fd66db22700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd64c06a170 0x7fd64c06c630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd658009a80 tx=0x7fd658008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:08.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.791+0000 7fd65e7fc700 1 -- 172.21.15.112:0/3466836631 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd664053c30 con 0x7fd668145cb0 2023-08-27T11:19:08.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.982+0000 7fd66fd86700 1 -- 172.21.15.112:0/3466836631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd66813c020 con 0x7fd668145cb0 2023-08-27T11:19:08.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.983+0000 7fd65e7fc700 1 -- 172.21.15.112:0/3466836631 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+14737 (secure 0 0 0) 0x7fd664057250 con 0x7fd668145cb0 2023-08-27T11:19:08.985 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:08.985 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":56,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:19:06.624859+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/3423469954":"2023-08-28T11:19:03.611345+0000","172.21.15.112:6833/3423469954":"2023-08-28T11:19:03.611345+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:19:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.985+0000 7fd66fd86700 1 -- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd64c06a170 msgr2=0x7fd64c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.985+0000 7fd66fd86700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd64c06a170 0x7fd64c06c630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd658009a80 tx=0x7fd658008040 comp rx=0 tx=0).stop 2023-08-27T11:19:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.985+0000 7fd66fd86700 1 -- 172.21.15.112:0/3466836631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd668145cb0 msgr2=0x7fd6681f2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.985+0000 7fd66fd86700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd668145cb0 0x7fd6681f2150 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd6640017a0 tx=0x7fd6640075f0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.985+0000 7fd66fd86700 1 -- 172.21.15.112:0/3466836631 shutdown_connections 2023-08-27T11:19:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.985+0000 7fd66fd86700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd64c06a170 0x7fd64c06c630 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.985+0000 7fd66fd86700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6681f2690 0x7fd66815f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.985+0000 7fd66fd86700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd654004d90 0x7fd668144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.985+0000 7fd66fd86700 1 --2- 172.21.15.112:0/3466836631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd668145cb0 0x7fd6681f2150 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.985+0000 7fd66fd86700 1 -- 172.21.15.112:0/3466836631 >> 172.21.15.112:0/3466836631 conn(0x7fd6680cdc50 msgr2=0x7fd6680ccdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.986+0000 7fd66fd86700 1 -- 172.21.15.112:0/3466836631 shutdown_connections 2023-08-27T11:19:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:08.986+0000 7fd66fd86700 1 -- 172.21.15.112:0/3466836631 wait complete. 2023-08-27T11:19:08.999 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:19:09.000 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:19:09.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.121+0000 7fa268c27700 1 Processor -- start 2023-08-27T11:19:09.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.121+0000 7fa268c27700 1 -- start start 2023-08-27T11:19:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.122+0000 7fa268c27700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa26406f790 0x7fa26406fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.122+0000 7fa268c27700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2640b5070 con 0x7fa2640b3440 2023-08-27T11:19:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.122+0000 7fa268c27700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa264147000 con 0x7fa264146c20 2023-08-27T11:19:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.122+0000 7fa268c27700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa264070160 con 0x7fa26406f790 2023-08-27T11:19:09.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.123+0000 7fa261d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa2640b3440 0x7fa26415e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53078/0 (socket says 172.21.15.112:53078) 2023-08-27T11:19:09.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.123+0000 7fa261d9b700 1 -- 172.21.15.112:0/280591792 learned_addr learned my addr 172.21.15.112:0/280591792 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:09.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.123+0000 7fa262d9d700 1 --2- 172.21.15.112:0/280591792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa26406f790 0x7fa26406fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.123+0000 7fa26159a700 1 -- 172.21.15.112:0/280591792 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067111608 0 0) 0x7fa254001110 con 0x7fa2640b3440 2023-08-27T11:19:09.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.123+0000 7fa26159a700 1 -- 172.21.15.112:0/280591792 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa24c0035b0 con 0x7fa2640b3440 2023-08-27T11:19:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.123+0000 7fa262d9d700 1 -- 172.21.15.112:0/280591792 >> v1:172.21.15.138:6789/0 conn(0x7fa264146c20 legacy=0x7fa264160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.123+0000 7fa262d9d700 1 -- 172.21.15.112:0/280591792 >> v1:172.21.15.112:6789/0 conn(0x7fa2640b3440 legacy=0x7fa26415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.123+0000 7fa26159a700 1 -- 172.21.15.112:0/280591792 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176288478 0 0) 0x7fa264147000 con 0x7fa264146c20 2023-08-27T11:19:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.123+0000 7fa262d9d700 1 -- 172.21.15.112:0/280591792 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa264161740 con 0x7fa26406f790 2023-08-27T11:19:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.123+0000 7fa262d9d700 1 --2- 172.21.15.112:0/280591792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa26406f790 0x7fa26406fc20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009dc0 tx=0x7fa25c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c67b1dda5fe08b18 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.123+0000 7fa26159a700 1 -- 172.21.15.112:0/280591792 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c00e070 con 0x7fa26406f790 2023-08-27T11:19:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.123+0000 7fa26159a700 1 -- 172.21.15.112:0/280591792 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25c00bb20 con 0x7fa26406f790 2023-08-27T11:19:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.123+0000 7fa26159a700 1 -- 172.21.15.112:0/280591792 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c00be90 con 0x7fa26406f790 2023-08-27T11:19:09.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.124+0000 7fa268c27700 1 -- 172.21.15.112:0/280591792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa26406f790 msgr2=0x7fa26406fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.124+0000 7fa268c27700 1 --2- 172.21.15.112:0/280591792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa26406f790 0x7fa26406fc20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009dc0 tx=0x7fa25c00b330 comp rx=0 tx=0).stop 2023-08-27T11:19:09.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.124+0000 7fa268c27700 1 -- 172.21.15.112:0/280591792 shutdown_connections 2023-08-27T11:19:09.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.124+0000 7fa268c27700 1 --2- 172.21.15.112:0/280591792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa26406f790 0x7fa26406fc20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.124+0000 7fa268c27700 1 -- 172.21.15.112:0/280591792 >> 172.21.15.112:0/280591792 conn(0x7fa2640b1e90 msgr2=0x7fa2640b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:09.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.124+0000 7fa268c27700 1 -- 172.21.15.112:0/280591792 shutdown_connections 2023-08-27T11:19:09.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.124+0000 7fa268c27700 1 -- 172.21.15.112:0/280591792 wait complete. 2023-08-27T11:19:09.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.125+0000 7fa268c27700 1 Processor -- start 2023-08-27T11:19:09.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.125+0000 7fa268c27700 1 -- start start 2023-08-27T11:19:09.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.126+0000 7fa268c27700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa26406f790 0x7fa26415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.126+0000 7fa268c27700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2640b3440 0x7fa2640ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.126+0000 7fa268c27700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa264146c20 0x7fa2640af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.126+0000 7fa268c27700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640c6ba0 con 0x7fa26406f790 2023-08-27T11:19:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.126+0000 7fa268c27700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640adf80 con 0x7fa264146c20 2023-08-27T11:19:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.126+0000 7fa268c27700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640ae260 con 0x7fa2640b3440 2023-08-27T11:19:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.126+0000 7fa26259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa26406f790 0x7fa26415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.126+0000 7fa26259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa26406f790 0x7fa26415f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58700/0 (socket says 172.21.15.112:58700) 2023-08-27T11:19:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.126+0000 7fa26259c700 1 -- 172.21.15.112:0/1907461777 learned_addr learned my addr 172.21.15.112:0/1907461777 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.126+0000 7fa261d9b700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2640b3440 0x7fa2640ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.127+0000 7fa262d9d700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa264146c20 0x7fa2640af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.127+0000 7fa26259c700 1 -- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2640b3440 msgr2=0x7fa2640ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.127+0000 7fa26259c700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2640b3440 0x7fa2640ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.127+0000 7fa26259c700 1 -- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa264146c20 msgr2=0x7fa2640af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.127+0000 7fa26259c700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa264146c20 0x7fa2640af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.127+0000 7fa26259c700 1 -- 172.21.15.112:0/1907461777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c009a70 con 0x7fa26406f790 2023-08-27T11:19:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.127+0000 7fa262d9d700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa264146c20 0x7fa2640af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.127+0000 7fa26259c700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa26406f790 0x7fa26415f320 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa25800c500 tx=0x7fa25800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.127+0000 7fa252ffd700 1 -- 172.21.15.112:0/1907461777 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2580054e0 con 0x7fa26406f790 2023-08-27T11:19:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.127+0000 7fa252ffd700 1 -- 172.21.15.112:0/1907461777 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa258003680 con 0x7fa26406f790 2023-08-27T11:19:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.127+0000 7fa268c27700 1 -- 172.21.15.112:0/1907461777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2640ae540 con 0x7fa26406f790 2023-08-27T11:19:09.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.127+0000 7fa252ffd700 1 -- 172.21.15.112:0/1907461777 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa258003a30 con 0x7fa26406f790 2023-08-27T11:19:09.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.127+0000 7fa268c27700 1 -- 172.21.15.112:0/1907461777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2640aeb00 con 0x7fa26406f790 2023-08-27T11:19:09.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.128+0000 7fa268c27700 1 -- 172.21.15.112:0/1907461777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa264067e90 con 0x7fa26406f790 2023-08-27T11:19:09.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.128+0000 7fa252ffd700 1 -- 172.21.15.112:0/1907461777 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fa258003b90 con 0x7fa26406f790 2023-08-27T11:19:09.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.129+0000 7fa252ffd700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa24406a170 0x7fa24406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.129+0000 7fa252ffd700 1 -- 172.21.15.112:0/1907461777 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa2580877d0 con 0x7fa26406f790 2023-08-27T11:19:09.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.129+0000 7fa261d9b700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa24406a170 0x7fa24406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.129+0000 7fa261d9b700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa24406a170 0x7fa24406c630 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa2640cf460 tx=0x7fa254009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:09.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.133+0000 7fa252ffd700 1 -- 172.21.15.112:0/1907461777 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa258053100 con 0x7fa26406f790 2023-08-27T11:19:09.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.322+0000 7fa268c27700 1 -- 172.21.15.112:0/1907461777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa264067a20 con 0x7fa26406f790 2023-08-27T11:19:09.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.322+0000 7fa252ffd700 1 -- 172.21.15.112:0/1907461777 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa258056720 con 0x7fa26406f790 2023-08-27T11:19:09.324 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:19:09.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.324+0000 7fa268c27700 1 -- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa24406a170 msgr2=0x7fa24406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.324+0000 7fa268c27700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa24406a170 0x7fa24406c630 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa2640cf460 tx=0x7fa254009040 comp rx=0 tx=0).stop 2023-08-27T11:19:09.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.324+0000 7fa268c27700 1 -- 172.21.15.112:0/1907461777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa26406f790 msgr2=0x7fa26415f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.324+0000 7fa268c27700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa26406f790 0x7fa26415f320 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa25800c500 tx=0x7fa25800c8c0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.324+0000 7fa268c27700 1 -- 172.21.15.112:0/1907461777 shutdown_connections 2023-08-27T11:19:09.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.324+0000 7fa268c27700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa24406a170 0x7fa24406c630 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.324+0000 7fa268c27700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa264146c20 0x7fa2640af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.324+0000 7fa268c27700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa26406f790 0x7fa26415f320 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.324+0000 7fa268c27700 1 --2- 172.21.15.112:0/1907461777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2640b3440 0x7fa2640ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.324+0000 7fa268c27700 1 -- 172.21.15.112:0/1907461777 >> 172.21.15.112:0/1907461777 conn(0x7fa2640b1e90 msgr2=0x7fa264070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:09.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.324+0000 7fa268c27700 1 -- 172.21.15.112:0/1907461777 shutdown_connections 2023-08-27T11:19:09.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.324+0000 7fa268c27700 1 -- 172.21.15.112:0/1907461777 wait complete. 2023-08-27T11:19:09.338 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:19:09.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.459+0000 7fe957bc5700 1 Processor -- start 2023-08-27T11:19:09.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.460+0000 7fe957bc5700 1 -- start start 2023-08-27T11:19:09.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.460+0000 7fe957bc5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe950146c20 0x7fe950160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.460+0000 7fe957bc5700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe9500b3440 0x7fe95015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.460+0000 7fe957bc5700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9500b5070 con 0x7fe95015e6e0 2023-08-27T11:19:09.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.460+0000 7fe957bc5700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe950147000 con 0x7fe950146c20 2023-08-27T11:19:09.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.460+0000 7fe957bc5700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9501614a0 con 0x7fe9500b3440 2023-08-27T11:19:09.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe955961700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe950146c20 0x7fe950160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe956162700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe95015e6e0 0x7fe95006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53094/0 (socket says 172.21.15.112:53094) 2023-08-27T11:19:09.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe956162700 1 -- 172.21.15.112:0/4055235876 learned_addr learned my addr 172.21.15.112:0/4055235876 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:09.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe955160700 1 --2- 172.21.15.112:0/4055235876 >> v2:172.21.15.138:3300/0 conn(0x7fe9500b3440 0x7fe95015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe955961700 1 -- 172.21.15.112:0/4055235876 >> v2:172.21.15.138:3300/0 conn(0x7fe9500b3440 msgr2=0x7fe95015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:09.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe955961700 1 --2- 172.21.15.112:0/4055235876 >> v2:172.21.15.138:3300/0 conn(0x7fe9500b3440 0x7fe95015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe955961700 1 -- 172.21.15.112:0/4055235876 >> v1:172.21.15.112:6789/0 conn(0x7fe95015e6e0 legacy=0x7fe95006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe95495f700 1 -- 172.21.15.112:0/4055235876 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670618509 0 0) 0x7fe948001490 con 0x7fe95015e6e0 2023-08-27T11:19:09.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe955961700 1 -- 172.21.15.112:0/4055235876 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950161740 con 0x7fe950146c20 2023-08-27T11:19:09.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe955160700 1 --2- 172.21.15.112:0/4055235876 >> v2:172.21.15.138:3300/0 conn(0x7fe9500b3440 0x7fe95015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:09.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe955961700 1 --2- 172.21.15.112:0/4055235876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe950146c20 0x7fe950160f60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe9500b2d80 tx=0x7fe940009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b4dae0684931d2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:09.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe95495f700 1 -- 172.21.15.112:0/4055235876 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940011830 con 0x7fe950146c20 2023-08-27T11:19:09.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe95495f700 1 -- 172.21.15.112:0/4055235876 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe940011990 con 0x7fe950146c20 2023-08-27T11:19:09.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.461+0000 7fe95495f700 1 -- 172.21.15.112:0/4055235876 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940011d20 con 0x7fe950146c20 2023-08-27T11:19:09.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.462+0000 7fe957bc5700 1 -- 172.21.15.112:0/4055235876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe950146c20 msgr2=0x7fe950160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.462+0000 7fe957bc5700 1 --2- 172.21.15.112:0/4055235876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe950146c20 0x7fe950160f60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe9500b2d80 tx=0x7fe940009b20 comp rx=0 tx=0).stop 2023-08-27T11:19:09.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.462+0000 7fe957bc5700 1 -- 172.21.15.112:0/4055235876 shutdown_connections 2023-08-27T11:19:09.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.462+0000 7fe957bc5700 1 --2- 172.21.15.112:0/4055235876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe950146c20 0x7fe950160f60 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.462+0000 7fe957bc5700 1 --2- 172.21.15.112:0/4055235876 >> v2:172.21.15.138:3300/0 conn(0x7fe9500b3440 0x7fe95015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.462+0000 7fe957bc5700 1 -- 172.21.15.112:0/4055235876 >> 172.21.15.112:0/4055235876 conn(0x7fe9500b1e90 msgr2=0x7fe9500b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:09.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.462+0000 7fe957bc5700 1 -- 172.21.15.112:0/4055235876 shutdown_connections 2023-08-27T11:19:09.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.462+0000 7fe957bc5700 1 -- 172.21.15.112:0/4055235876 wait complete. 2023-08-27T11:19:09.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.463+0000 7fe957bc5700 1 Processor -- start 2023-08-27T11:19:09.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.463+0000 7fe957bc5700 1 -- start start 2023-08-27T11:19:09.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe957bc5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9500b3440 0x7fe9500c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe957bc5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe950146c20 0x7fe950144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe957bc5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95015e6e0 0x7fe950071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe957bc5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9500c72f0 con 0x7fe95015e6e0 2023-08-27T11:19:09.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe957bc5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe950143690 con 0x7fe9500b3440 2023-08-27T11:19:09.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe957bc5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9501455a0 con 0x7fe950146c20 2023-08-27T11:19:09.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe956162700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95015e6e0 0x7fe950071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe956162700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95015e6e0 0x7fe950071320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58718/0 (socket says 172.21.15.112:58718) 2023-08-27T11:19:09.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe956162700 1 -- 172.21.15.112:0/1373136528 learned_addr learned my addr 172.21.15.112:0/1373136528 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:09.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe955160700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe950146c20 0x7fe950144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe955961700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9500b3440 0x7fe9500c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe956162700 1 -- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe950146c20 msgr2=0x7fe950144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe956162700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe950146c20 0x7fe950144ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe956162700 1 -- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9500b3440 msgr2=0x7fe9500c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe956162700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9500b3440 0x7fe9500c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe956162700 1 -- 172.21.15.112:0/1373136528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9400094a0 con 0x7fe95015e6e0 2023-08-27T11:19:09.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe956162700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95015e6e0 0x7fe950071320 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe94800cc10 tx=0x7fe9480021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:09.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe9467fc700 1 -- 172.21.15.112:0/1373136528 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94800ab80 con 0x7fe95015e6e0 2023-08-27T11:19:09.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe9467fc700 1 -- 172.21.15.112:0/1373136528 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94800ad00 con 0x7fe95015e6e0 2023-08-27T11:19:09.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe957bc5700 1 -- 172.21.15.112:0/1373136528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9501438e0 con 0x7fe95015e6e0 2023-08-27T11:19:09.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.464+0000 7fe9467fc700 1 -- 172.21.15.112:0/1373136528 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948007310 con 0x7fe95015e6e0 2023-08-27T11:19:09.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.465+0000 7fe957bc5700 1 -- 172.21.15.112:0/1373136528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe950143ea0 con 0x7fe95015e6e0 2023-08-27T11:19:09.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.465+0000 7fe957bc5700 1 -- 172.21.15.112:0/1373136528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9501440c0 con 0x7fe95015e6e0 2023-08-27T11:19:09.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.466+0000 7fe9467fc700 1 -- 172.21.15.112:0/1373136528 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fe948007470 con 0x7fe95015e6e0 2023-08-27T11:19:09.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.466+0000 7fe9467fc700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe93406a170 0x7fe93406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.466+0000 7fe9467fc700 1 -- 172.21.15.112:0/1373136528 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe948087e90 con 0x7fe95015e6e0 2023-08-27T11:19:09.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.466+0000 7fe955961700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe93406a170 0x7fe93406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.467+0000 7fe955961700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe93406a170 0x7fe93406c630 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe9400090b0 tx=0x7fe940008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:09.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.470+0000 7fe9467fc700 1 -- 172.21.15.112:0/1373136528 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe948053940 con 0x7fe95015e6e0 2023-08-27T11:19:09.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.661+0000 7fe957bc5700 1 -- 172.21.15.112:0/1373136528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9500b5070 con 0x7fe95015e6e0 2023-08-27T11:19:09.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.661+0000 7fe9467fc700 1 -- 172.21.15.112:0/1373136528 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9500b5070 con 0x7fe95015e6e0 2023-08-27T11:19:09.663 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:19:09.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.663+0000 7fe957bc5700 1 -- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe93406a170 msgr2=0x7fe93406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.663+0000 7fe957bc5700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe93406a170 0x7fe93406c630 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe9400090b0 tx=0x7fe940008030 comp rx=0 tx=0).stop 2023-08-27T11:19:09.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.663+0000 7fe957bc5700 1 -- 172.21.15.112:0/1373136528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95015e6e0 msgr2=0x7fe950071320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.663+0000 7fe957bc5700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95015e6e0 0x7fe950071320 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe94800cc10 tx=0x7fe9480021c0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.664+0000 7fe957bc5700 1 -- 172.21.15.112:0/1373136528 shutdown_connections 2023-08-27T11:19:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.664+0000 7fe957bc5700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe93406a170 0x7fe93406c630 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.664+0000 7fe957bc5700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95015e6e0 0x7fe950071320 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.664+0000 7fe957bc5700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9500b3440 0x7fe9500c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.664+0000 7fe957bc5700 1 --2- 172.21.15.112:0/1373136528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe950146c20 0x7fe950144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.664+0000 7fe957bc5700 1 -- 172.21.15.112:0/1373136528 >> 172.21.15.112:0/1373136528 conn(0x7fe9500b1e90 msgr2=0x7fe95006de30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.664+0000 7fe957bc5700 1 -- 172.21.15.112:0/1373136528 shutdown_connections 2023-08-27T11:19:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.664+0000 7fe957bc5700 1 -- 172.21.15.112:0/1373136528 wait complete. 2023-08-27T11:19:09.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:19:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.799+0000 7efbf6802700 1 Processor -- start 2023-08-27T11:19:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.799+0000 7efbf6802700 1 -- start start 2023-08-27T11:19:09.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbf6802700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf00cca90 0x7efbf015f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbf6802700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efbf006f790 0x7efbf006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbf6802700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf015ee90 con 0x7efbf0144900 2023-08-27T11:19:09.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbf6802700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf0070140 con 0x7efbf006f790 2023-08-27T11:19:09.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbf6802700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf00b11d0 con 0x7efbf00cca90 2023-08-27T11:19:09.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbeffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7efbf0144900 0x7efbf00b09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53104/0 (socket says 172.21.15.112:53104) 2023-08-27T11:19:09.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbeffff700 1 -- 172.21.15.112:0/2164752099 learned_addr learned my addr 172.21.15.112:0/2164752099 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:09.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbf4d9f700 1 --2- 172.21.15.112:0/2164752099 >> v2:172.21.15.112:3300/0 conn(0x7efbf006f790 0x7efbf006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbeeffd700 1 -- 172.21.15.112:0/2164752099 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463979638 0 0) 0x7efbf015ee90 con 0x7efbf0144900 2023-08-27T11:19:09.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbeeffd700 1 -- 172.21.15.112:0/2164752099 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbd80035b0 con 0x7efbf0144900 2023-08-27T11:19:09.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbef7fe700 1 --2- 172.21.15.112:0/2164752099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf00cca90 0x7efbf015f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbf4d9f700 1 -- 172.21.15.112:0/2164752099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf00cca90 msgr2=0x7efbf015f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:09.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbf4d9f700 1 --2- 172.21.15.112:0/2164752099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf00cca90 0x7efbf015f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbf4d9f700 1 -- 172.21.15.112:0/2164752099 >> v1:172.21.15.112:6789/0 conn(0x7efbf0144900 legacy=0x7efbf00b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.800+0000 7efbf4d9f700 1 -- 172.21.15.112:0/2164752099 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf00b14b0 con 0x7efbf006f790 2023-08-27T11:19:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.801+0000 7efbf4d9f700 1 --2- 172.21.15.112:0/2164752099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbf006f790 0x7efbf006fc00 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009d90 tx=0x7efbe800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc3dade1f9e327c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.801+0000 7efbeeffd700 1 -- 172.21.15.112:0/2164752099 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe800e070 con 0x7efbf006f790 2023-08-27T11:19:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.801+0000 7efbeeffd700 1 -- 172.21.15.112:0/2164752099 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe800bb20 con 0x7efbf006f790 2023-08-27T11:19:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.801+0000 7efbeeffd700 1 -- 172.21.15.112:0/2164752099 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe8012430 con 0x7efbf006f790 2023-08-27T11:19:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.801+0000 7efbf6802700 1 -- 172.21.15.112:0/2164752099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbf006f790 msgr2=0x7efbf006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.801+0000 7efbf6802700 1 --2- 172.21.15.112:0/2164752099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbf006f790 0x7efbf006fc00 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009d90 tx=0x7efbe800b330 comp rx=0 tx=0).stop 2023-08-27T11:19:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.801+0000 7efbf6802700 1 -- 172.21.15.112:0/2164752099 shutdown_connections 2023-08-27T11:19:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.801+0000 7efbf6802700 1 --2- 172.21.15.112:0/2164752099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbf006f790 0x7efbf006fc00 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.801+0000 7efbf6802700 1 --2- 172.21.15.112:0/2164752099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf00cca90 0x7efbf015f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.801+0000 7efbf6802700 1 -- 172.21.15.112:0/2164752099 >> 172.21.15.112:0/2164752099 conn(0x7efbf00b1e90 msgr2=0x7efbf00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.801+0000 7efbf6802700 1 -- 172.21.15.112:0/2164752099 shutdown_connections 2023-08-27T11:19:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.801+0000 7efbf6802700 1 -- 172.21.15.112:0/2164752099 wait complete. 2023-08-27T11:19:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.802+0000 7efbf6802700 1 Processor -- start 2023-08-27T11:19:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.803+0000 7efbf6802700 1 -- start start 2023-08-27T11:19:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.803+0000 7efbf6802700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf006f790 0x7efbf0134700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.803+0000 7efbf6802700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbf00cca90 0x7efbf015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.803+0000 7efbf6802700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf0144900 0x7efbf01f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.803+0000 7efbf6802700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf0135b80 con 0x7efbf00cca90 2023-08-27T11:19:09.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.803+0000 7efbf6802700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf0134f70 con 0x7efbf0144900 2023-08-27T11:19:09.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.803+0000 7efbf6802700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf0135250 con 0x7efbf006f790 2023-08-27T11:19:09.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.803+0000 7efbef7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbf00cca90 0x7efbf015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.803+0000 7efbef7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbf00cca90 0x7efbf015a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58730/0 (socket says 172.21.15.112:58730) 2023-08-27T11:19:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.803+0000 7efbef7fe700 1 -- 172.21.15.112:0/1987734321 learned_addr learned my addr 172.21.15.112:0/1987734321 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.804+0000 7efbeffff700 1 --2- 172.21.15.112:0/1987734321 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf006f790 0x7efbf0134700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.804+0000 7efbef7fe700 1 -- 172.21.15.112:0/1987734321 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf006f790 msgr2=0x7efbf0134700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.804+0000 7efbef7fe700 1 --2- 172.21.15.112:0/1987734321 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf006f790 0x7efbf0134700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.804+0000 7efbef7fe700 1 -- 172.21.15.112:0/1987734321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf0144900 msgr2=0x7efbf01f39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.804+0000 7efbef7fe700 1 --2- 172.21.15.112:0/1987734321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf0144900 0x7efbf01f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.804+0000 7efbef7fe700 1 -- 172.21.15.112:0/1987734321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe8009a70 con 0x7efbf00cca90 2023-08-27T11:19:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.804+0000 7efbef7fe700 1 --2- 172.21.15.112:0/1987734321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbf00cca90 0x7efbf015a600 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7efbe000e810 tx=0x7efbe000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.804+0000 7efbecff9700 1 -- 172.21.15.112:0/1987734321 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe0013610 con 0x7efbf00cca90 2023-08-27T11:19:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.804+0000 7efbecff9700 1 -- 172.21.15.112:0/1987734321 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe0013770 con 0x7efbf00cca90 2023-08-27T11:19:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.804+0000 7efbf6802700 1 -- 172.21.15.112:0/1987734321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf0135530 con 0x7efbf00cca90 2023-08-27T11:19:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.804+0000 7efbecff9700 1 -- 172.21.15.112:0/1987734321 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe0013b20 con 0x7efbf00cca90 2023-08-27T11:19:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.804+0000 7efbf6802700 1 -- 172.21.15.112:0/1987734321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf0134c40 con 0x7efbf00cca90 2023-08-27T11:19:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.805+0000 7efbf6802700 1 -- 172.21.15.112:0/1987734321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf0067e90 con 0x7efbf00cca90 2023-08-27T11:19:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.805+0000 7efbecff9700 1 -- 172.21.15.112:0/1987734321 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7efbe0013c80 con 0x7efbf00cca90 2023-08-27T11:19:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.806+0000 7efbecff9700 1 --2- 172.21.15.112:0/1987734321 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efbdc06a170 0x7efbdc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.806+0000 7efbecff9700 1 -- 172.21.15.112:0/1987734321 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7efbe0087fe0 con 0x7efbf00cca90 2023-08-27T11:19:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.806+0000 7efbeffff700 1 --2- 172.21.15.112:0/1987734321 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efbdc06a170 0x7efbdc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.806+0000 7efbeffff700 1 --2- 172.21.15.112:0/1987734321 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efbdc06a170 0x7efbdc06c630 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efbe40011e0 tx=0x7efbe4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:09.810+0000 7efbecff9700 1 -- 172.21.15.112:0/1987734321 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efbe0053900 con 0x7efbf00cca90 2023-08-27T11:19:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.001+0000 7efbf6802700 1 -- 172.21.15.112:0/1987734321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efbf0067a20 con 0x7efbf00cca90 2023-08-27T11:19:10.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.007+0000 7efbecff9700 1 -- 172.21.15.112:0/1987734321 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7efbe0056f20 con 0x7efbf00cca90 2023-08-27T11:19:10.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.009+0000 7efbf6802700 1 -- 172.21.15.112:0/1987734321 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efbdc06a170 msgr2=0x7efbdc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:10.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.009+0000 7efbf6802700 1 --2- 172.21.15.112:0/1987734321 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efbdc06a170 0x7efbdc06c630 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efbe40011e0 tx=0x7efbe4009040 comp rx=0 tx=0).stop 2023-08-27T11:19:10.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.009+0000 7efbf6802700 1 -- 172.21.15.112:0/1987734321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbf00cca90 msgr2=0x7efbf015a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:10.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.009+0000 7efbf6802700 1 --2- 172.21.15.112:0/1987734321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbf00cca90 0x7efbf015a600 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7efbe000e810 tx=0x7efbe000ebd0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.009+0000 7efbf6802700 1 -- 172.21.15.112:0/1987734321 shutdown_connections 2023-08-27T11:19:10.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.009+0000 7efbf6802700 1 --2- 172.21.15.112:0/1987734321 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efbdc06a170 0x7efbdc06c630 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.009+0000 7efbf6802700 1 --2- 172.21.15.112:0/1987734321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf0144900 0x7efbf01f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.009+0000 7efbf6802700 1 --2- 172.21.15.112:0/1987734321 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf006f790 0x7efbf0134700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.009+0000 7efbf6802700 1 --2- 172.21.15.112:0/1987734321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbf00cca90 0x7efbf015a600 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.009+0000 7efbf6802700 1 -- 172.21.15.112:0/1987734321 >> 172.21.15.112:0/1987734321 conn(0x7efbf00b1e90 msgr2=0x7efbf00b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:10.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.009+0000 7efbf6802700 1 -- 172.21.15.112:0/1987734321 shutdown_connections 2023-08-27T11:19:10.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.009+0000 7efbf6802700 1 -- 172.21.15.112:0/1987734321 wait complete. 2023-08-27T11:19:10.022 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:19:10.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:19:10.071 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:19:10.118 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:19:10.118 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:10.119 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:10.136 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:19:10.138 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:19:10.138 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:10.158 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:19:10.159 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:10.160 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:10.193 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:19:10.194 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:19:10.194 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:10.217 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:19:10.217 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:19:10.218 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:19:10.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.352+0000 7f954f922700 1 Processor -- start 2023-08-27T11:19:10.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.352+0000 7f954f922700 1 -- start start 2023-08-27T11:19:10.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.353+0000 7f954f922700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f95480b1400 0x7f95480cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:10.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.353+0000 7f954f922700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95480b17e0 0x7f954806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:10.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.353+0000 7f954f922700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9548161850 con 0x7f9548161ce0 2023-08-27T11:19:10.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.353+0000 7f954f922700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954806e7d0 con 0x7f95480b1400 2023-08-27T11:19:10.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.353+0000 7f954f922700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95480cfc10 con 0x7f95480b17e0 2023-08-27T11:19:10.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.353+0000 7f954d6be700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f95480b1400 0x7f95480cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:10.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.353+0000 7f954cebd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95480b17e0 0x7f954806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:10.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.353+0000 7f954d6be700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f95480b1400 0x7f95480cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:59162/0 (socket says 172.21.15.112:59162) 2023-08-27T11:19:10.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.353+0000 7f954debf700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9548161ce0 0x7f954806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:60646/0 (socket says 172.21.15.112:60646) 2023-08-27T11:19:10.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.353+0000 7f954d6be700 1 -- 172.21.15.112:0/1048247053 learned_addr learned my addr 172.21.15.112:0/1048247053 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:10.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.353+0000 7f954d6be700 1 -- 172.21.15.112:0/1048247053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95480b17e0 msgr2=0x7f954806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:10.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.353+0000 7f954d6be700 1 --2- 172.21.15.112:0/1048247053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95480b17e0 0x7f954806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.354+0000 7f954d6be700 1 -- 172.21.15.112:0/1048247053 >> v1:172.21.15.138:6789/0 conn(0x7f9548161ce0 legacy=0x7f954806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:10.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.354+0000 7f954d6be700 1 -- 172.21.15.112:0/1048247053 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9548161200 con 0x7f95480b1400 2023-08-27T11:19:10.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.354+0000 7f954d6be700 1 --2- 172.21.15.112:0/1048247053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95480b1400 0x7f95480cf5a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f95440098f0 tx=0x7f9544009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b096cd9836cef843 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:10.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.354+0000 7f953ffff700 1 -- 172.21.15.112:0/1048247053 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544011830 con 0x7f95480b1400 2023-08-27T11:19:10.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.354+0000 7f953ffff700 1 -- 172.21.15.112:0/1048247053 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9544011990 con 0x7f95480b1400 2023-08-27T11:19:10.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.354+0000 7f953ffff700 1 -- 172.21.15.112:0/1048247053 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544011d20 con 0x7f95480b1400 2023-08-27T11:19:10.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.354+0000 7f954f922700 1 -- 172.21.15.112:0/1048247053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95480b1400 msgr2=0x7f95480cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:10.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.354+0000 7f954f922700 1 --2- 172.21.15.112:0/1048247053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95480b1400 0x7f95480cf5a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f95440098f0 tx=0x7f9544009c00 comp rx=0 tx=0).stop 2023-08-27T11:19:10.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.354+0000 7f954f922700 1 -- 172.21.15.112:0/1048247053 shutdown_connections 2023-08-27T11:19:10.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.354+0000 7f954f922700 1 --2- 172.21.15.112:0/1048247053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95480b1400 0x7f95480cf5a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.354+0000 7f954f922700 1 --2- 172.21.15.112:0/1048247053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95480b17e0 0x7f954806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.354+0000 7f954f922700 1 -- 172.21.15.112:0/1048247053 >> 172.21.15.112:0/1048247053 conn(0x7f95480d0000 msgr2=0x7f95480d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:10.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.355+0000 7f954f922700 1 -- 172.21.15.112:0/1048247053 shutdown_connections 2023-08-27T11:19:10.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.355+0000 7f954f922700 1 -- 172.21.15.112:0/1048247053 wait complete. 2023-08-27T11:19:10.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.356+0000 7f954f922700 1 Processor -- start 2023-08-27T11:19:10.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.356+0000 7f954f922700 1 -- start start 2023-08-27T11:19:10.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.356+0000 7f954f922700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95480b1400 0x7f95481f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:10.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.356+0000 7f954f922700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95480b17e0 0x7f95480c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:10.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.356+0000 7f954f922700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9548161ce0 0x7f954815fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:10.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.356+0000 7f954f922700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95481f40b0 con 0x7f95480b17e0 2023-08-27T11:19:10.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.356+0000 7f954f922700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95480acbc0 con 0x7f95480b1400 2023-08-27T11:19:10.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.356+0000 7f954f922700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95481f6ba0 con 0x7f9548161ce0 2023-08-27T11:19:10.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954cebd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95480b17e0 0x7f95480c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:10.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954cebd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95480b17e0 0x7f95480c6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58744/0 (socket says 172.21.15.112:58744) 2023-08-27T11:19:10.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954cebd700 1 -- 172.21.15.112:0/200824959 learned_addr learned my addr 172.21.15.112:0/200824959 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:10.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954d6be700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95480b1400 0x7f95481f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:10.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954debf700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9548161ce0 0x7f954815fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:10.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954cebd700 1 -- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9548161ce0 msgr2=0x7f954815fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:10.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954cebd700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9548161ce0 0x7f954815fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954cebd700 1 -- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95480b1400 msgr2=0x7f95481f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:10.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954cebd700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95480b1400 0x7f95481f3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954cebd700 1 -- 172.21.15.112:0/200824959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9544009550 con 0x7f95480b17e0 2023-08-27T11:19:10.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954debf700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9548161ce0 0x7f954815fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:10.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954cebd700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95480b17e0 0x7f95480c6380 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f953800e9d0 tx=0x7f953800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:10.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f953dffb700 1 -- 172.21.15.112:0/200824959 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538009730 con 0x7f95480b17e0 2023-08-27T11:19:10.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f953dffb700 1 -- 172.21.15.112:0/200824959 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953800c850 con 0x7f95480b17e0 2023-08-27T11:19:10.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954f922700 1 -- 172.21.15.112:0/200824959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95481f6e80 con 0x7f95480b17e0 2023-08-27T11:19:10.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f953dffb700 1 -- 172.21.15.112:0/200824959 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953800cbe0 con 0x7f95480b17e0 2023-08-27T11:19:10.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.357+0000 7f954f922700 1 -- 172.21.15.112:0/200824959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95481f7440 con 0x7f95480b17e0 2023-08-27T11:19:10.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.358+0000 7f954f922700 1 -- 172.21.15.112:0/200824959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9548067a20 con 0x7f95480b17e0 2023-08-27T11:19:10.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.359+0000 7f953dffb700 1 -- 172.21.15.112:0/200824959 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f953800cd40 con 0x7f95480b17e0 2023-08-27T11:19:10.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.360+0000 7f953dffb700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f952c06a170 0x7f952c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:10.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.360+0000 7f953dffb700 1 -- 172.21.15.112:0/200824959 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9538088bf0 con 0x7f95480b17e0 2023-08-27T11:19:10.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.360+0000 7f954d6be700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f952c06a170 0x7f952c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:10.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.360+0000 7f954d6be700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f952c06a170 0x7f952c06c630 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9544009c40 tx=0x7f9544008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:10.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.363+0000 7f953dffb700 1 -- 172.21.15.112:0/200824959 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9538054590 con 0x7f95480b17e0 2023-08-27T11:19:10.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.595+0000 7f954f922700 1 -- 172.21.15.112:0/200824959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95480ca330 con 0x7f95480b17e0 2023-08-27T11:19:10.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.596+0000 7f953dffb700 1 -- 172.21.15.112:0/200824959 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f9538021e90 con 0x7f95480b17e0 2023-08-27T11:19:10.598 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:10.598 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":33,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1692518554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1692518554},{"type":"v1","addr":"172.21.15.138:6837","nonce":1692518554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3294671617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3294671617},{"type":"v1","addr":"172.21.15.138:6836","nonce":3294671617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3012780135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3012780135},{"type":"v1","addr":"172.21.15.112:6834","nonce":3012780135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6021,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1878199656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1878199656},{"type":"v1","addr":"172.21.15.112:6835","nonce":1878199656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[]} 2023-08-27T11:19:10.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.598+0000 7f954f922700 1 -- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f952c06a170 msgr2=0x7f952c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:10.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.598+0000 7f954f922700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f952c06a170 0x7f952c06c630 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9544009c40 tx=0x7f9544008030 comp rx=0 tx=0).stop 2023-08-27T11:19:10.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.598+0000 7f954f922700 1 -- 172.21.15.112:0/200824959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95480b17e0 msgr2=0x7f95480c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:10.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.598+0000 7f954f922700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95480b17e0 0x7f95480c6380 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f953800e9d0 tx=0x7f953800ece0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.598+0000 7f954f922700 1 -- 172.21.15.112:0/200824959 shutdown_connections 2023-08-27T11:19:10.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.598+0000 7f954f922700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f952c06a170 0x7f952c06c630 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.598+0000 7f954f922700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9548161ce0 0x7f954815fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.598+0000 7f954f922700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95480b1400 0x7f95481f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.598+0000 7f954f922700 1 --2- 172.21.15.112:0/200824959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95480b17e0 0x7f95480c6380 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.598+0000 7f954f922700 1 -- 172.21.15.112:0/200824959 >> 172.21.15.112:0/200824959 conn(0x7f95480d0000 msgr2=0x7f95480cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:10.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.598+0000 7f954f922700 1 -- 172.21.15.112:0/200824959 shutdown_connections 2023-08-27T11:19:10.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.598+0000 7f954f922700 1 -- 172.21.15.112:0/200824959 wait complete. 2023-08-27T11:19:10.602 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 33 2023-08-27T11:19:10.613 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:19:10.613 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:19:10.614 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:19:10.614 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:19:10.614 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:19:10.614 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:19:10.615 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:19:10.615 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:19:10.615 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:19:10.616 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:19:10.616 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:19:10.616 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:19:10.617 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:19:10.617 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:19:10.617 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:19:10.617 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:19:10.625 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:19:10.625 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:19:10.626 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:19:10.626 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:19:10.627 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:19:10.628 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:19:10.629 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:19:10.629 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:19:10.667 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:19:10.668 DEBUG:teuthology.parallel:result is None 2023-08-27T11:19:10.669 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:19:10.670 DEBUG:teuthology.parallel:result is None 2023-08-27T11:19:10.670 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:19:10.671 DEBUG:teuthology.parallel:result is None 2023-08-27T11:19:10.671 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:19:10.672 DEBUG:teuthology.parallel:result is None 2023-08-27T11:19:10.672 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:19:10.747 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:19:10.747 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:19:10.758 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2023-08-27T11:19:10.758 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:19:10.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.799+0000 7f05562a8700 1 Processor -- start 2023-08-27T11:19:10.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.799+0000 7f05562a8700 1 -- start start 2023-08-27T11:19:10.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f05562a8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0550146990 0x7f05500b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:10.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f05562a8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055006f930 con 0x7f055006d510 2023-08-27T11:19:10.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f05562a8700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055006fc10 con 0x7f0550160a00 2023-08-27T11:19:10.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f05562a8700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055006fef0 con 0x7f0550146990 2023-08-27T11:19:10.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f0554845700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f055006d510 0x7f055006d910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53162/0 (socket says 172.21.15.112:53162) 2023-08-27T11:19:10.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f0554845700 1 -- 172.21.15.112:0/2827578064 learned_addr learned my addr 172.21.15.112:0/2827578064 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:10.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f054f7fe700 1 --2- 172.21.15.112:0/2827578064 >> v2:172.21.15.138:3300/0 conn(0x7f0550146990 0x7f05500b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:10.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f054effd700 1 -- 172.21.15.112:0/2827578064 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167390049 0 0) 0x7f055006f930 con 0x7f055006d510 2023-08-27T11:19:10.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f054effd700 1 -- 172.21.15.112:0/2827578064 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05380035b0 con 0x7f055006d510 2023-08-27T11:19:10.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f054effd700 1 -- 172.21.15.112:0/2827578064 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3774275211 0 0) 0x7f05380035b0 con 0x7f055006d510 2023-08-27T11:19:10.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f054f7fe700 1 -- 172.21.15.112:0/2827578064 >> v1:172.21.15.138:6789/0 conn(0x7f0550160a00 legacy=0x7f055015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:10.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f054f7fe700 1 -- 172.21.15.112:0/2827578064 >> v1:172.21.15.112:6789/0 conn(0x7f055006d510 legacy=0x7f055006d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:10.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f054f7fe700 1 -- 172.21.15.112:0/2827578064 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05500701d0 con 0x7f0550146990 2023-08-27T11:19:10.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.800+0000 7f054f7fe700 1 --2- 172.21.15.112:0/2827578064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0550146990 0x7f05500b46d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0544009840 tx=0x7f0544009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=353dcc2b835800fc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:10.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.801+0000 7f054effd700 1 -- 172.21.15.112:0/2827578064 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05480019f0 con 0x7f0550146990 2023-08-27T11:19:10.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.801+0000 7f054effd700 1 -- 172.21.15.112:0/2827578064 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0544004030 con 0x7f0550146990 2023-08-27T11:19:10.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.801+0000 7f054effd700 1 -- 172.21.15.112:0/2827578064 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05440043c0 con 0x7f0550146990 2023-08-27T11:19:10.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.801+0000 7f05562a8700 1 -- 172.21.15.112:0/2827578064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0550146990 msgr2=0x7f05500b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:10.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.801+0000 7f05562a8700 1 --2- 172.21.15.112:0/2827578064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0550146990 0x7f05500b46d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0544009840 tx=0x7f0544009b50 comp rx=0 tx=0).stop 2023-08-27T11:19:10.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.801+0000 7f05562a8700 1 -- 172.21.15.112:0/2827578064 shutdown_connections 2023-08-27T11:19:10.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.801+0000 7f05562a8700 1 --2- 172.21.15.112:0/2827578064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0550146990 0x7f05500b46d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.801+0000 7f05562a8700 1 -- 172.21.15.112:0/2827578064 >> 172.21.15.112:0/2827578064 conn(0x7f05500cfe50 msgr2=0x7f05500d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:10.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.801+0000 7f05562a8700 1 -- 172.21.15.112:0/2827578064 shutdown_connections 2023-08-27T11:19:10.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.801+0000 7f05562a8700 1 -- 172.21.15.112:0/2827578064 wait complete. 2023-08-27T11:19:10.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.802+0000 7f05562a8700 1 Processor -- start 2023-08-27T11:19:10.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.802+0000 7f05562a8700 1 -- start start 2023-08-27T11:19:10.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.803+0000 7f05562a8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f055006d510 0x7f05501419b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:10.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.803+0000 7f05562a8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0550146990 0x7f055015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:10.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.803+0000 7f05562a8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0550160a00 0x7f055015e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:10.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.803+0000 7f05562a8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0550140700 con 0x7f055006d510 2023-08-27T11:19:10.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.803+0000 7f05562a8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0550140910 con 0x7f0550146990 2023-08-27T11:19:10.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.803+0000 7f05562a8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0550140bf0 con 0x7f0550160a00 2023-08-27T11:19:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.803+0000 7f054ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f055006d510 0x7f05501419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.803+0000 7f054ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f055006d510 0x7f05501419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58782/0 (socket says 172.21.15.112:58782) 2023-08-27T11:19:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.803+0000 7f054ffff700 1 -- 172.21.15.112:0/1125872134 learned_addr learned my addr 172.21.15.112:0/1125872134 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.804+0000 7f054ffff700 1 -- 172.21.15.112:0/1125872134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0550160a00 msgr2=0x7f055015e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.804+0000 7f054ffff700 1 --2- 172.21.15.112:0/1125872134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0550160a00 0x7f055015e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.804+0000 7f054ffff700 1 -- 172.21.15.112:0/1125872134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0550146990 msgr2=0x7f055015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:10.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.804+0000 7f054f7fe700 1 --2- 172.21.15.112:0/1125872134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0550146990 0x7f055015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:10.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.804+0000 7f054ffff700 1 --2- 172.21.15.112:0/1125872134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0550146990 0x7f055015dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:10.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.804+0000 7f054ffff700 1 -- 172.21.15.112:0/1125872134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05440094a0 con 0x7f055006d510 2023-08-27T11:19:10.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.804+0000 7f054ffff700 1 --2- 172.21.15.112:0/1125872134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f055006d510 0x7f05501419b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f054000c550 tx=0x7f054000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:10.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.804+0000 7f054cff9700 1 -- 172.21.15.112:0/1125872134 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400054e0 con 0x7f055006d510 2023-08-27T11:19:10.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.804+0000 7f054cff9700 1 -- 172.21.15.112:0/1125872134 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0540003680 con 0x7f055006d510 2023-08-27T11:19:10.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.804+0000 7f05562a8700 1 -- 172.21.15.112:0/1125872134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0550140ed0 con 0x7f055006d510 2023-08-27T11:19:10.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.804+0000 7f054cff9700 1 -- 172.21.15.112:0/1125872134 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540003a30 con 0x7f055006d510 2023-08-27T11:19:10.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.804+0000 7f05562a8700 1 -- 172.21.15.112:0/1125872134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0550141490 con 0x7f055006d510 2023-08-27T11:19:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.805+0000 7f05562a8700 1 -- 172.21.15.112:0/1125872134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0550067a20 con 0x7f055006d510 2023-08-27T11:19:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.805+0000 7f054cff9700 1 -- 172.21.15.112:0/1125872134 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f0540003b90 con 0x7f055006d510 2023-08-27T11:19:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.806+0000 7f054cff9700 1 --2- 172.21.15.112:0/1125872134 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f053c06a170 0x7f053c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.806+0000 7f054cff9700 1 -- 172.21.15.112:0/1125872134 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0540087790 con 0x7f055006d510 2023-08-27T11:19:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.806+0000 7f054f7fe700 1 --2- 172.21.15.112:0/1125872134 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f053c06a170 0x7f053c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.806+0000 7f054f7fe700 1 --2- 172.21.15.112:0/1125872134 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f053c06a170 0x7f053c06c630 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f05501422e0 tx=0x7f0544008000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:10.810+0000 7f054cff9700 1 -- 172.21.15.112:0/1125872134 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f05400531b0 con 0x7f055006d510 2023-08-27T11:19:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:11.014+0000 7f05562a8700 1 -- 172.21.15.112:0/1125872134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f05500cf450 con 0x7f055006d510 2023-08-27T11:19:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.022+0000 7f054cff9700 1 -- 172.21.15.112:0/1125872134 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0540017080 con 0x7f055006d510 2023-08-27T11:19:12.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.025+0000 7f05562a8700 1 -- 172.21.15.112:0/1125872134 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f053c06a170 msgr2=0x7f053c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.025+0000 7f05562a8700 1 --2- 172.21.15.112:0/1125872134 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f053c06a170 0x7f053c06c630 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f05501422e0 tx=0x7f0544008000 comp rx=0 tx=0).stop 2023-08-27T11:19:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.025+0000 7f05562a8700 1 -- 172.21.15.112:0/1125872134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f055006d510 msgr2=0x7f05501419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.025+0000 7f05562a8700 1 --2- 172.21.15.112:0/1125872134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f055006d510 0x7f05501419b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f054000c550 tx=0x7f054000c910 comp rx=0 tx=0).stop 2023-08-27T11:19:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.025+0000 7f05562a8700 1 -- 172.21.15.112:0/1125872134 shutdown_connections 2023-08-27T11:19:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.025+0000 7f05562a8700 1 --2- 172.21.15.112:0/1125872134 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f053c06a170 0x7f053c06c630 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.025+0000 7f05562a8700 1 --2- 172.21.15.112:0/1125872134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0550160a00 0x7f055015e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.025+0000 7f05562a8700 1 --2- 172.21.15.112:0/1125872134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f055006d510 0x7f05501419b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.025+0000 7f05562a8700 1 --2- 172.21.15.112:0/1125872134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0550146990 0x7f055015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.025+0000 7f05562a8700 1 -- 172.21.15.112:0/1125872134 >> 172.21.15.112:0/1125872134 conn(0x7f05500cfe50 msgr2=0x7f055006d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:12.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.025+0000 7f05562a8700 1 -- 172.21.15.112:0/1125872134 shutdown_connections 2023-08-27T11:19:12.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.025+0000 7f05562a8700 1 -- 172.21.15.112:0/1125872134 wait complete. 2023-08-27T11:19:12.029 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2023-08-27T11:19:12.042 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:19:12.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.168+0000 7f73fc958700 1 Processor -- start 2023-08-27T11:19:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.168+0000 7f73fc958700 1 -- start start 2023-08-27T11:19:12.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73fc958700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f8163f90 0x7f73f8166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73fc958700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f80b5ae0 con 0x7f73f8144570 2023-08-27T11:19:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73fc958700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f80b45f0 con 0x7f73f80b3e50 2023-08-27T11:19:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73fc958700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80cf7a0 con 0x7f73f8163f90 2023-08-27T11:19:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f659c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f73f8144570 0x7f73f815ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53176/0 (socket says 172.21.15.112:53176) 2023-08-27T11:19:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f659c700 1 -- 172.21.15.112:0/3961049177 learned_addr learned my addr 172.21.15.112:0/3961049177 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:12.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f6d9d700 1 --2- 172.21.15.112:0/3961049177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f8163f90 0x7f73f8166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934080862 0 0) 0x7f73e8001110 con 0x7f73f8144570 2023-08-27T11:19:12.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73e00035b0 con 0x7f73f8144570 2023-08-27T11:19:12.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 676358259 0 0) 0x7f73e00035b0 con 0x7f73f8144570 2023-08-27T11:19:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f8163f90 msgr2=0x7f73f8166380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 --2- 172.21.15.112:0/3961049177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f8163f90 0x7f73f8166380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 >> v1:172.21.15.138:6789/0 conn(0x7f73f80b3e50 legacy=0x7f73f815f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f815fc40 con 0x7f73f8144570 2023-08-27T11:19:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f73e8001800 con 0x7f73f8144570 2023-08-27T11:19:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f6d9d700 1 --2- 172.21.15.112:0/3961049177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f8163f90 0x7f73f8166380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:19:12.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 >> v1:172.21.15.112:6789/0 conn(0x7f73f8144570 legacy=0x7f73f815ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 --2- 172.21.15.112:0/3961049177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73e0003a30 0x7f73e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 --2- 172.21.15.112:0/3961049177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f8163f90 0x7f73e0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 --2- 172.21.15.112:0/3961049177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73e0004d90 0x7f73e0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80cf7a0 con 0x7f73e0004d90 2023-08-27T11:19:12.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e000ba60 con 0x7f73e0003a30 2023-08-27T11:19:12.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.169+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e000bc20 con 0x7f73f8163f90 2023-08-27T11:19:12.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.170+0000 7f73f6d9d700 1 --2- 172.21.15.112:0/3961049177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73e0004d90 0x7f73e0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.170+0000 7f73f659c700 1 --2- 172.21.15.112:0/3961049177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73e0003a30 0x7f73e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.170+0000 7f73f5d9b700 1 --2- 172.21.15.112:0/3961049177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f8163f90 0x7f73e0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.170+0000 7f73f6d9d700 1 -- 172.21.15.112:0/3961049177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f8163f90 msgr2=0x7f73e0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.170+0000 7f73f6d9d700 1 --2- 172.21.15.112:0/3961049177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73f8163f90 0x7f73e0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.170+0000 7f73f6d9d700 1 -- 172.21.15.112:0/3961049177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73e0003a30 msgr2=0x7f73e0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.170+0000 7f73f6d9d700 1 --2- 172.21.15.112:0/3961049177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73e0003a30 0x7f73e0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.170+0000 7f73f6d9d700 1 -- 172.21.15.112:0/3961049177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0009a70 con 0x7f73e0004d90 2023-08-27T11:19:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.170+0000 7f73f6d9d700 1 --2- 172.21.15.112:0/3961049177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73e0004d90 0x7f73e0005210 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f73f806e630 tx=0x7f73e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe40b40c9b0e986 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.170+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f000ed30 con 0x7f73e0004d90 2023-08-27T11:19:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.170+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f000f040 con 0x7f73e0004d90 2023-08-27T11:19:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.170+0000 7f73f559a700 1 -- 172.21.15.112:0/3961049177 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0003c40 con 0x7f73e0004d90 2023-08-27T11:19:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.171+0000 7f73fc958700 1 -- 172.21.15.112:0/3961049177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73e0004d90 msgr2=0x7f73e0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.171+0000 7f73fc958700 1 --2- 172.21.15.112:0/3961049177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73e0004d90 0x7f73e0005210 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f73f806e630 tx=0x7f73e0005880 comp rx=0 tx=0).stop 2023-08-27T11:19:12.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.171+0000 7f73f6d9d700 1 -- 172.21.15.112:0/3961049177 reap_dead start 2023-08-27T11:19:12.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.171+0000 7f73fc958700 1 -- 172.21.15.112:0/3961049177 shutdown_connections 2023-08-27T11:19:12.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.171+0000 7f73fc958700 1 -- 172.21.15.112:0/3961049177 >> 172.21.15.112:0/3961049177 conn(0x7f73f806f970 msgr2=0x7f73f806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:12.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.171+0000 7f73fc958700 1 -- 172.21.15.112:0/3961049177 shutdown_connections 2023-08-27T11:19:12.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.171+0000 7f73fc958700 1 -- 172.21.15.112:0/3961049177 wait complete. 2023-08-27T11:19:12.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.172+0000 7f73fc958700 1 Processor -- start 2023-08-27T11:19:12.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.172+0000 7f73fc958700 1 -- start start 2023-08-27T11:19:12.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73fc958700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73e0004d90 0x7f73f813cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73fc958700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f81f6190 0x7f73f81f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73fc958700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f81562c0 0x7f73f8156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73fc958700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f8156c90 con 0x7f73f81562c0 2023-08-27T11:19:12.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73fc958700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f813a2e0 con 0x7f73f81f6190 2023-08-27T11:19:12.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73fc958700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f81f6fe0 con 0x7f73e0004d90 2023-08-27T11:19:12.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73f6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f81562c0 0x7f73f8156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73f6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f81562c0 0x7f73f8156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58810/0 (socket says 172.21.15.112:58810) 2023-08-27T11:19:12.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73f6d9d700 1 -- 172.21.15.112:0/1359361199 learned_addr learned my addr 172.21.15.112:0/1359361199 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:12.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73f659c700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73e0004d90 0x7f73f813cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73f5d9b700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f81f6190 0x7f73f81f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73f6d9d700 1 -- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73e0004d90 msgr2=0x7f73f813cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73f6d9d700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73e0004d90 0x7f73f813cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73f6d9d700 1 -- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f81f6190 msgr2=0x7f73f81f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73f6d9d700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f81f6190 0x7f73f81f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73f6d9d700 1 -- 172.21.15.112:0/1359361199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f81f6d00 con 0x7f73f81562c0 2023-08-27T11:19:12.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73f6d9d700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f81562c0 0x7f73f8156750 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f73f000b010 tx=0x7f73f0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:12.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73e6ffd700 1 -- 172.21.15.112:0/1359361199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0004440 con 0x7f73f81562c0 2023-08-27T11:19:12.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73e6ffd700 1 -- 172.21.15.112:0/1359361199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f00045a0 con 0x7f73f81562c0 2023-08-27T11:19:12.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73fc958700 1 -- 172.21.15.112:0/1359361199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f8070cb0 con 0x7f73f81562c0 2023-08-27T11:19:12.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.173+0000 7f73e6ffd700 1 -- 172.21.15.112:0/1359361199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0004950 con 0x7f73f81562c0 2023-08-27T11:19:12.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.174+0000 7f73fc958700 1 -- 172.21.15.112:0/1359361199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f8071270 con 0x7f73f81562c0 2023-08-27T11:19:12.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.174+0000 7f73fc958700 1 -- 172.21.15.112:0/1359361199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f8071400 con 0x7f73f81562c0 2023-08-27T11:19:12.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.175+0000 7f73e6ffd700 1 -- 172.21.15.112:0/1359361199 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f73f0024020 con 0x7f73f81562c0 2023-08-27T11:19:12.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.175+0000 7f73e6ffd700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73d806a170 0x7f73d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.175+0000 7f73e6ffd700 1 -- 172.21.15.112:0/1359361199 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f73f0089f20 con 0x7f73f81562c0 2023-08-27T11:19:12.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.175+0000 7f73f659c700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73d806a170 0x7f73d806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.176+0000 7f73f659c700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73d806a170 0x7f73d806c630 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f73e8000c00 tx=0x7f73e8007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:12.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.180+0000 7f73e6ffd700 1 -- 172.21.15.112:0/1359361199 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f73f0055b20 con 0x7f73f81562c0 2023-08-27T11:19:12.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.446+0000 7f73fc958700 1 -- 172.21.15.112:0/1359361199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f73f8067a20 con 0x7f73f81562c0 2023-08-27T11:19:12.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.447+0000 7f73e6ffd700 1 -- 172.21.15.112:0/1359361199 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f73f0009de0 con 0x7f73f81562c0 2023-08-27T11:19:12.449 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:12.449 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:19:12.450 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2023-08-27T11:19:12.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.451 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2023-08-27T11:19:12.451 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk2bnkLBAA+vxcTeJzucKk9iSF/Zw3Lg==", 2023-08-27T11:19:12.451 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.451 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:19:12.451 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:19:12.452 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:19:12.452 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.452 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.453 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2023-08-27T11:19:12.453 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk3zD5OBAASVnwG6yn/YQzfYyeMakFpA==", 2023-08-27T11:19:12.453 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.453 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:19:12.453 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:19:12.454 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:19:12.454 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.454 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.455 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2023-08-27T11:19:12.455 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtki7wJNBAAsIGgDLM82sWNcKnCDlSBMw==", 2023-08-27T11:19:12.455 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.456 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:19:12.456 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:19:12.456 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:19:12.456 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.457 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.457 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2023-08-27T11:19:12.457 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkCsW/BBAAFTPiTOpBo1L+rLo9px+AjQ==", 2023-08-27T11:19:12.458 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.458 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:19:12.459 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:19:12.459 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:19:12.459 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.459 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.460 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2023-08-27T11:19:12.460 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB7MOtkR/cSHhAAR8e8VRd+e0ESJkqQDYWbEw==", 2023-08-27T11:19:12.460 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.461 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:19:12.461 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:19:12.461 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.462 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.462 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2023-08-27T11:19:12.463 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB+MOtksBF3HRAA1OIXiIkhgPDYpQyYPaWOXg==", 2023-08-27T11:19:12.463 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.463 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:19:12.463 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:19:12.464 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.464 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.465 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2023-08-27T11:19:12.465 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCBMOtkWsrdHBAAkfWbiE6wAFzpt9+BxRh8fw==", 2023-08-27T11:19:12.465 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.466 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:19:12.466 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:19:12.466 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.466 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.467 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2023-08-27T11:19:12.467 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCEMOtkIkiKHRAARDoh/FH9ZdqUlZZ29Mc/Dw==", 2023-08-27T11:19:12.467 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:19:12.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:19:12.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.468 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.469 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2023-08-27T11:19:12.469 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCKMOtkPgDzGxAA+U4kbwD0d4lS8aNPG2CgHw==", 2023-08-27T11:19:12.469 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.470 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:19:12.470 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:19:12.470 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.470 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.471 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2023-08-27T11:19:12.471 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCNMOtkKomSGxAAGAQUcIH2H/uyZIU8vlq1pw==", 2023-08-27T11:19:12.471 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.471 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:19:12.472 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:19:12.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.472 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.473 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2023-08-27T11:19:12.473 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCQMOtkHI2MGhAA7sgbi8JXO8P9wnVvMy3PqA==", 2023-08-27T11:19:12.473 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.473 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:19:12.473 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:19:12.473 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.474 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.474 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2023-08-27T11:19:12.475 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCTMOtkVHpiGhAA4C4YJMWbQ0mmGgB4i8G/AA==", 2023-08-27T11:19:12.475 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:19:12.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:19:12.475 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.476 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.476 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2023-08-27T11:19:12.477 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA==", 2023-08-27T11:19:12.477 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.477 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:19:12.477 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2023-08-27T11:19:12.477 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:19:12.478 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.478 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2023-08-27T11:19:12.478 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtk66DbDxAAWZ5StELux0lriJ2gEN9icg==", 2023-08-27T11:19:12.478 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.479 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:19:12.479 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:19:12.479 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:19:12.479 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:19:12.480 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.480 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2023-08-27T11:19:12.480 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkViKJDBAAF2ofwO+78er9OyFfn2bO+g==", 2023-08-27T11:19:12.481 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.481 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:19:12.481 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:19:12.481 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:19:12.482 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:19:12.482 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.482 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2023-08-27T11:19:12.483 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkwt+QFRAAowWE69VqeRhQz0896rxx9Q==", 2023-08-27T11:19:12.483 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.483 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:19:12.483 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:19:12.483 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:19:12.483 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:19:12.484 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.484 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2023-08-27T11:19:12.484 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkjDA+DxAAjmOn82AC2OMpKlAVCD6hlg==", 2023-08-27T11:19:12.485 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.485 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:19:12.485 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:19:12.485 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:19:12.486 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.486 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.486 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2023-08-27T11:19:12.487 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkXguHKRAALHwzk8diVXqsMvTKr+AzLQ==", 2023-08-27T11:19:12.487 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.487 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:19:12.487 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:19:12.487 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.488 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:12.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:12.488 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2023-08-27T11:19:12.489 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkrRjBJRAAq3ftmIh81LKadpliVUiJ6g==", 2023-08-27T11:19:12.489 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:19:12.489 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:19:12.489 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:19:12.490 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:19:12.490 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.490 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:12.490 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:19:12.491 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:19:12.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.449+0000 7f73fc958700 1 -- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73d806a170 msgr2=0x7f73d806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.449+0000 7f73fc958700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73d806a170 0x7f73d806c630 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f73e8000c00 tx=0x7f73e8007eb0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.449+0000 7f73fc958700 1 -- 172.21.15.112:0/1359361199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f81562c0 msgr2=0x7f73f8156750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.449+0000 7f73fc958700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f81562c0 0x7f73f8156750 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f73f000b010 tx=0x7f73f0009750 comp rx=0 tx=0).stop 2023-08-27T11:19:12.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.449+0000 7f73fc958700 1 -- 172.21.15.112:0/1359361199 shutdown_connections 2023-08-27T11:19:12.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.449+0000 7f73fc958700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73d806a170 0x7f73d806c630 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.449+0000 7f73fc958700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f81562c0 0x7f73f8156750 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.449+0000 7f73fc958700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73e0004d90 0x7f73f813cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.449+0000 7f73fc958700 1 --2- 172.21.15.112:0/1359361199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f81f6190 0x7f73f81f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.449+0000 7f73fc958700 1 -- 172.21.15.112:0/1359361199 >> 172.21.15.112:0/1359361199 conn(0x7f73f806f970 msgr2=0x7f73f806ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:12.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.449+0000 7f73fc958700 1 -- 172.21.15.112:0/1359361199 shutdown_connections 2023-08-27T11:19:12.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.449+0000 7f73fc958700 1 -- 172.21.15.112:0/1359361199 wait complete. 2023-08-27T11:19:12.496 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:19:12.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.615+0000 7f0f6ab7a700 1 Processor -- start 2023-08-27T11:19:12.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.615+0000 7f0f6ab7a700 1 -- start start 2023-08-27T11:19:12.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.616+0000 7f0f6ab7a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0f640b1400 0x7f0f640cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.616+0000 7f0f6ab7a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0f64161ce0 0x7f0f6406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.616+0000 7f0f6ab7a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f64161850 con 0x7f0f640b17e0 2023-08-27T11:19:12.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.616+0000 7f0f6ab7a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f640cfc10 con 0x7f0f640b1400 2023-08-27T11:19:12.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.616+0000 7f0f6ab7a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f64161200 con 0x7f0f64161ce0 2023-08-27T11:19:12.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.616+0000 7f0f68916700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0f640b1400 0x7f0f640cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.616+0000 7f0f68916700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0f640b1400 0x7f0f640cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58814/0 (socket says 172.21.15.112:58814) 2023-08-27T11:19:12.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.616+0000 7f0f68916700 1 -- 172.21.15.112:0/1134048263 learned_addr learned my addr 172.21.15.112:0/1134048263 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:12.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.616+0000 7f0f69117700 1 --2- 172.21.15.112:0/1134048263 >> v2:172.21.15.138:3300/0 conn(0x7f0f64161ce0 0x7f0f6406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.616+0000 7f0f68916700 1 -- 172.21.15.112:0/1134048263 >> v2:172.21.15.138:3300/0 conn(0x7f0f64161ce0 msgr2=0x7f0f6406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:12.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.616+0000 7f0f68916700 1 --2- 172.21.15.112:0/1134048263 >> v2:172.21.15.138:3300/0 conn(0x7f0f64161ce0 0x7f0f6406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.616+0000 7f0f68916700 1 -- 172.21.15.112:0/1134048263 >> v1:172.21.15.138:6789/0 conn(0x7f0f640b17e0 legacy=0x7f0f6406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.616+0000 7f0f68916700 1 -- 172.21.15.112:0/1134048263 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f641614e0 con 0x7f0f640b1400 2023-08-27T11:19:12.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.617+0000 7f0f68916700 1 --2- 172.21.15.112:0/1134048263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f640b1400 0x7f0f640cf5a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0f6406df10 tx=0x7f0f58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8126ed02408ad899 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:12.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.617+0000 7f0f637fe700 1 -- 172.21.15.112:0/1134048263 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5800b870 con 0x7f0f640b1400 2023-08-27T11:19:12.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.617+0000 7f0f637fe700 1 -- 172.21.15.112:0/1134048263 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5800b9d0 con 0x7f0f640b1400 2023-08-27T11:19:12.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.617+0000 7f0f637fe700 1 -- 172.21.15.112:0/1134048263 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5800bd60 con 0x7f0f640b1400 2023-08-27T11:19:12.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.617+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/1134048263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f640b1400 msgr2=0x7f0f640cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.617+0000 7f0f6ab7a700 1 --2- 172.21.15.112:0/1134048263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f640b1400 0x7f0f640cf5a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0f6406df10 tx=0x7f0f58009bd0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.617+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/1134048263 shutdown_connections 2023-08-27T11:19:12.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.617+0000 7f0f6ab7a700 1 --2- 172.21.15.112:0/1134048263 >> v2:172.21.15.138:3300/0 conn(0x7f0f64161ce0 0x7f0f6406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.617+0000 7f0f6ab7a700 1 --2- 172.21.15.112:0/1134048263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f640b1400 0x7f0f640cf5a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.617+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/1134048263 >> 172.21.15.112:0/1134048263 conn(0x7f0f640d0000 msgr2=0x7f0f640d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:12.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.617+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/1134048263 shutdown_connections 2023-08-27T11:19:12.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.617+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/1134048263 wait complete. 2023-08-27T11:19:12.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.618+0000 7f0f6ab7a700 1 Processor -- start 2023-08-27T11:19:12.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.619+0000 7f0f6ab7a700 1 -- start start 2023-08-27T11:19:12.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.619+0000 7f0f6ab7a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f640b1400 0x7f0f641508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.619+0000 7f0f6ab7a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f640b17e0 0x7f0f640c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.619+0000 7f0f6ab7a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f64161ce0 0x7f0f641561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.619+0000 7f0f6ab7a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f641f66c0 con 0x7f0f640b1400 2023-08-27T11:19:12.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.619+0000 7f0f6ab7a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f641f69a0 con 0x7f0f640b17e0 2023-08-27T11:19:12.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.619+0000 7f0f6ab7a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f641f6c80 con 0x7f0f64161ce0 2023-08-27T11:19:12.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f68916700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f640b1400 0x7f0f641508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f68916700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f640b1400 0x7f0f641508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58820/0 (socket says 172.21.15.112:58820) 2023-08-27T11:19:12.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f68916700 1 -- 172.21.15.112:0/3528133110 learned_addr learned my addr 172.21.15.112:0/3528133110 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:12.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f63fff700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f640b17e0 0x7f0f640c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f69117700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f64161ce0 0x7f0f641561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f68916700 1 -- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f64161ce0 msgr2=0x7f0f641561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f68916700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f64161ce0 0x7f0f641561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f68916700 1 -- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f640b17e0 msgr2=0x7f0f640c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f68916700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f640b17e0 0x7f0f640c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f68916700 1 -- 172.21.15.112:0/3528133110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f58009550 con 0x7f0f640b1400 2023-08-27T11:19:12.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f63fff700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f640b17e0 0x7f0f640c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:12.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f69117700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f64161ce0 0x7f0f641561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:12.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f68916700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f640b1400 0x7f0f641508c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0f58000c00 tx=0x7f0f58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:12.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f617fa700 1 -- 172.21.15.112:0/3528133110 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58004520 con 0x7f0f640b1400 2023-08-27T11:19:12.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f617fa700 1 -- 172.21.15.112:0/3528133110 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5801e070 con 0x7f0f640b1400 2023-08-27T11:19:12.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/3528133110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f641f6f60 con 0x7f0f640b1400 2023-08-27T11:19:12.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f617fa700 1 -- 172.21.15.112:0/3528133110 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f580038a0 con 0x7f0f640b1400 2023-08-27T11:19:12.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.620+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/3528133110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f641514a0 con 0x7f0f640b1400 2023-08-27T11:19:12.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.621+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/3528133110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f64067a20 con 0x7f0f640b1400 2023-08-27T11:19:12.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.622+0000 7f0f617fa700 1 -- 172.21.15.112:0/3528133110 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f0f58003a00 con 0x7f0f640b1400 2023-08-27T11:19:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.622+0000 7f0f617fa700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0f4806a170 0x7f0f4806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.622+0000 7f0f617fa700 1 -- 172.21.15.112:0/3528133110 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0f58020070 con 0x7f0f640b1400 2023-08-27T11:19:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.622+0000 7f0f63fff700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0f4806a170 0x7f0f4806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.623+0000 7f0f63fff700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0f4806a170 0x7f0f4806c630 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0f640c3f30 tx=0x7f0f50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.626+0000 7f0f617fa700 1 -- 172.21.15.112:0/3528133110 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0f580552d0 con 0x7f0f640b1400 2023-08-27T11:19:12.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.817+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/3528133110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f6406db60 con 0x7f0f640b1400 2023-08-27T11:19:12.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.817+0000 7f0f617fa700 1 -- 172.21.15.112:0/3528133110 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+14606 (secure 0 0 0) 0x7f0f58006ce0 con 0x7f0f640b1400 2023-08-27T11:19:12.819 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:12.819 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":61,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:19:12.018944+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:19:12.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.820+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0f4806a170 msgr2=0x7f0f4806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.820+0000 7f0f6ab7a700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0f4806a170 0x7f0f4806c630 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0f640c3f30 tx=0x7f0f50009040 comp rx=0 tx=0).stop 2023-08-27T11:19:12.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.820+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/3528133110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f640b1400 msgr2=0x7f0f641508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.820+0000 7f0f6ab7a700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f640b1400 0x7f0f641508c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0f58000c00 tx=0x7f0f58004750 comp rx=0 tx=0).stop 2023-08-27T11:19:12.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.820+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/3528133110 shutdown_connections 2023-08-27T11:19:12.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.820+0000 7f0f6ab7a700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0f4806a170 0x7f0f4806c630 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.820+0000 7f0f6ab7a700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f64161ce0 0x7f0f641561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.820+0000 7f0f6ab7a700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f640b1400 0x7f0f641508c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.820+0000 7f0f6ab7a700 1 --2- 172.21.15.112:0/3528133110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f640b17e0 0x7f0f640c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.820+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/3528133110 >> 172.21.15.112:0/3528133110 conn(0x7f0f640d0000 msgr2=0x7f0f640cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:12.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.820+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/3528133110 shutdown_connections 2023-08-27T11:19:12.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.820+0000 7f0f6ab7a700 1 -- 172.21.15.112:0/3528133110 wait complete. 2023-08-27T11:19:12.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:19:12.834 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:19:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.955+0000 7fa517b4c700 1 Processor -- start 2023-08-27T11:19:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.956+0000 7fa517b4c700 1 -- start start 2023-08-27T11:19:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.956+0000 7fa517b4c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa51013f370 0x7fa51015e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.956+0000 7fa517b4c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa51006e4c0 0x7fa51015aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.956+0000 7fa517b4c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51015dc40 con 0x7fa51015a600 2023-08-27T11:19:12.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.956+0000 7fa517b4c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51015b110 con 0x7fa51006e4c0 2023-08-27T11:19:12.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.956+0000 7fa517b4c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51015b3f0 con 0x7fa51013f370 2023-08-27T11:19:12.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.957+0000 7fa5160e9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa51006e4c0 0x7fa51015aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.957+0000 7fa5150e7700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa51015a600 0x7fa51006ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53210/0 (socket says 172.21.15.112:53210) 2023-08-27T11:19:12.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.957+0000 7fa5150e7700 1 -- 172.21.15.112:0/1146734282 learned_addr learned my addr 172.21.15.112:0/1146734282 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:12.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.957+0000 7fa5158e8700 1 --2- 172.21.15.112:0/1146734282 >> v2:172.21.15.138:3300/0 conn(0x7fa51013f370 0x7fa51015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.957+0000 7fa5160e9700 1 -- 172.21.15.112:0/1146734282 >> v2:172.21.15.138:3300/0 conn(0x7fa51013f370 msgr2=0x7fa51015e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:12.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.957+0000 7fa5160e9700 1 --2- 172.21.15.112:0/1146734282 >> v2:172.21.15.138:3300/0 conn(0x7fa51013f370 0x7fa51015e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.957+0000 7fa5160e9700 1 -- 172.21.15.112:0/1146734282 >> v1:172.21.15.112:6789/0 conn(0x7fa51015a600 legacy=0x7fa51006ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.957+0000 7fa5160e9700 1 -- 172.21.15.112:0/1146734282 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51015de90 con 0x7fa51006e4c0 2023-08-27T11:19:12.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.957+0000 7fa5148e6700 1 -- 172.21.15.112:0/1146734282 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167499035 0 0) 0x7fa51015dc40 con 0x7fa51015a600 2023-08-27T11:19:12.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.957+0000 7fa5160e9700 1 --2- 172.21.15.112:0/1146734282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51006e4c0 0x7fa51015aa90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa51006bc30 tx=0x7fa50800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18c5feaa7ac8330 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:12.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.957+0000 7fa5148e6700 1 -- 172.21.15.112:0/1146734282 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50800e070 con 0x7fa51006e4c0 2023-08-27T11:19:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.957+0000 7fa5148e6700 1 -- 172.21.15.112:0/1146734282 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50800bbd0 con 0x7fa51006e4c0 2023-08-27T11:19:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.957+0000 7fa5148e6700 1 -- 172.21.15.112:0/1146734282 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508012430 con 0x7fa51006e4c0 2023-08-27T11:19:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.958+0000 7fa517b4c700 1 -- 172.21.15.112:0/1146734282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51006e4c0 msgr2=0x7fa51015aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.958+0000 7fa517b4c700 1 --2- 172.21.15.112:0/1146734282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51006e4c0 0x7fa51015aa90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa51006bc30 tx=0x7fa50800b330 comp rx=0 tx=0).stop 2023-08-27T11:19:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.958+0000 7fa517b4c700 1 -- 172.21.15.112:0/1146734282 shutdown_connections 2023-08-27T11:19:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.958+0000 7fa517b4c700 1 --2- 172.21.15.112:0/1146734282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51006e4c0 0x7fa51015aa90 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.958+0000 7fa517b4c700 1 --2- 172.21.15.112:0/1146734282 >> v2:172.21.15.138:3300/0 conn(0x7fa51013f370 0x7fa51015e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.958+0000 7fa517b4c700 1 -- 172.21.15.112:0/1146734282 >> 172.21.15.112:0/1146734282 conn(0x7fa5100cc240 msgr2=0x7fa5100cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:12.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.958+0000 7fa517b4c700 1 -- 172.21.15.112:0/1146734282 shutdown_connections 2023-08-27T11:19:12.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.958+0000 7fa517b4c700 1 -- 172.21.15.112:0/1146734282 wait complete. 2023-08-27T11:19:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.959+0000 7fa517b4c700 1 Processor -- start 2023-08-27T11:19:12.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.959+0000 7fa517b4c700 1 -- start start 2023-08-27T11:19:12.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa517b4c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa51006e4c0 0x7fa510159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa517b4c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51013f370 0x7fa51014f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa517b4c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa51015a600 0x7fa51015cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa517b4c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100ce2a0 con 0x7fa51013f370 2023-08-27T11:19:12.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa517b4c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51015c770 con 0x7fa51015a600 2023-08-27T11:19:12.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa517b4c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51015bba0 con 0x7fa51006e4c0 2023-08-27T11:19:12.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5150e7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51013f370 0x7fa51014f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5150e7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51013f370 0x7fa51014f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58836/0 (socket says 172.21.15.112:58836) 2023-08-27T11:19:12.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5150e7700 1 -- 172.21.15.112:0/2767290948 learned_addr learned my addr 172.21.15.112:0/2767290948 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:12.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5160e9700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa51015a600 0x7fa51015cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5158e8700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa51006e4c0 0x7fa510159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5150e7700 1 -- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa51006e4c0 msgr2=0x7fa510159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5150e7700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa51006e4c0 0x7fa510159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5150e7700 1 -- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa51015a600 msgr2=0x7fa51015cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:12.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5150e7700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa51015a600 0x7fa51015cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:12.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5150e7700 1 -- 172.21.15.112:0/2767290948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa508009a70 con 0x7fa51013f370 2023-08-27T11:19:12.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5158e8700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa51006e4c0 0x7fa510159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:12.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5150e7700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51013f370 0x7fa51014f4a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c550 tx=0x7fa50c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:12.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5067fc700 1 -- 172.21.15.112:0/2767290948 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c004c80 con 0x7fa51013f370 2023-08-27T11:19:12.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5067fc700 1 -- 172.21.15.112:0/2767290948 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50c004e00 con 0x7fa51013f370 2023-08-27T11:19:12.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa517b4c700 1 -- 172.21.15.112:0/2767290948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51015be40 con 0x7fa51013f370 2023-08-27T11:19:12.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.960+0000 7fa5067fc700 1 -- 172.21.15.112:0/2767290948 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c007310 con 0x7fa51013f370 2023-08-27T11:19:12.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.961+0000 7fa517b4c700 1 -- 172.21.15.112:0/2767290948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51015c400 con 0x7fa51013f370 2023-08-27T11:19:12.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.962+0000 7fa517b4c700 1 -- 172.21.15.112:0/2767290948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa51015c900 con 0x7fa51013f370 2023-08-27T11:19:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.962+0000 7fa5067fc700 1 -- 172.21.15.112:0/2767290948 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fa50c007470 con 0x7fa51013f370 2023-08-27T11:19:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.962+0000 7fa5067fc700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa4f406a170 0x7fa4f406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.962+0000 7fa5067fc700 1 -- 172.21.15.112:0/2767290948 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa50c087ea0 con 0x7fa51013f370 2023-08-27T11:19:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.962+0000 7fa5158e8700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa4f406a170 0x7fa4f406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.963+0000 7fa5158e8700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa4f406a170 0x7fa4f406c630 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa5000099f0 tx=0x7fa500008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:12.966+0000 7fa5067fc700 1 -- 172.21.15.112:0/2767290948 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa50c0538f0 con 0x7fa51013f370 2023-08-27T11:19:13.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.153+0000 7fa517b4c700 1 -- 172.21.15.112:0/2767290948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa510067e90 con 0x7fa51013f370 2023-08-27T11:19:13.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.153+0000 7fa5067fc700 1 -- 172.21.15.112:0/2767290948 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa50c056f10 con 0x7fa51013f370 2023-08-27T11:19:13.155 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:19:13.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.155+0000 7fa517b4c700 1 -- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa4f406a170 msgr2=0x7fa4f406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.155+0000 7fa517b4c700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa4f406a170 0x7fa4f406c630 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa5000099f0 tx=0x7fa500008040 comp rx=0 tx=0).stop 2023-08-27T11:19:13.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.155+0000 7fa517b4c700 1 -- 172.21.15.112:0/2767290948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51013f370 msgr2=0x7fa51014f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.155+0000 7fa517b4c700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51013f370 0x7fa51014f4a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c550 tx=0x7fa50c00c860 comp rx=0 tx=0).stop 2023-08-27T11:19:13.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.155+0000 7fa517b4c700 1 -- 172.21.15.112:0/2767290948 shutdown_connections 2023-08-27T11:19:13.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.155+0000 7fa517b4c700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa4f406a170 0x7fa4f406c630 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.155+0000 7fa517b4c700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa51015a600 0x7fa51015cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.155+0000 7fa517b4c700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa51006e4c0 0x7fa510159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.155+0000 7fa517b4c700 1 --2- 172.21.15.112:0/2767290948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51013f370 0x7fa51014f4a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.155+0000 7fa517b4c700 1 -- 172.21.15.112:0/2767290948 >> 172.21.15.112:0/2767290948 conn(0x7fa5100cc240 msgr2=0x7fa51013d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:13.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.155+0000 7fa517b4c700 1 -- 172.21.15.112:0/2767290948 shutdown_connections 2023-08-27T11:19:13.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.155+0000 7fa517b4c700 1 -- 172.21.15.112:0/2767290948 wait complete. 2023-08-27T11:19:13.168 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:19:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.290+0000 7feb1252b700 1 Processor -- start 2023-08-27T11:19:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.290+0000 7feb1252b700 1 -- start start 2023-08-27T11:19:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.290+0000 7feb1252b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feb0c146c20 0x7feb0c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.290+0000 7feb1252b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb0c06f790 0x7feb0c06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.290+0000 7feb1252b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c0b5070 con 0x7feb0c0b3440 2023-08-27T11:19:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.290+0000 7feb1252b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c070180 con 0x7feb0c146c20 2023-08-27T11:19:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.290+0000 7feb1252b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c147000 con 0x7feb0c06f790 2023-08-27T11:19:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.291+0000 7feb0bfff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feb0c146c20 0x7feb0c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.291+0000 7feb10ac8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb0c06f790 0x7feb0c06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.291+0000 7feb0b7fe700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7feb0c0b3440 0x7feb0c15e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:60748/0 (socket says 172.21.15.112:60748) 2023-08-27T11:19:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.291+0000 7feb0b7fe700 1 -- 172.21.15.112:0/1626479745 learned_addr learned my addr 172.21.15.112:0/1626479745 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.291+0000 7feb0bfff700 1 -- 172.21.15.112:0/1626479745 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb0c06f790 msgr2=0x7feb0c06fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.291+0000 7feb0bfff700 1 --2- 172.21.15.112:0/1626479745 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb0c06f790 0x7feb0c06fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.291+0000 7feb0bfff700 1 -- 172.21.15.112:0/1626479745 >> v1:172.21.15.138:6789/0 conn(0x7feb0c0b3440 legacy=0x7feb0c15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.291+0000 7feb0bfff700 1 -- 172.21.15.112:0/1626479745 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c1614a0 con 0x7feb0c146c20 2023-08-27T11:19:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.291+0000 7feb0bfff700 1 --2- 172.21.15.112:0/1626479745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb0c146c20 0x7feb0c160f60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feafc009840 tx=0x7feafc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=440e39597376acc7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:13.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.291+0000 7feb0affd700 1 -- 172.21.15.112:0/1626479745 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc011830 con 0x7feb0c146c20 2023-08-27T11:19:13.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.292+0000 7feb0affd700 1 -- 172.21.15.112:0/1626479745 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc011990 con 0x7feb0c146c20 2023-08-27T11:19:13.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.292+0000 7feb0affd700 1 -- 172.21.15.112:0/1626479745 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc011d20 con 0x7feb0c146c20 2023-08-27T11:19:13.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.292+0000 7feb1252b700 1 -- 172.21.15.112:0/1626479745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb0c146c20 msgr2=0x7feb0c160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.292+0000 7feb1252b700 1 --2- 172.21.15.112:0/1626479745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb0c146c20 0x7feb0c160f60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feafc009840 tx=0x7feafc009b50 comp rx=0 tx=0).stop 2023-08-27T11:19:13.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.292+0000 7feb1252b700 1 -- 172.21.15.112:0/1626479745 shutdown_connections 2023-08-27T11:19:13.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.292+0000 7feb1252b700 1 --2- 172.21.15.112:0/1626479745 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb0c06f790 0x7feb0c06fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.292+0000 7feb1252b700 1 --2- 172.21.15.112:0/1626479745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb0c146c20 0x7feb0c160f60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.292+0000 7feb1252b700 1 -- 172.21.15.112:0/1626479745 >> 172.21.15.112:0/1626479745 conn(0x7feb0c0b1e90 msgr2=0x7feb0c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:13.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.292+0000 7feb1252b700 1 -- 172.21.15.112:0/1626479745 shutdown_connections 2023-08-27T11:19:13.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.292+0000 7feb1252b700 1 -- 172.21.15.112:0/1626479745 wait complete. 2023-08-27T11:19:13.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.293+0000 7feb1252b700 1 Processor -- start 2023-08-27T11:19:13.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.294+0000 7feb1252b700 1 -- start start 2023-08-27T11:19:13.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.294+0000 7feb1252b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb0c06f790 0x7feb0c15cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.294+0000 7feb1252b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb0c0b3440 0x7feb0c131fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.294+0000 7feb1252b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb0c146c20 0x7feb0c132510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.294+0000 7feb1252b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0c57e0 con 0x7feb0c146c20 2023-08-27T11:19:13.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.294+0000 7feb1252b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c15d050 con 0x7feb0c06f790 2023-08-27T11:19:13.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.294+0000 7feb1252b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c143f50 con 0x7feb0c0b3440 2023-08-27T11:19:13.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.294+0000 7feb10ac8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb0c146c20 0x7feb0c132510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.294+0000 7feb10ac8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb0c146c20 0x7feb0c132510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58852/0 (socket says 172.21.15.112:58852) 2023-08-27T11:19:13.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.294+0000 7feb10ac8700 1 -- 172.21.15.112:0/3167931862 learned_addr learned my addr 172.21.15.112:0/3167931862 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:13.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb0bfff700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb0c06f790 0x7feb0c15cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb0b7fe700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb0c0b3440 0x7feb0c131fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb10ac8700 1 -- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb0c0b3440 msgr2=0x7feb0c131fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb10ac8700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb0c0b3440 0x7feb0c131fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb10ac8700 1 -- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb0c06f790 msgr2=0x7feb0c15cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb10ac8700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb0c06f790 0x7feb0c15cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb10ac8700 1 -- 172.21.15.112:0/3167931862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc0094a0 con 0x7feb0c146c20 2023-08-27T11:19:13.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb0b7fe700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb0c0b3440 0x7feb0c131fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:13.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb10ac8700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb0c146c20 0x7feb0c132510 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7feb0400efc0 tx=0x7feb0400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:13.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb08ff9700 1 -- 172.21.15.112:0/3167931862 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0400cb80 con 0x7feb0c146c20 2023-08-27T11:19:13.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb08ff9700 1 -- 172.21.15.112:0/3167931862 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0400f040 con 0x7feb0c146c20 2023-08-27T11:19:13.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb1252b700 1 -- 172.21.15.112:0/3167931862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c0c5190 con 0x7feb0c146c20 2023-08-27T11:19:13.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb08ff9700 1 -- 172.21.15.112:0/3167931862 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04014480 con 0x7feb0c146c20 2023-08-27T11:19:13.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.295+0000 7feb1252b700 1 -- 172.21.15.112:0/3167931862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c144230 con 0x7feb0c146c20 2023-08-27T11:19:13.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.296+0000 7feb1252b700 1 -- 172.21.15.112:0/3167931862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c067e90 con 0x7feb0c146c20 2023-08-27T11:19:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.296+0000 7feb08ff9700 1 -- 172.21.15.112:0/3167931862 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7feb040145e0 con 0x7feb0c146c20 2023-08-27T11:19:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.297+0000 7feb08ff9700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaf806a170 0x7feaf806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.297+0000 7feb08ff9700 1 -- 172.21.15.112:0/3167931862 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5210+0+0 (secure 0 0 0) 0x7feb04088bb0 con 0x7feb0c146c20 2023-08-27T11:19:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.297+0000 7feb0bfff700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaf806a170 0x7feaf806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.297+0000 7feb0bfff700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaf806a170 0x7feaf806c630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feafc000c00 tx=0x7feafc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.301+0000 7feb08ff9700 1 -- 172.21.15.112:0/3167931862 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7feb0408f080 con 0x7feb0c146c20 2023-08-27T11:19:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.487+0000 7feb1252b700 1 -- 172.21.15.112:0/3167931862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb0c067a20 con 0x7feb0c146c20 2023-08-27T11:19:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.487+0000 7feb08ff9700 1 -- 172.21.15.112:0/3167931862 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb040180a0 con 0x7feb0c146c20 2023-08-27T11:19:13.489 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:19:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.489+0000 7feb1252b700 1 -- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaf806a170 msgr2=0x7feaf806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.489+0000 7feb1252b700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaf806a170 0x7feaf806c630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feafc000c00 tx=0x7feafc008030 comp rx=0 tx=0).stop 2023-08-27T11:19:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.489+0000 7feb1252b700 1 -- 172.21.15.112:0/3167931862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb0c146c20 msgr2=0x7feb0c132510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.489+0000 7feb1252b700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb0c146c20 0x7feb0c132510 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7feb0400efc0 tx=0x7feb0400c0c0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.489+0000 7feb1252b700 1 -- 172.21.15.112:0/3167931862 shutdown_connections 2023-08-27T11:19:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.489+0000 7feb1252b700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaf806a170 0x7feaf806c630 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.489+0000 7feb1252b700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb0c146c20 0x7feb0c132510 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.489+0000 7feb1252b700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb0c06f790 0x7feb0c15cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.489+0000 7feb1252b700 1 --2- 172.21.15.112:0/3167931862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb0c0b3440 0x7feb0c131fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.489+0000 7feb1252b700 1 -- 172.21.15.112:0/3167931862 >> 172.21.15.112:0/3167931862 conn(0x7feb0c0b1e90 msgr2=0x7feb0c072010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:13.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.489+0000 7feb1252b700 1 -- 172.21.15.112:0/3167931862 shutdown_connections 2023-08-27T11:19:13.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.489+0000 7feb1252b700 1 -- 172.21.15.112:0/3167931862 wait complete. 2023-08-27T11:19:13.503 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:19:13.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.621+0000 7f804b3a9700 1 Processor -- start 2023-08-27T11:19:13.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.621+0000 7f804b3a9700 1 -- start start 2023-08-27T11:19:13.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f804b3a9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8044151060 0x7f804415e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f804b3a9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f804406b4c0 0x7f80440cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f804b3a9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80440b02f0 con 0x7f804415e990 2023-08-27T11:19:13.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f804b3a9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80440b05d0 con 0x7f804406b4c0 2023-08-27T11:19:13.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f804b3a9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80440b08b0 con 0x7f8044151060 2023-08-27T11:19:13.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f8049946700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f804406b4c0 0x7f80440cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f8048944700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f804415e990 0x7f804406adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53240/0 (socket says 172.21.15.112:53240) 2023-08-27T11:19:13.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f8048944700 1 -- 172.21.15.112:0/3808399994 learned_addr learned my addr 172.21.15.112:0/3808399994 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:13.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f8049145700 1 --2- 172.21.15.112:0/3808399994 >> v2:172.21.15.138:3300/0 conn(0x7f8044151060 0x7f804415e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f8049946700 1 -- 172.21.15.112:0/3808399994 >> v2:172.21.15.138:3300/0 conn(0x7f8044151060 msgr2=0x7f804415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:13.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f8049946700 1 --2- 172.21.15.112:0/3808399994 >> v2:172.21.15.138:3300/0 conn(0x7f8044151060 0x7f804415e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f8049946700 1 -- 172.21.15.112:0/3808399994 >> v1:172.21.15.112:6789/0 conn(0x7f804415e990 legacy=0x7f804406adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f8049946700 1 -- 172.21.15.112:0/3808399994 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440b0b90 con 0x7f804406b4c0 2023-08-27T11:19:13.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f803bfff700 1 -- 172.21.15.112:0/3808399994 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2877953243 0 0) 0x7f80440b02f0 con 0x7f804415e990 2023-08-27T11:19:13.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.622+0000 7f8049145700 1 --2- 172.21.15.112:0/3808399994 >> v2:172.21.15.138:3300/0 conn(0x7f8044151060 0x7f804415e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:13.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.623+0000 7f8049946700 1 --2- 172.21.15.112:0/3808399994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804406b4c0 0x7f80440cdbd0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f804406e8c0 tx=0x7f803c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f1f433a9e9433f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:13.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.623+0000 7f803bfff700 1 -- 172.21.15.112:0/3808399994 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c00e070 con 0x7f804406b4c0 2023-08-27T11:19:13.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.623+0000 7f803bfff700 1 -- 172.21.15.112:0/3808399994 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c00bbd0 con 0x7f804406b4c0 2023-08-27T11:19:13.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.623+0000 7f803bfff700 1 -- 172.21.15.112:0/3808399994 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c012430 con 0x7f804406b4c0 2023-08-27T11:19:13.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.623+0000 7f804b3a9700 1 -- 172.21.15.112:0/3808399994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804406b4c0 msgr2=0x7f80440cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.623+0000 7f804b3a9700 1 --2- 172.21.15.112:0/3808399994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804406b4c0 0x7f80440cdbd0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f804406e8c0 tx=0x7f803c00b330 comp rx=0 tx=0).stop 2023-08-27T11:19:13.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.623+0000 7f804b3a9700 1 -- 172.21.15.112:0/3808399994 shutdown_connections 2023-08-27T11:19:13.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.623+0000 7f804b3a9700 1 --2- 172.21.15.112:0/3808399994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804406b4c0 0x7f80440cdbd0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.623+0000 7f804b3a9700 1 --2- 172.21.15.112:0/3808399994 >> v2:172.21.15.138:3300/0 conn(0x7f8044151060 0x7f804415e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.623+0000 7f804b3a9700 1 -- 172.21.15.112:0/3808399994 >> 172.21.15.112:0/3808399994 conn(0x7f80440d0000 msgr2=0x7f80440d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:13.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.623+0000 7f804b3a9700 1 -- 172.21.15.112:0/3808399994 shutdown_connections 2023-08-27T11:19:13.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.623+0000 7f804b3a9700 1 -- 172.21.15.112:0/3808399994 wait complete. 2023-08-27T11:19:13.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.624+0000 7f804b3a9700 1 Processor -- start 2023-08-27T11:19:13.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.625+0000 7f804b3a9700 1 -- start start 2023-08-27T11:19:13.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.625+0000 7f804b3a9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f804406b4c0 0x7f804414e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.625+0000 7f804b3a9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8044151060 0x7f80440c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.625+0000 7f804b3a9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804415e990 0x7f80440c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.625+0000 7f804b3a9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804415d890 con 0x7f804415e990 2023-08-27T11:19:13.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.625+0000 7f804b3a9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804415cd10 con 0x7f804406b4c0 2023-08-27T11:19:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.625+0000 7f804b3a9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804415cff0 con 0x7f8044151060 2023-08-27T11:19:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.625+0000 7f8049946700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804415e990 0x7f80440c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.625+0000 7f8049946700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804415e990 0x7f80440c39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58866/0 (socket says 172.21.15.112:58866) 2023-08-27T11:19:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.625+0000 7f8049946700 1 -- 172.21.15.112:0/959213827 learned_addr learned my addr 172.21.15.112:0/959213827 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.626+0000 7f8049946700 1 -- 172.21.15.112:0/959213827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8044151060 msgr2=0x7f80440c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.626+0000 7f8049946700 1 --2- 172.21.15.112:0/959213827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8044151060 0x7f80440c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.626+0000 7f8049946700 1 -- 172.21.15.112:0/959213827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f804406b4c0 msgr2=0x7f804414e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.626+0000 7f8049946700 1 --2- 172.21.15.112:0/959213827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f804406b4c0 0x7f804414e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.626+0000 7f8049946700 1 -- 172.21.15.112:0/959213827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c009a70 con 0x7f804415e990 2023-08-27T11:19:13.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.626+0000 7f8049946700 1 --2- 172.21.15.112:0/959213827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804415e990 0x7f80440c39f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f803c00b370 tx=0x7f803c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:13.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.626+0000 7f8039ffb700 1 -- 172.21.15.112:0/959213827 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c012430 con 0x7f804415e990 2023-08-27T11:19:13.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.626+0000 7f8039ffb700 1 -- 172.21.15.112:0/959213827 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c020070 con 0x7f804415e990 2023-08-27T11:19:13.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.626+0000 7f804b3a9700 1 -- 172.21.15.112:0/959213827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804415d2d0 con 0x7f804415e990 2023-08-27T11:19:13.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.626+0000 7f8039ffb700 1 -- 172.21.15.112:0/959213827 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c004280 con 0x7f804415e990 2023-08-27T11:19:13.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.626+0000 7f804b3a9700 1 -- 172.21.15.112:0/959213827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804414e9a0 con 0x7f804415e990 2023-08-27T11:19:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.627+0000 7f804b3a9700 1 -- 172.21.15.112:0/959213827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8044067a20 con 0x7f804415e990 2023-08-27T11:19:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.627+0000 7f8039ffb700 1 -- 172.21.15.112:0/959213827 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f803c0043e0 con 0x7f804415e990 2023-08-27T11:19:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.628+0000 7f8039ffb700 1 --2- 172.21.15.112:0/959213827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f802806a170 0x7f802806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.628+0000 7f8039ffb700 1 -- 172.21.15.112:0/959213827 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f803c022070 con 0x7f804415e990 2023-08-27T11:19:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.628+0000 7f8049145700 1 --2- 172.21.15.112:0/959213827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f802806a170 0x7f802806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.628+0000 7f8049145700 1 --2- 172.21.15.112:0/959213827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f802806a170 0x7f802806c630 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8034007920 tx=0x7f8034008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.632+0000 7f8039ffb700 1 -- 172.21.15.112:0/959213827 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f803c055f90 con 0x7f804415e990 2023-08-27T11:19:13.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.823+0000 7f804b3a9700 1 -- 172.21.15.112:0/959213827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f80440cc4b0 con 0x7f804415e990 2023-08-27T11:19:13.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.823+0000 7f8039ffb700 1 -- 172.21.15.112:0/959213827 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+126 (secure 0 0 0) 0x7f803c01ae60 con 0x7f804415e990 2023-08-27T11:19:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:13.825 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:19:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2023-08-27T11:19:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "rbd" 2023-08-27T11:19:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:19:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 2, 2023-08-27T11:19:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2023-08-27T11:19:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:13.827 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:19:13.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.825+0000 7f804b3a9700 1 -- 172.21.15.112:0/959213827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f802806a170 msgr2=0x7f802806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.825+0000 7f804b3a9700 1 --2- 172.21.15.112:0/959213827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f802806a170 0x7f802806c630 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8034007920 tx=0x7f8034008040 comp rx=0 tx=0).stop 2023-08-27T11:19:13.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.825+0000 7f804b3a9700 1 -- 172.21.15.112:0/959213827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804415e990 msgr2=0x7f80440c39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.825+0000 7f804b3a9700 1 --2- 172.21.15.112:0/959213827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804415e990 0x7f80440c39f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f803c00b370 tx=0x7f803c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.825+0000 7f804b3a9700 1 -- 172.21.15.112:0/959213827 shutdown_connections 2023-08-27T11:19:13.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.825+0000 7f804b3a9700 1 --2- 172.21.15.112:0/959213827 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f802806a170 0x7f802806c630 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.825+0000 7f804b3a9700 1 --2- 172.21.15.112:0/959213827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804415e990 0x7f80440c39f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.825+0000 7f804b3a9700 1 --2- 172.21.15.112:0/959213827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f804406b4c0 0x7f804414e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.825+0000 7f804b3a9700 1 --2- 172.21.15.112:0/959213827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8044151060 0x7f80440c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.825+0000 7f804b3a9700 1 -- 172.21.15.112:0/959213827 >> 172.21.15.112:0/959213827 conn(0x7f80440d0000 msgr2=0x7f8044073380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:13.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.825+0000 7f804b3a9700 1 -- 172.21.15.112:0/959213827 shutdown_connections 2023-08-27T11:19:13.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.825+0000 7f804b3a9700 1 -- 172.21.15.112:0/959213827 wait complete. 2023-08-27T11:19:13.838 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:19:13.839 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:19:13.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.962+0000 7f32a44fd700 1 Processor -- start 2023-08-27T11:19:13.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.962+0000 7f32a44fd700 1 -- start start 2023-08-27T11:19:13.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a44fd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f329c157a70 0x7f329c15da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a44fd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f329c0b3d50 0x7f329c161860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a44fd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f329c161da0 0x7f329c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a44fd700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329c0b1a80 con 0x7f329c0b3d50 2023-08-27T11:19:13.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a44fd700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329c1613d0 con 0x7f329c161da0 2023-08-27T11:19:13.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a44fd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c15e310 con 0x7f329c157a70 2023-08-27T11:19:13.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a1a98700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f329c0b3d50 0x7f329c161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a1a98700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f329c0b3d50 0x7f329c161860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58870/0 (socket says 172.21.15.112:58870) 2023-08-27T11:19:13.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a1a98700 1 -- 172.21.15.112:0/291191189 learned_addr learned my addr 172.21.15.112:0/291191189 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:13.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a2a9a700 1 --2- 172.21.15.112:0/291191189 >> v2:172.21.15.138:3300/0 conn(0x7f329c161da0 0x7f329c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a2299700 1 --2- 172.21.15.112:0/291191189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f329c157a70 0x7f329c15da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a1a98700 1 -- 172.21.15.112:0/291191189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f329c157a70 msgr2=0x7f329c15da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:13.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a1a98700 1 --2- 172.21.15.112:0/291191189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f329c157a70 0x7f329c15da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a1a98700 1 -- 172.21.15.112:0/291191189 >> v2:172.21.15.138:3300/0 conn(0x7f329c161da0 msgr2=0x7f329c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:13.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a1a98700 1 --2- 172.21.15.112:0/291191189 >> v2:172.21.15.138:3300/0 conn(0x7f329c161da0 0x7f329c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a1a98700 1 -- 172.21.15.112:0/291191189 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c15e5f0 con 0x7f329c0b3d50 2023-08-27T11:19:13.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a2a9a700 1 --2- 172.21.15.112:0/291191189 >> v2:172.21.15.138:3300/0 conn(0x7f329c161da0 0x7f329c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:13.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a2299700 1 --2- 172.21.15.112:0/291191189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f329c157a70 0x7f329c15da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:13.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.963+0000 7f32a1a98700 1 --2- 172.21.15.112:0/291191189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c0b3d50 0x7f329c161860 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f329c0afdc0 tx=0x7f3298009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c4fc5116111cb8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:13.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.964+0000 7f32a1297700 1 -- 172.21.15.112:0/291191189 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329800b870 con 0x7f329c0b3d50 2023-08-27T11:19:13.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.964+0000 7f32a1297700 1 -- 172.21.15.112:0/291191189 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329800b9d0 con 0x7f329c0b3d50 2023-08-27T11:19:13.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.964+0000 7f32a1297700 1 -- 172.21.15.112:0/291191189 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329800bd60 con 0x7f329c0b3d50 2023-08-27T11:19:13.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.964+0000 7f32a44fd700 1 -- 172.21.15.112:0/291191189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c0b3d50 msgr2=0x7f329c161860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.964+0000 7f32a44fd700 1 --2- 172.21.15.112:0/291191189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c0b3d50 0x7f329c161860 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f329c0afdc0 tx=0x7f3298009bd0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.964+0000 7f32a44fd700 1 -- 172.21.15.112:0/291191189 shutdown_connections 2023-08-27T11:19:13.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.964+0000 7f32a44fd700 1 --2- 172.21.15.112:0/291191189 >> v2:172.21.15.138:3300/0 conn(0x7f329c161da0 0x7f329c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.964+0000 7f32a44fd700 1 --2- 172.21.15.112:0/291191189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f329c157a70 0x7f329c15da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.964+0000 7f32a44fd700 1 --2- 172.21.15.112:0/291191189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c0b3d50 0x7f329c161860 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.964+0000 7f32a44fd700 1 -- 172.21.15.112:0/291191189 >> 172.21.15.112:0/291191189 conn(0x7f329c0b1be0 msgr2=0x7f329c0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:13.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.964+0000 7f32a44fd700 1 -- 172.21.15.112:0/291191189 shutdown_connections 2023-08-27T11:19:13.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.964+0000 7f32a44fd700 1 -- 172.21.15.112:0/291191189 wait complete. 2023-08-27T11:19:13.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.965+0000 7f32a44fd700 1 Processor -- start 2023-08-27T11:19:13.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.965+0000 7f32a44fd700 1 -- start start 2023-08-27T11:19:13.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.966+0000 7f32a44fd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c0b3d50 0x7f329c06f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.966+0000 7f32a44fd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c157a70 0x7f329c06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.966+0000 7f32a44fd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f329c161da0 0x7f329c146ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.966+0000 7f32a44fd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c157e50 con 0x7f329c157a70 2023-08-27T11:19:13.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.966+0000 7f32a44fd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c147ba0 con 0x7f329c0b3d50 2023-08-27T11:19:13.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.966+0000 7f32a44fd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c147e80 con 0x7f329c161da0 2023-08-27T11:19:13.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.966+0000 7f32a1a98700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c157a70 0x7f329c06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.966+0000 7f32a1a98700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c157a70 0x7f329c06fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58878/0 (socket says 172.21.15.112:58878) 2023-08-27T11:19:13.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.966+0000 7f32a1a98700 1 -- 172.21.15.112:0/3112299091 learned_addr learned my addr 172.21.15.112:0/3112299091 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:13.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f32a2a9a700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f329c161da0 0x7f329c146ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f32a2299700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c0b3d50 0x7f329c06f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f32a1a98700 1 -- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f329c161da0 msgr2=0x7f329c146ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f32a1a98700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f329c161da0 0x7f329c146ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f32a1a98700 1 -- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c0b3d50 msgr2=0x7f329c06f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:13.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f32a1a98700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c0b3d50 0x7f329c06f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:13.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f32a1a98700 1 -- 172.21.15.112:0/3112299091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3298009550 con 0x7f329c157a70 2023-08-27T11:19:13.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f32a2299700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c0b3d50 0x7f329c06f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:13.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f32a1a98700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c157a70 0x7f329c06fb40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3298014040 tx=0x7f3298004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:13.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f3292ffd700 1 -- 172.21.15.112:0/3112299091 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3298004520 con 0x7f329c157a70 2023-08-27T11:19:13.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f3292ffd700 1 -- 172.21.15.112:0/3112299091 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329801e070 con 0x7f329c157a70 2023-08-27T11:19:13.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f32a44fd700 1 -- 172.21.15.112:0/3112299091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c148160 con 0x7f329c157a70 2023-08-27T11:19:13.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f3292ffd700 1 -- 172.21.15.112:0/3112299091 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32980038e0 con 0x7f329c157a70 2023-08-27T11:19:13.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.967+0000 7f32a44fd700 1 -- 172.21.15.112:0/3112299091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c159a20 con 0x7f329c157a70 2023-08-27T11:19:13.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.968+0000 7f32a44fd700 1 -- 172.21.15.112:0/3112299091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c067a20 con 0x7f329c157a70 2023-08-27T11:19:13.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.968+0000 7f3292ffd700 1 -- 172.21.15.112:0/3112299091 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f3298003a40 con 0x7f329c157a70 2023-08-27T11:19:13.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.969+0000 7f3292ffd700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f328006a170 0x7f328006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:13.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.969+0000 7f3292ffd700 1 -- 172.21.15.112:0/3112299091 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3298020070 con 0x7f329c157a70 2023-08-27T11:19:13.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.969+0000 7f32a2299700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f328006a170 0x7f328006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:13.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.969+0000 7f32a2299700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f328006a170 0x7f328006c630 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f328c008940 tx=0x7f328c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:13.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:13.973+0000 7f3292ffd700 1 -- 172.21.15.112:0/3112299091 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3298055380 con 0x7f329c157a70 2023-08-27T11:19:14.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.160+0000 7f32a44fd700 1 -- 172.21.15.112:0/3112299091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f329c0afdf0 con 0x7f329c157a70 2023-08-27T11:19:14.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.657+0000 7f3292ffd700 1 -- 172.21.15.112:0/3112299091 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v62) v1 ==== 138+0+0 (secure 0 0 0) 0x7f32980150b0 con 0x7f329c157a70 2023-08-27T11:19:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.660+0000 7f32a44fd700 1 -- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f328006a170 msgr2=0x7f328006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.660+0000 7f32a44fd700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f328006a170 0x7f328006c630 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f328c008940 tx=0x7f328c008040 comp rx=0 tx=0).stop 2023-08-27T11:19:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.660+0000 7f32a44fd700 1 -- 172.21.15.112:0/3112299091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c157a70 msgr2=0x7f329c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.660+0000 7f32a44fd700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c157a70 0x7f329c06fb40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3298014040 tx=0x7f3298004750 comp rx=0 tx=0).stop 2023-08-27T11:19:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.660+0000 7f32a44fd700 1 -- 172.21.15.112:0/3112299091 shutdown_connections 2023-08-27T11:19:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.660+0000 7f32a44fd700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f328006a170 0x7f328006c630 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.660+0000 7f32a44fd700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f329c161da0 0x7f329c146ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.660+0000 7f32a44fd700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c0b3d50 0x7f329c06f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.660+0000 7f32a44fd700 1 --2- 172.21.15.112:0/3112299091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c157a70 0x7f329c06fb40 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.660+0000 7f32a44fd700 1 -- 172.21.15.112:0/3112299091 >> 172.21.15.112:0/3112299091 conn(0x7f329c0b1be0 msgr2=0x7f329c071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.660+0000 7f32a44fd700 1 -- 172.21.15.112:0/3112299091 shutdown_connections 2023-08-27T11:19:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.660+0000 7f32a44fd700 1 -- 172.21.15.112:0/3112299091 wait complete. 2023-08-27T11:19:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2023-08-27T11:19:14.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:19:14.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.803+0000 7f9a43eaa700 1 Processor -- start 2023-08-27T11:19:14.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.804+0000 7f9a43eaa700 1 -- start start 2023-08-27T11:19:14.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.804+0000 7f9a43eaa700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9a3c146c20 0x7f9a3c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.804+0000 7f9a43eaa700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9a3c06f790 0x7f9a3c06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.804+0000 7f9a43eaa700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3c0b5070 con 0x7f9a3c0b3440 2023-08-27T11:19:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.804+0000 7f9a43eaa700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3c070180 con 0x7f9a3c06f790 2023-08-27T11:19:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.804+0000 7f9a43eaa700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3c147000 con 0x7f9a3c146c20 2023-08-27T11:19:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a42447700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9a3c06f790 0x7f9a3c06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a41445700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9a3c0b3440 0x7f9a3c15e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53266/0 (socket says 172.21.15.112:53266) 2023-08-27T11:19:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a41445700 1 -- 172.21.15.112:0/2505267419 learned_addr learned my addr 172.21.15.112:0/2505267419 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a41c46700 1 --2- 172.21.15.112:0/2505267419 >> v2:172.21.15.138:3300/0 conn(0x7f9a3c146c20 0x7f9a3c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a42447700 1 -- 172.21.15.112:0/2505267419 >> v2:172.21.15.138:3300/0 conn(0x7f9a3c146c20 msgr2=0x7f9a3c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:14.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a42447700 1 --2- 172.21.15.112:0/2505267419 >> v2:172.21.15.138:3300/0 conn(0x7f9a3c146c20 0x7f9a3c160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:14.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a42447700 1 -- 172.21.15.112:0/2505267419 >> v1:172.21.15.112:6789/0 conn(0x7f9a3c0b3440 legacy=0x7f9a3c15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:14.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a40c44700 1 -- 172.21.15.112:0/2505267419 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618368446 0 0) 0x7f9a38001110 con 0x7f9a3c0b3440 2023-08-27T11:19:14.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a42447700 1 -- 172.21.15.112:0/2505267419 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c1614a0 con 0x7f9a3c06f790 2023-08-27T11:19:14.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a41c46700 1 --2- 172.21.15.112:0/2505267419 >> v2:172.21.15.138:3300/0 conn(0x7f9a3c146c20 0x7f9a3c160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a42447700 1 --2- 172.21.15.112:0/2505267419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c06f790 0x7f9a3c06fc40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0b2d00 tx=0x7f9a3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfc879bb0662095 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a40c44700 1 -- 172.21.15.112:0/2505267419 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3400e070 con 0x7f9a3c06f790 2023-08-27T11:19:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a40c44700 1 -- 172.21.15.112:0/2505267419 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3400bbd0 con 0x7f9a3c06f790 2023-08-27T11:19:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.805+0000 7f9a40c44700 1 -- 172.21.15.112:0/2505267419 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34012430 con 0x7f9a3c06f790 2023-08-27T11:19:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.806+0000 7f9a43eaa700 1 -- 172.21.15.112:0/2505267419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c06f790 msgr2=0x7f9a3c06fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.806+0000 7f9a43eaa700 1 --2- 172.21.15.112:0/2505267419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c06f790 0x7f9a3c06fc40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0b2d00 tx=0x7f9a3400b330 comp rx=0 tx=0).stop 2023-08-27T11:19:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.806+0000 7f9a43eaa700 1 -- 172.21.15.112:0/2505267419 shutdown_connections 2023-08-27T11:19:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.806+0000 7f9a43eaa700 1 --2- 172.21.15.112:0/2505267419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c06f790 0x7f9a3c06fc40 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.806+0000 7f9a43eaa700 1 --2- 172.21.15.112:0/2505267419 >> v2:172.21.15.138:3300/0 conn(0x7f9a3c146c20 0x7f9a3c160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.806+0000 7f9a43eaa700 1 -- 172.21.15.112:0/2505267419 >> 172.21.15.112:0/2505267419 conn(0x7f9a3c0b1e90 msgr2=0x7f9a3c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.806+0000 7f9a43eaa700 1 -- 172.21.15.112:0/2505267419 shutdown_connections 2023-08-27T11:19:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.806+0000 7f9a43eaa700 1 -- 172.21.15.112:0/2505267419 wait complete. 2023-08-27T11:19:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.807+0000 7f9a43eaa700 1 Processor -- start 2023-08-27T11:19:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.807+0000 7f9a43eaa700 1 -- start start 2023-08-27T11:19:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a43eaa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c06f790 0x7f9a3c0c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a43eaa700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a3c0b3440 0x7f9a3c144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a43eaa700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a3c146c20 0x7f9a3c145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a43eaa700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c0cf940 con 0x7f9a3c06f790 2023-08-27T11:19:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a43eaa700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c1382b0 con 0x7f9a3c146c20 2023-08-27T11:19:14.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a43eaa700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c0c6ab0 con 0x7f9a3c0b3440 2023-08-27T11:19:14.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a41c46700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c06f790 0x7f9a3c0c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:14.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a41c46700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c06f790 0x7f9a3c0c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58892/0 (socket says 172.21.15.112:58892) 2023-08-27T11:19:14.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a41c46700 1 -- 172.21.15.112:0/1294378594 learned_addr learned my addr 172.21.15.112:0/1294378594 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:14.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a42447700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a3c146c20 0x7f9a3c145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:14.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a41445700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a3c0b3440 0x7f9a3c144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:14.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a41c46700 1 -- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a3c0b3440 msgr2=0x7f9a3c144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:14.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a41c46700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a3c0b3440 0x7f9a3c144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:14.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a41c46700 1 -- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a3c146c20 msgr2=0x7f9a3c145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:14.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a41c46700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a3c146c20 0x7f9a3c145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:14.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a41c46700 1 -- 172.21.15.112:0/1294378594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a34009a70 con 0x7f9a3c06f790 2023-08-27T11:19:14.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.808+0000 7f9a41c46700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c06f790 0x7f9a3c0c6120 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c7f0 tx=0x7f9a2c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:14.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.809+0000 7f9a327fc700 1 -- 172.21.15.112:0/1294378594 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c011800 con 0x7f9a3c06f790 2023-08-27T11:19:14.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.809+0000 7f9a327fc700 1 -- 172.21.15.112:0/1294378594 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c011960 con 0x7f9a3c06f790 2023-08-27T11:19:14.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.809+0000 7f9a327fc700 1 -- 172.21.15.112:0/1294378594 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c011d10 con 0x7f9a3c06f790 2023-08-27T11:19:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.809+0000 7f9a43eaa700 1 -- 172.21.15.112:0/1294378594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3c0c6660 con 0x7f9a3c06f790 2023-08-27T11:19:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.809+0000 7f9a43eaa700 1 -- 172.21.15.112:0/1294378594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c1f6530 con 0x7f9a3c06f790 2023-08-27T11:19:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.809+0000 7f9a43eaa700 1 -- 172.21.15.112:0/1294378594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3c067e90 con 0x7f9a3c06f790 2023-08-27T11:19:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.810+0000 7f9a327fc700 1 -- 172.21.15.112:0/1294378594 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f9a2c018360 con 0x7f9a3c06f790 2023-08-27T11:19:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.810+0000 7f9a327fc700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9a2006a170 0x7f9a2006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.811+0000 7f9a327fc700 1 -- 172.21.15.112:0/1294378594 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f9a2c088460 con 0x7f9a3c06f790 2023-08-27T11:19:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.811+0000 7f9a41445700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9a2006a170 0x7f9a2006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.811+0000 7f9a41445700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9a2006a170 0x7f9a2006c630 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9a38000f60 tx=0x7f9a38007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:14.815+0000 7f9a327fc700 1 -- 172.21.15.112:0/1294378594 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9a2c0576c0 con 0x7f9a3c06f790 2023-08-27T11:19:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.002+0000 7f9a43eaa700 1 -- 172.21.15.112:0/1294378594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9a3c06fe10 con 0x7f9a3c06f790 2023-08-27T11:19:15.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.665+0000 7f9a327fc700 1 -- 172.21.15.112:0/1294378594 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v63) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9a2c057250 con 0x7f9a3c06f790 2023-08-27T11:19:15.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.670+0000 7f9a43eaa700 1 -- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9a2006a170 msgr2=0x7f9a2006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:15.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.670+0000 7f9a43eaa700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9a2006a170 0x7f9a2006c630 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9a38000f60 tx=0x7f9a38007a40 comp rx=0 tx=0).stop 2023-08-27T11:19:15.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.670+0000 7f9a43eaa700 1 -- 172.21.15.112:0/1294378594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c06f790 msgr2=0x7f9a3c0c6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:15.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.670+0000 7f9a43eaa700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c06f790 0x7f9a3c0c6120 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c7f0 tx=0x7f9a2c00cbb0 comp rx=0 tx=0).stop 2023-08-27T11:19:15.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.670+0000 7f9a43eaa700 1 -- 172.21.15.112:0/1294378594 shutdown_connections 2023-08-27T11:19:15.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.670+0000 7f9a43eaa700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9a2006a170 0x7f9a2006c630 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:15.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.670+0000 7f9a43eaa700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a3c146c20 0x7f9a3c145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:15.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.670+0000 7f9a43eaa700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c06f790 0x7f9a3c0c6120 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:15.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.670+0000 7f9a43eaa700 1 --2- 172.21.15.112:0/1294378594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a3c0b3440 0x7f9a3c144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:15.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.670+0000 7f9a43eaa700 1 -- 172.21.15.112:0/1294378594 >> 172.21.15.112:0/1294378594 conn(0x7f9a3c0b1e90 msgr2=0x7f9a3c1438e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:15.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.671+0000 7f9a43eaa700 1 -- 172.21.15.112:0/1294378594 shutdown_connections 2023-08-27T11:19:15.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.671+0000 7f9a43eaa700 1 -- 172.21.15.112:0/1294378594 wait complete. 2023-08-27T11:19:15.677 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2023-08-27T11:19:15.693 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:19:15.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.816+0000 7fdaa1106700 1 Processor -- start 2023-08-27T11:19:15.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.817+0000 7fdaa1106700 1 -- start start 2023-08-27T11:19:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.817+0000 7fdaa1106700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fda9c1437c0 0x7fda9c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.817+0000 7fdaa1106700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fda9c06d5b0 0x7fda9c06da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.817+0000 7fdaa1106700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda9c0cfd50 con 0x7fda9c143ba0 2023-08-27T11:19:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.817+0000 7fdaa1106700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9c06df80 con 0x7fda9c06d5b0 2023-08-27T11:19:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.817+0000 7fdaa1106700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9c0cf140 con 0x7fda9c1437c0 2023-08-27T11:19:15.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.818+0000 7fda9b59e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fda9c06d5b0 0x7fda9c06da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:15.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.818+0000 7fda9b59e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fda9c06d5b0 0x7fda9c06da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58902/0 (socket says 172.21.15.112:58902) 2023-08-27T11:19:15.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.818+0000 7fda9b59e700 1 -- 172.21.15.112:0/1361869111 learned_addr learned my addr 172.21.15.112:0/1361869111 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:15.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.818+0000 7fda9ad9d700 1 --2- 172.21.15.112:0/1361869111 >> v2:172.21.15.138:3300/0 conn(0x7fda9c1437c0 0x7fda9c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:15.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.818+0000 7fda9b59e700 1 -- 172.21.15.112:0/1361869111 >> v2:172.21.15.138:3300/0 conn(0x7fda9c1437c0 msgr2=0x7fda9c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:15.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.818+0000 7fda9b59e700 1 --2- 172.21.15.112:0/1361869111 >> v2:172.21.15.138:3300/0 conn(0x7fda9c1437c0 0x7fda9c0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:15.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.818+0000 7fda9b59e700 1 -- 172.21.15.112:0/1361869111 >> v1:172.21.15.138:6789/0 conn(0x7fda9c143ba0 legacy=0x7fda9c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:15.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.818+0000 7fda9b59e700 1 -- 172.21.15.112:0/1361869111 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c0cf420 con 0x7fda9c06d5b0 2023-08-27T11:19:15.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.818+0000 7fda9b59e700 1 --2- 172.21.15.112:0/1361869111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c06d5b0 0x7fda9c06da40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fda9c06e3d0 tx=0x7fda9400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a3c241abbeaa7a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:15.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.818+0000 7fda99d9b700 1 -- 172.21.15.112:0/1361869111 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94010070 con 0x7fda9c06d5b0 2023-08-27T11:19:15.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.818+0000 7fda99d9b700 1 -- 172.21.15.112:0/1361869111 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9400dce0 con 0x7fda9c06d5b0 2023-08-27T11:19:15.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.818+0000 7fda99d9b700 1 -- 172.21.15.112:0/1361869111 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9400b310 con 0x7fda9c06d5b0 2023-08-27T11:19:15.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.819+0000 7fdaa1106700 1 -- 172.21.15.112:0/1361869111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c06d5b0 msgr2=0x7fda9c06da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:15.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.819+0000 7fdaa1106700 1 --2- 172.21.15.112:0/1361869111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c06d5b0 0x7fda9c06da40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fda9c06e3d0 tx=0x7fda9400d430 comp rx=0 tx=0).stop 2023-08-27T11:19:15.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.819+0000 7fdaa1106700 1 -- 172.21.15.112:0/1361869111 shutdown_connections 2023-08-27T11:19:15.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.819+0000 7fdaa1106700 1 --2- 172.21.15.112:0/1361869111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c06d5b0 0x7fda9c06da40 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:15.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.819+0000 7fdaa1106700 1 --2- 172.21.15.112:0/1361869111 >> v2:172.21.15.138:3300/0 conn(0x7fda9c1437c0 0x7fda9c0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:15.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.819+0000 7fdaa1106700 1 -- 172.21.15.112:0/1361869111 >> 172.21.15.112:0/1361869111 conn(0x7fda9c0b1e90 msgr2=0x7fda9c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:15.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.819+0000 7fdaa1106700 1 -- 172.21.15.112:0/1361869111 shutdown_connections 2023-08-27T11:19:15.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.819+0000 7fdaa1106700 1 -- 172.21.15.112:0/1361869111 wait complete. 2023-08-27T11:19:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.820+0000 7fdaa1106700 1 Processor -- start 2023-08-27T11:19:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.820+0000 7fdaa1106700 1 -- start start 2023-08-27T11:19:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fdaa1106700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c06d5b0 0x7fda9c13ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fdaa1106700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c1437c0 0x7fda9c14aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fdaa1106700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda9c143ba0 0x7fda9c13aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fdaa1106700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c13d720 con 0x7fda9c06d5b0 2023-08-27T11:19:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fdaa1106700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c139a40 con 0x7fda9c1437c0 2023-08-27T11:19:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fdaa1106700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c13d200 con 0x7fda9c143ba0 2023-08-27T11:19:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fda9ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c06d5b0 0x7fda9c13ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fda9ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c06d5b0 0x7fda9c13ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58904/0 (socket says 172.21.15.112:58904) 2023-08-27T11:19:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fda9ad9d700 1 -- 172.21.15.112:0/3921614304 learned_addr learned my addr 172.21.15.112:0/3921614304 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fda9b59e700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda9c143ba0 0x7fda9c13aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fda9a59c700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c1437c0 0x7fda9c14aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fda9ad9d700 1 -- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda9c143ba0 msgr2=0x7fda9c13aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fda9ad9d700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda9c143ba0 0x7fda9c13aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fda9ad9d700 1 -- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c1437c0 msgr2=0x7fda9c14aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fda9ad9d700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c1437c0 0x7fda9c14aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fda9ad9d700 1 -- 172.21.15.112:0/3921614304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9400bd40 con 0x7fda9c06d5b0 2023-08-27T11:19:15.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.821+0000 7fda9a59c700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c1437c0 0x7fda9c14aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:15.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.822+0000 7fda9ad9d700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c06d5b0 0x7fda9c13ccc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fda9000e9b0 tx=0x7fda9000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:15.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.822+0000 7fda837fe700 1 -- 172.21.15.112:0/3921614304 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90009710 con 0x7fda9c06d5b0 2023-08-27T11:19:15.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.822+0000 7fda837fe700 1 -- 172.21.15.112:0/3921614304 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9000c850 con 0x7fda9c06d5b0 2023-08-27T11:19:15.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.822+0000 7fdaa1106700 1 -- 172.21.15.112:0/3921614304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9c14afc0 con 0x7fda9c06d5b0 2023-08-27T11:19:15.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.822+0000 7fda837fe700 1 -- 172.21.15.112:0/3921614304 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9000cbc0 con 0x7fda9c06d5b0 2023-08-27T11:19:15.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.822+0000 7fdaa1106700 1 -- 172.21.15.112:0/3921614304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9c139110 con 0x7fda9c06d5b0 2023-08-27T11:19:15.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.823+0000 7fdaa1106700 1 -- 172.21.15.112:0/3921614304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9c067e90 con 0x7fda9c06d5b0 2023-08-27T11:19:15.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.824+0000 7fda837fe700 1 -- 172.21.15.112:0/3921614304 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fda9000cd20 con 0x7fda9c06d5b0 2023-08-27T11:19:15.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.824+0000 7fda837fe700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fda8806a170 0x7fda8806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:15.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.824+0000 7fda837fe700 1 -- 172.21.15.112:0/3921614304 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fda90088cb0 con 0x7fda9c06d5b0 2023-08-27T11:19:15.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.824+0000 7fda9a59c700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fda8806a170 0x7fda8806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:15.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.825+0000 7fda9a59c700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fda8806a170 0x7fda8806c630 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda9c137520 tx=0x7fda8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:15.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:15.828+0000 7fda837fe700 1 -- 172.21.15.112:0/3921614304 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fda90054490 con 0x7fda9c06d5b0 2023-08-27T11:19:16.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.062+0000 7fdaa1106700 1 -- 172.21.15.112:0/3921614304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fda9c067a20 con 0x7fda9c06d5b0 2023-08-27T11:19:16.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.667+0000 7fda837fe700 1 -- 172.21.15.112:0/3921614304 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v35) v1 ==== 174+0+0 (secure 0 0 0) 0x7fda900170c0 con 0x7fda9c06d5b0 2023-08-27T11:19:16.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.672+0000 7fdaa1106700 1 -- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fda8806a170 msgr2=0x7fda8806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:16.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.672+0000 7fdaa1106700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fda8806a170 0x7fda8806c630 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda9c137520 tx=0x7fda8c009040 comp rx=0 tx=0).stop 2023-08-27T11:19:16.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.672+0000 7fdaa1106700 1 -- 172.21.15.112:0/3921614304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c06d5b0 msgr2=0x7fda9c13ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:16.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.672+0000 7fdaa1106700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c06d5b0 0x7fda9c13ccc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fda9000e9b0 tx=0x7fda9000ecc0 comp rx=0 tx=0).stop 2023-08-27T11:19:16.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.672+0000 7fdaa1106700 1 -- 172.21.15.112:0/3921614304 shutdown_connections 2023-08-27T11:19:16.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.672+0000 7fdaa1106700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fda8806a170 0x7fda8806c630 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:16.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.672+0000 7fdaa1106700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda9c143ba0 0x7fda9c13aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:16.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.672+0000 7fdaa1106700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c06d5b0 0x7fda9c13ccc0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:16.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.672+0000 7fdaa1106700 1 --2- 172.21.15.112:0/3921614304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c1437c0 0x7fda9c14aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:16.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.672+0000 7fdaa1106700 1 -- 172.21.15.112:0/3921614304 >> 172.21.15.112:0/3921614304 conn(0x7fda9c0b1e90 msgr2=0x7fda9c141b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:16.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.673+0000 7fdaa1106700 1 -- 172.21.15.112:0/3921614304 shutdown_connections 2023-08-27T11:19:16.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.673+0000 7fdaa1106700 1 -- 172.21.15.112:0/3921614304 wait complete. 2023-08-27T11:19:16.677 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 11 and data pool 12 2023-08-27T11:19:16.689 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:19:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.809+0000 7fd8aefe9700 1 Processor -- start 2023-08-27T11:19:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.809+0000 7fd8aefe9700 1 -- start start 2023-08-27T11:19:16.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.809+0000 7fd8aefe9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd8a8144570 0x7fd8a815ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:16.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.809+0000 7fd8aefe9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a80b45f0 con 0x7fd8a80b4ad0 2023-08-27T11:19:16.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.809+0000 7fd8aefe9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a80b4310 con 0x7fd8a80b3e50 2023-08-27T11:19:16.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.809+0000 7fd8aefe9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a80cf7a0 con 0x7fd8a8144570 2023-08-27T11:19:16.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.809+0000 7fd8acd85700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd8a8144570 0x7fd8a815ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:16.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.810+0000 7fd8ad586700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd8a80b4ad0 0x7fd8a80b4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53310/0 (socket says 172.21.15.112:53310) 2023-08-27T11:19:16.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.810+0000 7fd8acd85700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd8a8144570 0x7fd8a815ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58926/0 (socket says 172.21.15.112:58926) 2023-08-27T11:19:16.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.810+0000 7fd8ad586700 1 -- 172.21.15.112:0/2070360337 learned_addr learned my addr 172.21.15.112:0/2070360337 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:16.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.810+0000 7fd8acd85700 1 -- 172.21.15.112:0/2070360337 >> v1:172.21.15.138:6789/0 conn(0x7fd8a80b3e50 legacy=0x7fd8a815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:16.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.810+0000 7fd8a77fe700 1 -- 172.21.15.112:0/2070360337 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26025328 0 0) 0x7fd8a80b45f0 con 0x7fd8a80b4ad0 2023-08-27T11:19:16.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.810+0000 7fd8acd85700 1 -- 172.21.15.112:0/2070360337 >> v1:172.21.15.112:6789/0 conn(0x7fd8a80b4ad0 legacy=0x7fd8a80b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:16.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.810+0000 7fd8acd85700 1 -- 172.21.15.112:0/2070360337 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a80cdcb0 con 0x7fd8a8144570 2023-08-27T11:19:16.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.810+0000 7fd8acd85700 1 --2- 172.21.15.112:0/2070360337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a8144570 0x7fd8a815ee20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009840 tx=0x7fd89c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a211d40ff8d216e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:16.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.810+0000 7fd8a77fe700 1 -- 172.21.15.112:0/2070360337 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c011830 con 0x7fd8a8144570 2023-08-27T11:19:16.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.810+0000 7fd8a77fe700 1 -- 172.21.15.112:0/2070360337 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c011990 con 0x7fd8a8144570 2023-08-27T11:19:16.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.810+0000 7fd8a77fe700 1 -- 172.21.15.112:0/2070360337 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c011d20 con 0x7fd8a8144570 2023-08-27T11:19:16.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.811+0000 7fd8aefe9700 1 -- 172.21.15.112:0/2070360337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a8144570 msgr2=0x7fd8a815ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:16.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.811+0000 7fd8aefe9700 1 --2- 172.21.15.112:0/2070360337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a8144570 0x7fd8a815ee20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009840 tx=0x7fd89c009b50 comp rx=0 tx=0).stop 2023-08-27T11:19:16.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.811+0000 7fd8aefe9700 1 -- 172.21.15.112:0/2070360337 shutdown_connections 2023-08-27T11:19:16.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.811+0000 7fd8aefe9700 1 --2- 172.21.15.112:0/2070360337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a8144570 0x7fd8a815ee20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:16.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.811+0000 7fd8aefe9700 1 -- 172.21.15.112:0/2070360337 >> 172.21.15.112:0/2070360337 conn(0x7fd8a806f970 msgr2=0x7fd8a806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:16.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.811+0000 7fd8aefe9700 1 -- 172.21.15.112:0/2070360337 shutdown_connections 2023-08-27T11:19:16.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.811+0000 7fd8aefe9700 1 -- 172.21.15.112:0/2070360337 wait complete. 2023-08-27T11:19:16.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.812+0000 7fd8aefe9700 1 Processor -- start 2023-08-27T11:19:16.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.812+0000 7fd8aefe9700 1 -- start start 2023-08-27T11:19:16.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8aefe9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a80b3e50 0x7fd8a8159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:16.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8aefe9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8a80b4ad0 0x7fd8a81f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:16.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8aefe9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8a8144570 0x7fd8a81368f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:16.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8aefe9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a815ebb0 con 0x7fd8a80b3e50 2023-08-27T11:19:16.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8aefe9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a81f47f0 con 0x7fd8a80b4ad0 2023-08-27T11:19:16.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8aefe9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a81f4ad0 con 0x7fd8a8144570 2023-08-27T11:19:16.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8acd85700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a80b3e50 0x7fd8a8159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:16.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8acd85700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a80b3e50 0x7fd8a8159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58930/0 (socket says 172.21.15.112:58930) 2023-08-27T11:19:16.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8acd85700 1 -- 172.21.15.112:0/4268106421 learned_addr learned my addr 172.21.15.112:0/4268106421 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:16.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8ad586700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8a8144570 0x7fd8a81368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:16.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8a7fff700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8a80b4ad0 0x7fd8a81f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:16.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8acd85700 1 -- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8a8144570 msgr2=0x7fd8a81368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:16.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8acd85700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8a8144570 0x7fd8a81368f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:16.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8acd85700 1 -- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8a80b4ad0 msgr2=0x7fd8a81f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:16.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8acd85700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8a80b4ad0 0x7fd8a81f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:16.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8acd85700 1 -- 172.21.15.112:0/4268106421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c0094a0 con 0x7fd8a80b3e50 2023-08-27T11:19:16.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.813+0000 7fd8acd85700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a80b3e50 0x7fd8a8159540 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0090b0 tx=0x7fd89c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:16.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.814+0000 7fd8a57fa700 1 -- 172.21.15.112:0/4268106421 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c01a900 con 0x7fd8a80b3e50 2023-08-27T11:19:16.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.814+0000 7fd8a57fa700 1 -- 172.21.15.112:0/4268106421 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c01aa60 con 0x7fd8a80b3e50 2023-08-27T11:19:16.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.814+0000 7fd8a57fa700 1 -- 172.21.15.112:0/4268106421 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c01add0 con 0x7fd8a80b3e50 2023-08-27T11:19:16.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.814+0000 7fd8aefe9700 1 -- 172.21.15.112:0/4268106421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a81f4db0 con 0x7fd8a80b3e50 2023-08-27T11:19:16.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.814+0000 7fd8aefe9700 1 -- 172.21.15.112:0/4268106421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a8159ec0 con 0x7fd8a80b3e50 2023-08-27T11:19:16.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.815+0000 7fd8aefe9700 1 -- 172.21.15.112:0/4268106421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a8067a20 con 0x7fd8a80b3e50 2023-08-27T11:19:16.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.815+0000 7fd8a57fa700 1 -- 172.21.15.112:0/4268106421 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fd89c022070 con 0x7fd8a80b3e50 2023-08-27T11:19:16.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.815+0000 7fd8a57fa700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd88c06a170 0x7fd88c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:16.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.815+0000 7fd8a57fa700 1 -- 172.21.15.112:0/4268106421 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd89c0898e0 con 0x7fd8a80b3e50 2023-08-27T11:19:16.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.816+0000 7fd8a7fff700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd88c06a170 0x7fd88c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:16.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.816+0000 7fd8a7fff700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd88c06a170 0x7fd88c06c630 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8140f70 tx=0x7fd894009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:16.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:16.820+0000 7fd8a57fa700 1 -- 172.21.15.112:0/4268106421 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd89c055240 con 0x7fd8a80b3e50 2023-08-27T11:19:17.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.008+0000 7fd8aefe9700 1 -- 172.21.15.112:0/4268106421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8a80cdf60 con 0x7fd8a80b3e50 2023-08-27T11:19:17.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.009+0000 7fd8a57fa700 1 -- 172.21.15.112:0/4268106421 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+18333 (secure 0 0 0) 0x7fd89c058860 con 0x7fd8a80b3e50 2023-08-27T11:19:17.011 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:17.012 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":64,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:19:16.064007+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:19:14.179813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-27T11:19:15.021524+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:19:17.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.011+0000 7fd8aefe9700 1 -- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd88c06a170 msgr2=0x7fd88c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:17.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.011+0000 7fd8aefe9700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd88c06a170 0x7fd88c06c630 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8140f70 tx=0x7fd894009040 comp rx=0 tx=0).stop 2023-08-27T11:19:17.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.011+0000 7fd8aefe9700 1 -- 172.21.15.112:0/4268106421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a80b3e50 msgr2=0x7fd8a8159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:17.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.011+0000 7fd8aefe9700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a80b3e50 0x7fd8a8159540 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0090b0 tx=0x7fd89c004750 comp rx=0 tx=0).stop 2023-08-27T11:19:17.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.012+0000 7fd8aefe9700 1 -- 172.21.15.112:0/4268106421 shutdown_connections 2023-08-27T11:19:17.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.012+0000 7fd8aefe9700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd88c06a170 0x7fd88c06c630 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.012+0000 7fd8aefe9700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8a8144570 0x7fd8a81368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.012+0000 7fd8aefe9700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a80b3e50 0x7fd8a8159540 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.012+0000 7fd8aefe9700 1 --2- 172.21.15.112:0/4268106421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8a80b4ad0 0x7fd8a81f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.012+0000 7fd8aefe9700 1 -- 172.21.15.112:0/4268106421 >> 172.21.15.112:0/4268106421 conn(0x7fd8a806f970 msgr2=0x7fd8a81e5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:17.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.012+0000 7fd8aefe9700 1 -- 172.21.15.112:0/4268106421 shutdown_connections 2023-08-27T11:19:17.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.012+0000 7fd8aefe9700 1 -- 172.21.15.112:0/4268106421 wait complete. 2023-08-27T11:19:17.025 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:19:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.146+0000 7f172f864700 1 Processor -- start 2023-08-27T11:19:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.147+0000 7f172f864700 1 -- start start 2023-08-27T11:19:17.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.147+0000 7f172f864700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f172815a120 0x7f17280b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.147+0000 7f172f864700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f172806da40 0x7f17280b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.147+0000 7f172f864700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172806deb0 con 0x7f172806d5b0 2023-08-27T11:19:17.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.147+0000 7f172f864700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172815b540 con 0x7f172815a120 2023-08-27T11:19:17.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.147+0000 7f172f864700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17280af830 con 0x7f172806da40 2023-08-27T11:19:17.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.148+0000 7f172d600700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f172815a120 0x7f17280b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.148+0000 7f172d600700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f172815a120 0x7f17280b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58938/0 (socket says 172.21.15.112:58938) 2023-08-27T11:19:17.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.148+0000 7f172d600700 1 -- 172.21.15.112:0/3999745350 learned_addr learned my addr 172.21.15.112:0/3999745350 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:17.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.148+0000 7f172de01700 1 --2- 172.21.15.112:0/3999745350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f172806da40 0x7f17280b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.148+0000 7f172d600700 1 -- 172.21.15.112:0/3999745350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f172806da40 msgr2=0x7f17280b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:17.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.148+0000 7f172d600700 1 --2- 172.21.15.112:0/3999745350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f172806da40 0x7f17280b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.148+0000 7f172d600700 1 -- 172.21.15.112:0/3999745350 >> v1:172.21.15.138:6789/0 conn(0x7f172806d5b0 legacy=0x7f17280b3440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:19:17.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.148+0000 7f172d600700 1 -- 172.21.15.112:0/3999745350 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17280afb10 con 0x7f172815a120 2023-08-27T11:19:17.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.148+0000 7f172d600700 1 --2- 172.21.15.112:0/3999745350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172815a120 0x7f17280b05f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f172400bb90 tx=0x7f172400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5721d3d0ff2c8d63 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:17.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.148+0000 7f171ffff700 1 -- 172.21.15.112:0/3999745350 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172400dd10 con 0x7f172815a120 2023-08-27T11:19:17.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.148+0000 7f171ffff700 1 -- 172.21.15.112:0/3999745350 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172400de70 con 0x7f172815a120 2023-08-27T11:19:17.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.148+0000 7f171ffff700 1 -- 172.21.15.112:0/3999745350 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724004230 con 0x7f172815a120 2023-08-27T11:19:17.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.149+0000 7f172f864700 1 -- 172.21.15.112:0/3999745350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172815a120 msgr2=0x7f17280b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:17.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.149+0000 7f172f864700 1 --2- 172.21.15.112:0/3999745350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172815a120 0x7f17280b05f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f172400bb90 tx=0x7f172400bea0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.149+0000 7f172f864700 1 -- 172.21.15.112:0/3999745350 shutdown_connections 2023-08-27T11:19:17.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.149+0000 7f172f864700 1 --2- 172.21.15.112:0/3999745350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f172806da40 0x7f17280b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.149+0000 7f172f864700 1 --2- 172.21.15.112:0/3999745350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172815a120 0x7f17280b05f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.149+0000 7f172f864700 1 -- 172.21.15.112:0/3999745350 >> 172.21.15.112:0/3999745350 conn(0x7f17280cff80 msgr2=0x7f17280d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:17.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.149+0000 7f172f864700 1 -- 172.21.15.112:0/3999745350 shutdown_connections 2023-08-27T11:19:17.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.149+0000 7f172f864700 1 -- 172.21.15.112:0/3999745350 wait complete. 2023-08-27T11:19:17.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.150+0000 7f172f864700 1 Processor -- start 2023-08-27T11:19:17.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.151+0000 7f172f864700 1 -- start start 2023-08-27T11:19:17.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.151+0000 7f172f864700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f172806d5b0 0x7f1728144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.151+0000 7f172f864700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172806da40 0x7f1728154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.151+0000 7f172f864700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f172815a120 0x7f17281551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.151+0000 7f172f864700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1728144640 con 0x7f172806da40 2023-08-27T11:19:17.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.151+0000 7f172f864700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1728146930 con 0x7f172806d5b0 2023-08-27T11:19:17.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.151+0000 7f172f864700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1728146b80 con 0x7f172815a120 2023-08-27T11:19:17.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f172cdff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172806da40 0x7f1728154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f172cdff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172806da40 0x7f1728154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58946/0 (socket says 172.21.15.112:58946) 2023-08-27T11:19:17.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f172cdff700 1 -- 172.21.15.112:0/1707573320 learned_addr learned my addr 172.21.15.112:0/1707573320 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:17.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f172cdff700 1 -- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f172815a120 msgr2=0x7f17281551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:17.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f172de01700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f172815a120 0x7f17281551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f172d600700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f172806d5b0 0x7f1728144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f172cdff700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f172815a120 0x7f17281551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f172cdff700 1 -- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f172806d5b0 msgr2=0x7f1728144100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:17.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f172cdff700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f172806d5b0 0x7f1728144100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f172cdff700 1 -- 172.21.15.112:0/1707573320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172400b7f0 con 0x7f172806da40 2023-08-27T11:19:17.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f172cdff700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172806da40 0x7f1728154ca0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f171800b550 tx=0x7f171800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:17.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f171dffb700 1 -- 172.21.15.112:0/1707573320 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718011830 con 0x7f172806da40 2023-08-27T11:19:17.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f171dffb700 1 -- 172.21.15.112:0/1707573320 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17180119b0 con 0x7f172806da40 2023-08-27T11:19:17.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f172f864700 1 -- 172.21.15.112:0/1707573320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1728146e60 con 0x7f172806da40 2023-08-27T11:19:17.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f171dffb700 1 -- 172.21.15.112:0/1707573320 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718011d60 con 0x7f172806da40 2023-08-27T11:19:17.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.152+0000 7f172f864700 1 -- 172.21.15.112:0/1707573320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1728147420 con 0x7f172806da40 2023-08-27T11:19:17.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.153+0000 7f172f864700 1 -- 172.21.15.112:0/1707573320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1728147750 con 0x7f172806da40 2023-08-27T11:19:17.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.154+0000 7f171dffb700 1 -- 172.21.15.112:0/1707573320 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f171801a400 con 0x7f172806da40 2023-08-27T11:19:17.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.154+0000 7f171dffb700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f170c06a170 0x7f170c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.154+0000 7f171dffb700 1 -- 172.21.15.112:0/1707573320 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1718088620 con 0x7f172806da40 2023-08-27T11:19:17.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.154+0000 7f172d600700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f170c06a170 0x7f170c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.155+0000 7f172d600700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f170c06a170 0x7f170c06c630 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1724000c00 tx=0x7f1724013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:17.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.159+0000 7f171dffb700 1 -- 172.21.15.112:0/1707573320 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1718053d20 con 0x7f172806da40 2023-08-27T11:19:17.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.351+0000 7f172f864700 1 -- 172.21.15.112:0/1707573320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1728067a20 con 0x7f172806da40 2023-08-27T11:19:17.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.353+0000 7f171dffb700 1 -- 172.21.15.112:0/1707573320 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18320 (secure 0 0 0) 0x7f1718057340 con 0x7f172806da40 2023-08-27T11:19:17.358 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:17.358 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":65,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:19:17.239518+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:19:14.179813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-27T11:19:15.021524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:19:17.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.355+0000 7f172f864700 1 -- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f170c06a170 msgr2=0x7f170c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:17.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.355+0000 7f172f864700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f170c06a170 0x7f170c06c630 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1724000c00 tx=0x7f1724013000 comp rx=0 tx=0).stop 2023-08-27T11:19:17.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.355+0000 7f172f864700 1 -- 172.21.15.112:0/1707573320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172806da40 msgr2=0x7f1728154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:17.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.355+0000 7f172f864700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172806da40 0x7f1728154ca0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f171800b550 tx=0x7f171800b910 comp rx=0 tx=0).stop 2023-08-27T11:19:17.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.355+0000 7f172f864700 1 -- 172.21.15.112:0/1707573320 shutdown_connections 2023-08-27T11:19:17.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.355+0000 7f172f864700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f170c06a170 0x7f170c06c630 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.355+0000 7f172f864700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f172815a120 0x7f17281551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.355+0000 7f172f864700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f172806d5b0 0x7f1728144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.355+0000 7f172f864700 1 --2- 172.21.15.112:0/1707573320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172806da40 0x7f1728154ca0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.355+0000 7f172f864700 1 -- 172.21.15.112:0/1707573320 >> 172.21.15.112:0/1707573320 conn(0x7f17280cff80 msgr2=0x7f17281542b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:17.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.355+0000 7f172f864700 1 -- 172.21.15.112:0/1707573320 shutdown_connections 2023-08-27T11:19:17.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.355+0000 7f172f864700 1 -- 172.21.15.112:0/1707573320 wait complete. 2023-08-27T11:19:17.370 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:19:17.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.494+0000 7f1db6480700 1 Processor -- start 2023-08-27T11:19:17.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.494+0000 7f1db6480700 1 -- start start 2023-08-27T11:19:17.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1db6480700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1db0144b80 0x7f1db00b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1db6480700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db00cef20 con 0x7f1db006c6e0 2023-08-27T11:19:17.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1db6480700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db00b42a0 con 0x7f1db00b3de0 2023-08-27T11:19:17.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1db6480700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00b4b40 con 0x7f1db0144b80 2023-08-27T11:19:17.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1db4a1d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1db006c6e0 0x7f1db00cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53338/0 (socket says 172.21.15.112:53338) 2023-08-27T11:19:17.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1db4a1d700 1 -- 172.21.15.112:0/870418225 learned_addr learned my addr 172.21.15.112:0/870418225 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:17.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daffff700 1 --2- 172.21.15.112:0/870418225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1db0144b80 0x7f1db00b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620191018 0 0) 0x7f1db00cef20 con 0x7f1db006c6e0 2023-08-27T11:19:17.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d980035b0 con 0x7f1db006c6e0 2023-08-27T11:19:17.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2143334106 0 0) 0x7f1d980035b0 con 0x7f1db006c6e0 2023-08-27T11:19:17.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1db0144b80 msgr2=0x7f1db00b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:17.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 --2- 172.21.15.112:0/870418225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1db0144b80 0x7f1db00b6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 >> v1:172.21.15.138:6789/0 conn(0x7f1db00b3de0 legacy=0x7f1db006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:17.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db00b4580 con 0x7f1db006c6e0 2023-08-27T11:19:17.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f1da80019f0 con 0x7f1db006c6e0 2023-08-27T11:19:17.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 >> v1:172.21.15.112:6789/0 conn(0x7f1db006c6e0 legacy=0x7f1db00cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:17.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 --2- 172.21.15.112:0/870418225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d98003a30 0x7f1d98003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 --2- 172.21.15.112:0/870418225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d98004850 0x7f1d98004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 --2- 172.21.15.112:0/870418225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1db0144b80 0x7f1d98005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00b4b40 con 0x7f1d98004850 2023-08-27T11:19:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9800bb40 con 0x7f1d98003a30 2023-08-27T11:19:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9800bd00 con 0x7f1db0144b80 2023-08-27T11:19:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.495+0000 7f1daf7fe700 1 --2- 172.21.15.112:0/870418225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d98004850 0x7f1d98004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.496+0000 7f1daffff700 1 --2- 172.21.15.112:0/870418225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d98003a30 0x7f1d98003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.496+0000 7f1db4a1d700 1 --2- 172.21.15.112:0/870418225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1db0144b80 0x7f1d98005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.496+0000 7f1daf7fe700 1 -- 172.21.15.112:0/870418225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1db0144b80 msgr2=0x7f1d98005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.496+0000 7f1daf7fe700 1 --2- 172.21.15.112:0/870418225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1db0144b80 0x7f1d98005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.496+0000 7f1daf7fe700 1 -- 172.21.15.112:0/870418225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d98003a30 msgr2=0x7f1d98003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.496+0000 7f1daf7fe700 1 --2- 172.21.15.112:0/870418225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d98003a30 0x7f1d98003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.496+0000 7f1daf7fe700 1 -- 172.21.15.112:0/870418225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d980035b0 con 0x7f1d98004850 2023-08-27T11:19:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.496+0000 7f1db4a1d700 1 --2- 172.21.15.112:0/870418225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1db0144b80 0x7f1d98005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.496+0000 7f1daf7fe700 1 --2- 172.21.15.112:0/870418225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d98004850 0x7f1d98004cd0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1da00017c0 tx=0x7f1d98005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcae524c3eafd9e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.496+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da00052d0 con 0x7f1d98004850 2023-08-27T11:19:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.496+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da0005430 con 0x7f1d98004850 2023-08-27T11:19:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.496+0000 7f1daeffd700 1 -- 172.21.15.112:0/870418225 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da00038c0 con 0x7f1d98004850 2023-08-27T11:19:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.497+0000 7f1db6480700 1 -- 172.21.15.112:0/870418225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d98004850 msgr2=0x7f1d98004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.497+0000 7f1db6480700 1 --2- 172.21.15.112:0/870418225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d98004850 0x7f1d98004cd0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1da00017c0 tx=0x7f1d98005880 comp rx=0 tx=0).stop 2023-08-27T11:19:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.497+0000 7f1db4a1d700 1 -- 172.21.15.112:0/870418225 reap_dead start 2023-08-27T11:19:17.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.497+0000 7f1db6480700 1 -- 172.21.15.112:0/870418225 shutdown_connections 2023-08-27T11:19:17.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.497+0000 7f1db6480700 1 -- 172.21.15.112:0/870418225 >> 172.21.15.112:0/870418225 conn(0x7f1db00cdc50 msgr2=0x7f1db00ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:17.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.497+0000 7f1db6480700 1 -- 172.21.15.112:0/870418225 shutdown_connections 2023-08-27T11:19:17.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.497+0000 7f1db6480700 1 -- 172.21.15.112:0/870418225 wait complete. 2023-08-27T11:19:17.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.498+0000 7f1db6480700 1 Processor -- start 2023-08-27T11:19:17.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.498+0000 7f1db6480700 1 -- start start 2023-08-27T11:19:17.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1db6480700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d98004850 0x7f1db00c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1db6480700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1db0142b10 0x7f1db0142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1db6480700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1db0159750 0x7f1db0159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1db6480700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db01434d0 con 0x7f1db0142b10 2023-08-27T11:19:17.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1db6480700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db015a480 con 0x7f1db0159750 2023-08-27T11:19:17.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1db6480700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db015a760 con 0x7f1d98004850 2023-08-27T11:19:17.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1daf7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1db0142b10 0x7f1db0142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1daf7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1db0142b10 0x7f1db0142f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58966/0 (socket says 172.21.15.112:58966) 2023-08-27T11:19:17.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1daf7fe700 1 -- 172.21.15.112:0/1695439853 learned_addr learned my addr 172.21.15.112:0/1695439853 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:17.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1db4a1d700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1db0159750 0x7f1db0159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1daffff700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d98004850 0x7f1db00c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1daf7fe700 1 -- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d98004850 msgr2=0x7f1db00c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:17.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1daf7fe700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d98004850 0x7f1db00c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1daf7fe700 1 -- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1db0159750 msgr2=0x7f1db0159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:17.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1daf7fe700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1db0159750 0x7f1db0159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1daf7fe700 1 -- 172.21.15.112:0/1695439853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db015aa40 con 0x7f1db0142b10 2023-08-27T11:19:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1daf7fe700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1db0142b10 0x7f1db0142f90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800bb10 tx=0x7f1da0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1dacff9700 1 -- 172.21.15.112:0/1695439853 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0006ea0 con 0x7f1db0142b10 2023-08-27T11:19:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1dacff9700 1 -- 172.21.15.112:0/1695439853 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da0007240 con 0x7f1db0142b10 2023-08-27T11:19:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1db6480700 1 -- 172.21.15.112:0/1695439853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db01434d0 con 0x7f1db0142b10 2023-08-27T11:19:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1dacff9700 1 -- 172.21.15.112:0/1695439853 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da00075f0 con 0x7f1db0142b10 2023-08-27T11:19:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.499+0000 7f1db6480700 1 -- 172.21.15.112:0/1695439853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db015b000 con 0x7f1db0142b10 2023-08-27T11:19:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.500+0000 7f1db6480700 1 -- 172.21.15.112:0/1695439853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db015b190 con 0x7f1db0142b10 2023-08-27T11:19:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.501+0000 7f1dacff9700 1 -- 172.21.15.112:0/1695439853 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f1da0003680 con 0x7f1db0142b10 2023-08-27T11:19:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.501+0000 7f1dacff9700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d9c06a170 0x7f1d9c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.501+0000 7f1dacff9700 1 -- 172.21.15.112:0/1695439853 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1da0089400 con 0x7f1db0142b10 2023-08-27T11:19:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.502+0000 7f1daffff700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d9c06a170 0x7f1d9c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.502+0000 7f1daffff700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d9c06a170 0x7f1d9c06c630 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009c60 tx=0x7f1da4009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.506+0000 7f1dacff9700 1 -- 172.21.15.112:0/1695439853 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1da0054a90 con 0x7f1db0142b10 2023-08-27T11:19:17.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:17.740+0000 7f1db6480700 1 -- 172.21.15.112:0/1695439853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1db0067a20 con 0x7f1db0142b10 2023-08-27T11:19:18.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.690+0000 7f1dacff9700 1 -- 172.21.15.112:0/1695439853 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v38) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1da00580b0 con 0x7f1db0142b10 2023-08-27T11:19:18.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.693+0000 7f1db6480700 1 -- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d9c06a170 msgr2=0x7f1d9c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:18.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.693+0000 7f1db6480700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d9c06a170 0x7f1d9c06c630 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009c60 tx=0x7f1da4009210 comp rx=0 tx=0).stop 2023-08-27T11:19:18.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.693+0000 7f1db6480700 1 -- 172.21.15.112:0/1695439853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1db0142b10 msgr2=0x7f1db0142f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:18.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.693+0000 7f1db6480700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1db0142b10 0x7f1db0142f90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800bb10 tx=0x7f1da0001120 comp rx=0 tx=0).stop 2023-08-27T11:19:18.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.693+0000 7f1db6480700 1 -- 172.21.15.112:0/1695439853 shutdown_connections 2023-08-27T11:19:18.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.693+0000 7f1db6480700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1d9c06a170 0x7f1d9c06c630 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:18.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.693+0000 7f1db6480700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1db0159750 0x7f1db0159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:18.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.693+0000 7f1db6480700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d98004850 0x7f1db00c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:18.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.693+0000 7f1db6480700 1 --2- 172.21.15.112:0/1695439853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1db0142b10 0x7f1db0142f90 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:18.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.693+0000 7f1db6480700 1 -- 172.21.15.112:0/1695439853 >> 172.21.15.112:0/1695439853 conn(0x7f1db00cdc50 msgr2=0x7f1db0143dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:18.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.693+0000 7f1db6480700 1 -- 172.21.15.112:0/1695439853 shutdown_connections 2023-08-27T11:19:18.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.693+0000 7f1db6480700 1 -- 172.21.15.112:0/1695439853 wait complete. 2023-08-27T11:19:18.709 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:19:18.709 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:19:18.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.836+0000 7f45afd4f700 1 Processor -- start 2023-08-27T11:19:18.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.836+0000 7f45afd4f700 1 -- start start 2023-08-27T11:19:18.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.837+0000 7f45afd4f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f45a81437c0 0x7f45a80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:18.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.837+0000 7f45afd4f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a8143ba0 0x7f45a80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:18.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.837+0000 7f45afd4f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a80cfd50 con 0x7f45a806d5b0 2023-08-27T11:19:18.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.837+0000 7f45afd4f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a80cf140 con 0x7f45a81437c0 2023-08-27T11:19:18.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.837+0000 7f45afd4f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a80cf420 con 0x7f45a8143ba0 2023-08-27T11:19:18.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.837+0000 7f45adaeb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f45a81437c0 0x7f45a80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:18.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.837+0000 7f45adaeb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f45a81437c0 0x7f45a80b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58972/0 (socket says 172.21.15.112:58972) 2023-08-27T11:19:18.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.837+0000 7f45adaeb700 1 -- 172.21.15.112:0/1556081740 learned_addr learned my addr 172.21.15.112:0/1556081740 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:18.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.837+0000 7f45ad2ea700 1 --2- 172.21.15.112:0/1556081740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a8143ba0 0x7f45a80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:18.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.837+0000 7f45adaeb700 1 -- 172.21.15.112:0/1556081740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a8143ba0 msgr2=0x7f45a80ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:18.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45adaeb700 1 --2- 172.21.15.112:0/1556081740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a8143ba0 0x7f45a80ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:18.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45adaeb700 1 -- 172.21.15.112:0/1556081740 >> v1:172.21.15.138:6789/0 conn(0x7f45a806d5b0 legacy=0x7f45a815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:18.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45adaeb700 1 -- 172.21.15.112:0/1556081740 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a80cf700 con 0x7f45a81437c0 2023-08-27T11:19:18.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45adaeb700 1 --2- 172.21.15.112:0/1556081740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45a81437c0 0x7f45a80b4670 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f45a000d150 tx=0x7f45a000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0db57063cf1ebec server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:18.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45acae9700 1 -- 172.21.15.112:0/1556081740 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a0010070 con 0x7f45a81437c0 2023-08-27T11:19:18.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45acae9700 1 -- 172.21.15.112:0/1556081740 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a000dd10 con 0x7f45a81437c0 2023-08-27T11:19:18.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45acae9700 1 -- 172.21.15.112:0/1556081740 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a000b310 con 0x7f45a81437c0 2023-08-27T11:19:18.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45afd4f700 1 -- 172.21.15.112:0/1556081740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45a81437c0 msgr2=0x7f45a80b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:18.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45afd4f700 1 --2- 172.21.15.112:0/1556081740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45a81437c0 0x7f45a80b4670 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f45a000d150 tx=0x7f45a000d460 comp rx=0 tx=0).stop 2023-08-27T11:19:18.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45afd4f700 1 -- 172.21.15.112:0/1556081740 shutdown_connections 2023-08-27T11:19:18.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45afd4f700 1 --2- 172.21.15.112:0/1556081740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45a81437c0 0x7f45a80b4670 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:18.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45afd4f700 1 --2- 172.21.15.112:0/1556081740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a8143ba0 0x7f45a80ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:18.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45afd4f700 1 -- 172.21.15.112:0/1556081740 >> 172.21.15.112:0/1556081740 conn(0x7f45a80b1e90 msgr2=0x7f45a80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:18.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.838+0000 7f45afd4f700 1 -- 172.21.15.112:0/1556081740 shutdown_connections 2023-08-27T11:19:18.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.839+0000 7f45afd4f700 1 -- 172.21.15.112:0/1556081740 wait complete. 2023-08-27T11:19:18.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.840+0000 7f45afd4f700 1 Processor -- start 2023-08-27T11:19:18.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.840+0000 7f45afd4f700 1 -- start start 2023-08-27T11:19:18.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.840+0000 7f45afd4f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806d5b0 0x7f45a8159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:18.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.840+0000 7f45afd4f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45a81437c0 0x7f45a80ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:18.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.840+0000 7f45afd4f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45a8143ba0 0x7f45a8140390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:18.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.840+0000 7f45afd4f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a81f63f0 con 0x7f45a81437c0 2023-08-27T11:19:18.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.840+0000 7f45afd4f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a81f6690 con 0x7f45a8143ba0 2023-08-27T11:19:18.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.840+0000 7f45afd4f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a81f6970 con 0x7f45a806d5b0 2023-08-27T11:19:18.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.840+0000 7f45ad2ea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45a81437c0 0x7f45a80ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:18.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.840+0000 7f45ad2ea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45a81437c0 0x7f45a80ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58980/0 (socket says 172.21.15.112:58980) 2023-08-27T11:19:18.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.840+0000 7f45ad2ea700 1 -- 172.21.15.112:0/531914438 learned_addr learned my addr 172.21.15.112:0/531914438 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:18.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f45adaeb700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806d5b0 0x7f45a8159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:18.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f45ae2ec700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45a8143ba0 0x7f45a8140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:18.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f45ad2ea700 1 -- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806d5b0 msgr2=0x7f45a8159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:18.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f45ad2ea700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806d5b0 0x7f45a8159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:18.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f45ad2ea700 1 -- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45a8143ba0 msgr2=0x7f45a8140390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:18.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f45ad2ea700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45a8143ba0 0x7f45a8140390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:18.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f45ad2ea700 1 -- 172.21.15.112:0/531914438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a000bd40 con 0x7f45a81437c0 2023-08-27T11:19:18.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f45ae2ec700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45a8143ba0 0x7f45a8140390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:18.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f45ad2ea700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45a81437c0 0x7f45a80ca4b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f459800e870 tx=0x7f459800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:18.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f459e7fc700 1 -- 172.21.15.112:0/531914438 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598009590 con 0x7f45a81437c0 2023-08-27T11:19:18.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f459e7fc700 1 -- 172.21.15.112:0/531914438 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4598013680 con 0x7f45a81437c0 2023-08-27T11:19:18.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f45afd4f700 1 -- 172.21.15.112:0/531914438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a81f6c50 con 0x7f45a81437c0 2023-08-27T11:19:18.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f459e7fc700 1 -- 172.21.15.112:0/531914438 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45980139f0 con 0x7f45a81437c0 2023-08-27T11:19:18.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.841+0000 7f45afd4f700 1 -- 172.21.15.112:0/531914438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a8159e50 con 0x7f45a81437c0 2023-08-27T11:19:18.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.842+0000 7f45afd4f700 1 -- 172.21.15.112:0/531914438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a8067e90 con 0x7f45a81437c0 2023-08-27T11:19:18.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.843+0000 7f459e7fc700 1 -- 172.21.15.112:0/531914438 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f4598013b50 con 0x7f45a81437c0 2023-08-27T11:19:18.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.843+0000 7f459e7fc700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f458c06a170 0x7f458c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:18.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.843+0000 7f459e7fc700 1 -- 172.21.15.112:0/531914438 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f45980883d0 con 0x7f45a81437c0 2023-08-27T11:19:18.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.843+0000 7f45adaeb700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f458c06a170 0x7f458c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:18.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.844+0000 7f45adaeb700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f458c06a170 0x7f458c06c630 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f45a0000f80 tx=0x7f45a000b000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:18.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:18.847+0000 7f459e7fc700 1 -- 172.21.15.112:0/531914438 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4598053b60 con 0x7f45a81437c0 2023-08-27T11:19:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.080+0000 7f45afd4f700 1 -- 172.21.15.112:0/531914438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f45a8067a20 con 0x7f45a81437c0 2023-08-27T11:19:19.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.692+0000 7f459e7fc700 1 -- 172.21.15.112:0/531914438 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v39) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4598057180 con 0x7f45a81437c0 2023-08-27T11:19:19.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.694+0000 7f45afd4f700 1 -- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f458c06a170 msgr2=0x7f458c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:19.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.694+0000 7f45afd4f700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f458c06a170 0x7f458c06c630 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f45a0000f80 tx=0x7f45a000b000 comp rx=0 tx=0).stop 2023-08-27T11:19:19.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.694+0000 7f45afd4f700 1 -- 172.21.15.112:0/531914438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45a81437c0 msgr2=0x7f45a80ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:19.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.694+0000 7f45afd4f700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45a81437c0 0x7f45a80ca4b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f459800e870 tx=0x7f459800eb80 comp rx=0 tx=0).stop 2023-08-27T11:19:19.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.694+0000 7f45afd4f700 1 -- 172.21.15.112:0/531914438 shutdown_connections 2023-08-27T11:19:19.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.694+0000 7f45afd4f700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f458c06a170 0x7f458c06c630 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:19.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.694+0000 7f45afd4f700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45a8143ba0 0x7f45a8140390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:19.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.694+0000 7f45afd4f700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806d5b0 0x7f45a8159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:19.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.694+0000 7f45afd4f700 1 --2- 172.21.15.112:0/531914438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45a81437c0 0x7f45a80ca4b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:19.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.694+0000 7f45afd4f700 1 -- 172.21.15.112:0/531914438 >> 172.21.15.112:0/531914438 conn(0x7f45a80b1e90 msgr2=0x7f45a80b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:19.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.694+0000 7f45afd4f700 1 -- 172.21.15.112:0/531914438 shutdown_connections 2023-08-27T11:19:19.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.694+0000 7f45afd4f700 1 -- 172.21.15.112:0/531914438 wait complete. 2023-08-27T11:19:19.709 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:19:19.710 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:19:19.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.830+0000 7f9d75b4b700 1 Processor -- start 2023-08-27T11:19:19.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.830+0000 7f9d75b4b700 1 -- start start 2023-08-27T11:19:19.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d75b4b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d700b17e0 0x7f9d700cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:19.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d75b4b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9d70161ce0 0x7f9d7006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:19.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d75b4b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d70161850 con 0x7f9d700b1400 2023-08-27T11:19:19.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d75b4b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d700cfc10 con 0x7f9d70161ce0 2023-08-27T11:19:19.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d75b4b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70161200 con 0x7f9d700b17e0 2023-08-27T11:19:19.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d6f7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9d700b1400 0x7f9d7006e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53368/0 (socket says 172.21.15.112:53368) 2023-08-27T11:19:19.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d6f7fe700 1 -- 172.21.15.112:0/3547210299 learned_addr learned my addr 172.21.15.112:0/3547210299 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:19.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d6effd700 1 --2- 172.21.15.112:0/3547210299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d700b17e0 0x7f9d700cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:19.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d6ffff700 1 --2- 172.21.15.112:0/3547210299 >> v2:172.21.15.138:3300/0 conn(0x7f9d70161ce0 0x7f9d7006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:19.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707316770 0 0) 0x7f9d70161850 con 0x7f9d700b1400 2023-08-27T11:19:19.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d5c0035b0 con 0x7f9d700b1400 2023-08-27T11:19:19.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1641461806 0 0) 0x7f9d5c0035b0 con 0x7f9d700b1400 2023-08-27T11:19:19.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d700b17e0 msgr2=0x7f9d700cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:19.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d6e7fc700 1 --2- 172.21.15.112:0/3547210299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d700b17e0 0x7f9d700cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:19.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 >> v2:172.21.15.138:3300/0 conn(0x7f9d70161ce0 msgr2=0x7f9d7006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:19.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.831+0000 7f9d6e7fc700 1 --2- 172.21.15.112:0/3547210299 >> v2:172.21.15.138:3300/0 conn(0x7f9d70161ce0 0x7f9d7006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:19.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d701614e0 con 0x7f9d700b1400 2023-08-27T11:19:19.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6ffff700 1 --2- 172.21.15.112:0/3547210299 >> v2:172.21.15.138:3300/0 conn(0x7f9d70161ce0 0x7f9d7006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:19:19.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f9d58001670 con 0x7f9d700b1400 2023-08-27T11:19:19.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 >> v1:172.21.15.112:6789/0 conn(0x7f9d700b1400 legacy=0x7f9d7006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:19.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6e7fc700 1 --2- 172.21.15.112:0/3547210299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d5c003a00 0x7f9d5c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:19.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6e7fc700 1 --2- 172.21.15.112:0/3547210299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d5c004850 0x7f9d5c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:19.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6e7fc700 1 --2- 172.21.15.112:0/3547210299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d700b17e0 0x7f9d5c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:19.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700cfc10 con 0x7f9d5c004850 2023-08-27T11:19:19.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70161200 con 0x7f9d5c003a00 2023-08-27T11:19:19.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c00ba10 con 0x7f9d700b17e0 2023-08-27T11:19:19.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6effd700 1 --2- 172.21.15.112:0/3547210299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d5c004850 0x7f9d5c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:19.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6f7fe700 1 --2- 172.21.15.112:0/3547210299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d5c003a00 0x7f9d5c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:19.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6ffff700 1 --2- 172.21.15.112:0/3547210299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d700b17e0 0x7f9d5c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:19.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6effd700 1 -- 172.21.15.112:0/3547210299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d700b17e0 msgr2=0x7f9d5c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:19.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6effd700 1 --2- 172.21.15.112:0/3547210299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d700b17e0 0x7f9d5c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:19.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6effd700 1 -- 172.21.15.112:0/3547210299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d5c003a00 msgr2=0x7f9d5c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:19.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6effd700 1 --2- 172.21.15.112:0/3547210299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d5c003a00 0x7f9d5c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:19.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6effd700 1 -- 172.21.15.112:0/3547210299 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c0035b0 con 0x7f9d5c004850 2023-08-27T11:19:19.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6effd700 1 --2- 172.21.15.112:0/3547210299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d5c004850 0x7f9d5c004cd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009b90 tx=0x7f9d5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31117fd7a37af06c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:19.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000e6c0 con 0x7f9d5c004850 2023-08-27T11:19:19.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6000e820 con 0x7f9d5c004850 2023-08-27T11:19:19.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.832+0000 7f9d6e7fc700 1 -- 172.21.15.112:0/3547210299 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000eb90 con 0x7f9d5c004850 2023-08-27T11:19:19.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.833+0000 7f9d75b4b700 1 -- 172.21.15.112:0/3547210299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d5c004850 msgr2=0x7f9d5c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:19.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.833+0000 7f9d75b4b700 1 --2- 172.21.15.112:0/3547210299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d5c004850 0x7f9d5c004cd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009b90 tx=0x7f9d5c005880 comp rx=0 tx=0).stop 2023-08-27T11:19:19.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.833+0000 7f9d6ffff700 1 -- 172.21.15.112:0/3547210299 reap_dead start 2023-08-27T11:19:19.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.833+0000 7f9d75b4b700 1 -- 172.21.15.112:0/3547210299 shutdown_connections 2023-08-27T11:19:19.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.833+0000 7f9d75b4b700 1 -- 172.21.15.112:0/3547210299 >> 172.21.15.112:0/3547210299 conn(0x7f9d700d0000 msgr2=0x7f9d700d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:19.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.833+0000 7f9d75b4b700 1 -- 172.21.15.112:0/3547210299 shutdown_connections 2023-08-27T11:19:19.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.833+0000 7f9d75b4b700 1 -- 172.21.15.112:0/3547210299 wait complete. 2023-08-27T11:19:19.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.834+0000 7f9d75b4b700 1 Processor -- start 2023-08-27T11:19:19.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.834+0000 7f9d75b4b700 1 -- start start 2023-08-27T11:19:19.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d75b4b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d5c004850 0x7f9d700c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:19.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d75b4b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d70152900 0x7f9d701414f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:19.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d75b4b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d700c90f0 0x7f9d700c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:19.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d75b4b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70155640 con 0x7f9d700c90f0 2023-08-27T11:19:19.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d75b4b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70151910 con 0x7f9d5c004850 2023-08-27T11:19:19.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d75b4b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70072260 con 0x7f9d70152900 2023-08-27T11:19:19.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d6ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d700c90f0 0x7f9d700c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:19.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d6ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d700c90f0 0x7f9d700c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59002/0 (socket says 172.21.15.112:59002) 2023-08-27T11:19:19.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d6ffff700 1 -- 172.21.15.112:0/2134489158 learned_addr learned my addr 172.21.15.112:0/2134489158 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:19.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d6effd700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d70152900 0x7f9d701414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:19.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d6f7fe700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d5c004850 0x7f9d700c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d6ffff700 1 -- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d70152900 msgr2=0x7f9d701414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d6ffff700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d70152900 0x7f9d701414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d6ffff700 1 -- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d5c004850 msgr2=0x7f9d700c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d6ffff700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d5c004850 0x7f9d700c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d6ffff700 1 -- 172.21.15.112:0/2134489158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d700c9ab0 con 0x7f9d700c90f0 2023-08-27T11:19:19.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.835+0000 7f9d6effd700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d70152900 0x7f9d701414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:19.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.836+0000 7f9d6ffff700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d700c90f0 0x7f9d700c9570 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9d7006c130 tx=0x7f9d6800f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:19.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.836+0000 7f9d67fff700 1 -- 172.21.15.112:0/2134489158 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6801c070 con 0x7f9d700c90f0 2023-08-27T11:19:19.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.836+0000 7f9d67fff700 1 -- 172.21.15.112:0/2134489158 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6800f040 con 0x7f9d700c90f0 2023-08-27T11:19:19.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.836+0000 7f9d75b4b700 1 -- 172.21.15.112:0/2134489158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d701558b0 con 0x7f9d700c90f0 2023-08-27T11:19:19.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.836+0000 7f9d67fff700 1 -- 172.21.15.112:0/2134489158 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6800e040 con 0x7f9d700c90f0 2023-08-27T11:19:19.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.836+0000 7f9d75b4b700 1 -- 172.21.15.112:0/2134489158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d70155b90 con 0x7f9d700c90f0 2023-08-27T11:19:19.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.837+0000 7f9d75b4b700 1 -- 172.21.15.112:0/2134489158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d70067a20 con 0x7f9d700c90f0 2023-08-27T11:19:19.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.837+0000 7f9d67fff700 1 -- 172.21.15.112:0/2134489158 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f9d6800fd60 con 0x7f9d700c90f0 2023-08-27T11:19:19.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.837+0000 7f9d67fff700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d5006a170 0x7f9d5006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:19.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.837+0000 7f9d67fff700 1 -- 172.21.15.112:0/2134489158 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9d68088490 con 0x7f9d700c90f0 2023-08-27T11:19:19.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.838+0000 7f9d6f7fe700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d5006a170 0x7f9d5006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:19.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.838+0000 7f9d6f7fe700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d5006a170 0x7f9d5006c630 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9d580015d0 tx=0x7f9d58007980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:19.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:19.842+0000 7f9d67fff700 1 -- 172.21.15.112:0/2134489158 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9d68057510 con 0x7f9d700c90f0 2023-08-27T11:19:20.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.072+0000 7f9d75b4b700 1 -- 172.21.15.112:0/2134489158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d7006db60 con 0x7f9d700c90f0 2023-08-27T11:19:20.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.073+0000 7f9d67fff700 1 -- 172.21.15.112:0/2134489158 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4886 (secure 0 0 0) 0x7f9d680570a0 con 0x7f9d700c90f0 2023-08-27T11:19:20.075 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:20.075 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2906075444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2906075444},{"type":"v1","addr":"172.21.15.138:6836","nonce":2906075444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5761,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1781004436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1781004436},{"type":"v1","addr":"172.21.15.138:6837","nonce":1781004436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1573349448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1573349448},{"type":"v1","addr":"172.21.15.112:6834","nonce":1573349448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:19:16.064335+0000","modified":"2023-08-27T11:19:19.690233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/174707415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":174707415},{"type":"v1","addr":"172.21.15.112:6835","nonce":174707415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-27T11:19:20.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.075+0000 7f9d75b4b700 1 -- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d5006a170 msgr2=0x7f9d5006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.075+0000 7f9d75b4b700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d5006a170 0x7f9d5006c630 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9d580015d0 tx=0x7f9d58007980 comp rx=0 tx=0).stop 2023-08-27T11:19:20.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.075+0000 7f9d75b4b700 1 -- 172.21.15.112:0/2134489158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d700c90f0 msgr2=0x7f9d700c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.075+0000 7f9d75b4b700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d700c90f0 0x7f9d700c9570 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9d7006c130 tx=0x7f9d6800f490 comp rx=0 tx=0).stop 2023-08-27T11:19:20.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.075+0000 7f9d75b4b700 1 -- 172.21.15.112:0/2134489158 shutdown_connections 2023-08-27T11:19:20.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.075+0000 7f9d75b4b700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d5006a170 0x7f9d5006c630 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.075+0000 7f9d75b4b700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d700c90f0 0x7f9d700c9570 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.075+0000 7f9d75b4b700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d5c004850 0x7f9d700c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.075+0000 7f9d75b4b700 1 --2- 172.21.15.112:0/2134489158 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d70152900 0x7f9d701414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.075+0000 7f9d75b4b700 1 -- 172.21.15.112:0/2134489158 >> 172.21.15.112:0/2134489158 conn(0x7f9d700d0000 msgr2=0x7f9d700c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:20.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.075+0000 7f9d75b4b700 1 -- 172.21.15.112:0/2134489158 shutdown_connections 2023-08-27T11:19:20.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.075+0000 7f9d75b4b700 1 -- 172.21.15.112:0/2134489158 wait complete. 2023-08-27T11:19:20.079 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 39 2023-08-27T11:19:20.088 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:19:20.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.207+0000 7f0959e37700 1 Processor -- start 2023-08-27T11:19:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.207+0000 7f0959e37700 1 -- start start 2023-08-27T11:19:20.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0959e37700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f095415a120 0x7f09540b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0959e37700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f095406d5b0 0x7f095406da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0959e37700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f09540b3440 0x7f09540b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0959e37700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095406df50 con 0x7f095406d5b0 2023-08-27T11:19:20.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0959e37700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095415b540 con 0x7f09540b3440 2023-08-27T11:19:20.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0959e37700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09540af830 con 0x7f095415a120 2023-08-27T11:19:20.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0952ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f095406d5b0 0x7f095406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0952ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f095406d5b0 0x7f095406da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59006/0 (socket says 172.21.15.112:59006) 2023-08-27T11:19:20.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0952ffd700 1 -- 172.21.15.112:0/906824069 learned_addr learned my addr 172.21.15.112:0/906824069 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:20.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f09537fe700 1 --2- 172.21.15.112:0/906824069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f095415a120 0x7f09540b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0953fff700 1 --2- 172.21.15.112:0/906824069 >> v2:172.21.15.138:3300/0 conn(0x7f09540b3440 0x7f09540b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0952ffd700 1 -- 172.21.15.112:0/906824069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f095415a120 msgr2=0x7f09540b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:20.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0952ffd700 1 --2- 172.21.15.112:0/906824069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f095415a120 0x7f09540b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0952ffd700 1 -- 172.21.15.112:0/906824069 >> v2:172.21.15.138:3300/0 conn(0x7f09540b3440 msgr2=0x7f09540b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:20.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0952ffd700 1 --2- 172.21.15.112:0/906824069 >> v2:172.21.15.138:3300/0 conn(0x7f09540b3440 0x7f09540b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.208+0000 7f0952ffd700 1 -- 172.21.15.112:0/906824069 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09540afb10 con 0x7f095406d5b0 2023-08-27T11:19:20.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.209+0000 7f0952ffd700 1 --2- 172.21.15.112:0/906824069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095406d5b0 0x7f095406da10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f09540cfb10 tx=0x7f0944009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69227ad708d53b3c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:20.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.209+0000 7f09527fc700 1 -- 172.21.15.112:0/906824069 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094400b870 con 0x7f095406d5b0 2023-08-27T11:19:20.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.209+0000 7f09527fc700 1 -- 172.21.15.112:0/906824069 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094400b9d0 con 0x7f095406d5b0 2023-08-27T11:19:20.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.209+0000 7f09527fc700 1 -- 172.21.15.112:0/906824069 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094400bd60 con 0x7f095406d5b0 2023-08-27T11:19:20.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.209+0000 7f0959e37700 1 -- 172.21.15.112:0/906824069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095406d5b0 msgr2=0x7f095406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.209+0000 7f0959e37700 1 --2- 172.21.15.112:0/906824069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095406d5b0 0x7f095406da10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f09540cfb10 tx=0x7f0944009bd0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.209+0000 7f0959e37700 1 -- 172.21.15.112:0/906824069 shutdown_connections 2023-08-27T11:19:20.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.209+0000 7f0959e37700 1 --2- 172.21.15.112:0/906824069 >> v2:172.21.15.138:3300/0 conn(0x7f09540b3440 0x7f09540b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.209+0000 7f0959e37700 1 --2- 172.21.15.112:0/906824069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f095415a120 0x7f09540b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.209+0000 7f0959e37700 1 --2- 172.21.15.112:0/906824069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095406d5b0 0x7f095406da10 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.209+0000 7f0959e37700 1 -- 172.21.15.112:0/906824069 >> 172.21.15.112:0/906824069 conn(0x7f09540cff80 msgr2=0x7f09540d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:20.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.209+0000 7f0959e37700 1 -- 172.21.15.112:0/906824069 shutdown_connections 2023-08-27T11:19:20.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.209+0000 7f0959e37700 1 -- 172.21.15.112:0/906824069 wait complete. 2023-08-27T11:19:20.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.210+0000 7f0959e37700 1 Processor -- start 2023-08-27T11:19:20.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.211+0000 7f0959e37700 1 -- start start 2023-08-27T11:19:20.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.211+0000 7f0959e37700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f095406d5b0 0x7f0954157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.211+0000 7f0959e37700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540b3440 0x7f0954154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.211+0000 7f0959e37700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095415a120 0x7f09541485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.211+0000 7f0959e37700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540724b0 con 0x7f095415a120 2023-08-27T11:19:20.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.211+0000 7f0959e37700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540721d0 con 0x7f09540b3440 2023-08-27T11:19:20.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.211+0000 7f0959e37700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09540cb800 con 0x7f095406d5b0 2023-08-27T11:19:20.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f0953fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095415a120 0x7f09541485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f0953fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095415a120 0x7f09541485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59012/0 (socket says 172.21.15.112:59012) 2023-08-27T11:19:20.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f0953fff700 1 -- 172.21.15.112:0/113665103 learned_addr learned my addr 172.21.15.112:0/113665103 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:20.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f09537fe700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f095406d5b0 0x7f0954157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f0952ffd700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540b3440 0x7f0954154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f0953fff700 1 -- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f095406d5b0 msgr2=0x7f0954157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f0953fff700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f095406d5b0 0x7f0954157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f0953fff700 1 -- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540b3440 msgr2=0x7f0954154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f0953fff700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540b3440 0x7f0954154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f0953fff700 1 -- 172.21.15.112:0/113665103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0944009550 con 0x7f095415a120 2023-08-27T11:19:20.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f09537fe700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f095406d5b0 0x7f0954157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:20.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f0952ffd700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540b3440 0x7f0954154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:20.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f0953fff700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095415a120 0x7f09541485e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f094c007e40 tx=0x7f094c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:20.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f093bfff700 1 -- 172.21.15.112:0/113665103 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00eea0 con 0x7f095415a120 2023-08-27T11:19:20.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f093bfff700 1 -- 172.21.15.112:0/113665103 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c014420 con 0x7f095415a120 2023-08-27T11:19:20.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f0959e37700 1 -- 172.21.15.112:0/113665103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09540cbae0 con 0x7f095415a120 2023-08-27T11:19:20.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f093bfff700 1 -- 172.21.15.112:0/113665103 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0147b0 con 0x7f095415a120 2023-08-27T11:19:20.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.212+0000 7f0959e37700 1 -- 172.21.15.112:0/113665103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0954157c50 con 0x7f095415a120 2023-08-27T11:19:20.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.213+0000 7f0959e37700 1 -- 172.21.15.112:0/113665103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09540c9f10 con 0x7f095415a120 2023-08-27T11:19:20.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.213+0000 7f093bfff700 1 -- 172.21.15.112:0/113665103 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f094c014910 con 0x7f095415a120 2023-08-27T11:19:20.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.214+0000 7f093bfff700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f094006a170 0x7f094006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.214+0000 7f093bfff700 1 -- 172.21.15.112:0/113665103 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f094c088d20 con 0x7f095415a120 2023-08-27T11:19:20.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.214+0000 7f09537fe700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f094006a170 0x7f094006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.214+0000 7f09537fe700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f094006a170 0x7f094006c630 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0954157360 tx=0x7f0948008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:20.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.218+0000 7f093bfff700 1 -- 172.21.15.112:0/113665103 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f094c0544b0 con 0x7f095415a120 2023-08-27T11:19:20.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.408+0000 7f0959e37700 1 -- 172.21.15.112:0/113665103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0954067a20 con 0x7f095415a120 2023-08-27T11:19:20.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.410+0000 7f093bfff700 1 -- 172.21.15.112:0/113665103 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18320 (secure 0 0 0) 0x7f094c057ad0 con 0x7f095415a120 2023-08-27T11:19:20.411 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:20.412 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":65,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:19:17.239518+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:19:14.179813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-27T11:19:15.021524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:19:20.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.412+0000 7f0959e37700 1 -- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f094006a170 msgr2=0x7f094006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.412+0000 7f0959e37700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f094006a170 0x7f094006c630 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0954157360 tx=0x7f0948008040 comp rx=0 tx=0).stop 2023-08-27T11:19:20.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.412+0000 7f0959e37700 1 -- 172.21.15.112:0/113665103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095415a120 msgr2=0x7f09541485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.412+0000 7f0959e37700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095415a120 0x7f09541485e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f094c007e40 tx=0x7f094c00e3c0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.412+0000 7f0959e37700 1 -- 172.21.15.112:0/113665103 shutdown_connections 2023-08-27T11:19:20.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.412+0000 7f0959e37700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f094006a170 0x7f094006c630 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.412+0000 7f0959e37700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095415a120 0x7f09541485e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.412+0000 7f0959e37700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f095406d5b0 0x7f0954157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.412+0000 7f0959e37700 1 --2- 172.21.15.112:0/113665103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540b3440 0x7f0954154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.412+0000 7f0959e37700 1 -- 172.21.15.112:0/113665103 >> 172.21.15.112:0/113665103 conn(0x7f09540cff80 msgr2=0x7f0954155f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:20.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.412+0000 7f0959e37700 1 -- 172.21.15.112:0/113665103 shutdown_connections 2023-08-27T11:19:20.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.412+0000 7f0959e37700 1 -- 172.21.15.112:0/113665103 wait complete. 2023-08-27T11:19:20.425 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:19:20.426 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:19:20.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.545+0000 7f4e5bc77700 1 Processor -- start 2023-08-27T11:19:20.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.545+0000 7f4e5bc77700 1 -- start start 2023-08-27T11:19:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e5bc77700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e54141b60 0x7f4e540cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e5bc77700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4e54144a40 0x7f4e540cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e5bc77700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4e540cc8e0 0x7f4e5406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e5bc77700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e540ccd50 con 0x7f4e54144a40 2023-08-27T11:19:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e5bc77700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5415f3d0 con 0x7f4e540cc8e0 2023-08-27T11:19:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e5bc77700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5415f6b0 con 0x7f4e54141b60 2023-08-27T11:19:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e59212700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4e54144a40 0x7f4e540cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e59212700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4e54144a40 0x7f4e540cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59020/0 (socket says 172.21.15.112:59020) 2023-08-27T11:19:20.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e59212700 1 -- 172.21.15.112:0/645697145 learned_addr learned my addr 172.21.15.112:0/645697145 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:20.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e5a214700 1 --2- 172.21.15.112:0/645697145 >> v2:172.21.15.138:3300/0 conn(0x7f4e540cc8e0 0x7f4e5406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e59a13700 1 --2- 172.21.15.112:0/645697145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e54141b60 0x7f4e540cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e59212700 1 -- 172.21.15.112:0/645697145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e54141b60 msgr2=0x7f4e540cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:20.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e59212700 1 --2- 172.21.15.112:0/645697145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e54141b60 0x7f4e540cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e59212700 1 -- 172.21.15.112:0/645697145 >> v2:172.21.15.138:3300/0 conn(0x7f4e540cc8e0 msgr2=0x7f4e5406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:20.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e59212700 1 --2- 172.21.15.112:0/645697145 >> v2:172.21.15.138:3300/0 conn(0x7f4e540cc8e0 0x7f4e5406bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e59212700 1 -- 172.21.15.112:0/645697145 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5415f990 con 0x7f4e54144a40 2023-08-27T11:19:20.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e59a13700 1 --2- 172.21.15.112:0/645697145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e54141b60 0x7f4e540cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e59212700 1 --2- 172.21.15.112:0/645697145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e54144a40 0x7f4e540cc3a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4e540cbf40 tx=0x7f4e44009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f308954da4e59c54 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.546+0000 7f4e58a11700 1 -- 172.21.15.112:0/645697145 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4400b870 con 0x7f4e54144a40 2023-08-27T11:19:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.547+0000 7f4e58a11700 1 -- 172.21.15.112:0/645697145 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4400b9d0 con 0x7f4e54144a40 2023-08-27T11:19:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.547+0000 7f4e58a11700 1 -- 172.21.15.112:0/645697145 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4400bd60 con 0x7f4e54144a40 2023-08-27T11:19:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.547+0000 7f4e5bc77700 1 -- 172.21.15.112:0/645697145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e54144a40 msgr2=0x7f4e540cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.547+0000 7f4e5bc77700 1 --2- 172.21.15.112:0/645697145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e54144a40 0x7f4e540cc3a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4e540cbf40 tx=0x7f4e44009bd0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.547+0000 7f4e5bc77700 1 -- 172.21.15.112:0/645697145 shutdown_connections 2023-08-27T11:19:20.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.547+0000 7f4e5bc77700 1 --2- 172.21.15.112:0/645697145 >> v2:172.21.15.138:3300/0 conn(0x7f4e540cc8e0 0x7f4e5406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.547+0000 7f4e5bc77700 1 --2- 172.21.15.112:0/645697145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e54141b60 0x7f4e540cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.547+0000 7f4e5bc77700 1 --2- 172.21.15.112:0/645697145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e54144a40 0x7f4e540cc3a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.547+0000 7f4e5bc77700 1 -- 172.21.15.112:0/645697145 >> 172.21.15.112:0/645697145 conn(0x7f4e54066010 msgr2=0x7f4e540663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:20.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.547+0000 7f4e5bc77700 1 -- 172.21.15.112:0/645697145 shutdown_connections 2023-08-27T11:19:20.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.547+0000 7f4e5bc77700 1 -- 172.21.15.112:0/645697145 wait complete. 2023-08-27T11:19:20.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.548+0000 7f4e5bc77700 1 Processor -- start 2023-08-27T11:19:20.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.548+0000 7f4e5bc77700 1 -- start start 2023-08-27T11:19:20.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e5bc77700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e540cc8e0 0x7f4e5413fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e5bc77700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e54141b60 0x7f4e540712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e5bc77700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e54144a40 0x7f4e5415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e5bc77700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e540724a0 con 0x7f4e540cc8e0 2023-08-27T11:19:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e5bc77700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5415cb80 con 0x7f4e54144a40 2023-08-27T11:19:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e5bc77700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5415bf70 con 0x7f4e54141b60 2023-08-27T11:19:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e59a13700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e540cc8e0 0x7f4e5413fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e59a13700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e540cc8e0 0x7f4e5413fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59024/0 (socket says 172.21.15.112:59024) 2023-08-27T11:19:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e59a13700 1 -- 172.21.15.112:0/3791885728 learned_addr learned my addr 172.21.15.112:0/3791885728 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e5a214700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e54144a40 0x7f4e5415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e59212700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e54141b60 0x7f4e540712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e59a13700 1 -- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e54141b60 msgr2=0x7f4e540712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e59a13700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e54141b60 0x7f4e540712b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e59a13700 1 -- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e54144a40 msgr2=0x7f4e5415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e59a13700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e54144a40 0x7f4e5415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.549+0000 7f4e59a13700 1 -- 172.21.15.112:0/3791885728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e44009550 con 0x7f4e540cc8e0 2023-08-27T11:19:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.550+0000 7f4e59a13700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e540cc8e0 0x7f4e5413fa10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000c750 tx=0x7f4e5000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:20.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.550+0000 7f4e4a7fc700 1 -- 172.21.15.112:0/3791885728 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50011800 con 0x7f4e540cc8e0 2023-08-27T11:19:20.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.550+0000 7f4e4a7fc700 1 -- 172.21.15.112:0/3791885728 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e50011960 con 0x7f4e540cc8e0 2023-08-27T11:19:20.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.550+0000 7f4e4a7fc700 1 -- 172.21.15.112:0/3791885728 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50011cd0 con 0x7f4e540cc8e0 2023-08-27T11:19:20.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.550+0000 7f4e5bc77700 1 -- 172.21.15.112:0/3791885728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5415c250 con 0x7f4e540cc8e0 2023-08-27T11:19:20.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.550+0000 7f4e5bc77700 1 -- 172.21.15.112:0/3791885728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5415c810 con 0x7f4e540cc8e0 2023-08-27T11:19:20.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.551+0000 7f4e5bc77700 1 -- 172.21.15.112:0/3791885728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e54067a20 con 0x7f4e540cc8e0 2023-08-27T11:19:20.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.551+0000 7f4e4a7fc700 1 -- 172.21.15.112:0/3791885728 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f4e50021350 con 0x7f4e540cc8e0 2023-08-27T11:19:20.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.552+0000 7f4e4a7fc700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e3806a170 0x7f4e3806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.552+0000 7f4e4a7fc700 1 -- 172.21.15.112:0/3791885728 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e500885a0 con 0x7f4e540cc8e0 2023-08-27T11:19:20.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.552+0000 7f4e59212700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e3806a170 0x7f4e3806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.552+0000 7f4e59212700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e3806a170 0x7f4e3806c630 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009c10 tx=0x7f4e4401d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:20.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.556+0000 7f4e4a7fc700 1 -- 172.21.15.112:0/3791885728 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4e50057590 con 0x7f4e540cc8e0 2023-08-27T11:19:20.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.741+0000 7f4e5bc77700 1 -- 172.21.15.112:0/3791885728 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e5415ff50 con 0x7f4e3806a170 2023-08-27T11:19:20.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.751+0000 7f4e4a7fc700 1 -- 172.21.15.112:0/3791885728 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312211 (secure 0 0 0) 0x7f4e4400bcc0 con 0x7f4e3806a170 2023-08-27T11:19:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:20.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.754+0000 7f4e5bc77700 1 -- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e3806a170 msgr2=0x7f4e3806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.754+0000 7f4e5bc77700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e3806a170 0x7f4e3806c630 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009c10 tx=0x7f4e4401d040 comp rx=0 tx=0).stop 2023-08-27T11:19:20.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.754+0000 7f4e5bc77700 1 -- 172.21.15.112:0/3791885728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e540cc8e0 msgr2=0x7f4e5413fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.754+0000 7f4e5bc77700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e540cc8e0 0x7f4e5413fa10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000c750 tx=0x7f4e5000ca60 comp rx=0 tx=0).stop 2023-08-27T11:19:20.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.754+0000 7f4e5bc77700 1 -- 172.21.15.112:0/3791885728 shutdown_connections 2023-08-27T11:19:20.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.754+0000 7f4e5bc77700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e3806a170 0x7f4e3806c630 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.754+0000 7f4e5bc77700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e54144a40 0x7f4e5415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.754+0000 7f4e5bc77700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e540cc8e0 0x7f4e5413fa10 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.754+0000 7f4e5bc77700 1 --2- 172.21.15.112:0/3791885728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e54141b60 0x7f4e540712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.754+0000 7f4e5bc77700 1 -- 172.21.15.112:0/3791885728 >> 172.21.15.112:0/3791885728 conn(0x7f4e54066010 msgr2=0x7f4e54140b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:20.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.754+0000 7f4e5bc77700 1 -- 172.21.15.112:0/3791885728 shutdown_connections 2023-08-27T11:19:20.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.754+0000 7f4e5bc77700 1 -- 172.21.15.112:0/3791885728 wait complete. 2023-08-27T11:19:20.762 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:19:20.768 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2023-08-27T11:19:20.256019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223464,"kb_used_data":7264,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754751256,"statfs":{"total":773094113280,"available":772865286144,"internally_reserved":0,"allocated":7438336,"data_stored":4003275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.593800"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262692+0000","last_change":"2023-08-27T11:19:16.259692+0000","last_active":"2023-08-27T11:19:16.262692+0000","last_peered":"2023-08-27T11:19:16.262692+0000","last_clean":"2023-08-27T11:19:16.262692+0000","last_became_active":"2023-08-27T11:19:16.259386+0000","last_became_peered":"2023-08-27T11:19:16.259386+0000","last_unstale":"2023-08-27T11:19:16.262692+0000","last_undegraded":"2023-08-27T11:19:16.262692+0000","last_fullsized":"2023-08-27T11:19:16.262692+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:53:25.637980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248443+0000","last_change":"2023-08-27T11:19:16.260287+0000","last_active":"2023-08-27T11:19:17.248443+0000","last_peered":"2023-08-27T11:19:17.248443+0000","last_clean":"2023-08-27T11:19:17.248443+0000","last_became_active":"2023-08-27T11:19:16.260031+0000","last_became_peered":"2023-08-27T11:19:16.260031+0000","last_unstale":"2023-08-27T11:19:17.248443+0000","last_undegraded":"2023-08-27T11:19:17.248443+0000","last_fullsized":"2023-08-27T11:19:17.248443+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:14:23.586906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264753+0000","last_change":"2023-08-27T11:19:16.260482+0000","last_active":"2023-08-27T11:19:16.264753+0000","last_peered":"2023-08-27T11:19:16.264753+0000","last_clean":"2023-08-27T11:19:16.264753+0000","last_became_active":"2023-08-27T11:19:16.260219+0000","last_became_peered":"2023-08-27T11:19:16.260219+0000","last_unstale":"2023-08-27T11:19:16.264753+0000","last_undegraded":"2023-08-27T11:19:16.264753+0000","last_fullsized":"2023-08-27T11:19:16.264753+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:03:03.572878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.258757+0000","last_change":"2023-08-27T11:19:16.255592+0000","last_active":"2023-08-27T11:19:16.258757+0000","last_peered":"2023-08-27T11:19:16.258757+0000","last_clean":"2023-08-27T11:19:16.258757+0000","last_became_active":"2023-08-27T11:19:16.254743+0000","last_became_peered":"2023-08-27T11:19:16.254743+0000","last_unstale":"2023-08-27T11:19:16.258757+0000","last_undegraded":"2023-08-27T11:19:16.258757+0000","last_fullsized":"2023-08-27T11:19:16.258757+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:30:25.750411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.265355+0000","last_change":"2023-08-27T11:19:16.263154+0000","last_active":"2023-08-27T11:19:16.265355+0000","last_peered":"2023-08-27T11:19:16.265355+0000","last_clean":"2023-08-27T11:19:16.265355+0000","last_became_active":"2023-08-27T11:19:16.259734+0000","last_became_peered":"2023-08-27T11:19:16.259734+0000","last_unstale":"2023-08-27T11:19:16.265355+0000","last_undegraded":"2023-08-27T11:19:16.265355+0000","last_fullsized":"2023-08-27T11:19:16.265355+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:40:49.102138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264017+0000","last_change":"2023-08-27T11:19:16.259340+0000","last_active":"2023-08-27T11:19:16.264017+0000","last_peered":"2023-08-27T11:19:16.264017+0000","last_clean":"2023-08-27T11:19:16.264017+0000","last_became_active":"2023-08-27T11:19:16.256467+0000","last_became_peered":"2023-08-27T11:19:16.256467+0000","last_unstale":"2023-08-27T11:19:16.264017+0000","last_undegraded":"2023-08-27T11:19:16.264017+0000","last_fullsized":"2023-08-27T11:19:16.264017+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:47:45.582187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.251958+0000","last_change":"2023-08-27T11:19:16.264271+0000","last_active":"2023-08-27T11:19:17.251958+0000","last_peered":"2023-08-27T11:19:17.251958+0000","last_clean":"2023-08-27T11:19:17.251958+0000","last_became_active":"2023-08-27T11:19:16.263971+0000","last_became_peered":"2023-08-27T11:19:16.263971+0000","last_unstale":"2023-08-27T11:19:17.251958+0000","last_undegraded":"2023-08-27T11:19:17.251958+0000","last_fullsized":"2023-08-27T11:19:17.251958+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:12:59.269333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.263447+0000","last_change":"2023-08-27T11:19:16.257924+0000","last_active":"2023-08-27T11:19:16.263447+0000","last_peered":"2023-08-27T11:19:16.263447+0000","last_clean":"2023-08-27T11:19:16.263447+0000","last_became_active":"2023-08-27T11:19:16.257571+0000","last_became_peered":"2023-08-27T11:19:16.257571+0000","last_unstale":"2023-08-27T11:19:16.263447+0000","last_undegraded":"2023-08-27T11:19:16.263447+0000","last_fullsized":"2023-08-27T11:19:16.263447+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:21.317637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252209+0000","last_change":"2023-08-27T11:19:16.250635+0000","last_active":"2023-08-27T11:19:17.252209+0000","last_peered":"2023-08-27T11:19:17.252209+0000","last_clean":"2023-08-27T11:19:17.252209+0000","last_became_active":"2023-08-27T11:19:16.250123+0000","last_became_peered":"2023-08-27T11:19:16.250123+0000","last_unstale":"2023-08-27T11:19:17.252209+0000","last_undegraded":"2023-08-27T11:19:17.252209+0000","last_fullsized":"2023-08-27T11:19:17.252209+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:31:31.717367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262921+0000","last_change":"2023-08-27T11:19:16.258355+0000","last_active":"2023-08-27T11:19:16.262921+0000","last_peered":"2023-08-27T11:19:16.262921+0000","last_clean":"2023-08-27T11:19:16.262921+0000","last_became_active":"2023-08-27T11:19:16.258025+0000","last_became_peered":"2023-08-27T11:19:16.258025+0000","last_unstale":"2023-08-27T11:19:16.262921+0000","last_undegraded":"2023-08-27T11:19:16.262921+0000","last_fullsized":"2023-08-27T11:19:16.262921+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:22:14.482419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248628+0000","last_change":"2023-08-27T11:19:16.260176+0000","last_active":"2023-08-27T11:19:17.248628+0000","last_peered":"2023-08-27T11:19:17.248628+0000","last_clean":"2023-08-27T11:19:17.248628+0000","last_became_active":"2023-08-27T11:19:16.259920+0000","last_became_peered":"2023-08-27T11:19:16.259920+0000","last_unstale":"2023-08-27T11:19:17.248628+0000","last_undegraded":"2023-08-27T11:19:17.248628+0000","last_fullsized":"2023-08-27T11:19:17.248628+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:21:13.100446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262978+0000","last_change":"2023-08-27T11:19:16.261417+0000","last_active":"2023-08-27T11:19:16.262978+0000","last_peered":"2023-08-27T11:19:16.262978+0000","last_clean":"2023-08-27T11:19:16.262978+0000","last_became_active":"2023-08-27T11:19:16.260964+0000","last_became_peered":"2023-08-27T11:19:16.260964+0000","last_unstale":"2023-08-27T11:19:16.262978+0000","last_undegraded":"2023-08-27T11:19:16.262978+0000","last_fullsized":"2023-08-27T11:19:16.262978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:53:18.080063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.263281+0000","last_change":"2023-08-27T11:19:16.256015+0000","last_active":"2023-08-27T11:19:16.263281+0000","last_peered":"2023-08-27T11:19:16.263281+0000","last_clean":"2023-08-27T11:19:16.263281+0000","last_became_active":"2023-08-27T11:19:16.255617+0000","last_became_peered":"2023-08-27T11:19:16.255617+0000","last_unstale":"2023-08-27T11:19:16.263281+0000","last_undegraded":"2023-08-27T11:19:16.263281+0000","last_fullsized":"2023-08-27T11:19:16.263281+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:16:54.974620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.263846+0000","last_change":"2023-08-27T11:19:16.257095+0000","last_active":"2023-08-27T11:19:16.263846+0000","last_peered":"2023-08-27T11:19:16.263846+0000","last_clean":"2023-08-27T11:19:16.263846+0000","last_became_active":"2023-08-27T11:19:16.256133+0000","last_became_peered":"2023-08-27T11:19:16.256133+0000","last_unstale":"2023-08-27T11:19:16.263846+0000","last_undegraded":"2023-08-27T11:19:16.263846+0000","last_fullsized":"2023-08-27T11:19:16.263846+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:30:25.750411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248142+0000","last_change":"2023-08-27T11:19:16.259002+0000","last_active":"2023-08-27T11:19:17.248142+0000","last_peered":"2023-08-27T11:19:17.248142+0000","last_clean":"2023-08-27T11:19:17.248142+0000","last_became_active":"2023-08-27T11:19:16.258740+0000","last_became_peered":"2023-08-27T11:19:16.258740+0000","last_unstale":"2023-08-27T11:19:17.248142+0000","last_undegraded":"2023-08-27T11:19:17.248142+0000","last_fullsized":"2023-08-27T11:19:17.248142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:45:44.045561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.257857+0000","last_change":"2023-08-27T11:19:16.253764+0000","last_active":"2023-08-27T11:19:16.257857+0000","last_peered":"2023-08-27T11:19:16.257857+0000","last_clean":"2023-08-27T11:19:16.257857+0000","last_became_active":"2023-08-27T11:19:16.253523+0000","last_became_peered":"2023-08-27T11:19:16.253523+0000","last_unstale":"2023-08-27T11:19:16.257857+0000","last_undegraded":"2023-08-27T11:19:16.257857+0000","last_fullsized":"2023-08-27T11:19:16.257857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:46:25.794832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.265363+0000","last_change":"2023-08-27T11:19:16.263188+0000","last_active":"2023-08-27T11:19:16.265363+0000","last_peered":"2023-08-27T11:19:16.265363+0000","last_clean":"2023-08-27T11:19:16.265363+0000","last_became_active":"2023-08-27T11:19:16.259623+0000","last_became_peered":"2023-08-27T11:19:16.259623+0000","last_unstale":"2023-08-27T11:19:16.265363+0000","last_undegraded":"2023-08-27T11:19:16.265363+0000","last_fullsized":"2023-08-27T11:19:16.265363+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:16.815337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.259764+0000","last_change":"2023-08-27T11:19:16.255571+0000","last_active":"2023-08-27T11:19:16.259764+0000","last_peered":"2023-08-27T11:19:16.259764+0000","last_clean":"2023-08-27T11:19:16.259764+0000","last_became_active":"2023-08-27T11:19:16.254779+0000","last_became_peered":"2023-08-27T11:19:16.254779+0000","last_unstale":"2023-08-27T11:19:16.259764+0000","last_undegraded":"2023-08-27T11:19:16.259764+0000","last_fullsized":"2023-08-27T11:19:16.259764+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:37:12.312663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262707+0000","last_change":"2023-08-27T11:19:16.257992+0000","last_active":"2023-08-27T11:19:16.262707+0000","last_peered":"2023-08-27T11:19:16.262707+0000","last_clean":"2023-08-27T11:19:16.262707+0000","last_became_active":"2023-08-27T11:19:16.257746+0000","last_became_peered":"2023-08-27T11:19:16.257746+0000","last_unstale":"2023-08-27T11:19:16.262707+0000","last_undegraded":"2023-08-27T11:19:16.262707+0000","last_fullsized":"2023-08-27T11:19:16.262707+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:41:02.020022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264185+0000","last_change":"2023-08-27T11:19:16.261640+0000","last_active":"2023-08-27T11:19:16.264185+0000","last_peered":"2023-08-27T11:19:16.264185+0000","last_clean":"2023-08-27T11:19:16.264185+0000","last_became_active":"2023-08-27T11:19:16.261355+0000","last_became_peered":"2023-08-27T11:19:16.261355+0000","last_unstale":"2023-08-27T11:19:16.264185+0000","last_undegraded":"2023-08-27T11:19:16.264185+0000","last_fullsized":"2023-08-27T11:19:16.264185+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:04.687161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252332+0000","last_change":"2023-08-27T11:19:16.255856+0000","last_active":"2023-08-27T11:19:17.252332+0000","last_peered":"2023-08-27T11:19:17.252332+0000","last_clean":"2023-08-27T11:19:17.252332+0000","last_became_active":"2023-08-27T11:19:16.255386+0000","last_became_peered":"2023-08-27T11:19:16.255386+0000","last_unstale":"2023-08-27T11:19:17.252332+0000","last_undegraded":"2023-08-27T11:19:17.252332+0000","last_fullsized":"2023-08-27T11:19:17.252332+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:31:18.071265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264159+0000","last_change":"2023-08-27T11:19:16.261655+0000","last_active":"2023-08-27T11:19:16.264159+0000","last_peered":"2023-08-27T11:19:16.264159+0000","last_clean":"2023-08-27T11:19:16.264159+0000","last_became_active":"2023-08-27T11:19:16.261418+0000","last_became_peered":"2023-08-27T11:19:16.261418+0000","last_unstale":"2023-08-27T11:19:16.264159+0000","last_undegraded":"2023-08-27T11:19:16.264159+0000","last_fullsized":"2023-08-27T11:19:16.264159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:57:04.356037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.251972+0000","last_change":"2023-08-27T11:19:16.253818+0000","last_active":"2023-08-27T11:19:17.251972+0000","last_peered":"2023-08-27T11:19:17.251972+0000","last_clean":"2023-08-27T11:19:17.251972+0000","last_became_active":"2023-08-27T11:19:16.253523+0000","last_became_peered":"2023-08-27T11:19:16.253523+0000","last_unstale":"2023-08-27T11:19:17.251972+0000","last_undegraded":"2023-08-27T11:19:17.251972+0000","last_fullsized":"2023-08-27T11:19:17.251972+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:04:27.242470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248381+0000","last_change":"2023-08-27T11:19:16.251666+0000","last_active":"2023-08-27T11:19:17.248381+0000","last_peered":"2023-08-27T11:19:17.248381+0000","last_clean":"2023-08-27T11:19:17.248381+0000","last_became_active":"2023-08-27T11:19:16.251405+0000","last_became_peered":"2023-08-27T11:19:16.251405+0000","last_unstale":"2023-08-27T11:19:17.248381+0000","last_undegraded":"2023-08-27T11:19:17.248381+0000","last_fullsized":"2023-08-27T11:19:17.248381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:29:19.506410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248372+0000","last_change":"2023-08-27T11:19:16.254142+0000","last_active":"2023-08-27T11:19:17.248372+0000","last_peered":"2023-08-27T11:19:17.248372+0000","last_clean":"2023-08-27T11:19:17.248372+0000","last_became_active":"2023-08-27T11:19:16.253894+0000","last_became_peered":"2023-08-27T11:19:16.253894+0000","last_unstale":"2023-08-27T11:19:17.248372+0000","last_undegraded":"2023-08-27T11:19:17.248372+0000","last_fullsized":"2023-08-27T11:19:17.248372+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:27:49.277713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262346+0000","last_change":"2023-08-27T11:19:16.257289+0000","last_active":"2023-08-27T11:19:16.262346+0000","last_peered":"2023-08-27T11:19:16.262346+0000","last_clean":"2023-08-27T11:19:16.262346+0000","last_became_active":"2023-08-27T11:19:16.257053+0000","last_became_peered":"2023-08-27T11:19:16.257053+0000","last_unstale":"2023-08-27T11:19:16.262346+0000","last_undegraded":"2023-08-27T11:19:16.262346+0000","last_fullsized":"2023-08-27T11:19:16.262346+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:54.446601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.257782+0000","last_change":"2023-08-27T11:19:16.253795+0000","last_active":"2023-08-27T11:19:16.257782+0000","last_peered":"2023-08-27T11:19:16.257782+0000","last_clean":"2023-08-27T11:19:16.257782+0000","last_became_active":"2023-08-27T11:19:16.253485+0000","last_became_peered":"2023-08-27T11:19:16.253485+0000","last_unstale":"2023-08-27T11:19:16.257782+0000","last_undegraded":"2023-08-27T11:19:16.257782+0000","last_fullsized":"2023-08-27T11:19:16.257782+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:03:41.841855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.263666+0000","last_change":"2023-08-27T11:19:16.256266+0000","last_active":"2023-08-27T11:19:16.263666+0000","last_peered":"2023-08-27T11:19:16.263666+0000","last_clean":"2023-08-27T11:19:16.263666+0000","last_became_active":"2023-08-27T11:19:16.256014+0000","last_became_peered":"2023-08-27T11:19:16.256014+0000","last_unstale":"2023-08-27T11:19:16.263666+0000","last_undegraded":"2023-08-27T11:19:16.263666+0000","last_fullsized":"2023-08-27T11:19:16.263666+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:18:12.235459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248315+0000","last_change":"2023-08-27T11:19:16.259027+0000","last_active":"2023-08-27T11:19:17.248315+0000","last_peered":"2023-08-27T11:19:17.248315+0000","last_clean":"2023-08-27T11:19:17.248315+0000","last_became_active":"2023-08-27T11:19:16.258748+0000","last_became_peered":"2023-08-27T11:19:16.258748+0000","last_unstale":"2023-08-27T11:19:17.248315+0000","last_undegraded":"2023-08-27T11:19:17.248315+0000","last_fullsized":"2023-08-27T11:19:17.248315+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:19:05.373592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262731+0000","last_change":"2023-08-27T11:19:16.260021+0000","last_active":"2023-08-27T11:19:16.262731+0000","last_peered":"2023-08-27T11:19:16.262731+0000","last_clean":"2023-08-27T11:19:16.262731+0000","last_became_active":"2023-08-27T11:19:16.259405+0000","last_became_peered":"2023-08-27T11:19:16.259405+0000","last_unstale":"2023-08-27T11:19:16.262731+0000","last_undegraded":"2023-08-27T11:19:16.262731+0000","last_fullsized":"2023-08-27T11:19:16.262731+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:32:50.423067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264786+0000","last_change":"2023-08-27T11:19:16.260535+0000","last_active":"2023-08-27T11:19:16.264786+0000","last_peered":"2023-08-27T11:19:16.264786+0000","last_clean":"2023-08-27T11:19:16.264786+0000","last_became_active":"2023-08-27T11:19:16.260207+0000","last_became_peered":"2023-08-27T11:19:16.260207+0000","last_unstale":"2023-08-27T11:19:16.264786+0000","last_undegraded":"2023-08-27T11:19:16.264786+0000","last_fullsized":"2023-08-27T11:19:16.264786+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:15:55.910012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.257874+0000","last_change":"2023-08-27T11:19:16.255856+0000","last_active":"2023-08-27T11:19:16.257874+0000","last_peered":"2023-08-27T11:19:16.257874+0000","last_clean":"2023-08-27T11:19:16.257874+0000","last_became_active":"2023-08-27T11:19:16.255580+0000","last_became_peered":"2023-08-27T11:19:16.255580+0000","last_unstale":"2023-08-27T11:19:16.257874+0000","last_undegraded":"2023-08-27T11:19:16.257874+0000","last_fullsized":"2023-08-27T11:19:16.257874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:34:49.226624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.256538+0000","last_change":"2023-08-27T11:19:16.254831+0000","last_active":"2023-08-27T11:19:16.256538+0000","last_peered":"2023-08-27T11:19:16.256538+0000","last_clean":"2023-08-27T11:19:16.256538+0000","last_became_active":"2023-08-27T11:19:16.254557+0000","last_became_peered":"2023-08-27T11:19:16.254557+0000","last_unstale":"2023-08-27T11:19:16.256538+0000","last_undegraded":"2023-08-27T11:19:16.256538+0000","last_fullsized":"2023-08-27T11:19:16.256538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:05:24.801073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"64'4","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.677440+0000","last_change":"2023-08-27T11:19:15.681714+0000","last_active":"2023-08-27T11:19:16.677440+0000","last_peered":"2023-08-27T11:19:16.677440+0000","last_clean":"2023-08-27T11:19:16.677440+0000","last_became_active":"2023-08-27T11:19:15.681225+0000","last_became_peered":"2023-08-27T11:19:15.681225+0000","last_unstale":"2023-08-27T11:19:16.677440+0000","last_undegraded":"2023-08-27T11:19:16.677440+0000","last_fullsized":"2023-08-27T11:19:16.677440+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:08:18.126251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248267+0000","last_change":"2023-08-27T11:19:16.258259+0000","last_active":"2023-08-27T11:19:17.248267+0000","last_peered":"2023-08-27T11:19:17.248267+0000","last_clean":"2023-08-27T11:19:17.248267+0000","last_became_active":"2023-08-27T11:19:16.257986+0000","last_became_peered":"2023-08-27T11:19:16.257986+0000","last_unstale":"2023-08-27T11:19:17.248267+0000","last_undegraded":"2023-08-27T11:19:17.248267+0000","last_fullsized":"2023-08-27T11:19:17.248267+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:03:02.291266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.678064+0000","last_change":"2023-08-27T11:19:15.678625+0000","last_active":"2023-08-27T11:19:16.678064+0000","last_peered":"2023-08-27T11:19:16.678064+0000","last_clean":"2023-08-27T11:19:16.678064+0000","last_became_active":"2023-08-27T11:19:15.678324+0000","last_became_peered":"2023-08-27T11:19:15.678324+0000","last_unstale":"2023-08-27T11:19:16.678064+0000","last_undegraded":"2023-08-27T11:19:16.678064+0000","last_fullsized":"2023-08-27T11:19:16.678064+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:56:07.395501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252341+0000","last_change":"2023-08-27T11:19:16.253871+0000","last_active":"2023-08-27T11:19:17.252341+0000","last_peered":"2023-08-27T11:19:17.252341+0000","last_clean":"2023-08-27T11:19:17.252341+0000","last_became_active":"2023-08-27T11:19:16.252750+0000","last_became_peered":"2023-08-27T11:19:16.252750+0000","last_unstale":"2023-08-27T11:19:17.252341+0000","last_undegraded":"2023-08-27T11:19:17.252341+0000","last_fullsized":"2023-08-27T11:19:17.252341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:02:35.688837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.676672+0000","last_change":"2023-08-27T11:19:15.681717+0000","last_active":"2023-08-27T11:19:16.676672+0000","last_peered":"2023-08-27T11:19:16.676672+0000","last_clean":"2023-08-27T11:19:16.676672+0000","last_became_active":"2023-08-27T11:19:15.681418+0000","last_became_peered":"2023-08-27T11:19:15.681418+0000","last_unstale":"2023-08-27T11:19:16.676672+0000","last_undegraded":"2023-08-27T11:19:16.676672+0000","last_fullsized":"2023-08-27T11:19:16.676672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:38.354190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252615+0000","last_change":"2023-08-27T11:19:16.255651+0000","last_active":"2023-08-27T11:19:17.252615+0000","last_peered":"2023-08-27T11:19:17.252615+0000","last_clean":"2023-08-27T11:19:17.252615+0000","last_became_active":"2023-08-27T11:19:16.255028+0000","last_became_peered":"2023-08-27T11:19:16.255028+0000","last_unstale":"2023-08-27T11:19:17.252615+0000","last_undegraded":"2023-08-27T11:19:17.252615+0000","last_fullsized":"2023-08-27T11:19:17.252615+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:27:05.200585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"64'2","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252387+0000","last_change":"2023-08-27T11:19:15.675923+0000","last_active":"2023-08-27T11:19:17.252387+0000","last_peered":"2023-08-27T11:19:17.252387+0000","last_clean":"2023-08-27T11:19:17.252387+0000","last_became_active":"2023-08-27T11:19:15.675598+0000","last_became_peered":"2023-08-27T11:19:15.675598+0000","last_unstale":"2023-08-27T11:19:17.252387+0000","last_undegraded":"2023-08-27T11:19:17.252387+0000","last_fullsized":"2023-08-27T11:19:17.252387+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:13:54.624802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248476+0000","last_change":"2023-08-27T11:19:16.263309+0000","last_active":"2023-08-27T11:19:17.248476+0000","last_peered":"2023-08-27T11:19:17.248476+0000","last_clean":"2023-08-27T11:19:17.248476+0000","last_became_active":"2023-08-27T11:19:16.262930+0000","last_became_peered":"2023-08-27T11:19:16.262930+0000","last_unstale":"2023-08-27T11:19:17.248476+0000","last_undegraded":"2023-08-27T11:19:17.248476+0000","last_fullsized":"2023-08-27T11:19:17.248476+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:19:02.011264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"64'1","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252762+0000","last_change":"2023-08-27T11:19:15.684528+0000","last_active":"2023-08-27T11:19:17.252762+0000","last_peered":"2023-08-27T11:19:17.252762+0000","last_clean":"2023-08-27T11:19:17.252762+0000","last_became_active":"2023-08-27T11:19:15.684220+0000","last_became_peered":"2023-08-27T11:19:15.684220+0000","last_unstale":"2023-08-27T11:19:17.252762+0000","last_undegraded":"2023-08-27T11:19:17.252762+0000","last_fullsized":"2023-08-27T11:19:17.252762+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:44:35.580620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248613+0000","last_change":"2023-08-27T11:19:16.260311+0000","last_active":"2023-08-27T11:19:17.248613+0000","last_peered":"2023-08-27T11:19:17.248613+0000","last_clean":"2023-08-27T11:19:17.248613+0000","last_became_active":"2023-08-27T11:19:16.260059+0000","last_became_peered":"2023-08-27T11:19:16.260059+0000","last_unstale":"2023-08-27T11:19:17.248613+0000","last_undegraded":"2023-08-27T11:19:17.248613+0000","last_fullsized":"2023-08-27T11:19:17.248613+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:22:56.884278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.674584+0000","last_change":"2023-08-27T11:19:15.681037+0000","last_active":"2023-08-27T11:19:16.674584+0000","last_peered":"2023-08-27T11:19:16.674584+0000","last_clean":"2023-08-27T11:19:16.674584+0000","last_became_active":"2023-08-27T11:19:15.680521+0000","last_became_peered":"2023-08-27T11:19:15.680521+0000","last_unstale":"2023-08-27T11:19:16.674584+0000","last_undegraded":"2023-08-27T11:19:16.674584+0000","last_fullsized":"2023-08-27T11:19:16.674584+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:49:02.157224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252531+0000","last_change":"2023-08-27T11:19:16.251391+0000","last_active":"2023-08-27T11:19:17.252531+0000","last_peered":"2023-08-27T11:19:17.252531+0000","last_clean":"2023-08-27T11:19:17.252531+0000","last_became_active":"2023-08-27T11:19:16.251068+0000","last_became_peered":"2023-08-27T11:19:16.251068+0000","last_unstale":"2023-08-27T11:19:17.252531+0000","last_undegraded":"2023-08-27T11:19:17.252531+0000","last_fullsized":"2023-08-27T11:19:17.252531+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:32:38.024405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248226+0000","last_change":"2023-08-27T11:19:15.684238+0000","last_active":"2023-08-27T11:19:17.248226+0000","last_peered":"2023-08-27T11:19:17.248226+0000","last_clean":"2023-08-27T11:19:17.248226+0000","last_became_active":"2023-08-27T11:19:15.683983+0000","last_became_peered":"2023-08-27T11:19:15.683983+0000","last_unstale":"2023-08-27T11:19:17.248226+0000","last_undegraded":"2023-08-27T11:19:17.248226+0000","last_fullsized":"2023-08-27T11:19:17.248226+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:00.357426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252536+0000","last_change":"2023-08-27T11:19:16.260858+0000","last_active":"2023-08-27T11:19:17.252536+0000","last_peered":"2023-08-27T11:19:17.252536+0000","last_clean":"2023-08-27T11:19:17.252536+0000","last_became_active":"2023-08-27T11:19:16.260544+0000","last_became_peered":"2023-08-27T11:19:16.260544+0000","last_unstale":"2023-08-27T11:19:17.252536+0000","last_undegraded":"2023-08-27T11:19:17.252536+0000","last_fullsized":"2023-08-27T11:19:17.252536+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:13:45.345824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.250170+0000","last_change":"2023-08-27T11:19:15.681421+0000","last_active":"2023-08-27T11:19:16.250170+0000","last_peered":"2023-08-27T11:19:16.250170+0000","last_clean":"2023-08-27T11:19:16.250170+0000","last_became_active":"2023-08-27T11:19:15.681119+0000","last_became_peered":"2023-08-27T11:19:15.681119+0000","last_unstale":"2023-08-27T11:19:16.250170+0000","last_undegraded":"2023-08-27T11:19:16.250170+0000","last_fullsized":"2023-08-27T11:19:16.250170+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:26.109313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.247986+0000","last_change":"2023-08-27T11:19:16.258682+0000","last_active":"2023-08-27T11:19:17.247986+0000","last_peered":"2023-08-27T11:19:17.247986+0000","last_clean":"2023-08-27T11:19:17.247986+0000","last_became_active":"2023-08-27T11:19:16.258421+0000","last_became_peered":"2023-08-27T11:19:16.258421+0000","last_unstale":"2023-08-27T11:19:17.247986+0000","last_undegraded":"2023-08-27T11:19:17.247986+0000","last_fullsized":"2023-08-27T11:19:17.247986+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:34:02.411369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.250399+0000","last_change":"2023-08-27T11:19:15.682371+0000","last_active":"2023-08-27T11:19:16.250399+0000","last_peered":"2023-08-27T11:19:16.250399+0000","last_clean":"2023-08-27T11:19:16.250399+0000","last_became_active":"2023-08-27T11:19:15.682092+0000","last_became_peered":"2023-08-27T11:19:15.682092+0000","last_unstale":"2023-08-27T11:19:16.250399+0000","last_undegraded":"2023-08-27T11:19:16.250399+0000","last_fullsized":"2023-08-27T11:19:16.250399+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:58.739608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.258758+0000","last_change":"2023-08-27T11:19:16.255461+0000","last_active":"2023-08-27T11:19:16.258758+0000","last_peered":"2023-08-27T11:19:16.258758+0000","last_clean":"2023-08-27T11:19:16.258758+0000","last_became_active":"2023-08-27T11:19:16.254827+0000","last_became_peered":"2023-08-27T11:19:16.254827+0000","last_unstale":"2023-08-27T11:19:16.258758+0000","last_undegraded":"2023-08-27T11:19:16.258758+0000","last_fullsized":"2023-08-27T11:19:16.258758+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:06.746338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.249380+0000","last_change":"2023-08-27T11:19:15.677219+0000","last_active":"2023-08-27T11:19:16.249380+0000","last_peered":"2023-08-27T11:19:16.249380+0000","last_clean":"2023-08-27T11:19:16.249380+0000","last_became_active":"2023-08-27T11:19:15.676685+0000","last_became_peered":"2023-08-27T11:19:15.676685+0000","last_unstale":"2023-08-27T11:19:16.249380+0000","last_undegraded":"2023-08-27T11:19:16.249380+0000","last_fullsized":"2023-08-27T11:19:16.249380+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:19.489206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.263822+0000","last_change":"2023-08-27T11:19:16.256966+0000","last_active":"2023-08-27T11:19:16.263822+0000","last_peered":"2023-08-27T11:19:16.263822+0000","last_clean":"2023-08-27T11:19:16.263822+0000","last_became_active":"2023-08-27T11:19:16.255805+0000","last_became_peered":"2023-08-27T11:19:16.255805+0000","last_unstale":"2023-08-27T11:19:16.263822+0000","last_undegraded":"2023-08-27T11:19:16.263822+0000","last_fullsized":"2023-08-27T11:19:16.263822+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:53:18.080063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"64'3","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.676214+0000","last_change":"2023-08-27T11:19:15.674041+0000","last_active":"2023-08-27T11:19:16.676214+0000","last_peered":"2023-08-27T11:19:16.676214+0000","last_clean":"2023-08-27T11:19:16.676214+0000","last_became_active":"2023-08-27T11:19:15.673759+0000","last_became_peered":"2023-08-27T11:19:15.673759+0000","last_unstale":"2023-08-27T11:19:16.676214+0000","last_undegraded":"2023-08-27T11:19:16.676214+0000","last_fullsized":"2023-08-27T11:19:16.676214+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:08:52.207889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264766+0000","last_change":"2023-08-27T11:19:16.260619+0000","last_active":"2023-08-27T11:19:16.264766+0000","last_peered":"2023-08-27T11:19:16.264766+0000","last_clean":"2023-08-27T11:19:16.264766+0000","last_became_active":"2023-08-27T11:19:16.260346+0000","last_became_peered":"2023-08-27T11:19:16.260346+0000","last_unstale":"2023-08-27T11:19:16.264766+0000","last_undegraded":"2023-08-27T11:19:16.264766+0000","last_fullsized":"2023-08-27T11:19:16.264766+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:04:19.737862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"64'6","reported_seq":19,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252509+0000","last_change":"2023-08-27T11:19:15.681619+0000","last_active":"2023-08-27T11:19:17.252509+0000","last_peered":"2023-08-27T11:19:17.252509+0000","last_clean":"2023-08-27T11:19:17.252509+0000","last_became_active":"2023-08-27T11:19:15.681317+0000","last_became_peered":"2023-08-27T11:19:15.681317+0000","last_unstale":"2023-08-27T11:19:17.252509+0000","last_undegraded":"2023-08-27T11:19:17.252509+0000","last_fullsized":"2023-08-27T11:19:17.252509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:54:22.388631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262911+0000","last_change":"2023-08-27T11:19:16.257958+0000","last_active":"2023-08-27T11:19:16.262911+0000","last_peered":"2023-08-27T11:19:16.262911+0000","last_clean":"2023-08-27T11:19:16.262911+0000","last_became_active":"2023-08-27T11:19:16.257670+0000","last_became_peered":"2023-08-27T11:19:16.257670+0000","last_unstale":"2023-08-27T11:19:16.262911+0000","last_undegraded":"2023-08-27T11:19:16.262911+0000","last_fullsized":"2023-08-27T11:19:16.262911+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:07:26.067523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.677525+0000","last_change":"2023-08-27T11:19:15.681748+0000","last_active":"2023-08-27T11:19:16.677525+0000","last_peered":"2023-08-27T11:19:16.677525+0000","last_clean":"2023-08-27T11:19:16.677525+0000","last_became_active":"2023-08-27T11:19:15.681468+0000","last_became_peered":"2023-08-27T11:19:15.681468+0000","last_unstale":"2023-08-27T11:19:16.677525+0000","last_undegraded":"2023-08-27T11:19:16.677525+0000","last_fullsized":"2023-08-27T11:19:16.677525+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:31.326789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252098+0000","last_change":"2023-08-27T11:19:16.254735+0000","last_active":"2023-08-27T11:19:17.252098+0000","last_peered":"2023-08-27T11:19:17.252098+0000","last_clean":"2023-08-27T11:19:17.252098+0000","last_became_active":"2023-08-27T11:19:16.254460+0000","last_became_peered":"2023-08-27T11:19:16.254460+0000","last_unstale":"2023-08-27T11:19:17.252098+0000","last_undegraded":"2023-08-27T11:19:17.252098+0000","last_fullsized":"2023-08-27T11:19:17.252098+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:47:27.241362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.246749+0000","last_change":"2023-08-27T11:19:15.680755+0000","last_active":"2023-08-27T11:19:16.246749+0000","last_peered":"2023-08-27T11:19:16.246749+0000","last_clean":"2023-08-27T11:19:16.246749+0000","last_became_active":"2023-08-27T11:19:15.680442+0000","last_became_peered":"2023-08-27T11:19:15.680442+0000","last_unstale":"2023-08-27T11:19:16.246749+0000","last_undegraded":"2023-08-27T11:19:16.246749+0000","last_fullsized":"2023-08-27T11:19:16.246749+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:07:07.837430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252693+0000","last_change":"2023-08-27T11:19:16.252681+0000","last_active":"2023-08-27T11:19:17.252693+0000","last_peered":"2023-08-27T11:19:17.252693+0000","last_clean":"2023-08-27T11:19:17.252693+0000","last_became_active":"2023-08-27T11:19:16.252361+0000","last_became_peered":"2023-08-27T11:19:16.252361+0000","last_unstale":"2023-08-27T11:19:17.252693+0000","last_undegraded":"2023-08-27T11:19:17.252693+0000","last_fullsized":"2023-08-27T11:19:17.252693+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:46:50.752398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.250640+0000","last_change":"2023-08-27T11:19:15.682094+0000","last_active":"2023-08-27T11:19:16.250640+0000","last_peered":"2023-08-27T11:19:16.250640+0000","last_clean":"2023-08-27T11:19:16.250640+0000","last_became_active":"2023-08-27T11:19:15.681828+0000","last_became_peered":"2023-08-27T11:19:15.681828+0000","last_unstale":"2023-08-27T11:19:16.250640+0000","last_undegraded":"2023-08-27T11:19:16.250640+0000","last_fullsized":"2023-08-27T11:19:16.250640+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:55:45.379063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252883+0000","last_change":"2023-08-27T11:19:16.252126+0000","last_active":"2023-08-27T11:19:17.252883+0000","last_peered":"2023-08-27T11:19:17.252883+0000","last_clean":"2023-08-27T11:19:17.252883+0000","last_became_active":"2023-08-27T11:19:16.251355+0000","last_became_peered":"2023-08-27T11:19:16.251355+0000","last_unstale":"2023-08-27T11:19:17.252883+0000","last_undegraded":"2023-08-27T11:19:17.252883+0000","last_fullsized":"2023-08-27T11:19:17.252883+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:31:18.071265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"64'8","reported_seq":21,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.247993+0000","last_change":"2023-08-27T11:19:15.676573+0000","last_active":"2023-08-27T11:19:17.247993+0000","last_peered":"2023-08-27T11:19:17.247993+0000","last_clean":"2023-08-27T11:19:17.247993+0000","last_became_active":"2023-08-27T11:19:15.676300+0000","last_became_peered":"2023-08-27T11:19:15.676300+0000","last_unstale":"2023-08-27T11:19:17.247993+0000","last_undegraded":"2023-08-27T11:19:17.247993+0000","last_fullsized":"2023-08-27T11:19:17.247993+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:16:52.142744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.251778+0000","last_change":"2023-08-27T11:19:16.249802+0000","last_active":"2023-08-27T11:19:17.251778+0000","last_peered":"2023-08-27T11:19:17.251778+0000","last_clean":"2023-08-27T11:19:17.251778+0000","last_became_active":"2023-08-27T11:19:16.249497+0000","last_became_peered":"2023-08-27T11:19:16.249497+0000","last_unstale":"2023-08-27T11:19:17.251778+0000","last_undegraded":"2023-08-27T11:19:17.251778+0000","last_fullsized":"2023-08-27T11:19:17.251778+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:55:24.372765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.676382+0000","last_change":"2023-08-27T11:19:15.679666+0000","last_active":"2023-08-27T11:19:16.676382+0000","last_peered":"2023-08-27T11:19:16.676382+0000","last_clean":"2023-08-27T11:19:16.676382+0000","last_became_active":"2023-08-27T11:19:15.679378+0000","last_became_peered":"2023-08-27T11:19:15.679378+0000","last_unstale":"2023-08-27T11:19:16.676382+0000","last_undegraded":"2023-08-27T11:19:16.676382+0000","last_fullsized":"2023-08-27T11:19:16.676382+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:47:44.577411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252559+0000","last_change":"2023-08-27T11:19:16.251660+0000","last_active":"2023-08-27T11:19:17.252559+0000","last_peered":"2023-08-27T11:19:17.252559+0000","last_clean":"2023-08-27T11:19:17.252559+0000","last_became_active":"2023-08-27T11:19:16.251387+0000","last_became_peered":"2023-08-27T11:19:16.251387+0000","last_unstale":"2023-08-27T11:19:17.252559+0000","last_undegraded":"2023-08-27T11:19:17.252559+0000","last_fullsized":"2023-08-27T11:19:17.252559+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:55:24.372765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248712+0000","last_change":"2023-08-27T11:19:15.677246+0000","last_active":"2023-08-27T11:19:17.248712+0000","last_peered":"2023-08-27T11:19:17.248712+0000","last_clean":"2023-08-27T11:19:17.248712+0000","last_became_active":"2023-08-27T11:19:15.676964+0000","last_became_peered":"2023-08-27T11:19:15.676964+0000","last_unstale":"2023-08-27T11:19:17.248712+0000","last_undegraded":"2023-08-27T11:19:17.248712+0000","last_fullsized":"2023-08-27T11:19:17.248712+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:43:44.164338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252294+0000","last_change":"2023-08-27T11:19:16.253130+0000","last_active":"2023-08-27T11:19:17.252294+0000","last_peered":"2023-08-27T11:19:17.252294+0000","last_clean":"2023-08-27T11:19:17.252294+0000","last_became_active":"2023-08-27T11:19:16.252627+0000","last_became_peered":"2023-08-27T11:19:16.252627+0000","last_unstale":"2023-08-27T11:19:17.252294+0000","last_undegraded":"2023-08-27T11:19:17.252294+0000","last_fullsized":"2023-08-27T11:19:17.252294+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:00:38.245757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.250933+0000","last_change":"2023-08-27T11:19:15.683177+0000","last_active":"2023-08-27T11:19:16.250933+0000","last_peered":"2023-08-27T11:19:16.250933+0000","last_clean":"2023-08-27T11:19:16.250933+0000","last_became_active":"2023-08-27T11:19:15.682862+0000","last_became_peered":"2023-08-27T11:19:15.682862+0000","last_unstale":"2023-08-27T11:19:16.250933+0000","last_undegraded":"2023-08-27T11:19:16.250933+0000","last_fullsized":"2023-08-27T11:19:16.250933+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:22:08.945339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248451+0000","last_change":"2023-08-27T11:19:16.253111+0000","last_active":"2023-08-27T11:19:17.248451+0000","last_peered":"2023-08-27T11:19:17.248451+0000","last_clean":"2023-08-27T11:19:17.248451+0000","last_became_active":"2023-08-27T11:19:16.252868+0000","last_became_peered":"2023-08-27T11:19:16.252868+0000","last_unstale":"2023-08-27T11:19:17.248451+0000","last_undegraded":"2023-08-27T11:19:17.248451+0000","last_fullsized":"2023-08-27T11:19:17.248451+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:24:29.530359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.249844+0000","last_change":"2023-08-27T11:19:15.678311+0000","last_active":"2023-08-27T11:19:16.249844+0000","last_peered":"2023-08-27T11:19:16.249844+0000","last_clean":"2023-08-27T11:19:16.249844+0000","last_became_active":"2023-08-27T11:19:15.677933+0000","last_became_peered":"2023-08-27T11:19:15.677933+0000","last_unstale":"2023-08-27T11:19:16.249844+0000","last_undegraded":"2023-08-27T11:19:16.249844+0000","last_fullsized":"2023-08-27T11:19:16.249844+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:41:50.956554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252198+0000","last_change":"2023-08-27T11:19:16.252009+0000","last_active":"2023-08-27T11:19:17.252198+0000","last_peered":"2023-08-27T11:19:17.252198+0000","last_clean":"2023-08-27T11:19:17.252198+0000","last_became_active":"2023-08-27T11:19:16.251709+0000","last_became_peered":"2023-08-27T11:19:16.251709+0000","last_unstale":"2023-08-27T11:19:17.252198+0000","last_undegraded":"2023-08-27T11:19:17.252198+0000","last_fullsized":"2023-08-27T11:19:17.252198+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:22:05.047563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252599+0000","last_change":"2023-08-27T11:19:15.675556+0000","last_active":"2023-08-27T11:19:17.252599+0000","last_peered":"2023-08-27T11:19:17.252599+0000","last_clean":"2023-08-27T11:19:17.252599+0000","last_became_active":"2023-08-27T11:19:15.675241+0000","last_became_peered":"2023-08-27T11:19:15.675241+0000","last_unstale":"2023-08-27T11:19:17.252599+0000","last_undegraded":"2023-08-27T11:19:17.252599+0000","last_fullsized":"2023-08-27T11:19:17.252599+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:04:26.237695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252244+0000","last_change":"2023-08-27T11:19:16.260556+0000","last_active":"2023-08-27T11:19:17.252244+0000","last_peered":"2023-08-27T11:19:17.252244+0000","last_clean":"2023-08-27T11:19:17.252244+0000","last_became_active":"2023-08-27T11:19:16.260269+0000","last_became_peered":"2023-08-27T11:19:16.260269+0000","last_unstale":"2023-08-27T11:19:17.252244+0000","last_undegraded":"2023-08-27T11:19:17.252244+0000","last_fullsized":"2023-08-27T11:19:17.252244+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:10:36.759506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.250569+0000","last_change":"2023-08-27T11:19:15.682286+0000","last_active":"2023-08-27T11:19:16.250569+0000","last_peered":"2023-08-27T11:19:16.250569+0000","last_clean":"2023-08-27T11:19:16.250569+0000","last_became_active":"2023-08-27T11:19:15.681833+0000","last_became_peered":"2023-08-27T11:19:15.681833+0000","last_unstale":"2023-08-27T11:19:16.250569+0000","last_undegraded":"2023-08-27T11:19:16.250569+0000","last_fullsized":"2023-08-27T11:19:16.250569+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:03:53.791487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264780+0000","last_change":"2023-08-27T11:19:16.261007+0000","last_active":"2023-08-27T11:19:16.264780+0000","last_peered":"2023-08-27T11:19:16.264780+0000","last_clean":"2023-08-27T11:19:16.264780+0000","last_became_active":"2023-08-27T11:19:16.260756+0000","last_became_peered":"2023-08-27T11:19:16.260756+0000","last_unstale":"2023-08-27T11:19:16.264780+0000","last_undegraded":"2023-08-27T11:19:16.264780+0000","last_fullsized":"2023-08-27T11:19:16.264780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:45:32.872489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252710+0000","last_change":"2023-08-27T11:19:15.681743+0000","last_active":"2023-08-27T11:19:17.252710+0000","last_peered":"2023-08-27T11:19:17.252710+0000","last_clean":"2023-08-27T11:19:17.252710+0000","last_became_active":"2023-08-27T11:19:15.681474+0000","last_became_peered":"2023-08-27T11:19:15.681474+0000","last_unstale":"2023-08-27T11:19:17.252710+0000","last_undegraded":"2023-08-27T11:19:17.252710+0000","last_fullsized":"2023-08-27T11:19:17.252710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:31:30.712591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.255361+0000","last_change":"2023-08-27T11:19:16.253638+0000","last_active":"2023-08-27T11:19:16.255361+0000","last_peered":"2023-08-27T11:19:16.255361+0000","last_clean":"2023-08-27T11:19:16.255361+0000","last_became_active":"2023-08-27T11:19:16.253420+0000","last_became_peered":"2023-08-27T11:19:16.253420+0000","last_unstale":"2023-08-27T11:19:16.255361+0000","last_undegraded":"2023-08-27T11:19:16.255361+0000","last_fullsized":"2023-08-27T11:19:16.255361+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:48:26.533059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.246125+0000","last_change":"2023-08-27T11:19:15.680648+0000","last_active":"2023-08-27T11:19:16.246125+0000","last_peered":"2023-08-27T11:19:16.246125+0000","last_clean":"2023-08-27T11:19:16.246125+0000","last_became_active":"2023-08-27T11:19:15.680400+0000","last_became_peered":"2023-08-27T11:19:15.680400+0000","last_unstale":"2023-08-27T11:19:16.246125+0000","last_undegraded":"2023-08-27T11:19:16.246125+0000","last_fullsized":"2023-08-27T11:19:16.246125+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:52:35.980582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264082+0000","last_change":"2023-08-27T11:19:16.259158+0000","last_active":"2023-08-27T11:19:16.264082+0000","last_peered":"2023-08-27T11:19:16.264082+0000","last_clean":"2023-08-27T11:19:16.264082+0000","last_became_active":"2023-08-27T11:19:16.258918+0000","last_became_peered":"2023-08-27T11:19:16.258918+0000","last_unstale":"2023-08-27T11:19:16.264082+0000","last_undegraded":"2023-08-27T11:19:16.264082+0000","last_fullsized":"2023-08-27T11:19:16.264082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:25:11.321371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.677742+0000","last_change":"2023-08-27T11:19:15.680156+0000","last_active":"2023-08-27T11:19:16.677742+0000","last_peered":"2023-08-27T11:19:16.677742+0000","last_clean":"2023-08-27T11:19:16.677742+0000","last_became_active":"2023-08-27T11:19:15.679591+0000","last_became_peered":"2023-08-27T11:19:15.679591+0000","last_unstale":"2023-08-27T11:19:16.677742+0000","last_undegraded":"2023-08-27T11:19:16.677742+0000","last_fullsized":"2023-08-27T11:19:16.677742+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:22:02.314053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252849+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:19:17.252849+0000","last_peered":"2023-08-27T11:19:17.252849+0000","last_clean":"2023-08-27T11:19:17.252849+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:19:17.252849+0000","last_undegraded":"2023-08-27T11:19:17.252849+0000","last_fullsized":"2023-08-27T11:19:17.252849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:00:18.336409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264010+0000","last_change":"2023-08-27T11:19:16.260953+0000","last_active":"2023-08-27T11:19:16.264010+0000","last_peered":"2023-08-27T11:19:16.264010+0000","last_clean":"2023-08-27T11:19:16.264010+0000","last_became_active":"2023-08-27T11:19:16.260700+0000","last_became_peered":"2023-08-27T11:19:16.260700+0000","last_unstale":"2023-08-27T11:19:16.264010+0000","last_undegraded":"2023-08-27T11:19:16.264010+0000","last_fullsized":"2023-08-27T11:19:16.264010+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:22:14.482419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.245733+0000","last_change":"2023-08-27T11:19:15.674181+0000","last_active":"2023-08-27T11:19:16.245733+0000","last_peered":"2023-08-27T11:19:16.245733+0000","last_clean":"2023-08-27T11:19:16.245733+0000","last_became_active":"2023-08-27T11:19:15.673928+0000","last_became_peered":"2023-08-27T11:19:15.673928+0000","last_unstale":"2023-08-27T11:19:16.245733+0000","last_undegraded":"2023-08-27T11:19:16.245733+0000","last_fullsized":"2023-08-27T11:19:16.245733+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:13.603402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252781+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:19:17.252781+0000","last_peered":"2023-08-27T11:19:17.252781+0000","last_clean":"2023-08-27T11:19:17.252781+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:19:17.252781+0000","last_undegraded":"2023-08-27T11:19:17.252781+0000","last_fullsized":"2023-08-27T11:19:17.252781+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:57:17.819601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248426+0000","last_change":"2023-08-27T11:19:16.259944+0000","last_active":"2023-08-27T11:19:17.248426+0000","last_peered":"2023-08-27T11:19:17.248426+0000","last_clean":"2023-08-27T11:19:17.248426+0000","last_became_active":"2023-08-27T11:19:16.259657+0000","last_became_peered":"2023-08-27T11:19:16.259657+0000","last_unstale":"2023-08-27T11:19:17.248426+0000","last_undegraded":"2023-08-27T11:19:17.248426+0000","last_fullsized":"2023-08-27T11:19:17.248426+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:37:02.028909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.675869+0000","last_change":"2023-08-27T11:19:15.678064+0000","last_active":"2023-08-27T11:19:16.675869+0000","last_peered":"2023-08-27T11:19:16.675869+0000","last_clean":"2023-08-27T11:19:16.675869+0000","last_became_active":"2023-08-27T11:19:15.677804+0000","last_became_peered":"2023-08-27T11:19:15.677804+0000","last_unstale":"2023-08-27T11:19:16.675869+0000","last_undegraded":"2023-08-27T11:19:16.675869+0000","last_fullsized":"2023-08-27T11:19:16.675869+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:07:25.062747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":90,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.247284+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:19:16.247284+0000","last_peered":"2023-08-27T11:19:16.247284+0000","last_clean":"2023-08-27T11:19:16.247284+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:19:16.247284+0000","last_undegraded":"2023-08-27T11:19:16.247284+0000","last_fullsized":"2023-08-27T11:19:16.247284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:38:44.667594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.257801+0000","last_change":"2023-08-27T11:19:16.256112+0000","last_active":"2023-08-27T11:19:16.257801+0000","last_peered":"2023-08-27T11:19:16.257801+0000","last_clean":"2023-08-27T11:19:16.257801+0000","last_became_active":"2023-08-27T11:19:16.255840+0000","last_became_peered":"2023-08-27T11:19:16.255840+0000","last_unstale":"2023-08-27T11:19:16.257801+0000","last_undegraded":"2023-08-27T11:19:16.257801+0000","last_fullsized":"2023-08-27T11:19:16.257801+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:47:15.078584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.240918+0000","last_change":"2023-08-27T11:19:15.682143+0000","last_active":"2023-08-27T11:19:16.240918+0000","last_peered":"2023-08-27T11:19:16.240918+0000","last_clean":"2023-08-27T11:19:16.240918+0000","last_became_active":"2023-08-27T11:19:15.681880+0000","last_became_peered":"2023-08-27T11:19:15.681880+0000","last_unstale":"2023-08-27T11:19:16.240918+0000","last_undegraded":"2023-08-27T11:19:16.240918+0000","last_fullsized":"2023-08-27T11:19:16.240918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:05.741563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":94,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.247181+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:19:16.247181+0000","last_peered":"2023-08-27T11:19:16.247181+0000","last_clean":"2023-08-27T11:19:16.247181+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:19:16.247181+0000","last_undegraded":"2023-08-27T11:19:16.247181+0000","last_fullsized":"2023-08-27T11:19:16.247181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:59:51.103458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252756+0000","last_change":"2023-08-27T11:19:16.256282+0000","last_active":"2023-08-27T11:19:17.252756+0000","last_peered":"2023-08-27T11:19:17.252756+0000","last_clean":"2023-08-27T11:19:17.252756+0000","last_became_active":"2023-08-27T11:19:16.254162+0000","last_became_peered":"2023-08-27T11:19:16.254162+0000","last_unstale":"2023-08-27T11:19:17.252756+0000","last_undegraded":"2023-08-27T11:19:17.252756+0000","last_fullsized":"2023-08-27T11:19:17.252756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:00:38.245757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.240967+0000","last_change":"2023-08-27T11:19:15.679327+0000","last_active":"2023-08-27T11:19:16.240967+0000","last_peered":"2023-08-27T11:19:16.240967+0000","last_clean":"2023-08-27T11:19:16.240967+0000","last_became_active":"2023-08-27T11:19:15.678898+0000","last_became_peered":"2023-08-27T11:19:15.678898+0000","last_unstale":"2023-08-27T11:19:16.240967+0000","last_undegraded":"2023-08-27T11:19:16.240967+0000","last_fullsized":"2023-08-27T11:19:16.240967+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:53.441825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.246840+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:19:16.246840+0000","last_peered":"2023-08-27T11:19:16.246840+0000","last_clean":"2023-08-27T11:19:16.246840+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:19:16.246840+0000","last_undegraded":"2023-08-27T11:19:16.246840+0000","last_fullsized":"2023-08-27T11:19:16.246840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:01:16.542542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248584+0000","last_change":"2023-08-27T11:19:16.255315+0000","last_active":"2023-08-27T11:19:17.248584+0000","last_peered":"2023-08-27T11:19:17.248584+0000","last_clean":"2023-08-27T11:19:17.248584+0000","last_became_active":"2023-08-27T11:19:16.255033+0000","last_became_peered":"2023-08-27T11:19:16.255033+0000","last_unstale":"2023-08-27T11:19:17.248584+0000","last_undegraded":"2023-08-27T11:19:17.248584+0000","last_fullsized":"2023-08-27T11:19:17.248584+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:52:18.509741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.241749+0000","last_change":"2023-08-27T11:19:15.679229+0000","last_active":"2023-08-27T11:19:16.241749+0000","last_peered":"2023-08-27T11:19:16.241749+0000","last_clean":"2023-08-27T11:19:16.241749+0000","last_became_active":"2023-08-27T11:19:15.678940+0000","last_became_peered":"2023-08-27T11:19:15.678940+0000","last_unstale":"2023-08-27T11:19:16.241749+0000","last_undegraded":"2023-08-27T11:19:16.241749+0000","last_fullsized":"2023-08-27T11:19:16.241749+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:19.489206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":244,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.247014+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:19:16.247014+0000","last_peered":"2023-08-27T11:19:16.247014+0000","last_clean":"2023-08-27T11:19:16.247014+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:19:16.247014+0000","last_undegraded":"2023-08-27T11:19:16.247014+0000","last_fullsized":"2023-08-27T11:19:16.247014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:58:13.188543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.249635+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:19:16.249635+0000","last_peered":"2023-08-27T11:19:16.249635+0000","last_clean":"2023-08-27T11:19:16.249635+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:19:16.249635+0000","last_undegraded":"2023-08-27T11:19:16.249635+0000","last_fullsized":"2023-08-27T11:19:16.249635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:58:58.287272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264042+0000","last_change":"2023-08-27T11:19:16.259098+0000","last_active":"2023-08-27T11:19:16.264042+0000","last_peered":"2023-08-27T11:19:16.264042+0000","last_clean":"2023-08-27T11:19:16.264042+0000","last_became_active":"2023-08-27T11:19:16.258824+0000","last_became_peered":"2023-08-27T11:19:16.258824+0000","last_unstale":"2023-08-27T11:19:16.264042+0000","last_undegraded":"2023-08-27T11:19:16.264042+0000","last_fullsized":"2023-08-27T11:19:16.264042+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:48.743606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"64'2","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252554+0000","last_change":"2023-08-27T11:19:15.680008+0000","last_active":"2023-08-27T11:19:17.252554+0000","last_peered":"2023-08-27T11:19:17.252554+0000","last_clean":"2023-08-27T11:19:17.252554+0000","last_became_active":"2023-08-27T11:19:15.679150+0000","last_became_peered":"2023-08-27T11:19:15.679150+0000","last_unstale":"2023-08-27T11:19:17.252554+0000","last_undegraded":"2023-08-27T11:19:17.252554+0000","last_fullsized":"2023-08-27T11:19:17.252554+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:58.293421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.241064+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:19:16.241064+0000","last_peered":"2023-08-27T11:19:16.241064+0000","last_clean":"2023-08-27T11:19:16.241064+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:19:16.241064+0000","last_undegraded":"2023-08-27T11:19:16.241064+0000","last_fullsized":"2023-08-27T11:19:16.241064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:46:45.809571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264053+0000","last_change":"2023-08-27T11:19:16.260971+0000","last_active":"2023-08-27T11:19:16.264053+0000","last_peered":"2023-08-27T11:19:16.264053+0000","last_clean":"2023-08-27T11:19:16.264053+0000","last_became_active":"2023-08-27T11:19:16.260712+0000","last_became_peered":"2023-08-27T11:19:16.260712+0000","last_unstale":"2023-08-27T11:19:16.264053+0000","last_undegraded":"2023-08-27T11:19:16.264053+0000","last_fullsized":"2023-08-27T11:19:16.264053+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:32.331565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.677770+0000","last_change":"2023-08-27T11:19:15.679538+0000","last_active":"2023-08-27T11:19:16.677770+0000","last_peered":"2023-08-27T11:19:16.677770+0000","last_clean":"2023-08-27T11:19:16.677770+0000","last_became_active":"2023-08-27T11:19:15.679102+0000","last_became_peered":"2023-08-27T11:19:15.679102+0000","last_unstale":"2023-08-27T11:19:16.677770+0000","last_undegraded":"2023-08-27T11:19:16.677770+0000","last_fullsized":"2023-08-27T11:19:16.677770+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:49:30.845159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252269+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:19:17.252269+0000","last_peered":"2023-08-27T11:19:17.252269+0000","last_clean":"2023-08-27T11:19:17.252269+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:19:17.252269+0000","last_undegraded":"2023-08-27T11:19:17.252269+0000","last_fullsized":"2023-08-27T11:19:17.252269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:57:17.819601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28368,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343472,"statfs":{"total":96636764160,"available":96607715328,"internally_reserved":0,"allocated":1372160,"data_stored":943127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47499999999999998,"15min":0.46700000000000003},"min":{"1min":0.32000000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.36199999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.77700000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54000000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47699999999999998,"15min":0.46899999999999997},"min":{"1min":0.32200000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.39600000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.82299999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58799999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.45000000000000001},"min":{"1min":0.374,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.42799999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76100000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.48699999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.34200000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81000000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60599999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68799999999999994,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.504,"15min":0.502},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.38300000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73499999999999999,"5min":0.747,"15min":0.747},"last":0.44500000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.50600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":352759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.33900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89400000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54300000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.42099999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.48599999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.40999999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.503},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60699999999999998,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.51600000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53100000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77200000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75600000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.622}]},{"osd":5,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.442},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59399999999999997,"15min":0.59299999999999997},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.33900000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57199999999999995,"15min":0.58299999999999996},"min":{"1min":0.35299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69399999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78700000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28364,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343476,"statfs":{"total":96636764160,"available":96607719424,"internally_reserved":0,"allocated":1380352,"data_stored":943239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.79700000000000004,"15min":0.877},"min":{"1min":0.223,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56100000000000005,"5min":11.971,"15min":11.971},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.318,"5min":1.288,"15min":1.45},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48099999999999998,"5min":23.045999999999999,"15min":23.045999999999999},"last":0.34899999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.82999999999999996,"15min":0.91100000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.42399999999999999,"5min":11.928000000000001,"15min":11.928000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.45300000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.26300000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.82599999999999996,"15min":0.90200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.52800000000000002,"5min":10.99,"15min":10.99},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.371,"5min":1.323,"15min":1.482},"min":{"1min":0.28699999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54200000000000004,"5min":22.893999999999998,"15min":22.893999999999998},"last":0.29699999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.44700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.73999999999999999,"5min":1.26,"15min":1.26},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.98499999999999999,"15min":1.0580000000000001},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73199999999999998,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.51300000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":1.514,"15min":1.671},"min":{"1min":0.41099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85399999999999998,"5min":23.016999999999999,"15min":23.016999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77700000000000002,"5min":1.296,"15min":1.296},"last":0.503}]},{"osd":6,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":1.034,"15min":1.1100000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79200000000000004,"5min":10.925000000000001,"15min":10.925000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":1.532,"15min":1.6879999999999999},"min":{"1min":0.45500000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.875,"5min":22.952000000000002,"15min":22.952000000000002},"last":0.58199999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":1.524,"15min":1.6830000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82599999999999996,"5min":22.988,"15min":22.988},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79600000000000004,"5min":1.327,"15min":1.327},"last":0.56599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":774144,"data_stored":351398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.317,"15min":0.318},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.436,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.35999999999999999,"15min":0.36599999999999999},"min":{"1min":0.23899999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.42099999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.34000000000000002,"15min":0.34300000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.48899999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34000000000000002,"15min":0.34300000000000003},"min":{"1min":0.247,"5min":0.224,"15min":0.224},"max":{"1min":0.41899999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.39900000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.35099999999999998,"15min":0.35399999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.47399999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.375,"15min":0.376},"min":{"1min":0.32300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.435,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.45700000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.755,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59999999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.57899999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.625},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55000000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.64000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":352759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.22500000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.316,"15min":0.315},"min":{"1min":0.23200000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.311}]},{"osd":1,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32800000000000001,"15min":0.32900000000000001},"min":{"1min":0.253,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.24399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.34899999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.33700000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.47099999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.46999999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.72299999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.42799999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.48099999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.79100000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.372}]},{"osd":6,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.501,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44500000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.629,"15min":0.629},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81399999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.39900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":352759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.29799999999999999,"15min":0.29399999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.436,"5min":0.438,"15min":0.438},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.30499999999999999,"15min":0.29999999999999999},"min":{"1min":0.23899999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.31900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.30199999999999999,"15min":0.29999999999999999},"min":{"1min":0.23899999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.35699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.29099999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.46999999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.378,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.40100000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57799999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.377,"5min":0.314,"15min":0.314},"max":{"1min":0.76200000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.42199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73199999999999998,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.73899999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.59799999999999998},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.82899999999999996,"5min":0.874,"15min":0.874},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81699999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52500000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":1.236,"15min":1.236},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":798720,"data_stored":354475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.499,"15min":0.504},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70199999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48199999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.60899999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.57499999999999996}]},{"osd":2,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66300000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.62}]},{"osd":3,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.52100000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.71199999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51800000000000002,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71699999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.38800000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.77600000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.68999999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.69499999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57599999999999996,"15min":0.57999999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.79000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":352759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65800000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79700000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.61899999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.63700000000000001,"5min":1.075,"15min":1.075},"last":0.56100000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55700000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73099999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57499999999999996}]},{"osd":3,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":1.03,"15min":1.03},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70599999999999996,"5min":1.026,"15min":1.026},"last":0.46800000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67000000000000004,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.77100000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54000000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.74399999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71399999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.49299999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81999999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.62},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78800000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:19:20.774 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:19:20.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.927+0000 7fd3723a0700 1 Processor -- start 2023-08-27T11:19:20.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.927+0000 7fd3723a0700 1 -- start start 2023-08-27T11:19:20.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.928+0000 7fd3723a0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd36c06f790 0x7fd36c06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.928+0000 7fd3723a0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36c146f00 con 0x7fd36c0ac890 2023-08-27T11:19:20.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.928+0000 7fd3723a0700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36c0b0a70 con 0x7fd36c146b20 2023-08-27T11:19:20.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.928+0000 7fd3723a0700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36c0b16f0 con 0x7fd36c06f790 2023-08-27T11:19:20.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.928+0000 7fd370b9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd36c06f790 0x7fd36c06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.928+0000 7fd371b9f700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd36c0ac890 0x7fd36c0acca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53418/0 (socket says 172.21.15.112:53418) 2023-08-27T11:19:20.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.928+0000 7fd371b9f700 1 -- 172.21.15.112:0/2179374674 learned_addr learned my addr 172.21.15.112:0/2179374674 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:20.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.929+0000 7fd370b9d700 1 -- 172.21.15.112:0/2179374674 >> v1:172.21.15.138:6789/0 conn(0x7fd36c146b20 legacy=0x7fd36c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:20.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.929+0000 7fd370b9d700 1 -- 172.21.15.112:0/2179374674 >> v1:172.21.15.112:6789/0 conn(0x7fd36c0ac890 legacy=0x7fd36c0acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.929+0000 7fd370b9d700 1 -- 172.21.15.112:0/2179374674 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36c0b0d50 con 0x7fd36c06f790 2023-08-27T11:19:20.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.929+0000 7fd370b9d700 1 --2- 172.21.15.112:0/2179374674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd36c06f790 0x7fd36c06fc40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd368009840 tx=0x7fd368009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3418bc85b7f98b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:20.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.929+0000 7fd363fff700 1 -- 172.21.15.112:0/2179374674 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368004230 con 0x7fd36c06f790 2023-08-27T11:19:20.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.929+0000 7fd363fff700 1 -- 172.21.15.112:0/2179374674 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd368004390 con 0x7fd36c06f790 2023-08-27T11:19:20.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.929+0000 7fd363fff700 1 -- 172.21.15.112:0/2179374674 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3680036a0 con 0x7fd36c06f790 2023-08-27T11:19:20.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.929+0000 7fd3723a0700 1 -- 172.21.15.112:0/2179374674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd36c06f790 msgr2=0x7fd36c06fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.929+0000 7fd3723a0700 1 --2- 172.21.15.112:0/2179374674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd36c06f790 0x7fd36c06fc40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd368009840 tx=0x7fd368009b50 comp rx=0 tx=0).stop 2023-08-27T11:19:20.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.929+0000 7fd3723a0700 1 -- 172.21.15.112:0/2179374674 shutdown_connections 2023-08-27T11:19:20.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.929+0000 7fd3723a0700 1 --2- 172.21.15.112:0/2179374674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd36c06f790 0x7fd36c06fc40 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.929+0000 7fd3723a0700 1 -- 172.21.15.112:0/2179374674 >> 172.21.15.112:0/2179374674 conn(0x7fd36c06b7a0 msgr2=0x7fd36c0cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:20.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.930+0000 7fd3723a0700 1 -- 172.21.15.112:0/2179374674 shutdown_connections 2023-08-27T11:19:20.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.930+0000 7fd3723a0700 1 -- 172.21.15.112:0/2179374674 wait complete. 2023-08-27T11:19:20.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.931+0000 7fd3723a0700 1 Processor -- start 2023-08-27T11:19:20.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.931+0000 7fd3723a0700 1 -- start start 2023-08-27T11:19:20.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.931+0000 7fd3723a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd36c06f790 0x7fd36c143e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.931+0000 7fd3723a0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd36c0ac890 0x7fd36c142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.931+0000 7fd3723a0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c146b20 0x7fd36c15f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.931+0000 7fd3723a0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c070180 con 0x7fd36c06f790 2023-08-27T11:19:20.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.931+0000 7fd3723a0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c143790 con 0x7fd36c146b20 2023-08-27T11:19:20.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.931+0000 7fd3723a0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c0c50f0 con 0x7fd36c0ac890 2023-08-27T11:19:20.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.931+0000 7fd37139e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd36c06f790 0x7fd36c143e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd37139e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd36c06f790 0x7fd36c143e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59038/0 (socket says 172.21.15.112:59038) 2023-08-27T11:19:20.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd37139e700 1 -- 172.21.15.112:0/3000862880 learned_addr learned my addr 172.21.15.112:0/3000862880 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:20.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd370b9d700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd36c0ac890 0x7fd36c142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd37139e700 1 -- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd36c0ac890 msgr2=0x7fd36c142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd37139e700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd36c0ac890 0x7fd36c142430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd371b9f700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c146b20 0x7fd36c15f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd37139e700 1 -- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c146b20 msgr2=0x7fd36c15f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:20.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd37139e700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c146b20 0x7fd36c15f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:20.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd37139e700 1 -- 172.21.15.112:0/3000862880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3680094a0 con 0x7fd36c06f790 2023-08-27T11:19:20.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd371b9f700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c146b20 0x7fd36c15f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:20.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd37139e700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd36c06f790 0x7fd36c143e10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00b470 tx=0x7fd35c00b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:20.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd361ffb700 1 -- 172.21.15.112:0/3000862880 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c011830 con 0x7fd36c06f790 2023-08-27T11:19:20.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd361ffb700 1 -- 172.21.15.112:0/3000862880 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c011990 con 0x7fd36c06f790 2023-08-27T11:19:20.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd3723a0700 1 -- 172.21.15.112:0/3000862880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36c142970 con 0x7fd36c06f790 2023-08-27T11:19:20.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd361ffb700 1 -- 172.21.15.112:0/3000862880 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c011d40 con 0x7fd36c06f790 2023-08-27T11:19:20.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.932+0000 7fd3723a0700 1 -- 172.21.15.112:0/3000862880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36c142ef0 con 0x7fd36c06f790 2023-08-27T11:19:20.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.933+0000 7fd3723a0700 1 -- 172.21.15.112:0/3000862880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36c061f50 con 0x7fd36c06f790 2023-08-27T11:19:20.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.934+0000 7fd361ffb700 1 -- 172.21.15.112:0/3000862880 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fd35c01a400 con 0x7fd36c06f790 2023-08-27T11:19:20.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.934+0000 7fd361ffb700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd35006a170 0x7fd35006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:20.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.934+0000 7fd361ffb700 1 -- 172.21.15.112:0/3000862880 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd35c0885a0 con 0x7fd36c06f790 2023-08-27T11:19:20.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.934+0000 7fd370b9d700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd35006a170 0x7fd35006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:20.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.935+0000 7fd370b9d700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd35006a170 0x7fd35006c630 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd36c140e20 tx=0x7fd368008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:20.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:20.938+0000 7fd361ffb700 1 -- 172.21.15.112:0/3000862880 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd35c053ca0 con 0x7fd36c06f790 2023-08-27T11:19:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.123+0000 7fd3723a0700 1 -- 172.21.15.112:0/3000862880 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd36c0b1030 con 0x7fd35006a170 2023-08-27T11:19:21.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.133+0000 7fd361ffb700 1 -- 172.21.15.112:0/3000862880 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312211 (secure 0 0 0) 0x7fd3680040b0 con 0x7fd35006a170 2023-08-27T11:19:21.139 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:21.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.136+0000 7fd3723a0700 1 -- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd35006a170 msgr2=0x7fd35006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:21.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.136+0000 7fd3723a0700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd35006a170 0x7fd35006c630 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd36c140e20 tx=0x7fd368008030 comp rx=0 tx=0).stop 2023-08-27T11:19:21.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.136+0000 7fd3723a0700 1 -- 172.21.15.112:0/3000862880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd36c06f790 msgr2=0x7fd36c143e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:21.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.136+0000 7fd3723a0700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd36c06f790 0x7fd36c143e10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00b470 tx=0x7fd35c00b830 comp rx=0 tx=0).stop 2023-08-27T11:19:21.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.136+0000 7fd3723a0700 1 -- 172.21.15.112:0/3000862880 shutdown_connections 2023-08-27T11:19:21.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.136+0000 7fd3723a0700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd35006a170 0x7fd35006c630 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.136+0000 7fd3723a0700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c146b20 0x7fd36c15f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.136+0000 7fd3723a0700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd36c06f790 0x7fd36c143e10 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.136+0000 7fd3723a0700 1 --2- 172.21.15.112:0/3000862880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd36c0ac890 0x7fd36c142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.136+0000 7fd3723a0700 1 -- 172.21.15.112:0/3000862880 >> 172.21.15.112:0/3000862880 conn(0x7fd36c06b7a0 msgr2=0x7fd36c0d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:21.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.136+0000 7fd3723a0700 1 -- 172.21.15.112:0/3000862880 shutdown_connections 2023-08-27T11:19:21.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.136+0000 7fd3723a0700 1 -- 172.21.15.112:0/3000862880 wait complete. 2023-08-27T11:19:21.147 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:19:21.149 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2023-08-27T11:19:20.256019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223464,"kb_used_data":7264,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754751256,"statfs":{"total":773094113280,"available":772865286144,"internally_reserved":0,"allocated":7438336,"data_stored":4003275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.593800"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262692+0000","last_change":"2023-08-27T11:19:16.259692+0000","last_active":"2023-08-27T11:19:16.262692+0000","last_peered":"2023-08-27T11:19:16.262692+0000","last_clean":"2023-08-27T11:19:16.262692+0000","last_became_active":"2023-08-27T11:19:16.259386+0000","last_became_peered":"2023-08-27T11:19:16.259386+0000","last_unstale":"2023-08-27T11:19:16.262692+0000","last_undegraded":"2023-08-27T11:19:16.262692+0000","last_fullsized":"2023-08-27T11:19:16.262692+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:53:25.637980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248443+0000","last_change":"2023-08-27T11:19:16.260287+0000","last_active":"2023-08-27T11:19:17.248443+0000","last_peered":"2023-08-27T11:19:17.248443+0000","last_clean":"2023-08-27T11:19:17.248443+0000","last_became_active":"2023-08-27T11:19:16.260031+0000","last_became_peered":"2023-08-27T11:19:16.260031+0000","last_unstale":"2023-08-27T11:19:17.248443+0000","last_undegraded":"2023-08-27T11:19:17.248443+0000","last_fullsized":"2023-08-27T11:19:17.248443+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:14:23.586906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264753+0000","last_change":"2023-08-27T11:19:16.260482+0000","last_active":"2023-08-27T11:19:16.264753+0000","last_peered":"2023-08-27T11:19:16.264753+0000","last_clean":"2023-08-27T11:19:16.264753+0000","last_became_active":"2023-08-27T11:19:16.260219+0000","last_became_peered":"2023-08-27T11:19:16.260219+0000","last_unstale":"2023-08-27T11:19:16.264753+0000","last_undegraded":"2023-08-27T11:19:16.264753+0000","last_fullsized":"2023-08-27T11:19:16.264753+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:03:03.572878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.258757+0000","last_change":"2023-08-27T11:19:16.255592+0000","last_active":"2023-08-27T11:19:16.258757+0000","last_peered":"2023-08-27T11:19:16.258757+0000","last_clean":"2023-08-27T11:19:16.258757+0000","last_became_active":"2023-08-27T11:19:16.254743+0000","last_became_peered":"2023-08-27T11:19:16.254743+0000","last_unstale":"2023-08-27T11:19:16.258757+0000","last_undegraded":"2023-08-27T11:19:16.258757+0000","last_fullsized":"2023-08-27T11:19:16.258757+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:30:25.750411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.265355+0000","last_change":"2023-08-27T11:19:16.263154+0000","last_active":"2023-08-27T11:19:16.265355+0000","last_peered":"2023-08-27T11:19:16.265355+0000","last_clean":"2023-08-27T11:19:16.265355+0000","last_became_active":"2023-08-27T11:19:16.259734+0000","last_became_peered":"2023-08-27T11:19:16.259734+0000","last_unstale":"2023-08-27T11:19:16.265355+0000","last_undegraded":"2023-08-27T11:19:16.265355+0000","last_fullsized":"2023-08-27T11:19:16.265355+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:40:49.102138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264017+0000","last_change":"2023-08-27T11:19:16.259340+0000","last_active":"2023-08-27T11:19:16.264017+0000","last_peered":"2023-08-27T11:19:16.264017+0000","last_clean":"2023-08-27T11:19:16.264017+0000","last_became_active":"2023-08-27T11:19:16.256467+0000","last_became_peered":"2023-08-27T11:19:16.256467+0000","last_unstale":"2023-08-27T11:19:16.264017+0000","last_undegraded":"2023-08-27T11:19:16.264017+0000","last_fullsized":"2023-08-27T11:19:16.264017+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:47:45.582187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.251958+0000","last_change":"2023-08-27T11:19:16.264271+0000","last_active":"2023-08-27T11:19:17.251958+0000","last_peered":"2023-08-27T11:19:17.251958+0000","last_clean":"2023-08-27T11:19:17.251958+0000","last_became_active":"2023-08-27T11:19:16.263971+0000","last_became_peered":"2023-08-27T11:19:16.263971+0000","last_unstale":"2023-08-27T11:19:17.251958+0000","last_undegraded":"2023-08-27T11:19:17.251958+0000","last_fullsized":"2023-08-27T11:19:17.251958+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:12:59.269333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.263447+0000","last_change":"2023-08-27T11:19:16.257924+0000","last_active":"2023-08-27T11:19:16.263447+0000","last_peered":"2023-08-27T11:19:16.263447+0000","last_clean":"2023-08-27T11:19:16.263447+0000","last_became_active":"2023-08-27T11:19:16.257571+0000","last_became_peered":"2023-08-27T11:19:16.257571+0000","last_unstale":"2023-08-27T11:19:16.263447+0000","last_undegraded":"2023-08-27T11:19:16.263447+0000","last_fullsized":"2023-08-27T11:19:16.263447+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:21.317637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252209+0000","last_change":"2023-08-27T11:19:16.250635+0000","last_active":"2023-08-27T11:19:17.252209+0000","last_peered":"2023-08-27T11:19:17.252209+0000","last_clean":"2023-08-27T11:19:17.252209+0000","last_became_active":"2023-08-27T11:19:16.250123+0000","last_became_peered":"2023-08-27T11:19:16.250123+0000","last_unstale":"2023-08-27T11:19:17.252209+0000","last_undegraded":"2023-08-27T11:19:17.252209+0000","last_fullsized":"2023-08-27T11:19:17.252209+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:31:31.717367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262921+0000","last_change":"2023-08-27T11:19:16.258355+0000","last_active":"2023-08-27T11:19:16.262921+0000","last_peered":"2023-08-27T11:19:16.262921+0000","last_clean":"2023-08-27T11:19:16.262921+0000","last_became_active":"2023-08-27T11:19:16.258025+0000","last_became_peered":"2023-08-27T11:19:16.258025+0000","last_unstale":"2023-08-27T11:19:16.262921+0000","last_undegraded":"2023-08-27T11:19:16.262921+0000","last_fullsized":"2023-08-27T11:19:16.262921+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:22:14.482419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248628+0000","last_change":"2023-08-27T11:19:16.260176+0000","last_active":"2023-08-27T11:19:17.248628+0000","last_peered":"2023-08-27T11:19:17.248628+0000","last_clean":"2023-08-27T11:19:17.248628+0000","last_became_active":"2023-08-27T11:19:16.259920+0000","last_became_peered":"2023-08-27T11:19:16.259920+0000","last_unstale":"2023-08-27T11:19:17.248628+0000","last_undegraded":"2023-08-27T11:19:17.248628+0000","last_fullsized":"2023-08-27T11:19:17.248628+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:21:13.100446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262978+0000","last_change":"2023-08-27T11:19:16.261417+0000","last_active":"2023-08-27T11:19:16.262978+0000","last_peered":"2023-08-27T11:19:16.262978+0000","last_clean":"2023-08-27T11:19:16.262978+0000","last_became_active":"2023-08-27T11:19:16.260964+0000","last_became_peered":"2023-08-27T11:19:16.260964+0000","last_unstale":"2023-08-27T11:19:16.262978+0000","last_undegraded":"2023-08-27T11:19:16.262978+0000","last_fullsized":"2023-08-27T11:19:16.262978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:53:18.080063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.263281+0000","last_change":"2023-08-27T11:19:16.256015+0000","last_active":"2023-08-27T11:19:16.263281+0000","last_peered":"2023-08-27T11:19:16.263281+0000","last_clean":"2023-08-27T11:19:16.263281+0000","last_became_active":"2023-08-27T11:19:16.255617+0000","last_became_peered":"2023-08-27T11:19:16.255617+0000","last_unstale":"2023-08-27T11:19:16.263281+0000","last_undegraded":"2023-08-27T11:19:16.263281+0000","last_fullsized":"2023-08-27T11:19:16.263281+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:16:54.974620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.263846+0000","last_change":"2023-08-27T11:19:16.257095+0000","last_active":"2023-08-27T11:19:16.263846+0000","last_peered":"2023-08-27T11:19:16.263846+0000","last_clean":"2023-08-27T11:19:16.263846+0000","last_became_active":"2023-08-27T11:19:16.256133+0000","last_became_peered":"2023-08-27T11:19:16.256133+0000","last_unstale":"2023-08-27T11:19:16.263846+0000","last_undegraded":"2023-08-27T11:19:16.263846+0000","last_fullsized":"2023-08-27T11:19:16.263846+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:30:25.750411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248142+0000","last_change":"2023-08-27T11:19:16.259002+0000","last_active":"2023-08-27T11:19:17.248142+0000","last_peered":"2023-08-27T11:19:17.248142+0000","last_clean":"2023-08-27T11:19:17.248142+0000","last_became_active":"2023-08-27T11:19:16.258740+0000","last_became_peered":"2023-08-27T11:19:16.258740+0000","last_unstale":"2023-08-27T11:19:17.248142+0000","last_undegraded":"2023-08-27T11:19:17.248142+0000","last_fullsized":"2023-08-27T11:19:17.248142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:45:44.045561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.257857+0000","last_change":"2023-08-27T11:19:16.253764+0000","last_active":"2023-08-27T11:19:16.257857+0000","last_peered":"2023-08-27T11:19:16.257857+0000","last_clean":"2023-08-27T11:19:16.257857+0000","last_became_active":"2023-08-27T11:19:16.253523+0000","last_became_peered":"2023-08-27T11:19:16.253523+0000","last_unstale":"2023-08-27T11:19:16.257857+0000","last_undegraded":"2023-08-27T11:19:16.257857+0000","last_fullsized":"2023-08-27T11:19:16.257857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:46:25.794832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.265363+0000","last_change":"2023-08-27T11:19:16.263188+0000","last_active":"2023-08-27T11:19:16.265363+0000","last_peered":"2023-08-27T11:19:16.265363+0000","last_clean":"2023-08-27T11:19:16.265363+0000","last_became_active":"2023-08-27T11:19:16.259623+0000","last_became_peered":"2023-08-27T11:19:16.259623+0000","last_unstale":"2023-08-27T11:19:16.265363+0000","last_undegraded":"2023-08-27T11:19:16.265363+0000","last_fullsized":"2023-08-27T11:19:16.265363+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:16.815337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.259764+0000","last_change":"2023-08-27T11:19:16.255571+0000","last_active":"2023-08-27T11:19:16.259764+0000","last_peered":"2023-08-27T11:19:16.259764+0000","last_clean":"2023-08-27T11:19:16.259764+0000","last_became_active":"2023-08-27T11:19:16.254779+0000","last_became_peered":"2023-08-27T11:19:16.254779+0000","last_unstale":"2023-08-27T11:19:16.259764+0000","last_undegraded":"2023-08-27T11:19:16.259764+0000","last_fullsized":"2023-08-27T11:19:16.259764+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:37:12.312663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262707+0000","last_change":"2023-08-27T11:19:16.257992+0000","last_active":"2023-08-27T11:19:16.262707+0000","last_peered":"2023-08-27T11:19:16.262707+0000","last_clean":"2023-08-27T11:19:16.262707+0000","last_became_active":"2023-08-27T11:19:16.257746+0000","last_became_peered":"2023-08-27T11:19:16.257746+0000","last_unstale":"2023-08-27T11:19:16.262707+0000","last_undegraded":"2023-08-27T11:19:16.262707+0000","last_fullsized":"2023-08-27T11:19:16.262707+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:41:02.020022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264185+0000","last_change":"2023-08-27T11:19:16.261640+0000","last_active":"2023-08-27T11:19:16.264185+0000","last_peered":"2023-08-27T11:19:16.264185+0000","last_clean":"2023-08-27T11:19:16.264185+0000","last_became_active":"2023-08-27T11:19:16.261355+0000","last_became_peered":"2023-08-27T11:19:16.261355+0000","last_unstale":"2023-08-27T11:19:16.264185+0000","last_undegraded":"2023-08-27T11:19:16.264185+0000","last_fullsized":"2023-08-27T11:19:16.264185+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:04.687161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252332+0000","last_change":"2023-08-27T11:19:16.255856+0000","last_active":"2023-08-27T11:19:17.252332+0000","last_peered":"2023-08-27T11:19:17.252332+0000","last_clean":"2023-08-27T11:19:17.252332+0000","last_became_active":"2023-08-27T11:19:16.255386+0000","last_became_peered":"2023-08-27T11:19:16.255386+0000","last_unstale":"2023-08-27T11:19:17.252332+0000","last_undegraded":"2023-08-27T11:19:17.252332+0000","last_fullsized":"2023-08-27T11:19:17.252332+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:31:18.071265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264159+0000","last_change":"2023-08-27T11:19:16.261655+0000","last_active":"2023-08-27T11:19:16.264159+0000","last_peered":"2023-08-27T11:19:16.264159+0000","last_clean":"2023-08-27T11:19:16.264159+0000","last_became_active":"2023-08-27T11:19:16.261418+0000","last_became_peered":"2023-08-27T11:19:16.261418+0000","last_unstale":"2023-08-27T11:19:16.264159+0000","last_undegraded":"2023-08-27T11:19:16.264159+0000","last_fullsized":"2023-08-27T11:19:16.264159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:57:04.356037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.251972+0000","last_change":"2023-08-27T11:19:16.253818+0000","last_active":"2023-08-27T11:19:17.251972+0000","last_peered":"2023-08-27T11:19:17.251972+0000","last_clean":"2023-08-27T11:19:17.251972+0000","last_became_active":"2023-08-27T11:19:16.253523+0000","last_became_peered":"2023-08-27T11:19:16.253523+0000","last_unstale":"2023-08-27T11:19:17.251972+0000","last_undegraded":"2023-08-27T11:19:17.251972+0000","last_fullsized":"2023-08-27T11:19:17.251972+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:04:27.242470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248381+0000","last_change":"2023-08-27T11:19:16.251666+0000","last_active":"2023-08-27T11:19:17.248381+0000","last_peered":"2023-08-27T11:19:17.248381+0000","last_clean":"2023-08-27T11:19:17.248381+0000","last_became_active":"2023-08-27T11:19:16.251405+0000","last_became_peered":"2023-08-27T11:19:16.251405+0000","last_unstale":"2023-08-27T11:19:17.248381+0000","last_undegraded":"2023-08-27T11:19:17.248381+0000","last_fullsized":"2023-08-27T11:19:17.248381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:29:19.506410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248372+0000","last_change":"2023-08-27T11:19:16.254142+0000","last_active":"2023-08-27T11:19:17.248372+0000","last_peered":"2023-08-27T11:19:17.248372+0000","last_clean":"2023-08-27T11:19:17.248372+0000","last_became_active":"2023-08-27T11:19:16.253894+0000","last_became_peered":"2023-08-27T11:19:16.253894+0000","last_unstale":"2023-08-27T11:19:17.248372+0000","last_undegraded":"2023-08-27T11:19:17.248372+0000","last_fullsized":"2023-08-27T11:19:17.248372+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:27:49.277713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262346+0000","last_change":"2023-08-27T11:19:16.257289+0000","last_active":"2023-08-27T11:19:16.262346+0000","last_peered":"2023-08-27T11:19:16.262346+0000","last_clean":"2023-08-27T11:19:16.262346+0000","last_became_active":"2023-08-27T11:19:16.257053+0000","last_became_peered":"2023-08-27T11:19:16.257053+0000","last_unstale":"2023-08-27T11:19:16.262346+0000","last_undegraded":"2023-08-27T11:19:16.262346+0000","last_fullsized":"2023-08-27T11:19:16.262346+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:54.446601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.257782+0000","last_change":"2023-08-27T11:19:16.253795+0000","last_active":"2023-08-27T11:19:16.257782+0000","last_peered":"2023-08-27T11:19:16.257782+0000","last_clean":"2023-08-27T11:19:16.257782+0000","last_became_active":"2023-08-27T11:19:16.253485+0000","last_became_peered":"2023-08-27T11:19:16.253485+0000","last_unstale":"2023-08-27T11:19:16.257782+0000","last_undegraded":"2023-08-27T11:19:16.257782+0000","last_fullsized":"2023-08-27T11:19:16.257782+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:03:41.841855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.263666+0000","last_change":"2023-08-27T11:19:16.256266+0000","last_active":"2023-08-27T11:19:16.263666+0000","last_peered":"2023-08-27T11:19:16.263666+0000","last_clean":"2023-08-27T11:19:16.263666+0000","last_became_active":"2023-08-27T11:19:16.256014+0000","last_became_peered":"2023-08-27T11:19:16.256014+0000","last_unstale":"2023-08-27T11:19:16.263666+0000","last_undegraded":"2023-08-27T11:19:16.263666+0000","last_fullsized":"2023-08-27T11:19:16.263666+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:18:12.235459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248315+0000","last_change":"2023-08-27T11:19:16.259027+0000","last_active":"2023-08-27T11:19:17.248315+0000","last_peered":"2023-08-27T11:19:17.248315+0000","last_clean":"2023-08-27T11:19:17.248315+0000","last_became_active":"2023-08-27T11:19:16.258748+0000","last_became_peered":"2023-08-27T11:19:16.258748+0000","last_unstale":"2023-08-27T11:19:17.248315+0000","last_undegraded":"2023-08-27T11:19:17.248315+0000","last_fullsized":"2023-08-27T11:19:17.248315+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:19:05.373592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262731+0000","last_change":"2023-08-27T11:19:16.260021+0000","last_active":"2023-08-27T11:19:16.262731+0000","last_peered":"2023-08-27T11:19:16.262731+0000","last_clean":"2023-08-27T11:19:16.262731+0000","last_became_active":"2023-08-27T11:19:16.259405+0000","last_became_peered":"2023-08-27T11:19:16.259405+0000","last_unstale":"2023-08-27T11:19:16.262731+0000","last_undegraded":"2023-08-27T11:19:16.262731+0000","last_fullsized":"2023-08-27T11:19:16.262731+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:32:50.423067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264786+0000","last_change":"2023-08-27T11:19:16.260535+0000","last_active":"2023-08-27T11:19:16.264786+0000","last_peered":"2023-08-27T11:19:16.264786+0000","last_clean":"2023-08-27T11:19:16.264786+0000","last_became_active":"2023-08-27T11:19:16.260207+0000","last_became_peered":"2023-08-27T11:19:16.260207+0000","last_unstale":"2023-08-27T11:19:16.264786+0000","last_undegraded":"2023-08-27T11:19:16.264786+0000","last_fullsized":"2023-08-27T11:19:16.264786+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:15:55.910012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.257874+0000","last_change":"2023-08-27T11:19:16.255856+0000","last_active":"2023-08-27T11:19:16.257874+0000","last_peered":"2023-08-27T11:19:16.257874+0000","last_clean":"2023-08-27T11:19:16.257874+0000","last_became_active":"2023-08-27T11:19:16.255580+0000","last_became_peered":"2023-08-27T11:19:16.255580+0000","last_unstale":"2023-08-27T11:19:16.257874+0000","last_undegraded":"2023-08-27T11:19:16.257874+0000","last_fullsized":"2023-08-27T11:19:16.257874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:34:49.226624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.256538+0000","last_change":"2023-08-27T11:19:16.254831+0000","last_active":"2023-08-27T11:19:16.256538+0000","last_peered":"2023-08-27T11:19:16.256538+0000","last_clean":"2023-08-27T11:19:16.256538+0000","last_became_active":"2023-08-27T11:19:16.254557+0000","last_became_peered":"2023-08-27T11:19:16.254557+0000","last_unstale":"2023-08-27T11:19:16.256538+0000","last_undegraded":"2023-08-27T11:19:16.256538+0000","last_fullsized":"2023-08-27T11:19:16.256538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:05:24.801073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"64'4","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.677440+0000","last_change":"2023-08-27T11:19:15.681714+0000","last_active":"2023-08-27T11:19:16.677440+0000","last_peered":"2023-08-27T11:19:16.677440+0000","last_clean":"2023-08-27T11:19:16.677440+0000","last_became_active":"2023-08-27T11:19:15.681225+0000","last_became_peered":"2023-08-27T11:19:15.681225+0000","last_unstale":"2023-08-27T11:19:16.677440+0000","last_undegraded":"2023-08-27T11:19:16.677440+0000","last_fullsized":"2023-08-27T11:19:16.677440+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:08:18.126251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248267+0000","last_change":"2023-08-27T11:19:16.258259+0000","last_active":"2023-08-27T11:19:17.248267+0000","last_peered":"2023-08-27T11:19:17.248267+0000","last_clean":"2023-08-27T11:19:17.248267+0000","last_became_active":"2023-08-27T11:19:16.257986+0000","last_became_peered":"2023-08-27T11:19:16.257986+0000","last_unstale":"2023-08-27T11:19:17.248267+0000","last_undegraded":"2023-08-27T11:19:17.248267+0000","last_fullsized":"2023-08-27T11:19:17.248267+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:03:02.291266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.678064+0000","last_change":"2023-08-27T11:19:15.678625+0000","last_active":"2023-08-27T11:19:16.678064+0000","last_peered":"2023-08-27T11:19:16.678064+0000","last_clean":"2023-08-27T11:19:16.678064+0000","last_became_active":"2023-08-27T11:19:15.678324+0000","last_became_peered":"2023-08-27T11:19:15.678324+0000","last_unstale":"2023-08-27T11:19:16.678064+0000","last_undegraded":"2023-08-27T11:19:16.678064+0000","last_fullsized":"2023-08-27T11:19:16.678064+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:56:07.395501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252341+0000","last_change":"2023-08-27T11:19:16.253871+0000","last_active":"2023-08-27T11:19:17.252341+0000","last_peered":"2023-08-27T11:19:17.252341+0000","last_clean":"2023-08-27T11:19:17.252341+0000","last_became_active":"2023-08-27T11:19:16.252750+0000","last_became_peered":"2023-08-27T11:19:16.252750+0000","last_unstale":"2023-08-27T11:19:17.252341+0000","last_undegraded":"2023-08-27T11:19:17.252341+0000","last_fullsized":"2023-08-27T11:19:17.252341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:02:35.688837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.676672+0000","last_change":"2023-08-27T11:19:15.681717+0000","last_active":"2023-08-27T11:19:16.676672+0000","last_peered":"2023-08-27T11:19:16.676672+0000","last_clean":"2023-08-27T11:19:16.676672+0000","last_became_active":"2023-08-27T11:19:15.681418+0000","last_became_peered":"2023-08-27T11:19:15.681418+0000","last_unstale":"2023-08-27T11:19:16.676672+0000","last_undegraded":"2023-08-27T11:19:16.676672+0000","last_fullsized":"2023-08-27T11:19:16.676672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:38.354190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252615+0000","last_change":"2023-08-27T11:19:16.255651+0000","last_active":"2023-08-27T11:19:17.252615+0000","last_peered":"2023-08-27T11:19:17.252615+0000","last_clean":"2023-08-27T11:19:17.252615+0000","last_became_active":"2023-08-27T11:19:16.255028+0000","last_became_peered":"2023-08-27T11:19:16.255028+0000","last_unstale":"2023-08-27T11:19:17.252615+0000","last_undegraded":"2023-08-27T11:19:17.252615+0000","last_fullsized":"2023-08-27T11:19:17.252615+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:27:05.200585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"64'2","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252387+0000","last_change":"2023-08-27T11:19:15.675923+0000","last_active":"2023-08-27T11:19:17.252387+0000","last_peered":"2023-08-27T11:19:17.252387+0000","last_clean":"2023-08-27T11:19:17.252387+0000","last_became_active":"2023-08-27T11:19:15.675598+0000","last_became_peered":"2023-08-27T11:19:15.675598+0000","last_unstale":"2023-08-27T11:19:17.252387+0000","last_undegraded":"2023-08-27T11:19:17.252387+0000","last_fullsized":"2023-08-27T11:19:17.252387+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:13:54.624802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248476+0000","last_change":"2023-08-27T11:19:16.263309+0000","last_active":"2023-08-27T11:19:17.248476+0000","last_peered":"2023-08-27T11:19:17.248476+0000","last_clean":"2023-08-27T11:19:17.248476+0000","last_became_active":"2023-08-27T11:19:16.262930+0000","last_became_peered":"2023-08-27T11:19:16.262930+0000","last_unstale":"2023-08-27T11:19:17.248476+0000","last_undegraded":"2023-08-27T11:19:17.248476+0000","last_fullsized":"2023-08-27T11:19:17.248476+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:19:02.011264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"64'1","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252762+0000","last_change":"2023-08-27T11:19:15.684528+0000","last_active":"2023-08-27T11:19:17.252762+0000","last_peered":"2023-08-27T11:19:17.252762+0000","last_clean":"2023-08-27T11:19:17.252762+0000","last_became_active":"2023-08-27T11:19:15.684220+0000","last_became_peered":"2023-08-27T11:19:15.684220+0000","last_unstale":"2023-08-27T11:19:17.252762+0000","last_undegraded":"2023-08-27T11:19:17.252762+0000","last_fullsized":"2023-08-27T11:19:17.252762+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:44:35.580620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248613+0000","last_change":"2023-08-27T11:19:16.260311+0000","last_active":"2023-08-27T11:19:17.248613+0000","last_peered":"2023-08-27T11:19:17.248613+0000","last_clean":"2023-08-27T11:19:17.248613+0000","last_became_active":"2023-08-27T11:19:16.260059+0000","last_became_peered":"2023-08-27T11:19:16.260059+0000","last_unstale":"2023-08-27T11:19:17.248613+0000","last_undegraded":"2023-08-27T11:19:17.248613+0000","last_fullsized":"2023-08-27T11:19:17.248613+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:22:56.884278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.674584+0000","last_change":"2023-08-27T11:19:15.681037+0000","last_active":"2023-08-27T11:19:16.674584+0000","last_peered":"2023-08-27T11:19:16.674584+0000","last_clean":"2023-08-27T11:19:16.674584+0000","last_became_active":"2023-08-27T11:19:15.680521+0000","last_became_peered":"2023-08-27T11:19:15.680521+0000","last_unstale":"2023-08-27T11:19:16.674584+0000","last_undegraded":"2023-08-27T11:19:16.674584+0000","last_fullsized":"2023-08-27T11:19:16.674584+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:49:02.157224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252531+0000","last_change":"2023-08-27T11:19:16.251391+0000","last_active":"2023-08-27T11:19:17.252531+0000","last_peered":"2023-08-27T11:19:17.252531+0000","last_clean":"2023-08-27T11:19:17.252531+0000","last_became_active":"2023-08-27T11:19:16.251068+0000","last_became_peered":"2023-08-27T11:19:16.251068+0000","last_unstale":"2023-08-27T11:19:17.252531+0000","last_undegraded":"2023-08-27T11:19:17.252531+0000","last_fullsized":"2023-08-27T11:19:17.252531+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:32:38.024405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248226+0000","last_change":"2023-08-27T11:19:15.684238+0000","last_active":"2023-08-27T11:19:17.248226+0000","last_peered":"2023-08-27T11:19:17.248226+0000","last_clean":"2023-08-27T11:19:17.248226+0000","last_became_active":"2023-08-27T11:19:15.683983+0000","last_became_peered":"2023-08-27T11:19:15.683983+0000","last_unstale":"2023-08-27T11:19:17.248226+0000","last_undegraded":"2023-08-27T11:19:17.248226+0000","last_fullsized":"2023-08-27T11:19:17.248226+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:00.357426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252536+0000","last_change":"2023-08-27T11:19:16.260858+0000","last_active":"2023-08-27T11:19:17.252536+0000","last_peered":"2023-08-27T11:19:17.252536+0000","last_clean":"2023-08-27T11:19:17.252536+0000","last_became_active":"2023-08-27T11:19:16.260544+0000","last_became_peered":"2023-08-27T11:19:16.260544+0000","last_unstale":"2023-08-27T11:19:17.252536+0000","last_undegraded":"2023-08-27T11:19:17.252536+0000","last_fullsized":"2023-08-27T11:19:17.252536+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:13:45.345824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.250170+0000","last_change":"2023-08-27T11:19:15.681421+0000","last_active":"2023-08-27T11:19:16.250170+0000","last_peered":"2023-08-27T11:19:16.250170+0000","last_clean":"2023-08-27T11:19:16.250170+0000","last_became_active":"2023-08-27T11:19:15.681119+0000","last_became_peered":"2023-08-27T11:19:15.681119+0000","last_unstale":"2023-08-27T11:19:16.250170+0000","last_undegraded":"2023-08-27T11:19:16.250170+0000","last_fullsized":"2023-08-27T11:19:16.250170+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:26.109313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.247986+0000","last_change":"2023-08-27T11:19:16.258682+0000","last_active":"2023-08-27T11:19:17.247986+0000","last_peered":"2023-08-27T11:19:17.247986+0000","last_clean":"2023-08-27T11:19:17.247986+0000","last_became_active":"2023-08-27T11:19:16.258421+0000","last_became_peered":"2023-08-27T11:19:16.258421+0000","last_unstale":"2023-08-27T11:19:17.247986+0000","last_undegraded":"2023-08-27T11:19:17.247986+0000","last_fullsized":"2023-08-27T11:19:17.247986+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:34:02.411369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.250399+0000","last_change":"2023-08-27T11:19:15.682371+0000","last_active":"2023-08-27T11:19:16.250399+0000","last_peered":"2023-08-27T11:19:16.250399+0000","last_clean":"2023-08-27T11:19:16.250399+0000","last_became_active":"2023-08-27T11:19:15.682092+0000","last_became_peered":"2023-08-27T11:19:15.682092+0000","last_unstale":"2023-08-27T11:19:16.250399+0000","last_undegraded":"2023-08-27T11:19:16.250399+0000","last_fullsized":"2023-08-27T11:19:16.250399+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:58.739608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.258758+0000","last_change":"2023-08-27T11:19:16.255461+0000","last_active":"2023-08-27T11:19:16.258758+0000","last_peered":"2023-08-27T11:19:16.258758+0000","last_clean":"2023-08-27T11:19:16.258758+0000","last_became_active":"2023-08-27T11:19:16.254827+0000","last_became_peered":"2023-08-27T11:19:16.254827+0000","last_unstale":"2023-08-27T11:19:16.258758+0000","last_undegraded":"2023-08-27T11:19:16.258758+0000","last_fullsized":"2023-08-27T11:19:16.258758+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:06.746338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.249380+0000","last_change":"2023-08-27T11:19:15.677219+0000","last_active":"2023-08-27T11:19:16.249380+0000","last_peered":"2023-08-27T11:19:16.249380+0000","last_clean":"2023-08-27T11:19:16.249380+0000","last_became_active":"2023-08-27T11:19:15.676685+0000","last_became_peered":"2023-08-27T11:19:15.676685+0000","last_unstale":"2023-08-27T11:19:16.249380+0000","last_undegraded":"2023-08-27T11:19:16.249380+0000","last_fullsized":"2023-08-27T11:19:16.249380+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:19.489206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.263822+0000","last_change":"2023-08-27T11:19:16.256966+0000","last_active":"2023-08-27T11:19:16.263822+0000","last_peered":"2023-08-27T11:19:16.263822+0000","last_clean":"2023-08-27T11:19:16.263822+0000","last_became_active":"2023-08-27T11:19:16.255805+0000","last_became_peered":"2023-08-27T11:19:16.255805+0000","last_unstale":"2023-08-27T11:19:16.263822+0000","last_undegraded":"2023-08-27T11:19:16.263822+0000","last_fullsized":"2023-08-27T11:19:16.263822+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:53:18.080063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"64'3","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.676214+0000","last_change":"2023-08-27T11:19:15.674041+0000","last_active":"2023-08-27T11:19:16.676214+0000","last_peered":"2023-08-27T11:19:16.676214+0000","last_clean":"2023-08-27T11:19:16.676214+0000","last_became_active":"2023-08-27T11:19:15.673759+0000","last_became_peered":"2023-08-27T11:19:15.673759+0000","last_unstale":"2023-08-27T11:19:16.676214+0000","last_undegraded":"2023-08-27T11:19:16.676214+0000","last_fullsized":"2023-08-27T11:19:16.676214+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:08:52.207889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264766+0000","last_change":"2023-08-27T11:19:16.260619+0000","last_active":"2023-08-27T11:19:16.264766+0000","last_peered":"2023-08-27T11:19:16.264766+0000","last_clean":"2023-08-27T11:19:16.264766+0000","last_became_active":"2023-08-27T11:19:16.260346+0000","last_became_peered":"2023-08-27T11:19:16.260346+0000","last_unstale":"2023-08-27T11:19:16.264766+0000","last_undegraded":"2023-08-27T11:19:16.264766+0000","last_fullsized":"2023-08-27T11:19:16.264766+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:04:19.737862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"64'6","reported_seq":19,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252509+0000","last_change":"2023-08-27T11:19:15.681619+0000","last_active":"2023-08-27T11:19:17.252509+0000","last_peered":"2023-08-27T11:19:17.252509+0000","last_clean":"2023-08-27T11:19:17.252509+0000","last_became_active":"2023-08-27T11:19:15.681317+0000","last_became_peered":"2023-08-27T11:19:15.681317+0000","last_unstale":"2023-08-27T11:19:17.252509+0000","last_undegraded":"2023-08-27T11:19:17.252509+0000","last_fullsized":"2023-08-27T11:19:17.252509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:54:22.388631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.262911+0000","last_change":"2023-08-27T11:19:16.257958+0000","last_active":"2023-08-27T11:19:16.262911+0000","last_peered":"2023-08-27T11:19:16.262911+0000","last_clean":"2023-08-27T11:19:16.262911+0000","last_became_active":"2023-08-27T11:19:16.257670+0000","last_became_peered":"2023-08-27T11:19:16.257670+0000","last_unstale":"2023-08-27T11:19:16.262911+0000","last_undegraded":"2023-08-27T11:19:16.262911+0000","last_fullsized":"2023-08-27T11:19:16.262911+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:07:26.067523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.677525+0000","last_change":"2023-08-27T11:19:15.681748+0000","last_active":"2023-08-27T11:19:16.677525+0000","last_peered":"2023-08-27T11:19:16.677525+0000","last_clean":"2023-08-27T11:19:16.677525+0000","last_became_active":"2023-08-27T11:19:15.681468+0000","last_became_peered":"2023-08-27T11:19:15.681468+0000","last_unstale":"2023-08-27T11:19:16.677525+0000","last_undegraded":"2023-08-27T11:19:16.677525+0000","last_fullsized":"2023-08-27T11:19:16.677525+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:31.326789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252098+0000","last_change":"2023-08-27T11:19:16.254735+0000","last_active":"2023-08-27T11:19:17.252098+0000","last_peered":"2023-08-27T11:19:17.252098+0000","last_clean":"2023-08-27T11:19:17.252098+0000","last_became_active":"2023-08-27T11:19:16.254460+0000","last_became_peered":"2023-08-27T11:19:16.254460+0000","last_unstale":"2023-08-27T11:19:17.252098+0000","last_undegraded":"2023-08-27T11:19:17.252098+0000","last_fullsized":"2023-08-27T11:19:17.252098+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:47:27.241362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.246749+0000","last_change":"2023-08-27T11:19:15.680755+0000","last_active":"2023-08-27T11:19:16.246749+0000","last_peered":"2023-08-27T11:19:16.246749+0000","last_clean":"2023-08-27T11:19:16.246749+0000","last_became_active":"2023-08-27T11:19:15.680442+0000","last_became_peered":"2023-08-27T11:19:15.680442+0000","last_unstale":"2023-08-27T11:19:16.246749+0000","last_undegraded":"2023-08-27T11:19:16.246749+0000","last_fullsized":"2023-08-27T11:19:16.246749+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:07:07.837430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252693+0000","last_change":"2023-08-27T11:19:16.252681+0000","last_active":"2023-08-27T11:19:17.252693+0000","last_peered":"2023-08-27T11:19:17.252693+0000","last_clean":"2023-08-27T11:19:17.252693+0000","last_became_active":"2023-08-27T11:19:16.252361+0000","last_became_peered":"2023-08-27T11:19:16.252361+0000","last_unstale":"2023-08-27T11:19:17.252693+0000","last_undegraded":"2023-08-27T11:19:17.252693+0000","last_fullsized":"2023-08-27T11:19:17.252693+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:46:50.752398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.250640+0000","last_change":"2023-08-27T11:19:15.682094+0000","last_active":"2023-08-27T11:19:16.250640+0000","last_peered":"2023-08-27T11:19:16.250640+0000","last_clean":"2023-08-27T11:19:16.250640+0000","last_became_active":"2023-08-27T11:19:15.681828+0000","last_became_peered":"2023-08-27T11:19:15.681828+0000","last_unstale":"2023-08-27T11:19:16.250640+0000","last_undegraded":"2023-08-27T11:19:16.250640+0000","last_fullsized":"2023-08-27T11:19:16.250640+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:55:45.379063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252883+0000","last_change":"2023-08-27T11:19:16.252126+0000","last_active":"2023-08-27T11:19:17.252883+0000","last_peered":"2023-08-27T11:19:17.252883+0000","last_clean":"2023-08-27T11:19:17.252883+0000","last_became_active":"2023-08-27T11:19:16.251355+0000","last_became_peered":"2023-08-27T11:19:16.251355+0000","last_unstale":"2023-08-27T11:19:17.252883+0000","last_undegraded":"2023-08-27T11:19:17.252883+0000","last_fullsized":"2023-08-27T11:19:17.252883+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:31:18.071265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"64'8","reported_seq":21,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.247993+0000","last_change":"2023-08-27T11:19:15.676573+0000","last_active":"2023-08-27T11:19:17.247993+0000","last_peered":"2023-08-27T11:19:17.247993+0000","last_clean":"2023-08-27T11:19:17.247993+0000","last_became_active":"2023-08-27T11:19:15.676300+0000","last_became_peered":"2023-08-27T11:19:15.676300+0000","last_unstale":"2023-08-27T11:19:17.247993+0000","last_undegraded":"2023-08-27T11:19:17.247993+0000","last_fullsized":"2023-08-27T11:19:17.247993+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:16:52.142744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.251778+0000","last_change":"2023-08-27T11:19:16.249802+0000","last_active":"2023-08-27T11:19:17.251778+0000","last_peered":"2023-08-27T11:19:17.251778+0000","last_clean":"2023-08-27T11:19:17.251778+0000","last_became_active":"2023-08-27T11:19:16.249497+0000","last_became_peered":"2023-08-27T11:19:16.249497+0000","last_unstale":"2023-08-27T11:19:17.251778+0000","last_undegraded":"2023-08-27T11:19:17.251778+0000","last_fullsized":"2023-08-27T11:19:17.251778+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:55:24.372765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.676382+0000","last_change":"2023-08-27T11:19:15.679666+0000","last_active":"2023-08-27T11:19:16.676382+0000","last_peered":"2023-08-27T11:19:16.676382+0000","last_clean":"2023-08-27T11:19:16.676382+0000","last_became_active":"2023-08-27T11:19:15.679378+0000","last_became_peered":"2023-08-27T11:19:15.679378+0000","last_unstale":"2023-08-27T11:19:16.676382+0000","last_undegraded":"2023-08-27T11:19:16.676382+0000","last_fullsized":"2023-08-27T11:19:16.676382+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:47:44.577411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252559+0000","last_change":"2023-08-27T11:19:16.251660+0000","last_active":"2023-08-27T11:19:17.252559+0000","last_peered":"2023-08-27T11:19:17.252559+0000","last_clean":"2023-08-27T11:19:17.252559+0000","last_became_active":"2023-08-27T11:19:16.251387+0000","last_became_peered":"2023-08-27T11:19:16.251387+0000","last_unstale":"2023-08-27T11:19:17.252559+0000","last_undegraded":"2023-08-27T11:19:17.252559+0000","last_fullsized":"2023-08-27T11:19:17.252559+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:55:24.372765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248712+0000","last_change":"2023-08-27T11:19:15.677246+0000","last_active":"2023-08-27T11:19:17.248712+0000","last_peered":"2023-08-27T11:19:17.248712+0000","last_clean":"2023-08-27T11:19:17.248712+0000","last_became_active":"2023-08-27T11:19:15.676964+0000","last_became_peered":"2023-08-27T11:19:15.676964+0000","last_unstale":"2023-08-27T11:19:17.248712+0000","last_undegraded":"2023-08-27T11:19:17.248712+0000","last_fullsized":"2023-08-27T11:19:17.248712+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:43:44.164338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252294+0000","last_change":"2023-08-27T11:19:16.253130+0000","last_active":"2023-08-27T11:19:17.252294+0000","last_peered":"2023-08-27T11:19:17.252294+0000","last_clean":"2023-08-27T11:19:17.252294+0000","last_became_active":"2023-08-27T11:19:16.252627+0000","last_became_peered":"2023-08-27T11:19:16.252627+0000","last_unstale":"2023-08-27T11:19:17.252294+0000","last_undegraded":"2023-08-27T11:19:17.252294+0000","last_fullsized":"2023-08-27T11:19:17.252294+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:00:38.245757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.250933+0000","last_change":"2023-08-27T11:19:15.683177+0000","last_active":"2023-08-27T11:19:16.250933+0000","last_peered":"2023-08-27T11:19:16.250933+0000","last_clean":"2023-08-27T11:19:16.250933+0000","last_became_active":"2023-08-27T11:19:15.682862+0000","last_became_peered":"2023-08-27T11:19:15.682862+0000","last_unstale":"2023-08-27T11:19:16.250933+0000","last_undegraded":"2023-08-27T11:19:16.250933+0000","last_fullsized":"2023-08-27T11:19:16.250933+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:22:08.945339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248451+0000","last_change":"2023-08-27T11:19:16.253111+0000","last_active":"2023-08-27T11:19:17.248451+0000","last_peered":"2023-08-27T11:19:17.248451+0000","last_clean":"2023-08-27T11:19:17.248451+0000","last_became_active":"2023-08-27T11:19:16.252868+0000","last_became_peered":"2023-08-27T11:19:16.252868+0000","last_unstale":"2023-08-27T11:19:17.248451+0000","last_undegraded":"2023-08-27T11:19:17.248451+0000","last_fullsized":"2023-08-27T11:19:17.248451+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:24:29.530359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.249844+0000","last_change":"2023-08-27T11:19:15.678311+0000","last_active":"2023-08-27T11:19:16.249844+0000","last_peered":"2023-08-27T11:19:16.249844+0000","last_clean":"2023-08-27T11:19:16.249844+0000","last_became_active":"2023-08-27T11:19:15.677933+0000","last_became_peered":"2023-08-27T11:19:15.677933+0000","last_unstale":"2023-08-27T11:19:16.249844+0000","last_undegraded":"2023-08-27T11:19:16.249844+0000","last_fullsized":"2023-08-27T11:19:16.249844+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:41:50.956554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252198+0000","last_change":"2023-08-27T11:19:16.252009+0000","last_active":"2023-08-27T11:19:17.252198+0000","last_peered":"2023-08-27T11:19:17.252198+0000","last_clean":"2023-08-27T11:19:17.252198+0000","last_became_active":"2023-08-27T11:19:16.251709+0000","last_became_peered":"2023-08-27T11:19:16.251709+0000","last_unstale":"2023-08-27T11:19:17.252198+0000","last_undegraded":"2023-08-27T11:19:17.252198+0000","last_fullsized":"2023-08-27T11:19:17.252198+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:22:05.047563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252599+0000","last_change":"2023-08-27T11:19:15.675556+0000","last_active":"2023-08-27T11:19:17.252599+0000","last_peered":"2023-08-27T11:19:17.252599+0000","last_clean":"2023-08-27T11:19:17.252599+0000","last_became_active":"2023-08-27T11:19:15.675241+0000","last_became_peered":"2023-08-27T11:19:15.675241+0000","last_unstale":"2023-08-27T11:19:17.252599+0000","last_undegraded":"2023-08-27T11:19:17.252599+0000","last_fullsized":"2023-08-27T11:19:17.252599+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:04:26.237695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252244+0000","last_change":"2023-08-27T11:19:16.260556+0000","last_active":"2023-08-27T11:19:17.252244+0000","last_peered":"2023-08-27T11:19:17.252244+0000","last_clean":"2023-08-27T11:19:17.252244+0000","last_became_active":"2023-08-27T11:19:16.260269+0000","last_became_peered":"2023-08-27T11:19:16.260269+0000","last_unstale":"2023-08-27T11:19:17.252244+0000","last_undegraded":"2023-08-27T11:19:17.252244+0000","last_fullsized":"2023-08-27T11:19:17.252244+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:10:36.759506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.250569+0000","last_change":"2023-08-27T11:19:15.682286+0000","last_active":"2023-08-27T11:19:16.250569+0000","last_peered":"2023-08-27T11:19:16.250569+0000","last_clean":"2023-08-27T11:19:16.250569+0000","last_became_active":"2023-08-27T11:19:15.681833+0000","last_became_peered":"2023-08-27T11:19:15.681833+0000","last_unstale":"2023-08-27T11:19:16.250569+0000","last_undegraded":"2023-08-27T11:19:16.250569+0000","last_fullsized":"2023-08-27T11:19:16.250569+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:03:53.791487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264780+0000","last_change":"2023-08-27T11:19:16.261007+0000","last_active":"2023-08-27T11:19:16.264780+0000","last_peered":"2023-08-27T11:19:16.264780+0000","last_clean":"2023-08-27T11:19:16.264780+0000","last_became_active":"2023-08-27T11:19:16.260756+0000","last_became_peered":"2023-08-27T11:19:16.260756+0000","last_unstale":"2023-08-27T11:19:16.264780+0000","last_undegraded":"2023-08-27T11:19:16.264780+0000","last_fullsized":"2023-08-27T11:19:16.264780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:45:32.872489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252710+0000","last_change":"2023-08-27T11:19:15.681743+0000","last_active":"2023-08-27T11:19:17.252710+0000","last_peered":"2023-08-27T11:19:17.252710+0000","last_clean":"2023-08-27T11:19:17.252710+0000","last_became_active":"2023-08-27T11:19:15.681474+0000","last_became_peered":"2023-08-27T11:19:15.681474+0000","last_unstale":"2023-08-27T11:19:17.252710+0000","last_undegraded":"2023-08-27T11:19:17.252710+0000","last_fullsized":"2023-08-27T11:19:17.252710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:31:30.712591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.255361+0000","last_change":"2023-08-27T11:19:16.253638+0000","last_active":"2023-08-27T11:19:16.255361+0000","last_peered":"2023-08-27T11:19:16.255361+0000","last_clean":"2023-08-27T11:19:16.255361+0000","last_became_active":"2023-08-27T11:19:16.253420+0000","last_became_peered":"2023-08-27T11:19:16.253420+0000","last_unstale":"2023-08-27T11:19:16.255361+0000","last_undegraded":"2023-08-27T11:19:16.255361+0000","last_fullsized":"2023-08-27T11:19:16.255361+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:48:26.533059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.246125+0000","last_change":"2023-08-27T11:19:15.680648+0000","last_active":"2023-08-27T11:19:16.246125+0000","last_peered":"2023-08-27T11:19:16.246125+0000","last_clean":"2023-08-27T11:19:16.246125+0000","last_became_active":"2023-08-27T11:19:15.680400+0000","last_became_peered":"2023-08-27T11:19:15.680400+0000","last_unstale":"2023-08-27T11:19:16.246125+0000","last_undegraded":"2023-08-27T11:19:16.246125+0000","last_fullsized":"2023-08-27T11:19:16.246125+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:52:35.980582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264082+0000","last_change":"2023-08-27T11:19:16.259158+0000","last_active":"2023-08-27T11:19:16.264082+0000","last_peered":"2023-08-27T11:19:16.264082+0000","last_clean":"2023-08-27T11:19:16.264082+0000","last_became_active":"2023-08-27T11:19:16.258918+0000","last_became_peered":"2023-08-27T11:19:16.258918+0000","last_unstale":"2023-08-27T11:19:16.264082+0000","last_undegraded":"2023-08-27T11:19:16.264082+0000","last_fullsized":"2023-08-27T11:19:16.264082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:25:11.321371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.677742+0000","last_change":"2023-08-27T11:19:15.680156+0000","last_active":"2023-08-27T11:19:16.677742+0000","last_peered":"2023-08-27T11:19:16.677742+0000","last_clean":"2023-08-27T11:19:16.677742+0000","last_became_active":"2023-08-27T11:19:15.679591+0000","last_became_peered":"2023-08-27T11:19:15.679591+0000","last_unstale":"2023-08-27T11:19:16.677742+0000","last_undegraded":"2023-08-27T11:19:16.677742+0000","last_fullsized":"2023-08-27T11:19:16.677742+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:22:02.314053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252849+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:19:17.252849+0000","last_peered":"2023-08-27T11:19:17.252849+0000","last_clean":"2023-08-27T11:19:17.252849+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:19:17.252849+0000","last_undegraded":"2023-08-27T11:19:17.252849+0000","last_fullsized":"2023-08-27T11:19:17.252849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:00:18.336409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264010+0000","last_change":"2023-08-27T11:19:16.260953+0000","last_active":"2023-08-27T11:19:16.264010+0000","last_peered":"2023-08-27T11:19:16.264010+0000","last_clean":"2023-08-27T11:19:16.264010+0000","last_became_active":"2023-08-27T11:19:16.260700+0000","last_became_peered":"2023-08-27T11:19:16.260700+0000","last_unstale":"2023-08-27T11:19:16.264010+0000","last_undegraded":"2023-08-27T11:19:16.264010+0000","last_fullsized":"2023-08-27T11:19:16.264010+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:22:14.482419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.245733+0000","last_change":"2023-08-27T11:19:15.674181+0000","last_active":"2023-08-27T11:19:16.245733+0000","last_peered":"2023-08-27T11:19:16.245733+0000","last_clean":"2023-08-27T11:19:16.245733+0000","last_became_active":"2023-08-27T11:19:15.673928+0000","last_became_peered":"2023-08-27T11:19:15.673928+0000","last_unstale":"2023-08-27T11:19:16.245733+0000","last_undegraded":"2023-08-27T11:19:16.245733+0000","last_fullsized":"2023-08-27T11:19:16.245733+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:13.603402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252781+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:19:17.252781+0000","last_peered":"2023-08-27T11:19:17.252781+0000","last_clean":"2023-08-27T11:19:17.252781+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:19:17.252781+0000","last_undegraded":"2023-08-27T11:19:17.252781+0000","last_fullsized":"2023-08-27T11:19:17.252781+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:57:17.819601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248426+0000","last_change":"2023-08-27T11:19:16.259944+0000","last_active":"2023-08-27T11:19:17.248426+0000","last_peered":"2023-08-27T11:19:17.248426+0000","last_clean":"2023-08-27T11:19:17.248426+0000","last_became_active":"2023-08-27T11:19:16.259657+0000","last_became_peered":"2023-08-27T11:19:16.259657+0000","last_unstale":"2023-08-27T11:19:17.248426+0000","last_undegraded":"2023-08-27T11:19:17.248426+0000","last_fullsized":"2023-08-27T11:19:17.248426+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:37:02.028909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.675869+0000","last_change":"2023-08-27T11:19:15.678064+0000","last_active":"2023-08-27T11:19:16.675869+0000","last_peered":"2023-08-27T11:19:16.675869+0000","last_clean":"2023-08-27T11:19:16.675869+0000","last_became_active":"2023-08-27T11:19:15.677804+0000","last_became_peered":"2023-08-27T11:19:15.677804+0000","last_unstale":"2023-08-27T11:19:16.675869+0000","last_undegraded":"2023-08-27T11:19:16.675869+0000","last_fullsized":"2023-08-27T11:19:16.675869+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:07:25.062747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":90,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.247284+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:19:16.247284+0000","last_peered":"2023-08-27T11:19:16.247284+0000","last_clean":"2023-08-27T11:19:16.247284+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:19:16.247284+0000","last_undegraded":"2023-08-27T11:19:16.247284+0000","last_fullsized":"2023-08-27T11:19:16.247284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:38:44.667594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.257801+0000","last_change":"2023-08-27T11:19:16.256112+0000","last_active":"2023-08-27T11:19:16.257801+0000","last_peered":"2023-08-27T11:19:16.257801+0000","last_clean":"2023-08-27T11:19:16.257801+0000","last_became_active":"2023-08-27T11:19:16.255840+0000","last_became_peered":"2023-08-27T11:19:16.255840+0000","last_unstale":"2023-08-27T11:19:16.257801+0000","last_undegraded":"2023-08-27T11:19:16.257801+0000","last_fullsized":"2023-08-27T11:19:16.257801+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:47:15.078584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.240918+0000","last_change":"2023-08-27T11:19:15.682143+0000","last_active":"2023-08-27T11:19:16.240918+0000","last_peered":"2023-08-27T11:19:16.240918+0000","last_clean":"2023-08-27T11:19:16.240918+0000","last_became_active":"2023-08-27T11:19:15.681880+0000","last_became_peered":"2023-08-27T11:19:15.681880+0000","last_unstale":"2023-08-27T11:19:16.240918+0000","last_undegraded":"2023-08-27T11:19:16.240918+0000","last_fullsized":"2023-08-27T11:19:16.240918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:05.741563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":94,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.247181+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:19:16.247181+0000","last_peered":"2023-08-27T11:19:16.247181+0000","last_clean":"2023-08-27T11:19:16.247181+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:19:16.247181+0000","last_undegraded":"2023-08-27T11:19:16.247181+0000","last_fullsized":"2023-08-27T11:19:16.247181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:59:51.103458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252756+0000","last_change":"2023-08-27T11:19:16.256282+0000","last_active":"2023-08-27T11:19:17.252756+0000","last_peered":"2023-08-27T11:19:17.252756+0000","last_clean":"2023-08-27T11:19:17.252756+0000","last_became_active":"2023-08-27T11:19:16.254162+0000","last_became_peered":"2023-08-27T11:19:16.254162+0000","last_unstale":"2023-08-27T11:19:17.252756+0000","last_undegraded":"2023-08-27T11:19:17.252756+0000","last_fullsized":"2023-08-27T11:19:17.252756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:00:38.245757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.240967+0000","last_change":"2023-08-27T11:19:15.679327+0000","last_active":"2023-08-27T11:19:16.240967+0000","last_peered":"2023-08-27T11:19:16.240967+0000","last_clean":"2023-08-27T11:19:16.240967+0000","last_became_active":"2023-08-27T11:19:15.678898+0000","last_became_peered":"2023-08-27T11:19:15.678898+0000","last_unstale":"2023-08-27T11:19:16.240967+0000","last_undegraded":"2023-08-27T11:19:16.240967+0000","last_fullsized":"2023-08-27T11:19:16.240967+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:53.441825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.246840+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:19:16.246840+0000","last_peered":"2023-08-27T11:19:16.246840+0000","last_clean":"2023-08-27T11:19:16.246840+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:19:16.246840+0000","last_undegraded":"2023-08-27T11:19:16.246840+0000","last_fullsized":"2023-08-27T11:19:16.246840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:01:16.542542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.248584+0000","last_change":"2023-08-27T11:19:16.255315+0000","last_active":"2023-08-27T11:19:17.248584+0000","last_peered":"2023-08-27T11:19:17.248584+0000","last_clean":"2023-08-27T11:19:17.248584+0000","last_became_active":"2023-08-27T11:19:16.255033+0000","last_became_peered":"2023-08-27T11:19:16.255033+0000","last_unstale":"2023-08-27T11:19:17.248584+0000","last_undegraded":"2023-08-27T11:19:17.248584+0000","last_fullsized":"2023-08-27T11:19:17.248584+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:52:18.509741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.241749+0000","last_change":"2023-08-27T11:19:15.679229+0000","last_active":"2023-08-27T11:19:16.241749+0000","last_peered":"2023-08-27T11:19:16.241749+0000","last_clean":"2023-08-27T11:19:16.241749+0000","last_became_active":"2023-08-27T11:19:15.678940+0000","last_became_peered":"2023-08-27T11:19:15.678940+0000","last_unstale":"2023-08-27T11:19:16.241749+0000","last_undegraded":"2023-08-27T11:19:16.241749+0000","last_fullsized":"2023-08-27T11:19:16.241749+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:19.489206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":244,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.247014+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:19:16.247014+0000","last_peered":"2023-08-27T11:19:16.247014+0000","last_clean":"2023-08-27T11:19:16.247014+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:19:16.247014+0000","last_undegraded":"2023-08-27T11:19:16.247014+0000","last_fullsized":"2023-08-27T11:19:16.247014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:58:13.188543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.249635+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:19:16.249635+0000","last_peered":"2023-08-27T11:19:16.249635+0000","last_clean":"2023-08-27T11:19:16.249635+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:19:16.249635+0000","last_undegraded":"2023-08-27T11:19:16.249635+0000","last_fullsized":"2023-08-27T11:19:16.249635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:58:58.287272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264042+0000","last_change":"2023-08-27T11:19:16.259098+0000","last_active":"2023-08-27T11:19:16.264042+0000","last_peered":"2023-08-27T11:19:16.264042+0000","last_clean":"2023-08-27T11:19:16.264042+0000","last_became_active":"2023-08-27T11:19:16.258824+0000","last_became_peered":"2023-08-27T11:19:16.258824+0000","last_unstale":"2023-08-27T11:19:16.264042+0000","last_undegraded":"2023-08-27T11:19:16.264042+0000","last_fullsized":"2023-08-27T11:19:16.264042+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:48.743606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"64'2","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252554+0000","last_change":"2023-08-27T11:19:15.680008+0000","last_active":"2023-08-27T11:19:17.252554+0000","last_peered":"2023-08-27T11:19:17.252554+0000","last_clean":"2023-08-27T11:19:17.252554+0000","last_became_active":"2023-08-27T11:19:15.679150+0000","last_became_peered":"2023-08-27T11:19:15.679150+0000","last_unstale":"2023-08-27T11:19:17.252554+0000","last_undegraded":"2023-08-27T11:19:17.252554+0000","last_fullsized":"2023-08-27T11:19:17.252554+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:58.293421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.241064+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:19:16.241064+0000","last_peered":"2023-08-27T11:19:16.241064+0000","last_clean":"2023-08-27T11:19:16.241064+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:19:16.241064+0000","last_undegraded":"2023-08-27T11:19:16.241064+0000","last_fullsized":"2023-08-27T11:19:16.241064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:46:45.809571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.264053+0000","last_change":"2023-08-27T11:19:16.260971+0000","last_active":"2023-08-27T11:19:16.264053+0000","last_peered":"2023-08-27T11:19:16.264053+0000","last_clean":"2023-08-27T11:19:16.264053+0000","last_became_active":"2023-08-27T11:19:16.260712+0000","last_became_peered":"2023-08-27T11:19:16.260712+0000","last_unstale":"2023-08-27T11:19:16.264053+0000","last_undegraded":"2023-08-27T11:19:16.264053+0000","last_fullsized":"2023-08-27T11:19:16.264053+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:15.657346+0000","last_clean_scrub_stamp":"2023-08-27T11:19:15.657346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:32.331565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-08-27T11:19:16.677770+0000","last_change":"2023-08-27T11:19:15.679538+0000","last_active":"2023-08-27T11:19:16.677770+0000","last_peered":"2023-08-27T11:19:16.677770+0000","last_clean":"2023-08-27T11:19:16.677770+0000","last_became_active":"2023-08-27T11:19:15.679102+0000","last_became_peered":"2023-08-27T11:19:15.679102+0000","last_unstale":"2023-08-27T11:19:16.677770+0000","last_undegraded":"2023-08-27T11:19:16.677770+0000","last_fullsized":"2023-08-27T11:19:16.677770+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:19:14.652571+0000","last_clean_scrub_stamp":"2023-08-27T11:19:14.652571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:49:30.845159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-08-27T11:19:17.252269+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:19:17.252269+0000","last_peered":"2023-08-27T11:19:17.252269+0000","last_clean":"2023-08-27T11:19:17.252269+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:19:17.252269+0000","last_undegraded":"2023-08-27T11:19:17.252269+0000","last_fullsized":"2023-08-27T11:19:17.252269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:57:17.819601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28368,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343472,"statfs":{"total":96636764160,"available":96607715328,"internally_reserved":0,"allocated":1372160,"data_stored":943127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47499999999999998,"15min":0.46700000000000003},"min":{"1min":0.32000000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.36199999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.77700000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54000000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47699999999999998,"15min":0.46899999999999997},"min":{"1min":0.32200000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.39600000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.82299999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58799999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.45000000000000001},"min":{"1min":0.374,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.42799999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76100000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.48699999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.34200000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81000000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60599999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68799999999999994,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.504,"15min":0.502},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.38300000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73499999999999999,"5min":0.747,"15min":0.747},"last":0.44500000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.50600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":352759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.33900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89400000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54300000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.42099999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.48599999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.40999999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.503},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60699999999999998,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.51600000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53100000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77200000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75600000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.622}]},{"osd":5,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.442},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59399999999999997,"15min":0.59299999999999997},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.33900000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57199999999999995,"15min":0.58299999999999996},"min":{"1min":0.35299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69399999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78700000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28364,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343476,"statfs":{"total":96636764160,"available":96607719424,"internally_reserved":0,"allocated":1380352,"data_stored":943239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.79700000000000004,"15min":0.877},"min":{"1min":0.223,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56100000000000005,"5min":11.971,"15min":11.971},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.318,"5min":1.288,"15min":1.45},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48099999999999998,"5min":23.045999999999999,"15min":23.045999999999999},"last":0.34899999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.82999999999999996,"15min":0.91100000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.42399999999999999,"5min":11.928000000000001,"15min":11.928000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.45300000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.26300000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.82599999999999996,"15min":0.90200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.52800000000000002,"5min":10.99,"15min":10.99},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.371,"5min":1.323,"15min":1.482},"min":{"1min":0.28699999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54200000000000004,"5min":22.893999999999998,"15min":22.893999999999998},"last":0.29699999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.44700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.73999999999999999,"5min":1.26,"15min":1.26},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.98499999999999999,"15min":1.0580000000000001},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73199999999999998,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.51300000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":1.514,"15min":1.671},"min":{"1min":0.41099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85399999999999998,"5min":23.016999999999999,"15min":23.016999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77700000000000002,"5min":1.296,"15min":1.296},"last":0.503}]},{"osd":6,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":1.034,"15min":1.1100000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79200000000000004,"5min":10.925000000000001,"15min":10.925000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":1.532,"15min":1.6879999999999999},"min":{"1min":0.45500000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.875,"5min":22.952000000000002,"15min":22.952000000000002},"last":0.58199999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":1.524,"15min":1.6830000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82599999999999996,"5min":22.988,"15min":22.988},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79600000000000004,"5min":1.327,"15min":1.327},"last":0.56599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":774144,"data_stored":351398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.317,"15min":0.318},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.436,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.35999999999999999,"15min":0.36599999999999999},"min":{"1min":0.23899999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.42099999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.34000000000000002,"15min":0.34300000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.48899999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34000000000000002,"15min":0.34300000000000003},"min":{"1min":0.247,"5min":0.224,"15min":0.224},"max":{"1min":0.41899999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.39900000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.35099999999999998,"15min":0.35399999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.47399999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.375,"15min":0.376},"min":{"1min":0.32300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.435,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.45700000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.755,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59999999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.57899999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.625},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55000000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.64000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":352759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.22500000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.316,"15min":0.315},"min":{"1min":0.23200000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.311}]},{"osd":1,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32800000000000001,"15min":0.32900000000000001},"min":{"1min":0.253,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.24399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.34899999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.33700000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.47099999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.46999999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.72299999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.42799999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.48099999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.79100000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.372}]},{"osd":6,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.501,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44500000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.629,"15min":0.629},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81399999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.39900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":352759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.29799999999999999,"15min":0.29399999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.436,"5min":0.438,"15min":0.438},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.30499999999999999,"15min":0.29999999999999999},"min":{"1min":0.23899999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.31900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.30199999999999999,"15min":0.29999999999999999},"min":{"1min":0.23899999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.35699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.29099999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.46999999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.378,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.40100000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57799999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.377,"5min":0.314,"15min":0.314},"max":{"1min":0.76200000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.42199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73199999999999998,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.73899999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.59799999999999998},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.82899999999999996,"5min":0.874,"15min":0.874},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81699999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52500000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":1.236,"15min":1.236},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":798720,"data_stored":354475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.499,"15min":0.504},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70199999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48199999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.60899999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.57499999999999996}]},{"osd":2,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66300000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.62}]},{"osd":3,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.52100000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.71199999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51800000000000002,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71699999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.38800000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.77600000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.68999999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.69499999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57599999999999996,"15min":0.57999999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.79000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":352759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65800000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79700000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.61899999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.63700000000000001,"5min":1.075,"15min":1.075},"last":0.56100000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55700000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73099999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57499999999999996}]},{"osd":3,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":1.03,"15min":1.03},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70599999999999996,"5min":1.026,"15min":1.026},"last":0.46800000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67000000000000004,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.77100000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54000000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.74399999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71399999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.49299999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81999999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.62},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78800000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:19:21.160 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:19:21.161 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:19:21.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.316+0000 7fb2d0106700 1 Processor -- start 2023-08-27T11:19:21.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.316+0000 7fb2d0106700 1 -- start start 2023-08-27T11:19:21.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2d0106700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2c81445b0 0x7fb2c80ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:21.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2d0106700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb2c80ccad0 0x7fb2c815b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:21.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2d0106700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c80b08e0 con 0x7fb2c806f790 2023-08-27T11:19:21.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2d0106700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c815bc70 con 0x7fb2c80ccad0 2023-08-27T11:19:21.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2d0106700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c80ccf50 con 0x7fb2c81445b0 2023-08-27T11:19:21.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2ce6a3700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb2c806f790 0x7fb2c815ef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53432/0 (socket says 172.21.15.112:53432) 2023-08-27T11:19:21.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2ce6a3700 1 -- 172.21.15.112:0/4157062818 learned_addr learned my addr 172.21.15.112:0/4157062818 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:21.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2cdea2700 1 --2- 172.21.15.112:0/4157062818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2c81445b0 0x7fb2c80ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:21.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2cd6a1700 1 --2- 172.21.15.112:0/4157062818 >> v2:172.21.15.138:3300/0 conn(0x7fb2c80ccad0 0x7fb2c815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:21.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2ccea0700 1 -- 172.21.15.112:0/4157062818 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652530898 0 0) 0x7fb2c80b08e0 con 0x7fb2c806f790 2023-08-27T11:19:21.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2ccea0700 1 -- 172.21.15.112:0/4157062818 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2b40035b0 con 0x7fb2c806f790 2023-08-27T11:19:21.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2ccea0700 1 -- 172.21.15.112:0/4157062818 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2500233672 0 0) 0x7fb2b40035b0 con 0x7fb2c806f790 2023-08-27T11:19:21.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2cd6a1700 1 -- 172.21.15.112:0/4157062818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2c81445b0 msgr2=0x7fb2c80ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2cd6a1700 1 --2- 172.21.15.112:0/4157062818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2c81445b0 0x7fb2c80ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2cd6a1700 1 -- 172.21.15.112:0/4157062818 >> v1:172.21.15.112:6789/0 conn(0x7fb2c806f790 legacy=0x7fb2c815ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2cd6a1700 1 -- 172.21.15.112:0/4157062818 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c80b0b30 con 0x7fb2c80ccad0 2023-08-27T11:19:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.317+0000 7fb2cdea2700 1 --2- 172.21.15.112:0/4157062818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2c81445b0 0x7fb2c80ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:19:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.318+0000 7fb2cd6a1700 1 --2- 172.21.15.112:0/4157062818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2c80ccad0 0x7fb2c815b730 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009840 tx=0x7fb2b8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7294a00aacdd604f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:21.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.318+0000 7fb2ccea0700 1 -- 172.21.15.112:0/4157062818 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00019f0 con 0x7fb2c80ccad0 2023-08-27T11:19:21.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.318+0000 7fb2ccea0700 1 -- 172.21.15.112:0/4157062818 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b8004030 con 0x7fb2c80ccad0 2023-08-27T11:19:21.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.318+0000 7fb2ccea0700 1 -- 172.21.15.112:0/4157062818 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80043a0 con 0x7fb2c80ccad0 2023-08-27T11:19:21.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.318+0000 7fb2d0106700 1 -- 172.21.15.112:0/4157062818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2c80ccad0 msgr2=0x7fb2c815b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:21.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.318+0000 7fb2d0106700 1 --2- 172.21.15.112:0/4157062818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2c80ccad0 0x7fb2c815b730 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009840 tx=0x7fb2b8009b50 comp rx=0 tx=0).stop 2023-08-27T11:19:21.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.318+0000 7fb2d0106700 1 -- 172.21.15.112:0/4157062818 shutdown_connections 2023-08-27T11:19:21.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.318+0000 7fb2d0106700 1 --2- 172.21.15.112:0/4157062818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2c81445b0 0x7fb2c80ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.318+0000 7fb2d0106700 1 --2- 172.21.15.112:0/4157062818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2c80ccad0 0x7fb2c815b730 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.318+0000 7fb2d0106700 1 -- 172.21.15.112:0/4157062818 >> 172.21.15.112:0/4157062818 conn(0x7fb2c80b1e90 msgr2=0x7fb2c80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:21.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.318+0000 7fb2d0106700 1 -- 172.21.15.112:0/4157062818 shutdown_connections 2023-08-27T11:19:21.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.318+0000 7fb2d0106700 1 -- 172.21.15.112:0/4157062818 wait complete. 2023-08-27T11:19:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.320+0000 7fb2d0106700 1 Processor -- start 2023-08-27T11:19:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.320+0000 7fb2d0106700 1 -- start start 2023-08-27T11:19:21.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.320+0000 7fb2d0106700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2c806f790 0x7fb2c813c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:21.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.320+0000 7fb2d0106700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2c80ccad0 0x7fb2c813c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:21.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.320+0000 7fb2d0106700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2c81445b0 0x7fb2c813cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:21.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.320+0000 7fb2d0106700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c813ec00 con 0x7fb2c80ccad0 2023-08-27T11:19:21.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.320+0000 7fb2d0106700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c813b520 con 0x7fb2c806f790 2023-08-27T11:19:21.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.320+0000 7fb2d0106700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c813b800 con 0x7fb2c81445b0 2023-08-27T11:19:21.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.320+0000 7fb2cd6a1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2c80ccad0 0x7fb2c813c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:21.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.320+0000 7fb2cd6a1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2c80ccad0 0x7fb2c813c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59054/0 (socket says 172.21.15.112:59054) 2023-08-27T11:19:21.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.320+0000 7fb2cd6a1700 1 -- 172.21.15.112:0/1386851267 learned_addr learned my addr 172.21.15.112:0/1386851267 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:21.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2ce6a3700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2c81445b0 0x7fb2c813cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:21.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2cdea2700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2c806f790 0x7fb2c813c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:21.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2cd6a1700 1 -- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2c81445b0 msgr2=0x7fb2c813cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:21.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2cd6a1700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2c81445b0 0x7fb2c813cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2cd6a1700 1 -- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2c806f790 msgr2=0x7fb2c813c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:21.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2cd6a1700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2c806f790 0x7fb2c813c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2cd6a1700 1 -- 172.21.15.112:0/1386851267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c40094a0 con 0x7fb2c80ccad0 2023-08-27T11:19:21.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2ce6a3700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2c81445b0 0x7fb2c813cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:21.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2cdea2700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2c806f790 0x7fb2c813c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:21.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2cd6a1700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2c80ccad0 0x7fb2c813c980 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009b90 tx=0x7fb2b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:21.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2be7fc700 1 -- 172.21.15.112:0/1386851267 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8003df0 con 0x7fb2c80ccad0 2023-08-27T11:19:21.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2be7fc700 1 -- 172.21.15.112:0/1386851267 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b801f070 con 0x7fb2c80ccad0 2023-08-27T11:19:21.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2d0106700 1 -- 172.21.15.112:0/1386851267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b80094a0 con 0x7fb2c80ccad0 2023-08-27T11:19:21.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2be7fc700 1 -- 172.21.15.112:0/1386851267 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8011870 con 0x7fb2c80ccad0 2023-08-27T11:19:21.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.321+0000 7fb2d0106700 1 -- 172.21.15.112:0/1386851267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c813f130 con 0x7fb2c80ccad0 2023-08-27T11:19:21.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.322+0000 7fb2d0106700 1 -- 172.21.15.112:0/1386851267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c8067e90 con 0x7fb2c80ccad0 2023-08-27T11:19:21.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.323+0000 7fb2be7fc700 1 -- 172.21.15.112:0/1386851267 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fb2b8021030 con 0x7fb2c80ccad0 2023-08-27T11:19:21.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.323+0000 7fb2be7fc700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb2ac06a170 0x7fb2ac06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:21.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.323+0000 7fb2be7fc700 1 -- 172.21.15.112:0/1386851267 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2b80898d0 con 0x7fb2c80ccad0 2023-08-27T11:19:21.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.323+0000 7fb2cdea2700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb2ac06a170 0x7fb2ac06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:21.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.324+0000 7fb2cdea2700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb2ac06a170 0x7fb2ac06c630 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009c50 tx=0x7fb2c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:21.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.327+0000 7fb2be7fc700 1 -- 172.21.15.112:0/1386851267 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb2b8054fe0 con 0x7fb2c80ccad0 2023-08-27T11:19:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.595+0000 7fb2d0106700 1 -- 172.21.15.112:0/1386851267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb2c8067a20 con 0x7fb2c80ccad0 2023-08-27T11:19:21.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.600+0000 7fb2be7fc700 1 -- 172.21.15.112:0/1386851267 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb2b8058600 con 0x7fb2c80ccad0 2023-08-27T11:19:21.602 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-08-27T11:19:21.602 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA== 2023-08-27T11:19:21.602 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2023-08-27T11:19:21.603 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2023-08-27T11:19:21.603 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:19:21.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.603+0000 7fb2d0106700 1 -- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb2ac06a170 msgr2=0x7fb2ac06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:21.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.603+0000 7fb2d0106700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb2ac06a170 0x7fb2ac06c630 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009c50 tx=0x7fb2c4008040 comp rx=0 tx=0).stop 2023-08-27T11:19:21.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.603+0000 7fb2d0106700 1 -- 172.21.15.112:0/1386851267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2c80ccad0 msgr2=0x7fb2c813c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:21.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.603+0000 7fb2d0106700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2c80ccad0 0x7fb2c813c980 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009b90 tx=0x7fb2b8004750 comp rx=0 tx=0).stop 2023-08-27T11:19:21.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.603+0000 7fb2d0106700 1 -- 172.21.15.112:0/1386851267 shutdown_connections 2023-08-27T11:19:21.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.603+0000 7fb2d0106700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb2ac06a170 0x7fb2ac06c630 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.603+0000 7fb2d0106700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2c81445b0 0x7fb2c813cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.603+0000 7fb2d0106700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2c806f790 0x7fb2c813c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.603+0000 7fb2d0106700 1 --2- 172.21.15.112:0/1386851267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2c80ccad0 0x7fb2c813c980 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.603+0000 7fb2d0106700 1 -- 172.21.15.112:0/1386851267 >> 172.21.15.112:0/1386851267 conn(0x7fb2c80b1e90 msgr2=0x7fb2c806fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:21.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.603+0000 7fb2d0106700 1 -- 172.21.15.112:0/1386851267 shutdown_connections 2023-08-27T11:19:21.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.603+0000 7fb2d0106700 1 -- 172.21.15.112:0/1386851267 wait complete. 2023-08-27T11:19:21.608 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2023-08-27T11:19:21.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:19:21.656 INFO:tasks.ceph.mds.a.smithi112.stderr:2023-08-27T11:19:21.655+0000 7f42ae503700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:19:21.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.735+0000 7f2044933700 1 Processor -- start 2023-08-27T11:19:21.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.735+0000 7f2044933700 1 -- start start 2023-08-27T11:19:21.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.736+0000 7f2044933700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f20400b17e0 0x7f20400cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:21.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.736+0000 7f2044933700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2040161ce0 0x7f204006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:21.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.736+0000 7f2044933700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2040161850 con 0x7f20400b1400 2023-08-27T11:19:21.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.736+0000 7f2044933700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20400cfc10 con 0x7f20400b17e0 2023-08-27T11:19:21.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.736+0000 7f2044933700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2040161200 con 0x7f2040161ce0 2023-08-27T11:19:21.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.736+0000 7f203dd9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f20400b17e0 0x7f20400cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:21.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.736+0000 7f203dd9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f20400b17e0 0x7f20400cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59062/0 (socket says 172.21.15.112:59062) 2023-08-27T11:19:21.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.736+0000 7f203dd9b700 1 -- 172.21.15.112:0/3737386862 learned_addr learned my addr 172.21.15.112:0/3737386862 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:21.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.736+0000 7f203ed9d700 1 --2- 172.21.15.112:0/3737386862 >> v2:172.21.15.138:3300/0 conn(0x7f2040161ce0 0x7f204006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:21.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.736+0000 7f203dd9b700 1 -- 172.21.15.112:0/3737386862 >> v2:172.21.15.138:3300/0 conn(0x7f2040161ce0 msgr2=0x7f204006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:21.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.736+0000 7f203dd9b700 1 --2- 172.21.15.112:0/3737386862 >> v2:172.21.15.138:3300/0 conn(0x7f2040161ce0 0x7f204006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.736+0000 7f203dd9b700 1 -- 172.21.15.112:0/3737386862 >> v1:172.21.15.138:6789/0 conn(0x7f20400b1400 legacy=0x7f204006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:21.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.737+0000 7f203dd9b700 1 -- 172.21.15.112:0/3737386862 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20401614e0 con 0x7f20400b17e0 2023-08-27T11:19:21.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.737+0000 7f203dd9b700 1 --2- 172.21.15.112:0/3737386862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20400b17e0 0x7f20400cf5a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f204006df10 tx=0x7f2030009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98dd4beb6e10e54 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:21.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.737+0000 7f203d59a700 1 -- 172.21.15.112:0/3737386862 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030004230 con 0x7f20400b17e0 2023-08-27T11:19:21.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.737+0000 7f203d59a700 1 -- 172.21.15.112:0/3737386862 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2030004390 con 0x7f20400b17e0 2023-08-27T11:19:21.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.737+0000 7f203d59a700 1 -- 172.21.15.112:0/3737386862 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20300036a0 con 0x7f20400b17e0 2023-08-27T11:19:21.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.737+0000 7f2044933700 1 -- 172.21.15.112:0/3737386862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20400b17e0 msgr2=0x7f20400cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:21.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.737+0000 7f2044933700 1 --2- 172.21.15.112:0/3737386862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20400b17e0 0x7f20400cf5a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f204006df10 tx=0x7f2030009bd0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.737+0000 7f2044933700 1 -- 172.21.15.112:0/3737386862 shutdown_connections 2023-08-27T11:19:21.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.737+0000 7f2044933700 1 --2- 172.21.15.112:0/3737386862 >> v2:172.21.15.138:3300/0 conn(0x7f2040161ce0 0x7f204006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.737+0000 7f2044933700 1 --2- 172.21.15.112:0/3737386862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20400b17e0 0x7f20400cf5a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.737+0000 7f2044933700 1 -- 172.21.15.112:0/3737386862 >> 172.21.15.112:0/3737386862 conn(0x7f20400d0000 msgr2=0x7f20400d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:21.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.737+0000 7f2044933700 1 -- 172.21.15.112:0/3737386862 shutdown_connections 2023-08-27T11:19:21.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.738+0000 7f2044933700 1 -- 172.21.15.112:0/3737386862 wait complete. 2023-08-27T11:19:21.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.739+0000 7f2044933700 1 Processor -- start 2023-08-27T11:19:21.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.739+0000 7f2044933700 1 -- start start 2023-08-27T11:19:21.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f2044933700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20400b1400 0x7f2040153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f2044933700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20400b17e0 0x7f20400c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f2044933700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2040161ce0 0x7f20401f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f2044933700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20400c8880 con 0x7f2040161ce0 2023-08-27T11:19:21.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f2044933700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20400c6f10 con 0x7f20400b17e0 2023-08-27T11:19:21.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f2044933700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20401f4180 con 0x7f20400b1400 2023-08-27T11:19:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f203ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2040161ce0 0x7f20401f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f203ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2040161ce0 0x7f20401f3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59070/0 (socket says 172.21.15.112:59070) 2023-08-27T11:19:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f203ed9d700 1 -- 172.21.15.112:0/2161598845 learned_addr learned my addr 172.21.15.112:0/2161598845 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f203dd9b700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20400b17e0 0x7f20400c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f203e59c700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20400b1400 0x7f2040153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f203ed9d700 1 -- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20400b1400 msgr2=0x7f2040153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f203ed9d700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20400b1400 0x7f2040153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f203ed9d700 1 -- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20400b17e0 msgr2=0x7f20400c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:21.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f203ed9d700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20400b17e0 0x7f20400c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f203ed9d700 1 -- 172.21.15.112:0/2161598845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2030009550 con 0x7f2040161ce0 2023-08-27T11:19:21.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f203dd9b700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20400b17e0 0x7f20400c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:21.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f203ed9d700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2040161ce0 0x7f20401f3170 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f203800cd70 tx=0x7f203800a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:21.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f202effd700 1 -- 172.21.15.112:0/2161598845 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038003a20 con 0x7f2040161ce0 2023-08-27T11:19:21.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f202effd700 1 -- 172.21.15.112:0/2161598845 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2038003b80 con 0x7f2040161ce0 2023-08-27T11:19:21.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f202effd700 1 -- 172.21.15.112:0/2161598845 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038004020 con 0x7f2040161ce0 2023-08-27T11:19:21.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f2044933700 1 -- 172.21.15.112:0/2161598845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20401f36b0 con 0x7f2040161ce0 2023-08-27T11:19:21.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.740+0000 7f2044933700 1 -- 172.21.15.112:0/2161598845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20401f3bc0 con 0x7f2040161ce0 2023-08-27T11:19:21.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.741+0000 7f2044933700 1 -- 172.21.15.112:0/2161598845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20400c89c0 con 0x7f2040161ce0 2023-08-27T11:19:21.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.742+0000 7f202effd700 1 -- 172.21.15.112:0/2161598845 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f2038004180 con 0x7f2040161ce0 2023-08-27T11:19:21.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.742+0000 7f202effd700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f202006a170 0x7f202006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:21.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.742+0000 7f202effd700 1 -- 172.21.15.112:0/2161598845 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2038012070 con 0x7f2040161ce0 2023-08-27T11:19:21.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.742+0000 7f203e59c700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f202006a170 0x7f202006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.743+0000 7f203e59c700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f202006a170 0x7f202006c630 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f20401f0d60 tx=0x7f2034008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.746+0000 7f202effd700 1 -- 172.21.15.112:0/2161598845 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f20400c89c0 con 0x7f2040161ce0 2023-08-27T11:19:21.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.975+0000 7f2044933700 1 -- 172.21.15.112:0/2161598845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20400c89c0 con 0x7f2040161ce0 2023-08-27T11:19:21.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.976+0000 7f202effd700 1 -- 172.21.15.112:0/2161598845 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4886 (secure 0 0 0) 0x7f20400c89c0 con 0x7f2040161ce0 2023-08-27T11:19:21.978 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:19:21.978 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2906075444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2906075444},{"type":"v1","addr":"172.21.15.138:6836","nonce":2906075444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5761,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1781004436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1781004436},{"type":"v1","addr":"172.21.15.138:6837","nonce":1781004436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1573349448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1573349448},{"type":"v1","addr":"172.21.15.112:6834","nonce":1573349448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:19:16.064335+0000","modified":"2023-08-27T11:19:19.690233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/174707415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":174707415},{"type":"v1","addr":"172.21.15.112:6835","nonce":174707415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-27T11:19:21.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.978+0000 7f2044933700 1 -- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f202006a170 msgr2=0x7f202006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:21.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.978+0000 7f2044933700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f202006a170 0x7f202006c630 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f20401f0d60 tx=0x7f2034008040 comp rx=0 tx=0).stop 2023-08-27T11:19:21.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.978+0000 7f2044933700 1 -- 172.21.15.112:0/2161598845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2040161ce0 msgr2=0x7f20401f3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:21.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.978+0000 7f2044933700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2040161ce0 0x7f20401f3170 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f203800cd70 tx=0x7f203800a570 comp rx=0 tx=0).stop 2023-08-27T11:19:21.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.978+0000 7f2044933700 1 -- 172.21.15.112:0/2161598845 shutdown_connections 2023-08-27T11:19:21.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.978+0000 7f2044933700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f202006a170 0x7f202006c630 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.978+0000 7f2044933700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2040161ce0 0x7f20401f3170 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.978+0000 7f2044933700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20400b1400 0x7f2040153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.978+0000 7f2044933700 1 --2- 172.21.15.112:0/2161598845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20400b17e0 0x7f20400c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:21.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.978+0000 7f2044933700 1 -- 172.21.15.112:0/2161598845 >> 172.21.15.112:0/2161598845 conn(0x7f20400d0000 msgr2=0x7f20400d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:21.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.978+0000 7f2044933700 1 -- 172.21.15.112:0/2161598845 shutdown_connections 2023-08-27T11:19:21.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:21.978+0000 7f2044933700 1 -- 172.21.15.112:0/2161598845 wait complete. 2023-08-27T11:19:21.982 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 39 2023-08-27T11:19:22.394 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:19:16.064335+0000', 'modified': '2023-08-27T11:19:19.690233+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6093}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6093': {'gid': 6093, 'name': 'a', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/174707415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 174707415}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 174707415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:19:22.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:19:22.395 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-27T11:19:22.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.516+0000 7fa79db11700 1 Processor -- start 2023-08-27T11:19:22.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.516+0000 7fa79db11700 1 -- start start 2023-08-27T11:19:22.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.517+0000 7fa79db11700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b17e0 0x7fa7980cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.517+0000 7fa79db11700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa798161ce0 0x7fa79806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.517+0000 7fa79db11700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa798161850 con 0x7fa7980b1400 2023-08-27T11:19:22.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.517+0000 7fa79db11700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7980cfc10 con 0x7fa798161ce0 2023-08-27T11:19:22.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.517+0000 7fa79db11700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa798161200 con 0x7fa7980b17e0 2023-08-27T11:19:22.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.517+0000 7fa7977fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa7980b1400 0x7fa79806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53456/0 (socket says 172.21.15.112:53456) 2023-08-27T11:19:22.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.517+0000 7fa7977fe700 1 -- 172.21.15.112:0/3080475488 learned_addr learned my addr 172.21.15.112:0/3080475488 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:22.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.517+0000 7fa796ffd700 1 --2- 172.21.15.112:0/3080475488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b17e0 0x7fa7980cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.517+0000 7fa797fff700 1 --2- 172.21.15.112:0/3080475488 >> v2:172.21.15.138:3300/0 conn(0x7fa798161ce0 0x7fa79806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.517+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256974865 0 0) 0x7fa798161850 con 0x7fa7980b1400 2023-08-27T11:19:22.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.517+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7840035b0 con 0x7fa7980b1400 2023-08-27T11:19:22.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 792260459 0 0) 0x7fa7840035b0 con 0x7fa7980b1400 2023-08-27T11:19:22.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b17e0 msgr2=0x7fa7980cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 --2- 172.21.15.112:0/3080475488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b17e0 0x7fa7980cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 >> v2:172.21.15.138:3300/0 conn(0x7fa798161ce0 msgr2=0x7fa79806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 --2- 172.21.15.112:0/3080475488 >> v2:172.21.15.138:3300/0 conn(0x7fa798161ce0 0x7fa79806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7981614e0 con 0x7fa7980b1400 2023-08-27T11:19:22.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fa7880019f0 con 0x7fa7980b1400 2023-08-27T11:19:22.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 >> v1:172.21.15.112:6789/0 conn(0x7fa7980b1400 legacy=0x7fa79806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 --2- 172.21.15.112:0/3080475488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b17e0 0x7fa784003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 --2- 172.21.15.112:0/3080475488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa784003f80 0x7fa784004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 --2- 172.21.15.112:0/3080475488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa784004d90 0x7fa784005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980cfc10 con 0x7fa784003f80 2023-08-27T11:19:22.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa798161200 con 0x7fa784004d90 2023-08-27T11:19:22.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78400ba60 con 0x7fa7980b17e0 2023-08-27T11:19:22.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa796ffd700 1 --2- 172.21.15.112:0/3080475488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa784003f80 0x7fa784004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7977fe700 1 --2- 172.21.15.112:0/3080475488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b17e0 0x7fa784003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa797fff700 1 --2- 172.21.15.112:0/3080475488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa784004d90 0x7fa784005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa796ffd700 1 -- 172.21.15.112:0/3080475488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b17e0 msgr2=0x7fa784003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa796ffd700 1 --2- 172.21.15.112:0/3080475488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7980b17e0 0x7fa784003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa796ffd700 1 -- 172.21.15.112:0/3080475488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa784004d90 msgr2=0x7fa784005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa796ffd700 1 --2- 172.21.15.112:0/3080475488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa784004d90 0x7fa784005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa796ffd700 1 -- 172.21.15.112:0/3080475488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7840035b0 con 0x7fa784003f80 2023-08-27T11:19:22.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa797fff700 1 --2- 172.21.15.112:0/3080475488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa784004d90 0x7fa784005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:22.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa796ffd700 1 --2- 172.21.15.112:0/3080475488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa784003f80 0x7fa784004850 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa780009b90 tx=0x7fa784005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08a0af62b86d5fa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:22.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78000e6c0 con 0x7fa784003f80 2023-08-27T11:19:22.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78000e820 con 0x7fa784003f80 2023-08-27T11:19:22.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.518+0000 7fa7967fc700 1 -- 172.21.15.112:0/3080475488 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78000eb90 con 0x7fa784003f80 2023-08-27T11:19:22.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.519+0000 7fa79db11700 1 -- 172.21.15.112:0/3080475488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa784003f80 msgr2=0x7fa784004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.519+0000 7fa79db11700 1 --2- 172.21.15.112:0/3080475488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa784003f80 0x7fa784004850 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa780009b90 tx=0x7fa784005880 comp rx=0 tx=0).stop 2023-08-27T11:19:22.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.519+0000 7fa797fff700 1 -- 172.21.15.112:0/3080475488 reap_dead start 2023-08-27T11:19:22.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.519+0000 7fa79db11700 1 -- 172.21.15.112:0/3080475488 shutdown_connections 2023-08-27T11:19:22.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.519+0000 7fa79db11700 1 -- 172.21.15.112:0/3080475488 >> 172.21.15.112:0/3080475488 conn(0x7fa7980d0000 msgr2=0x7fa7980d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:22.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.519+0000 7fa79db11700 1 -- 172.21.15.112:0/3080475488 shutdown_connections 2023-08-27T11:19:22.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.519+0000 7fa79db11700 1 -- 172.21.15.112:0/3080475488 wait complete. 2023-08-27T11:19:22.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.520+0000 7fa79db11700 1 Processor -- start 2023-08-27T11:19:22.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.520+0000 7fa79db11700 1 -- start start 2023-08-27T11:19:22.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.521+0000 7fa79db11700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa784003f80 0x7fa7980c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.521+0000 7fa79db11700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7980acb10 0x7fa798154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.521+0000 7fa79db11700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7980ae9c0 0x7fa7980aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.521+0000 7fa79db11700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7981f2d60 con 0x7fa7980ae9c0 2023-08-27T11:19:22.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.521+0000 7fa79db11700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980c6f30 con 0x7fa7980acb10 2023-08-27T11:19:22.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.521+0000 7fa79db11700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980ae270 con 0x7fa784003f80 2023-08-27T11:19:22.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.521+0000 7fa797fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7980ae9c0 0x7fa7980aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.521+0000 7fa797fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7980ae9c0 0x7fa7980aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59090/0 (socket says 172.21.15.112:59090) 2023-08-27T11:19:22.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.521+0000 7fa797fff700 1 -- 172.21.15.112:0/622904216 learned_addr learned my addr 172.21.15.112:0/622904216 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:22.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.521+0000 7fa796ffd700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7980acb10 0x7fa798154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.521+0000 7fa7977fe700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa784003f80 0x7fa7980c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.522+0000 7fa797fff700 1 -- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa784003f80 msgr2=0x7fa7980c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.522+0000 7fa797fff700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa784003f80 0x7fa7980c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.522+0000 7fa797fff700 1 -- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7980acb10 msgr2=0x7fa798154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.522+0000 7fa797fff700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7980acb10 0x7fa798154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.522+0000 7fa797fff700 1 -- 172.21.15.112:0/622904216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7981560a0 con 0x7fa7980ae9c0 2023-08-27T11:19:22.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.522+0000 7fa797fff700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7980ae9c0 0x7fa7980aee50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa79806df40 tx=0x7fa790010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:22.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.522+0000 7fa78ffff700 1 -- 172.21.15.112:0/622904216 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790004030 con 0x7fa7980ae9c0 2023-08-27T11:19:22.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.522+0000 7fa78ffff700 1 -- 172.21.15.112:0/622904216 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa790004190 con 0x7fa7980ae9c0 2023-08-27T11:19:22.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.522+0000 7fa79db11700 1 -- 172.21.15.112:0/622904216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa798155800 con 0x7fa7980ae9c0 2023-08-27T11:19:22.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.522+0000 7fa78ffff700 1 -- 172.21.15.112:0/622904216 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790004540 con 0x7fa7980ae9c0 2023-08-27T11:19:22.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.522+0000 7fa79db11700 1 -- 172.21.15.112:0/622904216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa798155dc0 con 0x7fa7980ae9c0 2023-08-27T11:19:22.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.524+0000 7fa78ffff700 1 -- 172.21.15.112:0/622904216 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fa790016750 con 0x7fa7980ae9c0 2023-08-27T11:19:22.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.524+0000 7fa78ffff700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa77806a170 0x7fa77806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.524+0000 7fa78ffff700 1 -- 172.21.15.112:0/622904216 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa790087d20 con 0x7fa7980ae9c0 2023-08-27T11:19:22.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.524+0000 7fa7977fe700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa77806a170 0x7fa77806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.525+0000 7fa7977fe700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa77806a170 0x7fa77806c630 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa788000f80 tx=0x7fa788006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:22.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.526+0000 7fa7a9cd3b80 1 Processor -- start 2023-08-27T11:19:22.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.526+0000 7fa7a9cd3b80 1 -- start start 2023-08-27T11:19:22.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7a9cd3b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 0x55f6907d4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7a9cd3b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907e5960 0x55f6907d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7a9cd3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 0x55f6907d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7a9cd3b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f690702730 con 0x55f6907d3f40 2023-08-27T11:19:22.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7a9cd3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f690700050 con 0x55f6907a73d0 2023-08-27T11:19:22.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7a9cd3b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6906fb940 con 0x55f6907e5960 2023-08-27T11:19:22.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7977fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 0x55f6907d4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7977fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 0x55f6907d4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59094/0 (socket says 172.21.15.112:59094) 2023-08-27T11:19:22.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7977fe700 1 -- 172.21.15.112:0/1981129096 learned_addr learned my addr 172.21.15.112:0/1981129096 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:22.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa796ffd700 1 --2- 172.21.15.112:0/1981129096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907e5960 0x55f6907d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa797fff700 1 --2- 172.21.15.112:0/1981129096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 0x55f6907d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7977fe700 1 -- 172.21.15.112:0/1981129096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907e5960 msgr2=0x55f6907d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7977fe700 1 --2- 172.21.15.112:0/1981129096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907e5960 0x55f6907d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7977fe700 1 -- 172.21.15.112:0/1981129096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 msgr2=0x55f6907d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7977fe700 1 --2- 172.21.15.112:0/1981129096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 0x55f6907d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7977fe700 1 -- 172.21.15.112:0/1981129096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f69057b5e0 con 0x55f6907d3f40 2023-08-27T11:19:22.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa7977fe700 1 --2- 172.21.15.112:0/1981129096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 0x55f6907d4320 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa788015700 tx=0x7fa788015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8e638edc1e59ba server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:22.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa78d7fa700 1 -- 172.21.15.112:0/1981129096 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78801a040 con 0x55f6907d3f40 2023-08-27T11:19:22.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa78d7fa700 1 -- 172.21.15.112:0/1981129096 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7880145b0 con 0x55f6907d3f40 2023-08-27T11:19:22.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.527+0000 7fa78d7fa700 1 -- 172.21.15.112:0/1981129096 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788015ea0 con 0x55f6907d3f40 2023-08-27T11:19:22.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.528+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/1981129096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 msgr2=0x55f6907d4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.528+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/1981129096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 0x55f6907d4320 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa788015700 tx=0x7fa788015000 comp rx=0 tx=0).stop 2023-08-27T11:19:22.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.528+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/1981129096 shutdown_connections 2023-08-27T11:19:22.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.528+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/1981129096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 0x55f6907d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.528+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/1981129096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 0x55f6907d4320 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.528+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/1981129096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907e5960 0x55f6907d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.528+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/1981129096 >> 172.21.15.112:0/1981129096 conn(0x55f690707c30 msgr2=0x55f6907eed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:22.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.528+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/1981129096 shutdown_connections 2023-08-27T11:19:22.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.528+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/1981129096 wait complete. 2023-08-27T11:19:22.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.529+0000 7fa7a9cd3b80 1 -- start start 2023-08-27T11:19:22.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.529+0000 7fa7a9cd3b80 1 Processor -- start 2023-08-27T11:19:22.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.529+0000 7fa7a9cd3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 0x55f6907f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.529+0000 7fa7a9cd3b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 0x55f6907fe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.529+0000 7fa7a9cd3b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907e5960 0x55f6907d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa7a9cd3b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f690699b60 con 0x55f6907d3f40 2023-08-27T11:19:22.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa7a9cd3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f690699860 con 0x55f6907a73d0 2023-08-27T11:19:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa7a9cd3b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f69057d8d0 con 0x55f6907e5960 2023-08-27T11:19:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa7a9cd3b80 10 client.0 _ll_register_callbacks cb 0x55f690697630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:19:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa796ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 0x55f6907fe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa796ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 0x55f6907fe070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59102/0 (socket says 172.21.15.112:59102) 2023-08-27T11:19:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa796ffd700 1 -- 172.21.15.112:0/2373218151 learned_addr learned my addr 172.21.15.112:0/2373218151 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa7977fe700 1 --2- 172.21.15.112:0/2373218151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 0x55f6907f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa797fff700 1 --2- 172.21.15.112:0/2373218151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907e5960 0x55f6907d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa796ffd700 1 -- 172.21.15.112:0/2373218151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907e5960 msgr2=0x55f6907d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa796ffd700 1 --2- 172.21.15.112:0/2373218151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907e5960 0x55f6907d2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa796ffd700 1 -- 172.21.15.112:0/2373218151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 msgr2=0x55f6907f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa796ffd700 1 --2- 172.21.15.112:0/2373218151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 0x55f6907f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa796ffd700 1 -- 172.21.15.112:0/2373218151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7981614e0 con 0x55f6907d3f40 2023-08-27T11:19:22.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa796ffd700 1 --2- 172.21.15.112:0/2373218151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 0x55f6907fe070 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa784004790 tx=0x7fa780004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9adba86b2ae1dbd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:22.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa7767fc700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:19:22.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa7767fc700 1 -- 172.21.15.112:0/2373218151 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780022040 con 0x55f6907d3f40 2023-08-27T11:19:22.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa7767fc700 1 -- 172.21.15.112:0/2373218151 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa780004450 con 0x55f6907d3f40 2023-08-27T11:19:22.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/2373218151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f69079c4f0 con 0x55f6907d3f40 2023-08-27T11:19:22.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa7767fc700 1 -- 172.21.15.112:0/2373218151 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa78000e6c0 con 0x55f6907d3f40 2023-08-27T11:19:22.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa7a9cd3b80 10 client.6243 fetch_fsmap learned FSMap version 39 2023-08-27T11:19:22.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/2373218151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55f690713750 con 0x55f6907d3f40 2023-08-27T11:19:22.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.530+0000 7fa7767fc700 1 -- 172.21.15.112:0/2373218151 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f69079c4f0 con 0x55f6907d3f40 2023-08-27T11:19:22.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.531+0000 7fa7767fc700 1 -- 172.21.15.112:0/2373218151 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa780011070 con 0x55f6907d3f40 2023-08-27T11:19:22.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.531+0000 7fa7a9cd3b80 10 client.6243 fetch_fsmap finished waiting for FSMap version 39 2023-08-27T11:19:22.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.531+0000 7fa7a9cd3b80 10 client.6243 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-27T11:19:22.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.531+0000 7fa7a9cd3b80 20 client.6243 populate_metadata read hostname 'smithi112' 2023-08-27T11:19:22.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.531+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/2373218151 >> [v2:172.21.15.112:6833/174707415,v1:172.21.15.112:6835/174707415] conn(0x55f6907e7040 0x55f6907e7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.531+0000 7fa7a9cd3b80 4 client.6243 mds_command: new command op to 6093 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:19:22.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.531+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/2373218151 --> [v2:172.21.15.112:6833/174707415,v1:172.21.15.112:6835/174707415] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6906f4920 con 0x55f6907e7040 2023-08-27T11:19:22.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.531+0000 7fa7977fe700 1 --2- 172.21.15.112:0/2373218151 >> [v2:172.21.15.112:6833/174707415,v1:172.21.15.112:6835/174707415] conn(0x55f6907e7040 0x55f6907e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.531+0000 7fa7767fc700 10 client.6243 ms_handle_connect on v2:172.21.15.112:6833/174707415 2023-08-27T11:19:22.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.531+0000 7fa7977fe700 1 --2- 172.21.15.112:0/2373218151 >> [v2:172.21.15.112:6833/174707415,v1:172.21.15.112:6835/174707415] conn(0x55f6907e7040 0x55f6907e7420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=491210afddc4346d server_cookie=f41f9b3bbd965fe4 in_seq=0 out_seq=0 2023-08-27T11:19:22.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7767fc700 1 -- 172.21.15.112:0/2373218151 <== mds.0 v2:172.21.15.112:6833/174707415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f6906f4920 con 0x55f6907e7040 2023-08-27T11:19:22.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7767fc700 10 client.6243 handle_command_reply: tid=0 2023-08-27T11:19:22.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7a9cd3b80 1 client.6243 shutdown 2023-08-27T11:19:22.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/2373218151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 msgr2=0x55f6907fe070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/2373218151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 0x55f6907fe070 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa784004790 tx=0x7fa780004750 comp rx=0 tx=0).stop 2023-08-27T11:19:22.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/2373218151 shutdown_connections 2023-08-27T11:19:22.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/2373218151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907e5960 0x55f6907d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/2373218151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 0x55f6907f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/2373218151 >> [v2:172.21.15.112:6833/174707415,v1:172.21.15.112:6835/174707415] conn(0x55f6907e7040 0x55f6907e7420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/2373218151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907d3f40 0x55f6907fe070 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7767fc700 0 client.6243 ms_handle_reset on v2:172.21.15.112:6833/174707415 2023-08-27T11:19:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/2373218151 >> 172.21.15.112:0/2373218151 conn(0x55f690707c30 msgr2=0x55f6907eeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/2373218151 shutdown_connections 2023-08-27T11:19:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/2373218151 wait complete. 2023-08-27T11:19:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.532+0000 7fa7a9cd3b80 20 client.6243 trim_cache size 0 max 16384 2023-08-27T11:19:22.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.549+0000 7fa7a9cd3b80 1 Processor -- start 2023-08-27T11:19:22.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.550+0000 7fa7a9cd3b80 1 -- start start 2023-08-27T11:19:22.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.550+0000 7fa7a9cd3b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907a73d0 0x55f6907a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.550+0000 7fa7a9cd3b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907d3f40 0x55f6907aad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.550+0000 7fa7a9cd3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907e5960 0x55f690807940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.550+0000 7fa7a9cd3b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6907ee1a0 con 0x55f6907a73d0 2023-08-27T11:19:22.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.550+0000 7fa7a9cd3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6907e56f0 con 0x55f6907e5960 2023-08-27T11:19:22.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.550+0000 7fa7a9cd3b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6907d81c0 con 0x55f6907d3f40 2023-08-27T11:19:22.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.550+0000 7fa7977fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907a73d0 0x55f6907a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.551+0000 7fa7977fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907a73d0 0x55f6907a67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59108/0 (socket says 172.21.15.112:59108) 2023-08-27T11:19:22.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.551+0000 7fa7977fe700 1 -- 172.21.15.112:0/100892030 learned_addr learned my addr 172.21.15.112:0/100892030 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:22.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.551+0000 7fa7977fe700 1 -- 172.21.15.112:0/100892030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907d3f40 msgr2=0x55f6907aad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.551+0000 7fa7977fe700 1 --2- 172.21.15.112:0/100892030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907d3f40 0x55f6907aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.551+0000 7fa7977fe700 1 -- 172.21.15.112:0/100892030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907e5960 msgr2=0x55f690807940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.551+0000 7fa7977fe700 1 --2- 172.21.15.112:0/100892030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907e5960 0x55f690807940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.551+0000 7fa7977fe700 1 -- 172.21.15.112:0/100892030 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f690527d90 con 0x55f6907a73d0 2023-08-27T11:19:22.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.551+0000 7fa7977fe700 1 --2- 172.21.15.112:0/100892030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907a73d0 0x55f6907a67c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa788006010 tx=0x7fa788017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4807c6fccd3bca8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:22.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.551+0000 7fa78dffb700 1 -- 172.21.15.112:0/100892030 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788033070 con 0x55f6907a73d0 2023-08-27T11:19:22.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.551+0000 7fa78dffb700 1 -- 172.21.15.112:0/100892030 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7880149c0 con 0x55f6907a73d0 2023-08-27T11:19:22.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.551+0000 7fa78dffb700 1 -- 172.21.15.112:0/100892030 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788015ea0 con 0x55f6907a73d0 2023-08-27T11:19:22.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.552+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/100892030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907a73d0 msgr2=0x55f6907a67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.552+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/100892030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907a73d0 0x55f6907a67c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa788006010 tx=0x7fa788017000 comp rx=0 tx=0).stop 2023-08-27T11:19:22.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.552+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/100892030 shutdown_connections 2023-08-27T11:19:22.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.552+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/100892030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907e5960 0x55f690807940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.552+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/100892030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907a73d0 0x55f6907a67c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.552+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/100892030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907d3f40 0x55f6907aad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.552+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/100892030 >> 172.21.15.112:0/100892030 conn(0x55f690707c30 msgr2=0x55f6907d3570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:22.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.552+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/100892030 shutdown_connections 2023-08-27T11:19:22.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.552+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/100892030 wait complete. 2023-08-27T11:19:22.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.552+0000 7fa7a9cd3b80 1 -- start start 2023-08-27T11:19:22.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.552+0000 7fa7a9cd3b80 1 Processor -- start 2023-08-27T11:19:22.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.553+0000 7fa7a9cd3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 0x55f6907fe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.553+0000 7fa7a9cd3b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907d3f40 0x55f6908080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.553+0000 7fa7a9cd3b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907e5960 0x55f69083a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.553+0000 7fa7a9cd3b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6907a8ee0 con 0x55f6907e5960 2023-08-27T11:19:22.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.553+0000 7fa7a9cd3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69083b050 con 0x55f6907a73d0 2023-08-27T11:19:22.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.553+0000 7fa7a9cd3b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6907a9d00 con 0x55f6907d3f40 2023-08-27T11:19:22.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.553+0000 7fa7a9cd3b80 10 client.0 _ll_register_callbacks cb 0x55f690697630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:19:22.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.553+0000 7fa797fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907e5960 0x55f69083a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.553+0000 7fa797fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907e5960 0x55f69083a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59118/0 (socket says 172.21.15.112:59118) 2023-08-27T11:19:22.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.553+0000 7fa797fff700 1 -- 172.21.15.112:0/551015215 learned_addr learned my addr 172.21.15.112:0/551015215 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:22.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.553+0000 7fa796ffd700 1 --2- 172.21.15.112:0/551015215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907d3f40 0x55f6908080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.553+0000 7fa7977fe700 1 --2- 172.21.15.112:0/551015215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 0x55f6907fe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa797fff700 1 -- 172.21.15.112:0/551015215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907d3f40 msgr2=0x55f6908080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa797fff700 1 --2- 172.21.15.112:0/551015215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907d3f40 0x55f6908080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa797fff700 1 -- 172.21.15.112:0/551015215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 msgr2=0x55f6907fe070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa797fff700 1 --2- 172.21.15.112:0/551015215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 0x55f6907fe070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa797fff700 1 -- 172.21.15.112:0/551015215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f69057b5e0 con 0x55f6907e5960 2023-08-27T11:19:22.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa797fff700 1 --2- 172.21.15.112:0/551015215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907e5960 0x55f69083a460 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa7900535e0 tx=0x7fa7900568b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2254722322d18da1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:22.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7767fc700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:19:22.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7767fc700 1 -- 172.21.15.112:0/551015215 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79005aea0 con 0x55f6907e5960 2023-08-27T11:19:22.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7767fc700 1 -- 172.21.15.112:0/551015215 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa79005edf0 con 0x55f6907e5960 2023-08-27T11:19:22.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/551015215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7880145b0 con 0x55f6907e5960 2023-08-27T11:19:22.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7767fc700 1 -- 172.21.15.112:0/551015215 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa79006e220 con 0x55f6907e5960 2023-08-27T11:19:22.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7a9cd3b80 10 client.6249 fetch_fsmap learned FSMap version 39 2023-08-27T11:19:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/551015215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55f690510c10 con 0x55f6907e5960 2023-08-27T11:19:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7767fc700 1 -- 172.21.15.112:0/551015215 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7880145b0 con 0x55f6907e5960 2023-08-27T11:19:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7767fc700 1 -- 172.21.15.112:0/551015215 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa790075280 con 0x55f6907e5960 2023-08-27T11:19:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7a9cd3b80 10 client.6249 fetch_fsmap finished waiting for FSMap version 39 2023-08-27T11:19:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7a9cd3b80 10 client.6249 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-27T11:19:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7a9cd3b80 20 client.6249 populate_metadata read hostname 'smithi112' 2023-08-27T11:19:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/551015215 >> [v2:172.21.15.112:6833/174707415,v1:172.21.15.112:6835/174707415] conn(0x55f6908070e0 0x55f6907a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7a9cd3b80 4 client.6249 mds_command: new command op to 6093 tid=0[{"prefix": "status"}] 2023-08-27T11:19:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/551015215 --> [v2:172.21.15.112:6833/174707415,v1:172.21.15.112:6835/174707415] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f6906b5000 con 0x55f6908070e0 2023-08-27T11:19:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.554+0000 7fa7977fe700 1 --2- 172.21.15.112:0/551015215 >> [v2:172.21.15.112:6833/174707415,v1:172.21.15.112:6835/174707415] conn(0x55f6908070e0 0x55f6907a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7767fc700 10 client.6249 ms_handle_connect on v2:172.21.15.112:6833/174707415 2023-08-27T11:19:22.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7977fe700 1 --2- 172.21.15.112:0/551015215 >> [v2:172.21.15.112:6833/174707415,v1:172.21.15.112:6835/174707415] conn(0x55f6908070e0 0x55f6907a2b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d77a9802057a74 server_cookie=6fa1c6520d410474 in_seq=0 out_seq=0 2023-08-27T11:19:22.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7767fc700 1 -- 172.21.15.112:0/551015215 <== mds.0 v2:172.21.15.112:6833/174707415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55f6906b5000 con 0x55f6908070e0 2023-08-27T11:19:22.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7767fc700 10 client.6249 handle_command_reply: tid=0 2023-08-27T11:19:22.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7a9cd3b80 1 client.6249 shutdown 2023-08-27T11:19:22.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/551015215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907e5960 msgr2=0x55f69083a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/551015215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907e5960 0x55f69083a460 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa7900535e0 tx=0x7fa7900568b0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/551015215 shutdown_connections 2023-08-27T11:19:22.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/551015215 >> [v2:172.21.15.112:6833/174707415,v1:172.21.15.112:6835/174707415] conn(0x55f6908070e0 0x55f6907a2b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/551015215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6907e5960 0x55f69083a460 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/551015215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6907a73d0 0x55f6907fe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7767fc700 0 client.6249 ms_handle_reset on v2:172.21.15.112:6833/174707415 2023-08-27T11:19:22.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7a9cd3b80 1 --2- 172.21.15.112:0/551015215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6907d3f40 0x55f6908080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/551015215 >> 172.21.15.112:0/551015215 conn(0x55f690707c30 msgr2=0x55f6907aab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:22.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/551015215 shutdown_connections 2023-08-27T11:19:22.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7a9cd3b80 1 -- 172.21.15.112:0/551015215 wait complete. 2023-08-27T11:19:22.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.555+0000 7fa7a9cd3b80 20 client.6249 trim_cache size 0 max 16384 2023-08-27T11:19:22.587 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:19:22.588 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e099a24f-c1cd-4f11-ac4f-ecacc10e5e98", 2023-08-27T11:19:22.588 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2023-08-27T11:19:22.588 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 6093, 2023-08-27T11:19:22.588 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2023-08-27T11:19:22.589 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2023-08-27T11:19:22.589 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2023-08-27T11:19:22.589 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 5.8894642780000002, 2023-08-27T11:19:22.589 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 39, 2023-08-27T11:19:22.589 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 65, 2023-08-27T11:19:22.590 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 65, 2023-08-27T11:19:22.590 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 11.807495977 2023-08-27T11:19:22.590 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:19:22.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.556+0000 7fa79db11700 1 -- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa77806a170 msgr2=0x7fa77806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.556+0000 7fa79db11700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa77806a170 0x7fa77806c630 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa788000f80 tx=0x7fa788006040 comp rx=0 tx=0).stop 2023-08-27T11:19:22.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.556+0000 7fa79db11700 1 -- 172.21.15.112:0/622904216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7980ae9c0 msgr2=0x7fa7980aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.556+0000 7fa79db11700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7980ae9c0 0x7fa7980aee50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa79806df40 tx=0x7fa790010ad0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.556+0000 7fa79db11700 1 -- 172.21.15.112:0/622904216 shutdown_connections 2023-08-27T11:19:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.556+0000 7fa79db11700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa77806a170 0x7fa77806c630 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.556+0000 7fa79db11700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7980ae9c0 0x7fa7980aee50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.556+0000 7fa79db11700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa784003f80 0x7fa7980c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.556+0000 7fa79db11700 1 --2- 172.21.15.112:0/622904216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7980acb10 0x7fa798154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.556+0000 7fa79db11700 1 -- 172.21.15.112:0/622904216 >> 172.21.15.112:0/622904216 conn(0x7fa7980d0000 msgr2=0x7fa7980d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.556+0000 7fa79db11700 1 -- 172.21.15.112:0/622904216 shutdown_connections 2023-08-27T11:19:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.556+0000 7fa79db11700 1 -- 172.21.15.112:0/622904216 wait complete. 2023-08-27T11:19:22.595 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:19:22.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.714+0000 7ff1996c2700 1 Processor -- start 2023-08-27T11:19:22.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.714+0000 7ff1996c2700 1 -- start start 2023-08-27T11:19:22.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff1996c2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff194157880 0x7ff194160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff1996c2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19415f110 con 0x7ff194161430 2023-08-27T11:19:22.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff1996c2700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1940d1490 con 0x7ff1940d0e30 2023-08-27T11:19:22.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff1996c2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1941618b0 con 0x7ff194157880 2023-08-27T11:19:22.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff1927fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff194161430 0x7ff1940d0720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53506/0 (socket says 172.21.15.112:53506) 2023-08-27T11:19:22.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff1927fc700 1 -- 172.21.15.112:0/1215900150 learned_addr learned my addr 172.21.15.112:0/1215900150 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:22.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff192ffd700 1 --2- 172.21.15.112:0/1215900150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff194157880 0x7ff194160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625208910 0 0) 0x7ff19415f110 con 0x7ff194161430 2023-08-27T11:19:22.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff17c0035b0 con 0x7ff194161430 2023-08-27T11:19:22.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2161248716 0 0) 0x7ff17c0035b0 con 0x7ff194161430 2023-08-27T11:19:22.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff194157880 msgr2=0x7ff194160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 --2- 172.21.15.112:0/1215900150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff194157880 0x7ff194160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 >> v1:172.21.15.138:6789/0 conn(0x7ff1940d0e30 legacy=0x7ff19415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1940d1770 con 0x7ff194161430 2023-08-27T11:19:22.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7ff188001670 con 0x7ff194161430 2023-08-27T11:19:22.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 >> v1:172.21.15.112:6789/0 conn(0x7ff194161430 legacy=0x7ff1940d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 --2- 172.21.15.112:0/1215900150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff17c003a30 0x7ff17c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 --2- 172.21.15.112:0/1215900150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c004850 0x7ff17c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 --2- 172.21.15.112:0/1215900150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff194157880 0x7ff17c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1941618b0 con 0x7ff17c003a30 2023-08-27T11:19:22.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c00bac0 con 0x7ff17c004850 2023-08-27T11:19:22.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.715+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c00bc80 con 0x7ff194157880 2023-08-27T11:19:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff192ffd700 1 --2- 172.21.15.112:0/1215900150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff17c003a30 0x7ff17c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff1927fc700 1 --2- 172.21.15.112:0/1215900150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c004850 0x7ff17c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff1937fe700 1 --2- 172.21.15.112:0/1215900150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff194157880 0x7ff17c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff192ffd700 1 -- 172.21.15.112:0/1215900150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff194157880 msgr2=0x7ff17c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff192ffd700 1 --2- 172.21.15.112:0/1215900150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff194157880 0x7ff17c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff192ffd700 1 -- 172.21.15.112:0/1215900150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c004850 msgr2=0x7ff17c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff192ffd700 1 --2- 172.21.15.112:0/1215900150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c004850 0x7ff17c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff192ffd700 1 -- 172.21.15.112:0/1215900150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0035b0 con 0x7ff17c003a30 2023-08-27T11:19:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff1937fe700 1 --2- 172.21.15.112:0/1215900150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff194157880 0x7ff17c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff192ffd700 1 --2- 172.21.15.112:0/1215900150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff17c003a30 0x7ff17c003e50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff184009b30 tx=0x7ff17c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566750eb584745cd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:22.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18400e6c0 con 0x7ff17c003a30 2023-08-27T11:19:22.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18400e820 con 0x7ff17c003a30 2023-08-27T11:19:22.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff191ffb700 1 -- 172.21.15.112:0/1215900150 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18400eb90 con 0x7ff17c003a30 2023-08-27T11:19:22.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff1996c2700 1 -- 172.21.15.112:0/1215900150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff17c003a30 msgr2=0x7ff17c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff1996c2700 1 --2- 172.21.15.112:0/1215900150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff17c003a30 0x7ff17c003e50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff184009b30 tx=0x7ff17c005880 comp rx=0 tx=0).stop 2023-08-27T11:19:22.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff1937fe700 1 -- 172.21.15.112:0/1215900150 reap_dead start 2023-08-27T11:19:22.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.716+0000 7ff1996c2700 1 -- 172.21.15.112:0/1215900150 shutdown_connections 2023-08-27T11:19:22.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.717+0000 7ff1996c2700 1 -- 172.21.15.112:0/1215900150 >> 172.21.15.112:0/1215900150 conn(0x7ff19406c880 msgr2=0x7ff1940717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:22.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.717+0000 7ff1996c2700 1 -- 172.21.15.112:0/1215900150 shutdown_connections 2023-08-27T11:19:22.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.717+0000 7ff1996c2700 1 -- 172.21.15.112:0/1215900150 wait complete. 2023-08-27T11:19:22.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.718+0000 7ff1996c2700 1 Processor -- start 2023-08-27T11:19:22.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.718+0000 7ff1996c2700 1 -- start start 2023-08-27T11:19:22.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1996c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c003a30 0x7ff1941f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1996c2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1941f36b0 0x7ff194070780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1996c2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff194070cc0 0x7ff194155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1996c2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff194071170 con 0x7ff194070cc0 2023-08-27T11:19:22.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1996c2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1941f5f80 con 0x7ff17c003a30 2023-08-27T11:19:22.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1996c2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1941f61d0 con 0x7ff1941f36b0 2023-08-27T11:19:22.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1937fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff194070cc0 0x7ff194155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1937fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff194070cc0 0x7ff194155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59138/0 (socket says 172.21.15.112:59138) 2023-08-27T11:19:22.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1937fe700 1 -- 172.21.15.112:0/3811504396 learned_addr learned my addr 172.21.15.112:0/3811504396 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:22.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff192ffd700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c003a30 0x7ff1941f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1927fc700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1941f36b0 0x7ff194070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1937fe700 1 -- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1941f36b0 msgr2=0x7ff194070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1937fe700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1941f36b0 0x7ff194070780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1937fe700 1 -- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c003a30 msgr2=0x7ff1941f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1937fe700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c003a30 0x7ff1941f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1937fe700 1 -- 172.21.15.112:0/3811504396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1941f64b0 con 0x7ff194070cc0 2023-08-27T11:19:22.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1937fe700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff194070cc0 0x7ff194155910 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff194160e30 tx=0x7ff18c00ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:22.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff17b7fe700 1 -- 172.21.15.112:0/3811504396 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c003e10 con 0x7ff194070cc0 2023-08-27T11:19:22.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff17b7fe700 1 -- 172.21.15.112:0/3811504396 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c005070 con 0x7ff194070cc0 2023-08-27T11:19:22.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff17b7fe700 1 -- 172.21.15.112:0/3811504396 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c005420 con 0x7ff194070cc0 2023-08-27T11:19:22.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1996c2700 1 -- 172.21.15.112:0/3811504396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1941f6790 con 0x7ff194070cc0 2023-08-27T11:19:22.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.719+0000 7ff1996c2700 1 -- 172.21.15.112:0/3811504396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1941538b0 con 0x7ff194070cc0 2023-08-27T11:19:22.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.720+0000 7ff1996c2700 1 -- 172.21.15.112:0/3811504396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1941f6cb0 con 0x7ff194070cc0 2023-08-27T11:19:22.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.721+0000 7ff17b7fe700 1 -- 172.21.15.112:0/3811504396 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7ff18c005580 con 0x7ff194070cc0 2023-08-27T11:19:22.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.721+0000 7ff17b7fe700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff18006a170 0x7ff18006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:22.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.721+0000 7ff17b7fe700 1 -- 172.21.15.112:0/3811504396 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff18c088100 con 0x7ff194070cc0 2023-08-27T11:19:22.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.721+0000 7ff192ffd700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff18006a170 0x7ff18006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:22.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.722+0000 7ff192ffd700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff18006a170 0x7ff18006c630 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff1840090b0 tx=0x7ff18401f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:22.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.725+0000 7ff17b7fe700 1 -- 172.21.15.112:0/3811504396 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff1941f6cb0 con 0x7ff194070cc0 2023-08-27T11:19:22.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.958+0000 7ff1996c2700 1 -- 172.21.15.112:0/3811504396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff1941f6cb0 con 0x7ff194070cc0 2023-08-27T11:19:22.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.958+0000 7ff17b7fe700 1 -- 172.21.15.112:0/3811504396 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff1941f6cb0 con 0x7ff194070cc0 2023-08-27T11:19:22.960 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:19:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.960+0000 7ff1996c2700 1 -- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff18006a170 msgr2=0x7ff18006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.960+0000 7ff1996c2700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff18006a170 0x7ff18006c630 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff1840090b0 tx=0x7ff18401f040 comp rx=0 tx=0).stop 2023-08-27T11:19:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.960+0000 7ff1996c2700 1 -- 172.21.15.112:0/3811504396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff194070cc0 msgr2=0x7ff194155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.960+0000 7ff1996c2700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff194070cc0 0x7ff194155910 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff194160e30 tx=0x7ff18c00ddc0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.960+0000 7ff1996c2700 1 -- 172.21.15.112:0/3811504396 shutdown_connections 2023-08-27T11:19:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.960+0000 7ff1996c2700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff18006a170 0x7ff18006c630 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.960+0000 7ff1996c2700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff194070cc0 0x7ff194155910 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.960+0000 7ff1996c2700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c003a30 0x7ff1941f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.960+0000 7ff1996c2700 1 --2- 172.21.15.112:0/3811504396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1941f36b0 0x7ff194070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.961+0000 7ff1996c2700 1 -- 172.21.15.112:0/3811504396 >> 172.21.15.112:0/3811504396 conn(0x7ff19406c880 msgr2=0x7ff1940b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.961+0000 7ff1996c2700 1 -- 172.21.15.112:0/3811504396 shutdown_connections 2023-08-27T11:19:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:22.961+0000 7ff1996c2700 1 -- 172.21.15.112:0/3811504396 wait complete. 2023-08-27T11:19:22.974 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:19:22.974 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:19:22.975 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-08-27T11:19:22.975 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:19:22.975 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:19:22.976 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:19:22.976 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:19:22.976 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:19:22.977 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:19:22.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:22.977 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-08-27T11:19:22.995 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:19:22.995 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:19:22.995 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:19:22.995 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:22.996 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:19:22.996 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:22.996 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:19:22.996 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-08-27T11:19:22.996 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:19:22.997 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-08-27T11:19:22.997 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:19:22.997 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:19:22.997 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:19:22.997 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:19:22.998 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:19:22.998 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42280sec preferred_lft 42280sec 2023-08-27T11:19:22.998 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8d6c:4fe6:6a84:8222/64 scope link noprefixroute 2023-08-27T11:19:22.998 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:22.998 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:19:22.999 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff 2023-08-27T11:19:22.999 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:19:22.999 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:22.999 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::d87c:a1ff:fe1d:b635/64 scope link 2023-08-27T11:19:22.999 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:23.000 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:19:23.000 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:19:23.000 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::cc14:79ff:feb3:452f/64 scope link 2023-08-27T11:19:23.000 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:23.000 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:19:23.001 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether e6:51:f0:78:04:02 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:19:23.001 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::e451:f0ff:fe78:402/64 scope link 2023-08-27T11:19:23.001 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:23.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:23.002 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:19:23.060 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-08-27T11:19:23.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:23.062 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-08-27T11:19:23.122 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-08-27T11:19:23.122 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:19:23.122 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:19:23.123 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:19:23.123 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:19:23.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:23.124 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:19:23.124 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:19:23.124 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:19:23.125 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:19:23.125 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:19:23.125 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:19:23.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:23.228 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:19:23.245 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:19:23.245 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:19:23.246 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:19:23.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:23.247 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-08-27T11:19:23.313 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:19:23.314 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:19:23.370 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:19:23.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:19:23.371 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:19:23.426 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:19:23.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:23.427 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:19:23.511 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:19:23.512 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:19:23.512 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:19:23.520 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:19:23.521 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:19:23.521 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:19:23.521 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:19:23.521 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:19:23.546 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:19:23.546 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:19:23.546 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:19:23.547 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:19:23.547 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:19:23.547 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:19:23.547 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:19:23.547 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:19:23.549 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:19:23.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:23.550 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:19:23.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:23.581 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:19:23.663 INFO:teuthology.orchestra.run.smithi112.stdout:client6258 2023-08-27T11:19:23.664 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:23.665 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:23.724 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:19:23.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.883+0000 7f09bf070700 1 Processor -- start 2023-08-27T11:19:23.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.883+0000 7f09bf070700 1 -- start start 2023-08-27T11:19:23.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.884+0000 7f09bf070700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f09b8140430 0x7f09b806e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:23.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.884+0000 7f09bf070700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09b806e690 0x7f09b80acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:23.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.884+0000 7f09bf070700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b806eaf0 con 0x7f09b80ad150 2023-08-27T11:19:23.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.884+0000 7f09bf070700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b8140810 con 0x7f09b8140430 2023-08-27T11:19:23.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.884+0000 7f09bf070700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b815db70 con 0x7f09b806e690 2023-08-27T11:19:23.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.884+0000 7f09bce0c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f09b8140430 0x7f09b806e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:23.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.884+0000 7f09bce0c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f09b8140430 0x7f09b806e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59146/0 (socket says 172.21.15.112:59146) 2023-08-27T11:19:23.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.884+0000 7f09bce0c700 1 -- 172.21.15.112:0/907274878 learned_addr learned my addr 172.21.15.112:0/907274878 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:23.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.884+0000 7f09bce0c700 1 -- 172.21.15.112:0/907274878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09b806e690 msgr2=0x7f09b80acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:23.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.884+0000 7f09bce0c700 1 --2- 172.21.15.112:0/907274878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09b806e690 0x7f09b80acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:23.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.884+0000 7f09bce0c700 1 -- 172.21.15.112:0/907274878 >> v1:172.21.15.138:6789/0 conn(0x7f09b80ad150 legacy=0x7f09b8160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:23.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.884+0000 7f09bce0c700 1 -- 172.21.15.112:0/907274878 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b815de50 con 0x7f09b8140430 2023-08-27T11:19:23.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.885+0000 7f09bce0c700 1 --2- 172.21.15.112:0/907274878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b8140430 0x7f09b806e150 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0098f0 tx=0x7f09ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40caff949baf49f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:23.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.885+0000 7f09b77fe700 1 -- 172.21.15.112:0/907274878 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac004230 con 0x7f09b8140430 2023-08-27T11:19:23.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.885+0000 7f09b77fe700 1 -- 172.21.15.112:0/907274878 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ac004390 con 0x7f09b8140430 2023-08-27T11:19:23.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.885+0000 7f09b77fe700 1 -- 172.21.15.112:0/907274878 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac0036a0 con 0x7f09b8140430 2023-08-27T11:19:23.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.885+0000 7f09bf070700 1 -- 172.21.15.112:0/907274878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b8140430 msgr2=0x7f09b806e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:23.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.885+0000 7f09bf070700 1 --2- 172.21.15.112:0/907274878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b8140430 0x7f09b806e150 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0098f0 tx=0x7f09ac009c00 comp rx=0 tx=0).stop 2023-08-27T11:19:23.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.885+0000 7f09bf070700 1 -- 172.21.15.112:0/907274878 shutdown_connections 2023-08-27T11:19:23.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.885+0000 7f09bf070700 1 --2- 172.21.15.112:0/907274878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b8140430 0x7f09b806e150 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:23.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.885+0000 7f09bf070700 1 --2- 172.21.15.112:0/907274878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09b806e690 0x7f09b80acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:23.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.885+0000 7f09bf070700 1 -- 172.21.15.112:0/907274878 >> 172.21.15.112:0/907274878 conn(0x7f09b806cd30 msgr2=0x7f09b80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:23.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.885+0000 7f09bf070700 1 -- 172.21.15.112:0/907274878 shutdown_connections 2023-08-27T11:19:23.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.885+0000 7f09bf070700 1 -- 172.21.15.112:0/907274878 wait complete. 2023-08-27T11:19:23.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.886+0000 7f09bf070700 1 Processor -- start 2023-08-27T11:19:23.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.886+0000 7f09bf070700 1 -- start start 2023-08-27T11:19:23.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.887+0000 7f09bf070700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09b806e690 0x7f09b81f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:23.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.887+0000 7f09bf070700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b80ad150 0x7f09b80af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:23.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.887+0000 7f09bf070700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09b8140430 0x7f09b80b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:23.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.887+0000 7f09bf070700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b8158db0 con 0x7f09b80ad150 2023-08-27T11:19:23.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.887+0000 7f09bf070700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b8158130 con 0x7f09b8140430 2023-08-27T11:19:23.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.887+0000 7f09bf070700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b8158410 con 0x7f09b806e690 2023-08-27T11:19:23.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.887+0000 7f09b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b80ad150 0x7f09b80af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:23.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.887+0000 7f09b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b80ad150 0x7f09b80af5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59152/0 (socket says 172.21.15.112:59152) 2023-08-27T11:19:23.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.887+0000 7f09b7fff700 1 -- 172.21.15.112:0/3768928605 learned_addr learned my addr 172.21.15.112:0/3768928605 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:23.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09bd60d700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09b8140430 0x7f09b80b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:23.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09bce0c700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09b806e690 0x7f09b81f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:23.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09b7fff700 1 -- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09b806e690 msgr2=0x7f09b81f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:23.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09b7fff700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09b806e690 0x7f09b81f3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:23.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09b7fff700 1 -- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09b8140430 msgr2=0x7f09b80b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:23.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09b7fff700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09b8140430 0x7f09b80b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:23.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09b7fff700 1 -- 172.21.15.112:0/3768928605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac009550 con 0x7f09b80ad150 2023-08-27T11:19:23.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09bd60d700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09b8140430 0x7f09b80b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:23.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09bce0c700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09b806e690 0x7f09b81f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:23.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09b7fff700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b80ad150 0x7f09b80af5e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f09a400b520 tx=0x7f09a400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:23.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09b57fa700 1 -- 172.21.15.112:0/3768928605 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4005aa0 con 0x7f09b80ad150 2023-08-27T11:19:23.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09b57fa700 1 -- 172.21.15.112:0/3768928605 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a4005c00 con 0x7f09b80ad150 2023-08-27T11:19:23.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09bf070700 1 -- 172.21.15.112:0/3768928605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b81586f0 con 0x7f09b80ad150 2023-08-27T11:19:23.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09b57fa700 1 -- 172.21.15.112:0/3768928605 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4011630 con 0x7f09b80ad150 2023-08-27T11:19:23.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.888+0000 7f09bf070700 1 -- 172.21.15.112:0/3768928605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b81f4290 con 0x7f09b80ad150 2023-08-27T11:19:23.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.889+0000 7f09bf070700 1 -- 172.21.15.112:0/3768928605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b80cd4b0 con 0x7f09b80ad150 2023-08-27T11:19:23.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.889+0000 7f09b57fa700 1 -- 172.21.15.112:0/3768928605 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f09a4011790 con 0x7f09b80ad150 2023-08-27T11:19:23.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.890+0000 7f09b57fa700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f099c06a170 0x7f099c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:23.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.890+0000 7f09b57fa700 1 -- 172.21.15.112:0/3768928605 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f09a4088200 con 0x7f09b80ad150 2023-08-27T11:19:23.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.890+0000 7f09bce0c700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f099c06a170 0x7f099c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:23.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.890+0000 7f09bce0c700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f099c06a170 0x7f099c06c630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f09b80720d0 tx=0x7f09ac008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:23.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:23.894+0000 7f09b57fa700 1 -- 172.21.15.112:0/3768928605 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f09a4053900 con 0x7f09b80ad150 2023-08-27T11:19:24.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.124+0000 7f09bf070700 1 -- 172.21.15.112:0/3768928605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f09b8158ef0 con 0x7f09b80ad150 2023-08-27T11:19:24.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.124+0000 7f09b57fa700 1 -- 172.21.15.112:0/3768928605 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7f09b8158ef0 con 0x7f09b80ad150 2023-08-27T11:19:24.126 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:19:24.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.126+0000 7f09bf070700 1 -- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f099c06a170 msgr2=0x7f099c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:24.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.126+0000 7f09bf070700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f099c06a170 0x7f099c06c630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f09b80720d0 tx=0x7f09ac008030 comp rx=0 tx=0).stop 2023-08-27T11:19:24.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.126+0000 7f09bf070700 1 -- 172.21.15.112:0/3768928605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b80ad150 msgr2=0x7f09b80af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:24.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.126+0000 7f09bf070700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b80ad150 0x7f09b80af5e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f09a400b520 tx=0x7f09a400b8e0 comp rx=0 tx=0).stop 2023-08-27T11:19:24.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.126+0000 7f09bf070700 1 -- 172.21.15.112:0/3768928605 shutdown_connections 2023-08-27T11:19:24.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.126+0000 7f09bf070700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f099c06a170 0x7f099c06c630 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:24.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.126+0000 7f09bf070700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09b8140430 0x7f09b80b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:24.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.126+0000 7f09bf070700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09b806e690 0x7f09b81f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:24.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.126+0000 7f09bf070700 1 --2- 172.21.15.112:0/3768928605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b80ad150 0x7f09b80af5e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.126+0000 7f09bf070700 1 -- 172.21.15.112:0/3768928605 >> 172.21.15.112:0/3768928605 conn(0x7f09b806cd30 msgr2=0x7f09b8071610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.126+0000 7f09bf070700 1 -- 172.21.15.112:0/3768928605 shutdown_connections 2023-08-27T11:19:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:24.126+0000 7f09bf070700 1 -- 172.21.15.112:0/3768928605 wait complete. 2023-08-27T11:19:24.140 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:19:24.140 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:19:24.141 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2023-08-27T11:19:24.141 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-27T11:19:24.142 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:19:24.142 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:19:24.142 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:19:24.143 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:19:24.143 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:19:24.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:24.144 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2023-08-27T11:19:24.166 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:19:24.166 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:19:24.166 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:19:24.166 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:19:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:24.167 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:19:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2023-08-27T11:19:24.168 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:19:24.168 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2023-08-27T11:19:24.168 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:19:24.168 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:19:24.168 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:19:24.169 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:19:24.169 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:19:24.169 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42294sec preferred_lft 42294sec 2023-08-27T11:19:24.169 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ac17:49c5:cb39:801a/64 scope link noprefixroute 2023-08-27T11:19:24.169 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:24.170 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:19:24.170 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:19:24.170 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:19:24.171 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:24.171 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c4b4:94ff:fe58:4e35/64 scope link 2023-08-27T11:19:24.171 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:24.171 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:19:24.172 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:19:24.172 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::5806:6aff:fe04:ef32/64 scope link 2023-08-27T11:19:24.172 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:24.172 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:19:24.173 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 8a:21:ba:15:69:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-27T11:19:24.173 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::8821:baff:fe15:6919/64 scope link 2023-08-27T11:19:24.173 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:19:24.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:24.174 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:19:24.236 INFO:teuthology.orchestra.run.smithi138.stdout:1 2023-08-27T11:19:24.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:24.241 DEBUG:teuthology.orchestra.run.smithi138:> route 2023-08-27T11:19:24.299 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2023-08-27T11:19:24.300 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:19:24.300 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:19:24.300 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:19:24.300 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:19:24.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:24.302 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:19:24.303 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:19:24.303 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:19:24.303 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:19:24.304 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:19:24.304 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:19:24.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:24.410 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:19:24.427 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-27T11:19:24.428 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-27T11:19:24.428 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:19:24.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:24.429 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2023-08-27T11:19:24.495 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:19:24.495 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-27T11:19:24.553 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:19:24.554 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:19:24.554 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:19:24.610 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:19:24.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:24.611 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:19:24.695 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:19:24.696 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:19:24.696 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:19:24.704 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:19:24.704 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:19:24.704 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:19:24.705 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:19:24.705 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:19:24.778 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:19:24.779 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:19:24.779 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:19:24.779 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:19:24.779 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:19:24.780 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:19:24.780 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:19:24.780 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:19:24.781 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:19:24.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:24.782 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:19:24.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:24.810 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:19:24.881 INFO:teuthology.orchestra.run.smithi138.stdout:client6267 2023-08-27T11:19:24.882 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:19:24.882 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:24.942 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-08-27T11:19:25.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.071+0000 7f3f63467700 1 Processor -- start 2023-08-27T11:19:25.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.071+0000 7f3f63467700 1 -- start start 2023-08-27T11:19:25.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.072+0000 7f3f63467700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3f5c157ae0 0x7f3f5c15e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:25.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.072+0000 7f3f63467700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f5c06f790 0x7f3f5c06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:25.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.072+0000 7f3f63467700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5c161390 con 0x7f3f5c0b3de0 2023-08-27T11:19:25.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.072+0000 7f3f63467700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c0b1aa0 con 0x7f3f5c06f790 2023-08-27T11:19:25.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.072+0000 7f3f63467700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c070160 con 0x7f3f5c157ae0 2023-08-27T11:19:25.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.072+0000 7f3f61a04700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f5c06f790 0x7f3f5c06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:25.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.073+0000 7f3f60a02700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3f5c0b3de0 0x7f3f5c161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:32834/0 (socket says 172.21.15.112:32834) 2023-08-27T11:19:25.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.073+0000 7f3f61a04700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f5c06f790 0x7f3f5c06fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59162/0 (socket says 172.21.15.112:59162) 2023-08-27T11:19:25.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.073+0000 7f3f60a02700 1 -- 172.21.15.112:0/3864261607 learned_addr learned my addr 172.21.15.112:0/3864261607 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:25.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.073+0000 7f3f61203700 1 --2- 172.21.15.112:0/3864261607 >> v2:172.21.15.138:3300/0 conn(0x7f3f5c157ae0 0x7f3f5c15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:25.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.073+0000 7f3f61a04700 1 -- 172.21.15.112:0/3864261607 >> v2:172.21.15.138:3300/0 conn(0x7f3f5c157ae0 msgr2=0x7f3f5c15e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:25.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.073+0000 7f3f61a04700 1 --2- 172.21.15.112:0/3864261607 >> v2:172.21.15.138:3300/0 conn(0x7f3f5c157ae0 0x7f3f5c15e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:25.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.073+0000 7f3f53fff700 1 -- 172.21.15.112:0/3864261607 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887572924 0 0) 0x7f3f5c161390 con 0x7f3f5c0b3de0 2023-08-27T11:19:25.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.073+0000 7f3f61a04700 1 -- 172.21.15.112:0/3864261607 >> v1:172.21.15.138:6789/0 conn(0x7f3f5c0b3de0 legacy=0x7f3f5c161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:25.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.073+0000 7f3f61a04700 1 -- 172.21.15.112:0/3864261607 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c0b35f0 con 0x7f3f5c06f790 2023-08-27T11:19:25.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.073+0000 7f3f61a04700 1 --2- 172.21.15.112:0/3864261607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c06f790 0x7f3f5c06fc20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c06e580 tx=0x7f3f5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586ef15fd597470f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:25.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.073+0000 7f3f53fff700 1 -- 172.21.15.112:0/3864261607 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400e070 con 0x7f3f5c06f790 2023-08-27T11:19:25.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.073+0000 7f3f53fff700 1 -- 172.21.15.112:0/3864261607 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5400bb20 con 0x7f3f5c06f790 2023-08-27T11:19:25.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.073+0000 7f3f53fff700 1 -- 172.21.15.112:0/3864261607 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54012430 con 0x7f3f5c06f790 2023-08-27T11:19:25.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.074+0000 7f3f63467700 1 -- 172.21.15.112:0/3864261607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c06f790 msgr2=0x7f3f5c06fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:25.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.074+0000 7f3f63467700 1 --2- 172.21.15.112:0/3864261607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c06f790 0x7f3f5c06fc20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c06e580 tx=0x7f3f5400b330 comp rx=0 tx=0).stop 2023-08-27T11:19:25.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.074+0000 7f3f63467700 1 -- 172.21.15.112:0/3864261607 shutdown_connections 2023-08-27T11:19:25.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.074+0000 7f3f63467700 1 --2- 172.21.15.112:0/3864261607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c06f790 0x7f3f5c06fc20 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:25.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.074+0000 7f3f63467700 1 --2- 172.21.15.112:0/3864261607 >> v2:172.21.15.138:3300/0 conn(0x7f3f5c157ae0 0x7f3f5c15e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:25.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.074+0000 7f3f63467700 1 -- 172.21.15.112:0/3864261607 >> 172.21.15.112:0/3864261607 conn(0x7f3f5c0b1be0 msgr2=0x7f3f5c0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:25.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.074+0000 7f3f63467700 1 -- 172.21.15.112:0/3864261607 shutdown_connections 2023-08-27T11:19:25.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.074+0000 7f3f63467700 1 -- 172.21.15.112:0/3864261607 wait complete. 2023-08-27T11:19:25.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.075+0000 7f3f63467700 1 Processor -- start 2023-08-27T11:19:25.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.075+0000 7f3f63467700 1 -- start start 2023-08-27T11:19:25.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f63467700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c06f790 0x7f3f5c0ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:25.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f63467700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f5c0b3de0 0x7f3f5c15e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:25.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f63467700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c157ae0 0x7f3f5c1f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:25.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f63467700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0ae800 con 0x7f3f5c157ae0 2023-08-27T11:19:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f63467700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c15f730 con 0x7f3f5c06f790 2023-08-27T11:19:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f63467700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c15fa10 con 0x7f3f5c0b3de0 2023-08-27T11:19:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f61a04700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c157ae0 0x7f3f5c1f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f61a04700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c157ae0 0x7f3f5c1f6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59168/0 (socket says 172.21.15.112:59168) 2023-08-27T11:19:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f61a04700 1 -- 172.21.15.112:0/2821567193 learned_addr learned my addr 172.21.15.112:0/2821567193 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f61203700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c06f790 0x7f3f5c0ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f60a02700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f5c0b3de0 0x7f3f5c15e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f61a04700 1 -- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f5c0b3de0 msgr2=0x7f3f5c15e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f61a04700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f5c0b3de0 0x7f3f5c15e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f61a04700 1 -- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c06f790 msgr2=0x7f3f5c0ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f61a04700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c06f790 0x7f3f5c0ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:25.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f61a04700 1 -- 172.21.15.112:0/2821567193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f54009a70 con 0x7f3f5c157ae0 2023-08-27T11:19:25.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.076+0000 7f3f61203700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c06f790 0x7f3f5c0ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:19:25.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.077+0000 7f3f61a04700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c157ae0 0x7f3f5c1f6550 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c06e930 tx=0x7f3f54004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:25.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.077+0000 7f3f51ffb700 1 -- 172.21.15.112:0/2821567193 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54012430 con 0x7f3f5c157ae0 2023-08-27T11:19:25.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.077+0000 7f3f51ffb700 1 -- 172.21.15.112:0/2821567193 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f54020070 con 0x7f3f5c157ae0 2023-08-27T11:19:25.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.077+0000 7f3f63467700 1 -- 172.21.15.112:0/2821567193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5c15fcf0 con 0x7f3f5c157ae0 2023-08-27T11:19:25.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.077+0000 7f3f51ffb700 1 -- 172.21.15.112:0/2821567193 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54004280 con 0x7f3f5c157ae0 2023-08-27T11:19:25.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.077+0000 7f3f63467700 1 -- 172.21.15.112:0/2821567193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5c1602b0 con 0x7f3f5c157ae0 2023-08-27T11:19:25.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.078+0000 7f3f63467700 1 -- 172.21.15.112:0/2821567193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5c067a20 con 0x7f3f5c157ae0 2023-08-27T11:19:25.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.078+0000 7f3f51ffb700 1 -- 172.21.15.112:0/2821567193 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f3f540043e0 con 0x7f3f5c157ae0 2023-08-27T11:19:25.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.079+0000 7f3f51ffb700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3f4006a170 0x7f3f4006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:25.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.079+0000 7f3f51ffb700 1 -- 172.21.15.112:0/2821567193 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f54022070 con 0x7f3f5c157ae0 2023-08-27T11:19:25.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.079+0000 7f3f61203700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3f4006a170 0x7f3f4006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:25.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.079+0000 7f3f61203700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3f4006a170 0x7f3f4006c630 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009f30 tx=0x7f3f4c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:25.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.083+0000 7f3f51ffb700 1 -- 172.21.15.112:0/2821567193 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3f54055cf0 con 0x7f3f5c157ae0 2023-08-27T11:19:25.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:25.316+0000 7f3f63467700 1 -- 172.21.15.112:0/2821567193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3f5c0af580 con 0x7f3f5c157ae0 2023-08-27T11:19:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.265+0000 7f3f51ffb700 1 -- 172.21.15.112:0/2821567193 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 136+0+0 (secure 0 0 0) 0x7f3f54059310 con 0x7f3f5c157ae0 2023-08-27T11:19:26.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.268+0000 7f3f63467700 1 -- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3f4006a170 msgr2=0x7f3f4006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:26.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.268+0000 7f3f63467700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3f4006a170 0x7f3f4006c630 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009f30 tx=0x7f3f4c009210 comp rx=0 tx=0).stop 2023-08-27T11:19:26.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.268+0000 7f3f63467700 1 -- 172.21.15.112:0/2821567193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c157ae0 msgr2=0x7f3f5c1f6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:26.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.268+0000 7f3f63467700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c157ae0 0x7f3f5c1f6550 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c06e930 tx=0x7f3f54004ad0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.268+0000 7f3f63467700 1 -- 172.21.15.112:0/2821567193 shutdown_connections 2023-08-27T11:19:26.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.268+0000 7f3f63467700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3f4006a170 0x7f3f4006c630 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.268+0000 7f3f63467700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c157ae0 0x7f3f5c1f6550 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.268+0000 7f3f63467700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c06f790 0x7f3f5c0ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.268+0000 7f3f63467700 1 --2- 172.21.15.112:0/2821567193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f5c0b3de0 0x7f3f5c15e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.268+0000 7f3f63467700 1 -- 172.21.15.112:0/2821567193 >> 172.21.15.112:0/2821567193 conn(0x7f3f5c0b1be0 msgr2=0x7f3f5c145ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:26.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.268+0000 7f3f63467700 1 -- 172.21.15.112:0/2821567193 shutdown_connections 2023-08-27T11:19:26.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.268+0000 7f3f63467700 1 -- 172.21.15.112:0/2821567193 wait complete. 2023-08-27T11:19:26.284 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:19:26.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.410+0000 7fd092b7f700 1 Processor -- start 2023-08-27T11:19:26.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.410+0000 7fd092b7f700 1 -- start start 2023-08-27T11:19:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd092b7f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd08c0b1400 0x7fd08c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd092b7f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd08c0b17e0 0x7fd08c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd092b7f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08c161850 con 0x7fd08c161ce0 2023-08-27T11:19:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd092b7f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08c06e7d0 con 0x7fd08c0b1400 2023-08-27T11:19:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd092b7f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0cfc10 con 0x7fd08c0b17e0 2023-08-27T11:19:26.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd09111c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd08c161ce0 0x7fd08c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53558/0 (socket says 172.21.15.112:53558) 2023-08-27T11:19:26.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd09111c700 1 -- 172.21.15.112:0/2853532203 learned_addr learned my addr 172.21.15.112:0/2853532203 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:26.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd08bfff700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd08c0b17e0 0x7fd08c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd09091b700 1 --2- 172.21.15.112:0/2853532203 >> v2:172.21.15.138:3300/0 conn(0x7fd08c0b1400 0x7fd08c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487930707 0 0) 0x7fd08c161850 con 0x7fd08c161ce0 2023-08-27T11:19:26.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd07c0035b0 con 0x7fd08c161ce0 2023-08-27T11:19:26.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4051673578 0 0) 0x7fd07c0035b0 con 0x7fd08c161ce0 2023-08-27T11:19:26.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd08c0b17e0 msgr2=0x7fd08c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:26.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd08b7fe700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd08c0b17e0 0x7fd08c06e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 >> v2:172.21.15.138:3300/0 conn(0x7fd08c0b1400 msgr2=0x7fd08c0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:26.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd08b7fe700 1 --2- 172.21.15.112:0/2853532203 >> v2:172.21.15.138:3300/0 conn(0x7fd08c0b1400 0x7fd08c0cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c161200 con 0x7fd08c161ce0 2023-08-27T11:19:26.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fd0840019f0 con 0x7fd08c161ce0 2023-08-27T11:19:26.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 >> v1:172.21.15.112:6789/0 conn(0x7fd08c161ce0 legacy=0x7fd08c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:26.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd08b7fe700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd07c003a00 0x7fd07c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.411+0000 7fd08b7fe700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd08c0b17e0 0x7fd07c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd08b7fe700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d90 0x7fd07c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c06e7d0 con 0x7fd07c003a00 2023-08-27T11:19:26.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0cfc10 con 0x7fd07c004d90 2023-08-27T11:19:26.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c00ba90 con 0x7fd08c0b17e0 2023-08-27T11:19:26.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd09091b700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd07c003a00 0x7fd07c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd08bfff700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd08c0b17e0 0x7fd07c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd09111c700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d90 0x7fd07c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd09091b700 1 -- 172.21.15.112:0/2853532203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd08c0b17e0 msgr2=0x7fd07c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd09091b700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd08c0b17e0 0x7fd07c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd09091b700 1 -- 172.21.15.112:0/2853532203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d90 msgr2=0x7fd07c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd09091b700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d90 0x7fd07c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd09091b700 1 -- 172.21.15.112:0/2853532203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c0035b0 con 0x7fd07c003a00 2023-08-27T11:19:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd08bfff700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd08c0b17e0 0x7fd07c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:26.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd09111c700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d90 0x7fd07c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:19:26.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd09091b700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd07c003a00 0x7fd07c003e20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd080009b60 tx=0x7fd07c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee1105d786feb09 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:26.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08000e6c0 con 0x7fd07c003a00 2023-08-27T11:19:26.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08000e820 con 0x7fd07c003a00 2023-08-27T11:19:26.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.412+0000 7fd08b7fe700 1 -- 172.21.15.112:0/2853532203 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08000eb90 con 0x7fd07c003a00 2023-08-27T11:19:26.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.413+0000 7fd092b7f700 1 -- 172.21.15.112:0/2853532203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd07c003a00 msgr2=0x7fd07c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:26.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.413+0000 7fd092b7f700 1 --2- 172.21.15.112:0/2853532203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd07c003a00 0x7fd07c003e20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd080009b60 tx=0x7fd07c005880 comp rx=0 tx=0).stop 2023-08-27T11:19:26.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.413+0000 7fd09111c700 1 -- 172.21.15.112:0/2853532203 reap_dead start 2023-08-27T11:19:26.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.413+0000 7fd092b7f700 1 -- 172.21.15.112:0/2853532203 shutdown_connections 2023-08-27T11:19:26.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.413+0000 7fd092b7f700 1 -- 172.21.15.112:0/2853532203 >> 172.21.15.112:0/2853532203 conn(0x7fd08c0d0000 msgr2=0x7fd08c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:26.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.413+0000 7fd092b7f700 1 -- 172.21.15.112:0/2853532203 shutdown_connections 2023-08-27T11:19:26.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.413+0000 7fd092b7f700 1 -- 172.21.15.112:0/2853532203 wait complete. 2023-08-27T11:19:26.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.414+0000 7fd092b7f700 1 Processor -- start 2023-08-27T11:19:26.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.414+0000 7fd092b7f700 1 -- start start 2023-08-27T11:19:26.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd092b7f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c003a00 0x7fd08c154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd092b7f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd08c0ac8a0 0x7fd08c0c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd092b7f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd08c0b20d0 0x7fd08c0b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd092b7f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0b2a90 con 0x7fd08c0b20d0 2023-08-27T11:19:26.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd092b7f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0ace60 con 0x7fd08c0ac8a0 2023-08-27T11:19:26.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd092b7f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0ad140 con 0x7fd07c003a00 2023-08-27T11:19:26.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd09111c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd08c0b20d0 0x7fd08c0b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd09111c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd08c0b20d0 0x7fd08c0b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59188/0 (socket says 172.21.15.112:59188) 2023-08-27T11:19:26.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd09111c700 1 -- 172.21.15.112:0/2952579550 learned_addr learned my addr 172.21.15.112:0/2952579550 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:26.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd09091b700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c003a00 0x7fd08c154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd08bfff700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd08c0ac8a0 0x7fd08c0c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd09111c700 1 -- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c003a00 msgr2=0x7fd08c154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:26.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd09111c700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c003a00 0x7fd08c154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd09111c700 1 -- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd08c0ac8a0 msgr2=0x7fd08c0c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:26.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd09111c700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd08c0ac8a0 0x7fd08c0c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd09111c700 1 -- 172.21.15.112:0/2952579550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c0ad420 con 0x7fd08c0b20d0 2023-08-27T11:19:26.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd09111c700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd08c0b20d0 0x7fd08c0b2550 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0ca580 tx=0x7fd08400e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:26.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.415+0000 7fd0897fa700 1 -- 172.21.15.112:0/2952579550 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08400ebc0 con 0x7fd08c0b20d0 2023-08-27T11:19:26.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.416+0000 7fd0897fa700 1 -- 172.21.15.112:0/2952579550 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08400ed20 con 0x7fd08c0b20d0 2023-08-27T11:19:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.416+0000 7fd092b7f700 1 -- 172.21.15.112:0/2952579550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08c0ad700 con 0x7fd08c0b20d0 2023-08-27T11:19:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.416+0000 7fd0897fa700 1 -- 172.21.15.112:0/2952579550 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084013660 con 0x7fd08c0b20d0 2023-08-27T11:19:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.416+0000 7fd092b7f700 1 -- 172.21.15.112:0/2952579550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08c0c9c70 con 0x7fd08c0b20d0 2023-08-27T11:19:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.416+0000 7fd092b7f700 1 -- 172.21.15.112:0/2952579550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08c0ad8f0 con 0x7fd08c0b20d0 2023-08-27T11:19:26.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.417+0000 7fd0897fa700 1 -- 172.21.15.112:0/2952579550 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fd0840137c0 con 0x7fd08c0b20d0 2023-08-27T11:19:26.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.417+0000 7fd0897fa700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd07006a1a0 0x7fd07006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.417+0000 7fd0897fa700 1 -- 172.21.15.112:0/2952579550 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd0840889f0 con 0x7fd08c0b20d0 2023-08-27T11:19:26.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.418+0000 7fd09091b700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd07006a1a0 0x7fd07006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.418+0000 7fd09091b700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd07006a1a0 0x7fd07006c660 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd0800090b0 tx=0x7fd08001f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:26.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.422+0000 7fd0897fa700 1 -- 172.21.15.112:0/2952579550 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd08408ae60 con 0x7fd08c0b20d0 2023-08-27T11:19:26.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.638+0000 7fd092b7f700 1 -- 172.21.15.112:0/2952579550 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd08c1614e0 con 0x7fd07006a1a0 2023-08-27T11:19:26.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.640+0000 7fd0897fa700 1 -- 172.21.15.112:0/2952579550 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd080006c60 con 0x7fd07006a1a0 2023-08-27T11:19:26.640 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:19:26.640 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:19:26.641 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs" 2023-08-27T11:19:26.641 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:19:26.641 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:19:26.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.641+0000 7fd092b7f700 1 -- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd07006a1a0 msgr2=0x7fd07006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:26.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.641+0000 7fd092b7f700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd07006a1a0 0x7fd07006c660 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd0800090b0 tx=0x7fd08001f040 comp rx=0 tx=0).stop 2023-08-27T11:19:26.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.641+0000 7fd092b7f700 1 -- 172.21.15.112:0/2952579550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd08c0b20d0 msgr2=0x7fd08c0b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:26.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.641+0000 7fd092b7f700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd08c0b20d0 0x7fd08c0b2550 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0ca580 tx=0x7fd08400e440 comp rx=0 tx=0).stop 2023-08-27T11:19:26.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.642+0000 7fd092b7f700 1 -- 172.21.15.112:0/2952579550 shutdown_connections 2023-08-27T11:19:26.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.642+0000 7fd092b7f700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd07006a1a0 0x7fd07006c660 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.642+0000 7fd092b7f700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd08c0b20d0 0x7fd08c0b2550 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.642+0000 7fd092b7f700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c003a00 0x7fd08c154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.642+0000 7fd092b7f700 1 --2- 172.21.15.112:0/2952579550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd08c0ac8a0 0x7fd08c0c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.642+0000 7fd092b7f700 1 -- 172.21.15.112:0/2952579550 >> 172.21.15.112:0/2952579550 conn(0x7fd08c0d0000 msgr2=0x7fd08c1559c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:26.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.642+0000 7fd092b7f700 1 -- 172.21.15.112:0/2952579550 shutdown_connections 2023-08-27T11:19:26.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.642+0000 7fd092b7f700 1 -- 172.21.15.112:0/2952579550 wait complete. 2023-08-27T11:19:26.655 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:19:26.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.779+0000 7fa37c556700 1 Processor -- start 2023-08-27T11:19:26.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.779+0000 7fa37c556700 1 -- start start 2023-08-27T11:19:26.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37c556700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa374152130 0x7fa37415c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37c556700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa37415cd60 0x7fa37415ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37c556700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa37415f240 0x7fa3740b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37c556700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37415f750 con 0x7fa374152130 2023-08-27T11:19:26.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37c556700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37415fa30 con 0x7fa37415f240 2023-08-27T11:19:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37c556700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740b60a0 con 0x7fa37415cd60 2023-08-27T11:19:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37a2f2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa374152130 0x7fa37415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37a2f2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa374152130 0x7fa37415c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59192/0 (socket says 172.21.15.112:59192) 2023-08-27T11:19:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37a2f2700 1 -- 172.21.15.112:0/2846038600 learned_addr learned my addr 172.21.15.112:0/2846038600 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37a2f2700 1 -- 172.21.15.112:0/2846038600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa37415cd60 msgr2=0x7fa37415ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37a2f2700 1 --2- 172.21.15.112:0/2846038600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa37415cd60 0x7fa37415ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37aaf3700 1 --2- 172.21.15.112:0/2846038600 >> v2:172.21.15.138:3300/0 conn(0x7fa37415f240 0x7fa3740b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37a2f2700 1 -- 172.21.15.112:0/2846038600 >> v2:172.21.15.138:3300/0 conn(0x7fa37415f240 msgr2=0x7fa3740b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37a2f2700 1 --2- 172.21.15.112:0/2846038600 >> v2:172.21.15.138:3300/0 conn(0x7fa37415f240 0x7fa3740b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37a2f2700 1 -- 172.21.15.112:0/2846038600 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3740b6380 con 0x7fa374152130 2023-08-27T11:19:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa37a2f2700 1 --2- 172.21.15.112:0/2846038600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374152130 0x7fa37415c820 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa37406e5c0 tx=0x7fa370009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162123045a7fc527 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.780+0000 7fa3792f0700 1 -- 172.21.15.112:0/2846038600 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37000b870 con 0x7fa374152130 2023-08-27T11:19:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.781+0000 7fa3792f0700 1 -- 172.21.15.112:0/2846038600 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37000b9d0 con 0x7fa374152130 2023-08-27T11:19:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.781+0000 7fa3792f0700 1 -- 172.21.15.112:0/2846038600 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37000bd60 con 0x7fa374152130 2023-08-27T11:19:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.781+0000 7fa37c556700 1 -- 172.21.15.112:0/2846038600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374152130 msgr2=0x7fa37415c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.781+0000 7fa37c556700 1 --2- 172.21.15.112:0/2846038600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374152130 0x7fa37415c820 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa37406e5c0 tx=0x7fa370009bd0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.781+0000 7fa37c556700 1 -- 172.21.15.112:0/2846038600 shutdown_connections 2023-08-27T11:19:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.781+0000 7fa37c556700 1 --2- 172.21.15.112:0/2846038600 >> v2:172.21.15.138:3300/0 conn(0x7fa37415f240 0x7fa3740b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.781+0000 7fa37c556700 1 --2- 172.21.15.112:0/2846038600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374152130 0x7fa37415c820 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.781+0000 7fa37c556700 1 --2- 172.21.15.112:0/2846038600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa37415cd60 0x7fa37415ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.781+0000 7fa37c556700 1 -- 172.21.15.112:0/2846038600 >> 172.21.15.112:0/2846038600 conn(0x7fa374066010 msgr2=0x7fa3740663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.781+0000 7fa37c556700 1 -- 172.21.15.112:0/2846038600 shutdown_connections 2023-08-27T11:19:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.781+0000 7fa37c556700 1 -- 172.21.15.112:0/2846038600 wait complete. 2023-08-27T11:19:26.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.782+0000 7fa37c556700 1 Processor -- start 2023-08-27T11:19:26.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.782+0000 7fa37c556700 1 -- start start 2023-08-27T11:19:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.783+0000 7fa37c556700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374152130 0x7fa37413ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.783+0000 7fa37c556700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa37415cd60 0x7fa3740b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.783+0000 7fa37c556700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa37415f240 0x7fa3740b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.783+0000 7fa37c556700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa374149460 con 0x7fa374152130 2023-08-27T11:19:26.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.783+0000 7fa37c556700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa374150530 con 0x7fa37415cd60 2023-08-27T11:19:26.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.783+0000 7fa37c556700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa374150810 con 0x7fa37415f240 2023-08-27T11:19:26.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.783+0000 7fa37a2f2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374152130 0x7fa37413ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.783+0000 7fa37a2f2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374152130 0x7fa37413ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59198/0 (socket says 172.21.15.112:59198) 2023-08-27T11:19:26.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.783+0000 7fa37a2f2700 1 -- 172.21.15.112:0/2144093768 learned_addr learned my addr 172.21.15.112:0/2144093768 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:26.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.783+0000 7fa37a2f2700 1 -- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa37415f240 msgr2=0x7fa3740b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:26.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.783+0000 7fa37a2f2700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa37415f240 0x7fa3740b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.783+0000 7fa379af1700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa37415cd60 0x7fa3740b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.783+0000 7fa37a2f2700 1 -- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa37415cd60 msgr2=0x7fa3740b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:26.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.784+0000 7fa37a2f2700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa37415cd60 0x7fa3740b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.784+0000 7fa37a2f2700 1 -- 172.21.15.112:0/2144093768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa370009550 con 0x7fa374152130 2023-08-27T11:19:26.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.784+0000 7fa379af1700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa37415cd60 0x7fa3740b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:19:26.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.784+0000 7fa37a2f2700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374152130 0x7fa37413ea20 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa37406d1c0 tx=0x7fa370004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:26.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.784+0000 7fa36affd700 1 -- 172.21.15.112:0/2144093768 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370004310 con 0x7fa374152130 2023-08-27T11:19:26.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.784+0000 7fa36affd700 1 -- 172.21.15.112:0/2144093768 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37001f070 con 0x7fa374152130 2023-08-27T11:19:26.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.784+0000 7fa36affd700 1 -- 172.21.15.112:0/2144093768 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3700036a0 con 0x7fa374152130 2023-08-27T11:19:26.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.784+0000 7fa37c556700 1 -- 172.21.15.112:0/2144093768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa374150af0 con 0x7fa374152130 2023-08-27T11:19:26.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.784+0000 7fa37c556700 1 -- 172.21.15.112:0/2144093768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37413ef60 con 0x7fa374152130 2023-08-27T11:19:26.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.785+0000 7fa37c556700 1 -- 172.21.15.112:0/2144093768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa374066f20 con 0x7fa374152130 2023-08-27T11:19:26.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.785+0000 7fa36affd700 1 -- 172.21.15.112:0/2144093768 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7fa370021030 con 0x7fa374152130 2023-08-27T11:19:26.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.785+0000 7fa36affd700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa35806a170 0x7fa35806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:26.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.786+0000 7fa36affd700 1 -- 172.21.15.112:0/2144093768 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3700899d0 con 0x7fa374152130 2023-08-27T11:19:26.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.786+0000 7fa379af1700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa35806a170 0x7fa35806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:26.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.786+0000 7fa379af1700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa35806a170 0x7fa35806c630 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa3740d1ae0 tx=0x7fa364008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:26.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.790+0000 7fa36affd700 1 -- 172.21.15.112:0/2144093768 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa370055280 con 0x7fa374152130 2023-08-27T11:19:26.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.980+0000 7fa37c556700 1 -- 172.21.15.112:0/2144093768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa374067a20 con 0x7fa374152130 2023-08-27T11:19:26.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.980+0000 7fa36affd700 1 -- 172.21.15.112:0/2144093768 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa3700588a0 con 0x7fa374152130 2023-08-27T11:19:26.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.982+0000 7fa37c556700 1 -- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa35806a170 msgr2=0x7fa35806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.982+0000 7fa37c556700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa35806a170 0x7fa35806c630 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa3740d1ae0 tx=0x7fa364008040 comp rx=0 tx=0).stop 2023-08-27T11:19:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.982+0000 7fa37c556700 1 -- 172.21.15.112:0/2144093768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374152130 msgr2=0x7fa37413ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.982+0000 7fa37c556700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374152130 0x7fa37413ea20 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa37406d1c0 tx=0x7fa370004750 comp rx=0 tx=0).stop 2023-08-27T11:19:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.982+0000 7fa37c556700 1 -- 172.21.15.112:0/2144093768 shutdown_connections 2023-08-27T11:19:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.982+0000 7fa37c556700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa35806a170 0x7fa35806c630 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.982+0000 7fa37c556700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa37415f240 0x7fa3740b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.982+0000 7fa37c556700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374152130 0x7fa37413ea20 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.982+0000 7fa37c556700 1 --2- 172.21.15.112:0/2144093768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa37415cd60 0x7fa3740b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.982+0000 7fa37c556700 1 -- 172.21.15.112:0/2144093768 >> 172.21.15.112:0/2144093768 conn(0x7fa374066010 msgr2=0x7fa37415d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.982+0000 7fa37c556700 1 -- 172.21.15.112:0/2144093768 shutdown_connections 2023-08-27T11:19:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:26.982+0000 7fa37c556700 1 -- 172.21.15.112:0/2144093768 wait complete. 2023-08-27T11:19:26.994 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000166348 --mode=777 2023-08-27T11:19:27.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.117+0000 7f23e90e3700 1 Processor -- start 2023-08-27T11:19:27.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.117+0000 7f23e90e3700 1 -- start start 2023-08-27T11:19:27.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.118+0000 7f23e90e3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f23e4144bd0 0x7f23e415b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:27.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.118+0000 7f23e90e3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e406bfd0 0x7f23e406c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:27.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.118+0000 7f23e90e3700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e40b6560 con 0x7f23e40b57a0 2023-08-27T11:19:27.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.118+0000 7f23e90e3700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e406c990 con 0x7f23e4144bd0 2023-08-27T11:19:27.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.118+0000 7f23e90e3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e40b6340 con 0x7f23e406bfd0 2023-08-27T11:19:27.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.119+0000 7f23e2d9d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f23e4144bd0 0x7f23e415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:27.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.119+0000 7f23e259c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e406bfd0 0x7f23e406c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:27.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.119+0000 7f23e2d9d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f23e4144bd0 0x7f23e415b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:59632/0 (socket says 172.21.15.112:59632) 2023-08-27T11:19:27.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.119+0000 7f23e2d9d700 1 -- 172.21.15.112:0/1739018682 learned_addr learned my addr 172.21.15.112:0/1739018682 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:27.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.119+0000 7f23e359e700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f23e40b57a0 0x7f23e40b5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:32884/0 (socket says 172.21.15.112:32884) 2023-08-27T11:19:27.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.119+0000 7f23e2d9d700 1 -- 172.21.15.112:0/1739018682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e406bfd0 msgr2=0x7f23e406c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:27.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.119+0000 7f23e2d9d700 1 --2- 172.21.15.112:0/1739018682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e406bfd0 0x7f23e406c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.119+0000 7f23e2d9d700 1 -- 172.21.15.112:0/1739018682 >> v1:172.21.15.138:6789/0 conn(0x7f23e40b57a0 legacy=0x7f23e40b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:27.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.119+0000 7f23e1d9b700 1 -- 172.21.15.112:0/1739018682 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737993874 0 0) 0x7f23e40b6560 con 0x7f23e40b57a0 2023-08-27T11:19:27.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.119+0000 7f23e2d9d700 1 -- 172.21.15.112:0/1739018682 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e40b6840 con 0x7f23e4144bd0 2023-08-27T11:19:27.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.119+0000 7f23e259c700 1 --2- 172.21.15.112:0/1739018682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e406bfd0 0x7f23e406c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:19:27.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.119+0000 7f23e2d9d700 1 --2- 172.21.15.112:0/1739018682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e4144bd0 0x7f23e415b910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f23d80098f0 tx=0x7f23d8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=811fd57e9d75c20f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:27.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.119+0000 7f23e1d9b700 1 -- 172.21.15.112:0/1739018682 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8011830 con 0x7f23e4144bd0 2023-08-27T11:19:27.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.120+0000 7f23e1d9b700 1 -- 172.21.15.112:0/1739018682 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d8011990 con 0x7f23e4144bd0 2023-08-27T11:19:27.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.120+0000 7f23e1d9b700 1 -- 172.21.15.112:0/1739018682 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8011d20 con 0x7f23e4144bd0 2023-08-27T11:19:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.120+0000 7f23e90e3700 1 -- 172.21.15.112:0/1739018682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e4144bd0 msgr2=0x7f23e415b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.120+0000 7f23e90e3700 1 --2- 172.21.15.112:0/1739018682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e4144bd0 0x7f23e415b910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f23d80098f0 tx=0x7f23d8009c00 comp rx=0 tx=0).stop 2023-08-27T11:19:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.120+0000 7f23e90e3700 1 -- 172.21.15.112:0/1739018682 shutdown_connections 2023-08-27T11:19:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.120+0000 7f23e90e3700 1 --2- 172.21.15.112:0/1739018682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e4144bd0 0x7f23e415b910 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.120+0000 7f23e90e3700 1 --2- 172.21.15.112:0/1739018682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e406bfd0 0x7f23e406c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.120+0000 7f23e90e3700 1 -- 172.21.15.112:0/1739018682 >> 172.21.15.112:0/1739018682 conn(0x7f23e4066010 msgr2=0x7f23e4066420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.120+0000 7f23e90e3700 1 -- 172.21.15.112:0/1739018682 shutdown_connections 2023-08-27T11:19:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.120+0000 7f23e90e3700 1 -- 172.21.15.112:0/1739018682 wait complete. 2023-08-27T11:19:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.122+0000 7f23e90e3700 1 Processor -- start 2023-08-27T11:19:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.122+0000 7f23e90e3700 1 -- start start 2023-08-27T11:19:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.123+0000 7f23e90e3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e406bfd0 0x7f23e4155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.123+0000 7f23e90e3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e40b57a0 0x7f23e41f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.123+0000 7f23e90e3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e4144bd0 0x7f23e41f0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.123+0000 7f23e90e3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e4141d40 con 0x7f23e4144bd0 2023-08-27T11:19:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.123+0000 7f23e90e3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41410c0 con 0x7f23e406bfd0 2023-08-27T11:19:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.123+0000 7f23e90e3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41413a0 con 0x7f23e40b57a0 2023-08-27T11:19:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.123+0000 7f23e359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e4144bd0 0x7f23e41f0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.123+0000 7f23e359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e4144bd0 0x7f23e41f0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59214/0 (socket says 172.21.15.112:59214) 2023-08-27T11:19:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.123+0000 7f23e359e700 1 -- 172.21.15.112:0/2634006330 learned_addr learned my addr 172.21.15.112:0/2634006330 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:27.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.123+0000 7f23e2d9d700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e406bfd0 0x7f23e4155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:27.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.123+0000 7f23e259c700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e40b57a0 0x7f23e41f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:27.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.124+0000 7f23e359e700 1 -- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e40b57a0 msgr2=0x7f23e41f02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:27.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.124+0000 7f23e359e700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e40b57a0 0x7f23e41f02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.124+0000 7f23e359e700 1 -- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e406bfd0 msgr2=0x7f23e4155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:27.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.124+0000 7f23e359e700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e406bfd0 0x7f23e4155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.124+0000 7f23e359e700 1 -- 172.21.15.112:0/2634006330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d40094a0 con 0x7f23e4144bd0 2023-08-27T11:19:27.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.124+0000 7f23e359e700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e4144bd0 0x7f23e41f0830 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f23dc002230 tx=0x7f23dc00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:27.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.124+0000 7f23cb7fe700 1 -- 172.21.15.112:0/2634006330 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc00aac0 con 0x7f23e4144bd0 2023-08-27T11:19:27.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.124+0000 7f23cb7fe700 1 -- 172.21.15.112:0/2634006330 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23dc00ac40 con 0x7f23e4144bd0 2023-08-27T11:19:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.124+0000 7f23e90e3700 1 -- 172.21.15.112:0/2634006330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d8009550 con 0x7f23e4144bd0 2023-08-27T11:19:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.124+0000 7f23cb7fe700 1 -- 172.21.15.112:0/2634006330 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc0070d0 con 0x7f23e4144bd0 2023-08-27T11:19:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.124+0000 7f23e90e3700 1 -- 172.21.15.112:0/2634006330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e4141960 con 0x7f23e4144bd0 2023-08-27T11:19:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.125+0000 7f23e90e3700 1 -- 172.21.15.112:0/2634006330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e4141e80 con 0x7f23e4144bd0 2023-08-27T11:19:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.126+0000 7f23cb7fe700 1 -- 172.21.15.112:0/2634006330 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f23dc007250 con 0x7f23e4144bd0 2023-08-27T11:19:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.126+0000 7f23cb7fe700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f23d006a170 0x7f23d006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:27.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.126+0000 7f23cb7fe700 1 -- 172.21.15.112:0/2634006330 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f23dc088370 con 0x7f23e4144bd0 2023-08-27T11:19:27.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.126+0000 7f23e2d9d700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f23d006a170 0x7f23d006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:27.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.127+0000 7f23e2d9d700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f23d006a170 0x7f23d006c630 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f23d8014040 tx=0x7f23d8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:27.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.130+0000 7f23cb7fe700 1 -- 172.21.15.112:0/2634006330 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f23e4141e80 con 0x7f23e4144bd0 2023-08-27T11:19:27.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.348+0000 7f23e90e3700 1 -- 172.21.15.112:0/2634006330 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000166348", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f23e40cf050 con 0x7f23d006a170 2023-08-27T11:19:27.355 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:19:27.353+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:19:27.356 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:19:27.353+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:19:27.356 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:19:27.353+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:19:27.356 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:19:27.353+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:19:27.356 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:19:27.353+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:19:27.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.391+0000 7f23cb7fe700 1 -- 172.21.15.112:0/2634006330 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f23d8011b80 con 0x7f23d006a170 2023-08-27T11:19:27.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.393+0000 7f23e90e3700 1 -- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f23d006a170 msgr2=0x7f23d006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:27.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.393+0000 7f23e90e3700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f23d006a170 0x7f23d006c630 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f23d8014040 tx=0x7f23d8008030 comp rx=0 tx=0).stop 2023-08-27T11:19:27.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.393+0000 7f23e90e3700 1 -- 172.21.15.112:0/2634006330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e4144bd0 msgr2=0x7f23e41f0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:27.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.393+0000 7f23e90e3700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e4144bd0 0x7f23e41f0830 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f23dc002230 tx=0x7f23dc00cf50 comp rx=0 tx=0).stop 2023-08-27T11:19:27.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.393+0000 7f23e90e3700 1 -- 172.21.15.112:0/2634006330 shutdown_connections 2023-08-27T11:19:27.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.393+0000 7f23e90e3700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f23d006a170 0x7f23d006c630 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.393+0000 7f23e90e3700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e4144bd0 0x7f23e41f0830 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.393+0000 7f23e90e3700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e406bfd0 0x7f23e4155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.393+0000 7f23e90e3700 1 --2- 172.21.15.112:0/2634006330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e40b57a0 0x7f23e41f02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.393+0000 7f23e90e3700 1 -- 172.21.15.112:0/2634006330 >> 172.21.15.112:0/2634006330 conn(0x7f23e4066010 msgr2=0x7f23e4159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:27.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.393+0000 7f23e90e3700 1 -- 172.21.15.112:0/2634006330 shutdown_connections 2023-08-27T11:19:27.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.393+0000 7f23e90e3700 1 -- 172.21.15.112:0/2634006330 wait complete. 2023-08-27T11:19:27.406 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000166348 2023-08-27T11:19:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.529+0000 7f512ff86700 1 Processor -- start 2023-08-27T11:19:27.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.529+0000 7f512ff86700 1 -- start start 2023-08-27T11:19:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.530+0000 7f512ff86700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f512806fea0 0x7f51280ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:27.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.530+0000 7f512ff86700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51280ccad0 0x7f512815b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:27.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.530+0000 7f512ff86700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51280b08e0 con 0x7f51281445b0 2023-08-27T11:19:27.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.530+0000 7f512ff86700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512815bc70 con 0x7f512806fea0 2023-08-27T11:19:27.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.530+0000 7f512ff86700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51280ccee0 con 0x7f51280ccad0 2023-08-27T11:19:27.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512e523700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51280ccad0 0x7f512815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:27.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512d521700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f512806fea0 0x7f51280ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:27.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512dd22700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f51281445b0 0x7f512806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:32894/0 (socket says 172.21.15.112:32894) 2023-08-27T11:19:27.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512d521700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f512806fea0 0x7f51280ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:59648/0 (socket says 172.21.15.112:59648) 2023-08-27T11:19:27.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512dd22700 1 -- 172.21.15.112:0/925571461 learned_addr learned my addr 172.21.15.112:0/925571461 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:19:27.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512e523700 1 -- 172.21.15.112:0/925571461 >> v2:172.21.15.138:3300/0 conn(0x7f512806fea0 msgr2=0x7f51280ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:19:27.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512e523700 1 --2- 172.21.15.112:0/925571461 >> v2:172.21.15.138:3300/0 conn(0x7f512806fea0 0x7f51280ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512e523700 1 -- 172.21.15.112:0/925571461 >> v1:172.21.15.138:6789/0 conn(0x7f51281445b0 legacy=0x7f512806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:27.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512e523700 1 -- 172.21.15.112:0/925571461 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51280b0b30 con 0x7f51280ccad0 2023-08-27T11:19:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512d521700 1 --2- 172.21.15.112:0/925571461 >> v2:172.21.15.138:3300/0 conn(0x7f512806fea0 0x7f51280ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:19:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512e523700 1 --2- 172.21.15.112:0/925571461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51280ccad0 0x7f512815b730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5120009d90 tx=0x7f512000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcc812a2f3619af3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512cd20700 1 -- 172.21.15.112:0/925571461 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512000e070 con 0x7f51280ccad0 2023-08-27T11:19:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512cd20700 1 -- 172.21.15.112:0/925571461 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512000bb20 con 0x7f51280ccad0 2023-08-27T11:19:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.531+0000 7f512cd20700 1 -- 172.21.15.112:0/925571461 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120012430 con 0x7f51280ccad0 2023-08-27T11:19:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.532+0000 7f512ff86700 1 -- 172.21.15.112:0/925571461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51280ccad0 msgr2=0x7f512815b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.532+0000 7f512ff86700 1 --2- 172.21.15.112:0/925571461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51280ccad0 0x7f512815b730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5120009d90 tx=0x7f512000b330 comp rx=0 tx=0).stop 2023-08-27T11:19:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.532+0000 7f512ff86700 1 -- 172.21.15.112:0/925571461 shutdown_connections 2023-08-27T11:19:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.532+0000 7f512ff86700 1 --2- 172.21.15.112:0/925571461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51280ccad0 0x7f512815b730 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.532+0000 7f512ff86700 1 --2- 172.21.15.112:0/925571461 >> v2:172.21.15.138:3300/0 conn(0x7f512806fea0 0x7f51280ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.532+0000 7f512ff86700 1 -- 172.21.15.112:0/925571461 >> 172.21.15.112:0/925571461 conn(0x7f51280b1e90 msgr2=0x7f51280b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:27.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.532+0000 7f512ff86700 1 -- 172.21.15.112:0/925571461 shutdown_connections 2023-08-27T11:19:27.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.532+0000 7f512ff86700 1 -- 172.21.15.112:0/925571461 wait complete. 2023-08-27T11:19:27.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.533+0000 7f512ff86700 1 Processor -- start 2023-08-27T11:19:27.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.533+0000 7f512ff86700 1 -- start start 2023-08-27T11:19:27.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512ff86700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f512806fea0 0x7f51280ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:27.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512ff86700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51280ccad0 0x7f51281426c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:27.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512ff86700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51281445b0 0x7f51281f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:27.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512ff86700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51280b4e20 con 0x7f512806fea0 2023-08-27T11:19:27.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512ff86700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5128141620 con 0x7f51281445b0 2023-08-27T11:19:27.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512ff86700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5128141900 con 0x7f51280ccad0 2023-08-27T11:19:27.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512dd22700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f512806fea0 0x7f51280ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:27.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512dd22700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f512806fea0 0x7f51280ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59226/0 (socket says 172.21.15.112:59226) 2023-08-27T11:19:27.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512dd22700 1 -- 172.21.15.112:0/960739967 learned_addr learned my addr 172.21.15.112:0/960739967 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:19:27.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512d521700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51280ccad0 0x7f51281426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:27.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512e523700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51281445b0 0x7f51281f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:27.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512dd22700 1 -- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51280ccad0 msgr2=0x7f51281426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:27.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512dd22700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51280ccad0 0x7f51281426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512dd22700 1 -- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51281445b0 msgr2=0x7f51281f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:27.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512dd22700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51281445b0 0x7f51281f60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512dd22700 1 -- 172.21.15.112:0/960739967 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240094a0 con 0x7f512806fea0 2023-08-27T11:19:27.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.534+0000 7f512dd22700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f512806fea0 0x7f51280ad2a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f511800c520 tx=0x7f511800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:27.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.535+0000 7f511e7fc700 1 -- 172.21.15.112:0/960739967 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118005d70 con 0x7f512806fea0 2023-08-27T11:19:27.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.535+0000 7f511e7fc700 1 -- 172.21.15.112:0/960739967 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51180070d0 con 0x7f512806fea0 2023-08-27T11:19:27.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.535+0000 7f511e7fc700 1 -- 172.21.15.112:0/960739967 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118007440 con 0x7f512806fea0 2023-08-27T11:19:27.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.535+0000 7f512ff86700 1 -- 172.21.15.112:0/960739967 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5120009a70 con 0x7f512806fea0 2023-08-27T11:19:27.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.535+0000 7f512ff86700 1 -- 172.21.15.112:0/960739967 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5128141c90 con 0x7f512806fea0 2023-08-27T11:19:27.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.536+0000 7f512ff86700 1 -- 172.21.15.112:0/960739967 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5128067e90 con 0x7f512806fea0 2023-08-27T11:19:27.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.536+0000 7f511e7fc700 1 -- 172.21.15.112:0/960739967 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 6) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f51180075a0 con 0x7f512806fea0 2023-08-27T11:19:27.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.536+0000 7f511e7fc700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f510c06a170 0x7f510c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:19:27.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.537+0000 7f511e7fc700 1 -- 172.21.15.112:0/960739967 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f51180877a0 con 0x7f512806fea0 2023-08-27T11:19:27.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.537+0000 7f512d521700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f510c06a170 0x7f510c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:19:27.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.537+0000 7f512d521700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f510c06a170 0x7f510c06c630 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f51280af4c0 tx=0x7f5124009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:19:27.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.541+0000 7f511e7fc700 1 -- 172.21.15.112:0/960739967 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5118052fe0 con 0x7f512806fea0 2023-08-27T11:19:27.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.761+0000 7f512ff86700 1 -- 172.21.15.112:0/960739967 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000166348", "target": ["mon-mgr", ""]}) v1 -- 0x7f51280b0e10 con 0x7f510c06a170 2023-08-27T11:19:27.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.768+0000 7f511e7fc700 1 -- 172.21.15.112:0/960739967 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f5124011b10 con 0x7f510c06a170 2023-08-27T11:19:27.769 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243 2023-08-27T11:19:27.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.770+0000 7f512ff86700 1 -- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f510c06a170 msgr2=0x7f510c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.770+0000 7f512ff86700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f510c06a170 0x7f510c06c630 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f51280af4c0 tx=0x7f5124009210 comp rx=0 tx=0).stop 2023-08-27T11:19:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.770+0000 7f512ff86700 1 -- 172.21.15.112:0/960739967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f512806fea0 msgr2=0x7f51280ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:19:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.770+0000 7f512ff86700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f512806fea0 0x7f51280ad2a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f511800c520 tx=0x7f511800c830 comp rx=0 tx=0).stop 2023-08-27T11:19:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.770+0000 7f512ff86700 1 -- 172.21.15.112:0/960739967 shutdown_connections 2023-08-27T11:19:27.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.770+0000 7f512ff86700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f510c06a170 0x7f510c06c630 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.770+0000 7f512ff86700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51281445b0 0x7f51281f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.770+0000 7f512ff86700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f512806fea0 0x7f51280ad2a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.770+0000 7f512ff86700 1 --2- 172.21.15.112:0/960739967 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51280ccad0 0x7f51281426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:19:27.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.770+0000 7f512ff86700 1 -- 172.21.15.112:0/960739967 >> 172.21.15.112:0/960739967 conn(0x7f51280b1e90 msgr2=0x7f5128071f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:19:27.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.770+0000 7f512ff86700 1 -- 172.21.15.112:0/960739967 shutdown_connections 2023-08-27T11:19:27.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:19:27.770+0000 7f512ff86700 1 -- 172.21.15.112:0/960739967 wait complete. 2023-08-27T11:19:27.782 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000166348 with 3000 files each 0.1MB size under directory volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243 2023-08-27T11:19:27.783 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:27.783 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:27.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:27.801 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.0 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:27.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:27.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:27.864 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4286e-05 s, 0.0 kB/s 2023-08-27T11:19:27.865 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:27.865 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:27.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:27.922 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:27.978 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:27.978 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:27.979 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8646e-05 s, 0.0 kB/s 2023-08-27T11:19:27.979 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:27.979 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:28.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:28.035 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:28.092 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:28.093 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:28.093 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6615e-05 s, 0.0 kB/s 2023-08-27T11:19:28.094 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:28.094 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:28.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:28.151 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.3 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:28.207 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:28.208 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:28.208 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9109e-05 s, 0.0 kB/s 2023-08-27T11:19:28.208 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:28.209 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:28.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:28.266 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.4 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:28.323 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:28.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:28.324 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8137e-05 s, 0.0 kB/s 2023-08-27T11:19:28.324 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:28.325 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:28.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:28.382 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.5 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:28.439 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:28.440 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:28.440 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.784e-05 s, 0.0 kB/s 2023-08-27T11:19:28.440 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:28.441 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:28.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:28.498 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.6 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:28.555 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8854e-05 s, 0.0 kB/s 2023-08-27T11:19:28.557 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:28.557 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:28.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:28.614 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.7 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:28.672 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:28.672 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:28.673 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.924e-05 s, 0.0 kB/s 2023-08-27T11:19:28.673 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:28.674 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:28.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:28.730 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.8 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:28.788 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:28.789 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:28.789 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6999e-05 s, 0.0 kB/s 2023-08-27T11:19:28.789 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:28.790 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:28.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:28.846 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.9 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6333e-05 s, 0.0 kB/s 2023-08-27T11:19:28.903 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:28.903 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:28.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:28.960 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.10 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:29.016 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:29.016 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:29.017 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1139e-05 s, 0.0 kB/s 2023-08-27T11:19:29.017 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:29.018 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:29.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:29.074 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.11 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:29.132 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:29.132 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:29.132 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8532e-05 s, 0.0 kB/s 2023-08-27T11:19:29.133 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:29.133 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:29.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:29.190 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.12 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:29.247 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:29.247 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:29.247 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.651e-05 s, 0.0 kB/s 2023-08-27T11:19:29.248 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:29.248 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:29.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:29.305 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.13 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:29.363 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:29.363 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:29.363 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9885e-05 s, 0.0 kB/s 2023-08-27T11:19:29.364 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:29.364 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:29.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:29.421 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.14 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:29.479 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9536e-05 s, 0.0 kB/s 2023-08-27T11:19:29.479 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:29.479 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:29.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:29.536 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.15 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:29.593 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:29.593 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6945e-05 s, 0.0 kB/s 2023-08-27T11:19:29.594 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:29.594 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:29.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:29.652 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.16 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:29.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:29.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:29.708 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7276e-05 s, 0.0 kB/s 2023-08-27T11:19:29.709 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:29.709 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:29.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:29.767 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.17 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:29.826 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:29.826 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:29.826 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7733e-05 s, 0.0 kB/s 2023-08-27T11:19:29.827 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:29.827 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:29.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:29.884 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.18 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:29.942 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0899e-05 s, 0.0 kB/s 2023-08-27T11:19:29.942 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:29.942 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:29.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:30.000 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.19 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:30.055 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:30.055 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:30.055 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8947e-05 s, 0.0 kB/s 2023-08-27T11:19:30.056 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:30.057 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:30.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:30.113 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.20 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:30.171 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:30.172 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:30.172 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8928e-05 s, 0.0 kB/s 2023-08-27T11:19:30.173 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:30.173 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:30.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:30.228 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.21 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:30.285 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:30.286 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:30.286 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3828e-05 s, 0.0 kB/s 2023-08-27T11:19:30.286 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:30.287 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:30.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:30.344 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.22 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:30.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:30.402 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:30.402 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8657e-05 s, 0.0 kB/s 2023-08-27T11:19:30.402 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:30.403 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:30.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:30.462 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.23 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:30.518 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:30.519 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:30.519 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.904e-05 s, 0.0 kB/s 2023-08-27T11:19:30.519 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:30.520 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:30.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:30.578 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.24 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:30.635 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:30.635 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:30.635 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8595e-05 s, 0.0 kB/s 2023-08-27T11:19:30.636 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:30.636 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:30.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:30.696 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.25 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:30.754 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:30.755 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:30.755 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.0061e-05 s, 0.0 kB/s 2023-08-27T11:19:30.755 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:30.757 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:30.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:30.818 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.26 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:30.878 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:30.879 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:30.879 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7469e-05 s, 0.0 kB/s 2023-08-27T11:19:30.879 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:30.880 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:30.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:30.936 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.27 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:30.993 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:30.994 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:30.994 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7155e-05 s, 0.0 kB/s 2023-08-27T11:19:30.994 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:30.995 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:31.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:31.052 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.28 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:31.108 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:31.108 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:31.109 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9444e-05 s, 0.0 kB/s 2023-08-27T11:19:31.110 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:31.110 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:31.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:31.166 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.29 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:31.222 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:31.222 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:31.223 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.685e-05 s, 0.0 kB/s 2023-08-27T11:19:31.223 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:31.223 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:31.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:31.280 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.30 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:31.338 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:31.338 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:31.338 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0891e-05 s, 0.0 kB/s 2023-08-27T11:19:31.338 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:31.339 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:31.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:31.399 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.31 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8144e-05 s, 0.0 kB/s 2023-08-27T11:19:31.457 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:31.457 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:31.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:31.514 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.32 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:31.572 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:31.572 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:31.572 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7252e-05 s, 0.0 kB/s 2023-08-27T11:19:31.573 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:31.573 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:31.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:31.630 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.33 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6112e-05 s, 0.0 kB/s 2023-08-27T11:19:31.687 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:31.688 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:31.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:31.745 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.34 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:31.802 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:31.803 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:31.803 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1207e-05 s, 0.0 kB/s 2023-08-27T11:19:31.803 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:31.804 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:31.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:31.861 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.35 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:31.918 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:31.918 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:31.918 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9029e-05 s, 0.0 kB/s 2023-08-27T11:19:31.919 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:31.919 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:31.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:31.976 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.36 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:32.032 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:32.033 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:32.033 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0927e-05 s, 0.0 kB/s 2023-08-27T11:19:32.033 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:32.034 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:32.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:32.090 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.37 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7883e-05 s, 0.0 kB/s 2023-08-27T11:19:32.148 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:32.149 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:32.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:32.206 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.38 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:32.264 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:32.264 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:32.264 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9185e-05 s, 0.0 kB/s 2023-08-27T11:19:32.265 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:32.265 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:32.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:32.325 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.39 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:32.380 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:32.381 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:32.381 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8127e-05 s, 0.0 kB/s 2023-08-27T11:19:32.381 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:32.382 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:32.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:32.439 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.40 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:32.495 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:32.496 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:32.496 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1991e-05 s, 0.0 kB/s 2023-08-27T11:19:32.496 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:32.497 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:32.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:32.554 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.41 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:32.611 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:32.611 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:32.612 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6199e-05 s, 0.0 kB/s 2023-08-27T11:19:32.612 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:32.612 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:32.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:32.671 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.42 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:32.739 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:32.739 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:32.739 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.09e-05 s, 0.0 kB/s 2023-08-27T11:19:32.740 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:32.740 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:32.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:32.801 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.43 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:32.859 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:32.860 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:32.860 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1011e-05 s, 0.0 kB/s 2023-08-27T11:19:32.860 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:32.861 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:32.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:32.917 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.44 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3389e-05 s, 0.0 kB/s 2023-08-27T11:19:32.976 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:32.978 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:33.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:33.033 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.45 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:33.091 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:33.091 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0053e-05 s, 0.0 kB/s 2023-08-27T11:19:33.092 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:33.093 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:33.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:33.157 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.46 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:33.216 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:33.217 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:33.218 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9872e-05 s, 0.0 kB/s 2023-08-27T11:19:33.218 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:33.219 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:33.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:33.278 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.47 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:33.338 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:33.339 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:33.339 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1911e-05 s, 0.0 kB/s 2023-08-27T11:19:33.339 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:33.340 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:33.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:33.402 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.48 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:33.462 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:33.463 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:33.463 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9468e-05 s, 0.0 kB/s 2023-08-27T11:19:33.463 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:33.464 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:33.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:33.524 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.49 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:33.585 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:33.585 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1658e-05 s, 0.0 kB/s 2023-08-27T11:19:33.586 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:33.586 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:33.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:33.650 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.50 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:33.710 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:33.710 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:33.710 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1037e-05 s, 0.0 kB/s 2023-08-27T11:19:33.711 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:33.711 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:33.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:33.773 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.51 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:33.836 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:33.837 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:33.837 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9909e-05 s, 0.0 kB/s 2023-08-27T11:19:33.837 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:33.838 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:33.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:33.899 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.52 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:33.961 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:33.961 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:33.962 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6331e-05 s, 0.0 kB/s 2023-08-27T11:19:33.962 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:33.963 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:34.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:34.023 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.53 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:34.085 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:34.086 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:34.086 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7679e-05 s, 0.0 kB/s 2023-08-27T11:19:34.086 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:34.087 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:34.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:34.145 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.54 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:34.203 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:34.203 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:34.204 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6825e-05 s, 0.0 kB/s 2023-08-27T11:19:34.204 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:34.204 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:34.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:34.262 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.55 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0788e-05 s, 0.0 kB/s 2023-08-27T11:19:34.322 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:34.322 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:34.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:34.382 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.56 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:34.438 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6691e-05 s, 0.0 kB/s 2023-08-27T11:19:34.439 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:34.440 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:34.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:34.497 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.57 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:34.554 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9571e-05 s, 0.0 kB/s 2023-08-27T11:19:34.554 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:34.555 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:34.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:34.611 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.58 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:34.669 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:34.670 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:34.670 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2244e-05 s, 0.0 kB/s 2023-08-27T11:19:34.671 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:34.671 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:34.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:34.728 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.59 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:34.784 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:34.785 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:34.785 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9333e-05 s, 0.0 kB/s 2023-08-27T11:19:34.785 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:34.786 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:34.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:34.843 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.60 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:34.901 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:34.901 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6661e-05 s, 0.0 kB/s 2023-08-27T11:19:34.901 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:34.902 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:34.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:34.959 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.61 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:35.016 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:35.016 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:35.017 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.5905e-05 s, 0.0 kB/s 2023-08-27T11:19:35.017 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:35.018 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:35.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:35.074 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.62 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:35.132 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:35.132 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:35.132 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7823e-05 s, 0.0 kB/s 2023-08-27T11:19:35.132 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:35.133 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:35.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:35.191 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.63 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:35.249 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:35.249 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6448e-05 s, 0.0 kB/s 2023-08-27T11:19:35.250 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:35.250 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:35.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:35.307 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.64 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:35.365 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:35.365 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:35.366 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7709e-05 s, 0.0 kB/s 2023-08-27T11:19:35.366 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:35.366 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:35.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:35.427 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.65 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:35.484 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:35.484 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:35.485 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0748e-05 s, 0.0 kB/s 2023-08-27T11:19:35.485 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:35.487 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:35.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:35.542 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.66 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:35.598 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:35.598 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:35.598 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8596e-05 s, 0.0 kB/s 2023-08-27T11:19:35.599 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:35.599 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:35.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:35.656 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.67 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:35.712 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:35.713 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:35.713 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7434e-05 s, 0.0 kB/s 2023-08-27T11:19:35.713 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:35.714 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:35.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:35.770 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.68 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:35.813 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:19:35.818 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:19:35.833 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:35.833 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:35.833 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9847e-05 s, 0.0 kB/s 2023-08-27T11:19:35.834 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:35.834 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:35.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:35.929 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.69 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:35.986 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:35.986 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:35.987 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8752e-05 s, 0.0 kB/s 2023-08-27T11:19:35.987 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:35.988 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:36.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:36.046 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.70 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:36.103 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:36.103 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:36.104 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1636e-05 s, 0.0 kB/s 2023-08-27T11:19:36.104 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:36.104 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:36.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:36.160 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.71 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:36.217 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:36.218 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:36.218 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.955e-05 s, 0.0 kB/s 2023-08-27T11:19:36.220 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:36.221 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:36.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:36.277 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.72 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1304e-05 s, 0.0 kB/s 2023-08-27T11:19:36.336 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:36.336 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:36.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:36.396 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.73 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:36.453 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:36.453 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:36.454 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7168e-05 s, 0.0 kB/s 2023-08-27T11:19:36.454 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:36.455 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:36.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:36.513 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.74 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:36.570 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:36.570 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:36.571 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6179e-05 s, 0.0 kB/s 2023-08-27T11:19:36.571 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:36.571 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:36.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:36.630 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.75 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:36.688 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:36.688 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:36.688 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2507e-05 s, 0.0 kB/s 2023-08-27T11:19:36.689 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:36.689 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:36.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:36.746 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.76 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:36.804 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:36.804 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:36.804 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7962e-05 s, 0.0 kB/s 2023-08-27T11:19:36.804 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:36.805 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:36.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:36.862 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.77 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:36.919 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:36.920 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:36.920 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.757e-05 s, 0.0 kB/s 2023-08-27T11:19:36.920 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:36.921 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:36.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:36.978 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.78 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:37.036 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:37.036 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:37.037 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.949e-05 s, 0.0 kB/s 2023-08-27T11:19:37.037 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:37.037 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:37.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:37.093 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.79 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:37.149 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:37.150 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:37.150 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8602e-05 s, 0.0 kB/s 2023-08-27T11:19:37.150 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:37.151 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:37.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:37.207 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.80 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:37.266 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:37.267 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:37.267 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8641e-05 s, 0.0 kB/s 2023-08-27T11:19:37.267 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:37.268 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:37.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:37.324 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.81 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:37.382 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:37.382 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:37.382 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8981e-05 s, 0.0 kB/s 2023-08-27T11:19:37.383 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:37.383 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:37.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:37.440 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.82 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6186e-05 s, 0.0 kB/s 2023-08-27T11:19:37.497 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:37.498 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:37.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:37.554 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.83 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:37.612 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:37.612 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:37.612 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2599e-05 s, 0.0 kB/s 2023-08-27T11:19:37.613 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:37.613 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:37.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:37.671 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.84 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:37.727 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:37.728 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:37.728 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6967e-05 s, 0.0 kB/s 2023-08-27T11:19:37.728 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:37.729 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:37.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:37.786 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.85 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:37.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:37.843 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:37.843 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6071e-05 s, 0.0 kB/s 2023-08-27T11:19:37.843 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:37.844 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:37.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:37.901 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.86 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:37.958 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:37.959 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:37.959 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8407e-05 s, 0.0 kB/s 2023-08-27T11:19:37.959 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:37.960 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:38.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:38.018 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.87 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:38.083 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:38.084 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:38.084 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9689e-05 s, 0.0 kB/s 2023-08-27T11:19:38.086 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:38.086 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:38.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:38.149 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.88 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:38.210 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:38.210 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:38.210 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1413e-05 s, 0.0 kB/s 2023-08-27T11:19:38.211 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:38.211 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:38.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:38.272 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.89 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0918e-05 s, 0.0 kB/s 2023-08-27T11:19:38.332 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:38.333 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:38.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:38.392 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.90 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:38.454 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:38.454 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:38.454 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0684e-05 s, 0.0 kB/s 2023-08-27T11:19:38.455 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:38.456 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:38.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:38.520 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.91 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:38.582 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:38.582 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:38.582 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0182e-05 s, 0.0 kB/s 2023-08-27T11:19:38.583 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:38.583 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:38.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:38.651 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.92 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:38.711 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:38.711 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:38.711 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0477e-05 s, 0.0 kB/s 2023-08-27T11:19:38.712 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:38.712 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:38.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:38.774 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.93 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:38.831 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:38.831 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:38.831 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8078e-05 s, 0.0 kB/s 2023-08-27T11:19:38.832 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:38.832 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:38.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:38.892 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.94 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:38.948 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:38.949 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:38.949 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6271e-05 s, 0.0 kB/s 2023-08-27T11:19:38.949 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:38.950 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:39.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:39.019 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.95 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:39.078 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:39.078 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:39.078 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9379e-05 s, 0.0 kB/s 2023-08-27T11:19:39.079 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:39.079 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:39.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:39.147 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.96 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6978e-05 s, 0.0 kB/s 2023-08-27T11:19:39.205 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:39.206 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:39.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:39.263 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.97 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:39.321 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:39.321 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:39.322 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7589e-05 s, 0.0 kB/s 2023-08-27T11:19:39.322 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:39.322 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:39.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:39.380 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.98 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:39.438 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:39.439 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:39.439 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6642e-05 s, 0.0 kB/s 2023-08-27T11:19:39.439 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:39.440 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:39.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:39.497 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.99 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:39.556 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:39.556 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9525e-05 s, 0.0 kB/s 2023-08-27T11:19:39.556 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:39.557 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:39.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:39.614 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.100 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:39.672 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.622e-05 s, 0.0 kB/s 2023-08-27T11:19:39.672 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:39.672 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:39.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:39.729 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.101 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:39.787 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:39.787 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:39.788 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7291e-05 s, 0.0 kB/s 2023-08-27T11:19:39.788 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:39.788 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:39.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:39.845 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.102 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:39.902 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0906e-05 s, 0.0 kB/s 2023-08-27T11:19:39.904 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:39.905 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:39.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:39.960 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.103 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:40.018 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:40.019 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:40.019 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8552e-05 s, 0.0 kB/s 2023-08-27T11:19:40.019 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:40.020 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:40.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:40.077 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.104 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:40.134 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:40.134 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:40.134 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7857e-05 s, 0.0 kB/s 2023-08-27T11:19:40.135 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:40.135 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:40.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:40.193 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.105 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:40.251 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:40.251 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:40.251 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9129e-05 s, 0.0 kB/s 2023-08-27T11:19:40.252 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:40.252 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:40.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:40.309 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.106 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:40.370 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:40.370 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:40.370 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9223e-05 s, 0.0 kB/s 2023-08-27T11:19:40.371 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:40.371 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:40.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:40.428 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.107 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:40.485 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:40.486 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:40.486 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.973e-05 s, 0.0 kB/s 2023-08-27T11:19:40.486 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:40.487 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:40.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:40.543 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.108 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:40.600 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:40.601 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:40.601 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7292e-05 s, 0.0 kB/s 2023-08-27T11:19:40.601 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:40.602 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:40.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:40.659 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.109 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:40.716 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:40.716 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:40.717 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8504e-05 s, 0.0 kB/s 2023-08-27T11:19:40.717 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:40.718 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:40.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:40.774 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.110 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:40.832 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:40.832 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:40.832 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8759e-05 s, 0.0 kB/s 2023-08-27T11:19:40.833 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:40.833 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:40.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:40.890 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.111 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:40.946 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:40.946 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:40.947 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6276e-05 s, 0.0 kB/s 2023-08-27T11:19:40.947 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:40.947 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:41.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:41.005 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.112 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9056e-05 s, 0.0 kB/s 2023-08-27T11:19:41.062 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:41.062 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:41.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:41.119 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.113 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:41.175 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:41.176 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:41.176 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.824e-05 s, 0.0 kB/s 2023-08-27T11:19:41.177 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:41.177 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:41.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:41.234 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.114 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:41.291 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:41.292 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:41.292 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6629e-05 s, 0.0 kB/s 2023-08-27T11:19:41.292 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:41.293 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:41.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:41.351 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.115 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:41.407 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:41.407 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:41.408 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8791e-05 s, 0.0 kB/s 2023-08-27T11:19:41.408 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:41.408 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:41.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:41.465 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.116 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:41.524 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.996e-05 s, 0.0 kB/s 2023-08-27T11:19:41.524 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:41.525 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:41.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:41.581 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.117 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:41.638 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:41.638 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:41.638 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9246e-05 s, 0.0 kB/s 2023-08-27T11:19:41.639 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:41.639 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:41.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:41.698 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.118 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:41.753 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:41.754 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:41.754 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9533e-05 s, 0.0 kB/s 2023-08-27T11:19:41.755 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:41.755 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:41.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:41.811 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.119 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:41.869 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:41.870 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:41.870 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9791e-05 s, 0.0 kB/s 2023-08-27T11:19:41.870 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:41.871 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:41.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:41.927 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.120 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:41.986 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:41.986 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:41.986 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.721e-05 s, 0.0 kB/s 2023-08-27T11:19:41.986 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:41.987 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:42.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:42.044 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.121 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:42.102 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:42.102 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:42.102 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9916e-05 s, 0.0 kB/s 2023-08-27T11:19:42.103 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:42.103 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:42.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:42.159 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.122 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:42.217 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:42.217 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:42.217 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6817e-05 s, 0.0 kB/s 2023-08-27T11:19:42.218 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:42.218 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:42.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:42.275 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.123 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9359e-05 s, 0.0 kB/s 2023-08-27T11:19:42.333 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:42.333 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:42.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:42.390 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.124 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:42.448 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:42.448 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:42.449 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8342e-05 s, 0.0 kB/s 2023-08-27T11:19:42.449 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:42.449 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:42.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:42.505 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.125 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:42.562 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:42.563 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:42.563 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1117e-05 s, 0.0 kB/s 2023-08-27T11:19:42.563 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:42.564 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:42.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:42.620 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.126 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:42.677 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:42.677 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:42.678 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6658e-05 s, 0.0 kB/s 2023-08-27T11:19:42.678 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:42.678 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:42.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:42.735 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.127 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:42.794 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:42.794 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:42.794 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9526e-05 s, 0.0 kB/s 2023-08-27T11:19:42.795 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:42.795 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:42.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:42.852 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.128 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:42.909 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:42.909 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:42.909 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9642e-05 s, 0.0 kB/s 2023-08-27T11:19:42.910 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:42.910 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:42.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:42.966 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.129 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:43.023 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:43.024 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:43.025 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7401e-05 s, 0.0 kB/s 2023-08-27T11:19:43.025 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:43.026 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:43.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:43.082 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.130 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:43.139 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:43.140 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:43.140 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7933e-05 s, 0.0 kB/s 2023-08-27T11:19:43.140 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:43.141 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:43.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:43.197 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.131 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7699e-05 s, 0.0 kB/s 2023-08-27T11:19:43.255 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:43.257 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:43.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:43.315 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.132 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:43.372 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.907e-05 s, 0.0 kB/s 2023-08-27T11:19:43.453 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:43.454 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:43.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:43.473 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.133 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:43.529 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:43.530 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:43.530 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9949e-05 s, 0.0 kB/s 2023-08-27T11:19:43.531 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:43.531 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:43.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:43.588 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.134 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:43.645 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:43.645 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:43.646 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0888e-05 s, 0.0 kB/s 2023-08-27T11:19:43.646 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:43.646 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:43.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:43.703 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.135 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:43.761 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:43.762 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:43.762 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0911e-05 s, 0.0 kB/s 2023-08-27T11:19:43.762 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:43.763 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:43.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:43.820 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.136 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:43.877 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:43.877 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:43.877 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7322e-05 s, 0.0 kB/s 2023-08-27T11:19:43.878 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:43.878 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:43.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:43.935 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.137 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:43.993 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:43.993 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:43.993 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8866e-05 s, 0.0 kB/s 2023-08-27T11:19:43.994 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:43.994 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:44.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:44.051 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.138 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:44.109 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:44.109 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:44.109 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0754e-05 s, 0.0 kB/s 2023-08-27T11:19:44.110 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:44.110 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:44.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:44.167 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.139 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:44.223 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:44.224 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:44.224 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8203e-05 s, 0.0 kB/s 2023-08-27T11:19:44.225 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:44.225 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:44.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:44.281 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.140 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:44.338 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:44.339 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:44.339 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8384e-05 s, 0.0 kB/s 2023-08-27T11:19:44.340 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:44.340 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:44.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:44.397 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.141 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:44.453 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:44.453 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:44.453 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.269e-05 s, 0.0 kB/s 2023-08-27T11:19:44.454 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:44.454 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:44.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:44.511 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.142 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:44.568 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:44.568 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:44.568 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1766e-05 s, 0.0 kB/s 2023-08-27T11:19:44.569 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:44.569 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:44.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:44.627 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.143 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:44.683 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:44.683 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:44.684 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8164e-05 s, 0.0 kB/s 2023-08-27T11:19:44.684 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:44.685 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:44.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:44.741 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.144 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6064e-05 s, 0.0 kB/s 2023-08-27T11:19:44.799 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:44.799 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:44.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:44.861 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.145 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8641e-05 s, 0.0 kB/s 2023-08-27T11:19:44.918 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:44.919 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:44.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:44.974 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.146 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7425e-05 s, 0.0 kB/s 2023-08-27T11:19:45.033 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:45.033 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:45.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:45.090 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.147 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:45.147 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:45.147 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:45.148 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2129e-05 s, 0.0 kB/s 2023-08-27T11:19:45.148 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:45.149 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:45.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:45.205 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.148 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:45.261 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:45.262 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:45.262 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6822e-05 s, 0.0 kB/s 2023-08-27T11:19:45.262 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:45.262 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:45.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:45.321 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.149 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9771e-05 s, 0.0 kB/s 2023-08-27T11:19:45.380 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:45.381 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:45.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:45.436 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.150 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:45.493 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:45.494 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:45.494 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.595e-05 s, 0.0 kB/s 2023-08-27T11:19:45.494 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:45.497 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:45.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:45.552 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.151 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:45.609 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:45.610 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:45.610 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8191e-05 s, 0.0 kB/s 2023-08-27T11:19:45.610 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:45.611 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:45.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:45.668 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.152 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:45.726 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:45.726 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:45.726 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7146e-05 s, 0.0 kB/s 2023-08-27T11:19:45.727 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:45.727 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:45.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:45.783 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.153 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:45.841 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:45.841 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:45.841 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0499e-05 s, 0.0 kB/s 2023-08-27T11:19:45.842 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:45.842 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:45.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:45.899 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.154 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:45.955 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:45.955 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1185e-05 s, 0.0 kB/s 2023-08-27T11:19:45.956 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:45.956 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:46.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:46.013 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.155 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:46.070 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:46.070 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:46.071 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0982e-05 s, 0.0 kB/s 2023-08-27T11:19:46.071 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:46.072 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:46.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:46.128 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.156 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:46.186 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:46.187 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:46.187 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0335e-05 s, 0.0 kB/s 2023-08-27T11:19:46.187 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:46.188 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:46.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:46.244 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.157 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:46.302 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:46.302 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1765e-05 s, 0.0 kB/s 2023-08-27T11:19:46.303 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:46.304 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:46.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:46.360 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.158 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:46.418 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:46.418 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7175e-05 s, 0.0 kB/s 2023-08-27T11:19:46.418 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:46.419 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:46.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:46.476 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.159 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:46.533 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:46.534 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:46.534 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8066e-05 s, 0.0 kB/s 2023-08-27T11:19:46.534 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:46.535 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:46.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:46.594 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.160 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:46.650 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:46.651 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:46.651 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7974e-05 s, 0.0 kB/s 2023-08-27T11:19:46.651 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:46.652 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:46.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:46.709 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.161 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:46.765 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:46.765 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2722e-05 s, 0.0 kB/s 2023-08-27T11:19:46.766 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:46.766 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:46.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:46.822 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.162 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:46.880 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:46.880 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0329e-05 s, 0.0 kB/s 2023-08-27T11:19:46.880 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:46.881 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:46.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:46.939 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.163 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:46.996 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7122e-05 s, 0.0 kB/s 2023-08-27T11:19:46.996 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:46.997 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:47.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:47.052 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.164 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:47.109 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8988e-05 s, 0.0 kB/s 2023-08-27T11:19:47.110 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:47.112 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:47.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:47.168 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.165 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:47.224 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:47.224 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:47.224 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9006e-05 s, 0.0 kB/s 2023-08-27T11:19:47.225 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:47.225 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:47.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:47.282 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.166 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:47.339 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:47.340 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:47.340 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8415e-05 s, 0.0 kB/s 2023-08-27T11:19:47.341 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:47.341 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:47.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:47.398 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.167 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8331e-05 s, 0.0 kB/s 2023-08-27T11:19:47.456 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:47.457 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:47.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:47.514 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.168 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:47.571 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:47.571 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:47.572 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8974e-05 s, 0.0 kB/s 2023-08-27T11:19:47.572 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:47.572 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:47.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:47.629 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.169 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.884e-05 s, 0.0 kB/s 2023-08-27T11:19:47.687 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:47.687 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:47.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:47.745 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.170 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:47.806 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:47.806 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:47.807 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.841e-05 s, 0.0 kB/s 2023-08-27T11:19:47.807 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:47.808 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:47.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:47.864 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.171 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:47.922 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:47.922 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:47.922 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8835e-05 s, 0.0 kB/s 2023-08-27T11:19:47.923 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:47.923 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:47.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:47.979 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.172 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:48.037 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:48.037 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:48.037 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0188e-05 s, 0.0 kB/s 2023-08-27T11:19:48.038 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:48.038 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:48.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:48.097 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.173 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:48.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:48.154 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:48.154 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6155e-05 s, 0.0 kB/s 2023-08-27T11:19:48.154 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:48.155 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:48.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:48.214 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.174 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:48.273 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:48.274 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:48.274 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7906e-05 s, 0.0 kB/s 2023-08-27T11:19:48.274 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:48.274 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:48.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:48.331 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.175 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:48.388 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:48.389 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:48.389 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0135e-05 s, 0.0 kB/s 2023-08-27T11:19:48.389 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:48.390 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:48.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:48.446 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.176 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:48.502 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:48.502 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9523e-05 s, 0.0 kB/s 2023-08-27T11:19:48.503 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:48.503 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:48.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:48.560 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.177 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:48.617 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1701e-05 s, 0.0 kB/s 2023-08-27T11:19:48.617 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:48.617 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:48.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:48.674 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.178 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:48.731 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:48.732 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:48.732 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9584e-05 s, 0.0 kB/s 2023-08-27T11:19:48.732 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:48.733 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:48.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:48.789 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.179 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:48.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:48.846 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:48.846 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6902e-05 s, 0.0 kB/s 2023-08-27T11:19:48.846 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:48.847 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:48.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:48.905 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.180 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:48.961 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:48.961 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:48.961 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9391e-05 s, 0.0 kB/s 2023-08-27T11:19:48.962 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:48.962 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:49.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:49.018 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.181 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:49.075 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:49.076 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:49.077 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7315e-05 s, 0.0 kB/s 2023-08-27T11:19:49.077 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:49.077 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:49.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:49.133 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.182 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:49.190 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:49.190 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:49.190 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7515e-05 s, 0.0 kB/s 2023-08-27T11:19:49.191 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:49.191 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:49.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:49.249 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.183 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:49.306 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:49.306 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:49.306 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6505e-05 s, 0.0 kB/s 2023-08-27T11:19:49.307 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:49.307 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:49.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:49.364 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.184 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:49.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:49.421 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:49.421 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8537e-05 s, 0.0 kB/s 2023-08-27T11:19:49.421 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:49.422 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:49.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:49.482 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.185 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:49.537 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:49.538 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:49.538 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0908e-05 s, 0.0 kB/s 2023-08-27T11:19:49.538 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:49.539 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:49.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:49.596 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.186 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:49.654 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:49.654 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:49.654 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9774e-05 s, 0.0 kB/s 2023-08-27T11:19:49.655 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:49.655 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:49.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:49.712 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.187 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:49.769 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:49.769 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:49.770 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8707e-05 s, 0.0 kB/s 2023-08-27T11:19:49.770 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:49.770 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:49.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:49.826 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.188 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:49.883 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:49.883 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:49.884 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8014e-05 s, 0.0 kB/s 2023-08-27T11:19:49.884 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:49.884 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:49.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:49.940 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.189 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0611e-05 s, 0.0 kB/s 2023-08-27T11:19:50.000 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:50.001 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:50.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:50.058 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.190 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6509e-05 s, 0.0 kB/s 2023-08-27T11:19:50.115 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:50.116 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:50.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:50.173 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.191 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8859e-05 s, 0.0 kB/s 2023-08-27T11:19:50.231 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:50.231 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:50.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:50.288 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.192 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8857e-05 s, 0.0 kB/s 2023-08-27T11:19:50.349 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:50.350 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:50.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:50.403 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.193 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:50.459 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:50.460 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:50.460 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7589e-05 s, 0.0 kB/s 2023-08-27T11:19:50.460 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:50.461 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:50.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:50.516 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.194 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:50.574 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:50.574 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:50.575 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8982e-05 s, 0.0 kB/s 2023-08-27T11:19:50.575 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:50.575 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:50.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:50.632 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.195 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:50.688 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:50.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:50.689 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9288e-05 s, 0.0 kB/s 2023-08-27T11:19:50.689 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:50.690 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:50.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:50.747 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.196 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:50.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:50.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:50.805 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0398e-05 s, 0.0 kB/s 2023-08-27T11:19:50.806 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:50.806 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:50.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:50.864 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.197 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:50.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:50.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:50.922 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0452e-05 s, 0.0 kB/s 2023-08-27T11:19:50.922 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:50.922 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:50.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:50.979 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.198 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:51.036 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:51.036 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:51.036 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.122e-05 s, 0.0 kB/s 2023-08-27T11:19:51.037 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:51.037 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:51.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:51.094 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.199 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:51.152 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6878e-05 s, 0.0 kB/s 2023-08-27T11:19:51.153 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:51.154 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:51.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:51.210 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.200 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:51.268 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:51.268 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:51.268 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0364e-05 s, 0.0 kB/s 2023-08-27T11:19:51.269 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:51.269 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:51.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:51.326 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.201 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:51.386 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:51.386 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:51.387 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9972e-05 s, 0.0 kB/s 2023-08-27T11:19:51.387 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:51.387 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:51.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:51.444 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.202 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:51.502 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:51.502 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:51.502 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0486e-05 s, 0.0 kB/s 2023-08-27T11:19:51.502 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:51.503 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:51.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:51.560 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.203 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:51.617 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:51.618 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:51.618 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.74e-05 s, 0.0 kB/s 2023-08-27T11:19:51.618 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:51.618 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:51.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:51.676 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.204 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:51.732 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:51.733 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:51.733 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8834e-05 s, 0.0 kB/s 2023-08-27T11:19:51.734 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:51.734 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:51.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:51.790 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.205 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:51.847 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:51.847 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:51.848 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9874e-05 s, 0.0 kB/s 2023-08-27T11:19:51.848 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:51.848 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:51.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:51.905 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.206 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:51.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:51.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:51.963 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0891e-05 s, 0.0 kB/s 2023-08-27T11:19:51.963 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:51.964 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:52.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:52.020 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.207 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:52.076 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:52.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:52.077 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7275e-05 s, 0.0 kB/s 2023-08-27T11:19:52.078 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:52.078 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:52.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:52.135 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.208 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:52.194 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6836e-05 s, 0.0 kB/s 2023-08-27T11:19:52.194 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:52.195 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:52.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:52.251 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.209 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:52.309 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:52.309 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:52.309 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1818e-05 s, 0.0 kB/s 2023-08-27T11:19:52.310 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:52.310 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:52.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:52.367 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.210 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:52.424 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:52.424 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:52.425 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1495e-05 s, 0.0 kB/s 2023-08-27T11:19:52.425 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:52.426 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:52.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:52.482 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.211 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:52.539 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:52.540 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:52.540 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9495e-05 s, 0.0 kB/s 2023-08-27T11:19:52.540 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:52.541 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:52.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:52.598 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.212 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:52.655 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:52.655 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:52.656 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9742e-05 s, 0.0 kB/s 2023-08-27T11:19:52.656 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:52.656 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:52.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:52.713 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.213 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:52.770 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:52.771 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:52.771 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7431e-05 s, 0.0 kB/s 2023-08-27T11:19:52.771 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:52.772 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:52.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:52.829 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.214 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:52.885 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:52.886 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:52.886 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9005e-05 s, 0.0 kB/s 2023-08-27T11:19:52.886 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:52.887 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:52.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:52.944 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.215 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:53.002 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:53.002 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:53.002 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8252e-05 s, 0.0 kB/s 2023-08-27T11:19:53.003 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:53.003 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:53.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:53.060 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.216 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:53.116 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:53.117 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:53.117 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.568e-05 s, 0.0 kB/s 2023-08-27T11:19:53.117 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:53.118 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:53.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:53.175 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.217 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:53.231 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:53.232 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:53.232 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.682e-05 s, 0.0 kB/s 2023-08-27T11:19:53.233 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:53.234 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:53.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:53.291 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.218 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:53.349 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:53.350 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:53.350 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9316e-05 s, 0.0 kB/s 2023-08-27T11:19:53.350 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:53.351 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:53.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:53.407 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.219 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:53.466 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:53.466 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:53.466 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.82e-05 s, 0.0 kB/s 2023-08-27T11:19:53.467 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:53.467 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:53.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:53.524 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.220 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:53.581 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:53.582 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:53.582 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8394e-05 s, 0.0 kB/s 2023-08-27T11:19:53.582 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:53.583 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:53.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:53.640 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.221 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:53.697 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:53.698 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:53.698 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0667e-05 s, 0.0 kB/s 2023-08-27T11:19:53.698 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:53.699 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:53.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:53.757 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.222 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:53.814 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:53.814 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:53.814 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.985e-05 s, 0.0 kB/s 2023-08-27T11:19:53.815 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:53.815 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:53.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:53.872 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.223 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:53.930 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:53.930 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.05e-05 s, 0.0 kB/s 2023-08-27T11:19:53.930 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:53.931 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:53.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:53.988 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.224 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:54.044 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:54.045 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:54.045 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.713e-05 s, 0.0 kB/s 2023-08-27T11:19:54.045 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:54.046 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:54.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:54.103 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.225 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1059e-05 s, 0.0 kB/s 2023-08-27T11:19:54.161 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:54.161 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:54.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:54.221 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.226 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:54.278 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:54.278 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:54.278 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0063e-05 s, 0.0 kB/s 2023-08-27T11:19:54.279 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:54.279 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:54.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:54.337 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.227 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:54.394 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:54.394 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:54.395 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9058e-05 s, 0.0 kB/s 2023-08-27T11:19:54.395 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:54.395 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:54.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:54.452 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.228 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:54.508 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:54.508 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:54.509 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.4929e-05 s, 0.0 kB/s 2023-08-27T11:19:54.509 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:54.510 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:54.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:54.566 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.229 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:54.622 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:54.623 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:54.623 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7871e-05 s, 0.0 kB/s 2023-08-27T11:19:54.623 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:54.624 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:54.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:54.681 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.230 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:54.737 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:54.737 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:54.738 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8453e-05 s, 0.0 kB/s 2023-08-27T11:19:54.738 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:54.738 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:54.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:54.796 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.231 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:54.853 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:54.853 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:54.853 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8066e-05 s, 0.0 kB/s 2023-08-27T11:19:54.854 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:54.854 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:54.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:54.911 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.232 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:54.970 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:54.970 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0734e-05 s, 0.0 kB/s 2023-08-27T11:19:54.970 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:54.971 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:55.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:55.028 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.233 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:55.085 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:55.085 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:55.086 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0815e-05 s, 0.0 kB/s 2023-08-27T11:19:55.086 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:55.087 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:55.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:55.143 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.234 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:55.200 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:55.200 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:55.200 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3552e-05 s, 0.0 kB/s 2023-08-27T11:19:55.201 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:55.201 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:55.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:55.258 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.235 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9589e-05 s, 0.0 kB/s 2023-08-27T11:19:55.316 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:55.316 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:55.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:55.375 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.236 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:55.432 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:55.432 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:55.433 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8486e-05 s, 0.0 kB/s 2023-08-27T11:19:55.433 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:55.434 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:55.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:55.490 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.237 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:55.547 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:55.548 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:55.548 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9387e-05 s, 0.0 kB/s 2023-08-27T11:19:55.548 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:55.549 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:55.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:55.606 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.238 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:55.662 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:55.662 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:55.663 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1784e-05 s, 0.0 kB/s 2023-08-27T11:19:55.663 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:55.664 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:55.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:55.720 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.239 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0016e-05 s, 0.0 kB/s 2023-08-27T11:19:55.778 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:55.778 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:55.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:55.837 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.240 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:55.893 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:55.893 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:55.893 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7971e-05 s, 0.0 kB/s 2023-08-27T11:19:55.894 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:55.894 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:55.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:55.952 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.241 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:56.009 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:56.010 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:56.010 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7399e-05 s, 0.0 kB/s 2023-08-27T11:19:56.010 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:56.011 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:56.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:56.070 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.242 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:56.130 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:56.131 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8368e-05 s, 0.0 kB/s 2023-08-27T11:19:56.131 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:56.132 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:56.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:56.185 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.243 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:56.242 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:56.243 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:56.243 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8848e-05 s, 0.0 kB/s 2023-08-27T11:19:56.243 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:56.244 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:56.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:56.301 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.244 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:56.357 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:56.357 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:56.358 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8363e-05 s, 0.0 kB/s 2023-08-27T11:19:56.358 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:56.358 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:56.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:56.415 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.245 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:56.473 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:56.473 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7261e-05 s, 0.0 kB/s 2023-08-27T11:19:56.474 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:56.474 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:56.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:56.531 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.246 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:56.588 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:56.588 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:56.588 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0017e-05 s, 0.0 kB/s 2023-08-27T11:19:56.589 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:56.589 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:56.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:56.646 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.247 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:56.704 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:56.705 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:56.705 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9722e-05 s, 0.0 kB/s 2023-08-27T11:19:56.705 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:56.706 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:56.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:56.763 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.248 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:56.821 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:56.824 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:56.824 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3073e-05 s, 0.0 kB/s 2023-08-27T11:19:56.825 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:56.825 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:56.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:56.879 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.249 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:56.936 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:56.936 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:56.937 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7114e-05 s, 0.0 kB/s 2023-08-27T11:19:56.937 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:56.937 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:56.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:56.994 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.250 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:57.052 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:57.052 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:57.052 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9088e-05 s, 0.0 kB/s 2023-08-27T11:19:57.053 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:57.053 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:57.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:57.110 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.251 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:57.167 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:57.168 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:57.168 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7103e-05 s, 0.0 kB/s 2023-08-27T11:19:57.168 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:57.169 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:57.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:57.225 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.252 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:57.282 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:57.282 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:57.282 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8806e-05 s, 0.0 kB/s 2023-08-27T11:19:57.283 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:57.283 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:57.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:57.343 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.253 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:57.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:57.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:57.402 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7518e-05 s, 0.0 kB/s 2023-08-27T11:19:57.402 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:57.403 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:57.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:57.460 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.254 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:57.519 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:57.519 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:57.519 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2784e-05 s, 0.0 kB/s 2023-08-27T11:19:57.520 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:57.520 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:57.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:57.577 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.255 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:57.634 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:57.635 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:57.635 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9135e-05 s, 0.0 kB/s 2023-08-27T11:19:57.635 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:57.636 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:57.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:57.694 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.256 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:57.750 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:57.750 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:57.751 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.912e-05 s, 0.0 kB/s 2023-08-27T11:19:57.751 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:57.752 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:57.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:57.808 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.257 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:57.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0864e-05 s, 0.0 kB/s 2023-08-27T11:19:57.865 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:57.866 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:57.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:57.926 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.258 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9712e-05 s, 0.0 kB/s 2023-08-27T11:19:57.983 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:57.984 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:58.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:58.043 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.259 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:58.099 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:58.100 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:58.100 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.69e-05 s, 0.0 kB/s 2023-08-27T11:19:58.100 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:58.101 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:58.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:58.158 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.260 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:58.215 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:58.215 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:58.215 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9895e-05 s, 0.0 kB/s 2023-08-27T11:19:58.216 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:58.216 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:58.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:58.272 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.261 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:58.329 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:58.329 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:58.330 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1927e-05 s, 0.0 kB/s 2023-08-27T11:19:58.330 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:58.330 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:58.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:58.387 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.262 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6936e-05 s, 0.0 kB/s 2023-08-27T11:19:58.444 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:58.444 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:58.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:58.501 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.263 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:58.558 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:58.558 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:58.558 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8675e-05 s, 0.0 kB/s 2023-08-27T11:19:58.558 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:58.559 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:58.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:58.616 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.264 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:58.672 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:58.673 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:58.673 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8333e-05 s, 0.0 kB/s 2023-08-27T11:19:58.673 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:58.674 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:58.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:58.732 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.265 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:58.789 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:58.789 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9781e-05 s, 0.0 kB/s 2023-08-27T11:19:58.790 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:58.790 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:58.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:58.847 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.266 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:58.903 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:58.904 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:58.904 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6584e-05 s, 0.0 kB/s 2023-08-27T11:19:58.904 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:58.905 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:58.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:58.962 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.267 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:59.018 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:59.019 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:59.019 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7759e-05 s, 0.0 kB/s 2023-08-27T11:19:59.020 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:59.020 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:59.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:59.076 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.268 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:59.133 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:59.133 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:59.134 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8068e-05 s, 0.0 kB/s 2023-08-27T11:19:59.134 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:59.135 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:59.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:59.191 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.269 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:59.247 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:59.248 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:59.248 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9519e-05 s, 0.0 kB/s 2023-08-27T11:19:59.249 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:59.249 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:59.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:59.305 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.270 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:59.361 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:59.362 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:59.362 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.983e-05 s, 0.0 kB/s 2023-08-27T11:19:59.363 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:59.363 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:59.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:59.420 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.271 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:59.477 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9082e-05 s, 0.0 kB/s 2023-08-27T11:19:59.478 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:59.479 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:59.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:59.537 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.272 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:59.594 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:59.594 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:59.594 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9315e-05 s, 0.0 kB/s 2023-08-27T11:19:59.595 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:59.595 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:59.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:59.656 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.273 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:59.712 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:59.712 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:59.713 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6129e-05 s, 0.0 kB/s 2023-08-27T11:19:59.713 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:59.713 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:59.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:59.772 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.274 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:59.829 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:59.830 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:59.830 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7955e-05 s, 0.0 kB/s 2023-08-27T11:19:59.830 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:59.831 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:19:59.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:19:59.888 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.275 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:19:59.946 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:19:59.946 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:19:59.946 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.785e-05 s, 0.0 kB/s 2023-08-27T11:19:59.947 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:19:59.947 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:00.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:00.007 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.276 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:00.064 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:00.064 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:00.064 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.82e-05 s, 0.0 kB/s 2023-08-27T11:20:00.065 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:00.065 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:00.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:00.122 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.277 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:00.179 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9606e-05 s, 0.0 kB/s 2023-08-27T11:20:00.180 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:00.181 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:00.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:00.238 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.278 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:00.295 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:00.296 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:00.296 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4673e-05 s, 0.0 kB/s 2023-08-27T11:20:00.296 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:00.297 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:00.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:00.354 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.279 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:00.411 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8237e-05 s, 0.0 kB/s 2023-08-27T11:20:00.412 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:00.413 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:00.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:00.469 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.280 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:00.526 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:00.526 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8511e-05 s, 0.0 kB/s 2023-08-27T11:20:00.527 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:00.528 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:00.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:00.586 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.281 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:00.644 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:00.644 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:00.644 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.079e-05 s, 0.0 kB/s 2023-08-27T11:20:00.645 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:00.645 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:00.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:00.703 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.282 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:00.759 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:00.760 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:00.760 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1362e-05 s, 0.0 kB/s 2023-08-27T11:20:00.760 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:00.761 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:00.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:00.820 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.283 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:00.877 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:00.877 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9639e-05 s, 0.0 kB/s 2023-08-27T11:20:00.878 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:00.878 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:00.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:00.935 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.284 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:00.992 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:00.993 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:00.993 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9903e-05 s, 0.0 kB/s 2023-08-27T11:20:00.993 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:00.994 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:01.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:01.050 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.285 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:01.107 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:01.107 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:01.108 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8538e-05 s, 0.0 kB/s 2023-08-27T11:20:01.108 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:01.108 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:01.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:01.165 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.286 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:01.222 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:01.224 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:01.224 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7773e-05 s, 0.0 kB/s 2023-08-27T11:20:01.225 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:01.226 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:01.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:01.280 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.287 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:01.336 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:01.337 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:01.338 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0538e-05 s, 0.0 kB/s 2023-08-27T11:20:01.338 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:01.339 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:01.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:01.394 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.288 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:01.451 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:01.452 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:01.452 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.02e-05 s, 0.0 kB/s 2023-08-27T11:20:01.452 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:01.453 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:01.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:01.510 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.289 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:01.567 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8534e-05 s, 0.0 kB/s 2023-08-27T11:20:01.568 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:01.568 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:01.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:01.625 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.290 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:01.683 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:01.683 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:01.683 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9431e-05 s, 0.0 kB/s 2023-08-27T11:20:01.684 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:01.684 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:01.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:01.743 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.291 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:01.800 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:01.800 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:01.800 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2926e-05 s, 0.0 kB/s 2023-08-27T11:20:01.801 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:01.801 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:01.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:01.858 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.292 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:01.914 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:01.915 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:01.915 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.152e-05 s, 0.0 kB/s 2023-08-27T11:20:01.915 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:01.916 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:01.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:01.972 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.293 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:02.029 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:02.030 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:02.030 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8574e-05 s, 0.0 kB/s 2023-08-27T11:20:02.030 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:02.031 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:02.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:02.088 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.294 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:02.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:02.145 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:02.145 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9078e-05 s, 0.0 kB/s 2023-08-27T11:20:02.145 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:02.146 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:02.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:02.203 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.295 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:02.259 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:02.260 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:02.260 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8333e-05 s, 0.0 kB/s 2023-08-27T11:20:02.260 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:02.261 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:02.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:02.319 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.296 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:02.376 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:02.377 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:02.377 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7645e-05 s, 0.0 kB/s 2023-08-27T11:20:02.378 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:02.378 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:02.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:02.434 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.297 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:02.491 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:02.491 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:02.492 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9368e-05 s, 0.0 kB/s 2023-08-27T11:20:02.492 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:02.492 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:02.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:02.549 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.298 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:02.605 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:02.606 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:02.606 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8852e-05 s, 0.0 kB/s 2023-08-27T11:20:02.606 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:02.607 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:02.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:02.664 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.299 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:02.722 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:02.722 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:02.723 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7205e-05 s, 0.0 kB/s 2023-08-27T11:20:02.723 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:02.723 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:02.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:02.780 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.300 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:02.837 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:02.837 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:02.838 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6997e-05 s, 0.0 kB/s 2023-08-27T11:20:02.838 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:02.838 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:02.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:02.895 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.301 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:02.953 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:02.953 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:02.953 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8963e-05 s, 0.0 kB/s 2023-08-27T11:20:02.954 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:02.954 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:03.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:03.012 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.302 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:03.069 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:03.069 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:03.069 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8133e-05 s, 0.0 kB/s 2023-08-27T11:20:03.070 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:03.070 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:03.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:03.127 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.303 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:03.183 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:03.183 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:03.184 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7394e-05 s, 0.0 kB/s 2023-08-27T11:20:03.184 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:03.185 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:03.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:03.242 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.304 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:03.299 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:03.299 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:03.300 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.898e-05 s, 0.0 kB/s 2023-08-27T11:20:03.300 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:03.300 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:03.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:03.359 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.305 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:03.416 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:03.416 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:03.416 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8724e-05 s, 0.0 kB/s 2023-08-27T11:20:03.417 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:03.417 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:03.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:03.474 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.306 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:03.531 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:03.532 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:03.532 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9181e-05 s, 0.0 kB/s 2023-08-27T11:20:03.532 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:03.533 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:03.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:03.589 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.307 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:03.645 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:03.646 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:03.646 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7224e-05 s, 0.0 kB/s 2023-08-27T11:20:03.646 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:03.647 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:03.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:03.704 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.308 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1343e-05 s, 0.0 kB/s 2023-08-27T11:20:03.763 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:03.763 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:03.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:03.820 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.309 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:03.879 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:03.880 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:03.880 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9397e-05 s, 0.0 kB/s 2023-08-27T11:20:03.881 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:03.881 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:03.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:03.953 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.310 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:04.023 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1393e-05 s, 0.0 kB/s 2023-08-27T11:20:04.025 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:04.025 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:04.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:04.081 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.311 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:04.142 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:04.143 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:04.143 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1643e-05 s, 0.0 kB/s 2023-08-27T11:20:04.143 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:04.144 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:04.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:04.202 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.312 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:04.264 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:04.264 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:04.264 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9542e-05 s, 0.0 kB/s 2023-08-27T11:20:04.265 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:04.265 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:04.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:04.325 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.313 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:04.387 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:04.388 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:04.388 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8504e-05 s, 0.0 kB/s 2023-08-27T11:20:04.389 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:04.389 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:04.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:04.448 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.314 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:04.508 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:04.509 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:04.509 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7261e-05 s, 0.0 kB/s 2023-08-27T11:20:04.510 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:04.510 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:04.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:04.572 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.315 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8784e-05 s, 0.0 kB/s 2023-08-27T11:20:04.631 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:04.631 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:04.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:04.689 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.316 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8575e-05 s, 0.0 kB/s 2023-08-27T11:20:04.748 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:04.750 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:04.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:04.810 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.317 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:04.867 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:04.868 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:04.868 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7883e-05 s, 0.0 kB/s 2023-08-27T11:20:04.868 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:04.869 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:04.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:04.931 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.318 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:04.987 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:04.988 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:04.988 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7023e-05 s, 0.0 kB/s 2023-08-27T11:20:04.989 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:04.989 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:05.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:05.049 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.319 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:05.110 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:05.111 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:05.111 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0034e-05 s, 0.0 kB/s 2023-08-27T11:20:05.111 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:05.112 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:05.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:05.176 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.320 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.846e-05 s, 0.0 kB/s 2023-08-27T11:20:05.238 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:05.238 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:05.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:05.302 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.321 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:05.361 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:05.361 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:05.362 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0149e-05 s, 0.0 kB/s 2023-08-27T11:20:05.362 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:05.364 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:05.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:05.421 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.322 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:05.478 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:05.478 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:05.479 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9535e-05 s, 0.0 kB/s 2023-08-27T11:20:05.479 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:05.479 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:05.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:05.536 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.323 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:05.592 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:05.593 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:05.593 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7659e-05 s, 0.0 kB/s 2023-08-27T11:20:05.593 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:05.594 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:05.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:05.651 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.324 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:05.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:05.709 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:05.709 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.881e-05 s, 0.0 kB/s 2023-08-27T11:20:05.710 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:05.710 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:05.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:05.767 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.325 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:05.825 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:05.825 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:05.825 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7768e-05 s, 0.0 kB/s 2023-08-27T11:20:05.826 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:05.826 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:05.874 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:20:05.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:05.884 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.326 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:05.885 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:20:05.941 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:05.941 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:05.942 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9512e-05 s, 0.0 kB/s 2023-08-27T11:20:05.942 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:05.943 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:06.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:06.015 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.327 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:06.072 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:06.072 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:06.072 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.931e-05 s, 0.0 kB/s 2023-08-27T11:20:06.072 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:06.073 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:06.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:06.130 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.328 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:06.187 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:06.188 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:06.188 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1976e-05 s, 0.0 kB/s 2023-08-27T11:20:06.188 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:06.189 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:06.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:06.253 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.329 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:06.308 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:06.309 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:06.309 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.129e-05 s, 0.0 kB/s 2023-08-27T11:20:06.309 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:06.310 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:06.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:06.367 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.330 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:06.424 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:06.425 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:06.426 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6321e-05 s, 0.0 kB/s 2023-08-27T11:20:06.426 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:06.427 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:06.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:06.482 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.331 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:06.539 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:06.539 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:06.540 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6794e-05 s, 0.0 kB/s 2023-08-27T11:20:06.540 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:06.542 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:06.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:06.598 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.332 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9931e-05 s, 0.0 kB/s 2023-08-27T11:20:06.657 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:06.658 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:06.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:06.715 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.333 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:06.772 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:06.772 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1881e-05 s, 0.0 kB/s 2023-08-27T11:20:06.773 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:06.774 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:06.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:06.831 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.334 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:06.888 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:06.888 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:06.888 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8578e-05 s, 0.0 kB/s 2023-08-27T11:20:06.889 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:06.889 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:06.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:06.947 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.335 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:07.005 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:07.006 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:07.006 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.021e-05 s, 0.0 kB/s 2023-08-27T11:20:07.007 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:07.008 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:07.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:07.069 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.336 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:07.126 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:07.126 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:07.126 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7571e-05 s, 0.0 kB/s 2023-08-27T11:20:07.126 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:07.127 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:07.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:07.185 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.337 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:07.242 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:07.243 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:07.243 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9206e-05 s, 0.0 kB/s 2023-08-27T11:20:07.243 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:07.244 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:07.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:07.300 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.338 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8765e-05 s, 0.0 kB/s 2023-08-27T11:20:07.359 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:07.359 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:07.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:07.416 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.339 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:07.474 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:07.474 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:07.474 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8365e-05 s, 0.0 kB/s 2023-08-27T11:20:07.475 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:07.475 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:07.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:07.533 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.340 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:07.590 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:07.590 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:07.590 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.819e-05 s, 0.0 kB/s 2023-08-27T11:20:07.591 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:07.591 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:07.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:07.648 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.341 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:07.704 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:07.705 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:07.705 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6128e-05 s, 0.0 kB/s 2023-08-27T11:20:07.705 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:07.706 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:07.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:07.763 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.342 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:07.820 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7033e-05 s, 0.0 kB/s 2023-08-27T11:20:07.821 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:07.822 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:07.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:07.878 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.343 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:07.936 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:07.936 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:07.937 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5487e-05 s, 0.0 kB/s 2023-08-27T11:20:07.937 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:07.937 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:07.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:07.994 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.344 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:08.052 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:08.053 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:08.053 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0662e-05 s, 0.0 kB/s 2023-08-27T11:20:08.054 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:08.054 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:08.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:08.111 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.345 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:08.167 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:08.168 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:08.168 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8344e-05 s, 0.0 kB/s 2023-08-27T11:20:08.168 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:08.169 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:08.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:08.226 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.346 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:08.282 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:08.283 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:08.283 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6955e-05 s, 0.0 kB/s 2023-08-27T11:20:08.283 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:08.283 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:08.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:08.341 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.347 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:08.397 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:08.398 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:08.398 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7704e-05 s, 0.0 kB/s 2023-08-27T11:20:08.398 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:08.399 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:08.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:08.455 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.348 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:08.511 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:08.512 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:08.512 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1453e-05 s, 0.0 kB/s 2023-08-27T11:20:08.512 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:08.513 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:08.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:08.569 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.349 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:08.626 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:08.627 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:08.627 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9178e-05 s, 0.0 kB/s 2023-08-27T11:20:08.627 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:08.628 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:08.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:08.685 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.350 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:08.742 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:08.742 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:08.742 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.957e-05 s, 0.0 kB/s 2023-08-27T11:20:08.743 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:08.743 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:08.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:08.801 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.351 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:08.859 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:08.859 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:08.860 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1869e-05 s, 0.0 kB/s 2023-08-27T11:20:08.862 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:08.862 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:08.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:08.917 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.352 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:08.973 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:08.974 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:08.974 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0336e-05 s, 0.0 kB/s 2023-08-27T11:20:08.974 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:08.975 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:09.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:09.032 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.353 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:09.089 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:09.089 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:09.089 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8176e-05 s, 0.0 kB/s 2023-08-27T11:20:09.090 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:09.090 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:09.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:09.147 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.354 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:09.203 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:09.204 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:09.204 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8743e-05 s, 0.0 kB/s 2023-08-27T11:20:09.204 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:09.205 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:09.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:09.265 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.355 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:09.329 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:09.329 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:09.329 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1278e-05 s, 0.0 kB/s 2023-08-27T11:20:09.330 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:09.330 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:09.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:09.390 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.356 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:09.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:09.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:09.420 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7633e-05 s, 0.0 kB/s 2023-08-27T11:20:09.420 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:09.421 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:09.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:09.483 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.357 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:09.547 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:09.547 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:09.548 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8132e-05 s, 0.0 kB/s 2023-08-27T11:20:09.548 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:09.549 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:09.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:09.618 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.358 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:09.679 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:09.679 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:09.680 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9725e-05 s, 0.0 kB/s 2023-08-27T11:20:09.680 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:09.681 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:09.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:09.740 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.359 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:09.798 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:09.798 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:09.798 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8551e-05 s, 0.0 kB/s 2023-08-27T11:20:09.801 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:09.801 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:09.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:09.856 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.360 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:09.913 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:09.914 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:09.914 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8655e-05 s, 0.0 kB/s 2023-08-27T11:20:09.914 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:09.915 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:09.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:09.977 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.361 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:10.038 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:10.040 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:10.040 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9481e-05 s, 0.0 kB/s 2023-08-27T11:20:10.040 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:10.041 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:10.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:10.099 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.362 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:10.163 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:10.163 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:10.164 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0697e-05 s, 0.0 kB/s 2023-08-27T11:20:10.164 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:10.165 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:10.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:10.225 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.363 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:10.285 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:10.285 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:10.285 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8089e-05 s, 0.0 kB/s 2023-08-27T11:20:10.286 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:10.286 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:10.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:10.345 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.364 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:10.406 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:10.406 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:10.406 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.683e-05 s, 0.0 kB/s 2023-08-27T11:20:10.407 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:10.407 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:10.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:10.465 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.365 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:10.523 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:10.523 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:10.523 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9243e-05 s, 0.0 kB/s 2023-08-27T11:20:10.524 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:10.524 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:10.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:10.588 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.366 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:10.647 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:10.647 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:10.647 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9707e-05 s, 0.0 kB/s 2023-08-27T11:20:10.648 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:10.648 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:10.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:10.707 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.367 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:10.766 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:10.766 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:10.766 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0277e-05 s, 0.0 kB/s 2023-08-27T11:20:10.767 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:10.767 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:10.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:10.824 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.368 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:10.880 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:10.881 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:10.881 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.236e-05 s, 0.0 kB/s 2023-08-27T11:20:10.882 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:10.882 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:10.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:10.939 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.369 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:10.995 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:10.996 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:10.996 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9873e-05 s, 0.0 kB/s 2023-08-27T11:20:10.996 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:10.997 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:11.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:11.054 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.370 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:11.110 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:11.111 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:11.111 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9941e-05 s, 0.0 kB/s 2023-08-27T11:20:11.111 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:11.112 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:11.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:11.168 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.371 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:11.227 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:11.227 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:11.227 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1784e-05 s, 0.0 kB/s 2023-08-27T11:20:11.228 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:11.228 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:11.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:11.285 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.372 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:11.342 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:11.343 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:11.343 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7083e-05 s, 0.0 kB/s 2023-08-27T11:20:11.344 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:11.344 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:11.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:11.401 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.373 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:11.458 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:11.459 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:11.459 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7976e-05 s, 0.0 kB/s 2023-08-27T11:20:11.460 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:11.460 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:11.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:11.517 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.374 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:11.574 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:11.574 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:11.575 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9039e-05 s, 0.0 kB/s 2023-08-27T11:20:11.575 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:11.575 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:11.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:11.631 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.375 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:11.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:11.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:11.689 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0428e-05 s, 0.0 kB/s 2023-08-27T11:20:11.690 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:11.690 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:11.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:11.747 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.376 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:11.803 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:11.803 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:11.804 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6532e-05 s, 0.0 kB/s 2023-08-27T11:20:11.804 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:11.805 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:11.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:11.862 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.377 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:11.919 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:11.919 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:11.919 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8807e-05 s, 0.0 kB/s 2023-08-27T11:20:11.920 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:11.920 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:11.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:11.977 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.378 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:12.034 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:12.034 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5577e-05 s, 0.0 kB/s 2023-08-27T11:20:12.035 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:12.035 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:12.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:12.092 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.379 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:12.149 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:12.150 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:12.150 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1849e-05 s, 0.0 kB/s 2023-08-27T11:20:12.150 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:12.151 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:12.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:12.207 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.380 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:12.267 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:12.267 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:12.268 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8484e-05 s, 0.0 kB/s 2023-08-27T11:20:12.268 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:12.268 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:12.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:12.326 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.381 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:12.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:12.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:12.384 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7155e-05 s, 0.0 kB/s 2023-08-27T11:20:12.384 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:12.385 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:12.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:12.441 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.382 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:12.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:12.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:12.500 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8933e-05 s, 0.0 kB/s 2023-08-27T11:20:12.501 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:12.501 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:12.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:12.557 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.383 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:12.615 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:12.615 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:12.615 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2022e-05 s, 0.0 kB/s 2023-08-27T11:20:12.615 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:12.616 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:12.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:12.675 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.384 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:12.732 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6383e-05 s, 0.0 kB/s 2023-08-27T11:20:12.732 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:12.732 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:12.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:12.788 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.385 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:12.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:12.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:12.845 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9932e-05 s, 0.0 kB/s 2023-08-27T11:20:12.846 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:12.846 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:12.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:12.902 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.386 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:12.960 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:12.960 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9126e-05 s, 0.0 kB/s 2023-08-27T11:20:12.960 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:12.961 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:13.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:13.018 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.387 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:13.074 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:13.074 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:13.075 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7761e-05 s, 0.0 kB/s 2023-08-27T11:20:13.075 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:13.075 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:13.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:13.132 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.388 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:13.188 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:13.188 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:13.188 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5748e-05 s, 0.0 kB/s 2023-08-27T11:20:13.189 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:13.190 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:13.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:13.246 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.389 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:13.305 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:13.305 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:13.305 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6319e-05 s, 0.0 kB/s 2023-08-27T11:20:13.306 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:13.306 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:13.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:13.364 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.390 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:13.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:13.421 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:13.421 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7494e-05 s, 0.0 kB/s 2023-08-27T11:20:13.421 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:13.422 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:13.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:13.478 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.391 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:13.535 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:13.535 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:13.535 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9314e-05 s, 0.0 kB/s 2023-08-27T11:20:13.536 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:13.536 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:13.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:13.593 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.392 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:13.649 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:13.650 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:13.650 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1453e-05 s, 0.0 kB/s 2023-08-27T11:20:13.650 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:13.651 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:13.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:13.707 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.393 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:13.763 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:13.764 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:13.764 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2107e-05 s, 0.0 kB/s 2023-08-27T11:20:13.764 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:13.765 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:13.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:13.822 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.394 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:13.878 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:13.878 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:13.878 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9167e-05 s, 0.0 kB/s 2023-08-27T11:20:13.879 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:13.879 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:13.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:13.935 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.395 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:13.992 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:13.992 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:13.993 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.72e-05 s, 0.0 kB/s 2023-08-27T11:20:13.993 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:13.993 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:14.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:14.050 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.396 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:14.109 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:14.109 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:14.109 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.895e-05 s, 0.0 kB/s 2023-08-27T11:20:14.110 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:14.110 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:14.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:14.167 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.397 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:14.224 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:14.225 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:14.225 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6786e-05 s, 0.0 kB/s 2023-08-27T11:20:14.225 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:14.226 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:14.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:14.283 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.398 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:14.340 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:14.340 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8788e-05 s, 0.0 kB/s 2023-08-27T11:20:14.341 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:14.341 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:14.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:14.397 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.399 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:14.455 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:14.455 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1654e-05 s, 0.0 kB/s 2023-08-27T11:20:14.456 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:14.456 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:14.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:14.513 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.400 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:14.569 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:14.570 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:14.570 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8278e-05 s, 0.0 kB/s 2023-08-27T11:20:14.570 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:14.570 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:14.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:14.657 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.401 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:14.684 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:14.685 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:14.685 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0959e-05 s, 0.0 kB/s 2023-08-27T11:20:14.685 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:14.686 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:14.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:14.743 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.402 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:14.800 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:14.801 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:14.801 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.294e-05 s, 0.0 kB/s 2023-08-27T11:20:14.802 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:14.802 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:14.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:14.859 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.403 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:14.915 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:14.915 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:14.916 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7365e-05 s, 0.0 kB/s 2023-08-27T11:20:14.916 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:14.916 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:14.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:14.973 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.404 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:15.029 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:15.029 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:15.029 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7492e-05 s, 0.0 kB/s 2023-08-27T11:20:15.030 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:15.030 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:15.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:15.086 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.405 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:15.143 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:15.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:15.144 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0703e-05 s, 0.0 kB/s 2023-08-27T11:20:15.144 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:15.145 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:15.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:15.201 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.406 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:15.257 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:15.257 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:15.258 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7565e-05 s, 0.0 kB/s 2023-08-27T11:20:15.258 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:15.258 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:15.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:15.315 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.407 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2698e-05 s, 0.0 kB/s 2023-08-27T11:20:15.374 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:15.374 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:15.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:15.432 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.408 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0704e-05 s, 0.0 kB/s 2023-08-27T11:20:15.489 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:15.490 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:15.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:15.547 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.409 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:15.604 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:15.604 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:15.605 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7157e-05 s, 0.0 kB/s 2023-08-27T11:20:15.605 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:15.606 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:15.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:15.662 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.410 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:15.720 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:15.720 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:15.720 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9173e-05 s, 0.0 kB/s 2023-08-27T11:20:15.720 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:15.721 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:15.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:15.779 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.411 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:15.835 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:15.835 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:15.836 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0205e-05 s, 0.0 kB/s 2023-08-27T11:20:15.836 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:15.836 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:15.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:15.893 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.412 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:15.950 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:15.950 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:15.950 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0036e-05 s, 0.0 kB/s 2023-08-27T11:20:15.951 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:15.951 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:16.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:16.008 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.413 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:16.066 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:16.066 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:16.066 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.051e-05 s, 0.0 kB/s 2023-08-27T11:20:16.067 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:16.067 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:16.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:16.124 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.414 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6023e-05 s, 0.0 kB/s 2023-08-27T11:20:16.183 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:16.183 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:16.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:16.240 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.415 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8921e-05 s, 0.0 kB/s 2023-08-27T11:20:16.298 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:16.298 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:16.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:16.355 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.416 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:16.412 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:16.413 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:16.413 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8082e-05 s, 0.0 kB/s 2023-08-27T11:20:16.413 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:16.414 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:16.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:16.471 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.417 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:16.527 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:16.528 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:16.528 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0797e-05 s, 0.0 kB/s 2023-08-27T11:20:16.529 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:16.529 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:16.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:16.585 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.418 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:16.642 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:16.643 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:16.643 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.732e-05 s, 0.0 kB/s 2023-08-27T11:20:16.643 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:16.644 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:16.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:16.702 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.419 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:16.758 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:16.759 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:16.759 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7157e-05 s, 0.0 kB/s 2023-08-27T11:20:16.759 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:16.760 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:16.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:16.816 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.420 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8395e-05 s, 0.0 kB/s 2023-08-27T11:20:16.874 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:16.874 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:16.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:16.931 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.421 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:16.987 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:16.988 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:16.988 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6602e-05 s, 0.0 kB/s 2023-08-27T11:20:16.988 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:16.989 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:17.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:17.045 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.422 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:17.103 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:17.103 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:17.103 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6372e-05 s, 0.0 kB/s 2023-08-27T11:20:17.104 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:17.104 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:17.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:17.161 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.423 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:17.218 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:17.218 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:17.218 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8999e-05 s, 0.0 kB/s 2023-08-27T11:20:17.219 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:17.219 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:17.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:17.276 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.424 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:17.332 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:17.332 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:17.332 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9856e-05 s, 0.0 kB/s 2023-08-27T11:20:17.333 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:17.333 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:17.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:17.391 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.425 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:17.448 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:17.448 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:17.449 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8683e-05 s, 0.0 kB/s 2023-08-27T11:20:17.449 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:17.449 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:17.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:17.507 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.426 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:17.565 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:17.565 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:17.566 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9713e-05 s, 0.0 kB/s 2023-08-27T11:20:17.566 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:17.567 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:17.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:17.623 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.427 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:17.681 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1476e-05 s, 0.0 kB/s 2023-08-27T11:20:17.681 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:17.681 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:17.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:17.738 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.428 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:17.796 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:17.797 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:17.797 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1013e-05 s, 0.0 kB/s 2023-08-27T11:20:17.801 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:17.801 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:17.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:17.855 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.429 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:17.911 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:17.912 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:17.912 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1437e-05 s, 0.0 kB/s 2023-08-27T11:20:17.912 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:17.913 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:17.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:17.969 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.430 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.5278e-05 s, 0.0 kB/s 2023-08-27T11:20:18.026 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:18.027 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:18.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:18.084 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.431 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:18.140 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:18.141 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:18.141 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.1695e-05 s, 0.0 kB/s 2023-08-27T11:20:18.141 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:18.142 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:18.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:18.201 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.432 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:18.258 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:18.259 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:18.259 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8767e-05 s, 0.0 kB/s 2023-08-27T11:20:18.260 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:18.260 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:18.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:18.319 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.433 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:18.377 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:18.377 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:18.377 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9621e-05 s, 0.0 kB/s 2023-08-27T11:20:18.378 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:18.379 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:18.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:18.435 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.434 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:18.493 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:18.493 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:18.493 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8179e-05 s, 0.0 kB/s 2023-08-27T11:20:18.494 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:18.494 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:18.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:18.551 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.435 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:18.607 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:18.608 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:18.608 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7567e-05 s, 0.0 kB/s 2023-08-27T11:20:18.608 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:18.609 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:18.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:18.666 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.436 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9917e-05 s, 0.0 kB/s 2023-08-27T11:20:18.723 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:18.724 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:18.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:18.780 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.437 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:18.838 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.873e-05 s, 0.0 kB/s 2023-08-27T11:20:18.839 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:18.840 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:18.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:18.899 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.438 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:18.956 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:18.956 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9526e-05 s, 0.0 kB/s 2023-08-27T11:20:18.957 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:18.957 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:19.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:19.015 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.439 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:19.072 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:19.072 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:19.072 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.978e-05 s, 0.0 kB/s 2023-08-27T11:20:19.073 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:19.073 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:19.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:19.130 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.440 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:19.187 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:19.188 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:19.188 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9229e-05 s, 0.0 kB/s 2023-08-27T11:20:19.188 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:19.189 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:19.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:19.246 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.441 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:19.303 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:19.304 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:19.304 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8801e-05 s, 0.0 kB/s 2023-08-27T11:20:19.304 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:19.305 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:19.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:19.363 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.442 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:19.419 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.891e-05 s, 0.0 kB/s 2023-08-27T11:20:19.420 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:19.421 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:19.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:19.478 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.443 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:19.538 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:19.538 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:19.539 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0637e-05 s, 0.0 kB/s 2023-08-27T11:20:19.539 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:19.540 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:19.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:19.595 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.444 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:19.652 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:19.652 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:19.652 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.862e-05 s, 0.0 kB/s 2023-08-27T11:20:19.653 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:19.653 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:19.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:19.711 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.445 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:19.767 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:19.767 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8201e-05 s, 0.0 kB/s 2023-08-27T11:20:19.768 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:19.769 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:19.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:19.825 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.446 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:19.882 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:19.882 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:19.883 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0418e-05 s, 0.0 kB/s 2023-08-27T11:20:19.883 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:19.883 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:19.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:19.940 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.447 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:19.998 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:19.998 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:19.998 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.235e-05 s, 0.0 kB/s 2023-08-27T11:20:19.998 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:19.999 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:20.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:20.057 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.448 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:20.113 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:20.114 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:20.114 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1292e-05 s, 0.0 kB/s 2023-08-27T11:20:20.114 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:20.115 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:20.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:20.172 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.449 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:20.228 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:20.228 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:20.229 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0726e-05 s, 0.0 kB/s 2023-08-27T11:20:20.229 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:20.229 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:20.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:20.287 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.450 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2996e-05 s, 0.0 kB/s 2023-08-27T11:20:20.345 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:20.345 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:20.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:20.402 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.451 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:20.459 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:20.459 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:20.459 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0817e-05 s, 0.0 kB/s 2023-08-27T11:20:20.460 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:20.460 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:20.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:20.517 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.452 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:20.574 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0798e-05 s, 0.0 kB/s 2023-08-27T11:20:20.575 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:20.576 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:20.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:20.632 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.453 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:20.690 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:20.690 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:20.690 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1347e-05 s, 0.0 kB/s 2023-08-27T11:20:20.691 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:20.691 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:20.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:20.748 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.454 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:20.806 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:20.806 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:20.806 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2532e-05 s, 0.0 kB/s 2023-08-27T11:20:20.807 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:20.807 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:20.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:20.867 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.455 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:20.923 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:20.923 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:20.923 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2419e-05 s, 0.0 kB/s 2023-08-27T11:20:20.924 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:20.924 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:20.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:20.982 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.456 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.5769e-05 s, 0.0 kB/s 2023-08-27T11:20:21.039 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:21.040 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:21.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:21.098 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.457 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0012e-05 s, 0.0 kB/s 2023-08-27T11:20:21.155 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:21.156 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:21.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:21.213 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.458 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2697e-05 s, 0.0 kB/s 2023-08-27T11:20:21.272 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:21.272 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:21.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:21.329 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.459 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:21.387 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:21.387 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:21.388 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9355e-05 s, 0.0 kB/s 2023-08-27T11:20:21.388 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:21.389 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:21.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:21.445 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.460 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:21.502 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:21.502 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:21.502 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8899e-05 s, 0.0 kB/s 2023-08-27T11:20:21.503 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:21.503 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:21.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:21.560 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.461 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:21.617 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:21.617 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:21.617 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0651e-05 s, 0.0 kB/s 2023-08-27T11:20:21.618 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:21.618 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:21.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:21.675 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.462 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:21.732 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:21.732 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:21.732 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0454e-05 s, 0.0 kB/s 2023-08-27T11:20:21.732 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:21.733 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:21.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:21.790 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.463 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:21.847 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:21.847 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:21.848 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.15e-05 s, 0.0 kB/s 2023-08-27T11:20:21.848 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:21.848 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:21.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:21.905 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.464 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:21.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:21.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:21.963 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9163e-05 s, 0.0 kB/s 2023-08-27T11:20:21.963 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:21.963 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:22.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:22.020 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.465 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:22.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:22.078 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:22.078 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6605e-05 s, 0.0 kB/s 2023-08-27T11:20:22.079 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:22.079 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:22.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:22.136 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.466 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:22.194 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:22.194 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:22.195 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7972e-05 s, 0.0 kB/s 2023-08-27T11:20:22.195 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:22.195 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:22.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:22.252 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.467 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:22.308 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:22.309 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:22.309 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0145e-05 s, 0.0 kB/s 2023-08-27T11:20:22.310 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:22.310 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:22.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:22.367 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.468 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:22.424 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:22.424 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8199e-05 s, 0.0 kB/s 2023-08-27T11:20:22.425 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:22.425 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:22.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:22.482 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.469 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:22.538 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:22.539 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:22.539 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9044e-05 s, 0.0 kB/s 2023-08-27T11:20:22.539 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:22.540 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:22.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:22.597 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.470 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:22.653 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:22.654 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:22.654 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9635e-05 s, 0.0 kB/s 2023-08-27T11:20:22.654 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:22.655 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:22.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:22.714 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.471 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:22.772 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6039e-05 s, 0.0 kB/s 2023-08-27T11:20:22.772 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:22.772 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:22.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:22.829 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.472 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:22.886 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:22.887 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:22.887 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9382e-05 s, 0.0 kB/s 2023-08-27T11:20:22.887 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:22.888 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:22.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:22.945 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.473 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:23.001 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:23.002 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:23.002 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0564e-05 s, 0.0 kB/s 2023-08-27T11:20:23.003 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:23.003 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:23.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:23.060 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.474 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:23.118 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:23.118 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:23.118 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8589e-05 s, 0.0 kB/s 2023-08-27T11:20:23.120 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:23.120 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:23.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:23.175 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.475 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:23.232 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9139e-05 s, 0.0 kB/s 2023-08-27T11:20:23.233 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:23.234 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:23.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:23.291 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.476 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4394e-05 s, 0.0 kB/s 2023-08-27T11:20:23.349 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:23.350 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:23.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:23.406 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.477 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:23.463 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:23.464 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:23.464 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7553e-05 s, 0.0 kB/s 2023-08-27T11:20:23.464 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:23.465 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:23.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:23.521 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.478 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:23.577 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:23.578 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:23.578 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9055e-05 s, 0.0 kB/s 2023-08-27T11:20:23.578 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:23.579 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:23.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:23.636 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.479 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:23.694 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:23.695 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:23.695 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3445e-05 s, 0.0 kB/s 2023-08-27T11:20:23.695 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:23.696 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:23.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:23.752 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.480 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:23.809 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:23.810 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:23.810 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8622e-05 s, 0.0 kB/s 2023-08-27T11:20:23.810 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:23.811 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:23.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:23.868 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.481 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:23.924 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8998e-05 s, 0.0 kB/s 2023-08-27T11:20:23.926 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:23.926 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:23.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:23.981 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.482 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:24.038 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:24.038 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:24.039 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9392e-05 s, 0.0 kB/s 2023-08-27T11:20:24.039 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:24.039 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:24.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:24.096 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.483 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7531e-05 s, 0.0 kB/s 2023-08-27T11:20:24.155 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:24.155 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:24.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:24.212 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.484 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:24.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:24.270 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:24.270 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7528e-05 s, 0.0 kB/s 2023-08-27T11:20:24.270 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:24.271 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:24.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:24.328 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.485 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:24.384 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:24.385 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:24.385 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7549e-05 s, 0.0 kB/s 2023-08-27T11:20:24.385 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:24.386 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:24.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:24.444 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.486 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:24.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:24.500 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:24.500 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7638e-05 s, 0.0 kB/s 2023-08-27T11:20:24.501 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:24.501 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:24.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:24.559 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.487 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:24.615 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:24.615 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:24.616 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9825e-05 s, 0.0 kB/s 2023-08-27T11:20:24.616 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:24.616 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:24.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:24.673 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.488 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:24.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:24.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7641e-05 s, 0.0 kB/s 2023-08-27T11:20:24.731 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:24.731 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:24.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:24.789 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.489 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:24.846 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:24.846 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:24.846 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0885e-05 s, 0.0 kB/s 2023-08-27T11:20:24.847 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:24.847 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:24.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:24.904 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.490 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:24.961 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:24.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:24.962 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8229e-05 s, 0.0 kB/s 2023-08-27T11:20:24.962 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:24.963 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:25.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:25.020 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.491 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:25.076 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:25.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:25.077 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0675e-05 s, 0.0 kB/s 2023-08-27T11:20:25.077 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:25.078 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:25.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:25.137 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.492 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:25.195 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:25.195 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6309e-05 s, 0.0 kB/s 2023-08-27T11:20:25.195 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:25.196 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:25.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:25.253 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.493 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:25.310 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:25.310 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:25.310 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7247e-05 s, 0.0 kB/s 2023-08-27T11:20:25.311 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:25.311 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:25.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:25.368 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.494 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:25.425 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:25.425 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:25.425 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8471e-05 s, 0.0 kB/s 2023-08-27T11:20:25.426 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:25.426 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:25.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:25.483 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.495 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:25.540 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7982e-05 s, 0.0 kB/s 2023-08-27T11:20:25.541 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:25.542 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:25.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:25.599 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.496 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:25.657 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:25.657 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:25.657 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0653e-05 s, 0.0 kB/s 2023-08-27T11:20:25.658 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:25.658 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:25.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:25.716 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.497 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:25.774 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:25.774 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:25.775 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0406e-05 s, 0.0 kB/s 2023-08-27T11:20:25.775 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:25.775 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:25.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:25.834 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.498 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:25.891 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:25.892 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:25.892 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2305e-05 s, 0.0 kB/s 2023-08-27T11:20:25.892 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:25.894 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:25.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:25.950 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.499 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6392e-05 s, 0.0 kB/s 2023-08-27T11:20:26.010 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:26.011 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:26.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:26.067 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.500 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:26.125 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:26.125 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:26.125 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1509e-05 s, 0.0 kB/s 2023-08-27T11:20:26.126 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:26.126 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:26.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:26.183 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.501 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:26.242 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:26.242 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:26.242 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.191e-05 s, 0.0 kB/s 2023-08-27T11:20:26.243 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:26.243 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:26.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:26.300 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.502 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:26.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:26.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:26.358 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9679e-05 s, 0.0 kB/s 2023-08-27T11:20:26.359 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:26.359 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:26.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:26.416 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.503 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:26.473 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:26.473 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:26.474 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7133e-05 s, 0.0 kB/s 2023-08-27T11:20:26.474 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:26.475 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:26.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:26.531 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.504 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:26.588 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:26.589 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:26.589 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7891e-05 s, 0.0 kB/s 2023-08-27T11:20:26.589 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:26.590 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:26.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:26.646 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.505 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:26.702 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:26.702 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:26.703 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1136e-05 s, 0.0 kB/s 2023-08-27T11:20:26.703 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:26.703 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:26.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:26.759 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.506 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:26.817 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:26.818 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:26.818 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8189e-05 s, 0.0 kB/s 2023-08-27T11:20:26.818 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:26.819 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:26.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:26.876 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.507 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:26.933 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:26.934 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:26.934 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1444e-05 s, 0.0 kB/s 2023-08-27T11:20:26.934 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:26.935 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:26.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:26.992 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.508 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:27.049 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:27.049 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:27.050 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0942e-05 s, 0.0 kB/s 2023-08-27T11:20:27.050 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:27.050 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:27.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:27.107 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.509 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:27.164 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:27.164 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:27.164 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0144e-05 s, 0.0 kB/s 2023-08-27T11:20:27.165 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:27.165 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:27.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:27.221 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.510 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:27.279 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:27.279 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:27.280 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.129e-05 s, 0.0 kB/s 2023-08-27T11:20:27.280 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:27.280 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:27.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:27.337 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.511 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:27.394 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:27.394 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:27.394 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0402e-05 s, 0.0 kB/s 2023-08-27T11:20:27.395 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:27.395 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:27.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:27.452 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.512 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:27.508 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0537e-05 s, 0.0 kB/s 2023-08-27T11:20:27.510 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:27.510 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:27.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:27.566 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.513 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:27.623 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:27.623 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:27.623 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.653e-05 s, 0.0 kB/s 2023-08-27T11:20:27.624 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:27.624 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:27.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:27.681 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.514 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0617e-05 s, 0.0 kB/s 2023-08-27T11:20:27.739 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:27.739 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:27.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:27.796 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.515 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:27.854 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:27.854 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:27.855 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2034e-05 s, 0.0 kB/s 2023-08-27T11:20:27.855 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:27.855 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:27.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:27.912 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.516 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:27.970 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:27.970 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:27.970 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0166e-05 s, 0.0 kB/s 2023-08-27T11:20:27.970 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:27.971 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:28.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:28.028 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.517 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:28.086 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:28.086 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:28.086 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0994e-05 s, 0.0 kB/s 2023-08-27T11:20:28.087 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:28.087 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:28.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:28.144 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.518 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:28.202 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:28.202 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:28.203 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8231e-05 s, 0.0 kB/s 2023-08-27T11:20:28.203 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:28.204 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:28.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:28.261 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.519 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7624e-05 s, 0.0 kB/s 2023-08-27T11:20:28.319 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:28.319 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:28.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:28.376 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.520 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:28.433 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:28.434 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:28.434 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8178e-05 s, 0.0 kB/s 2023-08-27T11:20:28.434 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:28.435 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:28.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:28.492 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.521 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:28.550 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6929e-05 s, 0.0 kB/s 2023-08-27T11:20:28.551 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:28.552 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:28.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:28.608 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.522 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:28.665 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:28.666 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:28.666 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0995e-05 s, 0.0 kB/s 2023-08-27T11:20:28.667 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:28.667 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:28.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:28.724 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.523 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7777e-05 s, 0.0 kB/s 2023-08-27T11:20:28.782 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:28.783 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:28.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:28.839 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.524 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:28.897 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:28.897 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:28.897 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0778e-05 s, 0.0 kB/s 2023-08-27T11:20:28.898 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:28.898 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:28.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:28.956 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.525 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:29.012 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:29.013 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:29.013 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.939e-05 s, 0.0 kB/s 2023-08-27T11:20:29.013 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:29.014 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:29.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:29.070 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.526 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:29.127 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:29.127 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:29.128 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2015e-05 s, 0.0 kB/s 2023-08-27T11:20:29.128 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:29.128 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:29.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:29.185 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.527 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:29.242 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:29.242 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:29.243 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8094e-05 s, 0.0 kB/s 2023-08-27T11:20:29.243 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:29.243 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:29.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:29.300 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.528 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:29.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:29.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:29.359 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7051e-05 s, 0.0 kB/s 2023-08-27T11:20:29.359 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:29.359 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:29.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:29.416 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.529 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:29.472 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:29.473 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:29.473 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1576e-05 s, 0.0 kB/s 2023-08-27T11:20:29.474 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:29.474 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:29.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:29.530 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.530 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:29.586 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:29.587 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:29.587 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8132e-05 s, 0.0 kB/s 2023-08-27T11:20:29.587 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:29.588 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:29.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:29.645 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.531 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:29.703 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:29.703 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:29.703 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.79e-05 s, 0.0 kB/s 2023-08-27T11:20:29.704 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:29.704 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:29.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:29.763 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.532 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:29.820 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:29.820 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:29.820 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8165e-05 s, 0.0 kB/s 2023-08-27T11:20:29.821 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:29.821 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:29.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:29.880 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.533 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:29.938 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6022e-05 s, 0.0 kB/s 2023-08-27T11:20:29.938 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:29.939 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:29.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:29.995 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.534 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:30.053 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:30.053 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:30.053 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0088e-05 s, 0.0 kB/s 2023-08-27T11:20:30.054 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:30.054 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:30.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:30.111 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.535 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:30.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:30.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:30.169 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8466e-05 s, 0.0 kB/s 2023-08-27T11:20:30.170 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:30.170 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:30.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:30.227 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.536 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:30.285 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:30.285 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:30.286 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8801e-05 s, 0.0 kB/s 2023-08-27T11:20:30.286 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:30.288 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:30.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:30.345 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.537 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:30.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:30.402 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:30.402 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.851e-05 s, 0.0 kB/s 2023-08-27T11:20:30.403 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:30.403 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:30.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:30.461 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.538 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:30.518 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:30.518 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:30.518 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.5478e-05 s, 0.0 kB/s 2023-08-27T11:20:30.519 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:30.520 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:30.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:30.576 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.539 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:30.632 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:30.633 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:30.633 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9338e-05 s, 0.0 kB/s 2023-08-27T11:20:30.633 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:30.634 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:30.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:30.691 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.540 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:30.748 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:30.749 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:30.749 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8351e-05 s, 0.0 kB/s 2023-08-27T11:20:30.749 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:30.750 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:30.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:30.807 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.541 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:30.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:30.865 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:30.865 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9346e-05 s, 0.0 kB/s 2023-08-27T11:20:30.865 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:30.866 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:30.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:30.923 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.542 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:30.981 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:30.981 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:30.982 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9016e-05 s, 0.0 kB/s 2023-08-27T11:20:30.982 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:30.983 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:31.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:31.039 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.543 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:31.097 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:31.097 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:31.097 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.748e-05 s, 0.0 kB/s 2023-08-27T11:20:31.098 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:31.098 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:31.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:31.157 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.544 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:31.213 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:31.214 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:31.214 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9081e-05 s, 0.0 kB/s 2023-08-27T11:20:31.215 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:31.216 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:31.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:31.274 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.545 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:31.331 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1215e-05 s, 0.0 kB/s 2023-08-27T11:20:31.333 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:31.333 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:31.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:31.389 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.546 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:31.446 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:31.446 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:31.446 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.108e-05 s, 0.0 kB/s 2023-08-27T11:20:31.447 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:31.447 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:31.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:31.503 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.547 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9392e-05 s, 0.0 kB/s 2023-08-27T11:20:31.562 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:31.562 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:31.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:31.617 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.548 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9409e-05 s, 0.0 kB/s 2023-08-27T11:20:31.675 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:31.676 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:31.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:31.733 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.549 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:31.789 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:31.790 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:31.790 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.124e-05 s, 0.0 kB/s 2023-08-27T11:20:31.790 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:31.791 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:31.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:31.848 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.550 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:31.905 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:31.905 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:31.905 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7824e-05 s, 0.0 kB/s 2023-08-27T11:20:31.906 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:31.906 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:31.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:31.962 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.551 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:32.020 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:32.020 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:32.021 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8325e-05 s, 0.0 kB/s 2023-08-27T11:20:32.021 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:32.021 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:32.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:32.079 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.552 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:32.136 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:32.136 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.973e-05 s, 0.0 kB/s 2023-08-27T11:20:32.136 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:32.137 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:32.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:32.194 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.553 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:32.251 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:32.251 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:32.251 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8902e-05 s, 0.0 kB/s 2023-08-27T11:20:32.252 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:32.252 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:32.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:32.310 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.554 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:32.367 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:32.367 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:32.367 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0326e-05 s, 0.0 kB/s 2023-08-27T11:20:32.368 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:32.368 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:32.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:32.425 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.555 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:32.481 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:32.482 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:32.482 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3102e-05 s, 0.0 kB/s 2023-08-27T11:20:32.482 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:32.483 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:32.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:32.541 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.556 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:32.597 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:32.598 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:32.598 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7828e-05 s, 0.0 kB/s 2023-08-27T11:20:32.598 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:32.599 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:32.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:32.656 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.557 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:32.713 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:32.713 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:32.713 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8667e-05 s, 0.0 kB/s 2023-08-27T11:20:32.714 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:32.714 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:32.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:32.771 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.558 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:32.828 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:32.829 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:32.829 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1868e-05 s, 0.0 kB/s 2023-08-27T11:20:32.829 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:32.830 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:32.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:32.887 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.559 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:32.943 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:32.943 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:32.944 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1543e-05 s, 0.0 kB/s 2023-08-27T11:20:32.944 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:32.944 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:33.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:33.001 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.560 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:33.061 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1364e-05 s, 0.0 kB/s 2023-08-27T11:20:33.061 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:33.061 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:33.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:33.118 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.561 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:33.175 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:33.176 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:33.176 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9201e-05 s, 0.0 kB/s 2023-08-27T11:20:33.176 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:33.177 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:33.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:33.232 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.562 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:33.289 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:33.289 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:33.290 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7397e-05 s, 0.0 kB/s 2023-08-27T11:20:33.290 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:33.290 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:33.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:33.347 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.563 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:33.403 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:33.404 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:33.404 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6846e-05 s, 0.0 kB/s 2023-08-27T11:20:33.404 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:33.405 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:33.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:33.462 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.564 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:33.520 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:33.520 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:33.520 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9098e-05 s, 0.0 kB/s 2023-08-27T11:20:33.521 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:33.521 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:33.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:33.578 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.565 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:33.636 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:33.636 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9913e-05 s, 0.0 kB/s 2023-08-27T11:20:33.636 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:33.637 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:33.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:33.694 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.566 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:33.751 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:33.751 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:33.751 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7424e-05 s, 0.0 kB/s 2023-08-27T11:20:33.752 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:33.752 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:33.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:33.810 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.567 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:33.867 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:33.868 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:33.868 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6156e-05 s, 0.0 kB/s 2023-08-27T11:20:33.869 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:33.869 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:33.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:33.927 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.568 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:33.983 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:33.984 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:33.984 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7948e-05 s, 0.0 kB/s 2023-08-27T11:20:33.984 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:33.985 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:34.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:34.042 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.569 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:34.100 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:34.100 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:34.100 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.855e-05 s, 0.0 kB/s 2023-08-27T11:20:34.100 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:34.101 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:34.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:34.159 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.570 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:34.216 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:34.216 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:34.216 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6958e-05 s, 0.0 kB/s 2023-08-27T11:20:34.217 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:34.217 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:34.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:34.276 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.571 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:34.332 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:34.333 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:34.333 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8115e-05 s, 0.0 kB/s 2023-08-27T11:20:34.334 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:34.334 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:34.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:34.391 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.572 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:34.448 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:34.449 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:34.449 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.082e-05 s, 0.0 kB/s 2023-08-27T11:20:34.449 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:34.451 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:34.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:34.507 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.573 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:34.564 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:34.565 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:34.565 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0667e-05 s, 0.0 kB/s 2023-08-27T11:20:34.566 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:34.566 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:34.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:34.623 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.574 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:34.679 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:34.680 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:34.680 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7338e-05 s, 0.0 kB/s 2023-08-27T11:20:34.680 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:34.681 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:34.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:34.738 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.575 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:34.795 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:34.796 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:34.796 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8209e-05 s, 0.0 kB/s 2023-08-27T11:20:34.797 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:34.797 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:34.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:34.854 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.576 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:34.911 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:34.911 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:34.912 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8773e-05 s, 0.0 kB/s 2023-08-27T11:20:34.912 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:34.913 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:34.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:34.970 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.577 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:35.026 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:35.027 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:35.027 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7912e-05 s, 0.0 kB/s 2023-08-27T11:20:35.027 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:35.028 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:35.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:35.086 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.578 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:35.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:35.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:35.144 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9115e-05 s, 0.0 kB/s 2023-08-27T11:20:35.145 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:35.145 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:35.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:35.209 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.579 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:35.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:35.270 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:35.270 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7879e-05 s, 0.0 kB/s 2023-08-27T11:20:35.270 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:35.271 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:35.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:35.331 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.580 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:35.368 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:35.368 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5974e-05 s, 0.0 kB/s 2023-08-27T11:20:35.368 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:35.369 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:35.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:35.430 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.581 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:35.489 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:35.489 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:35.490 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4955e-05 s, 0.0 kB/s 2023-08-27T11:20:35.490 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:35.490 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:35.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:35.550 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.582 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:35.611 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:35.612 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:35.612 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.859e-05 s, 0.0 kB/s 2023-08-27T11:20:35.612 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:35.613 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:35.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:35.669 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.583 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:35.733 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:35.733 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1743e-05 s, 0.0 kB/s 2023-08-27T11:20:35.733 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:35.734 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:35.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:35.796 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.584 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:35.857 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:35.857 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.117e-05 s, 0.0 kB/s 2023-08-27T11:20:35.858 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:35.859 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:35.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:35.920 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.585 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:35.957 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:20:35.967 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:20:35.980 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:35.980 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:35.980 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.015e-05 s, 0.0 kB/s 2023-08-27T11:20:35.981 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:35.982 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:36.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:36.077 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.586 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:36.134 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:36.135 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:36.135 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.654e-05 s, 0.0 kB/s 2023-08-27T11:20:36.135 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:36.136 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:36.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:36.194 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.587 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:36.256 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:36.256 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:36.256 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2953e-05 s, 0.0 kB/s 2023-08-27T11:20:36.257 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:36.257 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:36.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:36.315 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.588 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:36.372 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:36.372 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:36.372 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0424e-05 s, 0.0 kB/s 2023-08-27T11:20:36.373 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:36.373 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:36.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:36.431 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.589 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:36.490 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:36.490 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3305e-05 s, 0.0 kB/s 2023-08-27T11:20:36.491 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:36.491 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:36.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:36.550 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.590 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:36.610 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:36.610 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:36.610 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9267e-05 s, 0.0 kB/s 2023-08-27T11:20:36.610 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:36.611 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:36.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:36.668 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.591 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:36.724 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:36.725 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:36.725 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2011e-05 s, 0.0 kB/s 2023-08-27T11:20:36.725 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:36.726 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:36.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:36.782 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.592 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0897e-05 s, 0.0 kB/s 2023-08-27T11:20:36.840 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:36.840 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:36.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:36.897 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.593 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8085e-05 s, 0.0 kB/s 2023-08-27T11:20:36.955 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:36.955 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:37.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:37.012 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.594 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:37.069 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:37.069 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:37.069 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0405e-05 s, 0.0 kB/s 2023-08-27T11:20:37.070 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:37.070 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:37.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:37.127 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.595 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:37.183 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:37.184 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:37.184 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0077e-05 s, 0.0 kB/s 2023-08-27T11:20:37.184 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:37.185 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:37.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:37.242 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.596 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:37.300 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:37.300 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:37.300 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2223e-05 s, 0.0 kB/s 2023-08-27T11:20:37.301 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:37.301 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:37.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:37.359 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.597 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:37.416 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:37.417 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:37.417 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1305e-05 s, 0.0 kB/s 2023-08-27T11:20:37.417 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:37.418 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:37.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:37.473 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.598 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:37.530 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:37.531 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:37.531 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0886e-05 s, 0.0 kB/s 2023-08-27T11:20:37.531 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:37.532 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:37.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:37.588 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.599 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:37.646 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:37.646 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:37.646 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9864e-05 s, 0.0 kB/s 2023-08-27T11:20:37.647 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:37.647 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:37.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:37.704 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.600 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:37.761 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:37.762 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:37.762 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9359e-05 s, 0.0 kB/s 2023-08-27T11:20:37.763 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:37.763 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:37.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:37.820 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.601 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:37.877 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:37.877 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:37.877 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0522e-05 s, 0.0 kB/s 2023-08-27T11:20:37.878 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:37.878 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:37.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:37.934 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.602 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:37.992 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:37.992 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1895e-05 s, 0.0 kB/s 2023-08-27T11:20:37.993 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:37.994 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:38.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:38.050 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.603 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:38.106 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:38.107 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:38.107 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.871e-05 s, 0.0 kB/s 2023-08-27T11:20:38.107 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:38.108 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:38.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:38.165 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.604 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:38.222 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:38.222 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:38.223 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.197e-05 s, 0.0 kB/s 2023-08-27T11:20:38.223 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:38.224 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:38.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:38.280 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.605 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:38.337 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:38.338 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:38.338 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0593e-05 s, 0.0 kB/s 2023-08-27T11:20:38.338 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:38.339 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:38.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:38.396 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.606 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:38.453 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:38.453 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:38.454 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0921e-05 s, 0.0 kB/s 2023-08-27T11:20:38.454 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:38.455 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:38.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:38.512 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.607 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9039e-05 s, 0.0 kB/s 2023-08-27T11:20:38.569 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:38.570 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:38.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:38.626 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.608 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:38.683 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:38.683 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:38.684 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8358e-05 s, 0.0 kB/s 2023-08-27T11:20:38.684 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:38.685 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:38.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:38.741 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.609 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:38.798 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:38.798 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:38.798 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9568e-05 s, 0.0 kB/s 2023-08-27T11:20:38.799 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:38.799 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:38.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:38.856 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.610 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:38.914 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:38.915 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:38.915 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1252e-05 s, 0.0 kB/s 2023-08-27T11:20:38.915 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:38.916 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:38.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:38.977 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.611 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:39.030 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:39.031 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:39.031 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8801e-05 s, 0.0 kB/s 2023-08-27T11:20:39.031 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:39.032 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:39.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:39.090 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.612 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:39.146 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:39.147 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:39.147 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8037e-05 s, 0.0 kB/s 2023-08-27T11:20:39.147 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:39.148 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:39.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:39.204 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.613 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:39.262 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:39.262 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:39.262 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8161e-05 s, 0.0 kB/s 2023-08-27T11:20:39.263 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:39.263 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:39.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:39.320 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.614 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:39.377 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:39.377 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:39.378 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8218e-05 s, 0.0 kB/s 2023-08-27T11:20:39.379 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:39.379 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:39.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:39.438 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.615 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:39.496 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:39.496 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:39.496 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0328e-05 s, 0.0 kB/s 2023-08-27T11:20:39.497 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:39.497 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:39.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:39.555 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.616 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1699e-05 s, 0.0 kB/s 2023-08-27T11:20:39.611 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:39.612 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:39.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:39.668 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.617 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:39.725 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:39.726 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:39.726 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9452e-05 s, 0.0 kB/s 2023-08-27T11:20:39.726 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:39.727 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:39.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:39.783 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.618 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:39.839 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:39.840 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:39.840 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.874e-05 s, 0.0 kB/s 2023-08-27T11:20:39.840 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:39.841 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:39.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:39.897 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.619 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:39.955 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:39.955 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:39.955 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9987e-05 s, 0.0 kB/s 2023-08-27T11:20:39.956 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:39.956 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:40.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:40.012 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.620 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8266e-05 s, 0.0 kB/s 2023-08-27T11:20:40.071 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:40.072 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:40.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:40.128 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.621 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:40.184 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:40.185 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:40.185 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4436e-05 s, 0.0 kB/s 2023-08-27T11:20:40.185 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:40.186 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:40.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:40.242 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.622 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:40.298 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:40.299 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:40.299 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9711e-05 s, 0.0 kB/s 2023-08-27T11:20:40.299 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:40.300 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:40.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:40.358 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.623 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:40.418 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:40.418 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:40.418 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2402e-05 s, 0.0 kB/s 2023-08-27T11:20:40.419 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:40.419 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:40.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:40.476 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.624 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:40.541 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0785e-05 s, 0.0 kB/s 2023-08-27T11:20:40.541 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:40.541 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:40.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:40.600 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.625 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:40.670 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:40.671 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:40.671 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0349e-05 s, 0.0 kB/s 2023-08-27T11:20:40.671 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:40.672 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:40.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:40.736 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.626 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:40.794 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:40.795 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:40.795 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.289e-05 s, 0.0 kB/s 2023-08-27T11:20:40.795 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:40.796 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:40.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:40.856 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.627 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:40.914 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:40.915 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:40.915 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0088e-05 s, 0.0 kB/s 2023-08-27T11:20:40.915 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:40.916 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:40.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:40.983 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.628 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:41.045 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:41.046 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:41.046 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.5759e-05 s, 0.0 kB/s 2023-08-27T11:20:41.046 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:41.046 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:41.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:41.116 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.629 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.68e-05 s, 0.0 kB/s 2023-08-27T11:20:41.182 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:41.182 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:41.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:41.249 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.630 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:41.309 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:41.310 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:41.310 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9675e-05 s, 0.0 kB/s 2023-08-27T11:20:41.310 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:41.311 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:41.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:41.375 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.631 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:41.437 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:41.437 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:41.437 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7616e-05 s, 0.0 kB/s 2023-08-27T11:20:41.438 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:41.438 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:41.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:41.497 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.632 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:41.515 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:41.515 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:41.515 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8524e-05 s, 0.0 kB/s 2023-08-27T11:20:41.516 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:41.516 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:41.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:41.574 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.633 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:41.632 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:41.632 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:41.633 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.878e-05 s, 0.0 kB/s 2023-08-27T11:20:41.633 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:41.634 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:41.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:41.691 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.634 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:41.749 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:41.749 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:41.749 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8055e-05 s, 0.0 kB/s 2023-08-27T11:20:41.750 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:41.750 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:41.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:41.811 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.635 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:41.871 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:41.871 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:41.872 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2887e-05 s, 0.0 kB/s 2023-08-27T11:20:41.872 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:41.874 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:41.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:41.933 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.636 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:41.993 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:41.994 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:41.994 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8068e-05 s, 0.0 kB/s 2023-08-27T11:20:41.995 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:41.995 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:42.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:42.053 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.637 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:42.118 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:42.118 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0644e-05 s, 0.0 kB/s 2023-08-27T11:20:42.119 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:42.120 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:42.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:42.175 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.638 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:42.231 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:42.232 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:42.232 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8107e-05 s, 0.0 kB/s 2023-08-27T11:20:42.232 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:42.233 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:42.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:42.290 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.639 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:42.348 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:42.348 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:42.348 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2422e-05 s, 0.0 kB/s 2023-08-27T11:20:42.349 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:42.349 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:42.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:42.407 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.640 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:42.465 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:42.465 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:42.465 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7235e-05 s, 0.0 kB/s 2023-08-27T11:20:42.466 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:42.466 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:42.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:42.523 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.641 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:42.580 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:42.580 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:42.580 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7117e-05 s, 0.0 kB/s 2023-08-27T11:20:42.581 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:42.581 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:42.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:42.639 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.642 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:42.695 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:42.695 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:42.696 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8397e-05 s, 0.0 kB/s 2023-08-27T11:20:42.696 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:42.696 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:42.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:42.755 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.643 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:42.813 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:42.814 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:42.814 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8582e-05 s, 0.0 kB/s 2023-08-27T11:20:42.814 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:42.815 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:42.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:42.872 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.644 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:42.928 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:42.928 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:42.928 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9372e-05 s, 0.0 kB/s 2023-08-27T11:20:42.929 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:42.929 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:42.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:42.986 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.645 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:43.043 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:43.043 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:43.044 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7031e-05 s, 0.0 kB/s 2023-08-27T11:20:43.044 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:43.045 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:43.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:43.101 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.646 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:43.158 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:43.158 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:43.158 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6536e-05 s, 0.0 kB/s 2023-08-27T11:20:43.159 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:43.159 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:43.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:43.216 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.647 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7487e-05 s, 0.0 kB/s 2023-08-27T11:20:43.274 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:43.274 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:43.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:43.331 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.648 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8729e-05 s, 0.0 kB/s 2023-08-27T11:20:43.388 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:43.389 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:43.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:43.447 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.649 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:43.504 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:43.504 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:43.505 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9375e-05 s, 0.0 kB/s 2023-08-27T11:20:43.505 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:43.505 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:43.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:43.562 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.650 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:43.620 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6207e-05 s, 0.0 kB/s 2023-08-27T11:20:43.620 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:43.620 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:43.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:43.676 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.651 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:43.734 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:43.734 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:43.735 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0587e-05 s, 0.0 kB/s 2023-08-27T11:20:43.735 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:43.736 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:43.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:43.792 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.652 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:43.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:43.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:43.850 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.066e-05 s, 0.0 kB/s 2023-08-27T11:20:43.851 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:43.851 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:43.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:43.908 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.653 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:43.966 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9746e-05 s, 0.0 kB/s 2023-08-27T11:20:43.966 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:43.966 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:44.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:44.023 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.654 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:44.080 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:44.080 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:44.081 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.87e-05 s, 0.0 kB/s 2023-08-27T11:20:44.081 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:44.081 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:44.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:44.138 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.655 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:44.195 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:44.196 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:44.196 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8988e-05 s, 0.0 kB/s 2023-08-27T11:20:44.196 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:44.196 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:44.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:44.253 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.656 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:44.311 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:44.311 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:44.311 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7357e-05 s, 0.0 kB/s 2023-08-27T11:20:44.312 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:44.312 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:44.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:44.369 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.657 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:44.426 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:44.426 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:44.426 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8521e-05 s, 0.0 kB/s 2023-08-27T11:20:44.427 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:44.427 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:44.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:44.484 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.658 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:44.542 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:44.542 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:44.543 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3027e-05 s, 0.0 kB/s 2023-08-27T11:20:44.543 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:44.543 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:44.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:44.602 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.659 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:44.662 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:44.662 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:44.662 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4731e-05 s, 0.0 kB/s 2023-08-27T11:20:44.663 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:44.663 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:44.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:44.719 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.660 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:44.776 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:44.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:44.777 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3501e-05 s, 0.0 kB/s 2023-08-27T11:20:44.777 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:44.778 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:44.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:44.834 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.661 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:44.890 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:44.891 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:44.891 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9424e-05 s, 0.0 kB/s 2023-08-27T11:20:44.892 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:44.892 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:44.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:44.949 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.662 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:45.007 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9769e-05 s, 0.0 kB/s 2023-08-27T11:20:45.007 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:45.007 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:45.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:45.064 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.663 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:45.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:45.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:45.121 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6997e-05 s, 0.0 kB/s 2023-08-27T11:20:45.121 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:45.121 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:45.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:45.177 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.664 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:45.234 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:45.234 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:45.235 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9631e-05 s, 0.0 kB/s 2023-08-27T11:20:45.235 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:45.236 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:45.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:45.292 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.665 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0048e-05 s, 0.0 kB/s 2023-08-27T11:20:45.350 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:45.350 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:45.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:45.408 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.666 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:45.464 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:45.465 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:45.465 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.001e-05 s, 0.0 kB/s 2023-08-27T11:20:45.465 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:45.466 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:45.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:45.523 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.667 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:45.580 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:45.580 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:45.580 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7958e-05 s, 0.0 kB/s 2023-08-27T11:20:45.581 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:45.581 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:45.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:45.638 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.668 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:45.695 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:45.695 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:45.695 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8647e-05 s, 0.0 kB/s 2023-08-27T11:20:45.696 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:45.696 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:45.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:45.756 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.669 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:45.814 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:45.814 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:45.815 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8826e-05 s, 0.0 kB/s 2023-08-27T11:20:45.815 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:45.816 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:45.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:45.873 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.670 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:45.930 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:45.930 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:45.931 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8301e-05 s, 0.0 kB/s 2023-08-27T11:20:45.931 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:45.931 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:45.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:45.988 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.671 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:46.044 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8215e-05 s, 0.0 kB/s 2023-08-27T11:20:46.045 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:46.046 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:46.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:46.102 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.672 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:46.158 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:46.159 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:46.159 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8819e-05 s, 0.0 kB/s 2023-08-27T11:20:46.159 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:46.160 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:46.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:46.217 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.673 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:46.275 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:46.276 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:46.276 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.155e-05 s, 0.0 kB/s 2023-08-27T11:20:46.276 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:46.277 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:46.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:46.333 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.674 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:46.390 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:46.390 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:46.390 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7619e-05 s, 0.0 kB/s 2023-08-27T11:20:46.391 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:46.391 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:46.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:46.448 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.675 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:46.505 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:46.505 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:46.506 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.65e-05 s, 0.0 kB/s 2023-08-27T11:20:46.506 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:46.506 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:46.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:46.563 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.676 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:46.620 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:46.621 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:46.621 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.205e-05 s, 0.0 kB/s 2023-08-27T11:20:46.622 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:46.622 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:46.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:46.678 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.677 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:46.736 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:46.736 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:46.736 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0429e-05 s, 0.0 kB/s 2023-08-27T11:20:46.737 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:46.738 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:46.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:46.795 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.678 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.901e-05 s, 0.0 kB/s 2023-08-27T11:20:46.852 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:46.852 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:46.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:46.909 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.679 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:46.966 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:46.966 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7091e-05 s, 0.0 kB/s 2023-08-27T11:20:46.966 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:46.967 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:47.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:47.024 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.680 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:47.081 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:47.081 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:47.082 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0945e-05 s, 0.0 kB/s 2023-08-27T11:20:47.082 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:47.082 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:47.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:47.139 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.681 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:47.198 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:47.198 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:47.198 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7538e-05 s, 0.0 kB/s 2023-08-27T11:20:47.199 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:47.199 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:47.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:47.255 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.682 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:47.313 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:47.313 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:47.313 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2347e-05 s, 0.0 kB/s 2023-08-27T11:20:47.314 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:47.314 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:47.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:47.371 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.683 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:47.427 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:47.428 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:47.428 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9169e-05 s, 0.0 kB/s 2023-08-27T11:20:47.429 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:47.429 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:47.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:47.488 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.684 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:47.545 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:47.546 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:47.546 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9794e-05 s, 0.0 kB/s 2023-08-27T11:20:47.546 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:47.547 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:47.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:47.604 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.685 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:47.662 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:47.662 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:47.663 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8653e-05 s, 0.0 kB/s 2023-08-27T11:20:47.663 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:47.664 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:47.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:47.720 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.686 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:47.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:47.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:47.778 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9472e-05 s, 0.0 kB/s 2023-08-27T11:20:47.778 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:47.778 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:47.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:47.839 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.687 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:47.897 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:47.897 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:47.897 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0609e-05 s, 0.0 kB/s 2023-08-27T11:20:47.898 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:47.898 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:47.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:47.955 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.688 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:48.011 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:48.011 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:48.012 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7129e-05 s, 0.0 kB/s 2023-08-27T11:20:48.012 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:48.012 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:48.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:48.069 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.689 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9631e-05 s, 0.0 kB/s 2023-08-27T11:20:48.126 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:48.126 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:48.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:48.183 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.690 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:48.239 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:48.240 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:48.240 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8622e-05 s, 0.0 kB/s 2023-08-27T11:20:48.240 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:48.241 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:48.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:48.298 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.691 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:48.355 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:48.355 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:48.355 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.104e-05 s, 0.0 kB/s 2023-08-27T11:20:48.356 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:48.356 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:48.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:48.415 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.692 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7438e-05 s, 0.0 kB/s 2023-08-27T11:20:48.471 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:48.472 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:48.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:48.528 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.693 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:48.586 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:48.586 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5222e-05 s, 0.0 kB/s 2023-08-27T11:20:48.586 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:48.587 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:48.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:48.644 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.694 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:48.701 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:48.702 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:48.702 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0241e-05 s, 0.0 kB/s 2023-08-27T11:20:48.703 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:48.703 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:48.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:48.759 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.695 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:48.816 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:48.816 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:48.816 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1953e-05 s, 0.0 kB/s 2023-08-27T11:20:48.817 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:48.818 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:48.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:48.875 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.696 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:48.932 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:48.932 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:48.932 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8409e-05 s, 0.0 kB/s 2023-08-27T11:20:48.933 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:48.933 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:48.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:48.990 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.697 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:49.048 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:49.048 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:49.048 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0929e-05 s, 0.0 kB/s 2023-08-27T11:20:49.049 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:49.049 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:49.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:49.106 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.698 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:49.163 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:49.163 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:49.163 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9949e-05 s, 0.0 kB/s 2023-08-27T11:20:49.164 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:49.164 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:49.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:49.222 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.699 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:49.279 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:49.280 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:49.280 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9737e-05 s, 0.0 kB/s 2023-08-27T11:20:49.280 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:49.281 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:49.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:49.336 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.700 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:49.394 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:49.395 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:49.395 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0801e-05 s, 0.0 kB/s 2023-08-27T11:20:49.395 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:49.396 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:49.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:49.453 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.701 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:49.509 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:49.510 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:49.510 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5633e-05 s, 0.0 kB/s 2023-08-27T11:20:49.510 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:49.511 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:49.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:49.569 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.702 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:49.627 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:49.627 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:49.627 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0347e-05 s, 0.0 kB/s 2023-08-27T11:20:49.628 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:49.628 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:49.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:49.685 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.703 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:49.742 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:49.742 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:49.742 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1661e-05 s, 0.0 kB/s 2023-08-27T11:20:49.743 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:49.743 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:49.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:49.801 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.704 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:49.857 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:49.858 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:49.858 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8384e-05 s, 0.0 kB/s 2023-08-27T11:20:49.858 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:49.859 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:49.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:49.917 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.705 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:49.975 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2875e-05 s, 0.0 kB/s 2023-08-27T11:20:49.975 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:49.976 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:50.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:50.033 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.706 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:50.089 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:50.090 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:50.090 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7478e-05 s, 0.0 kB/s 2023-08-27T11:20:50.090 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:50.091 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:50.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:50.149 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.707 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:50.206 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.676e-05 s, 0.0 kB/s 2023-08-27T11:20:50.206 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:50.206 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:50.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:50.263 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.708 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:50.319 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:50.319 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:50.320 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9489e-05 s, 0.0 kB/s 2023-08-27T11:20:50.320 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:50.320 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:50.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:50.377 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.709 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:50.434 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:50.434 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:50.435 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5995e-05 s, 0.0 kB/s 2023-08-27T11:20:50.435 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:50.435 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:50.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:50.492 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.710 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:50.550 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:50.550 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:50.550 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7643e-05 s, 0.0 kB/s 2023-08-27T11:20:50.551 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:50.551 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:50.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:50.608 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.711 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:50.664 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:50.664 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:50.665 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7839e-05 s, 0.0 kB/s 2023-08-27T11:20:50.665 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:50.665 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:50.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:50.723 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.712 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:50.784 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:50.784 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:50.784 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0661e-05 s, 0.0 kB/s 2023-08-27T11:20:50.785 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:50.785 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:50.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:50.842 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.713 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:50.899 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:50.900 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:50.900 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8475e-05 s, 0.0 kB/s 2023-08-27T11:20:50.900 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:50.901 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:50.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:50.958 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.714 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8411e-05 s, 0.0 kB/s 2023-08-27T11:20:51.016 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:51.016 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:51.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:51.073 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.715 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.875e-05 s, 0.0 kB/s 2023-08-27T11:20:51.132 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:51.132 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:51.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:51.188 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.716 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:51.245 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:51.246 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:51.246 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8615e-05 s, 0.0 kB/s 2023-08-27T11:20:51.246 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:51.247 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:51.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:51.306 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.717 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:51.363 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:51.363 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:51.364 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0129e-05 s, 0.0 kB/s 2023-08-27T11:20:51.364 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:51.365 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:51.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:51.423 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.718 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:51.486 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:51.486 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:51.486 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9796e-05 s, 0.0 kB/s 2023-08-27T11:20:51.487 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:51.487 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:51.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:51.545 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.719 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9105e-05 s, 0.0 kB/s 2023-08-27T11:20:51.604 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:51.605 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:51.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:51.661 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.720 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8325e-05 s, 0.0 kB/s 2023-08-27T11:20:51.719 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:51.719 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:51.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:51.777 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.721 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7102e-05 s, 0.0 kB/s 2023-08-27T11:20:51.835 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:51.836 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:51.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:51.892 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.722 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:51.949 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:51.950 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:51.950 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9506e-05 s, 0.0 kB/s 2023-08-27T11:20:51.950 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:51.951 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:52.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:52.011 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.723 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:52.067 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:52.067 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:52.068 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9585e-05 s, 0.0 kB/s 2023-08-27T11:20:52.068 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:52.069 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:52.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:52.125 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.724 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:52.182 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:52.182 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:52.182 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.749e-05 s, 0.0 kB/s 2023-08-27T11:20:52.182 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:52.183 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:52.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:52.240 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.725 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:52.321 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:52.322 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:52.322 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 6.1339e-05 s, 0.0 kB/s 2023-08-27T11:20:52.322 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:52.323 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:52.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:52.379 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.726 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:52.437 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:52.437 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:52.437 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9836e-05 s, 0.0 kB/s 2023-08-27T11:20:52.437 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:52.438 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:52.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:52.496 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.727 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:52.554 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:52.555 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:52.555 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7212e-05 s, 0.0 kB/s 2023-08-27T11:20:52.555 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:52.556 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:52.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:52.614 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.728 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7211e-05 s, 0.0 kB/s 2023-08-27T11:20:52.676 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:52.677 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:52.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:52.734 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.729 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:52.790 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:52.791 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:52.791 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8203e-05 s, 0.0 kB/s 2023-08-27T11:20:52.791 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:52.792 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:52.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:52.849 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.730 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:52.908 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:52.908 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:52.910 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7217e-05 s, 0.0 kB/s 2023-08-27T11:20:52.911 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:52.911 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:52.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:52.965 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.731 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:53.021 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:53.021 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:53.022 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9078e-05 s, 0.0 kB/s 2023-08-27T11:20:53.022 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:53.022 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:53.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:53.079 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.732 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8244e-05 s, 0.0 kB/s 2023-08-27T11:20:53.136 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:53.136 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:53.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:53.193 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.733 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:53.250 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:53.250 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:53.250 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9008e-05 s, 0.0 kB/s 2023-08-27T11:20:53.251 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:53.251 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:53.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:53.307 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.734 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:53.365 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:53.365 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:53.366 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9755e-05 s, 0.0 kB/s 2023-08-27T11:20:53.366 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:53.366 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:53.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:53.423 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.735 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:53.481 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:53.481 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:53.481 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7453e-05 s, 0.0 kB/s 2023-08-27T11:20:53.482 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:53.482 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:53.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:53.539 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.736 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:53.597 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:53.598 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:53.598 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7975e-05 s, 0.0 kB/s 2023-08-27T11:20:53.598 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:53.599 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:53.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:53.658 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.737 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:53.715 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:53.715 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:53.715 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9552e-05 s, 0.0 kB/s 2023-08-27T11:20:53.715 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:53.716 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:53.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:53.773 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.738 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:53.830 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:53.830 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:53.831 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8612e-05 s, 0.0 kB/s 2023-08-27T11:20:53.831 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:53.831 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:53.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:53.890 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.739 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:53.948 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:53.948 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:53.949 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8208e-05 s, 0.0 kB/s 2023-08-27T11:20:53.949 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:53.950 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:54.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:54.005 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.740 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:54.063 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:54.063 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.862e-05 s, 0.0 kB/s 2023-08-27T11:20:54.064 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:54.064 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:54.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:54.121 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.741 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:54.179 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0916e-05 s, 0.0 kB/s 2023-08-27T11:20:54.181 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:54.181 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:54.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:54.238 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.742 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:54.295 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:54.295 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:54.295 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8905e-05 s, 0.0 kB/s 2023-08-27T11:20:54.296 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:54.296 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:54.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:54.354 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.743 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:54.412 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:54.412 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:54.412 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9457e-05 s, 0.0 kB/s 2023-08-27T11:20:54.413 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:54.413 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:54.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:54.470 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.744 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:54.527 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:54.527 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:54.527 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.797e-05 s, 0.0 kB/s 2023-08-27T11:20:54.528 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:54.529 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:54.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:54.585 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.745 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:54.641 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:54.642 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:54.642 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9621e-05 s, 0.0 kB/s 2023-08-27T11:20:54.642 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:54.643 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:54.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:54.701 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.746 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:54.758 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:54.758 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:54.758 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8053e-05 s, 0.0 kB/s 2023-08-27T11:20:54.759 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:54.759 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:54.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:54.816 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.747 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:54.873 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:54.873 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:54.873 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.666e-05 s, 0.0 kB/s 2023-08-27T11:20:54.874 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:54.874 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:54.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:54.932 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.748 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:54.991 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:54.991 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:54.992 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8785e-05 s, 0.0 kB/s 2023-08-27T11:20:54.992 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:54.992 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:55.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:55.049 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.749 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:55.106 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:55.108 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:55.108 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1961e-05 s, 0.0 kB/s 2023-08-27T11:20:55.108 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:55.111 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:55.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:55.164 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.750 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:55.222 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:55.222 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:55.223 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.146e-05 s, 0.0 kB/s 2023-08-27T11:20:55.223 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:55.224 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:55.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:55.280 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.751 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:55.337 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:55.337 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:55.337 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9588e-05 s, 0.0 kB/s 2023-08-27T11:20:55.338 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:55.338 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:55.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:55.395 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.752 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:55.452 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:55.452 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:55.452 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0427e-05 s, 0.0 kB/s 2023-08-27T11:20:55.452 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:55.453 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:55.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:55.510 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.753 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:55.568 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:55.568 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2539e-05 s, 0.0 kB/s 2023-08-27T11:20:55.568 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:55.569 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:55.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:55.624 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.754 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:55.681 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:55.682 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:55.682 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.297e-05 s, 0.0 kB/s 2023-08-27T11:20:55.683 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:55.683 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:55.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:55.741 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.755 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:55.798 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:55.798 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:55.799 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1284e-05 s, 0.0 kB/s 2023-08-27T11:20:55.799 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:55.799 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:55.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:55.856 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.756 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:55.912 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:55.913 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:55.913 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1253e-05 s, 0.0 kB/s 2023-08-27T11:20:55.914 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:55.914 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:55.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:55.971 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.757 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2028e-05 s, 0.0 kB/s 2023-08-27T11:20:56.029 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:56.030 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:56.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:56.087 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.758 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:56.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:56.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:56.144 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9786e-05 s, 0.0 kB/s 2023-08-27T11:20:56.145 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:56.145 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:56.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:56.202 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.759 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:56.259 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:56.259 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:56.259 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0706e-05 s, 0.0 kB/s 2023-08-27T11:20:56.260 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:56.260 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:56.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:56.317 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.760 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:56.375 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:56.375 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:56.375 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0231e-05 s, 0.0 kB/s 2023-08-27T11:20:56.375 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:56.376 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:56.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:56.433 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.761 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:56.490 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:56.490 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:56.491 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9255e-05 s, 0.0 kB/s 2023-08-27T11:20:56.491 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:56.491 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:56.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:56.548 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.762 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:56.604 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:56.605 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:56.605 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.106e-05 s, 0.0 kB/s 2023-08-27T11:20:56.605 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:56.606 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:56.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:56.663 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.763 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:56.719 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:56.720 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:56.720 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.003e-05 s, 0.0 kB/s 2023-08-27T11:20:56.720 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:56.721 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:56.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:56.776 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.764 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:56.834 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0963e-05 s, 0.0 kB/s 2023-08-27T11:20:56.834 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:56.834 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:56.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:56.891 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.765 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:56.947 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:56.948 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:56.948 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7044e-05 s, 0.0 kB/s 2023-08-27T11:20:56.948 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:56.949 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:57.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:57.006 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.766 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:57.063 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:57.063 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:57.063 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2061e-05 s, 0.0 kB/s 2023-08-27T11:20:57.064 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:57.064 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:57.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:57.121 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.767 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:57.178 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:57.178 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:57.178 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0024e-05 s, 0.0 kB/s 2023-08-27T11:20:57.179 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:57.179 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:57.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:57.236 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.768 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:57.294 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:57.294 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:57.294 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0199e-05 s, 0.0 kB/s 2023-08-27T11:20:57.295 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:57.295 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:57.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:57.355 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.769 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:57.412 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:57.412 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:57.412 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0201e-05 s, 0.0 kB/s 2023-08-27T11:20:57.413 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:57.413 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:57.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:57.470 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.770 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:57.526 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:57.527 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:57.527 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8007e-05 s, 0.0 kB/s 2023-08-27T11:20:57.527 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:57.528 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:57.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:57.585 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.771 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:57.643 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:57.643 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:57.643 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8478e-05 s, 0.0 kB/s 2023-08-27T11:20:57.644 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:57.644 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:57.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:57.701 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.772 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9475e-05 s, 0.0 kB/s 2023-08-27T11:20:57.759 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:57.759 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:57.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:57.816 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.773 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9043e-05 s, 0.0 kB/s 2023-08-27T11:20:57.873 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:57.874 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:57.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:57.930 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.774 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9397e-05 s, 0.0 kB/s 2023-08-27T11:20:57.988 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:57.989 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:58.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:58.046 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.775 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:58.104 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:58.104 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:58.104 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9894e-05 s, 0.0 kB/s 2023-08-27T11:20:58.105 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:58.105 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:58.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:58.162 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.776 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:58.219 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:58.220 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:58.220 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9706e-05 s, 0.0 kB/s 2023-08-27T11:20:58.220 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:58.221 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:58.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:58.277 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.777 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:58.338 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:58.338 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:58.338 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0578e-05 s, 0.0 kB/s 2023-08-27T11:20:58.339 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:58.339 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:58.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:58.399 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.778 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:58.456 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:58.456 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:58.457 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2709e-05 s, 0.0 kB/s 2023-08-27T11:20:58.457 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:58.457 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:58.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:58.514 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.779 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:58.572 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:58.572 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:58.572 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8719e-05 s, 0.0 kB/s 2023-08-27T11:20:58.573 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:58.573 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:58.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:58.631 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.780 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:58.688 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:58.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:58.689 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8144e-05 s, 0.0 kB/s 2023-08-27T11:20:58.689 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:58.690 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:58.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:58.747 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.781 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:58.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:58.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:58.805 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2115e-05 s, 0.0 kB/s 2023-08-27T11:20:58.806 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:58.806 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:58.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:58.863 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.782 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:58.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:58.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:58.922 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1482e-05 s, 0.0 kB/s 2023-08-27T11:20:58.922 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:58.922 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:58.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:58.979 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.783 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:59.036 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:59.036 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:59.037 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7014e-05 s, 0.0 kB/s 2023-08-27T11:20:59.037 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:59.037 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:59.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:59.094 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.784 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:59.152 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:59.152 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:59.153 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9551e-05 s, 0.0 kB/s 2023-08-27T11:20:59.153 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:59.153 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:59.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:59.210 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.785 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9506e-05 s, 0.0 kB/s 2023-08-27T11:20:59.267 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:59.268 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:59.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:59.325 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.786 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:59.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:59.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:59.383 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1278e-05 s, 0.0 kB/s 2023-08-27T11:20:59.383 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:59.384 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:59.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:59.441 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.787 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9276e-05 s, 0.0 kB/s 2023-08-27T11:20:59.498 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:59.498 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:59.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:59.555 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.788 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:59.612 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:59.612 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.841e-05 s, 0.0 kB/s 2023-08-27T11:20:59.612 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:59.613 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:59.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:59.669 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.789 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:59.726 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:59.726 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:59.726 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8995e-05 s, 0.0 kB/s 2023-08-27T11:20:59.727 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:59.727 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:59.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:59.784 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.790 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:59.840 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:59.840 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:59.841 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7316e-05 s, 0.0 kB/s 2023-08-27T11:20:59.841 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:59.841 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:20:59.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:20:59.898 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.791 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:20:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:20:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:20:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8542e-05 s, 0.0 kB/s 2023-08-27T11:20:59.955 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:20:59.956 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:00.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:00.012 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.792 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:00.069 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:00.070 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:00.070 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0893e-05 s, 0.0 kB/s 2023-08-27T11:21:00.070 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:00.071 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:00.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:00.128 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.793 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:00.186 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:00.186 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:00.186 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0387e-05 s, 0.0 kB/s 2023-08-27T11:21:00.187 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:00.187 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:00.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:00.245 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.794 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0597e-05 s, 0.0 kB/s 2023-08-27T11:21:00.304 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:00.304 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:00.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:00.361 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.795 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1575e-05 s, 0.0 kB/s 2023-08-27T11:21:00.419 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:00.420 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:00.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:00.477 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.796 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:00.536 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:00.536 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:00.536 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7348e-05 s, 0.0 kB/s 2023-08-27T11:21:00.537 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:00.537 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:00.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:00.594 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.797 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:00.651 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:00.652 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:00.652 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8689e-05 s, 0.0 kB/s 2023-08-27T11:21:00.652 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:00.653 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:00.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:00.709 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.798 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:00.766 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:00.766 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:00.766 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1144e-05 s, 0.0 kB/s 2023-08-27T11:21:00.767 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:00.767 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:00.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:00.824 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.799 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:00.881 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:00.881 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:00.882 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1156e-05 s, 0.0 kB/s 2023-08-27T11:21:00.882 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:00.882 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:00.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:00.939 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.800 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:00.996 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:00.996 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:00.996 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0771e-05 s, 0.0 kB/s 2023-08-27T11:21:00.997 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:00.997 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:01.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:01.054 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.801 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:01.111 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:01.111 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9129e-05 s, 0.0 kB/s 2023-08-27T11:21:01.111 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:01.112 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:01.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:01.169 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.802 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8473e-05 s, 0.0 kB/s 2023-08-27T11:21:01.226 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:01.227 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:01.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:01.285 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.803 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:01.341 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:01.342 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:01.342 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9502e-05 s, 0.0 kB/s 2023-08-27T11:21:01.342 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:01.343 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:01.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:01.400 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.804 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:01.458 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:01.458 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:01.458 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7442e-05 s, 0.0 kB/s 2023-08-27T11:21:01.459 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:01.459 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:01.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:01.516 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.805 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:01.572 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:01.572 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:01.573 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.8415e-05 s, 0.0 kB/s 2023-08-27T11:21:01.573 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:01.573 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:01.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:01.630 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.806 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:01.688 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:01.688 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:01.688 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9501e-05 s, 0.0 kB/s 2023-08-27T11:21:01.689 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:01.689 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:01.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:01.746 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.807 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:01.802 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:01.802 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:01.802 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7768e-05 s, 0.0 kB/s 2023-08-27T11:21:01.803 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:01.803 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:01.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:01.861 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.808 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:01.917 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:01.917 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:01.918 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7553e-05 s, 0.0 kB/s 2023-08-27T11:21:01.918 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:01.918 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:01.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:01.975 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.809 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:02.031 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:02.031 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:02.032 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2072e-05 s, 0.0 kB/s 2023-08-27T11:21:02.032 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:02.032 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:02.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:02.088 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.810 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:02.145 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:02.145 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:02.145 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2187e-05 s, 0.0 kB/s 2023-08-27T11:21:02.146 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:02.146 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:02.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:02.203 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.811 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:02.260 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:02.261 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:02.261 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8288e-05 s, 0.0 kB/s 2023-08-27T11:21:02.261 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:02.262 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:02.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:02.318 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.812 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:02.375 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:02.375 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:02.376 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8823e-05 s, 0.0 kB/s 2023-08-27T11:21:02.376 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:02.376 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:02.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:02.436 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.813 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4485e-05 s, 0.0 kB/s 2023-08-27T11:21:02.502 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:02.503 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:02.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:02.560 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.814 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:02.616 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:02.617 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:02.617 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6441e-05 s, 0.0 kB/s 2023-08-27T11:21:02.617 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:02.618 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:02.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:02.676 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.815 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:02.732 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:02.732 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:02.733 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.102e-05 s, 0.0 kB/s 2023-08-27T11:21:02.733 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:02.733 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:02.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:02.789 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.816 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:02.846 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:02.847 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:02.847 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7886e-05 s, 0.0 kB/s 2023-08-27T11:21:02.847 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:02.848 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:02.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:02.905 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.817 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:02.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:02.963 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:02.963 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0377e-05 s, 0.0 kB/s 2023-08-27T11:21:02.963 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:02.964 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:03.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:03.020 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.818 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:03.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:03.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:03.078 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9011e-05 s, 0.0 kB/s 2023-08-27T11:21:03.079 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:03.079 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:03.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:03.135 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.819 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:03.191 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:03.192 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:03.192 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9793e-05 s, 0.0 kB/s 2023-08-27T11:21:03.193 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:03.193 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:03.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:03.250 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.820 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:03.307 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:03.307 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:03.307 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1364e-05 s, 0.0 kB/s 2023-08-27T11:21:03.308 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:03.308 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:03.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:03.366 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.821 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6598e-05 s, 0.0 kB/s 2023-08-27T11:21:03.425 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:03.425 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:03.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:03.482 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.822 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:03.538 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:03.538 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:03.538 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8932e-05 s, 0.0 kB/s 2023-08-27T11:21:03.538 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:03.539 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:03.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:03.596 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.823 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:03.653 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:03.653 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:03.654 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0926e-05 s, 0.0 kB/s 2023-08-27T11:21:03.654 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:03.655 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:03.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:03.711 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.824 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8154e-05 s, 0.0 kB/s 2023-08-27T11:21:03.768 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:03.769 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:03.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:03.827 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.825 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:03.884 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:03.884 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:03.884 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2218e-05 s, 0.0 kB/s 2023-08-27T11:21:03.885 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:03.885 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:03.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:03.942 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.826 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:03.998 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:03.999 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:03.999 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2934e-05 s, 0.0 kB/s 2023-08-27T11:21:03.999 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:04.000 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:04.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:04.056 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.827 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:04.114 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:04.114 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9568e-05 s, 0.0 kB/s 2023-08-27T11:21:04.115 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:04.116 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:04.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:04.172 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.828 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:04.228 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:04.228 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:04.229 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.043e-05 s, 0.0 kB/s 2023-08-27T11:21:04.229 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:04.229 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:04.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:04.286 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.829 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:04.344 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:04.344 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:04.345 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3923e-05 s, 0.0 kB/s 2023-08-27T11:21:04.345 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:04.346 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:04.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:04.402 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.830 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:04.460 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6108e-05 s, 0.0 kB/s 2023-08-27T11:21:04.460 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:04.460 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:04.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:04.517 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.831 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:04.574 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:04.574 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:04.575 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.153e-05 s, 0.0 kB/s 2023-08-27T11:21:04.575 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:04.575 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:04.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:04.632 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.832 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:04.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:04.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:04.690 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9169e-05 s, 0.0 kB/s 2023-08-27T11:21:04.690 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:04.690 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:04.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:04.747 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.833 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:04.804 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:04.804 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:04.805 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8901e-05 s, 0.0 kB/s 2023-08-27T11:21:04.805 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:04.805 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:04.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:04.865 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.834 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1317e-05 s, 0.0 kB/s 2023-08-27T11:21:04.923 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:04.923 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:04.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:04.980 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.835 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0513e-05 s, 0.0 kB/s 2023-08-27T11:21:05.037 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:05.038 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:05.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:05.094 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.836 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:05.151 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:05.151 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:05.152 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6738e-05 s, 0.0 kB/s 2023-08-27T11:21:05.152 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:05.152 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:05.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:05.208 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.837 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:05.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:05.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:05.266 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9402e-05 s, 0.0 kB/s 2023-08-27T11:21:05.266 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:05.266 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:05.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:05.323 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.838 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:05.381 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:05.382 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:05.382 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9257e-05 s, 0.0 kB/s 2023-08-27T11:21:05.383 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:05.383 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:05.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:05.439 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.839 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:05.497 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8431e-05 s, 0.0 kB/s 2023-08-27T11:21:05.497 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:05.498 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:05.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:05.553 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.840 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7812e-05 s, 0.0 kB/s 2023-08-27T11:21:05.613 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:05.613 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:05.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:05.670 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.841 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:05.726 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8558e-05 s, 0.0 kB/s 2023-08-27T11:21:05.727 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:05.728 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:05.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:05.785 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.842 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:05.841 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:05.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:05.842 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8211e-05 s, 0.0 kB/s 2023-08-27T11:21:05.842 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:05.843 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:05.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:05.900 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.843 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:05.956 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:05.957 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:05.957 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0268e-05 s, 0.0 kB/s 2023-08-27T11:21:05.957 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:05.958 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:06.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:06.014 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.844 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:06.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:21:06.057 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:21:06.071 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:06.071 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:06.072 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1894e-05 s, 0.0 kB/s 2023-08-27T11:21:06.072 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:06.072 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:06.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:06.129 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.845 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:06.185 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:06.186 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:06.186 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7e-05 s, 0.0 kB/s 2023-08-27T11:21:06.186 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:06.187 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:06.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:06.244 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.846 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:06.301 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:06.301 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2312e-05 s, 0.0 kB/s 2023-08-27T11:21:06.302 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:06.302 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:06.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:06.368 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.847 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:06.388 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:06.389 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:06.389 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9406e-05 s, 0.0 kB/s 2023-08-27T11:21:06.390 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:06.390 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:06.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:06.455 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.848 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:06.513 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:06.514 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:06.514 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7883e-05 s, 0.0 kB/s 2023-08-27T11:21:06.514 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:06.516 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:06.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:06.575 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.849 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:06.642 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:06.643 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:06.643 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9379e-05 s, 0.0 kB/s 2023-08-27T11:21:06.643 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:06.644 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:06.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:06.708 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.850 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:06.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:06.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:06.769 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0011e-05 s, 0.0 kB/s 2023-08-27T11:21:06.769 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:06.770 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:06.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:06.830 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.851 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:06.894 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:06.895 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:06.895 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8344e-05 s, 0.0 kB/s 2023-08-27T11:21:06.895 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:06.896 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:06.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:06.961 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.852 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:07.024 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8018e-05 s, 0.0 kB/s 2023-08-27T11:21:07.025 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:07.026 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:07.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:07.085 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.853 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:07.152 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:07.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:07.153 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8579e-05 s, 0.0 kB/s 2023-08-27T11:21:07.154 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:07.154 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:07.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:07.220 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.854 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:07.281 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:07.282 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:07.282 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.986e-05 s, 0.0 kB/s 2023-08-27T11:21:07.282 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:07.283 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:07.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:07.340 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.855 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:07.397 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:07.398 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:07.398 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9867e-05 s, 0.0 kB/s 2023-08-27T11:21:07.400 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:07.400 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:07.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:07.463 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.856 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:07.523 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:07.523 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:07.524 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9998e-05 s, 0.0 kB/s 2023-08-27T11:21:07.524 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:07.525 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:07.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:07.588 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.857 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:07.648 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:07.649 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:07.649 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0211e-05 s, 0.0 kB/s 2023-08-27T11:21:07.649 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:07.650 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:07.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:07.713 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.858 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8327e-05 s, 0.0 kB/s 2023-08-27T11:21:07.774 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:07.774 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:07.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:07.831 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.859 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:07.894 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:07.895 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:07.895 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0515e-05 s, 0.0 kB/s 2023-08-27T11:21:07.895 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:07.896 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:07.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:07.952 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.860 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1868e-05 s, 0.0 kB/s 2023-08-27T11:21:08.010 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:08.011 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:08.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:08.068 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.861 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:08.130 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:08.130 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:08.130 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9365e-05 s, 0.0 kB/s 2023-08-27T11:21:08.131 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:08.131 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:08.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:08.187 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.862 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:08.244 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:08.245 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:08.245 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9593e-05 s, 0.0 kB/s 2023-08-27T11:21:08.245 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:08.246 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:08.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:08.303 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.863 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:08.360 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:08.360 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:08.361 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2294e-05 s, 0.0 kB/s 2023-08-27T11:21:08.361 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:08.362 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:08.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:08.418 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.864 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:08.475 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:08.475 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:08.475 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.749e-05 s, 0.0 kB/s 2023-08-27T11:21:08.475 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:08.476 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:08.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:08.533 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.865 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:08.592 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:08.592 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:08.592 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0862e-05 s, 0.0 kB/s 2023-08-27T11:21:08.593 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:08.593 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:08.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:08.650 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.866 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:08.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:08.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:08.708 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8203e-05 s, 0.0 kB/s 2023-08-27T11:21:08.709 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:08.709 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:08.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:08.766 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.867 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:08.824 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:08.825 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:08.825 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8436e-05 s, 0.0 kB/s 2023-08-27T11:21:08.825 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:08.826 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:08.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:08.883 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.868 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:08.941 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:08.941 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:08.941 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.047e-05 s, 0.0 kB/s 2023-08-27T11:21:08.942 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:08.942 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:08.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:08.998 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.869 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:09.057 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:09.064 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:09.064 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9046e-05 s, 0.0 kB/s 2023-08-27T11:21:09.065 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:09.065 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:09.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:09.115 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.870 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:09.172 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:09.173 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:09.173 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1475e-05 s, 0.0 kB/s 2023-08-27T11:21:09.173 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:09.174 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:09.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:09.233 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.871 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2682e-05 s, 0.0 kB/s 2023-08-27T11:21:09.290 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:09.291 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:09.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:09.347 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.872 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:09.405 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0393e-05 s, 0.0 kB/s 2023-08-27T11:21:09.406 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:09.407 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:09.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:09.463 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.873 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8899e-05 s, 0.0 kB/s 2023-08-27T11:21:09.522 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:09.522 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:09.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:09.579 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.874 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:09.638 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:09.638 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:09.638 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7218e-05 s, 0.0 kB/s 2023-08-27T11:21:09.639 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:09.639 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:09.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:09.696 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.875 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.958e-05 s, 0.0 kB/s 2023-08-27T11:21:09.755 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:09.755 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:09.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:09.812 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.876 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9354e-05 s, 0.0 kB/s 2023-08-27T11:21:09.870 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:09.870 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:09.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:09.931 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.877 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6798e-05 s, 0.0 kB/s 2023-08-27T11:21:09.990 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:09.990 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:10.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:10.047 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.878 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:10.105 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:10.105 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:10.105 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.947e-05 s, 0.0 kB/s 2023-08-27T11:21:10.106 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:10.106 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:10.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:10.163 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.879 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:10.220 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:10.221 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:10.222 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.869e-05 s, 0.0 kB/s 2023-08-27T11:21:10.222 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:10.223 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:10.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:10.279 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.880 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:10.337 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:10.337 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:10.337 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6349e-05 s, 0.0 kB/s 2023-08-27T11:21:10.338 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:10.338 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:10.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:10.395 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.881 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:10.455 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:10.455 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:10.455 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7999e-05 s, 0.0 kB/s 2023-08-27T11:21:10.456 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:10.457 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:10.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:10.515 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.882 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:10.572 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:10.572 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:10.573 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7549e-05 s, 0.0 kB/s 2023-08-27T11:21:10.573 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:10.573 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:10.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:10.630 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.883 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:10.687 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:10.687 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:10.687 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1013e-05 s, 0.0 kB/s 2023-08-27T11:21:10.688 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:10.689 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:10.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:10.747 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.884 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:10.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:10.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:10.806 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2129e-05 s, 0.0 kB/s 2023-08-27T11:21:10.806 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:10.806 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:10.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:10.863 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.885 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:10.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:10.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:10.921 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1138e-05 s, 0.0 kB/s 2023-08-27T11:21:10.922 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:10.922 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:10.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:10.979 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.886 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:11.037 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:11.037 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:11.038 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7453e-05 s, 0.0 kB/s 2023-08-27T11:21:11.038 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:11.038 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:11.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:11.095 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.887 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:11.152 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:11.152 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:11.152 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1717e-05 s, 0.0 kB/s 2023-08-27T11:21:11.153 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:11.153 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:11.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:11.210 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.888 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:11.267 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:11.268 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:11.268 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.005e-05 s, 0.0 kB/s 2023-08-27T11:21:11.268 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:11.269 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:11.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:11.326 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.889 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:11.385 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2471e-05 s, 0.0 kB/s 2023-08-27T11:21:11.385 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:11.385 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:11.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:11.442 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.890 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:11.498 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:11.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:11.499 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8637e-05 s, 0.0 kB/s 2023-08-27T11:21:11.499 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:11.500 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:11.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:11.557 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.891 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:11.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:11.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:11.614 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.084e-05 s, 0.0 kB/s 2023-08-27T11:21:11.614 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:11.615 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:11.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:11.672 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.892 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:11.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:11.730 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:11.730 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.124e-05 s, 0.0 kB/s 2023-08-27T11:21:11.730 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:11.732 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:11.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:11.796 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.893 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:11.857 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:11.857 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:11.858 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0274e-05 s, 0.0 kB/s 2023-08-27T11:21:11.858 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:11.859 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:11.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:11.916 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.894 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:11.974 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:11.974 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:11.974 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7604e-05 s, 0.0 kB/s 2023-08-27T11:21:11.975 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:11.975 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:12.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:12.033 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.895 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:12.091 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2169e-05 s, 0.0 kB/s 2023-08-27T11:21:12.092 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:12.092 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:12.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:12.151 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.896 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:12.214 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:12.215 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:12.215 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9149e-05 s, 0.0 kB/s 2023-08-27T11:21:12.216 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:12.216 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:12.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:12.285 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.897 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:12.346 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3348e-05 s, 0.0 kB/s 2023-08-27T11:21:12.348 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:12.348 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:12.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:12.409 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.898 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:12.470 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:12.471 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:12.471 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.212e-05 s, 0.0 kB/s 2023-08-27T11:21:12.471 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:12.472 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:12.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:12.534 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.899 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:12.592 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:12.592 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:12.592 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.951e-05 s, 0.0 kB/s 2023-08-27T11:21:12.593 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:12.593 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:12.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:12.652 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.900 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:12.676 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:12.676 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:12.676 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1549e-05 s, 0.0 kB/s 2023-08-27T11:21:12.677 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:12.677 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:12.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:12.737 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.901 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:12.795 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:12.796 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:12.796 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8604e-05 s, 0.0 kB/s 2023-08-27T11:21:12.796 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:12.797 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:12.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:12.854 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.902 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:12.913 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:12.913 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:12.913 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7378e-05 s, 0.0 kB/s 2023-08-27T11:21:12.914 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:12.914 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:12.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:12.973 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.903 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:13.033 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:13.034 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:13.034 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9114e-05 s, 0.0 kB/s 2023-08-27T11:21:13.035 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:13.035 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:13.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:13.094 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.904 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:13.154 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:13.155 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:13.155 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0989e-05 s, 0.0 kB/s 2023-08-27T11:21:13.156 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:13.156 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:13.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:13.215 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.905 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:13.274 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:13.274 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:13.275 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0699e-05 s, 0.0 kB/s 2023-08-27T11:21:13.276 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:13.276 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:13.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:13.333 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.906 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:13.390 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:13.390 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:13.390 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8385e-05 s, 0.0 kB/s 2023-08-27T11:21:13.391 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:13.391 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:13.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:13.448 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.907 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7369e-05 s, 0.0 kB/s 2023-08-27T11:21:13.505 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:13.505 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:13.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:13.562 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.908 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:13.618 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:13.619 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:13.619 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6405e-05 s, 0.0 kB/s 2023-08-27T11:21:13.619 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:13.620 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:13.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:13.676 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.909 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:13.734 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:13.734 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:13.735 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8492e-05 s, 0.0 kB/s 2023-08-27T11:21:13.735 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:13.735 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:13.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:13.792 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.910 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8366e-05 s, 0.0 kB/s 2023-08-27T11:21:13.851 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:13.851 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:13.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:13.908 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.911 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:13.966 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:13.966 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:13.966 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9129e-05 s, 0.0 kB/s 2023-08-27T11:21:13.967 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:13.967 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:14.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:14.023 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.912 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:14.080 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:14.081 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:14.081 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3103e-05 s, 0.0 kB/s 2023-08-27T11:21:14.081 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:14.082 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:14.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:14.137 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.913 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:14.195 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:14.195 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.031e-05 s, 0.0 kB/s 2023-08-27T11:21:14.195 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:14.195 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:14.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:14.253 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.914 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:14.310 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:14.311 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:14.311 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7675e-05 s, 0.0 kB/s 2023-08-27T11:21:14.311 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:14.312 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:14.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:14.369 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.915 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:14.428 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8718e-05 s, 0.0 kB/s 2023-08-27T11:21:14.428 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:14.429 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:14.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:14.485 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.916 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:14.543 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:14.544 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:14.544 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8117e-05 s, 0.0 kB/s 2023-08-27T11:21:14.545 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:14.545 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:14.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:14.602 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.917 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:14.660 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8946e-05 s, 0.0 kB/s 2023-08-27T11:21:14.662 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:14.663 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:14.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:14.719 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.918 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:14.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:14.778 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:14.778 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0191e-05 s, 0.0 kB/s 2023-08-27T11:21:14.778 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:14.778 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:14.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:14.836 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.919 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:14.894 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:14.894 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:14.894 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9056e-05 s, 0.0 kB/s 2023-08-27T11:21:14.895 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:14.895 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:14.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:14.952 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.920 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:15.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:15.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:15.009 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0264e-05 s, 0.0 kB/s 2023-08-27T11:21:15.009 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:15.009 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:15.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:15.066 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.921 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:15.122 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:15.122 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:15.122 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7377e-05 s, 0.0 kB/s 2023-08-27T11:21:15.123 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:15.123 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:15.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:15.181 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.922 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:15.238 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:15.238 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:15.238 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8182e-05 s, 0.0 kB/s 2023-08-27T11:21:15.238 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:15.239 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:15.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:15.296 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.923 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9419e-05 s, 0.0 kB/s 2023-08-27T11:21:15.353 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:15.353 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:15.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:15.410 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.924 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:15.467 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:15.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:15.468 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0289e-05 s, 0.0 kB/s 2023-08-27T11:21:15.468 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:15.469 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:15.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:15.525 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.925 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6361e-05 s, 0.0 kB/s 2023-08-27T11:21:15.583 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:15.583 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:15.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:15.640 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.926 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:15.697 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:15.697 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:15.698 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9193e-05 s, 0.0 kB/s 2023-08-27T11:21:15.698 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:15.698 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:15.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:15.754 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.927 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2533e-05 s, 0.0 kB/s 2023-08-27T11:21:15.813 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:15.813 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:15.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:15.870 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.928 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:15.927 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:15.928 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:15.928 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7762e-05 s, 0.0 kB/s 2023-08-27T11:21:15.928 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:15.928 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:15.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:15.987 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.929 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:16.043 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:16.043 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:16.044 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8354e-05 s, 0.0 kB/s 2023-08-27T11:21:16.044 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:16.044 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:16.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:16.101 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.930 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8877e-05 s, 0.0 kB/s 2023-08-27T11:21:16.160 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:16.161 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:16.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:16.217 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.931 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:16.275 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:16.275 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:16.275 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1393e-05 s, 0.0 kB/s 2023-08-27T11:21:16.276 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:16.276 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:16.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:16.333 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.932 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8262e-05 s, 0.0 kB/s 2023-08-27T11:21:16.391 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:16.392 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:16.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:16.448 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.933 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8906e-05 s, 0.0 kB/s 2023-08-27T11:21:16.506 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:16.507 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:16.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:16.566 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.934 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:16.623 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9328e-05 s, 0.0 kB/s 2023-08-27T11:21:16.623 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:16.623 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:16.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:16.680 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.935 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0933e-05 s, 0.0 kB/s 2023-08-27T11:21:16.737 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:16.737 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:16.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:16.795 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.936 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:16.852 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:16.852 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:16.852 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9501e-05 s, 0.0 kB/s 2023-08-27T11:21:16.853 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:16.853 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:16.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:16.910 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.937 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:16.967 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:16.967 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:16.967 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0106e-05 s, 0.0 kB/s 2023-08-27T11:21:16.967 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:16.968 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:17.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:17.024 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.938 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:17.081 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:17.109 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:17.109 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9978e-05 s, 0.0 kB/s 2023-08-27T11:21:17.109 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:17.110 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:17.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:17.139 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.939 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:17.197 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:17.197 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:17.197 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8146e-05 s, 0.0 kB/s 2023-08-27T11:21:17.198 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:17.199 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:17.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:17.256 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.940 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:17.313 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:17.313 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8232e-05 s, 0.0 kB/s 2023-08-27T11:21:17.314 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:17.314 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:17.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:17.371 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.941 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:17.428 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:17.428 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:17.428 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7822e-05 s, 0.0 kB/s 2023-08-27T11:21:17.429 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:17.429 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:17.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:17.487 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.942 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:17.544 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8825e-05 s, 0.0 kB/s 2023-08-27T11:21:17.544 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:17.544 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:17.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:17.600 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.943 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:17.657 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:17.657 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:17.657 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2407e-05 s, 0.0 kB/s 2023-08-27T11:21:17.658 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:17.658 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:17.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:17.715 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.944 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:17.771 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:17.772 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:17.772 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.148e-05 s, 0.0 kB/s 2023-08-27T11:21:17.772 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:17.773 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:17.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:17.828 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.945 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:17.886 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:17.886 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:17.886 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8509e-05 s, 0.0 kB/s 2023-08-27T11:21:17.887 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:17.887 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:17.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:17.944 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.946 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:18.001 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:18.002 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:18.002 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.739e-05 s, 0.0 kB/s 2023-08-27T11:21:18.002 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:18.003 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:18.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:18.060 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.947 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:18.117 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1081e-05 s, 0.0 kB/s 2023-08-27T11:21:18.117 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:18.117 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:18.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:18.173 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.948 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:18.232 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:18.232 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:18.232 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9364e-05 s, 0.0 kB/s 2023-08-27T11:21:18.233 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:18.233 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:18.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:18.290 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.949 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:18.346 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:18.346 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:18.347 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0584e-05 s, 0.0 kB/s 2023-08-27T11:21:18.347 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:18.347 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:18.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:18.404 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.950 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:18.461 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:18.461 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:18.461 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.078e-05 s, 0.0 kB/s 2023-08-27T11:21:18.462 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:18.462 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:18.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:18.518 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.951 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:18.575 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:18.575 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7061e-05 s, 0.0 kB/s 2023-08-27T11:21:18.576 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:18.576 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:18.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:18.632 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.952 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:18.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:18.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:18.690 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7541e-05 s, 0.0 kB/s 2023-08-27T11:21:18.690 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:18.690 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:18.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:18.746 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.953 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:18.803 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:18.803 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:18.804 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.9414e-05 s, 0.0 kB/s 2023-08-27T11:21:18.804 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:18.804 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:18.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:18.861 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.954 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:18.920 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8619e-05 s, 0.0 kB/s 2023-08-27T11:21:18.921 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:18.922 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:18.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:18.978 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.955 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:19.035 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:19.036 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:19.036 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8403e-05 s, 0.0 kB/s 2023-08-27T11:21:19.036 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:19.037 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:19.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:19.093 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.956 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8246e-05 s, 0.0 kB/s 2023-08-27T11:21:19.152 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:19.152 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:19.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:19.208 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.957 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:19.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:19.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:19.266 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8646e-05 s, 0.0 kB/s 2023-08-27T11:21:19.266 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:19.266 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:19.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:19.324 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.958 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.7679e-05 s, 0.0 kB/s 2023-08-27T11:21:19.401 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:19.402 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:19.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:19.459 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.959 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:19.516 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:19.517 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:19.517 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8546e-05 s, 0.0 kB/s 2023-08-27T11:21:19.517 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:19.518 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:19.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:19.574 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.960 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:19.631 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:19.631 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:19.631 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7605e-05 s, 0.0 kB/s 2023-08-27T11:21:19.632 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:19.633 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:19.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:19.689 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.961 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:19.745 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:19.746 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:19.746 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.1805e-05 s, 0.0 kB/s 2023-08-27T11:21:19.746 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:19.746 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:19.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:19.803 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.962 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:19.861 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:19.862 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:19.862 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8907e-05 s, 0.0 kB/s 2023-08-27T11:21:19.862 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:19.863 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:19.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:19.919 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.963 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:19.977 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:19.977 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:19.977 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0971e-05 s, 0.0 kB/s 2023-08-27T11:21:19.978 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:19.978 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:20.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:20.036 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.964 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:20.092 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:20.093 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:20.093 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9575e-05 s, 0.0 kB/s 2023-08-27T11:21:20.093 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:20.094 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:20.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:20.154 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.965 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:20.210 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:20.210 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1355e-05 s, 0.0 kB/s 2023-08-27T11:21:20.210 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:20.211 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:20.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:20.267 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.966 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:20.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:20.325 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:20.326 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0233e-05 s, 0.0 kB/s 2023-08-27T11:21:20.326 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:20.327 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:20.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:20.382 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.967 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:20.438 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:20.439 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:20.439 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9629e-05 s, 0.0 kB/s 2023-08-27T11:21:20.439 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:20.440 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:20.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:20.500 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.968 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:20.557 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:20.558 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:20.558 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8133e-05 s, 0.0 kB/s 2023-08-27T11:21:20.558 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:20.559 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:20.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:20.616 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.969 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:20.672 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:20.673 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:20.674 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8515e-05 s, 0.0 kB/s 2023-08-27T11:21:20.674 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:20.674 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:20.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:20.731 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.970 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:20.787 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:20.787 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:20.787 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0679e-05 s, 0.0 kB/s 2023-08-27T11:21:20.788 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:20.788 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:20.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:20.845 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.971 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:20.902 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:20.902 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:20.902 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1711e-05 s, 0.0 kB/s 2023-08-27T11:21:20.903 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:20.903 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:20.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:20.959 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.972 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:21.016 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7338e-05 s, 0.0 kB/s 2023-08-27T11:21:21.017 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:21.018 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:21.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:21.074 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.973 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:21.132 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:21.132 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:21.132 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0606e-05 s, 0.0 kB/s 2023-08-27T11:21:21.133 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:21.133 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:21.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:21.191 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.974 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:21.248 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:21.248 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:21.249 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9282e-05 s, 0.0 kB/s 2023-08-27T11:21:21.249 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:21.250 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:21.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:21.306 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.975 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:21.362 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:21.362 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:21.362 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0795e-05 s, 0.0 kB/s 2023-08-27T11:21:21.363 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:21.363 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:21.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:21.420 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.976 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:21.477 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:21.478 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:21.478 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1138e-05 s, 0.0 kB/s 2023-08-27T11:21:21.478 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:21.479 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:21.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:21.536 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.977 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:21.594 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:21.594 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:21.594 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9961e-05 s, 0.0 kB/s 2023-08-27T11:21:21.595 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:21.595 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:21.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:21.652 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.978 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:21.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:21.709 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:21.709 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.5781e-05 s, 0.0 kB/s 2023-08-27T11:21:21.709 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:21.710 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:21.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:21.767 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.979 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:21.827 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.925e-05 s, 0.0 kB/s 2023-08-27T11:21:21.828 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:21.829 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:21.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:21.886 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.980 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9035e-05 s, 0.0 kB/s 2023-08-27T11:21:21.945 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:21.945 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:22.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:22.001 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.981 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:22.059 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:22.059 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:22.060 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0278e-05 s, 0.0 kB/s 2023-08-27T11:21:22.060 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:22.060 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:22.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:22.116 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.982 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:22.173 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:22.174 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:22.174 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6706e-05 s, 0.0 kB/s 2023-08-27T11:21:22.175 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:22.175 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:22.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:22.232 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.983 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:22.288 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:22.288 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:22.289 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8749e-05 s, 0.0 kB/s 2023-08-27T11:21:22.289 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:22.289 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:22.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:22.347 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.984 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:22.404 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:22.404 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:22.404 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9895e-05 s, 0.0 kB/s 2023-08-27T11:21:22.405 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:22.405 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:22.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:22.463 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.985 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:22.519 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:22.519 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:22.519 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0856e-05 s, 0.0 kB/s 2023-08-27T11:21:22.520 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:22.520 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:22.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:22.577 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.986 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:22.633 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:22.634 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:22.634 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8539e-05 s, 0.0 kB/s 2023-08-27T11:21:22.634 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:22.635 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:22.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:22.692 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.987 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:22.748 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:22.749 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:22.749 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8522e-05 s, 0.0 kB/s 2023-08-27T11:21:22.750 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:22.750 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:22.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:22.807 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.988 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:22.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:22.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:22.864 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6526e-05 s, 0.0 kB/s 2023-08-27T11:21:22.864 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:22.865 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:22.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:22.922 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.989 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:22.978 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:22.979 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:22.979 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0644e-05 s, 0.0 kB/s 2023-08-27T11:21:22.979 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:22.980 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:23.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:23.036 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.990 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:23.094 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:23.095 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:23.095 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7276e-05 s, 0.0 kB/s 2023-08-27T11:21:23.095 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:23.096 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:23.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:23.153 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.991 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:23.211 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:23.211 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:23.211 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 5.3194e-05 s, 0.0 kB/s 2023-08-27T11:21:23.212 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:23.212 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:23.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:23.269 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.992 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:23.326 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:23.326 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:23.326 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8898e-05 s, 0.0 kB/s 2023-08-27T11:21:23.327 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:23.327 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:23.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:23.384 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.993 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:23.440 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:23.441 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:23.441 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8575e-05 s, 0.0 kB/s 2023-08-27T11:21:23.441 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:23.442 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:23.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:23.498 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.994 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:23.555 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3038e-05 s, 0.0 kB/s 2023-08-27T11:21:23.556 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:23.557 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:23.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:23.613 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.995 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:23.670 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:23.670 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:23.671 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7239e-05 s, 0.0 kB/s 2023-08-27T11:21:23.671 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:23.672 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:23.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:23.728 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.996 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:23.785 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:23.785 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:23.786 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9019e-05 s, 0.0 kB/s 2023-08-27T11:21:23.786 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:23.786 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:23.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:23.843 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.997 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:23.900 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:23.901 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:23.901 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7967e-05 s, 0.0 kB/s 2023-08-27T11:21:23.901 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:23.902 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:23.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:23.958 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.998 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:24.015 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:24.016 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:24.016 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8951e-05 s, 0.0 kB/s 2023-08-27T11:21:24.016 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:24.017 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:24.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:24.073 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.999 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:24.131 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0163e-05 s, 0.0 kB/s 2023-08-27T11:21:24.133 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:24.133 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:24.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:24.197 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1000 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:24.255 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:24.255 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:24.255 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.329e-05 s, 0.0 kB/s 2023-08-27T11:21:24.256 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:24.256 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:24.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:24.313 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1001 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:24.371 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:24.371 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:24.371 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0605e-05 s, 0.0 kB/s 2023-08-27T11:21:24.372 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:24.372 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:24.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:24.429 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1002 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:24.486 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:24.487 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:24.487 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9113e-05 s, 0.0 kB/s 2023-08-27T11:21:24.488 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:24.488 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:24.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:24.544 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1003 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:24.601 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:24.602 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:24.602 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0897e-05 s, 0.0 kB/s 2023-08-27T11:21:24.602 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:24.603 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:24.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:24.660 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1004 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:24.717 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:24.717 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2153e-05 s, 0.0 kB/s 2023-08-27T11:21:24.718 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:24.718 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:24.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:24.777 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1005 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:24.833 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1785e-05 s, 0.0 kB/s 2023-08-27T11:21:24.834 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:24.835 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:24.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:24.892 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1006 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:24.949 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:24.950 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:24.950 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8165e-05 s, 0.0 kB/s 2023-08-27T11:21:24.951 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:24.952 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:25.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:25.009 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1007 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:25.066 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:25.066 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:25.066 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9774e-05 s, 0.0 kB/s 2023-08-27T11:21:25.067 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:25.067 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:25.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:25.124 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1008 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:25.181 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:25.181 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:25.182 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.105e-05 s, 0.0 kB/s 2023-08-27T11:21:25.182 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:25.182 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:25.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:25.239 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1009 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:25.297 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:25.297 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:25.298 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9955e-05 s, 0.0 kB/s 2023-08-27T11:21:25.298 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:25.298 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:25.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:25.355 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1010 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:25.413 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:25.413 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:25.414 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0831e-05 s, 0.0 kB/s 2023-08-27T11:21:25.414 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:25.415 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:25.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:25.472 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1011 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:25.528 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:25.529 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:25.529 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8752e-05 s, 0.0 kB/s 2023-08-27T11:21:25.529 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:25.530 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:25.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:25.587 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1012 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:25.644 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:25.645 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:25.645 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9398e-05 s, 0.0 kB/s 2023-08-27T11:21:25.645 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:25.646 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:25.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:25.702 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1013 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:25.761 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:25.761 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:25.761 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5006e-05 s, 0.0 kB/s 2023-08-27T11:21:25.762 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:25.763 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:25.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:25.820 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1014 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:25.877 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:25.877 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:25.877 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2116e-05 s, 0.0 kB/s 2023-08-27T11:21:25.878 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:25.878 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:25.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:25.935 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1015 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:25.992 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:25.993 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:25.993 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1094e-05 s, 0.0 kB/s 2023-08-27T11:21:25.993 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:25.994 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:26.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:26.049 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1016 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0953e-05 s, 0.0 kB/s 2023-08-27T11:21:26.108 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:26.108 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:26.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:26.164 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1017 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:26.222 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:26.222 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:26.222 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9545e-05 s, 0.0 kB/s 2023-08-27T11:21:26.224 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:26.224 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:26.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:26.280 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1018 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:26.336 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6717e-05 s, 0.0 kB/s 2023-08-27T11:21:26.337 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:26.338 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:26.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:26.395 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1019 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:26.451 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:26.452 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:26.452 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2007e-05 s, 0.0 kB/s 2023-08-27T11:21:26.453 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:26.453 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:26.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:26.510 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1020 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.6165e-05 s, 0.0 kB/s 2023-08-27T11:21:26.569 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:26.570 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:26.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:26.626 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1021 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:26.683 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:26.684 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:26.684 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0052e-05 s, 0.0 kB/s 2023-08-27T11:21:26.684 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:26.685 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:26.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:26.742 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1022 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:26.798 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:26.799 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:26.799 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.752e-05 s, 0.0 kB/s 2023-08-27T11:21:26.800 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:26.800 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:26.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:26.857 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1023 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:26.914 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:26.914 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:26.915 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9925e-05 s, 0.0 kB/s 2023-08-27T11:21:26.915 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:26.915 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:26.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:26.971 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1024 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:27.028 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:27.029 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:27.029 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8646e-05 s, 0.0 kB/s 2023-08-27T11:21:27.029 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:27.030 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:27.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:27.087 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1025 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8246e-05 s, 0.0 kB/s 2023-08-27T11:21:27.145 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:27.146 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:27.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:27.202 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1026 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:27.260 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:27.260 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:27.260 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9343e-05 s, 0.0 kB/s 2023-08-27T11:21:27.261 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:27.261 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:27.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:27.318 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1027 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:27.380 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:27.381 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:27.381 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2567e-05 s, 0.0 kB/s 2023-08-27T11:21:27.382 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:27.382 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:27.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:27.442 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1028 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:27.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:27.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:27.499 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.018e-05 s, 0.0 kB/s 2023-08-27T11:21:27.499 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:27.500 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:27.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:27.557 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1029 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:27.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:27.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:27.614 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1591e-05 s, 0.0 kB/s 2023-08-27T11:21:27.615 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:27.615 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:27.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:27.672 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1030 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:27.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:27.730 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:27.730 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0954e-05 s, 0.0 kB/s 2023-08-27T11:21:27.730 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:27.731 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:27.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:27.788 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1031 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:27.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:27.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:27.845 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.805e-05 s, 0.0 kB/s 2023-08-27T11:21:27.846 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:27.847 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:27.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:27.904 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1032 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:27.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:27.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:27.962 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0632e-05 s, 0.0 kB/s 2023-08-27T11:21:27.963 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:27.963 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:28.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:28.020 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1033 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:28.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:28.078 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:28.078 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9953e-05 s, 0.0 kB/s 2023-08-27T11:21:28.078 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:28.079 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:28.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:28.136 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1034 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:28.192 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:28.193 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:28.193 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1693e-05 s, 0.0 kB/s 2023-08-27T11:21:28.193 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:28.194 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:28.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:28.251 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1035 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:28.310 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:28.310 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:28.310 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0024e-05 s, 0.0 kB/s 2023-08-27T11:21:28.311 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:28.311 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:28.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:28.368 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1036 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:28.425 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:28.426 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:28.426 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8162e-05 s, 0.0 kB/s 2023-08-27T11:21:28.427 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:28.427 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:28.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:28.484 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1037 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:28.541 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:28.542 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:28.542 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8874e-05 s, 0.0 kB/s 2023-08-27T11:21:28.542 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:28.543 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:28.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:28.601 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1038 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:28.658 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:28.659 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:28.659 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1738e-05 s, 0.0 kB/s 2023-08-27T11:21:28.659 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:28.659 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:28.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:28.716 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1039 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:28.772 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:28.773 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:28.773 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7549e-05 s, 0.0 kB/s 2023-08-27T11:21:28.773 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:28.774 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:28.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:28.831 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1040 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:28.889 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:28.889 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:28.889 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3162e-05 s, 0.0 kB/s 2023-08-27T11:21:28.890 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:28.890 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:28.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:28.947 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1041 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:29.003 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:29.004 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:29.004 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7738e-05 s, 0.0 kB/s 2023-08-27T11:21:29.004 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:29.005 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:29.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:29.062 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1042 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:29.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:29.121 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:29.121 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2642e-05 s, 0.0 kB/s 2023-08-27T11:21:29.121 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:29.122 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:29.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:29.178 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1043 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:29.236 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:29.236 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:29.236 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.839e-05 s, 0.0 kB/s 2023-08-27T11:21:29.236 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:29.237 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:29.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:29.294 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1044 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:29.351 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:29.352 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:29.352 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1031e-05 s, 0.0 kB/s 2023-08-27T11:21:29.352 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:29.353 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:29.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:29.410 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1045 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:29.467 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:29.467 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:29.468 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0019e-05 s, 0.0 kB/s 2023-08-27T11:21:29.468 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:29.468 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:29.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:29.525 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1046 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:29.581 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:29.581 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:29.582 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1959e-05 s, 0.0 kB/s 2023-08-27T11:21:29.582 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:29.582 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:29.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:29.639 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1047 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:29.696 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:29.697 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:29.697 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7967e-05 s, 0.0 kB/s 2023-08-27T11:21:29.697 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:29.697 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:29.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:29.754 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1048 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:29.810 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:29.810 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:29.811 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.087e-05 s, 0.0 kB/s 2023-08-27T11:21:29.811 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:29.811 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:29.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:29.868 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1049 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8647e-05 s, 0.0 kB/s 2023-08-27T11:21:29.925 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:29.926 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:29.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:29.983 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1050 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:30.040 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:30.041 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:30.041 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.799e-05 s, 0.0 kB/s 2023-08-27T11:21:30.041 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:30.042 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:30.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:30.099 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1051 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:30.155 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:30.155 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9896e-05 s, 0.0 kB/s 2023-08-27T11:21:30.156 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:30.156 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:30.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:30.213 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1052 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:30.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:30.270 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:30.270 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6951e-05 s, 0.0 kB/s 2023-08-27T11:21:30.270 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:30.271 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:30.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:30.328 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1053 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:30.385 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:30.385 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:30.386 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2016e-05 s, 0.0 kB/s 2023-08-27T11:21:30.386 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:30.387 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:30.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:30.445 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1054 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:30.502 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:30.502 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:30.502 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8677e-05 s, 0.0 kB/s 2023-08-27T11:21:30.502 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:30.503 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:30.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:30.560 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1055 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:30.618 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:30.618 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:30.619 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7902e-05 s, 0.0 kB/s 2023-08-27T11:21:30.619 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:30.619 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:30.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:30.675 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1056 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:30.733 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:30.733 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:30.733 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9437e-05 s, 0.0 kB/s 2023-08-27T11:21:30.734 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:30.734 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:30.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:30.791 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1057 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:30.847 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:30.848 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:30.848 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0983e-05 s, 0.0 kB/s 2023-08-27T11:21:30.848 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:30.849 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:30.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:30.905 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1058 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:30.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:30.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:30.962 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7015e-05 s, 0.0 kB/s 2023-08-27T11:21:30.963 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:30.963 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:31.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:31.020 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1059 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:31.076 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:31.076 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:31.076 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1363e-05 s, 0.0 kB/s 2023-08-27T11:21:31.077 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:31.077 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:31.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:31.134 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1060 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:31.190 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:31.191 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:31.191 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0251e-05 s, 0.0 kB/s 2023-08-27T11:21:31.191 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:31.192 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:31.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:31.249 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1061 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:31.306 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:31.306 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:31.307 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8678e-05 s, 0.0 kB/s 2023-08-27T11:21:31.307 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:31.307 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:31.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:31.364 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1062 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:31.422 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:31.422 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:31.422 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0666e-05 s, 0.0 kB/s 2023-08-27T11:21:31.423 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:31.423 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:31.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:31.480 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1063 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:31.538 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:31.538 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:31.538 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6928e-05 s, 0.0 kB/s 2023-08-27T11:21:31.539 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:31.539 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:31.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:31.596 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1064 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:31.654 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:31.654 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0895e-05 s, 0.0 kB/s 2023-08-27T11:21:31.654 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:31.655 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:31.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:31.711 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1065 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:31.769 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:31.769 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:31.769 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0215e-05 s, 0.0 kB/s 2023-08-27T11:21:31.770 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:31.770 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:31.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:31.827 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1066 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:31.883 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:31.884 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:31.884 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7892e-05 s, 0.0 kB/s 2023-08-27T11:21:31.884 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:31.885 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:31.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:31.942 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1067 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:31.999 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:32.000 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:32.000 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8833e-05 s, 0.0 kB/s 2023-08-27T11:21:32.000 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:32.001 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:32.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:32.058 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1068 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:32.115 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:32.115 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:32.115 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.792e-05 s, 0.0 kB/s 2023-08-27T11:21:32.116 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:32.116 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:32.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:32.173 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1069 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:32.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:32.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:32.230 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6798e-05 s, 0.0 kB/s 2023-08-27T11:21:32.231 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:32.231 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:32.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:32.288 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1070 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:32.345 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:32.346 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:32.346 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2989e-05 s, 0.0 kB/s 2023-08-27T11:21:32.346 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:32.347 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:32.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:32.404 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1071 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:32.462 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:32.462 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:32.462 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8529e-05 s, 0.0 kB/s 2023-08-27T11:21:32.463 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:32.463 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:32.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:32.519 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1072 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:32.577 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:32.577 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:32.577 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8537e-05 s, 0.0 kB/s 2023-08-27T11:21:32.578 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:32.578 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:32.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:32.635 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1073 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:32.691 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:32.692 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:32.692 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.06e-05 s, 0.0 kB/s 2023-08-27T11:21:32.692 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:32.693 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:32.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:32.749 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1074 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:32.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:32.806 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:32.806 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8756e-05 s, 0.0 kB/s 2023-08-27T11:21:32.806 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:32.807 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:32.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:32.863 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1075 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.904e-05 s, 0.0 kB/s 2023-08-27T11:21:32.921 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:32.922 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:32.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:32.978 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1076 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:33.035 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:33.036 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:33.036 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0132e-05 s, 0.0 kB/s 2023-08-27T11:21:33.036 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:33.037 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:33.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:33.093 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1077 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:33.150 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:33.151 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:33.151 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0204e-05 s, 0.0 kB/s 2023-08-27T11:21:33.151 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:33.152 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:33.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:33.209 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1078 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:33.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:33.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:33.266 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7346e-05 s, 0.0 kB/s 2023-08-27T11:21:33.266 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:33.266 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:33.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:33.323 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1079 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:33.381 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:33.381 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1621e-05 s, 0.0 kB/s 2023-08-27T11:21:33.382 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:33.383 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:33.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:33.439 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1080 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:33.497 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:33.497 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:33.497 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.734e-05 s, 0.0 kB/s 2023-08-27T11:21:33.498 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:33.498 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:33.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:33.555 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1081 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1717e-05 s, 0.0 kB/s 2023-08-27T11:21:33.613 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:33.616 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:33.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:33.672 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1082 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:33.730 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:33.730 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:33.730 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8969e-05 s, 0.0 kB/s 2023-08-27T11:21:33.731 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:33.731 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:33.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:33.787 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1083 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:33.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:33.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:33.846 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.651e-05 s, 0.0 kB/s 2023-08-27T11:21:33.846 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:33.846 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:33.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:33.908 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1084 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2134e-05 s, 0.0 kB/s 2023-08-27T11:21:33.967 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:33.967 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:34.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:34.024 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1085 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:34.080 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:34.081 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:34.081 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7324e-05 s, 0.0 kB/s 2023-08-27T11:21:34.081 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:34.082 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:34.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:34.142 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1086 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:34.199 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:34.199 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:34.199 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7245e-05 s, 0.0 kB/s 2023-08-27T11:21:34.200 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:34.200 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:34.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:34.258 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1087 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2311e-05 s, 0.0 kB/s 2023-08-27T11:21:34.316 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:34.316 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:34.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:34.373 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1088 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:34.430 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:34.431 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:34.431 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1265e-05 s, 0.0 kB/s 2023-08-27T11:21:34.431 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:34.432 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:34.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:34.489 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1089 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:34.546 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:34.547 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:34.547 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1121e-05 s, 0.0 kB/s 2023-08-27T11:21:34.547 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:34.548 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:34.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:34.605 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1090 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:34.662 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:34.662 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:34.662 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8882e-05 s, 0.0 kB/s 2023-08-27T11:21:34.663 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:34.663 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:34.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:34.720 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1091 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:34.776 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0783e-05 s, 0.0 kB/s 2023-08-27T11:21:34.777 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:34.778 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:34.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:34.835 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1092 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:34.891 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:34.893 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:34.893 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4534e-05 s, 0.0 kB/s 2023-08-27T11:21:34.893 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:34.894 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:34.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:34.951 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1093 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6583e-05 s, 0.0 kB/s 2023-08-27T11:21:35.009 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:35.009 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:35.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:35.066 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1094 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:35.123 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:35.123 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:35.124 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7224e-05 s, 0.0 kB/s 2023-08-27T11:21:35.124 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:35.124 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:35.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:35.181 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1095 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:35.239 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:35.239 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:35.239 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9976e-05 s, 0.0 kB/s 2023-08-27T11:21:35.240 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:35.240 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:35.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:35.298 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1096 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:35.356 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:35.356 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:35.356 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6853e-05 s, 0.0 kB/s 2023-08-27T11:21:35.356 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:35.357 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:35.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:35.414 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1097 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:35.470 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:35.470 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:35.470 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9826e-05 s, 0.0 kB/s 2023-08-27T11:21:35.472 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:35.472 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:35.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:35.530 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1098 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:35.587 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:35.587 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:35.587 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.008e-05 s, 0.0 kB/s 2023-08-27T11:21:35.587 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:35.588 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:35.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:35.645 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1099 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:35.702 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:35.703 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:35.704 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1478e-05 s, 0.0 kB/s 2023-08-27T11:21:35.704 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:35.704 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:35.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:35.761 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1100 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:35.818 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:35.819 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:35.819 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.815e-05 s, 0.0 kB/s 2023-08-27T11:21:35.819 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:35.820 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:35.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:35.876 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1101 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:35.932 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:35.932 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.972e-05 s, 0.0 kB/s 2023-08-27T11:21:35.934 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:35.934 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:35.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:35.993 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1102 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:36.050 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8349e-05 s, 0.0 kB/s 2023-08-27T11:21:36.051 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:36.052 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:36.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:36.108 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1103 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:36.131 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:21:36.151 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:21:36.165 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:36.165 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:36.165 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8392e-05 s, 0.0 kB/s 2023-08-27T11:21:36.166 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:36.166 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:36.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:36.223 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1104 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:36.279 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:36.280 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:36.280 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9754e-05 s, 0.0 kB/s 2023-08-27T11:21:36.280 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:36.281 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:36.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:36.338 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1105 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:36.394 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:36.395 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:36.395 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7144e-05 s, 0.0 kB/s 2023-08-27T11:21:36.395 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:36.396 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:36.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:36.452 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1106 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:36.509 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:36.510 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:36.510 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.979e-05 s, 0.0 kB/s 2023-08-27T11:21:36.511 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:36.511 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:36.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:36.568 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1107 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:36.627 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9481e-05 s, 0.0 kB/s 2023-08-27T11:21:36.627 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:36.627 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:36.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:36.684 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1108 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:36.740 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:36.741 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:36.741 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.552e-05 s, 0.0 kB/s 2023-08-27T11:21:36.741 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:36.742 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:36.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:36.799 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1109 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:36.855 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:36.856 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:36.856 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0096e-05 s, 0.0 kB/s 2023-08-27T11:21:36.856 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:36.857 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:36.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:36.913 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1110 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7056e-05 s, 0.0 kB/s 2023-08-27T11:21:36.973 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:36.973 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:37.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:37.029 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1111 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:37.087 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:37.087 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:37.087 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9444e-05 s, 0.0 kB/s 2023-08-27T11:21:37.088 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:37.088 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:37.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:37.145 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1112 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:37.201 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:37.202 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:37.202 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8819e-05 s, 0.0 kB/s 2023-08-27T11:21:37.205 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:37.206 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:37.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:37.259 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1113 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:37.316 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:37.316 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0289e-05 s, 0.0 kB/s 2023-08-27T11:21:37.316 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:37.317 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:37.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:37.374 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1114 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:37.431 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:37.431 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:37.431 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8948e-05 s, 0.0 kB/s 2023-08-27T11:21:37.432 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:37.432 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:37.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:37.489 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1115 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9989e-05 s, 0.0 kB/s 2023-08-27T11:21:37.547 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:37.547 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:37.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:37.606 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1116 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:37.664 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:37.664 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:37.665 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0068e-05 s, 0.0 kB/s 2023-08-27T11:21:37.665 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:37.665 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:37.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:37.735 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1117 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:37.792 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:37.793 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:37.793 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0691e-05 s, 0.0 kB/s 2023-08-27T11:21:37.793 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:37.794 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:37.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:37.851 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1118 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:37.913 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:37.913 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:37.913 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.103e-05 s, 0.0 kB/s 2023-08-27T11:21:37.914 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:37.915 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:37.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:37.974 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1119 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:38.040 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:38.040 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:38.040 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0616e-05 s, 0.0 kB/s 2023-08-27T11:21:38.040 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:38.041 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:38.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:38.110 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1120 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:38.176 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:38.177 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:38.177 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2487e-05 s, 0.0 kB/s 2023-08-27T11:21:38.177 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:38.179 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:38.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:38.238 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1121 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:38.305 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:38.306 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.126e-05 s, 0.0 kB/s 2023-08-27T11:21:38.307 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:38.308 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:38.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:38.366 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1122 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:38.430 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:38.430 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:38.431 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.856e-05 s, 0.0 kB/s 2023-08-27T11:21:38.431 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:38.432 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:38.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:38.492 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1123 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:38.555 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8633e-05 s, 0.0 kB/s 2023-08-27T11:21:38.556 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:38.557 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:38.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:38.613 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1124 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:38.674 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:38.674 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:38.674 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1921e-05 s, 0.0 kB/s 2023-08-27T11:21:38.674 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:38.675 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:38.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:38.735 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1125 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:38.795 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:38.795 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:38.795 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7931e-05 s, 0.0 kB/s 2023-08-27T11:21:38.796 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:38.797 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:38.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:38.856 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1126 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:38.916 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:38.917 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:38.917 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0021e-05 s, 0.0 kB/s 2023-08-27T11:21:38.917 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:38.918 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:38.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:38.978 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1127 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:39.037 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:39.038 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:39.038 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1831e-05 s, 0.0 kB/s 2023-08-27T11:21:39.038 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:39.039 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:39.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:39.096 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1128 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:39.154 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:39.154 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:39.154 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0822e-05 s, 0.0 kB/s 2023-08-27T11:21:39.155 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:39.155 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:39.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:39.212 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1129 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:39.268 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:39.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:39.269 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0649e-05 s, 0.0 kB/s 2023-08-27T11:21:39.269 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:39.270 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:39.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:39.326 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1130 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:39.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:39.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:39.383 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9552e-05 s, 0.0 kB/s 2023-08-27T11:21:39.384 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:39.384 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:39.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:39.442 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1131 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:39.498 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:39.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:39.499 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.832e-05 s, 0.0 kB/s 2023-08-27T11:21:39.499 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:39.500 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:39.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:39.557 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1132 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:39.613 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:39.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:39.614 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0736e-05 s, 0.0 kB/s 2023-08-27T11:21:39.614 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:39.614 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:39.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:39.671 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1133 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:39.728 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:39.728 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:39.728 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8446e-05 s, 0.0 kB/s 2023-08-27T11:21:39.729 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:39.729 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:39.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:39.785 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1134 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:39.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:39.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:39.842 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.983e-05 s, 0.0 kB/s 2023-08-27T11:21:39.843 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:39.843 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:39.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:39.899 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1135 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:39.956 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:39.957 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:39.957 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8897e-05 s, 0.0 kB/s 2023-08-27T11:21:39.957 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:39.958 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:40.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:40.014 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1136 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6654e-05 s, 0.0 kB/s 2023-08-27T11:21:40.072 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:40.072 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:40.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:40.130 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1137 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7828e-05 s, 0.0 kB/s 2023-08-27T11:21:40.191 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:40.191 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:40.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:40.248 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1138 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:40.305 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:40.305 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:40.306 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1261e-05 s, 0.0 kB/s 2023-08-27T11:21:40.306 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:40.307 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:40.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:40.363 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1139 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:40.421 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0343e-05 s, 0.0 kB/s 2023-08-27T11:21:40.421 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:40.421 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:40.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:40.481 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1140 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:40.537 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:40.537 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:40.537 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8368e-05 s, 0.0 kB/s 2023-08-27T11:21:40.538 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:40.538 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:40.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:40.595 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1141 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:40.652 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:40.652 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0949e-05 s, 0.0 kB/s 2023-08-27T11:21:40.652 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:40.653 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:40.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:40.710 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1142 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:40.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:40.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:40.768 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8673e-05 s, 0.0 kB/s 2023-08-27T11:21:40.769 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:40.769 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:40.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:40.826 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1143 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:40.884 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:40.884 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:40.884 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8854e-05 s, 0.0 kB/s 2023-08-27T11:21:40.885 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:40.885 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:40.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:40.942 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1144 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:41.000 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:41.000 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:41.000 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8161e-05 s, 0.0 kB/s 2023-08-27T11:21:41.001 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:41.001 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:41.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:41.058 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1145 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9407e-05 s, 0.0 kB/s 2023-08-27T11:21:41.117 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:41.118 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:41.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:41.173 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1146 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:41.231 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:41.231 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:41.231 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.782e-05 s, 0.0 kB/s 2023-08-27T11:21:41.231 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:41.232 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:41.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:41.290 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1147 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:41.346 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:41.346 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:41.347 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7366e-05 s, 0.0 kB/s 2023-08-27T11:21:41.347 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:41.348 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:41.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:41.404 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1148 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:41.462 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:41.462 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:41.462 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2143e-05 s, 0.0 kB/s 2023-08-27T11:21:41.463 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:41.463 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:41.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:41.520 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1149 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9258e-05 s, 0.0 kB/s 2023-08-27T11:21:41.580 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:41.580 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:41.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:41.636 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1150 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3678e-05 s, 0.0 kB/s 2023-08-27T11:21:41.695 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:41.696 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:41.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:41.752 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1151 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:41.810 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:41.810 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:41.810 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9674e-05 s, 0.0 kB/s 2023-08-27T11:21:41.811 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:41.811 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:41.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:41.868 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1152 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:41.925 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:41.925 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8633e-05 s, 0.0 kB/s 2023-08-27T11:21:41.925 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:41.926 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:41.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:41.983 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1153 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:42.041 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:42.041 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:42.041 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2049e-05 s, 0.0 kB/s 2023-08-27T11:21:42.042 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:42.042 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:42.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:42.099 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1154 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:42.157 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:42.157 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:42.157 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8487e-05 s, 0.0 kB/s 2023-08-27T11:21:42.158 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:42.158 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:42.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:42.215 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1155 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:42.272 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6113e-05 s, 0.0 kB/s 2023-08-27T11:21:42.273 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:42.274 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:42.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:42.331 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1156 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:42.389 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:42.389 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:42.389 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2405e-05 s, 0.0 kB/s 2023-08-27T11:21:42.390 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:42.390 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:42.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:42.447 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1157 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:42.504 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:42.504 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:42.505 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0758e-05 s, 0.0 kB/s 2023-08-27T11:21:42.505 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:42.505 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:42.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:42.562 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1158 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:42.618 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:42.619 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:42.619 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9842e-05 s, 0.0 kB/s 2023-08-27T11:21:42.619 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:42.620 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:42.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:42.677 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1159 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:42.735 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:42.735 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:42.735 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8804e-05 s, 0.0 kB/s 2023-08-27T11:21:42.736 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:42.736 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:42.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:42.793 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1160 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:42.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:42.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:42.850 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9958e-05 s, 0.0 kB/s 2023-08-27T11:21:42.851 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:42.851 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:42.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:42.908 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1161 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.968e-05 s, 0.0 kB/s 2023-08-27T11:21:42.976 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:42.976 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:43.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:43.039 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1162 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:43.103 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:43.103 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0315e-05 s, 0.0 kB/s 2023-08-27T11:21:43.104 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:43.105 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:43.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:43.167 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1163 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:43.226 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:43.227 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:43.227 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8631e-05 s, 0.0 kB/s 2023-08-27T11:21:43.228 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:43.228 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:43.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:43.286 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1164 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:43.348 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:43.349 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:43.349 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0556e-05 s, 0.0 kB/s 2023-08-27T11:21:43.350 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:43.351 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:43.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:43.408 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1165 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:43.466 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:43.466 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:43.467 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1641e-05 s, 0.0 kB/s 2023-08-27T11:21:43.467 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:43.467 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:43.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:43.524 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1166 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:43.590 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:43.590 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:43.591 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1159e-05 s, 0.0 kB/s 2023-08-27T11:21:43.591 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:43.591 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:43.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:43.655 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1167 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:43.716 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:43.717 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:43.717 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7212e-05 s, 0.0 kB/s 2023-08-27T11:21:43.717 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:43.718 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:43.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:43.781 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1168 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:43.843 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:43.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:43.845 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7755e-05 s, 0.0 kB/s 2023-08-27T11:21:43.845 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:43.846 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:43.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:43.907 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1169 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:43.966 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:43.966 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:43.967 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.997e-05 s, 0.0 kB/s 2023-08-27T11:21:43.967 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:43.968 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:44.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:44.046 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1170 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:44.106 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:44.107 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:44.107 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2315e-05 s, 0.0 kB/s 2023-08-27T11:21:44.107 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:44.108 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:44.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:44.166 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1171 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:44.229 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:44.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:44.230 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.84e-05 s, 0.0 kB/s 2023-08-27T11:21:44.230 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:44.231 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:44.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:44.290 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1172 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:44.348 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:44.348 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:44.349 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8656e-05 s, 0.0 kB/s 2023-08-27T11:21:44.349 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:44.350 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:44.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:44.407 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1173 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:44.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:44.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:44.468 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9886e-05 s, 0.0 kB/s 2023-08-27T11:21:44.469 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:44.469 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:44.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:44.526 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1174 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:44.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:44.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:44.584 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0659e-05 s, 0.0 kB/s 2023-08-27T11:21:44.584 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:44.584 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:44.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:44.641 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1175 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:44.698 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:44.699 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:44.699 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9765e-05 s, 0.0 kB/s 2023-08-27T11:21:44.699 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:44.700 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:44.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:44.756 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1176 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:44.813 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:44.813 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:44.813 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7575e-05 s, 0.0 kB/s 2023-08-27T11:21:44.814 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:44.814 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:44.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:44.874 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1177 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:44.931 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:44.931 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9783e-05 s, 0.0 kB/s 2023-08-27T11:21:44.932 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:44.932 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:44.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:44.989 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1178 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:45.047 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:45.047 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9379e-05 s, 0.0 kB/s 2023-08-27T11:21:45.047 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:45.048 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:45.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:45.106 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1179 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:45.163 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:45.163 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:45.163 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6242e-05 s, 0.0 kB/s 2023-08-27T11:21:45.164 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:45.165 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:45.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:45.222 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1180 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:45.278 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7698e-05 s, 0.0 kB/s 2023-08-27T11:21:45.279 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:45.280 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:45.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:45.337 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1181 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:45.395 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:45.395 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:45.395 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9355e-05 s, 0.0 kB/s 2023-08-27T11:21:45.396 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:45.396 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:45.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:45.452 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1182 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:45.509 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:45.509 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:45.510 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6972e-05 s, 0.0 kB/s 2023-08-27T11:21:45.510 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:45.510 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:45.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:45.567 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1183 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:45.626 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6932e-05 s, 0.0 kB/s 2023-08-27T11:21:45.626 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:45.626 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:45.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:45.683 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1184 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:45.740 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:45.740 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:45.740 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6658e-05 s, 0.0 kB/s 2023-08-27T11:21:45.741 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:45.742 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:45.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:45.798 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1185 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:45.856 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:45.856 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:45.856 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.991e-05 s, 0.0 kB/s 2023-08-27T11:21:45.857 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:45.857 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:45.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:45.913 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1186 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:45.970 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:45.971 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:45.971 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6806e-05 s, 0.0 kB/s 2023-08-27T11:21:45.971 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:45.972 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:46.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:46.029 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1187 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:46.087 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:46.087 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:46.087 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7667e-05 s, 0.0 kB/s 2023-08-27T11:21:46.088 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:46.088 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:46.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:46.146 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1188 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:46.204 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:46.204 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:46.204 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8219e-05 s, 0.0 kB/s 2023-08-27T11:21:46.205 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:46.205 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:46.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:46.262 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1189 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:46.319 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:46.319 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:46.319 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7698e-05 s, 0.0 kB/s 2023-08-27T11:21:46.320 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:46.320 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:46.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:46.379 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1190 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:46.434 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:46.435 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:46.435 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0777e-05 s, 0.0 kB/s 2023-08-27T11:21:46.435 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:46.435 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:46.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:46.492 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1191 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:46.548 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:46.549 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:46.549 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8331e-05 s, 0.0 kB/s 2023-08-27T11:21:46.549 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:46.550 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:46.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:46.607 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1192 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:46.664 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:46.664 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:46.664 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7454e-05 s, 0.0 kB/s 2023-08-27T11:21:46.665 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:46.665 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:46.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:46.722 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1193 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:46.780 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8464e-05 s, 0.0 kB/s 2023-08-27T11:21:46.780 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:46.780 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:46.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:46.838 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1194 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8792e-05 s, 0.0 kB/s 2023-08-27T11:21:46.896 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:46.897 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:46.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:46.954 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1195 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:47.010 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:47.011 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:47.011 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8589e-05 s, 0.0 kB/s 2023-08-27T11:21:47.011 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:47.012 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:47.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:47.069 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1196 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6592e-05 s, 0.0 kB/s 2023-08-27T11:21:47.127 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:47.128 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:47.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:47.184 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1197 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:47.241 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:47.241 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:47.241 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0061e-05 s, 0.0 kB/s 2023-08-27T11:21:47.242 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:47.242 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:47.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:47.301 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1198 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:47.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:47.359 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:47.359 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3263e-05 s, 0.0 kB/s 2023-08-27T11:21:47.359 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:47.360 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:47.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:47.416 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1199 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:47.473 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:47.474 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:47.474 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1414e-05 s, 0.0 kB/s 2023-08-27T11:21:47.474 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:47.475 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:47.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:47.531 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1200 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:47.589 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:47.590 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:47.590 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0178e-05 s, 0.0 kB/s 2023-08-27T11:21:47.591 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:47.591 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:47.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:47.649 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1201 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:47.706 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:47.707 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:47.707 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9872e-05 s, 0.0 kB/s 2023-08-27T11:21:47.708 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:47.709 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:47.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:47.767 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1202 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:47.823 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:47.823 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:47.824 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9224e-05 s, 0.0 kB/s 2023-08-27T11:21:47.824 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:47.824 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:47.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:47.883 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1203 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:47.940 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:47.940 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:47.941 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9787e-05 s, 0.0 kB/s 2023-08-27T11:21:47.941 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:47.942 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:47.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:47.998 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1204 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:48.054 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:48.055 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:48.055 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9359e-05 s, 0.0 kB/s 2023-08-27T11:21:48.055 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:48.056 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:48.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:48.112 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1205 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0075e-05 s, 0.0 kB/s 2023-08-27T11:21:48.171 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:48.172 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:48.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:48.228 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1206 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:48.286 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:48.286 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:48.286 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7902e-05 s, 0.0 kB/s 2023-08-27T11:21:48.287 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:48.287 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:48.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:48.344 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1207 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:48.400 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:48.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:48.401 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9679e-05 s, 0.0 kB/s 2023-08-27T11:21:48.401 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:48.402 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:48.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:48.459 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1208 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:48.515 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:48.516 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:48.516 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1445e-05 s, 0.0 kB/s 2023-08-27T11:21:48.516 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:48.517 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:48.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:48.574 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1209 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:48.631 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:48.631 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:48.631 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9072e-05 s, 0.0 kB/s 2023-08-27T11:21:48.632 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:48.632 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:48.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:48.689 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1210 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:48.746 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:48.746 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:48.746 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0999e-05 s, 0.0 kB/s 2023-08-27T11:21:48.747 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:48.747 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:48.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:48.805 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1211 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:48.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:48.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:48.866 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.978e-05 s, 0.0 kB/s 2023-08-27T11:21:48.867 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:48.868 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:48.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:48.924 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1212 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:48.981 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:48.981 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:48.981 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0355e-05 s, 0.0 kB/s 2023-08-27T11:21:48.984 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:48.984 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:49.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:49.039 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1213 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:49.096 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:49.096 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:49.096 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8844e-05 s, 0.0 kB/s 2023-08-27T11:21:49.096 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:49.097 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:49.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:49.155 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1214 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:49.213 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:49.213 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7864e-05 s, 0.0 kB/s 2023-08-27T11:21:49.214 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:49.215 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:49.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:49.271 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1215 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:49.328 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:49.329 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:49.330 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9719e-05 s, 0.0 kB/s 2023-08-27T11:21:49.331 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:49.331 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:49.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:49.387 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1216 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:49.444 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:49.445 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:49.445 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9921e-05 s, 0.0 kB/s 2023-08-27T11:21:49.445 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:49.446 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:49.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:49.504 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1217 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:49.561 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:49.561 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7735e-05 s, 0.0 kB/s 2023-08-27T11:21:49.562 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:49.562 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:49.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:49.620 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1218 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:49.677 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:49.677 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:49.677 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7101e-05 s, 0.0 kB/s 2023-08-27T11:21:49.678 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:49.678 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:49.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:49.735 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1219 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:49.793 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:49.794 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:49.794 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1182e-05 s, 0.0 kB/s 2023-08-27T11:21:49.794 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:49.795 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:49.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:49.851 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1220 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:49.910 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:49.910 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:49.910 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9861e-05 s, 0.0 kB/s 2023-08-27T11:21:49.911 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:49.911 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:49.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:49.968 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1221 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:50.024 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:50.025 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:50.025 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0629e-05 s, 0.0 kB/s 2023-08-27T11:21:50.026 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:50.026 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:50.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:50.085 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1222 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:50.141 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:50.142 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:50.142 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7896e-05 s, 0.0 kB/s 2023-08-27T11:21:50.142 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:50.143 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:50.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:50.199 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1223 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:50.257 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8864e-05 s, 0.0 kB/s 2023-08-27T11:21:50.257 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:50.257 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:50.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:50.314 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1224 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:50.372 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:50.372 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:50.372 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0931e-05 s, 0.0 kB/s 2023-08-27T11:21:50.373 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:50.373 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:50.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:50.439 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1225 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:50.496 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:50.496 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:50.496 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0981e-05 s, 0.0 kB/s 2023-08-27T11:21:50.497 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:50.497 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:50.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:50.556 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1226 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:50.612 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:50.612 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:50.613 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0235e-05 s, 0.0 kB/s 2023-08-27T11:21:50.613 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:50.613 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:50.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:50.671 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1227 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:50.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:50.730 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:50.730 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0016e-05 s, 0.0 kB/s 2023-08-27T11:21:50.730 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:50.731 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:50.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:50.788 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1228 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:50.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:50.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:50.845 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0545e-05 s, 0.0 kB/s 2023-08-27T11:21:50.846 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:50.846 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:50.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:50.903 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1229 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:50.960 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:50.961 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:50.961 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7011e-05 s, 0.0 kB/s 2023-08-27T11:21:50.961 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:50.962 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:51.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:51.019 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1230 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:51.075 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:51.076 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:51.076 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7616e-05 s, 0.0 kB/s 2023-08-27T11:21:51.077 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:51.077 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:51.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:51.133 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1231 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:51.191 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8233e-05 s, 0.0 kB/s 2023-08-27T11:21:51.191 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:51.191 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:51.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:51.248 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1232 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:51.306 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:51.306 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:51.307 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3892e-05 s, 0.0 kB/s 2023-08-27T11:21:51.307 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:51.308 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:51.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:51.364 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1233 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:51.422 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:51.422 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:51.422 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1054e-05 s, 0.0 kB/s 2023-08-27T11:21:51.423 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:51.423 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:51.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:51.480 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1234 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.7345e-05 s, 0.0 kB/s 2023-08-27T11:21:51.537 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:51.538 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:51.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:51.595 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1235 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:51.663 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:51.663 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:51.664 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4319e-05 s, 0.0 kB/s 2023-08-27T11:21:51.664 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:51.665 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:51.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:51.721 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1236 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8528e-05 s, 0.0 kB/s 2023-08-27T11:21:51.779 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:51.779 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:51.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:51.836 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1237 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:51.892 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:51.893 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:51.893 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8493e-05 s, 0.0 kB/s 2023-08-27T11:21:51.893 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:51.894 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:51.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:51.951 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1238 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:52.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:52.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:52.009 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1804e-05 s, 0.0 kB/s 2023-08-27T11:21:52.009 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:52.010 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:52.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:52.066 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1239 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:52.124 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:52.124 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:52.124 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7398e-05 s, 0.0 kB/s 2023-08-27T11:21:52.124 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:52.125 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:52.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:52.182 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1240 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:52.239 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:52.239 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:52.239 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1058e-05 s, 0.0 kB/s 2023-08-27T11:21:52.240 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:52.240 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:52.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:52.297 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1241 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:52.354 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:52.354 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:52.355 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0226e-05 s, 0.0 kB/s 2023-08-27T11:21:52.355 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:52.355 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:52.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:52.412 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1242 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:52.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:52.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:52.469 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1402e-05 s, 0.0 kB/s 2023-08-27T11:21:52.469 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:52.472 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:52.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:52.526 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1243 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:52.582 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:52.582 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:52.583 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8427e-05 s, 0.0 kB/s 2023-08-27T11:21:52.583 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:52.583 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:52.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:52.641 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1244 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:52.699 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:52.699 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1444e-05 s, 0.0 kB/s 2023-08-27T11:21:52.699 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:52.700 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:52.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:52.756 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1245 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:52.813 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:52.814 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:52.814 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0488e-05 s, 0.0 kB/s 2023-08-27T11:21:52.814 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:52.815 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:52.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:52.872 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1246 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:52.929 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:52.929 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:52.929 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.8471e-05 s, 0.0 kB/s 2023-08-27T11:21:52.930 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:52.930 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:52.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:52.987 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1247 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:53.044 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:53.044 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:53.044 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1026e-05 s, 0.0 kB/s 2023-08-27T11:21:53.045 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:53.045 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:53.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:53.103 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1248 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:53.160 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:53.161 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:53.161 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8389e-05 s, 0.0 kB/s 2023-08-27T11:21:53.161 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:53.162 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:53.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:53.218 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1249 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8899e-05 s, 0.0 kB/s 2023-08-27T11:21:53.276 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:53.276 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:53.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:53.334 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1250 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:53.393 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:53.393 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:53.393 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2343e-05 s, 0.0 kB/s 2023-08-27T11:21:53.394 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:53.394 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:53.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:53.450 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1251 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:53.508 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:53.508 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:53.508 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8809e-05 s, 0.0 kB/s 2023-08-27T11:21:53.509 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:53.509 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:53.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:53.566 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1252 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:53.624 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:53.624 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:53.624 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0719e-05 s, 0.0 kB/s 2023-08-27T11:21:53.625 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:53.625 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:53.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:53.682 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1253 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:53.739 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:53.739 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:53.740 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9339e-05 s, 0.0 kB/s 2023-08-27T11:21:53.740 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:53.740 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:53.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:53.797 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1254 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:53.853 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:53.854 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:53.854 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8111e-05 s, 0.0 kB/s 2023-08-27T11:21:53.854 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:53.855 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:53.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:53.913 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1255 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:53.970 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:53.971 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:53.971 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0642e-05 s, 0.0 kB/s 2023-08-27T11:21:53.971 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:53.972 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:54.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:54.028 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1256 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:54.085 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:54.085 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:54.085 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0861e-05 s, 0.0 kB/s 2023-08-27T11:21:54.086 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:54.086 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:54.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:54.144 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1257 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:54.201 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:54.201 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:54.202 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.308e-05 s, 0.0 kB/s 2023-08-27T11:21:54.202 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:54.203 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:54.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:54.259 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1258 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:54.315 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:54.316 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:54.316 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2856e-05 s, 0.0 kB/s 2023-08-27T11:21:54.316 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:54.317 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:54.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:54.375 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1259 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:54.432 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:54.432 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:54.432 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0899e-05 s, 0.0 kB/s 2023-08-27T11:21:54.432 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:54.433 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:54.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:54.491 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1260 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:54.548 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:54.548 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:54.548 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7624e-05 s, 0.0 kB/s 2023-08-27T11:21:54.549 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:54.549 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:54.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:54.606 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1261 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:54.664 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:54.664 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:54.664 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0577e-05 s, 0.0 kB/s 2023-08-27T11:21:54.664 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:54.665 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:54.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:54.723 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1262 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:54.780 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:54.780 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:54.781 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0113e-05 s, 0.0 kB/s 2023-08-27T11:21:54.781 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:54.781 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:54.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:54.839 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1263 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2629e-05 s, 0.0 kB/s 2023-08-27T11:21:54.898 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:54.898 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:54.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:54.955 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1264 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:55.012 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:55.012 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:55.012 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0695e-05 s, 0.0 kB/s 2023-08-27T11:21:55.013 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:55.013 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:55.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:55.070 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1265 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:55.129 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:55.129 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:55.130 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8346e-05 s, 0.0 kB/s 2023-08-27T11:21:55.130 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:55.130 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:55.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:55.187 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1266 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:55.244 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:55.244 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:55.245 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8506e-05 s, 0.0 kB/s 2023-08-27T11:21:55.245 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:55.247 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:55.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:55.302 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1267 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:55.360 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:55.360 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:55.360 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.974e-05 s, 0.0 kB/s 2023-08-27T11:21:55.361 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:55.361 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:55.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:55.418 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1268 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:55.476 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:55.476 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:55.477 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2436e-05 s, 0.0 kB/s 2023-08-27T11:21:55.477 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:55.478 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:55.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:55.534 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1269 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:55.591 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:55.592 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:55.592 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0576e-05 s, 0.0 kB/s 2023-08-27T11:21:55.592 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:55.593 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:55.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:55.650 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1270 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:55.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:55.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:55.708 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9399e-05 s, 0.0 kB/s 2023-08-27T11:21:55.709 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:55.709 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:55.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:55.766 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1271 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:55.826 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:55.826 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:55.826 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7845e-05 s, 0.0 kB/s 2023-08-27T11:21:55.827 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:55.827 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:55.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:55.884 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1272 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:55.941 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:55.942 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:55.942 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2519e-05 s, 0.0 kB/s 2023-08-27T11:21:55.942 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:55.943 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:55.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:55.999 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1273 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:56.055 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:56.055 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:56.056 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0694e-05 s, 0.0 kB/s 2023-08-27T11:21:56.056 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:56.056 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:56.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:56.112 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1274 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:56.170 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:56.170 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.7983e-05 s, 0.0 kB/s 2023-08-27T11:21:56.171 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:56.171 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:56.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:56.229 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1275 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:56.286 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:56.286 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:56.286 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2235e-05 s, 0.0 kB/s 2023-08-27T11:21:56.287 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:56.288 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:56.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:56.344 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1276 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:56.402 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:56.402 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:56.402 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7259e-05 s, 0.0 kB/s 2023-08-27T11:21:56.403 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:56.403 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:56.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:56.460 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1277 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:56.516 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:56.517 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:56.517 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9571e-05 s, 0.0 kB/s 2023-08-27T11:21:56.518 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:56.518 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:56.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:56.575 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1278 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:56.633 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:56.633 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:56.633 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8389e-05 s, 0.0 kB/s 2023-08-27T11:21:56.634 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:56.634 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:56.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:56.691 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1279 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:56.749 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:56.750 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:56.750 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0321e-05 s, 0.0 kB/s 2023-08-27T11:21:56.750 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:56.751 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:56.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:56.808 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1280 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7786e-05 s, 0.0 kB/s 2023-08-27T11:21:56.866 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:56.867 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:56.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:56.923 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1281 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:56.980 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:56.981 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:56.981 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.198e-05 s, 0.0 kB/s 2023-08-27T11:21:56.981 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:56.982 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:57.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:57.037 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1282 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:57.094 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:57.094 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0727e-05 s, 0.0 kB/s 2023-08-27T11:21:57.095 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:57.095 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:57.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:57.152 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1283 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:57.209 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:57.209 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9846e-05 s, 0.0 kB/s 2023-08-27T11:21:57.210 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:57.210 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:57.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:57.267 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1284 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:57.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:57.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:57.325 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7065e-05 s, 0.0 kB/s 2023-08-27T11:21:57.325 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:57.325 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:57.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:57.383 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1285 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:57.440 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:57.440 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0562e-05 s, 0.0 kB/s 2023-08-27T11:21:57.441 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:57.441 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:57.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:57.498 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1286 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:57.556 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:57.556 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:57.556 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1178e-05 s, 0.0 kB/s 2023-08-27T11:21:57.557 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:57.557 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:57.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:57.614 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1287 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1166e-05 s, 0.0 kB/s 2023-08-27T11:21:57.672 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:57.673 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:57.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:57.730 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1288 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:57.802 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:57.802 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:57.802 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5133e-05 s, 0.0 kB/s 2023-08-27T11:21:57.803 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:57.803 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:57.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:57.860 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1289 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:57.917 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:57.918 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:57.918 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8776e-05 s, 0.0 kB/s 2023-08-27T11:21:57.918 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:57.919 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:57.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:57.977 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1290 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:58.034 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:58.034 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:58.035 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.159e-05 s, 0.0 kB/s 2023-08-27T11:21:58.035 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:58.035 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:58.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:58.091 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1291 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:58.149 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:58.150 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:58.150 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0566e-05 s, 0.0 kB/s 2023-08-27T11:21:58.152 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:58.152 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:58.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:58.207 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1292 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:58.263 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:58.263 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.029e-05 s, 0.0 kB/s 2023-08-27T11:21:58.264 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:58.264 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:58.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:58.320 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1293 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:58.377 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:58.377 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:58.377 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2873e-05 s, 0.0 kB/s 2023-08-27T11:21:58.378 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:58.379 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:58.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:58.436 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1294 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:58.493 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:58.493 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:58.494 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6831e-05 s, 0.0 kB/s 2023-08-27T11:21:58.494 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:58.495 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:58.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:58.551 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1295 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8677e-05 s, 0.0 kB/s 2023-08-27T11:21:58.610 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:58.610 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:58.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:58.667 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1296 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:58.725 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:58.725 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:58.725 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8494e-05 s, 0.0 kB/s 2023-08-27T11:21:58.726 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:58.727 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:58.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:58.783 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1297 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:58.841 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:58.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:58.842 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8185e-05 s, 0.0 kB/s 2023-08-27T11:21:58.842 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:58.843 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:58.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:58.898 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1298 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:58.956 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:58.957 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:58.957 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2309e-05 s, 0.0 kB/s 2023-08-27T11:21:58.957 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:58.958 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:59.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:59.014 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1299 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:59.072 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:59.072 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:59.073 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7873e-05 s, 0.0 kB/s 2023-08-27T11:21:59.073 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:59.073 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:59.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:59.130 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1300 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:59.187 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:59.187 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:59.188 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8778e-05 s, 0.0 kB/s 2023-08-27T11:21:59.188 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:59.189 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:59.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:59.245 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1301 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:59.303 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:59.303 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:59.304 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6764e-05 s, 0.0 kB/s 2023-08-27T11:21:59.304 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:59.304 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:59.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:59.360 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1302 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:59.418 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:59.418 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:59.418 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.5281e-05 s, 0.0 kB/s 2023-08-27T11:21:59.419 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:59.419 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:59.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:59.477 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1303 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:59.534 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:59.534 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8552e-05 s, 0.0 kB/s 2023-08-27T11:21:59.534 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:59.535 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:59.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:59.592 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1304 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:59.649 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:59.649 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:59.650 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7691e-05 s, 0.0 kB/s 2023-08-27T11:21:59.650 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:59.650 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:59.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:59.707 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1305 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:59.765 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:59.765 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:59.766 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.823e-05 s, 0.0 kB/s 2023-08-27T11:21:59.766 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:59.766 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:59.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:59.824 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1306 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:59.880 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:59.880 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:59.881 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8944e-05 s, 0.0 kB/s 2023-08-27T11:21:59.882 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:59.882 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:21:59.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:21:59.939 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1307 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:21:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:21:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:21:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7811e-05 s, 0.0 kB/s 2023-08-27T11:21:59.998 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:21:59.998 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:00.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:00.055 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1308 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:00.113 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:00.113 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7748e-05 s, 0.0 kB/s 2023-08-27T11:22:00.114 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:00.114 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:00.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:00.172 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1309 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0742e-05 s, 0.0 kB/s 2023-08-27T11:22:00.229 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:00.230 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:00.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:00.285 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1310 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0439e-05 s, 0.0 kB/s 2023-08-27T11:22:00.343 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:00.343 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:00.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:00.400 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1311 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1552e-05 s, 0.0 kB/s 2023-08-27T11:22:00.458 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:00.458 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:00.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:00.515 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1312 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:00.572 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:00.572 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:00.572 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.841e-05 s, 0.0 kB/s 2023-08-27T11:22:00.573 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:00.573 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:00.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:00.630 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1313 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:00.687 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:00.688 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:00.688 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3204e-05 s, 0.0 kB/s 2023-08-27T11:22:00.688 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:00.689 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:00.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:00.746 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1314 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:00.803 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:00.804 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:00.804 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8203e-05 s, 0.0 kB/s 2023-08-27T11:22:00.804 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:00.805 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:00.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:00.862 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1315 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:00.919 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:00.919 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9956e-05 s, 0.0 kB/s 2023-08-27T11:22:00.920 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:00.920 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:00.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:00.976 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1316 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:01.034 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:01.034 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:01.034 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7602e-05 s, 0.0 kB/s 2023-08-27T11:22:01.035 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:01.035 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:01.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:01.092 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1317 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7478e-05 s, 0.0 kB/s 2023-08-27T11:22:01.149 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:01.149 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:01.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:01.206 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1318 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7559e-05 s, 0.0 kB/s 2023-08-27T11:22:01.264 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:01.264 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:01.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:01.322 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1319 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:01.380 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.166e-05 s, 0.0 kB/s 2023-08-27T11:22:01.382 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:01.382 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:01.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:01.439 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1320 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:01.496 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:01.497 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:01.497 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8838e-05 s, 0.0 kB/s 2023-08-27T11:22:01.497 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:01.498 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:01.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:01.555 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1321 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:01.611 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:01.612 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:01.612 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9937e-05 s, 0.0 kB/s 2023-08-27T11:22:01.612 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:01.613 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:01.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:01.670 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1322 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:01.727 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:01.728 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:01.728 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0209e-05 s, 0.0 kB/s 2023-08-27T11:22:01.729 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:01.730 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:01.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:01.786 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1323 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:01.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:01.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:01.843 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1945e-05 s, 0.0 kB/s 2023-08-27T11:22:01.843 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:01.843 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:01.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:01.901 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1324 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:01.957 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:01.958 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:01.958 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1355e-05 s, 0.0 kB/s 2023-08-27T11:22:01.959 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:01.959 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:02.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:02.015 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1325 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:02.072 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:02.072 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:02.072 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.052e-05 s, 0.0 kB/s 2023-08-27T11:22:02.073 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:02.073 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:02.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:02.131 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1326 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:02.188 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0861e-05 s, 0.0 kB/s 2023-08-27T11:22:02.188 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:02.188 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:02.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:02.244 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1327 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:02.301 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:02.301 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:02.302 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8836e-05 s, 0.0 kB/s 2023-08-27T11:22:02.302 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:02.302 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:02.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:02.359 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1328 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:02.417 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:02.417 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:02.418 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8801e-05 s, 0.0 kB/s 2023-08-27T11:22:02.418 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:02.418 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:02.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:02.474 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1329 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:02.531 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:02.531 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:02.532 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2936e-05 s, 0.0 kB/s 2023-08-27T11:22:02.532 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:02.532 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:02.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:02.588 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1330 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:02.646 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:02.647 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:02.647 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0355e-05 s, 0.0 kB/s 2023-08-27T11:22:02.647 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:02.648 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:02.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:02.704 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1331 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:02.761 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:02.762 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:02.762 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9982e-05 s, 0.0 kB/s 2023-08-27T11:22:02.763 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:02.763 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:02.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:02.821 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1332 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:02.878 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:02.878 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:02.879 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2077e-05 s, 0.0 kB/s 2023-08-27T11:22:02.879 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:02.879 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:02.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:02.936 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1333 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:02.993 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:02.994 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:02.994 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2146e-05 s, 0.0 kB/s 2023-08-27T11:22:02.995 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:02.996 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:03.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:03.052 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1334 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:03.108 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:03.109 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:03.109 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1926e-05 s, 0.0 kB/s 2023-08-27T11:22:03.109 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:03.110 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:03.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:03.167 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1335 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:03.224 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:03.224 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:03.224 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1182e-05 s, 0.0 kB/s 2023-08-27T11:22:03.225 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:03.225 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:03.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:03.282 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1336 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:03.339 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:03.339 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:03.339 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.3716e-05 s, 0.0 kB/s 2023-08-27T11:22:03.340 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:03.340 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:03.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:03.399 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1337 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:03.456 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:03.456 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:03.456 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8632e-05 s, 0.0 kB/s 2023-08-27T11:22:03.456 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:03.457 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:03.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:03.514 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1338 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:03.570 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:03.571 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:03.571 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0564e-05 s, 0.0 kB/s 2023-08-27T11:22:03.571 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:03.572 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:03.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:03.628 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1339 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:03.686 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:03.686 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:03.687 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9098e-05 s, 0.0 kB/s 2023-08-27T11:22:03.687 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:03.687 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:03.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:03.750 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1340 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:03.810 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:03.810 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0443e-05 s, 0.0 kB/s 2023-08-27T11:22:03.810 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:03.811 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:03.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:03.867 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1341 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:03.923 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:03.924 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:03.924 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1588e-05 s, 0.0 kB/s 2023-08-27T11:22:03.924 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:03.925 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:03.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:03.982 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1342 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:04.039 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:04.039 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:04.039 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0432e-05 s, 0.0 kB/s 2023-08-27T11:22:04.040 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:04.040 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:04.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:04.098 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1343 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1177e-05 s, 0.0 kB/s 2023-08-27T11:22:04.155 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:04.155 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:04.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:04.212 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1344 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1039e-05 s, 0.0 kB/s 2023-08-27T11:22:04.270 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:04.271 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:04.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:04.327 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1345 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:04.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:04.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:04.383 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8627e-05 s, 0.0 kB/s 2023-08-27T11:22:04.384 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:04.385 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:04.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:04.442 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1346 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:04.498 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0969e-05 s, 0.0 kB/s 2023-08-27T11:22:04.499 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:04.500 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:04.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:04.558 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1347 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9382e-05 s, 0.0 kB/s 2023-08-27T11:22:04.615 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:04.615 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:04.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:04.672 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1348 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.129e-05 s, 0.0 kB/s 2023-08-27T11:22:04.730 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:04.730 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:04.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:04.787 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1349 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:04.844 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:04.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:04.845 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9452e-05 s, 0.0 kB/s 2023-08-27T11:22:04.845 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:04.846 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:04.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:04.903 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1350 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:04.961 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:04.961 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:04.962 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8098e-05 s, 0.0 kB/s 2023-08-27T11:22:04.962 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:04.962 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:05.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:05.019 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1351 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:05.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:05.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:05.077 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9405e-05 s, 0.0 kB/s 2023-08-27T11:22:05.078 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:05.078 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:05.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:05.135 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1352 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8908e-05 s, 0.0 kB/s 2023-08-27T11:22:05.194 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:05.194 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:05.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:05.251 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1353 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:05.308 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:05.308 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7231e-05 s, 0.0 kB/s 2023-08-27T11:22:05.309 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:05.309 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:05.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:05.366 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1354 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:05.421 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8794e-05 s, 0.0 kB/s 2023-08-27T11:22:05.423 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:05.424 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:05.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:05.480 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1355 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:05.537 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:05.537 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:05.538 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0781e-05 s, 0.0 kB/s 2023-08-27T11:22:05.538 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:05.538 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:05.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:05.596 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1356 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:05.652 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:05.653 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:05.653 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8884e-05 s, 0.0 kB/s 2023-08-27T11:22:05.654 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:05.654 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:05.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:05.711 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1357 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:05.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:05.769 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:05.769 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8906e-05 s, 0.0 kB/s 2023-08-27T11:22:05.769 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:05.770 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:05.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:05.827 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1358 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:05.884 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:05.884 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:05.884 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0656e-05 s, 0.0 kB/s 2023-08-27T11:22:05.885 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:05.885 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:05.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:05.942 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1359 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:06.000 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:06.001 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:06.001 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7821e-05 s, 0.0 kB/s 2023-08-27T11:22:06.001 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:06.002 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:06.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:06.058 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1360 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:06.115 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:06.116 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:06.116 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7497e-05 s, 0.0 kB/s 2023-08-27T11:22:06.116 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:06.117 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:06.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:06.174 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1361 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:06.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:06.231 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:06.231 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8341e-05 s, 0.0 kB/s 2023-08-27T11:22:06.231 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:06.232 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:06.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:22:06.274 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:22:06.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:06.288 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1362 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:06.346 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:06.346 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:06.346 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9861e-05 s, 0.0 kB/s 2023-08-27T11:22:06.347 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:06.347 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:06.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:06.404 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1363 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:06.461 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:06.461 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:06.461 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2294e-05 s, 0.0 kB/s 2023-08-27T11:22:06.462 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:06.462 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:06.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:06.519 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1364 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:06.575 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:06.576 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:06.576 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8708e-05 s, 0.0 kB/s 2023-08-27T11:22:06.576 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:06.577 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:06.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:06.632 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1365 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:06.690 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7519e-05 s, 0.0 kB/s 2023-08-27T11:22:06.691 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:06.691 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:06.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:06.748 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1366 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:06.804 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:06.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:06.805 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9281e-05 s, 0.0 kB/s 2023-08-27T11:22:06.805 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:06.806 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:06.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:06.862 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1367 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:06.918 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:06.919 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:06.919 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8484e-05 s, 0.0 kB/s 2023-08-27T11:22:06.920 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:06.920 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:06.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:06.978 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1368 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:07.034 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:07.034 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:07.035 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9517e-05 s, 0.0 kB/s 2023-08-27T11:22:07.035 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:07.035 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:07.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:07.092 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1369 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:07.151 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1084e-05 s, 0.0 kB/s 2023-08-27T11:22:07.151 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:07.151 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:07.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:07.208 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1370 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:07.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:07.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:07.266 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6208e-05 s, 0.0 kB/s 2023-08-27T11:22:07.266 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:07.266 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:07.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:07.323 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1371 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9424e-05 s, 0.0 kB/s 2023-08-27T11:22:07.382 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:07.382 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:07.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:07.441 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1372 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:07.498 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:07.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:07.499 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9447e-05 s, 0.0 kB/s 2023-08-27T11:22:07.499 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:07.500 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:07.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:07.556 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1373 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:07.613 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:07.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:07.614 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1353e-05 s, 0.0 kB/s 2023-08-27T11:22:07.614 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:07.615 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:07.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:07.672 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1374 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1014e-05 s, 0.0 kB/s 2023-08-27T11:22:07.729 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:07.730 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:07.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:07.785 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1375 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:07.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:07.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:07.842 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9422e-05 s, 0.0 kB/s 2023-08-27T11:22:07.843 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:07.843 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:07.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:07.899 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1376 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:07.957 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:07.957 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:07.957 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0881e-05 s, 0.0 kB/s 2023-08-27T11:22:07.958 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:07.958 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:08.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:08.015 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1377 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8479e-05 s, 0.0 kB/s 2023-08-27T11:22:08.072 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:08.073 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:08.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:08.130 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1378 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:08.186 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:08.187 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:08.187 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8081e-05 s, 0.0 kB/s 2023-08-27T11:22:08.187 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:08.188 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:08.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:08.244 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1379 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:08.300 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:08.301 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:08.301 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3612e-05 s, 0.0 kB/s 2023-08-27T11:22:08.301 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:08.302 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:08.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:08.358 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1380 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:08.414 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:08.415 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:08.415 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7351e-05 s, 0.0 kB/s 2023-08-27T11:22:08.415 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:08.416 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:08.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:08.472 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1381 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:08.530 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:08.530 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:08.530 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8825e-05 s, 0.0 kB/s 2023-08-27T11:22:08.531 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:08.531 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:08.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:08.588 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1382 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:08.645 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:08.648 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:08.649 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.7006e-05 s, 0.0 kB/s 2023-08-27T11:22:08.649 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:08.650 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:08.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:08.702 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1383 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:08.760 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:08.760 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:08.760 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9329e-05 s, 0.0 kB/s 2023-08-27T11:22:08.761 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:08.761 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:08.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:08.829 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1384 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:08.890 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:08.891 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:08.891 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0276e-05 s, 0.0 kB/s 2023-08-27T11:22:08.892 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:08.892 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:08.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:08.961 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1385 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:09.036 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:09.037 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:09.037 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1026e-05 s, 0.0 kB/s 2023-08-27T11:22:09.038 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:09.038 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:09.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:09.100 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1386 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:09.162 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:09.162 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:09.163 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7833e-05 s, 0.0 kB/s 2023-08-27T11:22:09.164 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:09.164 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:09.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:09.227 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1387 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:09.287 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9487e-05 s, 0.0 kB/s 2023-08-27T11:22:09.288 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:09.289 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:09.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:09.348 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1388 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:09.408 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:09.408 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:09.409 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.79e-05 s, 0.0 kB/s 2023-08-27T11:22:09.409 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:09.409 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:09.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:09.471 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1389 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:09.529 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:09.529 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:09.529 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7461e-05 s, 0.0 kB/s 2023-08-27T11:22:09.530 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:09.530 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:09.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:09.587 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1390 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:09.646 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:09.646 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:09.646 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8371e-05 s, 0.0 kB/s 2023-08-27T11:22:09.647 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:09.647 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:09.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:09.705 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1391 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:09.763 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:09.763 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:09.763 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8228e-05 s, 0.0 kB/s 2023-08-27T11:22:09.764 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:09.764 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:09.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:09.821 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1392 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:09.882 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3487e-05 s, 0.0 kB/s 2023-08-27T11:22:09.882 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:09.883 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:09.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:09.943 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1393 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7404e-05 s, 0.0 kB/s 2023-08-27T11:22:10.003 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:10.004 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:10.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:10.062 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1394 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:10.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:10.121 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:10.121 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2747e-05 s, 0.0 kB/s 2023-08-27T11:22:10.122 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:10.122 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:10.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:10.182 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1395 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:10.239 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:10.239 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:10.239 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8179e-05 s, 0.0 kB/s 2023-08-27T11:22:10.240 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:10.240 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:10.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:10.298 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1396 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:10.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:10.359 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:10.359 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1109e-05 s, 0.0 kB/s 2023-08-27T11:22:10.359 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:10.360 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:10.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:10.416 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1397 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:10.473 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:10.474 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:10.474 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9307e-05 s, 0.0 kB/s 2023-08-27T11:22:10.474 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:10.474 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:10.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:10.531 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1398 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:10.588 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:10.589 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:10.589 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3349e-05 s, 0.0 kB/s 2023-08-27T11:22:10.589 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:10.590 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:10.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:10.647 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1399 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:10.705 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:10.705 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:10.705 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.906e-05 s, 0.0 kB/s 2023-08-27T11:22:10.706 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:10.706 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:10.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:10.763 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1400 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:10.821 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:10.821 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:10.821 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8856e-05 s, 0.0 kB/s 2023-08-27T11:22:10.823 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:10.823 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:10.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:10.879 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1401 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:10.936 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:10.936 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:10.936 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7169e-05 s, 0.0 kB/s 2023-08-27T11:22:10.937 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:10.937 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:10.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:10.994 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1402 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:11.050 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:11.051 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:11.051 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9299e-05 s, 0.0 kB/s 2023-08-27T11:22:11.051 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:11.052 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:11.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:11.107 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1403 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:11.165 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:11.165 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:11.165 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9946e-05 s, 0.0 kB/s 2023-08-27T11:22:11.165 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:11.166 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:11.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:11.222 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1404 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:11.279 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:11.279 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0098e-05 s, 0.0 kB/s 2023-08-27T11:22:11.280 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:11.280 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:11.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:11.338 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1405 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:11.395 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:11.395 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:11.395 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2259e-05 s, 0.0 kB/s 2023-08-27T11:22:11.396 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:11.396 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:11.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:11.453 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1406 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:11.509 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:11.509 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.204e-05 s, 0.0 kB/s 2023-08-27T11:22:11.510 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:11.510 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:11.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:11.567 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1407 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:11.624 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:11.624 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:11.624 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8225e-05 s, 0.0 kB/s 2023-08-27T11:22:11.625 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:11.625 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:11.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:11.683 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1408 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:11.740 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:11.740 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:11.740 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9558e-05 s, 0.0 kB/s 2023-08-27T11:22:11.740 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:11.741 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:11.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:11.798 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1409 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.849e-05 s, 0.0 kB/s 2023-08-27T11:22:11.855 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:11.855 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:11.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:11.912 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1410 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:11.969 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:11.969 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:11.969 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7355e-05 s, 0.0 kB/s 2023-08-27T11:22:11.970 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:11.970 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:12.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:12.027 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1411 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:12.084 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:12.084 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2025e-05 s, 0.0 kB/s 2023-08-27T11:22:12.085 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:12.085 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:12.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:12.142 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1412 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:12.200 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:12.201 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:12.201 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0973e-05 s, 0.0 kB/s 2023-08-27T11:22:12.201 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:12.202 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:12.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:12.258 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1413 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:12.317 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:12.318 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:12.318 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8707e-05 s, 0.0 kB/s 2023-08-27T11:22:12.318 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:12.319 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:12.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:12.377 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1414 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:12.433 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:12.433 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:12.433 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0121e-05 s, 0.0 kB/s 2023-08-27T11:22:12.434 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:12.434 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:12.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:12.492 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1415 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:12.548 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:12.549 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:12.549 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8388e-05 s, 0.0 kB/s 2023-08-27T11:22:12.550 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:12.550 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:12.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:12.607 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1416 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:12.664 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:12.665 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:12.665 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7544e-05 s, 0.0 kB/s 2023-08-27T11:22:12.665 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:12.666 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:12.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:12.723 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1417 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:12.780 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:12.780 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9523e-05 s, 0.0 kB/s 2023-08-27T11:22:12.780 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:12.781 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:12.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:12.837 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1418 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:12.894 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:12.894 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:12.894 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2537e-05 s, 0.0 kB/s 2023-08-27T11:22:12.895 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:12.895 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:12.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:12.952 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1419 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1454e-05 s, 0.0 kB/s 2023-08-27T11:22:13.011 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:13.011 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:13.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:13.070 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1420 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:13.127 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:13.127 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6833e-05 s, 0.0 kB/s 2023-08-27T11:22:13.127 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:13.128 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:13.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:13.185 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1421 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:13.242 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:13.242 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:13.242 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.96e-05 s, 0.0 kB/s 2023-08-27T11:22:13.243 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:13.243 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:13.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:13.300 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1422 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:13.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:13.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:13.358 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0444e-05 s, 0.0 kB/s 2023-08-27T11:22:13.359 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:13.359 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:13.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:13.417 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1423 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:13.474 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:13.474 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:13.474 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8498e-05 s, 0.0 kB/s 2023-08-27T11:22:13.475 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:13.475 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:13.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:13.533 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1424 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:13.590 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:13.590 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:13.590 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7331e-05 s, 0.0 kB/s 2023-08-27T11:22:13.591 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:13.591 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:13.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:13.649 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1425 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:13.707 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7369e-05 s, 0.0 kB/s 2023-08-27T11:22:13.708 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:13.709 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:13.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:13.765 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1426 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:13.821 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:13.821 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:13.822 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.974e-05 s, 0.0 kB/s 2023-08-27T11:22:13.822 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:13.822 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:13.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:13.880 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1427 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:13.936 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:13.937 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:13.937 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4688e-05 s, 0.0 kB/s 2023-08-27T11:22:13.937 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:13.938 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:13.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:13.994 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1428 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:14.052 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:14.052 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:14.053 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0029e-05 s, 0.0 kB/s 2023-08-27T11:22:14.053 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:14.053 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:14.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:14.111 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1429 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:14.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:14.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:14.169 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8739e-05 s, 0.0 kB/s 2023-08-27T11:22:14.170 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:14.170 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:14.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:14.236 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1430 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:14.258 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0401e-05 s, 0.0 kB/s 2023-08-27T11:22:14.260 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:14.261 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:14.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:14.326 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1431 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:14.389 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:14.389 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:14.390 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7229e-05 s, 0.0 kB/s 2023-08-27T11:22:14.390 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:14.390 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:14.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:14.449 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1432 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:14.508 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:14.508 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:14.508 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.189e-05 s, 0.0 kB/s 2023-08-27T11:22:14.510 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:14.510 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:14.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:14.570 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1433 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9425e-05 s, 0.0 kB/s 2023-08-27T11:22:14.628 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:14.629 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:14.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:14.686 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1434 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:14.744 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:14.744 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:14.744 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7093e-05 s, 0.0 kB/s 2023-08-27T11:22:14.745 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:14.745 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:14.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:14.802 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1435 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:14.860 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:14.860 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:14.860 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1627e-05 s, 0.0 kB/s 2023-08-27T11:22:14.861 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:14.861 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:14.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:14.918 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1436 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:14.975 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:14.975 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:14.975 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9773e-05 s, 0.0 kB/s 2023-08-27T11:22:14.976 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:14.976 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:15.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:15.033 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1437 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8167e-05 s, 0.0 kB/s 2023-08-27T11:22:15.091 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:15.091 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:15.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:15.148 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1438 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:15.204 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:15.205 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:15.205 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7233e-05 s, 0.0 kB/s 2023-08-27T11:22:15.205 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:15.206 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:15.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:15.263 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1439 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:15.322 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:15.322 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:15.322 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1085e-05 s, 0.0 kB/s 2023-08-27T11:22:15.322 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:15.323 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:15.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:15.381 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1440 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:15.437 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:15.437 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:15.437 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7909e-05 s, 0.0 kB/s 2023-08-27T11:22:15.438 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:15.439 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:15.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:15.495 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1441 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:15.552 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:15.552 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:15.552 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.082e-05 s, 0.0 kB/s 2023-08-27T11:22:15.553 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:15.553 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:15.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:15.611 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1442 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:15.670 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:15.670 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:15.670 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.07e-05 s, 0.0 kB/s 2023-08-27T11:22:15.671 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:15.671 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:15.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:15.728 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1443 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9837e-05 s, 0.0 kB/s 2023-08-27T11:22:15.786 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:15.787 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:15.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:15.845 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1444 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:15.902 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:15.902 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:15.902 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2217e-05 s, 0.0 kB/s 2023-08-27T11:22:15.903 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:15.903 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:15.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:15.959 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1445 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:16.016 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:16.017 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:16.017 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3629e-05 s, 0.0 kB/s 2023-08-27T11:22:16.017 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:16.018 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:16.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:16.074 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1446 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:16.130 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:16.130 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:16.131 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3473e-05 s, 0.0 kB/s 2023-08-27T11:22:16.131 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:16.131 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:16.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:16.188 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1447 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:16.244 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:16.245 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:16.245 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.932e-05 s, 0.0 kB/s 2023-08-27T11:22:16.245 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:16.246 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:16.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:16.303 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1448 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:16.361 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:16.361 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:16.361 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.26e-05 s, 0.0 kB/s 2023-08-27T11:22:16.362 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:16.362 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:16.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:16.419 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1449 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:16.476 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:16.476 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:16.476 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.39e-05 s, 0.0 kB/s 2023-08-27T11:22:16.477 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:16.477 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:16.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:16.533 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1450 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:16.590 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:16.591 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:16.591 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1953e-05 s, 0.0 kB/s 2023-08-27T11:22:16.591 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:16.592 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:16.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:16.648 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1451 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:16.705 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:16.705 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0812e-05 s, 0.0 kB/s 2023-08-27T11:22:16.705 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:16.706 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:16.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:16.762 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1452 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:16.818 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:16.819 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:16.819 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0244e-05 s, 0.0 kB/s 2023-08-27T11:22:16.819 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:16.820 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:16.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:16.876 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1453 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:16.934 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:16.935 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:16.935 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9681e-05 s, 0.0 kB/s 2023-08-27T11:22:16.935 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:16.936 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:16.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:16.993 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1454 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:17.049 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:17.050 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:17.050 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8267e-05 s, 0.0 kB/s 2023-08-27T11:22:17.050 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:17.051 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:17.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:17.108 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1455 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:17.165 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:17.165 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:17.165 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7385e-05 s, 0.0 kB/s 2023-08-27T11:22:17.166 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:17.166 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:17.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:17.223 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1456 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:17.280 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:17.280 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:17.280 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0976e-05 s, 0.0 kB/s 2023-08-27T11:22:17.281 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:17.281 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:17.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:17.338 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1457 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:17.396 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:17.397 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:17.397 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0309e-05 s, 0.0 kB/s 2023-08-27T11:22:17.397 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:17.398 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:17.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:17.455 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1458 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:17.513 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9834e-05 s, 0.0 kB/s 2023-08-27T11:22:17.513 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:17.513 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:17.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:17.571 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1459 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:17.628 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:17.628 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:17.629 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7618e-05 s, 0.0 kB/s 2023-08-27T11:22:17.629 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:17.629 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:17.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:17.686 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1460 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:17.743 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:17.744 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:17.744 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1605e-05 s, 0.0 kB/s 2023-08-27T11:22:17.744 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:17.744 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:17.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:17.801 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1461 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:17.857 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:17.858 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:17.858 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9512e-05 s, 0.0 kB/s 2023-08-27T11:22:17.858 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:17.859 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:17.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:17.916 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1462 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:17.972 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:17.973 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:17.973 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2404e-05 s, 0.0 kB/s 2023-08-27T11:22:17.973 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:17.974 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:18.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:18.031 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1463 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:18.087 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:18.088 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:18.088 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.947e-05 s, 0.0 kB/s 2023-08-27T11:22:18.088 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:18.089 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:18.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:18.144 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1464 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:18.202 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:18.202 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:18.202 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7766e-05 s, 0.0 kB/s 2023-08-27T11:22:18.203 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:18.203 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:18.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:18.260 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1465 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:18.317 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:18.318 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:18.318 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2604e-05 s, 0.0 kB/s 2023-08-27T11:22:18.318 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:18.319 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:18.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:18.376 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1466 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:18.433 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:18.434 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:18.434 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2504e-05 s, 0.0 kB/s 2023-08-27T11:22:18.434 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:18.435 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:18.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:18.492 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1467 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:18.549 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:18.550 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:18.550 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2011e-05 s, 0.0 kB/s 2023-08-27T11:22:18.551 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:18.551 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:18.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:18.607 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1468 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:18.663 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:18.664 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:18.664 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2957e-05 s, 0.0 kB/s 2023-08-27T11:22:18.664 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:18.665 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:18.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:18.721 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1469 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:18.778 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:18.778 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:18.778 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.601e-05 s, 0.0 kB/s 2023-08-27T11:22:18.779 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:18.779 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:18.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:18.836 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1470 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:18.892 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:18.893 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:18.893 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4119e-05 s, 0.0 kB/s 2023-08-27T11:22:18.894 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:18.894 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:18.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:18.951 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1471 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1205e-05 s, 0.0 kB/s 2023-08-27T11:22:19.009 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:19.009 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:19.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:19.067 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1472 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2168e-05 s, 0.0 kB/s 2023-08-27T11:22:19.126 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:19.126 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:19.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:19.183 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1473 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:19.240 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:19.240 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:19.241 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0489e-05 s, 0.0 kB/s 2023-08-27T11:22:19.241 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:19.241 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:19.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:19.324 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1474 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:19.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:19.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:19.358 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0545e-05 s, 0.0 kB/s 2023-08-27T11:22:19.359 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:19.359 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:19.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:19.416 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1475 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0114e-05 s, 0.0 kB/s 2023-08-27T11:22:19.474 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:19.474 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:19.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:19.534 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1476 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:19.590 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:19.590 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:19.591 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0326e-05 s, 0.0 kB/s 2023-08-27T11:22:19.591 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:19.591 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:19.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:19.648 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1477 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:19.705 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:19.705 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:19.705 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1876e-05 s, 0.0 kB/s 2023-08-27T11:22:19.706 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:19.706 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:19.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:19.763 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1478 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:19.822 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:19.822 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:19.822 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9784e-05 s, 0.0 kB/s 2023-08-27T11:22:19.823 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:19.823 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:19.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:19.881 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1479 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:19.938 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:19.938 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:19.939 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5689e-05 s, 0.0 kB/s 2023-08-27T11:22:19.939 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:19.940 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:19.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:19.996 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1480 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:20.052 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:20.053 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:20.053 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8278e-05 s, 0.0 kB/s 2023-08-27T11:22:20.053 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:20.054 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:20.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:20.111 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1481 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:20.168 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:20.168 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:20.168 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0583e-05 s, 0.0 kB/s 2023-08-27T11:22:20.169 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:20.169 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:20.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:20.226 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1482 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:20.285 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:20.285 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:20.285 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2373e-05 s, 0.0 kB/s 2023-08-27T11:22:20.286 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:20.286 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:20.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:20.343 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1483 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:20.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:20.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:20.401 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2358e-05 s, 0.0 kB/s 2023-08-27T11:22:20.402 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:20.402 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:20.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:20.459 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1484 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:20.518 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:20.518 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:20.519 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.117e-05 s, 0.0 kB/s 2023-08-27T11:22:20.519 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:20.520 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:20.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:20.577 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1485 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:20.633 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:20.634 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:20.634 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7893e-05 s, 0.0 kB/s 2023-08-27T11:22:20.634 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:20.635 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:20.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:20.692 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1486 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:20.750 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:20.750 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:20.750 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9168e-05 s, 0.0 kB/s 2023-08-27T11:22:20.751 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:20.751 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:20.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:20.808 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1487 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:20.865 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:20.866 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:20.866 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0157e-05 s, 0.0 kB/s 2023-08-27T11:22:20.866 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:20.867 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:20.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:20.923 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1488 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:20.979 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:20.979 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:20.979 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9401e-05 s, 0.0 kB/s 2023-08-27T11:22:20.979 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:20.981 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:21.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:21.037 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1489 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:21.094 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:21.095 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:21.095 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0002e-05 s, 0.0 kB/s 2023-08-27T11:22:21.095 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:21.096 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:21.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:21.153 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1490 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:21.209 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:21.210 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:21.210 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2068e-05 s, 0.0 kB/s 2023-08-27T11:22:21.210 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:21.210 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:21.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:21.267 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1491 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:21.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:21.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:21.324 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2901e-05 s, 0.0 kB/s 2023-08-27T11:22:21.325 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:21.325 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:21.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:21.384 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1492 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:21.441 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:21.442 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:21.442 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1593e-05 s, 0.0 kB/s 2023-08-27T11:22:21.442 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:21.443 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:21.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:21.499 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1493 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:21.557 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:21.557 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:21.558 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0287e-05 s, 0.0 kB/s 2023-08-27T11:22:21.558 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:21.558 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:21.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:21.615 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1494 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:21.672 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:21.672 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:21.672 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9695e-05 s, 0.0 kB/s 2023-08-27T11:22:21.673 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:21.673 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:21.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:21.730 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1495 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9881e-05 s, 0.0 kB/s 2023-08-27T11:22:21.790 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:21.790 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:21.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:21.847 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1496 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:21.904 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:21.904 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:21.905 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9652e-05 s, 0.0 kB/s 2023-08-27T11:22:21.905 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:21.905 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:21.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:21.962 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1497 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1273e-05 s, 0.0 kB/s 2023-08-27T11:22:22.022 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:22.022 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:22.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:22.079 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1498 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:22.136 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:22.136 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:22.137 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0666e-05 s, 0.0 kB/s 2023-08-27T11:22:22.137 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:22.138 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:22.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:22.193 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1499 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:22.251 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:22.251 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:22.252 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9295e-05 s, 0.0 kB/s 2023-08-27T11:22:22.252 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:22.252 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:22.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:22.309 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1500 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:22.367 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:22.367 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:22.367 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2888e-05 s, 0.0 kB/s 2023-08-27T11:22:22.368 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:22.368 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:22.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:22.425 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1501 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.273e-05 s, 0.0 kB/s 2023-08-27T11:22:22.483 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:22.483 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:22.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:22.540 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1502 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:22.597 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:22.598 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:22.598 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.4664e-05 s, 0.0 kB/s 2023-08-27T11:22:22.598 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:22.599 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:22.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:22.656 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1503 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:22.712 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:22.713 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:22.713 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0395e-05 s, 0.0 kB/s 2023-08-27T11:22:22.713 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:22.714 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:22.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:22.769 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1504 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:22.827 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:22.827 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:22.828 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.821e-05 s, 0.0 kB/s 2023-08-27T11:22:22.828 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:22.828 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:22.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:22.885 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1505 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:22.943 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:22.943 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:22.943 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7887e-05 s, 0.0 kB/s 2023-08-27T11:22:22.944 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:22.944 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:23.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:23.001 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1506 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9329e-05 s, 0.0 kB/s 2023-08-27T11:22:23.059 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:23.059 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:23.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:23.116 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1507 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:23.172 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:23.173 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:23.173 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1726e-05 s, 0.0 kB/s 2023-08-27T11:22:23.174 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:23.174 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:23.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:23.233 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1508 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:23.290 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:23.290 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:23.290 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7932e-05 s, 0.0 kB/s 2023-08-27T11:22:23.291 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:23.291 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:23.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:23.348 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1509 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:23.405 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:23.405 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:23.406 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0828e-05 s, 0.0 kB/s 2023-08-27T11:22:23.406 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:23.406 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:23.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:23.463 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1510 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:23.519 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:23.520 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:23.520 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0765e-05 s, 0.0 kB/s 2023-08-27T11:22:23.520 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:23.521 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:23.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:23.578 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1511 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:23.637 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:23.637 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:23.638 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1912e-05 s, 0.0 kB/s 2023-08-27T11:22:23.638 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:23.638 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:23.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:23.695 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1512 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:23.751 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:23.752 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:23.752 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9096e-05 s, 0.0 kB/s 2023-08-27T11:22:23.752 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:23.753 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:23.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:23.810 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1513 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:23.867 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:23.867 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:23.868 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7284e-05 s, 0.0 kB/s 2023-08-27T11:22:23.868 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:23.868 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:23.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:23.924 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1514 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:23.981 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:23.982 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:23.982 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7258e-05 s, 0.0 kB/s 2023-08-27T11:22:23.982 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:23.983 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:24.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:24.039 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1515 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:24.095 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:24.096 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:24.096 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6688e-05 s, 0.0 kB/s 2023-08-27T11:22:24.097 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:24.098 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:24.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:24.157 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1516 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9162e-05 s, 0.0 kB/s 2023-08-27T11:22:24.215 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:24.216 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:24.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:24.271 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1517 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:24.329 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:24.329 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:24.329 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9571e-05 s, 0.0 kB/s 2023-08-27T11:22:24.330 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:24.330 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:24.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:24.387 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1518 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:24.444 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:24.444 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:24.444 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9876e-05 s, 0.0 kB/s 2023-08-27T11:22:24.445 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:24.445 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:24.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:24.502 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1519 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:24.558 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:24.559 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:24.559 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9129e-05 s, 0.0 kB/s 2023-08-27T11:22:24.559 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:24.560 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:24.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:24.616 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1520 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:24.674 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8964e-05 s, 0.0 kB/s 2023-08-27T11:22:24.674 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:24.675 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:24.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:24.731 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1521 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:24.787 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:24.787 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:24.787 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9893e-05 s, 0.0 kB/s 2023-08-27T11:22:24.788 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:24.788 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:24.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:24.845 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1522 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:24.902 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:24.902 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:24.902 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7999e-05 s, 0.0 kB/s 2023-08-27T11:22:24.903 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:24.903 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:24.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:24.960 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1523 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:25.017 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:25.017 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:25.017 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7719e-05 s, 0.0 kB/s 2023-08-27T11:22:25.018 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:25.018 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:25.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:25.076 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1524 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:25.133 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:25.134 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:25.134 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2119e-05 s, 0.0 kB/s 2023-08-27T11:22:25.134 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:25.135 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:25.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:25.192 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1525 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:25.250 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:25.250 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9418e-05 s, 0.0 kB/s 2023-08-27T11:22:25.250 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:25.251 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:25.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:25.307 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1526 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:25.363 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:25.364 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:25.364 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8288e-05 s, 0.0 kB/s 2023-08-27T11:22:25.364 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:25.365 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:25.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:25.422 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1527 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:25.481 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:25.481 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:25.482 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9414e-05 s, 0.0 kB/s 2023-08-27T11:22:25.482 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:25.483 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:25.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:25.540 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1528 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2595e-05 s, 0.0 kB/s 2023-08-27T11:22:25.598 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:25.598 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:25.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:25.654 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1529 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:25.712 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:25.712 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:25.713 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2116e-05 s, 0.0 kB/s 2023-08-27T11:22:25.713 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:25.713 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:25.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:25.770 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1530 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:25.828 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:25.828 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:25.828 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8644e-05 s, 0.0 kB/s 2023-08-27T11:22:25.829 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:25.829 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:25.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:25.886 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1531 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:25.942 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:25.942 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:25.943 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.086e-05 s, 0.0 kB/s 2023-08-27T11:22:25.943 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:25.943 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:25.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:26.000 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1532 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.156e-05 s, 0.0 kB/s 2023-08-27T11:22:26.058 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:26.058 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:26.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:26.114 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1533 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:26.171 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2631e-05 s, 0.0 kB/s 2023-08-27T11:22:26.172 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:26.173 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:26.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:26.230 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1534 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9842e-05 s, 0.0 kB/s 2023-08-27T11:22:26.287 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:26.288 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:26.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:26.345 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1535 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:26.402 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:26.402 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:26.402 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0679e-05 s, 0.0 kB/s 2023-08-27T11:22:26.403 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:26.403 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:26.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:26.460 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1536 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:26.516 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:26.517 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:26.517 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6232e-05 s, 0.0 kB/s 2023-08-27T11:22:26.517 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:26.518 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:26.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:26.577 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1537 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:26.633 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:26.633 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:26.633 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9491e-05 s, 0.0 kB/s 2023-08-27T11:22:26.634 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:26.634 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:26.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:26.691 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1538 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:26.749 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:26.749 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:26.749 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8878e-05 s, 0.0 kB/s 2023-08-27T11:22:26.750 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:26.750 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:26.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:26.807 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1539 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:26.863 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:26.863 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:26.864 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9056e-05 s, 0.0 kB/s 2023-08-27T11:22:26.864 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:26.864 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:26.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:26.921 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1540 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:26.978 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:26.978 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:26.979 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1319e-05 s, 0.0 kB/s 2023-08-27T11:22:26.979 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:26.979 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:27.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:27.036 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1541 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:27.092 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:27.093 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:27.093 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8829e-05 s, 0.0 kB/s 2023-08-27T11:22:27.093 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:27.094 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:27.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:27.150 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1542 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:27.207 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:27.208 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:27.208 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9877e-05 s, 0.0 kB/s 2023-08-27T11:22:27.208 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:27.209 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:27.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:27.266 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1543 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:27.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:27.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:27.324 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2734e-05 s, 0.0 kB/s 2023-08-27T11:22:27.325 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:27.325 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:27.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:27.382 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1544 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:27.438 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:27.438 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:27.439 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.859e-05 s, 0.0 kB/s 2023-08-27T11:22:27.439 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:27.439 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:27.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:27.496 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1545 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:27.554 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:27.554 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:27.554 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8875e-05 s, 0.0 kB/s 2023-08-27T11:22:27.555 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:27.555 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:27.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:27.612 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1546 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:27.667 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:27.668 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:27.668 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2313e-05 s, 0.0 kB/s 2023-08-27T11:22:27.669 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:27.670 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:27.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:27.731 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1547 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:27.789 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:27.789 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:27.790 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1985e-05 s, 0.0 kB/s 2023-08-27T11:22:27.790 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:27.790 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:27.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:27.847 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1548 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:27.903 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:27.904 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:27.904 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1234e-05 s, 0.0 kB/s 2023-08-27T11:22:27.904 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:27.905 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:27.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:27.962 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1549 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:28.019 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0815e-05 s, 0.0 kB/s 2023-08-27T11:22:28.019 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:28.019 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:28.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:28.076 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1550 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:28.132 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:28.133 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:28.133 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1955e-05 s, 0.0 kB/s 2023-08-27T11:22:28.133 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:28.134 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:28.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:28.190 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1551 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:28.247 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:28.247 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:28.248 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9845e-05 s, 0.0 kB/s 2023-08-27T11:22:28.248 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:28.248 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:28.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:28.305 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1552 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:28.363 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:28.363 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:28.363 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1548e-05 s, 0.0 kB/s 2023-08-27T11:22:28.363 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:28.364 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:28.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:28.421 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1553 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:28.479 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:28.479 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:28.479 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2261e-05 s, 0.0 kB/s 2023-08-27T11:22:28.480 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:28.480 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:28.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:28.537 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1554 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:28.593 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:28.594 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:28.594 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5212e-05 s, 0.0 kB/s 2023-08-27T11:22:28.594 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:28.595 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:28.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:28.652 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1555 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:28.709 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:28.710 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:28.710 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8779e-05 s, 0.0 kB/s 2023-08-27T11:22:28.710 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:28.710 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:28.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:28.767 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1556 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:28.824 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:28.825 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:28.825 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0705e-05 s, 0.0 kB/s 2023-08-27T11:22:28.825 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:28.826 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:28.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:28.886 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1557 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:28.943 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:28.944 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:28.944 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0371e-05 s, 0.0 kB/s 2023-08-27T11:22:28.944 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:28.945 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:29.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:29.002 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1558 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:29.060 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:29.060 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:29.060 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8941e-05 s, 0.0 kB/s 2023-08-27T11:22:29.061 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:29.061 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:29.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:29.118 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1559 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.097e-05 s, 0.0 kB/s 2023-08-27T11:22:29.177 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:29.177 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:29.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:29.234 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1560 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:29.291 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:29.291 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:29.291 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1315e-05 s, 0.0 kB/s 2023-08-27T11:22:29.292 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:29.292 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:29.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:29.349 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1561 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:29.407 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:29.407 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:29.407 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9135e-05 s, 0.0 kB/s 2023-08-27T11:22:29.408 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:29.408 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:29.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:29.465 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1562 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:29.522 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:29.523 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:29.523 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5302e-05 s, 0.0 kB/s 2023-08-27T11:22:29.523 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:29.524 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:29.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:29.582 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1563 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:29.640 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:29.640 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:29.640 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9808e-05 s, 0.0 kB/s 2023-08-27T11:22:29.640 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:29.641 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:29.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:29.698 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1564 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:29.754 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:29.755 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:29.755 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1613e-05 s, 0.0 kB/s 2023-08-27T11:22:29.755 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:29.756 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:29.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:29.813 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1565 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:29.870 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:29.871 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:29.871 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8466e-05 s, 0.0 kB/s 2023-08-27T11:22:29.871 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:29.872 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:29.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:29.929 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1566 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:29.987 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:29.987 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:29.987 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0088e-05 s, 0.0 kB/s 2023-08-27T11:22:29.988 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:29.988 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:30.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:30.045 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1567 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2029e-05 s, 0.0 kB/s 2023-08-27T11:22:30.102 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:30.102 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:30.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:30.159 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1568 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:30.217 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:30.217 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:30.217 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1034e-05 s, 0.0 kB/s 2023-08-27T11:22:30.218 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:30.218 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:30.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:30.275 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1569 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:30.333 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:30.333 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:30.333 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.021e-05 s, 0.0 kB/s 2023-08-27T11:22:30.334 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:30.334 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:30.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:30.391 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1570 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:30.448 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:30.449 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:30.449 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8324e-05 s, 0.0 kB/s 2023-08-27T11:22:30.449 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:30.450 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:30.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:30.507 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1571 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:30.565 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:30.566 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:30.566 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2032e-05 s, 0.0 kB/s 2023-08-27T11:22:30.566 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:30.567 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:30.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:30.623 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1572 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:30.680 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:30.680 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:30.681 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0914e-05 s, 0.0 kB/s 2023-08-27T11:22:30.681 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:30.681 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:30.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:30.738 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1573 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:30.795 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:30.795 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:30.795 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.125e-05 s, 0 B/s 2023-08-27T11:22:30.795 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:30.796 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:30.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:30.854 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1574 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1639e-05 s, 0.0 kB/s 2023-08-27T11:22:30.912 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:30.913 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:30.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:30.969 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1575 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:31.025 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:31.026 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:31.026 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9632e-05 s, 0.0 kB/s 2023-08-27T11:22:31.026 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:31.027 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:31.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:31.083 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1576 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:31.140 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:31.141 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:31.141 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2009e-05 s, 0.0 kB/s 2023-08-27T11:22:31.141 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:31.142 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:31.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:31.199 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1577 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:31.256 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:31.257 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:31.257 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.7671e-05 s, 0.0 kB/s 2023-08-27T11:22:31.257 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:31.258 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:31.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:31.315 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1578 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:31.373 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:31.373 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:31.374 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3428e-05 s, 0.0 kB/s 2023-08-27T11:22:31.374 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:31.374 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:31.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:31.431 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1579 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:31.488 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:31.488 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:31.488 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2719e-05 s, 0.0 kB/s 2023-08-27T11:22:31.489 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:31.489 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:31.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:31.548 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1580 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:31.604 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:31.605 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:31.605 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0418e-05 s, 0.0 kB/s 2023-08-27T11:22:31.606 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:31.606 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:31.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:31.662 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1581 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:31.719 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:31.719 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:31.719 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.928e-05 s, 0.0 kB/s 2023-08-27T11:22:31.720 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:31.720 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:31.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:31.777 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1582 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:31.834 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:31.834 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:31.834 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7035e-05 s, 0.0 kB/s 2023-08-27T11:22:31.835 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:31.835 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:31.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:31.891 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1583 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:31.950 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:31.950 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:31.950 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.842e-05 s, 0.0 kB/s 2023-08-27T11:22:31.951 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:31.952 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:32.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:32.008 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1584 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:32.065 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:32.066 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:32.066 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7592e-05 s, 0.0 kB/s 2023-08-27T11:22:32.066 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:32.067 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:32.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:32.124 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1585 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:32.180 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:32.180 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:32.181 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0539e-05 s, 0.0 kB/s 2023-08-27T11:22:32.181 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:32.181 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:32.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:32.239 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1586 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5823e-05 s, 0.0 kB/s 2023-08-27T11:22:32.299 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:32.299 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:32.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:32.357 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1587 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:32.414 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:32.415 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:32.415 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1501e-05 s, 0.0 kB/s 2023-08-27T11:22:32.416 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:32.416 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:32.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:32.472 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1588 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:32.528 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:32.529 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:32.529 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8467e-05 s, 0.0 kB/s 2023-08-27T11:22:32.529 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:32.530 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:32.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:32.589 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1589 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:32.649 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:32.649 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:32.650 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.003e-05 s, 0.0 kB/s 2023-08-27T11:22:32.650 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:32.650 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:32.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:32.706 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1590 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:32.763 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:32.763 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:32.764 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0557e-05 s, 0.0 kB/s 2023-08-27T11:22:32.764 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:32.764 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:32.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:32.820 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1591 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:32.877 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:32.877 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8334e-05 s, 0.0 kB/s 2023-08-27T11:22:32.878 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:32.878 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:32.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:32.935 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1592 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:32.992 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:32.992 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:32.992 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9395e-05 s, 0.0 kB/s 2023-08-27T11:22:32.993 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:32.993 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:33.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:33.050 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1593 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:33.106 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:33.107 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:33.107 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9607e-05 s, 0.0 kB/s 2023-08-27T11:22:33.107 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:33.108 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:33.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:33.165 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1594 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:33.221 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:33.222 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:33.222 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8763e-05 s, 0.0 kB/s 2023-08-27T11:22:33.222 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:33.223 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:33.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:33.279 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1595 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:33.335 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:33.335 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:33.335 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7701e-05 s, 0.0 kB/s 2023-08-27T11:22:33.336 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:33.336 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:33.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:33.393 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1596 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8092e-05 s, 0.0 kB/s 2023-08-27T11:22:33.451 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:33.451 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:33.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:33.508 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1597 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:33.564 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:33.564 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:33.565 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1672e-05 s, 0.0 kB/s 2023-08-27T11:22:33.565 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:33.565 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:33.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:33.622 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1598 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7614e-05 s, 0.0 kB/s 2023-08-27T11:22:33.681 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:33.682 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:33.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:33.738 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1599 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:33.796 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:33.796 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:33.796 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8059e-05 s, 0.0 kB/s 2023-08-27T11:22:33.797 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:33.797 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:33.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:33.855 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1600 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:33.913 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:33.913 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:33.913 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9187e-05 s, 0.0 kB/s 2023-08-27T11:22:33.914 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:33.914 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:33.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:33.971 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1601 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:34.028 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:34.029 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:34.029 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3393e-05 s, 0.0 kB/s 2023-08-27T11:22:34.029 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:34.030 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:34.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:34.089 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1602 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:34.147 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:34.147 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:34.147 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.9267e-05 s, 0.0 kB/s 2023-08-27T11:22:34.148 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:34.148 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:34.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:34.205 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1603 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:34.263 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:34.263 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:34.263 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2048e-05 s, 0.0 kB/s 2023-08-27T11:22:34.264 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:34.264 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:34.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:34.321 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1604 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:34.378 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9858e-05 s, 0.0 kB/s 2023-08-27T11:22:34.379 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:34.380 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:34.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:34.437 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1605 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:34.494 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:34.495 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:34.495 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9022e-05 s, 0.0 kB/s 2023-08-27T11:22:34.495 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:34.496 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:34.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:34.553 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1606 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0545e-05 s, 0.0 kB/s 2023-08-27T11:22:34.611 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:34.612 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:34.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:34.668 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1607 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:34.724 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:34.725 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:34.725 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8072e-05 s, 0.0 kB/s 2023-08-27T11:22:34.725 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:34.726 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:34.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:34.784 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1608 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:34.840 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:34.840 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:34.841 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4223e-05 s, 0.0 kB/s 2023-08-27T11:22:34.841 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:34.841 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:34.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:34.898 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1609 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:34.954 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:34.955 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:34.955 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8036e-05 s, 0.0 kB/s 2023-08-27T11:22:34.955 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:34.956 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:35.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:35.012 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1610 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:35.069 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:35.069 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:35.070 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9929e-05 s, 0.0 kB/s 2023-08-27T11:22:35.070 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:35.071 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:35.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:35.128 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1611 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:35.184 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:35.184 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:35.185 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1044e-05 s, 0.0 kB/s 2023-08-27T11:22:35.185 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:35.186 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:35.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:35.242 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1612 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:35.299 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:35.300 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:35.300 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0079e-05 s, 0.0 kB/s 2023-08-27T11:22:35.301 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:35.301 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:35.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:35.358 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1613 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1327e-05 s, 0.0 kB/s 2023-08-27T11:22:35.416 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:35.416 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:35.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:35.473 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1614 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:35.530 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:35.530 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8087e-05 s, 0.0 kB/s 2023-08-27T11:22:35.530 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:35.531 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:35.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:35.588 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1615 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:35.645 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:35.645 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:35.645 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4327e-05 s, 0.0 kB/s 2023-08-27T11:22:35.646 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:35.646 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:35.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:35.703 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1616 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:35.760 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:35.760 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:35.760 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3275e-05 s, 0.0 kB/s 2023-08-27T11:22:35.761 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:35.761 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:35.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:35.818 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1617 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:35.875 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:35.876 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:35.876 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0676e-05 s, 0.0 kB/s 2023-08-27T11:22:35.876 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:35.877 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:35.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:35.934 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1618 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:35.990 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:35.991 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:35.991 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7565e-05 s, 0.0 kB/s 2023-08-27T11:22:35.991 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:35.992 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:36.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:36.048 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1619 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:36.104 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:36.105 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:36.105 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.938e-05 s, 0.0 kB/s 2023-08-27T11:22:36.105 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:36.106 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:36.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:36.163 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1620 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:36.220 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:36.221 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:36.221 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1781e-05 s, 0.0 kB/s 2023-08-27T11:22:36.221 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:36.222 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:36.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:36.279 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1621 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:36.335 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:22:36.336 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:36.336 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:36.337 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0158e-05 s, 0.0 kB/s 2023-08-27T11:22:36.337 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:36.338 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:36.378 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:22:36.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:36.394 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1622 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:36.484 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:36.484 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:36.484 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0424e-05 s, 0.0 kB/s 2023-08-27T11:22:36.485 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:36.485 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:36.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:36.542 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1623 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:36.598 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:36.598 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:36.599 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1744e-05 s, 0.0 kB/s 2023-08-27T11:22:36.599 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:36.599 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:36.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:36.656 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1624 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:36.714 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:36.715 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:36.715 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2588e-05 s, 0.0 kB/s 2023-08-27T11:22:36.715 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:36.716 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:36.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:36.773 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1625 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:36.829 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:36.829 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:36.830 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8815e-05 s, 0.0 kB/s 2023-08-27T11:22:36.830 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:36.830 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:36.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:36.887 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1626 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:36.944 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:36.944 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:36.945 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0838e-05 s, 0.0 kB/s 2023-08-27T11:22:36.945 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:36.945 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:37.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:37.001 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1627 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:37.058 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:37.059 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:37.059 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8381e-05 s, 0.0 kB/s 2023-08-27T11:22:37.059 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:37.060 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:37.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:37.116 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1628 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:37.174 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:37.175 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:37.175 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6734e-05 s, 0.0 kB/s 2023-08-27T11:22:37.175 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:37.176 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:37.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:37.232 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1629 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:37.288 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:37.288 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:37.289 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1607e-05 s, 0.0 kB/s 2023-08-27T11:22:37.289 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:37.289 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:37.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:37.346 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1630 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8242e-05 s, 0.0 kB/s 2023-08-27T11:22:37.403 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:37.404 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:37.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:37.461 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1631 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:37.518 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:37.518 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:37.518 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0179e-05 s, 0.0 kB/s 2023-08-27T11:22:37.518 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:37.519 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:37.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:37.576 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1632 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:37.632 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:37.633 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:37.633 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8249e-05 s, 0.0 kB/s 2023-08-27T11:22:37.633 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:37.634 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:37.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:37.690 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1633 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:37.747 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:37.748 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:37.748 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0207e-05 s, 0.0 kB/s 2023-08-27T11:22:37.748 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:37.749 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:37.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:37.806 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1634 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:37.863 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:37.863 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:37.863 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.032e-05 s, 0.0 kB/s 2023-08-27T11:22:37.863 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:37.864 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:37.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:37.921 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1635 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:37.977 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:37.978 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:37.978 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0719e-05 s, 0.0 kB/s 2023-08-27T11:22:37.978 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:37.979 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:38.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:38.035 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1636 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:38.092 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:38.093 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:38.093 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6572e-05 s, 0.0 kB/s 2023-08-27T11:22:38.093 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:38.094 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:38.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:38.150 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1637 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:38.207 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:38.207 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:38.207 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0261e-05 s, 0.0 kB/s 2023-08-27T11:22:38.208 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:38.208 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:38.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:38.266 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1638 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9408e-05 s, 0.0 kB/s 2023-08-27T11:22:38.324 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:38.325 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:38.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:38.382 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1639 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:38.438 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:38.438 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:38.439 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2526e-05 s, 0.0 kB/s 2023-08-27T11:22:38.439 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:38.439 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:38.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:38.496 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1640 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:38.552 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:38.553 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:38.553 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1366e-05 s, 0.0 kB/s 2023-08-27T11:22:38.553 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:38.554 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:38.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:38.610 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1641 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:38.667 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:38.668 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:38.668 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0045e-05 s, 0.0 kB/s 2023-08-27T11:22:38.669 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:38.669 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:38.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:38.725 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1642 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:38.781 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:38.782 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:38.782 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.867e-05 s, 0.0 kB/s 2023-08-27T11:22:38.782 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:38.783 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:38.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:38.840 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1643 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:38.898 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:38.898 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:38.898 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7866e-05 s, 0.0 kB/s 2023-08-27T11:22:38.899 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:38.899 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:38.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:38.956 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1644 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0301e-05 s, 0.0 kB/s 2023-08-27T11:22:39.014 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:39.016 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:39.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:39.071 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1645 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:39.127 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:39.127 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7468e-05 s, 0.0 kB/s 2023-08-27T11:22:39.128 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:39.128 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:39.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:39.185 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1646 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:39.243 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:39.243 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:39.243 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0973e-05 s, 0.0 kB/s 2023-08-27T11:22:39.244 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:39.244 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:39.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:39.301 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1647 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:39.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:39.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:39.358 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7378e-05 s, 0.0 kB/s 2023-08-27T11:22:39.358 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:39.359 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:39.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:39.416 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1648 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8573e-05 s, 0.0 kB/s 2023-08-27T11:22:39.474 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:39.474 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:39.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:39.531 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1649 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:39.588 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:39.588 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:39.589 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0051e-05 s, 0.0 kB/s 2023-08-27T11:22:39.589 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:39.590 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:39.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:39.647 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1650 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:39.704 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:39.704 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:39.704 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.861e-05 s, 0.0 kB/s 2023-08-27T11:22:39.705 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:39.705 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:39.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:39.762 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1651 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.235e-05 s, 0.0 kB/s 2023-08-27T11:22:39.825 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:39.825 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:39.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:39.886 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1652 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:39.959 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:39.960 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:39.960 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1418e-05 s, 0.0 kB/s 2023-08-27T11:22:39.960 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:39.961 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:40.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:40.021 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1653 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:40.078 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:40.078 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:40.079 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9439e-05 s, 0.0 kB/s 2023-08-27T11:22:40.079 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:40.079 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:40.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:40.136 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1654 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:40.194 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:40.194 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:40.194 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8029e-05 s, 0.0 kB/s 2023-08-27T11:22:40.195 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:40.195 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:40.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:40.252 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1655 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:40.311 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:40.311 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:40.311 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0777e-05 s, 0.0 kB/s 2023-08-27T11:22:40.312 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:40.312 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:40.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:40.369 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1656 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0885e-05 s, 0.0 kB/s 2023-08-27T11:22:40.428 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:40.428 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:40.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:40.486 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1657 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:40.544 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:40.544 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:40.545 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.895e-05 s, 0.0 kB/s 2023-08-27T11:22:40.545 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:40.545 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:40.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:40.603 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1658 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:40.663 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:40.663 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:40.663 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1044e-05 s, 0.0 kB/s 2023-08-27T11:22:40.664 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:40.664 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:40.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:40.725 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1659 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:40.787 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:40.788 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:40.788 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1554e-05 s, 0.0 kB/s 2023-08-27T11:22:40.789 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:40.789 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:40.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:40.847 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1660 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:40.909 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:40.909 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:40.909 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0051e-05 s, 0.0 kB/s 2023-08-27T11:22:40.910 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:40.910 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:40.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:40.971 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1661 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:41.032 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:41.032 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:41.032 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0358e-05 s, 0.0 kB/s 2023-08-27T11:22:41.033 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:41.034 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:41.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:41.092 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1662 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:41.151 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:41.151 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:41.152 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.944e-05 s, 0.0 kB/s 2023-08-27T11:22:41.152 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:41.152 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:41.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:41.211 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1663 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:41.270 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:41.272 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9751e-05 s, 0.0 kB/s 2023-08-27T11:22:41.273 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:41.273 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:41.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:41.328 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1664 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0473e-05 s, 0.0 kB/s 2023-08-27T11:22:41.385 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:41.386 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:41.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:41.444 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1665 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:41.500 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:41.501 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:41.501 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1618e-05 s, 0.0 kB/s 2023-08-27T11:22:41.501 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:41.502 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:41.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:41.558 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1666 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:41.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:41.615 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:41.615 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7549e-05 s, 0.0 kB/s 2023-08-27T11:22:41.615 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:41.616 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:41.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:41.672 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1667 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7001e-05 s, 0.0 kB/s 2023-08-27T11:22:41.729 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:41.730 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:41.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:41.786 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1668 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:41.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:41.843 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:41.843 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1088e-05 s, 0.0 kB/s 2023-08-27T11:22:41.843 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:41.844 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:41.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:41.900 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1669 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:41.956 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:41.956 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:41.957 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0889e-05 s, 0.0 kB/s 2023-08-27T11:22:41.957 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:41.957 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:42.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:42.014 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1670 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:42.070 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:42.071 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:42.071 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1127e-05 s, 0.0 kB/s 2023-08-27T11:22:42.071 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:42.072 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:42.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:42.128 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1671 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:42.186 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:42.186 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:42.187 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9358e-05 s, 0.0 kB/s 2023-08-27T11:22:42.187 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:42.187 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:42.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:42.243 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1672 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:42.301 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:42.301 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:42.301 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.713e-05 s, 0.0 kB/s 2023-08-27T11:22:42.302 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:42.302 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:42.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:42.359 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1673 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:42.416 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:42.417 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:42.417 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0036e-05 s, 0.0 kB/s 2023-08-27T11:22:42.417 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:42.418 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:42.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:42.473 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1674 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:42.530 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0838e-05 s, 0.0 kB/s 2023-08-27T11:22:42.531 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:42.532 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:42.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:42.587 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1675 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:42.645 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:42.646 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:42.646 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0683e-05 s, 0.0 kB/s 2023-08-27T11:22:42.646 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:42.647 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:42.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:42.703 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1676 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:42.760 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:42.761 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:42.761 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1499e-05 s, 0.0 kB/s 2023-08-27T11:22:42.762 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:42.762 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:42.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:42.818 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1677 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:42.875 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:42.875 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8385e-05 s, 0.0 kB/s 2023-08-27T11:22:42.875 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:42.876 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:42.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:42.933 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1678 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:42.990 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:42.990 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1901e-05 s, 0.0 kB/s 2023-08-27T11:22:42.990 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:42.991 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:43.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:43.047 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1679 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:43.105 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.066e-05 s, 0.0 kB/s 2023-08-27T11:22:43.105 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:43.105 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:43.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:43.162 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1680 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1725e-05 s, 0.0 kB/s 2023-08-27T11:22:43.221 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:43.221 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:43.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:43.278 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1681 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:43.335 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:43.336 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:43.336 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0764e-05 s, 0.0 kB/s 2023-08-27T11:22:43.336 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:43.337 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:43.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:43.393 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1682 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2992e-05 s, 0.0 kB/s 2023-08-27T11:22:43.455 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:43.455 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:43.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:43.511 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1683 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:43.569 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:43.569 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:43.569 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3121e-05 s, 0.0 kB/s 2023-08-27T11:22:43.570 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:43.571 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:43.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:43.627 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1684 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:43.685 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:43.685 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:43.685 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.8494e-05 s, 0.0 kB/s 2023-08-27T11:22:43.686 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:43.686 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:43.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:43.743 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1685 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:43.801 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:43.801 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:43.802 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9895e-05 s, 0.0 kB/s 2023-08-27T11:22:43.802 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:43.803 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:43.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:43.858 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1686 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:43.916 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1132e-05 s, 0.0 kB/s 2023-08-27T11:22:43.917 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:43.918 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:43.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:43.973 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1687 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:44.030 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:44.031 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:44.031 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8775e-05 s, 0.0 kB/s 2023-08-27T11:22:44.031 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:44.032 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:44.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:44.091 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1688 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:44.148 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:44.149 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:44.149 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3813e-05 s, 0.0 kB/s 2023-08-27T11:22:44.149 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:44.150 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:44.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:44.206 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1689 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:44.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:44.266 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:44.266 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.802e-05 s, 0.0 kB/s 2023-08-27T11:22:44.266 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:44.267 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:44.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:44.325 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1690 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:44.382 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:44.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:44.383 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.388e-05 s, 0.0 kB/s 2023-08-27T11:22:44.384 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:44.384 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:44.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:44.440 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1691 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:44.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:44.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:44.499 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1827e-05 s, 0.0 kB/s 2023-08-27T11:22:44.500 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:44.500 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:44.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:44.557 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1692 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:44.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:44.615 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:44.615 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2533e-05 s, 0.0 kB/s 2023-08-27T11:22:44.615 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:44.616 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:44.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:44.674 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1693 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:44.731 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:44.731 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:44.732 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2873e-05 s, 0.0 kB/s 2023-08-27T11:22:44.732 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:44.733 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:44.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:44.789 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1694 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:44.847 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:44.848 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:44.848 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1079e-05 s, 0.0 kB/s 2023-08-27T11:22:44.849 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:44.850 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:44.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:44.906 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1695 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:44.964 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:44.964 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:44.964 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7675e-05 s, 0.0 kB/s 2023-08-27T11:22:44.965 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:44.965 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:45.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:45.022 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1696 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:45.078 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:45.079 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:45.079 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9873e-05 s, 0.0 kB/s 2023-08-27T11:22:45.080 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:45.080 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:45.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:45.137 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1697 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:45.194 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9786e-05 s, 0.0 kB/s 2023-08-27T11:22:45.195 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:45.196 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:45.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:45.254 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1698 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:45.311 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:45.312 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:45.312 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0457e-05 s, 0.0 kB/s 2023-08-27T11:22:45.312 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:45.313 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:45.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:45.370 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1699 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:45.428 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:45.428 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:45.429 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2285e-05 s, 0.0 kB/s 2023-08-27T11:22:45.429 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:45.429 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:45.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:45.486 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1700 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:45.549 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2751e-05 s, 0.0 kB/s 2023-08-27T11:22:45.550 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:45.551 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:45.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:45.607 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1701 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0682e-05 s, 0.0 kB/s 2023-08-27T11:22:45.666 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:45.666 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:45.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:45.723 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1702 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:45.779 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:45.779 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:45.779 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.4095e-05 s, 0.0 kB/s 2023-08-27T11:22:45.780 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:45.780 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:45.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:45.836 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1703 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:45.893 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:45.894 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:45.894 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9947e-05 s, 0.0 kB/s 2023-08-27T11:22:45.894 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:45.895 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:45.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:45.954 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1704 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:46.011 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:46.011 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:46.011 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8252e-05 s, 0.0 kB/s 2023-08-27T11:22:46.012 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:46.012 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:46.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:46.069 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1705 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:46.127 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:46.127 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:46.127 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8653e-05 s, 0.0 kB/s 2023-08-27T11:22:46.128 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:46.128 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:46.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:46.185 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1706 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:46.241 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:46.242 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:46.242 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8759e-05 s, 0.0 kB/s 2023-08-27T11:22:46.242 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:46.243 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:46.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:46.299 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1707 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:46.356 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:46.356 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:46.356 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.876e-05 s, 0.0 kB/s 2023-08-27T11:22:46.357 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:46.357 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:46.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:46.415 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1708 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:46.471 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:46.472 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:46.472 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0686e-05 s, 0.0 kB/s 2023-08-27T11:22:46.472 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:46.473 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:46.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:46.529 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1709 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:46.586 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:46.586 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:46.587 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.899e-05 s, 0.0 kB/s 2023-08-27T11:22:46.587 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:46.587 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:46.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:46.645 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1710 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:46.702 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:46.703 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:46.703 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1015e-05 s, 0.0 kB/s 2023-08-27T11:22:46.703 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:46.704 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:46.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:46.763 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1711 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:46.820 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:46.820 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1007e-05 s, 0.0 kB/s 2023-08-27T11:22:46.820 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:46.821 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:46.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:46.877 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1712 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:46.934 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:46.935 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:46.935 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8671e-05 s, 0.0 kB/s 2023-08-27T11:22:46.935 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:46.936 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:46.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:46.993 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1713 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:47.051 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:47.052 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:47.052 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0967e-05 s, 0.0 kB/s 2023-08-27T11:22:47.052 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:47.053 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:47.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:47.109 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1714 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:47.166 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:47.166 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:47.166 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0335e-05 s, 0.0 kB/s 2023-08-27T11:22:47.167 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:47.167 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:47.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:47.224 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1715 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:47.281 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:47.282 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:47.282 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8765e-05 s, 0.0 kB/s 2023-08-27T11:22:47.282 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:47.283 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:47.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:47.340 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1716 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:47.396 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:47.397 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:47.397 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2807e-05 s, 0.0 kB/s 2023-08-27T11:22:47.397 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:47.397 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:47.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:47.455 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1717 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:47.513 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:47.513 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:47.513 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0118e-05 s, 0.0 kB/s 2023-08-27T11:22:47.514 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:47.514 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:47.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:47.571 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1718 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:47.628 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:47.628 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:47.628 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8544e-05 s, 0.0 kB/s 2023-08-27T11:22:47.629 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:47.629 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:47.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:47.686 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1719 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:47.744 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:47.744 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:47.744 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2777e-05 s, 0.0 kB/s 2023-08-27T11:22:47.745 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:47.745 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:47.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:47.802 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1720 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:47.859 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:47.860 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:47.860 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9604e-05 s, 0.0 kB/s 2023-08-27T11:22:47.860 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:47.861 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:47.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:47.917 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1721 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:47.975 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:47.975 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:47.975 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8609e-05 s, 0.0 kB/s 2023-08-27T11:22:47.976 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:47.976 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:48.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:48.035 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1722 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:48.090 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:48.090 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:48.090 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8802e-05 s, 0.0 kB/s 2023-08-27T11:22:48.090 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:48.091 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:48.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:48.147 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1723 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:48.204 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:48.204 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:48.204 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.915e-05 s, 0.0 kB/s 2023-08-27T11:22:48.205 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:48.206 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:48.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:48.263 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1724 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:48.321 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:48.321 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:48.321 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0025e-05 s, 0.0 kB/s 2023-08-27T11:22:48.322 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:48.322 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:48.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:48.379 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1725 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:48.436 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:48.436 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:48.436 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1797e-05 s, 0.0 kB/s 2023-08-27T11:22:48.437 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:48.437 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:48.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:48.494 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1726 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:48.551 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:48.551 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:48.551 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1223e-05 s, 0.0 kB/s 2023-08-27T11:22:48.552 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:48.552 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:48.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:48.609 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1727 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:48.668 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:48.668 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:48.668 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9646e-05 s, 0.0 kB/s 2023-08-27T11:22:48.668 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:48.669 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:48.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:48.726 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1728 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:48.783 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:48.783 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:48.784 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9102e-05 s, 0.0 kB/s 2023-08-27T11:22:48.784 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:48.784 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:48.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:48.841 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1729 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:48.898 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:48.898 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:48.898 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0249e-05 s, 0.0 kB/s 2023-08-27T11:22:48.898 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:48.899 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:48.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:48.956 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1730 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:49.013 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:49.013 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:49.013 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7492e-05 s, 0.0 kB/s 2023-08-27T11:22:49.014 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:49.014 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:49.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:49.073 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1731 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:49.130 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:49.130 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:49.130 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1517e-05 s, 0.0 kB/s 2023-08-27T11:22:49.131 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:49.131 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:49.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:49.188 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1732 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:49.246 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:49.246 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:49.247 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8796e-05 s, 0.0 kB/s 2023-08-27T11:22:49.247 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:49.247 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:49.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:49.304 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1733 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:49.362 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:49.362 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:49.362 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2713e-05 s, 0.0 kB/s 2023-08-27T11:22:49.363 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:49.363 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:49.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:49.420 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1734 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:49.477 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:49.477 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:49.477 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9666e-05 s, 0.0 kB/s 2023-08-27T11:22:49.478 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:49.478 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:49.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:49.535 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1735 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9589e-05 s, 0.0 kB/s 2023-08-27T11:22:49.594 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:49.594 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:49.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:49.651 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1736 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:49.707 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:49.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:49.708 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9669e-05 s, 0.0 kB/s 2023-08-27T11:22:49.708 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:49.709 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:49.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:49.766 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1737 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:49.823 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:49.823 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:49.823 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0588e-05 s, 0.0 kB/s 2023-08-27T11:22:49.824 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:49.824 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:49.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:49.880 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1738 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:49.938 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:49.938 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:49.938 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6911e-05 s, 0.0 kB/s 2023-08-27T11:22:49.939 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:49.939 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:49.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:49.995 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1739 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:50.053 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:50.053 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:50.053 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0043e-05 s, 0.0 kB/s 2023-08-27T11:22:50.053 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:50.054 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:50.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:50.111 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1740 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:50.168 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:50.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:50.169 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7388e-05 s, 0.0 kB/s 2023-08-27T11:22:50.169 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:50.170 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:50.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:50.226 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1741 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:50.283 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:50.284 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:50.284 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9951e-05 s, 0.0 kB/s 2023-08-27T11:22:50.284 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:50.285 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:50.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:50.342 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1742 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:50.400 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:50.400 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:50.401 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3396e-05 s, 0.0 kB/s 2023-08-27T11:22:50.401 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:50.401 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:50.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:50.458 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1743 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:50.515 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:50.515 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:50.516 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9712e-05 s, 0.0 kB/s 2023-08-27T11:22:50.516 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:50.517 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:50.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:50.573 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1744 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:50.631 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:50.631 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1676e-05 s, 0.0 kB/s 2023-08-27T11:22:50.631 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:50.632 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:50.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:50.689 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1745 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:50.746 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:50.746 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:50.747 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8751e-05 s, 0.0 kB/s 2023-08-27T11:22:50.747 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:50.747 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:50.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:50.805 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1746 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:50.863 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:50.863 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:50.864 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0628e-05 s, 0.0 kB/s 2023-08-27T11:22:50.864 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:50.865 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:50.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:50.920 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1747 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:50.977 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0141e-05 s, 0.0 kB/s 2023-08-27T11:22:50.978 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:50.979 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:51.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:51.035 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1748 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:51.091 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:51.092 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:51.092 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.329e-05 s, 0.0 kB/s 2023-08-27T11:22:51.092 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:51.093 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:51.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:51.150 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1749 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:51.207 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:51.207 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:51.207 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0314e-05 s, 0.0 kB/s 2023-08-27T11:22:51.208 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:51.208 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:51.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:51.267 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1750 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:51.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:51.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:51.324 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0622e-05 s, 0.0 kB/s 2023-08-27T11:22:51.325 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:51.325 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:51.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:51.381 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1751 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.5235e-05 s, 0.0 kB/s 2023-08-27T11:22:51.440 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:51.440 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:51.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:51.496 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1752 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:51.554 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:51.554 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0962e-05 s, 0.0 kB/s 2023-08-27T11:22:51.554 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:51.555 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:51.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:51.613 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1753 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:51.671 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:51.671 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:51.671 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8442e-05 s, 0.0 kB/s 2023-08-27T11:22:51.672 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:51.672 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:51.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:51.732 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1754 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:51.790 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:51.790 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:51.791 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8752e-05 s, 0.0 kB/s 2023-08-27T11:22:51.791 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:51.792 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:51.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:51.848 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1755 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7689e-05 s, 0.0 kB/s 2023-08-27T11:22:51.907 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:51.907 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:51.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:51.962 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1756 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:52.019 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:52.019 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:52.020 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6526e-05 s, 0.0 kB/s 2023-08-27T11:22:52.020 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:52.020 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:52.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:52.077 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1757 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:52.133 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:52.134 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:52.134 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8812e-05 s, 0.0 kB/s 2023-08-27T11:22:52.134 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:52.135 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:52.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:52.192 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1758 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:52.250 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:52.250 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:52.250 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0065e-05 s, 0.0 kB/s 2023-08-27T11:22:52.251 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:52.251 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:52.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:52.308 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1759 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:52.366 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:52.366 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:52.366 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8314e-05 s, 0.0 kB/s 2023-08-27T11:22:52.367 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:52.367 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:52.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:52.424 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1760 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:52.480 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:52.480 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:52.481 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9509e-05 s, 0.0 kB/s 2023-08-27T11:22:52.481 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:52.481 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:52.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:52.538 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1761 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:52.594 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:52.595 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:52.595 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8223e-05 s, 0.0 kB/s 2023-08-27T11:22:52.595 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:52.596 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:52.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:52.653 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1762 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:52.710 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:52.711 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:52.711 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7042e-05 s, 0.0 kB/s 2023-08-27T11:22:52.711 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:52.712 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:52.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:52.769 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1763 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:52.827 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:52.827 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:52.827 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6839e-05 s, 0.0 kB/s 2023-08-27T11:22:52.828 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:52.828 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:52.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:52.885 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1764 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:52.941 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:52.942 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:52.942 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8282e-05 s, 0.0 kB/s 2023-08-27T11:22:52.942 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:52.943 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:52.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:52.999 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1765 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:53.055 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:53.056 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:53.056 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.886e-05 s, 0.0 kB/s 2023-08-27T11:22:53.056 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:53.057 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:53.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:53.113 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1766 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9257e-05 s, 0.0 kB/s 2023-08-27T11:22:53.172 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:53.172 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:53.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:53.229 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1767 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:53.286 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:53.286 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:53.286 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1152e-05 s, 0.0 kB/s 2023-08-27T11:22:53.287 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:53.287 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:53.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:53.344 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1768 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4042e-05 s, 0.0 kB/s 2023-08-27T11:22:53.403 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:53.403 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:53.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:53.460 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1769 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:53.517 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:53.517 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:53.517 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2578e-05 s, 0.0 kB/s 2023-08-27T11:22:53.518 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:53.518 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:53.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:53.575 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1770 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:53.632 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:53.633 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:53.633 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.769e-05 s, 0.0 kB/s 2023-08-27T11:22:53.634 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:53.634 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:53.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:53.690 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1771 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:53.748 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:53.748 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:53.748 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7721e-05 s, 0.0 kB/s 2023-08-27T11:22:53.749 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:53.749 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:53.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:53.806 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1772 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.958e-05 s, 0.0 kB/s 2023-08-27T11:22:53.865 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:53.865 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:53.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:53.923 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1773 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:53.979 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:53.979 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:53.979 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6433e-05 s, 0.0 kB/s 2023-08-27T11:22:53.980 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:53.980 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:54.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:54.037 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1774 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:54.095 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:54.095 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:54.095 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8815e-05 s, 0.0 kB/s 2023-08-27T11:22:54.096 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:54.096 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:54.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:54.153 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1775 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:54.210 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:54.210 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:54.210 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6882e-05 s, 0.0 kB/s 2023-08-27T11:22:54.211 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:54.211 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:54.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:54.267 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1776 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:54.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:54.325 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:54.325 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.861e-05 s, 0.0 kB/s 2023-08-27T11:22:54.325 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:54.326 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:54.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:54.383 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1777 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:54.441 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:54.442 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:54.442 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9753e-05 s, 0.0 kB/s 2023-08-27T11:22:54.443 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:54.443 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:54.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:54.500 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1778 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:54.556 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:54.557 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:54.557 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9789e-05 s, 0.0 kB/s 2023-08-27T11:22:54.557 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:54.558 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:54.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:54.615 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1779 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:54.672 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:54.673 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:54.673 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0825e-05 s, 0.0 kB/s 2023-08-27T11:22:54.673 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:54.674 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:54.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:54.731 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1780 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:54.788 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:54.789 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:54.789 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8372e-05 s, 0.0 kB/s 2023-08-27T11:22:54.789 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:54.790 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:54.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:54.847 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1781 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0381e-05 s, 0.0 kB/s 2023-08-27T11:22:54.904 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:54.905 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:54.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:54.962 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1782 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9906e-05 s, 0.0 kB/s 2023-08-27T11:22:55.020 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:55.020 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:55.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:55.079 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1783 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:55.136 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:55.136 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:55.137 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0919e-05 s, 0.0 kB/s 2023-08-27T11:22:55.137 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:55.137 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:55.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:55.194 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1784 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8218e-05 s, 0.0 kB/s 2023-08-27T11:22:55.251 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:55.252 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:55.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:55.309 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1785 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:55.365 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:55.365 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:55.366 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.058e-05 s, 0.0 kB/s 2023-08-27T11:22:55.366 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:55.368 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:55.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:55.423 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1786 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:55.479 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:55.480 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:55.480 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0901e-05 s, 0.0 kB/s 2023-08-27T11:22:55.480 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:55.481 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:55.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:55.539 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1787 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:55.594 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.891e-05 s, 0.0 kB/s 2023-08-27T11:22:55.595 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:55.596 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:55.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:55.653 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1788 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:55.710 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:55.711 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:55.711 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9347e-05 s, 0.0 kB/s 2023-08-27T11:22:55.711 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:55.712 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:55.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:55.768 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1789 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:55.825 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:55.825 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:55.826 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9399e-05 s, 0.0 kB/s 2023-08-27T11:22:55.826 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:55.826 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:55.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:55.883 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1790 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:55.940 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:55.940 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:55.941 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0018e-05 s, 0.0 kB/s 2023-08-27T11:22:55.941 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:55.941 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:55.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:55.998 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1791 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:56.055 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:56.055 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:56.055 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0496e-05 s, 0.0 kB/s 2023-08-27T11:22:56.056 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:56.056 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:56.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:56.113 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1792 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:56.170 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:56.171 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:56.171 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1402e-05 s, 0.0 kB/s 2023-08-27T11:22:56.171 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:56.172 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:56.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:56.227 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1793 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:56.285 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:56.285 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:56.286 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9007e-05 s, 0.0 kB/s 2023-08-27T11:22:56.286 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:56.286 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:56.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:56.344 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1794 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:56.400 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:56.400 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:56.401 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6499e-05 s, 0.0 kB/s 2023-08-27T11:22:56.401 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:56.401 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:56.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:56.459 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1795 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:56.515 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:56.516 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:56.516 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7299e-05 s, 0.0 kB/s 2023-08-27T11:22:56.516 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:56.517 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:56.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:56.574 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1796 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:56.632 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:56.632 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:56.632 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9498e-05 s, 0.0 kB/s 2023-08-27T11:22:56.633 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:56.633 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:56.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:56.692 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1797 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:56.747 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9364e-05 s, 0.0 kB/s 2023-08-27T11:22:56.748 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:56.749 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:56.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:56.806 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1798 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1294e-05 s, 0.0 kB/s 2023-08-27T11:22:56.864 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:56.865 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:56.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:56.924 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1799 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:56.981 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:56.982 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:56.982 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9066e-05 s, 0.0 kB/s 2023-08-27T11:22:56.982 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:56.983 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:57.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:57.039 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1800 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:57.096 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.6099e-05 s, 0.0 kB/s 2023-08-27T11:22:57.096 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:57.096 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:57.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:57.153 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1801 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2614e-05 s, 0.0 kB/s 2023-08-27T11:22:57.211 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:57.211 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:57.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:57.271 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1802 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:57.327 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:57.328 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:57.328 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8717e-05 s, 0.0 kB/s 2023-08-27T11:22:57.328 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:57.329 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:57.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:57.386 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1803 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:57.443 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:57.443 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:57.443 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1516e-05 s, 0.0 kB/s 2023-08-27T11:22:57.444 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:57.444 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:57.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:57.501 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1804 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:57.557 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:57.558 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:57.558 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5244e-05 s, 0.0 kB/s 2023-08-27T11:22:57.558 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:57.559 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:57.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:57.615 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1805 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9821e-05 s, 0.0 kB/s 2023-08-27T11:22:57.672 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:57.673 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:57.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:57.729 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1806 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:57.787 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:57.787 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:57.788 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8326e-05 s, 0.0 kB/s 2023-08-27T11:22:57.788 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:57.788 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:57.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:57.845 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1807 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:57.902 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:57.902 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:57.902 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8217e-05 s, 0.0 kB/s 2023-08-27T11:22:57.902 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:57.903 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:57.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:57.960 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1808 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:58.017 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:58.017 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:58.017 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8738e-05 s, 0.0 kB/s 2023-08-27T11:22:58.018 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:58.018 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:58.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:58.075 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1809 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:58.134 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:58.135 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:58.135 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0144e-05 s, 0.0 kB/s 2023-08-27T11:22:58.135 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:58.136 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:58.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:58.192 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1810 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9078e-05 s, 0.0 kB/s 2023-08-27T11:22:58.249 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:58.249 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:58.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:58.306 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1811 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:58.363 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9932e-05 s, 0.0 kB/s 2023-08-27T11:22:58.364 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:58.365 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:58.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:58.421 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1812 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:58.479 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:58.479 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:58.480 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1424e-05 s, 0.0 kB/s 2023-08-27T11:22:58.480 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:58.481 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:58.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:58.536 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1813 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1801e-05 s, 0.0 kB/s 2023-08-27T11:22:58.595 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:58.595 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:58.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:58.652 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1814 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:58.730 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:58.730 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:58.730 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.7949e-05 s, 0.0 kB/s 2023-08-27T11:22:58.731 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:58.731 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:58.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:58.789 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1815 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:58.846 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:58.846 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:58.846 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8353e-05 s, 0.0 kB/s 2023-08-27T11:22:58.847 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:58.847 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:58.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:58.904 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1816 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:58.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:58.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:58.962 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8304e-05 s, 0.0 kB/s 2023-08-27T11:22:58.963 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:58.963 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:59.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:59.019 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1817 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:59.078 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:59.078 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:59.078 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0931e-05 s, 0.0 kB/s 2023-08-27T11:22:59.078 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:59.079 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:59.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:59.136 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1818 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:59.193 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:59.194 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:59.194 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1739e-05 s, 0.0 kB/s 2023-08-27T11:22:59.194 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:59.195 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:59.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:59.253 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1819 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:59.309 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:59.310 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:59.310 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9924e-05 s, 0.0 kB/s 2023-08-27T11:22:59.310 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:59.310 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:59.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:59.368 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1820 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:59.425 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0752e-05 s, 0.0 kB/s 2023-08-27T11:22:59.426 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:59.427 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:59.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:59.484 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1821 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:59.542 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:59.543 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:59.543 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5689e-05 s, 0.0 kB/s 2023-08-27T11:22:59.543 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:59.544 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:59.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:59.600 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1822 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:59.657 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:59.658 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:59.658 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7311e-05 s, 0.0 kB/s 2023-08-27T11:22:59.658 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:59.659 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:59.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:59.716 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1823 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:59.773 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:59.773 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:59.773 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2258e-05 s, 0.0 kB/s 2023-08-27T11:22:59.774 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:59.774 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:59.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:59.831 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1824 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:22:59.889 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:22:59.889 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:22:59.889 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2172e-05 s, 0.0 kB/s 2023-08-27T11:22:59.889 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:22:59.890 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:22:59.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:22:59.947 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1825 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1112e-05 s, 0.0 kB/s 2023-08-27T11:23:00.004 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:00.005 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:00.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:00.062 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1826 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:00.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:00.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:00.121 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8537e-05 s, 0.0 kB/s 2023-08-27T11:23:00.121 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:00.121 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:00.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:00.178 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1827 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:00.236 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:00.236 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:00.237 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.988e-05 s, 0.0 kB/s 2023-08-27T11:23:00.237 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:00.238 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:00.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:00.294 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1828 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:00.351 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:00.351 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1251e-05 s, 0.0 kB/s 2023-08-27T11:23:00.352 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:00.353 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:00.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:00.410 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1829 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:00.467 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:00.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:00.468 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8151e-05 s, 0.0 kB/s 2023-08-27T11:23:00.468 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:00.469 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:00.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:00.525 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1830 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:00.582 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:00.582 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:00.582 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6831e-05 s, 0.0 kB/s 2023-08-27T11:23:00.583 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:00.583 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:00.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:00.640 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1831 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:00.697 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9839e-05 s, 0.0 kB/s 2023-08-27T11:23:00.697 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:00.697 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:00.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:00.754 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1832 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:00.812 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:00.812 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:00.812 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0078e-05 s, 0.0 kB/s 2023-08-27T11:23:00.813 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:00.813 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:00.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:00.870 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1833 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:00.927 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:00.927 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:00.927 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9198e-05 s, 0.0 kB/s 2023-08-27T11:23:00.928 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:00.928 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:00.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:00.985 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1834 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:01.042 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:01.042 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:01.042 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.964e-05 s, 0.0 kB/s 2023-08-27T11:23:01.043 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:01.043 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:01.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:01.100 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1835 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:01.157 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8612e-05 s, 0.0 kB/s 2023-08-27T11:23:01.159 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:01.159 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:01.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:01.216 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1836 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:01.272 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:01.273 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:01.273 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8696e-05 s, 0.0 kB/s 2023-08-27T11:23:01.273 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:01.274 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:01.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:01.331 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1837 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:01.389 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:01.389 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:01.390 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0029e-05 s, 0.0 kB/s 2023-08-27T11:23:01.390 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:01.391 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:01.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:01.447 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1838 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:01.503 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:01.504 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:01.504 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9018e-05 s, 0.0 kB/s 2023-08-27T11:23:01.504 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:01.504 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:01.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:01.561 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1839 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:01.617 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:01.618 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:01.618 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7787e-05 s, 0.0 kB/s 2023-08-27T11:23:01.618 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:01.619 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:01.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:01.675 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1840 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:01.732 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:01.732 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:01.732 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8582e-05 s, 0.0 kB/s 2023-08-27T11:23:01.733 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:01.733 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:01.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:01.789 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1841 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:01.846 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:01.847 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:01.847 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0471e-05 s, 0.0 kB/s 2023-08-27T11:23:01.847 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:01.848 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:01.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:01.905 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1842 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:01.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:01.962 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.9904e-05 s, 0.0 kB/s 2023-08-27T11:23:01.962 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:01.963 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:02.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:02.019 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1843 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:02.076 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:02.076 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:02.076 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8859e-05 s, 0.0 kB/s 2023-08-27T11:23:02.077 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:02.077 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:02.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:02.134 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1844 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:02.191 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:02.191 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:02.192 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0785e-05 s, 0.0 kB/s 2023-08-27T11:23:02.192 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:02.192 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:02.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:02.249 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1845 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:02.305 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:02.305 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:02.306 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.205e-05 s, 0.0 kB/s 2023-08-27T11:23:02.306 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:02.306 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:02.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:02.363 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1846 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:02.421 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:02.422 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:02.422 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4328e-05 s, 0.0 kB/s 2023-08-27T11:23:02.423 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:02.423 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:02.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:02.479 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1847 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:02.536 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:02.536 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:02.537 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1284e-05 s, 0.0 kB/s 2023-08-27T11:23:02.537 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:02.538 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:02.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:02.594 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1848 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:02.651 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:02.651 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:02.651 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2672e-05 s, 0.0 kB/s 2023-08-27T11:23:02.652 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:02.652 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:02.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:02.709 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1849 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:02.766 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:02.767 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:02.767 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9888e-05 s, 0.0 kB/s 2023-08-27T11:23:02.767 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:02.767 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:02.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:02.825 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1850 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:02.881 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:02.882 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:02.882 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2643e-05 s, 0.0 kB/s 2023-08-27T11:23:02.882 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:02.883 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:02.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:02.940 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1851 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:02.997 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:02.997 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:02.997 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9175e-05 s, 0.0 kB/s 2023-08-27T11:23:02.998 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:02.998 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:03.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:03.057 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1852 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:03.112 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:03.112 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:03.112 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9426e-05 s, 0.0 kB/s 2023-08-27T11:23:03.112 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:03.113 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:03.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:03.170 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1853 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:03.227 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:03.227 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:03.227 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.981e-05 s, 0.0 kB/s 2023-08-27T11:23:03.228 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:03.228 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:03.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:03.285 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1854 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:03.342 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:03.342 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:03.342 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9282e-05 s, 0.0 kB/s 2023-08-27T11:23:03.343 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:03.343 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:03.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:03.400 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1855 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:03.458 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:03.458 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:03.458 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1006e-05 s, 0.0 kB/s 2023-08-27T11:23:03.459 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:03.459 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:03.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:03.516 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1856 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:03.573 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:03.574 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:03.574 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0329e-05 s, 0.0 kB/s 2023-08-27T11:23:03.574 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:03.575 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:03.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:03.632 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1857 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:03.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:03.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:03.689 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8734e-05 s, 0.0 kB/s 2023-08-27T11:23:03.690 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:03.690 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:03.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:03.746 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1858 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7659e-05 s, 0.0 kB/s 2023-08-27T11:23:03.805 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:03.805 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:03.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:03.862 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1859 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:03.918 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:03.919 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:03.919 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9455e-05 s, 0.0 kB/s 2023-08-27T11:23:03.919 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:03.920 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:03.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:03.977 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1860 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:04.034 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:04.035 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:04.035 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7393e-05 s, 0.0 kB/s 2023-08-27T11:23:04.036 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:04.036 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:04.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:04.095 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1861 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1214e-05 s, 0.0 kB/s 2023-08-27T11:23:04.154 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:04.154 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:04.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:04.211 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1862 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8907e-05 s, 0.0 kB/s 2023-08-27T11:23:04.270 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:04.270 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:04.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:04.327 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1863 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:04.384 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:04.384 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:04.385 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3954e-05 s, 0.0 kB/s 2023-08-27T11:23:04.385 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:04.386 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:04.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:04.442 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1864 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9986e-05 s, 0.0 kB/s 2023-08-27T11:23:04.500 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:04.500 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:04.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:04.558 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1865 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8154e-05 s, 0.0 kB/s 2023-08-27T11:23:04.616 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:04.616 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:04.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:04.673 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1866 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6207e-05 s, 0.0 kB/s 2023-08-27T11:23:04.730 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:04.731 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:04.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:04.788 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1867 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:04.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:04.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:04.845 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.7726e-05 s, 0.0 kB/s 2023-08-27T11:23:04.846 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:04.847 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:04.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:04.904 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1868 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:04.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:04.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:04.962 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2778e-05 s, 0.0 kB/s 2023-08-27T11:23:04.963 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:04.963 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:05.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:05.020 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1869 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:05.076 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:05.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:05.077 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3686e-05 s, 0.0 kB/s 2023-08-27T11:23:05.077 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:05.078 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:05.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:05.134 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1870 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0403e-05 s, 0.0 kB/s 2023-08-27T11:23:05.191 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:05.192 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:05.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:05.248 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1871 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:05.305 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:05.306 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:05.306 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0531e-05 s, 0.0 kB/s 2023-08-27T11:23:05.306 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:05.307 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:05.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:05.363 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1872 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:05.421 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9259e-05 s, 0.0 kB/s 2023-08-27T11:23:05.422 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:05.423 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:05.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:05.479 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1873 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.676e-05 s, 0.0 kB/s 2023-08-27T11:23:05.537 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:05.537 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:05.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:05.594 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1874 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:05.652 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:05.652 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:05.652 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9169e-05 s, 0.0 kB/s 2023-08-27T11:23:05.653 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:05.653 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:05.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:05.711 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1875 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:05.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:05.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:05.768 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1233e-05 s, 0.0 kB/s 2023-08-27T11:23:05.769 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:05.769 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:05.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:05.826 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1876 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:05.885 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:05.885 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:05.886 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0726e-05 s, 0.0 kB/s 2023-08-27T11:23:05.886 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:05.886 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:05.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:05.943 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1877 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:06.000 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:06.001 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:06.001 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2529e-05 s, 0.0 kB/s 2023-08-27T11:23:06.001 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:06.002 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:06.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:06.057 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1878 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:06.114 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:06.115 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:06.115 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9926e-05 s, 0.0 kB/s 2023-08-27T11:23:06.115 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:06.116 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:06.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:06.172 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1879 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:06.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:06.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:06.230 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0524e-05 s, 0.0 kB/s 2023-08-27T11:23:06.230 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:06.231 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:06.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:06.288 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1880 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:06.345 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:06.346 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:06.346 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1202e-05 s, 0.0 kB/s 2023-08-27T11:23:06.346 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:06.346 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:06.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:06.404 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1881 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:06.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:23:06.447 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:23:06.462 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:06.462 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:06.462 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1063e-05 s, 0.0 kB/s 2023-08-27T11:23:06.463 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:06.463 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:06.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:06.560 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1882 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:06.617 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:06.617 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:06.618 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1078e-05 s, 0.0 kB/s 2023-08-27T11:23:06.618 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:06.618 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:06.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:06.675 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1883 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:06.732 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:06.733 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:06.733 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2818e-05 s, 0.0 kB/s 2023-08-27T11:23:06.733 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:06.734 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:06.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:06.790 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1884 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:06.847 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:06.847 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:06.848 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9195e-05 s, 0.0 kB/s 2023-08-27T11:23:06.848 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:06.848 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:06.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:06.905 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1885 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:06.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:06.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:06.962 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9761e-05 s, 0.0 kB/s 2023-08-27T11:23:06.963 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:06.963 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:07.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:07.019 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1886 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:07.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:07.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:07.077 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8161e-05 s, 0.0 kB/s 2023-08-27T11:23:07.078 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:07.078 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:07.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:07.135 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1887 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:07.192 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:07.193 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:07.193 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0782e-05 s, 0.0 kB/s 2023-08-27T11:23:07.193 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:07.194 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:07.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:07.250 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1888 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:07.307 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:07.308 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:07.308 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2994e-05 s, 0.0 kB/s 2023-08-27T11:23:07.308 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:07.309 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:07.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:07.365 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1889 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:07.423 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:07.423 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:07.423 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0796e-05 s, 0.0 kB/s 2023-08-27T11:23:07.424 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:07.424 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:07.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:07.481 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1890 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:07.539 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:07.539 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:07.540 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2381e-05 s, 0.0 kB/s 2023-08-27T11:23:07.540 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:07.540 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:07.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:07.597 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1891 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:07.654 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:07.654 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:07.654 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1725e-05 s, 0.0 kB/s 2023-08-27T11:23:07.655 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:07.655 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:07.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:07.712 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1892 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:07.770 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:07.770 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:07.771 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7887e-05 s, 0.0 kB/s 2023-08-27T11:23:07.771 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:07.772 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:07.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:07.828 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1893 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:07.885 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:07.885 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.966e-05 s, 0.0 kB/s 2023-08-27T11:23:07.886 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:07.886 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:07.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:07.944 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1894 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:07.999 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:08.000 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:08.000 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7865e-05 s, 0.0 kB/s 2023-08-27T11:23:08.000 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:08.001 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:08.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:08.056 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1895 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:08.114 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:08.114 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:08.114 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4273e-05 s, 0.0 kB/s 2023-08-27T11:23:08.115 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:08.116 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:08.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:08.172 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1896 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:08.228 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:08.229 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:08.229 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7443e-05 s, 0.0 kB/s 2023-08-27T11:23:08.229 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:08.230 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:08.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:08.287 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1897 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:08.343 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:08.343 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:08.344 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8914e-05 s, 0.0 kB/s 2023-08-27T11:23:08.345 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:08.345 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:08.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:08.404 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1898 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:08.462 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:08.462 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:08.463 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8487e-05 s, 0.0 kB/s 2023-08-27T11:23:08.463 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:08.463 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:08.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:08.520 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1899 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:08.577 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:08.577 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0756e-05 s, 0.0 kB/s 2023-08-27T11:23:08.577 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:08.578 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:08.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:08.635 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1900 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1489e-05 s, 0.0 kB/s 2023-08-27T11:23:08.694 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:08.694 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:08.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:08.751 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1901 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:08.808 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:08.809 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:08.809 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.05e-05 s, 0.0 kB/s 2023-08-27T11:23:08.809 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:08.810 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:08.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:08.866 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1902 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:08.923 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:08.924 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:08.924 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9511e-05 s, 0.0 kB/s 2023-08-27T11:23:08.925 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:08.925 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:08.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:08.981 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1903 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:09.038 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:09.039 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:09.039 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.186e-05 s, 0.0 kB/s 2023-08-27T11:23:09.039 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:09.040 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:09.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:09.097 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1904 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:09.154 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:09.154 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:09.154 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8067e-05 s, 0.0 kB/s 2023-08-27T11:23:09.155 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:09.155 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:09.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:09.212 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1905 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:09.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:09.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:09.270 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2932e-05 s, 0.0 kB/s 2023-08-27T11:23:09.270 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:09.270 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:09.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:09.327 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1906 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:09.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:09.384 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:09.384 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9451e-05 s, 0.0 kB/s 2023-08-27T11:23:09.384 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:09.385 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:09.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:09.442 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1907 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:09.498 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:09.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:09.499 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1836e-05 s, 0.0 kB/s 2023-08-27T11:23:09.500 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:09.500 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:09.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:09.556 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1908 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:09.613 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1231e-05 s, 0.0 kB/s 2023-08-27T11:23:09.614 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:09.614 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:09.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:09.671 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1909 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1219e-05 s, 0.0 kB/s 2023-08-27T11:23:09.728 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:09.729 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:09.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:09.786 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1910 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.394e-05 s, 0.0 kB/s 2023-08-27T11:23:09.843 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:09.843 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:09.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:09.900 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1911 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:09.958 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:09.958 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2029e-05 s, 0.0 kB/s 2023-08-27T11:23:09.958 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:09.958 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:10.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:10.016 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1912 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:10.073 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:10.074 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:10.074 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1405e-05 s, 0.0 kB/s 2023-08-27T11:23:10.075 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:10.075 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:10.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:10.132 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1913 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:10.188 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:10.189 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:10.189 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9182e-05 s, 0.0 kB/s 2023-08-27T11:23:10.189 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:10.189 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:10.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:10.245 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1914 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:10.302 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9825e-05 s, 0.0 kB/s 2023-08-27T11:23:10.303 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:10.303 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:10.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:10.361 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1915 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:10.419 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:10.419 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:10.419 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9882e-05 s, 0.0 kB/s 2023-08-27T11:23:10.420 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:10.420 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:10.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:10.476 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1916 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1239e-05 s, 0.0 kB/s 2023-08-27T11:23:10.536 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:10.536 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:10.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:10.593 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1917 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:10.650 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:10.650 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:10.651 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3311e-05 s, 0.0 kB/s 2023-08-27T11:23:10.651 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:10.652 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:10.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:10.708 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1918 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:10.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:10.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:10.769 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8432e-05 s, 0.0 kB/s 2023-08-27T11:23:10.769 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:10.769 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:10.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:10.828 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1919 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:10.886 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:10.886 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:10.887 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0995e-05 s, 0.0 kB/s 2023-08-27T11:23:10.887 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:10.888 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:10.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:10.948 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1920 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:11.023 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:11.023 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:11.024 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0214e-05 s, 0.0 kB/s 2023-08-27T11:23:11.024 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:11.025 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:11.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:11.092 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1921 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:11.162 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1706e-05 s, 0.0 kB/s 2023-08-27T11:23:11.163 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:11.164 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:11.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:11.222 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1922 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0921e-05 s, 0.0 kB/s 2023-08-27T11:23:11.290 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:11.290 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:11.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:11.356 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1923 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:11.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:11.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:11.420 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2337e-05 s, 0.0 kB/s 2023-08-27T11:23:11.421 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:11.422 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:11.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:11.484 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1924 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:11.502 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1925e-05 s, 0.0 kB/s 2023-08-27T11:23:11.503 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:11.504 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:11.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:11.565 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1925 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:11.626 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:11.626 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:11.626 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2655e-05 s, 0.0 kB/s 2023-08-27T11:23:11.627 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:11.627 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:11.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:11.688 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1926 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:11.750 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:11.750 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1077e-05 s, 0.0 kB/s 2023-08-27T11:23:11.751 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:11.751 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:11.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:11.815 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1927 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:11.877 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:11.877 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:11.877 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8898e-05 s, 0.0 kB/s 2023-08-27T11:23:11.878 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:11.878 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:11.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:11.938 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1928 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0528e-05 s, 0.0 kB/s 2023-08-27T11:23:11.996 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:11.997 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:12.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:12.054 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1929 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1278e-05 s, 0.0 kB/s 2023-08-27T11:23:12.113 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:12.113 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:12.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:12.171 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1930 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:12.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:12.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:12.230 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8071e-05 s, 0.0 kB/s 2023-08-27T11:23:12.230 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:12.231 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:12.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:12.289 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1931 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:12.346 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:12.346 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:12.346 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9267e-05 s, 0.0 kB/s 2023-08-27T11:23:12.347 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:12.348 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:12.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:12.405 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1932 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:12.462 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:12.463 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:12.463 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3361e-05 s, 0.0 kB/s 2023-08-27T11:23:12.463 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:12.464 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:12.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:12.521 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1933 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:12.577 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:12.577 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:12.578 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0414e-05 s, 0.0 kB/s 2023-08-27T11:23:12.578 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:12.579 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:12.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:12.635 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1934 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:12.692 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:12.693 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:12.693 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1651e-05 s, 0.0 kB/s 2023-08-27T11:23:12.693 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:12.694 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:12.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:12.751 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1935 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:12.808 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:12.809 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:12.809 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7452e-05 s, 0.0 kB/s 2023-08-27T11:23:12.809 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:12.810 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:12.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:12.866 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1936 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:12.923 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:12.924 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:12.924 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.203e-05 s, 0.0 kB/s 2023-08-27T11:23:12.924 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:12.925 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:12.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:12.982 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1937 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:13.039 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:13.039 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:13.040 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0416e-05 s, 0.0 kB/s 2023-08-27T11:23:13.040 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:13.040 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:13.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:13.097 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1938 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:13.155 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:13.155 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:13.156 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2589e-05 s, 0.0 kB/s 2023-08-27T11:23:13.156 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:13.156 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:13.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:13.213 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1939 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:13.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:13.270 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:13.270 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0054e-05 s, 0.0 kB/s 2023-08-27T11:23:13.270 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:13.271 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:13.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:13.327 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1940 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:13.384 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:13.385 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:13.385 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4379e-05 s, 0.0 kB/s 2023-08-27T11:23:13.386 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:13.386 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:13.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:13.445 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1941 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:13.502 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:13.502 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:13.502 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0095e-05 s, 0.0 kB/s 2023-08-27T11:23:13.503 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:13.503 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:13.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:13.560 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1942 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:13.618 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:13.619 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:13.620 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9617e-05 s, 0.0 kB/s 2023-08-27T11:23:13.620 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:13.620 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:13.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:13.677 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1943 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:13.735 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:13.735 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:13.735 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8869e-05 s, 0.0 kB/s 2023-08-27T11:23:13.735 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:13.736 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:13.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:13.793 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1944 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0537e-05 s, 0.0 kB/s 2023-08-27T11:23:13.851 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:13.851 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:13.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:13.908 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1945 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:13.965 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:13.965 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:13.965 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7841e-05 s, 0.0 kB/s 2023-08-27T11:23:13.966 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:13.966 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:14.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:14.023 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1946 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:14.080 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:14.081 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:14.081 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9212e-05 s, 0.0 kB/s 2023-08-27T11:23:14.081 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:14.082 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:14.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:14.139 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1947 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:14.197 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:14.197 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:14.197 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9876e-05 s, 0.0 kB/s 2023-08-27T11:23:14.198 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:14.198 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:14.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:14.254 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1948 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:14.313 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:14.313 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:14.313 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4179e-05 s, 0.0 kB/s 2023-08-27T11:23:14.314 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:14.314 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:14.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:14.370 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1949 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:14.429 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:14.429 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:14.429 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1087e-05 s, 0.0 kB/s 2023-08-27T11:23:14.430 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:14.430 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:14.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:14.487 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1950 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:14.545 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:14.545 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:14.546 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2874e-05 s, 0.0 kB/s 2023-08-27T11:23:14.546 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:14.546 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:14.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:14.603 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1951 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:14.660 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:14.660 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:14.661 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3182e-05 s, 0.0 kB/s 2023-08-27T11:23:14.661 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:14.661 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:14.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:14.718 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1952 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:14.775 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:14.775 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:14.775 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8009e-05 s, 0.0 kB/s 2023-08-27T11:23:14.776 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:14.776 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:14.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:14.833 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1953 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:14.889 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:14.890 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:14.890 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8038e-05 s, 0.0 kB/s 2023-08-27T11:23:14.890 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:14.891 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:14.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:14.948 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1954 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:15.005 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:15.005 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:15.005 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8595e-05 s, 0.0 kB/s 2023-08-27T11:23:15.006 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:15.006 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:15.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:15.063 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1955 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:15.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:15.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:15.120 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.017e-05 s, 0.0 kB/s 2023-08-27T11:23:15.121 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:15.121 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:15.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:15.178 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1956 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:15.236 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:15.236 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:15.236 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0182e-05 s, 0.0 kB/s 2023-08-27T11:23:15.236 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:15.237 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:15.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:15.294 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1957 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:15.350 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:15.351 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:15.351 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0458e-05 s, 0.0 kB/s 2023-08-27T11:23:15.352 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:15.352 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:15.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:15.409 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1958 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:15.465 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:15.466 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:15.466 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7794e-05 s, 0.0 kB/s 2023-08-27T11:23:15.466 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:15.467 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:15.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:15.523 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1959 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9705e-05 s, 0.0 kB/s 2023-08-27T11:23:15.582 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:15.582 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:15.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:15.639 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1960 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:15.697 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:15.697 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:15.697 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9491e-05 s, 0.0 kB/s 2023-08-27T11:23:15.698 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:15.698 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:15.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:15.759 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1961 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:15.817 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:15.818 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:15.818 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0119e-05 s, 0.0 kB/s 2023-08-27T11:23:15.818 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:15.819 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:15.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:15.876 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1962 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:15.934 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:15.934 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:15.934 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5989e-05 s, 0.0 kB/s 2023-08-27T11:23:15.935 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:15.935 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:15.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:15.992 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1963 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:16.048 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:16.049 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:16.049 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9386e-05 s, 0.0 kB/s 2023-08-27T11:23:16.050 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:16.050 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:16.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:16.107 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1964 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:16.164 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:16.165 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:16.165 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9793e-05 s, 0.0 kB/s 2023-08-27T11:23:16.165 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:16.166 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:16.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:16.223 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1965 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:16.279 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:16.280 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:16.280 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1284e-05 s, 0.0 kB/s 2023-08-27T11:23:16.280 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:16.281 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:16.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:16.338 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1966 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:16.396 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:16.396 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:16.396 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0282e-05 s, 0.0 kB/s 2023-08-27T11:23:16.397 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:16.397 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:16.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:16.455 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1967 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:16.511 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:16.512 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:16.512 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7872e-05 s, 0.0 kB/s 2023-08-27T11:23:16.512 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:16.513 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:16.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:16.574 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1968 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:16.630 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:16.630 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:16.630 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0798e-05 s, 0.0 kB/s 2023-08-27T11:23:16.631 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:16.631 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:16.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:16.688 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1969 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:16.745 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:16.745 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:16.745 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8326e-05 s, 0.0 kB/s 2023-08-27T11:23:16.746 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:16.746 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:16.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:16.803 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1970 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:16.860 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:16.860 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:16.860 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9568e-05 s, 0.0 kB/s 2023-08-27T11:23:16.861 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:16.861 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:16.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:16.918 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1971 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:16.976 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:16.976 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:16.976 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8452e-05 s, 0.0 kB/s 2023-08-27T11:23:16.977 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:16.977 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:17.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:17.034 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1972 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:17.091 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:17.091 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.277e-05 s, 0.0 kB/s 2023-08-27T11:23:17.092 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:17.093 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:17.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:17.150 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1973 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:17.206 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:17.207 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:17.207 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8051e-05 s, 0.0 kB/s 2023-08-27T11:23:17.207 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:17.208 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:17.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:17.264 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1974 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:17.321 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:17.321 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:17.321 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.878e-05 s, 0.0 kB/s 2023-08-27T11:23:17.322 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:17.322 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:17.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:17.379 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1975 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2357e-05 s, 0.0 kB/s 2023-08-27T11:23:17.437 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:17.437 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:17.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:17.493 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1976 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:17.551 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:17.552 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:17.552 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3304e-05 s, 0.0 kB/s 2023-08-27T11:23:17.552 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:17.553 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:17.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:17.609 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1977 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:17.666 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:17.667 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:17.667 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0599e-05 s, 0.0 kB/s 2023-08-27T11:23:17.667 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:17.668 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:17.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:17.725 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1978 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:17.782 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:17.782 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:17.782 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1978e-05 s, 0.0 kB/s 2023-08-27T11:23:17.783 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:17.783 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:17.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:17.840 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1979 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:17.897 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:17.897 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:17.898 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2102e-05 s, 0.0 kB/s 2023-08-27T11:23:17.898 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:17.898 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:17.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:17.955 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1980 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:18.013 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:18.013 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:18.014 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7404e-05 s, 0.0 kB/s 2023-08-27T11:23:18.014 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:18.014 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:18.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:18.071 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1981 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:18.127 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:18.128 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:18.128 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0608e-05 s, 0.0 kB/s 2023-08-27T11:23:18.128 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:18.129 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:18.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:18.186 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1982 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:18.243 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:18.243 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:18.243 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8595e-05 s, 0.0 kB/s 2023-08-27T11:23:18.243 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:18.244 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:18.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:18.301 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1983 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:18.357 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:18.358 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:18.358 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0156e-05 s, 0.0 kB/s 2023-08-27T11:23:18.358 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:18.359 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:18.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:18.417 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1984 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:18.474 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:18.475 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:18.475 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0748e-05 s, 0.0 kB/s 2023-08-27T11:23:18.475 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:18.476 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:18.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:18.532 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1985 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:18.589 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:18.589 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:18.589 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.818e-05 s, 0.0 kB/s 2023-08-27T11:23:18.590 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:18.590 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:18.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:18.646 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1986 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:18.704 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:18.704 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:18.704 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.051e-05 s, 0.0 kB/s 2023-08-27T11:23:18.705 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:18.705 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:18.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:18.762 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1987 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1718e-05 s, 0.0 kB/s 2023-08-27T11:23:18.820 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:18.820 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:18.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:18.878 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1988 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:18.936 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:18.937 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:18.937 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1341e-05 s, 0.0 kB/s 2023-08-27T11:23:18.937 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:18.938 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:18.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:18.994 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1989 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:19.051 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:19.052 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:19.052 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2239e-05 s, 0.0 kB/s 2023-08-27T11:23:19.052 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:19.053 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:19.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:19.112 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1990 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:19.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:19.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:19.169 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9707e-05 s, 0.0 kB/s 2023-08-27T11:23:19.170 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:19.170 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:19.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:19.229 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1991 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:19.286 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:19.287 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:19.287 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5114e-05 s, 0.0 kB/s 2023-08-27T11:23:19.287 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:19.287 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:19.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:19.344 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1992 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2481e-05 s, 0.0 kB/s 2023-08-27T11:23:19.402 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:19.402 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:19.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:19.459 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1993 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:19.515 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:19.515 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:19.516 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7735e-05 s, 0.0 kB/s 2023-08-27T11:23:19.516 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:19.516 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:19.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:19.573 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1994 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9438e-05 s, 0.0 kB/s 2023-08-27T11:23:19.630 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:19.631 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:19.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:19.686 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1995 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:19.744 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:19.744 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:19.744 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9511e-05 s, 0.0 kB/s 2023-08-27T11:23:19.745 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:19.745 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:19.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:19.803 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1996 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:19.858 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:19.859 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:19.859 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9399e-05 s, 0.0 kB/s 2023-08-27T11:23:19.859 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:19.859 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:19.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:19.917 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1997 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:19.975 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:19.975 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:19.975 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7725e-05 s, 0.0 kB/s 2023-08-27T11:23:19.976 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:19.976 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:20.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:20.033 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1998 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:20.091 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:20.091 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:20.091 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0872e-05 s, 0.0 kB/s 2023-08-27T11:23:20.092 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:20.092 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:20.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:20.149 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.1999 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:20.205 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:20.206 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:20.206 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3104e-05 s, 0.0 kB/s 2023-08-27T11:23:20.206 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:20.207 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:20.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:20.263 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2000 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:20.320 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:20.320 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:20.320 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.222e-05 s, 0.0 kB/s 2023-08-27T11:23:20.321 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:20.321 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:20.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:20.378 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2001 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:20.435 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:20.437 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:20.437 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7819e-05 s, 0.0 kB/s 2023-08-27T11:23:20.437 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:20.438 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:20.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:20.493 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2002 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:20.549 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.851e-05 s, 0.0 kB/s 2023-08-27T11:23:20.550 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:20.551 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:20.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:20.608 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2003 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.043e-05 s, 0.0 kB/s 2023-08-27T11:23:20.667 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:20.669 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:20.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:20.724 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2004 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:20.781 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:20.782 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:20.782 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2714e-05 s, 0.0 kB/s 2023-08-27T11:23:20.782 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:20.783 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:20.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:20.839 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2005 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:20.896 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:20.897 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:20.897 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.9826e-05 s, 0.0 kB/s 2023-08-27T11:23:20.897 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:20.898 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:20.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:20.954 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2006 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:21.011 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0354e-05 s, 0.0 kB/s 2023-08-27T11:23:21.012 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:21.013 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:21.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:21.070 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2007 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8904e-05 s, 0.0 kB/s 2023-08-27T11:23:21.129 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:21.129 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:21.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:21.187 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2008 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:21.244 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:21.244 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:21.245 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2761e-05 s, 0.0 kB/s 2023-08-27T11:23:21.245 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:21.245 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:21.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:21.302 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2009 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:21.360 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:21.360 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:21.360 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6649e-05 s, 0.0 kB/s 2023-08-27T11:23:21.361 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:21.361 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:21.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:21.418 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2010 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:21.476 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:21.476 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:21.476 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7688e-05 s, 0.0 kB/s 2023-08-27T11:23:21.477 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:21.477 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:21.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:21.534 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2011 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:21.592 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:21.592 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:21.592 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.256e-05 s, 0.0 kB/s 2023-08-27T11:23:21.593 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:21.593 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:21.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:21.650 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2012 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:21.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:21.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:21.708 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.857e-05 s, 0.0 kB/s 2023-08-27T11:23:21.709 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:21.709 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:21.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:21.766 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2013 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:21.823 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:21.823 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:21.823 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1207e-05 s, 0.0 kB/s 2023-08-27T11:23:21.824 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:21.824 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:21.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:21.883 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2014 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:21.939 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:21.939 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:21.940 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1039e-05 s, 0.0 kB/s 2023-08-27T11:23:21.940 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:21.941 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:21.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:21.996 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2015 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:22.055 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:22.056 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:22.056 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9411e-05 s, 0.0 kB/s 2023-08-27T11:23:22.057 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:22.058 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:22.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:22.113 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2016 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:22.171 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:22.171 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:22.171 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8947e-05 s, 0.0 kB/s 2023-08-27T11:23:22.172 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:22.172 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:22.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:22.228 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2017 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:22.286 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:22.286 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:22.286 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9832e-05 s, 0.0 kB/s 2023-08-27T11:23:22.287 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:22.287 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:22.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:22.344 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2018 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1493e-05 s, 0.0 kB/s 2023-08-27T11:23:22.401 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:22.402 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:22.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:22.459 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2019 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:22.515 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:22.516 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:22.516 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2296e-05 s, 0.0 kB/s 2023-08-27T11:23:22.516 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:22.517 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:22.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:22.572 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2020 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:22.630 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:22.630 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:22.631 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.045e-05 s, 0.0 kB/s 2023-08-27T11:23:22.631 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:22.631 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:22.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:22.688 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2021 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:22.745 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:22.745 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1658e-05 s, 0.0 kB/s 2023-08-27T11:23:22.745 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:22.746 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:22.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:22.803 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2022 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:22.859 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:22.860 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:22.860 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5161e-05 s, 0.0 kB/s 2023-08-27T11:23:22.860 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:22.860 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:22.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:22.917 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2023 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:22.974 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:22.974 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:22.974 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9599e-05 s, 0.0 kB/s 2023-08-27T11:23:22.975 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:22.975 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:23.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:23.032 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2024 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9518e-05 s, 0.0 kB/s 2023-08-27T11:23:23.091 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:23.091 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:23.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:23.148 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2025 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:23.205 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:23.205 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:23.205 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0293e-05 s, 0.0 kB/s 2023-08-27T11:23:23.206 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:23.206 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:23.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:23.263 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2026 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:23.320 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:23.320 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:23.320 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2731e-05 s, 0.0 kB/s 2023-08-27T11:23:23.321 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:23.321 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:23.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:23.378 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2027 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:23.435 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:23.435 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:23.435 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7523e-05 s, 0.0 kB/s 2023-08-27T11:23:23.436 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:23.436 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:23.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:23.493 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2028 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0611e-05 s, 0.0 kB/s 2023-08-27T11:23:23.551 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:23.551 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:23.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:23.608 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2029 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:23.664 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:23.665 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:23.665 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8399e-05 s, 0.0 kB/s 2023-08-27T11:23:23.665 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:23.666 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:23.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:23.724 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2030 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:23.781 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:23.781 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:23.781 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.154e-05 s, 0.0 kB/s 2023-08-27T11:23:23.782 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:23.782 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:23.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:23.839 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2031 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:23.897 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:23.897 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:23.897 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0778e-05 s, 0.0 kB/s 2023-08-27T11:23:23.898 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:23.898 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:23.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:23.955 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2032 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:24.012 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:24.012 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:24.013 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1766e-05 s, 0.0 kB/s 2023-08-27T11:23:24.013 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:24.014 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:24.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:24.070 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2033 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:24.127 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:24.128 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:24.128 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0656e-05 s, 0.0 kB/s 2023-08-27T11:23:24.128 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:24.129 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:24.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:24.186 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2034 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:24.242 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:24.243 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:24.243 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9131e-05 s, 0.0 kB/s 2023-08-27T11:23:24.243 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:24.244 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:24.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:24.301 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2035 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:24.361 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:24.361 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:24.361 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9596e-05 s, 0.0 kB/s 2023-08-27T11:23:24.362 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:24.362 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:24.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:24.420 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2036 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:24.476 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:24.477 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:24.477 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.7369e-05 s, 0.0 kB/s 2023-08-27T11:23:24.477 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:24.478 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:24.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:24.534 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2037 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:24.592 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:24.592 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:24.592 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3079e-05 s, 0.0 kB/s 2023-08-27T11:23:24.593 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:24.593 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:24.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:24.649 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2038 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9772e-05 s, 0.0 kB/s 2023-08-27T11:23:24.708 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:24.708 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:24.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:24.765 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2039 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:24.822 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:24.822 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:24.822 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9259e-05 s, 0.0 kB/s 2023-08-27T11:23:24.823 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:24.823 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:24.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:24.879 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2040 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:24.937 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:24.937 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8876e-05 s, 0.0 kB/s 2023-08-27T11:23:24.937 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:24.938 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:24.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:24.993 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2041 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:25.051 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:25.051 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:25.051 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7404e-05 s, 0.0 kB/s 2023-08-27T11:23:25.051 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:25.052 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:25.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:25.109 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2042 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:25.168 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:25.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:25.169 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9301e-05 s, 0.0 kB/s 2023-08-27T11:23:25.169 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:25.170 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:25.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:25.223 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2043 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:25.280 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:25.280 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:25.281 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4076e-05 s, 0.0 kB/s 2023-08-27T11:23:25.281 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:25.281 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:25.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:25.337 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2044 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:25.394 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:25.395 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:25.395 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0076e-05 s, 0.0 kB/s 2023-08-27T11:23:25.395 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:25.396 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:25.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:25.453 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2045 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:25.510 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:25.510 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:25.510 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2044e-05 s, 0.0 kB/s 2023-08-27T11:23:25.511 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:25.511 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:25.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:25.567 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2046 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:25.625 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:25.625 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:25.625 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2219e-05 s, 0.0 kB/s 2023-08-27T11:23:25.626 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:25.626 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:25.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:25.683 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2047 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:25.741 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:25.741 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:25.742 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1825e-05 s, 0.0 kB/s 2023-08-27T11:23:25.743 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:25.743 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:25.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:25.799 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2048 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:25.856 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:25.856 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:25.856 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3316e-05 s, 0.0 kB/s 2023-08-27T11:23:25.857 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:25.857 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:25.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:25.914 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2049 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:25.970 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:25.971 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:25.971 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9884e-05 s, 0.0 kB/s 2023-08-27T11:23:25.972 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:25.972 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:26.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:26.028 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2050 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:26.085 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:26.086 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:26.086 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2948e-05 s, 0.0 kB/s 2023-08-27T11:23:26.086 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:26.087 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:26.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:26.143 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2051 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.972e-05 s, 0.0 kB/s 2023-08-27T11:23:26.200 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:26.201 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:26.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:26.258 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2052 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0304e-05 s, 0.0 kB/s 2023-08-27T11:23:26.316 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:26.316 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:26.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:26.372 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2053 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:26.428 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:26.429 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:26.429 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.221e-05 s, 0.0 kB/s 2023-08-27T11:23:26.429 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:26.430 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:26.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:26.486 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2054 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:26.543 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:26.543 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:26.543 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9165e-05 s, 0.0 kB/s 2023-08-27T11:23:26.544 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:26.544 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:26.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:26.601 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2055 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:26.658 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:26.658 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:26.658 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0683e-05 s, 0.0 kB/s 2023-08-27T11:23:26.659 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:26.659 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:26.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:26.716 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2056 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:26.773 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:26.774 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:26.774 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8618e-05 s, 0.0 kB/s 2023-08-27T11:23:26.774 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:26.775 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:26.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:26.832 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2057 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:26.889 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:26.890 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:26.890 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2632e-05 s, 0.0 kB/s 2023-08-27T11:23:26.890 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:26.891 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:26.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:26.948 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2058 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9107e-05 s, 0.0 kB/s 2023-08-27T11:23:27.006 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:27.006 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:27.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:27.064 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2059 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:27.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:27.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:27.121 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0212e-05 s, 0.0 kB/s 2023-08-27T11:23:27.121 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:27.121 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:27.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:27.177 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2060 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:27.235 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:27.235 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:27.235 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7802e-05 s, 0.0 kB/s 2023-08-27T11:23:27.236 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:27.236 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:27.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:27.293 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2061 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:27.350 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:27.351 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:27.351 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.044e-05 s, 0.0 kB/s 2023-08-27T11:23:27.351 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:27.352 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:27.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:27.408 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2062 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:27.465 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1798e-05 s, 0.0 kB/s 2023-08-27T11:23:27.466 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:27.467 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:27.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:27.524 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2063 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:27.581 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:27.581 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:27.581 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7299e-05 s, 0.0 kB/s 2023-08-27T11:23:27.582 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:27.582 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:27.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:27.640 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2064 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:27.696 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:27.697 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:27.697 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.848e-05 s, 0.0 kB/s 2023-08-27T11:23:27.697 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:27.698 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:27.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:27.754 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2065 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:27.810 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:27.810 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:27.811 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1325e-05 s, 0.0 kB/s 2023-08-27T11:23:27.811 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:27.811 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:27.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:27.867 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2066 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:27.924 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:27.925 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:27.925 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9648e-05 s, 0.0 kB/s 2023-08-27T11:23:27.925 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:27.926 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:27.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:27.982 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2067 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:28.038 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:28.038 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:28.038 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7588e-05 s, 0.0 kB/s 2023-08-27T11:23:28.039 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:28.040 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:28.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:28.097 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2068 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:28.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:28.154 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:28.154 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8201e-05 s, 0.0 kB/s 2023-08-27T11:23:28.154 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:28.154 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:28.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:28.211 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2069 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:28.268 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:28.268 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:28.268 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.147e-05 s, 0.0 kB/s 2023-08-27T11:23:28.269 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:28.269 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:28.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:28.326 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2070 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:28.384 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:28.384 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:28.385 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3893e-05 s, 0.0 kB/s 2023-08-27T11:23:28.385 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:28.385 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:28.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:28.442 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2071 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:28.498 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:28.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:28.499 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2806e-05 s, 0.0 kB/s 2023-08-27T11:23:28.499 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:28.500 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:28.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:28.557 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2072 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:28.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:28.614 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0755e-05 s, 0.0 kB/s 2023-08-27T11:23:28.614 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:28.615 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:28.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:28.672 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2073 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:28.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:28.730 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:28.730 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9871e-05 s, 0.0 kB/s 2023-08-27T11:23:28.730 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:28.731 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:28.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:28.788 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2074 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:28.849 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:28.849 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:28.849 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8506e-05 s, 0.0 kB/s 2023-08-27T11:23:28.850 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:28.850 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:28.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:28.907 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2075 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:28.965 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:28.965 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:28.965 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8512e-05 s, 0.0 kB/s 2023-08-27T11:23:28.966 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:28.966 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:29.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:29.023 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2076 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:29.080 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:29.080 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:29.081 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1285e-05 s, 0.0 kB/s 2023-08-27T11:23:29.081 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:29.082 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:29.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:29.138 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2077 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:29.195 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:29.196 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:29.196 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8326e-05 s, 0.0 kB/s 2023-08-27T11:23:29.197 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:29.197 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:29.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:29.255 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2078 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:29.312 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:29.312 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:29.313 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0744e-05 s, 0.0 kB/s 2023-08-27T11:23:29.313 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:29.313 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:29.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:29.370 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2079 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:29.427 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:29.427 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2232e-05 s, 0.0 kB/s 2023-08-27T11:23:29.428 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:29.428 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:29.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:29.485 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2080 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:29.541 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:29.542 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:29.542 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1119e-05 s, 0.0 kB/s 2023-08-27T11:23:29.542 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:29.543 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:29.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:29.600 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2081 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:29.658 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:29.658 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:29.658 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7745e-05 s, 0.0 kB/s 2023-08-27T11:23:29.659 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:29.659 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:29.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:29.716 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2082 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1137e-05 s, 0.0 kB/s 2023-08-27T11:23:29.774 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:29.774 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:29.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:29.832 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2083 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:29.890 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:29.890 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:29.890 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0688e-05 s, 0.0 kB/s 2023-08-27T11:23:29.891 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:29.892 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:29.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:29.948 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2084 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:30.004 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:30.005 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:30.005 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0784e-05 s, 0.0 kB/s 2023-08-27T11:23:30.005 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:30.006 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:30.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:30.063 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2085 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:30.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:30.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:30.120 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1045e-05 s, 0.0 kB/s 2023-08-27T11:23:30.121 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:30.121 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:30.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:30.178 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2086 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:30.236 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:30.236 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:30.236 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8969e-05 s, 0.0 kB/s 2023-08-27T11:23:30.237 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:30.237 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:30.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:30.294 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2087 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:30.352 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:30.352 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:30.352 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8098e-05 s, 0.0 kB/s 2023-08-27T11:23:30.353 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:30.353 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:30.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:30.410 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2088 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2232e-05 s, 0.0 kB/s 2023-08-27T11:23:30.469 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:30.469 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:30.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:30.526 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2089 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:30.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:30.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:30.583 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9273e-05 s, 0.0 kB/s 2023-08-27T11:23:30.583 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:30.584 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:30.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:30.640 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2090 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:30.698 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5014e-05 s, 0.0 kB/s 2023-08-27T11:23:30.699 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:30.700 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:30.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:30.755 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2091 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:30.813 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:30.813 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:30.814 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2277e-05 s, 0.0 kB/s 2023-08-27T11:23:30.814 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:30.814 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:30.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:30.871 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2092 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:30.927 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:30.927 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:30.928 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0886e-05 s, 0.0 kB/s 2023-08-27T11:23:30.928 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:30.928 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:30.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:30.985 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2093 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:31.042 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:31.042 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:31.042 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.888e-05 s, 0.0 kB/s 2023-08-27T11:23:31.043 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:31.043 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:31.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:31.101 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2094 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9221e-05 s, 0.0 kB/s 2023-08-27T11:23:31.159 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:31.159 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:31.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:31.216 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2095 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1734e-05 s, 0.0 kB/s 2023-08-27T11:23:31.275 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:31.275 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:31.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:31.332 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2096 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:31.390 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:31.391 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:31.391 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1167e-05 s, 0.0 kB/s 2023-08-27T11:23:31.391 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:31.392 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:31.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:31.447 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2097 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.264e-05 s, 0.0 kB/s 2023-08-27T11:23:31.505 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:31.505 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:31.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:31.562 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2098 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2964e-05 s, 0.0 kB/s 2023-08-27T11:23:31.621 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:31.621 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:31.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:31.678 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2099 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:31.735 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:31.735 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:31.735 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3341e-05 s, 0.0 kB/s 2023-08-27T11:23:31.736 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:31.736 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:31.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:31.793 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2100 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:31.848 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:31.849 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:31.849 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7648e-05 s, 0.0 kB/s 2023-08-27T11:23:31.850 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:31.850 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:31.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:31.907 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2101 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:31.964 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:31.965 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:31.965 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7811e-05 s, 0.0 kB/s 2023-08-27T11:23:31.965 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:31.966 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:32.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:32.024 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2102 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:32.080 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:32.081 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:32.081 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4064e-05 s, 0.0 kB/s 2023-08-27T11:23:32.081 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:32.082 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:32.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:32.139 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2103 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:32.196 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:32.197 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:32.197 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9739e-05 s, 0.0 kB/s 2023-08-27T11:23:32.197 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:32.198 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:32.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:32.255 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2104 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:32.312 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:32.312 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:32.313 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3059e-05 s, 0.0 kB/s 2023-08-27T11:23:32.313 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:32.314 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:32.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:32.371 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2105 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1375e-05 s, 0.0 kB/s 2023-08-27T11:23:32.429 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:32.430 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:32.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:32.488 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2106 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:32.546 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:32.546 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:32.547 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9105e-05 s, 0.0 kB/s 2023-08-27T11:23:32.547 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:32.548 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:32.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:32.604 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2107 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:32.661 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:32.662 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:32.662 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0144e-05 s, 0.0 kB/s 2023-08-27T11:23:32.663 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:32.663 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:32.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:32.721 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2108 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:32.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:32.778 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:32.778 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9863e-05 s, 0.0 kB/s 2023-08-27T11:23:32.778 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:32.779 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:32.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:32.836 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2109 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:32.892 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:32.892 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:32.892 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2345e-05 s, 0.0 kB/s 2023-08-27T11:23:32.893 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:32.894 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:32.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:32.953 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2110 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:33.011 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:33.011 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:33.011 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0711e-05 s, 0.0 kB/s 2023-08-27T11:23:33.012 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:33.012 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:33.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:33.068 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2111 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:33.125 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:33.125 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:33.126 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9898e-05 s, 0.0 kB/s 2023-08-27T11:23:33.126 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:33.126 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:33.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:33.183 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2112 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:33.240 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:33.240 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:33.240 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3139e-05 s, 0.0 kB/s 2023-08-27T11:23:33.241 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:33.241 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:33.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:33.298 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2113 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:33.356 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:33.356 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:33.357 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9374e-05 s, 0.0 kB/s 2023-08-27T11:23:33.357 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:33.357 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:33.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:33.414 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2114 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1451e-05 s, 0.0 kB/s 2023-08-27T11:23:33.473 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:33.473 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:33.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:33.530 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2115 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:33.588 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:33.588 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:33.588 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.754e-05 s, 0.0 kB/s 2023-08-27T11:23:33.589 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:33.589 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:33.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:33.646 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2116 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:33.702 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:33.703 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:33.703 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8442e-05 s, 0.0 kB/s 2023-08-27T11:23:33.703 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:33.704 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:33.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:33.761 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2117 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:33.818 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:33.818 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:33.819 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.287e-05 s, 0.0 kB/s 2023-08-27T11:23:33.819 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:33.820 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:33.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:33.876 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2118 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:33.933 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:33.934 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:33.934 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2318e-05 s, 0.0 kB/s 2023-08-27T11:23:33.934 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:33.935 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:33.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:33.991 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2119 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:34.047 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:34.048 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:34.048 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.15e-05 s, 0.0 kB/s 2023-08-27T11:23:34.048 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:34.049 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:34.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:34.105 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2120 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:34.162 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:34.162 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:34.162 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0149e-05 s, 0.0 kB/s 2023-08-27T11:23:34.163 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:34.163 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:34.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:34.220 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2121 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:34.278 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:34.278 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:34.278 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7892e-05 s, 0.0 kB/s 2023-08-27T11:23:34.278 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:34.279 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:34.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:34.336 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2122 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:34.393 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:34.394 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:34.394 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8085e-05 s, 0.0 kB/s 2023-08-27T11:23:34.394 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:34.395 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:34.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:34.451 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2123 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:34.508 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:34.508 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:34.508 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.238e-05 s, 0.0 kB/s 2023-08-27T11:23:34.508 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:34.509 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:34.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:34.566 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2124 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:34.622 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.105e-05 s, 0.0 kB/s 2023-08-27T11:23:34.623 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:34.624 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:34.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:34.681 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2125 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:34.737 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:34.738 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:34.738 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0791e-05 s, 0.0 kB/s 2023-08-27T11:23:34.738 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:34.739 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:34.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:34.796 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2126 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:34.852 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:34.853 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:34.853 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2209e-05 s, 0.0 kB/s 2023-08-27T11:23:34.853 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:34.854 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:34.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:34.912 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2127 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:34.968 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:34.969 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:34.969 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1552e-05 s, 0.0 kB/s 2023-08-27T11:23:34.970 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:34.970 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:35.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:35.026 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2128 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:35.083 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:35.083 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:35.083 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8486e-05 s, 0.0 kB/s 2023-08-27T11:23:35.084 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:35.084 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:35.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:35.141 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2129 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:35.201 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:35.201 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:35.201 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2506e-05 s, 0.0 kB/s 2023-08-27T11:23:35.202 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:35.202 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:35.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:35.259 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2130 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:35.315 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:35.316 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:35.316 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3405e-05 s, 0.0 kB/s 2023-08-27T11:23:35.316 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:35.317 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:35.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:35.373 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2131 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:35.431 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:35.431 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:35.431 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1426e-05 s, 0.0 kB/s 2023-08-27T11:23:35.432 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:35.432 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:35.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:35.489 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2132 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:35.546 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:35.547 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:35.547 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6168e-05 s, 0.0 kB/s 2023-08-27T11:23:35.547 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:35.547 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:35.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:35.604 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2133 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:35.661 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:35.661 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:35.661 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9239e-05 s, 0.0 kB/s 2023-08-27T11:23:35.662 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:35.662 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:35.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:35.719 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2134 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:35.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:35.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:35.777 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.6366e-05 s, 0.0 kB/s 2023-08-27T11:23:35.778 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:35.778 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:35.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:35.835 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2135 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:35.893 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:35.893 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:35.893 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.265e-05 s, 0.0 kB/s 2023-08-27T11:23:35.894 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:35.894 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:35.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:35.951 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2136 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:36.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:36.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:36.008 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8211e-05 s, 0.0 kB/s 2023-08-27T11:23:36.009 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:36.009 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:36.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:36.066 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2137 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:36.124 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:36.124 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0255e-05 s, 0.0 kB/s 2023-08-27T11:23:36.124 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:36.125 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:36.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:36.182 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2138 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1307e-05 s, 0.0 kB/s 2023-08-27T11:23:36.243 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:36.244 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:36.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:36.301 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2139 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:36.359 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:36.359 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:36.359 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9161e-05 s, 0.0 kB/s 2023-08-27T11:23:36.360 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:36.360 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:36.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:36.417 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2140 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:36.474 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:36.474 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:36.474 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0834e-05 s, 0.0 kB/s 2023-08-27T11:23:36.475 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:36.475 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:36.509 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:23:36.517 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:23:36.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:36.531 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2141 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.167e-05 s, 0.0 kB/s 2023-08-27T11:23:36.623 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:36.623 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:36.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:36.680 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2142 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:36.737 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:36.738 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:36.738 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.998e-05 s, 0.0 kB/s 2023-08-27T11:23:36.738 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:36.739 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:36.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:36.795 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2143 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:36.852 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:36.852 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9682e-05 s, 0.0 kB/s 2023-08-27T11:23:36.853 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:36.853 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:36.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:36.910 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2144 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:36.966 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:36.966 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:36.967 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8105e-05 s, 0.0 kB/s 2023-08-27T11:23:36.968 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:36.968 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:37.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:37.026 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2145 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:37.083 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:37.083 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:37.083 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2955e-05 s, 0.0 kB/s 2023-08-27T11:23:37.083 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:37.084 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:37.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:37.141 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2146 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:37.198 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:37.198 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:37.198 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.429e-05 s, 0.0 kB/s 2023-08-27T11:23:37.199 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:37.199 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:37.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:37.256 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2147 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:37.313 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:37.314 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:37.314 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2701e-05 s, 0.0 kB/s 2023-08-27T11:23:37.314 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:37.315 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:37.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:37.372 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2148 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:37.430 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:37.430 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:37.431 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9969e-05 s, 0.0 kB/s 2023-08-27T11:23:37.431 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:37.431 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:37.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:37.488 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2149 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.79e-05 s, 0.0 kB/s 2023-08-27T11:23:37.546 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:37.546 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:37.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:37.603 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2150 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:37.661 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:37.661 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:37.662 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9578e-05 s, 0.0 kB/s 2023-08-27T11:23:37.662 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:37.663 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:37.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:37.719 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2151 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:37.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:37.778 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:37.778 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3993e-05 s, 0.0 kB/s 2023-08-27T11:23:37.779 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:37.779 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:37.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:37.836 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2152 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:37.893 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:37.893 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:37.893 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9962e-05 s, 0.0 kB/s 2023-08-27T11:23:37.894 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:37.894 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:37.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:37.951 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2153 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1465e-05 s, 0.0 kB/s 2023-08-27T11:23:38.009 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:38.009 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:38.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:38.066 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2154 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:38.123 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:38.123 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:38.123 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9901e-05 s, 0.0 kB/s 2023-08-27T11:23:38.124 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:38.124 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:38.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:38.181 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2155 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:38.239 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3038e-05 s, 0.0 kB/s 2023-08-27T11:23:38.239 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:38.239 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:38.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:38.296 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2156 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:38.354 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:38.354 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:38.355 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.248e-05 s, 0.0 kB/s 2023-08-27T11:23:38.355 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:38.355 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:38.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:38.412 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2157 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:38.470 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:38.471 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:38.471 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0621e-05 s, 0.0 kB/s 2023-08-27T11:23:38.471 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:38.472 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:38.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:38.528 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2158 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8789e-05 s, 0.0 kB/s 2023-08-27T11:23:38.587 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:38.587 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:38.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:38.644 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2159 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2351e-05 s, 0.0 kB/s 2023-08-27T11:23:38.704 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:38.704 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:38.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:38.761 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2160 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:38.818 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:38.818 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:38.819 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0483e-05 s, 0.0 kB/s 2023-08-27T11:23:38.819 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:38.819 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:38.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:38.876 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2161 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:38.934 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:38.934 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:38.934 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.767e-05 s, 0.0 kB/s 2023-08-27T11:23:38.935 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:38.935 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:38.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:38.992 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2162 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2379e-05 s, 0.0 kB/s 2023-08-27T11:23:39.050 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:39.051 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:39.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:39.108 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2163 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:39.165 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:39.166 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:39.166 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0254e-05 s, 0.0 kB/s 2023-08-27T11:23:39.166 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:39.167 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:39.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:39.223 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2164 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:39.281 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:39.281 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:39.281 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3399e-05 s, 0.0 kB/s 2023-08-27T11:23:39.282 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:39.282 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:39.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:39.339 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2165 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:39.397 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:39.397 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:39.397 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0344e-05 s, 0.0 kB/s 2023-08-27T11:23:39.398 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:39.398 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:39.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:39.455 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2166 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:39.514 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:39.514 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:39.514 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9434e-05 s, 0.0 kB/s 2023-08-27T11:23:39.515 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:39.515 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:39.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:39.572 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2167 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:39.629 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:39.629 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:39.629 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.367e-05 s, 0.0 kB/s 2023-08-27T11:23:39.630 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:39.630 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:39.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:39.687 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2168 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:39.743 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:39.744 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:39.744 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0218e-05 s, 0.0 kB/s 2023-08-27T11:23:39.745 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:39.745 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:39.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:39.803 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2169 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1405e-05 s, 0.0 kB/s 2023-08-27T11:23:39.862 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:39.862 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:39.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:39.919 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2170 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:39.975 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:39.976 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:39.976 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4449e-05 s, 0.0 kB/s 2023-08-27T11:23:39.976 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:39.977 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:40.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:40.033 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2171 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:40.091 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:40.092 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:40.092 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9169e-05 s, 0.0 kB/s 2023-08-27T11:23:40.093 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:40.093 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:40.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:40.151 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2172 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9499e-05 s, 0.0 kB/s 2023-08-27T11:23:40.208 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:40.209 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:40.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:40.267 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2173 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:40.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:40.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:40.324 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8417e-05 s, 0.0 kB/s 2023-08-27T11:23:40.325 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:40.325 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:40.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:40.382 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2174 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:40.439 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:40.439 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:40.439 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8737e-05 s, 0.0 kB/s 2023-08-27T11:23:40.440 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:40.440 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:40.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:40.498 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2175 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:40.558 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:40.558 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:40.559 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0543e-05 s, 0.0 kB/s 2023-08-27T11:23:40.559 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:40.559 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:40.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:40.616 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2176 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:40.674 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:40.674 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:40.674 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9811e-05 s, 0.0 kB/s 2023-08-27T11:23:40.675 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:40.675 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:40.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:40.732 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2177 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:40.789 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:40.789 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:40.790 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1078e-05 s, 0.0 kB/s 2023-08-27T11:23:40.791 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:40.791 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:40.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:40.847 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2178 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:40.904 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:40.904 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:40.904 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9639e-05 s, 0.0 kB/s 2023-08-27T11:23:40.905 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:40.906 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:40.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:40.964 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2179 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:41.021 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:41.022 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:41.022 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8754e-05 s, 0.0 kB/s 2023-08-27T11:23:41.022 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:41.023 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:41.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:41.080 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2180 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:41.136 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:41.136 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:41.137 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.996e-05 s, 0.0 kB/s 2023-08-27T11:23:41.137 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:41.137 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:41.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:41.194 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2181 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:41.251 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:41.251 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0211e-05 s, 0.0 kB/s 2023-08-27T11:23:41.251 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:41.252 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:41.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:41.309 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2182 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:41.366 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:41.366 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:41.366 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8313e-05 s, 0.0 kB/s 2023-08-27T11:23:41.367 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:41.367 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:41.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:41.424 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2183 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:41.480 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:41.481 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:41.481 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9745e-05 s, 0.0 kB/s 2023-08-27T11:23:41.481 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:41.482 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:41.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:41.539 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2184 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:41.597 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:41.598 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:41.598 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8182e-05 s, 0.0 kB/s 2023-08-27T11:23:41.599 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:41.599 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:41.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:41.655 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2185 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:41.714 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1116e-05 s, 0.0 kB/s 2023-08-27T11:23:41.714 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:41.714 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:41.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:41.788 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2186 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:41.848 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:41.848 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:41.848 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.0005e-05 s, 0.0 kB/s 2023-08-27T11:23:41.849 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:41.849 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:41.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:41.910 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2187 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:41.970 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:41.971 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:41.971 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4512e-05 s, 0.0 kB/s 2023-08-27T11:23:41.971 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:41.972 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:42.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:42.034 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2188 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:42.091 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:42.091 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:42.091 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2246e-05 s, 0.0 kB/s 2023-08-27T11:23:42.092 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:42.092 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:42.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:42.149 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2189 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:42.206 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:42.207 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:42.207 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8424e-05 s, 0.0 kB/s 2023-08-27T11:23:42.207 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:42.208 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:42.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:42.269 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2190 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:42.328 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:42.329 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:42.329 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7846e-05 s, 0.0 kB/s 2023-08-27T11:23:42.329 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:42.330 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:42.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:42.391 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2191 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:42.454 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:42.455 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:42.455 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.058e-05 s, 0.0 kB/s 2023-08-27T11:23:42.455 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:42.456 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:42.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:42.520 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2192 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:42.576 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1816e-05 s, 0.0 kB/s 2023-08-27T11:23:42.578 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:42.578 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:42.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:42.640 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2193 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3129e-05 s, 0.0 kB/s 2023-08-27T11:23:42.703 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:42.703 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:42.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:42.769 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2194 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:42.831 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:42.831 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:42.831 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.11e-05 s, 0.0 kB/s 2023-08-27T11:23:42.832 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:42.833 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:42.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:42.894 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2195 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:42.953 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:42.954 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:42.954 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0038e-05 s, 0.0 kB/s 2023-08-27T11:23:42.954 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:42.955 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:43.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:43.013 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2196 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3009e-05 s, 0.0 kB/s 2023-08-27T11:23:43.075 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:43.075 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:43.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:43.133 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2197 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:43.191 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:43.191 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0707e-05 s, 0.0 kB/s 2023-08-27T11:23:43.192 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:43.193 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:43.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:43.250 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2198 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:43.308 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:43.308 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:43.308 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6527e-05 s, 0.0 kB/s 2023-08-27T11:23:43.309 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:43.309 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:43.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:43.366 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2199 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:43.425 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:43.425 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:43.425 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.268e-05 s, 0.0 kB/s 2023-08-27T11:23:43.426 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:43.426 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:43.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:43.483 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2200 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:43.540 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:43.541 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:43.541 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.402e-05 s, 0.0 kB/s 2023-08-27T11:23:43.541 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:43.542 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:43.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:43.598 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2201 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:43.656 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:43.656 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:43.656 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0608e-05 s, 0.0 kB/s 2023-08-27T11:23:43.657 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:43.657 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:43.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:43.714 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2202 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:43.771 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:43.771 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:43.771 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1028e-05 s, 0.0 kB/s 2023-08-27T11:23:43.772 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:43.772 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:43.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:43.829 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2203 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:43.886 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:43.886 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:43.886 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2379e-05 s, 0.0 kB/s 2023-08-27T11:23:43.887 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:43.887 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:43.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:43.944 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2204 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:44.001 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:44.002 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:44.002 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8265e-05 s, 0.0 kB/s 2023-08-27T11:23:44.003 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:44.003 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:44.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:44.060 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2205 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:44.117 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:44.117 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:44.117 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7561e-05 s, 0.0 kB/s 2023-08-27T11:23:44.118 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:44.118 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:44.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:44.176 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2206 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:44.233 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:44.234 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:44.234 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2749e-05 s, 0.0 kB/s 2023-08-27T11:23:44.235 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:44.235 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:44.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:44.293 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2207 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.062e-05 s, 0.0 kB/s 2023-08-27T11:23:44.352 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:44.352 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:44.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:44.410 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2208 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:44.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:44.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:44.468 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9697e-05 s, 0.0 kB/s 2023-08-27T11:23:44.469 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:44.469 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:44.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:44.526 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2209 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:44.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:44.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:44.584 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.062e-05 s, 0.0 kB/s 2023-08-27T11:23:44.584 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:44.585 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:44.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:44.646 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2210 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:44.707 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:44.707 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:44.707 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1548e-05 s, 0.0 kB/s 2023-08-27T11:23:44.708 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:44.708 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:44.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:44.765 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2211 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0586e-05 s, 0.0 kB/s 2023-08-27T11:23:44.824 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:44.824 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:44.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:44.882 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2212 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:44.940 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:44.941 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:44.941 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9206e-05 s, 0.0 kB/s 2023-08-27T11:23:44.941 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:44.942 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:44.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:45.000 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2213 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:45.056 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:45.056 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:45.057 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8271e-05 s, 0.0 kB/s 2023-08-27T11:23:45.057 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:45.058 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:45.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:45.114 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2214 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:45.172 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:45.172 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:45.172 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7784e-05 s, 0.0 kB/s 2023-08-27T11:23:45.173 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:45.173 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:45.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:45.230 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2215 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9921e-05 s, 0.0 kB/s 2023-08-27T11:23:45.288 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:45.288 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:45.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:45.345 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2216 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:45.402 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:45.402 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:45.402 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0014e-05 s, 0.0 kB/s 2023-08-27T11:23:45.403 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:45.403 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:45.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:45.460 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2217 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:45.518 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:45.518 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:45.518 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7674e-05 s, 0.0 kB/s 2023-08-27T11:23:45.519 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:45.519 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:45.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:45.576 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2218 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:45.633 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:45.633 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8065e-05 s, 0.0 kB/s 2023-08-27T11:23:45.634 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:45.635 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:45.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:45.693 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2219 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1081e-05 s, 0.0 kB/s 2023-08-27T11:23:45.751 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:45.751 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:45.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:45.808 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2220 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:45.865 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:45.865 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:45.865 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1193e-05 s, 0.0 kB/s 2023-08-27T11:23:45.866 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:45.866 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:45.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:45.924 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2221 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:45.981 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:45.981 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:45.981 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3085e-05 s, 0.0 kB/s 2023-08-27T11:23:45.982 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:45.982 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:46.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:46.039 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2222 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:46.097 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:46.097 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:46.098 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.167e-05 s, 0.0 kB/s 2023-08-27T11:23:46.098 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:46.098 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:46.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:46.155 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2223 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:46.212 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:46.213 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:46.213 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2696e-05 s, 0.0 kB/s 2023-08-27T11:23:46.213 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:46.214 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:46.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:46.271 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2224 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:46.329 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:46.330 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:46.330 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0895e-05 s, 0.0 kB/s 2023-08-27T11:23:46.330 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:46.331 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:46.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:46.387 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2225 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:46.445 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:46.445 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:46.445 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9655e-05 s, 0.0 kB/s 2023-08-27T11:23:46.446 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:46.446 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:46.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:46.506 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2226 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:46.563 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:46.563 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:46.563 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9479e-05 s, 0.0 kB/s 2023-08-27T11:23:46.564 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:46.564 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:46.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:46.621 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2227 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:46.678 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:46.678 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:46.679 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1506e-05 s, 0.0 kB/s 2023-08-27T11:23:46.679 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:46.680 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:46.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:46.736 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2228 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:46.793 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:46.793 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:46.793 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1335e-05 s, 0.0 kB/s 2023-08-27T11:23:46.794 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:46.794 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:46.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:46.854 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2229 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:46.912 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:46.913 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:46.913 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7016e-05 s, 0.0 kB/s 2023-08-27T11:23:46.914 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:46.914 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:46.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:46.970 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2230 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:47.026 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:47.027 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:47.027 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0257e-05 s, 0.0 kB/s 2023-08-27T11:23:47.027 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:47.028 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:47.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:47.085 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2231 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:47.143 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.7035e-05 s, 0.0 kB/s 2023-08-27T11:23:47.144 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:47.145 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:47.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:47.201 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2232 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:47.258 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:47.259 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:47.259 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8438e-05 s, 0.0 kB/s 2023-08-27T11:23:47.260 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:47.260 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:47.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:47.318 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2233 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:47.375 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:47.375 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:47.376 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1498e-05 s, 0.0 kB/s 2023-08-27T11:23:47.376 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:47.376 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:47.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:47.433 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2234 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:47.490 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:47.490 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:47.490 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2388e-05 s, 0.0 kB/s 2023-08-27T11:23:47.491 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:47.491 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:47.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:47.548 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2235 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:47.604 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:47.605 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:47.605 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9449e-05 s, 0.0 kB/s 2023-08-27T11:23:47.605 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:47.606 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:47.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:47.663 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2236 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:47.721 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:47.721 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:47.721 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9499e-05 s, 0.0 kB/s 2023-08-27T11:23:47.722 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:47.722 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:47.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:47.778 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2237 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:47.836 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:47.836 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:47.836 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9842e-05 s, 0.0 kB/s 2023-08-27T11:23:47.837 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:47.837 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:47.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:47.894 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2238 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:47.951 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:47.951 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:47.951 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1878e-05 s, 0.0 kB/s 2023-08-27T11:23:47.952 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:47.952 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:48.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:48.009 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2239 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:48.065 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:48.065 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:48.066 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.922e-05 s, 0.0 kB/s 2023-08-27T11:23:48.067 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:48.067 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:48.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:48.125 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2240 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:48.182 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2541e-05 s, 0.0 kB/s 2023-08-27T11:23:48.183 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:48.183 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:48.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:48.240 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2241 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:48.297 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:48.297 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:48.298 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.124e-05 s, 0.0 kB/s 2023-08-27T11:23:48.298 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:48.298 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:48.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:48.355 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2242 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:48.412 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:48.413 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:48.413 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1129e-05 s, 0.0 kB/s 2023-08-27T11:23:48.413 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:48.414 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:48.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:48.471 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2243 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:48.529 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:48.529 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:48.530 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.121e-05 s, 0.0 kB/s 2023-08-27T11:23:48.530 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:48.531 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:48.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:48.587 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2244 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:48.648 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:48.648 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:48.649 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.253e-05 s, 0.0 kB/s 2023-08-27T11:23:48.649 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:48.650 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:48.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:48.706 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2245 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:48.764 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:48.764 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:48.764 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4122e-05 s, 0.0 kB/s 2023-08-27T11:23:48.765 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:48.765 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:48.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:48.823 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2246 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:48.880 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:48.880 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:48.880 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.14e-05 s, 0.0 kB/s 2023-08-27T11:23:48.881 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:48.881 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:48.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:48.939 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2247 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:48.997 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:48.997 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:48.997 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1771e-05 s, 0.0 kB/s 2023-08-27T11:23:48.998 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:48.998 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:49.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:49.057 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2248 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:49.114 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:49.114 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:49.114 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4717e-05 s, 0.0 kB/s 2023-08-27T11:23:49.115 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:49.116 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:49.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:49.172 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2249 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:49.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:49.231 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:49.231 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1777e-05 s, 0.0 kB/s 2023-08-27T11:23:49.231 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:49.232 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:49.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:49.289 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2250 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:49.346 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:49.346 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:49.346 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1508e-05 s, 0.0 kB/s 2023-08-27T11:23:49.347 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:49.347 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:49.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:49.404 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2251 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:49.461 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:49.461 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:49.461 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.976e-05 s, 0.0 kB/s 2023-08-27T11:23:49.462 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:49.462 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:49.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:49.522 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2252 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:49.578 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:49.579 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:49.579 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2039e-05 s, 0.0 kB/s 2023-08-27T11:23:49.580 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:49.581 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:49.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:49.636 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2253 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:49.693 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:49.693 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:49.693 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9387e-05 s, 0.0 kB/s 2023-08-27T11:23:49.694 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:49.694 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:49.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:49.751 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2254 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:49.808 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:49.809 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:49.809 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9972e-05 s, 0.0 kB/s 2023-08-27T11:23:49.809 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:49.810 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:49.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:49.866 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2255 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:49.923 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:49.923 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:49.924 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1084e-05 s, 0.0 kB/s 2023-08-27T11:23:49.924 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:49.924 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:49.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:49.981 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2256 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:50.037 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:50.038 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:50.038 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6279e-05 s, 0.0 kB/s 2023-08-27T11:23:50.038 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:50.039 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:50.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:50.095 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2257 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:50.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:50.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:50.153 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0079e-05 s, 0.0 kB/s 2023-08-27T11:23:50.154 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:50.154 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:50.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:50.211 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2258 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:50.269 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8738e-05 s, 0.0 kB/s 2023-08-27T11:23:50.269 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:50.270 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:50.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:50.325 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2259 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:50.382 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:50.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:50.383 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2235e-05 s, 0.0 kB/s 2023-08-27T11:23:50.383 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:50.384 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:50.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:50.440 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2260 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:50.497 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:50.498 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:50.498 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2608e-05 s, 0.0 kB/s 2023-08-27T11:23:50.498 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:50.499 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:50.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:50.555 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2261 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:50.613 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:50.613 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:50.614 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9799e-05 s, 0.0 kB/s 2023-08-27T11:23:50.614 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:50.614 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:50.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:50.670 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2262 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:50.728 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:50.728 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:50.728 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3223e-05 s, 0.0 kB/s 2023-08-27T11:23:50.729 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:50.729 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:50.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:50.786 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2263 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:50.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:50.843 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:50.843 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0963e-05 s, 0.0 kB/s 2023-08-27T11:23:50.843 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:50.844 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:50.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:50.901 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2264 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:50.957 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.895e-05 s, 0.0 kB/s 2023-08-27T11:23:50.958 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:50.959 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:51.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:51.016 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2265 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:51.072 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:51.072 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8083e-05 s, 0.0 kB/s 2023-08-27T11:23:51.072 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:51.073 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:51.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:51.130 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2266 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:51.186 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:51.187 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:51.187 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1362e-05 s, 0.0 kB/s 2023-08-27T11:23:51.187 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:51.188 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:51.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:51.244 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2267 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:51.302 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:51.302 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:51.302 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.125e-05 s, 0 B/s 2023-08-27T11:23:51.303 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:51.303 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:51.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:51.359 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2268 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:51.417 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:51.417 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:51.417 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.135e-05 s, 0.0 kB/s 2023-08-27T11:23:51.418 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:51.418 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:51.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:51.474 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2269 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8038e-05 s, 0.0 kB/s 2023-08-27T11:23:51.532 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:51.533 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:51.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:51.590 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2270 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:51.649 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:51.649 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:51.649 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9898e-05 s, 0.0 kB/s 2023-08-27T11:23:51.650 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:51.650 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:51.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:51.707 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2271 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:51.763 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9678e-05 s, 0.0 kB/s 2023-08-27T11:23:51.764 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:51.765 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:51.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:51.822 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2272 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7855e-05 s, 0.0 kB/s 2023-08-27T11:23:51.880 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:51.880 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:51.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:51.937 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2273 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:51.994 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:51.995 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:51.995 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8812e-05 s, 0.0 kB/s 2023-08-27T11:23:51.995 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:51.996 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:52.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:52.052 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2274 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:52.109 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:52.109 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.777e-05 s, 0.0 kB/s 2023-08-27T11:23:52.110 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:52.110 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:52.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:52.167 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2275 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:52.224 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:52.240 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:52.241 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9767e-05 s, 0.0 kB/s 2023-08-27T11:23:52.241 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:52.242 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:52.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:52.283 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2276 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:52.342 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:52.342 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:52.342 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.117e-05 s, 0.0 kB/s 2023-08-27T11:23:52.343 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:52.343 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:52.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:52.400 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2277 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:52.458 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:52.458 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:52.458 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.035e-05 s, 0.0 kB/s 2023-08-27T11:23:52.459 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:52.459 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:52.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:52.517 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2278 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:52.573 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:52.574 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:52.574 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1589e-05 s, 0.0 kB/s 2023-08-27T11:23:52.574 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:52.575 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:52.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:52.631 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2279 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:52.688 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:52.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:52.689 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7206e-05 s, 0.0 kB/s 2023-08-27T11:23:52.689 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:52.690 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:52.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:52.746 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2280 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:52.803 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:52.803 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:52.803 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7542e-05 s, 0.0 kB/s 2023-08-27T11:23:52.804 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:52.804 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:52.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:52.862 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2281 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.083e-05 s, 0.0 kB/s 2023-08-27T11:23:52.922 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:52.922 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:52.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:52.979 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2282 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:53.036 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:53.036 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:53.036 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7068e-05 s, 0.0 kB/s 2023-08-27T11:23:53.037 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:53.037 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:53.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:53.094 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2283 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:53.151 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:53.151 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:53.151 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8177e-05 s, 0.0 kB/s 2023-08-27T11:23:53.152 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:53.152 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:53.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:53.209 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2284 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7713e-05 s, 0.0 kB/s 2023-08-27T11:23:53.269 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:53.269 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:53.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:53.324 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2285 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8509e-05 s, 0.0 kB/s 2023-08-27T11:23:53.383 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:53.384 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:53.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:53.441 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2286 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:53.513 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:53.513 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:53.513 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.8181e-05 s, 0.0 kB/s 2023-08-27T11:23:53.514 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:53.514 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:53.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:53.571 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2287 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:53.627 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:53.628 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:53.628 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1392e-05 s, 0.0 kB/s 2023-08-27T11:23:53.628 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:53.629 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:53.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:53.685 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2288 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:53.742 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:53.743 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:53.743 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0566e-05 s, 0.0 kB/s 2023-08-27T11:23:53.743 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:53.744 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:53.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:53.800 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2289 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1527e-05 s, 0.0 kB/s 2023-08-27T11:23:53.859 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:53.860 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:53.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:53.919 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2290 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:53.976 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:53.976 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:53.976 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9328e-05 s, 0.0 kB/s 2023-08-27T11:23:53.977 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:53.977 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:54.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:54.034 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2291 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:54.090 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:54.091 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:54.091 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9844e-05 s, 0.0 kB/s 2023-08-27T11:23:54.091 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:54.092 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:54.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:54.149 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2292 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:54.205 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:54.206 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:54.206 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2223e-05 s, 0.0 kB/s 2023-08-27T11:23:54.206 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:54.207 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:54.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:54.264 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2293 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:54.320 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:54.321 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:54.321 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1155e-05 s, 0.0 kB/s 2023-08-27T11:23:54.322 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:54.322 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:54.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:54.381 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2294 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:54.438 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:54.439 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:54.439 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1544e-05 s, 0.0 kB/s 2023-08-27T11:23:54.440 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:54.441 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:54.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:54.497 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2295 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:54.555 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:54.555 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:54.555 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0396e-05 s, 0.0 kB/s 2023-08-27T11:23:54.556 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:54.556 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:54.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:54.612 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2296 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:54.670 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:54.670 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:54.671 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2219e-05 s, 0.0 kB/s 2023-08-27T11:23:54.671 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:54.671 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:54.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:54.728 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2297 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:54.785 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:54.786 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:54.786 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0793e-05 s, 0.0 kB/s 2023-08-27T11:23:54.787 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:54.787 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:54.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:54.844 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2298 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.035e-05 s, 0.0 kB/s 2023-08-27T11:23:54.904 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:54.905 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:54.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:54.962 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2299 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8162e-05 s, 0.0 kB/s 2023-08-27T11:23:55.019 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:55.019 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:55.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:55.075 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2300 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:55.132 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:55.133 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:55.133 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1565e-05 s, 0.0 kB/s 2023-08-27T11:23:55.133 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:55.134 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:55.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:55.191 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2301 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:55.248 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:55.248 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:55.248 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8248e-05 s, 0.0 kB/s 2023-08-27T11:23:55.249 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:55.249 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:55.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:55.306 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2302 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:55.362 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:55.363 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:55.363 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2449e-05 s, 0.0 kB/s 2023-08-27T11:23:55.363 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:55.364 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:55.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:55.421 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2303 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:55.479 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:55.480 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:55.480 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0388e-05 s, 0.0 kB/s 2023-08-27T11:23:55.480 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:55.480 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:55.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:55.537 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2304 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:55.594 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0527e-05 s, 0.0 kB/s 2023-08-27T11:23:55.595 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:55.596 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:55.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:55.654 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2305 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:55.711 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:55.711 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:55.711 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0505e-05 s, 0.0 kB/s 2023-08-27T11:23:55.713 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:55.713 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:55.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:55.769 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2306 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:55.826 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:55.826 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:55.826 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1874e-05 s, 0.0 kB/s 2023-08-27T11:23:55.827 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:55.827 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:55.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:55.885 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2307 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:55.942 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:55.943 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:55.943 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1121e-05 s, 0.0 kB/s 2023-08-27T11:23:55.943 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:55.944 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:56.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:56.000 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2308 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:56.057 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:56.058 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:56.058 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8893e-05 s, 0.0 kB/s 2023-08-27T11:23:56.058 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:56.059 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:56.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:56.116 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2309 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:56.172 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:56.172 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:56.173 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1855e-05 s, 0.0 kB/s 2023-08-27T11:23:56.173 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:56.173 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:56.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:56.230 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2310 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:56.288 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:56.288 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:56.288 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3762e-05 s, 0.0 kB/s 2023-08-27T11:23:56.289 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:56.289 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:56.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:56.346 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2311 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:56.404 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.9811e-05 s, 0.0 kB/s 2023-08-27T11:23:56.405 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:56.406 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:56.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:56.463 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2312 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:56.520 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:56.521 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:56.521 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9458e-05 s, 0.0 kB/s 2023-08-27T11:23:56.521 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:56.522 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:56.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:56.579 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2313 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:56.637 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:56.638 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:56.638 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9624e-05 s, 0.0 kB/s 2023-08-27T11:23:56.638 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:56.639 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:56.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:56.695 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2314 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:56.754 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:56.754 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:56.755 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1939e-05 s, 0.0 kB/s 2023-08-27T11:23:56.755 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:56.756 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:56.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:56.812 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2315 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:56.870 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:56.870 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:56.870 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1792e-05 s, 0.0 kB/s 2023-08-27T11:23:56.871 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:56.871 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:56.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:56.927 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2316 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:56.985 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:56.985 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:56.985 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1267e-05 s, 0.0 kB/s 2023-08-27T11:23:56.986 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:56.986 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:57.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:57.043 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2317 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:57.100 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:57.100 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:57.100 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8871e-05 s, 0.0 kB/s 2023-08-27T11:23:57.101 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:57.101 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:57.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:57.158 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2318 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:57.215 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:57.216 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:57.216 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.7109e-05 s, 0.0 kB/s 2023-08-27T11:23:57.216 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:57.217 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:57.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:57.274 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2319 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0219e-05 s, 0.0 kB/s 2023-08-27T11:23:57.332 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:57.332 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:57.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:57.391 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2320 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:57.448 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:57.448 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:57.448 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9719e-05 s, 0.0 kB/s 2023-08-27T11:23:57.449 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:57.449 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:57.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:57.506 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2321 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:57.565 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:57.565 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:57.565 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.122e-05 s, 0.0 kB/s 2023-08-27T11:23:57.566 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:57.566 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:57.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:57.623 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2322 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1761e-05 s, 0.0 kB/s 2023-08-27T11:23:57.681 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:57.681 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:57.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:57.738 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2323 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:57.795 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:57.795 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:57.795 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9233e-05 s, 0.0 kB/s 2023-08-27T11:23:57.796 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:57.796 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:57.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:57.854 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2324 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3935e-05 s, 0.0 kB/s 2023-08-27T11:23:57.913 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:57.913 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:57.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:57.972 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2325 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:58.029 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:58.029 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:58.029 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1207e-05 s, 0.0 kB/s 2023-08-27T11:23:58.030 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:58.030 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:58.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:58.087 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2326 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:58.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:58.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:58.144 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3011e-05 s, 0.0 kB/s 2023-08-27T11:23:58.145 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:58.145 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:58.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:58.202 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2327 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:58.260 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:58.260 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:58.260 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7953e-05 s, 0.0 kB/s 2023-08-27T11:23:58.261 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:58.261 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:58.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:58.318 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2328 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:58.375 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:58.375 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1416e-05 s, 0.0 kB/s 2023-08-27T11:23:58.375 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:58.376 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:58.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:58.433 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2329 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9345e-05 s, 0.0 kB/s 2023-08-27T11:23:58.492 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:58.492 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:58.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:58.549 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2330 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:58.607 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:58.607 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.159e-05 s, 0.0 kB/s 2023-08-27T11:23:58.607 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:58.608 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:58.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:58.664 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2331 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:58.721 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:58.722 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:58.723 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0689e-05 s, 0.0 kB/s 2023-08-27T11:23:58.723 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:58.724 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:58.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:58.780 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2332 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:58.837 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:58.838 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:58.838 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9867e-05 s, 0.0 kB/s 2023-08-27T11:23:58.838 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:58.839 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:58.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:58.895 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2333 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:58.953 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:58.953 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:58.953 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6766e-05 s, 0.0 kB/s 2023-08-27T11:23:58.954 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:58.954 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:59.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:59.013 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2334 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:59.070 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:59.070 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:59.071 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1227e-05 s, 0.0 kB/s 2023-08-27T11:23:59.071 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:59.071 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:59.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:59.127 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2335 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:59.185 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:59.185 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:59.185 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9498e-05 s, 0.0 kB/s 2023-08-27T11:23:59.186 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:59.186 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:59.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:59.246 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2336 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:59.304 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:59.304 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:59.304 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.885e-05 s, 0.0 kB/s 2023-08-27T11:23:59.305 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:59.305 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:59.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:59.362 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2337 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:59.419 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:59.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:59.420 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0498e-05 s, 0.0 kB/s 2023-08-27T11:23:59.420 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:59.421 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:59.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:59.478 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2338 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:59.535 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:59.536 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:59.536 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.73e-05 s, 0.0 kB/s 2023-08-27T11:23:59.536 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:59.537 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:59.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:59.593 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2339 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:59.650 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:59.650 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:59.651 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1142e-05 s, 0.0 kB/s 2023-08-27T11:23:59.652 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:59.652 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:59.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:59.710 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2340 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:59.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:59.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:59.768 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.5387e-05 s, 0.0 kB/s 2023-08-27T11:23:59.769 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:59.769 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:59.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:59.827 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2341 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:59.884 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:23:59.884 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:23:59.884 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6309e-05 s, 0.0 kB/s 2023-08-27T11:23:59.885 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:23:59.885 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:23:59.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:23:59.942 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2342 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:23:59.999 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:00.000 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:00.000 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.897e-05 s, 0.0 kB/s 2023-08-27T11:24:00.000 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:00.001 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:00.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:00.057 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2343 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:00.115 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:00.115 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:00.115 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1965e-05 s, 0.0 kB/s 2023-08-27T11:24:00.116 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:00.116 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:00.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:00.172 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2344 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.054e-05 s, 0.0 kB/s 2023-08-27T11:24:00.230 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:00.231 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:00.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:00.288 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2345 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:00.344 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:00.345 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:00.345 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8141e-05 s, 0.0 kB/s 2023-08-27T11:24:00.345 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:00.346 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:00.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:00.403 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2346 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2542e-05 s, 0.0 kB/s 2023-08-27T11:24:00.462 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:00.462 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:00.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:00.518 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2347 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:00.576 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:00.576 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.92e-05 s, 0.0 kB/s 2023-08-27T11:24:00.576 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:00.577 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:00.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:00.632 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2348 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:00.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:00.690 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:00.690 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.568e-05 s, 0.0 kB/s 2023-08-27T11:24:00.690 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:00.691 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:00.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:00.747 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2349 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:00.804 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:00.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:00.805 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8664e-05 s, 0.0 kB/s 2023-08-27T11:24:00.805 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:00.806 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:00.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:00.863 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2350 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:00.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:00.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:00.921 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3059e-05 s, 0.0 kB/s 2023-08-27T11:24:00.922 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:00.922 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:00.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:00.979 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2351 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:01.037 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:01.037 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:01.038 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8552e-05 s, 0.0 kB/s 2023-08-27T11:24:01.038 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:01.039 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:01.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:01.096 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2352 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:01.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:01.154 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:01.154 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1078e-05 s, 0.0 kB/s 2023-08-27T11:24:01.154 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:01.155 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:01.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:01.211 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2353 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:01.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:01.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:01.269 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.575e-05 s, 0.0 kB/s 2023-08-27T11:24:01.270 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:01.270 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:01.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:01.327 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2354 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:01.387 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:01.387 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:01.387 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1085e-05 s, 0.0 kB/s 2023-08-27T11:24:01.388 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:01.388 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:01.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:01.445 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2355 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:01.503 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:01.503 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:01.503 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9234e-05 s, 0.0 kB/s 2023-08-27T11:24:01.504 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:01.504 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:01.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:01.561 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2356 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:01.619 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:01.619 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:01.619 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2206e-05 s, 0.0 kB/s 2023-08-27T11:24:01.619 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:01.620 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:01.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:01.677 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2357 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:01.735 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:01.735 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:01.735 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9677e-05 s, 0.0 kB/s 2023-08-27T11:24:01.736 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:01.736 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:01.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:01.793 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2358 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:01.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:01.851 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:01.851 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1888e-05 s, 0.0 kB/s 2023-08-27T11:24:01.851 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:01.852 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:01.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:01.910 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2359 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:01.966 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:01.967 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:01.967 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7791e-05 s, 0.0 kB/s 2023-08-27T11:24:01.968 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:01.968 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:02.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:02.025 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2360 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:02.083 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:02.083 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:02.084 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0284e-05 s, 0.0 kB/s 2023-08-27T11:24:02.084 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:02.084 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:02.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:02.141 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2361 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:02.197 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9608e-05 s, 0.0 kB/s 2023-08-27T11:24:02.198 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:02.199 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:02.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:02.256 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2362 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:02.313 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:02.313 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:02.314 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2461e-05 s, 0.0 kB/s 2023-08-27T11:24:02.314 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:02.314 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:02.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:02.371 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2363 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:02.429 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:02.429 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:02.430 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5247e-05 s, 0.0 kB/s 2023-08-27T11:24:02.430 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:02.430 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:02.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:02.488 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2364 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:02.544 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:02.545 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:02.545 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9667e-05 s, 0.0 kB/s 2023-08-27T11:24:02.545 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:02.546 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:02.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:02.603 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2365 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0308e-05 s, 0.0 kB/s 2023-08-27T11:24:02.664 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:02.664 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:02.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:02.719 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2366 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:02.776 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:02.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:02.777 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0465e-05 s, 0.0 kB/s 2023-08-27T11:24:02.777 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:02.778 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:02.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:02.835 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2367 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:02.891 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:02.892 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:02.892 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8976e-05 s, 0.0 kB/s 2023-08-27T11:24:02.892 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:02.893 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:02.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:02.950 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2368 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:03.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:03.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:03.008 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1861e-05 s, 0.0 kB/s 2023-08-27T11:24:03.009 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:03.009 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:03.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:03.066 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2369 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:03.125 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.208e-05 s, 0.0 kB/s 2023-08-27T11:24:03.125 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:03.125 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:03.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:03.182 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2370 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:03.239 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:03.239 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:03.239 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8192e-05 s, 0.0 kB/s 2023-08-27T11:24:03.240 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:03.240 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:03.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:03.297 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2371 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:03.353 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:03.353 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:03.354 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3138e-05 s, 0.0 kB/s 2023-08-27T11:24:03.354 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:03.354 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:03.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:03.411 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2372 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:03.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:03.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:03.469 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3559e-05 s, 0.0 kB/s 2023-08-27T11:24:03.469 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:03.469 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:03.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:03.526 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2373 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:03.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:03.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:03.583 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0058e-05 s, 0.0 kB/s 2023-08-27T11:24:03.584 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:03.584 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:03.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:03.641 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2374 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:03.697 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:03.698 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:03.698 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7273e-05 s, 0.0 kB/s 2023-08-27T11:24:03.698 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:03.699 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:03.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:03.756 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2375 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:03.814 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:03.815 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:03.815 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3316e-05 s, 0.0 kB/s 2023-08-27T11:24:03.815 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:03.816 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:03.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:03.873 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2376 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:03.930 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:03.931 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:03.931 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9351e-05 s, 0.0 kB/s 2023-08-27T11:24:03.931 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:03.932 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:03.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:03.988 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2377 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:04.045 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:04.046 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:04.046 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9431e-05 s, 0.0 kB/s 2023-08-27T11:24:04.046 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:04.047 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:04.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:04.103 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2378 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8513e-05 s, 0.0 kB/s 2023-08-27T11:24:04.162 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:04.162 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:04.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:04.219 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2379 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:04.276 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:04.276 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:04.277 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1898e-05 s, 0.0 kB/s 2023-08-27T11:24:04.277 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:04.277 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:04.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:04.334 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2380 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:04.390 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:04.390 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:04.391 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8024e-05 s, 0.0 kB/s 2023-08-27T11:24:04.391 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:04.391 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:04.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:04.448 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2381 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:04.504 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:04.505 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:04.505 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9393e-05 s, 0.0 kB/s 2023-08-27T11:24:04.505 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:04.506 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:04.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:04.562 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2382 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:04.619 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:04.620 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:04.620 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1382e-05 s, 0.0 kB/s 2023-08-27T11:24:04.620 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:04.621 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:04.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:04.677 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2383 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:04.734 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:04.734 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8427e-05 s, 0.0 kB/s 2023-08-27T11:24:04.735 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:04.735 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:04.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:04.792 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2384 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:04.849 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:04.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:04.850 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9084e-05 s, 0.0 kB/s 2023-08-27T11:24:04.850 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:04.851 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:04.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:04.908 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2385 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:04.964 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:04.965 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:04.965 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8769e-05 s, 0.0 kB/s 2023-08-27T11:24:04.965 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:04.966 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:05.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:05.022 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2386 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:05.078 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:05.079 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:05.079 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3564e-05 s, 0.0 kB/s 2023-08-27T11:24:05.079 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:05.080 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:05.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:05.136 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2387 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9651e-05 s, 0.0 kB/s 2023-08-27T11:24:05.194 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:05.194 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:05.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:05.251 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2388 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:05.307 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:05.308 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:05.308 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.285e-05 s, 0.0 kB/s 2023-08-27T11:24:05.308 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:05.309 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:05.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:05.367 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2389 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:05.424 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:05.425 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:05.425 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4574e-05 s, 0.0 kB/s 2023-08-27T11:24:05.425 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:05.426 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:05.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:05.482 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2390 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:05.539 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:05.539 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:05.540 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.809e-05 s, 0.0 kB/s 2023-08-27T11:24:05.540 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:05.540 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:05.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:05.597 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2391 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:05.654 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:05.654 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:05.654 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.872e-05 s, 0.0 kB/s 2023-08-27T11:24:05.655 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:05.655 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:05.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:05.711 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2392 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:05.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:05.769 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:05.769 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5264e-05 s, 0.0 kB/s 2023-08-27T11:24:05.769 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:05.770 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:05.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:05.828 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2393 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:05.884 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:05.885 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:05.885 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1852e-05 s, 0.0 kB/s 2023-08-27T11:24:05.885 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:05.886 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:05.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:05.943 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2394 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:06.001 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:06.001 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:06.002 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1993e-05 s, 0.0 kB/s 2023-08-27T11:24:06.002 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:06.003 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:06.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:06.059 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2395 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:06.115 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:06.115 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:06.115 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9306e-05 s, 0.0 kB/s 2023-08-27T11:24:06.116 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:06.116 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:06.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:06.173 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2396 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:06.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:06.230 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:06.231 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7762e-05 s, 0.0 kB/s 2023-08-27T11:24:06.231 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:06.231 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:06.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:06.288 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2397 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:06.345 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:06.345 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:06.345 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9792e-05 s, 0.0 kB/s 2023-08-27T11:24:06.346 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:06.346 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:06.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:06.403 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2398 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:06.459 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:06.459 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:06.460 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7618e-05 s, 0.0 kB/s 2023-08-27T11:24:06.460 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:06.460 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:06.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:06.516 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2399 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:06.564 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:24:06.574 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:06.574 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:06.574 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0547e-05 s, 0.0 kB/s 2023-08-27T11:24:06.575 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:06.575 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:06.576 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:24:06.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:06.632 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2400 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:06.690 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:06.690 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1455e-05 s, 0.0 kB/s 2023-08-27T11:24:06.691 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:06.691 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:06.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:06.748 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2401 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:06.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:06.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:06.805 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9213e-05 s, 0.0 kB/s 2023-08-27T11:24:06.806 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:06.806 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:06.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:06.863 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2402 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:06.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:06.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:06.921 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1764e-05 s, 0.0 kB/s 2023-08-27T11:24:06.922 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:06.922 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:06.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:06.979 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2403 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:07.037 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:07.037 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:07.038 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.003e-05 s, 0.0 kB/s 2023-08-27T11:24:07.038 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:07.038 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:07.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:07.095 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2404 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:07.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:07.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:07.154 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2611e-05 s, 0.0 kB/s 2023-08-27T11:24:07.154 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:07.154 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:07.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:07.211 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2405 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:07.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:07.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:07.269 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2848e-05 s, 0.0 kB/s 2023-08-27T11:24:07.269 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:07.270 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:07.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:07.331 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2406 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:07.389 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:07.389 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:07.389 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1433e-05 s, 0.0 kB/s 2023-08-27T11:24:07.390 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:07.390 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:07.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:07.447 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2407 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:07.503 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:07.503 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:07.504 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2859e-05 s, 0.0 kB/s 2023-08-27T11:24:07.504 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:07.504 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:07.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:07.562 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2408 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:07.619 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:07.620 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:07.620 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8332e-05 s, 0.0 kB/s 2023-08-27T11:24:07.620 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:07.621 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:07.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:07.678 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2409 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:07.734 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:07.734 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:07.734 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9642e-05 s, 0.0 kB/s 2023-08-27T11:24:07.735 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:07.735 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:07.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:07.792 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2410 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:07.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:07.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:07.850 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2391e-05 s, 0.0 kB/s 2023-08-27T11:24:07.851 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:07.851 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:07.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:07.908 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2411 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:07.965 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:07.965 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:07.965 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0285e-05 s, 0.0 kB/s 2023-08-27T11:24:07.966 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:07.966 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:08.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:08.023 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2412 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:08.080 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:08.080 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:08.080 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.944e-05 s, 0.0 kB/s 2023-08-27T11:24:08.081 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:08.081 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:08.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:08.138 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2413 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:08.195 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:08.196 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:08.196 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.949e-05 s, 0.0 kB/s 2023-08-27T11:24:08.196 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:08.197 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:08.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:08.254 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2414 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1049e-05 s, 0.0 kB/s 2023-08-27T11:24:08.312 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:08.312 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:08.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:08.370 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2415 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:08.427 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:08.427 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:08.427 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1407e-05 s, 0.0 kB/s 2023-08-27T11:24:08.428 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:08.428 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:08.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:08.484 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2416 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:08.542 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:08.542 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:08.542 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.964e-05 s, 0.0 kB/s 2023-08-27T11:24:08.543 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:08.543 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:08.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:08.600 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2417 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:08.656 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0915e-05 s, 0.0 kB/s 2023-08-27T11:24:08.658 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:08.658 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:08.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:08.715 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2418 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:08.771 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:08.772 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:08.772 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9843e-05 s, 0.0 kB/s 2023-08-27T11:24:08.773 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:08.773 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:08.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:08.830 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2419 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:08.888 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:08.889 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:08.889 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8532e-05 s, 0.0 kB/s 2023-08-27T11:24:08.889 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:08.891 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:08.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:08.947 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2420 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:09.004 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:09.004 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:09.004 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8469e-05 s, 0.0 kB/s 2023-08-27T11:24:09.005 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:09.005 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:09.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:09.062 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2421 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:09.118 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:09.119 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:09.119 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1033e-05 s, 0.0 kB/s 2023-08-27T11:24:09.119 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:09.120 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:09.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:09.177 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2422 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:09.235 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:09.235 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:09.235 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9999e-05 s, 0.0 kB/s 2023-08-27T11:24:09.236 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:09.236 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:09.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:09.293 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2423 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:09.351 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:09.351 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:09.351 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1204e-05 s, 0.0 kB/s 2023-08-27T11:24:09.352 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:09.352 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:09.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:09.409 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2424 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:09.465 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:09.466 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:09.466 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9773e-05 s, 0.0 kB/s 2023-08-27T11:24:09.466 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:09.467 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:09.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:09.525 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2425 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:09.581 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:09.582 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:09.582 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9314e-05 s, 0.0 kB/s 2023-08-27T11:24:09.582 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:09.583 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:09.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:09.639 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2426 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9985e-05 s, 0.0 kB/s 2023-08-27T11:24:09.697 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:09.697 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:09.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:09.754 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2427 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9413e-05 s, 0.0 kB/s 2023-08-27T11:24:09.813 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:09.813 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:09.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:09.871 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2428 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:09.929 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:09.929 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:09.929 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7562e-05 s, 0.0 kB/s 2023-08-27T11:24:09.929 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:09.930 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:09.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:09.987 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2429 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:10.044 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:10.045 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:10.045 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5283e-05 s, 0.0 kB/s 2023-08-27T11:24:10.045 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:10.046 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:10.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:10.105 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2430 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.829e-05 s, 0.0 kB/s 2023-08-27T11:24:10.163 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:10.163 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:10.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:10.220 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2431 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:10.276 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:10.277 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:10.277 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9505e-05 s, 0.0 kB/s 2023-08-27T11:24:10.278 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:10.278 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:10.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:10.335 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2432 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:10.413 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:10.413 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:10.413 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.8752e-05 s, 0.0 kB/s 2023-08-27T11:24:10.414 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:10.414 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:10.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:10.471 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2433 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:10.529 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:10.529 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:10.529 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0294e-05 s, 0.0 kB/s 2023-08-27T11:24:10.530 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:10.530 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:10.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:10.587 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2434 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:10.644 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:10.644 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:10.645 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2449e-05 s, 0.0 kB/s 2023-08-27T11:24:10.645 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:10.646 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:10.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:10.702 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2435 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:10.759 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:10.759 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:10.760 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4601e-05 s, 0.0 kB/s 2023-08-27T11:24:10.760 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:10.760 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:10.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:10.817 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2436 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:10.874 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:10.875 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:10.875 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3725e-05 s, 0.0 kB/s 2023-08-27T11:24:10.875 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:10.876 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:10.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:10.933 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2437 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:10.989 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:10.990 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:10.990 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0357e-05 s, 0.0 kB/s 2023-08-27T11:24:10.990 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:10.991 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:11.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:11.048 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2438 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:11.104 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:11.105 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:11.105 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.955e-05 s, 0.0 kB/s 2023-08-27T11:24:11.105 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:11.106 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:11.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:11.163 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2439 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:11.219 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:11.220 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:11.220 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.978e-05 s, 0.0 kB/s 2023-08-27T11:24:11.220 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:11.221 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:11.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:11.277 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2440 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:11.334 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:11.335 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:11.335 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8725e-05 s, 0.0 kB/s 2023-08-27T11:24:11.335 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:11.336 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:11.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:11.391 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2441 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:11.449 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:11.449 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:11.450 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.111e-05 s, 0.0 kB/s 2023-08-27T11:24:11.450 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:11.450 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:11.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:11.507 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2442 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:11.564 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:11.564 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:11.564 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2084e-05 s, 0.0 kB/s 2023-08-27T11:24:11.565 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:11.565 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:11.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:11.622 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2443 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:11.680 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:11.680 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:11.680 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8307e-05 s, 0.0 kB/s 2023-08-27T11:24:11.681 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:11.682 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:11.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:11.738 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2444 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:11.796 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:11.796 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:11.796 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9975e-05 s, 0.0 kB/s 2023-08-27T11:24:11.797 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:11.797 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:11.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:11.854 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2445 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:11.911 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:11.911 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:11.911 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8584e-05 s, 0.0 kB/s 2023-08-27T11:24:11.912 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:11.912 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:11.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:11.969 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2446 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9324e-05 s, 0.0 kB/s 2023-08-27T11:24:12.027 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:12.028 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:12.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:12.085 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2447 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:12.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:12.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:12.144 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8894e-05 s, 0.0 kB/s 2023-08-27T11:24:12.145 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:12.145 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:12.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:12.201 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2448 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:12.259 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:12.259 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:12.259 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1318e-05 s, 0.0 kB/s 2023-08-27T11:24:12.260 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:12.260 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:12.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:12.317 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2449 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:12.374 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1446e-05 s, 0.0 kB/s 2023-08-27T11:24:12.377 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:12.378 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:12.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:12.436 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2450 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:12.494 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:12.494 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:12.494 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1939e-05 s, 0.0 kB/s 2023-08-27T11:24:12.495 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:12.495 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:12.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:12.553 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2451 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:12.610 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:12.610 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:12.610 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7922e-05 s, 0.0 kB/s 2023-08-27T11:24:12.611 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:12.611 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:12.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:12.667 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2452 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:12.726 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:12.726 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:12.726 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0181e-05 s, 0.0 kB/s 2023-08-27T11:24:12.727 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:12.728 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:12.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:12.789 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2453 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:12.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:12.850 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:12.850 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6969e-05 s, 0.0 kB/s 2023-08-27T11:24:12.851 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:12.851 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:12.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:12.915 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2454 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:12.980 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:12.980 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:12.980 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8372e-05 s, 0.0 kB/s 2023-08-27T11:24:12.981 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:12.981 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:13.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:13.044 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2455 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9151e-05 s, 0.0 kB/s 2023-08-27T11:24:13.103 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:13.103 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:13.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:13.163 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2456 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:13.222 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:13.223 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:13.223 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8491e-05 s, 0.0 kB/s 2023-08-27T11:24:13.223 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:13.224 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:13.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:13.282 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2457 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:13.340 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:13.340 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:13.340 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9771e-05 s, 0.0 kB/s 2023-08-27T11:24:13.341 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:13.342 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:13.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:13.398 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2458 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:13.455 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:13.455 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:13.456 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0206e-05 s, 0.0 kB/s 2023-08-27T11:24:13.456 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:13.456 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:13.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:13.514 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2459 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:13.572 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:13.572 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:13.572 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0774e-05 s, 0.0 kB/s 2023-08-27T11:24:13.573 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:13.573 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:13.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:13.630 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2460 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:13.686 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:13.686 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:13.687 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0641e-05 s, 0.0 kB/s 2023-08-27T11:24:13.687 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:13.687 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:13.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:13.744 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2461 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:13.802 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:13.802 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:13.802 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8151e-05 s, 0.0 kB/s 2023-08-27T11:24:13.803 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:13.803 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:13.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:13.860 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2462 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0808e-05 s, 0.0 kB/s 2023-08-27T11:24:13.917 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:13.918 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:13.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:13.974 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2463 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:14.032 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:14.033 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:14.033 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9017e-05 s, 0.0 kB/s 2023-08-27T11:24:14.033 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:14.034 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:14.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:14.091 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2464 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:14.149 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:14.149 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:14.149 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7721e-05 s, 0.0 kB/s 2023-08-27T11:24:14.150 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:14.150 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:14.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:14.207 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2465 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:14.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:14.266 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:14.266 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0877e-05 s, 0.0 kB/s 2023-08-27T11:24:14.266 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:14.267 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:14.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:14.324 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2466 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:14.381 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:14.382 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:14.382 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4389e-05 s, 0.0 kB/s 2023-08-27T11:24:14.382 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:14.383 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:14.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:14.439 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2467 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:14.496 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:14.496 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:14.496 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0603e-05 s, 0.0 kB/s 2023-08-27T11:24:14.497 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:14.497 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:14.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:14.554 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2468 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:14.612 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:14.612 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:14.613 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2759e-05 s, 0.0 kB/s 2023-08-27T11:24:14.613 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:14.613 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:14.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:14.670 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2469 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:14.727 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:14.728 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:14.728 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.238e-05 s, 0.0 kB/s 2023-08-27T11:24:14.728 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:14.729 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:14.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:14.786 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2470 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:14.844 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:14.844 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:14.844 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8851e-05 s, 0.0 kB/s 2023-08-27T11:24:14.845 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:14.845 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:14.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:14.901 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2471 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:14.960 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:14.960 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:14.960 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0854e-05 s, 0.0 kB/s 2023-08-27T11:24:14.961 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:14.961 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:15.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:15.020 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2472 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:15.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:15.077 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8703e-05 s, 0.0 kB/s 2023-08-27T11:24:15.077 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:15.078 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:15.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:15.135 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2473 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:15.193 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:15.193 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:15.194 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9994e-05 s, 0.0 kB/s 2023-08-27T11:24:15.194 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:15.194 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:15.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:15.258 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2474 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:15.315 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:15.315 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:15.316 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0801e-05 s, 0.0 kB/s 2023-08-27T11:24:15.316 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:15.316 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:15.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:15.373 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2475 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:15.431 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:15.431 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:15.432 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9639e-05 s, 0.0 kB/s 2023-08-27T11:24:15.432 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:15.432 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:15.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:15.489 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2476 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:15.547 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:15.547 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:15.547 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9772e-05 s, 0.0 kB/s 2023-08-27T11:24:15.547 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:15.548 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:15.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:15.605 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2477 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:15.662 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:15.662 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:15.662 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8469e-05 s, 0.0 kB/s 2023-08-27T11:24:15.663 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:15.663 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:15.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:15.720 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2478 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:15.780 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:15.780 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:15.780 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.0198e-05 s, 0.0 kB/s 2023-08-27T11:24:15.781 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:15.781 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:15.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:15.838 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2479 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:15.895 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:15.896 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:15.896 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9283e-05 s, 0.0 kB/s 2023-08-27T11:24:15.896 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:15.897 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:15.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:15.954 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2480 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:16.010 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:16.011 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:16.011 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6102e-05 s, 0.0 kB/s 2023-08-27T11:24:16.011 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:16.012 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:16.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:16.074 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2481 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:16.131 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:16.131 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:16.131 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.073e-05 s, 0.0 kB/s 2023-08-27T11:24:16.132 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:16.132 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:16.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:16.189 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2482 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:16.246 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:16.246 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:16.246 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4732e-05 s, 0.0 kB/s 2023-08-27T11:24:16.247 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:16.247 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:16.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:16.304 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2483 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:16.361 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:16.361 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:16.362 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9002e-05 s, 0.0 kB/s 2023-08-27T11:24:16.362 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:16.362 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:16.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:16.419 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2484 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:16.475 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:16.476 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:16.476 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.138e-05 s, 0.0 kB/s 2023-08-27T11:24:16.476 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:16.477 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:16.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:16.534 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2485 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:16.591 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:16.591 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:16.592 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9582e-05 s, 0.0 kB/s 2023-08-27T11:24:16.592 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:16.592 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:16.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:16.649 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2486 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:16.707 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:16.707 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:16.707 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1541e-05 s, 0.0 kB/s 2023-08-27T11:24:16.708 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:16.708 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:16.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:16.765 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2487 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:16.822 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:16.822 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:16.822 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7464e-05 s, 0.0 kB/s 2023-08-27T11:24:16.823 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:16.823 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:16.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:16.880 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2488 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:16.937 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:16.937 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:16.937 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0849e-05 s, 0.0 kB/s 2023-08-27T11:24:16.937 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:16.938 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:16.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:16.995 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2489 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:17.052 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:17.052 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:17.052 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1768e-05 s, 0.0 kB/s 2023-08-27T11:24:17.053 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:17.053 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:17.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:17.112 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2490 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:17.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:17.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:17.169 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9411e-05 s, 0.0 kB/s 2023-08-27T11:24:17.170 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:17.170 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:17.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:17.227 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2491 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:17.284 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:17.284 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:17.284 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6982e-05 s, 0.0 kB/s 2023-08-27T11:24:17.285 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:17.285 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:17.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:17.342 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2492 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:17.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2729e-05 s, 0.0 kB/s 2023-08-27T11:24:17.403 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:17.403 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:17.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:17.459 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2493 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1888e-05 s, 0.0 kB/s 2023-08-27T11:24:17.518 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:17.519 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:17.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:17.575 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2494 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8234e-05 s, 0.0 kB/s 2023-08-27T11:24:17.632 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:17.633 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:17.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:17.690 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2495 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:17.748 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:17.748 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:17.748 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0888e-05 s, 0.0 kB/s 2023-08-27T11:24:17.749 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:17.749 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:17.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:17.806 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2496 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8131e-05 s, 0.0 kB/s 2023-08-27T11:24:17.863 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:17.864 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:17.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:17.921 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2497 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:17.977 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:17.978 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:17.978 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8321e-05 s, 0.0 kB/s 2023-08-27T11:24:17.979 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:17.979 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:18.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:18.035 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2498 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:18.093 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:18.094 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:18.094 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2917e-05 s, 0.0 kB/s 2023-08-27T11:24:18.094 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:18.095 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:18.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:18.152 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2499 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:18.209 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:18.210 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:18.210 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0483e-05 s, 0.0 kB/s 2023-08-27T11:24:18.210 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:18.211 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:18.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:18.268 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2500 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:18.327 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0757e-05 s, 0.0 kB/s 2023-08-27T11:24:18.327 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:18.327 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:18.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:18.384 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2501 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:18.442 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:18.442 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:18.443 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8609e-05 s, 0.0 kB/s 2023-08-27T11:24:18.443 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:18.443 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:18.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:18.501 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2502 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:18.558 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:18.558 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:18.558 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1049e-05 s, 0.0 kB/s 2023-08-27T11:24:18.559 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:18.559 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:18.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:18.616 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2503 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:18.674 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:18.674 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:18.674 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0138e-05 s, 0.0 kB/s 2023-08-27T11:24:18.675 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:18.675 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:18.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:18.733 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2504 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2809e-05 s, 0.0 kB/s 2023-08-27T11:24:18.792 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:18.792 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:18.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:18.849 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2505 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:18.907 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:18.907 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:18.907 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3058e-05 s, 0.0 kB/s 2023-08-27T11:24:18.908 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:18.908 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:18.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:18.965 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2506 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0605e-05 s, 0.0 kB/s 2023-08-27T11:24:19.025 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:19.026 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:19.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:19.083 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2507 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:19.140 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:19.141 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:19.141 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1705e-05 s, 0.0 kB/s 2023-08-27T11:24:19.141 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:19.142 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:19.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:19.198 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2508 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:19.256 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:19.256 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:19.257 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8601e-05 s, 0.0 kB/s 2023-08-27T11:24:19.257 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:19.257 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:19.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:19.314 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2509 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:19.371 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:19.372 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:19.372 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9634e-05 s, 0.0 kB/s 2023-08-27T11:24:19.372 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:19.373 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:19.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:19.430 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2510 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:19.488 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:19.488 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:19.489 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6908e-05 s, 0.0 kB/s 2023-08-27T11:24:19.489 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:19.489 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:19.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:19.546 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2511 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:19.604 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:19.604 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:19.604 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.977e-05 s, 0.0 kB/s 2023-08-27T11:24:19.605 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:19.605 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:19.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:19.662 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2512 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:19.720 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:19.721 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:19.721 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2619e-05 s, 0.0 kB/s 2023-08-27T11:24:19.721 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:19.722 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:19.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:19.779 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2513 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:19.836 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:19.836 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:19.836 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3544e-05 s, 0.0 kB/s 2023-08-27T11:24:19.837 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:19.837 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:19.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:19.894 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2514 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:19.951 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:19.951 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:19.952 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1473e-05 s, 0.0 kB/s 2023-08-27T11:24:19.952 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:19.952 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:20.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:20.011 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2515 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:20.068 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:20.068 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:20.069 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8437e-05 s, 0.0 kB/s 2023-08-27T11:24:20.069 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:20.070 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:20.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:20.126 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2516 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:20.183 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:20.183 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:20.183 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8873e-05 s, 0.0 kB/s 2023-08-27T11:24:20.183 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:20.184 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:20.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:20.241 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2517 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:20.298 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:20.298 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:20.299 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9277e-05 s, 0.0 kB/s 2023-08-27T11:24:20.299 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:20.299 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:20.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:20.356 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2518 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:20.414 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:20.414 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:20.415 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8462e-05 s, 0.0 kB/s 2023-08-27T11:24:20.415 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:20.416 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:20.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:20.472 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2519 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:20.528 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:20.528 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:20.529 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8701e-05 s, 0.0 kB/s 2023-08-27T11:24:20.530 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:20.530 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:20.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:20.589 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2520 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:20.648 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:20.648 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:20.649 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1388e-05 s, 0.0 kB/s 2023-08-27T11:24:20.649 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:20.649 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:20.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:20.705 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2521 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.7248e-05 s, 0.0 kB/s 2023-08-27T11:24:20.764 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:20.764 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:20.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:20.822 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2522 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:20.880 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:20.880 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:20.880 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8315e-05 s, 0.0 kB/s 2023-08-27T11:24:20.881 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:20.881 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:20.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:20.938 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2523 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:20.997 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:20.997 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:20.998 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9038e-05 s, 0.0 kB/s 2023-08-27T11:24:20.998 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:20.998 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:21.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:21.055 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2524 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:21.111 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:21.111 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:21.111 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9596e-05 s, 0.0 kB/s 2023-08-27T11:24:21.112 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:21.113 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:21.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:21.170 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2525 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7402e-05 s, 0.0 kB/s 2023-08-27T11:24:21.227 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:21.228 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:21.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:21.284 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2526 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:21.342 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:21.342 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:21.343 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2047e-05 s, 0.0 kB/s 2023-08-27T11:24:21.343 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:21.344 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:21.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:21.400 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2527 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:21.458 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:21.458 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:21.458 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.074e-05 s, 0.0 kB/s 2023-08-27T11:24:21.459 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:21.459 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:21.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:21.515 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2528 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:21.573 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:21.574 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:21.574 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1349e-05 s, 0.0 kB/s 2023-08-27T11:24:21.574 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:21.575 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:21.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:21.631 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2529 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:21.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:21.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:21.689 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8715e-05 s, 0.0 kB/s 2023-08-27T11:24:21.690 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:21.690 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:21.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:21.747 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2530 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:21.804 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:21.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:21.805 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.933e-05 s, 0.0 kB/s 2023-08-27T11:24:21.805 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:21.806 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:21.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:21.864 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2531 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:21.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:21.922 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:21.922 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8102e-05 s, 0.0 kB/s 2023-08-27T11:24:21.923 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:21.924 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:21.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:21.980 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2532 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:22.038 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:22.038 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:22.038 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.129e-05 s, 0.0 kB/s 2023-08-27T11:24:22.039 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:22.039 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:22.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:22.095 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2533 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:22.152 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:22.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:22.153 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2526e-05 s, 0.0 kB/s 2023-08-27T11:24:22.153 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:22.154 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:22.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:22.210 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2534 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:22.267 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:22.267 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:22.268 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9705e-05 s, 0.0 kB/s 2023-08-27T11:24:22.268 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:22.269 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:22.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:22.327 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2535 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0684e-05 s, 0.0 kB/s 2023-08-27T11:24:22.384 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:22.385 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:22.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:22.442 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2536 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:22.500 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:22.500 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:22.500 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1384e-05 s, 0.0 kB/s 2023-08-27T11:24:22.501 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:22.501 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:22.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:22.558 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2537 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:22.616 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:22.617 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:22.617 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3249e-05 s, 0.0 kB/s 2023-08-27T11:24:22.620 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:22.621 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:22.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:22.674 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2538 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:22.730 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:22.731 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:22.731 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.12e-05 s, 0.0 kB/s 2023-08-27T11:24:22.731 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:22.732 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:22.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:22.789 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2539 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:22.846 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:22.846 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:22.846 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2622e-05 s, 0.0 kB/s 2023-08-27T11:24:22.847 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:22.847 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:22.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:22.904 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2540 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8499e-05 s, 0.0 kB/s 2023-08-27T11:24:22.963 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:22.963 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:23.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:23.020 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2541 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:23.076 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:23.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:23.077 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.921e-05 s, 0.0 kB/s 2023-08-27T11:24:23.077 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:23.078 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:23.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:23.135 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2542 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:23.192 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:23.195 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:23.196 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1047e-05 s, 0.0 kB/s 2023-08-27T11:24:23.196 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:23.197 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:23.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:23.251 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2543 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:23.308 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:23.309 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:23.310 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.076e-05 s, 0.0 kB/s 2023-08-27T11:24:23.310 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:23.311 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:23.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:23.367 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2544 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:23.424 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:23.424 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:23.425 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4911e-05 s, 0.0 kB/s 2023-08-27T11:24:23.425 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:23.425 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:23.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:23.482 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2545 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9607e-05 s, 0.0 kB/s 2023-08-27T11:24:23.541 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:23.541 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:23.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:23.599 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2546 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:23.658 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8809e-05 s, 0.0 kB/s 2023-08-27T11:24:23.659 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:23.660 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:23.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:23.718 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2547 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:23.774 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:23.775 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:23.775 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2462e-05 s, 0.0 kB/s 2023-08-27T11:24:23.775 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:23.776 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:23.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:23.833 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2548 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:23.890 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:23.890 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:23.890 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2296e-05 s, 0.0 kB/s 2023-08-27T11:24:23.891 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:23.891 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:23.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:23.948 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2549 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:24.004 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:24.005 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:24.005 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9548e-05 s, 0.0 kB/s 2023-08-27T11:24:24.005 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:24.006 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:24.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:24.062 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2550 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:24.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:24.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:24.121 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.8436e-05 s, 0.0 kB/s 2023-08-27T11:24:24.121 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:24.122 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:24.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:24.179 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2551 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:24.236 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:24.236 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:24.236 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8324e-05 s, 0.0 kB/s 2023-08-27T11:24:24.237 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:24.237 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:24.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:24.294 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2552 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:24.351 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:24.351 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:24.352 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0046e-05 s, 0.0 kB/s 2023-08-27T11:24:24.352 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:24.352 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:24.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:24.409 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2553 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:24.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:24.469 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:24.469 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8454e-05 s, 0.0 kB/s 2023-08-27T11:24:24.470 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:24.470 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:24.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:24.526 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2554 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:24.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:24.584 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:24.584 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1284e-05 s, 0.0 kB/s 2023-08-27T11:24:24.584 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:24.585 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:24.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:24.642 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2555 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:24.700 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:24.700 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1566e-05 s, 0.0 kB/s 2023-08-27T11:24:24.701 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:24.701 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:24.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:24.758 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2556 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:24.814 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:24.815 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:24.815 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1965e-05 s, 0.0 kB/s 2023-08-27T11:24:24.815 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:24.816 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:24.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:24.875 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2557 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:24.932 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:24.933 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:24.933 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3314e-05 s, 0.0 kB/s 2023-08-27T11:24:24.933 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:24.934 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:24.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:24.990 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2558 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:25.047 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.318e-05 s, 0.0 kB/s 2023-08-27T11:24:25.048 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:25.049 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:25.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:25.105 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2559 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:25.162 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:25.163 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:25.163 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1332e-05 s, 0.0 kB/s 2023-08-27T11:24:25.163 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:25.164 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:25.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:25.221 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2560 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:25.278 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:25.278 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:25.278 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6544e-05 s, 0.0 kB/s 2023-08-27T11:24:25.280 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:25.281 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:25.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:25.336 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2561 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:25.392 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:25.393 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:25.393 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8668e-05 s, 0.0 kB/s 2023-08-27T11:24:25.393 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:25.394 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:25.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:25.452 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2562 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:25.509 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:25.509 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:25.510 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9424e-05 s, 0.0 kB/s 2023-08-27T11:24:25.510 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:25.510 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:25.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:25.567 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2563 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:25.623 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:25.624 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:25.624 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9682e-05 s, 0.0 kB/s 2023-08-27T11:24:25.624 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:25.625 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:25.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:25.682 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2564 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:25.739 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:25.740 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:25.740 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.789e-05 s, 0.0 kB/s 2023-08-27T11:24:25.740 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:25.741 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:25.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:25.798 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2565 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9129e-05 s, 0.0 kB/s 2023-08-27T11:24:25.856 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:25.856 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:25.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:25.913 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2566 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:25.969 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:25.970 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:25.970 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0182e-05 s, 0.0 kB/s 2023-08-27T11:24:25.971 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:25.971 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:26.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:26.029 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2567 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:26.086 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:26.086 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:26.087 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.866e-05 s, 0.0 kB/s 2023-08-27T11:24:26.087 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:26.087 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:26.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:26.143 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2568 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:26.201 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:26.201 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:26.202 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9162e-05 s, 0.0 kB/s 2023-08-27T11:24:26.202 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:26.202 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:26.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:26.259 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2569 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:26.317 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:26.318 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:26.318 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2578e-05 s, 0.0 kB/s 2023-08-27T11:24:26.318 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:26.319 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:26.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:26.375 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2570 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:26.431 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0272e-05 s, 0.0 kB/s 2023-08-27T11:24:26.433 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:26.433 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:26.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:26.490 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2571 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0164e-05 s, 0.0 kB/s 2023-08-27T11:24:26.550 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:26.550 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:26.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:26.607 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2572 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:26.663 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:26.664 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:26.664 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5377e-05 s, 0.0 kB/s 2023-08-27T11:24:26.664 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:26.665 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:26.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:26.720 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2573 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:26.797 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:26.797 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6807e-05 s, 0.0 kB/s 2023-08-27T11:24:26.797 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:26.798 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:26.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:26.855 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2574 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9281e-05 s, 0.0 kB/s 2023-08-27T11:24:26.913 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:26.913 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:26.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:26.970 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2575 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:27.028 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:27.028 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:27.028 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9964e-05 s, 0.0 kB/s 2023-08-27T11:24:27.029 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:27.029 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:27.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:27.087 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2576 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1356e-05 s, 0.0 kB/s 2023-08-27T11:24:27.145 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:27.145 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:27.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:27.202 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2577 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:27.258 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:27.258 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:27.258 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7558e-05 s, 0.0 kB/s 2023-08-27T11:24:27.259 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:27.260 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:27.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:27.317 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2578 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:27.374 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:27.374 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:27.374 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1843e-05 s, 0.0 kB/s 2023-08-27T11:24:27.375 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:27.375 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:27.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:27.433 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2579 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:27.490 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:27.490 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:27.491 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9155e-05 s, 0.0 kB/s 2023-08-27T11:24:27.491 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:27.491 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:27.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:27.550 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2580 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:27.607 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:27.607 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:27.607 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1813e-05 s, 0.0 kB/s 2023-08-27T11:24:27.608 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:27.608 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:27.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:27.665 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2581 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:27.726 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:27.726 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1149e-05 s, 0.0 kB/s 2023-08-27T11:24:27.726 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:27.727 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:27.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:27.785 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2582 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:27.843 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:27.844 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:27.844 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.103e-05 s, 0.0 kB/s 2023-08-27T11:24:27.844 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:27.845 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:27.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:27.901 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2583 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:27.958 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:27.958 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:27.959 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7818e-05 s, 0.0 kB/s 2023-08-27T11:24:27.959 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:27.960 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:28.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:28.016 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2584 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:28.073 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:28.074 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:28.074 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7685e-05 s, 0.0 kB/s 2023-08-27T11:24:28.075 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:28.075 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:28.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:28.131 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2585 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2952e-05 s, 0.0 kB/s 2023-08-27T11:24:28.190 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:28.190 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:28.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:28.247 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2586 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0874e-05 s, 0.0 kB/s 2023-08-27T11:24:28.306 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:28.306 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:28.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:28.362 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2587 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1106e-05 s, 0.0 kB/s 2023-08-27T11:24:28.421 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:28.421 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:28.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:28.478 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2588 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:28.534 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:28.535 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:28.535 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.9501e-05 s, 0.0 kB/s 2023-08-27T11:24:28.535 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:28.536 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:28.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:28.594 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2589 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:28.652 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:28.652 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:28.652 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1651e-05 s, 0.0 kB/s 2023-08-27T11:24:28.653 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:28.653 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:28.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:28.710 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2590 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2514e-05 s, 0.0 kB/s 2023-08-27T11:24:28.767 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:28.768 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:28.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:28.825 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2591 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:28.883 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:28.883 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1449e-05 s, 0.0 kB/s 2023-08-27T11:24:28.883 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:28.884 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:28.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:28.941 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2592 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:28.999 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:28.999 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:28.999 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0543e-05 s, 0.0 kB/s 2023-08-27T11:24:28.999 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:29.000 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:29.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.057 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2593 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:29.113 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:29.113 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:29.114 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8856e-05 s, 0.0 kB/s 2023-08-27T11:24:29.115 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:29.115 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:29.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.173 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2594 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:29.232 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8263e-05 s, 0.0 kB/s 2023-08-27T11:24:29.232 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:29.232 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:29.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.289 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2595 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9994e-05 s, 0.0 kB/s 2023-08-27T11:24:29.348 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:29.348 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:29.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.404 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2596 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:29.462 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:29.462 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:29.463 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9037e-05 s, 0.0 kB/s 2023-08-27T11:24:29.463 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:29.463 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:29.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.520 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2597 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8825e-05 s, 0.0 kB/s 2023-08-27T11:24:29.578 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:29.578 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:29.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.635 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2598 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:29.691 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:29.692 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:29.692 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0985e-05 s, 0.0 kB/s 2023-08-27T11:24:29.692 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:29.693 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:29.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.750 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2599 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:29.807 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:29.807 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:29.807 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9148e-05 s, 0.0 kB/s 2023-08-27T11:24:29.808 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:29.808 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:29.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.865 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2600 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:29.922 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:29.922 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:29.922 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0283e-05 s, 0.0 kB/s 2023-08-27T11:24:29.923 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:29.923 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:29.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.980 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2601 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:30.038 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:30.039 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:30.039 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2026e-05 s, 0.0 kB/s 2023-08-27T11:24:30.039 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:30.040 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:30.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.095 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2602 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:30.152 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:30.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:30.153 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3162e-05 s, 0.0 kB/s 2023-08-27T11:24:30.153 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:30.154 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:30.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.210 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2603 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:30.266 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:30.267 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:30.267 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8996e-05 s, 0.0 kB/s 2023-08-27T11:24:30.267 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:30.268 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:30.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.324 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2604 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:30.382 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:30.382 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:30.382 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1312e-05 s, 0.0 kB/s 2023-08-27T11:24:30.383 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:30.383 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:30.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.441 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2605 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:30.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:30.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:30.499 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9197e-05 s, 0.0 kB/s 2023-08-27T11:24:30.500 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:30.500 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:30.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.557 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2606 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:30.615 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:30.615 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:30.616 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0299e-05 s, 0.0 kB/s 2023-08-27T11:24:30.616 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:30.616 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:30.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.673 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2607 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:30.731 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:30.731 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:30.732 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0679e-05 s, 0.0 kB/s 2023-08-27T11:24:30.732 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:30.732 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:30.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.789 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2608 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:30.847 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:30.848 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:30.848 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0752e-05 s, 0.0 kB/s 2023-08-27T11:24:30.848 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:30.849 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:30.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.906 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2609 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:30.963 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:30.964 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:30.964 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1186e-05 s, 0.0 kB/s 2023-08-27T11:24:30.964 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:30.965 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:31.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.021 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2610 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:31.077 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:31.078 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:31.078 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8708e-05 s, 0.0 kB/s 2023-08-27T11:24:31.078 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:31.079 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:31.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.136 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2611 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:31.193 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:31.194 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:31.194 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9636e-05 s, 0.0 kB/s 2023-08-27T11:24:31.194 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:31.195 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:31.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.252 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2612 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:31.310 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:31.310 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:31.310 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0467e-05 s, 0.0 kB/s 2023-08-27T11:24:31.311 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:31.311 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:31.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.367 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2613 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:31.425 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:31.425 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:31.425 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1706e-05 s, 0.0 kB/s 2023-08-27T11:24:31.426 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:31.426 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:31.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.485 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2614 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:31.546 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:31.546 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:31.546 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9599e-05 s, 0.0 kB/s 2023-08-27T11:24:31.547 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:31.548 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:31.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.604 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2615 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:31.661 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:31.661 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:31.662 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2029e-05 s, 0.0 kB/s 2023-08-27T11:24:31.662 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:31.662 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:31.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.719 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2616 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:31.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:31.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:31.777 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.929e-05 s, 0.0 kB/s 2023-08-27T11:24:31.778 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:31.778 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:31.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.835 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2617 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:31.893 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0431e-05 s, 0.0 kB/s 2023-08-27T11:24:31.895 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:31.896 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:31.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.953 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2618 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:32.011 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:32.012 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:32.012 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2694e-05 s, 0.0 kB/s 2023-08-27T11:24:32.012 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:32.013 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:32.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.069 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2619 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:32.126 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:32.126 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:32.127 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8883e-05 s, 0.0 kB/s 2023-08-27T11:24:32.127 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:32.127 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:32.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.183 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2620 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:32.241 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:32.241 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:32.241 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2868e-05 s, 0.0 kB/s 2023-08-27T11:24:32.242 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:32.242 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:32.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.299 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2621 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:32.356 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:32.356 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:32.357 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1778e-05 s, 0.0 kB/s 2023-08-27T11:24:32.357 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:32.357 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:32.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.413 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2622 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:32.471 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:32.471 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:32.472 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9258e-05 s, 0.0 kB/s 2023-08-27T11:24:32.472 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:32.472 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:32.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.529 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2623 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:32.586 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:32.586 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:32.586 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.068e-05 s, 0.0 kB/s 2023-08-27T11:24:32.587 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:32.587 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:32.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.644 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2624 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:32.701 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:32.701 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:32.701 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3251e-05 s, 0.0 kB/s 2023-08-27T11:24:32.702 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:32.702 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:32.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.759 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2625 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:32.816 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:32.816 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:32.816 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2759e-05 s, 0.0 kB/s 2023-08-27T11:24:32.817 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:32.817 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:32.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.875 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2626 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:32.932 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:32.932 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:32.932 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2354e-05 s, 0.0 kB/s 2023-08-27T11:24:32.933 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:32.933 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:32.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.990 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2627 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:33.047 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:33.047 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:33.048 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.8714e-05 s, 0.0 kB/s 2023-08-27T11:24:33.048 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:33.048 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:33.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.105 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2628 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0738e-05 s, 0.0 kB/s 2023-08-27T11:24:33.163 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:33.163 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:33.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.220 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2629 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:33.276 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:33.277 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:33.277 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7012e-05 s, 0.0 kB/s 2023-08-27T11:24:33.277 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:33.278 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:33.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.334 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2630 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:33.392 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:33.392 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:33.392 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0885e-05 s, 0.0 kB/s 2023-08-27T11:24:33.393 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:33.393 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:33.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.450 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2631 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:33.507 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:33.508 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:33.508 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1064e-05 s, 0.0 kB/s 2023-08-27T11:24:33.508 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:33.508 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:33.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.565 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2632 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:33.622 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:33.623 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:33.623 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0422e-05 s, 0.0 kB/s 2023-08-27T11:24:33.623 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:33.624 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:33.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.681 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2633 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:33.737 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:33.738 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:33.738 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4518e-05 s, 0.0 kB/s 2023-08-27T11:24:33.738 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:33.738 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:33.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.795 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2634 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:33.852 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:33.852 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:33.852 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.5816e-05 s, 0.0 kB/s 2023-08-27T11:24:33.852 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:33.853 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:33.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.910 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2635 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:33.967 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:33.968 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:33.968 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2748e-05 s, 0.0 kB/s 2023-08-27T11:24:33.968 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:33.969 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:34.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.026 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2636 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:34.083 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:34.083 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:34.083 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.845e-05 s, 0.0 kB/s 2023-08-27T11:24:34.084 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:34.084 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:34.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.141 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2637 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:34.197 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:34.198 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:34.198 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3043e-05 s, 0.0 kB/s 2023-08-27T11:24:34.198 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:34.199 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:34.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.255 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2638 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0418e-05 s, 0.0 kB/s 2023-08-27T11:24:34.314 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:34.314 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:34.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.371 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2639 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:34.429 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:34.429 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:34.430 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.068e-05 s, 0.0 kB/s 2023-08-27T11:24:34.430 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:34.430 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:34.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.486 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2640 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:34.544 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:34.544 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:34.544 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2313e-05 s, 0.0 kB/s 2023-08-27T11:24:34.545 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:34.545 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:34.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.604 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2641 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:34.660 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:34.661 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:34.661 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1414e-05 s, 0.0 kB/s 2023-08-27T11:24:34.661 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:34.662 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:34.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.719 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2642 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:34.775 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:34.776 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:34.776 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2962e-05 s, 0.0 kB/s 2023-08-27T11:24:34.776 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:34.776 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:34.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.833 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2643 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:34.891 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:34.891 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:34.891 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1512e-05 s, 0.0 kB/s 2023-08-27T11:24:34.892 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:34.892 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:34.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.949 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2644 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.93e-05 s, 0.0 kB/s 2023-08-27T11:24:35.008 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:35.009 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:35.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.064 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2645 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:35.121 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:35.121 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:35.122 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6627e-05 s, 0.0 kB/s 2023-08-27T11:24:35.123 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:35.123 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:35.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.181 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2646 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.011e-05 s, 0.0 kB/s 2023-08-27T11:24:35.238 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:35.239 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:35.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.296 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2647 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:35.353 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:35.354 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:35.354 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2353e-05 s, 0.0 kB/s 2023-08-27T11:24:35.355 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:35.355 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:35.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.411 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2648 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:35.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:35.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:35.468 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1455e-05 s, 0.0 kB/s 2023-08-27T11:24:35.469 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:35.469 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:35.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.526 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2649 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:35.585 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:35.586 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:35.587 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9088e-05 s, 0.0 kB/s 2023-08-27T11:24:35.587 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:35.587 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:35.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.643 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2650 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:35.700 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:35.701 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:35.701 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0927e-05 s, 0.0 kB/s 2023-08-27T11:24:35.701 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:35.702 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:35.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.758 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2651 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:35.815 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:35.815 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:35.816 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.149e-05 s, 0.0 kB/s 2023-08-27T11:24:35.816 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:35.816 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:35.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.873 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2652 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:35.930 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0736e-05 s, 0.0 kB/s 2023-08-27T11:24:35.931 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:35.932 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:35.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.990 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2653 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:36.048 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:36.048 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:36.049 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8812e-05 s, 0.0 kB/s 2023-08-27T11:24:36.049 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:36.050 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:36.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.106 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2654 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:36.163 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:36.164 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:36.164 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2101e-05 s, 0.0 kB/s 2023-08-27T11:24:36.164 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:36.165 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:36.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.222 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2655 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:36.280 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:36.280 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:36.280 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2163e-05 s, 0.0 kB/s 2023-08-27T11:24:36.281 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:36.281 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:36.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.338 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2656 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:36.396 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:36.396 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:36.397 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3554e-05 s, 0.0 kB/s 2023-08-27T11:24:36.397 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:36.397 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:36.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.454 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2657 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:36.511 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:36.512 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:36.512 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1087e-05 s, 0.0 kB/s 2023-08-27T11:24:36.512 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:36.513 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:36.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.569 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2658 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1786e-05 s, 0.0 kB/s 2023-08-27T11:24:36.627 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:36.627 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:36.628 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:24:36.669 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:24:36.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.683 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2659 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:36.778 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:36.778 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0145e-05 s, 0.0 kB/s 2023-08-27T11:24:36.778 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:36.779 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:36.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.836 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2660 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:36.895 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:36.895 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:36.895 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.742e-05 s, 0.0 kB/s 2023-08-27T11:24:36.896 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:36.896 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:36.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.953 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2661 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:37.010 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:37.010 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:37.010 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0001e-05 s, 0.0 kB/s 2023-08-27T11:24:37.011 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:37.011 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:37.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.068 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2662 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.126e-05 s, 0.0 kB/s 2023-08-27T11:24:37.127 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:37.127 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:37.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.184 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2663 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:37.241 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:37.241 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:37.241 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0277e-05 s, 0.0 kB/s 2023-08-27T11:24:37.242 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:37.242 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:37.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.299 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2664 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:37.357 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:37.357 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:37.357 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3228e-05 s, 0.0 kB/s 2023-08-27T11:24:37.358 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:37.358 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:37.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.415 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2665 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:37.472 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:37.472 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:37.473 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1655e-05 s, 0.0 kB/s 2023-08-27T11:24:37.473 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:37.474 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:37.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.530 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2666 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:37.588 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:37.588 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:37.589 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1278e-05 s, 0.0 kB/s 2023-08-27T11:24:37.589 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:37.589 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:37.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.646 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2667 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:37.703 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:37.704 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:37.704 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2694e-05 s, 0.0 kB/s 2023-08-27T11:24:37.704 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:37.705 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:37.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.762 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2668 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:37.819 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:37.819 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:37.820 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7316e-05 s, 0.0 kB/s 2023-08-27T11:24:37.820 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:37.821 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:37.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.879 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2669 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:37.936 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:37.936 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:37.936 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.07e-05 s, 0.0 kB/s 2023-08-27T11:24:37.937 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:37.937 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:37.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.994 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2670 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:38.053 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:38.053 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:38.053 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8603e-05 s, 0.0 kB/s 2023-08-27T11:24:38.054 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:38.054 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:38.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.111 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2671 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:38.168 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:38.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:38.169 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4803e-05 s, 0.0 kB/s 2023-08-27T11:24:38.169 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:38.170 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:38.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.226 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2672 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9304e-05 s, 0.0 kB/s 2023-08-27T11:24:38.284 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:38.284 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:38.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.341 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2673 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:38.397 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:38.398 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:38.398 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0161e-05 s, 0.0 kB/s 2023-08-27T11:24:38.399 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:38.399 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:38.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.456 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2674 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:38.513 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:38.514 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:38.514 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9783e-05 s, 0.0 kB/s 2023-08-27T11:24:38.514 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:38.515 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:38.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.572 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2675 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:38.628 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:38.629 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:38.629 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0152e-05 s, 0.0 kB/s 2023-08-27T11:24:38.629 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:38.630 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:38.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.687 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2676 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:38.743 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:38.744 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:38.744 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1488e-05 s, 0.0 kB/s 2023-08-27T11:24:38.744 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:38.745 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:38.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.801 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2677 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:38.858 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:38.858 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:38.860 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.269e-05 s, 0.0 kB/s 2023-08-27T11:24:38.860 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:38.861 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:38.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.916 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2678 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:38.973 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:38.974 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:38.974 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0241e-05 s, 0.0 kB/s 2023-08-27T11:24:38.974 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:38.975 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:39.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.031 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2679 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:39.089 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:39.089 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:39.089 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3024e-05 s, 0.0 kB/s 2023-08-27T11:24:39.089 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:39.090 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:39.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.147 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2680 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1531e-05 s, 0.0 kB/s 2023-08-27T11:24:39.204 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:39.205 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:39.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.262 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2681 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:39.319 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:39.319 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:39.320 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9677e-05 s, 0.0 kB/s 2023-08-27T11:24:39.320 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:39.321 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:39.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.377 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2682 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:39.434 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:39.434 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:39.435 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.994e-05 s, 0.0 kB/s 2023-08-27T11:24:39.435 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:39.435 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:39.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.492 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2683 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:39.549 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:39.549 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:39.550 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0845e-05 s, 0.0 kB/s 2023-08-27T11:24:39.550 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:39.550 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:39.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.607 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2684 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:39.664 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:39.665 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:39.665 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9745e-05 s, 0.0 kB/s 2023-08-27T11:24:39.665 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:39.666 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:39.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.723 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2685 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:39.779 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:39.780 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:39.780 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8696e-05 s, 0.0 kB/s 2023-08-27T11:24:39.780 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:39.781 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:39.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.838 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2686 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0289e-05 s, 0.0 kB/s 2023-08-27T11:24:39.895 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:39.896 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:39.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.953 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2687 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8573e-05 s, 0.0 kB/s 2023-08-27T11:24:40.012 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:40.012 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:40.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.069 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2688 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:40.127 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:40.127 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:40.127 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9104e-05 s, 0.0 kB/s 2023-08-27T11:24:40.128 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:40.128 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:40.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.185 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2689 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:40.242 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:40.242 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:40.243 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8826e-05 s, 0.0 kB/s 2023-08-27T11:24:40.243 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:40.243 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:40.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.301 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2690 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:40.359 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:40.360 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:40.360 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2436e-05 s, 0.0 kB/s 2023-08-27T11:24:40.360 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:40.361 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:40.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.418 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2691 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:40.475 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:40.475 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:40.475 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9095e-05 s, 0.0 kB/s 2023-08-27T11:24:40.476 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:40.476 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:40.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.532 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2692 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:40.590 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:40.590 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:40.590 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0371e-05 s, 0.0 kB/s 2023-08-27T11:24:40.591 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:40.591 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:40.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.648 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2693 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:40.705 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:40.705 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:40.705 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8444e-05 s, 0.0 kB/s 2023-08-27T11:24:40.706 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:40.706 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:40.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.763 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2694 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:40.821 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:40.822 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:40.822 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1063e-05 s, 0.0 kB/s 2023-08-27T11:24:40.822 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:40.823 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:40.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.879 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2695 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:40.937 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:40.938 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:40.938 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0176e-05 s, 0.0 kB/s 2023-08-27T11:24:40.938 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:40.939 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:40.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.995 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2696 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:41.052 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:41.052 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:41.052 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0868e-05 s, 0.0 kB/s 2023-08-27T11:24:41.053 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:41.053 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:41.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.110 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2697 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.274e-05 s, 0.0 kB/s 2023-08-27T11:24:41.168 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:41.168 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:41.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.225 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2698 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:41.283 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:41.283 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:41.284 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2544e-05 s, 0.0 kB/s 2023-08-27T11:24:41.284 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:41.284 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:41.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.341 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2699 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:41.399 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:41.399 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2764e-05 s, 0.0 kB/s 2023-08-27T11:24:41.400 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:41.400 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:41.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.457 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2700 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:41.514 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:41.515 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:41.515 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7906e-05 s, 0.0 kB/s 2023-08-27T11:24:41.515 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:41.516 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:41.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.573 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2701 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:41.630 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:41.630 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:41.631 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7467e-05 s, 0.0 kB/s 2023-08-27T11:24:41.631 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:41.631 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:41.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.688 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2702 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:41.745 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:41.745 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:41.745 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9492e-05 s, 0.0 kB/s 2023-08-27T11:24:41.746 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:41.746 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:41.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.803 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2703 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:41.860 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:41.860 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:41.861 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0203e-05 s, 0.0 kB/s 2023-08-27T11:24:41.861 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:41.862 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:41.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.918 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2704 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:41.975 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:41.975 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:41.976 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2298e-05 s, 0.0 kB/s 2023-08-27T11:24:41.976 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:41.976 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:42.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:42.033 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2705 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:42.091 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:42.091 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:42.091 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0174e-05 s, 0.0 kB/s 2023-08-27T11:24:42.092 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:42.092 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:42.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:42.151 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2706 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:42.207 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:42.208 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:42.208 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9037e-05 s, 0.0 kB/s 2023-08-27T11:24:42.208 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:42.209 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:42.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:42.265 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2707 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:42.323 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:42.324 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:42.324 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0598e-05 s, 0.0 kB/s 2023-08-27T11:24:42.325 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:42.326 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:42.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:42.382 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2708 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:42.440 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:42.440 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:42.440 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9726e-05 s, 0.0 kB/s 2023-08-27T11:24:42.441 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:42.441 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:42.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:42.498 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2709 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:42.556 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:42.557 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:42.557 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0922e-05 s, 0.0 kB/s 2023-08-27T11:24:42.557 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:42.557 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:42.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:42.614 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2710 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2124e-05 s, 0.0 kB/s 2023-08-27T11:24:42.672 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:42.672 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:42.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:42.729 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2711 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:42.786 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:42.787 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:42.787 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1406e-05 s, 0.0 kB/s 2023-08-27T11:24:42.787 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:42.788 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:42.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:42.844 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2712 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:42.900 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:42.900 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:42.900 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9431e-05 s, 0.0 kB/s 2023-08-27T11:24:42.901 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:42.901 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:42.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:42.957 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2713 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:43.014 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:43.015 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:43.015 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0985e-05 s, 0.0 kB/s 2023-08-27T11:24:43.015 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:43.016 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:43.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:43.072 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2714 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9117e-05 s, 0.0 kB/s 2023-08-27T11:24:43.130 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:43.130 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:43.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:43.187 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2715 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:43.244 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:43.245 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:43.245 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7823e-05 s, 0.0 kB/s 2023-08-27T11:24:43.245 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:43.246 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:43.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:43.302 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2716 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:43.360 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:43.360 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:43.361 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.04e-05 s, 0.0 kB/s 2023-08-27T11:24:43.361 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:43.362 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:43.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:43.418 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2717 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:43.476 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:43.476 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:43.477 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9017e-05 s, 0.0 kB/s 2023-08-27T11:24:43.477 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:43.478 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:43.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:43.533 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2718 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:43.591 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:43.591 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:43.591 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8249e-05 s, 0.0 kB/s 2023-08-27T11:24:43.592 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:43.592 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:43.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:43.649 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2719 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:43.706 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:43.707 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:43.707 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.782e-05 s, 0.0 kB/s 2023-08-27T11:24:43.707 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:43.707 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:43.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:43.765 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2720 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0473e-05 s, 0.0 kB/s 2023-08-27T11:24:43.823 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:43.824 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:43.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:43.881 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2721 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:43.938 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:43.938 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:43.938 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0984e-05 s, 0.0 kB/s 2023-08-27T11:24:43.938 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:43.939 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:43.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:43.996 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2722 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:44.054 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:44.054 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:44.054 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1988e-05 s, 0.0 kB/s 2023-08-27T11:24:44.055 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:44.055 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:44.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:44.112 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2723 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:44.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:44.169 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:44.169 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1975e-05 s, 0.0 kB/s 2023-08-27T11:24:44.170 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:44.170 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:44.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:44.228 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2724 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:44.285 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:44.286 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:44.286 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9652e-05 s, 0.0 kB/s 2023-08-27T11:24:44.286 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:44.287 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:44.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:44.344 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2725 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:44.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:44.401 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:44.402 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3399e-05 s, 0.0 kB/s 2023-08-27T11:24:44.402 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:44.402 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:44.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:44.459 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2726 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:44.517 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:44.518 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:44.518 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1318e-05 s, 0.0 kB/s 2023-08-27T11:24:44.518 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:44.519 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:44.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:44.576 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2727 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:44.633 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:44.634 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:44.634 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8582e-05 s, 0.0 kB/s 2023-08-27T11:24:44.634 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:44.635 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:44.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:44.693 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2728 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:44.750 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:44.750 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:44.750 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1069e-05 s, 0.0 kB/s 2023-08-27T11:24:44.751 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:44.751 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:44.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:44.808 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2729 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:44.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:44.865 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:44.865 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0621e-05 s, 0.0 kB/s 2023-08-27T11:24:44.865 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:44.866 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:44.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:44.923 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2730 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:44.980 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9524e-05 s, 0.0 kB/s 2023-08-27T11:24:44.981 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:44.982 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:45.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:45.038 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2731 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:45.095 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:45.095 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:45.095 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.012e-05 s, 0.0 kB/s 2023-08-27T11:24:45.096 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:45.096 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:45.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:45.153 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2732 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:45.211 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:45.211 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:45.212 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2586e-05 s, 0.0 kB/s 2023-08-27T11:24:45.212 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:45.212 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:45.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:45.269 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2733 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2284e-05 s, 0.0 kB/s 2023-08-27T11:24:45.327 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:45.328 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:45.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:45.384 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2734 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:45.441 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:45.442 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0976e-05 s, 0.0 kB/s 2023-08-27T11:24:45.443 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:45.444 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:45.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:45.500 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2735 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:45.559 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:45.559 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1622e-05 s, 0.0 kB/s 2023-08-27T11:24:45.559 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:45.560 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:45.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:45.617 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2736 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:45.675 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:45.675 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:45.675 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8773e-05 s, 0.0 kB/s 2023-08-27T11:24:45.675 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:45.676 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:45.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:45.733 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2737 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:45.790 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:45.791 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:45.791 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.255e-05 s, 0.0 kB/s 2023-08-27T11:24:45.791 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:45.792 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:45.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:45.848 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2738 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:45.906 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:45.906 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:45.907 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9586e-05 s, 0.0 kB/s 2023-08-27T11:24:45.907 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:45.907 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:45.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:45.964 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2739 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.012e-05 s, 0.0 kB/s 2023-08-27T11:24:46.023 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:46.023 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:46.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:46.080 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2740 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:46.136 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:46.137 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:46.137 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.744e-05 s, 0.0 kB/s 2023-08-27T11:24:46.137 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:46.138 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:46.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:46.194 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2741 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:46.252 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:46.253 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:46.253 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2258e-05 s, 0.0 kB/s 2023-08-27T11:24:46.253 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:46.254 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:46.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:46.310 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2742 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:46.368 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:46.368 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:46.368 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.108e-05 s, 0.0 kB/s 2023-08-27T11:24:46.369 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:46.369 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:46.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:46.426 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2743 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:46.483 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:46.483 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:46.484 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8153e-05 s, 0.0 kB/s 2023-08-27T11:24:46.484 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:46.484 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:46.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:46.540 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2744 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:46.598 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:46.599 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:46.599 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0221e-05 s, 0.0 kB/s 2023-08-27T11:24:46.599 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:46.599 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:46.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:46.656 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2745 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:46.713 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:46.713 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0351e-05 s, 0.0 kB/s 2023-08-27T11:24:46.714 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:46.714 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:46.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:46.771 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2746 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:46.828 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:46.828 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:46.828 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7704e-05 s, 0.0 kB/s 2023-08-27T11:24:46.829 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:46.829 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:46.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:46.886 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2747 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:46.943 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:46.944 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:46.944 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8989e-05 s, 0.0 kB/s 2023-08-27T11:24:46.944 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:46.945 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:47.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:47.002 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2748 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:47.058 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:47.059 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:47.059 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8425e-05 s, 0.0 kB/s 2023-08-27T11:24:47.059 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:47.060 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:47.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:47.117 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2749 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:47.174 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:47.174 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:47.174 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7935e-05 s, 0.0 kB/s 2023-08-27T11:24:47.175 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:47.176 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:47.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:47.232 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2750 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.874e-05 s, 0.0 kB/s 2023-08-27T11:24:47.290 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:47.291 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:47.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:47.347 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2751 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:47.406 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8858e-05 s, 0.0 kB/s 2023-08-27T11:24:47.406 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:47.406 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:47.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:47.463 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2752 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0738e-05 s, 0.0 kB/s 2023-08-27T11:24:47.521 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:47.521 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:47.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:47.578 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2753 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:47.635 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:47.636 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:47.636 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2207e-05 s, 0.0 kB/s 2023-08-27T11:24:47.636 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:47.637 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:47.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:47.693 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2754 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:47.751 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:47.751 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:47.751 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3563e-05 s, 0.0 kB/s 2023-08-27T11:24:47.752 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:47.752 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:47.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:47.809 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2755 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:47.865 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.8195e-05 s, 0.0 kB/s 2023-08-27T11:24:47.866 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:47.867 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:47.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:47.924 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2756 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:47.982 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:47.982 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:47.982 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0078e-05 s, 0.0 kB/s 2023-08-27T11:24:47.983 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:47.983 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:48.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:48.040 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2757 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:48.098 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:48.098 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:48.099 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1421e-05 s, 0.0 kB/s 2023-08-27T11:24:48.100 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:48.100 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:48.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:48.156 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2758 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1026e-05 s, 0.0 kB/s 2023-08-27T11:24:48.214 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:48.214 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:48.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:48.271 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2759 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:48.327 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:48.327 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:48.328 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8353e-05 s, 0.0 kB/s 2023-08-27T11:24:48.328 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:48.328 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:48.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:48.385 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2760 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:48.442 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:48.442 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:48.442 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9658e-05 s, 0.0 kB/s 2023-08-27T11:24:48.443 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:48.443 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:48.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:48.500 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2761 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:48.556 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:48.556 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:48.557 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9079e-05 s, 0.0 kB/s 2023-08-27T11:24:48.557 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:48.557 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:48.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:48.614 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2762 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:48.670 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:48.671 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:48.671 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8993e-05 s, 0.0 kB/s 2023-08-27T11:24:48.671 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:48.672 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:48.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:48.729 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2763 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:48.786 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:48.786 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0472e-05 s, 0.0 kB/s 2023-08-27T11:24:48.787 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:48.787 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:48.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:48.844 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2764 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:48.901 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:48.902 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:48.902 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.7906e-05 s, 0.0 kB/s 2023-08-27T11:24:48.902 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:48.903 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:48.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:48.960 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2765 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:49.018 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:49.018 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:49.018 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0814e-05 s, 0.0 kB/s 2023-08-27T11:24:49.018 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:49.019 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:49.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:49.076 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2766 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:49.134 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:49.134 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:49.135 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0249e-05 s, 0.0 kB/s 2023-08-27T11:24:49.135 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:49.135 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:49.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:49.192 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2767 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:49.249 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:49.249 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:49.250 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0506e-05 s, 0.0 kB/s 2023-08-27T11:24:49.250 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:49.250 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:49.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:49.308 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2768 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:49.365 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:49.366 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:49.366 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3165e-05 s, 0.0 kB/s 2023-08-27T11:24:49.367 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:49.367 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:49.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:49.426 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2769 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:49.482 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:49.483 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:49.483 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3367e-05 s, 0.0 kB/s 2023-08-27T11:24:49.484 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:49.484 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:49.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:49.542 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2770 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:49.599 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:49.599 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:49.599 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1947e-05 s, 0.0 kB/s 2023-08-27T11:24:49.600 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:49.600 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:49.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:49.657 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2771 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:49.715 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:49.715 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:49.716 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0117e-05 s, 0.0 kB/s 2023-08-27T11:24:49.716 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:49.716 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:49.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:49.773 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2772 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:49.830 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:49.830 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:49.830 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1237e-05 s, 0.0 kB/s 2023-08-27T11:24:49.831 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:49.831 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:49.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:49.890 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2773 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:49.947 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:49.947 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:49.948 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1333e-05 s, 0.0 kB/s 2023-08-27T11:24:49.948 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:49.948 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:50.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:50.005 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2774 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:50.062 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:50.062 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:50.062 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8623e-05 s, 0.0 kB/s 2023-08-27T11:24:50.063 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:50.063 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:50.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:50.120 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2775 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:50.178 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:50.178 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:50.178 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8644e-05 s, 0.0 kB/s 2023-08-27T11:24:50.179 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:50.179 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:50.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:50.241 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2776 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:50.299 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:50.299 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:50.299 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.203e-05 s, 0.0 kB/s 2023-08-27T11:24:50.300 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:50.300 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:50.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:50.357 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2777 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:50.414 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:50.414 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:50.414 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8728e-05 s, 0.0 kB/s 2023-08-27T11:24:50.415 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:50.415 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:50.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:50.472 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2778 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:50.530 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:50.530 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:50.530 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.192e-05 s, 0.0 kB/s 2023-08-27T11:24:50.531 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:50.531 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:50.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:50.588 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2779 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:50.646 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:50.646 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:50.646 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2175e-05 s, 0.0 kB/s 2023-08-27T11:24:50.647 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:50.647 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:50.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:50.703 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2780 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:50.762 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:50.763 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:50.763 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4247e-05 s, 0.0 kB/s 2023-08-27T11:24:50.764 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:50.764 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:50.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:50.820 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2781 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:50.878 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:50.878 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:50.878 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.0131e-05 s, 0.0 kB/s 2023-08-27T11:24:50.879 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:50.879 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:50.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:50.937 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2782 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0338e-05 s, 0.0 kB/s 2023-08-27T11:24:50.994 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:50.995 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:51.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:51.053 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2783 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:51.110 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:51.110 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:51.110 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9185e-05 s, 0.0 kB/s 2023-08-27T11:24:51.111 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:51.111 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:51.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:51.168 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2784 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:51.225 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:51.226 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:51.226 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2379e-05 s, 0.0 kB/s 2023-08-27T11:24:51.227 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:51.227 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:51.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:51.285 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2785 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:51.343 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:51.344 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:51.344 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0521e-05 s, 0.0 kB/s 2023-08-27T11:24:51.345 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:51.345 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:51.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:51.402 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2786 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:51.459 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:51.459 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:51.459 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0273e-05 s, 0.0 kB/s 2023-08-27T11:24:51.460 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:51.460 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:51.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:51.517 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2787 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:51.573 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1663e-05 s, 0.0 kB/s 2023-08-27T11:24:51.574 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:51.575 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:51.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:51.632 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2788 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:51.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:51.689 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:51.689 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.4864e-05 s, 0.0 kB/s 2023-08-27T11:24:51.690 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:51.690 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:51.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:51.747 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2789 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:51.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:51.805 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:51.806 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9864e-05 s, 0.0 kB/s 2023-08-27T11:24:51.806 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:51.806 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:51.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:51.863 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2790 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:51.920 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1173e-05 s, 0.0 kB/s 2023-08-27T11:24:51.922 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:51.922 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:51.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:51.978 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2791 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:52.035 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:52.035 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:52.036 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2159e-05 s, 0.0 kB/s 2023-08-27T11:24:52.036 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:52.036 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:52.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:52.093 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2792 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:52.150 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:52.151 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:52.151 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0802e-05 s, 0.0 kB/s 2023-08-27T11:24:52.152 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:52.152 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:52.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:52.208 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2793 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:52.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:52.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:52.265 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 4.6578e-05 s, 0.0 kB/s 2023-08-27T11:24:52.266 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:52.267 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:52.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:52.325 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2794 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:52.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:52.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:52.384 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4637e-05 s, 0.0 kB/s 2023-08-27T11:24:52.384 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:52.384 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:52.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:52.441 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2795 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:52.498 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:52.499 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:52.499 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9342e-05 s, 0.0 kB/s 2023-08-27T11:24:52.499 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:52.500 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:52.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:52.556 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2796 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:52.613 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:52.613 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:52.614 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2379e-05 s, 0.0 kB/s 2023-08-27T11:24:52.614 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:52.614 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:52.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:52.671 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2797 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:52.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:52.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:52.730 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9716e-05 s, 0.0 kB/s 2023-08-27T11:24:52.730 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:52.730 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:52.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:52.788 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2798 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:52.845 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:52.846 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:52.846 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6438e-05 s, 0.0 kB/s 2023-08-27T11:24:52.846 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:52.847 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:52.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:52.904 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2799 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:52.960 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:52.960 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:52.961 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9992e-05 s, 0.0 kB/s 2023-08-27T11:24:52.961 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:52.962 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:53.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:53.018 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2800 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:53.075 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:53.075 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:53.075 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3928e-05 s, 0.0 kB/s 2023-08-27T11:24:53.076 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:53.076 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:53.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:53.133 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2801 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:53.191 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:53.191 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8875e-05 s, 0.0 kB/s 2023-08-27T11:24:53.192 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:53.192 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:53.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:53.249 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2802 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:53.306 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:53.307 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:53.307 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9676e-05 s, 0.0 kB/s 2023-08-27T11:24:53.308 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:53.308 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:53.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:53.364 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2803 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:53.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:53.421 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:53.421 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2516e-05 s, 0.0 kB/s 2023-08-27T11:24:53.421 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:53.422 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:53.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:53.478 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2804 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:53.537 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:53.537 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:53.537 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5273e-05 s, 0.0 kB/s 2023-08-27T11:24:53.538 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:53.538 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:53.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:53.595 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2805 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:53.654 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:53.654 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1928e-05 s, 0.0 kB/s 2023-08-27T11:24:53.654 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:53.655 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:53.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:53.711 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2806 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:53.769 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1149e-05 s, 0.0 kB/s 2023-08-27T11:24:53.769 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:53.769 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:53.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:53.827 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2807 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:53.885 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:53.885 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:53.885 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9589e-05 s, 0.0 kB/s 2023-08-27T11:24:53.886 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:53.886 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:53.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:53.943 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2808 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:54.001 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:54.001 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:54.002 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0959e-05 s, 0.0 kB/s 2023-08-27T11:24:54.002 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:54.002 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:54.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:54.059 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2809 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:54.116 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:54.117 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:54.117 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0168e-05 s, 0.0 kB/s 2023-08-27T11:24:54.118 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:54.118 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:54.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:54.176 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2810 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:54.234 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:54.234 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9721e-05 s, 0.0 kB/s 2023-08-27T11:24:54.236 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:54.237 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:54.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:54.294 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2811 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:54.352 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:54.352 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:54.353 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4644e-05 s, 0.0 kB/s 2023-08-27T11:24:54.353 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:54.354 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:54.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:54.410 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2812 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:54.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:54.468 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:54.469 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9195e-05 s, 0.0 kB/s 2023-08-27T11:24:54.469 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:54.470 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:54.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:54.526 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2813 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:54.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:54.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:54.583 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.279e-05 s, 0.0 kB/s 2023-08-27T11:24:54.584 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:54.584 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:54.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:54.641 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2814 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:54.697 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:54.698 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:54.698 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2538e-05 s, 0.0 kB/s 2023-08-27T11:24:54.699 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:54.700 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:54.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:54.757 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2815 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8927e-05 s, 0.0 kB/s 2023-08-27T11:24:54.816 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:54.817 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:54.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:54.873 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2816 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:54.930 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:54.930 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:54.931 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6683e-05 s, 0.0 kB/s 2023-08-27T11:24:54.931 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:54.932 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:54.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:54.988 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2817 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:55.046 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:55.046 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:55.046 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.799e-05 s, 0.0 kB/s 2023-08-27T11:24:55.047 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:55.047 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:55.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:55.104 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2818 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:55.161 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:55.161 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:55.161 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7913e-05 s, 0.0 kB/s 2023-08-27T11:24:55.162 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:55.162 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:55.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:55.219 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2819 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:55.275 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:55.276 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:55.276 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9886e-05 s, 0.0 kB/s 2023-08-27T11:24:55.276 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:55.276 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:55.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:55.333 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2820 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:55.391 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:55.391 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:55.391 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6584e-05 s, 0.0 kB/s 2023-08-27T11:24:55.391 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:55.392 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:55.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:55.449 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2821 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7169e-05 s, 0.0 kB/s 2023-08-27T11:24:55.507 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:55.507 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:55.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:55.564 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2822 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:55.622 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1715e-05 s, 0.0 kB/s 2023-08-27T11:24:55.622 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:55.622 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:55.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:55.679 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2823 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:55.735 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:55.736 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:55.736 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0688e-05 s, 0.0 kB/s 2023-08-27T11:24:55.736 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:55.737 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:55.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:55.794 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2824 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:55.851 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:55.851 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:55.852 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9108e-05 s, 0.0 kB/s 2023-08-27T11:24:55.852 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:55.852 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:55.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:55.910 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2825 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:55.968 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:55.968 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:55.968 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3487e-05 s, 0.0 kB/s 2023-08-27T11:24:55.969 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:55.969 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:56.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:56.026 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2826 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:56.084 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:56.085 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:56.085 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.408e-05 s, 0.0 kB/s 2023-08-27T11:24:56.085 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:56.086 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:56.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:56.142 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2827 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:56.199 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:56.200 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:56.200 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0214e-05 s, 0.0 kB/s 2023-08-27T11:24:56.200 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:56.201 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:56.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:56.259 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2828 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:56.317 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:56.317 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:56.318 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3203e-05 s, 0.0 kB/s 2023-08-27T11:24:56.318 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:56.318 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:56.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:56.375 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2829 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:56.433 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:56.433 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:56.433 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9511e-05 s, 0.0 kB/s 2023-08-27T11:24:56.434 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:56.434 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:56.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:56.491 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2830 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:56.555 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:56.555 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:56.556 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0756e-05 s, 0.0 kB/s 2023-08-27T11:24:56.556 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:56.556 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:56.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:56.613 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2831 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4791e-05 s, 0.0 kB/s 2023-08-27T11:24:56.671 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:56.672 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:56.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:56.729 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2832 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:56.786 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:56.786 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:56.786 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0669e-05 s, 0.0 kB/s 2023-08-27T11:24:56.786 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:56.787 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:56.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:56.844 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2833 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2153e-05 s, 0.0 kB/s 2023-08-27T11:24:56.902 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:56.902 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:56.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:56.959 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2834 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:57.017 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:57.017 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:57.018 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7278e-05 s, 0.0 kB/s 2023-08-27T11:24:57.018 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:57.018 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:57.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:57.075 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2835 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:57.132 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:57.132 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:57.132 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7436e-05 s, 0.0 kB/s 2023-08-27T11:24:57.133 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:57.133 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:57.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:57.189 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2836 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.83e-05 s, 0.0 kB/s 2023-08-27T11:24:57.248 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:57.248 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:57.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:57.305 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2837 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:57.362 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:57.362 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:57.362 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2223e-05 s, 0.0 kB/s 2023-08-27T11:24:57.363 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:57.363 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:57.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:57.420 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2838 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:57.478 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.331e-05 s, 0.0 kB/s 2023-08-27T11:24:57.478 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:57.478 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:57.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:57.535 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2839 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:57.593 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:57.593 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:57.593 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.024e-05 s, 0.0 kB/s 2023-08-27T11:24:57.594 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:57.594 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:57.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:57.651 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2840 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7755e-05 s, 0.0 kB/s 2023-08-27T11:24:57.709 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:57.709 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:57.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:57.766 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2841 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:57.825 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:57.826 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:57.826 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9167e-05 s, 0.0 kB/s 2023-08-27T11:24:57.827 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:57.827 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:57.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:57.883 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2842 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:57.941 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:57.941 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:57.942 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9631e-05 s, 0.0 kB/s 2023-08-27T11:24:57.942 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:57.942 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:57.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:57.999 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2843 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:58.056 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:58.056 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:58.057 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3737e-05 s, 0.0 kB/s 2023-08-27T11:24:58.057 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:58.057 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:58.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:58.114 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2844 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:58.170 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.205e-05 s, 0.0 kB/s 2023-08-27T11:24:58.171 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:58.172 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:58.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:58.229 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2845 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8655e-05 s, 0.0 kB/s 2023-08-27T11:24:58.286 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:58.287 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:58.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:58.342 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2846 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7919e-05 s, 0.0 kB/s 2023-08-27T11:24:58.401 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:58.401 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:58.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:58.458 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2847 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:58.516 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:58.516 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:58.516 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1679e-05 s, 0.0 kB/s 2023-08-27T11:24:58.517 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:58.517 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:58.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:58.574 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2848 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:58.632 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:58.632 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:58.632 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0927e-05 s, 0.0 kB/s 2023-08-27T11:24:58.633 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:58.633 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:58.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:58.690 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2849 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:58.747 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:58.747 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:58.748 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7388e-05 s, 0.0 kB/s 2023-08-27T11:24:58.748 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:58.748 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:58.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:58.806 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2850 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:58.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:58.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:58.864 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9283e-05 s, 0.0 kB/s 2023-08-27T11:24:58.865 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:58.865 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:58.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:58.922 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2851 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9456e-05 s, 0.0 kB/s 2023-08-27T11:24:58.981 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:58.981 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:59.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:59.038 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2852 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:59.101 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:59.101 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:59.102 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.233e-05 s, 0.0 kB/s 2023-08-27T11:24:59.102 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:59.102 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:59.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:59.159 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2853 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:59.216 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:59.216 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:59.218 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1045e-05 s, 0.0 kB/s 2023-08-27T11:24:59.218 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:59.219 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:59.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:59.275 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2854 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:59.332 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:59.332 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:59.333 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8792e-05 s, 0.0 kB/s 2023-08-27T11:24:59.333 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:59.333 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:59.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:59.390 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2855 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:59.448 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:59.449 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:59.449 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1624e-05 s, 0.0 kB/s 2023-08-27T11:24:59.449 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:59.450 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:59.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:59.507 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2856 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:59.565 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:59.565 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:59.566 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.958e-05 s, 0.0 kB/s 2023-08-27T11:24:59.566 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:59.566 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:59.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:59.623 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2857 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:59.681 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:59.681 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:59.681 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.011e-05 s, 0.0 kB/s 2023-08-27T11:24:59.681 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:59.682 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:59.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:59.740 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2858 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:59.797 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:59.797 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:59.798 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2989e-05 s, 0.0 kB/s 2023-08-27T11:24:59.798 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:59.798 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:59.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:59.856 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2859 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:24:59.913 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:24:59.913 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:24:59.914 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.117e-05 s, 0.0 kB/s 2023-08-27T11:24:59.914 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:24:59.915 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:59.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:59.971 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2860 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:00.028 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:00.028 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:00.028 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8915e-05 s, 0.0 kB/s 2023-08-27T11:25:00.028 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:00.029 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:00.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:00.086 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2861 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:00.142 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.899e-05 s, 0.0 kB/s 2023-08-27T11:25:00.144 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:00.144 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:00.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:00.202 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2862 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:00.258 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:00.259 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:00.259 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9552e-05 s, 0.0 kB/s 2023-08-27T11:25:00.259 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:00.260 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:00.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:00.317 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2863 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:00.377 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:00.377 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:00.378 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2075e-05 s, 0.0 kB/s 2023-08-27T11:25:00.378 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:00.378 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:00.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:00.436 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2864 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:00.493 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:00.493 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:00.494 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8921e-05 s, 0.0 kB/s 2023-08-27T11:25:00.494 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:00.494 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:00.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:00.552 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2865 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:00.610 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:00.610 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:00.611 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2697e-05 s, 0.0 kB/s 2023-08-27T11:25:00.611 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:00.611 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:00.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:00.668 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2866 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:00.725 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:00.726 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:00.726 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.954e-05 s, 0.0 kB/s 2023-08-27T11:25:00.726 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:00.727 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:00.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:00.785 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2867 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:00.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:00.842 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:00.843 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2145e-05 s, 0.0 kB/s 2023-08-27T11:25:00.843 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:00.843 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:00.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:00.901 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2868 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:00.959 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0959e-05 s, 0.0 kB/s 2023-08-27T11:25:00.961 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:00.961 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:01.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:01.017 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2869 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3516e-05 s, 0.0 kB/s 2023-08-27T11:25:01.074 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:01.074 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:01.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:01.130 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2870 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:01.188 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:01.188 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:01.189 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1817e-05 s, 0.0 kB/s 2023-08-27T11:25:01.189 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:01.189 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:01.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:01.248 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2871 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:01.305 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:01.305 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:01.305 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0902e-05 s, 0.0 kB/s 2023-08-27T11:25:01.306 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:01.306 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:01.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:01.363 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2872 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:01.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:01.420 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:01.420 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2168e-05 s, 0.0 kB/s 2023-08-27T11:25:01.420 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:01.421 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:01.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:01.478 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2873 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:01.535 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:01.536 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:01.536 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7964e-05 s, 0.0 kB/s 2023-08-27T11:25:01.536 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:01.537 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:01.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:01.593 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2874 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:01.650 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:01.651 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:01.651 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1286e-05 s, 0.0 kB/s 2023-08-27T11:25:01.651 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:01.652 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:01.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:01.709 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2875 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:01.766 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:01.767 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:01.767 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.228e-05 s, 0.0 kB/s 2023-08-27T11:25:01.767 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:01.768 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:01.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:01.825 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2876 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:01.882 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:01.883 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:01.883 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2318e-05 s, 0.0 kB/s 2023-08-27T11:25:01.883 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:01.884 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:01.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:01.941 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2877 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.6498e-05 s, 0.0 kB/s 2023-08-27T11:25:01.998 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:01.999 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:02.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:02.056 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2878 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:02.113 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:02.114 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:02.114 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.777e-05 s, 0.0 kB/s 2023-08-27T11:25:02.114 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:02.115 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:02.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:02.172 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2879 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:02.230 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9728e-05 s, 0.0 kB/s 2023-08-27T11:25:02.230 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:02.230 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:02.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:02.286 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2880 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:02.343 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:02.343 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:02.343 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0369e-05 s, 0.0 kB/s 2023-08-27T11:25:02.344 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:02.345 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:02.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:02.402 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2881 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:02.459 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:02.460 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:02.460 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.97e-05 s, 0.0 kB/s 2023-08-27T11:25:02.460 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:02.461 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:02.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:02.518 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2882 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8549e-05 s, 0.0 kB/s 2023-08-27T11:25:02.576 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:02.577 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:02.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:02.634 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2883 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:02.693 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:02.693 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:02.693 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0071e-05 s, 0.0 kB/s 2023-08-27T11:25:02.694 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:02.694 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:02.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:02.751 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2884 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:02.808 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:02.808 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:02.809 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2552e-05 s, 0.0 kB/s 2023-08-27T11:25:02.809 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:02.809 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:02.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:02.866 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2885 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:02.923 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:02.923 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:02.923 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9428e-05 s, 0.0 kB/s 2023-08-27T11:25:02.924 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:02.924 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:02.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:02.981 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2886 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:03.038 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:03.038 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:03.039 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4209e-05 s, 0.0 kB/s 2023-08-27T11:25:03.039 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:03.039 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:03.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:03.096 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2887 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:03.152 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:03.153 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:03.153 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8517e-05 s, 0.0 kB/s 2023-08-27T11:25:03.153 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:03.154 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:03.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:03.211 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2888 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:03.269 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:03.270 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:03.270 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9001e-05 s, 0.0 kB/s 2023-08-27T11:25:03.270 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:03.271 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:03.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:03.327 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2889 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9021e-05 s, 0.0 kB/s 2023-08-27T11:25:03.384 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:03.385 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:03.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:03.441 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2890 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:03.498 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:03.498 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:03.499 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9931e-05 s, 0.0 kB/s 2023-08-27T11:25:03.499 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:03.499 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:03.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:03.557 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2891 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:03.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:03.614 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:03.614 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8964e-05 s, 0.0 kB/s 2023-08-27T11:25:03.615 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:03.615 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:03.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:03.671 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2892 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1868e-05 s, 0.0 kB/s 2023-08-27T11:25:03.729 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:03.730 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:03.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:03.787 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2893 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0094e-05 s, 0.0 kB/s 2023-08-27T11:25:03.847 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:03.848 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:03.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:03.904 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2894 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:03.961 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:03.963 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:03.963 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9619e-05 s, 0.0 kB/s 2023-08-27T11:25:03.964 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:03.964 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:04.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:04.019 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2895 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:04.075 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:04.076 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:04.076 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0136e-05 s, 0.0 kB/s 2023-08-27T11:25:04.076 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:04.077 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:04.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:04.134 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2896 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:04.195 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:04.195 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:04.196 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9338e-05 s, 0.0 kB/s 2023-08-27T11:25:04.196 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:04.196 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:04.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:04.253 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2897 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:04.311 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:04.311 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:04.311 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2474e-05 s, 0.0 kB/s 2023-08-27T11:25:04.312 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:04.312 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:04.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:04.370 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2898 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:04.427 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:04.428 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:04.428 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5454e-05 s, 0.0 kB/s 2023-08-27T11:25:04.428 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:04.429 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:04.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:04.485 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2899 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:04.541 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:04.542 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:04.542 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.7225e-05 s, 0.0 kB/s 2023-08-27T11:25:04.542 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:04.543 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:04.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:04.599 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2900 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:04.656 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0643e-05 s, 0.0 kB/s 2023-08-27T11:25:04.657 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:04.658 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:04.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:04.715 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2901 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:04.772 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:04.773 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:04.773 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1441e-05 s, 0.0 kB/s 2023-08-27T11:25:04.773 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:04.774 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:04.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:04.830 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2902 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0225e-05 s, 0.0 kB/s 2023-08-27T11:25:04.888 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:04.888 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:04.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:04.944 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2903 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9597e-05 s, 0.0 kB/s 2023-08-27T11:25:05.003 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:05.003 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:05.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:05.061 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2904 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:05.119 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:05.119 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:05.120 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.6911e-05 s, 0.0 kB/s 2023-08-27T11:25:05.120 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:05.120 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:05.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:05.177 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2905 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:05.234 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:05.234 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:05.234 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1996e-05 s, 0.0 kB/s 2023-08-27T11:25:05.235 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:05.235 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:05.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:05.292 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2906 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:05.350 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:05.350 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:05.351 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0599e-05 s, 0.0 kB/s 2023-08-27T11:25:05.351 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:05.351 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:05.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:05.409 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2907 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:05.467 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:05.467 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:05.468 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2333e-05 s, 0.0 kB/s 2023-08-27T11:25:05.468 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:05.468 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:05.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:05.525 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2908 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:05.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:05.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:05.584 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8863e-05 s, 0.0 kB/s 2023-08-27T11:25:05.584 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:05.584 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:05.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:05.644 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2909 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:05.699 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:05.700 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:05.700 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.251e-05 s, 0.0 kB/s 2023-08-27T11:25:05.700 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:05.701 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:05.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:05.758 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2910 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:05.815 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:05.815 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2394e-05 s, 0.0 kB/s 2023-08-27T11:25:05.816 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:05.816 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:05.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:05.872 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2911 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:05.929 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:05.930 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:05.930 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8804e-05 s, 0.0 kB/s 2023-08-27T11:25:05.930 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:05.931 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:05.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:05.989 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2912 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:06.046 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:06.046 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:06.046 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0423e-05 s, 0.0 kB/s 2023-08-27T11:25:06.047 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:06.047 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:06.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:06.104 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2913 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:06.161 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:06.161 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:06.161 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2947e-05 s, 0.0 kB/s 2023-08-27T11:25:06.162 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:06.162 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:06.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:06.219 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2914 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3754e-05 s, 0.0 kB/s 2023-08-27T11:25:06.277 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:06.277 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:06.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:06.334 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2915 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:06.392 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:06.392 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:06.393 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0761e-05 s, 0.0 kB/s 2023-08-27T11:25:06.393 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:06.393 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:06.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:06.450 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2916 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:06.507 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:06.507 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:06.508 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.079e-05 s, 0.0 kB/s 2023-08-27T11:25:06.508 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:06.508 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:06.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:06.565 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2917 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:06.622 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:06.622 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:06.622 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1185e-05 s, 0.0 kB/s 2023-08-27T11:25:06.623 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:06.623 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:06.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:06.680 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2918 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:06.738 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:06.738 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:06.739 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0862e-05 s, 0.0 kB/s 2023-08-27T11:25:06.739 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:06.739 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:06.758 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:25:06.781 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:25:06.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:06.795 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2919 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:06.886 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:06.886 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:06.887 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9938e-05 s, 0.0 kB/s 2023-08-27T11:25:06.887 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:06.887 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:06.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:06.944 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2920 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7856e-05 s, 0.0 kB/s 2023-08-27T11:25:07.002 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:07.002 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:07.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:07.060 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2921 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:07.117 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:07.117 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:07.117 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0319e-05 s, 0.0 kB/s 2023-08-27T11:25:07.118 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:07.119 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:07.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:07.175 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2922 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:07.232 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:07.233 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:07.233 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8563e-05 s, 0.0 kB/s 2023-08-27T11:25:07.234 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:07.235 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:07.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:07.292 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2923 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:07.349 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:07.350 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:07.350 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2179e-05 s, 0.0 kB/s 2023-08-27T11:25:07.350 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:07.351 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:07.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:07.408 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2924 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:07.465 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:07.465 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:07.466 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0125e-05 s, 0.0 kB/s 2023-08-27T11:25:07.467 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:07.467 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:07.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:07.525 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2925 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:07.583 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:07.584 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:07.584 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0663e-05 s, 0.0 kB/s 2023-08-27T11:25:07.584 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:07.585 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:07.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:07.640 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2926 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:07.698 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:07.698 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:07.699 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8399e-05 s, 0.0 kB/s 2023-08-27T11:25:07.699 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:07.699 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:07.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:07.757 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2927 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:07.815 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:07.815 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0972e-05 s, 0.0 kB/s 2023-08-27T11:25:07.816 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:07.816 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:07.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:07.873 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2928 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:07.930 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:07.931 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:07.931 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.121e-05 s, 0.0 kB/s 2023-08-27T11:25:07.931 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:07.932 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:07.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:07.989 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2929 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:08.047 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:08.047 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:08.047 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8564e-05 s, 0.0 kB/s 2023-08-27T11:25:08.047 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:08.048 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:08.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:08.105 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2930 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:08.164 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0043e-05 s, 0.0 kB/s 2023-08-27T11:25:08.165 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:08.166 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:08.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:08.223 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2931 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:08.281 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:08.282 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:08.282 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1861e-05 s, 0.0 kB/s 2023-08-27T11:25:08.282 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:08.283 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:08.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:08.340 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2932 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:08.396 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:08.397 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:08.397 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0648e-05 s, 0.0 kB/s 2023-08-27T11:25:08.397 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:08.398 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:08.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:08.455 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2933 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:08.513 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:08.513 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:08.514 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9245e-05 s, 0.0 kB/s 2023-08-27T11:25:08.514 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:08.515 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:08.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:08.575 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2934 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:08.632 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:08.632 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:08.632 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9769e-05 s, 0.0 kB/s 2023-08-27T11:25:08.633 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:08.633 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:08.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:08.690 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2935 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:08.748 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:08.749 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:08.749 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.898e-05 s, 0.0 kB/s 2023-08-27T11:25:08.750 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:08.750 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:08.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:08.807 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2936 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:08.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:08.864 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:08.865 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0508e-05 s, 0.0 kB/s 2023-08-27T11:25:08.865 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:08.865 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:08.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:08.922 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2937 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:08.978 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:08.979 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:08.979 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8128e-05 s, 0.0 kB/s 2023-08-27T11:25:08.979 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:08.980 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:09.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:09.037 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2938 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:09.094 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:09.094 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:09.094 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9595e-05 s, 0.0 kB/s 2023-08-27T11:25:09.095 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:09.095 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:09.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:09.152 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2939 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:09.209 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:09.209 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:09.209 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8586e-05 s, 0.0 kB/s 2023-08-27T11:25:09.210 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:09.210 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:09.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:09.268 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2940 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:09.325 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:09.326 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:09.326 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8418e-05 s, 0.0 kB/s 2023-08-27T11:25:09.326 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:09.327 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:09.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:09.384 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2941 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:09.442 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:09.443 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:09.443 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1221e-05 s, 0.0 kB/s 2023-08-27T11:25:09.443 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:09.444 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:09.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:09.499 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2942 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:09.557 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:09.557 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:09.557 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3058e-05 s, 0.0 kB/s 2023-08-27T11:25:09.558 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:09.558 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:09.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:09.615 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2943 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:09.672 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:09.673 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:09.673 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7518e-05 s, 0.0 kB/s 2023-08-27T11:25:09.674 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:09.674 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:09.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:09.732 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2944 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:09.790 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:09.791 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:09.791 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9559e-05 s, 0.0 kB/s 2023-08-27T11:25:09.791 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:09.792 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:09.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:09.849 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2945 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:09.906 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:09.906 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:09.907 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9855e-05 s, 0.0 kB/s 2023-08-27T11:25:09.907 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:09.907 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:09.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:09.967 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2946 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:10.024 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:10.025 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:10.025 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8157e-05 s, 0.0 kB/s 2023-08-27T11:25:10.026 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:10.026 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:10.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:10.083 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2947 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:10.141 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7179e-05 s, 0.0 kB/s 2023-08-27T11:25:10.143 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:10.143 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:10.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:10.200 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2948 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:10.256 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:10.257 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:10.257 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0778e-05 s, 0.0 kB/s 2023-08-27T11:25:10.258 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:10.258 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:10.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:10.315 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2949 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:10.372 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:10.372 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:10.373 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0119e-05 s, 0.0 kB/s 2023-08-27T11:25:10.373 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:10.373 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:10.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:10.430 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2950 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:10.488 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:10.488 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:10.488 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0153e-05 s, 0.0 kB/s 2023-08-27T11:25:10.489 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:10.489 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:10.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:10.546 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2951 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:10.603 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:10.604 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:10.604 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3152e-05 s, 0.0 kB/s 2023-08-27T11:25:10.604 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:10.605 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:10.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:10.661 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2952 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:10.718 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:10.718 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:10.718 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.881e-05 s, 0.0 kB/s 2023-08-27T11:25:10.719 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:10.719 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:10.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:10.776 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2953 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:10.834 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:10.835 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:10.835 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2826e-05 s, 0.0 kB/s 2023-08-27T11:25:10.835 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:10.836 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:10.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:10.893 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2954 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1589e-05 s, 0.0 kB/s 2023-08-27T11:25:10.951 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:10.951 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:11.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:11.008 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2955 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:11.066 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:11.066 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:11.066 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2074e-05 s, 0.0 kB/s 2023-08-27T11:25:11.067 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:11.067 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:11.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:11.124 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2956 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:11.182 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:11.182 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:11.182 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9539e-05 s, 0.0 kB/s 2023-08-27T11:25:11.183 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:11.183 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:11.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:11.240 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2957 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8781e-05 s, 0.0 kB/s 2023-08-27T11:25:11.298 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:11.298 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:11.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:11.355 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2958 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:11.412 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:11.412 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:11.412 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0824e-05 s, 0.0 kB/s 2023-08-27T11:25:11.413 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:11.413 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:11.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:11.470 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2959 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:11.528 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:11.528 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:11.528 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1801e-05 s, 0.0 kB/s 2023-08-27T11:25:11.529 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:11.529 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:11.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:11.586 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2960 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:11.644 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:11.645 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:11.645 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9661e-05 s, 0.0 kB/s 2023-08-27T11:25:11.645 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:11.646 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:11.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:11.702 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2961 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:11.760 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:11.760 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:11.760 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2424e-05 s, 0.0 kB/s 2023-08-27T11:25:11.761 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:11.761 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:11.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:11.818 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2962 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:11.876 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:11.876 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:11.877 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1564e-05 s, 0.0 kB/s 2023-08-27T11:25:11.877 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:11.877 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:11.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:11.934 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2963 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2489e-05 s, 0.0 kB/s 2023-08-27T11:25:11.992 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:11.992 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:12.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:12.049 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2964 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:12.105 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:12.106 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:12.106 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.01e-05 s, 0.0 kB/s 2023-08-27T11:25:12.106 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:12.107 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:12.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:12.164 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2965 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:12.220 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:12.221 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:12.221 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.943e-05 s, 0.0 kB/s 2023-08-27T11:25:12.221 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:12.222 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:12.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:12.280 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2966 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4163e-05 s, 0.0 kB/s 2023-08-27T11:25:12.338 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:12.338 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:12.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:12.396 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2967 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:12.453 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:12.454 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:12.454 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.8393e-05 s, 0.0 kB/s 2023-08-27T11:25:12.454 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:12.455 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:12.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:12.511 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2968 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1487e-05 s, 0.0 kB/s 2023-08-27T11:25:12.570 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:12.570 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:12.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:12.627 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2969 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:12.686 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:12.686 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:12.686 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0807e-05 s, 0.0 kB/s 2023-08-27T11:25:12.687 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:12.687 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:12.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:12.744 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2970 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:12.800 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:12.801 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:12.801 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.295e-05 s, 0.0 kB/s 2023-08-27T11:25:12.802 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:12.802 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:12.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:12.858 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2971 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:12.916 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:12.917 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:12.917 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2067e-05 s, 0.0 kB/s 2023-08-27T11:25:12.917 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:12.918 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:12.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:12.974 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2972 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:13.031 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:13.031 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:13.032 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9174e-05 s, 0.0 kB/s 2023-08-27T11:25:13.033 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:13.033 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:13.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:13.091 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2973 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1022e-05 s, 0.0 kB/s 2023-08-27T11:25:13.150 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:13.151 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:13.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:13.207 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2974 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:13.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:13.265 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:13.265 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.308e-05 s, 0.0 kB/s 2023-08-27T11:25:13.265 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:13.266 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:13.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:13.323 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2975 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0247e-05 s, 0.0 kB/s 2023-08-27T11:25:13.382 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:13.383 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:13.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:13.439 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2976 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:13.496 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.171e-05 s, 0.0 kB/s 2023-08-27T11:25:13.497 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:13.498 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:13.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:13.555 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2977 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:13.613 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:13.613 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:13.614 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1533e-05 s, 0.0 kB/s 2023-08-27T11:25:13.614 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:13.614 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:13.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:13.671 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2978 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:13.728 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:13.728 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:13.728 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1348e-05 s, 0.0 kB/s 2023-08-27T11:25:13.729 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:13.729 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:13.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:13.786 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2979 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:13.843 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.301e-05 s, 0.0 kB/s 2023-08-27T11:25:13.844 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:13.845 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:13.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:13.902 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2980 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:13.959 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:13.959 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:13.959 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2471e-05 s, 0.0 kB/s 2023-08-27T11:25:13.960 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:13.960 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:14.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:14.017 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2981 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:14.075 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:14.075 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:14.075 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0849e-05 s, 0.0 kB/s 2023-08-27T11:25:14.076 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:14.076 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:14.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:14.135 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2982 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:14.192 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:14.192 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:14.193 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.5488e-05 s, 0.0 kB/s 2023-08-27T11:25:14.193 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:14.194 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:14.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:14.250 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2983 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:14.308 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:14.308 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:14.309 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.3711e-05 s, 0.0 kB/s 2023-08-27T11:25:14.309 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:14.309 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:14.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:14.365 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2984 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.165e-05 s, 0.0 kB/s 2023-08-27T11:25:14.425 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:14.425 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:14.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:14.483 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2985 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:14.541 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:14.541 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:14.541 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0677e-05 s, 0.0 kB/s 2023-08-27T11:25:14.542 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:14.542 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:14.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:14.599 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2986 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:14.657 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:14.657 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:14.657 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.1591e-05 s, 0.0 kB/s 2023-08-27T11:25:14.658 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:14.658 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:14.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:14.716 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2987 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:14.773 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:14.773 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:14.773 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9693e-05 s, 0.0 kB/s 2023-08-27T11:25:14.773 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:14.774 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:14.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:14.831 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2988 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:14.887 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:14.888 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:14.888 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.43e-05 s, 0.0 kB/s 2023-08-27T11:25:14.888 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:14.889 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:14.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:14.946 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2989 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.7799e-05 s, 0.0 kB/s 2023-08-27T11:25:15.003 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:15.004 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:15.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:15.061 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2990 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:15.118 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:15.120 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:15.120 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2373e-05 s, 0.0 kB/s 2023-08-27T11:25:15.120 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:15.121 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:15.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:15.176 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2991 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:15.232 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:15.233 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:15.233 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0791e-05 s, 0.0 kB/s 2023-08-27T11:25:15.233 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:15.234 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:15.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:15.291 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2992 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9894e-05 s, 0.0 kB/s 2023-08-27T11:25:15.350 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:15.350 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:15.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:15.407 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2993 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:15.465 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:15.465 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:15.466 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.4032e-05 s, 0.0 kB/s 2023-08-27T11:25:15.466 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:15.466 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:15.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:15.523 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2994 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9295e-05 s, 0.0 kB/s 2023-08-27T11:25:15.582 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:15.582 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:15.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:15.639 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2995 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:15.696 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:15.697 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:15.697 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0535e-05 s, 0.0 kB/s 2023-08-27T11:25:15.697 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:15.698 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:15.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:15.755 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2996 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:15.813 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:15.813 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.218e-05 s, 0.0 kB/s 2023-08-27T11:25:15.813 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:15.814 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:15.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:15.870 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2997 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:15.927 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:15.927 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:15.927 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.0213e-05 s, 0.0 kB/s 2023-08-27T11:25:15.928 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:15.928 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:15.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:15.985 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2998 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:16.043 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:16.043 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:16.044 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 2.9068e-05 s, 0.0 kB/s 2023-08-27T11:25:16.044 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:16.045 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:16.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:16.101 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000166348/240fc75e-a6c2-4d2a-84f5-3ba690d1e243/subvolume_file.2999 bs=1M conv=fdatasync count=0 seek=0) 2023-08-27T11:25:16.156 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records in 2023-08-27T11:25:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:0+0 records out 2023-08-27T11:25:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:0 bytes copied, 3.2959e-05 s, 0.0 kB/s 2023-08-27T11:25:16.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000166348 2023-08-27T11:25:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.324+0000 7f1eb29a8700 1 Processor -- start 2023-08-27T11:25:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.324+0000 7f1eb29a8700 1 -- start start 2023-08-27T11:25:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eb29a8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eac06ba50 0x7f1eac15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eb29a8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1eac15ae00 0x7f1eac0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eb29a8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eac15b8d0 con 0x7f1eac0b4d00 2023-08-27T11:25:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eb29a8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eac15b680 con 0x7f1eac15ae00 2023-08-27T11:25:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eb29a8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac15ec10 con 0x7f1eac06ba50 2023-08-27T11:25:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eb0f45700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1eac15ae00 0x7f1eac0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eabfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1eac0b4d00 0x7f1eac06b340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53622/0 (socket says 172.21.15.112:53622) 2023-08-27T11:25:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eabfff700 1 -- 172.21.15.112:0/3007729357 learned_addr learned my addr 172.21.15.112:0/3007729357 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eab7fe700 1 --2- 172.21.15.112:0/3007729357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eac06ba50 0x7f1eac15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eb0f45700 1 -- 172.21.15.112:0/3007729357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eac06ba50 msgr2=0x7f1eac15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:16.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eb0f45700 1 --2- 172.21.15.112:0/3007729357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eac06ba50 0x7f1eac15a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eb0f45700 1 -- 172.21.15.112:0/3007729357 >> v1:172.21.15.112:6789/0 conn(0x7f1eac0b4d00 legacy=0x7f1eac06b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eb0f45700 1 -- 172.21.15.112:0/3007729357 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac15de50 con 0x7f1eac15ae00 2023-08-27T11:25:16.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eaaffd700 1 -- 172.21.15.112:0/3007729357 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480537276 0 0) 0x7f1eac15b8d0 con 0x7f1eac0b4d00 2023-08-27T11:25:16.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eb0f45700 1 --2- 172.21.15.112:0/3007729357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eac15ae00 0x7f1eac0b3440 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009d90 tx=0x7f1ea400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6971c0a4b0779174 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:16.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eaaffd700 1 -- 172.21.15.112:0/3007729357 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea400e070 con 0x7f1eac15ae00 2023-08-27T11:25:16.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eaaffd700 1 -- 172.21.15.112:0/3007729357 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea400bbd0 con 0x7f1eac15ae00 2023-08-27T11:25:16.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.325+0000 7f1eaaffd700 1 -- 172.21.15.112:0/3007729357 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4012430 con 0x7f1eac15ae00 2023-08-27T11:25:16.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.326+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3007729357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eac15ae00 msgr2=0x7f1eac0b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.326+0000 7f1eb29a8700 1 --2- 172.21.15.112:0/3007729357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eac15ae00 0x7f1eac0b3440 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009d90 tx=0x7f1ea400b330 comp rx=0 tx=0).stop 2023-08-27T11:25:16.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.326+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3007729357 shutdown_connections 2023-08-27T11:25:16.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.326+0000 7f1eb29a8700 1 --2- 172.21.15.112:0/3007729357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eac15ae00 0x7f1eac0b3440 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.326+0000 7f1eb29a8700 1 --2- 172.21.15.112:0/3007729357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eac06ba50 0x7f1eac15a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.326+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3007729357 >> 172.21.15.112:0/3007729357 conn(0x7f1eac0d03e0 msgr2=0x7f1eac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:16.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.326+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3007729357 shutdown_connections 2023-08-27T11:25:16.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.326+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3007729357 wait complete. 2023-08-27T11:25:16.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.327+0000 7f1eb29a8700 1 Processor -- start 2023-08-27T11:25:16.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.327+0000 7f1eb29a8700 1 -- start start 2023-08-27T11:25:16.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.328+0000 7f1eb29a8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eac06ba50 0x7f1eac159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.328+0000 7f1eb29a8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eac0b4d00 0x7f1eac1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.328+0000 7f1eb29a8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eac15ae00 0x7f1eac155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.328+0000 7f1eb29a8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac1402b0 con 0x7f1eac0b4d00 2023-08-27T11:25:16.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.328+0000 7f1eb29a8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac155750 con 0x7f1eac15ae00 2023-08-27T11:25:16.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.328+0000 7f1eb29a8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac153a50 con 0x7f1eac06ba50 2023-08-27T11:25:16.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.328+0000 7f1eab7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eac0b4d00 0x7f1eac1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.328+0000 7f1eab7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eac0b4d00 0x7f1eac1533c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59244/0 (socket says 172.21.15.112:59244) 2023-08-27T11:25:16.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.328+0000 7f1eab7fe700 1 -- 172.21.15.112:0/3445914871 learned_addr learned my addr 172.21.15.112:0/3445914871 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:16.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.328+0000 7f1eabfff700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eac06ba50 0x7f1eac159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.329+0000 7f1eab7fe700 1 -- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eac06ba50 msgr2=0x7f1eac159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.329+0000 7f1eb0f45700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eac15ae00 0x7f1eac155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.329+0000 7f1eab7fe700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eac06ba50 0x7f1eac159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.329+0000 7f1eab7fe700 1 -- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eac15ae00 msgr2=0x7f1eac155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.329+0000 7f1eab7fe700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eac15ae00 0x7f1eac155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.329+0000 7f1eab7fe700 1 -- 172.21.15.112:0/3445914871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea4009a70 con 0x7f1eac0b4d00 2023-08-27T11:25:16.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.329+0000 7f1eabfff700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eac06ba50 0x7f1eac159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:16.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.329+0000 7f1eab7fe700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eac0b4d00 0x7f1eac1533c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000e810 tx=0x7f1ea000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:16.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.329+0000 7f1ea8ff9700 1 -- 172.21.15.112:0/3445914871 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0013610 con 0x7f1eac0b4d00 2023-08-27T11:25:16.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.329+0000 7f1ea8ff9700 1 -- 172.21.15.112:0/3445914871 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea0013770 con 0x7f1eac0b4d00 2023-08-27T11:25:16.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.329+0000 7f1ea8ff9700 1 -- 172.21.15.112:0/3445914871 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0013b20 con 0x7f1eac0b4d00 2023-08-27T11:25:16.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.329+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3445914871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eac153d30 con 0x7f1eac0b4d00 2023-08-27T11:25:16.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.329+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3445914871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eac1542f0 con 0x7f1eac0b4d00 2023-08-27T11:25:16.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.330+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3445914871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eac06e8f0 con 0x7f1eac0b4d00 2023-08-27T11:25:16.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.330+0000 7f1ea8ff9700 1 -- 172.21.15.112:0/3445914871 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f1ea0013c80 con 0x7f1eac0b4d00 2023-08-27T11:25:16.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.331+0000 7f1ea8ff9700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e9806a170 0x7f1e9806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.331+0000 7f1ea8ff9700 1 -- 172.21.15.112:0/3445914871 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1ea0088440 con 0x7f1eac0b4d00 2023-08-27T11:25:16.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.331+0000 7f1eabfff700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e9806a170 0x7f1e9806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.331+0000 7f1eabfff700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e9806a170 0x7f1e9806c630 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c000f60 tx=0x7f1e9c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:16.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.335+0000 7f1ea8ff9700 1 -- 172.21.15.112:0/3445914871 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1ea0053940 con 0x7f1eac0b4d00 2023-08-27T11:25:16.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.557+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3445914871 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000166348", "target": ["mon-mgr", ""]}) v1 -- 0x7f1eac15e130 con 0x7f1e9806a170 2023-08-27T11:25:16.563 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.563+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.563 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.563+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.563 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.563+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.563 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.563+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.564 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.563+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.599+0000 7f1ea8ff9700 1 -- 172.21.15.112:0/3445914871 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1e9c001cc0 con 0x7f1e9806a170 2023-08-27T11:25:16.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.601+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e9806a170 msgr2=0x7f1e9806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.601+0000 7f1eb29a8700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e9806a170 0x7f1e9806c630 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c000f60 tx=0x7f1e9c009040 comp rx=0 tx=0).stop 2023-08-27T11:25:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.601+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3445914871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eac0b4d00 msgr2=0x7f1eac1533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.601+0000 7f1eb29a8700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eac0b4d00 0x7f1eac1533c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000e810 tx=0x7f1ea000ebd0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.601+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3445914871 shutdown_connections 2023-08-27T11:25:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.601+0000 7f1eb29a8700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e9806a170 0x7f1e9806c630 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.601+0000 7f1eb29a8700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eac15ae00 0x7f1eac155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.601+0000 7f1eb29a8700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eac06ba50 0x7f1eac159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.601+0000 7f1eb29a8700 1 --2- 172.21.15.112:0/3445914871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eac0b4d00 0x7f1eac1533c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.601+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3445914871 >> 172.21.15.112:0/3445914871 conn(0x7f1eac0d03e0 msgr2=0x7f1eac06b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.601+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3445914871 shutdown_connections 2023-08-27T11:25:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.601+0000 7f1eb29a8700 1 -- 172.21.15.112:0/3445914871 wait complete. 2023-08-27T11:25:16.605 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.602+0000 7f285cc7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.605 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.602+0000 7f285cc7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.605 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.602+0000 7f285cc7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.606 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.602+0000 7f285cc7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.606 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.602+0000 7f285cc7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-08-27T11:25:16.620 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.619+0000 7f285b47c700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.620 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.619+0000 7f285b47c700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.620 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.619+0000 7f285b47c700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.620 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.619+0000 7f285b47c700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.621 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:16.619+0000 7f285b47c700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:16.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.742+0000 7f92e830d700 1 Processor -- start 2023-08-27T11:25:16.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.742+0000 7f92e830d700 1 -- start start 2023-08-27T11:25:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.743+0000 7f92e830d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f92e015df40 0x7f92e006dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.743+0000 7f92e830d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e0066410 0x7f92e00cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.743+0000 7f92e830d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e00a84f0 con 0x7f92e015e320 2023-08-27T11:25:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.743+0000 7f92e830d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e0072060 con 0x7f92e015df40 2023-08-27T11:25:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.743+0000 7f92e830d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00713e0 con 0x7f92e0066410 2023-08-27T11:25:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.743+0000 7f92e68aa700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f92e015e320 0x7f92e015a6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53636/0 (socket says 172.21.15.112:53636) 2023-08-27T11:25:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e68aa700 1 -- 172.21.15.112:0/686391168 learned_addr learned my addr 172.21.15.112:0/686391168 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e58a8700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e0066410 0x7f92e00cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e60a9700 1 --2- 172.21.15.112:0/686391168 >> v2:172.21.15.138:3300/0 conn(0x7f92e015df40 0x7f92e006dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998371048 0 0) 0x7f92e00a84f0 con 0x7f92e015e320 2023-08-27T11:25:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92cc0035b0 con 0x7f92e015e320 2023-08-27T11:25:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 220697590 0 0) 0x7f92cc0035b0 con 0x7f92e015e320 2023-08-27T11:25:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e0066410 msgr2=0x7f92e00cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e0066410 0x7f92e00cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 >> v2:172.21.15.138:3300/0 conn(0x7f92e015df40 msgr2=0x7f92e006dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 --2- 172.21.15.112:0/686391168 >> v2:172.21.15.138:3300/0 conn(0x7f92e015df40 0x7f92e006dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e00716c0 con 0x7f92e015e320 2023-08-27T11:25:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f92d80019f0 con 0x7f92e015e320 2023-08-27T11:25:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 >> v1:172.21.15.112:6789/0 conn(0x7f92e015e320 legacy=0x7f92e015a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92cc003a00 0x7f92cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92cc004850 0x7f92cc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e0066410 0x7f92cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0072060 con 0x7f92cc003a00 2023-08-27T11:25:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00713e0 con 0x7f92cc004850 2023-08-27T11:25:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc00dc20 con 0x7f92e0066410 2023-08-27T11:25:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e60a9700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92cc003a00 0x7f92cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e58a8700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92cc004850 0x7f92cc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e68aa700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e0066410 0x7f92cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e60a9700 1 -- 172.21.15.112:0/686391168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e0066410 msgr2=0x7f92cc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e60a9700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e0066410 0x7f92cc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e60a9700 1 -- 172.21.15.112:0/686391168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92cc004850 msgr2=0x7f92cc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e60a9700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92cc004850 0x7f92cc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e60a9700 1 -- 172.21.15.112:0/686391168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc0035b0 con 0x7f92cc003a00 2023-08-27T11:25:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e68aa700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e0066410 0x7f92cc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.744+0000 7f92e58a8700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92cc004850 0x7f92cc004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.745+0000 7f92e60a9700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92cc003a00 0x7f92cc003e20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009b60 tx=0x7f92cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491338ea75afc4d7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.745+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc00e6c0 con 0x7f92cc003a00 2023-08-27T11:25:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.745+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc00e820 con 0x7f92cc003a00 2023-08-27T11:25:16.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.745+0000 7f92e50a7700 1 -- 172.21.15.112:0/686391168 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc00eb90 con 0x7f92cc003a00 2023-08-27T11:25:16.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.745+0000 7f92e830d700 1 -- 172.21.15.112:0/686391168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92cc003a00 msgr2=0x7f92cc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.745+0000 7f92e830d700 1 --2- 172.21.15.112:0/686391168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92cc003a00 0x7f92cc003e20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009b60 tx=0x7f92cc005880 comp rx=0 tx=0).stop 2023-08-27T11:25:16.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.745+0000 7f92e68aa700 1 -- 172.21.15.112:0/686391168 reap_dead start 2023-08-27T11:25:16.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.746+0000 7f92e830d700 1 -- 172.21.15.112:0/686391168 shutdown_connections 2023-08-27T11:25:16.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.746+0000 7f92e830d700 1 -- 172.21.15.112:0/686391168 >> 172.21.15.112:0/686391168 conn(0x7f92e00b1380 msgr2=0x7f92e00b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:16.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.746+0000 7f92e830d700 1 -- 172.21.15.112:0/686391168 shutdown_connections 2023-08-27T11:25:16.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.746+0000 7f92e830d700 1 -- 172.21.15.112:0/686391168 wait complete. 2023-08-27T11:25:16.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.747+0000 7f92e830d700 1 Processor -- start 2023-08-27T11:25:16.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.747+0000 7f92e830d700 1 -- start start 2023-08-27T11:25:16.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e830d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92cc003a00 0x7f92e0154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e830d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e013b850 0x7f92e00701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e830d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92e013bcd0 0x7f92e00706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e830d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e01583d0 con 0x7f92e013bcd0 2023-08-27T11:25:16.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e830d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0154da0 con 0x7f92e013b850 2023-08-27T11:25:16.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e830d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0155060 con 0x7f92cc003a00 2023-08-27T11:25:16.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e68aa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92e013bcd0 0x7f92e00706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e60a9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92cc003a00 0x7f92e0154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e58a8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e013b850 0x7f92e00701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e68aa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92e013bcd0 0x7f92e00706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59266/0 (socket says 172.21.15.112:59266) 2023-08-27T11:25:16.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e60a9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92cc003a00 0x7f92e0154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:42684/0 (socket says 172.21.15.112:42684) 2023-08-27T11:25:16.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e68aa700 1 -- 172.21.15.112:0/3078413619 learned_addr learned my addr 172.21.15.112:0/3078413619 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:16.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e68aa700 1 -- 172.21.15.112:0/3078413619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92cc003a00 msgr2=0x7f92e0154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e68aa700 1 --2- 172.21.15.112:0/3078413619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92cc003a00 0x7f92e0154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e68aa700 1 -- 172.21.15.112:0/3078413619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e013b850 msgr2=0x7f92e00701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e68aa700 1 --2- 172.21.15.112:0/3078413619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e013b850 0x7f92e00701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e68aa700 1 -- 172.21.15.112:0/3078413619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e013d000 con 0x7f92e013bcd0 2023-08-27T11:25:16.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e68aa700 1 --2- 172.21.15.112:0/3078413619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92e013bcd0 0x7f92e00706e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f92e015e700 tx=0x7f92d800e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92d6ffd700 1 -- 172.21.15.112:0/3078413619 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d800ec10 con 0x7f92e013bcd0 2023-08-27T11:25:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92d6ffd700 1 -- 172.21.15.112:0/3078413619 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d800ed70 con 0x7f92e013bcd0 2023-08-27T11:25:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e830d700 1 -- 172.21.15.112:0/3078413619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e013d2e0 con 0x7f92e013bcd0 2023-08-27T11:25:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92d6ffd700 1 -- 172.21.15.112:0/3078413619 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d800eee0 con 0x7f92e013bcd0 2023-08-27T11:25:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.748+0000 7f92e830d700 1 -- 172.21.15.112:0/3078413619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e0158950 con 0x7f92e013bcd0 2023-08-27T11:25:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.750+0000 7f92e830d700 1 -- 172.21.15.112:0/3078413619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e013d4d0 con 0x7f92e013bcd0 2023-08-27T11:25:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.750+0000 7f92d6ffd700 1 -- 172.21.15.112:0/3078413619 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 88392+0+0 (secure 0 0 0) 0x7f92d8013660 con 0x7f92e013bcd0 2023-08-27T11:25:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.750+0000 7f92d6ffd700 1 --2- 172.21.15.112:0/3078413619 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f92c406a170 0x7f92c406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.751+0000 7f92d6ffd700 1 -- 172.21.15.112:0/3078413619 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f92d80888c0 con 0x7f92e013bcd0 2023-08-27T11:25:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.751+0000 7f92e60a9700 1 --2- 172.21.15.112:0/3078413619 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f92c406a170 0x7f92c406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.751+0000 7f92e60a9700 1 --2- 172.21.15.112:0/3078413619 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f92c406a170 0x7f92c406c630 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0090b0 tx=0x7f92dc01f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.755+0000 7f92d6ffd700 1 -- 172.21.15.112:0/3078413619 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f92d8054050 con 0x7f92e013bcd0 2023-08-27T11:25:16.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:16.990+0000 7f92e830d700 1 -- 172.21.15.112:0/3078413619 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f92e0071c80 con 0x7f92c406a170 2023-08-27T11:25:17.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.243+0000 7f92d6ffd700 1 -- 172.21.15.112:0/3078413619 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+539 (secure 0 0 0) 0x7f92dc006c60 con 0x7f92c406a170 2023-08-27T11:25:17.245 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:25:17.245 INFO:teuthology.orchestra.run.smithi112.stdout: "mon_addrs": [ 2023-08-27T11:25:17.246 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.112:6789", 2023-08-27T11:25:17.246 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.138:6789", 2023-08-27T11:25:17.246 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.138:6790" 2023-08-27T11:25:17.246 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2023-08-27T11:25:17.246 INFO:teuthology.orchestra.run.smithi112.stdout: "pending_subvolume_deletions": 1, 2023-08-27T11:25:17.247 INFO:teuthology.orchestra.run.smithi112.stdout: "pools": { 2023-08-27T11:25:17.247 INFO:teuthology.orchestra.run.smithi112.stdout: "data": [ 2023-08-27T11:25:17.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:17.247 INFO:teuthology.orchestra.run.smithi112.stdout: "avail": 367072247808, 2023-08-27T11:25:17.248 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs_data", 2023-08-27T11:25:17.248 INFO:teuthology.orchestra.run.smithi112.stdout: "used": 8192 2023-08-27T11:25:17.248 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:17.248 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2023-08-27T11:25:17.248 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": [ 2023-08-27T11:25:17.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:17.249 INFO:teuthology.orchestra.run.smithi112.stdout: "avail": 367072247808, 2023-08-27T11:25:17.249 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs_metadata", 2023-08-27T11:25:17.249 INFO:teuthology.orchestra.run.smithi112.stdout: "used": 24559616 2023-08-27T11:25:17.249 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:17.250 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:25:17.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:17.250 INFO:teuthology.orchestra.run.smithi112.stdout: "used_size": 145 2023-08-27T11:25:17.250 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:25:17.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.246+0000 7f92e830d700 1 -- 172.21.15.112:0/3078413619 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f92c406a170 msgr2=0x7f92c406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:17.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.246+0000 7f92e830d700 1 --2- 172.21.15.112:0/3078413619 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f92c406a170 0x7f92c406c630 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0090b0 tx=0x7f92dc01f040 comp rx=0 tx=0).stop 2023-08-27T11:25:17.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.246+0000 7f92e830d700 1 -- 172.21.15.112:0/3078413619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92e013bcd0 msgr2=0x7f92e00706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:17.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.246+0000 7f92e830d700 1 --2- 172.21.15.112:0/3078413619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92e013bcd0 0x7f92e00706e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f92e015e700 tx=0x7f92d800e440 comp rx=0 tx=0).stop 2023-08-27T11:25:17.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.246+0000 7f92e830d700 1 -- 172.21.15.112:0/3078413619 shutdown_connections 2023-08-27T11:25:17.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.246+0000 7f92e830d700 1 --2- 172.21.15.112:0/3078413619 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f92c406a170 0x7f92c406c630 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.246+0000 7f92e830d700 1 --2- 172.21.15.112:0/3078413619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92e013bcd0 0x7f92e00706e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.246+0000 7f92e830d700 1 --2- 172.21.15.112:0/3078413619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92cc003a00 0x7f92e0154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.246+0000 7f92e830d700 1 --2- 172.21.15.112:0/3078413619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e013b850 0x7f92e00701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.246+0000 7f92e830d700 1 -- 172.21.15.112:0/3078413619 >> 172.21.15.112:0/3078413619 conn(0x7f92e00b1380 msgr2=0x7f92e013f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:17.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.246+0000 7f92e830d700 1 -- 172.21.15.112:0/3078413619 shutdown_connections 2023-08-27T11:25:17.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.246+0000 7f92e830d700 1 -- 172.21.15.112:0/3078413619 wait complete. 2023-08-27T11:25:17.261 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:25:17.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:25:17.304 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:25:17.349 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:25:17.349 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:17.350 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:17.367 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:17.367 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:17.425 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:17.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.588+0000 7fd651460700 1 Processor -- start 2023-08-27T11:25:17.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.588+0000 7fd651460700 1 -- start start 2023-08-27T11:25:17.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.589+0000 7fd651460700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd64c160a00 0x7fd64c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:17.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.589+0000 7fd651460700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64c06f930 con 0x7fd64c06d510 2023-08-27T11:25:17.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.589+0000 7fd651460700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64c06fc10 con 0x7fd64c146990 2023-08-27T11:25:17.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.589+0000 7fd651460700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64c06fef0 con 0x7fd64c160a00 2023-08-27T11:25:17.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.589+0000 7fd64b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd64c06d510 0x7fd64c06d910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53656/0 (socket says 172.21.15.112:53656) 2023-08-27T11:25:17.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.589+0000 7fd64b7fe700 1 -- 172.21.15.112:0/2532954534 learned_addr learned my addr 172.21.15.112:0/2532954534 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:17.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.589+0000 7fd64affd700 1 --2- 172.21.15.112:0/2532954534 >> v2:172.21.15.138:3300/0 conn(0x7fd64c160a00 0x7fd64c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:17.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.590+0000 7fd649ffb700 1 -- 172.21.15.112:0/2532954534 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480014768 0 0) 0x7fd64c06f930 con 0x7fd64c06d510 2023-08-27T11:25:17.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.590+0000 7fd649ffb700 1 -- 172.21.15.112:0/2532954534 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6340035b0 con 0x7fd64c06d510 2023-08-27T11:25:17.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.590+0000 7fd649ffb700 1 -- 172.21.15.112:0/2532954534 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2945004626 0 0) 0x7fd64c06fc10 con 0x7fd64c146990 2023-08-27T11:25:17.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.590+0000 7fd649ffb700 1 -- 172.21.15.112:0/2532954534 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd64c06f930 con 0x7fd64c146990 2023-08-27T11:25:17.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.590+0000 7fd64affd700 1 -- 172.21.15.112:0/2532954534 >> v1:172.21.15.138:6789/0 conn(0x7fd64c146990 legacy=0x7fd64c15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:17.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.590+0000 7fd64affd700 1 -- 172.21.15.112:0/2532954534 >> v1:172.21.15.112:6789/0 conn(0x7fd64c06d510 legacy=0x7fd64c06d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:17.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.590+0000 7fd64affd700 1 -- 172.21.15.112:0/2532954534 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c0701d0 con 0x7fd64c160a00 2023-08-27T11:25:17.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.590+0000 7fd64affd700 1 --2- 172.21.15.112:0/2532954534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd64c160a00 0x7fd64c0b46d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009840 tx=0x7fd63c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1525a8a86231518d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:17.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.590+0000 7fd649ffb700 1 -- 172.21.15.112:0/2532954534 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c011830 con 0x7fd64c160a00 2023-08-27T11:25:17.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.590+0000 7fd649ffb700 1 -- 172.21.15.112:0/2532954534 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63c011990 con 0x7fd64c160a00 2023-08-27T11:25:17.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.590+0000 7fd649ffb700 1 -- 172.21.15.112:0/2532954534 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c011d20 con 0x7fd64c160a00 2023-08-27T11:25:17.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.591+0000 7fd651460700 1 -- 172.21.15.112:0/2532954534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd64c160a00 msgr2=0x7fd64c0b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:17.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.591+0000 7fd651460700 1 --2- 172.21.15.112:0/2532954534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd64c160a00 0x7fd64c0b46d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009840 tx=0x7fd63c009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:17.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.591+0000 7fd651460700 1 -- 172.21.15.112:0/2532954534 shutdown_connections 2023-08-27T11:25:17.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.591+0000 7fd651460700 1 --2- 172.21.15.112:0/2532954534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd64c160a00 0x7fd64c0b46d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.591+0000 7fd651460700 1 -- 172.21.15.112:0/2532954534 >> 172.21.15.112:0/2532954534 conn(0x7fd64c0cfe50 msgr2=0x7fd64c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:17.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.591+0000 7fd651460700 1 -- 172.21.15.112:0/2532954534 shutdown_connections 2023-08-27T11:25:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.591+0000 7fd651460700 1 -- 172.21.15.112:0/2532954534 wait complete. 2023-08-27T11:25:17.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.592+0000 7fd651460700 1 Processor -- start 2023-08-27T11:25:17.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.592+0000 7fd651460700 1 -- start start 2023-08-27T11:25:17.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd651460700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd64c06d510 0x7fd64c13c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:17.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd651460700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd64c146990 0x7fd64c1f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:17.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd651460700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd64c160a00 0x7fd64c144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:17.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd651460700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c1f6520 con 0x7fd64c160a00 2023-08-27T11:25:17.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd651460700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c13cee0 con 0x7fd64c146990 2023-08-27T11:25:17.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd651460700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c13d1c0 con 0x7fd64c06d510 2023-08-27T11:25:17.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd64b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd64c160a00 0x7fd64c144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:17.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd64b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd64c160a00 0x7fd64c144290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59280/0 (socket says 172.21.15.112:59280) 2023-08-27T11:25:17.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd64b7fe700 1 -- 172.21.15.112:0/2555431965 learned_addr learned my addr 172.21.15.112:0/2555431965 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:17.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd64affd700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd64c06d510 0x7fd64c13c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:17.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd64a7fc700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd64c146990 0x7fd64c1f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:17.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd64b7fe700 1 -- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd64c06d510 msgr2=0x7fd64c13c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:17.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd64b7fe700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd64c06d510 0x7fd64c13c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd64b7fe700 1 -- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd64c146990 msgr2=0x7fd64c1f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:17.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd64b7fe700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd64c146990 0x7fd64c1f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.593+0000 7fd64b7fe700 1 -- 172.21.15.112:0/2555431965 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0094a0 con 0x7fd64c160a00 2023-08-27T11:25:17.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.594+0000 7fd64b7fe700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd64c160a00 0x7fd64c144290 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd64400cb40 tx=0x7fd64400ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:17.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.594+0000 7fd6337fe700 1 -- 172.21.15.112:0/2555431965 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64400a9e0 con 0x7fd64c160a00 2023-08-27T11:25:17.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.594+0000 7fd6337fe700 1 -- 172.21.15.112:0/2555431965 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64400ab40 con 0x7fd64c160a00 2023-08-27T11:25:17.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.594+0000 7fd6337fe700 1 -- 172.21.15.112:0/2555431965 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644012400 con 0x7fd64c160a00 2023-08-27T11:25:17.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.594+0000 7fd651460700 1 -- 172.21.15.112:0/2555431965 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c1f67c0 con 0x7fd64c160a00 2023-08-27T11:25:17.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.594+0000 7fd651460700 1 -- 172.21.15.112:0/2555431965 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c13d4a0 con 0x7fd64c160a00 2023-08-27T11:25:17.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.595+0000 7fd651460700 1 -- 172.21.15.112:0/2555431965 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64c1f6e10 con 0x7fd64c160a00 2023-08-27T11:25:17.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.595+0000 7fd6337fe700 1 -- 172.21.15.112:0/2555431965 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fd64401b3b0 con 0x7fd64c160a00 2023-08-27T11:25:17.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.596+0000 7fd6337fe700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd63806a320 0x7fd63806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:17.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.596+0000 7fd6337fe700 1 -- 172.21.15.112:0/2555431965 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd644088400 con 0x7fd64c160a00 2023-08-27T11:25:17.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.596+0000 7fd64affd700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd63806a320 0x7fd63806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:17.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.596+0000 7fd64affd700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd63806a320 0x7fd63806c7e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009fd0 tx=0x7fd63c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:17.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.600+0000 7fd6337fe700 1 -- 172.21.15.112:0/2555431965 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd6440538d0 con 0x7fd64c160a00 2023-08-27T11:25:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.803+0000 7fd651460700 1 -- 172.21.15.112:0/2555431965 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd64c067a20 con 0x7fd64c160a00 2023-08-27T11:25:17.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.804+0000 7fd6337fe700 1 -- 172.21.15.112:0/2555431965 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18320 (secure 0 0 0) 0x7fd644056ef0 con 0x7fd64c160a00 2023-08-27T11:25:17.806 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":65,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:19:17.239518+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:19:14.179813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-27T11:19:15.021524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:17.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.807+0000 7fd651460700 1 -- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd63806a320 msgr2=0x7fd63806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:17.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.807+0000 7fd651460700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd63806a320 0x7fd63806c7e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009fd0 tx=0x7fd63c01c040 comp rx=0 tx=0).stop 2023-08-27T11:25:17.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.807+0000 7fd651460700 1 -- 172.21.15.112:0/2555431965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd64c160a00 msgr2=0x7fd64c144290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:17.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.807+0000 7fd651460700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd64c160a00 0x7fd64c144290 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd64400cb40 tx=0x7fd64400ce50 comp rx=0 tx=0).stop 2023-08-27T11:25:17.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.807+0000 7fd651460700 1 -- 172.21.15.112:0/2555431965 shutdown_connections 2023-08-27T11:25:17.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.807+0000 7fd651460700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd63806a320 0x7fd63806c7e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.807+0000 7fd651460700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd64c160a00 0x7fd64c144290 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.807+0000 7fd651460700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd64c06d510 0x7fd64c13c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.807+0000 7fd651460700 1 --2- 172.21.15.112:0/2555431965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd64c146990 0x7fd64c1f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.807+0000 7fd651460700 1 -- 172.21.15.112:0/2555431965 >> 172.21.15.112:0/2555431965 conn(0x7fd64c0cfe50 msgr2=0x7fd64c0d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:17.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.807+0000 7fd651460700 1 -- 172.21.15.112:0/2555431965 shutdown_connections 2023-08-27T11:25:17.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.807+0000 7fd651460700 1 -- 172.21.15.112:0/2555431965 wait complete. 2023-08-27T11:25:17.821 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:19:14.179813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:19:15.021524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:25:17.822 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:25:17.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.953+0000 7f3a0207c700 1 Processor -- start 2023-08-27T11:25:17.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.953+0000 7f3a0207c700 1 -- start start 2023-08-27T11:25:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.954+0000 7f3a0207c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39fc1437c0 0x7f39fc0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.954+0000 7f3a0207c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f39fc06d5b0 0x7f39fc06da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.954+0000 7f3a0207c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39fc0cfd50 con 0x7f39fc143ba0 2023-08-27T11:25:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.954+0000 7f3a0207c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39fc06df40 con 0x7f39fc06d5b0 2023-08-27T11:25:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.954+0000 7f3a0207c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc0cf140 con 0x7f39fc1437c0 2023-08-27T11:25:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.954+0000 7f39faffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f39fc143ba0 0x7f39fc0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53668/0 (socket says 172.21.15.112:53668) 2023-08-27T11:25:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.954+0000 7f39faffd700 1 -- 172.21.15.112:0/3184478732 learned_addr learned my addr 172.21.15.112:0/3184478732 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.954+0000 7f39fbfff700 1 --2- 172.21.15.112:0/3184478732 >> v2:172.21.15.112:3300/0 conn(0x7f39fc06d5b0 0x7f39fc06da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.954+0000 7f39fb7fe700 1 --2- 172.21.15.112:0/3184478732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39fc1437c0 0x7f39fc0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.955+0000 7f39fa7fc700 1 -- 172.21.15.112:0/3184478732 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701036122 0 0) 0x7f39fc0cfd50 con 0x7f39fc143ba0 2023-08-27T11:25:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.955+0000 7f39fa7fc700 1 -- 172.21.15.112:0/3184478732 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39e40035b0 con 0x7f39fc143ba0 2023-08-27T11:25:17.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.955+0000 7f39fbfff700 1 -- 172.21.15.112:0/3184478732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39fc1437c0 msgr2=0x7f39fc0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:17.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.955+0000 7f39fbfff700 1 --2- 172.21.15.112:0/3184478732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39fc1437c0 0x7f39fc0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.955+0000 7f39fbfff700 1 -- 172.21.15.112:0/3184478732 >> v1:172.21.15.112:6789/0 conn(0x7f39fc143ba0 legacy=0x7f39fc0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:17.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.955+0000 7f39fbfff700 1 -- 172.21.15.112:0/3184478732 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0cf420 con 0x7f39fc06d5b0 2023-08-27T11:25:17.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.955+0000 7f39fbfff700 1 --2- 172.21.15.112:0/3184478732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39fc06d5b0 0x7f39fc06da00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009d90 tx=0x7f39f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adcede25198082e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.955+0000 7f39fa7fc700 1 -- 172.21.15.112:0/3184478732 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f400e070 con 0x7f39fc06d5b0 2023-08-27T11:25:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.955+0000 7f39fa7fc700 1 -- 172.21.15.112:0/3184478732 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f400bb20 con 0x7f39fc06d5b0 2023-08-27T11:25:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.955+0000 7f39fa7fc700 1 -- 172.21.15.112:0/3184478732 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4012430 con 0x7f39fc06d5b0 2023-08-27T11:25:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.956+0000 7f3a0207c700 1 -- 172.21.15.112:0/3184478732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39fc06d5b0 msgr2=0x7f39fc06da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.956+0000 7f3a0207c700 1 --2- 172.21.15.112:0/3184478732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39fc06d5b0 0x7f39fc06da00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009d90 tx=0x7f39f400b330 comp rx=0 tx=0).stop 2023-08-27T11:25:17.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.956+0000 7f3a0207c700 1 -- 172.21.15.112:0/3184478732 shutdown_connections 2023-08-27T11:25:17.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.956+0000 7f3a0207c700 1 --2- 172.21.15.112:0/3184478732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39fc06d5b0 0x7f39fc06da00 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.956+0000 7f3a0207c700 1 --2- 172.21.15.112:0/3184478732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39fc1437c0 0x7f39fc0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.956+0000 7f3a0207c700 1 -- 172.21.15.112:0/3184478732 >> 172.21.15.112:0/3184478732 conn(0x7f39fc0b1e90 msgr2=0x7f39fc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.956+0000 7f3a0207c700 1 -- 172.21.15.112:0/3184478732 shutdown_connections 2023-08-27T11:25:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.956+0000 7f3a0207c700 1 -- 172.21.15.112:0/3184478732 wait complete. 2023-08-27T11:25:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.957+0000 7f3a0207c700 1 Processor -- start 2023-08-27T11:25:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.957+0000 7f3a0207c700 1 -- start start 2023-08-27T11:25:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f3a0207c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39fc06d5b0 0x7f39fc15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f3a0207c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39fc1437c0 0x7f39fc071070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f3a0207c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39fc143ba0 0x7f39fc1594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f3a0207c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc071b30 con 0x7f39fc06d5b0 2023-08-27T11:25:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f3a0207c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc071e10 con 0x7f39fc143ba0 2023-08-27T11:25:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f3a0207c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc0720f0 con 0x7f39fc1437c0 2023-08-27T11:25:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f39fb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39fc06d5b0 0x7f39fc15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f39fb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39fc06d5b0 0x7f39fc15c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59290/0 (socket says 172.21.15.112:59290) 2023-08-27T11:25:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f39fb7fe700 1 -- 172.21.15.112:0/2157002005 learned_addr learned my addr 172.21.15.112:0/2157002005 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f39faffd700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39fc1437c0 0x7f39fc071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f39fbfff700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39fc143ba0 0x7f39fc1594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f39fb7fe700 1 -- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39fc1437c0 msgr2=0x7f39fc071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f39fb7fe700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39fc1437c0 0x7f39fc071070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f39fb7fe700 1 -- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39fc143ba0 msgr2=0x7f39fc1594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f39fb7fe700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39fc143ba0 0x7f39fc1594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f39fb7fe700 1 -- 172.21.15.112:0/2157002005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f4009a70 con 0x7f39fc06d5b0 2023-08-27T11:25:17.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.958+0000 7f39fb7fe700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39fc06d5b0 0x7f39fc15c9a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00e9b0 tx=0x7f39ec00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:17.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.959+0000 7f39e3fff700 1 -- 172.21.15.112:0/2157002005 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec009710 con 0x7f39fc06d5b0 2023-08-27T11:25:17.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.959+0000 7f39e3fff700 1 -- 172.21.15.112:0/2157002005 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec00ee50 con 0x7f39fc06d5b0 2023-08-27T11:25:17.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.959+0000 7f3a0207c700 1 -- 172.21.15.112:0/2157002005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39fc0723d0 con 0x7f39fc06d5b0 2023-08-27T11:25:17.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.959+0000 7f39e3fff700 1 -- 172.21.15.112:0/2157002005 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec00ca50 con 0x7f39fc06d5b0 2023-08-27T11:25:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.959+0000 7f3a0207c700 1 -- 172.21.15.112:0/2157002005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39fc135120 con 0x7f39fc06d5b0 2023-08-27T11:25:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.960+0000 7f3a0207c700 1 -- 172.21.15.112:0/2157002005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39fc0c9870 con 0x7f39fc06d5b0 2023-08-27T11:25:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.960+0000 7f39e3fff700 1 -- 172.21.15.112:0/2157002005 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f39ec00cbb0 con 0x7f39fc06d5b0 2023-08-27T11:25:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.961+0000 7f39e3fff700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39e806a320 0x7f39e806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.961+0000 7f39e3fff700 1 -- 172.21.15.112:0/2157002005 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39ec088990 con 0x7f39fc06d5b0 2023-08-27T11:25:17.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.961+0000 7f39faffd700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39e806a320 0x7f39e806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:17.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.961+0000 7f39faffd700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39e806a320 0x7f39e806c7e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f39fc15db80 tx=0x7f39f0007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:17.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:17.968+0000 7f39e3fff700 1 -- 172.21.15.112:0/2157002005 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f39ec0577b0 con 0x7f39fc06d5b0 2023-08-27T11:25:18.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.201+0000 7f3a0207c700 1 -- 172.21.15.112:0/2157002005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f39fc067e90 con 0x7f39fc06d5b0 2023-08-27T11:25:18.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.201+0000 7f39e3fff700 1 -- 172.21.15.112:0/2157002005 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f39ec017020 con 0x7f39fc06d5b0 2023-08-27T11:25:18.203 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:25:18.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.203+0000 7f3a0207c700 1 -- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39e806a320 msgr2=0x7f39e806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.203+0000 7f3a0207c700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39e806a320 0x7f39e806c7e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f39fc15db80 tx=0x7f39f0007dc0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.203+0000 7f3a0207c700 1 -- 172.21.15.112:0/2157002005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39fc06d5b0 msgr2=0x7f39fc15c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.203+0000 7f3a0207c700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39fc06d5b0 0x7f39fc15c9a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00e9b0 tx=0x7f39ec00ecc0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.204+0000 7f3a0207c700 1 -- 172.21.15.112:0/2157002005 shutdown_connections 2023-08-27T11:25:18.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.204+0000 7f3a0207c700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39e806a320 0x7f39e806c7e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.204+0000 7f3a0207c700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39fc143ba0 0x7f39fc1594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.204+0000 7f3a0207c700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39fc06d5b0 0x7f39fc15c9a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.204+0000 7f3a0207c700 1 --2- 172.21.15.112:0/2157002005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39fc1437c0 0x7f39fc071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.204+0000 7f3a0207c700 1 -- 172.21.15.112:0/2157002005 >> 172.21.15.112:0/2157002005 conn(0x7f39fc0b1e90 msgr2=0x7f39fc0b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:18.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.204+0000 7f3a0207c700 1 -- 172.21.15.112:0/2157002005 shutdown_connections 2023-08-27T11:25:18.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.204+0000 7f3a0207c700 1 -- 172.21.15.112:0/2157002005 wait complete. 2023-08-27T11:25:18.218 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:25:18.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.345+0000 7f83f2078700 1 Processor -- start 2023-08-27T11:25:18.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.345+0000 7f83f2078700 1 -- start start 2023-08-27T11:25:18.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.345+0000 7f83f2078700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83ec0d1a60 0x7f83ec15e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:18.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.345+0000 7f83f2078700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f83ec071360 0x7f83ec0717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:18.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.345+0000 7f83f2078700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec071d10 con 0x7f83ec15af90 2023-08-27T11:25:18.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.345+0000 7f83f2078700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec15b370 con 0x7f83ec071360 2023-08-27T11:25:18.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.345+0000 7f83f2078700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0b6e70 con 0x7f83ec0d1a60 2023-08-27T11:25:18.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83eaffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83ec0d1a60 0x7f83ec15e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:18.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83ebfff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f83ec071360 0x7f83ec0717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:18.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83eb7fe700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f83ec15af90 0x7f83ec0b6680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:32972/0 (socket says 172.21.15.112:32972) 2023-08-27T11:25:18.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83eb7fe700 1 -- 172.21.15.112:0/2246185057 learned_addr learned my addr 172.21.15.112:0/2246185057 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:18.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83eaffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83ec0d1a60 0x7f83ec15e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:42722/0 (socket says 172.21.15.112:42722) 2023-08-27T11:25:18.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83ea7fc700 1 -- 172.21.15.112:0/2246185057 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179131617 0 0) 0x7f83ec071d10 con 0x7f83ec15af90 2023-08-27T11:25:18.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83ea7fc700 1 -- 172.21.15.112:0/2246185057 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83d40035b0 con 0x7f83ec15af90 2023-08-27T11:25:18.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83eaffd700 1 -- 172.21.15.112:0/2246185057 >> v2:172.21.15.138:3300/0 conn(0x7f83ec071360 msgr2=0x7f83ec0717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:18.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83eaffd700 1 --2- 172.21.15.112:0/2246185057 >> v2:172.21.15.138:3300/0 conn(0x7f83ec071360 0x7f83ec0717d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83eaffd700 1 -- 172.21.15.112:0/2246185057 >> v1:172.21.15.138:6789/0 conn(0x7f83ec15af90 legacy=0x7f83ec0b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83eaffd700 1 -- 172.21.15.112:0/2246185057 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0b7150 con 0x7f83ec0d1a60 2023-08-27T11:25:18.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83ebfff700 1 --2- 172.21.15.112:0/2246185057 >> v2:172.21.15.138:3300/0 conn(0x7f83ec071360 0x7f83ec0717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:18.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83eaffd700 1 --2- 172.21.15.112:0/2246185057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83ec0d1a60 0x7f83ec15e470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009840 tx=0x7f83e0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40aceb272e523736 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:18.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83ea7fc700 1 -- 172.21.15.112:0/2246185057 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0011830 con 0x7f83ec0d1a60 2023-08-27T11:25:18.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83ea7fc700 1 -- 172.21.15.112:0/2246185057 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e0011990 con 0x7f83ec0d1a60 2023-08-27T11:25:18.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.346+0000 7f83ea7fc700 1 -- 172.21.15.112:0/2246185057 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0011d20 con 0x7f83ec0d1a60 2023-08-27T11:25:18.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.347+0000 7f83f2078700 1 -- 172.21.15.112:0/2246185057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83ec0d1a60 msgr2=0x7f83ec15e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.347+0000 7f83f2078700 1 --2- 172.21.15.112:0/2246185057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83ec0d1a60 0x7f83ec15e470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009840 tx=0x7f83e0009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:18.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.347+0000 7f83f2078700 1 -- 172.21.15.112:0/2246185057 shutdown_connections 2023-08-27T11:25:18.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.347+0000 7f83f2078700 1 --2- 172.21.15.112:0/2246185057 >> v2:172.21.15.138:3300/0 conn(0x7f83ec071360 0x7f83ec0717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.347+0000 7f83f2078700 1 --2- 172.21.15.112:0/2246185057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83ec0d1a60 0x7f83ec15e470 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.347+0000 7f83f2078700 1 -- 172.21.15.112:0/2246185057 >> 172.21.15.112:0/2246185057 conn(0x7f83ec066010 msgr2=0x7f83ec0663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:18.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.347+0000 7f83f2078700 1 -- 172.21.15.112:0/2246185057 shutdown_connections 2023-08-27T11:25:18.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.347+0000 7f83f2078700 1 -- 172.21.15.112:0/2246185057 wait complete. 2023-08-27T11:25:18.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.349+0000 7f83f2078700 1 Processor -- start 2023-08-27T11:25:18.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.349+0000 7f83f2078700 1 -- start start 2023-08-27T11:25:18.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83f2078700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83ec071360 0x7f83ec14b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:18.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83f2078700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83ec0d1a60 0x7f83ec148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:18.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83f2078700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83ec15af90 0x7f83ec149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:18.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83f2078700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec148bd0 con 0x7f83ec15af90 2023-08-27T11:25:18.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83f2078700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec148e30 con 0x7f83ec071360 2023-08-27T11:25:18.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83f2078700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec149110 con 0x7f83ec0d1a60 2023-08-27T11:25:18.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83eb7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83ec071360 0x7f83ec14b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:18.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83ebfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83ec15af90 0x7f83ec149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:18.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83eaffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83ec0d1a60 0x7f83ec148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:18.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83eb7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83ec071360 0x7f83ec14b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:59730/0 (socket says 172.21.15.112:59730) 2023-08-27T11:25:18.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83ebfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83ec15af90 0x7f83ec149ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59308/0 (socket says 172.21.15.112:59308) 2023-08-27T11:25:18.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83eb7fe700 1 -- 172.21.15.112:0/939176445 learned_addr learned my addr 172.21.15.112:0/939176445 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:18.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83eaffd700 1 -- 172.21.15.112:0/939176445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83ec071360 msgr2=0x7f83ec14b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83eaffd700 1 --2- 172.21.15.112:0/939176445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83ec071360 0x7f83ec14b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83eaffd700 1 -- 172.21.15.112:0/939176445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83ec15af90 msgr2=0x7f83ec149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83eaffd700 1 --2- 172.21.15.112:0/939176445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83ec15af90 0x7f83ec149ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83eaffd700 1 -- 172.21.15.112:0/939176445 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e4009a70 con 0x7f83ec0d1a60 2023-08-27T11:25:18.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83ebfff700 1 --2- 172.21.15.112:0/939176445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83ec15af90 0x7f83ec149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:18.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.350+0000 7f83eaffd700 1 --2- 172.21.15.112:0/939176445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83ec0d1a60 0x7f83ec148690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f83e0016000 tx=0x7f83e0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:18.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.351+0000 7f83d3fff700 1 -- 172.21.15.112:0/939176445 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0004550 con 0x7f83ec0d1a60 2023-08-27T11:25:18.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.351+0000 7f83d3fff700 1 -- 172.21.15.112:0/939176445 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e001e070 con 0x7f83ec0d1a60 2023-08-27T11:25:18.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.351+0000 7f83f2078700 1 -- 172.21.15.112:0/939176445 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e00094a0 con 0x7f83ec0d1a60 2023-08-27T11:25:18.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.351+0000 7f83d3fff700 1 -- 172.21.15.112:0/939176445 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00038e0 con 0x7f83ec0d1a60 2023-08-27T11:25:18.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.351+0000 7f83f2078700 1 -- 172.21.15.112:0/939176445 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec1496d0 con 0x7f83ec0d1a60 2023-08-27T11:25:18.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.352+0000 7f83f2078700 1 -- 172.21.15.112:0/939176445 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ec067a20 con 0x7f83ec0d1a60 2023-08-27T11:25:18.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.352+0000 7f83d3fff700 1 -- 172.21.15.112:0/939176445 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f83e0003a40 con 0x7f83ec0d1a60 2023-08-27T11:25:18.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.352+0000 7f83d3fff700 1 --2- 172.21.15.112:0/939176445 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f83d806a320 0x7f83d806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:18.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.352+0000 7f83d3fff700 1 -- 172.21.15.112:0/939176445 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f83e0089860 con 0x7f83ec0d1a60 2023-08-27T11:25:18.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.352+0000 7f83eb7fe700 1 --2- 172.21.15.112:0/939176445 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f83d806a320 0x7f83d806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:18.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.353+0000 7f83eb7fe700 1 --2- 172.21.15.112:0/939176445 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f83d806a320 0x7f83d806c7e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0011e0 tx=0x7f83dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:18.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.358+0000 7f83d3fff700 1 -- 172.21.15.112:0/939176445 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f83e0054c00 con 0x7f83ec0d1a60 2023-08-27T11:25:18.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.557+0000 7f83f2078700 1 -- 172.21.15.112:0/939176445 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83ec0d0500 con 0x7f83ec0d1a60 2023-08-27T11:25:18.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.557+0000 7f83d3fff700 1 -- 172.21.15.112:0/939176445 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83e0058220 con 0x7f83ec0d1a60 2023-08-27T11:25:18.559 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:25:18.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.559+0000 7f83f2078700 1 -- 172.21.15.112:0/939176445 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f83d806a320 msgr2=0x7f83d806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.559+0000 7f83f2078700 1 --2- 172.21.15.112:0/939176445 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f83d806a320 0x7f83d806c7e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0011e0 tx=0x7f83dc009040 comp rx=0 tx=0).stop 2023-08-27T11:25:18.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.559+0000 7f83f2078700 1 -- 172.21.15.112:0/939176445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83ec0d1a60 msgr2=0x7f83ec148690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.559+0000 7f83f2078700 1 --2- 172.21.15.112:0/939176445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83ec0d1a60 0x7f83ec148690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f83e0016000 tx=0x7f83e0004750 comp rx=0 tx=0).stop 2023-08-27T11:25:18.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.559+0000 7f83f2078700 1 -- 172.21.15.112:0/939176445 shutdown_connections 2023-08-27T11:25:18.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.559+0000 7f83f2078700 1 --2- 172.21.15.112:0/939176445 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f83d806a320 0x7f83d806c7e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.559+0000 7f83f2078700 1 --2- 172.21.15.112:0/939176445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83ec15af90 0x7f83ec149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.559+0000 7f83f2078700 1 --2- 172.21.15.112:0/939176445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83ec071360 0x7f83ec14b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.559+0000 7f83f2078700 1 --2- 172.21.15.112:0/939176445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83ec0d1a60 0x7f83ec148690 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.559+0000 7f83f2078700 1 -- 172.21.15.112:0/939176445 >> 172.21.15.112:0/939176445 conn(0x7f83ec066010 msgr2=0x7f83ec0cda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:18.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.560+0000 7f83f2078700 1 -- 172.21.15.112:0/939176445 shutdown_connections 2023-08-27T11:25:18.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.560+0000 7f83f2078700 1 -- 172.21.15.112:0/939176445 wait complete. 2023-08-27T11:25:18.573 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:25:18.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.700+0000 7fcb5f525700 1 Processor -- start 2023-08-27T11:25:18.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.700+0000 7fcb5f525700 1 -- start start 2023-08-27T11:25:18.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.701+0000 7fcb5f525700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcb58146c20 0x7fcb58160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:18.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.701+0000 7fcb5f525700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcb5806f790 0x7fcb5806fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:18.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.701+0000 7fcb5f525700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb580b5070 con 0x7fcb580b3440 2023-08-27T11:25:18.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.701+0000 7fcb5f525700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb58070180 con 0x7fcb58146c20 2023-08-27T11:25:18.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.701+0000 7fcb5f525700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb58147000 con 0x7fcb5806f790 2023-08-27T11:25:18.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.701+0000 7fcb5d2c1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcb58146c20 0x7fcb58160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.701+0000 7fcb5d2c1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcb58146c20 0x7fcb58160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59312/0 (socket says 172.21.15.112:59312) 2023-08-27T11:25:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.701+0000 7fcb5d2c1700 1 -- 172.21.15.112:0/2553456056 learned_addr learned my addr 172.21.15.112:0/2553456056 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.701+0000 7fcb5dac2700 1 --2- 172.21.15.112:0/2553456056 >> v2:172.21.15.138:3300/0 conn(0x7fcb5806f790 0x7fcb5806fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.701+0000 7fcb5d2c1700 1 -- 172.21.15.112:0/2553456056 >> v2:172.21.15.138:3300/0 conn(0x7fcb5806f790 msgr2=0x7fcb5806fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.701+0000 7fcb5d2c1700 1 --2- 172.21.15.112:0/2553456056 >> v2:172.21.15.138:3300/0 conn(0x7fcb5806f790 0x7fcb5806fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.701+0000 7fcb5d2c1700 1 -- 172.21.15.112:0/2553456056 >> v1:172.21.15.138:6789/0 conn(0x7fcb580b3440 legacy=0x7fcb5815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.702+0000 7fcb5d2c1700 1 -- 172.21.15.112:0/2553456056 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb581614a0 con 0x7fcb58146c20 2023-08-27T11:25:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.702+0000 7fcb5d2c1700 1 --2- 172.21.15.112:0/2553456056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb58146c20 0x7fcb58160f60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcb580b1c80 tx=0x7fcb54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fad8643d3dffa94 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.702+0000 7fcb4ffff700 1 -- 172.21.15.112:0/2553456056 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54011830 con 0x7fcb58146c20 2023-08-27T11:25:18.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.702+0000 7fcb4ffff700 1 -- 172.21.15.112:0/2553456056 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54011990 con 0x7fcb58146c20 2023-08-27T11:25:18.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.702+0000 7fcb4ffff700 1 -- 172.21.15.112:0/2553456056 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54011d20 con 0x7fcb58146c20 2023-08-27T11:25:18.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.703+0000 7fcb5f525700 1 -- 172.21.15.112:0/2553456056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb58146c20 msgr2=0x7fcb58160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.703+0000 7fcb5f525700 1 --2- 172.21.15.112:0/2553456056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb58146c20 0x7fcb58160f60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcb580b1c80 tx=0x7fcb54009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.703+0000 7fcb5f525700 1 -- 172.21.15.112:0/2553456056 shutdown_connections 2023-08-27T11:25:18.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.703+0000 7fcb5f525700 1 --2- 172.21.15.112:0/2553456056 >> v2:172.21.15.138:3300/0 conn(0x7fcb5806f790 0x7fcb5806fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.703+0000 7fcb5f525700 1 --2- 172.21.15.112:0/2553456056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb58146c20 0x7fcb58160f60 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.703+0000 7fcb5f525700 1 -- 172.21.15.112:0/2553456056 >> 172.21.15.112:0/2553456056 conn(0x7fcb580b1e90 msgr2=0x7fcb580b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:18.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.703+0000 7fcb5f525700 1 -- 172.21.15.112:0/2553456056 shutdown_connections 2023-08-27T11:25:18.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.703+0000 7fcb5f525700 1 -- 172.21.15.112:0/2553456056 wait complete. 2023-08-27T11:25:18.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.705+0000 7fcb5f525700 1 Processor -- start 2023-08-27T11:25:18.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.705+0000 7fcb5f525700 1 -- start start 2023-08-27T11:25:18.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5f525700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb5806f790 0x7fcb581f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:18.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5f525700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb580b3440 0x7fcb58070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:18.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5f525700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb58146c20 0x7fcb580ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:18.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5f525700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581455f0 con 0x7fcb5806f790 2023-08-27T11:25:18.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5f525700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb58144810 con 0x7fcb580b3440 2023-08-27T11:25:18.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5f525700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581f3920 con 0x7fcb58146c20 2023-08-27T11:25:18.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5d2c1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb5806f790 0x7fcb581f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:18.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5d2c1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb5806f790 0x7fcb581f33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59318/0 (socket says 172.21.15.112:59318) 2023-08-27T11:25:18.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5d2c1700 1 -- 172.21.15.112:0/2848130890 learned_addr learned my addr 172.21.15.112:0/2848130890 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:18.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5cac0700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb580b3440 0x7fcb58070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:18.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5dac2700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb58146c20 0x7fcb580ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:18.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5d2c1700 1 -- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb58146c20 msgr2=0x7fcb580ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5d2c1700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb58146c20 0x7fcb580ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5d2c1700 1 -- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb580b3440 msgr2=0x7fcb58070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5d2c1700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb580b3440 0x7fcb58070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5d2c1700 1 -- 172.21.15.112:0/2848130890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb54009550 con 0x7fcb5806f790 2023-08-27T11:25:18.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5d2c1700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb5806f790 0x7fcb581f33e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009c10 tx=0x7fcb54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:18.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb4dffb700 1 -- 172.21.15.112:0/2848130890 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54004510 con 0x7fcb5806f790 2023-08-27T11:25:18.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb4dffb700 1 -- 172.21.15.112:0/2848130890 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5401e070 con 0x7fcb5806f790 2023-08-27T11:25:18.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb5f525700 1 -- 172.21.15.112:0/2848130890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb58143c00 con 0x7fcb5806f790 2023-08-27T11:25:18.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.706+0000 7fcb4dffb700 1 -- 172.21.15.112:0/2848130890 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540038e0 con 0x7fcb5806f790 2023-08-27T11:25:18.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.707+0000 7fcb5f525700 1 -- 172.21.15.112:0/2848130890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb581441c0 con 0x7fcb5806f790 2023-08-27T11:25:18.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.708+0000 7fcb5f525700 1 -- 172.21.15.112:0/2848130890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb58067e90 con 0x7fcb5806f790 2023-08-27T11:25:18.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.708+0000 7fcb4dffb700 1 -- 172.21.15.112:0/2848130890 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fcb54003a40 con 0x7fcb5806f790 2023-08-27T11:25:18.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.709+0000 7fcb4dffb700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcb3c06a320 0x7fcb3c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:18.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.709+0000 7fcb4dffb700 1 -- 172.21.15.112:0/2848130890 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb54089640 con 0x7fcb5806f790 2023-08-27T11:25:18.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.709+0000 7fcb5cac0700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcb3c06a320 0x7fcb3c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:18.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.709+0000 7fcb5cac0700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcb3c06a320 0x7fcb3c06c7e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb581f5eb0 tx=0x7fcb48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:18.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.715+0000 7fcb4dffb700 1 -- 172.21.15.112:0/2848130890 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcb54054ab0 con 0x7fcb5806f790 2023-08-27T11:25:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.954+0000 7fcb5f525700 1 -- 172.21.15.112:0/2848130890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcb58067a20 con 0x7fcb5806f790 2023-08-27T11:25:18.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.954+0000 7fcb4dffb700 1 -- 172.21.15.112:0/2848130890 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcb540580d0 con 0x7fcb5806f790 2023-08-27T11:25:18.956 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:25:18.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.957+0000 7fcb5f525700 1 -- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcb3c06a320 msgr2=0x7fcb3c06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.957+0000 7fcb5f525700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcb3c06a320 0x7fcb3c06c7e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb581f5eb0 tx=0x7fcb48009040 comp rx=0 tx=0).stop 2023-08-27T11:25:18.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.957+0000 7fcb5f525700 1 -- 172.21.15.112:0/2848130890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb5806f790 msgr2=0x7fcb581f33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:18.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.957+0000 7fcb5f525700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb5806f790 0x7fcb581f33e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009c10 tx=0x7fcb54004750 comp rx=0 tx=0).stop 2023-08-27T11:25:18.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.957+0000 7fcb5f525700 1 -- 172.21.15.112:0/2848130890 shutdown_connections 2023-08-27T11:25:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.957+0000 7fcb5f525700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcb3c06a320 0x7fcb3c06c7e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.957+0000 7fcb5f525700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb58146c20 0x7fcb580ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.957+0000 7fcb5f525700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb5806f790 0x7fcb581f33e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.957+0000 7fcb5f525700 1 --2- 172.21.15.112:0/2848130890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb580b3440 0x7fcb58070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.957+0000 7fcb5f525700 1 -- 172.21.15.112:0/2848130890 >> 172.21.15.112:0/2848130890 conn(0x7fcb580b1e90 msgr2=0x7fcb58071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.957+0000 7fcb5f525700 1 -- 172.21.15.112:0/2848130890 shutdown_connections 2023-08-27T11:25:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:18.957+0000 7fcb5f525700 1 -- 172.21.15.112:0/2848130890 wait complete. 2023-08-27T11:25:18.974 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:25:19.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.101+0000 7ff5fce43700 1 Processor -- start 2023-08-27T11:25:19.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.101+0000 7ff5fce43700 1 -- start start 2023-08-27T11:25:19.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.102+0000 7ff5fce43700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff5f8146bf0 0x7ff5f80b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:19.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.102+0000 7ff5fce43700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f80d1500 con 0x7ff5f8146fd0 2023-08-27T11:25:19.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.102+0000 7ff5fce43700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f80d0460 con 0x7ff5f8070eb0 2023-08-27T11:25:19.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.102+0000 7ff5fce43700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f80d0710 con 0x7ff5f8146bf0 2023-08-27T11:25:19.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.102+0000 7ff5f659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff5f8146bf0 0x7ff5f80b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:19.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.102+0000 7ff5f5d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff5f8146fd0 0x7ff5f80cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53710/0 (socket says 172.21.15.112:53710) 2023-08-27T11:25:19.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.102+0000 7ff5f5d9b700 1 -- 172.21.15.112:0/242513239 learned_addr learned my addr 172.21.15.112:0/242513239 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:19.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.103+0000 7ff5f659c700 1 -- 172.21.15.112:0/242513239 >> v1:172.21.15.138:6789/0 conn(0x7ff5f8070eb0 legacy=0x7ff5f80d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:19.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.103+0000 7ff5f659c700 1 -- 172.21.15.112:0/242513239 >> v1:172.21.15.112:6789/0 conn(0x7ff5f8146fd0 legacy=0x7ff5f80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:19.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.103+0000 7ff5f559a700 1 -- 172.21.15.112:0/242513239 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071392266 0 0) 0x7ff5f80d1500 con 0x7ff5f8146fd0 2023-08-27T11:25:19.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.103+0000 7ff5f659c700 1 -- 172.21.15.112:0/242513239 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f80d09f0 con 0x7ff5f8146bf0 2023-08-27T11:25:19.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.103+0000 7ff5f659c700 1 --2- 172.21.15.112:0/242513239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8146bf0 0x7ff5f80b5db0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009840 tx=0x7ff5e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79499d59baf81b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:19.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.103+0000 7ff5f559a700 1 -- 172.21.15.112:0/242513239 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8004230 con 0x7ff5f8146bf0 2023-08-27T11:25:19.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.103+0000 7ff5f559a700 1 -- 172.21.15.112:0/242513239 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e8004390 con 0x7ff5f8146bf0 2023-08-27T11:25:19.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.103+0000 7ff5f559a700 1 -- 172.21.15.112:0/242513239 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80036a0 con 0x7ff5f8146bf0 2023-08-27T11:25:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.103+0000 7ff5fce43700 1 -- 172.21.15.112:0/242513239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8146bf0 msgr2=0x7ff5f80b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.103+0000 7ff5fce43700 1 --2- 172.21.15.112:0/242513239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8146bf0 0x7ff5f80b5db0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009840 tx=0x7ff5e8009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.104+0000 7ff5fce43700 1 -- 172.21.15.112:0/242513239 shutdown_connections 2023-08-27T11:25:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.104+0000 7ff5fce43700 1 --2- 172.21.15.112:0/242513239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8146bf0 0x7ff5f80b5db0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.104+0000 7ff5fce43700 1 -- 172.21.15.112:0/242513239 >> 172.21.15.112:0/242513239 conn(0x7ff5f806c450 msgr2=0x7ff5f806f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.104+0000 7ff5fce43700 1 -- 172.21.15.112:0/242513239 shutdown_connections 2023-08-27T11:25:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.104+0000 7ff5fce43700 1 -- 172.21.15.112:0/242513239 wait complete. 2023-08-27T11:25:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.105+0000 7ff5fce43700 1 Processor -- start 2023-08-27T11:25:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.105+0000 7ff5fce43700 1 -- start start 2023-08-27T11:25:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.106+0000 7ff5fce43700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f8070eb0 0x7ff5f80b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.106+0000 7ff5fce43700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f8146bf0 0x7ff5f8159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.106+0000 7ff5fce43700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8146fd0 0x7ff5f8141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.106+0000 7ff5fce43700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f80b9470 con 0x7ff5f8146fd0 2023-08-27T11:25:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.106+0000 7ff5fce43700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f80b8700 con 0x7ff5f8146bf0 2023-08-27T11:25:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.106+0000 7ff5fce43700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f80b7940 con 0x7ff5f8070eb0 2023-08-27T11:25:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.106+0000 7ff5f6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8146fd0 0x7ff5f8141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.106+0000 7ff5f5d9b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f8146bf0 0x7ff5f8159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.106+0000 7ff5f6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8146fd0 0x7ff5f8141870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59336/0 (socket says 172.21.15.112:59336) 2023-08-27T11:25:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.106+0000 7ff5f659c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f8070eb0 0x7ff5f80b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.106+0000 7ff5f6d9d700 1 -- 172.21.15.112:0/624291107 learned_addr learned my addr 172.21.15.112:0/624291107 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.107+0000 7ff5f6d9d700 1 -- 172.21.15.112:0/624291107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f8070eb0 msgr2=0x7ff5f80b8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.107+0000 7ff5f6d9d700 1 --2- 172.21.15.112:0/624291107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f8070eb0 0x7ff5f80b8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.107+0000 7ff5f6d9d700 1 -- 172.21.15.112:0/624291107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f8146bf0 msgr2=0x7ff5f8159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.107+0000 7ff5f6d9d700 1 --2- 172.21.15.112:0/624291107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f8146bf0 0x7ff5f8159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.107+0000 7ff5f6d9d700 1 -- 172.21.15.112:0/624291107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e80094a0 con 0x7ff5f8146fd0 2023-08-27T11:25:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.107+0000 7ff5f5d9b700 1 --2- 172.21.15.112:0/624291107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f8146bf0 0x7ff5f8159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.107+0000 7ff5f6d9d700 1 --2- 172.21.15.112:0/624291107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8146fd0 0x7ff5f8141870 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000cc10 tx=0x7ff5f000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.107+0000 7ff5e6ffd700 1 -- 172.21.15.112:0/624291107 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f000aa60 con 0x7ff5f8146fd0 2023-08-27T11:25:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.107+0000 7ff5e6ffd700 1 -- 172.21.15.112:0/624291107 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f000abc0 con 0x7ff5f8146fd0 2023-08-27T11:25:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.107+0000 7ff5fce43700 1 -- 172.21.15.112:0/624291107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f80b7c20 con 0x7ff5f8146fd0 2023-08-27T11:25:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.107+0000 7ff5e6ffd700 1 -- 172.21.15.112:0/624291107 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0012420 con 0x7ff5f8146fd0 2023-08-27T11:25:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.107+0000 7ff5fce43700 1 -- 172.21.15.112:0/624291107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f80b81e0 con 0x7ff5f8146fd0 2023-08-27T11:25:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.108+0000 7ff5fce43700 1 -- 172.21.15.112:0/624291107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f8067e90 con 0x7ff5f8146fd0 2023-08-27T11:25:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.108+0000 7ff5e6ffd700 1 -- 172.21.15.112:0/624291107 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7ff5f0012580 con 0x7ff5f8146fd0 2023-08-27T11:25:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.109+0000 7ff5e6ffd700 1 --2- 172.21.15.112:0/624291107 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff5d806a320 0x7ff5d806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.109+0000 7ff5e6ffd700 1 -- 172.21.15.112:0/624291107 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff5f0088ba0 con 0x7ff5f8146fd0 2023-08-27T11:25:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.109+0000 7ff5f659c700 1 --2- 172.21.15.112:0/624291107 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff5d806a320 0x7ff5d806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.110+0000 7ff5f659c700 1 --2- 172.21.15.112:0/624291107 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff5d806a320 0x7ff5d806c7e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80090b0 tx=0x7ff5e801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:19.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.113+0000 7ff5e6ffd700 1 -- 172.21.15.112:0/624291107 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff5f0053fc0 con 0x7ff5f8146fd0 2023-08-27T11:25:19.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.315+0000 7ff5fce43700 1 -- 172.21.15.112:0/624291107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5f8067a20 con 0x7ff5f8146fd0 2023-08-27T11:25:19.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.316+0000 7ff5e6ffd700 1 -- 172.21.15.112:0/624291107 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5f00575e0 con 0x7ff5f8146fd0 2023-08-27T11:25:19.318 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:25:19.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.318+0000 7ff5fce43700 1 -- 172.21.15.112:0/624291107 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff5d806a320 msgr2=0x7ff5d806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:19.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.318+0000 7ff5fce43700 1 --2- 172.21.15.112:0/624291107 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff5d806a320 0x7ff5d806c7e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80090b0 tx=0x7ff5e801c040 comp rx=0 tx=0).stop 2023-08-27T11:25:19.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.318+0000 7ff5fce43700 1 -- 172.21.15.112:0/624291107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8146fd0 msgr2=0x7ff5f8141870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:19.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.318+0000 7ff5fce43700 1 --2- 172.21.15.112:0/624291107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8146fd0 0x7ff5f8141870 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000cc10 tx=0x7ff5f000cf20 comp rx=0 tx=0).stop 2023-08-27T11:25:19.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.318+0000 7ff5fce43700 1 -- 172.21.15.112:0/624291107 shutdown_connections 2023-08-27T11:25:19.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.318+0000 7ff5fce43700 1 --2- 172.21.15.112:0/624291107 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff5d806a320 0x7ff5d806c7e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.318+0000 7ff5fce43700 1 --2- 172.21.15.112:0/624291107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8146fd0 0x7ff5f8141870 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.318+0000 7ff5fce43700 1 --2- 172.21.15.112:0/624291107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f8070eb0 0x7ff5f80b8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.318+0000 7ff5fce43700 1 --2- 172.21.15.112:0/624291107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f8146bf0 0x7ff5f8159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.318+0000 7ff5fce43700 1 -- 172.21.15.112:0/624291107 >> 172.21.15.112:0/624291107 conn(0x7ff5f806c450 msgr2=0x7ff5f806f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:19.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.318+0000 7ff5fce43700 1 -- 172.21.15.112:0/624291107 shutdown_connections 2023-08-27T11:25:19.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.318+0000 7ff5fce43700 1 -- 172.21.15.112:0/624291107 wait complete. 2023-08-27T11:25:19.335 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:25:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.461+0000 7f9269c54700 1 Processor -- start 2023-08-27T11:25:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.461+0000 7f9269c54700 1 -- start start 2023-08-27T11:25:19.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.462+0000 7f9269c54700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9264160a00 0x7f92640b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:19.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.462+0000 7f9269c54700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9264146990 0x7f926415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:19.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.462+0000 7f9269c54700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926415e140 con 0x7f926406d510 2023-08-27T11:25:19.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.462+0000 7f9269c54700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926406f930 con 0x7f9264160a00 2023-08-27T11:25:19.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.462+0000 7f9269c54700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926406fc10 con 0x7f9264146990 2023-08-27T11:25:19.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.462+0000 7f92637fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9264160a00 0x7f92640b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:19.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.462+0000 7f925affd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9264146990 0x7f926415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:19.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.462+0000 7f9263fff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f926406d510 0x7f9264161400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:33018/0 (socket says 172.21.15.112:33018) 2023-08-27T11:25:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.462+0000 7f9263fff700 1 -- 172.21.15.112:0/3505994722 learned_addr learned my addr 172.21.15.112:0/3505994722 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.463+0000 7f92637fe700 1 -- 172.21.15.112:0/3505994722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9264146990 msgr2=0x7f926415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.463+0000 7f92637fe700 1 --2- 172.21.15.112:0/3505994722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9264146990 0x7f926415dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.463+0000 7f92637fe700 1 -- 172.21.15.112:0/3505994722 >> v1:172.21.15.138:6789/0 conn(0x7f926406d510 legacy=0x7f9264161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.463+0000 7f92637fe700 1 -- 172.21.15.112:0/3505994722 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926406fef0 con 0x7f9264160a00 2023-08-27T11:25:19.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.463+0000 7f9262ffd700 1 -- 172.21.15.112:0/3505994722 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764908669 0 0) 0x7f926415e140 con 0x7f926406d510 2023-08-27T11:25:19.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.463+0000 7f925affd700 1 --2- 172.21.15.112:0/3505994722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9264146990 0x7f926415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:19.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.463+0000 7f92637fe700 1 --2- 172.21.15.112:0/3505994722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9264160a00 0x7f92640b46d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9254009840 tx=0x7f9254009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a6b2cd89aee6ee4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:19.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.463+0000 7f9262ffd700 1 -- 172.21.15.112:0/3505994722 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254011830 con 0x7f9264160a00 2023-08-27T11:25:19.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.463+0000 7f9262ffd700 1 -- 172.21.15.112:0/3505994722 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9254011990 con 0x7f9264160a00 2023-08-27T11:25:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.463+0000 7f9262ffd700 1 -- 172.21.15.112:0/3505994722 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254011d20 con 0x7f9264160a00 2023-08-27T11:25:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.464+0000 7f9269c54700 1 -- 172.21.15.112:0/3505994722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9264160a00 msgr2=0x7f92640b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.464+0000 7f9269c54700 1 --2- 172.21.15.112:0/3505994722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9264160a00 0x7f92640b46d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9254009840 tx=0x7f9254009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.464+0000 7f9269c54700 1 -- 172.21.15.112:0/3505994722 shutdown_connections 2023-08-27T11:25:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.464+0000 7f9269c54700 1 --2- 172.21.15.112:0/3505994722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9264160a00 0x7f92640b46d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.464+0000 7f9269c54700 1 --2- 172.21.15.112:0/3505994722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9264146990 0x7f926415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.464+0000 7f9269c54700 1 -- 172.21.15.112:0/3505994722 >> 172.21.15.112:0/3505994722 conn(0x7f92640cfe50 msgr2=0x7f92640d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.464+0000 7f9269c54700 1 -- 172.21.15.112:0/3505994722 shutdown_connections 2023-08-27T11:25:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.464+0000 7f9269c54700 1 -- 172.21.15.112:0/3505994722 wait complete. 2023-08-27T11:25:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.465+0000 7f9269c54700 1 Processor -- start 2023-08-27T11:25:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.465+0000 7f9269c54700 1 -- start start 2023-08-27T11:25:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.466+0000 7f9269c54700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f926406d510 0x7f92641e6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.466+0000 7f9269c54700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9264146990 0x7f92640b8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.466+0000 7f9269c54700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9264160a00 0x7f92640afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.466+0000 7f9269c54700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92640b8e60 con 0x7f926406d510 2023-08-27T11:25:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.466+0000 7f9269c54700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9264139880 con 0x7f9264160a00 2023-08-27T11:25:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.466+0000 7f9269c54700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9264133820 con 0x7f9264146990 2023-08-27T11:25:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f92637fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f926406d510 0x7f92641e6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:19.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f92637fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f926406d510 0x7f92641e6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59346/0 (socket says 172.21.15.112:59346) 2023-08-27T11:25:19.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f92637fe700 1 -- 172.21.15.112:0/1091941033 learned_addr learned my addr 172.21.15.112:0/1091941033 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:19.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f925affd700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9264146990 0x7f92640b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:19.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f9263fff700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9264160a00 0x7f92640afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:19.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f92637fe700 1 -- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9264146990 msgr2=0x7f92640b8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:19.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f92637fe700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9264146990 0x7f92640b8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f92637fe700 1 -- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9264160a00 msgr2=0x7f92640afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f92637fe700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9264160a00 0x7f92640afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f92637fe700 1 -- 172.21.15.112:0/1091941033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c0094a0 con 0x7f926406d510 2023-08-27T11:25:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f92637fe700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f926406d510 0x7f92641e6080 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9254009fd0 tx=0x7f9254004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f9260ff9700 1 -- 172.21.15.112:0/1091941033 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92540036a0 con 0x7f926406d510 2023-08-27T11:25:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f9269c54700 1 -- 172.21.15.112:0/1091941033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92540094a0 con 0x7f926406d510 2023-08-27T11:25:19.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f9260ff9700 1 -- 172.21.15.112:0/1091941033 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925401e070 con 0x7f926406d510 2023-08-27T11:25:19.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f9260ff9700 1 -- 172.21.15.112:0/1091941033 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254003a60 con 0x7f926406d510 2023-08-27T11:25:19.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.467+0000 7f9269c54700 1 -- 172.21.15.112:0/1091941033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92641e6940 con 0x7f926406d510 2023-08-27T11:25:19.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.468+0000 7f9269c54700 1 -- 172.21.15.112:0/1091941033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9264067a20 con 0x7f926406d510 2023-08-27T11:25:19.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.469+0000 7f9260ff9700 1 -- 172.21.15.112:0/1091941033 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f9254003bc0 con 0x7f926406d510 2023-08-27T11:25:19.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.469+0000 7f9260ff9700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f924406a320 0x7f924406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.469+0000 7f9260ff9700 1 -- 172.21.15.112:0/1091941033 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f92540898d0 con 0x7f926406d510 2023-08-27T11:25:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.469+0000 7f925affd700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f924406a320 0x7f924406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.469+0000 7f925affd700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f924406a320 0x7f924406c7e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f92641e73c0 tx=0x7f924c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.473+0000 7f9260ff9700 1 -- 172.21.15.112:0/1091941033 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9254054c70 con 0x7f926406d510 2023-08-27T11:25:19.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.711+0000 7f9269c54700 1 -- 172.21.15.112:0/1091941033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f92640af9a0 con 0x7f926406d510 2023-08-27T11:25:19.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.711+0000 7f9260ff9700 1 -- 172.21.15.112:0/1091941033 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9254058290 con 0x7f926406d510 2023-08-27T11:25:19.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.713+0000 7f9269c54700 1 -- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f924406a320 msgr2=0x7f924406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:19.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.713+0000 7f9269c54700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f924406a320 0x7f924406c7e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f92641e73c0 tx=0x7f924c009210 comp rx=0 tx=0).stop 2023-08-27T11:25:19.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.713+0000 7f9269c54700 1 -- 172.21.15.112:0/1091941033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f926406d510 msgr2=0x7f92641e6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:19.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.713+0000 7f9269c54700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f926406d510 0x7f92641e6080 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9254009fd0 tx=0x7f9254004750 comp rx=0 tx=0).stop 2023-08-27T11:25:19.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.713+0000 7f9269c54700 1 -- 172.21.15.112:0/1091941033 shutdown_connections 2023-08-27T11:25:19.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.713+0000 7f9269c54700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f924406a320 0x7f924406c7e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.713+0000 7f9269c54700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9264160a00 0x7f92640afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.713+0000 7f9269c54700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f926406d510 0x7f92641e6080 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.713+0000 7f9269c54700 1 --2- 172.21.15.112:0/1091941033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9264146990 0x7f92640b8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:19.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.713+0000 7f9269c54700 1 -- 172.21.15.112:0/1091941033 >> 172.21.15.112:0/1091941033 conn(0x7f92640cfe50 msgr2=0x7f926415e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:19.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.713+0000 7f9269c54700 1 -- 172.21.15.112:0/1091941033 shutdown_connections 2023-08-27T11:25:19.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:19.713+0000 7f9269c54700 1 -- 172.21.15.112:0/1091941033 wait complete. 2023-08-27T11:25:19.717 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:25:19.727 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:25:19.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:19.728 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:25:20.452 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:25:20.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:20.453 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:25:20.471 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:25:20.471 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:20.489 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:25:20.489 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:20.546 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:20.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.692+0000 7fed9a540700 1 Processor -- start 2023-08-27T11:25:20.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.692+0000 7fed9a540700 1 -- start start 2023-08-27T11:25:20.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed9a540700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fed940ce260 0x7fed9406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed9a540700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9406ff70 0x7fed9406cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed9a540700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9406cb00 con 0x7fed9413f910 2023-08-27T11:25:20.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed9a540700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed940707a0 con 0x7fed940ce260 2023-08-27T11:25:20.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed9a540700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed94070a80 con 0x7fed9406ff70 2023-08-27T11:25:20.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed937fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fed940ce260 0x7fed9406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed937fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fed940ce260 0x7fed9406fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59356/0 (socket says 172.21.15.112:59356) 2023-08-27T11:25:20.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed937fe700 1 -- 172.21.15.112:0/2449721088 learned_addr learned my addr 172.21.15.112:0/2449721088 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:20.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed98add700 1 --2- 172.21.15.112:0/2449721088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9406ff70 0x7fed9406cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed937fe700 1 -- 172.21.15.112:0/2449721088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9406ff70 msgr2=0x7fed9406cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:20.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed937fe700 1 --2- 172.21.15.112:0/2449721088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9406ff70 0x7fed9406cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed937fe700 1 -- 172.21.15.112:0/2449721088 >> v1:172.21.15.112:6789/0 conn(0x7fed9413f910 legacy=0x7fed940cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed937fe700 1 -- 172.21.15.112:0/2449721088 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed94070d60 con 0x7fed940ce260 2023-08-27T11:25:20.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed92ffd700 1 -- 172.21.15.112:0/2449721088 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116738622 0 0) 0x7fed9406cb00 con 0x7fed9413f910 2023-08-27T11:25:20.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.693+0000 7fed937fe700 1 --2- 172.21.15.112:0/2449721088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed940ce260 0x7fed9406fa30 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fed8800bb90 tx=0x7fed8800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe8b42b2d787496 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:20.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.694+0000 7fed92ffd700 1 -- 172.21.15.112:0/2449721088 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8800dc20 con 0x7fed940ce260 2023-08-27T11:25:20.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.694+0000 7fed92ffd700 1 -- 172.21.15.112:0/2449721088 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8800dd80 con 0x7fed940ce260 2023-08-27T11:25:20.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.694+0000 7fed92ffd700 1 -- 172.21.15.112:0/2449721088 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88017630 con 0x7fed940ce260 2023-08-27T11:25:20.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.694+0000 7fed9a540700 1 -- 172.21.15.112:0/2449721088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed940ce260 msgr2=0x7fed9406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.694+0000 7fed9a540700 1 --2- 172.21.15.112:0/2449721088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed940ce260 0x7fed9406fa30 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fed8800bb90 tx=0x7fed8800bea0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.695+0000 7fed9a540700 1 -- 172.21.15.112:0/2449721088 shutdown_connections 2023-08-27T11:25:20.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.695+0000 7fed9a540700 1 --2- 172.21.15.112:0/2449721088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9406ff70 0x7fed9406cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.695+0000 7fed9a540700 1 --2- 172.21.15.112:0/2449721088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed940ce260 0x7fed9406fa30 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.695+0000 7fed9a540700 1 -- 172.21.15.112:0/2449721088 >> 172.21.15.112:0/2449721088 conn(0x7fed940b0bf0 msgr2=0x7fed940b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:20.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.695+0000 7fed9a540700 1 -- 172.21.15.112:0/2449721088 shutdown_connections 2023-08-27T11:25:20.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.695+0000 7fed9a540700 1 -- 172.21.15.112:0/2449721088 wait complete. 2023-08-27T11:25:20.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.696+0000 7fed9a540700 1 Processor -- start 2023-08-27T11:25:20.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.696+0000 7fed9a540700 1 -- start start 2023-08-27T11:25:20.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.697+0000 7fed9a540700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9406ff70 0x7fed9415b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.697+0000 7fed9a540700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed940ce260 0x7fed94158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.697+0000 7fed9a540700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed9413f910 0x7fed940c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.697+0000 7fed9a540700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9415bda0 con 0x7fed940ce260 2023-08-27T11:25:20.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.697+0000 7fed9a540700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9415c080 con 0x7fed9413f910 2023-08-27T11:25:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.697+0000 7fed9a540700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9415c360 con 0x7fed9406ff70 2023-08-27T11:25:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.697+0000 7fed937fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed940ce260 0x7fed94158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.697+0000 7fed937fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed940ce260 0x7fed94158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59362/0 (socket says 172.21.15.112:59362) 2023-08-27T11:25:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.697+0000 7fed937fe700 1 -- 172.21.15.112:0/1902223366 learned_addr learned my addr 172.21.15.112:0/1902223366 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.697+0000 7fed93fff700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9406ff70 0x7fed9415b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.697+0000 7fed98add700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed9413f910 0x7fed940c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.698+0000 7fed937fe700 1 -- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9406ff70 msgr2=0x7fed9415b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.698+0000 7fed937fe700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9406ff70 0x7fed9415b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.698+0000 7fed937fe700 1 -- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed9413f910 msgr2=0x7fed940c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.698+0000 7fed937fe700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed9413f910 0x7fed940c2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.698+0000 7fed937fe700 1 -- 172.21.15.112:0/1902223366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8800b7f0 con 0x7fed940ce260 2023-08-27T11:25:20.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.698+0000 7fed937fe700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed940ce260 0x7fed94158e00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fed88019040 tx=0x7fed88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:20.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.698+0000 7fed90ff9700 1 -- 172.21.15.112:0/1902223366 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88009050 con 0x7fed940ce260 2023-08-27T11:25:20.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.698+0000 7fed90ff9700 1 -- 172.21.15.112:0/1902223366 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8800a040 con 0x7fed940ce260 2023-08-27T11:25:20.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.698+0000 7fed90ff9700 1 -- 172.21.15.112:0/1902223366 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88017c20 con 0x7fed940ce260 2023-08-27T11:25:20.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.698+0000 7fed9a540700 1 -- 172.21.15.112:0/1902223366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9415c640 con 0x7fed940ce260 2023-08-27T11:25:20.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.698+0000 7fed9a540700 1 -- 172.21.15.112:0/1902223366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9415ba20 con 0x7fed940ce260 2023-08-27T11:25:20.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.699+0000 7fed9a540700 1 -- 172.21.15.112:0/1902223366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed9415cb60 con 0x7fed940ce260 2023-08-27T11:25:20.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.700+0000 7fed90ff9700 1 -- 172.21.15.112:0/1902223366 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fed88015070 con 0x7fed940ce260 2023-08-27T11:25:20.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.700+0000 7fed90ff9700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fed8006a320 0x7fed8006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.700+0000 7fed90ff9700 1 -- 172.21.15.112:0/1902223366 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fed88089ef0 con 0x7fed940ce260 2023-08-27T11:25:20.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.700+0000 7fed93fff700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fed8006a320 0x7fed8006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.701+0000 7fed93fff700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fed8006a320 0x7fed8006c7e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fed84007fd0 tx=0x7fed84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:20.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.705+0000 7fed90ff9700 1 -- 172.21.15.112:0/1902223366 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fed9415cb60 con 0x7fed940ce260 2023-08-27T11:25:20.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.931+0000 7fed9a540700 1 -- 172.21.15.112:0/1902223366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed9415cb60 con 0x7fed940ce260 2023-08-27T11:25:20.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.932+0000 7fed90ff9700 1 -- 172.21.15.112:0/1902223366 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18320 (secure 0 0 0) 0x7fed9415cb60 con 0x7fed940ce260 2023-08-27T11:25:20.934 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:20.934 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":65,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:19:17.239518+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:19:14.179813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-27T11:19:15.021524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:20.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.934+0000 7fed9a540700 1 -- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fed8006a320 msgr2=0x7fed8006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.934+0000 7fed9a540700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fed8006a320 0x7fed8006c7e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fed84007fd0 tx=0x7fed84009040 comp rx=0 tx=0).stop 2023-08-27T11:25:20.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.934+0000 7fed9a540700 1 -- 172.21.15.112:0/1902223366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed940ce260 msgr2=0x7fed94158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.934+0000 7fed9a540700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed940ce260 0x7fed94158e00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fed88019040 tx=0x7fed88004750 comp rx=0 tx=0).stop 2023-08-27T11:25:20.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.934+0000 7fed9a540700 1 -- 172.21.15.112:0/1902223366 shutdown_connections 2023-08-27T11:25:20.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.934+0000 7fed9a540700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fed8006a320 0x7fed8006c7e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.934+0000 7fed9a540700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed9413f910 0x7fed940c2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.934+0000 7fed9a540700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9406ff70 0x7fed9415b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.934+0000 7fed9a540700 1 --2- 172.21.15.112:0/1902223366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed940ce260 0x7fed94158e00 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.934+0000 7fed9a540700 1 -- 172.21.15.112:0/1902223366 >> 172.21.15.112:0/1902223366 conn(0x7fed940b0bf0 msgr2=0x7fed940af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:20.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.934+0000 7fed9a540700 1 -- 172.21.15.112:0/1902223366 shutdown_connections 2023-08-27T11:25:20.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:20.934+0000 7fed9a540700 1 -- 172.21.15.112:0/1902223366 wait complete. 2023-08-27T11:25:20.949 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:19:14.179813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:19:15.021524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:25:20.949 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:25:21.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.076+0000 7f5c9a825700 1 Processor -- start 2023-08-27T11:25:21.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.076+0000 7f5c9a825700 1 -- start start 2023-08-27T11:25:21.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c9a825700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c9415af90 0x7f5c9415e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c9a825700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c940d1a60 0x7f5c940b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c9a825700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9415b370 con 0x7f5c94071360 2023-08-27T11:25:21.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c9a825700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c940b6bc0 con 0x7f5c9415af90 2023-08-27T11:25:21.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c9a825700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c940b6e70 con 0x7f5c940d1a60 2023-08-27T11:25:21.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c93fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c9415af90 0x7f5c9415e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c937fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c940d1a60 0x7f5c940b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c93fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c9415af90 0x7f5c9415e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59368/0 (socket says 172.21.15.112:59368) 2023-08-27T11:25:21.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c98dc2700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5c94071360 0x7f5c940717b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53754/0 (socket says 172.21.15.112:53754) 2023-08-27T11:25:21.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c93fff700 1 -- 172.21.15.112:0/2063585827 learned_addr learned my addr 172.21.15.112:0/2063585827 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:21.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c937fe700 1 -- 172.21.15.112:0/2063585827 >> v2:172.21.15.112:3300/0 conn(0x7f5c9415af90 msgr2=0x7f5c9415e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c937fe700 1 --2- 172.21.15.112:0/2063585827 >> v2:172.21.15.112:3300/0 conn(0x7f5c9415af90 0x7f5c9415e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c937fe700 1 -- 172.21.15.112:0/2063585827 >> v1:172.21.15.112:6789/0 conn(0x7f5c94071360 legacy=0x7f5c940717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c937fe700 1 -- 172.21.15.112:0/2063585827 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c940b7150 con 0x7f5c940d1a60 2023-08-27T11:25:21.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c93fff700 1 --2- 172.21.15.112:0/2063585827 >> v2:172.21.15.112:3300/0 conn(0x7f5c9415af90 0x7f5c9415e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:21.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c937fe700 1 --2- 172.21.15.112:0/2063585827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c940d1a60 0x7f5c940b6680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009840 tx=0x7f5c88009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f9e0a91a3193d4b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.077+0000 7f5c92ffd700 1 -- 172.21.15.112:0/2063585827 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c88011830 con 0x7f5c940d1a60 2023-08-27T11:25:21.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.078+0000 7f5c92ffd700 1 -- 172.21.15.112:0/2063585827 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c88011990 con 0x7f5c940d1a60 2023-08-27T11:25:21.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.078+0000 7f5c92ffd700 1 -- 172.21.15.112:0/2063585827 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c88011d20 con 0x7f5c940d1a60 2023-08-27T11:25:21.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.078+0000 7f5c9a825700 1 -- 172.21.15.112:0/2063585827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c940d1a60 msgr2=0x7f5c940b6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.078+0000 7f5c9a825700 1 --2- 172.21.15.112:0/2063585827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c940d1a60 0x7f5c940b6680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009840 tx=0x7f5c88009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:21.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.078+0000 7f5c9a825700 1 -- 172.21.15.112:0/2063585827 shutdown_connections 2023-08-27T11:25:21.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.078+0000 7f5c9a825700 1 --2- 172.21.15.112:0/2063585827 >> v2:172.21.15.112:3300/0 conn(0x7f5c9415af90 0x7f5c9415e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.078+0000 7f5c9a825700 1 --2- 172.21.15.112:0/2063585827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c940d1a60 0x7f5c940b6680 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.078+0000 7f5c9a825700 1 -- 172.21.15.112:0/2063585827 >> 172.21.15.112:0/2063585827 conn(0x7f5c94066010 msgr2=0x7f5c940663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:21.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.078+0000 7f5c9a825700 1 -- 172.21.15.112:0/2063585827 shutdown_connections 2023-08-27T11:25:21.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.078+0000 7f5c9a825700 1 -- 172.21.15.112:0/2063585827 wait complete. 2023-08-27T11:25:21.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.079+0000 7f5c9a825700 1 Processor -- start 2023-08-27T11:25:21.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.080+0000 7f5c9a825700 1 -- start start 2023-08-27T11:25:21.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.080+0000 7f5c9a825700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c94071360 0x7f5c9414dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.080+0000 7f5c9a825700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c940d1a60 0x7f5c9415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.080+0000 7f5c9a825700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c9415af90 0x7f5c9415f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.080+0000 7f5c9a825700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9415bf30 con 0x7f5c9415af90 2023-08-27T11:25:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.080+0000 7f5c9a825700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c940cc5d0 con 0x7f5c94071360 2023-08-27T11:25:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.080+0000 7f5c9a825700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9415b8e0 con 0x7f5c940d1a60 2023-08-27T11:25:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c98dc2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c9415af90 0x7f5c9415f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c98dc2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c9415af90 0x7f5c9415f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59378/0 (socket says 172.21.15.112:59378) 2023-08-27T11:25:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c98dc2700 1 -- 172.21.15.112:0/2525931380 learned_addr learned my addr 172.21.15.112:0/2525931380 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c93fff700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c94071360 0x7f5c9414dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c937fe700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c940d1a60 0x7f5c9415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c98dc2700 1 -- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c940d1a60 msgr2=0x7f5c9415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c98dc2700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c940d1a60 0x7f5c9415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c98dc2700 1 -- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c94071360 msgr2=0x7f5c9414dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c98dc2700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c94071360 0x7f5c9414dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c98dc2700 1 -- 172.21.15.112:0/2525931380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c84009550 con 0x7f5c9415af90 2023-08-27T11:25:21.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c98dc2700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c9415af90 0x7f5c9415f2c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00cbc0 tx=0x7f5c8c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c90ff9700 1 -- 172.21.15.112:0/2525931380 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c00a9e0 con 0x7f5c9415af90 2023-08-27T11:25:21.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c90ff9700 1 -- 172.21.15.112:0/2525931380 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8c00ab40 con 0x7f5c9415af90 2023-08-27T11:25:21.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c9a825700 1 -- 172.21.15.112:0/2525931380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c880094a0 con 0x7f5c9415af90 2023-08-27T11:25:21.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c90ff9700 1 -- 172.21.15.112:0/2525931380 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c007070 con 0x7f5c9415af90 2023-08-27T11:25:21.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.081+0000 7f5c9a825700 1 -- 172.21.15.112:0/2525931380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c940b3690 con 0x7f5c9415af90 2023-08-27T11:25:21.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.082+0000 7f5c9a825700 1 -- 172.21.15.112:0/2525931380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9415b740 con 0x7f5c9415af90 2023-08-27T11:25:21.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.083+0000 7f5c90ff9700 1 -- 172.21.15.112:0/2525931380 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f5c8c0071d0 con 0x7f5c9415af90 2023-08-27T11:25:21.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.083+0000 7f5c90ff9700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5c8006a350 0x7f5c8006c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.083+0000 7f5c90ff9700 1 -- 172.21.15.112:0/2525931380 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c8c0884f0 con 0x7f5c9415af90 2023-08-27T11:25:21.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.083+0000 7f5c93fff700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5c8006a350 0x7f5c8006c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.084+0000 7f5c93fff700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5c8006a350 0x7f5c8006c810 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5c84006d30 tx=0x7f5c840092c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.087+0000 7f5c90ff9700 1 -- 172.21.15.112:0/2525931380 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5c9415b740 con 0x7f5c9415af90 2023-08-27T11:25:21.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.285+0000 7f5c9a825700 1 -- 172.21.15.112:0/2525931380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5c9415b740 con 0x7f5c9415af90 2023-08-27T11:25:21.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.285+0000 7f5c90ff9700 1 -- 172.21.15.112:0/2525931380 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5c8c017020 con 0x7f5c9415af90 2023-08-27T11:25:21.287 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:25:21.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.288+0000 7f5c9a825700 1 -- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5c8006a350 msgr2=0x7f5c8006c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.288+0000 7f5c9a825700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5c8006a350 0x7f5c8006c810 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5c84006d30 tx=0x7f5c840092c0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.288+0000 7f5c9a825700 1 -- 172.21.15.112:0/2525931380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c9415af90 msgr2=0x7f5c9415f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.288+0000 7f5c9a825700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c9415af90 0x7f5c9415f2c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00cbc0 tx=0x7f5c8c00ced0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.288+0000 7f5c9a825700 1 -- 172.21.15.112:0/2525931380 shutdown_connections 2023-08-27T11:25:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.288+0000 7f5c9a825700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5c8006a350 0x7f5c8006c810 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.288+0000 7f5c9a825700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c9415af90 0x7f5c9415f2c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.288+0000 7f5c9a825700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c94071360 0x7f5c9414dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.288+0000 7f5c9a825700 1 --2- 172.21.15.112:0/2525931380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c940d1a60 0x7f5c9415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.288+0000 7f5c9a825700 1 -- 172.21.15.112:0/2525931380 >> 172.21.15.112:0/2525931380 conn(0x7f5c94066010 msgr2=0x7f5c940cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.288+0000 7f5c9a825700 1 -- 172.21.15.112:0/2525931380 shutdown_connections 2023-08-27T11:25:21.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.288+0000 7f5c9a825700 1 -- 172.21.15.112:0/2525931380 wait complete. 2023-08-27T11:25:21.302 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:25:21.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.428+0000 7f3867f4e700 1 Processor -- start 2023-08-27T11:25:21.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.428+0000 7f3867f4e700 1 -- start start 2023-08-27T11:25:21.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.429+0000 7f3867f4e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38601445b0 0x7f38600ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.429+0000 7f3867f4e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38600b08e0 con 0x7f38600ccad0 2023-08-27T11:25:21.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.429+0000 7f3867f4e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38600b0b30 con 0x7f386006fea0 2023-08-27T11:25:21.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.429+0000 7f3867f4e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38600ccf50 con 0x7f38601445b0 2023-08-27T11:25:21.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.429+0000 7f38654e9700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f38600ccad0 0x7f386006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53764/0 (socket says 172.21.15.112:53764) 2023-08-27T11:25:21.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.429+0000 7f38654e9700 1 -- 172.21.15.112:0/3591466007 learned_addr learned my addr 172.21.15.112:0/3591466007 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:21.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.429+0000 7f3865cea700 1 --2- 172.21.15.112:0/3591466007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38601445b0 0x7f38600ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.430+0000 7f3864ce8700 1 -- 172.21.15.112:0/3591466007 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246194904 0 0) 0x7f38600b08e0 con 0x7f38600ccad0 2023-08-27T11:25:21.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.430+0000 7f3864ce8700 1 -- 172.21.15.112:0/3591466007 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f384c0035b0 con 0x7f38600ccad0 2023-08-27T11:25:21.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.430+0000 7f3864ce8700 1 -- 172.21.15.112:0/3591466007 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282792463 0 0) 0x7f38600b0b30 con 0x7f386006fea0 2023-08-27T11:25:21.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.430+0000 7f3864ce8700 1 -- 172.21.15.112:0/3591466007 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38600b08e0 con 0x7f386006fea0 2023-08-27T11:25:21.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.430+0000 7f3865cea700 1 -- 172.21.15.112:0/3591466007 >> v1:172.21.15.138:6789/0 conn(0x7f386006fea0 legacy=0x7f386015ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.430+0000 7f3865cea700 1 -- 172.21.15.112:0/3591466007 >> v1:172.21.15.112:6789/0 conn(0x7f38600ccad0 legacy=0x7f386006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.430+0000 7f3865cea700 1 -- 172.21.15.112:0/3591466007 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38600b0e10 con 0x7f38601445b0 2023-08-27T11:25:21.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.430+0000 7f3865cea700 1 --2- 172.21.15.112:0/3591466007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38601445b0 0x7f38600ce210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f385c009870 tx=0x7f385c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76912b790f373dd7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.430+0000 7f3864ce8700 1 -- 172.21.15.112:0/3591466007 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c011830 con 0x7f38601445b0 2023-08-27T11:25:21.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.430+0000 7f3864ce8700 1 -- 172.21.15.112:0/3591466007 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c011990 con 0x7f38601445b0 2023-08-27T11:25:21.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.430+0000 7f3864ce8700 1 -- 172.21.15.112:0/3591466007 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c011d20 con 0x7f38601445b0 2023-08-27T11:25:21.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.431+0000 7f3867f4e700 1 -- 172.21.15.112:0/3591466007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38601445b0 msgr2=0x7f38600ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.431+0000 7f3867f4e700 1 --2- 172.21.15.112:0/3591466007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38601445b0 0x7f38600ce210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f385c009870 tx=0x7f385c009b80 comp rx=0 tx=0).stop 2023-08-27T11:25:21.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.431+0000 7f3867f4e700 1 -- 172.21.15.112:0/3591466007 shutdown_connections 2023-08-27T11:25:21.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.431+0000 7f3867f4e700 1 --2- 172.21.15.112:0/3591466007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38601445b0 0x7f38600ce210 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.431+0000 7f3867f4e700 1 -- 172.21.15.112:0/3591466007 >> 172.21.15.112:0/3591466007 conn(0x7f38600b1e90 msgr2=0x7f38600b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:21.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.431+0000 7f3867f4e700 1 -- 172.21.15.112:0/3591466007 shutdown_connections 2023-08-27T11:25:21.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.431+0000 7f3867f4e700 1 -- 172.21.15.112:0/3591466007 wait complete. 2023-08-27T11:25:21.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.432+0000 7f3867f4e700 1 Processor -- start 2023-08-27T11:25:21.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.432+0000 7f3867f4e700 1 -- start start 2023-08-27T11:25:21.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f3867f4e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f386006fea0 0x7f38600cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f3867f4e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38600ccad0 0x7f38600b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f3867f4e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38601445b0 0x7f38600cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f3867f4e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38600b4b10 con 0x7f38600ccad0 2023-08-27T11:25:21.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f3867f4e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38600b3e90 con 0x7f38601445b0 2023-08-27T11:25:21.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f3867f4e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38600b4170 con 0x7f386006fea0 2023-08-27T11:25:21.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f38654e9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38600ccad0 0x7f38600b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f38654e9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38600ccad0 0x7f38600b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59390/0 (socket says 172.21.15.112:59390) 2023-08-27T11:25:21.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f38654e9700 1 -- 172.21.15.112:0/225332229 learned_addr learned my addr 172.21.15.112:0/225332229 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:21.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f3865cea700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f386006fea0 0x7f38600cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f38664eb700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38601445b0 0x7f38600cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f38654e9700 1 -- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f386006fea0 msgr2=0x7f38600cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f38654e9700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f386006fea0 0x7f38600cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f38654e9700 1 -- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38601445b0 msgr2=0x7f38600cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f38654e9700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38601445b0 0x7f38600cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f38654e9700 1 -- 172.21.15.112:0/225332229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c0094a0 con 0x7f38600ccad0 2023-08-27T11:25:21.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f38664eb700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38601445b0 0x7f38600cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:21.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.433+0000 7f3865cea700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f386006fea0 0x7f38600cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:21.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.434+0000 7f38654e9700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38600ccad0 0x7f38600b3440 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f385000c700 tx=0x7f385000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.434+0000 7f38567fc700 1 -- 172.21.15.112:0/225332229 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850005d70 con 0x7f38600ccad0 2023-08-27T11:25:21.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.434+0000 7f38567fc700 1 -- 172.21.15.112:0/225332229 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38500070d0 con 0x7f38600ccad0 2023-08-27T11:25:21.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.434+0000 7f3867f4e700 1 -- 172.21.15.112:0/225332229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38600b4450 con 0x7f38600ccad0 2023-08-27T11:25:21.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.434+0000 7f38567fc700 1 -- 172.21.15.112:0/225332229 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850007460 con 0x7f38600ccad0 2023-08-27T11:25:21.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.434+0000 7f3867f4e700 1 -- 172.21.15.112:0/225332229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38600cb360 con 0x7f38600ccad0 2023-08-27T11:25:21.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.435+0000 7f3867f4e700 1 -- 172.21.15.112:0/225332229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3860067e90 con 0x7f38600ccad0 2023-08-27T11:25:21.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.435+0000 7f38567fc700 1 -- 172.21.15.112:0/225332229 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f38500075c0 con 0x7f38600ccad0 2023-08-27T11:25:21.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.436+0000 7f38567fc700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f384406a320 0x7f384406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.436+0000 7f38567fc700 1 -- 172.21.15.112:0/225332229 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3850087d40 con 0x7f38600ccad0 2023-08-27T11:25:21.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.436+0000 7f3865cea700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f384406a320 0x7f384406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.436+0000 7f3865cea700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f384406a320 0x7f384406c7e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f385c0090b0 tx=0x7f385c01d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.440+0000 7f38567fc700 1 -- 172.21.15.112:0/225332229 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3850053160 con 0x7f38600ccad0 2023-08-27T11:25:21.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.695+0000 7f3867f4e700 1 -- 172.21.15.112:0/225332229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3860067a20 con 0x7f38600ccad0 2023-08-27T11:25:21.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.695+0000 7f38567fc700 1 -- 172.21.15.112:0/225332229 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3850017070 con 0x7f38600ccad0 2023-08-27T11:25:21.697 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:25:21.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.698+0000 7f3867f4e700 1 -- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f384406a320 msgr2=0x7f384406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.698+0000 7f3867f4e700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f384406a320 0x7f384406c7e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f385c0090b0 tx=0x7f385c01d040 comp rx=0 tx=0).stop 2023-08-27T11:25:21.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.698+0000 7f3867f4e700 1 -- 172.21.15.112:0/225332229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38600ccad0 msgr2=0x7f38600b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.698+0000 7f3867f4e700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38600ccad0 0x7f38600b3440 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f385000c700 tx=0x7f385000ca10 comp rx=0 tx=0).stop 2023-08-27T11:25:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.698+0000 7f3867f4e700 1 -- 172.21.15.112:0/225332229 shutdown_connections 2023-08-27T11:25:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.698+0000 7f3867f4e700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f384406a320 0x7f384406c7e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.698+0000 7f3867f4e700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38601445b0 0x7f38600cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.698+0000 7f3867f4e700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f386006fea0 0x7f38600cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.698+0000 7f3867f4e700 1 --2- 172.21.15.112:0/225332229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38600ccad0 0x7f38600b3440 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.698+0000 7f3867f4e700 1 -- 172.21.15.112:0/225332229 >> 172.21.15.112:0/225332229 conn(0x7f38600b1e90 msgr2=0x7f38600cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:21.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.698+0000 7f3867f4e700 1 -- 172.21.15.112:0/225332229 shutdown_connections 2023-08-27T11:25:21.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.699+0000 7f3867f4e700 1 -- 172.21.15.112:0/225332229 wait complete. 2023-08-27T11:25:21.712 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:25:21.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.844+0000 7ff9ecaea700 1 Processor -- start 2023-08-27T11:25:21.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.844+0000 7ff9ecaea700 1 -- start start 2023-08-27T11:25:21.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.845+0000 7ff9ecaea700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff9e8146bf0 0x7ff9e80b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.845+0000 7ff9ecaea700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e8146fd0 0x7ff9e80cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.845+0000 7ff9ecaea700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e80d1500 con 0x7ff9e80d0290 2023-08-27T11:25:21.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.845+0000 7ff9ecaea700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e80d0710 con 0x7ff9e8146bf0 2023-08-27T11:25:21.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.845+0000 7ff9ecaea700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e80d09f0 con 0x7ff9e8146fd0 2023-08-27T11:25:21.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.846+0000 7ff9e659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff9e8146bf0 0x7ff9e80b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.846+0000 7ff9e659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff9e8146bf0 0x7ff9e80b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59396/0 (socket says 172.21.15.112:59396) 2023-08-27T11:25:21.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.846+0000 7ff9e659c700 1 -- 172.21.15.112:0/580991641 learned_addr learned my addr 172.21.15.112:0/580991641 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:21.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.846+0000 7ff9e5d9b700 1 --2- 172.21.15.112:0/580991641 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e8146fd0 0x7ff9e80cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.846+0000 7ff9e659c700 1 -- 172.21.15.112:0/580991641 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e8146fd0 msgr2=0x7ff9e80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.846+0000 7ff9e659c700 1 --2- 172.21.15.112:0/580991641 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e8146fd0 0x7ff9e80cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.846+0000 7ff9e659c700 1 -- 172.21.15.112:0/580991641 >> v1:172.21.15.112:6789/0 conn(0x7ff9e80d0290 legacy=0x7ff9e8070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.846+0000 7ff9e659c700 1 -- 172.21.15.112:0/580991641 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e80d0cd0 con 0x7ff9e8146bf0 2023-08-27T11:25:21.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.846+0000 7ff9e659c700 1 --2- 172.21.15.112:0/580991641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9e8146bf0 0x7ff9e80b5db0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80098f0 tx=0x7ff9d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b746ec28455bc665 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.846+0000 7ff9e559a700 1 -- 172.21.15.112:0/580991641 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800b8a0 con 0x7ff9e8146bf0 2023-08-27T11:25:21.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.846+0000 7ff9e559a700 1 -- 172.21.15.112:0/580991641 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d800ba00 con 0x7ff9e8146bf0 2023-08-27T11:25:21.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.846+0000 7ff9e559a700 1 -- 172.21.15.112:0/580991641 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800bd90 con 0x7ff9e8146bf0 2023-08-27T11:25:21.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.847+0000 7ff9ecaea700 1 -- 172.21.15.112:0/580991641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9e8146bf0 msgr2=0x7ff9e80b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.847+0000 7ff9ecaea700 1 --2- 172.21.15.112:0/580991641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9e8146bf0 0x7ff9e80b5db0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80098f0 tx=0x7ff9d8009c00 comp rx=0 tx=0).stop 2023-08-27T11:25:21.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.847+0000 7ff9ecaea700 1 -- 172.21.15.112:0/580991641 shutdown_connections 2023-08-27T11:25:21.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.847+0000 7ff9ecaea700 1 --2- 172.21.15.112:0/580991641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9e8146bf0 0x7ff9e80b5db0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.847+0000 7ff9ecaea700 1 --2- 172.21.15.112:0/580991641 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e8146fd0 0x7ff9e80cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.847+0000 7ff9ecaea700 1 -- 172.21.15.112:0/580991641 >> 172.21.15.112:0/580991641 conn(0x7ff9e806c450 msgr2=0x7ff9e806f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:21.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.847+0000 7ff9ecaea700 1 -- 172.21.15.112:0/580991641 shutdown_connections 2023-08-27T11:25:21.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.847+0000 7ff9ecaea700 1 -- 172.21.15.112:0/580991641 wait complete. 2023-08-27T11:25:21.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.849+0000 7ff9ecaea700 1 Processor -- start 2023-08-27T11:25:21.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.849+0000 7ff9ecaea700 1 -- start start 2023-08-27T11:25:21.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.849+0000 7ff9ecaea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9e80d0290 0x7ff9e813cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.849+0000 7ff9ecaea700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e8146bf0 0x7ff9e8144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.849+0000 7ff9ecaea700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e8146fd0 0x7ff9e81f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.849+0000 7ff9ecaea700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e813d1e0 con 0x7ff9e80d0290 2023-08-27T11:25:21.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.849+0000 7ff9ecaea700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e8144a00 con 0x7ff9e8146bf0 2023-08-27T11:25:21.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.849+0000 7ff9ecaea700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e8144ce0 con 0x7ff9e8146fd0 2023-08-27T11:25:21.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9e659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9e80d0290 0x7ff9e813cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9e659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9e80d0290 0x7ff9e813cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59402/0 (socket says 172.21.15.112:59402) 2023-08-27T11:25:21.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9e659c700 1 -- 172.21.15.112:0/2593001510 learned_addr learned my addr 172.21.15.112:0/2593001510 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:21.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9e6d9d700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e8146fd0 0x7ff9e81f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9e5d9b700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e8146bf0 0x7ff9e8144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9e659c700 1 -- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e8146fd0 msgr2=0x7ff9e81f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9e659c700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e8146fd0 0x7ff9e81f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9e659c700 1 -- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e8146bf0 msgr2=0x7ff9e8144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9e659c700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e8146bf0 0x7ff9e8144090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9e659c700 1 -- 172.21.15.112:0/2593001510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d8009550 con 0x7ff9e80d0290 2023-08-27T11:25:21.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9e659c700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9e80d0290 0x7ff9e813cca0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009c40 tx=0x7ff9d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9d6ffd700 1 -- 172.21.15.112:0/2593001510 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8004450 con 0x7ff9e80d0290 2023-08-27T11:25:21.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9d6ffd700 1 -- 172.21.15.112:0/2593001510 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d80045b0 con 0x7ff9e80d0290 2023-08-27T11:25:21.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9d6ffd700 1 -- 172.21.15.112:0/2593001510 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80038e0 con 0x7ff9e80d0290 2023-08-27T11:25:21.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9ecaea700 1 -- 172.21.15.112:0/2593001510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e8144fc0 con 0x7ff9e80d0290 2023-08-27T11:25:21.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.850+0000 7ff9ecaea700 1 -- 172.21.15.112:0/2593001510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e8145580 con 0x7ff9e80d0290 2023-08-27T11:25:21.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.851+0000 7ff9ecaea700 1 -- 172.21.15.112:0/2593001510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e8067e90 con 0x7ff9e80d0290 2023-08-27T11:25:21.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.852+0000 7ff9d6ffd700 1 -- 172.21.15.112:0/2593001510 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7ff9d8003a40 con 0x7ff9e80d0290 2023-08-27T11:25:21.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.852+0000 7ff9d6ffd700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff9c806a320 0x7ff9c806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.852+0000 7ff9d6ffd700 1 -- 172.21.15.112:0/2593001510 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff9d8089930 con 0x7ff9e80d0290 2023-08-27T11:25:21.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.852+0000 7ff9e5d9b700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff9c806a320 0x7ff9c806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.853+0000 7ff9e5d9b700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff9c806a320 0x7ff9c806c7e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff9e815e2c0 tx=0x7ff9dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:21.857+0000 7ff9d6ffd700 1 -- 172.21.15.112:0/2593001510 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff9d8054e70 con 0x7ff9e80d0290 2023-08-27T11:25:22.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.058+0000 7ff9ecaea700 1 -- 172.21.15.112:0/2593001510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9e8067a20 con 0x7ff9e80d0290 2023-08-27T11:25:22.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.058+0000 7ff9d6ffd700 1 -- 172.21.15.112:0/2593001510 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9d8058490 con 0x7ff9e80d0290 2023-08-27T11:25:22.060 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:25:22.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.060+0000 7ff9ecaea700 1 -- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff9c806a320 msgr2=0x7ff9c806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.060+0000 7ff9ecaea700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff9c806a320 0x7ff9c806c7e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff9e815e2c0 tx=0x7ff9dc008040 comp rx=0 tx=0).stop 2023-08-27T11:25:22.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.060+0000 7ff9ecaea700 1 -- 172.21.15.112:0/2593001510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9e80d0290 msgr2=0x7ff9e813cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.060+0000 7ff9ecaea700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9e80d0290 0x7ff9e813cca0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009c40 tx=0x7ff9d8004750 comp rx=0 tx=0).stop 2023-08-27T11:25:22.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.061+0000 7ff9ecaea700 1 -- 172.21.15.112:0/2593001510 shutdown_connections 2023-08-27T11:25:22.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.061+0000 7ff9ecaea700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff9c806a320 0x7ff9c806c7e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.061+0000 7ff9ecaea700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e8146fd0 0x7ff9e81f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.061+0000 7ff9ecaea700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9e80d0290 0x7ff9e813cca0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.061+0000 7ff9ecaea700 1 --2- 172.21.15.112:0/2593001510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e8146bf0 0x7ff9e8144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.061+0000 7ff9ecaea700 1 -- 172.21.15.112:0/2593001510 >> 172.21.15.112:0/2593001510 conn(0x7ff9e806c450 msgr2=0x7ff9e815e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:22.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.061+0000 7ff9ecaea700 1 -- 172.21.15.112:0/2593001510 shutdown_connections 2023-08-27T11:25:22.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.061+0000 7ff9ecaea700 1 -- 172.21.15.112:0/2593001510 wait complete. 2023-08-27T11:25:22.075 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:25:22.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.210+0000 7f53f5e25700 1 Processor -- start 2023-08-27T11:25:22.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.210+0000 7f53f5e25700 1 -- start start 2023-08-27T11:25:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.210+0000 7f53f5e25700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f53f00723f0 0x7f53f006ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53f5e25700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f00b16e0 con 0x7f53f0143390 2023-08-27T11:25:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53f5e25700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f0143770 con 0x7f53f006e2f0 2023-08-27T11:25:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53f5e25700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f0070000 con 0x7f53f00723f0 2023-08-27T11:25:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53ef7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f53f0143390 0x7f53f006f820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53792/0 (socket says 172.21.15.112:53792) 2023-08-27T11:25:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53ef7fe700 1 -- 172.21.15.112:0/203017969 learned_addr learned my addr 172.21.15.112:0/203017969 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53eeffd700 1 --2- 172.21.15.112:0/203017969 >> v2:172.21.15.138:3300/0 conn(0x7f53f00723f0 0x7f53f006ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53ee7fc700 1 -- 172.21.15.112:0/203017969 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229351879 0 0) 0x7f53f00b16e0 con 0x7f53f0143390 2023-08-27T11:25:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53ee7fc700 1 -- 172.21.15.112:0/203017969 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53d80035b0 con 0x7f53f0143390 2023-08-27T11:25:22.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53eeffd700 1 -- 172.21.15.112:0/203017969 >> v1:172.21.15.138:6789/0 conn(0x7f53f006e2f0 legacy=0x7f53f00af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53eeffd700 1 -- 172.21.15.112:0/203017969 >> v1:172.21.15.112:6789/0 conn(0x7f53f0143390 legacy=0x7f53f006f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53eeffd700 1 -- 172.21.15.112:0/203017969 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f0070f40 con 0x7f53f00723f0 2023-08-27T11:25:22.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53eeffd700 1 --2- 172.21.15.112:0/203017969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f00723f0 0x7f53f006ddb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f53e400a9e0 tx=0x7f53e400acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3ae7207d4ca25e0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:22.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53ee7fc700 1 -- 172.21.15.112:0/203017969 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e400cb70 con 0x7f53f00723f0 2023-08-27T11:25:22.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53ee7fc700 1 -- 172.21.15.112:0/203017969 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e400ccd0 con 0x7f53f00723f0 2023-08-27T11:25:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.211+0000 7f53ee7fc700 1 -- 172.21.15.112:0/203017969 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4018620 con 0x7f53f00723f0 2023-08-27T11:25:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.212+0000 7f53f5e25700 1 -- 172.21.15.112:0/203017969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f00723f0 msgr2=0x7f53f006ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.212+0000 7f53f5e25700 1 --2- 172.21.15.112:0/203017969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f00723f0 0x7f53f006ddb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f53e400a9e0 tx=0x7f53e400acf0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.212+0000 7f53f5e25700 1 -- 172.21.15.112:0/203017969 shutdown_connections 2023-08-27T11:25:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.212+0000 7f53f5e25700 1 --2- 172.21.15.112:0/203017969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f00723f0 0x7f53f006ddb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.212+0000 7f53f5e25700 1 -- 172.21.15.112:0/203017969 >> 172.21.15.112:0/203017969 conn(0x7f53f00cc240 msgr2=0x7f53f00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:22.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.212+0000 7f53f5e25700 1 -- 172.21.15.112:0/203017969 shutdown_connections 2023-08-27T11:25:22.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.212+0000 7f53f5e25700 1 -- 172.21.15.112:0/203017969 wait complete. 2023-08-27T11:25:22.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.213+0000 7f53f5e25700 1 Processor -- start 2023-08-27T11:25:22.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.214+0000 7f53f5e25700 1 -- start start 2023-08-27T11:25:22.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.214+0000 7f53f5e25700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f006e2f0 0x7f53f013a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:22.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.214+0000 7f53f5e25700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53f00723f0 0x7f53f015a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:22.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.214+0000 7f53f5e25700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53f0143390 0x7f53f015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:22.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.214+0000 7f53f5e25700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f015b6b0 con 0x7f53f0143390 2023-08-27T11:25:22.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.214+0000 7f53f5e25700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f0156c20 con 0x7f53f006e2f0 2023-08-27T11:25:22.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.214+0000 7f53f5e25700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f01f6620 con 0x7f53f00723f0 2023-08-27T11:25:22.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.214+0000 7f53ef7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f006e2f0 0x7f53f013a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:22.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.214+0000 7f53ef7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f006e2f0 0x7f53f013a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:59840/0 (socket says 172.21.15.112:59840) 2023-08-27T11:25:22.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.214+0000 7f53eeffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53f00723f0 0x7f53f015a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:22.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.214+0000 7f53ef7fe700 1 -- 172.21.15.112:0/2409521647 learned_addr learned my addr 172.21.15.112:0/2409521647 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:22.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.215+0000 7f53effff700 1 --2- 172.21.15.112:0/2409521647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53f0143390 0x7f53f015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:22.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.215+0000 7f53ef7fe700 1 -- 172.21.15.112:0/2409521647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53f00723f0 msgr2=0x7f53f015a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.215+0000 7f53ef7fe700 1 --2- 172.21.15.112:0/2409521647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53f00723f0 0x7f53f015a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.215+0000 7f53ef7fe700 1 -- 172.21.15.112:0/2409521647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53f0143390 msgr2=0x7f53f015aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.215+0000 7f53ef7fe700 1 --2- 172.21.15.112:0/2409521647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53f0143390 0x7f53f015aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.215+0000 7f53ef7fe700 1 -- 172.21.15.112:0/2409521647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e400a640 con 0x7f53f006e2f0 2023-08-27T11:25:22.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.215+0000 7f53ef7fe700 1 --2- 172.21.15.112:0/2409521647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f006e2f0 0x7f53f013a4b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c700 tx=0x7f53e000ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:22.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.215+0000 7f53d7fff700 1 -- 172.21.15.112:0/2409521647 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0005d70 con 0x7f53f006e2f0 2023-08-27T11:25:22.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.215+0000 7f53d7fff700 1 -- 172.21.15.112:0/2409521647 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e00070d0 con 0x7f53f006e2f0 2023-08-27T11:25:22.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.215+0000 7f53f5e25700 1 -- 172.21.15.112:0/2409521647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f015b0f0 con 0x7f53f006e2f0 2023-08-27T11:25:22.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.215+0000 7f53d7fff700 1 -- 172.21.15.112:0/2409521647 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0007460 con 0x7f53f006e2f0 2023-08-27T11:25:22.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.215+0000 7f53f5e25700 1 -- 172.21.15.112:0/2409521647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f0156e70 con 0x7f53f006e2f0 2023-08-27T11:25:22.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.216+0000 7f53f5e25700 1 -- 172.21.15.112:0/2409521647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f0067e90 con 0x7f53f006e2f0 2023-08-27T11:25:22.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.217+0000 7f53d7fff700 1 -- 172.21.15.112:0/2409521647 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f53e00075c0 con 0x7f53f006e2f0 2023-08-27T11:25:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.217+0000 7f53d7fff700 1 --2- 172.21.15.112:0/2409521647 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f53dc06a320 0x7f53dc06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.217+0000 7f53d7fff700 1 -- 172.21.15.112:0/2409521647 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f53e0087df0 con 0x7f53f006e2f0 2023-08-27T11:25:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.217+0000 7f53eeffd700 1 --2- 172.21.15.112:0/2409521647 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f53dc06a320 0x7f53dc06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.218+0000 7f53eeffd700 1 --2- 172.21.15.112:0/2409521647 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f53dc06a320 0x7f53dc06c7e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f53e400a1c0 tx=0x7f53e4012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.222+0000 7f53d7fff700 1 -- 172.21.15.112:0/2409521647 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f53e0053180 con 0x7f53f006e2f0 2023-08-27T11:25:22.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.424+0000 7f53f5e25700 1 -- 172.21.15.112:0/2409521647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f53f0067a20 con 0x7f53f006e2f0 2023-08-27T11:25:22.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.425+0000 7f53d7fff700 1 -- 172.21.15.112:0/2409521647 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f53e005ae60 con 0x7f53f006e2f0 2023-08-27T11:25:22.427 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:25:22.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.427+0000 7f53f5e25700 1 -- 172.21.15.112:0/2409521647 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f53dc06a320 msgr2=0x7f53dc06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.427+0000 7f53f5e25700 1 --2- 172.21.15.112:0/2409521647 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f53dc06a320 0x7f53dc06c7e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f53e400a1c0 tx=0x7f53e4012000 comp rx=0 tx=0).stop 2023-08-27T11:25:22.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.427+0000 7f53f5e25700 1 -- 172.21.15.112:0/2409521647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f006e2f0 msgr2=0x7f53f013a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.427+0000 7f53f5e25700 1 --2- 172.21.15.112:0/2409521647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f006e2f0 0x7f53f013a4b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c700 tx=0x7f53e000ca10 comp rx=0 tx=0).stop 2023-08-27T11:25:22.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.427+0000 7f53f5e25700 1 -- 172.21.15.112:0/2409521647 shutdown_connections 2023-08-27T11:25:22.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.427+0000 7f53f5e25700 1 --2- 172.21.15.112:0/2409521647 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f53dc06a320 0x7f53dc06c7e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.427+0000 7f53f5e25700 1 --2- 172.21.15.112:0/2409521647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53f0143390 0x7f53f015aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.427+0000 7f53f5e25700 1 --2- 172.21.15.112:0/2409521647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f006e2f0 0x7f53f013a4b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.427+0000 7f53f5e25700 1 --2- 172.21.15.112:0/2409521647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53f00723f0 0x7f53f015a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.427+0000 7f53f5e25700 1 -- 172.21.15.112:0/2409521647 >> 172.21.15.112:0/2409521647 conn(0x7f53f00cc240 msgr2=0x7f53f0159340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:22.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.427+0000 7f53f5e25700 1 -- 172.21.15.112:0/2409521647 shutdown_connections 2023-08-27T11:25:22.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.427+0000 7f53f5e25700 1 -- 172.21.15.112:0/2409521647 wait complete. 2023-08-27T11:25:22.441 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:25:22.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.566+0000 7f1a7216a700 1 Processor -- start 2023-08-27T11:25:22.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.566+0000 7f1a7216a700 1 -- start start 2023-08-27T11:25:22.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.566+0000 7f1a7216a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1a6c15bf90 0x7f1a6c15f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:22.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a7216a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c15eed0 con 0x7f1a6c0cddb0 2023-08-27T11:25:22.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a7216a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c15fe60 con 0x7f1a6c0ce190 2023-08-27T11:25:22.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a7216a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c15f500 con 0x7f1a6c15bf90 2023-08-27T11:25:22.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a6affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1a6c15bf90 0x7f1a6c15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:22.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a6affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1a6c15bf90 0x7f1a6c15f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59424/0 (socket says 172.21.15.112:59424) 2023-08-27T11:25:22.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a6affd700 1 -- 172.21.15.112:0/2780943196 learned_addr learned my addr 172.21.15.112:0/2780943196 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:22.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a6b7fe700 1 --1- 172.21.15.112:0/2780943196 >> v1:172.21.15.112:6789/0 conn(0x7f1a6c0cddb0 0x7f1a6c15b880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53804/0 (socket says 172.21.15.112:53804) 2023-08-27T11:25:22.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a6affd700 1 -- 172.21.15.112:0/2780943196 >> v1:172.21.15.138:6789/0 conn(0x7f1a6c0ce190 legacy=0x7f1a6c0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a6affd700 1 -- 172.21.15.112:0/2780943196 >> v1:172.21.15.112:6789/0 conn(0x7f1a6c0cddb0 legacy=0x7f1a6c15b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a6a7fc700 1 -- 172.21.15.112:0/2780943196 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401228376 0 0) 0x7f1a6c15fe60 con 0x7f1a6c0ce190 2023-08-27T11:25:22.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a6affd700 1 -- 172.21.15.112:0/2780943196 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c15c450 con 0x7f1a6c15bf90 2023-08-27T11:25:22.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a6affd700 1 --2- 172.21.15.112:0/2780943196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a6c15bf90 0x7f1a6c15f920 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1a600098f0 tx=0x7f1a60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ed305bde51367 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:22.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a6a7fc700 1 -- 172.21.15.112:0/2780943196 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60004230 con 0x7f1a6c15bf90 2023-08-27T11:25:22.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a6a7fc700 1 -- 172.21.15.112:0/2780943196 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a60004390 con 0x7f1a6c15bf90 2023-08-27T11:25:22.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.567+0000 7f1a6a7fc700 1 -- 172.21.15.112:0/2780943196 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a600036a0 con 0x7f1a6c15bf90 2023-08-27T11:25:22.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.568+0000 7f1a7216a700 1 -- 172.21.15.112:0/2780943196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a6c15bf90 msgr2=0x7f1a6c15f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.568+0000 7f1a7216a700 1 --2- 172.21.15.112:0/2780943196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a6c15bf90 0x7f1a6c15f920 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1a600098f0 tx=0x7f1a60009c00 comp rx=0 tx=0).stop 2023-08-27T11:25:22.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.568+0000 7f1a7216a700 1 -- 172.21.15.112:0/2780943196 shutdown_connections 2023-08-27T11:25:22.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.568+0000 7f1a7216a700 1 --2- 172.21.15.112:0/2780943196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a6c15bf90 0x7f1a6c15f920 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.568+0000 7f1a7216a700 1 -- 172.21.15.112:0/2780943196 >> 172.21.15.112:0/2780943196 conn(0x7f1a6c06f970 msgr2=0x7f1a6c06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:22.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.568+0000 7f1a7216a700 1 -- 172.21.15.112:0/2780943196 shutdown_connections 2023-08-27T11:25:22.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.568+0000 7f1a7216a700 1 -- 172.21.15.112:0/2780943196 wait complete. 2023-08-27T11:25:22.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.570+0000 7f1a7216a700 1 Processor -- start 2023-08-27T11:25:22.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.570+0000 7f1a7216a700 1 -- start start 2023-08-27T11:25:22.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a7216a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a6c0cddb0 0x7f1a6c147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:22.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a7216a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0ce190 0x7f1a6c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:22.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a7216a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a6c15bf90 0x7f1a6c0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:22.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a7216a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c1f6270 con 0x7f1a6c15bf90 2023-08-27T11:25:22.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a7216a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c1f70d0 con 0x7f1a6c0ce190 2023-08-27T11:25:22.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a7216a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c1f64c0 con 0x7f1a6c0cddb0 2023-08-27T11:25:22.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a6bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a6c15bf90 0x7f1a6c0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:22.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a6bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a6c15bf90 0x7f1a6c0ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59432/0 (socket says 172.21.15.112:59432) 2023-08-27T11:25:22.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a6bfff700 1 -- 172.21.15.112:0/2857072773 learned_addr learned my addr 172.21.15.112:0/2857072773 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:22.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a6affd700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0ce190 0x7f1a6c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:22.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a6b7fe700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a6c0cddb0 0x7f1a6c147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:22.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a6bfff700 1 -- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a6c0cddb0 msgr2=0x7f1a6c147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a6bfff700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a6c0cddb0 0x7f1a6c147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a6bfff700 1 -- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0ce190 msgr2=0x7f1a6c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a6bfff700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0ce190 0x7f1a6c070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a6bfff700 1 -- 172.21.15.112:0/2857072773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a60009550 con 0x7f1a6c15bf90 2023-08-27T11:25:22.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a6affd700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0ce190 0x7f1a6c070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:22.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.571+0000 7f1a6bfff700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a6c15bf90 0x7f1a6c0ca510 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400c7c0 tx=0x7f1a6400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:22.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.572+0000 7f1a53fff700 1 -- 172.21.15.112:0/2857072773 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64003e30 con 0x7f1a6c15bf90 2023-08-27T11:25:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.572+0000 7f1a53fff700 1 -- 172.21.15.112:0/2857072773 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a64005070 con 0x7f1a6c15bf90 2023-08-27T11:25:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.572+0000 7f1a53fff700 1 -- 172.21.15.112:0/2857072773 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64005420 con 0x7f1a6c15bf90 2023-08-27T11:25:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.572+0000 7f1a7216a700 1 -- 172.21.15.112:0/2857072773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c1f67a0 con 0x7f1a6c15bf90 2023-08-27T11:25:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.572+0000 7f1a7216a700 1 -- 172.21.15.112:0/2857072773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c1f6d60 con 0x7f1a6c15bf90 2023-08-27T11:25:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.573+0000 7f1a7216a700 1 -- 172.21.15.112:0/2857072773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c15f100 con 0x7f1a6c15bf90 2023-08-27T11:25:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.573+0000 7f1a53fff700 1 -- 172.21.15.112:0/2857072773 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f1a64005580 con 0x7f1a6c15bf90 2023-08-27T11:25:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.573+0000 7f1a53fff700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1a5806a320 0x7f1a5806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.573+0000 7f1a53fff700 1 -- 172.21.15.112:0/2857072773 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a64088690 con 0x7f1a6c15bf90 2023-08-27T11:25:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.574+0000 7f1a6b7fe700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1a5806a320 0x7f1a5806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.574+0000 7f1a6b7fe700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1a5806a320 0x7f1a5806c7e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c007c20 tx=0x7f1a5c007d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:22.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.578+0000 7f1a53fff700 1 -- 172.21.15.112:0/2857072773 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1a6c15f100 con 0x7f1a6c15bf90 2023-08-27T11:25:22.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.794+0000 7f1a7216a700 1 -- 172.21.15.112:0/2857072773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1a6c0712f0 con 0x7f1a6c15bf90 2023-08-27T11:25:22.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.794+0000 7f1a53fff700 1 -- 172.21.15.112:0/2857072773 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1a6c0712f0 con 0x7f1a6c15bf90 2023-08-27T11:25:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.797+0000 7f1a7216a700 1 -- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1a5806a320 msgr2=0x7f1a5806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.797+0000 7f1a7216a700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1a5806a320 0x7f1a5806c7e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c007c20 tx=0x7f1a5c007d70 comp rx=0 tx=0).stop 2023-08-27T11:25:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.797+0000 7f1a7216a700 1 -- 172.21.15.112:0/2857072773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a6c15bf90 msgr2=0x7f1a6c0ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.797+0000 7f1a7216a700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a6c15bf90 0x7f1a6c0ca510 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400c7c0 tx=0x7f1a6400cb80 comp rx=0 tx=0).stop 2023-08-27T11:25:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.797+0000 7f1a7216a700 1 -- 172.21.15.112:0/2857072773 shutdown_connections 2023-08-27T11:25:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.797+0000 7f1a7216a700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1a5806a320 0x7f1a5806c7e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.797+0000 7f1a7216a700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a6c15bf90 0x7f1a6c0ca510 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.797+0000 7f1a7216a700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a6c0cddb0 0x7f1a6c147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.797+0000 7f1a7216a700 1 --2- 172.21.15.112:0/2857072773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0ce190 0x7f1a6c070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.797+0000 7f1a7216a700 1 -- 172.21.15.112:0/2857072773 >> 172.21.15.112:0/2857072773 conn(0x7f1a6c06f970 msgr2=0x7f1a6c154f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.797+0000 7f1a7216a700 1 -- 172.21.15.112:0/2857072773 shutdown_connections 2023-08-27T11:25:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:22.797+0000 7f1a7216a700 1 -- 172.21.15.112:0/2857072773 wait complete. 2023-08-27T11:25:22.803 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:25:22.811 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-27T11:25:22.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:22.811 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-08-27T11:25:23.291 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:25:23.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:23.292 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-27T11:25:23.309 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:23.309 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:23.327 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:25:23.327 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:25:23.327 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:23.366 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:25:23.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:25:23.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.489+0000 7f081c5de700 1 Processor -- start 2023-08-27T11:25:23.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.489+0000 7f081c5de700 1 -- start start 2023-08-27T11:25:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f081c5de700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 0x7f08140b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f081c5de700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0814161030 con 0x7f08141466c0 2023-08-27T11:25:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f081c5de700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0814161310 con 0x7f081406fea0 2023-08-27T11:25:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f081c5de700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08140b57e0 con 0x7f08140b4760 2023-08-27T11:25:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f081a37a700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f08141466c0 0x7f081406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53818/0 (socket says 172.21.15.112:53818) 2023-08-27T11:25:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f081a37a700 1 -- 172.21.15.112:0/2113723904 learned_addr learned my addr 172.21.15.112:0/2113723904 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f081ab7b700 1 --2- 172.21.15.112:0/2113723904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 0x7f08140b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606338340 0 0) 0x7f0814161030 con 0x7f08141466c0 2023-08-27T11:25:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08000035b0 con 0x7f08141466c0 2023-08-27T11:25:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 407258767 0 0) 0x7f08000035b0 con 0x7f08141466c0 2023-08-27T11:25:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 msgr2=0x7f08140b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f0819378700 1 --2- 172.21.15.112:0/2113723904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 0x7f08140b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 >> v1:172.21.15.138:6789/0 conn(0x7f081406fea0 legacy=0x7f08140b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08141615f0 con 0x7f08141466c0 2023-08-27T11:25:23.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f0810001670 con 0x7f08141466c0 2023-08-27T11:25:23.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.490+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 >> v1:172.21.15.112:6789/0 conn(0x7f08141466c0 legacy=0x7f081406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f0819378700 1 --2- 172.21.15.112:0/2113723904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 0x7f0800003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f0819378700 1 --2- 172.21.15.112:0/2113723904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0800003fb0 0x7f0800004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f0819378700 1 --2- 172.21.15.112:0/2113723904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800004d90 0x7f0800005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080000bac0 con 0x7f0800004d90 2023-08-27T11:25:23.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080000bc30 con 0x7f0800003fb0 2023-08-27T11:25:23.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080000bdf0 con 0x7f08140b4760 2023-08-27T11:25:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f081a37a700 1 --2- 172.21.15.112:0/2113723904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 0x7f0800003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f081ab7b700 1 --2- 172.21.15.112:0/2113723904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800004d90 0x7f0800005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f0819b79700 1 --2- 172.21.15.112:0/2113723904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0800003fb0 0x7f0800004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f081a37a700 1 -- 172.21.15.112:0/2113723904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0800003fb0 msgr2=0x7f0800004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f081a37a700 1 --2- 172.21.15.112:0/2113723904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0800003fb0 0x7f0800004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f081a37a700 1 -- 172.21.15.112:0/2113723904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800004d90 msgr2=0x7f0800005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f081a37a700 1 --2- 172.21.15.112:0/2113723904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800004d90 0x7f0800005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f081a37a700 1 -- 172.21.15.112:0/2113723904 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000035b0 con 0x7f08140b4760 2023-08-27T11:25:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f081ab7b700 1 --2- 172.21.15.112:0/2113723904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800004d90 0x7f0800005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f081a37a700 1 --2- 172.21.15.112:0/2113723904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 0x7f0800003a30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f081000c240 tx=0x7f0800005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d19e11687b0cfa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810007c10 con 0x7f08140b4760 2023-08-27T11:25:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081000cde0 con 0x7f08140b4760 2023-08-27T11:25:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.491+0000 7f0819378700 1 -- 172.21.15.112:0/2113723904 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810005ba0 con 0x7f08140b4760 2023-08-27T11:25:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.492+0000 7f081c5de700 1 -- 172.21.15.112:0/2113723904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 msgr2=0x7f0800003a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.492+0000 7f081c5de700 1 --2- 172.21.15.112:0/2113723904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 0x7f0800003a30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f081000c240 tx=0x7f0800005880 comp rx=0 tx=0).stop 2023-08-27T11:25:23.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.492+0000 7f081ab7b700 1 -- 172.21.15.112:0/2113723904 reap_dead start 2023-08-27T11:25:23.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.492+0000 7f081c5de700 1 -- 172.21.15.112:0/2113723904 shutdown_connections 2023-08-27T11:25:23.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.492+0000 7f081c5de700 1 -- 172.21.15.112:0/2113723904 >> 172.21.15.112:0/2113723904 conn(0x7f08140cff80 msgr2=0x7f08140d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:23.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.492+0000 7f081c5de700 1 -- 172.21.15.112:0/2113723904 shutdown_connections 2023-08-27T11:25:23.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.492+0000 7f081c5de700 1 -- 172.21.15.112:0/2113723904 wait complete. 2023-08-27T11:25:23.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.493+0000 7f081c5de700 1 Processor -- start 2023-08-27T11:25:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.493+0000 7f081c5de700 1 -- start start 2023-08-27T11:25:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081c5de700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 0x7f081413c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081c5de700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f081415a560 0x7f081415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081c5de700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081415c560 0x7f081415c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081c5de700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141f62b0 con 0x7f081415c560 2023-08-27T11:25:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081c5de700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141f6550 con 0x7f081415a560 2023-08-27T11:25:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081c5de700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08141f6830 con 0x7f08140b4760 2023-08-27T11:25:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081ab7b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081415c560 0x7f081415c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081a37a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 0x7f081413c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081ab7b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081415c560 0x7f081415c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59452/0 (socket says 172.21.15.112:59452) 2023-08-27T11:25:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081ab7b700 1 -- 172.21.15.112:0/3774542171 learned_addr learned my addr 172.21.15.112:0/3774542171 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f0819b79700 1 --2- 172.21.15.112:0/3774542171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f081415a560 0x7f081415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081ab7b700 1 -- 172.21.15.112:0/3774542171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 msgr2=0x7f081413c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081ab7b700 1 --2- 172.21.15.112:0/3774542171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 0x7f081413c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081ab7b700 1 -- 172.21.15.112:0/3774542171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f081415a560 msgr2=0x7f081415a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081ab7b700 1 --2- 172.21.15.112:0/3774542171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f081415a560 0x7f081415a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.494+0000 7f081ab7b700 1 -- 172.21.15.112:0/3774542171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08141f6b10 con 0x7f081415c560 2023-08-27T11:25:23.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.495+0000 7f081ab7b700 1 --2- 172.21.15.112:0/3774542171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081415c560 0x7f081415c9e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f08140b5db0 tx=0x7f080c00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:23.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.495+0000 7f080affd700 1 -- 172.21.15.112:0/3774542171 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c01c070 con 0x7f081415c560 2023-08-27T11:25:23.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.495+0000 7f080affd700 1 -- 172.21.15.112:0/3774542171 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c00f040 con 0x7f081415c560 2023-08-27T11:25:23.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.495+0000 7f080affd700 1 -- 172.21.15.112:0/3774542171 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c00e040 con 0x7f081415c560 2023-08-27T11:25:23.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.495+0000 7f081c5de700 1 -- 172.21.15.112:0/3774542171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081415cf20 con 0x7f081415c560 2023-08-27T11:25:23.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.495+0000 7f081c5de700 1 -- 172.21.15.112:0/3774542171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08141f7030 con 0x7f081415c560 2023-08-27T11:25:23.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.496+0000 7f081c5de700 1 -- 172.21.15.112:0/3774542171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08140b3470 con 0x7f081415c560 2023-08-27T11:25:23.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.496+0000 7f080affd700 1 -- 172.21.15.112:0/3774542171 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f080c015430 con 0x7f081415c560 2023-08-27T11:25:23.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.497+0000 7f080affd700 1 --2- 172.21.15.112:0/3774542171 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f07f806a320 0x7f07f806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.497+0000 7f080affd700 1 -- 172.21.15.112:0/3774542171 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f080c0885f0 con 0x7f081415c560 2023-08-27T11:25:23.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.497+0000 7f081a37a700 1 --2- 172.21.15.112:0/3774542171 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f07f806a320 0x7f07f806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.497+0000 7f081a37a700 1 --2- 172.21.15.112:0/3774542171 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f07f806a320 0x7f07f806c7e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0810015040 tx=0x7f0810007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:23.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.501+0000 7f080affd700 1 -- 172.21.15.112:0/3774542171 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f080c053870 con 0x7f081415c560 2023-08-27T11:25:23.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.784+0000 7f081c5de700 1 -- 172.21.15.112:0/3774542171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0814067a20 con 0x7f081415c560 2023-08-27T11:25:23.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.784+0000 7f080affd700 1 -- 172.21.15.112:0/3774542171 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4886 (secure 0 0 0) 0x7f080c056e90 con 0x7f081415c560 2023-08-27T11:25:23.786 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:23.786 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2906075444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2906075444},{"type":"v1","addr":"172.21.15.138:6836","nonce":2906075444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5761,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1781004436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1781004436},{"type":"v1","addr":"172.21.15.138:6837","nonce":1781004436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1573349448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1573349448},{"type":"v1","addr":"172.21.15.112:6834","nonce":1573349448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:19:16.064335+0000","modified":"2023-08-27T11:19:19.690233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/174707415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":174707415},{"type":"v1","addr":"172.21.15.112:6835","nonce":174707415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-27T11:25:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.786+0000 7f081c5de700 1 -- 172.21.15.112:0/3774542171 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f07f806a320 msgr2=0x7f07f806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.786+0000 7f081c5de700 1 --2- 172.21.15.112:0/3774542171 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f07f806a320 0x7f07f806c7e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0810015040 tx=0x7f0810007000 comp rx=0 tx=0).stop 2023-08-27T11:25:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.786+0000 7f081c5de700 1 -- 172.21.15.112:0/3774542171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081415c560 msgr2=0x7f081415c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.786+0000 7f081c5de700 1 --2- 172.21.15.112:0/3774542171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081415c560 0x7f081415c9e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f08140b5db0 tx=0x7f080c00f6c0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.786+0000 7f081c5de700 1 -- 172.21.15.112:0/3774542171 shutdown_connections 2023-08-27T11:25:23.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.786+0000 7f081c5de700 1 --2- 172.21.15.112:0/3774542171 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f07f806a320 0x7f07f806c7e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.786+0000 7f081c5de700 1 --2- 172.21.15.112:0/3774542171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081415c560 0x7f081415c9e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.786+0000 7f081c5de700 1 --2- 172.21.15.112:0/3774542171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140b4760 0x7f081413c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.786+0000 7f081c5de700 1 --2- 172.21.15.112:0/3774542171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f081415a560 0x7f081415a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.786+0000 7f081c5de700 1 -- 172.21.15.112:0/3774542171 >> 172.21.15.112:0/3774542171 conn(0x7f08140cff80 msgr2=0x7f08140d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:23.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.787+0000 7f081c5de700 1 -- 172.21.15.112:0/3774542171 shutdown_connections 2023-08-27T11:25:23.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.787+0000 7f081c5de700 1 -- 172.21.15.112:0/3774542171 wait complete. 2023-08-27T11:25:23.791 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 40 2023-08-27T11:25:23.800 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:23.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.935+0000 7f1e6cbe4700 1 Processor -- start 2023-08-27T11:25:23.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.936+0000 7f1e6cbe4700 1 -- start start 2023-08-27T11:25:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e6cbe4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1e6806b230 0x7f1e680ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e6cbe4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6815eb60 con 0x7f1e6806adb0 2023-08-27T11:25:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e6cbe4700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e680b02f0 con 0x7f1e68151060 2023-08-27T11:25:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e6cbe4700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e680b05d0 con 0x7f1e6806b230 2023-08-27T11:25:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e65d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1e6806adb0 0x7f1e680cdbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53840/0 (socket says 172.21.15.112:53840) 2023-08-27T11:25:23.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e65d9b700 1 -- 172.21.15.112:0/2622062505 learned_addr learned my addr 172.21.15.112:0/2622062505 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:23.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e66d9d700 1 --2- 172.21.15.112:0/2622062505 >> v2:172.21.15.138:3300/0 conn(0x7f1e6806b230 0x7f1e680ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e6559a700 1 -- 172.21.15.112:0/2622062505 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700337272 0 0) 0x7f1e6815eb60 con 0x7f1e6806adb0 2023-08-27T11:25:23.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e6559a700 1 -- 172.21.15.112:0/2622062505 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e500035b0 con 0x7f1e6806adb0 2023-08-27T11:25:23.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e6559a700 1 -- 172.21.15.112:0/2622062505 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010981139 0 0) 0x7f1e680b02f0 con 0x7f1e68151060 2023-08-27T11:25:23.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e6559a700 1 -- 172.21.15.112:0/2622062505 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e6815eb60 con 0x7f1e68151060 2023-08-27T11:25:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e66d9d700 1 -- 172.21.15.112:0/2622062505 >> v1:172.21.15.138:6789/0 conn(0x7f1e68151060 legacy=0x7f1e6815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e66d9d700 1 -- 172.21.15.112:0/2622062505 >> v1:172.21.15.112:6789/0 conn(0x7f1e6806adb0 legacy=0x7f1e680cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e66d9d700 1 -- 172.21.15.112:0/2622062505 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e680b08b0 con 0x7f1e6806b230 2023-08-27T11:25:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.937+0000 7f1e66d9d700 1 --2- 172.21.15.112:0/2622062505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e6806b230 0x7f1e680ce2e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009d90 tx=0x7f1e6000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f0468c858e69d94 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.938+0000 7f1e6559a700 1 -- 172.21.15.112:0/2622062505 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6000e070 con 0x7f1e6806b230 2023-08-27T11:25:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.938+0000 7f1e6559a700 1 -- 172.21.15.112:0/2622062505 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6000bb20 con 0x7f1e6806b230 2023-08-27T11:25:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.938+0000 7f1e6559a700 1 -- 172.21.15.112:0/2622062505 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60012430 con 0x7f1e6806b230 2023-08-27T11:25:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.938+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/2622062505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e6806b230 msgr2=0x7f1e680ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.938+0000 7f1e6cbe4700 1 --2- 172.21.15.112:0/2622062505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e6806b230 0x7f1e680ce2e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009d90 tx=0x7f1e6000b330 comp rx=0 tx=0).stop 2023-08-27T11:25:23.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.938+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/2622062505 shutdown_connections 2023-08-27T11:25:23.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.938+0000 7f1e6cbe4700 1 --2- 172.21.15.112:0/2622062505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e6806b230 0x7f1e680ce2e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.938+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/2622062505 >> 172.21.15.112:0/2622062505 conn(0x7f1e680d0000 msgr2=0x7f1e680d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:23.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.938+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/2622062505 shutdown_connections 2023-08-27T11:25:23.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.938+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/2622062505 wait complete. 2023-08-27T11:25:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.940+0000 7f1e6cbe4700 1 Processor -- start 2023-08-27T11:25:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.940+0000 7f1e6cbe4700 1 -- start start 2023-08-27T11:25:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.941+0000 7f1e6cbe4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6806adb0 0x7f1e680c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.941+0000 7f1e6cbe4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e6806b230 0x7f1e68147a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.941+0000 7f1e6cbe4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e68151060 0x7f1e68147fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.941+0000 7f1e6cbe4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680c7f30 con 0x7f1e6806adb0 2023-08-27T11:25:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.941+0000 7f1e6cbe4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6814ff10 con 0x7f1e68151060 2023-08-27T11:25:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.941+0000 7f1e6cbe4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680c7aa0 con 0x7f1e6806b230 2023-08-27T11:25:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.941+0000 7f1e6659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6806adb0 0x7f1e680c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.941+0000 7f1e6659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6806adb0 0x7f1e680c2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59462/0 (socket says 172.21.15.112:59462) 2023-08-27T11:25:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.941+0000 7f1e6659c700 1 -- 172.21.15.112:0/1228277101 learned_addr learned my addr 172.21.15.112:0/1228277101 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.941+0000 7f1e65d9b700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e6806b230 0x7f1e68147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.941+0000 7f1e66d9d700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e68151060 0x7f1e68147fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.942+0000 7f1e6659c700 1 -- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e6806b230 msgr2=0x7f1e68147a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.942+0000 7f1e6659c700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e6806b230 0x7f1e68147a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.942+0000 7f1e6659c700 1 -- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e68151060 msgr2=0x7f1e68147fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.942+0000 7f1e6659c700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e68151060 0x7f1e68147fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.942+0000 7f1e6659c700 1 -- 172.21.15.112:0/1228277101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e60009a70 con 0x7f1e6806adb0 2023-08-27T11:25:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.942+0000 7f1e6659c700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6806adb0 0x7f1e680c2f00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800c530 tx=0x7f1e5800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.942+0000 7f1e56ffd700 1 -- 172.21.15.112:0/1228277101 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580052d0 con 0x7f1e6806adb0 2023-08-27T11:25:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.942+0000 7f1e56ffd700 1 -- 172.21.15.112:0/1228277101 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e58005450 con 0x7f1e6806adb0 2023-08-27T11:25:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.942+0000 7f1e56ffd700 1 -- 172.21.15.112:0/1228277101 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580038c0 con 0x7f1e6806adb0 2023-08-27T11:25:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.942+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/1228277101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e680ca010 con 0x7f1e6806adb0 2023-08-27T11:25:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.942+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/1228277101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e680c7450 con 0x7f1e6806adb0 2023-08-27T11:25:23.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.943+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/1228277101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e680b0b70 con 0x7f1e6806adb0 2023-08-27T11:25:23.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.943+0000 7f1e56ffd700 1 -- 172.21.15.112:0/1228277101 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f1e58003a20 con 0x7f1e6806adb0 2023-08-27T11:25:23.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.944+0000 7f1e56ffd700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e4806a320 0x7f1e4806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.944+0000 7f1e56ffd700 1 -- 172.21.15.112:0/1228277101 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1e58087c00 con 0x7f1e6806adb0 2023-08-27T11:25:23.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.944+0000 7f1e65d9b700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e4806a320 0x7f1e4806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.944+0000 7f1e65d9b700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e4806a320 0x7f1e4806c7e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1e68150600 tx=0x7f1e5c007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:23.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:23.952+0000 7f1e56ffd700 1 -- 172.21.15.112:0/1228277101 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1e680b0b70 con 0x7f1e6806adb0 2023-08-27T11:25:24.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.156+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/1228277101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e680b0b70 con 0x7f1e6806adb0 2023-08-27T11:25:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.157+0000 7f1e56ffd700 1 -- 172.21.15.112:0/1228277101 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18320 (secure 0 0 0) 0x7f1e680b0b70 con 0x7f1e6806adb0 2023-08-27T11:25:24.159 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:24.160 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":65,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:19:17.239518+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:19:14.179813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-27T11:19:15.021524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:24.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.159+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e4806a320 msgr2=0x7f1e4806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.159+0000 7f1e6cbe4700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e4806a320 0x7f1e4806c7e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1e68150600 tx=0x7f1e5c007dc0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.159+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/1228277101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6806adb0 msgr2=0x7f1e680c2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.159+0000 7f1e6cbe4700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6806adb0 0x7f1e680c2f00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800c530 tx=0x7f1e5800c840 comp rx=0 tx=0).stop 2023-08-27T11:25:24.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.160+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/1228277101 shutdown_connections 2023-08-27T11:25:24.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.160+0000 7f1e6cbe4700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e4806a320 0x7f1e4806c7e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.160+0000 7f1e6cbe4700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e68151060 0x7f1e68147fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.160+0000 7f1e6cbe4700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6806adb0 0x7f1e680c2f00 secure :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800c530 tx=0x7f1e5800c840 comp rx=0 tx=0).stop 2023-08-27T11:25:24.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.160+0000 7f1e6cbe4700 1 --2- 172.21.15.112:0/1228277101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e6806b230 0x7f1e68147a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.160+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/1228277101 >> 172.21.15.112:0/1228277101 conn(0x7f1e680d0000 msgr2=0x7f1e680708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:24.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.160+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/1228277101 shutdown_connections 2023-08-27T11:25:24.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.160+0000 7f1e6cbe4700 1 -- 172.21.15.112:0/1228277101 wait complete. 2023-08-27T11:25:24.176 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:19:14.179813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:19:15.021524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:25:24.177 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:25:24.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.320+0000 7f0a19e17700 1 Processor -- start 2023-08-27T11:25:24.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.320+0000 7f0a19e17700 1 -- start start 2023-08-27T11:25:24.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a19e17700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0a1406dcc0 0x7f0a140b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a19e17700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a140cfd50 con 0x7f0a141437c0 2023-08-27T11:25:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a19e17700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a140cf140 con 0x7f0a14143ba0 2023-08-27T11:25:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a19e17700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a140cf420 con 0x7f0a1406dcc0 2023-08-27T11:25:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a137fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0a141437c0 0x7f0a140ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53854/0 (socket says 172.21.15.112:53854) 2023-08-27T11:25:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a137fe700 1 -- 172.21.15.112:0/2494634132 learned_addr learned my addr 172.21.15.112:0/2494634132 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a13fff700 1 --2- 172.21.15.112:0/2494634132 >> v2:172.21.15.138:3300/0 conn(0x7f0a1406dcc0 0x7f0a140b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a127fc700 1 -- 172.21.15.112:0/2494634132 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236451607 0 0) 0x7f0a140cfd50 con 0x7f0a141437c0 2023-08-27T11:25:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a127fc700 1 -- 172.21.15.112:0/2494634132 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09fc0035b0 con 0x7f0a141437c0 2023-08-27T11:25:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a127fc700 1 -- 172.21.15.112:0/2494634132 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552032763 0 0) 0x7f0a140cf140 con 0x7f0a14143ba0 2023-08-27T11:25:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a127fc700 1 -- 172.21.15.112:0/2494634132 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a140cfd50 con 0x7f0a14143ba0 2023-08-27T11:25:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a13fff700 1 -- 172.21.15.112:0/2494634132 >> v1:172.21.15.138:6789/0 conn(0x7f0a14143ba0 legacy=0x7f0a1406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a13fff700 1 -- 172.21.15.112:0/2494634132 >> v1:172.21.15.112:6789/0 conn(0x7f0a141437c0 legacy=0x7f0a140ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.321+0000 7f0a13fff700 1 -- 172.21.15.112:0/2494634132 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a140cf700 con 0x7f0a1406dcc0 2023-08-27T11:25:24.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.322+0000 7f0a13fff700 1 --2- 172.21.15.112:0/2494634132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1406dcc0 0x7f0a140b4670 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009d90 tx=0x7f0a0c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6eea0835d212de1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:24.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.322+0000 7f0a127fc700 1 -- 172.21.15.112:0/2494634132 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c00e070 con 0x7f0a1406dcc0 2023-08-27T11:25:24.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.322+0000 7f0a127fc700 1 -- 172.21.15.112:0/2494634132 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c00bb20 con 0x7f0a1406dcc0 2023-08-27T11:25:24.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.322+0000 7f0a127fc700 1 -- 172.21.15.112:0/2494634132 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c012430 con 0x7f0a1406dcc0 2023-08-27T11:25:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.323+0000 7f0a19e17700 1 -- 172.21.15.112:0/2494634132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1406dcc0 msgr2=0x7f0a140b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.323+0000 7f0a19e17700 1 --2- 172.21.15.112:0/2494634132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1406dcc0 0x7f0a140b4670 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009d90 tx=0x7f0a0c00b330 comp rx=0 tx=0).stop 2023-08-27T11:25:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.323+0000 7f0a19e17700 1 -- 172.21.15.112:0/2494634132 shutdown_connections 2023-08-27T11:25:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.323+0000 7f0a19e17700 1 --2- 172.21.15.112:0/2494634132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1406dcc0 0x7f0a140b4670 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.323+0000 7f0a19e17700 1 -- 172.21.15.112:0/2494634132 >> 172.21.15.112:0/2494634132 conn(0x7f0a140b1e90 msgr2=0x7f0a140b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.323+0000 7f0a19e17700 1 -- 172.21.15.112:0/2494634132 shutdown_connections 2023-08-27T11:25:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.323+0000 7f0a19e17700 1 -- 172.21.15.112:0/2494634132 wait complete. 2023-08-27T11:25:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.325+0000 7f0a19e17700 1 Processor -- start 2023-08-27T11:25:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.325+0000 7f0a19e17700 1 -- start start 2023-08-27T11:25:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.326+0000 7f0a19e17700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1406dcc0 0x7f0a14158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.326+0000 7f0a19e17700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a141437c0 0x7f0a141f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.326+0000 7f0a19e17700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a14143ba0 0x7f0a1406f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.326+0000 7f0a19e17700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1413d530 con 0x7f0a14143ba0 2023-08-27T11:25:24.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.326+0000 7f0a19e17700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a141f73d0 con 0x7f0a1406dcc0 2023-08-27T11:25:24.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.326+0000 7f0a19e17700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1413af90 con 0x7f0a141437c0 2023-08-27T11:25:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.326+0000 7f0a13fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a14143ba0 0x7f0a1406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.326+0000 7f0a12ffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a141437c0 0x7f0a141f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.326+0000 7f0a13fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a14143ba0 0x7f0a1406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59480/0 (socket says 172.21.15.112:59480) 2023-08-27T11:25:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.326+0000 7f0a12ffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a141437c0 0x7f0a141f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:42900/0 (socket says 172.21.15.112:42900) 2023-08-27T11:25:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.326+0000 7f0a13fff700 1 -- 172.21.15.112:0/1577342250 learned_addr learned my addr 172.21.15.112:0/1577342250 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.326+0000 7f0a137fe700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1406dcc0 0x7f0a14158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.327+0000 7f0a12ffd700 1 -- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1406dcc0 msgr2=0x7f0a14158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.327+0000 7f0a12ffd700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1406dcc0 0x7f0a14158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.327+0000 7f0a12ffd700 1 -- 172.21.15.112:0/1577342250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a14143ba0 msgr2=0x7f0a1406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.327+0000 7f0a12ffd700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a14143ba0 0x7f0a1406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.327+0000 7f0a12ffd700 1 -- 172.21.15.112:0/1577342250 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c009a70 con 0x7f0a141437c0 2023-08-27T11:25:24.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.327+0000 7f0a13fff700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a14143ba0 0x7f0a1406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:24.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.327+0000 7f0a12ffd700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a141437c0 0x7f0a141f6e90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800c3f0 tx=0x7f0a0800c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:24.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.327+0000 7f09fbfff700 1 -- 172.21.15.112:0/1577342250 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080052d0 con 0x7f0a141437c0 2023-08-27T11:25:24.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.327+0000 7f09fbfff700 1 -- 172.21.15.112:0/1577342250 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a08005450 con 0x7f0a141437c0 2023-08-27T11:25:24.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.327+0000 7f0a19e17700 1 -- 172.21.15.112:0/1577342250 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1413b790 con 0x7f0a141437c0 2023-08-27T11:25:24.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.327+0000 7f09fbfff700 1 -- 172.21.15.112:0/1577342250 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080038c0 con 0x7f0a141437c0 2023-08-27T11:25:24.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.327+0000 7f0a19e17700 1 -- 172.21.15.112:0/1577342250 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1413d150 con 0x7f0a141437c0 2023-08-27T11:25:24.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.328+0000 7f0a19e17700 1 -- 172.21.15.112:0/1577342250 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1413d6e0 con 0x7f0a141437c0 2023-08-27T11:25:24.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.328+0000 7f09fbfff700 1 -- 172.21.15.112:0/1577342250 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f0a08003a20 con 0x7f0a141437c0 2023-08-27T11:25:24.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.329+0000 7f09fbfff700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a0006a320 0x7f0a0006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.329+0000 7f09fbfff700 1 -- 172.21.15.112:0/1577342250 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0a08087a90 con 0x7f0a141437c0 2023-08-27T11:25:24.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.329+0000 7f0a137fe700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a0006a320 0x7f0a0006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.329+0000 7f0a137fe700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a0006a320 0x7f0a0006c7e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0a04007fd0 tx=0x7f0a04007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:24.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.334+0000 7f09fbfff700 1 -- 172.21.15.112:0/1577342250 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0a1413d6e0 con 0x7f0a141437c0 2023-08-27T11:25:24.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.654+0000 7f0a19e17700 1 -- 172.21.15.112:0/1577342250 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0a1413d6e0 con 0x7f0a141437c0 2023-08-27T11:25:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.654+0000 7f09fbfff700 1 -- 172.21.15.112:0/1577342250 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0a08017020 con 0x7f0a141437c0 2023-08-27T11:25:24.671 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:25:24.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.658+0000 7f0a19e17700 1 -- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a0006a320 msgr2=0x7f0a0006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.658+0000 7f0a19e17700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a0006a320 0x7f0a0006c7e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0a04007fd0 tx=0x7f0a04007dc0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.658+0000 7f0a19e17700 1 -- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a141437c0 msgr2=0x7f0a141f6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.658+0000 7f0a19e17700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a141437c0 0x7f0a141f6e90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800c3f0 tx=0x7f0a0800c700 comp rx=0 tx=0).stop 2023-08-27T11:25:24.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.659+0000 7f0a19e17700 1 -- 172.21.15.112:0/1577342250 shutdown_connections 2023-08-27T11:25:24.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.659+0000 7f0a19e17700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a0006a320 0x7f0a0006c7e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.659+0000 7f0a19e17700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a14143ba0 0x7f0a1406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.659+0000 7f0a19e17700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1406dcc0 0x7f0a14158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.659+0000 7f0a19e17700 1 --2- 172.21.15.112:0/1577342250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a141437c0 0x7f0a141f6e90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.659+0000 7f0a19e17700 1 -- 172.21.15.112:0/1577342250 >> 172.21.15.112:0/1577342250 conn(0x7f0a140b1e90 msgr2=0x7f0a14141960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.659+0000 7f0a19e17700 1 -- 172.21.15.112:0/1577342250 shutdown_connections 2023-08-27T11:25:24.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.659+0000 7f0a19e17700 1 -- 172.21.15.112:0/1577342250 wait complete. 2023-08-27T11:25:24.680 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:25:24.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.808+0000 7fe73f268700 1 Processor -- start 2023-08-27T11:25:24.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.808+0000 7fe73f268700 1 -- start start 2023-08-27T11:25:24.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.809+0000 7fe73f268700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe73813f470 0x7fe73815b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.809+0000 7fe73f268700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe738071510 0x7fe73813f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.809+0000 7fe73f268700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe73813fd90 0x7fe73806ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.809+0000 7fe73f268700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7381408f0 con 0x7fe738071510 2023-08-27T11:25:24.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.809+0000 7fe73f268700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73815a650 con 0x7fe73813fd90 2023-08-27T11:25:24.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.809+0000 7fe73f268700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73815abd0 con 0x7fe73813f470 2023-08-27T11:25:24.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe73d805700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe73813fd90 0x7fe73806ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe73d805700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe73813fd90 0x7fe73806ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:59914/0 (socket says 172.21.15.112:59914) 2023-08-27T11:25:24.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe73d805700 1 -- 172.21.15.112:0/759836583 learned_addr learned my addr 172.21.15.112:0/759836583 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:24.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe73c803700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe738071510 0x7fe73813f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe73d004700 1 --2- 172.21.15.112:0/759836583 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe73813f470 0x7fe73815b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe73d805700 1 -- 172.21.15.112:0/759836583 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe73813f470 msgr2=0x7fe73815b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:24.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe73d805700 1 --2- 172.21.15.112:0/759836583 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe73813f470 0x7fe73815b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe73d805700 1 -- 172.21.15.112:0/759836583 >> v2:172.21.15.112:3300/0 conn(0x7fe738071510 msgr2=0x7fe73813f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:24.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe73d805700 1 --2- 172.21.15.112:0/759836583 >> v2:172.21.15.112:3300/0 conn(0x7fe738071510 0x7fe73813f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe73d805700 1 -- 172.21.15.112:0/759836583 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe738140330 con 0x7fe73813fd90 2023-08-27T11:25:24.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe73d004700 1 --2- 172.21.15.112:0/759836583 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe73813f470 0x7fe73815b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:24.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe73c803700 1 --2- 172.21.15.112:0/759836583 >> v2:172.21.15.112:3300/0 conn(0x7fe738071510 0x7fe73813f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:24.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe73d805700 1 --2- 172.21.15.112:0/759836583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73813fd90 0x7fe73806ddb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe7380af4f0 tx=0x7fe73000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b119063aff7fadc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:24.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe72ffff700 1 -- 172.21.15.112:0/759836583 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73000e070 con 0x7fe73813fd90 2023-08-27T11:25:24.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe72ffff700 1 -- 172.21.15.112:0/759836583 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73000b910 con 0x7fe73813fd90 2023-08-27T11:25:24.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.810+0000 7fe72ffff700 1 -- 172.21.15.112:0/759836583 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73000bcc0 con 0x7fe73813fd90 2023-08-27T11:25:24.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.811+0000 7fe73f268700 1 -- 172.21.15.112:0/759836583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73813fd90 msgr2=0x7fe73806ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.811+0000 7fe73f268700 1 --2- 172.21.15.112:0/759836583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73813fd90 0x7fe73806ddb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe7380af4f0 tx=0x7fe73000b330 comp rx=0 tx=0).stop 2023-08-27T11:25:24.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.811+0000 7fe73f268700 1 -- 172.21.15.112:0/759836583 shutdown_connections 2023-08-27T11:25:24.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.811+0000 7fe73f268700 1 --2- 172.21.15.112:0/759836583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73813fd90 0x7fe73806ddb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.811+0000 7fe73f268700 1 --2- 172.21.15.112:0/759836583 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe73813f470 0x7fe73815b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.811+0000 7fe73f268700 1 --2- 172.21.15.112:0/759836583 >> v2:172.21.15.112:3300/0 conn(0x7fe738071510 0x7fe73813f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.811+0000 7fe73f268700 1 -- 172.21.15.112:0/759836583 >> 172.21.15.112:0/759836583 conn(0x7fe7380cc240 msgr2=0x7fe7380cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:24.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.811+0000 7fe73f268700 1 -- 172.21.15.112:0/759836583 shutdown_connections 2023-08-27T11:25:24.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.811+0000 7fe73f268700 1 -- 172.21.15.112:0/759836583 wait complete. 2023-08-27T11:25:24.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.812+0000 7fe73f268700 1 Processor -- start 2023-08-27T11:25:24.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.812+0000 7fe73f268700 1 -- start start 2023-08-27T11:25:24.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.813+0000 7fe73f268700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe738071510 0x7fe7381396d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.813+0000 7fe73f268700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73813f470 0x7fe73813cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.813+0000 7fe73f268700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe73813fd90 0x7fe738155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.813+0000 7fe73f268700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe738150fb0 con 0x7fe73813fd90 2023-08-27T11:25:24.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.813+0000 7fe73f268700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7381534e0 con 0x7fe73813f470 2023-08-27T11:25:24.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.813+0000 7fe73f268700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7381537c0 con 0x7fe738071510 2023-08-27T11:25:24.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.813+0000 7fe73d805700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe73813fd90 0x7fe738155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.813+0000 7fe73d805700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe73813fd90 0x7fe738155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59494/0 (socket says 172.21.15.112:59494) 2023-08-27T11:25:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.813+0000 7fe73d805700 1 -- 172.21.15.112:0/2867452238 learned_addr learned my addr 172.21.15.112:0/2867452238 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.813+0000 7fe73d004700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe738071510 0x7fe7381396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.813+0000 7fe73c803700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73813f470 0x7fe73813cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.814+0000 7fe73d805700 1 -- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe738071510 msgr2=0x7fe7381396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.814+0000 7fe73d805700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe738071510 0x7fe7381396d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.814+0000 7fe73d805700 1 -- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73813f470 msgr2=0x7fe73813cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.814+0000 7fe73d805700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73813f470 0x7fe73813cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.814+0000 7fe73d805700 1 -- 172.21.15.112:0/2867452238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7280094a0 con 0x7fe73813fd90 2023-08-27T11:25:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.814+0000 7fe73d805700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe73813fd90 0x7fe738155b20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe73815a790 tx=0x7fe730004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.814+0000 7fe72dffb700 1 -- 172.21.15.112:0/2867452238 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7300099b0 con 0x7fe73813fd90 2023-08-27T11:25:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.814+0000 7fe72dffb700 1 -- 172.21.15.112:0/2867452238 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73000ce80 con 0x7fe73813fd90 2023-08-27T11:25:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.814+0000 7fe73f268700 1 -- 172.21.15.112:0/2867452238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7340094a0 con 0x7fe73813fd90 2023-08-27T11:25:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.814+0000 7fe72dffb700 1 -- 172.21.15.112:0/2867452238 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730004470 con 0x7fe73813fd90 2023-08-27T11:25:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.814+0000 7fe73f268700 1 -- 172.21.15.112:0/2867452238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe730009b30 con 0x7fe73813fd90 2023-08-27T11:25:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.815+0000 7fe73f268700 1 -- 172.21.15.112:0/2867452238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe738067e90 con 0x7fe73813fd90 2023-08-27T11:25:24.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.816+0000 7fe72dffb700 1 -- 172.21.15.112:0/2867452238 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fe7300045d0 con 0x7fe73813fd90 2023-08-27T11:25:24.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.816+0000 7fe72dffb700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe71c06a320 0x7fe71c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.816+0000 7fe72dffb700 1 -- 172.21.15.112:0/2867452238 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe730089ad0 con 0x7fe73813fd90 2023-08-27T11:25:24.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.816+0000 7fe73d004700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe71c06a320 0x7fe71c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.816+0000 7fe73d004700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe71c06a320 0x7fe71c06c7e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe734009d70 tx=0x7fe734009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:24.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:24.820+0000 7fe72dffb700 1 -- 172.21.15.112:0/2867452238 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe730054e10 con 0x7fe73813fd90 2023-08-27T11:25:25.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.068+0000 7fe73f268700 1 -- 172.21.15.112:0/2867452238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7381405e0 con 0x7fe73813fd90 2023-08-27T11:25:25.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.068+0000 7fe72dffb700 1 -- 172.21.15.112:0/2867452238 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe730004880 con 0x7fe73813fd90 2023-08-27T11:25:25.070 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:25:25.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.071+0000 7fe73f268700 1 -- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe71c06a320 msgr2=0x7fe71c06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.071+0000 7fe73f268700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe71c06a320 0x7fe71c06c7e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe734009d70 tx=0x7fe734009210 comp rx=0 tx=0).stop 2023-08-27T11:25:25.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.071+0000 7fe73f268700 1 -- 172.21.15.112:0/2867452238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe73813fd90 msgr2=0x7fe738155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.071+0000 7fe73f268700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe73813fd90 0x7fe738155b20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe73815a790 tx=0x7fe730004ad0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.071+0000 7fe73f268700 1 -- 172.21.15.112:0/2867452238 shutdown_connections 2023-08-27T11:25:25.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.071+0000 7fe73f268700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe71c06a320 0x7fe71c06c7e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.071+0000 7fe73f268700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe73813fd90 0x7fe738155b20 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.071+0000 7fe73f268700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe738071510 0x7fe7381396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.071+0000 7fe73f268700 1 --2- 172.21.15.112:0/2867452238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73813f470 0x7fe73813cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.071+0000 7fe73f268700 1 -- 172.21.15.112:0/2867452238 >> 172.21.15.112:0/2867452238 conn(0x7fe7380cc240 msgr2=0x7fe73815b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:25.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.071+0000 7fe73f268700 1 -- 172.21.15.112:0/2867452238 shutdown_connections 2023-08-27T11:25:25.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.071+0000 7fe73f268700 1 -- 172.21.15.112:0/2867452238 wait complete. 2023-08-27T11:25:25.085 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:25:25.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.219+0000 7f3473b64700 1 Processor -- start 2023-08-27T11:25:25.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.219+0000 7f3473b64700 1 -- start start 2023-08-27T11:25:25.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.220+0000 7f3473b64700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346c1437c0 0x7f346c0cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.220+0000 7f3473b64700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f346c143ba0 0x7f346c06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.220+0000 7f3473b64700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346c0cf410 con 0x7f346c06dac0 2023-08-27T11:25:25.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.220+0000 7f3473b64700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f346c06df70 con 0x7f346c143ba0 2023-08-27T11:25:25.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.220+0000 7f3473b64700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c0cfde0 con 0x7f346c1437c0 2023-08-27T11:25:25.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.220+0000 7f34710ff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f346c143ba0 0x7f346c06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.221+0000 7f3472101700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f346c06dac0 0x7f346c0afcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:33176/0 (socket says 172.21.15.112:33176) 2023-08-27T11:25:25.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.221+0000 7f3472101700 1 -- 172.21.15.112:0/1009663046 learned_addr learned my addr 172.21.15.112:0/1009663046 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:25.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.221+0000 7f3471900700 1 --2- 172.21.15.112:0/1009663046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346c1437c0 0x7f346c0cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.221+0000 7f34710ff700 1 -- 172.21.15.112:0/1009663046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346c1437c0 msgr2=0x7f346c0cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:25.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.221+0000 7f34710ff700 1 --2- 172.21.15.112:0/1009663046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346c1437c0 0x7f346c0cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.221+0000 7f34710ff700 1 -- 172.21.15.112:0/1009663046 >> v1:172.21.15.138:6789/0 conn(0x7f346c06dac0 legacy=0x7f346c0afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.221+0000 7f34708fe700 1 -- 172.21.15.112:0/1009663046 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488003362 0 0) 0x7f346c0cf410 con 0x7f346c06dac0 2023-08-27T11:25:25.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.221+0000 7f34710ff700 1 -- 172.21.15.112:0/1009663046 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c0b09e0 con 0x7f346c143ba0 2023-08-27T11:25:25.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.221+0000 7f34710ff700 1 --2- 172.21.15.112:0/1009663046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346c143ba0 0x7f346c06d580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f345c009840 tx=0x7f345c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aea2f82d601dc5fa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:25.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.221+0000 7f34708fe700 1 -- 172.21.15.112:0/1009663046 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c011830 con 0x7f346c143ba0 2023-08-27T11:25:25.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.221+0000 7f34708fe700 1 -- 172.21.15.112:0/1009663046 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345c011990 con 0x7f346c143ba0 2023-08-27T11:25:25.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.221+0000 7f34708fe700 1 -- 172.21.15.112:0/1009663046 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c011d20 con 0x7f346c143ba0 2023-08-27T11:25:25.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.222+0000 7f3473b64700 1 -- 172.21.15.112:0/1009663046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346c143ba0 msgr2=0x7f346c06d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.222+0000 7f3473b64700 1 --2- 172.21.15.112:0/1009663046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346c143ba0 0x7f346c06d580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f345c009840 tx=0x7f345c009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:25.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.222+0000 7f3473b64700 1 -- 172.21.15.112:0/1009663046 shutdown_connections 2023-08-27T11:25:25.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.222+0000 7f3473b64700 1 --2- 172.21.15.112:0/1009663046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346c1437c0 0x7f346c0cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.222+0000 7f3473b64700 1 --2- 172.21.15.112:0/1009663046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346c143ba0 0x7f346c06d580 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.222+0000 7f3473b64700 1 -- 172.21.15.112:0/1009663046 >> 172.21.15.112:0/1009663046 conn(0x7f346c0b1e90 msgr2=0x7f346c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:25.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.222+0000 7f3473b64700 1 -- 172.21.15.112:0/1009663046 shutdown_connections 2023-08-27T11:25:25.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.222+0000 7f3473b64700 1 -- 172.21.15.112:0/1009663046 wait complete. 2023-08-27T11:25:25.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.223+0000 7f3473b64700 1 Processor -- start 2023-08-27T11:25:25.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.223+0000 7f3473b64700 1 -- start start 2023-08-27T11:25:25.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f3473b64700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346c06dac0 0x7f346c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f3473b64700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c1437c0 0x7f346c1eff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f3473b64700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346c143ba0 0x7f346c1f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f3473b64700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c138930 con 0x7f346c1437c0 2023-08-27T11:25:25.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f3473b64700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c0706e0 con 0x7f346c06dac0 2023-08-27T11:25:25.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f3473b64700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c1f4000 con 0x7f346c143ba0 2023-08-27T11:25:25.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f3471900700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346c06dac0 0x7f346c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f34710ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c1437c0 0x7f346c1eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f34710ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c1437c0 0x7f346c1eff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59506/0 (socket says 172.21.15.112:59506) 2023-08-27T11:25:25.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f3471900700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346c06dac0 0x7f346c0701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:59930/0 (socket says 172.21.15.112:59930) 2023-08-27T11:25:25.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f34710ff700 1 -- 172.21.15.112:0/914062110 learned_addr learned my addr 172.21.15.112:0/914062110 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:25.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f3472101700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346c143ba0 0x7f346c1f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f3471900700 1 -- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346c143ba0 msgr2=0x7f346c1f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.224+0000 7f3471900700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346c143ba0 0x7f346c1f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.225+0000 7f3471900700 1 -- 172.21.15.112:0/914062110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c1437c0 msgr2=0x7f346c1eff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.225+0000 7f3471900700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c1437c0 0x7f346c1eff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.225+0000 7f3471900700 1 -- 172.21.15.112:0/914062110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c0094a0 con 0x7f346c06dac0 2023-08-27T11:25:25.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.225+0000 7f34710ff700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c1437c0 0x7f346c1eff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:25.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.225+0000 7f3472101700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346c143ba0 0x7f346c1f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:25.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.225+0000 7f3471900700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346c06dac0 0x7f346c0701a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f346800e900 tx=0x7f346800ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:25.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.225+0000 7f34627fc700 1 -- 172.21.15.112:0/914062110 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468013680 con 0x7f346c06dac0 2023-08-27T11:25:25.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.225+0000 7f34627fc700 1 -- 172.21.15.112:0/914062110 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34680137e0 con 0x7f346c06dac0 2023-08-27T11:25:25.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.225+0000 7f3473b64700 1 -- 172.21.15.112:0/914062110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346c1f3cd0 con 0x7f346c06dac0 2023-08-27T11:25:25.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.225+0000 7f34627fc700 1 -- 172.21.15.112:0/914062110 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468013b50 con 0x7f346c06dac0 2023-08-27T11:25:25.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.225+0000 7f3473b64700 1 -- 172.21.15.112:0/914062110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346c138c40 con 0x7f346c06dac0 2023-08-27T11:25:25.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.226+0000 7f3473b64700 1 -- 172.21.15.112:0/914062110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346c1f42e0 con 0x7f346c06dac0 2023-08-27T11:25:25.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.226+0000 7f34627fc700 1 -- 172.21.15.112:0/914062110 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f3468013cb0 con 0x7f346c06dac0 2023-08-27T11:25:25.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.227+0000 7f34627fc700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f345006a320 0x7f345006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.227+0000 7f34627fc700 1 -- 172.21.15.112:0/914062110 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3468088be0 con 0x7f346c06dac0 2023-08-27T11:25:25.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.227+0000 7f34710ff700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f345006a320 0x7f345006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.227+0000 7f34710ff700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f345006a320 0x7f345006c7e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f345c009b90 tx=0x7f345c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:25.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.232+0000 7f34627fc700 1 -- 172.21.15.112:0/914062110 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3468057a00 con 0x7f346c06dac0 2023-08-27T11:25:25.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.433+0000 7f3473b64700 1 -- 172.21.15.112:0/914062110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f346c067a20 con 0x7f346c06dac0 2023-08-27T11:25:25.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.434+0000 7f34627fc700 1 -- 172.21.15.112:0/914062110 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3468057590 con 0x7f346c06dac0 2023-08-27T11:25:25.436 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:25:25.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.436+0000 7f3473b64700 1 -- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f345006a320 msgr2=0x7f345006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.436+0000 7f3473b64700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f345006a320 0x7f345006c7e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f345c009b90 tx=0x7f345c008030 comp rx=0 tx=0).stop 2023-08-27T11:25:25.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.436+0000 7f3473b64700 1 -- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346c06dac0 msgr2=0x7f346c0701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.436+0000 7f3473b64700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346c06dac0 0x7f346c0701a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f346800e900 tx=0x7f346800ec10 comp rx=0 tx=0).stop 2023-08-27T11:25:25.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.436+0000 7f3473b64700 1 -- 172.21.15.112:0/914062110 shutdown_connections 2023-08-27T11:25:25.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.436+0000 7f3473b64700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f345006a320 0x7f345006c7e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.436+0000 7f3473b64700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346c143ba0 0x7f346c1f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.436+0000 7f3473b64700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346c06dac0 0x7f346c0701a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.436+0000 7f3473b64700 1 --2- 172.21.15.112:0/914062110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c1437c0 0x7f346c1eff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.436+0000 7f3473b64700 1 -- 172.21.15.112:0/914062110 >> 172.21.15.112:0/914062110 conn(0x7f346c0b1e90 msgr2=0x7f346c06ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:25.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.436+0000 7f3473b64700 1 -- 172.21.15.112:0/914062110 shutdown_connections 2023-08-27T11:25:25.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.436+0000 7f3473b64700 1 -- 172.21.15.112:0/914062110 wait complete. 2023-08-27T11:25:25.450 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:25:25.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.581+0000 7f4eb103e700 1 Processor -- start 2023-08-27T11:25:25.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.581+0000 7f4eb103e700 1 -- start start 2023-08-27T11:25:25.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eb103e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4eac143ba0 0x7f4eac0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eb103e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4eac06d5b0 0x7f4eac06da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eb103e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eac0cfd50 con 0x7f4eac1437c0 2023-08-27T11:25:25.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eb103e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eac06df40 con 0x7f4eac06d5b0 2023-08-27T11:25:25.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eb103e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac0cf140 con 0x7f4eac143ba0 2023-08-27T11:25:25.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eaa59c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4eac143ba0 0x7f4eac0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eaad9d700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4eac1437c0 0x7f4eac0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:33186/0 (socket says 172.21.15.112:33186) 2023-08-27T11:25:25.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eab59e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4eac06d5b0 0x7f4eac06da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eaad9d700 1 -- 172.21.15.112:0/3412370009 learned_addr learned my addr 172.21.15.112:0/3412370009 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:25.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4ea9d9b700 1 -- 172.21.15.112:0/3412370009 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124942342 0 0) 0x7f4eac0cfd50 con 0x7f4eac1437c0 2023-08-27T11:25:25.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4ea9d9b700 1 -- 172.21.15.112:0/3412370009 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e980035b0 con 0x7f4eac1437c0 2023-08-27T11:25:25.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eab59e700 1 -- 172.21.15.112:0/3412370009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4eac143ba0 msgr2=0x7f4eac0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:25.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eab59e700 1 --2- 172.21.15.112:0/3412370009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4eac143ba0 0x7f4eac0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eab59e700 1 -- 172.21.15.112:0/3412370009 >> v1:172.21.15.138:6789/0 conn(0x7f4eac1437c0 legacy=0x7f4eac0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eab59e700 1 -- 172.21.15.112:0/3412370009 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac0cf420 con 0x7f4eac06d5b0 2023-08-27T11:25:25.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eaa59c700 1 --2- 172.21.15.112:0/3412370009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4eac143ba0 0x7f4eac0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:25.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4eab59e700 1 --2- 172.21.15.112:0/3412370009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eac06d5b0 0x7f4eac06da00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009840 tx=0x7f4e9c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d469eb2719f658a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:25.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4ea9d9b700 1 -- 172.21.15.112:0/3412370009 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c011830 con 0x7f4eac06d5b0 2023-08-27T11:25:25.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.582+0000 7f4ea9d9b700 1 -- 172.21.15.112:0/3412370009 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c011990 con 0x7f4eac06d5b0 2023-08-27T11:25:25.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.583+0000 7f4ea9d9b700 1 -- 172.21.15.112:0/3412370009 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c011d20 con 0x7f4eac06d5b0 2023-08-27T11:25:25.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.583+0000 7f4eb103e700 1 -- 172.21.15.112:0/3412370009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eac06d5b0 msgr2=0x7f4eac06da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.583+0000 7f4eb103e700 1 --2- 172.21.15.112:0/3412370009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eac06d5b0 0x7f4eac06da00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009840 tx=0x7f4e9c009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:25.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.583+0000 7f4eb103e700 1 -- 172.21.15.112:0/3412370009 shutdown_connections 2023-08-27T11:25:25.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.583+0000 7f4eb103e700 1 --2- 172.21.15.112:0/3412370009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eac06d5b0 0x7f4eac06da00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.583+0000 7f4eb103e700 1 --2- 172.21.15.112:0/3412370009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4eac143ba0 0x7f4eac0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.583+0000 7f4eb103e700 1 -- 172.21.15.112:0/3412370009 >> 172.21.15.112:0/3412370009 conn(0x7f4eac0b1e90 msgr2=0x7f4eac0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:25.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.583+0000 7f4eb103e700 1 -- 172.21.15.112:0/3412370009 shutdown_connections 2023-08-27T11:25:25.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.583+0000 7f4eb103e700 1 -- 172.21.15.112:0/3412370009 wait complete. 2023-08-27T11:25:25.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.585+0000 7f4eb103e700 1 Processor -- start 2023-08-27T11:25:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.585+0000 7f4eb103e700 1 -- start start 2023-08-27T11:25:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.585+0000 7f4eb103e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4eac06d5b0 0x7f4eac157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.585+0000 7f4eb103e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac1437c0 0x7f4eac13a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.585+0000 7f4eb103e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eac143ba0 0x7f4eac0cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.585+0000 7f4eb103e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac13ae50 con 0x7f4eac1437c0 2023-08-27T11:25:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.585+0000 7f4eb103e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac13b130 con 0x7f4eac143ba0 2023-08-27T11:25:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.585+0000 7f4eb103e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac13b410 con 0x7f4eac06d5b0 2023-08-27T11:25:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4eaa59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac1437c0 0x7f4eac13a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4eaa59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac1437c0 0x7f4eac13a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59520/0 (socket says 172.21.15.112:59520) 2023-08-27T11:25:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4eaa59c700 1 -- 172.21.15.112:0/2996591480 learned_addr learned my addr 172.21.15.112:0/2996591480 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4eab59e700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eac143ba0 0x7f4eac0cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4eaad9d700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4eac06d5b0 0x7f4eac157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4eaa59c700 1 -- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4eac06d5b0 msgr2=0x7f4eac157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4eaa59c700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4eac06d5b0 0x7f4eac157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4eaa59c700 1 -- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eac143ba0 msgr2=0x7f4eac0cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4eaa59c700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eac143ba0 0x7f4eac0cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4eaa59c700 1 -- 172.21.15.112:0/2996591480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e940094a0 con 0x7f4eac1437c0 2023-08-27T11:25:25.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4eaa59c700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac1437c0 0x7f4eac13a3b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400ebe0 tx=0x7f4e9400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:25.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4ea37fe700 1 -- 172.21.15.112:0/2996591480 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94009760 con 0x7f4eac1437c0 2023-08-27T11:25:25.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4ea37fe700 1 -- 172.21.15.112:0/2996591480 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9400c850 con 0x7f4eac1437c0 2023-08-27T11:25:25.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4eb103e700 1 -- 172.21.15.112:0/2996591480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c0094a0 con 0x7f4eac1437c0 2023-08-27T11:25:25.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4ea37fe700 1 -- 172.21.15.112:0/2996591480 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9400cbe0 con 0x7f4eac1437c0 2023-08-27T11:25:25.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.586+0000 7f4eb103e700 1 -- 172.21.15.112:0/2996591480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eac13b9d0 con 0x7f4eac1437c0 2023-08-27T11:25:25.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.587+0000 7f4eb103e700 1 -- 172.21.15.112:0/2996591480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eac13bd40 con 0x7f4eac1437c0 2023-08-27T11:25:25.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.588+0000 7f4ea37fe700 1 -- 172.21.15.112:0/2996591480 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f4e9400cd40 con 0x7f4eac1437c0 2023-08-27T11:25:25.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.588+0000 7f4ea37fe700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e8c06a320 0x7f4e8c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.589+0000 7f4ea37fe700 1 -- 172.21.15.112:0/2996591480 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e94088990 con 0x7f4eac1437c0 2023-08-27T11:25:25.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.589+0000 7f4eaad9d700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e8c06a320 0x7f4e8c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.589+0000 7f4eaad9d700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e8c06a320 0x7f4e8c06c7e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4001560 tx=0x7f4ea4001a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:25.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.595+0000 7f4ea37fe700 1 -- 172.21.15.112:0/2996591480 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4e94053d10 con 0x7f4eac1437c0 2023-08-27T11:25:25.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.805+0000 7f4eb103e700 1 -- 172.21.15.112:0/2996591480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4eac067e90 con 0x7f4eac1437c0 2023-08-27T11:25:25.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.806+0000 7f4ea37fe700 1 -- 172.21.15.112:0/2996591480 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e940210a0 con 0x7f4eac1437c0 2023-08-27T11:25:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:25:25.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.809+0000 7f4eb103e700 1 -- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e8c06a320 msgr2=0x7f4e8c06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.809+0000 7f4eb103e700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e8c06a320 0x7f4e8c06c7e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4001560 tx=0x7f4ea4001a90 comp rx=0 tx=0).stop 2023-08-27T11:25:25.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.809+0000 7f4eb103e700 1 -- 172.21.15.112:0/2996591480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac1437c0 msgr2=0x7f4eac13a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.809+0000 7f4eb103e700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac1437c0 0x7f4eac13a3b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400ebe0 tx=0x7f4e9400eef0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.809+0000 7f4eb103e700 1 -- 172.21.15.112:0/2996591480 shutdown_connections 2023-08-27T11:25:25.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.809+0000 7f4eb103e700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e8c06a320 0x7f4e8c06c7e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.809+0000 7f4eb103e700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eac143ba0 0x7f4eac0cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.809+0000 7f4eb103e700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4eac06d5b0 0x7f4eac157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.809+0000 7f4eb103e700 1 --2- 172.21.15.112:0/2996591480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac1437c0 0x7f4eac13a3b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.809+0000 7f4eb103e700 1 -- 172.21.15.112:0/2996591480 >> 172.21.15.112:0/2996591480 conn(0x7f4eac0b1e90 msgr2=0x7f4eac070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:25.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.809+0000 7f4eb103e700 1 -- 172.21.15.112:0/2996591480 shutdown_connections 2023-08-27T11:25:25.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.809+0000 7f4eb103e700 1 -- 172.21.15.112:0/2996591480 wait complete. 2023-08-27T11:25:25.823 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:25:25.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.945+0000 7fb199222700 1 Processor -- start 2023-08-27T11:25:25.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.945+0000 7fb199222700 1 -- start start 2023-08-27T11:25:25.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.946+0000 7fb199222700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb19406fea0 0x7fb1940b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.946+0000 7fb199222700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19406e3a0 0x7fb1940af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.946+0000 7fb199222700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb19415c190 con 0x7fb194158e20 2023-08-27T11:25:25.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.946+0000 7fb199222700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1940b1730 con 0x7fb19406fea0 2023-08-27T11:25:25.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.946+0000 7fb199222700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb194159200 con 0x7fb19406e3a0 2023-08-27T11:25:25.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.946+0000 7fb19259c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb19406fea0 0x7fb1940b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.946+0000 7fb19259c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb19406fea0 0x7fb1940b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59528/0 (socket says 172.21.15.112:59528) 2023-08-27T11:25:25.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.946+0000 7fb19259c700 1 -- 172.21.15.112:0/400509937 learned_addr learned my addr 172.21.15.112:0/400509937 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:25.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.946+0000 7fb19359e700 1 --2- 172.21.15.112:0/400509937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19406e3a0 0x7fb1940af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.946+0000 7fb19259c700 1 -- 172.21.15.112:0/400509937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19406e3a0 msgr2=0x7fb1940af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:25.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.946+0000 7fb19259c700 1 --2- 172.21.15.112:0/400509937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19406e3a0 0x7fb1940af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.946+0000 7fb19259c700 1 -- 172.21.15.112:0/400509937 >> v1:172.21.15.138:6789/0 conn(0x7fb194158e20 legacy=0x7fb19406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:25.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.946+0000 7fb19259c700 1 -- 172.21.15.112:0/400509937 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1940b1910 con 0x7fb19406fea0 2023-08-27T11:25:25.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.947+0000 7fb19259c700 1 --2- 172.21.15.112:0/400509937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb19406fea0 0x7fb1940b11f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb1880098f0 tx=0x7fb188009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6946ffb3436036f9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:25.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.947+0000 7fb191d9b700 1 -- 172.21.15.112:0/400509937 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188004230 con 0x7fb19406fea0 2023-08-27T11:25:25.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.947+0000 7fb191d9b700 1 -- 172.21.15.112:0/400509937 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb188004390 con 0x7fb19406fea0 2023-08-27T11:25:25.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.947+0000 7fb191d9b700 1 -- 172.21.15.112:0/400509937 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1880036a0 con 0x7fb19406fea0 2023-08-27T11:25:25.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.947+0000 7fb199222700 1 -- 172.21.15.112:0/400509937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb19406fea0 msgr2=0x7fb1940b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.947+0000 7fb199222700 1 --2- 172.21.15.112:0/400509937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb19406fea0 0x7fb1940b11f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb1880098f0 tx=0x7fb188009c00 comp rx=0 tx=0).stop 2023-08-27T11:25:25.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.947+0000 7fb199222700 1 -- 172.21.15.112:0/400509937 shutdown_connections 2023-08-27T11:25:25.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.947+0000 7fb199222700 1 --2- 172.21.15.112:0/400509937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19406e3a0 0x7fb1940af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.947+0000 7fb199222700 1 --2- 172.21.15.112:0/400509937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb19406fea0 0x7fb1940b11f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.947+0000 7fb199222700 1 -- 172.21.15.112:0/400509937 >> 172.21.15.112:0/400509937 conn(0x7fb1940d0000 msgr2=0x7fb1940d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:25.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.947+0000 7fb199222700 1 -- 172.21.15.112:0/400509937 shutdown_connections 2023-08-27T11:25:25.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.947+0000 7fb199222700 1 -- 172.21.15.112:0/400509937 wait complete. 2023-08-27T11:25:25.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.949+0000 7fb199222700 1 Processor -- start 2023-08-27T11:25:25.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.949+0000 7fb199222700 1 -- start start 2023-08-27T11:25:25.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.950+0000 7fb199222700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb19406e3a0 0x7fb194152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.950+0000 7fb199222700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19406fea0 0x7fb1940c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.950+0000 7fb199222700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb194158e20 0x7fb1940c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.950+0000 7fb199222700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb194156030 con 0x7fb19406e3a0 2023-08-27T11:25:25.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.950+0000 7fb199222700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940c8810 con 0x7fb194158e20 2023-08-27T11:25:25.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.950+0000 7fb199222700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb194156810 con 0x7fb19406fea0 2023-08-27T11:25:25.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.950+0000 7fb192d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb19406e3a0 0x7fb194152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.950+0000 7fb19259c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19406fea0 0x7fb1940c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.950+0000 7fb192d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb19406e3a0 0x7fb194152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59532/0 (socket says 172.21.15.112:59532) 2023-08-27T11:25:25.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.950+0000 7fb192d9d700 1 -- 172.21.15.112:0/3173952531 learned_addr learned my addr 172.21.15.112:0/3173952531 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:25.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.951+0000 7fb19359e700 1 --2- 172.21.15.112:0/3173952531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb194158e20 0x7fb1940c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.951+0000 7fb192d9d700 1 -- 172.21.15.112:0/3173952531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19406fea0 msgr2=0x7fb1940c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.951+0000 7fb192d9d700 1 --2- 172.21.15.112:0/3173952531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19406fea0 0x7fb1940c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.951+0000 7fb192d9d700 1 -- 172.21.15.112:0/3173952531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb194158e20 msgr2=0x7fb1940c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.951+0000 7fb192d9d700 1 --2- 172.21.15.112:0/3173952531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb194158e20 0x7fb1940c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.951+0000 7fb192d9d700 1 -- 172.21.15.112:0/3173952531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb188009550 con 0x7fb19406e3a0 2023-08-27T11:25:25.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.951+0000 7fb192d9d700 1 --2- 172.21.15.112:0/3173952531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb19406e3a0 0x7fb194152f60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb18400b4a0 tx=0x7fb18400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:25.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.951+0000 7fb17b7fe700 1 -- 172.21.15.112:0/3173952531 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184011830 con 0x7fb19406e3a0 2023-08-27T11:25:25.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.951+0000 7fb199222700 1 -- 172.21.15.112:0/3173952531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1940c4d70 con 0x7fb19406e3a0 2023-08-27T11:25:25.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.951+0000 7fb17b7fe700 1 -- 172.21.15.112:0/3173952531 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb184011990 con 0x7fb19406e3a0 2023-08-27T11:25:25.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.951+0000 7fb17b7fe700 1 -- 172.21.15.112:0/3173952531 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184011d40 con 0x7fb19406e3a0 2023-08-27T11:25:25.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.951+0000 7fb199222700 1 -- 172.21.15.112:0/3173952531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1940c8250 con 0x7fb19406e3a0 2023-08-27T11:25:25.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.952+0000 7fb199222700 1 -- 172.21.15.112:0/3173952531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb194067a20 con 0x7fb19406e3a0 2023-08-27T11:25:25.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.953+0000 7fb17b7fe700 1 -- 172.21.15.112:0/3173952531 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fb18401a400 con 0x7fb19406e3a0 2023-08-27T11:25:25.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.953+0000 7fb17b7fe700 1 --2- 172.21.15.112:0/3173952531 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb18006a320 0x7fb18006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.953+0000 7fb17b7fe700 1 -- 172.21.15.112:0/3173952531 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb184088510 con 0x7fb19406e3a0 2023-08-27T11:25:25.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.953+0000 7fb19259c700 1 --2- 172.21.15.112:0/3173952531 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb18006a320 0x7fb18006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.954+0000 7fb19259c700 1 --2- 172.21.15.112:0/3173952531 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb18006a320 0x7fb18006c7e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb188009130 tx=0x7fb188008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:25.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:25.957+0000 7fb17b7fe700 1 -- 172.21.15.112:0/3173952531 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb1840538b0 con 0x7fb19406e3a0 2023-08-27T11:25:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.209+0000 7fb199222700 1 -- 172.21.15.112:0/3173952531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb194071da0 con 0x7fb19406e3a0 2023-08-27T11:25:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.209+0000 7fb17b7fe700 1 -- 172.21.15.112:0/3173952531 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4886 (secure 0 0 0) 0x7fb184056ed0 con 0x7fb19406e3a0 2023-08-27T11:25:26.211 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:26.211 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2906075444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2906075444},{"type":"v1","addr":"172.21.15.138:6836","nonce":2906075444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5761,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1781004436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1781004436},{"type":"v1","addr":"172.21.15.138:6837","nonce":1781004436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1573349448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1573349448},{"type":"v1","addr":"172.21.15.112:6834","nonce":1573349448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:19:16.064335+0000","modified":"2023-08-27T11:19:19.690233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/174707415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":174707415},{"type":"v1","addr":"172.21.15.112:6835","nonce":174707415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-27T11:25:26.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.211+0000 7fb199222700 1 -- 172.21.15.112:0/3173952531 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb18006a320 msgr2=0x7fb18006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:26.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.211+0000 7fb199222700 1 --2- 172.21.15.112:0/3173952531 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb18006a320 0x7fb18006c7e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb188009130 tx=0x7fb188008030 comp rx=0 tx=0).stop 2023-08-27T11:25:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.211+0000 7fb199222700 1 -- 172.21.15.112:0/3173952531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb19406e3a0 msgr2=0x7fb194152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.211+0000 7fb199222700 1 --2- 172.21.15.112:0/3173952531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb19406e3a0 0x7fb194152f60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb18400b4a0 tx=0x7fb18400b7b0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.211+0000 7fb199222700 1 -- 172.21.15.112:0/3173952531 shutdown_connections 2023-08-27T11:25:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.211+0000 7fb199222700 1 --2- 172.21.15.112:0/3173952531 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb18006a320 0x7fb18006c7e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.211+0000 7fb199222700 1 --2- 172.21.15.112:0/3173952531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb194158e20 0x7fb1940c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.211+0000 7fb199222700 1 --2- 172.21.15.112:0/3173952531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb19406e3a0 0x7fb194152f60 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.211+0000 7fb199222700 1 --2- 172.21.15.112:0/3173952531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb19406fea0 0x7fb1940c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.211+0000 7fb199222700 1 -- 172.21.15.112:0/3173952531 >> 172.21.15.112:0/3173952531 conn(0x7fb1940d0000 msgr2=0x7fb19415e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:26.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.212+0000 7fb199222700 1 -- 172.21.15.112:0/3173952531 shutdown_connections 2023-08-27T11:25:26.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.212+0000 7fb199222700 1 -- 172.21.15.112:0/3173952531 wait complete. 2023-08-27T11:25:26.216 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 40 2023-08-27T11:25:26.226 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.350+0000 7f34661f7700 1 Processor -- start 2023-08-27T11:25:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.350+0000 7f34661f7700 1 -- start start 2023-08-27T11:25:26.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.351+0000 7f34661f7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346006fea0 0x7f34600b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:26.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.351+0000 7f34661f7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f346006e3a0 0x7f34600af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:26.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.351+0000 7f34661f7700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346015c190 con 0x7f3460158e20 2023-08-27T11:25:26.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.351+0000 7f34661f7700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34600b1730 con 0x7f346006e3a0 2023-08-27T11:25:26.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.351+0000 7f34661f7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3460159200 con 0x7f346006fea0 2023-08-27T11:25:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.351+0000 7f345f7fe700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3460158e20 0x7f346006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:33214/0 (socket says 172.21.15.112:33214) 2023-08-27T11:25:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.351+0000 7f345effd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346006fea0 0x7f34600b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.351+0000 7f345f7fe700 1 -- 172.21.15.112:0/139034936 learned_addr learned my addr 172.21.15.112:0/139034936 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.351+0000 7f345ffff700 1 --2- 172.21.15.112:0/139034936 >> v2:172.21.15.138:3300/0 conn(0x7f346006e3a0 0x7f34600af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.352+0000 7f345e7fc700 1 -- 172.21.15.112:0/139034936 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1479485037 0 0) 0x7f346015c190 con 0x7f3460158e20 2023-08-27T11:25:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.352+0000 7f345e7fc700 1 -- 172.21.15.112:0/139034936 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34480035b0 con 0x7f3460158e20 2023-08-27T11:25:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.352+0000 7f345effd700 1 -- 172.21.15.112:0/139034936 >> v2:172.21.15.138:3300/0 conn(0x7f346006e3a0 msgr2=0x7f34600af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.352+0000 7f345effd700 1 --2- 172.21.15.112:0/139034936 >> v2:172.21.15.138:3300/0 conn(0x7f346006e3a0 0x7f34600af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.352+0000 7f345effd700 1 -- 172.21.15.112:0/139034936 >> v1:172.21.15.138:6789/0 conn(0x7f3460158e20 legacy=0x7f346006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.352+0000 7f345effd700 1 -- 172.21.15.112:0/139034936 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34600b1910 con 0x7f346006fea0 2023-08-27T11:25:26.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.352+0000 7f345ffff700 1 --2- 172.21.15.112:0/139034936 >> v2:172.21.15.138:3300/0 conn(0x7f346006e3a0 0x7f34600af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:26.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.352+0000 7f345effd700 1 --2- 172.21.15.112:0/139034936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346006fea0 0x7f34600b11f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3454009840 tx=0x7f3454009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb496c303599eb6a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:26.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.352+0000 7f345e7fc700 1 -- 172.21.15.112:0/139034936 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454011830 con 0x7f346006fea0 2023-08-27T11:25:26.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.352+0000 7f345e7fc700 1 -- 172.21.15.112:0/139034936 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3454011990 con 0x7f346006fea0 2023-08-27T11:25:26.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.352+0000 7f345e7fc700 1 -- 172.21.15.112:0/139034936 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454011d20 con 0x7f346006fea0 2023-08-27T11:25:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.353+0000 7f34661f7700 1 -- 172.21.15.112:0/139034936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346006fea0 msgr2=0x7f34600b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.353+0000 7f34661f7700 1 --2- 172.21.15.112:0/139034936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346006fea0 0x7f34600b11f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3454009840 tx=0x7f3454009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.353+0000 7f34661f7700 1 -- 172.21.15.112:0/139034936 shutdown_connections 2023-08-27T11:25:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.353+0000 7f34661f7700 1 --2- 172.21.15.112:0/139034936 >> v2:172.21.15.138:3300/0 conn(0x7f346006e3a0 0x7f34600af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.353+0000 7f34661f7700 1 --2- 172.21.15.112:0/139034936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346006fea0 0x7f34600b11f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.353+0000 7f34661f7700 1 -- 172.21.15.112:0/139034936 >> 172.21.15.112:0/139034936 conn(0x7f34600d0000 msgr2=0x7f34600d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:26.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.353+0000 7f34661f7700 1 -- 172.21.15.112:0/139034936 shutdown_connections 2023-08-27T11:25:26.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.353+0000 7f34661f7700 1 -- 172.21.15.112:0/139034936 wait complete. 2023-08-27T11:25:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.355+0000 7f34661f7700 1 Processor -- start 2023-08-27T11:25:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.355+0000 7f34661f7700 1 -- start start 2023-08-27T11:25:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f34661f7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346006e3a0 0x7f34600c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f34661f7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346006fea0 0x7f3460154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f34661f7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3460158e20 0x7f34601548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f34661f7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34601f51e0 con 0x7f3460158e20 2023-08-27T11:25:26.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f34661f7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34601f4d50 con 0x7f346006fea0 2023-08-27T11:25:26.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f34661f7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34601f4420 con 0x7f346006e3a0 2023-08-27T11:25:26.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f345ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3460158e20 0x7f34601548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:26.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f345ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3460158e20 0x7f34601548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59550/0 (socket says 172.21.15.112:59550) 2023-08-27T11:25:26.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f345ffff700 1 -- 172.21.15.112:0/1840480477 learned_addr learned my addr 172.21.15.112:0/1840480477 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f345f7fe700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346006e3a0 0x7f34600c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f345effd700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346006fea0 0x7f3460154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f345ffff700 1 -- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346006e3a0 msgr2=0x7f34600c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f345ffff700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346006e3a0 0x7f34600c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f345ffff700 1 -- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346006fea0 msgr2=0x7f3460154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f345ffff700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346006fea0 0x7f3460154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f345ffff700 1 -- 172.21.15.112:0/1840480477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3458009a70 con 0x7f3460158e20 2023-08-27T11:25:26.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.356+0000 7f345ffff700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3460158e20 0x7f34601548b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f345800efc0 tx=0x7f345800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:26.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.357+0000 7f3447fff700 1 -- 172.21.15.112:0/1840480477 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345800ca90 con 0x7f3460158e20 2023-08-27T11:25:26.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.357+0000 7f3447fff700 1 -- 172.21.15.112:0/1840480477 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345800f040 con 0x7f3460158e20 2023-08-27T11:25:26.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.357+0000 7f3447fff700 1 -- 172.21.15.112:0/1840480477 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345800ce10 con 0x7f3460158e20 2023-08-27T11:25:26.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.357+0000 7f34661f7700 1 -- 172.21.15.112:0/1840480477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34540094a0 con 0x7f3460158e20 2023-08-27T11:25:26.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.357+0000 7f34661f7700 1 -- 172.21.15.112:0/1840480477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34601f49e0 con 0x7f3460158e20 2023-08-27T11:25:26.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.358+0000 7f34661f7700 1 -- 172.21.15.112:0/1840480477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34601f4ee0 con 0x7f3460158e20 2023-08-27T11:25:26.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.358+0000 7f3447fff700 1 -- 172.21.15.112:0/1840480477 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f3458014480 con 0x7f3460158e20 2023-08-27T11:25:26.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.359+0000 7f3447fff700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f344c06a320 0x7f344c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:26.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.359+0000 7f3447fff700 1 -- 172.21.15.112:0/1840480477 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3458089110 con 0x7f3460158e20 2023-08-27T11:25:26.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.359+0000 7f345f7fe700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f344c06a320 0x7f344c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:26.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.359+0000 7f345f7fe700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f344c06a320 0x7f344c06c7e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3450007fd0 tx=0x7f3450007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:26.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.365+0000 7f3447fff700 1 -- 172.21.15.112:0/1840480477 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3458057e20 con 0x7f3460158e20 2023-08-27T11:25:26.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.574+0000 7f34661f7700 1 -- 172.21.15.112:0/1840480477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3460067a20 con 0x7f3460158e20 2023-08-27T11:25:26.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.575+0000 7f3447fff700 1 -- 172.21.15.112:0/1840480477 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18320 (secure 0 0 0) 0x7f3458009d70 con 0x7f3460158e20 2023-08-27T11:25:26.577 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:26.577 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":65,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:19:17.239518+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:19:14.179813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-27T11:19:15.021524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:26.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.577+0000 7f34661f7700 1 -- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f344c06a320 msgr2=0x7f344c06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:26.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.577+0000 7f34661f7700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f344c06a320 0x7f344c06c7e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3450007fd0 tx=0x7f3450007dc0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.577+0000 7f34661f7700 1 -- 172.21.15.112:0/1840480477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3460158e20 msgr2=0x7f34601548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:26.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.577+0000 7f34661f7700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3460158e20 0x7f34601548b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f345800efc0 tx=0x7f345800c3a0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.577+0000 7f34661f7700 1 -- 172.21.15.112:0/1840480477 shutdown_connections 2023-08-27T11:25:26.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.577+0000 7f34661f7700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f344c06a320 0x7f344c06c7e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.577+0000 7f34661f7700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3460158e20 0x7f34601548b0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.577+0000 7f34661f7700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f346006e3a0 0x7f34600c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.577+0000 7f34661f7700 1 --2- 172.21.15.112:0/1840480477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346006fea0 0x7f3460154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.577+0000 7f34661f7700 1 -- 172.21.15.112:0/1840480477 >> 172.21.15.112:0/1840480477 conn(0x7f34600d0000 msgr2=0x7f346015ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:26.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.577+0000 7f34661f7700 1 -- 172.21.15.112:0/1840480477 shutdown_connections 2023-08-27T11:25:26.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.577+0000 7f34661f7700 1 -- 172.21.15.112:0/1840480477 wait complete. 2023-08-27T11:25:26.592 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:25:26.592 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:25:26.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.719+0000 7fc5d42e1700 1 Processor -- start 2023-08-27T11:25:26.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.719+0000 7fc5d42e1700 1 -- start start 2023-08-27T11:25:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.720+0000 7fc5d42e1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc5cc144570 0x7fc5cc15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.720+0000 7fc5d42e1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5cc0b4ad0 0x7fc5cc0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.720+0000 7fc5d42e1700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5cc0b45f0 con 0x7fc5cc0b3e50 2023-08-27T11:25:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.720+0000 7fc5d42e1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5cc0cf7a0 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.720+0000 7fc5d42e1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5cc144950 con 0x7fc5cc144570 2023-08-27T11:25:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.720+0000 7fc5d287e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5cc0b4ad0 0x7fc5cc0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.720+0000 7fc5d287e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5cc0b4ad0 0x7fc5cc0b4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59556/0 (socket says 172.21.15.112:59556) 2023-08-27T11:25:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.720+0000 7fc5d287e700 1 -- 172.21.15.112:0/3814913463 learned_addr learned my addr 172.21.15.112:0/3814913463 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.720+0000 7fc5d207d700 1 --2- 172.21.15.112:0/3814913463 >> v2:172.21.15.138:3300/0 conn(0x7fc5cc144570 0x7fc5cc15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.720+0000 7fc5d287e700 1 -- 172.21.15.112:0/3814913463 >> v2:172.21.15.138:3300/0 conn(0x7fc5cc144570 msgr2=0x7fc5cc15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.720+0000 7fc5d287e700 1 --2- 172.21.15.112:0/3814913463 >> v2:172.21.15.138:3300/0 conn(0x7fc5cc144570 0x7fc5cc15ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.720+0000 7fc5d287e700 1 -- 172.21.15.112:0/3814913463 >> v1:172.21.15.138:6789/0 conn(0x7fc5cc0b3e50 legacy=0x7fc5cc15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.720+0000 7fc5d287e700 1 -- 172.21.15.112:0/3814913463 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc0b4310 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.721+0000 7fc5d287e700 1 --2- 172.21.15.112:0/3814913463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc0b4ad0 0x7fc5cc0b4f20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc06b7b0 tx=0x7fc5c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773befd7a3109efd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.721+0000 7fc5d107b700 1 -- 172.21.15.112:0/3814913463 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400e070 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.721+0000 7fc5d107b700 1 -- 172.21.15.112:0/3814913463 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c400b8d0 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.721+0000 7fc5d107b700 1 -- 172.21.15.112:0/3814913463 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400bc80 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.721+0000 7fc5d42e1700 1 -- 172.21.15.112:0/3814913463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc0b4ad0 msgr2=0x7fc5cc0b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.721+0000 7fc5d42e1700 1 --2- 172.21.15.112:0/3814913463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc0b4ad0 0x7fc5cc0b4f20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc06b7b0 tx=0x7fc5c400b330 comp rx=0 tx=0).stop 2023-08-27T11:25:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.721+0000 7fc5d42e1700 1 -- 172.21.15.112:0/3814913463 shutdown_connections 2023-08-27T11:25:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.721+0000 7fc5d42e1700 1 --2- 172.21.15.112:0/3814913463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc0b4ad0 0x7fc5cc0b4f20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.721+0000 7fc5d42e1700 1 --2- 172.21.15.112:0/3814913463 >> v2:172.21.15.138:3300/0 conn(0x7fc5cc144570 0x7fc5cc15ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.721+0000 7fc5d42e1700 1 -- 172.21.15.112:0/3814913463 >> 172.21.15.112:0/3814913463 conn(0x7fc5cc06f970 msgr2=0x7fc5cc06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.721+0000 7fc5d42e1700 1 -- 172.21.15.112:0/3814913463 shutdown_connections 2023-08-27T11:25:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.721+0000 7fc5d42e1700 1 -- 172.21.15.112:0/3814913463 wait complete. 2023-08-27T11:25:26.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.723+0000 7fc5d42e1700 1 Processor -- start 2023-08-27T11:25:26.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.723+0000 7fc5d42e1700 1 -- start start 2023-08-27T11:25:26.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.723+0000 7fc5d42e1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5cc0b3e50 0x7fc5cc0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:26.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.723+0000 7fc5d42e1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc0b4ad0 0x7fc5cc1f0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:26.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.723+0000 7fc5d42e1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5cc144570 0x7fc5cc0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:26.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.723+0000 7fc5d42e1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0cc8d0 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.723+0000 7fc5d42e1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc1f0560 con 0x7fc5cc0b3e50 2023-08-27T11:25:26.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.723+0000 7fc5d42e1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0ca440 con 0x7fc5cc144570 2023-08-27T11:25:26.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5d187c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc0b4ad0 0x7fc5cc1f0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:26.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5d187c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc0b4ad0 0x7fc5cc1f0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59560/0 (socket says 172.21.15.112:59560) 2023-08-27T11:25:26.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5d187c700 1 -- 172.21.15.112:0/2243318170 learned_addr learned my addr 172.21.15.112:0/2243318170 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:26.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5d287e700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5cc144570 0x7fc5cc0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:26.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5d207d700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5cc0b3e50 0x7fc5cc0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:26.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5d187c700 1 -- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5cc144570 msgr2=0x7fc5cc0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:26.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5d187c700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5cc144570 0x7fc5cc0c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5d187c700 1 -- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5cc0b3e50 msgr2=0x7fc5cc0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:26.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5d187c700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5cc0b3e50 0x7fc5cc0cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:26.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5d187c700 1 -- 172.21.15.112:0/2243318170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c4009b30 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5d187c700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc0b4ad0 0x7fc5cc1f0020 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00c490 tx=0x7fc5bc00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:26.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5c2ffd700 1 -- 172.21.15.112:0/2243318170 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0052d0 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5c2ffd700 1 -- 172.21.15.112:0/2243318170 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5bc005450 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5d42e1700 1 -- 172.21.15.112:0/2243318170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc0ccae0 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5c2ffd700 1 -- 172.21.15.112:0/2243318170 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc006aa0 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.724+0000 7fc5d42e1700 1 -- 172.21.15.112:0/2243318170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc0cd0a0 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.725+0000 7fc5d42e1700 1 -- 172.21.15.112:0/2243318170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5cc067a20 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.726+0000 7fc5c2ffd700 1 -- 172.21.15.112:0/2243318170 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fc5bc003680 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.727+0000 7fc5c2ffd700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc5b006a320 0x7fc5b006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:26.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.727+0000 7fc5c2ffd700 1 -- 172.21.15.112:0/2243318170 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc5bc087e90 con 0x7fc5cc0b4ad0 2023-08-27T11:25:26.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.727+0000 7fc5d207d700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc5b006a320 0x7fc5b006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:26.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.727+0000 7fc5d207d700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc5b006a320 0x7fc5b006c7e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009b40 tx=0x7fc5c8009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:26.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:26.733+0000 7fc5c2ffd700 1 -- 172.21.15.112:0/2243318170 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc5bc0533d0 con 0x7fc5cc0b4ad0 2023-08-27T11:25:27.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.048+0000 7fc5d42e1700 1 -- 172.21.15.112:0/2243318170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5cc0b3c10 con 0x7fc5cc0b4ad0 2023-08-27T11:25:27.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.049+0000 7fc5c2ffd700 1 -- 172.21.15.112:0/2243318170 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4886 (secure 0 0 0) 0x7fc5bc020090 con 0x7fc5cc0b4ad0 2023-08-27T11:25:27.051 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:27.051 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2906075444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2906075444},{"type":"v1","addr":"172.21.15.138:6836","nonce":2906075444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5761,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1781004436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1781004436},{"type":"v1","addr":"172.21.15.138:6837","nonce":1781004436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1573349448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1573349448},{"type":"v1","addr":"172.21.15.112:6834","nonce":1573349448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:19:16.064335+0000","modified":"2023-08-27T11:19:19.690233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/174707415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":174707415},{"type":"v1","addr":"172.21.15.112:6835","nonce":174707415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-27T11:25:27.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.052+0000 7fc5d42e1700 1 -- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc5b006a320 msgr2=0x7fc5b006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.052+0000 7fc5d42e1700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc5b006a320 0x7fc5b006c7e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009b40 tx=0x7fc5c8009210 comp rx=0 tx=0).stop 2023-08-27T11:25:27.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.052+0000 7fc5d42e1700 1 -- 172.21.15.112:0/2243318170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc0b4ad0 msgr2=0x7fc5cc1f0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.052+0000 7fc5d42e1700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc0b4ad0 0x7fc5cc1f0020 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00c490 tx=0x7fc5bc00c7a0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.052+0000 7fc5d42e1700 1 -- 172.21.15.112:0/2243318170 shutdown_connections 2023-08-27T11:25:27.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.052+0000 7fc5d42e1700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc5b006a320 0x7fc5b006c7e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.052+0000 7fc5d42e1700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5cc144570 0x7fc5cc0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.052+0000 7fc5d42e1700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5cc0b3e50 0x7fc5cc0cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.052+0000 7fc5d42e1700 1 --2- 172.21.15.112:0/2243318170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc0b4ad0 0x7fc5cc1f0020 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.052+0000 7fc5d42e1700 1 -- 172.21.15.112:0/2243318170 >> 172.21.15.112:0/2243318170 conn(0x7fc5cc06f970 msgr2=0x7fc5cc1403b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:27.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.052+0000 7fc5d42e1700 1 -- 172.21.15.112:0/2243318170 shutdown_connections 2023-08-27T11:25:27.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.052+0000 7fc5d42e1700 1 -- 172.21.15.112:0/2243318170 wait complete. 2023-08-27T11:25:27.059 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 40 2023-08-27T11:25:27.073 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:25:27.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.198+0000 7f77e96dc700 1 Processor -- start 2023-08-27T11:25:27.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.198+0000 7f77e96dc700 1 -- start start 2023-08-27T11:25:27.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e96dc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f77e40b5470 0x7f77e40b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e96dc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 0x7f77e40cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e96dc700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e4161e40 con 0x7f77e4146be0 2023-08-27T11:25:27.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e96dc700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e40b3980 con 0x7f77e40b5470 2023-08-27T11:25:27.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e96dc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e40b3b70 con 0x7f77e41619b0 2023-08-27T11:25:27.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 0x7f77e40cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e27fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f77e40b5470 0x7f77e40b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 0x7f77e40cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:42992/0 (socket says 172.21.15.112:42992) 2023-08-27T11:25:27.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e37fe700 1 -- 172.21.15.112:0/3254977289 learned_addr learned my addr 172.21.15.112:0/3254977289 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:27.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e1ffb700 1 -- 172.21.15.112:0/3254977289 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43568441 0 0) 0x7f77e4161e40 con 0x7f77e4146be0 2023-08-27T11:25:27.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e37fe700 1 -- 172.21.15.112:0/3254977289 >> v2:172.21.15.138:3300/0 conn(0x7f77e40b5470 msgr2=0x7f77e40b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:27.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e37fe700 1 --2- 172.21.15.112:0/3254977289 >> v2:172.21.15.138:3300/0 conn(0x7f77e40b5470 0x7f77e40b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e37fe700 1 -- 172.21.15.112:0/3254977289 >> v1:172.21.15.138:6789/0 conn(0x7f77e4146be0 legacy=0x7f77e40b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e37fe700 1 -- 172.21.15.112:0/3254977289 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e40b3e50 con 0x7f77e41619b0 2023-08-27T11:25:27.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.199+0000 7f77e37fe700 1 --2- 172.21.15.112:0/3254977289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 0x7f77e40cfa40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009e50 tx=0x7f77dc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf57121bfd72f543 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:27.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.200+0000 7f77e1ffb700 1 -- 172.21.15.112:0/3254977289 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc00e070 con 0x7f77e41619b0 2023-08-27T11:25:27.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.200+0000 7f77e1ffb700 1 -- 172.21.15.112:0/3254977289 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77dc00b910 con 0x7f77e41619b0 2023-08-27T11:25:27.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.200+0000 7f77e1ffb700 1 -- 172.21.15.112:0/3254977289 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc00bcc0 con 0x7f77e41619b0 2023-08-27T11:25:27.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.200+0000 7f77e96dc700 1 -- 172.21.15.112:0/3254977289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 msgr2=0x7f77e40cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.200+0000 7f77e96dc700 1 --2- 172.21.15.112:0/3254977289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 0x7f77e40cfa40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009e50 tx=0x7f77dc00b330 comp rx=0 tx=0).stop 2023-08-27T11:25:27.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.200+0000 7f77e96dc700 1 -- 172.21.15.112:0/3254977289 shutdown_connections 2023-08-27T11:25:27.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.200+0000 7f77e96dc700 1 --2- 172.21.15.112:0/3254977289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 0x7f77e40cfa40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.200+0000 7f77e96dc700 1 --2- 172.21.15.112:0/3254977289 >> v2:172.21.15.138:3300/0 conn(0x7f77e40b5470 0x7f77e40b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.200+0000 7f77e96dc700 1 -- 172.21.15.112:0/3254977289 >> 172.21.15.112:0/3254977289 conn(0x7f77e406c880 msgr2=0x7f77e40717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:27.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.200+0000 7f77e96dc700 1 -- 172.21.15.112:0/3254977289 shutdown_connections 2023-08-27T11:25:27.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.200+0000 7f77e96dc700 1 -- 172.21.15.112:0/3254977289 wait complete. 2023-08-27T11:25:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.202+0000 7f77e96dc700 1 Processor -- start 2023-08-27T11:25:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.202+0000 7f77e96dc700 1 -- start start 2023-08-27T11:25:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.202+0000 7f77e96dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77e40b5470 0x7f77e415fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.202+0000 7f77e96dc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e4146be0 0x7f77e413abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.202+0000 7f77e96dc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 0x7f77e413b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.202+0000 7f77e96dc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e41f6c30 con 0x7f77e4146be0 2023-08-27T11:25:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.202+0000 7f77e96dc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e413b630 con 0x7f77e40b5470 2023-08-27T11:25:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.202+0000 7f77e96dc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e41f5fb0 con 0x7f77e41619b0 2023-08-27T11:25:27.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 0x7f77e413b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e4146be0 0x7f77e413abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 0x7f77e413b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:42998/0 (socket says 172.21.15.112:42998) 2023-08-27T11:25:27.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e37fe700 1 -- 172.21.15.112:0/3465794389 learned_addr learned my addr 172.21.15.112:0/3465794389 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:27.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e2ffd700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77e40b5470 0x7f77e415fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e37fe700 1 -- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77e40b5470 msgr2=0x7f77e415fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e37fe700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77e40b5470 0x7f77e415fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e37fe700 1 -- 172.21.15.112:0/3465794389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e4146be0 msgr2=0x7f77e413abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e37fe700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e4146be0 0x7f77e413abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e37fe700 1 -- 172.21.15.112:0/3465794389 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc009b30 con 0x7f77e41619b0 2023-08-27T11:25:27.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e27fc700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e4146be0 0x7f77e413abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:27.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e37fe700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 0x7f77e413b0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b680 tx=0x7f77dc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:27.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77cb7fe700 1 -- 172.21.15.112:0/3465794389 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0048b0 con 0x7f77e41619b0 2023-08-27T11:25:27.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77cb7fe700 1 -- 172.21.15.112:0/3465794389 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77dc0092e0 con 0x7f77e41619b0 2023-08-27T11:25:27.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e96dc700 1 -- 172.21.15.112:0/3465794389 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e41f6290 con 0x7f77e41619b0 2023-08-27T11:25:27.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77cb7fe700 1 -- 172.21.15.112:0/3465794389 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc01c660 con 0x7f77e41619b0 2023-08-27T11:25:27.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.203+0000 7f77e96dc700 1 -- 172.21.15.112:0/3465794389 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e41f6850 con 0x7f77e41619b0 2023-08-27T11:25:27.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.204+0000 7f77e96dc700 1 -- 172.21.15.112:0/3465794389 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e41f6d70 con 0x7f77e41619b0 2023-08-27T11:25:27.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.205+0000 7f77cb7fe700 1 -- 172.21.15.112:0/3465794389 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f77dc01c7c0 con 0x7f77e41619b0 2023-08-27T11:25:27.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.205+0000 7f77cb7fe700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f77d006a320 0x7f77d006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.205+0000 7f77cb7fe700 1 -- 172.21.15.112:0/3465794389 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f77dc08a240 con 0x7f77e41619b0 2023-08-27T11:25:27.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.205+0000 7f77e2ffd700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f77d006a320 0x7f77d006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.206+0000 7f77e2ffd700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f77d006a320 0x7f77d006c7e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f77d4007fd0 tx=0x7f77d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:27.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.210+0000 7f77cb7fe700 1 -- 172.21.15.112:0/3465794389 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f77e41f6d70 con 0x7f77e41619b0 2023-08-27T11:25:27.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.442+0000 7f77e96dc700 1 -- 172.21.15.112:0/3465794389 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77e41f6d70 con 0x7f77e41619b0 2023-08-27T11:25:27.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.442+0000 7f77cb7fe700 1 -- 172.21.15.112:0/3465794389 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4886 (secure 0 0 0) 0x7f77e41f6d70 con 0x7f77e41619b0 2023-08-27T11:25:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:27.444 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2906075444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2906075444},{"type":"v1","addr":"172.21.15.138:6836","nonce":2906075444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5761,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1781004436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1781004436},{"type":"v1","addr":"172.21.15.138:6837","nonce":1781004436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1573349448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1573349448},{"type":"v1","addr":"172.21.15.112:6834","nonce":1573349448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:19:16.064335+0000","modified":"2023-08-27T11:19:19.690233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/174707415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":174707415},{"type":"v1","addr":"172.21.15.112:6835","nonce":174707415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-27T11:25:27.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.445+0000 7f77e96dc700 1 -- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f77d006a320 msgr2=0x7f77d006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.445+0000 7f77e96dc700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f77d006a320 0x7f77d006c7e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f77d4007fd0 tx=0x7f77d4009040 comp rx=0 tx=0).stop 2023-08-27T11:25:27.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.445+0000 7f77e96dc700 1 -- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 msgr2=0x7f77e413b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.445+0000 7f77e96dc700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 0x7f77e413b0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b680 tx=0x7f77dc004ad0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.445+0000 7f77e96dc700 1 -- 172.21.15.112:0/3465794389 shutdown_connections 2023-08-27T11:25:27.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.445+0000 7f77e96dc700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f77d006a320 0x7f77d006c7e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.445+0000 7f77e96dc700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e41619b0 0x7f77e413b0f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.445+0000 7f77e96dc700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77e40b5470 0x7f77e415fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.445+0000 7f77e96dc700 1 --2- 172.21.15.112:0/3465794389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e4146be0 0x7f77e413abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.445+0000 7f77e96dc700 1 -- 172.21.15.112:0/3465794389 >> 172.21.15.112:0/3465794389 conn(0x7f77e406c880 msgr2=0x7f77e40cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:27.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.445+0000 7f77e96dc700 1 -- 172.21.15.112:0/3465794389 shutdown_connections 2023-08-27T11:25:27.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.445+0000 7f77e96dc700 1 -- 172.21.15.112:0/3465794389 wait complete. 2023-08-27T11:25:27.449 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 40 2023-08-27T11:25:27.458 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:27.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.577+0000 7f97cbe9e700 1 Processor -- start 2023-08-27T11:25:27.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.577+0000 7f97cbe9e700 1 -- start start 2023-08-27T11:25:27.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97cbe9e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f97c40b17e0 0x7f97c40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97cbe9e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97c4161ce0 0x7f97c406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97cbe9e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c4161850 con 0x7f97c40b1400 2023-08-27T11:25:27.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97cbe9e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c40cfc10 con 0x7f97c40b17e0 2023-08-27T11:25:27.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97cbe9e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c4161200 con 0x7f97c4161ce0 2023-08-27T11:25:27.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97c9439700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f97c40b17e0 0x7f97c40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97c9c3a700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f97c40b1400 0x7f97c406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53962/0 (socket says 172.21.15.112:53962) 2023-08-27T11:25:27.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97c9439700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f97c40b17e0 0x7f97c40cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59582/0 (socket says 172.21.15.112:59582) 2023-08-27T11:25:27.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97c9c3a700 1 -- 172.21.15.112:0/2459756242 learned_addr learned my addr 172.21.15.112:0/2459756242 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:27.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97ca43b700 1 --2- 172.21.15.112:0/2459756242 >> v2:172.21.15.138:3300/0 conn(0x7f97c4161ce0 0x7f97c406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97c9439700 1 -- 172.21.15.112:0/2459756242 >> v2:172.21.15.138:3300/0 conn(0x7f97c4161ce0 msgr2=0x7f97c406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:27.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97c9439700 1 --2- 172.21.15.112:0/2459756242 >> v2:172.21.15.138:3300/0 conn(0x7f97c4161ce0 0x7f97c406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97c9439700 1 -- 172.21.15.112:0/2459756242 >> v1:172.21.15.112:6789/0 conn(0x7f97c40b1400 legacy=0x7f97c406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97c9439700 1 -- 172.21.15.112:0/2459756242 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c41614e0 con 0x7f97c40b17e0 2023-08-27T11:25:27.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97c9439700 1 --2- 172.21.15.112:0/2459756242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c40b17e0 0x7f97c40cf5a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f97c406df10 tx=0x7f97b4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded2de5c50c1114a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:27.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97c8c38700 1 -- 172.21.15.112:0/2459756242 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4011830 con 0x7f97c40b17e0 2023-08-27T11:25:27.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97c8c38700 1 -- 172.21.15.112:0/2459756242 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b4011990 con 0x7f97c40b17e0 2023-08-27T11:25:27.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.578+0000 7f97c8c38700 1 -- 172.21.15.112:0/2459756242 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4011d20 con 0x7f97c40b17e0 2023-08-27T11:25:27.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.579+0000 7f97cbe9e700 1 -- 172.21.15.112:0/2459756242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c40b17e0 msgr2=0x7f97c40cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.579+0000 7f97cbe9e700 1 --2- 172.21.15.112:0/2459756242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c40b17e0 0x7f97c40cf5a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f97c406df10 tx=0x7f97b4009b20 comp rx=0 tx=0).stop 2023-08-27T11:25:27.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.579+0000 7f97cbe9e700 1 -- 172.21.15.112:0/2459756242 shutdown_connections 2023-08-27T11:25:27.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.579+0000 7f97cbe9e700 1 --2- 172.21.15.112:0/2459756242 >> v2:172.21.15.138:3300/0 conn(0x7f97c4161ce0 0x7f97c406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.579+0000 7f97cbe9e700 1 --2- 172.21.15.112:0/2459756242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c40b17e0 0x7f97c40cf5a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.579+0000 7f97cbe9e700 1 -- 172.21.15.112:0/2459756242 >> 172.21.15.112:0/2459756242 conn(0x7f97c40d0000 msgr2=0x7f97c40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:27.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.579+0000 7f97cbe9e700 1 -- 172.21.15.112:0/2459756242 shutdown_connections 2023-08-27T11:25:27.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.579+0000 7f97cbe9e700 1 -- 172.21.15.112:0/2459756242 wait complete. 2023-08-27T11:25:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.580+0000 7f97cbe9e700 1 Processor -- start 2023-08-27T11:25:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.580+0000 7f97cbe9e700 1 -- start start 2023-08-27T11:25:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97cbe9e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c40b1400 0x7f97c4152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97cbe9e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c40b17e0 0x7f97c40c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97cbe9e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c4161ce0 0x7f97c41f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97cbe9e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c4153050 con 0x7f97c4161ce0 2023-08-27T11:25:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97cbe9e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c4070f50 con 0x7f97c40b1400 2023-08-27T11:25:27.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97cbe9e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c4071230 con 0x7f97c40b17e0 2023-08-27T11:25:27.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97ca43b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c4161ce0 0x7f97c41f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97ca43b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c4161ce0 0x7f97c41f6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59590/0 (socket says 172.21.15.112:59590) 2023-08-27T11:25:27.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97ca43b700 1 -- 172.21.15.112:0/957768211 learned_addr learned my addr 172.21.15.112:0/957768211 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:27.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97c9439700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c40b17e0 0x7f97c40c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97c9c3a700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c40b1400 0x7f97c4152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97ca43b700 1 -- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c40b17e0 msgr2=0x7f97c40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97ca43b700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c40b17e0 0x7f97c40c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97ca43b700 1 -- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c40b1400 msgr2=0x7f97c4152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97ca43b700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c40b1400 0x7f97c4152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97ca43b700 1 -- 172.21.15.112:0/957768211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b40094a0 con 0x7f97c4161ce0 2023-08-27T11:25:27.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.581+0000 7f97c9c3a700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c40b1400 0x7f97c4152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:27.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.582+0000 7f97ca43b700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c4161ce0 0x7f97c41f6590 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f97bc007e40 tx=0x7f97bc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:27.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.582+0000 7f97ba7fc700 1 -- 172.21.15.112:0/957768211 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc00eea0 con 0x7f97c4161ce0 2023-08-27T11:25:27.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.582+0000 7f97ba7fc700 1 -- 172.21.15.112:0/957768211 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc014420 con 0x7f97c4161ce0 2023-08-27T11:25:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.582+0000 7f97cbe9e700 1 -- 172.21.15.112:0/957768211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c4071510 con 0x7f97c4161ce0 2023-08-27T11:25:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.582+0000 7f97ba7fc700 1 -- 172.21.15.112:0/957768211 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0147b0 con 0x7f97c4161ce0 2023-08-27T11:25:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.582+0000 7f97cbe9e700 1 -- 172.21.15.112:0/957768211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c4071ad0 con 0x7f97c4161ce0 2023-08-27T11:25:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.583+0000 7f97cbe9e700 1 -- 172.21.15.112:0/957768211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c4071cc0 con 0x7f97c4161ce0 2023-08-27T11:25:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.583+0000 7f97ba7fc700 1 -- 172.21.15.112:0/957768211 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f97bc014910 con 0x7f97c4161ce0 2023-08-27T11:25:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.584+0000 7f97ba7fc700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f97a806a320 0x7f97a806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.584+0000 7f97ba7fc700 1 -- 172.21.15.112:0/957768211 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97bc0890f0 con 0x7f97c4161ce0 2023-08-27T11:25:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.584+0000 7f97c9c3a700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f97a806a320 0x7f97a806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.584+0000 7f97c9c3a700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f97a806a320 0x7f97a806c7e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f97c0005910 tx=0x7f97c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:27.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.588+0000 7f97ba7fc700 1 -- 172.21.15.112:0/957768211 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f97bc054550 con 0x7f97c4161ce0 2023-08-27T11:25:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.782+0000 7f97cbe9e700 1 -- 172.21.15.112:0/957768211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97c4067a20 con 0x7f97c4161ce0 2023-08-27T11:25:27.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.783+0000 7f97ba7fc700 1 -- 172.21.15.112:0/957768211 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18320 (secure 0 0 0) 0x7f97bc057b70 con 0x7f97c4161ce0 2023-08-27T11:25:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:27.785 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":65,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:19:17.239518+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:19:14.179813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-27T11:19:15.021524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.785+0000 7f97cbe9e700 1 -- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f97a806a320 msgr2=0x7f97a806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.785+0000 7f97cbe9e700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f97a806a320 0x7f97a806c7e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f97c0005910 tx=0x7f97c0009040 comp rx=0 tx=0).stop 2023-08-27T11:25:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.785+0000 7f97cbe9e700 1 -- 172.21.15.112:0/957768211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c4161ce0 msgr2=0x7f97c41f6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.785+0000 7f97cbe9e700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c4161ce0 0x7f97c41f6590 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f97bc007e40 tx=0x7f97bc00e3c0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.785+0000 7f97cbe9e700 1 -- 172.21.15.112:0/957768211 shutdown_connections 2023-08-27T11:25:27.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.785+0000 7f97cbe9e700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f97a806a320 0x7f97a806c7e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.785+0000 7f97cbe9e700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c4161ce0 0x7f97c41f6590 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.785+0000 7f97cbe9e700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c40b1400 0x7f97c4152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.785+0000 7f97cbe9e700 1 --2- 172.21.15.112:0/957768211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c40b17e0 0x7f97c40c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.785+0000 7f97cbe9e700 1 -- 172.21.15.112:0/957768211 >> 172.21.15.112:0/957768211 conn(0x7f97c40d0000 msgr2=0x7f97c415eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:27.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.785+0000 7f97cbe9e700 1 -- 172.21.15.112:0/957768211 shutdown_connections 2023-08-27T11:25:27.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.785+0000 7f97cbe9e700 1 -- 172.21.15.112:0/957768211 wait complete. 2023-08-27T11:25:27.799 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:25:27.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.925+0000 7fade833a700 1 Processor -- start 2023-08-27T11:25:27.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.925+0000 7fade833a700 1 -- start start 2023-08-27T11:25:27.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.926+0000 7fade833a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fade013f990 0x7fade00cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.926+0000 7fade833a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade013ff50 con 0x7fade00cd270 2023-08-27T11:25:27.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.926+0000 7fade833a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade0140230 con 0x7fade006e150 2023-08-27T11:25:27.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.926+0000 7fade833a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade00cecb0 con 0x7fade013f990 2023-08-27T11:25:27.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.926+0000 7fade58d5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fade013f990 0x7fade00cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.926+0000 7fade58d5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fade013f990 0x7fade00cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59596/0 (socket says 172.21.15.112:59596) 2023-08-27T11:25:27.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.926+0000 7fade58d5700 1 -- 172.21.15.112:0/3897462432 learned_addr learned my addr 172.21.15.112:0/3897462432 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:27.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.926+0000 7fade58d5700 1 -- 172.21.15.112:0/3897462432 >> v1:172.21.15.138:6789/0 conn(0x7fade006e150 legacy=0x7fade006e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.926+0000 7fade58d5700 1 -- 172.21.15.112:0/3897462432 >> v1:172.21.15.112:6789/0 conn(0x7fade00cd270 legacy=0x7fade015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.926+0000 7fade50d4700 1 -- 172.21.15.112:0/3897462432 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915071343 0 0) 0x7fade0140230 con 0x7fade006e150 2023-08-27T11:25:27.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.926+0000 7fade58d5700 1 -- 172.21.15.112:0/3897462432 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0140510 con 0x7fade013f990 2023-08-27T11:25:27.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.926+0000 7fade58d5700 1 --2- 172.21.15.112:0/3897462432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade013f990 0x7fade00cf030 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fadd0006fd0 tx=0x7fadd000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e846c456daf0c4e5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:27.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.927+0000 7fade50d4700 1 -- 172.21.15.112:0/3897462432 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0005e30 con 0x7fade013f990 2023-08-27T11:25:27.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.927+0000 7fade50d4700 1 -- 172.21.15.112:0/3897462432 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd0004030 con 0x7fade013f990 2023-08-27T11:25:27.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.927+0000 7fade50d4700 1 -- 172.21.15.112:0/3897462432 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd00043e0 con 0x7fade013f990 2023-08-27T11:25:27.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.927+0000 7fade833a700 1 -- 172.21.15.112:0/3897462432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade013f990 msgr2=0x7fade00cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.927+0000 7fade833a700 1 --2- 172.21.15.112:0/3897462432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade013f990 0x7fade00cf030 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fadd0006fd0 tx=0x7fadd000be70 comp rx=0 tx=0).stop 2023-08-27T11:25:27.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.928+0000 7fade833a700 1 -- 172.21.15.112:0/3897462432 shutdown_connections 2023-08-27T11:25:27.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.928+0000 7fade833a700 1 --2- 172.21.15.112:0/3897462432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade013f990 0x7fade00cf030 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.928+0000 7fade833a700 1 -- 172.21.15.112:0/3897462432 >> 172.21.15.112:0/3897462432 conn(0x7fade006cd30 msgr2=0x7fade00717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:27.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.928+0000 7fade833a700 1 -- 172.21.15.112:0/3897462432 shutdown_connections 2023-08-27T11:25:27.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.928+0000 7fade833a700 1 -- 172.21.15.112:0/3897462432 wait complete. 2023-08-27T11:25:27.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.929+0000 7fade833a700 1 Processor -- start 2023-08-27T11:25:27.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.929+0000 7fade833a700 1 -- start start 2023-08-27T11:25:27.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade833a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fade006e150 0x7fade015c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade833a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade00cd270 0x7fade0156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade833a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fade013f990 0x7fade00ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade833a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade015c710 con 0x7fade00cd270 2023-08-27T11:25:27.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade833a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade0156ca0 con 0x7fade013f990 2023-08-27T11:25:27.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade833a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade015b130 con 0x7fade006e150 2023-08-27T11:25:27.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade58d5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade00cd270 0x7fade0156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade58d5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade00cd270 0x7fade0156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59602/0 (socket says 172.21.15.112:59602) 2023-08-27T11:25:27.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade58d5700 1 -- 172.21.15.112:0/875275281 learned_addr learned my addr 172.21.15.112:0/875275281 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:27.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade60d6700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fade006e150 0x7fade015c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade68d7700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fade013f990 0x7fade00ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade58d5700 1 -- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fade006e150 msgr2=0x7fade015c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade58d5700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fade006e150 0x7fade015c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade58d5700 1 -- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fade013f990 msgr2=0x7fade00ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:27.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade58d5700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fade013f990 0x7fade00ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:27.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.930+0000 7fade58d5700 1 -- 172.21.15.112:0/875275281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd000b7a0 con 0x7fade00cd270 2023-08-27T11:25:27.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.931+0000 7fade58d5700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade00cd270 0x7fade0156760 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fade006eeb0 tx=0x7fadd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:27.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.931+0000 7fadd6ffd700 1 -- 172.21.15.112:0/875275281 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0020070 con 0x7fade00cd270 2023-08-27T11:25:27.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.931+0000 7fadd6ffd700 1 -- 172.21.15.112:0/875275281 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd00036a0 con 0x7fade00cd270 2023-08-27T11:25:27.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.931+0000 7fade833a700 1 -- 172.21.15.112:0/875275281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade015b410 con 0x7fade00cd270 2023-08-27T11:25:27.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.931+0000 7fadd6ffd700 1 -- 172.21.15.112:0/875275281 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0003a30 con 0x7fade00cd270 2023-08-27T11:25:27.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.931+0000 7fade833a700 1 -- 172.21.15.112:0/875275281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade015b9d0 con 0x7fade00cd270 2023-08-27T11:25:27.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.932+0000 7fade833a700 1 -- 172.21.15.112:0/875275281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade0067a20 con 0x7fade00cd270 2023-08-27T11:25:27.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.933+0000 7fadd6ffd700 1 -- 172.21.15.112:0/875275281 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fadd0003b90 con 0x7fade00cd270 2023-08-27T11:25:27.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.933+0000 7fadd6ffd700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fadc406a320 0x7fadc406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:27.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.933+0000 7fadd6ffd700 1 -- 172.21.15.112:0/875275281 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fadd008bda0 con 0x7fade00cd270 2023-08-27T11:25:27.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.933+0000 7fade60d6700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fadc406a320 0x7fadc406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:27.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.934+0000 7fade60d6700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fadc406a320 0x7fadc406c7e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7faddc000f60 tx=0x7faddc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:27.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:27.937+0000 7fadd6ffd700 1 -- 172.21.15.112:0/875275281 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fadd0056fa0 con 0x7fade00cd270 2023-08-27T11:25:28.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.225+0000 7fade833a700 1 -- 172.21.15.112:0/875275281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fade00cf4a0 con 0x7fade00cd270 2023-08-27T11:25:28.227 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:25:28.225+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:25:28.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.230+0000 7fadd6ffd700 1 -- 172.21.15.112:0/875275281 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7fadd005a5c0 con 0x7fade00cd270 2023-08-27T11:25:28.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.232+0000 7fade833a700 1 -- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fadc406a320 msgr2=0x7fadc406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:28.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.232+0000 7fade833a700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fadc406a320 0x7fadc406c7e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7faddc000f60 tx=0x7faddc009040 comp rx=0 tx=0).stop 2023-08-27T11:25:28.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.232+0000 7fade833a700 1 -- 172.21.15.112:0/875275281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade00cd270 msgr2=0x7fade0156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:28.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.232+0000 7fade833a700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade00cd270 0x7fade0156760 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fade006eeb0 tx=0x7fadd0004750 comp rx=0 tx=0).stop 2023-08-27T11:25:28.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.232+0000 7fade833a700 1 -- 172.21.15.112:0/875275281 shutdown_connections 2023-08-27T11:25:28.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.232+0000 7fade833a700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fadc406a320 0x7fadc406c7e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:28.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.232+0000 7fade833a700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fade013f990 0x7fade00ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:28.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.232+0000 7fade833a700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fade006e150 0x7fade015c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:28.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.232+0000 7fade833a700 1 --2- 172.21.15.112:0/875275281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade00cd270 0x7fade0156760 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:28.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.232+0000 7fade833a700 1 -- 172.21.15.112:0/875275281 >> 172.21.15.112:0/875275281 conn(0x7fade006cd30 msgr2=0x7fade0159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:28.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.232+0000 7fade833a700 1 -- 172.21.15.112:0/875275281 shutdown_connections 2023-08-27T11:25:28.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.232+0000 7fade833a700 1 -- 172.21.15.112:0/875275281 wait complete. 2023-08-27T11:25:28.236 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:25:28.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:25:28.268 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:25:28.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.375+0000 7f0656a58700 1 Processor -- start 2023-08-27T11:25:28.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.375+0000 7f0656a58700 1 -- start start 2023-08-27T11:25:28.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.376+0000 7f0656a58700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0650146c30 0x7f065006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:28.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.376+0000 7f0656a58700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f065006fea0 0x7f065006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:28.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.376+0000 7f0656a58700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06500b3ee0 con 0x7f06500b4c10 2023-08-27T11:25:28.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.376+0000 7f0656a58700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0650070310 con 0x7f0650146c30 2023-08-27T11:25:28.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.376+0000 7f0656a58700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0650070500 con 0x7f065006fea0 2023-08-27T11:25:28.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.376+0000 7f064ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0650146c30 0x7f065006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:28.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.376+0000 7f064f7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f06500b4c10 0x7f065006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54002/0 (socket says 172.21.15.112:54002) 2023-08-27T11:25:28.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.376+0000 7f064f7fe700 1 -- 172.21.15.112:0/2287370081 learned_addr learned my addr 172.21.15.112:0/2287370081 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:28.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.376+0000 7f0654ff5700 1 --2- 172.21.15.112:0/2287370081 >> v2:172.21.15.138:3300/0 conn(0x7f065006fea0 0x7f065006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:28.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.376+0000 7f064ffff700 1 -- 172.21.15.112:0/2287370081 >> v2:172.21.15.138:3300/0 conn(0x7f065006fea0 msgr2=0x7f065006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:28.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.376+0000 7f064ffff700 1 --2- 172.21.15.112:0/2287370081 >> v2:172.21.15.138:3300/0 conn(0x7f065006fea0 0x7f065006d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:28.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f064ffff700 1 -- 172.21.15.112:0/2287370081 >> v1:172.21.15.112:6789/0 conn(0x7f06500b4c10 legacy=0x7f065006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:28.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f064ffff700 1 -- 172.21.15.112:0/2287370081 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06500b4130 con 0x7f0650146c30 2023-08-27T11:25:28.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f064effd700 1 -- 172.21.15.112:0/2287370081 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593457393 0 0) 0x7f06500b3ee0 con 0x7f06500b4c10 2023-08-27T11:25:28.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f064ffff700 1 --2- 172.21.15.112:0/2287370081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650146c30 0x7f065006d4a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f06500707e0 tx=0x7f0644009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f325cd7ca46d56 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:28.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f064effd700 1 -- 172.21.15.112:0/2287370081 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644011830 con 0x7f0650146c30 2023-08-27T11:25:28.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f064effd700 1 -- 172.21.15.112:0/2287370081 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0644011990 con 0x7f0650146c30 2023-08-27T11:25:28.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f064effd700 1 -- 172.21.15.112:0/2287370081 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644011d20 con 0x7f0650146c30 2023-08-27T11:25:28.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f0656a58700 1 -- 172.21.15.112:0/2287370081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650146c30 msgr2=0x7f065006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:28.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f0656a58700 1 --2- 172.21.15.112:0/2287370081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650146c30 0x7f065006d4a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f06500707e0 tx=0x7f0644009b20 comp rx=0 tx=0).stop 2023-08-27T11:25:28.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f0656a58700 1 -- 172.21.15.112:0/2287370081 shutdown_connections 2023-08-27T11:25:28.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f0656a58700 1 --2- 172.21.15.112:0/2287370081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650146c30 0x7f065006d4a0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:28.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f0656a58700 1 --2- 172.21.15.112:0/2287370081 >> v2:172.21.15.138:3300/0 conn(0x7f065006fea0 0x7f065006d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:28.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f0656a58700 1 -- 172.21.15.112:0/2287370081 >> 172.21.15.112:0/2287370081 conn(0x7f06500cffa0 msgr2=0x7f06500d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:28.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f0656a58700 1 -- 172.21.15.112:0/2287370081 shutdown_connections 2023-08-27T11:25:28.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.377+0000 7f0656a58700 1 -- 172.21.15.112:0/2287370081 wait complete. 2023-08-27T11:25:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.379+0000 7f0656a58700 1 Processor -- start 2023-08-27T11:25:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.379+0000 7f0656a58700 1 -- start start 2023-08-27T11:25:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0656a58700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065006fea0 0x7f065015cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:28.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0656a58700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06500b4c10 0x7f06501440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:28.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0656a58700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650146c30 0x7f06501f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:28.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0656a58700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065013e2c0 con 0x7f0650146c30 2023-08-27T11:25:28.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0656a58700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0650140a80 con 0x7f06500b4c10 2023-08-27T11:25:28.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0656a58700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0650140d60 con 0x7f065006fea0 2023-08-27T11:25:28.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0654ff5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650146c30 0x7f06501f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:28.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0654ff5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650146c30 0x7f06501f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59630/0 (socket says 172.21.15.112:59630) 2023-08-27T11:25:28.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0654ff5700 1 -- 172.21.15.112:0/1121144535 learned_addr learned my addr 172.21.15.112:0/1121144535 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:28.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f064f7fe700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06500b4c10 0x7f06501440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:28.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f064ffff700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065006fea0 0x7f065015cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0654ff5700 1 -- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065006fea0 msgr2=0x7f065015cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0654ff5700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065006fea0 0x7f065015cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0654ff5700 1 -- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06500b4c10 msgr2=0x7f06501440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0654ff5700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06500b4c10 0x7f06501440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.380+0000 7f0654ff5700 1 -- 172.21.15.112:0/1121144535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06440094a0 con 0x7f0650146c30 2023-08-27T11:25:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.381+0000 7f0654ff5700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650146c30 0x7f06501f5fa0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0648007e40 tx=0x7f064800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.381+0000 7f064cff9700 1 -- 172.21.15.112:0/1121144535 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064800eea0 con 0x7f0650146c30 2023-08-27T11:25:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.381+0000 7f064cff9700 1 -- 172.21.15.112:0/1121144535 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0648014420 con 0x7f0650146c30 2023-08-27T11:25:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.381+0000 7f0656a58700 1 -- 172.21.15.112:0/1121144535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065013dc00 con 0x7f0650146c30 2023-08-27T11:25:28.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.381+0000 7f064cff9700 1 -- 172.21.15.112:0/1121144535 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06480147b0 con 0x7f0650146c30 2023-08-27T11:25:28.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.381+0000 7f0656a58700 1 -- 172.21.15.112:0/1121144535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0650144610 con 0x7f0650146c30 2023-08-27T11:25:28.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.382+0000 7f0656a58700 1 -- 172.21.15.112:0/1121144535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065013e400 con 0x7f0650146c30 2023-08-27T11:25:28.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.382+0000 7f064cff9700 1 -- 172.21.15.112:0/1121144535 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f0648014910 con 0x7f0650146c30 2023-08-27T11:25:28.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.383+0000 7f064cff9700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063c06a320 0x7f063c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:28.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.383+0000 7f064cff9700 1 -- 172.21.15.112:0/1121144535 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0648089060 con 0x7f0650146c30 2023-08-27T11:25:28.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.383+0000 7f064ffff700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063c06a320 0x7f063c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:28.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.383+0000 7f064ffff700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063c06a320 0x7f063c06c7e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f06440090b0 tx=0x7f064401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:28.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.387+0000 7f064cff9700 1 -- 172.21.15.112:0/1121144535 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f065013e400 con 0x7f0650146c30 2023-08-27T11:25:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:28.619+0000 7f0656a58700 1 -- 172.21.15.112:0/1121144535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f065013e400 con 0x7f0650146c30 2023-08-27T11:25:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.230+0000 7f064cff9700 1 -- 172.21.15.112:0/1121144535 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7f065013e400 con 0x7f0650146c30 2023-08-27T11:25:29.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.232+0000 7f0656a58700 1 -- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063c06a320 msgr2=0x7f063c06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:29.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.232+0000 7f0656a58700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063c06a320 0x7f063c06c7e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f06440090b0 tx=0x7f064401c040 comp rx=0 tx=0).stop 2023-08-27T11:25:29.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.232+0000 7f0656a58700 1 -- 172.21.15.112:0/1121144535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650146c30 msgr2=0x7f06501f5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:29.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.232+0000 7f0656a58700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650146c30 0x7f06501f5fa0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0648007e40 tx=0x7f064800e3c0 comp rx=0 tx=0).stop 2023-08-27T11:25:29.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.232+0000 7f0656a58700 1 -- 172.21.15.112:0/1121144535 shutdown_connections 2023-08-27T11:25:29.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.232+0000 7f0656a58700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063c06a320 0x7f063c06c7e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:29.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.232+0000 7f0656a58700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650146c30 0x7f06501f5fa0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:29.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.232+0000 7f0656a58700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065006fea0 0x7f065015cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:29.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.232+0000 7f0656a58700 1 --2- 172.21.15.112:0/1121144535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06500b4c10 0x7f06501440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:29.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.232+0000 7f0656a58700 1 -- 172.21.15.112:0/1121144535 >> 172.21.15.112:0/1121144535 conn(0x7f06500cffa0 msgr2=0x7f065006f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:29.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.233+0000 7f0656a58700 1 -- 172.21.15.112:0/1121144535 shutdown_connections 2023-08-27T11:25:29.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.233+0000 7f0656a58700 1 -- 172.21.15.112:0/1121144535 wait complete. 2023-08-27T11:25:29.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:25:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.416+0000 7f281a17d700 1 Processor -- start 2023-08-27T11:25:29.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.416+0000 7f281a17d700 1 -- start start 2023-08-27T11:25:29.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.417+0000 7f281a17d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f28140b1420 0x7f28140b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:29.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.417+0000 7f281a17d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28140b0b10 con 0x7f28141439e0 2023-08-27T11:25:29.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.417+0000 7f281a17d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281406d850 con 0x7f2814143dc0 2023-08-27T11:25:29.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.417+0000 7f281a17d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281415b460 con 0x7f28140b1420 2023-08-27T11:25:29.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.417+0000 7f2813fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f28140b1420 0x7f28140b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:29.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.417+0000 7f28137fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f28141439e0 0x7f28140b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54018/0 (socket says 172.21.15.112:54018) 2023-08-27T11:25:29.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.417+0000 7f28137fe700 1 -- 172.21.15.112:0/3026643474 learned_addr learned my addr 172.21.15.112:0/3026643474 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:29.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.417+0000 7f2813fff700 1 -- 172.21.15.112:0/3026643474 >> v1:172.21.15.138:6789/0 conn(0x7f2814143dc0 legacy=0x7f281415a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:29.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.417+0000 7f2813fff700 1 -- 172.21.15.112:0/3026643474 >> v1:172.21.15.112:6789/0 conn(0x7f28141439e0 legacy=0x7f28140b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:29.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.417+0000 7f2813fff700 1 -- 172.21.15.112:0/3026643474 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281415b740 con 0x7f28140b1420 2023-08-27T11:25:29.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.417+0000 7f28127fc700 1 -- 172.21.15.112:0/3026643474 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729863763 0 0) 0x7f28140b0b10 con 0x7f28141439e0 2023-08-27T11:25:29.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.417+0000 7f2813fff700 1 --2- 172.21.15.112:0/3026643474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28140b1420 0x7f28140b05d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f280c009d90 tx=0x7f280c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93804b0349b7e4b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:29.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.418+0000 7f28127fc700 1 -- 172.21.15.112:0/3026643474 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c00e070 con 0x7f28140b1420 2023-08-27T11:25:29.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.418+0000 7f28127fc700 1 -- 172.21.15.112:0/3026643474 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c00bbd0 con 0x7f28140b1420 2023-08-27T11:25:29.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.418+0000 7f28127fc700 1 -- 172.21.15.112:0/3026643474 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c012430 con 0x7f28140b1420 2023-08-27T11:25:29.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.418+0000 7f281a17d700 1 -- 172.21.15.112:0/3026643474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28140b1420 msgr2=0x7f28140b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:29.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.418+0000 7f281a17d700 1 --2- 172.21.15.112:0/3026643474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28140b1420 0x7f28140b05d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f280c009d90 tx=0x7f280c00b330 comp rx=0 tx=0).stop 2023-08-27T11:25:29.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.418+0000 7f281a17d700 1 -- 172.21.15.112:0/3026643474 shutdown_connections 2023-08-27T11:25:29.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.418+0000 7f281a17d700 1 --2- 172.21.15.112:0/3026643474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28140b1420 0x7f28140b05d0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:29.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.418+0000 7f281a17d700 1 -- 172.21.15.112:0/3026643474 >> 172.21.15.112:0/3026643474 conn(0x7f28140b1be0 msgr2=0x7f28140b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:29.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.418+0000 7f281a17d700 1 -- 172.21.15.112:0/3026643474 shutdown_connections 2023-08-27T11:25:29.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.418+0000 7f281a17d700 1 -- 172.21.15.112:0/3026643474 wait complete. 2023-08-27T11:25:29.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.419+0000 7f281a17d700 1 Processor -- start 2023-08-27T11:25:29.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.420+0000 7f281a17d700 1 -- start start 2023-08-27T11:25:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f281a17d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28140b1420 0x7f281415e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f281a17d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28141439e0 0x7f281413d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f281a17d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f281415d030 0x7f2814139960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f281a17d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281413d8e0 con 0x7f28141439e0 2023-08-27T11:25:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f281a17d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281413d6b0 con 0x7f28140b1420 2023-08-27T11:25:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f281a17d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281413a770 con 0x7f281415d030 2023-08-27T11:25:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f2812ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28141439e0 0x7f281413d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f2812ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28141439e0 0x7f281413d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59644/0 (socket says 172.21.15.112:59644) 2023-08-27T11:25:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f2812ffd700 1 -- 172.21.15.112:0/2162912979 learned_addr learned my addr 172.21.15.112:0/2162912979 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f2813fff700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f281415d030 0x7f2814139960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f28137fe700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28140b1420 0x7f281415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f2812ffd700 1 -- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f281415d030 msgr2=0x7f2814139960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f2812ffd700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f281415d030 0x7f2814139960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f2812ffd700 1 -- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28140b1420 msgr2=0x7f281415e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f2812ffd700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28140b1420 0x7f281415e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f2812ffd700 1 -- 172.21.15.112:0/2162912979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c009a70 con 0x7f28141439e0 2023-08-27T11:25:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f28137fe700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28140b1420 0x7f281415e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f2812ffd700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28141439e0 0x7f281413d170 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f280400c400 tx=0x7f280400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.421+0000 7f280bfff700 1 -- 172.21.15.112:0/2162912979 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28040052d0 con 0x7f28141439e0 2023-08-27T11:25:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.422+0000 7f280bfff700 1 -- 172.21.15.112:0/2162912979 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2804005430 con 0x7f28141439e0 2023-08-27T11:25:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.422+0000 7f281a17d700 1 -- 172.21.15.112:0/2162912979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281415d950 con 0x7f28141439e0 2023-08-27T11:25:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.422+0000 7f280bfff700 1 -- 172.21.15.112:0/2162912979 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28040038c0 con 0x7f28141439e0 2023-08-27T11:25:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.422+0000 7f281a17d700 1 -- 172.21.15.112:0/2162912979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281415df10 con 0x7f28141439e0 2023-08-27T11:25:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.422+0000 7f281a17d700 1 -- 172.21.15.112:0/2162912979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281415d6a0 con 0x7f28141439e0 2023-08-27T11:25:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.423+0000 7f280bfff700 1 -- 172.21.15.112:0/2162912979 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f2804003a20 con 0x7f28141439e0 2023-08-27T11:25:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.423+0000 7f280bfff700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27f406a320 0x7f27f406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.423+0000 7f280bfff700 1 -- 172.21.15.112:0/2162912979 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f2804087d50 con 0x7f28141439e0 2023-08-27T11:25:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.423+0000 7f28137fe700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27f406a320 0x7f27f406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.424+0000 7f28137fe700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27f406a320 0x7f27f406c7e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f27fc000f60 tx=0x7f27fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:29.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.427+0000 7f280bfff700 1 -- 172.21.15.112:0/2162912979 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f280408b030 con 0x7f28141439e0 2023-08-27T11:25:29.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:29.616+0000 7f281a17d700 1 -- 172.21.15.112:0/2162912979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2814067a20 con 0x7f28141439e0 2023-08-27T11:25:30.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.351+0000 7f280bfff700 1 -- 172.21.15.112:0/2162912979 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 181+0+0 (secure 0 0 0) 0x7f28040530d0 con 0x7f28141439e0 2023-08-27T11:25:30.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.354+0000 7f281a17d700 1 -- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27f406a320 msgr2=0x7f27f406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.354+0000 7f281a17d700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27f406a320 0x7f27f406c7e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f27fc000f60 tx=0x7f27fc009040 comp rx=0 tx=0).stop 2023-08-27T11:25:30.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.354+0000 7f281a17d700 1 -- 172.21.15.112:0/2162912979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28141439e0 msgr2=0x7f281413d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.354+0000 7f281a17d700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28141439e0 0x7f281413d170 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f280400c400 tx=0x7f280400c7c0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.354+0000 7f281a17d700 1 -- 172.21.15.112:0/2162912979 shutdown_connections 2023-08-27T11:25:30.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.354+0000 7f281a17d700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27f406a320 0x7f27f406c7e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.354+0000 7f281a17d700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f281415d030 0x7f2814139960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.354+0000 7f281a17d700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28140b1420 0x7f281415e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.354+0000 7f281a17d700 1 --2- 172.21.15.112:0/2162912979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28141439e0 0x7f281413d170 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.354+0000 7f281a17d700 1 -- 172.21.15.112:0/2162912979 >> 172.21.15.112:0/2162912979 conn(0x7f28140b1be0 msgr2=0x7f281415a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:30.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.354+0000 7f281a17d700 1 -- 172.21.15.112:0/2162912979 shutdown_connections 2023-08-27T11:25:30.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.354+0000 7f281a17d700 1 -- 172.21.15.112:0/2162912979 wait complete. 2023-08-27T11:25:30.358 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:25:30.370 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:25:30.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.494+0000 7f2a88fce700 1 Processor -- start 2023-08-27T11:25:30.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.494+0000 7f2a88fce700 1 -- start start 2023-08-27T11:25:30.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.495+0000 7f2a88fce700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2a8406c6e0 0x7f2a8415b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.495+0000 7f2a88fce700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a840b6560 con 0x7f2a840b57a0 2023-08-27T11:25:30.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.495+0000 7f2a88fce700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a840b6840 con 0x7f2a84144bd0 2023-08-27T11:25:30.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.495+0000 7f2a88fce700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a840b6340 con 0x7f2a8406c6e0 2023-08-27T11:25:30.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.496+0000 7f2a82d9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2a840b57a0 0x7f2a840b5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54034/0 (socket says 172.21.15.112:54034) 2023-08-27T11:25:30.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.496+0000 7f2a81d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2a8406c6e0 0x7f2a8415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.496+0000 7f2a82d9d700 1 -- 172.21.15.112:0/3353423452 learned_addr learned my addr 172.21.15.112:0/3353423452 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:30.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.496+0000 7f2a8159a700 1 -- 172.21.15.112:0/3353423452 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147976395 0 0) 0x7f2a840b6560 con 0x7f2a840b57a0 2023-08-27T11:25:30.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.496+0000 7f2a8159a700 1 -- 172.21.15.112:0/3353423452 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a6c0035b0 con 0x7f2a840b57a0 2023-08-27T11:25:30.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.496+0000 7f2a81d9b700 1 -- 172.21.15.112:0/3353423452 >> v1:172.21.15.138:6789/0 conn(0x7f2a84144bd0 legacy=0x7f2a8406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.496+0000 7f2a81d9b700 1 -- 172.21.15.112:0/3353423452 >> v1:172.21.15.112:6789/0 conn(0x7f2a840b57a0 legacy=0x7f2a840b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.496+0000 7f2a81d9b700 1 -- 172.21.15.112:0/3353423452 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a840b6b20 con 0x7f2a8406c6e0 2023-08-27T11:25:30.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.496+0000 7f2a81d9b700 1 --2- 172.21.15.112:0/3353423452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8406c6e0 0x7f2a8415b910 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009840 tx=0x7f2a74009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc726daf075b7a3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:30.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.496+0000 7f2a8159a700 1 -- 172.21.15.112:0/3353423452 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74011830 con 0x7f2a8406c6e0 2023-08-27T11:25:30.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.496+0000 7f2a8159a700 1 -- 172.21.15.112:0/3353423452 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a74011990 con 0x7f2a8406c6e0 2023-08-27T11:25:30.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.496+0000 7f2a8159a700 1 -- 172.21.15.112:0/3353423452 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74011d20 con 0x7f2a8406c6e0 2023-08-27T11:25:30.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.497+0000 7f2a88fce700 1 -- 172.21.15.112:0/3353423452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8406c6e0 msgr2=0x7f2a8415b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.497+0000 7f2a88fce700 1 --2- 172.21.15.112:0/3353423452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8406c6e0 0x7f2a8415b910 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009840 tx=0x7f2a74009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:30.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.497+0000 7f2a88fce700 1 -- 172.21.15.112:0/3353423452 shutdown_connections 2023-08-27T11:25:30.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.497+0000 7f2a88fce700 1 --2- 172.21.15.112:0/3353423452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8406c6e0 0x7f2a8415b910 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.497+0000 7f2a88fce700 1 -- 172.21.15.112:0/3353423452 >> 172.21.15.112:0/3353423452 conn(0x7f2a84066010 msgr2=0x7f2a84066420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:30.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.497+0000 7f2a88fce700 1 -- 172.21.15.112:0/3353423452 shutdown_connections 2023-08-27T11:25:30.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.497+0000 7f2a88fce700 1 -- 172.21.15.112:0/3353423452 wait complete. 2023-08-27T11:25:30.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.498+0000 7f2a88fce700 1 Processor -- start 2023-08-27T11:25:30.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.498+0000 7f2a88fce700 1 -- start start 2023-08-27T11:25:30.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a88fce700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a8406c6e0 0x7f2a84159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a88fce700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840b57a0 0x7f2a84142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a88fce700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a84144bd0 0x7f2a8413d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a88fce700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8415dba0 con 0x7f2a840b57a0 2023-08-27T11:25:30.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a88fce700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8413e670 con 0x7f2a84144bd0 2023-08-27T11:25:30.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a88fce700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8415ddd0 con 0x7f2a8406c6e0 2023-08-27T11:25:30.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a81d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840b57a0 0x7f2a84142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a81d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840b57a0 0x7f2a84142020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59656/0 (socket says 172.21.15.112:59656) 2023-08-27T11:25:30.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a81d9b700 1 -- 172.21.15.112:0/1846831580 learned_addr learned my addr 172.21.15.112:0/1846831580 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:30.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a82d9d700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a84144bd0 0x7f2a8413d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a8259c700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a8406c6e0 0x7f2a84159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a81d9b700 1 -- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a8406c6e0 msgr2=0x7f2a84159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a81d9b700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a8406c6e0 0x7f2a84159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a81d9b700 1 -- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a84144bd0 msgr2=0x7f2a8413d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a81d9b700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a84144bd0 0x7f2a8413d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a81d9b700 1 -- 172.21.15.112:0/1846831580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a740094a0 con 0x7f2a840b57a0 2023-08-27T11:25:30.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a82d9d700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a84144bd0 0x7f2a8413d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:30.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.499+0000 7f2a81d9b700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840b57a0 0x7f2a84142020 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009fd0 tx=0x7f2a74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:30.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.500+0000 7f2a72ffd700 1 -- 172.21.15.112:0/1846831580 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a740038a0 con 0x7f2a840b57a0 2023-08-27T11:25:30.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.500+0000 7f2a72ffd700 1 -- 172.21.15.112:0/1846831580 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7401f070 con 0x7f2a840b57a0 2023-08-27T11:25:30.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.500+0000 7f2a88fce700 1 -- 172.21.15.112:0/1846831580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8413e0b0 con 0x7f2a840b57a0 2023-08-27T11:25:30.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.500+0000 7f2a72ffd700 1 -- 172.21.15.112:0/1846831580 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74003c40 con 0x7f2a840b57a0 2023-08-27T11:25:30.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.500+0000 7f2a88fce700 1 -- 172.21.15.112:0/1846831580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8415e070 con 0x7f2a840b57a0 2023-08-27T11:25:30.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.501+0000 7f2a88fce700 1 -- 172.21.15.112:0/1846831580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a84067a20 con 0x7f2a840b57a0 2023-08-27T11:25:30.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.501+0000 7f2a72ffd700 1 -- 172.21.15.112:0/1846831580 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f2a74026480 con 0x7f2a840b57a0 2023-08-27T11:25:30.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.501+0000 7f2a72ffd700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2a6406a320 0x7f2a6406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.501+0000 7f2a72ffd700 1 -- 172.21.15.112:0/1846831580 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f2a74021070 con 0x7f2a840b57a0 2023-08-27T11:25:30.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.502+0000 7f2a8259c700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2a6406a320 0x7f2a6406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.502+0000 7f2a8259c700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2a6406a320 0x7f2a6406c7e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2a78000f60 tx=0x7f2a78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:30.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.506+0000 7f2a72ffd700 1 -- 172.21.15.112:0/1846831580 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2a740bc990 con 0x7f2a840b57a0 2023-08-27T11:25:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:30.699+0000 7f2a88fce700 1 -- 172.21.15.112:0/1846831580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2a840d1cb0 con 0x7f2a840b57a0 2023-08-27T11:25:31.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.359+0000 7f2a72ffd700 1 -- 172.21.15.112:0/1846831580 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2a740150d0 con 0x7f2a840b57a0 2023-08-27T11:25:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.363+0000 7f2a88fce700 1 -- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2a6406a320 msgr2=0x7f2a6406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.363+0000 7f2a88fce700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2a6406a320 0x7f2a6406c7e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2a78000f60 tx=0x7f2a78009040 comp rx=0 tx=0).stop 2023-08-27T11:25:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.363+0000 7f2a88fce700 1 -- 172.21.15.112:0/1846831580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840b57a0 msgr2=0x7f2a84142020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.363+0000 7f2a88fce700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840b57a0 0x7f2a84142020 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009fd0 tx=0x7f2a74004750 comp rx=0 tx=0).stop 2023-08-27T11:25:31.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.363+0000 7f2a88fce700 1 -- 172.21.15.112:0/1846831580 shutdown_connections 2023-08-27T11:25:31.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.363+0000 7f2a88fce700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2a6406a320 0x7f2a6406c7e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.363+0000 7f2a88fce700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a84144bd0 0x7f2a8413d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.363+0000 7f2a88fce700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a8406c6e0 0x7f2a84159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.363+0000 7f2a88fce700 1 --2- 172.21.15.112:0/1846831580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840b57a0 0x7f2a84142020 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.363+0000 7f2a88fce700 1 -- 172.21.15.112:0/1846831580 >> 172.21.15.112:0/1846831580 conn(0x7f2a84066010 msgr2=0x7f2a84159db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:31.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.363+0000 7f2a88fce700 1 -- 172.21.15.112:0/1846831580 shutdown_connections 2023-08-27T11:25:31.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.364+0000 7f2a88fce700 1 -- 172.21.15.112:0/1846831580 wait complete. 2023-08-27T11:25:31.367 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2023-08-27T11:25:31.380 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:25:31.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.504+0000 7fefd83ac700 1 Processor -- start 2023-08-27T11:25:31.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.504+0000 7fefd83ac700 1 -- start start 2023-08-27T11:25:31.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.505+0000 7fefd83ac700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefd0071980 0x7fefd00cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:31.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.505+0000 7fefd83ac700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fefd015ea00 0x7fefd00ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:31.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.505+0000 7fefd83ac700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd015b270 con 0x7fefd015a760 2023-08-27T11:25:31.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.505+0000 7fefd83ac700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd015e5e0 con 0x7fefd015ea00 2023-08-27T11:25:31.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.505+0000 7fefd83ac700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd015b510 con 0x7fefd0071980 2023-08-27T11:25:31.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.505+0000 7fefd5947700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fefd015ea00 0x7fefd00ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:31.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.505+0000 7fefd6148700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefd0071980 0x7fefd00cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:31.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.505+0000 7fefd5947700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fefd015ea00 0x7fefd00ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:60092/0 (socket says 172.21.15.112:60092) 2023-08-27T11:25:31.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.505+0000 7fefd5947700 1 -- 172.21.15.112:0/1931866017 learned_addr learned my addr 172.21.15.112:0/1931866017 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:31.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.505+0000 7fefd6949700 1 --1- 172.21.15.112:0/1931866017 >> v1:172.21.15.138:6789/0 conn(0x7fefd015a760 0x7fefd015ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:33342/0 (socket says 172.21.15.112:33342) 2023-08-27T11:25:31.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.505+0000 7fefd6148700 1 --2- 172.21.15.112:0/1931866017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefd0071980 0x7fefd00cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:43086/0 (socket says 172.21.15.112:43086) 2023-08-27T11:25:31.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.506+0000 7fefd6148700 1 -- 172.21.15.112:0/1931866017 >> v2:172.21.15.138:3300/0 conn(0x7fefd015ea00 msgr2=0x7fefd00ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:31.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.506+0000 7fefd6148700 1 --2- 172.21.15.112:0/1931866017 >> v2:172.21.15.138:3300/0 conn(0x7fefd015ea00 0x7fefd00ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.506+0000 7fefd6148700 1 -- 172.21.15.112:0/1931866017 >> v1:172.21.15.138:6789/0 conn(0x7fefd015a760 legacy=0x7fefd015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:31.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.506+0000 7fefd6148700 1 -- 172.21.15.112:0/1931866017 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd015df20 con 0x7fefd0071980 2023-08-27T11:25:31.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.506+0000 7fefd5947700 1 --2- 172.21.15.112:0/1931866017 >> v2:172.21.15.138:3300/0 conn(0x7fefd015ea00 0x7fefd00ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:31.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.506+0000 7fefd6148700 1 --2- 172.21.15.112:0/1931866017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefd0071980 0x7fefd00cc960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009840 tx=0x7fefcc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42d153c26e5fc736 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:31.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.506+0000 7fefd5146700 1 -- 172.21.15.112:0/1931866017 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc011830 con 0x7fefd0071980 2023-08-27T11:25:31.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.506+0000 7fefd5146700 1 -- 172.21.15.112:0/1931866017 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc011990 con 0x7fefd0071980 2023-08-27T11:25:31.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.506+0000 7fefd5146700 1 -- 172.21.15.112:0/1931866017 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc011d20 con 0x7fefd0071980 2023-08-27T11:25:31.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.506+0000 7fefd83ac700 1 -- 172.21.15.112:0/1931866017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefd0071980 msgr2=0x7fefd00cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:31.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.506+0000 7fefd83ac700 1 --2- 172.21.15.112:0/1931866017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefd0071980 0x7fefd00cc960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009840 tx=0x7fefcc009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:31.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.507+0000 7fefd83ac700 1 -- 172.21.15.112:0/1931866017 shutdown_connections 2023-08-27T11:25:31.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.507+0000 7fefd83ac700 1 --2- 172.21.15.112:0/1931866017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefd0071980 0x7fefd00cc960 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.507+0000 7fefd83ac700 1 --2- 172.21.15.112:0/1931866017 >> v2:172.21.15.138:3300/0 conn(0x7fefd015ea00 0x7fefd00ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.507+0000 7fefd83ac700 1 -- 172.21.15.112:0/1931866017 >> 172.21.15.112:0/1931866017 conn(0x7fefd006bbf0 msgr2=0x7fefd00cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:31.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.507+0000 7fefd83ac700 1 -- 172.21.15.112:0/1931866017 shutdown_connections 2023-08-27T11:25:31.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.507+0000 7fefd83ac700 1 -- 172.21.15.112:0/1931866017 wait complete. 2023-08-27T11:25:31.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.508+0000 7fefd83ac700 1 Processor -- start 2023-08-27T11:25:31.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.508+0000 7fefd83ac700 1 -- start start 2023-08-27T11:25:31.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd83ac700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefd0071980 0x7fefd01410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:31.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd83ac700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd015a760 0x7fefd006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:31.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd83ac700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefd015ea00 0x7fefd006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:31.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd83ac700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd00b5c30 con 0x7fefd015a760 2023-08-27T11:25:31.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd83ac700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd01583b0 con 0x7fefd015ea00 2023-08-27T11:25:31.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd83ac700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd015cf80 con 0x7fefd0071980 2023-08-27T11:25:31.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd5947700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd015a760 0x7fefd006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:31.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd6148700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefd0071980 0x7fefd01410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:31.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd5947700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd015a760 0x7fefd006d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59672/0 (socket says 172.21.15.112:59672) 2023-08-27T11:25:31.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd5947700 1 -- 172.21.15.112:0/144098577 learned_addr learned my addr 172.21.15.112:0/144098577 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:31.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd6949700 1 --2- 172.21.15.112:0/144098577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefd015ea00 0x7fefd006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:31.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd5947700 1 -- 172.21.15.112:0/144098577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefd0071980 msgr2=0x7fefd01410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:31.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd5947700 1 --2- 172.21.15.112:0/144098577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefd0071980 0x7fefd01410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd5947700 1 -- 172.21.15.112:0/144098577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefd015ea00 msgr2=0x7fefd006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:31.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd5947700 1 --2- 172.21.15.112:0/144098577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefd015ea00 0x7fefd006dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd5947700 1 -- 172.21.15.112:0/144098577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc0009550 con 0x7fefd015a760 2023-08-27T11:25:31.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd5947700 1 --2- 172.21.15.112:0/144098577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd015a760 0x7fefd006d6f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fefc000e9b0 tx=0x7fefc000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:31.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefc6ffd700 1 -- 172.21.15.112:0/144098577 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc000ee50 con 0x7fefd015a760 2023-08-27T11:25:31.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefc6ffd700 1 -- 172.21.15.112:0/144098577 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc000c850 con 0x7fefd015a760 2023-08-27T11:25:31.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd83ac700 1 -- 172.21.15.112:0/144098577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefcc0094a0 con 0x7fefd015a760 2023-08-27T11:25:31.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefc6ffd700 1 -- 172.21.15.112:0/144098577 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc000cbc0 con 0x7fefd015a760 2023-08-27T11:25:31.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.509+0000 7fefd83ac700 1 -- 172.21.15.112:0/144098577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd015d540 con 0x7fefd015a760 2023-08-27T11:25:31.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.510+0000 7fefd83ac700 1 -- 172.21.15.112:0/144098577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd0066fa0 con 0x7fefd015a760 2023-08-27T11:25:31.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.511+0000 7fefc6ffd700 1 -- 172.21.15.112:0/144098577 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fefc000cd20 con 0x7fefd015a760 2023-08-27T11:25:31.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.511+0000 7fefc6ffd700 1 --2- 172.21.15.112:0/144098577 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fefb406a320 0x7fefb406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:31.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.511+0000 7fefc6ffd700 1 -- 172.21.15.112:0/144098577 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fefc0088620 con 0x7fefd015a760 2023-08-27T11:25:31.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.511+0000 7fefd6148700 1 --2- 172.21.15.112:0/144098577 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fefb406a320 0x7fefb406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:31.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.512+0000 7fefd6148700 1 --2- 172.21.15.112:0/144098577 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fefb406a320 0x7fefb406c7e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fefcc000c00 tx=0x7fefcc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:31.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.515+0000 7fefc6ffd700 1 -- 172.21.15.112:0/144098577 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fefc0053cb0 con 0x7fefd015a760 2023-08-27T11:25:31.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.713+0000 7fefd83ac700 1 -- 172.21.15.112:0/144098577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fefd0158560 con 0x7fefd015a760 2023-08-27T11:25:31.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.717+0000 7fefc6ffd700 1 -- 172.21.15.112:0/144098577 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fefd0158560 con 0x7fefd015a760 2023-08-27T11:25:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.719+0000 7fefd83ac700 1 -- 172.21.15.112:0/144098577 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fefb406a320 msgr2=0x7fefb406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.719+0000 7fefd83ac700 1 --2- 172.21.15.112:0/144098577 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fefb406a320 0x7fefb406c7e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fefcc000c00 tx=0x7fefcc008030 comp rx=0 tx=0).stop 2023-08-27T11:25:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.719+0000 7fefd83ac700 1 -- 172.21.15.112:0/144098577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd015a760 msgr2=0x7fefd006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.719+0000 7fefd83ac700 1 --2- 172.21.15.112:0/144098577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd015a760 0x7fefd006d6f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fefc000e9b0 tx=0x7fefc000ecc0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.719+0000 7fefd83ac700 1 -- 172.21.15.112:0/144098577 shutdown_connections 2023-08-27T11:25:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.719+0000 7fefd83ac700 1 --2- 172.21.15.112:0/144098577 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fefb406a320 0x7fefb406c7e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.719+0000 7fefd83ac700 1 --2- 172.21.15.112:0/144098577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefd015ea00 0x7fefd006dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.719+0000 7fefd83ac700 1 --2- 172.21.15.112:0/144098577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefd0071980 0x7fefd01410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.719+0000 7fefd83ac700 1 --2- 172.21.15.112:0/144098577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd015a760 0x7fefd006d6f0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.719+0000 7fefd83ac700 1 -- 172.21.15.112:0/144098577 >> 172.21.15.112:0/144098577 conn(0x7fefd006bbf0 msgr2=0x7fefd015ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:31.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.719+0000 7fefd83ac700 1 -- 172.21.15.112:0/144098577 shutdown_connections 2023-08-27T11:25:31.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.719+0000 7fefd83ac700 1 -- 172.21.15.112:0/144098577 wait complete. 2023-08-27T11:25:31.733 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2023-08-27T11:25:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.853+0000 7f80448f5700 1 Processor -- start 2023-08-27T11:25:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.853+0000 7f80448f5700 1 -- start start 2023-08-27T11:25:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f80448f5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8040151060 0x7f804015e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f80448f5700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f804015e990 0x7f804006adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f80448f5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f804006b2f0 0x7f804006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f80448f5700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804006b780 con 0x7f804006b2f0 2023-08-27T11:25:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f80448f5700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80400b02f0 con 0x7f804015e990 2023-08-27T11:25:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f80448f5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80400b05d0 con 0x7f8040151060 2023-08-27T11:25:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803ed9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f804006b2f0 0x7f804006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803ed9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f804006b2f0 0x7f804006d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59682/0 (socket says 172.21.15.112:59682) 2023-08-27T11:25:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803ed9d700 1 -- 172.21.15.112:0/935346419 learned_addr learned my addr 172.21.15.112:0/935346419 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803e59c700 1 --2- 172.21.15.112:0/935346419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8040151060 0x7f804015e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803dd9b700 1 --2- 172.21.15.112:0/935346419 >> v2:172.21.15.138:3300/0 conn(0x7f804015e990 0x7f804006adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803ed9d700 1 -- 172.21.15.112:0/935346419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8040151060 msgr2=0x7f804015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803ed9d700 1 --2- 172.21.15.112:0/935346419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8040151060 0x7f804015e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803ed9d700 1 -- 172.21.15.112:0/935346419 >> v2:172.21.15.138:3300/0 conn(0x7f804015e990 msgr2=0x7f804006adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:31.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803ed9d700 1 --2- 172.21.15.112:0/935346419 >> v2:172.21.15.138:3300/0 conn(0x7f804015e990 0x7f804006adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803ed9d700 1 -- 172.21.15.112:0/935346419 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80400b08b0 con 0x7f804006b2f0 2023-08-27T11:25:31.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803e59c700 1 --2- 172.21.15.112:0/935346419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8040151060 0x7f804015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:31.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803dd9b700 1 --2- 172.21.15.112:0/935346419 >> v2:172.21.15.138:3300/0 conn(0x7f804015e990 0x7f804006adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:31.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803ed9d700 1 --2- 172.21.15.112:0/935346419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804006b2f0 0x7f804006d580 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f80400b2c30 tx=0x7f803800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b07a18fd7e319a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:31.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803d59a700 1 -- 172.21.15.112:0/935346419 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803800e070 con 0x7f804006b2f0 2023-08-27T11:25:31.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.854+0000 7f803d59a700 1 -- 172.21.15.112:0/935346419 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803800b9c0 con 0x7f804006b2f0 2023-08-27T11:25:31.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.855+0000 7f803d59a700 1 -- 172.21.15.112:0/935346419 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803800bd70 con 0x7f804006b2f0 2023-08-27T11:25:31.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.855+0000 7f80448f5700 1 -- 172.21.15.112:0/935346419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804006b2f0 msgr2=0x7f804006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:31.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.855+0000 7f80448f5700 1 --2- 172.21.15.112:0/935346419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804006b2f0 0x7f804006d580 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f80400b2c30 tx=0x7f803800b330 comp rx=0 tx=0).stop 2023-08-27T11:25:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.855+0000 7f80448f5700 1 -- 172.21.15.112:0/935346419 shutdown_connections 2023-08-27T11:25:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.855+0000 7f80448f5700 1 --2- 172.21.15.112:0/935346419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804006b2f0 0x7f804006d580 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.855+0000 7f80448f5700 1 --2- 172.21.15.112:0/935346419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8040151060 0x7f804015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.855+0000 7f80448f5700 1 --2- 172.21.15.112:0/935346419 >> v2:172.21.15.138:3300/0 conn(0x7f804015e990 0x7f804006adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.855+0000 7f80448f5700 1 -- 172.21.15.112:0/935346419 >> 172.21.15.112:0/935346419 conn(0x7f80400d0000 msgr2=0x7f80400d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.855+0000 7f80448f5700 1 -- 172.21.15.112:0/935346419 shutdown_connections 2023-08-27T11:25:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.855+0000 7f80448f5700 1 -- 172.21.15.112:0/935346419 wait complete. 2023-08-27T11:25:31.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.856+0000 7f80448f5700 1 Processor -- start 2023-08-27T11:25:31.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.857+0000 7f80448f5700 1 -- start start 2023-08-27T11:25:31.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.857+0000 7f80448f5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f804006b2f0 0x7f80400c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:31.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.857+0000 7f80448f5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040151060 0x7f80400c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:31.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.857+0000 7f80448f5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804015e990 0x7f80401f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:31.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.857+0000 7f80448f5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804015cca0 con 0x7f804015e990 2023-08-27T11:25:31.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.857+0000 7f80448f5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804015cef0 con 0x7f8040151060 2023-08-27T11:25:31.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.857+0000 7f80448f5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804015d1d0 con 0x7f804006b2f0 2023-08-27T11:25:31.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.857+0000 7f803ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804015e990 0x7f80401f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:31.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.857+0000 7f803ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804015e990 0x7f80401f66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59688/0 (socket says 172.21.15.112:59688) 2023-08-27T11:25:31.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.857+0000 7f803ed9d700 1 -- 172.21.15.112:0/2202565716 learned_addr learned my addr 172.21.15.112:0/2202565716 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:31.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.857+0000 7f803e59c700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f804006b2f0 0x7f80400c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:31.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.857+0000 7f803dd9b700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040151060 0x7f80400c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:31.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.858+0000 7f803ed9d700 1 -- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f804006b2f0 msgr2=0x7f80400c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:31.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.858+0000 7f803ed9d700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f804006b2f0 0x7f80400c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.858+0000 7f803ed9d700 1 -- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040151060 msgr2=0x7f80400c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:31.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.858+0000 7f803ed9d700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040151060 0x7f80400c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.858+0000 7f803ed9d700 1 -- 172.21.15.112:0/2202565716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8038009b30 con 0x7f804015e990 2023-08-27T11:25:31.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.858+0000 7f803e59c700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f804006b2f0 0x7f80400c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:31.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.858+0000 7f803ed9d700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804015e990 0x7f80401f66a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f804006e590 tx=0x7f8038004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:31.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.858+0000 7f802effd700 1 -- 172.21.15.112:0/2202565716 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380099b0 con 0x7f804015e990 2023-08-27T11:25:31.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.858+0000 7f802effd700 1 -- 172.21.15.112:0/2202565716 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8038004250 con 0x7f804015e990 2023-08-27T11:25:31.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.858+0000 7f80448f5700 1 -- 172.21.15.112:0/2202565716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804015d4b0 con 0x7f804015e990 2023-08-27T11:25:31.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.858+0000 7f802effd700 1 -- 172.21.15.112:0/2202565716 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8038004600 con 0x7f804015e990 2023-08-27T11:25:31.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.858+0000 7f80448f5700 1 -- 172.21.15.112:0/2202565716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804015d790 con 0x7f804015e990 2023-08-27T11:25:31.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.859+0000 7f80448f5700 1 -- 172.21.15.112:0/2202565716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80401f61d0 con 0x7f804015e990 2023-08-27T11:25:31.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.860+0000 7f802effd700 1 -- 172.21.15.112:0/2202565716 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f8038004760 con 0x7f804015e990 2023-08-27T11:25:31.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.860+0000 7f802effd700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f802006a320 0x7f802006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:31.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.860+0000 7f802effd700 1 -- 172.21.15.112:0/2202565716 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8038089850 con 0x7f804015e990 2023-08-27T11:25:31.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.860+0000 7f803e59c700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f802006a320 0x7f802006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:31.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.861+0000 7f803e59c700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f802006a320 0x7f802006c7e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f804015cc30 tx=0x7f8030008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:31.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:31.864+0000 7f802effd700 1 -- 172.21.15.112:0/2202565716 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8038054e60 con 0x7f804015e990 2023-08-27T11:25:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.130+0000 7f80448f5700 1 -- 172.21.15.112:0/2202565716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) v1 -- 0x7f8040067a20 con 0x7f804015e990 2023-08-27T11:25:32.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.362+0000 7f802effd700 1 -- 172.21.15.112:0/2202565716 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v348) v1 ==== 158+0+0 (secure 0 0 0) 0x7f803801f0e0 con 0x7f804015e990 2023-08-27T11:25:32.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.364+0000 7f80448f5700 1 -- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f802006a320 msgr2=0x7f802006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:32.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.364+0000 7f80448f5700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f802006a320 0x7f802006c7e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f804015cc30 tx=0x7f8030008040 comp rx=0 tx=0).stop 2023-08-27T11:25:32.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.364+0000 7f80448f5700 1 -- 172.21.15.112:0/2202565716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804015e990 msgr2=0x7f80401f66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:32.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.364+0000 7f80448f5700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804015e990 0x7f80401f66a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f804006e590 tx=0x7f8038004ad0 comp rx=0 tx=0).stop 2023-08-27T11:25:32.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.364+0000 7f80448f5700 1 -- 172.21.15.112:0/2202565716 shutdown_connections 2023-08-27T11:25:32.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.364+0000 7f80448f5700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f802006a320 0x7f802006c7e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:32.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.364+0000 7f80448f5700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804015e990 0x7f80401f66a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:32.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.364+0000 7f80448f5700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f804006b2f0 0x7f80400c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:32.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.364+0000 7f80448f5700 1 --2- 172.21.15.112:0/2202565716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040151060 0x7f80400c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.364+0000 7f80448f5700 1 -- 172.21.15.112:0/2202565716 >> 172.21.15.112:0/2202565716 conn(0x7f80400d0000 msgr2=0x7f80400d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.364+0000 7f80448f5700 1 -- 172.21.15.112:0/2202565716 shutdown_connections 2023-08-27T11:25:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.364+0000 7f80448f5700 1 -- 172.21.15.112:0/2202565716 wait complete. 2023-08-27T11:25:32.378 INFO:tasks.cephfs_test_runner:Tests the pending_subvolume_deletions in 'fs volume info' command ... ok 2023-08-27T11:25:32.378 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_with_human_readable_flag (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info --human_readable' command 2023-08-27T11:25:32.379 INFO:tasks.cephfs_test_runner:test_volume_info_with_human_readable_flag (tasks.cephfs.test_volumes.TestVolumes) 2023-08-27T11:25:32.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag' 2023-08-27T11:25:32.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.498+0000 7f5f09cf9700 1 Processor -- start 2023-08-27T11:25:32.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.498+0000 7f5f09cf9700 1 -- start start 2023-08-27T11:25:32.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f09cf9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f0406adb0 0x7f5f0406b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:32.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f09cf9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0415eb60 con 0x7f5f04151060 2023-08-27T11:25:32.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f09cf9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f040b02f0 con 0x7f5f040cdbd0 2023-08-27T11:25:32.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f09cf9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0406b770 con 0x7f5f0406adb0 2023-08-27T11:25:32.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f037fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5f04151060 0x7f5f0415e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54074/0 (socket says 172.21.15.112:54074) 2023-08-27T11:25:32.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f037fe700 1 -- 172.21.15.112:0/1566688308 learned_addr learned my addr 172.21.15.112:0/1566688308 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:32.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f02ffd700 1 --2- 172.21.15.112:0/1566688308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f0406adb0 0x7f5f0406b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:32.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f027fc700 1 -- 172.21.15.112:0/1566688308 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474807181 0 0) 0x7f5f0415eb60 con 0x7f5f04151060 2023-08-27T11:25:32.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f027fc700 1 -- 172.21.15.112:0/1566688308 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ef00035b0 con 0x7f5f04151060 2023-08-27T11:25:32.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f027fc700 1 -- 172.21.15.112:0/1566688308 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204975229 0 0) 0x7f5f040b02f0 con 0x7f5f040cdbd0 2023-08-27T11:25:32.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f027fc700 1 -- 172.21.15.112:0/1566688308 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f0415eb60 con 0x7f5f040cdbd0 2023-08-27T11:25:32.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f027fc700 1 -- 172.21.15.112:0/1566688308 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 356611136 0 0) 0x7f5ef00035b0 con 0x7f5f04151060 2023-08-27T11:25:32.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f02ffd700 1 -- 172.21.15.112:0/1566688308 >> v1:172.21.15.138:6789/0 conn(0x7f5f040cdbd0 legacy=0x7f5f040ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:32.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f02ffd700 1 -- 172.21.15.112:0/1566688308 >> v1:172.21.15.112:6789/0 conn(0x7f5f04151060 legacy=0x7f5f0415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:32.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.499+0000 7f5f02ffd700 1 -- 172.21.15.112:0/1566688308 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f040b05d0 con 0x7f5f0406adb0 2023-08-27T11:25:32.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.500+0000 7f5f02ffd700 1 --2- 172.21.15.112:0/1566688308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f0406adb0 0x7f5f0406b230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009870 tx=0x7f5ef4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8fe07ff17892b5e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:32.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.500+0000 7f5f027fc700 1 -- 172.21.15.112:0/1566688308 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec001670 con 0x7f5f0406adb0 2023-08-27T11:25:32.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.500+0000 7f5f027fc700 1 -- 172.21.15.112:0/1566688308 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef4004030 con 0x7f5f0406adb0 2023-08-27T11:25:32.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.500+0000 7f5f027fc700 1 -- 172.21.15.112:0/1566688308 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef40043a0 con 0x7f5f0406adb0 2023-08-27T11:25:32.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.500+0000 7f5f09cf9700 1 -- 172.21.15.112:0/1566688308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f0406adb0 msgr2=0x7f5f0406b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:32.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.500+0000 7f5f09cf9700 1 --2- 172.21.15.112:0/1566688308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f0406adb0 0x7f5f0406b230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009870 tx=0x7f5ef4009b80 comp rx=0 tx=0).stop 2023-08-27T11:25:32.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.500+0000 7f5f09cf9700 1 -- 172.21.15.112:0/1566688308 shutdown_connections 2023-08-27T11:25:32.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.500+0000 7f5f09cf9700 1 --2- 172.21.15.112:0/1566688308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f0406adb0 0x7f5f0406b230 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:32.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.500+0000 7f5f09cf9700 1 -- 172.21.15.112:0/1566688308 >> 172.21.15.112:0/1566688308 conn(0x7f5f040d0000 msgr2=0x7f5f040d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.500+0000 7f5f09cf9700 1 -- 172.21.15.112:0/1566688308 shutdown_connections 2023-08-27T11:25:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.500+0000 7f5f09cf9700 1 -- 172.21.15.112:0/1566688308 wait complete. 2023-08-27T11:25:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.502+0000 7f5f09cf9700 1 Processor -- start 2023-08-27T11:25:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.502+0000 7f5f09cf9700 1 -- start start 2023-08-27T11:25:32.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f09cf9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f0406adb0 0x7f5f040c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:32.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f09cf9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f040cdbd0 0x7f5f041f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:32.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f09cf9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f04151060 0x7f5f04159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:32.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f09cf9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f041f3ec0 con 0x7f5f040cdbd0 2023-08-27T11:25:32.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f09cf9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f041f0b40 con 0x7f5f04151060 2023-08-27T11:25:32.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f09cf9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f041f6960 con 0x7f5f0406adb0 2023-08-27T11:25:32.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f02ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f040cdbd0 0x7f5f041f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:32.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f02ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f040cdbd0 0x7f5f041f3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59700/0 (socket says 172.21.15.112:59700) 2023-08-27T11:25:32.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f02ffd700 1 -- 172.21.15.112:0/358752052 learned_addr learned my addr 172.21.15.112:0/358752052 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:32.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f037fe700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f0406adb0 0x7f5f040c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:32.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f03fff700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f04151060 0x7f5f04159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:32.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f02ffd700 1 -- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f0406adb0 msgr2=0x7f5f040c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:32.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f02ffd700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f0406adb0 0x7f5f040c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:32.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f02ffd700 1 -- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f04151060 msgr2=0x7f5f04159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:32.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f02ffd700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f04151060 0x7f5f04159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:32.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f02ffd700 1 -- 172.21.15.112:0/358752052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40094a0 con 0x7f5f040cdbd0 2023-08-27T11:25:32.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f03fff700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f04151060 0x7f5f04159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:32.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f02ffd700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f040cdbd0 0x7f5f041f3980 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40097c0 tx=0x7f5ef4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:32.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5efbfff700 1 -- 172.21.15.112:0/358752052 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4003b80 con 0x7f5f040cdbd0 2023-08-27T11:25:32.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5efbfff700 1 -- 172.21.15.112:0/358752052 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef401f070 con 0x7f5f040cdbd0 2023-08-27T11:25:32.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f09cf9700 1 -- 172.21.15.112:0/358752052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f041f6c40 con 0x7f5f040cdbd0 2023-08-27T11:25:32.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5efbfff700 1 -- 172.21.15.112:0/358752052 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4011630 con 0x7f5f040cdbd0 2023-08-27T11:25:32.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.503+0000 7f5f09cf9700 1 -- 172.21.15.112:0/358752052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f041f7200 con 0x7f5f040cdbd0 2023-08-27T11:25:32.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.504+0000 7f5f09cf9700 1 -- 172.21.15.112:0/358752052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f041f7390 con 0x7f5f040cdbd0 2023-08-27T11:25:32.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.505+0000 7f5efbfff700 1 -- 172.21.15.112:0/358752052 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f5ef4021030 con 0x7f5f040cdbd0 2023-08-27T11:25:32.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.505+0000 7f5efbfff700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ee406a350 0x7f5ee406c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:32.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.505+0000 7f5efbfff700 1 -- 172.21.15.112:0/358752052 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5ef4089870 con 0x7f5f040cdbd0 2023-08-27T11:25:32.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.505+0000 7f5f037fe700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ee406a350 0x7f5ee406c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:32.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.506+0000 7f5f037fe700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ee406a350 0x7f5ee406c810 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5f040ca0d0 tx=0x7f5eec000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:32.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.509+0000 7f5efbfff700 1 -- 172.21.15.112:0/358752052 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5ef4054f90 con 0x7f5f040cdbd0 2023-08-27T11:25:32.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:32.778+0000 7f5f09cf9700 1 -- 172.21.15.112:0/358752052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]} v 0) v1 -- 0x7f5f04067a20 con 0x7f5f040cdbd0 2023-08-27T11:25:33.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.361+0000 7f5efbfff700 1 -- 172.21.15.112:0/358752052 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]}]=0 v349) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5ef40585b0 con 0x7f5f040cdbd0 2023-08-27T11:25:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.363+0000 7f5f09cf9700 1 -- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ee406a350 msgr2=0x7f5ee406c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.363+0000 7f5f09cf9700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ee406a350 0x7f5ee406c810 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5f040ca0d0 tx=0x7f5eec000bc0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.363+0000 7f5f09cf9700 1 -- 172.21.15.112:0/358752052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f040cdbd0 msgr2=0x7f5f041f3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.363+0000 7f5f09cf9700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f040cdbd0 0x7f5f041f3980 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40097c0 tx=0x7f5ef4004750 comp rx=0 tx=0).stop 2023-08-27T11:25:33.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.363+0000 7f5f09cf9700 1 -- 172.21.15.112:0/358752052 shutdown_connections 2023-08-27T11:25:33.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.363+0000 7f5f09cf9700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ee406a350 0x7f5ee406c810 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.363+0000 7f5f09cf9700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f04151060 0x7f5f04159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.363+0000 7f5f09cf9700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f0406adb0 0x7f5f040c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.363+0000 7f5f09cf9700 1 --2- 172.21.15.112:0/358752052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f040cdbd0 0x7f5f041f3980 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.363+0000 7f5f09cf9700 1 -- 172.21.15.112:0/358752052 >> 172.21.15.112:0/358752052 conn(0x7f5f040d0000 msgr2=0x7f5f0406d480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.364+0000 7f5f09cf9700 1 -- 172.21.15.112:0/358752052 shutdown_connections 2023-08-27T11:25:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.364+0000 7f5f09cf9700 1 -- 172.21.15.112:0/358752052 wait complete. 2023-08-27T11:25:33.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.501+0000 7f48f93d7700 1 Processor -- start 2023-08-27T11:25:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.501+0000 7f48f93d7700 1 -- start start 2023-08-27T11:25:33.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f93d7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f48f41466c0 0x7f48f406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f93d7700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f406e3d0 con 0x7f48f40b15c0 2023-08-27T11:25:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f93d7700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f4161030 con 0x7f48f40b4760 2023-08-27T11:25:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f93d7700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f40b57e0 con 0x7f48f41466c0 2023-08-27T11:25:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f37fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f48f40b15c0 0x7f48f406dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54092/0 (socket says 172.21.15.112:54092) 2023-08-27T11:25:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f37fe700 1 -- 172.21.15.112:0/3162991960 learned_addr learned my addr 172.21.15.112:0/3162991960 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:33.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101478021 0 0) 0x7f48f406e3d0 con 0x7f48f40b15c0 2023-08-27T11:25:33.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48dc0035b0 con 0x7f48f40b15c0 2023-08-27T11:25:33.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f2ffd700 1 --2- 172.21.15.112:0/3162991960 >> v2:172.21.15.138:3300/0 conn(0x7f48f41466c0 0x7f48f406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2041548286 0 0) 0x7f48dc0035b0 con 0x7f48f40b15c0 2023-08-27T11:25:33.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 >> v2:172.21.15.138:3300/0 conn(0x7f48f41466c0 msgr2=0x7f48f406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:33.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 --2- 172.21.15.112:0/3162991960 >> v2:172.21.15.138:3300/0 conn(0x7f48f41466c0 0x7f48f406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 >> v1:172.21.15.138:6789/0 conn(0x7f48f40b4760 legacy=0x7f48f406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f4161310 con 0x7f48f40b15c0 2023-08-27T11:25:33.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f48ec0019f0 con 0x7f48f40b15c0 2023-08-27T11:25:33.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 >> v1:172.21.15.112:6789/0 conn(0x7f48f40b15c0 legacy=0x7f48f406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 --2- 172.21.15.112:0/3162991960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48dc003a00 0x7f48dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 --2- 172.21.15.112:0/3162991960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48dc004850 0x7f48dc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 --2- 172.21.15.112:0/3162991960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48dc005210 0x7f48dc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc00c130 con 0x7f48dc005210 2023-08-27T11:25:33.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc00c2a0 con 0x7f48dc003a00 2023-08-27T11:25:33.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.502+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc00c460 con 0x7f48dc004850 2023-08-27T11:25:33.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.503+0000 7f48f37fe700 1 --2- 172.21.15.112:0/3162991960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48dc005210 0x7f48dc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.503+0000 7f48f2ffd700 1 --2- 172.21.15.112:0/3162991960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48dc003a00 0x7f48dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.503+0000 7f48f27fc700 1 --2- 172.21.15.112:0/3162991960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48dc004850 0x7f48dc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.503+0000 7f48f37fe700 1 -- 172.21.15.112:0/3162991960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48dc004850 msgr2=0x7f48dc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:33.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.503+0000 7f48f37fe700 1 --2- 172.21.15.112:0/3162991960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48dc004850 0x7f48dc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.503+0000 7f48f37fe700 1 -- 172.21.15.112:0/3162991960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48dc003a00 msgr2=0x7f48dc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:33.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.503+0000 7f48f37fe700 1 --2- 172.21.15.112:0/3162991960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48dc003a00 0x7f48dc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.503+0000 7f48f37fe700 1 -- 172.21.15.112:0/3162991960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc0035b0 con 0x7f48dc005210 2023-08-27T11:25:33.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.503+0000 7f48f37fe700 1 -- 172.21.15.112:0/3162991960 reap_dead start 2023-08-27T11:25:33.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.503+0000 7f48f37fe700 1 --2- 172.21.15.112:0/3162991960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48dc005210 0x7f48dc00ba10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00a830 tx=0x7f48dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e113c3502cac1e22 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:33.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.503+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00a2f0 con 0x7f48dc005210 2023-08-27T11:25:33.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.503+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec012620 con 0x7f48dc005210 2023-08-27T11:25:33.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.503+0000 7f48f1ffb700 1 -- 172.21.15.112:0/3162991960 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0129d0 con 0x7f48dc005210 2023-08-27T11:25:33.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.504+0000 7f48f93d7700 1 -- 172.21.15.112:0/3162991960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48dc005210 msgr2=0x7f48dc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.504+0000 7f48f93d7700 1 --2- 172.21.15.112:0/3162991960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48dc005210 0x7f48dc00ba10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00a830 tx=0x7f48dc005880 comp rx=0 tx=0).stop 2023-08-27T11:25:33.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.504+0000 7f48f93d7700 1 -- 172.21.15.112:0/3162991960 shutdown_connections 2023-08-27T11:25:33.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.504+0000 7f48f93d7700 1 --2- 172.21.15.112:0/3162991960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48dc005210 0x7f48dc00ba10 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.504+0000 7f48f93d7700 1 -- 172.21.15.112:0/3162991960 >> 172.21.15.112:0/3162991960 conn(0x7f48f40cff80 msgr2=0x7f48f40d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.504+0000 7f48f93d7700 1 -- 172.21.15.112:0/3162991960 shutdown_connections 2023-08-27T11:25:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.504+0000 7f48f93d7700 1 -- 172.21.15.112:0/3162991960 wait complete. 2023-08-27T11:25:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.505+0000 7f48f93d7700 1 Processor -- start 2023-08-27T11:25:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.505+0000 7f48f93d7700 1 -- start start 2023-08-27T11:25:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f93d7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48dc005210 0x7f48f4144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f93d7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40ac890 0x7f48f41426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f93d7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48f40accf0 0x7f48f4142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f93d7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40ad880 con 0x7f48f40accf0 2023-08-27T11:25:33.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f93d7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40ae0c0 con 0x7f48dc005210 2023-08-27T11:25:33.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f93d7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40ad2c0 con 0x7f48f40ac890 2023-08-27T11:25:33.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48f40accf0 0x7f48f4142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48f40accf0 0x7f48f4142bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59722/0 (socket says 172.21.15.112:59722) 2023-08-27T11:25:33.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f37fe700 1 -- 172.21.15.112:0/3655059729 learned_addr learned my addr 172.21.15.112:0/3655059729 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:33.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f27fc700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40ac890 0x7f48f41426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f2ffd700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48dc005210 0x7f48f4144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f37fe700 1 -- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40ac890 msgr2=0x7f48f41426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f37fe700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40ac890 0x7f48f41426b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f37fe700 1 -- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48dc005210 msgr2=0x7f48f4144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f37fe700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48dc005210 0x7f48f4144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f37fe700 1 -- 172.21.15.112:0/3655059729 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f40ad5a0 con 0x7f48f40accf0 2023-08-27T11:25:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48f37fe700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48f40accf0 0x7f48f4142bf0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f48f40b3010 tx=0x7f48ec001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48db7fe700 1 -- 172.21.15.112:0/3655059729 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec024070 con 0x7f48f40accf0 2023-08-27T11:25:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.506+0000 7f48db7fe700 1 -- 172.21.15.112:0/3655059729 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec00a210 con 0x7f48f40accf0 2023-08-27T11:25:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.507+0000 7f48f93d7700 1 -- 172.21.15.112:0/3655059729 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f40ae360 con 0x7f48f40accf0 2023-08-27T11:25:33.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.507+0000 7f48db7fe700 1 -- 172.21.15.112:0/3655059729 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec01d420 con 0x7f48f40accf0 2023-08-27T11:25:33.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.507+0000 7f48f93d7700 1 -- 172.21.15.112:0/3655059729 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f4144b40 con 0x7f48f40accf0 2023-08-27T11:25:33.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.507+0000 7f48f93d7700 1 -- 172.21.15.112:0/3655059729 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f40ae5a0 con 0x7f48f40accf0 2023-08-27T11:25:33.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.508+0000 7f48db7fe700 1 -- 172.21.15.112:0/3655059729 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f48ec01d790 con 0x7f48f40accf0 2023-08-27T11:25:33.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.508+0000 7f48db7fe700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48e006a320 0x7f48e006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.508+0000 7f48db7fe700 1 -- 172.21.15.112:0/3655059729 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f48ec089e50 con 0x7f48f40accf0 2023-08-27T11:25:33.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.508+0000 7f48f2ffd700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48e006a320 0x7f48e006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.509+0000 7f48f2ffd700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48e006a320 0x7f48e006c7e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009980 tx=0x7f48e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:33.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.512+0000 7f48db7fe700 1 -- 172.21.15.112:0/3655059729 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f48f40ae5a0 con 0x7f48f40accf0 2023-08-27T11:25:33.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.703+0000 7f48f93d7700 1 -- 172.21.15.112:0/3655059729 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48f4067a20 con 0x7f48f40accf0 2023-08-27T11:25:33.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.704+0000 7f48db7fe700 1 -- 172.21.15.112:0/3655059729 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+14735 (secure 0 0 0) 0x7f48ec055610 con 0x7f48f40accf0 2023-08-27T11:25:33.706 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:33.706 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":68,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:25:31.351096+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/174707415":"2023-08-28T11:25:28.226554+0000","172.21.15.112:6833/174707415":"2023-08-28T11:25:28.226554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:33.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.707+0000 7f48f93d7700 1 -- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48e006a320 msgr2=0x7f48e006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.707+0000 7f48f93d7700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48e006a320 0x7f48e006c7e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009980 tx=0x7f48e4008040 comp rx=0 tx=0).stop 2023-08-27T11:25:33.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.707+0000 7f48f93d7700 1 -- 172.21.15.112:0/3655059729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48f40accf0 msgr2=0x7f48f4142bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.707+0000 7f48f93d7700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48f40accf0 0x7f48f4142bf0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f48f40b3010 tx=0x7f48ec001d80 comp rx=0 tx=0).stop 2023-08-27T11:25:33.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.707+0000 7f48f93d7700 1 -- 172.21.15.112:0/3655059729 shutdown_connections 2023-08-27T11:25:33.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.707+0000 7f48f93d7700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48e006a320 0x7f48e006c7e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.707+0000 7f48f93d7700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48f40accf0 0x7f48f4142bf0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.707+0000 7f48f93d7700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48dc005210 0x7f48f4144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.707+0000 7f48f93d7700 1 --2- 172.21.15.112:0/3655059729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40ac890 0x7f48f41426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.707+0000 7f48f93d7700 1 -- 172.21.15.112:0/3655059729 >> 172.21.15.112:0/3655059729 conn(0x7f48f40cff80 msgr2=0x7f48f4070f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:33.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.707+0000 7f48f93d7700 1 -- 172.21.15.112:0/3655059729 shutdown_connections 2023-08-27T11:25:33.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.707+0000 7f48f93d7700 1 -- 172.21.15.112:0/3655059729 wait complete. 2023-08-27T11:25:33.721 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:25:33.721 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:25:33.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.843+0000 7f7582612700 1 Processor -- start 2023-08-27T11:25:33.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.843+0000 7f7582612700 1 -- start start 2023-08-27T11:25:33.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.844+0000 7f7582612700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f757c06f790 0x7f757c06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.844+0000 7f7582612700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c0b5070 con 0x7f757c0b3440 2023-08-27T11:25:33.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.844+0000 7f7582612700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c147000 con 0x7f757c146c20 2023-08-27T11:25:33.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.844+0000 7f7582612700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757c070160 con 0x7f757c06f790 2023-08-27T11:25:33.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.844+0000 7f757b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f757c0b3440 0x7f757c15e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54110/0 (socket says 172.21.15.112:54110) 2023-08-27T11:25:33.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.844+0000 7f757b7fe700 1 -- 172.21.15.112:0/2128467007 learned_addr learned my addr 172.21.15.112:0/2128467007 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:33.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.844+0000 7f7580baf700 1 --2- 172.21.15.112:0/2128467007 >> v2:172.21.15.138:3300/0 conn(0x7f757c06f790 0x7f757c06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.844+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653295374 0 0) 0x7f7570001110 con 0x7f757c0b3440 2023-08-27T11:25:33.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.844+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75640035b0 con 0x7f757c0b3440 2023-08-27T11:25:33.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1537310571 0 0) 0x7f75640035b0 con 0x7f757c0b3440 2023-08-27T11:25:33.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 >> v2:172.21.15.138:3300/0 conn(0x7f757c06f790 msgr2=0x7f757c06fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:33.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 --2- 172.21.15.112:0/2128467007 >> v2:172.21.15.138:3300/0 conn(0x7f757c06f790 0x7f757c06fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 >> v1:172.21.15.138:6789/0 conn(0x7f757c146c20 legacy=0x7f757c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c161740 con 0x7f757c0b3440 2023-08-27T11:25:33.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f7570001800 con 0x7f757c0b3440 2023-08-27T11:25:33.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 >> v1:172.21.15.112:6789/0 conn(0x7f757c0b3440 legacy=0x7f757c15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 --2- 172.21.15.112:0/2128467007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7564003a00 0x7f7564003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 --2- 172.21.15.112:0/2128467007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7564004850 0x7f7564004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 --2- 172.21.15.112:0/2128467007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564005210 0x7f756400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c070160 con 0x7f7564005210 2023-08-27T11:25:33.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756400c1b0 con 0x7f7564004850 2023-08-27T11:25:33.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756400c370 con 0x7f7564003a00 2023-08-27T11:25:33.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f7580baf700 1 --2- 172.21.15.112:0/2128467007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564005210 0x7f756400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757bfff700 1 --2- 172.21.15.112:0/2128467007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7564003a00 0x7f7564003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757b7fe700 1 --2- 172.21.15.112:0/2128467007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7564004850 0x7f7564004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f7580baf700 1 -- 172.21.15.112:0/2128467007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7564003a00 msgr2=0x7f7564003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:33.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f7580baf700 1 --2- 172.21.15.112:0/2128467007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7564003a00 0x7f7564003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f7580baf700 1 -- 172.21.15.112:0/2128467007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7564004850 msgr2=0x7f7564004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:33.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f7580baf700 1 --2- 172.21.15.112:0/2128467007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7564004850 0x7f7564004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f7580baf700 1 -- 172.21.15.112:0/2128467007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75640035b0 con 0x7f7564005210 2023-08-27T11:25:33.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757b7fe700 1 --2- 172.21.15.112:0/2128467007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7564004850 0x7f7564004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:33.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f7580baf700 1 -- 172.21.15.112:0/2128467007 reap_dead start 2023-08-27T11:25:33.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f7580baf700 1 --2- 172.21.15.112:0/2128467007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564005210 0x7f756400ba10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7574009fc0 tx=0x7f7564005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88689a12f5437a7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:33.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574003a20 con 0x7f7564005210 2023-08-27T11:25:33.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757400f040 con 0x7f7564005210 2023-08-27T11:25:33.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.845+0000 7f757affd700 1 -- 172.21.15.112:0/2128467007 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574003de0 con 0x7f7564005210 2023-08-27T11:25:33.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.846+0000 7f7582612700 1 -- 172.21.15.112:0/2128467007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564005210 msgr2=0x7f756400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.846+0000 7f7582612700 1 --2- 172.21.15.112:0/2128467007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564005210 0x7f756400ba10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7574009fc0 tx=0x7f7564005880 comp rx=0 tx=0).stop 2023-08-27T11:25:33.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.846+0000 7f7582612700 1 -- 172.21.15.112:0/2128467007 shutdown_connections 2023-08-27T11:25:33.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.846+0000 7f7582612700 1 --2- 172.21.15.112:0/2128467007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564005210 0x7f756400ba10 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.846+0000 7f7582612700 1 -- 172.21.15.112:0/2128467007 >> 172.21.15.112:0/2128467007 conn(0x7f757c0b1e90 msgr2=0x7f757c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:33.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.846+0000 7f7582612700 1 -- 172.21.15.112:0/2128467007 shutdown_connections 2023-08-27T11:25:33.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.846+0000 7f7582612700 1 -- 172.21.15.112:0/2128467007 wait complete. 2023-08-27T11:25:33.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.847+0000 7f7582612700 1 Processor -- start 2023-08-27T11:25:33.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.847+0000 7f7582612700 1 -- start start 2023-08-27T11:25:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f7582612700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564005210 0x7f757c1449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f7582612700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0c73f0 0x7f757c0c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f7582612700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c1450c0 0x7f757c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f7582612700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c144f00 con 0x7f7564005210 2023-08-27T11:25:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f7582612700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0c6350 con 0x7f757c0c73f0 2023-08-27T11:25:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f7582612700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0c6630 con 0x7f757c1450c0 2023-08-27T11:25:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f757bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564005210 0x7f757c1449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f757bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564005210 0x7f757c1449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59738/0 (socket says 172.21.15.112:59738) 2023-08-27T11:25:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f757bfff700 1 -- 172.21.15.112:0/154835957 learned_addr learned my addr 172.21.15.112:0/154835957 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f7580baf700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c1450c0 0x7f757c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f757b7fe700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0c73f0 0x7f757c0c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f757bfff700 1 -- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c1450c0 msgr2=0x7f757c06d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f757bfff700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c1450c0 0x7f757c06d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f757bfff700 1 -- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0c73f0 msgr2=0x7f757c0c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f757bfff700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0c73f0 0x7f757c0c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f757bfff700 1 -- 172.21.15.112:0/154835957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0c6910 con 0x7f7564005210 2023-08-27T11:25:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f757b7fe700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0c73f0 0x7f757c0c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:33.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.848+0000 7f7580baf700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c1450c0 0x7f757c06d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:33.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.849+0000 7f757bfff700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564005210 0x7f757c1449c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f756c00f820 tx=0x7f756c00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:33.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.849+0000 7f7578ff9700 1 -- 172.21.15.112:0/154835957 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0052e0 con 0x7f7564005210 2023-08-27T11:25:33.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.849+0000 7f7578ff9700 1 -- 172.21.15.112:0/154835957 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c005460 con 0x7f7564005210 2023-08-27T11:25:33.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.849+0000 7f7582612700 1 -- 172.21.15.112:0/154835957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c0c6bf0 con 0x7f7564005210 2023-08-27T11:25:33.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.849+0000 7f7578ff9700 1 -- 172.21.15.112:0/154835957 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0038c0 con 0x7f7564005210 2023-08-27T11:25:33.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.849+0000 7f7582612700 1 -- 172.21.15.112:0/154835957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c0c5f90 con 0x7f7564005210 2023-08-27T11:25:33.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.850+0000 7f7582612700 1 -- 172.21.15.112:0/154835957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c067e90 con 0x7f7564005210 2023-08-27T11:25:33.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.850+0000 7f7578ff9700 1 -- 172.21.15.112:0/154835957 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f756c003a20 con 0x7f7564005210 2023-08-27T11:25:33.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.850+0000 7f7578ff9700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f756806a320 0x7f756806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:33.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.851+0000 7f7578ff9700 1 -- 172.21.15.112:0/154835957 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f756c0884e0 con 0x7f7564005210 2023-08-27T11:25:33.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.851+0000 7f757b7fe700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f756806a320 0x7f756806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:33.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.851+0000 7f757b7fe700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f756806a320 0x7f756806c7e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7570001390 tx=0x7f7570000ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:33.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:33.855+0000 7f7578ff9700 1 -- 172.21.15.112:0/154835957 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f756c053af0 con 0x7f7564005210 2023-08-27T11:25:34.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.042+0000 7f7582612700 1 -- 172.21.15.112:0/154835957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f757c0c6120 con 0x7f7564005210 2023-08-27T11:25:34.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.043+0000 7f7578ff9700 1 -- 172.21.15.112:0/154835957 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v68) v1 ==== 92+0+10 (secure 0 0 0) 0x7f756c057110 con 0x7f7564005210 2023-08-27T11:25:34.045 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:25:34.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.045+0000 7f7582612700 1 -- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f756806a320 msgr2=0x7f756806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.045+0000 7f7582612700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f756806a320 0x7f756806c7e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7570001390 tx=0x7f7570000ef0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.045+0000 7f7582612700 1 -- 172.21.15.112:0/154835957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564005210 msgr2=0x7f757c1449c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.045+0000 7f7582612700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564005210 0x7f757c1449c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f756c00f820 tx=0x7f756c00fbe0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.045+0000 7f7582612700 1 -- 172.21.15.112:0/154835957 shutdown_connections 2023-08-27T11:25:34.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.045+0000 7f7582612700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f756806a320 0x7f756806c7e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.045+0000 7f7582612700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c1450c0 0x7f757c06d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.045+0000 7f7582612700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564005210 0x7f757c1449c0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.045+0000 7f7582612700 1 --2- 172.21.15.112:0/154835957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0c73f0 0x7f757c0c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.045+0000 7f7582612700 1 -- 172.21.15.112:0/154835957 >> 172.21.15.112:0/154835957 conn(0x7f757c0b1e90 msgr2=0x7f757c15d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:34.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.045+0000 7f7582612700 1 -- 172.21.15.112:0/154835957 shutdown_connections 2023-08-27T11:25:34.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.045+0000 7f7582612700 1 -- 172.21.15.112:0/154835957 wait complete. 2023-08-27T11:25:34.059 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:25:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.180+0000 7f68e6153700 1 Processor -- start 2023-08-27T11:25:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.180+0000 7f68e6153700 1 -- start start 2023-08-27T11:25:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.181+0000 7f68e6153700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f68e006dc90 0x7f68e00cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.181+0000 7f68e6153700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f68e0143ba0 0x7f68e00afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.181+0000 7f68e6153700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e00cf410 con 0x7f68e01437c0 2023-08-27T11:25:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.181+0000 7f68e6153700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e00cfde0 con 0x7f68e0143ba0 2023-08-27T11:25:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.181+0000 7f68e6153700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e00b09e0 con 0x7f68e006dc90 2023-08-27T11:25:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.181+0000 7f68dffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f68e0143ba0 0x7f68e00afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.181+0000 7f68df7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f68e01437c0 0x7f68e006d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54128/0 (socket says 172.21.15.112:54128) 2023-08-27T11:25:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.181+0000 7f68df7fe700 1 -- 172.21.15.112:0/136879476 learned_addr learned my addr 172.21.15.112:0/136879476 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.182+0000 7f68deffd700 1 --2- 172.21.15.112:0/136879476 >> v2:172.21.15.138:3300/0 conn(0x7f68e006dc90 0x7f68e00cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.182+0000 7f68dffff700 1 -- 172.21.15.112:0/136879476 >> v2:172.21.15.138:3300/0 conn(0x7f68e006dc90 msgr2=0x7f68e00cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.182+0000 7f68dffff700 1 --2- 172.21.15.112:0/136879476 >> v2:172.21.15.138:3300/0 conn(0x7f68e006dc90 0x7f68e00cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.182+0000 7f68dffff700 1 -- 172.21.15.112:0/136879476 >> v1:172.21.15.112:6789/0 conn(0x7f68e01437c0 legacy=0x7f68e006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.182+0000 7f68dffff700 1 -- 172.21.15.112:0/136879476 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e00ceae0 con 0x7f68e0143ba0 2023-08-27T11:25:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.182+0000 7f68de7fc700 1 -- 172.21.15.112:0/136879476 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053679679 0 0) 0x7f68e00cf410 con 0x7f68e01437c0 2023-08-27T11:25:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.182+0000 7f68dffff700 1 --2- 172.21.15.112:0/136879476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e0143ba0 0x7f68e00afcb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009d90 tx=0x7f68d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bf3bfe44fe0f38 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.182+0000 7f68de7fc700 1 -- 172.21.15.112:0/136879476 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d800e070 con 0x7f68e0143ba0 2023-08-27T11:25:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.182+0000 7f68de7fc700 1 -- 172.21.15.112:0/136879476 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d800bb90 con 0x7f68e0143ba0 2023-08-27T11:25:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.182+0000 7f68de7fc700 1 -- 172.21.15.112:0/136879476 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8012430 con 0x7f68e0143ba0 2023-08-27T11:25:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.182+0000 7f68e6153700 1 -- 172.21.15.112:0/136879476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e0143ba0 msgr2=0x7f68e00afcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.182+0000 7f68e6153700 1 --2- 172.21.15.112:0/136879476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e0143ba0 0x7f68e00afcb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009d90 tx=0x7f68d800b330 comp rx=0 tx=0).stop 2023-08-27T11:25:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.183+0000 7f68e6153700 1 -- 172.21.15.112:0/136879476 shutdown_connections 2023-08-27T11:25:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.183+0000 7f68e6153700 1 --2- 172.21.15.112:0/136879476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e0143ba0 0x7f68e00afcb0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.183+0000 7f68e6153700 1 --2- 172.21.15.112:0/136879476 >> v2:172.21.15.138:3300/0 conn(0x7f68e006dc90 0x7f68e00cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.183+0000 7f68e6153700 1 -- 172.21.15.112:0/136879476 >> 172.21.15.112:0/136879476 conn(0x7f68e00b1e90 msgr2=0x7f68e00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.183+0000 7f68e6153700 1 -- 172.21.15.112:0/136879476 shutdown_connections 2023-08-27T11:25:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.183+0000 7f68e6153700 1 -- 172.21.15.112:0/136879476 wait complete. 2023-08-27T11:25:34.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.184+0000 7f68e6153700 1 Processor -- start 2023-08-27T11:25:34.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.184+0000 7f68e6153700 1 -- start start 2023-08-27T11:25:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68e6153700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68e006dc90 0x7f68e01f0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68e6153700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68e01437c0 0x7f68e0071c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68e6153700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e0143ba0 0x7f68e006fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68e6153700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e0138af0 con 0x7f68e0143ba0 2023-08-27T11:25:34.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68e6153700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e0138db0 con 0x7f68e01437c0 2023-08-27T11:25:34.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68e6153700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e0139090 con 0x7f68e006dc90 2023-08-27T11:25:34.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68dffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e0143ba0 0x7f68e006fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:34.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68dffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e0143ba0 0x7f68e006fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59756/0 (socket says 172.21.15.112:59756) 2023-08-27T11:25:34.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68dffff700 1 -- 172.21.15.112:0/3789061850 learned_addr learned my addr 172.21.15.112:0/3789061850 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:34.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68deffd700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68e01437c0 0x7f68e0071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:34.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68df7fe700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68e006dc90 0x7f68e01f0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:34.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68dffff700 1 -- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68e006dc90 msgr2=0x7f68e01f0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68dffff700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68e006dc90 0x7f68e01f0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68dffff700 1 -- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68e01437c0 msgr2=0x7f68e0071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68dffff700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68e01437c0 0x7f68e0071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68dffff700 1 -- 172.21.15.112:0/3789061850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d8009a70 con 0x7f68e0143ba0 2023-08-27T11:25:34.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68dffff700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e0143ba0 0x7f68e006fde0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f68d8014040 tx=0x7f68d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:34.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68c7fff700 1 -- 172.21.15.112:0/3789061850 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8012430 con 0x7f68e0143ba0 2023-08-27T11:25:34.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68c7fff700 1 -- 172.21.15.112:0/3789061850 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d8020070 con 0x7f68e0143ba0 2023-08-27T11:25:34.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68e6153700 1 -- 172.21.15.112:0/3789061850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e0139370 con 0x7f68e0143ba0 2023-08-27T11:25:34.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68c7fff700 1 -- 172.21.15.112:0/3789061850 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8004280 con 0x7f68e0143ba0 2023-08-27T11:25:34.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.185+0000 7f68e6153700 1 -- 172.21.15.112:0/3789061850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e00af6a0 con 0x7f68e0143ba0 2023-08-27T11:25:34.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.186+0000 7f68e6153700 1 -- 172.21.15.112:0/3789061850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e013fd50 con 0x7f68e0143ba0 2023-08-27T11:25:34.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.187+0000 7f68c7fff700 1 -- 172.21.15.112:0/3789061850 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f68d80043e0 con 0x7f68e0143ba0 2023-08-27T11:25:34.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.187+0000 7f68c7fff700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f68cc06a320 0x7f68cc06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:34.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.187+0000 7f68c7fff700 1 -- 172.21.15.112:0/3789061850 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f68d8022070 con 0x7f68e0143ba0 2023-08-27T11:25:34.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.187+0000 7f68df7fe700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f68cc06a320 0x7f68cc06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:34.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.188+0000 7f68df7fe700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f68cc06a320 0x7f68cc06c7e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f68d4007fd0 tx=0x7f68d4007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:34.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.192+0000 7f68c7fff700 1 -- 172.21.15.112:0/3789061850 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f68e013fd50 con 0x7f68e0143ba0 2023-08-27T11:25:34.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.383+0000 7f68e6153700 1 -- 172.21.15.112:0/3789061850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68e013fd50 con 0x7f68e0143ba0 2023-08-27T11:25:34.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.384+0000 7f68c7fff700 1 -- 172.21.15.112:0/3789061850 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68e013fd50 con 0x7f68e0143ba0 2023-08-27T11:25:34.387 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:25:34.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.386+0000 7f68e6153700 1 -- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f68cc06a320 msgr2=0x7f68cc06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.386+0000 7f68e6153700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f68cc06a320 0x7f68cc06c7e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f68d4007fd0 tx=0x7f68d4007de0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.386+0000 7f68e6153700 1 -- 172.21.15.112:0/3789061850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e0143ba0 msgr2=0x7f68e006fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.386+0000 7f68e6153700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e0143ba0 0x7f68e006fde0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f68d8014040 tx=0x7f68d8004ad0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.386+0000 7f68e6153700 1 -- 172.21.15.112:0/3789061850 shutdown_connections 2023-08-27T11:25:34.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.386+0000 7f68e6153700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f68cc06a320 0x7f68cc06c7e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.386+0000 7f68e6153700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e0143ba0 0x7f68e006fde0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.386+0000 7f68e6153700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68e006dc90 0x7f68e01f0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.386+0000 7f68e6153700 1 --2- 172.21.15.112:0/3789061850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68e01437c0 0x7f68e0071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.386+0000 7f68e6153700 1 -- 172.21.15.112:0/3789061850 >> 172.21.15.112:0/3789061850 conn(0x7f68e00b1e90 msgr2=0x7f68e0159940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:34.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.386+0000 7f68e6153700 1 -- 172.21.15.112:0/3789061850 shutdown_connections 2023-08-27T11:25:34.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.386+0000 7f68e6153700 1 -- 172.21.15.112:0/3789061850 wait complete. 2023-08-27T11:25:34.399 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:25:34.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.521+0000 7fa471c71700 1 Processor -- start 2023-08-27T11:25:34.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.521+0000 7fa471c71700 1 -- start start 2023-08-27T11:25:34.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa471c71700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa46c0aee70 0x7fa46c0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:34.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa471c71700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa46c15d9b0 0x7fa46c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:34.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa471c71700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa46c0af7f0 con 0x7fa46c15af20 2023-08-27T11:25:34.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa471c71700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa46c06bce0 con 0x7fa46c15d9b0 2023-08-27T11:25:34.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa471c71700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c161190 con 0x7fa46c0aee70 2023-08-27T11:25:34.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa46bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa46c15d9b0 0x7fa46c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:34.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa46b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa46c15af20 0x7fa46c06b1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54142/0 (socket says 172.21.15.112:54142) 2023-08-27T11:25:34.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa46b7fe700 1 -- 172.21.15.112:0/3567501849 learned_addr learned my addr 172.21.15.112:0/3567501849 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:34.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa46bfff700 1 -- 172.21.15.112:0/3567501849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa46c0aee70 msgr2=0x7fa46c0af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:34.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa46bfff700 1 --2- 172.21.15.112:0/3567501849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa46c0aee70 0x7fa46c0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa46bfff700 1 -- 172.21.15.112:0/3567501849 >> v1:172.21.15.112:6789/0 conn(0x7fa46c15af20 legacy=0x7fa46c06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa46bfff700 1 -- 172.21.15.112:0/3567501849 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c161470 con 0x7fa46c15d9b0 2023-08-27T11:25:34.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa46a7fc700 1 -- 172.21.15.112:0/3567501849 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324914207 0 0) 0x7fa46c0af7f0 con 0x7fa46c15af20 2023-08-27T11:25:34.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.522+0000 7fa46bfff700 1 --2- 172.21.15.112:0/3567501849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa46c15d9b0 0x7fa46c15de40 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa464009d90 tx=0x7fa46400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c27767cd521aa1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:34.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.523+0000 7fa46a7fc700 1 -- 172.21.15.112:0/3567501849 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46400e070 con 0x7fa46c15d9b0 2023-08-27T11:25:34.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.523+0000 7fa46a7fc700 1 -- 172.21.15.112:0/3567501849 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46400bbd0 con 0x7fa46c15d9b0 2023-08-27T11:25:34.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.523+0000 7fa46a7fc700 1 -- 172.21.15.112:0/3567501849 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464012430 con 0x7fa46c15d9b0 2023-08-27T11:25:34.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.523+0000 7fa471c71700 1 -- 172.21.15.112:0/3567501849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa46c15d9b0 msgr2=0x7fa46c15de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.523+0000 7fa471c71700 1 --2- 172.21.15.112:0/3567501849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa46c15d9b0 0x7fa46c15de40 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa464009d90 tx=0x7fa46400b330 comp rx=0 tx=0).stop 2023-08-27T11:25:34.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.523+0000 7fa471c71700 1 -- 172.21.15.112:0/3567501849 shutdown_connections 2023-08-27T11:25:34.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.523+0000 7fa471c71700 1 --2- 172.21.15.112:0/3567501849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa46c15d9b0 0x7fa46c15de40 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.523+0000 7fa471c71700 1 --2- 172.21.15.112:0/3567501849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa46c0aee70 0x7fa46c0af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.523+0000 7fa471c71700 1 -- 172.21.15.112:0/3567501849 >> 172.21.15.112:0/3567501849 conn(0x7fa46c0d03e0 msgr2=0x7fa46c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:34.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.523+0000 7fa471c71700 1 -- 172.21.15.112:0/3567501849 shutdown_connections 2023-08-27T11:25:34.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.523+0000 7fa471c71700 1 -- 172.21.15.112:0/3567501849 wait complete. 2023-08-27T11:25:34.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.524+0000 7fa471c71700 1 Processor -- start 2023-08-27T11:25:34.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.525+0000 7fa471c71700 1 -- start start 2023-08-27T11:25:34.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.525+0000 7fa471c71700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa46c0aee70 0x7fa46c14c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:34.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.525+0000 7fa471c71700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa46c15af20 0x7fa46c0ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:34.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.525+0000 7fa471c71700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa46c15d9b0 0x7fa46c15cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:34.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.525+0000 7fa471c71700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c15d010 con 0x7fa46c0aee70 2023-08-27T11:25:34.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.525+0000 7fa471c71700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c1535d0 con 0x7fa46c15af20 2023-08-27T11:25:34.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.525+0000 7fa471c71700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c15d240 con 0x7fa46c15d9b0 2023-08-27T11:25:34.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.525+0000 7fa46b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa46c0aee70 0x7fa46c14c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:34.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.525+0000 7fa46b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa46c0aee70 0x7fa46c14c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59766/0 (socket says 172.21.15.112:59766) 2023-08-27T11:25:34.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.525+0000 7fa46b7fe700 1 -- 172.21.15.112:0/3056871505 learned_addr learned my addr 172.21.15.112:0/3056871505 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:34.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa46affd700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa46c15af20 0x7fa46c0ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:34.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa46bfff700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa46c15d9b0 0x7fa46c15cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:34.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa46b7fe700 1 -- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa46c15d9b0 msgr2=0x7fa46c15cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa46b7fe700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa46c15d9b0 0x7fa46c15cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa46b7fe700 1 -- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa46c15af20 msgr2=0x7fa46c0ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa46b7fe700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa46c15af20 0x7fa46c0ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa46b7fe700 1 -- 172.21.15.112:0/3056871505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa464009a70 con 0x7fa46c0aee70 2023-08-27T11:25:34.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa46affd700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa46c15af20 0x7fa46c0ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:34.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa46bfff700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa46c15d9b0 0x7fa46c15cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:34.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa46b7fe700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa46c0aee70 0x7fa46c14c3a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa4600011e0 tx=0x7fa46000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:34.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa453fff700 1 -- 172.21.15.112:0/3056871505 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460004c80 con 0x7fa46c0aee70 2023-08-27T11:25:34.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa453fff700 1 -- 172.21.15.112:0/3056871505 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa460004e00 con 0x7fa46c0aee70 2023-08-27T11:25:34.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa453fff700 1 -- 172.21.15.112:0/3056871505 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460007310 con 0x7fa46c0aee70 2023-08-27T11:25:34.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa471c71700 1 -- 172.21.15.112:0/3056871505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46c153890 con 0x7fa46c0aee70 2023-08-27T11:25:34.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.526+0000 7fa471c71700 1 -- 172.21.15.112:0/3056871505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46c15d520 con 0x7fa46c0aee70 2023-08-27T11:25:34.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.527+0000 7fa471c71700 1 -- 172.21.15.112:0/3056871505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46c06da70 con 0x7fa46c0aee70 2023-08-27T11:25:34.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.527+0000 7fa453fff700 1 -- 172.21.15.112:0/3056871505 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fa460007470 con 0x7fa46c0aee70 2023-08-27T11:25:34.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.528+0000 7fa453fff700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa45806a320 0x7fa45806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:34.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.528+0000 7fa453fff700 1 -- 172.21.15.112:0/3056871505 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa460088400 con 0x7fa46c0aee70 2023-08-27T11:25:34.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.528+0000 7fa46affd700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa45806a320 0x7fa45806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:34.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.528+0000 7fa46affd700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa45806a320 0x7fa45806c7e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa46c1f5dc0 tx=0x7fa45c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:34.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.532+0000 7fa453fff700 1 -- 172.21.15.112:0/3056871505 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa460053a90 con 0x7fa46c0aee70 2023-08-27T11:25:34.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.721+0000 7fa471c71700 1 -- 172.21.15.112:0/3056871505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa46c153db0 con 0x7fa46c0aee70 2023-08-27T11:25:34.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.725+0000 7fa453fff700 1 -- 172.21.15.112:0/3056871505 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa46c153db0 con 0x7fa46c0aee70 2023-08-27T11:25:34.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.727+0000 7fa471c71700 1 -- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa45806a320 msgr2=0x7fa45806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.727+0000 7fa471c71700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa45806a320 0x7fa45806c7e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa46c1f5dc0 tx=0x7fa45c008040 comp rx=0 tx=0).stop 2023-08-27T11:25:34.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.727+0000 7fa471c71700 1 -- 172.21.15.112:0/3056871505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa46c0aee70 msgr2=0x7fa46c14c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:34.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.727+0000 7fa471c71700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa46c0aee70 0x7fa46c14c3a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa4600011e0 tx=0x7fa46000cae0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.727+0000 7fa471c71700 1 -- 172.21.15.112:0/3056871505 shutdown_connections 2023-08-27T11:25:34.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.727+0000 7fa471c71700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa45806a320 0x7fa45806c7e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.727+0000 7fa471c71700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa46c15d9b0 0x7fa46c15cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.727+0000 7fa471c71700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa46c0aee70 0x7fa46c14c3a0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.727+0000 7fa471c71700 1 --2- 172.21.15.112:0/3056871505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa46c15af20 0x7fa46c0ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:34.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.727+0000 7fa471c71700 1 -- 172.21.15.112:0/3056871505 >> 172.21.15.112:0/3056871505 conn(0x7fa46c0d03e0 msgr2=0x7fa46c0b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:34.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.727+0000 7fa471c71700 1 -- 172.21.15.112:0/3056871505 shutdown_connections 2023-08-27T11:25:34.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:34.727+0000 7fa471c71700 1 -- 172.21.15.112:0/3056871505 wait complete. 2023-08-27T11:25:34.740 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:25:34.741 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:25:34.782 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:25:34.825 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:25:34.825 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:34.826 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:34.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:25:34.844 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:25:34.844 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:34.886 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:25:34.887 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:34.887 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:34.904 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:25:34.905 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:25:34.905 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:34.945 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:25:34.946 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:25:34.946 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:25:35.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.064+0000 7fc849e52700 1 Processor -- start 2023-08-27T11:25:35.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.064+0000 7fc849e52700 1 -- start start 2023-08-27T11:25:35.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.065+0000 7fc849e52700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8440b3e50 0x7fc84415f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:35.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.065+0000 7fc849e52700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8440b45f0 con 0x7fc844144570 2023-08-27T11:25:35.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.065+0000 7fc849e52700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8440b4310 con 0x7fc8440b4ad0 2023-08-27T11:25:35.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.065+0000 7fc849e52700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8440cf7a0 con 0x7fc8440b3e50 2023-08-27T11:25:35.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.065+0000 7fc8437fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc844144570 0x7fc84415ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54156/0 (socket says 172.21.15.112:54156) 2023-08-27T11:25:35.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.065+0000 7fc8437fe700 1 -- 172.21.15.112:0/3719266045 learned_addr learned my addr 172.21.15.112:0/3719266045 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:35.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.065+0000 7fc842ffd700 1 --2- 172.21.15.112:0/3719266045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8440b3e50 0x7fc84415f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:35.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151350906 0 0) 0x7fc8440b45f0 con 0x7fc844144570 2023-08-27T11:25:35.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc82c0035b0 con 0x7fc844144570 2023-08-27T11:25:35.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1785550182 0 0) 0x7fc82c0035b0 con 0x7fc844144570 2023-08-27T11:25:35.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8440b3e50 msgr2=0x7fc84415f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:35.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 --2- 172.21.15.112:0/3719266045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8440b3e50 0x7fc84415f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:35.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 >> v1:172.21.15.138:6789/0 conn(0x7fc8440b4ad0 legacy=0x7fc8440b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:35.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8440cdcb0 con 0x7fc844144570 2023-08-27T11:25:35.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fc838001670 con 0x7fc844144570 2023-08-27T11:25:35.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 >> v1:172.21.15.112:6789/0 conn(0x7fc844144570 legacy=0x7fc84415ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:35.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 --2- 172.21.15.112:0/3719266045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8440b3e50 0x7fc82c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:35.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 --2- 172.21.15.112:0/3719266045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82c003f70 0x7fc82c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:35.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 --2- 172.21.15.112:0/3719266045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc82c004d90 0x7fc82c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:35.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8440cf7a0 con 0x7fc82c003f70 2023-08-27T11:25:35.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c00bb20 con 0x7fc82c004d90 2023-08-27T11:25:35.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c00bce0 con 0x7fc8440b3e50 2023-08-27T11:25:35.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc842ffd700 1 --2- 172.21.15.112:0/3719266045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82c003f70 0x7fc82c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:35.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc8437fe700 1 --2- 172.21.15.112:0/3719266045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8440b3e50 0x7fc82c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:35.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc843fff700 1 --2- 172.21.15.112:0/3719266045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc82c004d90 0x7fc82c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:35.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc842ffd700 1 -- 172.21.15.112:0/3719266045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8440b3e50 msgr2=0x7fc82c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:35.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc842ffd700 1 --2- 172.21.15.112:0/3719266045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8440b3e50 0x7fc82c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:35.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc842ffd700 1 -- 172.21.15.112:0/3719266045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc82c004d90 msgr2=0x7fc82c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:35.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc842ffd700 1 --2- 172.21.15.112:0/3719266045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc82c004d90 0x7fc82c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:35.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc842ffd700 1 -- 172.21.15.112:0/3719266045 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82c0035b0 con 0x7fc82c003f70 2023-08-27T11:25:35.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.066+0000 7fc842ffd700 1 --2- 172.21.15.112:0/3719266045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82c003f70 0x7fc82c004850 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc834009b60 tx=0x7fc82c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1590fbf6ecd8922c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:35.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.067+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83400e6c0 con 0x7fc82c003f70 2023-08-27T11:25:35.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.067+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83400e820 con 0x7fc82c003f70 2023-08-27T11:25:35.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.067+0000 7fc8427fc700 1 -- 172.21.15.112:0/3719266045 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83400eb90 con 0x7fc82c003f70 2023-08-27T11:25:35.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.067+0000 7fc849e52700 1 -- 172.21.15.112:0/3719266045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82c003f70 msgr2=0x7fc82c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:35.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.067+0000 7fc849e52700 1 --2- 172.21.15.112:0/3719266045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82c003f70 0x7fc82c004850 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc834009b60 tx=0x7fc82c005880 comp rx=0 tx=0).stop 2023-08-27T11:25:35.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.067+0000 7fc843fff700 1 -- 172.21.15.112:0/3719266045 reap_dead start 2023-08-27T11:25:35.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.067+0000 7fc849e52700 1 -- 172.21.15.112:0/3719266045 shutdown_connections 2023-08-27T11:25:35.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.067+0000 7fc849e52700 1 -- 172.21.15.112:0/3719266045 >> 172.21.15.112:0/3719266045 conn(0x7fc84406f970 msgr2=0x7fc84406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:35.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.067+0000 7fc849e52700 1 -- 172.21.15.112:0/3719266045 shutdown_connections 2023-08-27T11:25:35.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.067+0000 7fc849e52700 1 -- 172.21.15.112:0/3719266045 wait complete. 2023-08-27T11:25:35.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.068+0000 7fc849e52700 1 Processor -- start 2023-08-27T11:25:35.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.069+0000 7fc849e52700 1 -- start start 2023-08-27T11:25:35.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.069+0000 7fc849e52700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82c003f70 0x7fc8440cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:35.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.069+0000 7fc849e52700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440710b0 0x7fc8441f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:35.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.069+0000 7fc849e52700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8440cb930 0x7fc84415a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:35.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.069+0000 7fc849e52700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8441f3790 con 0x7fc8440710b0 2023-08-27T11:25:35.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.069+0000 7fc849e52700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8441e5970 con 0x7fc8440cb930 2023-08-27T11:25:35.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.069+0000 7fc849e52700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8441e4b10 con 0x7fc82c003f70 2023-08-27T11:25:35.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.069+0000 7fc842ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440710b0 0x7fc8441f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:35.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.069+0000 7fc842ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440710b0 0x7fc8441f3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59788/0 (socket says 172.21.15.112:59788) 2023-08-27T11:25:35.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.069+0000 7fc842ffd700 1 -- 172.21.15.112:0/1610744393 learned_addr learned my addr 172.21.15.112:0/1610744393 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:35.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.069+0000 7fc843fff700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8440cb930 0x7fc84415a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:35.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.069+0000 7fc8437fe700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82c003f70 0x7fc8440cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:35.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.070+0000 7fc842ffd700 1 -- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82c003f70 msgr2=0x7fc8440cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:35.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.070+0000 7fc842ffd700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82c003f70 0x7fc8440cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:35.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.070+0000 7fc842ffd700 1 -- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8440cb930 msgr2=0x7fc84415a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:35.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.070+0000 7fc842ffd700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8440cb930 0x7fc84415a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:35.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.070+0000 7fc842ffd700 1 -- 172.21.15.112:0/1610744393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8441e4df0 con 0x7fc8440710b0 2023-08-27T11:25:35.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.070+0000 7fc843fff700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8440cb930 0x7fc84415a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:35.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.070+0000 7fc842ffd700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440710b0 0x7fc8441f3250 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00baf0 tx=0x7fc8340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:35.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.070+0000 7fc82bfff700 1 -- 172.21.15.112:0/1610744393 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83400e6c0 con 0x7fc8440710b0 2023-08-27T11:25:35.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.070+0000 7fc82bfff700 1 -- 172.21.15.112:0/1610744393 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc834003ad0 con 0x7fc8440710b0 2023-08-27T11:25:35.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.070+0000 7fc849e52700 1 -- 172.21.15.112:0/1610744393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8441e50d0 con 0x7fc8440710b0 2023-08-27T11:25:35.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.070+0000 7fc82bfff700 1 -- 172.21.15.112:0/1610744393 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834003e80 con 0x7fc8440710b0 2023-08-27T11:25:35.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.070+0000 7fc849e52700 1 -- 172.21.15.112:0/1610744393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8441e5690 con 0x7fc8440710b0 2023-08-27T11:25:35.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.071+0000 7fc849e52700 1 -- 172.21.15.112:0/1610744393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8440b2f40 con 0x7fc8440710b0 2023-08-27T11:25:35.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.071+0000 7fc82bfff700 1 -- 172.21.15.112:0/1610744393 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fc834013470 con 0x7fc8440710b0 2023-08-27T11:25:35.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.072+0000 7fc82bfff700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc83006a320 0x7fc83006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:35.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.072+0000 7fc82bfff700 1 -- 172.21.15.112:0/1610744393 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc83408a080 con 0x7fc8440710b0 2023-08-27T11:25:35.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.072+0000 7fc8437fe700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc83006a320 0x7fc83006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:35.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.072+0000 7fc8437fe700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc83006a320 0x7fc83006c7e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc8440cf260 tx=0x7fc838009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:35.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.076+0000 7fc82bfff700 1 -- 172.21.15.112:0/1610744393 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc834055850 con 0x7fc8440710b0 2023-08-27T11:25:35.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.307+0000 7fc849e52700 1 -- 172.21.15.112:0/1610744393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc844067a20 con 0x7fc8440710b0 2023-08-27T11:25:35.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.307+0000 7fc82bfff700 1 -- 172.21.15.112:0/1610744393 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+3711 (secure 0 0 0) 0x7fc834058e70 con 0x7fc8440710b0 2023-08-27T11:25:35.309 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:35.309 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5609,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1928177352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1928177352},{"type":"v1","addr":"172.21.15.112:6835","nonce":1928177352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2906075444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2906075444},{"type":"v1","addr":"172.21.15.138:6836","nonce":2906075444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5761,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1781004436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1781004436},{"type":"v1","addr":"172.21.15.138:6837","nonce":1781004436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1573349448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1573349448},{"type":"v1","addr":"172.21.15.112:6834","nonce":1573349448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[]} 2023-08-27T11:25:35.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.309+0000 7fc849e52700 1 -- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc83006a320 msgr2=0x7fc83006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:35.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.309+0000 7fc849e52700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc83006a320 0x7fc83006c7e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc8440cf260 tx=0x7fc838009040 comp rx=0 tx=0).stop 2023-08-27T11:25:35.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.309+0000 7fc849e52700 1 -- 172.21.15.112:0/1610744393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440710b0 msgr2=0x7fc8441f3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:35.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.309+0000 7fc849e52700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440710b0 0x7fc8441f3250 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00baf0 tx=0x7fc8340047c0 comp rx=0 tx=0).stop 2023-08-27T11:25:35.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.309+0000 7fc849e52700 1 -- 172.21.15.112:0/1610744393 shutdown_connections 2023-08-27T11:25:35.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.309+0000 7fc849e52700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc83006a320 0x7fc83006c7e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:35.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.309+0000 7fc849e52700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8440cb930 0x7fc84415a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:35.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.309+0000 7fc849e52700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82c003f70 0x7fc8440cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:35.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.309+0000 7fc849e52700 1 --2- 172.21.15.112:0/1610744393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440710b0 0x7fc8441f3250 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:35.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.309+0000 7fc849e52700 1 -- 172.21.15.112:0/1610744393 >> 172.21.15.112:0/1610744393 conn(0x7fc84406f970 msgr2=0x7fc8441427f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:35.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.309+0000 7fc849e52700 1 -- 172.21.15.112:0/1610744393 shutdown_connections 2023-08-27T11:25:35.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.309+0000 7fc849e52700 1 -- 172.21.15.112:0/1610744393 wait complete. 2023-08-27T11:25:35.313 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 42 2023-08-27T11:25:35.323 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:25:35.323 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:25:35.324 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:25:35.324 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:25:35.324 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:25:35.325 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:25:35.325 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:25:35.325 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:25:35.325 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:25:35.326 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:25:35.326 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:25:35.326 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:25:35.327 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:25:35.327 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:25:35.327 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:25:35.327 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:25:35.333 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:25:35.334 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:25:35.335 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:25:35.335 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:25:35.337 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:25:35.337 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:25:35.338 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:25:35.338 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:25:35.377 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:25:35.378 DEBUG:teuthology.parallel:result is None 2023-08-27T11:25:35.378 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:25:35.379 DEBUG:teuthology.parallel:result is None 2023-08-27T11:25:35.381 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:25:35.381 DEBUG:teuthology.parallel:result is None 2023-08-27T11:25:35.382 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:25:35.382 DEBUG:teuthology.parallel:result is None 2023-08-27T11:25:35.383 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:25:35.455 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:25:35.456 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:25:35.456 INFO:tasks.ceph.mds.c.smithi112.stderr:2023-08-27T11:25:35.454+0000 7face5431700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-08-27T11:25:35.466 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2023-08-27T11:25:35.467 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:25:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.509+0000 7f39ce3ae700 1 Processor -- start 2023-08-27T11:25:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.509+0000 7f39ce3ae700 1 -- start start 2023-08-27T11:25:35.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.509+0000 7f39ce3ae700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c80b11f0 0x7f39c806e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:35.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.509+0000 7f39ce3ae700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c815c190 con 0x7f39c806fea0 2023-08-27T11:25:35.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.509+0000 7f39ce3ae700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c8159200 con 0x7f39c8158e20 2023-08-27T11:25:35.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.509+0000 7f39ce3ae700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c806e8e0 con 0x7f39c80b11f0 2023-08-27T11:25:35.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39c77fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f39c806fea0 0x7f39c80af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54202/0 (socket says 172.21.15.112:54202) 2023-08-27T11:25:35.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39c77fe700 1 -- 172.21.15.112:0/1248487816 learned_addr learned my addr 172.21.15.112:0/1248487816 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:35.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39cc94b700 1 --2- 172.21.15.112:0/1248487816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c80b11f0 0x7f39c806e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:35.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39c6ffd700 1 -- 172.21.15.112:0/1248487816 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870385151 0 0) 0x7f39c815c190 con 0x7f39c806fea0 2023-08-27T11:25:35.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39c6ffd700 1 -- 172.21.15.112:0/1248487816 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39b00035b0 con 0x7f39c806fea0 2023-08-27T11:25:35.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39c6ffd700 1 -- 172.21.15.112:0/1248487816 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340663142 0 0) 0x7f39c8159200 con 0x7f39c8158e20 2023-08-27T11:25:35.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39c6ffd700 1 -- 172.21.15.112:0/1248487816 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39c815c190 con 0x7f39c8158e20 2023-08-27T11:25:35.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39cc94b700 1 -- 172.21.15.112:0/1248487816 >> v1:172.21.15.138:6789/0 conn(0x7f39c8158e20 legacy=0x7f39c806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:35.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39cc94b700 1 -- 172.21.15.112:0/1248487816 >> v1:172.21.15.112:6789/0 conn(0x7f39c806fea0 legacy=0x7f39c80af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:35.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39cc94b700 1 -- 172.21.15.112:0/1248487816 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c80b1640 con 0x7f39c80b11f0 2023-08-27T11:25:35.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39cc94b700 1 --2- 172.21.15.112:0/1248487816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c80b11f0 0x7f39c806e3a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009dc0 tx=0x7f39c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de7e0014ca7e543 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:35.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39c6ffd700 1 -- 172.21.15.112:0/1248487816 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c000e070 con 0x7f39c80b11f0 2023-08-27T11:25:35.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39c6ffd700 1 -- 172.21.15.112:0/1248487816 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c000bb20 con 0x7f39c80b11f0 2023-08-27T11:25:35.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.510+0000 7f39c6ffd700 1 -- 172.21.15.112:0/1248487816 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c000be90 con 0x7f39c80b11f0 2023-08-27T11:25:35.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.511+0000 7f39ce3ae700 1 -- 172.21.15.112:0/1248487816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c80b11f0 msgr2=0x7f39c806e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:35.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.511+0000 7f39ce3ae700 1 --2- 172.21.15.112:0/1248487816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c80b11f0 0x7f39c806e3a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009dc0 tx=0x7f39c000b330 comp rx=0 tx=0).stop 2023-08-27T11:25:35.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.511+0000 7f39ce3ae700 1 -- 172.21.15.112:0/1248487816 shutdown_connections 2023-08-27T11:25:35.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.511+0000 7f39ce3ae700 1 --2- 172.21.15.112:0/1248487816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c80b11f0 0x7f39c806e3a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:35.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.511+0000 7f39ce3ae700 1 -- 172.21.15.112:0/1248487816 >> 172.21.15.112:0/1248487816 conn(0x7f39c80d0000 msgr2=0x7f39c80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:35.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.511+0000 7f39ce3ae700 1 -- 172.21.15.112:0/1248487816 shutdown_connections 2023-08-27T11:25:35.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.511+0000 7f39ce3ae700 1 -- 172.21.15.112:0/1248487816 wait complete. 2023-08-27T11:25:35.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.512+0000 7f39ce3ae700 1 Processor -- start 2023-08-27T11:25:35.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.512+0000 7f39ce3ae700 1 -- start start 2023-08-27T11:25:35.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39ce3ae700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c806fea0 0x7f39c815cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:35.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39ce3ae700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c80b11f0 0x7f39c815e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:35.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39ce3ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39c8158e20 0x7f39c814f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:35.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39ce3ae700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c815e960 con 0x7f39c8158e20 2023-08-27T11:25:35.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39ce3ae700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80c4ef0 con 0x7f39c80b11f0 2023-08-27T11:25:35.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39ce3ae700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c814fcf0 con 0x7f39c806fea0 2023-08-27T11:25:35.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39cc94b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39c8158e20 0x7f39c814f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:35.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39cc94b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39c8158e20 0x7f39c814f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59828/0 (socket says 172.21.15.112:59828) 2023-08-27T11:25:35.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39cc94b700 1 -- 172.21.15.112:0/2373264399 learned_addr learned my addr 172.21.15.112:0/2373264399 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:35.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39c7fff700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c806fea0 0x7f39c815cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:35.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39c77fe700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c80b11f0 0x7f39c815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:35.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39cc94b700 1 -- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c806fea0 msgr2=0x7f39c815cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:35.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39cc94b700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c806fea0 0x7f39c815cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:35.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39cc94b700 1 -- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c80b11f0 msgr2=0x7f39c815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:35.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39cc94b700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c80b11f0 0x7f39c815e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:35.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39cc94b700 1 -- 172.21.15.112:0/2373264399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c0009a70 con 0x7f39c8158e20 2023-08-27T11:25:35.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.513+0000 7f39cc94b700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39c8158e20 0x7f39c814f7b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f39c0014040 tx=0x7f39c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:35.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.514+0000 7f39c4ff9700 1 -- 172.21.15.112:0/2373264399 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c00048c0 con 0x7f39c8158e20 2023-08-27T11:25:35.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.514+0000 7f39c4ff9700 1 -- 172.21.15.112:0/2373264399 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c0020070 con 0x7f39c8158e20 2023-08-27T11:25:35.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.514+0000 7f39ce3ae700 1 -- 172.21.15.112:0/2373264399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c80c81d0 con 0x7f39c8158e20 2023-08-27T11:25:35.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.514+0000 7f39c4ff9700 1 -- 172.21.15.112:0/2373264399 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0027680 con 0x7f39c8158e20 2023-08-27T11:25:35.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.514+0000 7f39ce3ae700 1 -- 172.21.15.112:0/2373264399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c815d880 con 0x7f39c8158e20 2023-08-27T11:25:35.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.515+0000 7f39ce3ae700 1 -- 172.21.15.112:0/2373264399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c80c8420 con 0x7f39c8158e20 2023-08-27T11:25:35.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.515+0000 7f39c4ff9700 1 -- 172.21.15.112:0/2373264399 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f39c0012430 con 0x7f39c8158e20 2023-08-27T11:25:35.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.515+0000 7f39c4ff9700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39b406a320 0x7f39b406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:35.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.515+0000 7f39c4ff9700 1 -- 172.21.15.112:0/2373264399 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f39c0022070 con 0x7f39c8158e20 2023-08-27T11:25:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.516+0000 7f39c7fff700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39b406a320 0x7f39b406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.516+0000 7f39c7fff700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39b406a320 0x7f39b406c7e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f39c806def0 tx=0x7f39b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.520+0000 7f39c4ff9700 1 -- 172.21.15.112:0/2373264399 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f39c80c8420 con 0x7f39c8158e20 2023-08-27T11:25:35.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:35.725+0000 7f39ce3ae700 1 -- 172.21.15.112:0/2373264399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f39c80c8420 con 0x7f39c8158e20 2023-08-27T11:25:36.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:25:36.849 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:25:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.326+0000 7f39c4ff9700 1 -- 172.21.15.112:0/2373264399 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v73) v1 ==== 97+0+0 (secure 0 0 0) 0x7f39c80c8420 con 0x7f39c8158e20 2023-08-27T11:25:37.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.328+0000 7f39ce3ae700 1 -- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39b406a320 msgr2=0x7f39b406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:37.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.328+0000 7f39ce3ae700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39b406a320 0x7f39b406c7e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f39c806def0 tx=0x7f39b8009040 comp rx=0 tx=0).stop 2023-08-27T11:25:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.328+0000 7f39ce3ae700 1 -- 172.21.15.112:0/2373264399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39c8158e20 msgr2=0x7f39c814f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.328+0000 7f39ce3ae700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39c8158e20 0x7f39c814f7b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f39c0014040 tx=0x7f39c0004ad0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.328+0000 7f39ce3ae700 1 -- 172.21.15.112:0/2373264399 shutdown_connections 2023-08-27T11:25:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.328+0000 7f39ce3ae700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39b406a320 0x7f39b406c7e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.329+0000 7f39ce3ae700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39c8158e20 0x7f39c814f7b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.329+0000 7f39ce3ae700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c806fea0 0x7f39c815cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.329+0000 7f39ce3ae700 1 --2- 172.21.15.112:0/2373264399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c80b11f0 0x7f39c815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.329+0000 7f39ce3ae700 1 -- 172.21.15.112:0/2373264399 >> 172.21.15.112:0/2373264399 conn(0x7f39c80d0000 msgr2=0x7f39c806e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:37.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.329+0000 7f39ce3ae700 1 -- 172.21.15.112:0/2373264399 shutdown_connections 2023-08-27T11:25:37.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.329+0000 7f39ce3ae700 1 -- 172.21.15.112:0/2373264399 wait complete. 2023-08-27T11:25:37.333 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2023-08-27T11:25:37.346 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:25:37.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.472+0000 7f13e539e700 1 Processor -- start 2023-08-27T11:25:37.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.472+0000 7f13e539e700 1 -- start start 2023-08-27T11:25:37.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.472+0000 7f13e539e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e013f910 0x7f13e00cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:37.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.472+0000 7f13e539e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13e00ce090 0x7f13e006fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:37.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.472+0000 7f13e539e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e006cb00 con 0x7f13e006ff70 2023-08-27T11:25:37.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.472+0000 7f13e539e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e00707a0 con 0x7f13e00ce090 2023-08-27T11:25:37.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.472+0000 7f13e539e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e0070a80 con 0x7f13e013f910 2023-08-27T11:25:37.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.473+0000 7f13deffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e013f910 0x7f13e00cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:37.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.473+0000 7f13de7fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13e00ce090 0x7f13e006fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:37.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.473+0000 7f13deffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e013f910 0x7f13e00cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:43258/0 (socket says 172.21.15.112:43258) 2023-08-27T11:25:37.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.473+0000 7f13df7fe700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f13e006ff70 0x7f13e015d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:33512/0 (socket says 172.21.15.112:33512) 2023-08-27T11:25:37.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.473+0000 7f13de7fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13e00ce090 0x7f13e006fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:60266/0 (socket says 172.21.15.112:60266) 2023-08-27T11:25:37.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.473+0000 7f13deffd700 1 -- 172.21.15.112:0/1608191862 learned_addr learned my addr 172.21.15.112:0/1608191862 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.473+0000 7f13deffd700 1 -- 172.21.15.112:0/1608191862 >> v2:172.21.15.138:3300/0 conn(0x7f13e00ce090 msgr2=0x7f13e006fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.473+0000 7f13deffd700 1 --2- 172.21.15.112:0/1608191862 >> v2:172.21.15.138:3300/0 conn(0x7f13e00ce090 0x7f13e006fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.473+0000 7f13deffd700 1 -- 172.21.15.112:0/1608191862 >> v1:172.21.15.138:6789/0 conn(0x7f13e006ff70 legacy=0x7f13e015d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.473+0000 7f13deffd700 1 -- 172.21.15.112:0/1608191862 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e0070d60 con 0x7f13e013f910 2023-08-27T11:25:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.473+0000 7f13de7fc700 1 --2- 172.21.15.112:0/1608191862 >> v2:172.21.15.138:3300/0 conn(0x7f13e00ce090 0x7f13e006fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.473+0000 7f13deffd700 1 --2- 172.21.15.112:0/1608191862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e013f910 0x7f13e00cdb50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f13d000bb90 tx=0x7f13d000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc820a928903dafe server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.474+0000 7f13ddffb700 1 -- 172.21.15.112:0/1608191862 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d000dd30 con 0x7f13e013f910 2023-08-27T11:25:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.474+0000 7f13ddffb700 1 -- 172.21.15.112:0/1608191862 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d000de90 con 0x7f13e013f910 2023-08-27T11:25:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.474+0000 7f13ddffb700 1 -- 172.21.15.112:0/1608191862 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0010690 con 0x7f13e013f910 2023-08-27T11:25:37.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.474+0000 7f13e539e700 1 -- 172.21.15.112:0/1608191862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e013f910 msgr2=0x7f13e00cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:37.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.474+0000 7f13e539e700 1 --2- 172.21.15.112:0/1608191862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e013f910 0x7f13e00cdb50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f13d000bb90 tx=0x7f13d000bea0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.474+0000 7f13e539e700 1 -- 172.21.15.112:0/1608191862 shutdown_connections 2023-08-27T11:25:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.474+0000 7f13e539e700 1 --2- 172.21.15.112:0/1608191862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e013f910 0x7f13e00cdb50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.474+0000 7f13e539e700 1 --2- 172.21.15.112:0/1608191862 >> v2:172.21.15.138:3300/0 conn(0x7f13e00ce090 0x7f13e006fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.474+0000 7f13e539e700 1 -- 172.21.15.112:0/1608191862 >> 172.21.15.112:0/1608191862 conn(0x7f13e00b0bf0 msgr2=0x7f13e00b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:37.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.474+0000 7f13e539e700 1 -- 172.21.15.112:0/1608191862 shutdown_connections 2023-08-27T11:25:37.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.474+0000 7f13e539e700 1 -- 172.21.15.112:0/1608191862 wait complete. 2023-08-27T11:25:37.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.475+0000 7f13e539e700 1 Processor -- start 2023-08-27T11:25:37.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.476+0000 7f13e539e700 1 -- start start 2023-08-27T11:25:37.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.476+0000 7f13e539e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e006ff70 0x7f13e01f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:37.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.476+0000 7f13e539e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e00ce090 0x7f13e01e2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:37.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.476+0000 7f13e539e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e013f910 0x7f13e01e3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:37.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.476+0000 7f13e539e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e01f5f90 con 0x7f13e013f910 2023-08-27T11:25:37.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.476+0000 7f13e539e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e013c710 con 0x7f13e00ce090 2023-08-27T11:25:37.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.476+0000 7f13e539e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e01f61e0 con 0x7f13e006ff70 2023-08-27T11:25:37.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.476+0000 7f13df7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e013f910 0x7f13e01e3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:37.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.476+0000 7f13df7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e013f910 0x7f13e01e3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59846/0 (socket says 172.21.15.112:59846) 2023-08-27T11:25:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.476+0000 7f13df7fe700 1 -- 172.21.15.112:0/1082994219 learned_addr learned my addr 172.21.15.112:0/1082994219 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.476+0000 7f13deffd700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e006ff70 0x7f13e01f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.477+0000 7f13de7fc700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e00ce090 0x7f13e01e2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.477+0000 7f13df7fe700 1 -- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e006ff70 msgr2=0x7f13e01f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.477+0000 7f13df7fe700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e006ff70 0x7f13e01f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.477+0000 7f13df7fe700 1 -- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e00ce090 msgr2=0x7f13e01e2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.477+0000 7f13df7fe700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e00ce090 0x7f13e01e2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.477+0000 7f13df7fe700 1 -- 172.21.15.112:0/1082994219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c80094a0 con 0x7f13e013f910 2023-08-27T11:25:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.477+0000 7f13df7fe700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e013f910 0x7f13e01e3300 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f13d800cbc0 tx=0x7f13d800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.477+0000 7f13d77fe700 1 -- 172.21.15.112:0/1082994219 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d800a9e0 con 0x7f13e013f910 2023-08-27T11:25:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.477+0000 7f13d77fe700 1 -- 172.21.15.112:0/1082994219 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d800ab40 con 0x7f13e013f910 2023-08-27T11:25:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.477+0000 7f13e539e700 1 -- 172.21.15.112:0/1082994219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d000b7f0 con 0x7f13e013f910 2023-08-27T11:25:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.477+0000 7f13d77fe700 1 -- 172.21.15.112:0/1082994219 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d8007070 con 0x7f13e013f910 2023-08-27T11:25:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.477+0000 7f13e539e700 1 -- 172.21.15.112:0/1082994219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e01f67a0 con 0x7f13e013f910 2023-08-27T11:25:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.478+0000 7f13e539e700 1 -- 172.21.15.112:0/1082994219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e0067a20 con 0x7f13e013f910 2023-08-27T11:25:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.478+0000 7f13d77fe700 1 -- 172.21.15.112:0/1082994219 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f13d80071d0 con 0x7f13e013f910 2023-08-27T11:25:37.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.479+0000 7f13d77fe700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13c006a320 0x7f13c006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:37.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.479+0000 7f13d77fe700 1 -- 172.21.15.112:0/1082994219 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f13d8088100 con 0x7f13e013f910 2023-08-27T11:25:37.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.479+0000 7f13deffd700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13c006a320 0x7f13c006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:37.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.480+0000 7f13deffd700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13c006a320 0x7f13c006c7e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f13d000bee0 tx=0x7f13d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:37.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.483+0000 7f13d77fe700 1 -- 172.21.15.112:0/1082994219 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f13d8053920 con 0x7f13e013f910 2023-08-27T11:25:37.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.754+0000 7f13e539e700 1 -- 172.21.15.112:0/1082994219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f13e00ccc50 con 0x7f13e013f910 2023-08-27T11:25:37.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.754+0000 7f13d77fe700 1 -- 172.21.15.112:0/1082994219 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f13d8056f40 con 0x7f13e013f910 2023-08-27T11:25:37.756 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:37.756 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:25:37.757 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2023-08-27T11:25:37.757 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.757 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2023-08-27T11:25:37.757 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk2bnkLBAA+vxcTeJzucKk9iSF/Zw3Lg==", 2023-08-27T11:25:37.757 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.758 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:25:37.758 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:25:37.758 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:25:37.758 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.758 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.759 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.759 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2023-08-27T11:25:37.759 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk3zD5OBAASVnwG6yn/YQzfYyeMakFpA==", 2023-08-27T11:25:37.759 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.760 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:25:37.760 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:25:37.760 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:25:37.760 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.760 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.761 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2023-08-27T11:25:37.761 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtki7wJNBAAsIGgDLM82sWNcKnCDlSBMw==", 2023-08-27T11:25:37.761 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.762 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:25:37.762 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:25:37.762 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:25:37.762 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.762 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.763 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2023-08-27T11:25:37.763 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkCsW/BBAAFTPiTOpBo1L+rLo9px+AjQ==", 2023-08-27T11:25:37.763 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.763 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:25:37.764 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:25:37.764 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:25:37.764 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.764 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.765 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.765 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2023-08-27T11:25:37.765 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB7MOtkR/cSHhAAR8e8VRd+e0ESJkqQDYWbEw==", 2023-08-27T11:25:37.765 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.765 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:37.766 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:25:37.766 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.766 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.767 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2023-08-27T11:25:37.767 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB+MOtksBF3HRAA1OIXiIkhgPDYpQyYPaWOXg==", 2023-08-27T11:25:37.767 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.767 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:37.767 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:25:37.767 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.768 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.768 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2023-08-27T11:25:37.768 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCBMOtkWsrdHBAAkfWbiE6wAFzpt9+BxRh8fw==", 2023-08-27T11:25:37.769 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.769 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:37.769 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:25:37.769 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.769 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.770 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2023-08-27T11:25:37.770 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCEMOtkIkiKHRAARDoh/FH9ZdqUlZZ29Mc/Dw==", 2023-08-27T11:25:37.770 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.771 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:37.771 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:25:37.771 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.771 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.772 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2023-08-27T11:25:37.772 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCKMOtkPgDzGxAA+U4kbwD0d4lS8aNPG2CgHw==", 2023-08-27T11:25:37.772 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.772 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:37.772 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:25:37.773 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.773 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.773 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2023-08-27T11:25:37.774 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCNMOtkKomSGxAAGAQUcIH2H/uyZIU8vlq1pw==", 2023-08-27T11:25:37.774 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.774 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:37.774 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:25:37.774 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.775 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.775 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2023-08-27T11:25:37.775 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCQMOtkHI2MGhAA7sgbi8JXO8P9wnVvMy3PqA==", 2023-08-27T11:25:37.776 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.776 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:37.776 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:25:37.776 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.776 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.777 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2023-08-27T11:25:37.777 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCTMOtkVHpiGhAA4C4YJMWbQ0mmGgB4i8G/AA==", 2023-08-27T11:25:37.778 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.778 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:37.778 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:25:37.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.778 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.779 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2023-08-27T11:25:37.779 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA==", 2023-08-27T11:25:37.779 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.780 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:25:37.780 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2023-08-27T11:25:37.780 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:25:37.780 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.781 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2023-08-27T11:25:37.781 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtk66DbDxAAWZ5StELux0lriJ2gEN9icg==", 2023-08-27T11:25:37.782 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.782 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:25:37.782 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:25:37.782 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:25:37.782 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:25:37.782 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.783 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2023-08-27T11:25:37.783 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkViKJDBAAF2ofwO+78er9OyFfn2bO+g==", 2023-08-27T11:25:37.783 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.784 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:25:37.784 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:25:37.784 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:25:37.784 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:25:37.784 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.785 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.785 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2023-08-27T11:25:37.785 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkwt+QFRAAowWE69VqeRhQz0896rxx9Q==", 2023-08-27T11:25:37.785 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.786 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:25:37.786 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:25:37.786 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:25:37.786 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:25:37.786 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.787 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2023-08-27T11:25:37.787 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkjDA+DxAAjmOn82AC2OMpKlAVCD6hlg==", 2023-08-27T11:25:37.787 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.787 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:25:37.788 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:25:37.788 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:25:37.788 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.788 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.789 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.789 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2023-08-27T11:25:37.789 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkXguHKRAALHwzk8diVXqsMvTKr+AzLQ==", 2023-08-27T11:25:37.789 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.789 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:25:37.790 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:25:37.790 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.790 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:37.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:37.791 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2023-08-27T11:25:37.791 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkrRjBJRAAq3ftmIh81LKadpliVUiJ6g==", 2023-08-27T11:25:37.792 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:25:37.792 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:25:37.792 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:25:37.793 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:25:37.793 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.793 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:37.794 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:25:37.794 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:25:37.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.756+0000 7f13e539e700 1 -- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13c006a320 msgr2=0x7f13c006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:37.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.756+0000 7f13e539e700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13c006a320 0x7f13c006c7e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f13d000bee0 tx=0x7f13d0008040 comp rx=0 tx=0).stop 2023-08-27T11:25:37.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.756+0000 7f13e539e700 1 -- 172.21.15.112:0/1082994219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e013f910 msgr2=0x7f13e01e3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:37.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.756+0000 7f13e539e700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e013f910 0x7f13e01e3300 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f13d800cbc0 tx=0x7f13d800ced0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.756+0000 7f13e539e700 1 -- 172.21.15.112:0/1082994219 shutdown_connections 2023-08-27T11:25:37.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.756+0000 7f13e539e700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13c006a320 0x7f13c006c7e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.756+0000 7f13e539e700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e013f910 0x7f13e01e3300 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.756+0000 7f13e539e700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e006ff70 0x7f13e01f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.756+0000 7f13e539e700 1 --2- 172.21.15.112:0/1082994219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e00ce090 0x7f13e01e2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.756+0000 7f13e539e700 1 -- 172.21.15.112:0/1082994219 >> 172.21.15.112:0/1082994219 conn(0x7f13e00b0bf0 msgr2=0x7f13e0158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:37.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.757+0000 7f13e539e700 1 -- 172.21.15.112:0/1082994219 shutdown_connections 2023-08-27T11:25:37.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.757+0000 7f13e539e700 1 -- 172.21.15.112:0/1082994219 wait complete. 2023-08-27T11:25:37.800 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:37.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.919+0000 7f0b46b86700 1 Processor -- start 2023-08-27T11:25:37.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.919+0000 7f0b46b86700 1 -- start start 2023-08-27T11:25:37.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b46b86700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0b40151060 0x7f0b4015e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:37.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b46b86700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0b4006b4c0 0x7f0b400cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:37.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b46b86700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b400b02f0 con 0x7f0b4015e990 2023-08-27T11:25:37.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b46b86700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b400b05d0 con 0x7f0b4006b4c0 2023-08-27T11:25:37.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b46b86700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b400b08b0 con 0x7f0b40151060 2023-08-27T11:25:37.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b45123700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0b4006b4c0 0x7f0b400cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:37.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b45123700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0b4006b4c0 0x7f0b400cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59852/0 (socket says 172.21.15.112:59852) 2023-08-27T11:25:37.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b45123700 1 -- 172.21.15.112:0/1932243035 learned_addr learned my addr 172.21.15.112:0/1932243035 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:37.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b44922700 1 --2- 172.21.15.112:0/1932243035 >> v2:172.21.15.138:3300/0 conn(0x7f0b40151060 0x7f0b4015e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:37.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b45123700 1 -- 172.21.15.112:0/1932243035 >> v2:172.21.15.138:3300/0 conn(0x7f0b40151060 msgr2=0x7f0b4015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:37.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b45123700 1 --2- 172.21.15.112:0/1932243035 >> v2:172.21.15.138:3300/0 conn(0x7f0b40151060 0x7f0b4015e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b45123700 1 -- 172.21.15.112:0/1932243035 >> v1:172.21.15.138:6789/0 conn(0x7f0b4015e990 legacy=0x7f0b4006adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:37.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b45123700 1 -- 172.21.15.112:0/1932243035 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400b0b90 con 0x7f0b4006b4c0 2023-08-27T11:25:37.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.920+0000 7f0b44922700 1 --2- 172.21.15.112:0/1932243035 >> v2:172.21.15.138:3300/0 conn(0x7f0b40151060 0x7f0b4015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:37.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.921+0000 7f0b45123700 1 --2- 172.21.15.112:0/1932243035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4006b4c0 0x7f0b400cdbd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0b4006e8c0 tx=0x7f0b3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31196cb5b3fb02a0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:37.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.921+0000 7f0b3f7fe700 1 -- 172.21.15.112:0/1932243035 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3800e070 con 0x7f0b4006b4c0 2023-08-27T11:25:37.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.921+0000 7f0b3f7fe700 1 -- 172.21.15.112:0/1932243035 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b38003a20 con 0x7f0b4006b4c0 2023-08-27T11:25:37.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.921+0000 7f0b3f7fe700 1 -- 172.21.15.112:0/1932243035 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38003dd0 con 0x7f0b4006b4c0 2023-08-27T11:25:37.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.921+0000 7f0b46b86700 1 -- 172.21.15.112:0/1932243035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4006b4c0 msgr2=0x7f0b400cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:37.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.921+0000 7f0b46b86700 1 --2- 172.21.15.112:0/1932243035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4006b4c0 0x7f0b400cdbd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0b4006e8c0 tx=0x7f0b3800b330 comp rx=0 tx=0).stop 2023-08-27T11:25:37.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.921+0000 7f0b46b86700 1 -- 172.21.15.112:0/1932243035 shutdown_connections 2023-08-27T11:25:37.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.921+0000 7f0b46b86700 1 --2- 172.21.15.112:0/1932243035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4006b4c0 0x7f0b400cdbd0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.921+0000 7f0b46b86700 1 --2- 172.21.15.112:0/1932243035 >> v2:172.21.15.138:3300/0 conn(0x7f0b40151060 0x7f0b4015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.921+0000 7f0b46b86700 1 -- 172.21.15.112:0/1932243035 >> 172.21.15.112:0/1932243035 conn(0x7f0b400d0000 msgr2=0x7f0b400d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:37.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.921+0000 7f0b46b86700 1 -- 172.21.15.112:0/1932243035 shutdown_connections 2023-08-27T11:25:37.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.921+0000 7f0b46b86700 1 -- 172.21.15.112:0/1932243035 wait complete. 2023-08-27T11:25:37.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.923+0000 7f0b46b86700 1 Processor -- start 2023-08-27T11:25:37.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.923+0000 7f0b46b86700 1 -- start start 2023-08-27T11:25:37.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.923+0000 7f0b46b86700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b4006b4c0 0x7f0b400c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:37.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.923+0000 7f0b46b86700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b40151060 0x7f0b401433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:37.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.923+0000 7f0b46b86700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4015e990 0x7f0b4014e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:37.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b46b86700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4014e9a0 con 0x7f0b4015e990 2023-08-27T11:25:37.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b46b86700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400c5ba0 con 0x7f0b4006b4c0 2023-08-27T11:25:37.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b46b86700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400c4b00 con 0x7f0b40151060 2023-08-27T11:25:37.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b45123700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4015e990 0x7f0b4014e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:37.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b45123700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4015e990 0x7f0b4014e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59856/0 (socket says 172.21.15.112:59856) 2023-08-27T11:25:37.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b45123700 1 -- 172.21.15.112:0/2225834033 learned_addr learned my addr 172.21.15.112:0/2225834033 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:37.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b3ffff700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b40151060 0x7f0b401433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:37.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b44922700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b4006b4c0 0x7f0b400c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:37.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b45123700 1 -- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b40151060 msgr2=0x7f0b401433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:37.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b45123700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b40151060 0x7f0b401433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b45123700 1 -- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b4006b4c0 msgr2=0x7f0b400c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:37.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b45123700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b4006b4c0 0x7f0b400c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:37.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b45123700 1 -- 172.21.15.112:0/2225834033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b38009b30 con 0x7f0b4015e990 2023-08-27T11:25:37.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b45123700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4015e990 0x7f0b4014e460 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0b38015040 tx=0x7f0b38004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:37.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b3d7fa700 1 -- 172.21.15.112:0/2225834033 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38004440 con 0x7f0b4015e990 2023-08-27T11:25:37.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b3d7fa700 1 -- 172.21.15.112:0/2225834033 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b380092e0 con 0x7f0b4015e990 2023-08-27T11:25:37.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b46b86700 1 -- 172.21.15.112:0/2225834033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b400c4de0 con 0x7f0b4015e990 2023-08-27T11:25:37.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b3d7fa700 1 -- 172.21.15.112:0/2225834033 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b380047e0 con 0x7f0b4015e990 2023-08-27T11:25:37.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.924+0000 7f0b46b86700 1 -- 172.21.15.112:0/2225834033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b400c53a0 con 0x7f0b4015e990 2023-08-27T11:25:37.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.925+0000 7f0b46b86700 1 -- 172.21.15.112:0/2225834033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b40067a20 con 0x7f0b4015e990 2023-08-27T11:25:37.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.926+0000 7f0b3d7fa700 1 -- 172.21.15.112:0/2225834033 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f0b38012430 con 0x7f0b4015e990 2023-08-27T11:25:37.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.926+0000 7f0b3d7fa700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b2406a320 0x7f0b2406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:37.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.926+0000 7f0b3d7fa700 1 -- 172.21.15.112:0/2225834033 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0b38089a90 con 0x7f0b4015e990 2023-08-27T11:25:37.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.926+0000 7f0b44922700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b2406a320 0x7f0b2406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:37.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.927+0000 7f0b44922700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b2406a320 0x7f0b2406c7e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0b34006fd0 tx=0x7f0b34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:37.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:37.930+0000 7f0b3d7fa700 1 -- 172.21.15.112:0/2225834033 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0b380550f0 con 0x7f0b4015e990 2023-08-27T11:25:38.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.120+0000 7f0b46b86700 1 -- 172.21.15.112:0/2225834033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b400cb6f0 con 0x7f0b4015e990 2023-08-27T11:25:38.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.121+0000 7f0b3d7fa700 1 -- 172.21.15.112:0/2225834033 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+14606 (secure 0 0 0) 0x7f0b38058710 con 0x7f0b4015e990 2023-08-27T11:25:38.123 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:38.123 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":73,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:25:37.318658+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:38.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.123+0000 7f0b46b86700 1 -- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b2406a320 msgr2=0x7f0b2406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.123+0000 7f0b46b86700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b2406a320 0x7f0b2406c7e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0b34006fd0 tx=0x7f0b34008040 comp rx=0 tx=0).stop 2023-08-27T11:25:38.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.123+0000 7f0b46b86700 1 -- 172.21.15.112:0/2225834033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4015e990 msgr2=0x7f0b4014e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.123+0000 7f0b46b86700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4015e990 0x7f0b4014e460 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0b38015040 tx=0x7f0b38004ad0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.123+0000 7f0b46b86700 1 -- 172.21.15.112:0/2225834033 shutdown_connections 2023-08-27T11:25:38.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.123+0000 7f0b46b86700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b2406a320 0x7f0b2406c7e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.123+0000 7f0b46b86700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4015e990 0x7f0b4014e460 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.123+0000 7f0b46b86700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b4006b4c0 0x7f0b400c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.123+0000 7f0b46b86700 1 --2- 172.21.15.112:0/2225834033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b40151060 0x7f0b401433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.123+0000 7f0b46b86700 1 -- 172.21.15.112:0/2225834033 >> 172.21.15.112:0/2225834033 conn(0x7f0b400d0000 msgr2=0x7f0b4014f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:38.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.123+0000 7f0b46b86700 1 -- 172.21.15.112:0/2225834033 shutdown_connections 2023-08-27T11:25:38.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.123+0000 7f0b46b86700 1 -- 172.21.15.112:0/2225834033 wait complete. 2023-08-27T11:25:38.137 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:25:38.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:25:38.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.260+0000 7f524c9e3700 1 Processor -- start 2023-08-27T11:25:38.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.260+0000 7f524c9e3700 1 -- start start 2023-08-27T11:25:38.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f524c9e3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52480b3de0 0x7f524806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f524c9e3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52480cef20 con 0x7f524806c510 2023-08-27T11:25:38.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f524c9e3700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524806c940 con 0x7f5248144b80 2023-08-27T11:25:38.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f524c9e3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52480b4b40 con 0x7f52480b3de0 2023-08-27T11:25:38.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f5246d9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f524806c510 0x7f52480cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54248/0 (socket says 172.21.15.112:54248) 2023-08-27T11:25:38.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f5246d9d700 1 -- 172.21.15.112:0/1739391701 learned_addr learned my addr 172.21.15.112:0/1739391701 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:38.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f5245d9b700 1 --2- 172.21.15.112:0/1739391701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52480b3de0 0x7f524806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f524559a700 1 -- 172.21.15.112:0/1739391701 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163350071 0 0) 0x7f52480cef20 con 0x7f524806c510 2023-08-27T11:25:38.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f524559a700 1 -- 172.21.15.112:0/1739391701 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52300035b0 con 0x7f524806c510 2023-08-27T11:25:38.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f524559a700 1 -- 172.21.15.112:0/1739391701 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842231539 0 0) 0x7f524806c940 con 0x7f5248144b80 2023-08-27T11:25:38.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f524559a700 1 -- 172.21.15.112:0/1739391701 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52480cef20 con 0x7f5248144b80 2023-08-27T11:25:38.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f5245d9b700 1 -- 172.21.15.112:0/1739391701 >> v1:172.21.15.138:6789/0 conn(0x7f5248144b80 legacy=0x7f52480b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f5245d9b700 1 -- 172.21.15.112:0/1739391701 >> v1:172.21.15.112:6789/0 conn(0x7f524806c510 legacy=0x7f52480cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.261+0000 7f5245d9b700 1 -- 172.21.15.112:0/1739391701 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52480b7310 con 0x7f52480b3de0 2023-08-27T11:25:38.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.262+0000 7f5245d9b700 1 --2- 172.21.15.112:0/1739391701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52480b3de0 0x7f524806bfd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f523c009870 tx=0x7f523c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a093d2ae0f95dd9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:38.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.262+0000 7f524559a700 1 -- 172.21.15.112:0/1739391701 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c011830 con 0x7f52480b3de0 2023-08-27T11:25:38.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.262+0000 7f524559a700 1 -- 172.21.15.112:0/1739391701 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c011990 con 0x7f52480b3de0 2023-08-27T11:25:38.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.262+0000 7f524559a700 1 -- 172.21.15.112:0/1739391701 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c011d20 con 0x7f52480b3de0 2023-08-27T11:25:38.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.262+0000 7f524c9e3700 1 -- 172.21.15.112:0/1739391701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52480b3de0 msgr2=0x7f524806bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.262+0000 7f524c9e3700 1 --2- 172.21.15.112:0/1739391701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52480b3de0 0x7f524806bfd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f523c009870 tx=0x7f523c009b80 comp rx=0 tx=0).stop 2023-08-27T11:25:38.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.262+0000 7f524c9e3700 1 -- 172.21.15.112:0/1739391701 shutdown_connections 2023-08-27T11:25:38.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.262+0000 7f524c9e3700 1 --2- 172.21.15.112:0/1739391701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52480b3de0 0x7f524806bfd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.262+0000 7f524c9e3700 1 -- 172.21.15.112:0/1739391701 >> 172.21.15.112:0/1739391701 conn(0x7f52480cdc50 msgr2=0x7f52480ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.262+0000 7f524c9e3700 1 -- 172.21.15.112:0/1739391701 shutdown_connections 2023-08-27T11:25:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.262+0000 7f524c9e3700 1 -- 172.21.15.112:0/1739391701 wait complete. 2023-08-27T11:25:38.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.264+0000 7f524c9e3700 1 Processor -- start 2023-08-27T11:25:38.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.264+0000 7f524c9e3700 1 -- start start 2023-08-27T11:25:38.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.264+0000 7f524c9e3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f524806c510 0x7f52481f0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.264+0000 7f524c9e3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52480b3de0 0x7f5248143840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.264+0000 7f524c9e3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5248144b80 0x7f5248154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.264+0000 7f524c9e3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5248156ce0 con 0x7f52480b3de0 2023-08-27T11:25:38.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.264+0000 7f524c9e3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52481f0ba0 con 0x7f524806c510 2023-08-27T11:25:38.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.264+0000 7f524c9e3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52481560d0 con 0x7f5248144b80 2023-08-27T11:25:38.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f5245d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52480b3de0 0x7f5248143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f5245d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52480b3de0 0x7f5248143840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59870/0 (socket says 172.21.15.112:59870) 2023-08-27T11:25:38.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f5245d9b700 1 -- 172.21.15.112:0/694479184 learned_addr learned my addr 172.21.15.112:0/694479184 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:38.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f5246d9d700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5248144b80 0x7f5248154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f524659c700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f524806c510 0x7f52481f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f5245d9b700 1 -- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5248144b80 msgr2=0x7f5248154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f5245d9b700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5248144b80 0x7f5248154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f5245d9b700 1 -- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f524806c510 msgr2=0x7f52481f0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f5245d9b700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f524806c510 0x7f52481f0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f5245d9b700 1 -- 172.21.15.112:0/694479184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c0094a0 con 0x7f52480b3de0 2023-08-27T11:25:38.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f5245d9b700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52480b3de0 0x7f5248143840 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f523c0090b0 tx=0x7f523c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:38.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f5236ffd700 1 -- 172.21.15.112:0/694479184 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c01a6e0 con 0x7f52480b3de0 2023-08-27T11:25:38.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f5236ffd700 1 -- 172.21.15.112:0/694479184 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c01a840 con 0x7f52480b3de0 2023-08-27T11:25:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f5236ffd700 1 -- 172.21.15.112:0/694479184 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c01abb0 con 0x7f52480b3de0 2023-08-27T11:25:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f524c9e3700 1 -- 172.21.15.112:0/694479184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52481563b0 con 0x7f52480b3de0 2023-08-27T11:25:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.265+0000 7f524c9e3700 1 -- 172.21.15.112:0/694479184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5248156970 con 0x7f52480b3de0 2023-08-27T11:25:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.266+0000 7f524c9e3700 1 -- 172.21.15.112:0/694479184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5248156e90 con 0x7f52480b3de0 2023-08-27T11:25:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.267+0000 7f5236ffd700 1 -- 172.21.15.112:0/694479184 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f523c022070 con 0x7f52480b3de0 2023-08-27T11:25:38.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.267+0000 7f5236ffd700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f522806a320 0x7f522806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.267+0000 7f5236ffd700 1 -- 172.21.15.112:0/694479184 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f523c089100 con 0x7f52480b3de0 2023-08-27T11:25:38.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.267+0000 7f524659c700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f522806a320 0x7f522806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.268+0000 7f524659c700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f522806a320 0x7f522806c7e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f52380011e0 tx=0x7f5238009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:38.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.271+0000 7f5236ffd700 1 -- 172.21.15.112:0/694479184 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f523c054920 con 0x7f52480b3de0 2023-08-27T11:25:38.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.458+0000 7f524c9e3700 1 -- 172.21.15.112:0/694479184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5248067a20 con 0x7f52480b3de0 2023-08-27T11:25:38.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.459+0000 7f5236ffd700 1 -- 172.21.15.112:0/694479184 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v73) v1 ==== 92+0+10 (secure 0 0 0) 0x7f523c015020 con 0x7f52480b3de0 2023-08-27T11:25:38.461 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:25:38.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.461+0000 7f524c9e3700 1 -- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f522806a320 msgr2=0x7f522806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.461+0000 7f524c9e3700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f522806a320 0x7f522806c7e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f52380011e0 tx=0x7f5238009040 comp rx=0 tx=0).stop 2023-08-27T11:25:38.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.461+0000 7f524c9e3700 1 -- 172.21.15.112:0/694479184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52480b3de0 msgr2=0x7f5248143840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.461+0000 7f524c9e3700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52480b3de0 0x7f5248143840 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f523c0090b0 tx=0x7f523c004750 comp rx=0 tx=0).stop 2023-08-27T11:25:38.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.461+0000 7f524c9e3700 1 -- 172.21.15.112:0/694479184 shutdown_connections 2023-08-27T11:25:38.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.461+0000 7f524c9e3700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f522806a320 0x7f522806c7e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.461+0000 7f524c9e3700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5248144b80 0x7f5248154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.461+0000 7f524c9e3700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f524806c510 0x7f52481f0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.461+0000 7f524c9e3700 1 --2- 172.21.15.112:0/694479184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52480b3de0 0x7f5248143840 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.461+0000 7f524c9e3700 1 -- 172.21.15.112:0/694479184 >> 172.21.15.112:0/694479184 conn(0x7f52480cdc50 msgr2=0x7f52480c6170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:38.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.461+0000 7f524c9e3700 1 -- 172.21.15.112:0/694479184 shutdown_connections 2023-08-27T11:25:38.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.461+0000 7f524c9e3700 1 -- 172.21.15.112:0/694479184 wait complete. 2023-08-27T11:25:38.474 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:25:38.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.596+0000 7f81a6743700 1 Processor -- start 2023-08-27T11:25:38.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.597+0000 7f81a6743700 1 -- start start 2023-08-27T11:25:38.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.597+0000 7f81a6743700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f81a0143390 0x7f81a00723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.597+0000 7f81a6743700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a006f820 0x7f81a006ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.597+0000 7f81a6743700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f81a006e2f0 0x7f81a015d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.597+0000 7f81a6743700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a0070000 con 0x7f81a006e2f0 2023-08-27T11:25:38.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.597+0000 7f81a6743700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a006e780 con 0x7f81a0143390 2023-08-27T11:25:38.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.597+0000 7f81a6743700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00b16e0 con 0x7f81a006f820 2023-08-27T11:25:38.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f81a4ce0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f81a006e2f0 0x7f81a015d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f81a4ce0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f81a006e2f0 0x7f81a015d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59880/0 (socket says 172.21.15.112:59880) 2023-08-27T11:25:38.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f81a4ce0700 1 -- 172.21.15.112:0/3787218691 learned_addr learned my addr 172.21.15.112:0/3787218691 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:38.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f819ffff700 1 --2- 172.21.15.112:0/3787218691 >> v2:172.21.15.138:3300/0 conn(0x7f81a0143390 0x7f81a00723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f819f7fe700 1 --2- 172.21.15.112:0/3787218691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a006f820 0x7f81a006ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f81a4ce0700 1 -- 172.21.15.112:0/3787218691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a006f820 msgr2=0x7f81a006ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:38.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f81a4ce0700 1 --2- 172.21.15.112:0/3787218691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a006f820 0x7f81a006ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f81a4ce0700 1 -- 172.21.15.112:0/3787218691 >> v2:172.21.15.138:3300/0 conn(0x7f81a0143390 msgr2=0x7f81a00723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:38.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f81a4ce0700 1 --2- 172.21.15.112:0/3787218691 >> v2:172.21.15.138:3300/0 conn(0x7f81a0143390 0x7f81a00723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f81a4ce0700 1 -- 172.21.15.112:0/3787218691 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a0143770 con 0x7f81a006e2f0 2023-08-27T11:25:38.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f81a4ce0700 1 --2- 172.21.15.112:0/3787218691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a006e2f0 0x7f81a015d450 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f81a00cd0f0 tx=0x7f819800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2868578592f349d6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:38.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f819effd700 1 -- 172.21.15.112:0/3787218691 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198010070 con 0x7f81a006e2f0 2023-08-27T11:25:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f819effd700 1 -- 172.21.15.112:0/3787218691 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819800dd90 con 0x7f81a006e2f0 2023-08-27T11:25:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.598+0000 7f819effd700 1 -- 172.21.15.112:0/3787218691 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819800b310 con 0x7f81a006e2f0 2023-08-27T11:25:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.599+0000 7f81a6743700 1 -- 172.21.15.112:0/3787218691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a006e2f0 msgr2=0x7f81a015d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.599+0000 7f81a6743700 1 --2- 172.21.15.112:0/3787218691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a006e2f0 0x7f81a015d450 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f81a00cd0f0 tx=0x7f819800d430 comp rx=0 tx=0).stop 2023-08-27T11:25:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.599+0000 7f81a6743700 1 -- 172.21.15.112:0/3787218691 shutdown_connections 2023-08-27T11:25:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.599+0000 7f81a6743700 1 --2- 172.21.15.112:0/3787218691 >> v2:172.21.15.138:3300/0 conn(0x7f81a0143390 0x7f81a00723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.599+0000 7f81a6743700 1 --2- 172.21.15.112:0/3787218691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a006e2f0 0x7f81a015d450 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.599+0000 7f81a6743700 1 --2- 172.21.15.112:0/3787218691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a006f820 0x7f81a006ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.599+0000 7f81a6743700 1 -- 172.21.15.112:0/3787218691 >> 172.21.15.112:0/3787218691 conn(0x7f81a00cc240 msgr2=0x7f81a00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:38.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.599+0000 7f81a6743700 1 -- 172.21.15.112:0/3787218691 shutdown_connections 2023-08-27T11:25:38.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.599+0000 7f81a6743700 1 -- 172.21.15.112:0/3787218691 wait complete. 2023-08-27T11:25:38.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.600+0000 7f81a6743700 1 Processor -- start 2023-08-27T11:25:38.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.601+0000 7f81a6743700 1 -- start start 2023-08-27T11:25:38.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.601+0000 7f81a6743700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a006e2f0 0x7f81a0159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.601+0000 7f81a6743700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a006f820 0x7f81a013f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.601+0000 7f81a6743700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a0143390 0x7f81a013ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.601+0000 7f81a6743700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0159760 con 0x7f81a006f820 2023-08-27T11:25:38.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.601+0000 7f81a6743700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a013ff50 con 0x7f81a006e2f0 2023-08-27T11:25:38.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.601+0000 7f81a6743700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0140230 con 0x7f81a0143390 2023-08-27T11:25:38.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.601+0000 7f819f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a006f820 0x7f81a013f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f819f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a006f820 0x7f81a013f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59884/0 (socket says 172.21.15.112:59884) 2023-08-27T11:25:38.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f819f7fe700 1 -- 172.21.15.112:0/2081536615 learned_addr learned my addr 172.21.15.112:0/2081536615 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:38.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f81a4ce0700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a0143390 0x7f81a013ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f819ffff700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a006e2f0 0x7f81a0159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f819f7fe700 1 -- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a0143390 msgr2=0x7f81a013ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f819f7fe700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a0143390 0x7f81a013ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f819f7fe700 1 -- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a006e2f0 msgr2=0x7f81a0159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f819f7fe700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a006e2f0 0x7f81a0159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f819f7fe700 1 -- 172.21.15.112:0/2081536615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819800bd40 con 0x7f81a006f820 2023-08-27T11:25:38.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f819ffff700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a006e2f0 0x7f81a0159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:38.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f819f7fe700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a006f820 0x7f81a013f620 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f819000e810 tx=0x7f819000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:38.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f819cff9700 1 -- 172.21.15.112:0/2081536615 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190013610 con 0x7f81a006f820 2023-08-27T11:25:38.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f819cff9700 1 -- 172.21.15.112:0/2081536615 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8190013770 con 0x7f81a006f820 2023-08-27T11:25:38.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f81a6743700 1 -- 172.21.15.112:0/2081536615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a0140510 con 0x7f81a006f820 2023-08-27T11:25:38.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f819cff9700 1 -- 172.21.15.112:0/2081536615 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190013b00 con 0x7f81a006f820 2023-08-27T11:25:38.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.602+0000 7f81a6743700 1 -- 172.21.15.112:0/2081536615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a01eebf0 con 0x7f81a006f820 2023-08-27T11:25:38.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.603+0000 7f81a6743700 1 -- 172.21.15.112:0/2081536615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a0067e90 con 0x7f81a006f820 2023-08-27T11:25:38.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.604+0000 7f819cff9700 1 -- 172.21.15.112:0/2081536615 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f8190013c60 con 0x7f81a006f820 2023-08-27T11:25:38.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.604+0000 7f819cff9700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f818c06a320 0x7f818c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.604+0000 7f819cff9700 1 -- 172.21.15.112:0/2081536615 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f81900883a0 con 0x7f81a006f820 2023-08-27T11:25:38.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.604+0000 7f819ffff700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f818c06a320 0x7f818c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.605+0000 7f819ffff700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f818c06a320 0x7f818c06c7e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8194007a50 tx=0x7f8194008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:38.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.609+0000 7f819cff9700 1 -- 172.21.15.112:0/2081536615 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8190053a00 con 0x7f81a006f820 2023-08-27T11:25:38.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.796+0000 7f81a6743700 1 -- 172.21.15.112:0/2081536615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f81a0067a20 con 0x7f81a006f820 2023-08-27T11:25:38.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.796+0000 7f819cff9700 1 -- 172.21.15.112:0/2081536615 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8190057020 con 0x7f81a006f820 2023-08-27T11:25:38.798 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:25:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.798+0000 7f81a6743700 1 -- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f818c06a320 msgr2=0x7f818c06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.798+0000 7f81a6743700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f818c06a320 0x7f818c06c7e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8194007a50 tx=0x7f8194008040 comp rx=0 tx=0).stop 2023-08-27T11:25:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.798+0000 7f81a6743700 1 -- 172.21.15.112:0/2081536615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a006f820 msgr2=0x7f81a013f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.798+0000 7f81a6743700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a006f820 0x7f81a013f620 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f819000e810 tx=0x7f819000ebd0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.798+0000 7f81a6743700 1 -- 172.21.15.112:0/2081536615 shutdown_connections 2023-08-27T11:25:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.798+0000 7f81a6743700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f818c06a320 0x7f818c06c7e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.798+0000 7f81a6743700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a0143390 0x7f81a013ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.798+0000 7f81a6743700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a006e2f0 0x7f81a0159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.798+0000 7f81a6743700 1 --2- 172.21.15.112:0/2081536615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a006f820 0x7f81a013f620 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.798+0000 7f81a6743700 1 -- 172.21.15.112:0/2081536615 >> 172.21.15.112:0/2081536615 conn(0x7f81a00cc240 msgr2=0x7f81a00cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.798+0000 7f81a6743700 1 -- 172.21.15.112:0/2081536615 shutdown_connections 2023-08-27T11:25:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.798+0000 7f81a6743700 1 -- 172.21.15.112:0/2081536615 wait complete. 2023-08-27T11:25:38.812 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:25:38.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.931+0000 7f77cd0e6700 1 Processor -- start 2023-08-27T11:25:38.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.931+0000 7f77cd0e6700 1 -- start start 2023-08-27T11:25:38.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77cd0e6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f77c80b1400 0x7f77c80cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77cd0e6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77c80b17e0 0x7f77c806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77cd0e6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f77c8161ce0 0x7f77c806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77cd0e6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c8161850 con 0x7f77c8161ce0 2023-08-27T11:25:38.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77cd0e6700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c806e7d0 con 0x7f77c80b1400 2023-08-27T11:25:38.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77cd0e6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80cfc10 con 0x7f77c80b17e0 2023-08-27T11:25:38.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77c759e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f77c8161ce0 0x7f77c806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77c759e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f77c8161ce0 0x7f77c806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59894/0 (socket says 172.21.15.112:59894) 2023-08-27T11:25:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77c759e700 1 -- 172.21.15.112:0/2325836846 learned_addr learned my addr 172.21.15.112:0/2325836846 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77c659c700 1 --2- 172.21.15.112:0/2325836846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77c80b17e0 0x7f77c806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77c6d9d700 1 --2- 172.21.15.112:0/2325836846 >> v2:172.21.15.138:3300/0 conn(0x7f77c80b1400 0x7f77c80cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77c759e700 1 -- 172.21.15.112:0/2325836846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77c80b17e0 msgr2=0x7f77c806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77c759e700 1 --2- 172.21.15.112:0/2325836846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77c80b17e0 0x7f77c806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77c759e700 1 -- 172.21.15.112:0/2325836846 >> v2:172.21.15.138:3300/0 conn(0x7f77c80b1400 msgr2=0x7f77c80cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77c759e700 1 --2- 172.21.15.112:0/2325836846 >> v2:172.21.15.138:3300/0 conn(0x7f77c80b1400 0x7f77c80cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.932+0000 7f77c759e700 1 -- 172.21.15.112:0/2325836846 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c8161200 con 0x7f77c8161ce0 2023-08-27T11:25:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.933+0000 7f77c759e700 1 --2- 172.21.15.112:0/2325836846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c8161ce0 0x7f77c806f790 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f77c8070370 tx=0x7f77b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38209fbf6b2de8ba server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.933+0000 7f77c5d9b700 1 -- 172.21.15.112:0/2325836846 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b800b870 con 0x7f77c8161ce0 2023-08-27T11:25:38.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.933+0000 7f77c5d9b700 1 -- 172.21.15.112:0/2325836846 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b800b9d0 con 0x7f77c8161ce0 2023-08-27T11:25:38.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.933+0000 7f77c5d9b700 1 -- 172.21.15.112:0/2325836846 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b800bd60 con 0x7f77c8161ce0 2023-08-27T11:25:38.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.933+0000 7f77cd0e6700 1 -- 172.21.15.112:0/2325836846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c8161ce0 msgr2=0x7f77c806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.933+0000 7f77cd0e6700 1 --2- 172.21.15.112:0/2325836846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c8161ce0 0x7f77c806f790 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f77c8070370 tx=0x7f77b8009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.933+0000 7f77cd0e6700 1 -- 172.21.15.112:0/2325836846 shutdown_connections 2023-08-27T11:25:38.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.933+0000 7f77cd0e6700 1 --2- 172.21.15.112:0/2325836846 >> v2:172.21.15.138:3300/0 conn(0x7f77c80b1400 0x7f77c80cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.933+0000 7f77cd0e6700 1 --2- 172.21.15.112:0/2325836846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c8161ce0 0x7f77c806f790 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.933+0000 7f77cd0e6700 1 --2- 172.21.15.112:0/2325836846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77c80b17e0 0x7f77c806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.933+0000 7f77cd0e6700 1 -- 172.21.15.112:0/2325836846 >> 172.21.15.112:0/2325836846 conn(0x7f77c80d0000 msgr2=0x7f77c80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:38.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.933+0000 7f77cd0e6700 1 -- 172.21.15.112:0/2325836846 shutdown_connections 2023-08-27T11:25:38.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.933+0000 7f77cd0e6700 1 -- 172.21.15.112:0/2325836846 wait complete. 2023-08-27T11:25:38.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.935+0000 7f77cd0e6700 1 Processor -- start 2023-08-27T11:25:38.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.935+0000 7f77cd0e6700 1 -- start start 2023-08-27T11:25:38.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.935+0000 7f77cd0e6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c80b1400 0x7f77c8154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.935+0000 7f77cd0e6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c80b17e0 0x7f77c81548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77cd0e6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77c8161ce0 0x7f77c806ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77cd0e6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80c5a70 con 0x7f77c80b17e0 2023-08-27T11:25:38.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77cd0e6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80c4b80 con 0x7f77c80b1400 2023-08-27T11:25:38.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77cd0e6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80c4e60 con 0x7f77c8161ce0 2023-08-27T11:25:38.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77c659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c80b17e0 0x7f77c81548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77c659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c80b17e0 0x7f77c81548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59896/0 (socket says 172.21.15.112:59896) 2023-08-27T11:25:38.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77c659c700 1 -- 172.21.15.112:0/1549723023 learned_addr learned my addr 172.21.15.112:0/1549723023 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:38.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77c6d9d700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c80b1400 0x7f77c8154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77c759e700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77c8161ce0 0x7f77c806ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77c659c700 1 -- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77c8161ce0 msgr2=0x7f77c806ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77c659c700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77c8161ce0 0x7f77c806ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77c659c700 1 -- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c80b1400 msgr2=0x7f77c8154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:38.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77c659c700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c80b1400 0x7f77c8154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:38.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77c659c700 1 -- 172.21.15.112:0/1549723023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b8009550 con 0x7f77c80b17e0 2023-08-27T11:25:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77c759e700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77c8161ce0 0x7f77c806ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77c659c700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c80b17e0 0x7f77c81548b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00e810 tx=0x7f77bc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77af7fe700 1 -- 172.21.15.112:0/1549723023 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc013610 con 0x7f77c80b17e0 2023-08-27T11:25:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77af7fe700 1 -- 172.21.15.112:0/1549723023 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc013770 con 0x7f77c80b17e0 2023-08-27T11:25:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77cd0e6700 1 -- 172.21.15.112:0/1549723023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c80c5140 con 0x7f77c80b17e0 2023-08-27T11:25:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77af7fe700 1 -- 172.21.15.112:0/1549723023 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc013b20 con 0x7f77c80b17e0 2023-08-27T11:25:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.936+0000 7f77cd0e6700 1 -- 172.21.15.112:0/1549723023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c80c5700 con 0x7f77c80b17e0 2023-08-27T11:25:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.937+0000 7f77cd0e6700 1 -- 172.21.15.112:0/1549723023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c8067a20 con 0x7f77c80b17e0 2023-08-27T11:25:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.938+0000 7f77af7fe700 1 -- 172.21.15.112:0/1549723023 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f77bc013c80 con 0x7f77c80b17e0 2023-08-27T11:25:38.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.938+0000 7f77af7fe700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f77b406a320 0x7f77b406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:38.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.938+0000 7f77af7fe700 1 -- 172.21.15.112:0/1549723023 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f77bc088440 con 0x7f77c80b17e0 2023-08-27T11:25:38.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.938+0000 7f77c6d9d700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f77b406a320 0x7f77b406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:38.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.939+0000 7f77c6d9d700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f77b406a320 0x7f77b406c7e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f77c0007fd0 tx=0x7f77c0007560 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:38.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:38.942+0000 7f77af7fe700 1 -- 172.21.15.112:0/1549723023 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f77bc053a20 con 0x7f77c80b17e0 2023-08-27T11:25:39.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.133+0000 7f77cd0e6700 1 -- 172.21.15.112:0/1549723023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f77c806db60 con 0x7f77c80b17e0 2023-08-27T11:25:39.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.133+0000 7f77af7fe700 1 -- 172.21.15.112:0/1549723023 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) v1 ==== 84+0+126 (secure 0 0 0) 0x7f77bc057040 con 0x7f77c80b17e0 2023-08-27T11:25:39.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:39.135 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:25:39.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:39.136 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2023-08-27T11:25:39.136 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "rbd" 2023-08-27T11:25:39.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:39.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:39.137 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 2, 2023-08-27T11:25:39.137 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2023-08-27T11:25:39.137 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:39.137 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:25:39.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.135+0000 7f77cd0e6700 1 -- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f77b406a320 msgr2=0x7f77b406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:39.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.135+0000 7f77cd0e6700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f77b406a320 0x7f77b406c7e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f77c0007fd0 tx=0x7f77c0007560 comp rx=0 tx=0).stop 2023-08-27T11:25:39.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.135+0000 7f77cd0e6700 1 -- 172.21.15.112:0/1549723023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c80b17e0 msgr2=0x7f77c81548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:39.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.135+0000 7f77cd0e6700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c80b17e0 0x7f77c81548b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00e810 tx=0x7f77bc00ebd0 comp rx=0 tx=0).stop 2023-08-27T11:25:39.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.135+0000 7f77cd0e6700 1 -- 172.21.15.112:0/1549723023 shutdown_connections 2023-08-27T11:25:39.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.135+0000 7f77cd0e6700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f77b406a320 0x7f77b406c7e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:39.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.135+0000 7f77cd0e6700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77c8161ce0 0x7f77c806ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:39.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.135+0000 7f77cd0e6700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c80b1400 0x7f77c8154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:39.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.135+0000 7f77cd0e6700 1 --2- 172.21.15.112:0/1549723023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c80b17e0 0x7f77c81548b0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:39.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.135+0000 7f77cd0e6700 1 -- 172.21.15.112:0/1549723023 >> 172.21.15.112:0/1549723023 conn(0x7f77c80d0000 msgr2=0x7f77c80d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:39.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.135+0000 7f77cd0e6700 1 -- 172.21.15.112:0/1549723023 shutdown_connections 2023-08-27T11:25:39.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.135+0000 7f77cd0e6700 1 -- 172.21.15.112:0/1549723023 wait complete. 2023-08-27T11:25:39.149 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:25:39.149 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:25:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.271+0000 7f0789b63700 1 Processor -- start 2023-08-27T11:25:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.271+0000 7f0789b63700 1 -- start start 2023-08-27T11:25:39.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.272+0000 7f0789b63700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0784144b10 0x7f07840ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:39.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.272+0000 7f0789b63700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f07840b5d60 0x7f078415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:39.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.272+0000 7f0789b63700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07840ced80 con 0x7f078415f8a0 2023-08-27T11:25:39.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.272+0000 7f0789b63700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07840cefd0 con 0x7f07840b5d60 2023-08-27T11:25:39.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.272+0000 7f0789b63700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07840cf2b0 con 0x7f0784144b10 2023-08-27T11:25:39.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.272+0000 7f0782ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f078415f8a0 0x7f07840b5650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54286/0 (socket says 172.21.15.112:54286) 2023-08-27T11:25:39.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.272+0000 7f0782ffd700 1 -- 172.21.15.112:0/475620272 learned_addr learned my addr 172.21.15.112:0/475620272 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:39.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f0783fff700 1 --2- 172.21.15.112:0/475620272 >> v2:172.21.15.138:3300/0 conn(0x7f07840b5d60 0x7f078415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:39.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07837fe700 1 --2- 172.21.15.112:0/475620272 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0784144b10 0x7f07840ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:39.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013095628 0 0) 0x7f07840ced80 con 0x7f078415f8a0 2023-08-27T11:25:39.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f076c0035b0 con 0x7f078415f8a0 2023-08-27T11:25:39.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2638454533 0 0) 0x7f076c0035b0 con 0x7f078415f8a0 2023-08-27T11:25:39.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0784144b10 msgr2=0x7f07840ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:39.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 --2- 172.21.15.112:0/475620272 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0784144b10 0x7f07840ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:39.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 >> v2:172.21.15.138:3300/0 conn(0x7f07840b5d60 msgr2=0x7f078415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:39.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 --2- 172.21.15.112:0/475620272 >> v2:172.21.15.138:3300/0 conn(0x7f07840b5d60 0x7f078415baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:39.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07840cf590 con 0x7f078415f8a0 2023-08-27T11:25:39.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f0774001670 con 0x7f078415f8a0 2023-08-27T11:25:39.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07837fe700 1 --2- 172.21.15.112:0/475620272 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0784144b10 0x7f07840ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:39.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 >> v1:172.21.15.112:6789/0 conn(0x7f078415f8a0 legacy=0x7f07840b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:39.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 --2- 172.21.15.112:0/475620272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f076c003a00 0x7f076c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:39.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 --2- 172.21.15.112:0/475620272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f076c004850 0x7f076c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:39.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 --2- 172.21.15.112:0/475620272 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0784144b10 0x7f076c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:39.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07840cefd0 con 0x7f076c004850 2023-08-27T11:25:39.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07840cf2b0 con 0x7f076c003a00 2023-08-27T11:25:39.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076c00ba10 con 0x7f0784144b10 2023-08-27T11:25:39.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f0783fff700 1 --2- 172.21.15.112:0/475620272 >> v2:172.21.15.138:3300/0 conn(0x7f07840b5d60 0x7f078415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:39.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f0782ffd700 1 --2- 172.21.15.112:0/475620272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f076c004850 0x7f076c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:39.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f07837fe700 1 --2- 172.21.15.112:0/475620272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f076c003a00 0x7f076c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:39.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f0783fff700 1 --2- 172.21.15.112:0/475620272 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0784144b10 0x7f076c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:39.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f0782ffd700 1 -- 172.21.15.112:0/475620272 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0784144b10 msgr2=0x7f076c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:39.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f0782ffd700 1 --2- 172.21.15.112:0/475620272 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0784144b10 0x7f076c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:39.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f0782ffd700 1 -- 172.21.15.112:0/475620272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f076c003a00 msgr2=0x7f076c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:39.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f0782ffd700 1 --2- 172.21.15.112:0/475620272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f076c003a00 0x7f076c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:39.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.273+0000 7f0782ffd700 1 -- 172.21.15.112:0/475620272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07780094a0 con 0x7f076c004850 2023-08-27T11:25:39.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.274+0000 7f0782ffd700 1 --2- 172.21.15.112:0/475620272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f076c004850 0x7f076c004cd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f077400c320 tx=0x7f076c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a9635a26ce1cdc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:39.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.274+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077400ce60 con 0x7f076c004850 2023-08-27T11:25:39.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.274+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0774005960 con 0x7f076c004850 2023-08-27T11:25:39.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.274+0000 7f07827fc700 1 -- 172.21.15.112:0/475620272 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774005cf0 con 0x7f076c004850 2023-08-27T11:25:39.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.274+0000 7f0789b63700 1 -- 172.21.15.112:0/475620272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f076c004850 msgr2=0x7f076c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:39.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.274+0000 7f0789b63700 1 --2- 172.21.15.112:0/475620272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f076c004850 0x7f076c004cd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f077400c320 tx=0x7f076c005880 comp rx=0 tx=0).stop 2023-08-27T11:25:39.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.274+0000 7f0783fff700 1 -- 172.21.15.112:0/475620272 reap_dead start 2023-08-27T11:25:39.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.274+0000 7f0789b63700 1 -- 172.21.15.112:0/475620272 shutdown_connections 2023-08-27T11:25:39.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.274+0000 7f0789b63700 1 -- 172.21.15.112:0/475620272 >> 172.21.15.112:0/475620272 conn(0x7f07840b1be0 msgr2=0x7f07840b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:39.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.274+0000 7f0789b63700 1 -- 172.21.15.112:0/475620272 shutdown_connections 2023-08-27T11:25:39.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.274+0000 7f0789b63700 1 -- 172.21.15.112:0/475620272 wait complete. 2023-08-27T11:25:39.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.276+0000 7f0789b63700 1 Processor -- start 2023-08-27T11:25:39.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.276+0000 7f0789b63700 1 -- start start 2023-08-27T11:25:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f0789b63700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f076c004850 0x7f07840ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f0789b63700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07840b5240 0x7f078415a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f0789b63700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07840b3e20 0x7f0784070840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f0789b63700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07840ad980 con 0x7f076c004850 2023-08-27T11:25:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f0789b63700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07840adc20 con 0x7f07840b5240 2023-08-27T11:25:39.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f0789b63700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07840adf00 con 0x7f07840b3e20 2023-08-27T11:25:39.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f07837fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f076c004850 0x7f07840ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:39.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f07837fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f076c004850 0x7f07840ae9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59916/0 (socket says 172.21.15.112:59916) 2023-08-27T11:25:39.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f07837fe700 1 -- 172.21.15.112:0/2386807860 learned_addr learned my addr 172.21.15.112:0/2386807860 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:39.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f0783fff700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07840b3e20 0x7f0784070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:39.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f0782ffd700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07840b5240 0x7f078415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:39.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f07837fe700 1 -- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07840b3e20 msgr2=0x7f0784070840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:39.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f07837fe700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07840b3e20 0x7f0784070840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:39.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f07837fe700 1 -- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07840b5240 msgr2=0x7f078415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:39.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f07837fe700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07840b5240 0x7f078415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:39.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f07837fe700 1 -- 172.21.15.112:0/2386807860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07840ae1e0 con 0x7f076c004850 2023-08-27T11:25:39.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f0782ffd700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07840b5240 0x7f078415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:39.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.277+0000 7f0783fff700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07840b3e20 0x7f0784070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:39.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.278+0000 7f07837fe700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f076c004850 0x7f07840ae9e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0778011ca0 tx=0x7f077800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:39.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.278+0000 7f076bfff700 1 -- 172.21.15.112:0/2386807860 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077800fb30 con 0x7f076c004850 2023-08-27T11:25:39.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.278+0000 7f076bfff700 1 -- 172.21.15.112:0/2386807860 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077800fc90 con 0x7f076c004850 2023-08-27T11:25:39.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.278+0000 7f0789b63700 1 -- 172.21.15.112:0/2386807860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07840ae4c0 con 0x7f076c004850 2023-08-27T11:25:39.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.278+0000 7f076bfff700 1 -- 172.21.15.112:0/2386807860 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778004020 con 0x7f076c004850 2023-08-27T11:25:39.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.278+0000 7f0789b63700 1 -- 172.21.15.112:0/2386807860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07840b4270 con 0x7f076c004850 2023-08-27T11:25:39.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.279+0000 7f0789b63700 1 -- 172.21.15.112:0/2386807860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0784067a20 con 0x7f076c004850 2023-08-27T11:25:39.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.279+0000 7f076bfff700 1 -- 172.21.15.112:0/2386807860 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f0778004180 con 0x7f076c004850 2023-08-27T11:25:39.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.280+0000 7f076bfff700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f077006a320 0x7f077006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:39.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.280+0000 7f076bfff700 1 -- 172.21.15.112:0/2386807860 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0778088b40 con 0x7f076c004850 2023-08-27T11:25:39.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.280+0000 7f0782ffd700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f077006a320 0x7f077006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:39.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.280+0000 7f0782ffd700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f077006a320 0x7f077006c7e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0774008fd0 tx=0x7f077400a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:39.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.284+0000 7f076bfff700 1 -- 172.21.15.112:0/2386807860 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0778054220 con 0x7f076c004850 2023-08-27T11:25:39.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:39.476+0000 7f0789b63700 1 -- 172.21.15.112:0/2386807860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f078406ded0 con 0x7f076c004850 2023-08-27T11:25:40.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.390+0000 7f076bfff700 1 -- 172.21.15.112:0/2386807860 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) v1 ==== 138+0+0 (secure 0 0 0) 0x7f07780220b0 con 0x7f076c004850 2023-08-27T11:25:40.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.395+0000 7f0789b63700 1 -- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f077006a320 msgr2=0x7f077006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:40.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.395+0000 7f0789b63700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f077006a320 0x7f077006c7e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0774008fd0 tx=0x7f077400a000 comp rx=0 tx=0).stop 2023-08-27T11:25:40.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.395+0000 7f0789b63700 1 -- 172.21.15.112:0/2386807860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f076c004850 msgr2=0x7f07840ae9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:40.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.395+0000 7f0789b63700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f076c004850 0x7f07840ae9e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0778011ca0 tx=0x7f077800f520 comp rx=0 tx=0).stop 2023-08-27T11:25:40.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.395+0000 7f0789b63700 1 -- 172.21.15.112:0/2386807860 shutdown_connections 2023-08-27T11:25:40.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.395+0000 7f0789b63700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f077006a320 0x7f077006c7e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:40.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.395+0000 7f0789b63700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07840b3e20 0x7f0784070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:40.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.395+0000 7f0789b63700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f076c004850 0x7f07840ae9e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:40.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.395+0000 7f0789b63700 1 --2- 172.21.15.112:0/2386807860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07840b5240 0x7f078415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:40.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.395+0000 7f0789b63700 1 -- 172.21.15.112:0/2386807860 >> 172.21.15.112:0/2386807860 conn(0x7f07840b1be0 msgr2=0x7f07840ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:40.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.396+0000 7f0789b63700 1 -- 172.21.15.112:0/2386807860 shutdown_connections 2023-08-27T11:25:40.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.396+0000 7f0789b63700 1 -- 172.21.15.112:0/2386807860 wait complete. 2023-08-27T11:25:40.400 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2023-08-27T11:25:40.413 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:25:40.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.544+0000 7f5808f82700 1 Processor -- start 2023-08-27T11:25:40.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.544+0000 7f5808f82700 1 -- start start 2023-08-27T11:25:40.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f5808f82700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f58041437c0 0x7f58040b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:40.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f5808f82700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f580406d5b0 0x7f580406da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:40.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f5808f82700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58040cfd50 con 0x7f5804143ba0 2023-08-27T11:25:40.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f5808f82700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580406df80 con 0x7f58041437c0 2023-08-27T11:25:40.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f5808f82700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58040cf140 con 0x7f580406d5b0 2023-08-27T11:25:40.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f580259c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f58041437c0 0x7f58040b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:40.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f580259c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f58041437c0 0x7f58040b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59924/0 (socket says 172.21.15.112:59924) 2023-08-27T11:25:40.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f580259c700 1 -- 172.21.15.112:0/4074177409 learned_addr learned my addr 172.21.15.112:0/4074177409 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:40.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f580259c700 1 -- 172.21.15.112:0/4074177409 >> v2:172.21.15.138:3300/0 conn(0x7f580406d5b0 msgr2=0x7f580406da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:40.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f5802d9d700 1 --2- 172.21.15.112:0/4074177409 >> v2:172.21.15.138:3300/0 conn(0x7f580406d5b0 0x7f580406da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:40.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f580259c700 1 --2- 172.21.15.112:0/4074177409 >> v2:172.21.15.138:3300/0 conn(0x7f580406d5b0 0x7f580406da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:40.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f580259c700 1 -- 172.21.15.112:0/4074177409 >> v1:172.21.15.138:6789/0 conn(0x7f5804143ba0 legacy=0x7f58040ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:40.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f580259c700 1 -- 172.21.15.112:0/4074177409 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58040cf420 con 0x7f58041437c0 2023-08-27T11:25:40.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f5802d9d700 1 --2- 172.21.15.112:0/4074177409 >> v2:172.21.15.138:3300/0 conn(0x7f580406d5b0 0x7f580406da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:40.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.545+0000 7f580259c700 1 --2- 172.21.15.112:0/4074177409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58041437c0 0x7f58040b4670 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f58040cfcc0 tx=0x7f57f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124b3338c4f34de5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:40.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.546+0000 7f580159a700 1 -- 172.21.15.112:0/4074177409 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4004230 con 0x7f58041437c0 2023-08-27T11:25:40.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.546+0000 7f580159a700 1 -- 172.21.15.112:0/4074177409 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f4004390 con 0x7f58041437c0 2023-08-27T11:25:40.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.546+0000 7f580159a700 1 -- 172.21.15.112:0/4074177409 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f40036a0 con 0x7f58041437c0 2023-08-27T11:25:40.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.546+0000 7f5808f82700 1 -- 172.21.15.112:0/4074177409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58041437c0 msgr2=0x7f58040b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:40.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.546+0000 7f5808f82700 1 --2- 172.21.15.112:0/4074177409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58041437c0 0x7f58040b4670 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f58040cfcc0 tx=0x7f57f4009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:40.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.546+0000 7f5808f82700 1 -- 172.21.15.112:0/4074177409 shutdown_connections 2023-08-27T11:25:40.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.546+0000 7f5808f82700 1 --2- 172.21.15.112:0/4074177409 >> v2:172.21.15.138:3300/0 conn(0x7f580406d5b0 0x7f580406da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:40.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.546+0000 7f5808f82700 1 --2- 172.21.15.112:0/4074177409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58041437c0 0x7f58040b4670 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:40.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.546+0000 7f5808f82700 1 -- 172.21.15.112:0/4074177409 >> 172.21.15.112:0/4074177409 conn(0x7f58040b1e90 msgr2=0x7f58040b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:40.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.546+0000 7f5808f82700 1 -- 172.21.15.112:0/4074177409 shutdown_connections 2023-08-27T11:25:40.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.546+0000 7f5808f82700 1 -- 172.21.15.112:0/4074177409 wait complete. 2023-08-27T11:25:40.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.548+0000 7f5808f82700 1 Processor -- start 2023-08-27T11:25:40.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.548+0000 7f5808f82700 1 -- start start 2023-08-27T11:25:40.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.548+0000 7f5808f82700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580406d5b0 0x7f580413ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:40.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.548+0000 7f5808f82700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58041437c0 0x7f5804159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:40.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.548+0000 7f5808f82700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5804143ba0 0x7f58040cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:40.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.548+0000 7f5808f82700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58040cd070 con 0x7f5804143ba0 2023-08-27T11:25:40.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.548+0000 7f5808f82700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58040c9f10 con 0x7f580406d5b0 2023-08-27T11:25:40.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.548+0000 7f5808f82700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58040ca1f0 con 0x7f58041437c0 2023-08-27T11:25:40.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f5802d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5804143ba0 0x7f58040cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:40.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f5802d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5804143ba0 0x7f58040cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59934/0 (socket says 172.21.15.112:59934) 2023-08-27T11:25:40.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f5802d9d700 1 -- 172.21.15.112:0/690840658 learned_addr learned my addr 172.21.15.112:0/690840658 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:40.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f5801d9b700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58041437c0 0x7f5804159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:40.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f580259c700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580406d5b0 0x7f580413ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:40.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f5802d9d700 1 -- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58041437c0 msgr2=0x7f5804159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:40.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f5802d9d700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58041437c0 0x7f5804159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:40.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f5802d9d700 1 -- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580406d5b0 msgr2=0x7f580413ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:40.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f5802d9d700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580406d5b0 0x7f580413ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:40.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f5802d9d700 1 -- 172.21.15.112:0/690840658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f4009550 con 0x7f5804143ba0 2023-08-27T11:25:40.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f5801d9b700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58041437c0 0x7f5804159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:40.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f5802d9d700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5804143ba0 0x7f58040cc160 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00cef0 tx=0x7f57fc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:40.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f57f2ffd700 1 -- 172.21.15.112:0/690840658 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc00ae50 con 0x7f5804143ba0 2023-08-27T11:25:40.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f57f2ffd700 1 -- 172.21.15.112:0/690840658 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57fc015440 con 0x7f5804143ba0 2023-08-27T11:25:40.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f57f2ffd700 1 -- 172.21.15.112:0/690840658 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0157f0 con 0x7f5804143ba0 2023-08-27T11:25:40.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f5808f82700 1 -- 172.21.15.112:0/690840658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58040ca4d0 con 0x7f5804143ba0 2023-08-27T11:25:40.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.549+0000 7f5808f82700 1 -- 172.21.15.112:0/690840658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58040caa90 con 0x7f5804143ba0 2023-08-27T11:25:40.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.550+0000 7f5808f82700 1 -- 172.21.15.112:0/690840658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5804067e90 con 0x7f5804143ba0 2023-08-27T11:25:40.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.551+0000 7f57f2ffd700 1 -- 172.21.15.112:0/690840658 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f57fc00d430 con 0x7f5804143ba0 2023-08-27T11:25:40.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.551+0000 7f57f2ffd700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f57e406a320 0x7f57e406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:40.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.551+0000 7f57f2ffd700 1 -- 172.21.15.112:0/690840658 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f57fc088e80 con 0x7f5804143ba0 2023-08-27T11:25:40.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.551+0000 7f580259c700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f57e406a320 0x7f57e406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:40.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.552+0000 7f580259c700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f57e406a320 0x7f57e406c7e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f57f4000c00 tx=0x7f57f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:40.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.556+0000 7f57f2ffd700 1 -- 172.21.15.112:0/690840658 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f57fc0542e0 con 0x7f5804143ba0 2023-08-27T11:25:40.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:40.746+0000 7f5808f82700 1 -- 172.21.15.112:0/690840658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5804067a20 con 0x7f5804143ba0 2023-08-27T11:25:41.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.393+0000 7f57f2ffd700 1 -- 172.21.15.112:0/690840658 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) v1 ==== 170+0+0 (secure 0 0 0) 0x7f57fc01a020 con 0x7f5804143ba0 2023-08-27T11:25:41.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.397+0000 7f5808f82700 1 -- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f57e406a320 msgr2=0x7f57e406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:41.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.397+0000 7f5808f82700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f57e406a320 0x7f57e406c7e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f57f4000c00 tx=0x7f57f4008030 comp rx=0 tx=0).stop 2023-08-27T11:25:41.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.397+0000 7f5808f82700 1 -- 172.21.15.112:0/690840658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5804143ba0 msgr2=0x7f58040cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:41.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.397+0000 7f5808f82700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5804143ba0 0x7f58040cc160 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00cef0 tx=0x7f57fc00a340 comp rx=0 tx=0).stop 2023-08-27T11:25:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.397+0000 7f5808f82700 1 -- 172.21.15.112:0/690840658 shutdown_connections 2023-08-27T11:25:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.397+0000 7f5808f82700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f57e406a320 0x7f57e406c7e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.397+0000 7f5808f82700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5804143ba0 0x7f58040cc160 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.397+0000 7f5808f82700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580406d5b0 0x7f580413ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:41.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.397+0000 7f5808f82700 1 --2- 172.21.15.112:0/690840658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58041437c0 0x7f5804159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:41.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.397+0000 7f5808f82700 1 -- 172.21.15.112:0/690840658 >> 172.21.15.112:0/690840658 conn(0x7f58040b1e90 msgr2=0x7f5804140790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:41.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.401+0000 7f5808f82700 1 -- 172.21.15.112:0/690840658 shutdown_connections 2023-08-27T11:25:41.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.401+0000 7f5808f82700 1 -- 172.21.15.112:0/690840658 wait complete. 2023-08-27T11:25:41.410 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2023-08-27T11:25:41.424 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:25:41.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.548+0000 7fc7e50b5700 1 Processor -- start 2023-08-27T11:25:41.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.548+0000 7fc7e50b5700 1 -- start start 2023-08-27T11:25:41.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.549+0000 7fc7e50b5700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc7e00b3440 0x7fc7e015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:41.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.549+0000 7fc7e50b5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7e015e6e0 0x7fc7e006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:41.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.549+0000 7fc7e50b5700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e00b5070 con 0x7fc7e0146c20 2023-08-27T11:25:41.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.549+0000 7fc7e50b5700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e0147000 con 0x7fc7e00b3440 2023-08-27T11:25:41.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.549+0000 7fc7e50b5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e0161740 con 0x7fc7e015e6e0 2023-08-27T11:25:41.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.550+0000 7fc7de59c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc7e00b3440 0x7fc7e015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:41.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.550+0000 7fc7df59e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7e015e6e0 0x7fc7e006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:41.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.550+0000 7fc7de59c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc7e00b3440 0x7fc7e015e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:60366/0 (socket says 172.21.15.112:60366) 2023-08-27T11:25:41.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.550+0000 7fc7df59e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7e015e6e0 0x7fc7e006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:43364/0 (socket says 172.21.15.112:43364) 2023-08-27T11:25:41.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.550+0000 7fc7de59c700 1 -- 172.21.15.112:0/881708868 learned_addr learned my addr 172.21.15.112:0/881708868 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:41.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.550+0000 7fc7ded9d700 1 --1- 172.21.15.112:0/881708868 >> v1:172.21.15.138:6789/0 conn(0x7fc7e0146c20 0x7fc7e0160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:33612/0 (socket says 172.21.15.112:33612) 2023-08-27T11:25:41.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.550+0000 7fc7de59c700 1 -- 172.21.15.112:0/881708868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7e015e6e0 msgr2=0x7fc7e006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.550+0000 7fc7de59c700 1 --2- 172.21.15.112:0/881708868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7e015e6e0 0x7fc7e006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.550+0000 7fc7de59c700 1 -- 172.21.15.112:0/881708868 >> v1:172.21.15.138:6789/0 conn(0x7fc7e0146c20 legacy=0x7fc7e0160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.550+0000 7fc7de59c700 1 -- 172.21.15.112:0/881708868 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e0161a20 con 0x7fc7e00b3440 2023-08-27T11:25:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.550+0000 7fc7df59e700 1 --2- 172.21.15.112:0/881708868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7e015e6e0 0x7fc7e006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.550+0000 7fc7de59c700 1 --2- 172.21.15.112:0/881708868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e00b3440 0x7fc7e015e1a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00098f0 tx=0x7fc7d0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98dc69d879e2ca6c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.550+0000 7fc7ddd9b700 1 -- 172.21.15.112:0/881708868 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0011830 con 0x7fc7e00b3440 2023-08-27T11:25:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.551+0000 7fc7ddd9b700 1 -- 172.21.15.112:0/881708868 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d0011990 con 0x7fc7e00b3440 2023-08-27T11:25:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.551+0000 7fc7ddd9b700 1 -- 172.21.15.112:0/881708868 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0011d20 con 0x7fc7e00b3440 2023-08-27T11:25:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.551+0000 7fc7e50b5700 1 -- 172.21.15.112:0/881708868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e00b3440 msgr2=0x7fc7e015e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.551+0000 7fc7e50b5700 1 --2- 172.21.15.112:0/881708868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e00b3440 0x7fc7e015e1a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00098f0 tx=0x7fc7d0009c00 comp rx=0 tx=0).stop 2023-08-27T11:25:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.551+0000 7fc7e50b5700 1 -- 172.21.15.112:0/881708868 shutdown_connections 2023-08-27T11:25:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.551+0000 7fc7e50b5700 1 --2- 172.21.15.112:0/881708868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7e015e6e0 0x7fc7e006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:41.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.551+0000 7fc7e50b5700 1 --2- 172.21.15.112:0/881708868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e00b3440 0x7fc7e015e1a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:41.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.551+0000 7fc7e50b5700 1 -- 172.21.15.112:0/881708868 >> 172.21.15.112:0/881708868 conn(0x7fc7e00b1e90 msgr2=0x7fc7e00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:41.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.551+0000 7fc7e50b5700 1 -- 172.21.15.112:0/881708868 shutdown_connections 2023-08-27T11:25:41.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.551+0000 7fc7e50b5700 1 -- 172.21.15.112:0/881708868 wait complete. 2023-08-27T11:25:41.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.553+0000 7fc7e50b5700 1 Processor -- start 2023-08-27T11:25:41.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.553+0000 7fc7e50b5700 1 -- start start 2023-08-27T11:25:41.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.553+0000 7fc7e50b5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e00b3440 0x7fc7e0145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:41.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.553+0000 7fc7e50b5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7e0146c20 0x7fc7e015aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:41.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.553+0000 7fc7e50b5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e015e6e0 0x7fc7e0160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:41.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.553+0000 7fc7e50b5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e01440b0 con 0x7fc7e00b3440 2023-08-27T11:25:41.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.553+0000 7fc7e50b5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e0144ed0 con 0x7fc7e015e6e0 2023-08-27T11:25:41.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.553+0000 7fc7e50b5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e01458a0 con 0x7fc7e0146c20 2023-08-27T11:25:41.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.553+0000 7fc7ded9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e00b3440 0x7fc7e0145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:41.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.553+0000 7fc7ded9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e00b3440 0x7fc7e0145360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59944/0 (socket says 172.21.15.112:59944) 2023-08-27T11:25:41.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.553+0000 7fc7ded9d700 1 -- 172.21.15.112:0/1570465110 learned_addr learned my addr 172.21.15.112:0/1570465110 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:41.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.554+0000 7fc7df59e700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e015e6e0 0x7fc7e0160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:41.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.554+0000 7fc7de59c700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7e0146c20 0x7fc7e015aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:41.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.554+0000 7fc7ded9d700 1 -- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7e0146c20 msgr2=0x7fc7e015aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:41.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.554+0000 7fc7ded9d700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7e0146c20 0x7fc7e015aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:41.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.554+0000 7fc7ded9d700 1 -- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e015e6e0 msgr2=0x7fc7e0160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:41.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.554+0000 7fc7ded9d700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e015e6e0 0x7fc7e0160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:41.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.554+0000 7fc7ded9d700 1 -- 172.21.15.112:0/1570465110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d8009a70 con 0x7fc7e00b3440 2023-08-27T11:25:41.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.554+0000 7fc7ded9d700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e00b3440 0x7fc7e0145360 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800c550 tx=0x7fc7c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:41.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.554+0000 7fc7d77fe700 1 -- 172.21.15.112:0/1570465110 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8005cb0 con 0x7fc7e00b3440 2023-08-27T11:25:41.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.554+0000 7fc7d77fe700 1 -- 172.21.15.112:0/1570465110 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c8005e10 con 0x7fc7e00b3440 2023-08-27T11:25:41.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.554+0000 7fc7e50b5700 1 -- 172.21.15.112:0/1570465110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d0009550 con 0x7fc7e00b3440 2023-08-27T11:25:41.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.554+0000 7fc7d77fe700 1 -- 172.21.15.112:0/1570465110 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c80072d0 con 0x7fc7e00b3440 2023-08-27T11:25:41.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.554+0000 7fc7e50b5700 1 -- 172.21.15.112:0/1570465110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e01442c0 con 0x7fc7e00b3440 2023-08-27T11:25:41.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.555+0000 7fc7e50b5700 1 -- 172.21.15.112:0/1570465110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e0067e90 con 0x7fc7e00b3440 2023-08-27T11:25:41.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.556+0000 7fc7d77fe700 1 -- 172.21.15.112:0/1570465110 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fc7c8007430 con 0x7fc7e00b3440 2023-08-27T11:25:41.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.556+0000 7fc7d77fe700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7c006a320 0x7fc7c006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:41.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.556+0000 7fc7d77fe700 1 -- 172.21.15.112:0/1570465110 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fc7c8087f80 con 0x7fc7e00b3440 2023-08-27T11:25:41.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.556+0000 7fc7de59c700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7c006a320 0x7fc7c006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:41.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.557+0000 7fc7de59c700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7c006a320 0x7fc7c006c7e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009c40 tx=0x7fc7d0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:41.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.560+0000 7fc7d77fe700 1 -- 172.21.15.112:0/1570465110 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc7c8053500 con 0x7fc7e00b3440 2023-08-27T11:25:41.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:41.792+0000 7fc7e50b5700 1 -- 172.21.15.112:0/1570465110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc7e00b5070 con 0x7fc7e00b3440 2023-08-27T11:25:42.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.425+0000 7fc7d77fe700 1 -- 172.21.15.112:0/1570465110 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v44) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc7e00b5070 con 0x7fc7e00b3440 2023-08-27T11:25:42.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.434+0000 7fc7e50b5700 1 -- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7c006a320 msgr2=0x7fc7c006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:42.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.434+0000 7fc7e50b5700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7c006a320 0x7fc7c006c7e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009c40 tx=0x7fc7d0008030 comp rx=0 tx=0).stop 2023-08-27T11:25:42.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.434+0000 7fc7e50b5700 1 -- 172.21.15.112:0/1570465110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e00b3440 msgr2=0x7fc7e0145360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:42.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.434+0000 7fc7e50b5700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e00b3440 0x7fc7e0145360 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800c550 tx=0x7fc7c800c910 comp rx=0 tx=0).stop 2023-08-27T11:25:42.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.434+0000 7fc7e50b5700 1 -- 172.21.15.112:0/1570465110 shutdown_connections 2023-08-27T11:25:42.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.434+0000 7fc7e50b5700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc7c006a320 0x7fc7c006c7e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.434+0000 7fc7e50b5700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e015e6e0 0x7fc7e0160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.434+0000 7fc7e50b5700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e00b3440 0x7fc7e0145360 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.434+0000 7fc7e50b5700 1 --2- 172.21.15.112:0/1570465110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7e0146c20 0x7fc7e015aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.434+0000 7fc7e50b5700 1 -- 172.21.15.112:0/1570465110 >> 172.21.15.112:0/1570465110 conn(0x7fc7e00b1e90 msgr2=0x7fc7e00b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:42.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.434+0000 7fc7e50b5700 1 -- 172.21.15.112:0/1570465110 shutdown_connections 2023-08-27T11:25:42.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.435+0000 7fc7e50b5700 1 -- 172.21.15.112:0/1570465110 wait complete. 2023-08-27T11:25:42.439 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 13 and data pool 14 2023-08-27T11:25:42.452 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:42.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.571+0000 7f7b00f0c700 1 Processor -- start 2023-08-27T11:25:42.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.571+0000 7f7b00f0c700 1 -- start start 2023-08-27T11:25:42.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.572+0000 7f7b00f0c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7afc0b17e0 0x7f7afc0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:42.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.572+0000 7f7b00f0c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7afc161ce0 0x7f7afc06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:42.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.572+0000 7f7b00f0c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7afc161850 con 0x7f7afc0b1400 2023-08-27T11:25:42.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.572+0000 7f7b00f0c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7afc0cfc10 con 0x7f7afc0b17e0 2023-08-27T11:25:42.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.572+0000 7f7b00f0c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7afc161200 con 0x7f7afc161ce0 2023-08-27T11:25:42.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.572+0000 7f7af9d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7afc0b17e0 0x7f7afc0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.572+0000 7f7af9d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7afc0b17e0 0x7f7afc0cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59966/0 (socket says 172.21.15.112:59966) 2023-08-27T11:25:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.572+0000 7f7af9d9b700 1 -- 172.21.15.112:0/920991924 learned_addr learned my addr 172.21.15.112:0/920991924 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7afad9d700 1 --2- 172.21.15.112:0/920991924 >> v2:172.21.15.138:3300/0 conn(0x7f7afc161ce0 0x7f7afc06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7af9d9b700 1 -- 172.21.15.112:0/920991924 >> v2:172.21.15.138:3300/0 conn(0x7f7afc161ce0 msgr2=0x7f7afc06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7af9d9b700 1 --2- 172.21.15.112:0/920991924 >> v2:172.21.15.138:3300/0 conn(0x7f7afc161ce0 0x7f7afc06f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7af9d9b700 1 -- 172.21.15.112:0/920991924 >> v1:172.21.15.138:6789/0 conn(0x7f7afc0b1400 legacy=0x7f7afc06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7af9d9b700 1 -- 172.21.15.112:0/920991924 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc1614e0 con 0x7f7afc0b17e0 2023-08-27T11:25:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7af9d9b700 1 --2- 172.21.15.112:0/920991924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7afc0b17e0 0x7f7afc0cf5a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7afc06df10 tx=0x7f7aec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9f5dee1325011d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:42.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7af959a700 1 -- 172.21.15.112:0/920991924 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec004230 con 0x7f7afc0b17e0 2023-08-27T11:25:42.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7af959a700 1 -- 172.21.15.112:0/920991924 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec004390 con 0x7f7afc0b17e0 2023-08-27T11:25:42.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7af959a700 1 -- 172.21.15.112:0/920991924 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0036a0 con 0x7f7afc0b17e0 2023-08-27T11:25:42.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7b00f0c700 1 -- 172.21.15.112:0/920991924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7afc0b17e0 msgr2=0x7f7afc0cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:42.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7b00f0c700 1 --2- 172.21.15.112:0/920991924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7afc0b17e0 0x7f7afc0cf5a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7afc06df10 tx=0x7f7aec009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7b00f0c700 1 -- 172.21.15.112:0/920991924 shutdown_connections 2023-08-27T11:25:42.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7b00f0c700 1 --2- 172.21.15.112:0/920991924 >> v2:172.21.15.138:3300/0 conn(0x7f7afc161ce0 0x7f7afc06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7b00f0c700 1 --2- 172.21.15.112:0/920991924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7afc0b17e0 0x7f7afc0cf5a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.573+0000 7f7b00f0c700 1 -- 172.21.15.112:0/920991924 >> 172.21.15.112:0/920991924 conn(0x7f7afc0d0000 msgr2=0x7f7afc0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:42.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.574+0000 7f7b00f0c700 1 -- 172.21.15.112:0/920991924 shutdown_connections 2023-08-27T11:25:42.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.574+0000 7f7b00f0c700 1 -- 172.21.15.112:0/920991924 wait complete. 2023-08-27T11:25:42.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.575+0000 7f7b00f0c700 1 Processor -- start 2023-08-27T11:25:42.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.575+0000 7f7b00f0c700 1 -- start start 2023-08-27T11:25:42.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.576+0000 7f7b00f0c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7afc0b1400 0x7f7afc153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:42.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.576+0000 7f7b00f0c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7afc0b17e0 0x7f7afc0c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:42.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.576+0000 7f7b00f0c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7afc161ce0 0x7f7afc1f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:42.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.576+0000 7f7b00f0c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0c8880 con 0x7f7afc0b17e0 2023-08-27T11:25:42.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.576+0000 7f7b00f0c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0c6f10 con 0x7f7afc0b1400 2023-08-27T11:25:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.576+0000 7f7b00f0c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1f4180 con 0x7f7afc161ce0 2023-08-27T11:25:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.576+0000 7f7af9d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7afc0b17e0 0x7f7afc0c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.576+0000 7f7af9d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7afc0b17e0 0x7f7afc0c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59972/0 (socket says 172.21.15.112:59972) 2023-08-27T11:25:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.576+0000 7f7af9d9b700 1 -- 172.21.15.112:0/2367438428 learned_addr learned my addr 172.21.15.112:0/2367438428 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7afa59c700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7afc0b1400 0x7f7afc153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7afad9d700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7afc161ce0 0x7f7afc1f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7af9d9b700 1 -- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7afc161ce0 msgr2=0x7f7afc1f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7af9d9b700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7afc161ce0 0x7f7afc1f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7af9d9b700 1 -- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7afc0b1400 msgr2=0x7f7afc153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7af9d9b700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7afc0b1400 0x7f7afc153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7af9d9b700 1 -- 172.21.15.112:0/2367438428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec009550 con 0x7f7afc0b17e0 2023-08-27T11:25:42.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7afad9d700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7afc161ce0 0x7f7afc1f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:42.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7af9d9b700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7afc0b17e0 0x7f7afc0c69d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f7aec004000 tx=0x7f7aec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:42.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7aeaffd700 1 -- 172.21.15.112:0/2367438428 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec003d80 con 0x7f7afc0b17e0 2023-08-27T11:25:42.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7aeaffd700 1 -- 172.21.15.112:0/2367438428 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec01f070 con 0x7f7afc0b17e0 2023-08-27T11:25:42.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7aeaffd700 1 -- 172.21.15.112:0/2367438428 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec011870 con 0x7f7afc0b17e0 2023-08-27T11:25:42.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7b00f0c700 1 -- 172.21.15.112:0/2367438428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc1f36b0 con 0x7f7afc0b17e0 2023-08-27T11:25:42.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.577+0000 7f7b00f0c700 1 -- 172.21.15.112:0/2367438428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc1f3bc0 con 0x7f7afc0b17e0 2023-08-27T11:25:42.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.578+0000 7f7b00f0c700 1 -- 172.21.15.112:0/2367438428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7afc0c85c0 con 0x7f7afc0b17e0 2023-08-27T11:25:42.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.578+0000 7f7aeaffd700 1 -- 172.21.15.112:0/2367438428 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f7aec021030 con 0x7f7afc0b17e0 2023-08-27T11:25:42.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.579+0000 7f7aeaffd700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7adc06a320 0x7f7adc06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:42.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.579+0000 7f7aeaffd700 1 -- 172.21.15.112:0/2367438428 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7aec089f70 con 0x7f7afc0b17e0 2023-08-27T11:25:42.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.579+0000 7f7afa59c700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7adc06a320 0x7f7adc06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:42.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.579+0000 7f7afa59c700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7adc06a320 0x7f7adc06c7e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7afc1f0d60 tx=0x7f7af0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:42.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.583+0000 7f7aeaffd700 1 -- 172.21.15.112:0/2367438428 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7aec0554e0 con 0x7f7afc0b17e0 2023-08-27T11:25:42.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.773+0000 7f7b00f0c700 1 -- 172.21.15.112:0/2367438428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7afc067a20 con 0x7f7afc0b17e0 2023-08-27T11:25:42.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.774+0000 7f7aeaffd700 1 -- 172.21.15.112:0/2367438428 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18273 (secure 0 0 0) 0x7f7aec058b00 con 0x7f7afc0b17e0 2023-08-27T11:25:42.776 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":76,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:25:41.794397+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:39.495757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:40.765773+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:42.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.776+0000 7f7b00f0c700 1 -- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7adc06a320 msgr2=0x7f7adc06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:42.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.776+0000 7f7b00f0c700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7adc06a320 0x7f7adc06c7e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7afc1f0d60 tx=0x7f7af0008040 comp rx=0 tx=0).stop 2023-08-27T11:25:42.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.776+0000 7f7b00f0c700 1 -- 172.21.15.112:0/2367438428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7afc0b17e0 msgr2=0x7f7afc0c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:42.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.776+0000 7f7b00f0c700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7afc0b17e0 0x7f7afc0c69d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f7aec004000 tx=0x7f7aec004750 comp rx=0 tx=0).stop 2023-08-27T11:25:42.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.777+0000 7f7b00f0c700 1 -- 172.21.15.112:0/2367438428 shutdown_connections 2023-08-27T11:25:42.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.777+0000 7f7b00f0c700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7adc06a320 0x7f7adc06c7e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.777+0000 7f7b00f0c700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7afc161ce0 0x7f7afc1f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.777+0000 7f7b00f0c700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7afc0b1400 0x7f7afc153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.777+0000 7f7b00f0c700 1 --2- 172.21.15.112:0/2367438428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7afc0b17e0 0x7f7afc0c69d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.777+0000 7f7b00f0c700 1 -- 172.21.15.112:0/2367438428 >> 172.21.15.112:0/2367438428 conn(0x7f7afc0d0000 msgr2=0x7f7afc0d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:42.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.777+0000 7f7b00f0c700 1 -- 172.21.15.112:0/2367438428 shutdown_connections 2023-08-27T11:25:42.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.777+0000 7f7b00f0c700 1 -- 172.21.15.112:0/2367438428 wait complete. 2023-08-27T11:25:42.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:42.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.909+0000 7f4ea909e700 1 Processor -- start 2023-08-27T11:25:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.909+0000 7f4ea909e700 1 -- start start 2023-08-27T11:25:42.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea909e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4ea40b1400 0x7f4ea40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:42.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea909e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ea4161ce0 0x7f4ea406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:42.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea909e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea4161850 con 0x7f4ea40b17e0 2023-08-27T11:25:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea909e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea40cfc10 con 0x7f4ea40b1400 2023-08-27T11:25:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea909e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4161200 con 0x7f4ea4161ce0 2023-08-27T11:25:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea2d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4ea40b1400 0x7f4ea40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea259c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4ea40b17e0 0x7f4ea406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54360/0 (socket says 172.21.15.112:54360) 2023-08-27T11:25:42.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea2d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4ea40b1400 0x7f4ea40cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59978/0 (socket says 172.21.15.112:59978) 2023-08-27T11:25:42.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea259c700 1 -- 172.21.15.112:0/4102197895 learned_addr learned my addr 172.21.15.112:0/4102197895 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:42.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea359e700 1 --2- 172.21.15.112:0/4102197895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ea4161ce0 0x7f4ea406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:42.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea2d9d700 1 -- 172.21.15.112:0/4102197895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ea4161ce0 msgr2=0x7f4ea406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:42.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea2d9d700 1 --2- 172.21.15.112:0/4102197895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ea4161ce0 0x7f4ea406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea2d9d700 1 -- 172.21.15.112:0/4102197895 >> v1:172.21.15.112:6789/0 conn(0x7f4ea40b17e0 legacy=0x7f4ea406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:42.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea2d9d700 1 -- 172.21.15.112:0/4102197895 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea41614e0 con 0x7f4ea40b1400 2023-08-27T11:25:42.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea1d9b700 1 -- 172.21.15.112:0/4102197895 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284587499 0 0) 0x7f4ea4161850 con 0x7f4ea40b17e0 2023-08-27T11:25:42.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea359e700 1 --2- 172.21.15.112:0/4102197895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ea4161ce0 0x7f4ea406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:42.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea2d9d700 1 --2- 172.21.15.112:0/4102197895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40b1400 0x7f4ea40cf5a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009840 tx=0x7f4e94009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a258e6826ba9db4b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:42.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea1d9b700 1 -- 172.21.15.112:0/4102197895 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94011830 con 0x7f4ea40b1400 2023-08-27T11:25:42.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea1d9b700 1 -- 172.21.15.112:0/4102197895 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e94011990 con 0x7f4ea40b1400 2023-08-27T11:25:42.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.910+0000 7f4ea1d9b700 1 -- 172.21.15.112:0/4102197895 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94011d20 con 0x7f4ea40b1400 2023-08-27T11:25:42.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.911+0000 7f4ea909e700 1 -- 172.21.15.112:0/4102197895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40b1400 msgr2=0x7f4ea40cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:42.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.911+0000 7f4ea909e700 1 --2- 172.21.15.112:0/4102197895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40b1400 0x7f4ea40cf5a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009840 tx=0x7f4e94009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:42.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.911+0000 7f4ea909e700 1 -- 172.21.15.112:0/4102197895 shutdown_connections 2023-08-27T11:25:42.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.911+0000 7f4ea909e700 1 --2- 172.21.15.112:0/4102197895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ea4161ce0 0x7f4ea406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.911+0000 7f4ea909e700 1 --2- 172.21.15.112:0/4102197895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40b1400 0x7f4ea40cf5a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.911+0000 7f4ea909e700 1 -- 172.21.15.112:0/4102197895 >> 172.21.15.112:0/4102197895 conn(0x7f4ea40d0000 msgr2=0x7f4ea40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:42.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.911+0000 7f4ea909e700 1 -- 172.21.15.112:0/4102197895 shutdown_connections 2023-08-27T11:25:42.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.911+0000 7f4ea909e700 1 -- 172.21.15.112:0/4102197895 wait complete. 2023-08-27T11:25:42.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.913+0000 7f4ea909e700 1 Processor -- start 2023-08-27T11:25:42.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.913+0000 7f4ea909e700 1 -- start start 2023-08-27T11:25:42.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea909e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ea40b1400 0x7f4ea40c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:42.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea909e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40b17e0 0x7f4ea4152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:42.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea909e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea4161ce0 0x7f4ea41f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:42.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea909e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40c8830 con 0x7f4ea40b17e0 2023-08-27T11:25:42.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea909e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4152e30 con 0x7f4ea4161ce0 2023-08-27T11:25:42.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea909e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4153110 con 0x7f4ea40b1400 2023-08-27T11:25:42.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40b17e0 0x7f4ea4152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:42.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40b17e0 0x7f4ea4152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59984/0 (socket says 172.21.15.112:59984) 2023-08-27T11:25:42.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea259c700 1 -- 172.21.15.112:0/1868364106 learned_addr learned my addr 172.21.15.112:0/1868364106 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:42.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea2d9d700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ea40b1400 0x7f4ea40c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:42.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea359e700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea4161ce0 0x7f4ea41f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:42.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea259c700 1 -- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ea40b1400 msgr2=0x7f4ea40c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:42.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea259c700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ea40b1400 0x7f4ea40c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea259c700 1 -- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea4161ce0 msgr2=0x7f4ea41f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:42.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea259c700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea4161ce0 0x7f4ea41f6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:42.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea259c700 1 -- 172.21.15.112:0/1868364106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e940094a0 con 0x7f4ea40b17e0 2023-08-27T11:25:42.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea259c700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40b17e0 0x7f4ea4152450 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800c540 tx=0x7f4e9800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:42.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4e8b7fe700 1 -- 172.21.15.112:0/1868364106 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98004c80 con 0x7f4ea40b17e0 2023-08-27T11:25:42.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4e8b7fe700 1 -- 172.21.15.112:0/1868364106 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e98004de0 con 0x7f4ea40b17e0 2023-08-27T11:25:42.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea909e700 1 -- 172.21.15.112:0/1868364106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea41533f0 con 0x7f4ea40b17e0 2023-08-27T11:25:42.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4e8b7fe700 1 -- 172.21.15.112:0/1868364106 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98007310 con 0x7f4ea40b17e0 2023-08-27T11:25:42.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.914+0000 7f4ea909e700 1 -- 172.21.15.112:0/1868364106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea40c55b0 con 0x7f4ea40b17e0 2023-08-27T11:25:42.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.915+0000 7f4ea909e700 1 -- 172.21.15.112:0/1868364106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea4153910 con 0x7f4ea40b17e0 2023-08-27T11:25:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.916+0000 7f4e8b7fe700 1 -- 172.21.15.112:0/1868364106 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f4e98007470 con 0x7f4ea40b17e0 2023-08-27T11:25:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.916+0000 7f4e8b7fe700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e9006a320 0x7f4e9006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.916+0000 7f4e8b7fe700 1 -- 172.21.15.112:0/1868364106 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e98088720 con 0x7f4ea40b17e0 2023-08-27T11:25:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.916+0000 7f4ea2d9d700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e9006a320 0x7f4e9006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.917+0000 7f4ea2d9d700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e9006a320 0x7f4e9006c7e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4e940090b0 tx=0x7f4e94008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:42.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:42.921+0000 7f4e8b7fe700 1 -- 172.21.15.112:0/1868364106 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4ea4153910 con 0x7f4ea40b17e0 2023-08-27T11:25:43.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.108+0000 7f4ea909e700 1 -- 172.21.15.112:0/1868364106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ea4153910 con 0x7f4ea40b17e0 2023-08-27T11:25:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.109+0000 7f4e8b7fe700 1 -- 172.21.15.112:0/1868364106 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18273 (secure 0 0 0) 0x7f4ea4153910 con 0x7f4ea40b17e0 2023-08-27T11:25:43.111 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:43.111 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":76,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:25:41.794397+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:39.495757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:40.765773+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:43.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.111+0000 7f4ea909e700 1 -- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e9006a320 msgr2=0x7f4e9006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:43.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.111+0000 7f4ea909e700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e9006a320 0x7f4e9006c7e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4e940090b0 tx=0x7f4e94008030 comp rx=0 tx=0).stop 2023-08-27T11:25:43.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.111+0000 7f4ea909e700 1 -- 172.21.15.112:0/1868364106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40b17e0 msgr2=0x7f4ea4152450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:43.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.111+0000 7f4ea909e700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40b17e0 0x7f4ea4152450 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800c540 tx=0x7f4e9800c900 comp rx=0 tx=0).stop 2023-08-27T11:25:43.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.111+0000 7f4ea909e700 1 -- 172.21.15.112:0/1868364106 shutdown_connections 2023-08-27T11:25:43.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.111+0000 7f4ea909e700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e9006a320 0x7f4e9006c7e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:43.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.111+0000 7f4ea909e700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea4161ce0 0x7f4ea41f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.111+0000 7f4ea909e700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ea40b1400 0x7f4ea40c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.111+0000 7f4ea909e700 1 --2- 172.21.15.112:0/1868364106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40b17e0 0x7f4ea4152450 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.111+0000 7f4ea909e700 1 -- 172.21.15.112:0/1868364106 >> 172.21.15.112:0/1868364106 conn(0x7f4ea40d0000 msgr2=0x7f4ea415ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:43.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.111+0000 7f4ea909e700 1 -- 172.21.15.112:0/1868364106 shutdown_connections 2023-08-27T11:25:43.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.111+0000 7f4ea909e700 1 -- 172.21.15.112:0/1868364106 wait complete. 2023-08-27T11:25:43.126 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:25:43.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.249+0000 7f9437fdb700 1 Processor -- start 2023-08-27T11:25:43.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.249+0000 7f9437fdb700 1 -- start start 2023-08-27T11:25:43.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.250+0000 7f9437fdb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f943015ed00 0x7f94300b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:43.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.250+0000 7f9437fdb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f943015f180 0x7f94300b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:43.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.250+0000 7f9437fdb700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943015f750 con 0x7f9430152130 2023-08-27T11:25:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.250+0000 7f9437fdb700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943015fa30 con 0x7f943015ed00 2023-08-27T11:25:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.250+0000 7f9437fdb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94300b60a0 con 0x7f943015f180 2023-08-27T11:25:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9436578700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f943015f180 0x7f94300b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9435576700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f943015ed00 0x7f94300b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9435d77700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9430152130 0x7f943015c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:33664/0 (socket says 172.21.15.112:33664) 2023-08-27T11:25:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9436578700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f943015f180 0x7f94300b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:43416/0 (socket says 172.21.15.112:43416) 2023-08-27T11:25:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9435576700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f943015ed00 0x7f94300b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:60418/0 (socket says 172.21.15.112:60418) 2023-08-27T11:25:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9435d77700 1 -- 172.21.15.112:0/143796387 learned_addr learned my addr 172.21.15.112:0/143796387 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9436578700 1 -- 172.21.15.112:0/143796387 >> v2:172.21.15.138:3300/0 conn(0x7f943015ed00 msgr2=0x7f94300b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9436578700 1 --2- 172.21.15.112:0/143796387 >> v2:172.21.15.138:3300/0 conn(0x7f943015ed00 0x7f94300b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9436578700 1 -- 172.21.15.112:0/143796387 >> v1:172.21.15.138:6789/0 conn(0x7f9430152130 legacy=0x7f943015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9436578700 1 -- 172.21.15.112:0/143796387 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94300b6380 con 0x7f943015f180 2023-08-27T11:25:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9435576700 1 --2- 172.21.15.112:0/143796387 >> v2:172.21.15.138:3300/0 conn(0x7f943015ed00 0x7f94300b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9436578700 1 --2- 172.21.15.112:0/143796387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f943015f180 0x7f94300b4230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9428009d90 tx=0x7f942800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c18de3198e3d08e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9434d75700 1 -- 172.21.15.112:0/143796387 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942800e070 con 0x7f943015f180 2023-08-27T11:25:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9434d75700 1 -- 172.21.15.112:0/143796387 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942800bb20 con 0x7f943015f180 2023-08-27T11:25:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.251+0000 7f9434d75700 1 -- 172.21.15.112:0/143796387 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428012430 con 0x7f943015f180 2023-08-27T11:25:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.252+0000 7f9437fdb700 1 -- 172.21.15.112:0/143796387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f943015f180 msgr2=0x7f94300b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.252+0000 7f9437fdb700 1 --2- 172.21.15.112:0/143796387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f943015f180 0x7f94300b4230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9428009d90 tx=0x7f942800b330 comp rx=0 tx=0).stop 2023-08-27T11:25:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.252+0000 7f9437fdb700 1 -- 172.21.15.112:0/143796387 shutdown_connections 2023-08-27T11:25:43.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.252+0000 7f9437fdb700 1 --2- 172.21.15.112:0/143796387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f943015f180 0x7f94300b4230 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:43.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.252+0000 7f9437fdb700 1 --2- 172.21.15.112:0/143796387 >> v2:172.21.15.138:3300/0 conn(0x7f943015ed00 0x7f94300b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:43.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.252+0000 7f9437fdb700 1 -- 172.21.15.112:0/143796387 >> 172.21.15.112:0/143796387 conn(0x7f9430066010 msgr2=0x7f94300663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:43.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.252+0000 7f9437fdb700 1 -- 172.21.15.112:0/143796387 shutdown_connections 2023-08-27T11:25:43.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.252+0000 7f9437fdb700 1 -- 172.21.15.112:0/143796387 wait complete. 2023-08-27T11:25:43.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.253+0000 7f9437fdb700 1 Processor -- start 2023-08-27T11:25:43.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.253+0000 7f9437fdb700 1 -- start start 2023-08-27T11:25:43.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9437fdb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9430152130 0x7f94301f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:43.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9437fdb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f943015ed00 0x7f9430071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:43.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9437fdb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f943015f180 0x7f943015a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:43.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9437fdb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94301f2ad0 con 0x7f943015ed00 2023-08-27T11:25:43.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9437fdb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9430071bb0 con 0x7f9430152130 2023-08-27T11:25:43.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9437fdb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9430152880 con 0x7f943015f180 2023-08-27T11:25:43.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9435576700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f943015ed00 0x7f9430071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:43.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9435576700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f943015ed00 0x7f9430071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59998/0 (socket says 172.21.15.112:59998) 2023-08-27T11:25:43.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9435576700 1 -- 172.21.15.112:0/1136381328 learned_addr learned my addr 172.21.15.112:0/1136381328 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:43.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9436578700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f943015f180 0x7f943015a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:43.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9435d77700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9430152130 0x7f94301f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:43.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9435576700 1 -- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f943015f180 msgr2=0x7f943015a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:43.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9435576700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f943015f180 0x7f943015a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:43.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9435576700 1 -- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9430152130 msgr2=0x7f94301f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:43.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9435576700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9430152130 0x7f94301f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:43.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.254+0000 7f9435576700 1 -- 172.21.15.112:0/1136381328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0094a0 con 0x7f943015ed00 2023-08-27T11:25:43.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.255+0000 7f9435576700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f943015ed00 0x7f9430071280 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f942c00ebe0 tx=0x7f942c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:43.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.255+0000 7f94267fc700 1 -- 172.21.15.112:0/1136381328 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c00c850 con 0x7f943015ed00 2023-08-27T11:25:43.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.255+0000 7f94267fc700 1 -- 172.21.15.112:0/1136381328 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c00c9b0 con 0x7f943015ed00 2023-08-27T11:25:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.255+0000 7f9437fdb700 1 -- 172.21.15.112:0/1136381328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9428009a70 con 0x7f943015ed00 2023-08-27T11:25:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.255+0000 7f94267fc700 1 -- 172.21.15.112:0/1136381328 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c00cd40 con 0x7f943015ed00 2023-08-27T11:25:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.255+0000 7f9437fdb700 1 -- 172.21.15.112:0/1136381328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9430152e40 con 0x7f943015ed00 2023-08-27T11:25:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.256+0000 7f9437fdb700 1 -- 172.21.15.112:0/1136381328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94300b4e70 con 0x7f943015ed00 2023-08-27T11:25:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.256+0000 7f94267fc700 1 -- 172.21.15.112:0/1136381328 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f942c013470 con 0x7f943015ed00 2023-08-27T11:25:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.257+0000 7f94267fc700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f941406a320 0x7f941406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.257+0000 7f94267fc700 1 -- 172.21.15.112:0/1136381328 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f942c088990 con 0x7f943015ed00 2023-08-27T11:25:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.257+0000 7f9435d77700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f941406a320 0x7f941406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.257+0000 7f9435d77700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f941406a320 0x7f941406c7e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9420005910 tx=0x7f9420009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.261+0000 7f94267fc700 1 -- 172.21.15.112:0/1136381328 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f942c053db0 con 0x7f943015ed00 2023-08-27T11:25:43.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:43.497+0000 7f9437fdb700 1 -- 172.21.15.112:0/1136381328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9430067a20 con 0x7f943015ed00 2023-08-27T11:25:44.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.437+0000 7f94267fc700 1 -- 172.21.15.112:0/1136381328 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) v1 ==== 118+0+0 (secure 0 0 0) 0x7f942c00cea0 con 0x7f943015ed00 2023-08-27T11:25:44.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.440+0000 7f9437fdb700 1 -- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f941406a320 msgr2=0x7f941406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:44.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.440+0000 7f9437fdb700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f941406a320 0x7f941406c7e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9420005910 tx=0x7f9420009040 comp rx=0 tx=0).stop 2023-08-27T11:25:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.440+0000 7f9437fdb700 1 -- 172.21.15.112:0/1136381328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f943015ed00 msgr2=0x7f9430071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.440+0000 7f9437fdb700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f943015ed00 0x7f9430071280 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f942c00ebe0 tx=0x7f942c00eef0 comp rx=0 tx=0).stop 2023-08-27T11:25:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.440+0000 7f9437fdb700 1 -- 172.21.15.112:0/1136381328 shutdown_connections 2023-08-27T11:25:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.440+0000 7f9437fdb700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f941406a320 0x7f941406c7e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.440+0000 7f9437fdb700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f943015f180 0x7f943015a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:44.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.440+0000 7f9437fdb700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9430152130 0x7f94301f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:44.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.440+0000 7f9437fdb700 1 --2- 172.21.15.112:0/1136381328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f943015ed00 0x7f9430071280 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:44.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.440+0000 7f9437fdb700 1 -- 172.21.15.112:0/1136381328 >> 172.21.15.112:0/1136381328 conn(0x7f9430066010 msgr2=0x7f943015b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:44.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.440+0000 7f9437fdb700 1 -- 172.21.15.112:0/1136381328 shutdown_connections 2023-08-27T11:25:44.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.440+0000 7f9437fdb700 1 -- 172.21.15.112:0/1136381328 wait complete. 2023-08-27T11:25:44.457 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:25:44.457 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:25:44.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.583+0000 7f97e4e43700 1 Processor -- start 2023-08-27T11:25:44.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.584+0000 7f97e4e43700 1 -- start start 2023-08-27T11:25:44.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.584+0000 7f97e4e43700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97e0146c20 0x7f97e0160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:44.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.584+0000 7f97e4e43700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e00b5070 con 0x7f97e006f790 2023-08-27T11:25:44.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.584+0000 7f97e4e43700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e0147000 con 0x7f97e00b3440 2023-08-27T11:25:44.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.584+0000 7f97e4e43700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e01614a0 con 0x7f97e0146c20 2023-08-27T11:25:44.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97ded9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f97e006f790 0x7f97e00b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54390/0 (socket says 172.21.15.112:54390) 2023-08-27T11:25:44.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97ded9d700 1 -- 172.21.15.112:0/3179902520 learned_addr learned my addr 172.21.15.112:0/3179902520 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:44.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584835588 0 0) 0x7f97d8001490 con 0x7f97e006f790 2023-08-27T11:25:44.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97de59c700 1 --2- 172.21.15.112:0/3179902520 >> v2:172.21.15.138:3300/0 conn(0x7f97e0146c20 0x7f97e0160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:44.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97c80035b0 con 0x7f97e006f790 2023-08-27T11:25:44.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3883838888 0 0) 0x7f97c80035b0 con 0x7f97e006f790 2023-08-27T11:25:44.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 >> v2:172.21.15.138:3300/0 conn(0x7f97e0146c20 msgr2=0x7f97e0160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:44.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 --2- 172.21.15.112:0/3179902520 >> v2:172.21.15.138:3300/0 conn(0x7f97e0146c20 0x7f97e0160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:44.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 >> v1:172.21.15.138:6789/0 conn(0x7f97e00b3440 legacy=0x7f97e015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:44.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e0161740 con 0x7f97e006f790 2023-08-27T11:25:44.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f97d8001b80 con 0x7f97e006f790 2023-08-27T11:25:44.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 >> v1:172.21.15.112:6789/0 conn(0x7f97e006f790 legacy=0x7f97e00b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:44.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 --2- 172.21.15.112:0/3179902520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c8003a00 0x7f97c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:44.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 --2- 172.21.15.112:0/3179902520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8004850 0x7f97c8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:44.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 --2- 172.21.15.112:0/3179902520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c8005210 0x7f97c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:44.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e01614a0 con 0x7f97c8003a00 2023-08-27T11:25:44.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c800c1b0 con 0x7f97c8004850 2023-08-27T11:25:44.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c800c370 con 0x7f97c8005210 2023-08-27T11:25:44.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97de59c700 1 --2- 172.21.15.112:0/3179902520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c8003a00 0x7f97c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:44.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97ddd9b700 1 --2- 172.21.15.112:0/3179902520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8004850 0x7f97c8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:44.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.585+0000 7f97ded9d700 1 --2- 172.21.15.112:0/3179902520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c8005210 0x7f97c800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:44.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97de59c700 1 -- 172.21.15.112:0/3179902520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c8005210 msgr2=0x7f97c800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:44.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97de59c700 1 --2- 172.21.15.112:0/3179902520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c8005210 0x7f97c800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:44.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97de59c700 1 -- 172.21.15.112:0/3179902520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8004850 msgr2=0x7f97c8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:44.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97de59c700 1 --2- 172.21.15.112:0/3179902520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8004850 0x7f97c8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:44.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97de59c700 1 -- 172.21.15.112:0/3179902520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c80035b0 con 0x7f97c8003a00 2023-08-27T11:25:44.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97ded9d700 1 -- 172.21.15.112:0/3179902520 reap_dead start 2023-08-27T11:25:44.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97de59c700 1 --2- 172.21.15.112:0/3179902520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c8003a00 0x7f97c8003e20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009b60 tx=0x7f97c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe88e4c7a0e203f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:44.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d400e6c0 con 0x7f97c8003a00 2023-08-27T11:25:44.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d400e820 con 0x7f97c8003a00 2023-08-27T11:25:44.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97dd59a700 1 -- 172.21.15.112:0/3179902520 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d400ebb0 con 0x7f97c8003a00 2023-08-27T11:25:44.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97e4e43700 1 -- 172.21.15.112:0/3179902520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c8003a00 msgr2=0x7f97c8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:44.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97e4e43700 1 --2- 172.21.15.112:0/3179902520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c8003a00 0x7f97c8003e20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009b60 tx=0x7f97c8005880 comp rx=0 tx=0).stop 2023-08-27T11:25:44.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97e4e43700 1 -- 172.21.15.112:0/3179902520 shutdown_connections 2023-08-27T11:25:44.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97e4e43700 1 --2- 172.21.15.112:0/3179902520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c8003a00 0x7f97c8003e20 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:44.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97e4e43700 1 -- 172.21.15.112:0/3179902520 >> 172.21.15.112:0/3179902520 conn(0x7f97e00b1e90 msgr2=0x7f97e00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:44.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97e4e43700 1 -- 172.21.15.112:0/3179902520 shutdown_connections 2023-08-27T11:25:44.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.586+0000 7f97e4e43700 1 -- 172.21.15.112:0/3179902520 wait complete. 2023-08-27T11:25:44.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.588+0000 7f97e4e43700 1 Processor -- start 2023-08-27T11:25:44.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.588+0000 7f97e4e43700 1 -- start start 2023-08-27T11:25:44.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.588+0000 7f97e4e43700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c8003a00 0x7f97e0071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:44.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.588+0000 7f97e4e43700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97e0140570 0x7f97e01409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:44.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.588+0000 7f97e4e43700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e0142b70 0x7f97e0142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:44.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97e4e43700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e0140f30 con 0x7f97e0142b70 2023-08-27T11:25:44.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97e4e43700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e01f33b0 con 0x7f97e0140570 2023-08-27T11:25:44.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97e4e43700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e01438e0 con 0x7f97c8003a00 2023-08-27T11:25:44.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97ded9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e0142b70 0x7f97e0142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:44.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97ded9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e0142b70 0x7f97e0142ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60018/0 (socket says 172.21.15.112:60018) 2023-08-27T11:25:44.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97ded9d700 1 -- 172.21.15.112:0/1850571587 learned_addr learned my addr 172.21.15.112:0/1850571587 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:44.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97ddd9b700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97e0140570 0x7f97e01409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:44.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97de59c700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c8003a00 0x7f97e0071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:44.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97ded9d700 1 -- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c8003a00 msgr2=0x7f97e0071ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:44.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97ded9d700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c8003a00 0x7f97e0071ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:44.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97ded9d700 1 -- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97e0140570 msgr2=0x7f97e01409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:44.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97ded9d700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97e0140570 0x7f97e01409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:44.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97ded9d700 1 -- 172.21.15.112:0/1850571587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e0143bc0 con 0x7f97e0142b70 2023-08-27T11:25:44.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97ded9d700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e0142b70 0x7f97e0142ff0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f97e00b0a60 tx=0x7f97d800e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:44.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97ceffd700 1 -- 172.21.15.112:0/1850571587 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d800eb80 con 0x7f97e0142b70 2023-08-27T11:25:44.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97ceffd700 1 -- 172.21.15.112:0/1850571587 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d800ece0 con 0x7f97e0142b70 2023-08-27T11:25:44.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97e4e43700 1 -- 172.21.15.112:0/1850571587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e0143ea0 con 0x7f97e0142b70 2023-08-27T11:25:44.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97ceffd700 1 -- 172.21.15.112:0/1850571587 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8013660 con 0x7f97e0142b70 2023-08-27T11:25:44.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.589+0000 7f97e4e43700 1 -- 172.21.15.112:0/1850571587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e0143530 con 0x7f97e0142b70 2023-08-27T11:25:44.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.590+0000 7f97e4e43700 1 -- 172.21.15.112:0/1850571587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e01f35b0 con 0x7f97e0142b70 2023-08-27T11:25:44.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.591+0000 7f97ceffd700 1 -- 172.21.15.112:0/1850571587 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f97d80137c0 con 0x7f97e0142b70 2023-08-27T11:25:44.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.591+0000 7f97ceffd700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f97c006a320 0x7f97c006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:44.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.591+0000 7f97ceffd700 1 -- 172.21.15.112:0/1850571587 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97d8088a90 con 0x7f97e0142b70 2023-08-27T11:25:44.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.591+0000 7f97de59c700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f97c006a320 0x7f97c006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:44.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.592+0000 7f97de59c700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f97c006a320 0x7f97c006c7e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f97d40090b0 tx=0x7f97d401e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:44.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.595+0000 7f97ceffd700 1 -- 172.21.15.112:0/1850571587 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f97e01f35b0 con 0x7f97e0142b70 2023-08-27T11:25:44.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:44.830+0000 7f97e4e43700 1 -- 172.21.15.112:0/1850571587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f97e00b5070 con 0x7f97e0142b70 2023-08-27T11:25:45.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.438+0000 7f97ceffd700 1 -- 172.21.15.112:0/1850571587 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v48) v1 ==== 122+0+0 (secure 0 0 0) 0x7f97e00b5070 con 0x7f97e0142b70 2023-08-27T11:25:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.441+0000 7f97e4e43700 1 -- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f97c006a320 msgr2=0x7f97c006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.441+0000 7f97e4e43700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f97c006a320 0x7f97c006c7e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f97d40090b0 tx=0x7f97d401e040 comp rx=0 tx=0).stop 2023-08-27T11:25:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.441+0000 7f97e4e43700 1 -- 172.21.15.112:0/1850571587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e0142b70 msgr2=0x7f97e0142ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.441+0000 7f97e4e43700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e0142b70 0x7f97e0142ff0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f97e00b0a60 tx=0x7f97d800e440 comp rx=0 tx=0).stop 2023-08-27T11:25:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.441+0000 7f97e4e43700 1 -- 172.21.15.112:0/1850571587 shutdown_connections 2023-08-27T11:25:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.441+0000 7f97e4e43700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f97c006a320 0x7f97c006c7e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.441+0000 7f97e4e43700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e0142b70 0x7f97e0142ff0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.441+0000 7f97e4e43700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c8003a00 0x7f97e0071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.441+0000 7f97e4e43700 1 --2- 172.21.15.112:0/1850571587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97e0140570 0x7f97e01409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.441+0000 7f97e4e43700 1 -- 172.21.15.112:0/1850571587 >> 172.21.15.112:0/1850571587 conn(0x7f97e00b1e90 msgr2=0x7f97e01413f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.441+0000 7f97e4e43700 1 -- 172.21.15.112:0/1850571587 shutdown_connections 2023-08-27T11:25:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.441+0000 7f97e4e43700 1 -- 172.21.15.112:0/1850571587 wait complete. 2023-08-27T11:25:45.456 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:25:45.457 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:25:45.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.579+0000 7f262cc32700 1 Processor -- start 2023-08-27T11:25:45.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.579+0000 7f262cc32700 1 -- start start 2023-08-27T11:25:45.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f262cc32700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f26280b17e0 0x7f26280cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:45.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f262cc32700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2628161850 con 0x7f2628161ce0 2023-08-27T11:25:45.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f262cc32700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26280cfc10 con 0x7f26280b1400 2023-08-27T11:25:45.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f262cc32700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2628161200 con 0x7f26280b17e0 2023-08-27T11:25:45.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f2625d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f26280b17e0 0x7f26280cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:45.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f2626d9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2628161ce0 0x7f262806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54408/0 (socket says 172.21.15.112:54408) 2023-08-27T11:25:45.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f2625d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f26280b17e0 0x7f26280cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60024/0 (socket says 172.21.15.112:60024) 2023-08-27T11:25:45.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f2626d9d700 1 -- 172.21.15.112:0/4196123277 learned_addr learned my addr 172.21.15.112:0/4196123277 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:45.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f2625d9b700 1 -- 172.21.15.112:0/4196123277 >> v1:172.21.15.138:6789/0 conn(0x7f26280b1400 legacy=0x7f262806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:45.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f2625d9b700 1 -- 172.21.15.112:0/4196123277 >> v1:172.21.15.112:6789/0 conn(0x7f2628161ce0 legacy=0x7f262806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:45.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f2625d9b700 1 -- 172.21.15.112:0/4196123277 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26281614e0 con 0x7f26280b17e0 2023-08-27T11:25:45.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f2625d9b700 1 --2- 172.21.15.112:0/4196123277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26280b17e0 0x7f26280cf5a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f261c009840 tx=0x7f261c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c528830b38c7dce7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:45.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f262559a700 1 -- 172.21.15.112:0/4196123277 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c004230 con 0x7f26280b17e0 2023-08-27T11:25:45.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.580+0000 7f262559a700 1 -- 172.21.15.112:0/4196123277 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c004390 con 0x7f26280b17e0 2023-08-27T11:25:45.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.581+0000 7f262559a700 1 -- 172.21.15.112:0/4196123277 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c0036a0 con 0x7f26280b17e0 2023-08-27T11:25:45.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.581+0000 7f262cc32700 1 -- 172.21.15.112:0/4196123277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26280b17e0 msgr2=0x7f26280cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:45.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.581+0000 7f262cc32700 1 --2- 172.21.15.112:0/4196123277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26280b17e0 0x7f26280cf5a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f261c009840 tx=0x7f261c009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:45.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.581+0000 7f262cc32700 1 -- 172.21.15.112:0/4196123277 shutdown_connections 2023-08-27T11:25:45.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.581+0000 7f262cc32700 1 --2- 172.21.15.112:0/4196123277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26280b17e0 0x7f26280cf5a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.581+0000 7f262cc32700 1 -- 172.21.15.112:0/4196123277 >> 172.21.15.112:0/4196123277 conn(0x7f26280d0000 msgr2=0x7f26280d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:45.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.581+0000 7f262cc32700 1 -- 172.21.15.112:0/4196123277 shutdown_connections 2023-08-27T11:25:45.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.581+0000 7f262cc32700 1 -- 172.21.15.112:0/4196123277 wait complete. 2023-08-27T11:25:45.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.582+0000 7f262cc32700 1 Processor -- start 2023-08-27T11:25:45.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.583+0000 7f262cc32700 1 -- start start 2023-08-27T11:25:45.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.583+0000 7f262cc32700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26280b1400 0x7f26280c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:45.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.583+0000 7f262cc32700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26280b17e0 0x7f26281f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:45.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.583+0000 7f262cc32700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2628161ce0 0x7f26281436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:45.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.583+0000 7f262cc32700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280c7610 con 0x7f2628161ce0 2023-08-27T11:25:45.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.583+0000 7f262cc32700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26281417e0 con 0x7f26280b1400 2023-08-27T11:25:45.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.583+0000 7f262cc32700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2628141ac0 con 0x7f26280b17e0 2023-08-27T11:25:45.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.583+0000 7f2626d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2628161ce0 0x7f26281436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:45.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.583+0000 7f2626d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2628161ce0 0x7f26281436f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60032/0 (socket says 172.21.15.112:60032) 2023-08-27T11:25:45.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.583+0000 7f2626d9d700 1 -- 172.21.15.112:0/855614195 learned_addr learned my addr 172.21.15.112:0/855614195 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:45.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.584+0000 7f2626d9d700 1 -- 172.21.15.112:0/855614195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26280b17e0 msgr2=0x7f26281f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:45.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.584+0000 7f2626d9d700 1 --2- 172.21.15.112:0/855614195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26280b17e0 0x7f26281f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.584+0000 7f2626d9d700 1 -- 172.21.15.112:0/855614195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26280b1400 msgr2=0x7f26280c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:45.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.584+0000 7f2626d9d700 1 --2- 172.21.15.112:0/855614195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26280b1400 0x7f26280c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.584+0000 7f2626d9d700 1 -- 172.21.15.112:0/855614195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c0094a0 con 0x7f2628161ce0 2023-08-27T11:25:45.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.584+0000 7f2626d9d700 1 --2- 172.21.15.112:0/855614195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2628161ce0 0x7f26281436f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f262000ca80 tx=0x7f262000ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:45.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.584+0000 7f2616ffd700 1 -- 172.21.15.112:0/855614195 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262000aaa0 con 0x7f2628161ce0 2023-08-27T11:25:45.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.584+0000 7f2616ffd700 1 -- 172.21.15.112:0/855614195 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262000ac00 con 0x7f2628161ce0 2023-08-27T11:25:45.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.584+0000 7f262cc32700 1 -- 172.21.15.112:0/855614195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2628141da0 con 0x7f2628161ce0 2023-08-27T11:25:45.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.584+0000 7f2616ffd700 1 -- 172.21.15.112:0/855614195 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620012400 con 0x7f2628161ce0 2023-08-27T11:25:45.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.584+0000 7f262cc32700 1 -- 172.21.15.112:0/855614195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2628142360 con 0x7f2628161ce0 2023-08-27T11:25:45.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.585+0000 7f262cc32700 1 -- 172.21.15.112:0/855614195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26280c6d90 con 0x7f2628161ce0 2023-08-27T11:25:45.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.585+0000 7f2616ffd700 1 -- 172.21.15.112:0/855614195 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f262001b3b0 con 0x7f2628161ce0 2023-08-27T11:25:45.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.586+0000 7f2616ffd700 1 --2- 172.21.15.112:0/855614195 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f260806a320 0x7f260806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:45.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.586+0000 7f2616ffd700 1 -- 172.21.15.112:0/855614195 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2620088520 con 0x7f2628161ce0 2023-08-27T11:25:45.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.586+0000 7f262659c700 1 --2- 172.21.15.112:0/855614195 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f260806a320 0x7f260806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:45.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.586+0000 7f262659c700 1 --2- 172.21.15.112:0/855614195 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f260806a320 0x7f260806c7e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2618003df0 tx=0x7f2618007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:45.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.590+0000 7f2616ffd700 1 -- 172.21.15.112:0/855614195 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2620053960 con 0x7f2628161ce0 2023-08-27T11:25:45.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.818+0000 7f262cc32700 1 -- 172.21.15.112:0/855614195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2628142550 con 0x7f2628161ce0 2023-08-27T11:25:45.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.819+0000 7f2616ffd700 1 -- 172.21.15.112:0/855614195 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4889 (secure 0 0 0) 0x7f2620056f80 con 0x7f2628161ce0 2023-08-27T11:25:45.821 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:45.822 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6166,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3262556844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3262556844},{"type":"v1","addr":"172.21.15.138:6836","nonce":3262556844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2232724696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2232724696},{"type":"v1","addr":"172.21.15.138:6837","nonce":2232724696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6570,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1511159838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1511159838},{"type":"v1","addr":"172.21.15.112:6834","nonce":1511159838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:25:41.794717+0000","modified":"2023-08-27T11:25:45.437236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2318724344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2318724344},{"type":"v1","addr":"172.21.15.112:6835","nonce":2318724344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-27T11:25:45.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.821+0000 7f262cc32700 1 -- 172.21.15.112:0/855614195 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f260806a320 msgr2=0x7f260806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:45.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.821+0000 7f262cc32700 1 --2- 172.21.15.112:0/855614195 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f260806a320 0x7f260806c7e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2618003df0 tx=0x7f2618007a40 comp rx=0 tx=0).stop 2023-08-27T11:25:45.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.821+0000 7f262cc32700 1 -- 172.21.15.112:0/855614195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2628161ce0 msgr2=0x7f26281436f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:45.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.821+0000 7f262cc32700 1 --2- 172.21.15.112:0/855614195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2628161ce0 0x7f26281436f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f262000ca80 tx=0x7f262000ce40 comp rx=0 tx=0).stop 2023-08-27T11:25:45.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.821+0000 7f262cc32700 1 -- 172.21.15.112:0/855614195 shutdown_connections 2023-08-27T11:25:45.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.821+0000 7f262cc32700 1 --2- 172.21.15.112:0/855614195 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f260806a320 0x7f260806c7e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.821+0000 7f262cc32700 1 --2- 172.21.15.112:0/855614195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2628161ce0 0x7f26281436f0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.821+0000 7f262cc32700 1 --2- 172.21.15.112:0/855614195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26280b1400 0x7f26280c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.821+0000 7f262cc32700 1 --2- 172.21.15.112:0/855614195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26280b17e0 0x7f26281f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.821+0000 7f262cc32700 1 -- 172.21.15.112:0/855614195 >> 172.21.15.112:0/855614195 conn(0x7f26280d0000 msgr2=0x7f26280d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:45.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.821+0000 7f262cc32700 1 -- 172.21.15.112:0/855614195 shutdown_connections 2023-08-27T11:25:45.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.821+0000 7f262cc32700 1 -- 172.21.15.112:0/855614195 wait complete. 2023-08-27T11:25:45.825 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 48 2023-08-27T11:25:45.834 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:45.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.954+0000 7f13f8a04700 1 Processor -- start 2023-08-27T11:25:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.954+0000 7f13f8a04700 1 -- start start 2023-08-27T11:25:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f8a04700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13f4143a60 0x7f13f40cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:45.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f8a04700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f13f415e290 0x7f13f415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:45.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f8a04700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f4143e40 con 0x7f13f40b4960 2023-08-27T11:25:45.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f8a04700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f40b5340 con 0x7f13f415e290 2023-08-27T11:25:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f8a04700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f40ce9a0 con 0x7f13f4143a60 2023-08-27T11:25:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f2d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f13f415e290 0x7f13f415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f1d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f13f40b4960 0x7f13f415db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54418/0 (socket says 172.21.15.112:54418) 2023-08-27T11:25:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f2d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f13f415e290 0x7f13f415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60040/0 (socket says 172.21.15.112:60040) 2023-08-27T11:25:45.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f1d9b700 1 -- 172.21.15.112:0/3935051590 learned_addr learned my addr 172.21.15.112:0/3935051590 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:45.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f259c700 1 --2- 172.21.15.112:0/3935051590 >> v2:172.21.15.138:3300/0 conn(0x7f13f4143a60 0x7f13f40cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:45.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f2d9d700 1 -- 172.21.15.112:0/3935051590 >> v2:172.21.15.138:3300/0 conn(0x7f13f4143a60 msgr2=0x7f13f40cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:45.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f2d9d700 1 --2- 172.21.15.112:0/3935051590 >> v2:172.21.15.138:3300/0 conn(0x7f13f4143a60 0x7f13f40cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f2d9d700 1 -- 172.21.15.112:0/3935051590 >> v1:172.21.15.112:6789/0 conn(0x7f13f40b4960 legacy=0x7f13f415db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:45.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f2d9d700 1 -- 172.21.15.112:0/3935051590 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f40cec80 con 0x7f13f415e290 2023-08-27T11:25:45.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f159a700 1 -- 172.21.15.112:0/3935051590 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069917852 0 0) 0x7f13f4143e40 con 0x7f13f40b4960 2023-08-27T11:25:45.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f2d9d700 1 --2- 172.21.15.112:0/3935051590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13f415e290 0x7f13f415b6a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f13f40b1820 tx=0x7f13ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93054dce365b8b8b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:45.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f159a700 1 -- 172.21.15.112:0/3935051590 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec00e070 con 0x7f13f415e290 2023-08-27T11:25:45.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f159a700 1 -- 172.21.15.112:0/3935051590 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec00bbd0 con 0x7f13f415e290 2023-08-27T11:25:45.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.955+0000 7f13f159a700 1 -- 172.21.15.112:0/3935051590 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec012430 con 0x7f13f415e290 2023-08-27T11:25:45.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.956+0000 7f13f8a04700 1 -- 172.21.15.112:0/3935051590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13f415e290 msgr2=0x7f13f415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:45.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.956+0000 7f13f8a04700 1 --2- 172.21.15.112:0/3935051590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13f415e290 0x7f13f415b6a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f13f40b1820 tx=0x7f13ec00b330 comp rx=0 tx=0).stop 2023-08-27T11:25:45.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.956+0000 7f13f8a04700 1 -- 172.21.15.112:0/3935051590 shutdown_connections 2023-08-27T11:25:45.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.956+0000 7f13f8a04700 1 --2- 172.21.15.112:0/3935051590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13f415e290 0x7f13f415b6a0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.956+0000 7f13f8a04700 1 --2- 172.21.15.112:0/3935051590 >> v2:172.21.15.138:3300/0 conn(0x7f13f4143a60 0x7f13f40cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.956+0000 7f13f8a04700 1 -- 172.21.15.112:0/3935051590 >> 172.21.15.112:0/3935051590 conn(0x7f13f406adb0 msgr2=0x7f13f406b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:45.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.956+0000 7f13f8a04700 1 -- 172.21.15.112:0/3935051590 shutdown_connections 2023-08-27T11:25:45.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.956+0000 7f13f8a04700 1 -- 172.21.15.112:0/3935051590 wait complete. 2023-08-27T11:25:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.957+0000 7f13f8a04700 1 Processor -- start 2023-08-27T11:25:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.957+0000 7f13f8a04700 1 -- start start 2023-08-27T11:25:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.958+0000 7f13f8a04700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13f40b4960 0x7f13f413fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.958+0000 7f13f8a04700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13f4143a60 0x7f13f41f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.958+0000 7f13f8a04700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13f415e290 0x7f13f4153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.958+0000 7f13f8a04700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40c9620 con 0x7f13f415e290 2023-08-27T11:25:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.958+0000 7f13f8a04700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40c9870 con 0x7f13f4143a60 2023-08-27T11:25:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.958+0000 7f13f8a04700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40c9b50 con 0x7f13f40b4960 2023-08-27T11:25:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.958+0000 7f13f2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13f415e290 0x7f13f4153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.958+0000 7f13f2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13f415e290 0x7f13f4153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60046/0 (socket says 172.21.15.112:60046) 2023-08-27T11:25:45.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.958+0000 7f13f2d9d700 1 -- 172.21.15.112:0/1011862554 learned_addr learned my addr 172.21.15.112:0/1011862554 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:45.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.958+0000 7f13f1d9b700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13f4143a60 0x7f13f41f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:45.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.958+0000 7f13f259c700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13f40b4960 0x7f13f413fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:45.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.959+0000 7f13f2d9d700 1 -- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13f40b4960 msgr2=0x7f13f413fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:45.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.959+0000 7f13f2d9d700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13f40b4960 0x7f13f413fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.959+0000 7f13f2d9d700 1 -- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13f4143a60 msgr2=0x7f13f41f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:45.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.959+0000 7f13f2d9d700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13f4143a60 0x7f13f41f2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:45.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.959+0000 7f13f2d9d700 1 -- 172.21.15.112:0/1011862554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec009a70 con 0x7f13f415e290 2023-08-27T11:25:45.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.959+0000 7f13f1d9b700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13f4143a60 0x7f13f41f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:45.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.959+0000 7f13f2d9d700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13f415e290 0x7f13f4153470 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f13ec013000 tx=0x7f13ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:45.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.959+0000 7f13e2ffd700 1 -- 172.21.15.112:0/1011862554 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec012430 con 0x7f13f415e290 2023-08-27T11:25:45.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.959+0000 7f13e2ffd700 1 -- 172.21.15.112:0/1011862554 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec020070 con 0x7f13f415e290 2023-08-27T11:25:45.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.959+0000 7f13f8a04700 1 -- 172.21.15.112:0/1011862554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f40c9e30 con 0x7f13f415e290 2023-08-27T11:25:45.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.959+0000 7f13e2ffd700 1 -- 172.21.15.112:0/1011862554 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec004280 con 0x7f13f415e290 2023-08-27T11:25:45.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.959+0000 7f13f8a04700 1 -- 172.21.15.112:0/1011862554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f4140260 con 0x7f13f415e290 2023-08-27T11:25:45.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.960+0000 7f13f8a04700 1 -- 172.21.15.112:0/1011862554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f40ca350 con 0x7f13f415e290 2023-08-27T11:25:45.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.960+0000 7f13e2ffd700 1 -- 172.21.15.112:0/1011862554 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f13ec0043e0 con 0x7f13f415e290 2023-08-27T11:25:45.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.960+0000 7f13e2ffd700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13d406a320 0x7f13d406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:45.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.961+0000 7f13e2ffd700 1 -- 172.21.15.112:0/1011862554 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f13ec022070 con 0x7f13f415e290 2023-08-27T11:25:45.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.961+0000 7f13f259c700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13d406a320 0x7f13d406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:45.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.961+0000 7f13f259c700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13d406a320 0x7f13d406c7e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f13e4006fd0 tx=0x7f13e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:45.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:45.965+0000 7f13e2ffd700 1 -- 172.21.15.112:0/1011862554 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f13ec055d20 con 0x7f13f415e290 2023-08-27T11:25:46.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.155+0000 7f13f8a04700 1 -- 172.21.15.112:0/1011862554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13f4067a20 con 0x7f13f415e290 2023-08-27T11:25:46.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.156+0000 7f13e2ffd700 1 -- 172.21.15.112:0/1011862554 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18260 (secure 0 0 0) 0x7f13ec059340 con 0x7f13f415e290 2023-08-27T11:25:46.158 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:46.158 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":77,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:25:43.408876+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:39.495757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:40.765773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:46.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.158+0000 7f13f8a04700 1 -- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13d406a320 msgr2=0x7f13d406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.158+0000 7f13f8a04700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13d406a320 0x7f13d406c7e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f13e4006fd0 tx=0x7f13e4008040 comp rx=0 tx=0).stop 2023-08-27T11:25:46.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.158+0000 7f13f8a04700 1 -- 172.21.15.112:0/1011862554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13f415e290 msgr2=0x7f13f4153470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.158+0000 7f13f8a04700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13f415e290 0x7f13f4153470 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f13ec013000 tx=0x7f13ec004ad0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.158+0000 7f13f8a04700 1 -- 172.21.15.112:0/1011862554 shutdown_connections 2023-08-27T11:25:46.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.158+0000 7f13f8a04700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13d406a320 0x7f13d406c7e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.158+0000 7f13f8a04700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13f415e290 0x7f13f4153470 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.158+0000 7f13f8a04700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13f40b4960 0x7f13f413fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.158+0000 7f13f8a04700 1 --2- 172.21.15.112:0/1011862554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13f4143a60 0x7f13f41f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.158+0000 7f13f8a04700 1 -- 172.21.15.112:0/1011862554 >> 172.21.15.112:0/1011862554 conn(0x7f13f406adb0 msgr2=0x7f13f415c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:46.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.158+0000 7f13f8a04700 1 -- 172.21.15.112:0/1011862554 shutdown_connections 2023-08-27T11:25:46.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.158+0000 7f13f8a04700 1 -- 172.21.15.112:0/1011862554 wait complete. 2023-08-27T11:25:46.172 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:25:46.173 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:25:46.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.292+0000 7f2306cd3700 1 Processor -- start 2023-08-27T11:25:46.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.292+0000 7f2306cd3700 1 -- start start 2023-08-27T11:25:46.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f2306cd3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f23000b17e0 0x7f23000cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:46.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f2306cd3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2300161ce0 0x7f230006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:46.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f2306cd3700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2300161850 con 0x7f23000b1400 2023-08-27T11:25:46.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f2306cd3700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23000cfc10 con 0x7f23000b17e0 2023-08-27T11:25:46.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f2306cd3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2300161200 con 0x7f2300161ce0 2023-08-27T11:25:46.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f2305270700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2300161ce0 0x7f230006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:46.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f22fffff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f23000b17e0 0x7f23000cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:46.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f2305270700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2300161ce0 0x7f230006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:43476/0 (socket says 172.21.15.112:43476) 2023-08-27T11:25:46.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f2304a6f700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f23000b1400 0x7f230006e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:33724/0 (socket says 172.21.15.112:33724) 2023-08-27T11:25:46.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f22fffff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f23000b17e0 0x7f23000cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:60478/0 (socket says 172.21.15.112:60478) 2023-08-27T11:25:46.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f2305270700 1 -- 172.21.15.112:0/1507254811 learned_addr learned my addr 172.21.15.112:0/1507254811 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:46.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f2305270700 1 -- 172.21.15.112:0/1507254811 >> v2:172.21.15.138:3300/0 conn(0x7f23000b17e0 msgr2=0x7f23000cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:46.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f2305270700 1 --2- 172.21.15.112:0/1507254811 >> v2:172.21.15.138:3300/0 conn(0x7f23000b17e0 0x7f23000cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f2305270700 1 -- 172.21.15.112:0/1507254811 >> v1:172.21.15.138:6789/0 conn(0x7f23000b1400 legacy=0x7f230006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f2305270700 1 -- 172.21.15.112:0/1507254811 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23001614e0 con 0x7f2300161ce0 2023-08-27T11:25:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.293+0000 7f22fffff700 1 --2- 172.21.15.112:0/1507254811 >> v2:172.21.15.138:3300/0 conn(0x7f23000b17e0 0x7f23000cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.294+0000 7f2305270700 1 --2- 172.21.15.112:0/1507254811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2300161ce0 0x7f230006f790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009e50 tx=0x7f22f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8093622c15d93426 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.294+0000 7f22ff7fe700 1 -- 172.21.15.112:0/1507254811 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f800e070 con 0x7f2300161ce0 2023-08-27T11:25:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.294+0000 7f22ff7fe700 1 -- 172.21.15.112:0/1507254811 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f800b910 con 0x7f2300161ce0 2023-08-27T11:25:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.294+0000 7f22ff7fe700 1 -- 172.21.15.112:0/1507254811 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f800bcc0 con 0x7f2300161ce0 2023-08-27T11:25:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.294+0000 7f2306cd3700 1 -- 172.21.15.112:0/1507254811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2300161ce0 msgr2=0x7f230006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.294+0000 7f2306cd3700 1 --2- 172.21.15.112:0/1507254811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2300161ce0 0x7f230006f790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009e50 tx=0x7f22f800b330 comp rx=0 tx=0).stop 2023-08-27T11:25:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.294+0000 7f2306cd3700 1 -- 172.21.15.112:0/1507254811 shutdown_connections 2023-08-27T11:25:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.294+0000 7f2306cd3700 1 --2- 172.21.15.112:0/1507254811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2300161ce0 0x7f230006f790 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.294+0000 7f2306cd3700 1 --2- 172.21.15.112:0/1507254811 >> v2:172.21.15.138:3300/0 conn(0x7f23000b17e0 0x7f23000cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.294+0000 7f2306cd3700 1 -- 172.21.15.112:0/1507254811 >> 172.21.15.112:0/1507254811 conn(0x7f23000d0000 msgr2=0x7f23000d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.294+0000 7f2306cd3700 1 -- 172.21.15.112:0/1507254811 shutdown_connections 2023-08-27T11:25:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.294+0000 7f2306cd3700 1 -- 172.21.15.112:0/1507254811 wait complete. 2023-08-27T11:25:46.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.295+0000 7f2306cd3700 1 Processor -- start 2023-08-27T11:25:46.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.295+0000 7f2306cd3700 1 -- start start 2023-08-27T11:25:46.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.296+0000 7f2306cd3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23000b1400 0x7f23000c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.296+0000 7f2306cd3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23000b17e0 0x7f23000c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.296+0000 7f2306cd3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2300161ce0 0x7f23000c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.296+0000 7f2306cd3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23000c34b0 con 0x7f2300161ce0 2023-08-27T11:25:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.296+0000 7f2306cd3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2300070950 con 0x7f23000b1400 2023-08-27T11:25:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.296+0000 7f2306cd3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2300070c30 con 0x7f23000b17e0 2023-08-27T11:25:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f2305270700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2300161ce0 0x7f23000c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f2305270700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2300161ce0 0x7f23000c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60060/0 (socket says 172.21.15.112:60060) 2023-08-27T11:25:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f2305270700 1 -- 172.21.15.112:0/3354448876 learned_addr learned my addr 172.21.15.112:0/3354448876 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f2304a6f700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23000b1400 0x7f23000c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f22fffff700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23000b17e0 0x7f23000c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:46.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f2305270700 1 -- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23000b17e0 msgr2=0x7f23000c2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f2305270700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23000b17e0 0x7f23000c2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f2305270700 1 -- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23000b1400 msgr2=0x7f23000c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f2305270700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23000b1400 0x7f23000c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f2305270700 1 -- 172.21.15.112:0/3354448876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec0094a0 con 0x7f2300161ce0 2023-08-27T11:25:46.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f2304a6f700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23000b1400 0x7f23000c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:46.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f2305270700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2300161ce0 0x7f23000c62d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2300161340 tx=0x7f22f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:46.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f22fd7fa700 1 -- 172.21.15.112:0/3354448876 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f8003ea0 con 0x7f2300161ce0 2023-08-27T11:25:46.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f22fd7fa700 1 -- 172.21.15.112:0/3354448876 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f80099b0 con 0x7f2300161ce0 2023-08-27T11:25:46.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f2306cd3700 1 -- 172.21.15.112:0/3354448876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f8009b30 con 0x7f2300161ce0 2023-08-27T11:25:46.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f22fd7fa700 1 -- 172.21.15.112:0/3354448876 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f801c680 con 0x7f2300161ce0 2023-08-27T11:25:46.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.297+0000 7f2306cd3700 1 -- 172.21.15.112:0/3354448876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23000711f0 con 0x7f2300161ce0 2023-08-27T11:25:46.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.298+0000 7f2306cd3700 1 -- 172.21.15.112:0/3354448876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2300067a20 con 0x7f2300161ce0 2023-08-27T11:25:46.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.298+0000 7f22fd7fa700 1 -- 172.21.15.112:0/3354448876 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f22f801c7e0 con 0x7f2300161ce0 2023-08-27T11:25:46.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.299+0000 7f22fd7fa700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22e406a320 0x7f22e406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:46.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.299+0000 7f22fd7fa700 1 -- 172.21.15.112:0/3354448876 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f22f808a730 con 0x7f2300161ce0 2023-08-27T11:25:46.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.299+0000 7f2304a6f700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22e406a320 0x7f22e406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:46.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.300+0000 7f2304a6f700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22e406a320 0x7f22e406c7e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f23000c3f80 tx=0x7f22f4007e10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:46.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.303+0000 7f22fd7fa700 1 -- 172.21.15.112:0/3354448876 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f22f80559c0 con 0x7f2300161ce0 2023-08-27T11:25:46.379 INFO:tasks.ceph.mds.c.smithi112.stderr:2023-08-27T11:25:46.378+0000 7facdf425700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:25:46.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.494+0000 7f2306cd3700 1 -- 172.21.15.112:0/3354448876 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f230006db60 con 0x7f22e406a320 2023-08-27T11:25:46.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.503+0000 7f22fd7fa700 1 -- 172.21.15.112:0/3354448876 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306600 (secure 0 0 0) 0x7f230006db60 con 0x7f22e406a320 2023-08-27T11:25:46.508 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:46.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.506+0000 7f2306cd3700 1 -- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22e406a320 msgr2=0x7f22e406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.506+0000 7f2306cd3700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22e406a320 0x7f22e406c7e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f23000c3f80 tx=0x7f22f4007e10 comp rx=0 tx=0).stop 2023-08-27T11:25:46.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.506+0000 7f2306cd3700 1 -- 172.21.15.112:0/3354448876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2300161ce0 msgr2=0x7f23000c62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.506+0000 7f2306cd3700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2300161ce0 0x7f23000c62d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2300161340 tx=0x7f22f8004ad0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.506+0000 7f2306cd3700 1 -- 172.21.15.112:0/3354448876 shutdown_connections 2023-08-27T11:25:46.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.506+0000 7f2306cd3700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22e406a320 0x7f22e406c7e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.506+0000 7f2306cd3700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2300161ce0 0x7f23000c62d0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.506+0000 7f2306cd3700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23000b1400 0x7f23000c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.506+0000 7f2306cd3700 1 --2- 172.21.15.112:0/3354448876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23000b17e0 0x7f23000c2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.506+0000 7f2306cd3700 1 -- 172.21.15.112:0/3354448876 >> 172.21.15.112:0/3354448876 conn(0x7f23000d0000 msgr2=0x7f23000d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:46.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.506+0000 7f2306cd3700 1 -- 172.21.15.112:0/3354448876 shutdown_connections 2023-08-27T11:25:46.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.507+0000 7f2306cd3700 1 -- 172.21.15.112:0/3354448876 wait complete. 2023-08-27T11:25:46.514 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:25:46.520 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2023-08-27T11:25:46.359922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257828,"kb_used_data":8356,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754716892,"statfs":{"total":773094113280,"available":772830097408,"internally_reserved":0,"allocated":8556544,"data_stored":4526431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.968705"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396787+0000","last_change":"2023-08-27T11:25:41.396787+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396787+0000","last_undegraded":"2023-08-27T11:25:41.396787+0000","last_fullsized":"2023-08-27T11:25:41.396787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.408023+0000","last_change":"2023-08-27T11:25:41.408023+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.408023+0000","last_undegraded":"2023-08-27T11:25:41.408023+0000","last_fullsized":"2023-08-27T11:25:41.408023+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396335+0000","last_change":"2023-08-27T11:25:41.396335+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396335+0000","last_undegraded":"2023-08-27T11:25:41.396335+0000","last_fullsized":"2023-08-27T11:25:41.396335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397019+0000","last_change":"2023-08-27T11:25:41.397019+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397019+0000","last_undegraded":"2023-08-27T11:25:41.397019+0000","last_fullsized":"2023-08-27T11:25:41.397019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399662+0000","last_change":"2023-08-27T11:25:41.399662+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399662+0000","last_undegraded":"2023-08-27T11:25:41.399662+0000","last_fullsized":"2023-08-27T11:25:41.399662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.406043+0000","last_change":"2023-08-27T11:25:41.406043+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.406043+0000","last_undegraded":"2023-08-27T11:25:41.406043+0000","last_fullsized":"2023-08-27T11:25:41.406043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400258+0000","last_change":"2023-08-27T11:25:41.400258+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400258+0000","last_undegraded":"2023-08-27T11:25:41.400258+0000","last_fullsized":"2023-08-27T11:25:41.400258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398869+0000","last_change":"2023-08-27T11:25:41.398869+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398869+0000","last_undegraded":"2023-08-27T11:25:41.398869+0000","last_fullsized":"2023-08-27T11:25:41.398869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398915+0000","last_change":"2023-08-27T11:25:41.398915+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398915+0000","last_undegraded":"2023-08-27T11:25:41.398915+0000","last_fullsized":"2023-08-27T11:25:41.398915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398563+0000","last_change":"2023-08-27T11:25:41.398563+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398563+0000","last_undegraded":"2023-08-27T11:25:41.398563+0000","last_fullsized":"2023-08-27T11:25:41.398563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.401519+0000","last_change":"2023-08-27T11:25:41.401519+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.401519+0000","last_undegraded":"2023-08-27T11:25:41.401519+0000","last_fullsized":"2023-08-27T11:25:41.401519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396135+0000","last_change":"2023-08-27T11:25:41.396135+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396135+0000","last_undegraded":"2023-08-27T11:25:41.396135+0000","last_fullsized":"2023-08-27T11:25:41.396135+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398767+0000","last_change":"2023-08-27T11:25:41.398767+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398767+0000","last_undegraded":"2023-08-27T11:25:41.398767+0000","last_fullsized":"2023-08-27T11:25:41.398767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.402238+0000","last_change":"2023-08-27T11:25:41.402238+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.402238+0000","last_undegraded":"2023-08-27T11:25:41.402238+0000","last_fullsized":"2023-08-27T11:25:41.402238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400006+0000","last_change":"2023-08-27T11:25:41.400006+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400006+0000","last_undegraded":"2023-08-27T11:25:41.400006+0000","last_fullsized":"2023-08-27T11:25:41.400006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403883+0000","last_change":"2023-08-27T11:25:41.403883+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403883+0000","last_undegraded":"2023-08-27T11:25:41.403883+0000","last_fullsized":"2023-08-27T11:25:41.403883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403836+0000","last_change":"2023-08-27T11:25:41.403836+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403836+0000","last_undegraded":"2023-08-27T11:25:41.403836+0000","last_fullsized":"2023-08-27T11:25:41.403836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396268+0000","last_change":"2023-08-27T11:25:41.396268+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396268+0000","last_undegraded":"2023-08-27T11:25:41.396268+0000","last_fullsized":"2023-08-27T11:25:41.396268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400999+0000","last_change":"2023-08-27T11:25:41.400999+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400999+0000","last_undegraded":"2023-08-27T11:25:41.400999+0000","last_fullsized":"2023-08-27T11:25:41.400999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399588+0000","last_change":"2023-08-27T11:25:41.399588+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399588+0000","last_undegraded":"2023-08-27T11:25:41.399588+0000","last_fullsized":"2023-08-27T11:25:41.399588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399193+0000","last_change":"2023-08-27T11:25:41.399193+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399193+0000","last_undegraded":"2023-08-27T11:25:41.399193+0000","last_fullsized":"2023-08-27T11:25:41.399193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400790+0000","last_change":"2023-08-27T11:25:41.400790+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400790+0000","last_undegraded":"2023-08-27T11:25:41.400790+0000","last_fullsized":"2023-08-27T11:25:41.400790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.407276+0000","last_change":"2023-08-27T11:25:41.407276+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.407276+0000","last_undegraded":"2023-08-27T11:25:41.407276+0000","last_fullsized":"2023-08-27T11:25:41.407276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395723+0000","last_change":"2023-08-27T11:25:41.395723+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395723+0000","last_undegraded":"2023-08-27T11:25:41.395723+0000","last_fullsized":"2023-08-27T11:25:41.395723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.408337+0000","last_change":"2023-08-27T11:25:41.408337+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.408337+0000","last_undegraded":"2023-08-27T11:25:41.408337+0000","last_fullsized":"2023-08-27T11:25:41.408337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.401694+0000","last_change":"2023-08-27T11:25:41.401694+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.401694+0000","last_undegraded":"2023-08-27T11:25:41.401694+0000","last_fullsized":"2023-08-27T11:25:41.401694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395708+0000","last_change":"2023-08-27T11:25:41.395708+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395708+0000","last_undegraded":"2023-08-27T11:25:41.395708+0000","last_fullsized":"2023-08-27T11:25:41.395708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399209+0000","last_change":"2023-08-27T11:25:41.399209+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399209+0000","last_undegraded":"2023-08-27T11:25:41.399209+0000","last_fullsized":"2023-08-27T11:25:41.399209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399718+0000","last_change":"2023-08-27T11:25:41.399718+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399718+0000","last_undegraded":"2023-08-27T11:25:41.399718+0000","last_fullsized":"2023-08-27T11:25:41.399718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400210+0000","last_change":"2023-08-27T11:25:41.400210+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400210+0000","last_undegraded":"2023-08-27T11:25:41.400210+0000","last_fullsized":"2023-08-27T11:25:41.400210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.409672+0000","last_change":"2023-08-27T11:25:41.409672+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.409672+0000","last_undegraded":"2023-08-27T11:25:41.409672+0000","last_fullsized":"2023-08-27T11:25:41.409672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403514+0000","last_change":"2023-08-27T11:25:41.403514+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403514+0000","last_undegraded":"2023-08-27T11:25:41.403514+0000","last_fullsized":"2023-08-27T11:25:41.403514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.402400+0000","last_change":"2023-08-27T11:25:41.402400+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.402400+0000","last_undegraded":"2023-08-27T11:25:41.402400+0000","last_fullsized":"2023-08-27T11:25:41.402400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.419031+0000","last_change":"2023-08-27T11:25:41.414884+0000","last_active":"2023-08-27T11:25:41.419031+0000","last_peered":"2023-08-27T11:25:41.419031+0000","last_clean":"2023-08-27T11:25:41.419031+0000","last_became_active":"2023-08-27T11:25:41.414549+0000","last_became_peered":"2023-08-27T11:25:41.414549+0000","last_unstale":"2023-08-27T11:25:41.419031+0000","last_undegraded":"2023-08-27T11:25:41.419031+0000","last_fullsized":"2023-08-27T11:25:41.419031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:31:13.509254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403856+0000","last_change":"2023-08-27T11:25:41.403856+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403856+0000","last_undegraded":"2023-08-27T11:25:41.403856+0000","last_fullsized":"2023-08-27T11:25:41.403856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.416537+0000","last_change":"2023-08-27T11:25:41.414847+0000","last_active":"2023-08-27T11:25:41.416537+0000","last_peered":"2023-08-27T11:25:41.416537+0000","last_clean":"2023-08-27T11:25:41.416537+0000","last_became_active":"2023-08-27T11:25:41.414532+0000","last_became_peered":"2023-08-27T11:25:41.414532+0000","last_unstale":"2023-08-27T11:25:41.416537+0000","last_undegraded":"2023-08-27T11:25:41.416537+0000","last_fullsized":"2023-08-27T11:25:41.416537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:52:27.302892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399003+0000","last_change":"2023-08-27T11:25:41.399003+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399003+0000","last_undegraded":"2023-08-27T11:25:41.399003+0000","last_fullsized":"2023-08-27T11:25:41.399003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.416340+0000","last_change":"2023-08-27T11:25:41.412179+0000","last_active":"2023-08-27T11:25:41.416340+0000","last_peered":"2023-08-27T11:25:41.416340+0000","last_clean":"2023-08-27T11:25:41.416340+0000","last_became_active":"2023-08-27T11:25:41.411940+0000","last_became_peered":"2023-08-27T11:25:41.411940+0000","last_unstale":"2023-08-27T11:25:41.416340+0000","last_undegraded":"2023-08-27T11:25:41.416340+0000","last_fullsized":"2023-08-27T11:25:41.416340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:01:10.875523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403413+0000","last_change":"2023-08-27T11:25:41.403413+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403413+0000","last_undegraded":"2023-08-27T11:25:41.403413+0000","last_fullsized":"2023-08-27T11:25:41.403413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418955+0000","last_change":"2023-08-27T11:25:41.414002+0000","last_active":"2023-08-27T11:25:41.418955+0000","last_peered":"2023-08-27T11:25:41.418955+0000","last_clean":"2023-08-27T11:25:41.418955+0000","last_became_active":"2023-08-27T11:25:41.413740+0000","last_became_peered":"2023-08-27T11:25:41.413740+0000","last_unstale":"2023-08-27T11:25:41.418955+0000","last_undegraded":"2023-08-27T11:25:41.418955+0000","last_fullsized":"2023-08-27T11:25:41.418955+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:26:23.153384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397310+0000","last_change":"2023-08-27T11:25:41.397310+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397310+0000","last_undegraded":"2023-08-27T11:25:41.397310+0000","last_fullsized":"2023-08-27T11:25:41.397310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.406853+0000","last_change":"2023-08-27T11:25:41.405587+0000","last_active":"2023-08-27T11:25:41.406853+0000","last_peered":"2023-08-27T11:25:41.406853+0000","last_clean":"2023-08-27T11:25:41.406853+0000","last_became_active":"2023-08-27T11:25:41.405255+0000","last_became_peered":"2023-08-27T11:25:41.405255+0000","last_unstale":"2023-08-27T11:25:41.406853+0000","last_undegraded":"2023-08-27T11:25:41.406853+0000","last_fullsized":"2023-08-27T11:25:41.406853+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:09:28.315159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.406628+0000","last_change":"2023-08-27T11:25:41.406628+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.406628+0000","last_undegraded":"2023-08-27T11:25:41.406628+0000","last_fullsized":"2023-08-27T11:25:41.406628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.409295+0000","last_change":"2023-08-27T11:25:41.406787+0000","last_active":"2023-08-27T11:25:41.409295+0000","last_peered":"2023-08-27T11:25:41.409295+0000","last_clean":"2023-08-27T11:25:41.409295+0000","last_became_active":"2023-08-27T11:25:41.406253+0000","last_became_peered":"2023-08-27T11:25:41.406253+0000","last_unstale":"2023-08-27T11:25:41.409295+0000","last_undegraded":"2023-08-27T11:25:41.409295+0000","last_fullsized":"2023-08-27T11:25:41.409295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:38.768166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397479+0000","last_change":"2023-08-27T11:25:41.397479+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397479+0000","last_undegraded":"2023-08-27T11:25:41.397479+0000","last_fullsized":"2023-08-27T11:25:41.397479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.408833+0000","last_change":"2023-08-27T11:25:41.404377+0000","last_active":"2023-08-27T11:25:41.408833+0000","last_peered":"2023-08-27T11:25:41.408833+0000","last_clean":"2023-08-27T11:25:41.408833+0000","last_became_active":"2023-08-27T11:25:41.404056+0000","last_became_peered":"2023-08-27T11:25:41.404056+0000","last_unstale":"2023-08-27T11:25:41.408833+0000","last_undegraded":"2023-08-27T11:25:41.408833+0000","last_fullsized":"2023-08-27T11:25:41.408833+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:16:51.600797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396184+0000","last_change":"2023-08-27T11:25:41.396184+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396184+0000","last_undegraded":"2023-08-27T11:25:41.396184+0000","last_fullsized":"2023-08-27T11:25:41.396184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411092+0000","last_change":"2023-08-27T11:25:41.407540+0000","last_active":"2023-08-27T11:25:41.411092+0000","last_peered":"2023-08-27T11:25:41.411092+0000","last_clean":"2023-08-27T11:25:41.411092+0000","last_became_active":"2023-08-27T11:25:41.406794+0000","last_became_peered":"2023-08-27T11:25:41.406794+0000","last_unstale":"2023-08-27T11:25:41.411092+0000","last_undegraded":"2023-08-27T11:25:41.411092+0000","last_fullsized":"2023-08-27T11:25:41.411092+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:10:38.722806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398510+0000","last_change":"2023-08-27T11:25:41.398510+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398510+0000","last_undegraded":"2023-08-27T11:25:41.398510+0000","last_fullsized":"2023-08-27T11:25:41.398510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421891+0000","last_change":"2023-08-27T11:25:41.417604+0000","last_active":"2023-08-27T11:25:41.421891+0000","last_peered":"2023-08-27T11:25:41.421891+0000","last_clean":"2023-08-27T11:25:41.421891+0000","last_became_active":"2023-08-27T11:25:41.416727+0000","last_became_peered":"2023-08-27T11:25:41.416727+0000","last_unstale":"2023-08-27T11:25:41.421891+0000","last_undegraded":"2023-08-27T11:25:41.421891+0000","last_fullsized":"2023-08-27T11:25:41.421891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:30.061949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398968+0000","last_change":"2023-08-27T11:25:41.398968+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398968+0000","last_undegraded":"2023-08-27T11:25:41.398968+0000","last_fullsized":"2023-08-27T11:25:41.398968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.406341+0000","last_change":"2023-08-27T11:25:41.402868+0000","last_active":"2023-08-27T11:25:41.406341+0000","last_peered":"2023-08-27T11:25:41.406341+0000","last_clean":"2023-08-27T11:25:41.406341+0000","last_became_active":"2023-08-27T11:25:41.402585+0000","last_became_peered":"2023-08-27T11:25:41.402585+0000","last_unstale":"2023-08-27T11:25:41.406341+0000","last_undegraded":"2023-08-27T11:25:41.406341+0000","last_fullsized":"2023-08-27T11:25:41.406341+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:24:45.015907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395873+0000","last_change":"2023-08-27T11:25:41.395873+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395873+0000","last_undegraded":"2023-08-27T11:25:41.395873+0000","last_fullsized":"2023-08-27T11:25:41.395873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.413847+0000","last_change":"2023-08-27T11:25:41.408134+0000","last_active":"2023-08-27T11:25:41.413847+0000","last_peered":"2023-08-27T11:25:41.413847+0000","last_clean":"2023-08-27T11:25:41.413847+0000","last_became_active":"2023-08-27T11:25:41.407806+0000","last_became_peered":"2023-08-27T11:25:41.407806+0000","last_unstale":"2023-08-27T11:25:41.413847+0000","last_undegraded":"2023-08-27T11:25:41.413847+0000","last_fullsized":"2023-08-27T11:25:41.413847+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:38:30.962549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396189+0000","last_change":"2023-08-27T11:25:41.396189+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396189+0000","last_undegraded":"2023-08-27T11:25:41.396189+0000","last_fullsized":"2023-08-27T11:25:41.396189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.415329+0000","last_change":"2023-08-27T11:25:41.414641+0000","last_active":"2023-08-27T11:25:41.415329+0000","last_peered":"2023-08-27T11:25:41.415329+0000","last_clean":"2023-08-27T11:25:41.415329+0000","last_became_active":"2023-08-27T11:25:41.414361+0000","last_became_peered":"2023-08-27T11:25:41.414361+0000","last_unstale":"2023-08-27T11:25:41.415329+0000","last_undegraded":"2023-08-27T11:25:41.415329+0000","last_fullsized":"2023-08-27T11:25:41.415329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:14.192009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396572+0000","last_change":"2023-08-27T11:25:41.396572+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396572+0000","last_undegraded":"2023-08-27T11:25:41.396572+0000","last_fullsized":"2023-08-27T11:25:41.396572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.413558+0000","last_change":"2023-08-27T11:25:41.410338+0000","last_active":"2023-08-27T11:25:41.413558+0000","last_peered":"2023-08-27T11:25:41.413558+0000","last_clean":"2023-08-27T11:25:41.413558+0000","last_became_active":"2023-08-27T11:25:41.410005+0000","last_became_peered":"2023-08-27T11:25:41.410005+0000","last_unstale":"2023-08-27T11:25:41.413558+0000","last_undegraded":"2023-08-27T11:25:41.413558+0000","last_fullsized":"2023-08-27T11:25:41.413558+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:22:28.572307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398507+0000","last_change":"2023-08-27T11:25:41.398507+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398507+0000","last_undegraded":"2023-08-27T11:25:41.398507+0000","last_fullsized":"2023-08-27T11:25:41.398507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.412023+0000","last_change":"2023-08-27T11:25:41.408888+0000","last_active":"2023-08-27T11:25:41.412023+0000","last_peered":"2023-08-27T11:25:41.412023+0000","last_clean":"2023-08-27T11:25:41.412023+0000","last_became_active":"2023-08-27T11:25:41.408609+0000","last_became_peered":"2023-08-27T11:25:41.408609+0000","last_unstale":"2023-08-27T11:25:41.412023+0000","last_undegraded":"2023-08-27T11:25:41.412023+0000","last_fullsized":"2023-08-27T11:25:41.412023+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:12:28.171979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395468+0000","last_change":"2023-08-27T11:25:41.395468+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395468+0000","last_undegraded":"2023-08-27T11:25:41.395468+0000","last_fullsized":"2023-08-27T11:25:41.395468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.415697+0000","last_change":"2023-08-27T11:25:41.414552+0000","last_active":"2023-08-27T11:25:41.415697+0000","last_peered":"2023-08-27T11:25:41.415697+0000","last_clean":"2023-08-27T11:25:41.415697+0000","last_became_active":"2023-08-27T11:25:41.414281+0000","last_became_peered":"2023-08-27T11:25:41.414281+0000","last_unstale":"2023-08-27T11:25:41.415697+0000","last_undegraded":"2023-08-27T11:25:41.415697+0000","last_fullsized":"2023-08-27T11:25:41.415697+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:40:04.874008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395170+0000","last_change":"2023-08-27T11:25:41.395170+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395170+0000","last_undegraded":"2023-08-27T11:25:41.395170+0000","last_fullsized":"2023-08-27T11:25:41.395170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411147+0000","last_change":"2023-08-27T11:25:41.407400+0000","last_active":"2023-08-27T11:25:41.411147+0000","last_peered":"2023-08-27T11:25:41.411147+0000","last_clean":"2023-08-27T11:25:41.411147+0000","last_became_active":"2023-08-27T11:25:41.405982+0000","last_became_peered":"2023-08-27T11:25:41.405982+0000","last_unstale":"2023-08-27T11:25:41.411147+0000","last_undegraded":"2023-08-27T11:25:41.411147+0000","last_fullsized":"2023-08-27T11:25:41.411147+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:59.630816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.407857+0000","last_change":"2023-08-27T11:25:41.407857+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.407857+0000","last_undegraded":"2023-08-27T11:25:41.407857+0000","last_fullsized":"2023-08-27T11:25:41.407857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418905+0000","last_change":"2023-08-27T11:25:41.415726+0000","last_active":"2023-08-27T11:25:41.418905+0000","last_peered":"2023-08-27T11:25:41.418905+0000","last_clean":"2023-08-27T11:25:41.418905+0000","last_became_active":"2023-08-27T11:25:41.415364+0000","last_became_peered":"2023-08-27T11:25:41.415364+0000","last_unstale":"2023-08-27T11:25:41.418905+0000","last_undegraded":"2023-08-27T11:25:41.418905+0000","last_fullsized":"2023-08-27T11:25:41.418905+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:18:37.632552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397995+0000","last_change":"2023-08-27T11:25:41.397995+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397995+0000","last_undegraded":"2023-08-27T11:25:41.397995+0000","last_fullsized":"2023-08-27T11:25:41.397995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418831+0000","last_change":"2023-08-27T11:25:41.417094+0000","last_active":"2023-08-27T11:25:41.418831+0000","last_peered":"2023-08-27T11:25:41.418831+0000","last_clean":"2023-08-27T11:25:41.418831+0000","last_became_active":"2023-08-27T11:25:41.416795+0000","last_became_peered":"2023-08-27T11:25:41.416795+0000","last_unstale":"2023-08-27T11:25:41.418831+0000","last_undegraded":"2023-08-27T11:25:41.418831+0000","last_fullsized":"2023-08-27T11:25:41.418831+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:33:47.717016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397502+0000","last_change":"2023-08-27T11:25:41.397502+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397502+0000","last_undegraded":"2023-08-27T11:25:41.397502+0000","last_fullsized":"2023-08-27T11:25:41.397502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.413823+0000","last_change":"2023-08-27T11:25:41.408039+0000","last_active":"2023-08-27T11:25:41.413823+0000","last_peered":"2023-08-27T11:25:41.413823+0000","last_clean":"2023-08-27T11:25:41.413823+0000","last_became_active":"2023-08-27T11:25:41.407709+0000","last_became_peered":"2023-08-27T11:25:41.407709+0000","last_unstale":"2023-08-27T11:25:41.413823+0000","last_undegraded":"2023-08-27T11:25:41.413823+0000","last_fullsized":"2023-08-27T11:25:41.413823+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:34:44.092855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.404340+0000","last_change":"2023-08-27T11:25:41.404340+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.404340+0000","last_undegraded":"2023-08-27T11:25:41.404340+0000","last_fullsized":"2023-08-27T11:25:41.404340+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418975+0000","last_change":"2023-08-27T11:25:41.414018+0000","last_active":"2023-08-27T11:25:41.418975+0000","last_peered":"2023-08-27T11:25:41.418975+0000","last_clean":"2023-08-27T11:25:41.418975+0000","last_became_active":"2023-08-27T11:25:41.413719+0000","last_became_peered":"2023-08-27T11:25:41.413719+0000","last_unstale":"2023-08-27T11:25:41.418975+0000","last_undegraded":"2023-08-27T11:25:41.418975+0000","last_fullsized":"2023-08-27T11:25:41.418975+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:51:04.452644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398022+0000","last_change":"2023-08-27T11:25:41.398022+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398022+0000","last_undegraded":"2023-08-27T11:25:41.398022+0000","last_fullsized":"2023-08-27T11:25:41.398022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411981+0000","last_change":"2023-08-27T11:25:41.408665+0000","last_active":"2023-08-27T11:25:41.411981+0000","last_peered":"2023-08-27T11:25:41.411981+0000","last_clean":"2023-08-27T11:25:41.411981+0000","last_became_active":"2023-08-27T11:25:41.408402+0000","last_became_peered":"2023-08-27T11:25:41.408402+0000","last_unstale":"2023-08-27T11:25:41.411981+0000","last_undegraded":"2023-08-27T11:25:41.411981+0000","last_fullsized":"2023-08-27T11:25:41.411981+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:25:55.452027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395170+0000","last_change":"2023-08-27T11:25:41.395170+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395170+0000","last_undegraded":"2023-08-27T11:25:41.395170+0000","last_fullsized":"2023-08-27T11:25:41.395170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421291+0000","last_change":"2023-08-27T11:25:41.411569+0000","last_active":"2023-08-27T11:25:41.421291+0000","last_peered":"2023-08-27T11:25:41.421291+0000","last_clean":"2023-08-27T11:25:41.421291+0000","last_became_active":"2023-08-27T11:25:41.411296+0000","last_became_peered":"2023-08-27T11:25:41.411296+0000","last_unstale":"2023-08-27T11:25:41.421291+0000","last_undegraded":"2023-08-27T11:25:41.421291+0000","last_fullsized":"2023-08-27T11:25:41.421291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:19:40.101939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.406076+0000","last_change":"2023-08-27T11:25:41.406076+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.406076+0000","last_undegraded":"2023-08-27T11:25:41.406076+0000","last_fullsized":"2023-08-27T11:25:41.406076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.416239+0000","last_change":"2023-08-27T11:25:41.412882+0000","last_active":"2023-08-27T11:25:41.416239+0000","last_peered":"2023-08-27T11:25:41.416239+0000","last_clean":"2023-08-27T11:25:41.416239+0000","last_became_active":"2023-08-27T11:25:41.412635+0000","last_became_peered":"2023-08-27T11:25:41.412635+0000","last_unstale":"2023-08-27T11:25:41.416239+0000","last_undegraded":"2023-08-27T11:25:41.416239+0000","last_fullsized":"2023-08-27T11:25:41.416239+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:10.546956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396619+0000","last_change":"2023-08-27T11:25:41.396619+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396619+0000","last_undegraded":"2023-08-27T11:25:41.396619+0000","last_fullsized":"2023-08-27T11:25:41.396619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.406484+0000","last_change":"2023-08-27T11:25:41.405653+0000","last_active":"2023-08-27T11:25:41.406484+0000","last_peered":"2023-08-27T11:25:41.406484+0000","last_clean":"2023-08-27T11:25:41.406484+0000","last_became_active":"2023-08-27T11:25:41.405385+0000","last_became_peered":"2023-08-27T11:25:41.405385+0000","last_unstale":"2023-08-27T11:25:41.406484+0000","last_undegraded":"2023-08-27T11:25:41.406484+0000","last_fullsized":"2023-08-27T11:25:41.406484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:55.963991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395911+0000","last_change":"2023-08-27T11:25:41.395911+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395911+0000","last_undegraded":"2023-08-27T11:25:41.395911+0000","last_fullsized":"2023-08-27T11:25:41.395911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421836+0000","last_change":"2023-08-27T11:25:41.417399+0000","last_active":"2023-08-27T11:25:41.421836+0000","last_peered":"2023-08-27T11:25:41.421836+0000","last_clean":"2023-08-27T11:25:41.421836+0000","last_became_active":"2023-08-27T11:25:41.417076+0000","last_became_peered":"2023-08-27T11:25:41.417076+0000","last_unstale":"2023-08-27T11:25:41.421836+0000","last_undegraded":"2023-08-27T11:25:41.421836+0000","last_fullsized":"2023-08-27T11:25:41.421836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:55:39.763853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":133,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.409138+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:25:41.409138+0000","last_peered":"2023-08-27T11:25:41.409138+0000","last_clean":"2023-08-27T11:25:41.409138+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:25:41.409138+0000","last_undegraded":"2023-08-27T11:25:41.409138+0000","last_fullsized":"2023-08-27T11:25:41.409138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:02:20.142155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.404145+0000","last_change":"2023-08-27T11:25:41.404145+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.404145+0000","last_undegraded":"2023-08-27T11:25:41.404145+0000","last_fullsized":"2023-08-27T11:25:41.404145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411964+0000","last_change":"2023-08-27T11:25:41.405709+0000","last_active":"2023-08-27T11:25:41.411964+0000","last_peered":"2023-08-27T11:25:41.411964+0000","last_clean":"2023-08-27T11:25:41.411964+0000","last_became_active":"2023-08-27T11:25:41.405421+0000","last_became_peered":"2023-08-27T11:25:41.405421+0000","last_unstale":"2023-08-27T11:25:41.411964+0000","last_undegraded":"2023-08-27T11:25:41.411964+0000","last_fullsized":"2023-08-27T11:25:41.411964+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:50:55.448426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":133,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.408951+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:25:41.408951+0000","last_peered":"2023-08-27T11:25:41.408951+0000","last_clean":"2023-08-27T11:25:41.408951+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:25:41.408951+0000","last_undegraded":"2023-08-27T11:25:41.408951+0000","last_fullsized":"2023-08-27T11:25:41.408951+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:50:30.904830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400360+0000","last_change":"2023-08-27T11:25:41.400360+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400360+0000","last_undegraded":"2023-08-27T11:25:41.400360+0000","last_fullsized":"2023-08-27T11:25:41.400360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.410077+0000","last_change":"2023-08-27T11:25:41.407112+0000","last_active":"2023-08-27T11:25:41.410077+0000","last_peered":"2023-08-27T11:25:41.410077+0000","last_clean":"2023-08-27T11:25:41.410077+0000","last_became_active":"2023-08-27T11:25:41.406794+0000","last_became_peered":"2023-08-27T11:25:41.406794+0000","last_unstale":"2023-08-27T11:25:41.410077+0000","last_undegraded":"2023-08-27T11:25:41.410077+0000","last_fullsized":"2023-08-27T11:25:41.410077+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:01:40.046194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411219+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:25:41.411219+0000","last_peered":"2023-08-27T11:25:41.411219+0000","last_clean":"2023-08-27T11:25:41.411219+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:25:41.411219+0000","last_undegraded":"2023-08-27T11:25:41.411219+0000","last_fullsized":"2023-08-27T11:25:41.411219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:26:47.289516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.401250+0000","last_change":"2023-08-27T11:25:41.401250+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.401250+0000","last_undegraded":"2023-08-27T11:25:41.401250+0000","last_fullsized":"2023-08-27T11:25:41.401250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418369+0000","last_change":"2023-08-27T11:25:41.417819+0000","last_active":"2023-08-27T11:25:41.418369+0000","last_peered":"2023-08-27T11:25:41.418369+0000","last_clean":"2023-08-27T11:25:41.418369+0000","last_became_active":"2023-08-27T11:25:41.417532+0000","last_became_peered":"2023-08-27T11:25:41.417532+0000","last_unstale":"2023-08-27T11:25:41.418369+0000","last_undegraded":"2023-08-27T11:25:41.418369+0000","last_fullsized":"2023-08-27T11:25:41.418369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:34.714411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.399323+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:25:41.399323+0000","last_peered":"2023-08-27T11:25:41.399323+0000","last_clean":"2023-08-27T11:25:41.399323+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:25:41.399323+0000","last_undegraded":"2023-08-27T11:25:41.399323+0000","last_fullsized":"2023-08-27T11:25:41.399323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:14:18.441956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.405779+0000","last_change":"2023-08-27T11:25:41.405779+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.405779+0000","last_undegraded":"2023-08-27T11:25:41.405779+0000","last_fullsized":"2023-08-27T11:25:41.405779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421309+0000","last_change":"2023-08-27T11:25:41.417161+0000","last_active":"2023-08-27T11:25:41.421309+0000","last_peered":"2023-08-27T11:25:41.421309+0000","last_clean":"2023-08-27T11:25:41.421309+0000","last_became_active":"2023-08-27T11:25:41.416886+0000","last_became_peered":"2023-08-27T11:25:41.416886+0000","last_unstale":"2023-08-27T11:25:41.421309+0000","last_undegraded":"2023-08-27T11:25:41.421309+0000","last_fullsized":"2023-08-27T11:25:41.421309+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:28:04.377394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.400152+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:25:41.400152+0000","last_peered":"2023-08-27T11:25:41.400152+0000","last_clean":"2023-08-27T11:25:41.400152+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:25:41.400152+0000","last_undegraded":"2023-08-27T11:25:41.400152+0000","last_fullsized":"2023-08-27T11:25:41.400152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:28.303971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399721+0000","last_change":"2023-08-27T11:25:41.399721+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399721+0000","last_undegraded":"2023-08-27T11:25:41.399721+0000","last_fullsized":"2023-08-27T11:25:41.399721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.419014+0000","last_change":"2023-08-27T11:25:41.414866+0000","last_active":"2023-08-27T11:25:41.419014+0000","last_peered":"2023-08-27T11:25:41.419014+0000","last_clean":"2023-08-27T11:25:41.419014+0000","last_became_active":"2023-08-27T11:25:41.414525+0000","last_became_peered":"2023-08-27T11:25:41.414525+0000","last_unstale":"2023-08-27T11:25:41.419014+0000","last_undegraded":"2023-08-27T11:25:41.419014+0000","last_fullsized":"2023-08-27T11:25:41.419014+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:04.857971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":263,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.399176+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:25:41.399176+0000","last_peered":"2023-08-27T11:25:41.399176+0000","last_clean":"2023-08-27T11:25:41.399176+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:25:41.399176+0000","last_undegraded":"2023-08-27T11:25:41.399176+0000","last_fullsized":"2023-08-27T11:25:41.399176+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:00.081196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.399025+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:25:41.399025+0000","last_peered":"2023-08-27T11:25:41.399025+0000","last_clean":"2023-08-27T11:25:41.399025+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:25:41.399025+0000","last_undegraded":"2023-08-27T11:25:41.399025+0000","last_fullsized":"2023-08-27T11:25:41.399025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:39.476760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396941+0000","last_change":"2023-08-27T11:25:41.396941+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396941+0000","last_undegraded":"2023-08-27T11:25:41.396941+0000","last_fullsized":"2023-08-27T11:25:41.396941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.417585+0000","last_change":"2023-08-27T11:25:41.416796+0000","last_active":"2023-08-27T11:25:41.417585+0000","last_peered":"2023-08-27T11:25:41.417585+0000","last_clean":"2023-08-27T11:25:41.417585+0000","last_became_active":"2023-08-27T11:25:41.416559+0000","last_became_peered":"2023-08-27T11:25:41.416559+0000","last_unstale":"2023-08-27T11:25:41.417585+0000","last_undegraded":"2023-08-27T11:25:41.417585+0000","last_fullsized":"2023-08-27T11:25:41.417585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:06:23.811399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.393189+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:25:41.393189+0000","last_peered":"2023-08-27T11:25:41.393189+0000","last_clean":"2023-08-27T11:25:41.393189+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:25:41.393189+0000","last_undegraded":"2023-08-27T11:25:41.393189+0000","last_fullsized":"2023-08-27T11:25:41.393189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:49:20.590638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.404181+0000","last_change":"2023-08-27T11:25:41.404181+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.404181+0000","last_undegraded":"2023-08-27T11:25:41.404181+0000","last_fullsized":"2023-08-27T11:25:41.404181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421868+0000","last_change":"2023-08-27T11:25:41.417495+0000","last_active":"2023-08-27T11:25:41.421868+0000","last_peered":"2023-08-27T11:25:41.421868+0000","last_clean":"2023-08-27T11:25:41.421868+0000","last_became_active":"2023-08-27T11:25:41.416993+0000","last_became_peered":"2023-08-27T11:25:41.416993+0000","last_unstale":"2023-08-27T11:25:41.421868+0000","last_undegraded":"2023-08-27T11:25:41.421868+0000","last_fullsized":"2023-08-27T11:25:41.421868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:30.109945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.401782+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:25:41.401782+0000","last_peered":"2023-08-27T11:25:41.401782+0000","last_clean":"2023-08-27T11:25:41.401782+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:25:41.401782+0000","last_undegraded":"2023-08-27T11:25:41.401782+0000","last_fullsized":"2023-08-27T11:25:41.401782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:38:57.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":1008750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51200000000000001,"15min":0.48899999999999999},"min":{"1min":0.34000000000000002,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.66200000000000003,"5min":0.79600000000000004,"15min":0.93500000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.49399999999999999},"min":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.252},"max":{"1min":0.73499999999999999,"5min":0.74099999999999999,"15min":0.81499999999999995},"last":0.42799999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.34200000000000003,"15min":0.26600000000000001},"max":{"1min":0.72299999999999998,"5min":0.77500000000000002,"15min":0.872},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54200000000000004,"15min":0.51100000000000001},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.253},"max":{"1min":0.67800000000000005,"5min":0.71999999999999997,"15min":0.84999999999999998},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.48199999999999998},"min":{"1min":0.32400000000000001,"5min":0.29099999999999998,"15min":0.28299999999999997},"max":{"1min":0.63,"5min":0.71699999999999997,"15min":0.79500000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56499999999999995,"15min":0.54400000000000004},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.33800000000000002},"max":{"1min":0.73999999999999999,"5min":0.78700000000000003,"15min":0.90200000000000002},"last":0.44700000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.54800000000000004},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.748,"15min":0.84299999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.48199999999999998,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.70399999999999996,"5min":0.83499999999999996,"15min":0.91700000000000004},"last":0.50600000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49399999999999999,"15min":0.50800000000000001},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.29899999999999999},"max":{"1min":0.72499999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.312},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52900000000000003,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72099999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38200000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.31900000000000001},"max":{"1min":0.749,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.43099999999999999,"5min":0.377,"15min":0.36799999999999999},"max":{"1min":0.77300000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.35799999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80800000000000005,"5min":0.82399999999999995,"15min":0.86099999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.755,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73099999999999998,"5min":0.86199999999999999,"15min":0.89800000000000002},"last":0.54600000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.316,"5min":0.312,"15min":0.312},"max":{"1min":0.75900000000000001,"5min":0.80600000000000005,"15min":0.873},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75600000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58799999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64700000000000002,"5min":0.74199999999999999,"15min":0.85899999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55800000000000005,"15min":0.58799999999999997},"min":{"1min":0.41699999999999998,"5min":0.39400000000000002,"15min":0.377},"max":{"1min":0.68600000000000005,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.60099999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.78900000000000003,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.70199999999999996,"5min":0.84699999999999998,"15min":0.93799999999999994},"last":0.56999999999999995}]},{"osd":4,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.32400000000000001,"5min":0.307,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.437,"5min":0.375,"15min":0.374},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.61799999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33400000000000002},"max":{"1min":0.63400000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.60099999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.875,"15min":0.875},"last":0.70599999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.47599999999999998,"5min":0.47099999999999997,"15min":0.34899999999999998},"max":{"1min":0.68200000000000005,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":1008750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.31,"15min":0.63400000000000001},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":0.502,"5min":0.502,"15min":11.971},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.96899999999999997},"min":{"1min":0.22500000000000001,"5min":0.221,"15min":0.20799999999999999},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.23300000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33700000000000002,"15min":0.66400000000000003},"min":{"1min":0.247,"5min":0.221,"15min":0.221},"max":{"1min":0.46600000000000003,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.33800000000000002,"15min":0.34399999999999997},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":0.48599999999999999,"5min":0.53700000000000003,"15min":0.58599999999999997},"last":0.32300000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.38,"15min":0.67600000000000005},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":10.99},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36699999999999999,"15min":1.002},"min":{"1min":0.29399999999999998,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.505,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.35499999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57199999999999995,"15min":0.60299999999999998},"min":{"1min":0.44500000000000001,"5min":0.36099999999999999,"15min":0.33400000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":1.26},"last":0.623},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55800000000000005,"15min":0.84199999999999997},"min":{"1min":0.38900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.77600000000000002,"5min":0.86299999999999999,"15min":10.957000000000001},"last":0.54500000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57299999999999995,"15min":1.2},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.36199999999999999},"max":{"1min":0.79300000000000004,"5min":0.85299999999999998,"15min":23.016999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.60299999999999998},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.296},"last":0.503}]},{"osd":6,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.57999999999999996,"15min":0.88300000000000001},"min":{"1min":0.434,"5min":0.371,"15min":0.371},"max":{"1min":0.84899999999999998,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":1.2170000000000001},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.52300000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":1.212},"min":{"1min":0.45400000000000001,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.76500000000000001,"5min":0.89600000000000002,"15min":22.988},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.63900000000000001},"min":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86199999999999999,"5min":0.91600000000000004,"15min":1.327},"last":0.57399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":417021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.29299999999999998,"15min":0.31},"min":{"1min":0.214,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.45300000000000001,"5min":0.50900000000000001,"15min":0.52300000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.35099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.62},"last":0.27700000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.33900000000000002},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.54400000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.33700000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.223},"max":{"1min":0.49299999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.25}]},{"osd":3,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.34499999999999997},"min":{"1min":0.249,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.50800000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.36199999999999999,"15min":0.372},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.53800000000000003,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.309}]},{"osd":4,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.56000000000000005},"min":{"1min":0.35399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85399999999999998,"5min":0.92500000000000004,"15min":1.214},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.35799999999999998,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.81000000000000005,"15min":1.2569999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.371,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81000000000000005,"5min":0.82599999999999996,"15min":0.91400000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83999999999999997,"5min":0.91100000000000003,"15min":1.603},"last":0.52900000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79900000000000004,"5min":0.86099999999999999,"15min":0.97599999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73999999999999999,"5min":0.871,"15min":0.871},"last":0.48199999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75800000000000001,"5min":0.94199999999999995,"15min":1.383},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86199999999999999,"5min":0.88900000000000001,"15min":0.92000000000000004},"last":0.56899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.30099999999999999,"15min":0.30199999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20200000000000001},"max":{"1min":0.48899999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.317},"min":{"1min":0.23100000000000001,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.46700000000000003,"5min":0.52500000000000002,"15min":0.629},"last":0.30199999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34200000000000003,"15min":0.34799999999999998},"min":{"1min":0.23000000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52200000000000002,"5min":0.749,"15min":0.749},"last":0.28899999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36299999999999999,"15min":0.38400000000000001},"min":{"1min":0.26600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57099999999999995,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.27500000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.69499999999999995},"last":0.34100000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.42699999999999999,"5min":0.39600000000000002,"15min":0.35599999999999998},"max":{"1min":0.82199999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.42599999999999999,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.75,"5min":0.879,"15min":0.879},"last":0.433}]},{"osd":5,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51300000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.73499999999999999,"5min":0.85199999999999998,"15min":0.85299999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55600000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59699999999999998,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.79500000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574958,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.27700000000000002,"5min":0.30099999999999999,"15min":0.29999999999999999},"min":{"1min":0.21199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.34100000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.374},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32600000000000001,"15min":0.312},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46200000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.246}]},{"osd":1,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.32700000000000001,"15min":0.312},"min":{"1min":0.22700000000000001,"5min":0.21299999999999999,"15min":0.19500000000000001},"max":{"1min":0.48599999999999999,"5min":0.50600000000000001,"15min":0.60399999999999998},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.308,"5min":0.33400000000000002,"15min":0.34000000000000002},"min":{"1min":0.26500000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.38200000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"last":0.30199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44400000000000001,"5min":0.57199999999999995,"15min":0.57599999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.42999999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.26700000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.436,"5min":0.316,"15min":0.28599999999999998},"max":{"1min":0.68200000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68700000000000006,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.78400000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.36799999999999999,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.57299999999999995},"min":{"1min":0.442,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.67000000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.67400000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.373},"max":{"1min":0.73999999999999999,"5min":1.091,"15min":1.091},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60099999999999998,"15min":0.61499999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75600000000000001,"5min":1.111,"15min":1.111},"last":0.60299999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.61799999999999999},"min":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.36699999999999999},"max":{"1min":0.70199999999999996,"5min":1.032,"15min":1.236},"last":0.623},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.58499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68200000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.80700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.32700000000000001,"15min":0.30399999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.44},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.379,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.65100000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54800000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76300000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.433,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.70699999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.44,"5min":0.41799999999999998,"15min":0.41399999999999998},"max":{"1min":0.751,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66100000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58599999999999997,"15min":0.59499999999999997},"min":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.379},"max":{"1min":0.85999999999999999,"5min":0.879,"15min":0.879},"last":0.625}]},{"osd":6,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.55000000000000004},"min":{"1min":0.39600000000000002,"5min":0.37,"15min":0.31},"max":{"1min":0.78900000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.62,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.37,"15min":0.35299999999999998},"max":{"1min":0.76700000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.39300000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.82499999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.34200000000000003},"max":{"1min":0.79200000000000004,"5min":0.83799999999999997,"15min":0.89300000000000002},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574957,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.63300000000000001,"5min":0.78600000000000003,"15min":0.96899999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.29199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.60099999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70499999999999996,"5min":0.747,"15min":0.92200000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52400000000000002,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.63200000000000001,"5min":0.72899999999999998,"15min":1.075},"last":0.53500000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.78000000000000003,"15min":0.95699999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54600000000000004,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.65300000000000002,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.56299999999999994}]},{"osd":3,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72299999999999998,"5min":0.81000000000000005,"15min":1.03},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58199999999999996,"15min":0.59499999999999997},"min":{"1min":0.48499999999999999,"5min":0.36699999999999999,"15min":0.35199999999999998},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":1.026},"last":0.58699999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72999999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.441},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.315,"15min":0.26900000000000002},"max":{"1min":0.78900000000000003,"5min":0.86399999999999999,"15min":0.91400000000000003},"last":0.42099999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.76500000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.48399999999999999,"5min":0.36599999999999999,"15min":0.28799999999999998},"max":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.94199999999999995},"last":0.5}]},{"osd":7,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.51300000000000001,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.95899999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.61199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.83699999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:25:46.527 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:25:46.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.679+0000 7fd6a0fed700 1 Processor -- start 2023-08-27T11:25:46.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.679+0000 7fd6a0fed700 1 -- start start 2023-08-27T11:25:46.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.680+0000 7fd6a0fed700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd69c0b17e0 0x7fd69c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:46.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.680+0000 7fd6a0fed700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69c161850 con 0x7fd69c0b1400 2023-08-27T11:25:46.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.680+0000 7fd6a0fed700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69c0cfc10 con 0x7fd69c161ce0 2023-08-27T11:25:46.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.680+0000 7fd6a0fed700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69c161200 con 0x7fd69c0b17e0 2023-08-27T11:25:46.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.680+0000 7fd699d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd69c0b17e0 0x7fd69c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:46.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.680+0000 7fd69a59c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd69c0b1400 0x7fd69c06e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54446/0 (socket says 172.21.15.112:54446) 2023-08-27T11:25:46.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.680+0000 7fd69a59c700 1 -- 172.21.15.112:0/3564242055 learned_addr learned my addr 172.21.15.112:0/3564242055 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:46.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.680+0000 7fd699d9b700 1 -- 172.21.15.112:0/3564242055 >> v1:172.21.15.138:6789/0 conn(0x7fd69c161ce0 legacy=0x7fd69c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:46.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.680+0000 7fd699d9b700 1 -- 172.21.15.112:0/3564242055 >> v1:172.21.15.112:6789/0 conn(0x7fd69c0b1400 legacy=0x7fd69c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.680+0000 7fd699d9b700 1 -- 172.21.15.112:0/3564242055 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c1614e0 con 0x7fd69c0b17e0 2023-08-27T11:25:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.680+0000 7fd69959a700 1 -- 172.21.15.112:0/3564242055 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237366064 0 0) 0x7fd69c161850 con 0x7fd69c0b1400 2023-08-27T11:25:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.680+0000 7fd699d9b700 1 --2- 172.21.15.112:0/3564242055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c0b17e0 0x7fd69c0cf5a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd690009840 tx=0x7fd690009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6c24900dfaad9e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.681+0000 7fd69959a700 1 -- 172.21.15.112:0/3564242055 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690004230 con 0x7fd69c0b17e0 2023-08-27T11:25:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.681+0000 7fd69959a700 1 -- 172.21.15.112:0/3564242055 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd690004390 con 0x7fd69c0b17e0 2023-08-27T11:25:46.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.681+0000 7fd69959a700 1 -- 172.21.15.112:0/3564242055 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6900036a0 con 0x7fd69c0b17e0 2023-08-27T11:25:46.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.681+0000 7fd6a0fed700 1 -- 172.21.15.112:0/3564242055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c0b17e0 msgr2=0x7fd69c0cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.681+0000 7fd6a0fed700 1 --2- 172.21.15.112:0/3564242055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c0b17e0 0x7fd69c0cf5a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd690009840 tx=0x7fd690009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:46.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.681+0000 7fd6a0fed700 1 -- 172.21.15.112:0/3564242055 shutdown_connections 2023-08-27T11:25:46.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.681+0000 7fd6a0fed700 1 --2- 172.21.15.112:0/3564242055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c0b17e0 0x7fd69c0cf5a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.681+0000 7fd6a0fed700 1 -- 172.21.15.112:0/3564242055 >> 172.21.15.112:0/3564242055 conn(0x7fd69c0d0000 msgr2=0x7fd69c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:46.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.681+0000 7fd6a0fed700 1 -- 172.21.15.112:0/3564242055 shutdown_connections 2023-08-27T11:25:46.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.681+0000 7fd6a0fed700 1 -- 172.21.15.112:0/3564242055 wait complete. 2023-08-27T11:25:46.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.682+0000 7fd6a0fed700 1 Processor -- start 2023-08-27T11:25:46.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.683+0000 7fd6a0fed700 1 -- start start 2023-08-27T11:25:46.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.683+0000 7fd6a0fed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd69c0b1400 0x7fd69c147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:46.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.683+0000 7fd6a0fed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69c0b17e0 0x7fd69c0c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:46.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.683+0000 7fd6a0fed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c161ce0 0x7fd69c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:46.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.683+0000 7fd6a0fed700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0c7ba0 con 0x7fd69c161ce0 2023-08-27T11:25:46.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.683+0000 7fd6a0fed700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0c5ae0 con 0x7fd69c0b17e0 2023-08-27T11:25:46.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.683+0000 7fd6a0fed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0ca010 con 0x7fd69c0b1400 2023-08-27T11:25:46.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.683+0000 7fd69ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c161ce0 0x7fd69c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:46.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.683+0000 7fd69ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c161ce0 0x7fd69c15d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60072/0 (socket says 172.21.15.112:60072) 2023-08-27T11:25:46.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.683+0000 7fd69ad9d700 1 -- 172.21.15.112:0/2705136159 learned_addr learned my addr 172.21.15.112:0/2705136159 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:46.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.683+0000 7fd699d9b700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69c0b17e0 0x7fd69c0c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:46.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.683+0000 7fd69a59c700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd69c0b1400 0x7fd69c147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:46.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.684+0000 7fd69ad9d700 1 -- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd69c0b1400 msgr2=0x7fd69c147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.684+0000 7fd69ad9d700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd69c0b1400 0x7fd69c147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.684+0000 7fd69ad9d700 1 -- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69c0b17e0 msgr2=0x7fd69c0c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.684+0000 7fd69ad9d700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69c0b17e0 0x7fd69c0c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.684+0000 7fd69ad9d700 1 -- 172.21.15.112:0/2705136159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6900094a0 con 0x7fd69c161ce0 2023-08-27T11:25:46.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.684+0000 7fd69ad9d700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c161ce0 0x7fd69c15d900 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd69400b990 tx=0x7fd69400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:46.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.684+0000 7fd68affd700 1 -- 172.21.15.112:0/2705136159 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69400c9d0 con 0x7fd69c161ce0 2023-08-27T11:25:46.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.684+0000 7fd68affd700 1 -- 172.21.15.112:0/2705136159 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69400cb30 con 0x7fd69c161ce0 2023-08-27T11:25:46.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.684+0000 7fd6a0fed700 1 -- 172.21.15.112:0/2705136159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69c0c7270 con 0x7fd69c161ce0 2023-08-27T11:25:46.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.684+0000 7fd68affd700 1 -- 172.21.15.112:0/2705136159 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694014440 con 0x7fd69c161ce0 2023-08-27T11:25:46.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.684+0000 7fd6a0fed700 1 -- 172.21.15.112:0/2705136159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69c0c7830 con 0x7fd69c161ce0 2023-08-27T11:25:46.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.685+0000 7fd6a0fed700 1 -- 172.21.15.112:0/2705136159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69c0c7d50 con 0x7fd69c161ce0 2023-08-27T11:25:46.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.685+0000 7fd68affd700 1 -- 172.21.15.112:0/2705136159 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fd694006d50 con 0x7fd69c161ce0 2023-08-27T11:25:46.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.686+0000 7fd68affd700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd67c06a320 0x7fd67c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:46.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.686+0000 7fd68affd700 1 -- 172.21.15.112:0/2705136159 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd694088400 con 0x7fd69c161ce0 2023-08-27T11:25:46.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.686+0000 7fd69a59c700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd67c06a320 0x7fd67c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:46.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.686+0000 7fd69a59c700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd67c06a320 0x7fd67c06c7e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd68c000f60 tx=0x7fd68c007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:46.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.690+0000 7fd68affd700 1 -- 172.21.15.112:0/2705136159 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd69c0c7d50 con 0x7fd69c161ce0 2023-08-27T11:25:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.873+0000 7fd6a0fed700 1 -- 172.21.15.112:0/2705136159 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd69c153bf0 con 0x7fd67c06a320 2023-08-27T11:25:46.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.883+0000 7fd68affd700 1 -- 172.21.15.112:0/2705136159 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306600 (secure 0 0 0) 0x7fd68c0012b0 con 0x7fd67c06a320 2023-08-27T11:25:46.887 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:46.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.886+0000 7fd6a0fed700 1 -- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd67c06a320 msgr2=0x7fd67c06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.886+0000 7fd6a0fed700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd67c06a320 0x7fd67c06c7e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd68c000f60 tx=0x7fd68c007c00 comp rx=0 tx=0).stop 2023-08-27T11:25:46.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.886+0000 7fd6a0fed700 1 -- 172.21.15.112:0/2705136159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c161ce0 msgr2=0x7fd69c15d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:46.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.886+0000 7fd6a0fed700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c161ce0 0x7fd69c15d900 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd69400b990 tx=0x7fd69400bd50 comp rx=0 tx=0).stop 2023-08-27T11:25:46.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.886+0000 7fd6a0fed700 1 -- 172.21.15.112:0/2705136159 shutdown_connections 2023-08-27T11:25:46.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.886+0000 7fd6a0fed700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd67c06a320 0x7fd67c06c7e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.886+0000 7fd6a0fed700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c161ce0 0x7fd69c15d900 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.886+0000 7fd6a0fed700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd69c0b1400 0x7fd69c147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.886+0000 7fd6a0fed700 1 --2- 172.21.15.112:0/2705136159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69c0b17e0 0x7fd69c0c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.886+0000 7fd6a0fed700 1 -- 172.21.15.112:0/2705136159 >> 172.21.15.112:0/2705136159 conn(0x7fd69c0d0000 msgr2=0x7fd69c0d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.886+0000 7fd6a0fed700 1 -- 172.21.15.112:0/2705136159 shutdown_connections 2023-08-27T11:25:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:46.886+0000 7fd6a0fed700 1 -- 172.21.15.112:0/2705136159 wait complete. 2023-08-27T11:25:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:25:46.900 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2023-08-27T11:25:46.359922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257828,"kb_used_data":8356,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754716892,"statfs":{"total":773094113280,"available":772830097408,"internally_reserved":0,"allocated":8556544,"data_stored":4526431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.968705"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396787+0000","last_change":"2023-08-27T11:25:41.396787+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396787+0000","last_undegraded":"2023-08-27T11:25:41.396787+0000","last_fullsized":"2023-08-27T11:25:41.396787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.408023+0000","last_change":"2023-08-27T11:25:41.408023+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.408023+0000","last_undegraded":"2023-08-27T11:25:41.408023+0000","last_fullsized":"2023-08-27T11:25:41.408023+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396335+0000","last_change":"2023-08-27T11:25:41.396335+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396335+0000","last_undegraded":"2023-08-27T11:25:41.396335+0000","last_fullsized":"2023-08-27T11:25:41.396335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397019+0000","last_change":"2023-08-27T11:25:41.397019+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397019+0000","last_undegraded":"2023-08-27T11:25:41.397019+0000","last_fullsized":"2023-08-27T11:25:41.397019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399662+0000","last_change":"2023-08-27T11:25:41.399662+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399662+0000","last_undegraded":"2023-08-27T11:25:41.399662+0000","last_fullsized":"2023-08-27T11:25:41.399662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.406043+0000","last_change":"2023-08-27T11:25:41.406043+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.406043+0000","last_undegraded":"2023-08-27T11:25:41.406043+0000","last_fullsized":"2023-08-27T11:25:41.406043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400258+0000","last_change":"2023-08-27T11:25:41.400258+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400258+0000","last_undegraded":"2023-08-27T11:25:41.400258+0000","last_fullsized":"2023-08-27T11:25:41.400258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398869+0000","last_change":"2023-08-27T11:25:41.398869+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398869+0000","last_undegraded":"2023-08-27T11:25:41.398869+0000","last_fullsized":"2023-08-27T11:25:41.398869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398915+0000","last_change":"2023-08-27T11:25:41.398915+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398915+0000","last_undegraded":"2023-08-27T11:25:41.398915+0000","last_fullsized":"2023-08-27T11:25:41.398915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398563+0000","last_change":"2023-08-27T11:25:41.398563+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398563+0000","last_undegraded":"2023-08-27T11:25:41.398563+0000","last_fullsized":"2023-08-27T11:25:41.398563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.401519+0000","last_change":"2023-08-27T11:25:41.401519+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.401519+0000","last_undegraded":"2023-08-27T11:25:41.401519+0000","last_fullsized":"2023-08-27T11:25:41.401519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396135+0000","last_change":"2023-08-27T11:25:41.396135+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396135+0000","last_undegraded":"2023-08-27T11:25:41.396135+0000","last_fullsized":"2023-08-27T11:25:41.396135+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398767+0000","last_change":"2023-08-27T11:25:41.398767+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398767+0000","last_undegraded":"2023-08-27T11:25:41.398767+0000","last_fullsized":"2023-08-27T11:25:41.398767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.402238+0000","last_change":"2023-08-27T11:25:41.402238+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.402238+0000","last_undegraded":"2023-08-27T11:25:41.402238+0000","last_fullsized":"2023-08-27T11:25:41.402238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400006+0000","last_change":"2023-08-27T11:25:41.400006+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400006+0000","last_undegraded":"2023-08-27T11:25:41.400006+0000","last_fullsized":"2023-08-27T11:25:41.400006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403883+0000","last_change":"2023-08-27T11:25:41.403883+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403883+0000","last_undegraded":"2023-08-27T11:25:41.403883+0000","last_fullsized":"2023-08-27T11:25:41.403883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403836+0000","last_change":"2023-08-27T11:25:41.403836+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403836+0000","last_undegraded":"2023-08-27T11:25:41.403836+0000","last_fullsized":"2023-08-27T11:25:41.403836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396268+0000","last_change":"2023-08-27T11:25:41.396268+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396268+0000","last_undegraded":"2023-08-27T11:25:41.396268+0000","last_fullsized":"2023-08-27T11:25:41.396268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400999+0000","last_change":"2023-08-27T11:25:41.400999+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400999+0000","last_undegraded":"2023-08-27T11:25:41.400999+0000","last_fullsized":"2023-08-27T11:25:41.400999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399588+0000","last_change":"2023-08-27T11:25:41.399588+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399588+0000","last_undegraded":"2023-08-27T11:25:41.399588+0000","last_fullsized":"2023-08-27T11:25:41.399588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399193+0000","last_change":"2023-08-27T11:25:41.399193+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399193+0000","last_undegraded":"2023-08-27T11:25:41.399193+0000","last_fullsized":"2023-08-27T11:25:41.399193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400790+0000","last_change":"2023-08-27T11:25:41.400790+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400790+0000","last_undegraded":"2023-08-27T11:25:41.400790+0000","last_fullsized":"2023-08-27T11:25:41.400790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.407276+0000","last_change":"2023-08-27T11:25:41.407276+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.407276+0000","last_undegraded":"2023-08-27T11:25:41.407276+0000","last_fullsized":"2023-08-27T11:25:41.407276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395723+0000","last_change":"2023-08-27T11:25:41.395723+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395723+0000","last_undegraded":"2023-08-27T11:25:41.395723+0000","last_fullsized":"2023-08-27T11:25:41.395723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.408337+0000","last_change":"2023-08-27T11:25:41.408337+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.408337+0000","last_undegraded":"2023-08-27T11:25:41.408337+0000","last_fullsized":"2023-08-27T11:25:41.408337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.401694+0000","last_change":"2023-08-27T11:25:41.401694+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.401694+0000","last_undegraded":"2023-08-27T11:25:41.401694+0000","last_fullsized":"2023-08-27T11:25:41.401694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395708+0000","last_change":"2023-08-27T11:25:41.395708+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395708+0000","last_undegraded":"2023-08-27T11:25:41.395708+0000","last_fullsized":"2023-08-27T11:25:41.395708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399209+0000","last_change":"2023-08-27T11:25:41.399209+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399209+0000","last_undegraded":"2023-08-27T11:25:41.399209+0000","last_fullsized":"2023-08-27T11:25:41.399209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399718+0000","last_change":"2023-08-27T11:25:41.399718+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399718+0000","last_undegraded":"2023-08-27T11:25:41.399718+0000","last_fullsized":"2023-08-27T11:25:41.399718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400210+0000","last_change":"2023-08-27T11:25:41.400210+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400210+0000","last_undegraded":"2023-08-27T11:25:41.400210+0000","last_fullsized":"2023-08-27T11:25:41.400210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.409672+0000","last_change":"2023-08-27T11:25:41.409672+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.409672+0000","last_undegraded":"2023-08-27T11:25:41.409672+0000","last_fullsized":"2023-08-27T11:25:41.409672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403514+0000","last_change":"2023-08-27T11:25:41.403514+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403514+0000","last_undegraded":"2023-08-27T11:25:41.403514+0000","last_fullsized":"2023-08-27T11:25:41.403514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.402400+0000","last_change":"2023-08-27T11:25:41.402400+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.402400+0000","last_undegraded":"2023-08-27T11:25:41.402400+0000","last_fullsized":"2023-08-27T11:25:41.402400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.419031+0000","last_change":"2023-08-27T11:25:41.414884+0000","last_active":"2023-08-27T11:25:41.419031+0000","last_peered":"2023-08-27T11:25:41.419031+0000","last_clean":"2023-08-27T11:25:41.419031+0000","last_became_active":"2023-08-27T11:25:41.414549+0000","last_became_peered":"2023-08-27T11:25:41.414549+0000","last_unstale":"2023-08-27T11:25:41.419031+0000","last_undegraded":"2023-08-27T11:25:41.419031+0000","last_fullsized":"2023-08-27T11:25:41.419031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:31:13.509254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403856+0000","last_change":"2023-08-27T11:25:41.403856+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403856+0000","last_undegraded":"2023-08-27T11:25:41.403856+0000","last_fullsized":"2023-08-27T11:25:41.403856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.416537+0000","last_change":"2023-08-27T11:25:41.414847+0000","last_active":"2023-08-27T11:25:41.416537+0000","last_peered":"2023-08-27T11:25:41.416537+0000","last_clean":"2023-08-27T11:25:41.416537+0000","last_became_active":"2023-08-27T11:25:41.414532+0000","last_became_peered":"2023-08-27T11:25:41.414532+0000","last_unstale":"2023-08-27T11:25:41.416537+0000","last_undegraded":"2023-08-27T11:25:41.416537+0000","last_fullsized":"2023-08-27T11:25:41.416537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:52:27.302892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399003+0000","last_change":"2023-08-27T11:25:41.399003+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399003+0000","last_undegraded":"2023-08-27T11:25:41.399003+0000","last_fullsized":"2023-08-27T11:25:41.399003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.416340+0000","last_change":"2023-08-27T11:25:41.412179+0000","last_active":"2023-08-27T11:25:41.416340+0000","last_peered":"2023-08-27T11:25:41.416340+0000","last_clean":"2023-08-27T11:25:41.416340+0000","last_became_active":"2023-08-27T11:25:41.411940+0000","last_became_peered":"2023-08-27T11:25:41.411940+0000","last_unstale":"2023-08-27T11:25:41.416340+0000","last_undegraded":"2023-08-27T11:25:41.416340+0000","last_fullsized":"2023-08-27T11:25:41.416340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:01:10.875523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403413+0000","last_change":"2023-08-27T11:25:41.403413+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403413+0000","last_undegraded":"2023-08-27T11:25:41.403413+0000","last_fullsized":"2023-08-27T11:25:41.403413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418955+0000","last_change":"2023-08-27T11:25:41.414002+0000","last_active":"2023-08-27T11:25:41.418955+0000","last_peered":"2023-08-27T11:25:41.418955+0000","last_clean":"2023-08-27T11:25:41.418955+0000","last_became_active":"2023-08-27T11:25:41.413740+0000","last_became_peered":"2023-08-27T11:25:41.413740+0000","last_unstale":"2023-08-27T11:25:41.418955+0000","last_undegraded":"2023-08-27T11:25:41.418955+0000","last_fullsized":"2023-08-27T11:25:41.418955+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:26:23.153384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397310+0000","last_change":"2023-08-27T11:25:41.397310+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397310+0000","last_undegraded":"2023-08-27T11:25:41.397310+0000","last_fullsized":"2023-08-27T11:25:41.397310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.406853+0000","last_change":"2023-08-27T11:25:41.405587+0000","last_active":"2023-08-27T11:25:41.406853+0000","last_peered":"2023-08-27T11:25:41.406853+0000","last_clean":"2023-08-27T11:25:41.406853+0000","last_became_active":"2023-08-27T11:25:41.405255+0000","last_became_peered":"2023-08-27T11:25:41.405255+0000","last_unstale":"2023-08-27T11:25:41.406853+0000","last_undegraded":"2023-08-27T11:25:41.406853+0000","last_fullsized":"2023-08-27T11:25:41.406853+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:09:28.315159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.406628+0000","last_change":"2023-08-27T11:25:41.406628+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.406628+0000","last_undegraded":"2023-08-27T11:25:41.406628+0000","last_fullsized":"2023-08-27T11:25:41.406628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.409295+0000","last_change":"2023-08-27T11:25:41.406787+0000","last_active":"2023-08-27T11:25:41.409295+0000","last_peered":"2023-08-27T11:25:41.409295+0000","last_clean":"2023-08-27T11:25:41.409295+0000","last_became_active":"2023-08-27T11:25:41.406253+0000","last_became_peered":"2023-08-27T11:25:41.406253+0000","last_unstale":"2023-08-27T11:25:41.409295+0000","last_undegraded":"2023-08-27T11:25:41.409295+0000","last_fullsized":"2023-08-27T11:25:41.409295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:38.768166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397479+0000","last_change":"2023-08-27T11:25:41.397479+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397479+0000","last_undegraded":"2023-08-27T11:25:41.397479+0000","last_fullsized":"2023-08-27T11:25:41.397479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.408833+0000","last_change":"2023-08-27T11:25:41.404377+0000","last_active":"2023-08-27T11:25:41.408833+0000","last_peered":"2023-08-27T11:25:41.408833+0000","last_clean":"2023-08-27T11:25:41.408833+0000","last_became_active":"2023-08-27T11:25:41.404056+0000","last_became_peered":"2023-08-27T11:25:41.404056+0000","last_unstale":"2023-08-27T11:25:41.408833+0000","last_undegraded":"2023-08-27T11:25:41.408833+0000","last_fullsized":"2023-08-27T11:25:41.408833+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:16:51.600797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396184+0000","last_change":"2023-08-27T11:25:41.396184+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396184+0000","last_undegraded":"2023-08-27T11:25:41.396184+0000","last_fullsized":"2023-08-27T11:25:41.396184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411092+0000","last_change":"2023-08-27T11:25:41.407540+0000","last_active":"2023-08-27T11:25:41.411092+0000","last_peered":"2023-08-27T11:25:41.411092+0000","last_clean":"2023-08-27T11:25:41.411092+0000","last_became_active":"2023-08-27T11:25:41.406794+0000","last_became_peered":"2023-08-27T11:25:41.406794+0000","last_unstale":"2023-08-27T11:25:41.411092+0000","last_undegraded":"2023-08-27T11:25:41.411092+0000","last_fullsized":"2023-08-27T11:25:41.411092+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:10:38.722806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398510+0000","last_change":"2023-08-27T11:25:41.398510+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398510+0000","last_undegraded":"2023-08-27T11:25:41.398510+0000","last_fullsized":"2023-08-27T11:25:41.398510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421891+0000","last_change":"2023-08-27T11:25:41.417604+0000","last_active":"2023-08-27T11:25:41.421891+0000","last_peered":"2023-08-27T11:25:41.421891+0000","last_clean":"2023-08-27T11:25:41.421891+0000","last_became_active":"2023-08-27T11:25:41.416727+0000","last_became_peered":"2023-08-27T11:25:41.416727+0000","last_unstale":"2023-08-27T11:25:41.421891+0000","last_undegraded":"2023-08-27T11:25:41.421891+0000","last_fullsized":"2023-08-27T11:25:41.421891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:30.061949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398968+0000","last_change":"2023-08-27T11:25:41.398968+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398968+0000","last_undegraded":"2023-08-27T11:25:41.398968+0000","last_fullsized":"2023-08-27T11:25:41.398968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.406341+0000","last_change":"2023-08-27T11:25:41.402868+0000","last_active":"2023-08-27T11:25:41.406341+0000","last_peered":"2023-08-27T11:25:41.406341+0000","last_clean":"2023-08-27T11:25:41.406341+0000","last_became_active":"2023-08-27T11:25:41.402585+0000","last_became_peered":"2023-08-27T11:25:41.402585+0000","last_unstale":"2023-08-27T11:25:41.406341+0000","last_undegraded":"2023-08-27T11:25:41.406341+0000","last_fullsized":"2023-08-27T11:25:41.406341+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:24:45.015907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395873+0000","last_change":"2023-08-27T11:25:41.395873+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395873+0000","last_undegraded":"2023-08-27T11:25:41.395873+0000","last_fullsized":"2023-08-27T11:25:41.395873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.413847+0000","last_change":"2023-08-27T11:25:41.408134+0000","last_active":"2023-08-27T11:25:41.413847+0000","last_peered":"2023-08-27T11:25:41.413847+0000","last_clean":"2023-08-27T11:25:41.413847+0000","last_became_active":"2023-08-27T11:25:41.407806+0000","last_became_peered":"2023-08-27T11:25:41.407806+0000","last_unstale":"2023-08-27T11:25:41.413847+0000","last_undegraded":"2023-08-27T11:25:41.413847+0000","last_fullsized":"2023-08-27T11:25:41.413847+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:38:30.962549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396189+0000","last_change":"2023-08-27T11:25:41.396189+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396189+0000","last_undegraded":"2023-08-27T11:25:41.396189+0000","last_fullsized":"2023-08-27T11:25:41.396189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.415329+0000","last_change":"2023-08-27T11:25:41.414641+0000","last_active":"2023-08-27T11:25:41.415329+0000","last_peered":"2023-08-27T11:25:41.415329+0000","last_clean":"2023-08-27T11:25:41.415329+0000","last_became_active":"2023-08-27T11:25:41.414361+0000","last_became_peered":"2023-08-27T11:25:41.414361+0000","last_unstale":"2023-08-27T11:25:41.415329+0000","last_undegraded":"2023-08-27T11:25:41.415329+0000","last_fullsized":"2023-08-27T11:25:41.415329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:14.192009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396572+0000","last_change":"2023-08-27T11:25:41.396572+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396572+0000","last_undegraded":"2023-08-27T11:25:41.396572+0000","last_fullsized":"2023-08-27T11:25:41.396572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.413558+0000","last_change":"2023-08-27T11:25:41.410338+0000","last_active":"2023-08-27T11:25:41.413558+0000","last_peered":"2023-08-27T11:25:41.413558+0000","last_clean":"2023-08-27T11:25:41.413558+0000","last_became_active":"2023-08-27T11:25:41.410005+0000","last_became_peered":"2023-08-27T11:25:41.410005+0000","last_unstale":"2023-08-27T11:25:41.413558+0000","last_undegraded":"2023-08-27T11:25:41.413558+0000","last_fullsized":"2023-08-27T11:25:41.413558+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:22:28.572307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398507+0000","last_change":"2023-08-27T11:25:41.398507+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398507+0000","last_undegraded":"2023-08-27T11:25:41.398507+0000","last_fullsized":"2023-08-27T11:25:41.398507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.412023+0000","last_change":"2023-08-27T11:25:41.408888+0000","last_active":"2023-08-27T11:25:41.412023+0000","last_peered":"2023-08-27T11:25:41.412023+0000","last_clean":"2023-08-27T11:25:41.412023+0000","last_became_active":"2023-08-27T11:25:41.408609+0000","last_became_peered":"2023-08-27T11:25:41.408609+0000","last_unstale":"2023-08-27T11:25:41.412023+0000","last_undegraded":"2023-08-27T11:25:41.412023+0000","last_fullsized":"2023-08-27T11:25:41.412023+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:12:28.171979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395468+0000","last_change":"2023-08-27T11:25:41.395468+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395468+0000","last_undegraded":"2023-08-27T11:25:41.395468+0000","last_fullsized":"2023-08-27T11:25:41.395468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.415697+0000","last_change":"2023-08-27T11:25:41.414552+0000","last_active":"2023-08-27T11:25:41.415697+0000","last_peered":"2023-08-27T11:25:41.415697+0000","last_clean":"2023-08-27T11:25:41.415697+0000","last_became_active":"2023-08-27T11:25:41.414281+0000","last_became_peered":"2023-08-27T11:25:41.414281+0000","last_unstale":"2023-08-27T11:25:41.415697+0000","last_undegraded":"2023-08-27T11:25:41.415697+0000","last_fullsized":"2023-08-27T11:25:41.415697+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:40:04.874008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395170+0000","last_change":"2023-08-27T11:25:41.395170+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395170+0000","last_undegraded":"2023-08-27T11:25:41.395170+0000","last_fullsized":"2023-08-27T11:25:41.395170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411147+0000","last_change":"2023-08-27T11:25:41.407400+0000","last_active":"2023-08-27T11:25:41.411147+0000","last_peered":"2023-08-27T11:25:41.411147+0000","last_clean":"2023-08-27T11:25:41.411147+0000","last_became_active":"2023-08-27T11:25:41.405982+0000","last_became_peered":"2023-08-27T11:25:41.405982+0000","last_unstale":"2023-08-27T11:25:41.411147+0000","last_undegraded":"2023-08-27T11:25:41.411147+0000","last_fullsized":"2023-08-27T11:25:41.411147+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:59.630816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.407857+0000","last_change":"2023-08-27T11:25:41.407857+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.407857+0000","last_undegraded":"2023-08-27T11:25:41.407857+0000","last_fullsized":"2023-08-27T11:25:41.407857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418905+0000","last_change":"2023-08-27T11:25:41.415726+0000","last_active":"2023-08-27T11:25:41.418905+0000","last_peered":"2023-08-27T11:25:41.418905+0000","last_clean":"2023-08-27T11:25:41.418905+0000","last_became_active":"2023-08-27T11:25:41.415364+0000","last_became_peered":"2023-08-27T11:25:41.415364+0000","last_unstale":"2023-08-27T11:25:41.418905+0000","last_undegraded":"2023-08-27T11:25:41.418905+0000","last_fullsized":"2023-08-27T11:25:41.418905+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:18:37.632552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397995+0000","last_change":"2023-08-27T11:25:41.397995+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397995+0000","last_undegraded":"2023-08-27T11:25:41.397995+0000","last_fullsized":"2023-08-27T11:25:41.397995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418831+0000","last_change":"2023-08-27T11:25:41.417094+0000","last_active":"2023-08-27T11:25:41.418831+0000","last_peered":"2023-08-27T11:25:41.418831+0000","last_clean":"2023-08-27T11:25:41.418831+0000","last_became_active":"2023-08-27T11:25:41.416795+0000","last_became_peered":"2023-08-27T11:25:41.416795+0000","last_unstale":"2023-08-27T11:25:41.418831+0000","last_undegraded":"2023-08-27T11:25:41.418831+0000","last_fullsized":"2023-08-27T11:25:41.418831+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:33:47.717016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397502+0000","last_change":"2023-08-27T11:25:41.397502+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397502+0000","last_undegraded":"2023-08-27T11:25:41.397502+0000","last_fullsized":"2023-08-27T11:25:41.397502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.413823+0000","last_change":"2023-08-27T11:25:41.408039+0000","last_active":"2023-08-27T11:25:41.413823+0000","last_peered":"2023-08-27T11:25:41.413823+0000","last_clean":"2023-08-27T11:25:41.413823+0000","last_became_active":"2023-08-27T11:25:41.407709+0000","last_became_peered":"2023-08-27T11:25:41.407709+0000","last_unstale":"2023-08-27T11:25:41.413823+0000","last_undegraded":"2023-08-27T11:25:41.413823+0000","last_fullsized":"2023-08-27T11:25:41.413823+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:34:44.092855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.404340+0000","last_change":"2023-08-27T11:25:41.404340+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.404340+0000","last_undegraded":"2023-08-27T11:25:41.404340+0000","last_fullsized":"2023-08-27T11:25:41.404340+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418975+0000","last_change":"2023-08-27T11:25:41.414018+0000","last_active":"2023-08-27T11:25:41.418975+0000","last_peered":"2023-08-27T11:25:41.418975+0000","last_clean":"2023-08-27T11:25:41.418975+0000","last_became_active":"2023-08-27T11:25:41.413719+0000","last_became_peered":"2023-08-27T11:25:41.413719+0000","last_unstale":"2023-08-27T11:25:41.418975+0000","last_undegraded":"2023-08-27T11:25:41.418975+0000","last_fullsized":"2023-08-27T11:25:41.418975+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:51:04.452644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398022+0000","last_change":"2023-08-27T11:25:41.398022+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398022+0000","last_undegraded":"2023-08-27T11:25:41.398022+0000","last_fullsized":"2023-08-27T11:25:41.398022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411981+0000","last_change":"2023-08-27T11:25:41.408665+0000","last_active":"2023-08-27T11:25:41.411981+0000","last_peered":"2023-08-27T11:25:41.411981+0000","last_clean":"2023-08-27T11:25:41.411981+0000","last_became_active":"2023-08-27T11:25:41.408402+0000","last_became_peered":"2023-08-27T11:25:41.408402+0000","last_unstale":"2023-08-27T11:25:41.411981+0000","last_undegraded":"2023-08-27T11:25:41.411981+0000","last_fullsized":"2023-08-27T11:25:41.411981+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:25:55.452027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395170+0000","last_change":"2023-08-27T11:25:41.395170+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395170+0000","last_undegraded":"2023-08-27T11:25:41.395170+0000","last_fullsized":"2023-08-27T11:25:41.395170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421291+0000","last_change":"2023-08-27T11:25:41.411569+0000","last_active":"2023-08-27T11:25:41.421291+0000","last_peered":"2023-08-27T11:25:41.421291+0000","last_clean":"2023-08-27T11:25:41.421291+0000","last_became_active":"2023-08-27T11:25:41.411296+0000","last_became_peered":"2023-08-27T11:25:41.411296+0000","last_unstale":"2023-08-27T11:25:41.421291+0000","last_undegraded":"2023-08-27T11:25:41.421291+0000","last_fullsized":"2023-08-27T11:25:41.421291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:19:40.101939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.406076+0000","last_change":"2023-08-27T11:25:41.406076+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.406076+0000","last_undegraded":"2023-08-27T11:25:41.406076+0000","last_fullsized":"2023-08-27T11:25:41.406076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.416239+0000","last_change":"2023-08-27T11:25:41.412882+0000","last_active":"2023-08-27T11:25:41.416239+0000","last_peered":"2023-08-27T11:25:41.416239+0000","last_clean":"2023-08-27T11:25:41.416239+0000","last_became_active":"2023-08-27T11:25:41.412635+0000","last_became_peered":"2023-08-27T11:25:41.412635+0000","last_unstale":"2023-08-27T11:25:41.416239+0000","last_undegraded":"2023-08-27T11:25:41.416239+0000","last_fullsized":"2023-08-27T11:25:41.416239+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:10.546956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396619+0000","last_change":"2023-08-27T11:25:41.396619+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396619+0000","last_undegraded":"2023-08-27T11:25:41.396619+0000","last_fullsized":"2023-08-27T11:25:41.396619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.406484+0000","last_change":"2023-08-27T11:25:41.405653+0000","last_active":"2023-08-27T11:25:41.406484+0000","last_peered":"2023-08-27T11:25:41.406484+0000","last_clean":"2023-08-27T11:25:41.406484+0000","last_became_active":"2023-08-27T11:25:41.405385+0000","last_became_peered":"2023-08-27T11:25:41.405385+0000","last_unstale":"2023-08-27T11:25:41.406484+0000","last_undegraded":"2023-08-27T11:25:41.406484+0000","last_fullsized":"2023-08-27T11:25:41.406484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:55.963991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395911+0000","last_change":"2023-08-27T11:25:41.395911+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395911+0000","last_undegraded":"2023-08-27T11:25:41.395911+0000","last_fullsized":"2023-08-27T11:25:41.395911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421836+0000","last_change":"2023-08-27T11:25:41.417399+0000","last_active":"2023-08-27T11:25:41.421836+0000","last_peered":"2023-08-27T11:25:41.421836+0000","last_clean":"2023-08-27T11:25:41.421836+0000","last_became_active":"2023-08-27T11:25:41.417076+0000","last_became_peered":"2023-08-27T11:25:41.417076+0000","last_unstale":"2023-08-27T11:25:41.421836+0000","last_undegraded":"2023-08-27T11:25:41.421836+0000","last_fullsized":"2023-08-27T11:25:41.421836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:55:39.763853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":133,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.409138+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:25:41.409138+0000","last_peered":"2023-08-27T11:25:41.409138+0000","last_clean":"2023-08-27T11:25:41.409138+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:25:41.409138+0000","last_undegraded":"2023-08-27T11:25:41.409138+0000","last_fullsized":"2023-08-27T11:25:41.409138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:02:20.142155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.404145+0000","last_change":"2023-08-27T11:25:41.404145+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.404145+0000","last_undegraded":"2023-08-27T11:25:41.404145+0000","last_fullsized":"2023-08-27T11:25:41.404145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411964+0000","last_change":"2023-08-27T11:25:41.405709+0000","last_active":"2023-08-27T11:25:41.411964+0000","last_peered":"2023-08-27T11:25:41.411964+0000","last_clean":"2023-08-27T11:25:41.411964+0000","last_became_active":"2023-08-27T11:25:41.405421+0000","last_became_peered":"2023-08-27T11:25:41.405421+0000","last_unstale":"2023-08-27T11:25:41.411964+0000","last_undegraded":"2023-08-27T11:25:41.411964+0000","last_fullsized":"2023-08-27T11:25:41.411964+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:50:55.448426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":133,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.408951+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:25:41.408951+0000","last_peered":"2023-08-27T11:25:41.408951+0000","last_clean":"2023-08-27T11:25:41.408951+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:25:41.408951+0000","last_undegraded":"2023-08-27T11:25:41.408951+0000","last_fullsized":"2023-08-27T11:25:41.408951+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:50:30.904830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400360+0000","last_change":"2023-08-27T11:25:41.400360+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400360+0000","last_undegraded":"2023-08-27T11:25:41.400360+0000","last_fullsized":"2023-08-27T11:25:41.400360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.410077+0000","last_change":"2023-08-27T11:25:41.407112+0000","last_active":"2023-08-27T11:25:41.410077+0000","last_peered":"2023-08-27T11:25:41.410077+0000","last_clean":"2023-08-27T11:25:41.410077+0000","last_became_active":"2023-08-27T11:25:41.406794+0000","last_became_peered":"2023-08-27T11:25:41.406794+0000","last_unstale":"2023-08-27T11:25:41.410077+0000","last_undegraded":"2023-08-27T11:25:41.410077+0000","last_fullsized":"2023-08-27T11:25:41.410077+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:01:40.046194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411219+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:25:41.411219+0000","last_peered":"2023-08-27T11:25:41.411219+0000","last_clean":"2023-08-27T11:25:41.411219+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:25:41.411219+0000","last_undegraded":"2023-08-27T11:25:41.411219+0000","last_fullsized":"2023-08-27T11:25:41.411219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:26:47.289516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.401250+0000","last_change":"2023-08-27T11:25:41.401250+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.401250+0000","last_undegraded":"2023-08-27T11:25:41.401250+0000","last_fullsized":"2023-08-27T11:25:41.401250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418369+0000","last_change":"2023-08-27T11:25:41.417819+0000","last_active":"2023-08-27T11:25:41.418369+0000","last_peered":"2023-08-27T11:25:41.418369+0000","last_clean":"2023-08-27T11:25:41.418369+0000","last_became_active":"2023-08-27T11:25:41.417532+0000","last_became_peered":"2023-08-27T11:25:41.417532+0000","last_unstale":"2023-08-27T11:25:41.418369+0000","last_undegraded":"2023-08-27T11:25:41.418369+0000","last_fullsized":"2023-08-27T11:25:41.418369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:34.714411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.399323+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:25:41.399323+0000","last_peered":"2023-08-27T11:25:41.399323+0000","last_clean":"2023-08-27T11:25:41.399323+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:25:41.399323+0000","last_undegraded":"2023-08-27T11:25:41.399323+0000","last_fullsized":"2023-08-27T11:25:41.399323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:14:18.441956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.405779+0000","last_change":"2023-08-27T11:25:41.405779+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.405779+0000","last_undegraded":"2023-08-27T11:25:41.405779+0000","last_fullsized":"2023-08-27T11:25:41.405779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421309+0000","last_change":"2023-08-27T11:25:41.417161+0000","last_active":"2023-08-27T11:25:41.421309+0000","last_peered":"2023-08-27T11:25:41.421309+0000","last_clean":"2023-08-27T11:25:41.421309+0000","last_became_active":"2023-08-27T11:25:41.416886+0000","last_became_peered":"2023-08-27T11:25:41.416886+0000","last_unstale":"2023-08-27T11:25:41.421309+0000","last_undegraded":"2023-08-27T11:25:41.421309+0000","last_fullsized":"2023-08-27T11:25:41.421309+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:28:04.377394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.400152+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:25:41.400152+0000","last_peered":"2023-08-27T11:25:41.400152+0000","last_clean":"2023-08-27T11:25:41.400152+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:25:41.400152+0000","last_undegraded":"2023-08-27T11:25:41.400152+0000","last_fullsized":"2023-08-27T11:25:41.400152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:28.303971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399721+0000","last_change":"2023-08-27T11:25:41.399721+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399721+0000","last_undegraded":"2023-08-27T11:25:41.399721+0000","last_fullsized":"2023-08-27T11:25:41.399721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.419014+0000","last_change":"2023-08-27T11:25:41.414866+0000","last_active":"2023-08-27T11:25:41.419014+0000","last_peered":"2023-08-27T11:25:41.419014+0000","last_clean":"2023-08-27T11:25:41.419014+0000","last_became_active":"2023-08-27T11:25:41.414525+0000","last_became_peered":"2023-08-27T11:25:41.414525+0000","last_unstale":"2023-08-27T11:25:41.419014+0000","last_undegraded":"2023-08-27T11:25:41.419014+0000","last_fullsized":"2023-08-27T11:25:41.419014+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:04.857971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":263,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.399176+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:25:41.399176+0000","last_peered":"2023-08-27T11:25:41.399176+0000","last_clean":"2023-08-27T11:25:41.399176+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:25:41.399176+0000","last_undegraded":"2023-08-27T11:25:41.399176+0000","last_fullsized":"2023-08-27T11:25:41.399176+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:00.081196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.399025+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:25:41.399025+0000","last_peered":"2023-08-27T11:25:41.399025+0000","last_clean":"2023-08-27T11:25:41.399025+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:25:41.399025+0000","last_undegraded":"2023-08-27T11:25:41.399025+0000","last_fullsized":"2023-08-27T11:25:41.399025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:39.476760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396941+0000","last_change":"2023-08-27T11:25:41.396941+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396941+0000","last_undegraded":"2023-08-27T11:25:41.396941+0000","last_fullsized":"2023-08-27T11:25:41.396941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.417585+0000","last_change":"2023-08-27T11:25:41.416796+0000","last_active":"2023-08-27T11:25:41.417585+0000","last_peered":"2023-08-27T11:25:41.417585+0000","last_clean":"2023-08-27T11:25:41.417585+0000","last_became_active":"2023-08-27T11:25:41.416559+0000","last_became_peered":"2023-08-27T11:25:41.416559+0000","last_unstale":"2023-08-27T11:25:41.417585+0000","last_undegraded":"2023-08-27T11:25:41.417585+0000","last_fullsized":"2023-08-27T11:25:41.417585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:06:23.811399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.393189+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:25:41.393189+0000","last_peered":"2023-08-27T11:25:41.393189+0000","last_clean":"2023-08-27T11:25:41.393189+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:25:41.393189+0000","last_undegraded":"2023-08-27T11:25:41.393189+0000","last_fullsized":"2023-08-27T11:25:41.393189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:49:20.590638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.404181+0000","last_change":"2023-08-27T11:25:41.404181+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.404181+0000","last_undegraded":"2023-08-27T11:25:41.404181+0000","last_fullsized":"2023-08-27T11:25:41.404181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421868+0000","last_change":"2023-08-27T11:25:41.417495+0000","last_active":"2023-08-27T11:25:41.421868+0000","last_peered":"2023-08-27T11:25:41.421868+0000","last_clean":"2023-08-27T11:25:41.421868+0000","last_became_active":"2023-08-27T11:25:41.416993+0000","last_became_peered":"2023-08-27T11:25:41.416993+0000","last_unstale":"2023-08-27T11:25:41.421868+0000","last_undegraded":"2023-08-27T11:25:41.421868+0000","last_fullsized":"2023-08-27T11:25:41.421868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:30.109945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.401782+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:25:41.401782+0000","last_peered":"2023-08-27T11:25:41.401782+0000","last_clean":"2023-08-27T11:25:41.401782+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:25:41.401782+0000","last_undegraded":"2023-08-27T11:25:41.401782+0000","last_fullsized":"2023-08-27T11:25:41.401782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:38:57.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":1008750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51200000000000001,"15min":0.48899999999999999},"min":{"1min":0.34000000000000002,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.66200000000000003,"5min":0.79600000000000004,"15min":0.93500000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.49399999999999999},"min":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.252},"max":{"1min":0.73499999999999999,"5min":0.74099999999999999,"15min":0.81499999999999995},"last":0.42799999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.34200000000000003,"15min":0.26600000000000001},"max":{"1min":0.72299999999999998,"5min":0.77500000000000002,"15min":0.872},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54200000000000004,"15min":0.51100000000000001},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.253},"max":{"1min":0.67800000000000005,"5min":0.71999999999999997,"15min":0.84999999999999998},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.48199999999999998},"min":{"1min":0.32400000000000001,"5min":0.29099999999999998,"15min":0.28299999999999997},"max":{"1min":0.63,"5min":0.71699999999999997,"15min":0.79500000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56499999999999995,"15min":0.54400000000000004},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.33800000000000002},"max":{"1min":0.73999999999999999,"5min":0.78700000000000003,"15min":0.90200000000000002},"last":0.44700000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.54800000000000004},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.748,"15min":0.84299999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.48199999999999998,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.70399999999999996,"5min":0.83499999999999996,"15min":0.91700000000000004},"last":0.50600000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49399999999999999,"15min":0.50800000000000001},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.29899999999999999},"max":{"1min":0.72499999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.312},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52900000000000003,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72099999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38200000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.31900000000000001},"max":{"1min":0.749,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.43099999999999999,"5min":0.377,"15min":0.36799999999999999},"max":{"1min":0.77300000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.35799999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80800000000000005,"5min":0.82399999999999995,"15min":0.86099999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.755,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73099999999999998,"5min":0.86199999999999999,"15min":0.89800000000000002},"last":0.54600000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.316,"5min":0.312,"15min":0.312},"max":{"1min":0.75900000000000001,"5min":0.80600000000000005,"15min":0.873},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75600000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58799999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64700000000000002,"5min":0.74199999999999999,"15min":0.85899999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55800000000000005,"15min":0.58799999999999997},"min":{"1min":0.41699999999999998,"5min":0.39400000000000002,"15min":0.377},"max":{"1min":0.68600000000000005,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.60099999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.78900000000000003,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.70199999999999996,"5min":0.84699999999999998,"15min":0.93799999999999994},"last":0.56999999999999995}]},{"osd":4,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.32400000000000001,"5min":0.307,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.437,"5min":0.375,"15min":0.374},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.61799999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33400000000000002},"max":{"1min":0.63400000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.60099999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.875,"15min":0.875},"last":0.70599999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.47599999999999998,"5min":0.47099999999999997,"15min":0.34899999999999998},"max":{"1min":0.68200000000000005,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":1008750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.31,"15min":0.63400000000000001},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":0.502,"5min":0.502,"15min":11.971},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.96899999999999997},"min":{"1min":0.22500000000000001,"5min":0.221,"15min":0.20799999999999999},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.23300000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33700000000000002,"15min":0.66400000000000003},"min":{"1min":0.247,"5min":0.221,"15min":0.221},"max":{"1min":0.46600000000000003,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.33800000000000002,"15min":0.34399999999999997},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":0.48599999999999999,"5min":0.53700000000000003,"15min":0.58599999999999997},"last":0.32300000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.38,"15min":0.67600000000000005},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":10.99},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36699999999999999,"15min":1.002},"min":{"1min":0.29399999999999998,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.505,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.35499999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57199999999999995,"15min":0.60299999999999998},"min":{"1min":0.44500000000000001,"5min":0.36099999999999999,"15min":0.33400000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":1.26},"last":0.623},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55800000000000005,"15min":0.84199999999999997},"min":{"1min":0.38900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.77600000000000002,"5min":0.86299999999999999,"15min":10.957000000000001},"last":0.54500000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57299999999999995,"15min":1.2},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.36199999999999999},"max":{"1min":0.79300000000000004,"5min":0.85299999999999998,"15min":23.016999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.60299999999999998},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.296},"last":0.503}]},{"osd":6,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.57999999999999996,"15min":0.88300000000000001},"min":{"1min":0.434,"5min":0.371,"15min":0.371},"max":{"1min":0.84899999999999998,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":1.2170000000000001},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.52300000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":1.212},"min":{"1min":0.45400000000000001,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.76500000000000001,"5min":0.89600000000000002,"15min":22.988},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.63900000000000001},"min":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86199999999999999,"5min":0.91600000000000004,"15min":1.327},"last":0.57399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":417021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.29299999999999998,"15min":0.31},"min":{"1min":0.214,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.45300000000000001,"5min":0.50900000000000001,"15min":0.52300000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.35099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.62},"last":0.27700000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.33900000000000002},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.54400000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.33700000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.223},"max":{"1min":0.49299999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.25}]},{"osd":3,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.34499999999999997},"min":{"1min":0.249,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.50800000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.36199999999999999,"15min":0.372},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.53800000000000003,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.309}]},{"osd":4,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.56000000000000005},"min":{"1min":0.35399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85399999999999998,"5min":0.92500000000000004,"15min":1.214},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.35799999999999998,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.81000000000000005,"15min":1.2569999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.371,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81000000000000005,"5min":0.82599999999999996,"15min":0.91400000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83999999999999997,"5min":0.91100000000000003,"15min":1.603},"last":0.52900000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79900000000000004,"5min":0.86099999999999999,"15min":0.97599999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73999999999999999,"5min":0.871,"15min":0.871},"last":0.48199999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75800000000000001,"5min":0.94199999999999995,"15min":1.383},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86199999999999999,"5min":0.88900000000000001,"15min":0.92000000000000004},"last":0.56899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.30099999999999999,"15min":0.30199999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20200000000000001},"max":{"1min":0.48899999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.317},"min":{"1min":0.23100000000000001,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.46700000000000003,"5min":0.52500000000000002,"15min":0.629},"last":0.30199999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34200000000000003,"15min":0.34799999999999998},"min":{"1min":0.23000000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52200000000000002,"5min":0.749,"15min":0.749},"last":0.28899999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36299999999999999,"15min":0.38400000000000001},"min":{"1min":0.26600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57099999999999995,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.27500000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.69499999999999995},"last":0.34100000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.42699999999999999,"5min":0.39600000000000002,"15min":0.35599999999999998},"max":{"1min":0.82199999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.42599999999999999,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.75,"5min":0.879,"15min":0.879},"last":0.433}]},{"osd":5,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51300000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.73499999999999999,"5min":0.85199999999999998,"15min":0.85299999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55600000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59699999999999998,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.79500000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574958,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.27700000000000002,"5min":0.30099999999999999,"15min":0.29999999999999999},"min":{"1min":0.21199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.34100000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.374},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32600000000000001,"15min":0.312},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46200000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.246}]},{"osd":1,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.32700000000000001,"15min":0.312},"min":{"1min":0.22700000000000001,"5min":0.21299999999999999,"15min":0.19500000000000001},"max":{"1min":0.48599999999999999,"5min":0.50600000000000001,"15min":0.60399999999999998},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.308,"5min":0.33400000000000002,"15min":0.34000000000000002},"min":{"1min":0.26500000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.38200000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"last":0.30199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44400000000000001,"5min":0.57199999999999995,"15min":0.57599999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.42999999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.26700000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.436,"5min":0.316,"15min":0.28599999999999998},"max":{"1min":0.68200000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68700000000000006,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.78400000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.36799999999999999,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.57299999999999995},"min":{"1min":0.442,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.67000000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.67400000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.373},"max":{"1min":0.73999999999999999,"5min":1.091,"15min":1.091},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60099999999999998,"15min":0.61499999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75600000000000001,"5min":1.111,"15min":1.111},"last":0.60299999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.61799999999999999},"min":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.36699999999999999},"max":{"1min":0.70199999999999996,"5min":1.032,"15min":1.236},"last":0.623},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.58499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68200000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.80700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.32700000000000001,"15min":0.30399999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.44},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.379,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.65100000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54800000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76300000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.433,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.70699999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.44,"5min":0.41799999999999998,"15min":0.41399999999999998},"max":{"1min":0.751,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66100000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58599999999999997,"15min":0.59499999999999997},"min":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.379},"max":{"1min":0.85999999999999999,"5min":0.879,"15min":0.879},"last":0.625}]},{"osd":6,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.55000000000000004},"min":{"1min":0.39600000000000002,"5min":0.37,"15min":0.31},"max":{"1min":0.78900000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.62,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.37,"15min":0.35299999999999998},"max":{"1min":0.76700000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.39300000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.82499999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.34200000000000003},"max":{"1min":0.79200000000000004,"5min":0.83799999999999997,"15min":0.89300000000000002},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574957,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.63300000000000001,"5min":0.78600000000000003,"15min":0.96899999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.29199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.60099999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70499999999999996,"5min":0.747,"15min":0.92200000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52400000000000002,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.63200000000000001,"5min":0.72899999999999998,"15min":1.075},"last":0.53500000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.78000000000000003,"15min":0.95699999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54600000000000004,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.65300000000000002,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.56299999999999994}]},{"osd":3,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72299999999999998,"5min":0.81000000000000005,"15min":1.03},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58199999999999996,"15min":0.59499999999999997},"min":{"1min":0.48499999999999999,"5min":0.36699999999999999,"15min":0.35199999999999998},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":1.026},"last":0.58699999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72999999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.441},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.315,"15min":0.26900000000000002},"max":{"1min":0.78900000000000003,"5min":0.86399999999999999,"15min":0.91400000000000003},"last":0.42099999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.76500000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.48399999999999999,"5min":0.36599999999999999,"15min":0.28799999999999998},"max":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.94199999999999995},"last":0.5}]},{"osd":7,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.51300000000000001,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.95899999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.61199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.83699999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:25:46.906 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:25:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.058+0000 7f8cf4391700 1 Processor -- start 2023-08-27T11:25:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.058+0000 7f8cf4391700 1 -- start start 2023-08-27T11:25:47.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.059+0000 7f8cf4391700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8cec144570 0x7f8cec15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.059+0000 7f8cf4391700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cec0b45f0 con 0x7f8cec0b4ad0 2023-08-27T11:25:47.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.059+0000 7f8cf4391700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cec0b4310 con 0x7f8cec0b3e50 2023-08-27T11:25:47.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.059+0000 7f8cf4391700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cec0cf7a0 con 0x7f8cec144570 2023-08-27T11:25:47.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.060+0000 7f8cf292e700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8cec0b4ad0 0x7f8cec0b4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54462/0 (socket says 172.21.15.112:54462) 2023-08-27T11:25:47.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.060+0000 7f8cf292e700 1 -- 172.21.15.112:0/2527486988 learned_addr learned my addr 172.21.15.112:0/2527486988 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:47.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.060+0000 7f8cf212d700 1 --2- 172.21.15.112:0/2527486988 >> v2:172.21.15.138:3300/0 conn(0x7f8cec144570 0x7f8cec15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.060+0000 7f8cf112b700 1 -- 172.21.15.112:0/2527486988 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102006968 0 0) 0x7f8cec0b45f0 con 0x7f8cec0b4ad0 2023-08-27T11:25:47.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.060+0000 7f8cf112b700 1 -- 172.21.15.112:0/2527486988 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cd80035b0 con 0x7f8cec0b4ad0 2023-08-27T11:25:47.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.060+0000 7f8cf212d700 1 -- 172.21.15.112:0/2527486988 >> v1:172.21.15.138:6789/0 conn(0x7f8cec0b3e50 legacy=0x7f8cec15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.060+0000 7f8cf212d700 1 -- 172.21.15.112:0/2527486988 >> v1:172.21.15.112:6789/0 conn(0x7f8cec0b4ad0 legacy=0x7f8cec0b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.060+0000 7f8cf112b700 1 -- 172.21.15.112:0/2527486988 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 901612224 0 0) 0x7f8cd80035b0 con 0x7f8cec0b4ad0 2023-08-27T11:25:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.060+0000 7f8cf212d700 1 -- 172.21.15.112:0/2527486988 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec0cdcb0 con 0x7f8cec144570 2023-08-27T11:25:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.060+0000 7f8cf212d700 1 --2- 172.21.15.112:0/2527486988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cec144570 0x7f8cec15ee20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009840 tx=0x7f8cdc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdb0dad0d51b032e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.060+0000 7f8cf112b700 1 -- 172.21.15.112:0/2527486988 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce40019f0 con 0x7f8cec144570 2023-08-27T11:25:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.060+0000 7f8cf112b700 1 -- 172.21.15.112:0/2527486988 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cdc004030 con 0x7f8cec144570 2023-08-27T11:25:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.061+0000 7f8cf112b700 1 -- 172.21.15.112:0/2527486988 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc0043a0 con 0x7f8cec144570 2023-08-27T11:25:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.061+0000 7f8cf4391700 1 -- 172.21.15.112:0/2527486988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cec144570 msgr2=0x7f8cec15ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.061+0000 7f8cf4391700 1 --2- 172.21.15.112:0/2527486988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cec144570 0x7f8cec15ee20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009840 tx=0x7f8cdc009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.061+0000 7f8cf4391700 1 -- 172.21.15.112:0/2527486988 shutdown_connections 2023-08-27T11:25:47.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.061+0000 7f8cf4391700 1 --2- 172.21.15.112:0/2527486988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cec144570 0x7f8cec15ee20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.061+0000 7f8cf4391700 1 -- 172.21.15.112:0/2527486988 >> 172.21.15.112:0/2527486988 conn(0x7f8cec06f970 msgr2=0x7f8cec06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:47.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.061+0000 7f8cf4391700 1 -- 172.21.15.112:0/2527486988 shutdown_connections 2023-08-27T11:25:47.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.061+0000 7f8cf4391700 1 -- 172.21.15.112:0/2527486988 wait complete. 2023-08-27T11:25:47.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.062+0000 7f8cf4391700 1 Processor -- start 2023-08-27T11:25:47.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.062+0000 7f8cf4391700 1 -- start start 2023-08-27T11:25:47.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf4391700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cec0b3e50 0x7f8cec1f07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf4391700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cec0b4ad0 0x7f8cec1f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf4391700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cec144570 0x7f8cec139e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf4391700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec137ed0 con 0x7f8cec144570 2023-08-27T11:25:47.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf4391700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec137530 con 0x7f8cec0b4ad0 2023-08-27T11:25:47.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf4391700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec137810 con 0x7f8cec0b3e50 2023-08-27T11:25:47.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf292e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cec144570 0x7f8cec139e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf292e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cec144570 0x7f8cec139e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60086/0 (socket says 172.21.15.112:60086) 2023-08-27T11:25:47.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf292e700 1 -- 172.21.15.112:0/3960454232 learned_addr learned my addr 172.21.15.112:0/3960454232 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:47.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf212d700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cec0b3e50 0x7f8cec1f07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf192c700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cec0b4ad0 0x7f8cec1f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf292e700 1 -- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cec0b3e50 msgr2=0x7f8cec1f07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf292e700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cec0b3e50 0x7f8cec1f07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf292e700 1 -- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cec0b4ad0 msgr2=0x7f8cec1f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.063+0000 7f8cf292e700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cec0b4ad0 0x7f8cec1f34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.064+0000 7f8cf292e700 1 -- 172.21.15.112:0/3960454232 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc0094a0 con 0x7f8cec144570 2023-08-27T11:25:47.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.064+0000 7f8cf212d700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cec0b3e50 0x7f8cec1f07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:47.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.064+0000 7f8cf192c700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cec0b4ad0 0x7f8cec1f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:47.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.064+0000 7f8cf292e700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cec144570 0x7f8cec139e00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4001ba0 tx=0x7f8ce400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:47.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.064+0000 7f8ce2ffd700 1 -- 172.21.15.112:0/3960454232 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400ac20 con 0x7f8cec144570 2023-08-27T11:25:47.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.064+0000 7f8ce2ffd700 1 -- 172.21.15.112:0/3960454232 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce400ad80 con 0x7f8cec144570 2023-08-27T11:25:47.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.064+0000 7f8cf4391700 1 -- 172.21.15.112:0/3960454232 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cec137af0 con 0x7f8cec144570 2023-08-27T11:25:47.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.064+0000 7f8ce2ffd700 1 -- 172.21.15.112:0/3960454232 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4007310 con 0x7f8cec144570 2023-08-27T11:25:47.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.064+0000 7f8cf4391700 1 -- 172.21.15.112:0/3960454232 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cec1f3a20 con 0x7f8cec144570 2023-08-27T11:25:47.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.065+0000 7f8cf4391700 1 -- 172.21.15.112:0/3960454232 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cec067a20 con 0x7f8cec144570 2023-08-27T11:25:47.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.065+0000 7f8ce2ffd700 1 -- 172.21.15.112:0/3960454232 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f8ce4007470 con 0x7f8cec144570 2023-08-27T11:25:47.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.066+0000 7f8ce2ffd700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cd006a320 0x7f8cd006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.066+0000 7f8ce2ffd700 1 -- 172.21.15.112:0/3960454232 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8ce40884a0 con 0x7f8cec144570 2023-08-27T11:25:47.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.066+0000 7f8cf212d700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cd006a320 0x7f8cd006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.066+0000 7f8cf212d700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cd006a320 0x7f8cd006c7e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0090b0 tx=0x7f8cdc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:47.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.070+0000 7f8ce2ffd700 1 -- 172.21.15.112:0/3960454232 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8ce40538c0 con 0x7f8cec144570 2023-08-27T11:25:47.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.253+0000 7f8cf4391700 1 -- 172.21.15.112:0/3960454232 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8cec0cdf90 con 0x7f8cd006a320 2023-08-27T11:25:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.262+0000 7f8ce2ffd700 1 -- 172.21.15.112:0/3960454232 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306600 (secure 0 0 0) 0x7f8cdc0036a0 con 0x7f8cd006a320 2023-08-27T11:25:47.268 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:47.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.265+0000 7f8cf4391700 1 -- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cd006a320 msgr2=0x7f8cd006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.265+0000 7f8cf4391700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cd006a320 0x7f8cd006c7e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0090b0 tx=0x7f8cdc008030 comp rx=0 tx=0).stop 2023-08-27T11:25:47.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.265+0000 7f8cf4391700 1 -- 172.21.15.112:0/3960454232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cec144570 msgr2=0x7f8cec139e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.265+0000 7f8cf4391700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cec144570 0x7f8cec139e00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4001ba0 tx=0x7f8ce400a060 comp rx=0 tx=0).stop 2023-08-27T11:25:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.265+0000 7f8cf4391700 1 -- 172.21.15.112:0/3960454232 shutdown_connections 2023-08-27T11:25:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.265+0000 7f8cf4391700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8cd006a320 0x7f8cd006c7e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.265+0000 7f8cf4391700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cec144570 0x7f8cec139e00 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.265+0000 7f8cf4391700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cec0b3e50 0x7f8cec1f07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.265+0000 7f8cf4391700 1 --2- 172.21.15.112:0/3960454232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cec0b4ad0 0x7f8cec1f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.265+0000 7f8cf4391700 1 -- 172.21.15.112:0/3960454232 >> 172.21.15.112:0/3960454232 conn(0x7f8cec06f970 msgr2=0x7f8cec140190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.266+0000 7f8cf4391700 1 -- 172.21.15.112:0/3960454232 shutdown_connections 2023-08-27T11:25:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.266+0000 7f8cf4391700 1 -- 172.21.15.112:0/3960454232 wait complete. 2023-08-27T11:25:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:25:47.279 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2023-08-27T11:25:46.359922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257828,"kb_used_data":8356,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754716892,"statfs":{"total":773094113280,"available":772830097408,"internally_reserved":0,"allocated":8556544,"data_stored":4526431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.968705"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396787+0000","last_change":"2023-08-27T11:25:41.396787+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396787+0000","last_undegraded":"2023-08-27T11:25:41.396787+0000","last_fullsized":"2023-08-27T11:25:41.396787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.408023+0000","last_change":"2023-08-27T11:25:41.408023+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.408023+0000","last_undegraded":"2023-08-27T11:25:41.408023+0000","last_fullsized":"2023-08-27T11:25:41.408023+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396335+0000","last_change":"2023-08-27T11:25:41.396335+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396335+0000","last_undegraded":"2023-08-27T11:25:41.396335+0000","last_fullsized":"2023-08-27T11:25:41.396335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397019+0000","last_change":"2023-08-27T11:25:41.397019+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397019+0000","last_undegraded":"2023-08-27T11:25:41.397019+0000","last_fullsized":"2023-08-27T11:25:41.397019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399662+0000","last_change":"2023-08-27T11:25:41.399662+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399662+0000","last_undegraded":"2023-08-27T11:25:41.399662+0000","last_fullsized":"2023-08-27T11:25:41.399662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.406043+0000","last_change":"2023-08-27T11:25:41.406043+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.406043+0000","last_undegraded":"2023-08-27T11:25:41.406043+0000","last_fullsized":"2023-08-27T11:25:41.406043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400258+0000","last_change":"2023-08-27T11:25:41.400258+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400258+0000","last_undegraded":"2023-08-27T11:25:41.400258+0000","last_fullsized":"2023-08-27T11:25:41.400258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398869+0000","last_change":"2023-08-27T11:25:41.398869+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398869+0000","last_undegraded":"2023-08-27T11:25:41.398869+0000","last_fullsized":"2023-08-27T11:25:41.398869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398915+0000","last_change":"2023-08-27T11:25:41.398915+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398915+0000","last_undegraded":"2023-08-27T11:25:41.398915+0000","last_fullsized":"2023-08-27T11:25:41.398915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398563+0000","last_change":"2023-08-27T11:25:41.398563+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398563+0000","last_undegraded":"2023-08-27T11:25:41.398563+0000","last_fullsized":"2023-08-27T11:25:41.398563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.401519+0000","last_change":"2023-08-27T11:25:41.401519+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.401519+0000","last_undegraded":"2023-08-27T11:25:41.401519+0000","last_fullsized":"2023-08-27T11:25:41.401519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396135+0000","last_change":"2023-08-27T11:25:41.396135+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396135+0000","last_undegraded":"2023-08-27T11:25:41.396135+0000","last_fullsized":"2023-08-27T11:25:41.396135+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398767+0000","last_change":"2023-08-27T11:25:41.398767+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398767+0000","last_undegraded":"2023-08-27T11:25:41.398767+0000","last_fullsized":"2023-08-27T11:25:41.398767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.402238+0000","last_change":"2023-08-27T11:25:41.402238+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.402238+0000","last_undegraded":"2023-08-27T11:25:41.402238+0000","last_fullsized":"2023-08-27T11:25:41.402238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400006+0000","last_change":"2023-08-27T11:25:41.400006+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400006+0000","last_undegraded":"2023-08-27T11:25:41.400006+0000","last_fullsized":"2023-08-27T11:25:41.400006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403883+0000","last_change":"2023-08-27T11:25:41.403883+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403883+0000","last_undegraded":"2023-08-27T11:25:41.403883+0000","last_fullsized":"2023-08-27T11:25:41.403883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403836+0000","last_change":"2023-08-27T11:25:41.403836+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403836+0000","last_undegraded":"2023-08-27T11:25:41.403836+0000","last_fullsized":"2023-08-27T11:25:41.403836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396268+0000","last_change":"2023-08-27T11:25:41.396268+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396268+0000","last_undegraded":"2023-08-27T11:25:41.396268+0000","last_fullsized":"2023-08-27T11:25:41.396268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400999+0000","last_change":"2023-08-27T11:25:41.400999+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400999+0000","last_undegraded":"2023-08-27T11:25:41.400999+0000","last_fullsized":"2023-08-27T11:25:41.400999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399588+0000","last_change":"2023-08-27T11:25:41.399588+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399588+0000","last_undegraded":"2023-08-27T11:25:41.399588+0000","last_fullsized":"2023-08-27T11:25:41.399588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399193+0000","last_change":"2023-08-27T11:25:41.399193+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399193+0000","last_undegraded":"2023-08-27T11:25:41.399193+0000","last_fullsized":"2023-08-27T11:25:41.399193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400790+0000","last_change":"2023-08-27T11:25:41.400790+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400790+0000","last_undegraded":"2023-08-27T11:25:41.400790+0000","last_fullsized":"2023-08-27T11:25:41.400790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.407276+0000","last_change":"2023-08-27T11:25:41.407276+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.407276+0000","last_undegraded":"2023-08-27T11:25:41.407276+0000","last_fullsized":"2023-08-27T11:25:41.407276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395723+0000","last_change":"2023-08-27T11:25:41.395723+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395723+0000","last_undegraded":"2023-08-27T11:25:41.395723+0000","last_fullsized":"2023-08-27T11:25:41.395723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.408337+0000","last_change":"2023-08-27T11:25:41.408337+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.408337+0000","last_undegraded":"2023-08-27T11:25:41.408337+0000","last_fullsized":"2023-08-27T11:25:41.408337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.401694+0000","last_change":"2023-08-27T11:25:41.401694+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.401694+0000","last_undegraded":"2023-08-27T11:25:41.401694+0000","last_fullsized":"2023-08-27T11:25:41.401694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395708+0000","last_change":"2023-08-27T11:25:41.395708+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395708+0000","last_undegraded":"2023-08-27T11:25:41.395708+0000","last_fullsized":"2023-08-27T11:25:41.395708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399209+0000","last_change":"2023-08-27T11:25:41.399209+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399209+0000","last_undegraded":"2023-08-27T11:25:41.399209+0000","last_fullsized":"2023-08-27T11:25:41.399209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399718+0000","last_change":"2023-08-27T11:25:41.399718+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399718+0000","last_undegraded":"2023-08-27T11:25:41.399718+0000","last_fullsized":"2023-08-27T11:25:41.399718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400210+0000","last_change":"2023-08-27T11:25:41.400210+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400210+0000","last_undegraded":"2023-08-27T11:25:41.400210+0000","last_fullsized":"2023-08-27T11:25:41.400210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.409672+0000","last_change":"2023-08-27T11:25:41.409672+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.409672+0000","last_undegraded":"2023-08-27T11:25:41.409672+0000","last_fullsized":"2023-08-27T11:25:41.409672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403514+0000","last_change":"2023-08-27T11:25:41.403514+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403514+0000","last_undegraded":"2023-08-27T11:25:41.403514+0000","last_fullsized":"2023-08-27T11:25:41.403514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.402400+0000","last_change":"2023-08-27T11:25:41.402400+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.402400+0000","last_undegraded":"2023-08-27T11:25:41.402400+0000","last_fullsized":"2023-08-27T11:25:41.402400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.419031+0000","last_change":"2023-08-27T11:25:41.414884+0000","last_active":"2023-08-27T11:25:41.419031+0000","last_peered":"2023-08-27T11:25:41.419031+0000","last_clean":"2023-08-27T11:25:41.419031+0000","last_became_active":"2023-08-27T11:25:41.414549+0000","last_became_peered":"2023-08-27T11:25:41.414549+0000","last_unstale":"2023-08-27T11:25:41.419031+0000","last_undegraded":"2023-08-27T11:25:41.419031+0000","last_fullsized":"2023-08-27T11:25:41.419031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:31:13.509254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403856+0000","last_change":"2023-08-27T11:25:41.403856+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403856+0000","last_undegraded":"2023-08-27T11:25:41.403856+0000","last_fullsized":"2023-08-27T11:25:41.403856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.416537+0000","last_change":"2023-08-27T11:25:41.414847+0000","last_active":"2023-08-27T11:25:41.416537+0000","last_peered":"2023-08-27T11:25:41.416537+0000","last_clean":"2023-08-27T11:25:41.416537+0000","last_became_active":"2023-08-27T11:25:41.414532+0000","last_became_peered":"2023-08-27T11:25:41.414532+0000","last_unstale":"2023-08-27T11:25:41.416537+0000","last_undegraded":"2023-08-27T11:25:41.416537+0000","last_fullsized":"2023-08-27T11:25:41.416537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:52:27.302892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399003+0000","last_change":"2023-08-27T11:25:41.399003+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399003+0000","last_undegraded":"2023-08-27T11:25:41.399003+0000","last_fullsized":"2023-08-27T11:25:41.399003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.416340+0000","last_change":"2023-08-27T11:25:41.412179+0000","last_active":"2023-08-27T11:25:41.416340+0000","last_peered":"2023-08-27T11:25:41.416340+0000","last_clean":"2023-08-27T11:25:41.416340+0000","last_became_active":"2023-08-27T11:25:41.411940+0000","last_became_peered":"2023-08-27T11:25:41.411940+0000","last_unstale":"2023-08-27T11:25:41.416340+0000","last_undegraded":"2023-08-27T11:25:41.416340+0000","last_fullsized":"2023-08-27T11:25:41.416340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:01:10.875523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403413+0000","last_change":"2023-08-27T11:25:41.403413+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403413+0000","last_undegraded":"2023-08-27T11:25:41.403413+0000","last_fullsized":"2023-08-27T11:25:41.403413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418955+0000","last_change":"2023-08-27T11:25:41.414002+0000","last_active":"2023-08-27T11:25:41.418955+0000","last_peered":"2023-08-27T11:25:41.418955+0000","last_clean":"2023-08-27T11:25:41.418955+0000","last_became_active":"2023-08-27T11:25:41.413740+0000","last_became_peered":"2023-08-27T11:25:41.413740+0000","last_unstale":"2023-08-27T11:25:41.418955+0000","last_undegraded":"2023-08-27T11:25:41.418955+0000","last_fullsized":"2023-08-27T11:25:41.418955+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:26:23.153384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397310+0000","last_change":"2023-08-27T11:25:41.397310+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397310+0000","last_undegraded":"2023-08-27T11:25:41.397310+0000","last_fullsized":"2023-08-27T11:25:41.397310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.406853+0000","last_change":"2023-08-27T11:25:41.405587+0000","last_active":"2023-08-27T11:25:41.406853+0000","last_peered":"2023-08-27T11:25:41.406853+0000","last_clean":"2023-08-27T11:25:41.406853+0000","last_became_active":"2023-08-27T11:25:41.405255+0000","last_became_peered":"2023-08-27T11:25:41.405255+0000","last_unstale":"2023-08-27T11:25:41.406853+0000","last_undegraded":"2023-08-27T11:25:41.406853+0000","last_fullsized":"2023-08-27T11:25:41.406853+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:09:28.315159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.406628+0000","last_change":"2023-08-27T11:25:41.406628+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.406628+0000","last_undegraded":"2023-08-27T11:25:41.406628+0000","last_fullsized":"2023-08-27T11:25:41.406628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.409295+0000","last_change":"2023-08-27T11:25:41.406787+0000","last_active":"2023-08-27T11:25:41.409295+0000","last_peered":"2023-08-27T11:25:41.409295+0000","last_clean":"2023-08-27T11:25:41.409295+0000","last_became_active":"2023-08-27T11:25:41.406253+0000","last_became_peered":"2023-08-27T11:25:41.406253+0000","last_unstale":"2023-08-27T11:25:41.409295+0000","last_undegraded":"2023-08-27T11:25:41.409295+0000","last_fullsized":"2023-08-27T11:25:41.409295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:38.768166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397479+0000","last_change":"2023-08-27T11:25:41.397479+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397479+0000","last_undegraded":"2023-08-27T11:25:41.397479+0000","last_fullsized":"2023-08-27T11:25:41.397479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.408833+0000","last_change":"2023-08-27T11:25:41.404377+0000","last_active":"2023-08-27T11:25:41.408833+0000","last_peered":"2023-08-27T11:25:41.408833+0000","last_clean":"2023-08-27T11:25:41.408833+0000","last_became_active":"2023-08-27T11:25:41.404056+0000","last_became_peered":"2023-08-27T11:25:41.404056+0000","last_unstale":"2023-08-27T11:25:41.408833+0000","last_undegraded":"2023-08-27T11:25:41.408833+0000","last_fullsized":"2023-08-27T11:25:41.408833+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:16:51.600797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396184+0000","last_change":"2023-08-27T11:25:41.396184+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396184+0000","last_undegraded":"2023-08-27T11:25:41.396184+0000","last_fullsized":"2023-08-27T11:25:41.396184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411092+0000","last_change":"2023-08-27T11:25:41.407540+0000","last_active":"2023-08-27T11:25:41.411092+0000","last_peered":"2023-08-27T11:25:41.411092+0000","last_clean":"2023-08-27T11:25:41.411092+0000","last_became_active":"2023-08-27T11:25:41.406794+0000","last_became_peered":"2023-08-27T11:25:41.406794+0000","last_unstale":"2023-08-27T11:25:41.411092+0000","last_undegraded":"2023-08-27T11:25:41.411092+0000","last_fullsized":"2023-08-27T11:25:41.411092+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:10:38.722806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398510+0000","last_change":"2023-08-27T11:25:41.398510+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398510+0000","last_undegraded":"2023-08-27T11:25:41.398510+0000","last_fullsized":"2023-08-27T11:25:41.398510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421891+0000","last_change":"2023-08-27T11:25:41.417604+0000","last_active":"2023-08-27T11:25:41.421891+0000","last_peered":"2023-08-27T11:25:41.421891+0000","last_clean":"2023-08-27T11:25:41.421891+0000","last_became_active":"2023-08-27T11:25:41.416727+0000","last_became_peered":"2023-08-27T11:25:41.416727+0000","last_unstale":"2023-08-27T11:25:41.421891+0000","last_undegraded":"2023-08-27T11:25:41.421891+0000","last_fullsized":"2023-08-27T11:25:41.421891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:30.061949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398968+0000","last_change":"2023-08-27T11:25:41.398968+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398968+0000","last_undegraded":"2023-08-27T11:25:41.398968+0000","last_fullsized":"2023-08-27T11:25:41.398968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.406341+0000","last_change":"2023-08-27T11:25:41.402868+0000","last_active":"2023-08-27T11:25:41.406341+0000","last_peered":"2023-08-27T11:25:41.406341+0000","last_clean":"2023-08-27T11:25:41.406341+0000","last_became_active":"2023-08-27T11:25:41.402585+0000","last_became_peered":"2023-08-27T11:25:41.402585+0000","last_unstale":"2023-08-27T11:25:41.406341+0000","last_undegraded":"2023-08-27T11:25:41.406341+0000","last_fullsized":"2023-08-27T11:25:41.406341+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:24:45.015907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395873+0000","last_change":"2023-08-27T11:25:41.395873+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395873+0000","last_undegraded":"2023-08-27T11:25:41.395873+0000","last_fullsized":"2023-08-27T11:25:41.395873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.413847+0000","last_change":"2023-08-27T11:25:41.408134+0000","last_active":"2023-08-27T11:25:41.413847+0000","last_peered":"2023-08-27T11:25:41.413847+0000","last_clean":"2023-08-27T11:25:41.413847+0000","last_became_active":"2023-08-27T11:25:41.407806+0000","last_became_peered":"2023-08-27T11:25:41.407806+0000","last_unstale":"2023-08-27T11:25:41.413847+0000","last_undegraded":"2023-08-27T11:25:41.413847+0000","last_fullsized":"2023-08-27T11:25:41.413847+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:38:30.962549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396189+0000","last_change":"2023-08-27T11:25:41.396189+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396189+0000","last_undegraded":"2023-08-27T11:25:41.396189+0000","last_fullsized":"2023-08-27T11:25:41.396189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.415329+0000","last_change":"2023-08-27T11:25:41.414641+0000","last_active":"2023-08-27T11:25:41.415329+0000","last_peered":"2023-08-27T11:25:41.415329+0000","last_clean":"2023-08-27T11:25:41.415329+0000","last_became_active":"2023-08-27T11:25:41.414361+0000","last_became_peered":"2023-08-27T11:25:41.414361+0000","last_unstale":"2023-08-27T11:25:41.415329+0000","last_undegraded":"2023-08-27T11:25:41.415329+0000","last_fullsized":"2023-08-27T11:25:41.415329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:14.192009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396572+0000","last_change":"2023-08-27T11:25:41.396572+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396572+0000","last_undegraded":"2023-08-27T11:25:41.396572+0000","last_fullsized":"2023-08-27T11:25:41.396572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.413558+0000","last_change":"2023-08-27T11:25:41.410338+0000","last_active":"2023-08-27T11:25:41.413558+0000","last_peered":"2023-08-27T11:25:41.413558+0000","last_clean":"2023-08-27T11:25:41.413558+0000","last_became_active":"2023-08-27T11:25:41.410005+0000","last_became_peered":"2023-08-27T11:25:41.410005+0000","last_unstale":"2023-08-27T11:25:41.413558+0000","last_undegraded":"2023-08-27T11:25:41.413558+0000","last_fullsized":"2023-08-27T11:25:41.413558+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:22:28.572307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398507+0000","last_change":"2023-08-27T11:25:41.398507+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398507+0000","last_undegraded":"2023-08-27T11:25:41.398507+0000","last_fullsized":"2023-08-27T11:25:41.398507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.412023+0000","last_change":"2023-08-27T11:25:41.408888+0000","last_active":"2023-08-27T11:25:41.412023+0000","last_peered":"2023-08-27T11:25:41.412023+0000","last_clean":"2023-08-27T11:25:41.412023+0000","last_became_active":"2023-08-27T11:25:41.408609+0000","last_became_peered":"2023-08-27T11:25:41.408609+0000","last_unstale":"2023-08-27T11:25:41.412023+0000","last_undegraded":"2023-08-27T11:25:41.412023+0000","last_fullsized":"2023-08-27T11:25:41.412023+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:12:28.171979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395468+0000","last_change":"2023-08-27T11:25:41.395468+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395468+0000","last_undegraded":"2023-08-27T11:25:41.395468+0000","last_fullsized":"2023-08-27T11:25:41.395468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.415697+0000","last_change":"2023-08-27T11:25:41.414552+0000","last_active":"2023-08-27T11:25:41.415697+0000","last_peered":"2023-08-27T11:25:41.415697+0000","last_clean":"2023-08-27T11:25:41.415697+0000","last_became_active":"2023-08-27T11:25:41.414281+0000","last_became_peered":"2023-08-27T11:25:41.414281+0000","last_unstale":"2023-08-27T11:25:41.415697+0000","last_undegraded":"2023-08-27T11:25:41.415697+0000","last_fullsized":"2023-08-27T11:25:41.415697+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:40:04.874008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395170+0000","last_change":"2023-08-27T11:25:41.395170+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395170+0000","last_undegraded":"2023-08-27T11:25:41.395170+0000","last_fullsized":"2023-08-27T11:25:41.395170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411147+0000","last_change":"2023-08-27T11:25:41.407400+0000","last_active":"2023-08-27T11:25:41.411147+0000","last_peered":"2023-08-27T11:25:41.411147+0000","last_clean":"2023-08-27T11:25:41.411147+0000","last_became_active":"2023-08-27T11:25:41.405982+0000","last_became_peered":"2023-08-27T11:25:41.405982+0000","last_unstale":"2023-08-27T11:25:41.411147+0000","last_undegraded":"2023-08-27T11:25:41.411147+0000","last_fullsized":"2023-08-27T11:25:41.411147+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:59.630816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.407857+0000","last_change":"2023-08-27T11:25:41.407857+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.407857+0000","last_undegraded":"2023-08-27T11:25:41.407857+0000","last_fullsized":"2023-08-27T11:25:41.407857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418905+0000","last_change":"2023-08-27T11:25:41.415726+0000","last_active":"2023-08-27T11:25:41.418905+0000","last_peered":"2023-08-27T11:25:41.418905+0000","last_clean":"2023-08-27T11:25:41.418905+0000","last_became_active":"2023-08-27T11:25:41.415364+0000","last_became_peered":"2023-08-27T11:25:41.415364+0000","last_unstale":"2023-08-27T11:25:41.418905+0000","last_undegraded":"2023-08-27T11:25:41.418905+0000","last_fullsized":"2023-08-27T11:25:41.418905+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:18:37.632552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397995+0000","last_change":"2023-08-27T11:25:41.397995+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397995+0000","last_undegraded":"2023-08-27T11:25:41.397995+0000","last_fullsized":"2023-08-27T11:25:41.397995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418831+0000","last_change":"2023-08-27T11:25:41.417094+0000","last_active":"2023-08-27T11:25:41.418831+0000","last_peered":"2023-08-27T11:25:41.418831+0000","last_clean":"2023-08-27T11:25:41.418831+0000","last_became_active":"2023-08-27T11:25:41.416795+0000","last_became_peered":"2023-08-27T11:25:41.416795+0000","last_unstale":"2023-08-27T11:25:41.418831+0000","last_undegraded":"2023-08-27T11:25:41.418831+0000","last_fullsized":"2023-08-27T11:25:41.418831+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:33:47.717016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397502+0000","last_change":"2023-08-27T11:25:41.397502+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397502+0000","last_undegraded":"2023-08-27T11:25:41.397502+0000","last_fullsized":"2023-08-27T11:25:41.397502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.413823+0000","last_change":"2023-08-27T11:25:41.408039+0000","last_active":"2023-08-27T11:25:41.413823+0000","last_peered":"2023-08-27T11:25:41.413823+0000","last_clean":"2023-08-27T11:25:41.413823+0000","last_became_active":"2023-08-27T11:25:41.407709+0000","last_became_peered":"2023-08-27T11:25:41.407709+0000","last_unstale":"2023-08-27T11:25:41.413823+0000","last_undegraded":"2023-08-27T11:25:41.413823+0000","last_fullsized":"2023-08-27T11:25:41.413823+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:34:44.092855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.404340+0000","last_change":"2023-08-27T11:25:41.404340+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.404340+0000","last_undegraded":"2023-08-27T11:25:41.404340+0000","last_fullsized":"2023-08-27T11:25:41.404340+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418975+0000","last_change":"2023-08-27T11:25:41.414018+0000","last_active":"2023-08-27T11:25:41.418975+0000","last_peered":"2023-08-27T11:25:41.418975+0000","last_clean":"2023-08-27T11:25:41.418975+0000","last_became_active":"2023-08-27T11:25:41.413719+0000","last_became_peered":"2023-08-27T11:25:41.413719+0000","last_unstale":"2023-08-27T11:25:41.418975+0000","last_undegraded":"2023-08-27T11:25:41.418975+0000","last_fullsized":"2023-08-27T11:25:41.418975+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:51:04.452644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398022+0000","last_change":"2023-08-27T11:25:41.398022+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398022+0000","last_undegraded":"2023-08-27T11:25:41.398022+0000","last_fullsized":"2023-08-27T11:25:41.398022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411981+0000","last_change":"2023-08-27T11:25:41.408665+0000","last_active":"2023-08-27T11:25:41.411981+0000","last_peered":"2023-08-27T11:25:41.411981+0000","last_clean":"2023-08-27T11:25:41.411981+0000","last_became_active":"2023-08-27T11:25:41.408402+0000","last_became_peered":"2023-08-27T11:25:41.408402+0000","last_unstale":"2023-08-27T11:25:41.411981+0000","last_undegraded":"2023-08-27T11:25:41.411981+0000","last_fullsized":"2023-08-27T11:25:41.411981+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:25:55.452027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395170+0000","last_change":"2023-08-27T11:25:41.395170+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395170+0000","last_undegraded":"2023-08-27T11:25:41.395170+0000","last_fullsized":"2023-08-27T11:25:41.395170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421291+0000","last_change":"2023-08-27T11:25:41.411569+0000","last_active":"2023-08-27T11:25:41.421291+0000","last_peered":"2023-08-27T11:25:41.421291+0000","last_clean":"2023-08-27T11:25:41.421291+0000","last_became_active":"2023-08-27T11:25:41.411296+0000","last_became_peered":"2023-08-27T11:25:41.411296+0000","last_unstale":"2023-08-27T11:25:41.421291+0000","last_undegraded":"2023-08-27T11:25:41.421291+0000","last_fullsized":"2023-08-27T11:25:41.421291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:19:40.101939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.406076+0000","last_change":"2023-08-27T11:25:41.406076+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.406076+0000","last_undegraded":"2023-08-27T11:25:41.406076+0000","last_fullsized":"2023-08-27T11:25:41.406076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.416239+0000","last_change":"2023-08-27T11:25:41.412882+0000","last_active":"2023-08-27T11:25:41.416239+0000","last_peered":"2023-08-27T11:25:41.416239+0000","last_clean":"2023-08-27T11:25:41.416239+0000","last_became_active":"2023-08-27T11:25:41.412635+0000","last_became_peered":"2023-08-27T11:25:41.412635+0000","last_unstale":"2023-08-27T11:25:41.416239+0000","last_undegraded":"2023-08-27T11:25:41.416239+0000","last_fullsized":"2023-08-27T11:25:41.416239+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:10.546956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396619+0000","last_change":"2023-08-27T11:25:41.396619+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396619+0000","last_undegraded":"2023-08-27T11:25:41.396619+0000","last_fullsized":"2023-08-27T11:25:41.396619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.406484+0000","last_change":"2023-08-27T11:25:41.405653+0000","last_active":"2023-08-27T11:25:41.406484+0000","last_peered":"2023-08-27T11:25:41.406484+0000","last_clean":"2023-08-27T11:25:41.406484+0000","last_became_active":"2023-08-27T11:25:41.405385+0000","last_became_peered":"2023-08-27T11:25:41.405385+0000","last_unstale":"2023-08-27T11:25:41.406484+0000","last_undegraded":"2023-08-27T11:25:41.406484+0000","last_fullsized":"2023-08-27T11:25:41.406484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:55.963991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395911+0000","last_change":"2023-08-27T11:25:41.395911+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395911+0000","last_undegraded":"2023-08-27T11:25:41.395911+0000","last_fullsized":"2023-08-27T11:25:41.395911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421836+0000","last_change":"2023-08-27T11:25:41.417399+0000","last_active":"2023-08-27T11:25:41.421836+0000","last_peered":"2023-08-27T11:25:41.421836+0000","last_clean":"2023-08-27T11:25:41.421836+0000","last_became_active":"2023-08-27T11:25:41.417076+0000","last_became_peered":"2023-08-27T11:25:41.417076+0000","last_unstale":"2023-08-27T11:25:41.421836+0000","last_undegraded":"2023-08-27T11:25:41.421836+0000","last_fullsized":"2023-08-27T11:25:41.421836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:55:39.763853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":133,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.409138+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:25:41.409138+0000","last_peered":"2023-08-27T11:25:41.409138+0000","last_clean":"2023-08-27T11:25:41.409138+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:25:41.409138+0000","last_undegraded":"2023-08-27T11:25:41.409138+0000","last_fullsized":"2023-08-27T11:25:41.409138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:02:20.142155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.404145+0000","last_change":"2023-08-27T11:25:41.404145+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.404145+0000","last_undegraded":"2023-08-27T11:25:41.404145+0000","last_fullsized":"2023-08-27T11:25:41.404145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411964+0000","last_change":"2023-08-27T11:25:41.405709+0000","last_active":"2023-08-27T11:25:41.411964+0000","last_peered":"2023-08-27T11:25:41.411964+0000","last_clean":"2023-08-27T11:25:41.411964+0000","last_became_active":"2023-08-27T11:25:41.405421+0000","last_became_peered":"2023-08-27T11:25:41.405421+0000","last_unstale":"2023-08-27T11:25:41.411964+0000","last_undegraded":"2023-08-27T11:25:41.411964+0000","last_fullsized":"2023-08-27T11:25:41.411964+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:50:55.448426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":133,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.408951+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:25:41.408951+0000","last_peered":"2023-08-27T11:25:41.408951+0000","last_clean":"2023-08-27T11:25:41.408951+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:25:41.408951+0000","last_undegraded":"2023-08-27T11:25:41.408951+0000","last_fullsized":"2023-08-27T11:25:41.408951+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:50:30.904830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400360+0000","last_change":"2023-08-27T11:25:41.400360+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400360+0000","last_undegraded":"2023-08-27T11:25:41.400360+0000","last_fullsized":"2023-08-27T11:25:41.400360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.410077+0000","last_change":"2023-08-27T11:25:41.407112+0000","last_active":"2023-08-27T11:25:41.410077+0000","last_peered":"2023-08-27T11:25:41.410077+0000","last_clean":"2023-08-27T11:25:41.410077+0000","last_became_active":"2023-08-27T11:25:41.406794+0000","last_became_peered":"2023-08-27T11:25:41.406794+0000","last_unstale":"2023-08-27T11:25:41.410077+0000","last_undegraded":"2023-08-27T11:25:41.410077+0000","last_fullsized":"2023-08-27T11:25:41.410077+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:01:40.046194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411219+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:25:41.411219+0000","last_peered":"2023-08-27T11:25:41.411219+0000","last_clean":"2023-08-27T11:25:41.411219+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:25:41.411219+0000","last_undegraded":"2023-08-27T11:25:41.411219+0000","last_fullsized":"2023-08-27T11:25:41.411219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:26:47.289516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.401250+0000","last_change":"2023-08-27T11:25:41.401250+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.401250+0000","last_undegraded":"2023-08-27T11:25:41.401250+0000","last_fullsized":"2023-08-27T11:25:41.401250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418369+0000","last_change":"2023-08-27T11:25:41.417819+0000","last_active":"2023-08-27T11:25:41.418369+0000","last_peered":"2023-08-27T11:25:41.418369+0000","last_clean":"2023-08-27T11:25:41.418369+0000","last_became_active":"2023-08-27T11:25:41.417532+0000","last_became_peered":"2023-08-27T11:25:41.417532+0000","last_unstale":"2023-08-27T11:25:41.418369+0000","last_undegraded":"2023-08-27T11:25:41.418369+0000","last_fullsized":"2023-08-27T11:25:41.418369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:34.714411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.399323+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:25:41.399323+0000","last_peered":"2023-08-27T11:25:41.399323+0000","last_clean":"2023-08-27T11:25:41.399323+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:25:41.399323+0000","last_undegraded":"2023-08-27T11:25:41.399323+0000","last_fullsized":"2023-08-27T11:25:41.399323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:14:18.441956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.405779+0000","last_change":"2023-08-27T11:25:41.405779+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.405779+0000","last_undegraded":"2023-08-27T11:25:41.405779+0000","last_fullsized":"2023-08-27T11:25:41.405779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421309+0000","last_change":"2023-08-27T11:25:41.417161+0000","last_active":"2023-08-27T11:25:41.421309+0000","last_peered":"2023-08-27T11:25:41.421309+0000","last_clean":"2023-08-27T11:25:41.421309+0000","last_became_active":"2023-08-27T11:25:41.416886+0000","last_became_peered":"2023-08-27T11:25:41.416886+0000","last_unstale":"2023-08-27T11:25:41.421309+0000","last_undegraded":"2023-08-27T11:25:41.421309+0000","last_fullsized":"2023-08-27T11:25:41.421309+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:28:04.377394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.400152+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:25:41.400152+0000","last_peered":"2023-08-27T11:25:41.400152+0000","last_clean":"2023-08-27T11:25:41.400152+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:25:41.400152+0000","last_undegraded":"2023-08-27T11:25:41.400152+0000","last_fullsized":"2023-08-27T11:25:41.400152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:28.303971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399721+0000","last_change":"2023-08-27T11:25:41.399721+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399721+0000","last_undegraded":"2023-08-27T11:25:41.399721+0000","last_fullsized":"2023-08-27T11:25:41.399721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.419014+0000","last_change":"2023-08-27T11:25:41.414866+0000","last_active":"2023-08-27T11:25:41.419014+0000","last_peered":"2023-08-27T11:25:41.419014+0000","last_clean":"2023-08-27T11:25:41.419014+0000","last_became_active":"2023-08-27T11:25:41.414525+0000","last_became_peered":"2023-08-27T11:25:41.414525+0000","last_unstale":"2023-08-27T11:25:41.419014+0000","last_undegraded":"2023-08-27T11:25:41.419014+0000","last_fullsized":"2023-08-27T11:25:41.419014+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:04.857971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":263,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.399176+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:25:41.399176+0000","last_peered":"2023-08-27T11:25:41.399176+0000","last_clean":"2023-08-27T11:25:41.399176+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:25:41.399176+0000","last_undegraded":"2023-08-27T11:25:41.399176+0000","last_fullsized":"2023-08-27T11:25:41.399176+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:00.081196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.399025+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:25:41.399025+0000","last_peered":"2023-08-27T11:25:41.399025+0000","last_clean":"2023-08-27T11:25:41.399025+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:25:41.399025+0000","last_undegraded":"2023-08-27T11:25:41.399025+0000","last_fullsized":"2023-08-27T11:25:41.399025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:39.476760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396941+0000","last_change":"2023-08-27T11:25:41.396941+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396941+0000","last_undegraded":"2023-08-27T11:25:41.396941+0000","last_fullsized":"2023-08-27T11:25:41.396941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.417585+0000","last_change":"2023-08-27T11:25:41.416796+0000","last_active":"2023-08-27T11:25:41.417585+0000","last_peered":"2023-08-27T11:25:41.417585+0000","last_clean":"2023-08-27T11:25:41.417585+0000","last_became_active":"2023-08-27T11:25:41.416559+0000","last_became_peered":"2023-08-27T11:25:41.416559+0000","last_unstale":"2023-08-27T11:25:41.417585+0000","last_undegraded":"2023-08-27T11:25:41.417585+0000","last_fullsized":"2023-08-27T11:25:41.417585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:06:23.811399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.393189+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:25:41.393189+0000","last_peered":"2023-08-27T11:25:41.393189+0000","last_clean":"2023-08-27T11:25:41.393189+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:25:41.393189+0000","last_undegraded":"2023-08-27T11:25:41.393189+0000","last_fullsized":"2023-08-27T11:25:41.393189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:49:20.590638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.404181+0000","last_change":"2023-08-27T11:25:41.404181+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.404181+0000","last_undegraded":"2023-08-27T11:25:41.404181+0000","last_fullsized":"2023-08-27T11:25:41.404181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421868+0000","last_change":"2023-08-27T11:25:41.417495+0000","last_active":"2023-08-27T11:25:41.421868+0000","last_peered":"2023-08-27T11:25:41.421868+0000","last_clean":"2023-08-27T11:25:41.421868+0000","last_became_active":"2023-08-27T11:25:41.416993+0000","last_became_peered":"2023-08-27T11:25:41.416993+0000","last_unstale":"2023-08-27T11:25:41.421868+0000","last_undegraded":"2023-08-27T11:25:41.421868+0000","last_fullsized":"2023-08-27T11:25:41.421868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:30.109945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.401782+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:25:41.401782+0000","last_peered":"2023-08-27T11:25:41.401782+0000","last_clean":"2023-08-27T11:25:41.401782+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:25:41.401782+0000","last_undegraded":"2023-08-27T11:25:41.401782+0000","last_fullsized":"2023-08-27T11:25:41.401782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:38:57.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":1008750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51200000000000001,"15min":0.48899999999999999},"min":{"1min":0.34000000000000002,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.66200000000000003,"5min":0.79600000000000004,"15min":0.93500000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.49399999999999999},"min":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.252},"max":{"1min":0.73499999999999999,"5min":0.74099999999999999,"15min":0.81499999999999995},"last":0.42799999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.34200000000000003,"15min":0.26600000000000001},"max":{"1min":0.72299999999999998,"5min":0.77500000000000002,"15min":0.872},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54200000000000004,"15min":0.51100000000000001},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.253},"max":{"1min":0.67800000000000005,"5min":0.71999999999999997,"15min":0.84999999999999998},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.48199999999999998},"min":{"1min":0.32400000000000001,"5min":0.29099999999999998,"15min":0.28299999999999997},"max":{"1min":0.63,"5min":0.71699999999999997,"15min":0.79500000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56499999999999995,"15min":0.54400000000000004},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.33800000000000002},"max":{"1min":0.73999999999999999,"5min":0.78700000000000003,"15min":0.90200000000000002},"last":0.44700000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.54800000000000004},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.748,"15min":0.84299999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.48199999999999998,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.70399999999999996,"5min":0.83499999999999996,"15min":0.91700000000000004},"last":0.50600000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49399999999999999,"15min":0.50800000000000001},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.29899999999999999},"max":{"1min":0.72499999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.312},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52900000000000003,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72099999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38200000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.31900000000000001},"max":{"1min":0.749,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.43099999999999999,"5min":0.377,"15min":0.36799999999999999},"max":{"1min":0.77300000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.35799999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80800000000000005,"5min":0.82399999999999995,"15min":0.86099999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.755,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73099999999999998,"5min":0.86199999999999999,"15min":0.89800000000000002},"last":0.54600000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.316,"5min":0.312,"15min":0.312},"max":{"1min":0.75900000000000001,"5min":0.80600000000000005,"15min":0.873},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75600000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58799999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64700000000000002,"5min":0.74199999999999999,"15min":0.85899999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55800000000000005,"15min":0.58799999999999997},"min":{"1min":0.41699999999999998,"5min":0.39400000000000002,"15min":0.377},"max":{"1min":0.68600000000000005,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.60099999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.78900000000000003,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.70199999999999996,"5min":0.84699999999999998,"15min":0.93799999999999994},"last":0.56999999999999995}]},{"osd":4,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.32400000000000001,"5min":0.307,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.437,"5min":0.375,"15min":0.374},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.61799999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33400000000000002},"max":{"1min":0.63400000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.60099999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.875,"15min":0.875},"last":0.70599999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.47599999999999998,"5min":0.47099999999999997,"15min":0.34899999999999998},"max":{"1min":0.68200000000000005,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":1008750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.31,"15min":0.63400000000000001},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":0.502,"5min":0.502,"15min":11.971},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.96899999999999997},"min":{"1min":0.22500000000000001,"5min":0.221,"15min":0.20799999999999999},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.23300000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33700000000000002,"15min":0.66400000000000003},"min":{"1min":0.247,"5min":0.221,"15min":0.221},"max":{"1min":0.46600000000000003,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.33800000000000002,"15min":0.34399999999999997},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":0.48599999999999999,"5min":0.53700000000000003,"15min":0.58599999999999997},"last":0.32300000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.38,"15min":0.67600000000000005},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":10.99},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36699999999999999,"15min":1.002},"min":{"1min":0.29399999999999998,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.505,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.35499999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57199999999999995,"15min":0.60299999999999998},"min":{"1min":0.44500000000000001,"5min":0.36099999999999999,"15min":0.33400000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":1.26},"last":0.623},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55800000000000005,"15min":0.84199999999999997},"min":{"1min":0.38900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.77600000000000002,"5min":0.86299999999999999,"15min":10.957000000000001},"last":0.54500000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57299999999999995,"15min":1.2},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.36199999999999999},"max":{"1min":0.79300000000000004,"5min":0.85299999999999998,"15min":23.016999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.60299999999999998},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.296},"last":0.503}]},{"osd":6,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.57999999999999996,"15min":0.88300000000000001},"min":{"1min":0.434,"5min":0.371,"15min":0.371},"max":{"1min":0.84899999999999998,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":1.2170000000000001},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.52300000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":1.212},"min":{"1min":0.45400000000000001,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.76500000000000001,"5min":0.89600000000000002,"15min":22.988},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.63900000000000001},"min":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86199999999999999,"5min":0.91600000000000004,"15min":1.327},"last":0.57399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":417021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.29299999999999998,"15min":0.31},"min":{"1min":0.214,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.45300000000000001,"5min":0.50900000000000001,"15min":0.52300000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.35099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.62},"last":0.27700000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.33900000000000002},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.54400000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.33700000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.223},"max":{"1min":0.49299999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.25}]},{"osd":3,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.34499999999999997},"min":{"1min":0.249,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.50800000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.36199999999999999,"15min":0.372},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.53800000000000003,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.309}]},{"osd":4,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.56000000000000005},"min":{"1min":0.35399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85399999999999998,"5min":0.92500000000000004,"15min":1.214},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.35799999999999998,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.81000000000000005,"15min":1.2569999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.371,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81000000000000005,"5min":0.82599999999999996,"15min":0.91400000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83999999999999997,"5min":0.91100000000000003,"15min":1.603},"last":0.52900000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79900000000000004,"5min":0.86099999999999999,"15min":0.97599999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73999999999999999,"5min":0.871,"15min":0.871},"last":0.48199999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75800000000000001,"5min":0.94199999999999995,"15min":1.383},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86199999999999999,"5min":0.88900000000000001,"15min":0.92000000000000004},"last":0.56899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.30099999999999999,"15min":0.30199999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20200000000000001},"max":{"1min":0.48899999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.317},"min":{"1min":0.23100000000000001,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.46700000000000003,"5min":0.52500000000000002,"15min":0.629},"last":0.30199999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34200000000000003,"15min":0.34799999999999998},"min":{"1min":0.23000000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52200000000000002,"5min":0.749,"15min":0.749},"last":0.28899999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36299999999999999,"15min":0.38400000000000001},"min":{"1min":0.26600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57099999999999995,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.27500000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.69499999999999995},"last":0.34100000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.42699999999999999,"5min":0.39600000000000002,"15min":0.35599999999999998},"max":{"1min":0.82199999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.42599999999999999,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.75,"5min":0.879,"15min":0.879},"last":0.433}]},{"osd":5,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51300000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.73499999999999999,"5min":0.85199999999999998,"15min":0.85299999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55600000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59699999999999998,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.79500000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574958,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.27700000000000002,"5min":0.30099999999999999,"15min":0.29999999999999999},"min":{"1min":0.21199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.34100000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.374},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32600000000000001,"15min":0.312},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46200000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.246}]},{"osd":1,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.32700000000000001,"15min":0.312},"min":{"1min":0.22700000000000001,"5min":0.21299999999999999,"15min":0.19500000000000001},"max":{"1min":0.48599999999999999,"5min":0.50600000000000001,"15min":0.60399999999999998},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.308,"5min":0.33400000000000002,"15min":0.34000000000000002},"min":{"1min":0.26500000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.38200000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"last":0.30199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44400000000000001,"5min":0.57199999999999995,"15min":0.57599999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.42999999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.26700000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.436,"5min":0.316,"15min":0.28599999999999998},"max":{"1min":0.68200000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68700000000000006,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.78400000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.36799999999999999,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.57299999999999995},"min":{"1min":0.442,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.67000000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.67400000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.373},"max":{"1min":0.73999999999999999,"5min":1.091,"15min":1.091},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60099999999999998,"15min":0.61499999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75600000000000001,"5min":1.111,"15min":1.111},"last":0.60299999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.61799999999999999},"min":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.36699999999999999},"max":{"1min":0.70199999999999996,"5min":1.032,"15min":1.236},"last":0.623},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.58499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68200000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.80700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.32700000000000001,"15min":0.30399999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.44},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.379,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.65100000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54800000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76300000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.433,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.70699999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.44,"5min":0.41799999999999998,"15min":0.41399999999999998},"max":{"1min":0.751,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66100000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58599999999999997,"15min":0.59499999999999997},"min":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.379},"max":{"1min":0.85999999999999999,"5min":0.879,"15min":0.879},"last":0.625}]},{"osd":6,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.55000000000000004},"min":{"1min":0.39600000000000002,"5min":0.37,"15min":0.31},"max":{"1min":0.78900000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.62,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.37,"15min":0.35299999999999998},"max":{"1min":0.76700000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.39300000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.82499999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.34200000000000003},"max":{"1min":0.79200000000000004,"5min":0.83799999999999997,"15min":0.89300000000000002},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574957,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.63300000000000001,"5min":0.78600000000000003,"15min":0.96899999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.29199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.60099999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70499999999999996,"5min":0.747,"15min":0.92200000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52400000000000002,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.63200000000000001,"5min":0.72899999999999998,"15min":1.075},"last":0.53500000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.78000000000000003,"15min":0.95699999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54600000000000004,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.65300000000000002,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.56299999999999994}]},{"osd":3,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72299999999999998,"5min":0.81000000000000005,"15min":1.03},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58199999999999996,"15min":0.59499999999999997},"min":{"1min":0.48499999999999999,"5min":0.36699999999999999,"15min":0.35199999999999998},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":1.026},"last":0.58699999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72999999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.441},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.315,"15min":0.26900000000000002},"max":{"1min":0.78900000000000003,"5min":0.86399999999999999,"15min":0.91400000000000003},"last":0.42099999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.76500000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.48399999999999999,"5min":0.36599999999999999,"15min":0.28799999999999998},"max":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.94199999999999995},"last":0.5}]},{"osd":7,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.51300000000000001,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.95899999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.61199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.83699999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:25:47.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3f is not active+clean 2023-08-27T11:25:47.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.396787+0000', 'last_change': '2023-08-27T11:25:41.396787+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.396787+0000', 'last_undegraded': '2023-08-27T11:25:41.396787+0000', 'last_fullsized': '2023-08-27T11:25:41.396787+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:25:47.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3e is not active+clean 2023-08-27T11:25:47.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.408023+0000', 'last_change': '2023-08-27T11:25:41.408023+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.408023+0000', 'last_undegraded': '2023-08-27T11:25:41.408023+0000', 'last_fullsized': '2023-08-27T11:25:41.408023+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:25:47.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3d is not active+clean 2023-08-27T11:25:47.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.396335+0000', 'last_change': '2023-08-27T11:25:41.396335+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.396335+0000', 'last_undegraded': '2023-08-27T11:25:41.396335+0000', 'last_fullsized': '2023-08-27T11:25:41.396335+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:25:47.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3c is not active+clean 2023-08-27T11:25:47.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.397019+0000', 'last_change': '2023-08-27T11:25:41.397019+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.397019+0000', 'last_undegraded': '2023-08-27T11:25:41.397019+0000', 'last_fullsized': '2023-08-27T11:25:41.397019+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:47.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3b is not active+clean 2023-08-27T11:25:47.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.399662+0000', 'last_change': '2023-08-27T11:25:41.399662+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.399662+0000', 'last_undegraded': '2023-08-27T11:25:41.399662+0000', 'last_fullsized': '2023-08-27T11:25:41.399662+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:47.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3a is not active+clean 2023-08-27T11:25:47.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.406043+0000', 'last_change': '2023-08-27T11:25:41.406043+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.406043+0000', 'last_undegraded': '2023-08-27T11:25:41.406043+0000', 'last_fullsized': '2023-08-27T11:25:41.406043+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.39 is not active+clean 2023-08-27T11:25:47.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.400258+0000', 'last_change': '2023-08-27T11:25:41.400258+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.400258+0000', 'last_undegraded': '2023-08-27T11:25:41.400258+0000', 'last_fullsized': '2023-08-27T11:25:41.400258+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:25:47.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.38 is not active+clean 2023-08-27T11:25:47.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.398869+0000', 'last_change': '2023-08-27T11:25:41.398869+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.398869+0000', 'last_undegraded': '2023-08-27T11:25:41.398869+0000', 'last_fullsized': '2023-08-27T11:25:41.398869+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:47.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.37 is not active+clean 2023-08-27T11:25:47.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.398915+0000', 'last_change': '2023-08-27T11:25:41.398915+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.398915+0000', 'last_undegraded': '2023-08-27T11:25:41.398915+0000', 'last_fullsized': '2023-08-27T11:25:41.398915+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:47.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.36 is not active+clean 2023-08-27T11:25:47.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.398563+0000', 'last_change': '2023-08-27T11:25:41.398563+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.398563+0000', 'last_undegraded': '2023-08-27T11:25:41.398563+0000', 'last_fullsized': '2023-08-27T11:25:41.398563+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:25:47.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.35 is not active+clean 2023-08-27T11:25:47.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.401519+0000', 'last_change': '2023-08-27T11:25:41.401519+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.401519+0000', 'last_undegraded': '2023-08-27T11:25:41.401519+0000', 'last_fullsized': '2023-08-27T11:25:41.401519+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:47.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.34 is not active+clean 2023-08-27T11:25:47.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.396135+0000', 'last_change': '2023-08-27T11:25:41.396135+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.396135+0000', 'last_undegraded': '2023-08-27T11:25:41.396135+0000', 'last_fullsized': '2023-08-27T11:25:41.396135+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:25:47.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.33 is not active+clean 2023-08-27T11:25:47.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.398767+0000', 'last_change': '2023-08-27T11:25:41.398767+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.398767+0000', 'last_undegraded': '2023-08-27T11:25:41.398767+0000', 'last_fullsized': '2023-08-27T11:25:41.398767+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:47.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.32 is not active+clean 2023-08-27T11:25:47.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.402238+0000', 'last_change': '2023-08-27T11:25:41.402238+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.402238+0000', 'last_undegraded': '2023-08-27T11:25:41.402238+0000', 'last_fullsized': '2023-08-27T11:25:41.402238+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:47.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.31 is not active+clean 2023-08-27T11:25:47.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.400006+0000', 'last_change': '2023-08-27T11:25:41.400006+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.400006+0000', 'last_undegraded': '2023-08-27T11:25:41.400006+0000', 'last_fullsized': '2023-08-27T11:25:41.400006+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:47.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.30 is not active+clean 2023-08-27T11:25:47.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.403883+0000', 'last_change': '2023-08-27T11:25:41.403883+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.403883+0000', 'last_undegraded': '2023-08-27T11:25:41.403883+0000', 'last_fullsized': '2023-08-27T11:25:41.403883+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2f is not active+clean 2023-08-27T11:25:47.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.403836+0000', 'last_change': '2023-08-27T11:25:41.403836+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.403836+0000', 'last_undegraded': '2023-08-27T11:25:41.403836+0000', 'last_fullsized': '2023-08-27T11:25:41.403836+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2e is not active+clean 2023-08-27T11:25:47.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.396268+0000', 'last_change': '2023-08-27T11:25:41.396268+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.396268+0000', 'last_undegraded': '2023-08-27T11:25:41.396268+0000', 'last_fullsized': '2023-08-27T11:25:41.396268+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:25:47.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2d is not active+clean 2023-08-27T11:25:47.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.400999+0000', 'last_change': '2023-08-27T11:25:41.400999+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.400999+0000', 'last_undegraded': '2023-08-27T11:25:41.400999+0000', 'last_fullsized': '2023-08-27T11:25:41.400999+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:47.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2c is not active+clean 2023-08-27T11:25:47.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.399588+0000', 'last_change': '2023-08-27T11:25:41.399588+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.399588+0000', 'last_undegraded': '2023-08-27T11:25:41.399588+0000', 'last_fullsized': '2023-08-27T11:25:41.399588+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:47.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2b is not active+clean 2023-08-27T11:25:47.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.399193+0000', 'last_change': '2023-08-27T11:25:41.399193+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.399193+0000', 'last_undegraded': '2023-08-27T11:25:41.399193+0000', 'last_fullsized': '2023-08-27T11:25:41.399193+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:47.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2a is not active+clean 2023-08-27T11:25:47.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.400790+0000', 'last_change': '2023-08-27T11:25:41.400790+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.400790+0000', 'last_undegraded': '2023-08-27T11:25:41.400790+0000', 'last_fullsized': '2023-08-27T11:25:41.400790+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:47.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2023-08-27T11:25:47.304 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.407276+0000', 'last_change': '2023-08-27T11:25:41.407276+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.407276+0000', 'last_undegraded': '2023-08-27T11:25:41.407276+0000', 'last_fullsized': '2023-08-27T11:25:41.407276+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:25:47.304 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.28 is not active+clean 2023-08-27T11:25:47.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.395723+0000', 'last_change': '2023-08-27T11:25:41.395723+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.395723+0000', 'last_undegraded': '2023-08-27T11:25:41.395723+0000', 'last_fullsized': '2023-08-27T11:25:41.395723+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:25:47.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.27 is not active+clean 2023-08-27T11:25:47.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.408337+0000', 'last_change': '2023-08-27T11:25:41.408337+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.408337+0000', 'last_undegraded': '2023-08-27T11:25:41.408337+0000', 'last_fullsized': '2023-08-27T11:25:41.408337+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:25:47.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.26 is not active+clean 2023-08-27T11:25:47.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.401694+0000', 'last_change': '2023-08-27T11:25:41.401694+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.401694+0000', 'last_undegraded': '2023-08-27T11:25:41.401694+0000', 'last_fullsized': '2023-08-27T11:25:41.401694+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:47.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.25 is not active+clean 2023-08-27T11:25:47.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.395708+0000', 'last_change': '2023-08-27T11:25:41.395708+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.395708+0000', 'last_undegraded': '2023-08-27T11:25:41.395708+0000', 'last_fullsized': '2023-08-27T11:25:41.395708+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:25:47.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.24 is not active+clean 2023-08-27T11:25:47.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.399209+0000', 'last_change': '2023-08-27T11:25:41.399209+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.399209+0000', 'last_undegraded': '2023-08-27T11:25:41.399209+0000', 'last_fullsized': '2023-08-27T11:25:41.399209+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:47.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.23 is not active+clean 2023-08-27T11:25:47.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.399718+0000', 'last_change': '2023-08-27T11:25:41.399718+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.399718+0000', 'last_undegraded': '2023-08-27T11:25:41.399718+0000', 'last_fullsized': '2023-08-27T11:25:41.399718+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:25:47.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.22 is not active+clean 2023-08-27T11:25:47.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.400210+0000', 'last_change': '2023-08-27T11:25:41.400210+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.400210+0000', 'last_undegraded': '2023-08-27T11:25:41.400210+0000', 'last_fullsized': '2023-08-27T11:25:41.400210+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:47.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.21 is not active+clean 2023-08-27T11:25:47.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.409672+0000', 'last_change': '2023-08-27T11:25:41.409672+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.409672+0000', 'last_undegraded': '2023-08-27T11:25:41.409672+0000', 'last_fullsized': '2023-08-27T11:25:41.409672+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:25:47.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.20 is not active+clean 2023-08-27T11:25:47.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.403514+0000', 'last_change': '2023-08-27T11:25:41.403514+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.403514+0000', 'last_undegraded': '2023-08-27T11:25:41.403514+0000', 'last_fullsized': '2023-08-27T11:25:41.403514+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2023-08-27T11:25:47.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.402400+0000', 'last_change': '2023-08-27T11:25:41.402400+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.402400+0000', 'last_undegraded': '2023-08-27T11:25:41.402400+0000', 'last_fullsized': '2023-08-27T11:25:41.402400+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2023-08-27T11:25:47.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.403856+0000', 'last_change': '2023-08-27T11:25:41.403856+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.403856+0000', 'last_undegraded': '2023-08-27T11:25:41.403856+0000', 'last_fullsized': '2023-08-27T11:25:41.403856+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2023-08-27T11:25:47.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.399003+0000', 'last_change': '2023-08-27T11:25:41.399003+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.399003+0000', 'last_undegraded': '2023-08-27T11:25:41.399003+0000', 'last_fullsized': '2023-08-27T11:25:41.399003+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:25:47.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2023-08-27T11:25:47.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.403413+0000', 'last_change': '2023-08-27T11:25:41.403413+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.403413+0000', 'last_undegraded': '2023-08-27T11:25:41.403413+0000', 'last_fullsized': '2023-08-27T11:25:41.403413+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2023-08-27T11:25:47.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.397310+0000', 'last_change': '2023-08-27T11:25:41.397310+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.397310+0000', 'last_undegraded': '2023-08-27T11:25:41.397310+0000', 'last_fullsized': '2023-08-27T11:25:41.397310+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:47.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2023-08-27T11:25:47.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.406628+0000', 'last_change': '2023-08-27T11:25:41.406628+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.406628+0000', 'last_undegraded': '2023-08-27T11:25:41.406628+0000', 'last_fullsized': '2023-08-27T11:25:41.406628+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:25:47.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2023-08-27T11:25:47.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.397479+0000', 'last_change': '2023-08-27T11:25:41.397479+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.397479+0000', 'last_undegraded': '2023-08-27T11:25:41.397479+0000', 'last_fullsized': '2023-08-27T11:25:41.397479+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2023-08-27T11:25:47.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.396184+0000', 'last_change': '2023-08-27T11:25:41.396184+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.396184+0000', 'last_undegraded': '2023-08-27T11:25:41.396184+0000', 'last_fullsized': '2023-08-27T11:25:41.396184+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:47.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2023-08-27T11:25:47.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.398510+0000', 'last_change': '2023-08-27T11:25:41.398510+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.398510+0000', 'last_undegraded': '2023-08-27T11:25:41.398510+0000', 'last_fullsized': '2023-08-27T11:25:41.398510+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:47.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2023-08-27T11:25:47.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.398968+0000', 'last_change': '2023-08-27T11:25:41.398968+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.398968+0000', 'last_undegraded': '2023-08-27T11:25:41.398968+0000', 'last_fullsized': '2023-08-27T11:25:41.398968+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:25:47.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2023-08-27T11:25:47.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.395873+0000', 'last_change': '2023-08-27T11:25:41.395873+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.395873+0000', 'last_undegraded': '2023-08-27T11:25:41.395873+0000', 'last_fullsized': '2023-08-27T11:25:41.395873+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:25:47.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2023-08-27T11:25:47.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.396189+0000', 'last_change': '2023-08-27T11:25:41.396189+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.396189+0000', 'last_undegraded': '2023-08-27T11:25:41.396189+0000', 'last_fullsized': '2023-08-27T11:25:41.396189+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:47.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2023-08-27T11:25:47.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.396572+0000', 'last_change': '2023-08-27T11:25:41.396572+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.396572+0000', 'last_undegraded': '2023-08-27T11:25:41.396572+0000', 'last_fullsized': '2023-08-27T11:25:41.396572+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:47.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2023-08-27T11:25:47.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.398507+0000', 'last_change': '2023-08-27T11:25:41.398507+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.398507+0000', 'last_undegraded': '2023-08-27T11:25:41.398507+0000', 'last_fullsized': '2023-08-27T11:25:41.398507+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:47.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2023-08-27T11:25:47.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.395468+0000', 'last_change': '2023-08-27T11:25:41.395468+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.395468+0000', 'last_undegraded': '2023-08-27T11:25:41.395468+0000', 'last_fullsized': '2023-08-27T11:25:41.395468+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:25:47.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2023-08-27T11:25:47.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.395170+0000', 'last_change': '2023-08-27T11:25:41.395170+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.395170+0000', 'last_undegraded': '2023-08-27T11:25:41.395170+0000', 'last_fullsized': '2023-08-27T11:25:41.395170+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:25:47.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2023-08-27T11:25:47.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.407857+0000', 'last_change': '2023-08-27T11:25:41.407857+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.407857+0000', 'last_undegraded': '2023-08-27T11:25:41.407857+0000', 'last_fullsized': '2023-08-27T11:25:41.407857+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:25:47.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2023-08-27T11:25:47.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.397995+0000', 'last_change': '2023-08-27T11:25:41.397995+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.397995+0000', 'last_undegraded': '2023-08-27T11:25:41.397995+0000', 'last_fullsized': '2023-08-27T11:25:41.397995+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2023-08-27T11:25:47.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.397502+0000', 'last_change': '2023-08-27T11:25:41.397502+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.397502+0000', 'last_undegraded': '2023-08-27T11:25:41.397502+0000', 'last_fullsized': '2023-08-27T11:25:41.397502+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2023-08-27T11:25:47.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.404340+0000', 'last_change': '2023-08-27T11:25:41.404340+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.404340+0000', 'last_undegraded': '2023-08-27T11:25:41.404340+0000', 'last_fullsized': '2023-08-27T11:25:41.404340+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:25:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2023-08-27T11:25:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.398022+0000', 'last_change': '2023-08-27T11:25:41.398022+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.398022+0000', 'last_undegraded': '2023-08-27T11:25:41.398022+0000', 'last_fullsized': '2023-08-27T11:25:41.398022+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:25:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2023-08-27T11:25:47.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.395170+0000', 'last_change': '2023-08-27T11:25:41.395170+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.395170+0000', 'last_undegraded': '2023-08-27T11:25:41.395170+0000', 'last_fullsized': '2023-08-27T11:25:41.395170+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:25:47.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2023-08-27T11:25:47.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.406076+0000', 'last_change': '2023-08-27T11:25:41.406076+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.406076+0000', 'last_undegraded': '2023-08-27T11:25:41.406076+0000', 'last_fullsized': '2023-08-27T11:25:41.406076+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:25:47.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2023-08-27T11:25:47.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.396619+0000', 'last_change': '2023-08-27T11:25:41.396619+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.396619+0000', 'last_undegraded': '2023-08-27T11:25:41.396619+0000', 'last_fullsized': '2023-08-27T11:25:41.396619+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2023-08-27T11:25:47.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.395911+0000', 'last_change': '2023-08-27T11:25:41.395911+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.395911+0000', 'last_undegraded': '2023-08-27T11:25:41.395911+0000', 'last_fullsized': '2023-08-27T11:25:41.395911+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2023-08-27T11:25:47.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.404145+0000', 'last_change': '2023-08-27T11:25:41.404145+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.404145+0000', 'last_undegraded': '2023-08-27T11:25:41.404145+0000', 'last_fullsized': '2023-08-27T11:25:41.404145+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:25:47.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2023-08-27T11:25:47.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.400360+0000', 'last_change': '2023-08-27T11:25:41.400360+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.400360+0000', 'last_undegraded': '2023-08-27T11:25:41.400360+0000', 'last_fullsized': '2023-08-27T11:25:41.400360+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:47.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2023-08-27T11:25:47.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.401250+0000', 'last_change': '2023-08-27T11:25:41.401250+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.401250+0000', 'last_undegraded': '2023-08-27T11:25:41.401250+0000', 'last_fullsized': '2023-08-27T11:25:41.401250+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:25:47.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2023-08-27T11:25:47.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.405779+0000', 'last_change': '2023-08-27T11:25:41.405779+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.405779+0000', 'last_undegraded': '2023-08-27T11:25:41.405779+0000', 'last_fullsized': '2023-08-27T11:25:41.405779+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:25:47.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2023-08-27T11:25:47.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.399721+0000', 'last_change': '2023-08-27T11:25:41.399721+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.399721+0000', 'last_undegraded': '2023-08-27T11:25:41.399721+0000', 'last_fullsized': '2023-08-27T11:25:41.399721+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2023-08-27T11:25:47.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.396941+0000', 'last_change': '2023-08-27T11:25:41.396941+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.396941+0000', 'last_undegraded': '2023-08-27T11:25:41.396941+0000', 'last_fullsized': '2023-08-27T11:25:41.396941+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:47.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2023-08-27T11:25:47.336 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 75, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:41.404181+0000', 'last_change': '2023-08-27T11:25:41.404181+0000', 'last_active': '2023-08-27T11:25:41.389437+0000', 'last_peered': '2023-08-27T11:25:41.389437+0000', 'last_clean': '2023-08-27T11:25:41.389437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:41.404181+0000', 'last_undegraded': '2023-08-27T11:25:41.404181+0000', 'last_fullsized': '2023-08-27T11:25:41.404181+0000', 'mapping_epoch': 75, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:41.389437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:25:47.336 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-27T11:25:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.455+0000 7f9fd0594700 1 Processor -- start 2023-08-27T11:25:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.455+0000 7f9fd0594700 1 -- start start 2023-08-27T11:25:47.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.457+0000 7f9fd0594700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9fc806adb0 0x7f9fc806b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.457+0000 7f9fd0594700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fc80cdbd0 0x7f9fc80ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.457+0000 7f9fd0594700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc815eb60 con 0x7f9fc8151060 2023-08-27T11:25:47.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.457+0000 7f9fd0594700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc806b770 con 0x7f9fc806adb0 2023-08-27T11:25:47.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.457+0000 7f9fd0594700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc80b02f0 con 0x7f9fc80cdbd0 2023-08-27T11:25:47.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.457+0000 7f9fcdb2f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9fc806adb0 0x7f9fc806b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.457+0000 7f9fceb31700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fc80cdbd0 0x7f9fc80ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.457+0000 7f9fce330700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9fc8151060 0x7f9fc815e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:33764/0 (socket says 172.21.15.112:33764) 2023-08-27T11:25:47.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.457+0000 7f9fce330700 1 -- 172.21.15.112:0/1357254269 learned_addr learned my addr 172.21.15.112:0/1357254269 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:47.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.457+0000 7f9fcdb2f700 1 -- 172.21.15.112:0/1357254269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fc80cdbd0 msgr2=0x7f9fc80ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:47.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.457+0000 7f9fcdb2f700 1 --2- 172.21.15.112:0/1357254269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fc80cdbd0 0x7f9fc80ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.457+0000 7f9fcdb2f700 1 -- 172.21.15.112:0/1357254269 >> v1:172.21.15.138:6789/0 conn(0x7f9fc8151060 legacy=0x7f9fc815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.457+0000 7f9fcd32e700 1 -- 172.21.15.112:0/1357254269 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572803824 0 0) 0x7f9fc815eb60 con 0x7f9fc8151060 2023-08-27T11:25:47.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.458+0000 7f9fcdb2f700 1 -- 172.21.15.112:0/1357254269 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc80b05d0 con 0x7f9fc806adb0 2023-08-27T11:25:47.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.458+0000 7f9fceb31700 1 --2- 172.21.15.112:0/1357254269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fc80cdbd0 0x7f9fc80ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:47.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.458+0000 7f9fcdb2f700 1 --2- 172.21.15.112:0/1357254269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc806adb0 0x7f9fc806b230 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009840 tx=0x7f9fc4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53178b1090f90019 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:47.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.458+0000 7f9fcd32e700 1 -- 172.21.15.112:0/1357254269 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4011830 con 0x7f9fc806adb0 2023-08-27T11:25:47.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.458+0000 7f9fcd32e700 1 -- 172.21.15.112:0/1357254269 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc4011990 con 0x7f9fc806adb0 2023-08-27T11:25:47.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.458+0000 7f9fcd32e700 1 -- 172.21.15.112:0/1357254269 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4011d20 con 0x7f9fc806adb0 2023-08-27T11:25:47.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.458+0000 7f9fd0594700 1 -- 172.21.15.112:0/1357254269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc806adb0 msgr2=0x7f9fc806b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.458+0000 7f9fd0594700 1 --2- 172.21.15.112:0/1357254269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc806adb0 0x7f9fc806b230 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009840 tx=0x7f9fc4009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:47.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.458+0000 7f9fd0594700 1 -- 172.21.15.112:0/1357254269 shutdown_connections 2023-08-27T11:25:47.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.459+0000 7f9fd0594700 1 --2- 172.21.15.112:0/1357254269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fc80cdbd0 0x7f9fc80ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.459+0000 7f9fd0594700 1 --2- 172.21.15.112:0/1357254269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc806adb0 0x7f9fc806b230 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.459+0000 7f9fd0594700 1 -- 172.21.15.112:0/1357254269 >> 172.21.15.112:0/1357254269 conn(0x7f9fc80d0000 msgr2=0x7f9fc80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:47.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.459+0000 7f9fd0594700 1 -- 172.21.15.112:0/1357254269 shutdown_connections 2023-08-27T11:25:47.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.459+0000 7f9fd0594700 1 -- 172.21.15.112:0/1357254269 wait complete. 2023-08-27T11:25:47.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.460+0000 7f9fd0594700 1 Processor -- start 2023-08-27T11:25:47.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.460+0000 7f9fd0594700 1 -- start start 2023-08-27T11:25:47.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.461+0000 7f9fd0594700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc806adb0 0x7f9fc814cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.461+0000 7f9fd0594700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fc80cdbd0 0x7f9fc81f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.461+0000 7f9fd0594700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fc8151060 0x7f9fc80b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.461+0000 7f9fd0594700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc814d440 con 0x7f9fc8151060 2023-08-27T11:25:47.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.461+0000 7f9fd0594700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc80c8080 con 0x7f9fc806adb0 2023-08-27T11:25:47.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.461+0000 7f9fd0594700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc81f72d0 con 0x7f9fc80cdbd0 2023-08-27T11:25:47.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.461+0000 7f9fceb31700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fc8151060 0x7f9fc80b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.461+0000 7f9fceb31700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fc8151060 0x7f9fc80b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60100/0 (socket says 172.21.15.112:60100) 2023-08-27T11:25:47.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.461+0000 7f9fceb31700 1 -- 172.21.15.112:0/3543036191 learned_addr learned my addr 172.21.15.112:0/3543036191 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:47.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.461+0000 7f9fce330700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc806adb0 0x7f9fc814cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.461+0000 7f9fcdb2f700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fc80cdbd0 0x7f9fc81f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.461+0000 7f9fceb31700 1 -- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fc80cdbd0 msgr2=0x7f9fc81f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.462+0000 7f9fceb31700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fc80cdbd0 0x7f9fc81f6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.462+0000 7f9fceb31700 1 -- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc806adb0 msgr2=0x7f9fc814cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.462+0000 7f9fceb31700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc806adb0 0x7f9fc814cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.462+0000 7f9fceb31700 1 -- 172.21.15.112:0/3543036191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc000bc80 con 0x7f9fc8151060 2023-08-27T11:25:47.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.462+0000 7f9fce330700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc806adb0 0x7f9fc814cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:47.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.462+0000 7f9fceb31700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fc8151060 0x7f9fc80b1c20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00093c0 tx=0x7f9fc00093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:47.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.462+0000 7f9fbeffd700 1 -- 172.21.15.112:0/3543036191 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0010040 con 0x7f9fc8151060 2023-08-27T11:25:47.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.462+0000 7f9fbeffd700 1 -- 172.21.15.112:0/3543036191 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc000ea90 con 0x7f9fc8151060 2023-08-27T11:25:47.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.462+0000 7f9fd0594700 1 -- 172.21.15.112:0/3543036191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc40094a0 con 0x7f9fc8151060 2023-08-27T11:25:47.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.462+0000 7f9fbeffd700 1 -- 172.21.15.112:0/3543036191 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000ee00 con 0x7f9fc8151060 2023-08-27T11:25:47.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.462+0000 7f9fd0594700 1 -- 172.21.15.112:0/3543036191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc80c7750 con 0x7f9fc8151060 2023-08-27T11:25:47.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.463+0000 7f9fd0594700 1 -- 172.21.15.112:0/3543036191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc80c8210 con 0x7f9fc8151060 2023-08-27T11:25:47.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.463+0000 7f9fbeffd700 1 -- 172.21.15.112:0/3543036191 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f9fc0009540 con 0x7f9fc8151060 2023-08-27T11:25:47.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.464+0000 7f9fbeffd700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9fac06a320 0x7f9fac06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.464+0000 7f9fbeffd700 1 -- 172.21.15.112:0/3543036191 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9fc0088c80 con 0x7f9fc8151060 2023-08-27T11:25:47.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.464+0000 7f9fce330700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9fac06a320 0x7f9fac06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.464+0000 7f9fce330700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9fac06a320 0x7f9fac06c7e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8000f60 tx=0x7f9fb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:47.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.468+0000 7f9fbeffd700 1 -- 172.21.15.112:0/3543036191 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9fc0053f00 con 0x7f9fc8151060 2023-08-27T11:25:47.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.736+0000 7f9fd0594700 1 -- 172.21.15.112:0/3543036191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9fc8067a20 con 0x7f9fc8151060 2023-08-27T11:25:47.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.737+0000 7f9fbeffd700 1 -- 172.21.15.112:0/3543036191 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1034 (secure 0 0 0) 0x7f9fc0057520 con 0x7f9fc8151060 2023-08-27T11:25:47.739 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:47.739 INFO:teuthology.orchestra.run.smithi112.stdout:{"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":541,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":77,"num_osds":8,"num_up_osds":8,"osd_up_since":1693135017,"num_in_osds":8,"osd_in_since":1693135009,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":264015872,"bytes_avail":772830097408,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":48,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":6573}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-08-27T11:20:22.272298+0000","services":{}},"progress_events":{}} 2023-08-27T11:25:47.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.739+0000 7f9fd0594700 1 -- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9fac06a320 msgr2=0x7f9fac06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.739+0000 7f9fd0594700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9fac06a320 0x7f9fac06c7e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8000f60 tx=0x7f9fb8009040 comp rx=0 tx=0).stop 2023-08-27T11:25:47.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.739+0000 7f9fd0594700 1 -- 172.21.15.112:0/3543036191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fc8151060 msgr2=0x7f9fc80b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.739+0000 7f9fd0594700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fc8151060 0x7f9fc80b1c20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00093c0 tx=0x7f9fc00093f0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.739+0000 7f9fd0594700 1 -- 172.21.15.112:0/3543036191 shutdown_connections 2023-08-27T11:25:47.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.739+0000 7f9fd0594700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9fac06a320 0x7f9fac06c7e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.739+0000 7f9fd0594700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fc8151060 0x7f9fc80b1c20 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.739+0000 7f9fd0594700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc806adb0 0x7f9fc814cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.739+0000 7f9fd0594700 1 --2- 172.21.15.112:0/3543036191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fc80cdbd0 0x7f9fc81f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.739+0000 7f9fd0594700 1 -- 172.21.15.112:0/3543036191 >> 172.21.15.112:0/3543036191 conn(0x7f9fc80d0000 msgr2=0x7f9fc814dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:47.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.739+0000 7f9fd0594700 1 -- 172.21.15.112:0/3543036191 shutdown_connections 2023-08-27T11:25:47.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.739+0000 7f9fd0594700 1 -- 172.21.15.112:0/3543036191 wait complete. 2023-08-27T11:25:47.752 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-27T11:25:47.752 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:25:47.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.871+0000 7fd16603f700 1 Processor -- start 2023-08-27T11:25:47.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.871+0000 7fd16603f700 1 -- start start 2023-08-27T11:25:47.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.871+0000 7fd16603f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1600b1400 0x7fd1600cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.871+0000 7fd16603f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd160161ce0 0x7fd16006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.871+0000 7fd16603f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd160161850 con 0x7fd1600b17e0 2023-08-27T11:25:47.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.871+0000 7fd16603f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1600cfc10 con 0x7fd160161ce0 2023-08-27T11:25:47.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd16603f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd160161200 con 0x7fd1600b1400 2023-08-27T11:25:47.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd15ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd160161ce0 0x7fd16006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd15ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd160161ce0 0x7fd16006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60108/0 (socket says 172.21.15.112:60108) 2023-08-27T11:25:47.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd15ffff700 1 -- 172.21.15.112:0/4057360063 learned_addr learned my addr 172.21.15.112:0/4057360063 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:47.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd15f7fe700 1 --2- 172.21.15.112:0/4057360063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1600b1400 0x7fd1600cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd15ffff700 1 -- 172.21.15.112:0/4057360063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1600b1400 msgr2=0x7fd1600cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:47.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd15ffff700 1 --2- 172.21.15.112:0/4057360063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1600b1400 0x7fd1600cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd15ffff700 1 -- 172.21.15.112:0/4057360063 >> v1:172.21.15.138:6789/0 conn(0x7fd1600b17e0 legacy=0x7fd16006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:47.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd15ffff700 1 -- 172.21.15.112:0/4057360063 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1601614e0 con 0x7fd160161ce0 2023-08-27T11:25:47.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd15f7fe700 1 --2- 172.21.15.112:0/4057360063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1600b1400 0x7fd1600cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:47.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd15ffff700 1 --2- 172.21.15.112:0/4057360063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd160161ce0 0x7fd16006f790 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd158009e50 tx=0x7fd15800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa43e14714247c9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:47.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd15e7fc700 1 -- 172.21.15.112:0/4057360063 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15800e070 con 0x7fd160161ce0 2023-08-27T11:25:47.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd15e7fc700 1 -- 172.21.15.112:0/4057360063 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd158003a20 con 0x7fd160161ce0 2023-08-27T11:25:47.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.872+0000 7fd15e7fc700 1 -- 172.21.15.112:0/4057360063 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158003dd0 con 0x7fd160161ce0 2023-08-27T11:25:47.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.873+0000 7fd16603f700 1 -- 172.21.15.112:0/4057360063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd160161ce0 msgr2=0x7fd16006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.873+0000 7fd16603f700 1 --2- 172.21.15.112:0/4057360063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd160161ce0 0x7fd16006f790 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd158009e50 tx=0x7fd15800b330 comp rx=0 tx=0).stop 2023-08-27T11:25:47.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.873+0000 7fd16603f700 1 -- 172.21.15.112:0/4057360063 shutdown_connections 2023-08-27T11:25:47.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.873+0000 7fd16603f700 1 --2- 172.21.15.112:0/4057360063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd160161ce0 0x7fd16006f790 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.873+0000 7fd16603f700 1 --2- 172.21.15.112:0/4057360063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1600b1400 0x7fd1600cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.873+0000 7fd16603f700 1 -- 172.21.15.112:0/4057360063 >> 172.21.15.112:0/4057360063 conn(0x7fd1600d0000 msgr2=0x7fd1600d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.873+0000 7fd16603f700 1 -- 172.21.15.112:0/4057360063 shutdown_connections 2023-08-27T11:25:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.873+0000 7fd16603f700 1 -- 172.21.15.112:0/4057360063 wait complete. 2023-08-27T11:25:47.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.874+0000 7fd16603f700 1 Processor -- start 2023-08-27T11:25:47.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.874+0000 7fd16603f700 1 -- start start 2023-08-27T11:25:47.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd16603f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1600b1400 0x7fd160155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd16603f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1600b17e0 0x7fd16015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd16603f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd160161ce0 0x7fd16015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd16603f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600c8710 con 0x7fd1600b1400 2023-08-27T11:25:47.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd16603f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd160156610 con 0x7fd1600b17e0 2023-08-27T11:25:47.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd16603f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd160155550 con 0x7fd160161ce0 2023-08-27T11:25:47.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd15f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1600b1400 0x7fd160155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd15f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1600b1400 0x7fd160155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60110/0 (socket says 172.21.15.112:60110) 2023-08-27T11:25:47.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd15f7fe700 1 -- 172.21.15.112:0/3429770726 learned_addr learned my addr 172.21.15.112:0/3429770726 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:47.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd15ffff700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd160161ce0 0x7fd16015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd15effd700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1600b17e0 0x7fd16015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd15f7fe700 1 -- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd160161ce0 msgr2=0x7fd16015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd15f7fe700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd160161ce0 0x7fd16015f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd15f7fe700 1 -- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1600b17e0 msgr2=0x7fd16015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:47.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd15f7fe700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1600b17e0 0x7fd16015ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:47.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.875+0000 7fd15f7fe700 1 -- 172.21.15.112:0/3429770726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd158009b30 con 0x7fd1600b1400 2023-08-27T11:25:47.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.876+0000 7fd15f7fe700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1600b1400 0x7fd160155010 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd15000c750 tx=0x7fd15000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:47.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.876+0000 7fd147fff700 1 -- 172.21.15.112:0/3429770726 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd150011800 con 0x7fd1600b1400 2023-08-27T11:25:47.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.876+0000 7fd147fff700 1 -- 172.21.15.112:0/3429770726 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd150011960 con 0x7fd1600b1400 2023-08-27T11:25:47.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.876+0000 7fd16603f700 1 -- 172.21.15.112:0/3429770726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd160156330 con 0x7fd1600b1400 2023-08-27T11:25:47.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.876+0000 7fd147fff700 1 -- 172.21.15.112:0/3429770726 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd150011cd0 con 0x7fd1600b1400 2023-08-27T11:25:47.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.876+0000 7fd16603f700 1 -- 172.21.15.112:0/3429770726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd160156c70 con 0x7fd1600b1400 2023-08-27T11:25:47.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.877+0000 7fd16603f700 1 -- 172.21.15.112:0/3429770726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16015f6f0 con 0x7fd1600b1400 2023-08-27T11:25:47.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.877+0000 7fd147fff700 1 -- 172.21.15.112:0/3429770726 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fd150021350 con 0x7fd1600b1400 2023-08-27T11:25:47.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.877+0000 7fd147fff700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd14c06a320 0x7fd14c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:47.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.878+0000 7fd147fff700 1 -- 172.21.15.112:0/3429770726 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd1500889a0 con 0x7fd1600b1400 2023-08-27T11:25:47.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.878+0000 7fd15effd700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd14c06a320 0x7fd14c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:47.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.878+0000 7fd15effd700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd14c06a320 0x7fd14c06c7e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd1600ca160 tx=0x7fd154009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:47.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:47.882+0000 7fd147fff700 1 -- 172.21.15.112:0/3429770726 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd16015f6f0 con 0x7fd1600b1400 2023-08-27T11:25:48.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.066+0000 7fd16603f700 1 -- 172.21.15.112:0/3429770726 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd16015f6f0 con 0x7fd14c06a320 2023-08-27T11:25:48.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.075+0000 7fd147fff700 1 -- 172.21.15.112:0/3429770726 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306600 (secure 0 0 0) 0x7fd16015f6f0 con 0x7fd14c06a320 2023-08-27T11:25:48.080 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:48.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.078+0000 7fd16603f700 1 -- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd14c06a320 msgr2=0x7fd14c06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:48.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.078+0000 7fd16603f700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd14c06a320 0x7fd14c06c7e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd1600ca160 tx=0x7fd154009040 comp rx=0 tx=0).stop 2023-08-27T11:25:48.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.078+0000 7fd16603f700 1 -- 172.21.15.112:0/3429770726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1600b1400 msgr2=0x7fd160155010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:48.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.078+0000 7fd16603f700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1600b1400 0x7fd160155010 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd15000c750 tx=0x7fd15000cb10 comp rx=0 tx=0).stop 2023-08-27T11:25:48.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.078+0000 7fd16603f700 1 -- 172.21.15.112:0/3429770726 shutdown_connections 2023-08-27T11:25:48.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.078+0000 7fd16603f700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd14c06a320 0x7fd14c06c7e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:48.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.078+0000 7fd16603f700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd160161ce0 0x7fd16015f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:48.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.078+0000 7fd16603f700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1600b1400 0x7fd160155010 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:48.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.078+0000 7fd16603f700 1 --2- 172.21.15.112:0/3429770726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1600b17e0 0x7fd16015ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:48.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.078+0000 7fd16603f700 1 -- 172.21.15.112:0/3429770726 >> 172.21.15.112:0/3429770726 conn(0x7fd1600d0000 msgr2=0x7fd1600ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:48.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.079+0000 7fd16603f700 1 -- 172.21.15.112:0/3429770726 shutdown_connections 2023-08-27T11:25:48.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:48.079+0000 7fd16603f700 1 -- 172.21.15.112:0/3429770726 wait complete. 2023-08-27T11:25:48.085 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:25:48.092 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2023-08-27T11:25:46.359922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257828,"kb_used_data":8356,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754716892,"statfs":{"total":773094113280,"available":772830097408,"internally_reserved":0,"allocated":8556544,"data_stored":4526431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.968705"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396787+0000","last_change":"2023-08-27T11:25:41.396787+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396787+0000","last_undegraded":"2023-08-27T11:25:41.396787+0000","last_fullsized":"2023-08-27T11:25:41.396787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.408023+0000","last_change":"2023-08-27T11:25:41.408023+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.408023+0000","last_undegraded":"2023-08-27T11:25:41.408023+0000","last_fullsized":"2023-08-27T11:25:41.408023+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396335+0000","last_change":"2023-08-27T11:25:41.396335+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396335+0000","last_undegraded":"2023-08-27T11:25:41.396335+0000","last_fullsized":"2023-08-27T11:25:41.396335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397019+0000","last_change":"2023-08-27T11:25:41.397019+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397019+0000","last_undegraded":"2023-08-27T11:25:41.397019+0000","last_fullsized":"2023-08-27T11:25:41.397019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399662+0000","last_change":"2023-08-27T11:25:41.399662+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399662+0000","last_undegraded":"2023-08-27T11:25:41.399662+0000","last_fullsized":"2023-08-27T11:25:41.399662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.406043+0000","last_change":"2023-08-27T11:25:41.406043+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.406043+0000","last_undegraded":"2023-08-27T11:25:41.406043+0000","last_fullsized":"2023-08-27T11:25:41.406043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400258+0000","last_change":"2023-08-27T11:25:41.400258+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400258+0000","last_undegraded":"2023-08-27T11:25:41.400258+0000","last_fullsized":"2023-08-27T11:25:41.400258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398869+0000","last_change":"2023-08-27T11:25:41.398869+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398869+0000","last_undegraded":"2023-08-27T11:25:41.398869+0000","last_fullsized":"2023-08-27T11:25:41.398869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398915+0000","last_change":"2023-08-27T11:25:41.398915+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398915+0000","last_undegraded":"2023-08-27T11:25:41.398915+0000","last_fullsized":"2023-08-27T11:25:41.398915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398563+0000","last_change":"2023-08-27T11:25:41.398563+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398563+0000","last_undegraded":"2023-08-27T11:25:41.398563+0000","last_fullsized":"2023-08-27T11:25:41.398563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.401519+0000","last_change":"2023-08-27T11:25:41.401519+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.401519+0000","last_undegraded":"2023-08-27T11:25:41.401519+0000","last_fullsized":"2023-08-27T11:25:41.401519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396135+0000","last_change":"2023-08-27T11:25:41.396135+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396135+0000","last_undegraded":"2023-08-27T11:25:41.396135+0000","last_fullsized":"2023-08-27T11:25:41.396135+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398767+0000","last_change":"2023-08-27T11:25:41.398767+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398767+0000","last_undegraded":"2023-08-27T11:25:41.398767+0000","last_fullsized":"2023-08-27T11:25:41.398767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.402238+0000","last_change":"2023-08-27T11:25:41.402238+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.402238+0000","last_undegraded":"2023-08-27T11:25:41.402238+0000","last_fullsized":"2023-08-27T11:25:41.402238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400006+0000","last_change":"2023-08-27T11:25:41.400006+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400006+0000","last_undegraded":"2023-08-27T11:25:41.400006+0000","last_fullsized":"2023-08-27T11:25:41.400006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403883+0000","last_change":"2023-08-27T11:25:41.403883+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403883+0000","last_undegraded":"2023-08-27T11:25:41.403883+0000","last_fullsized":"2023-08-27T11:25:41.403883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403836+0000","last_change":"2023-08-27T11:25:41.403836+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403836+0000","last_undegraded":"2023-08-27T11:25:41.403836+0000","last_fullsized":"2023-08-27T11:25:41.403836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396268+0000","last_change":"2023-08-27T11:25:41.396268+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396268+0000","last_undegraded":"2023-08-27T11:25:41.396268+0000","last_fullsized":"2023-08-27T11:25:41.396268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400999+0000","last_change":"2023-08-27T11:25:41.400999+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400999+0000","last_undegraded":"2023-08-27T11:25:41.400999+0000","last_fullsized":"2023-08-27T11:25:41.400999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399588+0000","last_change":"2023-08-27T11:25:41.399588+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399588+0000","last_undegraded":"2023-08-27T11:25:41.399588+0000","last_fullsized":"2023-08-27T11:25:41.399588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399193+0000","last_change":"2023-08-27T11:25:41.399193+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399193+0000","last_undegraded":"2023-08-27T11:25:41.399193+0000","last_fullsized":"2023-08-27T11:25:41.399193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400790+0000","last_change":"2023-08-27T11:25:41.400790+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400790+0000","last_undegraded":"2023-08-27T11:25:41.400790+0000","last_fullsized":"2023-08-27T11:25:41.400790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.407276+0000","last_change":"2023-08-27T11:25:41.407276+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.407276+0000","last_undegraded":"2023-08-27T11:25:41.407276+0000","last_fullsized":"2023-08-27T11:25:41.407276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395723+0000","last_change":"2023-08-27T11:25:41.395723+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395723+0000","last_undegraded":"2023-08-27T11:25:41.395723+0000","last_fullsized":"2023-08-27T11:25:41.395723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.408337+0000","last_change":"2023-08-27T11:25:41.408337+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.408337+0000","last_undegraded":"2023-08-27T11:25:41.408337+0000","last_fullsized":"2023-08-27T11:25:41.408337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.401694+0000","last_change":"2023-08-27T11:25:41.401694+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.401694+0000","last_undegraded":"2023-08-27T11:25:41.401694+0000","last_fullsized":"2023-08-27T11:25:41.401694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395708+0000","last_change":"2023-08-27T11:25:41.395708+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395708+0000","last_undegraded":"2023-08-27T11:25:41.395708+0000","last_fullsized":"2023-08-27T11:25:41.395708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399209+0000","last_change":"2023-08-27T11:25:41.399209+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399209+0000","last_undegraded":"2023-08-27T11:25:41.399209+0000","last_fullsized":"2023-08-27T11:25:41.399209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399718+0000","last_change":"2023-08-27T11:25:41.399718+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399718+0000","last_undegraded":"2023-08-27T11:25:41.399718+0000","last_fullsized":"2023-08-27T11:25:41.399718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400210+0000","last_change":"2023-08-27T11:25:41.400210+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400210+0000","last_undegraded":"2023-08-27T11:25:41.400210+0000","last_fullsized":"2023-08-27T11:25:41.400210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.409672+0000","last_change":"2023-08-27T11:25:41.409672+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.409672+0000","last_undegraded":"2023-08-27T11:25:41.409672+0000","last_fullsized":"2023-08-27T11:25:41.409672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403514+0000","last_change":"2023-08-27T11:25:41.403514+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403514+0000","last_undegraded":"2023-08-27T11:25:41.403514+0000","last_fullsized":"2023-08-27T11:25:41.403514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.402400+0000","last_change":"2023-08-27T11:25:41.402400+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.402400+0000","last_undegraded":"2023-08-27T11:25:41.402400+0000","last_fullsized":"2023-08-27T11:25:41.402400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.419031+0000","last_change":"2023-08-27T11:25:41.414884+0000","last_active":"2023-08-27T11:25:41.419031+0000","last_peered":"2023-08-27T11:25:41.419031+0000","last_clean":"2023-08-27T11:25:41.419031+0000","last_became_active":"2023-08-27T11:25:41.414549+0000","last_became_peered":"2023-08-27T11:25:41.414549+0000","last_unstale":"2023-08-27T11:25:41.419031+0000","last_undegraded":"2023-08-27T11:25:41.419031+0000","last_fullsized":"2023-08-27T11:25:41.419031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:31:13.509254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403856+0000","last_change":"2023-08-27T11:25:41.403856+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403856+0000","last_undegraded":"2023-08-27T11:25:41.403856+0000","last_fullsized":"2023-08-27T11:25:41.403856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.416537+0000","last_change":"2023-08-27T11:25:41.414847+0000","last_active":"2023-08-27T11:25:41.416537+0000","last_peered":"2023-08-27T11:25:41.416537+0000","last_clean":"2023-08-27T11:25:41.416537+0000","last_became_active":"2023-08-27T11:25:41.414532+0000","last_became_peered":"2023-08-27T11:25:41.414532+0000","last_unstale":"2023-08-27T11:25:41.416537+0000","last_undegraded":"2023-08-27T11:25:41.416537+0000","last_fullsized":"2023-08-27T11:25:41.416537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:52:27.302892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399003+0000","last_change":"2023-08-27T11:25:41.399003+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399003+0000","last_undegraded":"2023-08-27T11:25:41.399003+0000","last_fullsized":"2023-08-27T11:25:41.399003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.416340+0000","last_change":"2023-08-27T11:25:41.412179+0000","last_active":"2023-08-27T11:25:41.416340+0000","last_peered":"2023-08-27T11:25:41.416340+0000","last_clean":"2023-08-27T11:25:41.416340+0000","last_became_active":"2023-08-27T11:25:41.411940+0000","last_became_peered":"2023-08-27T11:25:41.411940+0000","last_unstale":"2023-08-27T11:25:41.416340+0000","last_undegraded":"2023-08-27T11:25:41.416340+0000","last_fullsized":"2023-08-27T11:25:41.416340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:01:10.875523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.403413+0000","last_change":"2023-08-27T11:25:41.403413+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.403413+0000","last_undegraded":"2023-08-27T11:25:41.403413+0000","last_fullsized":"2023-08-27T11:25:41.403413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418955+0000","last_change":"2023-08-27T11:25:41.414002+0000","last_active":"2023-08-27T11:25:41.418955+0000","last_peered":"2023-08-27T11:25:41.418955+0000","last_clean":"2023-08-27T11:25:41.418955+0000","last_became_active":"2023-08-27T11:25:41.413740+0000","last_became_peered":"2023-08-27T11:25:41.413740+0000","last_unstale":"2023-08-27T11:25:41.418955+0000","last_undegraded":"2023-08-27T11:25:41.418955+0000","last_fullsized":"2023-08-27T11:25:41.418955+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:26:23.153384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397310+0000","last_change":"2023-08-27T11:25:41.397310+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397310+0000","last_undegraded":"2023-08-27T11:25:41.397310+0000","last_fullsized":"2023-08-27T11:25:41.397310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.406853+0000","last_change":"2023-08-27T11:25:41.405587+0000","last_active":"2023-08-27T11:25:41.406853+0000","last_peered":"2023-08-27T11:25:41.406853+0000","last_clean":"2023-08-27T11:25:41.406853+0000","last_became_active":"2023-08-27T11:25:41.405255+0000","last_became_peered":"2023-08-27T11:25:41.405255+0000","last_unstale":"2023-08-27T11:25:41.406853+0000","last_undegraded":"2023-08-27T11:25:41.406853+0000","last_fullsized":"2023-08-27T11:25:41.406853+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:09:28.315159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.406628+0000","last_change":"2023-08-27T11:25:41.406628+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.406628+0000","last_undegraded":"2023-08-27T11:25:41.406628+0000","last_fullsized":"2023-08-27T11:25:41.406628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.409295+0000","last_change":"2023-08-27T11:25:41.406787+0000","last_active":"2023-08-27T11:25:41.409295+0000","last_peered":"2023-08-27T11:25:41.409295+0000","last_clean":"2023-08-27T11:25:41.409295+0000","last_became_active":"2023-08-27T11:25:41.406253+0000","last_became_peered":"2023-08-27T11:25:41.406253+0000","last_unstale":"2023-08-27T11:25:41.409295+0000","last_undegraded":"2023-08-27T11:25:41.409295+0000","last_fullsized":"2023-08-27T11:25:41.409295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:38.768166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397479+0000","last_change":"2023-08-27T11:25:41.397479+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397479+0000","last_undegraded":"2023-08-27T11:25:41.397479+0000","last_fullsized":"2023-08-27T11:25:41.397479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.408833+0000","last_change":"2023-08-27T11:25:41.404377+0000","last_active":"2023-08-27T11:25:41.408833+0000","last_peered":"2023-08-27T11:25:41.408833+0000","last_clean":"2023-08-27T11:25:41.408833+0000","last_became_active":"2023-08-27T11:25:41.404056+0000","last_became_peered":"2023-08-27T11:25:41.404056+0000","last_unstale":"2023-08-27T11:25:41.408833+0000","last_undegraded":"2023-08-27T11:25:41.408833+0000","last_fullsized":"2023-08-27T11:25:41.408833+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:16:51.600797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396184+0000","last_change":"2023-08-27T11:25:41.396184+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396184+0000","last_undegraded":"2023-08-27T11:25:41.396184+0000","last_fullsized":"2023-08-27T11:25:41.396184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411092+0000","last_change":"2023-08-27T11:25:41.407540+0000","last_active":"2023-08-27T11:25:41.411092+0000","last_peered":"2023-08-27T11:25:41.411092+0000","last_clean":"2023-08-27T11:25:41.411092+0000","last_became_active":"2023-08-27T11:25:41.406794+0000","last_became_peered":"2023-08-27T11:25:41.406794+0000","last_unstale":"2023-08-27T11:25:41.411092+0000","last_undegraded":"2023-08-27T11:25:41.411092+0000","last_fullsized":"2023-08-27T11:25:41.411092+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:10:38.722806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398510+0000","last_change":"2023-08-27T11:25:41.398510+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398510+0000","last_undegraded":"2023-08-27T11:25:41.398510+0000","last_fullsized":"2023-08-27T11:25:41.398510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421891+0000","last_change":"2023-08-27T11:25:41.417604+0000","last_active":"2023-08-27T11:25:41.421891+0000","last_peered":"2023-08-27T11:25:41.421891+0000","last_clean":"2023-08-27T11:25:41.421891+0000","last_became_active":"2023-08-27T11:25:41.416727+0000","last_became_peered":"2023-08-27T11:25:41.416727+0000","last_unstale":"2023-08-27T11:25:41.421891+0000","last_undegraded":"2023-08-27T11:25:41.421891+0000","last_fullsized":"2023-08-27T11:25:41.421891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:30.061949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398968+0000","last_change":"2023-08-27T11:25:41.398968+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398968+0000","last_undegraded":"2023-08-27T11:25:41.398968+0000","last_fullsized":"2023-08-27T11:25:41.398968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.406341+0000","last_change":"2023-08-27T11:25:41.402868+0000","last_active":"2023-08-27T11:25:41.406341+0000","last_peered":"2023-08-27T11:25:41.406341+0000","last_clean":"2023-08-27T11:25:41.406341+0000","last_became_active":"2023-08-27T11:25:41.402585+0000","last_became_peered":"2023-08-27T11:25:41.402585+0000","last_unstale":"2023-08-27T11:25:41.406341+0000","last_undegraded":"2023-08-27T11:25:41.406341+0000","last_fullsized":"2023-08-27T11:25:41.406341+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:24:45.015907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395873+0000","last_change":"2023-08-27T11:25:41.395873+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395873+0000","last_undegraded":"2023-08-27T11:25:41.395873+0000","last_fullsized":"2023-08-27T11:25:41.395873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.413847+0000","last_change":"2023-08-27T11:25:41.408134+0000","last_active":"2023-08-27T11:25:41.413847+0000","last_peered":"2023-08-27T11:25:41.413847+0000","last_clean":"2023-08-27T11:25:41.413847+0000","last_became_active":"2023-08-27T11:25:41.407806+0000","last_became_peered":"2023-08-27T11:25:41.407806+0000","last_unstale":"2023-08-27T11:25:41.413847+0000","last_undegraded":"2023-08-27T11:25:41.413847+0000","last_fullsized":"2023-08-27T11:25:41.413847+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:38:30.962549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396189+0000","last_change":"2023-08-27T11:25:41.396189+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396189+0000","last_undegraded":"2023-08-27T11:25:41.396189+0000","last_fullsized":"2023-08-27T11:25:41.396189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.415329+0000","last_change":"2023-08-27T11:25:41.414641+0000","last_active":"2023-08-27T11:25:41.415329+0000","last_peered":"2023-08-27T11:25:41.415329+0000","last_clean":"2023-08-27T11:25:41.415329+0000","last_became_active":"2023-08-27T11:25:41.414361+0000","last_became_peered":"2023-08-27T11:25:41.414361+0000","last_unstale":"2023-08-27T11:25:41.415329+0000","last_undegraded":"2023-08-27T11:25:41.415329+0000","last_fullsized":"2023-08-27T11:25:41.415329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:14.192009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396572+0000","last_change":"2023-08-27T11:25:41.396572+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396572+0000","last_undegraded":"2023-08-27T11:25:41.396572+0000","last_fullsized":"2023-08-27T11:25:41.396572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.413558+0000","last_change":"2023-08-27T11:25:41.410338+0000","last_active":"2023-08-27T11:25:41.413558+0000","last_peered":"2023-08-27T11:25:41.413558+0000","last_clean":"2023-08-27T11:25:41.413558+0000","last_became_active":"2023-08-27T11:25:41.410005+0000","last_became_peered":"2023-08-27T11:25:41.410005+0000","last_unstale":"2023-08-27T11:25:41.413558+0000","last_undegraded":"2023-08-27T11:25:41.413558+0000","last_fullsized":"2023-08-27T11:25:41.413558+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:22:28.572307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398507+0000","last_change":"2023-08-27T11:25:41.398507+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398507+0000","last_undegraded":"2023-08-27T11:25:41.398507+0000","last_fullsized":"2023-08-27T11:25:41.398507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.412023+0000","last_change":"2023-08-27T11:25:41.408888+0000","last_active":"2023-08-27T11:25:41.412023+0000","last_peered":"2023-08-27T11:25:41.412023+0000","last_clean":"2023-08-27T11:25:41.412023+0000","last_became_active":"2023-08-27T11:25:41.408609+0000","last_became_peered":"2023-08-27T11:25:41.408609+0000","last_unstale":"2023-08-27T11:25:41.412023+0000","last_undegraded":"2023-08-27T11:25:41.412023+0000","last_fullsized":"2023-08-27T11:25:41.412023+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:12:28.171979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395468+0000","last_change":"2023-08-27T11:25:41.395468+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395468+0000","last_undegraded":"2023-08-27T11:25:41.395468+0000","last_fullsized":"2023-08-27T11:25:41.395468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.415697+0000","last_change":"2023-08-27T11:25:41.414552+0000","last_active":"2023-08-27T11:25:41.415697+0000","last_peered":"2023-08-27T11:25:41.415697+0000","last_clean":"2023-08-27T11:25:41.415697+0000","last_became_active":"2023-08-27T11:25:41.414281+0000","last_became_peered":"2023-08-27T11:25:41.414281+0000","last_unstale":"2023-08-27T11:25:41.415697+0000","last_undegraded":"2023-08-27T11:25:41.415697+0000","last_fullsized":"2023-08-27T11:25:41.415697+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:40:04.874008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395170+0000","last_change":"2023-08-27T11:25:41.395170+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395170+0000","last_undegraded":"2023-08-27T11:25:41.395170+0000","last_fullsized":"2023-08-27T11:25:41.395170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411147+0000","last_change":"2023-08-27T11:25:41.407400+0000","last_active":"2023-08-27T11:25:41.411147+0000","last_peered":"2023-08-27T11:25:41.411147+0000","last_clean":"2023-08-27T11:25:41.411147+0000","last_became_active":"2023-08-27T11:25:41.405982+0000","last_became_peered":"2023-08-27T11:25:41.405982+0000","last_unstale":"2023-08-27T11:25:41.411147+0000","last_undegraded":"2023-08-27T11:25:41.411147+0000","last_fullsized":"2023-08-27T11:25:41.411147+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:59.630816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.407857+0000","last_change":"2023-08-27T11:25:41.407857+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.407857+0000","last_undegraded":"2023-08-27T11:25:41.407857+0000","last_fullsized":"2023-08-27T11:25:41.407857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418905+0000","last_change":"2023-08-27T11:25:41.415726+0000","last_active":"2023-08-27T11:25:41.418905+0000","last_peered":"2023-08-27T11:25:41.418905+0000","last_clean":"2023-08-27T11:25:41.418905+0000","last_became_active":"2023-08-27T11:25:41.415364+0000","last_became_peered":"2023-08-27T11:25:41.415364+0000","last_unstale":"2023-08-27T11:25:41.418905+0000","last_undegraded":"2023-08-27T11:25:41.418905+0000","last_fullsized":"2023-08-27T11:25:41.418905+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:18:37.632552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397995+0000","last_change":"2023-08-27T11:25:41.397995+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397995+0000","last_undegraded":"2023-08-27T11:25:41.397995+0000","last_fullsized":"2023-08-27T11:25:41.397995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418831+0000","last_change":"2023-08-27T11:25:41.417094+0000","last_active":"2023-08-27T11:25:41.418831+0000","last_peered":"2023-08-27T11:25:41.418831+0000","last_clean":"2023-08-27T11:25:41.418831+0000","last_became_active":"2023-08-27T11:25:41.416795+0000","last_became_peered":"2023-08-27T11:25:41.416795+0000","last_unstale":"2023-08-27T11:25:41.418831+0000","last_undegraded":"2023-08-27T11:25:41.418831+0000","last_fullsized":"2023-08-27T11:25:41.418831+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:33:47.717016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.397502+0000","last_change":"2023-08-27T11:25:41.397502+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.397502+0000","last_undegraded":"2023-08-27T11:25:41.397502+0000","last_fullsized":"2023-08-27T11:25:41.397502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.413823+0000","last_change":"2023-08-27T11:25:41.408039+0000","last_active":"2023-08-27T11:25:41.413823+0000","last_peered":"2023-08-27T11:25:41.413823+0000","last_clean":"2023-08-27T11:25:41.413823+0000","last_became_active":"2023-08-27T11:25:41.407709+0000","last_became_peered":"2023-08-27T11:25:41.407709+0000","last_unstale":"2023-08-27T11:25:41.413823+0000","last_undegraded":"2023-08-27T11:25:41.413823+0000","last_fullsized":"2023-08-27T11:25:41.413823+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:34:44.092855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.404340+0000","last_change":"2023-08-27T11:25:41.404340+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.404340+0000","last_undegraded":"2023-08-27T11:25:41.404340+0000","last_fullsized":"2023-08-27T11:25:41.404340+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418975+0000","last_change":"2023-08-27T11:25:41.414018+0000","last_active":"2023-08-27T11:25:41.418975+0000","last_peered":"2023-08-27T11:25:41.418975+0000","last_clean":"2023-08-27T11:25:41.418975+0000","last_became_active":"2023-08-27T11:25:41.413719+0000","last_became_peered":"2023-08-27T11:25:41.413719+0000","last_unstale":"2023-08-27T11:25:41.418975+0000","last_undegraded":"2023-08-27T11:25:41.418975+0000","last_fullsized":"2023-08-27T11:25:41.418975+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:51:04.452644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.398022+0000","last_change":"2023-08-27T11:25:41.398022+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.398022+0000","last_undegraded":"2023-08-27T11:25:41.398022+0000","last_fullsized":"2023-08-27T11:25:41.398022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411981+0000","last_change":"2023-08-27T11:25:41.408665+0000","last_active":"2023-08-27T11:25:41.411981+0000","last_peered":"2023-08-27T11:25:41.411981+0000","last_clean":"2023-08-27T11:25:41.411981+0000","last_became_active":"2023-08-27T11:25:41.408402+0000","last_became_peered":"2023-08-27T11:25:41.408402+0000","last_unstale":"2023-08-27T11:25:41.411981+0000","last_undegraded":"2023-08-27T11:25:41.411981+0000","last_fullsized":"2023-08-27T11:25:41.411981+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:25:55.452027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395170+0000","last_change":"2023-08-27T11:25:41.395170+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395170+0000","last_undegraded":"2023-08-27T11:25:41.395170+0000","last_fullsized":"2023-08-27T11:25:41.395170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421291+0000","last_change":"2023-08-27T11:25:41.411569+0000","last_active":"2023-08-27T11:25:41.421291+0000","last_peered":"2023-08-27T11:25:41.421291+0000","last_clean":"2023-08-27T11:25:41.421291+0000","last_became_active":"2023-08-27T11:25:41.411296+0000","last_became_peered":"2023-08-27T11:25:41.411296+0000","last_unstale":"2023-08-27T11:25:41.421291+0000","last_undegraded":"2023-08-27T11:25:41.421291+0000","last_fullsized":"2023-08-27T11:25:41.421291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:19:40.101939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.406076+0000","last_change":"2023-08-27T11:25:41.406076+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.406076+0000","last_undegraded":"2023-08-27T11:25:41.406076+0000","last_fullsized":"2023-08-27T11:25:41.406076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.416239+0000","last_change":"2023-08-27T11:25:41.412882+0000","last_active":"2023-08-27T11:25:41.416239+0000","last_peered":"2023-08-27T11:25:41.416239+0000","last_clean":"2023-08-27T11:25:41.416239+0000","last_became_active":"2023-08-27T11:25:41.412635+0000","last_became_peered":"2023-08-27T11:25:41.412635+0000","last_unstale":"2023-08-27T11:25:41.416239+0000","last_undegraded":"2023-08-27T11:25:41.416239+0000","last_fullsized":"2023-08-27T11:25:41.416239+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:10.546956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396619+0000","last_change":"2023-08-27T11:25:41.396619+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396619+0000","last_undegraded":"2023-08-27T11:25:41.396619+0000","last_fullsized":"2023-08-27T11:25:41.396619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.406484+0000","last_change":"2023-08-27T11:25:41.405653+0000","last_active":"2023-08-27T11:25:41.406484+0000","last_peered":"2023-08-27T11:25:41.406484+0000","last_clean":"2023-08-27T11:25:41.406484+0000","last_became_active":"2023-08-27T11:25:41.405385+0000","last_became_peered":"2023-08-27T11:25:41.405385+0000","last_unstale":"2023-08-27T11:25:41.406484+0000","last_undegraded":"2023-08-27T11:25:41.406484+0000","last_fullsized":"2023-08-27T11:25:41.406484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:55.963991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.395911+0000","last_change":"2023-08-27T11:25:41.395911+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.395911+0000","last_undegraded":"2023-08-27T11:25:41.395911+0000","last_fullsized":"2023-08-27T11:25:41.395911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421836+0000","last_change":"2023-08-27T11:25:41.417399+0000","last_active":"2023-08-27T11:25:41.421836+0000","last_peered":"2023-08-27T11:25:41.421836+0000","last_clean":"2023-08-27T11:25:41.421836+0000","last_became_active":"2023-08-27T11:25:41.417076+0000","last_became_peered":"2023-08-27T11:25:41.417076+0000","last_unstale":"2023-08-27T11:25:41.421836+0000","last_undegraded":"2023-08-27T11:25:41.421836+0000","last_fullsized":"2023-08-27T11:25:41.421836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:55:39.763853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":133,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.409138+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:25:41.409138+0000","last_peered":"2023-08-27T11:25:41.409138+0000","last_clean":"2023-08-27T11:25:41.409138+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:25:41.409138+0000","last_undegraded":"2023-08-27T11:25:41.409138+0000","last_fullsized":"2023-08-27T11:25:41.409138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:02:20.142155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.404145+0000","last_change":"2023-08-27T11:25:41.404145+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.404145+0000","last_undegraded":"2023-08-27T11:25:41.404145+0000","last_fullsized":"2023-08-27T11:25:41.404145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411964+0000","last_change":"2023-08-27T11:25:41.405709+0000","last_active":"2023-08-27T11:25:41.411964+0000","last_peered":"2023-08-27T11:25:41.411964+0000","last_clean":"2023-08-27T11:25:41.411964+0000","last_became_active":"2023-08-27T11:25:41.405421+0000","last_became_peered":"2023-08-27T11:25:41.405421+0000","last_unstale":"2023-08-27T11:25:41.411964+0000","last_undegraded":"2023-08-27T11:25:41.411964+0000","last_fullsized":"2023-08-27T11:25:41.411964+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:50:55.448426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":133,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.408951+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:25:41.408951+0000","last_peered":"2023-08-27T11:25:41.408951+0000","last_clean":"2023-08-27T11:25:41.408951+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:25:41.408951+0000","last_undegraded":"2023-08-27T11:25:41.408951+0000","last_fullsized":"2023-08-27T11:25:41.408951+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:50:30.904830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.400360+0000","last_change":"2023-08-27T11:25:41.400360+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.400360+0000","last_undegraded":"2023-08-27T11:25:41.400360+0000","last_fullsized":"2023-08-27T11:25:41.400360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.410077+0000","last_change":"2023-08-27T11:25:41.407112+0000","last_active":"2023-08-27T11:25:41.410077+0000","last_peered":"2023-08-27T11:25:41.410077+0000","last_clean":"2023-08-27T11:25:41.410077+0000","last_became_active":"2023-08-27T11:25:41.406794+0000","last_became_peered":"2023-08-27T11:25:41.406794+0000","last_unstale":"2023-08-27T11:25:41.410077+0000","last_undegraded":"2023-08-27T11:25:41.410077+0000","last_fullsized":"2023-08-27T11:25:41.410077+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:01:40.046194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.411219+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:25:41.411219+0000","last_peered":"2023-08-27T11:25:41.411219+0000","last_clean":"2023-08-27T11:25:41.411219+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:25:41.411219+0000","last_undegraded":"2023-08-27T11:25:41.411219+0000","last_fullsized":"2023-08-27T11:25:41.411219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:26:47.289516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.401250+0000","last_change":"2023-08-27T11:25:41.401250+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.401250+0000","last_undegraded":"2023-08-27T11:25:41.401250+0000","last_fullsized":"2023-08-27T11:25:41.401250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.418369+0000","last_change":"2023-08-27T11:25:41.417819+0000","last_active":"2023-08-27T11:25:41.418369+0000","last_peered":"2023-08-27T11:25:41.418369+0000","last_clean":"2023-08-27T11:25:41.418369+0000","last_became_active":"2023-08-27T11:25:41.417532+0000","last_became_peered":"2023-08-27T11:25:41.417532+0000","last_unstale":"2023-08-27T11:25:41.418369+0000","last_undegraded":"2023-08-27T11:25:41.418369+0000","last_fullsized":"2023-08-27T11:25:41.418369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:34.714411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.399323+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:25:41.399323+0000","last_peered":"2023-08-27T11:25:41.399323+0000","last_clean":"2023-08-27T11:25:41.399323+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:25:41.399323+0000","last_undegraded":"2023-08-27T11:25:41.399323+0000","last_fullsized":"2023-08-27T11:25:41.399323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:14:18.441956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.405779+0000","last_change":"2023-08-27T11:25:41.405779+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.405779+0000","last_undegraded":"2023-08-27T11:25:41.405779+0000","last_fullsized":"2023-08-27T11:25:41.405779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421309+0000","last_change":"2023-08-27T11:25:41.417161+0000","last_active":"2023-08-27T11:25:41.421309+0000","last_peered":"2023-08-27T11:25:41.421309+0000","last_clean":"2023-08-27T11:25:41.421309+0000","last_became_active":"2023-08-27T11:25:41.416886+0000","last_became_peered":"2023-08-27T11:25:41.416886+0000","last_unstale":"2023-08-27T11:25:41.421309+0000","last_undegraded":"2023-08-27T11:25:41.421309+0000","last_fullsized":"2023-08-27T11:25:41.421309+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:28:04.377394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.400152+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:25:41.400152+0000","last_peered":"2023-08-27T11:25:41.400152+0000","last_clean":"2023-08-27T11:25:41.400152+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:25:41.400152+0000","last_undegraded":"2023-08-27T11:25:41.400152+0000","last_fullsized":"2023-08-27T11:25:41.400152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:28.303971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.399721+0000","last_change":"2023-08-27T11:25:41.399721+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.399721+0000","last_undegraded":"2023-08-27T11:25:41.399721+0000","last_fullsized":"2023-08-27T11:25:41.399721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.419014+0000","last_change":"2023-08-27T11:25:41.414866+0000","last_active":"2023-08-27T11:25:41.419014+0000","last_peered":"2023-08-27T11:25:41.419014+0000","last_clean":"2023-08-27T11:25:41.419014+0000","last_became_active":"2023-08-27T11:25:41.414525+0000","last_became_peered":"2023-08-27T11:25:41.414525+0000","last_unstale":"2023-08-27T11:25:41.419014+0000","last_undegraded":"2023-08-27T11:25:41.419014+0000","last_fullsized":"2023-08-27T11:25:41.419014+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:04.857971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":263,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.399176+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:25:41.399176+0000","last_peered":"2023-08-27T11:25:41.399176+0000","last_clean":"2023-08-27T11:25:41.399176+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:25:41.399176+0000","last_undegraded":"2023-08-27T11:25:41.399176+0000","last_fullsized":"2023-08-27T11:25:41.399176+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:00.081196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.399025+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:25:41.399025+0000","last_peered":"2023-08-27T11:25:41.399025+0000","last_clean":"2023-08-27T11:25:41.399025+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:25:41.399025+0000","last_undegraded":"2023-08-27T11:25:41.399025+0000","last_fullsized":"2023-08-27T11:25:41.399025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:39.476760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.396941+0000","last_change":"2023-08-27T11:25:41.396941+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.396941+0000","last_undegraded":"2023-08-27T11:25:41.396941+0000","last_fullsized":"2023-08-27T11:25:41.396941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.417585+0000","last_change":"2023-08-27T11:25:41.416796+0000","last_active":"2023-08-27T11:25:41.417585+0000","last_peered":"2023-08-27T11:25:41.417585+0000","last_clean":"2023-08-27T11:25:41.417585+0000","last_became_active":"2023-08-27T11:25:41.416559+0000","last_became_peered":"2023-08-27T11:25:41.416559+0000","last_unstale":"2023-08-27T11:25:41.417585+0000","last_undegraded":"2023-08-27T11:25:41.417585+0000","last_fullsized":"2023-08-27T11:25:41.417585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:06:23.811399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.393189+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:25:41.393189+0000","last_peered":"2023-08-27T11:25:41.393189+0000","last_clean":"2023-08-27T11:25:41.393189+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:25:41.393189+0000","last_undegraded":"2023-08-27T11:25:41.393189+0000","last_fullsized":"2023-08-27T11:25:41.393189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:49:20.590638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2023-08-27T11:25:41.404181+0000","last_change":"2023-08-27T11:25:41.404181+0000","last_active":"2023-08-27T11:25:41.389437+0000","last_peered":"2023-08-27T11:25:41.389437+0000","last_clean":"2023-08-27T11:25:41.389437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:41.404181+0000","last_undegraded":"2023-08-27T11:25:41.404181+0000","last_fullsized":"2023-08-27T11:25:41.404181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.421868+0000","last_change":"2023-08-27T11:25:41.417495+0000","last_active":"2023-08-27T11:25:41.421868+0000","last_peered":"2023-08-27T11:25:41.421868+0000","last_clean":"2023-08-27T11:25:41.421868+0000","last_became_active":"2023-08-27T11:25:41.416993+0000","last_became_peered":"2023-08-27T11:25:41.416993+0000","last_unstale":"2023-08-27T11:25:41.421868+0000","last_undegraded":"2023-08-27T11:25:41.421868+0000","last_fullsized":"2023-08-27T11:25:41.421868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:30.109945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-08-27T11:25:41.401782+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:25:41.401782+0000","last_peered":"2023-08-27T11:25:41.401782+0000","last_clean":"2023-08-27T11:25:41.401782+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:25:41.401782+0000","last_undegraded":"2023-08-27T11:25:41.401782+0000","last_fullsized":"2023-08-27T11:25:41.401782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:38:57.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":1008750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51200000000000001,"15min":0.48899999999999999},"min":{"1min":0.34000000000000002,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.66200000000000003,"5min":0.79600000000000004,"15min":0.93500000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.49399999999999999},"min":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.252},"max":{"1min":0.73499999999999999,"5min":0.74099999999999999,"15min":0.81499999999999995},"last":0.42799999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.34200000000000003,"15min":0.26600000000000001},"max":{"1min":0.72299999999999998,"5min":0.77500000000000002,"15min":0.872},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54200000000000004,"15min":0.51100000000000001},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.253},"max":{"1min":0.67800000000000005,"5min":0.71999999999999997,"15min":0.84999999999999998},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.48199999999999998},"min":{"1min":0.32400000000000001,"5min":0.29099999999999998,"15min":0.28299999999999997},"max":{"1min":0.63,"5min":0.71699999999999997,"15min":0.79500000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56499999999999995,"15min":0.54400000000000004},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.33800000000000002},"max":{"1min":0.73999999999999999,"5min":0.78700000000000003,"15min":0.90200000000000002},"last":0.44700000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.54800000000000004},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.748,"15min":0.84299999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.48199999999999998,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.70399999999999996,"5min":0.83499999999999996,"15min":0.91700000000000004},"last":0.50600000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49399999999999999,"15min":0.50800000000000001},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.29899999999999999},"max":{"1min":0.72499999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.312},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52900000000000003,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72099999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38200000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.31900000000000001},"max":{"1min":0.749,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.43099999999999999,"5min":0.377,"15min":0.36799999999999999},"max":{"1min":0.77300000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.35799999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80800000000000005,"5min":0.82399999999999995,"15min":0.86099999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.755,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73099999999999998,"5min":0.86199999999999999,"15min":0.89800000000000002},"last":0.54600000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.316,"5min":0.312,"15min":0.312},"max":{"1min":0.75900000000000001,"5min":0.80600000000000005,"15min":0.873},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75600000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58799999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64700000000000002,"5min":0.74199999999999999,"15min":0.85899999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55800000000000005,"15min":0.58799999999999997},"min":{"1min":0.41699999999999998,"5min":0.39400000000000002,"15min":0.377},"max":{"1min":0.68600000000000005,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.60099999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.78900000000000003,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.70199999999999996,"5min":0.84699999999999998,"15min":0.93799999999999994},"last":0.56999999999999995}]},{"osd":4,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.32400000000000001,"5min":0.307,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.437,"5min":0.375,"15min":0.374},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.61799999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33400000000000002},"max":{"1min":0.63400000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.60099999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.875,"15min":0.875},"last":0.70599999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.47599999999999998,"5min":0.47099999999999997,"15min":0.34899999999999998},"max":{"1min":0.68200000000000005,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":1008750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.31,"15min":0.63400000000000001},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":0.502,"5min":0.502,"15min":11.971},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.96899999999999997},"min":{"1min":0.22500000000000001,"5min":0.221,"15min":0.20799999999999999},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.23300000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33700000000000002,"15min":0.66400000000000003},"min":{"1min":0.247,"5min":0.221,"15min":0.221},"max":{"1min":0.46600000000000003,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.33800000000000002,"15min":0.34399999999999997},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":0.48599999999999999,"5min":0.53700000000000003,"15min":0.58599999999999997},"last":0.32300000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.38,"15min":0.67600000000000005},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":10.99},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36699999999999999,"15min":1.002},"min":{"1min":0.29399999999999998,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.505,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.35499999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57199999999999995,"15min":0.60299999999999998},"min":{"1min":0.44500000000000001,"5min":0.36099999999999999,"15min":0.33400000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":1.26},"last":0.623},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55800000000000005,"15min":0.84199999999999997},"min":{"1min":0.38900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.77600000000000002,"5min":0.86299999999999999,"15min":10.957000000000001},"last":0.54500000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57299999999999995,"15min":1.2},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.36199999999999999},"max":{"1min":0.79300000000000004,"5min":0.85299999999999998,"15min":23.016999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.60299999999999998},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.296},"last":0.503}]},{"osd":6,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.57999999999999996,"15min":0.88300000000000001},"min":{"1min":0.434,"5min":0.371,"15min":0.371},"max":{"1min":0.84899999999999998,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":1.2170000000000001},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.52300000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":1.212},"min":{"1min":0.45400000000000001,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.76500000000000001,"5min":0.89600000000000002,"15min":22.988},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.63900000000000001},"min":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86199999999999999,"5min":0.91600000000000004,"15min":1.327},"last":0.57399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":417021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.29299999999999998,"15min":0.31},"min":{"1min":0.214,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.45300000000000001,"5min":0.50900000000000001,"15min":0.52300000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.35099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.62},"last":0.27700000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.33900000000000002},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.54400000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.33700000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.223},"max":{"1min":0.49299999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.25}]},{"osd":3,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.34499999999999997},"min":{"1min":0.249,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.50800000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.36199999999999999,"15min":0.372},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.53800000000000003,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.309}]},{"osd":4,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.56000000000000005},"min":{"1min":0.35399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85399999999999998,"5min":0.92500000000000004,"15min":1.214},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.35799999999999998,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.81000000000000005,"15min":1.2569999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.371,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81000000000000005,"5min":0.82599999999999996,"15min":0.91400000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83999999999999997,"5min":0.91100000000000003,"15min":1.603},"last":0.52900000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79900000000000004,"5min":0.86099999999999999,"15min":0.97599999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73999999999999999,"5min":0.871,"15min":0.871},"last":0.48199999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75800000000000001,"5min":0.94199999999999995,"15min":1.383},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86199999999999999,"5min":0.88900000000000001,"15min":0.92000000000000004},"last":0.56899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.30099999999999999,"15min":0.30199999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20200000000000001},"max":{"1min":0.48899999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.317},"min":{"1min":0.23100000000000001,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.46700000000000003,"5min":0.52500000000000002,"15min":0.629},"last":0.30199999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34200000000000003,"15min":0.34799999999999998},"min":{"1min":0.23000000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52200000000000002,"5min":0.749,"15min":0.749},"last":0.28899999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36299999999999999,"15min":0.38400000000000001},"min":{"1min":0.26600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57099999999999995,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.27500000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.69499999999999995},"last":0.34100000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.42699999999999999,"5min":0.39600000000000002,"15min":0.35599999999999998},"max":{"1min":0.82199999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.42599999999999999,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.75,"5min":0.879,"15min":0.879},"last":0.433}]},{"osd":5,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51300000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.73499999999999999,"5min":0.85199999999999998,"15min":0.85299999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55600000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59699999999999998,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.79500000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574958,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.27700000000000002,"5min":0.30099999999999999,"15min":0.29999999999999999},"min":{"1min":0.21199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.34100000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.374},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32600000000000001,"15min":0.312},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46200000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.246}]},{"osd":1,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.32700000000000001,"15min":0.312},"min":{"1min":0.22700000000000001,"5min":0.21299999999999999,"15min":0.19500000000000001},"max":{"1min":0.48599999999999999,"5min":0.50600000000000001,"15min":0.60399999999999998},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.308,"5min":0.33400000000000002,"15min":0.34000000000000002},"min":{"1min":0.26500000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.38200000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"last":0.30199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44400000000000001,"5min":0.57199999999999995,"15min":0.57599999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.42999999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.26700000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.436,"5min":0.316,"15min":0.28599999999999998},"max":{"1min":0.68200000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68700000000000006,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.78400000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.36799999999999999,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.57299999999999995},"min":{"1min":0.442,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.67000000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.67400000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.373},"max":{"1min":0.73999999999999999,"5min":1.091,"15min":1.091},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60099999999999998,"15min":0.61499999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75600000000000001,"5min":1.111,"15min":1.111},"last":0.60299999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.61799999999999999},"min":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.36699999999999999},"max":{"1min":0.70199999999999996,"5min":1.032,"15min":1.236},"last":0.623},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.58499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68200000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.80700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.32700000000000001,"15min":0.30399999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.44},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.379,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.65100000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54800000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76300000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.433,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.70699999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.44,"5min":0.41799999999999998,"15min":0.41399999999999998},"max":{"1min":0.751,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66100000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58599999999999997,"15min":0.59499999999999997},"min":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.379},"max":{"1min":0.85999999999999999,"5min":0.879,"15min":0.879},"last":0.625}]},{"osd":6,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.55000000000000004},"min":{"1min":0.39600000000000002,"5min":0.37,"15min":0.31},"max":{"1min":0.78900000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.62,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.37,"15min":0.35299999999999998},"max":{"1min":0.76700000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.39300000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.82499999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.34200000000000003},"max":{"1min":0.79200000000000004,"5min":0.83799999999999997,"15min":0.89300000000000002},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574957,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":418382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.63300000000000001,"5min":0.78600000000000003,"15min":0.96899999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.29199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.60099999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70499999999999996,"5min":0.747,"15min":0.92200000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52400000000000002,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.63200000000000001,"5min":0.72899999999999998,"15min":1.075},"last":0.53500000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.78000000000000003,"15min":0.95699999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54600000000000004,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.65300000000000002,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.56299999999999994}]},{"osd":3,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72299999999999998,"5min":0.81000000000000005,"15min":1.03},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58199999999999996,"15min":0.59499999999999997},"min":{"1min":0.48499999999999999,"5min":0.36699999999999999,"15min":0.35199999999999998},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":1.026},"last":0.58699999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72999999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.441},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.315,"15min":0.26900000000000002},"max":{"1min":0.78900000000000003,"5min":0.86399999999999999,"15min":0.91400000000000003},"last":0.42099999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.76500000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.48399999999999999,"5min":0.36599999999999999,"15min":0.28799999999999998},"max":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.94199999999999995},"last":0.5}]},{"osd":7,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.51300000000000001,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.95899999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.61199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.83699999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:25:51.100 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:25:51.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.232+0000 7f066215c700 1 Processor -- start 2023-08-27T11:25:51.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.232+0000 7f066215c700 1 -- start start 2023-08-27T11:25:51.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.233+0000 7f066215c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f065c158c90 0x7f065c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.233+0000 7f066215c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065c159070 0x7f065c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.233+0000 7f066215c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065c15f5d0 con 0x7f065c0b1270 2023-08-27T11:25:51.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.233+0000 7f066215c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065c0ca990 con 0x7f065c158c90 2023-08-27T11:25:51.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.233+0000 7f066215c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c06e870 con 0x7f065c159070 2023-08-27T11:25:51.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.233+0000 7f065bfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f065c0b1270 0x7f065c15b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54504/0 (socket says 172.21.15.112:54504) 2023-08-27T11:25:51.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.233+0000 7f065bfff700 1 -- 172.21.15.112:0/501173906 learned_addr learned my addr 172.21.15.112:0/501173906 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:51.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065affd700 1 --2- 172.21.15.112:0/501173906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065c159070 0x7f065c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065b7fe700 1 --2- 172.21.15.112:0/501173906 >> v2:172.21.15.138:3300/0 conn(0x7f065c158c90 0x7f065c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942040679 0 0) 0x7f065c15f5d0 con 0x7f065c0b1270 2023-08-27T11:25:51.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06480035b0 con 0x7f065c0b1270 2023-08-27T11:25:51.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1229330124 0 0) 0x7f06480035b0 con 0x7f065c0b1270 2023-08-27T11:25:51.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065c159070 msgr2=0x7f065c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:51.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 --2- 172.21.15.112:0/501173906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065c159070 0x7f065c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 >> v2:172.21.15.138:3300/0 conn(0x7f065c158c90 msgr2=0x7f065c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:51.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 --2- 172.21.15.112:0/501173906 >> v2:172.21.15.138:3300/0 conn(0x7f065c158c90 0x7f065c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c0b1a10 con 0x7f065c0b1270 2023-08-27T11:25:51.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f06540019f0 con 0x7f065c0b1270 2023-08-27T11:25:51.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 >> v1:172.21.15.112:6789/0 conn(0x7f065c0b1270 legacy=0x7f065c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:51.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 --2- 172.21.15.112:0/501173906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0648003a00 0x7f0648003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 --2- 172.21.15.112:0/501173906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065c159070 0x7f0648004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 --2- 172.21.15.112:0/501173906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0648004d90 0x7f0648005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0ca990 con 0x7f0648004d90 2023-08-27T11:25:51.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c06e870 con 0x7f0648003a00 2023-08-27T11:25:51.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064800ba10 con 0x7f065c159070 2023-08-27T11:25:51.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065bfff700 1 --2- 172.21.15.112:0/501173906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0648004d90 0x7f0648005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065b7fe700 1 --2- 172.21.15.112:0/501173906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0648003a00 0x7f0648003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065affd700 1 --2- 172.21.15.112:0/501173906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065c159070 0x7f0648004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065bfff700 1 -- 172.21.15.112:0/501173906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065c159070 msgr2=0x7f0648004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:51.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065bfff700 1 --2- 172.21.15.112:0/501173906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065c159070 0x7f0648004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065bfff700 1 -- 172.21.15.112:0/501173906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0648003a00 msgr2=0x7f0648003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:51.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065bfff700 1 --2- 172.21.15.112:0/501173906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0648003a00 0x7f0648003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.234+0000 7f065bfff700 1 -- 172.21.15.112:0/501173906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06480035b0 con 0x7f0648004d90 2023-08-27T11:25:51.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.235+0000 7f065bfff700 1 --2- 172.21.15.112:0/501173906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0648004d90 0x7f0648005210 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f065c0d1100 tx=0x7f0648005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e582e7f2d48be127 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:51.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.235+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065400a320 con 0x7f0648004d90 2023-08-27T11:25:51.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.235+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0654012600 con 0x7f0648004d90 2023-08-27T11:25:51.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.235+0000 7f065a7fc700 1 -- 172.21.15.112:0/501173906 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06540129b0 con 0x7f0648004d90 2023-08-27T11:25:51.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.235+0000 7f066215c700 1 -- 172.21.15.112:0/501173906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0648004d90 msgr2=0x7f0648005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:51.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.235+0000 7f066215c700 1 --2- 172.21.15.112:0/501173906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0648004d90 0x7f0648005210 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f065c0d1100 tx=0x7f0648005880 comp rx=0 tx=0).stop 2023-08-27T11:25:51.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.235+0000 7f065bfff700 1 -- 172.21.15.112:0/501173906 reap_dead start 2023-08-27T11:25:51.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.235+0000 7f066215c700 1 -- 172.21.15.112:0/501173906 shutdown_connections 2023-08-27T11:25:51.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.235+0000 7f066215c700 1 -- 172.21.15.112:0/501173906 >> 172.21.15.112:0/501173906 conn(0x7f065c0d0000 msgr2=0x7f065c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:51.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.235+0000 7f066215c700 1 -- 172.21.15.112:0/501173906 shutdown_connections 2023-08-27T11:25:51.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.235+0000 7f066215c700 1 -- 172.21.15.112:0/501173906 wait complete. 2023-08-27T11:25:51.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.237+0000 7f066215c700 1 Processor -- start 2023-08-27T11:25:51.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.237+0000 7f066215c700 1 -- start start 2023-08-27T11:25:51.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.237+0000 7f066215c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0648004d90 0x7f065c1f0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.237+0000 7f066215c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065c15a600 0x7f065c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.237+0000 7f066215c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f065c0c8dc0 0x7f065c0c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.237+0000 7f066215c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c15afc0 con 0x7f0648004d90 2023-08-27T11:25:51.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.237+0000 7f066215c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c141d40 con 0x7f065c0c8dc0 2023-08-27T11:25:51.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.237+0000 7f066215c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0c9780 con 0x7f065c15a600 2023-08-27T11:25:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f065b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0648004d90 0x7f065c1f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f065b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0648004d90 0x7f065c1f0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60130/0 (socket says 172.21.15.112:60130) 2023-08-27T11:25:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f065b7fe700 1 -- 172.21.15.112:0/4119787174 learned_addr learned my addr 172.21.15.112:0/4119787174 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f065affd700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065c15a600 0x7f065c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f065bfff700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f065c0c8dc0 0x7f065c0c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f065b7fe700 1 -- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065c15a600 msgr2=0x7f065c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:51.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f065b7fe700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065c15a600 0x7f065c15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f065b7fe700 1 -- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f065c0c8dc0 msgr2=0x7f065c0c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:51.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f065b7fe700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f065c0c8dc0 0x7f065c0c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f065b7fe700 1 -- 172.21.15.112:0/4119787174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c141fc0 con 0x7f0648004d90 2023-08-27T11:25:51.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f065b7fe700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0648004d90 0x7f065c1f0390 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0644011aa0 tx=0x7f0644011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:51.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f0653fff700 1 -- 172.21.15.112:0/4119787174 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644011ed0 con 0x7f0648004d90 2023-08-27T11:25:51.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f0653fff700 1 -- 172.21.15.112:0/4119787174 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064400f940 con 0x7f0648004d90 2023-08-27T11:25:51.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f066215c700 1 -- 172.21.15.112:0/4119787174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065c1422a0 con 0x7f0648004d90 2023-08-27T11:25:51.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f0653fff700 1 -- 172.21.15.112:0/4119787174 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064400fd10 con 0x7f0648004d90 2023-08-27T11:25:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.238+0000 7f066215c700 1 -- 172.21.15.112:0/4119787174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065c142860 con 0x7f0648004d90 2023-08-27T11:25:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.239+0000 7f066215c700 1 -- 172.21.15.112:0/4119787174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065c142bd0 con 0x7f0648004d90 2023-08-27T11:25:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.240+0000 7f0653fff700 1 -- 172.21.15.112:0/4119787174 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f0644014440 con 0x7f0648004d90 2023-08-27T11:25:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.240+0000 7f0653fff700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063c06a320 0x7f063c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.240+0000 7f0653fff700 1 -- 172.21.15.112:0/4119787174 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0644089040 con 0x7f0648004d90 2023-08-27T11:25:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.240+0000 7f065affd700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063c06a320 0x7f063c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.241+0000 7f065affd700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063c06a320 0x7f063c06c7e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f065c15dc60 tx=0x7f064c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.244+0000 7f0653fff700 1 -- 172.21.15.112:0/4119787174 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0644057f20 con 0x7f0648004d90 2023-08-27T11:25:51.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.430+0000 7f066215c700 1 -- 172.21.15.112:0/4119787174 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f065c15ef80 con 0x7f063c06a320 2023-08-27T11:25:51.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.440+0000 7f0653fff700 1 -- 172.21.15.112:0/4119787174 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311908 (secure 0 0 0) 0x7f064c019070 con 0x7f063c06a320 2023-08-27T11:25:51.445 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:51.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.443+0000 7f066215c700 1 -- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063c06a320 msgr2=0x7f063c06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:51.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.443+0000 7f066215c700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063c06a320 0x7f063c06c7e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f065c15dc60 tx=0x7f064c009210 comp rx=0 tx=0).stop 2023-08-27T11:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.443+0000 7f066215c700 1 -- 172.21.15.112:0/4119787174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0648004d90 msgr2=0x7f065c1f0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.443+0000 7f066215c700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0648004d90 0x7f065c1f0390 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0644011aa0 tx=0x7f0644011db0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.443+0000 7f066215c700 1 -- 172.21.15.112:0/4119787174 shutdown_connections 2023-08-27T11:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.443+0000 7f066215c700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f063c06a320 0x7f063c06c7e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.443+0000 7f066215c700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f065c0c8dc0 0x7f065c0c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.443+0000 7f066215c700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0648004d90 0x7f065c1f0390 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.443+0000 7f066215c700 1 --2- 172.21.15.112:0/4119787174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f065c15a600 0x7f065c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.443+0000 7f066215c700 1 -- 172.21.15.112:0/4119787174 >> 172.21.15.112:0/4119787174 conn(0x7f065c0d0000 msgr2=0x7f065c071820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.443+0000 7f066215c700 1 -- 172.21.15.112:0/4119787174 shutdown_connections 2023-08-27T11:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.443+0000 7f066215c700 1 -- 172.21.15.112:0/4119787174 wait complete. 2023-08-27T11:25:51.450 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:25:51.457 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2023-08-27T11:25:50.361003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":198,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258084,"kb_used_data":8612,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754716636,"statfs":{"total":773094113280,"available":772829835264,"internally_reserved":0,"allocated":8818688,"data_stored":4641499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002227"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652295+0000","last_change":"2023-08-27T11:25:42.420266+0000","last_active":"2023-08-27T11:25:43.652295+0000","last_peered":"2023-08-27T11:25:43.652295+0000","last_clean":"2023-08-27T11:25:43.652295+0000","last_became_active":"2023-08-27T11:25:42.420004+0000","last_became_peered":"2023-08-27T11:25:42.420004+0000","last_unstale":"2023-08-27T11:25:43.652295+0000","last_undegraded":"2023-08-27T11:25:43.652295+0000","last_fullsized":"2023-08-27T11:25:43.652295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:03:23.151422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652067+0000","last_change":"2023-08-27T11:25:42.439094+0000","last_active":"2023-08-27T11:25:43.652067+0000","last_peered":"2023-08-27T11:25:43.652067+0000","last_clean":"2023-08-27T11:25:43.652067+0000","last_became_active":"2023-08-27T11:25:42.438773+0000","last_became_peered":"2023-08-27T11:25:42.438773+0000","last_unstale":"2023-08-27T11:25:43.652067+0000","last_undegraded":"2023-08-27T11:25:43.652067+0000","last_fullsized":"2023-08-27T11:25:43.652067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:35:20.991741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651378+0000","last_change":"2023-08-27T11:25:42.427019+0000","last_active":"2023-08-27T11:25:43.651378+0000","last_peered":"2023-08-27T11:25:43.651378+0000","last_clean":"2023-08-27T11:25:43.651378+0000","last_became_active":"2023-08-27T11:25:42.426754+0000","last_became_peered":"2023-08-27T11:25:42.426754+0000","last_unstale":"2023-08-27T11:25:43.651378+0000","last_undegraded":"2023-08-27T11:25:43.651378+0000","last_fullsized":"2023-08-27T11:25:43.651378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:18.357004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412762+0000","last_change":"2023-08-27T11:25:42.429468+0000","last_active":"2023-08-27T11:25:43.412762+0000","last_peered":"2023-08-27T11:25:43.412762+0000","last_clean":"2023-08-27T11:25:43.412762+0000","last_became_active":"2023-08-27T11:25:42.429224+0000","last_became_peered":"2023-08-27T11:25:42.429224+0000","last_unstale":"2023-08-27T11:25:43.412762+0000","last_undegraded":"2023-08-27T11:25:43.412762+0000","last_fullsized":"2023-08-27T11:25:43.412762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:25:56.457183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414546+0000","last_change":"2023-08-27T11:25:42.435079+0000","last_active":"2023-08-27T11:25:43.414546+0000","last_peered":"2023-08-27T11:25:43.414546+0000","last_clean":"2023-08-27T11:25:43.414546+0000","last_became_active":"2023-08-27T11:25:42.434812+0000","last_became_peered":"2023-08-27T11:25:42.434812+0000","last_unstale":"2023-08-27T11:25:43.414546+0000","last_undegraded":"2023-08-27T11:25:43.414546+0000","last_fullsized":"2023-08-27T11:25:43.414546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:31.067105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651191+0000","last_change":"2023-08-27T11:25:42.422645+0000","last_active":"2023-08-27T11:25:43.651191+0000","last_peered":"2023-08-27T11:25:43.651191+0000","last_clean":"2023-08-27T11:25:43.651191+0000","last_became_active":"2023-08-27T11:25:42.422042+0000","last_became_peered":"2023-08-27T11:25:42.422042+0000","last_unstale":"2023-08-27T11:25:43.651191+0000","last_undegraded":"2023-08-27T11:25:43.651191+0000","last_fullsized":"2023-08-27T11:25:43.651191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:56.061378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651564+0000","last_change":"2023-08-27T11:25:42.437768+0000","last_active":"2023-08-27T11:25:43.651564+0000","last_peered":"2023-08-27T11:25:43.651564+0000","last_clean":"2023-08-27T11:25:43.651564+0000","last_became_active":"2023-08-27T11:25:42.437467+0000","last_became_peered":"2023-08-27T11:25:42.437467+0000","last_unstale":"2023-08-27T11:25:43.651564+0000","last_undegraded":"2023-08-27T11:25:43.651564+0000","last_fullsized":"2023-08-27T11:25:43.651564+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:32:29.485178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412347+0000","last_change":"2023-08-27T11:25:42.435249+0000","last_active":"2023-08-27T11:25:43.412347+0000","last_peered":"2023-08-27T11:25:43.412347+0000","last_clean":"2023-08-27T11:25:43.412347+0000","last_became_active":"2023-08-27T11:25:42.434977+0000","last_became_peered":"2023-08-27T11:25:42.434977+0000","last_unstale":"2023-08-27T11:25:43.412347+0000","last_undegraded":"2023-08-27T11:25:43.412347+0000","last_fullsized":"2023-08-27T11:25:43.412347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:28:59.149115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414551+0000","last_change":"2023-08-27T11:25:42.436603+0000","last_active":"2023-08-27T11:25:43.414551+0000","last_peered":"2023-08-27T11:25:43.414551+0000","last_clean":"2023-08-27T11:25:43.414551+0000","last_became_active":"2023-08-27T11:25:42.436302+0000","last_became_peered":"2023-08-27T11:25:42.436302+0000","last_unstale":"2023-08-27T11:25:43.414551+0000","last_undegraded":"2023-08-27T11:25:43.414551+0000","last_fullsized":"2023-08-27T11:25:43.414551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:55:40.769009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652249+0000","last_change":"2023-08-27T11:25:42.425085+0000","last_active":"2023-08-27T11:25:43.652249+0000","last_peered":"2023-08-27T11:25:43.652249+0000","last_clean":"2023-08-27T11:25:43.652249+0000","last_became_active":"2023-08-27T11:25:42.424883+0000","last_became_peered":"2023-08-27T11:25:42.424883+0000","last_unstale":"2023-08-27T11:25:43.652249+0000","last_undegraded":"2023-08-27T11:25:43.652249+0000","last_fullsized":"2023-08-27T11:25:43.652249+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:02:43.528771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414836+0000","last_change":"2023-08-27T11:25:42.437615+0000","last_active":"2023-08-27T11:25:43.414836+0000","last_peered":"2023-08-27T11:25:43.414836+0000","last_clean":"2023-08-27T11:25:43.414836+0000","last_became_active":"2023-08-27T11:25:42.437077+0000","last_became_peered":"2023-08-27T11:25:42.437077+0000","last_unstale":"2023-08-27T11:25:43.414836+0000","last_undegraded":"2023-08-27T11:25:43.414836+0000","last_fullsized":"2023-08-27T11:25:43.414836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:33.316913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652000+0000","last_change":"2023-08-27T11:25:42.430295+0000","last_active":"2023-08-27T11:25:43.652000+0000","last_peered":"2023-08-27T11:25:43.652000+0000","last_clean":"2023-08-27T11:25:43.652000+0000","last_became_active":"2023-08-27T11:25:42.430026+0000","last_became_peered":"2023-08-27T11:25:42.430026+0000","last_unstale":"2023-08-27T11:25:43.652000+0000","last_undegraded":"2023-08-27T11:25:43.652000+0000","last_fullsized":"2023-08-27T11:25:43.652000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:44:04.704190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412526+0000","last_change":"2023-08-27T11:25:42.434907+0000","last_active":"2023-08-27T11:25:43.412526+0000","last_peered":"2023-08-27T11:25:43.412526+0000","last_clean":"2023-08-27T11:25:43.412526+0000","last_became_active":"2023-08-27T11:25:42.434649+0000","last_became_peered":"2023-08-27T11:25:42.434649+0000","last_unstale":"2023-08-27T11:25:43.412526+0000","last_undegraded":"2023-08-27T11:25:43.412526+0000","last_fullsized":"2023-08-27T11:25:43.412526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:04.687388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413182+0000","last_change":"2023-08-27T11:25:42.433344+0000","last_active":"2023-08-27T11:25:43.413182+0000","last_peered":"2023-08-27T11:25:43.413182+0000","last_clean":"2023-08-27T11:25:43.413182+0000","last_became_active":"2023-08-27T11:25:42.433033+0000","last_became_peered":"2023-08-27T11:25:42.433033+0000","last_unstale":"2023-08-27T11:25:43.413182+0000","last_undegraded":"2023-08-27T11:25:43.413182+0000","last_fullsized":"2023-08-27T11:25:43.413182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:12:40.439919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412220+0000","last_change":"2023-08-27T11:25:42.418285+0000","last_active":"2023-08-27T11:25:43.412220+0000","last_peered":"2023-08-27T11:25:43.412220+0000","last_clean":"2023-08-27T11:25:43.412220+0000","last_became_active":"2023-08-27T11:25:42.417901+0000","last_became_peered":"2023-08-27T11:25:42.417901+0000","last_unstale":"2023-08-27T11:25:43.412220+0000","last_undegraded":"2023-08-27T11:25:43.412220+0000","last_fullsized":"2023-08-27T11:25:43.412220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:05:02.418791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651112+0000","last_change":"2023-08-27T11:25:42.424663+0000","last_active":"2023-08-27T11:25:43.651112+0000","last_peered":"2023-08-27T11:25:43.651112+0000","last_clean":"2023-08-27T11:25:43.651112+0000","last_became_active":"2023-08-27T11:25:42.424342+0000","last_became_peered":"2023-08-27T11:25:42.424342+0000","last_unstale":"2023-08-27T11:25:43.651112+0000","last_undegraded":"2023-08-27T11:25:43.651112+0000","last_fullsized":"2023-08-27T11:25:43.651112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:34:42.236180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652412+0000","last_change":"2023-08-27T11:25:42.421588+0000","last_active":"2023-08-27T11:25:43.652412+0000","last_peered":"2023-08-27T11:25:43.652412+0000","last_clean":"2023-08-27T11:25:43.652412+0000","last_became_active":"2023-08-27T11:25:42.421270+0000","last_became_peered":"2023-08-27T11:25:42.421270+0000","last_unstale":"2023-08-27T11:25:43.652412+0000","last_undegraded":"2023-08-27T11:25:43.652412+0000","last_fullsized":"2023-08-27T11:25:43.652412+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:09:29.320316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651505+0000","last_change":"2023-08-27T11:25:42.433905+0000","last_active":"2023-08-27T11:25:43.651505+0000","last_peered":"2023-08-27T11:25:43.651505+0000","last_clean":"2023-08-27T11:25:43.651505+0000","last_became_active":"2023-08-27T11:25:42.433732+0000","last_became_peered":"2023-08-27T11:25:42.433732+0000","last_unstale":"2023-08-27T11:25:43.651505+0000","last_undegraded":"2023-08-27T11:25:43.651505+0000","last_fullsized":"2023-08-27T11:25:43.651505+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:25.010381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414722+0000","last_change":"2023-08-27T11:25:42.429718+0000","last_active":"2023-08-27T11:25:43.414722+0000","last_peered":"2023-08-27T11:25:43.414722+0000","last_clean":"2023-08-27T11:25:43.414722+0000","last_became_active":"2023-08-27T11:25:42.429416+0000","last_became_peered":"2023-08-27T11:25:42.429416+0000","last_unstale":"2023-08-27T11:25:43.414722+0000","last_undegraded":"2023-08-27T11:25:43.414722+0000","last_fullsized":"2023-08-27T11:25:43.414722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:18.357004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412638+0000","last_change":"2023-08-27T11:25:42.422139+0000","last_active":"2023-08-27T11:25:43.412638+0000","last_peered":"2023-08-27T11:25:43.412638+0000","last_clean":"2023-08-27T11:25:43.412638+0000","last_became_active":"2023-08-27T11:25:42.421885+0000","last_became_peered":"2023-08-27T11:25:42.421885+0000","last_unstale":"2023-08-27T11:25:43.412638+0000","last_undegraded":"2023-08-27T11:25:43.412638+0000","last_fullsized":"2023-08-27T11:25:43.412638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:19.212468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414654+0000","last_change":"2023-08-27T11:25:42.431105+0000","last_active":"2023-08-27T11:25:43.414654+0000","last_peered":"2023-08-27T11:25:43.414654+0000","last_clean":"2023-08-27T11:25:43.414654+0000","last_became_active":"2023-08-27T11:25:42.430833+0000","last_became_peered":"2023-08-27T11:25:42.430833+0000","last_unstale":"2023-08-27T11:25:43.414654+0000","last_undegraded":"2023-08-27T11:25:43.414654+0000","last_fullsized":"2023-08-27T11:25:43.414654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:18:54.288640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414229+0000","last_change":"2023-08-27T11:25:42.427498+0000","last_active":"2023-08-27T11:25:43.414229+0000","last_peered":"2023-08-27T11:25:43.414229+0000","last_clean":"2023-08-27T11:25:43.414229+0000","last_became_active":"2023-08-27T11:25:42.427193+0000","last_became_peered":"2023-08-27T11:25:42.427193+0000","last_unstale":"2023-08-27T11:25:43.414229+0000","last_undegraded":"2023-08-27T11:25:43.414229+0000","last_fullsized":"2023-08-27T11:25:43.414229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:33.774360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413656+0000","last_change":"2023-08-27T11:25:42.422531+0000","last_active":"2023-08-27T11:25:43.413656+0000","last_peered":"2023-08-27T11:25:43.413656+0000","last_clean":"2023-08-27T11:25:43.413656+0000","last_became_active":"2023-08-27T11:25:42.422231+0000","last_became_peered":"2023-08-27T11:25:42.422231+0000","last_unstale":"2023-08-27T11:25:43.413656+0000","last_undegraded":"2023-08-27T11:25:43.413656+0000","last_fullsized":"2023-08-27T11:25:43.413656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:31.115102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651646+0000","last_change":"2023-08-27T11:25:42.423364+0000","last_active":"2023-08-27T11:25:43.651646+0000","last_peered":"2023-08-27T11:25:43.651646+0000","last_clean":"2023-08-27T11:25:43.651646+0000","last_became_active":"2023-08-27T11:25:42.423105+0000","last_became_peered":"2023-08-27T11:25:42.423105+0000","last_unstale":"2023-08-27T11:25:43.651646+0000","last_undegraded":"2023-08-27T11:25:43.651646+0000","last_fullsized":"2023-08-27T11:25:43.651646+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:33.316913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652052+0000","last_change":"2023-08-27T11:25:42.439263+0000","last_active":"2023-08-27T11:25:43.652052+0000","last_peered":"2023-08-27T11:25:43.652052+0000","last_clean":"2023-08-27T11:25:43.652052+0000","last_became_active":"2023-08-27T11:25:42.438720+0000","last_became_peered":"2023-08-27T11:25:42.438720+0000","last_unstale":"2023-08-27T11:25:43.652052+0000","last_undegraded":"2023-08-27T11:25:43.652052+0000","last_fullsized":"2023-08-27T11:25:43.652052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:08:25.805826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414674+0000","last_change":"2023-08-27T11:25:42.427526+0000","last_active":"2023-08-27T11:25:43.414674+0000","last_peered":"2023-08-27T11:25:43.414674+0000","last_clean":"2023-08-27T11:25:43.414674+0000","last_became_active":"2023-08-27T11:25:42.427212+0000","last_became_peered":"2023-08-27T11:25:42.427212+0000","last_unstale":"2023-08-27T11:25:43.414674+0000","last_undegraded":"2023-08-27T11:25:43.414674+0000","last_fullsized":"2023-08-27T11:25:43.414674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:19:27.592047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651475+0000","last_change":"2023-08-27T11:25:42.424075+0000","last_active":"2023-08-27T11:25:43.651475+0000","last_peered":"2023-08-27T11:25:43.651475+0000","last_clean":"2023-08-27T11:25:43.651475+0000","last_became_active":"2023-08-27T11:25:42.423798+0000","last_became_peered":"2023-08-27T11:25:42.423798+0000","last_unstale":"2023-08-27T11:25:43.651475+0000","last_undegraded":"2023-08-27T11:25:43.651475+0000","last_fullsized":"2023-08-27T11:25:43.651475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:28:45.094430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412620+0000","last_change":"2023-08-27T11:25:42.427486+0000","last_active":"2023-08-27T11:25:43.412620+0000","last_peered":"2023-08-27T11:25:43.412620+0000","last_clean":"2023-08-27T11:25:43.412620+0000","last_became_active":"2023-08-27T11:25:42.427243+0000","last_became_peered":"2023-08-27T11:25:42.427243+0000","last_unstale":"2023-08-27T11:25:43.412620+0000","last_undegraded":"2023-08-27T11:25:43.412620+0000","last_fullsized":"2023-08-27T11:25:43.412620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:29:58.073297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651830+0000","last_change":"2023-08-27T11:25:42.435078+0000","last_active":"2023-08-27T11:25:43.651830+0000","last_peered":"2023-08-27T11:25:43.651830+0000","last_clean":"2023-08-27T11:25:43.651830+0000","last_became_active":"2023-08-27T11:25:42.434793+0000","last_became_peered":"2023-08-27T11:25:42.434793+0000","last_unstale":"2023-08-27T11:25:43.651830+0000","last_undegraded":"2023-08-27T11:25:43.651830+0000","last_fullsized":"2023-08-27T11:25:43.651830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:37:04.205444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414271+0000","last_change":"2023-08-27T11:25:42.426898+0000","last_active":"2023-08-27T11:25:43.414271+0000","last_peered":"2023-08-27T11:25:43.414271+0000","last_clean":"2023-08-27T11:25:43.414271+0000","last_became_active":"2023-08-27T11:25:42.426550+0000","last_became_peered":"2023-08-27T11:25:42.426550+0000","last_unstale":"2023-08-27T11:25:43.414271+0000","last_undegraded":"2023-08-27T11:25:43.414271+0000","last_fullsized":"2023-08-27T11:25:43.414271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:00:40.536833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651488+0000","last_change":"2023-08-27T11:25:42.428905+0000","last_active":"2023-08-27T11:25:43.651488+0000","last_peered":"2023-08-27T11:25:43.651488+0000","last_clean":"2023-08-27T11:25:43.651488+0000","last_became_active":"2023-08-27T11:25:42.428568+0000","last_became_peered":"2023-08-27T11:25:42.428568+0000","last_unstale":"2023-08-27T11:25:43.651488+0000","last_undegraded":"2023-08-27T11:25:43.651488+0000","last_fullsized":"2023-08-27T11:25:43.651488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:44:29.713492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.650833+0000","last_change":"2023-08-27T11:25:42.418975+0000","last_active":"2023-08-27T11:25:43.650833+0000","last_peered":"2023-08-27T11:25:43.650833+0000","last_clean":"2023-08-27T11:25:43.650833+0000","last_became_active":"2023-08-27T11:25:42.418715+0000","last_became_peered":"2023-08-27T11:25:42.418715+0000","last_unstale":"2023-08-27T11:25:43.650833+0000","last_undegraded":"2023-08-27T11:25:43.650833+0000","last_fullsized":"2023-08-27T11:25:43.650833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:40:21.119531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651249+0000","last_change":"2023-08-27T11:25:42.431264+0000","last_active":"2023-08-27T11:25:43.651249+0000","last_peered":"2023-08-27T11:25:43.651249+0000","last_clean":"2023-08-27T11:25:43.651249+0000","last_became_active":"2023-08-27T11:25:42.430681+0000","last_became_peered":"2023-08-27T11:25:42.430681+0000","last_unstale":"2023-08-27T11:25:43.651249+0000","last_undegraded":"2023-08-27T11:25:43.651249+0000","last_fullsized":"2023-08-27T11:25:43.651249+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:02:13.385283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"76'4","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413659+0000","last_change":"2023-08-27T11:25:41.414884+0000","last_active":"2023-08-27T11:25:43.413659+0000","last_peered":"2023-08-27T11:25:43.413659+0000","last_clean":"2023-08-27T11:25:43.413659+0000","last_became_active":"2023-08-27T11:25:41.414549+0000","last_became_peered":"2023-08-27T11:25:41.414549+0000","last_unstale":"2023-08-27T11:25:43.413659+0000","last_undegraded":"2023-08-27T11:25:43.413659+0000","last_fullsized":"2023-08-27T11:25:43.413659+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:30.061949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651823+0000","last_change":"2023-08-27T11:25:42.425794+0000","last_active":"2023-08-27T11:25:43.651823+0000","last_peered":"2023-08-27T11:25:43.651823+0000","last_clean":"2023-08-27T11:25:43.651823+0000","last_became_active":"2023-08-27T11:25:42.425471+0000","last_became_peered":"2023-08-27T11:25:42.425471+0000","last_unstale":"2023-08-27T11:25:43.651823+0000","last_undegraded":"2023-08-27T11:25:43.651823+0000","last_fullsized":"2023-08-27T11:25:43.651823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:40:18.154178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"76'1","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414684+0000","last_change":"2023-08-27T11:25:41.414847+0000","last_active":"2023-08-27T11:25:43.414684+0000","last_peered":"2023-08-27T11:25:43.414684+0000","last_clean":"2023-08-27T11:25:43.414684+0000","last_became_active":"2023-08-27T11:25:41.414532+0000","last_became_peered":"2023-08-27T11:25:41.414532+0000","last_unstale":"2023-08-27T11:25:43.414684+0000","last_undegraded":"2023-08-27T11:25:43.414684+0000","last_fullsized":"2023-08-27T11:25:43.414684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:59.531534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651993+0000","last_change":"2023-08-27T11:25:42.435164+0000","last_active":"2023-08-27T11:25:43.651993+0000","last_peered":"2023-08-27T11:25:43.651993+0000","last_clean":"2023-08-27T11:25:43.651993+0000","last_became_active":"2023-08-27T11:25:42.434880+0000","last_became_peered":"2023-08-27T11:25:42.434880+0000","last_unstale":"2023-08-27T11:25:43.651993+0000","last_undegraded":"2023-08-27T11:25:43.651993+0000","last_fullsized":"2023-08-27T11:25:43.651993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:54:13.363165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"76'2","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652026+0000","last_change":"2023-08-27T11:25:41.412179+0000","last_active":"2023-08-27T11:25:43.652026+0000","last_peered":"2023-08-27T11:25:43.652026+0000","last_clean":"2023-08-27T11:25:43.652026+0000","last_became_active":"2023-08-27T11:25:41.411940+0000","last_became_peered":"2023-08-27T11:25:41.411940+0000","last_unstale":"2023-08-27T11:25:43.652026+0000","last_undegraded":"2023-08-27T11:25:43.652026+0000","last_fullsized":"2023-08-27T11:25:43.652026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:14:57.684645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652483+0000","last_change":"2023-08-27T11:25:42.420074+0000","last_active":"2023-08-27T11:25:43.652483+0000","last_peered":"2023-08-27T11:25:43.652483+0000","last_clean":"2023-08-27T11:25:43.652483+0000","last_became_active":"2023-08-27T11:25:42.419814+0000","last_became_peered":"2023-08-27T11:25:42.419814+0000","last_unstale":"2023-08-27T11:25:43.652483+0000","last_undegraded":"2023-08-27T11:25:43.652483+0000","last_fullsized":"2023-08-27T11:25:43.652483+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:34:45.098011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"76'2","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413889+0000","last_change":"2023-08-27T11:25:41.414002+0000","last_active":"2023-08-27T11:25:43.413889+0000","last_peered":"2023-08-27T11:25:43.413889+0000","last_clean":"2023-08-27T11:25:43.413889+0000","last_became_active":"2023-08-27T11:25:41.413740+0000","last_became_peered":"2023-08-27T11:25:41.413740+0000","last_unstale":"2023-08-27T11:25:43.413889+0000","last_undegraded":"2023-08-27T11:25:43.413889+0000","last_fullsized":"2023-08-27T11:25:43.413889+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:19:26.586891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412239+0000","last_change":"2023-08-27T11:25:42.422364+0000","last_active":"2023-08-27T11:25:43.412239+0000","last_peered":"2023-08-27T11:25:43.412239+0000","last_clean":"2023-08-27T11:25:43.412239+0000","last_became_active":"2023-08-27T11:25:42.422118+0000","last_became_peered":"2023-08-27T11:25:42.422118+0000","last_unstale":"2023-08-27T11:25:43.412239+0000","last_undegraded":"2023-08-27T11:25:43.412239+0000","last_fullsized":"2023-08-27T11:25:43.412239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:20:38.826741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"76'1","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414805+0000","last_change":"2023-08-27T11:25:41.405587+0000","last_active":"2023-08-27T11:25:43.414805+0000","last_peered":"2023-08-27T11:25:43.414805+0000","last_clean":"2023-08-27T11:25:43.414805+0000","last_became_active":"2023-08-27T11:25:41.405255+0000","last_became_peered":"2023-08-27T11:25:41.405255+0000","last_unstale":"2023-08-27T11:25:43.414805+0000","last_undegraded":"2023-08-27T11:25:43.414805+0000","last_fullsized":"2023-08-27T11:25:43.414805+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:24:59.711845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413801+0000","last_change":"2023-08-27T11:25:42.420562+0000","last_active":"2023-08-27T11:25:43.413801+0000","last_peered":"2023-08-27T11:25:43.413801+0000","last_clean":"2023-08-27T11:25:43.413801+0000","last_became_active":"2023-08-27T11:25:42.420220+0000","last_became_peered":"2023-08-27T11:25:42.420220+0000","last_unstale":"2023-08-27T11:25:43.413801+0000","last_undegraded":"2023-08-27T11:25:43.413801+0000","last_fullsized":"2023-08-27T11:25:43.413801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:18:54.288640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"76'1","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412421+0000","last_change":"2023-08-27T11:25:41.406787+0000","last_active":"2023-08-27T11:25:43.412421+0000","last_peered":"2023-08-27T11:25:43.412421+0000","last_clean":"2023-08-27T11:25:43.412421+0000","last_became_active":"2023-08-27T11:25:41.406253+0000","last_became_peered":"2023-08-27T11:25:41.406253+0000","last_unstale":"2023-08-27T11:25:43.412421+0000","last_undegraded":"2023-08-27T11:25:43.412421+0000","last_fullsized":"2023-08-27T11:25:43.412421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:09.352101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651049+0000","last_change":"2023-08-27T11:25:42.422808+0000","last_active":"2023-08-27T11:25:43.651049+0000","last_peered":"2023-08-27T11:25:43.651049+0000","last_clean":"2023-08-27T11:25:43.651049+0000","last_became_active":"2023-08-27T11:25:42.422401+0000","last_became_peered":"2023-08-27T11:25:42.422401+0000","last_unstale":"2023-08-27T11:25:43.651049+0000","last_undegraded":"2023-08-27T11:25:43.651049+0000","last_fullsized":"2023-08-27T11:25:43.651049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:56:58.624053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651672+0000","last_change":"2023-08-27T11:25:41.404377+0000","last_active":"2023-08-27T11:25:43.651672+0000","last_peered":"2023-08-27T11:25:43.651672+0000","last_clean":"2023-08-27T11:25:43.651672+0000","last_became_active":"2023-08-27T11:25:41.404056+0000","last_became_peered":"2023-08-27T11:25:41.404056+0000","last_unstale":"2023-08-27T11:25:43.651672+0000","last_undegraded":"2023-08-27T11:25:43.651672+0000","last_fullsized":"2023-08-27T11:25:43.651672+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:21:45.842462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412357+0000","last_change":"2023-08-27T11:25:42.420397+0000","last_active":"2023-08-27T11:25:43.412357+0000","last_peered":"2023-08-27T11:25:43.412357+0000","last_clean":"2023-08-27T11:25:43.412357+0000","last_became_active":"2023-08-27T11:25:42.419758+0000","last_became_peered":"2023-08-27T11:25:42.419758+0000","last_unstale":"2023-08-27T11:25:43.412357+0000","last_undegraded":"2023-08-27T11:25:43.412357+0000","last_fullsized":"2023-08-27T11:25:43.412357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:32:44.872652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651450+0000","last_change":"2023-08-27T11:25:41.407540+0000","last_active":"2023-08-27T11:25:43.651450+0000","last_peered":"2023-08-27T11:25:43.651450+0000","last_clean":"2023-08-27T11:25:43.651450+0000","last_became_active":"2023-08-27T11:25:41.406794+0000","last_became_peered":"2023-08-27T11:25:41.406794+0000","last_unstale":"2023-08-27T11:25:43.651450+0000","last_undegraded":"2023-08-27T11:25:43.651450+0000","last_fullsized":"2023-08-27T11:25:43.651450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:03.342451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414507+0000","last_change":"2023-08-27T11:25:42.437085+0000","last_active":"2023-08-27T11:25:43.414507+0000","last_peered":"2023-08-27T11:25:43.414507+0000","last_clean":"2023-08-27T11:25:43.414507+0000","last_became_active":"2023-08-27T11:25:42.436805+0000","last_became_peered":"2023-08-27T11:25:42.436805+0000","last_unstale":"2023-08-27T11:25:43.414507+0000","last_undegraded":"2023-08-27T11:25:43.414507+0000","last_fullsized":"2023-08-27T11:25:43.414507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:31.115102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652234+0000","last_change":"2023-08-27T11:25:41.417604+0000","last_active":"2023-08-27T11:25:43.652234+0000","last_peered":"2023-08-27T11:25:43.652234+0000","last_clean":"2023-08-27T11:25:43.652234+0000","last_became_active":"2023-08-27T11:25:41.416727+0000","last_became_peered":"2023-08-27T11:25:41.416727+0000","last_unstale":"2023-08-27T11:25:43.652234+0000","last_undegraded":"2023-08-27T11:25:43.652234+0000","last_fullsized":"2023-08-27T11:25:43.652234+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:55:23.851131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651791+0000","last_change":"2023-08-27T11:25:42.439516+0000","last_active":"2023-08-27T11:25:43.651791+0000","last_peered":"2023-08-27T11:25:43.651791+0000","last_clean":"2023-08-27T11:25:43.651791+0000","last_became_active":"2023-08-27T11:25:42.439192+0000","last_became_peered":"2023-08-27T11:25:42.439192+0000","last_unstale":"2023-08-27T11:25:43.651791+0000","last_undegraded":"2023-08-27T11:25:43.651791+0000","last_fullsized":"2023-08-27T11:25:43.651791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:15:28.719952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651787+0000","last_change":"2023-08-27T11:25:41.402868+0000","last_active":"2023-08-27T11:25:43.651787+0000","last_peered":"2023-08-27T11:25:43.651787+0000","last_clean":"2023-08-27T11:25:43.651787+0000","last_became_active":"2023-08-27T11:25:41.402585+0000","last_became_peered":"2023-08-27T11:25:41.402585+0000","last_unstale":"2023-08-27T11:25:43.651787+0000","last_undegraded":"2023-08-27T11:25:43.651787+0000","last_fullsized":"2023-08-27T11:25:43.651787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:00:22.620129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651614+0000","last_change":"2023-08-27T11:25:42.424398+0000","last_active":"2023-08-27T11:25:43.651614+0000","last_peered":"2023-08-27T11:25:43.651614+0000","last_clean":"2023-08-27T11:25:43.651614+0000","last_became_active":"2023-08-27T11:25:42.424159+0000","last_became_peered":"2023-08-27T11:25:42.424159+0000","last_unstale":"2023-08-27T11:25:43.651614+0000","last_undegraded":"2023-08-27T11:25:43.651614+0000","last_fullsized":"2023-08-27T11:25:43.651614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:45:01.126128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"76'3","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414815+0000","last_change":"2023-08-27T11:25:41.408134+0000","last_active":"2023-08-27T11:25:43.414815+0000","last_peered":"2023-08-27T11:25:43.414815+0000","last_clean":"2023-08-27T11:25:43.414815+0000","last_became_active":"2023-08-27T11:25:41.407806+0000","last_became_peered":"2023-08-27T11:25:41.407806+0000","last_unstale":"2023-08-27T11:25:43.414815+0000","last_undegraded":"2023-08-27T11:25:43.414815+0000","last_fullsized":"2023-08-27T11:25:43.414815+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:24.005225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412703+0000","last_change":"2023-08-27T11:25:42.427577+0000","last_active":"2023-08-27T11:25:43.412703+0000","last_peered":"2023-08-27T11:25:43.412703+0000","last_clean":"2023-08-27T11:25:43.412703+0000","last_became_active":"2023-08-27T11:25:42.426837+0000","last_became_peered":"2023-08-27T11:25:42.426837+0000","last_unstale":"2023-08-27T11:25:43.412703+0000","last_undegraded":"2023-08-27T11:25:43.412703+0000","last_fullsized":"2023-08-27T11:25:43.412703+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:12:29.177135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"76'6","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651373+0000","last_change":"2023-08-27T11:25:41.414641+0000","last_active":"2023-08-27T11:25:43.651373+0000","last_peered":"2023-08-27T11:25:43.651373+0000","last_clean":"2023-08-27T11:25:43.651373+0000","last_became_active":"2023-08-27T11:25:41.414361+0000","last_became_peered":"2023-08-27T11:25:41.414361+0000","last_unstale":"2023-08-27T11:25:43.651373+0000","last_undegraded":"2023-08-27T11:25:43.651373+0000","last_fullsized":"2023-08-27T11:25:43.651373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:09:15.705283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412098+0000","last_change":"2023-08-27T11:25:42.434704+0000","last_active":"2023-08-27T11:25:43.412098+0000","last_peered":"2023-08-27T11:25:43.412098+0000","last_clean":"2023-08-27T11:25:43.412098+0000","last_became_active":"2023-08-27T11:25:42.434416+0000","last_became_peered":"2023-08-27T11:25:42.434416+0000","last_unstale":"2023-08-27T11:25:43.412098+0000","last_undegraded":"2023-08-27T11:25:43.412098+0000","last_fullsized":"2023-08-27T11:25:43.412098+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:05:59.243349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"76'2","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651980+0000","last_change":"2023-08-27T11:25:41.410338+0000","last_active":"2023-08-27T11:25:43.651980+0000","last_peered":"2023-08-27T11:25:43.651980+0000","last_clean":"2023-08-27T11:25:43.651980+0000","last_became_active":"2023-08-27T11:25:41.410005+0000","last_became_peered":"2023-08-27T11:25:41.410005+0000","last_unstale":"2023-08-27T11:25:43.651980+0000","last_undegraded":"2023-08-27T11:25:43.651980+0000","last_fullsized":"2023-08-27T11:25:43.651980+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:59.490107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413100+0000","last_change":"2023-08-27T11:25:42.432689+0000","last_active":"2023-08-27T11:25:43.413100+0000","last_peered":"2023-08-27T11:25:43.413100+0000","last_clean":"2023-08-27T11:25:43.413100+0000","last_became_active":"2023-08-27T11:25:42.432403+0000","last_became_peered":"2023-08-27T11:25:42.432403+0000","last_unstale":"2023-08-27T11:25:43.413100+0000","last_undegraded":"2023-08-27T11:25:43.413100+0000","last_fullsized":"2023-08-27T11:25:43.413100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:16:52.605954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651307+0000","last_change":"2023-08-27T11:25:41.408888+0000","last_active":"2023-08-27T11:25:43.651307+0000","last_peered":"2023-08-27T11:25:43.651307+0000","last_clean":"2023-08-27T11:25:43.651307+0000","last_became_active":"2023-08-27T11:25:41.408609+0000","last_became_peered":"2023-08-27T11:25:41.408609+0000","last_unstale":"2023-08-27T11:25:43.651307+0000","last_undegraded":"2023-08-27T11:25:43.651307+0000","last_fullsized":"2023-08-27T11:25:43.651307+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:56:11.740062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651577+0000","last_change":"2023-08-27T11:25:42.424730+0000","last_active":"2023-08-27T11:25:43.651577+0000","last_peered":"2023-08-27T11:25:43.651577+0000","last_clean":"2023-08-27T11:25:43.651577+0000","last_became_active":"2023-08-27T11:25:42.424502+0000","last_became_peered":"2023-08-27T11:25:42.424502+0000","last_unstale":"2023-08-27T11:25:43.651577+0000","last_undegraded":"2023-08-27T11:25:43.651577+0000","last_fullsized":"2023-08-27T11:25:43.651577+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:43:31.016709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412227+0000","last_change":"2023-08-27T11:25:41.414552+0000","last_active":"2023-08-27T11:25:43.412227+0000","last_peered":"2023-08-27T11:25:43.412227+0000","last_clean":"2023-08-27T11:25:43.412227+0000","last_became_active":"2023-08-27T11:25:41.414281+0000","last_became_peered":"2023-08-27T11:25:41.414281+0000","last_unstale":"2023-08-27T11:25:43.412227+0000","last_undegraded":"2023-08-27T11:25:43.412227+0000","last_fullsized":"2023-08-27T11:25:43.412227+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:01:43.436813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651982+0000","last_change":"2023-08-27T11:25:42.427386+0000","last_active":"2023-08-27T11:25:43.651982+0000","last_peered":"2023-08-27T11:25:43.651982+0000","last_clean":"2023-08-27T11:25:43.651982+0000","last_became_active":"2023-08-27T11:25:42.427136+0000","last_became_peered":"2023-08-27T11:25:42.427136+0000","last_unstale":"2023-08-27T11:25:43.651982+0000","last_undegraded":"2023-08-27T11:25:43.651982+0000","last_fullsized":"2023-08-27T11:25:43.651982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:59:01.331026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"76'8","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412189+0000","last_change":"2023-08-27T11:25:41.407400+0000","last_active":"2023-08-27T11:25:43.412189+0000","last_peered":"2023-08-27T11:25:43.412189+0000","last_clean":"2023-08-27T11:25:43.412189+0000","last_became_active":"2023-08-27T11:25:41.405982+0000","last_became_peered":"2023-08-27T11:25:41.405982+0000","last_unstale":"2023-08-27T11:25:43.412189+0000","last_undegraded":"2023-08-27T11:25:43.412189+0000","last_fullsized":"2023-08-27T11:25:43.412189+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:11:53.557264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413685+0000","last_change":"2023-08-27T11:25:42.426821+0000","last_active":"2023-08-27T11:25:43.413685+0000","last_peered":"2023-08-27T11:25:43.413685+0000","last_clean":"2023-08-27T11:25:43.413685+0000","last_became_active":"2023-08-27T11:25:42.426553+0000","last_became_peered":"2023-08-27T11:25:42.426553+0000","last_unstale":"2023-08-27T11:25:43.413685+0000","last_undegraded":"2023-08-27T11:25:43.413685+0000","last_fullsized":"2023-08-27T11:25:43.413685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:55:40.769009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"76'2","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651831+0000","last_change":"2023-08-27T11:25:41.415726+0000","last_active":"2023-08-27T11:25:43.651831+0000","last_peered":"2023-08-27T11:25:43.651831+0000","last_clean":"2023-08-27T11:25:43.651831+0000","last_became_active":"2023-08-27T11:25:41.415364+0000","last_became_peered":"2023-08-27T11:25:41.415364+0000","last_unstale":"2023-08-27T11:25:43.651831+0000","last_undegraded":"2023-08-27T11:25:43.651831+0000","last_fullsized":"2023-08-27T11:25:43.651831+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:27.193402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652103+0000","last_change":"2023-08-27T11:25:42.429086+0000","last_active":"2023-08-27T11:25:43.652103+0000","last_peered":"2023-08-27T11:25:43.652103+0000","last_clean":"2023-08-27T11:25:43.652103+0000","last_became_active":"2023-08-27T11:25:42.428779+0000","last_became_peered":"2023-08-27T11:25:42.428779+0000","last_unstale":"2023-08-27T11:25:43.652103+0000","last_undegraded":"2023-08-27T11:25:43.652103+0000","last_fullsized":"2023-08-27T11:25:43.652103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:12:29.142733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"77'1","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414865+0000","last_change":"2023-08-27T11:25:41.417094+0000","last_active":"2023-08-27T11:25:43.414865+0000","last_peered":"2023-08-27T11:25:43.414865+0000","last_clean":"2023-08-27T11:25:43.414865+0000","last_became_active":"2023-08-27T11:25:41.416795+0000","last_became_peered":"2023-08-27T11:25:41.416795+0000","last_unstale":"2023-08-27T11:25:43.414865+0000","last_undegraded":"2023-08-27T11:25:43.414865+0000","last_fullsized":"2023-08-27T11:25:43.414865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:45:00.120971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651516+0000","last_change":"2023-08-27T11:25:42.425993+0000","last_active":"2023-08-27T11:25:43.651516+0000","last_peered":"2023-08-27T11:25:43.651516+0000","last_clean":"2023-08-27T11:25:43.651516+0000","last_became_active":"2023-08-27T11:25:42.425704+0000","last_became_peered":"2023-08-27T11:25:42.425704+0000","last_unstale":"2023-08-27T11:25:43.651516+0000","last_undegraded":"2023-08-27T11:25:43.651516+0000","last_fullsized":"2023-08-27T11:25:43.651516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:56:27.812945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414598+0000","last_change":"2023-08-27T11:25:41.408039+0000","last_active":"2023-08-27T11:25:43.414598+0000","last_peered":"2023-08-27T11:25:43.414598+0000","last_clean":"2023-08-27T11:25:43.414598+0000","last_became_active":"2023-08-27T11:25:41.407709+0000","last_became_peered":"2023-08-27T11:25:41.407709+0000","last_unstale":"2023-08-27T11:25:43.414598+0000","last_undegraded":"2023-08-27T11:25:43.414598+0000","last_fullsized":"2023-08-27T11:25:43.414598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:29.368878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652240+0000","last_change":"2023-08-27T11:25:42.431042+0000","last_active":"2023-08-27T11:25:43.652240+0000","last_peered":"2023-08-27T11:25:43.652240+0000","last_clean":"2023-08-27T11:25:43.652240+0000","last_became_active":"2023-08-27T11:25:42.430738+0000","last_became_peered":"2023-08-27T11:25:42.430738+0000","last_unstale":"2023-08-27T11:25:43.652240+0000","last_undegraded":"2023-08-27T11:25:43.652240+0000","last_fullsized":"2023-08-27T11:25:43.652240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:26:32.208279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413973+0000","last_change":"2023-08-27T11:25:41.414018+0000","last_active":"2023-08-27T11:25:43.413973+0000","last_peered":"2023-08-27T11:25:43.413973+0000","last_clean":"2023-08-27T11:25:43.413973+0000","last_became_active":"2023-08-27T11:25:41.413719+0000","last_became_peered":"2023-08-27T11:25:41.413719+0000","last_unstale":"2023-08-27T11:25:43.413973+0000","last_undegraded":"2023-08-27T11:25:43.413973+0000","last_fullsized":"2023-08-27T11:25:43.413973+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:24:59.711845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651111+0000","last_change":"2023-08-27T11:25:42.426777+0000","last_active":"2023-08-27T11:25:43.651111+0000","last_peered":"2023-08-27T11:25:43.651111+0000","last_clean":"2023-08-27T11:25:43.651111+0000","last_became_active":"2023-08-27T11:25:42.426492+0000","last_became_peered":"2023-08-27T11:25:42.426492+0000","last_unstale":"2023-08-27T11:25:43.651111+0000","last_undegraded":"2023-08-27T11:25:43.651111+0000","last_fullsized":"2023-08-27T11:25:43.651111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:45:30.247555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651812+0000","last_change":"2023-08-27T11:25:41.408665+0000","last_active":"2023-08-27T11:25:43.651812+0000","last_peered":"2023-08-27T11:25:43.651812+0000","last_clean":"2023-08-27T11:25:43.651812+0000","last_became_active":"2023-08-27T11:25:41.408402+0000","last_became_peered":"2023-08-27T11:25:41.408402+0000","last_unstale":"2023-08-27T11:25:43.651812+0000","last_undegraded":"2023-08-27T11:25:43.651812+0000","last_fullsized":"2023-08-27T11:25:43.651812+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:28.624353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651495+0000","last_change":"2023-08-27T11:25:42.423556+0000","last_active":"2023-08-27T11:25:43.651495+0000","last_peered":"2023-08-27T11:25:43.651495+0000","last_clean":"2023-08-27T11:25:43.651495+0000","last_became_active":"2023-08-27T11:25:42.422721+0000","last_became_peered":"2023-08-27T11:25:42.422721+0000","last_unstale":"2023-08-27T11:25:43.651495+0000","last_undegraded":"2023-08-27T11:25:43.651495+0000","last_fullsized":"2023-08-27T11:25:43.651495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:25:00.717001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652270+0000","last_change":"2023-08-27T11:25:41.411569+0000","last_active":"2023-08-27T11:25:43.652270+0000","last_peered":"2023-08-27T11:25:43.652270+0000","last_clean":"2023-08-27T11:25:43.652270+0000","last_became_active":"2023-08-27T11:25:41.411296+0000","last_became_peered":"2023-08-27T11:25:41.411296+0000","last_unstale":"2023-08-27T11:25:43.652270+0000","last_undegraded":"2023-08-27T11:25:43.652270+0000","last_fullsized":"2023-08-27T11:25:43.652270+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:41:21.227801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413269+0000","last_change":"2023-08-27T11:25:42.425087+0000","last_active":"2023-08-27T11:25:43.413269+0000","last_peered":"2023-08-27T11:25:43.413269+0000","last_clean":"2023-08-27T11:25:43.413269+0000","last_became_active":"2023-08-27T11:25:42.424790+0000","last_became_peered":"2023-08-27T11:25:42.424790+0000","last_unstale":"2023-08-27T11:25:43.413269+0000","last_undegraded":"2023-08-27T11:25:43.413269+0000","last_fullsized":"2023-08-27T11:25:43.413269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:00:40.536833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413946+0000","last_change":"2023-08-27T11:25:41.412882+0000","last_active":"2023-08-27T11:25:43.413946+0000","last_peered":"2023-08-27T11:25:43.413946+0000","last_clean":"2023-08-27T11:25:43.413946+0000","last_became_active":"2023-08-27T11:25:41.412635+0000","last_became_peered":"2023-08-27T11:25:41.412635+0000","last_unstale":"2023-08-27T11:25:43.413946+0000","last_undegraded":"2023-08-27T11:25:43.413946+0000","last_fullsized":"2023-08-27T11:25:43.413946+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:17.351848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651698+0000","last_change":"2023-08-27T11:25:42.425442+0000","last_active":"2023-08-27T11:25:43.651698+0000","last_peered":"2023-08-27T11:25:43.651698+0000","last_clean":"2023-08-27T11:25:43.651698+0000","last_became_active":"2023-08-27T11:25:42.423501+0000","last_became_peered":"2023-08-27T11:25:42.423501+0000","last_unstale":"2023-08-27T11:25:43.651698+0000","last_undegraded":"2023-08-27T11:25:43.651698+0000","last_fullsized":"2023-08-27T11:25:43.651698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:12:50.518148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651922+0000","last_change":"2023-08-27T11:25:41.405653+0000","last_active":"2023-08-27T11:25:43.651922+0000","last_peered":"2023-08-27T11:25:43.651922+0000","last_clean":"2023-08-27T11:25:43.651922+0000","last_became_active":"2023-08-27T11:25:41.405385+0000","last_became_peered":"2023-08-27T11:25:41.405385+0000","last_unstale":"2023-08-27T11:25:43.651922+0000","last_undegraded":"2023-08-27T11:25:43.651922+0000","last_fullsized":"2023-08-27T11:25:43.651922+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:41:56.778277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651233+0000","last_change":"2023-08-27T11:25:42.419269+0000","last_active":"2023-08-27T11:25:43.651233+0000","last_peered":"2023-08-27T11:25:43.651233+0000","last_clean":"2023-08-27T11:25:43.651233+0000","last_became_active":"2023-08-27T11:25:42.418440+0000","last_became_peered":"2023-08-27T11:25:42.418440+0000","last_unstale":"2023-08-27T11:25:43.651233+0000","last_undegraded":"2023-08-27T11:25:43.651233+0000","last_fullsized":"2023-08-27T11:25:43.651233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:59:38.887388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"76'2","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651603+0000","last_change":"2023-08-27T11:25:41.417399+0000","last_active":"2023-08-27T11:25:43.651603+0000","last_peered":"2023-08-27T11:25:43.651603+0000","last_clean":"2023-08-27T11:25:43.651603+0000","last_became_active":"2023-08-27T11:25:41.417076+0000","last_became_peered":"2023-08-27T11:25:41.417076+0000","last_unstale":"2023-08-27T11:25:43.651603+0000","last_undegraded":"2023-08-27T11:25:43.651603+0000","last_fullsized":"2023-08-27T11:25:43.651603+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:57.010141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":137,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413886+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:25:43.413886+0000","last_peered":"2023-08-27T11:25:43.413886+0000","last_clean":"2023-08-27T11:25:43.413886+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:25:43.413886+0000","last_undegraded":"2023-08-27T11:25:43.413886+0000","last_fullsized":"2023-08-27T11:25:43.413886+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:17.452171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651750+0000","last_change":"2023-08-27T11:25:42.436516+0000","last_active":"2023-08-27T11:25:43.651750+0000","last_peered":"2023-08-27T11:25:43.651750+0000","last_clean":"2023-08-27T11:25:43.651750+0000","last_became_active":"2023-08-27T11:25:42.436127+0000","last_became_peered":"2023-08-27T11:25:42.436127+0000","last_unstale":"2023-08-27T11:25:43.651750+0000","last_undegraded":"2023-08-27T11:25:43.651750+0000","last_fullsized":"2023-08-27T11:25:43.651750+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:36:03.520085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651699+0000","last_change":"2023-08-27T11:25:41.405709+0000","last_active":"2023-08-27T11:25:43.651699+0000","last_peered":"2023-08-27T11:25:43.651699+0000","last_clean":"2023-08-27T11:25:43.651699+0000","last_became_active":"2023-08-27T11:25:41.405421+0000","last_became_peered":"2023-08-27T11:25:41.405421+0000","last_unstale":"2023-08-27T11:25:43.651699+0000","last_undegraded":"2023-08-27T11:25:43.651699+0000","last_fullsized":"2023-08-27T11:25:43.651699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:46:27.115229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":137,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413504+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:25:43.413504+0000","last_peered":"2023-08-27T11:25:43.413504+0000","last_clean":"2023-08-27T11:25:43.413504+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:25:43.413504+0000","last_undegraded":"2023-08-27T11:25:43.413504+0000","last_fullsized":"2023-08-27T11:25:43.413504+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:58.022576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414463+0000","last_change":"2023-08-27T11:25:42.437402+0000","last_active":"2023-08-27T11:25:43.414463+0000","last_peered":"2023-08-27T11:25:43.414463+0000","last_clean":"2023-08-27T11:25:43.414463+0000","last_became_active":"2023-08-27T11:25:42.436362+0000","last_became_peered":"2023-08-27T11:25:42.436362+0000","last_unstale":"2023-08-27T11:25:43.414463+0000","last_undegraded":"2023-08-27T11:25:43.414463+0000","last_fullsized":"2023-08-27T11:25:43.414463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:19:41.107095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"76'1","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414913+0000","last_change":"2023-08-27T11:25:41.407112+0000","last_active":"2023-08-27T11:25:43.414913+0000","last_peered":"2023-08-27T11:25:43.414913+0000","last_clean":"2023-08-27T11:25:43.414913+0000","last_became_active":"2023-08-27T11:25:41.406794+0000","last_became_peered":"2023-08-27T11:25:41.406794+0000","last_unstale":"2023-08-27T11:25:43.414913+0000","last_undegraded":"2023-08-27T11:25:43.414913+0000","last_fullsized":"2023-08-27T11:25:43.414913+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:21:45.842462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652344+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:25:43.652344+0000","last_peered":"2023-08-27T11:25:43.652344+0000","last_clean":"2023-08-27T11:25:43.652344+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:25:43.652344+0000","last_undegraded":"2023-08-27T11:25:43.652344+0000","last_fullsized":"2023-08-27T11:25:43.652344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:52:48.271155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651809+0000","last_change":"2023-08-27T11:25:42.437096+0000","last_active":"2023-08-27T11:25:43.651809+0000","last_peered":"2023-08-27T11:25:43.651809+0000","last_clean":"2023-08-27T11:25:43.651809+0000","last_became_active":"2023-08-27T11:25:42.436777+0000","last_became_peered":"2023-08-27T11:25:42.436777+0000","last_unstale":"2023-08-27T11:25:43.651809+0000","last_undegraded":"2023-08-27T11:25:43.651809+0000","last_fullsized":"2023-08-27T11:25:43.651809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:44:37.646415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412774+0000","last_change":"2023-08-27T11:25:41.417819+0000","last_active":"2023-08-27T11:25:43.412774+0000","last_peered":"2023-08-27T11:25:43.412774+0000","last_clean":"2023-08-27T11:25:43.412774+0000","last_became_active":"2023-08-27T11:25:41.417532+0000","last_became_peered":"2023-08-27T11:25:41.417532+0000","last_unstale":"2023-08-27T11:25:43.412774+0000","last_undegraded":"2023-08-27T11:25:43.412774+0000","last_fullsized":"2023-08-27T11:25:43.412774+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:14.192009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652112+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:25:43.652112+0000","last_peered":"2023-08-27T11:25:43.652112+0000","last_clean":"2023-08-27T11:25:43.652112+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:25:43.652112+0000","last_undegraded":"2023-08-27T11:25:43.652112+0000","last_fullsized":"2023-08-27T11:25:43.652112+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:17:05.598771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413348+0000","last_change":"2023-08-27T11:25:42.428787+0000","last_active":"2023-08-27T11:25:43.413348+0000","last_peered":"2023-08-27T11:25:43.413348+0000","last_clean":"2023-08-27T11:25:43.413348+0000","last_became_active":"2023-08-27T11:25:42.428492+0000","last_became_peered":"2023-08-27T11:25:42.428492+0000","last_unstale":"2023-08-27T11:25:43.413348+0000","last_undegraded":"2023-08-27T11:25:43.413348+0000","last_fullsized":"2023-08-27T11:25:43.413348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:21:44.148493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"76'2","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651881+0000","last_change":"2023-08-27T11:25:41.417161+0000","last_active":"2023-08-27T11:25:43.651881+0000","last_peered":"2023-08-27T11:25:43.651881+0000","last_clean":"2023-08-27T11:25:43.651881+0000","last_became_active":"2023-08-27T11:25:41.416886+0000","last_became_peered":"2023-08-27T11:25:41.416886+0000","last_unstale":"2023-08-27T11:25:43.651881+0000","last_undegraded":"2023-08-27T11:25:43.651881+0000","last_fullsized":"2023-08-27T11:25:43.651881+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:57:29.491054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":117,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651641+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:25:43.651641+0000","last_peered":"2023-08-27T11:25:43.651641+0000","last_clean":"2023-08-27T11:25:43.651641+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:25:43.651641+0000","last_undegraded":"2023-08-27T11:25:43.651641+0000","last_fullsized":"2023-08-27T11:25:43.651641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:22.782420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651103+0000","last_change":"2023-08-27T11:25:42.431395+0000","last_active":"2023-08-27T11:25:43.651103+0000","last_peered":"2023-08-27T11:25:43.651103+0000","last_clean":"2023-08-27T11:25:43.651103+0000","last_became_active":"2023-08-27T11:25:42.431008+0000","last_became_peered":"2023-08-27T11:25:42.431008+0000","last_unstale":"2023-08-27T11:25:43.651103+0000","last_undegraded":"2023-08-27T11:25:43.651103+0000","last_fullsized":"2023-08-27T11:25:43.651103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:50.087332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"76'1","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.413704+0000","last_change":"2023-08-27T11:25:41.414866+0000","last_active":"2023-08-27T11:25:43.413704+0000","last_peered":"2023-08-27T11:25:43.413704+0000","last_clean":"2023-08-27T11:25:43.413704+0000","last_became_active":"2023-08-27T11:25:41.414525+0000","last_became_peered":"2023-08-27T11:25:41.414525+0000","last_unstale":"2023-08-27T11:25:43.413704+0000","last_undegraded":"2023-08-27T11:25:43.413704+0000","last_fullsized":"2023-08-27T11:25:43.413704+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:29.368878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":267,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651642+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:25:43.651642+0000","last_peered":"2023-08-27T11:25:43.651642+0000","last_clean":"2023-08-27T11:25:43.651642+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:25:43.651642+0000","last_undegraded":"2023-08-27T11:25:43.651642+0000","last_fullsized":"2023-08-27T11:25:43.651642+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:38:27.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":121,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.412462+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:25:43.412462+0000","last_peered":"2023-08-27T11:25:43.412462+0000","last_clean":"2023-08-27T11:25:43.412462+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:25:43.412462+0000","last_undegraded":"2023-08-27T11:25:43.412462+0000","last_fullsized":"2023-08-27T11:25:43.412462+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:41:54.956126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651487+0000","last_change":"2023-08-27T11:25:42.428357+0000","last_active":"2023-08-27T11:25:43.651487+0000","last_peered":"2023-08-27T11:25:43.651487+0000","last_clean":"2023-08-27T11:25:43.651487+0000","last_became_active":"2023-08-27T11:25:42.428029+0000","last_became_peered":"2023-08-27T11:25:42.428029+0000","last_unstale":"2023-08-27T11:25:43.651487+0000","last_undegraded":"2023-08-27T11:25:43.651487+0000","last_fullsized":"2023-08-27T11:25:43.651487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:49:01.172962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651238+0000","last_change":"2023-08-27T11:25:41.416796+0000","last_active":"2023-08-27T11:25:43.651238+0000","last_peered":"2023-08-27T11:25:43.651238+0000","last_clean":"2023-08-27T11:25:43.651238+0000","last_became_active":"2023-08-27T11:25:41.416559+0000","last_became_peered":"2023-08-27T11:25:41.416559+0000","last_unstale":"2023-08-27T11:25:43.651238+0000","last_undegraded":"2023-08-27T11:25:43.651238+0000","last_fullsized":"2023-08-27T11:25:43.651238+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:59:47.275628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":115,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651346+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:25:43.651346+0000","last_peered":"2023-08-27T11:25:43.651346+0000","last_clean":"2023-08-27T11:25:43.651346+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:25:43.651346+0000","last_undegraded":"2023-08-27T11:25:43.651346+0000","last_fullsized":"2023-08-27T11:25:43.651346+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:17.452171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.651749+0000","last_change":"2023-08-27T11:25:42.432962+0000","last_active":"2023-08-27T11:25:43.651749+0000","last_peered":"2023-08-27T11:25:43.651749+0000","last_clean":"2023-08-27T11:25:43.651749+0000","last_became_active":"2023-08-27T11:25:42.432704+0000","last_became_peered":"2023-08-27T11:25:42.432704+0000","last_unstale":"2023-08-27T11:25:43.651749+0000","last_undegraded":"2023-08-27T11:25:43.651749+0000","last_fullsized":"2023-08-27T11:25:43.651749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:41.389437+0000","last_clean_scrub_stamp":"2023-08-27T11:25:41.389437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:56:25.216408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.652167+0000","last_change":"2023-08-27T11:25:41.417495+0000","last_active":"2023-08-27T11:25:43.652167+0000","last_peered":"2023-08-27T11:25:43.652167+0000","last_clean":"2023-08-27T11:25:43.652167+0000","last_became_active":"2023-08-27T11:25:41.416993+0000","last_became_peered":"2023-08-27T11:25:41.416993+0000","last_unstale":"2023-08-27T11:25:43.652167+0000","last_undegraded":"2023-08-27T11:25:43.652167+0000","last_fullsized":"2023-08-27T11:25:43.652167+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:40.384280+0000","last_clean_scrub_stamp":"2023-08-27T11:25:40.384280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:28.842898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-08-27T11:25:43.414442+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:25:43.414442+0000","last_peered":"2023-08-27T11:25:43.414442+0000","last_clean":"2023-08-27T11:25:43.414442+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:25:43.414442+0000","last_undegraded":"2023-08-27T11:25:43.414442+0000","last_fullsized":"2023-08-27T11:25:43.414442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:13:01.063974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574958,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32696,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1548288,"data_stored":1022587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51200000000000001,"15min":0.48899999999999999},"min":{"1min":0.34000000000000002,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.66200000000000003,"5min":0.79600000000000004,"15min":0.93500000000000005},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.49399999999999999},"min":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.252},"max":{"1min":0.73499999999999999,"5min":0.74099999999999999,"15min":0.81499999999999995},"last":0.40100000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.34200000000000003,"15min":0.26600000000000001},"max":{"1min":0.72299999999999998,"5min":0.77500000000000002,"15min":0.872},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54200000000000004,"15min":0.51100000000000001},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.253},"max":{"1min":0.67800000000000005,"5min":0.71999999999999997,"15min":0.84999999999999998},"last":0.55100000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.48199999999999998},"min":{"1min":0.32400000000000001,"5min":0.29099999999999998,"15min":0.28299999999999997},"max":{"1min":0.63,"5min":0.71699999999999997,"15min":0.79500000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56499999999999995,"15min":0.54400000000000004},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.33800000000000002},"max":{"1min":0.73999999999999999,"5min":0.78700000000000003,"15min":0.90200000000000002},"last":0.57699999999999996}]},{"osd":3,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.54800000000000004},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.748,"15min":0.84299999999999997},"last":0.495},{"interface":"front","average":{"1min":0.625,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.48199999999999998,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.70399999999999996,"5min":0.83499999999999996,"15min":0.91700000000000004},"last":0.61099999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49399999999999999,"15min":0.50800000000000001},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.29899999999999999},"max":{"1min":0.72499999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52900000000000003,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72099999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53400000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.31900000000000001},"max":{"1min":0.749,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.43099999999999999,"5min":0.377,"15min":0.36799999999999999},"max":{"1min":0.77300000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.59499999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80800000000000005,"5min":0.82399999999999995,"15min":0.86099999999999999},"last":0.623},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.755,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574959,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":966656,"data_stored":434379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73099999999999998,"5min":0.86199999999999999,"15min":0.89800000000000002},"last":0.54600000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.316,"5min":0.312,"15min":0.312},"max":{"1min":0.75900000000000001,"5min":0.80600000000000005,"15min":0.873},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75600000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58799999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64700000000000002,"5min":0.74199999999999999,"15min":0.85899999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55800000000000005,"15min":0.58799999999999997},"min":{"1min":0.41699999999999998,"5min":0.39400000000000002,"15min":0.377},"max":{"1min":0.68600000000000005,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.60099999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.78900000000000003,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.70199999999999996,"5min":0.84699999999999998,"15min":0.93799999999999994},"last":0.56999999999999995}]},{"osd":4,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.32400000000000001,"5min":0.307,"15min":0.28999999999999998},"max":{"1min":0.70499999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.437,"5min":0.375,"15min":0.374},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.61799999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33400000000000002},"max":{"1min":0.63400000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.60099999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.875,"15min":0.875},"last":0.70599999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.47599999999999998,"5min":0.47099999999999997,"15min":0.34899999999999998},"max":{"1min":0.68200000000000005,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1544192,"data_stored":1022601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.31,"15min":0.63400000000000001},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":0.502,"5min":0.502,"15min":11.971},"last":0.224},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.96899999999999997},"min":{"1min":0.22500000000000001,"5min":0.221,"15min":0.20799999999999999},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.26800000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33700000000000002,"15min":0.66400000000000003},"min":{"1min":0.247,"5min":0.221,"15min":0.221},"max":{"1min":0.46600000000000003,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.33800000000000002,"15min":0.34399999999999997},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":0.48599999999999999,"5min":0.53700000000000003,"15min":0.58599999999999997},"last":0.28899999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.38,"15min":0.67600000000000005},"min":{"1min":0.30099999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":10.99},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36699999999999999,"15min":1.002},"min":{"1min":0.29399999999999998,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.505,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.34699999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57199999999999995,"15min":0.60299999999999998},"min":{"1min":0.44500000000000001,"5min":0.36099999999999999,"15min":0.33400000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":1.26},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55800000000000005,"15min":0.84199999999999997},"min":{"1min":0.38900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.77600000000000002,"5min":0.86299999999999999,"15min":10.957000000000001},"last":0.61799999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57299999999999995,"15min":1.2},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.36199999999999999},"max":{"1min":0.79300000000000004,"5min":0.85299999999999998,"15min":23.016999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.60299999999999998},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.296},"last":0.59399999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.57999999999999996,"15min":0.88300000000000001},"min":{"1min":0.434,"5min":0.371,"15min":0.371},"max":{"1min":0.84899999999999998,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":1.2170000000000001},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.74299999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":1.212},"min":{"1min":0.45400000000000001,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.76500000000000001,"5min":0.89600000000000002,"15min":22.988},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.63900000000000001},"min":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86199999999999999,"5min":0.91600000000000004,"15min":1.327},"last":0.68600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574958,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":966656,"data_stored":433120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.29299999999999998,"15min":0.31},"min":{"1min":0.214,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.45300000000000001,"5min":0.50900000000000001,"15min":0.52300000000000002},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.35099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.62},"last":0.29999999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.33900000000000002},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.54400000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.245},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.33700000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.223},"max":{"1min":0.49299999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.34300000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.34499999999999997},"min":{"1min":0.249,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.50800000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.36199999999999999,"15min":0.372},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.53800000000000003,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.33200000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.56000000000000005},"min":{"1min":0.35399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85399999999999998,"5min":0.92500000000000004,"15min":1.214},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.35799999999999998,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.81000000000000005,"15min":1.2569999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.371,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81000000000000005,"5min":0.82599999999999996,"15min":0.91400000000000003},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59899999999999998},"min":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83999999999999997,"5min":0.91100000000000003,"15min":1.603},"last":0.51900000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79900000000000004,"5min":0.86099999999999999,"15min":0.97599999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73999999999999999,"5min":0.871,"15min":0.871},"last":0.57999999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75800000000000001,"5min":0.94199999999999995,"15min":1.383},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86199999999999999,"5min":0.88900000000000001,"15min":0.92000000000000004},"last":0.69699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574959,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":432163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.30099999999999999,"15min":0.30199999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20200000000000001},"max":{"1min":0.48899999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.317},"min":{"1min":0.23100000000000001,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.46700000000000003,"5min":0.52500000000000002,"15min":0.629},"last":0.33200000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34200000000000003,"15min":0.34799999999999998},"min":{"1min":0.23000000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52200000000000002,"5min":0.749,"15min":0.749},"last":0.247}]},{"osd":3,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36299999999999999,"15min":0.38400000000000001},"min":{"1min":0.26600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57099999999999995,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.318},{"interface":"front","average":{"1min":0.378,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.27500000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.69499999999999995},"last":0.30599999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.42699999999999999,"5min":0.39600000000000002,"15min":0.35599999999999998},"max":{"1min":0.82199999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.42599999999999999,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.75,"5min":0.879,"15min":0.879},"last":0.45700000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.432},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51100000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.73499999999999999,"5min":0.85199999999999998,"15min":0.85299999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76400000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.441}]},{"osd":7,"last update":"Sun Aug 27 11:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59699999999999998,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.79500000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":432163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.27700000000000002,"5min":0.30099999999999999,"15min":0.29999999999999999},"min":{"1min":0.21199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.34100000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32600000000000001,"15min":0.312},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46200000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.23100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.32700000000000001,"15min":0.312},"min":{"1min":0.22700000000000001,"5min":0.21299999999999999,"15min":0.19500000000000001},"max":{"1min":0.48599999999999999,"5min":0.50600000000000001,"15min":0.60399999999999998},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.308,"5min":0.33400000000000002,"15min":0.34000000000000002},"min":{"1min":0.26500000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.38200000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"last":0.308}]},{"osd":2,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44400000000000001,"5min":0.57199999999999995,"15min":0.57599999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.42999999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.34000000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.436,"5min":0.316,"15min":0.28599999999999998},"max":{"1min":0.68200000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68700000000000006,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56399999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.36799999999999999,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.57299999999999995},"min":{"1min":0.442,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.67000000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58899999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.373},"max":{"1min":0.73999999999999999,"5min":1.091,"15min":1.091},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60099999999999998,"15min":0.61499999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75600000000000001,"5min":1.111,"15min":1.111},"last":0.57799999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.61799999999999999},"min":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.36699999999999999},"max":{"1min":0.70199999999999996,"5min":1.032,"15min":1.236},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.58499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68200000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.64000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":954368,"data_stored":432323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.32700000000000001,"15min":0.30399999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.441},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.379,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.65100000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46800000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76300000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.433,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.57399999999999995}]},{"osd":3,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.70699999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.44,"5min":0.41799999999999998,"15min":0.41399999999999998},"max":{"1min":0.751,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59299999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58599999999999997,"15min":0.59499999999999997},"min":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.379},"max":{"1min":0.85999999999999999,"5min":0.879,"15min":0.879},"last":0.60899999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.55000000000000004},"min":{"1min":0.39600000000000002,"5min":0.37,"15min":0.31},"max":{"1min":0.78900000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.625},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.62,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.37,"15min":0.35299999999999998},"max":{"1min":0.76700000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.65600000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.82499999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.34200000000000003},"max":{"1min":0.79200000000000004,"5min":0.83799999999999997,"15min":0.89300000000000002},"last":0.68100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":432163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.63300000000000001,"5min":0.78600000000000003,"15min":0.96899999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.29199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.60099999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70499999999999996,"5min":0.747,"15min":0.92200000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52400000000000002,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.63200000000000001,"5min":0.72899999999999998,"15min":1.075},"last":0.53500000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.78000000000000003,"15min":0.95699999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54600000000000004,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.65300000000000002,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.56299999999999994}]},{"osd":3,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72299999999999998,"5min":0.81000000000000005,"15min":1.03},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58199999999999996,"15min":0.59499999999999997},"min":{"1min":0.48499999999999999,"5min":0.36699999999999999,"15min":0.35199999999999998},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":1.026},"last":0.58699999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72999999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.441},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.315,"15min":0.26900000000000002},"max":{"1min":0.78900000000000003,"5min":0.86399999999999999,"15min":0.91400000000000003},"last":0.42099999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.76500000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.48399999999999999,"5min":0.36599999999999999,"15min":0.28799999999999998},"max":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.94199999999999995},"last":0.5}]},{"osd":7,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.51300000000000001,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.95899999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.61199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.83699999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:25:51.463 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:25:51.464 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:25:51.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.624+0000 7f89499be700 1 Processor -- start 2023-08-27T11:25:51.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.624+0000 7f89499be700 1 -- start start 2023-08-27T11:25:51.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.624+0000 7f89499be700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8944146b30 0x7f894406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.624+0000 7f89499be700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89440715c0 0x7f8944161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.624+0000 7f89499be700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894406ce30 con 0x7f89440d1580 2023-08-27T11:25:51.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.624+0000 7f89499be700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894406c870 con 0x7f8944146b30 2023-08-27T11:25:51.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.624+0000 7f89499be700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894406cb50 con 0x7f89440715c0 2023-08-27T11:25:51.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f8942ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8944146b30 0x7f894406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f89437fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f89440d1580 0x7f894415eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54522/0 (socket says 172.21.15.112:54522) 2023-08-27T11:25:51.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f89437fe700 1 -- 172.21.15.112:0/2583385752 learned_addr learned my addr 172.21.15.112:0/2583385752 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:51.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f8942ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8944146b30 0x7f894406bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60138/0 (socket says 172.21.15.112:60138) 2023-08-27T11:25:51.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f89427fc700 1 --2- 172.21.15.112:0/2583385752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89440715c0 0x7f8944161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f8942ffd700 1 -- 172.21.15.112:0/2583385752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89440715c0 msgr2=0x7f8944161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:51.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f8942ffd700 1 --2- 172.21.15.112:0/2583385752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89440715c0 0x7f8944161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f8942ffd700 1 -- 172.21.15.112:0/2583385752 >> v1:172.21.15.112:6789/0 conn(0x7f89440d1580 legacy=0x7f894415eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:51.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f8942ffd700 1 -- 172.21.15.112:0/2583385752 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8944161020 con 0x7f8944146b30 2023-08-27T11:25:51.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f8941ffb700 1 -- 172.21.15.112:0/2583385752 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040910102 0 0) 0x7f894406ce30 con 0x7f89440d1580 2023-08-27T11:25:51.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f8942ffd700 1 --2- 172.21.15.112:0/2583385752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8944146b30 0x7f894406bfd0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8938009840 tx=0x7f8938009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2317875d2f3f815 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:51.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f8941ffb700 1 -- 172.21.15.112:0/2583385752 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938011830 con 0x7f8944146b30 2023-08-27T11:25:51.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f8941ffb700 1 -- 172.21.15.112:0/2583385752 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8938011990 con 0x7f8944146b30 2023-08-27T11:25:51.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.625+0000 7f8941ffb700 1 -- 172.21.15.112:0/2583385752 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938011d20 con 0x7f8944146b30 2023-08-27T11:25:51.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.626+0000 7f89499be700 1 -- 172.21.15.112:0/2583385752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8944146b30 msgr2=0x7f894406bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:51.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.626+0000 7f89499be700 1 --2- 172.21.15.112:0/2583385752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8944146b30 0x7f894406bfd0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8938009840 tx=0x7f8938009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:51.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.626+0000 7f89499be700 1 -- 172.21.15.112:0/2583385752 shutdown_connections 2023-08-27T11:25:51.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.626+0000 7f89499be700 1 --2- 172.21.15.112:0/2583385752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8944146b30 0x7f894406bfd0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.626+0000 7f89499be700 1 --2- 172.21.15.112:0/2583385752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89440715c0 0x7f8944161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.626+0000 7f89499be700 1 -- 172.21.15.112:0/2583385752 >> 172.21.15.112:0/2583385752 conn(0x7f894406b1d0 msgr2=0x7f894406b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:51.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.626+0000 7f89499be700 1 -- 172.21.15.112:0/2583385752 shutdown_connections 2023-08-27T11:25:51.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.626+0000 7f89499be700 1 -- 172.21.15.112:0/2583385752 wait complete. 2023-08-27T11:25:51.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.627+0000 7f89499be700 1 Processor -- start 2023-08-27T11:25:51.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.627+0000 7f89499be700 1 -- start start 2023-08-27T11:25:51.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f89499be700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89440715c0 0x7f8944142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f89499be700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89440d1580 0x7f89441f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f89499be700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8944146b30 0x7f89440cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f89499be700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89440cbfb0 con 0x7f89440715c0 2023-08-27T11:25:51.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f89499be700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89440ca640 con 0x7f89440d1580 2023-08-27T11:25:51.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f89499be700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89440cc1e0 con 0x7f8944146b30 2023-08-27T11:25:51.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f8942ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89440715c0 0x7f8944142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f8942ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89440715c0 0x7f8944142fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60144/0 (socket says 172.21.15.112:60144) 2023-08-27T11:25:51.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f8942ffd700 1 -- 172.21.15.112:0/3011974036 learned_addr learned my addr 172.21.15.112:0/3011974036 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:51.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f89427fc700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89440d1580 0x7f89441f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f89437fe700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8944146b30 0x7f89440cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f8942ffd700 1 -- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8944146b30 msgr2=0x7f89440cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:51.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f8942ffd700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8944146b30 0x7f89440cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f8942ffd700 1 -- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89440d1580 msgr2=0x7f89441f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:51.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f8942ffd700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89440d1580 0x7f89441f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f8942ffd700 1 -- 172.21.15.112:0/3011974036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380094a0 con 0x7f89440715c0 2023-08-27T11:25:51.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.628+0000 7f8942ffd700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89440715c0 0x7f8944142fc0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8938000c00 tx=0x7f8938004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:51.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.629+0000 7f892bfff700 1 -- 172.21.15.112:0/3011974036 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89380036a0 con 0x7f89440715c0 2023-08-27T11:25:51.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.629+0000 7f892bfff700 1 -- 172.21.15.112:0/3011974036 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893801e070 con 0x7f89440715c0 2023-08-27T11:25:51.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.629+0000 7f89499be700 1 -- 172.21.15.112:0/3011974036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89440cc4c0 con 0x7f89440715c0 2023-08-27T11:25:51.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.629+0000 7f892bfff700 1 -- 172.21.15.112:0/3011974036 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938003a60 con 0x7f89440715c0 2023-08-27T11:25:51.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.629+0000 7f89499be700 1 -- 172.21.15.112:0/3011974036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89440cca80 con 0x7f89440715c0 2023-08-27T11:25:51.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.630+0000 7f89499be700 1 -- 172.21.15.112:0/3011974036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8944067a20 con 0x7f89440715c0 2023-08-27T11:25:51.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.630+0000 7f892bfff700 1 -- 172.21.15.112:0/3011974036 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f8938003bc0 con 0x7f89440715c0 2023-08-27T11:25:51.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.630+0000 7f892bfff700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f893006a320 0x7f893006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:51.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.631+0000 7f892bfff700 1 -- 172.21.15.112:0/3011974036 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f89380899e0 con 0x7f89440715c0 2023-08-27T11:25:51.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.631+0000 7f89427fc700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f893006a320 0x7f893006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:51.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.631+0000 7f89427fc700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f893006a320 0x7f893006c7e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f89440cd8e0 tx=0x7f8934008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:51.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.635+0000 7f892bfff700 1 -- 172.21.15.112:0/3011974036 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8938054e00 con 0x7f89440715c0 2023-08-27T11:25:51.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.899+0000 7f89499be700 1 -- 172.21.15.112:0/3011974036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f89440d0c70 con 0x7f89440715c0 2023-08-27T11:25:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.903+0000 7f892bfff700 1 -- 172.21.15.112:0/3011974036 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f89380150b0 con 0x7f89440715c0 2023-08-27T11:25:51.904 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-08-27T11:25:51.904 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA== 2023-08-27T11:25:51.904 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2023-08-27T11:25:51.904 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2023-08-27T11:25:51.904 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:25:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.905+0000 7f89499be700 1 -- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f893006a320 msgr2=0x7f893006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.905+0000 7f89499be700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f893006a320 0x7f893006c7e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f89440cd8e0 tx=0x7f8934008040 comp rx=0 tx=0).stop 2023-08-27T11:25:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.905+0000 7f89499be700 1 -- 172.21.15.112:0/3011974036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89440715c0 msgr2=0x7f8944142fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.905+0000 7f89499be700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89440715c0 0x7f8944142fc0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8938000c00 tx=0x7f8938004750 comp rx=0 tx=0).stop 2023-08-27T11:25:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.905+0000 7f89499be700 1 -- 172.21.15.112:0/3011974036 shutdown_connections 2023-08-27T11:25:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.905+0000 7f89499be700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f893006a320 0x7f893006c7e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.905+0000 7f89499be700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8944146b30 0x7f89440cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.905+0000 7f89499be700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89440715c0 0x7f8944142fc0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.905+0000 7f89499be700 1 --2- 172.21.15.112:0/3011974036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89440d1580 0x7f89441f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.905+0000 7f89499be700 1 -- 172.21.15.112:0/3011974036 >> 172.21.15.112:0/3011974036 conn(0x7f894406b1d0 msgr2=0x7f89440cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.905+0000 7f89499be700 1 -- 172.21.15.112:0/3011974036 shutdown_connections 2023-08-27T11:25:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:51.905+0000 7f89499be700 1 -- 172.21.15.112:0/3011974036 wait complete. 2023-08-27T11:25:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2023-08-27T11:25:51.918 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:25:52.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.037+0000 7f19f6df5700 1 Processor -- start 2023-08-27T11:25:52.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.037+0000 7f19f6df5700 1 -- start start 2023-08-27T11:25:52.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.038+0000 7f19f6df5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f0146bb0 0x7f19f015e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.038+0000 7f19f6df5700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19f0146f90 0x7f19f00708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.038+0000 7f19f6df5700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f01610d0 con 0x7f19f0072230 2023-08-27T11:25:52.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.038+0000 7f19f6df5700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f015ea50 con 0x7f19f0146f90 2023-08-27T11:25:52.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.038+0000 7f19f6df5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f00727b0 con 0x7f19f0146bb0 2023-08-27T11:25:52.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.038+0000 7f19effff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19f0146f90 0x7f19f00708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.038+0000 7f19f4b91700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f0146bb0 0x7f19f015e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.038+0000 7f19f5392700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f19f0072230 0x7f19f00cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:33830/0 (socket says 172.21.15.112:33830) 2023-08-27T11:25:52.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.038+0000 7f19f5392700 1 -- 172.21.15.112:0/3186777881 learned_addr learned my addr 172.21.15.112:0/3186777881 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:52.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.039+0000 7f19effff700 1 -- 172.21.15.112:0/3186777881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f0146bb0 msgr2=0x7f19f015e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:52.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.039+0000 7f19effff700 1 --2- 172.21.15.112:0/3186777881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f0146bb0 0x7f19f015e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.039+0000 7f19effff700 1 -- 172.21.15.112:0/3186777881 >> v1:172.21.15.138:6789/0 conn(0x7f19f0072230 legacy=0x7f19f00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.039+0000 7f19effff700 1 -- 172.21.15.112:0/3186777881 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f00d0a40 con 0x7f19f0146f90 2023-08-27T11:25:52.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.039+0000 7f19ef7fe700 1 -- 172.21.15.112:0/3186777881 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1479443430 0 0) 0x7f19f01610d0 con 0x7f19f0072230 2023-08-27T11:25:52.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.039+0000 7f19effff700 1 --2- 172.21.15.112:0/3186777881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f0146f90 0x7f19f00708a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009840 tx=0x7f19dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=707fe850f8dc560e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:52.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.039+0000 7f19ef7fe700 1 -- 172.21.15.112:0/3186777881 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc011830 con 0x7f19f0146f90 2023-08-27T11:25:52.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.039+0000 7f19ef7fe700 1 -- 172.21.15.112:0/3186777881 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc011990 con 0x7f19f0146f90 2023-08-27T11:25:52.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.039+0000 7f19ef7fe700 1 -- 172.21.15.112:0/3186777881 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc011d20 con 0x7f19f0146f90 2023-08-27T11:25:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.040+0000 7f19f6df5700 1 -- 172.21.15.112:0/3186777881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f0146f90 msgr2=0x7f19f00708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.040+0000 7f19f6df5700 1 --2- 172.21.15.112:0/3186777881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f0146f90 0x7f19f00708a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009840 tx=0x7f19dc009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.040+0000 7f19f6df5700 1 -- 172.21.15.112:0/3186777881 shutdown_connections 2023-08-27T11:25:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.040+0000 7f19f6df5700 1 --2- 172.21.15.112:0/3186777881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f0146bb0 0x7f19f015e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.040+0000 7f19f6df5700 1 --2- 172.21.15.112:0/3186777881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f0146f90 0x7f19f00708a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.040+0000 7f19f6df5700 1 -- 172.21.15.112:0/3186777881 >> 172.21.15.112:0/3186777881 conn(0x7f19f006f970 msgr2=0x7f19f006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:52.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.040+0000 7f19f6df5700 1 -- 172.21.15.112:0/3186777881 shutdown_connections 2023-08-27T11:25:52.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.040+0000 7f19f6df5700 1 -- 172.21.15.112:0/3186777881 wait complete. 2023-08-27T11:25:52.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.041+0000 7f19f6df5700 1 Processor -- start 2023-08-27T11:25:52.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.041+0000 7f19f6df5700 1 -- start start 2023-08-27T11:25:52.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.042+0000 7f19f6df5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f0072230 0x7f19f015ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.042+0000 7f19f6df5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f0146bb0 0x7f19f0144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.042+0000 7f19f6df5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f0146f90 0x7f19f00d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.042+0000 7f19f6df5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f013da40 con 0x7f19f0146bb0 2023-08-27T11:25:52.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.042+0000 7f19f6df5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f013ded0 con 0x7f19f0072230 2023-08-27T11:25:52.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.042+0000 7f19f6df5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f013d110 con 0x7f19f0146f90 2023-08-27T11:25:52.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.042+0000 7f19effff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f0146bb0 0x7f19f0144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.042+0000 7f19effff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f0146bb0 0x7f19f0144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60160/0 (socket says 172.21.15.112:60160) 2023-08-27T11:25:52.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19effff700 1 -- 172.21.15.112:0/2528030414 learned_addr learned my addr 172.21.15.112:0/2528030414 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:52.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19f5392700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f0146f90 0x7f19f00d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19f4b91700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f0072230 0x7f19f015ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19effff700 1 -- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f0146f90 msgr2=0x7f19f00d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19effff700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f0146f90 0x7f19f00d10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19effff700 1 -- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f0072230 msgr2=0x7f19f015ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19effff700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f0072230 0x7f19f015ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19effff700 1 -- 172.21.15.112:0/2528030414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc0094a0 con 0x7f19f0146bb0 2023-08-27T11:25:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19effff700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f0146bb0 0x7f19f0144000 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f19dc014040 tx=0x7f19dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19ed7fa700 1 -- 172.21.15.112:0/2528030414 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc0036a0 con 0x7f19f0146bb0 2023-08-27T11:25:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19ed7fa700 1 -- 172.21.15.112:0/2528030414 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc01f070 con 0x7f19f0146bb0 2023-08-27T11:25:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19f6df5700 1 -- 172.21.15.112:0/2528030414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f013d3f0 con 0x7f19f0146bb0 2023-08-27T11:25:52.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19ed7fa700 1 -- 172.21.15.112:0/2528030414 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc003a00 con 0x7f19f0146bb0 2023-08-27T11:25:52.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.043+0000 7f19f6df5700 1 -- 172.21.15.112:0/2528030414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f0144540 con 0x7f19f0146bb0 2023-08-27T11:25:52.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.044+0000 7f19f6df5700 1 -- 172.21.15.112:0/2528030414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f013dbb0 con 0x7f19f0146bb0 2023-08-27T11:25:52.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.045+0000 7f19ed7fa700 1 -- 172.21.15.112:0/2528030414 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f19dc003b60 con 0x7f19f0146bb0 2023-08-27T11:25:52.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.045+0000 7f19ed7fa700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f19d406a320 0x7f19d406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.045+0000 7f19ed7fa700 1 -- 172.21.15.112:0/2528030414 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f19dc021070 con 0x7f19f0146bb0 2023-08-27T11:25:52.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.045+0000 7f19f4b91700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f19d406a320 0x7f19d406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.046+0000 7f19f4b91700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f19d406a320 0x7f19d406c7e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f19f01f6010 tx=0x7f19e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.049+0000 7f19ed7fa700 1 -- 172.21.15.112:0/2528030414 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f19dc055620 con 0x7f19f0146bb0 2023-08-27T11:25:52.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.277+0000 7f19f6df5700 1 -- 172.21.15.112:0/2528030414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19f0067a20 con 0x7f19f0146bb0 2023-08-27T11:25:52.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.277+0000 7f19ed7fa700 1 -- 172.21.15.112:0/2528030414 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4889 (secure 0 0 0) 0x7f19dc058c40 con 0x7f19f0146bb0 2023-08-27T11:25:52.278 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:52.279 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6166,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3262556844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3262556844},{"type":"v1","addr":"172.21.15.138:6836","nonce":3262556844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2232724696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2232724696},{"type":"v1","addr":"172.21.15.138:6837","nonce":2232724696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6570,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1511159838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1511159838},{"type":"v1","addr":"172.21.15.112:6834","nonce":1511159838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:25:41.794717+0000","modified":"2023-08-27T11:25:45.437236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2318724344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2318724344},{"type":"v1","addr":"172.21.15.112:6835","nonce":2318724344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-27T11:25:52.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.280+0000 7f19f6df5700 1 -- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f19d406a320 msgr2=0x7f19d406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.280+0000 7f19f6df5700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f19d406a320 0x7f19d406c7e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f19f01f6010 tx=0x7f19e4008040 comp rx=0 tx=0).stop 2023-08-27T11:25:52.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.280+0000 7f19f6df5700 1 -- 172.21.15.112:0/2528030414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f0146bb0 msgr2=0x7f19f0144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.280+0000 7f19f6df5700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f0146bb0 0x7f19f0144000 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f19dc014040 tx=0x7f19dc004750 comp rx=0 tx=0).stop 2023-08-27T11:25:52.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.280+0000 7f19f6df5700 1 -- 172.21.15.112:0/2528030414 shutdown_connections 2023-08-27T11:25:52.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.280+0000 7f19f6df5700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f19d406a320 0x7f19d406c7e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.280+0000 7f19f6df5700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f0146f90 0x7f19f00d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.280+0000 7f19f6df5700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f0072230 0x7f19f015ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.280+0000 7f19f6df5700 1 --2- 172.21.15.112:0/2528030414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f0146bb0 0x7f19f0144000 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.280+0000 7f19f6df5700 1 -- 172.21.15.112:0/2528030414 >> 172.21.15.112:0/2528030414 conn(0x7f19f006f970 msgr2=0x7f19f01607c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:52.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.280+0000 7f19f6df5700 1 -- 172.21.15.112:0/2528030414 shutdown_connections 2023-08-27T11:25:52.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.280+0000 7f19f6df5700 1 -- 172.21.15.112:0/2528030414 wait complete. 2023-08-27T11:25:52.283 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 48 2023-08-27T11:25:52.695 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:25:41.794717+0000', 'modified': '2023-08-27T11:25:45.437236+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6573}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6573': {'gid': 6573, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/2318724344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 2318724344}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 2318724344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:25:52.695 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:25:52.696 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-27T11:25:52.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.818+0000 7f0095284700 1 Processor -- start 2023-08-27T11:25:52.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.818+0000 7f0095284700 1 -- start start 2023-08-27T11:25:52.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.819+0000 7f0095284700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f00900b5470 0x7f00900b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.819+0000 7f0095284700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00900b3b70 con 0x7f00901619b0 2023-08-27T11:25:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.819+0000 7f0095284700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00900b3e50 con 0x7f0090146be0 2023-08-27T11:25:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.819+0000 7f0095284700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00900b3980 con 0x7f00900b5470 2023-08-27T11:25:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.819+0000 7f008e7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f00900b5470 0x7f00900b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.819+0000 7f008f7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f00901619b0 0x7f00900cfa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54552/0 (socket says 172.21.15.112:54552) 2023-08-27T11:25:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.819+0000 7f008f7fe700 1 -- 172.21.15.112:0/1911798547 learned_addr learned my addr 172.21.15.112:0/1911798547 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:52.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.819+0000 7f008e7fc700 1 -- 172.21.15.112:0/1911798547 >> v1:172.21.15.138:6789/0 conn(0x7f0090146be0 legacy=0x7f00900b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:52.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.819+0000 7f008e7fc700 1 -- 172.21.15.112:0/1911798547 >> v1:172.21.15.112:6789/0 conn(0x7f00901619b0 legacy=0x7f00900cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.819+0000 7f008e7fc700 1 -- 172.21.15.112:0/1911798547 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00900b4130 con 0x7f00900b5470 2023-08-27T11:25:52.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.819+0000 7f008dffb700 1 -- 172.21.15.112:0/1911798547 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4260183084 0 0) 0x7f00900b3b70 con 0x7f00901619b0 2023-08-27T11:25:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.819+0000 7f008e7fc700 1 --2- 172.21.15.112:0/1911798547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00900b5470 0x7f00900b3440 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0084009840 tx=0x7f0084009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8a74a2a3b5e28b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.819+0000 7f008dffb700 1 -- 172.21.15.112:0/1911798547 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084011830 con 0x7f00900b5470 2023-08-27T11:25:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.820+0000 7f008dffb700 1 -- 172.21.15.112:0/1911798547 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0084011990 con 0x7f00900b5470 2023-08-27T11:25:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.820+0000 7f008dffb700 1 -- 172.21.15.112:0/1911798547 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084011d20 con 0x7f00900b5470 2023-08-27T11:25:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.820+0000 7f0095284700 1 -- 172.21.15.112:0/1911798547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00900b5470 msgr2=0x7f00900b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.820+0000 7f0095284700 1 --2- 172.21.15.112:0/1911798547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00900b5470 0x7f00900b3440 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0084009840 tx=0x7f0084009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:52.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.820+0000 7f0095284700 1 -- 172.21.15.112:0/1911798547 shutdown_connections 2023-08-27T11:25:52.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.820+0000 7f0095284700 1 --2- 172.21.15.112:0/1911798547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00900b5470 0x7f00900b3440 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.820+0000 7f0095284700 1 -- 172.21.15.112:0/1911798547 >> 172.21.15.112:0/1911798547 conn(0x7f009006c880 msgr2=0x7f00900717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:52.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.820+0000 7f0095284700 1 -- 172.21.15.112:0/1911798547 shutdown_connections 2023-08-27T11:25:52.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.820+0000 7f0095284700 1 -- 172.21.15.112:0/1911798547 wait complete. 2023-08-27T11:25:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.821+0000 7f0095284700 1 Processor -- start 2023-08-27T11:25:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.821+0000 7f0095284700 1 -- start start 2023-08-27T11:25:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f0095284700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00900b5470 0x7f0090141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f0095284700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0090146be0 0x7f009013c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f0095284700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00901619b0 0x7f0090143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f0095284700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00900715e0 con 0x7f00900b5470 2023-08-27T11:25:52.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f0095284700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009013c550 con 0x7f00901619b0 2023-08-27T11:25:52.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f0095284700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009013c7d0 con 0x7f0090146be0 2023-08-27T11:25:52.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f008effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00900b5470 0x7f0090141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f008effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00900b5470 0x7f0090141930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60172/0 (socket says 172.21.15.112:60172) 2023-08-27T11:25:52.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f008effd700 1 -- 172.21.15.112:0/1120724800 learned_addr learned my addr 172.21.15.112:0/1120724800 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:52.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f008effd700 1 -- 172.21.15.112:0/1120724800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0090146be0 msgr2=0x7f009013c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f008effd700 1 --2- 172.21.15.112:0/1120724800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0090146be0 0x7f009013c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f008effd700 1 -- 172.21.15.112:0/1120724800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00901619b0 msgr2=0x7f0090143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f008effd700 1 --2- 172.21.15.112:0/1120724800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00901619b0 0x7f0090143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.822+0000 7f008effd700 1 -- 172.21.15.112:0/1120724800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00840094a0 con 0x7f00900b5470 2023-08-27T11:25:52.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.823+0000 7f008effd700 1 --2- 172.21.15.112:0/1120724800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00900b5470 0x7f0090141930 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f008000c4e0 tx=0x7f008000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:52.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.823+0000 7f00777fe700 1 -- 172.21.15.112:0/1120724800 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00800052d0 con 0x7f00900b5470 2023-08-27T11:25:52.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.823+0000 7f00777fe700 1 -- 172.21.15.112:0/1120724800 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0080005430 con 0x7f00900b5470 2023-08-27T11:25:52.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.823+0000 7f0095284700 1 -- 172.21.15.112:0/1120724800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009013cab0 con 0x7f00900b5470 2023-08-27T11:25:52.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.823+0000 7f00777fe700 1 -- 172.21.15.112:0/1120724800 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00800038c0 con 0x7f00900b5470 2023-08-27T11:25:52.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.823+0000 7f0095284700 1 -- 172.21.15.112:0/1120724800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009013d070 con 0x7f00900b5470 2023-08-27T11:25:52.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.824+0000 7f00777fe700 1 -- 172.21.15.112:0/1120724800 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f0080003a20 con 0x7f00900b5470 2023-08-27T11:25:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.825+0000 7f00777fe700 1 --2- 172.21.15.112:0/1120724800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f007c06a320 0x7f007c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.825+0000 7f00777fe700 1 -- 172.21.15.112:0/1120724800 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0080087d10 con 0x7f00900b5470 2023-08-27T11:25:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.825+0000 7f008e7fc700 1 --2- 172.21.15.112:0/1120724800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f007c06a320 0x7f007c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.825+0000 7f008e7fc700 1 --2- 172.21.15.112:0/1120724800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f007c06a320 0x7f007c06c7e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f00900d1ce0 tx=0x7f0084010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.827+0000 7f00a1446b80 1 Processor -- start 2023-08-27T11:25:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.827+0000 7f00a1446b80 1 -- start start 2023-08-27T11:25:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.827+0000 7f00a1446b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 0x562418ad1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.827+0000 7f00a1446b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ae2960 0x562418acdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f00a1446b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 0x562418ace0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f00a1446b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624189ff730 con 0x562418ad0f40 2023-08-27T11:25:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f00a1446b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624189fd050 con 0x562418ae2960 2023-08-27T11:25:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f00a1446b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624189f8940 con 0x562418aa43d0 2023-08-27T11:25:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f008effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 0x562418ad1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f008effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 0x562418ad1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60180/0 (socket says 172.21.15.112:60180) 2023-08-27T11:25:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f008effd700 1 -- 172.21.15.112:0/2273936125 learned_addr learned my addr 172.21.15.112:0/2273936125 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f008e7fc700 1 --2- 172.21.15.112:0/2273936125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ae2960 0x562418acdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f008f7fe700 1 --2- 172.21.15.112:0/2273936125 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 0x562418ace0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f008effd700 1 -- 172.21.15.112:0/2273936125 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 msgr2=0x562418ace0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:52.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f008effd700 1 --2- 172.21.15.112:0/2273936125 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 0x562418ace0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f008effd700 1 -- 172.21.15.112:0/2273936125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ae2960 msgr2=0x562418acdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:52.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f008effd700 1 --2- 172.21.15.112:0/2273936125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ae2960 0x562418acdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f008effd700 1 -- 172.21.15.112:0/2273936125 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624188785e0 con 0x562418ad0f40 2023-08-27T11:25:52.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f008effd700 1 --2- 172.21.15.112:0/2273936125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 0x562418ad1320 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f008000cca0 tx=0x7f00800568a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f88a9e96c8456 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:52.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f0074ff9700 1 -- 172.21.15.112:0/2273936125 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00800575b0 con 0x562418ad0f40 2023-08-27T11:25:52.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f0074ff9700 1 -- 172.21.15.112:0/2273936125 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008005ac80 con 0x562418ad0f40 2023-08-27T11:25:52.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.828+0000 7f0074ff9700 1 -- 172.21.15.112:0/2273936125 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008005e940 con 0x562418ad0f40 2023-08-27T11:25:52.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.829+0000 7f00a1446b80 1 -- 172.21.15.112:0/2273936125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 msgr2=0x562418ad1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.829+0000 7f00a1446b80 1 --2- 172.21.15.112:0/2273936125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 0x562418ad1320 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f008000cca0 tx=0x7f00800568a0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.829+0000 7f00a1446b80 1 -- 172.21.15.112:0/2273936125 shutdown_connections 2023-08-27T11:25:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.829+0000 7f00a1446b80 1 --2- 172.21.15.112:0/2273936125 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 0x562418ace0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.829+0000 7f00a1446b80 1 --2- 172.21.15.112:0/2273936125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 0x562418ad1320 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.829+0000 7f00a1446b80 1 --2- 172.21.15.112:0/2273936125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ae2960 0x562418acdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.829+0000 7f00a1446b80 1 -- 172.21.15.112:0/2273936125 >> 172.21.15.112:0/2273936125 conn(0x562418a04c30 msgr2=0x562418aebd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:52.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.829+0000 7f00a1446b80 1 -- 172.21.15.112:0/2273936125 shutdown_connections 2023-08-27T11:25:52.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.829+0000 7f00a1446b80 1 -- 172.21.15.112:0/2273936125 wait complete. 2023-08-27T11:25:52.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.830+0000 7f00a1446b80 1 -- start start 2023-08-27T11:25:52.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.830+0000 7f00a1446b80 1 Processor -- start 2023-08-27T11:25:52.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f00a1446b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418aa43d0 0x562418aefb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f00a1446b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ad0f40 0x562418aed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f00a1446b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418ae2960 0x562418aed710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f00a1446b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x562418996b60 con 0x562418aa43d0 2023-08-27T11:25:52.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f00a1446b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562418996860 con 0x562418ad0f40 2023-08-27T11:25:52.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f00a1446b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x56241887a8d0 con 0x562418ae2960 2023-08-27T11:25:52.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f008effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418aa43d0 0x562418aefb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f00a1446b80 10 client.0 _ll_register_callbacks cb 0x562418994630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:25:52.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f008effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418aa43d0 0x562418aefb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60186/0 (socket says 172.21.15.112:60186) 2023-08-27T11:25:52.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f008effd700 1 -- 172.21.15.112:0/3518893437 learned_addr learned my addr 172.21.15.112:0/3518893437 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:52.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f008f7fe700 1 --2- 172.21.15.112:0/3518893437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418ae2960 0x562418aed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f008e7fc700 1 --2- 172.21.15.112:0/3518893437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ad0f40 0x562418aed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f008effd700 1 -- 172.21.15.112:0/3518893437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418ae2960 msgr2=0x562418aed710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f008effd700 1 --2- 172.21.15.112:0/3518893437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418ae2960 0x562418aed710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f008effd700 1 -- 172.21.15.112:0/3518893437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ad0f40 msgr2=0x562418aed1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f008effd700 1 --2- 172.21.15.112:0/3518893437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ad0f40 0x562418aed1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f008effd700 1 -- 172.21.15.112:0/3518893437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009013d070 con 0x562418aa43d0 2023-08-27T11:25:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f008effd700 1 --2- 172.21.15.112:0/3518893437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418aa43d0 0x562418aefb30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0080056990 tx=0x7f0080014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b910ac4907cca4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.831+0000 7f006dffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:25:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f006dffb700 1 -- 172.21.15.112:0/3518893437 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080052cb0 con 0x562418aa43d0 2023-08-27T11:25:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f006dffb700 1 -- 172.21.15.112:0/3518893437 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0080052e10 con 0x562418aa43d0 2023-08-27T11:25:52.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f00a1446b80 1 -- 172.21.15.112:0/3518893437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562418a994f0 con 0x562418aa43d0 2023-08-27T11:25:52.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f006dffb700 1 -- 172.21.15.112:0/3518893437 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0080052ff0 con 0x562418aa43d0 2023-08-27T11:25:52.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f00a1446b80 10 client.6738 fetch_fsmap learned FSMap version 48 2023-08-27T11:25:52.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f00a1446b80 1 -- 172.21.15.112:0/3518893437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x562418a10750 con 0x562418aa43d0 2023-08-27T11:25:52.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f006dffb700 1 -- 172.21.15.112:0/3518893437 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562418a994f0 con 0x562418aa43d0 2023-08-27T11:25:52.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f006dffb700 1 -- 172.21.15.112:0/3518893437 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f00800751a0 con 0x562418aa43d0 2023-08-27T11:25:52.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f00a1446b80 10 client.6738 fetch_fsmap finished waiting for FSMap version 48 2023-08-27T11:25:52.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f00a1446b80 10 client.6738 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:25:52.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f00a1446b80 20 client.6738 populate_metadata read hostname 'smithi112' 2023-08-27T11:25:52.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f00a1446b80 1 --2- 172.21.15.112:0/3518893437 >> [v2:172.21.15.112:6833/2318724344,v1:172.21.15.112:6835/2318724344] conn(0x562418ab9a40 0x562418ab9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f00a1446b80 4 client.6738 mds_command: new command op to 6573 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:25:52.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f00a1446b80 1 -- 172.21.15.112:0/3518893437 --> [v2:172.21.15.112:6833/2318724344,v1:172.21.15.112:6835/2318724344] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624189f1920 con 0x562418ab9a40 2023-08-27T11:25:52.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f008e7fc700 1 --2- 172.21.15.112:0/3518893437 >> [v2:172.21.15.112:6833/2318724344,v1:172.21.15.112:6835/2318724344] conn(0x562418ab9a40 0x562418ab9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f006dffb700 10 client.6738 ms_handle_connect on v2:172.21.15.112:6833/2318724344 2023-08-27T11:25:52.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.832+0000 7f008e7fc700 1 --2- 172.21.15.112:0/3518893437 >> [v2:172.21.15.112:6833/2318724344,v1:172.21.15.112:6835/2318724344] conn(0x562418ab9a40 0x562418ab9e20 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85f4a394686e9b60 server_cookie=92379a883828e67c in_seq=0 out_seq=0 2023-08-27T11:25:52.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f006dffb700 1 -- 172.21.15.112:0/3518893437 <== mds.0 v2:172.21.15.112:6833/2318724344 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5624189f1920 con 0x562418ab9a40 2023-08-27T11:25:52.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f006dffb700 10 client.6738 handle_command_reply: tid=0 2023-08-27T11:25:52.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f00a1446b80 1 client.6738 shutdown 2023-08-27T11:25:52.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f00a1446b80 1 -- 172.21.15.112:0/3518893437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418aa43d0 msgr2=0x562418aefb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f00a1446b80 1 --2- 172.21.15.112:0/3518893437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418aa43d0 0x562418aefb30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0080056990 tx=0x7f0080014070 comp rx=0 tx=0).stop 2023-08-27T11:25:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f00a1446b80 1 -- 172.21.15.112:0/3518893437 shutdown_connections 2023-08-27T11:25:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f00a1446b80 1 --2- 172.21.15.112:0/3518893437 >> [v2:172.21.15.112:6833/2318724344,v1:172.21.15.112:6835/2318724344] conn(0x562418ab9a40 0x562418ab9e20 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f00a1446b80 1 --2- 172.21.15.112:0/3518893437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418ae2960 0x562418aed710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f00a1446b80 1 --2- 172.21.15.112:0/3518893437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418aa43d0 0x562418aefb30 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f006dffb700 0 client.6738 ms_handle_reset on v2:172.21.15.112:6833/2318724344 2023-08-27T11:25:52.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f00a1446b80 1 --2- 172.21.15.112:0/3518893437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ad0f40 0x562418aed1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f00a1446b80 1 -- 172.21.15.112:0/3518893437 >> 172.21.15.112:0/3518893437 conn(0x562418a04c30 msgr2=0x562418acaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:52.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f00a1446b80 1 -- 172.21.15.112:0/3518893437 shutdown_connections 2023-08-27T11:25:52.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f00a1446b80 1 -- 172.21.15.112:0/3518893437 wait complete. 2023-08-27T11:25:52.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.833+0000 7f00a1446b80 20 client.6738 trim_cache size 0 max 16384 2023-08-27T11:25:52.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.851+0000 7f00a1446b80 1 Processor -- start 2023-08-27T11:25:52.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.851+0000 7f00a1446b80 1 -- start start 2023-08-27T11:25:52.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.851+0000 7f00a1446b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 0x562418aeb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.851+0000 7f00a1446b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ad0f40 0x562418aead80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.851+0000 7f00a1446b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ae2960 0x562418ad4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.851+0000 7f00a1446b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x562418ac8cb0 con 0x562418ae2960 2023-08-27T11:25:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.851+0000 7f00a1446b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562418b0d2c0 con 0x562418ad0f40 2023-08-27T11:25:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.851+0000 7f00a1446b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x562418acaa80 con 0x562418aa43d0 2023-08-27T11:25:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.851+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ae2960 0x562418ad4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.852+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ae2960 0x562418ad4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60198/0 (socket says 172.21.15.112:60198) 2023-08-27T11:25:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.852+0000 7f008f7fe700 1 -- 172.21.15.112:0/2830090379 learned_addr learned my addr 172.21.15.112:0/2830090379 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.852+0000 7f008f7fe700 1 -- 172.21.15.112:0/2830090379 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 msgr2=0x562418aeb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.852+0000 7f008f7fe700 1 --2- 172.21.15.112:0/2830090379 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 0x562418aeb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.852+0000 7f008f7fe700 1 -- 172.21.15.112:0/2830090379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ad0f40 msgr2=0x562418aead80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.852+0000 7f008f7fe700 1 --2- 172.21.15.112:0/2830090379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ad0f40 0x562418aead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.852+0000 7f008f7fe700 1 -- 172.21.15.112:0/2830090379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562418824d90 con 0x562418ae2960 2023-08-27T11:25:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.852+0000 7f008f7fe700 1 --2- 172.21.15.112:0/2830090379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ae2960 0x562418ad4920 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0088002210 tx=0x7f0088001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9135f87e97c4f08f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:52.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.852+0000 7f00757fa700 1 -- 172.21.15.112:0/2830090379 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088015470 con 0x562418ae2960 2023-08-27T11:25:52.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.852+0000 7f00757fa700 1 -- 172.21.15.112:0/2830090379 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00880155d0 con 0x562418ae2960 2023-08-27T11:25:52.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.852+0000 7f00757fa700 1 -- 172.21.15.112:0/2830090379 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088015960 con 0x562418ae2960 2023-08-27T11:25:52.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.852+0000 7f00a1446b80 1 -- 172.21.15.112:0/2830090379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ae2960 msgr2=0x562418ad4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.852+0000 7f00a1446b80 1 --2- 172.21.15.112:0/2830090379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ae2960 0x562418ad4920 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0088002210 tx=0x7f0088001a10 comp rx=0 tx=0).stop 2023-08-27T11:25:52.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.853+0000 7f00a1446b80 1 -- 172.21.15.112:0/2830090379 shutdown_connections 2023-08-27T11:25:52.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.853+0000 7f00a1446b80 1 --2- 172.21.15.112:0/2830090379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ae2960 0x562418ad4920 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.853+0000 7f00a1446b80 1 --2- 172.21.15.112:0/2830090379 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 0x562418aeb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.853+0000 7f00a1446b80 1 --2- 172.21.15.112:0/2830090379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ad0f40 0x562418aead80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.853+0000 7f00a1446b80 1 -- 172.21.15.112:0/2830090379 >> 172.21.15.112:0/2830090379 conn(0x562418a04c30 msgr2=0x562418ab9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:52.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.853+0000 7f00a1446b80 1 -- 172.21.15.112:0/2830090379 shutdown_connections 2023-08-27T11:25:52.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.853+0000 7f00a1446b80 1 -- 172.21.15.112:0/2830090379 wait complete. 2023-08-27T11:25:52.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.853+0000 7f00a1446b80 1 -- start start 2023-08-27T11:25:52.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.853+0000 7f00a1446b80 1 Processor -- start 2023-08-27T11:25:52.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f00a1446b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 0x562418a9fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f00a1446b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 0x562418ac9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f00a1446b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ae2960 0x562418aead80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f00a1446b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x562418aef600 con 0x562418ad0f40 2023-08-27T11:25:52.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f00a1446b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x562418aef9c0 con 0x562418ae2960 2023-08-27T11:25:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f00a1446b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x562418aa4c60 con 0x562418aa43d0 2023-08-27T11:25:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f00a1446b80 10 client.0 _ll_register_callbacks cb 0x562418994630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:25:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f008e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 0x562418ac9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f008e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 0x562418ac9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60202/0 (socket says 172.21.15.112:60202) 2023-08-27T11:25:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f008e7fc700 1 -- 172.21.15.112:0/711415950 learned_addr learned my addr 172.21.15.112:0/711415950 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f008effd700 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 0x562418a9fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f008f7fe700 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ae2960 0x562418aead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f008e7fc700 1 -- 172.21.15.112:0/711415950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 msgr2=0x562418a9fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f008e7fc700 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 0x562418a9fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f008e7fc700 1 -- 172.21.15.112:0/711415950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ae2960 msgr2=0x562418aead80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f008e7fc700 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ae2960 0x562418aead80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f008e7fc700 1 -- 172.21.15.112:0/711415950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0088001ef0 con 0x562418ad0f40 2023-08-27T11:25:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f008f7fe700 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ae2960 0x562418aead80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f008e7fc700 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 0x562418ac9b60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0084013000 tx=0x7f0084004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d53eef24d8b861c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:52.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.854+0000 7f006dffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:25:52.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f006dffb700 1 -- 172.21.15.112:0/711415950 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084033270 con 0x562418ad0f40 2023-08-27T11:25:52.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f006dffb700 1 -- 172.21.15.112:0/711415950 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0084033e40 con 0x562418ad0f40 2023-08-27T11:25:52.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f00a1446b80 1 -- 172.21.15.112:0/711415950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f008005ac80 con 0x562418ad0f40 2023-08-27T11:25:52.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f006dffb700 1 -- 172.21.15.112:0/711415950 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00840333d0 con 0x562418ad0f40 2023-08-27T11:25:52.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f00a1446b80 10 client.6744 fetch_fsmap learned FSMap version 48 2023-08-27T11:25:52.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f00a1446b80 1 -- 172.21.15.112:0/711415950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x56241880dc10 con 0x562418ad0f40 2023-08-27T11:25:52.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f006dffb700 1 -- 172.21.15.112:0/711415950 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008005ac80 con 0x562418ad0f40 2023-08-27T11:25:52.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f006dffb700 1 -- 172.21.15.112:0/711415950 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f008403f430 con 0x562418ad0f40 2023-08-27T11:25:52.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f00a1446b80 10 client.6744 fetch_fsmap finished waiting for FSMap version 48 2023-08-27T11:25:52.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f00a1446b80 10 client.6744 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:25:52.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f00a1446b80 20 client.6744 populate_metadata read hostname 'smithi112' 2023-08-27T11:25:52.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f00a1446b80 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.112:6833/2318724344,v1:172.21.15.112:6835/2318724344] conn(0x562418aeb2c0 0x562418ad3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:52.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f00a1446b80 4 client.6744 mds_command: new command op to 6573 tid=0[{"prefix": "status"}] 2023-08-27T11:25:52.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f00a1446b80 1 -- 172.21.15.112:0/711415950 --> [v2:172.21.15.112:6833/2318724344,v1:172.21.15.112:6835/2318724344] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5624189b2000 con 0x562418aeb2c0 2023-08-27T11:25:52.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f008effd700 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.112:6833/2318724344,v1:172.21.15.112:6835/2318724344] conn(0x562418aeb2c0 0x562418ad3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:52.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f006dffb700 10 client.6744 ms_handle_connect on v2:172.21.15.112:6833/2318724344 2023-08-27T11:25:52.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.855+0000 7f008effd700 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.112:6833/2318724344,v1:172.21.15.112:6835/2318724344] conn(0x562418aeb2c0 0x562418ad3330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d184de1f8e15bb02 server_cookie=35879585dab584f1 in_seq=0 out_seq=0 2023-08-27T11:25:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f006dffb700 1 -- 172.21.15.112:0/711415950 <== mds.0 v2:172.21.15.112:6833/2318724344 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5624189b2000 con 0x562418aeb2c0 2023-08-27T11:25:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f006dffb700 10 client.6744 handle_command_reply: tid=0 2023-08-27T11:25:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f00a1446b80 1 client.6744 shutdown 2023-08-27T11:25:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f00a1446b80 1 -- 172.21.15.112:0/711415950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 msgr2=0x562418ac9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f00a1446b80 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 0x562418ac9b60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0084013000 tx=0x7f0084004750 comp rx=0 tx=0).stop 2023-08-27T11:25:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f00a1446b80 1 -- 172.21.15.112:0/711415950 shutdown_connections 2023-08-27T11:25:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f00a1446b80 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.112:6833/2318724344,v1:172.21.15.112:6835/2318724344] conn(0x562418aeb2c0 0x562418ad3330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f00a1446b80 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562418ae2960 0x562418aead80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f006dffb700 0 client.6744 ms_handle_reset on v2:172.21.15.112:6833/2318724344 2023-08-27T11:25:52.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f00a1446b80 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562418aa43d0 0x562418a9fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f00a1446b80 1 --2- 172.21.15.112:0/711415950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562418ad0f40 0x562418ac9b60 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f00a1446b80 1 -- 172.21.15.112:0/711415950 >> 172.21.15.112:0/711415950 conn(0x562418a04c30 msgr2=0x562418ac66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:52.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f00a1446b80 1 -- 172.21.15.112:0/711415950 shutdown_connections 2023-08-27T11:25:52.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f00a1446b80 1 -- 172.21.15.112:0/711415950 wait complete. 2023-08-27T11:25:52.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f00a1446b80 20 client.6744 trim_cache size 0 max 16384 2023-08-27T11:25:52.884 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:25:52.884 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e099a24f-c1cd-4f11-ac4f-ecacc10e5e98", 2023-08-27T11:25:52.885 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2023-08-27T11:25:52.885 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 6573, 2023-08-27T11:25:52.885 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2023-08-27T11:25:52.885 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2023-08-27T11:25:52.886 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2023-08-27T11:25:52.886 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 10.434445423, 2023-08-27T11:25:52.886 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 48, 2023-08-27T11:25:52.886 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 77, 2023-08-27T11:25:52.886 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 77, 2023-08-27T11:25:52.887 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 17.399490377999999 2023-08-27T11:25:52.887 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:25:52.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f0095284700 1 -- 172.21.15.112:0/1120724800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f007c06a320 msgr2=0x7f007c06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f0095284700 1 --2- 172.21.15.112:0/1120724800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f007c06a320 0x7f007c06c7e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f00900d1ce0 tx=0x7f0084010000 comp rx=0 tx=0).stop 2023-08-27T11:25:52.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f0095284700 1 -- 172.21.15.112:0/1120724800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00900b5470 msgr2=0x7f0090141930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:52.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f0095284700 1 --2- 172.21.15.112:0/1120724800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00900b5470 0x7f0090141930 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f008000c4e0 tx=0x7f008000c8a0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f0095284700 1 -- 172.21.15.112:0/1120724800 shutdown_connections 2023-08-27T11:25:52.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f0095284700 1 --2- 172.21.15.112:0/1120724800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f007c06a320 0x7f007c06c7e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f0095284700 1 --2- 172.21.15.112:0/1120724800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00901619b0 0x7f0090143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f0095284700 1 --2- 172.21.15.112:0/1120724800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00900b5470 0x7f0090141930 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f0095284700 1 --2- 172.21.15.112:0/1120724800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0090146be0 0x7f009013c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:52.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f0095284700 1 -- 172.21.15.112:0/1120724800 >> 172.21.15.112:0/1120724800 conn(0x7f009006c880 msgr2=0x7f0090144550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:52.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.856+0000 7f0095284700 1 -- 172.21.15.112:0/1120724800 shutdown_connections 2023-08-27T11:25:52.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:52.857+0000 7f0095284700 1 -- 172.21.15.112:0/1120724800 wait complete. 2023-08-27T11:25:52.893 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:25:53.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.012+0000 7f07132bd700 1 Processor -- start 2023-08-27T11:25:53.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.012+0000 7f07132bd700 1 -- start start 2023-08-27T11:25:53.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.013+0000 7f07132bd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f070c15a120 0x7f070c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:53.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.013+0000 7f07132bd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f070c06da40 0x7f070c0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:53.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.013+0000 7f07132bd700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f070c06deb0 con 0x7f070c06d5b0 2023-08-27T11:25:53.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.013+0000 7f07132bd700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070c15b540 con 0x7f070c15a120 2023-08-27T11:25:53.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.013+0000 7f07132bd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0af830 con 0x7f070c06da40 2023-08-27T11:25:53.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.013+0000 7f0711059700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f070c15a120 0x7f070c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:53.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.013+0000 7f0711059700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f070c15a120 0x7f070c0b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60208/0 (socket says 172.21.15.112:60208) 2023-08-27T11:25:53.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.013+0000 7f0711059700 1 -- 172.21.15.112:0/2956735037 learned_addr learned my addr 172.21.15.112:0/2956735037 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:53.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.013+0000 7f0710858700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f070c06d5b0 0x7f070c0b3440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54590/0 (socket says 172.21.15.112:54590) 2023-08-27T11:25:53.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.013+0000 7f071185a700 1 --2- 172.21.15.112:0/2956735037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f070c06da40 0x7f070c0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:53.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.013+0000 7f0711059700 1 -- 172.21.15.112:0/2956735037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f070c06da40 msgr2=0x7f070c0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:53.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f0703fff700 1 -- 172.21.15.112:0/2956735037 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4095845123 0 0) 0x7f070c06deb0 con 0x7f070c06d5b0 2023-08-27T11:25:53.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f0711059700 1 --2- 172.21.15.112:0/2956735037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f070c06da40 0x7f070c0b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:53.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f0711059700 1 -- 172.21.15.112:0/2956735037 >> v1:172.21.15.112:6789/0 conn(0x7f070c06d5b0 legacy=0x7f070c0b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:53.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f0711059700 1 -- 172.21.15.112:0/2956735037 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c0afb10 con 0x7f070c15a120 2023-08-27T11:25:53.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f071185a700 1 --2- 172.21.15.112:0/2956735037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f070c06da40 0x7f070c0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:53.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f0711059700 1 --2- 172.21.15.112:0/2956735037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f070c15a120 0x7f070c0b05f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00bb90 tx=0x7f06fc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c5e3c00cbfc29f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:53.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f0703fff700 1 -- 172.21.15.112:0/2956735037 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc00dd50 con 0x7f070c15a120 2023-08-27T11:25:53.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f0703fff700 1 -- 172.21.15.112:0/2956735037 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06fc010470 con 0x7f070c15a120 2023-08-27T11:25:53.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f0703fff700 1 -- 172.21.15.112:0/2956735037 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc010800 con 0x7f070c15a120 2023-08-27T11:25:53.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f07132bd700 1 -- 172.21.15.112:0/2956735037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f070c15a120 msgr2=0x7f070c0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:53.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f07132bd700 1 --2- 172.21.15.112:0/2956735037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f070c15a120 0x7f070c0b05f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00bb90 tx=0x7f06fc00bea0 comp rx=0 tx=0).stop 2023-08-27T11:25:53.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f07132bd700 1 -- 172.21.15.112:0/2956735037 shutdown_connections 2023-08-27T11:25:53.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f07132bd700 1 --2- 172.21.15.112:0/2956735037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f070c06da40 0x7f070c0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:53.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f07132bd700 1 --2- 172.21.15.112:0/2956735037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f070c15a120 0x7f070c0b05f0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:53.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.014+0000 7f07132bd700 1 -- 172.21.15.112:0/2956735037 >> 172.21.15.112:0/2956735037 conn(0x7f070c0cff80 msgr2=0x7f070c0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:53.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.015+0000 7f07132bd700 1 -- 172.21.15.112:0/2956735037 shutdown_connections 2023-08-27T11:25:53.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.015+0000 7f07132bd700 1 -- 172.21.15.112:0/2956735037 wait complete. 2023-08-27T11:25:53.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.016+0000 7f07132bd700 1 Processor -- start 2023-08-27T11:25:53.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.016+0000 7f07132bd700 1 -- start start 2023-08-27T11:25:53.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.016+0000 7f07132bd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f070c06d5b0 0x7f070c148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:53.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.016+0000 7f07132bd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f070c06da40 0x7f070c0ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:53.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.016+0000 7f07132bd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f070c15a120 0x7f070c153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:53.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.016+0000 7f07132bd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0c9500 con 0x7f070c15a120 2023-08-27T11:25:53.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.016+0000 7f07132bd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0c97e0 con 0x7f070c06da40 2023-08-27T11:25:53.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.016+0000 7f07132bd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0c9ac0 con 0x7f070c06d5b0 2023-08-27T11:25:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f071185a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f070c15a120 0x7f070c153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f071185a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f070c15a120 0x7f070c153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60218/0 (socket says 172.21.15.112:60218) 2023-08-27T11:25:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f071185a700 1 -- 172.21.15.112:0/203120975 learned_addr learned my addr 172.21.15.112:0/203120975 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f0711059700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f070c06d5b0 0x7f070c148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f0710858700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f070c06da40 0x7f070c0ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f071185a700 1 -- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f070c06d5b0 msgr2=0x7f070c148cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:53.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f071185a700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f070c06d5b0 0x7f070c148cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:53.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f071185a700 1 -- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f070c06da40 msgr2=0x7f070c0ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:53.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f071185a700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f070c06da40 0x7f070c0ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:53.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f071185a700 1 -- 172.21.15.112:0/203120975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc00b7f0 con 0x7f070c15a120 2023-08-27T11:25:53.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f0711059700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f070c06d5b0 0x7f070c148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:53.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f071185a700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f070c15a120 0x7f070c153730 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f070400cef0 tx=0x7f070400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:53.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f0701ffb700 1 -- 172.21.15.112:0/203120975 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070400ae50 con 0x7f070c15a120 2023-08-27T11:25:53.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f0701ffb700 1 -- 172.21.15.112:0/203120975 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0704015440 con 0x7f070c15a120 2023-08-27T11:25:53.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f07132bd700 1 -- 172.21.15.112:0/203120975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070c0c9da0 con 0x7f070c15a120 2023-08-27T11:25:53.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f0701ffb700 1 -- 172.21.15.112:0/203120975 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07040157f0 con 0x7f070c15a120 2023-08-27T11:25:53.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.017+0000 7f07132bd700 1 -- 172.21.15.112:0/203120975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070c0ca360 con 0x7f070c15a120 2023-08-27T11:25:53.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.018+0000 7f07132bd700 1 -- 172.21.15.112:0/203120975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070c0ca580 con 0x7f070c15a120 2023-08-27T11:25:53.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.019+0000 7f0701ffb700 1 -- 172.21.15.112:0/203120975 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f070400d430 con 0x7f070c15a120 2023-08-27T11:25:53.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.019+0000 7f0701ffb700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f06f006a320 0x7f06f006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:53.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.019+0000 7f0701ffb700 1 -- 172.21.15.112:0/203120975 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0704088fb0 con 0x7f070c15a120 2023-08-27T11:25:53.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.019+0000 7f0711059700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f06f006a320 0x7f06f006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:53.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.020+0000 7f0711059700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f06f006a320 0x7f06f006c7e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00bee0 tx=0x7f06fc013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:53.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.023+0000 7f0701ffb700 1 -- 172.21.15.112:0/203120975 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f07040bb990 con 0x7f070c15a120 2023-08-27T11:25:53.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.256+0000 7f07132bd700 1 -- 172.21.15.112:0/203120975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f070c067a20 con 0x7f070c15a120 2023-08-27T11:25:53.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.256+0000 7f0701ffb700 1 -- 172.21.15.112:0/203120975 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7f070401a020 con 0x7f070c15a120 2023-08-27T11:25:53.257 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:25:53.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.258+0000 7f07132bd700 1 -- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f06f006a320 msgr2=0x7f06f006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:53.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.258+0000 7f07132bd700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f06f006a320 0x7f06f006c7e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00bee0 tx=0x7f06fc013000 comp rx=0 tx=0).stop 2023-08-27T11:25:53.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.258+0000 7f07132bd700 1 -- 172.21.15.112:0/203120975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f070c15a120 msgr2=0x7f070c153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:53.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.258+0000 7f07132bd700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f070c15a120 0x7f070c153730 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f070400cef0 tx=0x7f070400a340 comp rx=0 tx=0).stop 2023-08-27T11:25:53.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.258+0000 7f07132bd700 1 -- 172.21.15.112:0/203120975 shutdown_connections 2023-08-27T11:25:53.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.258+0000 7f07132bd700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f06f006a320 0x7f06f006c7e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:53.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.258+0000 7f07132bd700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f070c15a120 0x7f070c153730 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:53.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.258+0000 7f07132bd700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f070c06d5b0 0x7f070c148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:53.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.258+0000 7f07132bd700 1 --2- 172.21.15.112:0/203120975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f070c06da40 0x7f070c0ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:53.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.258+0000 7f07132bd700 1 -- 172.21.15.112:0/203120975 >> 172.21.15.112:0/203120975 conn(0x7f070c0cff80 msgr2=0x7f070c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:53.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.259+0000 7f07132bd700 1 -- 172.21.15.112:0/203120975 shutdown_connections 2023-08-27T11:25:53.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:53.259+0000 7f07132bd700 1 -- 172.21.15.112:0/203120975 wait complete. 2023-08-27T11:25:53.271 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:25:53.271 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:25:53.272 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-08-27T11:25:53.272 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:25:53.272 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:25:53.273 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:25:53.273 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:25:53.273 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:25:53.274 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:25:53.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:53.274 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-08-27T11:25:53.291 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:25:53.292 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:25:53.292 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:25:53.292 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:53.292 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:25:53.292 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:53.293 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:25:53.293 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:53.293 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:25:53.293 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:53.294 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:25:53.294 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:53.294 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:25:53.294 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:53.294 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:25:53.295 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 41889sec preferred_lft 41889sec 2023-08-27T11:25:53.295 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8d6c:4fe6:6a84:8222/64 scope link noprefixroute 2023-08-27T11:25:53.295 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:53.295 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:25:53.295 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:53.296 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:25:53.296 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:53.296 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::d87c:a1ff:fe1d:b635/64 scope link 2023-08-27T11:25:53.296 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:53.296 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:25:53.297 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:25:53.297 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::cc14:79ff:feb3:452f/64 scope link 2023-08-27T11:25:53.297 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:53.297 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:25:53.297 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether e6:51:f0:78:04:02 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:25:53.298 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::e451:f0ff:fe78:402/64 scope link 2023-08-27T11:25:53.298 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:53.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:53.299 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:25:53.357 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-08-27T11:25:53.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:53.359 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-08-27T11:25:53.419 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-08-27T11:25:53.419 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:25:53.419 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:25:53.420 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:25:53.420 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:25:53.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:53.421 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:25:53.421 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:25:53.421 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:25:53.422 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:25:53.422 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:25:53.423 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:25:53.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:53.524 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:25:53.541 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:25:53.541 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:25:53.542 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:25:53.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:53.543 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-08-27T11:25:53.608 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:25:53.608 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:25:53.665 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:25:53.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:25:53.666 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:25:53.721 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:25:53.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:53.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:25:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:25:53.807 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:25:53.807 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:25:53.815 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:25:53.816 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:25:53.816 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:25:53.816 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:25:53.816 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:25:53.890 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:25:53.890 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:25:53.891 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:25:53.891 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:25:53.891 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:25:53.891 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:25:53.891 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:25:53.892 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:25:53.892 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:25:53.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:53.893 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:25:53.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:53.926 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:25:54.006 INFO:teuthology.orchestra.run.smithi112.stdout:client5783 2023-08-27T11:25:54.006 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:54.007 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:54.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:25:54.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.223+0000 7fd173ae0700 1 Processor -- start 2023-08-27T11:25:54.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.223+0000 7fd173ae0700 1 -- start start 2023-08-27T11:25:54.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.224+0000 7fd173ae0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd16c0b1400 0x7fd16c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:54.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.224+0000 7fd173ae0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd16c0b17e0 0x7fd16c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:54.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.224+0000 7fd173ae0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd16c161ce0 0x7fd16c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:54.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.224+0000 7fd173ae0700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16c161850 con 0x7fd16c161ce0 2023-08-27T11:25:54.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.224+0000 7fd173ae0700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16c06e7d0 con 0x7fd16c0b17e0 2023-08-27T11:25:54.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.224+0000 7fd173ae0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c0cfc10 con 0x7fd16c0b1400 2023-08-27T11:25:54.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.224+0000 7fd17207d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd16c161ce0 0x7fd16c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:54.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.224+0000 7fd17207d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd16c161ce0 0x7fd16c06f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60228/0 (socket says 172.21.15.112:60228) 2023-08-27T11:25:54.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.224+0000 7fd17207d700 1 -- 172.21.15.112:0/785729072 learned_addr learned my addr 172.21.15.112:0/785729072 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:54.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd17207d700 1 -- 172.21.15.112:0/785729072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd16c0b1400 msgr2=0x7fd16c0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:54.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd17107b700 1 --2- 172.21.15.112:0/785729072 >> v2:172.21.15.138:3300/0 conn(0x7fd16c0b17e0 0x7fd16c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:54.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd17187c700 1 --2- 172.21.15.112:0/785729072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd16c0b1400 0x7fd16c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:54.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd17207d700 1 --2- 172.21.15.112:0/785729072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd16c0b1400 0x7fd16c0cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:54.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd17207d700 1 -- 172.21.15.112:0/785729072 >> v2:172.21.15.138:3300/0 conn(0x7fd16c0b17e0 msgr2=0x7fd16c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:54.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd17207d700 1 --2- 172.21.15.112:0/785729072 >> v2:172.21.15.138:3300/0 conn(0x7fd16c0b17e0 0x7fd16c06e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:54.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd17207d700 1 -- 172.21.15.112:0/785729072 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c161200 con 0x7fd16c161ce0 2023-08-27T11:25:54.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd17207d700 1 --2- 172.21.15.112:0/785729072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd16c161ce0 0x7fd16c06f790 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0d0db0 tx=0x7fd16400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4e10b33f1f7fcf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:54.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd17087a700 1 -- 172.21.15.112:0/785729072 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16400e070 con 0x7fd16c161ce0 2023-08-27T11:25:54.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd17087a700 1 -- 172.21.15.112:0/785729072 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16400b9c0 con 0x7fd16c161ce0 2023-08-27T11:25:54.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd17087a700 1 -- 172.21.15.112:0/785729072 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16400bd70 con 0x7fd16c161ce0 2023-08-27T11:25:54.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd173ae0700 1 -- 172.21.15.112:0/785729072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd16c161ce0 msgr2=0x7fd16c06f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:54.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd173ae0700 1 --2- 172.21.15.112:0/785729072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd16c161ce0 0x7fd16c06f790 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0d0db0 tx=0x7fd16400b330 comp rx=0 tx=0).stop 2023-08-27T11:25:54.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd173ae0700 1 -- 172.21.15.112:0/785729072 shutdown_connections 2023-08-27T11:25:54.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd173ae0700 1 --2- 172.21.15.112:0/785729072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd16c161ce0 0x7fd16c06f790 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd173ae0700 1 --2- 172.21.15.112:0/785729072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd16c0b1400 0x7fd16c0cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd173ae0700 1 --2- 172.21.15.112:0/785729072 >> v2:172.21.15.138:3300/0 conn(0x7fd16c0b17e0 0x7fd16c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.225+0000 7fd173ae0700 1 -- 172.21.15.112:0/785729072 >> 172.21.15.112:0/785729072 conn(0x7fd16c0d0000 msgr2=0x7fd16c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.226+0000 7fd173ae0700 1 -- 172.21.15.112:0/785729072 shutdown_connections 2023-08-27T11:25:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.226+0000 7fd173ae0700 1 -- 172.21.15.112:0/785729072 wait complete. 2023-08-27T11:25:54.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.227+0000 7fd173ae0700 1 Processor -- start 2023-08-27T11:25:54.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.227+0000 7fd173ae0700 1 -- start start 2023-08-27T11:25:54.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd173ae0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd16c0b1400 0x7fd16c0c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd173ae0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd16c0b17e0 0x7fd16c15eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd173ae0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd16c161ce0 0x7fd16c160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd173ae0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c160bc0 con 0x7fd16c0b1400 2023-08-27T11:25:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd173ae0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c0c7390 con 0x7fd16c0b17e0 2023-08-27T11:25:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd173ae0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c15f5e0 con 0x7fd16c161ce0 2023-08-27T11:25:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd17187c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd16c0b1400 0x7fd16c0c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd17187c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd16c0b1400 0x7fd16c0c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60232/0 (socket says 172.21.15.112:60232) 2023-08-27T11:25:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd17187c700 1 -- 172.21.15.112:0/2207806366 learned_addr learned my addr 172.21.15.112:0/2207806366 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd17107b700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd16c0b17e0 0x7fd16c15eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd17207d700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd16c161ce0 0x7fd16c160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:54.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd17187c700 1 -- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd16c161ce0 msgr2=0x7fd16c160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:54.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd17187c700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd16c161ce0 0x7fd16c160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:54.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd17187c700 1 -- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd16c0b17e0 msgr2=0x7fd16c15eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:54.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd17187c700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd16c0b17e0 0x7fd16c15eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:54.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.228+0000 7fd17187c700 1 -- 172.21.15.112:0/2207806366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd164009b30 con 0x7fd16c0b1400 2023-08-27T11:25:54.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.229+0000 7fd17187c700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd16c0b1400 0x7fd16c0c9c20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00c7f0 tx=0x7fd15c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:54.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.229+0000 7fd1627fc700 1 -- 172.21.15.112:0/2207806366 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c011800 con 0x7fd16c0b1400 2023-08-27T11:25:54.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.229+0000 7fd1627fc700 1 -- 172.21.15.112:0/2207806366 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c011960 con 0x7fd16c0b1400 2023-08-27T11:25:54.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.229+0000 7fd173ae0700 1 -- 172.21.15.112:0/2207806366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16c15f8c0 con 0x7fd16c0b1400 2023-08-27T11:25:54.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.229+0000 7fd1627fc700 1 -- 172.21.15.112:0/2207806366 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c011d10 con 0x7fd16c0b1400 2023-08-27T11:25:54.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.229+0000 7fd173ae0700 1 -- 172.21.15.112:0/2207806366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16c15fe80 con 0x7fd16c0b1400 2023-08-27T11:25:54.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.230+0000 7fd173ae0700 1 -- 172.21.15.112:0/2207806366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16c067a20 con 0x7fd16c0b1400 2023-08-27T11:25:54.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.230+0000 7fd1627fc700 1 -- 172.21.15.112:0/2207806366 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fd15c018360 con 0x7fd16c0b1400 2023-08-27T11:25:54.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.231+0000 7fd1627fc700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd15006a320 0x7fd15006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:54.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.231+0000 7fd1627fc700 1 -- 172.21.15.112:0/2207806366 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fd15c088b40 con 0x7fd16c0b1400 2023-08-27T11:25:54.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.231+0000 7fd17107b700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd15006a320 0x7fd15006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:54.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.231+0000 7fd17107b700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd15006a320 0x7fd15006c7e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0ca520 tx=0x7fd168008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:54.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.235+0000 7fd1627fc700 1 -- 172.21.15.112:0/2207806366 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd15c053cb0 con 0x7fd16c0b1400 2023-08-27T11:25:54.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.465+0000 7fd173ae0700 1 -- 172.21.15.112:0/2207806366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd16c06bbe0 con 0x7fd16c0b1400 2023-08-27T11:25:54.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.465+0000 7fd1627fc700 1 -- 172.21.15.112:0/2207806366 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd15c015070 con 0x7fd16c0b1400 2023-08-27T11:25:54.466 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:25:54.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.467+0000 7fd173ae0700 1 -- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd15006a320 msgr2=0x7fd15006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:54.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.467+0000 7fd173ae0700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd15006a320 0x7fd15006c7e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0ca520 tx=0x7fd168008040 comp rx=0 tx=0).stop 2023-08-27T11:25:54.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.467+0000 7fd173ae0700 1 -- 172.21.15.112:0/2207806366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd16c0b1400 msgr2=0x7fd16c0c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:54.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.467+0000 7fd173ae0700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd16c0b1400 0x7fd16c0c9c20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00c7f0 tx=0x7fd15c00cbb0 comp rx=0 tx=0).stop 2023-08-27T11:25:54.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.467+0000 7fd173ae0700 1 -- 172.21.15.112:0/2207806366 shutdown_connections 2023-08-27T11:25:54.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.467+0000 7fd173ae0700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd15006a320 0x7fd15006c7e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:54.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.467+0000 7fd173ae0700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd16c161ce0 0x7fd16c160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:54.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.467+0000 7fd173ae0700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd16c0b1400 0x7fd16c0c9c20 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:54.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.467+0000 7fd173ae0700 1 --2- 172.21.15.112:0/2207806366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd16c0b17e0 0x7fd16c15eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:54.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.467+0000 7fd173ae0700 1 -- 172.21.15.112:0/2207806366 >> 172.21.15.112:0/2207806366 conn(0x7fd16c0d0000 msgr2=0x7fd16c071740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:54.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.468+0000 7fd173ae0700 1 -- 172.21.15.112:0/2207806366 shutdown_connections 2023-08-27T11:25:54.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:54.468+0000 7fd173ae0700 1 -- 172.21.15.112:0/2207806366 wait complete. 2023-08-27T11:25:54.480 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:25:54.481 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:25:54.481 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2023-08-27T11:25:54.482 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-27T11:25:54.482 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:25:54.482 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:25:54.482 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:25:54.483 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:25:54.483 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:25:54.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:54.484 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2023-08-27T11:25:54.506 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:25:54.507 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:25:54.507 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:25:54.507 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:54.508 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:25:54.508 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:54.508 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:25:54.508 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:54.509 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:25:54.509 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:54.509 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:25:54.510 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:54.510 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:25:54.510 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:54.510 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:25:54.511 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 41904sec preferred_lft 41904sec 2023-08-27T11:25:54.511 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ac17:49c5:cb39:801a/64 scope link noprefixroute 2023-08-27T11:25:54.511 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:54.512 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:25:54.512 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:54.512 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:25:54.512 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:54.513 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c4b4:94ff:fe58:4e35/64 scope link 2023-08-27T11:25:54.513 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:54.513 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:25:54.513 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:25:54.514 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::5806:6aff:fe04:ef32/64 scope link 2023-08-27T11:25:54.514 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:54.514 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:25:54.515 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 8a:21:ba:15:69:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-27T11:25:54.515 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::8821:baff:fe15:6919/64 scope link 2023-08-27T11:25:54.515 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:54.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:54.516 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:25:54.578 INFO:teuthology.orchestra.run.smithi138.stdout:1 2023-08-27T11:25:54.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:54.580 DEBUG:teuthology.orchestra.run.smithi138:> route 2023-08-27T11:25:54.642 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2023-08-27T11:25:54.643 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:25:54.643 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:25:54.643 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:25:54.643 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:25:54.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:54.644 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:25:54.645 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:25:54.645 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:25:54.645 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:25:54.646 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:25:54.646 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:25:54.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:54.757 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:25:54.775 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-27T11:25:54.775 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-27T11:25:54.776 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:25:54.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:54.777 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2023-08-27T11:25:54.843 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:25:54.844 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-27T11:25:54.900 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:25:54.901 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:25:54.901 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:25:54.957 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:25:54.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:54.958 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:25:55.043 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:25:55.044 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:25:55.044 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:25:55.052 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:25:55.053 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:25:55.053 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:25:55.053 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:25:55.053 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:25:55.128 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:25:55.129 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:25:55.129 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:25:55.129 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:25:55.129 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:25:55.130 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:25:55.130 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:25:55.130 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:25:55.131 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:25:55.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:55.132 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:25:55.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:55.159 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:25:55.230 INFO:teuthology.orchestra.run.smithi138.stdout:client6762 2023-08-27T11:25:55.231 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:25:55.231 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:55.287 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-08-27T11:25:55.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.414+0000 7f8a419ec700 1 Processor -- start 2023-08-27T11:25:55.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.414+0000 7f8a419ec700 1 -- start start 2023-08-27T11:25:55.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.414+0000 7f8a419ec700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8a3c1465f0 0x7f8a3c0b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.414+0000 7f8a419ec700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3c0b64e0 con 0x7f8a3c0b4390 2023-08-27T11:25:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.414+0000 7f8a419ec700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3c0b67c0 con 0x7f8a3c0b7580 2023-08-27T11:25:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.414+0000 7f8a419ec700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3c0b6aa0 con 0x7f8a3c1465f0 2023-08-27T11:25:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a3a7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8a3c0b4390 0x7f8a3c0d0bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54622/0 (socket says 172.21.15.112:54622) 2023-08-27T11:25:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a3a7fc700 1 -- 172.21.15.112:0/3656693278 learned_addr learned my addr 172.21.15.112:0/3656693278 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a3affd700 1 --2- 172.21.15.112:0/3656693278 >> v2:172.21.15.138:3300/0 conn(0x7f8a3c1465f0 0x7f8a3c0b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a39ffb700 1 -- 172.21.15.112:0/3656693278 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453820083 0 0) 0x7f8a3c0b64e0 con 0x7f8a3c0b4390 2023-08-27T11:25:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a39ffb700 1 -- 172.21.15.112:0/3656693278 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a240035b0 con 0x7f8a3c0b4390 2023-08-27T11:25:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a39ffb700 1 -- 172.21.15.112:0/3656693278 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167925752 0 0) 0x7f8a3c0b67c0 con 0x7f8a3c0b7580 2023-08-27T11:25:55.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a39ffb700 1 -- 172.21.15.112:0/3656693278 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a3c0b64e0 con 0x7f8a3c0b7580 2023-08-27T11:25:55.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a3affd700 1 -- 172.21.15.112:0/3656693278 >> v1:172.21.15.138:6789/0 conn(0x7f8a3c0b7580 legacy=0x7f8a3c0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:55.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a3affd700 1 -- 172.21.15.112:0/3656693278 >> v1:172.21.15.112:6789/0 conn(0x7f8a3c0b4390 legacy=0x7f8a3c0d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:55.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a39ffb700 1 -- 172.21.15.112:0/3656693278 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 382182134 0 0) 0x7f8a240035b0 con 0x7f8a3c0b4390 2023-08-27T11:25:55.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a3affd700 1 -- 172.21.15.112:0/3656693278 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c0b6d80 con 0x7f8a3c1465f0 2023-08-27T11:25:55.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a3affd700 1 --2- 172.21.15.112:0/3656693278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3c1465f0 0x7f8a3c0b3e50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009840 tx=0x7f8a2c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88f6f77578070080 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:55.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a39ffb700 1 -- 172.21.15.112:0/3656693278 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a30001670 con 0x7f8a3c1465f0 2023-08-27T11:25:55.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a39ffb700 1 -- 172.21.15.112:0/3656693278 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c004030 con 0x7f8a3c1465f0 2023-08-27T11:25:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.415+0000 7f8a39ffb700 1 -- 172.21.15.112:0/3656693278 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0043c0 con 0x7f8a3c1465f0 2023-08-27T11:25:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.416+0000 7f8a419ec700 1 -- 172.21.15.112:0/3656693278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3c1465f0 msgr2=0x7f8a3c0b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.416+0000 7f8a419ec700 1 --2- 172.21.15.112:0/3656693278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3c1465f0 0x7f8a3c0b3e50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009840 tx=0x7f8a2c009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.416+0000 7f8a419ec700 1 -- 172.21.15.112:0/3656693278 shutdown_connections 2023-08-27T11:25:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.416+0000 7f8a419ec700 1 --2- 172.21.15.112:0/3656693278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3c1465f0 0x7f8a3c0b3e50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:55.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.416+0000 7f8a419ec700 1 -- 172.21.15.112:0/3656693278 >> 172.21.15.112:0/3656693278 conn(0x7f8a3c06d330 msgr2=0x7f8a3c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:55.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.416+0000 7f8a419ec700 1 -- 172.21.15.112:0/3656693278 shutdown_connections 2023-08-27T11:25:55.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.416+0000 7f8a419ec700 1 -- 172.21.15.112:0/3656693278 wait complete. 2023-08-27T11:25:55.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.417+0000 7f8a419ec700 1 Processor -- start 2023-08-27T11:25:55.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.417+0000 7f8a419ec700 1 -- start start 2023-08-27T11:25:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a419ec700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3c0b4390 0x7f8a3c0b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a419ec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c0b7580 0x7f8a3c15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a419ec700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a3c1465f0 0x7f8a3c0af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a419ec700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c0b1e90 con 0x7f8a3c0b7580 2023-08-27T11:25:55.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a419ec700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c0b1560 con 0x7f8a3c0b4390 2023-08-27T11:25:55.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a419ec700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c15d270 con 0x7f8a3c1465f0 2023-08-27T11:25:55.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a3a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c0b7580 0x7f8a3c15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:55.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a3a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c0b7580 0x7f8a3c15cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60246/0 (socket says 172.21.15.112:60246) 2023-08-27T11:25:55.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a3a7fc700 1 -- 172.21.15.112:0/863110309 learned_addr learned my addr 172.21.15.112:0/863110309 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:55.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a3affd700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3c0b4390 0x7f8a3c0b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:55.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a3a7fc700 1 -- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a3c1465f0 msgr2=0x7f8a3c0af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:55.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a3b7fe700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a3c1465f0 0x7f8a3c0af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:55.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a3a7fc700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a3c1465f0 0x7f8a3c0af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:55.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a3a7fc700 1 -- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3c0b4390 msgr2=0x7f8a3c0b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:55.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a3a7fc700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3c0b4390 0x7f8a3c0b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:55.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.418+0000 7f8a3a7fc700 1 -- 172.21.15.112:0/863110309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c0094a0 con 0x7f8a3c0b7580 2023-08-27T11:25:55.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.419+0000 7f8a3affd700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3c0b4390 0x7f8a3c0b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:55.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.419+0000 7f8a3a7fc700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c0b7580 0x7f8a3c15cd30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000c700 tx=0x7f8a3000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:55.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.419+0000 7f8a23fff700 1 -- 172.21.15.112:0/863110309 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a30004c80 con 0x7f8a3c0b7580 2023-08-27T11:25:55.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.419+0000 7f8a23fff700 1 -- 172.21.15.112:0/863110309 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a30004de0 con 0x7f8a3c0b7580 2023-08-27T11:25:55.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.419+0000 7f8a419ec700 1 -- 172.21.15.112:0/863110309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3c0b17d0 con 0x7f8a3c0b7580 2023-08-27T11:25:55.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.419+0000 7f8a23fff700 1 -- 172.21.15.112:0/863110309 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a300072f0 con 0x7f8a3c0b7580 2023-08-27T11:25:55.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.419+0000 7f8a419ec700 1 -- 172.21.15.112:0/863110309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3c1f3b10 con 0x7f8a3c0b7580 2023-08-27T11:25:55.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.420+0000 7f8a419ec700 1 -- 172.21.15.112:0/863110309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a3c0b1fd0 con 0x7f8a3c0b7580 2023-08-27T11:25:55.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.420+0000 7f8a23fff700 1 -- 172.21.15.112:0/863110309 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f8a30007450 con 0x7f8a3c0b7580 2023-08-27T11:25:55.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.421+0000 7f8a23fff700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a2806a320 0x7f8a2806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:55.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.421+0000 7f8a23fff700 1 -- 172.21.15.112:0/863110309 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f8a30088760 con 0x7f8a3c0b7580 2023-08-27T11:25:55.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.421+0000 7f8a3affd700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a2806a320 0x7f8a2806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:55.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.421+0000 7f8a3affd700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a2806a320 0x7f8a2806c7e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0090b0 tx=0x7f8a2c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:55.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.425+0000 7f8a23fff700 1 -- 172.21.15.112:0/863110309 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8a3c0b1fd0 con 0x7f8a3c0b7580 2023-08-27T11:25:55.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.656+0000 7f8a419ec700 1 -- 172.21.15.112:0/863110309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8a3c0b1fd0 con 0x7f8a3c0b7580 2023-08-27T11:25:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.920+0000 7f8a23fff700 1 -- 172.21.15.112:0/863110309 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 136+0+0 (secure 0 0 0) 0x7f8a3c0b1fd0 con 0x7f8a3c0b7580 2023-08-27T11:25:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.922+0000 7f8a419ec700 1 -- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a2806a320 msgr2=0x7f8a2806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.922+0000 7f8a419ec700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a2806a320 0x7f8a2806c7e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0090b0 tx=0x7f8a2c008030 comp rx=0 tx=0).stop 2023-08-27T11:25:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.922+0000 7f8a419ec700 1 -- 172.21.15.112:0/863110309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c0b7580 msgr2=0x7f8a3c15cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.922+0000 7f8a419ec700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c0b7580 0x7f8a3c15cd30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000c700 tx=0x7f8a3000cac0 comp rx=0 tx=0).stop 2023-08-27T11:25:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.922+0000 7f8a419ec700 1 -- 172.21.15.112:0/863110309 shutdown_connections 2023-08-27T11:25:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.922+0000 7f8a419ec700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a2806a320 0x7f8a2806c7e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.922+0000 7f8a419ec700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a3c1465f0 0x7f8a3c0af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.922+0000 7f8a419ec700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3c0b4390 0x7f8a3c0b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.922+0000 7f8a419ec700 1 --2- 172.21.15.112:0/863110309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c0b7580 0x7f8a3c15cd30 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.922+0000 7f8a419ec700 1 -- 172.21.15.112:0/863110309 >> 172.21.15.112:0/863110309 conn(0x7f8a3c06d330 msgr2=0x7f8a3c0b7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.923+0000 7f8a419ec700 1 -- 172.21.15.112:0/863110309 shutdown_connections 2023-08-27T11:25:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:55.923+0000 7f8a419ec700 1 -- 172.21.15.112:0/863110309 wait complete. 2023-08-27T11:25:55.937 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:25:56.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.057+0000 7ff2868da700 1 Processor -- start 2023-08-27T11:25:56.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.057+0000 7ff2868da700 1 -- start start 2023-08-27T11:25:56.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.058+0000 7ff2868da700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff280141420 0x7ff2800aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.058+0000 7ff2868da700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2800ab440 con 0x7ff2800a9970 2023-08-27T11:25:56.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.058+0000 7ff2868da700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28015eed0 con 0x7ff28006e150 2023-08-27T11:25:56.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.058+0000 7ff2868da700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff28015f1b0 con 0x7ff280141420 2023-08-27T11:25:56.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.058+0000 7ff284e77700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff2800a9970 0x7ff2800cc780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54638/0 (socket says 172.21.15.112:54638) 2023-08-27T11:25:56.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.058+0000 7ff284e77700 1 -- 172.21.15.112:0/742945762 learned_addr learned my addr 172.21.15.112:0/742945762 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:56.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.058+0000 7ff27ffff700 1 --2- 172.21.15.112:0/742945762 >> v2:172.21.15.138:3300/0 conn(0x7ff280141420 0x7ff2800aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511069858 0 0) 0x7ff2800ab440 con 0x7ff2800a9970 2023-08-27T11:25:56.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff26c0035b0 con 0x7ff2800a9970 2023-08-27T11:25:56.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3297252523 0 0) 0x7ff26c0035b0 con 0x7ff2800a9970 2023-08-27T11:25:56.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 >> v2:172.21.15.138:3300/0 conn(0x7ff280141420 msgr2=0x7ff2800aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:56.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 --2- 172.21.15.112:0/742945762 >> v2:172.21.15.138:3300/0 conn(0x7ff280141420 0x7ff2800aaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 >> v1:172.21.15.138:6789/0 conn(0x7ff28006e150 legacy=0x7ff2800cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28015f490 con 0x7ff2800a9970 2023-08-27T11:25:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7ff2780019f0 con 0x7ff2800a9970 2023-08-27T11:25:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 >> v1:172.21.15.112:6789/0 conn(0x7ff2800a9970 legacy=0x7ff2800cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 --2- 172.21.15.112:0/742945762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26c003a00 0x7ff26c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 --2- 172.21.15.112:0/742945762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26c004850 0x7ff26c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 --2- 172.21.15.112:0/742945762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff26c005330 0x7ff26c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28015f1b0 con 0x7ff26c005330 2023-08-27T11:25:56.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c00c200 con 0x7ff26c003a00 2023-08-27T11:25:56.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c00c3c0 con 0x7ff26c004850 2023-08-27T11:25:56.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff284e77700 1 --2- 172.21.15.112:0/742945762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff26c005330 0x7ff26c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27ffff700 1 --2- 172.21.15.112:0/742945762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26c003a00 0x7ff26c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27f7fe700 1 --2- 172.21.15.112:0/742945762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26c004850 0x7ff26c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff284e77700 1 -- 172.21.15.112:0/742945762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26c004850 msgr2=0x7ff26c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:56.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff284e77700 1 --2- 172.21.15.112:0/742945762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26c004850 0x7ff26c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff284e77700 1 -- 172.21.15.112:0/742945762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26c003a00 msgr2=0x7ff26c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:56.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff284e77700 1 --2- 172.21.15.112:0/742945762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26c003a00 0x7ff26c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff284e77700 1 -- 172.21.15.112:0/742945762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c0035b0 con 0x7ff26c005330 2023-08-27T11:25:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff27f7fe700 1 --2- 172.21.15.112:0/742945762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26c004850 0x7ff26c004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff284e77700 1 -- 172.21.15.112:0/742945762 reap_dead start 2023-08-27T11:25:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.059+0000 7ff284e77700 1 --2- 172.21.15.112:0/742945762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff26c005330 0x7ff26c00ba10 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff27800a800 tx=0x7ff26c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67559d7c2e57ac15 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.060+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27800a2f0 con 0x7ff26c005330 2023-08-27T11:25:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.060+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff278012600 con 0x7ff26c005330 2023-08-27T11:25:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.060+0000 7ff27effd700 1 -- 172.21.15.112:0/742945762 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2780129b0 con 0x7ff26c005330 2023-08-27T11:25:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.060+0000 7ff2868da700 1 -- 172.21.15.112:0/742945762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff26c005330 msgr2=0x7ff26c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.060+0000 7ff2868da700 1 --2- 172.21.15.112:0/742945762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff26c005330 0x7ff26c00ba10 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff27800a800 tx=0x7ff26c005880 comp rx=0 tx=0).stop 2023-08-27T11:25:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.060+0000 7ff2868da700 1 -- 172.21.15.112:0/742945762 shutdown_connections 2023-08-27T11:25:56.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.060+0000 7ff2868da700 1 --2- 172.21.15.112:0/742945762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff26c005330 0x7ff26c00ba10 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.060+0000 7ff2868da700 1 -- 172.21.15.112:0/742945762 >> 172.21.15.112:0/742945762 conn(0x7ff28006cd30 msgr2=0x7ff2800717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:56.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.060+0000 7ff2868da700 1 -- 172.21.15.112:0/742945762 shutdown_connections 2023-08-27T11:25:56.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.060+0000 7ff2868da700 1 -- 172.21.15.112:0/742945762 wait complete. 2023-08-27T11:25:56.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.061+0000 7ff2868da700 1 Processor -- start 2023-08-27T11:25:56.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.062+0000 7ff2868da700 1 -- start start 2023-08-27T11:25:56.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.062+0000 7ff2868da700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26c005330 0x7ff280141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.062+0000 7ff2868da700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff280143e10 0x7ff2801f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.062+0000 7ff2868da700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff28013fde0 0x7ff280151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.062+0000 7ff2868da700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff280144e70 con 0x7ff28013fde0 2023-08-27T11:25:56.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.062+0000 7ff2868da700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2801f18f0 con 0x7ff26c005330 2023-08-27T11:25:56.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.062+0000 7ff2868da700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2801442f0 con 0x7ff280143e10 2023-08-27T11:25:56.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff284e77700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff28013fde0 0x7ff280151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff284e77700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff28013fde0 0x7ff280151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60268/0 (socket says 172.21.15.112:60268) 2023-08-27T11:25:56.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff284e77700 1 -- 172.21.15.112:0/219654285 learned_addr learned my addr 172.21.15.112:0/219654285 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:56.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff27f7fe700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff280143e10 0x7ff2801f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff27ffff700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26c005330 0x7ff280141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff284e77700 1 -- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff280143e10 msgr2=0x7ff2801f13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff284e77700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff280143e10 0x7ff2801f13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff284e77700 1 -- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26c005330 msgr2=0x7ff280141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff284e77700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26c005330 0x7ff280141b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff284e77700 1 -- 172.21.15.112:0/219654285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2801445d0 con 0x7ff28013fde0 2023-08-27T11:25:56.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff284e77700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff28013fde0 0x7ff280151b90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff2800a8530 tx=0x7ff278001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:56.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff27cff9700 1 -- 172.21.15.112:0/219654285 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278024030 con 0x7ff28013fde0 2023-08-27T11:25:56.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff27cff9700 1 -- 172.21.15.112:0/219654285 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2780019b0 con 0x7ff28013fde0 2023-08-27T11:25:56.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff2868da700 1 -- 172.21.15.112:0/219654285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2801448b0 con 0x7ff28013fde0 2023-08-27T11:25:56.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff27cff9700 1 -- 172.21.15.112:0/219654285 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27801e670 con 0x7ff28013fde0 2023-08-27T11:25:56.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.063+0000 7ff2868da700 1 -- 172.21.15.112:0/219654285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff280156980 con 0x7ff28013fde0 2023-08-27T11:25:56.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.064+0000 7ff2868da700 1 -- 172.21.15.112:0/219654285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff280067a20 con 0x7ff28013fde0 2023-08-27T11:25:56.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.064+0000 7ff27cff9700 1 -- 172.21.15.112:0/219654285 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7ff278027070 con 0x7ff28013fde0 2023-08-27T11:25:56.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.065+0000 7ff27cff9700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff26006a320 0x7ff26006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.065+0000 7ff27cff9700 1 -- 172.21.15.112:0/219654285 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7ff278025070 con 0x7ff28013fde0 2023-08-27T11:25:56.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.065+0000 7ff27ffff700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff26006a320 0x7ff26006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.065+0000 7ff27ffff700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff26006a320 0x7ff26006c7e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff280143da0 tx=0x7ff268008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:56.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.069+0000 7ff27cff9700 1 -- 172.21.15.112:0/219654285 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff2780560f0 con 0x7ff28013fde0 2023-08-27T11:25:56.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.286+0000 7ff2868da700 1 -- 172.21.15.112:0/219654285 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff28015fa50 con 0x7ff26006a320 2023-08-27T11:25:56.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.288+0000 7ff27cff9700 1 -- 172.21.15.112:0/219654285 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff268009770 con 0x7ff26006a320 2023-08-27T11:25:56.290 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:25:56.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:56.290 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs" 2023-08-27T11:25:56.291 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:56.291 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:25:56.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.291+0000 7ff2868da700 1 -- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff26006a320 msgr2=0x7ff26006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.291+0000 7ff2868da700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff26006a320 0x7ff26006c7e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff280143da0 tx=0x7ff268008040 comp rx=0 tx=0).stop 2023-08-27T11:25:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.291+0000 7ff2868da700 1 -- 172.21.15.112:0/219654285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff28013fde0 msgr2=0x7ff280151b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.291+0000 7ff2868da700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff28013fde0 0x7ff280151b90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff2800a8530 tx=0x7ff278001d80 comp rx=0 tx=0).stop 2023-08-27T11:25:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.291+0000 7ff2868da700 1 -- 172.21.15.112:0/219654285 shutdown_connections 2023-08-27T11:25:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.291+0000 7ff2868da700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff26006a320 0x7ff26006c7e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.291+0000 7ff2868da700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff28013fde0 0x7ff280151b90 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.291+0000 7ff2868da700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26c005330 0x7ff280141b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.291+0000 7ff2868da700 1 --2- 172.21.15.112:0/219654285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff280143e10 0x7ff2801f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.291+0000 7ff2868da700 1 -- 172.21.15.112:0/219654285 >> 172.21.15.112:0/219654285 conn(0x7ff28006cd30 msgr2=0x7ff2800a8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.291+0000 7ff2868da700 1 -- 172.21.15.112:0/219654285 shutdown_connections 2023-08-27T11:25:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.291+0000 7ff2868da700 1 -- 172.21.15.112:0/219654285 wait complete. 2023-08-27T11:25:56.304 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:25:56.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.427+0000 7f3602f65700 1 Processor -- start 2023-08-27T11:25:56.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.427+0000 7f3602f65700 1 -- start start 2023-08-27T11:25:56.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.427+0000 7f3602f65700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35fc1445b0 0x7f35fc0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.427+0000 7f3602f65700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f35fc0ccad0 0x7f35fc15b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3602f65700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f35fc06f790 0x7f35fc06fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3602f65700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc15bc70 con 0x7f35fc06f790 2023-08-27T11:25:56.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3602f65700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc0ccf50 con 0x7f35fc0ccad0 2023-08-27T11:25:56.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3602f65700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc070120 con 0x7f35fc1445b0 2023-08-27T11:25:56.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3601502700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f35fc06f790 0x7f35fc06fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3601502700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f35fc06f790 0x7f35fc06fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60276/0 (socket says 172.21.15.112:60276) 2023-08-27T11:25:56.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3601502700 1 -- 172.21.15.112:0/3887646098 learned_addr learned my addr 172.21.15.112:0/3887646098 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:56.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f35fbfff700 1 --2- 172.21.15.112:0/3887646098 >> v2:172.21.15.138:3300/0 conn(0x7f35fc0ccad0 0x7f35fc15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3600d01700 1 --2- 172.21.15.112:0/3887646098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35fc1445b0 0x7f35fc0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3601502700 1 -- 172.21.15.112:0/3887646098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35fc1445b0 msgr2=0x7f35fc0ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:56.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3601502700 1 --2- 172.21.15.112:0/3887646098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35fc1445b0 0x7f35fc0ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3601502700 1 -- 172.21.15.112:0/3887646098 >> v2:172.21.15.138:3300/0 conn(0x7f35fc0ccad0 msgr2=0x7f35fc15b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:56.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3601502700 1 --2- 172.21.15.112:0/3887646098 >> v2:172.21.15.138:3300/0 conn(0x7f35fc0ccad0 0x7f35fc15b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3601502700 1 -- 172.21.15.112:0/3887646098 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc0b08e0 con 0x7f35fc06f790 2023-08-27T11:25:56.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f3601502700 1 --2- 172.21.15.112:0/3887646098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35fc06f790 0x7f35fc06fbe0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0b2db0 tx=0x7f35f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8316d11091d0aa2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:56.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f35fb7fe700 1 -- 172.21.15.112:0/3887646098 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f400e070 con 0x7f35fc06f790 2023-08-27T11:25:56.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f35fb7fe700 1 -- 172.21.15.112:0/3887646098 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f400b9c0 con 0x7f35fc06f790 2023-08-27T11:25:56.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.428+0000 7f35fb7fe700 1 -- 172.21.15.112:0/3887646098 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f400bd70 con 0x7f35fc06f790 2023-08-27T11:25:56.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.429+0000 7f3602f65700 1 -- 172.21.15.112:0/3887646098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35fc06f790 msgr2=0x7f35fc06fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.429+0000 7f3602f65700 1 --2- 172.21.15.112:0/3887646098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35fc06f790 0x7f35fc06fbe0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0b2db0 tx=0x7f35f400b330 comp rx=0 tx=0).stop 2023-08-27T11:25:56.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.429+0000 7f3602f65700 1 -- 172.21.15.112:0/3887646098 shutdown_connections 2023-08-27T11:25:56.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.429+0000 7f3602f65700 1 --2- 172.21.15.112:0/3887646098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35fc06f790 0x7f35fc06fbe0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.429+0000 7f3602f65700 1 --2- 172.21.15.112:0/3887646098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35fc1445b0 0x7f35fc0ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.429+0000 7f3602f65700 1 --2- 172.21.15.112:0/3887646098 >> v2:172.21.15.138:3300/0 conn(0x7f35fc0ccad0 0x7f35fc15b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.429+0000 7f3602f65700 1 -- 172.21.15.112:0/3887646098 >> 172.21.15.112:0/3887646098 conn(0x7f35fc0b1e90 msgr2=0x7f35fc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:56.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.429+0000 7f3602f65700 1 -- 172.21.15.112:0/3887646098 shutdown_connections 2023-08-27T11:25:56.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.429+0000 7f3602f65700 1 -- 172.21.15.112:0/3887646098 wait complete. 2023-08-27T11:25:56.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.430+0000 7f3602f65700 1 Processor -- start 2023-08-27T11:25:56.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.430+0000 7f3602f65700 1 -- start start 2023-08-27T11:25:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3602f65700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35fc06f790 0x7f35fc0ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3602f65700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35fc0ccad0 0x7f35fc1408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3602f65700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35fc1445b0 0x7f35fc140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3602f65700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0aa1c0 con 0x7f35fc1445b0 2023-08-27T11:25:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3602f65700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0aa4a0 con 0x7f35fc0ccad0 2023-08-27T11:25:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3602f65700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0aa780 con 0x7f35fc06f790 2023-08-27T11:25:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3601502700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35fc1445b0 0x7f35fc140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3601502700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35fc1445b0 0x7f35fc140e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60282/0 (socket says 172.21.15.112:60282) 2023-08-27T11:25:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3601502700 1 -- 172.21.15.112:0/2513235482 learned_addr learned my addr 172.21.15.112:0/2513235482 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f35fbfff700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35fc0ccad0 0x7f35fc1408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3600d01700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35fc06f790 0x7f35fc0ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3601502700 1 -- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35fc06f790 msgr2=0x7f35fc0ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3601502700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35fc06f790 0x7f35fc0ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3601502700 1 -- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35fc0ccad0 msgr2=0x7f35fc1408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3601502700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35fc0ccad0 0x7f35fc1408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.431+0000 7f3601502700 1 -- 172.21.15.112:0/2513235482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f4009b30 con 0x7f35fc1445b0 2023-08-27T11:25:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.432+0000 7f3601502700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35fc1445b0 0x7f35fc140e20 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0b5830 tx=0x7f35f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.432+0000 7f35f97fa700 1 -- 172.21.15.112:0/2513235482 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4004250 con 0x7f35fc1445b0 2023-08-27T11:25:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.432+0000 7f35f97fa700 1 -- 172.21.15.112:0/2513235482 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f40043d0 con 0x7f35fc1445b0 2023-08-27T11:25:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.432+0000 7f3602f65700 1 -- 172.21.15.112:0/2513235482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc0aaa60 con 0x7f35fc1445b0 2023-08-27T11:25:56.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.432+0000 7f35f97fa700 1 -- 172.21.15.112:0/2513235482 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4004780 con 0x7f35fc1445b0 2023-08-27T11:25:56.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.432+0000 7f3602f65700 1 -- 172.21.15.112:0/2513235482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc0aeb00 con 0x7f35fc1445b0 2023-08-27T11:25:56.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.433+0000 7f3602f65700 1 -- 172.21.15.112:0/2513235482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35fc067e90 con 0x7f35fc1445b0 2023-08-27T11:25:56.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.433+0000 7f35f97fa700 1 -- 172.21.15.112:0/2513235482 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f35f401b430 con 0x7f35fc1445b0 2023-08-27T11:25:56.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.433+0000 7f35f97fa700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f35e006a320 0x7f35e006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.433+0000 7f35f97fa700 1 -- 172.21.15.112:0/2513235482 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f35f4089ae0 con 0x7f35fc1445b0 2023-08-27T11:25:56.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.434+0000 7f3600d01700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f35e006a320 0x7f35e006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.434+0000 7f3600d01700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f35e006a320 0x7f35e006c7e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f35fc06e2f0 tx=0x7f35f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:56.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.438+0000 7f35f97fa700 1 -- 172.21.15.112:0/2513235482 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f35f4054d70 con 0x7f35fc1445b0 2023-08-27T11:25:56.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.629+0000 7f3602f65700 1 -- 172.21.15.112:0/2513235482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f35fc0aeb00 con 0x7f35fc1445b0 2023-08-27T11:25:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.629+0000 7f35f97fa700 1 -- 172.21.15.112:0/2513235482 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f35fc0aeb00 con 0x7f35fc1445b0 2023-08-27T11:25:56.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.631+0000 7f3602f65700 1 -- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f35e006a320 msgr2=0x7f35e006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.631+0000 7f3602f65700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f35e006a320 0x7f35e006c7e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f35fc06e2f0 tx=0x7f35f0008040 comp rx=0 tx=0).stop 2023-08-27T11:25:56.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.631+0000 7f3602f65700 1 -- 172.21.15.112:0/2513235482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35fc1445b0 msgr2=0x7f35fc140e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.631+0000 7f3602f65700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35fc1445b0 0x7f35fc140e20 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0b5830 tx=0x7f35f4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.631+0000 7f3602f65700 1 -- 172.21.15.112:0/2513235482 shutdown_connections 2023-08-27T11:25:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.631+0000 7f3602f65700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f35e006a320 0x7f35e006c7e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.631+0000 7f3602f65700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35fc1445b0 0x7f35fc140e20 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.631+0000 7f3602f65700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35fc06f790 0x7f35fc0ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.631+0000 7f3602f65700 1 --2- 172.21.15.112:0/2513235482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35fc0ccad0 0x7f35fc1408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.631+0000 7f3602f65700 1 -- 172.21.15.112:0/2513235482 >> 172.21.15.112:0/2513235482 conn(0x7f35fc0b1e90 msgr2=0x7f35fc143980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:56.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.631+0000 7f3602f65700 1 -- 172.21.15.112:0/2513235482 shutdown_connections 2023-08-27T11:25:56.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.631+0000 7f3602f65700 1 -- 172.21.15.112:0/2513235482 wait complete. 2023-08-27T11:25:56.645 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000682889 2023-08-27T11:25:56.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.766+0000 7fe04aed3700 1 Processor -- start 2023-08-27T11:25:56.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.766+0000 7fe04aed3700 1 -- start start 2023-08-27T11:25:56.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.767+0000 7fe04aed3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe0441437c0 0x7fe0440b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.767+0000 7fe04aed3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe044143ba0 0x7fe0440ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.767+0000 7fe04aed3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe04406d5b0 0x7fe04406da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.767+0000 7fe04aed3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04406df80 con 0x7fe0441437c0 2023-08-27T11:25:56.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.767+0000 7fe04aed3700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0440cfd50 con 0x7fe044143ba0 2023-08-27T11:25:56.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.767+0000 7fe04aed3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0440cf140 con 0x7fe04406d5b0 2023-08-27T11:25:56.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.767+0000 7fe048c6f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe0441437c0 0x7fe0440b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.767+0000 7fe048c6f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe0441437c0 0x7fe0440b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60286/0 (socket says 172.21.15.112:60286) 2023-08-27T11:25:56.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.767+0000 7fe048c6f700 1 -- 172.21.15.112:0/3102731690 learned_addr learned my addr 172.21.15.112:0/3102731690 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:56.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.768+0000 7fe043fff700 1 --2- 172.21.15.112:0/3102731690 >> v2:172.21.15.138:3300/0 conn(0x7fe044143ba0 0x7fe0440ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.768+0000 7fe049470700 1 --2- 172.21.15.112:0/3102731690 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe04406d5b0 0x7fe04406da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.768+0000 7fe048c6f700 1 -- 172.21.15.112:0/3102731690 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe04406d5b0 msgr2=0x7fe04406da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:56.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.768+0000 7fe048c6f700 1 --2- 172.21.15.112:0/3102731690 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe04406d5b0 0x7fe04406da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.768+0000 7fe048c6f700 1 -- 172.21.15.112:0/3102731690 >> v2:172.21.15.138:3300/0 conn(0x7fe044143ba0 msgr2=0x7fe0440ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:56.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.768+0000 7fe048c6f700 1 --2- 172.21.15.112:0/3102731690 >> v2:172.21.15.138:3300/0 conn(0x7fe044143ba0 0x7fe0440ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.768+0000 7fe048c6f700 1 -- 172.21.15.112:0/3102731690 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0440cf420 con 0x7fe0441437c0 2023-08-27T11:25:56.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.768+0000 7fe048c6f700 1 --2- 172.21.15.112:0/3102731690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0441437c0 0x7fe0440b4670 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe04406e1a0 tx=0x7fe038009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94ac21930f1a111 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:56.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.768+0000 7fe0437fe700 1 -- 172.21.15.112:0/3102731690 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038011830 con 0x7fe0441437c0 2023-08-27T11:25:56.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.768+0000 7fe0437fe700 1 -- 172.21.15.112:0/3102731690 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe038011990 con 0x7fe0441437c0 2023-08-27T11:25:56.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.768+0000 7fe0437fe700 1 -- 172.21.15.112:0/3102731690 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038011d20 con 0x7fe0441437c0 2023-08-27T11:25:56.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.768+0000 7fe04aed3700 1 -- 172.21.15.112:0/3102731690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0441437c0 msgr2=0x7fe0440b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.768+0000 7fe04aed3700 1 --2- 172.21.15.112:0/3102731690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0441437c0 0x7fe0440b4670 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe04406e1a0 tx=0x7fe038009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.769+0000 7fe04aed3700 1 -- 172.21.15.112:0/3102731690 shutdown_connections 2023-08-27T11:25:56.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.769+0000 7fe04aed3700 1 --2- 172.21.15.112:0/3102731690 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe04406d5b0 0x7fe04406da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.769+0000 7fe04aed3700 1 --2- 172.21.15.112:0/3102731690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0441437c0 0x7fe0440b4670 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.769+0000 7fe04aed3700 1 --2- 172.21.15.112:0/3102731690 >> v2:172.21.15.138:3300/0 conn(0x7fe044143ba0 0x7fe0440ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.769+0000 7fe04aed3700 1 -- 172.21.15.112:0/3102731690 >> 172.21.15.112:0/3102731690 conn(0x7fe0440b1e90 msgr2=0x7fe0440b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:56.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.769+0000 7fe04aed3700 1 -- 172.21.15.112:0/3102731690 shutdown_connections 2023-08-27T11:25:56.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.769+0000 7fe04aed3700 1 -- 172.21.15.112:0/3102731690 wait complete. 2023-08-27T11:25:56.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.770+0000 7fe04aed3700 1 Processor -- start 2023-08-27T11:25:56.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.770+0000 7fe04aed3700 1 -- start start 2023-08-27T11:25:56.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe04aed3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04406d5b0 0x7fe044139620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe04aed3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0441437c0 0x7fe044071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe04aed3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe044143ba0 0x7fe044071ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe04aed3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04413a870 con 0x7fe04406d5b0 2023-08-27T11:25:56.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe04aed3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04415afb0 con 0x7fe044143ba0 2023-08-27T11:25:56.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe04aed3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe044139b60 con 0x7fe0441437c0 2023-08-27T11:25:56.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe048c6f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04406d5b0 0x7fe044139620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe048c6f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04406d5b0 0x7fe044139620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60292/0 (socket says 172.21.15.112:60292) 2023-08-27T11:25:56.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe048c6f700 1 -- 172.21.15.112:0/3121903740 learned_addr learned my addr 172.21.15.112:0/3121903740 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:56.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe043fff700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0441437c0 0x7fe044071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe049470700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe044143ba0 0x7fe044071ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe048c6f700 1 -- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0441437c0 msgr2=0x7fe044071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe048c6f700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0441437c0 0x7fe044071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe048c6f700 1 -- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe044143ba0 msgr2=0x7fe044071ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:56.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe048c6f700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe044143ba0 0x7fe044071ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:56.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe048c6f700 1 -- 172.21.15.112:0/3121903740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe038009550 con 0x7fe04406d5b0 2023-08-27T11:25:56.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe048c6f700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04406d5b0 0x7fe044139620 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe038009fd0 tx=0x7fe038004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:56.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe0417fa700 1 -- 172.21.15.112:0/3121903740 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038004450 con 0x7fe04406d5b0 2023-08-27T11:25:56.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe0417fa700 1 -- 172.21.15.112:0/3121903740 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03801f070 con 0x7fe04406d5b0 2023-08-27T11:25:56.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe04aed3700 1 -- 172.21.15.112:0/3121903740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04415b1c0 con 0x7fe04406d5b0 2023-08-27T11:25:56.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe0417fa700 1 -- 172.21.15.112:0/3121903740 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0380038c0 con 0x7fe04406d5b0 2023-08-27T11:25:56.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.771+0000 7fe04aed3700 1 -- 172.21.15.112:0/3121903740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04415b780 con 0x7fe04406d5b0 2023-08-27T11:25:56.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.772+0000 7fe04aed3700 1 -- 172.21.15.112:0/3121903740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe044067e90 con 0x7fe04406d5b0 2023-08-27T11:25:56.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.773+0000 7fe0417fa700 1 -- 172.21.15.112:0/3121903740 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fe038003a20 con 0x7fe04406d5b0 2023-08-27T11:25:56.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.773+0000 7fe0417fa700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe02806a320 0x7fe02806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:56.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.773+0000 7fe0417fa700 1 -- 172.21.15.112:0/3121903740 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fe038021070 con 0x7fe04406d5b0 2023-08-27T11:25:56.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.773+0000 7fe043fff700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe02806a320 0x7fe02806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:56.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.774+0000 7fe043fff700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe02806a320 0x7fe02806c7e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe0440b0af0 tx=0x7fe030009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:56.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.777+0000 7fe0417fa700 1 -- 172.21.15.112:0/3121903740 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe0380554b0 con 0x7fe04406d5b0 2023-08-27T11:25:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:56.995+0000 7fe04aed3700 1 -- 172.21.15.112:0/3121903740 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group_0000000000682889", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0440cf9e0 con 0x7fe02806a320 2023-08-27T11:25:57.004 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:57.002+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:57.004 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:57.002+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:57.004 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:57.002+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:57.005 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:57.002+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:57.005 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:25:57.002+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:57.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.018+0000 7fe0417fa700 1 -- 172.21.15.112:0/3121903740 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe030004020 con 0x7fe02806a320 2023-08-27T11:25:57.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.020+0000 7fe04aed3700 1 -- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe02806a320 msgr2=0x7fe02806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.020+0000 7fe04aed3700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe02806a320 0x7fe02806c7e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe0440b0af0 tx=0x7fe030009210 comp rx=0 tx=0).stop 2023-08-27T11:25:57.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.020+0000 7fe04aed3700 1 -- 172.21.15.112:0/3121903740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04406d5b0 msgr2=0x7fe044139620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.020+0000 7fe04aed3700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04406d5b0 0x7fe044139620 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe038009fd0 tx=0x7fe038004750 comp rx=0 tx=0).stop 2023-08-27T11:25:57.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.020+0000 7fe04aed3700 1 -- 172.21.15.112:0/3121903740 shutdown_connections 2023-08-27T11:25:57.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.020+0000 7fe04aed3700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe02806a320 0x7fe02806c7e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.020+0000 7fe04aed3700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe044143ba0 0x7fe044071ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.020+0000 7fe04aed3700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04406d5b0 0x7fe044139620 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.020+0000 7fe04aed3700 1 --2- 172.21.15.112:0/3121903740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0441437c0 0x7fe044071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.020+0000 7fe04aed3700 1 -- 172.21.15.112:0/3121903740 >> 172.21.15.112:0/3121903740 conn(0x7fe0440b1e90 msgr2=0x7fe04415dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:57.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.020+0000 7fe04aed3700 1 -- 172.21.15.112:0/3121903740 shutdown_connections 2023-08-27T11:25:57.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.020+0000 7fe04aed3700 1 -- 172.21.15.112:0/3121903740 wait complete. 2023-08-27T11:25:57.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs --human_readable 2023-08-27T11:25:57.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.155+0000 7f2c9f4f1700 1 Processor -- start 2023-08-27T11:25:57.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.155+0000 7f2c9f4f1700 1 -- start start 2023-08-27T11:25:57.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.156+0000 7f2c9f4f1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2c9806f790 0x7f2c9806fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:57.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.156+0000 7f2c9f4f1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c980b5070 con 0x7f2c98146c20 2023-08-27T11:25:57.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.156+0000 7f2c9f4f1700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c98147000 con 0x7f2c980b3440 2023-08-27T11:25:57.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.156+0000 7f2c9f4f1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c98070160 con 0x7f2c9806f790 2023-08-27T11:25:57.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.156+0000 7f2c9d28d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2c98146c20 0x7f2c98160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54682/0 (socket says 172.21.15.112:54682) 2023-08-27T11:25:57.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.156+0000 7f2c9da8e700 1 --2- 172.21.15.112:0/2475267805 >> v2:172.21.15.112:3300/0 conn(0x7f2c9806f790 0x7f2c9806fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:57.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.156+0000 7f2c9d28d700 1 -- 172.21.15.112:0/2475267805 learned_addr learned my addr 172.21.15.112:0/2475267805 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:57.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.156+0000 7f2c9da8e700 1 -- 172.21.15.112:0/2475267805 >> v1:172.21.15.138:6789/0 conn(0x7f2c980b3440 legacy=0x7f2c9815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:57.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.157+0000 7f2c9da8e700 1 -- 172.21.15.112:0/2475267805 >> v1:172.21.15.112:6789/0 conn(0x7f2c98146c20 legacy=0x7f2c98160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.157+0000 7f2c9da8e700 1 -- 172.21.15.112:0/2475267805 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c98161740 con 0x7f2c9806f790 2023-08-27T11:25:57.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.157+0000 7f2c8ffff700 1 -- 172.21.15.112:0/2475267805 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783450452 0 0) 0x7f2c88001110 con 0x7f2c98146c20 2023-08-27T11:25:57.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.157+0000 7f2c9da8e700 1 --2- 172.21.15.112:0/2475267805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9806f790 0x7f2c9806fc20 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009d90 tx=0x7f2c9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62d115e38ed28e0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:57.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.157+0000 7f2c8ffff700 1 -- 172.21.15.112:0/2475267805 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9000e070 con 0x7f2c9806f790 2023-08-27T11:25:57.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.157+0000 7f2c8ffff700 1 -- 172.21.15.112:0/2475267805 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9000bb90 con 0x7f2c9806f790 2023-08-27T11:25:57.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.157+0000 7f2c8ffff700 1 -- 172.21.15.112:0/2475267805 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c90003a20 con 0x7f2c9806f790 2023-08-27T11:25:57.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.157+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/2475267805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9806f790 msgr2=0x7f2c9806fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.157+0000 7f2c9f4f1700 1 --2- 172.21.15.112:0/2475267805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9806f790 0x7f2c9806fc20 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009d90 tx=0x7f2c9000b330 comp rx=0 tx=0).stop 2023-08-27T11:25:57.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.157+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/2475267805 shutdown_connections 2023-08-27T11:25:57.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.157+0000 7f2c9f4f1700 1 --2- 172.21.15.112:0/2475267805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9806f790 0x7f2c9806fc20 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.157+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/2475267805 >> 172.21.15.112:0/2475267805 conn(0x7f2c980b1e90 msgr2=0x7f2c980b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:57.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.157+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/2475267805 shutdown_connections 2023-08-27T11:25:57.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.158+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/2475267805 wait complete. 2023-08-27T11:25:57.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.159+0000 7f2c9f4f1700 1 Processor -- start 2023-08-27T11:25:57.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.159+0000 7f2c9f4f1700 1 -- start start 2023-08-27T11:25:57.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.159+0000 7f2c9f4f1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c9806f790 0x7f2c9815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:57.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.159+0000 7f2c9f4f1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c980b3440 0x7f2c980719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:57.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.159+0000 7f2c9f4f1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c98146c20 0x7f2c98071f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:57.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.159+0000 7f2c9f4f1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9815fff0 con 0x7f2c980b3440 2023-08-27T11:25:57.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.159+0000 7f2c9f4f1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9815f3e0 con 0x7f2c9806f790 2023-08-27T11:25:57.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.159+0000 7f2c9f4f1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9815f6c0 con 0x7f2c98146c20 2023-08-27T11:25:57.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.159+0000 7f2c9ca8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c980b3440 0x7f2c980719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:57.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.159+0000 7f2c9ca8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c980b3440 0x7f2c980719e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60308/0 (socket says 172.21.15.112:60308) 2023-08-27T11:25:57.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.159+0000 7f2c9ca8c700 1 -- 172.21.15.112:0/1632011248 learned_addr learned my addr 172.21.15.112:0/1632011248 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:57.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.160+0000 7f2c9ca8c700 1 -- 172.21.15.112:0/1632011248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c98146c20 msgr2=0x7f2c98071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.160+0000 7f2c9ca8c700 1 --2- 172.21.15.112:0/1632011248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c98146c20 0x7f2c98071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.160+0000 7f2c9ca8c700 1 -- 172.21.15.112:0/1632011248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c9806f790 msgr2=0x7f2c9815ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.160+0000 7f2c9d28d700 1 --2- 172.21.15.112:0/1632011248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c9806f790 0x7f2c9815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.160+0000 7f2c9ca8c700 1 --2- 172.21.15.112:0/1632011248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c9806f790 0x7f2c9815ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.160+0000 7f2c9ca8c700 1 -- 172.21.15.112:0/1632011248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c90009a70 con 0x7f2c980b3440 2023-08-27T11:25:57.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.160+0000 7f2c9ca8c700 1 --2- 172.21.15.112:0/1632011248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c980b3440 0x7f2c980719e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400c4d0 tx=0x7f2c9400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:57.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.160+0000 7f2c8dffb700 1 -- 172.21.15.112:0/1632011248 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94011790 con 0x7f2c980b3440 2023-08-27T11:25:57.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.160+0000 7f2c8dffb700 1 -- 172.21.15.112:0/1632011248 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c940118f0 con 0x7f2c980b3440 2023-08-27T11:25:57.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.160+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/1632011248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9815f9a0 con 0x7f2c980b3440 2023-08-27T11:25:57.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.160+0000 7f2c8dffb700 1 -- 172.21.15.112:0/1632011248 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94011ca0 con 0x7f2c980b3440 2023-08-27T11:25:57.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.160+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/1632011248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9815f0f0 con 0x7f2c980b3440 2023-08-27T11:25:57.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.161+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/1632011248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c981601a0 con 0x7f2c980b3440 2023-08-27T11:25:57.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.161+0000 7f2c8dffb700 1 -- 172.21.15.112:0/1632011248 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f2c9401c460 con 0x7f2c980b3440 2023-08-27T11:25:57.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.162+0000 7f2c8dffb700 1 --2- 172.21.15.112:0/1632011248 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c7c06a320 0x7f2c7c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:57.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.162+0000 7f2c8dffb700 1 -- 172.21.15.112:0/1632011248 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f2c94088780 con 0x7f2c980b3440 2023-08-27T11:25:57.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.162+0000 7f2c9d28d700 1 --2- 172.21.15.112:0/1632011248 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c7c06a320 0x7f2c7c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:57.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.163+0000 7f2c9d28d700 1 --2- 172.21.15.112:0/1632011248 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c7c06a320 0x7f2c7c06c7e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2c88000f60 tx=0x7f2c88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:57.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.166+0000 7f2c8dffb700 1 -- 172.21.15.112:0/1632011248 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2c981601a0 con 0x7f2c980b3440 2023-08-27T11:25:57.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.382+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/1632011248 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "human_readable": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2c98161a20 con 0x7f2c7c06a320 2023-08-27T11:25:57.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.385+0000 7f2c8dffb700 1 -- 172.21.15.112:0/1632011248 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+535 (secure 0 0 0) 0x7f2c88004870 con 0x7f2c7c06a320 2023-08-27T11:25:57.387 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:25:57.387 INFO:teuthology.orchestra.run.smithi112.stdout: "mon_addrs": [ 2023-08-27T11:25:57.388 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.112:6789", 2023-08-27T11:25:57.388 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.138:6789", 2023-08-27T11:25:57.388 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.138:6790" 2023-08-27T11:25:57.388 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2023-08-27T11:25:57.388 INFO:teuthology.orchestra.run.smithi112.stdout: "pending_subvolume_deletions": 0, 2023-08-27T11:25:57.388 INFO:teuthology.orchestra.run.smithi112.stdout: "pools": { 2023-08-27T11:25:57.389 INFO:teuthology.orchestra.run.smithi112.stdout: "data": [ 2023-08-27T11:25:57.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:57.389 INFO:teuthology.orchestra.run.smithi112.stdout: "avail": " 341G", 2023-08-27T11:25:57.389 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs_data", 2023-08-27T11:25:57.390 INFO:teuthology.orchestra.run.smithi112.stdout: "used": " 0 " 2023-08-27T11:25:57.390 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:57.390 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2023-08-27T11:25:57.390 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": [ 2023-08-27T11:25:57.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:25:57.390 INFO:teuthology.orchestra.run.smithi112.stdout: "avail": " 341G", 2023-08-27T11:25:57.391 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs_metadata", 2023-08-27T11:25:57.391 INFO:teuthology.orchestra.run.smithi112.stdout: "used": "64.0k" 2023-08-27T11:25:57.391 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:25:57.391 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:25:57.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:25:57.392 INFO:teuthology.orchestra.run.smithi112.stdout: "used_size": " 0 " 2023-08-27T11:25:57.392 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:25:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.388+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/1632011248 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c7c06a320 msgr2=0x7f2c7c06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.388+0000 7f2c9f4f1700 1 --2- 172.21.15.112:0/1632011248 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c7c06a320 0x7f2c7c06c7e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2c88000f60 tx=0x7f2c88009040 comp rx=0 tx=0).stop 2023-08-27T11:25:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.388+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/1632011248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c980b3440 msgr2=0x7f2c980719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.388+0000 7f2c9f4f1700 1 --2- 172.21.15.112:0/1632011248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c980b3440 0x7f2c980719e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400c4d0 tx=0x7f2c9400c890 comp rx=0 tx=0).stop 2023-08-27T11:25:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.388+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/1632011248 shutdown_connections 2023-08-27T11:25:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.388+0000 7f2c9f4f1700 1 --2- 172.21.15.112:0/1632011248 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c7c06a320 0x7f2c7c06c7e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.388+0000 7f2c9f4f1700 1 --2- 172.21.15.112:0/1632011248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c98146c20 0x7f2c98071f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.388+0000 7f2c9f4f1700 1 --2- 172.21.15.112:0/1632011248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c9806f790 0x7f2c9815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.388+0000 7f2c9f4f1700 1 --2- 172.21.15.112:0/1632011248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c980b3440 0x7f2c980719e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.388+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/1632011248 >> 172.21.15.112:0/1632011248 conn(0x7f2c980b1e90 msgr2=0x7f2c980b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.388+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/1632011248 shutdown_connections 2023-08-27T11:25:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.388+0000 7f2c9f4f1700 1 -- 172.21.15.112:0/1632011248 wait complete. 2023-08-27T11:25:57.401 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:25:57.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:25:57.444 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:25:57.489 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:25:57.490 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:57.490 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:57.507 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:25:57.508 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:57.564 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:57.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.722+0000 7f9251f8d700 1 Processor -- start 2023-08-27T11:25:57.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.722+0000 7f9251f8d700 1 -- start start 2023-08-27T11:25:57.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.723+0000 7f9251f8d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f924c0b17e0 0x7f924c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:57.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.723+0000 7f9251f8d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f924c161ce0 0x7f924c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:57.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.723+0000 7f9251f8d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f924c161850 con 0x7f924c0b1400 2023-08-27T11:25:57.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.723+0000 7f9251f8d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924c0cfc10 con 0x7f924c0b17e0 2023-08-27T11:25:57.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.723+0000 7f9251f8d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c161200 con 0x7f924c161ce0 2023-08-27T11:25:57.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.723+0000 7f924bfff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f924c161ce0 0x7f924c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:57.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.723+0000 7f924affd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f924c0b17e0 0x7f924c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:57.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.723+0000 7f924bfff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f924c161ce0 0x7f924c06f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:43740/0 (socket says 172.21.15.112:43740) 2023-08-27T11:25:57.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.723+0000 7f924affd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f924c0b17e0 0x7f924c0cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:60742/0 (socket says 172.21.15.112:60742) 2023-08-27T11:25:57.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.723+0000 7f924b7fe700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f924c0b1400 0x7f924c06e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:33988/0 (socket says 172.21.15.112:33988) 2023-08-27T11:25:57.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.723+0000 7f924bfff700 1 -- 172.21.15.112:0/2113040317 learned_addr learned my addr 172.21.15.112:0/2113040317 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:57.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.723+0000 7f924affd700 1 -- 172.21.15.112:0/2113040317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f924c161ce0 msgr2=0x7f924c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.723+0000 7f924affd700 1 --2- 172.21.15.112:0/2113040317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f924c161ce0 0x7f924c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f924affd700 1 -- 172.21.15.112:0/2113040317 >> v1:172.21.15.138:6789/0 conn(0x7f924c0b1400 legacy=0x7f924c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f924affd700 1 -- 172.21.15.112:0/2113040317 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c1614e0 con 0x7f924c0b17e0 2023-08-27T11:25:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f924a7fc700 1 -- 172.21.15.112:0/2113040317 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753779752 0 0) 0x7f924c161850 con 0x7f924c0b1400 2023-08-27T11:25:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f924bfff700 1 --2- 172.21.15.112:0/2113040317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f924c161ce0 0x7f924c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:57.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f924affd700 1 --2- 172.21.15.112:0/2113040317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f924c0b17e0 0x7f924c0cf5a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f923c009840 tx=0x7f923c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1894259f6a13d978 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:57.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f924a7fc700 1 -- 172.21.15.112:0/2113040317 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c011830 con 0x7f924c0b17e0 2023-08-27T11:25:57.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f924a7fc700 1 -- 172.21.15.112:0/2113040317 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c011990 con 0x7f924c0b17e0 2023-08-27T11:25:57.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f924a7fc700 1 -- 172.21.15.112:0/2113040317 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c011d20 con 0x7f924c0b17e0 2023-08-27T11:25:57.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f9251f8d700 1 -- 172.21.15.112:0/2113040317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f924c0b17e0 msgr2=0x7f924c0cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f9251f8d700 1 --2- 172.21.15.112:0/2113040317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f924c0b17e0 0x7f924c0cf5a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f923c009840 tx=0x7f923c009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f9251f8d700 1 -- 172.21.15.112:0/2113040317 shutdown_connections 2023-08-27T11:25:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f9251f8d700 1 --2- 172.21.15.112:0/2113040317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f924c161ce0 0x7f924c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f9251f8d700 1 --2- 172.21.15.112:0/2113040317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f924c0b17e0 0x7f924c0cf5a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f9251f8d700 1 -- 172.21.15.112:0/2113040317 >> 172.21.15.112:0/2113040317 conn(0x7f924c0d0000 msgr2=0x7f924c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f9251f8d700 1 -- 172.21.15.112:0/2113040317 shutdown_connections 2023-08-27T11:25:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.724+0000 7f9251f8d700 1 -- 172.21.15.112:0/2113040317 wait complete. 2023-08-27T11:25:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.726+0000 7f9251f8d700 1 Processor -- start 2023-08-27T11:25:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.726+0000 7f9251f8d700 1 -- start start 2023-08-27T11:25:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.726+0000 7f9251f8d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f924c0b1400 0x7f924c146480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.726+0000 7f9251f8d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f924c0b17e0 0x7f924c0c3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.726+0000 7f9251f8d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c161ce0 0x7f924c155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.726+0000 7f9251f8d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c1f7240 con 0x7f924c161ce0 2023-08-27T11:25:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.726+0000 7f9251f8d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c1469c0 con 0x7f924c0b17e0 2023-08-27T11:25:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.726+0000 7f9251f8d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0c6a10 con 0x7f924c0b1400 2023-08-27T11:25:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f924bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c161ce0 0x7f924c155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f924bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c161ce0 0x7f924c155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60324/0 (socket says 172.21.15.112:60324) 2023-08-27T11:25:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f924bfff700 1 -- 172.21.15.112:0/2060307924 learned_addr learned my addr 172.21.15.112:0/2060307924 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:57.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f924affd700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f924c0b17e0 0x7f924c0c3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:57.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f924b7fe700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f924c0b1400 0x7f924c146480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:57.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f924bfff700 1 -- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f924c0b1400 msgr2=0x7f924c146480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f924bfff700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f924c0b1400 0x7f924c146480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f924bfff700 1 -- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f924c0b17e0 msgr2=0x7f924c0c3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f924bfff700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f924c0b17e0 0x7f924c0c3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f924bfff700 1 -- 172.21.15.112:0/2060307924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9244009b30 con 0x7f924c161ce0 2023-08-27T11:25:57.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f924bfff700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c161ce0 0x7f924c155670 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f9244009e70 tx=0x7f924400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:57.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f9243fff700 1 -- 172.21.15.112:0/2060307924 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244018070 con 0x7f924c161ce0 2023-08-27T11:25:57.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f9243fff700 1 -- 172.21.15.112:0/2060307924 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924400f040 con 0x7f924c161ce0 2023-08-27T11:25:57.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f9251f8d700 1 -- 172.21.15.112:0/2060307924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923c0094a0 con 0x7f924c161ce0 2023-08-27T11:25:57.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f9243fff700 1 -- 172.21.15.112:0/2060307924 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244003c40 con 0x7f924c161ce0 2023-08-27T11:25:57.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.727+0000 7f9251f8d700 1 -- 172.21.15.112:0/2060307924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924c1f68a0 con 0x7f924c161ce0 2023-08-27T11:25:57.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.728+0000 7f9251f8d700 1 -- 172.21.15.112:0/2060307924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924c1f7380 con 0x7f924c161ce0 2023-08-27T11:25:57.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.729+0000 7f9243fff700 1 -- 172.21.15.112:0/2060307924 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7f924400c880 con 0x7f924c161ce0 2023-08-27T11:25:57.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.729+0000 7f9243fff700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f922c06a320 0x7f922c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:57.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.729+0000 7f9243fff700 1 -- 172.21.15.112:0/2060307924 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f9244088b90 con 0x7f924c161ce0 2023-08-27T11:25:57.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.729+0000 7f924b7fe700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f922c06a320 0x7f922c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:57.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.730+0000 7f924b7fe700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f922c06a320 0x7f922c06c7e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9234007fd0 tx=0x7f9234007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:57.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.733+0000 7f9243fff700 1 -- 172.21.15.112:0/2060307924 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f924c1f7380 con 0x7f924c161ce0 2023-08-27T11:25:57.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.924+0000 7f9251f8d700 1 -- 172.21.15.112:0/2060307924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f924c1f7380 con 0x7f924c161ce0 2023-08-27T11:25:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.925+0000 7f9243fff700 1 -- 172.21.15.112:0/2060307924 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18453 (secure 0 0 0) 0x7f924c1f7380 con 0x7f924c161ce0 2023-08-27T11:25:57.927 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:25:57.927 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":79,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:25:54.909142+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:39.495757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:40.765773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.2d","mappings":[{"from":1,"to":7}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:57.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.927+0000 7f9251f8d700 1 -- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f922c06a320 msgr2=0x7f922c06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.927+0000 7f9251f8d700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f922c06a320 0x7f922c06c7e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9234007fd0 tx=0x7f9234007de0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.927+0000 7f9251f8d700 1 -- 172.21.15.112:0/2060307924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c161ce0 msgr2=0x7f924c155670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:57.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.927+0000 7f9251f8d700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c161ce0 0x7f924c155670 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f9244009e70 tx=0x7f924400c3a0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.927+0000 7f9251f8d700 1 -- 172.21.15.112:0/2060307924 shutdown_connections 2023-08-27T11:25:57.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.927+0000 7f9251f8d700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f922c06a320 0x7f922c06c7e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.927+0000 7f9251f8d700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c161ce0 0x7f924c155670 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.927+0000 7f9251f8d700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f924c0b1400 0x7f924c146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.927+0000 7f9251f8d700 1 --2- 172.21.15.112:0/2060307924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f924c0b17e0 0x7f924c0c3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:57.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.927+0000 7f9251f8d700 1 -- 172.21.15.112:0/2060307924 >> 172.21.15.112:0/2060307924 conn(0x7f924c0d0000 msgr2=0x7f924c15eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:57.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.927+0000 7f9251f8d700 1 -- 172.21.15.112:0/2060307924 shutdown_connections 2023-08-27T11:25:57.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:57.927+0000 7f9251f8d700 1 -- 172.21.15.112:0/2060307924 wait complete. 2023-08-27T11:25:57.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:25:39.495757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:25:40.765773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:25:57.942 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:25:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.063+0000 7fb70bdba700 1 Processor -- start 2023-08-27T11:25:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.063+0000 7fb70bdba700 1 -- start start 2023-08-27T11:25:58.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.064+0000 7fb70bdba700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb704146be0 0x7fb70406b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.064+0000 7fb70bdba700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb704161790 0x7fb7040af950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.064+0000 7fb70bdba700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb70406bc10 con 0x7fb70406b7b0 2023-08-27T11:25:58.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.064+0000 7fb70bdba700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70415dac0 con 0x7fb704146be0 2023-08-27T11:25:58.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.064+0000 7fb70bdba700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70415dd30 con 0x7fb704161790 2023-08-27T11:25:58.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.064+0000 7fb709b56700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb704146be0 0x7fb70406b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.064+0000 7fb709b56700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb704146be0 0x7fb70406b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60328/0 (socket says 172.21.15.112:60328) 2023-08-27T11:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.064+0000 7fb709b56700 1 -- 172.21.15.112:0/3518403048 learned_addr learned my addr 172.21.15.112:0/3518403048 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.065+0000 7fb70a357700 1 --2- 172.21.15.112:0/3518403048 >> v2:172.21.15.138:3300/0 conn(0x7fb704161790 0x7fb7040af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.065+0000 7fb709b56700 1 -- 172.21.15.112:0/3518403048 >> v2:172.21.15.138:3300/0 conn(0x7fb704161790 msgr2=0x7fb7040af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.065+0000 7fb709b56700 1 --2- 172.21.15.112:0/3518403048 >> v2:172.21.15.138:3300/0 conn(0x7fb704161790 0x7fb7040af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.065+0000 7fb709b56700 1 -- 172.21.15.112:0/3518403048 >> v1:172.21.15.138:6789/0 conn(0x7fb70406b7b0 legacy=0x7fb704161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.065+0000 7fb709b56700 1 -- 172.21.15.112:0/3518403048 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70415e010 con 0x7fb704146be0 2023-08-27T11:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.065+0000 7fb70a357700 1 --2- 172.21.15.112:0/3518403048 >> v2:172.21.15.138:3300/0 conn(0x7fb704161790 0x7fb7040af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.065+0000 7fb709b56700 1 --2- 172.21.15.112:0/3518403048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704146be0 0x7fb70406b270 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb70406e830 tx=0x7fb6f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e705397f73f29c76 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.065+0000 7fb708b54700 1 -- 172.21.15.112:0/3518403048 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4004230 con 0x7fb704146be0 2023-08-27T11:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.065+0000 7fb708b54700 1 -- 172.21.15.112:0/3518403048 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f4004390 con 0x7fb704146be0 2023-08-27T11:25:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.065+0000 7fb708b54700 1 -- 172.21.15.112:0/3518403048 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40036a0 con 0x7fb704146be0 2023-08-27T11:25:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.065+0000 7fb70bdba700 1 -- 172.21.15.112:0/3518403048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704146be0 msgr2=0x7fb70406b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.065+0000 7fb70bdba700 1 --2- 172.21.15.112:0/3518403048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704146be0 0x7fb70406b270 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb70406e830 tx=0x7fb6f4009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.066+0000 7fb70bdba700 1 -- 172.21.15.112:0/3518403048 shutdown_connections 2023-08-27T11:25:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.066+0000 7fb70bdba700 1 --2- 172.21.15.112:0/3518403048 >> v2:172.21.15.138:3300/0 conn(0x7fb704161790 0x7fb7040af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.066+0000 7fb70bdba700 1 --2- 172.21.15.112:0/3518403048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704146be0 0x7fb70406b270 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.066+0000 7fb70bdba700 1 -- 172.21.15.112:0/3518403048 >> 172.21.15.112:0/3518403048 conn(0x7fb7040d03e0 msgr2=0x7fb7040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.066+0000 7fb70bdba700 1 -- 172.21.15.112:0/3518403048 shutdown_connections 2023-08-27T11:25:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.066+0000 7fb70bdba700 1 -- 172.21.15.112:0/3518403048 wait complete. 2023-08-27T11:25:58.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.067+0000 7fb70bdba700 1 Processor -- start 2023-08-27T11:25:58.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.067+0000 7fb70bdba700 1 -- start start 2023-08-27T11:25:58.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.067+0000 7fb70bdba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb70406b7b0 0x7fb7041f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.067+0000 7fb70bdba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704146be0 0x7fb7040cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.067+0000 7fb70bdba700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb704161790 0x7fb7040719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.067+0000 7fb70bdba700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040ce3d0 con 0x7fb704146be0 2023-08-27T11:25:58.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.067+0000 7fb70bdba700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040cd7c0 con 0x7fb70406b7b0 2023-08-27T11:25:58.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.067+0000 7fb70bdba700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040cdaa0 con 0x7fb704161790 2023-08-27T11:25:58.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb709355700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704146be0 0x7fb7040cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb709355700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704146be0 0x7fb7040cf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60334/0 (socket says 172.21.15.112:60334) 2023-08-27T11:25:58.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb709355700 1 -- 172.21.15.112:0/3594040815 learned_addr learned my addr 172.21.15.112:0/3594040815 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:58.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb709b56700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb70406b7b0 0x7fb7041f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb70a357700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb704161790 0x7fb7040719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb709355700 1 -- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb704161790 msgr2=0x7fb7040719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb709355700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb704161790 0x7fb7040719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb709355700 1 -- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb70406b7b0 msgr2=0x7fb7041f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb709355700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb70406b7b0 0x7fb7041f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb709355700 1 -- 172.21.15.112:0/3594040815 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f4009550 con 0x7fb704146be0 2023-08-27T11:25:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb70a357700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb704161790 0x7fb7040719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb709355700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704146be0 0x7fb7040cf0c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb70000b4a0 tx=0x7fb70000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb6fa7fc700 1 -- 172.21.15.112:0/3594040815 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700011830 con 0x7fb704146be0 2023-08-27T11:25:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb6fa7fc700 1 -- 172.21.15.112:0/3594040815 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb700011990 con 0x7fb704146be0 2023-08-27T11:25:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb70bdba700 1 -- 172.21.15.112:0/3594040815 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7040cdd80 con 0x7fb704146be0 2023-08-27T11:25:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb6fa7fc700 1 -- 172.21.15.112:0/3594040815 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700011d40 con 0x7fb704146be0 2023-08-27T11:25:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.068+0000 7fb70bdba700 1 -- 172.21.15.112:0/3594040815 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7041f5c40 con 0x7fb704146be0 2023-08-27T11:25:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.069+0000 7fb70bdba700 1 -- 172.21.15.112:0/3594040815 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb704067e90 con 0x7fb704146be0 2023-08-27T11:25:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.070+0000 7fb6fa7fc700 1 -- 172.21.15.112:0/3594040815 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 88545+0+0 (secure 0 0 0) 0x7fb70001a400 con 0x7fb704146be0 2023-08-27T11:25:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.070+0000 7fb6fa7fc700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6e806a320 0x7fb6e806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.070+0000 7fb6fa7fc700 1 -- 172.21.15.112:0/3594040815 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fb700088710 con 0x7fb704146be0 2023-08-27T11:25:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.070+0000 7fb709b56700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6e806a320 0x7fb6e806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.071+0000 7fb709b56700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6e806a320 0x7fb6e806c7e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009130 tx=0x7fb6f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:58.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.074+0000 7fb6fa7fc700 1 -- 172.21.15.112:0/3594040815 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb700053a20 con 0x7fb704146be0 2023-08-27T11:25:58.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.262+0000 7fb70bdba700 1 -- 172.21.15.112:0/3594040815 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb704067a20 con 0x7fb704146be0 2023-08-27T11:25:58.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.262+0000 7fb6fa7fc700 1 -- 172.21.15.112:0/3594040815 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v79) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb700057040 con 0x7fb704146be0 2023-08-27T11:25:58.264 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:25:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.264+0000 7fb70bdba700 1 -- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6e806a320 msgr2=0x7fb6e806c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.264+0000 7fb70bdba700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6e806a320 0x7fb6e806c7e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009130 tx=0x7fb6f4008030 comp rx=0 tx=0).stop 2023-08-27T11:25:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.264+0000 7fb70bdba700 1 -- 172.21.15.112:0/3594040815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704146be0 msgr2=0x7fb7040cf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.264+0000 7fb70bdba700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704146be0 0x7fb7040cf0c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb70000b4a0 tx=0x7fb70000b7b0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.264+0000 7fb70bdba700 1 -- 172.21.15.112:0/3594040815 shutdown_connections 2023-08-27T11:25:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.264+0000 7fb70bdba700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6e806a320 0x7fb6e806c7e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.264+0000 7fb70bdba700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb704161790 0x7fb7040719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.264+0000 7fb70bdba700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb70406b7b0 0x7fb7041f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.264+0000 7fb70bdba700 1 --2- 172.21.15.112:0/3594040815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704146be0 0x7fb7040cf0c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.264+0000 7fb70bdba700 1 -- 172.21.15.112:0/3594040815 >> 172.21.15.112:0/3594040815 conn(0x7fb7040d03e0 msgr2=0x7fb704145e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:58.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.264+0000 7fb70bdba700 1 -- 172.21.15.112:0/3594040815 shutdown_connections 2023-08-27T11:25:58.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.264+0000 7fb70bdba700 1 -- 172.21.15.112:0/3594040815 wait complete. 2023-08-27T11:25:58.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:25:58.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.401+0000 7ff6322a7700 1 Processor -- start 2023-08-27T11:25:58.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.401+0000 7ff6322a7700 1 -- start start 2023-08-27T11:25:58.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.401+0000 7ff6322a7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff62c146c20 0x7ff62c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.401+0000 7ff6322a7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff62c06f790 0x7ff62c06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.401+0000 7ff6322a7700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62c0b5070 con 0x7ff62c0b3440 2023-08-27T11:25:58.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.401+0000 7ff6322a7700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62c070180 con 0x7ff62c146c20 2023-08-27T11:25:58.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.401+0000 7ff6322a7700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62c147000 con 0x7ff62c06f790 2023-08-27T11:25:58.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.402+0000 7ff62bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff62c146c20 0x7ff62c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.402+0000 7ff62bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff62c146c20 0x7ff62c160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60340/0 (socket says 172.21.15.112:60340) 2023-08-27T11:25:58.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.402+0000 7ff62bfff700 1 -- 172.21.15.112:0/3377177682 learned_addr learned my addr 172.21.15.112:0/3377177682 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:58.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.402+0000 7ff630844700 1 --2- 172.21.15.112:0/3377177682 >> v2:172.21.15.138:3300/0 conn(0x7ff62c06f790 0x7ff62c06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.402+0000 7ff62bfff700 1 -- 172.21.15.112:0/3377177682 >> v2:172.21.15.138:3300/0 conn(0x7ff62c06f790 msgr2=0x7ff62c06fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:58.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.402+0000 7ff62bfff700 1 --2- 172.21.15.112:0/3377177682 >> v2:172.21.15.138:3300/0 conn(0x7ff62c06f790 0x7ff62c06fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.402+0000 7ff62bfff700 1 -- 172.21.15.112:0/3377177682 >> v1:172.21.15.138:6789/0 conn(0x7ff62c0b3440 legacy=0x7ff62c15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:58.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.402+0000 7ff62bfff700 1 -- 172.21.15.112:0/3377177682 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c1614a0 con 0x7ff62c146c20 2023-08-27T11:25:58.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.402+0000 7ff62bfff700 1 --2- 172.21.15.112:0/3377177682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff62c146c20 0x7ff62c160f60 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0b1c80 tx=0x7ff61c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6d39b17c1be3fd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:58.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.402+0000 7ff62affd700 1 -- 172.21.15.112:0/3377177682 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c004230 con 0x7ff62c146c20 2023-08-27T11:25:58.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.402+0000 7ff62affd700 1 -- 172.21.15.112:0/3377177682 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61c004390 con 0x7ff62c146c20 2023-08-27T11:25:58.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.402+0000 7ff62affd700 1 -- 172.21.15.112:0/3377177682 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c0036a0 con 0x7ff62c146c20 2023-08-27T11:25:58.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.403+0000 7ff6322a7700 1 -- 172.21.15.112:0/3377177682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff62c146c20 msgr2=0x7ff62c160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.403+0000 7ff6322a7700 1 --2- 172.21.15.112:0/3377177682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff62c146c20 0x7ff62c160f60 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0b1c80 tx=0x7ff61c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.403+0000 7ff6322a7700 1 -- 172.21.15.112:0/3377177682 shutdown_connections 2023-08-27T11:25:58.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.403+0000 7ff6322a7700 1 --2- 172.21.15.112:0/3377177682 >> v2:172.21.15.138:3300/0 conn(0x7ff62c06f790 0x7ff62c06fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.403+0000 7ff6322a7700 1 --2- 172.21.15.112:0/3377177682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff62c146c20 0x7ff62c160f60 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.403+0000 7ff6322a7700 1 -- 172.21.15.112:0/3377177682 >> 172.21.15.112:0/3377177682 conn(0x7ff62c0b1e90 msgr2=0x7ff62c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:58.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.403+0000 7ff6322a7700 1 -- 172.21.15.112:0/3377177682 shutdown_connections 2023-08-27T11:25:58.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.403+0000 7ff6322a7700 1 -- 172.21.15.112:0/3377177682 wait complete. 2023-08-27T11:25:58.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.404+0000 7ff6322a7700 1 Processor -- start 2023-08-27T11:25:58.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.404+0000 7ff6322a7700 1 -- start start 2023-08-27T11:25:58.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff6322a7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c06f790 0x7ff62c132c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff6322a7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c0b3440 0x7ff62c0c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff6322a7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff62c146c20 0x7ff62c1f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff6322a7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c0c5630 con 0x7ff62c146c20 2023-08-27T11:25:58.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff6322a7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c0c6470 con 0x7ff62c0b3440 2023-08-27T11:25:58.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff6322a7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c0c5860 con 0x7ff62c06f790 2023-08-27T11:25:58.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff630844700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff62c146c20 0x7ff62c1f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff630844700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff62c146c20 0x7ff62c1f2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60348/0 (socket says 172.21.15.112:60348) 2023-08-27T11:25:58.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff630844700 1 -- 172.21.15.112:0/2685792301 learned_addr learned my addr 172.21.15.112:0/2685792301 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:58.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff62bfff700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c06f790 0x7ff62c132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff62b7fe700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c0b3440 0x7ff62c0c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff630844700 1 -- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c06f790 msgr2=0x7ff62c132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff630844700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c06f790 0x7ff62c132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff630844700 1 -- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c0b3440 msgr2=0x7ff62c0c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff630844700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c0b3440 0x7ff62c0c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff630844700 1 -- 172.21.15.112:0/2685792301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c009550 con 0x7ff62c146c20 2023-08-27T11:25:58.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff62b7fe700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c0b3440 0x7ff62c0c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:58.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff630844700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff62c146c20 0x7ff62c1f2460 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff624007e40 tx=0x7ff62400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:58.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.405+0000 7ff628ff9700 1 -- 172.21.15.112:0/2685792301 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62400ede0 con 0x7ff62c146c20 2023-08-27T11:25:58.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.406+0000 7ff628ff9700 1 -- 172.21.15.112:0/2685792301 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff624014420 con 0x7ff62c146c20 2023-08-27T11:25:58.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.406+0000 7ff6322a7700 1 -- 172.21.15.112:0/2685792301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62c0c5b40 con 0x7ff62c146c20 2023-08-27T11:25:58.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.406+0000 7ff628ff9700 1 -- 172.21.15.112:0/2685792301 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240147d0 con 0x7ff62c146c20 2023-08-27T11:25:58.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.406+0000 7ff6322a7700 1 -- 172.21.15.112:0/2685792301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62c0c6100 con 0x7ff62c146c20 2023-08-27T11:25:58.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.406+0000 7ff6322a7700 1 -- 172.21.15.112:0/2685792301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62c067e90 con 0x7ff62c146c20 2023-08-27T11:25:58.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.407+0000 7ff628ff9700 1 -- 172.21.15.112:0/2685792301 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7ff624014930 con 0x7ff62c146c20 2023-08-27T11:25:58.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.407+0000 7ff628ff9700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff61806a3e0 0x7ff61806c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.407+0000 7ff628ff9700 1 -- 172.21.15.112:0/2685792301 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7ff624089190 con 0x7ff62c146c20 2023-08-27T11:25:58.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.408+0000 7ff62bfff700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff61806a3e0 0x7ff61806c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.408+0000 7ff62bfff700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff61806a3e0 0x7ff61806c8a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009130 tx=0x7ff61c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:58.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.412+0000 7ff628ff9700 1 -- 172.21.15.112:0/2685792301 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff624057eb0 con 0x7ff62c146c20 2023-08-27T11:25:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.603+0000 7ff6322a7700 1 -- 172.21.15.112:0/2685792301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff62c067a20 con 0x7ff62c146c20 2023-08-27T11:25:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.604+0000 7ff628ff9700 1 -- 172.21.15.112:0/2685792301 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff624057a40 con 0x7ff62c146c20 2023-08-27T11:25:58.606 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:25:58.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.606+0000 7ff6322a7700 1 -- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff61806a3e0 msgr2=0x7ff61806c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.606+0000 7ff6322a7700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff61806a3e0 0x7ff61806c8a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009130 tx=0x7ff61c008030 comp rx=0 tx=0).stop 2023-08-27T11:25:58.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.606+0000 7ff6322a7700 1 -- 172.21.15.112:0/2685792301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff62c146c20 msgr2=0x7ff62c1f2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.606+0000 7ff6322a7700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff62c146c20 0x7ff62c1f2460 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff624007e40 tx=0x7ff62400e3c0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.606+0000 7ff6322a7700 1 -- 172.21.15.112:0/2685792301 shutdown_connections 2023-08-27T11:25:58.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.606+0000 7ff6322a7700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff61806a3e0 0x7ff61806c8a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.606+0000 7ff6322a7700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff62c146c20 0x7ff62c1f2460 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.606+0000 7ff6322a7700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c06f790 0x7ff62c132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.606+0000 7ff6322a7700 1 --2- 172.21.15.112:0/2685792301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c0b3440 0x7ff62c0c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.606+0000 7ff6322a7700 1 -- 172.21.15.112:0/2685792301 >> 172.21.15.112:0/2685792301 conn(0x7ff62c0b1e90 msgr2=0x7ff62c071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.606+0000 7ff6322a7700 1 -- 172.21.15.112:0/2685792301 shutdown_connections 2023-08-27T11:25:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.606+0000 7ff6322a7700 1 -- 172.21.15.112:0/2685792301 wait complete. 2023-08-27T11:25:58.619 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:25:58.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.742+0000 7fbf744ec700 1 Processor -- start 2023-08-27T11:25:58.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.742+0000 7fbf744ec700 1 -- start start 2023-08-27T11:25:58.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.743+0000 7fbf744ec700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbf6c0b3de0 0x7fbf6c06bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.743+0000 7fbf744ec700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6c0cef20 con 0x7fbf6c06c510 2023-08-27T11:25:58.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.743+0000 7fbf744ec700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6c0b7310 con 0x7fbf6c144b80 2023-08-27T11:25:58.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.743+0000 7fbf744ec700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c0b4b40 con 0x7fbf6c0b3de0 2023-08-27T11:25:58.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.743+0000 7fbf72a89700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbf6c06c510 0x7fbf6c0cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54736/0 (socket says 172.21.15.112:54736) 2023-08-27T11:25:58.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.743+0000 7fbf72a89700 1 -- 172.21.15.112:0/3259510028 learned_addr learned my addr 172.21.15.112:0/3259510028 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:58.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.743+0000 7fbf71a87700 1 --2- 172.21.15.112:0/3259510028 >> v2:172.21.15.138:3300/0 conn(0x7fbf6c0b3de0 0x7fbf6c06bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.743+0000 7fbf71286700 1 -- 172.21.15.112:0/3259510028 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282454054 0 0) 0x7fbf6c0cef20 con 0x7fbf6c06c510 2023-08-27T11:25:58.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.743+0000 7fbf71286700 1 -- 172.21.15.112:0/3259510028 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf580035b0 con 0x7fbf6c06c510 2023-08-27T11:25:58.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.743+0000 7fbf71286700 1 -- 172.21.15.112:0/3259510028 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3797472251 0 0) 0x7fbf580035b0 con 0x7fbf6c06c510 2023-08-27T11:25:58.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.743+0000 7fbf71a87700 1 -- 172.21.15.112:0/3259510028 >> v1:172.21.15.138:6789/0 conn(0x7fbf6c144b80 legacy=0x7fbf6c0b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.743+0000 7fbf71a87700 1 -- 172.21.15.112:0/3259510028 >> v1:172.21.15.112:6789/0 conn(0x7fbf6c06c510 legacy=0x7fbf6c0cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.743+0000 7fbf71a87700 1 -- 172.21.15.112:0/3259510028 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c06c910 con 0x7fbf6c0b3de0 2023-08-27T11:25:58.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.744+0000 7fbf71a87700 1 --2- 172.21.15.112:0/3259510028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf6c0b3de0 0x7fbf6c06bfd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009840 tx=0x7fbf5c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f615643e14a2aaf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:58.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.744+0000 7fbf71286700 1 -- 172.21.15.112:0/3259510028 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf640019f0 con 0x7fbf6c0b3de0 2023-08-27T11:25:58.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.744+0000 7fbf71286700 1 -- 172.21.15.112:0/3259510028 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c004030 con 0x7fbf6c0b3de0 2023-08-27T11:25:58.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.744+0000 7fbf71286700 1 -- 172.21.15.112:0/3259510028 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c0043c0 con 0x7fbf6c0b3de0 2023-08-27T11:25:58.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.744+0000 7fbf744ec700 1 -- 172.21.15.112:0/3259510028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf6c0b3de0 msgr2=0x7fbf6c06bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.744+0000 7fbf744ec700 1 --2- 172.21.15.112:0/3259510028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf6c0b3de0 0x7fbf6c06bfd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009840 tx=0x7fbf5c009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:58.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.744+0000 7fbf744ec700 1 -- 172.21.15.112:0/3259510028 shutdown_connections 2023-08-27T11:25:58.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.744+0000 7fbf744ec700 1 --2- 172.21.15.112:0/3259510028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf6c0b3de0 0x7fbf6c06bfd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.744+0000 7fbf744ec700 1 -- 172.21.15.112:0/3259510028 >> 172.21.15.112:0/3259510028 conn(0x7fbf6c0cdc50 msgr2=0x7fbf6c0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:58.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.744+0000 7fbf744ec700 1 -- 172.21.15.112:0/3259510028 shutdown_connections 2023-08-27T11:25:58.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.744+0000 7fbf744ec700 1 -- 172.21.15.112:0/3259510028 wait complete. 2023-08-27T11:25:58.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.746+0000 7fbf744ec700 1 Processor -- start 2023-08-27T11:25:58.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.746+0000 7fbf744ec700 1 -- start start 2023-08-27T11:25:58.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.746+0000 7fbf744ec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf6c06c510 0x7fbf6c140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.746+0000 7fbf744ec700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf6c0b3de0 0x7fbf6c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.746+0000 7fbf744ec700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf6c144b80 0x7fbf6c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.746+0000 7fbf744ec700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c158aa0 con 0x7fbf6c06c510 2023-08-27T11:25:58.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.746+0000 7fbf744ec700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c140fe0 con 0x7fbf6c0b3de0 2023-08-27T11:25:58.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.746+0000 7fbf744ec700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c15a220 con 0x7fbf6c144b80 2023-08-27T11:25:58.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.746+0000 7fbf72288700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf6c06c510 0x7fbf6c140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.746+0000 7fbf72288700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf6c06c510 0x7fbf6c140aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60358/0 (socket says 172.21.15.112:60358) 2023-08-27T11:25:58.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.746+0000 7fbf72288700 1 -- 172.21.15.112:0/1273937196 learned_addr learned my addr 172.21.15.112:0/1273937196 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:58.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf71a87700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf6c0b3de0 0x7fbf6c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf72a89700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf6c144b80 0x7fbf6c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf72288700 1 -- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf6c144b80 msgr2=0x7fbf6c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf72288700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf6c144b80 0x7fbf6c0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf72288700 1 -- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf6c0b3de0 msgr2=0x7fbf6c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf72288700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf6c0b3de0 0x7fbf6c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf72288700 1 -- 172.21.15.112:0/1273937196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c0094a0 con 0x7fbf6c06c510 2023-08-27T11:25:58.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf71a87700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf6c0b3de0 0x7fbf6c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:58.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf72288700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf6c06c510 0x7fbf6c140aa0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800c480 tx=0x7fbf6800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:58.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf62ffd700 1 -- 172.21.15.112:0/1273937196 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf680052d0 con 0x7fbf6c06c510 2023-08-27T11:25:58.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf62ffd700 1 -- 172.21.15.112:0/1273937196 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf68005430 con 0x7fbf6c06c510 2023-08-27T11:25:58.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf744ec700 1 -- 172.21.15.112:0/1273937196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6c15a500 con 0x7fbf6c06c510 2023-08-27T11:25:58.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf62ffd700 1 -- 172.21.15.112:0/1273937196 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf680038c0 con 0x7fbf6c06c510 2023-08-27T11:25:58.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.747+0000 7fbf744ec700 1 -- 172.21.15.112:0/1273937196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6c15aac0 con 0x7fbf6c06c510 2023-08-27T11:25:58.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.748+0000 7fbf744ec700 1 -- 172.21.15.112:0/1273937196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6c067a20 con 0x7fbf6c06c510 2023-08-27T11:25:58.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.748+0000 7fbf62ffd700 1 -- 172.21.15.112:0/1273937196 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fbf68003a20 con 0x7fbf6c06c510 2023-08-27T11:25:58.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.749+0000 7fbf62ffd700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf5006a3b0 0x7fbf5006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:58.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.749+0000 7fbf62ffd700 1 -- 172.21.15.112:0/1273937196 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fbf68087ef0 con 0x7fbf6c06c510 2023-08-27T11:25:58.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.749+0000 7fbf71a87700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf5006a3b0 0x7fbf5006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:58.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.750+0000 7fbf71a87700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf5006a3b0 0x7fbf5006c870 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0090b0 tx=0x7fbf5c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:58.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.753+0000 7fbf62ffd700 1 -- 172.21.15.112:0/1273937196 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbf680531d0 con 0x7fbf6c06c510 2023-08-27T11:25:58.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.941+0000 7fbf744ec700 1 -- 172.21.15.112:0/1273937196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf6c0b2b30 con 0x7fbf6c06c510 2023-08-27T11:25:58.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.942+0000 7fbf62ffd700 1 -- 172.21.15.112:0/1273937196 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf680567f0 con 0x7fbf6c06c510 2023-08-27T11:25:58.944 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:25:58.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.944+0000 7fbf744ec700 1 -- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf5006a3b0 msgr2=0x7fbf5006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.944+0000 7fbf744ec700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf5006a3b0 0x7fbf5006c870 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0090b0 tx=0x7fbf5c008030 comp rx=0 tx=0).stop 2023-08-27T11:25:58.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.944+0000 7fbf744ec700 1 -- 172.21.15.112:0/1273937196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf6c06c510 msgr2=0x7fbf6c140aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:58.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.944+0000 7fbf744ec700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf6c06c510 0x7fbf6c140aa0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800c480 tx=0x7fbf6800c840 comp rx=0 tx=0).stop 2023-08-27T11:25:58.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.944+0000 7fbf744ec700 1 -- 172.21.15.112:0/1273937196 shutdown_connections 2023-08-27T11:25:58.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.944+0000 7fbf744ec700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf5006a3b0 0x7fbf5006c870 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.944+0000 7fbf744ec700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf6c144b80 0x7fbf6c0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.944+0000 7fbf744ec700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf6c06c510 0x7fbf6c140aa0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.944+0000 7fbf744ec700 1 --2- 172.21.15.112:0/1273937196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf6c0b3de0 0x7fbf6c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:58.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.944+0000 7fbf744ec700 1 -- 172.21.15.112:0/1273937196 >> 172.21.15.112:0/1273937196 conn(0x7fbf6c0cdc50 msgr2=0x7fbf6c0b68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:58.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.944+0000 7fbf744ec700 1 -- 172.21.15.112:0/1273937196 shutdown_connections 2023-08-27T11:25:58.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:58.944+0000 7fbf744ec700 1 -- 172.21.15.112:0/1273937196 wait complete. 2023-08-27T11:25:58.958 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:25:59.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.080+0000 7f0a9e7ed700 1 Processor -- start 2023-08-27T11:25:59.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.080+0000 7f0a9e7ed700 1 -- start start 2023-08-27T11:25:59.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a9e7ed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a98146c20 0x7f0a98160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a9e7ed700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0a980b3440 0x7f0a9815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a9e7ed700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0a9815e6e0 0x7f0a9806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a9e7ed700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a980b5070 con 0x7f0a980b3440 2023-08-27T11:25:59.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a9e7ed700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a98147000 con 0x7f0a9815e6e0 2023-08-27T11:25:59.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a9e7ed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a981614a0 con 0x7f0a98146c20 2023-08-27T11:25:59.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a977fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0a980b3440 0x7f0a9815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a977fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0a980b3440 0x7f0a9815e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60366/0 (socket says 172.21.15.112:60366) 2023-08-27T11:25:59.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a977fe700 1 -- 172.21.15.112:0/3783130500 learned_addr learned my addr 172.21.15.112:0/3783130500 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:59.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a9cd8a700 1 --2- 172.21.15.112:0/3783130500 >> v2:172.21.15.138:3300/0 conn(0x7f0a9815e6e0 0x7f0a9806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a97fff700 1 --2- 172.21.15.112:0/3783130500 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a98146c20 0x7f0a98160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a977fe700 1 -- 172.21.15.112:0/3783130500 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a98146c20 msgr2=0x7f0a98160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:59.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a977fe700 1 --2- 172.21.15.112:0/3783130500 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a98146c20 0x7f0a98160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a977fe700 1 -- 172.21.15.112:0/3783130500 >> v2:172.21.15.138:3300/0 conn(0x7f0a9815e6e0 msgr2=0x7f0a9806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:59.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a977fe700 1 --2- 172.21.15.112:0/3783130500 >> v2:172.21.15.138:3300/0 conn(0x7f0a9815e6e0 0x7f0a9806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a977fe700 1 -- 172.21.15.112:0/3783130500 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a98161740 con 0x7f0a980b3440 2023-08-27T11:25:59.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a97fff700 1 --2- 172.21.15.112:0/3783130500 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a98146c20 0x7f0a98160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:59.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a977fe700 1 --2- 172.21.15.112:0/3783130500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a980b3440 0x7f0a9815e1a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0a980b1c80 tx=0x7f0a80009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8323ffdad45716b8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:59.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.081+0000 7f0a96ffd700 1 -- 172.21.15.112:0/3783130500 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80011830 con 0x7f0a980b3440 2023-08-27T11:25:59.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.082+0000 7f0a96ffd700 1 -- 172.21.15.112:0/3783130500 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a80011990 con 0x7f0a980b3440 2023-08-27T11:25:59.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.082+0000 7f0a96ffd700 1 -- 172.21.15.112:0/3783130500 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80011d20 con 0x7f0a980b3440 2023-08-27T11:25:59.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.082+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/3783130500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a980b3440 msgr2=0x7f0a9815e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.082+0000 7f0a9e7ed700 1 --2- 172.21.15.112:0/3783130500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a980b3440 0x7f0a9815e1a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0a980b1c80 tx=0x7f0a80009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.082+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/3783130500 shutdown_connections 2023-08-27T11:25:59.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.082+0000 7f0a9e7ed700 1 --2- 172.21.15.112:0/3783130500 >> v2:172.21.15.138:3300/0 conn(0x7f0a9815e6e0 0x7f0a9806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.082+0000 7f0a9e7ed700 1 --2- 172.21.15.112:0/3783130500 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a98146c20 0x7f0a98160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.082+0000 7f0a9e7ed700 1 --2- 172.21.15.112:0/3783130500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a980b3440 0x7f0a9815e1a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.082+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/3783130500 >> 172.21.15.112:0/3783130500 conn(0x7f0a980b1e90 msgr2=0x7f0a980b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:59.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.082+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/3783130500 shutdown_connections 2023-08-27T11:25:59.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.082+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/3783130500 wait complete. 2023-08-27T11:25:59.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.083+0000 7f0a9e7ed700 1 Processor -- start 2023-08-27T11:25:59.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.083+0000 7f0a9e7ed700 1 -- start start 2023-08-27T11:25:59.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.084+0000 7f0a9e7ed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a980b3440 0x7f0a9813db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.084+0000 7f0a9e7ed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a98146c20 0x7f0a980ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.084+0000 7f0a9e7ed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a9815e6e0 0x7f0a9815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.084+0000 7f0a9e7ed700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9815d800 con 0x7f0a98146c20 2023-08-27T11:25:59.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.084+0000 7f0a9e7ed700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a981f3530 con 0x7f0a980b3440 2023-08-27T11:25:59.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.084+0000 7f0a9e7ed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9815bcd0 con 0x7f0a9815e6e0 2023-08-27T11:25:59.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.084+0000 7f0a977fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a98146c20 0x7f0a980ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.084+0000 7f0a977fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a98146c20 0x7f0a980ad690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60372/0 (socket says 172.21.15.112:60372) 2023-08-27T11:25:59.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.084+0000 7f0a977fe700 1 -- 172.21.15.112:0/2131550510 learned_addr learned my addr 172.21.15.112:0/2131550510 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:59.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.085+0000 7f0a977fe700 1 -- 172.21.15.112:0/2131550510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a9815e6e0 msgr2=0x7f0a9815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.085+0000 7f0a977fe700 1 --2- 172.21.15.112:0/2131550510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a9815e6e0 0x7f0a9815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.085+0000 7f0a977fe700 1 -- 172.21.15.112:0/2131550510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a980b3440 msgr2=0x7f0a9813db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.085+0000 7f0a977fe700 1 --2- 172.21.15.112:0/2131550510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a980b3440 0x7f0a9813db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.085+0000 7f0a977fe700 1 -- 172.21.15.112:0/2131550510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a80009550 con 0x7f0a98146c20 2023-08-27T11:25:59.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.085+0000 7f0a977fe700 1 --2- 172.21.15.112:0/2131550510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a98146c20 0x7f0a980ad690 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009c10 tx=0x7f0a80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:59.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.085+0000 7f0a94ff9700 1 -- 172.21.15.112:0/2131550510 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80004590 con 0x7f0a98146c20 2023-08-27T11:25:59.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.085+0000 7f0a94ff9700 1 -- 172.21.15.112:0/2131550510 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8001e070 con 0x7f0a98146c20 2023-08-27T11:25:59.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.085+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/2131550510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9815bfb0 con 0x7f0a98146c20 2023-08-27T11:25:59.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.085+0000 7f0a94ff9700 1 -- 172.21.15.112:0/2131550510 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a800038e0 con 0x7f0a98146c20 2023-08-27T11:25:59.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.085+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/2131550510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a981f3810 con 0x7f0a98146c20 2023-08-27T11:25:59.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.086+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/2131550510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9815bb30 con 0x7f0a98146c20 2023-08-27T11:25:59.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.086+0000 7f0a94ff9700 1 -- 172.21.15.112:0/2131550510 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f0a80003a40 con 0x7f0a98146c20 2023-08-27T11:25:59.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.087+0000 7f0a94ff9700 1 --2- 172.21.15.112:0/2131550510 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a7806a3b0 0x7f0a7806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.087+0000 7f0a94ff9700 1 -- 172.21.15.112:0/2131550510 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f0a800898a0 con 0x7f0a98146c20 2023-08-27T11:25:59.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.087+0000 7f0a97fff700 1 --2- 172.21.15.112:0/2131550510 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a7806a3b0 0x7f0a7806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.087+0000 7f0a97fff700 1 --2- 172.21.15.112:0/2131550510 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a7806a3b0 0x7f0a7806c870 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0a88006fd0 tx=0x7f0a88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:59.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.091+0000 7f0a94ff9700 1 -- 172.21.15.112:0/2131550510 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0a9815bb30 con 0x7f0a98146c20 2023-08-27T11:25:59.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.278+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/2131550510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a9815c4d0 con 0x7f0a98146c20 2023-08-27T11:25:59.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.279+0000 7f0a94ff9700 1 -- 172.21.15.112:0/2131550510 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a9815c4d0 con 0x7f0a98146c20 2023-08-27T11:25:59.281 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:25:59.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.281+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/2131550510 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a7806a3b0 msgr2=0x7f0a7806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.281+0000 7f0a9e7ed700 1 --2- 172.21.15.112:0/2131550510 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a7806a3b0 0x7f0a7806c870 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0a88006fd0 tx=0x7f0a88008040 comp rx=0 tx=0).stop 2023-08-27T11:25:59.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.281+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/2131550510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a98146c20 msgr2=0x7f0a980ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.281+0000 7f0a9e7ed700 1 --2- 172.21.15.112:0/2131550510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a98146c20 0x7f0a980ad690 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009c10 tx=0x7f0a80004750 comp rx=0 tx=0).stop 2023-08-27T11:25:59.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.281+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/2131550510 shutdown_connections 2023-08-27T11:25:59.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.281+0000 7f0a9e7ed700 1 --2- 172.21.15.112:0/2131550510 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0a7806a3b0 0x7f0a7806c870 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.281+0000 7f0a9e7ed700 1 --2- 172.21.15.112:0/2131550510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a9815e6e0 0x7f0a9815c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.281+0000 7f0a9e7ed700 1 --2- 172.21.15.112:0/2131550510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a980b3440 0x7f0a9813db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.281+0000 7f0a9e7ed700 1 --2- 172.21.15.112:0/2131550510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a98146c20 0x7f0a980ad690 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.281+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/2131550510 >> 172.21.15.112:0/2131550510 conn(0x7f0a980b1e90 msgr2=0x7f0a980b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:59.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.281+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/2131550510 shutdown_connections 2023-08-27T11:25:59.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.281+0000 7f0a9e7ed700 1 -- 172.21.15.112:0/2131550510 wait complete. 2023-08-27T11:25:59.295 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:25:59.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.414+0000 7f5632550700 1 Processor -- start 2023-08-27T11:25:59.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.414+0000 7f5632550700 1 -- start start 2023-08-27T11:25:59.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.415+0000 7f5632550700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f562c151060 0x7f562c15e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.415+0000 7f5632550700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f562c15e990 0x7f562c06adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.415+0000 7f5632550700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c06b710 con 0x7f562c06b2f0 2023-08-27T11:25:59.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.415+0000 7f5632550700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562c0b02f0 con 0x7f562c151060 2023-08-27T11:25:59.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.415+0000 7f5632550700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562c0b05d0 con 0x7f562c15e990 2023-08-27T11:25:59.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.415+0000 7f562bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f562c151060 0x7f562c15e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.415+0000 7f562bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f562c151060 0x7f562c15e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60380/0 (socket says 172.21.15.112:60380) 2023-08-27T11:25:59.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.415+0000 7f562bfff700 1 -- 172.21.15.112:0/250677225 learned_addr learned my addr 172.21.15.112:0/250677225 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:59.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.415+0000 7f562b7fe700 1 --2- 172.21.15.112:0/250677225 >> v2:172.21.15.138:3300/0 conn(0x7f562c15e990 0x7f562c06adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f562bfff700 1 -- 172.21.15.112:0/250677225 >> v2:172.21.15.138:3300/0 conn(0x7f562c15e990 msgr2=0x7f562c06adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:59.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f562bfff700 1 --2- 172.21.15.112:0/250677225 >> v2:172.21.15.138:3300/0 conn(0x7f562c15e990 0x7f562c06adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f562bfff700 1 -- 172.21.15.112:0/250677225 >> v1:172.21.15.138:6789/0 conn(0x7f562c06b2f0 legacy=0x7f562c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f562bfff700 1 -- 172.21.15.112:0/250677225 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c0b08b0 con 0x7f562c151060 2023-08-27T11:25:59.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f562bfff700 1 --2- 172.21.15.112:0/250677225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f562c151060 0x7f562c15e450 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f562c06ccb0 tx=0x7f5620009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c77b5b75cbceb1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:59.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f562affd700 1 -- 172.21.15.112:0/250677225 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620011830 con 0x7f562c151060 2023-08-27T11:25:59.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f562affd700 1 -- 172.21.15.112:0/250677225 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5620011990 con 0x7f562c151060 2023-08-27T11:25:59.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f562affd700 1 -- 172.21.15.112:0/250677225 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620011d20 con 0x7f562c151060 2023-08-27T11:25:59.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f5632550700 1 -- 172.21.15.112:0/250677225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f562c151060 msgr2=0x7f562c15e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f5632550700 1 --2- 172.21.15.112:0/250677225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f562c151060 0x7f562c15e450 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f562c06ccb0 tx=0x7f5620009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f5632550700 1 -- 172.21.15.112:0/250677225 shutdown_connections 2023-08-27T11:25:59.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f5632550700 1 --2- 172.21.15.112:0/250677225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f562c151060 0x7f562c15e450 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f5632550700 1 --2- 172.21.15.112:0/250677225 >> v2:172.21.15.138:3300/0 conn(0x7f562c15e990 0x7f562c06adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f5632550700 1 -- 172.21.15.112:0/250677225 >> 172.21.15.112:0/250677225 conn(0x7f562c0d0000 msgr2=0x7f562c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:59.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f5632550700 1 -- 172.21.15.112:0/250677225 shutdown_connections 2023-08-27T11:25:59.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.416+0000 7f5632550700 1 -- 172.21.15.112:0/250677225 wait complete. 2023-08-27T11:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.418+0000 7f5632550700 1 Processor -- start 2023-08-27T11:25:59.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.418+0000 7f5632550700 1 -- start start 2023-08-27T11:25:59.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.418+0000 7f5632550700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f562c06b2f0 0x7f562c0af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.418+0000 7f5632550700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f562c151060 0x7f562c0b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f5632550700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f562c15e990 0x7f562c0c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f5632550700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c1f6a10 con 0x7f562c06b2f0 2023-08-27T11:25:59.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f5632550700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c1f6cf0 con 0x7f562c151060 2023-08-27T11:25:59.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f5632550700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c1f6fd0 con 0x7f562c15e990 2023-08-27T11:25:59.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f562bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f562c06b2f0 0x7f562c0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f562bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f562c06b2f0 0x7f562c0af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60384/0 (socket says 172.21.15.112:60384) 2023-08-27T11:25:59.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f562bfff700 1 -- 172.21.15.112:0/1160097833 learned_addr learned my addr 172.21.15.112:0/1160097833 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:59.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f562b7fe700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f562c151060 0x7f562c0b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f5630aed700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f562c15e990 0x7f562c0c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f562bfff700 1 -- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f562c15e990 msgr2=0x7f562c0c28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f562bfff700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f562c15e990 0x7f562c0c28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f562bfff700 1 -- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f562c151060 msgr2=0x7f562c0b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f562bfff700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f562c151060 0x7f562c0b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f562bfff700 1 -- 172.21.15.112:0/1160097833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5620009550 con 0x7f562c06b2f0 2023-08-27T11:25:59.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f5630aed700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f562c15e990 0x7f562c0c28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:59.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f562bfff700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f562c06b2f0 0x7f562c0af180 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5620009c10 tx=0x7f5620004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:59.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f5628ff9700 1 -- 172.21.15.112:0/1160097833 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620004590 con 0x7f562c06b2f0 2023-08-27T11:25:59.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f5628ff9700 1 -- 172.21.15.112:0/1160097833 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562001e070 con 0x7f562c06b2f0 2023-08-27T11:25:59.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f5632550700 1 -- 172.21.15.112:0/1160097833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562c1f72b0 con 0x7f562c06b2f0 2023-08-27T11:25:59.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f5628ff9700 1 -- 172.21.15.112:0/1160097833 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56200038e0 con 0x7f562c06b2f0 2023-08-27T11:25:59.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.419+0000 7f5632550700 1 -- 172.21.15.112:0/1160097833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562c1f7870 con 0x7f562c06b2f0 2023-08-27T11:25:59.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.420+0000 7f5632550700 1 -- 172.21.15.112:0/1160097833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562c067a20 con 0x7f562c06b2f0 2023-08-27T11:25:59.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.421+0000 7f5628ff9700 1 -- 172.21.15.112:0/1160097833 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f5620003a40 con 0x7f562c06b2f0 2023-08-27T11:25:59.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.421+0000 7f5628ff9700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f561806a3b0 0x7f561806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.421+0000 7f5628ff9700 1 -- 172.21.15.112:0/1160097833 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f5620089860 con 0x7f562c06b2f0 2023-08-27T11:25:59.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.421+0000 7f562b7fe700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f561806a3b0 0x7f561806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.422+0000 7f562b7fe700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f561806a3b0 0x7f561806c870 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f562c1f6970 tx=0x7f561c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:59.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.425+0000 7f5628ff9700 1 -- 172.21.15.112:0/1160097833 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5620054c10 con 0x7f562c06b2f0 2023-08-27T11:25:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.633+0000 7f5632550700 1 -- 172.21.15.112:0/1160097833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f562c0ca6f0 con 0x7f562c06b2f0 2023-08-27T11:25:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.633+0000 7f5628ff9700 1 -- 172.21.15.112:0/1160097833 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v79) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5620058230 con 0x7f562c06b2f0 2023-08-27T11:25:59.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.635+0000 7f5632550700 1 -- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f561806a3b0 msgr2=0x7f561806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.635+0000 7f5632550700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f561806a3b0 0x7f561806c870 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f562c1f6970 tx=0x7f561c009210 comp rx=0 tx=0).stop 2023-08-27T11:25:59.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.635+0000 7f5632550700 1 -- 172.21.15.112:0/1160097833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f562c06b2f0 msgr2=0x7f562c0af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.635+0000 7f5632550700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f562c06b2f0 0x7f562c0af180 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5620009c10 tx=0x7f5620004750 comp rx=0 tx=0).stop 2023-08-27T11:25:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.635+0000 7f5632550700 1 -- 172.21.15.112:0/1160097833 shutdown_connections 2023-08-27T11:25:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.635+0000 7f5632550700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f561806a3b0 0x7f561806c870 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.635+0000 7f5632550700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f562c15e990 0x7f562c0c28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.635+0000 7f5632550700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f562c06b2f0 0x7f562c0af180 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.635+0000 7f5632550700 1 --2- 172.21.15.112:0/1160097833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f562c151060 0x7f562c0b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.635+0000 7f5632550700 1 -- 172.21.15.112:0/1160097833 >> 172.21.15.112:0/1160097833 conn(0x7f562c0d0000 msgr2=0x7f562c071950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:59.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.635+0000 7f5632550700 1 -- 172.21.15.112:0/1160097833 shutdown_connections 2023-08-27T11:25:59.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.635+0000 7f5632550700 1 -- 172.21.15.112:0/1160097833 wait complete. 2023-08-27T11:25:59.639 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:25:59.649 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:25:59.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:59.650 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:25:59.697 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:25:59.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:59.698 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:25:59.715 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:25:59.716 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:59.735 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:25:59.736 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:59.795 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:59.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.915+0000 7f654ce13700 1 Processor -- start 2023-08-27T11:25:59.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.915+0000 7f654ce13700 1 -- start start 2023-08-27T11:25:59.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654ce13700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65480b1400 0x7f65480cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654ce13700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6548161850 con 0x7f6548161ce0 2023-08-27T11:25:59.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654ce13700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65480cfc10 con 0x7f65480b17e0 2023-08-27T11:25:59.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654ce13700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6548161200 con 0x7f65480b1400 2023-08-27T11:25:59.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f6546d9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6548161ce0 0x7f654806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54776/0 (socket says 172.21.15.112:54776) 2023-08-27T11:25:59.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f6546d9d700 1 -- 172.21.15.112:0/60837206 learned_addr learned my addr 172.21.15.112:0/60837206 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:25:59.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912924577 0 0) 0x7f6548161850 con 0x7f6548161ce0 2023-08-27T11:25:59.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654659c700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65480b1400 0x7f65480cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65300035b0 con 0x7f6548161ce0 2023-08-27T11:25:59.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1590129653 0 0) 0x7f65300035b0 con 0x7f6548161ce0 2023-08-27T11:25:59.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65480b1400 msgr2=0x7f65480cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:59.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654559a700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65480b1400 0x7f65480cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 >> v1:172.21.15.138:6789/0 conn(0x7f65480b17e0 legacy=0x7f654806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65481614e0 con 0x7f6548161ce0 2023-08-27T11:25:59.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f65400019f0 con 0x7f6548161ce0 2023-08-27T11:25:59.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 >> v1:172.21.15.112:6789/0 conn(0x7f6548161ce0 legacy=0x7f654806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654559a700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6530003a30 0x7f6530003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.916+0000 7f654559a700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6530004850 0x7f6530004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f654559a700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6530005260 0x7f653000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6548161200 con 0x7f6530005260 2023-08-27T11:25:59.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653000c160 con 0x7f6530003a30 2023-08-27T11:25:59.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653000c320 con 0x7f6530004850 2023-08-27T11:25:59.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f654659c700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65480b1400 0x7f65480cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:59.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f6546d9d700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6530005260 0x7f653000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f6545d9b700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6530004850 0x7f6530004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f654659c700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6530003a30 0x7f6530003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f6546d9d700 1 -- 172.21.15.112:0/60837206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6530004850 msgr2=0x7f6530004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:59.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f6546d9d700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6530004850 0x7f6530004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f6546d9d700 1 -- 172.21.15.112:0/60837206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6530003a30 msgr2=0x7f6530003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:59.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f6546d9d700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6530003a30 0x7f6530003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f6546d9d700 1 -- 172.21.15.112:0/60837206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65300035b0 con 0x7f6530005260 2023-08-27T11:25:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f654659c700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6530003a30 0x7f6530003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f6546d9d700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6530005260 0x7f653000ba10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f654000a830 tx=0x7f6530005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bc1cc6b71adee4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654000a2f0 con 0x7f6530005260 2023-08-27T11:25:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6540012620 con 0x7f6530005260 2023-08-27T11:25:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.917+0000 7f654559a700 1 -- 172.21.15.112:0/60837206 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65400129d0 con 0x7f6530005260 2023-08-27T11:25:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.918+0000 7f654ce13700 1 -- 172.21.15.112:0/60837206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6530005260 msgr2=0x7f653000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.918+0000 7f654ce13700 1 --2- 172.21.15.112:0/60837206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6530005260 0x7f653000ba10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f654000a830 tx=0x7f6530005880 comp rx=0 tx=0).stop 2023-08-27T11:25:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.918+0000 7f6546d9d700 1 -- 172.21.15.112:0/60837206 reap_dead start 2023-08-27T11:25:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.918+0000 7f654ce13700 1 -- 172.21.15.112:0/60837206 shutdown_connections 2023-08-27T11:25:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.918+0000 7f654ce13700 1 -- 172.21.15.112:0/60837206 >> 172.21.15.112:0/60837206 conn(0x7f65480d0000 msgr2=0x7f65480d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:59.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.918+0000 7f654ce13700 1 -- 172.21.15.112:0/60837206 shutdown_connections 2023-08-27T11:25:59.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.918+0000 7f654ce13700 1 -- 172.21.15.112:0/60837206 wait complete. 2023-08-27T11:25:59.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.920+0000 7f654ce13700 1 Processor -- start 2023-08-27T11:25:59.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.920+0000 7f654ce13700 1 -- start start 2023-08-27T11:25:59.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654ce13700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6530005260 0x7f6548150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654ce13700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65480714a0 0x7f65480c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654ce13700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65480c9950 0x7f65480c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654ce13700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65480ca340 con 0x7f6530005260 2023-08-27T11:25:59.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654ce13700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6548150c70 con 0x7f65480714a0 2023-08-27T11:25:59.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654ce13700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65480719a0 con 0x7f65480c9950 2023-08-27T11:25:59.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6530005260 0x7f6548150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6530005260 0x7f6548150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60404/0 (socket says 172.21.15.112:60404) 2023-08-27T11:25:59.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654659c700 1 -- 172.21.15.112:0/1757084918 learned_addr learned my addr 172.21.15.112:0/1757084918 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:25:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f6545d9b700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65480714a0 0x7f65480c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654659c700 1 -- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65480c9950 msgr2=0x7f65480c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f6546d9d700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65480c9950 0x7f65480c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654659c700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65480c9950 0x7f65480c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654659c700 1 -- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65480714a0 msgr2=0x7f65480c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:59.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654659c700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65480714a0 0x7f65480c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:59.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654659c700 1 -- 172.21.15.112:0/1757084918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6548071c80 con 0x7f6530005260 2023-08-27T11:25:59.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654659c700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6530005260 0x7f6548150730 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f653c00f4c0 tx=0x7f653c00f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:59.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f6536ffd700 1 -- 172.21.15.112:0/1757084918 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c00c070 con 0x7f6530005260 2023-08-27T11:25:59.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f6536ffd700 1 -- 172.21.15.112:0/1757084918 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653c014890 con 0x7f6530005260 2023-08-27T11:25:59.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654ce13700 1 -- 172.21.15.112:0/1757084918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6548071f60 con 0x7f6530005260 2023-08-27T11:25:59.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f6536ffd700 1 -- 172.21.15.112:0/1757084918 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c014c00 con 0x7f6530005260 2023-08-27T11:25:59.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.921+0000 7f654ce13700 1 -- 172.21.15.112:0/1757084918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6548151040 con 0x7f6530005260 2023-08-27T11:25:59.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.922+0000 7f654ce13700 1 -- 172.21.15.112:0/1757084918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65481513b0 con 0x7f6530005260 2023-08-27T11:25:59.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.923+0000 7f6536ffd700 1 -- 172.21.15.112:0/1757084918 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f653c01d400 con 0x7f6530005260 2023-08-27T11:25:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.923+0000 7f6536ffd700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f652806a3b0 0x7f652806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.923+0000 7f6536ffd700 1 -- 172.21.15.112:0/1757084918 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f653c088f60 con 0x7f6530005260 2023-08-27T11:25:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.923+0000 7f6545d9b700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f652806a3b0 0x7f652806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.924+0000 7f6545d9b700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f652806a3b0 0x7f652806c870 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6548070a40 tx=0x7f6538009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:25:59.927+0000 7f6536ffd700 1 -- 172.21.15.112:0/1757084918 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f653c057b20 con 0x7f6530005260 2023-08-27T11:26:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.116+0000 7f654ce13700 1 -- 172.21.15.112:0/1757084918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65480cf560 con 0x7f6530005260 2023-08-27T11:26:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.117+0000 7f6536ffd700 1 -- 172.21.15.112:0/1757084918 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18453 (secure 0 0 0) 0x7f653c0576b0 con 0x7f6530005260 2023-08-27T11:26:00.119 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:00.119 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":79,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:25:54.909142+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:39.495757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:40.765773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.2d","mappings":[{"from":1,"to":7}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:00.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.119+0000 7f654ce13700 1 -- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f652806a3b0 msgr2=0x7f652806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.119+0000 7f654ce13700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f652806a3b0 0x7f652806c870 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6548070a40 tx=0x7f6538009040 comp rx=0 tx=0).stop 2023-08-27T11:26:00.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.119+0000 7f654ce13700 1 -- 172.21.15.112:0/1757084918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6530005260 msgr2=0x7f6548150730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.119+0000 7f654ce13700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6530005260 0x7f6548150730 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f653c00f4c0 tx=0x7f653c00f880 comp rx=0 tx=0).stop 2023-08-27T11:26:00.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.119+0000 7f654ce13700 1 -- 172.21.15.112:0/1757084918 shutdown_connections 2023-08-27T11:26:00.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.119+0000 7f654ce13700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f652806a3b0 0x7f652806c870 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.119+0000 7f654ce13700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65480c9950 0x7f65480c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.119+0000 7f654ce13700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6530005260 0x7f6548150730 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.119+0000 7f654ce13700 1 --2- 172.21.15.112:0/1757084918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65480714a0 0x7f65480c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.119+0000 7f654ce13700 1 -- 172.21.15.112:0/1757084918 >> 172.21.15.112:0/1757084918 conn(0x7f65480d0000 msgr2=0x7f65480cb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:00.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.119+0000 7f654ce13700 1 -- 172.21.15.112:0/1757084918 shutdown_connections 2023-08-27T11:26:00.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.119+0000 7f654ce13700 1 -- 172.21.15.112:0/1757084918 wait complete. 2023-08-27T11:26:00.133 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:25:39.495757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:25:40.765773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:26:00.134 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:26:00.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.255+0000 7fb10f4b3700 1 Processor -- start 2023-08-27T11:26:00.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.255+0000 7fb10f4b3700 1 -- start start 2023-08-27T11:26:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb10f4b3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb10815ed00 0x7fb1080b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb10f4b3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb10815f180 0x7fb1080b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb10f4b3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10815f750 con 0x7fb108152130 2023-08-27T11:26:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb10f4b3700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10815fa30 con 0x7fb10815ed00 2023-08-27T11:26:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb10f4b3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080b60a0 con 0x7fb10815f180 2023-08-27T11:26:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb10d24f700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb108152130 0x7fb10815c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54794/0 (socket says 172.21.15.112:54794) 2023-08-27T11:26:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb10d24f700 1 -- 172.21.15.112:0/1385548005 learned_addr learned my addr 172.21.15.112:0/1385548005 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb10da50700 1 --2- 172.21.15.112:0/1385548005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb10815f180 0x7fb1080b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb10ca4e700 1 --2- 172.21.15.112:0/1385548005 >> v2:172.21.15.138:3300/0 conn(0x7fb10815ed00 0x7fb1080b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493089128 0 0) 0x7fb10815f750 con 0x7fb108152130 2023-08-27T11:26:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f40035b0 con 0x7fb108152130 2023-08-27T11:26:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3946205536 0 0) 0x7fb0f40035b0 con 0x7fb108152130 2023-08-27T11:26:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb10815f180 msgr2=0x7fb1080b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 --2- 172.21.15.112:0/1385548005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb10815f180 0x7fb1080b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 >> v2:172.21.15.138:3300/0 conn(0x7fb10815ed00 msgr2=0x7fb1080b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 --2- 172.21.15.112:0/1385548005 >> v2:172.21.15.138:3300/0 conn(0x7fb10815ed00 0x7fb1080b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1080b6380 con 0x7fb108152130 2023-08-27T11:26:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fb104001670 con 0x7fb108152130 2023-08-27T11:26:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 >> v1:172.21.15.112:6789/0 conn(0x7fb108152130 legacy=0x7fb10815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 --2- 172.21.15.112:0/1385548005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f4003a00 0x7fb0f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 --2- 172.21.15.112:0/1385548005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb10815f180 0x7fb0f4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 --2- 172.21.15.112:0/1385548005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f4004d90 0x7fb0f4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10815fa30 con 0x7fb0f4003a00 2023-08-27T11:26:00.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080b60a0 con 0x7fb0f4004d90 2023-08-27T11:26:00.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.256+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f400ba10 con 0x7fb10815f180 2023-08-27T11:26:00.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.257+0000 7fb10d24f700 1 --2- 172.21.15.112:0/1385548005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f4003a00 0x7fb0f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.257+0000 7fb10ca4e700 1 --2- 172.21.15.112:0/1385548005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb10815f180 0x7fb0f4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.257+0000 7fb10da50700 1 --2- 172.21.15.112:0/1385548005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f4004d90 0x7fb0f4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.257+0000 7fb10d24f700 1 -- 172.21.15.112:0/1385548005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb10815f180 msgr2=0x7fb0f4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:00.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.257+0000 7fb10d24f700 1 --2- 172.21.15.112:0/1385548005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb10815f180 0x7fb0f4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.257+0000 7fb10d24f700 1 -- 172.21.15.112:0/1385548005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f4004d90 msgr2=0x7fb0f4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:00.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.257+0000 7fb10d24f700 1 --2- 172.21.15.112:0/1385548005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f4004d90 0x7fb0f4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.257+0000 7fb10d24f700 1 -- 172.21.15.112:0/1385548005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f40035b0 con 0x7fb0f4003a00 2023-08-27T11:26:00.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.257+0000 7fb10da50700 1 --2- 172.21.15.112:0/1385548005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f4004d90 0x7fb0f4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:00.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.257+0000 7fb10d24f700 1 --2- 172.21.15.112:0/1385548005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f4003a00 0x7fb0f4003e20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb10400c370 tx=0x7fb0f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c5868732f1fcc5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:00.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.257+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104007c60 con 0x7fb0f4003a00 2023-08-27T11:26:00.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.257+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb104005960 con 0x7fb0f4003a00 2023-08-27T11:26:00.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.257+0000 7fb0fffff700 1 -- 172.21.15.112:0/1385548005 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104005d10 con 0x7fb0f4003a00 2023-08-27T11:26:00.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.258+0000 7fb10f4b3700 1 -- 172.21.15.112:0/1385548005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f4003a00 msgr2=0x7fb0f4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.258+0000 7fb10f4b3700 1 --2- 172.21.15.112:0/1385548005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f4003a00 0x7fb0f4003e20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb10400c370 tx=0x7fb0f4005880 comp rx=0 tx=0).stop 2023-08-27T11:26:00.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.258+0000 7fb10da50700 1 -- 172.21.15.112:0/1385548005 reap_dead start 2023-08-27T11:26:00.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.258+0000 7fb10f4b3700 1 -- 172.21.15.112:0/1385548005 shutdown_connections 2023-08-27T11:26:00.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.258+0000 7fb10f4b3700 1 -- 172.21.15.112:0/1385548005 >> 172.21.15.112:0/1385548005 conn(0x7fb108066010 msgr2=0x7fb1080663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:00.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.258+0000 7fb10f4b3700 1 -- 172.21.15.112:0/1385548005 shutdown_connections 2023-08-27T11:26:00.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.258+0000 7fb10f4b3700 1 -- 172.21.15.112:0/1385548005 wait complete. 2023-08-27T11:26:00.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.259+0000 7fb10f4b3700 1 Processor -- start 2023-08-27T11:26:00.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.259+0000 7fb10f4b3700 1 -- start start 2023-08-27T11:26:00.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10f4b3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f4003a00 0x7fb108151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10f4b3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb10815d0b0 0x7fb10815d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10f4b3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb108153920 0x7fb108153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10f4b3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10815da70 con 0x7fb0f4003a00 2023-08-27T11:26:00.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10f4b3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb108155240 con 0x7fb10815d0b0 2023-08-27T11:26:00.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10f4b3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1081556d0 con 0x7fb108153920 2023-08-27T11:26:00.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10d24f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f4003a00 0x7fb108151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10d24f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f4003a00 0x7fb108151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60424/0 (socket says 172.21.15.112:60424) 2023-08-27T11:26:00.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10d24f700 1 -- 172.21.15.112:0/3152279528 learned_addr learned my addr 172.21.15.112:0/3152279528 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:00.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10ca4e700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb10815d0b0 0x7fb10815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10da50700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb108153920 0x7fb108153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10d24f700 1 -- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb108153920 msgr2=0x7fb108153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10d24f700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb108153920 0x7fb108153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10d24f700 1 -- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb10815d0b0 msgr2=0x7fb10815d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10d24f700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb10815d0b0 0x7fb10815d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10d24f700 1 -- 172.21.15.112:0/3152279528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb108154630 con 0x7fb0f4003a00 2023-08-27T11:26:00.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10ca4e700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb10815d0b0 0x7fb10815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:00.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10d24f700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f4003a00 0x7fb108151290 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb104000c00 tx=0x7fb104000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:00.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb0fdffb700 1 -- 172.21.15.112:0/3152279528 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104004870 con 0x7fb0f4003a00 2023-08-27T11:26:00.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb0fdffb700 1 -- 172.21.15.112:0/3152279528 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1040049d0 con 0x7fb0f4003a00 2023-08-27T11:26:00.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb10f4b3700 1 -- 172.21.15.112:0/3152279528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb108154910 con 0x7fb0f4003a00 2023-08-27T11:26:00.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.260+0000 7fb0fdffb700 1 -- 172.21.15.112:0/3152279528 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104004d80 con 0x7fb0f4003a00 2023-08-27T11:26:00.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.261+0000 7fb10f4b3700 1 -- 172.21.15.112:0/3152279528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb108154ed0 con 0x7fb0f4003a00 2023-08-27T11:26:00.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.262+0000 7fb10f4b3700 1 -- 172.21.15.112:0/3152279528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb108067a20 con 0x7fb0f4003a00 2023-08-27T11:26:00.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.262+0000 7fb0fdffb700 1 -- 172.21.15.112:0/3152279528 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fb1040070d0 con 0x7fb0f4003a00 2023-08-27T11:26:00.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.262+0000 7fb0fdffb700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb0ec06a3b0 0x7fb0ec06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.262+0000 7fb0fdffb700 1 -- 172.21.15.112:0/3152279528 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fb10408a100 con 0x7fb0f4003a00 2023-08-27T11:26:00.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.262+0000 7fb10ca4e700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb0ec06a3b0 0x7fb0ec06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.263+0000 7fb10ca4e700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb0ec06a3b0 0x7fb0ec06c870 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb1080cd6f0 tx=0x7fb0f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:00.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.267+0000 7fb0fdffb700 1 -- 172.21.15.112:0/3152279528 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb1040554b0 con 0x7fb0f4003a00 2023-08-27T11:26:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.455+0000 7fb10f4b3700 1 -- 172.21.15.112:0/3152279528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb10815da70 con 0x7fb0f4003a00 2023-08-27T11:26:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.455+0000 7fb0fdffb700 1 -- 172.21.15.112:0/3152279528 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v79) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb104007dc0 con 0x7fb0f4003a00 2023-08-27T11:26:00.457 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:26:00.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.457+0000 7fb10f4b3700 1 -- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb0ec06a3b0 msgr2=0x7fb0ec06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.457+0000 7fb10f4b3700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb0ec06a3b0 0x7fb0ec06c870 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb1080cd6f0 tx=0x7fb0f8008040 comp rx=0 tx=0).stop 2023-08-27T11:26:00.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.457+0000 7fb10f4b3700 1 -- 172.21.15.112:0/3152279528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f4003a00 msgr2=0x7fb108151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.457+0000 7fb10f4b3700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f4003a00 0x7fb108151290 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb104000c00 tx=0x7fb104000fe0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.457+0000 7fb10f4b3700 1 -- 172.21.15.112:0/3152279528 shutdown_connections 2023-08-27T11:26:00.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.457+0000 7fb10f4b3700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb0ec06a3b0 0x7fb0ec06c870 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.457+0000 7fb10f4b3700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb108153920 0x7fb108153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.457+0000 7fb10f4b3700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f4003a00 0x7fb108151290 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.457+0000 7fb10f4b3700 1 --2- 172.21.15.112:0/3152279528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb10815d0b0 0x7fb10815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.457+0000 7fb10f4b3700 1 -- 172.21.15.112:0/3152279528 >> 172.21.15.112:0/3152279528 conn(0x7fb108066010 msgr2=0x7fb10815c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.457+0000 7fb10f4b3700 1 -- 172.21.15.112:0/3152279528 shutdown_connections 2023-08-27T11:26:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.457+0000 7fb10f4b3700 1 -- 172.21.15.112:0/3152279528 wait complete. 2023-08-27T11:26:00.471 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:26:00.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.593+0000 7f8a82717700 1 Processor -- start 2023-08-27T11:26:00.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.593+0000 7f8a82717700 1 -- start start 2023-08-27T11:26:00.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.594+0000 7f8a82717700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a7c1449e0 0x7f8a7c0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.594+0000 7f8a82717700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7c0ce5c0 con 0x7f8a7c15be30 2023-08-27T11:26:00.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.594+0000 7f8a82717700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7c0b1850 con 0x7f8a7c0b09f0 2023-08-27T11:26:00.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.594+0000 7f8a82717700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c15b750 con 0x7f8a7c1449e0 2023-08-27T11:26:00.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.594+0000 7f8a80cb4700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8a7c15be30 0x7f8a7c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54814/0 (socket says 172.21.15.112:54814) 2023-08-27T11:26:00.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.594+0000 7f8a80cb4700 1 -- 172.21.15.112:0/4240582544 learned_addr learned my addr 172.21.15.112:0/4240582544 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:00.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7bfff700 1 --2- 172.21.15.112:0/4240582544 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a7c1449e0 0x7f8a7c0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366180810 0 0) 0x7f8a7c0ce5c0 con 0x7f8a7c15be30 2023-08-27T11:26:00.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a640035b0 con 0x7f8a7c15be30 2023-08-27T11:26:00.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2119456218 0 0) 0x7f8a640035b0 con 0x7f8a7c15be30 2023-08-27T11:26:00.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a7c1449e0 msgr2=0x7f8a7c0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:00.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 --2- 172.21.15.112:0/4240582544 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a7c1449e0 0x7f8a7c0ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 >> v1:172.21.15.138:6789/0 conn(0x7f8a7c0b09f0 legacy=0x7f8a7c15c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c15b950 con 0x7f8a7c15be30 2023-08-27T11:26:00.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f8a740019f0 con 0x7f8a7c15be30 2023-08-27T11:26:00.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 >> v1:172.21.15.112:6789/0 conn(0x7f8a7c15be30 legacy=0x7f8a7c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 --2- 172.21.15.112:0/4240582544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a64003a30 0x7f8a64003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 --2- 172.21.15.112:0/4240582544 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a7c1449e0 0x7f8a64004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 --2- 172.21.15.112:0/4240582544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a64004d90 0x7f8a64005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c15b750 con 0x7f8a64004d90 2023-08-27T11:26:00.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6400ba10 con 0x7f8a64003a30 2023-08-27T11:26:00.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6400bbd0 con 0x7f8a7c1449e0 2023-08-27T11:26:00.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a80cb4700 1 --2- 172.21.15.112:0/4240582544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a64004d90 0x7f8a64005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7bfff700 1 --2- 172.21.15.112:0/4240582544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a64003a30 0x7f8a64003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a7b7fe700 1 --2- 172.21.15.112:0/4240582544 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a7c1449e0 0x7f8a64004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a80cb4700 1 -- 172.21.15.112:0/4240582544 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a7c1449e0 msgr2=0x7f8a64004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:00.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a80cb4700 1 --2- 172.21.15.112:0/4240582544 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a7c1449e0 0x7f8a64004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a80cb4700 1 -- 172.21.15.112:0/4240582544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a64003a30 msgr2=0x7f8a64003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:00.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a80cb4700 1 --2- 172.21.15.112:0/4240582544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a64003a30 0x7f8a64003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a80cb4700 1 -- 172.21.15.112:0/4240582544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a640035b0 con 0x7f8a64004d90 2023-08-27T11:26:00.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.595+0000 7f8a80cb4700 1 --2- 172.21.15.112:0/4240582544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a64004d90 0x7f8a64005210 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400a840 tx=0x7f8a64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f127efe125b075 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:00.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.596+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74012600 con 0x7f8a64004d90 2023-08-27T11:26:00.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.596+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a74012760 con 0x7f8a64004d90 2023-08-27T11:26:00.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.596+0000 7f8a7affd700 1 -- 172.21.15.112:0/4240582544 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74012af0 con 0x7f8a64004d90 2023-08-27T11:26:00.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.596+0000 7f8a82717700 1 -- 172.21.15.112:0/4240582544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a64004d90 msgr2=0x7f8a64005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.596+0000 7f8a82717700 1 --2- 172.21.15.112:0/4240582544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a64004d90 0x7f8a64005210 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400a840 tx=0x7f8a64005880 comp rx=0 tx=0).stop 2023-08-27T11:26:00.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.596+0000 7f8a80cb4700 1 -- 172.21.15.112:0/4240582544 reap_dead start 2023-08-27T11:26:00.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.596+0000 7f8a82717700 1 -- 172.21.15.112:0/4240582544 shutdown_connections 2023-08-27T11:26:00.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.596+0000 7f8a82717700 1 -- 172.21.15.112:0/4240582544 >> 172.21.15.112:0/4240582544 conn(0x7f8a7c0b1e90 msgr2=0x7f8a7c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:00.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.596+0000 7f8a82717700 1 -- 172.21.15.112:0/4240582544 shutdown_connections 2023-08-27T11:26:00.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.596+0000 7f8a82717700 1 -- 172.21.15.112:0/4240582544 wait complete. 2023-08-27T11:26:00.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.597+0000 7f8a82717700 1 Processor -- start 2023-08-27T11:26:00.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.598+0000 7f8a82717700 1 -- start start 2023-08-27T11:26:00.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.598+0000 7f8a82717700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a64004d90 0x7f8a7c1f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.598+0000 7f8a82717700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a7c0b4ed0 0x7f8a7c0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.598+0000 7f8a82717700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c0c9500 0x7f8a7c15d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.598+0000 7f8a82717700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c1f30e0 con 0x7f8a7c0c9500 2023-08-27T11:26:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.598+0000 7f8a82717700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c1f33c0 con 0x7f8a7c0b4ed0 2023-08-27T11:26:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.598+0000 7f8a82717700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c1f36a0 con 0x7f8a64004d90 2023-08-27T11:26:00.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.598+0000 7f8a80cb4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c0c9500 0x7f8a7c15d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a80cb4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c0c9500 0x7f8a7c15d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60448/0 (socket says 172.21.15.112:60448) 2023-08-27T11:26:00.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a80cb4700 1 -- 172.21.15.112:0/3073762878 learned_addr learned my addr 172.21.15.112:0/3073762878 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:00.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a7bfff700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a64004d90 0x7f8a7c1f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a7b7fe700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a7c0b4ed0 0x7f8a7c0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a80cb4700 1 -- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a64004d90 msgr2=0x7f8a7c1f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a80cb4700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a64004d90 0x7f8a7c1f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a80cb4700 1 -- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a7c0b4ed0 msgr2=0x7f8a7c0ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a80cb4700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a7c0b4ed0 0x7f8a7c0ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a80cb4700 1 -- 172.21.15.112:0/3073762878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c1f3980 con 0x7f8a7c0c9500 2023-08-27T11:26:00.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a7b7fe700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a7c0b4ed0 0x7f8a7c0ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:00.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a80cb4700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c0c9500 0x7f8a7c15d650 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8a74001cb0 tx=0x7f8a7c15bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:00.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a78ff9700 1 -- 172.21.15.112:0/3073762878 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a740019b0 con 0x7f8a7c0c9500 2023-08-27T11:26:00.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a78ff9700 1 -- 172.21.15.112:0/3073762878 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7400a280 con 0x7f8a7c0c9500 2023-08-27T11:26:00.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a82717700 1 -- 172.21.15.112:0/3073762878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7c0b34e0 con 0x7f8a7c0c9500 2023-08-27T11:26:00.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a78ff9700 1 -- 172.21.15.112:0/3073762878 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74027690 con 0x7f8a7c0c9500 2023-08-27T11:26:00.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.599+0000 7f8a82717700 1 -- 172.21.15.112:0/3073762878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7c0b5330 con 0x7f8a7c0c9500 2023-08-27T11:26:00.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.600+0000 7f8a82717700 1 -- 172.21.15.112:0/3073762878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7c067a20 con 0x7f8a7c0c9500 2023-08-27T11:26:00.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.600+0000 7f8a78ff9700 1 -- 172.21.15.112:0/3073762878 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f8a74024070 con 0x7f8a7c0c9500 2023-08-27T11:26:00.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.601+0000 7f8a78ff9700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a6806a3b0 0x7f8a6806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.601+0000 7f8a78ff9700 1 -- 172.21.15.112:0/3073762878 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f8a7408afc0 con 0x7f8a7c0c9500 2023-08-27T11:26:00.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.601+0000 7f8a7bfff700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a6806a3b0 0x7f8a6806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.601+0000 7f8a7bfff700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a6806a3b0 0x7f8a6806c870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c15e330 tx=0x7f8a70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:00.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.605+0000 7f8a78ff9700 1 -- 172.21.15.112:0/3073762878 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8a74056490 con 0x7f8a7c0c9500 2023-08-27T11:26:00.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.789+0000 7f8a82717700 1 -- 172.21.15.112:0/3073762878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a7c0b04b0 con 0x7f8a7c0c9500 2023-08-27T11:26:00.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.789+0000 7f8a78ff9700 1 -- 172.21.15.112:0/3073762878 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a7405f030 con 0x7f8a7c0c9500 2023-08-27T11:26:00.791 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:26:00.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.791+0000 7f8a82717700 1 -- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a6806a3b0 msgr2=0x7f8a6806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.791+0000 7f8a82717700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a6806a3b0 0x7f8a6806c870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c15e330 tx=0x7f8a70008040 comp rx=0 tx=0).stop 2023-08-27T11:26:00.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.792+0000 7f8a82717700 1 -- 172.21.15.112:0/3073762878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c0c9500 msgr2=0x7f8a7c15d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.792+0000 7f8a82717700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c0c9500 0x7f8a7c15d650 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8a74001cb0 tx=0x7f8a7c15bae0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.792+0000 7f8a82717700 1 -- 172.21.15.112:0/3073762878 shutdown_connections 2023-08-27T11:26:00.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.792+0000 7f8a82717700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a6806a3b0 0x7f8a6806c870 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.792+0000 7f8a82717700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c0c9500 0x7f8a7c15d650 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.792+0000 7f8a82717700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a64004d90 0x7f8a7c1f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.792+0000 7f8a82717700 1 --2- 172.21.15.112:0/3073762878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a7c0b4ed0 0x7f8a7c0ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.792+0000 7f8a82717700 1 -- 172.21.15.112:0/3073762878 >> 172.21.15.112:0/3073762878 conn(0x7f8a7c0b1e90 msgr2=0x7f8a7c15a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.792+0000 7f8a82717700 1 -- 172.21.15.112:0/3073762878 shutdown_connections 2023-08-27T11:26:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.792+0000 7f8a82717700 1 -- 172.21.15.112:0/3073762878 wait complete. 2023-08-27T11:26:00.805 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:26:00.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.926+0000 7fde5e2c8700 1 Processor -- start 2023-08-27T11:26:00.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.927+0000 7fde5e2c8700 1 -- start start 2023-08-27T11:26:00.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.927+0000 7fde5e2c8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fde580b3440 0x7fde5815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.927+0000 7fde5e2c8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde580b5070 con 0x7fde58146c20 2023-08-27T11:26:00.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.927+0000 7fde5e2c8700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde58147000 con 0x7fde5815e6e0 2023-08-27T11:26:00.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.927+0000 7fde5e2c8700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde58161740 con 0x7fde580b3440 2023-08-27T11:26:00.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde57fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fde58146c20 0x7fde58160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54834/0 (socket says 172.21.15.112:54834) 2023-08-27T11:26:00.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde57fff700 1 -- 172.21.15.112:0/1110575195 learned_addr learned my addr 172.21.15.112:0/1110575195 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:00.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714501920 0 0) 0x7fde48001110 con 0x7fde58146c20 2023-08-27T11:26:00.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde400035b0 con 0x7fde58146c20 2023-08-27T11:26:00.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde577fe700 1 --2- 172.21.15.112:0/1110575195 >> v2:172.21.15.138:3300/0 conn(0x7fde580b3440 0x7fde5815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 361819802 0 0) 0x7fde400035b0 con 0x7fde58146c20 2023-08-27T11:26:00.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 >> v2:172.21.15.138:3300/0 conn(0x7fde580b3440 msgr2=0x7fde5815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:00.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 --2- 172.21.15.112:0/1110575195 >> v2:172.21.15.138:3300/0 conn(0x7fde580b3440 0x7fde5815e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 >> v1:172.21.15.138:6789/0 conn(0x7fde5815e6e0 legacy=0x7fde5806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde58161a20 con 0x7fde58146c20 2023-08-27T11:26:00.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fde48001800 con 0x7fde58146c20 2023-08-27T11:26:00.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 >> v1:172.21.15.112:6789/0 conn(0x7fde58146c20 legacy=0x7fde58160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 --2- 172.21.15.112:0/1110575195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde40003a00 0x7fde40003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 --2- 172.21.15.112:0/1110575195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde40004850 0x7fde40004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 --2- 172.21.15.112:0/1110575195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde40005210 0x7fde4000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde58161740 con 0x7fde40003a00 2023-08-27T11:26:00.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4000c1b0 con 0x7fde40005210 2023-08-27T11:26:00.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4000c370 con 0x7fde40004850 2023-08-27T11:26:00.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde57fff700 1 --2- 172.21.15.112:0/1110575195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde40003a00 0x7fde40003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde577fe700 1 --2- 172.21.15.112:0/1110575195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde40004850 0x7fde40004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde5c865700 1 --2- 172.21.15.112:0/1110575195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde40005210 0x7fde4000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde57fff700 1 -- 172.21.15.112:0/1110575195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde40004850 msgr2=0x7fde40004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:00.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde57fff700 1 --2- 172.21.15.112:0/1110575195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde40004850 0x7fde40004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde57fff700 1 -- 172.21.15.112:0/1110575195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde40005210 msgr2=0x7fde4000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:00.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde57fff700 1 --2- 172.21.15.112:0/1110575195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde40005210 0x7fde4000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde57fff700 1 -- 172.21.15.112:0/1110575195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde400035b0 con 0x7fde40003a00 2023-08-27T11:26:00.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde5c865700 1 -- 172.21.15.112:0/1110575195 reap_dead start 2023-08-27T11:26:00.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.928+0000 7fde577fe700 1 --2- 172.21.15.112:0/1110575195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde40004850 0x7fde40004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:00.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.929+0000 7fde57fff700 1 --2- 172.21.15.112:0/1110575195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde40003a00 0x7fde40003e20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c240 tx=0x7fde40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81544e66867f6d55 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:00.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.929+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48007db0 con 0x7fde40003a00 2023-08-27T11:26:00.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.929+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4800ce00 con 0x7fde40003a00 2023-08-27T11:26:00.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.929+0000 7fde56ffd700 1 -- 172.21.15.112:0/1110575195 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48005ba0 con 0x7fde40003a00 2023-08-27T11:26:00.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.929+0000 7fde5e2c8700 1 -- 172.21.15.112:0/1110575195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde40003a00 msgr2=0x7fde40003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.929+0000 7fde5e2c8700 1 --2- 172.21.15.112:0/1110575195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde40003a00 0x7fde40003e20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c240 tx=0x7fde40005880 comp rx=0 tx=0).stop 2023-08-27T11:26:00.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.929+0000 7fde5e2c8700 1 -- 172.21.15.112:0/1110575195 shutdown_connections 2023-08-27T11:26:00.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.929+0000 7fde5e2c8700 1 --2- 172.21.15.112:0/1110575195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde40003a00 0x7fde40003e20 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.929+0000 7fde5e2c8700 1 -- 172.21.15.112:0/1110575195 >> 172.21.15.112:0/1110575195 conn(0x7fde580b1e90 msgr2=0x7fde580b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:00.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.929+0000 7fde5e2c8700 1 -- 172.21.15.112:0/1110575195 shutdown_connections 2023-08-27T11:26:00.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.929+0000 7fde5e2c8700 1 -- 172.21.15.112:0/1110575195 wait complete. 2023-08-27T11:26:00.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.930+0000 7fde5e2c8700 1 Processor -- start 2023-08-27T11:26:00.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.931+0000 7fde5e2c8700 1 -- start start 2023-08-27T11:26:00.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.931+0000 7fde5e2c8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde40003a00 0x7fde5815abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.931+0000 7fde5e2c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde5815c630 0x7fde5815cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.931+0000 7fde5e2c8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde581453d0 0x7fde58145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.931+0000 7fde5e2c8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde58143980 con 0x7fde581453d0 2023-08-27T11:26:00.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.931+0000 7fde5e2c8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde58145d90 con 0x7fde5815c630 2023-08-27T11:26:00.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.931+0000 7fde5e2c8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5815b0f0 con 0x7fde40003a00 2023-08-27T11:26:00.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.931+0000 7fde5c865700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde581453d0 0x7fde58145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde57fff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde40003a00 0x7fde5815abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde5c865700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde581453d0 0x7fde58145850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60468/0 (socket says 172.21.15.112:60468) 2023-08-27T11:26:00.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde5c865700 1 -- 172.21.15.112:0/465827141 learned_addr learned my addr 172.21.15.112:0/465827141 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:00.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde577fe700 1 --2- 172.21.15.112:0/465827141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde5815c630 0x7fde5815cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde5c865700 1 -- 172.21.15.112:0/465827141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde40003a00 msgr2=0x7fde5815abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde5c865700 1 --2- 172.21.15.112:0/465827141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde40003a00 0x7fde5815abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde5c865700 1 -- 172.21.15.112:0/465827141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde5815c630 msgr2=0x7fde5815cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:00.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde5c865700 1 --2- 172.21.15.112:0/465827141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde5815c630 0x7fde5815cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:00.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde5c865700 1 -- 172.21.15.112:0/465827141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde58142d00 con 0x7fde581453d0 2023-08-27T11:26:00.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde5c865700 1 --2- 172.21.15.112:0/465827141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde581453d0 0x7fde58145850 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fde580b0a60 tx=0x7fde5000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:00.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde54ff9700 1 -- 172.21.15.112:0/465827141 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50003c10 con 0x7fde581453d0 2023-08-27T11:26:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde54ff9700 1 -- 172.21.15.112:0/465827141 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde50003d90 con 0x7fde581453d0 2023-08-27T11:26:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde54ff9700 1 -- 172.21.15.112:0/465827141 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50005310 con 0x7fde581453d0 2023-08-27T11:26:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde5e2c8700 1 -- 172.21.15.112:0/465827141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde58142fe0 con 0x7fde581453d0 2023-08-27T11:26:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.932+0000 7fde5e2c8700 1 -- 172.21.15.112:0/465827141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde581435a0 con 0x7fde581453d0 2023-08-27T11:26:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.933+0000 7fde5e2c8700 1 -- 172.21.15.112:0/465827141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde58143760 con 0x7fde581453d0 2023-08-27T11:26:00.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.933+0000 7fde54ff9700 1 -- 172.21.15.112:0/465827141 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fde50005470 con 0x7fde581453d0 2023-08-27T11:26:00.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.934+0000 7fde54ff9700 1 --2- 172.21.15.112:0/465827141 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fde4406a3b0 0x7fde4406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:00.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.934+0000 7fde54ff9700 1 -- 172.21.15.112:0/465827141 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fde500888d0 con 0x7fde581453d0 2023-08-27T11:26:00.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.934+0000 7fde57fff700 1 --2- 172.21.15.112:0/465827141 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fde4406a3b0 0x7fde4406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:00.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.934+0000 7fde57fff700 1 --2- 172.21.15.112:0/465827141 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fde4406a3b0 0x7fde4406c870 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fde48000c00 tx=0x7fde48007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:00.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:00.938+0000 7fde54ff9700 1 -- 172.21.15.112:0/465827141 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fde50053cc0 con 0x7fde581453d0 2023-08-27T11:26:01.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.125+0000 7fde5e2c8700 1 -- 172.21.15.112:0/465827141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde58143ac0 con 0x7fde581453d0 2023-08-27T11:26:01.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.125+0000 7fde54ff9700 1 -- 172.21.15.112:0/465827141 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde58143ac0 con 0x7fde581453d0 2023-08-27T11:26:01.127 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:26:01.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.127+0000 7fde5e2c8700 1 -- 172.21.15.112:0/465827141 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fde4406a3b0 msgr2=0x7fde4406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.127+0000 7fde5e2c8700 1 --2- 172.21.15.112:0/465827141 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fde4406a3b0 0x7fde4406c870 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fde48000c00 tx=0x7fde48007000 comp rx=0 tx=0).stop 2023-08-27T11:26:01.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.127+0000 7fde5e2c8700 1 -- 172.21.15.112:0/465827141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde581453d0 msgr2=0x7fde58145850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.127+0000 7fde5e2c8700 1 --2- 172.21.15.112:0/465827141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde581453d0 0x7fde58145850 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fde580b0a60 tx=0x7fde5000edd0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.127+0000 7fde5e2c8700 1 -- 172.21.15.112:0/465827141 shutdown_connections 2023-08-27T11:26:01.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.127+0000 7fde5e2c8700 1 --2- 172.21.15.112:0/465827141 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fde4406a3b0 0x7fde4406c870 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.127+0000 7fde5e2c8700 1 --2- 172.21.15.112:0/465827141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde581453d0 0x7fde58145850 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.127+0000 7fde5e2c8700 1 --2- 172.21.15.112:0/465827141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde40003a00 0x7fde5815abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.127+0000 7fde5e2c8700 1 --2- 172.21.15.112:0/465827141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde5815c630 0x7fde5815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.127+0000 7fde5e2c8700 1 -- 172.21.15.112:0/465827141 >> 172.21.15.112:0/465827141 conn(0x7fde580b1e90 msgr2=0x7fde5806d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:01.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.128+0000 7fde5e2c8700 1 -- 172.21.15.112:0/465827141 shutdown_connections 2023-08-27T11:26:01.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.128+0000 7fde5e2c8700 1 -- 172.21.15.112:0/465827141 wait complete. 2023-08-27T11:26:01.141 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:26:01.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.262+0000 7f9862f31700 1 Processor -- start 2023-08-27T11:26:01.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.262+0000 7f9862f31700 1 -- start start 2023-08-27T11:26:01.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.263+0000 7f9862f31700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f985c143390 0x7f985c0723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.263+0000 7f9862f31700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f985c06f820 0x7f985c06ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.263+0000 7f9862f31700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c0b16e0 con 0x7f985c06e2f0 2023-08-27T11:26:01.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.263+0000 7f9862f31700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985c070000 con 0x7f985c06f820 2023-08-27T11:26:01.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.263+0000 7f9862f31700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c06e780 con 0x7f985c143390 2023-08-27T11:26:01.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.263+0000 7f985bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f985c06f820 0x7f985c06ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.263+0000 7f985bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f985c06f820 0x7f985c06ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60472/0 (socket says 172.21.15.112:60472) 2023-08-27T11:26:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.263+0000 7f985bfff700 1 -- 172.21.15.112:0/745385282 learned_addr learned my addr 172.21.15.112:0/745385282 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.263+0000 7f9860ccd700 1 --2- 172.21.15.112:0/745385282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f985c143390 0x7f985c0723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f985bfff700 1 -- 172.21.15.112:0/745385282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f985c143390 msgr2=0x7f985c0723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f985bfff700 1 --2- 172.21.15.112:0/745385282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f985c143390 0x7f985c0723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f985bfff700 1 -- 172.21.15.112:0/745385282 >> v1:172.21.15.138:6789/0 conn(0x7f985c06e2f0 legacy=0x7f985c15d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f985bfff700 1 -- 172.21.15.112:0/745385282 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c0b0010 con 0x7f985c06f820 2023-08-27T11:26:01.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f985bfff700 1 --2- 172.21.15.112:0/745385282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985c06f820 0x7f985c06ddb0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f98480098f0 tx=0x7f9848009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951b226efc792593 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:01.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f985b7fe700 1 -- 172.21.15.112:0/745385282 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848011830 con 0x7f985c06f820 2023-08-27T11:26:01.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f985b7fe700 1 -- 172.21.15.112:0/745385282 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9848011990 con 0x7f985c06f820 2023-08-27T11:26:01.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f985b7fe700 1 -- 172.21.15.112:0/745385282 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848011d20 con 0x7f985c06f820 2023-08-27T11:26:01.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f9862f31700 1 -- 172.21.15.112:0/745385282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985c06f820 msgr2=0x7f985c06ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f9862f31700 1 --2- 172.21.15.112:0/745385282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985c06f820 0x7f985c06ddb0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f98480098f0 tx=0x7f9848009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:01.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f9862f31700 1 -- 172.21.15.112:0/745385282 shutdown_connections 2023-08-27T11:26:01.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f9862f31700 1 --2- 172.21.15.112:0/745385282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f985c143390 0x7f985c0723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f9862f31700 1 --2- 172.21.15.112:0/745385282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985c06f820 0x7f985c06ddb0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f9862f31700 1 -- 172.21.15.112:0/745385282 >> 172.21.15.112:0/745385282 conn(0x7f985c0cc240 msgr2=0x7f985c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:01.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f9862f31700 1 -- 172.21.15.112:0/745385282 shutdown_connections 2023-08-27T11:26:01.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.264+0000 7f9862f31700 1 -- 172.21.15.112:0/745385282 wait complete. 2023-08-27T11:26:01.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.266+0000 7f9862f31700 1 Processor -- start 2023-08-27T11:26:01.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.266+0000 7f9862f31700 1 -- start start 2023-08-27T11:26:01.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.266+0000 7f9862f31700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985c06e2f0 0x7f985c1401f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.266+0000 7f9862f31700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f985c06f820 0x7f985c0adef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.266+0000 7f9862f31700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985c143390 0x7f985c158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.266+0000 7f9862f31700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c0ad2f0 con 0x7f985c06e2f0 2023-08-27T11:26:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.266+0000 7f9862f31700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c1407d0 con 0x7f985c143390 2023-08-27T11:26:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.266+0000 7f9862f31700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c140ab0 con 0x7f985c06f820 2023-08-27T11:26:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f9860ccd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985c06e2f0 0x7f985c1401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f9860ccd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985c06e2f0 0x7f985c1401f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60478/0 (socket says 172.21.15.112:60478) 2023-08-27T11:26:01.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f9860ccd700 1 -- 172.21.15.112:0/3407135702 learned_addr learned my addr 172.21.15.112:0/3407135702 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:01.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f985bfff700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f985c06f820 0x7f985c0adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f98614ce700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985c143390 0x7f985c158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f9860ccd700 1 -- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f985c06f820 msgr2=0x7f985c0adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f9860ccd700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f985c06f820 0x7f985c0adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f9860ccd700 1 -- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985c143390 msgr2=0x7f985c158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f9860ccd700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985c143390 0x7f985c158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f9860ccd700 1 -- 172.21.15.112:0/3407135702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9848009550 con 0x7f985c06e2f0 2023-08-27T11:26:01.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f9860ccd700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985c06e2f0 0x7f985c1401f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f985000e870 tx=0x7f985000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:01.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f98597fa700 1 -- 172.21.15.112:0/3407135702 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850009590 con 0x7f985c06e2f0 2023-08-27T11:26:01.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f98597fa700 1 -- 172.21.15.112:0/3407135702 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9850013680 con 0x7f985c06e2f0 2023-08-27T11:26:01.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f98597fa700 1 -- 172.21.15.112:0/3407135702 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98500139f0 con 0x7f985c06e2f0 2023-08-27T11:26:01.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f9862f31700 1 -- 172.21.15.112:0/3407135702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985c140d90 con 0x7f985c06e2f0 2023-08-27T11:26:01.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.267+0000 7f9862f31700 1 -- 172.21.15.112:0/3407135702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985c141350 con 0x7f985c06e2f0 2023-08-27T11:26:01.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.268+0000 7f9862f31700 1 -- 172.21.15.112:0/3407135702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985c1f2ee0 con 0x7f985c06e2f0 2023-08-27T11:26:01.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.269+0000 7f98597fa700 1 -- 172.21.15.112:0/3407135702 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f9850013b50 con 0x7f985c06e2f0 2023-08-27T11:26:01.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.269+0000 7f98597fa700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f984006a3b0 0x7f984006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.269+0000 7f98597fa700 1 -- 172.21.15.112:0/3407135702 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f98500887f0 con 0x7f985c06e2f0 2023-08-27T11:26:01.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.269+0000 7f985bfff700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f984006a3b0 0x7f984006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.270+0000 7f985bfff700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f984006a3b0 0x7f984006c870 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9848000c00 tx=0x7f9848008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:01.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.273+0000 7f98597fa700 1 -- 172.21.15.112:0/3407135702 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9850057630 con 0x7f985c06e2f0 2023-08-27T11:26:01.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.463+0000 7f9862f31700 1 -- 172.21.15.112:0/3407135702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f985c070ef0 con 0x7f985c06e2f0 2023-08-27T11:26:01.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.463+0000 7f98597fa700 1 -- 172.21.15.112:0/3407135702 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7f98500571c0 con 0x7f985c06e2f0 2023-08-27T11:26:01.465 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:26:01.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.465+0000 7f9862f31700 1 -- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f984006a3b0 msgr2=0x7f984006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.465+0000 7f9862f31700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f984006a3b0 0x7f984006c870 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9848000c00 tx=0x7f9848008030 comp rx=0 tx=0).stop 2023-08-27T11:26:01.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.465+0000 7f9862f31700 1 -- 172.21.15.112:0/3407135702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985c06e2f0 msgr2=0x7f985c1401f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.465+0000 7f9862f31700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985c06e2f0 0x7f985c1401f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f985000e870 tx=0x7f985000eb80 comp rx=0 tx=0).stop 2023-08-27T11:26:01.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.465+0000 7f9862f31700 1 -- 172.21.15.112:0/3407135702 shutdown_connections 2023-08-27T11:26:01.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.465+0000 7f9862f31700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f984006a3b0 0x7f984006c870 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.465+0000 7f9862f31700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985c143390 0x7f985c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.465+0000 7f9862f31700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985c06e2f0 0x7f985c1401f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.465+0000 7f9862f31700 1 --2- 172.21.15.112:0/3407135702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f985c06f820 0x7f985c0adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.465+0000 7f9862f31700 1 -- 172.21.15.112:0/3407135702 >> 172.21.15.112:0/3407135702 conn(0x7f985c0cc240 msgr2=0x7f985c159510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:01.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.465+0000 7f9862f31700 1 -- 172.21.15.112:0/3407135702 shutdown_connections 2023-08-27T11:26:01.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.465+0000 7f9862f31700 1 -- 172.21.15.112:0/3407135702 wait complete. 2023-08-27T11:26:01.479 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:26:01.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.598+0000 7efe669bf700 1 Processor -- start 2023-08-27T11:26:01.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.598+0000 7efe669bf700 1 -- start start 2023-08-27T11:26:01.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.599+0000 7efe669bf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe600b3440 0x7efe600b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.599+0000 7efe669bf700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe600725e0 0x7efe6006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.599+0000 7efe669bf700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe60160db0 con 0x7efe601609d0 2023-08-27T11:26:01.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.599+0000 7efe669bf700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe600b38d0 con 0x7efe600725e0 2023-08-27T11:26:01.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.599+0000 7efe669bf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe600b3bb0 con 0x7efe600b3440 2023-08-27T11:26:01.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.599+0000 7efe5ffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7efe601609d0 0x7efe60071ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54868/0 (socket says 172.21.15.112:54868) 2023-08-27T11:26:01.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.599+0000 7efe5ffff700 1 -- 172.21.15.112:0/1258751958 learned_addr learned my addr 172.21.15.112:0/1258751958 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:01.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5f7fe700 1 --2- 172.21.15.112:0/1258751958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe600b3440 0x7efe600b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe64f5c700 1 --2- 172.21.15.112:0/1258751958 >> v2:172.21.15.138:3300/0 conn(0x7efe600725e0 0x7efe6006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213781593 0 0) 0x7efe60160db0 con 0x7efe601609d0 2023-08-27T11:26:01.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe4c0035b0 con 0x7efe601609d0 2023-08-27T11:26:01.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3970339949 0 0) 0x7efe4c0035b0 con 0x7efe601609d0 2023-08-27T11:26:01.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe600b3440 msgr2=0x7efe600b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:01.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 --2- 172.21.15.112:0/1258751958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe600b3440 0x7efe600b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 >> v2:172.21.15.138:3300/0 conn(0x7efe600725e0 msgr2=0x7efe6006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:01.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 --2- 172.21.15.112:0/1258751958 >> v2:172.21.15.138:3300/0 conn(0x7efe600725e0 0x7efe6006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe600b3e90 con 0x7efe601609d0 2023-08-27T11:26:01.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7efe48001670 con 0x7efe601609d0 2023-08-27T11:26:01.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 >> v1:172.21.15.112:6789/0 conn(0x7efe601609d0 legacy=0x7efe60071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 --2- 172.21.15.112:0/1258751958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe4c003a70 0x7efe4c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 --2- 172.21.15.112:0/1258751958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe4c004850 0x7efe4c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 --2- 172.21.15.112:0/1258751958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe4c0051d0 0x7efe4c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe600b38d0 con 0x7efe4c0051d0 2023-08-27T11:26:01.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe600b3bb0 con 0x7efe4c003a70 2023-08-27T11:26:01.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0046c0 con 0x7efe4c004850 2023-08-27T11:26:01.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe64f5c700 1 --2- 172.21.15.112:0/1258751958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe4c0051d0 0x7efe4c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5ffff700 1 --2- 172.21.15.112:0/1258751958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe4c003a70 0x7efe4c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe5f7fe700 1 --2- 172.21.15.112:0/1258751958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe4c004850 0x7efe4c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe64f5c700 1 -- 172.21.15.112:0/1258751958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe4c004850 msgr2=0x7efe4c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:01.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe64f5c700 1 --2- 172.21.15.112:0/1258751958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe4c004850 0x7efe4c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe64f5c700 1 -- 172.21.15.112:0/1258751958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe4c003a70 msgr2=0x7efe4c003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:01.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.600+0000 7efe64f5c700 1 --2- 172.21.15.112:0/1258751958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe4c003a70 0x7efe4c003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.601+0000 7efe64f5c700 1 -- 172.21.15.112:0/1258751958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0035b0 con 0x7efe4c0051d0 2023-08-27T11:26:01.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.601+0000 7efe64f5c700 1 --2- 172.21.15.112:0/1258751958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe4c0051d0 0x7efe4c00ba10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efe6006d530 tx=0x7efe4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4354147f7ee54 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:01.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.601+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5800ebd0 con 0x7efe4c0051d0 2023-08-27T11:26:01.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.601+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5800f040 con 0x7efe4c0051d0 2023-08-27T11:26:01.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.601+0000 7efe5effd700 1 -- 172.21.15.112:0/1258751958 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58014480 con 0x7efe4c0051d0 2023-08-27T11:26:01.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.601+0000 7efe669bf700 1 -- 172.21.15.112:0/1258751958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe4c0051d0 msgr2=0x7efe4c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.601+0000 7efe669bf700 1 --2- 172.21.15.112:0/1258751958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe4c0051d0 0x7efe4c00ba10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efe6006d530 tx=0x7efe4c005880 comp rx=0 tx=0).stop 2023-08-27T11:26:01.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.601+0000 7efe64f5c700 1 -- 172.21.15.112:0/1258751958 reap_dead start 2023-08-27T11:26:01.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.601+0000 7efe669bf700 1 -- 172.21.15.112:0/1258751958 shutdown_connections 2023-08-27T11:26:01.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.601+0000 7efe669bf700 1 -- 172.21.15.112:0/1258751958 >> 172.21.15.112:0/1258751958 conn(0x7efe60066400 msgr2=0x7efe600cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:01.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.601+0000 7efe669bf700 1 -- 172.21.15.112:0/1258751958 shutdown_connections 2023-08-27T11:26:01.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.601+0000 7efe669bf700 1 -- 172.21.15.112:0/1258751958 wait complete. 2023-08-27T11:26:01.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.603+0000 7efe669bf700 1 Processor -- start 2023-08-27T11:26:01.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.603+0000 7efe669bf700 1 -- start start 2023-08-27T11:26:01.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe669bf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe601f6050 0x7efe601f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe669bf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe60150360 0x7efe600afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe669bf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe600b0220 0x7efe6014c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe669bf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6014dd40 con 0x7efe600b0220 2023-08-27T11:26:01.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe669bf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe600b06b0 con 0x7efe60150360 2023-08-27T11:26:01.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe669bf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe601f6970 con 0x7efe601f6050 2023-08-27T11:26:01.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe64f5c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe600b0220 0x7efe6014c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe64f5c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe600b0220 0x7efe6014c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60502/0 (socket says 172.21.15.112:60502) 2023-08-27T11:26:01.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe5ffff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe601f6050 0x7efe601f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe64f5c700 1 -- 172.21.15.112:0/2694205081 learned_addr learned my addr 172.21.15.112:0/2694205081 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:01.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe5f7fe700 1 --2- 172.21.15.112:0/2694205081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe60150360 0x7efe600afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe64f5c700 1 -- 172.21.15.112:0/2694205081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe601f6050 msgr2=0x7efe601f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe64f5c700 1 --2- 172.21.15.112:0/2694205081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe601f6050 0x7efe601f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe64f5c700 1 -- 172.21.15.112:0/2694205081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe60150360 msgr2=0x7efe600afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe64f5c700 1 --2- 172.21.15.112:0/2694205081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe60150360 0x7efe600afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.604+0000 7efe64f5c700 1 -- 172.21.15.112:0/2694205081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe600b08f0 con 0x7efe600b0220 2023-08-27T11:26:01.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.605+0000 7efe64f5c700 1 --2- 172.21.15.112:0/2694205081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe600b0220 0x7efe6014c980 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7efe58015000 tx=0x7efe58009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:01.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.605+0000 7efe5cff9700 1 -- 172.21.15.112:0/2694205081 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58007000 con 0x7efe600b0220 2023-08-27T11:26:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.605+0000 7efe5cff9700 1 -- 172.21.15.112:0/2694205081 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe58022070 con 0x7efe600b0220 2023-08-27T11:26:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.605+0000 7efe669bf700 1 -- 172.21.15.112:0/2694205081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe600b0bd0 con 0x7efe600b0220 2023-08-27T11:26:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.605+0000 7efe5cff9700 1 -- 172.21.15.112:0/2694205081 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58014ad0 con 0x7efe600b0220 2023-08-27T11:26:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.605+0000 7efe669bf700 1 -- 172.21.15.112:0/2694205081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe600b1190 con 0x7efe600b0220 2023-08-27T11:26:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.606+0000 7efe669bf700 1 -- 172.21.15.112:0/2694205081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe600b16b0 con 0x7efe600b0220 2023-08-27T11:26:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.606+0000 7efe5cff9700 1 -- 172.21.15.112:0/2694205081 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7efe58014c30 con 0x7efe600b0220 2023-08-27T11:26:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.607+0000 7efe5cff9700 1 --2- 172.21.15.112:0/2694205081 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efe4006a3b0 0x7efe4006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.607+0000 7efe5cff9700 1 -- 172.21.15.112:0/2694205081 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7efe58024070 con 0x7efe600b0220 2023-08-27T11:26:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.607+0000 7efe5ffff700 1 --2- 172.21.15.112:0/2694205081 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efe4006a3b0 0x7efe4006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.607+0000 7efe5ffff700 1 --2- 172.21.15.112:0/2694205081 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efe4006a3b0 0x7efe4006c870 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efe48000c00 tx=0x7efe48007980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.611+0000 7efe5cff9700 1 -- 172.21.15.112:0/2694205081 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efe600b16b0 con 0x7efe600b0220 2023-08-27T11:26:01.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.815+0000 7efe669bf700 1 -- 172.21.15.112:0/2694205081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efe600b16b0 con 0x7efe600b0220 2023-08-27T11:26:01.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.816+0000 7efe5cff9700 1 -- 172.21.15.112:0/2694205081 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v79) v1 ==== 80+0+0 (secure 0 0 0) 0x7efe600b16b0 con 0x7efe600b0220 2023-08-27T11:26:01.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.818+0000 7efe669bf700 1 -- 172.21.15.112:0/2694205081 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efe4006a3b0 msgr2=0x7efe4006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.818+0000 7efe669bf700 1 --2- 172.21.15.112:0/2694205081 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efe4006a3b0 0x7efe4006c870 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efe48000c00 tx=0x7efe48007980 comp rx=0 tx=0).stop 2023-08-27T11:26:01.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.818+0000 7efe669bf700 1 -- 172.21.15.112:0/2694205081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe600b0220 msgr2=0x7efe6014c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:01.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.818+0000 7efe669bf700 1 --2- 172.21.15.112:0/2694205081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe600b0220 0x7efe6014c980 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7efe58015000 tx=0x7efe58009750 comp rx=0 tx=0).stop 2023-08-27T11:26:01.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.818+0000 7efe669bf700 1 -- 172.21.15.112:0/2694205081 shutdown_connections 2023-08-27T11:26:01.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.818+0000 7efe669bf700 1 --2- 172.21.15.112:0/2694205081 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efe4006a3b0 0x7efe4006c870 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.818+0000 7efe669bf700 1 --2- 172.21.15.112:0/2694205081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe600b0220 0x7efe6014c980 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.818+0000 7efe669bf700 1 --2- 172.21.15.112:0/2694205081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe601f6050 0x7efe601f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.818+0000 7efe669bf700 1 --2- 172.21.15.112:0/2694205081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe60150360 0x7efe600afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:01.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.818+0000 7efe669bf700 1 -- 172.21.15.112:0/2694205081 >> 172.21.15.112:0/2694205081 conn(0x7efe60066400 msgr2=0x7efe6015de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:01.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.818+0000 7efe669bf700 1 -- 172.21.15.112:0/2694205081 shutdown_connections 2023-08-27T11:26:01.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:01.818+0000 7efe669bf700 1 -- 172.21.15.112:0/2694205081 wait complete. 2023-08-27T11:26:01.822 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:26:01.831 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-27T11:26:01.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:01.832 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-08-27T11:26:01.883 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:26:01.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:01.884 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-27T11:26:01.903 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:26:01.904 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:01.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:26:01.922 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:26:01.922 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:01.963 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:26:01.964 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:26:02.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.082+0000 7f6531eac700 1 Processor -- start 2023-08-27T11:26:02.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.082+0000 7f6531eac700 1 -- start start 2023-08-27T11:26:02.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.083+0000 7f6531eac700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b1400 0x7f652c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.083+0000 7f6531eac700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f652c0b17e0 0x7f652c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.083+0000 7f6531eac700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652c161850 con 0x7f652c161ce0 2023-08-27T11:26:02.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.083+0000 7f6531eac700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652c06e7d0 con 0x7f652c0b17e0 2023-08-27T11:26:02.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.083+0000 7f6531eac700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652c0cfc10 con 0x7f652c0b1400 2023-08-27T11:26:02.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.084+0000 7f652b7fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b1400 0x7f652c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.084+0000 7f652affd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f652c0b17e0 0x7f652c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.084+0000 7f652b7fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b1400 0x7f652c0cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:43928/0 (socket says 172.21.15.112:43928) 2023-08-27T11:26:02.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.084+0000 7f652b7fe700 1 -- 172.21.15.112:0/3342808675 learned_addr learned my addr 172.21.15.112:0/3342808675 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:02.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.084+0000 7f652b7fe700 1 -- 172.21.15.112:0/3342808675 >> v2:172.21.15.138:3300/0 conn(0x7f652c0b17e0 msgr2=0x7f652c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:02.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.084+0000 7f652b7fe700 1 --2- 172.21.15.112:0/3342808675 >> v2:172.21.15.138:3300/0 conn(0x7f652c0b17e0 0x7f652c06e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.084+0000 7f652b7fe700 1 -- 172.21.15.112:0/3342808675 >> v1:172.21.15.138:6789/0 conn(0x7f652c161ce0 legacy=0x7f652c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.084+0000 7f652b7fe700 1 -- 172.21.15.112:0/3342808675 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652c161200 con 0x7f652c0b1400 2023-08-27T11:26:02.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.084+0000 7f652affd700 1 --2- 172.21.15.112:0/3342808675 >> v2:172.21.15.138:3300/0 conn(0x7f652c0b17e0 0x7f652c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:02.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.084+0000 7f652b7fe700 1 --2- 172.21.15.112:0/3342808675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b1400 0x7f652c0cf5a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6524009e50 tx=0x7f652400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f23d745af851fdeb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:02.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.084+0000 7f652a7fc700 1 -- 172.21.15.112:0/3342808675 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652400e070 con 0x7f652c0b1400 2023-08-27T11:26:02.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.084+0000 7f652a7fc700 1 -- 172.21.15.112:0/3342808675 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652400b910 con 0x7f652c0b1400 2023-08-27T11:26:02.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.084+0000 7f652a7fc700 1 -- 172.21.15.112:0/3342808675 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652400bcc0 con 0x7f652c0b1400 2023-08-27T11:26:02.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.085+0000 7f6531eac700 1 -- 172.21.15.112:0/3342808675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b1400 msgr2=0x7f652c0cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.085+0000 7f6531eac700 1 --2- 172.21.15.112:0/3342808675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b1400 0x7f652c0cf5a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6524009e50 tx=0x7f652400b330 comp rx=0 tx=0).stop 2023-08-27T11:26:02.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.085+0000 7f6531eac700 1 -- 172.21.15.112:0/3342808675 shutdown_connections 2023-08-27T11:26:02.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.085+0000 7f6531eac700 1 --2- 172.21.15.112:0/3342808675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b1400 0x7f652c0cf5a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.085+0000 7f6531eac700 1 --2- 172.21.15.112:0/3342808675 >> v2:172.21.15.138:3300/0 conn(0x7f652c0b17e0 0x7f652c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.085+0000 7f6531eac700 1 -- 172.21.15.112:0/3342808675 >> 172.21.15.112:0/3342808675 conn(0x7f652c0d0000 msgr2=0x7f652c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:02.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.085+0000 7f6531eac700 1 -- 172.21.15.112:0/3342808675 shutdown_connections 2023-08-27T11:26:02.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.085+0000 7f6531eac700 1 -- 172.21.15.112:0/3342808675 wait complete. 2023-08-27T11:26:02.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.086+0000 7f6531eac700 1 Processor -- start 2023-08-27T11:26:02.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.087+0000 7f6531eac700 1 -- start start 2023-08-27T11:26:02.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f6531eac700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f652c0b1400 0x7f652c0c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f6531eac700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b17e0 0x7f652c1482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f6531eac700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c161ce0 0x7f652c0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f6531eac700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c0c64a0 con 0x7f652c161ce0 2023-08-27T11:26:02.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f6531eac700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c148830 con 0x7f652c0b1400 2023-08-27T11:26:02.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f6531eac700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652c0c8ad0 con 0x7f652c0b17e0 2023-08-27T11:26:02.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652b7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f652c0b1400 0x7f652c0c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652affd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b17e0 0x7f652c1482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c161ce0 0x7f652c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652affd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b17e0 0x7f652c1482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:43934/0 (socket says 172.21.15.112:43934) 2023-08-27T11:26:02.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652b7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f652c0b1400 0x7f652c0c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:60936/0 (socket says 172.21.15.112:60936) 2023-08-27T11:26:02.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652affd700 1 -- 172.21.15.112:0/2955141900 learned_addr learned my addr 172.21.15.112:0/2955141900 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:02.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652bfff700 1 -- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b17e0 msgr2=0x7f652c1482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652bfff700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b17e0 0x7f652c1482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652bfff700 1 -- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f652c0b1400 msgr2=0x7f652c0c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652bfff700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f652c0b1400 0x7f652c0c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652bfff700 1 -- 172.21.15.112:0/2955141900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200094a0 con 0x7f652c161ce0 2023-08-27T11:26:02.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652affd700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b17e0 0x7f652c1482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:02.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652b7fe700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f652c0b1400 0x7f652c0c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:02.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f652bfff700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c161ce0 0x7f652c0c95b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f651c000f60 tx=0x7f651c00c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f6513fff700 1 -- 172.21.15.112:0/2955141900 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c0052d0 con 0x7f652c161ce0 2023-08-27T11:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f6513fff700 1 -- 172.21.15.112:0/2955141900 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651c005430 con 0x7f652c161ce0 2023-08-27T11:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f6531eac700 1 -- 172.21.15.112:0/2955141900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6524009b30 con 0x7f652c161ce0 2023-08-27T11:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.088+0000 7f6513fff700 1 -- 172.21.15.112:0/2955141900 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c0038c0 con 0x7f652c161ce0 2023-08-27T11:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.089+0000 7f6531eac700 1 -- 172.21.15.112:0/2955141900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652c0c5e50 con 0x7f652c161ce0 2023-08-27T11:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.089+0000 7f6531eac700 1 -- 172.21.15.112:0/2955141900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652c0c6650 con 0x7f652c161ce0 2023-08-27T11:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.090+0000 7f6513fff700 1 -- 172.21.15.112:0/2955141900 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f651c003a20 con 0x7f652c161ce0 2023-08-27T11:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.090+0000 7f6513fff700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f651806a3b0 0x7f651806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.090+0000 7f6513fff700 1 -- 172.21.15.112:0/2955141900 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f651c087c70 con 0x7f652c161ce0 2023-08-27T11:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.090+0000 7f652b7fe700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f651806a3b0 0x7f651806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.091+0000 7f652b7fe700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f651806a3b0 0x7f651806c870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f652400b370 tx=0x7f6524009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:02.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.094+0000 7f6513fff700 1 -- 172.21.15.112:0/2955141900 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f652c0c6650 con 0x7f652c161ce0 2023-08-27T11:26:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.326+0000 7f6531eac700 1 -- 172.21.15.112:0/2955141900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f652c067a20 con 0x7f652c161ce0 2023-08-27T11:26:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.327+0000 7f6513fff700 1 -- 172.21.15.112:0/2955141900 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4889 (secure 0 0 0) 0x7f651c017020 con 0x7f652c161ce0 2023-08-27T11:26:02.329 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:02.329 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6166,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3262556844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3262556844},{"type":"v1","addr":"172.21.15.138:6836","nonce":3262556844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2232724696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2232724696},{"type":"v1","addr":"172.21.15.138:6837","nonce":2232724696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6570,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1511159838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1511159838},{"type":"v1","addr":"172.21.15.112:6834","nonce":1511159838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:25:41.794717+0000","modified":"2023-08-27T11:25:45.437236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2318724344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2318724344},{"type":"v1","addr":"172.21.15.112:6835","nonce":2318724344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-27T11:26:02.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.329+0000 7f6531eac700 1 -- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f651806a3b0 msgr2=0x7f651806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.329+0000 7f6531eac700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f651806a3b0 0x7f651806c870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f652400b370 tx=0x7f6524009000 comp rx=0 tx=0).stop 2023-08-27T11:26:02.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.329+0000 7f6531eac700 1 -- 172.21.15.112:0/2955141900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c161ce0 msgr2=0x7f652c0c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.329+0000 7f6531eac700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c161ce0 0x7f652c0c95b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f651c000f60 tx=0x7f651c00c670 comp rx=0 tx=0).stop 2023-08-27T11:26:02.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.329+0000 7f6531eac700 1 -- 172.21.15.112:0/2955141900 shutdown_connections 2023-08-27T11:26:02.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.329+0000 7f6531eac700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f651806a3b0 0x7f651806c870 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.329+0000 7f6531eac700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c161ce0 0x7f652c0c95b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.329+0000 7f6531eac700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f652c0b1400 0x7f652c0c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.329+0000 7f6531eac700 1 --2- 172.21.15.112:0/2955141900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f652c0b17e0 0x7f652c1482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.329+0000 7f6531eac700 1 -- 172.21.15.112:0/2955141900 >> 172.21.15.112:0/2955141900 conn(0x7f652c0d0000 msgr2=0x7f652c0ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:02.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.329+0000 7f6531eac700 1 -- 172.21.15.112:0/2955141900 shutdown_connections 2023-08-27T11:26:02.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.329+0000 7f6531eac700 1 -- 172.21.15.112:0/2955141900 wait complete. 2023-08-27T11:26:02.333 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 49 2023-08-27T11:26:02.343 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.462+0000 7f873be98700 1 Processor -- start 2023-08-27T11:26:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.462+0000 7f873be98700 1 -- start start 2023-08-27T11:26:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f873be98700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87340ad430 0x7f87340ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f873be98700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87340af000 0x7f87340af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f873be98700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87341470e0 con 0x7f8734146d00 2023-08-27T11:26:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f873be98700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87340aeb70 con 0x7f87340af000 2023-08-27T11:26:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f873be98700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87340adf60 con 0x7f87340ad430 2023-08-27T11:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f873a435700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87340af000 0x7f87340af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f8739c34700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8734146d00 0x7f87340b3440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54900/0 (socket says 172.21.15.112:54900) 2023-08-27T11:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f8739c34700 1 -- 172.21.15.112:0/169557159 learned_addr learned my addr 172.21.15.112:0/169557159 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f8739433700 1 --2- 172.21.15.112:0/169557159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87340ad430 0x7f87340ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f873a435700 1 -- 172.21.15.112:0/169557159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87340ad430 msgr2=0x7f87340ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f873a435700 1 --2- 172.21.15.112:0/169557159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87340ad430 0x7f87340ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f873a435700 1 -- 172.21.15.112:0/169557159 >> v1:172.21.15.112:6789/0 conn(0x7f8734146d00 legacy=0x7f87340b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f873a435700 1 -- 172.21.15.112:0/169557159 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87340ae240 con 0x7f87340af000 2023-08-27T11:26:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.463+0000 7f8738c32700 1 -- 172.21.15.112:0/169557159 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964016985 0 0) 0x7f87341470e0 con 0x7f8734146d00 2023-08-27T11:26:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.464+0000 7f873a435700 1 --2- 172.21.15.112:0/169557159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340af000 0x7f87340af490 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f872c009d90 tx=0x7f872c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4222f3c301b7a4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.464+0000 7f8738c32700 1 -- 172.21.15.112:0/169557159 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00e070 con 0x7f87340af000 2023-08-27T11:26:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.464+0000 7f8738c32700 1 -- 172.21.15.112:0/169557159 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c00bbd0 con 0x7f87340af000 2023-08-27T11:26:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.464+0000 7f8738c32700 1 -- 172.21.15.112:0/169557159 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c012430 con 0x7f87340af000 2023-08-27T11:26:02.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.464+0000 7f873be98700 1 -- 172.21.15.112:0/169557159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340af000 msgr2=0x7f87340af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.464+0000 7f873be98700 1 --2- 172.21.15.112:0/169557159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340af000 0x7f87340af490 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f872c009d90 tx=0x7f872c00b330 comp rx=0 tx=0).stop 2023-08-27T11:26:02.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.464+0000 7f873be98700 1 -- 172.21.15.112:0/169557159 shutdown_connections 2023-08-27T11:26:02.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.464+0000 7f873be98700 1 --2- 172.21.15.112:0/169557159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340af000 0x7f87340af490 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.464+0000 7f873be98700 1 --2- 172.21.15.112:0/169557159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87340ad430 0x7f87340ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.464+0000 7f873be98700 1 -- 172.21.15.112:0/169557159 >> 172.21.15.112:0/169557159 conn(0x7f873406adb0 msgr2=0x7f873406b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:02.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.464+0000 7f873be98700 1 -- 172.21.15.112:0/169557159 shutdown_connections 2023-08-27T11:26:02.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.464+0000 7f873be98700 1 -- 172.21.15.112:0/169557159 wait complete. 2023-08-27T11:26:02.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.466+0000 7f873be98700 1 Processor -- start 2023-08-27T11:26:02.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.466+0000 7f873be98700 1 -- start start 2023-08-27T11:26:02.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.466+0000 7f873be98700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340ad430 0x7f87340cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.466+0000 7f873be98700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87340af000 0x7f873415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.466+0000 7f873be98700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8734146d00 0x7f87340b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.466+0000 7f873be98700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87340cd7e0 con 0x7f87340ad430 2023-08-27T11:26:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.466+0000 7f873be98700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873415f3e0 con 0x7f8734146d00 2023-08-27T11:26:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.466+0000 7f873be98700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873415ee40 con 0x7f87340af000 2023-08-27T11:26:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.466+0000 7f8739c34700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340ad430 0x7f87340cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.466+0000 7f8739c34700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340ad430 0x7f87340cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60524/0 (socket says 172.21.15.112:60524) 2023-08-27T11:26:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.466+0000 7f8739c34700 1 -- 172.21.15.112:0/2285271 learned_addr learned my addr 172.21.15.112:0/2285271 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f8739433700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87340af000 0x7f873415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f873a435700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8734146d00 0x7f87340b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f8739c34700 1 -- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87340af000 msgr2=0x7f873415e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f8739c34700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87340af000 0x7f873415e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f8739c34700 1 -- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8734146d00 msgr2=0x7f87340b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f8739c34700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8734146d00 0x7f87340b0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f8739c34700 1 -- 172.21.15.112:0/2285271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c009a70 con 0x7f87340ad430 2023-08-27T11:26:02.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f873a435700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8734146d00 0x7f87340b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:02.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f8739c34700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340ad430 0x7f87340cd0a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f872400c4a0 tx=0x7f872400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:02.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f872a7fc700 1 -- 172.21.15.112:0/2285271 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87240074e0 con 0x7f87340ad430 2023-08-27T11:26:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f872a7fc700 1 -- 172.21.15.112:0/2285271 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8724007640 con 0x7f87340ad430 2023-08-27T11:26:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f873be98700 1 -- 172.21.15.112:0/2285271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873415f100 con 0x7f87340ad430 2023-08-27T11:26:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f872a7fc700 1 -- 172.21.15.112:0/2285271 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724005a70 con 0x7f87340ad430 2023-08-27T11:26:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.467+0000 7f873be98700 1 -- 172.21.15.112:0/2285271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87340cdca0 con 0x7f87340ad430 2023-08-27T11:26:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.468+0000 7f873be98700 1 -- 172.21.15.112:0/2285271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8734067a20 con 0x7f87340ad430 2023-08-27T11:26:02.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.468+0000 7f872a7fc700 1 -- 172.21.15.112:0/2285271 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f8724005bd0 con 0x7f87340ad430 2023-08-27T11:26:02.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.469+0000 7f872a7fc700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f871806a3b0 0x7f871806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.469+0000 7f872a7fc700 1 -- 172.21.15.112:0/2285271 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f8724087ec0 con 0x7f87340ad430 2023-08-27T11:26:02.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.469+0000 7f8739433700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f871806a3b0 0x7f871806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.470+0000 7f8739433700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f871806a3b0 0x7f871806c870 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8734160e20 tx=0x7f8730008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:02.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.473+0000 7f872a7fc700 1 -- 172.21.15.112:0/2285271 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8724053230 con 0x7f87340ad430 2023-08-27T11:26:02.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.664+0000 7f873be98700 1 -- 172.21.15.112:0/2285271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87340cf860 con 0x7f87340ad430 2023-08-27T11:26:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.665+0000 7f872a7fc700 1 -- 172.21.15.112:0/2285271 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18453 (secure 0 0 0) 0x7f8724017090 con 0x7f87340ad430 2023-08-27T11:26:02.667 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:02.667 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":79,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:25:54.909142+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:39.495757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:40.765773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.2d","mappings":[{"from":1,"to":7}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:02.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.667+0000 7f873be98700 1 -- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f871806a3b0 msgr2=0x7f871806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.667+0000 7f873be98700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f871806a3b0 0x7f871806c870 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8734160e20 tx=0x7f8730008040 comp rx=0 tx=0).stop 2023-08-27T11:26:02.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.667+0000 7f873be98700 1 -- 172.21.15.112:0/2285271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340ad430 msgr2=0x7f87340cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.667+0000 7f873be98700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340ad430 0x7f87340cd0a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f872400c4a0 tx=0x7f872400c860 comp rx=0 tx=0).stop 2023-08-27T11:26:02.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.667+0000 7f873be98700 1 -- 172.21.15.112:0/2285271 shutdown_connections 2023-08-27T11:26:02.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.667+0000 7f873be98700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f871806a3b0 0x7f871806c870 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.667+0000 7f873be98700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8734146d00 0x7f87340b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.667+0000 7f873be98700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87340ad430 0x7f87340cd0a0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.667+0000 7f873be98700 1 --2- 172.21.15.112:0/2285271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87340af000 0x7f873415e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.667+0000 7f873be98700 1 -- 172.21.15.112:0/2285271 >> 172.21.15.112:0/2285271 conn(0x7f873406adb0 msgr2=0x7f8734070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:02.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.667+0000 7f873be98700 1 -- 172.21.15.112:0/2285271 shutdown_connections 2023-08-27T11:26:02.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.667+0000 7f873be98700 1 -- 172.21.15.112:0/2285271 wait complete. 2023-08-27T11:26:02.681 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:25:39.495757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:25:40.765773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:26:02.682 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:26:02.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.803+0000 7f22c9e95700 1 Processor -- start 2023-08-27T11:26:02.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.803+0000 7f22c9e95700 1 -- start start 2023-08-27T11:26:02.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c9e95700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22c4146c20 0x7f22c4160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c9e95700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f22c40b3440 0x7f22c415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c9e95700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c40b5070 con 0x7f22c415e6e0 2023-08-27T11:26:02.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c9e95700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c4147000 con 0x7f22c40b3440 2023-08-27T11:26:02.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c9e95700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c41614a0 con 0x7f22c4146c20 2023-08-27T11:26:02.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c2ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f22c40b3440 0x7f22c415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c3fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f22c415e6e0 0x7f22c406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54918/0 (socket says 172.21.15.112:54918) 2023-08-27T11:26:02.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c2ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f22c40b3440 0x7f22c415e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60534/0 (socket says 172.21.15.112:60534) 2023-08-27T11:26:02.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c3fff700 1 -- 172.21.15.112:0/4022819739 learned_addr learned my addr 172.21.15.112:0/4022819739 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:02.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c37fe700 1 --2- 172.21.15.112:0/4022819739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22c4146c20 0x7f22c4160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c2ffd700 1 -- 172.21.15.112:0/4022819739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22c4146c20 msgr2=0x7f22c4160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:02.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c2ffd700 1 --2- 172.21.15.112:0/4022819739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22c4146c20 0x7f22c4160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c2ffd700 1 -- 172.21.15.112:0/4022819739 >> v1:172.21.15.112:6789/0 conn(0x7f22c415e6e0 legacy=0x7f22c406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c2ffd700 1 -- 172.21.15.112:0/4022819739 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c4161740 con 0x7f22c40b3440 2023-08-27T11:26:02.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c27fc700 1 -- 172.21.15.112:0/4022819739 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385310031 0 0) 0x7f22bc001490 con 0x7f22c415e6e0 2023-08-27T11:26:02.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c2ffd700 1 --2- 172.21.15.112:0/4022819739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c40b3440 0x7f22c415e1a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009840 tx=0x7f22b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ade5f1a0ef435ea server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:02.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.804+0000 7f22c27fc700 1 -- 172.21.15.112:0/4022819739 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4011830 con 0x7f22c40b3440 2023-08-27T11:26:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.805+0000 7f22c27fc700 1 -- 172.21.15.112:0/4022819739 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b4011990 con 0x7f22c40b3440 2023-08-27T11:26:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.805+0000 7f22c27fc700 1 -- 172.21.15.112:0/4022819739 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4011d20 con 0x7f22c40b3440 2023-08-27T11:26:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.805+0000 7f22c9e95700 1 -- 172.21.15.112:0/4022819739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c40b3440 msgr2=0x7f22c415e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.805+0000 7f22c9e95700 1 --2- 172.21.15.112:0/4022819739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c40b3440 0x7f22c415e1a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009840 tx=0x7f22b4009b50 comp rx=0 tx=0).stop 2023-08-27T11:26:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.805+0000 7f22c9e95700 1 -- 172.21.15.112:0/4022819739 shutdown_connections 2023-08-27T11:26:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.805+0000 7f22c9e95700 1 --2- 172.21.15.112:0/4022819739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22c4146c20 0x7f22c4160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.805+0000 7f22c9e95700 1 --2- 172.21.15.112:0/4022819739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c40b3440 0x7f22c415e1a0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.805+0000 7f22c9e95700 1 -- 172.21.15.112:0/4022819739 >> 172.21.15.112:0/4022819739 conn(0x7f22c40b1e90 msgr2=0x7f22c40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.805+0000 7f22c9e95700 1 -- 172.21.15.112:0/4022819739 shutdown_connections 2023-08-27T11:26:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.805+0000 7f22c9e95700 1 -- 172.21.15.112:0/4022819739 wait complete. 2023-08-27T11:26:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.806+0000 7f22c9e95700 1 Processor -- start 2023-08-27T11:26:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.806+0000 7f22c9e95700 1 -- start start 2023-08-27T11:26:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c9e95700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22c40b3440 0x7f22c4142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c9e95700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c4146c20 0x7f22c4160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c9e95700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22c415e6e0 0x7f22c406ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c9e95700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c41431d0 con 0x7f22c4146c20 2023-08-27T11:26:02.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c9e95700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c41f6980 con 0x7f22c415e6e0 2023-08-27T11:26:02.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c9e95700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c41f5db0 con 0x7f22c40b3440 2023-08-27T11:26:02.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c4146c20 0x7f22c4160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c4146c20 0x7f22c4160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60540/0 (socket says 172.21.15.112:60540) 2023-08-27T11:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c2ffd700 1 -- 172.21.15.112:0/2888273848 learned_addr learned my addr 172.21.15.112:0/2888273848 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c37fe700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22c40b3440 0x7f22c4142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c3fff700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22c415e6e0 0x7f22c406ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c2ffd700 1 -- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22c40b3440 msgr2=0x7f22c4142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c2ffd700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22c40b3440 0x7f22c4142c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.807+0000 7f22c2ffd700 1 -- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22c415e6e0 msgr2=0x7f22c406ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.808+0000 7f22c2ffd700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22c415e6e0 0x7f22c406ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.808+0000 7f22c2ffd700 1 -- 172.21.15.112:0/2888273848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b40094a0 con 0x7f22c4146c20 2023-08-27T11:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.808+0000 7f22c37fe700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22c40b3440 0x7f22c4142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.808+0000 7f22c3fff700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22c415e6e0 0x7f22c406ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:02.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.808+0000 7f22c2ffd700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c4146c20 0x7f22c4160210 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f22b40090b0 tx=0x7f22b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:02.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.808+0000 7f22abfff700 1 -- 172.21.15.112:0/2888273848 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b40038a0 con 0x7f22c4146c20 2023-08-27T11:26:02.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.808+0000 7f22abfff700 1 -- 172.21.15.112:0/2888273848 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b401f070 con 0x7f22c4146c20 2023-08-27T11:26:02.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.808+0000 7f22abfff700 1 -- 172.21.15.112:0/2888273848 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4003c40 con 0x7f22c4146c20 2023-08-27T11:26:02.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.808+0000 7f22c9e95700 1 -- 172.21.15.112:0/2888273848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c41f6050 con 0x7f22c4146c20 2023-08-27T11:26:02.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.808+0000 7f22c9e95700 1 -- 172.21.15.112:0/2888273848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c41f6610 con 0x7f22c4146c20 2023-08-27T11:26:02.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.809+0000 7f22c9e95700 1 -- 172.21.15.112:0/2888273848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c4067e90 con 0x7f22c4146c20 2023-08-27T11:26:02.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.809+0000 7f22abfff700 1 -- 172.21.15.112:0/2888273848 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f22b4026480 con 0x7f22c4146c20 2023-08-27T11:26:02.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.809+0000 7f22abfff700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22b006a3b0 0x7f22b006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:02.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.810+0000 7f22abfff700 1 -- 172.21.15.112:0/2888273848 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f22b4021070 con 0x7f22c4146c20 2023-08-27T11:26:02.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.810+0000 7f22c37fe700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22b006a3b0 0x7f22b006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:02.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.810+0000 7f22c37fe700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22b006a3b0 0x7f22b006c870 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f22b8006fd0 tx=0x7f22b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:02.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:02.814+0000 7f22abfff700 1 -- 172.21.15.112:0/2888273848 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f22b4055880 con 0x7f22c4146c20 2023-08-27T11:26:03.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.001+0000 7f22c9e95700 1 -- 172.21.15.112:0/2888273848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f22c4067a20 con 0x7f22c4146c20 2023-08-27T11:26:03.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.001+0000 7f22abfff700 1 -- 172.21.15.112:0/2888273848 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v79) v1 ==== 92+0+10 (secure 0 0 0) 0x7f22b40045b0 con 0x7f22c4146c20 2023-08-27T11:26:03.003 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:26:03.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.003+0000 7f22c9e95700 1 -- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22b006a3b0 msgr2=0x7f22b006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.003+0000 7f22c9e95700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22b006a3b0 0x7f22b006c870 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f22b8006fd0 tx=0x7f22b8008040 comp rx=0 tx=0).stop 2023-08-27T11:26:03.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.003+0000 7f22c9e95700 1 -- 172.21.15.112:0/2888273848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c4146c20 msgr2=0x7f22c4160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.003+0000 7f22c9e95700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c4146c20 0x7f22c4160210 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f22b40090b0 tx=0x7f22b4004750 comp rx=0 tx=0).stop 2023-08-27T11:26:03.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.003+0000 7f22c9e95700 1 -- 172.21.15.112:0/2888273848 shutdown_connections 2023-08-27T11:26:03.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.003+0000 7f22c9e95700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22b006a3b0 0x7f22b006c870 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.003+0000 7f22c9e95700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22c415e6e0 0x7f22c406ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.003+0000 7f22c9e95700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22c40b3440 0x7f22c4142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.003+0000 7f22c9e95700 1 --2- 172.21.15.112:0/2888273848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c4146c20 0x7f22c4160210 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.003+0000 7f22c9e95700 1 -- 172.21.15.112:0/2888273848 >> 172.21.15.112:0/2888273848 conn(0x7f22c40b1e90 msgr2=0x7f22c40b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:03.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.003+0000 7f22c9e95700 1 -- 172.21.15.112:0/2888273848 shutdown_connections 2023-08-27T11:26:03.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.003+0000 7f22c9e95700 1 -- 172.21.15.112:0/2888273848 wait complete. 2023-08-27T11:26:03.017 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:26:03.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.138+0000 7f5fb974c700 1 Processor -- start 2023-08-27T11:26:03.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.138+0000 7f5fb974c700 1 -- start start 2023-08-27T11:26:03.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.139+0000 7f5fb974c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fb4146bf0 0x7f5fb40b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.139+0000 7f5fb974c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb40d1500 con 0x7f5fb4146fd0 2023-08-27T11:26:03.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.139+0000 7f5fb974c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb40d0460 con 0x7f5fb4070eb0 2023-08-27T11:26:03.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.139+0000 7f5fb974c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40d0710 con 0x7f5fb4146bf0 2023-08-27T11:26:03.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.139+0000 7f5fb27fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5fb4146fd0 0x7f5fb40cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54928/0 (socket says 172.21.15.112:54928) 2023-08-27T11:26:03.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.139+0000 7f5fb27fc700 1 -- 172.21.15.112:0/1974694936 learned_addr learned my addr 172.21.15.112:0/1974694936 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:03.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.139+0000 7f5fb2ffd700 1 --2- 172.21.15.112:0/1974694936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fb4146bf0 0x7f5fb40b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.139+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276030570 0 0) 0x7f5fb40d1500 con 0x7f5fb4146fd0 2023-08-27T11:26:03.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.139+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f9c0035b0 con 0x7f5fb4146fd0 2023-08-27T11:26:03.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1149396222 0 0) 0x7f5f9c0035b0 con 0x7f5fb4146fd0 2023-08-27T11:26:03.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fb4146bf0 msgr2=0x7f5fb40b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:03.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 --2- 172.21.15.112:0/1974694936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fb4146bf0 0x7f5fb40b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 >> v1:172.21.15.138:6789/0 conn(0x7f5fb4070eb0 legacy=0x7f5fb40d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb40d09f0 con 0x7f5fb4146fd0 2023-08-27T11:26:03.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f5fa4001670 con 0x7f5fb4146fd0 2023-08-27T11:26:03.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 >> v1:172.21.15.112:6789/0 conn(0x7f5fb4146fd0 legacy=0x7f5fb40cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 --2- 172.21.15.112:0/1974694936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c003a30 0x7f5f9c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 --2- 172.21.15.112:0/1974694936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fb4146bf0 0x7f5f9c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 --2- 172.21.15.112:0/1974694936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f9c004d90 0x7f5f9c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40d0710 con 0x7f5f9c003a30 2023-08-27T11:26:03.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c00bb40 con 0x7f5f9c004d90 2023-08-27T11:26:03.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c00bd00 con 0x7f5fb4146bf0 2023-08-27T11:26:03.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb2ffd700 1 --2- 172.21.15.112:0/1974694936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c003a30 0x7f5f9c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb27fc700 1 --2- 172.21.15.112:0/1974694936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fb4146bf0 0x7f5f9c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb37fe700 1 --2- 172.21.15.112:0/1974694936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f9c004d90 0x7f5f9c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb2ffd700 1 -- 172.21.15.112:0/1974694936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fb4146bf0 msgr2=0x7f5f9c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:03.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb2ffd700 1 --2- 172.21.15.112:0/1974694936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fb4146bf0 0x7f5f9c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb2ffd700 1 -- 172.21.15.112:0/1974694936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f9c004d90 msgr2=0x7f5f9c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:03.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb2ffd700 1 --2- 172.21.15.112:0/1974694936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f9c004d90 0x7f5f9c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb2ffd700 1 -- 172.21.15.112:0/1974694936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c0035b0 con 0x7f5f9c003a30 2023-08-27T11:26:03.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb2ffd700 1 --2- 172.21.15.112:0/1974694936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c003a30 0x7f5f9c003e50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009b60 tx=0x7f5f9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acceb6aa3b3c8996 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:03.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa800e6c0 con 0x7f5f9c003a30 2023-08-27T11:26:03.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa800e820 con 0x7f5f9c003a30 2023-08-27T11:26:03.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.140+0000 7f5fb1ffb700 1 -- 172.21.15.112:0/1974694936 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa800eb90 con 0x7f5f9c003a30 2023-08-27T11:26:03.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.141+0000 7f5fb974c700 1 -- 172.21.15.112:0/1974694936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c003a30 msgr2=0x7f5f9c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.141+0000 7f5fb974c700 1 --2- 172.21.15.112:0/1974694936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c003a30 0x7f5f9c003e50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009b60 tx=0x7f5f9c005880 comp rx=0 tx=0).stop 2023-08-27T11:26:03.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.141+0000 7f5fb37fe700 1 -- 172.21.15.112:0/1974694936 reap_dead start 2023-08-27T11:26:03.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.141+0000 7f5fb974c700 1 -- 172.21.15.112:0/1974694936 shutdown_connections 2023-08-27T11:26:03.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.141+0000 7f5fb974c700 1 -- 172.21.15.112:0/1974694936 >> 172.21.15.112:0/1974694936 conn(0x7f5fb406c450 msgr2=0x7f5fb406f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:03.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.141+0000 7f5fb974c700 1 -- 172.21.15.112:0/1974694936 shutdown_connections 2023-08-27T11:26:03.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.141+0000 7f5fb974c700 1 -- 172.21.15.112:0/1974694936 wait complete. 2023-08-27T11:26:03.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.142+0000 7f5fb974c700 1 Processor -- start 2023-08-27T11:26:03.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.142+0000 7f5fb974c700 1 -- start start 2023-08-27T11:26:03.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb974c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f9c003a30 0x7f5fb40ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb974c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb40b2400 0x7f5fb415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb974c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb4145560 0x7f5fb41459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb974c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40b03e0 con 0x7f5fb40b2400 2023-08-27T11:26:03.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb974c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb4145f20 con 0x7f5fb4145560 2023-08-27T11:26:03.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb974c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40ad7a0 con 0x7f5f9c003a30 2023-08-27T11:26:03.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb40b2400 0x7f5fb415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb40b2400 0x7f5fb415cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60562/0 (socket says 172.21.15.112:60562) 2023-08-27T11:26:03.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb27fc700 1 -- 172.21.15.112:0/1262541895 learned_addr learned my addr 172.21.15.112:0/1262541895 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:03.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb2ffd700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f9c003a30 0x7f5fb40ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb37fe700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb4145560 0x7f5fb41459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb27fc700 1 -- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f9c003a30 msgr2=0x7f5fb40ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb27fc700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f9c003a30 0x7f5fb40ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb27fc700 1 -- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb4145560 msgr2=0x7f5fb41459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb27fc700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb4145560 0x7f5fb41459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb27fc700 1 -- 172.21.15.112:0/1262541895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb41f7090 con 0x7f5fb40b2400 2023-08-27T11:26:03.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.143+0000 7f5fb2ffd700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f9c003a30 0x7f5fb40ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:03.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.144+0000 7f5fb27fc700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb40b2400 0x7f5fb415cf30 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4007290 tx=0x7f5fa4007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:03.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.144+0000 7f5f9b7fe700 1 -- 172.21.15.112:0/1262541895 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa400f6c0 con 0x7f5fb40b2400 2023-08-27T11:26:03.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.144+0000 7f5f9b7fe700 1 -- 172.21.15.112:0/1262541895 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa400f840 con 0x7f5fb40b2400 2023-08-27T11:26:03.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.144+0000 7f5f9b7fe700 1 -- 172.21.15.112:0/1262541895 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa400fbf0 con 0x7f5fb40b2400 2023-08-27T11:26:03.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.144+0000 7f5fb974c700 1 -- 172.21.15.112:0/1262541895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb41f6480 con 0x7f5fb40b2400 2023-08-27T11:26:03.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.144+0000 7f5fb974c700 1 -- 172.21.15.112:0/1262541895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb41f6a40 con 0x7f5fb40b2400 2023-08-27T11:26:03.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.145+0000 7f5fb974c700 1 -- 172.21.15.112:0/1262541895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb41f7220 con 0x7f5fb40b2400 2023-08-27T11:26:03.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.145+0000 7f5f9b7fe700 1 -- 172.21.15.112:0/1262541895 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f5fa400fd50 con 0x7f5fb40b2400 2023-08-27T11:26:03.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.145+0000 7f5f9b7fe700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fa006a3b0 0x7f5fa006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.146+0000 7f5f9b7fe700 1 -- 172.21.15.112:0/1262541895 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f5fa4088a20 con 0x7f5fb40b2400 2023-08-27T11:26:03.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.146+0000 7f5fb2ffd700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fa006a3b0 0x7f5fa006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.146+0000 7f5fb2ffd700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fa006a3b0 0x7f5fa006c870 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80090b0 tx=0x7f5fa8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:03.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.150+0000 7f5f9b7fe700 1 -- 172.21.15.112:0/1262541895 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5fa4053dd0 con 0x7f5fb40b2400 2023-08-27T11:26:03.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.337+0000 7f5fb974c700 1 -- 172.21.15.112:0/1262541895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5fb4067e90 con 0x7f5fb40b2400 2023-08-27T11:26:03.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.338+0000 7f5f9b7fe700 1 -- 172.21.15.112:0/1262541895 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5fa40573f0 con 0x7f5fb40b2400 2023-08-27T11:26:03.340 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:26:03.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.340+0000 7f5fb974c700 1 -- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fa006a3b0 msgr2=0x7f5fa006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.340+0000 7f5fb974c700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fa006a3b0 0x7f5fa006c870 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80090b0 tx=0x7f5fa8008030 comp rx=0 tx=0).stop 2023-08-27T11:26:03.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.340+0000 7f5fb974c700 1 -- 172.21.15.112:0/1262541895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb40b2400 msgr2=0x7f5fb415cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.340+0000 7f5fb974c700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb40b2400 0x7f5fb415cf30 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4007290 tx=0x7f5fa4007650 comp rx=0 tx=0).stop 2023-08-27T11:26:03.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.340+0000 7f5fb974c700 1 -- 172.21.15.112:0/1262541895 shutdown_connections 2023-08-27T11:26:03.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.340+0000 7f5fb974c700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fa006a3b0 0x7f5fa006c870 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.340+0000 7f5fb974c700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb4145560 0x7f5fb41459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.340+0000 7f5fb974c700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f9c003a30 0x7f5fb40ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.340+0000 7f5fb974c700 1 --2- 172.21.15.112:0/1262541895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb40b2400 0x7f5fb415cf30 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.340+0000 7f5fb974c700 1 -- 172.21.15.112:0/1262541895 >> 172.21.15.112:0/1262541895 conn(0x7f5fb406c450 msgr2=0x7f5fb40b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:03.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.340+0000 7f5fb974c700 1 -- 172.21.15.112:0/1262541895 shutdown_connections 2023-08-27T11:26:03.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.340+0000 7f5fb974c700 1 -- 172.21.15.112:0/1262541895 wait complete. 2023-08-27T11:26:03.353 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:26:03.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.477+0000 7f87a5a55700 1 Processor -- start 2023-08-27T11:26:03.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.477+0000 7f87a5a55700 1 -- start start 2023-08-27T11:26:03.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.477+0000 7f87a5a55700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87a0146fd0 0x7f87a00b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.477+0000 7f87a5a55700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f87a0070eb0 0x7f87a0071320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f87a5a55700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a00d1500 con 0x7f87a0146bf0 2023-08-27T11:26:03.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f87a5a55700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a0071860 con 0x7f87a0146fd0 2023-08-27T11:26:03.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f87a5a55700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a00d0460 con 0x7f87a0070eb0 2023-08-27T11:26:03.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f879e7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87a0146fd0 0x7f87a00b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f879e7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87a0146fd0 0x7f87a00b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60568/0 (socket says 172.21.15.112:60568) 2023-08-27T11:26:03.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f879e7fc700 1 -- 172.21.15.112:0/1681597778 learned_addr learned my addr 172.21.15.112:0/1681597778 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:03.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f879e7fc700 1 -- 172.21.15.112:0/1681597778 >> v2:172.21.15.138:3300/0 conn(0x7f87a0070eb0 msgr2=0x7f87a0071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:03.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f879e7fc700 1 --2- 172.21.15.112:0/1681597778 >> v2:172.21.15.138:3300/0 conn(0x7f87a0070eb0 0x7f87a0071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f879e7fc700 1 -- 172.21.15.112:0/1681597778 >> v1:172.21.15.138:6789/0 conn(0x7f87a0146bf0 legacy=0x7f87a00cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:03.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f879e7fc700 1 -- 172.21.15.112:0/1681597778 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a00d0710 con 0x7f87a0146fd0 2023-08-27T11:26:03.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f879e7fc700 1 --2- 172.21.15.112:0/1681597778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0146fd0 0x7f87a00b5db0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f87a00b58f0 tx=0x7f8794009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2ae7a402f3a6d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:03.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f879dffb700 1 -- 172.21.15.112:0/1681597778 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794004230 con 0x7f87a0146fd0 2023-08-27T11:26:03.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f879dffb700 1 -- 172.21.15.112:0/1681597778 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8794004390 con 0x7f87a0146fd0 2023-08-27T11:26:03.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.478+0000 7f879dffb700 1 -- 172.21.15.112:0/1681597778 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87940036a0 con 0x7f87a0146fd0 2023-08-27T11:26:03.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.479+0000 7f87a5a55700 1 -- 172.21.15.112:0/1681597778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0146fd0 msgr2=0x7f87a00b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.479+0000 7f87a5a55700 1 --2- 172.21.15.112:0/1681597778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0146fd0 0x7f87a00b5db0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f87a00b58f0 tx=0x7f8794009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.479+0000 7f87a5a55700 1 -- 172.21.15.112:0/1681597778 shutdown_connections 2023-08-27T11:26:03.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.479+0000 7f87a5a55700 1 --2- 172.21.15.112:0/1681597778 >> v2:172.21.15.138:3300/0 conn(0x7f87a0070eb0 0x7f87a0071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.479+0000 7f87a5a55700 1 --2- 172.21.15.112:0/1681597778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0146fd0 0x7f87a00b5db0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.479+0000 7f87a5a55700 1 -- 172.21.15.112:0/1681597778 >> 172.21.15.112:0/1681597778 conn(0x7f87a006c450 msgr2=0x7f87a006f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:03.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.479+0000 7f87a5a55700 1 -- 172.21.15.112:0/1681597778 shutdown_connections 2023-08-27T11:26:03.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.479+0000 7f87a5a55700 1 -- 172.21.15.112:0/1681597778 wait complete. 2023-08-27T11:26:03.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.480+0000 7f87a5a55700 1 Processor -- start 2023-08-27T11:26:03.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.480+0000 7f87a5a55700 1 -- start start 2023-08-27T11:26:03.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f87a5a55700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87a0070eb0 0x7f87a0141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f87a5a55700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87a0146bf0 0x7f87a00b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f87a5a55700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0146fd0 0x7f87a00b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f87a5a55700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00b62b0 con 0x7f87a0146fd0 2023-08-27T11:26:03.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f87a5a55700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00b6550 con 0x7f87a0146bf0 2023-08-27T11:26:03.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f87a5a55700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00b6830 con 0x7f87a0070eb0 2023-08-27T11:26:03.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f879f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0146fd0 0x7f87a00b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f879f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0146fd0 0x7f87a00b75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60576/0 (socket says 172.21.15.112:60576) 2023-08-27T11:26:03.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f879f7fe700 1 -- 172.21.15.112:0/675405148 learned_addr learned my addr 172.21.15.112:0/675405148 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:03.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f879effd700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87a0070eb0 0x7f87a0141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f879e7fc700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87a0146bf0 0x7f87a00b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f879f7fe700 1 -- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87a0070eb0 msgr2=0x7f87a0141fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f879f7fe700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87a0070eb0 0x7f87a0141fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f879f7fe700 1 -- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87a0146bf0 msgr2=0x7f87a00b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f879f7fe700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87a0146bf0 0x7f87a00b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.481+0000 7f879f7fe700 1 -- 172.21.15.112:0/675405148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8794009550 con 0x7f87a0146fd0 2023-08-27T11:26:03.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.482+0000 7f879f7fe700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0146fd0 0x7f87a00b75f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f879800b970 tx=0x7f879800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:03.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.482+0000 7f8787fff700 1 -- 172.21.15.112:0/675405148 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879800d860 con 0x7f87a0146fd0 2023-08-27T11:26:03.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.482+0000 7f8787fff700 1 -- 172.21.15.112:0/675405148 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879800d9e0 con 0x7f87a0146fd0 2023-08-27T11:26:03.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.482+0000 7f87a5a55700 1 -- 172.21.15.112:0/675405148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a00b6b10 con 0x7f87a0146fd0 2023-08-27T11:26:03.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.482+0000 7f8787fff700 1 -- 172.21.15.112:0/675405148 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879800dd90 con 0x7f87a0146fd0 2023-08-27T11:26:03.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.482+0000 7f87a5a55700 1 -- 172.21.15.112:0/675405148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a00b70d0 con 0x7f87a0146fd0 2023-08-27T11:26:03.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.483+0000 7f87a5a55700 1 -- 172.21.15.112:0/675405148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a0067e90 con 0x7f87a0146fd0 2023-08-27T11:26:03.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.483+0000 7f8787fff700 1 -- 172.21.15.112:0/675405148 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f879800e430 con 0x7f87a0146fd0 2023-08-27T11:26:03.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.483+0000 7f8787fff700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f878c06a3b0 0x7f878c06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.484+0000 7f8787fff700 1 -- 172.21.15.112:0/675405148 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f8798006e40 con 0x7f87a0146fd0 2023-08-27T11:26:03.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.484+0000 7f879effd700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f878c06a3b0 0x7f878c06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.484+0000 7f879effd700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f878c06a3b0 0x7f878c06c870 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8790007fd0 tx=0x7f8790009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:03.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.488+0000 7f8787fff700 1 -- 172.21.15.112:0/675405148 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f879808ae60 con 0x7f87a0146fd0 2023-08-27T11:26:03.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.677+0000 7f87a5a55700 1 -- 172.21.15.112:0/675405148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87a00d0ca0 con 0x7f87a0146fd0 2023-08-27T11:26:03.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.677+0000 7f8787fff700 1 -- 172.21.15.112:0/675405148 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8798053e50 con 0x7f87a0146fd0 2023-08-27T11:26:03.679 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:26:03.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.679+0000 7f87a5a55700 1 -- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f878c06a3b0 msgr2=0x7f878c06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.679+0000 7f87a5a55700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f878c06a3b0 0x7f878c06c870 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8790007fd0 tx=0x7f8790009040 comp rx=0 tx=0).stop 2023-08-27T11:26:03.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.679+0000 7f87a5a55700 1 -- 172.21.15.112:0/675405148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0146fd0 msgr2=0x7f87a00b75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.679+0000 7f87a5a55700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0146fd0 0x7f87a00b75f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f879800b970 tx=0x7f879800bc80 comp rx=0 tx=0).stop 2023-08-27T11:26:03.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.679+0000 7f87a5a55700 1 -- 172.21.15.112:0/675405148 shutdown_connections 2023-08-27T11:26:03.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.679+0000 7f87a5a55700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f878c06a3b0 0x7f878c06c870 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.679+0000 7f87a5a55700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0146fd0 0x7f87a00b75f0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.679+0000 7f87a5a55700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87a0070eb0 0x7f87a0141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.679+0000 7f87a5a55700 1 --2- 172.21.15.112:0/675405148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87a0146bf0 0x7f87a00b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.679+0000 7f87a5a55700 1 -- 172.21.15.112:0/675405148 >> 172.21.15.112:0/675405148 conn(0x7f87a006c450 msgr2=0x7f87a006dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:03.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.680+0000 7f87a5a55700 1 -- 172.21.15.112:0/675405148 shutdown_connections 2023-08-27T11:26:03.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.680+0000 7f87a5a55700 1 -- 172.21.15.112:0/675405148 wait complete. 2023-08-27T11:26:03.693 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:26:03.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.815+0000 7fc6ecd24700 1 Processor -- start 2023-08-27T11:26:03.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.815+0000 7fc6ecd24700 1 -- start start 2023-08-27T11:26:03.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6ecd24700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc6e80b3440 0x7fc6e815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6ecd24700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc6e815e6e0 0x7fc6e806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6ecd24700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e80b5070 con 0x7fc6e8146c20 2023-08-27T11:26:03.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6ecd24700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e8147000 con 0x7fc6e815e6e0 2023-08-27T11:26:03.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6ecd24700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e8161740 con 0x7fc6e80b3440 2023-08-27T11:26:03.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6e6d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc6e815e6e0 0x7fc6e806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6e6d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc6e815e6e0 0x7fc6e806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60584/0 (socket says 172.21.15.112:60584) 2023-08-27T11:26:03.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6e6d9d700 1 -- 172.21.15.112:0/2217626704 learned_addr learned my addr 172.21.15.112:0/2217626704 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:03.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6e5d9b700 1 --2- 172.21.15.112:0/2217626704 >> v2:172.21.15.138:3300/0 conn(0x7fc6e80b3440 0x7fc6e815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6e6d9d700 1 -- 172.21.15.112:0/2217626704 >> v2:172.21.15.138:3300/0 conn(0x7fc6e80b3440 msgr2=0x7fc6e815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:03.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6e6d9d700 1 --2- 172.21.15.112:0/2217626704 >> v2:172.21.15.138:3300/0 conn(0x7fc6e80b3440 0x7fc6e815e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6e6d9d700 1 -- 172.21.15.112:0/2217626704 >> v1:172.21.15.138:6789/0 conn(0x7fc6e8146c20 legacy=0x7fc6e8160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:03.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6e6d9d700 1 -- 172.21.15.112:0/2217626704 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e8161a20 con 0x7fc6e815e6e0 2023-08-27T11:26:03.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6e6d9d700 1 --2- 172.21.15.112:0/2217626704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6e815e6e0 0x7fc6e806f790 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80b3f00 tx=0x7fc6e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692c6a30212ef600 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:03.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.816+0000 7fc6e559a700 1 -- 172.21.15.112:0/2217626704 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000e070 con 0x7fc6e815e6e0 2023-08-27T11:26:03.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.817+0000 7fc6e559a700 1 -- 172.21.15.112:0/2217626704 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e0003a20 con 0x7fc6e815e6e0 2023-08-27T11:26:03.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.817+0000 7fc6e559a700 1 -- 172.21.15.112:0/2217626704 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0003dd0 con 0x7fc6e815e6e0 2023-08-27T11:26:03.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.817+0000 7fc6ecd24700 1 -- 172.21.15.112:0/2217626704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6e815e6e0 msgr2=0x7fc6e806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.817+0000 7fc6ecd24700 1 --2- 172.21.15.112:0/2217626704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6e815e6e0 0x7fc6e806f790 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80b3f00 tx=0x7fc6e000b330 comp rx=0 tx=0).stop 2023-08-27T11:26:03.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.817+0000 7fc6ecd24700 1 -- 172.21.15.112:0/2217626704 shutdown_connections 2023-08-27T11:26:03.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.817+0000 7fc6ecd24700 1 --2- 172.21.15.112:0/2217626704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6e815e6e0 0x7fc6e806f790 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.817+0000 7fc6ecd24700 1 --2- 172.21.15.112:0/2217626704 >> v2:172.21.15.138:3300/0 conn(0x7fc6e80b3440 0x7fc6e815e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.817+0000 7fc6ecd24700 1 -- 172.21.15.112:0/2217626704 >> 172.21.15.112:0/2217626704 conn(0x7fc6e80b1e90 msgr2=0x7fc6e80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:03.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.817+0000 7fc6ecd24700 1 -- 172.21.15.112:0/2217626704 shutdown_connections 2023-08-27T11:26:03.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.817+0000 7fc6ecd24700 1 -- 172.21.15.112:0/2217626704 wait complete. 2023-08-27T11:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.819+0000 7fc6ecd24700 1 Processor -- start 2023-08-27T11:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.819+0000 7fc6ecd24700 1 -- start start 2023-08-27T11:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6ecd24700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6e80b3440 0x7fc6e815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6ecd24700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6e8146c20 0x7fc6e80ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6ecd24700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e815e6e0 0x7fc6e80aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6ecd24700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e815b1b0 con 0x7fc6e80b3440 2023-08-27T11:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6ecd24700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e815eaf0 con 0x7fc6e815e6e0 2023-08-27T11:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6ecd24700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e815cd50 con 0x7fc6e8146c20 2023-08-27T11:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6e659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6e80b3440 0x7fc6e815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6e659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6e80b3440 0x7fc6e815ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60586/0 (socket says 172.21.15.112:60586) 2023-08-27T11:26:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6e659c700 1 -- 172.21.15.112:0/1873832039 learned_addr learned my addr 172.21.15.112:0/1873832039 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6e6d9d700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e815e6e0 0x7fc6e80aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6e5d9b700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6e8146c20 0x7fc6e80ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6e659c700 1 -- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6e8146c20 msgr2=0x7fc6e80ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6e659c700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6e8146c20 0x7fc6e80ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6e659c700 1 -- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e815e6e0 msgr2=0x7fc6e80aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:03.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6e659c700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e815e6e0 0x7fc6e80aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:03.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6e659c700 1 -- 172.21.15.112:0/1873832039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e0009b30 con 0x7fc6e80b3440 2023-08-27T11:26:03.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6e6d9d700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e815e6e0 0x7fc6e80aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:03.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6e659c700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6e80b3440 0x7fc6e815ac70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c4d0 tx=0x7fc6d800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:03.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.820+0000 7fc6d6ffd700 1 -- 172.21.15.112:0/1873832039 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8011790 con 0x7fc6e80b3440 2023-08-27T11:26:03.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.821+0000 7fc6d6ffd700 1 -- 172.21.15.112:0/1873832039 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d80118f0 con 0x7fc6e80b3440 2023-08-27T11:26:03.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.821+0000 7fc6ecd24700 1 -- 172.21.15.112:0/1873832039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e815d030 con 0x7fc6e80b3440 2023-08-27T11:26:03.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.821+0000 7fc6d6ffd700 1 -- 172.21.15.112:0/1873832039 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8011ca0 con 0x7fc6e80b3440 2023-08-27T11:26:03.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.821+0000 7fc6ecd24700 1 -- 172.21.15.112:0/1873832039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e815f020 con 0x7fc6e80b3440 2023-08-27T11:26:03.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.821+0000 7fc6ecd24700 1 -- 172.21.15.112:0/1873832039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e8067e90 con 0x7fc6e80b3440 2023-08-27T11:26:03.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.822+0000 7fc6d6ffd700 1 -- 172.21.15.112:0/1873832039 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fc6d801c460 con 0x7fc6e80b3440 2023-08-27T11:26:03.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.822+0000 7fc6d6ffd700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc6c806a3b0 0x7fc6c806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:03.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.822+0000 7fc6d6ffd700 1 -- 172.21.15.112:0/1873832039 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fc6d8088810 con 0x7fc6e80b3440 2023-08-27T11:26:03.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.822+0000 7fc6e5d9b700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc6c806a3b0 0x7fc6c806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:03.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.823+0000 7fc6e5d9b700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc6c806a3b0 0x7fc6c806c870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009950 tx=0x7fc6dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:03.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:03.827+0000 7fc6d6ffd700 1 -- 172.21.15.112:0/1873832039 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc6d8053bc0 con 0x7fc6e80b3440 2023-08-27T11:26:04.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.015+0000 7fc6ecd24700 1 -- 172.21.15.112:0/1873832039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6e8067a20 con 0x7fc6e80b3440 2023-08-27T11:26:04.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.015+0000 7fc6d6ffd700 1 -- 172.21.15.112:0/1873832039 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6d80571e0 con 0x7fc6e80b3440 2023-08-27T11:26:04.017 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:26:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.017+0000 7fc6ecd24700 1 -- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc6c806a3b0 msgr2=0x7fc6c806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.017+0000 7fc6ecd24700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc6c806a3b0 0x7fc6c806c870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009950 tx=0x7fc6dc008040 comp rx=0 tx=0).stop 2023-08-27T11:26:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.017+0000 7fc6ecd24700 1 -- 172.21.15.112:0/1873832039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6e80b3440 msgr2=0x7fc6e815ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.017+0000 7fc6ecd24700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6e80b3440 0x7fc6e815ac70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c4d0 tx=0x7fc6d800c890 comp rx=0 tx=0).stop 2023-08-27T11:26:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.017+0000 7fc6ecd24700 1 -- 172.21.15.112:0/1873832039 shutdown_connections 2023-08-27T11:26:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.017+0000 7fc6ecd24700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc6c806a3b0 0x7fc6c806c870 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.017+0000 7fc6ecd24700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e815e6e0 0x7fc6e80aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.017+0000 7fc6ecd24700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6e80b3440 0x7fc6e815ac70 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.017+0000 7fc6ecd24700 1 --2- 172.21.15.112:0/1873832039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6e8146c20 0x7fc6e80ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.017+0000 7fc6ecd24700 1 -- 172.21.15.112:0/1873832039 >> 172.21.15.112:0/1873832039 conn(0x7fc6e80b1e90 msgr2=0x7fc6e80b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.017+0000 7fc6ecd24700 1 -- 172.21.15.112:0/1873832039 shutdown_connections 2023-08-27T11:26:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.017+0000 7fc6ecd24700 1 -- 172.21.15.112:0/1873832039 wait complete. 2023-08-27T11:26:04.031 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:26:04.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.149+0000 7fa07c18e700 1 Processor -- start 2023-08-27T11:26:04.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.149+0000 7fa07c18e700 1 -- start start 2023-08-27T11:26:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa07c18e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0740b1400 0x7fa0740cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa07c18e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0740b17e0 0x7fa07406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa07c18e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa074161ce0 0x7fa07406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa07c18e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa074161850 con 0x7fa0740b1400 2023-08-27T11:26:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa07c18e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07406e7d0 con 0x7fa074161ce0 2023-08-27T11:26:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa07c18e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740cfc10 con 0x7fa0740b17e0 2023-08-27T11:26:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa079f2a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0740b1400 0x7fa0740cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa079f2a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0740b1400 0x7fa0740cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60594/0 (socket says 172.21.15.112:60594) 2023-08-27T11:26:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa079f2a700 1 -- 172.21.15.112:0/1532179422 learned_addr learned my addr 172.21.15.112:0/1532179422 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa079729700 1 --2- 172.21.15.112:0/1532179422 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0740b17e0 0x7fa07406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa07a72b700 1 --2- 172.21.15.112:0/1532179422 >> v2:172.21.15.138:3300/0 conn(0x7fa074161ce0 0x7fa07406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa079f2a700 1 -- 172.21.15.112:0/1532179422 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0740b17e0 msgr2=0x7fa07406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa079f2a700 1 --2- 172.21.15.112:0/1532179422 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0740b17e0 0x7fa07406e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa079f2a700 1 -- 172.21.15.112:0/1532179422 >> v2:172.21.15.138:3300/0 conn(0x7fa074161ce0 msgr2=0x7fa07406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa079f2a700 1 --2- 172.21.15.112:0/1532179422 >> v2:172.21.15.138:3300/0 conn(0x7fa074161ce0 0x7fa07406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa079f2a700 1 -- 172.21.15.112:0/1532179422 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa074161200 con 0x7fa0740b1400 2023-08-27T11:26:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa079f2a700 1 --2- 172.21.15.112:0/1532179422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0740b1400 0x7fa0740cf5a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa0740ca550 tx=0x7fa064009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671ac8968264b102 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa078f28700 1 -- 172.21.15.112:0/1532179422 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06400b870 con 0x7fa0740b1400 2023-08-27T11:26:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa078f28700 1 -- 172.21.15.112:0/1532179422 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06400b9d0 con 0x7fa0740b1400 2023-08-27T11:26:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.150+0000 7fa078f28700 1 -- 172.21.15.112:0/1532179422 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06400bd60 con 0x7fa0740b1400 2023-08-27T11:26:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.151+0000 7fa07c18e700 1 -- 172.21.15.112:0/1532179422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0740b1400 msgr2=0x7fa0740cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.151+0000 7fa07c18e700 1 --2- 172.21.15.112:0/1532179422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0740b1400 0x7fa0740cf5a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa0740ca550 tx=0x7fa064009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.151+0000 7fa07c18e700 1 -- 172.21.15.112:0/1532179422 shutdown_connections 2023-08-27T11:26:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.151+0000 7fa07c18e700 1 --2- 172.21.15.112:0/1532179422 >> v2:172.21.15.138:3300/0 conn(0x7fa074161ce0 0x7fa07406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.151+0000 7fa07c18e700 1 --2- 172.21.15.112:0/1532179422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0740b1400 0x7fa0740cf5a0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.151+0000 7fa07c18e700 1 --2- 172.21.15.112:0/1532179422 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0740b17e0 0x7fa07406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.151+0000 7fa07c18e700 1 -- 172.21.15.112:0/1532179422 >> 172.21.15.112:0/1532179422 conn(0x7fa0740d0000 msgr2=0x7fa0740d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.151+0000 7fa07c18e700 1 -- 172.21.15.112:0/1532179422 shutdown_connections 2023-08-27T11:26:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.151+0000 7fa07c18e700 1 -- 172.21.15.112:0/1532179422 wait complete. 2023-08-27T11:26:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.152+0000 7fa07c18e700 1 Processor -- start 2023-08-27T11:26:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.152+0000 7fa07c18e700 1 -- start start 2023-08-27T11:26:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa07c18e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0740b1400 0x7fa0740c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa07c18e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0740b17e0 0x7fa074157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa07c18e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa074161ce0 0x7fa0740ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa07c18e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07415f250 con 0x7fa0740b17e0 2023-08-27T11:26:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa07c18e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740c9f00 con 0x7fa074161ce0 2023-08-27T11:26:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa07c18e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740ca1e0 con 0x7fa0740b1400 2023-08-27T11:26:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa079729700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0740b17e0 0x7fa074157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa079729700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0740b17e0 0x7fa074157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60602/0 (socket says 172.21.15.112:60602) 2023-08-27T11:26:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa079729700 1 -- 172.21.15.112:0/2812557481 learned_addr learned my addr 172.21.15.112:0/2812557481 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa07a72b700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa074161ce0 0x7fa0740ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa079729700 1 -- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0740b1400 msgr2=0x7fa0740c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa079f2a700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0740b1400 0x7fa0740c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa079729700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0740b1400 0x7fa0740c9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa079729700 1 -- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa074161ce0 msgr2=0x7fa0740ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa079729700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa074161ce0 0x7fa0740ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa079729700 1 -- 172.21.15.112:0/2812557481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064009550 con 0x7fa0740b17e0 2023-08-27T11:26:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.153+0000 7fa079729700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0740b17e0 0x7fa074157390 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c7f0 tx=0x7fa07000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.154+0000 7fa06a7fc700 1 -- 172.21.15.112:0/2812557481 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070011800 con 0x7fa0740b17e0 2023-08-27T11:26:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.154+0000 7fa06a7fc700 1 -- 172.21.15.112:0/2812557481 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa070011960 con 0x7fa0740b17e0 2023-08-27T11:26:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.154+0000 7fa07c18e700 1 -- 172.21.15.112:0/2812557481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07415ec00 con 0x7fa0740b17e0 2023-08-27T11:26:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.154+0000 7fa06a7fc700 1 -- 172.21.15.112:0/2812557481 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070011d10 con 0x7fa0740b17e0 2023-08-27T11:26:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.154+0000 7fa07c18e700 1 -- 172.21.15.112:0/2812557481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0740c9bd0 con 0x7fa0740b17e0 2023-08-27T11:26:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.154+0000 7fa07c18e700 1 -- 172.21.15.112:0/2812557481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07415f400 con 0x7fa0740b17e0 2023-08-27T11:26:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.155+0000 7fa06a7fc700 1 -- 172.21.15.112:0/2812557481 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fa070018360 con 0x7fa0740b17e0 2023-08-27T11:26:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.155+0000 7fa06a7fc700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa05806a3b0 0x7fa05806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.155+0000 7fa06a7fc700 1 -- 172.21.15.112:0/2812557481 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fa070088b10 con 0x7fa0740b17e0 2023-08-27T11:26:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.155+0000 7fa079f2a700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa05806a3b0 0x7fa05806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.156+0000 7fa079f2a700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa05806a3b0 0x7fa05806c870 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa074160900 tx=0x7fa064008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.160+0000 7fa06a7fc700 1 -- 172.21.15.112:0/2812557481 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa07415f400 con 0x7fa0740b17e0 2023-08-27T11:26:04.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.391+0000 7fa07c18e700 1 -- 172.21.15.112:0/2812557481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa074067a20 con 0x7fa0740b17e0 2023-08-27T11:26:04.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.392+0000 7fa06a7fc700 1 -- 172.21.15.112:0/2812557481 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4889 (secure 0 0 0) 0x7fa070015090 con 0x7fa0740b17e0 2023-08-27T11:26:04.394 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:04.394 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6166,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3262556844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3262556844},{"type":"v1","addr":"172.21.15.138:6836","nonce":3262556844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2232724696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2232724696},{"type":"v1","addr":"172.21.15.138:6837","nonce":2232724696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6570,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1511159838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1511159838},{"type":"v1","addr":"172.21.15.112:6834","nonce":1511159838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:25:41.794717+0000","modified":"2023-08-27T11:25:45.437236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2318724344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2318724344},{"type":"v1","addr":"172.21.15.112:6835","nonce":2318724344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-27T11:26:04.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.394+0000 7fa07c18e700 1 -- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa05806a3b0 msgr2=0x7fa05806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.394+0000 7fa07c18e700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa05806a3b0 0x7fa05806c870 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa074160900 tx=0x7fa064008030 comp rx=0 tx=0).stop 2023-08-27T11:26:04.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.394+0000 7fa07c18e700 1 -- 172.21.15.112:0/2812557481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0740b17e0 msgr2=0x7fa074157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.394+0000 7fa07c18e700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0740b17e0 0x7fa074157390 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c7f0 tx=0x7fa07000cbb0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.394+0000 7fa07c18e700 1 -- 172.21.15.112:0/2812557481 shutdown_connections 2023-08-27T11:26:04.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.394+0000 7fa07c18e700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa05806a3b0 0x7fa05806c870 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.394+0000 7fa07c18e700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa074161ce0 0x7fa0740ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.394+0000 7fa07c18e700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0740b1400 0x7fa0740c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.394+0000 7fa07c18e700 1 --2- 172.21.15.112:0/2812557481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0740b17e0 0x7fa074157390 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.394+0000 7fa07c18e700 1 -- 172.21.15.112:0/2812557481 >> 172.21.15.112:0/2812557481 conn(0x7fa0740d0000 msgr2=0x7fa074070530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:04.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.394+0000 7fa07c18e700 1 -- 172.21.15.112:0/2812557481 shutdown_connections 2023-08-27T11:26:04.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.394+0000 7fa07c18e700 1 -- 172.21.15.112:0/2812557481 wait complete. 2023-08-27T11:26:04.398 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 49 2023-08-27T11:26:04.407 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.527+0000 7fe822472700 1 Processor -- start 2023-08-27T11:26:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.527+0000 7fe822472700 1 -- start start 2023-08-27T11:26:04.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.528+0000 7fe822472700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c144570 0x7fe81c15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.528+0000 7fe822472700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe81c0b4ad0 0x7fe81c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.528+0000 7fe822472700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c0b45f0 con 0x7fe81c0b3e50 2023-08-27T11:26:04.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.528+0000 7fe822472700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c0cf7a0 con 0x7fe81c0b4ad0 2023-08-27T11:26:04.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.528+0000 7fe822472700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c144950 con 0x7fe81c144570 2023-08-27T11:26:04.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.528+0000 7fe81b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe81c0b3e50 0x7fe81c15f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54992/0 (socket says 172.21.15.112:54992) 2023-08-27T11:26:04.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.528+0000 7fe81b7fe700 1 -- 172.21.15.112:0/3088345236 learned_addr learned my addr 172.21.15.112:0/3088345236 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:04.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.528+0000 7fe820a0f700 1 --2- 172.21.15.112:0/3088345236 >> v2:172.21.15.138:3300/0 conn(0x7fe81c0b4ad0 0x7fe81c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.528+0000 7fe81bfff700 1 --2- 172.21.15.112:0/3088345236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c144570 0x7fe81c15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.528+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693406738 0 0) 0x7fe81c0b45f0 con 0x7fe81c0b3e50 2023-08-27T11:26:04.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.528+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8040035b0 con 0x7fe81c0b3e50 2023-08-27T11:26:04.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 889338282 0 0) 0x7fe8040035b0 con 0x7fe81c0b3e50 2023-08-27T11:26:04.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c144570 msgr2=0x7fe81c15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:04.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 --2- 172.21.15.112:0/3088345236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c144570 0x7fe81c15ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 >> v2:172.21.15.138:3300/0 conn(0x7fe81c0b4ad0 msgr2=0x7fe81c0b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:04.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 --2- 172.21.15.112:0/3088345236 >> v2:172.21.15.138:3300/0 conn(0x7fe81c0b4ad0 0x7fe81c0b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c0b4310 con 0x7fe81c0b3e50 2023-08-27T11:26:04.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fe80c001670 con 0x7fe81c0b3e50 2023-08-27T11:26:04.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 >> v1:172.21.15.112:6789/0 conn(0x7fe81c0b3e50 legacy=0x7fe81c15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 --2- 172.21.15.112:0/3088345236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c144570 0x7fe804003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 --2- 172.21.15.112:0/3088345236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe804003fb0 0x7fe804004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 --2- 172.21.15.112:0/3088345236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe804004d90 0x7fe804005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c0cf7a0 con 0x7fe804004d90 2023-08-27T11:26:04.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c144950 con 0x7fe804003fb0 2023-08-27T11:26:04.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80400ba90 con 0x7fe81c144570 2023-08-27T11:26:04.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe820a0f700 1 --2- 172.21.15.112:0/3088345236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe804004d90 0x7fe804005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81bfff700 1 --2- 172.21.15.112:0/3088345236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c144570 0x7fe804003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81b7fe700 1 --2- 172.21.15.112:0/3088345236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe804003fb0 0x7fe804004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe820a0f700 1 -- 172.21.15.112:0/3088345236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c144570 msgr2=0x7fe804003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:04.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe820a0f700 1 --2- 172.21.15.112:0/3088345236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe81c144570 0x7fe804003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe820a0f700 1 -- 172.21.15.112:0/3088345236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe804003fb0 msgr2=0x7fe804004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:04.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe820a0f700 1 --2- 172.21.15.112:0/3088345236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe804003fb0 0x7fe804004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe820a0f700 1 -- 172.21.15.112:0/3088345236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8040035b0 con 0x7fe804004d90 2023-08-27T11:26:04.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe820a0f700 1 --2- 172.21.15.112:0/3088345236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe804004d90 0x7fe804005210 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe814009fc0 tx=0x7fe804005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d668596b600ab86 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:04.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81400ec30 con 0x7fe804004d90 2023-08-27T11:26:04.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81400f040 con 0x7fe804004d90 2023-08-27T11:26:04.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.529+0000 7fe81affd700 1 -- 172.21.15.112:0/3088345236 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814014480 con 0x7fe804004d90 2023-08-27T11:26:04.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.530+0000 7fe822472700 1 -- 172.21.15.112:0/3088345236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe804004d90 msgr2=0x7fe804005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.530+0000 7fe822472700 1 --2- 172.21.15.112:0/3088345236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe804004d90 0x7fe804005210 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe814009fc0 tx=0x7fe804005880 comp rx=0 tx=0).stop 2023-08-27T11:26:04.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.530+0000 7fe820a0f700 1 -- 172.21.15.112:0/3088345236 reap_dead start 2023-08-27T11:26:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.530+0000 7fe822472700 1 -- 172.21.15.112:0/3088345236 shutdown_connections 2023-08-27T11:26:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.530+0000 7fe822472700 1 -- 172.21.15.112:0/3088345236 >> 172.21.15.112:0/3088345236 conn(0x7fe81c06f970 msgr2=0x7fe81c06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.530+0000 7fe822472700 1 -- 172.21.15.112:0/3088345236 shutdown_connections 2023-08-27T11:26:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.530+0000 7fe822472700 1 -- 172.21.15.112:0/3088345236 wait complete. 2023-08-27T11:26:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.531+0000 7fe822472700 1 Processor -- start 2023-08-27T11:26:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.531+0000 7fe822472700 1 -- start start 2023-08-27T11:26:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.532+0000 7fe822472700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe804004d90 0x7fe81c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.532+0000 7fe822472700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe81c0c7f00 0x7fe81c0c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.532+0000 7fe822472700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe81c15a350 0x7fe81c15a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.532+0000 7fe822472700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c15d010 con 0x7fe81c0c7f00 2023-08-27T11:26:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.532+0000 7fe822472700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c15c140 con 0x7fe81c15a350 2023-08-27T11:26:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.532+0000 7fe822472700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c0c88c0 con 0x7fe804004d90 2023-08-27T11:26:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.532+0000 7fe81b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe81c0c7f00 0x7fe81c0c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.532+0000 7fe81b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe81c0c7f00 0x7fe81c0c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60622/0 (socket says 172.21.15.112:60622) 2023-08-27T11:26:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.532+0000 7fe81b7fe700 1 -- 172.21.15.112:0/3406049267 learned_addr learned my addr 172.21.15.112:0/3406049267 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.532+0000 7fe81bfff700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe804004d90 0x7fe81c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.532+0000 7fe820a0f700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe81c15a350 0x7fe81c15a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.533+0000 7fe81b7fe700 1 -- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe804004d90 msgr2=0x7fe81c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.533+0000 7fe81b7fe700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe804004d90 0x7fe81c15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.533+0000 7fe81b7fe700 1 -- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe81c15a350 msgr2=0x7fe81c15a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.533+0000 7fe81b7fe700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe81c15a350 0x7fe81c15a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.533+0000 7fe81b7fe700 1 -- 172.21.15.112:0/3406049267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c15acf0 con 0x7fe81c0c7f00 2023-08-27T11:26:04.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.533+0000 7fe81bfff700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe804004d90 0x7fe81c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:04.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.533+0000 7fe820a0f700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe81c15a350 0x7fe81c15a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:04.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.533+0000 7fe81b7fe700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe81c0c7f00 0x7fe81c0c8380 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0072c0 tx=0x7fe80c007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:04.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.533+0000 7fe818ff9700 1 -- 172.21.15.112:0/3406049267 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c0014c0 con 0x7fe81c0c7f00 2023-08-27T11:26:04.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.533+0000 7fe818ff9700 1 -- 172.21.15.112:0/3406049267 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c00f8d0 con 0x7fe81c0c7f00 2023-08-27T11:26:04.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.533+0000 7fe822472700 1 -- 172.21.15.112:0/3406049267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c15c390 con 0x7fe81c0c7f00 2023-08-27T11:26:04.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.533+0000 7fe818ff9700 1 -- 172.21.15.112:0/3406049267 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c00fc80 con 0x7fe81c0c7f00 2023-08-27T11:26:04.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.533+0000 7fe822472700 1 -- 172.21.15.112:0/3406049267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c15c950 con 0x7fe81c0c7f00 2023-08-27T11:26:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.534+0000 7fe822472700 1 -- 172.21.15.112:0/3406049267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c067a20 con 0x7fe81c0c7f00 2023-08-27T11:26:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.534+0000 7fe818ff9700 1 -- 172.21.15.112:0/3406049267 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fe80c01d460 con 0x7fe81c0c7f00 2023-08-27T11:26:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.535+0000 7fe818ff9700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe80806a3b0 0x7fe80806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.535+0000 7fe818ff9700 1 -- 172.21.15.112:0/3406049267 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fe80c088af0 con 0x7fe81c0c7f00 2023-08-27T11:26:04.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.535+0000 7fe81bfff700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe80806a3b0 0x7fe80806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.535+0000 7fe81bfff700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe80806a3b0 0x7fe80806c870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe810009c60 tx=0x7fe810008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:04.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.539+0000 7fe818ff9700 1 -- 172.21.15.112:0/3406049267 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe80c053e10 con 0x7fe81c0c7f00 2023-08-27T11:26:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.728+0000 7fe822472700 1 -- 172.21.15.112:0/3406049267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe81c1e5860 con 0x7fe81c0c7f00 2023-08-27T11:26:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.729+0000 7fe818ff9700 1 -- 172.21.15.112:0/3406049267 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18453 (secure 0 0 0) 0x7fe80c057430 con 0x7fe81c0c7f00 2023-08-27T11:26:04.731 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:04.731 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":79,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:25:54.909142+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:39.495757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:40.765773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.2d","mappings":[{"from":1,"to":7}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:04.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.731+0000 7fe822472700 1 -- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe80806a3b0 msgr2=0x7fe80806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.731+0000 7fe822472700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe80806a3b0 0x7fe80806c870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe810009c60 tx=0x7fe810008040 comp rx=0 tx=0).stop 2023-08-27T11:26:04.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.731+0000 7fe822472700 1 -- 172.21.15.112:0/3406049267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe81c0c7f00 msgr2=0x7fe81c0c8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.731+0000 7fe822472700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe81c0c7f00 0x7fe81c0c8380 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0072c0 tx=0x7fe80c007680 comp rx=0 tx=0).stop 2023-08-27T11:26:04.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.731+0000 7fe822472700 1 -- 172.21.15.112:0/3406049267 shutdown_connections 2023-08-27T11:26:04.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.731+0000 7fe822472700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe80806a3b0 0x7fe80806c870 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.731+0000 7fe822472700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe81c15a350 0x7fe81c15a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.731+0000 7fe822472700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe804004d90 0x7fe81c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.731+0000 7fe822472700 1 --2- 172.21.15.112:0/3406049267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe81c0c7f00 0x7fe81c0c8380 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.731+0000 7fe822472700 1 -- 172.21.15.112:0/3406049267 >> 172.21.15.112:0/3406049267 conn(0x7fe81c06f970 msgr2=0x7fe81c13ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.731+0000 7fe822472700 1 -- 172.21.15.112:0/3406049267 shutdown_connections 2023-08-27T11:26:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.731+0000 7fe822472700 1 -- 172.21.15.112:0/3406049267 wait complete. 2023-08-27T11:26:04.745 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:26:04.746 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:26:04.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.864+0000 7f7fa16cb700 1 Processor -- start 2023-08-27T11:26:04.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.865+0000 7f7fa16cb700 1 -- start start 2023-08-27T11:26:04.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.865+0000 7f7fa16cb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9c143190 0x7f7f9c0ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.865+0000 7f7fa16cb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7f9c0cf1e0 0x7f7f9c06e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.865+0000 7f7fa16cb700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9c0a9b70 con 0x7f7f9c06e690 2023-08-27T11:26:04.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.865+0000 7f7fa16cb700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9c06eb20 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.865+0000 7f7fa16cb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0cc200 con 0x7f7f9c143190 2023-08-27T11:26:04.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.865+0000 7f7f9a7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7f9c0cf1e0 0x7f7f9c06e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.865+0000 7f7f9b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7f9c06e690 0x7f7f9c0cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55014/0 (socket says 172.21.15.112:55014) 2023-08-27T11:26:04.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.865+0000 7f7f9b7fe700 1 -- 172.21.15.112:0/1377646355 learned_addr learned my addr 172.21.15.112:0/1377646355 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:04.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.866+0000 7f7f9affd700 1 --2- 172.21.15.112:0/1377646355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9c143190 0x7f7f9c0ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.866+0000 7f7f9a7fc700 1 -- 172.21.15.112:0/1377646355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9c143190 msgr2=0x7f7f9c0ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:04.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.866+0000 7f7f9a7fc700 1 --2- 172.21.15.112:0/1377646355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9c143190 0x7f7f9c0ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.866+0000 7f7f9a7fc700 1 -- 172.21.15.112:0/1377646355 >> v1:172.21.15.112:6789/0 conn(0x7f7f9c06e690 legacy=0x7f7f9c0cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.866+0000 7f7f99ffb700 1 -- 172.21.15.112:0/1377646355 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890786008 0 0) 0x7f7f9c0a9b70 con 0x7f7f9c06e690 2023-08-27T11:26:04.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.866+0000 7f7f9a7fc700 1 -- 172.21.15.112:0/1377646355 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c0cc4e0 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.866+0000 7f7f9a7fc700 1 --2- 172.21.15.112:0/1377646355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f9c0cf1e0 0x7f7f9c06e150 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009820 tx=0x7f7f8c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f9ba4f9288d952 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.866+0000 7f7f99ffb700 1 -- 172.21.15.112:0/1377646355 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c005e30 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.866+0000 7f7f99ffb700 1 -- 172.21.15.112:0/1377646355 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c006030 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.866+0000 7f7f99ffb700 1 -- 172.21.15.112:0/1377646355 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c0063e0 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.866+0000 7f7fa16cb700 1 -- 172.21.15.112:0/1377646355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f9c0cf1e0 msgr2=0x7f7f9c06e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.866+0000 7f7fa16cb700 1 --2- 172.21.15.112:0/1377646355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f9c0cf1e0 0x7f7f9c06e150 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009820 tx=0x7f7f8c009b30 comp rx=0 tx=0).stop 2023-08-27T11:26:04.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.867+0000 7f7fa16cb700 1 -- 172.21.15.112:0/1377646355 shutdown_connections 2023-08-27T11:26:04.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.867+0000 7f7fa16cb700 1 --2- 172.21.15.112:0/1377646355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9c143190 0x7f7f9c0ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.867+0000 7f7fa16cb700 1 --2- 172.21.15.112:0/1377646355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f9c0cf1e0 0x7f7f9c06e150 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.867+0000 7f7fa16cb700 1 -- 172.21.15.112:0/1377646355 >> 172.21.15.112:0/1377646355 conn(0x7f7f9c06cd30 msgr2=0x7f7f9c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:04.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.867+0000 7f7fa16cb700 1 -- 172.21.15.112:0/1377646355 shutdown_connections 2023-08-27T11:26:04.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.867+0000 7f7fa16cb700 1 -- 172.21.15.112:0/1377646355 wait complete. 2023-08-27T11:26:04.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.868+0000 7f7fa16cb700 1 Processor -- start 2023-08-27T11:26:04.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.868+0000 7f7fa16cb700 1 -- start start 2023-08-27T11:26:04.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7fa16cb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9c06e690 0x7f7f9c155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7fa16cb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f9c0cf1e0 0x7f7f9c153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7fa16cb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f9c143190 0x7f7f9c0710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7fa16cb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c070cc0 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7fa16cb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c06fdf0 con 0x7f7f9c143190 2023-08-27T11:26:04.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7fa16cb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c070040 con 0x7f7f9c06e690 2023-08-27T11:26:04.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f9a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f9c0cf1e0 0x7f7f9c153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f9a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f9c0cf1e0 0x7f7f9c153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60636/0 (socket says 172.21.15.112:60636) 2023-08-27T11:26:04.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f9a7fc700 1 -- 172.21.15.112:0/2631855547 learned_addr learned my addr 172.21.15.112:0/2631855547 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:04.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f9b7fe700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f9c143190 0x7f7f9c0710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f9affd700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9c06e690 0x7f7f9c155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f9a7fc700 1 -- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9c06e690 msgr2=0x7f7f9c155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f9a7fc700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9c06e690 0x7f7f9c155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f9a7fc700 1 -- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f9c143190 msgr2=0x7f7f9c0710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f9a7fc700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f9c143190 0x7f7f9c0710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f9a7fc700 1 -- 172.21.15.112:0/2631855547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c009430 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f9a7fc700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f9c0cf1e0 0x7f7f9c153630 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c06eeb0 tx=0x7f7f8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f837fe700 1 -- 172.21.15.112:0/2631855547 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c01f070 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f837fe700 1 -- 172.21.15.112:0/2631855547 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c019720 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7fa16cb700 1 -- 172.21.15.112:0/2631855547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9c070320 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.869+0000 7f7f837fe700 1 -- 172.21.15.112:0/2631855547 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c019ad0 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.870+0000 7f7fa16cb700 1 -- 172.21.15.112:0/2631855547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9c0708e0 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.870+0000 7f7fa16cb700 1 -- 172.21.15.112:0/2631855547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9c070e00 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.871+0000 7f7f837fe700 1 -- 172.21.15.112:0/2631855547 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f7f8c019c30 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.871+0000 7f7f837fe700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f8806a3b0 0x7f7f8806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:04.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.871+0000 7f7f837fe700 1 -- 172.21.15.112:0/2631855547 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f7f8c089b90 con 0x7f7f9c0cf1e0 2023-08-27T11:26:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.871+0000 7f7f9affd700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f8806a3b0 0x7f7f8806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.872+0000 7f7f9affd700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f8806a3b0 0x7f7f8806c870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000b9b0 tx=0x7f7f90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:04.875+0000 7f7f837fe700 1 -- 172.21.15.112:0/2631855547 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7f9c070e00 con 0x7f7f9c0cf1e0 2023-08-27T11:26:05.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.106+0000 7f7fa16cb700 1 -- 172.21.15.112:0/2631855547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f9c070e00 con 0x7f7f9c0cf1e0 2023-08-27T11:26:05.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.106+0000 7f7f837fe700 1 -- 172.21.15.112:0/2631855547 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4889 (secure 0 0 0) 0x7f7f9c070e00 con 0x7f7f9c0cf1e0 2023-08-27T11:26:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:05.108 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6166,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3262556844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3262556844},{"type":"v1","addr":"172.21.15.138:6836","nonce":3262556844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2232724696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2232724696},{"type":"v1","addr":"172.21.15.138:6837","nonce":2232724696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6570,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1511159838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1511159838},{"type":"v1","addr":"172.21.15.112:6834","nonce":1511159838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:25:41.794717+0000","modified":"2023-08-27T11:25:45.437236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2318724344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2318724344},{"type":"v1","addr":"172.21.15.112:6835","nonce":2318724344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-27T11:26:05.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.108+0000 7f7fa16cb700 1 -- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f8806a3b0 msgr2=0x7f7f8806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.108+0000 7f7fa16cb700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f8806a3b0 0x7f7f8806c870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000b9b0 tx=0x7f7f90008040 comp rx=0 tx=0).stop 2023-08-27T11:26:05.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.108+0000 7f7fa16cb700 1 -- 172.21.15.112:0/2631855547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f9c0cf1e0 msgr2=0x7f7f9c153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.108+0000 7f7fa16cb700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f9c0cf1e0 0x7f7f9c153630 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c06eeb0 tx=0x7f7f8c004750 comp rx=0 tx=0).stop 2023-08-27T11:26:05.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.108+0000 7f7fa16cb700 1 -- 172.21.15.112:0/2631855547 shutdown_connections 2023-08-27T11:26:05.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.108+0000 7f7fa16cb700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f8806a3b0 0x7f7f8806c870 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.108+0000 7f7fa16cb700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f9c143190 0x7f7f9c0710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.108+0000 7f7fa16cb700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9c06e690 0x7f7f9c155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.108+0000 7f7fa16cb700 1 --2- 172.21.15.112:0/2631855547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f9c0cf1e0 0x7f7f9c153630 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.108+0000 7f7fa16cb700 1 -- 172.21.15.112:0/2631855547 >> 172.21.15.112:0/2631855547 conn(0x7f7f9c06cd30 msgr2=0x7f7f9c140d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.108+0000 7f7fa16cb700 1 -- 172.21.15.112:0/2631855547 shutdown_connections 2023-08-27T11:26:05.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.108+0000 7f7fa16cb700 1 -- 172.21.15.112:0/2631855547 wait complete. 2023-08-27T11:26:05.116 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 49 2023-08-27T11:26:05.122 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:26:05.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.242+0000 7f9435430700 1 Processor -- start 2023-08-27T11:26:05.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.242+0000 7f9435430700 1 -- start start 2023-08-27T11:26:05.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.242+0000 7f9435430700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9430071ac0 0x7f94300b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.242+0000 7f9435430700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94300b3440 con 0x7f94300716e0 2023-08-27T11:26:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f9435430700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94300b3720 con 0x7f94300b4a20 2023-08-27T11:26:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f9435430700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94300b3a00 con 0x7f9430071ac0 2023-08-27T11:26:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942effd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f94300716e0 0x7f94300d0cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55024/0 (socket says 172.21.15.112:55024) 2023-08-27T11:26:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942effd700 1 -- 172.21.15.112:0/1434191217 learned_addr learned my addr 172.21.15.112:0/1434191217 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967145500 0 0) 0x7f94300b3440 con 0x7f94300716e0 2023-08-27T11:26:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942e7fc700 1 --2- 172.21.15.112:0/1434191217 >> v2:172.21.15.138:3300/0 conn(0x7f9430071ac0 0x7f94300b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94180035b0 con 0x7f94300716e0 2023-08-27T11:26:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2111884915 0 0) 0x7f94180035b0 con 0x7f94300716e0 2023-08-27T11:26:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 >> v2:172.21.15.138:3300/0 conn(0x7f9430071ac0 msgr2=0x7f94300b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 --2- 172.21.15.112:0/1434191217 >> v2:172.21.15.138:3300/0 conn(0x7f9430071ac0 0x7f94300b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 >> v1:172.21.15.138:6789/0 conn(0x7f94300b4a20 legacy=0x7f94300701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94300b3ce0 con 0x7f94300716e0 2023-08-27T11:26:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f9420001670 con 0x7f94300716e0 2023-08-27T11:26:05.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 >> v1:172.21.15.112:6789/0 conn(0x7f94300716e0 legacy=0x7f94300d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 --2- 172.21.15.112:0/1434191217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9418003a00 0x7f9418003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 --2- 172.21.15.112:0/1434191217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418004850 0x7f9418004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 --2- 172.21.15.112:0/1434191217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9418005210 0x7f941800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94300b3a00 con 0x7f9418003a00 2023-08-27T11:26:05.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941800c0b0 con 0x7f9418004850 2023-08-27T11:26:05.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941800c270 con 0x7f9418005210 2023-08-27T11:26:05.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942effd700 1 --2- 172.21.15.112:0/1434191217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9418003a00 0x7f9418003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942e7fc700 1 --2- 172.21.15.112:0/1434191217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418004850 0x7f9418004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.243+0000 7f942f7fe700 1 --2- 172.21.15.112:0/1434191217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9418005210 0x7f941800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.244+0000 7f942effd700 1 -- 172.21.15.112:0/1434191217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9418005210 msgr2=0x7f941800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.244+0000 7f942effd700 1 --2- 172.21.15.112:0/1434191217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9418005210 0x7f941800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.244+0000 7f942effd700 1 -- 172.21.15.112:0/1434191217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418004850 msgr2=0x7f9418004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.244+0000 7f942effd700 1 --2- 172.21.15.112:0/1434191217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418004850 0x7f9418004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.244+0000 7f942effd700 1 -- 172.21.15.112:0/1434191217 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180035b0 con 0x7f9418003a00 2023-08-27T11:26:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.244+0000 7f942e7fc700 1 --2- 172.21.15.112:0/1434191217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418004850 0x7f9418004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.244+0000 7f942f7fe700 1 -- 172.21.15.112:0/1434191217 reap_dead start 2023-08-27T11:26:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.244+0000 7f942effd700 1 --2- 172.21.15.112:0/1434191217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9418003a00 0x7f9418003e20 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f942000c240 tx=0x7f9418005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32caa817629954b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.244+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420007c10 con 0x7f9418003a00 2023-08-27T11:26:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.244+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942000ce00 con 0x7f9418003a00 2023-08-27T11:26:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.244+0000 7f942dffb700 1 -- 172.21.15.112:0/1434191217 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420005ba0 con 0x7f9418003a00 2023-08-27T11:26:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.244+0000 7f9435430700 1 -- 172.21.15.112:0/1434191217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9418003a00 msgr2=0x7f9418003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.244+0000 7f9435430700 1 --2- 172.21.15.112:0/1434191217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9418003a00 0x7f9418003e20 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f942000c240 tx=0x7f9418005880 comp rx=0 tx=0).stop 2023-08-27T11:26:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.245+0000 7f9435430700 1 -- 172.21.15.112:0/1434191217 shutdown_connections 2023-08-27T11:26:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.245+0000 7f9435430700 1 --2- 172.21.15.112:0/1434191217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9418003a00 0x7f9418003e20 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.245+0000 7f9435430700 1 -- 172.21.15.112:0/1434191217 >> 172.21.15.112:0/1434191217 conn(0x7f943006c920 msgr2=0x7f943006cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.245+0000 7f9435430700 1 -- 172.21.15.112:0/1434191217 shutdown_connections 2023-08-27T11:26:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.245+0000 7f9435430700 1 -- 172.21.15.112:0/1434191217 wait complete. 2023-08-27T11:26:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.246+0000 7f9435430700 1 Processor -- start 2023-08-27T11:26:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.246+0000 7f9435430700 1 -- start start 2023-08-27T11:26:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.247+0000 7f9435430700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9418003a00 0x7f9430147360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.247+0000 7f9435430700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9430155f40 0x7f943015e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.247+0000 7f9435430700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94300d1cc0 0x7f94300d2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.247+0000 7f9435430700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94300d26b0 con 0x7f9418003a00 2023-08-27T11:26:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.247+0000 7f9435430700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94301f2970 con 0x7f9430155f40 2023-08-27T11:26:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.247+0000 7f9435430700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9430156700 con 0x7f94300d1cc0 2023-08-27T11:26:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.247+0000 7f942effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9418003a00 0x7f9430147360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.247+0000 7f942effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9418003a00 0x7f9430147360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60654/0 (socket says 172.21.15.112:60654) 2023-08-27T11:26:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.247+0000 7f942effd700 1 -- 172.21.15.112:0/2741245672 learned_addr learned my addr 172.21.15.112:0/2741245672 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:05.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.247+0000 7f942effd700 1 -- 172.21.15.112:0/2741245672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94300d1cc0 msgr2=0x7f94300d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.247+0000 7f942effd700 1 --2- 172.21.15.112:0/2741245672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94300d1cc0 0x7f94300d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.248+0000 7f942effd700 1 -- 172.21.15.112:0/2741245672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9430155f40 msgr2=0x7f943015e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.248+0000 7f942effd700 1 --2- 172.21.15.112:0/2741245672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9430155f40 0x7f943015e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.248+0000 7f942effd700 1 -- 172.21.15.112:0/2741245672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94301569e0 con 0x7f9418003a00 2023-08-27T11:26:05.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.248+0000 7f942effd700 1 --2- 172.21.15.112:0/2741245672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9418003a00 0x7f9430147360 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f9420000c00 tx=0x7f9420001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:05.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.248+0000 7f94177fe700 1 -- 172.21.15.112:0/2741245672 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420004d40 con 0x7f9418003a00 2023-08-27T11:26:05.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.248+0000 7f94177fe700 1 -- 172.21.15.112:0/2741245672 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94200070d0 con 0x7f9418003a00 2023-08-27T11:26:05.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.248+0000 7f9435430700 1 -- 172.21.15.112:0/2741245672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9430156cc0 con 0x7f9418003a00 2023-08-27T11:26:05.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.248+0000 7f94177fe700 1 -- 172.21.15.112:0/2741245672 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420007480 con 0x7f9418003a00 2023-08-27T11:26:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.248+0000 7f9435430700 1 -- 172.21.15.112:0/2741245672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94301563c0 con 0x7f9418003a00 2023-08-27T11:26:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.249+0000 7f9435430700 1 -- 172.21.15.112:0/2741245672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9430067a20 con 0x7f9418003a00 2023-08-27T11:26:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.249+0000 7f94177fe700 1 -- 172.21.15.112:0/2741245672 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f94200075e0 con 0x7f9418003a00 2023-08-27T11:26:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.250+0000 7f94177fe700 1 --2- 172.21.15.112:0/2741245672 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f941c06a3b0 0x7f941c06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.250+0000 7f94177fe700 1 -- 172.21.15.112:0/2741245672 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f9420089320 con 0x7f9418003a00 2023-08-27T11:26:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.250+0000 7f942e7fc700 1 --2- 172.21.15.112:0/2741245672 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f941c06a3b0 0x7f941c06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.250+0000 7f942e7fc700 1 --2- 172.21.15.112:0/2741245672 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f941c06a3b0 0x7f941c06c870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9430070330 tx=0x7f9424008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:05.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.254+0000 7f94177fe700 1 -- 172.21.15.112:0/2741245672 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f94200546d0 con 0x7f9418003a00 2023-08-27T11:26:05.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.489+0000 7f9435430700 1 -- 172.21.15.112:0/2741245672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9430072360 con 0x7f9418003a00 2023-08-27T11:26:05.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.490+0000 7f94177fe700 1 -- 172.21.15.112:0/2741245672 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4889 (secure 0 0 0) 0x7f9420057cf0 con 0x7f9418003a00 2023-08-27T11:26:05.492 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:05.492 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6166,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3262556844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3262556844},{"type":"v1","addr":"172.21.15.138:6836","nonce":3262556844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2232724696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2232724696},{"type":"v1","addr":"172.21.15.138:6837","nonce":2232724696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6570,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1511159838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1511159838},{"type":"v1","addr":"172.21.15.112:6834","nonce":1511159838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:25:41.794717+0000","modified":"2023-08-27T11:25:45.437236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2318724344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2318724344},{"type":"v1","addr":"172.21.15.112:6835","nonce":2318724344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-27T11:26:05.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.492+0000 7f9435430700 1 -- 172.21.15.112:0/2741245672 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f941c06a3b0 msgr2=0x7f941c06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.492+0000 7f9435430700 1 --2- 172.21.15.112:0/2741245672 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f941c06a3b0 0x7f941c06c870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9430070330 tx=0x7f9424008040 comp rx=0 tx=0).stop 2023-08-27T11:26:05.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.492+0000 7f9435430700 1 -- 172.21.15.112:0/2741245672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9418003a00 msgr2=0x7f9430147360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.492+0000 7f9435430700 1 --2- 172.21.15.112:0/2741245672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9418003a00 0x7f9430147360 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f9420000c00 tx=0x7f9420001a00 comp rx=0 tx=0).stop 2023-08-27T11:26:05.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.492+0000 7f9435430700 1 -- 172.21.15.112:0/2741245672 shutdown_connections 2023-08-27T11:26:05.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.492+0000 7f9435430700 1 --2- 172.21.15.112:0/2741245672 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f941c06a3b0 0x7f941c06c870 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.492+0000 7f9435430700 1 --2- 172.21.15.112:0/2741245672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94300d1cc0 0x7f94300d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.492+0000 7f9435430700 1 --2- 172.21.15.112:0/2741245672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9418003a00 0x7f9430147360 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.492+0000 7f9435430700 1 --2- 172.21.15.112:0/2741245672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9430155f40 0x7f943015e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.492+0000 7f9435430700 1 -- 172.21.15.112:0/2741245672 >> 172.21.15.112:0/2741245672 conn(0x7f943006c920 msgr2=0x7f94300b77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.492+0000 7f9435430700 1 -- 172.21.15.112:0/2741245672 shutdown_connections 2023-08-27T11:26:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.492+0000 7f9435430700 1 -- 172.21.15.112:0/2741245672 wait complete. 2023-08-27T11:26:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 49 2023-08-27T11:26:05.505 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:05.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.624+0000 7f03a729b700 1 Processor -- start 2023-08-27T11:26:05.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.625+0000 7f03a729b700 1 -- start start 2023-08-27T11:26:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.625+0000 7f03a729b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f03a006adb0 0x7f03a006b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.625+0000 7f03a729b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f03a00cdbd0 0x7f03a00ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.625+0000 7f03a729b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a015eb60 con 0x7f03a0151060 2023-08-27T11:26:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.625+0000 7f03a729b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a006b770 con 0x7f03a006adb0 2023-08-27T11:26:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.625+0000 7f03a729b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a00b02f0 con 0x7f03a00cdbd0 2023-08-27T11:26:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.625+0000 7f03a4836700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f03a006adb0 0x7f03a006b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.626+0000 7f03a4836700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f03a006adb0 0x7f03a006b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60664/0 (socket says 172.21.15.112:60664) 2023-08-27T11:26:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.626+0000 7f03a4836700 1 -- 172.21.15.112:0/2597172780 learned_addr learned my addr 172.21.15.112:0/2597172780 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.626+0000 7f03a5838700 1 --2- 172.21.15.112:0/2597172780 >> v2:172.21.15.138:3300/0 conn(0x7f03a00cdbd0 0x7f03a00ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.626+0000 7f03a4836700 1 -- 172.21.15.112:0/2597172780 >> v2:172.21.15.138:3300/0 conn(0x7f03a00cdbd0 msgr2=0x7f03a00ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:05.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.626+0000 7f03a4836700 1 --2- 172.21.15.112:0/2597172780 >> v2:172.21.15.138:3300/0 conn(0x7f03a00cdbd0 0x7f03a00ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.626+0000 7f03a4836700 1 -- 172.21.15.112:0/2597172780 >> v1:172.21.15.138:6789/0 conn(0x7f03a0151060 legacy=0x7f03a015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:05.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.626+0000 7f03a4836700 1 -- 172.21.15.112:0/2597172780 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a00b05d0 con 0x7f03a006adb0 2023-08-27T11:26:05.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.626+0000 7f03a4836700 1 --2- 172.21.15.112:0/2597172780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a006adb0 0x7f03a006b230 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f03a00b1960 tx=0x7f039c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539c045501521bee server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:05.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.626+0000 7f0397fff700 1 -- 172.21.15.112:0/2597172780 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c00dd30 con 0x7f03a006adb0 2023-08-27T11:26:05.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.626+0000 7f0397fff700 1 -- 172.21.15.112:0/2597172780 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c00de90 con 0x7f03a006adb0 2023-08-27T11:26:05.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.626+0000 7f0397fff700 1 -- 172.21.15.112:0/2597172780 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c010690 con 0x7f03a006adb0 2023-08-27T11:26:05.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.627+0000 7f03a729b700 1 -- 172.21.15.112:0/2597172780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a006adb0 msgr2=0x7f03a006b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.627+0000 7f03a729b700 1 --2- 172.21.15.112:0/2597172780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a006adb0 0x7f03a006b230 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f03a00b1960 tx=0x7f039c00be70 comp rx=0 tx=0).stop 2023-08-27T11:26:05.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.627+0000 7f03a729b700 1 -- 172.21.15.112:0/2597172780 shutdown_connections 2023-08-27T11:26:05.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.627+0000 7f03a729b700 1 --2- 172.21.15.112:0/2597172780 >> v2:172.21.15.138:3300/0 conn(0x7f03a00cdbd0 0x7f03a00ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.627+0000 7f03a729b700 1 --2- 172.21.15.112:0/2597172780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a006adb0 0x7f03a006b230 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.627+0000 7f03a729b700 1 -- 172.21.15.112:0/2597172780 >> 172.21.15.112:0/2597172780 conn(0x7f03a00d0000 msgr2=0x7f03a00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:05.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.627+0000 7f03a729b700 1 -- 172.21.15.112:0/2597172780 shutdown_connections 2023-08-27T11:26:05.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.627+0000 7f03a729b700 1 -- 172.21.15.112:0/2597172780 wait complete. 2023-08-27T11:26:05.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.628+0000 7f03a729b700 1 Processor -- start 2023-08-27T11:26:05.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.628+0000 7f03a729b700 1 -- start start 2023-08-27T11:26:05.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a729b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a006adb0 0x7f03a006f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a729b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03a00cdbd0 0x7f03a00c5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a729b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a0151060 0x7f03a00c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a729b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a00c59b0 con 0x7f03a006adb0 2023-08-27T11:26:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a729b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a00707d0 con 0x7f03a0151060 2023-08-27T11:26:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a729b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a006fc50 con 0x7f03a00cdbd0 2023-08-27T11:26:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a5037700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a006adb0 0x7f03a006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a5037700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a006adb0 0x7f03a006f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60668/0 (socket says 172.21.15.112:60668) 2023-08-27T11:26:05.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a5037700 1 -- 172.21.15.112:0/1919829590 learned_addr learned my addr 172.21.15.112:0/1919829590 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:05.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a5838700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a0151060 0x7f03a00c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a4836700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03a00cdbd0 0x7f03a00c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a5037700 1 -- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03a00cdbd0 msgr2=0x7f03a00c5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a5037700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03a00cdbd0 0x7f03a00c5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a5037700 1 -- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a0151060 msgr2=0x7f03a00c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a5037700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a0151060 0x7f03a00c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a5037700 1 -- 172.21.15.112:0/1919829590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c00b7f0 con 0x7f03a006adb0 2023-08-27T11:26:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a5037700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a006adb0 0x7f03a006f600 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f039000c4e0 tx=0x7f039000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f0395ffb700 1 -- 172.21.15.112:0/1919829590 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03900052d0 con 0x7f03a006adb0 2023-08-27T11:26:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f0395ffb700 1 -- 172.21.15.112:0/1919829590 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0390005430 con 0x7f03a006adb0 2023-08-27T11:26:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f03a729b700 1 -- 172.21.15.112:0/1919829590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a006ff30 con 0x7f03a006adb0 2023-08-27T11:26:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.629+0000 7f0395ffb700 1 -- 172.21.15.112:0/1919829590 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03900038c0 con 0x7f03a006adb0 2023-08-27T11:26:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.630+0000 7f03a729b700 1 -- 172.21.15.112:0/1919829590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a00704f0 con 0x7f03a006adb0 2023-08-27T11:26:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.630+0000 7f03a729b700 1 -- 172.21.15.112:0/1919829590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a014ee20 con 0x7f03a006adb0 2023-08-27T11:26:05.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.631+0000 7f0395ffb700 1 -- 172.21.15.112:0/1919829590 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f0390003a20 con 0x7f03a006adb0 2023-08-27T11:26:05.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.631+0000 7f0395ffb700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f038406a3b0 0x7f038406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.631+0000 7f0395ffb700 1 -- 172.21.15.112:0/1919829590 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f0390087e70 con 0x7f03a006adb0 2023-08-27T11:26:05.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.632+0000 7f03a4836700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f038406a3b0 0x7f038406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.632+0000 7f03a4836700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f038406a3b0 0x7f038406c870 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f039c00b6f0 tx=0x7f039c012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:05.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.636+0000 7f0395ffb700 1 -- 172.21.15.112:0/1919829590 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f03a014ee20 con 0x7f03a006adb0 2023-08-27T11:26:05.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.825+0000 7f03a729b700 1 -- 172.21.15.112:0/1919829590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03a014ee20 con 0x7f03a006adb0 2023-08-27T11:26:05.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.826+0000 7f0395ffb700 1 -- 172.21.15.112:0/1919829590 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18453 (secure 0 0 0) 0x7f03a014ee20 con 0x7f03a006adb0 2023-08-27T11:26:05.828 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:05.829 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":79,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:25:54.909142+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:39.495757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:40.765773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.2d","mappings":[{"from":1,"to":7}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:05.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.828+0000 7f03a729b700 1 -- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f038406a3b0 msgr2=0x7f038406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.828+0000 7f03a729b700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f038406a3b0 0x7f038406c870 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f039c00b6f0 tx=0x7f039c012000 comp rx=0 tx=0).stop 2023-08-27T11:26:05.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.828+0000 7f03a729b700 1 -- 172.21.15.112:0/1919829590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a006adb0 msgr2=0x7f03a006f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.828+0000 7f03a729b700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a006adb0 0x7f03a006f600 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f039000c4e0 tx=0x7f039000c7f0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.829+0000 7f03a729b700 1 -- 172.21.15.112:0/1919829590 shutdown_connections 2023-08-27T11:26:05.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.829+0000 7f03a729b700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f038406a3b0 0x7f038406c870 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.829+0000 7f03a729b700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a0151060 0x7f03a00c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.829+0000 7f03a729b700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a006adb0 0x7f03a006f600 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.829+0000 7f03a729b700 1 --2- 172.21.15.112:0/1919829590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03a00cdbd0 0x7f03a00c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.829+0000 7f03a729b700 1 -- 172.21.15.112:0/1919829590 >> 172.21.15.112:0/1919829590 conn(0x7f03a00d0000 msgr2=0x7f03a00b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:05.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.829+0000 7f03a729b700 1 -- 172.21.15.112:0/1919829590 shutdown_connections 2023-08-27T11:26:05.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.829+0000 7f03a729b700 1 -- 172.21.15.112:0/1919829590 wait complete. 2023-08-27T11:26:05.842 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:26:05.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.962+0000 7f10da79f700 1 Processor -- start 2023-08-27T11:26:05.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.962+0000 7f10da79f700 1 -- start start 2023-08-27T11:26:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10da79f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f10d4144570 0x7f10d415ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10da79f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10d40b4ad0 0x7f10d40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10da79f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d40b45f0 con 0x7f10d40b3e50 2023-08-27T11:26:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10da79f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d40cf7a0 con 0x7f10d4144570 2023-08-27T11:26:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10da79f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d4144950 con 0x7f10d40b4ad0 2023-08-27T11:26:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10d3fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f10d4144570 0x7f10d415ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10d3fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f10d4144570 0x7f10d415ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60676/0 (socket says 172.21.15.112:60676) 2023-08-27T11:26:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10d3fff700 1 -- 172.21.15.112:0/3685474971 learned_addr learned my addr 172.21.15.112:0/3685474971 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10d8d3c700 1 --2- 172.21.15.112:0/3685474971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10d40b4ad0 0x7f10d40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10d3fff700 1 -- 172.21.15.112:0/3685474971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10d40b4ad0 msgr2=0x7f10d40b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:05.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10d3fff700 1 --2- 172.21.15.112:0/3685474971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10d40b4ad0 0x7f10d40b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10d3fff700 1 -- 172.21.15.112:0/3685474971 >> v1:172.21.15.138:6789/0 conn(0x7f10d40b3e50 legacy=0x7f10d415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10d3fff700 1 -- 172.21.15.112:0/3685474971 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d40b4310 con 0x7f10d4144570 2023-08-27T11:26:05.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10d3fff700 1 --2- 172.21.15.112:0/3685474971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d4144570 0x7f10d415ee20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009920 tx=0x7f10c4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a5a272e60492ae server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10d2ffd700 1 -- 172.21.15.112:0/3685474971 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4011830 con 0x7f10d4144570 2023-08-27T11:26:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10d2ffd700 1 -- 172.21.15.112:0/3685474971 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c4011990 con 0x7f10d4144570 2023-08-27T11:26:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.963+0000 7f10d2ffd700 1 -- 172.21.15.112:0/3685474971 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4011d20 con 0x7f10d4144570 2023-08-27T11:26:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.964+0000 7f10da79f700 1 -- 172.21.15.112:0/3685474971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d4144570 msgr2=0x7f10d415ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.964+0000 7f10da79f700 1 --2- 172.21.15.112:0/3685474971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d4144570 0x7f10d415ee20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009920 tx=0x7f10c4009c30 comp rx=0 tx=0).stop 2023-08-27T11:26:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.964+0000 7f10da79f700 1 -- 172.21.15.112:0/3685474971 shutdown_connections 2023-08-27T11:26:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.964+0000 7f10da79f700 1 --2- 172.21.15.112:0/3685474971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10d40b4ad0 0x7f10d40b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.964+0000 7f10da79f700 1 --2- 172.21.15.112:0/3685474971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d4144570 0x7f10d415ee20 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.964+0000 7f10da79f700 1 -- 172.21.15.112:0/3685474971 >> 172.21.15.112:0/3685474971 conn(0x7f10d406f970 msgr2=0x7f10d406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.964+0000 7f10da79f700 1 -- 172.21.15.112:0/3685474971 shutdown_connections 2023-08-27T11:26:05.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.964+0000 7f10da79f700 1 -- 172.21.15.112:0/3685474971 wait complete. 2023-08-27T11:26:05.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.965+0000 7f10da79f700 1 Processor -- start 2023-08-27T11:26:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.965+0000 7f10da79f700 1 -- start start 2023-08-27T11:26:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10da79f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10d40b3e50 0x7f10d415e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10da79f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10d40b4ad0 0x7f10d413ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10da79f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d4144570 0x7f10d40ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10da79f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d413cfa0 con 0x7f10d4144570 2023-08-27T11:26:05.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10da79f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d415db70 con 0x7f10d40b3e50 2023-08-27T11:26:05.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10da79f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d415cf60 con 0x7f10d40b4ad0 2023-08-27T11:26:05.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10d8d3c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d4144570 0x7f10d40ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10d8d3c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d4144570 0x7f10d40ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60686/0 (socket says 172.21.15.112:60686) 2023-08-27T11:26:05.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10d8d3c700 1 -- 172.21.15.112:0/3992909042 learned_addr learned my addr 172.21.15.112:0/3992909042 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10d3fff700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10d40b3e50 0x7f10d415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10d37fe700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10d40b4ad0 0x7f10d413ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10d8d3c700 1 -- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10d40b4ad0 msgr2=0x7f10d413ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10d8d3c700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10d40b4ad0 0x7f10d413ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10d8d3c700 1 -- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10d40b3e50 msgr2=0x7f10d415e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:05.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10d8d3c700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10d40b3e50 0x7f10d415e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:05.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10d8d3c700 1 -- 172.21.15.112:0/3992909042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c4009550 con 0x7f10d4144570 2023-08-27T11:26:05.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10d37fe700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10d40b4ad0 0x7f10d413ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:05.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.966+0000 7f10d3fff700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10d40b3e50 0x7f10d415e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:05.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.967+0000 7f10d8d3c700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d4144570 0x7f10d40ca580 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00efc0 tx=0x7f10cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.967+0000 7f10d0ff9700 1 -- 172.21.15.112:0/3992909042 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc00cd40 con 0x7f10d4144570 2023-08-27T11:26:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.967+0000 7f10d0ff9700 1 -- 172.21.15.112:0/3992909042 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10cc00f040 con 0x7f10d4144570 2023-08-27T11:26:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.967+0000 7f10da79f700 1 -- 172.21.15.112:0/3992909042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d415d240 con 0x7f10d4144570 2023-08-27T11:26:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.967+0000 7f10d0ff9700 1 -- 172.21.15.112:0/3992909042 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc0146c0 con 0x7f10d4144570 2023-08-27T11:26:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.967+0000 7f10da79f700 1 -- 172.21.15.112:0/3992909042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d415d800 con 0x7f10d4144570 2023-08-27T11:26:05.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.968+0000 7f10da79f700 1 -- 172.21.15.112:0/3992909042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d4067a20 con 0x7f10d4144570 2023-08-27T11:26:05.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.968+0000 7f10d0ff9700 1 -- 172.21.15.112:0/3992909042 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f10cc014820 con 0x7f10d4144570 2023-08-27T11:26:05.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.968+0000 7f10d0ff9700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10c006a3b0 0x7f10c006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:05.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.969+0000 7f10d0ff9700 1 -- 172.21.15.112:0/3992909042 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f10cc089670 con 0x7f10d4144570 2023-08-27T11:26:05.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.969+0000 7f10d3fff700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10c006a3b0 0x7f10c006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.969+0000 7f10d3fff700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10c006a3b0 0x7f10c006c870 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f10c400a010 tx=0x7f10c4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:05.973+0000 7f10d0ff9700 1 -- 172.21.15.112:0/3992909042 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f10cc058230 con 0x7f10d4144570 2023-08-27T11:26:06.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.203+0000 7f10da79f700 1 -- 172.21.15.112:0/3992909042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f10d415dcb0 con 0x7f10d4144570 2023-08-27T11:26:06.205 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:26:06.204+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:26:06.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.208+0000 7f10d0ff9700 1 -- 172.21.15.112:0/3992909042 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7f10cc00cea0 con 0x7f10d4144570 2023-08-27T11:26:06.210 INFO:tasks.ceph.mds.c.smithi112.stderr: -5864> 2023-08-27T11:25:35.454+0000 7face5431700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-08-27T11:26:06.211 INFO:tasks.ceph.mds.c.smithi112.stderr: -4693> 2023-08-27T11:25:46.378+0000 7facdf425700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:26:06.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.210+0000 7f10da79f700 1 -- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10c006a3b0 msgr2=0x7f10c006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:06.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.210+0000 7f10da79f700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10c006a3b0 0x7f10c006c870 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f10c400a010 tx=0x7f10c4008030 comp rx=0 tx=0).stop 2023-08-27T11:26:06.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.210+0000 7f10da79f700 1 -- 172.21.15.112:0/3992909042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d4144570 msgr2=0x7f10d40ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:06.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.210+0000 7f10da79f700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d4144570 0x7f10d40ca580 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00efc0 tx=0x7f10cc00c3a0 comp rx=0 tx=0).stop 2023-08-27T11:26:06.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.210+0000 7f10da79f700 1 -- 172.21.15.112:0/3992909042 shutdown_connections 2023-08-27T11:26:06.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.210+0000 7f10da79f700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10c006a3b0 0x7f10c006c870 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:06.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.210+0000 7f10da79f700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d4144570 0x7f10d40ca580 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:06.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.210+0000 7f10da79f700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10d40b3e50 0x7f10d415e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:06.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.210+0000 7f10da79f700 1 --2- 172.21.15.112:0/3992909042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10d40b4ad0 0x7f10d413ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:06.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.211+0000 7f10da79f700 1 -- 172.21.15.112:0/3992909042 >> 172.21.15.112:0/3992909042 conn(0x7f10d406f970 msgr2=0x7f10d41433e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:06.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.211+0000 7f10da79f700 1 -- 172.21.15.112:0/3992909042 shutdown_connections 2023-08-27T11:26:06.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.211+0000 7f10da79f700 1 -- 172.21.15.112:0/3992909042 wait complete. 2023-08-27T11:26:06.215 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:26:06.224 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:26:06.242 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.347+0000 7f51e19df700 1 Processor -- start 2023-08-27T11:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.347+0000 7f51e19df700 1 -- start start 2023-08-27T11:26:06.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.348+0000 7f51e19df700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc071980 0x7f51dc0cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:06.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.348+0000 7f51e19df700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f51dc15ea00 0x7f51dc0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:06.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.348+0000 7f51e19df700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51dc15b270 con 0x7f51dc15a760 2023-08-27T11:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.348+0000 7f51e19df700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc15e5e0 con 0x7f51dc15ea00 2023-08-27T11:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.348+0000 7f51e19df700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc15b510 con 0x7f51dc071980 2023-08-27T11:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.348+0000 7f51db7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f51dc15a760 0x7f51dc15ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55088/0 (socket says 172.21.15.112:55088) 2023-08-27T11:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.348+0000 7f51db7fe700 1 -- 172.21.15.112:0/2983498216 learned_addr learned my addr 172.21.15.112:0/2983498216 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.348+0000 7f51da7fc700 1 --2- 172.21.15.112:0/2983498216 >> v2:172.21.15.138:3300/0 conn(0x7f51dc15ea00 0x7f51dc0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.348+0000 7f51daffd700 1 --2- 172.21.15.112:0/2983498216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc071980 0x7f51dc0cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.349+0000 7f51d9ffb700 1 -- 172.21.15.112:0/2983498216 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433730643 0 0) 0x7f51dc15b270 con 0x7f51dc15a760 2023-08-27T11:26:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.349+0000 7f51d9ffb700 1 -- 172.21.15.112:0/2983498216 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51c40057c0 con 0x7f51dc15a760 2023-08-27T11:26:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.349+0000 7f51daffd700 1 -- 172.21.15.112:0/2983498216 >> v2:172.21.15.138:3300/0 conn(0x7f51dc15ea00 msgr2=0x7f51dc0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:06.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.349+0000 7f51daffd700 1 --2- 172.21.15.112:0/2983498216 >> v2:172.21.15.138:3300/0 conn(0x7f51dc15ea00 0x7f51dc0ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:06.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.349+0000 7f51daffd700 1 -- 172.21.15.112:0/2983498216 >> v1:172.21.15.112:6789/0 conn(0x7f51dc15a760 legacy=0x7f51dc15ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:06.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.349+0000 7f51daffd700 1 -- 172.21.15.112:0/2983498216 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc15df20 con 0x7f51dc071980 2023-08-27T11:26:06.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.349+0000 7f51da7fc700 1 --2- 172.21.15.112:0/2983498216 >> v2:172.21.15.138:3300/0 conn(0x7f51dc15ea00 0x7f51dc0ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:06.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.349+0000 7f51daffd700 1 --2- 172.21.15.112:0/2983498216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc071980 0x7f51dc0cc960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009840 tx=0x7f51cc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d932b4603af8f4ad server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.350+0000 7f51d9ffb700 1 -- 172.21.15.112:0/2983498216 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc011830 con 0x7f51dc071980 2023-08-27T11:26:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.350+0000 7f51d9ffb700 1 -- 172.21.15.112:0/2983498216 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc011990 con 0x7f51dc071980 2023-08-27T11:26:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.350+0000 7f51d9ffb700 1 -- 172.21.15.112:0/2983498216 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc011d00 con 0x7f51dc071980 2023-08-27T11:26:06.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.350+0000 7f51e19df700 1 -- 172.21.15.112:0/2983498216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc071980 msgr2=0x7f51dc0cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:06.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.350+0000 7f51e19df700 1 --2- 172.21.15.112:0/2983498216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc071980 0x7f51dc0cc960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009840 tx=0x7f51cc009b50 comp rx=0 tx=0).stop 2023-08-27T11:26:06.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.350+0000 7f51e19df700 1 -- 172.21.15.112:0/2983498216 shutdown_connections 2023-08-27T11:26:06.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.350+0000 7f51e19df700 1 --2- 172.21.15.112:0/2983498216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc071980 0x7f51dc0cc960 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:06.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.350+0000 7f51e19df700 1 --2- 172.21.15.112:0/2983498216 >> v2:172.21.15.138:3300/0 conn(0x7f51dc15ea00 0x7f51dc0ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:06.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.350+0000 7f51e19df700 1 -- 172.21.15.112:0/2983498216 >> 172.21.15.112:0/2983498216 conn(0x7f51dc06bbf0 msgr2=0x7f51dc0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:06.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.350+0000 7f51e19df700 1 -- 172.21.15.112:0/2983498216 shutdown_connections 2023-08-27T11:26:06.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.350+0000 7f51e19df700 1 -- 172.21.15.112:0/2983498216 wait complete. 2023-08-27T11:26:06.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.351+0000 7f51e19df700 1 Processor -- start 2023-08-27T11:26:06.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.352+0000 7f51e19df700 1 -- start start 2023-08-27T11:26:06.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.352+0000 7f51e19df700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51dc071980 0x7f51dc14acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:06.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.352+0000 7f51e19df700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc15a760 0x7f51dc15c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:06.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.352+0000 7f51e19df700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51dc15ea00 0x7f51dc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:06.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.352+0000 7f51e19df700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc1533b0 con 0x7f51dc15ea00 2023-08-27T11:26:06.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.352+0000 7f51e19df700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc153690 con 0x7f51dc071980 2023-08-27T11:26:06.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.352+0000 7f51e19df700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc153970 con 0x7f51dc15a760 2023-08-27T11:26:06.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51db7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51dc15ea00 0x7f51dc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:06.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51db7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51dc15ea00 0x7f51dc0b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60712/0 (socket says 172.21.15.112:60712) 2023-08-27T11:26:06.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51db7fe700 1 -- 172.21.15.112:0/1506938616 learned_addr learned my addr 172.21.15.112:0/1506938616 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:06.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51daffd700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51dc071980 0x7f51dc14acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:06.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51da7fc700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc15a760 0x7f51dc15c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:06.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51db7fe700 1 -- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc15a760 msgr2=0x7f51dc15c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:06.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51db7fe700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc15a760 0x7f51dc15c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:06.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51db7fe700 1 -- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51dc071980 msgr2=0x7f51dc14acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:06.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51db7fe700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51dc071980 0x7f51dc14acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:06.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51db7fe700 1 -- 172.21.15.112:0/1506938616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d00094a0 con 0x7f51dc15ea00 2023-08-27T11:26:06.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51db7fe700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51dc15ea00 0x7f51dc0b4db0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f51d400cb30 tx=0x7f51d400ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:06.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51c3fff700 1 -- 172.21.15.112:0/1506938616 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d400a9e0 con 0x7f51dc15ea00 2023-08-27T11:26:06.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51c3fff700 1 -- 172.21.15.112:0/1506938616 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d400ab40 con 0x7f51dc15ea00 2023-08-27T11:26:06.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51e19df700 1 -- 172.21.15.112:0/1506938616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51cc0094a0 con 0x7f51dc15ea00 2023-08-27T11:26:06.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51c3fff700 1 -- 172.21.15.112:0/1506938616 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4012400 con 0x7f51dc15ea00 2023-08-27T11:26:06.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.353+0000 7f51e19df700 1 -- 172.21.15.112:0/1506938616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc153f30 con 0x7f51dc15ea00 2023-08-27T11:26:06.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.354+0000 7f51e19df700 1 -- 172.21.15.112:0/1506938616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc066fa0 con 0x7f51dc15ea00 2023-08-27T11:26:06.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.355+0000 7f51c3fff700 1 -- 172.21.15.112:0/1506938616 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f51d401b3b0 con 0x7f51dc15ea00 2023-08-27T11:26:06.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.355+0000 7f51c3fff700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f51c806a3b0 0x7f51c806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:06.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.355+0000 7f51c3fff700 1 -- 172.21.15.112:0/1506938616 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f51d40885f0 con 0x7f51dc15ea00 2023-08-27T11:26:06.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.355+0000 7f51daffd700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f51c806a3b0 0x7f51c806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:06.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.356+0000 7f51daffd700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f51c806a3b0 0x7f51c806c870 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009fd0 tx=0x7f51cc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:06.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.359+0000 7f51c3fff700 1 -- 172.21.15.112:0/1506938616 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f51d4053930 con 0x7f51dc15ea00 2023-08-27T11:26:06.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:06.596+0000 7f51e19df700 1 -- 172.21.15.112:0/1506938616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f51dc067a20 con 0x7f51dc15ea00 2023-08-27T11:26:06.918 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:26:06.921 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:26:07.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.209+0000 7f51c3fff700 1 -- 172.21.15.112:0/1506938616 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f51d4056f50 con 0x7f51dc15ea00 2023-08-27T11:26:07.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.211+0000 7f51e19df700 1 -- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f51c806a3b0 msgr2=0x7f51c806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:07.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.211+0000 7f51e19df700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f51c806a3b0 0x7f51c806c870 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009fd0 tx=0x7f51cc008030 comp rx=0 tx=0).stop 2023-08-27T11:26:07.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.211+0000 7f51e19df700 1 -- 172.21.15.112:0/1506938616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51dc15ea00 msgr2=0x7f51dc0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:07.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.211+0000 7f51e19df700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51dc15ea00 0x7f51dc0b4db0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f51d400cb30 tx=0x7f51d400ce40 comp rx=0 tx=0).stop 2023-08-27T11:26:07.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.211+0000 7f51e19df700 1 -- 172.21.15.112:0/1506938616 shutdown_connections 2023-08-27T11:26:07.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.211+0000 7f51e19df700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f51c806a3b0 0x7f51c806c870 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:07.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.211+0000 7f51e19df700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51dc15ea00 0x7f51dc0b4db0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:07.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.211+0000 7f51e19df700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51dc071980 0x7f51dc14acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:07.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.211+0000 7f51e19df700 1 --2- 172.21.15.112:0/1506938616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc15a760 0x7f51dc15c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:07.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.211+0000 7f51e19df700 1 -- 172.21.15.112:0/1506938616 >> 172.21.15.112:0/1506938616 conn(0x7f51dc06bbf0 msgr2=0x7f51dc156f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:07.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.211+0000 7f51e19df700 1 -- 172.21.15.112:0/1506938616 shutdown_connections 2023-08-27T11:26:07.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.211+0000 7f51e19df700 1 -- 172.21.15.112:0/1506938616 wait complete. 2023-08-27T11:26:07.226 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:26:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.361+0000 7fb6de41a700 1 Processor -- start 2023-08-27T11:26:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.362+0000 7fb6de41a700 1 -- start start 2023-08-27T11:26:07.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.362+0000 7fb6de41a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6d80ce770 0x7fb6d80ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.362+0000 7fb6de41a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb6d8071eb0 0x7fb6d8072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.362+0000 7fb6de41a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d80a8480 con 0x7fb6d8140460 2023-08-27T11:26:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.362+0000 7fb6de41a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d8072880 con 0x7fb6d8071eb0 2023-08-27T11:26:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.362+0000 7fb6de41a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80a92e0 con 0x7fb6d80ce770 2023-08-27T11:26:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.363+0000 7fb6dc9b7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb6d8071eb0 0x7fb6d8072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.363+0000 7fb6d77fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6d80ce770 0x7fb6d80ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.363+0000 7fb6d7fff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb6d8140460 0x7fb6d815b850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:34392/0 (socket says 172.21.15.112:34392) 2023-08-27T11:26:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.363+0000 7fb6d7fff700 1 -- 172.21.15.112:0/576637711 learned_addr learned my addr 172.21.15.112:0/576637711 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.363+0000 7fb6dc9b7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb6d8071eb0 0x7fb6d8072340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:32916/0 (socket says 172.21.15.112:32916) 2023-08-27T11:26:07.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.363+0000 7fb6dc9b7700 1 -- 172.21.15.112:0/576637711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6d80ce770 msgr2=0x7fb6d80ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:07.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.363+0000 7fb6dc9b7700 1 --2- 172.21.15.112:0/576637711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6d80ce770 0x7fb6d80ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:07.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.363+0000 7fb6dc9b7700 1 -- 172.21.15.112:0/576637711 >> v1:172.21.15.138:6789/0 conn(0x7fb6d8140460 legacy=0x7fb6d815b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:07.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.363+0000 7fb6dc9b7700 1 -- 172.21.15.112:0/576637711 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d80a86d0 con 0x7fb6d8071eb0 2023-08-27T11:26:07.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.363+0000 7fb6dc9b7700 1 --2- 172.21.15.112:0/576637711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d8071eb0 0x7fb6d8072340 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009d90 tx=0x7fb6d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91f2e9f77df3a94b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:07.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.363+0000 7fb6d6ffd700 1 -- 172.21.15.112:0/576637711 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d000e070 con 0x7fb6d8071eb0 2023-08-27T11:26:07.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.363+0000 7fb6d6ffd700 1 -- 172.21.15.112:0/576637711 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d000bb20 con 0x7fb6d8071eb0 2023-08-27T11:26:07.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.363+0000 7fb6d6ffd700 1 -- 172.21.15.112:0/576637711 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0012430 con 0x7fb6d8071eb0 2023-08-27T11:26:07.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.364+0000 7fb6de41a700 1 -- 172.21.15.112:0/576637711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d8071eb0 msgr2=0x7fb6d8072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:07.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.364+0000 7fb6de41a700 1 --2- 172.21.15.112:0/576637711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d8071eb0 0x7fb6d8072340 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009d90 tx=0x7fb6d000b330 comp rx=0 tx=0).stop 2023-08-27T11:26:07.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.364+0000 7fb6de41a700 1 -- 172.21.15.112:0/576637711 shutdown_connections 2023-08-27T11:26:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.364+0000 7fb6de41a700 1 --2- 172.21.15.112:0/576637711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d8071eb0 0x7fb6d8072340 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.364+0000 7fb6de41a700 1 --2- 172.21.15.112:0/576637711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6d80ce770 0x7fb6d80ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.364+0000 7fb6de41a700 1 -- 172.21.15.112:0/576637711 >> 172.21.15.112:0/576637711 conn(0x7fb6d806ccc0 msgr2=0x7fb6d80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.364+0000 7fb6de41a700 1 -- 172.21.15.112:0/576637711 shutdown_connections 2023-08-27T11:26:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.364+0000 7fb6de41a700 1 -- 172.21.15.112:0/576637711 wait complete. 2023-08-27T11:26:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.365+0000 7fb6de41a700 1 Processor -- start 2023-08-27T11:26:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.365+0000 7fb6de41a700 1 -- start start 2023-08-27T11:26:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6de41a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6d8071eb0 0x7fb6d8157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6de41a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6d80ce770 0x7fb6d81f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6de41a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d8140460 0x7fb6d81f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6de41a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8157b90 con 0x7fb6d8071eb0 2023-08-27T11:26:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6de41a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8156960 con 0x7fb6d8140460 2023-08-27T11:26:07.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6de41a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d81571c0 con 0x7fb6d80ce770 2023-08-27T11:26:07.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6d8071eb0 0x7fb6d8157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:07.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6d8071eb0 0x7fb6d8157650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60724/0 (socket says 172.21.15.112:60724) 2023-08-27T11:26:07.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6d7fff700 1 -- 172.21.15.112:0/2038980858 learned_addr learned my addr 172.21.15.112:0/2038980858 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:07.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6dc9b7700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d8140460 0x7fb6d81f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6d77fe700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6d80ce770 0x7fb6d81f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6d7fff700 1 -- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6d80ce770 msgr2=0x7fb6d81f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6d7fff700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6d80ce770 0x7fb6d81f6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6d7fff700 1 -- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d8140460 msgr2=0x7fb6d81f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6d7fff700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d8140460 0x7fb6d81f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.366+0000 7fb6d7fff700 1 -- 172.21.15.112:0/2038980858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d0009a70 con 0x7fb6d8071eb0 2023-08-27T11:26:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.367+0000 7fb6d7fff700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6d8071eb0 0x7fb6d8157650 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800c480 tx=0x7fb6c800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.367+0000 7fb6d4ff9700 1 -- 172.21.15.112:0/2038980858 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8005d70 con 0x7fb6d8071eb0 2023-08-27T11:26:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.367+0000 7fb6d4ff9700 1 -- 172.21.15.112:0/2038980858 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c80070d0 con 0x7fb6d8071eb0 2023-08-27T11:26:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.367+0000 7fb6d4ff9700 1 -- 172.21.15.112:0/2038980858 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8007440 con 0x7fb6d8071eb0 2023-08-27T11:26:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.367+0000 7fb6de41a700 1 -- 172.21.15.112:0/2038980858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d81f6b50 con 0x7fb6d8071eb0 2023-08-27T11:26:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.367+0000 7fb6de41a700 1 -- 172.21.15.112:0/2038980858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d80b1710 con 0x7fb6d8071eb0 2023-08-27T11:26:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.368+0000 7fb6de41a700 1 -- 172.21.15.112:0/2038980858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d8067a20 con 0x7fb6d8071eb0 2023-08-27T11:26:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.368+0000 7fb6d4ff9700 1 -- 172.21.15.112:0/2038980858 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fb6c80075a0 con 0x7fb6d8071eb0 2023-08-27T11:26:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.369+0000 7fb6d4ff9700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6c406a3b0 0x7fb6c406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.369+0000 7fb6d4ff9700 1 -- 172.21.15.112:0/2038980858 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6264+0+0 (secure 0 0 0) 0x7fb6c8087e60 con 0x7fb6d8071eb0 2023-08-27T11:26:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.369+0000 7fb6d77fe700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6c406a3b0 0x7fb6c406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.369+0000 7fb6d77fe700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6c406a3b0 0x7fb6c406c870 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8143db0 tx=0x7fb6cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.373+0000 7fb6d4ff9700 1 -- 172.21.15.112:0/2038980858 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb6c80531e0 con 0x7fb6d8071eb0 2023-08-27T11:26:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:07.561+0000 7fb6de41a700 1 -- 172.21.15.112:0/2038980858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb6d80ce090 con 0x7fb6d8071eb0 2023-08-27T11:26:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.361+0000 7fb6d4ff9700 1 -- 172.21.15.112:0/2038980858 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v81) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb6c8056800 con 0x7fb6d8071eb0 2023-08-27T11:26:08.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.364+0000 7fb6de41a700 1 -- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6c406a3b0 msgr2=0x7fb6c406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:08.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.364+0000 7fb6de41a700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6c406a3b0 0x7fb6c406c870 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8143db0 tx=0x7fb6cc008040 comp rx=0 tx=0).stop 2023-08-27T11:26:08.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.365+0000 7fb6de41a700 1 -- 172.21.15.112:0/2038980858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6d8071eb0 msgr2=0x7fb6d8157650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:08.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.365+0000 7fb6de41a700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6d8071eb0 0x7fb6d8157650 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800c480 tx=0x7fb6c800c840 comp rx=0 tx=0).stop 2023-08-27T11:26:08.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.365+0000 7fb6de41a700 1 -- 172.21.15.112:0/2038980858 shutdown_connections 2023-08-27T11:26:08.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.365+0000 7fb6de41a700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6c406a3b0 0x7fb6c406c870 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:08.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.365+0000 7fb6de41a700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d8140460 0x7fb6d81f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:08.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.365+0000 7fb6de41a700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6d8071eb0 0x7fb6d8157650 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:08.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.365+0000 7fb6de41a700 1 --2- 172.21.15.112:0/2038980858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6d80ce770 0x7fb6d81f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:08.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.365+0000 7fb6de41a700 1 -- 172.21.15.112:0/2038980858 >> 172.21.15.112:0/2038980858 conn(0x7fb6d806ccc0 msgr2=0x7fb6d813e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:08.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.365+0000 7fb6de41a700 1 -- 172.21.15.112:0/2038980858 shutdown_connections 2023-08-27T11:26:08.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.365+0000 7fb6de41a700 1 -- 172.21.15.112:0/2038980858 wait complete. 2023-08-27T11:26:08.369 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:26:08.382 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:26:08.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.512+0000 7fbcee02d700 1 Processor -- start 2023-08-27T11:26:08.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.512+0000 7fbcee02d700 1 -- start start 2023-08-27T11:26:08.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.513+0000 7fbcee02d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbce8143dc0 0x7fbce80b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:08.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.513+0000 7fbcee02d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbce80b1420 0x7fbce815a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:08.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.513+0000 7fbcee02d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce80b0b10 con 0x7fbce81439e0 2023-08-27T11:26:08.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.513+0000 7fbcee02d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce806d850 con 0x7fbce80b1420 2023-08-27T11:26:08.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.513+0000 7fbcee02d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce815b460 con 0x7fbce8143dc0 2023-08-27T11:26:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce7fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbce80b1420 0x7fbce815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce77fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbce81439e0 0x7fbce80b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55114/0 (socket says 172.21.15.112:55114) 2023-08-27T11:26:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce77fe700 1 -- 172.21.15.112:0/206735089 learned_addr learned my addr 172.21.15.112:0/206735089 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce6ffd700 1 --2- 172.21.15.112:0/206735089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbce8143dc0 0x7fbce80b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:08.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce7fff700 1 -- 172.21.15.112:0/206735089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbce8143dc0 msgr2=0x7fbce80b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce7fff700 1 --2- 172.21.15.112:0/206735089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbce8143dc0 0x7fbce80b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce7fff700 1 -- 172.21.15.112:0/206735089 >> v1:172.21.15.112:6789/0 conn(0x7fbce81439e0 legacy=0x7fbce80b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce7fff700 1 -- 172.21.15.112:0/206735089 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce815b740 con 0x7fbce80b1420 2023-08-27T11:26:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce67fc700 1 -- 172.21.15.112:0/206735089 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858635629 0 0) 0x7fbce80b0b10 con 0x7fbce81439e0 2023-08-27T11:26:08.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce6ffd700 1 --2- 172.21.15.112:0/206735089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbce8143dc0 0x7fbce80b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce7fff700 1 --2- 172.21.15.112:0/206735089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce80b1420 0x7fbce815a810 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009d90 tx=0x7fbce000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a343e6caf34e4c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce67fc700 1 -- 172.21.15.112:0/206735089 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce000e070 con 0x7fbce80b1420 2023-08-27T11:26:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce67fc700 1 -- 172.21.15.112:0/206735089 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce000bbd0 con 0x7fbce80b1420 2023-08-27T11:26:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.514+0000 7fbce67fc700 1 -- 172.21.15.112:0/206735089 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0012430 con 0x7fbce80b1420 2023-08-27T11:26:08.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.515+0000 7fbcee02d700 1 -- 172.21.15.112:0/206735089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce80b1420 msgr2=0x7fbce815a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:08.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.515+0000 7fbcee02d700 1 --2- 172.21.15.112:0/206735089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce80b1420 0x7fbce815a810 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009d90 tx=0x7fbce000b330 comp rx=0 tx=0).stop 2023-08-27T11:26:08.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.515+0000 7fbcee02d700 1 -- 172.21.15.112:0/206735089 shutdown_connections 2023-08-27T11:26:08.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.515+0000 7fbcee02d700 1 --2- 172.21.15.112:0/206735089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce80b1420 0x7fbce815a810 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:08.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.515+0000 7fbcee02d700 1 --2- 172.21.15.112:0/206735089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbce8143dc0 0x7fbce80b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:08.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.515+0000 7fbcee02d700 1 -- 172.21.15.112:0/206735089 >> 172.21.15.112:0/206735089 conn(0x7fbce80b1be0 msgr2=0x7fbce80b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:08.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.515+0000 7fbcee02d700 1 -- 172.21.15.112:0/206735089 shutdown_connections 2023-08-27T11:26:08.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.515+0000 7fbcee02d700 1 -- 172.21.15.112:0/206735089 wait complete. 2023-08-27T11:26:08.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.516+0000 7fbcee02d700 1 Processor -- start 2023-08-27T11:26:08.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.516+0000 7fbcee02d700 1 -- start start 2023-08-27T11:26:08.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbcee02d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbce80b1420 0x7fbce80cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:08.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbcee02d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbce81439e0 0x7fbce806fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:08.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbcee02d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce80d0360 0x7fbce81586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:08.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbcee02d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce80cf800 con 0x7fbce80d0360 2023-08-27T11:26:08.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbcee02d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce80cdc70 con 0x7fbce81439e0 2023-08-27T11:26:08.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbcee02d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce80ceab0 con 0x7fbce80b1420 2023-08-27T11:26:08.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbce7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce80d0360 0x7fbce81586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:08.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbce7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce80d0360 0x7fbce81586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60742/0 (socket says 172.21.15.112:60742) 2023-08-27T11:26:08.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbce7fff700 1 -- 172.21.15.112:0/4245168554 learned_addr learned my addr 172.21.15.112:0/4245168554 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:08.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbce6ffd700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbce81439e0 0x7fbce806fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:08.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbce77fe700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbce80b1420 0x7fbce80cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:08.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbce7fff700 1 -- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbce80b1420 msgr2=0x7fbce80cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:08.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbce7fff700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbce80b1420 0x7fbce80cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:08.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbce7fff700 1 -- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbce81439e0 msgr2=0x7fbce806fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:08.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbce7fff700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbce81439e0 0x7fbce806fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:08.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbce7fff700 1 -- 172.21.15.112:0/4245168554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce0009a70 con 0x7fbce80d0360 2023-08-27T11:26:08.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.517+0000 7fbce6ffd700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbce81439e0 0x7fbce806fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:08.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.518+0000 7fbce7fff700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce80d0360 0x7fbce81586d0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbce000b730 tx=0x7fbce0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:08.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.518+0000 7fbccffff700 1 -- 172.21.15.112:0/4245168554 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0012430 con 0x7fbce80d0360 2023-08-27T11:26:08.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.518+0000 7fbccffff700 1 -- 172.21.15.112:0/4245168554 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce0020070 con 0x7fbce80d0360 2023-08-27T11:26:08.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.518+0000 7fbcee02d700 1 -- 172.21.15.112:0/4245168554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce80cdea0 con 0x7fbce80d0360 2023-08-27T11:26:08.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.518+0000 7fbccffff700 1 -- 172.21.15.112:0/4245168554 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0004280 con 0x7fbce80d0360 2023-08-27T11:26:08.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.518+0000 7fbcee02d700 1 -- 172.21.15.112:0/4245168554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce80ce460 con 0x7fbce80d0360 2023-08-27T11:26:08.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.519+0000 7fbcee02d700 1 -- 172.21.15.112:0/4245168554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce81f2a40 con 0x7fbce80d0360 2023-08-27T11:26:08.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.519+0000 7fbccffff700 1 -- 172.21.15.112:0/4245168554 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fbce00043e0 con 0x7fbce80d0360 2023-08-27T11:26:08.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.520+0000 7fbccffff700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbcd406a3b0 0x7fbcd406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:08.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.520+0000 7fbccffff700 1 -- 172.21.15.112:0/4245168554 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5826+0+0 (secure 0 0 0) 0x7fbce0022070 con 0x7fbce80d0360 2023-08-27T11:26:08.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.520+0000 7fbce77fe700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbcd406a3b0 0x7fbcd406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:08.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.520+0000 7fbce77fe700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbcd406a3b0 0x7fbcd406c870 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc000f60 tx=0x7fbcdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:08.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.524+0000 7fbccffff700 1 -- 172.21.15.112:0/4245168554 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbce81f2a40 con 0x7fbce80d0360 2023-08-27T11:26:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:08.713+0000 7fbcee02d700 1 -- 172.21.15.112:0/4245168554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbce81f2a40 con 0x7fbce80d0360 2023-08-27T11:26:09.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.371+0000 7fbccffff700 1 -- 172.21.15.112:0/4245168554 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v82) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbce81f2a40 con 0x7fbce80d0360 2023-08-27T11:26:09.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.375+0000 7fbcee02d700 1 -- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbcd406a3b0 msgr2=0x7fbcd406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:09.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.375+0000 7fbcee02d700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbcd406a3b0 0x7fbcd406c870 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc000f60 tx=0x7fbcdc009040 comp rx=0 tx=0).stop 2023-08-27T11:26:09.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.375+0000 7fbcee02d700 1 -- 172.21.15.112:0/4245168554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce80d0360 msgr2=0x7fbce81586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:09.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.375+0000 7fbcee02d700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce80d0360 0x7fbce81586d0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbce000b730 tx=0x7fbce0004ad0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.375+0000 7fbcee02d700 1 -- 172.21.15.112:0/4245168554 shutdown_connections 2023-08-27T11:26:09.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.375+0000 7fbcee02d700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbcd406a3b0 0x7fbcd406c870 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.375+0000 7fbcee02d700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce80d0360 0x7fbce81586d0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.375+0000 7fbcee02d700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbce80b1420 0x7fbce80cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.375+0000 7fbcee02d700 1 --2- 172.21.15.112:0/4245168554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbce81439e0 0x7fbce806fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.375+0000 7fbcee02d700 1 -- 172.21.15.112:0/4245168554 >> 172.21.15.112:0/4245168554 conn(0x7fbce80b1be0 msgr2=0x7fbce806da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:09.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.375+0000 7fbcee02d700 1 -- 172.21.15.112:0/4245168554 shutdown_connections 2023-08-27T11:26:09.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.375+0000 7fbcee02d700 1 -- 172.21.15.112:0/4245168554 wait complete. 2023-08-27T11:26:09.381 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2023-08-27T11:26:09.392 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:26:09.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.520+0000 7fbdc0d0d700 1 Processor -- start 2023-08-27T11:26:09.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.520+0000 7fbdc0d0d700 1 -- start start 2023-08-27T11:26:09.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdc0d0d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbdbc15e190 0x7fbdbc15e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:09.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdc0d0d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbdbc161dd0 0x7fbdbc0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdc0d0d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdbc0cc980 0x7fbdbc0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdc0d0d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdbc0cd340 con 0x7fbdbc161dd0 2023-08-27T11:26:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdc0d0d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdbc0b46c0 con 0x7fbdbc15e190 2023-08-27T11:26:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdc0d0d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc0b49a0 con 0x7fbdbc0cc980 2023-08-27T11:26:09.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdb9d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbdbc161dd0 0x7fbdbc0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:09.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdb9d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbdbc161dd0 0x7fbdbc0b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60746/0 (socket says 172.21.15.112:60746) 2023-08-27T11:26:09.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdb9d9b700 1 -- 172.21.15.112:0/2010207861 learned_addr learned my addr 172.21.15.112:0/2010207861 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:09.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdbad9d700 1 --2- 172.21.15.112:0/2010207861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdbc0cc980 0x7fbdbc0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:09.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdba59c700 1 --2- 172.21.15.112:0/2010207861 >> v2:172.21.15.138:3300/0 conn(0x7fbdbc15e190 0x7fbdbc15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:09.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdb9d9b700 1 -- 172.21.15.112:0/2010207861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdbc0cc980 msgr2=0x7fbdbc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:09.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdb9d9b700 1 --2- 172.21.15.112:0/2010207861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdbc0cc980 0x7fbdbc0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdb9d9b700 1 -- 172.21.15.112:0/2010207861 >> v2:172.21.15.138:3300/0 conn(0x7fbdbc15e190 msgr2=0x7fbdbc15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:09.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdb9d9b700 1 --2- 172.21.15.112:0/2010207861 >> v2:172.21.15.138:3300/0 conn(0x7fbdbc15e190 0x7fbdbc15e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.521+0000 7fbdb9d9b700 1 -- 172.21.15.112:0/2010207861 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc1612f0 con 0x7fbdbc161dd0 2023-08-27T11:26:09.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.522+0000 7fbdb9d9b700 1 --2- 172.21.15.112:0/2010207861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc161dd0 0x7fbdbc0b3dc0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0d0fb0 tx=0x7fbdac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fe54020084baa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:09.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.522+0000 7fbdb959a700 1 -- 172.21.15.112:0/2010207861 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00b870 con 0x7fbdbc161dd0 2023-08-27T11:26:09.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.522+0000 7fbdb959a700 1 -- 172.21.15.112:0/2010207861 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac00b9d0 con 0x7fbdbc161dd0 2023-08-27T11:26:09.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.522+0000 7fbdb959a700 1 -- 172.21.15.112:0/2010207861 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00bd60 con 0x7fbdbc161dd0 2023-08-27T11:26:09.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.522+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/2010207861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc161dd0 msgr2=0x7fbdbc0b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:09.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.522+0000 7fbdc0d0d700 1 --2- 172.21.15.112:0/2010207861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc161dd0 0x7fbdbc0b3dc0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0d0fb0 tx=0x7fbdac009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.522+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/2010207861 shutdown_connections 2023-08-27T11:26:09.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.522+0000 7fbdc0d0d700 1 --2- 172.21.15.112:0/2010207861 >> v2:172.21.15.138:3300/0 conn(0x7fbdbc15e190 0x7fbdbc15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.522+0000 7fbdc0d0d700 1 --2- 172.21.15.112:0/2010207861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdbc0cc980 0x7fbdbc0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.522+0000 7fbdc0d0d700 1 --2- 172.21.15.112:0/2010207861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc161dd0 0x7fbdbc0b3dc0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.522+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/2010207861 >> 172.21.15.112:0/2010207861 conn(0x7fbdbc06bbf0 msgr2=0x7fbdbc0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:09.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.522+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/2010207861 shutdown_connections 2023-08-27T11:26:09.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.522+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/2010207861 wait complete. 2023-08-27T11:26:09.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.524+0000 7fbdc0d0d700 1 Processor -- start 2023-08-27T11:26:09.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.524+0000 7fbdc0d0d700 1 -- start start 2023-08-27T11:26:09.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdc0d0d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdbc0cc980 0x7fbdbc14e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:09.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdc0d0d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdbc15e190 0x7fbdbc06f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:09.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdc0d0d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc161dd0 0x7fbdbc0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:09.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdc0d0d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc14ed00 con 0x7fbdbc161dd0 2023-08-27T11:26:09.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdc0d0d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc06ff30 con 0x7fbdbc15e190 2023-08-27T11:26:09.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdc0d0d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc070210 con 0x7fbdbc0cc980 2023-08-27T11:26:09.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdbad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc161dd0 0x7fbdbc0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:09.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdbad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc161dd0 0x7fbdbc0cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60756/0 (socket says 172.21.15.112:60756) 2023-08-27T11:26:09.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdbad9d700 1 -- 172.21.15.112:0/334607459 learned_addr learned my addr 172.21.15.112:0/334607459 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:09.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdb9d9b700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdbc15e190 0x7fbdbc06f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:09.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdba59c700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdbc0cc980 0x7fbdbc14e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:09.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdbad9d700 1 -- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdbc0cc980 msgr2=0x7fbdbc14e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:09.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdbad9d700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdbc0cc980 0x7fbdbc14e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdbad9d700 1 -- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdbc15e190 msgr2=0x7fbdbc06f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:09.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdbad9d700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdbc15e190 0x7fbdbc06f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdbad9d700 1 -- 172.21.15.112:0/334607459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdac009550 con 0x7fbdbc161dd0 2023-08-27T11:26:09.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdbad9d700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc161dd0 0x7fbdbc0cec50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4007e60 tx=0x7fbdb400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:09.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdaaffd700 1 -- 172.21.15.112:0/334607459 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb400ed30 con 0x7fbdbc161dd0 2023-08-27T11:26:09.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdaaffd700 1 -- 172.21.15.112:0/334607459 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb4014420 con 0x7fbdbc161dd0 2023-08-27T11:26:09.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdaaffd700 1 -- 172.21.15.112:0/334607459 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40147d0 con 0x7fbdbc161dd0 2023-08-27T11:26:09.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/334607459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdbc0704f0 con 0x7fbdbc161dd0 2023-08-27T11:26:09.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.525+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/334607459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdbc070f80 con 0x7fbdbc161dd0 2023-08-27T11:26:09.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.526+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/334607459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdbc066fa0 con 0x7fbdbc161dd0 2023-08-27T11:26:09.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.527+0000 7fbdaaffd700 1 -- 172.21.15.112:0/334607459 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fbdb4014930 con 0x7fbdbc161dd0 2023-08-27T11:26:09.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.527+0000 7fbdaaffd700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd9c06a3b0 0x7fbd9c06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:09.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.527+0000 7fbdaaffd700 1 -- 172.21.15.112:0/334607459 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbdb4088ca0 con 0x7fbdbc161dd0 2023-08-27T11:26:09.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.527+0000 7fbdba59c700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd9c06a3b0 0x7fbd9c06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:09.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.528+0000 7fbdba59c700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd9c06a3b0 0x7fbd9c06c870 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0006b20 tx=0x7fbdb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.531+0000 7fbdaaffd700 1 -- 172.21.15.112:0/334607459 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbdb4054340 con 0x7fbdbc161dd0 2023-08-27T11:26:09.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.729+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/334607459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbdbc0711a0 con 0x7fbdbc161dd0 2023-08-27T11:26:09.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.732+0000 7fbdaaffd700 1 -- 172.21.15.112:0/334607459 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbdb4057960 con 0x7fbdbc161dd0 2023-08-27T11:26:09.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.734+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd9c06a3b0 msgr2=0x7fbd9c06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:09.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.734+0000 7fbdc0d0d700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd9c06a3b0 0x7fbd9c06c870 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0006b20 tx=0x7fbdb0008040 comp rx=0 tx=0).stop 2023-08-27T11:26:09.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.734+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/334607459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc161dd0 msgr2=0x7fbdbc0cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:09.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.734+0000 7fbdc0d0d700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc161dd0 0x7fbdbc0cec50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4007e60 tx=0x7fbdb400e3c0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.734+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/334607459 shutdown_connections 2023-08-27T11:26:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.734+0000 7fbdc0d0d700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd9c06a3b0 0x7fbd9c06c870 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.734+0000 7fbdc0d0d700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc161dd0 0x7fbdbc0cec50 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.734+0000 7fbdc0d0d700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdbc0cc980 0x7fbdbc14e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.734+0000 7fbdc0d0d700 1 --2- 172.21.15.112:0/334607459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdbc15e190 0x7fbdbc06f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.734+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/334607459 >> 172.21.15.112:0/334607459 conn(0x7fbdbc06bbf0 msgr2=0x7fbdbc0ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:09.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.734+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/334607459 shutdown_connections 2023-08-27T11:26:09.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.734+0000 7fbdc0d0d700 1 -- 172.21.15.112:0/334607459 wait complete. 2023-08-27T11:26:09.748 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag' 2023-08-27T11:26:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.867+0000 7fd7088b9700 1 Processor -- start 2023-08-27T11:26:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.867+0000 7fd7088b9700 1 -- start start 2023-08-27T11:26:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.868+0000 7fd7088b9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd70406adb0 0x7fd70406b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.868+0000 7fd7088b9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd7040cdbd0 0x7fd7040ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:09.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.868+0000 7fd7088b9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70415eb60 con 0x7fd704151060 2023-08-27T11:26:09.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.868+0000 7fd7088b9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70406b770 con 0x7fd7040cdbd0 2023-08-27T11:26:09.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.868+0000 7fd7088b9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7040b02f0 con 0x7fd70406adb0 2023-08-27T11:26:09.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.868+0000 7fd701d9b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd70406adb0 0x7fd70406b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:09.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.868+0000 7fd701d9b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd70406adb0 0x7fd70406b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:44184/0 (socket says 172.21.15.112:44184) 2023-08-27T11:26:09.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.868+0000 7fd701d9b700 1 -- 172.21.15.112:0/4067841010 learned_addr learned my addr 172.21.15.112:0/4067841010 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:09.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.868+0000 7fd702d9d700 1 --2- 172.21.15.112:0/4067841010 >> v2:172.21.15.138:3300/0 conn(0x7fd7040cdbd0 0x7fd7040ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:09.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.869+0000 7fd701d9b700 1 -- 172.21.15.112:0/4067841010 >> v2:172.21.15.138:3300/0 conn(0x7fd7040cdbd0 msgr2=0x7fd7040ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.869+0000 7fd701d9b700 1 --2- 172.21.15.112:0/4067841010 >> v2:172.21.15.138:3300/0 conn(0x7fd7040cdbd0 0x7fd7040ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.869+0000 7fd701d9b700 1 -- 172.21.15.112:0/4067841010 >> v1:172.21.15.138:6789/0 conn(0x7fd704151060 legacy=0x7fd70415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.869+0000 7fd701d9b700 1 -- 172.21.15.112:0/4067841010 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7040b05d0 con 0x7fd70406adb0 2023-08-27T11:26:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.869+0000 7fd701d9b700 1 --2- 172.21.15.112:0/4067841010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd70406adb0 0x7fd70406b230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80098f0 tx=0x7fd6f8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36705ea5bceb0920 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.869+0000 7fd70159a700 1 -- 172.21.15.112:0/4067841010 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8011830 con 0x7fd70406adb0 2023-08-27T11:26:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.869+0000 7fd70159a700 1 -- 172.21.15.112:0/4067841010 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f8011990 con 0x7fd70406adb0 2023-08-27T11:26:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.869+0000 7fd70159a700 1 -- 172.21.15.112:0/4067841010 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8011d20 con 0x7fd70406adb0 2023-08-27T11:26:09.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.869+0000 7fd7088b9700 1 -- 172.21.15.112:0/4067841010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd70406adb0 msgr2=0x7fd70406b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:09.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.869+0000 7fd7088b9700 1 --2- 172.21.15.112:0/4067841010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd70406adb0 0x7fd70406b230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80098f0 tx=0x7fd6f8009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:09.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.870+0000 7fd7088b9700 1 -- 172.21.15.112:0/4067841010 shutdown_connections 2023-08-27T11:26:09.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.870+0000 7fd7088b9700 1 --2- 172.21.15.112:0/4067841010 >> v2:172.21.15.138:3300/0 conn(0x7fd7040cdbd0 0x7fd7040ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.870+0000 7fd7088b9700 1 --2- 172.21.15.112:0/4067841010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd70406adb0 0x7fd70406b230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.870+0000 7fd7088b9700 1 -- 172.21.15.112:0/4067841010 >> 172.21.15.112:0/4067841010 conn(0x7fd7040d0000 msgr2=0x7fd7040d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:09.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.870+0000 7fd7088b9700 1 -- 172.21.15.112:0/4067841010 shutdown_connections 2023-08-27T11:26:09.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.870+0000 7fd7088b9700 1 -- 172.21.15.112:0/4067841010 wait complete. 2023-08-27T11:26:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.871+0000 7fd7088b9700 1 Processor -- start 2023-08-27T11:26:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.871+0000 7fd7088b9700 1 -- start start 2023-08-27T11:26:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd7088b9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70406adb0 0x7fd70414d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd7088b9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7040cdbd0 0x7fd7040c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:09.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd7088b9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd704151060 0x7fd70415c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:09.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd7088b9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70414db00 con 0x7fd7040cdbd0 2023-08-27T11:26:09.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd7088b9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7040c7a70 con 0x7fd70406adb0 2023-08-27T11:26:09.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd7088b9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7040c8310 con 0x7fd704151060 2023-08-27T11:26:09.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd701d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7040cdbd0 0x7fd7040c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd70259c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70406adb0 0x7fd70414d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd701d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7040cdbd0 0x7fd7040c4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60768/0 (socket says 172.21.15.112:60768) 2023-08-27T11:26:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd701d9b700 1 -- 172.21.15.112:0/3624009309 learned_addr learned my addr 172.21.15.112:0/3624009309 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd702d9d700 1 --2- 172.21.15.112:0/3624009309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd704151060 0x7fd70415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd701d9b700 1 -- 172.21.15.112:0/3624009309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd704151060 msgr2=0x7fd70415c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:09.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd701d9b700 1 --2- 172.21.15.112:0/3624009309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd704151060 0x7fd70415c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd701d9b700 1 -- 172.21.15.112:0/3624009309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70406adb0 msgr2=0x7fd70414d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:09.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd701d9b700 1 --2- 172.21.15.112:0/3624009309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70406adb0 0x7fd70414d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:09.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd701d9b700 1 -- 172.21.15.112:0/3624009309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f8009550 con 0x7fd7040cdbd0 2023-08-27T11:26:09.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd701d9b700 1 --2- 172.21.15.112:0/3624009309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7040cdbd0 0x7fd7040c4a40 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009c40 tx=0x7fd6f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:09.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd6f2ffd700 1 -- 172.21.15.112:0/3624009309 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8004590 con 0x7fd7040cdbd0 2023-08-27T11:26:09.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd6f2ffd700 1 -- 172.21.15.112:0/3624009309 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f801e070 con 0x7fd7040cdbd0 2023-08-27T11:26:09.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.872+0000 7fd6f2ffd700 1 -- 172.21.15.112:0/3624009309 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f80038e0 con 0x7fd7040cdbd0 2023-08-27T11:26:09.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.873+0000 7fd7088b9700 1 -- 172.21.15.112:0/3624009309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70414d560 con 0x7fd7040cdbd0 2023-08-27T11:26:09.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.873+0000 7fd7088b9700 1 -- 172.21.15.112:0/3624009309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7040c7cc0 con 0x7fd7040cdbd0 2023-08-27T11:26:09.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.873+0000 7fd7088b9700 1 -- 172.21.15.112:0/3624009309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7040c84c0 con 0x7fd7040cdbd0 2023-08-27T11:26:09.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.874+0000 7fd6f2ffd700 1 -- 172.21.15.112:0/3624009309 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fd6f8003a40 con 0x7fd7040cdbd0 2023-08-27T11:26:09.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.874+0000 7fd6f2ffd700 1 --2- 172.21.15.112:0/3624009309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6e406a3b0 0x7fd6e406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:09.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.874+0000 7fd6f2ffd700 1 -- 172.21.15.112:0/3624009309 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd6f8089490 con 0x7fd7040cdbd0 2023-08-27T11:26:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.874+0000 7fd70259c700 1 --2- 172.21.15.112:0/3624009309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6e406a3b0 0x7fd6e406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.875+0000 7fd70259c700 1 --2- 172.21.15.112:0/3624009309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6e406a3b0 0x7fd6e406c870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd7041f2d90 tx=0x7fd6f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:09.878+0000 7fd6f2ffd700 1 -- 172.21.15.112:0/3624009309 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd7040c84c0 con 0x7fd7040cdbd0 2023-08-27T11:26:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.147+0000 7fd7088b9700 1 -- 172.21.15.112:0/3624009309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]} v 0) v1 -- 0x7fd7040c8480 con 0x7fd7040cdbd0 2023-08-27T11:26:10.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.370+0000 7fd6f2ffd700 1 -- 172.21.15.112:0/3624009309 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]}]=0 v384) v1 ==== 158+0+0 (secure 0 0 0) 0x7fd6f8054bc0 con 0x7fd7040cdbd0 2023-08-27T11:26:10.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.372+0000 7fd7088b9700 1 -- 172.21.15.112:0/3624009309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6e406a3b0 msgr2=0x7fd6e406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:10.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.372+0000 7fd7088b9700 1 --2- 172.21.15.112:0/3624009309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6e406a3b0 0x7fd6e406c870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd7041f2d90 tx=0x7fd6f4009040 comp rx=0 tx=0).stop 2023-08-27T11:26:10.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.372+0000 7fd7088b9700 1 -- 172.21.15.112:0/3624009309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7040cdbd0 msgr2=0x7fd7040c4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:10.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.372+0000 7fd7088b9700 1 --2- 172.21.15.112:0/3624009309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7040cdbd0 0x7fd7040c4a40 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009c40 tx=0x7fd6f8004750 comp rx=0 tx=0).stop 2023-08-27T11:26:10.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.372+0000 7fd7088b9700 1 -- 172.21.15.112:0/3624009309 shutdown_connections 2023-08-27T11:26:10.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.372+0000 7fd7088b9700 1 --2- 172.21.15.112:0/3624009309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6e406a3b0 0x7fd6e406c870 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:10.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.372+0000 7fd7088b9700 1 --2- 172.21.15.112:0/3624009309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd704151060 0x7fd70415c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:10.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.372+0000 7fd7088b9700 1 --2- 172.21.15.112:0/3624009309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70406adb0 0x7fd70414d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:10.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.372+0000 7fd7088b9700 1 --2- 172.21.15.112:0/3624009309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7040cdbd0 0x7fd7040c4a40 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:10.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.372+0000 7fd7088b9700 1 -- 172.21.15.112:0/3624009309 >> 172.21.15.112:0/3624009309 conn(0x7fd7040d0000 msgr2=0x7fd70415e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:10.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.372+0000 7fd7088b9700 1 -- 172.21.15.112:0/3624009309 shutdown_connections 2023-08-27T11:26:10.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.372+0000 7fd7088b9700 1 -- 172.21.15.112:0/3624009309 wait complete. 2023-08-27T11:26:10.386 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info --human_readable' command ... ok 2023-08-27T11:26:10.386 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_with_human_readable_flag_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info --human_readable' command without subvolume group 2023-08-27T11:26:10.387 INFO:tasks.cephfs_test_runner:test_volume_info_with_human_readable_flag_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) 2023-08-27T11:26:10.387 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup' 2023-08-27T11:26:10.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.505+0000 7fbf1026d700 1 Processor -- start 2023-08-27T11:26:10.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.505+0000 7fbf1026d700 1 -- start start 2023-08-27T11:26:10.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.506+0000 7fbf1026d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf0815d560 0x7fbf080ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:10.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.506+0000 7fbf1026d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf080cd7c0 con 0x7fbf080d14c0 2023-08-27T11:26:10.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.506+0000 7fbf1026d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf080cdaa0 con 0x7fbf080cd240 2023-08-27T11:26:10.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.506+0000 7fbf1026d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080cdd80 con 0x7fbf0815d560 2023-08-27T11:26:10.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.506+0000 7fbf0e009700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbf080d14c0 0x7fbf080d18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55156/0 (socket says 172.21.15.112:55156) 2023-08-27T11:26:10.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.506+0000 7fbf0e009700 1 -- 172.21.15.112:0/2902331944 learned_addr learned my addr 172.21.15.112:0/2902331944 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:10.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d808700 1 --2- 172.21.15.112:0/2902331944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf0815d560 0x7fbf080ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:10.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130537258 0 0) 0x7fbf080cd7c0 con 0x7fbf080d14c0 2023-08-27T11:26:10.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbef40035b0 con 0x7fbf080d14c0 2023-08-27T11:26:10.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3606158215 0 0) 0x7fbef40035b0 con 0x7fbf080d14c0 2023-08-27T11:26:10.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf0815d560 msgr2=0x7fbf080ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:10.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 --2- 172.21.15.112:0/2902331944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf0815d560 0x7fbf080ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:10.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 >> v1:172.21.15.138:6789/0 conn(0x7fbf080cd240 legacy=0x7fbf080ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:10.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf080ce060 con 0x7fbf080d14c0 2023-08-27T11:26:10.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fbef8001670 con 0x7fbf080d14c0 2023-08-27T11:26:10.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 >> v1:172.21.15.112:6789/0 conn(0x7fbf080d14c0 legacy=0x7fbf080d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:10.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 --2- 172.21.15.112:0/2902331944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4003a30 0x7fbef4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:10.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 --2- 172.21.15.112:0/2902331944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf0815d560 0x7fbef4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:10.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 --2- 172.21.15.112:0/2902331944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef4004d90 0x7fbef4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:10.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080cdd80 con 0x7fbef4004d90 2023-08-27T11:26:10.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef400ba10 con 0x7fbef4003a30 2023-08-27T11:26:10.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef400bbd0 con 0x7fbf0815d560 2023-08-27T11:26:10.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0e80a700 1 --2- 172.21.15.112:0/2902331944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef4004d90 0x7fbef4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:10.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0e009700 1 --2- 172.21.15.112:0/2902331944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4003a30 0x7fbef4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:10.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0d808700 1 --2- 172.21.15.112:0/2902331944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf0815d560 0x7fbef4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:10.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0e80a700 1 -- 172.21.15.112:0/2902331944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf0815d560 msgr2=0x7fbef4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:10.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0e80a700 1 --2- 172.21.15.112:0/2902331944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf0815d560 0x7fbef4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:10.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0e80a700 1 -- 172.21.15.112:0/2902331944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4003a30 msgr2=0x7fbef4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:10.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0e80a700 1 --2- 172.21.15.112:0/2902331944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4003a30 0x7fbef4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:10.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.507+0000 7fbf0e80a700 1 -- 172.21.15.112:0/2902331944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40035b0 con 0x7fbef4004d90 2023-08-27T11:26:10.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.508+0000 7fbf0e80a700 1 --2- 172.21.15.112:0/2902331944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef4004d90 0x7fbef4005210 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000c260 tx=0x7fbef4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efee15c9affc30fd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:10.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.508+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00001e70 con 0x7fbef4004d90 2023-08-27T11:26:10.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.508+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0000ce20 con 0x7fbef4004d90 2023-08-27T11:26:10.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.508+0000 7fbf0d007700 1 -- 172.21.15.112:0/2902331944 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00012660 con 0x7fbef4004d90 2023-08-27T11:26:10.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.508+0000 7fbf1026d700 1 -- 172.21.15.112:0/2902331944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef4004d90 msgr2=0x7fbef4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:10.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.508+0000 7fbf1026d700 1 --2- 172.21.15.112:0/2902331944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef4004d90 0x7fbef4005210 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000c260 tx=0x7fbef4005880 comp rx=0 tx=0).stop 2023-08-27T11:26:10.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.508+0000 7fbf0e80a700 1 -- 172.21.15.112:0/2902331944 reap_dead start 2023-08-27T11:26:10.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.508+0000 7fbf1026d700 1 -- 172.21.15.112:0/2902331944 shutdown_connections 2023-08-27T11:26:10.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.508+0000 7fbf1026d700 1 -- 172.21.15.112:0/2902331944 >> 172.21.15.112:0/2902331944 conn(0x7fbf0806b1d0 msgr2=0x7fbf0806b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:10.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.508+0000 7fbf1026d700 1 -- 172.21.15.112:0/2902331944 shutdown_connections 2023-08-27T11:26:10.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.508+0000 7fbf1026d700 1 -- 172.21.15.112:0/2902331944 wait complete. 2023-08-27T11:26:10.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.510+0000 7fbf1026d700 1 Processor -- start 2023-08-27T11:26:10.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.510+0000 7fbf1026d700 1 -- start start 2023-08-27T11:26:10.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf1026d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef4004d90 0x7fbf0814cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:10.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf1026d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf080b3420 0x7fbf080b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:10.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf1026d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf080b3d80 0x7fbf080b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:10.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf1026d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf08156960 con 0x7fbf080b3d80 2023-08-27T11:26:10.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf1026d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080b4750 con 0x7fbf080b3420 2023-08-27T11:26:10.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf1026d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080b6c10 con 0x7fbef4004d90 2023-08-27T11:26:10.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf0e80a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf080b3d80 0x7fbf080b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:10.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf0e80a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf080b3d80 0x7fbf080b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60790/0 (socket says 172.21.15.112:60790) 2023-08-27T11:26:10.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf0e80a700 1 -- 172.21.15.112:0/326087865 learned_addr learned my addr 172.21.15.112:0/326087865 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:10.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf0e009700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef4004d90 0x7fbf0814cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:10.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf0d808700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf080b3420 0x7fbf080b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:10.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf0e80a700 1 -- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef4004d90 msgr2=0x7fbf0814cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:10.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf0e80a700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef4004d90 0x7fbf0814cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:10.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf0e80a700 1 -- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf080b3420 msgr2=0x7fbf080b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:10.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf0e80a700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf080b3420 0x7fbf080b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:10.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf0e80a700 1 -- 172.21.15.112:0/326087865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0815de30 con 0x7fbf080b3d80 2023-08-27T11:26:10.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf0e80a700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf080b3d80 0x7fbf080b4210 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbf00013000 tx=0x7fbf000015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:10.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbefeffd700 1 -- 172.21.15.112:0/326087865 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00003e10 con 0x7fbf080b3d80 2023-08-27T11:26:10.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbefeffd700 1 -- 172.21.15.112:0/326087865 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf000050c0 con 0x7fbf080b3d80 2023-08-27T11:26:10.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbf1026d700 1 -- 172.21.15.112:0/326087865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf080b6650 con 0x7fbf080b3d80 2023-08-27T11:26:10.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.511+0000 7fbefeffd700 1 -- 172.21.15.112:0/326087865 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00005470 con 0x7fbf080b3d80 2023-08-27T11:26:10.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.512+0000 7fbf1026d700 1 -- 172.21.15.112:0/326087865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0815dac0 con 0x7fbf080b3d80 2023-08-27T11:26:10.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.513+0000 7fbf1026d700 1 -- 172.21.15.112:0/326087865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0815dfc0 con 0x7fbf080b3d80 2023-08-27T11:26:10.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.513+0000 7fbefeffd700 1 -- 172.21.15.112:0/326087865 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fbf00023070 con 0x7fbf080b3d80 2023-08-27T11:26:10.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.513+0000 7fbefeffd700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbeec06a3b0 0x7fbeec06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:10.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.513+0000 7fbefeffd700 1 -- 172.21.15.112:0/326087865 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbf00089e30 con 0x7fbf080b3d80 2023-08-27T11:26:10.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.513+0000 7fbf0e009700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbeec06a3b0 0x7fbeec06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:10.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.514+0000 7fbf0e009700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbeec06a3b0 0x7fbeec06c870 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbef8000c00 tx=0x7fbef8000f80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:10.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.518+0000 7fbefeffd700 1 -- 172.21.15.112:0/326087865 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbf00055520 con 0x7fbf080b3d80 2023-08-27T11:26:10.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:10.783+0000 7fbf1026d700 1 -- 172.21.15.112:0/326087865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]} v 0) v1 -- 0x7fbf08067a20 con 0x7fbf080b3d80 2023-08-27T11:26:11.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.370+0000 7fbefeffd700 1 -- 172.21.15.112:0/326087865 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]}]=0 v385) v1 ==== 184+0+0 (secure 0 0 0) 0x7fbf00058b40 con 0x7fbf080b3d80 2023-08-27T11:26:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.373+0000 7fbf1026d700 1 -- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbeec06a3b0 msgr2=0x7fbeec06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.373+0000 7fbf1026d700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbeec06a3b0 0x7fbeec06c870 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbef8000c00 tx=0x7fbef8000f80 comp rx=0 tx=0).stop 2023-08-27T11:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.373+0000 7fbf1026d700 1 -- 172.21.15.112:0/326087865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf080b3d80 msgr2=0x7fbf080b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.373+0000 7fbf1026d700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf080b3d80 0x7fbf080b4210 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbf00013000 tx=0x7fbf000015b0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.373+0000 7fbf1026d700 1 -- 172.21.15.112:0/326087865 shutdown_connections 2023-08-27T11:26:11.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.373+0000 7fbf1026d700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbeec06a3b0 0x7fbeec06c870 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.373+0000 7fbf1026d700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf080b3d80 0x7fbf080b4210 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.373+0000 7fbf1026d700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef4004d90 0x7fbf0814cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.373+0000 7fbf1026d700 1 --2- 172.21.15.112:0/326087865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf080b3420 0x7fbf080b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.373+0000 7fbf1026d700 1 -- 172.21.15.112:0/326087865 >> 172.21.15.112:0/326087865 conn(0x7fbf0806b1d0 msgr2=0x7fbf080d18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:11.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.373+0000 7fbf1026d700 1 -- 172.21.15.112:0/326087865 shutdown_connections 2023-08-27T11:26:11.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.373+0000 7fbf1026d700 1 -- 172.21.15.112:0/326087865 wait complete. 2023-08-27T11:26:11.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:11.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.511+0000 7f435e69e700 1 Processor -- start 2023-08-27T11:26:11.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.511+0000 7f435e69e700 1 -- start start 2023-08-27T11:26:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f435e69e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4358144570 0x7f435815ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f435e69e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f43580b4ad0 0x7f43580b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f435e69e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43580b45f0 con 0x7f43580b3e50 2023-08-27T11:26:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f435e69e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43580cf7a0 con 0x7f43580b4ad0 2023-08-27T11:26:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f435e69e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4358144950 con 0x7f4358144570 2023-08-27T11:26:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f435cc3b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f43580b4ad0 0x7f43580b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:11.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f43577fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f43580b3e50 0x7f435815f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55176/0 (socket says 172.21.15.112:55176) 2023-08-27T11:26:11.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f43577fe700 1 -- 172.21.15.112:0/673704771 learned_addr learned my addr 172.21.15.112:0/673704771 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:11.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f4357fff700 1 --2- 172.21.15.112:0/673704771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4358144570 0x7f435815ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:11.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f435cc3b700 1 -- 172.21.15.112:0/673704771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4358144570 msgr2=0x7f435815ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:11.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f435cc3b700 1 --2- 172.21.15.112:0/673704771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4358144570 0x7f435815ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f435cc3b700 1 -- 172.21.15.112:0/673704771 >> v1:172.21.15.112:6789/0 conn(0x7f43580b3e50 legacy=0x7f435815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:11.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f435cc3b700 1 -- 172.21.15.112:0/673704771 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43580b4310 con 0x7f43580b4ad0 2023-08-27T11:26:11.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f4356ffd700 1 -- 172.21.15.112:0/673704771 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570791068 0 0) 0x7f43580b45f0 con 0x7f43580b3e50 2023-08-27T11:26:11.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f435cc3b700 1 --2- 172.21.15.112:0/673704771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43580b4ad0 0x7f43580b4f80 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4350009dc0 tx=0x7f435000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fe9db55c4ed4bf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:11.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.512+0000 7f4356ffd700 1 -- 172.21.15.112:0/673704771 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435000e070 con 0x7f43580b4ad0 2023-08-27T11:26:11.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.513+0000 7f4356ffd700 1 -- 172.21.15.112:0/673704771 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435000bb90 con 0x7f43580b4ad0 2023-08-27T11:26:11.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.513+0000 7f4356ffd700 1 -- 172.21.15.112:0/673704771 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350012430 con 0x7f43580b4ad0 2023-08-27T11:26:11.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.513+0000 7f435e69e700 1 -- 172.21.15.112:0/673704771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43580b4ad0 msgr2=0x7f43580b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:11.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.513+0000 7f435e69e700 1 --2- 172.21.15.112:0/673704771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43580b4ad0 0x7f43580b4f80 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4350009dc0 tx=0x7f435000b330 comp rx=0 tx=0).stop 2023-08-27T11:26:11.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.513+0000 7f435e69e700 1 -- 172.21.15.112:0/673704771 shutdown_connections 2023-08-27T11:26:11.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.513+0000 7f435e69e700 1 --2- 172.21.15.112:0/673704771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43580b4ad0 0x7f43580b4f80 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.513+0000 7f435e69e700 1 --2- 172.21.15.112:0/673704771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4358144570 0x7f435815ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.513+0000 7f435e69e700 1 -- 172.21.15.112:0/673704771 >> 172.21.15.112:0/673704771 conn(0x7f435806f970 msgr2=0x7f435806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:11.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.513+0000 7f435e69e700 1 -- 172.21.15.112:0/673704771 shutdown_connections 2023-08-27T11:26:11.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.513+0000 7f435e69e700 1 -- 172.21.15.112:0/673704771 wait complete. 2023-08-27T11:26:11.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.515+0000 7f435e69e700 1 Processor -- start 2023-08-27T11:26:11.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.515+0000 7f435e69e700 1 -- start start 2023-08-27T11:26:11.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f435e69e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43580b3e50 0x7f435815e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:11.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f435e69e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43580b4ad0 0x7f435815a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:11.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f435e69e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4358144570 0x7f435813f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:11.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f435e69e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435815dcf0 con 0x7f43580b4ad0 2023-08-27T11:26:11.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f435e69e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435815cc50 con 0x7f4358144570 2023-08-27T11:26:11.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f435e69e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435815cf30 con 0x7f43580b3e50 2023-08-27T11:26:11.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f43577fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43580b4ad0 0x7f435815a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:11.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f43577fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43580b4ad0 0x7f435815a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60800/0 (socket says 172.21.15.112:60800) 2023-08-27T11:26:11.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f43577fe700 1 -- 172.21.15.112:0/3769560043 learned_addr learned my addr 172.21.15.112:0/3769560043 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:11.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f4357fff700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43580b3e50 0x7f435815e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:11.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f43577fe700 1 -- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43580b3e50 msgr2=0x7f435815e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:11.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f43577fe700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43580b3e50 0x7f435815e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f43577fe700 1 -- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4358144570 msgr2=0x7f435813f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:11.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f43577fe700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4358144570 0x7f435813f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f43577fe700 1 -- 172.21.15.112:0/3769560043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4350009a70 con 0x7f43580b4ad0 2023-08-27T11:26:11.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f4357fff700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43580b3e50 0x7f435815e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:11.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f43577fe700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43580b4ad0 0x7f435815a530 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f43480011e0 tx=0x7f434800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:11.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f4354ff9700 1 -- 172.21.15.112:0/3769560043 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348004c80 con 0x7f43580b4ad0 2023-08-27T11:26:11.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f4354ff9700 1 -- 172.21.15.112:0/3769560043 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4348004e00 con 0x7f43580b4ad0 2023-08-27T11:26:11.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f435e69e700 1 -- 172.21.15.112:0/3769560043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435815d210 con 0x7f43580b4ad0 2023-08-27T11:26:11.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f4354ff9700 1 -- 172.21.15.112:0/3769560043 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348007310 con 0x7f43580b4ad0 2023-08-27T11:26:11.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.516+0000 7f435e69e700 1 -- 172.21.15.112:0/3769560043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435815d7d0 con 0x7f43580b4ad0 2023-08-27T11:26:11.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.517+0000 7f435e69e700 1 -- 172.21.15.112:0/3769560043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435815d9c0 con 0x7f43580b4ad0 2023-08-27T11:26:11.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.518+0000 7f4354ff9700 1 -- 172.21.15.112:0/3769560043 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f4348007470 con 0x7f43580b4ad0 2023-08-27T11:26:11.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.518+0000 7f4354ff9700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f434406a3b0 0x7f434406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:11.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.518+0000 7f4354ff9700 1 -- 172.21.15.112:0/3769560043 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4348088560 con 0x7f43580b4ad0 2023-08-27T11:26:11.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.518+0000 7f4357fff700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f434406a3b0 0x7f434406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:11.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.519+0000 7f4357fff700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f434406a3b0 0x7f434406c870 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f434c006fd0 tx=0x7f434c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:11.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.522+0000 7f4354ff9700 1 -- 172.21.15.112:0/3769560043 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4348053a90 con 0x7f43580b4ad0 2023-08-27T11:26:11.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.716+0000 7f435e69e700 1 -- 172.21.15.112:0/3769560043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4358067a20 con 0x7f43580b4ad0 2023-08-27T11:26:11.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.717+0000 7f4354ff9700 1 -- 172.21.15.112:0/3769560043 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+14737 (secure 0 0 0) 0x7f43480570b0 con 0x7f43580b4ad0 2023-08-27T11:26:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:11.719 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":82,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:09.360653+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/2318724344":"2023-08-28T11:26:06.205314+0000","172.21.15.112:6833/2318724344":"2023-08-28T11:26:06.205314+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:11.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.719+0000 7f435e69e700 1 -- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f434406a3b0 msgr2=0x7f434406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:11.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.719+0000 7f435e69e700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f434406a3b0 0x7f434406c870 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f434c006fd0 tx=0x7f434c008040 comp rx=0 tx=0).stop 2023-08-27T11:26:11.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.719+0000 7f435e69e700 1 -- 172.21.15.112:0/3769560043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43580b4ad0 msgr2=0x7f435815a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:11.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.719+0000 7f435e69e700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43580b4ad0 0x7f435815a530 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f43480011e0 tx=0x7f434800cae0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.719+0000 7f435e69e700 1 -- 172.21.15.112:0/3769560043 shutdown_connections 2023-08-27T11:26:11.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.719+0000 7f435e69e700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f434406a3b0 0x7f434406c870 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.719+0000 7f435e69e700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4358144570 0x7f435813f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.719+0000 7f435e69e700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43580b3e50 0x7f435815e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.719+0000 7f435e69e700 1 --2- 172.21.15.112:0/3769560043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43580b4ad0 0x7f435815a530 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.719+0000 7f435e69e700 1 -- 172.21.15.112:0/3769560043 >> 172.21.15.112:0/3769560043 conn(0x7f435806f970 msgr2=0x7f4358143260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:11.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.719+0000 7f435e69e700 1 -- 172.21.15.112:0/3769560043 shutdown_connections 2023-08-27T11:26:11.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.719+0000 7f435e69e700 1 -- 172.21.15.112:0/3769560043 wait complete. 2023-08-27T11:26:11.732 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:26:11.733 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:26:11.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.854+0000 7f3e03e6f700 1 Processor -- start 2023-08-27T11:26:11.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.854+0000 7f3e03e6f700 1 -- start start 2023-08-27T11:26:11.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.855+0000 7f3e03e6f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3dfc1445b0 0x7f3dfc0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:11.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.855+0000 7f3e03e6f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dfc0ccad0 0x7f3dfc15b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:11.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.855+0000 7f3e03e6f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dfc0b08e0 con 0x7f3dfc06f790 2023-08-27T11:26:11.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.855+0000 7f3e03e6f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dfc0ccf80 con 0x7f3dfc1445b0 2023-08-27T11:26:11.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.855+0000 7f3e03e6f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc15bc70 con 0x7f3dfc0ccad0 2023-08-27T11:26:11.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.855+0000 7f3e01c0b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3dfc1445b0 0x7f3dfc0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:11.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.855+0000 7f3e01c0b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3dfc1445b0 0x7f3dfc0ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60806/0 (socket says 172.21.15.112:60806) 2023-08-27T11:26:11.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.855+0000 7f3e01c0b700 1 -- 172.21.15.112:0/762879380 learned_addr learned my addr 172.21.15.112:0/762879380 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:11.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.855+0000 7f3e0140a700 1 --2- 172.21.15.112:0/762879380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dfc0ccad0 0x7f3dfc15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:11.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.855+0000 7f3e01c0b700 1 -- 172.21.15.112:0/762879380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dfc0ccad0 msgr2=0x7f3dfc15b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:11.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.855+0000 7f3e01c0b700 1 --2- 172.21.15.112:0/762879380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dfc0ccad0 0x7f3dfc15b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.855+0000 7f3e01c0b700 1 -- 172.21.15.112:0/762879380 >> v1:172.21.15.138:6789/0 conn(0x7f3dfc06f790 legacy=0x7f3dfc15ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:11.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.856+0000 7f3e01c0b700 1 -- 172.21.15.112:0/762879380 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc0b0b30 con 0x7f3dfc1445b0 2023-08-27T11:26:11.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.856+0000 7f3e01c0b700 1 --2- 172.21.15.112:0/762879380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc1445b0 0x7f3dfc0ce210 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3dec0098f0 tx=0x7f3dec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52fd86adb971d91 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:11.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.856+0000 7f3e00c09700 1 -- 172.21.15.112:0/762879380 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec004230 con 0x7f3dfc1445b0 2023-08-27T11:26:11.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.856+0000 7f3e00c09700 1 -- 172.21.15.112:0/762879380 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dec004390 con 0x7f3dfc1445b0 2023-08-27T11:26:11.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.856+0000 7f3e00c09700 1 -- 172.21.15.112:0/762879380 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec0036a0 con 0x7f3dfc1445b0 2023-08-27T11:26:11.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.856+0000 7f3e03e6f700 1 -- 172.21.15.112:0/762879380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc1445b0 msgr2=0x7f3dfc0ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:11.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.856+0000 7f3e03e6f700 1 --2- 172.21.15.112:0/762879380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc1445b0 0x7f3dfc0ce210 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3dec0098f0 tx=0x7f3dec009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:11.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.856+0000 7f3e03e6f700 1 -- 172.21.15.112:0/762879380 shutdown_connections 2023-08-27T11:26:11.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.856+0000 7f3e03e6f700 1 --2- 172.21.15.112:0/762879380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc1445b0 0x7f3dfc0ce210 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.856+0000 7f3e03e6f700 1 --2- 172.21.15.112:0/762879380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dfc0ccad0 0x7f3dfc15b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.856+0000 7f3e03e6f700 1 -- 172.21.15.112:0/762879380 >> 172.21.15.112:0/762879380 conn(0x7f3dfc0b1e90 msgr2=0x7f3dfc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.856+0000 7f3e03e6f700 1 -- 172.21.15.112:0/762879380 shutdown_connections 2023-08-27T11:26:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.856+0000 7f3e03e6f700 1 -- 172.21.15.112:0/762879380 wait complete. 2023-08-27T11:26:11.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.858+0000 7f3e03e6f700 1 Processor -- start 2023-08-27T11:26:11.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.858+0000 7f3e03e6f700 1 -- start start 2023-08-27T11:26:11.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.858+0000 7f3e03e6f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dfc06f790 0x7f3dfc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:11.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.858+0000 7f3e03e6f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dfc0ccad0 0x7f3dfc1f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:11.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.858+0000 7f3e03e6f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc1445b0 0x7f3dfc1422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:11.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.858+0000 7f3e03e6f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc0af370 con 0x7f3dfc1445b0 2023-08-27T11:26:11.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.858+0000 7f3e03e6f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc1f3780 con 0x7f3dfc06f790 2023-08-27T11:26:11.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.858+0000 7f3e03e6f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc1f2ca0 con 0x7f3dfc0ccad0 2023-08-27T11:26:11.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.858+0000 7f3e0240c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc1445b0 0x7f3dfc1422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:11.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.858+0000 7f3e0240c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc1445b0 0x7f3dfc1422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60816/0 (socket says 172.21.15.112:60816) 2023-08-27T11:26:11.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.858+0000 7f3e0240c700 1 -- 172.21.15.112:0/3122508273 learned_addr learned my addr 172.21.15.112:0/3122508273 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:11.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3e0140a700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dfc0ccad0 0x7f3dfc1f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:11.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3e0240c700 1 -- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dfc0ccad0 msgr2=0x7f3dfc1f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:11.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3e0240c700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dfc0ccad0 0x7f3dfc1f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3e01c0b700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dfc06f790 0x7f3dfc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:11.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3e0240c700 1 -- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dfc06f790 msgr2=0x7f3dfc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:11.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3e0240c700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dfc06f790 0x7f3dfc0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:11.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3e0240c700 1 -- 172.21.15.112:0/3122508273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec009550 con 0x7f3dfc1445b0 2023-08-27T11:26:11.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3e0140a700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dfc0ccad0 0x7f3dfc1f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:11.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3e01c0b700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dfc06f790 0x7f3dfc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:11.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3e0240c700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc1445b0 0x7f3dfc1422d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3df400b9d0 tx=0x7f3df400bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:11.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3df27fc700 1 -- 172.21.15.112:0/3122508273 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4003c00 con 0x7f3dfc1445b0 2023-08-27T11:26:11.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3df27fc700 1 -- 172.21.15.112:0/3122508273 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df4003d60 con 0x7f3dfc1445b0 2023-08-27T11:26:11.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3e03e6f700 1 -- 172.21.15.112:0/3122508273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dfc0c96a0 con 0x7f3dfc1445b0 2023-08-27T11:26:11.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3df27fc700 1 -- 172.21.15.112:0/3122508273 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df400c990 con 0x7f3dfc1445b0 2023-08-27T11:26:11.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.859+0000 7f3e03e6f700 1 -- 172.21.15.112:0/3122508273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dfc1f31c0 con 0x7f3dfc1445b0 2023-08-27T11:26:11.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.860+0000 7f3e03e6f700 1 -- 172.21.15.112:0/3122508273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dfc067e90 con 0x7f3dfc1445b0 2023-08-27T11:26:11.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.860+0000 7f3df27fc700 1 -- 172.21.15.112:0/3122508273 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f3df400caf0 con 0x7f3dfc1445b0 2023-08-27T11:26:11.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.861+0000 7f3df27fc700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3de006a3b0 0x7f3de006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:11.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.861+0000 7f3df27fc700 1 -- 172.21.15.112:0/3122508273 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3df4088240 con 0x7f3dfc1445b0 2023-08-27T11:26:11.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.861+0000 7f3e01c0b700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3de006a3b0 0x7f3de006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:11.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.861+0000 7f3e01c0b700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3de006a3b0 0x7f3de006c870 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0704a0 tx=0x7f3dec008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:11.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:11.865+0000 7f3df27fc700 1 -- 172.21.15.112:0/3122508273 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3df4053890 con 0x7f3dfc1445b0 2023-08-27T11:26:12.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.054+0000 7f3e03e6f700 1 -- 172.21.15.112:0/3122508273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3dfc067a20 con 0x7f3dfc1445b0 2023-08-27T11:26:12.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.054+0000 7f3df27fc700 1 -- 172.21.15.112:0/3122508273 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v82) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3df4056eb0 con 0x7f3dfc1445b0 2023-08-27T11:26:12.056 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:26:12.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.056+0000 7f3e03e6f700 1 -- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3de006a3b0 msgr2=0x7f3de006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:12.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.056+0000 7f3e03e6f700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3de006a3b0 0x7f3de006c870 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0704a0 tx=0x7f3dec008030 comp rx=0 tx=0).stop 2023-08-27T11:26:12.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.056+0000 7f3e03e6f700 1 -- 172.21.15.112:0/3122508273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc1445b0 msgr2=0x7f3dfc1422d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:12.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.056+0000 7f3e03e6f700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc1445b0 0x7f3dfc1422d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3df400b9d0 tx=0x7f3df400bd90 comp rx=0 tx=0).stop 2023-08-27T11:26:12.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.056+0000 7f3e03e6f700 1 -- 172.21.15.112:0/3122508273 shutdown_connections 2023-08-27T11:26:12.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.056+0000 7f3e03e6f700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3de006a3b0 0x7f3de006c870 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.056+0000 7f3e03e6f700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc1445b0 0x7f3dfc1422d0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.056+0000 7f3e03e6f700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dfc06f790 0x7f3dfc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.056+0000 7f3e03e6f700 1 --2- 172.21.15.112:0/3122508273 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dfc0ccad0 0x7f3dfc1f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.056+0000 7f3e03e6f700 1 -- 172.21.15.112:0/3122508273 >> 172.21.15.112:0/3122508273 conn(0x7f3dfc0b1e90 msgr2=0x7f3dfc06dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.056+0000 7f3e03e6f700 1 -- 172.21.15.112:0/3122508273 shutdown_connections 2023-08-27T11:26:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.056+0000 7f3e03e6f700 1 -- 172.21.15.112:0/3122508273 wait complete. 2023-08-27T11:26:12.070 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:26:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.191+0000 7f452cce2700 1 Processor -- start 2023-08-27T11:26:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.191+0000 7f452cce2700 1 -- start start 2023-08-27T11:26:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.192+0000 7f452cce2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f45280723f0 0x7f452806ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.192+0000 7f452cce2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f452806e2f0 0x7f45280af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.192+0000 7f452cce2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45280b16e0 con 0x7f4528143390 2023-08-27T11:26:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.192+0000 7f452cce2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4528070000 con 0x7f452806e2f0 2023-08-27T11:26:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.192+0000 7f452cce2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452806e760 con 0x7f45280723f0 2023-08-27T11:26:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.192+0000 7f4526d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f452806e2f0 0x7f45280af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.192+0000 7f4526d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f452806e2f0 0x7f45280af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60824/0 (socket says 172.21.15.112:60824) 2023-08-27T11:26:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.192+0000 7f4526d9d700 1 -- 172.21.15.112:0/4069670079 learned_addr learned my addr 172.21.15.112:0/4069670079 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.192+0000 7f452659c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4528143390 0x7f452806f820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55202/0 (socket says 172.21.15.112:55202) 2023-08-27T11:26:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.193+0000 7f4525d9b700 1 --2- 172.21.15.112:0/4069670079 >> v2:172.21.15.138:3300/0 conn(0x7f45280723f0 0x7f452806ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.193+0000 7f4526d9d700 1 -- 172.21.15.112:0/4069670079 >> v2:172.21.15.138:3300/0 conn(0x7f45280723f0 msgr2=0x7f452806ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.193+0000 7f4526d9d700 1 --2- 172.21.15.112:0/4069670079 >> v2:172.21.15.138:3300/0 conn(0x7f45280723f0 0x7f452806ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.193+0000 7f4526d9d700 1 -- 172.21.15.112:0/4069670079 >> v1:172.21.15.112:6789/0 conn(0x7f4528143390 legacy=0x7f452806f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:12.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.193+0000 7f4526d9d700 1 -- 172.21.15.112:0/4069670079 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528143770 con 0x7f452806e2f0 2023-08-27T11:26:12.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.193+0000 7f452559a700 1 -- 172.21.15.112:0/4069670079 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120572789 0 0) 0x7f45280b16e0 con 0x7f4528143390 2023-08-27T11:26:12.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.193+0000 7f4526d9d700 1 --2- 172.21.15.112:0/4069670079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452806e2f0 0x7f45280af000 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f45280b09c0 tx=0x7f452000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e026966a8a2efae server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:12.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.193+0000 7f452559a700 1 -- 172.21.15.112:0/4069670079 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452000e070 con 0x7f452806e2f0 2023-08-27T11:26:12.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.193+0000 7f452559a700 1 -- 172.21.15.112:0/4069670079 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452000b9c0 con 0x7f452806e2f0 2023-08-27T11:26:12.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.193+0000 7f452559a700 1 -- 172.21.15.112:0/4069670079 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452000bd70 con 0x7f452806e2f0 2023-08-27T11:26:12.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.193+0000 7f452cce2700 1 -- 172.21.15.112:0/4069670079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452806e2f0 msgr2=0x7f45280af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:12.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.193+0000 7f452cce2700 1 --2- 172.21.15.112:0/4069670079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452806e2f0 0x7f45280af000 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f45280b09c0 tx=0x7f452000b330 comp rx=0 tx=0).stop 2023-08-27T11:26:12.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.194+0000 7f452cce2700 1 -- 172.21.15.112:0/4069670079 shutdown_connections 2023-08-27T11:26:12.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.194+0000 7f452cce2700 1 --2- 172.21.15.112:0/4069670079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452806e2f0 0x7f45280af000 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.194+0000 7f452cce2700 1 --2- 172.21.15.112:0/4069670079 >> v2:172.21.15.138:3300/0 conn(0x7f45280723f0 0x7f452806ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.194+0000 7f452cce2700 1 -- 172.21.15.112:0/4069670079 >> 172.21.15.112:0/4069670079 conn(0x7f45280cc240 msgr2=0x7f45280cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:12.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.194+0000 7f452cce2700 1 -- 172.21.15.112:0/4069670079 shutdown_connections 2023-08-27T11:26:12.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.194+0000 7f452cce2700 1 -- 172.21.15.112:0/4069670079 wait complete. 2023-08-27T11:26:12.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.195+0000 7f452cce2700 1 Processor -- start 2023-08-27T11:26:12.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.195+0000 7f452cce2700 1 -- start start 2023-08-27T11:26:12.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f452cce2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f452806e2f0 0x7f4528135c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:12.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f452cce2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45280723f0 0x7f452815b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:12.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f452cce2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528143390 0x7f4528138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:12.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f452cce2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528152580 con 0x7f4528143390 2023-08-27T11:26:12.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f452cce2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528156000 con 0x7f452806e2f0 2023-08-27T11:26:12.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f452cce2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45281562e0 con 0x7f45280723f0 2023-08-27T11:26:12.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f4526d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528143390 0x7f4528138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:12.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f4526d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528143390 0x7f4528138660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60830/0 (socket says 172.21.15.112:60830) 2023-08-27T11:26:12.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f4526d9d700 1 -- 172.21.15.112:0/2106776107 learned_addr learned my addr 172.21.15.112:0/2106776107 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:12.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f4525d9b700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45280723f0 0x7f452815b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:12.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f452659c700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f452806e2f0 0x7f4528135c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:12.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f4526d9d700 1 -- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45280723f0 msgr2=0x7f452815b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:12.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f4526d9d700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45280723f0 0x7f452815b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f4526d9d700 1 -- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f452806e2f0 msgr2=0x7f4528135c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:12.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f4526d9d700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f452806e2f0 0x7f4528135c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f4526d9d700 1 -- 172.21.15.112:0/2106776107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4520009b30 con 0x7f4528143390 2023-08-27T11:26:12.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f452659c700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f452806e2f0 0x7f4528135c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:12.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f4526d9d700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528143390 0x7f4528138660 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f452000b370 tx=0x7f4520004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:12.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f4516ffd700 1 -- 172.21.15.112:0/2106776107 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4520004930 con 0x7f4528143390 2023-08-27T11:26:12.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f4516ffd700 1 -- 172.21.15.112:0/2106776107 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45200092e0 con 0x7f4528143390 2023-08-27T11:26:12.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f452cce2700 1 -- 172.21.15.112:0/2106776107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45281565c0 con 0x7f4528143390 2023-08-27T11:26:12.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f4516ffd700 1 -- 172.21.15.112:0/2106776107 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452001a700 con 0x7f4528143390 2023-08-27T11:26:12.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.196+0000 7f452cce2700 1 -- 172.21.15.112:0/2106776107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45280cdfa0 con 0x7f4528143390 2023-08-27T11:26:12.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.197+0000 7f452cce2700 1 -- 172.21.15.112:0/2106776107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45280cdb20 con 0x7f4528143390 2023-08-27T11:26:12.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.198+0000 7f4516ffd700 1 -- 172.21.15.112:0/2106776107 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f452001a860 con 0x7f4528143390 2023-08-27T11:26:12.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.198+0000 7f4516ffd700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f450806a3b0 0x7f450806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:12.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.198+0000 7f4516ffd700 1 -- 172.21.15.112:0/2106776107 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f452008bd90 con 0x7f4528143390 2023-08-27T11:26:12.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.198+0000 7f452659c700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f450806a3b0 0x7f450806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:12.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.199+0000 7f452659c700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f450806a3b0 0x7f450806c870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f451c000f60 tx=0x7f451c007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:12.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.203+0000 7f4516ffd700 1 -- 172.21.15.112:0/2106776107 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f45280cdb20 con 0x7f4528143390 2023-08-27T11:26:12.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.392+0000 7f452cce2700 1 -- 172.21.15.112:0/2106776107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45280cdb20 con 0x7f4528143390 2023-08-27T11:26:12.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.393+0000 7f4516ffd700 1 -- 172.21.15.112:0/2106776107 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45280cdb20 con 0x7f4528143390 2023-08-27T11:26:12.395 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:26:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.395+0000 7f452cce2700 1 -- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f450806a3b0 msgr2=0x7f450806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.395+0000 7f452cce2700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f450806a3b0 0x7f450806c870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f451c000f60 tx=0x7f451c007d50 comp rx=0 tx=0).stop 2023-08-27T11:26:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.395+0000 7f452cce2700 1 -- 172.21.15.112:0/2106776107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528143390 msgr2=0x7f4528138660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:12.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.395+0000 7f452cce2700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528143390 0x7f4528138660 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f452000b370 tx=0x7f4520004ad0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.395+0000 7f452cce2700 1 -- 172.21.15.112:0/2106776107 shutdown_connections 2023-08-27T11:26:12.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.395+0000 7f452cce2700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f450806a3b0 0x7f450806c870 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.395+0000 7f452cce2700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528143390 0x7f4528138660 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.395+0000 7f452cce2700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f452806e2f0 0x7f4528135c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.395+0000 7f452cce2700 1 --2- 172.21.15.112:0/2106776107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45280723f0 0x7f452815b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.395+0000 7f452cce2700 1 -- 172.21.15.112:0/2106776107 >> 172.21.15.112:0/2106776107 conn(0x7f45280cc240 msgr2=0x7f45280cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.395+0000 7f452cce2700 1 -- 172.21.15.112:0/2106776107 shutdown_connections 2023-08-27T11:26:12.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.395+0000 7f452cce2700 1 -- 172.21.15.112:0/2106776107 wait complete. 2023-08-27T11:26:12.409 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:26:12.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.531+0000 7f34b7cbd700 1 Processor -- start 2023-08-27T11:26:12.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.531+0000 7f34b7cbd700 1 -- start start 2023-08-27T11:26:12.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b7cbd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b00b4d00 0x7f34b006b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:12.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b7cbd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f34b006b880 0x7f34b015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:12.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b7cbd700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b015b8d0 con 0x7f34b015ae00 2023-08-27T11:26:12.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b7cbd700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b006bd00 con 0x7f34b006b880 2023-08-27T11:26:12.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b7cbd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b015b680 con 0x7f34b00b4d00 2023-08-27T11:26:12.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b5258700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f34b006b880 0x7f34b015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:12.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b5258700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f34b006b880 0x7f34b015a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60834/0 (socket says 172.21.15.112:60834) 2023-08-27T11:26:12.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b5258700 1 -- 172.21.15.112:0/1845723311 learned_addr learned my addr 172.21.15.112:0/1845723311 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:12.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b5a59700 1 --2- 172.21.15.112:0/1845723311 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b00b4d00 0x7f34b006b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:12.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b5258700 1 -- 172.21.15.112:0/1845723311 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b00b4d00 msgr2=0x7f34b006b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:12.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b5258700 1 --2- 172.21.15.112:0/1845723311 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b00b4d00 0x7f34b006b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b5258700 1 -- 172.21.15.112:0/1845723311 >> v1:172.21.15.138:6789/0 conn(0x7f34b015ae00 legacy=0x7f34b006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:12.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b5258700 1 -- 172.21.15.112:0/1845723311 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b015ec10 con 0x7f34b006b880 2023-08-27T11:26:12.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.532+0000 7f34b5258700 1 --2- 172.21.15.112:0/1845723311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34b006b880 0x7f34b015a8c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f34a00098f0 tx=0x7f34a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae42df6e97c01d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.533+0000 7f34b4a57700 1 -- 172.21.15.112:0/1845723311 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0004230 con 0x7f34b006b880 2023-08-27T11:26:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.533+0000 7f34b4a57700 1 -- 172.21.15.112:0/1845723311 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a0004390 con 0x7f34b006b880 2023-08-27T11:26:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.533+0000 7f34b4a57700 1 -- 172.21.15.112:0/1845723311 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a00036a0 con 0x7f34b006b880 2023-08-27T11:26:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.533+0000 7f34b7cbd700 1 -- 172.21.15.112:0/1845723311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34b006b880 msgr2=0x7f34b015a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.533+0000 7f34b7cbd700 1 --2- 172.21.15.112:0/1845723311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34b006b880 0x7f34b015a8c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f34a00098f0 tx=0x7f34a0009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.533+0000 7f34b7cbd700 1 -- 172.21.15.112:0/1845723311 shutdown_connections 2023-08-27T11:26:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.533+0000 7f34b7cbd700 1 --2- 172.21.15.112:0/1845723311 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b00b4d00 0x7f34b006b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.533+0000 7f34b7cbd700 1 --2- 172.21.15.112:0/1845723311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34b006b880 0x7f34b015a8c0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.533+0000 7f34b7cbd700 1 -- 172.21.15.112:0/1845723311 >> 172.21.15.112:0/1845723311 conn(0x7f34b00d03e0 msgr2=0x7f34b00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.533+0000 7f34b7cbd700 1 -- 172.21.15.112:0/1845723311 shutdown_connections 2023-08-27T11:26:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.533+0000 7f34b7cbd700 1 -- 172.21.15.112:0/1845723311 wait complete. 2023-08-27T11:26:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.534+0000 7f34b7cbd700 1 Processor -- start 2023-08-27T11:26:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.534+0000 7f34b7cbd700 1 -- start start 2023-08-27T11:26:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.535+0000 7f34b7cbd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b006b880 0x7f34b00c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.535+0000 7f34b7cbd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34b00b4d00 0x7f34b0141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.535+0000 7f34b7cbd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34b015ae00 0x7f34b0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.535+0000 7f34b7cbd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b00c9e10 con 0x7f34b015ae00 2023-08-27T11:26:12.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.535+0000 7f34b7cbd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b01422d0 con 0x7f34b00b4d00 2023-08-27T11:26:12.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.535+0000 7f34b7cbd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b00c8830 con 0x7f34b006b880 2023-08-27T11:26:12.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.535+0000 7f34b625a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34b015ae00 0x7f34b0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:12.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.535+0000 7f34b625a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34b015ae00 0x7f34b0071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60842/0 (socket says 172.21.15.112:60842) 2023-08-27T11:26:12.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.535+0000 7f34b625a700 1 -- 172.21.15.112:0/3392654530 learned_addr learned my addr 172.21.15.112:0/3392654530 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.535+0000 7f34b5a59700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b006b880 0x7f34b00c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.535+0000 7f34b5258700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34b00b4d00 0x7f34b0141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.536+0000 7f34b625a700 1 -- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b006b880 msgr2=0x7f34b00c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.536+0000 7f34b625a700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b006b880 0x7f34b00c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.536+0000 7f34b625a700 1 -- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34b00b4d00 msgr2=0x7f34b0141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:12.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.536+0000 7f34b625a700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34b00b4d00 0x7f34b0141d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.536+0000 7f34b625a700 1 -- 172.21.15.112:0/3392654530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a0009550 con 0x7f34b015ae00 2023-08-27T11:26:12.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.536+0000 7f34b625a700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34b015ae00 0x7f34b0071a80 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f34a800db90 tx=0x7f34a800dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:12.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.536+0000 7f34a67fc700 1 -- 172.21.15.112:0/3392654530 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a800e6f0 con 0x7f34b015ae00 2023-08-27T11:26:12.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.536+0000 7f34a67fc700 1 -- 172.21.15.112:0/3392654530 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a800e850 con 0x7f34b015ae00 2023-08-27T11:26:12.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.536+0000 7f34b7cbd700 1 -- 172.21.15.112:0/3392654530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b00c8b10 con 0x7f34b015ae00 2023-08-27T11:26:12.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.536+0000 7f34a67fc700 1 -- 172.21.15.112:0/3392654530 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a800ec00 con 0x7f34b015ae00 2023-08-27T11:26:12.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.536+0000 7f34b7cbd700 1 -- 172.21.15.112:0/3392654530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b00c90d0 con 0x7f34b015ae00 2023-08-27T11:26:12.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.537+0000 7f34b7cbd700 1 -- 172.21.15.112:0/3392654530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b006e8f0 con 0x7f34b015ae00 2023-08-27T11:26:12.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.537+0000 7f34a67fc700 1 -- 172.21.15.112:0/3392654530 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f34a8009560 con 0x7f34b015ae00 2023-08-27T11:26:12.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.538+0000 7f34a67fc700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f349406a3b0 0x7f349406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:12.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.538+0000 7f34a67fc700 1 -- 172.21.15.112:0/3392654530 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f34a808a220 con 0x7f34b015ae00 2023-08-27T11:26:12.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.538+0000 7f34b5a59700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f349406a3b0 0x7f349406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:12.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.538+0000 7f34b5a59700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f349406a3b0 0x7f349406c870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00ab60 tx=0x7f34ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:12.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.542+0000 7f34a67fc700 1 -- 172.21.15.112:0/3392654530 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f34a8055830 con 0x7f34b015ae00 2023-08-27T11:26:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.730+0000 7f34b7cbd700 1 -- 172.21.15.112:0/3392654530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f34b0067e90 con 0x7f34b015ae00 2023-08-27T11:26:12.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.734+0000 7f34a67fc700 1 -- 172.21.15.112:0/3392654530 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f34a8058e50 con 0x7f34b015ae00 2023-08-27T11:26:12.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.736+0000 7f34b7cbd700 1 -- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f349406a3b0 msgr2=0x7f349406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:12.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.736+0000 7f34b7cbd700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f349406a3b0 0x7f349406c870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00ab60 tx=0x7f34ac008040 comp rx=0 tx=0).stop 2023-08-27T11:26:12.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.736+0000 7f34b7cbd700 1 -- 172.21.15.112:0/3392654530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34b015ae00 msgr2=0x7f34b0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:12.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.736+0000 7f34b7cbd700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34b015ae00 0x7f34b0071a80 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f34a800db90 tx=0x7f34a800dea0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.736+0000 7f34b7cbd700 1 -- 172.21.15.112:0/3392654530 shutdown_connections 2023-08-27T11:26:12.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.736+0000 7f34b7cbd700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f349406a3b0 0x7f349406c870 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.736+0000 7f34b7cbd700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34b015ae00 0x7f34b0071a80 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.736+0000 7f34b7cbd700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b006b880 0x7f34b00c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.736+0000 7f34b7cbd700 1 --2- 172.21.15.112:0/3392654530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34b00b4d00 0x7f34b0141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:12.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.736+0000 7f34b7cbd700 1 -- 172.21.15.112:0/3392654530 >> 172.21.15.112:0/3392654530 conn(0x7f34b00d03e0 msgr2=0x7f34b006d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:12.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.736+0000 7f34b7cbd700 1 -- 172.21.15.112:0/3392654530 shutdown_connections 2023-08-27T11:26:12.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:12.736+0000 7f34b7cbd700 1 -- 172.21.15.112:0/3392654530 wait complete. 2023-08-27T11:26:12.749 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:26:12.750 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:26:12.801 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:26:12.869 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:26:12.870 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:26:12.870 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:12.887 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:26:12.888 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:26:12.889 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:12.908 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:26:12.909 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:26:12.909 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:12.949 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:26:12.949 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:26:12.950 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:12.968 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:26:12.969 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:26:12.969 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:26:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.107+0000 7f764ffa9700 1 Processor -- start 2023-08-27T11:26:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.107+0000 7f764ffa9700 1 -- start start 2023-08-27T11:26:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764ffa9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f76480b4760 0x7f76480b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764ffa9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7648161030 con 0x7f76481466c0 2023-08-27T11:26:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764ffa9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7648161310 con 0x7f764806fea0 2023-08-27T11:26:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764ffa9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76480b57e0 con 0x7f76480b4760 2023-08-27T11:26:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764e546700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f76480b4760 0x7f76480b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764dd45700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f76481466c0 0x7f764806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55228/0 (socket says 172.21.15.112:55228) 2023-08-27T11:26:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764dd45700 1 -- 172.21.15.112:0/3365636974 learned_addr learned my addr 172.21.15.112:0/3365636974 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764e546700 1 -- 172.21.15.112:0/3365636974 >> v1:172.21.15.138:6789/0 conn(0x7f764806fea0 legacy=0x7f76480b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764e546700 1 -- 172.21.15.112:0/3365636974 >> v1:172.21.15.112:6789/0 conn(0x7f76481466c0 legacy=0x7f764806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764e546700 1 -- 172.21.15.112:0/3365636974 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76481615f0 con 0x7f76480b4760 2023-08-27T11:26:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764cd43700 1 -- 172.21.15.112:0/3365636974 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657782182 0 0) 0x7f7648161030 con 0x7f76481466c0 2023-08-27T11:26:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764e546700 1 --2- 172.21.15.112:0/3365636974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76480b4760 0x7f76480b0cc0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7640009d90 tx=0x7f764000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cdfd366a1ec162 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764cd43700 1 -- 172.21.15.112:0/3365636974 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764000e070 con 0x7f76480b4760 2023-08-27T11:26:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764cd43700 1 -- 172.21.15.112:0/3365636974 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764000bbd0 con 0x7f76480b4760 2023-08-27T11:26:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.108+0000 7f764cd43700 1 -- 172.21.15.112:0/3365636974 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640012430 con 0x7f76480b4760 2023-08-27T11:26:13.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.109+0000 7f764ffa9700 1 -- 172.21.15.112:0/3365636974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76480b4760 msgr2=0x7f76480b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:13.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.109+0000 7f764ffa9700 1 --2- 172.21.15.112:0/3365636974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76480b4760 0x7f76480b0cc0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7640009d90 tx=0x7f764000b330 comp rx=0 tx=0).stop 2023-08-27T11:26:13.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.109+0000 7f764ffa9700 1 -- 172.21.15.112:0/3365636974 shutdown_connections 2023-08-27T11:26:13.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.109+0000 7f764ffa9700 1 --2- 172.21.15.112:0/3365636974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76480b4760 0x7f76480b0cc0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:13.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.109+0000 7f764ffa9700 1 -- 172.21.15.112:0/3365636974 >> 172.21.15.112:0/3365636974 conn(0x7f76480cff80 msgr2=0x7f76480d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:13.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.109+0000 7f764ffa9700 1 -- 172.21.15.112:0/3365636974 shutdown_connections 2023-08-27T11:26:13.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.109+0000 7f764ffa9700 1 -- 172.21.15.112:0/3365636974 wait complete. 2023-08-27T11:26:13.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.110+0000 7f764ffa9700 1 Processor -- start 2023-08-27T11:26:13.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.110+0000 7f764ffa9700 1 -- start start 2023-08-27T11:26:13.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764ffa9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764806fea0 0x7f7648145700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:13.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764ffa9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76480b4760 0x7f76481f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:13.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764ffa9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76481466c0 0x7f76481601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:13.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764ffa9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764814eb60 con 0x7f76480b4760 2023-08-27T11:26:13.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764ffa9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764813e210 con 0x7f764806fea0 2023-08-27T11:26:13.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764ffa9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764813e4f0 con 0x7f76481466c0 2023-08-27T11:26:13.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764d544700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76480b4760 0x7f76481f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:13.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764d544700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76480b4760 0x7f76481f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60854/0 (socket says 172.21.15.112:60854) 2023-08-27T11:26:13.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764d544700 1 -- 172.21.15.112:0/3954697253 learned_addr learned my addr 172.21.15.112:0/3954697253 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:13.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764dd45700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764806fea0 0x7f7648145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:13.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764e546700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76481466c0 0x7f76481601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:13.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764d544700 1 -- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76481466c0 msgr2=0x7f76481601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:13.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764d544700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76481466c0 0x7f76481601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:13.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764d544700 1 -- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764806fea0 msgr2=0x7f7648145700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764d544700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764806fea0 0x7f7648145700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764d544700 1 -- 172.21.15.112:0/3954697253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7640009a70 con 0x7f76480b4760 2023-08-27T11:26:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.111+0000 7f764d544700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76480b4760 0x7f76481f23b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f763800c450 tx=0x7f763800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.112+0000 7f763e7fc700 1 -- 172.21.15.112:0/3954697253 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380052d0 con 0x7f76480b4760 2023-08-27T11:26:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.112+0000 7f763e7fc700 1 -- 172.21.15.112:0/3954697253 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7638005430 con 0x7f76480b4760 2023-08-27T11:26:13.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.112+0000 7f764ffa9700 1 -- 172.21.15.112:0/3954697253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764814e7f0 con 0x7f76480b4760 2023-08-27T11:26:13.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.112+0000 7f763e7fc700 1 -- 172.21.15.112:0/3954697253 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380038c0 con 0x7f76480b4760 2023-08-27T11:26:13.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.112+0000 7f764ffa9700 1 -- 172.21.15.112:0/3954697253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7648151440 con 0x7f76480b4760 2023-08-27T11:26:13.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.113+0000 7f764ffa9700 1 -- 172.21.15.112:0/3954697253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7648067a20 con 0x7f76480b4760 2023-08-27T11:26:13.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.113+0000 7f763e7fc700 1 -- 172.21.15.112:0/3954697253 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f7638003a20 con 0x7f76480b4760 2023-08-27T11:26:13.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.113+0000 7f763e7fc700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f762c06a3b0 0x7f762c06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:13.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.113+0000 7f763e7fc700 1 -- 172.21.15.112:0/3954697253 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7638087980 con 0x7f76480b4760 2023-08-27T11:26:13.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.114+0000 7f764dd45700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f762c06a3b0 0x7f762c06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:13.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.114+0000 7f764dd45700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f762c06a3b0 0x7f762c06c870 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7644007fd0 tx=0x7f7644007c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:13.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.118+0000 7f763e7fc700 1 -- 172.21.15.112:0/3954697253 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f76380530b0 con 0x7f76480b4760 2023-08-27T11:26:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.349+0000 7f764ffa9700 1 -- 172.21.15.112:0/3954697253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7648070b20 con 0x7f76480b4760 2023-08-27T11:26:13.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.350+0000 7f763e7fc700 1 -- 172.21.15.112:0/3954697253 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f76380566d0 con 0x7f76480b4760 2023-08-27T11:26:13.352 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:13.352 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6166,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3262556844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3262556844},{"type":"v1","addr":"172.21.15.138:6836","nonce":3262556844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2232724696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2232724696},{"type":"v1","addr":"172.21.15.138:6837","nonce":2232724696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6570,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1511159838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1511159838},{"type":"v1","addr":"172.21.15.112:6834","nonce":1511159838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6957,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2282779316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2282779316},{"type":"v1","addr":"172.21.15.112:6835","nonce":2282779316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[]} 2023-08-27T11:26:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.352+0000 7f764ffa9700 1 -- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f762c06a3b0 msgr2=0x7f762c06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.352+0000 7f764ffa9700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f762c06a3b0 0x7f762c06c870 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7644007fd0 tx=0x7f7644007c20 comp rx=0 tx=0).stop 2023-08-27T11:26:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.352+0000 7f764ffa9700 1 -- 172.21.15.112:0/3954697253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76480b4760 msgr2=0x7f76481f23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.352+0000 7f764ffa9700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76480b4760 0x7f76481f23b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f763800c450 tx=0x7f763800c810 comp rx=0 tx=0).stop 2023-08-27T11:26:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.352+0000 7f764ffa9700 1 -- 172.21.15.112:0/3954697253 shutdown_connections 2023-08-27T11:26:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.352+0000 7f764ffa9700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f762c06a3b0 0x7f762c06c870 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.352+0000 7f764ffa9700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76481466c0 0x7f76481601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.352+0000 7f764ffa9700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764806fea0 0x7f7648145700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.352+0000 7f764ffa9700 1 --2- 172.21.15.112:0/3954697253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76480b4760 0x7f76481f23b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.352+0000 7f764ffa9700 1 -- 172.21.15.112:0/3954697253 >> 172.21.15.112:0/3954697253 conn(0x7f76480cff80 msgr2=0x7f76480b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.352+0000 7f764ffa9700 1 -- 172.21.15.112:0/3954697253 shutdown_connections 2023-08-27T11:26:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.352+0000 7f764ffa9700 1 -- 172.21.15.112:0/3954697253 wait complete. 2023-08-27T11:26:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 51 2023-08-27T11:26:13.366 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:26:13.367 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:26:13.367 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:26:13.367 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:26:13.370 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:26:13.370 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:26:13.370 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:26:13.371 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:26:13.371 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:26:13.371 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:26:13.371 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:26:13.372 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:26:13.372 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:26:13.372 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:26:13.373 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:26:13.373 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:26:13.377 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:26:13.377 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:26:13.381 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:26:13.381 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:26:13.384 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:26:13.385 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:26:13.386 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:26:13.386 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:26:13.423 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:26:13.424 DEBUG:teuthology.parallel:result is None 2023-08-27T11:26:13.424 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:26:13.425 DEBUG:teuthology.parallel:result is None 2023-08-27T11:26:13.428 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:26:13.428 DEBUG:teuthology.parallel:result is None 2023-08-27T11:26:13.429 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:26:13.430 DEBUG:teuthology.parallel:result is None 2023-08-27T11:26:13.430 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:26:13.501 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:26:13.501 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:26:13.512 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2023-08-27T11:26:13.513 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:26:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.556+0000 7f2302d23700 1 Processor -- start 2023-08-27T11:26:13.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.556+0000 7f2302d23700 1 -- start start 2023-08-27T11:26:13.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f2302d23700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22fc0b4760 0x7f22fc0b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:13.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f2302d23700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22fc161030 con 0x7f22fc06fea0 2023-08-27T11:26:13.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f2302d23700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22fc161310 con 0x7f22fc1466c0 2023-08-27T11:26:13.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f2302d23700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc0b57e0 con 0x7f22fc0b4760 2023-08-27T11:26:13.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f22fbfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f22fc06fea0 0x7f22fc0b05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55268/0 (socket says 172.21.15.112:55268) 2023-08-27T11:26:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f22fbfff700 1 -- 172.21.15.112:0/3279765931 learned_addr learned my addr 172.21.15.112:0/3279765931 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f23012c0700 1 --2- 172.21.15.112:0/3279765931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22fc0b4760 0x7f22fc0b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f22fb7fe700 1 -- 172.21.15.112:0/3279765931 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530569636 0 0) 0x7f22fc161030 con 0x7f22fc06fea0 2023-08-27T11:26:13.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f22fb7fe700 1 -- 172.21.15.112:0/3279765931 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22ec0035b0 con 0x7f22fc06fea0 2023-08-27T11:26:13.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f22fb7fe700 1 -- 172.21.15.112:0/3279765931 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351435602 0 0) 0x7f22fc161310 con 0x7f22fc1466c0 2023-08-27T11:26:13.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f22fb7fe700 1 -- 172.21.15.112:0/3279765931 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22fc161030 con 0x7f22fc1466c0 2023-08-27T11:26:13.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f23012c0700 1 -- 172.21.15.112:0/3279765931 >> v1:172.21.15.138:6789/0 conn(0x7f22fc1466c0 legacy=0x7f22fc06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:13.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f23012c0700 1 -- 172.21.15.112:0/3279765931 >> v1:172.21.15.112:6789/0 conn(0x7f22fc06fea0 legacy=0x7f22fc0b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:13.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.557+0000 7f23012c0700 1 -- 172.21.15.112:0/3279765931 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc1615f0 con 0x7f22fc0b4760 2023-08-27T11:26:13.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.558+0000 7f23012c0700 1 --2- 172.21.15.112:0/3279765931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22fc0b4760 0x7f22fc0b0cc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009dc0 tx=0x7f22f400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0b13f4866cb040 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:13.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.558+0000 7f22fb7fe700 1 -- 172.21.15.112:0/3279765931 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f400e070 con 0x7f22fc0b4760 2023-08-27T11:26:13.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.558+0000 7f22fb7fe700 1 -- 172.21.15.112:0/3279765931 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f400bb20 con 0x7f22fc0b4760 2023-08-27T11:26:13.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.558+0000 7f22fb7fe700 1 -- 172.21.15.112:0/3279765931 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f4012430 con 0x7f22fc0b4760 2023-08-27T11:26:13.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.558+0000 7f2302d23700 1 -- 172.21.15.112:0/3279765931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22fc0b4760 msgr2=0x7f22fc0b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:13.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.558+0000 7f2302d23700 1 --2- 172.21.15.112:0/3279765931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22fc0b4760 0x7f22fc0b0cc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009dc0 tx=0x7f22f400b330 comp rx=0 tx=0).stop 2023-08-27T11:26:13.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.558+0000 7f2302d23700 1 -- 172.21.15.112:0/3279765931 shutdown_connections 2023-08-27T11:26:13.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.558+0000 7f2302d23700 1 --2- 172.21.15.112:0/3279765931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22fc0b4760 0x7f22fc0b0cc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:13.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.558+0000 7f2302d23700 1 -- 172.21.15.112:0/3279765931 >> 172.21.15.112:0/3279765931 conn(0x7f22fc0cff80 msgr2=0x7f22fc0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:13.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.558+0000 7f2302d23700 1 -- 172.21.15.112:0/3279765931 shutdown_connections 2023-08-27T11:26:13.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.558+0000 7f2302d23700 1 -- 172.21.15.112:0/3279765931 wait complete. 2023-08-27T11:26:13.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.559+0000 7f2302d23700 1 Processor -- start 2023-08-27T11:26:13.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.559+0000 7f2302d23700 1 -- start start 2023-08-27T11:26:13.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.560+0000 7f2302d23700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22fc06fea0 0x7f22fc13cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:13.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.560+0000 7f2302d23700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22fc0b4760 0x7f22fc1f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:13.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.560+0000 7f2302d23700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22fc1466c0 0x7f22fc1f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:13.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.560+0000 7f2302d23700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc0af5c0 con 0x7f22fc1466c0 2023-08-27T11:26:13.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.560+0000 7f2302d23700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc0af2e0 con 0x7f22fc0b4760 2023-08-27T11:26:13.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.560+0000 7f2302d23700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc13d2c0 con 0x7f22fc06fea0 2023-08-27T11:26:13.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.560+0000 7f23012c0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22fc1466c0 0x7f22fc1f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:13.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.560+0000 7f23012c0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22fc1466c0 0x7f22fc1f6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60894/0 (socket says 172.21.15.112:60894) 2023-08-27T11:26:13.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.560+0000 7f23012c0700 1 -- 172.21.15.112:0/3615220616 learned_addr learned my addr 172.21.15.112:0/3615220616 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:13.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.560+0000 7f22fbfff700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22fc0b4760 0x7f22fc1f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:13.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.560+0000 7f2300abf700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22fc06fea0 0x7f22fc13cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:13.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.561+0000 7f23012c0700 1 -- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22fc06fea0 msgr2=0x7f22fc13cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:13.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.561+0000 7f23012c0700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22fc06fea0 0x7f22fc13cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:13.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.561+0000 7f23012c0700 1 -- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22fc0b4760 msgr2=0x7f22fc1f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:13.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.561+0000 7f23012c0700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22fc0b4760 0x7f22fc1f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:13.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.561+0000 7f23012c0700 1 -- 172.21.15.112:0/3615220616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f4009a70 con 0x7f22fc1466c0 2023-08-27T11:26:13.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.561+0000 7f23012c0700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22fc1466c0 0x7f22fc1f6680 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f22f400b370 tx=0x7f22f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:13.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.561+0000 7f22f97fa700 1 -- 172.21.15.112:0/3615220616 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f401b810 con 0x7f22fc1466c0 2023-08-27T11:26:13.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.561+0000 7f22f97fa700 1 -- 172.21.15.112:0/3615220616 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f4020070 con 0x7f22fc1466c0 2023-08-27T11:26:13.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.561+0000 7f2302d23700 1 -- 172.21.15.112:0/3615220616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22fc13d5a0 con 0x7f22fc1466c0 2023-08-27T11:26:13.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.561+0000 7f22f97fa700 1 -- 172.21.15.112:0/3615220616 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f401bbb0 con 0x7f22fc1466c0 2023-08-27T11:26:13.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.561+0000 7f2302d23700 1 -- 172.21.15.112:0/3615220616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc0aefb0 con 0x7f22fc1466c0 2023-08-27T11:26:13.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.562+0000 7f2302d23700 1 -- 172.21.15.112:0/3615220616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22fc13dac0 con 0x7f22fc1466c0 2023-08-27T11:26:13.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.562+0000 7f22f97fa700 1 -- 172.21.15.112:0/3615220616 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f22f401bd10 con 0x7f22fc1466c0 2023-08-27T11:26:13.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.563+0000 7f22f97fa700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22e006a3b0 0x7f22e006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:13.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.563+0000 7f22f97fa700 1 -- 172.21.15.112:0/3615220616 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f22f4022070 con 0x7f22fc1466c0 2023-08-27T11:26:13.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.563+0000 7f2300abf700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22e006a3b0 0x7f22e006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:13.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.563+0000 7f2300abf700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22e006a3b0 0x7f22e006c870 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f22f0001220 tx=0x7f22f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:13.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.567+0000 7f22f97fa700 1 -- 172.21.15.112:0/3615220616 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f22fc13dac0 con 0x7f22fc1466c0 2023-08-27T11:26:13.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:13.772+0000 7f2302d23700 1 -- 172.21.15.112:0/3615220616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f22fc13dac0 con 0x7f22fc1466c0 2023-08-27T11:26:14.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.388+0000 7f22f97fa700 1 -- 172.21.15.112:0/3615220616 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v84) v1 ==== 97+0+0 (secure 0 0 0) 0x7f22fc13dac0 con 0x7f22fc1466c0 2023-08-27T11:26:14.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.391+0000 7f2302d23700 1 -- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22e006a3b0 msgr2=0x7f22e006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:14.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.391+0000 7f2302d23700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22e006a3b0 0x7f22e006c870 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f22f0001220 tx=0x7f22f0009040 comp rx=0 tx=0).stop 2023-08-27T11:26:14.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.391+0000 7f2302d23700 1 -- 172.21.15.112:0/3615220616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22fc1466c0 msgr2=0x7f22fc1f6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:14.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.391+0000 7f2302d23700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22fc1466c0 0x7f22fc1f6680 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f22f400b370 tx=0x7f22f4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.391+0000 7f2302d23700 1 -- 172.21.15.112:0/3615220616 shutdown_connections 2023-08-27T11:26:14.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.391+0000 7f2302d23700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f22e006a3b0 0x7f22e006c870 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.391+0000 7f2302d23700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22fc1466c0 0x7f22fc1f6680 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.391+0000 7f2302d23700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22fc06fea0 0x7f22fc13cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.391+0000 7f2302d23700 1 --2- 172.21.15.112:0/3615220616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22fc0b4760 0x7f22fc1f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.391+0000 7f2302d23700 1 -- 172.21.15.112:0/3615220616 >> 172.21.15.112:0/3615220616 conn(0x7f22fc0cff80 msgr2=0x7f22fc160400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:14.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.391+0000 7f2302d23700 1 -- 172.21.15.112:0/3615220616 shutdown_connections 2023-08-27T11:26:14.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.391+0000 7f2302d23700 1 -- 172.21.15.112:0/3615220616 wait complete. 2023-08-27T11:26:14.397 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2023-08-27T11:26:14.407 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:26:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.530+0000 7f977db4d700 1 Processor -- start 2023-08-27T11:26:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.530+0000 7f977db4d700 1 -- start start 2023-08-27T11:26:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f977db4d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f977815a120 0x7f97780b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f977db4d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f977806d5b0 0x7f977806da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f977db4d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977806df50 con 0x7f97780b3440 2023-08-27T11:26:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f977db4d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977815b540 con 0x7f977815a120 2023-08-27T11:26:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f977db4d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97780af830 con 0x7f977806d5b0 2023-08-27T11:26:14.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f97777fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f977815a120 0x7f97780b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:14.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f97777fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f977815a120 0x7f97780b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60900/0 (socket says 172.21.15.112:60900) 2023-08-27T11:26:14.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f97777fe700 1 -- 172.21.15.112:0/1327066856 learned_addr learned my addr 172.21.15.112:0/1327066856 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:14.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f9776ffd700 1 --2- 172.21.15.112:0/1327066856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f977806d5b0 0x7f977806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:14.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f97777fe700 1 -- 172.21.15.112:0/1327066856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f977806d5b0 msgr2=0x7f977806da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f97777fe700 1 --2- 172.21.15.112:0/1327066856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f977806d5b0 0x7f977806da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f97777fe700 1 -- 172.21.15.112:0/1327066856 >> v1:172.21.15.138:6789/0 conn(0x7f97780b3440 legacy=0x7f97780b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f97777fe700 1 -- 172.21.15.112:0/1327066856 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97780afb10 con 0x7f977815a120 2023-08-27T11:26:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f97777fe700 1 --2- 172.21.15.112:0/1327066856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977815a120 0x7f97780b05f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f97680098f0 tx=0x7f9768009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8565358bd9aa3132 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f97767fc700 1 -- 172.21.15.112:0/1327066856 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976800b8a0 con 0x7f977815a120 2023-08-27T11:26:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f97767fc700 1 -- 172.21.15.112:0/1327066856 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976800ba00 con 0x7f977815a120 2023-08-27T11:26:14.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.531+0000 7f97767fc700 1 -- 172.21.15.112:0/1327066856 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976800bd90 con 0x7f977815a120 2023-08-27T11:26:14.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.532+0000 7f977db4d700 1 -- 172.21.15.112:0/1327066856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977815a120 msgr2=0x7f97780b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:14.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.532+0000 7f977db4d700 1 --2- 172.21.15.112:0/1327066856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977815a120 0x7f97780b05f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f97680098f0 tx=0x7f9768009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:14.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.532+0000 7f977db4d700 1 -- 172.21.15.112:0/1327066856 shutdown_connections 2023-08-27T11:26:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.532+0000 7f977db4d700 1 --2- 172.21.15.112:0/1327066856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977815a120 0x7f97780b05f0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.532+0000 7f977db4d700 1 --2- 172.21.15.112:0/1327066856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f977806d5b0 0x7f977806da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.532+0000 7f977db4d700 1 -- 172.21.15.112:0/1327066856 >> 172.21.15.112:0/1327066856 conn(0x7f97780cff80 msgr2=0x7f97780d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:14.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.532+0000 7f977db4d700 1 -- 172.21.15.112:0/1327066856 shutdown_connections 2023-08-27T11:26:14.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.532+0000 7f977db4d700 1 -- 172.21.15.112:0/1327066856 wait complete. 2023-08-27T11:26:14.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.533+0000 7f977db4d700 1 Processor -- start 2023-08-27T11:26:14.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.533+0000 7f977db4d700 1 -- start start 2023-08-27T11:26:14.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.534+0000 7f977db4d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f977806d5b0 0x7f9778158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:14.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.534+0000 7f977db4d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97780b3440 0x7f9778141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:14.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.534+0000 7f977db4d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977815a120 0x7f97780c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:14.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.534+0000 7f977db4d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9778158740 con 0x7f977815a120 2023-08-27T11:26:14.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.534+0000 7f977db4d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9778158a20 con 0x7f97780b3440 2023-08-27T11:26:14.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.534+0000 7f977db4d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9778158d00 con 0x7f977806d5b0 2023-08-27T11:26:14.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.534+0000 7f9777fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977815a120 0x7f97780c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:14.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.534+0000 7f9777fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977815a120 0x7f97780c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60910/0 (socket says 172.21.15.112:60910) 2023-08-27T11:26:14.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.534+0000 7f9777fff700 1 -- 172.21.15.112:0/2735103170 learned_addr learned my addr 172.21.15.112:0/2735103170 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:14.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.534+0000 7f9776ffd700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97780b3440 0x7f9778141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:14.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.534+0000 7f97777fe700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f977806d5b0 0x7f9778158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:14.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.534+0000 7f9777fff700 1 -- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f977806d5b0 msgr2=0x7f9778158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:14.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.534+0000 7f9777fff700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f977806d5b0 0x7f9778158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.535+0000 7f9777fff700 1 -- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97780b3440 msgr2=0x7f9778141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:14.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.535+0000 7f9777fff700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97780b3440 0x7f9778141a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.535+0000 7f9777fff700 1 -- 172.21.15.112:0/2735103170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9768009550 con 0x7f977815a120 2023-08-27T11:26:14.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.535+0000 7f9776ffd700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97780b3440 0x7f9778141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:14.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.535+0000 7f9777fff700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977815a120 0x7f97780c5c30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f977000cc50 tx=0x7f977000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:14.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.535+0000 7f975ffff700 1 -- 172.21.15.112:0/2735103170 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977000ad80 con 0x7f977815a120 2023-08-27T11:26:14.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.535+0000 7f975ffff700 1 -- 172.21.15.112:0/2735103170 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977000ce50 con 0x7f977815a120 2023-08-27T11:26:14.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.535+0000 7f977db4d700 1 -- 172.21.15.112:0/2735103170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9778158fe0 con 0x7f977815a120 2023-08-27T11:26:14.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.535+0000 7f975ffff700 1 -- 172.21.15.112:0/2735103170 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770003c40 con 0x7f977815a120 2023-08-27T11:26:14.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.535+0000 7f977db4d700 1 -- 172.21.15.112:0/2735103170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9778141f80 con 0x7f977815a120 2023-08-27T11:26:14.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.536+0000 7f977db4d700 1 -- 172.21.15.112:0/2735103170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9778159500 con 0x7f977815a120 2023-08-27T11:26:14.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.536+0000 7f975ffff700 1 -- 172.21.15.112:0/2735103170 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f97700050d0 con 0x7f977815a120 2023-08-27T11:26:14.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.537+0000 7f975ffff700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f976406a3b0 0x7f976406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:14.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.537+0000 7f975ffff700 1 -- 172.21.15.112:0/2735103170 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9770088820 con 0x7f977815a120 2023-08-27T11:26:14.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.537+0000 7f97777fe700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f976406a3b0 0x7f976406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:14.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.537+0000 7f97777fe700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f976406a3b0 0x7f976406c870 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9768014040 tx=0x7f9768008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:14.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.541+0000 7f975ffff700 1 -- 172.21.15.112:0/2735103170 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9778159500 con 0x7f977815a120 2023-08-27T11:26:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.807+0000 7f977db4d700 1 -- 172.21.15.112:0/2735103170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9778159500 con 0x7f977815a120 2023-08-27T11:26:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.807+0000 7f975ffff700 1 -- 172.21.15.112:0/2735103170 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9778159500 con 0x7f977815a120 2023-08-27T11:26:14.809 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:14.810 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:26:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2023-08-27T11:26:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2023-08-27T11:26:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk2bnkLBAA+vxcTeJzucKk9iSF/Zw3Lg==", 2023-08-27T11:26:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:26:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:26:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.812 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.812 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2023-08-27T11:26:14.812 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk3zD5OBAASVnwG6yn/YQzfYyeMakFpA==", 2023-08-27T11:26:14.813 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.813 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:14.813 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:26:14.813 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:26:14.813 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.814 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.814 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2023-08-27T11:26:14.814 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtki7wJNBAAsIGgDLM82sWNcKnCDlSBMw==", 2023-08-27T11:26:14.815 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.815 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:14.815 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:26:14.815 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:26:14.815 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.816 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.816 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2023-08-27T11:26:14.816 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkCsW/BBAAFTPiTOpBo1L+rLo9px+AjQ==", 2023-08-27T11:26:14.817 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.817 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:14.817 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:26:14.817 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:26:14.818 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.818 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.818 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2023-08-27T11:26:14.819 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB7MOtkR/cSHhAAR8e8VRd+e0ESJkqQDYWbEw==", 2023-08-27T11:26:14.819 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.819 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:14.819 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:14.820 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.820 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.820 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2023-08-27T11:26:14.821 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB+MOtksBF3HRAA1OIXiIkhgPDYpQyYPaWOXg==", 2023-08-27T11:26:14.821 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.821 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:14.821 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:14.821 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.822 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.822 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2023-08-27T11:26:14.822 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCBMOtkWsrdHBAAkfWbiE6wAFzpt9+BxRh8fw==", 2023-08-27T11:26:14.822 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.823 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:14.823 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:14.823 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.823 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.824 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2023-08-27T11:26:14.824 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCEMOtkIkiKHRAARDoh/FH9ZdqUlZZ29Mc/Dw==", 2023-08-27T11:26:14.824 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.824 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:14.825 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:14.825 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.825 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.826 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2023-08-27T11:26:14.826 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCKMOtkPgDzGxAA+U4kbwD0d4lS8aNPG2CgHw==", 2023-08-27T11:26:14.826 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.826 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:14.826 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:14.827 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.827 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.827 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2023-08-27T11:26:14.828 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCNMOtkKomSGxAAGAQUcIH2H/uyZIU8vlq1pw==", 2023-08-27T11:26:14.828 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.828 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:14.828 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:14.828 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.828 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.829 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2023-08-27T11:26:14.829 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCQMOtkHI2MGhAA7sgbi8JXO8P9wnVvMy3PqA==", 2023-08-27T11:26:14.829 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.830 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:14.830 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:14.830 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.830 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.831 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2023-08-27T11:26:14.831 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCTMOtkVHpiGhAA4C4YJMWbQ0mmGgB4i8G/AA==", 2023-08-27T11:26:14.831 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.831 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:14.831 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:14.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.832 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.832 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2023-08-27T11:26:14.832 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA==", 2023-08-27T11:26:14.833 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.833 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:14.833 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2023-08-27T11:26:14.833 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:26:14.833 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.834 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2023-08-27T11:26:14.834 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtk66DbDxAAWZ5StELux0lriJ2gEN9icg==", 2023-08-27T11:26:14.834 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.835 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:14.835 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:26:14.835 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:26:14.835 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:26:14.835 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.836 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.836 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2023-08-27T11:26:14.836 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkViKJDBAAF2ofwO+78er9OyFfn2bO+g==", 2023-08-27T11:26:14.836 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.837 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:14.837 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:26:14.837 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:26:14.837 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:26:14.837 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.838 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.838 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2023-08-27T11:26:14.838 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkwt+QFRAAowWE69VqeRhQz0896rxx9Q==", 2023-08-27T11:26:14.838 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.839 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:14.839 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:26:14.839 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:26:14.839 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:26:14.839 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2023-08-27T11:26:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkjDA+DxAAjmOn82AC2OMpKlAVCD6hlg==", 2023-08-27T11:26:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:26:14.841 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:26:14.841 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:26:14.841 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.841 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.842 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2023-08-27T11:26:14.842 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkXguHKRAALHwzk8diVXqsMvTKr+AzLQ==", 2023-08-27T11:26:14.842 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.842 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:26:14.843 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:26:14.843 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.843 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:14.843 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:14.844 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2023-08-27T11:26:14.844 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkrRjBJRAAq3ftmIh81LKadpliVUiJ6g==", 2023-08-27T11:26:14.844 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:14.844 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:26:14.844 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:26:14.845 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:14.845 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.845 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:14.845 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:26:14.845 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:26:14.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.810+0000 7f977db4d700 1 -- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f976406a3b0 msgr2=0x7f976406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:14.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.810+0000 7f977db4d700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f976406a3b0 0x7f976406c870 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9768014040 tx=0x7f9768008030 comp rx=0 tx=0).stop 2023-08-27T11:26:14.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.810+0000 7f977db4d700 1 -- 172.21.15.112:0/2735103170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977815a120 msgr2=0x7f97780c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:14.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.810+0000 7f977db4d700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977815a120 0x7f97780c5c30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f977000cc50 tx=0x7f977000a340 comp rx=0 tx=0).stop 2023-08-27T11:26:14.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.810+0000 7f977db4d700 1 -- 172.21.15.112:0/2735103170 shutdown_connections 2023-08-27T11:26:14.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.810+0000 7f977db4d700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f976406a3b0 0x7f976406c870 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.810+0000 7f977db4d700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977815a120 0x7f97780c5c30 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.810+0000 7f977db4d700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f977806d5b0 0x7f9778158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.810+0000 7f977db4d700 1 --2- 172.21.15.112:0/2735103170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97780b3440 0x7f9778141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.810+0000 7f977db4d700 1 -- 172.21.15.112:0/2735103170 >> 172.21.15.112:0/2735103170 conn(0x7f97780cff80 msgr2=0x7f97780b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:14.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.810+0000 7f977db4d700 1 -- 172.21.15.112:0/2735103170 shutdown_connections 2023-08-27T11:26:14.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.810+0000 7f977db4d700 1 -- 172.21.15.112:0/2735103170 wait complete. 2023-08-27T11:26:14.852 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:14.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.970+0000 7f69e3f2b700 1 Processor -- start 2023-08-27T11:26:14.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.971+0000 7f69e3f2b700 1 -- start start 2023-08-27T11:26:14.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.971+0000 7f69e3f2b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f69dc163f90 0x7f69dc166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:14.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.971+0000 7f69e3f2b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69dc0b5ae0 con 0x7f69dc0b3e50 2023-08-27T11:26:14.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.971+0000 7f69e3f2b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69dc0b45f0 con 0x7f69dc144570 2023-08-27T11:26:14.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.971+0000 7f69e3f2b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69dc0cf7a0 con 0x7f69dc163f90 2023-08-27T11:26:14.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.972+0000 7f69e24c8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f69dc163f90 0x7f69dc166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.972+0000 7f69e14c6700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f69dc0b3e50 0x7f69dc15f530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55296/0 (socket says 172.21.15.112:55296) 2023-08-27T11:26:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.972+0000 7f69e24c8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f69dc163f90 0x7f69dc166380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60918/0 (socket says 172.21.15.112:60918) 2023-08-27T11:26:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.972+0000 7f69e14c6700 1 -- 172.21.15.112:0/1335159037 learned_addr learned my addr 172.21.15.112:0/1335159037 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.972+0000 7f69e24c8700 1 -- 172.21.15.112:0/1335159037 >> v1:172.21.15.138:6789/0 conn(0x7f69dc144570 legacy=0x7f69dc15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.972+0000 7f69e24c8700 1 -- 172.21.15.112:0/1335159037 >> v1:172.21.15.112:6789/0 conn(0x7f69dc0b3e50 legacy=0x7f69dc15f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:14.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.972+0000 7f69e24c8700 1 -- 172.21.15.112:0/1335159037 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc15fc40 con 0x7f69dc163f90 2023-08-27T11:26:14.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.972+0000 7f69e0cc5700 1 -- 172.21.15.112:0/1335159037 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 259954145 0 0) 0x7f69d8001110 con 0x7f69dc0b3e50 2023-08-27T11:26:14.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.972+0000 7f69e24c8700 1 --2- 172.21.15.112:0/1335159037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69dc163f90 0x7f69dc166380 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009d90 tx=0x7f69d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b616850828a18fa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:14.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.972+0000 7f69e0cc5700 1 -- 172.21.15.112:0/1335159037 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d400e070 con 0x7f69dc163f90 2023-08-27T11:26:14.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.972+0000 7f69e0cc5700 1 -- 172.21.15.112:0/1335159037 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d400bb90 con 0x7f69dc163f90 2023-08-27T11:26:14.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.972+0000 7f69e0cc5700 1 -- 172.21.15.112:0/1335159037 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d4003a20 con 0x7f69dc163f90 2023-08-27T11:26:14.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.973+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1335159037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69dc163f90 msgr2=0x7f69dc166380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:14.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.973+0000 7f69e3f2b700 1 --2- 172.21.15.112:0/1335159037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69dc163f90 0x7f69dc166380 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009d90 tx=0x7f69d400b330 comp rx=0 tx=0).stop 2023-08-27T11:26:14.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.973+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1335159037 shutdown_connections 2023-08-27T11:26:14.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.973+0000 7f69e3f2b700 1 --2- 172.21.15.112:0/1335159037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69dc163f90 0x7f69dc166380 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.973+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1335159037 >> 172.21.15.112:0/1335159037 conn(0x7f69dc06f970 msgr2=0x7f69dc06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:14.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.973+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1335159037 shutdown_connections 2023-08-27T11:26:14.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.973+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1335159037 wait complete. 2023-08-27T11:26:14.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.974+0000 7f69e3f2b700 1 Processor -- start 2023-08-27T11:26:14.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.974+0000 7f69e3f2b700 1 -- start start 2023-08-27T11:26:14.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e3f2b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69dc0b3e50 0x7f69dc13dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:14.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e3f2b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69dc144570 0x7f69dc072180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:14.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e3f2b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69dc163f90 0x7f69dc143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:14.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e3f2b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc1f3810 con 0x7f69dc0b3e50 2023-08-27T11:26:14.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e3f2b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc13e1e0 con 0x7f69dc163f90 2023-08-27T11:26:14.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e3f2b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc1f2c90 con 0x7f69dc144570 2023-08-27T11:26:14.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e1cc7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69dc0b3e50 0x7f69dc13dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:14.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e1cc7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69dc0b3e50 0x7f69dc13dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60920/0 (socket says 172.21.15.112:60920) 2023-08-27T11:26:14.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e1cc7700 1 -- 172.21.15.112:0/1622468521 learned_addr learned my addr 172.21.15.112:0/1622468521 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:14.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e24c8700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69dc163f90 0x7f69dc143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:14.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e1cc7700 1 -- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69dc144570 msgr2=0x7f69dc072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:14.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e14c6700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69dc144570 0x7f69dc072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:14.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e1cc7700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69dc144570 0x7f69dc072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e1cc7700 1 -- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69dc163f90 msgr2=0x7f69dc143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:14.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e1cc7700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69dc163f90 0x7f69dc143110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:14.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e1cc7700 1 -- 172.21.15.112:0/1622468521 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d4009a70 con 0x7f69dc0b3e50 2023-08-27T11:26:14.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e24c8700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69dc163f90 0x7f69dc143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:14.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e1cc7700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69dc0b3e50 0x7f69dc13dca0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00c4d0 tx=0x7f69cc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:14.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69d27fc700 1 -- 172.21.15.112:0/1622468521 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69cc011790 con 0x7f69dc0b3e50 2023-08-27T11:26:14.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69d27fc700 1 -- 172.21.15.112:0/1622468521 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69cc0118f0 con 0x7f69dc0b3e50 2023-08-27T11:26:14.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1622468521 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69dc1f2f70 con 0x7f69dc0b3e50 2023-08-27T11:26:14.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.975+0000 7f69d27fc700 1 -- 172.21.15.112:0/1622468521 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69cc011ca0 con 0x7f69dc0b3e50 2023-08-27T11:26:14.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.976+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1622468521 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69dc1f3530 con 0x7f69dc0b3e50 2023-08-27T11:26:14.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.976+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1622468521 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69dc0d0c40 con 0x7f69dc0b3e50 2023-08-27T11:26:14.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.977+0000 7f69d27fc700 1 -- 172.21.15.112:0/1622468521 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f69cc01c460 con 0x7f69dc0b3e50 2023-08-27T11:26:14.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.977+0000 7f69d27fc700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f69c006a3b0 0x7f69c006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:14.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.977+0000 7f69d27fc700 1 -- 172.21.15.112:0/1622468521 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f69cc088410 con 0x7f69dc0b3e50 2023-08-27T11:26:14.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.977+0000 7f69e14c6700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f69c006a3b0 0x7f69c006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:14.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.978+0000 7f69e14c6700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f69c006a3b0 0x7f69c006c870 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0663c0 tx=0x7f69d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:14.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:14.981+0000 7f69d27fc700 1 -- 172.21.15.112:0/1622468521 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f69cc053b90 con 0x7f69dc0b3e50 2023-08-27T11:26:15.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.170+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1622468521 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69dc067a20 con 0x7f69dc0b3e50 2023-08-27T11:26:15.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.171+0000 7f69d27fc700 1 -- 172.21.15.112:0/1622468521 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+14606 (secure 0 0 0) 0x7f69cc0571b0 con 0x7f69dc0b3e50 2023-08-27T11:26:15.173 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:15.173 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":86,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:14.396149+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:15.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.173+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f69c006a3b0 msgr2=0x7f69c006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.173+0000 7f69e3f2b700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f69c006a3b0 0x7f69c006c870 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0663c0 tx=0x7f69d8009040 comp rx=0 tx=0).stop 2023-08-27T11:26:15.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.173+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1622468521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69dc0b3e50 msgr2=0x7f69dc13dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.173+0000 7f69e3f2b700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69dc0b3e50 0x7f69dc13dca0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00c4d0 tx=0x7f69cc00c890 comp rx=0 tx=0).stop 2023-08-27T11:26:15.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.173+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1622468521 shutdown_connections 2023-08-27T11:26:15.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.173+0000 7f69e3f2b700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f69c006a3b0 0x7f69c006c870 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.173+0000 7f69e3f2b700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69dc163f90 0x7f69dc143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.173+0000 7f69e3f2b700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69dc0b3e50 0x7f69dc13dca0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.173+0000 7f69e3f2b700 1 --2- 172.21.15.112:0/1622468521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69dc144570 0x7f69dc072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.173+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1622468521 >> 172.21.15.112:0/1622468521 conn(0x7f69dc06f970 msgr2=0x7f69dc15acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:15.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.173+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1622468521 shutdown_connections 2023-08-27T11:26:15.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.173+0000 7f69e3f2b700 1 -- 172.21.15.112:0/1622468521 wait complete. 2023-08-27T11:26:15.187 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:26:15.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:26:15.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.310+0000 7f3774d16700 1 Processor -- start 2023-08-27T11:26:15.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.310+0000 7f3774d16700 1 -- start start 2023-08-27T11:26:15.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.311+0000 7f3774d16700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3770158ed0 0x7f377015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.311+0000 7f3774d16700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3770072000 0x7f377006ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.311+0000 7f3774d16700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377015ca50 con 0x7f377006e2f0 2023-08-27T11:26:15.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.311+0000 7f3774d16700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377006e780 con 0x7f3770072000 2023-08-27T11:26:15.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.311+0000 7f3774d16700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3770072480 con 0x7f3770158ed0 2023-08-27T11:26:15.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.312+0000 7f376e59c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3770158ed0 0x7f377015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.312+0000 7f376dd9b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3770072000 0x7f377006ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.312+0000 7f376dd9b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3770072000 0x7f377006ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:33124/0 (socket says 172.21.15.112:33124) 2023-08-27T11:26:15.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.312+0000 7f376dd9b700 1 -- 172.21.15.112:0/2373721491 learned_addr learned my addr 172.21.15.112:0/2373721491 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:15.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.312+0000 7f376dd9b700 1 -- 172.21.15.112:0/2373721491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3770158ed0 msgr2=0x7f377015b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:15.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.312+0000 7f376dd9b700 1 --2- 172.21.15.112:0/2373721491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3770158ed0 0x7f377015b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.312+0000 7f376dd9b700 1 -- 172.21.15.112:0/2373721491 >> v1:172.21.15.138:6789/0 conn(0x7f377006e2f0 legacy=0x7f377015eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.312+0000 7f376dd9b700 1 -- 172.21.15.112:0/2373721491 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377015be20 con 0x7f3770072000 2023-08-27T11:26:15.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.312+0000 7f376e59c700 1 --2- 172.21.15.112:0/2373721491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3770158ed0 0x7f377015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:15.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.312+0000 7f376dd9b700 1 --2- 172.21.15.112:0/2373721491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3770072000 0x7f377006ddb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f37600098f0 tx=0x7f3760009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d0a2abf98d0faf1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.312+0000 7f376d59a700 1 -- 172.21.15.112:0/2373721491 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760011830 con 0x7f3770072000 2023-08-27T11:26:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.313+0000 7f376d59a700 1 -- 172.21.15.112:0/2373721491 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3760011990 con 0x7f3770072000 2023-08-27T11:26:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.313+0000 7f376d59a700 1 -- 172.21.15.112:0/2373721491 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760011d20 con 0x7f3770072000 2023-08-27T11:26:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.313+0000 7f3774d16700 1 -- 172.21.15.112:0/2373721491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3770072000 msgr2=0x7f377006ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.313+0000 7f3774d16700 1 --2- 172.21.15.112:0/2373721491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3770072000 0x7f377006ddb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f37600098f0 tx=0x7f3760009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:15.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.313+0000 7f3774d16700 1 -- 172.21.15.112:0/2373721491 shutdown_connections 2023-08-27T11:26:15.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.313+0000 7f3774d16700 1 --2- 172.21.15.112:0/2373721491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3770158ed0 0x7f377015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.313+0000 7f3774d16700 1 --2- 172.21.15.112:0/2373721491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3770072000 0x7f377006ddb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.313+0000 7f3774d16700 1 -- 172.21.15.112:0/2373721491 >> 172.21.15.112:0/2373721491 conn(0x7f37700cc240 msgr2=0x7f37700cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:15.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.313+0000 7f3774d16700 1 -- 172.21.15.112:0/2373721491 shutdown_connections 2023-08-27T11:26:15.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.313+0000 7f3774d16700 1 -- 172.21.15.112:0/2373721491 wait complete. 2023-08-27T11:26:15.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.314+0000 7f3774d16700 1 Processor -- start 2023-08-27T11:26:15.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.314+0000 7f3774d16700 1 -- start start 2023-08-27T11:26:15.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.315+0000 7f3774d16700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f377006e2f0 0x7f37700af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.315+0000 7f3774d16700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3770072000 0x7f3770156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.315+0000 7f3774d16700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3770158ed0 0x7f3770157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.315+0000 7f3774d16700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37700abac0 con 0x7f3770158ed0 2023-08-27T11:26:15.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.315+0000 7f3774d16700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3770157a40 con 0x7f377006e2f0 2023-08-27T11:26:15.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.315+0000 7f3774d16700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37700aaf40 con 0x7f3770072000 2023-08-27T11:26:15.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.315+0000 7f376ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3770158ed0 0x7f3770157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.315+0000 7f376ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3770158ed0 0x7f3770157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60938/0 (socket says 172.21.15.112:60938) 2023-08-27T11:26:15.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.315+0000 7f376ed9d700 1 -- 172.21.15.112:0/2709920546 learned_addr learned my addr 172.21.15.112:0/2709920546 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:15.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.315+0000 7f376e59c700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f377006e2f0 0x7f37700af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.316+0000 7f376dd9b700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3770072000 0x7f3770156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.316+0000 7f376ed9d700 1 -- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3770072000 msgr2=0x7f3770156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.316+0000 7f376ed9d700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3770072000 0x7f3770156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.316+0000 7f376ed9d700 1 -- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f377006e2f0 msgr2=0x7f37700af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.316+0000 7f376ed9d700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f377006e2f0 0x7f37700af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.316+0000 7f376ed9d700 1 -- 172.21.15.112:0/2709920546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37640094a0 con 0x7f3770158ed0 2023-08-27T11:26:15.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.316+0000 7f376e59c700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f377006e2f0 0x7f37700af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.316+0000 7f376ed9d700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3770158ed0 0x7f3770157500 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f376800cc80 tx=0x7f376800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.316+0000 7f375effd700 1 -- 172.21.15.112:0/2709920546 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376800ad40 con 0x7f3770158ed0 2023-08-27T11:26:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.316+0000 7f375effd700 1 -- 172.21.15.112:0/2709920546 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376800ce80 con 0x7f3770158ed0 2023-08-27T11:26:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.316+0000 7f375effd700 1 -- 172.21.15.112:0/2709920546 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376800aea0 con 0x7f3770158ed0 2023-08-27T11:26:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.316+0000 7f3774d16700 1 -- 172.21.15.112:0/2709920546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3760009550 con 0x7f3770158ed0 2023-08-27T11:26:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.316+0000 7f3774d16700 1 -- 172.21.15.112:0/2709920546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37700ab500 con 0x7f3770158ed0 2023-08-27T11:26:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.317+0000 7f3774d16700 1 -- 172.21.15.112:0/2709920546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37700acdb0 con 0x7f3770158ed0 2023-08-27T11:26:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.317+0000 7f375effd700 1 -- 172.21.15.112:0/2709920546 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f3768003c40 con 0x7f3770158ed0 2023-08-27T11:26:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.318+0000 7f375effd700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f375006a3b0 0x7f375006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.318+0000 7f375effd700 1 -- 172.21.15.112:0/2709920546 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3768088750 con 0x7f3770158ed0 2023-08-27T11:26:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.318+0000 7f376e59c700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f375006a3b0 0x7f375006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.318+0000 7f376e59c700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f375006a3b0 0x7f375006c870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3764009d20 tx=0x7f3764008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.322+0000 7f375effd700 1 -- 172.21.15.112:0/2709920546 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3768053e40 con 0x7f3770158ed0 2023-08-27T11:26:15.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.513+0000 7f3774d16700 1 -- 172.21.15.112:0/2709920546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3770067e90 con 0x7f3770158ed0 2023-08-27T11:26:15.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.514+0000 7f375effd700 1 -- 172.21.15.112:0/2709920546 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3768057460 con 0x7f3770158ed0 2023-08-27T11:26:15.516 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:26:15.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.516+0000 7f3774d16700 1 -- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f375006a3b0 msgr2=0x7f375006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.516+0000 7f3774d16700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f375006a3b0 0x7f375006c870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3764009d20 tx=0x7f3764008040 comp rx=0 tx=0).stop 2023-08-27T11:26:15.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.516+0000 7f3774d16700 1 -- 172.21.15.112:0/2709920546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3770158ed0 msgr2=0x7f3770157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.516+0000 7f3774d16700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3770158ed0 0x7f3770157500 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f376800cc80 tx=0x7f376800a340 comp rx=0 tx=0).stop 2023-08-27T11:26:15.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.516+0000 7f3774d16700 1 -- 172.21.15.112:0/2709920546 shutdown_connections 2023-08-27T11:26:15.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.516+0000 7f3774d16700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f375006a3b0 0x7f375006c870 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.516+0000 7f3774d16700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3770158ed0 0x7f3770157500 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.516+0000 7f3774d16700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f377006e2f0 0x7f37700af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.516+0000 7f3774d16700 1 --2- 172.21.15.112:0/2709920546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3770072000 0x7f3770156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.516+0000 7f3774d16700 1 -- 172.21.15.112:0/2709920546 >> 172.21.15.112:0/2709920546 conn(0x7f37700cc240 msgr2=0x7f3770157e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:15.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.516+0000 7f3774d16700 1 -- 172.21.15.112:0/2709920546 shutdown_connections 2023-08-27T11:26:15.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.516+0000 7f3774d16700 1 -- 172.21.15.112:0/2709920546 wait complete. 2023-08-27T11:26:15.530 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:26:15.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.651+0000 7f8db8640700 1 Processor -- start 2023-08-27T11:26:15.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.651+0000 7f8db8640700 1 -- start start 2023-08-27T11:26:15.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.652+0000 7f8db8640700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db01437c0 0x7f8db00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.652+0000 7f8db8640700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8db0143ba0 0x7f8db00ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.652+0000 7f8db8640700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8db006d5b0 0x7f8db006da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.652+0000 7f8db8640700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db00cfd50 con 0x7f8db0143ba0 2023-08-27T11:26:15.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.652+0000 7f8db8640700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db006df40 con 0x7f8db006d5b0 2023-08-27T11:26:15.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.652+0000 7f8db8640700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00cf140 con 0x7f8db01437c0 2023-08-27T11:26:15.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.652+0000 7f8db5bdb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8db0143ba0 0x7f8db00ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.652+0000 7f8db5bdb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8db0143ba0 0x7f8db00ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60944/0 (socket says 172.21.15.112:60944) 2023-08-27T11:26:15.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.652+0000 7f8db5bdb700 1 -- 172.21.15.112:0/1250690728 learned_addr learned my addr 172.21.15.112:0/1250690728 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:15.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.653+0000 7f8db63dc700 1 --2- 172.21.15.112:0/1250690728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db01437c0 0x7f8db00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.653+0000 7f8db6bdd700 1 --2- 172.21.15.112:0/1250690728 >> v2:172.21.15.138:3300/0 conn(0x7f8db006d5b0 0x7f8db006da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.653+0000 7f8db5bdb700 1 -- 172.21.15.112:0/1250690728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db01437c0 msgr2=0x7f8db00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:15.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.653+0000 7f8db5bdb700 1 --2- 172.21.15.112:0/1250690728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db01437c0 0x7f8db00b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.653+0000 7f8db5bdb700 1 -- 172.21.15.112:0/1250690728 >> v2:172.21.15.138:3300/0 conn(0x7f8db006d5b0 msgr2=0x7f8db006da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:15.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.653+0000 7f8db5bdb700 1 --2- 172.21.15.112:0/1250690728 >> v2:172.21.15.138:3300/0 conn(0x7f8db006d5b0 0x7f8db006da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.653+0000 7f8db5bdb700 1 -- 172.21.15.112:0/1250690728 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db00cf420 con 0x7f8db0143ba0 2023-08-27T11:26:15.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.653+0000 7f8db5bdb700 1 --2- 172.21.15.112:0/1250690728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db0143ba0 0x7f8db00ce860 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8db00b3af0 tx=0x7f8dac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b50b16eb0207b5a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:15.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.653+0000 7f8db53da700 1 -- 172.21.15.112:0/1250690728 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac00b870 con 0x7f8db0143ba0 2023-08-27T11:26:15.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.653+0000 7f8db53da700 1 -- 172.21.15.112:0/1250690728 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac00b9d0 con 0x7f8db0143ba0 2023-08-27T11:26:15.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.653+0000 7f8db53da700 1 -- 172.21.15.112:0/1250690728 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac00bd60 con 0x7f8db0143ba0 2023-08-27T11:26:15.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.653+0000 7f8db8640700 1 -- 172.21.15.112:0/1250690728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db0143ba0 msgr2=0x7f8db00ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.653+0000 7f8db8640700 1 --2- 172.21.15.112:0/1250690728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db0143ba0 0x7f8db00ce860 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8db00b3af0 tx=0x7f8dac009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.654+0000 7f8db8640700 1 -- 172.21.15.112:0/1250690728 shutdown_connections 2023-08-27T11:26:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.654+0000 7f8db8640700 1 --2- 172.21.15.112:0/1250690728 >> v2:172.21.15.138:3300/0 conn(0x7f8db006d5b0 0x7f8db006da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.654+0000 7f8db8640700 1 --2- 172.21.15.112:0/1250690728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db01437c0 0x7f8db00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.654+0000 7f8db8640700 1 --2- 172.21.15.112:0/1250690728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db0143ba0 0x7f8db00ce860 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.654+0000 7f8db8640700 1 -- 172.21.15.112:0/1250690728 >> 172.21.15.112:0/1250690728 conn(0x7f8db00b1e90 msgr2=0x7f8db00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.654+0000 7f8db8640700 1 -- 172.21.15.112:0/1250690728 shutdown_connections 2023-08-27T11:26:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.654+0000 7f8db8640700 1 -- 172.21.15.112:0/1250690728 wait complete. 2023-08-27T11:26:15.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.655+0000 7f8db8640700 1 Processor -- start 2023-08-27T11:26:15.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.655+0000 7f8db8640700 1 -- start start 2023-08-27T11:26:15.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db8640700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db006d5b0 0x7f8db013f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db8640700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db01437c0 0x7f8db013cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db8640700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db0143ba0 0x7f8db0139500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db8640700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db013acb0 con 0x7f8db01437c0 2023-08-27T11:26:15.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db8640700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db013d9f0 con 0x7f8db0143ba0 2023-08-27T11:26:15.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db8640700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db013d710 con 0x7f8db006d5b0 2023-08-27T11:26:15.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db5bdb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db01437c0 0x7f8db013cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db5bdb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db01437c0 0x7f8db013cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60952/0 (socket says 172.21.15.112:60952) 2023-08-27T11:26:15.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db5bdb700 1 -- 172.21.15.112:0/2545030377 learned_addr learned my addr 172.21.15.112:0/2545030377 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:15.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db63dc700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db006d5b0 0x7f8db013f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db6bdd700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db0143ba0 0x7f8db0139500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db5bdb700 1 -- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db006d5b0 msgr2=0x7f8db013f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db5bdb700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db006d5b0 0x7f8db013f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db5bdb700 1 -- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db0143ba0 msgr2=0x7f8db0139500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db5bdb700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db0143ba0 0x7f8db0139500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db5bdb700 1 -- 172.21.15.112:0/2545030377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac009550 con 0x7f8db01437c0 2023-08-27T11:26:15.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db6bdd700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db0143ba0 0x7f8db0139500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:15.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db63dc700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db006d5b0 0x7f8db013f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:15.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db5bdb700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db01437c0 0x7f8db013cc70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f8dac014040 tx=0x7f8dac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:15.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8da6ffd700 1 -- 172.21.15.112:0/2545030377 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac004520 con 0x7f8db01437c0 2023-08-27T11:26:15.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8da6ffd700 1 -- 172.21.15.112:0/2545030377 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac01e070 con 0x7f8db01437c0 2023-08-27T11:26:15.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db8640700 1 -- 172.21.15.112:0/2545030377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db013a660 con 0x7f8db01437c0 2023-08-27T11:26:15.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8da6ffd700 1 -- 172.21.15.112:0/2545030377 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0038e0 con 0x7f8db01437c0 2023-08-27T11:26:15.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.656+0000 7f8db8640700 1 -- 172.21.15.112:0/2545030377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db01580b0 con 0x7f8db01437c0 2023-08-27T11:26:15.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.657+0000 7f8db8640700 1 -- 172.21.15.112:0/2545030377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db0067e90 con 0x7f8db01437c0 2023-08-27T11:26:15.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.658+0000 7f8da6ffd700 1 -- 172.21.15.112:0/2545030377 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f8dac003a40 con 0x7f8db01437c0 2023-08-27T11:26:15.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.658+0000 7f8da6ffd700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8d9406a3b0 0x7f8d9406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.658+0000 7f8da6ffd700 1 -- 172.21.15.112:0/2545030377 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8dac020070 con 0x7f8db01437c0 2023-08-27T11:26:15.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.658+0000 7f8db63dc700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8d9406a3b0 0x7f8d9406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.659+0000 7f8db63dc700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8d9406a3b0 0x7f8d9406c870 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8da000aa50 tx=0x7f8da0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:15.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.662+0000 7f8da6ffd700 1 -- 172.21.15.112:0/2545030377 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8dac055450 con 0x7f8db01437c0 2023-08-27T11:26:15.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.850+0000 7f8db8640700 1 -- 172.21.15.112:0/2545030377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8db0067a20 con 0x7f8db01437c0 2023-08-27T11:26:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.850+0000 7f8da6ffd700 1 -- 172.21.15.112:0/2545030377 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8dac015020 con 0x7f8db01437c0 2023-08-27T11:26:15.852 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:26:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.852+0000 7f8db8640700 1 -- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8d9406a3b0 msgr2=0x7f8d9406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.852+0000 7f8db8640700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8d9406a3b0 0x7f8d9406c870 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8da000aa50 tx=0x7f8da0008040 comp rx=0 tx=0).stop 2023-08-27T11:26:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.852+0000 7f8db8640700 1 -- 172.21.15.112:0/2545030377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db01437c0 msgr2=0x7f8db013cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.852+0000 7f8db8640700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db01437c0 0x7f8db013cc70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f8dac014040 tx=0x7f8dac004750 comp rx=0 tx=0).stop 2023-08-27T11:26:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.852+0000 7f8db8640700 1 -- 172.21.15.112:0/2545030377 shutdown_connections 2023-08-27T11:26:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.852+0000 7f8db8640700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8d9406a3b0 0x7f8d9406c870 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.852+0000 7f8db8640700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db0143ba0 0x7f8db0139500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.852+0000 7f8db8640700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db006d5b0 0x7f8db013f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.852+0000 7f8db8640700 1 --2- 172.21.15.112:0/2545030377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db01437c0 0x7f8db013cc70 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.852+0000 7f8db8640700 1 -- 172.21.15.112:0/2545030377 >> 172.21.15.112:0/2545030377 conn(0x7f8db00b1e90 msgr2=0x7f8db0071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.852+0000 7f8db8640700 1 -- 172.21.15.112:0/2545030377 shutdown_connections 2023-08-27T11:26:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.852+0000 7f8db8640700 1 -- 172.21.15.112:0/2545030377 wait complete. 2023-08-27T11:26:15.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:26:15.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.984+0000 7fa57eefb700 1 Processor -- start 2023-08-27T11:26:15.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.984+0000 7fa57eefb700 1 -- start start 2023-08-27T11:26:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.985+0000 7fa57eefb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa5780b17e0 0x7fa5780cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.985+0000 7fa57eefb700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa578161850 con 0x7fa5780b1400 2023-08-27T11:26:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.985+0000 7fa57eefb700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5780cfc10 con 0x7fa578161ce0 2023-08-27T11:26:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.985+0000 7fa57eefb700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa578161200 con 0x7fa5780b17e0 2023-08-27T11:26:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.985+0000 7fa577fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa5780b17e0 0x7fa5780cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.985+0000 7fa57cc97700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa5780b1400 0x7fa57806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55340/0 (socket says 172.21.15.112:55340) 2023-08-27T11:26:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.985+0000 7fa577fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa5780b17e0 0x7fa5780cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60960/0 (socket says 172.21.15.112:60960) 2023-08-27T11:26:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.985+0000 7fa57cc97700 1 -- 172.21.15.112:0/876250014 learned_addr learned my addr 172.21.15.112:0/876250014 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.985+0000 7fa577fff700 1 -- 172.21.15.112:0/876250014 >> v1:172.21.15.138:6789/0 conn(0x7fa578161ce0 legacy=0x7fa57806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:15.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.985+0000 7fa577fff700 1 -- 172.21.15.112:0/876250014 >> v1:172.21.15.112:6789/0 conn(0x7fa5780b1400 legacy=0x7fa57806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.985+0000 7fa577fff700 1 -- 172.21.15.112:0/876250014 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5781614e0 con 0x7fa5780b17e0 2023-08-27T11:26:15.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.985+0000 7fa5777fe700 1 -- 172.21.15.112:0/876250014 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566391121 0 0) 0x7fa578161850 con 0x7fa5780b1400 2023-08-27T11:26:15.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.985+0000 7fa577fff700 1 --2- 172.21.15.112:0/876250014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5780b17e0 0x7fa5780cf5a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa564009840 tx=0x7fa564009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dba14543058eb9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:15.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.986+0000 7fa5777fe700 1 -- 172.21.15.112:0/876250014 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564004230 con 0x7fa5780b17e0 2023-08-27T11:26:15.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.986+0000 7fa5777fe700 1 -- 172.21.15.112:0/876250014 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa564004390 con 0x7fa5780b17e0 2023-08-27T11:26:15.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.986+0000 7fa5777fe700 1 -- 172.21.15.112:0/876250014 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640036a0 con 0x7fa5780b17e0 2023-08-27T11:26:15.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.986+0000 7fa57eefb700 1 -- 172.21.15.112:0/876250014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5780b17e0 msgr2=0x7fa5780cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.986+0000 7fa57eefb700 1 --2- 172.21.15.112:0/876250014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5780b17e0 0x7fa5780cf5a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa564009840 tx=0x7fa564009b50 comp rx=0 tx=0).stop 2023-08-27T11:26:15.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.986+0000 7fa57eefb700 1 -- 172.21.15.112:0/876250014 shutdown_connections 2023-08-27T11:26:15.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.986+0000 7fa57eefb700 1 --2- 172.21.15.112:0/876250014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5780b17e0 0x7fa5780cf5a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.986+0000 7fa57eefb700 1 -- 172.21.15.112:0/876250014 >> 172.21.15.112:0/876250014 conn(0x7fa5780d0000 msgr2=0x7fa5780d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:15.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.986+0000 7fa57eefb700 1 -- 172.21.15.112:0/876250014 shutdown_connections 2023-08-27T11:26:15.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.986+0000 7fa57eefb700 1 -- 172.21.15.112:0/876250014 wait complete. 2023-08-27T11:26:15.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.987+0000 7fa57eefb700 1 Processor -- start 2023-08-27T11:26:15.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.987+0000 7fa57eefb700 1 -- start start 2023-08-27T11:26:15.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.988+0000 7fa57eefb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780b1400 0x7fa578150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.988+0000 7fa57eefb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5780b17e0 0x7fa5781f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.988+0000 7fa57eefb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa578161ce0 0x7fa5780c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:15.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.988+0000 7fa57eefb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5780c70d0 con 0x7fa5780b17e0 2023-08-27T11:26:15.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.988+0000 7fa57eefb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5780c73b0 con 0x7fa5780b1400 2023-08-27T11:26:15.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.988+0000 7fa57eefb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5780c7690 con 0x7fa578161ce0 2023-08-27T11:26:15.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.988+0000 7fa577fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5780b17e0 0x7fa5781f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.988+0000 7fa577fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5780b17e0 0x7fa5781f2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60966/0 (socket says 172.21.15.112:60966) 2023-08-27T11:26:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.988+0000 7fa577fff700 1 -- 172.21.15.112:0/3999978469 learned_addr learned my addr 172.21.15.112:0/3999978469 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.988+0000 7fa57cc97700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780b1400 0x7fa578150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.988+0000 7fa57d498700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa578161ce0 0x7fa5780c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.989+0000 7fa577fff700 1 -- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa578161ce0 msgr2=0x7fa5780c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.989+0000 7fa577fff700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa578161ce0 0x7fa5780c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.989+0000 7fa577fff700 1 -- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780b1400 msgr2=0x7fa578150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:15.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.989+0000 7fa577fff700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780b1400 0x7fa578150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:15.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.989+0000 7fa577fff700 1 -- 172.21.15.112:0/3999978469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5640094a0 con 0x7fa5780b17e0 2023-08-27T11:26:15.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.989+0000 7fa57cc97700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780b1400 0x7fa578150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:15.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.989+0000 7fa57d498700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa578161ce0 0x7fa5780c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:15.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.989+0000 7fa577fff700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5780b17e0 0x7fa5781f2f50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa5640090b0 tx=0x7fa564004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:15.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.989+0000 7fa5757fa700 1 -- 172.21.15.112:0/3999978469 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564011830 con 0x7fa5780b17e0 2023-08-27T11:26:15.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.989+0000 7fa5757fa700 1 -- 172.21.15.112:0/3999978469 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56401e070 con 0x7fa5780b17e0 2023-08-27T11:26:15.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.989+0000 7fa57eefb700 1 -- 172.21.15.112:0/3999978469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5780c7970 con 0x7fa5780b17e0 2023-08-27T11:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.989+0000 7fa5757fa700 1 -- 172.21.15.112:0/3999978469 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564011bb0 con 0x7fa5780b17e0 2023-08-27T11:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.989+0000 7fa57eefb700 1 -- 172.21.15.112:0/3999978469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5781f3490 con 0x7fa5780b17e0 2023-08-27T11:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.990+0000 7fa57eefb700 1 -- 172.21.15.112:0/3999978469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5780c7e90 con 0x7fa5780b17e0 2023-08-27T11:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.990+0000 7fa5757fa700 1 -- 172.21.15.112:0/3999978469 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fa564011d10 con 0x7fa5780b17e0 2023-08-27T11:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.990+0000 7fa5757fa700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa55c06a3b0 0x7fa55c06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:16.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.991+0000 7fa5757fa700 1 -- 172.21.15.112:0/3999978469 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa5640898a0 con 0x7fa5780b17e0 2023-08-27T11:26:16.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.991+0000 7fa57cc97700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa55c06a3b0 0x7fa55c06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:16.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.991+0000 7fa57cc97700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa55c06a3b0 0x7fa55c06c870 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa56c000f60 tx=0x7fa56c007cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:16.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:15.995+0000 7fa5757fa700 1 -- 172.21.15.112:0/3999978469 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa5780c7e90 con 0x7fa5780b17e0 2023-08-27T11:26:16.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.187+0000 7fa57eefb700 1 -- 172.21.15.112:0/3999978469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa5780c7e90 con 0x7fa5780b17e0 2023-08-27T11:26:16.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.188+0000 7fa5757fa700 1 -- 172.21.15.112:0/3999978469 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa5780c7e90 con 0x7fa5780b17e0 2023-08-27T11:26:16.190 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:16.190 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:26:16.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:16.190 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2023-08-27T11:26:16.191 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "rbd" 2023-08-27T11:26:16.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:16.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:16.191 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 2, 2023-08-27T11:26:16.191 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2023-08-27T11:26:16.192 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:16.192 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:26:16.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.190+0000 7fa57eefb700 1 -- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa55c06a3b0 msgr2=0x7fa55c06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:16.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.190+0000 7fa57eefb700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa55c06a3b0 0x7fa55c06c870 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa56c000f60 tx=0x7fa56c007cc0 comp rx=0 tx=0).stop 2023-08-27T11:26:16.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.190+0000 7fa57eefb700 1 -- 172.21.15.112:0/3999978469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5780b17e0 msgr2=0x7fa5781f2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:16.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.190+0000 7fa57eefb700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5780b17e0 0x7fa5781f2f50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa5640090b0 tx=0x7fa564004750 comp rx=0 tx=0).stop 2023-08-27T11:26:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.190+0000 7fa57eefb700 1 -- 172.21.15.112:0/3999978469 shutdown_connections 2023-08-27T11:26:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.190+0000 7fa57eefb700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa55c06a3b0 0x7fa55c06c870 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.190+0000 7fa57eefb700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa578161ce0 0x7fa5780c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.190+0000 7fa57eefb700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780b1400 0x7fa578150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.190+0000 7fa57eefb700 1 --2- 172.21.15.112:0/3999978469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5780b17e0 0x7fa5781f2f50 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:16.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.190+0000 7fa57eefb700 1 -- 172.21.15.112:0/3999978469 >> 172.21.15.112:0/3999978469 conn(0x7fa5780d0000 msgr2=0x7fa5780cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:16.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.190+0000 7fa57eefb700 1 -- 172.21.15.112:0/3999978469 shutdown_connections 2023-08-27T11:26:16.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.190+0000 7fa57eefb700 1 -- 172.21.15.112:0/3999978469 wait complete. 2023-08-27T11:26:16.203 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:26:16.204 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:26:16.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.327+0000 7f406e849700 1 Processor -- start 2023-08-27T11:26:16.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.328+0000 7f406e849700 1 -- start start 2023-08-27T11:26:16.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.328+0000 7f406e849700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4068146d50 0x7f40680b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:16.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.328+0000 7f406e849700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40680b69b0 0x7f40680b5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:16.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.328+0000 7f406e849700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40680b5580 0x7f40681610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:16.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.328+0000 7f406e849700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40680b5a10 con 0x7f40680b69b0 2023-08-27T11:26:16.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.328+0000 7f406e849700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40680b7f60 con 0x7f4068146d50 2023-08-27T11:26:16.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.328+0000 7f406e849700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40680b6ec0 con 0x7f40680b5580 2023-08-27T11:26:16.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.328+0000 7f40677fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40680b69b0 0x7f40680b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:16.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.328+0000 7f40677fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40680b69b0 0x7f40680b5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60972/0 (socket says 172.21.15.112:60972) 2023-08-27T11:26:16.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.328+0000 7f40677fe700 1 -- 172.21.15.112:0/1920079417 learned_addr learned my addr 172.21.15.112:0/1920079417 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:16.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.329+0000 7f40677fe700 1 -- 172.21.15.112:0/1920079417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40680b5580 msgr2=0x7f40681610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:16.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.329+0000 7f406cde6700 1 --2- 172.21.15.112:0/1920079417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40680b5580 0x7f40681610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:16.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.329+0000 7f4067fff700 1 --2- 172.21.15.112:0/1920079417 >> v2:172.21.15.138:3300/0 conn(0x7f4068146d50 0x7f40680b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:16.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.329+0000 7f40677fe700 1 --2- 172.21.15.112:0/1920079417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40680b5580 0x7f40681610d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:16.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.329+0000 7f40677fe700 1 -- 172.21.15.112:0/1920079417 >> v2:172.21.15.138:3300/0 conn(0x7f4068146d50 msgr2=0x7f40680b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:16.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.329+0000 7f40677fe700 1 --2- 172.21.15.112:0/1920079417 >> v2:172.21.15.138:3300/0 conn(0x7f4068146d50 0x7f40680b6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:16.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.329+0000 7f40677fe700 1 -- 172.21.15.112:0/1920079417 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40680b71a0 con 0x7f40680b69b0 2023-08-27T11:26:16.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.329+0000 7f40677fe700 1 --2- 172.21.15.112:0/1920079417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40680b69b0 0x7f40680b5040 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f406806a930 tx=0x7f4050009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b75598e1cc03cec server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:16.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.329+0000 7f4066ffd700 1 -- 172.21.15.112:0/1920079417 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405000b870 con 0x7f40680b69b0 2023-08-27T11:26:16.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.329+0000 7f4066ffd700 1 -- 172.21.15.112:0/1920079417 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405000b9d0 con 0x7f40680b69b0 2023-08-27T11:26:16.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.329+0000 7f4066ffd700 1 -- 172.21.15.112:0/1920079417 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405000bd60 con 0x7f40680b69b0 2023-08-27T11:26:16.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.330+0000 7f406e849700 1 -- 172.21.15.112:0/1920079417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40680b69b0 msgr2=0x7f40680b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:16.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.330+0000 7f406e849700 1 --2- 172.21.15.112:0/1920079417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40680b69b0 0x7f40680b5040 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f406806a930 tx=0x7f4050009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:16.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.330+0000 7f406e849700 1 -- 172.21.15.112:0/1920079417 shutdown_connections 2023-08-27T11:26:16.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.330+0000 7f406e849700 1 --2- 172.21.15.112:0/1920079417 >> v2:172.21.15.138:3300/0 conn(0x7f4068146d50 0x7f40680b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:16.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.330+0000 7f406e849700 1 --2- 172.21.15.112:0/1920079417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40680b5580 0x7f40681610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:16.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.330+0000 7f406e849700 1 --2- 172.21.15.112:0/1920079417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40680b69b0 0x7f40680b5040 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:16.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.330+0000 7f406e849700 1 -- 172.21.15.112:0/1920079417 >> 172.21.15.112:0/1920079417 conn(0x7f406806d330 msgr2=0x7f40680717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:16.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.330+0000 7f406e849700 1 -- 172.21.15.112:0/1920079417 shutdown_connections 2023-08-27T11:26:16.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.330+0000 7f406e849700 1 -- 172.21.15.112:0/1920079417 wait complete. 2023-08-27T11:26:16.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.331+0000 7f406e849700 1 Processor -- start 2023-08-27T11:26:16.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.331+0000 7f406e849700 1 -- start start 2023-08-27T11:26:16.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406e849700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40680b5580 0x7f40681e7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:16.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406e849700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40680b69b0 0x7f406815b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:16.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406e849700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4068146d50 0x7f406815b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:16.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406e849700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40681e6b20 con 0x7f4068146d50 2023-08-27T11:26:16.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406e849700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406815edc0 con 0x7f40680b5580 2023-08-27T11:26:16.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406e849700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40681e5ea0 con 0x7f40680b69b0 2023-08-27T11:26:16.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406cde6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4068146d50 0x7f406815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:16.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406cde6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4068146d50 0x7f406815b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60980/0 (socket says 172.21.15.112:60980) 2023-08-27T11:26:16.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406cde6700 1 -- 172.21.15.112:0/4270710678 learned_addr learned my addr 172.21.15.112:0/4270710678 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:16.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f4067fff700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40680b5580 0x7f40681e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:16.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f40677fe700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40680b69b0 0x7f406815b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:16.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406cde6700 1 -- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40680b69b0 msgr2=0x7f406815b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:16.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406cde6700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40680b69b0 0x7f406815b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:16.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406cde6700 1 -- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40680b5580 msgr2=0x7f40681e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:16.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406cde6700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40680b5580 0x7f40681e7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:16.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406cde6700 1 -- 172.21.15.112:0/4270710678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4050009550 con 0x7f4068146d50 2023-08-27T11:26:16.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406cde6700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4068146d50 0x7f406815b880 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f406000cd20 tx=0x7f406000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:16.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f4064ff9700 1 -- 172.21.15.112:0/4270710678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060003a20 con 0x7f4068146d50 2023-08-27T11:26:16.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f4064ff9700 1 -- 172.21.15.112:0/4270710678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4060003b80 con 0x7f4068146d50 2023-08-27T11:26:16.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f406e849700 1 -- 172.21.15.112:0/4270710678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40681e6180 con 0x7f4068146d50 2023-08-27T11:26:16.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.332+0000 7f4064ff9700 1 -- 172.21.15.112:0/4270710678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060004020 con 0x7f4068146d50 2023-08-27T11:26:16.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.333+0000 7f406e849700 1 -- 172.21.15.112:0/4270710678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40681e6740 con 0x7f4068146d50 2023-08-27T11:26:16.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.333+0000 7f406e849700 1 -- 172.21.15.112:0/4270710678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40681e6c60 con 0x7f4068146d50 2023-08-27T11:26:16.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.334+0000 7f4064ff9700 1 -- 172.21.15.112:0/4270710678 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f4060004180 con 0x7f4068146d50 2023-08-27T11:26:16.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.334+0000 7f4064ff9700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f404806a3b0 0x7f404806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:16.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.334+0000 7f4064ff9700 1 -- 172.21.15.112:0/4270710678 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4060012070 con 0x7f4068146d50 2023-08-27T11:26:16.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.334+0000 7f4067fff700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f404806a3b0 0x7f404806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:16.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.335+0000 7f4067fff700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f404806a3b0 0x7f404806c870 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f405800a950 tx=0x7f4058008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:16.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.339+0000 7f4064ff9700 1 -- 172.21.15.112:0/4270710678 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f40681e6c60 con 0x7f4068146d50 2023-08-27T11:26:16.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:16.529+0000 7f406e849700 1 -- 172.21.15.112:0/4270710678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f40681e6c60 con 0x7f4068146d50 2023-08-27T11:26:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.401+0000 7f4064ff9700 1 -- 172.21.15.112:0/4270710678 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) v1 ==== 138+0+0 (secure 0 0 0) 0x7f40681e6c60 con 0x7f4068146d50 2023-08-27T11:26:17.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.404+0000 7f406e849700 1 -- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f404806a3b0 msgr2=0x7f404806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:17.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.404+0000 7f406e849700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f404806a3b0 0x7f404806c870 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f405800a950 tx=0x7f4058008040 comp rx=0 tx=0).stop 2023-08-27T11:26:17.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.404+0000 7f406e849700 1 -- 172.21.15.112:0/4270710678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4068146d50 msgr2=0x7f406815b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:17.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.404+0000 7f406e849700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4068146d50 0x7f406815b880 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f406000cd20 tx=0x7f406000a570 comp rx=0 tx=0).stop 2023-08-27T11:26:17.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.404+0000 7f406e849700 1 -- 172.21.15.112:0/4270710678 shutdown_connections 2023-08-27T11:26:17.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.404+0000 7f406e849700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f404806a3b0 0x7f404806c870 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:17.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.405+0000 7f406e849700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4068146d50 0x7f406815b880 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:17.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.405+0000 7f406e849700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40680b5580 0x7f40681e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:17.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.405+0000 7f406e849700 1 --2- 172.21.15.112:0/4270710678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40680b69b0 0x7f406815b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:17.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.405+0000 7f406e849700 1 -- 172.21.15.112:0/4270710678 >> 172.21.15.112:0/4270710678 conn(0x7f406806d330 msgr2=0x7f4068070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:17.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.405+0000 7f406e849700 1 -- 172.21.15.112:0/4270710678 shutdown_connections 2023-08-27T11:26:17.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.405+0000 7f406e849700 1 -- 172.21.15.112:0/4270710678 wait complete. 2023-08-27T11:26:17.409 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2023-08-27T11:26:17.422 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:26:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.553+0000 7faef52a8700 1 Processor -- start 2023-08-27T11:26:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.553+0000 7faef52a8700 1 -- start start 2023-08-27T11:26:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.553+0000 7faef52a8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faef0157b80 0x7faef00d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.553+0000 7faef52a8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef00cf0f0 con 0x7faef00d23f0 2023-08-27T11:26:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.553+0000 7faef52a8700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef00ce570 con 0x7faef00ce030 2023-08-27T11:26:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.553+0000 7faef52a8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef00d2870 con 0x7faef0157b80 2023-08-27T11:26:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeee7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7faef00d23f0 0x7faef00cd920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55366/0 (socket says 172.21.15.112:55366) 2023-08-27T11:26:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeee7fc700 1 -- 172.21.15.112:0/603039017 learned_addr learned my addr 172.21.15.112:0/603039017 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeeeffd700 1 --2- 172.21.15.112:0/603039017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faef0157b80 0x7faef00d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883828330 0 0) 0x7faef00cf0f0 con 0x7faef00d23f0 2023-08-27T11:26:17.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faed80035b0 con 0x7faef00d23f0 2023-08-27T11:26:17.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3003738307 0 0) 0x7faed80035b0 con 0x7faef00d23f0 2023-08-27T11:26:17.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faef0157b80 msgr2=0x7faef00d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:17.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 --2- 172.21.15.112:0/603039017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faef0157b80 0x7faef00d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:17.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 >> v1:172.21.15.138:6789/0 conn(0x7faef00ce030 legacy=0x7faef00706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:17.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef00ce850 con 0x7faef00d23f0 2023-08-27T11:26:17.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7faee4001670 con 0x7faef00d23f0 2023-08-27T11:26:17.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 >> v1:172.21.15.112:6789/0 conn(0x7faef00d23f0 legacy=0x7faef00cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:17.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 --2- 172.21.15.112:0/603039017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faef0157b80 0x7faed8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:17.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 --2- 172.21.15.112:0/603039017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faed8003f70 0x7faed8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:17.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 --2- 172.21.15.112:0/603039017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed8004d90 0x7faed8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:17.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef00d2870 con 0x7faed8003f70 2023-08-27T11:26:17.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed800ba70 con 0x7faed8004d90 2023-08-27T11:26:17.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed800bc30 con 0x7faef0157b80 2023-08-27T11:26:17.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeee7fc700 1 --2- 172.21.15.112:0/603039017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faed8003f70 0x7faed8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:17.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeeeffd700 1 --2- 172.21.15.112:0/603039017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faef0157b80 0x7faed8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:17.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.554+0000 7faeef7fe700 1 --2- 172.21.15.112:0/603039017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed8004d90 0x7faed8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:17.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faeee7fc700 1 -- 172.21.15.112:0/603039017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faef0157b80 msgr2=0x7faed8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:17.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faeee7fc700 1 --2- 172.21.15.112:0/603039017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faef0157b80 0x7faed8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:17.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faeee7fc700 1 -- 172.21.15.112:0/603039017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed8004d90 msgr2=0x7faed8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:17.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faeee7fc700 1 --2- 172.21.15.112:0/603039017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed8004d90 0x7faed8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:17.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faeee7fc700 1 -- 172.21.15.112:0/603039017 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed80035b0 con 0x7faed8003f70 2023-08-27T11:26:17.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faeee7fc700 1 --2- 172.21.15.112:0/603039017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faed8003f70 0x7faed8004850 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7faee400c2f0 tx=0x7faed8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcec664094aca75d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:17.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee400cd70 con 0x7faed8003f70 2023-08-27T11:26:17.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee4005960 con 0x7faed8003f70 2023-08-27T11:26:17.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faeedffb700 1 -- 172.21.15.112:0/603039017 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee4005d10 con 0x7faed8003f70 2023-08-27T11:26:17.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faef52a8700 1 -- 172.21.15.112:0/603039017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faed8003f70 msgr2=0x7faed8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:17.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faef52a8700 1 --2- 172.21.15.112:0/603039017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faed8003f70 0x7faed8004850 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7faee400c2f0 tx=0x7faed8005880 comp rx=0 tx=0).stop 2023-08-27T11:26:17.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faeef7fe700 1 -- 172.21.15.112:0/603039017 reap_dead start 2023-08-27T11:26:17.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faef52a8700 1 -- 172.21.15.112:0/603039017 shutdown_connections 2023-08-27T11:26:17.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faef52a8700 1 -- 172.21.15.112:0/603039017 >> 172.21.15.112:0/603039017 conn(0x7faef00cfd50 msgr2=0x7faef00d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:17.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.555+0000 7faef52a8700 1 -- 172.21.15.112:0/603039017 shutdown_connections 2023-08-27T11:26:17.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.556+0000 7faef52a8700 1 -- 172.21.15.112:0/603039017 wait complete. 2023-08-27T11:26:17.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.557+0000 7faef52a8700 1 Processor -- start 2023-08-27T11:26:17.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.557+0000 7faef52a8700 1 -- start start 2023-08-27T11:26:17.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faef52a8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faed8003f70 0x7faef00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:17.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faef52a8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faef00cce90 0x7faef01f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:17.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faef52a8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faef01f6220 0x7faef0158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:17.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faef52a8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef00b3e30 con 0x7faed8003f70 2023-08-27T11:26:17.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faef52a8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef01f24c0 con 0x7faef00cce90 2023-08-27T11:26:17.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faef52a8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef00cd310 con 0x7faef01f6220 2023-08-27T11:26:17.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faeeeffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faed8003f70 0x7faef00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:17.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faeeeffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faed8003f70 0x7faef00cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60996/0 (socket says 172.21.15.112:60996) 2023-08-27T11:26:17.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faeeeffd700 1 -- 172.21.15.112:0/1961923746 learned_addr learned my addr 172.21.15.112:0/1961923746 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:17.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faeee7fc700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faef00cce90 0x7faef01f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:17.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faeef7fe700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faef01f6220 0x7faef0158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:17.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faeeeffd700 1 -- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faef01f6220 msgr2=0x7faef0158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:17.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faeeeffd700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faef01f6220 0x7faef0158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:17.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faeeeffd700 1 -- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faef00cce90 msgr2=0x7faef01f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:17.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faeeeffd700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faef00cce90 0x7faef01f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:17.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faeeeffd700 1 -- 172.21.15.112:0/1961923746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef01f3450 con 0x7faed8003f70 2023-08-27T11:26:17.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faeeeffd700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faed8003f70 0x7faef00cc950 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7faee0011ad0 tx=0x7faee0011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:17.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faed77fe700 1 -- 172.21.15.112:0/1961923746 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000f940 con 0x7faed8003f70 2023-08-27T11:26:17.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faed77fe700 1 -- 172.21.15.112:0/1961923746 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee000faa0 con 0x7faed8003f70 2023-08-27T11:26:17.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faef52a8700 1 -- 172.21.15.112:0/1961923746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef0147810 con 0x7faed8003f70 2023-08-27T11:26:17.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faed77fe700 1 -- 172.21.15.112:0/1961923746 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000fe50 con 0x7faed8003f70 2023-08-27T11:26:17.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.558+0000 7faef52a8700 1 -- 172.21.15.112:0/1961923746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef01f2970 con 0x7faed8003f70 2023-08-27T11:26:17.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.559+0000 7faef52a8700 1 -- 172.21.15.112:0/1961923746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef0067e90 con 0x7faed8003f70 2023-08-27T11:26:17.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.560+0000 7faed77fe700 1 -- 172.21.15.112:0/1961923746 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7faee0014440 con 0x7faed8003f70 2023-08-27T11:26:17.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.560+0000 7faed77fe700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faedc06a3b0 0x7faedc06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:17.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.560+0000 7faed77fe700 1 -- 172.21.15.112:0/1961923746 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5580+0+0 (secure 0 0 0) 0x7faee0088c80 con 0x7faed8003f70 2023-08-27T11:26:17.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.560+0000 7faeee7fc700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faedc06a3b0 0x7faedc06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:17.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.561+0000 7faeee7fc700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faedc06a3b0 0x7faedc06c870 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7faee4000c00 tx=0x7faee4007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:17.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.565+0000 7faed77fe700 1 -- 172.21.15.112:0/1961923746 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7faee00542c0 con 0x7faed8003f70 2023-08-27T11:26:17.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:17.754+0000 7faef52a8700 1 -- 172.21.15.112:0/1961923746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7faef01f3450 con 0x7faed8003f70 2023-08-27T11:26:18.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.406+0000 7faed77fe700 1 -- 172.21.15.112:0/1961923746 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) v1 ==== 170+0+0 (secure 0 0 0) 0x7faef01f3450 con 0x7faed8003f70 2023-08-27T11:26:18.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.410+0000 7faef52a8700 1 -- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faedc06a3b0 msgr2=0x7faedc06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:18.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.410+0000 7faef52a8700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faedc06a3b0 0x7faedc06c870 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7faee4000c00 tx=0x7faee4007000 comp rx=0 tx=0).stop 2023-08-27T11:26:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.410+0000 7faef52a8700 1 -- 172.21.15.112:0/1961923746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faed8003f70 msgr2=0x7faef00cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.410+0000 7faef52a8700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faed8003f70 0x7faef00cc950 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7faee0011ad0 tx=0x7faee0011de0 comp rx=0 tx=0).stop 2023-08-27T11:26:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.410+0000 7faef52a8700 1 -- 172.21.15.112:0/1961923746 shutdown_connections 2023-08-27T11:26:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.410+0000 7faef52a8700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faedc06a3b0 0x7faedc06c870 secure :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7faee4000c00 tx=0x7faee4007000 comp rx=0 tx=0).stop 2023-08-27T11:26:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.410+0000 7faef52a8700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faef01f6220 0x7faef0158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.410+0000 7faef52a8700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faed8003f70 0x7faef00cc950 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.410+0000 7faef52a8700 1 --2- 172.21.15.112:0/1961923746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faef00cce90 0x7faef01f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.410+0000 7faef52a8700 1 -- 172.21.15.112:0/1961923746 >> 172.21.15.112:0/1961923746 conn(0x7faef00cfd50 msgr2=0x7faef00c8530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:18.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.412+0000 7faef52a8700 1 -- 172.21.15.112:0/1961923746 shutdown_connections 2023-08-27T11:26:18.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.412+0000 7faef52a8700 1 -- 172.21.15.112:0/1961923746 wait complete. 2023-08-27T11:26:18.417 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2023-08-27T11:26:18.433 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:26:18.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.557+0000 7f9e2a417700 1 Processor -- start 2023-08-27T11:26:18.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.557+0000 7f9e2a417700 1 -- start start 2023-08-27T11:26:18.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.557+0000 7f9e2a417700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e24144b80 0x7f9e240b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:18.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.557+0000 7f9e2a417700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9e240b3de0 0x7f9e2406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:18.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.557+0000 7f9e2a417700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e2406c510 0x7f9e2415eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:18.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.557+0000 7f9e2a417700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e240b4b40 con 0x7f9e2406c510 2023-08-27T11:26:18.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.557+0000 7f9e2a417700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e240cef20 con 0x7f9e240b3de0 2023-08-27T11:26:18.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.557+0000 7f9e2a417700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2406c940 con 0x7f9e24144b80 2023-08-27T11:26:18.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e289b4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e2406c510 0x7f9e2415eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:18.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e289b4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e2406c510 0x7f9e2415eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32776/0 (socket says 172.21.15.112:32776) 2023-08-27T11:26:18.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e289b4700 1 -- 172.21.15.112:0/3636041949 learned_addr learned my addr 172.21.15.112:0/3636041949 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:18.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e237fe700 1 --2- 172.21.15.112:0/3636041949 >> v2:172.21.15.138:3300/0 conn(0x7f9e240b3de0 0x7f9e2406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:18.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e23fff700 1 --2- 172.21.15.112:0/3636041949 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e24144b80 0x7f9e240b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:18.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e289b4700 1 -- 172.21.15.112:0/3636041949 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e24144b80 msgr2=0x7f9e240b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:18.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e289b4700 1 --2- 172.21.15.112:0/3636041949 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e24144b80 0x7f9e240b6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e289b4700 1 -- 172.21.15.112:0/3636041949 >> v2:172.21.15.138:3300/0 conn(0x7f9e240b3de0 msgr2=0x7f9e2406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e289b4700 1 --2- 172.21.15.112:0/3636041949 >> v2:172.21.15.138:3300/0 conn(0x7f9e240b3de0 0x7f9e2406bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e289b4700 1 -- 172.21.15.112:0/3636041949 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e240b42a0 con 0x7f9e2406c510 2023-08-27T11:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e289b4700 1 --2- 172.21.15.112:0/3636041949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e2406c510 0x7f9e2415eeb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9e240b2e50 tx=0x7f9e1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0a5696054f8d9d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e22ffd700 1 -- 172.21.15.112:0/3636041949 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c00e070 con 0x7f9e2406c510 2023-08-27T11:26:18.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e22ffd700 1 -- 172.21.15.112:0/3636041949 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c00b9c0 con 0x7f9e2406c510 2023-08-27T11:26:18.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.558+0000 7f9e22ffd700 1 -- 172.21.15.112:0/3636041949 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c00bd70 con 0x7f9e2406c510 2023-08-27T11:26:18.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.559+0000 7f9e2a417700 1 -- 172.21.15.112:0/3636041949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e2406c510 msgr2=0x7f9e2415eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:18.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.559+0000 7f9e2a417700 1 --2- 172.21.15.112:0/3636041949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e2406c510 0x7f9e2415eeb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9e240b2e50 tx=0x7f9e1c00b330 comp rx=0 tx=0).stop 2023-08-27T11:26:18.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.559+0000 7f9e2a417700 1 -- 172.21.15.112:0/3636041949 shutdown_connections 2023-08-27T11:26:18.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.559+0000 7f9e2a417700 1 --2- 172.21.15.112:0/3636041949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e2406c510 0x7f9e2415eeb0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:18.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.559+0000 7f9e2a417700 1 --2- 172.21.15.112:0/3636041949 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e24144b80 0x7f9e240b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:18.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.559+0000 7f9e2a417700 1 --2- 172.21.15.112:0/3636041949 >> v2:172.21.15.138:3300/0 conn(0x7f9e240b3de0 0x7f9e2406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:18.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.559+0000 7f9e2a417700 1 -- 172.21.15.112:0/3636041949 >> 172.21.15.112:0/3636041949 conn(0x7f9e240cdc50 msgr2=0x7f9e240ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:18.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.559+0000 7f9e2a417700 1 -- 172.21.15.112:0/3636041949 shutdown_connections 2023-08-27T11:26:18.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.559+0000 7f9e2a417700 1 -- 172.21.15.112:0/3636041949 wait complete. 2023-08-27T11:26:18.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.560+0000 7f9e2a417700 1 Processor -- start 2023-08-27T11:26:18.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.560+0000 7f9e2a417700 1 -- start start 2023-08-27T11:26:18.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e2a417700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e2406c510 0x7f9e2415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:18.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e2a417700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e240b3de0 0x7f9e24142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:18.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e2a417700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e24144b80 0x7f9e24071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:18.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e2a417700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241f3ac0 con 0x7f9e240b3de0 2023-08-27T11:26:18.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e2a417700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241f3190 con 0x7f9e2406c510 2023-08-27T11:26:18.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e2a417700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241f3470 con 0x7f9e24144b80 2023-08-27T11:26:18.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e237fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e240b3de0 0x7f9e24142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:18.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e237fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e240b3de0 0x7f9e24142bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32780/0 (socket says 172.21.15.112:32780) 2023-08-27T11:26:18.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e237fe700 1 -- 172.21.15.112:0/1101629822 learned_addr learned my addr 172.21.15.112:0/1101629822 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:18.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e289b4700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e24144b80 0x7f9e24071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:18.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e23fff700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e2406c510 0x7f9e2415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:18.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e237fe700 1 -- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e24144b80 msgr2=0x7f9e24071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:18.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e237fe700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e24144b80 0x7f9e24071e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:18.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e237fe700 1 -- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e2406c510 msgr2=0x7f9e2415a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:18.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e237fe700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e2406c510 0x7f9e2415a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:18.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e237fe700 1 -- 172.21.15.112:0/1101629822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c009b30 con 0x7f9e240b3de0 2023-08-27T11:26:18.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e289b4700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e24144b80 0x7f9e24071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:18.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e23fff700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e2406c510 0x7f9e2415a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:18.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.561+0000 7f9e237fe700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e240b3de0 0x7f9e24142bf0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800e810 tx=0x7f9e1800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.562+0000 7f9e20ff9700 1 -- 172.21.15.112:0/1101629822 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18013610 con 0x7f9e240b3de0 2023-08-27T11:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.562+0000 7f9e20ff9700 1 -- 172.21.15.112:0/1101629822 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e18013770 con 0x7f9e240b3de0 2023-08-27T11:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.562+0000 7f9e2a417700 1 -- 172.21.15.112:0/1101629822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e241f3750 con 0x7f9e240b3de0 2023-08-27T11:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.562+0000 7f9e20ff9700 1 -- 172.21.15.112:0/1101629822 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18013b00 con 0x7f9e240b3de0 2023-08-27T11:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.562+0000 7f9e2a417700 1 -- 172.21.15.112:0/1101629822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e24158470 con 0x7f9e240b3de0 2023-08-27T11:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.563+0000 7f9e2a417700 1 -- 172.21.15.112:0/1101629822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e241f3c70 con 0x7f9e240b3de0 2023-08-27T11:26:18.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.563+0000 7f9e20ff9700 1 -- 172.21.15.112:0/1101629822 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f9e18013c60 con 0x7f9e240b3de0 2023-08-27T11:26:18.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.563+0000 7f9e20ff9700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e1006a3b0 0x7f9e1006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:18.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.564+0000 7f9e20ff9700 1 -- 172.21.15.112:0/1101629822 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f9e18088730 con 0x7f9e240b3de0 2023-08-27T11:26:18.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.564+0000 7f9e23fff700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e1006a3b0 0x7f9e1006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:18.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.564+0000 7f9e23fff700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e1006a3b0 0x7f9e1006c870 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9e241f0ac0 tx=0x7f9e14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:18.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.568+0000 7f9e20ff9700 1 -- 172.21.15.112:0/1101629822 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9e241f3c70 con 0x7f9e240b3de0 2023-08-27T11:26:18.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:18.803+0000 7f9e2a417700 1 -- 172.21.15.112:0/1101629822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9e240b4860 con 0x7f9e240b3de0 2023-08-27T11:26:19.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.408+0000 7f9e20ff9700 1 -- 172.21.15.112:0/1101629822 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v54) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9e18053940 con 0x7f9e240b3de0 2023-08-27T11:26:19.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.416+0000 7f9e2a417700 1 -- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e1006a3b0 msgr2=0x7f9e1006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:19.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.416+0000 7f9e2a417700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e1006a3b0 0x7f9e1006c870 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9e241f0ac0 tx=0x7f9e14008040 comp rx=0 tx=0).stop 2023-08-27T11:26:19.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.416+0000 7f9e2a417700 1 -- 172.21.15.112:0/1101629822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e240b3de0 msgr2=0x7f9e24142bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:19.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.416+0000 7f9e2a417700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e240b3de0 0x7f9e24142bf0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800e810 tx=0x7f9e1800ebd0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.420+0000 7f9e2a417700 1 -- 172.21.15.112:0/1101629822 shutdown_connections 2023-08-27T11:26:19.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.420+0000 7f9e2a417700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e1006a3b0 0x7f9e1006c870 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.420+0000 7f9e2a417700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e24144b80 0x7f9e24071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.420+0000 7f9e2a417700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e2406c510 0x7f9e2415a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.420+0000 7f9e2a417700 1 --2- 172.21.15.112:0/1101629822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e240b3de0 0x7f9e24142bf0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.420+0000 7f9e2a417700 1 -- 172.21.15.112:0/1101629822 >> 172.21.15.112:0/1101629822 conn(0x7f9e240cdc50 msgr2=0x7f9e240cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:19.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.422+0000 7f9e2a417700 1 -- 172.21.15.112:0/1101629822 shutdown_connections 2023-08-27T11:26:19.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.423+0000 7f9e2a417700 1 -- 172.21.15.112:0/1101629822 wait complete. 2023-08-27T11:26:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 15 and data pool 16 2023-08-27T11:26:19.446 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:19.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.566+0000 7f7293b57700 1 Processor -- start 2023-08-27T11:26:19.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.567+0000 7f7293b57700 1 -- start start 2023-08-27T11:26:19.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.567+0000 7f7293b57700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f728c06adb0 0x7f728c06b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:19.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.567+0000 7f7293b57700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c0cdbd0 0x7f728c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:19.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.567+0000 7f7293b57700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728c15eb60 con 0x7f728c151060 2023-08-27T11:26:19.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.567+0000 7f7293b57700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728c06b770 con 0x7f728c06adb0 2023-08-27T11:26:19.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.567+0000 7f7293b57700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0b02f0 con 0x7f728c0cdbd0 2023-08-27T11:26:19.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72910f2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f728c06adb0 0x7f728c06b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:19.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72910f2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f728c06adb0 0x7f728c06b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32802/0 (socket says 172.21.15.112:32802) 2023-08-27T11:26:19.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72918f3700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f728c151060 0x7f728c15e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55414/0 (socket says 172.21.15.112:55414) 2023-08-27T11:26:19.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72910f2700 1 -- 172.21.15.112:0/3232544222 learned_addr learned my addr 172.21.15.112:0/3232544222 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:19.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72920f4700 1 --2- 172.21.15.112:0/3232544222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c0cdbd0 0x7f728c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:19.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72910f2700 1 -- 172.21.15.112:0/3232544222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c0cdbd0 msgr2=0x7f728c0ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72910f2700 1 --2- 172.21.15.112:0/3232544222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c0cdbd0 0x7f728c0ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72910f2700 1 -- 172.21.15.112:0/3232544222 >> v1:172.21.15.112:6789/0 conn(0x7f728c151060 legacy=0x7f728c15e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72910f2700 1 -- 172.21.15.112:0/3232544222 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c0b05d0 con 0x7f728c06adb0 2023-08-27T11:26:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72908f1700 1 -- 172.21.15.112:0/3232544222 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045504977 0 0) 0x7f728c15eb60 con 0x7f728c151060 2023-08-27T11:26:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72920f4700 1 --2- 172.21.15.112:0/3232544222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c0cdbd0 0x7f728c0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72910f2700 1 --2- 172.21.15.112:0/3232544222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f728c06adb0 0x7f728c06b230 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f728800bb90 tx=0x7f728800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25acfecdea65bb7e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72908f1700 1 -- 172.21.15.112:0/3232544222 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728800dd50 con 0x7f728c06adb0 2023-08-27T11:26:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72908f1700 1 -- 172.21.15.112:0/3232544222 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7288010470 con 0x7f728c06adb0 2023-08-27T11:26:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.568+0000 7f72908f1700 1 -- 172.21.15.112:0/3232544222 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288010800 con 0x7f728c06adb0 2023-08-27T11:26:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.569+0000 7f7293b57700 1 -- 172.21.15.112:0/3232544222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f728c06adb0 msgr2=0x7f728c06b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:19.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.569+0000 7f7293b57700 1 --2- 172.21.15.112:0/3232544222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f728c06adb0 0x7f728c06b230 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f728800bb90 tx=0x7f728800bea0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.569+0000 7f7293b57700 1 -- 172.21.15.112:0/3232544222 shutdown_connections 2023-08-27T11:26:19.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.569+0000 7f7293b57700 1 --2- 172.21.15.112:0/3232544222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c0cdbd0 0x7f728c0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.569+0000 7f7293b57700 1 --2- 172.21.15.112:0/3232544222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f728c06adb0 0x7f728c06b230 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.569+0000 7f7293b57700 1 -- 172.21.15.112:0/3232544222 >> 172.21.15.112:0/3232544222 conn(0x7f728c0d0000 msgr2=0x7f728c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:19.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.569+0000 7f7293b57700 1 -- 172.21.15.112:0/3232544222 shutdown_connections 2023-08-27T11:26:19.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.569+0000 7f7293b57700 1 -- 172.21.15.112:0/3232544222 wait complete. 2023-08-27T11:26:19.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.570+0000 7f7293b57700 1 Processor -- start 2023-08-27T11:26:19.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.570+0000 7f7293b57700 1 -- start start 2023-08-27T11:26:19.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.571+0000 7f7293b57700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c06adb0 0x7f728c14bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:19.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.571+0000 7f7293b57700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f728c0cdbd0 0x7f728c153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:19.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.571+0000 7f7293b57700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c151060 0x7f728c14c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:19.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.571+0000 7f7293b57700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0c8600 con 0x7f728c0cdbd0 2023-08-27T11:26:19.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.571+0000 7f7293b57700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c14c070 con 0x7f728c06adb0 2023-08-27T11:26:19.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.571+0000 7f7293b57700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0c7560 con 0x7f728c151060 2023-08-27T11:26:19.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.571+0000 7f72910f2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f728c0cdbd0 0x7f728c153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:19.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.571+0000 7f72910f2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f728c0cdbd0 0x7f728c153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32808/0 (socket says 172.21.15.112:32808) 2023-08-27T11:26:19.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.571+0000 7f72910f2700 1 -- 172.21.15.112:0/230182448 learned_addr learned my addr 172.21.15.112:0/230182448 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:19.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f72920f4700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c151060 0x7f728c14c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:19.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f72918f3700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c06adb0 0x7f728c14bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:19.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f72910f2700 1 -- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c151060 msgr2=0x7f728c14c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:19.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f72910f2700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c151060 0x7f728c14c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f72910f2700 1 -- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c06adb0 msgr2=0x7f728c14bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:19.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f72910f2700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c06adb0 0x7f728c14bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f72910f2700 1 -- 172.21.15.112:0/230182448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728800b7f0 con 0x7f728c0cdbd0 2023-08-27T11:26:19.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f72918f3700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c06adb0 0x7f728c14bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:19.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f72910f2700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f728c0cdbd0 0x7f728c153540 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f728c06e590 tx=0x7f7288004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:19.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f72827fc700 1 -- 172.21.15.112:0/230182448 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728801f070 con 0x7f728c0cdbd0 2023-08-27T11:26:19.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f72827fc700 1 -- 172.21.15.112:0/230182448 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72880039f0 con 0x7f728c0cdbd0 2023-08-27T11:26:19.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f7293b57700 1 -- 172.21.15.112:0/230182448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728c0c7840 con 0x7f728c0cdbd0 2023-08-27T11:26:19.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f72827fc700 1 -- 172.21.15.112:0/230182448 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288003da0 con 0x7f728c0cdbd0 2023-08-27T11:26:19.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.572+0000 7f7293b57700 1 -- 172.21.15.112:0/230182448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728c0c7e00 con 0x7f728c0cdbd0 2023-08-27T11:26:19.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.573+0000 7f7293b57700 1 -- 172.21.15.112:0/230182448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728c067a20 con 0x7f728c0cdbd0 2023-08-27T11:26:19.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.574+0000 7f72827fc700 1 -- 172.21.15.112:0/230182448 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f7288019400 con 0x7f728c0cdbd0 2023-08-27T11:26:19.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.574+0000 7f72827fc700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f727006a3b0 0x7f727006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:19.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.574+0000 7f72827fc700 1 -- 172.21.15.112:0/230182448 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f728808c020 con 0x7f728c0cdbd0 2023-08-27T11:26:19.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.574+0000 7f72918f3700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f727006a3b0 0x7f727006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:19.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.575+0000 7f72918f3700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f727006a3b0 0x7f727006c870 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f727c000f60 tx=0x7f727c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:19.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.578+0000 7f72827fc700 1 -- 172.21.15.112:0/230182448 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7288057460 con 0x7f728c0cdbd0 2023-08-27T11:26:19.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.771+0000 7f7293b57700 1 -- 172.21.15.112:0/230182448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f728c0cf9f0 con 0x7f728c0cdbd0 2023-08-27T11:26:19.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.772+0000 7f72827fc700 1 -- 172.21.15.112:0/230182448 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+18261 (secure 0 0 0) 0x7f728805aa80 con 0x7f728c0cdbd0 2023-08-27T11:26:19.774 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:19.774 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":89,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:18.805053+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:16.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:17.773993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:19.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.774+0000 7f7293b57700 1 -- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f727006a3b0 msgr2=0x7f727006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:19.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.774+0000 7f7293b57700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f727006a3b0 0x7f727006c870 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f727c000f60 tx=0x7f727c009040 comp rx=0 tx=0).stop 2023-08-27T11:26:19.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.774+0000 7f7293b57700 1 -- 172.21.15.112:0/230182448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f728c0cdbd0 msgr2=0x7f728c153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:19.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.774+0000 7f7293b57700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f728c0cdbd0 0x7f728c153540 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f728c06e590 tx=0x7f7288004750 comp rx=0 tx=0).stop 2023-08-27T11:26:19.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.774+0000 7f7293b57700 1 -- 172.21.15.112:0/230182448 shutdown_connections 2023-08-27T11:26:19.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.774+0000 7f7293b57700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f727006a3b0 0x7f727006c870 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.774+0000 7f7293b57700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c151060 0x7f728c14c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.774+0000 7f7293b57700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c06adb0 0x7f728c14bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.774+0000 7f7293b57700 1 --2- 172.21.15.112:0/230182448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f728c0cdbd0 0x7f728c153540 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.774+0000 7f7293b57700 1 -- 172.21.15.112:0/230182448 >> 172.21.15.112:0/230182448 conn(0x7f728c0d0000 msgr2=0x7f728c06dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:19.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.774+0000 7f7293b57700 1 -- 172.21.15.112:0/230182448 shutdown_connections 2023-08-27T11:26:19.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.774+0000 7f7293b57700 1 -- 172.21.15.112:0/230182448 wait complete. 2023-08-27T11:26:19.788 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:19.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.906+0000 7fb255930700 1 Processor -- start 2023-08-27T11:26:19.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.906+0000 7fb255930700 1 -- start start 2023-08-27T11:26:19.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.907+0000 7fb255930700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb250143900 0x7fb2500708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:19.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.907+0000 7fb255930700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb25006dc90 0x7fb2500b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:19.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.907+0000 7fb255930700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb250070de0 con 0x7fb25015dcc0 2023-08-27T11:26:19.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.907+0000 7fb255930700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb250071010 con 0x7fb25006dc90 2023-08-27T11:26:19.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.907+0000 7fb255930700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2500712f0 con 0x7fb250143900 2023-08-27T11:26:19.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.907+0000 7fb24f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb25006dc90 0x7fb2500b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:19.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.907+0000 7fb24f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb25006dc90 0x7fb2500b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32818/0 (socket says 172.21.15.112:32818) 2023-08-27T11:26:19.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.907+0000 7fb24e7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb25015dcc0 0x7fb25006d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55428/0 (socket says 172.21.15.112:55428) 2023-08-27T11:26:19.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.907+0000 7fb24f7fe700 1 -- 172.21.15.112:0/584617981 learned_addr learned my addr 172.21.15.112:0/584617981 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:19.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.908+0000 7fb24effd700 1 --2- 172.21.15.112:0/584617981 >> v2:172.21.15.138:3300/0 conn(0x7fb250143900 0x7fb2500708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:19.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.908+0000 7fb24f7fe700 1 -- 172.21.15.112:0/584617981 >> v2:172.21.15.138:3300/0 conn(0x7fb250143900 msgr2=0x7fb2500708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:19.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.908+0000 7fb24f7fe700 1 --2- 172.21.15.112:0/584617981 >> v2:172.21.15.138:3300/0 conn(0x7fb250143900 0x7fb2500708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.908+0000 7fb24f7fe700 1 -- 172.21.15.112:0/584617981 >> v1:172.21.15.112:6789/0 conn(0x7fb25015dcc0 legacy=0x7fb25006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:19.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.908+0000 7fb24f7fe700 1 -- 172.21.15.112:0/584617981 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2500715d0 con 0x7fb25006dc90 2023-08-27T11:26:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.908+0000 7fb24dffb700 1 -- 172.21.15.112:0/584617981 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321448605 0 0) 0x7fb250070de0 con 0x7fb25015dcc0 2023-08-27T11:26:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.908+0000 7fb24f7fe700 1 --2- 172.21.15.112:0/584617981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb25006dc90 0x7fb2500b3820 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fb2500b2b10 tx=0x7fb24800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f115e56f64c02a29 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.908+0000 7fb24dffb700 1 -- 172.21.15.112:0/584617981 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24800e070 con 0x7fb25006dc90 2023-08-27T11:26:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.908+0000 7fb24dffb700 1 -- 172.21.15.112:0/584617981 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24800b9c0 con 0x7fb25006dc90 2023-08-27T11:26:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.908+0000 7fb24dffb700 1 -- 172.21.15.112:0/584617981 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24800bd70 con 0x7fb25006dc90 2023-08-27T11:26:19.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.908+0000 7fb255930700 1 -- 172.21.15.112:0/584617981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb25006dc90 msgr2=0x7fb2500b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:19.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.908+0000 7fb255930700 1 --2- 172.21.15.112:0/584617981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb25006dc90 0x7fb2500b3820 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fb2500b2b10 tx=0x7fb24800b330 comp rx=0 tx=0).stop 2023-08-27T11:26:19.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.909+0000 7fb255930700 1 -- 172.21.15.112:0/584617981 shutdown_connections 2023-08-27T11:26:19.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.909+0000 7fb255930700 1 --2- 172.21.15.112:0/584617981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb25006dc90 0x7fb2500b3820 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.909+0000 7fb255930700 1 --2- 172.21.15.112:0/584617981 >> v2:172.21.15.138:3300/0 conn(0x7fb250143900 0x7fb2500708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.909+0000 7fb255930700 1 -- 172.21.15.112:0/584617981 >> 172.21.15.112:0/584617981 conn(0x7fb250066400 msgr2=0x7fb2500cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:19.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.909+0000 7fb255930700 1 -- 172.21.15.112:0/584617981 shutdown_connections 2023-08-27T11:26:19.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.909+0000 7fb255930700 1 -- 172.21.15.112:0/584617981 wait complete. 2023-08-27T11:26:19.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.910+0000 7fb255930700 1 Processor -- start 2023-08-27T11:26:19.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.910+0000 7fb255930700 1 -- start start 2023-08-27T11:26:19.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb255930700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25006dc90 0x7fb250137140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:19.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb255930700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb250143900 0x7fb250142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:19.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb255930700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb25015dcc0 0x7fb25006f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:19.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb255930700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500cde50 con 0x7fb25015dcc0 2023-08-27T11:26:19.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb255930700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb250134610 con 0x7fb25006dc90 2023-08-27T11:26:19.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb255930700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb250134eb0 con 0x7fb250143900 2023-08-27T11:26:19.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb24f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb25015dcc0 0x7fb25006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:19.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb24f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb25015dcc0 0x7fb25006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32824/0 (socket says 172.21.15.112:32824) 2023-08-27T11:26:19.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb24f7fe700 1 -- 172.21.15.112:0/1476054584 learned_addr learned my addr 172.21.15.112:0/1476054584 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:19.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb24effd700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25006dc90 0x7fb250137140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:19.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb24e7fc700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb250143900 0x7fb250142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:19.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb24f7fe700 1 -- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb250143900 msgr2=0x7fb250142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:19.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb24f7fe700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb250143900 0x7fb250142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb24f7fe700 1 -- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25006dc90 msgr2=0x7fb250137140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:19.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb24f7fe700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25006dc90 0x7fb250137140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:19.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb24f7fe700 1 -- 172.21.15.112:0/1476054584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb248009b30 con 0x7fb25015dcc0 2023-08-27T11:26:19.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb24f7fe700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb25015dcc0 0x7fb25006f790 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fb2500d0cb0 tx=0x7fb248004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:19.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb237fff700 1 -- 172.21.15.112:0/1476054584 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248004750 con 0x7fb25015dcc0 2023-08-27T11:26:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb237fff700 1 -- 172.21.15.112:0/1476054584 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2480048d0 con 0x7fb25015dcc0 2023-08-27T11:26:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb255930700 1 -- 172.21.15.112:0/1476054584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb250137680 con 0x7fb25015dcc0 2023-08-27T11:26:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb237fff700 1 -- 172.21.15.112:0/1476054584 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24801c680 con 0x7fb25015dcc0 2023-08-27T11:26:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.911+0000 7fb255930700 1 -- 172.21.15.112:0/1476054584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb250134860 con 0x7fb25015dcc0 2023-08-27T11:26:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.912+0000 7fb255930700 1 -- 172.21.15.112:0/1476054584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb250061f50 con 0x7fb25015dcc0 2023-08-27T11:26:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.913+0000 7fb237fff700 1 -- 172.21.15.112:0/1476054584 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fb24801c7e0 con 0x7fb25015dcc0 2023-08-27T11:26:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.913+0000 7fb237fff700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb23c06a3b0 0x7fb23c06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.913+0000 7fb237fff700 1 -- 172.21.15.112:0/1476054584 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb24808a610 con 0x7fb25015dcc0 2023-08-27T11:26:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.913+0000 7fb24effd700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb23c06a3b0 0x7fb23c06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.914+0000 7fb24effd700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb23c06a3b0 0x7fb23c06c870 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb244009930 tx=0x7fb244008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:19.917+0000 7fb237fff700 1 -- 172.21.15.112:0/1476054584 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb248055930 con 0x7fb25015dcc0 2023-08-27T11:26:20.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.105+0000 7fb255930700 1 -- 172.21.15.112:0/1476054584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb25006ee90 con 0x7fb25015dcc0 2023-08-27T11:26:20.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.106+0000 7fb237fff700 1 -- 172.21.15.112:0/1476054584 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+18261 (secure 0 0 0) 0x7fb248058f50 con 0x7fb25015dcc0 2023-08-27T11:26:20.109 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:20.109 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":89,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:18.805053+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:16.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:17.773993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:20.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.108+0000 7fb255930700 1 -- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb23c06a3b0 msgr2=0x7fb23c06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:20.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.108+0000 7fb255930700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb23c06a3b0 0x7fb23c06c870 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb244009930 tx=0x7fb244008040 comp rx=0 tx=0).stop 2023-08-27T11:26:20.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.108+0000 7fb255930700 1 -- 172.21.15.112:0/1476054584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb25015dcc0 msgr2=0x7fb25006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:20.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.108+0000 7fb255930700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb25015dcc0 0x7fb25006f790 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fb2500d0cb0 tx=0x7fb248004ad0 comp rx=0 tx=0).stop 2023-08-27T11:26:20.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.108+0000 7fb255930700 1 -- 172.21.15.112:0/1476054584 shutdown_connections 2023-08-27T11:26:20.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.108+0000 7fb255930700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb23c06a3b0 0x7fb23c06c870 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:20.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.108+0000 7fb255930700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb25015dcc0 0x7fb25006f790 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:20.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.108+0000 7fb255930700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25006dc90 0x7fb250137140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:20.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.108+0000 7fb255930700 1 --2- 172.21.15.112:0/1476054584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb250143900 0x7fb250142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:20.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.108+0000 7fb255930700 1 -- 172.21.15.112:0/1476054584 >> 172.21.15.112:0/1476054584 conn(0x7fb250066400 msgr2=0x7fb250141d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:20.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.108+0000 7fb255930700 1 -- 172.21.15.112:0/1476054584 shutdown_connections 2023-08-27T11:26:20.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.108+0000 7fb255930700 1 -- 172.21.15.112:0/1476054584 wait complete. 2023-08-27T11:26:20.123 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:26:20.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.244+0000 7f7114ba1700 1 Processor -- start 2023-08-27T11:26:20.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.244+0000 7f7114ba1700 1 -- start start 2023-08-27T11:26:20.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f7114ba1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f711006f790 0x7f711006fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:20.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f7114ba1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71100b5070 con 0x7f7110146c20 2023-08-27T11:26:20.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f7114ba1700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7110147000 con 0x7f71100b3440 2023-08-27T11:26:20.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f7114ba1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7110070160 con 0x7f711006f790 2023-08-27T11:26:20.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f710e59c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7110146c20 0x7f7110160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55442/0 (socket says 172.21.15.112:55442) 2023-08-27T11:26:20.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f710e59c700 1 -- 172.21.15.112:0/2937164329 learned_addr learned my addr 172.21.15.112:0/2937164329 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:20.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783930794 0 0) 0x7f7104001110 con 0x7f7110146c20 2023-08-27T11:26:20.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70f80035b0 con 0x7f7110146c20 2023-08-27T11:26:20.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f710ed9d700 1 --2- 172.21.15.112:0/2937164329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f711006f790 0x7f711006fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:20.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3588989372 0 0) 0x7f70f80035b0 con 0x7f7110146c20 2023-08-27T11:26:20.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f711006f790 msgr2=0x7f711006fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:20.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f710d59a700 1 --2- 172.21.15.112:0/2937164329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f711006f790 0x7f711006fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:20.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 >> v1:172.21.15.138:6789/0 conn(0x7f71100b3440 legacy=0x7f711015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:20.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7110161740 con 0x7f7110146c20 2023-08-27T11:26:20.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.245+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f7104001800 con 0x7f7110146c20 2023-08-27T11:26:20.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 >> v1:172.21.15.112:6789/0 conn(0x7f7110146c20 legacy=0x7f7110160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:20.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710d59a700 1 --2- 172.21.15.112:0/2937164329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70f8003a30 0x7f70f8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:20.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710d59a700 1 --2- 172.21.15.112:0/2937164329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f711006f790 0x7f70f8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:20.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710d59a700 1 --2- 172.21.15.112:0/2937164329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f8004d90 0x7f70f8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:20.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7110070160 con 0x7f70f8004d90 2023-08-27T11:26:20.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f800bb70 con 0x7f70f8003a30 2023-08-27T11:26:20.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f800bd30 con 0x7f711006f790 2023-08-27T11:26:20.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710ed9d700 1 --2- 172.21.15.112:0/2937164329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f8004d90 0x7f70f8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:20.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710e59c700 1 --2- 172.21.15.112:0/2937164329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70f8003a30 0x7f70f8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710dd9b700 1 --2- 172.21.15.112:0/2937164329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f711006f790 0x7f70f8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710ed9d700 1 -- 172.21.15.112:0/2937164329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f711006f790 msgr2=0x7f70f8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710ed9d700 1 --2- 172.21.15.112:0/2937164329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f711006f790 0x7f70f8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710ed9d700 1 -- 172.21.15.112:0/2937164329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70f8003a30 msgr2=0x7f70f8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710ed9d700 1 --2- 172.21.15.112:0/2937164329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70f8003a30 0x7f70f8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710ed9d700 1 -- 172.21.15.112:0/2937164329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f80035b0 con 0x7f70f8004d90 2023-08-27T11:26:20.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710ed9d700 1 --2- 172.21.15.112:0/2937164329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f8004d90 0x7f70f8005210 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7108009fc0 tx=0x7f70f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706dabd89ea531a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:20.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710800ec30 con 0x7f70f8004d90 2023-08-27T11:26:20.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710800f040 con 0x7f70f8004d90 2023-08-27T11:26:20.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.246+0000 7f710d59a700 1 -- 172.21.15.112:0/2937164329 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108014480 con 0x7f70f8004d90 2023-08-27T11:26:20.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.247+0000 7f7114ba1700 1 -- 172.21.15.112:0/2937164329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f8004d90 msgr2=0x7f70f8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:20.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.247+0000 7f7114ba1700 1 --2- 172.21.15.112:0/2937164329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f8004d90 0x7f70f8005210 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7108009fc0 tx=0x7f70f8005880 comp rx=0 tx=0).stop 2023-08-27T11:26:20.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.247+0000 7f710ed9d700 1 -- 172.21.15.112:0/2937164329 reap_dead start 2023-08-27T11:26:20.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.247+0000 7f7114ba1700 1 -- 172.21.15.112:0/2937164329 shutdown_connections 2023-08-27T11:26:20.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.247+0000 7f7114ba1700 1 -- 172.21.15.112:0/2937164329 >> 172.21.15.112:0/2937164329 conn(0x7f71100b1e90 msgr2=0x7f71100b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:20.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.247+0000 7f7114ba1700 1 -- 172.21.15.112:0/2937164329 shutdown_connections 2023-08-27T11:26:20.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.247+0000 7f7114ba1700 1 -- 172.21.15.112:0/2937164329 wait complete. 2023-08-27T11:26:20.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.249+0000 7f7114ba1700 1 Processor -- start 2023-08-27T11:26:20.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.249+0000 7f7114ba1700 1 -- start start 2023-08-27T11:26:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.249+0000 7f7114ba1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70f8004d90 0x7f7110144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.249+0000 7f7114ba1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7110144650 0x7f71100ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f7114ba1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71101451c0 0x7f71100aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f7114ba1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71101f3730 con 0x7f71101451c0 2023-08-27T11:26:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f7114ba1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71101f3a10 con 0x7f7110144650 2023-08-27T11:26:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f7114ba1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71101f3cf0 con 0x7f70f8004d90 2023-08-27T11:26:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f710ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71101451c0 0x7f71100aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f710ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71101451c0 0x7f71100aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32844/0 (socket says 172.21.15.112:32844) 2023-08-27T11:26:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f710ed9d700 1 -- 172.21.15.112:0/1693497009 learned_addr learned my addr 172.21.15.112:0/1693497009 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f710e59c700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70f8004d90 0x7f7110144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f710dd9b700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7110144650 0x7f71100ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f710ed9d700 1 -- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70f8004d90 msgr2=0x7f7110144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f710ed9d700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70f8004d90 0x7f7110144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f710ed9d700 1 -- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7110144650 msgr2=0x7f71100ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f710ed9d700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7110144650 0x7f71100ae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f710ed9d700 1 -- 172.21.15.112:0/1693497009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7110143910 con 0x7f71101451c0 2023-08-27T11:26:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f710ed9d700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71101451c0 0x7f71100aea10 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f7108017040 tx=0x7f7108009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f70feffd700 1 -- 172.21.15.112:0/1693497009 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108007000 con 0x7f71101451c0 2023-08-27T11:26:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f70feffd700 1 -- 172.21.15.112:0/1693497009 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108003e50 con 0x7f71101451c0 2023-08-27T11:26:20.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f7114ba1700 1 -- 172.21.15.112:0/1693497009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7110143bf0 con 0x7f71101451c0 2023-08-27T11:26:20.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f70feffd700 1 -- 172.21.15.112:0/1693497009 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108014b50 con 0x7f71101451c0 2023-08-27T11:26:20.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.250+0000 7f7114ba1700 1 -- 172.21.15.112:0/1693497009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71101404d0 con 0x7f71101451c0 2023-08-27T11:26:20.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.251+0000 7f7114ba1700 1 -- 172.21.15.112:0/1693497009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7110067e90 con 0x7f71101451c0 2023-08-27T11:26:20.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.252+0000 7f70feffd700 1 -- 172.21.15.112:0/1693497009 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f7108023020 con 0x7f71101451c0 2023-08-27T11:26:20.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.252+0000 7f70feffd700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f70f006a3b0 0x7f70f006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:20.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.252+0000 7f70feffd700 1 -- 172.21.15.112:0/1693497009 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7108089ed0 con 0x7f71101451c0 2023-08-27T11:26:20.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.252+0000 7f710e59c700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f70f006a3b0 0x7f70f006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:20.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.253+0000 7f710e59c700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f70f006a3b0 0x7f70f006c870 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7104001050 tx=0x7f7104009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:20.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.257+0000 7f70feffd700 1 -- 172.21.15.112:0/1693497009 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7108055420 con 0x7f71101451c0 2023-08-27T11:26:20.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:20.491+0000 7f7114ba1700 1 -- 172.21.15.112:0/1693497009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7110067a20 con 0x7f71101451c0 2023-08-27T11:26:21.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.433+0000 7f70feffd700 1 -- 172.21.15.112:0/1693497009 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7f71080180b0 con 0x7f71101451c0 2023-08-27T11:26:21.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.436+0000 7f7114ba1700 1 -- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f70f006a3b0 msgr2=0x7f70f006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:21.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.436+0000 7f7114ba1700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f70f006a3b0 0x7f70f006c870 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7104001050 tx=0x7f7104009040 comp rx=0 tx=0).stop 2023-08-27T11:26:21.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.436+0000 7f7114ba1700 1 -- 172.21.15.112:0/1693497009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71101451c0 msgr2=0x7f71100aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:21.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.436+0000 7f7114ba1700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71101451c0 0x7f71100aea10 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f7108017040 tx=0x7f7108009750 comp rx=0 tx=0).stop 2023-08-27T11:26:21.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.436+0000 7f7114ba1700 1 -- 172.21.15.112:0/1693497009 shutdown_connections 2023-08-27T11:26:21.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.436+0000 7f7114ba1700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f70f006a3b0 0x7f70f006c870 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:21.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.436+0000 7f7114ba1700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71101451c0 0x7f71100aea10 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:21.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.436+0000 7f7114ba1700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70f8004d90 0x7f7110144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:21.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.436+0000 7f7114ba1700 1 --2- 172.21.15.112:0/1693497009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7110144650 0x7f71100ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:21.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.436+0000 7f7114ba1700 1 -- 172.21.15.112:0/1693497009 >> 172.21.15.112:0/1693497009 conn(0x7f71100b1e90 msgr2=0x7f711015ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:21.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.436+0000 7f7114ba1700 1 -- 172.21.15.112:0/1693497009 shutdown_connections 2023-08-27T11:26:21.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.436+0000 7f7114ba1700 1 -- 172.21.15.112:0/1693497009 wait complete. 2023-08-27T11:26:21.451 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:26:21.452 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:26:21.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.577+0000 7f8381e81700 1 Processor -- start 2023-08-27T11:26:21.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.577+0000 7f8381e81700 1 -- start start 2023-08-27T11:26:21.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f8381e81700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f837c0cb090 0x7f837c0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:21.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f8381e81700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f837c15a5d0 con 0x7f837c15a1f0 2023-08-27T11:26:21.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f8381e81700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f837c0aef20 con 0x7f837c070820 2023-08-27T11:26:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f8381e81700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f837c0cba80 con 0x7f837c0cb090 2023-08-27T11:26:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f837affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f837c0cb090 0x7f837c0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f837b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f837c15a1f0 0x7f837c0ae810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55462/0 (socket says 172.21.15.112:55462) 2023-08-27T11:26:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f837b7fe700 1 -- 172.21.15.112:0/2116737888 learned_addr learned my addr 172.21.15.112:0/2116737888 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f837affd700 1 -- 172.21.15.112:0/2116737888 >> v1:172.21.15.138:6789/0 conn(0x7f837c070820 legacy=0x7f837c070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f837affd700 1 -- 172.21.15.112:0/2116737888 >> v1:172.21.15.112:6789/0 conn(0x7f837c15a1f0 legacy=0x7f837c0ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f837affd700 1 -- 172.21.15.112:0/2116737888 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c0cddd0 con 0x7f837c0cb090 2023-08-27T11:26:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f837a7fc700 1 -- 172.21.15.112:0/2116737888 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127204509 0 0) 0x7f837c15a5d0 con 0x7f837c15a1f0 2023-08-27T11:26:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f837affd700 1 --2- 172.21.15.112:0/2116737888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f837c0cb090 0x7f837c0cb540 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f836c009840 tx=0x7f836c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f794e9f7c4c959c8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f837a7fc700 1 -- 172.21.15.112:0/2116737888 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c004230 con 0x7f837c0cb090 2023-08-27T11:26:21.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.578+0000 7f837a7fc700 1 -- 172.21.15.112:0/2116737888 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836c004390 con 0x7f837c0cb090 2023-08-27T11:26:21.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.579+0000 7f837a7fc700 1 -- 172.21.15.112:0/2116737888 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c0036a0 con 0x7f837c0cb090 2023-08-27T11:26:21.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.579+0000 7f8381e81700 1 -- 172.21.15.112:0/2116737888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f837c0cb090 msgr2=0x7f837c0cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:21.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.579+0000 7f8381e81700 1 --2- 172.21.15.112:0/2116737888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f837c0cb090 0x7f837c0cb540 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f836c009840 tx=0x7f836c009b50 comp rx=0 tx=0).stop 2023-08-27T11:26:21.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.579+0000 7f8381e81700 1 -- 172.21.15.112:0/2116737888 shutdown_connections 2023-08-27T11:26:21.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.579+0000 7f8381e81700 1 --2- 172.21.15.112:0/2116737888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f837c0cb090 0x7f837c0cb540 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:21.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.579+0000 7f8381e81700 1 -- 172.21.15.112:0/2116737888 >> 172.21.15.112:0/2116737888 conn(0x7f837c0cf750 msgr2=0x7f837c0cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:21.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.579+0000 7f8381e81700 1 -- 172.21.15.112:0/2116737888 shutdown_connections 2023-08-27T11:26:21.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.579+0000 7f8381e81700 1 -- 172.21.15.112:0/2116737888 wait complete. 2023-08-27T11:26:21.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.580+0000 7f8381e81700 1 Processor -- start 2023-08-27T11:26:21.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.580+0000 7f8381e81700 1 -- start start 2023-08-27T11:26:21.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f8381e81700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f837c070820 0x7f837c14d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:21.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f8381e81700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f837c0cb090 0x7f837c1531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:21.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f8381e81700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f837c15a1f0 0x7f837c1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:21.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f8381e81700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c15c830 con 0x7f837c0cb090 2023-08-27T11:26:21.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f8381e81700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c15cb10 con 0x7f837c15a1f0 2023-08-27T11:26:21.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f8381e81700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837c15cdf0 con 0x7f837c070820 2023-08-27T11:26:21.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f837affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f837c0cb090 0x7f837c1531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:21.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f837affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f837c0cb090 0x7f837c1531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32856/0 (socket says 172.21.15.112:32856) 2023-08-27T11:26:21.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f837affd700 1 -- 172.21.15.112:0/662758417 learned_addr learned my addr 172.21.15.112:0/662758417 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:21.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f837bfff700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f837c15a1f0 0x7f837c1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:21.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f837b7fe700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f837c070820 0x7f837c14d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:21.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f837affd700 1 -- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f837c070820 msgr2=0x7f837c14d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:21.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f837affd700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f837c070820 0x7f837c14d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:21.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f837affd700 1 -- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f837c15a1f0 msgr2=0x7f837c1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:21.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f837affd700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f837c15a1f0 0x7f837c1536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:21.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f837affd700 1 -- 172.21.15.112:0/662758417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c0094a0 con 0x7f837c0cb090 2023-08-27T11:26:21.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f837bfff700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f837c15a1f0 0x7f837c1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:21.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.581+0000 7f837b7fe700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f837c070820 0x7f837c14d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:21.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.582+0000 7f837affd700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f837c0cb090 0x7f837c1531b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f836c0090b0 tx=0x7f836c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:21.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.582+0000 7f8373fff700 1 -- 172.21.15.112:0/662758417 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c011830 con 0x7f837c0cb090 2023-08-27T11:26:21.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.582+0000 7f8373fff700 1 -- 172.21.15.112:0/662758417 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836c01e070 con 0x7f837c0cb090 2023-08-27T11:26:21.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.582+0000 7f8381e81700 1 -- 172.21.15.112:0/662758417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837c15d0d0 con 0x7f837c0cb090 2023-08-27T11:26:21.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.582+0000 7f8373fff700 1 -- 172.21.15.112:0/662758417 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c011bb0 con 0x7f837c0cb090 2023-08-27T11:26:21.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.582+0000 7f8381e81700 1 -- 172.21.15.112:0/662758417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837c14eac0 con 0x7f837c0cb090 2023-08-27T11:26:21.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.583+0000 7f8381e81700 1 -- 172.21.15.112:0/662758417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837c15d2e0 con 0x7f837c0cb090 2023-08-27T11:26:21.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.583+0000 7f8373fff700 1 -- 172.21.15.112:0/662758417 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f836c011d10 con 0x7f837c0cb090 2023-08-27T11:26:21.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.583+0000 7f8373fff700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f835c06a3b0 0x7f835c06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.584+0000 7f8373fff700 1 -- 172.21.15.112:0/662758417 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f836c089de0 con 0x7f837c0cb090 2023-08-27T11:26:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.584+0000 7f837b7fe700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f835c06a3b0 0x7f835c06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.584+0000 7f837b7fe700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f835c06a3b0 0x7f835c06c870 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8364000f60 tx=0x7f8364009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.588+0000 7f8373fff700 1 -- 172.21.15.112:0/662758417 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f836c055130 con 0x7f837c0cb090 2023-08-27T11:26:21.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:21.822+0000 7f8381e81700 1 -- 172.21.15.112:0/662758417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f837c0ce080 con 0x7f837c0cb090 2023-08-27T11:26:22.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.438+0000 7f8373fff700 1 -- 172.21.15.112:0/662758417 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f836c058750 con 0x7f837c0cb090 2023-08-27T11:26:22.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.441+0000 7f8381e81700 1 -- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f835c06a3b0 msgr2=0x7f835c06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:22.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.441+0000 7f8381e81700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f835c06a3b0 0x7f835c06c870 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8364000f60 tx=0x7f8364009040 comp rx=0 tx=0).stop 2023-08-27T11:26:22.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.441+0000 7f8381e81700 1 -- 172.21.15.112:0/662758417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f837c0cb090 msgr2=0x7f837c1531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:22.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.441+0000 7f8381e81700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f837c0cb090 0x7f837c1531b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f836c0090b0 tx=0x7f836c004750 comp rx=0 tx=0).stop 2023-08-27T11:26:22.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.441+0000 7f8381e81700 1 -- 172.21.15.112:0/662758417 shutdown_connections 2023-08-27T11:26:22.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.441+0000 7f8381e81700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f835c06a3b0 0x7f835c06c870 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.441+0000 7f8381e81700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f837c15a1f0 0x7f837c1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.441+0000 7f8381e81700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f837c070820 0x7f837c14d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.441+0000 7f8381e81700 1 --2- 172.21.15.112:0/662758417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f837c0cb090 0x7f837c1531b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.441+0000 7f8381e81700 1 -- 172.21.15.112:0/662758417 >> 172.21.15.112:0/662758417 conn(0x7f837c0cf750 msgr2=0x7f837c06cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:22.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.441+0000 7f8381e81700 1 -- 172.21.15.112:0/662758417 shutdown_connections 2023-08-27T11:26:22.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.441+0000 7f8381e81700 1 -- 172.21.15.112:0/662758417 wait complete. 2023-08-27T11:26:22.456 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:26:22.457 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:26:22.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.577+0000 7f95c71af700 1 Processor -- start 2023-08-27T11:26:22.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.577+0000 7f95c71af700 1 -- start start 2023-08-27T11:26:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95c71af700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f95c00b3e50 0x7f95c015f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95c71af700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c00b45f0 con 0x7f95c00b4ad0 2023-08-27T11:26:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95c71af700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c00b4310 con 0x7f95c0144570 2023-08-27T11:26:22.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95c71af700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c00cf7a0 con 0x7f95c00b3e50 2023-08-27T11:26:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95bffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f95c00b3e50 0x7f95c015f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95bffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f95c00b3e50 0x7f95c015f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32864/0 (socket says 172.21.15.112:32864) 2023-08-27T11:26:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95c574c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f95c00b4ad0 0x7f95c00b4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55480/0 (socket says 172.21.15.112:55480) 2023-08-27T11:26:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95bffff700 1 -- 172.21.15.112:0/4131856308 learned_addr learned my addr 172.21.15.112:0/4131856308 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:22.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95bffff700 1 -- 172.21.15.112:0/4131856308 >> v1:172.21.15.138:6789/0 conn(0x7f95c0144570 legacy=0x7f95c015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:22.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95bffff700 1 -- 172.21.15.112:0/4131856308 >> v1:172.21.15.112:6789/0 conn(0x7f95c00b4ad0 legacy=0x7f95c00b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:22.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95bf7fe700 1 -- 172.21.15.112:0/4131856308 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700792720 0 0) 0x7f95c00b45f0 con 0x7f95c00b4ad0 2023-08-27T11:26:22.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95bffff700 1 -- 172.21.15.112:0/4131856308 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c00cdcb0 con 0x7f95c00b3e50 2023-08-27T11:26:22.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95bffff700 1 --2- 172.21.15.112:0/4131856308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b3e50 0x7f95c015f530 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0098f0 tx=0x7f95ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c16af33da37f826 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:22.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95bf7fe700 1 -- 172.21.15.112:0/4131856308 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac004230 con 0x7f95c00b3e50 2023-08-27T11:26:22.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95bf7fe700 1 -- 172.21.15.112:0/4131856308 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac004390 con 0x7f95c00b3e50 2023-08-27T11:26:22.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.578+0000 7f95bf7fe700 1 -- 172.21.15.112:0/4131856308 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0036a0 con 0x7f95c00b3e50 2023-08-27T11:26:22.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.579+0000 7f95c71af700 1 -- 172.21.15.112:0/4131856308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b3e50 msgr2=0x7f95c015f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:22.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.579+0000 7f95c71af700 1 --2- 172.21.15.112:0/4131856308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b3e50 0x7f95c015f530 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0098f0 tx=0x7f95ac009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:22.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.579+0000 7f95c71af700 1 -- 172.21.15.112:0/4131856308 shutdown_connections 2023-08-27T11:26:22.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.579+0000 7f95c71af700 1 --2- 172.21.15.112:0/4131856308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b3e50 0x7f95c015f530 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.579+0000 7f95c71af700 1 -- 172.21.15.112:0/4131856308 >> 172.21.15.112:0/4131856308 conn(0x7f95c006f970 msgr2=0x7f95c006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:22.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.579+0000 7f95c71af700 1 -- 172.21.15.112:0/4131856308 shutdown_connections 2023-08-27T11:26:22.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.579+0000 7f95c71af700 1 -- 172.21.15.112:0/4131856308 wait complete. 2023-08-27T11:26:22.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.580+0000 7f95c71af700 1 Processor -- start 2023-08-27T11:26:22.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.580+0000 7f95c71af700 1 -- start start 2023-08-27T11:26:22.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.581+0000 7f95c71af700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95c00b3e50 0x7f95c00cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.581+0000 7f95c71af700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95c00b4ad0 0x7f95c014e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.581+0000 7f95c71af700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c0144570 0x7f95c014ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.581+0000 7f95c71af700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c0071630 con 0x7f95c0144570 2023-08-27T11:26:22.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.581+0000 7f95c71af700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c0071910 con 0x7f95c00b3e50 2023-08-27T11:26:22.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.581+0000 7f95c71af700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c0071bf0 con 0x7f95c00b4ad0 2023-08-27T11:26:22.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.581+0000 7f95c574c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c0144570 0x7f95c014ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.581+0000 7f95c574c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c0144570 0x7f95c014ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32872/0 (socket says 172.21.15.112:32872) 2023-08-27T11:26:22.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.581+0000 7f95c574c700 1 -- 172.21.15.112:0/1819990591 learned_addr learned my addr 172.21.15.112:0/1819990591 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:22.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95c4f4b700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95c00b3e50 0x7f95c00cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95c574c700 1 -- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95c00b4ad0 msgr2=0x7f95c014e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:22.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95bffff700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95c00b4ad0 0x7f95c014e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95c574c700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95c00b4ad0 0x7f95c014e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95c574c700 1 -- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95c00b3e50 msgr2=0x7f95c00cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:22.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95c574c700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95c00b3e50 0x7f95c00cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95c574c700 1 -- 172.21.15.112:0/1819990591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac009550 con 0x7f95c0144570 2023-08-27T11:26:22.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95c4f4b700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95c00b3e50 0x7f95c00cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:22.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95c574c700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c0144570 0x7f95c014ea50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f95b800cca0 tx=0x7f95b800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:22.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95bd7fa700 1 -- 172.21.15.112:0/1819990591 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b800cea0 con 0x7f95c0144570 2023-08-27T11:26:22.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95bd7fa700 1 -- 172.21.15.112:0/1819990591 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b800ade0 con 0x7f95c0144570 2023-08-27T11:26:22.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95bd7fa700 1 -- 172.21.15.112:0/1819990591 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8007310 con 0x7f95c0144570 2023-08-27T11:26:22.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95c71af700 1 -- 172.21.15.112:0/1819990591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c0071ed0 con 0x7f95c0144570 2023-08-27T11:26:22.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.582+0000 7f95c71af700 1 -- 172.21.15.112:0/1819990591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c00cd570 con 0x7f95c0144570 2023-08-27T11:26:22.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.583+0000 7f95c71af700 1 -- 172.21.15.112:0/1819990591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c0067a20 con 0x7f95c0144570 2023-08-27T11:26:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.583+0000 7f95bd7fa700 1 -- 172.21.15.112:0/1819990591 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f95b8007470 con 0x7f95c0144570 2023-08-27T11:26:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.584+0000 7f95bd7fa700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95a406a3b0 0x7f95a406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.584+0000 7f95bd7fa700 1 -- 172.21.15.112:0/1819990591 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f95b80886b0 con 0x7f95c0144570 2023-08-27T11:26:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.584+0000 7f95c4f4b700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95a406a3b0 0x7f95a406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.584+0000 7f95c4f4b700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95a406a3b0 0x7f95a406c870 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f95b4006fd0 tx=0x7f95b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.588+0000 7f95bd7fa700 1 -- 172.21.15.112:0/1819990591 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f95b8053a00 con 0x7f95c0144570 2023-08-27T11:26:22.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.816+0000 7f95c71af700 1 -- 172.21.15.112:0/1819990591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95c00cd760 con 0x7f95c0144570 2023-08-27T11:26:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.816+0000 7f95bd7fa700 1 -- 172.21.15.112:0/1819990591 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4886 (secure 0 0 0) 0x7f95b8057020 con 0x7f95c0144570 2023-08-27T11:26:22.818 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:22.818 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/120353149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":120353149},{"type":"v1","addr":"172.21.15.138:6836","nonce":120353149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1171600247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1171600247},{"type":"v1","addr":"172.21.15.138:6837","nonce":1171600247}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3930299696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3930299696},{"type":"v1","addr":"172.21.15.112:6834","nonce":3930299696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:26:18.805364+0000","modified":"2023-08-27T11:26:22.433680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7041":{"gid":7041,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2614833373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2614833373},{"type":"v1","addr":"172.21.15.112:6835","nonce":2614833373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-27T11:26:22.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.818+0000 7f95c71af700 1 -- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95a406a3b0 msgr2=0x7f95a406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:22.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.818+0000 7f95c71af700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95a406a3b0 0x7f95a406c870 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f95b4006fd0 tx=0x7f95b4008040 comp rx=0 tx=0).stop 2023-08-27T11:26:22.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.818+0000 7f95c71af700 1 -- 172.21.15.112:0/1819990591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c0144570 msgr2=0x7f95c014ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:22.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.818+0000 7f95c71af700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c0144570 0x7f95c014ea50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f95b800cca0 tx=0x7f95b800a340 comp rx=0 tx=0).stop 2023-08-27T11:26:22.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.818+0000 7f95c71af700 1 -- 172.21.15.112:0/1819990591 shutdown_connections 2023-08-27T11:26:22.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.818+0000 7f95c71af700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95a406a3b0 0x7f95a406c870 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.818+0000 7f95c71af700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c0144570 0x7f95c014ea50 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.818+0000 7f95c71af700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95c00b3e50 0x7f95c00cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.818+0000 7f95c71af700 1 --2- 172.21.15.112:0/1819990591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95c00b4ad0 0x7f95c014e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.818+0000 7f95c71af700 1 -- 172.21.15.112:0/1819990591 >> 172.21.15.112:0/1819990591 conn(0x7f95c006f970 msgr2=0x7f95c006ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.819+0000 7f95c71af700 1 -- 172.21.15.112:0/1819990591 shutdown_connections 2023-08-27T11:26:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.819+0000 7f95c71af700 1 -- 172.21.15.112:0/1819990591 wait complete. 2023-08-27T11:26:22.823 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 58 2023-08-27T11:26:22.832 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:22.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.950+0000 7fec43e8e700 1 Processor -- start 2023-08-27T11:26:22.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.950+0000 7fec43e8e700 1 -- start start 2023-08-27T11:26:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.951+0000 7fec43e8e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fec3c0b1400 0x7fec3c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.951+0000 7fec43e8e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec3c161ce0 0x7fec3c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.951+0000 7fec43e8e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3c161850 con 0x7fec3c0b17e0 2023-08-27T11:26:22.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.951+0000 7fec43e8e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec3c0cfc10 con 0x7fec3c0b1400 2023-08-27T11:26:22.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.951+0000 7fec43e8e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c161200 con 0x7fec3c161ce0 2023-08-27T11:26:22.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.951+0000 7fec41429700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fec3c0b17e0 0x7fec3c06e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55490/0 (socket says 172.21.15.112:55490) 2023-08-27T11:26:22.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.951+0000 7fec41429700 1 -- 172.21.15.112:0/2652137771 learned_addr learned my addr 172.21.15.112:0/2652137771 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:22.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.951+0000 7fec4242b700 1 --2- 172.21.15.112:0/2652137771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec3c161ce0 0x7fec3c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.951+0000 7fec41c2a700 1 --2- 172.21.15.112:0/2652137771 >> v2:172.21.15.138:3300/0 conn(0x7fec3c0b1400 0x7fec3c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.951+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191917252 0 0) 0x7fec3c161850 con 0x7fec3c0b17e0 2023-08-27T11:26:22.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.951+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec280035b0 con 0x7fec3c0b17e0 2023-08-27T11:26:22.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3409725742 0 0) 0x7fec280035b0 con 0x7fec3c0b17e0 2023-08-27T11:26:22.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec3c161ce0 msgr2=0x7fec3c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:22.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 --2- 172.21.15.112:0/2652137771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec3c161ce0 0x7fec3c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 >> v2:172.21.15.138:3300/0 conn(0x7fec3c0b1400 msgr2=0x7fec3c0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:22.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 --2- 172.21.15.112:0/2652137771 >> v2:172.21.15.138:3300/0 conn(0x7fec3c0b1400 0x7fec3c0cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c1614e0 con 0x7fec3c0b17e0 2023-08-27T11:26:22.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec41c2a700 1 --2- 172.21.15.112:0/2652137771 >> v2:172.21.15.138:3300/0 conn(0x7fec3c0b1400 0x7fec3c0cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:22.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fec38001670 con 0x7fec3c0b17e0 2023-08-27T11:26:22.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 >> v1:172.21.15.112:6789/0 conn(0x7fec3c0b17e0 legacy=0x7fec3c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:22.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 --2- 172.21.15.112:0/2652137771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec28003a00 0x7fec28003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 --2- 172.21.15.112:0/2652137771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec3c161ce0 0x7fec28004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 --2- 172.21.15.112:0/2652137771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec28004d90 0x7fec28005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0cfc10 con 0x7fec28003a00 2023-08-27T11:26:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c161200 con 0x7fec28004d90 2023-08-27T11:26:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2800bac0 con 0x7fec3c161ce0 2023-08-27T11:26:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec41c2a700 1 --2- 172.21.15.112:0/2652137771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec28003a00 0x7fec28003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec41429700 1 --2- 172.21.15.112:0/2652137771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec3c161ce0 0x7fec28004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec4242b700 1 --2- 172.21.15.112:0/2652137771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec28004d90 0x7fec28005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec41c2a700 1 -- 172.21.15.112:0/2652137771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec3c161ce0 msgr2=0x7fec28004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:22.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec41c2a700 1 --2- 172.21.15.112:0/2652137771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec3c161ce0 0x7fec28004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec41c2a700 1 -- 172.21.15.112:0/2652137771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec28004d90 msgr2=0x7fec28005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:22.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec41c2a700 1 --2- 172.21.15.112:0/2652137771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec28004d90 0x7fec28005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec41c2a700 1 -- 172.21.15.112:0/2652137771 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec280035b0 con 0x7fec28003a00 2023-08-27T11:26:22.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec4242b700 1 --2- 172.21.15.112:0/2652137771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec28004d90 0x7fec28005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:22.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec41c2a700 1 --2- 172.21.15.112:0/2652137771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec28003a00 0x7fec28003e20 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009d30 tx=0x7fec28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddfa67874d92bff server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:22.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c009e80 con 0x7fec28003a00 2023-08-27T11:26:22.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c00e9a0 con 0x7fec28003a00 2023-08-27T11:26:22.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.952+0000 7fec40c28700 1 -- 172.21.15.112:0/2652137771 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00ed30 con 0x7fec28003a00 2023-08-27T11:26:22.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.953+0000 7fec43e8e700 1 -- 172.21.15.112:0/2652137771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec28003a00 msgr2=0x7fec28003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:22.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.953+0000 7fec43e8e700 1 --2- 172.21.15.112:0/2652137771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec28003a00 0x7fec28003e20 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009d30 tx=0x7fec28005880 comp rx=0 tx=0).stop 2023-08-27T11:26:22.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.953+0000 7fec4242b700 1 -- 172.21.15.112:0/2652137771 reap_dead start 2023-08-27T11:26:22.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.953+0000 7fec43e8e700 1 -- 172.21.15.112:0/2652137771 shutdown_connections 2023-08-27T11:26:22.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.953+0000 7fec43e8e700 1 -- 172.21.15.112:0/2652137771 >> 172.21.15.112:0/2652137771 conn(0x7fec3c0d0000 msgr2=0x7fec3c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:22.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.953+0000 7fec43e8e700 1 -- 172.21.15.112:0/2652137771 shutdown_connections 2023-08-27T11:26:22.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.953+0000 7fec43e8e700 1 -- 172.21.15.112:0/2652137771 wait complete. 2023-08-27T11:26:22.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.954+0000 7fec43e8e700 1 Processor -- start 2023-08-27T11:26:22.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.954+0000 7fec43e8e700 1 -- start start 2023-08-27T11:26:22.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec43e8e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec28003a00 0x7fec3c0c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec43e8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c1496a0 0x7fec3c1f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec43e8e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec3c149b20 0x7fec3c1f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec43e8e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0c9db0 con 0x7fec3c1496a0 2023-08-27T11:26:22.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec43e8e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c159a80 con 0x7fec3c149b20 2023-08-27T11:26:22.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec43e8e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0c5fa0 con 0x7fec28003a00 2023-08-27T11:26:22.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec41429700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c1496a0 0x7fec3c1f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec41429700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c1496a0 0x7fec3c1f1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32888/0 (socket says 172.21.15.112:32888) 2023-08-27T11:26:22.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec41429700 1 -- 172.21.15.112:0/3115701580 learned_addr learned my addr 172.21.15.112:0/3115701580 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:22.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec41c2a700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec28003a00 0x7fec3c0c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec4242b700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec3c149b20 0x7fec3c1f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec41429700 1 -- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec28003a00 msgr2=0x7fec3c0c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:22.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec41429700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec28003a00 0x7fec3c0c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec41429700 1 -- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec3c149b20 msgr2=0x7fec3c1f24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:22.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec41429700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec3c149b20 0x7fec3c1f24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:22.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec41429700 1 -- 172.21.15.112:0/3115701580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c0c6280 con 0x7fec3c1496a0 2023-08-27T11:26:22.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec4242b700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec3c149b20 0x7fec3c1f24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:22.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.955+0000 7fec41429700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c1496a0 0x7fec3c1f1f70 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fec38000fb0 tx=0x7fec380075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:22.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.956+0000 7fec327fc700 1 -- 172.21.15.112:0/3115701580 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3800f5e0 con 0x7fec3c1496a0 2023-08-27T11:26:22.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.956+0000 7fec327fc700 1 -- 172.21.15.112:0/3115701580 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3800f780 con 0x7fec3c1496a0 2023-08-27T11:26:22.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.956+0000 7fec327fc700 1 -- 172.21.15.112:0/3115701580 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3800fb30 con 0x7fec3c1496a0 2023-08-27T11:26:22.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.956+0000 7fec43e8e700 1 -- 172.21.15.112:0/3115701580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c0c6560 con 0x7fec3c1496a0 2023-08-27T11:26:22.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.956+0000 7fec43e8e700 1 -- 172.21.15.112:0/3115701580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c1f29f0 con 0x7fec3c1496a0 2023-08-27T11:26:22.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.957+0000 7fec43e8e700 1 -- 172.21.15.112:0/3115701580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3c067a20 con 0x7fec3c1496a0 2023-08-27T11:26:22.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.957+0000 7fec327fc700 1 -- 172.21.15.112:0/3115701580 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fec3800fc90 con 0x7fec3c1496a0 2023-08-27T11:26:22.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.958+0000 7fec327fc700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fec2006a3b0 0x7fec2006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:22.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.958+0000 7fec327fc700 1 -- 172.21.15.112:0/3115701580 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fec380889b0 con 0x7fec3c1496a0 2023-08-27T11:26:22.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.958+0000 7fec41c2a700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fec2006a3b0 0x7fec2006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:22.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.958+0000 7fec41c2a700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fec2006a3b0 0x7fec2006c870 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fec2c000c00 tx=0x7fec2c01f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:22.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:22.962+0000 7fec327fc700 1 -- 172.21.15.112:0/3115701580 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fec38053d50 con 0x7fec3c1496a0 2023-08-27T11:26:23.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.151+0000 7fec43e8e700 1 -- 172.21.15.112:0/3115701580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec3c0ca330 con 0x7fec3c1496a0 2023-08-27T11:26:23.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.152+0000 7fec327fc700 1 -- 172.21.15.112:0/3115701580 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+18248 (secure 0 0 0) 0x7fec38057370 con 0x7fec3c1496a0 2023-08-27T11:26:23.154 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:23.154 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:20.416307+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:16.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:17.773993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:23.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.154+0000 7fec43e8e700 1 -- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fec2006a3b0 msgr2=0x7fec2006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.154+0000 7fec43e8e700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fec2006a3b0 0x7fec2006c870 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fec2c000c00 tx=0x7fec2c01f040 comp rx=0 tx=0).stop 2023-08-27T11:26:23.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.154+0000 7fec43e8e700 1 -- 172.21.15.112:0/3115701580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c1496a0 msgr2=0x7fec3c1f1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.154+0000 7fec43e8e700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c1496a0 0x7fec3c1f1f70 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fec38000fb0 tx=0x7fec380075f0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.154+0000 7fec43e8e700 1 -- 172.21.15.112:0/3115701580 shutdown_connections 2023-08-27T11:26:23.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.154+0000 7fec43e8e700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fec2006a3b0 0x7fec2006c870 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.154+0000 7fec43e8e700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec3c149b20 0x7fec3c1f24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.154+0000 7fec43e8e700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec28003a00 0x7fec3c0c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.154+0000 7fec43e8e700 1 --2- 172.21.15.112:0/3115701580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c1496a0 0x7fec3c1f1f70 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.154+0000 7fec43e8e700 1 -- 172.21.15.112:0/3115701580 >> 172.21.15.112:0/3115701580 conn(0x7fec3c0d0000 msgr2=0x7fec3c15cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:23.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.154+0000 7fec43e8e700 1 -- 172.21.15.112:0/3115701580 shutdown_connections 2023-08-27T11:26:23.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.154+0000 7fec43e8e700 1 -- 172.21.15.112:0/3115701580 wait complete. 2023-08-27T11:26:23.169 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:26:23.170 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:26:23.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.288+0000 7f5fcfd7f700 1 Processor -- start 2023-08-27T11:26:23.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.288+0000 7f5fcfd7f700 1 -- start start 2023-08-27T11:26:23.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.289+0000 7f5fcfd7f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5fc8151060 0x7f5fc815e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:23.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.289+0000 7f5fcfd7f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fc815e990 0x7f5fc806adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:23.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.289+0000 7f5fcfd7f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc80b02f0 con 0x7f5fc806b2f0 2023-08-27T11:26:23.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.289+0000 7f5fcfd7f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc806b780 con 0x7f5fc8151060 2023-08-27T11:26:23.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.289+0000 7f5fcfd7f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc80b05d0 con 0x7f5fc815e990 2023-08-27T11:26:23.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.289+0000 7f5fcd31a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fc815e990 0x7f5fc806adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.289+0000 7f5fcd31a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fc815e990 0x7f5fc806adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:44552/0 (socket says 172.21.15.112:44552) 2023-08-27T11:26:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.289+0000 7f5fcd31a700 1 -- 172.21.15.112:0/2787111656 learned_addr learned my addr 172.21.15.112:0/2787111656 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.289+0000 7f5fcdb1b700 1 --2- 172.21.15.112:0/2787111656 >> v2:172.21.15.138:3300/0 conn(0x7f5fc8151060 0x7f5fc815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.290+0000 7f5fcd31a700 1 -- 172.21.15.112:0/2787111656 >> v2:172.21.15.138:3300/0 conn(0x7f5fc8151060 msgr2=0x7f5fc815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:23.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.290+0000 7f5fcd31a700 1 --2- 172.21.15.112:0/2787111656 >> v2:172.21.15.138:3300/0 conn(0x7f5fc8151060 0x7f5fc815e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.290+0000 7f5fcd31a700 1 -- 172.21.15.112:0/2787111656 >> v1:172.21.15.138:6789/0 conn(0x7f5fc806b2f0 legacy=0x7f5fc806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.290+0000 7f5fcd31a700 1 -- 172.21.15.112:0/2787111656 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc80b08b0 con 0x7f5fc815e990 2023-08-27T11:26:23.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.290+0000 7f5fcdb1b700 1 --2- 172.21.15.112:0/2787111656 >> v2:172.21.15.138:3300/0 conn(0x7f5fc8151060 0x7f5fc815e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:23.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.290+0000 7f5fcd31a700 1 --2- 172.21.15.112:0/2787111656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fc815e990 0x7f5fc806adb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40098f0 tx=0x7f5fc4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=575c6d9c0ccfe34d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:23.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.290+0000 7f5fccb19700 1 -- 172.21.15.112:0/2787111656 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4011830 con 0x7f5fc815e990 2023-08-27T11:26:23.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.290+0000 7f5fccb19700 1 -- 172.21.15.112:0/2787111656 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc4011990 con 0x7f5fc815e990 2023-08-27T11:26:23.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.290+0000 7f5fccb19700 1 -- 172.21.15.112:0/2787111656 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4011d20 con 0x7f5fc815e990 2023-08-27T11:26:23.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.290+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2787111656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fc815e990 msgr2=0x7f5fc806adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.291+0000 7f5fcfd7f700 1 --2- 172.21.15.112:0/2787111656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fc815e990 0x7f5fc806adb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40098f0 tx=0x7f5fc4009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:23.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.291+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2787111656 shutdown_connections 2023-08-27T11:26:23.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.291+0000 7f5fcfd7f700 1 --2- 172.21.15.112:0/2787111656 >> v2:172.21.15.138:3300/0 conn(0x7f5fc8151060 0x7f5fc815e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.291+0000 7f5fcfd7f700 1 --2- 172.21.15.112:0/2787111656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fc815e990 0x7f5fc806adb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.291+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2787111656 >> 172.21.15.112:0/2787111656 conn(0x7f5fc80d0000 msgr2=0x7f5fc80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:23.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.291+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2787111656 shutdown_connections 2023-08-27T11:26:23.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.291+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2787111656 wait complete. 2023-08-27T11:26:23.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.292+0000 7f5fcfd7f700 1 Processor -- start 2023-08-27T11:26:23.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.292+0000 7f5fcfd7f700 1 -- start start 2023-08-27T11:26:23.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fcfd7f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fc806b2f0 0x7f5fc815a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:23.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fcfd7f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fc8151060 0x7f5fc8071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:23.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fcfd7f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fc815e990 0x7f5fc8072020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:23.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fcfd7f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc8142150 con 0x7f5fc815e990 2023-08-27T11:26:23.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fcfd7f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc80c4720 con 0x7f5fc806b2f0 2023-08-27T11:26:23.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fcfd7f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc80c4a00 con 0x7f5fc8151060 2023-08-27T11:26:23.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fce31c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fc815e990 0x7f5fc8072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:23.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fcd31a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fc8151060 0x7f5fc8071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:23.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fcdb1b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fc806b2f0 0x7f5fc815a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:23.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fce31c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fc815e990 0x7f5fc8072020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32906/0 (socket says 172.21.15.112:32906) 2023-08-27T11:26:23.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fce31c700 1 -- 172.21.15.112:0/2144699945 learned_addr learned my addr 172.21.15.112:0/2144699945 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:23.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fce31c700 1 -- 172.21.15.112:0/2144699945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fc8151060 msgr2=0x7f5fc8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fce31c700 1 --2- 172.21.15.112:0/2144699945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fc8151060 0x7f5fc8071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fce31c700 1 -- 172.21.15.112:0/2144699945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fc806b2f0 msgr2=0x7f5fc815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fce31c700 1 --2- 172.21.15.112:0/2144699945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fc806b2f0 0x7f5fc815a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fce31c700 1 -- 172.21.15.112:0/2144699945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb80094a0 con 0x7f5fc815e990 2023-08-27T11:26:23.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fcdb1b700 1 --2- 172.21.15.112:0/2144699945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fc806b2f0 0x7f5fc815a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:23.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fce31c700 1 --2- 172.21.15.112:0/2144699945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fc815e990 0x7f5fc8072020 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000cb50 tx=0x7f5fc000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:23.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fbe7fc700 1 -- 172.21.15.112:0/2144699945 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0003c00 con 0x7f5fc815e990 2023-08-27T11:26:23.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fbe7fc700 1 -- 172.21.15.112:0/2144699945 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc0003d60 con 0x7f5fc815e990 2023-08-27T11:26:23.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2144699945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc4009550 con 0x7f5fc815e990 2023-08-27T11:26:23.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fbe7fc700 1 -- 172.21.15.112:0/2144699945 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc000aa20 con 0x7f5fc815e990 2023-08-27T11:26:23.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.293+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2144699945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc80c4fc0 con 0x7f5fc815e990 2023-08-27T11:26:23.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.294+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2144699945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc8067a20 con 0x7f5fc815e990 2023-08-27T11:26:23.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.295+0000 7f5fbe7fc700 1 -- 172.21.15.112:0/2144699945 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f5fc000ab80 con 0x7f5fc815e990 2023-08-27T11:26:23.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.295+0000 7f5fbe7fc700 1 --2- 172.21.15.112:0/2144699945 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fac06a3b0 0x7f5fac06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:23.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.295+0000 7f5fbe7fc700 1 -- 172.21.15.112:0/2144699945 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5fc00889a0 con 0x7f5fc815e990 2023-08-27T11:26:23.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.295+0000 7f5fcdb1b700 1 --2- 172.21.15.112:0/2144699945 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fac06a3b0 0x7f5fac06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:23.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.296+0000 7f5fcdb1b700 1 --2- 172.21.15.112:0/2144699945 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fac06a3b0 0x7f5fac06c870 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80114b0 tx=0x7f5fb8009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:23.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.300+0000 7f5fbe7fc700 1 -- 172.21.15.112:0/2144699945 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5fc0053d40 con 0x7f5fc815e990 2023-08-27T11:26:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.484+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2144699945 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5fc80b0e70 con 0x7f5fac06a3b0 2023-08-27T11:26:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.493+0000 7f5fbe7fc700 1 -- 172.21.15.112:0/2144699945 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307418 (secure 0 0 0) 0x7f5fb8011ec0 con 0x7f5fac06a3b0 2023-08-27T11:26:23.497 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.496+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2144699945 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fac06a3b0 msgr2=0x7f5fac06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.496+0000 7f5fcfd7f700 1 --2- 172.21.15.112:0/2144699945 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fac06a3b0 0x7f5fac06c870 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80114b0 tx=0x7f5fb8009210 comp rx=0 tx=0).stop 2023-08-27T11:26:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.496+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2144699945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fc815e990 msgr2=0x7f5fc8072020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.496+0000 7f5fcfd7f700 1 --2- 172.21.15.112:0/2144699945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fc815e990 0x7f5fc8072020 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000cb50 tx=0x7f5fc000ce60 comp rx=0 tx=0).stop 2023-08-27T11:26:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.496+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2144699945 shutdown_connections 2023-08-27T11:26:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.496+0000 7f5fcfd7f700 1 --2- 172.21.15.112:0/2144699945 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fac06a3b0 0x7f5fac06c870 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.496+0000 7f5fcfd7f700 1 --2- 172.21.15.112:0/2144699945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fc815e990 0x7f5fc8072020 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.496+0000 7f5fcfd7f700 1 --2- 172.21.15.112:0/2144699945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fc806b2f0 0x7f5fc815a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.496+0000 7f5fcfd7f700 1 --2- 172.21.15.112:0/2144699945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fc8151060 0x7f5fc8071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.496+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2144699945 >> 172.21.15.112:0/2144699945 conn(0x7f5fc80d0000 msgr2=0x7f5fc80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:23.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.496+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2144699945 shutdown_connections 2023-08-27T11:26:23.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.496+0000 7f5fcfd7f700 1 -- 172.21.15.112:0/2144699945 wait complete. 2023-08-27T11:26:23.502 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:26:23.510 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2023-08-27T11:26:22.368926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592618,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":183,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":154,"acting":154,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259232,"kb_used_data":9764,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754715488,"statfs":{"total":773094113280,"available":772828659712,"internally_reserved":0,"allocated":9998336,"data_stored":5206526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2250,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.970088"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874566+0000","last_change":"2023-08-27T11:26:19.427207+0000","last_active":"2023-08-27T11:26:20.874566+0000","last_peered":"2023-08-27T11:26:20.874566+0000","last_clean":"2023-08-27T11:26:20.874566+0000","last_became_active":"2023-08-27T11:26:19.426963+0000","last_became_peered":"2023-08-27T11:26:19.426963+0000","last_unstale":"2023-08-27T11:26:20.874566+0000","last_undegraded":"2023-08-27T11:26:20.874566+0000","last_fullsized":"2023-08-27T11:26:20.874566+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:55:21.733240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430931+0000","last_change":"2023-08-27T11:26:19.431066+0000","last_active":"2023-08-27T11:26:20.430931+0000","last_peered":"2023-08-27T11:26:20.430931+0000","last_clean":"2023-08-27T11:26:20.430931+0000","last_became_active":"2023-08-27T11:26:19.430477+0000","last_became_peered":"2023-08-27T11:26:19.430477+0000","last_unstale":"2023-08-27T11:26:20.430931+0000","last_undegraded":"2023-08-27T11:26:20.430931+0000","last_fullsized":"2023-08-27T11:26:20.430931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:43:44.337358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874230+0000","last_change":"2023-08-27T11:26:19.434887+0000","last_active":"2023-08-27T11:26:20.874230+0000","last_peered":"2023-08-27T11:26:20.874230+0000","last_clean":"2023-08-27T11:26:20.874230+0000","last_became_active":"2023-08-27T11:26:19.434566+0000","last_became_peered":"2023-08-27T11:26:19.434566+0000","last_unstale":"2023-08-27T11:26:20.874230+0000","last_undegraded":"2023-08-27T11:26:20.874230+0000","last_fullsized":"2023-08-27T11:26:20.874230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:11:11.738619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425453+0000","last_change":"2023-08-27T11:26:19.426786+0000","last_active":"2023-08-27T11:26:20.425453+0000","last_peered":"2023-08-27T11:26:20.425453+0000","last_clean":"2023-08-27T11:26:20.425453+0000","last_became_active":"2023-08-27T11:26:19.426494+0000","last_became_peered":"2023-08-27T11:26:19.426494+0000","last_unstale":"2023-08-27T11:26:20.425453+0000","last_undegraded":"2023-08-27T11:26:20.425453+0000","last_fullsized":"2023-08-27T11:26:20.425453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:33:53.249169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057365+0000","last_change":"2023-08-27T11:26:19.424084+0000","last_active":"2023-08-27T11:26:21.057365+0000","last_peered":"2023-08-27T11:26:21.057365+0000","last_clean":"2023-08-27T11:26:21.057365+0000","last_became_active":"2023-08-27T11:26:19.422955+0000","last_became_peered":"2023-08-27T11:26:19.422955+0000","last_unstale":"2023-08-27T11:26:21.057365+0000","last_undegraded":"2023-08-27T11:26:21.057365+0000","last_fullsized":"2023-08-27T11:26:21.057365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:59:24.896402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430830+0000","last_change":"2023-08-27T11:26:19.424659+0000","last_active":"2023-08-27T11:26:20.430830+0000","last_peered":"2023-08-27T11:26:20.430830+0000","last_clean":"2023-08-27T11:26:20.430830+0000","last_became_active":"2023-08-27T11:26:19.424317+0000","last_became_peered":"2023-08-27T11:26:19.424317+0000","last_unstale":"2023-08-27T11:26:20.430830+0000","last_undegraded":"2023-08-27T11:26:20.430830+0000","last_fullsized":"2023-08-27T11:26:20.430830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:47:42.857298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430038+0000","last_change":"2023-08-27T11:26:19.421667+0000","last_active":"2023-08-27T11:26:20.430038+0000","last_peered":"2023-08-27T11:26:20.430038+0000","last_clean":"2023-08-27T11:26:20.430038+0000","last_became_active":"2023-08-27T11:26:19.421352+0000","last_became_peered":"2023-08-27T11:26:19.421352+0000","last_unstale":"2023-08-27T11:26:20.430038+0000","last_undegraded":"2023-08-27T11:26:20.430038+0000","last_fullsized":"2023-08-27T11:26:20.430038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:24.356051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384297+0000","last_change":"2023-08-27T11:26:19.430958+0000","last_active":"2023-08-27T11:26:21.384297+0000","last_peered":"2023-08-27T11:26:21.384297+0000","last_clean":"2023-08-27T11:26:21.384297+0000","last_became_active":"2023-08-27T11:26:19.430685+0000","last_became_peered":"2023-08-27T11:26:19.430685+0000","last_unstale":"2023-08-27T11:26:21.384297+0000","last_undegraded":"2023-08-27T11:26:21.384297+0000","last_fullsized":"2023-08-27T11:26:21.384297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:41.805565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431121+0000","last_change":"2023-08-27T11:26:19.430211+0000","last_active":"2023-08-27T11:26:20.431121+0000","last_peered":"2023-08-27T11:26:20.431121+0000","last_clean":"2023-08-27T11:26:20.431121+0000","last_became_active":"2023-08-27T11:26:19.429921+0000","last_became_peered":"2023-08-27T11:26:19.429921+0000","last_unstale":"2023-08-27T11:26:20.431121+0000","last_undegraded":"2023-08-27T11:26:20.431121+0000","last_fullsized":"2023-08-27T11:26:20.431121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:56:34.526955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057756+0000","last_change":"2023-08-27T11:26:19.428983+0000","last_active":"2023-08-27T11:26:21.057756+0000","last_peered":"2023-08-27T11:26:21.057756+0000","last_clean":"2023-08-27T11:26:21.057756+0000","last_became_active":"2023-08-27T11:26:19.428699+0000","last_became_peered":"2023-08-27T11:26:19.428699+0000","last_unstale":"2023-08-27T11:26:21.057756+0000","last_undegraded":"2023-08-27T11:26:21.057756+0000","last_fullsized":"2023-08-27T11:26:21.057756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:49:32.779304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425155+0000","last_change":"2023-08-27T11:26:19.436540+0000","last_active":"2023-08-27T11:26:20.425155+0000","last_peered":"2023-08-27T11:26:20.425155+0000","last_clean":"2023-08-27T11:26:20.425155+0000","last_became_active":"2023-08-27T11:26:19.436278+0000","last_became_peered":"2023-08-27T11:26:19.436278+0000","last_unstale":"2023-08-27T11:26:20.425155+0000","last_undegraded":"2023-08-27T11:26:20.425155+0000","last_fullsized":"2023-08-27T11:26:20.425155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:40:05.207374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874577+0000","last_change":"2023-08-27T11:26:19.434923+0000","last_active":"2023-08-27T11:26:20.874577+0000","last_peered":"2023-08-27T11:26:20.874577+0000","last_clean":"2023-08-27T11:26:20.874577+0000","last_became_active":"2023-08-27T11:26:19.434650+0000","last_became_peered":"2023-08-27T11:26:19.434650+0000","last_unstale":"2023-08-27T11:26:20.874577+0000","last_undegraded":"2023-08-27T11:26:20.874577+0000","last_fullsized":"2023-08-27T11:26:20.874577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:33:06.493994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430846+0000","last_change":"2023-08-27T11:26:19.422300+0000","last_active":"2023-08-27T11:26:20.430846+0000","last_peered":"2023-08-27T11:26:20.430846+0000","last_clean":"2023-08-27T11:26:20.430846+0000","last_became_active":"2023-08-27T11:26:19.421284+0000","last_became_peered":"2023-08-27T11:26:19.421284+0000","last_unstale":"2023-08-27T11:26:20.430846+0000","last_undegraded":"2023-08-27T11:26:20.430846+0000","last_fullsized":"2023-08-27T11:26:20.430846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:12.363650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874425+0000","last_change":"2023-08-27T11:26:19.418470+0000","last_active":"2023-08-27T11:26:20.874425+0000","last_peered":"2023-08-27T11:26:20.874425+0000","last_clean":"2023-08-27T11:26:20.874425+0000","last_became_active":"2023-08-27T11:26:19.418192+0000","last_became_peered":"2023-08-27T11:26:19.418192+0000","last_unstale":"2023-08-27T11:26:20.874425+0000","last_undegraded":"2023-08-27T11:26:20.874425+0000","last_fullsized":"2023-08-27T11:26:20.874425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:14:53.959765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384767+0000","last_change":"2023-08-27T11:26:19.433627+0000","last_active":"2023-08-27T11:26:21.384767+0000","last_peered":"2023-08-27T11:26:21.384767+0000","last_clean":"2023-08-27T11:26:21.384767+0000","last_became_active":"2023-08-27T11:26:19.433370+0000","last_became_peered":"2023-08-27T11:26:19.433370+0000","last_unstale":"2023-08-27T11:26:21.384767+0000","last_undegraded":"2023-08-27T11:26:21.384767+0000","last_fullsized":"2023-08-27T11:26:21.384767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:41.696204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057432+0000","last_change":"2023-08-27T11:26:19.433519+0000","last_active":"2023-08-27T11:26:21.057432+0000","last_peered":"2023-08-27T11:26:21.057432+0000","last_clean":"2023-08-27T11:26:21.057432+0000","last_became_active":"2023-08-27T11:26:19.433245+0000","last_became_peered":"2023-08-27T11:26:19.433245+0000","last_unstale":"2023-08-27T11:26:21.057432+0000","last_undegraded":"2023-08-27T11:26:21.057432+0000","last_fullsized":"2023-08-27T11:26:21.057432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:07:00.066813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874575+0000","last_change":"2023-08-27T11:26:19.434762+0000","last_active":"2023-08-27T11:26:20.874575+0000","last_peered":"2023-08-27T11:26:20.874575+0000","last_clean":"2023-08-27T11:26:20.874575+0000","last_became_active":"2023-08-27T11:26:19.434462+0000","last_became_peered":"2023-08-27T11:26:19.434462+0000","last_unstale":"2023-08-27T11:26:20.874575+0000","last_undegraded":"2023-08-27T11:26:20.874575+0000","last_fullsized":"2023-08-27T11:26:20.874575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:41.356424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057290+0000","last_change":"2023-08-27T11:26:19.423833+0000","last_active":"2023-08-27T11:26:21.057290+0000","last_peered":"2023-08-27T11:26:21.057290+0000","last_clean":"2023-08-27T11:26:21.057290+0000","last_became_active":"2023-08-27T11:26:19.423440+0000","last_became_peered":"2023-08-27T11:26:19.423440+0000","last_unstale":"2023-08-27T11:26:21.057290+0000","last_undegraded":"2023-08-27T11:26:21.057290+0000","last_fullsized":"2023-08-27T11:26:21.057290+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:41:13.840925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430610+0000","last_change":"2023-08-27T11:26:19.433762+0000","last_active":"2023-08-27T11:26:20.430610+0000","last_peered":"2023-08-27T11:26:20.430610+0000","last_clean":"2023-08-27T11:26:20.430610+0000","last_became_active":"2023-08-27T11:26:19.433489+0000","last_became_peered":"2023-08-27T11:26:19.433489+0000","last_unstale":"2023-08-27T11:26:20.430610+0000","last_undegraded":"2023-08-27T11:26:20.430610+0000","last_fullsized":"2023-08-27T11:26:20.430610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:34.644697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384668+0000","last_change":"2023-08-27T11:26:19.431428+0000","last_active":"2023-08-27T11:26:21.384668+0000","last_peered":"2023-08-27T11:26:21.384668+0000","last_clean":"2023-08-27T11:26:21.384668+0000","last_became_active":"2023-08-27T11:26:19.431169+0000","last_became_peered":"2023-08-27T11:26:19.431169+0000","last_unstale":"2023-08-27T11:26:21.384668+0000","last_undegraded":"2023-08-27T11:26:21.384668+0000","last_fullsized":"2023-08-27T11:26:21.384668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:30:35.082113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430321+0000","last_change":"2023-08-27T11:26:19.427535+0000","last_active":"2023-08-27T11:26:20.430321+0000","last_peered":"2023-08-27T11:26:20.430321+0000","last_clean":"2023-08-27T11:26:20.430321+0000","last_became_active":"2023-08-27T11:26:19.426191+0000","last_became_peered":"2023-08-27T11:26:19.426191+0000","last_unstale":"2023-08-27T11:26:20.430321+0000","last_undegraded":"2023-08-27T11:26:20.430321+0000","last_fullsized":"2023-08-27T11:26:20.430321+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:22:47.314202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384374+0000","last_change":"2023-08-27T11:26:19.424642+0000","last_active":"2023-08-27T11:26:21.384374+0000","last_peered":"2023-08-27T11:26:21.384374+0000","last_clean":"2023-08-27T11:26:21.384374+0000","last_became_active":"2023-08-27T11:26:19.424359+0000","last_became_peered":"2023-08-27T11:26:19.424359+0000","last_unstale":"2023-08-27T11:26:21.384374+0000","last_undegraded":"2023-08-27T11:26:21.384374+0000","last_fullsized":"2023-08-27T11:26:21.384374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:06:36.252165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874126+0000","last_change":"2023-08-27T11:26:19.414983+0000","last_active":"2023-08-27T11:26:20.874126+0000","last_peered":"2023-08-27T11:26:20.874126+0000","last_clean":"2023-08-27T11:26:20.874126+0000","last_became_active":"2023-08-27T11:26:19.414726+0000","last_became_peered":"2023-08-27T11:26:19.414726+0000","last_unstale":"2023-08-27T11:26:20.874126+0000","last_undegraded":"2023-08-27T11:26:20.874126+0000","last_fullsized":"2023-08-27T11:26:20.874126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:57:45.405543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"89'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425328+0000","last_change":"2023-08-27T11:26:18.432220+0000","last_active":"2023-08-27T11:26:20.425328+0000","last_peered":"2023-08-27T11:26:20.425328+0000","last_clean":"2023-08-27T11:26:20.425328+0000","last_became_active":"2023-08-27T11:26:18.431938+0000","last_became_peered":"2023-08-27T11:26:18.431938+0000","last_unstale":"2023-08-27T11:26:20.425328+0000","last_undegraded":"2023-08-27T11:26:20.425328+0000","last_fullsized":"2023-08-27T11:26:20.425328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:26:24.683316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384352+0000","last_change":"2023-08-27T11:26:19.427981+0000","last_active":"2023-08-27T11:26:21.384352+0000","last_peered":"2023-08-27T11:26:21.384352+0000","last_clean":"2023-08-27T11:26:21.384352+0000","last_became_active":"2023-08-27T11:26:19.427694+0000","last_became_peered":"2023-08-27T11:26:19.427694+0000","last_unstale":"2023-08-27T11:26:21.384352+0000","last_undegraded":"2023-08-27T11:26:21.384352+0000","last_fullsized":"2023-08-27T11:26:21.384352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:31.571236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384746+0000","last_change":"2023-08-27T11:26:18.416729+0000","last_active":"2023-08-27T11:26:21.384746+0000","last_peered":"2023-08-27T11:26:21.384746+0000","last_clean":"2023-08-27T11:26:21.384746+0000","last_became_active":"2023-08-27T11:26:18.416320+0000","last_became_peered":"2023-08-27T11:26:18.416320+0000","last_unstale":"2023-08-27T11:26:21.384746+0000","last_undegraded":"2023-08-27T11:26:21.384746+0000","last_fullsized":"2023-08-27T11:26:21.384746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:32.457209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057704+0000","last_change":"2023-08-27T11:26:18.426839+0000","last_active":"2023-08-27T11:26:21.057704+0000","last_peered":"2023-08-27T11:26:21.057704+0000","last_clean":"2023-08-27T11:26:21.057704+0000","last_became_active":"2023-08-27T11:26:18.426578+0000","last_became_peered":"2023-08-27T11:26:18.426578+0000","last_unstale":"2023-08-27T11:26:21.057704+0000","last_undegraded":"2023-08-27T11:26:21.057704+0000","last_fullsized":"2023-08-27T11:26:21.057704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:23.764319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425272+0000","last_change":"2023-08-27T11:26:19.426762+0000","last_active":"2023-08-27T11:26:20.425272+0000","last_peered":"2023-08-27T11:26:20.425272+0000","last_clean":"2023-08-27T11:26:20.425272+0000","last_became_active":"2023-08-27T11:26:19.426146+0000","last_became_peered":"2023-08-27T11:26:19.426146+0000","last_unstale":"2023-08-27T11:26:20.425272+0000","last_undegraded":"2023-08-27T11:26:20.425272+0000","last_fullsized":"2023-08-27T11:26:20.425272+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:54:50.371981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874343+0000","last_change":"2023-08-27T11:26:19.418506+0000","last_active":"2023-08-27T11:26:20.874343+0000","last_peered":"2023-08-27T11:26:20.874343+0000","last_clean":"2023-08-27T11:26:20.874343+0000","last_became_active":"2023-08-27T11:26:19.418235+0000","last_became_peered":"2023-08-27T11:26:19.418235+0000","last_unstale":"2023-08-27T11:26:20.874343+0000","last_undegraded":"2023-08-27T11:26:20.874343+0000","last_fullsized":"2023-08-27T11:26:20.874343+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:50.719209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430641+0000","last_change":"2023-08-27T11:26:19.431258+0000","last_active":"2023-08-27T11:26:20.430641+0000","last_peered":"2023-08-27T11:26:20.430641+0000","last_clean":"2023-08-27T11:26:20.430641+0000","last_became_active":"2023-08-27T11:26:19.430816+0000","last_became_peered":"2023-08-27T11:26:19.430816+0000","last_unstale":"2023-08-27T11:26:20.430641+0000","last_undegraded":"2023-08-27T11:26:20.430641+0000","last_fullsized":"2023-08-27T11:26:20.430641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:55.969633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425484+0000","last_change":"2023-08-27T11:26:19.426307+0000","last_active":"2023-08-27T11:26:20.425484+0000","last_peered":"2023-08-27T11:26:20.425484+0000","last_clean":"2023-08-27T11:26:20.425484+0000","last_became_active":"2023-08-27T11:26:19.425959+0000","last_became_peered":"2023-08-27T11:26:19.425959+0000","last_unstale":"2023-08-27T11:26:20.425484+0000","last_undegraded":"2023-08-27T11:26:20.425484+0000","last_fullsized":"2023-08-27T11:26:20.425484+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:10:02.440599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425034+0000","last_change":"2023-08-27T11:26:18.429003+0000","last_active":"2023-08-27T11:26:20.425034+0000","last_peered":"2023-08-27T11:26:20.425034+0000","last_clean":"2023-08-27T11:26:20.425034+0000","last_became_active":"2023-08-27T11:26:18.428701+0000","last_became_peered":"2023-08-27T11:26:18.428701+0000","last_unstale":"2023-08-27T11:26:20.425034+0000","last_undegraded":"2023-08-27T11:26:20.425034+0000","last_fullsized":"2023-08-27T11:26:20.425034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:42:33.783460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874118+0000","last_change":"2023-08-27T11:26:19.417293+0000","last_active":"2023-08-27T11:26:20.874118+0000","last_peered":"2023-08-27T11:26:20.874118+0000","last_clean":"2023-08-27T11:26:20.874118+0000","last_became_active":"2023-08-27T11:26:19.417018+0000","last_became_peered":"2023-08-27T11:26:19.417018+0000","last_unstale":"2023-08-27T11:26:20.874118+0000","last_undegraded":"2023-08-27T11:26:20.874118+0000","last_fullsized":"2023-08-27T11:26:20.874118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:02:56.092248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057401+0000","last_change":"2023-08-27T11:26:19.425413+0000","last_active":"2023-08-27T11:26:21.057401+0000","last_peered":"2023-08-27T11:26:21.057401+0000","last_clean":"2023-08-27T11:26:21.057401+0000","last_became_active":"2023-08-27T11:26:19.424482+0000","last_became_peered":"2023-08-27T11:26:19.424482+0000","last_unstale":"2023-08-27T11:26:21.057401+0000","last_undegraded":"2023-08-27T11:26:21.057401+0000","last_fullsized":"2023-08-27T11:26:21.057401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:36.587246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874328+0000","last_change":"2023-08-27T11:26:18.426356+0000","last_active":"2023-08-27T11:26:20.874328+0000","last_peered":"2023-08-27T11:26:20.874328+0000","last_clean":"2023-08-27T11:26:20.874328+0000","last_became_active":"2023-08-27T11:26:18.426041+0000","last_became_peered":"2023-08-27T11:26:18.426041+0000","last_unstale":"2023-08-27T11:26:20.874328+0000","last_undegraded":"2023-08-27T11:26:20.874328+0000","last_fullsized":"2023-08-27T11:26:20.874328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:28.381950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384393+0000","last_change":"2023-08-27T11:26:19.431679+0000","last_active":"2023-08-27T11:26:21.384393+0000","last_peered":"2023-08-27T11:26:21.384393+0000","last_clean":"2023-08-27T11:26:21.384393+0000","last_became_active":"2023-08-27T11:26:19.431405+0000","last_became_peered":"2023-08-27T11:26:19.431405+0000","last_unstale":"2023-08-27T11:26:21.384393+0000","last_undegraded":"2023-08-27T11:26:21.384393+0000","last_fullsized":"2023-08-27T11:26:21.384393+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:02:21.450786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057480+0000","last_change":"2023-08-27T11:26:19.424582+0000","last_active":"2023-08-27T11:26:21.057480+0000","last_peered":"2023-08-27T11:26:21.057480+0000","last_clean":"2023-08-27T11:26:21.057480+0000","last_became_active":"2023-08-27T11:26:19.424288+0000","last_became_peered":"2023-08-27T11:26:19.424288+0000","last_unstale":"2023-08-27T11:26:21.057480+0000","last_undegraded":"2023-08-27T11:26:21.057480+0000","last_fullsized":"2023-08-27T11:26:21.057480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:16.279109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430741+0000","last_change":"2023-08-27T11:26:18.421802+0000","last_active":"2023-08-27T11:26:20.430741+0000","last_peered":"2023-08-27T11:26:20.430741+0000","last_clean":"2023-08-27T11:26:20.430741+0000","last_became_active":"2023-08-27T11:26:18.421093+0000","last_became_peered":"2023-08-27T11:26:18.421093+0000","last_unstale":"2023-08-27T11:26:20.430741+0000","last_undegraded":"2023-08-27T11:26:20.430741+0000","last_fullsized":"2023-08-27T11:26:20.430741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:29.408679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057739+0000","last_change":"2023-08-27T11:26:18.421876+0000","last_active":"2023-08-27T11:26:21.057739+0000","last_peered":"2023-08-27T11:26:21.057739+0000","last_clean":"2023-08-27T11:26:21.057739+0000","last_became_active":"2023-08-27T11:26:18.421583+0000","last_became_peered":"2023-08-27T11:26:18.421583+0000","last_unstale":"2023-08-27T11:26:21.057739+0000","last_undegraded":"2023-08-27T11:26:21.057739+0000","last_fullsized":"2023-08-27T11:26:21.057739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:48:43.946799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430199+0000","last_change":"2023-08-27T11:26:19.430893+0000","last_active":"2023-08-27T11:26:20.430199+0000","last_peered":"2023-08-27T11:26:20.430199+0000","last_clean":"2023-08-27T11:26:20.430199+0000","last_became_active":"2023-08-27T11:26:19.430225+0000","last_became_peered":"2023-08-27T11:26:19.430225+0000","last_unstale":"2023-08-27T11:26:20.430199+0000","last_undegraded":"2023-08-27T11:26:20.430199+0000","last_fullsized":"2023-08-27T11:26:20.430199+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:43.123979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425475+0000","last_change":"2023-08-27T11:26:18.432084+0000","last_active":"2023-08-27T11:26:20.425475+0000","last_peered":"2023-08-27T11:26:20.425475+0000","last_clean":"2023-08-27T11:26:20.425475+0000","last_became_active":"2023-08-27T11:26:18.431877+0000","last_became_peered":"2023-08-27T11:26:18.431877+0000","last_unstale":"2023-08-27T11:26:20.425475+0000","last_undegraded":"2023-08-27T11:26:20.425475+0000","last_fullsized":"2023-08-27T11:26:20.425475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:12:10.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384677+0000","last_change":"2023-08-27T11:26:19.429400+0000","last_active":"2023-08-27T11:26:21.384677+0000","last_peered":"2023-08-27T11:26:21.384677+0000","last_clean":"2023-08-27T11:26:21.384677+0000","last_became_active":"2023-08-27T11:26:19.429146+0000","last_became_peered":"2023-08-27T11:26:19.429146+0000","last_unstale":"2023-08-27T11:26:21.384677+0000","last_undegraded":"2023-08-27T11:26:21.384677+0000","last_fullsized":"2023-08-27T11:26:21.384677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:06.185951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384453+0000","last_change":"2023-08-27T11:26:18.420633+0000","last_active":"2023-08-27T11:26:21.384453+0000","last_peered":"2023-08-27T11:26:21.384453+0000","last_clean":"2023-08-27T11:26:21.384453+0000","last_became_active":"2023-08-27T11:26:18.420312+0000","last_became_peered":"2023-08-27T11:26:18.420312+0000","last_unstale":"2023-08-27T11:26:21.384453+0000","last_undegraded":"2023-08-27T11:26:21.384453+0000","last_fullsized":"2023-08-27T11:26:21.384453+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:20.872678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874639+0000","last_change":"2023-08-27T11:26:19.434950+0000","last_active":"2023-08-27T11:26:20.874639+0000","last_peered":"2023-08-27T11:26:20.874639+0000","last_clean":"2023-08-27T11:26:20.874639+0000","last_became_active":"2023-08-27T11:26:19.434684+0000","last_became_peered":"2023-08-27T11:26:19.434684+0000","last_unstale":"2023-08-27T11:26:20.874639+0000","last_undegraded":"2023-08-27T11:26:20.874639+0000","last_fullsized":"2023-08-27T11:26:20.874639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:52:42.875755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874262+0000","last_change":"2023-08-27T11:26:19.425141+0000","last_active":"2023-08-27T11:26:20.874262+0000","last_peered":"2023-08-27T11:26:20.874262+0000","last_clean":"2023-08-27T11:26:20.874262+0000","last_became_active":"2023-08-27T11:26:19.424669+0000","last_became_peered":"2023-08-27T11:26:19.424669+0000","last_unstale":"2023-08-27T11:26:20.874262+0000","last_undegraded":"2023-08-27T11:26:20.874262+0000","last_fullsized":"2023-08-27T11:26:20.874262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:31:34.706537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057687+0000","last_change":"2023-08-27T11:26:18.421572+0000","last_active":"2023-08-27T11:26:21.057687+0000","last_peered":"2023-08-27T11:26:21.057687+0000","last_clean":"2023-08-27T11:26:21.057687+0000","last_became_active":"2023-08-27T11:26:18.421268+0000","last_became_peered":"2023-08-27T11:26:18.421268+0000","last_unstale":"2023-08-27T11:26:21.057687+0000","last_undegraded":"2023-08-27T11:26:21.057687+0000","last_fullsized":"2023-08-27T11:26:21.057687+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:44.389523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874451+0000","last_change":"2023-08-27T11:26:18.429524+0000","last_active":"2023-08-27T11:26:20.874451+0000","last_peered":"2023-08-27T11:26:20.874451+0000","last_clean":"2023-08-27T11:26:20.874451+0000","last_became_active":"2023-08-27T11:26:18.429224+0000","last_became_peered":"2023-08-27T11:26:18.429224+0000","last_unstale":"2023-08-27T11:26:20.874451+0000","last_undegraded":"2023-08-27T11:26:20.874451+0000","last_fullsized":"2023-08-27T11:26:20.874451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:13.776898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425320+0000","last_change":"2023-08-27T11:26:19.427555+0000","last_active":"2023-08-27T11:26:20.425320+0000","last_peered":"2023-08-27T11:26:20.425320+0000","last_clean":"2023-08-27T11:26:20.425320+0000","last_became_active":"2023-08-27T11:26:19.427295+0000","last_became_peered":"2023-08-27T11:26:19.427295+0000","last_unstale":"2023-08-27T11:26:20.425320+0000","last_undegraded":"2023-08-27T11:26:20.425320+0000","last_fullsized":"2023-08-27T11:26:20.425320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:35.698617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430342+0000","last_change":"2023-08-27T11:26:18.432922+0000","last_active":"2023-08-27T11:26:20.430342+0000","last_peered":"2023-08-27T11:26:20.430342+0000","last_clean":"2023-08-27T11:26:20.430342+0000","last_became_active":"2023-08-27T11:26:18.432631+0000","last_became_peered":"2023-08-27T11:26:18.432631+0000","last_unstale":"2023-08-27T11:26:20.430342+0000","last_undegraded":"2023-08-27T11:26:20.430342+0000","last_fullsized":"2023-08-27T11:26:20.430342+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:58.703884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057644+0000","last_change":"2023-08-27T11:26:18.422687+0000","last_active":"2023-08-27T11:26:21.057644+0000","last_peered":"2023-08-27T11:26:21.057644+0000","last_clean":"2023-08-27T11:26:21.057644+0000","last_became_active":"2023-08-27T11:26:18.422404+0000","last_became_peered":"2023-08-27T11:26:18.422404+0000","last_unstale":"2023-08-27T11:26:21.057644+0000","last_undegraded":"2023-08-27T11:26:21.057644+0000","last_fullsized":"2023-08-27T11:26:21.057644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:34.961579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384633+0000","last_change":"2023-08-27T11:26:19.425732+0000","last_active":"2023-08-27T11:26:21.384633+0000","last_peered":"2023-08-27T11:26:21.384633+0000","last_clean":"2023-08-27T11:26:21.384633+0000","last_became_active":"2023-08-27T11:26:19.425452+0000","last_became_peered":"2023-08-27T11:26:19.425452+0000","last_unstale":"2023-08-27T11:26:21.384633+0000","last_undegraded":"2023-08-27T11:26:21.384633+0000","last_fullsized":"2023-08-27T11:26:21.384633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:15.049461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384761+0000","last_change":"2023-08-27T11:26:19.425658+0000","last_active":"2023-08-27T11:26:21.384761+0000","last_peered":"2023-08-27T11:26:21.384761+0000","last_clean":"2023-08-27T11:26:21.384761+0000","last_became_active":"2023-08-27T11:26:19.425387+0000","last_became_peered":"2023-08-27T11:26:19.425387+0000","last_unstale":"2023-08-27T11:26:21.384761+0000","last_undegraded":"2023-08-27T11:26:21.384761+0000","last_fullsized":"2023-08-27T11:26:21.384761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:52.205981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384498+0000","last_change":"2023-08-27T11:26:18.426465+0000","last_active":"2023-08-27T11:26:21.384498+0000","last_peered":"2023-08-27T11:26:21.384498+0000","last_clean":"2023-08-27T11:26:21.384498+0000","last_became_active":"2023-08-27T11:26:18.426180+0000","last_became_peered":"2023-08-27T11:26:18.426180+0000","last_unstale":"2023-08-27T11:26:21.384498+0000","last_undegraded":"2023-08-27T11:26:21.384498+0000","last_fullsized":"2023-08-27T11:26:21.384498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:46.357284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430463+0000","last_change":"2023-08-27T11:26:18.432789+0000","last_active":"2023-08-27T11:26:20.430463+0000","last_peered":"2023-08-27T11:26:20.430463+0000","last_clean":"2023-08-27T11:26:20.430463+0000","last_became_active":"2023-08-27T11:26:18.432521+0000","last_became_peered":"2023-08-27T11:26:18.432521+0000","last_unstale":"2023-08-27T11:26:20.430463+0000","last_undegraded":"2023-08-27T11:26:20.430463+0000","last_fullsized":"2023-08-27T11:26:20.430463+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:06.592671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384455+0000","last_change":"2023-08-27T11:26:18.426963+0000","last_active":"2023-08-27T11:26:21.384455+0000","last_peered":"2023-08-27T11:26:21.384455+0000","last_clean":"2023-08-27T11:26:21.384455+0000","last_became_active":"2023-08-27T11:26:18.426752+0000","last_became_peered":"2023-08-27T11:26:18.426752+0000","last_unstale":"2023-08-27T11:26:21.384455+0000","last_undegraded":"2023-08-27T11:26:21.384455+0000","last_fullsized":"2023-08-27T11:26:21.384455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:29:35.149141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425190+0000","last_change":"2023-08-27T11:26:19.431940+0000","last_active":"2023-08-27T11:26:20.425190+0000","last_peered":"2023-08-27T11:26:20.425190+0000","last_clean":"2023-08-27T11:26:20.425190+0000","last_became_active":"2023-08-27T11:26:19.431631+0000","last_became_peered":"2023-08-27T11:26:19.431631+0000","last_unstale":"2023-08-27T11:26:20.425190+0000","last_undegraded":"2023-08-27T11:26:20.425190+0000","last_fullsized":"2023-08-27T11:26:20.425190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:37.504080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430548+0000","last_change":"2023-08-27T11:26:18.425472+0000","last_active":"2023-08-27T11:26:20.430548+0000","last_peered":"2023-08-27T11:26:20.430548+0000","last_clean":"2023-08-27T11:26:20.430548+0000","last_became_active":"2023-08-27T11:26:18.425150+0000","last_became_peered":"2023-08-27T11:26:18.425150+0000","last_unstale":"2023-08-27T11:26:20.430548+0000","last_undegraded":"2023-08-27T11:26:20.430548+0000","last_fullsized":"2023-08-27T11:26:20.430548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:10.608143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431200+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:26:20.431200+0000","last_peered":"2023-08-27T11:26:20.431200+0000","last_clean":"2023-08-27T11:26:20.431200+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:26:20.431200+0000","last_undegraded":"2023-08-27T11:26:20.431200+0000","last_fullsized":"2023-08-27T11:26:20.431200+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:53.465982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384795+0000","last_change":"2023-08-27T11:26:19.426312+0000","last_active":"2023-08-27T11:26:21.384795+0000","last_peered":"2023-08-27T11:26:21.384795+0000","last_clean":"2023-08-27T11:26:21.384795+0000","last_became_active":"2023-08-27T11:26:19.425963+0000","last_became_peered":"2023-08-27T11:26:19.425963+0000","last_unstale":"2023-08-27T11:26:21.384795+0000","last_undegraded":"2023-08-27T11:26:21.384795+0000","last_fullsized":"2023-08-27T11:26:21.384795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:46.854781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384674+0000","last_change":"2023-08-27T11:26:18.429275+0000","last_active":"2023-08-27T11:26:21.384674+0000","last_peered":"2023-08-27T11:26:21.384674+0000","last_clean":"2023-08-27T11:26:21.384674+0000","last_became_active":"2023-08-27T11:26:18.428994+0000","last_became_peered":"2023-08-27T11:26:18.428994+0000","last_unstale":"2023-08-27T11:26:21.384674+0000","last_undegraded":"2023-08-27T11:26:21.384674+0000","last_fullsized":"2023-08-27T11:26:21.384674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:55.212494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430413+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:26:20.430413+0000","last_peered":"2023-08-27T11:26:20.430413+0000","last_clean":"2023-08-27T11:26:20.430413+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:26:20.430413+0000","last_undegraded":"2023-08-27T11:26:20.430413+0000","last_fullsized":"2023-08-27T11:26:20.430413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:35:03.658156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384543+0000","last_change":"2023-08-27T11:26:19.434349+0000","last_active":"2023-08-27T11:26:21.384543+0000","last_peered":"2023-08-27T11:26:21.384543+0000","last_clean":"2023-08-27T11:26:21.384543+0000","last_became_active":"2023-08-27T11:26:19.434063+0000","last_became_peered":"2023-08-27T11:26:19.434063+0000","last_unstale":"2023-08-27T11:26:21.384543+0000","last_undegraded":"2023-08-27T11:26:21.384543+0000","last_fullsized":"2023-08-27T11:26:21.384543+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:15.835557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431167+0000","last_change":"2023-08-27T11:26:18.429449+0000","last_active":"2023-08-27T11:26:20.431167+0000","last_peered":"2023-08-27T11:26:20.431167+0000","last_clean":"2023-08-27T11:26:20.431167+0000","last_became_active":"2023-08-27T11:26:18.429134+0000","last_became_peered":"2023-08-27T11:26:18.429134+0000","last_unstale":"2023-08-27T11:26:20.431167+0000","last_undegraded":"2023-08-27T11:26:20.431167+0000","last_fullsized":"2023-08-27T11:26:20.431167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:47:49.142361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:16.089904+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:26:16.089904+0000","last_peered":"2023-08-27T11:26:16.089904+0000","last_clean":"2023-08-27T11:26:16.089904+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:26:16.089904+0000","last_undegraded":"2023-08-27T11:26:16.089904+0000","last_fullsized":"2023-08-27T11:26:16.089904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:48:47.411691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384819+0000","last_change":"2023-08-27T11:26:18.429297+0000","last_active":"2023-08-27T11:26:21.384819+0000","last_peered":"2023-08-27T11:26:21.384819+0000","last_clean":"2023-08-27T11:26:21.384819+0000","last_became_active":"2023-08-27T11:26:18.429028+0000","last_became_peered":"2023-08-27T11:26:18.429028+0000","last_unstale":"2023-08-27T11:26:21.384819+0000","last_undegraded":"2023-08-27T11:26:21.384819+0000","last_fullsized":"2023-08-27T11:26:21.384819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:25:56.014027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":133,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:14.402961+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:26:14.402961+0000","last_peered":"2023-08-27T11:26:14.402961+0000","last_clean":"2023-08-27T11:26:14.402961+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:26:14.402961+0000","last_undegraded":"2023-08-27T11:26:14.402961+0000","last_fullsized":"2023-08-27T11:26:14.402961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:18:41.917017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057479+0000","last_change":"2023-08-27T11:26:19.432826+0000","last_active":"2023-08-27T11:26:21.057479+0000","last_peered":"2023-08-27T11:26:21.057479+0000","last_clean":"2023-08-27T11:26:21.057479+0000","last_became_active":"2023-08-27T11:26:19.432562+0000","last_became_peered":"2023-08-27T11:26:19.432562+0000","last_unstale":"2023-08-27T11:26:21.057479+0000","last_undegraded":"2023-08-27T11:26:21.057479+0000","last_fullsized":"2023-08-27T11:26:21.057479+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:05.457562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":133,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:14.402955+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:26:14.402955+0000","last_peered":"2023-08-27T11:26:14.402955+0000","last_clean":"2023-08-27T11:26:14.402955+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:26:14.402955+0000","last_undegraded":"2023-08-27T11:26:14.402955+0000","last_fullsized":"2023-08-27T11:26:14.402955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:59:51.839342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430815+0000","last_change":"2023-08-27T11:26:19.427376+0000","last_active":"2023-08-27T11:26:20.430815+0000","last_peered":"2023-08-27T11:26:20.430815+0000","last_clean":"2023-08-27T11:26:20.430815+0000","last_became_active":"2023-08-27T11:26:19.426214+0000","last_became_peered":"2023-08-27T11:26:19.426214+0000","last_unstale":"2023-08-27T11:26:20.430815+0000","last_undegraded":"2023-08-27T11:26:20.430815+0000","last_fullsized":"2023-08-27T11:26:20.430815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:07:17.226611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":283,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:14.403153+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:26:14.403153+0000","last_peered":"2023-08-27T11:26:14.403153+0000","last_clean":"2023-08-27T11:26:14.403153+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:26:14.403153+0000","last_undegraded":"2023-08-27T11:26:14.403153+0000","last_fullsized":"2023-08-27T11:26:14.403153+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:26:15.453703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057579+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:26:21.057579+0000","last_peered":"2023-08-27T11:26:21.057579+0000","last_clean":"2023-08-27T11:26:21.057579+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:26:21.057579+0000","last_undegraded":"2023-08-27T11:26:21.057579+0000","last_fullsized":"2023-08-27T11:26:21.057579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:12:51.448320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874474+0000","last_change":"2023-08-27T11:26:19.420539+0000","last_active":"2023-08-27T11:26:20.874474+0000","last_peered":"2023-08-27T11:26:20.874474+0000","last_clean":"2023-08-27T11:26:20.874474+0000","last_became_active":"2023-08-27T11:26:19.420291+0000","last_became_peered":"2023-08-27T11:26:19.420291+0000","last_unstale":"2023-08-27T11:26:20.874474+0000","last_undegraded":"2023-08-27T11:26:20.874474+0000","last_fullsized":"2023-08-27T11:26:20.874474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:23:25.892210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384424+0000","last_change":"2023-08-27T11:26:18.432770+0000","last_active":"2023-08-27T11:26:21.384424+0000","last_peered":"2023-08-27T11:26:21.384424+0000","last_clean":"2023-08-27T11:26:21.384424+0000","last_became_active":"2023-08-27T11:26:18.432552+0000","last_became_peered":"2023-08-27T11:26:18.432552+0000","last_unstale":"2023-08-27T11:26:21.384424+0000","last_undegraded":"2023-08-27T11:26:21.384424+0000","last_fullsized":"2023-08-27T11:26:21.384424+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:05:38.418817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":139,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425350+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:26:20.425350+0000","last_peered":"2023-08-27T11:26:20.425350+0000","last_clean":"2023-08-27T11:26:20.425350+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:26:20.425350+0000","last_undegraded":"2023-08-27T11:26:20.425350+0000","last_fullsized":"2023-08-27T11:26:20.425350+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:38:27.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874280+0000","last_change":"2023-08-27T11:26:19.427286+0000","last_active":"2023-08-27T11:26:20.874280+0000","last_peered":"2023-08-27T11:26:20.874280+0000","last_clean":"2023-08-27T11:26:20.874280+0000","last_became_active":"2023-08-27T11:26:19.426940+0000","last_became_peered":"2023-08-27T11:26:19.426940+0000","last_unstale":"2023-08-27T11:26:20.874280+0000","last_undegraded":"2023-08-27T11:26:20.874280+0000","last_fullsized":"2023-08-27T11:26:20.874280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:31.231234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384817+0000","last_change":"2023-08-27T11:26:18.415885+0000","last_active":"2023-08-27T11:26:21.384817+0000","last_peered":"2023-08-27T11:26:21.384817+0000","last_clean":"2023-08-27T11:26:21.384817+0000","last_became_active":"2023-08-27T11:26:18.415623+0000","last_became_peered":"2023-08-27T11:26:18.415623+0000","last_unstale":"2023-08-27T11:26:21.384817+0000","last_undegraded":"2023-08-27T11:26:21.384817+0000","last_fullsized":"2023-08-27T11:26:21.384817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:10:07.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":145,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:16.089811+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:26:16.089811+0000","last_peered":"2023-08-27T11:26:16.089811+0000","last_clean":"2023-08-27T11:26:16.089811+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:26:16.089811+0000","last_undegraded":"2023-08-27T11:26:16.089811+0000","last_fullsized":"2023-08-27T11:26:16.089811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:05:17.706426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2250,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":44,"acting":44,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":5},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1658880,"data_stored":1076579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.33300000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.55000000000000004,"15min":0.50800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":0.77500000000000002,"15min":0.872},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.253},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.39000000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.48699999999999999},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.33800000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.90200000000000002},"last":0.53700000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.56799999999999995}]},{"osd":4,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.505},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.72699999999999998,"5min":0.72799999999999998,"15min":0.77400000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.497}]},{"osd":5,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.52400000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574966,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1134592,"data_stored":515006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51500000000000001,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.29099999999999998},"max":{"1min":0.626,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67300000000000004,"5min":0.86199999999999999,"15min":0.89800000000000002},"last":0.51100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.316,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.80600000000000005,"15min":0.873},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.62,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61299999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58299999999999996,"5min":0.74199999999999999,"15min":0.85899999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.65200000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.58199999999999996,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.70699999999999996,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.65200000000000002,"5min":0.84699999999999998,"15min":0.93799999999999994},"last":0.63400000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.70699999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.374},"max":{"1min":0.77800000000000002,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.72599999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.624,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.78100000000000003,"5min":0.875,"15min":0.875},"last":0.74199999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.748,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.34899999999999998},"max":{"1min":0.76700000000000002,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.77500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1712128,"data_stored":1103192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.308,"15min":0.59299999999999997},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.45800000000000002,"5min":0.502,"15min":11.971},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.89000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.51700000000000002,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.26800000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.625},"min":{"1min":0.23400000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.495,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.34300000000000003},"min":{"1min":0.246,"5min":0.224,"15min":0.224},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.58599999999999997},"last":0.28799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.64100000000000001},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":10.99},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.373,"15min":0.92500000000000004},"min":{"1min":0.318,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.54700000000000004,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.36099999999999999,"15min":0.33400000000000002},"max":{"1min":0.76900000000000002,"5min":0.90700000000000003,"15min":1.26},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.80700000000000005},"min":{"1min":0.434,"5min":0.37,"15min":0.29099999999999998},"max":{"1min":0.64500000000000002,"5min":0.77600000000000002,"15min":10.957000000000001},"last":0.61299999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":1.121},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.36199999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71499999999999997,"5min":0.88700000000000001,"15min":1.296},"last":0.624}]},{"osd":6,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.84299999999999997},"min":{"1min":0.44800000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":1.1379999999999999},"min":{"1min":0.435,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.74299999999999999,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.53900000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":1.1319999999999999},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84199999999999997,"15min":22.988},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.63500000000000001},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":1.327},"last":0.59899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":513131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.29699999999999999,"15min":0.311},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.33200000000000002,"15min":0.34999999999999998},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.27000000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.24399999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.628,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.33200000000000002,"15min":0.33900000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.374}]},{"osd":3,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.33700000000000002,"15min":0.34599999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.502,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.27500000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.54200000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.41499999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79100000000000004,"5min":0.92500000000000004,"15min":1.214},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.84299999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.77500000000000002,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85399999999999998,"5min":0.90200000000000002,"15min":1.603},"last":0.628}]},{"osd":6,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.83099999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.74099999999999999,"5min":0.94199999999999995,"15min":1.383},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82699999999999996,"5min":0.88900000000000001,"15min":0.92000000000000004},"last":0.78900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1118208,"data_stored":512824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29799999999999999,"15min":0.30199999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40500000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.314,"5min":0.316,"15min":0.317},"min":{"1min":0.22700000000000001,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.629},"last":0.26100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.317,"15min":0.32400000000000001},"min":{"1min":0.222,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.36499999999999999,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33800000000000002,"15min":0.34599999999999997},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47799999999999998,"5min":0.64200000000000002,"15min":0.749},"last":0.28299999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36099999999999999,"15min":0.38200000000000001},"min":{"1min":0.318,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41599999999999998,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.313},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.376,"15min":0.39800000000000002},"min":{"1min":0.30599999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.69499999999999995},"last":0.374}]},{"osd":4,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.68600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.433,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.76100000000000001,"5min":0.78400000000000003,"15min":0.879},"last":0.40799999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70699999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.40999999999999998,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.73399999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.495}]},{"osd":6,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77200000000000002,"5min":0.82899999999999996,"15min":0.85299999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.59399999999999997},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.65900000000000003,"5min":0.80500000000000005,"15min":0.83599999999999997},"last":0.54400000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.60899999999999999},"min":{"1min":0.438,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574966,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1118208,"data_stored":513372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29699999999999999,"15min":0.29999999999999999},"min":{"1min":0.21299999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.373},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32100000000000001,"15min":0.312},"min":{"1min":0.23100000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.38400000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.29899999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32300000000000001,"15min":0.313},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.19500000000000001},"max":{"1min":0.46700000000000003,"5min":0.50600000000000001,"15min":0.60399999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":0.44900000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"last":0.32100000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.28499999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.53300000000000003,"5min":0.57199999999999995,"15min":0.57599999999999996},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.52500000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.35399999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.28599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.503}]},{"osd":5,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.371,"15min":0.314},"max":{"1min":0.79100000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57899999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.373},"max":{"1min":0.80400000000000005,"5min":1.091,"15min":1.091},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.61199999999999999},"min":{"1min":0.41699999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78400000000000003,"5min":1.111,"15min":1.111},"last":0.624}]},{"osd":7,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.61199999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74399999999999999,"5min":1.032,"15min":1.236},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.61299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1064960,"data_stored":486211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.755,"5min":0.997,"15min":0.997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.68700000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53300000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.752,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.46300000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.51000000000000001},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.69199999999999995}]},{"osd":3,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50600000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.85899999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75700000000000001,"5min":0.85999999999999999,"15min":0.879},"last":0.67900000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70599999999999996,"5min":0.78900000000000003,"15min":0.90600000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59599999999999997,"15min":0.623},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.92300000000000004},"last":0.65200000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.80600000000000005,"15min":0.82499999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.89300000000000002},"last":0.66200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1064960,"data_stored":486211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.63300000000000001,"5min":0.78600000000000003,"15min":0.96899999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.29199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.60899999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70499999999999996,"5min":0.747,"15min":0.92200000000000004},"last":0.623},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52400000000000002,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.63200000000000001,"5min":0.72899999999999998,"15min":1.075},"last":0.56999999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.78000000000000003,"15min":0.95699999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54600000000000004,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.65300000000000002,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.51300000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72299999999999998,"5min":0.81000000000000005,"15min":1.03},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58199999999999996,"15min":0.59499999999999997},"min":{"1min":0.48499999999999999,"5min":0.36699999999999999,"15min":0.35199999999999998},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":1.026},"last":0.55300000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72999999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.495},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.315,"15min":0.26900000000000002},"max":{"1min":0.78900000000000003,"5min":0.86399999999999999,"15min":0.91400000000000003},"last":0.40600000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.76500000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.48399999999999999,"5min":0.36599999999999999,"15min":0.28799999999999998},"max":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.94199999999999995},"last":0.433}]},{"osd":7,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.51300000000000001,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.95899999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.61199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.83699999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:26:23.517 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:26:23.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.669+0000 7fb199312700 1 Processor -- start 2023-08-27T11:26:23.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.669+0000 7fb199312700 1 -- start start 2023-08-27T11:26:23.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.669+0000 7fb199312700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb194161600 0x7fb1940d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:23.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.669+0000 7fb199312700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1940d0c60 0x7fb19415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:23.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.669+0000 7fb199312700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb19415f110 con 0x7fb194157880 2023-08-27T11:26:23.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.669+0000 7fb199312700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1940d1490 con 0x7fb194161600 2023-08-27T11:26:23.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.669+0000 7fb199312700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1940d1770 con 0x7fb1940d0c60 2023-08-27T11:26:23.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.670+0000 7fb1927fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb194161600 0x7fb1940d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:23.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.670+0000 7fb192ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb194157880 0x7fb194160ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55526/0 (socket says 172.21.15.112:55526) 2023-08-27T11:26:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.670+0000 7fb192ffd700 1 -- 172.21.15.112:0/1937183510 learned_addr learned my addr 172.21.15.112:0/1937183510 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.670+0000 7fb1937fe700 1 --2- 172.21.15.112:0/1937183510 >> v2:172.21.15.138:3300/0 conn(0x7fb1940d0c60 0x7fb19415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.670+0000 7fb1927fc700 1 -- 172.21.15.112:0/1937183510 >> v2:172.21.15.138:3300/0 conn(0x7fb1940d0c60 msgr2=0x7fb19415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.670+0000 7fb1927fc700 1 --2- 172.21.15.112:0/1937183510 >> v2:172.21.15.138:3300/0 conn(0x7fb1940d0c60 0x7fb19415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.670+0000 7fb1927fc700 1 -- 172.21.15.112:0/1937183510 >> v1:172.21.15.112:6789/0 conn(0x7fb194157880 legacy=0x7fb194160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.670+0000 7fb191ffb700 1 -- 172.21.15.112:0/1937183510 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300252138 0 0) 0x7fb19415f110 con 0x7fb194157880 2023-08-27T11:26:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.670+0000 7fb1927fc700 1 -- 172.21.15.112:0/1937183510 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19415eac0 con 0x7fb194161600 2023-08-27T11:26:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.670+0000 7fb1927fc700 1 --2- 172.21.15.112:0/1937183510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb194161600 0x7fb1940d0720 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb19415f080 tx=0x7fb188009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4e60b98a27a347 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.670+0000 7fb191ffb700 1 -- 172.21.15.112:0/1937183510 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188011830 con 0x7fb194161600 2023-08-27T11:26:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.670+0000 7fb191ffb700 1 -- 172.21.15.112:0/1937183510 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb188011990 con 0x7fb194161600 2023-08-27T11:26:23.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.670+0000 7fb191ffb700 1 -- 172.21.15.112:0/1937183510 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188011d20 con 0x7fb194161600 2023-08-27T11:26:23.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.671+0000 7fb199312700 1 -- 172.21.15.112:0/1937183510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb194161600 msgr2=0x7fb1940d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.671+0000 7fb199312700 1 --2- 172.21.15.112:0/1937183510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb194161600 0x7fb1940d0720 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb19415f080 tx=0x7fb188009b20 comp rx=0 tx=0).stop 2023-08-27T11:26:23.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.671+0000 7fb199312700 1 -- 172.21.15.112:0/1937183510 shutdown_connections 2023-08-27T11:26:23.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.671+0000 7fb199312700 1 --2- 172.21.15.112:0/1937183510 >> v2:172.21.15.138:3300/0 conn(0x7fb1940d0c60 0x7fb19415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.671+0000 7fb199312700 1 --2- 172.21.15.112:0/1937183510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb194161600 0x7fb1940d0720 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.671+0000 7fb199312700 1 -- 172.21.15.112:0/1937183510 >> 172.21.15.112:0/1937183510 conn(0x7fb19406c880 msgr2=0x7fb1940717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:23.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.671+0000 7fb199312700 1 -- 172.21.15.112:0/1937183510 shutdown_connections 2023-08-27T11:26:23.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.671+0000 7fb199312700 1 -- 172.21.15.112:0/1937183510 wait complete. 2023-08-27T11:26:23.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.672+0000 7fb199312700 1 Processor -- start 2023-08-27T11:26:23.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.672+0000 7fb199312700 1 -- start start 2023-08-27T11:26:23.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb199312700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1940d0c60 0x7fb1941f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb199312700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb194157880 0x7fb194070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb199312700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb194161600 0x7fb1941f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb199312700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb194070d10 con 0x7fb194157880 2023-08-27T11:26:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb199312700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb194158040 con 0x7fb194161600 2023-08-27T11:26:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb199312700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb194158320 con 0x7fb1940d0c60 2023-08-27T11:26:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb1927fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb194157880 0x7fb194070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb1927fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb194157880 0x7fb194070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32920/0 (socket says 172.21.15.112:32920) 2023-08-27T11:26:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb1927fc700 1 -- 172.21.15.112:0/4059550559 learned_addr learned my addr 172.21.15.112:0/4059550559 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb192ffd700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1940d0c60 0x7fb1941f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb1937fe700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb194161600 0x7fb1941f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb1927fc700 1 -- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1940d0c60 msgr2=0x7fb1941f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb1927fc700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1940d0c60 0x7fb1941f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb1927fc700 1 -- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb194161600 msgr2=0x7fb1941f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb1927fc700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb194161600 0x7fb1941f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb1927fc700 1 -- 172.21.15.112:0/4059550559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1880094a0 con 0x7fb194157880 2023-08-27T11:26:23.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb1937fe700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb194161600 0x7fb1941f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:23.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.673+0000 7fb1927fc700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb194157880 0x7fb194070010 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb188014040 tx=0x7fb188004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:23.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.674+0000 7fb17b7fe700 1 -- 172.21.15.112:0/4059550559 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1880038a0 con 0x7fb194157880 2023-08-27T11:26:23.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.674+0000 7fb17b7fe700 1 -- 172.21.15.112:0/4059550559 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18801e070 con 0x7fb194157880 2023-08-27T11:26:23.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.674+0000 7fb199312700 1 -- 172.21.15.112:0/4059550559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb194158600 con 0x7fb194157880 2023-08-27T11:26:23.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.674+0000 7fb17b7fe700 1 -- 172.21.15.112:0/4059550559 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188003c20 con 0x7fb194157880 2023-08-27T11:26:23.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.674+0000 7fb199312700 1 -- 172.21.15.112:0/4059550559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb194158bc0 con 0x7fb194157880 2023-08-27T11:26:23.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.674+0000 7fb199312700 1 -- 172.21.15.112:0/4059550559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb194067a20 con 0x7fb194157880 2023-08-27T11:26:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.675+0000 7fb17b7fe700 1 -- 172.21.15.112:0/4059550559 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fb18802d450 con 0x7fb194157880 2023-08-27T11:26:23.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.675+0000 7fb17b7fe700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb18006a3b0 0x7fb18006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:23.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.676+0000 7fb17b7fe700 1 -- 172.21.15.112:0/4059550559 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb188089a00 con 0x7fb194157880 2023-08-27T11:26:23.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.676+0000 7fb192ffd700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb18006a3b0 0x7fb18006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:23.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.676+0000 7fb192ffd700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb18006a3b0 0x7fb18006c870 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb184000f60 tx=0x7fb184007be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:23.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.680+0000 7fb17b7fe700 1 -- 172.21.15.112:0/4059550559 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb188054df0 con 0x7fb194157880 2023-08-27T11:26:23.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.864+0000 7fb199312700 1 -- 172.21.15.112:0/4059550559 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb19415eda0 con 0x7fb18006a3b0 2023-08-27T11:26:23.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.873+0000 7fb17b7fe700 1 -- 172.21.15.112:0/4059550559 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307418 (secure 0 0 0) 0x7fb1840012b0 con 0x7fb18006a3b0 2023-08-27T11:26:23.880 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:23.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.876+0000 7fb199312700 1 -- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb18006a3b0 msgr2=0x7fb18006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.876+0000 7fb199312700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb18006a3b0 0x7fb18006c870 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb184000f60 tx=0x7fb184007be0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.876+0000 7fb199312700 1 -- 172.21.15.112:0/4059550559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb194157880 msgr2=0x7fb194070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:23.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.876+0000 7fb199312700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb194157880 0x7fb194070010 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb188014040 tx=0x7fb188004750 comp rx=0 tx=0).stop 2023-08-27T11:26:23.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.876+0000 7fb199312700 1 -- 172.21.15.112:0/4059550559 shutdown_connections 2023-08-27T11:26:23.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.876+0000 7fb199312700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb18006a3b0 0x7fb18006c870 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.876+0000 7fb199312700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb194161600 0x7fb1941f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.876+0000 7fb199312700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1940d0c60 0x7fb1941f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.876+0000 7fb199312700 1 --2- 172.21.15.112:0/4059550559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb194157880 0x7fb194070010 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:23.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.876+0000 7fb199312700 1 -- 172.21.15.112:0/4059550559 >> 172.21.15.112:0/4059550559 conn(0x7fb19406c880 msgr2=0x7fb19415cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:23.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.876+0000 7fb199312700 1 -- 172.21.15.112:0/4059550559 shutdown_connections 2023-08-27T11:26:23.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:23.876+0000 7fb199312700 1 -- 172.21.15.112:0/4059550559 wait complete. 2023-08-27T11:26:23.886 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:26:23.889 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2023-08-27T11:26:22.368926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592618,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":183,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":154,"acting":154,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259232,"kb_used_data":9764,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754715488,"statfs":{"total":773094113280,"available":772828659712,"internally_reserved":0,"allocated":9998336,"data_stored":5206526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2250,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.970088"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874566+0000","last_change":"2023-08-27T11:26:19.427207+0000","last_active":"2023-08-27T11:26:20.874566+0000","last_peered":"2023-08-27T11:26:20.874566+0000","last_clean":"2023-08-27T11:26:20.874566+0000","last_became_active":"2023-08-27T11:26:19.426963+0000","last_became_peered":"2023-08-27T11:26:19.426963+0000","last_unstale":"2023-08-27T11:26:20.874566+0000","last_undegraded":"2023-08-27T11:26:20.874566+0000","last_fullsized":"2023-08-27T11:26:20.874566+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:55:21.733240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430931+0000","last_change":"2023-08-27T11:26:19.431066+0000","last_active":"2023-08-27T11:26:20.430931+0000","last_peered":"2023-08-27T11:26:20.430931+0000","last_clean":"2023-08-27T11:26:20.430931+0000","last_became_active":"2023-08-27T11:26:19.430477+0000","last_became_peered":"2023-08-27T11:26:19.430477+0000","last_unstale":"2023-08-27T11:26:20.430931+0000","last_undegraded":"2023-08-27T11:26:20.430931+0000","last_fullsized":"2023-08-27T11:26:20.430931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:43:44.337358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874230+0000","last_change":"2023-08-27T11:26:19.434887+0000","last_active":"2023-08-27T11:26:20.874230+0000","last_peered":"2023-08-27T11:26:20.874230+0000","last_clean":"2023-08-27T11:26:20.874230+0000","last_became_active":"2023-08-27T11:26:19.434566+0000","last_became_peered":"2023-08-27T11:26:19.434566+0000","last_unstale":"2023-08-27T11:26:20.874230+0000","last_undegraded":"2023-08-27T11:26:20.874230+0000","last_fullsized":"2023-08-27T11:26:20.874230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:11:11.738619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425453+0000","last_change":"2023-08-27T11:26:19.426786+0000","last_active":"2023-08-27T11:26:20.425453+0000","last_peered":"2023-08-27T11:26:20.425453+0000","last_clean":"2023-08-27T11:26:20.425453+0000","last_became_active":"2023-08-27T11:26:19.426494+0000","last_became_peered":"2023-08-27T11:26:19.426494+0000","last_unstale":"2023-08-27T11:26:20.425453+0000","last_undegraded":"2023-08-27T11:26:20.425453+0000","last_fullsized":"2023-08-27T11:26:20.425453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:33:53.249169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057365+0000","last_change":"2023-08-27T11:26:19.424084+0000","last_active":"2023-08-27T11:26:21.057365+0000","last_peered":"2023-08-27T11:26:21.057365+0000","last_clean":"2023-08-27T11:26:21.057365+0000","last_became_active":"2023-08-27T11:26:19.422955+0000","last_became_peered":"2023-08-27T11:26:19.422955+0000","last_unstale":"2023-08-27T11:26:21.057365+0000","last_undegraded":"2023-08-27T11:26:21.057365+0000","last_fullsized":"2023-08-27T11:26:21.057365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:59:24.896402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430830+0000","last_change":"2023-08-27T11:26:19.424659+0000","last_active":"2023-08-27T11:26:20.430830+0000","last_peered":"2023-08-27T11:26:20.430830+0000","last_clean":"2023-08-27T11:26:20.430830+0000","last_became_active":"2023-08-27T11:26:19.424317+0000","last_became_peered":"2023-08-27T11:26:19.424317+0000","last_unstale":"2023-08-27T11:26:20.430830+0000","last_undegraded":"2023-08-27T11:26:20.430830+0000","last_fullsized":"2023-08-27T11:26:20.430830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:47:42.857298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430038+0000","last_change":"2023-08-27T11:26:19.421667+0000","last_active":"2023-08-27T11:26:20.430038+0000","last_peered":"2023-08-27T11:26:20.430038+0000","last_clean":"2023-08-27T11:26:20.430038+0000","last_became_active":"2023-08-27T11:26:19.421352+0000","last_became_peered":"2023-08-27T11:26:19.421352+0000","last_unstale":"2023-08-27T11:26:20.430038+0000","last_undegraded":"2023-08-27T11:26:20.430038+0000","last_fullsized":"2023-08-27T11:26:20.430038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:24.356051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384297+0000","last_change":"2023-08-27T11:26:19.430958+0000","last_active":"2023-08-27T11:26:21.384297+0000","last_peered":"2023-08-27T11:26:21.384297+0000","last_clean":"2023-08-27T11:26:21.384297+0000","last_became_active":"2023-08-27T11:26:19.430685+0000","last_became_peered":"2023-08-27T11:26:19.430685+0000","last_unstale":"2023-08-27T11:26:21.384297+0000","last_undegraded":"2023-08-27T11:26:21.384297+0000","last_fullsized":"2023-08-27T11:26:21.384297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:41.805565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431121+0000","last_change":"2023-08-27T11:26:19.430211+0000","last_active":"2023-08-27T11:26:20.431121+0000","last_peered":"2023-08-27T11:26:20.431121+0000","last_clean":"2023-08-27T11:26:20.431121+0000","last_became_active":"2023-08-27T11:26:19.429921+0000","last_became_peered":"2023-08-27T11:26:19.429921+0000","last_unstale":"2023-08-27T11:26:20.431121+0000","last_undegraded":"2023-08-27T11:26:20.431121+0000","last_fullsized":"2023-08-27T11:26:20.431121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:56:34.526955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057756+0000","last_change":"2023-08-27T11:26:19.428983+0000","last_active":"2023-08-27T11:26:21.057756+0000","last_peered":"2023-08-27T11:26:21.057756+0000","last_clean":"2023-08-27T11:26:21.057756+0000","last_became_active":"2023-08-27T11:26:19.428699+0000","last_became_peered":"2023-08-27T11:26:19.428699+0000","last_unstale":"2023-08-27T11:26:21.057756+0000","last_undegraded":"2023-08-27T11:26:21.057756+0000","last_fullsized":"2023-08-27T11:26:21.057756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:49:32.779304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425155+0000","last_change":"2023-08-27T11:26:19.436540+0000","last_active":"2023-08-27T11:26:20.425155+0000","last_peered":"2023-08-27T11:26:20.425155+0000","last_clean":"2023-08-27T11:26:20.425155+0000","last_became_active":"2023-08-27T11:26:19.436278+0000","last_became_peered":"2023-08-27T11:26:19.436278+0000","last_unstale":"2023-08-27T11:26:20.425155+0000","last_undegraded":"2023-08-27T11:26:20.425155+0000","last_fullsized":"2023-08-27T11:26:20.425155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:40:05.207374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874577+0000","last_change":"2023-08-27T11:26:19.434923+0000","last_active":"2023-08-27T11:26:20.874577+0000","last_peered":"2023-08-27T11:26:20.874577+0000","last_clean":"2023-08-27T11:26:20.874577+0000","last_became_active":"2023-08-27T11:26:19.434650+0000","last_became_peered":"2023-08-27T11:26:19.434650+0000","last_unstale":"2023-08-27T11:26:20.874577+0000","last_undegraded":"2023-08-27T11:26:20.874577+0000","last_fullsized":"2023-08-27T11:26:20.874577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:33:06.493994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430846+0000","last_change":"2023-08-27T11:26:19.422300+0000","last_active":"2023-08-27T11:26:20.430846+0000","last_peered":"2023-08-27T11:26:20.430846+0000","last_clean":"2023-08-27T11:26:20.430846+0000","last_became_active":"2023-08-27T11:26:19.421284+0000","last_became_peered":"2023-08-27T11:26:19.421284+0000","last_unstale":"2023-08-27T11:26:20.430846+0000","last_undegraded":"2023-08-27T11:26:20.430846+0000","last_fullsized":"2023-08-27T11:26:20.430846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:12.363650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874425+0000","last_change":"2023-08-27T11:26:19.418470+0000","last_active":"2023-08-27T11:26:20.874425+0000","last_peered":"2023-08-27T11:26:20.874425+0000","last_clean":"2023-08-27T11:26:20.874425+0000","last_became_active":"2023-08-27T11:26:19.418192+0000","last_became_peered":"2023-08-27T11:26:19.418192+0000","last_unstale":"2023-08-27T11:26:20.874425+0000","last_undegraded":"2023-08-27T11:26:20.874425+0000","last_fullsized":"2023-08-27T11:26:20.874425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:14:53.959765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384767+0000","last_change":"2023-08-27T11:26:19.433627+0000","last_active":"2023-08-27T11:26:21.384767+0000","last_peered":"2023-08-27T11:26:21.384767+0000","last_clean":"2023-08-27T11:26:21.384767+0000","last_became_active":"2023-08-27T11:26:19.433370+0000","last_became_peered":"2023-08-27T11:26:19.433370+0000","last_unstale":"2023-08-27T11:26:21.384767+0000","last_undegraded":"2023-08-27T11:26:21.384767+0000","last_fullsized":"2023-08-27T11:26:21.384767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:41.696204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057432+0000","last_change":"2023-08-27T11:26:19.433519+0000","last_active":"2023-08-27T11:26:21.057432+0000","last_peered":"2023-08-27T11:26:21.057432+0000","last_clean":"2023-08-27T11:26:21.057432+0000","last_became_active":"2023-08-27T11:26:19.433245+0000","last_became_peered":"2023-08-27T11:26:19.433245+0000","last_unstale":"2023-08-27T11:26:21.057432+0000","last_undegraded":"2023-08-27T11:26:21.057432+0000","last_fullsized":"2023-08-27T11:26:21.057432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:07:00.066813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874575+0000","last_change":"2023-08-27T11:26:19.434762+0000","last_active":"2023-08-27T11:26:20.874575+0000","last_peered":"2023-08-27T11:26:20.874575+0000","last_clean":"2023-08-27T11:26:20.874575+0000","last_became_active":"2023-08-27T11:26:19.434462+0000","last_became_peered":"2023-08-27T11:26:19.434462+0000","last_unstale":"2023-08-27T11:26:20.874575+0000","last_undegraded":"2023-08-27T11:26:20.874575+0000","last_fullsized":"2023-08-27T11:26:20.874575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:41.356424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057290+0000","last_change":"2023-08-27T11:26:19.423833+0000","last_active":"2023-08-27T11:26:21.057290+0000","last_peered":"2023-08-27T11:26:21.057290+0000","last_clean":"2023-08-27T11:26:21.057290+0000","last_became_active":"2023-08-27T11:26:19.423440+0000","last_became_peered":"2023-08-27T11:26:19.423440+0000","last_unstale":"2023-08-27T11:26:21.057290+0000","last_undegraded":"2023-08-27T11:26:21.057290+0000","last_fullsized":"2023-08-27T11:26:21.057290+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:41:13.840925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430610+0000","last_change":"2023-08-27T11:26:19.433762+0000","last_active":"2023-08-27T11:26:20.430610+0000","last_peered":"2023-08-27T11:26:20.430610+0000","last_clean":"2023-08-27T11:26:20.430610+0000","last_became_active":"2023-08-27T11:26:19.433489+0000","last_became_peered":"2023-08-27T11:26:19.433489+0000","last_unstale":"2023-08-27T11:26:20.430610+0000","last_undegraded":"2023-08-27T11:26:20.430610+0000","last_fullsized":"2023-08-27T11:26:20.430610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:34.644697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384668+0000","last_change":"2023-08-27T11:26:19.431428+0000","last_active":"2023-08-27T11:26:21.384668+0000","last_peered":"2023-08-27T11:26:21.384668+0000","last_clean":"2023-08-27T11:26:21.384668+0000","last_became_active":"2023-08-27T11:26:19.431169+0000","last_became_peered":"2023-08-27T11:26:19.431169+0000","last_unstale":"2023-08-27T11:26:21.384668+0000","last_undegraded":"2023-08-27T11:26:21.384668+0000","last_fullsized":"2023-08-27T11:26:21.384668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:30:35.082113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430321+0000","last_change":"2023-08-27T11:26:19.427535+0000","last_active":"2023-08-27T11:26:20.430321+0000","last_peered":"2023-08-27T11:26:20.430321+0000","last_clean":"2023-08-27T11:26:20.430321+0000","last_became_active":"2023-08-27T11:26:19.426191+0000","last_became_peered":"2023-08-27T11:26:19.426191+0000","last_unstale":"2023-08-27T11:26:20.430321+0000","last_undegraded":"2023-08-27T11:26:20.430321+0000","last_fullsized":"2023-08-27T11:26:20.430321+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:22:47.314202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384374+0000","last_change":"2023-08-27T11:26:19.424642+0000","last_active":"2023-08-27T11:26:21.384374+0000","last_peered":"2023-08-27T11:26:21.384374+0000","last_clean":"2023-08-27T11:26:21.384374+0000","last_became_active":"2023-08-27T11:26:19.424359+0000","last_became_peered":"2023-08-27T11:26:19.424359+0000","last_unstale":"2023-08-27T11:26:21.384374+0000","last_undegraded":"2023-08-27T11:26:21.384374+0000","last_fullsized":"2023-08-27T11:26:21.384374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:06:36.252165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874126+0000","last_change":"2023-08-27T11:26:19.414983+0000","last_active":"2023-08-27T11:26:20.874126+0000","last_peered":"2023-08-27T11:26:20.874126+0000","last_clean":"2023-08-27T11:26:20.874126+0000","last_became_active":"2023-08-27T11:26:19.414726+0000","last_became_peered":"2023-08-27T11:26:19.414726+0000","last_unstale":"2023-08-27T11:26:20.874126+0000","last_undegraded":"2023-08-27T11:26:20.874126+0000","last_fullsized":"2023-08-27T11:26:20.874126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:57:45.405543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"89'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425328+0000","last_change":"2023-08-27T11:26:18.432220+0000","last_active":"2023-08-27T11:26:20.425328+0000","last_peered":"2023-08-27T11:26:20.425328+0000","last_clean":"2023-08-27T11:26:20.425328+0000","last_became_active":"2023-08-27T11:26:18.431938+0000","last_became_peered":"2023-08-27T11:26:18.431938+0000","last_unstale":"2023-08-27T11:26:20.425328+0000","last_undegraded":"2023-08-27T11:26:20.425328+0000","last_fullsized":"2023-08-27T11:26:20.425328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:26:24.683316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384352+0000","last_change":"2023-08-27T11:26:19.427981+0000","last_active":"2023-08-27T11:26:21.384352+0000","last_peered":"2023-08-27T11:26:21.384352+0000","last_clean":"2023-08-27T11:26:21.384352+0000","last_became_active":"2023-08-27T11:26:19.427694+0000","last_became_peered":"2023-08-27T11:26:19.427694+0000","last_unstale":"2023-08-27T11:26:21.384352+0000","last_undegraded":"2023-08-27T11:26:21.384352+0000","last_fullsized":"2023-08-27T11:26:21.384352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:31.571236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384746+0000","last_change":"2023-08-27T11:26:18.416729+0000","last_active":"2023-08-27T11:26:21.384746+0000","last_peered":"2023-08-27T11:26:21.384746+0000","last_clean":"2023-08-27T11:26:21.384746+0000","last_became_active":"2023-08-27T11:26:18.416320+0000","last_became_peered":"2023-08-27T11:26:18.416320+0000","last_unstale":"2023-08-27T11:26:21.384746+0000","last_undegraded":"2023-08-27T11:26:21.384746+0000","last_fullsized":"2023-08-27T11:26:21.384746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:32.457209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057704+0000","last_change":"2023-08-27T11:26:18.426839+0000","last_active":"2023-08-27T11:26:21.057704+0000","last_peered":"2023-08-27T11:26:21.057704+0000","last_clean":"2023-08-27T11:26:21.057704+0000","last_became_active":"2023-08-27T11:26:18.426578+0000","last_became_peered":"2023-08-27T11:26:18.426578+0000","last_unstale":"2023-08-27T11:26:21.057704+0000","last_undegraded":"2023-08-27T11:26:21.057704+0000","last_fullsized":"2023-08-27T11:26:21.057704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:23.764319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425272+0000","last_change":"2023-08-27T11:26:19.426762+0000","last_active":"2023-08-27T11:26:20.425272+0000","last_peered":"2023-08-27T11:26:20.425272+0000","last_clean":"2023-08-27T11:26:20.425272+0000","last_became_active":"2023-08-27T11:26:19.426146+0000","last_became_peered":"2023-08-27T11:26:19.426146+0000","last_unstale":"2023-08-27T11:26:20.425272+0000","last_undegraded":"2023-08-27T11:26:20.425272+0000","last_fullsized":"2023-08-27T11:26:20.425272+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:54:50.371981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874343+0000","last_change":"2023-08-27T11:26:19.418506+0000","last_active":"2023-08-27T11:26:20.874343+0000","last_peered":"2023-08-27T11:26:20.874343+0000","last_clean":"2023-08-27T11:26:20.874343+0000","last_became_active":"2023-08-27T11:26:19.418235+0000","last_became_peered":"2023-08-27T11:26:19.418235+0000","last_unstale":"2023-08-27T11:26:20.874343+0000","last_undegraded":"2023-08-27T11:26:20.874343+0000","last_fullsized":"2023-08-27T11:26:20.874343+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:50.719209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430641+0000","last_change":"2023-08-27T11:26:19.431258+0000","last_active":"2023-08-27T11:26:20.430641+0000","last_peered":"2023-08-27T11:26:20.430641+0000","last_clean":"2023-08-27T11:26:20.430641+0000","last_became_active":"2023-08-27T11:26:19.430816+0000","last_became_peered":"2023-08-27T11:26:19.430816+0000","last_unstale":"2023-08-27T11:26:20.430641+0000","last_undegraded":"2023-08-27T11:26:20.430641+0000","last_fullsized":"2023-08-27T11:26:20.430641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:55.969633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425484+0000","last_change":"2023-08-27T11:26:19.426307+0000","last_active":"2023-08-27T11:26:20.425484+0000","last_peered":"2023-08-27T11:26:20.425484+0000","last_clean":"2023-08-27T11:26:20.425484+0000","last_became_active":"2023-08-27T11:26:19.425959+0000","last_became_peered":"2023-08-27T11:26:19.425959+0000","last_unstale":"2023-08-27T11:26:20.425484+0000","last_undegraded":"2023-08-27T11:26:20.425484+0000","last_fullsized":"2023-08-27T11:26:20.425484+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:10:02.440599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425034+0000","last_change":"2023-08-27T11:26:18.429003+0000","last_active":"2023-08-27T11:26:20.425034+0000","last_peered":"2023-08-27T11:26:20.425034+0000","last_clean":"2023-08-27T11:26:20.425034+0000","last_became_active":"2023-08-27T11:26:18.428701+0000","last_became_peered":"2023-08-27T11:26:18.428701+0000","last_unstale":"2023-08-27T11:26:20.425034+0000","last_undegraded":"2023-08-27T11:26:20.425034+0000","last_fullsized":"2023-08-27T11:26:20.425034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:42:33.783460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874118+0000","last_change":"2023-08-27T11:26:19.417293+0000","last_active":"2023-08-27T11:26:20.874118+0000","last_peered":"2023-08-27T11:26:20.874118+0000","last_clean":"2023-08-27T11:26:20.874118+0000","last_became_active":"2023-08-27T11:26:19.417018+0000","last_became_peered":"2023-08-27T11:26:19.417018+0000","last_unstale":"2023-08-27T11:26:20.874118+0000","last_undegraded":"2023-08-27T11:26:20.874118+0000","last_fullsized":"2023-08-27T11:26:20.874118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:02:56.092248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057401+0000","last_change":"2023-08-27T11:26:19.425413+0000","last_active":"2023-08-27T11:26:21.057401+0000","last_peered":"2023-08-27T11:26:21.057401+0000","last_clean":"2023-08-27T11:26:21.057401+0000","last_became_active":"2023-08-27T11:26:19.424482+0000","last_became_peered":"2023-08-27T11:26:19.424482+0000","last_unstale":"2023-08-27T11:26:21.057401+0000","last_undegraded":"2023-08-27T11:26:21.057401+0000","last_fullsized":"2023-08-27T11:26:21.057401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:36.587246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874328+0000","last_change":"2023-08-27T11:26:18.426356+0000","last_active":"2023-08-27T11:26:20.874328+0000","last_peered":"2023-08-27T11:26:20.874328+0000","last_clean":"2023-08-27T11:26:20.874328+0000","last_became_active":"2023-08-27T11:26:18.426041+0000","last_became_peered":"2023-08-27T11:26:18.426041+0000","last_unstale":"2023-08-27T11:26:20.874328+0000","last_undegraded":"2023-08-27T11:26:20.874328+0000","last_fullsized":"2023-08-27T11:26:20.874328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:28.381950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384393+0000","last_change":"2023-08-27T11:26:19.431679+0000","last_active":"2023-08-27T11:26:21.384393+0000","last_peered":"2023-08-27T11:26:21.384393+0000","last_clean":"2023-08-27T11:26:21.384393+0000","last_became_active":"2023-08-27T11:26:19.431405+0000","last_became_peered":"2023-08-27T11:26:19.431405+0000","last_unstale":"2023-08-27T11:26:21.384393+0000","last_undegraded":"2023-08-27T11:26:21.384393+0000","last_fullsized":"2023-08-27T11:26:21.384393+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:02:21.450786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057480+0000","last_change":"2023-08-27T11:26:19.424582+0000","last_active":"2023-08-27T11:26:21.057480+0000","last_peered":"2023-08-27T11:26:21.057480+0000","last_clean":"2023-08-27T11:26:21.057480+0000","last_became_active":"2023-08-27T11:26:19.424288+0000","last_became_peered":"2023-08-27T11:26:19.424288+0000","last_unstale":"2023-08-27T11:26:21.057480+0000","last_undegraded":"2023-08-27T11:26:21.057480+0000","last_fullsized":"2023-08-27T11:26:21.057480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:16.279109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430741+0000","last_change":"2023-08-27T11:26:18.421802+0000","last_active":"2023-08-27T11:26:20.430741+0000","last_peered":"2023-08-27T11:26:20.430741+0000","last_clean":"2023-08-27T11:26:20.430741+0000","last_became_active":"2023-08-27T11:26:18.421093+0000","last_became_peered":"2023-08-27T11:26:18.421093+0000","last_unstale":"2023-08-27T11:26:20.430741+0000","last_undegraded":"2023-08-27T11:26:20.430741+0000","last_fullsized":"2023-08-27T11:26:20.430741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:29.408679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057739+0000","last_change":"2023-08-27T11:26:18.421876+0000","last_active":"2023-08-27T11:26:21.057739+0000","last_peered":"2023-08-27T11:26:21.057739+0000","last_clean":"2023-08-27T11:26:21.057739+0000","last_became_active":"2023-08-27T11:26:18.421583+0000","last_became_peered":"2023-08-27T11:26:18.421583+0000","last_unstale":"2023-08-27T11:26:21.057739+0000","last_undegraded":"2023-08-27T11:26:21.057739+0000","last_fullsized":"2023-08-27T11:26:21.057739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:48:43.946799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430199+0000","last_change":"2023-08-27T11:26:19.430893+0000","last_active":"2023-08-27T11:26:20.430199+0000","last_peered":"2023-08-27T11:26:20.430199+0000","last_clean":"2023-08-27T11:26:20.430199+0000","last_became_active":"2023-08-27T11:26:19.430225+0000","last_became_peered":"2023-08-27T11:26:19.430225+0000","last_unstale":"2023-08-27T11:26:20.430199+0000","last_undegraded":"2023-08-27T11:26:20.430199+0000","last_fullsized":"2023-08-27T11:26:20.430199+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:43.123979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425475+0000","last_change":"2023-08-27T11:26:18.432084+0000","last_active":"2023-08-27T11:26:20.425475+0000","last_peered":"2023-08-27T11:26:20.425475+0000","last_clean":"2023-08-27T11:26:20.425475+0000","last_became_active":"2023-08-27T11:26:18.431877+0000","last_became_peered":"2023-08-27T11:26:18.431877+0000","last_unstale":"2023-08-27T11:26:20.425475+0000","last_undegraded":"2023-08-27T11:26:20.425475+0000","last_fullsized":"2023-08-27T11:26:20.425475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:12:10.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384677+0000","last_change":"2023-08-27T11:26:19.429400+0000","last_active":"2023-08-27T11:26:21.384677+0000","last_peered":"2023-08-27T11:26:21.384677+0000","last_clean":"2023-08-27T11:26:21.384677+0000","last_became_active":"2023-08-27T11:26:19.429146+0000","last_became_peered":"2023-08-27T11:26:19.429146+0000","last_unstale":"2023-08-27T11:26:21.384677+0000","last_undegraded":"2023-08-27T11:26:21.384677+0000","last_fullsized":"2023-08-27T11:26:21.384677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:06.185951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384453+0000","last_change":"2023-08-27T11:26:18.420633+0000","last_active":"2023-08-27T11:26:21.384453+0000","last_peered":"2023-08-27T11:26:21.384453+0000","last_clean":"2023-08-27T11:26:21.384453+0000","last_became_active":"2023-08-27T11:26:18.420312+0000","last_became_peered":"2023-08-27T11:26:18.420312+0000","last_unstale":"2023-08-27T11:26:21.384453+0000","last_undegraded":"2023-08-27T11:26:21.384453+0000","last_fullsized":"2023-08-27T11:26:21.384453+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:20.872678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874639+0000","last_change":"2023-08-27T11:26:19.434950+0000","last_active":"2023-08-27T11:26:20.874639+0000","last_peered":"2023-08-27T11:26:20.874639+0000","last_clean":"2023-08-27T11:26:20.874639+0000","last_became_active":"2023-08-27T11:26:19.434684+0000","last_became_peered":"2023-08-27T11:26:19.434684+0000","last_unstale":"2023-08-27T11:26:20.874639+0000","last_undegraded":"2023-08-27T11:26:20.874639+0000","last_fullsized":"2023-08-27T11:26:20.874639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:52:42.875755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874262+0000","last_change":"2023-08-27T11:26:19.425141+0000","last_active":"2023-08-27T11:26:20.874262+0000","last_peered":"2023-08-27T11:26:20.874262+0000","last_clean":"2023-08-27T11:26:20.874262+0000","last_became_active":"2023-08-27T11:26:19.424669+0000","last_became_peered":"2023-08-27T11:26:19.424669+0000","last_unstale":"2023-08-27T11:26:20.874262+0000","last_undegraded":"2023-08-27T11:26:20.874262+0000","last_fullsized":"2023-08-27T11:26:20.874262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:31:34.706537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057687+0000","last_change":"2023-08-27T11:26:18.421572+0000","last_active":"2023-08-27T11:26:21.057687+0000","last_peered":"2023-08-27T11:26:21.057687+0000","last_clean":"2023-08-27T11:26:21.057687+0000","last_became_active":"2023-08-27T11:26:18.421268+0000","last_became_peered":"2023-08-27T11:26:18.421268+0000","last_unstale":"2023-08-27T11:26:21.057687+0000","last_undegraded":"2023-08-27T11:26:21.057687+0000","last_fullsized":"2023-08-27T11:26:21.057687+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:44.389523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874451+0000","last_change":"2023-08-27T11:26:18.429524+0000","last_active":"2023-08-27T11:26:20.874451+0000","last_peered":"2023-08-27T11:26:20.874451+0000","last_clean":"2023-08-27T11:26:20.874451+0000","last_became_active":"2023-08-27T11:26:18.429224+0000","last_became_peered":"2023-08-27T11:26:18.429224+0000","last_unstale":"2023-08-27T11:26:20.874451+0000","last_undegraded":"2023-08-27T11:26:20.874451+0000","last_fullsized":"2023-08-27T11:26:20.874451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:13.776898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425320+0000","last_change":"2023-08-27T11:26:19.427555+0000","last_active":"2023-08-27T11:26:20.425320+0000","last_peered":"2023-08-27T11:26:20.425320+0000","last_clean":"2023-08-27T11:26:20.425320+0000","last_became_active":"2023-08-27T11:26:19.427295+0000","last_became_peered":"2023-08-27T11:26:19.427295+0000","last_unstale":"2023-08-27T11:26:20.425320+0000","last_undegraded":"2023-08-27T11:26:20.425320+0000","last_fullsized":"2023-08-27T11:26:20.425320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:35.698617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430342+0000","last_change":"2023-08-27T11:26:18.432922+0000","last_active":"2023-08-27T11:26:20.430342+0000","last_peered":"2023-08-27T11:26:20.430342+0000","last_clean":"2023-08-27T11:26:20.430342+0000","last_became_active":"2023-08-27T11:26:18.432631+0000","last_became_peered":"2023-08-27T11:26:18.432631+0000","last_unstale":"2023-08-27T11:26:20.430342+0000","last_undegraded":"2023-08-27T11:26:20.430342+0000","last_fullsized":"2023-08-27T11:26:20.430342+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:58.703884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057644+0000","last_change":"2023-08-27T11:26:18.422687+0000","last_active":"2023-08-27T11:26:21.057644+0000","last_peered":"2023-08-27T11:26:21.057644+0000","last_clean":"2023-08-27T11:26:21.057644+0000","last_became_active":"2023-08-27T11:26:18.422404+0000","last_became_peered":"2023-08-27T11:26:18.422404+0000","last_unstale":"2023-08-27T11:26:21.057644+0000","last_undegraded":"2023-08-27T11:26:21.057644+0000","last_fullsized":"2023-08-27T11:26:21.057644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:34.961579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384633+0000","last_change":"2023-08-27T11:26:19.425732+0000","last_active":"2023-08-27T11:26:21.384633+0000","last_peered":"2023-08-27T11:26:21.384633+0000","last_clean":"2023-08-27T11:26:21.384633+0000","last_became_active":"2023-08-27T11:26:19.425452+0000","last_became_peered":"2023-08-27T11:26:19.425452+0000","last_unstale":"2023-08-27T11:26:21.384633+0000","last_undegraded":"2023-08-27T11:26:21.384633+0000","last_fullsized":"2023-08-27T11:26:21.384633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:15.049461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384761+0000","last_change":"2023-08-27T11:26:19.425658+0000","last_active":"2023-08-27T11:26:21.384761+0000","last_peered":"2023-08-27T11:26:21.384761+0000","last_clean":"2023-08-27T11:26:21.384761+0000","last_became_active":"2023-08-27T11:26:19.425387+0000","last_became_peered":"2023-08-27T11:26:19.425387+0000","last_unstale":"2023-08-27T11:26:21.384761+0000","last_undegraded":"2023-08-27T11:26:21.384761+0000","last_fullsized":"2023-08-27T11:26:21.384761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:52.205981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384498+0000","last_change":"2023-08-27T11:26:18.426465+0000","last_active":"2023-08-27T11:26:21.384498+0000","last_peered":"2023-08-27T11:26:21.384498+0000","last_clean":"2023-08-27T11:26:21.384498+0000","last_became_active":"2023-08-27T11:26:18.426180+0000","last_became_peered":"2023-08-27T11:26:18.426180+0000","last_unstale":"2023-08-27T11:26:21.384498+0000","last_undegraded":"2023-08-27T11:26:21.384498+0000","last_fullsized":"2023-08-27T11:26:21.384498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:46.357284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430463+0000","last_change":"2023-08-27T11:26:18.432789+0000","last_active":"2023-08-27T11:26:20.430463+0000","last_peered":"2023-08-27T11:26:20.430463+0000","last_clean":"2023-08-27T11:26:20.430463+0000","last_became_active":"2023-08-27T11:26:18.432521+0000","last_became_peered":"2023-08-27T11:26:18.432521+0000","last_unstale":"2023-08-27T11:26:20.430463+0000","last_undegraded":"2023-08-27T11:26:20.430463+0000","last_fullsized":"2023-08-27T11:26:20.430463+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:06.592671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384455+0000","last_change":"2023-08-27T11:26:18.426963+0000","last_active":"2023-08-27T11:26:21.384455+0000","last_peered":"2023-08-27T11:26:21.384455+0000","last_clean":"2023-08-27T11:26:21.384455+0000","last_became_active":"2023-08-27T11:26:18.426752+0000","last_became_peered":"2023-08-27T11:26:18.426752+0000","last_unstale":"2023-08-27T11:26:21.384455+0000","last_undegraded":"2023-08-27T11:26:21.384455+0000","last_fullsized":"2023-08-27T11:26:21.384455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:29:35.149141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425190+0000","last_change":"2023-08-27T11:26:19.431940+0000","last_active":"2023-08-27T11:26:20.425190+0000","last_peered":"2023-08-27T11:26:20.425190+0000","last_clean":"2023-08-27T11:26:20.425190+0000","last_became_active":"2023-08-27T11:26:19.431631+0000","last_became_peered":"2023-08-27T11:26:19.431631+0000","last_unstale":"2023-08-27T11:26:20.425190+0000","last_undegraded":"2023-08-27T11:26:20.425190+0000","last_fullsized":"2023-08-27T11:26:20.425190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:37.504080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430548+0000","last_change":"2023-08-27T11:26:18.425472+0000","last_active":"2023-08-27T11:26:20.430548+0000","last_peered":"2023-08-27T11:26:20.430548+0000","last_clean":"2023-08-27T11:26:20.430548+0000","last_became_active":"2023-08-27T11:26:18.425150+0000","last_became_peered":"2023-08-27T11:26:18.425150+0000","last_unstale":"2023-08-27T11:26:20.430548+0000","last_undegraded":"2023-08-27T11:26:20.430548+0000","last_fullsized":"2023-08-27T11:26:20.430548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:10.608143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431200+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:26:20.431200+0000","last_peered":"2023-08-27T11:26:20.431200+0000","last_clean":"2023-08-27T11:26:20.431200+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:26:20.431200+0000","last_undegraded":"2023-08-27T11:26:20.431200+0000","last_fullsized":"2023-08-27T11:26:20.431200+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:53.465982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384795+0000","last_change":"2023-08-27T11:26:19.426312+0000","last_active":"2023-08-27T11:26:21.384795+0000","last_peered":"2023-08-27T11:26:21.384795+0000","last_clean":"2023-08-27T11:26:21.384795+0000","last_became_active":"2023-08-27T11:26:19.425963+0000","last_became_peered":"2023-08-27T11:26:19.425963+0000","last_unstale":"2023-08-27T11:26:21.384795+0000","last_undegraded":"2023-08-27T11:26:21.384795+0000","last_fullsized":"2023-08-27T11:26:21.384795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:46.854781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384674+0000","last_change":"2023-08-27T11:26:18.429275+0000","last_active":"2023-08-27T11:26:21.384674+0000","last_peered":"2023-08-27T11:26:21.384674+0000","last_clean":"2023-08-27T11:26:21.384674+0000","last_became_active":"2023-08-27T11:26:18.428994+0000","last_became_peered":"2023-08-27T11:26:18.428994+0000","last_unstale":"2023-08-27T11:26:21.384674+0000","last_undegraded":"2023-08-27T11:26:21.384674+0000","last_fullsized":"2023-08-27T11:26:21.384674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:55.212494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430413+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:26:20.430413+0000","last_peered":"2023-08-27T11:26:20.430413+0000","last_clean":"2023-08-27T11:26:20.430413+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:26:20.430413+0000","last_undegraded":"2023-08-27T11:26:20.430413+0000","last_fullsized":"2023-08-27T11:26:20.430413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:35:03.658156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384543+0000","last_change":"2023-08-27T11:26:19.434349+0000","last_active":"2023-08-27T11:26:21.384543+0000","last_peered":"2023-08-27T11:26:21.384543+0000","last_clean":"2023-08-27T11:26:21.384543+0000","last_became_active":"2023-08-27T11:26:19.434063+0000","last_became_peered":"2023-08-27T11:26:19.434063+0000","last_unstale":"2023-08-27T11:26:21.384543+0000","last_undegraded":"2023-08-27T11:26:21.384543+0000","last_fullsized":"2023-08-27T11:26:21.384543+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:15.835557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431167+0000","last_change":"2023-08-27T11:26:18.429449+0000","last_active":"2023-08-27T11:26:20.431167+0000","last_peered":"2023-08-27T11:26:20.431167+0000","last_clean":"2023-08-27T11:26:20.431167+0000","last_became_active":"2023-08-27T11:26:18.429134+0000","last_became_peered":"2023-08-27T11:26:18.429134+0000","last_unstale":"2023-08-27T11:26:20.431167+0000","last_undegraded":"2023-08-27T11:26:20.431167+0000","last_fullsized":"2023-08-27T11:26:20.431167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:47:49.142361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:16.089904+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:26:16.089904+0000","last_peered":"2023-08-27T11:26:16.089904+0000","last_clean":"2023-08-27T11:26:16.089904+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:26:16.089904+0000","last_undegraded":"2023-08-27T11:26:16.089904+0000","last_fullsized":"2023-08-27T11:26:16.089904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:48:47.411691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384819+0000","last_change":"2023-08-27T11:26:18.429297+0000","last_active":"2023-08-27T11:26:21.384819+0000","last_peered":"2023-08-27T11:26:21.384819+0000","last_clean":"2023-08-27T11:26:21.384819+0000","last_became_active":"2023-08-27T11:26:18.429028+0000","last_became_peered":"2023-08-27T11:26:18.429028+0000","last_unstale":"2023-08-27T11:26:21.384819+0000","last_undegraded":"2023-08-27T11:26:21.384819+0000","last_fullsized":"2023-08-27T11:26:21.384819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:25:56.014027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":133,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:14.402961+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:26:14.402961+0000","last_peered":"2023-08-27T11:26:14.402961+0000","last_clean":"2023-08-27T11:26:14.402961+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:26:14.402961+0000","last_undegraded":"2023-08-27T11:26:14.402961+0000","last_fullsized":"2023-08-27T11:26:14.402961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:18:41.917017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057479+0000","last_change":"2023-08-27T11:26:19.432826+0000","last_active":"2023-08-27T11:26:21.057479+0000","last_peered":"2023-08-27T11:26:21.057479+0000","last_clean":"2023-08-27T11:26:21.057479+0000","last_became_active":"2023-08-27T11:26:19.432562+0000","last_became_peered":"2023-08-27T11:26:19.432562+0000","last_unstale":"2023-08-27T11:26:21.057479+0000","last_undegraded":"2023-08-27T11:26:21.057479+0000","last_fullsized":"2023-08-27T11:26:21.057479+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:05.457562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":133,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:14.402955+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:26:14.402955+0000","last_peered":"2023-08-27T11:26:14.402955+0000","last_clean":"2023-08-27T11:26:14.402955+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:26:14.402955+0000","last_undegraded":"2023-08-27T11:26:14.402955+0000","last_fullsized":"2023-08-27T11:26:14.402955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:59:51.839342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430815+0000","last_change":"2023-08-27T11:26:19.427376+0000","last_active":"2023-08-27T11:26:20.430815+0000","last_peered":"2023-08-27T11:26:20.430815+0000","last_clean":"2023-08-27T11:26:20.430815+0000","last_became_active":"2023-08-27T11:26:19.426214+0000","last_became_peered":"2023-08-27T11:26:19.426214+0000","last_unstale":"2023-08-27T11:26:20.430815+0000","last_undegraded":"2023-08-27T11:26:20.430815+0000","last_fullsized":"2023-08-27T11:26:20.430815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:07:17.226611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":283,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:14.403153+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:26:14.403153+0000","last_peered":"2023-08-27T11:26:14.403153+0000","last_clean":"2023-08-27T11:26:14.403153+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:26:14.403153+0000","last_undegraded":"2023-08-27T11:26:14.403153+0000","last_fullsized":"2023-08-27T11:26:14.403153+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:26:15.453703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057579+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:26:21.057579+0000","last_peered":"2023-08-27T11:26:21.057579+0000","last_clean":"2023-08-27T11:26:21.057579+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:26:21.057579+0000","last_undegraded":"2023-08-27T11:26:21.057579+0000","last_fullsized":"2023-08-27T11:26:21.057579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:12:51.448320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874474+0000","last_change":"2023-08-27T11:26:19.420539+0000","last_active":"2023-08-27T11:26:20.874474+0000","last_peered":"2023-08-27T11:26:20.874474+0000","last_clean":"2023-08-27T11:26:20.874474+0000","last_became_active":"2023-08-27T11:26:19.420291+0000","last_became_peered":"2023-08-27T11:26:19.420291+0000","last_unstale":"2023-08-27T11:26:20.874474+0000","last_undegraded":"2023-08-27T11:26:20.874474+0000","last_fullsized":"2023-08-27T11:26:20.874474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:23:25.892210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384424+0000","last_change":"2023-08-27T11:26:18.432770+0000","last_active":"2023-08-27T11:26:21.384424+0000","last_peered":"2023-08-27T11:26:21.384424+0000","last_clean":"2023-08-27T11:26:21.384424+0000","last_became_active":"2023-08-27T11:26:18.432552+0000","last_became_peered":"2023-08-27T11:26:18.432552+0000","last_unstale":"2023-08-27T11:26:21.384424+0000","last_undegraded":"2023-08-27T11:26:21.384424+0000","last_fullsized":"2023-08-27T11:26:21.384424+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:05:38.418817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":139,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425350+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:26:20.425350+0000","last_peered":"2023-08-27T11:26:20.425350+0000","last_clean":"2023-08-27T11:26:20.425350+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:26:20.425350+0000","last_undegraded":"2023-08-27T11:26:20.425350+0000","last_fullsized":"2023-08-27T11:26:20.425350+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:38:27.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874280+0000","last_change":"2023-08-27T11:26:19.427286+0000","last_active":"2023-08-27T11:26:20.874280+0000","last_peered":"2023-08-27T11:26:20.874280+0000","last_clean":"2023-08-27T11:26:20.874280+0000","last_became_active":"2023-08-27T11:26:19.426940+0000","last_became_peered":"2023-08-27T11:26:19.426940+0000","last_unstale":"2023-08-27T11:26:20.874280+0000","last_undegraded":"2023-08-27T11:26:20.874280+0000","last_fullsized":"2023-08-27T11:26:20.874280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:31.231234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384817+0000","last_change":"2023-08-27T11:26:18.415885+0000","last_active":"2023-08-27T11:26:21.384817+0000","last_peered":"2023-08-27T11:26:21.384817+0000","last_clean":"2023-08-27T11:26:21.384817+0000","last_became_active":"2023-08-27T11:26:18.415623+0000","last_became_peered":"2023-08-27T11:26:18.415623+0000","last_unstale":"2023-08-27T11:26:21.384817+0000","last_undegraded":"2023-08-27T11:26:21.384817+0000","last_fullsized":"2023-08-27T11:26:21.384817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:10:07.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":145,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:16.089811+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:26:16.089811+0000","last_peered":"2023-08-27T11:26:16.089811+0000","last_clean":"2023-08-27T11:26:16.089811+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:26:16.089811+0000","last_undegraded":"2023-08-27T11:26:16.089811+0000","last_fullsized":"2023-08-27T11:26:16.089811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:05:17.706426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2250,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":44,"acting":44,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":5},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1658880,"data_stored":1076579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.33300000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.55000000000000004,"15min":0.50800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":0.77500000000000002,"15min":0.872},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.253},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.39000000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.48699999999999999},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.33800000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.90200000000000002},"last":0.53700000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.56799999999999995}]},{"osd":4,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.505},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.72699999999999998,"5min":0.72799999999999998,"15min":0.77400000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.497}]},{"osd":5,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.52400000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574966,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1134592,"data_stored":515006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51500000000000001,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.29099999999999998},"max":{"1min":0.626,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67300000000000004,"5min":0.86199999999999999,"15min":0.89800000000000002},"last":0.51100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.316,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.80600000000000005,"15min":0.873},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.62,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61299999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58299999999999996,"5min":0.74199999999999999,"15min":0.85899999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.65200000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.58199999999999996,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.70699999999999996,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.65200000000000002,"5min":0.84699999999999998,"15min":0.93799999999999994},"last":0.63400000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.70699999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.374},"max":{"1min":0.77800000000000002,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.72599999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.624,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.78100000000000003,"5min":0.875,"15min":0.875},"last":0.74199999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.748,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.34899999999999998},"max":{"1min":0.76700000000000002,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.77500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1712128,"data_stored":1103192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.308,"15min":0.59299999999999997},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.45800000000000002,"5min":0.502,"15min":11.971},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.89000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.51700000000000002,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.26800000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.625},"min":{"1min":0.23400000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.495,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.34300000000000003},"min":{"1min":0.246,"5min":0.224,"15min":0.224},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.58599999999999997},"last":0.28799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.64100000000000001},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":10.99},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.373,"15min":0.92500000000000004},"min":{"1min":0.318,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.54700000000000004,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.36099999999999999,"15min":0.33400000000000002},"max":{"1min":0.76900000000000002,"5min":0.90700000000000003,"15min":1.26},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.80700000000000005},"min":{"1min":0.434,"5min":0.37,"15min":0.29099999999999998},"max":{"1min":0.64500000000000002,"5min":0.77600000000000002,"15min":10.957000000000001},"last":0.61299999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":1.121},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.36199999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71499999999999997,"5min":0.88700000000000001,"15min":1.296},"last":0.624}]},{"osd":6,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.84299999999999997},"min":{"1min":0.44800000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":1.1379999999999999},"min":{"1min":0.435,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.74299999999999999,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.53900000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":1.1319999999999999},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84199999999999997,"15min":22.988},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.63500000000000001},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":1.327},"last":0.59899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":513131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.29699999999999999,"15min":0.311},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.33200000000000002,"15min":0.34999999999999998},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.27000000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.24399999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.628,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.33200000000000002,"15min":0.33900000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.374}]},{"osd":3,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.33700000000000002,"15min":0.34599999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.502,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.27500000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.54200000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.41499999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79100000000000004,"5min":0.92500000000000004,"15min":1.214},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.84299999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.77500000000000002,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85399999999999998,"5min":0.90200000000000002,"15min":1.603},"last":0.628}]},{"osd":6,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.83099999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.74099999999999999,"5min":0.94199999999999995,"15min":1.383},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82699999999999996,"5min":0.88900000000000001,"15min":0.92000000000000004},"last":0.78900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1118208,"data_stored":512824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29799999999999999,"15min":0.30199999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40500000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.314,"5min":0.316,"15min":0.317},"min":{"1min":0.22700000000000001,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.629},"last":0.26100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.317,"15min":0.32400000000000001},"min":{"1min":0.222,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.36499999999999999,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33800000000000002,"15min":0.34599999999999997},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47799999999999998,"5min":0.64200000000000002,"15min":0.749},"last":0.28299999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36099999999999999,"15min":0.38200000000000001},"min":{"1min":0.318,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41599999999999998,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.313},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.376,"15min":0.39800000000000002},"min":{"1min":0.30599999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.69499999999999995},"last":0.374}]},{"osd":4,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.68600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.433,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.76100000000000001,"5min":0.78400000000000003,"15min":0.879},"last":0.40799999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70699999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.40999999999999998,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.73399999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.495}]},{"osd":6,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77200000000000002,"5min":0.82899999999999996,"15min":0.85299999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.59399999999999997},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.65900000000000003,"5min":0.80500000000000005,"15min":0.83599999999999997},"last":0.54400000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.60899999999999999},"min":{"1min":0.438,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574966,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1118208,"data_stored":513372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29699999999999999,"15min":0.29999999999999999},"min":{"1min":0.21299999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.373},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32100000000000001,"15min":0.312},"min":{"1min":0.23100000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.38400000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.29899999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32300000000000001,"15min":0.313},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.19500000000000001},"max":{"1min":0.46700000000000003,"5min":0.50600000000000001,"15min":0.60399999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":0.44900000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"last":0.32100000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.28499999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.53300000000000003,"5min":0.57199999999999995,"15min":0.57599999999999996},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.52500000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.35399999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.28599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.503}]},{"osd":5,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.371,"15min":0.314},"max":{"1min":0.79100000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57899999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.373},"max":{"1min":0.80400000000000005,"5min":1.091,"15min":1.091},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.61199999999999999},"min":{"1min":0.41699999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78400000000000003,"5min":1.111,"15min":1.111},"last":0.624}]},{"osd":7,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.61199999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74399999999999999,"5min":1.032,"15min":1.236},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.61299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1064960,"data_stored":486211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.755,"5min":0.997,"15min":0.997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.68700000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53300000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.752,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.46300000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.51000000000000001},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.69199999999999995}]},{"osd":3,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50600000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.85899999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75700000000000001,"5min":0.85999999999999999,"15min":0.879},"last":0.67900000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70599999999999996,"5min":0.78900000000000003,"15min":0.90600000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59599999999999997,"15min":0.623},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.92300000000000004},"last":0.65200000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.80600000000000005,"15min":0.82499999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.89300000000000002},"last":0.66200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1064960,"data_stored":486211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.63300000000000001,"5min":0.78600000000000003,"15min":0.96899999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.29199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.60899999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70499999999999996,"5min":0.747,"15min":0.92200000000000004},"last":0.623},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52400000000000002,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.63200000000000001,"5min":0.72899999999999998,"15min":1.075},"last":0.56999999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.78000000000000003,"15min":0.95699999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54600000000000004,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.65300000000000002,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.51300000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72299999999999998,"5min":0.81000000000000005,"15min":1.03},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58199999999999996,"15min":0.59499999999999997},"min":{"1min":0.48499999999999999,"5min":0.36699999999999999,"15min":0.35199999999999998},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":1.026},"last":0.55300000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72999999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.495},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.315,"15min":0.26900000000000002},"max":{"1min":0.78900000000000003,"5min":0.86399999999999999,"15min":0.91400000000000003},"last":0.40600000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.76500000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.48399999999999999,"5min":0.36599999999999999,"15min":0.28799999999999998},"max":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.94199999999999995},"last":0.433}]},{"osd":7,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.51300000000000001,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.95899999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.61199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.83699999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:26:23.895 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:26:24.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.049+0000 7f2c6c20d700 1 Processor -- start 2023-08-27T11:26:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.049+0000 7f2c6c20d700 1 -- start start 2023-08-27T11:26:24.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.050+0000 7f2c6c20d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2c6415ae80 0x7f2c640b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.050+0000 7f2c6c20d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c640d0560 0x7f2c640d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.050+0000 7f2c6c20d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c640d0ef0 con 0x7f2c640ca740 2023-08-27T11:26:24.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.050+0000 7f2c6c20d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c640b0c50 con 0x7f2c6415ae80 2023-08-27T11:26:24.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.050+0000 7f2c6c20d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640cf2a0 con 0x7f2c640d0560 2023-08-27T11:26:24.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.050+0000 7f2c6a7aa700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c640d0560 0x7f2c640d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.050+0000 7f2c69fa9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2c6415ae80 0x7f2c640b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.050+0000 7f2c69fa9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2c6415ae80 0x7f2c640b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:33352/0 (socket says 172.21.15.112:33352) 2023-08-27T11:26:24.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.050+0000 7f2c69fa9700 1 -- 172.21.15.112:0/2040368555 learned_addr learned my addr 172.21.15.112:0/2040368555 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:24.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.051+0000 7f2c69fa9700 1 -- 172.21.15.112:0/2040368555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c640d0560 msgr2=0x7f2c640d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:24.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.051+0000 7f2c69fa9700 1 --2- 172.21.15.112:0/2040368555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c640d0560 0x7f2c640d09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.051+0000 7f2c69fa9700 1 -- 172.21.15.112:0/2040368555 >> v1:172.21.15.138:6789/0 conn(0x7f2c640ca740 legacy=0x7f2c640708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.051+0000 7f2c69fa9700 1 -- 172.21.15.112:0/2040368555 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c640cf580 con 0x7f2c6415ae80 2023-08-27T11:26:24.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.051+0000 7f2c6a7aa700 1 --2- 172.21.15.112:0/2040368555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c640d0560 0x7f2c640d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:24.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.051+0000 7f2c69fa9700 1 --2- 172.21.15.112:0/2040368555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c6415ae80 0x7f2c640b0710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2c540098f0 tx=0x7f2c54009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63be1b109ccef2d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:24.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.051+0000 7f2c68fa7700 1 -- 172.21.15.112:0/2040368555 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54011830 con 0x7f2c6415ae80 2023-08-27T11:26:24.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.051+0000 7f2c68fa7700 1 -- 172.21.15.112:0/2040368555 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c54011990 con 0x7f2c6415ae80 2023-08-27T11:26:24.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.051+0000 7f2c68fa7700 1 -- 172.21.15.112:0/2040368555 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54011d20 con 0x7f2c6415ae80 2023-08-27T11:26:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.052+0000 7f2c6c20d700 1 -- 172.21.15.112:0/2040368555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c6415ae80 msgr2=0x7f2c640b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.052+0000 7f2c6c20d700 1 --2- 172.21.15.112:0/2040368555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c6415ae80 0x7f2c640b0710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2c540098f0 tx=0x7f2c54009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.052+0000 7f2c6c20d700 1 -- 172.21.15.112:0/2040368555 shutdown_connections 2023-08-27T11:26:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.052+0000 7f2c6c20d700 1 --2- 172.21.15.112:0/2040368555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c640d0560 0x7f2c640d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.052+0000 7f2c6c20d700 1 --2- 172.21.15.112:0/2040368555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c6415ae80 0x7f2c640b0710 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.052+0000 7f2c6c20d700 1 -- 172.21.15.112:0/2040368555 >> 172.21.15.112:0/2040368555 conn(0x7f2c6406f970 msgr2=0x7f2c6406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:24.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.052+0000 7f2c6c20d700 1 -- 172.21.15.112:0/2040368555 shutdown_connections 2023-08-27T11:26:24.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.052+0000 7f2c6c20d700 1 -- 172.21.15.112:0/2040368555 wait complete. 2023-08-27T11:26:24.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.053+0000 7f2c6c20d700 1 Processor -- start 2023-08-27T11:26:24.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.053+0000 7f2c6c20d700 1 -- start start 2023-08-27T11:26:24.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c6c20d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c640ca740 0x7f2c64158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c6c20d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c640d0560 0x7f2c640cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c6c20d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c6415ae80 0x7f2c640cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c6c20d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c64144d00 con 0x7f2c640d0560 2023-08-27T11:26:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c6c20d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640b53d0 con 0x7f2c640ca740 2023-08-27T11:26:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c6c20d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c64144360 con 0x7f2c6415ae80 2023-08-27T11:26:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c697a8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c640d0560 0x7f2c640cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c697a8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c640d0560 0x7f2c640cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32932/0 (socket says 172.21.15.112:32932) 2023-08-27T11:26:24.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c697a8700 1 -- 172.21.15.112:0/280404266 learned_addr learned my addr 172.21.15.112:0/280404266 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:24.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c6a7aa700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c6415ae80 0x7f2c640cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c69fa9700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c640ca740 0x7f2c64158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c697a8700 1 -- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c6415ae80 msgr2=0x7f2c640cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c697a8700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c6415ae80 0x7f2c640cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c697a8700 1 -- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c640ca740 msgr2=0x7f2c64158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c697a8700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c640ca740 0x7f2c64158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c697a8700 1 -- 172.21.15.112:0/280404266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c009a70 con 0x7f2c640d0560 2023-08-27T11:26:24.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c69fa9700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c640ca740 0x7f2c64158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:24.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.054+0000 7f2c697a8700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c640d0560 0x7f2c640cc400 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000c4e0 tx=0x7f2c6000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:24.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.055+0000 7f2c5a7fc700 1 -- 172.21.15.112:0/280404266 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c600052d0 con 0x7f2c640d0560 2023-08-27T11:26:24.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.055+0000 7f2c5a7fc700 1 -- 172.21.15.112:0/280404266 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c60005430 con 0x7f2c640d0560 2023-08-27T11:26:24.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.055+0000 7f2c5a7fc700 1 -- 172.21.15.112:0/280404266 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c600038c0 con 0x7f2c640d0560 2023-08-27T11:26:24.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.055+0000 7f2c6c20d700 1 -- 172.21.15.112:0/280404266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c54009550 con 0x7f2c640d0560 2023-08-27T11:26:24.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.055+0000 7f2c6c20d700 1 -- 172.21.15.112:0/280404266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c64144920 con 0x7f2c640d0560 2023-08-27T11:26:24.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.055+0000 7f2c6c20d700 1 -- 172.21.15.112:0/280404266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c64144e40 con 0x7f2c640d0560 2023-08-27T11:26:24.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.056+0000 7f2c5a7fc700 1 -- 172.21.15.112:0/280404266 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f2c60003a20 con 0x7f2c640d0560 2023-08-27T11:26:24.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.056+0000 7f2c5a7fc700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c4806a3b0 0x7f2c4806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.056+0000 7f2c5a7fc700 1 -- 172.21.15.112:0/280404266 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2c60087d70 con 0x7f2c640d0560 2023-08-27T11:26:24.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.057+0000 7f2c69fa9700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c4806a3b0 0x7f2c4806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.057+0000 7f2c69fa9700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c4806a3b0 0x7f2c4806c870 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2c54014040 tx=0x7f2c54008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:24.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.061+0000 7f2c5a7fc700 1 -- 172.21.15.112:0/280404266 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2c64144e40 con 0x7f2c640d0560 2023-08-27T11:26:24.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.245+0000 7f2c6c20d700 1 -- 172.21.15.112:0/280404266 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c640cfd60 con 0x7f2c4806a3b0 2023-08-27T11:26:24.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.254+0000 7f2c5a7fc700 1 -- 172.21.15.112:0/280404266 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307418 (secure 0 0 0) 0x7f2c54011c00 con 0x7f2c4806a3b0 2023-08-27T11:26:24.259 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:24.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.257+0000 7f2c6c20d700 1 -- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c4806a3b0 msgr2=0x7f2c4806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.257+0000 7f2c6c20d700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c4806a3b0 0x7f2c4806c870 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2c54014040 tx=0x7f2c54008030 comp rx=0 tx=0).stop 2023-08-27T11:26:24.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.257+0000 7f2c6c20d700 1 -- 172.21.15.112:0/280404266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c640d0560 msgr2=0x7f2c640cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.257+0000 7f2c6c20d700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c640d0560 0x7f2c640cc400 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000c4e0 tx=0x7f2c6000c8a0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.257+0000 7f2c6c20d700 1 -- 172.21.15.112:0/280404266 shutdown_connections 2023-08-27T11:26:24.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.257+0000 7f2c6c20d700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c4806a3b0 0x7f2c4806c870 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.257+0000 7f2c6c20d700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c6415ae80 0x7f2c640cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.257+0000 7f2c6c20d700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c640ca740 0x7f2c64158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.257+0000 7f2c6c20d700 1 --2- 172.21.15.112:0/280404266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c640d0560 0x7f2c640cc400 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.257+0000 7f2c6c20d700 1 -- 172.21.15.112:0/280404266 >> 172.21.15.112:0/280404266 conn(0x7f2c6406f970 msgr2=0x7f2c64155c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:24.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.257+0000 7f2c6c20d700 1 -- 172.21.15.112:0/280404266 shutdown_connections 2023-08-27T11:26:24.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.257+0000 7f2c6c20d700 1 -- 172.21.15.112:0/280404266 wait complete. 2023-08-27T11:26:24.266 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:26:24.271 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2023-08-27T11:26:22.368926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592618,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":183,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":154,"acting":154,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259232,"kb_used_data":9764,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754715488,"statfs":{"total":773094113280,"available":772828659712,"internally_reserved":0,"allocated":9998336,"data_stored":5206526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2250,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.970088"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874566+0000","last_change":"2023-08-27T11:26:19.427207+0000","last_active":"2023-08-27T11:26:20.874566+0000","last_peered":"2023-08-27T11:26:20.874566+0000","last_clean":"2023-08-27T11:26:20.874566+0000","last_became_active":"2023-08-27T11:26:19.426963+0000","last_became_peered":"2023-08-27T11:26:19.426963+0000","last_unstale":"2023-08-27T11:26:20.874566+0000","last_undegraded":"2023-08-27T11:26:20.874566+0000","last_fullsized":"2023-08-27T11:26:20.874566+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:55:21.733240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430931+0000","last_change":"2023-08-27T11:26:19.431066+0000","last_active":"2023-08-27T11:26:20.430931+0000","last_peered":"2023-08-27T11:26:20.430931+0000","last_clean":"2023-08-27T11:26:20.430931+0000","last_became_active":"2023-08-27T11:26:19.430477+0000","last_became_peered":"2023-08-27T11:26:19.430477+0000","last_unstale":"2023-08-27T11:26:20.430931+0000","last_undegraded":"2023-08-27T11:26:20.430931+0000","last_fullsized":"2023-08-27T11:26:20.430931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:43:44.337358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874230+0000","last_change":"2023-08-27T11:26:19.434887+0000","last_active":"2023-08-27T11:26:20.874230+0000","last_peered":"2023-08-27T11:26:20.874230+0000","last_clean":"2023-08-27T11:26:20.874230+0000","last_became_active":"2023-08-27T11:26:19.434566+0000","last_became_peered":"2023-08-27T11:26:19.434566+0000","last_unstale":"2023-08-27T11:26:20.874230+0000","last_undegraded":"2023-08-27T11:26:20.874230+0000","last_fullsized":"2023-08-27T11:26:20.874230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:11:11.738619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425453+0000","last_change":"2023-08-27T11:26:19.426786+0000","last_active":"2023-08-27T11:26:20.425453+0000","last_peered":"2023-08-27T11:26:20.425453+0000","last_clean":"2023-08-27T11:26:20.425453+0000","last_became_active":"2023-08-27T11:26:19.426494+0000","last_became_peered":"2023-08-27T11:26:19.426494+0000","last_unstale":"2023-08-27T11:26:20.425453+0000","last_undegraded":"2023-08-27T11:26:20.425453+0000","last_fullsized":"2023-08-27T11:26:20.425453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:33:53.249169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057365+0000","last_change":"2023-08-27T11:26:19.424084+0000","last_active":"2023-08-27T11:26:21.057365+0000","last_peered":"2023-08-27T11:26:21.057365+0000","last_clean":"2023-08-27T11:26:21.057365+0000","last_became_active":"2023-08-27T11:26:19.422955+0000","last_became_peered":"2023-08-27T11:26:19.422955+0000","last_unstale":"2023-08-27T11:26:21.057365+0000","last_undegraded":"2023-08-27T11:26:21.057365+0000","last_fullsized":"2023-08-27T11:26:21.057365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:59:24.896402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430830+0000","last_change":"2023-08-27T11:26:19.424659+0000","last_active":"2023-08-27T11:26:20.430830+0000","last_peered":"2023-08-27T11:26:20.430830+0000","last_clean":"2023-08-27T11:26:20.430830+0000","last_became_active":"2023-08-27T11:26:19.424317+0000","last_became_peered":"2023-08-27T11:26:19.424317+0000","last_unstale":"2023-08-27T11:26:20.430830+0000","last_undegraded":"2023-08-27T11:26:20.430830+0000","last_fullsized":"2023-08-27T11:26:20.430830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:47:42.857298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430038+0000","last_change":"2023-08-27T11:26:19.421667+0000","last_active":"2023-08-27T11:26:20.430038+0000","last_peered":"2023-08-27T11:26:20.430038+0000","last_clean":"2023-08-27T11:26:20.430038+0000","last_became_active":"2023-08-27T11:26:19.421352+0000","last_became_peered":"2023-08-27T11:26:19.421352+0000","last_unstale":"2023-08-27T11:26:20.430038+0000","last_undegraded":"2023-08-27T11:26:20.430038+0000","last_fullsized":"2023-08-27T11:26:20.430038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:24.356051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384297+0000","last_change":"2023-08-27T11:26:19.430958+0000","last_active":"2023-08-27T11:26:21.384297+0000","last_peered":"2023-08-27T11:26:21.384297+0000","last_clean":"2023-08-27T11:26:21.384297+0000","last_became_active":"2023-08-27T11:26:19.430685+0000","last_became_peered":"2023-08-27T11:26:19.430685+0000","last_unstale":"2023-08-27T11:26:21.384297+0000","last_undegraded":"2023-08-27T11:26:21.384297+0000","last_fullsized":"2023-08-27T11:26:21.384297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:41.805565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431121+0000","last_change":"2023-08-27T11:26:19.430211+0000","last_active":"2023-08-27T11:26:20.431121+0000","last_peered":"2023-08-27T11:26:20.431121+0000","last_clean":"2023-08-27T11:26:20.431121+0000","last_became_active":"2023-08-27T11:26:19.429921+0000","last_became_peered":"2023-08-27T11:26:19.429921+0000","last_unstale":"2023-08-27T11:26:20.431121+0000","last_undegraded":"2023-08-27T11:26:20.431121+0000","last_fullsized":"2023-08-27T11:26:20.431121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:56:34.526955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057756+0000","last_change":"2023-08-27T11:26:19.428983+0000","last_active":"2023-08-27T11:26:21.057756+0000","last_peered":"2023-08-27T11:26:21.057756+0000","last_clean":"2023-08-27T11:26:21.057756+0000","last_became_active":"2023-08-27T11:26:19.428699+0000","last_became_peered":"2023-08-27T11:26:19.428699+0000","last_unstale":"2023-08-27T11:26:21.057756+0000","last_undegraded":"2023-08-27T11:26:21.057756+0000","last_fullsized":"2023-08-27T11:26:21.057756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:49:32.779304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425155+0000","last_change":"2023-08-27T11:26:19.436540+0000","last_active":"2023-08-27T11:26:20.425155+0000","last_peered":"2023-08-27T11:26:20.425155+0000","last_clean":"2023-08-27T11:26:20.425155+0000","last_became_active":"2023-08-27T11:26:19.436278+0000","last_became_peered":"2023-08-27T11:26:19.436278+0000","last_unstale":"2023-08-27T11:26:20.425155+0000","last_undegraded":"2023-08-27T11:26:20.425155+0000","last_fullsized":"2023-08-27T11:26:20.425155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:40:05.207374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874577+0000","last_change":"2023-08-27T11:26:19.434923+0000","last_active":"2023-08-27T11:26:20.874577+0000","last_peered":"2023-08-27T11:26:20.874577+0000","last_clean":"2023-08-27T11:26:20.874577+0000","last_became_active":"2023-08-27T11:26:19.434650+0000","last_became_peered":"2023-08-27T11:26:19.434650+0000","last_unstale":"2023-08-27T11:26:20.874577+0000","last_undegraded":"2023-08-27T11:26:20.874577+0000","last_fullsized":"2023-08-27T11:26:20.874577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:33:06.493994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430846+0000","last_change":"2023-08-27T11:26:19.422300+0000","last_active":"2023-08-27T11:26:20.430846+0000","last_peered":"2023-08-27T11:26:20.430846+0000","last_clean":"2023-08-27T11:26:20.430846+0000","last_became_active":"2023-08-27T11:26:19.421284+0000","last_became_peered":"2023-08-27T11:26:19.421284+0000","last_unstale":"2023-08-27T11:26:20.430846+0000","last_undegraded":"2023-08-27T11:26:20.430846+0000","last_fullsized":"2023-08-27T11:26:20.430846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:12.363650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874425+0000","last_change":"2023-08-27T11:26:19.418470+0000","last_active":"2023-08-27T11:26:20.874425+0000","last_peered":"2023-08-27T11:26:20.874425+0000","last_clean":"2023-08-27T11:26:20.874425+0000","last_became_active":"2023-08-27T11:26:19.418192+0000","last_became_peered":"2023-08-27T11:26:19.418192+0000","last_unstale":"2023-08-27T11:26:20.874425+0000","last_undegraded":"2023-08-27T11:26:20.874425+0000","last_fullsized":"2023-08-27T11:26:20.874425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:14:53.959765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384767+0000","last_change":"2023-08-27T11:26:19.433627+0000","last_active":"2023-08-27T11:26:21.384767+0000","last_peered":"2023-08-27T11:26:21.384767+0000","last_clean":"2023-08-27T11:26:21.384767+0000","last_became_active":"2023-08-27T11:26:19.433370+0000","last_became_peered":"2023-08-27T11:26:19.433370+0000","last_unstale":"2023-08-27T11:26:21.384767+0000","last_undegraded":"2023-08-27T11:26:21.384767+0000","last_fullsized":"2023-08-27T11:26:21.384767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:41.696204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057432+0000","last_change":"2023-08-27T11:26:19.433519+0000","last_active":"2023-08-27T11:26:21.057432+0000","last_peered":"2023-08-27T11:26:21.057432+0000","last_clean":"2023-08-27T11:26:21.057432+0000","last_became_active":"2023-08-27T11:26:19.433245+0000","last_became_peered":"2023-08-27T11:26:19.433245+0000","last_unstale":"2023-08-27T11:26:21.057432+0000","last_undegraded":"2023-08-27T11:26:21.057432+0000","last_fullsized":"2023-08-27T11:26:21.057432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:07:00.066813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874575+0000","last_change":"2023-08-27T11:26:19.434762+0000","last_active":"2023-08-27T11:26:20.874575+0000","last_peered":"2023-08-27T11:26:20.874575+0000","last_clean":"2023-08-27T11:26:20.874575+0000","last_became_active":"2023-08-27T11:26:19.434462+0000","last_became_peered":"2023-08-27T11:26:19.434462+0000","last_unstale":"2023-08-27T11:26:20.874575+0000","last_undegraded":"2023-08-27T11:26:20.874575+0000","last_fullsized":"2023-08-27T11:26:20.874575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:41.356424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057290+0000","last_change":"2023-08-27T11:26:19.423833+0000","last_active":"2023-08-27T11:26:21.057290+0000","last_peered":"2023-08-27T11:26:21.057290+0000","last_clean":"2023-08-27T11:26:21.057290+0000","last_became_active":"2023-08-27T11:26:19.423440+0000","last_became_peered":"2023-08-27T11:26:19.423440+0000","last_unstale":"2023-08-27T11:26:21.057290+0000","last_undegraded":"2023-08-27T11:26:21.057290+0000","last_fullsized":"2023-08-27T11:26:21.057290+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:41:13.840925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430610+0000","last_change":"2023-08-27T11:26:19.433762+0000","last_active":"2023-08-27T11:26:20.430610+0000","last_peered":"2023-08-27T11:26:20.430610+0000","last_clean":"2023-08-27T11:26:20.430610+0000","last_became_active":"2023-08-27T11:26:19.433489+0000","last_became_peered":"2023-08-27T11:26:19.433489+0000","last_unstale":"2023-08-27T11:26:20.430610+0000","last_undegraded":"2023-08-27T11:26:20.430610+0000","last_fullsized":"2023-08-27T11:26:20.430610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:34.644697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384668+0000","last_change":"2023-08-27T11:26:19.431428+0000","last_active":"2023-08-27T11:26:21.384668+0000","last_peered":"2023-08-27T11:26:21.384668+0000","last_clean":"2023-08-27T11:26:21.384668+0000","last_became_active":"2023-08-27T11:26:19.431169+0000","last_became_peered":"2023-08-27T11:26:19.431169+0000","last_unstale":"2023-08-27T11:26:21.384668+0000","last_undegraded":"2023-08-27T11:26:21.384668+0000","last_fullsized":"2023-08-27T11:26:21.384668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:30:35.082113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430321+0000","last_change":"2023-08-27T11:26:19.427535+0000","last_active":"2023-08-27T11:26:20.430321+0000","last_peered":"2023-08-27T11:26:20.430321+0000","last_clean":"2023-08-27T11:26:20.430321+0000","last_became_active":"2023-08-27T11:26:19.426191+0000","last_became_peered":"2023-08-27T11:26:19.426191+0000","last_unstale":"2023-08-27T11:26:20.430321+0000","last_undegraded":"2023-08-27T11:26:20.430321+0000","last_fullsized":"2023-08-27T11:26:20.430321+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:22:47.314202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384374+0000","last_change":"2023-08-27T11:26:19.424642+0000","last_active":"2023-08-27T11:26:21.384374+0000","last_peered":"2023-08-27T11:26:21.384374+0000","last_clean":"2023-08-27T11:26:21.384374+0000","last_became_active":"2023-08-27T11:26:19.424359+0000","last_became_peered":"2023-08-27T11:26:19.424359+0000","last_unstale":"2023-08-27T11:26:21.384374+0000","last_undegraded":"2023-08-27T11:26:21.384374+0000","last_fullsized":"2023-08-27T11:26:21.384374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:06:36.252165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874126+0000","last_change":"2023-08-27T11:26:19.414983+0000","last_active":"2023-08-27T11:26:20.874126+0000","last_peered":"2023-08-27T11:26:20.874126+0000","last_clean":"2023-08-27T11:26:20.874126+0000","last_became_active":"2023-08-27T11:26:19.414726+0000","last_became_peered":"2023-08-27T11:26:19.414726+0000","last_unstale":"2023-08-27T11:26:20.874126+0000","last_undegraded":"2023-08-27T11:26:20.874126+0000","last_fullsized":"2023-08-27T11:26:20.874126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:57:45.405543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"89'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425328+0000","last_change":"2023-08-27T11:26:18.432220+0000","last_active":"2023-08-27T11:26:20.425328+0000","last_peered":"2023-08-27T11:26:20.425328+0000","last_clean":"2023-08-27T11:26:20.425328+0000","last_became_active":"2023-08-27T11:26:18.431938+0000","last_became_peered":"2023-08-27T11:26:18.431938+0000","last_unstale":"2023-08-27T11:26:20.425328+0000","last_undegraded":"2023-08-27T11:26:20.425328+0000","last_fullsized":"2023-08-27T11:26:20.425328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:26:24.683316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384352+0000","last_change":"2023-08-27T11:26:19.427981+0000","last_active":"2023-08-27T11:26:21.384352+0000","last_peered":"2023-08-27T11:26:21.384352+0000","last_clean":"2023-08-27T11:26:21.384352+0000","last_became_active":"2023-08-27T11:26:19.427694+0000","last_became_peered":"2023-08-27T11:26:19.427694+0000","last_unstale":"2023-08-27T11:26:21.384352+0000","last_undegraded":"2023-08-27T11:26:21.384352+0000","last_fullsized":"2023-08-27T11:26:21.384352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:31.571236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384746+0000","last_change":"2023-08-27T11:26:18.416729+0000","last_active":"2023-08-27T11:26:21.384746+0000","last_peered":"2023-08-27T11:26:21.384746+0000","last_clean":"2023-08-27T11:26:21.384746+0000","last_became_active":"2023-08-27T11:26:18.416320+0000","last_became_peered":"2023-08-27T11:26:18.416320+0000","last_unstale":"2023-08-27T11:26:21.384746+0000","last_undegraded":"2023-08-27T11:26:21.384746+0000","last_fullsized":"2023-08-27T11:26:21.384746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:32.457209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057704+0000","last_change":"2023-08-27T11:26:18.426839+0000","last_active":"2023-08-27T11:26:21.057704+0000","last_peered":"2023-08-27T11:26:21.057704+0000","last_clean":"2023-08-27T11:26:21.057704+0000","last_became_active":"2023-08-27T11:26:18.426578+0000","last_became_peered":"2023-08-27T11:26:18.426578+0000","last_unstale":"2023-08-27T11:26:21.057704+0000","last_undegraded":"2023-08-27T11:26:21.057704+0000","last_fullsized":"2023-08-27T11:26:21.057704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:23.764319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425272+0000","last_change":"2023-08-27T11:26:19.426762+0000","last_active":"2023-08-27T11:26:20.425272+0000","last_peered":"2023-08-27T11:26:20.425272+0000","last_clean":"2023-08-27T11:26:20.425272+0000","last_became_active":"2023-08-27T11:26:19.426146+0000","last_became_peered":"2023-08-27T11:26:19.426146+0000","last_unstale":"2023-08-27T11:26:20.425272+0000","last_undegraded":"2023-08-27T11:26:20.425272+0000","last_fullsized":"2023-08-27T11:26:20.425272+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:54:50.371981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874343+0000","last_change":"2023-08-27T11:26:19.418506+0000","last_active":"2023-08-27T11:26:20.874343+0000","last_peered":"2023-08-27T11:26:20.874343+0000","last_clean":"2023-08-27T11:26:20.874343+0000","last_became_active":"2023-08-27T11:26:19.418235+0000","last_became_peered":"2023-08-27T11:26:19.418235+0000","last_unstale":"2023-08-27T11:26:20.874343+0000","last_undegraded":"2023-08-27T11:26:20.874343+0000","last_fullsized":"2023-08-27T11:26:20.874343+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:50.719209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430641+0000","last_change":"2023-08-27T11:26:19.431258+0000","last_active":"2023-08-27T11:26:20.430641+0000","last_peered":"2023-08-27T11:26:20.430641+0000","last_clean":"2023-08-27T11:26:20.430641+0000","last_became_active":"2023-08-27T11:26:19.430816+0000","last_became_peered":"2023-08-27T11:26:19.430816+0000","last_unstale":"2023-08-27T11:26:20.430641+0000","last_undegraded":"2023-08-27T11:26:20.430641+0000","last_fullsized":"2023-08-27T11:26:20.430641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:55.969633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425484+0000","last_change":"2023-08-27T11:26:19.426307+0000","last_active":"2023-08-27T11:26:20.425484+0000","last_peered":"2023-08-27T11:26:20.425484+0000","last_clean":"2023-08-27T11:26:20.425484+0000","last_became_active":"2023-08-27T11:26:19.425959+0000","last_became_peered":"2023-08-27T11:26:19.425959+0000","last_unstale":"2023-08-27T11:26:20.425484+0000","last_undegraded":"2023-08-27T11:26:20.425484+0000","last_fullsized":"2023-08-27T11:26:20.425484+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:10:02.440599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425034+0000","last_change":"2023-08-27T11:26:18.429003+0000","last_active":"2023-08-27T11:26:20.425034+0000","last_peered":"2023-08-27T11:26:20.425034+0000","last_clean":"2023-08-27T11:26:20.425034+0000","last_became_active":"2023-08-27T11:26:18.428701+0000","last_became_peered":"2023-08-27T11:26:18.428701+0000","last_unstale":"2023-08-27T11:26:20.425034+0000","last_undegraded":"2023-08-27T11:26:20.425034+0000","last_fullsized":"2023-08-27T11:26:20.425034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:42:33.783460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874118+0000","last_change":"2023-08-27T11:26:19.417293+0000","last_active":"2023-08-27T11:26:20.874118+0000","last_peered":"2023-08-27T11:26:20.874118+0000","last_clean":"2023-08-27T11:26:20.874118+0000","last_became_active":"2023-08-27T11:26:19.417018+0000","last_became_peered":"2023-08-27T11:26:19.417018+0000","last_unstale":"2023-08-27T11:26:20.874118+0000","last_undegraded":"2023-08-27T11:26:20.874118+0000","last_fullsized":"2023-08-27T11:26:20.874118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:02:56.092248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057401+0000","last_change":"2023-08-27T11:26:19.425413+0000","last_active":"2023-08-27T11:26:21.057401+0000","last_peered":"2023-08-27T11:26:21.057401+0000","last_clean":"2023-08-27T11:26:21.057401+0000","last_became_active":"2023-08-27T11:26:19.424482+0000","last_became_peered":"2023-08-27T11:26:19.424482+0000","last_unstale":"2023-08-27T11:26:21.057401+0000","last_undegraded":"2023-08-27T11:26:21.057401+0000","last_fullsized":"2023-08-27T11:26:21.057401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:36.587246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874328+0000","last_change":"2023-08-27T11:26:18.426356+0000","last_active":"2023-08-27T11:26:20.874328+0000","last_peered":"2023-08-27T11:26:20.874328+0000","last_clean":"2023-08-27T11:26:20.874328+0000","last_became_active":"2023-08-27T11:26:18.426041+0000","last_became_peered":"2023-08-27T11:26:18.426041+0000","last_unstale":"2023-08-27T11:26:20.874328+0000","last_undegraded":"2023-08-27T11:26:20.874328+0000","last_fullsized":"2023-08-27T11:26:20.874328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:28.381950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384393+0000","last_change":"2023-08-27T11:26:19.431679+0000","last_active":"2023-08-27T11:26:21.384393+0000","last_peered":"2023-08-27T11:26:21.384393+0000","last_clean":"2023-08-27T11:26:21.384393+0000","last_became_active":"2023-08-27T11:26:19.431405+0000","last_became_peered":"2023-08-27T11:26:19.431405+0000","last_unstale":"2023-08-27T11:26:21.384393+0000","last_undegraded":"2023-08-27T11:26:21.384393+0000","last_fullsized":"2023-08-27T11:26:21.384393+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:02:21.450786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057480+0000","last_change":"2023-08-27T11:26:19.424582+0000","last_active":"2023-08-27T11:26:21.057480+0000","last_peered":"2023-08-27T11:26:21.057480+0000","last_clean":"2023-08-27T11:26:21.057480+0000","last_became_active":"2023-08-27T11:26:19.424288+0000","last_became_peered":"2023-08-27T11:26:19.424288+0000","last_unstale":"2023-08-27T11:26:21.057480+0000","last_undegraded":"2023-08-27T11:26:21.057480+0000","last_fullsized":"2023-08-27T11:26:21.057480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:16.279109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430741+0000","last_change":"2023-08-27T11:26:18.421802+0000","last_active":"2023-08-27T11:26:20.430741+0000","last_peered":"2023-08-27T11:26:20.430741+0000","last_clean":"2023-08-27T11:26:20.430741+0000","last_became_active":"2023-08-27T11:26:18.421093+0000","last_became_peered":"2023-08-27T11:26:18.421093+0000","last_unstale":"2023-08-27T11:26:20.430741+0000","last_undegraded":"2023-08-27T11:26:20.430741+0000","last_fullsized":"2023-08-27T11:26:20.430741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:29.408679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057739+0000","last_change":"2023-08-27T11:26:18.421876+0000","last_active":"2023-08-27T11:26:21.057739+0000","last_peered":"2023-08-27T11:26:21.057739+0000","last_clean":"2023-08-27T11:26:21.057739+0000","last_became_active":"2023-08-27T11:26:18.421583+0000","last_became_peered":"2023-08-27T11:26:18.421583+0000","last_unstale":"2023-08-27T11:26:21.057739+0000","last_undegraded":"2023-08-27T11:26:21.057739+0000","last_fullsized":"2023-08-27T11:26:21.057739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:48:43.946799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430199+0000","last_change":"2023-08-27T11:26:19.430893+0000","last_active":"2023-08-27T11:26:20.430199+0000","last_peered":"2023-08-27T11:26:20.430199+0000","last_clean":"2023-08-27T11:26:20.430199+0000","last_became_active":"2023-08-27T11:26:19.430225+0000","last_became_peered":"2023-08-27T11:26:19.430225+0000","last_unstale":"2023-08-27T11:26:20.430199+0000","last_undegraded":"2023-08-27T11:26:20.430199+0000","last_fullsized":"2023-08-27T11:26:20.430199+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:43.123979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425475+0000","last_change":"2023-08-27T11:26:18.432084+0000","last_active":"2023-08-27T11:26:20.425475+0000","last_peered":"2023-08-27T11:26:20.425475+0000","last_clean":"2023-08-27T11:26:20.425475+0000","last_became_active":"2023-08-27T11:26:18.431877+0000","last_became_peered":"2023-08-27T11:26:18.431877+0000","last_unstale":"2023-08-27T11:26:20.425475+0000","last_undegraded":"2023-08-27T11:26:20.425475+0000","last_fullsized":"2023-08-27T11:26:20.425475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:12:10.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384677+0000","last_change":"2023-08-27T11:26:19.429400+0000","last_active":"2023-08-27T11:26:21.384677+0000","last_peered":"2023-08-27T11:26:21.384677+0000","last_clean":"2023-08-27T11:26:21.384677+0000","last_became_active":"2023-08-27T11:26:19.429146+0000","last_became_peered":"2023-08-27T11:26:19.429146+0000","last_unstale":"2023-08-27T11:26:21.384677+0000","last_undegraded":"2023-08-27T11:26:21.384677+0000","last_fullsized":"2023-08-27T11:26:21.384677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:06.185951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384453+0000","last_change":"2023-08-27T11:26:18.420633+0000","last_active":"2023-08-27T11:26:21.384453+0000","last_peered":"2023-08-27T11:26:21.384453+0000","last_clean":"2023-08-27T11:26:21.384453+0000","last_became_active":"2023-08-27T11:26:18.420312+0000","last_became_peered":"2023-08-27T11:26:18.420312+0000","last_unstale":"2023-08-27T11:26:21.384453+0000","last_undegraded":"2023-08-27T11:26:21.384453+0000","last_fullsized":"2023-08-27T11:26:21.384453+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:20.872678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874639+0000","last_change":"2023-08-27T11:26:19.434950+0000","last_active":"2023-08-27T11:26:20.874639+0000","last_peered":"2023-08-27T11:26:20.874639+0000","last_clean":"2023-08-27T11:26:20.874639+0000","last_became_active":"2023-08-27T11:26:19.434684+0000","last_became_peered":"2023-08-27T11:26:19.434684+0000","last_unstale":"2023-08-27T11:26:20.874639+0000","last_undegraded":"2023-08-27T11:26:20.874639+0000","last_fullsized":"2023-08-27T11:26:20.874639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:52:42.875755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874262+0000","last_change":"2023-08-27T11:26:19.425141+0000","last_active":"2023-08-27T11:26:20.874262+0000","last_peered":"2023-08-27T11:26:20.874262+0000","last_clean":"2023-08-27T11:26:20.874262+0000","last_became_active":"2023-08-27T11:26:19.424669+0000","last_became_peered":"2023-08-27T11:26:19.424669+0000","last_unstale":"2023-08-27T11:26:20.874262+0000","last_undegraded":"2023-08-27T11:26:20.874262+0000","last_fullsized":"2023-08-27T11:26:20.874262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:31:34.706537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057687+0000","last_change":"2023-08-27T11:26:18.421572+0000","last_active":"2023-08-27T11:26:21.057687+0000","last_peered":"2023-08-27T11:26:21.057687+0000","last_clean":"2023-08-27T11:26:21.057687+0000","last_became_active":"2023-08-27T11:26:18.421268+0000","last_became_peered":"2023-08-27T11:26:18.421268+0000","last_unstale":"2023-08-27T11:26:21.057687+0000","last_undegraded":"2023-08-27T11:26:21.057687+0000","last_fullsized":"2023-08-27T11:26:21.057687+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:44.389523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874451+0000","last_change":"2023-08-27T11:26:18.429524+0000","last_active":"2023-08-27T11:26:20.874451+0000","last_peered":"2023-08-27T11:26:20.874451+0000","last_clean":"2023-08-27T11:26:20.874451+0000","last_became_active":"2023-08-27T11:26:18.429224+0000","last_became_peered":"2023-08-27T11:26:18.429224+0000","last_unstale":"2023-08-27T11:26:20.874451+0000","last_undegraded":"2023-08-27T11:26:20.874451+0000","last_fullsized":"2023-08-27T11:26:20.874451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:13.776898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425320+0000","last_change":"2023-08-27T11:26:19.427555+0000","last_active":"2023-08-27T11:26:20.425320+0000","last_peered":"2023-08-27T11:26:20.425320+0000","last_clean":"2023-08-27T11:26:20.425320+0000","last_became_active":"2023-08-27T11:26:19.427295+0000","last_became_peered":"2023-08-27T11:26:19.427295+0000","last_unstale":"2023-08-27T11:26:20.425320+0000","last_undegraded":"2023-08-27T11:26:20.425320+0000","last_fullsized":"2023-08-27T11:26:20.425320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:35.698617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430342+0000","last_change":"2023-08-27T11:26:18.432922+0000","last_active":"2023-08-27T11:26:20.430342+0000","last_peered":"2023-08-27T11:26:20.430342+0000","last_clean":"2023-08-27T11:26:20.430342+0000","last_became_active":"2023-08-27T11:26:18.432631+0000","last_became_peered":"2023-08-27T11:26:18.432631+0000","last_unstale":"2023-08-27T11:26:20.430342+0000","last_undegraded":"2023-08-27T11:26:20.430342+0000","last_fullsized":"2023-08-27T11:26:20.430342+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:58.703884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057644+0000","last_change":"2023-08-27T11:26:18.422687+0000","last_active":"2023-08-27T11:26:21.057644+0000","last_peered":"2023-08-27T11:26:21.057644+0000","last_clean":"2023-08-27T11:26:21.057644+0000","last_became_active":"2023-08-27T11:26:18.422404+0000","last_became_peered":"2023-08-27T11:26:18.422404+0000","last_unstale":"2023-08-27T11:26:21.057644+0000","last_undegraded":"2023-08-27T11:26:21.057644+0000","last_fullsized":"2023-08-27T11:26:21.057644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:34.961579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384633+0000","last_change":"2023-08-27T11:26:19.425732+0000","last_active":"2023-08-27T11:26:21.384633+0000","last_peered":"2023-08-27T11:26:21.384633+0000","last_clean":"2023-08-27T11:26:21.384633+0000","last_became_active":"2023-08-27T11:26:19.425452+0000","last_became_peered":"2023-08-27T11:26:19.425452+0000","last_unstale":"2023-08-27T11:26:21.384633+0000","last_undegraded":"2023-08-27T11:26:21.384633+0000","last_fullsized":"2023-08-27T11:26:21.384633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:15.049461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384761+0000","last_change":"2023-08-27T11:26:19.425658+0000","last_active":"2023-08-27T11:26:21.384761+0000","last_peered":"2023-08-27T11:26:21.384761+0000","last_clean":"2023-08-27T11:26:21.384761+0000","last_became_active":"2023-08-27T11:26:19.425387+0000","last_became_peered":"2023-08-27T11:26:19.425387+0000","last_unstale":"2023-08-27T11:26:21.384761+0000","last_undegraded":"2023-08-27T11:26:21.384761+0000","last_fullsized":"2023-08-27T11:26:21.384761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:52.205981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384498+0000","last_change":"2023-08-27T11:26:18.426465+0000","last_active":"2023-08-27T11:26:21.384498+0000","last_peered":"2023-08-27T11:26:21.384498+0000","last_clean":"2023-08-27T11:26:21.384498+0000","last_became_active":"2023-08-27T11:26:18.426180+0000","last_became_peered":"2023-08-27T11:26:18.426180+0000","last_unstale":"2023-08-27T11:26:21.384498+0000","last_undegraded":"2023-08-27T11:26:21.384498+0000","last_fullsized":"2023-08-27T11:26:21.384498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:46.357284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430463+0000","last_change":"2023-08-27T11:26:18.432789+0000","last_active":"2023-08-27T11:26:20.430463+0000","last_peered":"2023-08-27T11:26:20.430463+0000","last_clean":"2023-08-27T11:26:20.430463+0000","last_became_active":"2023-08-27T11:26:18.432521+0000","last_became_peered":"2023-08-27T11:26:18.432521+0000","last_unstale":"2023-08-27T11:26:20.430463+0000","last_undegraded":"2023-08-27T11:26:20.430463+0000","last_fullsized":"2023-08-27T11:26:20.430463+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:06.592671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384455+0000","last_change":"2023-08-27T11:26:18.426963+0000","last_active":"2023-08-27T11:26:21.384455+0000","last_peered":"2023-08-27T11:26:21.384455+0000","last_clean":"2023-08-27T11:26:21.384455+0000","last_became_active":"2023-08-27T11:26:18.426752+0000","last_became_peered":"2023-08-27T11:26:18.426752+0000","last_unstale":"2023-08-27T11:26:21.384455+0000","last_undegraded":"2023-08-27T11:26:21.384455+0000","last_fullsized":"2023-08-27T11:26:21.384455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:29:35.149141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425190+0000","last_change":"2023-08-27T11:26:19.431940+0000","last_active":"2023-08-27T11:26:20.425190+0000","last_peered":"2023-08-27T11:26:20.425190+0000","last_clean":"2023-08-27T11:26:20.425190+0000","last_became_active":"2023-08-27T11:26:19.431631+0000","last_became_peered":"2023-08-27T11:26:19.431631+0000","last_unstale":"2023-08-27T11:26:20.425190+0000","last_undegraded":"2023-08-27T11:26:20.425190+0000","last_fullsized":"2023-08-27T11:26:20.425190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:37.504080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430548+0000","last_change":"2023-08-27T11:26:18.425472+0000","last_active":"2023-08-27T11:26:20.430548+0000","last_peered":"2023-08-27T11:26:20.430548+0000","last_clean":"2023-08-27T11:26:20.430548+0000","last_became_active":"2023-08-27T11:26:18.425150+0000","last_became_peered":"2023-08-27T11:26:18.425150+0000","last_unstale":"2023-08-27T11:26:20.430548+0000","last_undegraded":"2023-08-27T11:26:20.430548+0000","last_fullsized":"2023-08-27T11:26:20.430548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:10.608143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431200+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:26:20.431200+0000","last_peered":"2023-08-27T11:26:20.431200+0000","last_clean":"2023-08-27T11:26:20.431200+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:26:20.431200+0000","last_undegraded":"2023-08-27T11:26:20.431200+0000","last_fullsized":"2023-08-27T11:26:20.431200+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:53.465982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384795+0000","last_change":"2023-08-27T11:26:19.426312+0000","last_active":"2023-08-27T11:26:21.384795+0000","last_peered":"2023-08-27T11:26:21.384795+0000","last_clean":"2023-08-27T11:26:21.384795+0000","last_became_active":"2023-08-27T11:26:19.425963+0000","last_became_peered":"2023-08-27T11:26:19.425963+0000","last_unstale":"2023-08-27T11:26:21.384795+0000","last_undegraded":"2023-08-27T11:26:21.384795+0000","last_fullsized":"2023-08-27T11:26:21.384795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:46.854781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384674+0000","last_change":"2023-08-27T11:26:18.429275+0000","last_active":"2023-08-27T11:26:21.384674+0000","last_peered":"2023-08-27T11:26:21.384674+0000","last_clean":"2023-08-27T11:26:21.384674+0000","last_became_active":"2023-08-27T11:26:18.428994+0000","last_became_peered":"2023-08-27T11:26:18.428994+0000","last_unstale":"2023-08-27T11:26:21.384674+0000","last_undegraded":"2023-08-27T11:26:21.384674+0000","last_fullsized":"2023-08-27T11:26:21.384674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:55.212494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430413+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:26:20.430413+0000","last_peered":"2023-08-27T11:26:20.430413+0000","last_clean":"2023-08-27T11:26:20.430413+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:26:20.430413+0000","last_undegraded":"2023-08-27T11:26:20.430413+0000","last_fullsized":"2023-08-27T11:26:20.430413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:35:03.658156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384543+0000","last_change":"2023-08-27T11:26:19.434349+0000","last_active":"2023-08-27T11:26:21.384543+0000","last_peered":"2023-08-27T11:26:21.384543+0000","last_clean":"2023-08-27T11:26:21.384543+0000","last_became_active":"2023-08-27T11:26:19.434063+0000","last_became_peered":"2023-08-27T11:26:19.434063+0000","last_unstale":"2023-08-27T11:26:21.384543+0000","last_undegraded":"2023-08-27T11:26:21.384543+0000","last_fullsized":"2023-08-27T11:26:21.384543+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:15.835557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431167+0000","last_change":"2023-08-27T11:26:18.429449+0000","last_active":"2023-08-27T11:26:20.431167+0000","last_peered":"2023-08-27T11:26:20.431167+0000","last_clean":"2023-08-27T11:26:20.431167+0000","last_became_active":"2023-08-27T11:26:18.429134+0000","last_became_peered":"2023-08-27T11:26:18.429134+0000","last_unstale":"2023-08-27T11:26:20.431167+0000","last_undegraded":"2023-08-27T11:26:20.431167+0000","last_fullsized":"2023-08-27T11:26:20.431167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:47:49.142361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":133,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:16.089904+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:26:16.089904+0000","last_peered":"2023-08-27T11:26:16.089904+0000","last_clean":"2023-08-27T11:26:16.089904+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:26:16.089904+0000","last_undegraded":"2023-08-27T11:26:16.089904+0000","last_fullsized":"2023-08-27T11:26:16.089904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:48:47.411691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:18.398253+0000","last_change":"2023-08-27T11:26:18.398253+0000","last_active":"2023-08-27T11:26:18.398253+0000","last_peered":"2023-08-27T11:26:18.398253+0000","last_clean":"2023-08-27T11:26:18.398253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:18.398253+0000","last_undegraded":"2023-08-27T11:26:18.398253+0000","last_fullsized":"2023-08-27T11:26:18.398253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384819+0000","last_change":"2023-08-27T11:26:18.429297+0000","last_active":"2023-08-27T11:26:21.384819+0000","last_peered":"2023-08-27T11:26:21.384819+0000","last_clean":"2023-08-27T11:26:21.384819+0000","last_became_active":"2023-08-27T11:26:18.429028+0000","last_became_peered":"2023-08-27T11:26:18.429028+0000","last_unstale":"2023-08-27T11:26:21.384819+0000","last_undegraded":"2023-08-27T11:26:21.384819+0000","last_fullsized":"2023-08-27T11:26:21.384819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:25:56.014027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":133,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:14.402961+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:26:14.402961+0000","last_peered":"2023-08-27T11:26:14.402961+0000","last_clean":"2023-08-27T11:26:14.402961+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:26:14.402961+0000","last_undegraded":"2023-08-27T11:26:14.402961+0000","last_fullsized":"2023-08-27T11:26:14.402961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:18:41.917017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057479+0000","last_change":"2023-08-27T11:26:19.432826+0000","last_active":"2023-08-27T11:26:21.057479+0000","last_peered":"2023-08-27T11:26:21.057479+0000","last_clean":"2023-08-27T11:26:21.057479+0000","last_became_active":"2023-08-27T11:26:19.432562+0000","last_became_peered":"2023-08-27T11:26:19.432562+0000","last_unstale":"2023-08-27T11:26:21.057479+0000","last_undegraded":"2023-08-27T11:26:21.057479+0000","last_fullsized":"2023-08-27T11:26:21.057479+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:05.457562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":133,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:14.402955+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:26:14.402955+0000","last_peered":"2023-08-27T11:26:14.402955+0000","last_clean":"2023-08-27T11:26:14.402955+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:26:14.402955+0000","last_undegraded":"2023-08-27T11:26:14.402955+0000","last_fullsized":"2023-08-27T11:26:14.402955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:59:51.839342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430815+0000","last_change":"2023-08-27T11:26:19.427376+0000","last_active":"2023-08-27T11:26:20.430815+0000","last_peered":"2023-08-27T11:26:20.430815+0000","last_clean":"2023-08-27T11:26:20.430815+0000","last_became_active":"2023-08-27T11:26:19.426214+0000","last_became_peered":"2023-08-27T11:26:19.426214+0000","last_unstale":"2023-08-27T11:26:20.430815+0000","last_undegraded":"2023-08-27T11:26:20.430815+0000","last_fullsized":"2023-08-27T11:26:20.430815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:07:17.226611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:26:17.389463+0000","last_change":"2023-08-27T11:26:17.389463+0000","last_active":"2023-08-27T11:26:17.389463+0000","last_peered":"2023-08-27T11:26:17.389463+0000","last_clean":"2023-08-27T11:26:17.389463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:26:17.389463+0000","last_undegraded":"2023-08-27T11:26:17.389463+0000","last_fullsized":"2023-08-27T11:26:17.389463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":283,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:14.403153+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:26:14.403153+0000","last_peered":"2023-08-27T11:26:14.403153+0000","last_clean":"2023-08-27T11:26:14.403153+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:26:14.403153+0000","last_undegraded":"2023-08-27T11:26:14.403153+0000","last_fullsized":"2023-08-27T11:26:14.403153+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:26:15.453703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057579+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:26:21.057579+0000","last_peered":"2023-08-27T11:26:21.057579+0000","last_clean":"2023-08-27T11:26:21.057579+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:26:21.057579+0000","last_undegraded":"2023-08-27T11:26:21.057579+0000","last_fullsized":"2023-08-27T11:26:21.057579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:12:51.448320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874474+0000","last_change":"2023-08-27T11:26:19.420539+0000","last_active":"2023-08-27T11:26:20.874474+0000","last_peered":"2023-08-27T11:26:20.874474+0000","last_clean":"2023-08-27T11:26:20.874474+0000","last_became_active":"2023-08-27T11:26:19.420291+0000","last_became_peered":"2023-08-27T11:26:19.420291+0000","last_unstale":"2023-08-27T11:26:20.874474+0000","last_undegraded":"2023-08-27T11:26:20.874474+0000","last_fullsized":"2023-08-27T11:26:20.874474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:23:25.892210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384424+0000","last_change":"2023-08-27T11:26:18.432770+0000","last_active":"2023-08-27T11:26:21.384424+0000","last_peered":"2023-08-27T11:26:21.384424+0000","last_clean":"2023-08-27T11:26:21.384424+0000","last_became_active":"2023-08-27T11:26:18.432552+0000","last_became_peered":"2023-08-27T11:26:18.432552+0000","last_unstale":"2023-08-27T11:26:21.384424+0000","last_undegraded":"2023-08-27T11:26:21.384424+0000","last_fullsized":"2023-08-27T11:26:21.384424+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:05:38.418817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":139,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425350+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:26:20.425350+0000","last_peered":"2023-08-27T11:26:20.425350+0000","last_clean":"2023-08-27T11:26:20.425350+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:26:20.425350+0000","last_undegraded":"2023-08-27T11:26:20.425350+0000","last_fullsized":"2023-08-27T11:26:20.425350+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:38:27.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874280+0000","last_change":"2023-08-27T11:26:19.427286+0000","last_active":"2023-08-27T11:26:20.874280+0000","last_peered":"2023-08-27T11:26:20.874280+0000","last_clean":"2023-08-27T11:26:20.874280+0000","last_became_active":"2023-08-27T11:26:19.426940+0000","last_became_peered":"2023-08-27T11:26:19.426940+0000","last_unstale":"2023-08-27T11:26:20.874280+0000","last_undegraded":"2023-08-27T11:26:20.874280+0000","last_fullsized":"2023-08-27T11:26:20.874280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:31.231234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384817+0000","last_change":"2023-08-27T11:26:18.415885+0000","last_active":"2023-08-27T11:26:21.384817+0000","last_peered":"2023-08-27T11:26:21.384817+0000","last_clean":"2023-08-27T11:26:21.384817+0000","last_became_active":"2023-08-27T11:26:18.415623+0000","last_became_peered":"2023-08-27T11:26:18.415623+0000","last_unstale":"2023-08-27T11:26:21.384817+0000","last_undegraded":"2023-08-27T11:26:21.384817+0000","last_fullsized":"2023-08-27T11:26:21.384817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:10:07.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":145,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:26:16.089811+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:26:16.089811+0000","last_peered":"2023-08-27T11:26:16.089811+0000","last_clean":"2023-08-27T11:26:16.089811+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:26:16.089811+0000","last_undegraded":"2023-08-27T11:26:16.089811+0000","last_fullsized":"2023-08-27T11:26:16.089811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:05:17.706426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2250,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":44,"acting":44,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":5},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1658880,"data_stored":1076579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.33300000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.55000000000000004,"15min":0.50800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":0.77500000000000002,"15min":0.872},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.253},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.39000000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.48699999999999999},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.33800000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.90200000000000002},"last":0.53700000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.56799999999999995}]},{"osd":4,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.505},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.72699999999999998,"5min":0.72799999999999998,"15min":0.77400000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.497}]},{"osd":5,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.52400000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574966,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1134592,"data_stored":515006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51500000000000001,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.29099999999999998},"max":{"1min":0.626,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67300000000000004,"5min":0.86199999999999999,"15min":0.89800000000000002},"last":0.51100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.316,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.80600000000000005,"15min":0.873},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.62,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61299999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58299999999999996,"5min":0.74199999999999999,"15min":0.85899999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.65200000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.58199999999999996,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.70699999999999996,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.65200000000000002,"5min":0.84699999999999998,"15min":0.93799999999999994},"last":0.63400000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.70699999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.374},"max":{"1min":0.77800000000000002,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.72599999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.624,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.78100000000000003,"5min":0.875,"15min":0.875},"last":0.74199999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.748,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.34899999999999998},"max":{"1min":0.76700000000000002,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.77500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1712128,"data_stored":1103192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.308,"15min":0.59299999999999997},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.45800000000000002,"5min":0.502,"15min":11.971},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.89000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.51700000000000002,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.26800000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.625},"min":{"1min":0.23400000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.495,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.34300000000000003},"min":{"1min":0.246,"5min":0.224,"15min":0.224},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.58599999999999997},"last":0.28799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.64100000000000001},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":10.99},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.373,"15min":0.92500000000000004},"min":{"1min":0.318,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.54700000000000004,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.36099999999999999,"15min":0.33400000000000002},"max":{"1min":0.76900000000000002,"5min":0.90700000000000003,"15min":1.26},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.80700000000000005},"min":{"1min":0.434,"5min":0.37,"15min":0.29099999999999998},"max":{"1min":0.64500000000000002,"5min":0.77600000000000002,"15min":10.957000000000001},"last":0.61299999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":1.121},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.36199999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71499999999999997,"5min":0.88700000000000001,"15min":1.296},"last":0.624}]},{"osd":6,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.84299999999999997},"min":{"1min":0.44800000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":1.1379999999999999},"min":{"1min":0.435,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.74299999999999999,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.53900000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":1.1319999999999999},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84199999999999997,"15min":22.988},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.63500000000000001},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":1.327},"last":0.59899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":513131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.29699999999999999,"15min":0.311},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.33200000000000002,"15min":0.34999999999999998},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.27000000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.24399999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.628,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.33200000000000002,"15min":0.33900000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.374}]},{"osd":3,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.33700000000000002,"15min":0.34599999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.502,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.27500000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.54200000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.41499999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79100000000000004,"5min":0.92500000000000004,"15min":1.214},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.84299999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.77500000000000002,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85399999999999998,"5min":0.90200000000000002,"15min":1.603},"last":0.628}]},{"osd":6,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.83099999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.74099999999999999,"5min":0.94199999999999995,"15min":1.383},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82699999999999996,"5min":0.88900000000000001,"15min":0.92000000000000004},"last":0.78900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1118208,"data_stored":512824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29799999999999999,"15min":0.30199999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40500000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.314,"5min":0.316,"15min":0.317},"min":{"1min":0.22700000000000001,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.629},"last":0.26100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.317,"15min":0.32400000000000001},"min":{"1min":0.222,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.36499999999999999,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33800000000000002,"15min":0.34599999999999997},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47799999999999998,"5min":0.64200000000000002,"15min":0.749},"last":0.28299999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36099999999999999,"15min":0.38200000000000001},"min":{"1min":0.318,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41599999999999998,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.313},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.376,"15min":0.39800000000000002},"min":{"1min":0.30599999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.69499999999999995},"last":0.374}]},{"osd":4,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.68600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.433,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.76100000000000001,"5min":0.78400000000000003,"15min":0.879},"last":0.40799999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70699999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.40999999999999998,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.73399999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.495}]},{"osd":6,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77200000000000002,"5min":0.82899999999999996,"15min":0.85299999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.59399999999999997},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.65900000000000003,"5min":0.80500000000000005,"15min":0.83599999999999997},"last":0.54400000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.60899999999999999},"min":{"1min":0.438,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574966,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1118208,"data_stored":513372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29699999999999999,"15min":0.29999999999999999},"min":{"1min":0.21299999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.373},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32100000000000001,"15min":0.312},"min":{"1min":0.23100000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.38400000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.29899999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32300000000000001,"15min":0.313},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.19500000000000001},"max":{"1min":0.46700000000000003,"5min":0.50600000000000001,"15min":0.60399999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":0.44900000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"last":0.32100000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.28499999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.53300000000000003,"5min":0.57199999999999995,"15min":0.57599999999999996},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.52500000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.35399999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.28599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.503}]},{"osd":5,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.371,"15min":0.314},"max":{"1min":0.79100000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57899999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.373},"max":{"1min":0.80400000000000005,"5min":1.091,"15min":1.091},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.61199999999999999},"min":{"1min":0.41699999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78400000000000003,"5min":1.111,"15min":1.111},"last":0.624}]},{"osd":7,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.61199999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74399999999999999,"5min":1.032,"15min":1.236},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.61299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1064960,"data_stored":486211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.755,"5min":0.997,"15min":0.997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.68700000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53300000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.752,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.46300000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.51000000000000001},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.69199999999999995}]},{"osd":3,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50600000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.85899999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75700000000000001,"5min":0.85999999999999999,"15min":0.879},"last":0.67900000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70599999999999996,"5min":0.78900000000000003,"15min":0.90600000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59599999999999997,"15min":0.623},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.92300000000000004},"last":0.65200000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.80600000000000005,"15min":0.82499999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.89300000000000002},"last":0.66200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1064960,"data_stored":486211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.63300000000000001,"5min":0.78600000000000003,"15min":0.96899999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.29199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.60899999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70499999999999996,"5min":0.747,"15min":0.92200000000000004},"last":0.623},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52400000000000002,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.63200000000000001,"5min":0.72899999999999998,"15min":1.075},"last":0.56999999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.78000000000000003,"15min":0.95699999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54600000000000004,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.65300000000000002,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.51300000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72299999999999998,"5min":0.81000000000000005,"15min":1.03},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58199999999999996,"15min":0.59499999999999997},"min":{"1min":0.48499999999999999,"5min":0.36699999999999999,"15min":0.35199999999999998},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":1.026},"last":0.55300000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72999999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.495},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.315,"15min":0.26900000000000002},"max":{"1min":0.78900000000000003,"5min":0.86399999999999999,"15min":0.91400000000000003},"last":0.40600000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.59999999999999998},"min":{"1min":0.42699999999999999,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.76500000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.48399999999999999,"5min":0.36599999999999999,"15min":0.28799999999999998},"max":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.94199999999999995},"last":0.433}]},{"osd":7,"last update":"Sun Aug 27 11:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.51300000000000001,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.95899999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.61199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.83699999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:26:24.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.35 is not active+clean 2023-08-27T11:26:24.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2023-08-27T11:26:24.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.32 is not active+clean 2023-08-27T11:26:24.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.31 is not active+clean 2023-08-27T11:26:24.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.30 is not active+clean 2023-08-27T11:26:24.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2023-08-27T11:26:24.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2023-08-27T11:26:24.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2023-08-27T11:26:24.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.25 is not active+clean 2023-08-27T11:26:24.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2023-08-27T11:26:24.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2023-08-27T11:26:24.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2023-08-27T11:26:24.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:17.389463+0000', 'last_change': '2023-08-27T11:26:17.389463+0000', 'last_active': '2023-08-27T11:26:17.389463+0000', 'last_peered': '2023-08-27T11:26:17.389463+0000', 'last_clean': '2023-08-27T11:26:17.389463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:17.389463+0000', 'last_undegraded': '2023-08-27T11:26:17.389463+0000', 'last_fullsized': '2023-08-27T11:26:17.389463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2023-08-27T11:26:24.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2023-08-27T11:26:24.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:17.389463+0000', 'last_change': '2023-08-27T11:26:17.389463+0000', 'last_active': '2023-08-27T11:26:17.389463+0000', 'last_peered': '2023-08-27T11:26:17.389463+0000', 'last_clean': '2023-08-27T11:26:17.389463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:17.389463+0000', 'last_undegraded': '2023-08-27T11:26:17.389463+0000', 'last_fullsized': '2023-08-27T11:26:17.389463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2023-08-27T11:26:24.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:17.389463+0000', 'last_change': '2023-08-27T11:26:17.389463+0000', 'last_active': '2023-08-27T11:26:17.389463+0000', 'last_peered': '2023-08-27T11:26:17.389463+0000', 'last_clean': '2023-08-27T11:26:17.389463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:17.389463+0000', 'last_undegraded': '2023-08-27T11:26:17.389463+0000', 'last_fullsized': '2023-08-27T11:26:17.389463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2023-08-27T11:26:24.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:17.389463+0000', 'last_change': '2023-08-27T11:26:17.389463+0000', 'last_active': '2023-08-27T11:26:17.389463+0000', 'last_peered': '2023-08-27T11:26:17.389463+0000', 'last_clean': '2023-08-27T11:26:17.389463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:17.389463+0000', 'last_undegraded': '2023-08-27T11:26:17.389463+0000', 'last_fullsized': '2023-08-27T11:26:17.389463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2023-08-27T11:26:24.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:17.389463+0000', 'last_change': '2023-08-27T11:26:17.389463+0000', 'last_active': '2023-08-27T11:26:17.389463+0000', 'last_peered': '2023-08-27T11:26:17.389463+0000', 'last_clean': '2023-08-27T11:26:17.389463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:17.389463+0000', 'last_undegraded': '2023-08-27T11:26:17.389463+0000', 'last_fullsized': '2023-08-27T11:26:17.389463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2023-08-27T11:26:24.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:17.389463+0000', 'last_change': '2023-08-27T11:26:17.389463+0000', 'last_active': '2023-08-27T11:26:17.389463+0000', 'last_peered': '2023-08-27T11:26:17.389463+0000', 'last_clean': '2023-08-27T11:26:17.389463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:17.389463+0000', 'last_undegraded': '2023-08-27T11:26:17.389463+0000', 'last_fullsized': '2023-08-27T11:26:17.389463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2023-08-27T11:26:24.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2023-08-27T11:26:24.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:17.389463+0000', 'last_change': '2023-08-27T11:26:17.389463+0000', 'last_active': '2023-08-27T11:26:17.389463+0000', 'last_peered': '2023-08-27T11:26:17.389463+0000', 'last_clean': '2023-08-27T11:26:17.389463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:17.389463+0000', 'last_undegraded': '2023-08-27T11:26:17.389463+0000', 'last_fullsized': '2023-08-27T11:26:17.389463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2023-08-27T11:26:24.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2023-08-27T11:26:24.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2023-08-27T11:26:24.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:17.389463+0000', 'last_change': '2023-08-27T11:26:17.389463+0000', 'last_active': '2023-08-27T11:26:17.389463+0000', 'last_peered': '2023-08-27T11:26:17.389463+0000', 'last_clean': '2023-08-27T11:26:17.389463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:17.389463+0000', 'last_undegraded': '2023-08-27T11:26:17.389463+0000', 'last_fullsized': '2023-08-27T11:26:17.389463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1e is not active+clean 2023-08-27T11:26:24.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2023-08-27T11:26:24.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2023-08-27T11:26:24.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:18.398253+0000', 'last_change': '2023-08-27T11:26:18.398253+0000', 'last_active': '2023-08-27T11:26:18.398253+0000', 'last_peered': '2023-08-27T11:26:18.398253+0000', 'last_clean': '2023-08-27T11:26:18.398253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:18.398253+0000', 'last_undegraded': '2023-08-27T11:26:18.398253+0000', 'last_fullsized': '2023-08-27T11:26:18.398253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:18.398253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2023-08-27T11:26:24.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:17.389463+0000', 'last_change': '2023-08-27T11:26:17.389463+0000', 'last_active': '2023-08-27T11:26:17.389463+0000', 'last_peered': '2023-08-27T11:26:17.389463+0000', 'last_clean': '2023-08-27T11:26:17.389463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:17.389463+0000', 'last_undegraded': '2023-08-27T11:26:17.389463+0000', 'last_fullsized': '2023-08-27T11:26:17.389463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2023-08-27T11:26:24.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:26:17.389463+0000', 'last_change': '2023-08-27T11:26:17.389463+0000', 'last_active': '2023-08-27T11:26:17.389463+0000', 'last_peered': '2023-08-27T11:26:17.389463+0000', 'last_clean': '2023-08-27T11:26:17.389463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:26:17.389463+0000', 'last_undegraded': '2023-08-27T11:26:17.389463+0000', 'last_fullsized': '2023-08-27T11:26:17.389463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'last_clean_scrub_stamp': '2023-08-27T11:26:17.389463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:26:24.299 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-27T11:26:24.389 INFO:tasks.ceph.mds.c.smithi112.stderr:2023-08-27T11:26:24.388+0000 7f4c64695700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:26:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.430+0000 7f94fe61c700 1 Processor -- start 2023-08-27T11:26:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.430+0000 7f94fe61c700 1 -- start start 2023-08-27T11:26:24.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.430+0000 7f94fe61c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f94f80b1400 0x7f94f80cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.430+0000 7f94fe61c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f8161850 con 0x7f94f8161ce0 2023-08-27T11:26:24.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.430+0000 7f94fe61c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f80cfc10 con 0x7f94f80b17e0 2023-08-27T11:26:24.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.430+0000 7f94fe61c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f8161200 con 0x7f94f80b1400 2023-08-27T11:26:24.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.430+0000 7f94f7fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f94f80b1400 0x7f94f80cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.431+0000 7f94fcbb9700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f94f8161ce0 0x7f94f806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55556/0 (socket says 172.21.15.112:55556) 2023-08-27T11:26:24.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.431+0000 7f94f7fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f94f80b1400 0x7f94f80cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32940/0 (socket says 172.21.15.112:32940) 2023-08-27T11:26:24.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.431+0000 7f94fcbb9700 1 -- 172.21.15.112:0/2913781022 learned_addr learned my addr 172.21.15.112:0/2913781022 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:24.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.431+0000 7f94f7fff700 1 -- 172.21.15.112:0/2913781022 >> v1:172.21.15.138:6789/0 conn(0x7f94f80b17e0 legacy=0x7f94f806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.431+0000 7f94f7fff700 1 -- 172.21.15.112:0/2913781022 >> v1:172.21.15.112:6789/0 conn(0x7f94f8161ce0 legacy=0x7f94f806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.431+0000 7f94f7fff700 1 -- 172.21.15.112:0/2913781022 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f81614e0 con 0x7f94f80b1400 2023-08-27T11:26:24.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.431+0000 7f94f6ffd700 1 -- 172.21.15.112:0/2913781022 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418263553 0 0) 0x7f94f8161850 con 0x7f94f8161ce0 2023-08-27T11:26:24.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.431+0000 7f94f7fff700 1 --2- 172.21.15.112:0/2913781022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f80b1400 0x7f94f80cf5a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009840 tx=0x7f94e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6979054d6b906f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:24.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.431+0000 7f94f6ffd700 1 -- 172.21.15.112:0/2913781022 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8011830 con 0x7f94f80b1400 2023-08-27T11:26:24.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.431+0000 7f94f6ffd700 1 -- 172.21.15.112:0/2913781022 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e8011990 con 0x7f94f80b1400 2023-08-27T11:26:24.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.431+0000 7f94f6ffd700 1 -- 172.21.15.112:0/2913781022 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8011d20 con 0x7f94f80b1400 2023-08-27T11:26:24.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.432+0000 7f94fe61c700 1 -- 172.21.15.112:0/2913781022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f80b1400 msgr2=0x7f94f80cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.432+0000 7f94fe61c700 1 --2- 172.21.15.112:0/2913781022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f80b1400 0x7f94f80cf5a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009840 tx=0x7f94e8009b50 comp rx=0 tx=0).stop 2023-08-27T11:26:24.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.432+0000 7f94fe61c700 1 -- 172.21.15.112:0/2913781022 shutdown_connections 2023-08-27T11:26:24.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.432+0000 7f94fe61c700 1 --2- 172.21.15.112:0/2913781022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f80b1400 0x7f94f80cf5a0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.432+0000 7f94fe61c700 1 -- 172.21.15.112:0/2913781022 >> 172.21.15.112:0/2913781022 conn(0x7f94f80d0000 msgr2=0x7f94f80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:24.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.432+0000 7f94fe61c700 1 -- 172.21.15.112:0/2913781022 shutdown_connections 2023-08-27T11:26:24.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.432+0000 7f94fe61c700 1 -- 172.21.15.112:0/2913781022 wait complete. 2023-08-27T11:26:24.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.433+0000 7f94fe61c700 1 Processor -- start 2023-08-27T11:26:24.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.433+0000 7f94fe61c700 1 -- start start 2023-08-27T11:26:24.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94fe61c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94f80b1400 0x7f94f814f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94fe61c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f80b17e0 0x7f94f81f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94fe61c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f8161ce0 0x7f94f80c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94fe61c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f80c58d0 con 0x7f94f80b17e0 2023-08-27T11:26:24.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94fe61c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f80c4c50 con 0x7f94f8161ce0 2023-08-27T11:26:24.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94fe61c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f80c4f30 con 0x7f94f80b1400 2023-08-27T11:26:24.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94f77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f80b17e0 0x7f94f81f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94f77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f80b17e0 0x7f94f81f2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32948/0 (socket says 172.21.15.112:32948) 2023-08-27T11:26:24.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94f77fe700 1 -- 172.21.15.112:0/1336646775 learned_addr learned my addr 172.21.15.112:0/1336646775 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:24.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94fcbb9700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f8161ce0 0x7f94f80c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94f7fff700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94f80b1400 0x7f94f814f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94f77fe700 1 -- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94f80b1400 msgr2=0x7f94f814f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94f77fe700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94f80b1400 0x7f94f814f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94f77fe700 1 -- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f8161ce0 msgr2=0x7f94f80c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94f77fe700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f8161ce0 0x7f94f80c5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94f77fe700 1 -- 172.21.15.112:0/1336646775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e80094a0 con 0x7f94f80b17e0 2023-08-27T11:26:24.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94f77fe700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f80b17e0 0x7f94f81f2d40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f94e000c4b0 tx=0x7f94e000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:24.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.434+0000 7f94f4ff9700 1 -- 172.21.15.112:0/1336646775 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e00052d0 con 0x7f94f80b17e0 2023-08-27T11:26:24.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.435+0000 7f94f4ff9700 1 -- 172.21.15.112:0/1336646775 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e0005430 con 0x7f94f80b17e0 2023-08-27T11:26:24.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.435+0000 7f94fe61c700 1 -- 172.21.15.112:0/1336646775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f80c5210 con 0x7f94f80b17e0 2023-08-27T11:26:24.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.435+0000 7f94f4ff9700 1 -- 172.21.15.112:0/1336646775 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e00038c0 con 0x7f94f80b17e0 2023-08-27T11:26:24.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.435+0000 7f94fe61c700 1 -- 172.21.15.112:0/1336646775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f814f710 con 0x7f94f80b17e0 2023-08-27T11:26:24.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.435+0000 7f94fe61c700 1 -- 172.21.15.112:0/1336646775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f80c5a10 con 0x7f94f80b17e0 2023-08-27T11:26:24.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.436+0000 7f94f4ff9700 1 -- 172.21.15.112:0/1336646775 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f94e0003a20 con 0x7f94f80b17e0 2023-08-27T11:26:24.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.437+0000 7f94f4ff9700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f94d806a3b0 0x7f94d806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.437+0000 7f94f4ff9700 1 -- 172.21.15.112:0/1336646775 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f94e0087e80 con 0x7f94f80b17e0 2023-08-27T11:26:24.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.437+0000 7f94f7fff700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f94d806a3b0 0x7f94d806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.437+0000 7f94f7fff700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f94d806a3b0 0x7f94d806c870 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009b90 tx=0x7f94e801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:24.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.441+0000 7f94f4ff9700 1 -- 172.21.15.112:0/1336646775 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f94f80c5a10 con 0x7f94f80b17e0 2023-08-27T11:26:24.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.714+0000 7f94fe61c700 1 -- 172.21.15.112:0/1336646775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f94f80c5a10 con 0x7f94f80b17e0 2023-08-27T11:26:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.714+0000 7f94f4ff9700 1 -- 172.21.15.112:0/1336646775 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1433 (secure 0 0 0) 0x7f94f80c5a10 con 0x7f94f80b17e0 2023-08-27T11:26:24.716 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:24.717 INFO:teuthology.orchestra.run.smithi112.stdout:{"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":578,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1693135017,"num_in_osds":8,"osd_in_since":1693135009,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592778,"bytes_used":265613312,"bytes_avail":772828499968,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":58,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7041}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-08-27T11:26:06.365276+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-08-27T11:26:24.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.716+0000 7f94fe61c700 1 -- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f94d806a3b0 msgr2=0x7f94d806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.716+0000 7f94fe61c700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f94d806a3b0 0x7f94d806c870 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009b90 tx=0x7f94e801c040 comp rx=0 tx=0).stop 2023-08-27T11:26:24.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.716+0000 7f94fe61c700 1 -- 172.21.15.112:0/1336646775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f80b17e0 msgr2=0x7f94f81f2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.716+0000 7f94fe61c700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f80b17e0 0x7f94f81f2d40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f94e000c4b0 tx=0x7f94e000c870 comp rx=0 tx=0).stop 2023-08-27T11:26:24.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.716+0000 7f94fe61c700 1 -- 172.21.15.112:0/1336646775 shutdown_connections 2023-08-27T11:26:24.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.716+0000 7f94fe61c700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f94d806a3b0 0x7f94d806c870 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.716+0000 7f94fe61c700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f8161ce0 0x7f94f80c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.716+0000 7f94fe61c700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94f80b1400 0x7f94f814f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.716+0000 7f94fe61c700 1 --2- 172.21.15.112:0/1336646775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94f80b17e0 0x7f94f81f2d40 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.716+0000 7f94fe61c700 1 -- 172.21.15.112:0/1336646775 >> 172.21.15.112:0/1336646775 conn(0x7f94f80d0000 msgr2=0x7f94f80cb350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:24.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.716+0000 7f94fe61c700 1 -- 172.21.15.112:0/1336646775 shutdown_connections 2023-08-27T11:26:24.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.716+0000 7f94fe61c700 1 -- 172.21.15.112:0/1336646775 wait complete. 2023-08-27T11:26:24.730 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-27T11:26:24.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:26:24.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.849+0000 7efc35ce9700 1 Processor -- start 2023-08-27T11:26:24.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.850+0000 7efc35ce9700 1 -- start start 2023-08-27T11:26:24.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.850+0000 7efc35ce9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efc300b3e50 0x7efc3015f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.850+0000 7efc35ce9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc300b4ad0 0x7efc300b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.850+0000 7efc35ce9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc300b45f0 con 0x7efc30144570 2023-08-27T11:26:24.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.850+0000 7efc35ce9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc300cf7a0 con 0x7efc300b3e50 2023-08-27T11:26:24.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.850+0000 7efc35ce9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc30144950 con 0x7efc300b4ad0 2023-08-27T11:26:24.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.850+0000 7efc2effd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efc300b3e50 0x7efc3015f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.850+0000 7efc2f7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7efc30144570 0x7efc3015ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55568/0 (socket says 172.21.15.112:55568) 2023-08-27T11:26:24.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.851+0000 7efc2effd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efc300b3e50 0x7efc3015f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32956/0 (socket says 172.21.15.112:32956) 2023-08-27T11:26:24.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.851+0000 7efc2f7fe700 1 -- 172.21.15.112:0/234288686 learned_addr learned my addr 172.21.15.112:0/234288686 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:24.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.851+0000 7efc2ffff700 1 --2- 172.21.15.112:0/234288686 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc300b4ad0 0x7efc300b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.851+0000 7efc2effd700 1 -- 172.21.15.112:0/234288686 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc300b4ad0 msgr2=0x7efc300b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:24.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.851+0000 7efc2effd700 1 --2- 172.21.15.112:0/234288686 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc300b4ad0 0x7efc300b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.851+0000 7efc2effd700 1 -- 172.21.15.112:0/234288686 >> v1:172.21.15.112:6789/0 conn(0x7efc30144570 legacy=0x7efc3015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.851+0000 7efc2e7fc700 1 -- 172.21.15.112:0/234288686 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361946893 0 0) 0x7efc300b45f0 con 0x7efc30144570 2023-08-27T11:26:24.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.851+0000 7efc2effd700 1 -- 172.21.15.112:0/234288686 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc300b4310 con 0x7efc300b3e50 2023-08-27T11:26:24.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.851+0000 7efc2ffff700 1 --2- 172.21.15.112:0/234288686 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc300b4ad0 0x7efc300b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:24.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.851+0000 7efc2effd700 1 --2- 172.21.15.112:0/234288686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3e50 0x7efc3015f530 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7efc18009870 tx=0x7efc18009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468861c32d745a50 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:24.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.851+0000 7efc2e7fc700 1 -- 172.21.15.112:0/234288686 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18011830 con 0x7efc300b3e50 2023-08-27T11:26:24.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.851+0000 7efc2e7fc700 1 -- 172.21.15.112:0/234288686 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc18011990 con 0x7efc300b3e50 2023-08-27T11:26:24.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.851+0000 7efc2e7fc700 1 -- 172.21.15.112:0/234288686 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18011d20 con 0x7efc300b3e50 2023-08-27T11:26:24.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.852+0000 7efc35ce9700 1 -- 172.21.15.112:0/234288686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3e50 msgr2=0x7efc3015f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.852+0000 7efc35ce9700 1 --2- 172.21.15.112:0/234288686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3e50 0x7efc3015f530 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7efc18009870 tx=0x7efc18009b80 comp rx=0 tx=0).stop 2023-08-27T11:26:24.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.852+0000 7efc35ce9700 1 -- 172.21.15.112:0/234288686 shutdown_connections 2023-08-27T11:26:24.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.852+0000 7efc35ce9700 1 --2- 172.21.15.112:0/234288686 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc300b4ad0 0x7efc300b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.852+0000 7efc35ce9700 1 --2- 172.21.15.112:0/234288686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3e50 0x7efc3015f530 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.852+0000 7efc35ce9700 1 -- 172.21.15.112:0/234288686 >> 172.21.15.112:0/234288686 conn(0x7efc3006f970 msgr2=0x7efc3006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:24.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.852+0000 7efc35ce9700 1 -- 172.21.15.112:0/234288686 shutdown_connections 2023-08-27T11:26:24.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.852+0000 7efc35ce9700 1 -- 172.21.15.112:0/234288686 wait complete. 2023-08-27T11:26:24.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.853+0000 7efc35ce9700 1 Processor -- start 2023-08-27T11:26:24.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.853+0000 7efc35ce9700 1 -- start start 2023-08-27T11:26:24.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc35ce9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc300b3e50 0x7efc3013bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc35ce9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc300b4ad0 0x7efc30142490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc35ce9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30144570 0x7efc301429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc35ce9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300ccb60 con 0x7efc30144570 2023-08-27T11:26:24.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc35ce9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc301f0390 con 0x7efc300b3e50 2023-08-27T11:26:24.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc35ce9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3013c240 con 0x7efc300b4ad0 2023-08-27T11:26:24.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc2ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30144570 0x7efc301429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc2ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30144570 0x7efc301429d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32964/0 (socket says 172.21.15.112:32964) 2023-08-27T11:26:24.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc2ffff700 1 -- 172.21.15.112:0/2729952237 learned_addr learned my addr 172.21.15.112:0/2729952237 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:24.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc2effd700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc300b4ad0 0x7efc30142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc2f7fe700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc300b3e50 0x7efc3013bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc2ffff700 1 -- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc300b4ad0 msgr2=0x7efc30142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc2ffff700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc300b4ad0 0x7efc30142490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc2ffff700 1 -- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc300b3e50 msgr2=0x7efc3013bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:24.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc2ffff700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc300b3e50 0x7efc3013bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:24.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc2ffff700 1 -- 172.21.15.112:0/2729952237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc180094a0 con 0x7efc30144570 2023-08-27T11:26:24.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc2f7fe700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc300b3e50 0x7efc3013bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:24.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc2effd700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc300b4ad0 0x7efc30142490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:24.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc2ffff700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30144570 0x7efc301429d0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7efc2800cef0 tx=0x7efc2800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:24.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc27fff700 1 -- 172.21.15.112:0/2729952237 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2800ae50 con 0x7efc30144570 2023-08-27T11:26:24.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc27fff700 1 -- 172.21.15.112:0/2729952237 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc28015440 con 0x7efc30144570 2023-08-27T11:26:24.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.854+0000 7efc35ce9700 1 -- 172.21.15.112:0/2729952237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc300cbfe0 con 0x7efc30144570 2023-08-27T11:26:24.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.855+0000 7efc27fff700 1 -- 172.21.15.112:0/2729952237 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc280157f0 con 0x7efc30144570 2023-08-27T11:26:24.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.855+0000 7efc35ce9700 1 -- 172.21.15.112:0/2729952237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc300cc5a0 con 0x7efc30144570 2023-08-27T11:26:24.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.856+0000 7efc35ce9700 1 -- 172.21.15.112:0/2729952237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc30067a20 con 0x7efc30144570 2023-08-27T11:26:24.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.856+0000 7efc27fff700 1 -- 172.21.15.112:0/2729952237 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7efc2800d430 con 0x7efc30144570 2023-08-27T11:26:24.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.856+0000 7efc27fff700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc1006a3b0 0x7efc1006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:24.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.856+0000 7efc27fff700 1 -- 172.21.15.112:0/2729952237 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efc28088fd0 con 0x7efc30144570 2023-08-27T11:26:24.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.856+0000 7efc2f7fe700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc1006a3b0 0x7efc1006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:24.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.857+0000 7efc2f7fe700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc1006a3b0 0x7efc1006c870 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efc20000f60 tx=0x7efc20007ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:24.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:24.860+0000 7efc27fff700 1 -- 172.21.15.112:0/2729952237 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efc280bb990 con 0x7efc30144570 2023-08-27T11:26:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.043+0000 7efc35ce9700 1 -- 172.21.15.112:0/2729952237 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc300cdf90 con 0x7efc1006a3b0 2023-08-27T11:26:25.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.053+0000 7efc27fff700 1 -- 172.21.15.112:0/2729952237 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312009 (secure 0 0 0) 0x7efc20005bb0 con 0x7efc1006a3b0 2023-08-27T11:26:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:25.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.056+0000 7efc35ce9700 1 -- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc1006a3b0 msgr2=0x7efc1006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:25.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.056+0000 7efc35ce9700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc1006a3b0 0x7efc1006c870 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efc20000f60 tx=0x7efc20007ca0 comp rx=0 tx=0).stop 2023-08-27T11:26:25.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.056+0000 7efc35ce9700 1 -- 172.21.15.112:0/2729952237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30144570 msgr2=0x7efc301429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:25.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.056+0000 7efc35ce9700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30144570 0x7efc301429d0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7efc2800cef0 tx=0x7efc2800a340 comp rx=0 tx=0).stop 2023-08-27T11:26:25.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.056+0000 7efc35ce9700 1 -- 172.21.15.112:0/2729952237 shutdown_connections 2023-08-27T11:26:25.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.056+0000 7efc35ce9700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc1006a3b0 0x7efc1006c870 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:25.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.056+0000 7efc35ce9700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30144570 0x7efc301429d0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:25.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.056+0000 7efc35ce9700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc300b3e50 0x7efc3013bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:25.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.056+0000 7efc35ce9700 1 --2- 172.21.15.112:0/2729952237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc300b4ad0 0x7efc30142490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:25.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.056+0000 7efc35ce9700 1 -- 172.21.15.112:0/2729952237 >> 172.21.15.112:0/2729952237 conn(0x7efc3006f970 msgr2=0x7efc300cef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:25.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.056+0000 7efc35ce9700 1 -- 172.21.15.112:0/2729952237 shutdown_connections 2023-08-27T11:26:25.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:25.056+0000 7efc35ce9700 1 -- 172.21.15.112:0/2729952237 wait complete. 2023-08-27T11:26:25.065 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:26:25.070 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2023-08-27T11:26:24.369354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259388,"kb_used_data":9924,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754715332,"statfs":{"total":773094113280,"available":772828499968,"internally_reserved":0,"allocated":10162176,"data_stored":5286515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001405"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874566+0000","last_change":"2023-08-27T11:26:19.427207+0000","last_active":"2023-08-27T11:26:20.874566+0000","last_peered":"2023-08-27T11:26:20.874566+0000","last_clean":"2023-08-27T11:26:20.874566+0000","last_became_active":"2023-08-27T11:26:19.426963+0000","last_became_peered":"2023-08-27T11:26:19.426963+0000","last_unstale":"2023-08-27T11:26:20.874566+0000","last_undegraded":"2023-08-27T11:26:20.874566+0000","last_fullsized":"2023-08-27T11:26:20.874566+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:55:21.733240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430931+0000","last_change":"2023-08-27T11:26:19.431066+0000","last_active":"2023-08-27T11:26:20.430931+0000","last_peered":"2023-08-27T11:26:20.430931+0000","last_clean":"2023-08-27T11:26:20.430931+0000","last_became_active":"2023-08-27T11:26:19.430477+0000","last_became_peered":"2023-08-27T11:26:19.430477+0000","last_unstale":"2023-08-27T11:26:20.430931+0000","last_undegraded":"2023-08-27T11:26:20.430931+0000","last_fullsized":"2023-08-27T11:26:20.430931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:43:44.337358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874230+0000","last_change":"2023-08-27T11:26:19.434887+0000","last_active":"2023-08-27T11:26:20.874230+0000","last_peered":"2023-08-27T11:26:20.874230+0000","last_clean":"2023-08-27T11:26:20.874230+0000","last_became_active":"2023-08-27T11:26:19.434566+0000","last_became_peered":"2023-08-27T11:26:19.434566+0000","last_unstale":"2023-08-27T11:26:20.874230+0000","last_undegraded":"2023-08-27T11:26:20.874230+0000","last_fullsized":"2023-08-27T11:26:20.874230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:11:11.738619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425453+0000","last_change":"2023-08-27T11:26:19.426786+0000","last_active":"2023-08-27T11:26:20.425453+0000","last_peered":"2023-08-27T11:26:20.425453+0000","last_clean":"2023-08-27T11:26:20.425453+0000","last_became_active":"2023-08-27T11:26:19.426494+0000","last_became_peered":"2023-08-27T11:26:19.426494+0000","last_unstale":"2023-08-27T11:26:20.425453+0000","last_undegraded":"2023-08-27T11:26:20.425453+0000","last_fullsized":"2023-08-27T11:26:20.425453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:33:53.249169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057365+0000","last_change":"2023-08-27T11:26:19.424084+0000","last_active":"2023-08-27T11:26:21.057365+0000","last_peered":"2023-08-27T11:26:21.057365+0000","last_clean":"2023-08-27T11:26:21.057365+0000","last_became_active":"2023-08-27T11:26:19.422955+0000","last_became_peered":"2023-08-27T11:26:19.422955+0000","last_unstale":"2023-08-27T11:26:21.057365+0000","last_undegraded":"2023-08-27T11:26:21.057365+0000","last_fullsized":"2023-08-27T11:26:21.057365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:59:24.896402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430830+0000","last_change":"2023-08-27T11:26:19.424659+0000","last_active":"2023-08-27T11:26:20.430830+0000","last_peered":"2023-08-27T11:26:20.430830+0000","last_clean":"2023-08-27T11:26:20.430830+0000","last_became_active":"2023-08-27T11:26:19.424317+0000","last_became_peered":"2023-08-27T11:26:19.424317+0000","last_unstale":"2023-08-27T11:26:20.430830+0000","last_undegraded":"2023-08-27T11:26:20.430830+0000","last_fullsized":"2023-08-27T11:26:20.430830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:47:42.857298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430038+0000","last_change":"2023-08-27T11:26:19.421667+0000","last_active":"2023-08-27T11:26:20.430038+0000","last_peered":"2023-08-27T11:26:20.430038+0000","last_clean":"2023-08-27T11:26:20.430038+0000","last_became_active":"2023-08-27T11:26:19.421352+0000","last_became_peered":"2023-08-27T11:26:19.421352+0000","last_unstale":"2023-08-27T11:26:20.430038+0000","last_undegraded":"2023-08-27T11:26:20.430038+0000","last_fullsized":"2023-08-27T11:26:20.430038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:24.356051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384297+0000","last_change":"2023-08-27T11:26:19.430958+0000","last_active":"2023-08-27T11:26:21.384297+0000","last_peered":"2023-08-27T11:26:21.384297+0000","last_clean":"2023-08-27T11:26:21.384297+0000","last_became_active":"2023-08-27T11:26:19.430685+0000","last_became_peered":"2023-08-27T11:26:19.430685+0000","last_unstale":"2023-08-27T11:26:21.384297+0000","last_undegraded":"2023-08-27T11:26:21.384297+0000","last_fullsized":"2023-08-27T11:26:21.384297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:41.805565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431121+0000","last_change":"2023-08-27T11:26:19.430211+0000","last_active":"2023-08-27T11:26:20.431121+0000","last_peered":"2023-08-27T11:26:20.431121+0000","last_clean":"2023-08-27T11:26:20.431121+0000","last_became_active":"2023-08-27T11:26:19.429921+0000","last_became_peered":"2023-08-27T11:26:19.429921+0000","last_unstale":"2023-08-27T11:26:20.431121+0000","last_undegraded":"2023-08-27T11:26:20.431121+0000","last_fullsized":"2023-08-27T11:26:20.431121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:56:34.526955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057756+0000","last_change":"2023-08-27T11:26:19.428983+0000","last_active":"2023-08-27T11:26:21.057756+0000","last_peered":"2023-08-27T11:26:21.057756+0000","last_clean":"2023-08-27T11:26:21.057756+0000","last_became_active":"2023-08-27T11:26:19.428699+0000","last_became_peered":"2023-08-27T11:26:19.428699+0000","last_unstale":"2023-08-27T11:26:21.057756+0000","last_undegraded":"2023-08-27T11:26:21.057756+0000","last_fullsized":"2023-08-27T11:26:21.057756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:49:32.779304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.386528+0000","last_change":"2023-08-27T11:26:19.433591+0000","last_active":"2023-08-27T11:26:21.386528+0000","last_peered":"2023-08-27T11:26:21.386528+0000","last_clean":"2023-08-27T11:26:21.386528+0000","last_became_active":"2023-08-27T11:26:19.433328+0000","last_became_peered":"2023-08-27T11:26:19.433328+0000","last_unstale":"2023-08-27T11:26:21.386528+0000","last_undegraded":"2023-08-27T11:26:21.386528+0000","last_fullsized":"2023-08-27T11:26:21.386528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:30.417469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430987+0000","last_change":"2023-08-27T11:26:19.428345+0000","last_active":"2023-08-27T11:26:20.430987+0000","last_peered":"2023-08-27T11:26:20.430987+0000","last_clean":"2023-08-27T11:26:20.430987+0000","last_became_active":"2023-08-27T11:26:19.427975+0000","last_became_peered":"2023-08-27T11:26:19.427975+0000","last_unstale":"2023-08-27T11:26:20.430987+0000","last_undegraded":"2023-08-27T11:26:20.430987+0000","last_fullsized":"2023-08-27T11:26:20.430987+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:08:01.417089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425155+0000","last_change":"2023-08-27T11:26:19.436540+0000","last_active":"2023-08-27T11:26:20.425155+0000","last_peered":"2023-08-27T11:26:20.425155+0000","last_clean":"2023-08-27T11:26:20.425155+0000","last_became_active":"2023-08-27T11:26:19.436278+0000","last_became_peered":"2023-08-27T11:26:19.436278+0000","last_unstale":"2023-08-27T11:26:20.425155+0000","last_undegraded":"2023-08-27T11:26:20.425155+0000","last_fullsized":"2023-08-27T11:26:20.425155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:40:05.207374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385238+0000","last_change":"2023-08-27T11:26:19.429289+0000","last_active":"2023-08-27T11:26:21.385238+0000","last_peered":"2023-08-27T11:26:21.385238+0000","last_clean":"2023-08-27T11:26:21.385238+0000","last_became_active":"2023-08-27T11:26:19.429019+0000","last_became_peered":"2023-08-27T11:26:19.429019+0000","last_unstale":"2023-08-27T11:26:21.385238+0000","last_undegraded":"2023-08-27T11:26:21.385238+0000","last_fullsized":"2023-08-27T11:26:21.385238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:02.295653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384991+0000","last_change":"2023-08-27T11:26:19.437669+0000","last_active":"2023-08-27T11:26:21.384991+0000","last_peered":"2023-08-27T11:26:21.384991+0000","last_clean":"2023-08-27T11:26:21.384991+0000","last_became_active":"2023-08-27T11:26:19.437024+0000","last_became_peered":"2023-08-27T11:26:19.437024+0000","last_unstale":"2023-08-27T11:26:21.384991+0000","last_undegraded":"2023-08-27T11:26:21.384991+0000","last_fullsized":"2023-08-27T11:26:21.384991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:51.492355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385006+0000","last_change":"2023-08-27T11:26:19.429490+0000","last_active":"2023-08-27T11:26:21.385006+0000","last_peered":"2023-08-27T11:26:21.385006+0000","last_clean":"2023-08-27T11:26:21.385006+0000","last_became_active":"2023-08-27T11:26:19.429177+0000","last_became_peered":"2023-08-27T11:26:19.429177+0000","last_unstale":"2023-08-27T11:26:21.385006+0000","last_undegraded":"2023-08-27T11:26:21.385006+0000","last_fullsized":"2023-08-27T11:26:21.385006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:08:53.141523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874577+0000","last_change":"2023-08-27T11:26:19.434923+0000","last_active":"2023-08-27T11:26:20.874577+0000","last_peered":"2023-08-27T11:26:20.874577+0000","last_clean":"2023-08-27T11:26:20.874577+0000","last_became_active":"2023-08-27T11:26:19.434650+0000","last_became_peered":"2023-08-27T11:26:19.434650+0000","last_unstale":"2023-08-27T11:26:20.874577+0000","last_undegraded":"2023-08-27T11:26:20.874577+0000","last_fullsized":"2023-08-27T11:26:20.874577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:33:06.493994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430846+0000","last_change":"2023-08-27T11:26:19.422300+0000","last_active":"2023-08-27T11:26:20.430846+0000","last_peered":"2023-08-27T11:26:20.430846+0000","last_clean":"2023-08-27T11:26:20.430846+0000","last_became_active":"2023-08-27T11:26:19.421284+0000","last_became_peered":"2023-08-27T11:26:19.421284+0000","last_unstale":"2023-08-27T11:26:20.430846+0000","last_undegraded":"2023-08-27T11:26:20.430846+0000","last_fullsized":"2023-08-27T11:26:20.430846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:12.363650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874425+0000","last_change":"2023-08-27T11:26:19.418470+0000","last_active":"2023-08-27T11:26:20.874425+0000","last_peered":"2023-08-27T11:26:20.874425+0000","last_clean":"2023-08-27T11:26:20.874425+0000","last_became_active":"2023-08-27T11:26:19.418192+0000","last_became_peered":"2023-08-27T11:26:19.418192+0000","last_unstale":"2023-08-27T11:26:20.874425+0000","last_undegraded":"2023-08-27T11:26:20.874425+0000","last_fullsized":"2023-08-27T11:26:20.874425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:14:53.959765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430870+0000","last_change":"2023-08-27T11:26:19.431041+0000","last_active":"2023-08-27T11:26:20.430870+0000","last_peered":"2023-08-27T11:26:20.430870+0000","last_clean":"2023-08-27T11:26:20.430870+0000","last_became_active":"2023-08-27T11:26:19.430750+0000","last_became_peered":"2023-08-27T11:26:19.430750+0000","last_unstale":"2023-08-27T11:26:20.430870+0000","last_undegraded":"2023-08-27T11:26:20.430870+0000","last_fullsized":"2023-08-27T11:26:20.430870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:17:25.874882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384767+0000","last_change":"2023-08-27T11:26:19.433627+0000","last_active":"2023-08-27T11:26:21.384767+0000","last_peered":"2023-08-27T11:26:21.384767+0000","last_clean":"2023-08-27T11:26:21.384767+0000","last_became_active":"2023-08-27T11:26:19.433370+0000","last_became_peered":"2023-08-27T11:26:19.433370+0000","last_unstale":"2023-08-27T11:26:21.384767+0000","last_undegraded":"2023-08-27T11:26:21.384767+0000","last_fullsized":"2023-08-27T11:26:21.384767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:41.696204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057432+0000","last_change":"2023-08-27T11:26:19.433519+0000","last_active":"2023-08-27T11:26:21.057432+0000","last_peered":"2023-08-27T11:26:21.057432+0000","last_clean":"2023-08-27T11:26:21.057432+0000","last_became_active":"2023-08-27T11:26:19.433245+0000","last_became_peered":"2023-08-27T11:26:19.433245+0000","last_unstale":"2023-08-27T11:26:21.057432+0000","last_undegraded":"2023-08-27T11:26:21.057432+0000","last_fullsized":"2023-08-27T11:26:21.057432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:07:00.066813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874575+0000","last_change":"2023-08-27T11:26:19.434762+0000","last_active":"2023-08-27T11:26:20.874575+0000","last_peered":"2023-08-27T11:26:20.874575+0000","last_clean":"2023-08-27T11:26:20.874575+0000","last_became_active":"2023-08-27T11:26:19.434462+0000","last_became_peered":"2023-08-27T11:26:19.434462+0000","last_unstale":"2023-08-27T11:26:20.874575+0000","last_undegraded":"2023-08-27T11:26:20.874575+0000","last_fullsized":"2023-08-27T11:26:20.874575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:41.356424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057290+0000","last_change":"2023-08-27T11:26:19.423833+0000","last_active":"2023-08-27T11:26:21.057290+0000","last_peered":"2023-08-27T11:26:21.057290+0000","last_clean":"2023-08-27T11:26:21.057290+0000","last_became_active":"2023-08-27T11:26:19.423440+0000","last_became_peered":"2023-08-27T11:26:19.423440+0000","last_unstale":"2023-08-27T11:26:21.057290+0000","last_undegraded":"2023-08-27T11:26:21.057290+0000","last_fullsized":"2023-08-27T11:26:21.057290+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:41:13.840925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431105+0000","last_change":"2023-08-27T11:26:19.434228+0000","last_active":"2023-08-27T11:26:20.431105+0000","last_peered":"2023-08-27T11:26:20.431105+0000","last_clean":"2023-08-27T11:26:20.431105+0000","last_became_active":"2023-08-27T11:26:19.433958+0000","last_became_peered":"2023-08-27T11:26:19.433958+0000","last_unstale":"2023-08-27T11:26:20.431105+0000","last_undegraded":"2023-08-27T11:26:20.431105+0000","last_fullsized":"2023-08-27T11:26:20.431105+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:25:08.328313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431061+0000","last_change":"2023-08-27T11:26:19.432242+0000","last_active":"2023-08-27T11:26:20.431061+0000","last_peered":"2023-08-27T11:26:20.431061+0000","last_clean":"2023-08-27T11:26:20.431061+0000","last_became_active":"2023-08-27T11:26:19.432006+0000","last_became_peered":"2023-08-27T11:26:19.432006+0000","last_unstale":"2023-08-27T11:26:20.431061+0000","last_undegraded":"2023-08-27T11:26:20.431061+0000","last_fullsized":"2023-08-27T11:26:20.431061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:47:39.278669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385953+0000","last_change":"2023-08-27T11:26:19.432982+0000","last_active":"2023-08-27T11:26:21.385953+0000","last_peered":"2023-08-27T11:26:21.385953+0000","last_clean":"2023-08-27T11:26:21.385953+0000","last_became_active":"2023-08-27T11:26:19.432668+0000","last_became_peered":"2023-08-27T11:26:19.432668+0000","last_unstale":"2023-08-27T11:26:21.385953+0000","last_undegraded":"2023-08-27T11:26:21.385953+0000","last_fullsized":"2023-08-27T11:26:21.385953+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:55.969633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430610+0000","last_change":"2023-08-27T11:26:19.433762+0000","last_active":"2023-08-27T11:26:20.430610+0000","last_peered":"2023-08-27T11:26:20.430610+0000","last_clean":"2023-08-27T11:26:20.430610+0000","last_became_active":"2023-08-27T11:26:19.433489+0000","last_became_peered":"2023-08-27T11:26:19.433489+0000","last_unstale":"2023-08-27T11:26:20.430610+0000","last_undegraded":"2023-08-27T11:26:20.430610+0000","last_fullsized":"2023-08-27T11:26:20.430610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:34.644697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384668+0000","last_change":"2023-08-27T11:26:19.431428+0000","last_active":"2023-08-27T11:26:21.384668+0000","last_peered":"2023-08-27T11:26:21.384668+0000","last_clean":"2023-08-27T11:26:21.384668+0000","last_became_active":"2023-08-27T11:26:19.431169+0000","last_became_peered":"2023-08-27T11:26:19.431169+0000","last_unstale":"2023-08-27T11:26:21.384668+0000","last_undegraded":"2023-08-27T11:26:21.384668+0000","last_fullsized":"2023-08-27T11:26:21.384668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:30:35.082113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430552+0000","last_change":"2023-08-27T11:26:19.435080+0000","last_active":"2023-08-27T11:26:20.430552+0000","last_peered":"2023-08-27T11:26:20.430552+0000","last_clean":"2023-08-27T11:26:20.430552+0000","last_became_active":"2023-08-27T11:26:19.434778+0000","last_became_peered":"2023-08-27T11:26:19.434778+0000","last_unstale":"2023-08-27T11:26:20.430552+0000","last_undegraded":"2023-08-27T11:26:20.430552+0000","last_fullsized":"2023-08-27T11:26:20.430552+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:27:49.502862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430321+0000","last_change":"2023-08-27T11:26:19.427535+0000","last_active":"2023-08-27T11:26:20.430321+0000","last_peered":"2023-08-27T11:26:20.430321+0000","last_clean":"2023-08-27T11:26:20.430321+0000","last_became_active":"2023-08-27T11:26:19.426191+0000","last_became_peered":"2023-08-27T11:26:19.426191+0000","last_unstale":"2023-08-27T11:26:20.430321+0000","last_undegraded":"2023-08-27T11:26:20.430321+0000","last_fullsized":"2023-08-27T11:26:20.430321+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:22:47.314202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384374+0000","last_change":"2023-08-27T11:26:19.424642+0000","last_active":"2023-08-27T11:26:21.384374+0000","last_peered":"2023-08-27T11:26:21.384374+0000","last_clean":"2023-08-27T11:26:21.384374+0000","last_became_active":"2023-08-27T11:26:19.424359+0000","last_became_peered":"2023-08-27T11:26:19.424359+0000","last_unstale":"2023-08-27T11:26:21.384374+0000","last_undegraded":"2023-08-27T11:26:21.384374+0000","last_fullsized":"2023-08-27T11:26:21.384374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:06:36.252165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874126+0000","last_change":"2023-08-27T11:26:19.414983+0000","last_active":"2023-08-27T11:26:20.874126+0000","last_peered":"2023-08-27T11:26:20.874126+0000","last_clean":"2023-08-27T11:26:20.874126+0000","last_became_active":"2023-08-27T11:26:19.414726+0000","last_became_peered":"2023-08-27T11:26:19.414726+0000","last_unstale":"2023-08-27T11:26:20.874126+0000","last_undegraded":"2023-08-27T11:26:20.874126+0000","last_fullsized":"2023-08-27T11:26:20.874126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:57:45.405543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"89'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425328+0000","last_change":"2023-08-27T11:26:18.432220+0000","last_active":"2023-08-27T11:26:20.425328+0000","last_peered":"2023-08-27T11:26:20.425328+0000","last_clean":"2023-08-27T11:26:20.425328+0000","last_became_active":"2023-08-27T11:26:18.431938+0000","last_became_peered":"2023-08-27T11:26:18.431938+0000","last_unstale":"2023-08-27T11:26:20.425328+0000","last_undegraded":"2023-08-27T11:26:20.425328+0000","last_fullsized":"2023-08-27T11:26:20.425328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:26:24.683316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384352+0000","last_change":"2023-08-27T11:26:19.427981+0000","last_active":"2023-08-27T11:26:21.384352+0000","last_peered":"2023-08-27T11:26:21.384352+0000","last_clean":"2023-08-27T11:26:21.384352+0000","last_became_active":"2023-08-27T11:26:19.427694+0000","last_became_peered":"2023-08-27T11:26:19.427694+0000","last_unstale":"2023-08-27T11:26:21.384352+0000","last_undegraded":"2023-08-27T11:26:21.384352+0000","last_fullsized":"2023-08-27T11:26:21.384352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:31.571236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384746+0000","last_change":"2023-08-27T11:26:18.416729+0000","last_active":"2023-08-27T11:26:21.384746+0000","last_peered":"2023-08-27T11:26:21.384746+0000","last_clean":"2023-08-27T11:26:21.384746+0000","last_became_active":"2023-08-27T11:26:18.416320+0000","last_became_peered":"2023-08-27T11:26:18.416320+0000","last_unstale":"2023-08-27T11:26:21.384746+0000","last_undegraded":"2023-08-27T11:26:21.384746+0000","last_fullsized":"2023-08-27T11:26:21.384746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:32.457209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430696+0000","last_change":"2023-08-27T11:26:19.422239+0000","last_active":"2023-08-27T11:26:20.430696+0000","last_peered":"2023-08-27T11:26:20.430696+0000","last_clean":"2023-08-27T11:26:20.430696+0000","last_became_active":"2023-08-27T11:26:19.421958+0000","last_became_peered":"2023-08-27T11:26:19.421958+0000","last_unstale":"2023-08-27T11:26:20.430696+0000","last_undegraded":"2023-08-27T11:26:20.430696+0000","last_fullsized":"2023-08-27T11:26:20.430696+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:06:02.839193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430341+0000","last_change":"2023-08-27T11:26:18.426347+0000","last_active":"2023-08-27T11:26:20.430341+0000","last_peered":"2023-08-27T11:26:20.430341+0000","last_clean":"2023-08-27T11:26:20.430341+0000","last_became_active":"2023-08-27T11:26:18.426066+0000","last_became_peered":"2023-08-27T11:26:18.426066+0000","last_unstale":"2023-08-27T11:26:20.430341+0000","last_undegraded":"2023-08-27T11:26:20.430341+0000","last_fullsized":"2023-08-27T11:26:20.430341+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:48:56.592853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385761+0000","last_change":"2023-08-27T11:26:19.434508+0000","last_active":"2023-08-27T11:26:21.385761+0000","last_peered":"2023-08-27T11:26:21.385761+0000","last_clean":"2023-08-27T11:26:21.385761+0000","last_became_active":"2023-08-27T11:26:19.433793+0000","last_became_peered":"2023-08-27T11:26:19.433793+0000","last_unstale":"2023-08-27T11:26:21.385761+0000","last_undegraded":"2023-08-27T11:26:21.385761+0000","last_fullsized":"2023-08-27T11:26:21.385761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:59.712674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057704+0000","last_change":"2023-08-27T11:26:18.426839+0000","last_active":"2023-08-27T11:26:21.057704+0000","last_peered":"2023-08-27T11:26:21.057704+0000","last_clean":"2023-08-27T11:26:21.057704+0000","last_became_active":"2023-08-27T11:26:18.426578+0000","last_became_peered":"2023-08-27T11:26:18.426578+0000","last_unstale":"2023-08-27T11:26:21.057704+0000","last_undegraded":"2023-08-27T11:26:21.057704+0000","last_fullsized":"2023-08-27T11:26:21.057704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:23.764319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425272+0000","last_change":"2023-08-27T11:26:19.426762+0000","last_active":"2023-08-27T11:26:20.425272+0000","last_peered":"2023-08-27T11:26:20.425272+0000","last_clean":"2023-08-27T11:26:20.425272+0000","last_became_active":"2023-08-27T11:26:19.426146+0000","last_became_peered":"2023-08-27T11:26:19.426146+0000","last_unstale":"2023-08-27T11:26:20.425272+0000","last_undegraded":"2023-08-27T11:26:20.425272+0000","last_fullsized":"2023-08-27T11:26:20.425272+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:54:50.371981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385535+0000","last_change":"2023-08-27T11:26:18.423094+0000","last_active":"2023-08-27T11:26:21.385535+0000","last_peered":"2023-08-27T11:26:21.385535+0000","last_clean":"2023-08-27T11:26:21.385535+0000","last_became_active":"2023-08-27T11:26:18.422776+0000","last_became_peered":"2023-08-27T11:26:18.422776+0000","last_unstale":"2023-08-27T11:26:21.385535+0000","last_undegraded":"2023-08-27T11:26:21.385535+0000","last_fullsized":"2023-08-27T11:26:21.385535+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:29:26.967597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874343+0000","last_change":"2023-08-27T11:26:19.418506+0000","last_active":"2023-08-27T11:26:20.874343+0000","last_peered":"2023-08-27T11:26:20.874343+0000","last_clean":"2023-08-27T11:26:20.874343+0000","last_became_active":"2023-08-27T11:26:19.418235+0000","last_became_peered":"2023-08-27T11:26:19.418235+0000","last_unstale":"2023-08-27T11:26:20.874343+0000","last_undegraded":"2023-08-27T11:26:20.874343+0000","last_fullsized":"2023-08-27T11:26:20.874343+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:50.719209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385880+0000","last_change":"2023-08-27T11:26:18.429721+0000","last_active":"2023-08-27T11:26:21.385880+0000","last_peered":"2023-08-27T11:26:21.385880+0000","last_clean":"2023-08-27T11:26:21.385880+0000","last_became_active":"2023-08-27T11:26:18.429487+0000","last_became_peered":"2023-08-27T11:26:18.429487+0000","last_unstale":"2023-08-27T11:26:21.385880+0000","last_undegraded":"2023-08-27T11:26:21.385880+0000","last_fullsized":"2023-08-27T11:26:21.385880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:06.592671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430641+0000","last_change":"2023-08-27T11:26:19.431258+0000","last_active":"2023-08-27T11:26:20.430641+0000","last_peered":"2023-08-27T11:26:20.430641+0000","last_clean":"2023-08-27T11:26:20.430641+0000","last_became_active":"2023-08-27T11:26:19.430816+0000","last_became_peered":"2023-08-27T11:26:19.430816+0000","last_unstale":"2023-08-27T11:26:20.430641+0000","last_undegraded":"2023-08-27T11:26:20.430641+0000","last_fullsized":"2023-08-27T11:26:20.430641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:55.969633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430088+0000","last_change":"2023-08-27T11:26:18.426802+0000","last_active":"2023-08-27T11:26:20.430088+0000","last_peered":"2023-08-27T11:26:20.430088+0000","last_clean":"2023-08-27T11:26:20.430088+0000","last_became_active":"2023-08-27T11:26:18.426076+0000","last_became_peered":"2023-08-27T11:26:18.426076+0000","last_unstale":"2023-08-27T11:26:20.430088+0000","last_undegraded":"2023-08-27T11:26:20.430088+0000","last_fullsized":"2023-08-27T11:26:20.430088+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:04:08.369081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425484+0000","last_change":"2023-08-27T11:26:19.426307+0000","last_active":"2023-08-27T11:26:20.425484+0000","last_peered":"2023-08-27T11:26:20.425484+0000","last_clean":"2023-08-27T11:26:20.425484+0000","last_became_active":"2023-08-27T11:26:19.425959+0000","last_became_peered":"2023-08-27T11:26:19.425959+0000","last_unstale":"2023-08-27T11:26:20.425484+0000","last_undegraded":"2023-08-27T11:26:20.425484+0000","last_fullsized":"2023-08-27T11:26:20.425484+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:10:02.440599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425034+0000","last_change":"2023-08-27T11:26:18.429003+0000","last_active":"2023-08-27T11:26:20.425034+0000","last_peered":"2023-08-27T11:26:20.425034+0000","last_clean":"2023-08-27T11:26:20.425034+0000","last_became_active":"2023-08-27T11:26:18.428701+0000","last_became_peered":"2023-08-27T11:26:18.428701+0000","last_unstale":"2023-08-27T11:26:20.425034+0000","last_undegraded":"2023-08-27T11:26:20.425034+0000","last_fullsized":"2023-08-27T11:26:20.425034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:42:33.783460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874118+0000","last_change":"2023-08-27T11:26:19.417293+0000","last_active":"2023-08-27T11:26:20.874118+0000","last_peered":"2023-08-27T11:26:20.874118+0000","last_clean":"2023-08-27T11:26:20.874118+0000","last_became_active":"2023-08-27T11:26:19.417018+0000","last_became_peered":"2023-08-27T11:26:19.417018+0000","last_unstale":"2023-08-27T11:26:20.874118+0000","last_undegraded":"2023-08-27T11:26:20.874118+0000","last_fullsized":"2023-08-27T11:26:20.874118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:02:56.092248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431106+0000","last_change":"2023-08-27T11:26:18.429425+0000","last_active":"2023-08-27T11:26:20.431106+0000","last_peered":"2023-08-27T11:26:20.431106+0000","last_clean":"2023-08-27T11:26:20.431106+0000","last_became_active":"2023-08-27T11:26:18.429137+0000","last_became_peered":"2023-08-27T11:26:18.429137+0000","last_unstale":"2023-08-27T11:26:20.431106+0000","last_undegraded":"2023-08-27T11:26:20.431106+0000","last_fullsized":"2023-08-27T11:26:20.431106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:26:09.978662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057401+0000","last_change":"2023-08-27T11:26:19.425413+0000","last_active":"2023-08-27T11:26:21.057401+0000","last_peered":"2023-08-27T11:26:21.057401+0000","last_clean":"2023-08-27T11:26:21.057401+0000","last_became_active":"2023-08-27T11:26:19.424482+0000","last_became_peered":"2023-08-27T11:26:19.424482+0000","last_unstale":"2023-08-27T11:26:21.057401+0000","last_undegraded":"2023-08-27T11:26:21.057401+0000","last_fullsized":"2023-08-27T11:26:21.057401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:36.587246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874328+0000","last_change":"2023-08-27T11:26:18.426356+0000","last_active":"2023-08-27T11:26:20.874328+0000","last_peered":"2023-08-27T11:26:20.874328+0000","last_clean":"2023-08-27T11:26:20.874328+0000","last_became_active":"2023-08-27T11:26:18.426041+0000","last_became_peered":"2023-08-27T11:26:18.426041+0000","last_unstale":"2023-08-27T11:26:20.874328+0000","last_undegraded":"2023-08-27T11:26:20.874328+0000","last_fullsized":"2023-08-27T11:26:20.874328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:28.381950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384393+0000","last_change":"2023-08-27T11:26:19.431679+0000","last_active":"2023-08-27T11:26:21.384393+0000","last_peered":"2023-08-27T11:26:21.384393+0000","last_clean":"2023-08-27T11:26:21.384393+0000","last_became_active":"2023-08-27T11:26:19.431405+0000","last_became_peered":"2023-08-27T11:26:19.431405+0000","last_unstale":"2023-08-27T11:26:21.384393+0000","last_undegraded":"2023-08-27T11:26:21.384393+0000","last_fullsized":"2023-08-27T11:26:21.384393+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:02:21.450786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"89'3","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385384+0000","last_change":"2023-08-27T11:26:18.421366+0000","last_active":"2023-08-27T11:26:21.385384+0000","last_peered":"2023-08-27T11:26:21.385384+0000","last_clean":"2023-08-27T11:26:21.385384+0000","last_became_active":"2023-08-27T11:26:18.420937+0000","last_became_peered":"2023-08-27T11:26:18.420937+0000","last_unstale":"2023-08-27T11:26:21.385384+0000","last_undegraded":"2023-08-27T11:26:21.385384+0000","last_fullsized":"2023-08-27T11:26:21.385384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:44:04.672082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057480+0000","last_change":"2023-08-27T11:26:19.424582+0000","last_active":"2023-08-27T11:26:21.057480+0000","last_peered":"2023-08-27T11:26:21.057480+0000","last_clean":"2023-08-27T11:26:21.057480+0000","last_became_active":"2023-08-27T11:26:19.424288+0000","last_became_peered":"2023-08-27T11:26:19.424288+0000","last_unstale":"2023-08-27T11:26:21.057480+0000","last_undegraded":"2023-08-27T11:26:21.057480+0000","last_fullsized":"2023-08-27T11:26:21.057480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:16.279109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430741+0000","last_change":"2023-08-27T11:26:18.421802+0000","last_active":"2023-08-27T11:26:20.430741+0000","last_peered":"2023-08-27T11:26:20.430741+0000","last_clean":"2023-08-27T11:26:20.430741+0000","last_became_active":"2023-08-27T11:26:18.421093+0000","last_became_peered":"2023-08-27T11:26:18.421093+0000","last_unstale":"2023-08-27T11:26:20.430741+0000","last_undegraded":"2023-08-27T11:26:20.430741+0000","last_fullsized":"2023-08-27T11:26:20.430741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:29.408679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385662+0000","last_change":"2023-08-27T11:26:19.435932+0000","last_active":"2023-08-27T11:26:21.385662+0000","last_peered":"2023-08-27T11:26:21.385662+0000","last_clean":"2023-08-27T11:26:21.385662+0000","last_became_active":"2023-08-27T11:26:19.435665+0000","last_became_peered":"2023-08-27T11:26:19.435665+0000","last_unstale":"2023-08-27T11:26:21.385662+0000","last_undegraded":"2023-08-27T11:26:21.385662+0000","last_fullsized":"2023-08-27T11:26:21.385662+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:11.616933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057739+0000","last_change":"2023-08-27T11:26:18.421876+0000","last_active":"2023-08-27T11:26:21.057739+0000","last_peered":"2023-08-27T11:26:21.057739+0000","last_clean":"2023-08-27T11:26:21.057739+0000","last_became_active":"2023-08-27T11:26:18.421583+0000","last_became_peered":"2023-08-27T11:26:18.421583+0000","last_unstale":"2023-08-27T11:26:21.057739+0000","last_undegraded":"2023-08-27T11:26:21.057739+0000","last_fullsized":"2023-08-27T11:26:21.057739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:48:43.946799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430199+0000","last_change":"2023-08-27T11:26:19.430893+0000","last_active":"2023-08-27T11:26:20.430199+0000","last_peered":"2023-08-27T11:26:20.430199+0000","last_clean":"2023-08-27T11:26:20.430199+0000","last_became_active":"2023-08-27T11:26:19.430225+0000","last_became_peered":"2023-08-27T11:26:19.430225+0000","last_unstale":"2023-08-27T11:26:20.430199+0000","last_undegraded":"2023-08-27T11:26:20.430199+0000","last_fullsized":"2023-08-27T11:26:20.430199+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:43.123979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425475+0000","last_change":"2023-08-27T11:26:18.432084+0000","last_active":"2023-08-27T11:26:20.425475+0000","last_peered":"2023-08-27T11:26:20.425475+0000","last_clean":"2023-08-27T11:26:20.425475+0000","last_became_active":"2023-08-27T11:26:18.431877+0000","last_became_peered":"2023-08-27T11:26:18.431877+0000","last_unstale":"2023-08-27T11:26:20.425475+0000","last_undegraded":"2023-08-27T11:26:20.425475+0000","last_fullsized":"2023-08-27T11:26:20.425475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:12:10.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384677+0000","last_change":"2023-08-27T11:26:19.429400+0000","last_active":"2023-08-27T11:26:21.384677+0000","last_peered":"2023-08-27T11:26:21.384677+0000","last_clean":"2023-08-27T11:26:21.384677+0000","last_became_active":"2023-08-27T11:26:19.429146+0000","last_became_peered":"2023-08-27T11:26:19.429146+0000","last_unstale":"2023-08-27T11:26:21.384677+0000","last_undegraded":"2023-08-27T11:26:21.384677+0000","last_fullsized":"2023-08-27T11:26:21.384677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:06.185951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384453+0000","last_change":"2023-08-27T11:26:18.420633+0000","last_active":"2023-08-27T11:26:21.384453+0000","last_peered":"2023-08-27T11:26:21.384453+0000","last_clean":"2023-08-27T11:26:21.384453+0000","last_became_active":"2023-08-27T11:26:18.420312+0000","last_became_peered":"2023-08-27T11:26:18.420312+0000","last_unstale":"2023-08-27T11:26:21.384453+0000","last_undegraded":"2023-08-27T11:26:21.384453+0000","last_fullsized":"2023-08-27T11:26:21.384453+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:20.872678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874639+0000","last_change":"2023-08-27T11:26:19.434950+0000","last_active":"2023-08-27T11:26:20.874639+0000","last_peered":"2023-08-27T11:26:20.874639+0000","last_clean":"2023-08-27T11:26:20.874639+0000","last_became_active":"2023-08-27T11:26:19.434684+0000","last_became_peered":"2023-08-27T11:26:19.434684+0000","last_unstale":"2023-08-27T11:26:20.874639+0000","last_undegraded":"2023-08-27T11:26:20.874639+0000","last_fullsized":"2023-08-27T11:26:20.874639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:52:42.875755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"89'8","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430974+0000","last_change":"2023-08-27T11:26:18.429430+0000","last_active":"2023-08-27T11:26:20.430974+0000","last_peered":"2023-08-27T11:26:20.430974+0000","last_clean":"2023-08-27T11:26:20.430974+0000","last_became_active":"2023-08-27T11:26:18.429158+0000","last_became_peered":"2023-08-27T11:26:18.429158+0000","last_unstale":"2023-08-27T11:26:20.430974+0000","last_undegraded":"2023-08-27T11:26:20.430974+0000","last_fullsized":"2023-08-27T11:26:20.430974+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:53:21.490317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874262+0000","last_change":"2023-08-27T11:26:19.425141+0000","last_active":"2023-08-27T11:26:20.874262+0000","last_peered":"2023-08-27T11:26:20.874262+0000","last_clean":"2023-08-27T11:26:20.874262+0000","last_became_active":"2023-08-27T11:26:19.424669+0000","last_became_peered":"2023-08-27T11:26:19.424669+0000","last_unstale":"2023-08-27T11:26:20.874262+0000","last_undegraded":"2023-08-27T11:26:20.874262+0000","last_fullsized":"2023-08-27T11:26:20.874262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:31:34.706537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057687+0000","last_change":"2023-08-27T11:26:18.421572+0000","last_active":"2023-08-27T11:26:21.057687+0000","last_peered":"2023-08-27T11:26:21.057687+0000","last_clean":"2023-08-27T11:26:21.057687+0000","last_became_active":"2023-08-27T11:26:18.421268+0000","last_became_peered":"2023-08-27T11:26:18.421268+0000","last_unstale":"2023-08-27T11:26:21.057687+0000","last_undegraded":"2023-08-27T11:26:21.057687+0000","last_fullsized":"2023-08-27T11:26:21.057687+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:44.389523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.429894+0000","last_change":"2023-08-27T11:26:19.427139+0000","last_active":"2023-08-27T11:26:20.429894+0000","last_peered":"2023-08-27T11:26:20.429894+0000","last_clean":"2023-08-27T11:26:20.429894+0000","last_became_active":"2023-08-27T11:26:19.426793+0000","last_became_peered":"2023-08-27T11:26:19.426793+0000","last_unstale":"2023-08-27T11:26:20.429894+0000","last_undegraded":"2023-08-27T11:26:20.429894+0000","last_fullsized":"2023-08-27T11:26:20.429894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:05:13.559317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874451+0000","last_change":"2023-08-27T11:26:18.429524+0000","last_active":"2023-08-27T11:26:20.874451+0000","last_peered":"2023-08-27T11:26:20.874451+0000","last_clean":"2023-08-27T11:26:20.874451+0000","last_became_active":"2023-08-27T11:26:18.429224+0000","last_became_peered":"2023-08-27T11:26:18.429224+0000","last_unstale":"2023-08-27T11:26:20.874451+0000","last_undegraded":"2023-08-27T11:26:20.874451+0000","last_fullsized":"2023-08-27T11:26:20.874451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:13.776898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425320+0000","last_change":"2023-08-27T11:26:19.427555+0000","last_active":"2023-08-27T11:26:20.425320+0000","last_peered":"2023-08-27T11:26:20.425320+0000","last_clean":"2023-08-27T11:26:20.425320+0000","last_became_active":"2023-08-27T11:26:19.427295+0000","last_became_peered":"2023-08-27T11:26:19.427295+0000","last_unstale":"2023-08-27T11:26:20.425320+0000","last_undegraded":"2023-08-27T11:26:20.425320+0000","last_fullsized":"2023-08-27T11:26:20.425320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:35.698617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430342+0000","last_change":"2023-08-27T11:26:18.432922+0000","last_active":"2023-08-27T11:26:20.430342+0000","last_peered":"2023-08-27T11:26:20.430342+0000","last_clean":"2023-08-27T11:26:20.430342+0000","last_became_active":"2023-08-27T11:26:18.432631+0000","last_became_peered":"2023-08-27T11:26:18.432631+0000","last_unstale":"2023-08-27T11:26:20.430342+0000","last_undegraded":"2023-08-27T11:26:20.430342+0000","last_fullsized":"2023-08-27T11:26:20.430342+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:58.703884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385691+0000","last_change":"2023-08-27T11:26:19.431706+0000","last_active":"2023-08-27T11:26:21.385691+0000","last_peered":"2023-08-27T11:26:21.385691+0000","last_clean":"2023-08-27T11:26:21.385691+0000","last_became_active":"2023-08-27T11:26:19.431424+0000","last_became_peered":"2023-08-27T11:26:19.431424+0000","last_unstale":"2023-08-27T11:26:21.385691+0000","last_undegraded":"2023-08-27T11:26:21.385691+0000","last_fullsized":"2023-08-27T11:26:21.385691+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:43.123979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057644+0000","last_change":"2023-08-27T11:26:18.422687+0000","last_active":"2023-08-27T11:26:21.057644+0000","last_peered":"2023-08-27T11:26:21.057644+0000","last_clean":"2023-08-27T11:26:21.057644+0000","last_became_active":"2023-08-27T11:26:18.422404+0000","last_became_peered":"2023-08-27T11:26:18.422404+0000","last_unstale":"2023-08-27T11:26:21.057644+0000","last_undegraded":"2023-08-27T11:26:21.057644+0000","last_fullsized":"2023-08-27T11:26:21.057644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:34.961579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384633+0000","last_change":"2023-08-27T11:26:19.425732+0000","last_active":"2023-08-27T11:26:21.384633+0000","last_peered":"2023-08-27T11:26:21.384633+0000","last_clean":"2023-08-27T11:26:21.384633+0000","last_became_active":"2023-08-27T11:26:19.425452+0000","last_became_peered":"2023-08-27T11:26:19.425452+0000","last_unstale":"2023-08-27T11:26:21.384633+0000","last_undegraded":"2023-08-27T11:26:21.384633+0000","last_fullsized":"2023-08-27T11:26:21.384633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:15.049461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430151+0000","last_change":"2023-08-27T11:26:18.423038+0000","last_active":"2023-08-27T11:26:20.430151+0000","last_peered":"2023-08-27T11:26:20.430151+0000","last_clean":"2023-08-27T11:26:20.430151+0000","last_became_active":"2023-08-27T11:26:18.422607+0000","last_became_peered":"2023-08-27T11:26:18.422607+0000","last_unstale":"2023-08-27T11:26:20.430151+0000","last_undegraded":"2023-08-27T11:26:20.430151+0000","last_fullsized":"2023-08-27T11:26:20.430151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:33:42.505117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384761+0000","last_change":"2023-08-27T11:26:19.425658+0000","last_active":"2023-08-27T11:26:21.384761+0000","last_peered":"2023-08-27T11:26:21.384761+0000","last_clean":"2023-08-27T11:26:21.384761+0000","last_became_active":"2023-08-27T11:26:19.425387+0000","last_became_peered":"2023-08-27T11:26:19.425387+0000","last_unstale":"2023-08-27T11:26:21.384761+0000","last_undegraded":"2023-08-27T11:26:21.384761+0000","last_fullsized":"2023-08-27T11:26:21.384761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:52.205981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384498+0000","last_change":"2023-08-27T11:26:18.426465+0000","last_active":"2023-08-27T11:26:21.384498+0000","last_peered":"2023-08-27T11:26:21.384498+0000","last_clean":"2023-08-27T11:26:21.384498+0000","last_became_active":"2023-08-27T11:26:18.426180+0000","last_became_peered":"2023-08-27T11:26:18.426180+0000","last_unstale":"2023-08-27T11:26:21.384498+0000","last_undegraded":"2023-08-27T11:26:21.384498+0000","last_fullsized":"2023-08-27T11:26:21.384498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:46.357284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385448+0000","last_change":"2023-08-27T11:26:19.433993+0000","last_active":"2023-08-27T11:26:21.385448+0000","last_peered":"2023-08-27T11:26:21.385448+0000","last_clean":"2023-08-27T11:26:21.385448+0000","last_became_active":"2023-08-27T11:26:19.433735+0000","last_became_peered":"2023-08-27T11:26:19.433735+0000","last_unstale":"2023-08-27T11:26:21.385448+0000","last_undegraded":"2023-08-27T11:26:21.385448+0000","last_fullsized":"2023-08-27T11:26:21.385448+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:44:55.647335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430463+0000","last_change":"2023-08-27T11:26:18.432789+0000","last_active":"2023-08-27T11:26:20.430463+0000","last_peered":"2023-08-27T11:26:20.430463+0000","last_clean":"2023-08-27T11:26:20.430463+0000","last_became_active":"2023-08-27T11:26:18.432521+0000","last_became_peered":"2023-08-27T11:26:18.432521+0000","last_unstale":"2023-08-27T11:26:20.430463+0000","last_undegraded":"2023-08-27T11:26:20.430463+0000","last_fullsized":"2023-08-27T11:26:20.430463+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:06.592671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385904+0000","last_change":"2023-08-27T11:26:19.434177+0000","last_active":"2023-08-27T11:26:21.385904+0000","last_peered":"2023-08-27T11:26:21.385904+0000","last_clean":"2023-08-27T11:26:21.385904+0000","last_became_active":"2023-08-27T11:26:19.432686+0000","last_became_peered":"2023-08-27T11:26:19.432686+0000","last_unstale":"2023-08-27T11:26:21.385904+0000","last_undegraded":"2023-08-27T11:26:21.385904+0000","last_fullsized":"2023-08-27T11:26:21.385904+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:34.644697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384455+0000","last_change":"2023-08-27T11:26:18.426963+0000","last_active":"2023-08-27T11:26:21.384455+0000","last_peered":"2023-08-27T11:26:21.384455+0000","last_clean":"2023-08-27T11:26:21.384455+0000","last_became_active":"2023-08-27T11:26:18.426752+0000","last_became_peered":"2023-08-27T11:26:18.426752+0000","last_unstale":"2023-08-27T11:26:21.384455+0000","last_undegraded":"2023-08-27T11:26:21.384455+0000","last_fullsized":"2023-08-27T11:26:21.384455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:29:35.149141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425190+0000","last_change":"2023-08-27T11:26:19.431940+0000","last_active":"2023-08-27T11:26:20.425190+0000","last_peered":"2023-08-27T11:26:20.425190+0000","last_clean":"2023-08-27T11:26:20.425190+0000","last_became_active":"2023-08-27T11:26:19.431631+0000","last_became_peered":"2023-08-27T11:26:19.431631+0000","last_unstale":"2023-08-27T11:26:20.425190+0000","last_undegraded":"2023-08-27T11:26:20.425190+0000","last_fullsized":"2023-08-27T11:26:20.425190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:37.504080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430548+0000","last_change":"2023-08-27T11:26:18.425472+0000","last_active":"2023-08-27T11:26:20.430548+0000","last_peered":"2023-08-27T11:26:20.430548+0000","last_clean":"2023-08-27T11:26:20.430548+0000","last_became_active":"2023-08-27T11:26:18.425150+0000","last_became_peered":"2023-08-27T11:26:18.425150+0000","last_unstale":"2023-08-27T11:26:20.430548+0000","last_undegraded":"2023-08-27T11:26:20.430548+0000","last_fullsized":"2023-08-27T11:26:20.430548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:10.608143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431200+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:26:20.431200+0000","last_peered":"2023-08-27T11:26:20.431200+0000","last_clean":"2023-08-27T11:26:20.431200+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:26:20.431200+0000","last_undegraded":"2023-08-27T11:26:20.431200+0000","last_fullsized":"2023-08-27T11:26:20.431200+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:53.465982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384795+0000","last_change":"2023-08-27T11:26:19.426312+0000","last_active":"2023-08-27T11:26:21.384795+0000","last_peered":"2023-08-27T11:26:21.384795+0000","last_clean":"2023-08-27T11:26:21.384795+0000","last_became_active":"2023-08-27T11:26:19.425963+0000","last_became_peered":"2023-08-27T11:26:19.425963+0000","last_unstale":"2023-08-27T11:26:21.384795+0000","last_undegraded":"2023-08-27T11:26:21.384795+0000","last_fullsized":"2023-08-27T11:26:21.384795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:46.854781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384674+0000","last_change":"2023-08-27T11:26:18.429275+0000","last_active":"2023-08-27T11:26:21.384674+0000","last_peered":"2023-08-27T11:26:21.384674+0000","last_clean":"2023-08-27T11:26:21.384674+0000","last_became_active":"2023-08-27T11:26:18.428994+0000","last_became_peered":"2023-08-27T11:26:18.428994+0000","last_unstale":"2023-08-27T11:26:21.384674+0000","last_undegraded":"2023-08-27T11:26:21.384674+0000","last_fullsized":"2023-08-27T11:26:21.384674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:55.212494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430413+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:26:20.430413+0000","last_peered":"2023-08-27T11:26:20.430413+0000","last_clean":"2023-08-27T11:26:20.430413+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:26:20.430413+0000","last_undegraded":"2023-08-27T11:26:20.430413+0000","last_fullsized":"2023-08-27T11:26:20.430413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:35:03.658156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384543+0000","last_change":"2023-08-27T11:26:19.434349+0000","last_active":"2023-08-27T11:26:21.384543+0000","last_peered":"2023-08-27T11:26:21.384543+0000","last_clean":"2023-08-27T11:26:21.384543+0000","last_became_active":"2023-08-27T11:26:19.434063+0000","last_became_peered":"2023-08-27T11:26:19.434063+0000","last_unstale":"2023-08-27T11:26:21.384543+0000","last_undegraded":"2023-08-27T11:26:21.384543+0000","last_fullsized":"2023-08-27T11:26:21.384543+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:15.835557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431167+0000","last_change":"2023-08-27T11:26:18.429449+0000","last_active":"2023-08-27T11:26:20.431167+0000","last_peered":"2023-08-27T11:26:20.431167+0000","last_clean":"2023-08-27T11:26:20.431167+0000","last_became_active":"2023-08-27T11:26:18.429134+0000","last_became_peered":"2023-08-27T11:26:18.429134+0000","last_unstale":"2023-08-27T11:26:20.431167+0000","last_undegraded":"2023-08-27T11:26:20.431167+0000","last_fullsized":"2023-08-27T11:26:20.431167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:47:49.142361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":141,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385196+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:26:21.385196+0000","last_peered":"2023-08-27T11:26:21.385196+0000","last_clean":"2023-08-27T11:26:21.385196+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:26:21.385196+0000","last_undegraded":"2023-08-27T11:26:21.385196+0000","last_fullsized":"2023-08-27T11:26:21.385196+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:17:04.262716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385249+0000","last_change":"2023-08-27T11:26:19.437248+0000","last_active":"2023-08-27T11:26:21.385249+0000","last_peered":"2023-08-27T11:26:21.385249+0000","last_clean":"2023-08-27T11:26:21.385249+0000","last_became_active":"2023-08-27T11:26:19.436894+0000","last_became_peered":"2023-08-27T11:26:19.436894+0000","last_unstale":"2023-08-27T11:26:21.385249+0000","last_undegraded":"2023-08-27T11:26:21.385249+0000","last_fullsized":"2023-08-27T11:26:21.385249+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:46:39.788991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384819+0000","last_change":"2023-08-27T11:26:18.429297+0000","last_active":"2023-08-27T11:26:21.384819+0000","last_peered":"2023-08-27T11:26:21.384819+0000","last_clean":"2023-08-27T11:26:21.384819+0000","last_became_active":"2023-08-27T11:26:18.429028+0000","last_became_peered":"2023-08-27T11:26:18.429028+0000","last_unstale":"2023-08-27T11:26:21.384819+0000","last_undegraded":"2023-08-27T11:26:21.384819+0000","last_fullsized":"2023-08-27T11:26:21.384819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:25:56.014027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":141,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385082+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:26:21.385082+0000","last_peered":"2023-08-27T11:26:21.385082+0000","last_clean":"2023-08-27T11:26:21.385082+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:26:21.385082+0000","last_undegraded":"2023-08-27T11:26:21.385082+0000","last_fullsized":"2023-08-27T11:26:21.385082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:55.131492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057479+0000","last_change":"2023-08-27T11:26:19.432826+0000","last_active":"2023-08-27T11:26:21.057479+0000","last_peered":"2023-08-27T11:26:21.057479+0000","last_clean":"2023-08-27T11:26:21.057479+0000","last_became_active":"2023-08-27T11:26:19.432562+0000","last_became_peered":"2023-08-27T11:26:19.432562+0000","last_unstale":"2023-08-27T11:26:21.057479+0000","last_undegraded":"2023-08-27T11:26:21.057479+0000","last_fullsized":"2023-08-27T11:26:21.057479+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:05.457562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385725+0000","last_change":"2023-08-27T11:26:18.425845+0000","last_active":"2023-08-27T11:26:21.385725+0000","last_peered":"2023-08-27T11:26:21.385725+0000","last_clean":"2023-08-27T11:26:21.385725+0000","last_became_active":"2023-08-27T11:26:18.425533+0000","last_became_peered":"2023-08-27T11:26:18.425533+0000","last_unstale":"2023-08-27T11:26:21.385725+0000","last_undegraded":"2023-08-27T11:26:21.385725+0000","last_fullsized":"2023-08-27T11:26:21.385725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:22:46.305412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":141,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385273+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:26:21.385273+0000","last_peered":"2023-08-27T11:26:21.385273+0000","last_clean":"2023-08-27T11:26:21.385273+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:26:21.385273+0000","last_undegraded":"2023-08-27T11:26:21.385273+0000","last_fullsized":"2023-08-27T11:26:21.385273+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:27:20.122317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430815+0000","last_change":"2023-08-27T11:26:19.427376+0000","last_active":"2023-08-27T11:26:20.430815+0000","last_peered":"2023-08-27T11:26:20.430815+0000","last_clean":"2023-08-27T11:26:20.430815+0000","last_became_active":"2023-08-27T11:26:19.426214+0000","last_became_peered":"2023-08-27T11:26:19.426214+0000","last_unstale":"2023-08-27T11:26:20.430815+0000","last_undegraded":"2023-08-27T11:26:20.430815+0000","last_fullsized":"2023-08-27T11:26:20.430815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:07:17.226611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385541+0000","last_change":"2023-08-27T11:26:18.426780+0000","last_active":"2023-08-27T11:26:21.385541+0000","last_peered":"2023-08-27T11:26:21.385541+0000","last_clean":"2023-08-27T11:26:21.385541+0000","last_became_active":"2023-08-27T11:26:18.426462+0000","last_became_peered":"2023-08-27T11:26:18.426462+0000","last_unstale":"2023-08-27T11:26:21.385541+0000","last_undegraded":"2023-08-27T11:26:21.385541+0000","last_fullsized":"2023-08-27T11:26:21.385541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:06:04.639068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":291,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384913+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:26:21.384913+0000","last_peered":"2023-08-27T11:26:21.384913+0000","last_clean":"2023-08-27T11:26:21.384913+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:26:21.384913+0000","last_undegraded":"2023-08-27T11:26:21.384913+0000","last_fullsized":"2023-08-27T11:26:21.384913+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:30:11.743649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057579+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:26:21.057579+0000","last_peered":"2023-08-27T11:26:21.057579+0000","last_clean":"2023-08-27T11:26:21.057579+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:26:21.057579+0000","last_undegraded":"2023-08-27T11:26:21.057579+0000","last_fullsized":"2023-08-27T11:26:21.057579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:12:51.448320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874474+0000","last_change":"2023-08-27T11:26:19.420539+0000","last_active":"2023-08-27T11:26:20.874474+0000","last_peered":"2023-08-27T11:26:20.874474+0000","last_clean":"2023-08-27T11:26:20.874474+0000","last_became_active":"2023-08-27T11:26:19.420291+0000","last_became_peered":"2023-08-27T11:26:19.420291+0000","last_unstale":"2023-08-27T11:26:20.874474+0000","last_undegraded":"2023-08-27T11:26:20.874474+0000","last_fullsized":"2023-08-27T11:26:20.874474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:23:25.892210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384424+0000","last_change":"2023-08-27T11:26:18.432770+0000","last_active":"2023-08-27T11:26:21.384424+0000","last_peered":"2023-08-27T11:26:21.384424+0000","last_clean":"2023-08-27T11:26:21.384424+0000","last_became_active":"2023-08-27T11:26:18.432552+0000","last_became_peered":"2023-08-27T11:26:18.432552+0000","last_unstale":"2023-08-27T11:26:21.384424+0000","last_undegraded":"2023-08-27T11:26:21.384424+0000","last_fullsized":"2023-08-27T11:26:21.384424+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:05:38.418817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":139,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425350+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:26:20.425350+0000","last_peered":"2023-08-27T11:26:20.425350+0000","last_clean":"2023-08-27T11:26:20.425350+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:26:20.425350+0000","last_undegraded":"2023-08-27T11:26:20.425350+0000","last_fullsized":"2023-08-27T11:26:20.425350+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:38:27.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874280+0000","last_change":"2023-08-27T11:26:19.427286+0000","last_active":"2023-08-27T11:26:20.874280+0000","last_peered":"2023-08-27T11:26:20.874280+0000","last_clean":"2023-08-27T11:26:20.874280+0000","last_became_active":"2023-08-27T11:26:19.426940+0000","last_became_peered":"2023-08-27T11:26:19.426940+0000","last_unstale":"2023-08-27T11:26:20.874280+0000","last_undegraded":"2023-08-27T11:26:20.874280+0000","last_fullsized":"2023-08-27T11:26:20.874280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:31.231234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384817+0000","last_change":"2023-08-27T11:26:18.415885+0000","last_active":"2023-08-27T11:26:21.384817+0000","last_peered":"2023-08-27T11:26:21.384817+0000","last_clean":"2023-08-27T11:26:21.384817+0000","last_became_active":"2023-08-27T11:26:18.415623+0000","last_became_peered":"2023-08-27T11:26:18.415623+0000","last_unstale":"2023-08-27T11:26:21.384817+0000","last_undegraded":"2023-08-27T11:26:21.384817+0000","last_fullsized":"2023-08-27T11:26:21.384817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:10:07.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430427+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:26:20.430427+0000","last_peered":"2023-08-27T11:26:20.430427+0000","last_clean":"2023-08-27T11:26:20.430427+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:26:20.430427+0000","last_undegraded":"2023-08-27T11:26:20.430427+0000","last_fullsized":"2023-08-27T11:26:20.430427+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:43:08.686044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":1103318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.47699999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.55000000000000004,"15min":0.50800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":0.77500000000000002,"15min":0.872},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.253},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.53800000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.48699999999999999},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.435},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.33800000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.90200000000000002},"last":0.58299999999999996}]},{"osd":3,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.45900000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.505},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.72699999999999998,"5min":0.72799999999999998,"15min":0.77400000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.49099999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51200000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.62}]}]},{"osd":6,"up_from":13,"seq":55834574966,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1134592,"data_stored":515006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51500000000000001,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.29099999999999998},"max":{"1min":0.626,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67300000000000004,"5min":0.86199999999999999,"15min":0.89800000000000002},"last":0.51100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.316,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.80600000000000005,"15min":0.873},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.62,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61299999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58299999999999996,"5min":0.74199999999999999,"15min":0.85899999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.65200000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.58199999999999996,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.70699999999999996,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.65200000000000002,"5min":0.84699999999999998,"15min":0.93799999999999994},"last":0.63400000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.70699999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.374},"max":{"1min":0.77800000000000002,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.72599999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.624,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.78100000000000003,"5min":0.875,"15min":0.875},"last":0.74199999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.748,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.34899999999999998},"max":{"1min":0.76700000000000002,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.77500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1712128,"data_stored":1103192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.308,"15min":0.59299999999999997},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.45800000000000002,"5min":0.502,"15min":11.971},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.89000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.51700000000000002,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.26800000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.625},"min":{"1min":0.23400000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.495,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.34300000000000003},"min":{"1min":0.246,"5min":0.224,"15min":0.224},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.58599999999999997},"last":0.28799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.64100000000000001},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":10.99},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.373,"15min":0.92500000000000004},"min":{"1min":0.318,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.54700000000000004,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.36099999999999999,"15min":0.33400000000000002},"max":{"1min":0.76900000000000002,"5min":0.90700000000000003,"15min":1.26},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.80700000000000005},"min":{"1min":0.434,"5min":0.37,"15min":0.29099999999999998},"max":{"1min":0.64500000000000002,"5min":0.77600000000000002,"15min":10.957000000000001},"last":0.61299999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":1.121},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.36199999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71499999999999997,"5min":0.88700000000000001,"15min":1.296},"last":0.624}]},{"osd":6,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.84299999999999997},"min":{"1min":0.44800000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":1.1379999999999999},"min":{"1min":0.435,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.74299999999999999,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.53900000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":1.1319999999999999},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84199999999999997,"15min":22.988},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.63500000000000001},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":1.327},"last":0.59899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":513131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.29699999999999999,"15min":0.311},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.33200000000000002,"15min":0.34999999999999998},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.27000000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.24399999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.628,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.33200000000000002,"15min":0.33900000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.374}]},{"osd":3,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.33700000000000002,"15min":0.34599999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.502,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.27500000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.54200000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.41499999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79100000000000004,"5min":0.92500000000000004,"15min":1.214},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.84299999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.77500000000000002,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85399999999999998,"5min":0.90200000000000002,"15min":1.603},"last":0.628}]},{"osd":6,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.83099999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.74099999999999999,"5min":0.94199999999999995,"15min":1.383},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82699999999999996,"5min":0.88900000000000001,"15min":0.92000000000000004},"last":0.78900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1118208,"data_stored":512824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29799999999999999,"15min":0.30199999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40500000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.314,"5min":0.316,"15min":0.317},"min":{"1min":0.22700000000000001,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.629},"last":0.26100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.317,"15min":0.32400000000000001},"min":{"1min":0.222,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.36499999999999999,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33800000000000002,"15min":0.34599999999999997},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47799999999999998,"5min":0.64200000000000002,"15min":0.749},"last":0.28299999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36099999999999999,"15min":0.38200000000000001},"min":{"1min":0.318,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41599999999999998,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.313},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.376,"15min":0.39800000000000002},"min":{"1min":0.30599999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.69499999999999995},"last":0.374}]},{"osd":4,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.68600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.433,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.76100000000000001,"5min":0.78400000000000003,"15min":0.879},"last":0.40799999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70699999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.40999999999999998,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.73399999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.495}]},{"osd":6,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77200000000000002,"5min":0.82899999999999996,"15min":0.85299999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.59399999999999997},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.65900000000000003,"5min":0.80500000000000005,"15min":0.83599999999999997},"last":0.54400000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.60899999999999999},"min":{"1min":0.438,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574966,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1118208,"data_stored":513372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29699999999999999,"15min":0.29999999999999999},"min":{"1min":0.21299999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.373},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32100000000000001,"15min":0.312},"min":{"1min":0.23100000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.38400000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.29899999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32300000000000001,"15min":0.313},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.19500000000000001},"max":{"1min":0.46700000000000003,"5min":0.50600000000000001,"15min":0.60399999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":0.44900000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"last":0.32100000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.28499999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.53300000000000003,"5min":0.57199999999999995,"15min":0.57599999999999996},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.52500000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.35399999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.28599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.503}]},{"osd":5,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.371,"15min":0.314},"max":{"1min":0.79100000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57899999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.373},"max":{"1min":0.80400000000000005,"5min":1.091,"15min":1.091},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.61199999999999999},"min":{"1min":0.41699999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78400000000000003,"5min":1.111,"15min":1.111},"last":0.624}]},{"osd":7,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.61199999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74399999999999999,"5min":1.032,"15min":1.236},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.61299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1122304,"data_stored":512882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.755,"5min":0.997,"15min":0.997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.68700000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40500000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.498},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.752,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52800000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.51000000000000001},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38400000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54300000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.85899999999999999},"last":0.438},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75700000000000001,"5min":0.85999999999999999,"15min":0.879},"last":0.55700000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70599999999999996,"5min":0.78900000000000003,"15min":0.90600000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59599999999999997,"15min":0.623},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.92300000000000004},"last":0.57399999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.80600000000000005,"15min":0.82499999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.89300000000000002},"last":0.36099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":512790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.69699999999999995,"5min":0.78600000000000003,"15min":0.96899999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.64500000000000002,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.42299999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.54200000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66700000000000004,"5min":0.72899999999999998,"15min":1.075},"last":0.48399999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.54800000000000004},"min":{"1min":0.36399999999999999,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.68899999999999995,"5min":0.78000000000000003,"15min":0.95699999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54100000000000004,"15min":0.55700000000000005},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.67600000000000005,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.44900000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.56299999999999994,"15min":0.58099999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73399999999999999,"5min":0.81000000000000005,"15min":1.03},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.35199999999999998},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":1.026},"last":0.51300000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.31,"15min":0.27000000000000002},"max":{"1min":0.72799999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.435},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.73399999999999999,"5min":0.86399999999999999,"15min":0.91400000000000003},"last":0.46000000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59199999999999997,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.28799999999999998},"max":{"1min":0.77700000000000002,"5min":0.84799999999999998,"15min":0.94199999999999995},"last":0.53700000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.61499999999999999,"15min":0.629},"min":{"1min":0.432,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.63900000000000001,"5min":0.90700000000000003,"15min":0.95899999999999996},"last":0.501},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.31},"max":{"1min":0.69399999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.54800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:26:28.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:26:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.210+0000 7f69c2bb0700 1 Processor -- start 2023-08-27T11:26:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.210+0000 7f69c2bb0700 1 -- start start 2023-08-27T11:26:28.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69c2bb0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f69bc15a120 0x7f69bc0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69c2bb0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69bc06d5b0 0x7f69bc06da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69c2bb0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69bc06df50 con 0x7f69bc0b3440 2023-08-27T11:26:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69c2bb0700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69bc15b540 con 0x7f69bc15a120 2023-08-27T11:26:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69c2bb0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc0af830 con 0x7f69bc06d5b0 2023-08-27T11:26:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69c114d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f69bc0b3440 0x7f69bc0b38b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55586/0 (socket says 172.21.15.112:55586) 2023-08-27T11:26:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69c114d700 1 -- 172.21.15.112:0/3269266640 learned_addr learned my addr 172.21.15.112:0/3269266640 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69bbfff700 1 --2- 172.21.15.112:0/3269266640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69bc06d5b0 0x7f69bc06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69c094c700 1 --2- 172.21.15.112:0/3269266640 >> v2:172.21.15.138:3300/0 conn(0x7f69bc15a120 0x7f69bc0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69bb7fe700 1 -- 172.21.15.112:0/3269266640 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496996645 0 0) 0x7f69bc06df50 con 0x7f69bc0b3440 2023-08-27T11:26:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69bb7fe700 1 -- 172.21.15.112:0/3269266640 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69ac0035b0 con 0x7f69bc0b3440 2023-08-27T11:26:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69c094c700 1 -- 172.21.15.112:0/3269266640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69bc06d5b0 msgr2=0x7f69bc06da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69c094c700 1 --2- 172.21.15.112:0/3269266640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69bc06d5b0 0x7f69bc06da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69c094c700 1 -- 172.21.15.112:0/3269266640 >> v1:172.21.15.112:6789/0 conn(0x7f69bc0b3440 legacy=0x7f69bc0b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69c094c700 1 -- 172.21.15.112:0/3269266640 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69bc0afb10 con 0x7f69bc15a120 2023-08-27T11:26:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.211+0000 7f69bbfff700 1 --2- 172.21.15.112:0/3269266640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69bc06d5b0 0x7f69bc06da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:28.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.212+0000 7f69c094c700 1 --2- 172.21.15.112:0/3269266640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69bc15a120 0x7f69bc0b05f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009840 tx=0x7f69b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f2231837f3b0949 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:28.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.212+0000 7f69bb7fe700 1 -- 172.21.15.112:0/3269266640 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0011830 con 0x7f69bc15a120 2023-08-27T11:26:28.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.212+0000 7f69bb7fe700 1 -- 172.21.15.112:0/3269266640 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b0011990 con 0x7f69bc15a120 2023-08-27T11:26:28.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.212+0000 7f69bb7fe700 1 -- 172.21.15.112:0/3269266640 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0011d20 con 0x7f69bc15a120 2023-08-27T11:26:28.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.212+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3269266640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69bc15a120 msgr2=0x7f69bc0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.212+0000 7f69c2bb0700 1 --2- 172.21.15.112:0/3269266640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69bc15a120 0x7f69bc0b05f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009840 tx=0x7f69b0009b50 comp rx=0 tx=0).stop 2023-08-27T11:26:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.212+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3269266640 shutdown_connections 2023-08-27T11:26:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.212+0000 7f69c2bb0700 1 --2- 172.21.15.112:0/3269266640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69bc15a120 0x7f69bc0b05f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.212+0000 7f69c2bb0700 1 --2- 172.21.15.112:0/3269266640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69bc06d5b0 0x7f69bc06da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.212+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3269266640 >> 172.21.15.112:0/3269266640 conn(0x7f69bc0cff80 msgr2=0x7f69bc0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:28.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.212+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3269266640 shutdown_connections 2023-08-27T11:26:28.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.212+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3269266640 wait complete. 2023-08-27T11:26:28.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.214+0000 7f69c2bb0700 1 Processor -- start 2023-08-27T11:26:28.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.214+0000 7f69c2bb0700 1 -- start start 2023-08-27T11:26:28.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.214+0000 7f69c2bb0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69bc06d5b0 0x7f69bc071890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.214+0000 7f69c2bb0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69bc0b3440 0x7f69bc1538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.214+0000 7f69c2bb0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69bc15a120 0x7f69bc153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c2bb0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc071400 con 0x7f69bc15a120 2023-08-27T11:26:28.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c2bb0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc0707f0 con 0x7f69bc06d5b0 2023-08-27T11:26:28.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c2bb0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc070ad0 con 0x7f69bc0b3440 2023-08-27T11:26:28.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c114d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69bc15a120 0x7f69bc153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c114d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69bc15a120 0x7f69bc153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32978/0 (socket says 172.21.15.112:32978) 2023-08-27T11:26:28.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c114d700 1 -- 172.21.15.112:0/3370339111 learned_addr learned my addr 172.21.15.112:0/3370339111 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:28.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69bbfff700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69bc0b3440 0x7f69bc1538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c094c700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69bc06d5b0 0x7f69bc071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c114d700 1 -- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69bc0b3440 msgr2=0x7f69bc1538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:28.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c114d700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69bc0b3440 0x7f69bc1538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c114d700 1 -- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69bc06d5b0 msgr2=0x7f69bc071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:28.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c114d700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69bc06d5b0 0x7f69bc071890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c114d700 1 -- 172.21.15.112:0/3370339111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a80094a0 con 0x7f69bc15a120 2023-08-27T11:26:28.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c114d700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69bc15a120 0x7f69bc153e00 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f69b400cb40 tx=0x7f69b400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69b97fa700 1 -- 172.21.15.112:0/3370339111 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b400ab50 con 0x7f69bc15a120 2023-08-27T11:26:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69b97fa700 1 -- 172.21.15.112:0/3370339111 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b400acb0 con 0x7f69bc15a120 2023-08-27T11:26:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3370339111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b00094a0 con 0x7f69bc15a120 2023-08-27T11:26:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69b97fa700 1 -- 172.21.15.112:0/3370339111 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4012620 con 0x7f69bc15a120 2023-08-27T11:26:28.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.215+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3370339111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69bc071090 con 0x7f69bc15a120 2023-08-27T11:26:28.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.216+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3370339111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69bc071570 con 0x7f69bc15a120 2023-08-27T11:26:28.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.217+0000 7f69b97fa700 1 -- 172.21.15.112:0/3370339111 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f69b401b3b0 con 0x7f69bc15a120 2023-08-27T11:26:28.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.217+0000 7f69b97fa700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f69a006a3b0 0x7f69a006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.217+0000 7f69b97fa700 1 -- 172.21.15.112:0/3370339111 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f69b4088770 con 0x7f69bc15a120 2023-08-27T11:26:28.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.217+0000 7f69c094c700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f69a006a3b0 0x7f69a006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.218+0000 7f69c094c700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f69a006a3b0 0x7f69a006c870 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f69b0000c00 tx=0x7f69b001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:28.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.221+0000 7f69b97fa700 1 -- 172.21.15.112:0/3370339111 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f69b4053b10 con 0x7f69bc15a120 2023-08-27T11:26:28.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.407+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3370339111 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69bc0b00d0 con 0x7f69a006a3b0 2023-08-27T11:26:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.416+0000 7f69b97fa700 1 -- 172.21.15.112:0/3370339111 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311872 (secure 0 0 0) 0x7f69b0011b00 con 0x7f69a006a3b0 2023-08-27T11:26:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.419+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f69a006a3b0 msgr2=0x7f69a006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.419+0000 7f69c2bb0700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f69a006a3b0 0x7f69a006c870 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f69b0000c00 tx=0x7f69b001c040 comp rx=0 tx=0).stop 2023-08-27T11:26:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.419+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3370339111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69bc15a120 msgr2=0x7f69bc153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.419+0000 7f69c2bb0700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69bc15a120 0x7f69bc153e00 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f69b400cb40 tx=0x7f69b400cf00 comp rx=0 tx=0).stop 2023-08-27T11:26:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.419+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3370339111 shutdown_connections 2023-08-27T11:26:28.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.419+0000 7f69c2bb0700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f69a006a3b0 0x7f69a006c870 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.419+0000 7f69c2bb0700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69bc15a120 0x7f69bc153e00 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.419+0000 7f69c2bb0700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69bc06d5b0 0x7f69bc071890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.419+0000 7f69c2bb0700 1 --2- 172.21.15.112:0/3370339111 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69bc0b3440 0x7f69bc1538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.419+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3370339111 >> 172.21.15.112:0/3370339111 conn(0x7f69bc0cff80 msgr2=0x7f69bc0b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:28.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.419+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3370339111 shutdown_connections 2023-08-27T11:26:28.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.419+0000 7f69c2bb0700 1 -- 172.21.15.112:0/3370339111 wait complete. 2023-08-27T11:26:28.426 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:26:28.433 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2023-08-27T11:26:28.370172+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259388,"kb_used_data":9924,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754715332,"statfs":{"total":773094113280,"available":772828499968,"internally_reserved":0,"allocated":10162176,"data_stored":5286515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.959518"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874566+0000","last_change":"2023-08-27T11:26:19.427207+0000","last_active":"2023-08-27T11:26:20.874566+0000","last_peered":"2023-08-27T11:26:20.874566+0000","last_clean":"2023-08-27T11:26:20.874566+0000","last_became_active":"2023-08-27T11:26:19.426963+0000","last_became_peered":"2023-08-27T11:26:19.426963+0000","last_unstale":"2023-08-27T11:26:20.874566+0000","last_undegraded":"2023-08-27T11:26:20.874566+0000","last_fullsized":"2023-08-27T11:26:20.874566+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:55:21.733240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430931+0000","last_change":"2023-08-27T11:26:19.431066+0000","last_active":"2023-08-27T11:26:20.430931+0000","last_peered":"2023-08-27T11:26:20.430931+0000","last_clean":"2023-08-27T11:26:20.430931+0000","last_became_active":"2023-08-27T11:26:19.430477+0000","last_became_peered":"2023-08-27T11:26:19.430477+0000","last_unstale":"2023-08-27T11:26:20.430931+0000","last_undegraded":"2023-08-27T11:26:20.430931+0000","last_fullsized":"2023-08-27T11:26:20.430931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:43:44.337358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874230+0000","last_change":"2023-08-27T11:26:19.434887+0000","last_active":"2023-08-27T11:26:20.874230+0000","last_peered":"2023-08-27T11:26:20.874230+0000","last_clean":"2023-08-27T11:26:20.874230+0000","last_became_active":"2023-08-27T11:26:19.434566+0000","last_became_peered":"2023-08-27T11:26:19.434566+0000","last_unstale":"2023-08-27T11:26:20.874230+0000","last_undegraded":"2023-08-27T11:26:20.874230+0000","last_fullsized":"2023-08-27T11:26:20.874230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:11:11.738619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425453+0000","last_change":"2023-08-27T11:26:19.426786+0000","last_active":"2023-08-27T11:26:20.425453+0000","last_peered":"2023-08-27T11:26:20.425453+0000","last_clean":"2023-08-27T11:26:20.425453+0000","last_became_active":"2023-08-27T11:26:19.426494+0000","last_became_peered":"2023-08-27T11:26:19.426494+0000","last_unstale":"2023-08-27T11:26:20.425453+0000","last_undegraded":"2023-08-27T11:26:20.425453+0000","last_fullsized":"2023-08-27T11:26:20.425453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:33:53.249169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057365+0000","last_change":"2023-08-27T11:26:19.424084+0000","last_active":"2023-08-27T11:26:21.057365+0000","last_peered":"2023-08-27T11:26:21.057365+0000","last_clean":"2023-08-27T11:26:21.057365+0000","last_became_active":"2023-08-27T11:26:19.422955+0000","last_became_peered":"2023-08-27T11:26:19.422955+0000","last_unstale":"2023-08-27T11:26:21.057365+0000","last_undegraded":"2023-08-27T11:26:21.057365+0000","last_fullsized":"2023-08-27T11:26:21.057365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:59:24.896402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430830+0000","last_change":"2023-08-27T11:26:19.424659+0000","last_active":"2023-08-27T11:26:20.430830+0000","last_peered":"2023-08-27T11:26:20.430830+0000","last_clean":"2023-08-27T11:26:20.430830+0000","last_became_active":"2023-08-27T11:26:19.424317+0000","last_became_peered":"2023-08-27T11:26:19.424317+0000","last_unstale":"2023-08-27T11:26:20.430830+0000","last_undegraded":"2023-08-27T11:26:20.430830+0000","last_fullsized":"2023-08-27T11:26:20.430830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:47:42.857298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430038+0000","last_change":"2023-08-27T11:26:19.421667+0000","last_active":"2023-08-27T11:26:20.430038+0000","last_peered":"2023-08-27T11:26:20.430038+0000","last_clean":"2023-08-27T11:26:20.430038+0000","last_became_active":"2023-08-27T11:26:19.421352+0000","last_became_peered":"2023-08-27T11:26:19.421352+0000","last_unstale":"2023-08-27T11:26:20.430038+0000","last_undegraded":"2023-08-27T11:26:20.430038+0000","last_fullsized":"2023-08-27T11:26:20.430038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:24.356051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384297+0000","last_change":"2023-08-27T11:26:19.430958+0000","last_active":"2023-08-27T11:26:21.384297+0000","last_peered":"2023-08-27T11:26:21.384297+0000","last_clean":"2023-08-27T11:26:21.384297+0000","last_became_active":"2023-08-27T11:26:19.430685+0000","last_became_peered":"2023-08-27T11:26:19.430685+0000","last_unstale":"2023-08-27T11:26:21.384297+0000","last_undegraded":"2023-08-27T11:26:21.384297+0000","last_fullsized":"2023-08-27T11:26:21.384297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:41.805565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431121+0000","last_change":"2023-08-27T11:26:19.430211+0000","last_active":"2023-08-27T11:26:20.431121+0000","last_peered":"2023-08-27T11:26:20.431121+0000","last_clean":"2023-08-27T11:26:20.431121+0000","last_became_active":"2023-08-27T11:26:19.429921+0000","last_became_peered":"2023-08-27T11:26:19.429921+0000","last_unstale":"2023-08-27T11:26:20.431121+0000","last_undegraded":"2023-08-27T11:26:20.431121+0000","last_fullsized":"2023-08-27T11:26:20.431121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:56:34.526955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057756+0000","last_change":"2023-08-27T11:26:19.428983+0000","last_active":"2023-08-27T11:26:21.057756+0000","last_peered":"2023-08-27T11:26:21.057756+0000","last_clean":"2023-08-27T11:26:21.057756+0000","last_became_active":"2023-08-27T11:26:19.428699+0000","last_became_peered":"2023-08-27T11:26:19.428699+0000","last_unstale":"2023-08-27T11:26:21.057756+0000","last_undegraded":"2023-08-27T11:26:21.057756+0000","last_fullsized":"2023-08-27T11:26:21.057756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:49:32.779304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.386528+0000","last_change":"2023-08-27T11:26:19.433591+0000","last_active":"2023-08-27T11:26:21.386528+0000","last_peered":"2023-08-27T11:26:21.386528+0000","last_clean":"2023-08-27T11:26:21.386528+0000","last_became_active":"2023-08-27T11:26:19.433328+0000","last_became_peered":"2023-08-27T11:26:19.433328+0000","last_unstale":"2023-08-27T11:26:21.386528+0000","last_undegraded":"2023-08-27T11:26:21.386528+0000","last_fullsized":"2023-08-27T11:26:21.386528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:30.417469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430987+0000","last_change":"2023-08-27T11:26:19.428345+0000","last_active":"2023-08-27T11:26:20.430987+0000","last_peered":"2023-08-27T11:26:20.430987+0000","last_clean":"2023-08-27T11:26:20.430987+0000","last_became_active":"2023-08-27T11:26:19.427975+0000","last_became_peered":"2023-08-27T11:26:19.427975+0000","last_unstale":"2023-08-27T11:26:20.430987+0000","last_undegraded":"2023-08-27T11:26:20.430987+0000","last_fullsized":"2023-08-27T11:26:20.430987+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:08:01.417089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425155+0000","last_change":"2023-08-27T11:26:19.436540+0000","last_active":"2023-08-27T11:26:20.425155+0000","last_peered":"2023-08-27T11:26:20.425155+0000","last_clean":"2023-08-27T11:26:20.425155+0000","last_became_active":"2023-08-27T11:26:19.436278+0000","last_became_peered":"2023-08-27T11:26:19.436278+0000","last_unstale":"2023-08-27T11:26:20.425155+0000","last_undegraded":"2023-08-27T11:26:20.425155+0000","last_fullsized":"2023-08-27T11:26:20.425155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:40:05.207374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385238+0000","last_change":"2023-08-27T11:26:19.429289+0000","last_active":"2023-08-27T11:26:21.385238+0000","last_peered":"2023-08-27T11:26:21.385238+0000","last_clean":"2023-08-27T11:26:21.385238+0000","last_became_active":"2023-08-27T11:26:19.429019+0000","last_became_peered":"2023-08-27T11:26:19.429019+0000","last_unstale":"2023-08-27T11:26:21.385238+0000","last_undegraded":"2023-08-27T11:26:21.385238+0000","last_fullsized":"2023-08-27T11:26:21.385238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:02.295653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384991+0000","last_change":"2023-08-27T11:26:19.437669+0000","last_active":"2023-08-27T11:26:21.384991+0000","last_peered":"2023-08-27T11:26:21.384991+0000","last_clean":"2023-08-27T11:26:21.384991+0000","last_became_active":"2023-08-27T11:26:19.437024+0000","last_became_peered":"2023-08-27T11:26:19.437024+0000","last_unstale":"2023-08-27T11:26:21.384991+0000","last_undegraded":"2023-08-27T11:26:21.384991+0000","last_fullsized":"2023-08-27T11:26:21.384991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:51.492355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385006+0000","last_change":"2023-08-27T11:26:19.429490+0000","last_active":"2023-08-27T11:26:21.385006+0000","last_peered":"2023-08-27T11:26:21.385006+0000","last_clean":"2023-08-27T11:26:21.385006+0000","last_became_active":"2023-08-27T11:26:19.429177+0000","last_became_peered":"2023-08-27T11:26:19.429177+0000","last_unstale":"2023-08-27T11:26:21.385006+0000","last_undegraded":"2023-08-27T11:26:21.385006+0000","last_fullsized":"2023-08-27T11:26:21.385006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:08:53.141523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874577+0000","last_change":"2023-08-27T11:26:19.434923+0000","last_active":"2023-08-27T11:26:20.874577+0000","last_peered":"2023-08-27T11:26:20.874577+0000","last_clean":"2023-08-27T11:26:20.874577+0000","last_became_active":"2023-08-27T11:26:19.434650+0000","last_became_peered":"2023-08-27T11:26:19.434650+0000","last_unstale":"2023-08-27T11:26:20.874577+0000","last_undegraded":"2023-08-27T11:26:20.874577+0000","last_fullsized":"2023-08-27T11:26:20.874577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:33:06.493994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430846+0000","last_change":"2023-08-27T11:26:19.422300+0000","last_active":"2023-08-27T11:26:20.430846+0000","last_peered":"2023-08-27T11:26:20.430846+0000","last_clean":"2023-08-27T11:26:20.430846+0000","last_became_active":"2023-08-27T11:26:19.421284+0000","last_became_peered":"2023-08-27T11:26:19.421284+0000","last_unstale":"2023-08-27T11:26:20.430846+0000","last_undegraded":"2023-08-27T11:26:20.430846+0000","last_fullsized":"2023-08-27T11:26:20.430846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:12.363650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874425+0000","last_change":"2023-08-27T11:26:19.418470+0000","last_active":"2023-08-27T11:26:20.874425+0000","last_peered":"2023-08-27T11:26:20.874425+0000","last_clean":"2023-08-27T11:26:20.874425+0000","last_became_active":"2023-08-27T11:26:19.418192+0000","last_became_peered":"2023-08-27T11:26:19.418192+0000","last_unstale":"2023-08-27T11:26:20.874425+0000","last_undegraded":"2023-08-27T11:26:20.874425+0000","last_fullsized":"2023-08-27T11:26:20.874425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:14:53.959765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430870+0000","last_change":"2023-08-27T11:26:19.431041+0000","last_active":"2023-08-27T11:26:20.430870+0000","last_peered":"2023-08-27T11:26:20.430870+0000","last_clean":"2023-08-27T11:26:20.430870+0000","last_became_active":"2023-08-27T11:26:19.430750+0000","last_became_peered":"2023-08-27T11:26:19.430750+0000","last_unstale":"2023-08-27T11:26:20.430870+0000","last_undegraded":"2023-08-27T11:26:20.430870+0000","last_fullsized":"2023-08-27T11:26:20.430870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:17:25.874882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384767+0000","last_change":"2023-08-27T11:26:19.433627+0000","last_active":"2023-08-27T11:26:21.384767+0000","last_peered":"2023-08-27T11:26:21.384767+0000","last_clean":"2023-08-27T11:26:21.384767+0000","last_became_active":"2023-08-27T11:26:19.433370+0000","last_became_peered":"2023-08-27T11:26:19.433370+0000","last_unstale":"2023-08-27T11:26:21.384767+0000","last_undegraded":"2023-08-27T11:26:21.384767+0000","last_fullsized":"2023-08-27T11:26:21.384767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:41.696204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057432+0000","last_change":"2023-08-27T11:26:19.433519+0000","last_active":"2023-08-27T11:26:21.057432+0000","last_peered":"2023-08-27T11:26:21.057432+0000","last_clean":"2023-08-27T11:26:21.057432+0000","last_became_active":"2023-08-27T11:26:19.433245+0000","last_became_peered":"2023-08-27T11:26:19.433245+0000","last_unstale":"2023-08-27T11:26:21.057432+0000","last_undegraded":"2023-08-27T11:26:21.057432+0000","last_fullsized":"2023-08-27T11:26:21.057432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:07:00.066813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874575+0000","last_change":"2023-08-27T11:26:19.434762+0000","last_active":"2023-08-27T11:26:20.874575+0000","last_peered":"2023-08-27T11:26:20.874575+0000","last_clean":"2023-08-27T11:26:20.874575+0000","last_became_active":"2023-08-27T11:26:19.434462+0000","last_became_peered":"2023-08-27T11:26:19.434462+0000","last_unstale":"2023-08-27T11:26:20.874575+0000","last_undegraded":"2023-08-27T11:26:20.874575+0000","last_fullsized":"2023-08-27T11:26:20.874575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:41.356424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057290+0000","last_change":"2023-08-27T11:26:19.423833+0000","last_active":"2023-08-27T11:26:21.057290+0000","last_peered":"2023-08-27T11:26:21.057290+0000","last_clean":"2023-08-27T11:26:21.057290+0000","last_became_active":"2023-08-27T11:26:19.423440+0000","last_became_peered":"2023-08-27T11:26:19.423440+0000","last_unstale":"2023-08-27T11:26:21.057290+0000","last_undegraded":"2023-08-27T11:26:21.057290+0000","last_fullsized":"2023-08-27T11:26:21.057290+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:41:13.840925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431105+0000","last_change":"2023-08-27T11:26:19.434228+0000","last_active":"2023-08-27T11:26:20.431105+0000","last_peered":"2023-08-27T11:26:20.431105+0000","last_clean":"2023-08-27T11:26:20.431105+0000","last_became_active":"2023-08-27T11:26:19.433958+0000","last_became_peered":"2023-08-27T11:26:19.433958+0000","last_unstale":"2023-08-27T11:26:20.431105+0000","last_undegraded":"2023-08-27T11:26:20.431105+0000","last_fullsized":"2023-08-27T11:26:20.431105+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:25:08.328313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431061+0000","last_change":"2023-08-27T11:26:19.432242+0000","last_active":"2023-08-27T11:26:20.431061+0000","last_peered":"2023-08-27T11:26:20.431061+0000","last_clean":"2023-08-27T11:26:20.431061+0000","last_became_active":"2023-08-27T11:26:19.432006+0000","last_became_peered":"2023-08-27T11:26:19.432006+0000","last_unstale":"2023-08-27T11:26:20.431061+0000","last_undegraded":"2023-08-27T11:26:20.431061+0000","last_fullsized":"2023-08-27T11:26:20.431061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:47:39.278669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385953+0000","last_change":"2023-08-27T11:26:19.432982+0000","last_active":"2023-08-27T11:26:21.385953+0000","last_peered":"2023-08-27T11:26:21.385953+0000","last_clean":"2023-08-27T11:26:21.385953+0000","last_became_active":"2023-08-27T11:26:19.432668+0000","last_became_peered":"2023-08-27T11:26:19.432668+0000","last_unstale":"2023-08-27T11:26:21.385953+0000","last_undegraded":"2023-08-27T11:26:21.385953+0000","last_fullsized":"2023-08-27T11:26:21.385953+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:55.969633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430610+0000","last_change":"2023-08-27T11:26:19.433762+0000","last_active":"2023-08-27T11:26:20.430610+0000","last_peered":"2023-08-27T11:26:20.430610+0000","last_clean":"2023-08-27T11:26:20.430610+0000","last_became_active":"2023-08-27T11:26:19.433489+0000","last_became_peered":"2023-08-27T11:26:19.433489+0000","last_unstale":"2023-08-27T11:26:20.430610+0000","last_undegraded":"2023-08-27T11:26:20.430610+0000","last_fullsized":"2023-08-27T11:26:20.430610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:34.644697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384668+0000","last_change":"2023-08-27T11:26:19.431428+0000","last_active":"2023-08-27T11:26:21.384668+0000","last_peered":"2023-08-27T11:26:21.384668+0000","last_clean":"2023-08-27T11:26:21.384668+0000","last_became_active":"2023-08-27T11:26:19.431169+0000","last_became_peered":"2023-08-27T11:26:19.431169+0000","last_unstale":"2023-08-27T11:26:21.384668+0000","last_undegraded":"2023-08-27T11:26:21.384668+0000","last_fullsized":"2023-08-27T11:26:21.384668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:30:35.082113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430552+0000","last_change":"2023-08-27T11:26:19.435080+0000","last_active":"2023-08-27T11:26:20.430552+0000","last_peered":"2023-08-27T11:26:20.430552+0000","last_clean":"2023-08-27T11:26:20.430552+0000","last_became_active":"2023-08-27T11:26:19.434778+0000","last_became_peered":"2023-08-27T11:26:19.434778+0000","last_unstale":"2023-08-27T11:26:20.430552+0000","last_undegraded":"2023-08-27T11:26:20.430552+0000","last_fullsized":"2023-08-27T11:26:20.430552+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:27:49.502862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430321+0000","last_change":"2023-08-27T11:26:19.427535+0000","last_active":"2023-08-27T11:26:20.430321+0000","last_peered":"2023-08-27T11:26:20.430321+0000","last_clean":"2023-08-27T11:26:20.430321+0000","last_became_active":"2023-08-27T11:26:19.426191+0000","last_became_peered":"2023-08-27T11:26:19.426191+0000","last_unstale":"2023-08-27T11:26:20.430321+0000","last_undegraded":"2023-08-27T11:26:20.430321+0000","last_fullsized":"2023-08-27T11:26:20.430321+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:22:47.314202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384374+0000","last_change":"2023-08-27T11:26:19.424642+0000","last_active":"2023-08-27T11:26:21.384374+0000","last_peered":"2023-08-27T11:26:21.384374+0000","last_clean":"2023-08-27T11:26:21.384374+0000","last_became_active":"2023-08-27T11:26:19.424359+0000","last_became_peered":"2023-08-27T11:26:19.424359+0000","last_unstale":"2023-08-27T11:26:21.384374+0000","last_undegraded":"2023-08-27T11:26:21.384374+0000","last_fullsized":"2023-08-27T11:26:21.384374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:06:36.252165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874126+0000","last_change":"2023-08-27T11:26:19.414983+0000","last_active":"2023-08-27T11:26:20.874126+0000","last_peered":"2023-08-27T11:26:20.874126+0000","last_clean":"2023-08-27T11:26:20.874126+0000","last_became_active":"2023-08-27T11:26:19.414726+0000","last_became_peered":"2023-08-27T11:26:19.414726+0000","last_unstale":"2023-08-27T11:26:20.874126+0000","last_undegraded":"2023-08-27T11:26:20.874126+0000","last_fullsized":"2023-08-27T11:26:20.874126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:57:45.405543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"89'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425328+0000","last_change":"2023-08-27T11:26:18.432220+0000","last_active":"2023-08-27T11:26:20.425328+0000","last_peered":"2023-08-27T11:26:20.425328+0000","last_clean":"2023-08-27T11:26:20.425328+0000","last_became_active":"2023-08-27T11:26:18.431938+0000","last_became_peered":"2023-08-27T11:26:18.431938+0000","last_unstale":"2023-08-27T11:26:20.425328+0000","last_undegraded":"2023-08-27T11:26:20.425328+0000","last_fullsized":"2023-08-27T11:26:20.425328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:26:24.683316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384352+0000","last_change":"2023-08-27T11:26:19.427981+0000","last_active":"2023-08-27T11:26:21.384352+0000","last_peered":"2023-08-27T11:26:21.384352+0000","last_clean":"2023-08-27T11:26:21.384352+0000","last_became_active":"2023-08-27T11:26:19.427694+0000","last_became_peered":"2023-08-27T11:26:19.427694+0000","last_unstale":"2023-08-27T11:26:21.384352+0000","last_undegraded":"2023-08-27T11:26:21.384352+0000","last_fullsized":"2023-08-27T11:26:21.384352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:31.571236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384746+0000","last_change":"2023-08-27T11:26:18.416729+0000","last_active":"2023-08-27T11:26:21.384746+0000","last_peered":"2023-08-27T11:26:21.384746+0000","last_clean":"2023-08-27T11:26:21.384746+0000","last_became_active":"2023-08-27T11:26:18.416320+0000","last_became_peered":"2023-08-27T11:26:18.416320+0000","last_unstale":"2023-08-27T11:26:21.384746+0000","last_undegraded":"2023-08-27T11:26:21.384746+0000","last_fullsized":"2023-08-27T11:26:21.384746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:32.457209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430696+0000","last_change":"2023-08-27T11:26:19.422239+0000","last_active":"2023-08-27T11:26:20.430696+0000","last_peered":"2023-08-27T11:26:20.430696+0000","last_clean":"2023-08-27T11:26:20.430696+0000","last_became_active":"2023-08-27T11:26:19.421958+0000","last_became_peered":"2023-08-27T11:26:19.421958+0000","last_unstale":"2023-08-27T11:26:20.430696+0000","last_undegraded":"2023-08-27T11:26:20.430696+0000","last_fullsized":"2023-08-27T11:26:20.430696+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:06:02.839193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430341+0000","last_change":"2023-08-27T11:26:18.426347+0000","last_active":"2023-08-27T11:26:20.430341+0000","last_peered":"2023-08-27T11:26:20.430341+0000","last_clean":"2023-08-27T11:26:20.430341+0000","last_became_active":"2023-08-27T11:26:18.426066+0000","last_became_peered":"2023-08-27T11:26:18.426066+0000","last_unstale":"2023-08-27T11:26:20.430341+0000","last_undegraded":"2023-08-27T11:26:20.430341+0000","last_fullsized":"2023-08-27T11:26:20.430341+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:48:56.592853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385761+0000","last_change":"2023-08-27T11:26:19.434508+0000","last_active":"2023-08-27T11:26:21.385761+0000","last_peered":"2023-08-27T11:26:21.385761+0000","last_clean":"2023-08-27T11:26:21.385761+0000","last_became_active":"2023-08-27T11:26:19.433793+0000","last_became_peered":"2023-08-27T11:26:19.433793+0000","last_unstale":"2023-08-27T11:26:21.385761+0000","last_undegraded":"2023-08-27T11:26:21.385761+0000","last_fullsized":"2023-08-27T11:26:21.385761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:59.712674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057704+0000","last_change":"2023-08-27T11:26:18.426839+0000","last_active":"2023-08-27T11:26:21.057704+0000","last_peered":"2023-08-27T11:26:21.057704+0000","last_clean":"2023-08-27T11:26:21.057704+0000","last_became_active":"2023-08-27T11:26:18.426578+0000","last_became_peered":"2023-08-27T11:26:18.426578+0000","last_unstale":"2023-08-27T11:26:21.057704+0000","last_undegraded":"2023-08-27T11:26:21.057704+0000","last_fullsized":"2023-08-27T11:26:21.057704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:23.764319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425272+0000","last_change":"2023-08-27T11:26:19.426762+0000","last_active":"2023-08-27T11:26:20.425272+0000","last_peered":"2023-08-27T11:26:20.425272+0000","last_clean":"2023-08-27T11:26:20.425272+0000","last_became_active":"2023-08-27T11:26:19.426146+0000","last_became_peered":"2023-08-27T11:26:19.426146+0000","last_unstale":"2023-08-27T11:26:20.425272+0000","last_undegraded":"2023-08-27T11:26:20.425272+0000","last_fullsized":"2023-08-27T11:26:20.425272+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:54:50.371981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385535+0000","last_change":"2023-08-27T11:26:18.423094+0000","last_active":"2023-08-27T11:26:21.385535+0000","last_peered":"2023-08-27T11:26:21.385535+0000","last_clean":"2023-08-27T11:26:21.385535+0000","last_became_active":"2023-08-27T11:26:18.422776+0000","last_became_peered":"2023-08-27T11:26:18.422776+0000","last_unstale":"2023-08-27T11:26:21.385535+0000","last_undegraded":"2023-08-27T11:26:21.385535+0000","last_fullsized":"2023-08-27T11:26:21.385535+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:29:26.967597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874343+0000","last_change":"2023-08-27T11:26:19.418506+0000","last_active":"2023-08-27T11:26:20.874343+0000","last_peered":"2023-08-27T11:26:20.874343+0000","last_clean":"2023-08-27T11:26:20.874343+0000","last_became_active":"2023-08-27T11:26:19.418235+0000","last_became_peered":"2023-08-27T11:26:19.418235+0000","last_unstale":"2023-08-27T11:26:20.874343+0000","last_undegraded":"2023-08-27T11:26:20.874343+0000","last_fullsized":"2023-08-27T11:26:20.874343+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:50.719209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385880+0000","last_change":"2023-08-27T11:26:18.429721+0000","last_active":"2023-08-27T11:26:21.385880+0000","last_peered":"2023-08-27T11:26:21.385880+0000","last_clean":"2023-08-27T11:26:21.385880+0000","last_became_active":"2023-08-27T11:26:18.429487+0000","last_became_peered":"2023-08-27T11:26:18.429487+0000","last_unstale":"2023-08-27T11:26:21.385880+0000","last_undegraded":"2023-08-27T11:26:21.385880+0000","last_fullsized":"2023-08-27T11:26:21.385880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:06.592671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430641+0000","last_change":"2023-08-27T11:26:19.431258+0000","last_active":"2023-08-27T11:26:20.430641+0000","last_peered":"2023-08-27T11:26:20.430641+0000","last_clean":"2023-08-27T11:26:20.430641+0000","last_became_active":"2023-08-27T11:26:19.430816+0000","last_became_peered":"2023-08-27T11:26:19.430816+0000","last_unstale":"2023-08-27T11:26:20.430641+0000","last_undegraded":"2023-08-27T11:26:20.430641+0000","last_fullsized":"2023-08-27T11:26:20.430641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:55.969633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430088+0000","last_change":"2023-08-27T11:26:18.426802+0000","last_active":"2023-08-27T11:26:20.430088+0000","last_peered":"2023-08-27T11:26:20.430088+0000","last_clean":"2023-08-27T11:26:20.430088+0000","last_became_active":"2023-08-27T11:26:18.426076+0000","last_became_peered":"2023-08-27T11:26:18.426076+0000","last_unstale":"2023-08-27T11:26:20.430088+0000","last_undegraded":"2023-08-27T11:26:20.430088+0000","last_fullsized":"2023-08-27T11:26:20.430088+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:04:08.369081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425484+0000","last_change":"2023-08-27T11:26:19.426307+0000","last_active":"2023-08-27T11:26:20.425484+0000","last_peered":"2023-08-27T11:26:20.425484+0000","last_clean":"2023-08-27T11:26:20.425484+0000","last_became_active":"2023-08-27T11:26:19.425959+0000","last_became_peered":"2023-08-27T11:26:19.425959+0000","last_unstale":"2023-08-27T11:26:20.425484+0000","last_undegraded":"2023-08-27T11:26:20.425484+0000","last_fullsized":"2023-08-27T11:26:20.425484+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:10:02.440599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425034+0000","last_change":"2023-08-27T11:26:18.429003+0000","last_active":"2023-08-27T11:26:20.425034+0000","last_peered":"2023-08-27T11:26:20.425034+0000","last_clean":"2023-08-27T11:26:20.425034+0000","last_became_active":"2023-08-27T11:26:18.428701+0000","last_became_peered":"2023-08-27T11:26:18.428701+0000","last_unstale":"2023-08-27T11:26:20.425034+0000","last_undegraded":"2023-08-27T11:26:20.425034+0000","last_fullsized":"2023-08-27T11:26:20.425034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:42:33.783460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874118+0000","last_change":"2023-08-27T11:26:19.417293+0000","last_active":"2023-08-27T11:26:20.874118+0000","last_peered":"2023-08-27T11:26:20.874118+0000","last_clean":"2023-08-27T11:26:20.874118+0000","last_became_active":"2023-08-27T11:26:19.417018+0000","last_became_peered":"2023-08-27T11:26:19.417018+0000","last_unstale":"2023-08-27T11:26:20.874118+0000","last_undegraded":"2023-08-27T11:26:20.874118+0000","last_fullsized":"2023-08-27T11:26:20.874118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:02:56.092248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431106+0000","last_change":"2023-08-27T11:26:18.429425+0000","last_active":"2023-08-27T11:26:20.431106+0000","last_peered":"2023-08-27T11:26:20.431106+0000","last_clean":"2023-08-27T11:26:20.431106+0000","last_became_active":"2023-08-27T11:26:18.429137+0000","last_became_peered":"2023-08-27T11:26:18.429137+0000","last_unstale":"2023-08-27T11:26:20.431106+0000","last_undegraded":"2023-08-27T11:26:20.431106+0000","last_fullsized":"2023-08-27T11:26:20.431106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:26:09.978662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057401+0000","last_change":"2023-08-27T11:26:19.425413+0000","last_active":"2023-08-27T11:26:21.057401+0000","last_peered":"2023-08-27T11:26:21.057401+0000","last_clean":"2023-08-27T11:26:21.057401+0000","last_became_active":"2023-08-27T11:26:19.424482+0000","last_became_peered":"2023-08-27T11:26:19.424482+0000","last_unstale":"2023-08-27T11:26:21.057401+0000","last_undegraded":"2023-08-27T11:26:21.057401+0000","last_fullsized":"2023-08-27T11:26:21.057401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:36.587246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874328+0000","last_change":"2023-08-27T11:26:18.426356+0000","last_active":"2023-08-27T11:26:20.874328+0000","last_peered":"2023-08-27T11:26:20.874328+0000","last_clean":"2023-08-27T11:26:20.874328+0000","last_became_active":"2023-08-27T11:26:18.426041+0000","last_became_peered":"2023-08-27T11:26:18.426041+0000","last_unstale":"2023-08-27T11:26:20.874328+0000","last_undegraded":"2023-08-27T11:26:20.874328+0000","last_fullsized":"2023-08-27T11:26:20.874328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:28.381950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384393+0000","last_change":"2023-08-27T11:26:19.431679+0000","last_active":"2023-08-27T11:26:21.384393+0000","last_peered":"2023-08-27T11:26:21.384393+0000","last_clean":"2023-08-27T11:26:21.384393+0000","last_became_active":"2023-08-27T11:26:19.431405+0000","last_became_peered":"2023-08-27T11:26:19.431405+0000","last_unstale":"2023-08-27T11:26:21.384393+0000","last_undegraded":"2023-08-27T11:26:21.384393+0000","last_fullsized":"2023-08-27T11:26:21.384393+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:02:21.450786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"89'3","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385384+0000","last_change":"2023-08-27T11:26:18.421366+0000","last_active":"2023-08-27T11:26:21.385384+0000","last_peered":"2023-08-27T11:26:21.385384+0000","last_clean":"2023-08-27T11:26:21.385384+0000","last_became_active":"2023-08-27T11:26:18.420937+0000","last_became_peered":"2023-08-27T11:26:18.420937+0000","last_unstale":"2023-08-27T11:26:21.385384+0000","last_undegraded":"2023-08-27T11:26:21.385384+0000","last_fullsized":"2023-08-27T11:26:21.385384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:44:04.672082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057480+0000","last_change":"2023-08-27T11:26:19.424582+0000","last_active":"2023-08-27T11:26:21.057480+0000","last_peered":"2023-08-27T11:26:21.057480+0000","last_clean":"2023-08-27T11:26:21.057480+0000","last_became_active":"2023-08-27T11:26:19.424288+0000","last_became_peered":"2023-08-27T11:26:19.424288+0000","last_unstale":"2023-08-27T11:26:21.057480+0000","last_undegraded":"2023-08-27T11:26:21.057480+0000","last_fullsized":"2023-08-27T11:26:21.057480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:16.279109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430741+0000","last_change":"2023-08-27T11:26:18.421802+0000","last_active":"2023-08-27T11:26:20.430741+0000","last_peered":"2023-08-27T11:26:20.430741+0000","last_clean":"2023-08-27T11:26:20.430741+0000","last_became_active":"2023-08-27T11:26:18.421093+0000","last_became_peered":"2023-08-27T11:26:18.421093+0000","last_unstale":"2023-08-27T11:26:20.430741+0000","last_undegraded":"2023-08-27T11:26:20.430741+0000","last_fullsized":"2023-08-27T11:26:20.430741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:29.408679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385662+0000","last_change":"2023-08-27T11:26:19.435932+0000","last_active":"2023-08-27T11:26:21.385662+0000","last_peered":"2023-08-27T11:26:21.385662+0000","last_clean":"2023-08-27T11:26:21.385662+0000","last_became_active":"2023-08-27T11:26:19.435665+0000","last_became_peered":"2023-08-27T11:26:19.435665+0000","last_unstale":"2023-08-27T11:26:21.385662+0000","last_undegraded":"2023-08-27T11:26:21.385662+0000","last_fullsized":"2023-08-27T11:26:21.385662+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:11.616933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057739+0000","last_change":"2023-08-27T11:26:18.421876+0000","last_active":"2023-08-27T11:26:21.057739+0000","last_peered":"2023-08-27T11:26:21.057739+0000","last_clean":"2023-08-27T11:26:21.057739+0000","last_became_active":"2023-08-27T11:26:18.421583+0000","last_became_peered":"2023-08-27T11:26:18.421583+0000","last_unstale":"2023-08-27T11:26:21.057739+0000","last_undegraded":"2023-08-27T11:26:21.057739+0000","last_fullsized":"2023-08-27T11:26:21.057739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:48:43.946799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430199+0000","last_change":"2023-08-27T11:26:19.430893+0000","last_active":"2023-08-27T11:26:20.430199+0000","last_peered":"2023-08-27T11:26:20.430199+0000","last_clean":"2023-08-27T11:26:20.430199+0000","last_became_active":"2023-08-27T11:26:19.430225+0000","last_became_peered":"2023-08-27T11:26:19.430225+0000","last_unstale":"2023-08-27T11:26:20.430199+0000","last_undegraded":"2023-08-27T11:26:20.430199+0000","last_fullsized":"2023-08-27T11:26:20.430199+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:43.123979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425475+0000","last_change":"2023-08-27T11:26:18.432084+0000","last_active":"2023-08-27T11:26:20.425475+0000","last_peered":"2023-08-27T11:26:20.425475+0000","last_clean":"2023-08-27T11:26:20.425475+0000","last_became_active":"2023-08-27T11:26:18.431877+0000","last_became_peered":"2023-08-27T11:26:18.431877+0000","last_unstale":"2023-08-27T11:26:20.425475+0000","last_undegraded":"2023-08-27T11:26:20.425475+0000","last_fullsized":"2023-08-27T11:26:20.425475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:12:10.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384677+0000","last_change":"2023-08-27T11:26:19.429400+0000","last_active":"2023-08-27T11:26:21.384677+0000","last_peered":"2023-08-27T11:26:21.384677+0000","last_clean":"2023-08-27T11:26:21.384677+0000","last_became_active":"2023-08-27T11:26:19.429146+0000","last_became_peered":"2023-08-27T11:26:19.429146+0000","last_unstale":"2023-08-27T11:26:21.384677+0000","last_undegraded":"2023-08-27T11:26:21.384677+0000","last_fullsized":"2023-08-27T11:26:21.384677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:06.185951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384453+0000","last_change":"2023-08-27T11:26:18.420633+0000","last_active":"2023-08-27T11:26:21.384453+0000","last_peered":"2023-08-27T11:26:21.384453+0000","last_clean":"2023-08-27T11:26:21.384453+0000","last_became_active":"2023-08-27T11:26:18.420312+0000","last_became_peered":"2023-08-27T11:26:18.420312+0000","last_unstale":"2023-08-27T11:26:21.384453+0000","last_undegraded":"2023-08-27T11:26:21.384453+0000","last_fullsized":"2023-08-27T11:26:21.384453+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:33:20.872678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874639+0000","last_change":"2023-08-27T11:26:19.434950+0000","last_active":"2023-08-27T11:26:20.874639+0000","last_peered":"2023-08-27T11:26:20.874639+0000","last_clean":"2023-08-27T11:26:20.874639+0000","last_became_active":"2023-08-27T11:26:19.434684+0000","last_became_peered":"2023-08-27T11:26:19.434684+0000","last_unstale":"2023-08-27T11:26:20.874639+0000","last_undegraded":"2023-08-27T11:26:20.874639+0000","last_fullsized":"2023-08-27T11:26:20.874639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:52:42.875755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"89'8","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430974+0000","last_change":"2023-08-27T11:26:18.429430+0000","last_active":"2023-08-27T11:26:20.430974+0000","last_peered":"2023-08-27T11:26:20.430974+0000","last_clean":"2023-08-27T11:26:20.430974+0000","last_became_active":"2023-08-27T11:26:18.429158+0000","last_became_peered":"2023-08-27T11:26:18.429158+0000","last_unstale":"2023-08-27T11:26:20.430974+0000","last_undegraded":"2023-08-27T11:26:20.430974+0000","last_fullsized":"2023-08-27T11:26:20.430974+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:53:21.490317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874262+0000","last_change":"2023-08-27T11:26:19.425141+0000","last_active":"2023-08-27T11:26:20.874262+0000","last_peered":"2023-08-27T11:26:20.874262+0000","last_clean":"2023-08-27T11:26:20.874262+0000","last_became_active":"2023-08-27T11:26:19.424669+0000","last_became_peered":"2023-08-27T11:26:19.424669+0000","last_unstale":"2023-08-27T11:26:20.874262+0000","last_undegraded":"2023-08-27T11:26:20.874262+0000","last_fullsized":"2023-08-27T11:26:20.874262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:31:34.706537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057687+0000","last_change":"2023-08-27T11:26:18.421572+0000","last_active":"2023-08-27T11:26:21.057687+0000","last_peered":"2023-08-27T11:26:21.057687+0000","last_clean":"2023-08-27T11:26:21.057687+0000","last_became_active":"2023-08-27T11:26:18.421268+0000","last_became_peered":"2023-08-27T11:26:18.421268+0000","last_unstale":"2023-08-27T11:26:21.057687+0000","last_undegraded":"2023-08-27T11:26:21.057687+0000","last_fullsized":"2023-08-27T11:26:21.057687+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:44.389523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.429894+0000","last_change":"2023-08-27T11:26:19.427139+0000","last_active":"2023-08-27T11:26:20.429894+0000","last_peered":"2023-08-27T11:26:20.429894+0000","last_clean":"2023-08-27T11:26:20.429894+0000","last_became_active":"2023-08-27T11:26:19.426793+0000","last_became_peered":"2023-08-27T11:26:19.426793+0000","last_unstale":"2023-08-27T11:26:20.429894+0000","last_undegraded":"2023-08-27T11:26:20.429894+0000","last_fullsized":"2023-08-27T11:26:20.429894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:05:13.559317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"90'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:22.355460+0000","last_change":"2023-08-27T11:26:18.429524+0000","last_active":"2023-08-27T11:26:22.355460+0000","last_peered":"2023-08-27T11:26:22.355460+0000","last_clean":"2023-08-27T11:26:22.355460+0000","last_became_active":"2023-08-27T11:26:18.429224+0000","last_became_peered":"2023-08-27T11:26:18.429224+0000","last_unstale":"2023-08-27T11:26:22.355460+0000","last_undegraded":"2023-08-27T11:26:22.355460+0000","last_fullsized":"2023-08-27T11:26:22.355460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:13.776898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425320+0000","last_change":"2023-08-27T11:26:19.427555+0000","last_active":"2023-08-27T11:26:20.425320+0000","last_peered":"2023-08-27T11:26:20.425320+0000","last_clean":"2023-08-27T11:26:20.425320+0000","last_became_active":"2023-08-27T11:26:19.427295+0000","last_became_peered":"2023-08-27T11:26:19.427295+0000","last_unstale":"2023-08-27T11:26:20.425320+0000","last_undegraded":"2023-08-27T11:26:20.425320+0000","last_fullsized":"2023-08-27T11:26:20.425320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:35.698617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430342+0000","last_change":"2023-08-27T11:26:18.432922+0000","last_active":"2023-08-27T11:26:20.430342+0000","last_peered":"2023-08-27T11:26:20.430342+0000","last_clean":"2023-08-27T11:26:20.430342+0000","last_became_active":"2023-08-27T11:26:18.432631+0000","last_became_peered":"2023-08-27T11:26:18.432631+0000","last_unstale":"2023-08-27T11:26:20.430342+0000","last_undegraded":"2023-08-27T11:26:20.430342+0000","last_fullsized":"2023-08-27T11:26:20.430342+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:58.703884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385691+0000","last_change":"2023-08-27T11:26:19.431706+0000","last_active":"2023-08-27T11:26:21.385691+0000","last_peered":"2023-08-27T11:26:21.385691+0000","last_clean":"2023-08-27T11:26:21.385691+0000","last_became_active":"2023-08-27T11:26:19.431424+0000","last_became_peered":"2023-08-27T11:26:19.431424+0000","last_unstale":"2023-08-27T11:26:21.385691+0000","last_undegraded":"2023-08-27T11:26:21.385691+0000","last_fullsized":"2023-08-27T11:26:21.385691+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:43.123979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057644+0000","last_change":"2023-08-27T11:26:18.422687+0000","last_active":"2023-08-27T11:26:21.057644+0000","last_peered":"2023-08-27T11:26:21.057644+0000","last_clean":"2023-08-27T11:26:21.057644+0000","last_became_active":"2023-08-27T11:26:18.422404+0000","last_became_peered":"2023-08-27T11:26:18.422404+0000","last_unstale":"2023-08-27T11:26:21.057644+0000","last_undegraded":"2023-08-27T11:26:21.057644+0000","last_fullsized":"2023-08-27T11:26:21.057644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:34.961579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384633+0000","last_change":"2023-08-27T11:26:19.425732+0000","last_active":"2023-08-27T11:26:21.384633+0000","last_peered":"2023-08-27T11:26:21.384633+0000","last_clean":"2023-08-27T11:26:21.384633+0000","last_became_active":"2023-08-27T11:26:19.425452+0000","last_became_peered":"2023-08-27T11:26:19.425452+0000","last_unstale":"2023-08-27T11:26:21.384633+0000","last_undegraded":"2023-08-27T11:26:21.384633+0000","last_fullsized":"2023-08-27T11:26:21.384633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:15.049461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430151+0000","last_change":"2023-08-27T11:26:18.423038+0000","last_active":"2023-08-27T11:26:20.430151+0000","last_peered":"2023-08-27T11:26:20.430151+0000","last_clean":"2023-08-27T11:26:20.430151+0000","last_became_active":"2023-08-27T11:26:18.422607+0000","last_became_peered":"2023-08-27T11:26:18.422607+0000","last_unstale":"2023-08-27T11:26:20.430151+0000","last_undegraded":"2023-08-27T11:26:20.430151+0000","last_fullsized":"2023-08-27T11:26:20.430151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:33:42.505117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384761+0000","last_change":"2023-08-27T11:26:19.425658+0000","last_active":"2023-08-27T11:26:21.384761+0000","last_peered":"2023-08-27T11:26:21.384761+0000","last_clean":"2023-08-27T11:26:21.384761+0000","last_became_active":"2023-08-27T11:26:19.425387+0000","last_became_peered":"2023-08-27T11:26:19.425387+0000","last_unstale":"2023-08-27T11:26:21.384761+0000","last_undegraded":"2023-08-27T11:26:21.384761+0000","last_fullsized":"2023-08-27T11:26:21.384761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:52.205981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384498+0000","last_change":"2023-08-27T11:26:18.426465+0000","last_active":"2023-08-27T11:26:21.384498+0000","last_peered":"2023-08-27T11:26:21.384498+0000","last_clean":"2023-08-27T11:26:21.384498+0000","last_became_active":"2023-08-27T11:26:18.426180+0000","last_became_peered":"2023-08-27T11:26:18.426180+0000","last_unstale":"2023-08-27T11:26:21.384498+0000","last_undegraded":"2023-08-27T11:26:21.384498+0000","last_fullsized":"2023-08-27T11:26:21.384498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:46.357284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385448+0000","last_change":"2023-08-27T11:26:19.433993+0000","last_active":"2023-08-27T11:26:21.385448+0000","last_peered":"2023-08-27T11:26:21.385448+0000","last_clean":"2023-08-27T11:26:21.385448+0000","last_became_active":"2023-08-27T11:26:19.433735+0000","last_became_peered":"2023-08-27T11:26:19.433735+0000","last_unstale":"2023-08-27T11:26:21.385448+0000","last_undegraded":"2023-08-27T11:26:21.385448+0000","last_fullsized":"2023-08-27T11:26:21.385448+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:44:55.647335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430463+0000","last_change":"2023-08-27T11:26:18.432789+0000","last_active":"2023-08-27T11:26:20.430463+0000","last_peered":"2023-08-27T11:26:20.430463+0000","last_clean":"2023-08-27T11:26:20.430463+0000","last_became_active":"2023-08-27T11:26:18.432521+0000","last_became_peered":"2023-08-27T11:26:18.432521+0000","last_unstale":"2023-08-27T11:26:20.430463+0000","last_undegraded":"2023-08-27T11:26:20.430463+0000","last_fullsized":"2023-08-27T11:26:20.430463+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:06.592671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385904+0000","last_change":"2023-08-27T11:26:19.434177+0000","last_active":"2023-08-27T11:26:21.385904+0000","last_peered":"2023-08-27T11:26:21.385904+0000","last_clean":"2023-08-27T11:26:21.385904+0000","last_became_active":"2023-08-27T11:26:19.432686+0000","last_became_peered":"2023-08-27T11:26:19.432686+0000","last_unstale":"2023-08-27T11:26:21.385904+0000","last_undegraded":"2023-08-27T11:26:21.385904+0000","last_fullsized":"2023-08-27T11:26:21.385904+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:34.644697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384455+0000","last_change":"2023-08-27T11:26:18.426963+0000","last_active":"2023-08-27T11:26:21.384455+0000","last_peered":"2023-08-27T11:26:21.384455+0000","last_clean":"2023-08-27T11:26:21.384455+0000","last_became_active":"2023-08-27T11:26:18.426752+0000","last_became_peered":"2023-08-27T11:26:18.426752+0000","last_unstale":"2023-08-27T11:26:21.384455+0000","last_undegraded":"2023-08-27T11:26:21.384455+0000","last_fullsized":"2023-08-27T11:26:21.384455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:29:35.149141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425190+0000","last_change":"2023-08-27T11:26:19.431940+0000","last_active":"2023-08-27T11:26:20.425190+0000","last_peered":"2023-08-27T11:26:20.425190+0000","last_clean":"2023-08-27T11:26:20.425190+0000","last_became_active":"2023-08-27T11:26:19.431631+0000","last_became_peered":"2023-08-27T11:26:19.431631+0000","last_unstale":"2023-08-27T11:26:20.425190+0000","last_undegraded":"2023-08-27T11:26:20.425190+0000","last_fullsized":"2023-08-27T11:26:20.425190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:37.504080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430548+0000","last_change":"2023-08-27T11:26:18.425472+0000","last_active":"2023-08-27T11:26:20.430548+0000","last_peered":"2023-08-27T11:26:20.430548+0000","last_clean":"2023-08-27T11:26:20.430548+0000","last_became_active":"2023-08-27T11:26:18.425150+0000","last_became_peered":"2023-08-27T11:26:18.425150+0000","last_unstale":"2023-08-27T11:26:20.430548+0000","last_undegraded":"2023-08-27T11:26:20.430548+0000","last_fullsized":"2023-08-27T11:26:20.430548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:10.608143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431200+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:26:20.431200+0000","last_peered":"2023-08-27T11:26:20.431200+0000","last_clean":"2023-08-27T11:26:20.431200+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:26:20.431200+0000","last_undegraded":"2023-08-27T11:26:20.431200+0000","last_fullsized":"2023-08-27T11:26:20.431200+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:53.465982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384795+0000","last_change":"2023-08-27T11:26:19.426312+0000","last_active":"2023-08-27T11:26:21.384795+0000","last_peered":"2023-08-27T11:26:21.384795+0000","last_clean":"2023-08-27T11:26:21.384795+0000","last_became_active":"2023-08-27T11:26:19.425963+0000","last_became_peered":"2023-08-27T11:26:19.425963+0000","last_unstale":"2023-08-27T11:26:21.384795+0000","last_undegraded":"2023-08-27T11:26:21.384795+0000","last_fullsized":"2023-08-27T11:26:21.384795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:46.854781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384674+0000","last_change":"2023-08-27T11:26:18.429275+0000","last_active":"2023-08-27T11:26:21.384674+0000","last_peered":"2023-08-27T11:26:21.384674+0000","last_clean":"2023-08-27T11:26:21.384674+0000","last_became_active":"2023-08-27T11:26:18.428994+0000","last_became_peered":"2023-08-27T11:26:18.428994+0000","last_unstale":"2023-08-27T11:26:21.384674+0000","last_undegraded":"2023-08-27T11:26:21.384674+0000","last_fullsized":"2023-08-27T11:26:21.384674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:55.212494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430413+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:26:20.430413+0000","last_peered":"2023-08-27T11:26:20.430413+0000","last_clean":"2023-08-27T11:26:20.430413+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:26:20.430413+0000","last_undegraded":"2023-08-27T11:26:20.430413+0000","last_fullsized":"2023-08-27T11:26:20.430413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:35:03.658156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384543+0000","last_change":"2023-08-27T11:26:19.434349+0000","last_active":"2023-08-27T11:26:21.384543+0000","last_peered":"2023-08-27T11:26:21.384543+0000","last_clean":"2023-08-27T11:26:21.384543+0000","last_became_active":"2023-08-27T11:26:19.434063+0000","last_became_peered":"2023-08-27T11:26:19.434063+0000","last_unstale":"2023-08-27T11:26:21.384543+0000","last_undegraded":"2023-08-27T11:26:21.384543+0000","last_fullsized":"2023-08-27T11:26:21.384543+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:15.835557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.431167+0000","last_change":"2023-08-27T11:26:18.429449+0000","last_active":"2023-08-27T11:26:20.431167+0000","last_peered":"2023-08-27T11:26:20.431167+0000","last_clean":"2023-08-27T11:26:20.431167+0000","last_became_active":"2023-08-27T11:26:18.429134+0000","last_became_peered":"2023-08-27T11:26:18.429134+0000","last_unstale":"2023-08-27T11:26:20.431167+0000","last_undegraded":"2023-08-27T11:26:20.431167+0000","last_fullsized":"2023-08-27T11:26:20.431167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:47:49.142361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":141,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385196+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:26:21.385196+0000","last_peered":"2023-08-27T11:26:21.385196+0000","last_clean":"2023-08-27T11:26:21.385196+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:26:21.385196+0000","last_undegraded":"2023-08-27T11:26:21.385196+0000","last_fullsized":"2023-08-27T11:26:21.385196+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:17:04.262716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385249+0000","last_change":"2023-08-27T11:26:19.437248+0000","last_active":"2023-08-27T11:26:21.385249+0000","last_peered":"2023-08-27T11:26:21.385249+0000","last_clean":"2023-08-27T11:26:21.385249+0000","last_became_active":"2023-08-27T11:26:19.436894+0000","last_became_peered":"2023-08-27T11:26:19.436894+0000","last_unstale":"2023-08-27T11:26:21.385249+0000","last_undegraded":"2023-08-27T11:26:21.385249+0000","last_fullsized":"2023-08-27T11:26:21.385249+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:46:39.788991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384819+0000","last_change":"2023-08-27T11:26:18.429297+0000","last_active":"2023-08-27T11:26:21.384819+0000","last_peered":"2023-08-27T11:26:21.384819+0000","last_clean":"2023-08-27T11:26:21.384819+0000","last_became_active":"2023-08-27T11:26:18.429028+0000","last_became_peered":"2023-08-27T11:26:18.429028+0000","last_unstale":"2023-08-27T11:26:21.384819+0000","last_undegraded":"2023-08-27T11:26:21.384819+0000","last_fullsized":"2023-08-27T11:26:21.384819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:25:56.014027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":141,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385082+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:26:21.385082+0000","last_peered":"2023-08-27T11:26:21.385082+0000","last_clean":"2023-08-27T11:26:21.385082+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:26:21.385082+0000","last_undegraded":"2023-08-27T11:26:21.385082+0000","last_fullsized":"2023-08-27T11:26:21.385082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:55.131492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057479+0000","last_change":"2023-08-27T11:26:19.432826+0000","last_active":"2023-08-27T11:26:21.057479+0000","last_peered":"2023-08-27T11:26:21.057479+0000","last_clean":"2023-08-27T11:26:21.057479+0000","last_became_active":"2023-08-27T11:26:19.432562+0000","last_became_peered":"2023-08-27T11:26:19.432562+0000","last_unstale":"2023-08-27T11:26:21.057479+0000","last_undegraded":"2023-08-27T11:26:21.057479+0000","last_fullsized":"2023-08-27T11:26:21.057479+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:05.457562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385725+0000","last_change":"2023-08-27T11:26:18.425845+0000","last_active":"2023-08-27T11:26:21.385725+0000","last_peered":"2023-08-27T11:26:21.385725+0000","last_clean":"2023-08-27T11:26:21.385725+0000","last_became_active":"2023-08-27T11:26:18.425533+0000","last_became_peered":"2023-08-27T11:26:18.425533+0000","last_unstale":"2023-08-27T11:26:21.385725+0000","last_undegraded":"2023-08-27T11:26:21.385725+0000","last_fullsized":"2023-08-27T11:26:21.385725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:22:46.305412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":141,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385273+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:26:21.385273+0000","last_peered":"2023-08-27T11:26:21.385273+0000","last_clean":"2023-08-27T11:26:21.385273+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:26:21.385273+0000","last_undegraded":"2023-08-27T11:26:21.385273+0000","last_fullsized":"2023-08-27T11:26:21.385273+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:27:20.122317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430815+0000","last_change":"2023-08-27T11:26:19.427376+0000","last_active":"2023-08-27T11:26:20.430815+0000","last_peered":"2023-08-27T11:26:20.430815+0000","last_clean":"2023-08-27T11:26:20.430815+0000","last_became_active":"2023-08-27T11:26:19.426214+0000","last_became_peered":"2023-08-27T11:26:19.426214+0000","last_unstale":"2023-08-27T11:26:20.430815+0000","last_undegraded":"2023-08-27T11:26:20.430815+0000","last_fullsized":"2023-08-27T11:26:20.430815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:07:17.226611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.385541+0000","last_change":"2023-08-27T11:26:18.426780+0000","last_active":"2023-08-27T11:26:21.385541+0000","last_peered":"2023-08-27T11:26:21.385541+0000","last_clean":"2023-08-27T11:26:21.385541+0000","last_became_active":"2023-08-27T11:26:18.426462+0000","last_became_peered":"2023-08-27T11:26:18.426462+0000","last_unstale":"2023-08-27T11:26:21.385541+0000","last_undegraded":"2023-08-27T11:26:21.385541+0000","last_fullsized":"2023-08-27T11:26:21.385541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:06:04.639068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":291,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384913+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:26:21.384913+0000","last_peered":"2023-08-27T11:26:21.384913+0000","last_clean":"2023-08-27T11:26:21.384913+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:26:21.384913+0000","last_undegraded":"2023-08-27T11:26:21.384913+0000","last_fullsized":"2023-08-27T11:26:21.384913+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:30:11.743649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.057579+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:26:21.057579+0000","last_peered":"2023-08-27T11:26:21.057579+0000","last_clean":"2023-08-27T11:26:21.057579+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:26:21.057579+0000","last_undegraded":"2023-08-27T11:26:21.057579+0000","last_fullsized":"2023-08-27T11:26:21.057579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:12:51.448320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874474+0000","last_change":"2023-08-27T11:26:19.420539+0000","last_active":"2023-08-27T11:26:20.874474+0000","last_peered":"2023-08-27T11:26:20.874474+0000","last_clean":"2023-08-27T11:26:20.874474+0000","last_became_active":"2023-08-27T11:26:19.420291+0000","last_became_peered":"2023-08-27T11:26:19.420291+0000","last_unstale":"2023-08-27T11:26:20.874474+0000","last_undegraded":"2023-08-27T11:26:20.874474+0000","last_fullsized":"2023-08-27T11:26:20.874474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:23:25.892210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384424+0000","last_change":"2023-08-27T11:26:18.432770+0000","last_active":"2023-08-27T11:26:21.384424+0000","last_peered":"2023-08-27T11:26:21.384424+0000","last_clean":"2023-08-27T11:26:21.384424+0000","last_became_active":"2023-08-27T11:26:18.432552+0000","last_became_peered":"2023-08-27T11:26:18.432552+0000","last_unstale":"2023-08-27T11:26:21.384424+0000","last_undegraded":"2023-08-27T11:26:21.384424+0000","last_fullsized":"2023-08-27T11:26:21.384424+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:05:38.418817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":139,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.425350+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:26:20.425350+0000","last_peered":"2023-08-27T11:26:20.425350+0000","last_clean":"2023-08-27T11:26:20.425350+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:26:20.425350+0000","last_undegraded":"2023-08-27T11:26:20.425350+0000","last_fullsized":"2023-08-27T11:26:20.425350+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:38:27.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.874280+0000","last_change":"2023-08-27T11:26:19.427286+0000","last_active":"2023-08-27T11:26:20.874280+0000","last_peered":"2023-08-27T11:26:20.874280+0000","last_clean":"2023-08-27T11:26:20.874280+0000","last_became_active":"2023-08-27T11:26:19.426940+0000","last_became_peered":"2023-08-27T11:26:19.426940+0000","last_unstale":"2023-08-27T11:26:20.874280+0000","last_undegraded":"2023-08-27T11:26:20.874280+0000","last_fullsized":"2023-08-27T11:26:20.874280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:18.398253+0000","last_clean_scrub_stamp":"2023-08-27T11:26:18.398253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:31.231234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:21.384817+0000","last_change":"2023-08-27T11:26:18.415885+0000","last_active":"2023-08-27T11:26:21.384817+0000","last_peered":"2023-08-27T11:26:21.384817+0000","last_clean":"2023-08-27T11:26:21.384817+0000","last_became_active":"2023-08-27T11:26:18.415623+0000","last_became_peered":"2023-08-27T11:26:18.415623+0000","last_unstale":"2023-08-27T11:26:21.384817+0000","last_undegraded":"2023-08-27T11:26:21.384817+0000","last_fullsized":"2023-08-27T11:26:21.384817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:17.389463+0000","last_clean_scrub_stamp":"2023-08-27T11:26:17.389463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:10:07.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-27T11:26:20.430427+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:26:20.430427+0000","last_peered":"2023-08-27T11:26:20.430427+0000","last_clean":"2023-08-27T11:26:20.430427+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:26:20.430427+0000","last_undegraded":"2023-08-27T11:26:20.430427+0000","last_fullsized":"2023-08-27T11:26:20.430427+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:43:08.686044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":1103318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.38900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.55000000000000004,"15min":0.50800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":0.77500000000000002,"15min":0.872},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.253},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.32500000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.48699999999999999},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.33800000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.90200000000000002},"last":0.503}]},{"osd":3,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.55600000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.505},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.72699999999999998,"5min":0.72799999999999998,"15min":0.77400000000000002},"last":0.433},{"interface":"front","average":{"1min":0.5,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.57099999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47899999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.60199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1134592,"data_stored":515006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51500000000000001,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.29099999999999998},"max":{"1min":0.626,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67300000000000004,"5min":0.86199999999999999,"15min":0.89800000000000002},"last":0.58999999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.316,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.80600000000000005,"15min":0.873},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.62,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.72299999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58299999999999996,"5min":0.74199999999999999,"15min":0.85899999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.48899999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.58199999999999996,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.70699999999999996,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.65200000000000002,"5min":0.84699999999999998,"15min":0.93799999999999994},"last":0.65200000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.70699999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.374},"max":{"1min":0.77800000000000002,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.67300000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.624,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.78100000000000003,"5min":0.875,"15min":0.875},"last":0.63600000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.748,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.621},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.34899999999999998},"max":{"1min":0.76700000000000002,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1712128,"data_stored":1103192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.308,"15min":0.59299999999999997},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.45800000000000002,"5min":0.502,"15min":11.971},"last":0.38},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.89000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.51700000000000002,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.29199999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.625},"min":{"1min":0.23400000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.495,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.34300000000000003},"min":{"1min":0.246,"5min":0.224,"15min":0.224},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.58599999999999997},"last":0.314}]},{"osd":3,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.64100000000000001},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":10.99},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.373,"15min":0.92500000000000004},"min":{"1min":0.318,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.54700000000000004,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.36099999999999999,"15min":0.33400000000000002},"max":{"1min":0.76900000000000002,"5min":0.90700000000000003,"15min":1.26},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.80700000000000005},"min":{"1min":0.434,"5min":0.37,"15min":0.29099999999999998},"max":{"1min":0.64500000000000002,"5min":0.77600000000000002,"15min":10.957000000000001},"last":0.622}]},{"osd":5,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":1.121},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.36199999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71499999999999997,"5min":0.88700000000000001,"15min":1.296},"last":0.55100000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.84299999999999997},"min":{"1min":0.44800000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":1.1379999999999999},"min":{"1min":0.435,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.74299999999999999,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.58999999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":1.1319999999999999},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84199999999999997,"15min":22.988},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.63500000000000001},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":1.327},"last":0.60099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574966,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":513131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.29699999999999999,"15min":0.311},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.33200000000000002,"15min":0.34999999999999998},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.312}]},{"osd":2,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.24399999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.628,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.33200000000000002,"15min":0.33900000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.34799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.33700000000000002,"15min":0.34599999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.502,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.27500000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.54200000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.36399999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79100000000000004,"5min":0.92500000000000004,"15min":1.214},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.60999999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.77500000000000002,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85399999999999998,"5min":0.90200000000000002,"15min":1.603},"last":0.64500000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.65700000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.74099999999999999,"5min":0.94199999999999995,"15min":1.383},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82699999999999996,"5min":0.88900000000000001,"15min":0.92000000000000004},"last":0.68600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1118208,"data_stored":512824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29799999999999999,"15min":0.30199999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40500000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.245},{"interface":"front","average":{"1min":0.314,"5min":0.316,"15min":0.317},"min":{"1min":0.22700000000000001,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.629},"last":0.318}]},{"osd":1,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.317,"15min":0.32400000000000001},"min":{"1min":0.222,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.36499999999999999,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33800000000000002,"15min":0.34599999999999997},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47799999999999998,"5min":0.64200000000000002,"15min":0.749},"last":0.29699999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36099999999999999,"15min":0.38200000000000001},"min":{"1min":0.318,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41599999999999998,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.376,"15min":0.39800000000000002},"min":{"1min":0.30599999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.69499999999999995},"last":0.33500000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.68600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.433,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.76100000000000001,"5min":0.78400000000000003,"15min":0.879},"last":0.56200000000000006}]},{"osd":5,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70699999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.40999999999999998,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.73399999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77200000000000002,"5min":0.82899999999999996,"15min":0.85299999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.59399999999999997},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.65900000000000003,"5min":0.80500000000000005,"15min":0.83599999999999997},"last":0.61899999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.629},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.60899999999999999},"min":{"1min":0.438,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1118208,"data_stored":513372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29699999999999999,"15min":0.29999999999999999},"min":{"1min":0.21299999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32100000000000001,"15min":0.312},"min":{"1min":0.23100000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.38400000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.26000000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32300000000000001,"15min":0.313},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.19500000000000001},"max":{"1min":0.46700000000000003,"5min":0.50600000000000001,"15min":0.60399999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":0.44900000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"last":0.316}]},{"osd":2,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.28499999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.53300000000000003,"5min":0.57199999999999995,"15min":0.57599999999999996},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.52500000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.27500000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.28599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.44500000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.371,"15min":0.314},"max":{"1min":0.79100000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.40000000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.373},"max":{"1min":0.80400000000000005,"5min":1.091,"15min":1.091},"last":0.434},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.61199999999999999},"min":{"1min":0.41699999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78400000000000003,"5min":1.111,"15min":1.111},"last":0.48699999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.61199999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74399999999999999,"5min":1.032,"15min":1.236},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.41599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1122304,"data_stored":512882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.755,"5min":0.997,"15min":0.997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.68700000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.442}]},{"osd":1,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.752,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56999999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.51000000000000001},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.629}]},{"osd":3,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64300000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.85899999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75700000000000001,"5min":0.85999999999999999,"15min":0.879},"last":0.74299999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70599999999999996,"5min":0.78900000000000003,"15min":0.90600000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59599999999999997,"15min":0.623},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.92300000000000004},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.80600000000000005,"15min":0.82499999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.89300000000000002},"last":0.71399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":512790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.69699999999999995,"5min":0.78600000000000003,"15min":0.96899999999999997},"last":0.433},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.64500000000000002,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.55000000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.54200000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66700000000000004,"5min":0.72899999999999998,"15min":1.075},"last":0.53700000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.54800000000000004},"min":{"1min":0.36399999999999999,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.68899999999999995,"5min":0.78000000000000003,"15min":0.95699999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54100000000000004,"15min":0.55700000000000005},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.67600000000000005,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.41099999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.56299999999999994,"15min":0.58099999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73399999999999999,"5min":0.81000000000000005,"15min":1.03},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.35199999999999998},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":1.026},"last":0.47099999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.31,"15min":0.27000000000000002},"max":{"1min":0.72799999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.371},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.73399999999999999,"5min":0.86399999999999999,"15min":0.91400000000000003},"last":0.63700000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59199999999999997,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.28799999999999998},"max":{"1min":0.77700000000000002,"5min":0.84799999999999998,"15min":0.94199999999999995},"last":0.497}]},{"osd":7,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.61499999999999999,"15min":0.629},"min":{"1min":0.432,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.63900000000000001,"5min":0.90700000000000003,"15min":0.95899999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.31},"max":{"1min":0.69399999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:26:28.439 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:26:28.440 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:26:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.599+0000 7fd9decfb700 1 Processor -- start 2023-08-27T11:26:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.599+0000 7fd9decfb700 1 -- start start 2023-08-27T11:26:28.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.600+0000 7fd9decfb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d8146c20 0x7fd9d8160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.600+0000 7fd9decfb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd9d80b3440 0x7fd9d815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.600+0000 7fd9decfb700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d80b5070 con 0x7fd9d815e6e0 2023-08-27T11:26:28.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.600+0000 7fd9decfb700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d8147000 con 0x7fd9d80b3440 2023-08-27T11:26:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.600+0000 7fd9decfb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81614a0 con 0x7fd9d8146c20 2023-08-27T11:26:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.600+0000 7fd9dd298700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd9d815e6e0 0x7fd9d806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55600/0 (socket says 172.21.15.112:55600) 2023-08-27T11:26:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.600+0000 7fd9dd298700 1 -- 172.21.15.112:0/4068682062 learned_addr learned my addr 172.21.15.112:0/4068682062 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699433862 0 0) 0x7fd9d0001490 con 0x7fd9d815e6e0 2023-08-27T11:26:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9c80035b0 con 0x7fd9d815e6e0 2023-08-27T11:26:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9dca97700 1 --2- 172.21.15.112:0/4068682062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d8146c20 0x7fd9d8160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d7fff700 1 --2- 172.21.15.112:0/4068682062 >> v2:172.21.15.138:3300/0 conn(0x7fd9d80b3440 0x7fd9d815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 327004910 0 0) 0x7fd9c80035b0 con 0x7fd9d815e6e0 2023-08-27T11:26:28.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d8146c20 msgr2=0x7fd9d8160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:28.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 --2- 172.21.15.112:0/4068682062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d8146c20 0x7fd9d8160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 >> v2:172.21.15.138:3300/0 conn(0x7fd9d80b3440 msgr2=0x7fd9d815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:28.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 --2- 172.21.15.112:0/4068682062 >> v2:172.21.15.138:3300/0 conn(0x7fd9d80b3440 0x7fd9d815e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d8161740 con 0x7fd9d815e6e0 2023-08-27T11:26:28.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fd9d0001b80 con 0x7fd9d815e6e0 2023-08-27T11:26:28.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 >> v1:172.21.15.112:6789/0 conn(0x7fd9d815e6e0 legacy=0x7fd9d806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:28.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 --2- 172.21.15.112:0/4068682062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c8003a00 0x7fd9c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 --2- 172.21.15.112:0/4068682062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9c8004850 0x7fd9c8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 --2- 172.21.15.112:0/4068682062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d8146c20 0x7fd9c8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d8147000 con 0x7fd9c8003a00 2023-08-27T11:26:28.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81614a0 con 0x7fd9c8004850 2023-08-27T11:26:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c800ba10 con 0x7fd9d8146c20 2023-08-27T11:26:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9dca97700 1 --2- 172.21.15.112:0/4068682062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c8003a00 0x7fd9c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9d7fff700 1 --2- 172.21.15.112:0/4068682062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9c8004850 0x7fd9c8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9dd298700 1 --2- 172.21.15.112:0/4068682062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d8146c20 0x7fd9c8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9dca97700 1 -- 172.21.15.112:0/4068682062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d8146c20 msgr2=0x7fd9c8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9dca97700 1 --2- 172.21.15.112:0/4068682062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d8146c20 0x7fd9c8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9dca97700 1 -- 172.21.15.112:0/4068682062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9c8004850 msgr2=0x7fd9c8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9dca97700 1 --2- 172.21.15.112:0/4068682062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9c8004850 0x7fd9c8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.601+0000 7fd9dca97700 1 -- 172.21.15.112:0/4068682062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c80035b0 con 0x7fd9c8003a00 2023-08-27T11:26:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.602+0000 7fd9dca97700 1 --2- 172.21.15.112:0/4068682062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c8003a00 0x7fd9c8003e20 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009b90 tx=0x7fd9c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c27865b5d2f0e13 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:28.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.602+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00e6c0 con 0x7fd9c8003a00 2023-08-27T11:26:28.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.602+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc00e820 con 0x7fd9c8003a00 2023-08-27T11:26:28.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.602+0000 7fd9d77fe700 1 -- 172.21.15.112:0/4068682062 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00eb90 con 0x7fd9c8003a00 2023-08-27T11:26:28.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.602+0000 7fd9decfb700 1 -- 172.21.15.112:0/4068682062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c8003a00 msgr2=0x7fd9c8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:28.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.602+0000 7fd9decfb700 1 --2- 172.21.15.112:0/4068682062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c8003a00 0x7fd9c8003e20 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009b90 tx=0x7fd9c8005880 comp rx=0 tx=0).stop 2023-08-27T11:26:28.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.602+0000 7fd9dd298700 1 -- 172.21.15.112:0/4068682062 reap_dead start 2023-08-27T11:26:28.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.602+0000 7fd9decfb700 1 -- 172.21.15.112:0/4068682062 shutdown_connections 2023-08-27T11:26:28.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.602+0000 7fd9decfb700 1 -- 172.21.15.112:0/4068682062 >> 172.21.15.112:0/4068682062 conn(0x7fd9d80b1e90 msgr2=0x7fd9d80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:28.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.602+0000 7fd9decfb700 1 -- 172.21.15.112:0/4068682062 shutdown_connections 2023-08-27T11:26:28.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.602+0000 7fd9decfb700 1 -- 172.21.15.112:0/4068682062 wait complete. 2023-08-27T11:26:28.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.604+0000 7fd9decfb700 1 Processor -- start 2023-08-27T11:26:28.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.604+0000 7fd9decfb700 1 -- start start 2023-08-27T11:26:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9decfb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9c8003a00 0x7fd9d8135cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9decfb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d81332a0 0x7fd9d8145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9decfb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9d81457b0 0x7fd9d80adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9decfb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d8145c40 con 0x7fd9d81457b0 2023-08-27T11:26:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9decfb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81432a0 con 0x7fd9d81332a0 2023-08-27T11:26:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9decfb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80aec70 con 0x7fd9c8003a00 2023-08-27T11:26:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9dd298700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9d81457b0 0x7fd9d80adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9dd298700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9d81457b0 0x7fd9d80adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32996/0 (socket says 172.21.15.112:32996) 2023-08-27T11:26:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9dd298700 1 -- 172.21.15.112:0/2265603021 learned_addr learned my addr 172.21.15.112:0/2265603021 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9dca97700 1 --2- 172.21.15.112:0/2265603021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9c8003a00 0x7fd9d8135cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9dd298700 1 -- 172.21.15.112:0/2265603021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9c8003a00 msgr2=0x7fd9d8135cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9dd298700 1 --2- 172.21.15.112:0/2265603021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9c8003a00 0x7fd9d8135cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9dd298700 1 -- 172.21.15.112:0/2265603021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d81332a0 msgr2=0x7fd9d8145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:28.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9dd298700 1 --2- 172.21.15.112:0/2265603021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d81332a0 0x7fd9d8145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.605+0000 7fd9dd298700 1 -- 172.21.15.112:0/2265603021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d81434f0 con 0x7fd9d81457b0 2023-08-27T11:26:28.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.606+0000 7fd9dd298700 1 --2- 172.21.15.112:0/2265603021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9d81457b0 0x7fd9d80adeb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd9d815eac0 tx=0x7fd9d0007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:28.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.606+0000 7fd9d57fa700 1 -- 172.21.15.112:0/2265603021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0007b70 con 0x7fd9d81457b0 2023-08-27T11:26:28.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.606+0000 7fd9d57fa700 1 -- 172.21.15.112:0/2265603021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d00133f0 con 0x7fd9d81457b0 2023-08-27T11:26:28.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.606+0000 7fd9decfb700 1 -- 172.21.15.112:0/2265603021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d81437d0 con 0x7fd9d81457b0 2023-08-27T11:26:28.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.606+0000 7fd9d57fa700 1 -- 172.21.15.112:0/2265603021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00137a0 con 0x7fd9d81457b0 2023-08-27T11:26:28.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.606+0000 7fd9decfb700 1 -- 172.21.15.112:0/2265603021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d81361f0 con 0x7fd9d81457b0 2023-08-27T11:26:28.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.607+0000 7fd9decfb700 1 -- 172.21.15.112:0/2265603021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d8067e90 con 0x7fd9d81457b0 2023-08-27T11:26:28.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.607+0000 7fd9d57fa700 1 -- 172.21.15.112:0/2265603021 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fd9d0013900 con 0x7fd9d81457b0 2023-08-27T11:26:28.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.607+0000 7fd9d57fa700 1 --2- 172.21.15.112:0/2265603021 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd9bc06a3b0 0x7fd9bc06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:28.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.608+0000 7fd9d57fa700 1 -- 172.21.15.112:0/2265603021 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd9d0089190 con 0x7fd9d81457b0 2023-08-27T11:26:28.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.608+0000 7fd9dca97700 1 --2- 172.21.15.112:0/2265603021 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd9bc06a3b0 0x7fd9bc06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:28.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.608+0000 7fd9dca97700 1 --2- 172.21.15.112:0/2265603021 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd9bc06a3b0 0x7fd9bc06c870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80af0a0 tx=0x7fd9cc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:28.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.612+0000 7fd9d57fa700 1 -- 172.21.15.112:0/2265603021 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd9d0054530 con 0x7fd9d81457b0 2023-08-27T11:26:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.881+0000 7fd9decfb700 1 -- 172.21.15.112:0/2265603021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd9d80b5070 con 0x7fd9d81457b0 2023-08-27T11:26:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.885+0000 7fd9d57fa700 1 -- 172.21.15.112:0/2265603021 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd9d80b5070 con 0x7fd9d81457b0 2023-08-27T11:26:28.886 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-08-27T11:26:28.887 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA== 2023-08-27T11:26:28.887 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2023-08-27T11:26:28.887 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2023-08-27T11:26:28.887 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:26:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.887+0000 7fd9decfb700 1 -- 172.21.15.112:0/2265603021 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd9bc06a3b0 msgr2=0x7fd9bc06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:28.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.887+0000 7fd9decfb700 1 --2- 172.21.15.112:0/2265603021 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd9bc06a3b0 0x7fd9bc06c870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80af0a0 tx=0x7fd9cc008030 comp rx=0 tx=0).stop 2023-08-27T11:26:28.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.887+0000 7fd9decfb700 1 -- 172.21.15.112:0/2265603021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9d81457b0 msgr2=0x7fd9d80adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:28.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.887+0000 7fd9decfb700 1 --2- 172.21.15.112:0/2265603021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9d81457b0 0x7fd9d80adeb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd9d815eac0 tx=0x7fd9d0007340 comp rx=0 tx=0).stop 2023-08-27T11:26:28.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.887+0000 7fd9decfb700 1 -- 172.21.15.112:0/2265603021 shutdown_connections 2023-08-27T11:26:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.887+0000 7fd9decfb700 1 --2- 172.21.15.112:0/2265603021 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd9bc06a3b0 0x7fd9bc06c870 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.887+0000 7fd9decfb700 1 --2- 172.21.15.112:0/2265603021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9d81457b0 0x7fd9d80adeb0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.887+0000 7fd9decfb700 1 --2- 172.21.15.112:0/2265603021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9c8003a00 0x7fd9d8135cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.887+0000 7fd9decfb700 1 --2- 172.21.15.112:0/2265603021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d81332a0 0x7fd9d8145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.887+0000 7fd9decfb700 1 -- 172.21.15.112:0/2265603021 >> 172.21.15.112:0/2265603021 conn(0x7fd9d80b1e90 msgr2=0x7fd9d80b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.887+0000 7fd9decfb700 1 -- 172.21.15.112:0/2265603021 shutdown_connections 2023-08-27T11:26:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:28.887+0000 7fd9decfb700 1 -- 172.21.15.112:0/2265603021 wait complete. 2023-08-27T11:26:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2023-08-27T11:26:28.900 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:26:29.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.020+0000 7f8239730700 1 Processor -- start 2023-08-27T11:26:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.020+0000 7f8239730700 1 -- start start 2023-08-27T11:26:29.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.020+0000 7f8239730700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f823406dfa0 0x7f8234070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.020+0000 7f8239730700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823406cb00 con 0x7f82340ce260 2023-08-27T11:26:29.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.020+0000 7f8239730700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82340707a0 con 0x7f823413f910 2023-08-27T11:26:29.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.020+0000 7f8239730700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8234070a80 con 0x7f823406dfa0 2023-08-27T11:26:29.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f82327fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f82340ce260 0x7f823406fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55618/0 (socket says 172.21.15.112:55618) 2023-08-27T11:26:29.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f82327fc700 1 -- 172.21.15.112:0/2286405928 learned_addr learned my addr 172.21.15.112:0/2286405928 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:29.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694240360 0 0) 0x7f823406cb00 con 0x7f82340ce260 2023-08-27T11:26:29.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f821c0046c0 con 0x7f82340ce260 2023-08-27T11:26:29.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f82337fe700 1 --2- 172.21.15.112:0/2286405928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f823406dfa0 0x7f8234070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1056092128 0 0) 0x7f821c0046c0 con 0x7f82340ce260 2023-08-27T11:26:29.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f823406dfa0 msgr2=0x7f8234070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 --2- 172.21.15.112:0/2286405928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f823406dfa0 0x7f8234070140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 >> v1:172.21.15.138:6789/0 conn(0x7f823413f910 legacy=0x7f82340cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8234070d60 con 0x7f82340ce260 2023-08-27T11:26:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f8224001670 con 0x7f82340ce260 2023-08-27T11:26:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 >> v1:172.21.15.112:6789/0 conn(0x7f82340ce260 legacy=0x7f823406fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 --2- 172.21.15.112:0/2286405928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c004b40 0x7f821c004f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 --2- 172.21.15.112:0/2286405928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f823406dfa0 0x7f821c005960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 --2- 172.21.15.112:0/2286405928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c005ea0 0x7f821c006320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8234070a80 con 0x7f821c004b40 2023-08-27T11:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c00cc00 con 0x7f821c005ea0 2023-08-27T11:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c00cdc0 con 0x7f823406dfa0 2023-08-27T11:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f8232ffd700 1 --2- 172.21.15.112:0/2286405928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c004b40 0x7f821c004f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f82327fc700 1 --2- 172.21.15.112:0/2286405928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f823406dfa0 0x7f821c005960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.021+0000 7f82337fe700 1 --2- 172.21.15.112:0/2286405928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c005ea0 0x7f821c006320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8232ffd700 1 -- 172.21.15.112:0/2286405928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f823406dfa0 msgr2=0x7f821c005960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8232ffd700 1 --2- 172.21.15.112:0/2286405928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f823406dfa0 0x7f821c005960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8232ffd700 1 -- 172.21.15.112:0/2286405928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c005ea0 msgr2=0x7f821c006320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8232ffd700 1 --2- 172.21.15.112:0/2286405928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c005ea0 0x7f821c006320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8232ffd700 1 -- 172.21.15.112:0/2286405928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0046c0 con 0x7f821c004b40 2023-08-27T11:26:29.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8232ffd700 1 --2- 172.21.15.112:0/2286405928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c004b40 0x7f821c004f60 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8228007fd0 tx=0x7f821c006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df06ef78048f5798 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:29.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82280052d0 con 0x7f821c004b40 2023-08-27T11:26:29.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8228005470 con 0x7f821c004b40 2023-08-27T11:26:29.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8231ffb700 1 -- 172.21.15.112:0/2286405928 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82280038c0 con 0x7f821c004b40 2023-08-27T11:26:29.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8239730700 1 -- 172.21.15.112:0/2286405928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c004b40 msgr2=0x7f821c004f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8239730700 1 --2- 172.21.15.112:0/2286405928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c004b40 0x7f821c004f60 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8228007fd0 tx=0x7f821c006990 comp rx=0 tx=0).stop 2023-08-27T11:26:29.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f82337fe700 1 -- 172.21.15.112:0/2286405928 reap_dead start 2023-08-27T11:26:29.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8239730700 1 -- 172.21.15.112:0/2286405928 shutdown_connections 2023-08-27T11:26:29.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8239730700 1 -- 172.21.15.112:0/2286405928 >> 172.21.15.112:0/2286405928 conn(0x7f82340b0bf0 msgr2=0x7f82340b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:29.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8239730700 1 -- 172.21.15.112:0/2286405928 shutdown_connections 2023-08-27T11:26:29.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.022+0000 7f8239730700 1 -- 172.21.15.112:0/2286405928 wait complete. 2023-08-27T11:26:29.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.024+0000 7f8239730700 1 Processor -- start 2023-08-27T11:26:29.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.024+0000 7f8239730700 1 -- start start 2023-08-27T11:26:29.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.024+0000 7f8239730700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c004b40 0x7f8234159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.024+0000 7f8239730700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82340aae40 0x7f82340ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f8239730700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82340ab2c0 0x7f8234131170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f8239730700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8234159590 con 0x7f82340ab2c0 2023-08-27T11:26:29.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f8239730700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82340ab820 con 0x7f821c004b40 2023-08-27T11:26:29.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f8239730700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82340abb00 con 0x7f82340aae40 2023-08-27T11:26:29.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f82337fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82340ab2c0 0x7f8234131170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f82337fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82340ab2c0 0x7f8234131170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33018/0 (socket says 172.21.15.112:33018) 2023-08-27T11:26:29.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f82337fe700 1 -- 172.21.15.112:0/2827375754 learned_addr learned my addr 172.21.15.112:0/2827375754 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:29.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f82327fc700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82340aae40 0x7f82340ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f8232ffd700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c004b40 0x7f8234159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f82337fe700 1 -- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82340aae40 msgr2=0x7f82340ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f82337fe700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82340aae40 0x7f82340ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f82337fe700 1 -- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c004b40 msgr2=0x7f8234159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f82337fe700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c004b40 0x7f8234159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f82337fe700 1 -- 172.21.15.112:0/2827375754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82340abde0 con 0x7f82340ab2c0 2023-08-27T11:26:29.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f82337fe700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82340ab2c0 0x7f8234131170 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8234142910 tx=0x7f822c011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:29.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f821b7fe700 1 -- 172.21.15.112:0/2827375754 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00e040 con 0x7f82340ab2c0 2023-08-27T11:26:29.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f821b7fe700 1 -- 172.21.15.112:0/2827375754 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c00f040 con 0x7f82340ab2c0 2023-08-27T11:26:29.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f821b7fe700 1 -- 172.21.15.112:0/2827375754 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00fc40 con 0x7f82340ab2c0 2023-08-27T11:26:29.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f8239730700 1 -- 172.21.15.112:0/2827375754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82340ac0c0 con 0x7f82340ab2c0 2023-08-27T11:26:29.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.025+0000 7f8239730700 1 -- 172.21.15.112:0/2827375754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8234133ed0 con 0x7f82340ab2c0 2023-08-27T11:26:29.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.026+0000 7f8239730700 1 -- 172.21.15.112:0/2827375754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8234134090 con 0x7f82340ab2c0 2023-08-27T11:26:29.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.027+0000 7f821b7fe700 1 -- 172.21.15.112:0/2827375754 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f822c022460 con 0x7f82340ab2c0 2023-08-27T11:26:29.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.027+0000 7f821b7fe700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f822006a3b0 0x7f822006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.027+0000 7f821b7fe700 1 -- 172.21.15.112:0/2827375754 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f822c088e60 con 0x7f82340ab2c0 2023-08-27T11:26:29.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.027+0000 7f8232ffd700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f822006a3b0 0x7f822006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.028+0000 7f8232ffd700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f822006a3b0 0x7f822006c870 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8228007fd0 tx=0x7f8228009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:29.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.031+0000 7f821b7fe700 1 -- 172.21.15.112:0/2827375754 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f822c057b80 con 0x7f82340ab2c0 2023-08-27T11:26:29.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.261+0000 7f8239730700 1 -- 172.21.15.112:0/2827375754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8234067a20 con 0x7f82340ab2c0 2023-08-27T11:26:29.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.261+0000 7f821b7fe700 1 -- 172.21.15.112:0/2827375754 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4886 (secure 0 0 0) 0x7f822c057710 con 0x7f82340ab2c0 2023-08-27T11:26:29.263 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:29.264 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/120353149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":120353149},{"type":"v1","addr":"172.21.15.138:6836","nonce":120353149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1171600247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1171600247},{"type":"v1","addr":"172.21.15.138:6837","nonce":1171600247}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3930299696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3930299696},{"type":"v1","addr":"172.21.15.112:6834","nonce":3930299696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:26:18.805364+0000","modified":"2023-08-27T11:26:22.433680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7041":{"gid":7041,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2614833373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2614833373},{"type":"v1","addr":"172.21.15.112:6835","nonce":2614833373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-27T11:26:29.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.264+0000 7f8239730700 1 -- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f822006a3b0 msgr2=0x7f822006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.264+0000 7f8239730700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f822006a3b0 0x7f822006c870 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8228007fd0 tx=0x7f8228009030 comp rx=0 tx=0).stop 2023-08-27T11:26:29.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.264+0000 7f8239730700 1 -- 172.21.15.112:0/2827375754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82340ab2c0 msgr2=0x7f8234131170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.264+0000 7f8239730700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82340ab2c0 0x7f8234131170 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8234142910 tx=0x7f822c011ed0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.264+0000 7f8239730700 1 -- 172.21.15.112:0/2827375754 shutdown_connections 2023-08-27T11:26:29.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.264+0000 7f8239730700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f822006a3b0 0x7f822006c870 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.264+0000 7f8239730700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82340ab2c0 0x7f8234131170 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.264+0000 7f8239730700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c004b40 0x7f8234159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.264+0000 7f8239730700 1 --2- 172.21.15.112:0/2827375754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82340aae40 0x7f82340ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.264+0000 7f8239730700 1 -- 172.21.15.112:0/2827375754 >> 172.21.15.112:0/2827375754 conn(0x7f82340b0bf0 msgr2=0x7f823413e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:29.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.264+0000 7f8239730700 1 -- 172.21.15.112:0/2827375754 shutdown_connections 2023-08-27T11:26:29.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.264+0000 7f8239730700 1 -- 172.21.15.112:0/2827375754 wait complete. 2023-08-27T11:26:29.268 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 58 2023-08-27T11:26:29.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:26:18.805364+0000', 'modified': '2023-08-27T11:26:22.433680+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7041}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7041': {'gid': 7041, 'name': 'c', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/2614833373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6832', 'nonce': 2614833373}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 2614833373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:26:29.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:26:29.681 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-27T11:26:29.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.801+0000 7fc690b22700 1 Processor -- start 2023-08-27T11:26:29.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.801+0000 7fc690b22700 1 -- start start 2023-08-27T11:26:29.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.802+0000 7fc690b22700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc68c0b1400 0x7fc68c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.802+0000 7fc690b22700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc68c0b17e0 0x7fc68c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.802+0000 7fc690b22700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc68c161850 con 0x7fc68c161ce0 2023-08-27T11:26:29.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.802+0000 7fc690b22700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc68c06e7d0 con 0x7fc68c0b17e0 2023-08-27T11:26:29.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.802+0000 7fc690b22700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc68c0cfc10 con 0x7fc68c0b1400 2023-08-27T11:26:29.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.802+0000 7fc689d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc68c0b17e0 0x7fc68c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.802+0000 7fc689d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc68c0b17e0 0x7fc68c06e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33022/0 (socket says 172.21.15.112:33022) 2023-08-27T11:26:29.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.802+0000 7fc68ad9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc68c161ce0 0x7fc68c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55640/0 (socket says 172.21.15.112:55640) 2023-08-27T11:26:29.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.802+0000 7fc689d9b700 1 -- 172.21.15.112:0/3312750567 learned_addr learned my addr 172.21.15.112:0/3312750567 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:29.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc689d9b700 1 -- 172.21.15.112:0/3312750567 >> v2:172.21.15.138:3300/0 conn(0x7fc68c0b1400 msgr2=0x7fc68c0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc689d9b700 1 --2- 172.21.15.112:0/3312750567 >> v2:172.21.15.138:3300/0 conn(0x7fc68c0b1400 0x7fc68c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc689d9b700 1 -- 172.21.15.112:0/3312750567 >> v1:172.21.15.112:6789/0 conn(0x7fc68c161ce0 legacy=0x7fc68c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc689d9b700 1 -- 172.21.15.112:0/3312750567 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c161200 con 0x7fc68c0b17e0 2023-08-27T11:26:29.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc68959a700 1 -- 172.21.15.112:0/3312750567 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328728480 0 0) 0x7fc68c161850 con 0x7fc68c161ce0 2023-08-27T11:26:29.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc689d9b700 1 --2- 172.21.15.112:0/3312750567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c0b17e0 0x7fc68c06e290 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc68c070370 tx=0x7fc67c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0307e505eac52b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:29.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc68959a700 1 -- 172.21.15.112:0/3312750567 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c00b870 con 0x7fc68c0b17e0 2023-08-27T11:26:29.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc68959a700 1 -- 172.21.15.112:0/3312750567 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67c00b9d0 con 0x7fc68c0b17e0 2023-08-27T11:26:29.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc68959a700 1 -- 172.21.15.112:0/3312750567 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c00bd60 con 0x7fc68c0b17e0 2023-08-27T11:26:29.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc690b22700 1 -- 172.21.15.112:0/3312750567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c0b17e0 msgr2=0x7fc68c06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc690b22700 1 --2- 172.21.15.112:0/3312750567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c0b17e0 0x7fc68c06e290 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc68c070370 tx=0x7fc67c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc690b22700 1 -- 172.21.15.112:0/3312750567 shutdown_connections 2023-08-27T11:26:29.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc690b22700 1 --2- 172.21.15.112:0/3312750567 >> v2:172.21.15.138:3300/0 conn(0x7fc68c0b1400 0x7fc68c0cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc690b22700 1 --2- 172.21.15.112:0/3312750567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c0b17e0 0x7fc68c06e290 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.803+0000 7fc690b22700 1 -- 172.21.15.112:0/3312750567 >> 172.21.15.112:0/3312750567 conn(0x7fc68c0d0000 msgr2=0x7fc68c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:29.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.804+0000 7fc690b22700 1 -- 172.21.15.112:0/3312750567 shutdown_connections 2023-08-27T11:26:29.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.804+0000 7fc690b22700 1 -- 172.21.15.112:0/3312750567 wait complete. 2023-08-27T11:26:29.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.805+0000 7fc690b22700 1 Processor -- start 2023-08-27T11:26:29.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.805+0000 7fc690b22700 1 -- start start 2023-08-27T11:26:29.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc690b22700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c0b1400 0x7fc68c154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc690b22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc68c0b17e0 0x7fc68c0c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc690b22700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc68c161ce0 0x7fc68c0c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc690b22700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c0c70d0 con 0x7fc68c0b1400 2023-08-27T11:26:29.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc690b22700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c0c4a60 con 0x7fc68c0b17e0 2023-08-27T11:26:29.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc690b22700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c0c4d40 con 0x7fc68c161ce0 2023-08-27T11:26:29.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc68a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c0b1400 0x7fc68c154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc68a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c0b1400 0x7fc68c154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33028/0 (socket says 172.21.15.112:33028) 2023-08-27T11:26:29.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc68a59c700 1 -- 172.21.15.112:0/3237101225 learned_addr learned my addr 172.21.15.112:0/3237101225 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:29.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc689d9b700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc68c0b17e0 0x7fc68c0c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc68ad9d700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc68c161ce0 0x7fc68c0c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc68a59c700 1 -- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc68c161ce0 msgr2=0x7fc68c0c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc68a59c700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc68c161ce0 0x7fc68c0c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc68a59c700 1 -- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc68c0b17e0 msgr2=0x7fc68c0c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc68a59c700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc68c0b17e0 0x7fc68c0c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc68a59c700 1 -- 172.21.15.112:0/3237101225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c009550 con 0x7fc68c0b1400 2023-08-27T11:26:29.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc68a59c700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c0b1400 0x7fc68c154e70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fc68000b530 tx=0x7fc68000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:29.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc67affd700 1 -- 172.21.15.112:0/3237101225 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680011830 con 0x7fc68c0b1400 2023-08-27T11:26:29.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc67affd700 1 -- 172.21.15.112:0/3237101225 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc680011990 con 0x7fc68c0b1400 2023-08-27T11:26:29.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc690b22700 1 -- 172.21.15.112:0/3237101225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68c0c5020 con 0x7fc68c0b1400 2023-08-27T11:26:29.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc67affd700 1 -- 172.21.15.112:0/3237101225 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680011d40 con 0x7fc68c0b1400 2023-08-27T11:26:29.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.806+0000 7fc690b22700 1 -- 172.21.15.112:0/3237101225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68c1553b0 con 0x7fc68c0b1400 2023-08-27T11:26:29.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.808+0000 7fc67affd700 1 -- 172.21.15.112:0/3237101225 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fc68001a400 con 0x7fc68c0b1400 2023-08-27T11:26:29.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.808+0000 7fc67affd700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc66c06a3b0 0x7fc66c06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.808+0000 7fc67affd700 1 -- 172.21.15.112:0/3237101225 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc680088880 con 0x7fc68c0b1400 2023-08-27T11:26:29.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.808+0000 7fc689d9b700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc66c06a3b0 0x7fc66c06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.809+0000 7fc689d9b700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc66c06a3b0 0x7fc66c06c870 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0b1bf0 tx=0x7fc67c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:29.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.810+0000 7fc69cce4b80 1 Processor -- start 2023-08-27T11:26:29.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.810+0000 7fc69cce4b80 1 -- start start 2023-08-27T11:26:29.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc69cce4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453725320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc69cce4b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a453736960 0x55a453721b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc69cce4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4536f83d0 0x55a4537220b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc69cce4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a453653730 con 0x55a453724f40 2023-08-27T11:26:29.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc69cce4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a453651050 con 0x55a4536f83d0 2023-08-27T11:26:29.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc69cce4b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a45364c940 con 0x55a453736960 2023-08-27T11:26:29.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc68a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453725320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc68a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453725320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33036/0 (socket says 172.21.15.112:33036) 2023-08-27T11:26:29.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc68a59c700 1 -- 172.21.15.112:0/1621222378 learned_addr learned my addr 172.21.15.112:0/1621222378 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:29.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc689d9b700 1 --2- 172.21.15.112:0/1621222378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a453736960 0x55a453721b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc68ad9d700 1 --2- 172.21.15.112:0/1621222378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4536f83d0 0x55a4537220b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc68a59c700 1 -- 172.21.15.112:0/1621222378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a453736960 msgr2=0x55a453721b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc68a59c700 1 --2- 172.21.15.112:0/1621222378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a453736960 0x55a453721b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc68a59c700 1 -- 172.21.15.112:0/1621222378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4536f83d0 msgr2=0x55a4537220b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc68a59c700 1 --2- 172.21.15.112:0/1621222378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4536f83d0 0x55a4537220b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc68a59c700 1 -- 172.21.15.112:0/1621222378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4534cc5e0 con 0x55a453724f40 2023-08-27T11:26:29.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc68a59c700 1 --2- 172.21.15.112:0/1621222378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453725320 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc6800576e0 tx=0x7fc680058070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8cba9450296051 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:29.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc66bfff700 1 -- 172.21.15.112:0/1621222378 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68005f6f0 con 0x55a453724f40 2023-08-27T11:26:29.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc66bfff700 1 -- 172.21.15.112:0/1621222378 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc680056ed0 con 0x55a453724f40 2023-08-27T11:26:29.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.811+0000 7fc66bfff700 1 -- 172.21.15.112:0/1621222378 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68005f260 con 0x55a453724f40 2023-08-27T11:26:29.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.812+0000 7fc69cce4b80 1 -- 172.21.15.112:0/1621222378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 msgr2=0x55a453725320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.812+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/1621222378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453725320 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc6800576e0 tx=0x7fc680058070 comp rx=0 tx=0).stop 2023-08-27T11:26:29.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.812+0000 7fc69cce4b80 1 -- 172.21.15.112:0/1621222378 shutdown_connections 2023-08-27T11:26:29.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.812+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/1621222378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4536f83d0 0x55a4537220b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.812+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/1621222378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453725320 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.812+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/1621222378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a453736960 0x55a453721b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.812+0000 7fc69cce4b80 1 -- 172.21.15.112:0/1621222378 >> 172.21.15.112:0/1621222378 conn(0x55a453658c30 msgr2=0x55a45373fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:29.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.812+0000 7fc69cce4b80 1 -- 172.21.15.112:0/1621222378 shutdown_connections 2023-08-27T11:26:29.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.812+0000 7fc69cce4b80 1 -- 172.21.15.112:0/1621222378 wait complete. 2023-08-27T11:26:29.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.813+0000 7fc69cce4b80 1 -- start start 2023-08-27T11:26:29.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.813+0000 7fc69cce4b80 1 Processor -- start 2023-08-27T11:26:29.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc69cce4b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 0x55a453720b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc69cce4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453721060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc69cce4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 0x55a4537267e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc69cce4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4535eab60 con 0x55a453724f40 2023-08-27T11:26:29.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc69cce4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4535ea860 con 0x55a453736960 2023-08-27T11:26:29.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc69cce4b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4534ce8d0 con 0x55a4536f83d0 2023-08-27T11:26:29.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc69cce4b80 10 client.0 _ll_register_callbacks cb 0x55a4535e8630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:26:29.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc689d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453721060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc689d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453721060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33044/0 (socket says 172.21.15.112:33044) 2023-08-27T11:26:29.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc689d9b700 1 -- 172.21.15.112:0/2364165606 learned_addr learned my addr 172.21.15.112:0/2364165606 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:29.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc68a59c700 1 --2- 172.21.15.112:0/2364165606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 0x55a453720b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc68ad9d700 1 --2- 172.21.15.112:0/2364165606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 0x55a4537267e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc689d9b700 1 -- 172.21.15.112:0/2364165606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 msgr2=0x55a453720b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc689d9b700 1 --2- 172.21.15.112:0/2364165606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 0x55a453720b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc689d9b700 1 -- 172.21.15.112:0/2364165606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 msgr2=0x55a4537267e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc689d9b700 1 --2- 172.21.15.112:0/2364165606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 0x55a4537267e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc689d9b700 1 -- 172.21.15.112:0/2364165606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68c1553b0 con 0x55a453724f40 2023-08-27T11:26:29.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc689d9b700 1 --2- 172.21.15.112:0/2364165606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453721060 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009fd0 tx=0x7fc67c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52dd74c346fdd2f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:29.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc6697fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:26:29.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.814+0000 7fc6697fa700 1 -- 172.21.15.112:0/2364165606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c02e040 con 0x55a453724f40 2023-08-27T11:26:29.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc6697fa700 1 -- 172.21.15.112:0/2364165606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc67c034c40 con 0x55a453724f40 2023-08-27T11:26:29.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2364165606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4536ed4f0 con 0x55a453724f40 2023-08-27T11:26:29.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc6697fa700 1 -- 172.21.15.112:0/2364165606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc67c020070 con 0x55a453724f40 2023-08-27T11:26:29.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc69cce4b80 10 client.7224 fetch_fsmap learned FSMap version 58 2023-08-27T11:26:29.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2364165606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55a453664750 con 0x55a453724f40 2023-08-27T11:26:29.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc6697fa700 1 -- 172.21.15.112:0/2364165606 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a4536ed4f0 con 0x55a453724f40 2023-08-27T11:26:29.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc6697fa700 1 -- 172.21.15.112:0/2364165606 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc67c003a70 con 0x55a453724f40 2023-08-27T11:26:29.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc69cce4b80 10 client.7224 fetch_fsmap finished waiting for FSMap version 58 2023-08-27T11:26:29.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc69cce4b80 10 client.7224 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:26:29.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc69cce4b80 20 client.7224 populate_metadata read hostname 'smithi112' 2023-08-27T11:26:29.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2364165606 >> [v2:172.21.15.112:6832/2614833373,v1:172.21.15.112:6835/2614833373] conn(0x55a453743eb0 0x55a45371ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc69cce4b80 4 client.7224 mds_command: new command op to 7041 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:26:29.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2364165606 --> [v2:172.21.15.112:6832/2614833373,v1:172.21.15.112:6835/2614833373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a453645920 con 0x55a453743eb0 2023-08-27T11:26:29.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc68a59c700 1 --2- 172.21.15.112:0/2364165606 >> [v2:172.21.15.112:6832/2614833373,v1:172.21.15.112:6835/2614833373] conn(0x55a453743eb0 0x55a45371ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc6697fa700 10 client.7224 ms_handle_connect on v2:172.21.15.112:6832/2614833373 2023-08-27T11:26:29.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.815+0000 7fc68a59c700 1 --2- 172.21.15.112:0/2364165606 >> [v2:172.21.15.112:6832/2614833373,v1:172.21.15.112:6835/2614833373] conn(0x55a453743eb0 0x55a45371ffc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b16c17c79b95de server_cookie=a3fa349eef4306b4 in_seq=0 out_seq=0 2023-08-27T11:26:29.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc6697fa700 1 -- 172.21.15.112:0/2364165606 <== mds.0 v2:172.21.15.112:6832/2614833373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a453645920 con 0x55a453743eb0 2023-08-27T11:26:29.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc6697fa700 10 client.7224 handle_command_reply: tid=0 2023-08-27T11:26:29.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc69cce4b80 1 client.7224 shutdown 2023-08-27T11:26:29.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2364165606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 msgr2=0x55a453721060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2364165606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453721060 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009fd0 tx=0x7fc67c00e000 comp rx=0 tx=0).stop 2023-08-27T11:26:29.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2364165606 shutdown_connections 2023-08-27T11:26:29.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2364165606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 0x55a4537267e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2364165606 >> [v2:172.21.15.112:6832/2614833373,v1:172.21.15.112:6835/2614833373] conn(0x55a453743eb0 0x55a45371ffc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2364165606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 0x55a453720b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2364165606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453721060 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc6697fa700 0 client.7224 ms_handle_reset on v2:172.21.15.112:6832/2614833373 2023-08-27T11:26:29.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2364165606 >> 172.21.15.112:0/2364165606 conn(0x55a453658c30 msgr2=0x55a45371eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:29.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2364165606 shutdown_connections 2023-08-27T11:26:29.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2364165606 wait complete. 2023-08-27T11:26:29.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.816+0000 7fc69cce4b80 20 client.7224 trim_cache size 0 max 16384 2023-08-27T11:26:29.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.834+0000 7fc69cce4b80 1 Processor -- start 2023-08-27T11:26:29.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.834+0000 7fc69cce4b80 1 -- start start 2023-08-27T11:26:29.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.834+0000 7fc69cce4b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 0x55a453727320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.834+0000 7fc69cce4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453725320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.834+0000 7fc69cce4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 0x55a453725860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.834+0000 7fc69cce4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a45371e960 con 0x55a453724f40 2023-08-27T11:26:29.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.834+0000 7fc69cce4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4537232c0 con 0x55a453736960 2023-08-27T11:26:29.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.834+0000 7fc69cce4b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4537210b0 con 0x55a4536f83d0 2023-08-27T11:26:29.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc689d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453725320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc689d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453725320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33050/0 (socket says 172.21.15.112:33050) 2023-08-27T11:26:29.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc689d9b700 1 -- 172.21.15.112:0/2383634152 learned_addr learned my addr 172.21.15.112:0/2383634152 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:29.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc68a59c700 1 --2- 172.21.15.112:0/2383634152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 0x55a453727320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc68ad9d700 1 --2- 172.21.15.112:0/2383634152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 0x55a453725860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc689d9b700 1 -- 172.21.15.112:0/2383634152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 msgr2=0x55a453727320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc689d9b700 1 --2- 172.21.15.112:0/2383634152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 0x55a453727320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc689d9b700 1 -- 172.21.15.112:0/2383634152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 msgr2=0x55a453725860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc689d9b700 1 --2- 172.21.15.112:0/2383634152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 0x55a453725860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc689d9b700 1 -- 172.21.15.112:0/2383634152 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a453478d90 con 0x55a453724f40 2023-08-27T11:26:29.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc68a59c700 1 --2- 172.21.15.112:0/2383634152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 0x55a453727320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:29.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc68ad9d700 1 --2- 172.21.15.112:0/2383634152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 0x55a453725860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:29.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc689d9b700 1 --2- 172.21.15.112:0/2383634152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453725320 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009270 tx=0x7fc67c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f135221be9d334c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:29.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc678ff9700 1 -- 172.21.15.112:0/2383634152 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c02e040 con 0x55a453724f40 2023-08-27T11:26:29.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc678ff9700 1 -- 172.21.15.112:0/2383634152 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67c03d670 con 0x55a453724f40 2023-08-27T11:26:29.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.835+0000 7fc678ff9700 1 -- 172.21.15.112:0/2383634152 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c03da00 con 0x55a453724f40 2023-08-27T11:26:29.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.836+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2383634152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 msgr2=0x55a453725320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.836+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2383634152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453725320 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009270 tx=0x7fc67c020000 comp rx=0 tx=0).stop 2023-08-27T11:26:29.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.836+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2383634152 shutdown_connections 2023-08-27T11:26:29.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.836+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2383634152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 0x55a453725860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.836+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2383634152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 0x55a453727320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.836+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2383634152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453725320 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.836+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2383634152 >> 172.21.15.112:0/2383634152 conn(0x55a453658c30 msgr2=0x55a4536f4690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:29.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.836+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2383634152 shutdown_connections 2023-08-27T11:26:29.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.836+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2383634152 wait complete. 2023-08-27T11:26:29.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.836+0000 7fc69cce4b80 1 -- start start 2023-08-27T11:26:29.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.836+0000 7fc69cce4b80 1 Processor -- start 2023-08-27T11:26:29.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.837+0000 7fc69cce4b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 0x55a45373f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.837+0000 7fc69cce4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453739b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.837+0000 7fc69cce4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 0x55a4537a6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.837+0000 7fc69cce4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a45373f7b0 con 0x55a453724f40 2023-08-27T11:26:29.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.837+0000 7fc69cce4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a45373a0a0 con 0x55a453736960 2023-08-27T11:26:29.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.837+0000 7fc69cce4b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4537a7680 con 0x55a4536f83d0 2023-08-27T11:26:29.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.837+0000 7fc69cce4b80 10 client.0 _ll_register_callbacks cb 0x55a4535e8630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:26:29.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.837+0000 7fc689d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453739b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.837+0000 7fc689d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453739b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33056/0 (socket says 172.21.15.112:33056) 2023-08-27T11:26:29.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.837+0000 7fc689d9b700 1 -- 172.21.15.112:0/2470097625 learned_addr learned my addr 172.21.15.112:0/2470097625 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:29.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.837+0000 7fc68a59c700 1 --2- 172.21.15.112:0/2470097625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 0x55a45373f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.837+0000 7fc68ad9d700 1 --2- 172.21.15.112:0/2470097625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 0x55a4537a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc689d9b700 1 -- 172.21.15.112:0/2470097625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 msgr2=0x55a45373f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc689d9b700 1 --2- 172.21.15.112:0/2470097625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 0x55a45373f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc689d9b700 1 -- 172.21.15.112:0/2470097625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 msgr2=0x55a4537a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc689d9b700 1 --2- 172.21.15.112:0/2470097625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 0x55a4537a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc689d9b700 1 -- 172.21.15.112:0/2470097625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc67c034e70 con 0x55a453724f40 2023-08-27T11:26:29.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc689d9b700 1 --2- 172.21.15.112:0/2470097625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453739b60 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0094e0 tx=0x7fc67c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23222732a1eed36f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:29.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc6697fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:26:29.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc6697fa700 1 -- 172.21.15.112:0/2470097625 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c020040 con 0x55a453724f40 2023-08-27T11:26:29.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc6697fa700 1 -- 172.21.15.112:0/2470097625 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc67c03dda0 con 0x55a453724f40 2023-08-27T11:26:29.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2470097625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc680056ed0 con 0x55a453724f40 2023-08-27T11:26:29.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc6697fa700 1 -- 172.21.15.112:0/2470097625 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc67c02e040 con 0x55a453724f40 2023-08-27T11:26:29.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc69cce4b80 10 client.7230 fetch_fsmap learned FSMap version 58 2023-08-27T11:26:29.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2470097625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55a453461c10 con 0x55a453724f40 2023-08-27T11:26:29.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc6697fa700 1 -- 172.21.15.112:0/2470097625 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc680056ed0 con 0x55a453724f40 2023-08-27T11:26:29.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc6697fa700 1 -- 172.21.15.112:0/2470097625 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc67c050470 con 0x55a453724f40 2023-08-27T11:26:29.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc69cce4b80 10 client.7230 fetch_fsmap finished waiting for FSMap version 58 2023-08-27T11:26:29.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc69cce4b80 10 client.7230 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:26:29.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc69cce4b80 20 client.7230 populate_metadata read hostname 'smithi112' 2023-08-27T11:26:29.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2470097625 >> [v2:172.21.15.112:6832/2614833373,v1:172.21.15.112:6835/2614833373] conn(0x55a4537a6f00 0x55a453725320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc69cce4b80 4 client.7230 mds_command: new command op to 7041 tid=0[{"prefix": "status"}] 2023-08-27T11:26:29.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2470097625 --> [v2:172.21.15.112:6832/2614833373,v1:172.21.15.112:6835/2614833373] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a453606000 con 0x55a4537a6f00 2023-08-27T11:26:29.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.838+0000 7fc68a59c700 1 --2- 172.21.15.112:0/2470097625 >> [v2:172.21.15.112:6832/2614833373,v1:172.21.15.112:6835/2614833373] conn(0x55a4537a6f00 0x55a453725320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc6697fa700 10 client.7230 ms_handle_connect on v2:172.21.15.112:6832/2614833373 2023-08-27T11:26:29.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc68a59c700 1 --2- 172.21.15.112:0/2470097625 >> [v2:172.21.15.112:6832/2614833373,v1:172.21.15.112:6835/2614833373] conn(0x55a4537a6f00 0x55a453725320 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b334b9b1c9741c server_cookie=2684405dcf2b1e55 in_seq=0 out_seq=0 2023-08-27T11:26:29.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc6697fa700 1 -- 172.21.15.112:0/2470097625 <== mds.0 v2:172.21.15.112:6832/2614833373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x55a453606000 con 0x55a4537a6f00 2023-08-27T11:26:29.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc6697fa700 10 client.7230 handle_command_reply: tid=0 2023-08-27T11:26:29.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc69cce4b80 1 client.7230 shutdown 2023-08-27T11:26:29.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2470097625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 msgr2=0x55a453739b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2470097625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453739b60 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0094e0 tx=0x7fc67c0047c0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2470097625 shutdown_connections 2023-08-27T11:26:29.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2470097625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a453736960 0x55a4537a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2470097625 >> [v2:172.21.15.112:6832/2614833373,v1:172.21.15.112:6835/2614833373] conn(0x55a4537a6f00 0x55a453725320 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2470097625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4536f83d0 0x55a45373f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc6697fa700 0 client.7230 ms_handle_reset on v2:172.21.15.112:6832/2614833373 2023-08-27T11:26:29.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc69cce4b80 1 --2- 172.21.15.112:0/2470097625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a453724f40 0x55a453739b60 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2470097625 >> 172.21.15.112:0/2470097625 conn(0x55a453658c30 msgr2=0x55a4536fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:29.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2470097625 shutdown_connections 2023-08-27T11:26:29.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc69cce4b80 1 -- 172.21.15.112:0/2470097625 wait complete. 2023-08-27T11:26:29.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.839+0000 7fc69cce4b80 20 client.7230 trim_cache size 0 max 16384 2023-08-27T11:26:29.865 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:26:29.865 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e099a24f-c1cd-4f11-ac4f-ecacc10e5e98", 2023-08-27T11:26:29.866 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2023-08-27T11:26:29.866 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 7041, 2023-08-27T11:26:29.866 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2023-08-27T11:26:29.866 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2023-08-27T11:26:29.866 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2023-08-27T11:26:29.867 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 10.433776214, 2023-08-27T11:26:29.867 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 58, 2023-08-27T11:26:29.867 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 90, 2023-08-27T11:26:29.867 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 90, 2023-08-27T11:26:29.867 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 16.33793709 2023-08-27T11:26:29.868 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:26:29.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.840+0000 7fc690b22700 1 -- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc66c06a3b0 msgr2=0x7fc66c06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.840+0000 7fc690b22700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc66c06a3b0 0x7fc66c06c870 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0b1bf0 tx=0x7fc67c008030 comp rx=0 tx=0).stop 2023-08-27T11:26:29.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.840+0000 7fc690b22700 1 -- 172.21.15.112:0/3237101225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c0b1400 msgr2=0x7fc68c154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.840+0000 7fc690b22700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c0b1400 0x7fc68c154e70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fc68000b530 tx=0x7fc68000b840 comp rx=0 tx=0).stop 2023-08-27T11:26:29.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.840+0000 7fc690b22700 1 -- 172.21.15.112:0/3237101225 shutdown_connections 2023-08-27T11:26:29.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.840+0000 7fc690b22700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc66c06a3b0 0x7fc66c06c870 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.840+0000 7fc690b22700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc68c161ce0 0x7fc68c0c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.840+0000 7fc690b22700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c0b1400 0x7fc68c154e70 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.840+0000 7fc690b22700 1 --2- 172.21.15.112:0/3237101225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc68c0b17e0 0x7fc68c0c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.840+0000 7fc690b22700 1 -- 172.21.15.112:0/3237101225 >> 172.21.15.112:0/3237101225 conn(0x7fc68c0d0000 msgr2=0x7fc68c06ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:29.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.840+0000 7fc690b22700 1 -- 172.21.15.112:0/3237101225 shutdown_connections 2023-08-27T11:26:29.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.840+0000 7fc690b22700 1 -- 172.21.15.112:0/3237101225 wait complete. 2023-08-27T11:26:29.872 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:26:29.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.990+0000 7f9e6cc8f700 1 Processor -- start 2023-08-27T11:26:29.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.990+0000 7f9e6cc8f700 1 -- start start 2023-08-27T11:26:29.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.991+0000 7f9e6cc8f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e680b1400 0x7f9e680cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.991+0000 7f9e6cc8f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e680b17e0 0x7f9e6806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.991+0000 7f9e6cc8f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9e68161ce0 0x7f9e6806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:29.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.991+0000 7f9e6cc8f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e68161850 con 0x7f9e680b17e0 2023-08-27T11:26:29.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.991+0000 7f9e6cc8f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6806e7d0 con 0x7f9e68161ce0 2023-08-27T11:26:29.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.991+0000 7f9e6cc8f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e680cfc10 con 0x7f9e680b1400 2023-08-27T11:26:29.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.991+0000 7f9e65d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e680b17e0 0x7f9e6806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.991+0000 7f9e65d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e680b17e0 0x7f9e6806e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33064/0 (socket says 172.21.15.112:33064) 2023-08-27T11:26:29.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.991+0000 7f9e65d9b700 1 -- 172.21.15.112:0/3840902907 learned_addr learned my addr 172.21.15.112:0/3840902907 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:29.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.992+0000 7f9e66d9d700 1 --2- 172.21.15.112:0/3840902907 >> v2:172.21.15.138:3300/0 conn(0x7f9e68161ce0 0x7f9e6806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:29.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.992+0000 7f9e65d9b700 1 -- 172.21.15.112:0/3840902907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e680b1400 msgr2=0x7f9e680cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.992+0000 7f9e65d9b700 1 --2- 172.21.15.112:0/3840902907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e680b1400 0x7f9e680cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.992+0000 7f9e65d9b700 1 -- 172.21.15.112:0/3840902907 >> v2:172.21.15.138:3300/0 conn(0x7f9e68161ce0 msgr2=0x7f9e6806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:29.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.992+0000 7f9e65d9b700 1 --2- 172.21.15.112:0/3840902907 >> v2:172.21.15.138:3300/0 conn(0x7f9e68161ce0 0x7f9e6806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.992+0000 7f9e65d9b700 1 -- 172.21.15.112:0/3840902907 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e68161200 con 0x7f9e680b17e0 2023-08-27T11:26:29.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.992+0000 7f9e65d9b700 1 --2- 172.21.15.112:0/3840902907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e680b17e0 0x7f9e6806e290 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9e680ca550 tx=0x7f9e58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ff771d61cf9eea server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:29.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.992+0000 7f9e6559a700 1 -- 172.21.15.112:0/3840902907 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5800b870 con 0x7f9e680b17e0 2023-08-27T11:26:29.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.992+0000 7f9e6559a700 1 -- 172.21.15.112:0/3840902907 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5800b9d0 con 0x7f9e680b17e0 2023-08-27T11:26:29.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.992+0000 7f9e6559a700 1 -- 172.21.15.112:0/3840902907 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5800bd60 con 0x7f9e680b17e0 2023-08-27T11:26:29.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.992+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/3840902907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e680b17e0 msgr2=0x7f9e6806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:29.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.992+0000 7f9e6cc8f700 1 --2- 172.21.15.112:0/3840902907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e680b17e0 0x7f9e6806e290 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9e680ca550 tx=0x7f9e58009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.993+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/3840902907 shutdown_connections 2023-08-27T11:26:29.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.993+0000 7f9e6cc8f700 1 --2- 172.21.15.112:0/3840902907 >> v2:172.21.15.138:3300/0 conn(0x7f9e68161ce0 0x7f9e6806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.993+0000 7f9e6cc8f700 1 --2- 172.21.15.112:0/3840902907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e680b1400 0x7f9e680cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.993+0000 7f9e6cc8f700 1 --2- 172.21.15.112:0/3840902907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e680b17e0 0x7f9e6806e290 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:29.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.993+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/3840902907 >> 172.21.15.112:0/3840902907 conn(0x7f9e680d0000 msgr2=0x7f9e680d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:29.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.993+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/3840902907 shutdown_connections 2023-08-27T11:26:29.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.993+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/3840902907 wait complete. 2023-08-27T11:26:30.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.994+0000 7f9e6cc8f700 1 Processor -- start 2023-08-27T11:26:30.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.994+0000 7f9e6cc8f700 1 -- start start 2023-08-27T11:26:30.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e6cc8f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e680b1400 0x7f9e680c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:30.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e6cc8f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e680b17e0 0x7f9e6815ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:30.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e6cc8f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e68161ce0 0x7f9e681602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:30.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e6cc8f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6815f270 con 0x7f9e68161ce0 2023-08-27T11:26:30.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e6cc8f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e680c4e10 con 0x7f9e680b17e0 2023-08-27T11:26:30.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e6cc8f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e680c50f0 con 0x7f9e680b1400 2023-08-27T11:26:30.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e66d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e68161ce0 0x7f9e681602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:30.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e66d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e68161ce0 0x7f9e681602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33072/0 (socket says 172.21.15.112:33072) 2023-08-27T11:26:30.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e66d9d700 1 -- 172.21.15.112:0/492836572 learned_addr learned my addr 172.21.15.112:0/492836572 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:30.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e65d9b700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e680b17e0 0x7f9e6815ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:30.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e6659c700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e680b1400 0x7f9e680c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:30.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e66d9d700 1 -- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e680b1400 msgr2=0x7f9e680c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:30.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e66d9d700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e680b1400 0x7f9e680c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:30.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e66d9d700 1 -- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e680b17e0 msgr2=0x7f9e6815ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:30.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e66d9d700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e680b17e0 0x7f9e6815ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:30.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e66d9d700 1 -- 172.21.15.112:0/492836572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e58009550 con 0x7f9e68161ce0 2023-08-27T11:26:30.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e6659c700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e680b1400 0x7f9e680c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:30.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e65d9b700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e680b17e0 0x7f9e6815ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:30.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e66d9d700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e68161ce0 0x7f9e681602e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000cc30 tx=0x7f9e6000a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:30.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e56ffd700 1 -- 172.21.15.112:0/492836572 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6000ab70 con 0x7f9e68161ce0 2023-08-27T11:26:30.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e56ffd700 1 -- 172.21.15.112:0/492836572 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6000acd0 con 0x7f9e68161ce0 2023-08-27T11:26:30.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/492836572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e680c53d0 con 0x7f9e68161ce0 2023-08-27T11:26:30.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e56ffd700 1 -- 172.21.15.112:0/492836572 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60003c40 con 0x7f9e68161ce0 2023-08-27T11:26:30.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.995+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/492836572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e68160820 con 0x7f9e68161ce0 2023-08-27T11:26:30.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.996+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/492836572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e680c4c70 con 0x7f9e68161ce0 2023-08-27T11:26:30.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.997+0000 7f9e56ffd700 1 -- 172.21.15.112:0/492836572 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f9e6000f3f0 con 0x7f9e68161ce0 2023-08-27T11:26:30.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.997+0000 7f9e56ffd700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e4806a3b0 0x7f9e4806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:30.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.997+0000 7f9e56ffd700 1 -- 172.21.15.112:0/492836572 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e60088ca0 con 0x7f9e68161ce0 2023-08-27T11:26:30.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.997+0000 7f9e6659c700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e4806a3b0 0x7f9e4806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:30.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:29.998+0000 7f9e6659c700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e4806a3b0 0x7f9e4806c870 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9e6815da20 tx=0x7f9e5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:30.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.002+0000 7f9e56ffd700 1 -- 172.21.15.112:0/492836572 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9e680c4c70 con 0x7f9e68161ce0 2023-08-27T11:26:30.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.234+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/492836572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9e680c4c70 con 0x7f9e68161ce0 2023-08-27T11:26:30.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.234+0000 7f9e56ffd700 1 -- 172.21.15.112:0/492836572 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9e680c4c70 con 0x7f9e68161ce0 2023-08-27T11:26:30.236 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:26:30.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.236+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e4806a3b0 msgr2=0x7f9e4806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:30.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.236+0000 7f9e6cc8f700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e4806a3b0 0x7f9e4806c870 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9e6815da20 tx=0x7f9e5c008040 comp rx=0 tx=0).stop 2023-08-27T11:26:30.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.236+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/492836572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e68161ce0 msgr2=0x7f9e681602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:30.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.236+0000 7f9e6cc8f700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e68161ce0 0x7f9e681602e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000cc30 tx=0x7f9e6000a1f0 comp rx=0 tx=0).stop 2023-08-27T11:26:30.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.236+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/492836572 shutdown_connections 2023-08-27T11:26:30.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.236+0000 7f9e6cc8f700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e4806a3b0 0x7f9e4806c870 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:30.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.236+0000 7f9e6cc8f700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e68161ce0 0x7f9e681602e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:30.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.236+0000 7f9e6cc8f700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e680b1400 0x7f9e680c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:30.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.236+0000 7f9e6cc8f700 1 --2- 172.21.15.112:0/492836572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e680b17e0 0x7f9e6815ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:30.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.236+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/492836572 >> 172.21.15.112:0/492836572 conn(0x7f9e680d0000 msgr2=0x7f9e68070530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:30.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.236+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/492836572 shutdown_connections 2023-08-27T11:26:30.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:30.236+0000 7f9e6cc8f700 1 -- 172.21.15.112:0/492836572 wait complete. 2023-08-27T11:26:30.249 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:26:30.250 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:26:30.250 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-08-27T11:26:30.250 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:26:30.251 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:26:30.251 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:26:30.251 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:26:30.252 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:26:30.252 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:26:30.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:30.253 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-08-27T11:26:30.271 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:26:30.271 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:26:30.271 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:26:30.271 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:30.272 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:26:30.272 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:30.272 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:26:30.272 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-08-27T11:26:30.272 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:26:30.273 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-08-27T11:26:30.273 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:26:30.273 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:26:30.273 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:26:30.273 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:26:30.274 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:26:30.274 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 41853sec preferred_lft 41853sec 2023-08-27T11:26:30.274 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8d6c:4fe6:6a84:8222/64 scope link noprefixroute 2023-08-27T11:26:30.274 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:30.274 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:26:30.274 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff 2023-08-27T11:26:30.275 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:26:30.275 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:30.275 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::d87c:a1ff:fe1d:b635/64 scope link 2023-08-27T11:26:30.275 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:30.275 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:26:30.275 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:26:30.276 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::cc14:79ff:feb3:452f/64 scope link 2023-08-27T11:26:30.276 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:30.276 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:26:30.276 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether e6:51:f0:78:04:02 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:26:30.276 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::e451:f0ff:fe78:402/64 scope link 2023-08-27T11:26:30.277 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:30.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:30.277 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:26:30.336 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-08-27T11:26:30.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:30.338 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-08-27T11:26:30.395 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-08-27T11:26:30.395 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:26:30.395 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:26:30.396 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:26:30.396 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:26:30.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:30.397 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:26:30.397 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:26:30.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:26:30.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:26:30.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:26:30.399 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:26:30.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:30.499 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:26:30.516 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:26:30.516 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:26:30.516 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:26:30.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:30.517 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-08-27T11:26:30.582 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:26:30.583 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:26:30.640 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:26:30.640 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:26:30.640 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:26:30.697 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:26:30.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:30.698 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:26:30.782 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:26:30.782 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:26:30.782 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:26:30.790 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:26:30.791 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:26:30.791 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:26:30.791 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:26:30.791 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:26:30.865 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:26:30.865 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:26:30.865 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:26:30.866 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:26:30.866 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:26:30.866 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:26:30.866 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:26:30.867 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:26:30.867 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:26:30.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:30.868 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:26:30.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:30.896 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:26:30.977 INFO:teuthology.orchestra.run.smithi112.stdout:client6640 2023-08-27T11:26:30.977 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:26:30.978 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:31.035 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:26:31.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.194+0000 7f9bcf5f3700 1 Processor -- start 2023-08-27T11:26:31.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.195+0000 7f9bcf5f3700 1 -- start start 2023-08-27T11:26:31.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.195+0000 7f9bcf5f3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc80b4960 0x7f9bc815db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:31.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.195+0000 7f9bcf5f3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc80cfe70 con 0x7f9bc80b4dc0 2023-08-27T11:26:31.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.195+0000 7f9bcf5f3700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc8143e40 con 0x7f9bc8143a60 2023-08-27T11:26:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.195+0000 7f9bcf5f3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80b5340 con 0x7f9bc80b4960 2023-08-27T11:26:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bcdb90700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9bc80b4dc0 0x7f9bc815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55698/0 (socket says 172.21.15.112:55698) 2023-08-27T11:26:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bcdb90700 1 -- 172.21.15.112:0/2191596701 learned_addr learned my addr 172.21.15.112:0/2191596701 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bccb8e700 1 --2- 172.21.15.112:0/2191596701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc80b4960 0x7f9bc815db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434697697 0 0) 0x7f9bc80cfe70 con 0x7f9bc80b4dc0 2023-08-27T11:26:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb40035b0 con 0x7f9bc80b4dc0 2023-08-27T11:26:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797403614 0 0) 0x7f9bc8143e40 con 0x7f9bc8143a60 2023-08-27T11:26:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bc80cfe70 con 0x7f9bc8143a60 2023-08-27T11:26:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3150577596 0 0) 0x7f9bb40035b0 con 0x7f9bc80b4dc0 2023-08-27T11:26:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc80b4960 msgr2=0x7f9bc815db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 --2- 172.21.15.112:0/2191596701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc80b4960 0x7f9bc815db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 >> v1:172.21.15.138:6789/0 conn(0x7f9bc8143a60 legacy=0x7f9bc80cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc80ce9a0 con 0x7f9bc80b4dc0 2023-08-27T11:26:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f9bc00019f0 con 0x7f9bc80b4dc0 2023-08-27T11:26:31.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 >> v1:172.21.15.112:6789/0 conn(0x7f9bc80b4dc0 legacy=0x7f9bc815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:31.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 --2- 172.21.15.112:0/2191596701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bb4003bc0 0x7f9bb4003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:31.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 --2- 172.21.15.112:0/2191596701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc80b4960 0x7f9bb4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:31.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 --2- 172.21.15.112:0/2191596701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb4004d90 0x7f9bb4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:31.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80b5340 con 0x7f9bb4003bc0 2023-08-27T11:26:31.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb400bac0 con 0x7f9bb4004d90 2023-08-27T11:26:31.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb400bc30 con 0x7f9bc80b4960 2023-08-27T11:26:31.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bcd38f700 1 --2- 172.21.15.112:0/2191596701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bb4003bc0 0x7f9bb4003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:31.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bccb8e700 1 --2- 172.21.15.112:0/2191596701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc80b4960 0x7f9bb4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:31.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.196+0000 7f9bcdb90700 1 --2- 172.21.15.112:0/2191596701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb4004d90 0x7f9bb4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:31.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bcd38f700 1 -- 172.21.15.112:0/2191596701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc80b4960 msgr2=0x7f9bb4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:31.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bcd38f700 1 --2- 172.21.15.112:0/2191596701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc80b4960 0x7f9bb4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:31.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bcd38f700 1 -- 172.21.15.112:0/2191596701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb4004d90 msgr2=0x7f9bb4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:31.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bcd38f700 1 --2- 172.21.15.112:0/2191596701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb4004d90 0x7f9bb4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:31.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bcd38f700 1 -- 172.21.15.112:0/2191596701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb40035b0 con 0x7f9bb4003bc0 2023-08-27T11:26:31.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bcd38f700 1 --2- 172.21.15.112:0/2191596701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bb4003bc0 0x7f9bb4003fc0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f9bb80019a0 tx=0x7f9bb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415558898efd8cc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:31.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8001dd0 con 0x7f9bb4003bc0 2023-08-27T11:26:31.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb80052d0 con 0x7f9bb4003bc0 2023-08-27T11:26:31.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bbffff700 1 -- 172.21.15.112:0/2191596701 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8001460 con 0x7f9bb4003bc0 2023-08-27T11:26:31.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/2191596701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bb4003bc0 msgr2=0x7f9bb4003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:31.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bcf5f3700 1 --2- 172.21.15.112:0/2191596701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bb4003bc0 0x7f9bb4003fc0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f9bb80019a0 tx=0x7f9bb4005880 comp rx=0 tx=0).stop 2023-08-27T11:26:31.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bcdb90700 1 -- 172.21.15.112:0/2191596701 reap_dead start 2023-08-27T11:26:31.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/2191596701 shutdown_connections 2023-08-27T11:26:31.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.197+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/2191596701 >> 172.21.15.112:0/2191596701 conn(0x7f9bc806adb0 msgr2=0x7f9bc806b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:31.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.198+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/2191596701 shutdown_connections 2023-08-27T11:26:31.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.198+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/2191596701 wait complete. 2023-08-27T11:26:31.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.199+0000 7f9bcf5f3700 1 Processor -- start 2023-08-27T11:26:31.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.199+0000 7f9bcf5f3700 1 -- start start 2023-08-27T11:26:31.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.199+0000 7f9bcf5f3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb4003bc0 0x7f9bc8138d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:31.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.199+0000 7f9bcf5f3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc80c5e20 0x7f9bc80c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:31.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.199+0000 7f9bcf5f3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc813a8e0 0x7f9bc813d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:31.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.199+0000 7f9bcf5f3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc813bcc0 con 0x7f9bc813a8e0 2023-08-27T11:26:31.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.199+0000 7f9bcf5f3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc8139240 con 0x7f9bb4003bc0 2023-08-27T11:26:31.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.199+0000 7f9bcf5f3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80c77e0 con 0x7f9bc80c5e20 2023-08-27T11:26:31.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bcdb90700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc813a8e0 0x7f9bc813d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:31.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bcdb90700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc813a8e0 0x7f9bc813d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33096/0 (socket says 172.21.15.112:33096) 2023-08-27T11:26:31.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bcdb90700 1 -- 172.21.15.112:0/1867164574 learned_addr learned my addr 172.21.15.112:0/1867164574 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:31.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bcd38f700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb4003bc0 0x7f9bc8138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:31.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bccb8e700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc80c5e20 0x7f9bc80c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:31.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bcdb90700 1 -- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc80c5e20 msgr2=0x7f9bc80c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:31.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bcdb90700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc80c5e20 0x7f9bc80c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:31.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bcdb90700 1 -- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb4003bc0 msgr2=0x7f9bc8138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:31.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bcdb90700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb4003bc0 0x7f9bc8138d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:31.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bcdb90700 1 -- 172.21.15.112:0/1867164574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc813b320 con 0x7f9bc813a8e0 2023-08-27T11:26:31.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bcdb90700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc813a8e0 0x7f9bc813d870 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f9bc806e920 tx=0x7f9bc0007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:31.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bbdffb700 1 -- 172.21.15.112:0/1867164574 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000eb60 con 0x7f9bc813a8e0 2023-08-27T11:26:31.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bbdffb700 1 -- 172.21.15.112:0/1867164574 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc000ece0 con 0x7f9bc813a8e0 2023-08-27T11:26:31.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bbdffb700 1 -- 172.21.15.112:0/1867164574 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0013660 con 0x7f9bc813a8e0 2023-08-27T11:26:31.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/1867164574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc813b600 con 0x7f9bc813a8e0 2023-08-27T11:26:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.200+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/1867164574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc80c74d0 con 0x7f9bc813a8e0 2023-08-27T11:26:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.201+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/1867164574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc813be00 con 0x7f9bc813a8e0 2023-08-27T11:26:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.202+0000 7f9bbdffb700 1 -- 172.21.15.112:0/1867164574 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f9bc00137c0 con 0x7f9bc813a8e0 2023-08-27T11:26:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.202+0000 7f9bbdffb700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9bac06a3b0 0x7f9bac06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.202+0000 7f9bbdffb700 1 -- 172.21.15.112:0/1867164574 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9bc0088dd0 con 0x7f9bc813a8e0 2023-08-27T11:26:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.202+0000 7f9bcd38f700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9bac06a3b0 0x7f9bac06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.203+0000 7f9bcd38f700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9bac06a3b0 0x7f9bac06c870 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8000c00 tx=0x7f9bb800a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.206+0000 7f9bbdffb700 1 -- 172.21.15.112:0/1867164574 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9bc813be00 con 0x7f9bc813a8e0 2023-08-27T11:26:31.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.438+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/1867164574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9bc813be00 con 0x7f9bc813a8e0 2023-08-27T11:26:31.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.438+0000 7f9bbdffb700 1 -- 172.21.15.112:0/1867164574 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9bc813be00 con 0x7f9bc813a8e0 2023-08-27T11:26:31.439 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:26:31.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.441+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9bac06a3b0 msgr2=0x7f9bac06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:31.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.441+0000 7f9bcf5f3700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9bac06a3b0 0x7f9bac06c870 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8000c00 tx=0x7f9bb800a000 comp rx=0 tx=0).stop 2023-08-27T11:26:31.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.441+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/1867164574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc813a8e0 msgr2=0x7f9bc813d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:31.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.441+0000 7f9bcf5f3700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc813a8e0 0x7f9bc813d870 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f9bc806e920 tx=0x7f9bc0007c90 comp rx=0 tx=0).stop 2023-08-27T11:26:31.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.441+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/1867164574 shutdown_connections 2023-08-27T11:26:31.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.441+0000 7f9bcf5f3700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9bac06a3b0 0x7f9bac06c870 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:31.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.441+0000 7f9bcf5f3700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc813a8e0 0x7f9bc813d870 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:31.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.441+0000 7f9bcf5f3700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bb4003bc0 0x7f9bc8138d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:31.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.441+0000 7f9bcf5f3700 1 --2- 172.21.15.112:0/1867164574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc80c5e20 0x7f9bc80c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:31.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.441+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/1867164574 >> 172.21.15.112:0/1867164574 conn(0x7f9bc806adb0 msgr2=0x7f9bc80cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:31.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.441+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/1867164574 shutdown_connections 2023-08-27T11:26:31.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:31.441+0000 7f9bcf5f3700 1 -- 172.21.15.112:0/1867164574 wait complete. 2023-08-27T11:26:31.453 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:26:31.453 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:26:31.454 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2023-08-27T11:26:31.454 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-27T11:26:31.454 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:26:31.455 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:26:31.455 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:26:31.455 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:26:31.456 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:26:31.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:31.457 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2023-08-27T11:26:31.478 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:26:31.478 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:26:31.478 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:26:31.479 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:31.479 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:26:31.479 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:31.479 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:26:31.479 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2023-08-27T11:26:31.480 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:26:31.480 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2023-08-27T11:26:31.480 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:26:31.480 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:26:31.480 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:26:31.480 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:26:31.481 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:26:31.481 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 41867sec preferred_lft 41867sec 2023-08-27T11:26:31.481 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ac17:49c5:cb39:801a/64 scope link noprefixroute 2023-08-27T11:26:31.481 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:31.481 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:26:31.482 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:26:31.482 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:26:31.482 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:31.482 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c4b4:94ff:fe58:4e35/64 scope link 2023-08-27T11:26:31.482 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:31.483 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:26:31.483 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:26:31.483 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::5806:6aff:fe04:ef32/64 scope link 2023-08-27T11:26:31.483 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:31.483 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:26:31.484 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 8a:21:ba:15:69:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-27T11:26:31.484 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::8821:baff:fe15:6919/64 scope link 2023-08-27T11:26:31.484 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:26:31.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:31.485 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:26:31.548 INFO:teuthology.orchestra.run.smithi138.stdout:1 2023-08-27T11:26:31.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:31.550 DEBUG:teuthology.orchestra.run.smithi138:> route 2023-08-27T11:26:31.610 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2023-08-27T11:26:31.610 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:26:31.611 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:26:31.611 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:26:31.611 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:26:31.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:31.612 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:26:31.612 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:26:31.613 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:26:31.613 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:26:31.613 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:26:31.614 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:26:31.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:31.720 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:26:31.737 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-27T11:26:31.737 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-27T11:26:31.738 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:26:31.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:31.739 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2023-08-27T11:26:31.804 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:26:31.805 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-27T11:26:31.861 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:26:31.861 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:26:31.862 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:26:31.917 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:26:31.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:31.918 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:26:32.003 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:26:32.004 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:26:32.004 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:26:32.012 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:26:32.012 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:26:32.012 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:26:32.013 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:26:32.013 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:26:32.087 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:26:32.087 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:26:32.088 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:26:32.088 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:26:32.088 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:26:32.088 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:26:32.088 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:26:32.089 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:26:32.090 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:26:32.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:32.090 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:26:32.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:32.117 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:26:32.189 INFO:teuthology.orchestra.run.smithi138.stdout:client7245 2023-08-27T11:26:32.190 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:26:32.190 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:32.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-08-27T11:26:32.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.384+0000 7ff1a394d700 1 Processor -- start 2023-08-27T11:26:32.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.384+0000 7ff1a394d700 1 -- start start 2023-08-27T11:26:32.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a394d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff19c157ae0 0x7ff19c15e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:32.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a394d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff19c0b3de0 0x7ff19c161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:32.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a394d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c161390 con 0x7ff19c161d60 2023-08-27T11:26:32.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a394d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c0b1aa0 con 0x7ff19c0b3de0 2023-08-27T11:26:32.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a394d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c0b4bd0 con 0x7ff19c157ae0 2023-08-27T11:26:32.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a0ee8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff19c0b3de0 0x7ff19c161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:32.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a1eea700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff19c161d60 0x7ff19c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55718/0 (socket says 172.21.15.112:55718) 2023-08-27T11:26:32.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a0ee8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff19c0b3de0 0x7ff19c161820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33102/0 (socket says 172.21.15.112:33102) 2023-08-27T11:26:32.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a1eea700 1 -- 172.21.15.112:0/671140749 learned_addr learned my addr 172.21.15.112:0/671140749 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:32.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a16e9700 1 --2- 172.21.15.112:0/671140749 >> v2:172.21.15.138:3300/0 conn(0x7ff19c157ae0 0x7ff19c15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:32.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a0ee8700 1 -- 172.21.15.112:0/671140749 >> v2:172.21.15.138:3300/0 conn(0x7ff19c157ae0 msgr2=0x7ff19c15e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:32.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a0ee8700 1 --2- 172.21.15.112:0/671140749 >> v2:172.21.15.138:3300/0 conn(0x7ff19c157ae0 0x7ff19c15e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:32.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a0ee8700 1 -- 172.21.15.112:0/671140749 >> v1:172.21.15.112:6789/0 conn(0x7ff19c161d60 legacy=0x7ff19c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:32.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a0ee8700 1 -- 172.21.15.112:0/671140749 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0b35f0 con 0x7ff19c0b3de0 2023-08-27T11:26:32.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff193fff700 1 -- 172.21.15.112:0/671140749 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959000959 0 0) 0x7ff19c161390 con 0x7ff19c161d60 2023-08-27T11:26:32.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a16e9700 1 --2- 172.21.15.112:0/671140749 >> v2:172.21.15.138:3300/0 conn(0x7ff19c157ae0 0x7ff19c15e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:32.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.385+0000 7ff1a0ee8700 1 --2- 172.21.15.112:0/671140749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c0b3de0 0x7ff19c161820 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff19c06e580 tx=0x7ff198009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be291ad0809d4204 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:32.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.386+0000 7ff193fff700 1 -- 172.21.15.112:0/671140749 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198011830 con 0x7ff19c0b3de0 2023-08-27T11:26:32.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.386+0000 7ff193fff700 1 -- 172.21.15.112:0/671140749 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff198011990 con 0x7ff19c0b3de0 2023-08-27T11:26:32.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.386+0000 7ff193fff700 1 -- 172.21.15.112:0/671140749 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198011d20 con 0x7ff19c0b3de0 2023-08-27T11:26:32.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.386+0000 7ff1a394d700 1 -- 172.21.15.112:0/671140749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c0b3de0 msgr2=0x7ff19c161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:32.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.386+0000 7ff1a394d700 1 --2- 172.21.15.112:0/671140749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c0b3de0 0x7ff19c161820 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff19c06e580 tx=0x7ff198009b20 comp rx=0 tx=0).stop 2023-08-27T11:26:32.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.386+0000 7ff1a394d700 1 -- 172.21.15.112:0/671140749 shutdown_connections 2023-08-27T11:26:32.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.386+0000 7ff1a394d700 1 --2- 172.21.15.112:0/671140749 >> v2:172.21.15.138:3300/0 conn(0x7ff19c157ae0 0x7ff19c15e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:32.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.386+0000 7ff1a394d700 1 --2- 172.21.15.112:0/671140749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c0b3de0 0x7ff19c161820 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:32.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.386+0000 7ff1a394d700 1 -- 172.21.15.112:0/671140749 >> 172.21.15.112:0/671140749 conn(0x7ff19c0b1be0 msgr2=0x7ff19c0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:32.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.386+0000 7ff1a394d700 1 -- 172.21.15.112:0/671140749 shutdown_connections 2023-08-27T11:26:32.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.386+0000 7ff1a394d700 1 -- 172.21.15.112:0/671140749 wait complete. 2023-08-27T11:26:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.387+0000 7ff1a394d700 1 Processor -- start 2023-08-27T11:26:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.388+0000 7ff1a394d700 1 -- start start 2023-08-27T11:26:32.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.388+0000 7ff1a394d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff19c0b3de0 0x7ff19c13b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:32.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.388+0000 7ff1a394d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff19c157ae0 0x7ff19c15e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:32.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.388+0000 7ff1a394d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c161d60 0x7ff19c144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:32.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.388+0000 7ff1a394d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c15fe60 con 0x7ff19c161d60 2023-08-27T11:26:32.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.388+0000 7ff1a394d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c15f250 con 0x7ff19c157ae0 2023-08-27T11:26:32.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.388+0000 7ff1a394d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c15f530 con 0x7ff19c0b3de0 2023-08-27T11:26:32.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.388+0000 7ff1a1eea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c161d60 0x7ff19c144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:32.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.388+0000 7ff1a1eea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c161d60 0x7ff19c144420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33110/0 (socket says 172.21.15.112:33110) 2023-08-27T11:26:32.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.388+0000 7ff1a1eea700 1 -- 172.21.15.112:0/661181039 learned_addr learned my addr 172.21.15.112:0/661181039 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:32.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff1a0ee8700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff19c157ae0 0x7ff19c15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:32.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff1a16e9700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff19c0b3de0 0x7ff19c13b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:32.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff1a1eea700 1 -- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff19c0b3de0 msgr2=0x7ff19c13b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:32.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff1a1eea700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff19c0b3de0 0x7ff19c13b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:32.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff1a1eea700 1 -- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff19c157ae0 msgr2=0x7ff19c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:32.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff1a1eea700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff19c157ae0 0x7ff19c15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:32.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff1a1eea700 1 -- 172.21.15.112:0/661181039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1980094a0 con 0x7ff19c161d60 2023-08-27T11:26:32.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff1a0ee8700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff19c157ae0 0x7ff19c15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:32.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff1a1eea700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c161d60 0x7ff19c144420 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff19400cc10 tx=0x7ff1940021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:32.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff191ffb700 1 -- 172.21.15.112:0/661181039 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19400ab80 con 0x7ff19c161d60 2023-08-27T11:26:32.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff191ffb700 1 -- 172.21.15.112:0/661181039 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19400ad00 con 0x7ff19c161d60 2023-08-27T11:26:32.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff1a394d700 1 -- 172.21.15.112:0/661181039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c15f810 con 0x7ff19c161d60 2023-08-27T11:26:32.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff191ffb700 1 -- 172.21.15.112:0/661181039 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194007310 con 0x7ff19c161d60 2023-08-27T11:26:32.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.389+0000 7ff1a394d700 1 -- 172.21.15.112:0/661181039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c0aeb90 con 0x7ff19c161d60 2023-08-27T11:26:32.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.390+0000 7ff1a394d700 1 -- 172.21.15.112:0/661181039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c15ffd0 con 0x7ff19c161d60 2023-08-27T11:26:32.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.390+0000 7ff191ffb700 1 -- 172.21.15.112:0/661181039 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7ff194007470 con 0x7ff19c161d60 2023-08-27T11:26:32.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.391+0000 7ff191ffb700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff18006a3b0 0x7ff18006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:32.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.391+0000 7ff191ffb700 1 -- 172.21.15.112:0/661181039 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff1940887f0 con 0x7ff19c161d60 2023-08-27T11:26:32.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.391+0000 7ff1a16e9700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff18006a3b0 0x7ff18006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:32.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.391+0000 7ff1a16e9700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff18006a3b0 0x7ff18006c870 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009930 tx=0x7ff18c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:32.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.396+0000 7ff191ffb700 1 -- 172.21.15.112:0/661181039 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff194053b90 con 0x7ff19c161d60 2023-08-27T11:26:32.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:32.631+0000 7ff1a394d700 1 -- 172.21.15.112:0/661181039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff19c067a20 con 0x7ff19c161d60 2023-08-27T11:26:33.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.475+0000 7ff191ffb700 1 -- 172.21.15.112:0/661181039 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 136+0+0 (secure 0 0 0) 0x7ff1940571b0 con 0x7ff19c161d60 2023-08-27T11:26:33.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.478+0000 7ff1a394d700 1 -- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff18006a3b0 msgr2=0x7ff18006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:33.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.478+0000 7ff1a394d700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff18006a3b0 0x7ff18006c870 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009930 tx=0x7ff18c008040 comp rx=0 tx=0).stop 2023-08-27T11:26:33.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.478+0000 7ff1a394d700 1 -- 172.21.15.112:0/661181039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c161d60 msgr2=0x7ff19c144420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:33.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.478+0000 7ff1a394d700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c161d60 0x7ff19c144420 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff19400cc10 tx=0x7ff1940021c0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.478+0000 7ff1a394d700 1 -- 172.21.15.112:0/661181039 shutdown_connections 2023-08-27T11:26:33.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.478+0000 7ff1a394d700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff18006a3b0 0x7ff18006c870 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.478+0000 7ff1a394d700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff19c161d60 0x7ff19c144420 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.478+0000 7ff1a394d700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff19c0b3de0 0x7ff19c13b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.478+0000 7ff1a394d700 1 --2- 172.21.15.112:0/661181039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff19c157ae0 0x7ff19c15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.478+0000 7ff1a394d700 1 -- 172.21.15.112:0/661181039 >> 172.21.15.112:0/661181039 conn(0x7ff19c0b1be0 msgr2=0x7ff19c1458f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:33.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.478+0000 7ff1a394d700 1 -- 172.21.15.112:0/661181039 shutdown_connections 2023-08-27T11:26:33.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.478+0000 7ff1a394d700 1 -- 172.21.15.112:0/661181039 wait complete. 2023-08-27T11:26:33.493 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:26:33.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.619+0000 7f3cbd90f700 1 Processor -- start 2023-08-27T11:26:33.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.619+0000 7f3cbd90f700 1 -- start start 2023-08-27T11:26:33.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.619+0000 7f3cbd90f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3cb8157880 0x7f3cb8160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:33.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.619+0000 7f3cbd90f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3cb8161430 0x7f3cb80d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:33.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.619+0000 7f3cbd90f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb815f110 con 0x7f3cb80d0c60 2023-08-27T11:26:33.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.619+0000 7f3cbd90f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb81618c0 con 0x7f3cb8157880 2023-08-27T11:26:33.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.619+0000 7f3cbd90f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb80d1490 con 0x7f3cb8161430 2023-08-27T11:26:33.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.620+0000 7f3cb6ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3cb8157880 0x7f3cb8160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:33.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.620+0000 7f3cb6ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3cb8157880 0x7f3cb8160ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33114/0 (socket says 172.21.15.112:33114) 2023-08-27T11:26:33.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.620+0000 7f3cb6ffd700 1 -- 172.21.15.112:0/566285181 learned_addr learned my addr 172.21.15.112:0/566285181 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:33.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.620+0000 7f3cb67fc700 1 --2- 172.21.15.112:0/566285181 >> v2:172.21.15.138:3300/0 conn(0x7f3cb8161430 0x7f3cb80d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:33.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.620+0000 7f3cb6ffd700 1 -- 172.21.15.112:0/566285181 >> v2:172.21.15.138:3300/0 conn(0x7f3cb8161430 msgr2=0x7f3cb80d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:33.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.620+0000 7f3cb6ffd700 1 --2- 172.21.15.112:0/566285181 >> v2:172.21.15.138:3300/0 conn(0x7f3cb8161430 0x7f3cb80d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.620+0000 7f3cb6ffd700 1 -- 172.21.15.112:0/566285181 >> v1:172.21.15.138:6789/0 conn(0x7f3cb80d0c60 legacy=0x7f3cb815d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:33.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.620+0000 7f3cb6ffd700 1 -- 172.21.15.112:0/566285181 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb80d1770 con 0x7f3cb8157880 2023-08-27T11:26:33.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.620+0000 7f3cb6ffd700 1 --2- 172.21.15.112:0/566285181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8157880 0x7f3cb8160ef0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80720a0 tx=0x7f3cac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c748983a40e50bab server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:33.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.620+0000 7f3cb5ffb700 1 -- 172.21.15.112:0/566285181 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac004230 con 0x7f3cb8157880 2023-08-27T11:26:33.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.620+0000 7f3cb5ffb700 1 -- 172.21.15.112:0/566285181 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac004390 con 0x7f3cb8157880 2023-08-27T11:26:33.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.620+0000 7f3cb5ffb700 1 -- 172.21.15.112:0/566285181 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac0036a0 con 0x7f3cb8157880 2023-08-27T11:26:33.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.621+0000 7f3cbd90f700 1 -- 172.21.15.112:0/566285181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8157880 msgr2=0x7f3cb8160ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:33.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.621+0000 7f3cbd90f700 1 --2- 172.21.15.112:0/566285181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8157880 0x7f3cb8160ef0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80720a0 tx=0x7f3cac009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.621+0000 7f3cbd90f700 1 -- 172.21.15.112:0/566285181 shutdown_connections 2023-08-27T11:26:33.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.621+0000 7f3cbd90f700 1 --2- 172.21.15.112:0/566285181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8157880 0x7f3cb8160ef0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.621+0000 7f3cbd90f700 1 --2- 172.21.15.112:0/566285181 >> v2:172.21.15.138:3300/0 conn(0x7f3cb8161430 0x7f3cb80d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.621+0000 7f3cbd90f700 1 -- 172.21.15.112:0/566285181 >> 172.21.15.112:0/566285181 conn(0x7f3cb806c880 msgr2=0x7f3cb80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:33.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.621+0000 7f3cbd90f700 1 -- 172.21.15.112:0/566285181 shutdown_connections 2023-08-27T11:26:33.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.621+0000 7f3cbd90f700 1 -- 172.21.15.112:0/566285181 wait complete. 2023-08-27T11:26:33.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.622+0000 7f3cbd90f700 1 Processor -- start 2023-08-27T11:26:33.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.622+0000 7f3cbd90f700 1 -- start start 2023-08-27T11:26:33.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cbd90f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3cb80d0c60 0x7f3cb815c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:33.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cbd90f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cb8157880 0x7f3cb80b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:33.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cbd90f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8161430 0x7f3cb80b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:33.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cbd90f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb815be70 con 0x7f3cb8161430 2023-08-27T11:26:33.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cbd90f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb815b260 con 0x7f3cb8157880 2023-08-27T11:26:33.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cbd90f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb815b540 con 0x7f3cb80d0c60 2023-08-27T11:26:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cb77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8161430 0x7f3cb80b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cb77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8161430 0x7f3cb80b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33120/0 (socket says 172.21.15.112:33120) 2023-08-27T11:26:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cb77fe700 1 -- 172.21.15.112:0/2957002825 learned_addr learned my addr 172.21.15.112:0/2957002825 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cb67fc700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cb8157880 0x7f3cb80b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cb6ffd700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3cb80d0c60 0x7f3cb815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cb77fe700 1 -- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3cb80d0c60 msgr2=0x7f3cb815c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cb77fe700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3cb80d0c60 0x7f3cb815c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cb77fe700 1 -- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cb8157880 msgr2=0x7f3cb80b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cb77fe700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cb8157880 0x7f3cb80b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cb77fe700 1 -- 172.21.15.112:0/2957002825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac009550 con 0x7f3cb8161430 2023-08-27T11:26:33.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3cb77fe700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8161430 0x7f3cb80b5480 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000cc10 tx=0x7f3cb000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:33.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.623+0000 7f3c9ffff700 1 -- 172.21.15.112:0/2957002825 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb000abb0 con 0x7f3cb8161430 2023-08-27T11:26:33.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.624+0000 7f3c9ffff700 1 -- 172.21.15.112:0/2957002825 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb000ad10 con 0x7f3cb8161430 2023-08-27T11:26:33.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.624+0000 7f3cbd90f700 1 -- 172.21.15.112:0/2957002825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb815b820 con 0x7f3cb8161430 2023-08-27T11:26:33.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.624+0000 7f3c9ffff700 1 -- 172.21.15.112:0/2957002825 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0012660 con 0x7f3cb8161430 2023-08-27T11:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.624+0000 7f3cbd90f700 1 -- 172.21.15.112:0/2957002825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb814d5d0 con 0x7f3cb8161430 2023-08-27T11:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.625+0000 7f3cbd90f700 1 -- 172.21.15.112:0/2957002825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb815bfe0 con 0x7f3cb8161430 2023-08-27T11:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.625+0000 7f3c9ffff700 1 -- 172.21.15.112:0/2957002825 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f3cb00127c0 con 0x7f3cb8161430 2023-08-27T11:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.625+0000 7f3c9ffff700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3ca406a3b0 0x7f3ca406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.625+0000 7f3c9ffff700 1 -- 172.21.15.112:0/2957002825 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3cb0089240 con 0x7f3cb8161430 2023-08-27T11:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.626+0000 7f3cb6ffd700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3ca406a3b0 0x7f3ca406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.626+0000 7f3cb6ffd700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3ca406a3b0 0x7f3ca406c870 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009130 tx=0x7f3cac008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.630+0000 7f3c9ffff700 1 -- 172.21.15.112:0/2957002825 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3cb00545e0 con 0x7f3cb8161430 2023-08-27T11:26:33.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.842+0000 7f3cbd90f700 1 -- 172.21.15.112:0/2957002825 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cb815eda0 con 0x7f3ca406a3b0 2023-08-27T11:26:33.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.844+0000 7f3c9ffff700 1 -- 172.21.15.112:0/2957002825 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3cac0041b0 con 0x7f3ca406a3b0 2023-08-27T11:26:33.845 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:26:33.846 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:33.846 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs" 2023-08-27T11:26:33.846 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:33.846 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:26:33.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.846+0000 7f3cbd90f700 1 -- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3ca406a3b0 msgr2=0x7f3ca406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:33.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.846+0000 7f3cbd90f700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3ca406a3b0 0x7f3ca406c870 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009130 tx=0x7f3cac008030 comp rx=0 tx=0).stop 2023-08-27T11:26:33.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.846+0000 7f3cbd90f700 1 -- 172.21.15.112:0/2957002825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8161430 msgr2=0x7f3cb80b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:33.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.846+0000 7f3cbd90f700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8161430 0x7f3cb80b5480 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000cc10 tx=0x7f3cb000a060 comp rx=0 tx=0).stop 2023-08-27T11:26:33.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.846+0000 7f3cbd90f700 1 -- 172.21.15.112:0/2957002825 shutdown_connections 2023-08-27T11:26:33.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.846+0000 7f3cbd90f700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3ca406a3b0 0x7f3ca406c870 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.846+0000 7f3cbd90f700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8161430 0x7f3cb80b5480 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.846+0000 7f3cbd90f700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3cb80d0c60 0x7f3cb815c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.846+0000 7f3cbd90f700 1 --2- 172.21.15.112:0/2957002825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cb8157880 0x7f3cb80b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.846+0000 7f3cbd90f700 1 -- 172.21.15.112:0/2957002825 >> 172.21.15.112:0/2957002825 conn(0x7f3cb806c880 msgr2=0x7f3cb815fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:33.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.846+0000 7f3cbd90f700 1 -- 172.21.15.112:0/2957002825 shutdown_connections 2023-08-27T11:26:33.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.846+0000 7f3cbd90f700 1 -- 172.21.15.112:0/2957002825 wait complete. 2023-08-27T11:26:33.859 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:26:33.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.981+0000 7f1703245700 1 Processor -- start 2023-08-27T11:26:33.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.981+0000 7f1703245700 1 -- start start 2023-08-27T11:26:33.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f1703245700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f16fc146c20 0x7f16fc160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:33.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f1703245700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16fc0b3440 0x7f16fc15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:33.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f1703245700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16fc0b5070 con 0x7f16fc15e6e0 2023-08-27T11:26:33.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f1703245700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16fc147000 con 0x7f16fc146c20 2023-08-27T11:26:33.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f1703245700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc1614a0 con 0x7f16fc0b3440 2023-08-27T11:26:33.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f1700fe1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f16fc146c20 0x7f16fc160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:33.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f1700fe1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f16fc146c20 0x7f16fc160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33126/0 (socket says 172.21.15.112:33126) 2023-08-27T11:26:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f1700fe1700 1 -- 172.21.15.112:0/1993131104 learned_addr learned my addr 172.21.15.112:0/1993131104 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f16fbfff700 1 --2- 172.21.15.112:0/1993131104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16fc0b3440 0x7f16fc15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f1700fe1700 1 -- 172.21.15.112:0/1993131104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16fc0b3440 msgr2=0x7f16fc15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f1700fe1700 1 --2- 172.21.15.112:0/1993131104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16fc0b3440 0x7f16fc15e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f1700fe1700 1 -- 172.21.15.112:0/1993131104 >> v1:172.21.15.138:6789/0 conn(0x7f16fc15e6e0 legacy=0x7f16fc06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f1700fe1700 1 -- 172.21.15.112:0/1993131104 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc161740 con 0x7f16fc146c20 2023-08-27T11:26:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f1700fe1700 1 --2- 172.21.15.112:0/1993131104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc146c20 0x7f16fc160f60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f16f00098f0 tx=0x7f16f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863805ccf72be97a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:33.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f16fb7fe700 1 -- 172.21.15.112:0/1993131104 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0011830 con 0x7f16fc146c20 2023-08-27T11:26:33.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f16fb7fe700 1 -- 172.21.15.112:0/1993131104 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f0011990 con 0x7f16fc146c20 2023-08-27T11:26:33.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.982+0000 7f16fb7fe700 1 -- 172.21.15.112:0/1993131104 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0011d20 con 0x7f16fc146c20 2023-08-27T11:26:33.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.983+0000 7f1703245700 1 -- 172.21.15.112:0/1993131104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc146c20 msgr2=0x7f16fc160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:33.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.983+0000 7f1703245700 1 --2- 172.21.15.112:0/1993131104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc146c20 0x7f16fc160f60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f16f00098f0 tx=0x7f16f0009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:33.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.983+0000 7f1703245700 1 -- 172.21.15.112:0/1993131104 shutdown_connections 2023-08-27T11:26:33.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.983+0000 7f1703245700 1 --2- 172.21.15.112:0/1993131104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc146c20 0x7f16fc160f60 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.983+0000 7f1703245700 1 --2- 172.21.15.112:0/1993131104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16fc0b3440 0x7f16fc15e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.983+0000 7f1703245700 1 -- 172.21.15.112:0/1993131104 >> 172.21.15.112:0/1993131104 conn(0x7f16fc0b1e90 msgr2=0x7f16fc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:33.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.983+0000 7f1703245700 1 -- 172.21.15.112:0/1993131104 shutdown_connections 2023-08-27T11:26:33.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.983+0000 7f1703245700 1 -- 172.21.15.112:0/1993131104 wait complete. 2023-08-27T11:26:33.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.984+0000 7f1703245700 1 Processor -- start 2023-08-27T11:26:33.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.984+0000 7f1703245700 1 -- start start 2023-08-27T11:26:33.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f1703245700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0b3440 0x7f16fc1380e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:33.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f1703245700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16fc146c20 0x7f16fc0ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:33.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f1703245700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16fc15e6e0 0x7f16fc1f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:33.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f1703245700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc0ae8b0 con 0x7f16fc0b3440 2023-08-27T11:26:33.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f1703245700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc0ada00 con 0x7f16fc146c20 2023-08-27T11:26:33.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f1703245700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc0adc30 con 0x7f16fc15e6e0 2023-08-27T11:26:33.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f1700fe1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0b3440 0x7f16fc1380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:33.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f1700fe1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0b3440 0x7f16fc1380e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33132/0 (socket says 172.21.15.112:33132) 2023-08-27T11:26:33.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f1700fe1700 1 -- 172.21.15.112:0/1461969173 learned_addr learned my addr 172.21.15.112:0/1461969173 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f16fbfff700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16fc146c20 0x7f16fc0ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f17017e2700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16fc15e6e0 0x7f16fc1f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f1700fe1700 1 -- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16fc15e6e0 msgr2=0x7f16fc1f2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f1700fe1700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16fc15e6e0 0x7f16fc1f2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f1700fe1700 1 -- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16fc146c20 msgr2=0x7f16fc0ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.985+0000 7f1700fe1700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16fc146c20 0x7f16fc0ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.986+0000 7f1700fe1700 1 -- 172.21.15.112:0/1461969173 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f0009550 con 0x7f16fc0b3440 2023-08-27T11:26:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.986+0000 7f16fbfff700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16fc146c20 0x7f16fc0ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.986+0000 7f17017e2700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16fc15e6e0 0x7f16fc1f2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.986+0000 7f1700fe1700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0b3440 0x7f16fc1380e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f16f0000c00 tx=0x7f16f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:33.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.986+0000 7f16f97fa700 1 -- 172.21.15.112:0/1461969173 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0004570 con 0x7f16fc0b3440 2023-08-27T11:26:33.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.986+0000 7f16f97fa700 1 -- 172.21.15.112:0/1461969173 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f001f070 con 0x7f16fc0b3440 2023-08-27T11:26:33.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.986+0000 7f1703245700 1 -- 172.21.15.112:0/1461969173 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16fc0adf10 con 0x7f16fc0b3440 2023-08-27T11:26:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.986+0000 7f16f97fa700 1 -- 172.21.15.112:0/1461969173 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f00038c0 con 0x7f16fc0b3440 2023-08-27T11:26:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.986+0000 7f1703245700 1 -- 172.21.15.112:0/1461969173 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16fc0ae4d0 con 0x7f16fc0b3440 2023-08-27T11:26:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.987+0000 7f1703245700 1 -- 172.21.15.112:0/1461969173 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16fc067e90 con 0x7f16fc0b3440 2023-08-27T11:26:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.987+0000 7f16f97fa700 1 -- 172.21.15.112:0/1461969173 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f16f0021030 con 0x7f16fc0b3440 2023-08-27T11:26:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.988+0000 7f16f97fa700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f16e006a3b0 0x7f16e006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.988+0000 7f16f97fa700 1 -- 172.21.15.112:0/1461969173 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f16f0089e20 con 0x7f16fc0b3440 2023-08-27T11:26:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.988+0000 7f16fbfff700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f16e006a3b0 0x7f16e006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.988+0000 7f16fbfff700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f16e006a3b0 0x7f16e006c870 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f16fc142010 tx=0x7f16e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:33.992+0000 7f16f97fa700 1 -- 172.21.15.112:0/1461969173 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f16f0055210 con 0x7f16fc0b3440 2023-08-27T11:26:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.180+0000 7f1703245700 1 -- 172.21.15.112:0/1461969173 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f16fc067a20 con 0x7f16fc0b3440 2023-08-27T11:26:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.181+0000 7f16f97fa700 1 -- 172.21.15.112:0/1461969173 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f16f00150c0 con 0x7f16fc0b3440 2023-08-27T11:26:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.183+0000 7f1703245700 1 -- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f16e006a3b0 msgr2=0x7f16e006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.183+0000 7f1703245700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f16e006a3b0 0x7f16e006c870 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f16fc142010 tx=0x7f16e8008040 comp rx=0 tx=0).stop 2023-08-27T11:26:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.183+0000 7f1703245700 1 -- 172.21.15.112:0/1461969173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0b3440 msgr2=0x7f16fc1380e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.183+0000 7f1703245700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0b3440 0x7f16fc1380e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f16f0000c00 tx=0x7f16f0004750 comp rx=0 tx=0).stop 2023-08-27T11:26:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.183+0000 7f1703245700 1 -- 172.21.15.112:0/1461969173 shutdown_connections 2023-08-27T11:26:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.183+0000 7f1703245700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f16e006a3b0 0x7f16e006c870 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.183+0000 7f1703245700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16fc15e6e0 0x7f16fc1f2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.183+0000 7f1703245700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0b3440 0x7f16fc1380e0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.183+0000 7f1703245700 1 --2- 172.21.15.112:0/1461969173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16fc146c20 0x7f16fc0ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.183+0000 7f1703245700 1 -- 172.21.15.112:0/1461969173 >> 172.21.15.112:0/1461969173 conn(0x7f16fc0b1e90 msgr2=0x7f16fc06ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.183+0000 7f1703245700 1 -- 172.21.15.112:0/1461969173 shutdown_connections 2023-08-27T11:26:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.183+0000 7f1703245700 1 -- 172.21.15.112:0/1461969173 wait complete. 2023-08-27T11:26:34.196 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs --human_readable 2023-08-27T11:26:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.319+0000 7fd8cc699700 1 Processor -- start 2023-08-27T11:26:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.319+0000 7fd8cc699700 1 -- start start 2023-08-27T11:26:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8cc699700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8c41445b0 0x7fd8c40ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8cc699700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd8c40ccad0 0x7fd8c415b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8cc699700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c40b08e0 con 0x7fd8c406f790 2023-08-27T11:26:34.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8cc699700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c415bc70 con 0x7fd8c40ccad0 2023-08-27T11:26:34.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8cc699700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40ccf50 con 0x7fd8c41445b0 2023-08-27T11:26:34.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8c9c34700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd8c40ccad0 0x7fd8c415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:34.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8c9c34700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd8c40ccad0 0x7fd8c415b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33140/0 (socket says 172.21.15.112:33140) 2023-08-27T11:26:34.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8c9c34700 1 -- 172.21.15.112:0/3974322926 learned_addr learned my addr 172.21.15.112:0/3974322926 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:34.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8ca435700 1 --2- 172.21.15.112:0/3974322926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8c41445b0 0x7fd8c40ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:34.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8c9c34700 1 -- 172.21.15.112:0/3974322926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8c41445b0 msgr2=0x7fd8c40ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8c9c34700 1 --2- 172.21.15.112:0/3974322926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8c41445b0 0x7fd8c40ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8c9c34700 1 -- 172.21.15.112:0/3974322926 >> v1:172.21.15.138:6789/0 conn(0x7fd8c406f790 legacy=0x7fd8c415ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8c9c34700 1 -- 172.21.15.112:0/3974322926 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c40b0b30 con 0x7fd8c40ccad0 2023-08-27T11:26:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8c9c34700 1 --2- 172.21.15.112:0/3974322926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8c40ccad0 0x7fd8c415b730 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40098f0 tx=0x7fd8b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ee2cdbc1ce473d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8c9433700 1 -- 172.21.15.112:0/3974322926 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400b8a0 con 0x7fd8c40ccad0 2023-08-27T11:26:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8c9433700 1 -- 172.21.15.112:0/3974322926 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b400ba00 con 0x7fd8c40ccad0 2023-08-27T11:26:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.320+0000 7fd8c9433700 1 -- 172.21.15.112:0/3974322926 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400bd90 con 0x7fd8c40ccad0 2023-08-27T11:26:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.321+0000 7fd8cc699700 1 -- 172.21.15.112:0/3974322926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8c40ccad0 msgr2=0x7fd8c415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.321+0000 7fd8cc699700 1 --2- 172.21.15.112:0/3974322926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8c40ccad0 0x7fd8c415b730 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40098f0 tx=0x7fd8b4009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:34.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.321+0000 7fd8cc699700 1 -- 172.21.15.112:0/3974322926 shutdown_connections 2023-08-27T11:26:34.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.321+0000 7fd8cc699700 1 --2- 172.21.15.112:0/3974322926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8c41445b0 0x7fd8c40ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.321+0000 7fd8cc699700 1 --2- 172.21.15.112:0/3974322926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8c40ccad0 0x7fd8c415b730 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.321+0000 7fd8cc699700 1 -- 172.21.15.112:0/3974322926 >> 172.21.15.112:0/3974322926 conn(0x7fd8c40b1e90 msgr2=0x7fd8c40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:34.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.321+0000 7fd8cc699700 1 -- 172.21.15.112:0/3974322926 shutdown_connections 2023-08-27T11:26:34.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.321+0000 7fd8cc699700 1 -- 172.21.15.112:0/3974322926 wait complete. 2023-08-27T11:26:34.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.322+0000 7fd8cc699700 1 Processor -- start 2023-08-27T11:26:34.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.322+0000 7fd8cc699700 1 -- start start 2023-08-27T11:26:34.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cc699700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8c406f790 0x7fd8c40cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:34.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cc699700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c40ccad0 0x7fd8c4142c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:34.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cc699700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8c41445b0 0x7fd8c40708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:34.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cc699700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40cb9c0 con 0x7fd8c41445b0 2023-08-27T11:26:34.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cc699700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40cab10 con 0x7fd8c40ccad0 2023-08-27T11:26:34.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cc699700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40cadb0 con 0x7fd8c406f790 2023-08-27T11:26:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cac36700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8c41445b0 0x7fd8c40708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cac36700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8c41445b0 0x7fd8c40708a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33150/0 (socket says 172.21.15.112:33150) 2023-08-27T11:26:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cac36700 1 -- 172.21.15.112:0/2115090823 learned_addr learned my addr 172.21.15.112:0/2115090823 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8c9c34700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c40ccad0 0x7fd8c4142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8ca435700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8c406f790 0x7fd8c40cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cac36700 1 -- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8c406f790 msgr2=0x7fd8c40cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cac36700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8c406f790 0x7fd8c40cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cac36700 1 -- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c40ccad0 msgr2=0x7fd8c4142c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cac36700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c40ccad0 0x7fd8c4142c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cac36700 1 -- 172.21.15.112:0/2115090823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b4009550 con 0x7fd8c41445b0 2023-08-27T11:26:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8c9c34700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c40ccad0 0x7fd8c4142c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.323+0000 7fd8cac36700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8c41445b0 0x7fd8c40708a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00cb50 tx=0x7fd8bc00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.324+0000 7fd8baffd700 1 -- 172.21.15.112:0/2115090823 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc003c00 con 0x7fd8c41445b0 2023-08-27T11:26:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.324+0000 7fd8baffd700 1 -- 172.21.15.112:0/2115090823 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc003d60 con 0x7fd8c41445b0 2023-08-27T11:26:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.324+0000 7fd8cc699700 1 -- 172.21.15.112:0/2115090823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c40cb090 con 0x7fd8c41445b0 2023-08-27T11:26:34.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.324+0000 7fd8baffd700 1 -- 172.21.15.112:0/2115090823 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc00ab70 con 0x7fd8c41445b0 2023-08-27T11:26:34.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.324+0000 7fd8cc699700 1 -- 172.21.15.112:0/2115090823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c40cb650 con 0x7fd8c41445b0 2023-08-27T11:26:34.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.325+0000 7fd8cc699700 1 -- 172.21.15.112:0/2115090823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c4067e90 con 0x7fd8c41445b0 2023-08-27T11:26:34.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.325+0000 7fd8baffd700 1 -- 172.21.15.112:0/2115090823 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7fd8bc00acd0 con 0x7fd8c41445b0 2023-08-27T11:26:34.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.325+0000 7fd8baffd700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd8a806a3b0 0x7fd8a806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:34.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.326+0000 7fd8baffd700 1 -- 172.21.15.112:0/2115090823 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd8bc088a90 con 0x7fd8c41445b0 2023-08-27T11:26:34.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.326+0000 7fd8ca435700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd8a806a3b0 0x7fd8a806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:34.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.326+0000 7fd8ca435700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd8a806a3b0 0x7fd8a806c870 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009930 tx=0x7fd8c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:34.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.330+0000 7fd8baffd700 1 -- 172.21.15.112:0/2115090823 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd8bc053df0 con 0x7fd8c41445b0 2023-08-27T11:26:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.552+0000 7fd8cc699700 1 -- 172.21.15.112:0/2115090823 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "human_readable": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd8c40b10f0 con 0x7fd8a806a3b0 2023-08-27T11:26:34.560 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:26:34.559+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:26:34.560 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:26:34.559+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:26:34.560 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:26:34.559+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:26:34.561 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:26:34.559+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:26:34.561 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:26:34.559+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:26:34.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.566+0000 7fd8baffd700 1 -- 172.21.15.112:0/2115090823 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+471 (secure 0 0 0) 0x7fd8c0009e00 con 0x7fd8a806a3b0 2023-08-27T11:26:34.567 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:26:34.568 INFO:teuthology.orchestra.run.smithi112.stdout: "mon_addrs": [ 2023-08-27T11:26:34.568 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.112:6789", 2023-08-27T11:26:34.568 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.138:6789", 2023-08-27T11:26:34.568 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.138:6790" 2023-08-27T11:26:34.568 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2023-08-27T11:26:34.569 INFO:teuthology.orchestra.run.smithi112.stdout: "pools": { 2023-08-27T11:26:34.569 INFO:teuthology.orchestra.run.smithi112.stdout: "data": [ 2023-08-27T11:26:34.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:34.569 INFO:teuthology.orchestra.run.smithi112.stdout: "avail": " 341G", 2023-08-27T11:26:34.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs_data", 2023-08-27T11:26:34.570 INFO:teuthology.orchestra.run.smithi112.stdout: "used": " 0 " 2023-08-27T11:26:34.570 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:34.570 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2023-08-27T11:26:34.570 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": [ 2023-08-27T11:26:34.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:34.571 INFO:teuthology.orchestra.run.smithi112.stdout: "avail": " 341G", 2023-08-27T11:26:34.571 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs_metadata", 2023-08-27T11:26:34.571 INFO:teuthology.orchestra.run.smithi112.stdout: "used": "64.0k" 2023-08-27T11:26:34.571 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:34.572 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:26:34.572 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:34.572 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:26:34.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.568+0000 7fd8cc699700 1 -- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd8a806a3b0 msgr2=0x7fd8a806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:34.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.568+0000 7fd8cc699700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd8a806a3b0 0x7fd8a806c870 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009930 tx=0x7fd8c0008040 comp rx=0 tx=0).stop 2023-08-27T11:26:34.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.568+0000 7fd8cc699700 1 -- 172.21.15.112:0/2115090823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8c41445b0 msgr2=0x7fd8c40708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:34.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.568+0000 7fd8cc699700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8c41445b0 0x7fd8c40708a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00cb50 tx=0x7fd8bc00cf10 comp rx=0 tx=0).stop 2023-08-27T11:26:34.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.568+0000 7fd8cc699700 1 -- 172.21.15.112:0/2115090823 shutdown_connections 2023-08-27T11:26:34.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.568+0000 7fd8cc699700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd8a806a3b0 0x7fd8a806c870 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.568+0000 7fd8cc699700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8c41445b0 0x7fd8c40708a0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.568+0000 7fd8cc699700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8c406f790 0x7fd8c40cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.568+0000 7fd8cc699700 1 --2- 172.21.15.112:0/2115090823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c40ccad0 0x7fd8c4142c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.568+0000 7fd8cc699700 1 -- 172.21.15.112:0/2115090823 >> 172.21.15.112:0/2115090823 conn(0x7fd8c40b1e90 msgr2=0x7fd8c4071e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:34.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.568+0000 7fd8cc699700 1 -- 172.21.15.112:0/2115090823 shutdown_connections 2023-08-27T11:26:34.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.568+0000 7fd8cc699700 1 -- 172.21.15.112:0/2115090823 wait complete. 2023-08-27T11:26:34.581 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:26:34.582 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:26:34.619 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:26:34.670 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:26:34.671 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:26:34.671 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:34.688 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:26:34.689 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:34.745 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.903+0000 7f5fed19c700 1 Processor -- start 2023-08-27T11:26:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.903+0000 7f5fed19c700 1 -- start start 2023-08-27T11:26:34.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fed19c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fe8071ac0 0x7f5fe80b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:34.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fed19c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5fe80b4a20 0x7f5fe80701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:34.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fed19c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe80b3440 con 0x7f5fe80716e0 2023-08-27T11:26:34.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fed19c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe80b3720 con 0x7f5fe80b4a20 2023-08-27T11:26:34.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fed19c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80b3a00 con 0x7f5fe8071ac0 2023-08-27T11:26:34.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fe759e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5fe80b4a20 0x7f5fe80701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:34.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fe759e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5fe80b4a20 0x7f5fe80701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33158/0 (socket says 172.21.15.112:33158) 2023-08-27T11:26:34.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fe759e700 1 -- 172.21.15.112:0/1051809120 learned_addr learned my addr 172.21.15.112:0/1051809120 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:34.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fe659c700 1 --2- 172.21.15.112:0/1051809120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fe8071ac0 0x7f5fe80b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:34.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fe759e700 1 -- 172.21.15.112:0/1051809120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fe8071ac0 msgr2=0x7f5fe80b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:34.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fe759e700 1 --2- 172.21.15.112:0/1051809120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fe8071ac0 0x7f5fe80b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fe759e700 1 -- 172.21.15.112:0/1051809120 >> v1:172.21.15.138:6789/0 conn(0x7f5fe80716e0 legacy=0x7f5fe80d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:34.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fe759e700 1 -- 172.21.15.112:0/1051809120 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe80b3ce0 con 0x7f5fe80b4a20 2023-08-27T11:26:34.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.904+0000 7f5fe759e700 1 --2- 172.21.15.112:0/1051809120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe80b4a20 0x7f5fe80701a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8009e50 tx=0x7f5fd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c73f3b4af788f8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:34.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.905+0000 7f5fe5d9b700 1 -- 172.21.15.112:0/1051809120 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd800e070 con 0x7f5fe80b4a20 2023-08-27T11:26:34.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.905+0000 7f5fe5d9b700 1 -- 172.21.15.112:0/1051809120 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd8003a20 con 0x7f5fe80b4a20 2023-08-27T11:26:34.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.905+0000 7f5fe5d9b700 1 -- 172.21.15.112:0/1051809120 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8003dd0 con 0x7f5fe80b4a20 2023-08-27T11:26:34.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.905+0000 7f5fed19c700 1 -- 172.21.15.112:0/1051809120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe80b4a20 msgr2=0x7f5fe80701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:34.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.905+0000 7f5fed19c700 1 --2- 172.21.15.112:0/1051809120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe80b4a20 0x7f5fe80701a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8009e50 tx=0x7f5fd800b330 comp rx=0 tx=0).stop 2023-08-27T11:26:34.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.905+0000 7f5fed19c700 1 -- 172.21.15.112:0/1051809120 shutdown_connections 2023-08-27T11:26:34.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.905+0000 7f5fed19c700 1 --2- 172.21.15.112:0/1051809120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe80b4a20 0x7f5fe80701a0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.905+0000 7f5fed19c700 1 --2- 172.21.15.112:0/1051809120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fe8071ac0 0x7f5fe80b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.905+0000 7f5fed19c700 1 -- 172.21.15.112:0/1051809120 >> 172.21.15.112:0/1051809120 conn(0x7f5fe806c920 msgr2=0x7f5fe806cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:34.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.905+0000 7f5fed19c700 1 -- 172.21.15.112:0/1051809120 shutdown_connections 2023-08-27T11:26:34.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.905+0000 7f5fed19c700 1 -- 172.21.15.112:0/1051809120 wait complete. 2023-08-27T11:26:34.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.907+0000 7f5fed19c700 1 Processor -- start 2023-08-27T11:26:34.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.907+0000 7f5fed19c700 1 -- start start 2023-08-27T11:26:34.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.907+0000 7f5fed19c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fe80716e0 0x7f5fe8150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:34.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.907+0000 7f5fed19c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe8071ac0 0x7f5fe81528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:34.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.907+0000 7f5fed19c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe80b4a20 0x7f5fe81567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:34.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.907+0000 7f5fed19c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81484a0 con 0x7f5fe8071ac0 2023-08-27T11:26:34.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.907+0000 7f5fed19c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe8158470 con 0x7f5fe80b4a20 2023-08-27T11:26:34.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.907+0000 7f5fed19c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe8159220 con 0x7f5fe80716e0 2023-08-27T11:26:34.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.907+0000 7f5fe659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe8071ac0 0x7f5fe81528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:34.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.907+0000 7f5fe659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe8071ac0 0x7f5fe81528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33162/0 (socket says 172.21.15.112:33162) 2023-08-27T11:26:34.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.907+0000 7f5fe659c700 1 -- 172.21.15.112:0/4002492072 learned_addr learned my addr 172.21.15.112:0/4002492072 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:34.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.908+0000 7f5fe759e700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe80b4a20 0x7f5fe81567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:34.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.908+0000 7f5fe6d9d700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fe80716e0 0x7f5fe8150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:34.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.908+0000 7f5fe659c700 1 -- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fe80716e0 msgr2=0x7f5fe8150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:34.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.908+0000 7f5fe659c700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fe80716e0 0x7f5fe8150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.908+0000 7f5fe659c700 1 -- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe80b4a20 msgr2=0x7f5fe81567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:34.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.908+0000 7f5fe659c700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe80b4a20 0x7f5fe81567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:34.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.908+0000 7f5fe659c700 1 -- 172.21.15.112:0/4002492072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd8009b30 con 0x7f5fe8071ac0 2023-08-27T11:26:34.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.908+0000 7f5fe659c700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe8071ac0 0x7f5fe81528c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000c7f0 tx=0x7f5fd000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:34.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.908+0000 7f5fdf7fe700 1 -- 172.21.15.112:0/4002492072 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0011800 con 0x7f5fe8071ac0 2023-08-27T11:26:34.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.908+0000 7f5fdf7fe700 1 -- 172.21.15.112:0/4002492072 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd0011960 con 0x7f5fe8071ac0 2023-08-27T11:26:34.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.908+0000 7f5fed19c700 1 -- 172.21.15.112:0/4002492072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe81586a0 con 0x7f5fe8071ac0 2023-08-27T11:26:34.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.908+0000 7f5fdf7fe700 1 -- 172.21.15.112:0/4002492072 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0011cf0 con 0x7f5fe8071ac0 2023-08-27T11:26:34.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.908+0000 7f5fed19c700 1 -- 172.21.15.112:0/4002492072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe8158a30 con 0x7f5fe8071ac0 2023-08-27T11:26:34.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.909+0000 7f5fed19c700 1 -- 172.21.15.112:0/4002492072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe8067a20 con 0x7f5fe8071ac0 2023-08-27T11:26:34.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.909+0000 7f5fdf7fe700 1 -- 172.21.15.112:0/4002492072 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f5fd0018360 con 0x7f5fe8071ac0 2023-08-27T11:26:34.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.910+0000 7f5fdf7fe700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fc806a3b0 0x7f5fc806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:34.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.910+0000 7f5fdf7fe700 1 -- 172.21.15.112:0/4002492072 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5fd0088a60 con 0x7f5fe8071ac0 2023-08-27T11:26:34.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.910+0000 7f5fe6d9d700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fc806a3b0 0x7f5fc806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:34.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.911+0000 7f5fe6d9d700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fc806a3b0 0x7f5fc806c870 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0004f30 tx=0x7f5fe0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:34.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:34.914+0000 7f5fdf7fe700 1 -- 172.21.15.112:0/4002492072 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5fd0057790 con 0x7f5fe8071ac0 2023-08-27T11:26:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.104+0000 7f5fed19c700 1 -- 172.21.15.112:0/4002492072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fe81f5e00 con 0x7f5fe8071ac0 2023-08-27T11:26:35.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.106+0000 7f5fdf7fe700 1 -- 172.21.15.112:0/4002492072 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+18248 (secure 0 0 0) 0x7f5fd0057320 con 0x7f5fe8071ac0 2023-08-27T11:26:35.107 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:20.416307+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:16.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:17.773993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:35.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.108+0000 7f5fed19c700 1 -- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fc806a3b0 msgr2=0x7f5fc806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.108+0000 7f5fed19c700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fc806a3b0 0x7f5fc806c870 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0004f30 tx=0x7f5fe0009040 comp rx=0 tx=0).stop 2023-08-27T11:26:35.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.108+0000 7f5fed19c700 1 -- 172.21.15.112:0/4002492072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe8071ac0 msgr2=0x7f5fe81528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.108+0000 7f5fed19c700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe8071ac0 0x7f5fe81528c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000c7f0 tx=0x7f5fd000cbb0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.108+0000 7f5fed19c700 1 -- 172.21.15.112:0/4002492072 shutdown_connections 2023-08-27T11:26:35.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.108+0000 7f5fed19c700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5fc806a3b0 0x7f5fc806c870 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.108+0000 7f5fed19c700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe80b4a20 0x7f5fe81567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.108+0000 7f5fed19c700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fe80716e0 0x7f5fe8150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.108+0000 7f5fed19c700 1 --2- 172.21.15.112:0/4002492072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe8071ac0 0x7f5fe81528c0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.108+0000 7f5fed19c700 1 -- 172.21.15.112:0/4002492072 >> 172.21.15.112:0/4002492072 conn(0x7f5fe806c920 msgr2=0x7f5fe806f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:35.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.108+0000 7f5fed19c700 1 -- 172.21.15.112:0/4002492072 shutdown_connections 2023-08-27T11:26:35.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.108+0000 7f5fed19c700 1 -- 172.21.15.112:0/4002492072 wait complete. 2023-08-27T11:26:35.122 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:26:16.548692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:26:17.773993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:26:35.122 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:26:35.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.244+0000 7f6607954700 1 Processor -- start 2023-08-27T11:26:35.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.244+0000 7f6607954700 1 -- start start 2023-08-27T11:26:35.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f6607954700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66000708a0 0x7f66000cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f6607954700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66000cf230 con 0x7f66001466c0 2023-08-27T11:26:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f6607954700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66000cb6e0 con 0x7f660006bfd0 2023-08-27T11:26:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f6607954700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66000719b0 con 0x7f66000708a0 2023-08-27T11:26:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f66056f0700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f66001466c0 0x7f66000cafd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55782/0 (socket says 172.21.15.112:55782) 2023-08-27T11:26:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f66056f0700 1 -- 172.21.15.112:0/4030176015 learned_addr learned my addr 172.21.15.112:0/4030176015 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f6604eef700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66000708a0 0x7f66000cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171415845 0 0) 0x7f66000cf230 con 0x7f66001466c0 2023-08-27T11:26:35.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65ec0035b0 con 0x7f66001466c0 2023-08-27T11:26:35.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684929648 0 0) 0x7f66000cb6e0 con 0x7f660006bfd0 2023-08-27T11:26:35.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66000cf230 con 0x7f660006bfd0 2023-08-27T11:26:35.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 299304770 0 0) 0x7f65ec0035b0 con 0x7f66001466c0 2023-08-27T11:26:35.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66000708a0 msgr2=0x7f66000cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:35.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f65f7fff700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66000708a0 0x7f66000cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 >> v1:172.21.15.138:6789/0 conn(0x7f660006bfd0 legacy=0x7f6600071dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6600147000 con 0x7f66001466c0 2023-08-27T11:26:35.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.245+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f65fc001670 con 0x7f66001466c0 2023-08-27T11:26:35.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 >> v1:172.21.15.112:6789/0 conn(0x7f66001466c0 legacy=0x7f66000cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f65f7fff700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec003bc0 0x7f65ec003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f65f7fff700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65ec004850 0x7f65ec004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f65f7fff700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65ec005210 0x7f65ec00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000719b0 con 0x7f65ec003bc0 2023-08-27T11:26:35.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec00bf50 con 0x7f65ec004850 2023-08-27T11:26:35.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec00c090 con 0x7f65ec005210 2023-08-27T11:26:35.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f66056f0700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec003bc0 0x7f65ec003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f6604eef700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65ec004850 0x7f65ec004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f6605ef1700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65ec005210 0x7f65ec00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f66056f0700 1 -- 172.21.15.112:0/4030176015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65ec005210 msgr2=0x7f65ec00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:35.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f66056f0700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65ec005210 0x7f65ec00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f66056f0700 1 -- 172.21.15.112:0/4030176015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65ec004850 msgr2=0x7f65ec004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:35.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f66056f0700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65ec004850 0x7f65ec004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f66056f0700 1 -- 172.21.15.112:0/4030176015 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec0035b0 con 0x7f65ec003bc0 2023-08-27T11:26:35.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f6604eef700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65ec004850 0x7f65ec004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:35.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f6605ef1700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65ec005210 0x7f65ec00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:35.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f66056f0700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec003bc0 0x7f65ec003fc0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00c160 tx=0x7f65ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f986c44c176b740e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:35.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00cca0 con 0x7f65ec003bc0 2023-08-27T11:26:35.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc00ce20 con 0x7f65ec003bc0 2023-08-27T11:26:35.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.246+0000 7f65f7fff700 1 -- 172.21.15.112:0/4030176015 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc005ba0 con 0x7f65ec003bc0 2023-08-27T11:26:35.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.247+0000 7f6607954700 1 -- 172.21.15.112:0/4030176015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec003bc0 msgr2=0x7f65ec003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.247+0000 7f6607954700 1 --2- 172.21.15.112:0/4030176015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec003bc0 0x7f65ec003fc0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00c160 tx=0x7f65ec005880 comp rx=0 tx=0).stop 2023-08-27T11:26:35.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.247+0000 7f6605ef1700 1 -- 172.21.15.112:0/4030176015 reap_dead start 2023-08-27T11:26:35.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.247+0000 7f6607954700 1 -- 172.21.15.112:0/4030176015 shutdown_connections 2023-08-27T11:26:35.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.247+0000 7f6607954700 1 -- 172.21.15.112:0/4030176015 >> 172.21.15.112:0/4030176015 conn(0x7f66000cdc50 msgr2=0x7f66000ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:35.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.247+0000 7f6607954700 1 -- 172.21.15.112:0/4030176015 shutdown_connections 2023-08-27T11:26:35.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.247+0000 7f6607954700 1 -- 172.21.15.112:0/4030176015 wait complete. 2023-08-27T11:26:35.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.248+0000 7f6607954700 1 Processor -- start 2023-08-27T11:26:35.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.248+0000 7f6607954700 1 -- start start 2023-08-27T11:26:35.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f6607954700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec003bc0 0x7f6600072040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f6607954700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66001595e0 0x7f6600159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f6607954700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6600157790 0x7f6600157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f6607954700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660015bbb0 con 0x7f65ec003bc0 2023-08-27T11:26:35.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f6607954700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660015a810 con 0x7f66001595e0 2023-08-27T11:26:35.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f6607954700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660015b880 con 0x7f6600157790 2023-08-27T11:26:35.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f66056f0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec003bc0 0x7f6600072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f66056f0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec003bc0 0x7f6600072040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33180/0 (socket says 172.21.15.112:33180) 2023-08-27T11:26:35.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f66056f0700 1 -- 172.21.15.112:0/2291288603 learned_addr learned my addr 172.21.15.112:0/2291288603 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:35.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f6604eef700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66001595e0 0x7f6600159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f6605ef1700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6600157790 0x7f6600157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f66056f0700 1 -- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6600157790 msgr2=0x7f6600157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f66056f0700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6600157790 0x7f6600157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f66056f0700 1 -- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66001595e0 msgr2=0x7f6600159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f66056f0700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66001595e0 0x7f6600159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f66056f0700 1 -- 172.21.15.112:0/2291288603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6600159fa0 con 0x7f65ec003bc0 2023-08-27T11:26:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f66056f0700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec003bc0 0x7f6600072040 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f65ec005800 tx=0x7f65fc001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f65f5ffb700 1 -- 172.21.15.112:0/2291288603 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0070d0 con 0x7f65ec003bc0 2023-08-27T11:26:35.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f65f5ffb700 1 -- 172.21.15.112:0/2291288603 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc007230 con 0x7f65ec003bc0 2023-08-27T11:26:35.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.249+0000 7f6607954700 1 -- 172.21.15.112:0/2291288603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6600158150 con 0x7f65ec003bc0 2023-08-27T11:26:35.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.250+0000 7f65f5ffb700 1 -- 172.21.15.112:0/2291288603 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0075c0 con 0x7f65ec003bc0 2023-08-27T11:26:35.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.250+0000 7f6607954700 1 -- 172.21.15.112:0/2291288603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660015a530 con 0x7f65ec003bc0 2023-08-27T11:26:35.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.251+0000 7f6607954700 1 -- 172.21.15.112:0/2291288603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660015b9c0 con 0x7f65ec003bc0 2023-08-27T11:26:35.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.251+0000 7f65f5ffb700 1 -- 172.21.15.112:0/2291288603 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f65fc019030 con 0x7f65ec003bc0 2023-08-27T11:26:35.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.251+0000 7f65f5ffb700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65e406a3e0 0x7f65e406c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.251+0000 7f65f5ffb700 1 -- 172.21.15.112:0/2291288603 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f65fc089a50 con 0x7f65ec003bc0 2023-08-27T11:26:35.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.252+0000 7f6604eef700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65e406a3e0 0x7f65e406c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.252+0000 7f6604eef700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65e406a3e0 0x7f65e406c8a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6600143320 tx=0x7f65f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:35.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.256+0000 7f65f5ffb700 1 -- 172.21.15.112:0/2291288603 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f65fc054e80 con 0x7f65ec003bc0 2023-08-27T11:26:35.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.449+0000 7f6607954700 1 -- 172.21.15.112:0/2291288603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6600067a20 con 0x7f65ec003bc0 2023-08-27T11:26:35.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.450+0000 7f65f5ffb700 1 -- 172.21.15.112:0/2291288603 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v90) v1 ==== 92+0+10 (secure 0 0 0) 0x7f65fc0170a0 con 0x7f65ec003bc0 2023-08-27T11:26:35.452 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:26:35.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.452+0000 7f6607954700 1 -- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65e406a3e0 msgr2=0x7f65e406c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.452+0000 7f6607954700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65e406a3e0 0x7f65e406c8a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6600143320 tx=0x7f65f0008040 comp rx=0 tx=0).stop 2023-08-27T11:26:35.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.452+0000 7f6607954700 1 -- 172.21.15.112:0/2291288603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec003bc0 msgr2=0x7f6600072040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.452+0000 7f6607954700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec003bc0 0x7f6600072040 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f65ec005800 tx=0x7f65fc001930 comp rx=0 tx=0).stop 2023-08-27T11:26:35.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.452+0000 7f6607954700 1 -- 172.21.15.112:0/2291288603 shutdown_connections 2023-08-27T11:26:35.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.452+0000 7f6607954700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65e406a3e0 0x7f65e406c8a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.452+0000 7f6607954700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6600157790 0x7f6600157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.452+0000 7f6607954700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec003bc0 0x7f6600072040 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.452+0000 7f6607954700 1 --2- 172.21.15.112:0/2291288603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66001595e0 0x7f6600159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.452+0000 7f6607954700 1 -- 172.21.15.112:0/2291288603 >> 172.21.15.112:0/2291288603 conn(0x7f66000cdc50 msgr2=0x7f6600160a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:35.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.452+0000 7f6607954700 1 -- 172.21.15.112:0/2291288603 shutdown_connections 2023-08-27T11:26:35.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.452+0000 7f6607954700 1 -- 172.21.15.112:0/2291288603 wait complete. 2023-08-27T11:26:35.465 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:26:35.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.587+0000 7f59ee863700 1 Processor -- start 2023-08-27T11:26:35.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.587+0000 7f59ee863700 1 -- start start 2023-08-27T11:26:35.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59ee863700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59e80b3440 0x7f59e815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59ee863700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e80b5070 con 0x7f59e8146c20 2023-08-27T11:26:35.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59ee863700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e8147000 con 0x7f59e815e6e0 2023-08-27T11:26:35.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59ee863700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e8161740 con 0x7f59e80b3440 2023-08-27T11:26:35.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e7fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f59e8146c20 0x7f59e8160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55802/0 (socket says 172.21.15.112:55802) 2023-08-27T11:26:35.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e7fff700 1 -- 172.21.15.112:0/3951952710 learned_addr learned my addr 172.21.15.112:0/3951952710 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:35.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e77fe700 1 --2- 172.21.15.112:0/3951952710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59e80b3440 0x7f59e815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613835950 0 0) 0x7f59d8001110 con 0x7f59e8146c20 2023-08-27T11:26:35.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59d00035b0 con 0x7f59e8146c20 2023-08-27T11:26:35.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1633546408 0 0) 0x7f59d00035b0 con 0x7f59e8146c20 2023-08-27T11:26:35.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59e80b3440 msgr2=0x7f59e815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:35.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 --2- 172.21.15.112:0/3951952710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59e80b3440 0x7f59e815e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 >> v1:172.21.15.138:6789/0 conn(0x7f59e815e6e0 legacy=0x7f59e806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e8161a20 con 0x7f59e8146c20 2023-08-27T11:26:35.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f59d8001800 con 0x7f59e8146c20 2023-08-27T11:26:35.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 >> v1:172.21.15.112:6789/0 conn(0x7f59e8146c20 legacy=0x7f59e8160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 --2- 172.21.15.112:0/3951952710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d0003a30 0x7f59d0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 --2- 172.21.15.112:0/3951952710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59e80b3440 0x7f59d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 --2- 172.21.15.112:0/3951952710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d0004d90 0x7f59d0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e8161740 con 0x7f59d0003a30 2023-08-27T11:26:35.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d000bac0 con 0x7f59d0004d90 2023-08-27T11:26:35.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.588+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d000bc80 con 0x7f59e80b3440 2023-08-27T11:26:35.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.589+0000 7f59e7fff700 1 --2- 172.21.15.112:0/3951952710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d0003a30 0x7f59d0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.589+0000 7f59e77fe700 1 --2- 172.21.15.112:0/3951952710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59e80b3440 0x7f59d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.589+0000 7f59ece00700 1 --2- 172.21.15.112:0/3951952710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d0004d90 0x7f59d0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.589+0000 7f59e7fff700 1 -- 172.21.15.112:0/3951952710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59e80b3440 msgr2=0x7f59d0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:35.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.589+0000 7f59e7fff700 1 --2- 172.21.15.112:0/3951952710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59e80b3440 0x7f59d0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.589+0000 7f59e7fff700 1 -- 172.21.15.112:0/3951952710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d0004d90 msgr2=0x7f59d0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:35.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.589+0000 7f59e7fff700 1 --2- 172.21.15.112:0/3951952710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d0004d90 0x7f59d0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.589+0000 7f59e7fff700 1 -- 172.21.15.112:0/3951952710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d00035b0 con 0x7f59d0003a30 2023-08-27T11:26:35.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.589+0000 7f59ece00700 1 --2- 172.21.15.112:0/3951952710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d0004d90 0x7f59d0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:35.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.589+0000 7f59e7fff700 1 --2- 172.21.15.112:0/3951952710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d0003a30 0x7f59d0003e10 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f59d800c240 tx=0x7f59d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685e0e80cc9da5f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:35.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.589+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800cde0 con 0x7f59d0003a30 2023-08-27T11:26:35.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.589+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d8005960 con 0x7f59d0003a30 2023-08-27T11:26:35.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.589+0000 7f59e6ffd700 1 -- 172.21.15.112:0/3951952710 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8005cf0 con 0x7f59d0003a30 2023-08-27T11:26:35.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.590+0000 7f59ee863700 1 -- 172.21.15.112:0/3951952710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d0003a30 msgr2=0x7f59d0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.590+0000 7f59ee863700 1 --2- 172.21.15.112:0/3951952710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d0003a30 0x7f59d0003e10 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f59d800c240 tx=0x7f59d0005880 comp rx=0 tx=0).stop 2023-08-27T11:26:35.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.590+0000 7f59ece00700 1 -- 172.21.15.112:0/3951952710 reap_dead start 2023-08-27T11:26:35.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.590+0000 7f59ee863700 1 -- 172.21.15.112:0/3951952710 shutdown_connections 2023-08-27T11:26:35.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.590+0000 7f59ee863700 1 -- 172.21.15.112:0/3951952710 >> 172.21.15.112:0/3951952710 conn(0x7f59e80b1e90 msgr2=0x7f59e80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:35.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.590+0000 7f59ee863700 1 -- 172.21.15.112:0/3951952710 shutdown_connections 2023-08-27T11:26:35.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.590+0000 7f59ee863700 1 -- 172.21.15.112:0/3951952710 wait complete. 2023-08-27T11:26:35.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.591+0000 7f59ee863700 1 Processor -- start 2023-08-27T11:26:35.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.591+0000 7f59ee863700 1 -- start start 2023-08-27T11:26:35.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ee863700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d0003a30 0x7f59e80af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ee863700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59e80ae950 0x7f59e815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ee863700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59e815c710 0x7f59e815cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ee863700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e813f8d0 con 0x7f59e815c710 2023-08-27T11:26:35.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ee863700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e815d0d0 con 0x7f59d0003a30 2023-08-27T11:26:35.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ee863700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e813ece0 con 0x7f59e80ae950 2023-08-27T11:26:35.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ece00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59e815c710 0x7f59e815cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ece00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59e815c710 0x7f59e815cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33204/0 (socket says 172.21.15.112:33204) 2023-08-27T11:26:35.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ece00700 1 -- 172.21.15.112:0/917890716 learned_addr learned my addr 172.21.15.112:0/917890716 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:35.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59e77fe700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59e80ae950 0x7f59e815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59e7fff700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d0003a30 0x7f59e80af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ece00700 1 -- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59e80ae950 msgr2=0x7f59e815a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ece00700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59e80ae950 0x7f59e815a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ece00700 1 -- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d0003a30 msgr2=0x7f59e80af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ece00700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d0003a30 0x7f59e80af070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ece00700 1 -- 172.21.15.112:0/917890716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e813fcf0 con 0x7f59e815c710 2023-08-27T11:26:35.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.592+0000 7f59ece00700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59e815c710 0x7f59e815cb90 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f59e815eac0 tx=0x7f59e000c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:35.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.593+0000 7f59e4ff9700 1 -- 172.21.15.112:0/917890716 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0003c10 con 0x7f59e815c710 2023-08-27T11:26:35.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.593+0000 7f59e4ff9700 1 -- 172.21.15.112:0/917890716 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e0003d70 con 0x7f59e815c710 2023-08-27T11:26:35.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.593+0000 7f59ee863700 1 -- 172.21.15.112:0/917890716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e813ef30 con 0x7f59e815c710 2023-08-27T11:26:35.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.593+0000 7f59e4ff9700 1 -- 172.21.15.112:0/917890716 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e000c920 con 0x7f59e815c710 2023-08-27T11:26:35.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.593+0000 7f59ee863700 1 -- 172.21.15.112:0/917890716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e813f4f0 con 0x7f59e815c710 2023-08-27T11:26:35.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.594+0000 7f59ee863700 1 -- 172.21.15.112:0/917890716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e813f6b0 con 0x7f59e815c710 2023-08-27T11:26:35.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.594+0000 7f59e4ff9700 1 -- 172.21.15.112:0/917890716 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f59e000ca80 con 0x7f59e815c710 2023-08-27T11:26:35.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.595+0000 7f59e4ff9700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59d406a3b0 0x7f59d406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.595+0000 7f59e4ff9700 1 -- 172.21.15.112:0/917890716 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f59e0088b00 con 0x7f59e815c710 2023-08-27T11:26:35.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.595+0000 7f59e7fff700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59d406a3b0 0x7f59d406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.595+0000 7f59e7fff700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59d406a3b0 0x7f59d406c870 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f59d8015040 tx=0x7f59d8007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:35.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.599+0000 7f59e4ff9700 1 -- 172.21.15.112:0/917890716 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f59e0053ea0 con 0x7f59e815c710 2023-08-27T11:26:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.785+0000 7f59ee863700 1 -- 172.21.15.112:0/917890716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59e8161cd0 con 0x7f59e815c710 2023-08-27T11:26:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.786+0000 7f59e4ff9700 1 -- 172.21.15.112:0/917890716 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f59e00574c0 con 0x7f59e815c710 2023-08-27T11:26:35.788 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:26:35.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.788+0000 7f59ee863700 1 -- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59d406a3b0 msgr2=0x7f59d406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.788+0000 7f59ee863700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59d406a3b0 0x7f59d406c870 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f59d8015040 tx=0x7f59d8007000 comp rx=0 tx=0).stop 2023-08-27T11:26:35.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.788+0000 7f59ee863700 1 -- 172.21.15.112:0/917890716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59e815c710 msgr2=0x7f59e815cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.788+0000 7f59ee863700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59e815c710 0x7f59e815cb90 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f59e815eac0 tx=0x7f59e000c0e0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.788+0000 7f59ee863700 1 -- 172.21.15.112:0/917890716 shutdown_connections 2023-08-27T11:26:35.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.788+0000 7f59ee863700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59d406a3b0 0x7f59d406c870 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.788+0000 7f59ee863700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59e815c710 0x7f59e815cb90 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.788+0000 7f59ee863700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d0003a30 0x7f59e80af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.788+0000 7f59ee863700 1 --2- 172.21.15.112:0/917890716 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59e80ae950 0x7f59e815a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.788+0000 7f59ee863700 1 -- 172.21.15.112:0/917890716 >> 172.21.15.112:0/917890716 conn(0x7f59e80b1e90 msgr2=0x7f59e80cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:35.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.788+0000 7f59ee863700 1 -- 172.21.15.112:0/917890716 shutdown_connections 2023-08-27T11:26:35.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.788+0000 7f59ee863700 1 -- 172.21.15.112:0/917890716 wait complete. 2023-08-27T11:26:35.801 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:26:35.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.923+0000 7f2629cf5700 1 Processor -- start 2023-08-27T11:26:35.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.923+0000 7f2629cf5700 1 -- start start 2023-08-27T11:26:35.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2629cf5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2624146c20 0x7f2624160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2629cf5700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f26240b3440 0x7f262415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2629cf5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f262415e6e0 0x7f262406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2629cf5700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26240b5070 con 0x7f262415e6e0 2023-08-27T11:26:35.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2629cf5700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2624147000 con 0x7f26240b3440 2023-08-27T11:26:35.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2629cf5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26241614a0 con 0x7f2624146c20 2023-08-27T11:26:35.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2623fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f262415e6e0 0x7f262406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2623fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f262415e6e0 0x7f262406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33212/0 (socket says 172.21.15.112:33212) 2023-08-27T11:26:35.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2623fff700 1 -- 172.21.15.112:0/747914017 learned_addr learned my addr 172.21.15.112:0/747914017 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:35.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2622ffd700 1 --2- 172.21.15.112:0/747914017 >> v2:172.21.15.138:3300/0 conn(0x7f26240b3440 0x7f262415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f26237fe700 1 --2- 172.21.15.112:0/747914017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2624146c20 0x7f2624160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2623fff700 1 -- 172.21.15.112:0/747914017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2624146c20 msgr2=0x7f2624160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:35.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2623fff700 1 --2- 172.21.15.112:0/747914017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2624146c20 0x7f2624160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2623fff700 1 -- 172.21.15.112:0/747914017 >> v2:172.21.15.138:3300/0 conn(0x7f26240b3440 msgr2=0x7f262415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2623fff700 1 --2- 172.21.15.112:0/747914017 >> v2:172.21.15.138:3300/0 conn(0x7f26240b3440 0x7f262415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2623fff700 1 -- 172.21.15.112:0/747914017 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2624161740 con 0x7f262415e6e0 2023-08-27T11:26:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.924+0000 7f2623fff700 1 --2- 172.21.15.112:0/747914017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262415e6e0 0x7f262406f790 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f26240b2db0 tx=0x7f261c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2147ca1d5b6e89 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:35.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.925+0000 7f26227fc700 1 -- 172.21.15.112:0/747914017 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c00e070 con 0x7f262415e6e0 2023-08-27T11:26:35.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.925+0000 7f26227fc700 1 -- 172.21.15.112:0/747914017 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c00b9c0 con 0x7f262415e6e0 2023-08-27T11:26:35.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.925+0000 7f26227fc700 1 -- 172.21.15.112:0/747914017 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c00bd70 con 0x7f262415e6e0 2023-08-27T11:26:35.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.925+0000 7f2629cf5700 1 -- 172.21.15.112:0/747914017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262415e6e0 msgr2=0x7f262406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.925+0000 7f2629cf5700 1 --2- 172.21.15.112:0/747914017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262415e6e0 0x7f262406f790 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f26240b2db0 tx=0x7f261c00b330 comp rx=0 tx=0).stop 2023-08-27T11:26:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.925+0000 7f2629cf5700 1 -- 172.21.15.112:0/747914017 shutdown_connections 2023-08-27T11:26:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.925+0000 7f2629cf5700 1 --2- 172.21.15.112:0/747914017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262415e6e0 0x7f262406f790 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.925+0000 7f2629cf5700 1 --2- 172.21.15.112:0/747914017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2624146c20 0x7f2624160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.925+0000 7f2629cf5700 1 --2- 172.21.15.112:0/747914017 >> v2:172.21.15.138:3300/0 conn(0x7f26240b3440 0x7f262415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.925+0000 7f2629cf5700 1 -- 172.21.15.112:0/747914017 >> 172.21.15.112:0/747914017 conn(0x7f26240b1e90 msgr2=0x7f26240b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.925+0000 7f2629cf5700 1 -- 172.21.15.112:0/747914017 shutdown_connections 2023-08-27T11:26:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.925+0000 7f2629cf5700 1 -- 172.21.15.112:0/747914017 wait complete. 2023-08-27T11:26:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.926+0000 7f2629cf5700 1 Processor -- start 2023-08-27T11:26:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.927+0000 7f2629cf5700 1 -- start start 2023-08-27T11:26:35.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.927+0000 7f2629cf5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26240b3440 0x7f26241450f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.927+0000 7f2629cf5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2624146c20 0x7f262406f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.927+0000 7f2629cf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f262415e6e0 0x7f2624136720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.927+0000 7f2629cf5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26241604a0 con 0x7f2624146c20 2023-08-27T11:26:35.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.927+0000 7f2629cf5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262415fc90 con 0x7f262415e6e0 2023-08-27T11:26:35.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.927+0000 7f2629cf5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2624145630 con 0x7f26240b3440 2023-08-27T11:26:35.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.927+0000 7f2622ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2624146c20 0x7f262406f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.927+0000 7f2622ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2624146c20 0x7f262406f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33216/0 (socket says 172.21.15.112:33216) 2023-08-27T11:26:35.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.927+0000 7f2622ffd700 1 -- 172.21.15.112:0/549400687 learned_addr learned my addr 172.21.15.112:0/549400687 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:35.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f2623fff700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f262415e6e0 0x7f2624136720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f26237fe700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26240b3440 0x7f26241450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f2622ffd700 1 -- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26240b3440 msgr2=0x7f26241450f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f2622ffd700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26240b3440 0x7f26241450f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f2622ffd700 1 -- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f262415e6e0 msgr2=0x7f2624136720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:35.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f2622ffd700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f262415e6e0 0x7f2624136720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:35.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f2622ffd700 1 -- 172.21.15.112:0/549400687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c009b30 con 0x7f2624146c20 2023-08-27T11:26:35.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f26237fe700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26240b3440 0x7f26241450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:35.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f2622ffd700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2624146c20 0x7f262406f870 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f261800d800 tx=0x7f261800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:35.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f260bfff700 1 -- 172.21.15.112:0/549400687 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26180125f0 con 0x7f2624146c20 2023-08-27T11:26:35.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f260bfff700 1 -- 172.21.15.112:0/549400687 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2618012750 con 0x7f2624146c20 2023-08-27T11:26:35.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f260bfff700 1 -- 172.21.15.112:0/549400687 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618012ae0 con 0x7f2624146c20 2023-08-27T11:26:35.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f2629cf5700 1 -- 172.21.15.112:0/549400687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262415fee0 con 0x7f2624146c20 2023-08-27T11:26:35.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.928+0000 7f2629cf5700 1 -- 172.21.15.112:0/549400687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26240c5290 con 0x7f2624146c20 2023-08-27T11:26:35.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.929+0000 7f2629cf5700 1 -- 172.21.15.112:0/549400687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26240c50f0 con 0x7f2624146c20 2023-08-27T11:26:35.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.929+0000 7f260bfff700 1 -- 172.21.15.112:0/549400687 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f2618012c40 con 0x7f2624146c20 2023-08-27T11:26:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.930+0000 7f260bfff700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f261006a3b0 0x7f261006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.930+0000 7f260bfff700 1 -- 172.21.15.112:0/549400687 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2618088cd0 con 0x7f2624146c20 2023-08-27T11:26:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.930+0000 7f26237fe700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f261006a3b0 0x7f261006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.930+0000 7f26237fe700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f261006a3b0 0x7f261006c870 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2614009a00 tx=0x7f2614008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:35.934+0000 7f260bfff700 1 -- 172.21.15.112:0/549400687 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f26240c50f0 con 0x7f2624146c20 2023-08-27T11:26:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.121+0000 7f2629cf5700 1 -- 172.21.15.112:0/549400687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26240c50f0 con 0x7f2624146c20 2023-08-27T11:26:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.122+0000 7f260bfff700 1 -- 172.21.15.112:0/549400687 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26240c50f0 con 0x7f2624146c20 2023-08-27T11:26:36.124 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:26:36.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.123+0000 7f2629cf5700 1 -- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f261006a3b0 msgr2=0x7f261006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.123+0000 7f2629cf5700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f261006a3b0 0x7f261006c870 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2614009a00 tx=0x7f2614008040 comp rx=0 tx=0).stop 2023-08-27T11:26:36.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.123+0000 7f2629cf5700 1 -- 172.21.15.112:0/549400687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2624146c20 msgr2=0x7f262406f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.123+0000 7f2629cf5700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2624146c20 0x7f262406f870 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f261800d800 tx=0x7f261800dbc0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.124+0000 7f2629cf5700 1 -- 172.21.15.112:0/549400687 shutdown_connections 2023-08-27T11:26:36.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.124+0000 7f2629cf5700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f261006a3b0 0x7f261006c870 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.124+0000 7f2629cf5700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f262415e6e0 0x7f2624136720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.124+0000 7f2629cf5700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26240b3440 0x7f26241450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.124+0000 7f2629cf5700 1 --2- 172.21.15.112:0/549400687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2624146c20 0x7f262406f870 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.124+0000 7f2629cf5700 1 -- 172.21.15.112:0/549400687 >> 172.21.15.112:0/549400687 conn(0x7f26240b1e90 msgr2=0x7f262406dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:36.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.124+0000 7f2629cf5700 1 -- 172.21.15.112:0/549400687 shutdown_connections 2023-08-27T11:26:36.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.124+0000 7f2629cf5700 1 -- 172.21.15.112:0/549400687 wait complete. 2023-08-27T11:26:36.137 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:26:36.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.258+0000 7f91da432700 1 Processor -- start 2023-08-27T11:26:36.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.258+0000 7f91da432700 1 -- start start 2023-08-27T11:26:36.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.259+0000 7f91da432700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f91d415ed00 0x7f91d40b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:36.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.259+0000 7f91da432700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d415f750 con 0x7f91d415f180 2023-08-27T11:26:36.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.259+0000 7f91da432700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d415fa30 con 0x7f91d4152130 2023-08-27T11:26:36.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.259+0000 7f91da432700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d40b60a0 con 0x7f91d415ed00 2023-08-27T11:26:36.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.259+0000 7f91d37fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f91d415ed00 0x7f91d40b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:36.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.259+0000 7f91d89cf700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f91d415f180 0x7f91d40b68a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55840/0 (socket says 172.21.15.112:55840) 2023-08-27T11:26:36.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.259+0000 7f91d89cf700 1 -- 172.21.15.112:0/2210654193 learned_addr learned my addr 172.21.15.112:0/2210654193 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:36.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.259+0000 7f91d2ffd700 1 -- 172.21.15.112:0/2210654193 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678258950 0 0) 0x7f91d415f750 con 0x7f91d415f180 2023-08-27T11:26:36.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.259+0000 7f91d2ffd700 1 -- 172.21.15.112:0/2210654193 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91bc0035b0 con 0x7f91d415f180 2023-08-27T11:26:36.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.259+0000 7f91d37fe700 1 -- 172.21.15.112:0/2210654193 >> v1:172.21.15.138:6789/0 conn(0x7f91d4152130 legacy=0x7f91d415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.259+0000 7f91d37fe700 1 -- 172.21.15.112:0/2210654193 >> v1:172.21.15.112:6789/0 conn(0x7f91d415f180 legacy=0x7f91d40b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.259+0000 7f91d37fe700 1 -- 172.21.15.112:0/2210654193 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d40b6380 con 0x7f91d415ed00 2023-08-27T11:26:36.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.259+0000 7f91d37fe700 1 --2- 172.21.15.112:0/2210654193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d415ed00 0x7f91d40b3cf0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009840 tx=0x7f91c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5579a677f82d07 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:36.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.260+0000 7f91d2ffd700 1 -- 172.21.15.112:0/2210654193 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4011830 con 0x7f91d415ed00 2023-08-27T11:26:36.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.260+0000 7f91d2ffd700 1 -- 172.21.15.112:0/2210654193 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c4011990 con 0x7f91d415ed00 2023-08-27T11:26:36.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.260+0000 7f91d2ffd700 1 -- 172.21.15.112:0/2210654193 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4011d20 con 0x7f91d415ed00 2023-08-27T11:26:36.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.260+0000 7f91da432700 1 -- 172.21.15.112:0/2210654193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d415ed00 msgr2=0x7f91d40b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.260+0000 7f91da432700 1 --2- 172.21.15.112:0/2210654193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d415ed00 0x7f91d40b3cf0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009840 tx=0x7f91c4009b50 comp rx=0 tx=0).stop 2023-08-27T11:26:36.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.260+0000 7f91da432700 1 -- 172.21.15.112:0/2210654193 shutdown_connections 2023-08-27T11:26:36.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.260+0000 7f91da432700 1 --2- 172.21.15.112:0/2210654193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d415ed00 0x7f91d40b3cf0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.260+0000 7f91da432700 1 -- 172.21.15.112:0/2210654193 >> 172.21.15.112:0/2210654193 conn(0x7f91d4066010 msgr2=0x7f91d40663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:36.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.260+0000 7f91da432700 1 -- 172.21.15.112:0/2210654193 shutdown_connections 2023-08-27T11:26:36.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.260+0000 7f91da432700 1 -- 172.21.15.112:0/2210654193 wait complete. 2023-08-27T11:26:36.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.261+0000 7f91da432700 1 Processor -- start 2023-08-27T11:26:36.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.261+0000 7f91da432700 1 -- start start 2023-08-27T11:26:36.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.262+0000 7f91da432700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d4152130 0x7f91d40d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:36.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.262+0000 7f91da432700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d415ed00 0x7f91d413d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:36.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.262+0000 7f91da432700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d415f180 0x7f91d4140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:36.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.262+0000 7f91da432700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41545d0 con 0x7f91d4152130 2023-08-27T11:26:36.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.262+0000 7f91da432700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d415b7a0 con 0x7f91d415f180 2023-08-27T11:26:36.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.262+0000 7f91da432700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d4153fa0 con 0x7f91d415ed00 2023-08-27T11:26:36.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.262+0000 7f91d3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d4152130 0x7f91d40d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:36.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.262+0000 7f91d3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d4152130 0x7f91d40d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33228/0 (socket says 172.21.15.112:33228) 2023-08-27T11:26:36.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.262+0000 7f91d3fff700 1 -- 172.21.15.112:0/397481316 learned_addr learned my addr 172.21.15.112:0/397481316 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:36.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.262+0000 7f91d37fe700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d415ed00 0x7f91d413d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:36.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.262+0000 7f91d89cf700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d415f180 0x7f91d4140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:36.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.263+0000 7f91d3fff700 1 -- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d415ed00 msgr2=0x7f91d413d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.263+0000 7f91d3fff700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d415ed00 0x7f91d413d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.263+0000 7f91d3fff700 1 -- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d415f180 msgr2=0x7f91d4140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.263+0000 7f91d3fff700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d415f180 0x7f91d4140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.263+0000 7f91d3fff700 1 -- 172.21.15.112:0/397481316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c40094a0 con 0x7f91d4152130 2023-08-27T11:26:36.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.263+0000 7f91d89cf700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d415f180 0x7f91d4140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:36.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.263+0000 7f91d3fff700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d4152130 0x7f91d40d06b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f91c800c4e0 tx=0x7f91c800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:36.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.263+0000 7f91d0ff9700 1 -- 172.21.15.112:0/397481316 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c80052d0 con 0x7f91d4152130 2023-08-27T11:26:36.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.263+0000 7f91d0ff9700 1 -- 172.21.15.112:0/397481316 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c8005430 con 0x7f91d4152130 2023-08-27T11:26:36.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.263+0000 7f91da432700 1 -- 172.21.15.112:0/397481316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d41406d0 con 0x7f91d4152130 2023-08-27T11:26:36.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.263+0000 7f91d0ff9700 1 -- 172.21.15.112:0/397481316 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c80038c0 con 0x7f91d4152130 2023-08-27T11:26:36.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.263+0000 7f91da432700 1 -- 172.21.15.112:0/397481316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d40d0f70 con 0x7f91d4152130 2023-08-27T11:26:36.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.264+0000 7f91da432700 1 -- 172.21.15.112:0/397481316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d40d1160 con 0x7f91d4152130 2023-08-27T11:26:36.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.264+0000 7f91d0ff9700 1 -- 172.21.15.112:0/397481316 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) v1 ==== 88596+0+0 (secure 0 0 0) 0x7f91c8003a20 con 0x7f91d4152130 2023-08-27T11:26:36.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.265+0000 7f91d0ff9700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91c006a3b0 0x7f91c006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:36.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.265+0000 7f91d0ff9700 1 -- 172.21.15.112:0/397481316 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f91c8087db0 con 0x7f91d4152130 2023-08-27T11:26:36.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.265+0000 7f91d37fe700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91c006a3b0 0x7f91c006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:36.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.266+0000 7f91d37fe700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91c006a3b0 0x7f91c006c870 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f91c40090b0 tx=0x7f91c4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:36.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.269+0000 7f91d0ff9700 1 -- 172.21.15.112:0/397481316 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f91c8053150 con 0x7f91d4152130 2023-08-27T11:26:36.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.459+0000 7f91da432700 1 -- 172.21.15.112:0/397481316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91d4067a20 con 0x7f91d4152130 2023-08-27T11:26:36.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.459+0000 7f91d0ff9700 1 -- 172.21.15.112:0/397481316 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91c8017020 con 0x7f91d4152130 2023-08-27T11:26:36.461 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:26:36.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.462+0000 7f91da432700 1 -- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91c006a3b0 msgr2=0x7f91c006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.462+0000 7f91da432700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91c006a3b0 0x7f91c006c870 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f91c40090b0 tx=0x7f91c4008030 comp rx=0 tx=0).stop 2023-08-27T11:26:36.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.462+0000 7f91da432700 1 -- 172.21.15.112:0/397481316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d4152130 msgr2=0x7f91d40d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.462+0000 7f91da432700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d4152130 0x7f91d40d06b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f91c800c4e0 tx=0x7f91c800c7f0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.462+0000 7f91da432700 1 -- 172.21.15.112:0/397481316 shutdown_connections 2023-08-27T11:26:36.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.462+0000 7f91da432700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91c006a3b0 0x7f91c006c870 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.462+0000 7f91da432700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d415f180 0x7f91d4140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.462+0000 7f91da432700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d4152130 0x7f91d40d06b0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.462+0000 7f91da432700 1 --2- 172.21.15.112:0/397481316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d415ed00 0x7f91d413d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.462+0000 7f91da432700 1 -- 172.21.15.112:0/397481316 >> 172.21.15.112:0/397481316 conn(0x7f91d4066010 msgr2=0x7f91d40b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:36.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.462+0000 7f91da432700 1 -- 172.21.15.112:0/397481316 shutdown_connections 2023-08-27T11:26:36.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.462+0000 7f91da432700 1 -- 172.21.15.112:0/397481316 wait complete. 2023-08-27T11:26:36.475 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:26:36.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.596+0000 7fe9bf7b5700 1 Processor -- start 2023-08-27T11:26:36.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.597+0000 7fe9bf7b5700 1 -- start start 2023-08-27T11:26:36.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.597+0000 7fe9bf7b5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9b815a120 0x7fe9b80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:36.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.597+0000 7fe9bf7b5700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe9b806d5b0 0x7fe9b806da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:36.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.597+0000 7fe9bf7b5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe9b80b3440 0x7fe9b80b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:36.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.597+0000 7fe9bf7b5700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b806df50 con 0x7fe9b80b3440 2023-08-27T11:26:36.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.597+0000 7fe9bf7b5700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b815b540 con 0x7fe9b806d5b0 2023-08-27T11:26:36.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.597+0000 7fe9bf7b5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b80af830 con 0x7fe9b815a120 2023-08-27T11:26:36.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.597+0000 7fe9bdd52700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe9b80b3440 0x7fe9b80b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:36.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.597+0000 7fe9bdd52700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe9b80b3440 0x7fe9b80b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33240/0 (socket says 172.21.15.112:33240) 2023-08-27T11:26:36.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.597+0000 7fe9bdd52700 1 -- 172.21.15.112:0/4056970056 learned_addr learned my addr 172.21.15.112:0/4056970056 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:36.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.598+0000 7fe9bd551700 1 --2- 172.21.15.112:0/4056970056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9b815a120 0x7fe9b80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:36.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.598+0000 7fe9bcd50700 1 --2- 172.21.15.112:0/4056970056 >> v2:172.21.15.138:3300/0 conn(0x7fe9b806d5b0 0x7fe9b806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:36.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.598+0000 7fe9bdd52700 1 -- 172.21.15.112:0/4056970056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9b815a120 msgr2=0x7fe9b80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:36.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.598+0000 7fe9bdd52700 1 --2- 172.21.15.112:0/4056970056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9b815a120 0x7fe9b80b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.598+0000 7fe9bdd52700 1 -- 172.21.15.112:0/4056970056 >> v2:172.21.15.138:3300/0 conn(0x7fe9b806d5b0 msgr2=0x7fe9b806da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:36.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.598+0000 7fe9bdd52700 1 --2- 172.21.15.112:0/4056970056 >> v2:172.21.15.138:3300/0 conn(0x7fe9b806d5b0 0x7fe9b806da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.598+0000 7fe9bdd52700 1 -- 172.21.15.112:0/4056970056 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b80afb10 con 0x7fe9b80b3440 2023-08-27T11:26:36.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.598+0000 7fe9bd551700 1 --2- 172.21.15.112:0/4056970056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9b815a120 0x7fe9b80b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:36.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.598+0000 7fe9bcd50700 1 --2- 172.21.15.112:0/4056970056 >> v2:172.21.15.138:3300/0 conn(0x7fe9b806d5b0 0x7fe9b806da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.598+0000 7fe9bdd52700 1 --2- 172.21.15.112:0/4056970056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b80b3440 0x7fe9b80b38b0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe9b806f010 tx=0x7fe9b000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d4e55d94e9e218 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.598+0000 7fe9affff700 1 -- 172.21.15.112:0/4056970056 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0010070 con 0x7fe9b80b3440 2023-08-27T11:26:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.598+0000 7fe9affff700 1 -- 172.21.15.112:0/4056970056 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b000dd90 con 0x7fe9b80b3440 2023-08-27T11:26:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.598+0000 7fe9affff700 1 -- 172.21.15.112:0/4056970056 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b000b310 con 0x7fe9b80b3440 2023-08-27T11:26:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.599+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/4056970056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b80b3440 msgr2=0x7fe9b80b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.599+0000 7fe9bf7b5700 1 --2- 172.21.15.112:0/4056970056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b80b3440 0x7fe9b80b38b0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe9b806f010 tx=0x7fe9b000d430 comp rx=0 tx=0).stop 2023-08-27T11:26:36.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.599+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/4056970056 shutdown_connections 2023-08-27T11:26:36.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.599+0000 7fe9bf7b5700 1 --2- 172.21.15.112:0/4056970056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b80b3440 0x7fe9b80b38b0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.599+0000 7fe9bf7b5700 1 --2- 172.21.15.112:0/4056970056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9b815a120 0x7fe9b80b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.599+0000 7fe9bf7b5700 1 --2- 172.21.15.112:0/4056970056 >> v2:172.21.15.138:3300/0 conn(0x7fe9b806d5b0 0x7fe9b806da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.599+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/4056970056 >> 172.21.15.112:0/4056970056 conn(0x7fe9b80cff80 msgr2=0x7fe9b80d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.599+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/4056970056 shutdown_connections 2023-08-27T11:26:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.599+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/4056970056 wait complete. 2023-08-27T11:26:36.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.600+0000 7fe9bf7b5700 1 Processor -- start 2023-08-27T11:26:36.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.600+0000 7fe9bf7b5700 1 -- start start 2023-08-27T11:26:36.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bf7b5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b806d5b0 0x7fe9b814ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bf7b5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9b80b3440 0x7fe9b8151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bf7b5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9b815a120 0x7fe9b8157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bf7b5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b814e9e0 con 0x7fe9b806d5b0 2023-08-27T11:26:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bf7b5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b8151aa0 con 0x7fe9b815a120 2023-08-27T11:26:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bf7b5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b8151d80 con 0x7fe9b80b3440 2023-08-27T11:26:36.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bd551700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b806d5b0 0x7fe9b814ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:36.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bd551700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b806d5b0 0x7fe9b814ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33242/0 (socket says 172.21.15.112:33242) 2023-08-27T11:26:36.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bd551700 1 -- 172.21.15.112:0/1532545036 learned_addr learned my addr 172.21.15.112:0/1532545036 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:36.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bdd52700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9b815a120 0x7fe9b8157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:36.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bcd50700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9b80b3440 0x7fe9b8151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:36.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bd551700 1 -- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9b80b3440 msgr2=0x7fe9b8151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bd551700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9b80b3440 0x7fe9b8151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bd551700 1 -- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9b815a120 msgr2=0x7fe9b8157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bd551700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9b815a120 0x7fe9b8157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bd551700 1 -- 172.21.15.112:0/1532545036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b000bd40 con 0x7fe9b806d5b0 2023-08-27T11:26:36.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bd551700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b806d5b0 0x7fe9b814ce70 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400c7f0 tx=0x7fe9b400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:36.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9adffb700 1 -- 172.21.15.112:0/1532545036 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4011800 con 0x7fe9b806d5b0 2023-08-27T11:26:36.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9adffb700 1 -- 172.21.15.112:0/1532545036 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b4011960 con 0x7fe9b806d5b0 2023-08-27T11:26:36.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/1532545036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b8152060 con 0x7fe9b806d5b0 2023-08-27T11:26:36.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.601+0000 7fe9adffb700 1 -- 172.21.15.112:0/1532545036 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4011cf0 con 0x7fe9b806d5b0 2023-08-27T11:26:36.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.602+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/1532545036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b8152620 con 0x7fe9b806d5b0 2023-08-27T11:26:36.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.602+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/1532545036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b8157380 con 0x7fe9b806d5b0 2023-08-27T11:26:36.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.603+0000 7fe9adffb700 1 -- 172.21.15.112:0/1532545036 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fe9b4018360 con 0x7fe9b806d5b0 2023-08-27T11:26:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.603+0000 7fe9adffb700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe99c06a440 0x7fe99c06c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.603+0000 7fe9adffb700 1 -- 172.21.15.112:0/1532545036 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9b4088a20 con 0x7fe9b806d5b0 2023-08-27T11:26:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.604+0000 7fe9bcd50700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe99c06a440 0x7fe99c06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.604+0000 7fe9bcd50700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe99c06a440 0x7fe99c06c900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8006fd0 tx=0x7fe9a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.608+0000 7fe9adffb700 1 -- 172.21.15.112:0/1532545036 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe9b4053bb0 con 0x7fe9b806d5b0 2023-08-27T11:26:36.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.812+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/1532545036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe9b80afdc0 con 0x7fe9b806d5b0 2023-08-27T11:26:36.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.812+0000 7fe9adffb700 1 -- 172.21.15.112:0/1532545036 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v90) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe9b40571d0 con 0x7fe9b806d5b0 2023-08-27T11:26:36.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.814+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe99c06a440 msgr2=0x7fe99c06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.814+0000 7fe9bf7b5700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe99c06a440 0x7fe99c06c900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8006fd0 tx=0x7fe9a8008040 comp rx=0 tx=0).stop 2023-08-27T11:26:36.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.814+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/1532545036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b806d5b0 msgr2=0x7fe9b814ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:36.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.814+0000 7fe9bf7b5700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b806d5b0 0x7fe9b814ce70 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400c7f0 tx=0x7fe9b400cb00 comp rx=0 tx=0).stop 2023-08-27T11:26:36.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.814+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/1532545036 shutdown_connections 2023-08-27T11:26:36.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.814+0000 7fe9bf7b5700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe99c06a440 0x7fe99c06c900 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.814+0000 7fe9bf7b5700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9b815a120 0x7fe9b8157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.814+0000 7fe9bf7b5700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b806d5b0 0x7fe9b814ce70 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.815+0000 7fe9bf7b5700 1 --2- 172.21.15.112:0/1532545036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe9b80b3440 0x7fe9b8151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:36.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.815+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/1532545036 >> 172.21.15.112:0/1532545036 conn(0x7fe9b80cff80 msgr2=0x7fe9b80ce930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:36.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.815+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/1532545036 shutdown_connections 2023-08-27T11:26:36.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:36.815+0000 7fe9bf7b5700 1 -- 172.21.15.112:0/1532545036 wait complete. 2023-08-27T11:26:36.819 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:26:36.828 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:26:36.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:36.829 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:26:36.876 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:26:36.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:36.877 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:26:36.894 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:26:36.894 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:36.915 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:26:36.916 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:36.976 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:36.992 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:26:37.022 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:26:37.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.100+0000 7f7781b80700 1 Processor -- start 2023-08-27T11:26:37.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.100+0000 7f7781b80700 1 -- start start 2023-08-27T11:26:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f7781b80700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f777c06e150 0x7f777c06e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f7781b80700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777c15eed0 con 0x7f777c141420 2023-08-27T11:26:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f7781b80700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777c15f1b0 con 0x7f777c0a8960 2023-08-27T11:26:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f7781b80700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777c06eb40 con 0x7f777c06e150 2023-08-27T11:26:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f777b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f777c141420 0x7f777c0aaf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55864/0 (socket says 172.21.15.112:55864) 2023-08-27T11:26:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f777b7fe700 1 -- 172.21.15.112:0/447191814 learned_addr learned my addr 172.21.15.112:0/447191814 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f777affd700 1 --2- 172.21.15.112:0/447191814 >> v2:172.21.15.138:3300/0 conn(0x7f777c06e150 0x7f777c06e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f777a7fc700 1 -- 172.21.15.112:0/447191814 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057797179 0 0) 0x7f777c15eed0 con 0x7f777c141420 2023-08-27T11:26:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f777a7fc700 1 -- 172.21.15.112:0/447191814 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77640035b0 con 0x7f777c141420 2023-08-27T11:26:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f777a7fc700 1 -- 172.21.15.112:0/447191814 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574945562 0 0) 0x7f777c15f1b0 con 0x7f777c0a8960 2023-08-27T11:26:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f777a7fc700 1 -- 172.21.15.112:0/447191814 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f777c15eed0 con 0x7f777c0a8960 2023-08-27T11:26:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f777affd700 1 -- 172.21.15.112:0/447191814 >> v1:172.21.15.138:6789/0 conn(0x7f777c0a8960 legacy=0x7f777c0a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f777affd700 1 -- 172.21.15.112:0/447191814 >> v1:172.21.15.112:6789/0 conn(0x7f777c141420 legacy=0x7f777c0aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f777affd700 1 -- 172.21.15.112:0/447191814 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c15f490 con 0x7f777c06e150 2023-08-27T11:26:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.101+0000 7f777affd700 1 --2- 172.21.15.112:0/447191814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f777c06e150 0x7f777c06e600 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7770009820 tx=0x7f7770009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af659da2330c37ee server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.102+0000 7f777a7fc700 1 -- 172.21.15.112:0/447191814 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770005e30 con 0x7f777c06e150 2023-08-27T11:26:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.102+0000 7f777a7fc700 1 -- 172.21.15.112:0/447191814 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7770006030 con 0x7f777c06e150 2023-08-27T11:26:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.102+0000 7f777a7fc700 1 -- 172.21.15.112:0/447191814 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77700063a0 con 0x7f777c06e150 2023-08-27T11:26:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.102+0000 7f7781b80700 1 -- 172.21.15.112:0/447191814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f777c06e150 msgr2=0x7f777c06e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.102+0000 7f7781b80700 1 --2- 172.21.15.112:0/447191814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f777c06e150 0x7f777c06e600 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7770009820 tx=0x7f7770009b30 comp rx=0 tx=0).stop 2023-08-27T11:26:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.102+0000 7f7781b80700 1 -- 172.21.15.112:0/447191814 shutdown_connections 2023-08-27T11:26:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.102+0000 7f7781b80700 1 --2- 172.21.15.112:0/447191814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f777c06e150 0x7f777c06e600 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.102+0000 7f7781b80700 1 -- 172.21.15.112:0/447191814 >> 172.21.15.112:0/447191814 conn(0x7f777c06cd30 msgr2=0x7f777c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.102+0000 7f7781b80700 1 -- 172.21.15.112:0/447191814 shutdown_connections 2023-08-27T11:26:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.102+0000 7f7781b80700 1 -- 172.21.15.112:0/447191814 wait complete. 2023-08-27T11:26:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.104+0000 7f7781b80700 1 Processor -- start 2023-08-27T11:26:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.104+0000 7f7781b80700 1 -- start start 2023-08-27T11:26:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f7781b80700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777c06e150 0x7f777c13c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f7781b80700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f777c0a8960 0x7f777c141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f7781b80700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f777c141420 0x7f777c1f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f7781b80700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c13cb70 con 0x7f777c06e150 2023-08-27T11:26:37.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f7781b80700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c13ce50 con 0x7f777c141420 2023-08-27T11:26:37.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f7781b80700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777c13d130 con 0x7f777c0a8960 2023-08-27T11:26:37.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f777b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777c06e150 0x7f777c13c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f777b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777c06e150 0x7f777c13c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33256/0 (socket says 172.21.15.112:33256) 2023-08-27T11:26:37.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f777b7fe700 1 -- 172.21.15.112:0/3314021438 learned_addr learned my addr 172.21.15.112:0/3314021438 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:37.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f777bfff700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f777c141420 0x7f777c1f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f777affd700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f777c0a8960 0x7f777c141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f777b7fe700 1 -- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f777c0a8960 msgr2=0x7f777c141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f777b7fe700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f777c0a8960 0x7f777c141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f777b7fe700 1 -- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f777c141420 msgr2=0x7f777c1f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f777b7fe700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f777c141420 0x7f777c1f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f777b7fe700 1 -- 172.21.15.112:0/3314021438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7770009430 con 0x7f777c06e150 2023-08-27T11:26:37.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f777b7fe700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777c06e150 0x7f777c13c4e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f776c001eb0 tx=0x7f776c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:37.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f7763fff700 1 -- 172.21.15.112:0/3314021438 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c005d70 con 0x7f777c06e150 2023-08-27T11:26:37.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f7763fff700 1 -- 172.21.15.112:0/3314021438 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776c0070d0 con 0x7f777c06e150 2023-08-27T11:26:37.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f7781b80700 1 -- 172.21.15.112:0/3314021438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777c13d410 con 0x7f777c06e150 2023-08-27T11:26:37.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f7763fff700 1 -- 172.21.15.112:0/3314021438 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c007460 con 0x7f777c06e150 2023-08-27T11:26:37.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.105+0000 7f7781b80700 1 -- 172.21.15.112:0/3314021438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777c1f46f0 con 0x7f777c06e150 2023-08-27T11:26:37.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.106+0000 7f7781b80700 1 -- 172.21.15.112:0/3314021438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777c13d930 con 0x7f777c06e150 2023-08-27T11:26:37.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.107+0000 7f7763fff700 1 -- 172.21.15.112:0/3314021438 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f776c0075c0 con 0x7f777c06e150 2023-08-27T11:26:37.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.107+0000 7f7763fff700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f776806a440 0x7f776806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.107+0000 7f7763fff700 1 -- 172.21.15.112:0/3314021438 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f776c087f50 con 0x7f777c06e150 2023-08-27T11:26:37.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.107+0000 7f777affd700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f776806a440 0x7f776806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.108+0000 7f777affd700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f776806a440 0x7f776806c900 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7770009fd0 tx=0x7f7770008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:37.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.112+0000 7f7763fff700 1 -- 172.21.15.112:0/3314021438 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f777c13d930 con 0x7f777c06e150 2023-08-27T11:26:37.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.300+0000 7f7781b80700 1 -- 172.21.15.112:0/3314021438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f777c13d930 con 0x7f777c06e150 2023-08-27T11:26:37.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.302+0000 7f7763fff700 1 -- 172.21.15.112:0/3314021438 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+18248 (secure 0 0 0) 0x7f777c13d930 con 0x7f777c06e150 2023-08-27T11:26:37.303 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:37.304 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:20.416307+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:16.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:17.773993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:37.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.304+0000 7f7781b80700 1 -- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f776806a440 msgr2=0x7f776806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.304+0000 7f7781b80700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f776806a440 0x7f776806c900 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7770009fd0 tx=0x7f7770008070 comp rx=0 tx=0).stop 2023-08-27T11:26:37.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.304+0000 7f7781b80700 1 -- 172.21.15.112:0/3314021438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777c06e150 msgr2=0x7f777c13c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.304+0000 7f7781b80700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777c06e150 0x7f777c13c4e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f776c001eb0 tx=0x7f776c00c7b0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.304+0000 7f7781b80700 1 -- 172.21.15.112:0/3314021438 shutdown_connections 2023-08-27T11:26:37.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.304+0000 7f7781b80700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f776806a440 0x7f776806c900 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.304+0000 7f7781b80700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f777c141420 0x7f777c1f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.304+0000 7f7781b80700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777c06e150 0x7f777c13c4e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.304+0000 7f7781b80700 1 --2- 172.21.15.112:0/3314021438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f777c0a8960 0x7f777c141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.304+0000 7f7781b80700 1 -- 172.21.15.112:0/3314021438 >> 172.21.15.112:0/3314021438 conn(0x7f777c06cd30 msgr2=0x7f777c15dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:37.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.304+0000 7f7781b80700 1 -- 172.21.15.112:0/3314021438 shutdown_connections 2023-08-27T11:26:37.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.304+0000 7f7781b80700 1 -- 172.21.15.112:0/3314021438 wait complete. 2023-08-27T11:26:37.318 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:26:16.548692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:26:17.773993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:26:37.319 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:26:37.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.479+0000 7fa546cfb700 1 Processor -- start 2023-08-27T11:26:37.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.479+0000 7fa546cfb700 1 -- start start 2023-08-27T11:26:37.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa546cfb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5400aea60 0x7fa5400ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa546cfb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa54006d480 0x7fa54015bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa546cfb700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5400cb150 con 0x7fa5400aee40 2023-08-27T11:26:37.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa546cfb700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa54015b150 con 0x7fa5400aea60 2023-08-27T11:26:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa546cfb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54015b430 con 0x7fa54006d480 2023-08-27T11:26:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa53ffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa5400aee40 0x7fa54006cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55878/0 (socket says 172.21.15.112:55878) 2023-08-27T11:26:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa53ffff700 1 -- 172.21.15.112:0/103945722 learned_addr learned my addr 172.21.15.112:0/103945722 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa545298700 1 --2- 172.21.15.112:0/103945722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa54006d480 0x7fa54015bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa544a97700 1 --2- 172.21.15.112:0/103945722 >> v2:172.21.15.138:3300/0 conn(0x7fa5400aea60 0x7fa5400ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488156680 0 0) 0x7fa5400cb150 con 0x7fa5400aee40 2023-08-27T11:26:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5300046c0 con 0x7fa5400aee40 2023-08-27T11:26:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1771683014 0 0) 0x7fa5300046c0 con 0x7fa5400aee40 2023-08-27T11:26:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa54006d480 msgr2=0x7fa54015bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa53f7fe700 1 --2- 172.21.15.112:0/103945722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa54006d480 0x7fa54015bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 >> v2:172.21.15.138:3300/0 conn(0x7fa5400aea60 msgr2=0x7fa5400ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa53f7fe700 1 --2- 172.21.15.112:0/103945722 >> v2:172.21.15.138:3300/0 conn(0x7fa5400aea60 0x7fa5400ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54015b710 con 0x7fa5400aee40 2023-08-27T11:26:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.480+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fa52c001670 con 0x7fa5400aee40 2023-08-27T11:26:37.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 >> v1:172.21.15.112:6789/0 conn(0x7fa5400aee40 legacy=0x7fa54006cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa53f7fe700 1 --2- 172.21.15.112:0/103945722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa530004b10 0x7fa530004f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa53f7fe700 1 --2- 172.21.15.112:0/103945722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa530005960 0x7fa530005de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa53f7fe700 1 --2- 172.21.15.112:0/103945722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa54006d480 0x7fa530006320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54015b150 con 0x7fa530004b10 2023-08-27T11:26:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54015b430 con 0x7fa530005960 2023-08-27T11:26:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53000cb20 con 0x7fa54006d480 2023-08-27T11:26:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa544a97700 1 --2- 172.21.15.112:0/103945722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa530004b10 0x7fa530004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa545298700 1 --2- 172.21.15.112:0/103945722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa54006d480 0x7fa530006320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa53ffff700 1 --2- 172.21.15.112:0/103945722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa530005960 0x7fa530005de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa544a97700 1 -- 172.21.15.112:0/103945722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa54006d480 msgr2=0x7fa530006320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:37.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa544a97700 1 --2- 172.21.15.112:0/103945722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa54006d480 0x7fa530006320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa544a97700 1 -- 172.21.15.112:0/103945722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa530005960 msgr2=0x7fa530005de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:37.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa544a97700 1 --2- 172.21.15.112:0/103945722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa530005960 0x7fa530005de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa544a97700 1 -- 172.21.15.112:0/103945722 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5300046c0 con 0x7fa530004b10 2023-08-27T11:26:37.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa545298700 1 --2- 172.21.15.112:0/103945722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa54006d480 0x7fa530006320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:37.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa544a97700 1 --2- 172.21.15.112:0/103945722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa530004b10 0x7fa530004f30 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa534009b60 tx=0x7fa530006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec29b32d1ac19663 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:37.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53400e6c0 con 0x7fa530004b10 2023-08-27T11:26:37.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53400e820 con 0x7fa530004b10 2023-08-27T11:26:37.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.481+0000 7fa53f7fe700 1 -- 172.21.15.112:0/103945722 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53400eb90 con 0x7fa530004b10 2023-08-27T11:26:37.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.482+0000 7fa546cfb700 1 -- 172.21.15.112:0/103945722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa530004b10 msgr2=0x7fa530004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.482+0000 7fa546cfb700 1 --2- 172.21.15.112:0/103945722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa530004b10 0x7fa530004f30 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa534009b60 tx=0x7fa530006990 comp rx=0 tx=0).stop 2023-08-27T11:26:37.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.482+0000 7fa545298700 1 -- 172.21.15.112:0/103945722 reap_dead start 2023-08-27T11:26:37.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.482+0000 7fa546cfb700 1 -- 172.21.15.112:0/103945722 shutdown_connections 2023-08-27T11:26:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.482+0000 7fa546cfb700 1 -- 172.21.15.112:0/103945722 >> 172.21.15.112:0/103945722 conn(0x7fa5400cf750 msgr2=0x7fa5400cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.482+0000 7fa546cfb700 1 -- 172.21.15.112:0/103945722 shutdown_connections 2023-08-27T11:26:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.482+0000 7fa546cfb700 1 -- 172.21.15.112:0/103945722 wait complete. 2023-08-27T11:26:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.483+0000 7fa546cfb700 1 Processor -- start 2023-08-27T11:26:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.483+0000 7fa546cfb700 1 -- start start 2023-08-27T11:26:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa546cfb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa530004b10 0x7fa540070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa546cfb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa540072650 0x7fa5400af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa546cfb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5400af7d0 0x7fa5400afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa546cfb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa540153a80 con 0x7fa5400af7d0 2023-08-27T11:26:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa546cfb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa540070f20 con 0x7fa540072650 2023-08-27T11:26:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa546cfb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa540071200 con 0x7fa530004b10 2023-08-27T11:26:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa545298700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5400af7d0 0x7fa5400afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa545298700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5400af7d0 0x7fa5400afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33280/0 (socket says 172.21.15.112:33280) 2023-08-27T11:26:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa545298700 1 -- 172.21.15.112:0/539866040 learned_addr learned my addr 172.21.15.112:0/539866040 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa544a97700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa530004b10 0x7fa540070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa53ffff700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa540072650 0x7fa5400af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa545298700 1 -- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa530004b10 msgr2=0x7fa540070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa545298700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa530004b10 0x7fa540070620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa545298700 1 -- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa540072650 msgr2=0x7fa5400af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa545298700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa540072650 0x7fa5400af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa545298700 1 -- 172.21.15.112:0/539866040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5400714e0 con 0x7fa5400af7d0 2023-08-27T11:26:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa545298700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5400af7d0 0x7fa5400afc80 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa540067c50 tx=0x7fa5380114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa53d7fa700 1 -- 172.21.15.112:0/539866040 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53800e030 con 0x7fa5400af7d0 2023-08-27T11:26:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa53d7fa700 1 -- 172.21.15.112:0/539866040 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa538011b10 con 0x7fa5400af7d0 2023-08-27T11:26:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.484+0000 7fa546cfb700 1 -- 172.21.15.112:0/539866040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5400717c0 con 0x7fa5400af7d0 2023-08-27T11:26:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.485+0000 7fa53d7fa700 1 -- 172.21.15.112:0/539866040 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538007ea0 con 0x7fa5400af7d0 2023-08-27T11:26:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.485+0000 7fa546cfb700 1 -- 172.21.15.112:0/539866040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa540070b60 con 0x7fa5400af7d0 2023-08-27T11:26:37.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.485+0000 7fa546cfb700 1 -- 172.21.15.112:0/539866040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5401e32b0 con 0x7fa5400af7d0 2023-08-27T11:26:37.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.486+0000 7fa53d7fa700 1 -- 172.21.15.112:0/539866040 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fa538016470 con 0x7fa5400af7d0 2023-08-27T11:26:37.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.486+0000 7fa53d7fa700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa52406a440 0x7fa52406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.486+0000 7fa53d7fa700 1 -- 172.21.15.112:0/539866040 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa53800f070 con 0x7fa5400af7d0 2023-08-27T11:26:37.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.486+0000 7fa544a97700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa52406a440 0x7fa52406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.487+0000 7fa544a97700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa52406a440 0x7fa52406c900 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa5340090b0 tx=0x7fa53401f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:37.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.491+0000 7fa53d7fa700 1 -- 172.21.15.112:0/539866040 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa5401e32b0 con 0x7fa5400af7d0 2023-08-27T11:26:37.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.679+0000 7fa546cfb700 1 -- 172.21.15.112:0/539866040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa5401e32b0 con 0x7fa5400af7d0 2023-08-27T11:26:37.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.679+0000 7fa53d7fa700 1 -- 172.21.15.112:0/539866040 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v90) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa5401e32b0 con 0x7fa5400af7d0 2023-08-27T11:26:37.681 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:26:37.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.681+0000 7fa546cfb700 1 -- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa52406a440 msgr2=0x7fa52406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.681+0000 7fa546cfb700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa52406a440 0x7fa52406c900 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa5340090b0 tx=0x7fa53401f040 comp rx=0 tx=0).stop 2023-08-27T11:26:37.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.681+0000 7fa546cfb700 1 -- 172.21.15.112:0/539866040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5400af7d0 msgr2=0x7fa5400afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.681+0000 7fa546cfb700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5400af7d0 0x7fa5400afc80 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa540067c50 tx=0x7fa5380114b0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.681+0000 7fa546cfb700 1 -- 172.21.15.112:0/539866040 shutdown_connections 2023-08-27T11:26:37.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.681+0000 7fa546cfb700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa52406a440 0x7fa52406c900 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.681+0000 7fa546cfb700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5400af7d0 0x7fa5400afc80 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.681+0000 7fa546cfb700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa530004b10 0x7fa540070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.681+0000 7fa546cfb700 1 --2- 172.21.15.112:0/539866040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa540072650 0x7fa5400af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.681+0000 7fa546cfb700 1 -- 172.21.15.112:0/539866040 >> 172.21.15.112:0/539866040 conn(0x7fa5400cf750 msgr2=0x7fa540158320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:37.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.681+0000 7fa546cfb700 1 -- 172.21.15.112:0/539866040 shutdown_connections 2023-08-27T11:26:37.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.681+0000 7fa546cfb700 1 -- 172.21.15.112:0/539866040 wait complete. 2023-08-27T11:26:37.695 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:26:37.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.816+0000 7fb3712ed700 1 Processor -- start 2023-08-27T11:26:37.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.816+0000 7fb3712ed700 1 -- start start 2023-08-27T11:26:37.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.817+0000 7fb3712ed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb36c152130 0x7fb36c15c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.817+0000 7fb3712ed700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb36c15cd60 0x7fb36c15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.817+0000 7fb3712ed700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36c15f750 con 0x7fb36c15f240 2023-08-27T11:26:37.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.817+0000 7fb3712ed700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36c15fa30 con 0x7fb36c15cd60 2023-08-27T11:26:37.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.817+0000 7fb3712ed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c0b60a0 con 0x7fb36c152130 2023-08-27T11:26:37.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.817+0000 7fb36affd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb36c152130 0x7fb36c15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.817+0000 7fb36a7fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb36c15cd60 0x7fb36c15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.817+0000 7fb36affd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb36c152130 0x7fb36c15c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:44938/0 (socket says 172.21.15.112:44938) 2023-08-27T11:26:37.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.817+0000 7fb36a7fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb36c15cd60 0x7fb36c15ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:33710/0 (socket says 172.21.15.112:33710) 2023-08-27T11:26:37.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.817+0000 7fb36affd700 1 -- 172.21.15.112:0/110610582 learned_addr learned my addr 172.21.15.112:0/110610582 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:37.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.817+0000 7fb36b7fe700 1 --1- 172.21.15.112:0/110610582 >> v1:172.21.15.138:6789/0 conn(0x7fb36c15f240 0x7fb36c0b3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:35194/0 (socket says 172.21.15.112:35194) 2023-08-27T11:26:37.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.818+0000 7fb36a7fc700 1 -- 172.21.15.112:0/110610582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb36c152130 msgr2=0x7fb36c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:37.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.818+0000 7fb36a7fc700 1 --2- 172.21.15.112:0/110610582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb36c152130 0x7fb36c15c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.818+0000 7fb36a7fc700 1 -- 172.21.15.112:0/110610582 >> v1:172.21.15.138:6789/0 conn(0x7fb36c15f240 legacy=0x7fb36c0b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.818+0000 7fb36a7fc700 1 -- 172.21.15.112:0/110610582 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c0b6380 con 0x7fb36c15cd60 2023-08-27T11:26:37.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.818+0000 7fb36affd700 1 --2- 172.21.15.112:0/110610582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb36c152130 0x7fb36c15c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:37.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.818+0000 7fb36a7fc700 1 --2- 172.21.15.112:0/110610582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb36c15cd60 0x7fb36c15ed00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009840 tx=0x7fb35c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=187da8aa61a25b8f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:37.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.818+0000 7fb369ffb700 1 -- 172.21.15.112:0/110610582 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c011830 con 0x7fb36c15cd60 2023-08-27T11:26:37.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.818+0000 7fb369ffb700 1 -- 172.21.15.112:0/110610582 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35c011990 con 0x7fb36c15cd60 2023-08-27T11:26:37.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.818+0000 7fb369ffb700 1 -- 172.21.15.112:0/110610582 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c011d20 con 0x7fb36c15cd60 2023-08-27T11:26:37.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.819+0000 7fb3712ed700 1 -- 172.21.15.112:0/110610582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb36c15cd60 msgr2=0x7fb36c15ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.819+0000 7fb3712ed700 1 --2- 172.21.15.112:0/110610582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb36c15cd60 0x7fb36c15ed00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009840 tx=0x7fb35c009b50 comp rx=0 tx=0).stop 2023-08-27T11:26:37.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.819+0000 7fb3712ed700 1 -- 172.21.15.112:0/110610582 shutdown_connections 2023-08-27T11:26:37.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.819+0000 7fb3712ed700 1 --2- 172.21.15.112:0/110610582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb36c152130 0x7fb36c15c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.819+0000 7fb3712ed700 1 --2- 172.21.15.112:0/110610582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb36c15cd60 0x7fb36c15ed00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.819+0000 7fb3712ed700 1 -- 172.21.15.112:0/110610582 >> 172.21.15.112:0/110610582 conn(0x7fb36c066010 msgr2=0x7fb36c0663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:37.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.819+0000 7fb3712ed700 1 -- 172.21.15.112:0/110610582 shutdown_connections 2023-08-27T11:26:37.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.819+0000 7fb3712ed700 1 -- 172.21.15.112:0/110610582 wait complete. 2023-08-27T11:26:37.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.820+0000 7fb3712ed700 1 Processor -- start 2023-08-27T11:26:37.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.820+0000 7fb3712ed700 1 -- start start 2023-08-27T11:26:37.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb3712ed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb36c152130 0x7fb36c14b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb3712ed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb36c15cd60 0x7fb36c1f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb3712ed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb36c15f240 0x7fb36c1f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb3712ed700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c070ed0 con 0x7fb36c15f240 2023-08-27T11:26:37.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb3712ed700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c0708a0 con 0x7fb36c15cd60 2023-08-27T11:26:37.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb3712ed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c1f2f90 con 0x7fb36c152130 2023-08-27T11:26:37.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb36b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb36c15f240 0x7fb36c1f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb36b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb36c15f240 0x7fb36c1f2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33294/0 (socket says 172.21.15.112:33294) 2023-08-27T11:26:37.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb36b7fe700 1 -- 172.21.15.112:0/3543009240 learned_addr learned my addr 172.21.15.112:0/3543009240 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:37.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb36affd700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb36c152130 0x7fb36c14b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb36a7fc700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb36c15cd60 0x7fb36c1f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb36b7fe700 1 -- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb36c152130 msgr2=0x7fb36c14b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb36b7fe700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb36c152130 0x7fb36c14b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb36b7fe700 1 -- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb36c15cd60 msgr2=0x7fb36c1f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:37.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb36b7fe700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb36c15cd60 0x7fb36c1f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:37.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb36b7fe700 1 -- 172.21.15.112:0/3543009240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb360009550 con 0x7fb36c15f240 2023-08-27T11:26:37.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.821+0000 7fb36affd700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb36c152130 0x7fb36c14b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:37.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.822+0000 7fb36b7fe700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb36c15f240 0x7fb36c1f2940 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb36400cbc0 tx=0x7fb36400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:37.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.822+0000 7fb3537fe700 1 -- 172.21.15.112:0/3543009240 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36400aaf0 con 0x7fb36c15f240 2023-08-27T11:26:37.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.822+0000 7fb3537fe700 1 -- 172.21.15.112:0/3543009240 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36400ac50 con 0x7fb36c15f240 2023-08-27T11:26:37.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.822+0000 7fb3712ed700 1 -- 172.21.15.112:0/3543009240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35c0094a0 con 0x7fb36c15f240 2023-08-27T11:26:37.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.822+0000 7fb3537fe700 1 -- 172.21.15.112:0/3543009240 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb364007070 con 0x7fb36c15f240 2023-08-27T11:26:37.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.822+0000 7fb3712ed700 1 -- 172.21.15.112:0/3543009240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36c070b60 con 0x7fb36c15f240 2023-08-27T11:26:37.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.823+0000 7fb3712ed700 1 -- 172.21.15.112:0/3543009240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36c071040 con 0x7fb36c15f240 2023-08-27T11:26:37.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.823+0000 7fb3537fe700 1 -- 172.21.15.112:0/3543009240 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fb3640071d0 con 0x7fb36c15f240 2023-08-27T11:26:37.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.823+0000 7fb3537fe700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb35806a3b0 0x7fb35806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:37.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.824+0000 7fb3537fe700 1 -- 172.21.15.112:0/3543009240 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb364088740 con 0x7fb36c15f240 2023-08-27T11:26:37.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.824+0000 7fb36affd700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb35806a3b0 0x7fb35806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:37.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.824+0000 7fb36affd700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb35806a3b0 0x7fb35806c870 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb360009db0 tx=0x7fb360008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:37.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:37.828+0000 7fb3537fe700 1 -- 172.21.15.112:0/3543009240 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb364053ac0 con 0x7fb36c15f240 2023-08-27T11:26:38.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.015+0000 7fb3712ed700 1 -- 172.21.15.112:0/3543009240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb36c067a20 con 0x7fb36c15f240 2023-08-27T11:26:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.015+0000 7fb3537fe700 1 -- 172.21.15.112:0/3543009240 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3640570e0 con 0x7fb36c15f240 2023-08-27T11:26:38.018 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:26:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.017+0000 7fb3712ed700 1 -- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb35806a3b0 msgr2=0x7fb35806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.017+0000 7fb3712ed700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb35806a3b0 0x7fb35806c870 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb360009db0 tx=0x7fb360008040 comp rx=0 tx=0).stop 2023-08-27T11:26:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.017+0000 7fb3712ed700 1 -- 172.21.15.112:0/3543009240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb36c15f240 msgr2=0x7fb36c1f2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.017+0000 7fb3712ed700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb36c15f240 0x7fb36c1f2940 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb36400cbc0 tx=0x7fb36400cf80 comp rx=0 tx=0).stop 2023-08-27T11:26:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.017+0000 7fb3712ed700 1 -- 172.21.15.112:0/3543009240 shutdown_connections 2023-08-27T11:26:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.017+0000 7fb3712ed700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb35806a3b0 0x7fb35806c870 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.017+0000 7fb3712ed700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb36c15f240 0x7fb36c1f2940 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.017+0000 7fb3712ed700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb36c152130 0x7fb36c14b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.017+0000 7fb3712ed700 1 --2- 172.21.15.112:0/3543009240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb36c15cd60 0x7fb36c1f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.017+0000 7fb3712ed700 1 -- 172.21.15.112:0/3543009240 >> 172.21.15.112:0/3543009240 conn(0x7fb36c066010 msgr2=0x7fb36c0b4620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.017+0000 7fb3712ed700 1 -- 172.21.15.112:0/3543009240 shutdown_connections 2023-08-27T11:26:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.017+0000 7fb3712ed700 1 -- 172.21.15.112:0/3543009240 wait complete. 2023-08-27T11:26:38.030 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:26:38.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.152+0000 7f0f62dcf700 1 Processor -- start 2023-08-27T11:26:38.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.152+0000 7f0f62dcf700 1 -- start start 2023-08-27T11:26:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.153+0000 7f0f62dcf700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0f5c06dcc0 0x7f0f5c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.153+0000 7f0f62dcf700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c0cfd50 con 0x7f0f5c1437c0 2023-08-27T11:26:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.153+0000 7f0f62dcf700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c0cf140 con 0x7f0f5c143ba0 2023-08-27T11:26:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.153+0000 7f0f62dcf700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c0cf420 con 0x7f0f5c06dcc0 2023-08-27T11:26:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.153+0000 7f0f60b6b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0f5c1437c0 0x7f0f5c0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55912/0 (socket says 172.21.15.112:55912) 2023-08-27T11:26:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.153+0000 7f0f60b6b700 1 -- 172.21.15.112:0/1492361017 learned_addr learned my addr 172.21.15.112:0/1492361017 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.153+0000 7f0f6136c700 1 --2- 172.21.15.112:0/1492361017 >> v2:172.21.15.138:3300/0 conn(0x7f0f5c06dcc0 0x7f0f5c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.153+0000 7f0f5b7fe700 1 -- 172.21.15.112:0/1492361017 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613170982 0 0) 0x7f0f5c0cfd50 con 0x7f0f5c1437c0 2023-08-27T11:26:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.153+0000 7f0f5b7fe700 1 -- 172.21.15.112:0/1492361017 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f4c0035b0 con 0x7f0f5c1437c0 2023-08-27T11:26:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.153+0000 7f0f6136c700 1 -- 172.21.15.112:0/1492361017 >> v1:172.21.15.138:6789/0 conn(0x7f0f5c143ba0 legacy=0x7f0f5c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.153+0000 7f0f6136c700 1 -- 172.21.15.112:0/1492361017 >> v1:172.21.15.112:6789/0 conn(0x7f0f5c1437c0 legacy=0x7f0f5c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.154+0000 7f0f5b7fe700 1 -- 172.21.15.112:0/1492361017 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 856021442 0 0) 0x7f0f4c0035b0 con 0x7f0f5c1437c0 2023-08-27T11:26:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.154+0000 7f0f6136c700 1 -- 172.21.15.112:0/1492361017 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c0cf700 con 0x7f0f5c06dcc0 2023-08-27T11:26:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.154+0000 7f0f6136c700 1 --2- 172.21.15.112:0/1492361017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c06dcc0 0x7f0f5c0b4670 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009d90 tx=0x7f0f5400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeea0e9fd527e174 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.154+0000 7f0f5b7fe700 1 -- 172.21.15.112:0/1492361017 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400e070 con 0x7f0f5c06dcc0 2023-08-27T11:26:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.154+0000 7f0f5b7fe700 1 -- 172.21.15.112:0/1492361017 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f54003a20 con 0x7f0f5c06dcc0 2023-08-27T11:26:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.154+0000 7f0f5b7fe700 1 -- 172.21.15.112:0/1492361017 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50001670 con 0x7f0f5c06dcc0 2023-08-27T11:26:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.154+0000 7f0f62dcf700 1 -- 172.21.15.112:0/1492361017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c06dcc0 msgr2=0x7f0f5c0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.154+0000 7f0f62dcf700 1 --2- 172.21.15.112:0/1492361017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c06dcc0 0x7f0f5c0b4670 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009d90 tx=0x7f0f5400b330 comp rx=0 tx=0).stop 2023-08-27T11:26:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.154+0000 7f0f62dcf700 1 -- 172.21.15.112:0/1492361017 shutdown_connections 2023-08-27T11:26:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.154+0000 7f0f62dcf700 1 --2- 172.21.15.112:0/1492361017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c06dcc0 0x7f0f5c0b4670 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.154+0000 7f0f62dcf700 1 -- 172.21.15.112:0/1492361017 >> 172.21.15.112:0/1492361017 conn(0x7f0f5c0b1e90 msgr2=0x7f0f5c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.154+0000 7f0f62dcf700 1 -- 172.21.15.112:0/1492361017 shutdown_connections 2023-08-27T11:26:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.155+0000 7f0f62dcf700 1 -- 172.21.15.112:0/1492361017 wait complete. 2023-08-27T11:26:38.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.156+0000 7f0f62dcf700 1 Processor -- start 2023-08-27T11:26:38.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.156+0000 7f0f62dcf700 1 -- start start 2023-08-27T11:26:38.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f62dcf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f5c06dcc0 0x7f0f5c158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f62dcf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f5c1437c0 0x7f0f5c139970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f62dcf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c143ba0 0x7f0f5c13f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f62dcf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c13a2c0 con 0x7f0f5c1437c0 2023-08-27T11:26:38.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f62dcf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c13a5a0 con 0x7f0f5c143ba0 2023-08-27T11:26:38.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f62dcf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c13a880 con 0x7f0f5c06dcc0 2023-08-27T11:26:38.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f5bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f5c1437c0 0x7f0f5c139970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f5bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f5c1437c0 0x7f0f5c139970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33306/0 (socket says 172.21.15.112:33306) 2023-08-27T11:26:38.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f5bfff700 1 -- 172.21.15.112:0/3849053501 learned_addr learned my addr 172.21.15.112:0/3849053501 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:38.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f60b6b700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f5c06dcc0 0x7f0f5c158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f6136c700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c143ba0 0x7f0f5c13f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f5bfff700 1 -- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f5c06dcc0 msgr2=0x7f0f5c158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f5bfff700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f5c06dcc0 0x7f0f5c158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f5bfff700 1 -- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c143ba0 msgr2=0x7f0f5c13f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f5bfff700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c143ba0 0x7f0f5c13f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f5bfff700 1 -- 172.21.15.112:0/3849053501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f54009a70 con 0x7f0f5c1437c0 2023-08-27T11:26:38.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f5bfff700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f5c1437c0 0x7f0f5c139970 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e940 tx=0x7f0f4800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:38.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f597fa700 1 -- 172.21.15.112:0/3849053501 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480052d0 con 0x7f0f5c1437c0 2023-08-27T11:26:38.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f597fa700 1 -- 172.21.15.112:0/3849053501 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f48005430 con 0x7f0f5c1437c0 2023-08-27T11:26:38.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f62dcf700 1 -- 172.21.15.112:0/3849053501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c13ab60 con 0x7f0f5c1437c0 2023-08-27T11:26:38.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f597fa700 1 -- 172.21.15.112:0/3849053501 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480108e0 con 0x7f0f5c1437c0 2023-08-27T11:26:38.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.157+0000 7f0f62dcf700 1 -- 172.21.15.112:0/3849053501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c159330 con 0x7f0f5c1437c0 2023-08-27T11:26:38.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.158+0000 7f0f62dcf700 1 -- 172.21.15.112:0/3849053501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c0b0ed0 con 0x7f0f5c1437c0 2023-08-27T11:26:38.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.159+0000 7f0f597fa700 1 -- 172.21.15.112:0/3849053501 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f0f48010a40 con 0x7f0f5c1437c0 2023-08-27T11:26:38.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.159+0000 7f0f597fa700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0f4006a440 0x7f0f4006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.159+0000 7f0f597fa700 1 -- 172.21.15.112:0/3849053501 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f4808aad0 con 0x7f0f5c1437c0 2023-08-27T11:26:38.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.159+0000 7f0f60b6b700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0f4006a440 0x7f0f4006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.160+0000 7f0f60b6b700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0f4006a440 0x7f0f4006c900 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f50001eb0 tx=0x7f0f50001e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:38.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.164+0000 7f0f597fa700 1 -- 172.21.15.112:0/3849053501 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0f48055c30 con 0x7f0f5c1437c0 2023-08-27T11:26:38.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.353+0000 7f0f62dcf700 1 -- 172.21.15.112:0/3849053501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0f5c13b080 con 0x7f0f5c1437c0 2023-08-27T11:26:38.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.353+0000 7f0f597fa700 1 -- 172.21.15.112:0/3849053501 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0f4801f020 con 0x7f0f5c1437c0 2023-08-27T11:26:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:26:38.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.356+0000 7f0f62dcf700 1 -- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0f4006a440 msgr2=0x7f0f4006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.356+0000 7f0f62dcf700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0f4006a440 0x7f0f4006c900 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f50001eb0 tx=0x7f0f50001e40 comp rx=0 tx=0).stop 2023-08-27T11:26:38.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.356+0000 7f0f62dcf700 1 -- 172.21.15.112:0/3849053501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f5c1437c0 msgr2=0x7f0f5c139970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.356+0000 7f0f62dcf700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f5c1437c0 0x7f0f5c139970 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e940 tx=0x7f0f4800ec50 comp rx=0 tx=0).stop 2023-08-27T11:26:38.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.356+0000 7f0f62dcf700 1 -- 172.21.15.112:0/3849053501 shutdown_connections 2023-08-27T11:26:38.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.356+0000 7f0f62dcf700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0f4006a440 0x7f0f4006c900 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.356+0000 7f0f62dcf700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c143ba0 0x7f0f5c13f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.356+0000 7f0f62dcf700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f5c06dcc0 0x7f0f5c158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.356+0000 7f0f62dcf700 1 --2- 172.21.15.112:0/3849053501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f5c1437c0 0x7f0f5c139970 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.356+0000 7f0f62dcf700 1 -- 172.21.15.112:0/3849053501 >> 172.21.15.112:0/3849053501 conn(0x7f0f5c0b1e90 msgr2=0x7f0f5c1419f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:38.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.356+0000 7f0f62dcf700 1 -- 172.21.15.112:0/3849053501 shutdown_connections 2023-08-27T11:26:38.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.356+0000 7f0f62dcf700 1 -- 172.21.15.112:0/3849053501 wait complete. 2023-08-27T11:26:38.369 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:26:38.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.491+0000 7f6be726c700 1 Processor -- start 2023-08-27T11:26:38.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.492+0000 7f6be726c700 1 -- start start 2023-08-27T11:26:38.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.492+0000 7f6be726c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6be0146c20 0x7f6be0160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.492+0000 7f6be726c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be00b5070 con 0x7f6be006f790 2023-08-27T11:26:38.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.492+0000 7f6be726c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be0147000 con 0x7f6be00b3440 2023-08-27T11:26:38.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.492+0000 7f6be726c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be01614a0 con 0x7f6be0146c20 2023-08-27T11:26:38.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.492+0000 7f6be5809700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6be006f790 0x7f6be00b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55930/0 (socket says 172.21.15.112:55930) 2023-08-27T11:26:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.492+0000 7f6be5809700 1 -- 172.21.15.112:0/1930018544 learned_addr learned my addr 172.21.15.112:0/1930018544 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.493+0000 7f6be5008700 1 --2- 172.21.15.112:0/1930018544 >> v2:172.21.15.138:3300/0 conn(0x7f6be0146c20 0x7f6be0160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.493+0000 7f6be5008700 1 -- 172.21.15.112:0/1930018544 >> v1:172.21.15.138:6789/0 conn(0x7f6be00b3440 legacy=0x7f6be015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.493+0000 7f6be5008700 1 -- 172.21.15.112:0/1930018544 >> v1:172.21.15.112:6789/0 conn(0x7f6be006f790 legacy=0x7f6be00b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.493+0000 7f6be5008700 1 -- 172.21.15.112:0/1930018544 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be0161740 con 0x7f6be0146c20 2023-08-27T11:26:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.493+0000 7f6be5008700 1 --2- 172.21.15.112:0/1930018544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be0146c20 0x7f6be0160f60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009840 tx=0x7f6bdc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a078a0531ceb82c9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.495+0000 7f6bd7fff700 1 -- 172.21.15.112:0/1930018544 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc011830 con 0x7f6be0146c20 2023-08-27T11:26:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.495+0000 7f6bd7fff700 1 -- 172.21.15.112:0/1930018544 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc011990 con 0x7f6be0146c20 2023-08-27T11:26:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.495+0000 7f6bd7fff700 1 -- 172.21.15.112:0/1930018544 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc011d20 con 0x7f6be0146c20 2023-08-27T11:26:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.496+0000 7f6be726c700 1 -- 172.21.15.112:0/1930018544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be0146c20 msgr2=0x7f6be0160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.496+0000 7f6be726c700 1 --2- 172.21.15.112:0/1930018544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be0146c20 0x7f6be0160f60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009840 tx=0x7f6bdc009b50 comp rx=0 tx=0).stop 2023-08-27T11:26:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.496+0000 7f6be726c700 1 -- 172.21.15.112:0/1930018544 shutdown_connections 2023-08-27T11:26:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.496+0000 7f6be726c700 1 --2- 172.21.15.112:0/1930018544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be0146c20 0x7f6be0160f60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.496+0000 7f6be726c700 1 -- 172.21.15.112:0/1930018544 >> 172.21.15.112:0/1930018544 conn(0x7f6be00b1e90 msgr2=0x7f6be00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.496+0000 7f6be726c700 1 -- 172.21.15.112:0/1930018544 shutdown_connections 2023-08-27T11:26:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.496+0000 7f6be726c700 1 -- 172.21.15.112:0/1930018544 wait complete. 2023-08-27T11:26:38.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.497+0000 7f6be726c700 1 Processor -- start 2023-08-27T11:26:38.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.497+0000 7f6be726c700 1 -- start start 2023-08-27T11:26:38.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be726c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be006f790 0x7f6be01f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be726c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be00b3440 0x7f6be00c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be726c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be0146c20 0x7f6be00c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be726c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00c8d80 con 0x7f6be0146c20 2023-08-27T11:26:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be726c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be01f2810 con 0x7f6be00b3440 2023-08-27T11:26:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be726c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be013a9f0 con 0x7f6be006f790 2023-08-27T11:26:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be5809700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be0146c20 0x7f6be00c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be5809700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be0146c20 0x7f6be00c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33320/0 (socket says 172.21.15.112:33320) 2023-08-27T11:26:38.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be5809700 1 -- 172.21.15.112:0/436013064 learned_addr learned my addr 172.21.15.112:0/436013064 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:38.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be5008700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be006f790 0x7f6be01f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be4807700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be00b3440 0x7f6be00c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be5809700 1 -- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be006f790 msgr2=0x7f6be01f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be5809700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be006f790 0x7f6be01f22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be5809700 1 -- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be00b3440 msgr2=0x7f6be00c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be5809700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be00b3440 0x7f6be00c6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.498+0000 7f6be5809700 1 -- 172.21.15.112:0/436013064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc0094a0 con 0x7f6be0146c20 2023-08-27T11:26:38.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.499+0000 7f6be5809700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be0146c20 0x7f6be00c7090 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800ca80 tx=0x7f6bd800cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:38.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.500+0000 7f6bd5ffb700 1 -- 172.21.15.112:0/436013064 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800a9e0 con 0x7f6be0146c20 2023-08-27T11:26:38.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.500+0000 7f6bd5ffb700 1 -- 172.21.15.112:0/436013064 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd800ab40 con 0x7f6be0146c20 2023-08-27T11:26:38.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.500+0000 7f6be726c700 1 -- 172.21.15.112:0/436013064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be013aee0 con 0x7f6be0146c20 2023-08-27T11:26:38.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.500+0000 7f6bd5ffb700 1 -- 172.21.15.112:0/436013064 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8012400 con 0x7f6be0146c20 2023-08-27T11:26:38.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.500+0000 7f6be726c700 1 -- 172.21.15.112:0/436013064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be00c79f0 con 0x7f6be0146c20 2023-08-27T11:26:38.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.501+0000 7f6be726c700 1 -- 172.21.15.112:0/436013064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be0067e90 con 0x7f6be0146c20 2023-08-27T11:26:38.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.501+0000 7f6bd5ffb700 1 -- 172.21.15.112:0/436013064 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f6bd801b3b0 con 0x7f6be0146c20 2023-08-27T11:26:38.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.502+0000 7f6bd5ffb700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6bc406a440 0x7f6bc406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.502+0000 7f6bd5ffb700 1 -- 172.21.15.112:0/436013064 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6bd80883f0 con 0x7f6be0146c20 2023-08-27T11:26:38.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.502+0000 7f6be5008700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6bc406a440 0x7f6bc406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.502+0000 7f6be5008700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6bc406a440 0x7f6bc406c900 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0090b0 tx=0x7f6bdc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:38.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.506+0000 7f6bd5ffb700 1 -- 172.21.15.112:0/436013064 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6bd8053710 con 0x7f6be0146c20 2023-08-27T11:26:38.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.694+0000 7f6be726c700 1 -- 172.21.15.112:0/436013064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6be00b5070 con 0x7f6be0146c20 2023-08-27T11:26:38.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.694+0000 7f6bd5ffb700 1 -- 172.21.15.112:0/436013064 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6be00b5070 con 0x7f6be0146c20 2023-08-27T11:26:38.696 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:26:38.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.696+0000 7f6be726c700 1 -- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6bc406a440 msgr2=0x7f6bc406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.696+0000 7f6be726c700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6bc406a440 0x7f6bc406c900 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0090b0 tx=0x7f6bdc008030 comp rx=0 tx=0).stop 2023-08-27T11:26:38.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.696+0000 7f6be726c700 1 -- 172.21.15.112:0/436013064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be0146c20 msgr2=0x7f6be00c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.696+0000 7f6be726c700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be0146c20 0x7f6be00c7090 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800ca80 tx=0x7f6bd800cd90 comp rx=0 tx=0).stop 2023-08-27T11:26:38.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.696+0000 7f6be726c700 1 -- 172.21.15.112:0/436013064 shutdown_connections 2023-08-27T11:26:38.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.696+0000 7f6be726c700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6bc406a440 0x7f6bc406c900 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.696+0000 7f6be726c700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be0146c20 0x7f6be00c7090 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.696+0000 7f6be726c700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6be006f790 0x7f6be01f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.696+0000 7f6be726c700 1 --2- 172.21.15.112:0/436013064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be00b3440 0x7f6be00c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.696+0000 7f6be726c700 1 -- 172.21.15.112:0/436013064 >> 172.21.15.112:0/436013064 conn(0x7f6be00b1e90 msgr2=0x7f6be00b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:38.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.696+0000 7f6be726c700 1 -- 172.21.15.112:0/436013064 shutdown_connections 2023-08-27T11:26:38.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.696+0000 7f6be726c700 1 -- 172.21.15.112:0/436013064 wait complete. 2023-08-27T11:26:38.710 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:26:38.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.828+0000 7fc67d643700 1 Processor -- start 2023-08-27T11:26:38.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.828+0000 7fc67d643700 1 -- start start 2023-08-27T11:26:38.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.829+0000 7fc67d643700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc6780b1400 0x7fc6780cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.829+0000 7fc67d643700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6780b17e0 0x7fc67806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.829+0000 7fc67d643700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc678161850 con 0x7fc678161ce0 2023-08-27T11:26:38.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.829+0000 7fc67d643700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc67806e7d0 con 0x7fc6780b1400 2023-08-27T11:26:38.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.829+0000 7fc67d643700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780cfc10 con 0x7fc6780b17e0 2023-08-27T11:26:38.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.829+0000 7fc6777fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc678161ce0 0x7fc67806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55942/0 (socket says 172.21.15.112:55942) 2023-08-27T11:26:38.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.829+0000 7fc6777fe700 1 -- 172.21.15.112:0/169412644 learned_addr learned my addr 172.21.15.112:0/169412644 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:38.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819924280 0 0) 0x7fc678161850 con 0x7fc678161ce0 2023-08-27T11:26:38.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6600035b0 con 0x7fc678161ce0 2023-08-27T11:26:38.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc6767fc700 1 --2- 172.21.15.112:0/169412644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6780b17e0 0x7fc67806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc676ffd700 1 --2- 172.21.15.112:0/169412644 >> v2:172.21.15.138:3300/0 conn(0x7fc6780b1400 0x7fc6780cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 303113386 0 0) 0x7fc6600035b0 con 0x7fc678161ce0 2023-08-27T11:26:38.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6780b17e0 msgr2=0x7fc67806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:38.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 --2- 172.21.15.112:0/169412644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6780b17e0 0x7fc67806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 >> v2:172.21.15.138:3300/0 conn(0x7fc6780b1400 msgr2=0x7fc6780cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:38.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 --2- 172.21.15.112:0/169412644 >> v2:172.21.15.138:3300/0 conn(0x7fc6780b1400 0x7fc6780cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc678161200 con 0x7fc678161ce0 2023-08-27T11:26:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fc6700019f0 con 0x7fc678161ce0 2023-08-27T11:26:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 >> v1:172.21.15.112:6789/0 conn(0x7fc678161ce0 legacy=0x7fc67806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 --2- 172.21.15.112:0/169412644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc660003a00 0x7fc660003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 --2- 172.21.15.112:0/169412644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc660004850 0x7fc660004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 --2- 172.21.15.112:0/169412644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6780b17e0 0x7fc6600052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67806e7d0 con 0x7fc660004850 2023-08-27T11:26:38.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780cfc10 con 0x7fc660003a00 2023-08-27T11:26:38.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66000bbf0 con 0x7fc6780b17e0 2023-08-27T11:26:38.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc6767fc700 1 --2- 172.21.15.112:0/169412644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc660004850 0x7fc660004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc676ffd700 1 --2- 172.21.15.112:0/169412644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc660003a00 0x7fc660003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc6777fe700 1 --2- 172.21.15.112:0/169412644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6780b17e0 0x7fc6600052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc6767fc700 1 -- 172.21.15.112:0/169412644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6780b17e0 msgr2=0x7fc6600052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:38.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc6767fc700 1 --2- 172.21.15.112:0/169412644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6780b17e0 0x7fc6600052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc6767fc700 1 -- 172.21.15.112:0/169412644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc660003a00 msgr2=0x7fc660003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:38.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc6767fc700 1 --2- 172.21.15.112:0/169412644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc660003a00 0x7fc660003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc6767fc700 1 -- 172.21.15.112:0/169412644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6600035b0 con 0x7fc660004850 2023-08-27T11:26:38.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.830+0000 7fc6767fc700 1 --2- 172.21.15.112:0/169412644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc660004850 0x7fc660004cd0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc668009890 tx=0x7fc660005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3b1452b87b1c20 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:38.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.831+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66800c070 con 0x7fc660004850 2023-08-27T11:26:38.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.831+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc668013610 con 0x7fc660004850 2023-08-27T11:26:38.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.831+0000 7fc675ffb700 1 -- 172.21.15.112:0/169412644 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6680139a0 con 0x7fc660004850 2023-08-27T11:26:38.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.831+0000 7fc67d643700 1 -- 172.21.15.112:0/169412644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc660004850 msgr2=0x7fc660004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.831+0000 7fc67d643700 1 --2- 172.21.15.112:0/169412644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc660004850 0x7fc660004cd0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc668009890 tx=0x7fc660005880 comp rx=0 tx=0).stop 2023-08-27T11:26:38.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.831+0000 7fc6777fe700 1 -- 172.21.15.112:0/169412644 reap_dead start 2023-08-27T11:26:38.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.831+0000 7fc67d643700 1 -- 172.21.15.112:0/169412644 shutdown_connections 2023-08-27T11:26:38.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.831+0000 7fc67d643700 1 -- 172.21.15.112:0/169412644 >> 172.21.15.112:0/169412644 conn(0x7fc6780d0000 msgr2=0x7fc6780d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:38.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.831+0000 7fc67d643700 1 -- 172.21.15.112:0/169412644 shutdown_connections 2023-08-27T11:26:38.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.831+0000 7fc67d643700 1 -- 172.21.15.112:0/169412644 wait complete. 2023-08-27T11:26:38.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.833+0000 7fc67d643700 1 Processor -- start 2023-08-27T11:26:38.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.833+0000 7fc67d643700 1 -- start start 2023-08-27T11:26:38.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc67d643700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc660004850 0x7fc6781f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc67d643700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6780b02e0 0x7fc6780c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc67d643700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6780c98a0 0x7fc67815dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc67d643700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc678150d50 con 0x7fc6780b02e0 2023-08-27T11:26:38.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc67d643700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780ca410 con 0x7fc660004850 2023-08-27T11:26:38.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc67d643700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6781f7350 con 0x7fc6780c98a0 2023-08-27T11:26:38.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc6767fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6780b02e0 0x7fc6780c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc6767fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6780b02e0 0x7fc6780c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33338/0 (socket says 172.21.15.112:33338) 2023-08-27T11:26:38.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc6767fc700 1 -- 172.21.15.112:0/1450637784 learned_addr learned my addr 172.21.15.112:0/1450637784 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:38.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc676ffd700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc660004850 0x7fc6781f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc6777fe700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6780c98a0 0x7fc67815dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc6767fc700 1 -- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6780c98a0 msgr2=0x7fc67815dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc6767fc700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6780c98a0 0x7fc67815dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc6767fc700 1 -- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc660004850 msgr2=0x7fc6781f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:38.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc6767fc700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc660004850 0x7fc6781f6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:38.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc6767fc700 1 -- 172.21.15.112:0/1450637784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6780c9e50 con 0x7fc6780b02e0 2023-08-27T11:26:38.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc6767fc700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6780b02e0 0x7fc6780c9360 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc660005800 tx=0x7fc6680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:38.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc65f7fe700 1 -- 172.21.15.112:0/1450637784 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668013b90 con 0x7fc6780b02e0 2023-08-27T11:26:38.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc65f7fe700 1 -- 172.21.15.112:0/1450637784 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc668013d10 con 0x7fc6780b02e0 2023-08-27T11:26:38.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc67d643700 1 -- 172.21.15.112:0/1450637784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6780ca130 con 0x7fc6780b02e0 2023-08-27T11:26:38.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.834+0000 7fc65f7fe700 1 -- 172.21.15.112:0/1450637784 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6680038e0 con 0x7fc6780b02e0 2023-08-27T11:26:38.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.835+0000 7fc67d643700 1 -- 172.21.15.112:0/1450637784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6781f68f0 con 0x7fc6780b02e0 2023-08-27T11:26:38.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.836+0000 7fc67d643700 1 -- 172.21.15.112:0/1450637784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6781f6c60 con 0x7fc6780b02e0 2023-08-27T11:26:38.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.836+0000 7fc65f7fe700 1 -- 172.21.15.112:0/1450637784 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fc668003a40 con 0x7fc6780b02e0 2023-08-27T11:26:38.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.836+0000 7fc65f7fe700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc66406a440 0x7fc66406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:38.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.837+0000 7fc65f7fe700 1 -- 172.21.15.112:0/1450637784 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc668089710 con 0x7fc6780b02e0 2023-08-27T11:26:38.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.837+0000 7fc676ffd700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc66406a440 0x7fc66406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:38.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.837+0000 7fc676ffd700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc66406a440 0x7fc66406c900 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc6780c9150 tx=0x7fc66c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:38.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:38.841+0000 7fc65f7fe700 1 -- 172.21.15.112:0/1450637784 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc668054930 con 0x7fc6780b02e0 2023-08-27T11:26:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.044+0000 7fc67d643700 1 -- 172.21.15.112:0/1450637784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc6781f6470 con 0x7fc6780b02e0 2023-08-27T11:26:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.044+0000 7fc65f7fe700 1 -- 172.21.15.112:0/1450637784 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v90) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc6781f6470 con 0x7fc6780b02e0 2023-08-27T11:26:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.046+0000 7fc67d643700 1 -- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc66406a440 msgr2=0x7fc66406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.046+0000 7fc67d643700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc66406a440 0x7fc66406c900 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc6780c9150 tx=0x7fc66c008040 comp rx=0 tx=0).stop 2023-08-27T11:26:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.046+0000 7fc67d643700 1 -- 172.21.15.112:0/1450637784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6780b02e0 msgr2=0x7fc6780c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.046+0000 7fc67d643700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6780b02e0 0x7fc6780c9360 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc660005800 tx=0x7fc6680047c0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.046+0000 7fc67d643700 1 -- 172.21.15.112:0/1450637784 shutdown_connections 2023-08-27T11:26:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.046+0000 7fc67d643700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc66406a440 0x7fc66406c900 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.046+0000 7fc67d643700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6780c98a0 0x7fc67815dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.046+0000 7fc67d643700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc660004850 0x7fc6781f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.046+0000 7fc67d643700 1 --2- 172.21.15.112:0/1450637784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6780b02e0 0x7fc6780c9360 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.046+0000 7fc67d643700 1 -- 172.21.15.112:0/1450637784 >> 172.21.15.112:0/1450637784 conn(0x7fc6780d0000 msgr2=0x7fc6780cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.046+0000 7fc67d643700 1 -- 172.21.15.112:0/1450637784 shutdown_connections 2023-08-27T11:26:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.046+0000 7fc67d643700 1 -- 172.21.15.112:0/1450637784 wait complete. 2023-08-27T11:26:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:26:39.060 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-27T11:26:39.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:39.061 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-08-27T11:26:39.109 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:26:39.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:39.110 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-27T11:26:39.129 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:26:39.129 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:39.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:26:39.149 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:26:39.150 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:39.188 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:26:39.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:26:39.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.307+0000 7fabe0792700 1 Processor -- start 2023-08-27T11:26:39.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.307+0000 7fabe0792700 1 -- start start 2023-08-27T11:26:39.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.308+0000 7fabe0792700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fabd80b1400 0x7fabd80cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.308+0000 7fabe0792700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd8161850 con 0x7fabd80b17e0 2023-08-27T11:26:39.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.308+0000 7fabe0792700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd80cfc10 con 0x7fabd8161ce0 2023-08-27T11:26:39.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.308+0000 7fabe0792700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd8161200 con 0x7fabd80b1400 2023-08-27T11:26:39.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.308+0000 7fabddd2d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fabd80b17e0 0x7fabd806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55958/0 (socket says 172.21.15.112:55958) 2023-08-27T11:26:39.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.308+0000 7fabddd2d700 1 -- 172.21.15.112:0/3426761182 learned_addr learned my addr 172.21.15.112:0/3426761182 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:39.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.308+0000 7fabde52e700 1 --2- 172.21.15.112:0/3426761182 >> v2:172.21.15.138:3300/0 conn(0x7fabd80b1400 0x7fabd80cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:39.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.308+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525947295 0 0) 0x7fabd8161850 con 0x7fabd80b17e0 2023-08-27T11:26:39.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.308+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc40035b0 con 0x7fabd80b17e0 2023-08-27T11:26:39.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3109697487 0 0) 0x7fabc40035b0 con 0x7fabd80b17e0 2023-08-27T11:26:39.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 >> v2:172.21.15.138:3300/0 conn(0x7fabd80b1400 msgr2=0x7fabd80cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:39.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 --2- 172.21.15.112:0/3426761182 >> v2:172.21.15.138:3300/0 conn(0x7fabd80b1400 0x7fabd80cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 >> v1:172.21.15.138:6789/0 conn(0x7fabd8161ce0 legacy=0x7fabd806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd81614e0 con 0x7fabd80b17e0 2023-08-27T11:26:39.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fabd4001670 con 0x7fabd80b17e0 2023-08-27T11:26:39.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 >> v1:172.21.15.112:6789/0 conn(0x7fabd80b17e0 legacy=0x7fabd806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 --2- 172.21.15.112:0/3426761182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc4003a00 0x7fabc4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 --2- 172.21.15.112:0/3426761182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc4004850 0x7fabc4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 --2- 172.21.15.112:0/3426761182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc4005330 0x7fabc400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd8161200 con 0x7fabc4005330 2023-08-27T11:26:39.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc400c200 con 0x7fabc4003a00 2023-08-27T11:26:39.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc400c3c0 con 0x7fabc4004850 2023-08-27T11:26:39.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabded2f700 1 --2- 172.21.15.112:0/3426761182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc4005330 0x7fabc400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:39.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabde52e700 1 --2- 172.21.15.112:0/3426761182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc4003a00 0x7fabc4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:39.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabddd2d700 1 --2- 172.21.15.112:0/3426761182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc4004850 0x7fabc4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:39.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabded2f700 1 -- 172.21.15.112:0/3426761182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc4004850 msgr2=0x7fabc4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:39.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabded2f700 1 --2- 172.21.15.112:0/3426761182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc4004850 0x7fabc4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabded2f700 1 -- 172.21.15.112:0/3426761182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc4003a00 msgr2=0x7fabc4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:39.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabded2f700 1 --2- 172.21.15.112:0/3426761182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc4003a00 0x7fabc4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabded2f700 1 -- 172.21.15.112:0/3426761182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc40035b0 con 0x7fabc4005330 2023-08-27T11:26:39.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabded2f700 1 -- 172.21.15.112:0/3426761182 reap_dead start 2023-08-27T11:26:39.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabded2f700 1 --2- 172.21.15.112:0/3426761182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc4005330 0x7fabc400ba10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fabd000c160 tx=0x7fabc4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2c276990f172e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:39.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0001db0 con 0x7fabc4005330 2023-08-27T11:26:39.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.309+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd000cca0 con 0x7fabc4005330 2023-08-27T11:26:39.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.310+0000 7fabdd52c700 1 -- 172.21.15.112:0/3426761182 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0003c00 con 0x7fabc4005330 2023-08-27T11:26:39.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.310+0000 7fabe0792700 1 -- 172.21.15.112:0/3426761182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc4005330 msgr2=0x7fabc400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.310+0000 7fabe0792700 1 --2- 172.21.15.112:0/3426761182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc4005330 0x7fabc400ba10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fabd000c160 tx=0x7fabc4005880 comp rx=0 tx=0).stop 2023-08-27T11:26:39.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.310+0000 7fabe0792700 1 -- 172.21.15.112:0/3426761182 shutdown_connections 2023-08-27T11:26:39.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.310+0000 7fabe0792700 1 --2- 172.21.15.112:0/3426761182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc4005330 0x7fabc400ba10 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.310+0000 7fabe0792700 1 -- 172.21.15.112:0/3426761182 >> 172.21.15.112:0/3426761182 conn(0x7fabd80d0000 msgr2=0x7fabd80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:39.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.310+0000 7fabe0792700 1 -- 172.21.15.112:0/3426761182 shutdown_connections 2023-08-27T11:26:39.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.310+0000 7fabe0792700 1 -- 172.21.15.112:0/3426761182 wait complete. 2023-08-27T11:26:39.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.311+0000 7fabe0792700 1 Processor -- start 2023-08-27T11:26:39.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.311+0000 7fabe0792700 1 -- start start 2023-08-27T11:26:39.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabe0792700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc4005330 0x7fabd8150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabe0792700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabd80c9750 0x7fabd80c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabe0792700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabd8071510 0x7fabd8071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabe0792700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80c3b30 con 0x7fabd80c9750 2023-08-27T11:26:39.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabe0792700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd8150b70 con 0x7fabd8071510 2023-08-27T11:26:39.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabe0792700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd8070500 con 0x7fabc4005330 2023-08-27T11:26:39.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabddd2d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabd80c9750 0x7fabd80c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabddd2d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabd80c9750 0x7fabd80c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33358/0 (socket says 172.21.15.112:33358) 2023-08-27T11:26:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabddd2d700 1 -- 172.21.15.112:0/3648326366 learned_addr learned my addr 172.21.15.112:0/3648326366 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabded2f700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabd8071510 0x7fabd8071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabde52e700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc4005330 0x7fabd8150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabddd2d700 1 -- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc4005330 msgr2=0x7fabd8150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabddd2d700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc4005330 0x7fabd8150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabddd2d700 1 -- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabd8071510 msgr2=0x7fabd8071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabddd2d700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabd8071510 0x7fabd8071990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.312+0000 7fabddd2d700 1 -- 172.21.15.112:0/3648326366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd8070750 con 0x7fabd80c9750 2023-08-27T11:26:39.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.313+0000 7fabddd2d700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabd80c9750 0x7fabd80c9bd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fabd40072c0 tx=0x7fabd4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.313+0000 7fabceffd700 1 -- 172.21.15.112:0/3648326366 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd40010b0 con 0x7fabd80c9750 2023-08-27T11:26:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.313+0000 7fabceffd700 1 -- 172.21.15.112:0/3648326366 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd40077a0 con 0x7fabd80c9750 2023-08-27T11:26:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.313+0000 7fabe0792700 1 -- 172.21.15.112:0/3648326366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd8070a30 con 0x7fabd80c9750 2023-08-27T11:26:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.313+0000 7fabceffd700 1 -- 172.21.15.112:0/3648326366 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd400fab0 con 0x7fabd80c9750 2023-08-27T11:26:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.313+0000 7fabe0792700 1 -- 172.21.15.112:0/3648326366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd8070ff0 con 0x7fabd80c9750 2023-08-27T11:26:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.314+0000 7fabe0792700 1 -- 172.21.15.112:0/3648326366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd8067a20 con 0x7fabd80c9750 2023-08-27T11:26:39.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.314+0000 7fabceffd700 1 -- 172.21.15.112:0/3648326366 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fabd400fc10 con 0x7fabd80c9750 2023-08-27T11:26:39.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.314+0000 7fabceffd700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fabbc06a470 0x7fabbc06c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.314+0000 7fabceffd700 1 -- 172.21.15.112:0/3648326366 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fabd40889e0 con 0x7fabd80c9750 2023-08-27T11:26:39.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.315+0000 7fabde52e700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fabbc06a470 0x7fabbc06c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:39.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.315+0000 7fabde52e700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fabbc06a470 0x7fabbc06c930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009a20 tx=0x7fabc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:39.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.319+0000 7fabceffd700 1 -- 172.21.15.112:0/3648326366 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fabd4053c30 con 0x7fabd80c9750 2023-08-27T11:26:39.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.552+0000 7fabe0792700 1 -- 172.21.15.112:0/3648326366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabd806db60 con 0x7fabd80c9750 2023-08-27T11:26:39.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.553+0000 7fabceffd700 1 -- 172.21.15.112:0/3648326366 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4886 (secure 0 0 0) 0x7fabd4057250 con 0x7fabd80c9750 2023-08-27T11:26:39.555 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:39.555 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/120353149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":120353149},{"type":"v1","addr":"172.21.15.138:6836","nonce":120353149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1171600247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1171600247},{"type":"v1","addr":"172.21.15.138:6837","nonce":1171600247}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3930299696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3930299696},{"type":"v1","addr":"172.21.15.112:6834","nonce":3930299696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:26:18.805364+0000","modified":"2023-08-27T11:26:22.433680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7041":{"gid":7041,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2614833373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2614833373},{"type":"v1","addr":"172.21.15.112:6835","nonce":2614833373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-27T11:26:39.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.554+0000 7fabe0792700 1 -- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fabbc06a470 msgr2=0x7fabbc06c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.555+0000 7fabe0792700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fabbc06a470 0x7fabbc06c930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009a20 tx=0x7fabc8008040 comp rx=0 tx=0).stop 2023-08-27T11:26:39.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.555+0000 7fabe0792700 1 -- 172.21.15.112:0/3648326366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabd80c9750 msgr2=0x7fabd80c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.555+0000 7fabe0792700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabd80c9750 0x7fabd80c9bd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fabd40072c0 tx=0x7fabd4007680 comp rx=0 tx=0).stop 2023-08-27T11:26:39.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.555+0000 7fabe0792700 1 -- 172.21.15.112:0/3648326366 shutdown_connections 2023-08-27T11:26:39.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.555+0000 7fabe0792700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fabbc06a470 0x7fabbc06c930 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.555+0000 7fabe0792700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabd8071510 0x7fabd8071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.555+0000 7fabe0792700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc4005330 0x7fabd8150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.555+0000 7fabe0792700 1 --2- 172.21.15.112:0/3648326366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabd80c9750 0x7fabd80c9bd0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.555+0000 7fabe0792700 1 -- 172.21.15.112:0/3648326366 >> 172.21.15.112:0/3648326366 conn(0x7fabd80d0000 msgr2=0x7fabd80cada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:39.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.555+0000 7fabe0792700 1 -- 172.21.15.112:0/3648326366 shutdown_connections 2023-08-27T11:26:39.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.555+0000 7fabe0792700 1 -- 172.21.15.112:0/3648326366 wait complete. 2023-08-27T11:26:39.559 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 59 2023-08-27T11:26:39.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:39.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.688+0000 7fdfcf77c700 1 Processor -- start 2023-08-27T11:26:39.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.688+0000 7fdfcf77c700 1 -- start start 2023-08-27T11:26:39.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfcf77c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfc80ad210 0x7fdfc815e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfcf77c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdfc8141f10 0x7fdfc806e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfcf77c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc806ead0 con 0x7fdfc806e150 2023-08-27T11:26:39.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfcf77c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc8143bf0 con 0x7fdfc8141f10 2023-08-27T11:26:39.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfcf77c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80add70 con 0x7fdfc80ad210 2023-08-27T11:26:39.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfcdd19700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdfc8141f10 0x7fdfc806e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:39.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfcdd19700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdfc8141f10 0x7fdfc806e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33368/0 (socket says 172.21.15.112:33368) 2023-08-27T11:26:39.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfcdd19700 1 -- 172.21.15.112:0/3155668284 learned_addr learned my addr 172.21.15.112:0/3155668284 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfcdd19700 1 -- 172.21.15.112:0/3155668284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfc80ad210 msgr2=0x7fdfc815e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfcdd19700 1 --2- 172.21.15.112:0/3155668284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfc80ad210 0x7fdfc815e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfcdd19700 1 -- 172.21.15.112:0/3155668284 >> v1:172.21.15.138:6789/0 conn(0x7fdfc806e150 legacy=0x7fdfc8161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfcdd19700 1 -- 172.21.15.112:0/3155668284 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc8142fe0 con 0x7fdfc8141f10 2023-08-27T11:26:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfcdd19700 1 --2- 172.21.15.112:0/3155668284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc8141f10 0x7fdfc806e590 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009e80 tx=0x7fdfc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfc6ce956b56598 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:39.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfbffff700 1 -- 172.21.15.112:0/3155668284 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000e070 con 0x7fdfc8141f10 2023-08-27T11:26:39.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfbffff700 1 -- 172.21.15.112:0/3155668284 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc0003a20 con 0x7fdfc8141f10 2023-08-27T11:26:39.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.689+0000 7fdfbffff700 1 -- 172.21.15.112:0/3155668284 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0003dd0 con 0x7fdfc8141f10 2023-08-27T11:26:39.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.690+0000 7fdfcf77c700 1 -- 172.21.15.112:0/3155668284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc8141f10 msgr2=0x7fdfc806e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.690+0000 7fdfcf77c700 1 --2- 172.21.15.112:0/3155668284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc8141f10 0x7fdfc806e590 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009e80 tx=0x7fdfc000b330 comp rx=0 tx=0).stop 2023-08-27T11:26:39.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.690+0000 7fdfcf77c700 1 -- 172.21.15.112:0/3155668284 shutdown_connections 2023-08-27T11:26:39.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.690+0000 7fdfcf77c700 1 --2- 172.21.15.112:0/3155668284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc8141f10 0x7fdfc806e590 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.690+0000 7fdfcf77c700 1 --2- 172.21.15.112:0/3155668284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfc80ad210 0x7fdfc815e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.690+0000 7fdfcf77c700 1 -- 172.21.15.112:0/3155668284 >> 172.21.15.112:0/3155668284 conn(0x7fdfc806cd30 msgr2=0x7fdfc80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:39.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.690+0000 7fdfcf77c700 1 -- 172.21.15.112:0/3155668284 shutdown_connections 2023-08-27T11:26:39.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.690+0000 7fdfcf77c700 1 -- 172.21.15.112:0/3155668284 wait complete. 2023-08-27T11:26:39.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.691+0000 7fdfcf77c700 1 Processor -- start 2023-08-27T11:26:39.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.691+0000 7fdfcf77c700 1 -- start start 2023-08-27T11:26:39.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcf77c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfc806e150 0x7fdfc8156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcf77c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfc80ad210 0x7fdfc80b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcf77c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc8141f10 0x7fdfc80ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcf77c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc81571b0 con 0x7fdfc8141f10 2023-08-27T11:26:39.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcf77c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80b1d40 con 0x7fdfc806e150 2023-08-27T11:26:39.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcf77c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80b0ce0 con 0x7fdfc80ad210 2023-08-27T11:26:39.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcdd19700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc8141f10 0x7fdfc80ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:39.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcdd19700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc8141f10 0x7fdfc80ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33374/0 (socket says 172.21.15.112:33374) 2023-08-27T11:26:39.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcdd19700 1 -- 172.21.15.112:0/1415787059 learned_addr learned my addr 172.21.15.112:0/1415787059 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:39.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcdd19700 1 -- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfc80ad210 msgr2=0x7fdfc80b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcd518700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfc806e150 0x7fdfc8156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:39.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfccd17700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfc80ad210 0x7fdfc80b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:39.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcdd19700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfc80ad210 0x7fdfc80b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcdd19700 1 -- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfc806e150 msgr2=0x7fdfc8156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcdd19700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfc806e150 0x7fdfc8156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcdd19700 1 -- 172.21.15.112:0/1415787059 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc0009b30 con 0x7fdfc8141f10 2023-08-27T11:26:39.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.692+0000 7fdfcdd19700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc8141f10 0x7fdfc80ce100 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0004000 tx=0x7fdfc0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:39.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.693+0000 7fdfbdffb700 1 -- 172.21.15.112:0/1415787059 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0004240 con 0x7fdfc8141f10 2023-08-27T11:26:39.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.693+0000 7fdfbdffb700 1 -- 172.21.15.112:0/1415787059 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc00043e0 con 0x7fdfc8141f10 2023-08-27T11:26:39.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.693+0000 7fdfbdffb700 1 -- 172.21.15.112:0/1415787059 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0004790 con 0x7fdfc8141f10 2023-08-27T11:26:39.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.693+0000 7fdfcf77c700 1 -- 172.21.15.112:0/1415787059 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc80b0f80 con 0x7fdfc8141f10 2023-08-27T11:26:39.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.693+0000 7fdfcf77c700 1 -- 172.21.15.112:0/1415787059 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc80b1540 con 0x7fdfc8141f10 2023-08-27T11:26:39.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.694+0000 7fdfcf77c700 1 -- 172.21.15.112:0/1415787059 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc8067a20 con 0x7fdfc8141f10 2023-08-27T11:26:39.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.694+0000 7fdfbdffb700 1 -- 172.21.15.112:0/1415787059 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fdfc0019070 con 0x7fdfc8141f10 2023-08-27T11:26:39.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.694+0000 7fdfbdffb700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdfac06a440 0x7fdfac06c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:39.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.695+0000 7fdfbdffb700 1 -- 172.21.15.112:0/1415787059 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdfc0089b50 con 0x7fdfc8141f10 2023-08-27T11:26:39.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.695+0000 7fdfcd518700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdfac06a440 0x7fdfac06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:39.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.695+0000 7fdfcd518700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdfac06a440 0x7fdfac06c900 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4006fd0 tx=0x7fdfc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:39.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.699+0000 7fdfbdffb700 1 -- 172.21.15.112:0/1415787059 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdfc0054db0 con 0x7fdfc8141f10 2023-08-27T11:26:39.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.888+0000 7fdfcf77c700 1 -- 172.21.15.112:0/1415787059 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfc80acbf0 con 0x7fdfc8141f10 2023-08-27T11:26:39.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.890+0000 7fdfbdffb700 1 -- 172.21.15.112:0/1415787059 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+18248 (secure 0 0 0) 0x7fdfc0017090 con 0x7fdfc8141f10 2023-08-27T11:26:39.891 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:39.892 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:20.416307+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:16.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:17.773993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:39.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.892+0000 7fdfcf77c700 1 -- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdfac06a440 msgr2=0x7fdfac06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.892+0000 7fdfcf77c700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdfac06a440 0x7fdfac06c900 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4006fd0 tx=0x7fdfc4008040 comp rx=0 tx=0).stop 2023-08-27T11:26:39.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.892+0000 7fdfcf77c700 1 -- 172.21.15.112:0/1415787059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc8141f10 msgr2=0x7fdfc80ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:39.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.892+0000 7fdfcf77c700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc8141f10 0x7fdfc80ce100 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0004000 tx=0x7fdfc0004ad0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.892+0000 7fdfcf77c700 1 -- 172.21.15.112:0/1415787059 shutdown_connections 2023-08-27T11:26:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.892+0000 7fdfcf77c700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdfac06a440 0x7fdfac06c900 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.892+0000 7fdfcf77c700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc8141f10 0x7fdfc80ce100 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.892+0000 7fdfcf77c700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfc806e150 0x7fdfc8156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.892+0000 7fdfcf77c700 1 --2- 172.21.15.112:0/1415787059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfc80ad210 0x7fdfc80b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.892+0000 7fdfcf77c700 1 -- 172.21.15.112:0/1415787059 >> 172.21.15.112:0/1415787059 conn(0x7fdfc806cd30 msgr2=0x7fdfc80710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:39.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.892+0000 7fdfcf77c700 1 -- 172.21.15.112:0/1415787059 shutdown_connections 2023-08-27T11:26:39.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:39.892+0000 7fdfcf77c700 1 -- 172.21.15.112:0/1415787059 wait complete. 2023-08-27T11:26:39.906 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:26:16.548692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:26:17.773993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:26:39.906 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:26:40.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.027+0000 7f1673740700 1 Processor -- start 2023-08-27T11:26:40.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.027+0000 7f1673740700 1 -- start start 2023-08-27T11:26:40.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.028+0000 7f1673740700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f166c0ce210 0x7f166c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.028+0000 7f1673740700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166c0b08e0 con 0x7f166c1445b0 2023-08-27T11:26:40.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.028+0000 7f1673740700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166c0ce5f0 con 0x7f166c06fea0 2023-08-27T11:26:40.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.028+0000 7f1673740700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166c0b0b30 con 0x7f166c0ce210 2023-08-27T11:26:40.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.028+0000 7f16714dc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f166c1445b0 0x7f166c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55992/0 (socket says 172.21.15.112:55992) 2023-08-27T11:26:40.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.028+0000 7f1671cdd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f166c0ce210 0x7f166c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.028+0000 7f16714dc700 1 -- 172.21.15.112:0/1630585728 learned_addr learned my addr 172.21.15.112:0/1630585728 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.029+0000 7f1671cdd700 1 -- 172.21.15.112:0/1630585728 >> v1:172.21.15.138:6789/0 conn(0x7f166c06fea0 legacy=0x7f166c15ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.029+0000 7f1671cdd700 1 -- 172.21.15.112:0/1630585728 >> v1:172.21.15.112:6789/0 conn(0x7f166c1445b0 legacy=0x7f166c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.029+0000 7f1671cdd700 1 -- 172.21.15.112:0/1630585728 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c0b0e10 con 0x7f166c0ce210 2023-08-27T11:26:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.029+0000 7f1671cdd700 1 --2- 172.21.15.112:0/1630585728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f166c0ce210 0x7f166c0ccad0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1664009d90 tx=0x7f166400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecd42f1f09897cb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:40.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.029+0000 7f1663fff700 1 -- 172.21.15.112:0/1630585728 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166400e070 con 0x7f166c0ce210 2023-08-27T11:26:40.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.029+0000 7f1663fff700 1 -- 172.21.15.112:0/1630585728 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166400bb90 con 0x7f166c0ce210 2023-08-27T11:26:40.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.029+0000 7f1663fff700 1 -- 172.21.15.112:0/1630585728 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664003a20 con 0x7f166c0ce210 2023-08-27T11:26:40.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.029+0000 7f1673740700 1 -- 172.21.15.112:0/1630585728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f166c0ce210 msgr2=0x7f166c0ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.029+0000 7f1673740700 1 --2- 172.21.15.112:0/1630585728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f166c0ce210 0x7f166c0ccad0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1664009d90 tx=0x7f166400b330 comp rx=0 tx=0).stop 2023-08-27T11:26:40.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.030+0000 7f1673740700 1 -- 172.21.15.112:0/1630585728 shutdown_connections 2023-08-27T11:26:40.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.030+0000 7f1673740700 1 --2- 172.21.15.112:0/1630585728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f166c0ce210 0x7f166c0ccad0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.030+0000 7f1673740700 1 -- 172.21.15.112:0/1630585728 >> 172.21.15.112:0/1630585728 conn(0x7f166c0b1e90 msgr2=0x7f166c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:40.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.030+0000 7f1673740700 1 -- 172.21.15.112:0/1630585728 shutdown_connections 2023-08-27T11:26:40.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.030+0000 7f1673740700 1 -- 172.21.15.112:0/1630585728 wait complete. 2023-08-27T11:26:40.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.031+0000 7f1673740700 1 Processor -- start 2023-08-27T11:26:40.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.031+0000 7f1673740700 1 -- start start 2023-08-27T11:26:40.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.031+0000 7f1673740700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f166c06fea0 0x7f166c15a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1673740700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f166c0ce210 0x7f166c137350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1673740700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f166c1445b0 0x7f166c137890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1673740700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c1381b0 con 0x7f166c1445b0 2023-08-27T11:26:40.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1673740700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c135ea0 con 0x7f166c06fea0 2023-08-27T11:26:40.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1673740700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c136180 con 0x7f166c0ce210 2023-08-27T11:26:40.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1671cdd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f166c1445b0 0x7f166c137890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1671cdd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f166c1445b0 0x7f166c137890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33388/0 (socket says 172.21.15.112:33388) 2023-08-27T11:26:40.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1671cdd700 1 -- 172.21.15.112:0/2304055013 learned_addr learned my addr 172.21.15.112:0/2304055013 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:40.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f16714dc700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f166c06fea0 0x7f166c15a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1670cdb700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f166c0ce210 0x7f166c137350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1671cdd700 1 -- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f166c0ce210 msgr2=0x7f166c137350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1671cdd700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f166c0ce210 0x7f166c137350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1671cdd700 1 -- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f166c06fea0 msgr2=0x7f166c15a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1671cdd700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f166c06fea0 0x7f166c15a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1671cdd700 1 -- 172.21.15.112:0/2304055013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1664009a70 con 0x7f166c1445b0 2023-08-27T11:26:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1671cdd700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f166c1445b0 0x7f166c137890 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f166c0aff70 tx=0x7f1664004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1661ffb700 1 -- 172.21.15.112:0/2304055013 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664012630 con 0x7f166c1445b0 2023-08-27T11:26:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1661ffb700 1 -- 172.21.15.112:0/2304055013 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16640127b0 con 0x7f166c1445b0 2023-08-27T11:26:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1673740700 1 -- 172.21.15.112:0/2304055013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166c136460 con 0x7f166c1445b0 2023-08-27T11:26:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1661ffb700 1 -- 172.21.15.112:0/2304055013 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664012b60 con 0x7f166c1445b0 2023-08-27T11:26:40.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.032+0000 7f1673740700 1 -- 172.21.15.112:0/2304055013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166c136a20 con 0x7f166c1445b0 2023-08-27T11:26:40.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.033+0000 7f1673740700 1 -- 172.21.15.112:0/2304055013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166c136c40 con 0x7f166c1445b0 2023-08-27T11:26:40.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.034+0000 7f1661ffb700 1 -- 172.21.15.112:0/2304055013 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f1664012cc0 con 0x7f166c1445b0 2023-08-27T11:26:40.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.034+0000 7f1661ffb700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f165006a440 0x7f165006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.034+0000 7f1661ffb700 1 -- 172.21.15.112:0/2304055013 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f166408b270 con 0x7f166c1445b0 2023-08-27T11:26:40.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.035+0000 7f16714dc700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f165006a440 0x7f165006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.035+0000 7f16714dc700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f165006a440 0x7f165006c900 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f165c000f60 tx=0x7f165c007f10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:40.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.039+0000 7f1661ffb700 1 -- 172.21.15.112:0/2304055013 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1664056390 con 0x7f166c1445b0 2023-08-27T11:26:40.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.227+0000 7f1673740700 1 -- 172.21.15.112:0/2304055013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f166c067e90 con 0x7f166c1445b0 2023-08-27T11:26:40.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.228+0000 7f1661ffb700 1 -- 172.21.15.112:0/2304055013 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v90) v1 ==== 92+0+10 (secure 0 0 0) 0x7f16640599b0 con 0x7f166c1445b0 2023-08-27T11:26:40.230 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:26:40.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.230+0000 7f1673740700 1 -- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f165006a440 msgr2=0x7f165006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.230+0000 7f1673740700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f165006a440 0x7f165006c900 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f165c000f60 tx=0x7f165c007f10 comp rx=0 tx=0).stop 2023-08-27T11:26:40.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.230+0000 7f1673740700 1 -- 172.21.15.112:0/2304055013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f166c1445b0 msgr2=0x7f166c137890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.230+0000 7f1673740700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f166c1445b0 0x7f166c137890 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f166c0aff70 tx=0x7f1664004ad0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.230+0000 7f1673740700 1 -- 172.21.15.112:0/2304055013 shutdown_connections 2023-08-27T11:26:40.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.230+0000 7f1673740700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f165006a440 0x7f165006c900 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.230+0000 7f1673740700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f166c1445b0 0x7f166c137890 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.230+0000 7f1673740700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f166c06fea0 0x7f166c15a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.230+0000 7f1673740700 1 --2- 172.21.15.112:0/2304055013 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f166c0ce210 0x7f166c137350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.230+0000 7f1673740700 1 -- 172.21.15.112:0/2304055013 >> 172.21.15.112:0/2304055013 conn(0x7f166c0b1e90 msgr2=0x7f166c06f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:40.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.230+0000 7f1673740700 1 -- 172.21.15.112:0/2304055013 shutdown_connections 2023-08-27T11:26:40.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.230+0000 7f1673740700 1 -- 172.21.15.112:0/2304055013 wait complete. 2023-08-27T11:26:40.243 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:26:40.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.364+0000 7f080b9df700 1 Processor -- start 2023-08-27T11:26:40.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.365+0000 7f080b9df700 1 -- start start 2023-08-27T11:26:40.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.365+0000 7f080b9df700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08040b3440 0x7f080415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.365+0000 7f080b9df700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f080415e6e0 0x7f080406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.365+0000 7f080b9df700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08040b5070 con 0x7f0804146c20 2023-08-27T11:26:40.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.365+0000 7f080b9df700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0804147000 con 0x7f08040b3440 2023-08-27T11:26:40.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.365+0000 7f080b9df700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0804161740 con 0x7f080415e6e0 2023-08-27T11:26:40.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.365+0000 7f0808f7a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08040b3440 0x7f080415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.365+0000 7f0808f7a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08040b3440 0x7f080415e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33394/0 (socket says 172.21.15.112:33394) 2023-08-27T11:26:40.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.365+0000 7f0808f7a700 1 -- 172.21.15.112:0/3313224941 learned_addr learned my addr 172.21.15.112:0/3313224941 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:40.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.366+0000 7f0809f7c700 1 --2- 172.21.15.112:0/3313224941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f080415e6e0 0x7f080406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.366+0000 7f0808f7a700 1 -- 172.21.15.112:0/3313224941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f080415e6e0 msgr2=0x7f080406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:40.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.366+0000 7f0808f7a700 1 --2- 172.21.15.112:0/3313224941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f080415e6e0 0x7f080406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.366+0000 7f0808f7a700 1 -- 172.21.15.112:0/3313224941 >> v1:172.21.15.138:6789/0 conn(0x7f0804146c20 legacy=0x7f0804160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:40.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.366+0000 7f0808f7a700 1 -- 172.21.15.112:0/3313224941 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0804161a20 con 0x7f08040b3440 2023-08-27T11:26:40.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.366+0000 7f0808f7a700 1 --2- 172.21.15.112:0/3313224941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08040b3440 0x7f080415e1a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f08000098f0 tx=0x7f0800009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c49e0dddb6def06 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:40.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.366+0000 7f07fbfff700 1 -- 172.21.15.112:0/3313224941 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800004230 con 0x7f08040b3440 2023-08-27T11:26:40.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.366+0000 7f07fbfff700 1 -- 172.21.15.112:0/3313224941 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0800004390 con 0x7f08040b3440 2023-08-27T11:26:40.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.366+0000 7f07fbfff700 1 -- 172.21.15.112:0/3313224941 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08000036a0 con 0x7f08040b3440 2023-08-27T11:26:40.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.367+0000 7f080b9df700 1 -- 172.21.15.112:0/3313224941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08040b3440 msgr2=0x7f080415e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.367+0000 7f080b9df700 1 --2- 172.21.15.112:0/3313224941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08040b3440 0x7f080415e1a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f08000098f0 tx=0x7f0800009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:40.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.367+0000 7f080b9df700 1 -- 172.21.15.112:0/3313224941 shutdown_connections 2023-08-27T11:26:40.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.367+0000 7f080b9df700 1 --2- 172.21.15.112:0/3313224941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f080415e6e0 0x7f080406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.367+0000 7f080b9df700 1 --2- 172.21.15.112:0/3313224941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08040b3440 0x7f080415e1a0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.367+0000 7f080b9df700 1 -- 172.21.15.112:0/3313224941 >> 172.21.15.112:0/3313224941 conn(0x7f08040b1e90 msgr2=0x7f08040b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:40.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.367+0000 7f080b9df700 1 -- 172.21.15.112:0/3313224941 shutdown_connections 2023-08-27T11:26:40.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.367+0000 7f080b9df700 1 -- 172.21.15.112:0/3313224941 wait complete. 2023-08-27T11:26:40.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.368+0000 7f080b9df700 1 Processor -- start 2023-08-27T11:26:40.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.368+0000 7f080b9df700 1 -- start start 2023-08-27T11:26:40.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f080b9df700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08040b3440 0x7f080415a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f080b9df700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0804146c20 0x7f08040aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f080b9df700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f080415e6e0 0x7f0804144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f080b9df700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08040ae600 con 0x7f080415e6e0 2023-08-27T11:26:40.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f080b9df700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08040ae320 con 0x7f0804146c20 2023-08-27T11:26:40.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f080b9df700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0804145c30 con 0x7f08040b3440 2023-08-27T11:26:40.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f0809f7c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f080415e6e0 0x7f0804144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f0809f7c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f080415e6e0 0x7f0804144ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33402/0 (socket says 172.21.15.112:33402) 2023-08-27T11:26:40.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f0809f7c700 1 -- 172.21.15.112:0/3157768298 learned_addr learned my addr 172.21.15.112:0/3157768298 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:40.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f0808f7a700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0804146c20 0x7f08040aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f080977b700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08040b3440 0x7f080415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f0809f7c700 1 -- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08040b3440 msgr2=0x7f080415a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f0809f7c700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08040b3440 0x7f080415a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f0809f7c700 1 -- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0804146c20 msgr2=0x7f08040aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f0809f7c700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0804146c20 0x7f08040aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f0809f7c700 1 -- 172.21.15.112:0/3157768298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800009550 con 0x7f080415e6e0 2023-08-27T11:26:40.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f0809f7c700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f080415e6e0 0x7f0804144ed0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00cc30 tx=0x7f07fc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:40.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f07f9ffb700 1 -- 172.21.15.112:0/3157768298 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00aa50 con 0x7f080415e6e0 2023-08-27T11:26:40.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f07f9ffb700 1 -- 172.21.15.112:0/3157768298 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc00abb0 con 0x7f080415e6e0 2023-08-27T11:26:40.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f07f9ffb700 1 -- 172.21.15.112:0/3157768298 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc003a20 con 0x7f080415e6e0 2023-08-27T11:26:40.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.369+0000 7f080b9df700 1 -- 172.21.15.112:0/3157768298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0804145f10 con 0x7f080415e6e0 2023-08-27T11:26:40.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.370+0000 7f080b9df700 1 -- 172.21.15.112:0/3157768298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080415a7b0 con 0x7f080415e6e0 2023-08-27T11:26:40.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.371+0000 7f080b9df700 1 -- 172.21.15.112:0/3157768298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0804146430 con 0x7f080415e6e0 2023-08-27T11:26:40.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.371+0000 7f07f9ffb700 1 -- 172.21.15.112:0/3157768298 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f07fc00f3f0 con 0x7f080415e6e0 2023-08-27T11:26:40.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.371+0000 7f07f9ffb700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f07e806a440 0x7f07e806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.371+0000 7f07f9ffb700 1 -- 172.21.15.112:0/3157768298 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f07fc088da0 con 0x7f080415e6e0 2023-08-27T11:26:40.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.371+0000 7f080977b700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f07e806a440 0x7f07e806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.372+0000 7f080977b700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f07e806a440 0x7f07e806c900 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f07f4007fd0 tx=0x7f07f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:40.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.376+0000 7f07f9ffb700 1 -- 172.21.15.112:0/3157768298 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0804146430 con 0x7f080415e6e0 2023-08-27T11:26:40.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.570+0000 7f080b9df700 1 -- 172.21.15.112:0/3157768298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0804146430 con 0x7f080415e6e0 2023-08-27T11:26:40.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.570+0000 7f07f9ffb700 1 -- 172.21.15.112:0/3157768298 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0804146430 con 0x7f080415e6e0 2023-08-27T11:26:40.572 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:26:40.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.572+0000 7f080b9df700 1 -- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f07e806a440 msgr2=0x7f07e806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.572+0000 7f080b9df700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f07e806a440 0x7f07e806c900 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f07f4007fd0 tx=0x7f07f4009040 comp rx=0 tx=0).stop 2023-08-27T11:26:40.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.572+0000 7f080b9df700 1 -- 172.21.15.112:0/3157768298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f080415e6e0 msgr2=0x7f0804144ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.572+0000 7f080b9df700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f080415e6e0 0x7f0804144ed0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00cc30 tx=0x7f07fc00cf40 comp rx=0 tx=0).stop 2023-08-27T11:26:40.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.572+0000 7f080b9df700 1 -- 172.21.15.112:0/3157768298 shutdown_connections 2023-08-27T11:26:40.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.572+0000 7f080b9df700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f07e806a440 0x7f07e806c900 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.572+0000 7f080b9df700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f080415e6e0 0x7f0804144ed0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.572+0000 7f080b9df700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08040b3440 0x7f080415a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.572+0000 7f080b9df700 1 --2- 172.21.15.112:0/3157768298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0804146c20 0x7f08040aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.572+0000 7f080b9df700 1 -- 172.21.15.112:0/3157768298 >> 172.21.15.112:0/3157768298 conn(0x7f08040b1e90 msgr2=0x7f080406ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:40.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.572+0000 7f080b9df700 1 -- 172.21.15.112:0/3157768298 shutdown_connections 2023-08-27T11:26:40.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.573+0000 7f080b9df700 1 -- 172.21.15.112:0/3157768298 wait complete. 2023-08-27T11:26:40.586 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:26:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.707+0000 7f84d00ed700 1 Processor -- start 2023-08-27T11:26:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.708+0000 7f84d00ed700 1 -- start start 2023-08-27T11:26:40.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.708+0000 7f84d00ed700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f84c8153be0 0x7f84c806e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.708+0000 7f84d00ed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84c80ce660 0x7f84c806e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.709+0000 7f84d00ed700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c80cec80 con 0x7f84c8153fc0 2023-08-27T11:26:40.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.709+0000 7f84d00ed700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c80cf520 con 0x7f84c8153be0 2023-08-27T11:26:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.709+0000 7f84d00ed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80a9b50 con 0x7f84c80ce660 2023-08-27T11:26:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.709+0000 7f84ce68a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84c80ce660 0x7f84c806e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.709+0000 7f84ce68a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84c80ce660 0x7f84c806e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:45064/0 (socket says 172.21.15.112:45064) 2023-08-27T11:26:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.709+0000 7f84ce68a700 1 -- 172.21.15.112:0/3190579791 learned_addr learned my addr 172.21.15.112:0/3190579791 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:40.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.709+0000 7f84cde89700 1 --2- 172.21.15.112:0/3190579791 >> v2:172.21.15.138:3300/0 conn(0x7f84c8153be0 0x7f84c806e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.709+0000 7f84ce68a700 1 -- 172.21.15.112:0/3190579791 >> v2:172.21.15.138:3300/0 conn(0x7f84c8153be0 msgr2=0x7f84c806e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:40.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.709+0000 7f84ce68a700 1 --2- 172.21.15.112:0/3190579791 >> v2:172.21.15.138:3300/0 conn(0x7f84c8153be0 0x7f84c806e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.709+0000 7f84ce68a700 1 -- 172.21.15.112:0/3190579791 >> v1:172.21.15.138:6789/0 conn(0x7f84c8153fc0 legacy=0x7f84c80cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.709+0000 7f84ce68a700 1 -- 172.21.15.112:0/3190579791 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c80cef60 con 0x7f84c80ce660 2023-08-27T11:26:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.709+0000 7f84cde89700 1 --2- 172.21.15.112:0/3190579791 >> v2:172.21.15.138:3300/0 conn(0x7f84c8153be0 0x7f84c806e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.710+0000 7f84ce68a700 1 --2- 172.21.15.112:0/3190579791 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84c80ce660 0x7f84c806e700 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f84c00098f0 tx=0x7f84c0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=389ad05f9565b04 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.710+0000 7f84cce87700 1 -- 172.21.15.112:0/3190579791 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0011830 con 0x7f84c80ce660 2023-08-27T11:26:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.710+0000 7f84cce87700 1 -- 172.21.15.112:0/3190579791 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c0011990 con 0x7f84c80ce660 2023-08-27T11:26:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.710+0000 7f84cce87700 1 -- 172.21.15.112:0/3190579791 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0011d20 con 0x7f84c80ce660 2023-08-27T11:26:40.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.710+0000 7f84d00ed700 1 -- 172.21.15.112:0/3190579791 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84c80ce660 msgr2=0x7f84c806e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.710+0000 7f84d00ed700 1 --2- 172.21.15.112:0/3190579791 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84c80ce660 0x7f84c806e700 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f84c00098f0 tx=0x7f84c0009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:40.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.710+0000 7f84d00ed700 1 -- 172.21.15.112:0/3190579791 shutdown_connections 2023-08-27T11:26:40.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.710+0000 7f84d00ed700 1 --2- 172.21.15.112:0/3190579791 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84c80ce660 0x7f84c806e700 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.710+0000 7f84d00ed700 1 --2- 172.21.15.112:0/3190579791 >> v2:172.21.15.138:3300/0 conn(0x7f84c8153be0 0x7f84c806e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.710+0000 7f84d00ed700 1 -- 172.21.15.112:0/3190579791 >> 172.21.15.112:0/3190579791 conn(0x7f84c806cda0 msgr2=0x7f84c80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:40.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.710+0000 7f84d00ed700 1 -- 172.21.15.112:0/3190579791 shutdown_connections 2023-08-27T11:26:40.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.710+0000 7f84d00ed700 1 -- 172.21.15.112:0/3190579791 wait complete. 2023-08-27T11:26:40.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.712+0000 7f84d00ed700 1 Processor -- start 2023-08-27T11:26:40.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.712+0000 7f84d00ed700 1 -- start start 2023-08-27T11:26:40.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.712+0000 7f84d00ed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c80ce660 0x7f84c8140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.712+0000 7f84d00ed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c8153be0 0x7f84c813dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.712+0000 7f84d00ed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84c8153fc0 0x7f84c8152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.712+0000 7f84d00ed700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c813fc00 con 0x7f84c8153be0 2023-08-27T11:26:40.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.712+0000 7f84d00ed700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c813eff0 con 0x7f84c80ce660 2023-08-27T11:26:40.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.712+0000 7f84d00ed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c813f2d0 con 0x7f84c8153fc0 2023-08-27T11:26:40.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.712+0000 7f84cd688700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c8153be0 0x7f84c813dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.712+0000 7f84cd688700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c8153be0 0x7f84c813dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33414/0 (socket says 172.21.15.112:33414) 2023-08-27T11:26:40.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.712+0000 7f84cd688700 1 -- 172.21.15.112:0/2982945560 learned_addr learned my addr 172.21.15.112:0/2982945560 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:40.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.713+0000 7f84cde89700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c80ce660 0x7f84c8140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.713+0000 7f84ce68a700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84c8153fc0 0x7f84c8152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.713+0000 7f84cd688700 1 -- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84c8153fc0 msgr2=0x7f84c8152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.713+0000 7f84cd688700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84c8153fc0 0x7f84c8152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.713+0000 7f84cd688700 1 -- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c80ce660 msgr2=0x7f84c8140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.713+0000 7f84cd688700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c80ce660 0x7f84c8140090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.713+0000 7f84cd688700 1 -- 172.21.15.112:0/2982945560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b8009a70 con 0x7f84c8153be0 2023-08-27T11:26:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.713+0000 7f84cd688700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c8153be0 0x7f84c813dae0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00c450 tx=0x7f84bc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:40.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.713+0000 7f84c67fc700 1 -- 172.21.15.112:0/2982945560 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0052d0 con 0x7f84c8153be0 2023-08-27T11:26:40.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.713+0000 7f84c67fc700 1 -- 172.21.15.112:0/2982945560 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84bc005430 con 0x7f84c8153be0 2023-08-27T11:26:40.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.713+0000 7f84d00ed700 1 -- 172.21.15.112:0/2982945560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c0009550 con 0x7f84c8153be0 2023-08-27T11:26:40.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.713+0000 7f84c67fc700 1 -- 172.21.15.112:0/2982945560 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0038c0 con 0x7f84c8153be0 2023-08-27T11:26:40.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.713+0000 7f84d00ed700 1 -- 172.21.15.112:0/2982945560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c813f890 con 0x7f84c8153be0 2023-08-27T11:26:40.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.714+0000 7f84d00ed700 1 -- 172.21.15.112:0/2982945560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c8067a20 con 0x7f84c8153be0 2023-08-27T11:26:40.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.714+0000 7f84c67fc700 1 -- 172.21.15.112:0/2982945560 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f84bc003a20 con 0x7f84c8153be0 2023-08-27T11:26:40.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.715+0000 7f84c67fc700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f84b406a440 0x7f84b406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:40.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.715+0000 7f84c67fc700 1 -- 172.21.15.112:0/2982945560 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f84bc087ea0 con 0x7f84c8153be0 2023-08-27T11:26:40.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.715+0000 7f84cde89700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f84b406a440 0x7f84b406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:40.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.716+0000 7f84cde89700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f84b406a440 0x7f84b406c900 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f84c806e650 tx=0x7f84b800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:40.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.719+0000 7f84c67fc700 1 -- 172.21.15.112:0/2982945560 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f84bc053080 con 0x7f84c8153be0 2023-08-27T11:26:40.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.906+0000 7f84d00ed700 1 -- 172.21.15.112:0/2982945560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f84c813fd40 con 0x7f84c8153be0 2023-08-27T11:26:40.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.907+0000 7f84c67fc700 1 -- 172.21.15.112:0/2982945560 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84bc0566a0 con 0x7f84c8153be0 2023-08-27T11:26:40.909 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:26:40.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.908+0000 7f84d00ed700 1 -- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f84b406a440 msgr2=0x7f84b406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.908+0000 7f84d00ed700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f84b406a440 0x7f84b406c900 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f84c806e650 tx=0x7f84b800b040 comp rx=0 tx=0).stop 2023-08-27T11:26:40.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.908+0000 7f84d00ed700 1 -- 172.21.15.112:0/2982945560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c8153be0 msgr2=0x7f84c813dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:40.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.908+0000 7f84d00ed700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c8153be0 0x7f84c813dae0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00c450 tx=0x7f84bc00c810 comp rx=0 tx=0).stop 2023-08-27T11:26:40.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.909+0000 7f84d00ed700 1 -- 172.21.15.112:0/2982945560 shutdown_connections 2023-08-27T11:26:40.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.909+0000 7f84d00ed700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f84b406a440 0x7f84b406c900 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.909+0000 7f84d00ed700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84c8153fc0 0x7f84c8152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.909+0000 7f84d00ed700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c80ce660 0x7f84c8140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.909+0000 7f84d00ed700 1 --2- 172.21.15.112:0/2982945560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c8153be0 0x7f84c813dae0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:40.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.909+0000 7f84d00ed700 1 -- 172.21.15.112:0/2982945560 >> 172.21.15.112:0/2982945560 conn(0x7f84c806cda0 msgr2=0x7f84c81441e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:40.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.909+0000 7f84d00ed700 1 -- 172.21.15.112:0/2982945560 shutdown_connections 2023-08-27T11:26:40.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:40.909+0000 7f84d00ed700 1 -- 172.21.15.112:0/2982945560 wait complete. 2023-08-27T11:26:40.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:26:41.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.043+0000 7fc6c0aa2700 1 Processor -- start 2023-08-27T11:26:41.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.043+0000 7fc6c0aa2700 1 -- start start 2023-08-27T11:26:41.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.044+0000 7fc6c0aa2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc6bc146b30 0x7fc6bc06bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.044+0000 7fc6c0aa2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0d1580 0x7fc6bc0d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.044+0000 7fc6c0aa2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6bc06ce30 con 0x7fc6bc0715c0 2023-08-27T11:26:41.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.044+0000 7fc6c0aa2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6bc0d1f70 con 0x7fc6bc146b30 2023-08-27T11:26:41.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.044+0000 7fc6c0aa2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc06c870 con 0x7fc6bc0d1580 2023-08-27T11:26:41.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.044+0000 7fc6b9d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc6bc0715c0 0x7fc6bc161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56034/0 (socket says 172.21.15.112:56034) 2023-08-27T11:26:41.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.044+0000 7fc6b9d9b700 1 -- 172.21.15.112:0/1872741944 learned_addr learned my addr 172.21.15.112:0/1872741944 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:41.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.044+0000 7fc6ba59c700 1 --2- 172.21.15.112:0/1872741944 >> v2:172.21.15.138:3300/0 conn(0x7fc6bc146b30 0x7fc6bc06bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.044+0000 7fc6bad9d700 1 --2- 172.21.15.112:0/1872741944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0d1580 0x7fc6bc0d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.044+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764906581 0 0) 0x7fc6bc06ce30 con 0x7fc6bc0715c0 2023-08-27T11:26:41.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.044+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6a40035b0 con 0x7fc6bc0715c0 2023-08-27T11:26:41.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 780266551 0 0) 0x7fc6a40035b0 con 0x7fc6bc0715c0 2023-08-27T11:26:41.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0d1580 msgr2=0x7fc6bc0d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:41.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 --2- 172.21.15.112:0/1872741944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0d1580 0x7fc6bc0d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 >> v2:172.21.15.138:3300/0 conn(0x7fc6bc146b30 msgr2=0x7fc6bc06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:41.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 --2- 172.21.15.112:0/1872741944 >> v2:172.21.15.138:3300/0 conn(0x7fc6bc146b30 0x7fc6bc06bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc06cb50 con 0x7fc6bc0715c0 2023-08-27T11:26:41.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fc6ac001670 con 0x7fc6bc0715c0 2023-08-27T11:26:41.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 >> v1:172.21.15.112:6789/0 conn(0x7fc6bc0715c0 legacy=0x7fc6bc161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 --2- 172.21.15.112:0/1872741944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0d1580 0x7fc6a4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 --2- 172.21.15.112:0/1872741944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a4003f80 0x7fc6a4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 --2- 172.21.15.112:0/1872741944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a4004d90 0x7fc6a4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc0d1f70 con 0x7fc6a4003f80 2023-08-27T11:26:41.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc06c870 con 0x7fc6a4004d90 2023-08-27T11:26:41.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a400ba60 con 0x7fc6bc0d1580 2023-08-27T11:26:41.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6ba59c700 1 --2- 172.21.15.112:0/1872741944 >> v2:172.21.15.138:3300/0 conn(0x7fc6bc146b30 0x7fc6bc06bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:41.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b9d9b700 1 --2- 172.21.15.112:0/1872741944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a4003f80 0x7fc6a4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6bad9d700 1 --2- 172.21.15.112:0/1872741944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a4004d90 0x7fc6a4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6ba59c700 1 --2- 172.21.15.112:0/1872741944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0d1580 0x7fc6a4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b9d9b700 1 -- 172.21.15.112:0/1872741944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0d1580 msgr2=0x7fc6a4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:41.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b9d9b700 1 --2- 172.21.15.112:0/1872741944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0d1580 0x7fc6a4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b9d9b700 1 -- 172.21.15.112:0/1872741944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a4004d90 msgr2=0x7fc6a4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:41.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b9d9b700 1 --2- 172.21.15.112:0/1872741944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a4004d90 0x7fc6a4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b9d9b700 1 -- 172.21.15.112:0/1872741944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a40035b0 con 0x7fc6a4003f80 2023-08-27T11:26:41.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b9d9b700 1 --2- 172.21.15.112:0/1872741944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a4003f80 0x7fc6a4004850 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00c2f0 tx=0x7fc6a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899a2af92a902faa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:41.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac00cdf0 con 0x7fc6a4003f80 2023-08-27T11:26:41.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac005960 con 0x7fc6a4003f80 2023-08-27T11:26:41.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.045+0000 7fc6b959a700 1 -- 172.21.15.112:0/1872741944 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac005d10 con 0x7fc6a4003f80 2023-08-27T11:26:41.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.046+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1872741944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a4003f80 msgr2=0x7fc6a4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.046+0000 7fc6c0aa2700 1 --2- 172.21.15.112:0/1872741944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a4003f80 0x7fc6a4004850 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00c2f0 tx=0x7fc6a4005880 comp rx=0 tx=0).stop 2023-08-27T11:26:41.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.046+0000 7fc6bad9d700 1 -- 172.21.15.112:0/1872741944 reap_dead start 2023-08-27T11:26:41.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.046+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1872741944 shutdown_connections 2023-08-27T11:26:41.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.046+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1872741944 >> 172.21.15.112:0/1872741944 conn(0x7fc6bc06b1d0 msgr2=0x7fc6bc06b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:41.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.046+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1872741944 shutdown_connections 2023-08-27T11:26:41.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.046+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1872741944 wait complete. 2023-08-27T11:26:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.048+0000 7fc6c0aa2700 1 Processor -- start 2023-08-27T11:26:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.048+0000 7fc6c0aa2700 1 -- start start 2023-08-27T11:26:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6c0aa2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a4003f80 0x7fc6bc140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6c0aa2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0b5f40 0x7fc6bc0cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6c0aa2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6bc1f62a0 0x7fc6bc144730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6c0aa2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc141210 con 0x7fc6bc1f62a0 2023-08-27T11:26:41.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6c0aa2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1f7160 con 0x7fc6a4003f80 2023-08-27T11:26:41.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6c0aa2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1f6ba0 con 0x7fc6bc0b5f40 2023-08-27T11:26:41.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6bad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6bc1f62a0 0x7fc6bc144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6bad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6bc1f62a0 0x7fc6bc144730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33436/0 (socket says 172.21.15.112:33436) 2023-08-27T11:26:41.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6bad9d700 1 -- 172.21.15.112:0/1492577630 learned_addr learned my addr 172.21.15.112:0/1492577630 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:41.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6ba59c700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a4003f80 0x7fc6bc140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6b9d9b700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0b5f40 0x7fc6bc0cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6bad9d700 1 -- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0b5f40 msgr2=0x7fc6bc0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6bad9d700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0b5f40 0x7fc6bc0cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6bad9d700 1 -- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a4003f80 msgr2=0x7fc6bc140cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6bad9d700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a4003f80 0x7fc6bc140cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6bad9d700 1 -- 172.21.15.112:0/1492577630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc1f6e80 con 0x7fc6bc1f62a0 2023-08-27T11:26:41.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6b9d9b700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0b5f40 0x7fc6bc0cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:41.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6bad9d700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6bc1f62a0 0x7fc6bc144730 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc072260 tx=0x7fc6b4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:41.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6aaffd700 1 -- 172.21.15.112:0/1492577630 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400e040 con 0x7fc6bc1f62a0 2023-08-27T11:26:41.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6aaffd700 1 -- 172.21.15.112:0/1492577630 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b400f040 con 0x7fc6bc1f62a0 2023-08-27T11:26:41.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1492577630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6bc0cbd20 con 0x7fc6bc1f62a0 2023-08-27T11:26:41.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.049+0000 7fc6aaffd700 1 -- 172.21.15.112:0/1492577630 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400fc40 con 0x7fc6bc1f62a0 2023-08-27T11:26:41.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.050+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1492577630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc0cca60 con 0x7fc6bc1f62a0 2023-08-27T11:26:41.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.050+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1492577630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6bc067a20 con 0x7fc6bc1f62a0 2023-08-27T11:26:41.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.051+0000 7fc6aaffd700 1 -- 172.21.15.112:0/1492577630 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fc6b4015430 con 0x7fc6bc1f62a0 2023-08-27T11:26:41.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.051+0000 7fc6aaffd700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc69c06a440 0x7fc69c06c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.051+0000 7fc6aaffd700 1 -- 172.21.15.112:0/1492577630 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6b4089300 con 0x7fc6bc1f62a0 2023-08-27T11:26:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.051+0000 7fc6ba59c700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc69c06a440 0x7fc69c06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.052+0000 7fc6ba59c700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc69c06a440 0x7fc69c06c900 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009470 tx=0x7fc6b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.055+0000 7fc6aaffd700 1 -- 172.21.15.112:0/1492577630 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc6b4057ef0 con 0x7fc6bc1f62a0 2023-08-27T11:26:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.244+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1492577630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6bc161770 con 0x7fc6bc1f62a0 2023-08-27T11:26:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.245+0000 7fc6aaffd700 1 -- 172.21.15.112:0/1492577630 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6b4057a80 con 0x7fc6bc1f62a0 2023-08-27T11:26:41.247 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:26:41.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.247+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc69c06a440 msgr2=0x7fc69c06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.247+0000 7fc6c0aa2700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc69c06a440 0x7fc69c06c900 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009470 tx=0x7fc6b0008040 comp rx=0 tx=0).stop 2023-08-27T11:26:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.247+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1492577630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6bc1f62a0 msgr2=0x7fc6bc144730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.247+0000 7fc6c0aa2700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6bc1f62a0 0x7fc6bc144730 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc072260 tx=0x7fc6b4011e20 comp rx=0 tx=0).stop 2023-08-27T11:26:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.247+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1492577630 shutdown_connections 2023-08-27T11:26:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.247+0000 7fc6c0aa2700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc69c06a440 0x7fc69c06c900 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.247+0000 7fc6c0aa2700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6bc1f62a0 0x7fc6bc144730 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.247+0000 7fc6c0aa2700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a4003f80 0x7fc6bc140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.247+0000 7fc6c0aa2700 1 --2- 172.21.15.112:0/1492577630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6bc0b5f40 0x7fc6bc0cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.247+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1492577630 >> 172.21.15.112:0/1492577630 conn(0x7fc6bc06b1d0 msgr2=0x7fc6bc070a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.247+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1492577630 shutdown_connections 2023-08-27T11:26:41.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.247+0000 7fc6c0aa2700 1 -- 172.21.15.112:0/1492577630 wait complete. 2023-08-27T11:26:41.260 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:26:41.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.379+0000 7f6ce36b6700 1 Processor -- start 2023-08-27T11:26:41.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.380+0000 7f6ce36b6700 1 -- start start 2023-08-27T11:26:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.380+0000 7f6ce36b6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6cdc144570 0x7f6cdc15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.380+0000 7f6ce36b6700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc0b45f0 con 0x7f6cdc0b4ad0 2023-08-27T11:26:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.380+0000 7f6ce36b6700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc0b4310 con 0x7f6cdc0b3e50 2023-08-27T11:26:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.380+0000 7f6ce36b6700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cdc0cf7a0 con 0x7f6cdc144570 2023-08-27T11:26:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6ce1c53700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6cdc0b4ad0 0x7f6cdc0b4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56056/0 (socket says 172.21.15.112:56056) 2023-08-27T11:26:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6ce1c53700 1 -- 172.21.15.112:0/1670802115 learned_addr learned my addr 172.21.15.112:0/1670802115 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6ce1452700 1 --2- 172.21.15.112:0/1670802115 >> v2:172.21.15.138:3300/0 conn(0x7f6cdc144570 0x7f6cdc15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434827721 0 0) 0x7f6cdc0b45f0 con 0x7f6cdc0b4ad0 2023-08-27T11:26:41.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc80035b0 con 0x7f6cdc0b4ad0 2023-08-27T11:26:41.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690254107 0 0) 0x7f6cdc0b4310 con 0x7f6cdc0b3e50 2023-08-27T11:26:41.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cdc0b45f0 con 0x7f6cdc0b3e50 2023-08-27T11:26:41.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1361326287 0 0) 0x7f6cc80035b0 con 0x7f6cdc0b4ad0 2023-08-27T11:26:41.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 >> v2:172.21.15.138:3300/0 conn(0x7f6cdc144570 msgr2=0x7f6cdc15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:41.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 --2- 172.21.15.112:0/1670802115 >> v2:172.21.15.138:3300/0 conn(0x7f6cdc144570 0x7f6cdc15ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 >> v1:172.21.15.138:6789/0 conn(0x7f6cdc0b3e50 legacy=0x7f6cdc15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc0cdcb0 con 0x7f6cdc0b4ad0 2023-08-27T11:26:41.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f6cd40019f0 con 0x7f6cdc0b4ad0 2023-08-27T11:26:41.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 >> v1:172.21.15.112:6789/0 conn(0x7f6cdc0b4ad0 legacy=0x7f6cdc0b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 --2- 172.21.15.112:0/1670802115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cc8003b90 0x7f6cc8003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 --2- 172.21.15.112:0/1670802115 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cc8004850 0x7f6cc8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 --2- 172.21.15.112:0/1670802115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cc8005210 0x7f6cc800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0cf7a0 con 0x7f6cc8005210 2023-08-27T11:26:41.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc800c140 con 0x7f6cc8003b90 2023-08-27T11:26:41.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc800c300 con 0x7f6cc8004850 2023-08-27T11:26:41.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6ce1452700 1 --2- 172.21.15.112:0/1670802115 >> v2:172.21.15.138:3300/0 conn(0x7f6cdc144570 0x7f6cdc15ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:41.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6ce1c53700 1 --2- 172.21.15.112:0/1670802115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cc8005210 0x7f6cc800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6ce0c51700 1 --2- 172.21.15.112:0/1670802115 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cc8004850 0x7f6cc8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.381+0000 7f6ce1452700 1 --2- 172.21.15.112:0/1670802115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cc8003b90 0x7f6cc8003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6ce1c53700 1 -- 172.21.15.112:0/1670802115 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cc8004850 msgr2=0x7f6cc8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:41.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6ce1c53700 1 --2- 172.21.15.112:0/1670802115 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cc8004850 0x7f6cc8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6ce1c53700 1 -- 172.21.15.112:0/1670802115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cc8003b90 msgr2=0x7f6cc8003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:41.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6ce1c53700 1 --2- 172.21.15.112:0/1670802115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cc8003b90 0x7f6cc8003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6ce1c53700 1 -- 172.21.15.112:0/1670802115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80035b0 con 0x7f6cc8005210 2023-08-27T11:26:41.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6ce1452700 1 --2- 172.21.15.112:0/1670802115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cc8003b90 0x7f6cc8003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:41.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6ce1c53700 1 -- 172.21.15.112:0/1670802115 reap_dead start 2023-08-27T11:26:41.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6ce1c53700 1 --2- 172.21.15.112:0/1670802115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cc8005210 0x7f6cc800ba10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400a7c0 tx=0x7f6cc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f1bb96e0dee632 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:41.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4012600 con 0x7f6cc8005210 2023-08-27T11:26:41.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd4012780 con 0x7f6cc8005210 2023-08-27T11:26:41.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6cd3fff700 1 -- 172.21.15.112:0/1670802115 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4012b30 con 0x7f6cc8005210 2023-08-27T11:26:41.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6ce36b6700 1 -- 172.21.15.112:0/1670802115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cc8005210 msgr2=0x7f6cc800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6ce36b6700 1 --2- 172.21.15.112:0/1670802115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cc8005210 0x7f6cc800ba10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400a7c0 tx=0x7f6cc8005880 comp rx=0 tx=0).stop 2023-08-27T11:26:41.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6ce36b6700 1 -- 172.21.15.112:0/1670802115 shutdown_connections 2023-08-27T11:26:41.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6ce36b6700 1 --2- 172.21.15.112:0/1670802115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cc8005210 0x7f6cc800ba10 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.382+0000 7f6ce36b6700 1 -- 172.21.15.112:0/1670802115 >> 172.21.15.112:0/1670802115 conn(0x7f6cdc06f970 msgr2=0x7f6cdc06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:41.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.383+0000 7f6ce36b6700 1 -- 172.21.15.112:0/1670802115 shutdown_connections 2023-08-27T11:26:41.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.383+0000 7f6ce36b6700 1 -- 172.21.15.112:0/1670802115 wait complete. 2023-08-27T11:26:41.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.384+0000 7f6ce36b6700 1 Processor -- start 2023-08-27T11:26:41.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.384+0000 7f6ce36b6700 1 -- start start 2023-08-27T11:26:41.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.384+0000 7f6ce36b6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cc8005210 0x7f6cdc15a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.384+0000 7f6ce36b6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc071990 0x7f6cdc0ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.384+0000 7f6ce36b6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc0cbf90 0x7f6cdc13bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.384+0000 7f6ce36b6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0cbb70 con 0x7f6cdc0cbf90 2023-08-27T11:26:41.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.384+0000 7f6ce36b6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0caef0 con 0x7f6cdc071990 2023-08-27T11:26:41.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.384+0000 7f6ce36b6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0cb1d0 con 0x7f6cc8005210 2023-08-27T11:26:41.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6ce1c53700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc0cbf90 0x7f6cdc13bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6ce1c53700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc0cbf90 0x7f6cdc13bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33452/0 (socket says 172.21.15.112:33452) 2023-08-27T11:26:41.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6ce1c53700 1 -- 172.21.15.112:0/4069666253 learned_addr learned my addr 172.21.15.112:0/4069666253 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:41.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6ce0c51700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc071990 0x7f6cdc0ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6ce1452700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cc8005210 0x7f6cdc15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6ce1c53700 1 -- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cc8005210 msgr2=0x7f6cdc15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6ce1c53700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cc8005210 0x7f6cdc15a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6ce1c53700 1 -- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc071990 msgr2=0x7f6cdc0ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6ce1c53700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc071990 0x7f6cdc0ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6ce1c53700 1 -- 172.21.15.112:0/4069666253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc0cb4b0 con 0x7f6cdc0cbf90 2023-08-27T11:26:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6ce1c53700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc0cbf90 0x7f6cdc13bb50 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4000f80 tx=0x7f6cd4001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6cd1ffb700 1 -- 172.21.15.112:0/4069666253 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4012610 con 0x7f6cdc0cbf90 2023-08-27T11:26:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6cd1ffb700 1 -- 172.21.15.112:0/4069666253 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd40017b0 con 0x7f6cdc0cbf90 2023-08-27T11:26:41.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6cd1ffb700 1 -- 172.21.15.112:0/4069666253 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd401e430 con 0x7f6cdc0cbf90 2023-08-27T11:26:41.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6ce36b6700 1 -- 172.21.15.112:0/4069666253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc0cb790 con 0x7f6cdc0cbf90 2023-08-27T11:26:41.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.385+0000 7f6ce36b6700 1 -- 172.21.15.112:0/4069666253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc0caac0 con 0x7f6cdc0cbf90 2023-08-27T11:26:41.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.386+0000 7f6ce36b6700 1 -- 172.21.15.112:0/4069666253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cdc067a20 con 0x7f6cdc0cbf90 2023-08-27T11:26:41.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.387+0000 7f6cd1ffb700 1 -- 172.21.15.112:0/4069666253 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f6cd401e5b0 con 0x7f6cdc0cbf90 2023-08-27T11:26:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.387+0000 7f6cd1ffb700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cc006a440 0x7f6cc006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.387+0000 7f6cd1ffb700 1 -- 172.21.15.112:0/4069666253 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6cd408a3e0 con 0x7f6cdc0cbf90 2023-08-27T11:26:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.387+0000 7f6ce1452700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cc006a440 0x7f6cc006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.388+0000 7f6ce1452700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cc006a440 0x7f6cc006c900 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc006f90 tx=0x7f6ccc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.391+0000 7f6cd1ffb700 1 -- 172.21.15.112:0/4069666253 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6cd4055870 con 0x7f6cdc0cbf90 2023-08-27T11:26:41.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.626+0000 7f6ce36b6700 1 -- 172.21.15.112:0/4069666253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cdc067a20 con 0x7f6cdc0cbf90 2023-08-27T11:26:41.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.626+0000 7f6cd1ffb700 1 -- 172.21.15.112:0/4069666253 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4886 (secure 0 0 0) 0x7f6cd4058e90 con 0x7f6cdc0cbf90 2023-08-27T11:26:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:41.629 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/120353149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":120353149},{"type":"v1","addr":"172.21.15.138:6836","nonce":120353149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1171600247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1171600247},{"type":"v1","addr":"172.21.15.138:6837","nonce":1171600247}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3930299696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3930299696},{"type":"v1","addr":"172.21.15.112:6834","nonce":3930299696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:26:18.805364+0000","modified":"2023-08-27T11:26:22.433680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7041":{"gid":7041,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2614833373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2614833373},{"type":"v1","addr":"172.21.15.112:6835","nonce":2614833373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-27T11:26:41.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.628+0000 7f6ce36b6700 1 -- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cc006a440 msgr2=0x7f6cc006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.628+0000 7f6ce36b6700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cc006a440 0x7f6cc006c900 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc006f90 tx=0x7f6ccc008040 comp rx=0 tx=0).stop 2023-08-27T11:26:41.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.628+0000 7f6ce36b6700 1 -- 172.21.15.112:0/4069666253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc0cbf90 msgr2=0x7f6cdc13bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.628+0000 7f6ce36b6700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc0cbf90 0x7f6cdc13bb50 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4000f80 tx=0x7f6cd4001d00 comp rx=0 tx=0).stop 2023-08-27T11:26:41.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.628+0000 7f6ce36b6700 1 -- 172.21.15.112:0/4069666253 shutdown_connections 2023-08-27T11:26:41.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.628+0000 7f6ce36b6700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cc006a440 0x7f6cc006c900 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.628+0000 7f6ce36b6700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cdc0cbf90 0x7f6cdc13bb50 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.628+0000 7f6ce36b6700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cc8005210 0x7f6cdc15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.628+0000 7f6ce36b6700 1 --2- 172.21.15.112:0/4069666253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc071990 0x7f6cdc0ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.629+0000 7f6ce36b6700 1 -- 172.21.15.112:0/4069666253 >> 172.21.15.112:0/4069666253 conn(0x7f6cdc06f970 msgr2=0x7f6cdc138a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:41.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.629+0000 7f6ce36b6700 1 -- 172.21.15.112:0/4069666253 shutdown_connections 2023-08-27T11:26:41.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.629+0000 7f6ce36b6700 1 -- 172.21.15.112:0/4069666253 wait complete. 2023-08-27T11:26:41.633 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 59 2023-08-27T11:26:41.642 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:41.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.760+0000 7fdf378a6700 1 Processor -- start 2023-08-27T11:26:41.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.761+0000 7fdf378a6700 1 -- start start 2023-08-27T11:26:41.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.761+0000 7fdf378a6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf300b1400 0x7fdf300cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.761+0000 7fdf378a6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdf300b17e0 0x7fdf3006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.761+0000 7fdf378a6700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf30161850 con 0x7fdf30161ce0 2023-08-27T11:26:41.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.761+0000 7fdf378a6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3006e7d0 con 0x7fdf300b17e0 2023-08-27T11:26:41.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.761+0000 7fdf378a6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf300cfc10 con 0x7fdf300b1400 2023-08-27T11:26:41.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.762+0000 7fdf34e41700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdf300b17e0 0x7fdf3006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.762+0000 7fdf34e41700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdf300b17e0 0x7fdf3006e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33458/0 (socket says 172.21.15.112:33458) 2023-08-27T11:26:41.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.762+0000 7fdf34e41700 1 -- 172.21.15.112:0/4188087389 learned_addr learned my addr 172.21.15.112:0/4188087389 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:41.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.762+0000 7fdf35642700 1 --2- 172.21.15.112:0/4188087389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf300b1400 0x7fdf300cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.762+0000 7fdf34e41700 1 -- 172.21.15.112:0/4188087389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf300b1400 msgr2=0x7fdf300cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:41.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.762+0000 7fdf34e41700 1 --2- 172.21.15.112:0/4188087389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf300b1400 0x7fdf300cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.762+0000 7fdf34e41700 1 -- 172.21.15.112:0/4188087389 >> v1:172.21.15.138:6789/0 conn(0x7fdf30161ce0 legacy=0x7fdf3006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.762+0000 7fdf34e41700 1 -- 172.21.15.112:0/4188087389 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf30161200 con 0x7fdf300b17e0 2023-08-27T11:26:41.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.762+0000 7fdf34e41700 1 --2- 172.21.15.112:0/4188087389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf300b17e0 0x7fdf3006e290 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0098f0 tx=0x7fdf2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fb55a7057d9788 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:41.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.762+0000 7fdf27fff700 1 -- 172.21.15.112:0/4188087389 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c00b8a0 con 0x7fdf300b17e0 2023-08-27T11:26:41.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.762+0000 7fdf27fff700 1 -- 172.21.15.112:0/4188087389 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c00ba00 con 0x7fdf300b17e0 2023-08-27T11:26:41.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.762+0000 7fdf27fff700 1 -- 172.21.15.112:0/4188087389 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c00bd90 con 0x7fdf300b17e0 2023-08-27T11:26:41.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.763+0000 7fdf378a6700 1 -- 172.21.15.112:0/4188087389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf300b17e0 msgr2=0x7fdf3006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.763+0000 7fdf378a6700 1 --2- 172.21.15.112:0/4188087389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf300b17e0 0x7fdf3006e290 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0098f0 tx=0x7fdf2c009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:41.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.763+0000 7fdf378a6700 1 -- 172.21.15.112:0/4188087389 shutdown_connections 2023-08-27T11:26:41.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.763+0000 7fdf378a6700 1 --2- 172.21.15.112:0/4188087389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf300b1400 0x7fdf300cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.763+0000 7fdf378a6700 1 --2- 172.21.15.112:0/4188087389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf300b17e0 0x7fdf3006e290 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.763+0000 7fdf378a6700 1 -- 172.21.15.112:0/4188087389 >> 172.21.15.112:0/4188087389 conn(0x7fdf300d0000 msgr2=0x7fdf300d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:41.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.763+0000 7fdf378a6700 1 -- 172.21.15.112:0/4188087389 shutdown_connections 2023-08-27T11:26:41.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.763+0000 7fdf378a6700 1 -- 172.21.15.112:0/4188087389 wait complete. 2023-08-27T11:26:41.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.764+0000 7fdf378a6700 1 Processor -- start 2023-08-27T11:26:41.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.764+0000 7fdf378a6700 1 -- start start 2023-08-27T11:26:41.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf378a6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf300b1400 0x7fdf300715f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf378a6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf300b17e0 0x7fdf3015fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf378a6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf30161ce0 0x7fdf300c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf378a6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30071160 con 0x7fdf300b1400 2023-08-27T11:26:41.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf378a6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30160620 con 0x7fdf30161ce0 2023-08-27T11:26:41.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf378a6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30070b10 con 0x7fdf300b17e0 2023-08-27T11:26:41.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf35642700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf300b1400 0x7fdf300715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf35642700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf300b1400 0x7fdf300715f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33462/0 (socket says 172.21.15.112:33462) 2023-08-27T11:26:41.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf35642700 1 -- 172.21.15.112:0/3558516132 learned_addr learned my addr 172.21.15.112:0/3558516132 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:41.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf34e41700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf300b17e0 0x7fdf3015fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf35e43700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf30161ce0 0x7fdf300c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf35642700 1 -- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf300b17e0 msgr2=0x7fdf3015fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf35642700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf300b17e0 0x7fdf3015fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf35642700 1 -- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf30161ce0 msgr2=0x7fdf300c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf35642700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf30161ce0 0x7fdf300c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf35642700 1 -- 172.21.15.112:0/3558516132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c009550 con 0x7fdf300b1400 2023-08-27T11:26:41.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf35642700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf300b1400 0x7fdf300715f0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000e810 tx=0x7fdf2000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:41.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf25ffb700 1 -- 172.21.15.112:0/3558516132 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf20013610 con 0x7fdf300b1400 2023-08-27T11:26:41.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf25ffb700 1 -- 172.21.15.112:0/3558516132 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf20013770 con 0x7fdf300b1400 2023-08-27T11:26:41.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf25ffb700 1 -- 172.21.15.112:0/3558516132 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf20013b20 con 0x7fdf300b1400 2023-08-27T11:26:41.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.765+0000 7fdf378a6700 1 -- 172.21.15.112:0/3558516132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf30070df0 con 0x7fdf300b1400 2023-08-27T11:26:41.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.766+0000 7fdf378a6700 1 -- 172.21.15.112:0/3558516132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf30150af0 con 0x7fdf300b1400 2023-08-27T11:26:41.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.767+0000 7fdf378a6700 1 -- 172.21.15.112:0/3558516132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf30067a20 con 0x7fdf300b1400 2023-08-27T11:26:41.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.767+0000 7fdf25ffb700 1 -- 172.21.15.112:0/3558516132 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fdf20013c80 con 0x7fdf300b1400 2023-08-27T11:26:41.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.767+0000 7fdf25ffb700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdf1406a3b0 0x7fdf1406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:41.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.767+0000 7fdf25ffb700 1 -- 172.21.15.112:0/3558516132 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdf20088880 con 0x7fdf300b1400 2023-08-27T11:26:41.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.768+0000 7fdf34e41700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdf1406a3b0 0x7fdf1406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:41.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.768+0000 7fdf34e41700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdf1406a3b0 0x7fdf1406c870 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdf30153150 tx=0x7fdf2c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:41.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.772+0000 7fdf25ffb700 1 -- 172.21.15.112:0/3558516132 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdf20053a40 con 0x7fdf300b1400 2023-08-27T11:26:41.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.959+0000 7fdf378a6700 1 -- 172.21.15.112:0/3558516132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf3006db60 con 0x7fdf300b1400 2023-08-27T11:26:41.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.960+0000 7fdf25ffb700 1 -- 172.21.15.112:0/3558516132 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+18248 (secure 0 0 0) 0x7fdf20057060 con 0x7fdf300b1400 2023-08-27T11:26:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:41.963 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:20.416307+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:16.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:17.773993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:41.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.962+0000 7fdf378a6700 1 -- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdf1406a3b0 msgr2=0x7fdf1406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.962+0000 7fdf378a6700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdf1406a3b0 0x7fdf1406c870 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdf30153150 tx=0x7fdf2c008030 comp rx=0 tx=0).stop 2023-08-27T11:26:41.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.962+0000 7fdf378a6700 1 -- 172.21.15.112:0/3558516132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf300b1400 msgr2=0x7fdf300715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:41.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.962+0000 7fdf378a6700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf300b1400 0x7fdf300715f0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000e810 tx=0x7fdf2000ebd0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.962+0000 7fdf378a6700 1 -- 172.21.15.112:0/3558516132 shutdown_connections 2023-08-27T11:26:41.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.962+0000 7fdf378a6700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdf1406a3b0 0x7fdf1406c870 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.962+0000 7fdf378a6700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf30161ce0 0x7fdf300c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.962+0000 7fdf378a6700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf300b1400 0x7fdf300715f0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.962+0000 7fdf378a6700 1 --2- 172.21.15.112:0/3558516132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf300b17e0 0x7fdf3015fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:41.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.962+0000 7fdf378a6700 1 -- 172.21.15.112:0/3558516132 >> 172.21.15.112:0/3558516132 conn(0x7fdf300d0000 msgr2=0x7fdf300d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:41.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.962+0000 7fdf378a6700 1 -- 172.21.15.112:0/3558516132 shutdown_connections 2023-08-27T11:26:41.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:41.962+0000 7fdf378a6700 1 -- 172.21.15.112:0/3558516132 wait complete. 2023-08-27T11:26:41.979 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:26:41.980 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:26:42.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.099+0000 7f226bca9700 1 Processor -- start 2023-08-27T11:26:42.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.099+0000 7f226bca9700 1 -- start start 2023-08-27T11:26:42.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.100+0000 7f226bca9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2264146c00 0x7f226415e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.100+0000 7f226bca9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2264066720 0x7f22640708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.100+0000 7f226bca9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2264072250 con 0x7f22640cec70 2023-08-27T11:26:42.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.100+0000 7f226bca9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22640d0b60 con 0x7f2264146c00 2023-08-27T11:26:42.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.100+0000 7f226bca9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22640724a0 con 0x7f2264066720 2023-08-27T11:26:42.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.100+0000 7f2269a45700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2264146c00 0x7f226415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.100+0000 7f2269a45700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2264146c00 0x7f226415e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33470/0 (socket says 172.21.15.112:33470) 2023-08-27T11:26:42.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.100+0000 7f2269a45700 1 -- 172.21.15.112:0/3100099192 learned_addr learned my addr 172.21.15.112:0/3100099192 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:42.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.100+0000 7f226a246700 1 --2- 172.21.15.112:0/3100099192 >> v2:172.21.15.138:3300/0 conn(0x7f2264066720 0x7f22640708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.100+0000 7f2269a45700 1 -- 172.21.15.112:0/3100099192 >> v2:172.21.15.138:3300/0 conn(0x7f2264066720 msgr2=0x7f22640708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:42.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.100+0000 7f2269a45700 1 --2- 172.21.15.112:0/3100099192 >> v2:172.21.15.138:3300/0 conn(0x7f2264066720 0x7f22640708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.100+0000 7f2269a45700 1 -- 172.21.15.112:0/3100099192 >> v1:172.21.15.138:6789/0 conn(0x7f22640cec70 legacy=0x7f2264066010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:42.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.101+0000 7f2269a45700 1 -- 172.21.15.112:0/3100099192 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264072780 con 0x7f2264146c00 2023-08-27T11:26:42.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.101+0000 7f226a246700 1 --2- 172.21.15.112:0/3100099192 >> v2:172.21.15.138:3300/0 conn(0x7f2264066720 0x7f22640708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.101+0000 7f2269a45700 1 --2- 172.21.15.112:0/3100099192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2264146c00 0x7f226415e550 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f22640cb0b0 tx=0x7f2254009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65900a0190c3595b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.101+0000 7f2268a43700 1 -- 172.21.15.112:0/3100099192 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2254004230 con 0x7f2264146c00 2023-08-27T11:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.101+0000 7f2268a43700 1 -- 172.21.15.112:0/3100099192 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2254004390 con 0x7f2264146c00 2023-08-27T11:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.101+0000 7f2268a43700 1 -- 172.21.15.112:0/3100099192 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22540036a0 con 0x7f2264146c00 2023-08-27T11:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.101+0000 7f226bca9700 1 -- 172.21.15.112:0/3100099192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2264146c00 msgr2=0x7f226415e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.101+0000 7f226bca9700 1 --2- 172.21.15.112:0/3100099192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2264146c00 0x7f226415e550 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f22640cb0b0 tx=0x7f2254009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.101+0000 7f226bca9700 1 -- 172.21.15.112:0/3100099192 shutdown_connections 2023-08-27T11:26:42.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.101+0000 7f226bca9700 1 --2- 172.21.15.112:0/3100099192 >> v2:172.21.15.138:3300/0 conn(0x7f2264066720 0x7f22640708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.101+0000 7f226bca9700 1 --2- 172.21.15.112:0/3100099192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2264146c00 0x7f226415e550 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.101+0000 7f226bca9700 1 -- 172.21.15.112:0/3100099192 >> 172.21.15.112:0/3100099192 conn(0x7f226406f970 msgr2=0x7f226406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:42.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.102+0000 7f226bca9700 1 -- 172.21.15.112:0/3100099192 shutdown_connections 2023-08-27T11:26:42.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.102+0000 7f226bca9700 1 -- 172.21.15.112:0/3100099192 wait complete. 2023-08-27T11:26:42.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.103+0000 7f226bca9700 1 Processor -- start 2023-08-27T11:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.103+0000 7f226bca9700 1 -- start start 2023-08-27T11:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.103+0000 7f226bca9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2264066720 0x7f226413c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.103+0000 7f226bca9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22640cec70 0x7f22640713a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.103+0000 7f226bca9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2264146c00 0x7f22640718e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.103+0000 7f226bca9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22640cbec0 con 0x7f2264066720 2023-08-27T11:26:42.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.103+0000 7f226bca9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226415a5e0 con 0x7f2264146c00 2023-08-27T11:26:42.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.103+0000 7f226bca9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22640cb870 con 0x7f22640cec70 2023-08-27T11:26:42.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f2269a45700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2264066720 0x7f226413c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f2269a45700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2264066720 0x7f226413c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33476/0 (socket says 172.21.15.112:33476) 2023-08-27T11:26:42.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f2269a45700 1 -- 172.21.15.112:0/2736087972 learned_addr learned my addr 172.21.15.112:0/2736087972 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:42.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f226a246700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2264146c00 0x7f22640718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f2269244700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22640cec70 0x7f22640713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f2269a45700 1 -- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22640cec70 msgr2=0x7f22640713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f2269a45700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22640cec70 0x7f22640713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f2269a45700 1 -- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2264146c00 msgr2=0x7f22640718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f2269a45700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2264146c00 0x7f22640718e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f2269a45700 1 -- 172.21.15.112:0/2736087972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2254009550 con 0x7f2264066720 2023-08-27T11:26:42.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f2269244700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22640cec70 0x7f22640713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:42.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f2269a45700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2264066720 0x7f226413c490 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f22640ceab0 tx=0x7f2254004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f225a7fc700 1 -- 172.21.15.112:0/2736087972 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2254011630 con 0x7f2264066720 2023-08-27T11:26:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f225a7fc700 1 -- 172.21.15.112:0/2736087972 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2254011790 con 0x7f2264066720 2023-08-27T11:26:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f226bca9700 1 -- 172.21.15.112:0/2736087972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22640cbb50 con 0x7f2264066720 2023-08-27T11:26:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f225a7fc700 1 -- 172.21.15.112:0/2736087972 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2254011b40 con 0x7f2264066720 2023-08-27T11:26:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.104+0000 7f226bca9700 1 -- 172.21.15.112:0/2736087972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22641598b0 con 0x7f2264066720 2023-08-27T11:26:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.105+0000 7f226bca9700 1 -- 172.21.15.112:0/2736087972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2264067a20 con 0x7f2264066720 2023-08-27T11:26:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.106+0000 7f225a7fc700 1 -- 172.21.15.112:0/2736087972 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f2254011ca0 con 0x7f2264066720 2023-08-27T11:26:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.106+0000 7f225a7fc700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f224806a440 0x7f224806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.106+0000 7f225a7fc700 1 -- 172.21.15.112:0/2736087972 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f225408a070 con 0x7f2264066720 2023-08-27T11:26:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.106+0000 7f2269244700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f224806a440 0x7f224806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.107+0000 7f2269244700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f224806a440 0x7f224806c900 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2264160e00 tx=0x7f2260009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:42.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.110+0000 7f225a7fc700 1 -- 172.21.15.112:0/2736087972 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2254055490 con 0x7f2264066720 2023-08-27T11:26:42.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.339+0000 7f226bca9700 1 -- 172.21.15.112:0/2736087972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22640caa60 con 0x7f2264066720 2023-08-27T11:26:42.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.340+0000 7f225a7fc700 1 -- 172.21.15.112:0/2736087972 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4886 (secure 0 0 0) 0x7f2254058ab0 con 0x7f2264066720 2023-08-27T11:26:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:42.342 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/120353149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":120353149},{"type":"v1","addr":"172.21.15.138:6836","nonce":120353149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1171600247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1171600247},{"type":"v1","addr":"172.21.15.138:6837","nonce":1171600247}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3930299696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3930299696},{"type":"v1","addr":"172.21.15.112:6834","nonce":3930299696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:26:18.805364+0000","modified":"2023-08-27T11:26:22.433680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7041":{"gid":7041,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2614833373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2614833373},{"type":"v1","addr":"172.21.15.112:6835","nonce":2614833373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-27T11:26:42.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.342+0000 7f226bca9700 1 -- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f224806a440 msgr2=0x7f224806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.342+0000 7f226bca9700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f224806a440 0x7f224806c900 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2264160e00 tx=0x7f2260009040 comp rx=0 tx=0).stop 2023-08-27T11:26:42.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.342+0000 7f226bca9700 1 -- 172.21.15.112:0/2736087972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2264066720 msgr2=0x7f226413c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.342+0000 7f226bca9700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2264066720 0x7f226413c490 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f22640ceab0 tx=0x7f2254004750 comp rx=0 tx=0).stop 2023-08-27T11:26:42.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.342+0000 7f226bca9700 1 -- 172.21.15.112:0/2736087972 shutdown_connections 2023-08-27T11:26:42.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.342+0000 7f226bca9700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f224806a440 0x7f224806c900 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.342+0000 7f226bca9700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2264146c00 0x7f22640718e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.342+0000 7f226bca9700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2264066720 0x7f226413c490 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.342+0000 7f226bca9700 1 --2- 172.21.15.112:0/2736087972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22640cec70 0x7f22640713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.342+0000 7f226bca9700 1 -- 172.21.15.112:0/2736087972 >> 172.21.15.112:0/2736087972 conn(0x7f226406f970 msgr2=0x7f226415b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:42.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.342+0000 7f226bca9700 1 -- 172.21.15.112:0/2736087972 shutdown_connections 2023-08-27T11:26:42.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.342+0000 7f226bca9700 1 -- 172.21.15.112:0/2736087972 wait complete. 2023-08-27T11:26:42.347 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 59 2023-08-27T11:26:42.357 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:26:42.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.478+0000 7f6e38b73700 1 Processor -- start 2023-08-27T11:26:42.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.478+0000 7f6e38b73700 1 -- start start 2023-08-27T11:26:42.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e38b73700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6e340b3e50 0x7f6e3415f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e38b73700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e340b4ad0 0x7f6e340b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e38b73700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e340b45f0 con 0x7f6e34144570 2023-08-27T11:26:42.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e38b73700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e340cf7a0 con 0x7f6e340b3e50 2023-08-27T11:26:42.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e38b73700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34144950 con 0x7f6e340b4ad0 2023-08-27T11:26:42.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e32d9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e340b4ad0 0x7f6e340b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e31d9b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6e340b3e50 0x7f6e3415f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e31d9b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6e340b3e50 0x7f6e3415f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:33912/0 (socket says 172.21.15.112:33912) 2023-08-27T11:26:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e3259c700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6e34144570 0x7f6e3415ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:35390/0 (socket says 172.21.15.112:35390) 2023-08-27T11:26:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e32d9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e340b4ad0 0x7f6e340b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:45142/0 (socket says 172.21.15.112:45142) 2023-08-27T11:26:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e31d9b700 1 -- 172.21.15.112:0/160649400 learned_addr learned my addr 172.21.15.112:0/160649400 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e31d9b700 1 -- 172.21.15.112:0/160649400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e340b4ad0 msgr2=0x7f6e340b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e31d9b700 1 --2- 172.21.15.112:0/160649400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e340b4ad0 0x7f6e340b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e31d9b700 1 -- 172.21.15.112:0/160649400 >> v1:172.21.15.138:6789/0 conn(0x7f6e34144570 legacy=0x7f6e3415ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e31d9b700 1 -- 172.21.15.112:0/160649400 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e340b4310 con 0x7f6e340b3e50 2023-08-27T11:26:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e3159a700 1 -- 172.21.15.112:0/160649400 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373794583 0 0) 0x7f6e340b45f0 con 0x7f6e34144570 2023-08-27T11:26:42.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e32d9d700 1 --2- 172.21.15.112:0/160649400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e340b4ad0 0x7f6e340b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:42.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.479+0000 7f6e31d9b700 1 --2- 172.21.15.112:0/160649400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e340b3e50 0x7f6e3415f530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009920 tx=0x7f6e24009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f28edc88b829a58b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:42.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.480+0000 7f6e3159a700 1 -- 172.21.15.112:0/160649400 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24011830 con 0x7f6e340b3e50 2023-08-27T11:26:42.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.480+0000 7f6e3159a700 1 -- 172.21.15.112:0/160649400 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e24011990 con 0x7f6e340b3e50 2023-08-27T11:26:42.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.480+0000 7f6e3159a700 1 -- 172.21.15.112:0/160649400 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24011d20 con 0x7f6e340b3e50 2023-08-27T11:26:42.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.480+0000 7f6e38b73700 1 -- 172.21.15.112:0/160649400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e340b3e50 msgr2=0x7f6e3415f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.480+0000 7f6e38b73700 1 --2- 172.21.15.112:0/160649400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e340b3e50 0x7f6e3415f530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009920 tx=0x7f6e24009c30 comp rx=0 tx=0).stop 2023-08-27T11:26:42.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.480+0000 7f6e38b73700 1 -- 172.21.15.112:0/160649400 shutdown_connections 2023-08-27T11:26:42.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.480+0000 7f6e38b73700 1 --2- 172.21.15.112:0/160649400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e340b4ad0 0x7f6e340b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.480+0000 7f6e38b73700 1 --2- 172.21.15.112:0/160649400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e340b3e50 0x7f6e3415f530 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.480+0000 7f6e38b73700 1 -- 172.21.15.112:0/160649400 >> 172.21.15.112:0/160649400 conn(0x7f6e3406f970 msgr2=0x7f6e3406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:42.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.480+0000 7f6e38b73700 1 -- 172.21.15.112:0/160649400 shutdown_connections 2023-08-27T11:26:42.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.480+0000 7f6e38b73700 1 -- 172.21.15.112:0/160649400 wait complete. 2023-08-27T11:26:42.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.482+0000 7f6e38b73700 1 Processor -- start 2023-08-27T11:26:42.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.482+0000 7f6e38b73700 1 -- start start 2023-08-27T11:26:42.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.482+0000 7f6e38b73700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e340b3e50 0x7f6e340cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.482+0000 7f6e38b73700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e340b4ad0 0x7f6e340708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.482+0000 7f6e38b73700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e34144570 0x7f6e341388c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.482+0000 7f6e38b73700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34072350 con 0x7f6e340b4ad0 2023-08-27T11:26:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.482+0000 7f6e38b73700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34071ec0 con 0x7f6e340b3e50 2023-08-27T11:26:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.482+0000 7f6e38b73700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340cb820 con 0x7f6e34144570 2023-08-27T11:26:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.482+0000 7f6e31d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e340b4ad0 0x7f6e340708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e31d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e340b4ad0 0x7f6e340708a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33492/0 (socket says 172.21.15.112:33492) 2023-08-27T11:26:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e31d9b700 1 -- 172.21.15.112:0/4062681406 learned_addr learned my addr 172.21.15.112:0/4062681406 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e32d9d700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e34144570 0x7f6e341388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e3259c700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e340b3e50 0x7f6e340cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e31d9b700 1 -- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e34144570 msgr2=0x7f6e341388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e31d9b700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e34144570 0x7f6e341388c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e31d9b700 1 -- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e340b3e50 msgr2=0x7f6e340cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e31d9b700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e340b3e50 0x7f6e340cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e31d9b700 1 -- 172.21.15.112:0/4062681406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c009a70 con 0x7f6e340b4ad0 2023-08-27T11:26:42.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e32d9d700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e34144570 0x7f6e341388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:42.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e3259c700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e340b3e50 0x7f6e340cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:42.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e31d9b700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e340b4ad0 0x7f6e340708a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009130 tx=0x7f6e24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:42.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e2affd700 1 -- 172.21.15.112:0/4062681406 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240036a0 con 0x7f6e340b4ad0 2023-08-27T11:26:42.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e2affd700 1 -- 172.21.15.112:0/4062681406 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2401e070 con 0x7f6e340b4ad0 2023-08-27T11:26:42.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e38b73700 1 -- 172.21.15.112:0/4062681406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e24009550 con 0x7f6e340b4ad0 2023-08-27T11:26:42.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e2affd700 1 -- 172.21.15.112:0/4062681406 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24003a60 con 0x7f6e340b4ad0 2023-08-27T11:26:42.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.483+0000 7f6e38b73700 1 -- 172.21.15.112:0/4062681406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e34071590 con 0x7f6e340b4ad0 2023-08-27T11:26:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.484+0000 7f6e38b73700 1 -- 172.21.15.112:0/4062681406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e34072030 con 0x7f6e340b4ad0 2023-08-27T11:26:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.485+0000 7f6e2affd700 1 -- 172.21.15.112:0/4062681406 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f6e24003bc0 con 0x7f6e340b4ad0 2023-08-27T11:26:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.485+0000 7f6e2affd700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6e1406a490 0x7f6e1406c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.485+0000 7f6e2affd700 1 -- 172.21.15.112:0/4062681406 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6e240898e0 con 0x7f6e340b4ad0 2023-08-27T11:26:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.485+0000 7f6e3259c700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6e1406a490 0x7f6e1406c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.486+0000 7f6e3259c700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6e1406a490 0x7f6e1406c950 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000f60 tx=0x7f6e1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.489+0000 7f6e2affd700 1 -- 172.21.15.112:0/4062681406 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6e24054d50 con 0x7f6e340b4ad0 2023-08-27T11:26:42.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.724+0000 7f6e38b73700 1 -- 172.21.15.112:0/4062681406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e34067a20 con 0x7f6e340b4ad0 2023-08-27T11:26:42.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.724+0000 7f6e2affd700 1 -- 172.21.15.112:0/4062681406 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4886 (secure 0 0 0) 0x7f6e24058370 con 0x7f6e340b4ad0 2023-08-27T11:26:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/120353149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":120353149},{"type":"v1","addr":"172.21.15.138:6836","nonce":120353149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1171600247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1171600247},{"type":"v1","addr":"172.21.15.138:6837","nonce":1171600247}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3930299696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3930299696},{"type":"v1","addr":"172.21.15.112:6834","nonce":3930299696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:26:18.805364+0000","modified":"2023-08-27T11:26:22.433680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7041":{"gid":7041,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2614833373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2614833373},{"type":"v1","addr":"172.21.15.112:6835","nonce":2614833373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-27T11:26:42.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.726+0000 7f6e38b73700 1 -- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6e1406a490 msgr2=0x7f6e1406c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.726+0000 7f6e38b73700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6e1406a490 0x7f6e1406c950 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000f60 tx=0x7f6e1c009040 comp rx=0 tx=0).stop 2023-08-27T11:26:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.726+0000 7f6e38b73700 1 -- 172.21.15.112:0/4062681406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e340b4ad0 msgr2=0x7f6e340708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.726+0000 7f6e38b73700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e340b4ad0 0x7f6e340708a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009130 tx=0x7f6e24004750 comp rx=0 tx=0).stop 2023-08-27T11:26:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.726+0000 7f6e38b73700 1 -- 172.21.15.112:0/4062681406 shutdown_connections 2023-08-27T11:26:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.726+0000 7f6e38b73700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6e1406a490 0x7f6e1406c950 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.726+0000 7f6e38b73700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e34144570 0x7f6e341388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.726+0000 7f6e38b73700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e340b3e50 0x7f6e340cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.726+0000 7f6e38b73700 1 --2- 172.21.15.112:0/4062681406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e340b4ad0 0x7f6e340708a0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.726+0000 7f6e38b73700 1 -- 172.21.15.112:0/4062681406 >> 172.21.15.112:0/4062681406 conn(0x7f6e3406f970 msgr2=0x7f6e340cef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:42.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.726+0000 7f6e38b73700 1 -- 172.21.15.112:0/4062681406 shutdown_connections 2023-08-27T11:26:42.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.727+0000 7f6e38b73700 1 -- 172.21.15.112:0/4062681406 wait complete. 2023-08-27T11:26:42.733 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 59 2023-08-27T11:26:42.740 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:42.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.859+0000 7f38036b8700 1 Processor -- start 2023-08-27T11:26:42.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.859+0000 7f38036b8700 1 -- start start 2023-08-27T11:26:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f38036b8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f37fc0b17e0 0x7f37fc0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f38036b8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc161850 con 0x7f37fc161ce0 2023-08-27T11:26:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f38036b8700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc0cfc10 con 0x7f37fc0b1400 2023-08-27T11:26:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f38036b8700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37fc161200 con 0x7f37fc0b17e0 2023-08-27T11:26:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f3801c55700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f37fc161ce0 0x7f37fc06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56116/0 (socket says 172.21.15.112:56116) 2023-08-27T11:26:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f3801c55700 1 -- 172.21.15.112:0/3530807270 learned_addr learned my addr 172.21.15.112:0/3530807270 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f3800c53700 1 --2- 172.21.15.112:0/3530807270 >> v2:172.21.15.138:3300/0 conn(0x7f37fc0b17e0 0x7f37fc0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798034331 0 0) 0x7f37fc161850 con 0x7f37fc161ce0 2023-08-27T11:26:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e80035b0 con 0x7f37fc161ce0 2023-08-27T11:26:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2502141364 0 0) 0x7f37e80035b0 con 0x7f37fc161ce0 2023-08-27T11:26:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 >> v2:172.21.15.138:3300/0 conn(0x7f37fc0b17e0 msgr2=0x7f37fc0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 --2- 172.21.15.112:0/3530807270 >> v2:172.21.15.138:3300/0 conn(0x7f37fc0b17e0 0x7f37fc0cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 >> v1:172.21.15.138:6789/0 conn(0x7f37fc0b1400 legacy=0x7f37fc06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc1614e0 con 0x7f37fc161ce0 2023-08-27T11:26:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f37f40019f0 con 0x7f37fc161ce0 2023-08-27T11:26:42.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 >> v1:172.21.15.112:6789/0 conn(0x7f37fc161ce0 legacy=0x7f37fc06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 --2- 172.21.15.112:0/3530807270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e8003a00 0x7f37e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 --2- 172.21.15.112:0/3530807270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37e8004850 0x7f37e8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 --2- 172.21.15.112:0/3530807270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e8005330 0x7f37e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc161200 con 0x7f37e8005330 2023-08-27T11:26:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e800c200 con 0x7f37e8003a00 2023-08-27T11:26:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e800c3c0 con 0x7f37e8004850 2023-08-27T11:26:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.860+0000 7f3801c55700 1 --2- 172.21.15.112:0/3530807270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e8005330 0x7f37e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f3801454700 1 --2- 172.21.15.112:0/3530807270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e8003a00 0x7f37e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f3800c53700 1 --2- 172.21.15.112:0/3530807270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37e8004850 0x7f37e8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f3801c55700 1 -- 172.21.15.112:0/3530807270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37e8004850 msgr2=0x7f37e8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f3801c55700 1 --2- 172.21.15.112:0/3530807270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37e8004850 0x7f37e8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f3801c55700 1 -- 172.21.15.112:0/3530807270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e8003a00 msgr2=0x7f37e8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f3801c55700 1 --2- 172.21.15.112:0/3530807270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e8003a00 0x7f37e8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f3801c55700 1 -- 172.21.15.112:0/3530807270 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e80035b0 con 0x7f37e8005330 2023-08-27T11:26:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f3801454700 1 --2- 172.21.15.112:0/3530807270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e8003a00 0x7f37e8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f3801c55700 1 -- 172.21.15.112:0/3530807270 reap_dead start 2023-08-27T11:26:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f3801c55700 1 --2- 172.21.15.112:0/3530807270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e8005330 0x7f37e800ba10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f37f400a800 tx=0x7f37e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43e30c46cf16c60 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f400a2f0 con 0x7f37e8005330 2023-08-27T11:26:42.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f4012600 con 0x7f37e8005330 2023-08-27T11:26:42.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f37f3fff700 1 -- 172.21.15.112:0/3530807270 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f40129b0 con 0x7f37e8005330 2023-08-27T11:26:42.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f38036b8700 1 -- 172.21.15.112:0/3530807270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e8005330 msgr2=0x7f37e800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f38036b8700 1 --2- 172.21.15.112:0/3530807270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e8005330 0x7f37e800ba10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f37f400a800 tx=0x7f37e8005880 comp rx=0 tx=0).stop 2023-08-27T11:26:42.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f38036b8700 1 -- 172.21.15.112:0/3530807270 shutdown_connections 2023-08-27T11:26:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.861+0000 7f38036b8700 1 --2- 172.21.15.112:0/3530807270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e8005330 0x7f37e800ba10 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.862+0000 7f38036b8700 1 -- 172.21.15.112:0/3530807270 >> 172.21.15.112:0/3530807270 conn(0x7f37fc0d0000 msgr2=0x7f37fc0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.862+0000 7f38036b8700 1 -- 172.21.15.112:0/3530807270 shutdown_connections 2023-08-27T11:26:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.862+0000 7f38036b8700 1 -- 172.21.15.112:0/3530807270 wait complete. 2023-08-27T11:26:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.863+0000 7f38036b8700 1 Processor -- start 2023-08-27T11:26:42.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.863+0000 7f38036b8700 1 -- start start 2023-08-27T11:26:42.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.863+0000 7f38036b8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e8005330 0x7f37fc0ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.863+0000 7f38036b8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37fc0c9ab0 0x7f37fc15ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.863+0000 7f38036b8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37fc154e30 0x7f37fc1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.863+0000 7f38036b8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc1f6980 con 0x7f37fc0c9ab0 2023-08-27T11:26:42.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.863+0000 7f38036b8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc1557f0 con 0x7f37e8005330 2023-08-27T11:26:42.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.863+0000 7f38036b8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc1f2a80 con 0x7f37fc154e30 2023-08-27T11:26:42.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f3800c53700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37fc0c9ab0 0x7f37fc15ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f3800c53700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37fc0c9ab0 0x7f37fc15ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33512/0 (socket says 172.21.15.112:33512) 2023-08-27T11:26:42.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f3800c53700 1 -- 172.21.15.112:0/3891397359 learned_addr learned my addr 172.21.15.112:0/3891397359 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:42.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f3801454700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e8005330 0x7f37fc0ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f3801c55700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37fc154e30 0x7f37fc1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f3800c53700 1 -- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37fc154e30 msgr2=0x7f37fc1552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f3800c53700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37fc154e30 0x7f37fc1552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f3800c53700 1 -- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e8005330 msgr2=0x7f37fc0ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:42.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f3800c53700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e8005330 0x7f37fc0ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:42.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f3800c53700 1 -- 172.21.15.112:0/3891397359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc1f6090 con 0x7f37fc0c9ab0 2023-08-27T11:26:42.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f3800c53700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37fc0c9ab0 0x7f37fc15ef20 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f37f8010920 tx=0x7f37f8010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:42.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f37f1ffb700 1 -- 172.21.15.112:0/3891397359 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f800e8d0 con 0x7f37fc0c9ab0 2023-08-27T11:26:42.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f37f1ffb700 1 -- 172.21.15.112:0/3891397359 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f800ea30 con 0x7f37fc0c9ab0 2023-08-27T11:26:42.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f38036b8700 1 -- 172.21.15.112:0/3891397359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc1f6e10 con 0x7f37fc0c9ab0 2023-08-27T11:26:42.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f37f1ffb700 1 -- 172.21.15.112:0/3891397359 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f800ede0 con 0x7f37fc0c9ab0 2023-08-27T11:26:42.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.864+0000 7f38036b8700 1 -- 172.21.15.112:0/3891397359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37fc1f6610 con 0x7f37fc0c9ab0 2023-08-27T11:26:42.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.865+0000 7f38036b8700 1 -- 172.21.15.112:0/3891397359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37fc1f6b10 con 0x7f37fc0c9ab0 2023-08-27T11:26:42.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.866+0000 7f37f1ffb700 1 -- 172.21.15.112:0/3891397359 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f37f8016460 con 0x7f37fc0c9ab0 2023-08-27T11:26:42.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.866+0000 7f37f1ffb700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f37e006a440 0x7f37e006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:42.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.866+0000 7f37f1ffb700 1 -- 172.21.15.112:0/3891397359 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f37f8088dc0 con 0x7f37fc0c9ab0 2023-08-27T11:26:42.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.866+0000 7f3801454700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f37e006a440 0x7f37e006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:42.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.867+0000 7f3801454700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f37e006a440 0x7f37e006c900 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f37ec000c00 tx=0x7f37ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:42.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:42.871+0000 7f37f1ffb700 1 -- 172.21.15.112:0/3891397359 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f37f80540a0 con 0x7f37fc0c9ab0 2023-08-27T11:26:43.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.060+0000 7f38036b8700 1 -- 172.21.15.112:0/3891397359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37fc067a20 con 0x7f37fc0c9ab0 2023-08-27T11:26:43.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.061+0000 7f37f1ffb700 1 -- 172.21.15.112:0/3891397359 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+18248 (secure 0 0 0) 0x7f37f80576c0 con 0x7f37fc0c9ab0 2023-08-27T11:26:43.063 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:43.063 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:20.416307+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:16.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:17.773993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:43.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.063+0000 7f38036b8700 1 -- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f37e006a440 msgr2=0x7f37e006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:43.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.063+0000 7f38036b8700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f37e006a440 0x7f37e006c900 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f37ec000c00 tx=0x7f37ec009040 comp rx=0 tx=0).stop 2023-08-27T11:26:43.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.063+0000 7f38036b8700 1 -- 172.21.15.112:0/3891397359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37fc0c9ab0 msgr2=0x7f37fc15ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:43.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.063+0000 7f38036b8700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37fc0c9ab0 0x7f37fc15ef20 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f37f8010920 tx=0x7f37f8010ce0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.063+0000 7f38036b8700 1 -- 172.21.15.112:0/3891397359 shutdown_connections 2023-08-27T11:26:43.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.063+0000 7f38036b8700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f37e006a440 0x7f37e006c900 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.063+0000 7f38036b8700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37fc154e30 0x7f37fc1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.063+0000 7f38036b8700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e8005330 0x7f37fc0ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.063+0000 7f38036b8700 1 --2- 172.21.15.112:0/3891397359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37fc0c9ab0 0x7f37fc15ef20 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.063+0000 7f38036b8700 1 -- 172.21.15.112:0/3891397359 >> 172.21.15.112:0/3891397359 conn(0x7f37fc0d0000 msgr2=0x7f37fc071e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.063+0000 7f38036b8700 1 -- 172.21.15.112:0/3891397359 shutdown_connections 2023-08-27T11:26:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.063+0000 7f38036b8700 1 -- 172.21.15.112:0/3891397359 wait complete. 2023-08-27T11:26:43.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:26:43.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.197+0000 7f47d0003700 1 Processor -- start 2023-08-27T11:26:43.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.197+0000 7f47d0003700 1 -- start start 2023-08-27T11:26:43.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47d0003700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f47c8144570 0x7f47c815ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47d0003700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c80b3e50 0x7f47c815f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47d0003700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f47c815f8a0 0x7f47c80ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47d0003700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c8144950 con 0x7f47c8144570 2023-08-27T11:26:43.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47d0003700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c80b45f0 con 0x7f47c815f8a0 2023-08-27T11:26:43.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47d0003700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c80b4310 con 0x7f47c80b3e50 2023-08-27T11:26:43.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47cdd9f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f47c8144570 0x7f47c815ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47cdd9f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f47c8144570 0x7f47c815ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33518/0 (socket says 172.21.15.112:33518) 2023-08-27T11:26:43.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47cdd9f700 1 -- 172.21.15.112:0/2436930796 learned_addr learned my addr 172.21.15.112:0/2436930796 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:43.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47ce5a0700 1 --2- 172.21.15.112:0/2436930796 >> v2:172.21.15.138:3300/0 conn(0x7f47c815f8a0 0x7f47c80ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47cd59e700 1 --2- 172.21.15.112:0/2436930796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c80b3e50 0x7f47c815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47cdd9f700 1 -- 172.21.15.112:0/2436930796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c80b3e50 msgr2=0x7f47c815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:43.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47cdd9f700 1 --2- 172.21.15.112:0/2436930796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c80b3e50 0x7f47c815f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47cdd9f700 1 -- 172.21.15.112:0/2436930796 >> v2:172.21.15.138:3300/0 conn(0x7f47c815f8a0 msgr2=0x7f47c80ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:43.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47cdd9f700 1 --2- 172.21.15.112:0/2436930796 >> v2:172.21.15.138:3300/0 conn(0x7f47c815f8a0 0x7f47c80ce790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47cdd9f700 1 -- 172.21.15.112:0/2436930796 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c80cdcb0 con 0x7f47c8144570 2023-08-27T11:26:43.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47cdd9f700 1 --2- 172.21.15.112:0/2436930796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c8144570 0x7f47c815ee20 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f47c806e610 tx=0x7f47c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9680dde7738b35 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:43.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47ccd9d700 1 -- 172.21.15.112:0/2436930796 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c400b870 con 0x7f47c8144570 2023-08-27T11:26:43.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47ccd9d700 1 -- 172.21.15.112:0/2436930796 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c400b9d0 con 0x7f47c8144570 2023-08-27T11:26:43.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.198+0000 7f47ccd9d700 1 -- 172.21.15.112:0/2436930796 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c400bd60 con 0x7f47c8144570 2023-08-27T11:26:43.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.199+0000 7f47d0003700 1 -- 172.21.15.112:0/2436930796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c8144570 msgr2=0x7f47c815ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:43.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.199+0000 7f47d0003700 1 --2- 172.21.15.112:0/2436930796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c8144570 0x7f47c815ee20 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f47c806e610 tx=0x7f47c4009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.199+0000 7f47d0003700 1 -- 172.21.15.112:0/2436930796 shutdown_connections 2023-08-27T11:26:43.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.199+0000 7f47d0003700 1 --2- 172.21.15.112:0/2436930796 >> v2:172.21.15.138:3300/0 conn(0x7f47c815f8a0 0x7f47c80ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.199+0000 7f47d0003700 1 --2- 172.21.15.112:0/2436930796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c8144570 0x7f47c815ee20 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.199+0000 7f47d0003700 1 --2- 172.21.15.112:0/2436930796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c80b3e50 0x7f47c815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.199+0000 7f47d0003700 1 -- 172.21.15.112:0/2436930796 >> 172.21.15.112:0/2436930796 conn(0x7f47c806f970 msgr2=0x7f47c806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.199+0000 7f47d0003700 1 -- 172.21.15.112:0/2436930796 shutdown_connections 2023-08-27T11:26:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.199+0000 7f47d0003700 1 -- 172.21.15.112:0/2436930796 wait complete. 2023-08-27T11:26:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.200+0000 7f47d0003700 1 Processor -- start 2023-08-27T11:26:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.200+0000 7f47d0003700 1 -- start start 2023-08-27T11:26:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47d0003700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c80b3e50 0x7f47c81f0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47d0003700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c8144570 0x7f47c8143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47d0003700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c815f8a0 0x7f47c80b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47d0003700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c81f08d0 con 0x7f47c815f8a0 2023-08-27T11:26:43.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47d0003700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c81436c0 con 0x7f47c80b3e50 2023-08-27T11:26:43.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47d0003700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c81f4520 con 0x7f47c8144570 2023-08-27T11:26:43.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47ce5a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c815f8a0 0x7f47c80b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47ce5a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c815f8a0 0x7f47c80b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33528/0 (socket says 172.21.15.112:33528) 2023-08-27T11:26:43.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47ce5a0700 1 -- 172.21.15.112:0/4175779723 learned_addr learned my addr 172.21.15.112:0/4175779723 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:43.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47cdd9f700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c80b3e50 0x7f47c81f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47cd59e700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c8144570 0x7f47c8143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47ce5a0700 1 -- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c8144570 msgr2=0x7f47c8143180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47ce5a0700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c8144570 0x7f47c8143180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47ce5a0700 1 -- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c80b3e50 msgr2=0x7f47c81f0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47ce5a0700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c80b3e50 0x7f47c81f0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.201+0000 7f47ce5a0700 1 -- 172.21.15.112:0/4175779723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c4009550 con 0x7f47c815f8a0 2023-08-27T11:26:43.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.202+0000 7f47ce5a0700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c815f8a0 0x7f47c80b47a0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f47c000cc30 tx=0x7f47c000a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:43.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.202+0000 7f47be7fc700 1 -- 172.21.15.112:0/4175779723 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c000ab70 con 0x7f47c815f8a0 2023-08-27T11:26:43.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.202+0000 7f47be7fc700 1 -- 172.21.15.112:0/4175779723 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c000acd0 con 0x7f47c815f8a0 2023-08-27T11:26:43.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.202+0000 7f47d0003700 1 -- 172.21.15.112:0/4175779723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c81f4800 con 0x7f47c815f8a0 2023-08-27T11:26:43.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.202+0000 7f47be7fc700 1 -- 172.21.15.112:0/4175779723 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0003c40 con 0x7f47c815f8a0 2023-08-27T11:26:43.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.202+0000 7f47d0003700 1 -- 172.21.15.112:0/4175779723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c81f4dc0 con 0x7f47c815f8a0 2023-08-27T11:26:43.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.203+0000 7f47d0003700 1 -- 172.21.15.112:0/4175779723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c8067a20 con 0x7f47c815f8a0 2023-08-27T11:26:43.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.203+0000 7f47be7fc700 1 -- 172.21.15.112:0/4175779723 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f47c000f3f0 con 0x7f47c815f8a0 2023-08-27T11:26:43.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.204+0000 7f47be7fc700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f47ac06a440 0x7f47ac06c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.204+0000 7f47be7fc700 1 -- 172.21.15.112:0/4175779723 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47c0088ea0 con 0x7f47c815f8a0 2023-08-27T11:26:43.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.204+0000 7f47cdd9f700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f47ac06a440 0x7f47ac06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.204+0000 7f47cdd9f700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f47ac06a440 0x7f47ac06c900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f47c4000c00 tx=0x7f47c4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:43.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.208+0000 7f47be7fc700 1 -- 172.21.15.112:0/4175779723 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f47c0053f70 con 0x7f47c815f8a0 2023-08-27T11:26:43.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.443+0000 7f47d0003700 1 -- 172.21.15.112:0/4175779723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f47c815b2e0 con 0x7f47c815f8a0 2023-08-27T11:26:43.445 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:26:43.444+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:26:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.448+0000 7f47be7fc700 1 -- 172.21.15.112:0/4175779723 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) v1 ==== 161+0+0 (secure 0 0 0) 0x7f47c001d020 con 0x7f47c815f8a0 2023-08-27T11:26:43.452 INFO:tasks.ceph.mds.c.smithi112.stderr: -3541> 2023-08-27T11:26:24.388+0000 7f4c64695700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:26:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.451+0000 7f47d0003700 1 -- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f47ac06a440 msgr2=0x7f47ac06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.451+0000 7f47d0003700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f47ac06a440 0x7f47ac06c900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f47c4000c00 tx=0x7f47c4008030 comp rx=0 tx=0).stop 2023-08-27T11:26:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.451+0000 7f47d0003700 1 -- 172.21.15.112:0/4175779723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c815f8a0 msgr2=0x7f47c80b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.451+0000 7f47d0003700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c815f8a0 0x7f47c80b47a0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f47c000cc30 tx=0x7f47c000a1f0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.451+0000 7f47d0003700 1 -- 172.21.15.112:0/4175779723 shutdown_connections 2023-08-27T11:26:43.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.451+0000 7f47d0003700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f47ac06a440 0x7f47ac06c900 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.451+0000 7f47d0003700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c815f8a0 0x7f47c80b47a0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.451+0000 7f47d0003700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c80b3e50 0x7f47c81f0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.451+0000 7f47d0003700 1 --2- 172.21.15.112:0/4175779723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c8144570 0x7f47c8143180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.451+0000 7f47d0003700 1 -- 172.21.15.112:0/4175779723 >> 172.21.15.112:0/4175779723 conn(0x7f47c806f970 msgr2=0x7f47c815edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:43.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.451+0000 7f47d0003700 1 -- 172.21.15.112:0/4175779723 shutdown_connections 2023-08-27T11:26:43.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.451+0000 7f47d0003700 1 -- 172.21.15.112:0/4175779723 wait complete. 2023-08-27T11:26:43.456 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:26:43.467 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:26:43.484 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:26:43.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.589+0000 7f1fd7e96700 1 Processor -- start 2023-08-27T11:26:43.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.589+0000 7f1fd7e96700 1 -- start start 2023-08-27T11:26:43.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd7e96700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1fd0071980 0x7f1fd00cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd7e96700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd00cd2e0 con 0x7f1fd00ccea0 2023-08-27T11:26:43.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd7e96700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd015b510 con 0x7f1fd015ea00 2023-08-27T11:26:43.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd7e96700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd015e5e0 con 0x7f1fd0071980 2023-08-27T11:26:43.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd6433700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1fd00ccea0 0x7f1fd00ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56162/0 (socket says 172.21.15.112:56162) 2023-08-27T11:26:43.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd6433700 1 -- 172.21.15.112:0/1781637945 learned_addr learned my addr 172.21.15.112:0/1781637945 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:43.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833220842 0 0) 0x7f1fd00cd2e0 con 0x7f1fd00ccea0 2023-08-27T11:26:43.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd5c32700 1 --2- 172.21.15.112:0/1781637945 >> v2:172.21.15.138:3300/0 conn(0x7f1fd0071980 0x7f1fd00cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fbc0035b0 con 0x7f1fd00ccea0 2023-08-27T11:26:43.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3513685449 0 0) 0x7f1fbc0035b0 con 0x7f1fd00ccea0 2023-08-27T11:26:43.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 >> v2:172.21.15.138:3300/0 conn(0x7f1fd0071980 msgr2=0x7f1fd00cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:43.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd4c30700 1 --2- 172.21.15.112:0/1781637945 >> v2:172.21.15.138:3300/0 conn(0x7f1fd0071980 0x7f1fd00cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 >> v1:172.21.15.138:6789/0 conn(0x7f1fd015ea00 legacy=0x7f1fd015a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:43.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd015df20 con 0x7f1fd00ccea0 2023-08-27T11:26:43.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f1fc80019f0 con 0x7f1fd00ccea0 2023-08-27T11:26:43.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 >> v1:172.21.15.112:6789/0 conn(0x7f1fd00ccea0 legacy=0x7f1fd00ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:43.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd4c30700 1 --2- 172.21.15.112:0/1781637945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc003a00 0x7f1fbc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd4c30700 1 --2- 172.21.15.112:0/1781637945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fbc004850 0x7f1fbc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.590+0000 7f1fd4c30700 1 --2- 172.21.15.112:0/1781637945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc005410 0x7f1fbc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd015e5e0 con 0x7f1fbc004850 2023-08-27T11:26:43.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc00c2a0 con 0x7f1fbc003a00 2023-08-27T11:26:43.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc00e670 con 0x7f1fbc005410 2023-08-27T11:26:43.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd5431700 1 --2- 172.21.15.112:0/1781637945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fbc004850 0x7f1fbc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd5c32700 1 --2- 172.21.15.112:0/1781637945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc003a00 0x7f1fbc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd6433700 1 --2- 172.21.15.112:0/1781637945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc005410 0x7f1fbc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd5431700 1 -- 172.21.15.112:0/1781637945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc005410 msgr2=0x7f1fbc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:43.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd5431700 1 --2- 172.21.15.112:0/1781637945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc005410 0x7f1fbc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd5431700 1 -- 172.21.15.112:0/1781637945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc003a00 msgr2=0x7f1fbc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:43.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd5431700 1 --2- 172.21.15.112:0/1781637945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc003a00 0x7f1fbc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd5431700 1 -- 172.21.15.112:0/1781637945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc0035b0 con 0x7f1fbc004850 2023-08-27T11:26:43.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd6433700 1 --2- 172.21.15.112:0/1781637945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc005410 0x7f1fbc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd6433700 1 -- 172.21.15.112:0/1781637945 reap_dead start 2023-08-27T11:26:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd5431700 1 --2- 172.21.15.112:0/1781637945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fbc004850 0x7f1fbc004cd0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00017c0 tx=0x7f1fbc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb093b6c4a3231f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00052d0 con 0x7f1fbc004850 2023-08-27T11:26:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc0005430 con 0x7f1fbc004850 2023-08-27T11:26:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.591+0000 7f1fd4c30700 1 -- 172.21.15.112:0/1781637945 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00038c0 con 0x7f1fbc004850 2023-08-27T11:26:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.592+0000 7f1fd7e96700 1 -- 172.21.15.112:0/1781637945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fbc004850 msgr2=0x7f1fbc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.592+0000 7f1fd7e96700 1 --2- 172.21.15.112:0/1781637945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fbc004850 0x7f1fbc004cd0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00017c0 tx=0x7f1fbc005880 comp rx=0 tx=0).stop 2023-08-27T11:26:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.592+0000 7f1fd7e96700 1 -- 172.21.15.112:0/1781637945 shutdown_connections 2023-08-27T11:26:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.592+0000 7f1fd7e96700 1 --2- 172.21.15.112:0/1781637945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fbc004850 0x7f1fbc004cd0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.592+0000 7f1fd7e96700 1 -- 172.21.15.112:0/1781637945 >> 172.21.15.112:0/1781637945 conn(0x7f1fd006bbf0 msgr2=0x7f1fd00cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:43.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.592+0000 7f1fd7e96700 1 -- 172.21.15.112:0/1781637945 shutdown_connections 2023-08-27T11:26:43.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.592+0000 7f1fd7e96700 1 -- 172.21.15.112:0/1781637945 wait complete. 2023-08-27T11:26:43.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.593+0000 7f1fd7e96700 1 Processor -- start 2023-08-27T11:26:43.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.593+0000 7f1fd7e96700 1 -- start start 2023-08-27T11:26:43.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd7e96700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc004850 0x7f1fd0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd7e96700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fd01f3e60 0x7f1fd015bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd7e96700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd014ec20 0x7f1fd015c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd7e96700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd015c040 con 0x7f1fd014ec20 2023-08-27T11:26:43.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd7e96700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd01596f0 con 0x7f1fd01f3e60 2023-08-27T11:26:43.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd7e96700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd01599d0 con 0x7f1fbc004850 2023-08-27T11:26:43.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd6433700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd014ec20 0x7f1fd015c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd6433700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd014ec20 0x7f1fd015c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33560/0 (socket says 172.21.15.112:33560) 2023-08-27T11:26:43.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd6433700 1 -- 172.21.15.112:0/4102914975 learned_addr learned my addr 172.21.15.112:0/4102914975 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:43.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd5c32700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc004850 0x7f1fd0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd5431700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fd01f3e60 0x7f1fd015bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd6433700 1 -- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc004850 msgr2=0x7f1fd0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:43.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd6433700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc004850 0x7f1fd0158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd6433700 1 -- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fd01f3e60 msgr2=0x7f1fd015bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:43.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd6433700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fd01f3e60 0x7f1fd015bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:43.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd6433700 1 -- 172.21.15.112:0/4102914975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd0152c70 con 0x7f1fd014ec20 2023-08-27T11:26:43.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd6433700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd014ec20 0x7f1fd015c1f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00b6200 tx=0x7f1fc800e4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:43.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fc67fc700 1 -- 172.21.15.112:0/4102914975 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800ed80 con 0x7f1fd014ec20 2023-08-27T11:26:43.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fc67fc700 1 -- 172.21.15.112:0/4102914975 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc8013420 con 0x7f1fd014ec20 2023-08-27T11:26:43.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.594+0000 7f1fd7e96700 1 -- 172.21.15.112:0/4102914975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd0152f50 con 0x7f1fd014ec20 2023-08-27T11:26:43.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.595+0000 7f1fc67fc700 1 -- 172.21.15.112:0/4102914975 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc80137d0 con 0x7f1fd014ec20 2023-08-27T11:26:43.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.595+0000 7f1fd7e96700 1 -- 172.21.15.112:0/4102914975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd014f0b0 con 0x7f1fd014ec20 2023-08-27T11:26:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.595+0000 7f1fd7e96700 1 -- 172.21.15.112:0/4102914975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd0066fa0 con 0x7f1fd014ec20 2023-08-27T11:26:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.596+0000 7f1fc67fc700 1 -- 172.21.15.112:0/4102914975 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f1fc8013930 con 0x7f1fd014ec20 2023-08-27T11:26:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.596+0000 7f1fc67fc700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1fb406a440 0x7f1fb406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.596+0000 7f1fc67fc700 1 -- 172.21.15.112:0/4102914975 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f1fc8089170 con 0x7f1fd014ec20 2023-08-27T11:26:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.597+0000 7f1fd5c32700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1fb406a440 0x7f1fb406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:43.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.597+0000 7f1fd5c32700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1fb406a440 0x7f1fb406c900 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1fd01f61f0 tx=0x7f1fcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:43.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.601+0000 7f1fc67fc700 1 -- 172.21.15.112:0/4102914975 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1fc8054480 con 0x7f1fd014ec20 2023-08-27T11:26:43.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:43.833+0000 7f1fd7e96700 1 -- 172.21.15.112:0/4102914975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1fd0159830 con 0x7f1fd014ec20 2023-08-27T11:26:44.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.448+0000 7f1fc67fc700 1 -- 172.21.15.112:0/4102914975 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1fd0159830 con 0x7f1fd014ec20 2023-08-27T11:26:44.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.451+0000 7f1fd7e96700 1 -- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1fb406a440 msgr2=0x7f1fb406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:44.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.451+0000 7f1fd7e96700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1fb406a440 0x7f1fb406c900 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1fd01f61f0 tx=0x7f1fcc008040 comp rx=0 tx=0).stop 2023-08-27T11:26:44.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.451+0000 7f1fd7e96700 1 -- 172.21.15.112:0/4102914975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd014ec20 msgr2=0x7f1fd015c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:44.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.451+0000 7f1fd7e96700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd014ec20 0x7f1fd015c1f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00b6200 tx=0x7f1fc800e4f0 comp rx=0 tx=0).stop 2023-08-27T11:26:44.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.451+0000 7f1fd7e96700 1 -- 172.21.15.112:0/4102914975 shutdown_connections 2023-08-27T11:26:44.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.451+0000 7f1fd7e96700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1fb406a440 0x7f1fb406c900 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:44.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.451+0000 7f1fd7e96700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd014ec20 0x7f1fd015c1f0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:44.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.451+0000 7f1fd7e96700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc004850 0x7f1fd0158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:44.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.451+0000 7f1fd7e96700 1 --2- 172.21.15.112:0/4102914975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fd01f3e60 0x7f1fd015bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:44.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.451+0000 7f1fd7e96700 1 -- 172.21.15.112:0/4102914975 >> 172.21.15.112:0/4102914975 conn(0x7f1fd006bbf0 msgr2=0x7f1fd0156e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:44.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.451+0000 7f1fd7e96700 1 -- 172.21.15.112:0/4102914975 shutdown_connections 2023-08-27T11:26:44.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.451+0000 7f1fd7e96700 1 -- 172.21.15.112:0/4102914975 wait complete. 2023-08-27T11:26:44.466 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:26:44.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.632+0000 7f6d1655a700 1 Processor -- start 2023-08-27T11:26:44.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.632+0000 7f6d1655a700 1 -- start start 2023-08-27T11:26:44.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d1655a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6d10157ae0 0x7f6d1015e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d1655a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6d1006f790 0x7f6d1006fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d1655a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d10161390 con 0x7f6d100b3de0 2023-08-27T11:26:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d1655a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d100b1aa0 con 0x7f6d1006f790 2023-08-27T11:26:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d1655a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d10070160 con 0x7f6d10157ae0 2023-08-27T11:26:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d14af7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6d1006f790 0x7f6d1006fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d0f7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6d100b3de0 0x7f6d10161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56180/0 (socket says 172.21.15.112:56180) 2023-08-27T11:26:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d14af7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6d1006f790 0x7f6d1006fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33568/0 (socket says 172.21.15.112:33568) 2023-08-27T11:26:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d0f7fe700 1 -- 172.21.15.112:0/1775119038 learned_addr learned my addr 172.21.15.112:0/1775119038 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d14af7700 1 -- 172.21.15.112:0/1775119038 >> v2:172.21.15.138:3300/0 conn(0x7f6d10157ae0 msgr2=0x7f6d1015e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d14af7700 1 --2- 172.21.15.112:0/1775119038 >> v2:172.21.15.138:3300/0 conn(0x7f6d10157ae0 0x7f6d1015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:44.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d14af7700 1 -- 172.21.15.112:0/1775119038 >> v1:172.21.15.112:6789/0 conn(0x7f6d100b3de0 legacy=0x7f6d10161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:44.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d14af7700 1 -- 172.21.15.112:0/1775119038 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d100b35f0 con 0x7f6d1006f790 2023-08-27T11:26:44.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d0effd700 1 -- 172.21.15.112:0/1775119038 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110610675 0 0) 0x7f6d10161390 con 0x7f6d100b3de0 2023-08-27T11:26:44.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d14af7700 1 --2- 172.21.15.112:0/1775119038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d1006f790 0x7f6d1006fc20 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6d1006e580 tx=0x7f6d0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8876a38827fc34d5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:44.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.633+0000 7f6d0effd700 1 -- 172.21.15.112:0/1775119038 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0800e070 con 0x7f6d1006f790 2023-08-27T11:26:44.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.634+0000 7f6d0effd700 1 -- 172.21.15.112:0/1775119038 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0800bbd0 con 0x7f6d1006f790 2023-08-27T11:26:44.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.634+0000 7f6d0effd700 1 -- 172.21.15.112:0/1775119038 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08012430 con 0x7f6d1006f790 2023-08-27T11:26:44.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.634+0000 7f6d1655a700 1 -- 172.21.15.112:0/1775119038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d1006f790 msgr2=0x7f6d1006fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:44.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.634+0000 7f6d1655a700 1 --2- 172.21.15.112:0/1775119038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d1006f790 0x7f6d1006fc20 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6d1006e580 tx=0x7f6d0800b330 comp rx=0 tx=0).stop 2023-08-27T11:26:44.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.634+0000 7f6d1655a700 1 -- 172.21.15.112:0/1775119038 shutdown_connections 2023-08-27T11:26:44.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.634+0000 7f6d1655a700 1 --2- 172.21.15.112:0/1775119038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d1006f790 0x7f6d1006fc20 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:44.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.634+0000 7f6d1655a700 1 --2- 172.21.15.112:0/1775119038 >> v2:172.21.15.138:3300/0 conn(0x7f6d10157ae0 0x7f6d1015e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:44.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.634+0000 7f6d1655a700 1 -- 172.21.15.112:0/1775119038 >> 172.21.15.112:0/1775119038 conn(0x7f6d100b1be0 msgr2=0x7f6d100b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:44.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.634+0000 7f6d1655a700 1 -- 172.21.15.112:0/1775119038 shutdown_connections 2023-08-27T11:26:44.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.634+0000 7f6d1655a700 1 -- 172.21.15.112:0/1775119038 wait complete. 2023-08-27T11:26:44.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.635+0000 7f6d1655a700 1 Processor -- start 2023-08-27T11:26:44.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.635+0000 7f6d1655a700 1 -- start start 2023-08-27T11:26:44.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.636+0000 7f6d1655a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d1006f790 0x7f6d10157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:44.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.636+0000 7f6d1655a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d100b3de0 0x7f6d1015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:44.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.636+0000 7f6d1655a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d10157ae0 0x7f6d101f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:44.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.636+0000 7f6d1655a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d10158400 con 0x7f6d10157ae0 2023-08-27T11:26:44.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.636+0000 7f6d1655a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d10158660 con 0x7f6d100b3de0 2023-08-27T11:26:44.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.636+0000 7f6d1655a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d10158940 con 0x7f6d1006f790 2023-08-27T11:26:44.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.636+0000 7f6d14af7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d10157ae0 0x7f6d101f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:44.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.636+0000 7f6d14af7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d10157ae0 0x7f6d101f2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33574/0 (socket says 172.21.15.112:33574) 2023-08-27T11:26:44.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.636+0000 7f6d14af7700 1 -- 172.21.15.112:0/758684880 learned_addr learned my addr 172.21.15.112:0/758684880 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:44.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.636+0000 7f6d0f7fe700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d100b3de0 0x7f6d1015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:44.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.637+0000 7f6d0ffff700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d1006f790 0x7f6d10157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:44.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.637+0000 7f6d14af7700 1 -- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d1006f790 msgr2=0x7f6d10157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:44.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.637+0000 7f6d14af7700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d1006f790 0x7f6d10157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:44.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.637+0000 7f6d14af7700 1 -- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d100b3de0 msgr2=0x7f6d1015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:44.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.637+0000 7f6d14af7700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d100b3de0 0x7f6d1015a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.637+0000 7f6d14af7700 1 -- 172.21.15.112:0/758684880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d08009a70 con 0x7f6d10157ae0 2023-08-27T11:26:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.637+0000 7f6d0ffff700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d1006f790 0x7f6d10157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.637+0000 7f6d14af7700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d10157ae0 0x7f6d101f2cb0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6d08014040 tx=0x7f6d08004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.637+0000 7f6d0cff9700 1 -- 172.21.15.112:0/758684880 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08012430 con 0x7f6d10157ae0 2023-08-27T11:26:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.637+0000 7f6d0cff9700 1 -- 172.21.15.112:0/758684880 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d08020070 con 0x7f6d10157ae0 2023-08-27T11:26:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.637+0000 7f6d0cff9700 1 -- 172.21.15.112:0/758684880 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08004280 con 0x7f6d10157ae0 2023-08-27T11:26:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.637+0000 7f6d1655a700 1 -- 172.21.15.112:0/758684880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d10158c20 con 0x7f6d10157ae0 2023-08-27T11:26:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.637+0000 7f6d1655a700 1 -- 172.21.15.112:0/758684880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d101591e0 con 0x7f6d10157ae0 2023-08-27T11:26:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.638+0000 7f6d1655a700 1 -- 172.21.15.112:0/758684880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d10067a20 con 0x7f6d10157ae0 2023-08-27T11:26:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.638+0000 7f6d0cff9700 1 -- 172.21.15.112:0/758684880 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f6d080043e0 con 0x7f6d10157ae0 2023-08-27T11:26:44.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.639+0000 7f6d0cff9700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cfc06a440 0x7f6cfc06c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:44.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.639+0000 7f6d0cff9700 1 -- 172.21.15.112:0/758684880 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f6d08022070 con 0x7f6d10157ae0 2023-08-27T11:26:44.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.639+0000 7f6d0ffff700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cfc06a440 0x7f6cfc06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:44.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.640+0000 7f6d0ffff700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cfc06a440 0x7f6cfc06c900 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400a8f0 tx=0x7f6d04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:44.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.643+0000 7f6d0cff9700 1 -- 172.21.15.112:0/758684880 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6d08055d40 con 0x7f6d10157ae0 2023-08-27T11:26:44.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:44.834+0000 7f6d1655a700 1 -- 172.21.15.112:0/758684880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6d10160e60 con 0x7f6d10157ae0 2023-08-27T11:26:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.519+0000 7f6d0cff9700 1 -- 172.21.15.112:0/758684880 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6d08059360 con 0x7f6d10157ae0 2023-08-27T11:26:45.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.523+0000 7f6d1655a700 1 -- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cfc06a440 msgr2=0x7f6cfc06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:45.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.523+0000 7f6d1655a700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cfc06a440 0x7f6cfc06c900 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400a8f0 tx=0x7f6d04008040 comp rx=0 tx=0).stop 2023-08-27T11:26:45.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.523+0000 7f6d1655a700 1 -- 172.21.15.112:0/758684880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d10157ae0 msgr2=0x7f6d101f2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:45.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.523+0000 7f6d1655a700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d10157ae0 0x7f6d101f2cb0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6d08014040 tx=0x7f6d08004ad0 comp rx=0 tx=0).stop 2023-08-27T11:26:45.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.523+0000 7f6d1655a700 1 -- 172.21.15.112:0/758684880 shutdown_connections 2023-08-27T11:26:45.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.523+0000 7f6d1655a700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6cfc06a440 0x7f6cfc06c900 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:45.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.523+0000 7f6d1655a700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d10157ae0 0x7f6d101f2cb0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:45.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.523+0000 7f6d1655a700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d1006f790 0x7f6d10157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:45.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.523+0000 7f6d1655a700 1 --2- 172.21.15.112:0/758684880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d100b3de0 0x7f6d1015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:45.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.523+0000 7f6d1655a700 1 -- 172.21.15.112:0/758684880 >> 172.21.15.112:0/758684880 conn(0x7f6d100b1be0 msgr2=0x7f6d1015ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:45.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.523+0000 7f6d1655a700 1 -- 172.21.15.112:0/758684880 shutdown_connections 2023-08-27T11:26:45.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.523+0000 7f6d1655a700 1 -- 172.21.15.112:0/758684880 wait complete. 2023-08-27T11:26:45.527 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:26:45.537 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:26:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.660+0000 7f4b7a00d700 1 Processor -- start 2023-08-27T11:26:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.660+0000 7f4b7a00d700 1 -- start start 2023-08-27T11:26:45.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.661+0000 7f4b7a00d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7406e4c0 0x7f4b740acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:45.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.661+0000 7f4b7a00d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b74160fa0 con 0x7f4b740ad000 2023-08-27T11:26:45.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.661+0000 7f4b7a00d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b74146f40 con 0x7f4b74146b60 2023-08-27T11:26:45.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.661+0000 7f4b7a00d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b74161b70 con 0x7f4b7406e4c0 2023-08-27T11:26:45.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.661+0000 7f4b73fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4b740ad000 0x7f4b7415d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56196/0 (socket says 172.21.15.112:56196) 2023-08-27T11:26:45.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.661+0000 7f4b73fff700 1 -- 172.21.15.112:0/990930615 learned_addr learned my addr 172.21.15.112:0/990930615 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:45.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b72ffd700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7406e4c0 0x7f4b740acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:45.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264688321 0 0) 0x7f4b74160fa0 con 0x7f4b740ad000 2023-08-27T11:26:45.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b5c0035b0 con 0x7f4b740ad000 2023-08-27T11:26:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148188924 0 0) 0x7f4b68001110 con 0x7f4b74146b60 2023-08-27T11:26:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b74160fa0 con 0x7f4b74146b60 2023-08-27T11:26:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3315399518 0 0) 0x7f4b5c0035b0 con 0x7f4b740ad000 2023-08-27T11:26:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7406e4c0 msgr2=0x7f4b740acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7406e4c0 0x7f4b740acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 >> v1:172.21.15.138:6789/0 conn(0x7f4b74146b60 legacy=0x7f4b7406ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b741611d0 con 0x7f4b740ad000 2023-08-27T11:26:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f4b6c0019f0 con 0x7f4b740ad000 2023-08-27T11:26:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 >> v1:172.21.15.112:6789/0 conn(0x7f4b740ad000 legacy=0x7f4b7415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5c003bc0 0x7f4b5c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:45.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b5c004850 0x7f4b5c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:45.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7406e4c0 0x7f4b5c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:45.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b74161b70 con 0x7f4b5c004850 2023-08-27T11:26:45.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c00bac0 con 0x7f4b5c003bc0 2023-08-27T11:26:45.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c00bc50 con 0x7f4b7406e4c0 2023-08-27T11:26:45.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b72ffd700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b5c004850 0x7f4b5c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:45.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b737fe700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5c003bc0 0x7f4b5c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:45.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b73fff700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7406e4c0 0x7f4b5c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:45.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b72ffd700 1 -- 172.21.15.112:0/990930615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7406e4c0 msgr2=0x7f4b5c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:45.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b72ffd700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7406e4c0 0x7f4b5c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:45.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b72ffd700 1 -- 172.21.15.112:0/990930615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5c003bc0 msgr2=0x7f4b5c003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:45.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b72ffd700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5c003bc0 0x7f4b5c003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:45.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b72ffd700 1 -- 172.21.15.112:0/990930615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c0035b0 con 0x7f4b5c004850 2023-08-27T11:26:45.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b73fff700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7406e4c0 0x7f4b5c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:45.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.662+0000 7f4b737fe700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5c003bc0 0x7f4b5c003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.663+0000 7f4b72ffd700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b5c004850 0x7f4b5c004cd0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009b60 tx=0x7f4b5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695921338fc2a7b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.663+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400e6c0 con 0x7f4b5c004850 2023-08-27T11:26:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.663+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6400e820 con 0x7f4b5c004850 2023-08-27T11:26:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.663+0000 7f4b727fc700 1 -- 172.21.15.112:0/990930615 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400eb90 con 0x7f4b5c004850 2023-08-27T11:26:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.663+0000 7f4b7a00d700 1 -- 172.21.15.112:0/990930615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b5c004850 msgr2=0x7f4b5c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.663+0000 7f4b7a00d700 1 --2- 172.21.15.112:0/990930615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b5c004850 0x7f4b5c004cd0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009b60 tx=0x7f4b5c005880 comp rx=0 tx=0).stop 2023-08-27T11:26:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.663+0000 7f4b73fff700 1 -- 172.21.15.112:0/990930615 reap_dead start 2023-08-27T11:26:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.663+0000 7f4b7a00d700 1 -- 172.21.15.112:0/990930615 shutdown_connections 2023-08-27T11:26:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.663+0000 7f4b7a00d700 1 -- 172.21.15.112:0/990930615 >> 172.21.15.112:0/990930615 conn(0x7f4b740cc240 msgr2=0x7f4b740cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.663+0000 7f4b7a00d700 1 -- 172.21.15.112:0/990930615 shutdown_connections 2023-08-27T11:26:45.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.663+0000 7f4b7a00d700 1 -- 172.21.15.112:0/990930615 wait complete. 2023-08-27T11:26:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.665+0000 7f4b7a00d700 1 Processor -- start 2023-08-27T11:26:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.665+0000 7f4b7a00d700 1 -- start start 2023-08-27T11:26:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b7a00d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5c004850 0x7f4b7415cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b7a00d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7413b380 0x7f4b741f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b7a00d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b741f6b10 0x7f4b7406f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b7a00d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7415d190 con 0x7f4b741f6b10 2023-08-27T11:26:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b7a00d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741f6f90 con 0x7f4b5c004850 2023-08-27T11:26:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b7a00d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7413e6c0 con 0x7f4b7413b380 2023-08-27T11:26:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b73fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b741f6b10 0x7f4b7406f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b73fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b741f6b10 0x7f4b7406f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33594/0 (socket says 172.21.15.112:33594) 2023-08-27T11:26:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b73fff700 1 -- 172.21.15.112:0/1711819864 learned_addr learned my addr 172.21.15.112:0/1711819864 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b737fe700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5c004850 0x7f4b7415cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:45.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b72ffd700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7413b380 0x7f4b741f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:45.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b73fff700 1 -- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7413b380 msgr2=0x7f4b741f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:45.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b73fff700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7413b380 0x7f4b741f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:45.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b73fff700 1 -- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5c004850 msgr2=0x7f4b7415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:45.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b73fff700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5c004850 0x7f4b7415cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:45.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b73fff700 1 -- 172.21.15.112:0/1711819864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7413a3b0 con 0x7f4b741f6b10 2023-08-27T11:26:45.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.666+0000 7f4b737fe700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5c004850 0x7f4b7415cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:45.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.667+0000 7f4b73fff700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b741f6b10 0x7f4b7406f600 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4b740afd80 tx=0x7f4b6c007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:45.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.667+0000 7f4b5bfff700 1 -- 172.21.15.112:0/1711819864 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c00eb80 con 0x7f4b741f6b10 2023-08-27T11:26:45.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.667+0000 7f4b5bfff700 1 -- 172.21.15.112:0/1711819864 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c00ece0 con 0x7f4b741f6b10 2023-08-27T11:26:45.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.667+0000 7f4b7a00d700 1 -- 172.21.15.112:0/1711819864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b741395f0 con 0x7f4b741f6b10 2023-08-27T11:26:45.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.667+0000 7f4b5bfff700 1 -- 172.21.15.112:0/1711819864 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c013660 con 0x7f4b741f6b10 2023-08-27T11:26:45.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.667+0000 7f4b7a00d700 1 -- 172.21.15.112:0/1711819864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b74139bb0 con 0x7f4b741f6b10 2023-08-27T11:26:45.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.668+0000 7f4b7a00d700 1 -- 172.21.15.112:0/1711819864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b74067e90 con 0x7f4b741f6b10 2023-08-27T11:26:45.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.668+0000 7f4b5bfff700 1 -- 172.21.15.112:0/1711819864 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f4b6c0137c0 con 0x7f4b741f6b10 2023-08-27T11:26:45.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.668+0000 7f4b5bfff700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4b6006a440 0x7f4b6006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:45.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.668+0000 7f4b5bfff700 1 -- 172.21.15.112:0/1711819864 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f4b6c088d70 con 0x7f4b741f6b10 2023-08-27T11:26:45.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.669+0000 7f4b737fe700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4b6006a440 0x7f4b6006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:45.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.669+0000 7f4b737fe700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4b6006a440 0x7f4b6006c900 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4b68007b70 tx=0x7f4b68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:45.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.673+0000 7f4b5bfff700 1 -- 172.21.15.112:0/1711819864 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4b6c08d070 con 0x7f4b741f6b10 2023-08-27T11:26:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:45.867+0000 7f4b7a00d700 1 -- 172.21.15.112:0/1711819864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4b74067a20 con 0x7f4b741f6b10 2023-08-27T11:26:46.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.538+0000 7f4b5bfff700 1 -- 172.21.15.112:0/1711819864 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4b6c057cd0 con 0x7f4b741f6b10 2023-08-27T11:26:46.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.541+0000 7f4b7a00d700 1 -- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4b6006a440 msgr2=0x7f4b6006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:46.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.541+0000 7f4b7a00d700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4b6006a440 0x7f4b6006c900 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4b68007b70 tx=0x7f4b68009040 comp rx=0 tx=0).stop 2023-08-27T11:26:46.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.541+0000 7f4b7a00d700 1 -- 172.21.15.112:0/1711819864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b741f6b10 msgr2=0x7f4b7406f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:46.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.541+0000 7f4b7a00d700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b741f6b10 0x7f4b7406f600 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4b740afd80 tx=0x7f4b6c007c90 comp rx=0 tx=0).stop 2023-08-27T11:26:46.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.542+0000 7f4b7a00d700 1 -- 172.21.15.112:0/1711819864 shutdown_connections 2023-08-27T11:26:46.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.542+0000 7f4b7a00d700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4b6006a440 0x7f4b6006c900 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.542+0000 7f4b7a00d700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b741f6b10 0x7f4b7406f600 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.542+0000 7f4b7a00d700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5c004850 0x7f4b7415cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.542+0000 7f4b7a00d700 1 --2- 172.21.15.112:0/1711819864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b7413b380 0x7f4b741f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.542+0000 7f4b7a00d700 1 -- 172.21.15.112:0/1711819864 >> 172.21.15.112:0/1711819864 conn(0x7f4b740cc240 msgr2=0x7f4b74141a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:46.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.542+0000 7f4b7a00d700 1 -- 172.21.15.112:0/1711819864 shutdown_connections 2023-08-27T11:26:46.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.542+0000 7f4b7a00d700 1 -- 172.21.15.112:0/1711819864 wait complete. 2023-08-27T11:26:46.547 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2023-08-27T11:26:46.559 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:26:46.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.684+0000 7fbf261f2700 1 Processor -- start 2023-08-27T11:26:46.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.684+0000 7fbf261f2700 1 -- start start 2023-08-27T11:26:46.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.685+0000 7fbf261f2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbf20157c60 0x7fbf20160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:46.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.685+0000 7fbf261f2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbf200aee50 0x7fbf20161520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:46.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.685+0000 7fbf261f2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf20161e70 con 0x7fbf20161a60 2023-08-27T11:26:46.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.685+0000 7fbf261f2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf200adce0 con 0x7fbf200aee50 2023-08-27T11:26:46.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.685+0000 7fbf261f2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf200707a0 con 0x7fbf20157c60 2023-08-27T11:26:46.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.685+0000 7fbf1effd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbf200aee50 0x7fbf20161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:46.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.685+0000 7fbf1effd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbf200aee50 0x7fbf20161520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33598/0 (socket says 172.21.15.112:33598) 2023-08-27T11:26:46.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.685+0000 7fbf1effd700 1 -- 172.21.15.112:0/3885444285 learned_addr learned my addr 172.21.15.112:0/3885444285 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:46.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.685+0000 7fbf1ffff700 1 --1- 172.21.15.112:0/3885444285 >> v1:172.21.15.112:6789/0 conn(0x7fbf20161a60 0x7fbf2015d9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56216/0 (socket says 172.21.15.112:56216) 2023-08-27T11:26:46.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.686+0000 7fbf1effd700 1 -- 172.21.15.112:0/3885444285 >> v2:172.21.15.138:3300/0 conn(0x7fbf20157c60 msgr2=0x7fbf20160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:46.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.686+0000 7fbf1f7fe700 1 --2- 172.21.15.112:0/3885444285 >> v2:172.21.15.138:3300/0 conn(0x7fbf20157c60 0x7fbf20160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:46.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.686+0000 7fbf1e7fc700 1 -- 172.21.15.112:0/3885444285 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468689119 0 0) 0x7fbf20161e70 con 0x7fbf20161a60 2023-08-27T11:26:46.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.686+0000 7fbf1effd700 1 --2- 172.21.15.112:0/3885444285 >> v2:172.21.15.138:3300/0 conn(0x7fbf20157c60 0x7fbf20160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.686+0000 7fbf1effd700 1 -- 172.21.15.112:0/3885444285 >> v1:172.21.15.112:6789/0 conn(0x7fbf20161a60 legacy=0x7fbf2015d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:46.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.686+0000 7fbf1effd700 1 -- 172.21.15.112:0/3885444285 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2006e050 con 0x7fbf200aee50 2023-08-27T11:26:46.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.686+0000 7fbf1effd700 1 --2- 172.21.15.112:0/3885444285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf200aee50 0x7fbf20161520 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fbf20066820 tx=0x7fbf10009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44cf8e3c1a0082a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:46.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.686+0000 7fbf1e7fc700 1 -- 172.21.15.112:0/3885444285 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1000b870 con 0x7fbf200aee50 2023-08-27T11:26:46.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.686+0000 7fbf1e7fc700 1 -- 172.21.15.112:0/3885444285 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1000b9d0 con 0x7fbf200aee50 2023-08-27T11:26:46.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.686+0000 7fbf1e7fc700 1 -- 172.21.15.112:0/3885444285 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1000bd60 con 0x7fbf200aee50 2023-08-27T11:26:46.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.686+0000 7fbf261f2700 1 -- 172.21.15.112:0/3885444285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf200aee50 msgr2=0x7fbf20161520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:46.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.687+0000 7fbf261f2700 1 --2- 172.21.15.112:0/3885444285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf200aee50 0x7fbf20161520 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fbf20066820 tx=0x7fbf10009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.687+0000 7fbf261f2700 1 -- 172.21.15.112:0/3885444285 shutdown_connections 2023-08-27T11:26:46.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.687+0000 7fbf261f2700 1 --2- 172.21.15.112:0/3885444285 >> v2:172.21.15.138:3300/0 conn(0x7fbf20157c60 0x7fbf20160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.687+0000 7fbf261f2700 1 --2- 172.21.15.112:0/3885444285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf200aee50 0x7fbf20161520 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.687+0000 7fbf261f2700 1 -- 172.21.15.112:0/3885444285 >> 172.21.15.112:0/3885444285 conn(0x7fbf200b1380 msgr2=0x7fbf200b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:46.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.687+0000 7fbf261f2700 1 -- 172.21.15.112:0/3885444285 shutdown_connections 2023-08-27T11:26:46.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.687+0000 7fbf261f2700 1 -- 172.21.15.112:0/3885444285 wait complete. 2023-08-27T11:26:46.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.688+0000 7fbf261f2700 1 Processor -- start 2023-08-27T11:26:46.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.688+0000 7fbf261f2700 1 -- start start 2023-08-27T11:26:46.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf261f2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf200aee50 0x7fbf20152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:46.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf261f2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20157c60 0x7fbf201f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:46.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf261f2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf20161a60 0x7fbf20144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:46.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf261f2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2014c1a0 con 0x7fbf20161a60 2023-08-27T11:26:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf261f2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2014c6d0 con 0x7fbf200aee50 2023-08-27T11:26:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf261f2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf20151f60 con 0x7fbf20157c60 2023-08-27T11:26:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf1ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf20161a60 0x7fbf20144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf1ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf20161a60 0x7fbf20144ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33608/0 (socket says 172.21.15.112:33608) 2023-08-27T11:26:46.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf1ffff700 1 -- 172.21.15.112:0/1703884257 learned_addr learned my addr 172.21.15.112:0/1703884257 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:46.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf1f7fe700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf200aee50 0x7fbf20152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:46.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf1effd700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20157c60 0x7fbf201f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:46.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf1ffff700 1 -- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20157c60 msgr2=0x7fbf201f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:46.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf1ffff700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20157c60 0x7fbf201f6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf1ffff700 1 -- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf200aee50 msgr2=0x7fbf20152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:46.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf1ffff700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf200aee50 0x7fbf20152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf1ffff700 1 -- 172.21.15.112:0/1703884257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10009550 con 0x7fbf20161a60 2023-08-27T11:26:46.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf1ffff700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf20161a60 0x7fbf20144ac0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800cbd0 tx=0x7fbf1800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:46.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf17fff700 1 -- 172.21.15.112:0/1703884257 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800aa60 con 0x7fbf20161a60 2023-08-27T11:26:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf17fff700 1 -- 172.21.15.112:0/1703884257 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1800abe0 con 0x7fbf20161a60 2023-08-27T11:26:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf261f2700 1 -- 172.21.15.112:0/1703884257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf2014c3f0 con 0x7fbf20161a60 2023-08-27T11:26:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf17fff700 1 -- 172.21.15.112:0/1703884257 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180070d0 con 0x7fbf20161a60 2023-08-27T11:26:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.689+0000 7fbf261f2700 1 -- 172.21.15.112:0/1703884257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf201f6130 con 0x7fbf20161a60 2023-08-27T11:26:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.690+0000 7fbf261f2700 1 -- 172.21.15.112:0/1703884257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf20067a20 con 0x7fbf20161a60 2023-08-27T11:26:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.691+0000 7fbf17fff700 1 -- 172.21.15.112:0/1703884257 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fbf18007230 con 0x7fbf20161a60 2023-08-27T11:26:46.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.691+0000 7fbf17fff700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf0006a440 0x7fbf0006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:46.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.691+0000 7fbf17fff700 1 -- 172.21.15.112:0/1703884257 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbf180881d0 con 0x7fbf20161a60 2023-08-27T11:26:46.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.692+0000 7fbf1f7fe700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf0006a440 0x7fbf0006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:46.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.692+0000 7fbf1f7fe700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf0006a440 0x7fbf0006c900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbf08006fd0 tx=0x7fbf08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:46.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.696+0000 7fbf17fff700 1 -- 172.21.15.112:0/1703884257 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbf180536c0 con 0x7fbf20161a60 2023-08-27T11:26:46.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.884+0000 7fbf261f2700 1 -- 172.21.15.112:0/1703884257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbf201f6320 con 0x7fbf20161a60 2023-08-27T11:26:46.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.888+0000 7fbf17fff700 1 -- 172.21.15.112:0/1703884257 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbf18056ce0 con 0x7fbf20161a60 2023-08-27T11:26:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.890+0000 7fbf261f2700 1 -- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf0006a440 msgr2=0x7fbf0006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.890+0000 7fbf261f2700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf0006a440 0x7fbf0006c900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbf08006fd0 tx=0x7fbf08008040 comp rx=0 tx=0).stop 2023-08-27T11:26:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.890+0000 7fbf261f2700 1 -- 172.21.15.112:0/1703884257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf20161a60 msgr2=0x7fbf20144ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.890+0000 7fbf261f2700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf20161a60 0x7fbf20144ac0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800cbd0 tx=0x7fbf1800cee0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.890+0000 7fbf261f2700 1 -- 172.21.15.112:0/1703884257 shutdown_connections 2023-08-27T11:26:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.890+0000 7fbf261f2700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf0006a440 0x7fbf0006c900 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.890+0000 7fbf261f2700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf20161a60 0x7fbf20144ac0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.890+0000 7fbf261f2700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf200aee50 0x7fbf20152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.890+0000 7fbf261f2700 1 --2- 172.21.15.112:0/1703884257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20157c60 0x7fbf201f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.890+0000 7fbf261f2700 1 -- 172.21.15.112:0/1703884257 >> 172.21.15.112:0/1703884257 conn(0x7fbf200b1380 msgr2=0x7fbf2006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.890+0000 7fbf261f2700 1 -- 172.21.15.112:0/1703884257 shutdown_connections 2023-08-27T11:26:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:46.890+0000 7fbf261f2700 1 -- 172.21.15.112:0/1703884257 wait complete. 2023-08-27T11:26:46.904 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup' 2023-08-27T11:26:47.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.022+0000 7f5b3d819700 1 Processor -- start 2023-08-27T11:26:47.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.022+0000 7f5b3d819700 1 -- start start 2023-08-27T11:26:47.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b3d819700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b38157880 0x7f5b38160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:47.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b3d819700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5b38161430 0x7f5b380d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:47.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b3d819700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5b380d0c60 0x7f5b3815d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:47.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b3d819700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b381618b0 con 0x7f5b38161430 2023-08-27T11:26:47.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b3d819700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3815f110 con 0x7f5b380d0c60 2023-08-27T11:26:47.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b3d819700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b380d1490 con 0x7f5b38157880 2023-08-27T11:26:47.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b367fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5b38161430 0x7f5b380d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:47.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b367fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5b38161430 0x7f5b380d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33614/0 (socket says 172.21.15.112:33614) 2023-08-27T11:26:47.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b367fc700 1 -- 172.21.15.112:0/2351269616 learned_addr learned my addr 172.21.15.112:0/2351269616 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:47.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b36ffd700 1 --2- 172.21.15.112:0/2351269616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b38157880 0x7f5b38160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:47.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b377fe700 1 --2- 172.21.15.112:0/2351269616 >> v2:172.21.15.138:3300/0 conn(0x7f5b380d0c60 0x7f5b3815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:47.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b367fc700 1 -- 172.21.15.112:0/2351269616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b38157880 msgr2=0x7f5b38160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:47.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b367fc700 1 --2- 172.21.15.112:0/2351269616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b38157880 0x7f5b38160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b367fc700 1 -- 172.21.15.112:0/2351269616 >> v2:172.21.15.138:3300/0 conn(0x7f5b380d0c60 msgr2=0x7f5b3815d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:47.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b367fc700 1 --2- 172.21.15.112:0/2351269616 >> v2:172.21.15.138:3300/0 conn(0x7f5b380d0c60 0x7f5b3815d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b367fc700 1 -- 172.21.15.112:0/2351269616 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b380d1770 con 0x7f5b38161430 2023-08-27T11:26:47.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b377fe700 1 --2- 172.21.15.112:0/2351269616 >> v2:172.21.15.138:3300/0 conn(0x7f5b380d0c60 0x7f5b3815d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:47.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b36ffd700 1 --2- 172.21.15.112:0/2351269616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b38157880 0x7f5b38160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:47.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.023+0000 7f5b367fc700 1 --2- 172.21.15.112:0/2351269616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b38161430 0x7f5b380d0720 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f5b380b2cb0 tx=0x7f5b28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214d5d7c37341f4d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:47.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.024+0000 7f5b35ffb700 1 -- 172.21.15.112:0/2351269616 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800b870 con 0x7f5b38161430 2023-08-27T11:26:47.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.024+0000 7f5b35ffb700 1 -- 172.21.15.112:0/2351269616 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2800b9d0 con 0x7f5b38161430 2023-08-27T11:26:47.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.024+0000 7f5b35ffb700 1 -- 172.21.15.112:0/2351269616 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800bd60 con 0x7f5b38161430 2023-08-27T11:26:47.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.024+0000 7f5b3d819700 1 -- 172.21.15.112:0/2351269616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b38161430 msgr2=0x7f5b380d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:47.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.024+0000 7f5b3d819700 1 --2- 172.21.15.112:0/2351269616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b38161430 0x7f5b380d0720 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f5b380b2cb0 tx=0x7f5b28009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.024+0000 7f5b3d819700 1 -- 172.21.15.112:0/2351269616 shutdown_connections 2023-08-27T11:26:47.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.024+0000 7f5b3d819700 1 --2- 172.21.15.112:0/2351269616 >> v2:172.21.15.138:3300/0 conn(0x7f5b380d0c60 0x7f5b3815d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.024+0000 7f5b3d819700 1 --2- 172.21.15.112:0/2351269616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b38157880 0x7f5b38160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.024+0000 7f5b3d819700 1 --2- 172.21.15.112:0/2351269616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b38161430 0x7f5b380d0720 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.024+0000 7f5b3d819700 1 -- 172.21.15.112:0/2351269616 >> 172.21.15.112:0/2351269616 conn(0x7f5b3806c880 msgr2=0x7f5b380717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:47.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.024+0000 7f5b3d819700 1 -- 172.21.15.112:0/2351269616 shutdown_connections 2023-08-27T11:26:47.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.024+0000 7f5b3d819700 1 -- 172.21.15.112:0/2351269616 wait complete. 2023-08-27T11:26:47.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.025+0000 7f5b3d819700 1 Processor -- start 2023-08-27T11:26:47.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.026+0000 7f5b3d819700 1 -- start start 2023-08-27T11:26:47.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.026+0000 7f5b3d819700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b380d0c60 0x7f5b380b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:47.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.026+0000 7f5b3d819700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b38157880 0x7f5b380b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:47.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.026+0000 7f5b3d819700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b38161430 0x7f5b380b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:47.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.026+0000 7f5b3d819700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b38158cb0 con 0x7f5b380d0c60 2023-08-27T11:26:47.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.026+0000 7f5b3d819700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b38156410 con 0x7f5b38161430 2023-08-27T11:26:47.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.026+0000 7f5b3d819700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b381480d0 con 0x7f5b38157880 2023-08-27T11:26:47.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.026+0000 7f5b36ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b380d0c60 0x7f5b380b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.026+0000 7f5b36ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b380d0c60 0x7f5b380b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33618/0 (socket says 172.21.15.112:33618) 2023-08-27T11:26:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.026+0000 7f5b36ffd700 1 -- 172.21.15.112:0/3121181758 learned_addr learned my addr 172.21.15.112:0/3121181758 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.026+0000 7f5b367fc700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b38157880 0x7f5b380b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.026+0000 7f5b377fe700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b38161430 0x7f5b380b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.027+0000 7f5b36ffd700 1 -- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b38157880 msgr2=0x7f5b380b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:47.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.027+0000 7f5b36ffd700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b38157880 0x7f5b380b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.027+0000 7f5b36ffd700 1 -- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b38161430 msgr2=0x7f5b380b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:47.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.027+0000 7f5b36ffd700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b38161430 0x7f5b380b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.027+0000 7f5b36ffd700 1 -- 172.21.15.112:0/3121181758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28009550 con 0x7f5b380d0c60 2023-08-27T11:26:47.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.027+0000 7f5b377fe700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b38161430 0x7f5b380b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.027+0000 7f5b36ffd700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b380d0c60 0x7f5b380b4970 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00e810 tx=0x7f5b2c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.027+0000 7f5b1ffff700 1 -- 172.21.15.112:0/3121181758 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c004230 con 0x7f5b380d0c60 2023-08-27T11:26:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.027+0000 7f5b1ffff700 1 -- 172.21.15.112:0/3121181758 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c0043b0 con 0x7f5b380d0c60 2023-08-27T11:26:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.027+0000 7f5b3d819700 1 -- 172.21.15.112:0/3121181758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b38158f00 con 0x7f5b380d0c60 2023-08-27T11:26:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.027+0000 7f5b1ffff700 1 -- 172.21.15.112:0/3121181758 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0038e0 con 0x7f5b380d0c60 2023-08-27T11:26:47.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.027+0000 7f5b3d819700 1 -- 172.21.15.112:0/3121181758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b381594c0 con 0x7f5b380d0c60 2023-08-27T11:26:47.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.028+0000 7f5b3d819700 1 -- 172.21.15.112:0/3121181758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b38067a20 con 0x7f5b380d0c60 2023-08-27T11:26:47.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.028+0000 7f5b1ffff700 1 -- 172.21.15.112:0/3121181758 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f5b2c003a40 con 0x7f5b380d0c60 2023-08-27T11:26:47.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.029+0000 7f5b1ffff700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b2406a440 0x7f5b2406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:47.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.029+0000 7f5b1ffff700 1 -- 172.21.15.112:0/3121181758 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5b2c0882a0 con 0x7f5b380d0c60 2023-08-27T11:26:47.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.029+0000 7f5b367fc700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b2406a440 0x7f5b2406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:47.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.029+0000 7f5b367fc700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b2406a440 0x7f5b2406c900 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5b28014040 tx=0x7f5b28008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:47.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.033+0000 7f5b1ffff700 1 -- 172.21.15.112:0/3121181758 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5b2c057200 con 0x7f5b380d0c60 2023-08-27T11:26:47.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.297+0000 7f5b3d819700 1 -- 172.21.15.112:0/3121181758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]} v 0) v1 -- 0x7f5b380b9440 con 0x7f5b380d0c60 2023-08-27T11:26:47.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.540+0000 7f5b1ffff700 1 -- 172.21.15.112:0/3121181758 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]}]=0 v419) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5b2c0094c0 con 0x7f5b380d0c60 2023-08-27T11:26:47.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.542+0000 7f5b3d819700 1 -- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b2406a440 msgr2=0x7f5b2406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:47.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.542+0000 7f5b3d819700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b2406a440 0x7f5b2406c900 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5b28014040 tx=0x7f5b28008030 comp rx=0 tx=0).stop 2023-08-27T11:26:47.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.542+0000 7f5b3d819700 1 -- 172.21.15.112:0/3121181758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b380d0c60 msgr2=0x7f5b380b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:47.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.542+0000 7f5b3d819700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b380d0c60 0x7f5b380b4970 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00e810 tx=0x7f5b2c00eb20 comp rx=0 tx=0).stop 2023-08-27T11:26:47.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.542+0000 7f5b3d819700 1 -- 172.21.15.112:0/3121181758 shutdown_connections 2023-08-27T11:26:47.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.542+0000 7f5b3d819700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b2406a440 0x7f5b2406c900 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.542+0000 7f5b3d819700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b38161430 0x7f5b380b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.542+0000 7f5b3d819700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b380d0c60 0x7f5b380b4970 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.542+0000 7f5b3d819700 1 --2- 172.21.15.112:0/3121181758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b38157880 0x7f5b380b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.542+0000 7f5b3d819700 1 -- 172.21.15.112:0/3121181758 >> 172.21.15.112:0/3121181758 conn(0x7f5b3806c880 msgr2=0x7f5b3815b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:47.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.542+0000 7f5b3d819700 1 -- 172.21.15.112:0/3121181758 shutdown_connections 2023-08-27T11:26:47.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.542+0000 7f5b3d819700 1 -- 172.21.15.112:0/3121181758 wait complete. 2023-08-27T11:26:47.556 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info --human_readable' command without subvolume group ... ok 2023-08-27T11:26:47.556 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info' command without subvolume group 2023-08-27T11:26:47.556 INFO:tasks.cephfs_test_runner:test_volume_info_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) 2023-08-27T11:26:47.557 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup' 2023-08-27T11:26:47.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.675+0000 7f7b5b274700 1 Processor -- start 2023-08-27T11:26:47.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.676+0000 7f7b5b274700 1 -- start start 2023-08-27T11:26:47.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.676+0000 7f7b5b274700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7b5406dcc0 0x7f7b540b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:47.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.676+0000 7f7b5b274700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7b540b3440 0x7f7b540b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:47.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.676+0000 7f7b5b274700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5415b540 con 0x7f7b5415a120 2023-08-27T11:26:47.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.676+0000 7f7b5b274700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b540af830 con 0x7f7b5406dcc0 2023-08-27T11:26:47.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b5b274700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b540afb10 con 0x7f7b540b3440 2023-08-27T11:26:47.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b5880f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7b5406dcc0 0x7f7b540b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:47.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b59010700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7b5415a120 0x7f7b5406d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56240/0 (socket says 172.21.15.112:56240) 2023-08-27T11:26:47.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b59010700 1 -- 172.21.15.112:0/838708925 learned_addr learned my addr 172.21.15.112:0/838708925 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:47.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b59811700 1 --2- 172.21.15.112:0/838708925 >> v2:172.21.15.138:3300/0 conn(0x7f7b540b3440 0x7f7b540b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:47.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b5880f700 1 -- 172.21.15.112:0/838708925 >> v2:172.21.15.138:3300/0 conn(0x7f7b540b3440 msgr2=0x7f7b540b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:47.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b5880f700 1 --2- 172.21.15.112:0/838708925 >> v2:172.21.15.138:3300/0 conn(0x7f7b540b3440 0x7f7b540b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b5880f700 1 -- 172.21.15.112:0/838708925 >> v1:172.21.15.112:6789/0 conn(0x7f7b5415a120 legacy=0x7f7b5406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:47.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b5880f700 1 -- 172.21.15.112:0/838708925 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b540afdf0 con 0x7f7b5406dcc0 2023-08-27T11:26:47.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b4bfff700 1 -- 172.21.15.112:0/838708925 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 859555085 0 0) 0x7f7b5415b540 con 0x7f7b5415a120 2023-08-27T11:26:47.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b5880f700 1 --2- 172.21.15.112:0/838708925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b5406dcc0 0x7f7b540b05f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7b540b5cd0 tx=0x7f7b50009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67359324bfc9e829 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:47.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b4bfff700 1 -- 172.21.15.112:0/838708925 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50011830 con 0x7f7b5406dcc0 2023-08-27T11:26:47.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b4bfff700 1 -- 172.21.15.112:0/838708925 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b50011990 con 0x7f7b5406dcc0 2023-08-27T11:26:47.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.677+0000 7f7b4bfff700 1 -- 172.21.15.112:0/838708925 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50011d20 con 0x7f7b5406dcc0 2023-08-27T11:26:47.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.678+0000 7f7b5b274700 1 -- 172.21.15.112:0/838708925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b5406dcc0 msgr2=0x7f7b540b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:47.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.678+0000 7f7b5b274700 1 --2- 172.21.15.112:0/838708925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b5406dcc0 0x7f7b540b05f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7b540b5cd0 tx=0x7f7b50009b20 comp rx=0 tx=0).stop 2023-08-27T11:26:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.678+0000 7f7b5b274700 1 -- 172.21.15.112:0/838708925 shutdown_connections 2023-08-27T11:26:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.678+0000 7f7b5b274700 1 --2- 172.21.15.112:0/838708925 >> v2:172.21.15.138:3300/0 conn(0x7f7b540b3440 0x7f7b540b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.678+0000 7f7b5b274700 1 --2- 172.21.15.112:0/838708925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b5406dcc0 0x7f7b540b05f0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.678+0000 7f7b5b274700 1 -- 172.21.15.112:0/838708925 >> 172.21.15.112:0/838708925 conn(0x7f7b540cff80 msgr2=0x7f7b540d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.678+0000 7f7b5b274700 1 -- 172.21.15.112:0/838708925 shutdown_connections 2023-08-27T11:26:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.678+0000 7f7b5b274700 1 -- 172.21.15.112:0/838708925 wait complete. 2023-08-27T11:26:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.679+0000 7f7b5b274700 1 Processor -- start 2023-08-27T11:26:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.680+0000 7f7b5b274700 1 -- start start 2023-08-27T11:26:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.680+0000 7f7b5b274700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b5406dcc0 0x7f7b5414ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.680+0000 7f7b5b274700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b540b3440 0x7f7b541f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:47.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.680+0000 7f7b5b274700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5415a120 0x7f7b5406f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:47.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.680+0000 7f7b5b274700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5414ef90 con 0x7f7b5406dcc0 2023-08-27T11:26:47.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.680+0000 7f7b5b274700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540c88d0 con 0x7f7b540b3440 2023-08-27T11:26:47.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.680+0000 7f7b5b274700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540c8b70 con 0x7f7b5415a120 2023-08-27T11:26:47.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.680+0000 7f7b59010700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b5406dcc0 0x7f7b5414ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:47.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b59010700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b5406dcc0 0x7f7b5414ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33632/0 (socket says 172.21.15.112:33632) 2023-08-27T11:26:47.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b59010700 1 -- 172.21.15.112:0/4179908853 learned_addr learned my addr 172.21.15.112:0/4179908853 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:47.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b5880f700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b540b3440 0x7f7b541f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:47.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b59811700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5415a120 0x7f7b5406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:47.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b59010700 1 -- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5415a120 msgr2=0x7f7b5406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:47.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b59010700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5415a120 0x7f7b5406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b59010700 1 -- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b540b3440 msgr2=0x7f7b541f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:47.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b59010700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b540b3440 0x7f7b541f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:47.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b59010700 1 -- 172.21.15.112:0/4179908853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b500094a0 con 0x7f7b5406dcc0 2023-08-27T11:26:47.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b59010700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b5406dcc0 0x7f7b5414ea50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400c550 tx=0x7f7b4400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:47.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b49ffb700 1 -- 172.21.15.112:0/4179908853 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44004c80 con 0x7f7b5406dcc0 2023-08-27T11:26:47.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b49ffb700 1 -- 172.21.15.112:0/4179908853 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b44004e00 con 0x7f7b5406dcc0 2023-08-27T11:26:47.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b5b274700 1 -- 172.21.15.112:0/4179908853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b540c8e50 con 0x7f7b5406dcc0 2023-08-27T11:26:47.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b49ffb700 1 -- 172.21.15.112:0/4179908853 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44007310 con 0x7f7b5406dcc0 2023-08-27T11:26:47.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.681+0000 7f7b5b274700 1 -- 172.21.15.112:0/4179908853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b540c9410 con 0x7f7b5406dcc0 2023-08-27T11:26:47.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.682+0000 7f7b5b274700 1 -- 172.21.15.112:0/4179908853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b54067a20 con 0x7f7b5406dcc0 2023-08-27T11:26:47.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.682+0000 7f7b49ffb700 1 -- 172.21.15.112:0/4179908853 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f7b44007470 con 0x7f7b5406dcc0 2023-08-27T11:26:47.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.683+0000 7f7b49ffb700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b3806a440 0x7f7b3806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:47.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.683+0000 7f7b49ffb700 1 -- 172.21.15.112:0/4179908853 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7b44088570 con 0x7f7b5406dcc0 2023-08-27T11:26:47.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.683+0000 7f7b5880f700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b3806a440 0x7f7b3806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:47.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.684+0000 7f7b5880f700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b3806a440 0x7f7b3806c900 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7b500090b0 tx=0x7f7b50008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:47.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.687+0000 7f7b49ffb700 1 -- 172.21.15.112:0/4179908853 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7b44053a90 con 0x7f7b5406dcc0 2023-08-27T11:26:47.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:47.954+0000 7f7b5b274700 1 -- 172.21.15.112:0/4179908853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]} v 0) v1 -- 0x7f7b540c9600 con 0x7f7b5406dcc0 2023-08-27T11:26:48.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.544+0000 7f7b49ffb700 1 -- 172.21.15.112:0/4179908853 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]}]=0 v420) v1 ==== 159+0+0 (secure 0 0 0) 0x7f7b440570b0 con 0x7f7b5406dcc0 2023-08-27T11:26:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.546+0000 7f7b5b274700 1 -- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b3806a440 msgr2=0x7f7b3806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.546+0000 7f7b5b274700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b3806a440 0x7f7b3806c900 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7b500090b0 tx=0x7f7b50008030 comp rx=0 tx=0).stop 2023-08-27T11:26:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.546+0000 7f7b5b274700 1 -- 172.21.15.112:0/4179908853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b5406dcc0 msgr2=0x7f7b5414ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:48.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.546+0000 7f7b5b274700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b5406dcc0 0x7f7b5414ea50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400c550 tx=0x7f7b4400c910 comp rx=0 tx=0).stop 2023-08-27T11:26:48.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.546+0000 7f7b5b274700 1 -- 172.21.15.112:0/4179908853 shutdown_connections 2023-08-27T11:26:48.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.546+0000 7f7b5b274700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b3806a440 0x7f7b3806c900 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:48.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.546+0000 7f7b5b274700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5415a120 0x7f7b5406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:48.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.546+0000 7f7b5b274700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b5406dcc0 0x7f7b5414ea50 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:48.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.546+0000 7f7b5b274700 1 --2- 172.21.15.112:0/4179908853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b540b3440 0x7f7b541f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:48.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.546+0000 7f7b5b274700 1 -- 172.21.15.112:0/4179908853 >> 172.21.15.112:0/4179908853 conn(0x7f7b540cff80 msgr2=0x7f7b54073380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:48.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.546+0000 7f7b5b274700 1 -- 172.21.15.112:0/4179908853 shutdown_connections 2023-08-27T11:26:48.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.546+0000 7f7b5b274700 1 -- 172.21.15.112:0/4179908853 wait complete. 2023-08-27T11:26:48.562 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:48.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.682+0000 7f444e6ea700 1 Processor -- start 2023-08-27T11:26:48.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.683+0000 7f444e6ea700 1 -- start start 2023-08-27T11:26:48.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.683+0000 7f444e6ea700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4448141420 0x7f44480aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:48.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.683+0000 7f444e6ea700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f444806e150 0x7f444806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:48.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.683+0000 7f444e6ea700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44480ab440 con 0x7f44480cc070 2023-08-27T11:26:48.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.683+0000 7f444e6ea700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444806eb10 con 0x7f444806e150 2023-08-27T11:26:48.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.683+0000 7f444e6ea700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444815eed0 con 0x7f4448141420 2023-08-27T11:26:48.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f44477fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f444806e150 0x7f444806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:48.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f4447fff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4448141420 0x7f44480aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:48.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f44477fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f444806e150 0x7f444806e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:34066/0 (socket says 172.21.15.112:34066) 2023-08-27T11:26:48.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f44477fe700 1 -- 172.21.15.112:0/775852184 learned_addr learned my addr 172.21.15.112:0/775852184 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:48.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f444cc87700 1 --1- 172.21.15.112:0/775852184 >> v1:172.21.15.138:6789/0 conn(0x7f44480cc070 0x7f44480cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:35548/0 (socket says 172.21.15.112:35548) 2023-08-27T11:26:48.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f4447fff700 1 -- 172.21.15.112:0/775852184 >> v2:172.21.15.138:3300/0 conn(0x7f444806e150 msgr2=0x7f444806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:48.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f4447fff700 1 --2- 172.21.15.112:0/775852184 >> v2:172.21.15.138:3300/0 conn(0x7f444806e150 0x7f444806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:48.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f4447fff700 1 -- 172.21.15.112:0/775852184 >> v1:172.21.15.138:6789/0 conn(0x7f44480cc070 legacy=0x7f44480cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:48.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f4447fff700 1 -- 172.21.15.112:0/775852184 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444815f1b0 con 0x7f4448141420 2023-08-27T11:26:48.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f44477fe700 1 --2- 172.21.15.112:0/775852184 >> v2:172.21.15.138:3300/0 conn(0x7f444806e150 0x7f444806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:26:48.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f4447fff700 1 --2- 172.21.15.112:0/775852184 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4448141420 0x7f44480aaf00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4438009840 tx=0x7f4438009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150535b53dfdbe2e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:48.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f4446ffd700 1 -- 172.21.15.112:0/775852184 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438011830 con 0x7f4448141420 2023-08-27T11:26:48.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f4446ffd700 1 -- 172.21.15.112:0/775852184 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4438011990 con 0x7f4448141420 2023-08-27T11:26:48.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.684+0000 7f4446ffd700 1 -- 172.21.15.112:0/775852184 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438011d20 con 0x7f4448141420 2023-08-27T11:26:48.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.685+0000 7f444e6ea700 1 -- 172.21.15.112:0/775852184 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4448141420 msgr2=0x7f44480aaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:48.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.685+0000 7f444e6ea700 1 --2- 172.21.15.112:0/775852184 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4448141420 0x7f44480aaf00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4438009840 tx=0x7f4438009b50 comp rx=0 tx=0).stop 2023-08-27T11:26:48.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.685+0000 7f444e6ea700 1 -- 172.21.15.112:0/775852184 shutdown_connections 2023-08-27T11:26:48.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.685+0000 7f444e6ea700 1 --2- 172.21.15.112:0/775852184 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4448141420 0x7f44480aaf00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:48.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.685+0000 7f444e6ea700 1 --2- 172.21.15.112:0/775852184 >> v2:172.21.15.138:3300/0 conn(0x7f444806e150 0x7f444806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:48.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.685+0000 7f444e6ea700 1 -- 172.21.15.112:0/775852184 >> 172.21.15.112:0/775852184 conn(0x7f444806cd30 msgr2=0x7f44480717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:48.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.685+0000 7f444e6ea700 1 -- 172.21.15.112:0/775852184 shutdown_connections 2023-08-27T11:26:48.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.685+0000 7f444e6ea700 1 -- 172.21.15.112:0/775852184 wait complete. 2023-08-27T11:26:48.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.686+0000 7f444e6ea700 1 Processor -- start 2023-08-27T11:26:48.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.686+0000 7f444e6ea700 1 -- start start 2023-08-27T11:26:48.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.687+0000 7f444e6ea700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f444806e150 0x7f4448137210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:48.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.687+0000 7f444e6ea700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44480cc070 0x7f44480ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:48.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.687+0000 7f444e6ea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4448141420 0x7f4448142850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:48.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.687+0000 7f444e6ea700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444814a180 con 0x7f4448141420 2023-08-27T11:26:48.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.687+0000 7f444e6ea700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444814afe0 con 0x7f444806e150 2023-08-27T11:26:48.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.687+0000 7f444e6ea700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444814a3d0 con 0x7f44480cc070 2023-08-27T11:26:48.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.687+0000 7f444cc87700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4448141420 0x7f4448142850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:48.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.687+0000 7f444cc87700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4448141420 0x7f4448142850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33646/0 (socket says 172.21.15.112:33646) 2023-08-27T11:26:48.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.687+0000 7f444cc87700 1 -- 172.21.15.112:0/1383717182 learned_addr learned my addr 172.21.15.112:0/1383717182 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:48.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.687+0000 7f4447fff700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f444806e150 0x7f4448137210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:48.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.687+0000 7f44477fe700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44480cc070 0x7f44480ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:48.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.688+0000 7f444cc87700 1 -- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44480cc070 msgr2=0x7f44480ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:48.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.688+0000 7f444cc87700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44480cc070 0x7f44480ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:48.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.688+0000 7f444cc87700 1 -- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f444806e150 msgr2=0x7f4448137210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:48.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.688+0000 7f444cc87700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f444806e150 0x7f4448137210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:48.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.688+0000 7f444cc87700 1 -- 172.21.15.112:0/1383717182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c009500 con 0x7f4448141420 2023-08-27T11:26:48.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.688+0000 7f4447fff700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f444806e150 0x7f4448137210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:48.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.688+0000 7f444cc87700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4448141420 0x7f4448142850 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f444000cc00 tx=0x7f444000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:48.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.688+0000 7f4444ff9700 1 -- 172.21.15.112:0/1383717182 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444000aba0 con 0x7f4448141420 2023-08-27T11:26:48.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.688+0000 7f4444ff9700 1 -- 172.21.15.112:0/1383717182 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444000ad00 con 0x7f4448141420 2023-08-27T11:26:48.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.688+0000 7f444e6ea700 1 -- 172.21.15.112:0/1383717182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44380094a0 con 0x7f4448141420 2023-08-27T11:26:48.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.688+0000 7f4444ff9700 1 -- 172.21.15.112:0/1383717182 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440007270 con 0x7f4448141420 2023-08-27T11:26:48.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.688+0000 7f444e6ea700 1 -- 172.21.15.112:0/1383717182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444814a990 con 0x7f4448141420 2023-08-27T11:26:48.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.689+0000 7f444e6ea700 1 -- 172.21.15.112:0/1383717182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4448067a20 con 0x7f4448141420 2023-08-27T11:26:48.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.690+0000 7f4444ff9700 1 -- 172.21.15.112:0/1383717182 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f44400073d0 con 0x7f4448141420 2023-08-27T11:26:48.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.690+0000 7f4444ff9700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f443406a440 0x7f443406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:48.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.690+0000 7f4444ff9700 1 -- 172.21.15.112:0/1383717182 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f44400886f0 con 0x7f4448141420 2023-08-27T11:26:48.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.690+0000 7f4447fff700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f443406a440 0x7f443406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:48.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.691+0000 7f4447fff700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f443406a440 0x7f443406c900 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4438000c00 tx=0x7f4438008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:48.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.694+0000 7f4444ff9700 1 -- 172.21.15.112:0/1383717182 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4440053c00 con 0x7f4448141420 2023-08-27T11:26:48.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.884+0000 7f444e6ea700 1 -- 172.21.15.112:0/1383717182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f444814b120 con 0x7f4448141420 2023-08-27T11:26:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.885+0000 7f4444ff9700 1 -- 172.21.15.112:0/1383717182 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+14737 (secure 0 0 0) 0x7f4440057220 con 0x7f4448141420 2023-08-27T11:26:48.887 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:48.887 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":93,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:46.528653+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/2614833373":"2023-08-28T11:26:43.445146+0000","172.21.15.112:6832/2614833373":"2023-08-28T11:26:43.445146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:48.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.887+0000 7f444e6ea700 1 -- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f443406a440 msgr2=0x7f443406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:48.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.887+0000 7f444e6ea700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f443406a440 0x7f443406c900 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4438000c00 tx=0x7f4438008030 comp rx=0 tx=0).stop 2023-08-27T11:26:48.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.887+0000 7f444e6ea700 1 -- 172.21.15.112:0/1383717182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4448141420 msgr2=0x7f4448142850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:48.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.887+0000 7f444e6ea700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4448141420 0x7f4448142850 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f444000cc00 tx=0x7f444000a060 comp rx=0 tx=0).stop 2023-08-27T11:26:48.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.887+0000 7f444e6ea700 1 -- 172.21.15.112:0/1383717182 shutdown_connections 2023-08-27T11:26:48.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.887+0000 7f444e6ea700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f443406a440 0x7f443406c900 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:48.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.887+0000 7f444e6ea700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4448141420 0x7f4448142850 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:48.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.887+0000 7f444e6ea700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f444806e150 0x7f4448137210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:48.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.887+0000 7f444e6ea700 1 --2- 172.21.15.112:0/1383717182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44480cc070 0x7f44480ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:48.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.887+0000 7f444e6ea700 1 -- 172.21.15.112:0/1383717182 >> 172.21.15.112:0/1383717182 conn(0x7f444806cd30 msgr2=0x7f44481592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:48.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.887+0000 7f444e6ea700 1 -- 172.21.15.112:0/1383717182 shutdown_connections 2023-08-27T11:26:48.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:48.887+0000 7f444e6ea700 1 -- 172.21.15.112:0/1383717182 wait complete. 2023-08-27T11:26:48.901 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:26:48.902 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:26:49.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.025+0000 7faacd4f5700 1 Processor -- start 2023-08-27T11:26:49.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.026+0000 7faacd4f5700 1 -- start start 2023-08-27T11:26:49.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faacd4f5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faac80b4400 0x7faac80b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faacd4f5700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac815f750 con 0x7faac815ed00 2023-08-27T11:26:49.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faacd4f5700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac815fa30 con 0x7faac8152130 2023-08-27T11:26:49.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faacd4f5700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac80b60a0 con 0x7faac80b4400 2023-08-27T11:26:49.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faac67fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7faac815ed00 0x7faac80b3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56268/0 (socket says 172.21.15.112:56268) 2023-08-27T11:26:49.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faac67fc700 1 -- 172.21.15.112:0/3822372071 learned_addr learned my addr 172.21.15.112:0/3822372071 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:49.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faac77fe700 1 --2- 172.21.15.112:0/3822372071 >> v2:172.21.15.112:3300/0 conn(0x7faac80b4400 0x7faac80b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faac77fe700 1 -- 172.21.15.112:0/3822372071 >> v1:172.21.15.138:6789/0 conn(0x7faac8152130 legacy=0x7faac815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:49.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faac77fe700 1 -- 172.21.15.112:0/3822372071 >> v1:172.21.15.112:6789/0 conn(0x7faac815ed00 legacy=0x7faac80b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faac77fe700 1 -- 172.21.15.112:0/3822372071 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac80b6380 con 0x7faac80b4400 2023-08-27T11:26:49.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faac5ffb700 1 -- 172.21.15.112:0/3822372071 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135741053 0 0) 0x7faac815f750 con 0x7faac815ed00 2023-08-27T11:26:49.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faac77fe700 1 --2- 172.21.15.112:0/3822372071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac80b4400 0x7faac80b68a0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7faac815b750 tx=0x7faac000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0534a86b739b425 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:49.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faac5ffb700 1 -- 172.21.15.112:0/3822372071 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac000e070 con 0x7faac80b4400 2023-08-27T11:26:49.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faac5ffb700 1 -- 172.21.15.112:0/3822372071 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac000bb90 con 0x7faac80b4400 2023-08-27T11:26:49.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.027+0000 7faac5ffb700 1 -- 172.21.15.112:0/3822372071 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac0012430 con 0x7faac80b4400 2023-08-27T11:26:49.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.028+0000 7faacd4f5700 1 -- 172.21.15.112:0/3822372071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac80b4400 msgr2=0x7faac80b68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.028+0000 7faacd4f5700 1 --2- 172.21.15.112:0/3822372071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac80b4400 0x7faac80b68a0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7faac815b750 tx=0x7faac000b330 comp rx=0 tx=0).stop 2023-08-27T11:26:49.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.028+0000 7faacd4f5700 1 -- 172.21.15.112:0/3822372071 shutdown_connections 2023-08-27T11:26:49.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.028+0000 7faacd4f5700 1 --2- 172.21.15.112:0/3822372071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac80b4400 0x7faac80b68a0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.028+0000 7faacd4f5700 1 -- 172.21.15.112:0/3822372071 >> 172.21.15.112:0/3822372071 conn(0x7faac8066010 msgr2=0x7faac80663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:49.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.028+0000 7faacd4f5700 1 -- 172.21.15.112:0/3822372071 shutdown_connections 2023-08-27T11:26:49.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.028+0000 7faacd4f5700 1 -- 172.21.15.112:0/3822372071 wait complete. 2023-08-27T11:26:49.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.030+0000 7faacd4f5700 1 Processor -- start 2023-08-27T11:26:49.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.030+0000 7faacd4f5700 1 -- start start 2023-08-27T11:26:49.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faacd4f5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faac80b4400 0x7faac80b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faacd4f5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faac8152130 0x7faac815d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faacd4f5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac815ed00 0x7faac815df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faacd4f5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac80d11f0 con 0x7faac815ed00 2023-08-27T11:26:49.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faacd4f5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac8154d60 con 0x7faac8152130 2023-08-27T11:26:49.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faacd4f5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac80d0850 con 0x7faac80b4400 2023-08-27T11:26:49.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faac77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac815ed00 0x7faac815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faac77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac815ed00 0x7faac815df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33662/0 (socket says 172.21.15.112:33662) 2023-08-27T11:26:49.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faac77fe700 1 -- 172.21.15.112:0/363699169 learned_addr learned my addr 172.21.15.112:0/363699169 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faac67fc700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faac8152130 0x7faac815d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faac6ffd700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faac80b4400 0x7faac80b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faac77fe700 1 -- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faac80b4400 msgr2=0x7faac80b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faac77fe700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faac80b4400 0x7faac80b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faac77fe700 1 -- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faac8152130 msgr2=0x7faac815d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faac77fe700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faac8152130 0x7faac815d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faac77fe700 1 -- 172.21.15.112:0/363699169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac0009a70 con 0x7faac815ed00 2023-08-27T11:26:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faac77fe700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac815ed00 0x7faac815df00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7faac0013000 tx=0x7faac0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faabf7fe700 1 -- 172.21.15.112:0/363699169 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac0012430 con 0x7faac815ed00 2023-08-27T11:26:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faabf7fe700 1 -- 172.21.15.112:0/363699169 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac0020070 con 0x7faac815ed00 2023-08-27T11:26:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faacd4f5700 1 -- 172.21.15.112:0/363699169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac80d0b30 con 0x7faac815ed00 2023-08-27T11:26:49.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faabf7fe700 1 -- 172.21.15.112:0/363699169 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac0004280 con 0x7faac815ed00 2023-08-27T11:26:49.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.031+0000 7faacd4f5700 1 -- 172.21.15.112:0/363699169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac80b1320 con 0x7faac815ed00 2023-08-27T11:26:49.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.032+0000 7faacd4f5700 1 -- 172.21.15.112:0/363699169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac80d1330 con 0x7faac815ed00 2023-08-27T11:26:49.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.033+0000 7faabf7fe700 1 -- 172.21.15.112:0/363699169 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7faac00043e0 con 0x7faac815ed00 2023-08-27T11:26:49.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.033+0000 7faabf7fe700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faaa806a440 0x7faaa806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.033+0000 7faabf7fe700 1 -- 172.21.15.112:0/363699169 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5296+0+0 (secure 0 0 0) 0x7faac0022070 con 0x7faac815ed00 2023-08-27T11:26:49.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.033+0000 7faac6ffd700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faaa806a440 0x7faaa806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.034+0000 7faac6ffd700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faaa806a440 0x7faaa806c900 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faab0007fd0 tx=0x7faab0007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:49.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.038+0000 7faabf7fe700 1 -- 172.21.15.112:0/363699169 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7faac80d1330 con 0x7faac815ed00 2023-08-27T11:26:49.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.223+0000 7faacd4f5700 1 -- 172.21.15.112:0/363699169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faac80d1330 con 0x7faac815ed00 2023-08-27T11:26:49.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.224+0000 7faabf7fe700 1 -- 172.21.15.112:0/363699169 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v93) v1 ==== 92+0+10 (secure 0 0 0) 0x7faac80d1330 con 0x7faac815ed00 2023-08-27T11:26:49.226 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:26:49.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.226+0000 7faacd4f5700 1 -- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faaa806a440 msgr2=0x7faaa806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.226+0000 7faacd4f5700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faaa806a440 0x7faaa806c900 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faab0007fd0 tx=0x7faab0007ec0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.226+0000 7faacd4f5700 1 -- 172.21.15.112:0/363699169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac815ed00 msgr2=0x7faac815df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.226+0000 7faacd4f5700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac815ed00 0x7faac815df00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7faac0013000 tx=0x7faac0004ad0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.226+0000 7faacd4f5700 1 -- 172.21.15.112:0/363699169 shutdown_connections 2023-08-27T11:26:49.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.226+0000 7faacd4f5700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faaa806a440 0x7faaa806c900 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.226+0000 7faacd4f5700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faac815ed00 0x7faac815df00 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.226+0000 7faacd4f5700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faac80b4400 0x7faac80b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.226+0000 7faacd4f5700 1 --2- 172.21.15.112:0/363699169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faac8152130 0x7faac815d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.226+0000 7faacd4f5700 1 -- 172.21.15.112:0/363699169 >> 172.21.15.112:0/363699169 conn(0x7faac8066010 msgr2=0x7faac80b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:49.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.226+0000 7faacd4f5700 1 -- 172.21.15.112:0/363699169 shutdown_connections 2023-08-27T11:26:49.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.226+0000 7faacd4f5700 1 -- 172.21.15.112:0/363699169 wait complete. 2023-08-27T11:26:49.239 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:26:49.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.362+0000 7fa322ab6700 1 Processor -- start 2023-08-27T11:26:49.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.362+0000 7fa322ab6700 1 -- start start 2023-08-27T11:26:49.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.362+0000 7fa322ab6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa31c146c20 0x7fa31c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.362+0000 7fa322ab6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c06f790 0x7fa31c06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.362+0000 7fa322ab6700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31c0b5070 con 0x7fa31c0b3440 2023-08-27T11:26:49.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.362+0000 7fa322ab6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31c070180 con 0x7fa31c146c20 2023-08-27T11:26:49.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.362+0000 7fa322ab6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c147000 con 0x7fa31c06f790 2023-08-27T11:26:49.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.363+0000 7fa320852700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa31c146c20 0x7fa31c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.363+0000 7fa31bfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa31c0b3440 0x7fa31c15e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56282/0 (socket says 172.21.15.112:56282) 2023-08-27T11:26:49.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.363+0000 7fa31bfff700 1 -- 172.21.15.112:0/729462912 learned_addr learned my addr 172.21.15.112:0/729462912 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:49.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.363+0000 7fa321053700 1 --2- 172.21.15.112:0/729462912 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c06f790 0x7fa31c06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.363+0000 7fa320852700 1 -- 172.21.15.112:0/729462912 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c06f790 msgr2=0x7fa31c06fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.363+0000 7fa320852700 1 --2- 172.21.15.112:0/729462912 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c06f790 0x7fa31c06fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.363+0000 7fa320852700 1 -- 172.21.15.112:0/729462912 >> v1:172.21.15.112:6789/0 conn(0x7fa31c0b3440 legacy=0x7fa31c15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.363+0000 7fa320852700 1 -- 172.21.15.112:0/729462912 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c1614a0 con 0x7fa31c146c20 2023-08-27T11:26:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.363+0000 7fa31b7fe700 1 -- 172.21.15.112:0/729462912 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654493400 0 0) 0x7fa308001110 con 0x7fa31c0b3440 2023-08-27T11:26:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.363+0000 7fa320852700 1 --2- 172.21.15.112:0/729462912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c146c20 0x7fa31c160f60 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa310009840 tx=0x7fa310009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2e5a5b67a3b301 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:49.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.363+0000 7fa31b7fe700 1 -- 172.21.15.112:0/729462912 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310011830 con 0x7fa31c146c20 2023-08-27T11:26:49.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.363+0000 7fa31b7fe700 1 -- 172.21.15.112:0/729462912 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa310011990 con 0x7fa31c146c20 2023-08-27T11:26:49.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.363+0000 7fa31b7fe700 1 -- 172.21.15.112:0/729462912 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310011d20 con 0x7fa31c146c20 2023-08-27T11:26:49.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.364+0000 7fa322ab6700 1 -- 172.21.15.112:0/729462912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c146c20 msgr2=0x7fa31c160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.364+0000 7fa322ab6700 1 --2- 172.21.15.112:0/729462912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c146c20 0x7fa31c160f60 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa310009840 tx=0x7fa310009b50 comp rx=0 tx=0).stop 2023-08-27T11:26:49.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.364+0000 7fa322ab6700 1 -- 172.21.15.112:0/729462912 shutdown_connections 2023-08-27T11:26:49.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.364+0000 7fa322ab6700 1 --2- 172.21.15.112:0/729462912 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c06f790 0x7fa31c06fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.364+0000 7fa322ab6700 1 --2- 172.21.15.112:0/729462912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c146c20 0x7fa31c160f60 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.364+0000 7fa322ab6700 1 -- 172.21.15.112:0/729462912 >> 172.21.15.112:0/729462912 conn(0x7fa31c0b1e90 msgr2=0x7fa31c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:49.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.364+0000 7fa322ab6700 1 -- 172.21.15.112:0/729462912 shutdown_connections 2023-08-27T11:26:49.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.364+0000 7fa322ab6700 1 -- 172.21.15.112:0/729462912 wait complete. 2023-08-27T11:26:49.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.365+0000 7fa322ab6700 1 Processor -- start 2023-08-27T11:26:49.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.365+0000 7fa322ab6700 1 -- start start 2023-08-27T11:26:49.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa322ab6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c06f790 0x7fa31c145170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa322ab6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c0b3440 0x7fa31c1f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa322ab6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c146c20 0x7fa31c0b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa322ab6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c1f6610 con 0x7fa31c146c20 2023-08-27T11:26:49.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa322ab6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c1f68b0 con 0x7fa31c06f790 2023-08-27T11:26:49.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa322ab6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c1f6b90 con 0x7fa31c0b3440 2023-08-27T11:26:49.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa321053700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c146c20 0x7fa31c0b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa321053700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c146c20 0x7fa31c0b3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33676/0 (socket says 172.21.15.112:33676) 2023-08-27T11:26:49.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa321053700 1 -- 172.21.15.112:0/252233214 learned_addr learned my addr 172.21.15.112:0/252233214 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:49.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa320852700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c06f790 0x7fa31c145170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa31bfff700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c0b3440 0x7fa31c1f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa321053700 1 -- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c0b3440 msgr2=0x7fa31c1f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa321053700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c0b3440 0x7fa31c1f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa321053700 1 -- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c06f790 msgr2=0x7fa31c145170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa321053700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c06f790 0x7fa31c145170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa321053700 1 -- 172.21.15.112:0/252233214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3100094a0 con 0x7fa31c146c20 2023-08-27T11:26:49.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa320852700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c06f790 0x7fa31c145170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:49.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.366+0000 7fa321053700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c146c20 0x7fa31c0b3da0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa31400cd70 tx=0x7fa31400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:49.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.367+0000 7fa3197fa700 1 -- 172.21.15.112:0/252233214 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa314003a20 con 0x7fa31c146c20 2023-08-27T11:26:49.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.367+0000 7fa3197fa700 1 -- 172.21.15.112:0/252233214 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa314003b80 con 0x7fa31c146c20 2023-08-27T11:26:49.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.367+0000 7fa322ab6700 1 -- 172.21.15.112:0/252233214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c1f6e70 con 0x7fa31c146c20 2023-08-27T11:26:49.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.367+0000 7fa3197fa700 1 -- 172.21.15.112:0/252233214 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa314004020 con 0x7fa31c146c20 2023-08-27T11:26:49.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.367+0000 7fa322ab6700 1 -- 172.21.15.112:0/252233214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c1f2ae0 con 0x7fa31c146c20 2023-08-27T11:26:49.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.368+0000 7fa322ab6700 1 -- 172.21.15.112:0/252233214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31c067e90 con 0x7fa31c146c20 2023-08-27T11:26:49.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.368+0000 7fa3197fa700 1 -- 172.21.15.112:0/252233214 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fa314004180 con 0x7fa31c146c20 2023-08-27T11:26:49.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.368+0000 7fa3197fa700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa30006a440 0x7fa30006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.368+0000 7fa3197fa700 1 -- 172.21.15.112:0/252233214 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa314012070 con 0x7fa31c146c20 2023-08-27T11:26:49.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.369+0000 7fa320852700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa30006a440 0x7fa30006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.369+0000 7fa320852700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa30006a440 0x7fa30006c900 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa3100090b0 tx=0x7fa31001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:49.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.373+0000 7fa3197fa700 1 -- 172.21.15.112:0/252233214 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa314054590 con 0x7fa31c146c20 2023-08-27T11:26:49.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.562+0000 7fa322ab6700 1 -- 172.21.15.112:0/252233214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa31c067a20 con 0x7fa31c146c20 2023-08-27T11:26:49.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.563+0000 7fa3197fa700 1 -- 172.21.15.112:0/252233214 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa314057bb0 con 0x7fa31c146c20 2023-08-27T11:26:49.565 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:26:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.565+0000 7fa322ab6700 1 -- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa30006a440 msgr2=0x7fa30006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.565+0000 7fa322ab6700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa30006a440 0x7fa30006c900 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa3100090b0 tx=0x7fa31001c040 comp rx=0 tx=0).stop 2023-08-27T11:26:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.565+0000 7fa322ab6700 1 -- 172.21.15.112:0/252233214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c146c20 msgr2=0x7fa31c0b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.565+0000 7fa322ab6700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c146c20 0x7fa31c0b3da0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa31400cd70 tx=0x7fa31400a570 comp rx=0 tx=0).stop 2023-08-27T11:26:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.565+0000 7fa322ab6700 1 -- 172.21.15.112:0/252233214 shutdown_connections 2023-08-27T11:26:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.565+0000 7fa322ab6700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa30006a440 0x7fa30006c900 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.565+0000 7fa322ab6700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c146c20 0x7fa31c0b3da0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.565+0000 7fa322ab6700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c06f790 0x7fa31c145170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.565+0000 7fa322ab6700 1 --2- 172.21.15.112:0/252233214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c0b3440 0x7fa31c1f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.565+0000 7fa322ab6700 1 -- 172.21.15.112:0/252233214 >> 172.21.15.112:0/252233214 conn(0x7fa31c0b1e90 msgr2=0x7fa31c0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.565+0000 7fa322ab6700 1 -- 172.21.15.112:0/252233214 shutdown_connections 2023-08-27T11:26:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.565+0000 7fa322ab6700 1 -- 172.21.15.112:0/252233214 wait complete. 2023-08-27T11:26:49.578 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:26:49.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.700+0000 7f56a29ab700 1 Processor -- start 2023-08-27T11:26:49.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.701+0000 7f56a29ab700 1 -- start start 2023-08-27T11:26:49.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.701+0000 7f56a29ab700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f569c0b3440 0x7f569c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.701+0000 7f56a29ab700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f569c15e6e0 0x7f569c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.701+0000 7f56a29ab700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569c0b5070 con 0x7f569c146c20 2023-08-27T11:26:49.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.701+0000 7f56a29ab700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c147000 con 0x7f569c0b3440 2023-08-27T11:26:49.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.701+0000 7f56a29ab700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c161740 con 0x7f569c15e6e0 2023-08-27T11:26:49.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.701+0000 7f569b7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f569c0b3440 0x7f569c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.702+0000 7f569b7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f569c0b3440 0x7f569c15e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33682/0 (socket says 172.21.15.112:33682) 2023-08-27T11:26:49.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.702+0000 7f569b7fe700 1 -- 172.21.15.112:0/1549127002 learned_addr learned my addr 172.21.15.112:0/1549127002 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:49.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.702+0000 7f56a0f48700 1 --2- 172.21.15.112:0/1549127002 >> v2:172.21.15.138:3300/0 conn(0x7f569c15e6e0 0x7f569c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.702+0000 7f569b7fe700 1 -- 172.21.15.112:0/1549127002 >> v2:172.21.15.138:3300/0 conn(0x7f569c15e6e0 msgr2=0x7f569c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:49.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.702+0000 7f569b7fe700 1 --2- 172.21.15.112:0/1549127002 >> v2:172.21.15.138:3300/0 conn(0x7f569c15e6e0 0x7f569c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.702+0000 7f569b7fe700 1 -- 172.21.15.112:0/1549127002 >> v1:172.21.15.138:6789/0 conn(0x7f569c146c20 legacy=0x7f569c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.702+0000 7f569b7fe700 1 -- 172.21.15.112:0/1549127002 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c161a20 con 0x7f569c0b3440 2023-08-27T11:26:49.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.702+0000 7f56a0f48700 1 --2- 172.21.15.112:0/1549127002 >> v2:172.21.15.138:3300/0 conn(0x7f569c15e6e0 0x7f569c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:49.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.702+0000 7f569b7fe700 1 --2- 172.21.15.112:0/1549127002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c0b3440 0x7f569c15e1a0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f569c0b38c0 tx=0x7f5684009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b952a1dcde9704 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:49.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.702+0000 7f569affd700 1 -- 172.21.15.112:0/1549127002 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568400b870 con 0x7f569c0b3440 2023-08-27T11:26:49.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.702+0000 7f569affd700 1 -- 172.21.15.112:0/1549127002 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568400b9d0 con 0x7f569c0b3440 2023-08-27T11:26:49.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.702+0000 7f569affd700 1 -- 172.21.15.112:0/1549127002 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568400bd60 con 0x7f569c0b3440 2023-08-27T11:26:49.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.703+0000 7f56a29ab700 1 -- 172.21.15.112:0/1549127002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c0b3440 msgr2=0x7f569c15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.703+0000 7f56a29ab700 1 --2- 172.21.15.112:0/1549127002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c0b3440 0x7f569c15e1a0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f569c0b38c0 tx=0x7f5684009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.703+0000 7f56a29ab700 1 -- 172.21.15.112:0/1549127002 shutdown_connections 2023-08-27T11:26:49.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.703+0000 7f56a29ab700 1 --2- 172.21.15.112:0/1549127002 >> v2:172.21.15.138:3300/0 conn(0x7f569c15e6e0 0x7f569c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.703+0000 7f56a29ab700 1 --2- 172.21.15.112:0/1549127002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c0b3440 0x7f569c15e1a0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.703+0000 7f56a29ab700 1 -- 172.21.15.112:0/1549127002 >> 172.21.15.112:0/1549127002 conn(0x7f569c0b1e90 msgr2=0x7f569c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.703+0000 7f56a29ab700 1 -- 172.21.15.112:0/1549127002 shutdown_connections 2023-08-27T11:26:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.703+0000 7f56a29ab700 1 -- 172.21.15.112:0/1549127002 wait complete. 2023-08-27T11:26:49.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.704+0000 7f56a29ab700 1 Processor -- start 2023-08-27T11:26:49.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.704+0000 7f56a29ab700 1 -- start start 2023-08-27T11:26:49.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f56a29ab700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c0b3440 0x7f569c072260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f56a29ab700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f569c146c20 0x7f569c142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f56a29ab700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f569c15e6e0 0x7f569c1404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f56a29ab700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c140a20 con 0x7f569c0b3440 2023-08-27T11:26:49.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f56a29ab700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c070f40 con 0x7f569c146c20 2023-08-27T11:26:49.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f56a29ab700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c1f25b0 con 0x7f569c15e6e0 2023-08-27T11:26:49.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f569bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c0b3440 0x7f569c072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f569bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c0b3440 0x7f569c072260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33686/0 (socket says 172.21.15.112:33686) 2023-08-27T11:26:49.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f569bfff700 1 -- 172.21.15.112:0/2275328767 learned_addr learned my addr 172.21.15.112:0/2275328767 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:49.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f569b7fe700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f569c146c20 0x7f569c142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f569bfff700 1 -- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f569c15e6e0 msgr2=0x7f569c1404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f56a0f48700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f569c15e6e0 0x7f569c1404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f569bfff700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f569c15e6e0 0x7f569c1404e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f569bfff700 1 -- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f569c146c20 msgr2=0x7f569c142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f569bfff700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f569c146c20 0x7f569c142bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f569bfff700 1 -- 172.21.15.112:0/2275328767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684009550 con 0x7f569c0b3440 2023-08-27T11:26:49.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f569b7fe700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f569c146c20 0x7f569c142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:49.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.705+0000 7f569bfff700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c0b3440 0x7f569c072260 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f568c00c4e0 tx=0x7f568c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:49.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.706+0000 7f5698ff9700 1 -- 172.21.15.112:0/2275328767 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c0052d0 con 0x7f569c0b3440 2023-08-27T11:26:49.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.706+0000 7f5698ff9700 1 -- 172.21.15.112:0/2275328767 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c005430 con 0x7f569c0b3440 2023-08-27T11:26:49.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.706+0000 7f56a29ab700 1 -- 172.21.15.112:0/2275328767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c140c50 con 0x7f569c0b3440 2023-08-27T11:26:49.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.706+0000 7f5698ff9700 1 -- 172.21.15.112:0/2275328767 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c0038c0 con 0x7f569c0b3440 2023-08-27T11:26:49.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.706+0000 7f56a29ab700 1 -- 172.21.15.112:0/2275328767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c070bd0 con 0x7f569c0b3440 2023-08-27T11:26:49.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.707+0000 7f56a29ab700 1 -- 172.21.15.112:0/2275328767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569c070750 con 0x7f569c0b3440 2023-08-27T11:26:49.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.707+0000 7f5698ff9700 1 -- 172.21.15.112:0/2275328767 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f568c003a20 con 0x7f569c0b3440 2023-08-27T11:26:49.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.708+0000 7f5698ff9700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f567c06a440 0x7f567c06c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:49.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.708+0000 7f5698ff9700 1 -- 172.21.15.112:0/2275328767 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f568c087cd0 con 0x7f569c0b3440 2023-08-27T11:26:49.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.708+0000 7f569b7fe700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f567c06a440 0x7f567c06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:49.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.708+0000 7f569b7fe700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f567c06a440 0x7f567c06c900 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f569c0af5b0 tx=0x7f5684008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:49.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.711+0000 7f5698ff9700 1 -- 172.21.15.112:0/2275328767 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f568c0533b0 con 0x7f569c0b3440 2023-08-27T11:26:49.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.901+0000 7f56a29ab700 1 -- 172.21.15.112:0/2275328767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f569c0b5070 con 0x7f569c0b3440 2023-08-27T11:26:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.905+0000 7f5698ff9700 1 -- 172.21.15.112:0/2275328767 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f569c0b5070 con 0x7f569c0b3440 2023-08-27T11:26:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.907+0000 7f56a29ab700 1 -- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f567c06a440 msgr2=0x7f567c06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.907+0000 7f56a29ab700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f567c06a440 0x7f567c06c900 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f569c0af5b0 tx=0x7f5684008030 comp rx=0 tx=0).stop 2023-08-27T11:26:49.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.907+0000 7f56a29ab700 1 -- 172.21.15.112:0/2275328767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c0b3440 msgr2=0x7f569c072260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:49.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.907+0000 7f56a29ab700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c0b3440 0x7f569c072260 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f568c00c4e0 tx=0x7f568c00c8a0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.907+0000 7f56a29ab700 1 -- 172.21.15.112:0/2275328767 shutdown_connections 2023-08-27T11:26:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.907+0000 7f56a29ab700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f567c06a440 0x7f567c06c900 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.907+0000 7f56a29ab700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f569c15e6e0 0x7f569c1404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.907+0000 7f56a29ab700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c0b3440 0x7f569c072260 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.907+0000 7f56a29ab700 1 --2- 172.21.15.112:0/2275328767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f569c146c20 0x7f569c142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:49.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.907+0000 7f56a29ab700 1 -- 172.21.15.112:0/2275328767 >> 172.21.15.112:0/2275328767 conn(0x7f569c0b1e90 msgr2=0x7f569c1410a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:49.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.907+0000 7f56a29ab700 1 -- 172.21.15.112:0/2275328767 shutdown_connections 2023-08-27T11:26:49.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:49.907+0000 7f56a29ab700 1 -- 172.21.15.112:0/2275328767 wait complete. 2023-08-27T11:26:49.921 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:26:49.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:26:49.957 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:26:49.997 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:26:49.997 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:26:49.998 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:50.015 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:26:50.016 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:26:50.016 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:50.056 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:26:50.056 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:26:50.057 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:50.074 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:26:50.074 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:26:50.074 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:26:50.114 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:26:50.115 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:26:50.115 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:26:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.233+0000 7fba4bdd6700 1 Processor -- start 2023-08-27T11:26:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.233+0000 7fba4bdd6700 1 -- start start 2023-08-27T11:26:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.234+0000 7fba4bdd6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fba4406dc20 0x7fba440ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.234+0000 7fba4bdd6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba440708a0 0x7fba44070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.234+0000 7fba4bdd6700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba4415b3e0 con 0x7fba4406cf80 2023-08-27T11:26:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.234+0000 7fba4bdd6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba4415a5c0 con 0x7fba440708a0 2023-08-27T11:26:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.234+0000 7fba4bdd6700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba440711f0 con 0x7fba4406dc20 2023-08-27T11:26:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.234+0000 7fba4a373700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba440708a0 0x7fba44070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.234+0000 7fba49371700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fba4406cf80 0x7fba4415e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56310/0 (socket says 172.21.15.112:56310) 2023-08-27T11:26:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.234+0000 7fba49371700 1 -- 172.21.15.112:0/2223644345 learned_addr learned my addr 172.21.15.112:0/2223644345 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.234+0000 7fba49b72700 1 --2- 172.21.15.112:0/2223644345 >> v2:172.21.15.138:3300/0 conn(0x7fba4406dc20 0x7fba440ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.234+0000 7fba4a373700 1 -- 172.21.15.112:0/2223644345 >> v2:172.21.15.138:3300/0 conn(0x7fba4406dc20 msgr2=0x7fba440ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.234+0000 7fba4a373700 1 --2- 172.21.15.112:0/2223644345 >> v2:172.21.15.138:3300/0 conn(0x7fba4406dc20 0x7fba440ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba4a373700 1 -- 172.21.15.112:0/2223644345 >> v1:172.21.15.112:6789/0 conn(0x7fba4406cf80 legacy=0x7fba4415e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba4a373700 1 -- 172.21.15.112:0/2223644345 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4415a7d0 con 0x7fba440708a0 2023-08-27T11:26:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba49b72700 1 --2- 172.21.15.112:0/2223644345 >> v2:172.21.15.138:3300/0 conn(0x7fba4406dc20 0x7fba440ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba48b70700 1 -- 172.21.15.112:0/2223644345 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785958101 0 0) 0x7fba4415b3e0 con 0x7fba4406cf80 2023-08-27T11:26:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba4a373700 1 --2- 172.21.15.112:0/2223644345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba440708a0 0x7fba44070cb0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fba440cfee0 tx=0x7fba3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4654c5ceb5bb27ee server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba48b70700 1 -- 172.21.15.112:0/2223644345 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c00e070 con 0x7fba440708a0 2023-08-27T11:26:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba48b70700 1 -- 172.21.15.112:0/2223644345 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3c00bbd0 con 0x7fba440708a0 2023-08-27T11:26:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba48b70700 1 -- 172.21.15.112:0/2223644345 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c012430 con 0x7fba440708a0 2023-08-27T11:26:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba4bdd6700 1 -- 172.21.15.112:0/2223644345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba440708a0 msgr2=0x7fba44070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba4bdd6700 1 --2- 172.21.15.112:0/2223644345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba440708a0 0x7fba44070cb0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fba440cfee0 tx=0x7fba3c00b330 comp rx=0 tx=0).stop 2023-08-27T11:26:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba4bdd6700 1 -- 172.21.15.112:0/2223644345 shutdown_connections 2023-08-27T11:26:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba4bdd6700 1 --2- 172.21.15.112:0/2223644345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba440708a0 0x7fba44070cb0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba4bdd6700 1 --2- 172.21.15.112:0/2223644345 >> v2:172.21.15.138:3300/0 conn(0x7fba4406dc20 0x7fba440ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba4bdd6700 1 -- 172.21.15.112:0/2223644345 >> 172.21.15.112:0/2223644345 conn(0x7fba440b0bf0 msgr2=0x7fba440b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba4bdd6700 1 -- 172.21.15.112:0/2223644345 shutdown_connections 2023-08-27T11:26:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.235+0000 7fba4bdd6700 1 -- 172.21.15.112:0/2223644345 wait complete. 2023-08-27T11:26:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.237+0000 7fba4bdd6700 1 Processor -- start 2023-08-27T11:26:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.237+0000 7fba4bdd6700 1 -- start start 2023-08-27T11:26:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.237+0000 7fba4bdd6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba4406cf80 0x7fba4414d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.237+0000 7fba4bdd6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba4406dc20 0x7fba440ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:50.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.237+0000 7fba4bdd6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba440708a0 0x7fba44155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:50.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.237+0000 7fba4bdd6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba440adef0 con 0x7fba440708a0 2023-08-27T11:26:50.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.237+0000 7fba4bdd6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba440ad2e0 con 0x7fba4406dc20 2023-08-27T11:26:50.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.237+0000 7fba4bdd6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba440ad5c0 con 0x7fba4406cf80 2023-08-27T11:26:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.237+0000 7fba4a373700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba440708a0 0x7fba44155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.237+0000 7fba4a373700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba440708a0 0x7fba44155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33704/0 (socket says 172.21.15.112:33704) 2023-08-27T11:26:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.237+0000 7fba4a373700 1 -- 172.21.15.112:0/466555199 learned_addr learned my addr 172.21.15.112:0/466555199 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.238+0000 7fba4a373700 1 -- 172.21.15.112:0/466555199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba4406cf80 msgr2=0x7fba4414d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.238+0000 7fba4a373700 1 --2- 172.21.15.112:0/466555199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba4406cf80 0x7fba4414d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.238+0000 7fba4a373700 1 -- 172.21.15.112:0/466555199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba4406dc20 msgr2=0x7fba440ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.238+0000 7fba4a373700 1 --2- 172.21.15.112:0/466555199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba4406dc20 0x7fba440ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.238+0000 7fba4a373700 1 -- 172.21.15.112:0/466555199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c009a70 con 0x7fba440708a0 2023-08-27T11:26:50.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.238+0000 7fba4a373700 1 --2- 172.21.15.112:0/466555199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba440708a0 0x7fba44155190 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fba3c014040 tx=0x7fba3c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:50.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.238+0000 7fba3a7fc700 1 -- 172.21.15.112:0/466555199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c012430 con 0x7fba440708a0 2023-08-27T11:26:50.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.238+0000 7fba3a7fc700 1 -- 172.21.15.112:0/466555199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3c020070 con 0x7fba440708a0 2023-08-27T11:26:50.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.238+0000 7fba4bdd6700 1 -- 172.21.15.112:0/466555199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba440ad8a0 con 0x7fba440708a0 2023-08-27T11:26:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.238+0000 7fba3a7fc700 1 -- 172.21.15.112:0/466555199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c004280 con 0x7fba440708a0 2023-08-27T11:26:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.238+0000 7fba4bdd6700 1 -- 172.21.15.112:0/466555199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4414db70 con 0x7fba440708a0 2023-08-27T11:26:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.239+0000 7fba4bdd6700 1 -- 172.21.15.112:0/466555199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba44071890 con 0x7fba440708a0 2023-08-27T11:26:50.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.239+0000 7fba3a7fc700 1 -- 172.21.15.112:0/466555199 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fba3c0043e0 con 0x7fba440708a0 2023-08-27T11:26:50.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.240+0000 7fba3a7fc700 1 --2- 172.21.15.112:0/466555199 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fba2806a440 0x7fba2806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:50.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.240+0000 7fba3a7fc700 1 -- 172.21.15.112:0/466555199 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fba3c022070 con 0x7fba440708a0 2023-08-27T11:26:50.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.240+0000 7fba49b72700 1 --2- 172.21.15.112:0/466555199 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fba2806a440 0x7fba2806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:50.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.241+0000 7fba49b72700 1 --2- 172.21.15.112:0/466555199 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fba2806a440 0x7fba2806c900 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fba3400a8b0 tx=0x7fba34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:50.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.244+0000 7fba3a7fc700 1 -- 172.21.15.112:0/466555199 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fba3c056110 con 0x7fba440708a0 2023-08-27T11:26:50.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.480+0000 7fba4bdd6700 1 -- 172.21.15.112:0/466555199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba44067a20 con 0x7fba440708a0 2023-08-27T11:26:50.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.480+0000 7fba3a7fc700 1 -- 172.21.15.112:0/466555199 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fba3c059730 con 0x7fba440708a0 2023-08-27T11:26:50.482 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:50.483 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":61,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/120353149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":120353149},{"type":"v1","addr":"172.21.15.138:6836","nonce":120353149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1171600247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1171600247},{"type":"v1","addr":"172.21.15.138:6837","nonce":1171600247}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3930299696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3930299696},{"type":"v1","addr":"172.21.15.112:6834","nonce":3930299696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7425,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2232786572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2232786572},{"type":"v1","addr":"172.21.15.112:6835","nonce":2232786572}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[]} 2023-08-27T11:26:50.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.482+0000 7fba4bdd6700 1 -- 172.21.15.112:0/466555199 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fba2806a440 msgr2=0x7fba2806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:50.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.482+0000 7fba4bdd6700 1 --2- 172.21.15.112:0/466555199 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fba2806a440 0x7fba2806c900 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fba3400a8b0 tx=0x7fba34008040 comp rx=0 tx=0).stop 2023-08-27T11:26:50.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.482+0000 7fba4bdd6700 1 -- 172.21.15.112:0/466555199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba440708a0 msgr2=0x7fba44155190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:50.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.482+0000 7fba4bdd6700 1 --2- 172.21.15.112:0/466555199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba440708a0 0x7fba44155190 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fba3c014040 tx=0x7fba3c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.482+0000 7fba4bdd6700 1 -- 172.21.15.112:0/466555199 shutdown_connections 2023-08-27T11:26:50.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.482+0000 7fba4bdd6700 1 --2- 172.21.15.112:0/466555199 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fba2806a440 0x7fba2806c900 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.482+0000 7fba4bdd6700 1 --2- 172.21.15.112:0/466555199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba440708a0 0x7fba44155190 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.482+0000 7fba4bdd6700 1 --2- 172.21.15.112:0/466555199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba4406cf80 0x7fba4414d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.482+0000 7fba4bdd6700 1 --2- 172.21.15.112:0/466555199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba4406dc20 0x7fba440ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.482+0000 7fba4bdd6700 1 -- 172.21.15.112:0/466555199 >> 172.21.15.112:0/466555199 conn(0x7fba440b0bf0 msgr2=0x7fba4406ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:50.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.483+0000 7fba4bdd6700 1 -- 172.21.15.112:0/466555199 shutdown_connections 2023-08-27T11:26:50.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.483+0000 7fba4bdd6700 1 -- 172.21.15.112:0/466555199 wait complete. 2023-08-27T11:26:50.487 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 61 2023-08-27T11:26:50.496 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:26:50.496 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:26:50.497 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:26:50.497 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:26:50.497 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:26:50.498 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:26:50.498 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:26:50.498 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:26:50.498 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:26:50.499 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:26:50.499 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:26:50.499 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:26:50.499 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:26:50.500 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:26:50.500 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:26:50.500 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:26:50.505 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:26:50.505 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:26:50.506 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:26:50.507 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:26:50.513 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:26:50.513 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:26:50.514 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:26:50.514 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:26:50.547 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:26:50.548 DEBUG:teuthology.parallel:result is None 2023-08-27T11:26:50.548 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:26:50.549 DEBUG:teuthology.parallel:result is None 2023-08-27T11:26:50.556 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:26:50.556 DEBUG:teuthology.parallel:result is None 2023-08-27T11:26:50.557 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:26:50.557 DEBUG:teuthology.parallel:result is None 2023-08-27T11:26:50.558 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:26:50.626 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:26:50.626 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:26:50.644 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:26:50.644 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2023-08-27T11:26:50.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.685+0000 7fa213199700 1 Processor -- start 2023-08-27T11:26:50.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.685+0000 7fa213199700 1 -- start start 2023-08-27T11:26:50.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.685+0000 7fa213199700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa20c145d90 0x7fa20c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:50.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.685+0000 7fa213199700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa20c146170 0x7fa20c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:50.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.685+0000 7fa213199700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20c1619a0 con 0x7fa20c0afa90 2023-08-27T11:26:50.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.685+0000 7fa213199700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20c161e30 con 0x7fa20c145d90 2023-08-27T11:26:50.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.685+0000 7fa213199700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c161070 con 0x7fa20c146170 2023-08-27T11:26:50.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.686+0000 7fa210f35700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa20c145d90 0x7fa20c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:50.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.686+0000 7fa211736700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa20c0afa90 0x7fa20c06d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56346/0 (socket says 172.21.15.112:56346) 2023-08-27T11:26:50.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.686+0000 7fa211736700 1 -- 172.21.15.112:0/4284991295 learned_addr learned my addr 172.21.15.112:0/4284991295 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:50.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.686+0000 7fa20bfff700 1 --2- 172.21.15.112:0/4284991295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa20c146170 0x7fa20c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:50.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.686+0000 7fa210f35700 1 -- 172.21.15.112:0/4284991295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa20c146170 msgr2=0x7fa20c06fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:50.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.686+0000 7fa210f35700 1 --2- 172.21.15.112:0/4284991295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa20c146170 0x7fa20c06fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.686+0000 7fa210f35700 1 -- 172.21.15.112:0/4284991295 >> v1:172.21.15.112:6789/0 conn(0x7fa20c0afa90 legacy=0x7fa20c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:50.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.686+0000 7fa210f35700 1 -- 172.21.15.112:0/4284991295 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c161350 con 0x7fa20c145d90 2023-08-27T11:26:50.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.686+0000 7fa20b7fe700 1 -- 172.21.15.112:0/4284991295 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652631807 0 0) 0x7fa20c1619a0 con 0x7fa20c0afa90 2023-08-27T11:26:50.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.686+0000 7fa210f35700 1 --2- 172.21.15.112:0/4284991295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c145d90 0x7fa20c06f790 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fa200009840 tx=0x7fa200009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4d5cef4eef41b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:50.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.686+0000 7fa20b7fe700 1 -- 172.21.15.112:0/4284991295 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200011830 con 0x7fa20c145d90 2023-08-27T11:26:50.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.686+0000 7fa20b7fe700 1 -- 172.21.15.112:0/4284991295 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa200011990 con 0x7fa20c145d90 2023-08-27T11:26:50.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.686+0000 7fa20b7fe700 1 -- 172.21.15.112:0/4284991295 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200011d20 con 0x7fa20c145d90 2023-08-27T11:26:50.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.687+0000 7fa213199700 1 -- 172.21.15.112:0/4284991295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c145d90 msgr2=0x7fa20c06f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:50.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.687+0000 7fa213199700 1 --2- 172.21.15.112:0/4284991295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c145d90 0x7fa20c06f790 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fa200009840 tx=0x7fa200009b50 comp rx=0 tx=0).stop 2023-08-27T11:26:50.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.687+0000 7fa213199700 1 -- 172.21.15.112:0/4284991295 shutdown_connections 2023-08-27T11:26:50.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.687+0000 7fa213199700 1 --2- 172.21.15.112:0/4284991295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c145d90 0x7fa20c06f790 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.687+0000 7fa213199700 1 --2- 172.21.15.112:0/4284991295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa20c146170 0x7fa20c06fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.687+0000 7fa213199700 1 -- 172.21.15.112:0/4284991295 >> 172.21.15.112:0/4284991295 conn(0x7fa20c0cff80 msgr2=0x7fa20c0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:50.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.687+0000 7fa213199700 1 -- 172.21.15.112:0/4284991295 shutdown_connections 2023-08-27T11:26:50.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.687+0000 7fa213199700 1 -- 172.21.15.112:0/4284991295 wait complete. 2023-08-27T11:26:50.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.688+0000 7fa213199700 1 Processor -- start 2023-08-27T11:26:50.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.688+0000 7fa213199700 1 -- start start 2023-08-27T11:26:50.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa213199700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c0afa90 0x7fa20c13ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:50.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa213199700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c145d90 0x7fa20c143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:50.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa213199700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa20c146170 0x7fa20c143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:50.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa213199700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c13efd0 con 0x7fa20c0afa90 2023-08-27T11:26:50.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa213199700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0ae670 con 0x7fa20c145d90 2023-08-27T11:26:50.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa213199700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0ad5d0 con 0x7fa20c146170 2023-08-27T11:26:50.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa210f35700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c0afa90 0x7fa20c13ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:50.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa210f35700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c0afa90 0x7fa20c13ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33736/0 (socket says 172.21.15.112:33736) 2023-08-27T11:26:50.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa210f35700 1 -- 172.21.15.112:0/2250708125 learned_addr learned my addr 172.21.15.112:0/2250708125 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:50.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa211736700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa20c146170 0x7fa20c143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:50.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa20bfff700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c145d90 0x7fa20c143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:50.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa210f35700 1 -- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa20c146170 msgr2=0x7fa20c143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:50.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa210f35700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa20c146170 0x7fa20c143b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa210f35700 1 -- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c145d90 msgr2=0x7fa20c143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:50.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa210f35700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c145d90 0x7fa20c143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:50.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa210f35700 1 -- 172.21.15.112:0/2250708125 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2000094a0 con 0x7fa20c0afa90 2023-08-27T11:26:50.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa211736700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa20c146170 0x7fa20c143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:50.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.689+0000 7fa210f35700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c0afa90 0x7fa20c13ea90 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa2000090b0 tx=0x7fa200004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:50.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.690+0000 7fa2097fa700 1 -- 172.21.15.112:0/2250708125 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2000038a0 con 0x7fa20c0afa90 2023-08-27T11:26:50.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.690+0000 7fa2097fa700 1 -- 172.21.15.112:0/2250708125 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20001f070 con 0x7fa20c0afa90 2023-08-27T11:26:50.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.690+0000 7fa2097fa700 1 -- 172.21.15.112:0/2250708125 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200003c20 con 0x7fa20c0afa90 2023-08-27T11:26:50.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.690+0000 7fa213199700 1 -- 172.21.15.112:0/2250708125 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20c0ad8b0 con 0x7fa20c0afa90 2023-08-27T11:26:50.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.690+0000 7fa213199700 1 -- 172.21.15.112:0/2250708125 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20c0ade70 con 0x7fa20c0afa90 2023-08-27T11:26:50.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.690+0000 7fa213199700 1 -- 172.21.15.112:0/2250708125 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20c067a20 con 0x7fa20c0afa90 2023-08-27T11:26:50.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.691+0000 7fa2097fa700 1 -- 172.21.15.112:0/2250708125 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fa20001a430 con 0x7fa20c0afa90 2023-08-27T11:26:50.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.691+0000 7fa2097fa700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1f006a440 0x7fa1f006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:50.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.691+0000 7fa2097fa700 1 -- 172.21.15.112:0/2250708125 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa200021070 con 0x7fa20c0afa90 2023-08-27T11:26:50.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.691+0000 7fa20bfff700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1f006a440 0x7fa1f006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.692+0000 7fa20bfff700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1f006a440 0x7fa1f006c900 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0712e0 tx=0x7fa1f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.696+0000 7fa2097fa700 1 -- 172.21.15.112:0/2250708125 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa200055810 con 0x7fa20c0afa90 2023-08-27T11:26:50.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:50.899+0000 7fa213199700 1 -- 172.21.15.112:0/2250708125 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa20c071420 con 0x7fa20c0afa90 2023-08-27T11:26:51.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:51.925+0000 7fa2097fa700 1 -- 172.21.15.112:0/2250708125 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v98) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa200010380 con 0x7fa20c0afa90 2023-08-27T11:26:51.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:51.928+0000 7fa213199700 1 -- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1f006a440 msgr2=0x7fa1f006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:51.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:51.928+0000 7fa213199700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1f006a440 0x7fa1f006c900 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0712e0 tx=0x7fa1f8008040 comp rx=0 tx=0).stop 2023-08-27T11:26:51.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:51.928+0000 7fa213199700 1 -- 172.21.15.112:0/2250708125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c0afa90 msgr2=0x7fa20c13ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:51.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:51.928+0000 7fa213199700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c0afa90 0x7fa20c13ea90 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa2000090b0 tx=0x7fa200004750 comp rx=0 tx=0).stop 2023-08-27T11:26:51.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:51.928+0000 7fa213199700 1 -- 172.21.15.112:0/2250708125 shutdown_connections 2023-08-27T11:26:51.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:51.928+0000 7fa213199700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1f006a440 0x7fa1f006c900 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:51.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:51.928+0000 7fa213199700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa20c146170 0x7fa20c143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:51.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:51.928+0000 7fa213199700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c0afa90 0x7fa20c13ea90 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:51.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:51.928+0000 7fa213199700 1 --2- 172.21.15.112:0/2250708125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c145d90 0x7fa20c143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:51.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:51.928+0000 7fa213199700 1 -- 172.21.15.112:0/2250708125 >> 172.21.15.112:0/2250708125 conn(0x7fa20c0cff80 msgr2=0x7fa20c13f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:51.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:51.928+0000 7fa213199700 1 -- 172.21.15.112:0/2250708125 shutdown_connections 2023-08-27T11:26:51.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:51.928+0000 7fa213199700 1 -- 172.21.15.112:0/2250708125 wait complete. 2023-08-27T11:26:51.932 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2023-08-27T11:26:51.943 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:26:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.062+0000 7f355413d700 1 Processor -- start 2023-08-27T11:26:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.063+0000 7f355413d700 1 -- start start 2023-08-27T11:26:52.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.063+0000 7f355413d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f354c06adb0 0x7f354c06b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.063+0000 7f355413d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354c15eb60 con 0x7f354c0cdbd0 2023-08-27T11:26:52.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.063+0000 7f355413d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354c0b02f0 con 0x7f354c151060 2023-08-27T11:26:52.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.063+0000 7f355413d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354c06b770 con 0x7f354c06adb0 2023-08-27T11:26:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f35526da700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f354c0cdbd0 0x7f354c0ce020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56366/0 (socket says 172.21.15.112:56366) 2023-08-27T11:26:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f35526da700 1 -- 172.21.15.112:0/2152246982 learned_addr learned my addr 172.21.15.112:0/2152246982 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f35516d8700 1 --2- 172.21.15.112:0/2152246982 >> v2:172.21.15.138:3300/0 conn(0x7f354c06adb0 0x7f354c06b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3594036593 0 0) 0x7f354c15eb60 con 0x7f354c0cdbd0 2023-08-27T11:26:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35380035b0 con 0x7f354c0cdbd0 2023-08-27T11:26:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530803827 0 0) 0x7f354c0b02f0 con 0x7f354c151060 2023-08-27T11:26:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f354c15eb60 con 0x7f354c151060 2023-08-27T11:26:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2106945162 0 0) 0x7f35380035b0 con 0x7f354c0cdbd0 2023-08-27T11:26:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 >> v2:172.21.15.138:3300/0 conn(0x7f354c06adb0 msgr2=0x7f354c06b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:52.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 --2- 172.21.15.112:0/2152246982 >> v2:172.21.15.138:3300/0 conn(0x7f354c06adb0 0x7f354c06b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 >> v1:172.21.15.138:6789/0 conn(0x7f354c151060 legacy=0x7f354c15e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c0b05d0 con 0x7f354c0cdbd0 2023-08-27T11:26:52.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f35440019f0 con 0x7f354c0cdbd0 2023-08-27T11:26:52.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 >> v1:172.21.15.112:6789/0 conn(0x7f354c0cdbd0 legacy=0x7f354c0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 --2- 172.21.15.112:0/2152246982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3538003be0 0x7f3538003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 --2- 172.21.15.112:0/2152246982 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3538004850 0x7f3538004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 --2- 172.21.15.112:0/2152246982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3538005210 0x7f353800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c06b770 con 0x7f3538003be0 2023-08-27T11:26:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353800c0e0 con 0x7f3538005210 2023-08-27T11:26:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353800c2a0 con 0x7f3538004850 2023-08-27T11:26:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3551ed9700 1 --2- 172.21.15.112:0/2152246982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3538003be0 0x7f3538003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f35516d8700 1 --2- 172.21.15.112:0/2152246982 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3538004850 0x7f3538004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f35526da700 1 --2- 172.21.15.112:0/2152246982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3538005210 0x7f353800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3551ed9700 1 -- 172.21.15.112:0/2152246982 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3538004850 msgr2=0x7f3538004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:52.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3551ed9700 1 --2- 172.21.15.112:0/2152246982 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3538004850 0x7f3538004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3551ed9700 1 -- 172.21.15.112:0/2152246982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3538005210 msgr2=0x7f353800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:52.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.064+0000 7f3551ed9700 1 --2- 172.21.15.112:0/2152246982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3538005210 0x7f353800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f3551ed9700 1 -- 172.21.15.112:0/2152246982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35380035b0 con 0x7f3538003be0 2023-08-27T11:26:52.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f35526da700 1 -- 172.21.15.112:0/2152246982 reap_dead start 2023-08-27T11:26:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f35516d8700 1 --2- 172.21.15.112:0/2152246982 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3538004850 0x7f3538004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f3551ed9700 1 --2- 172.21.15.112:0/2152246982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3538003be0 0x7f3538003fe0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f353c001940 tx=0x7f3538005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efac39986951c6b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c001430 con 0x7f3538003be0 2023-08-27T11:26:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353c001d70 con 0x7f3538003be0 2023-08-27T11:26:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f3550ed7700 1 -- 172.21.15.112:0/2152246982 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c005530 con 0x7f3538003be0 2023-08-27T11:26:52.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f355413d700 1 -- 172.21.15.112:0/2152246982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3538003be0 msgr2=0x7f3538003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f355413d700 1 --2- 172.21.15.112:0/2152246982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3538003be0 0x7f3538003fe0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f353c001940 tx=0x7f3538005880 comp rx=0 tx=0).stop 2023-08-27T11:26:52.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f355413d700 1 -- 172.21.15.112:0/2152246982 shutdown_connections 2023-08-27T11:26:52.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f355413d700 1 --2- 172.21.15.112:0/2152246982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3538003be0 0x7f3538003fe0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f355413d700 1 -- 172.21.15.112:0/2152246982 >> 172.21.15.112:0/2152246982 conn(0x7f354c0d0000 msgr2=0x7f354c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:52.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f355413d700 1 -- 172.21.15.112:0/2152246982 shutdown_connections 2023-08-27T11:26:52.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.065+0000 7f355413d700 1 -- 172.21.15.112:0/2152246982 wait complete. 2023-08-27T11:26:52.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.067+0000 7f355413d700 1 Processor -- start 2023-08-27T11:26:52.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.067+0000 7f355413d700 1 -- start start 2023-08-27T11:26:52.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.067+0000 7f355413d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3538003be0 0x7f354c1f0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.067+0000 7f355413d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f354c070240 0x7f354c0cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.067+0000 7f355413d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f354c0cbb00 0x7f354c0c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.067+0000 7f355413d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0cbf80 con 0x7f354c0cbb00 2023-08-27T11:26:52.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.067+0000 7f355413d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0706c0 con 0x7f3538003be0 2023-08-27T11:26:52.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.067+0000 7f355413d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0c6990 con 0x7f354c070240 2023-08-27T11:26:52.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.067+0000 7f35526da700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f354c0cbb00 0x7f354c0c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.067+0000 7f35526da700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f354c0cbb00 0x7f354c0c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33764/0 (socket says 172.21.15.112:33764) 2023-08-27T11:26:52.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.067+0000 7f35526da700 1 -- 172.21.15.112:0/1645374566 learned_addr learned my addr 172.21.15.112:0/1645374566 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:52.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.068+0000 7f35526da700 1 -- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f354c070240 msgr2=0x7f354c0cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.068+0000 7f35516d8700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f354c070240 0x7f354c0cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.068+0000 7f3551ed9700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3538003be0 0x7f354c1f0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.068+0000 7f35526da700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f354c070240 0x7f354c0cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.068+0000 7f35526da700 1 -- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3538003be0 msgr2=0x7f354c1f0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.068+0000 7f35526da700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3538003be0 0x7f354c1f0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.068+0000 7f35526da700 1 -- 172.21.15.112:0/1645374566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c1f4e90 con 0x7f354c0cbb00 2023-08-27T11:26:52.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.068+0000 7f35526da700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f354c0cbb00 0x7f354c0c92e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f354c0c7080 tx=0x7f3544007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:52.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.068+0000 7f35427fc700 1 -- 172.21.15.112:0/1645374566 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544017070 con 0x7f354c0cbb00 2023-08-27T11:26:52.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.068+0000 7f35427fc700 1 -- 172.21.15.112:0/1645374566 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354400ec30 con 0x7f354c0cbb00 2023-08-27T11:26:52.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.068+0000 7f35427fc700 1 -- 172.21.15.112:0/1645374566 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354400ede0 con 0x7f354c0cbb00 2023-08-27T11:26:52.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.068+0000 7f355413d700 1 -- 172.21.15.112:0/1645374566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c0c63d0 con 0x7f354c0cbb00 2023-08-27T11:26:52.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.068+0000 7f355413d700 1 -- 172.21.15.112:0/1645374566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c06fec0 con 0x7f354c0cbb00 2023-08-27T11:26:52.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.069+0000 7f355413d700 1 -- 172.21.15.112:0/1645374566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c070080 con 0x7f354c0cbb00 2023-08-27T11:26:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.069+0000 7f35427fc700 1 -- 172.21.15.112:0/1645374566 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f3544013420 con 0x7f354c0cbb00 2023-08-27T11:26:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.070+0000 7f35427fc700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f353006a440 0x7f353006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.070+0000 7f35427fc700 1 -- 172.21.15.112:0/1645374566 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3544088400 con 0x7f354c0cbb00 2023-08-27T11:26:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.070+0000 7f3551ed9700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f353006a440 0x7f353006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.070+0000 7f3551ed9700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f353006a440 0x7f353006c900 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f353c005040 tx=0x7f353c00a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:52.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.074+0000 7f35427fc700 1 -- 172.21.15.112:0/1645374566 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3544053ac0 con 0x7f354c0cbb00 2023-08-27T11:26:52.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.338+0000 7f355413d700 1 -- 172.21.15.112:0/1645374566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f354c067a20 con 0x7f354c0cbb00 2023-08-27T11:26:52.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.339+0000 7f35427fc700 1 -- 172.21.15.112:0/1645374566 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f35440570e0 con 0x7f354c0cbb00 2023-08-27T11:26:52.340 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:52.341 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:26:52.341 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2023-08-27T11:26:52.341 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.341 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2023-08-27T11:26:52.342 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk2bnkLBAA+vxcTeJzucKk9iSF/Zw3Lg==", 2023-08-27T11:26:52.342 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.342 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:52.342 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:26:52.343 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:26:52.343 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.343 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.344 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2023-08-27T11:26:52.344 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk3zD5OBAASVnwG6yn/YQzfYyeMakFpA==", 2023-08-27T11:26:52.344 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.344 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:52.344 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:26:52.345 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:26:52.345 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.345 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.346 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.347 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2023-08-27T11:26:52.348 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtki7wJNBAAsIGgDLM82sWNcKnCDlSBMw==", 2023-08-27T11:26:52.348 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.348 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:52.348 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:26:52.349 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:26:52.349 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.349 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.350 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2023-08-27T11:26:52.350 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkCsW/BBAAFTPiTOpBo1L+rLo9px+AjQ==", 2023-08-27T11:26:52.350 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.350 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:52.350 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:26:52.351 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:26:52.351 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.351 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.352 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2023-08-27T11:26:52.352 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB7MOtkR/cSHhAAR8e8VRd+e0ESJkqQDYWbEw==", 2023-08-27T11:26:52.352 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.352 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:52.352 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:52.353 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.353 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.354 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2023-08-27T11:26:52.354 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB+MOtksBF3HRAA1OIXiIkhgPDYpQyYPaWOXg==", 2023-08-27T11:26:52.354 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.354 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:52.354 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:52.355 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.355 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.356 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2023-08-27T11:26:52.356 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCBMOtkWsrdHBAAkfWbiE6wAFzpt9+BxRh8fw==", 2023-08-27T11:26:52.356 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.356 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:52.356 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:52.357 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.357 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.357 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2023-08-27T11:26:52.358 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCEMOtkIkiKHRAARDoh/FH9ZdqUlZZ29Mc/Dw==", 2023-08-27T11:26:52.358 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.358 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:52.358 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:52.358 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.359 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.359 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2023-08-27T11:26:52.359 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCKMOtkPgDzGxAA+U4kbwD0d4lS8aNPG2CgHw==", 2023-08-27T11:26:52.360 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.360 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:52.360 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:52.360 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.360 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.361 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.361 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2023-08-27T11:26:52.361 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCNMOtkKomSGxAAGAQUcIH2H/uyZIU8vlq1pw==", 2023-08-27T11:26:52.361 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.362 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:52.362 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:52.362 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.362 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.363 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2023-08-27T11:26:52.363 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCQMOtkHI2MGhAA7sgbi8JXO8P9wnVvMy3PqA==", 2023-08-27T11:26:52.363 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.363 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:52.364 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:52.364 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.364 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.365 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2023-08-27T11:26:52.365 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCTMOtkVHpiGhAA4C4YJMWbQ0mmGgB4i8G/AA==", 2023-08-27T11:26:52.365 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.365 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:26:52.365 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:26:52.366 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.366 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.366 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2023-08-27T11:26:52.366 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA==", 2023-08-27T11:26:52.367 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.367 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:52.367 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2023-08-27T11:26:52.367 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:26:52.367 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.368 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2023-08-27T11:26:52.368 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtk66DbDxAAWZ5StELux0lriJ2gEN9icg==", 2023-08-27T11:26:52.369 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.369 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:52.369 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:26:52.369 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:26:52.369 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:26:52.370 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.370 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2023-08-27T11:26:52.370 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkViKJDBAAF2ofwO+78er9OyFfn2bO+g==", 2023-08-27T11:26:52.371 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.371 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:52.371 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:26:52.371 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:26:52.371 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:26:52.372 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.372 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2023-08-27T11:26:52.373 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkwt+QFRAAowWE69VqeRhQz0896rxx9Q==", 2023-08-27T11:26:52.373 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.373 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:26:52.373 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:26:52.373 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:26:52.374 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:26:52.374 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.374 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2023-08-27T11:26:52.375 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkjDA+DxAAjmOn82AC2OMpKlAVCD6hlg==", 2023-08-27T11:26:52.375 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.375 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:26:52.375 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:26:52.375 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:26:52.376 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.376 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.376 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2023-08-27T11:26:52.377 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkXguHKRAALHwzk8diVXqsMvTKr+AzLQ==", 2023-08-27T11:26:52.377 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.377 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:26:52.377 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:26:52.377 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.378 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:52.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:52.378 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2023-08-27T11:26:52.378 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkrRjBJRAAq3ftmIh81LKadpliVUiJ6g==", 2023-08-27T11:26:52.379 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:26:52.379 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:26:52.379 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:26:52.379 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:26:52.379 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.380 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:52.380 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:26:52.380 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:26:52.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.341+0000 7f355413d700 1 -- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f353006a440 msgr2=0x7f353006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.341+0000 7f355413d700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f353006a440 0x7f353006c900 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f353c005040 tx=0x7f353c00a000 comp rx=0 tx=0).stop 2023-08-27T11:26:52.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.341+0000 7f355413d700 1 -- 172.21.15.112:0/1645374566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f354c0cbb00 msgr2=0x7f354c0c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.341+0000 7f355413d700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f354c0cbb00 0x7f354c0c92e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f354c0c7080 tx=0x7f3544007c90 comp rx=0 tx=0).stop 2023-08-27T11:26:52.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.341+0000 7f355413d700 1 -- 172.21.15.112:0/1645374566 shutdown_connections 2023-08-27T11:26:52.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.341+0000 7f355413d700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f353006a440 0x7f353006c900 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.341+0000 7f355413d700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f354c0cbb00 0x7f354c0c92e0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.341+0000 7f355413d700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3538003be0 0x7f354c1f0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.341+0000 7f355413d700 1 --2- 172.21.15.112:0/1645374566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f354c070240 0x7f354c0cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.341+0000 7f355413d700 1 -- 172.21.15.112:0/1645374566 >> 172.21.15.112:0/1645374566 conn(0x7f354c0d0000 msgr2=0x7f354c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:52.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.341+0000 7f355413d700 1 -- 172.21.15.112:0/1645374566 shutdown_connections 2023-08-27T11:26:52.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.341+0000 7f355413d700 1 -- 172.21.15.112:0/1645374566 wait complete. 2023-08-27T11:26:52.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:52.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.503+0000 7f0b7fab1700 1 Processor -- start 2023-08-27T11:26:52.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.503+0000 7f0b7fab1700 1 -- start start 2023-08-27T11:26:52.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.504+0000 7f0b7fab1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b78151060 0x7f0b7815e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.504+0000 7f0b7fab1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0b7815e990 0x7f0b7806adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.504+0000 7f0b7fab1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b780b02f0 con 0x7f0b7806b2f0 2023-08-27T11:26:52.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.504+0000 7f0b7fab1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7806b780 con 0x7f0b7815e990 2023-08-27T11:26:52.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.504+0000 7f0b7fab1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780b05d0 con 0x7f0b78151060 2023-08-27T11:26:52.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.504+0000 7f0b7e04e700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0b7806b2f0 0x7f0b7806d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56384/0 (socket says 172.21.15.112:56384) 2023-08-27T11:26:52.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.504+0000 7f0b7e04e700 1 -- 172.21.15.112:0/1393629315 learned_addr learned my addr 172.21.15.112:0/1393629315 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:52.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.504+0000 7f0b7d84d700 1 --2- 172.21.15.112:0/1393629315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b78151060 0x7f0b7815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.504+0000 7f0b7d04c700 1 --2- 172.21.15.112:0/1393629315 >> v2:172.21.15.138:3300/0 conn(0x7f0b7815e990 0x7f0b7806adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240582050 0 0) 0x7f0b780b02f0 con 0x7f0b7806b2f0 2023-08-27T11:26:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b640035b0 con 0x7f0b7806b2f0 2023-08-27T11:26:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3402119787 0 0) 0x7f0b640035b0 con 0x7f0b7806b2f0 2023-08-27T11:26:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b78151060 msgr2=0x7f0b7815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 --2- 172.21.15.112:0/1393629315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b78151060 0x7f0b7815e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 >> v2:172.21.15.138:3300/0 conn(0x7f0b7815e990 msgr2=0x7f0b7806adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:52.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 --2- 172.21.15.112:0/1393629315 >> v2:172.21.15.138:3300/0 conn(0x7f0b7815e990 0x7f0b7806adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b780b08b0 con 0x7f0b7806b2f0 2023-08-27T11:26:52.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f0b700019f0 con 0x7f0b7806b2f0 2023-08-27T11:26:52.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 >> v1:172.21.15.112:6789/0 conn(0x7f0b7806b2f0 legacy=0x7f0b7806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 --2- 172.21.15.112:0/1393629315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b64003a00 0x7f0b64003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 --2- 172.21.15.112:0/1393629315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b64004850 0x7f0b64004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 --2- 172.21.15.112:0/1393629315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b78151060 0x7f0b64005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7806b780 con 0x7f0b64004850 2023-08-27T11:26:52.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780b05d0 con 0x7f0b64003a00 2023-08-27T11:26:52.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6400ba10 con 0x7f0b78151060 2023-08-27T11:26:52.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7d04c700 1 --2- 172.21.15.112:0/1393629315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b64004850 0x7f0b64004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7d84d700 1 --2- 172.21.15.112:0/1393629315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b64003a00 0x7f0b64003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7e04e700 1 --2- 172.21.15.112:0/1393629315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b78151060 0x7f0b64005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7d04c700 1 -- 172.21.15.112:0/1393629315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b78151060 msgr2=0x7f0b64005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7d04c700 1 --2- 172.21.15.112:0/1393629315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b78151060 0x7f0b64005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7d04c700 1 -- 172.21.15.112:0/1393629315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b64003a00 msgr2=0x7f0b64003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7d04c700 1 --2- 172.21.15.112:0/1393629315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b64003a00 0x7f0b64003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7d04c700 1 -- 172.21.15.112:0/1393629315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b640035b0 con 0x7f0b64004850 2023-08-27T11:26:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7e04e700 1 --2- 172.21.15.112:0/1393629315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b78151060 0x7f0b64005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7d04c700 1 --2- 172.21.15.112:0/1393629315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b64004850 0x7f0b64004cd0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009b60 tx=0x7f0b64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8293d0fb62e5541 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.505+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800e6c0 con 0x7f0b64004850 2023-08-27T11:26:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.506+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6800e820 con 0x7f0b64004850 2023-08-27T11:26:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.506+0000 7f0b7c84b700 1 -- 172.21.15.112:0/1393629315 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800eb90 con 0x7f0b64004850 2023-08-27T11:26:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.506+0000 7f0b7fab1700 1 -- 172.21.15.112:0/1393629315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b64004850 msgr2=0x7f0b64004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.506+0000 7f0b7fab1700 1 --2- 172.21.15.112:0/1393629315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b64004850 0x7f0b64004cd0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009b60 tx=0x7f0b64005880 comp rx=0 tx=0).stop 2023-08-27T11:26:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.506+0000 7f0b7e04e700 1 -- 172.21.15.112:0/1393629315 reap_dead start 2023-08-27T11:26:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.506+0000 7f0b7fab1700 1 -- 172.21.15.112:0/1393629315 shutdown_connections 2023-08-27T11:26:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.506+0000 7f0b7fab1700 1 -- 172.21.15.112:0/1393629315 >> 172.21.15.112:0/1393629315 conn(0x7f0b780d0000 msgr2=0x7f0b780d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.506+0000 7f0b7fab1700 1 -- 172.21.15.112:0/1393629315 shutdown_connections 2023-08-27T11:26:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.506+0000 7f0b7fab1700 1 -- 172.21.15.112:0/1393629315 wait complete. 2023-08-27T11:26:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.507+0000 7f0b7fab1700 1 Processor -- start 2023-08-27T11:26:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.507+0000 7f0b7fab1700 1 -- start start 2023-08-27T11:26:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7fab1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b64004850 0x7f0b7814e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7fab1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b780aefb0 0x7f0b781f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7fab1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b780708f0 0x7f0b78070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7fab1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780712b0 con 0x7f0b780aefb0 2023-08-27T11:26:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7fab1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7814eb50 con 0x7f0b64004850 2023-08-27T11:26:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7fab1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b781f4600 con 0x7f0b780708f0 2023-08-27T11:26:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7d04c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b780aefb0 0x7f0b781f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7d04c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b780aefb0 0x7f0b781f3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33780/0 (socket says 172.21.15.112:33780) 2023-08-27T11:26:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7d04c700 1 -- 172.21.15.112:0/149911136 learned_addr learned my addr 172.21.15.112:0/149911136 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7d84d700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b64004850 0x7f0b7814e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7e04e700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b780708f0 0x7f0b78070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7d04c700 1 -- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b780708f0 msgr2=0x7f0b78070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7d04c700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b780708f0 0x7f0b78070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7d04c700 1 -- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b64004850 msgr2=0x7f0b7814e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7d04c700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b64004850 0x7f0b7814e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.508+0000 7f0b7d04c700 1 -- 172.21.15.112:0/149911136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b781f48e0 con 0x7f0b780aefb0 2023-08-27T11:26:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.509+0000 7f0b7d04c700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b780aefb0 0x7f0b781f3dd0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0b680090b0 tx=0x7f0b680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.509+0000 7f0b6e7fc700 1 -- 172.21.15.112:0/149911136 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68021070 con 0x7f0b780aefb0 2023-08-27T11:26:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.509+0000 7f0b6e7fc700 1 -- 172.21.15.112:0/149911136 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b68004590 con 0x7f0b780aefb0 2023-08-27T11:26:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.509+0000 7f0b7fab1700 1 -- 172.21.15.112:0/149911136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b781f4bc0 con 0x7f0b780aefb0 2023-08-27T11:26:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.509+0000 7f0b6e7fc700 1 -- 172.21.15.112:0/149911136 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800e6c0 con 0x7f0b780aefb0 2023-08-27T11:26:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.509+0000 7f0b7fab1700 1 -- 172.21.15.112:0/149911136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b78151720 con 0x7f0b780aefb0 2023-08-27T11:26:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.510+0000 7f0b7fab1700 1 -- 172.21.15.112:0/149911136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b78151a90 con 0x7f0b780aefb0 2023-08-27T11:26:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.510+0000 7f0b6e7fc700 1 -- 172.21.15.112:0/149911136 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f0b68025070 con 0x7f0b780aefb0 2023-08-27T11:26:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.511+0000 7f0b6e7fc700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b5c06a440 0x7f0b5c06c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.511+0000 7f0b6e7fc700 1 -- 172.21.15.112:0/149911136 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0b680899f0 con 0x7f0b780aefb0 2023-08-27T11:26:52.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.511+0000 7f0b7d84d700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b5c06a440 0x7f0b5c06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.511+0000 7f0b7d84d700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b5c06a440 0x7f0b5c06c900 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0b781f3d30 tx=0x7f0b74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:52.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.515+0000 7f0b6e7fc700 1 -- 172.21.15.112:0/149911136 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0b680551d0 con 0x7f0b780aefb0 2023-08-27T11:26:52.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.706+0000 7f0b7fab1700 1 -- 172.21.15.112:0/149911136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b781518b0 con 0x7f0b780aefb0 2023-08-27T11:26:52.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.707+0000 7f0b6e7fc700 1 -- 172.21.15.112:0/149911136 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+14606 (secure 0 0 0) 0x7f0b680587f0 con 0x7f0b780aefb0 2023-08-27T11:26:52.708 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:52.709 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":98,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:51.918052+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:52.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.709+0000 7f0b7fab1700 1 -- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b5c06a440 msgr2=0x7f0b5c06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.709+0000 7f0b7fab1700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b5c06a440 0x7f0b5c06c900 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0b781f3d30 tx=0x7f0b74008040 comp rx=0 tx=0).stop 2023-08-27T11:26:52.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.709+0000 7f0b7fab1700 1 -- 172.21.15.112:0/149911136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b780aefb0 msgr2=0x7f0b781f3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.709+0000 7f0b7fab1700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b780aefb0 0x7f0b781f3dd0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0b680090b0 tx=0x7f0b680047c0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.709+0000 7f0b7fab1700 1 -- 172.21.15.112:0/149911136 shutdown_connections 2023-08-27T11:26:52.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.709+0000 7f0b7fab1700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b5c06a440 0x7f0b5c06c900 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.709+0000 7f0b7fab1700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b780708f0 0x7f0b78070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.709+0000 7f0b7fab1700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b64004850 0x7f0b7814e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.709+0000 7f0b7fab1700 1 --2- 172.21.15.112:0/149911136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b780aefb0 0x7f0b781f3dd0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.709+0000 7f0b7fab1700 1 -- 172.21.15.112:0/149911136 >> 172.21.15.112:0/149911136 conn(0x7f0b780d0000 msgr2=0x7f0b78071a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:52.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.709+0000 7f0b7fab1700 1 -- 172.21.15.112:0/149911136 shutdown_connections 2023-08-27T11:26:52.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.709+0000 7f0b7fab1700 1 -- 172.21.15.112:0/149911136 wait complete. 2023-08-27T11:26:52.722 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:26:52.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:26:52.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.844+0000 7fca33b74700 1 Processor -- start 2023-08-27T11:26:52.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.844+0000 7fca33b74700 1 -- start start 2023-08-27T11:26:52.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca33b74700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fca2c0b3440 0x7fca2c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca33b74700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fca2c15e6e0 0x7fca2c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca33b74700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2c0b5070 con 0x7fca2c146c20 2023-08-27T11:26:52.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca33b74700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2c147000 con 0x7fca2c0b3440 2023-08-27T11:26:52.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca33b74700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2c161740 con 0x7fca2c15e6e0 2023-08-27T11:26:52.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca3110f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fca2c0b3440 0x7fca2c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca3110f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fca2c0b3440 0x7fca2c15e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33790/0 (socket says 172.21.15.112:33790) 2023-08-27T11:26:52.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca3110f700 1 -- 172.21.15.112:0/1178845500 learned_addr learned my addr 172.21.15.112:0/1178845500 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:52.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca32111700 1 --2- 172.21.15.112:0/1178845500 >> v2:172.21.15.138:3300/0 conn(0x7fca2c15e6e0 0x7fca2c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca3110f700 1 -- 172.21.15.112:0/1178845500 >> v2:172.21.15.138:3300/0 conn(0x7fca2c15e6e0 msgr2=0x7fca2c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:52.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca3110f700 1 --2- 172.21.15.112:0/1178845500 >> v2:172.21.15.138:3300/0 conn(0x7fca2c15e6e0 0x7fca2c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca3110f700 1 -- 172.21.15.112:0/1178845500 >> v1:172.21.15.138:6789/0 conn(0x7fca2c146c20 legacy=0x7fca2c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:52.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca3110f700 1 -- 172.21.15.112:0/1178845500 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c161a20 con 0x7fca2c0b3440 2023-08-27T11:26:52.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca32111700 1 --2- 172.21.15.112:0/1178845500 >> v2:172.21.15.138:3300/0 conn(0x7fca2c15e6e0 0x7fca2c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:52.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.845+0000 7fca3110f700 1 --2- 172.21.15.112:0/1178845500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca2c0b3440 0x7fca2c15e1a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0b38c0 tx=0x7fca1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9549d2486df44722 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.846+0000 7fca3090e700 1 -- 172.21.15.112:0/1178845500 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c004230 con 0x7fca2c0b3440 2023-08-27T11:26:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.846+0000 7fca3090e700 1 -- 172.21.15.112:0/1178845500 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c004390 con 0x7fca2c0b3440 2023-08-27T11:26:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.846+0000 7fca3090e700 1 -- 172.21.15.112:0/1178845500 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0036a0 con 0x7fca2c0b3440 2023-08-27T11:26:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.846+0000 7fca33b74700 1 -- 172.21.15.112:0/1178845500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca2c0b3440 msgr2=0x7fca2c15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.846+0000 7fca33b74700 1 --2- 172.21.15.112:0/1178845500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca2c0b3440 0x7fca2c15e1a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0b38c0 tx=0x7fca1c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.846+0000 7fca33b74700 1 -- 172.21.15.112:0/1178845500 shutdown_connections 2023-08-27T11:26:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.846+0000 7fca33b74700 1 --2- 172.21.15.112:0/1178845500 >> v2:172.21.15.138:3300/0 conn(0x7fca2c15e6e0 0x7fca2c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.846+0000 7fca33b74700 1 --2- 172.21.15.112:0/1178845500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca2c0b3440 0x7fca2c15e1a0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.846+0000 7fca33b74700 1 -- 172.21.15.112:0/1178845500 >> 172.21.15.112:0/1178845500 conn(0x7fca2c0b1e90 msgr2=0x7fca2c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:52.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.846+0000 7fca33b74700 1 -- 172.21.15.112:0/1178845500 shutdown_connections 2023-08-27T11:26:52.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.846+0000 7fca33b74700 1 -- 172.21.15.112:0/1178845500 wait complete. 2023-08-27T11:26:52.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.847+0000 7fca33b74700 1 Processor -- start 2023-08-27T11:26:52.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.847+0000 7fca33b74700 1 -- start start 2023-08-27T11:26:52.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.848+0000 7fca33b74700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c0b3440 0x7fca2c1f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.848+0000 7fca33b74700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c146c20 0x7fca2c1f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.848+0000 7fca33b74700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca2c15e6e0 0x7fca2c0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.848+0000 7fca33b74700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c1f66b0 con 0x7fca2c15e6e0 2023-08-27T11:26:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.848+0000 7fca33b74700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c145070 con 0x7fca2c0b3440 2023-08-27T11:26:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.848+0000 7fca33b74700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c144210 con 0x7fca2c146c20 2023-08-27T11:26:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.848+0000 7fca32111700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca2c15e6e0 0x7fca2c0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.848+0000 7fca32111700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca2c15e6e0 0x7fca2c0aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33798/0 (socket says 172.21.15.112:33798) 2023-08-27T11:26:52.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.848+0000 7fca32111700 1 -- 172.21.15.112:0/1243068926 learned_addr learned my addr 172.21.15.112:0/1243068926 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:52.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.848+0000 7fca3110f700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c146c20 0x7fca2c1f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.848+0000 7fca31910700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c0b3440 0x7fca2c1f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.849+0000 7fca32111700 1 -- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c146c20 msgr2=0x7fca2c1f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.849+0000 7fca32111700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c146c20 0x7fca2c1f23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.849+0000 7fca32111700 1 -- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c0b3440 msgr2=0x7fca2c1f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:52.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.849+0000 7fca32111700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c0b3440 0x7fca2c1f6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:52.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.849+0000 7fca32111700 1 -- 172.21.15.112:0/1243068926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c009550 con 0x7fca2c15e6e0 2023-08-27T11:26:52.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.849+0000 7fca31910700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c0b3440 0x7fca2c1f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:52.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.849+0000 7fca32111700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca2c15e6e0 0x7fca2c0aeed0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fca2400cd80 tx=0x7fca2400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:52.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.849+0000 7fca227fc700 1 -- 172.21.15.112:0/1243068926 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2400f480 con 0x7fca2c15e6e0 2023-08-27T11:26:52.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.849+0000 7fca227fc700 1 -- 172.21.15.112:0/1243068926 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2400f600 con 0x7fca2c15e6e0 2023-08-27T11:26:52.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.849+0000 7fca227fc700 1 -- 172.21.15.112:0/1243068926 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2400f9b0 con 0x7fca2c15e6e0 2023-08-27T11:26:52.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.849+0000 7fca33b74700 1 -- 172.21.15.112:0/1243068926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2c1444f0 con 0x7fca2c15e6e0 2023-08-27T11:26:52.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.849+0000 7fca33b74700 1 -- 172.21.15.112:0/1243068926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2c144ab0 con 0x7fca2c15e6e0 2023-08-27T11:26:52.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.850+0000 7fca33b74700 1 -- 172.21.15.112:0/1243068926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2c067e90 con 0x7fca2c15e6e0 2023-08-27T11:26:52.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.850+0000 7fca227fc700 1 -- 172.21.15.112:0/1243068926 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fca2400b040 con 0x7fca2c15e6e0 2023-08-27T11:26:52.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.851+0000 7fca227fc700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fca1006a440 0x7fca1006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:52.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.851+0000 7fca227fc700 1 -- 172.21.15.112:0/1243068926 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fca24089120 con 0x7fca2c15e6e0 2023-08-27T11:26:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.851+0000 7fca31910700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fca1006a440 0x7fca1006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.851+0000 7fca31910700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fca1006a440 0x7fca1006c900 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fca2c145930 tx=0x7fca28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:52.855+0000 7fca227fc700 1 -- 172.21.15.112:0/1243068926 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fca24054640 con 0x7fca2c15e6e0 2023-08-27T11:26:53.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.044+0000 7fca33b74700 1 -- 172.21.15.112:0/1243068926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fca2c067a20 con 0x7fca2c15e6e0 2023-08-27T11:26:53.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.044+0000 7fca227fc700 1 -- 172.21.15.112:0/1243068926 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v98) v1 ==== 92+0+10 (secure 0 0 0) 0x7fca2401d020 con 0x7fca2c15e6e0 2023-08-27T11:26:53.046 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:26:53.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.047+0000 7fca33b74700 1 -- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fca1006a440 msgr2=0x7fca1006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.047+0000 7fca33b74700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fca1006a440 0x7fca1006c900 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fca2c145930 tx=0x7fca28009040 comp rx=0 tx=0).stop 2023-08-27T11:26:53.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.047+0000 7fca33b74700 1 -- 172.21.15.112:0/1243068926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca2c15e6e0 msgr2=0x7fca2c0aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.047+0000 7fca33b74700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca2c15e6e0 0x7fca2c0aeed0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fca2400cd80 tx=0x7fca2400a560 comp rx=0 tx=0).stop 2023-08-27T11:26:53.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.047+0000 7fca33b74700 1 -- 172.21.15.112:0/1243068926 shutdown_connections 2023-08-27T11:26:53.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.047+0000 7fca33b74700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fca1006a440 0x7fca1006c900 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.047+0000 7fca33b74700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca2c15e6e0 0x7fca2c0aeed0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.047+0000 7fca33b74700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c0b3440 0x7fca2c1f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.047+0000 7fca33b74700 1 --2- 172.21.15.112:0/1243068926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca2c146c20 0x7fca2c1f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.047+0000 7fca33b74700 1 -- 172.21.15.112:0/1243068926 >> 172.21.15.112:0/1243068926 conn(0x7fca2c0b1e90 msgr2=0x7fca2c142c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:53.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.047+0000 7fca33b74700 1 -- 172.21.15.112:0/1243068926 shutdown_connections 2023-08-27T11:26:53.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.047+0000 7fca33b74700 1 -- 172.21.15.112:0/1243068926 wait complete. 2023-08-27T11:26:53.060 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:26:53.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.182+0000 7f5bc17b2700 1 Processor -- start 2023-08-27T11:26:53.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.182+0000 7f5bc17b2700 1 -- start start 2023-08-27T11:26:53.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.182+0000 7f5bc17b2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5bbc157b80 0x7f5bbc0d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.182+0000 7f5bc17b2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bbc0d23f0 0x7f5bbc0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.182+0000 7f5bc17b2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5bbc0cde60 0x7f5bbc161020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.182+0000 7f5bc17b2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bbc15ddd0 con 0x7f5bbc0cde60 2023-08-27T11:26:53.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.182+0000 7f5bc17b2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bbc0d2850 con 0x7f5bbc157b80 2023-08-27T11:26:53.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.182+0000 7f5bc17b2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0ce2f0 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bbb7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5bbc0cde60 0x7f5bbc161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bbb7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5bbc0cde60 0x7f5bbc161020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33804/0 (socket says 172.21.15.112:33804) 2023-08-27T11:26:53.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bbb7fe700 1 -- 172.21.15.112:0/3888256209 learned_addr learned my addr 172.21.15.112:0/3888256209 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:53.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bbaffd700 1 --2- 172.21.15.112:0/3888256209 >> v2:172.21.15.138:3300/0 conn(0x7f5bbc157b80 0x7f5bbc0d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bba7fc700 1 --2- 172.21.15.112:0/3888256209 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bbc0d23f0 0x7f5bbc0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bbb7fe700 1 -- 172.21.15.112:0/3888256209 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bbc0d23f0 msgr2=0x7f5bbc0cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:53.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bbb7fe700 1 --2- 172.21.15.112:0/3888256209 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bbc0d23f0 0x7f5bbc0cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bbb7fe700 1 -- 172.21.15.112:0/3888256209 >> v2:172.21.15.138:3300/0 conn(0x7f5bbc157b80 msgr2=0x7f5bbc0d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:53.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bbb7fe700 1 --2- 172.21.15.112:0/3888256209 >> v2:172.21.15.138:3300/0 conn(0x7f5bbc157b80 0x7f5bbc0d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bbb7fe700 1 -- 172.21.15.112:0/3888256209 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc0cf0f0 con 0x7f5bbc0cde60 2023-08-27T11:26:53.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bbb7fe700 1 --2- 172.21.15.112:0/3888256209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bbc0cde60 0x7f5bbc161020 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0d3230 tx=0x7f5bb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18abe49517a8ad07 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:53.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bb9ffb700 1 -- 172.21.15.112:0/3888256209 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400e070 con 0x7f5bbc0cde60 2023-08-27T11:26:53.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bb9ffb700 1 -- 172.21.15.112:0/3888256209 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb400b8d0 con 0x7f5bbc0cde60 2023-08-27T11:26:53.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.183+0000 7f5bb9ffb700 1 -- 172.21.15.112:0/3888256209 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400bc80 con 0x7f5bbc0cde60 2023-08-27T11:26:53.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.184+0000 7f5bc17b2700 1 -- 172.21.15.112:0/3888256209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bbc0cde60 msgr2=0x7f5bbc161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.184+0000 7f5bc17b2700 1 --2- 172.21.15.112:0/3888256209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bbc0cde60 0x7f5bbc161020 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0d3230 tx=0x7f5bb400b330 comp rx=0 tx=0).stop 2023-08-27T11:26:53.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.184+0000 7f5bc17b2700 1 -- 172.21.15.112:0/3888256209 shutdown_connections 2023-08-27T11:26:53.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.184+0000 7f5bc17b2700 1 --2- 172.21.15.112:0/3888256209 >> v2:172.21.15.138:3300/0 conn(0x7f5bbc157b80 0x7f5bbc0d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.184+0000 7f5bc17b2700 1 --2- 172.21.15.112:0/3888256209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bbc0cde60 0x7f5bbc161020 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.184+0000 7f5bc17b2700 1 --2- 172.21.15.112:0/3888256209 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bbc0d23f0 0x7f5bbc0cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.184+0000 7f5bc17b2700 1 -- 172.21.15.112:0/3888256209 >> 172.21.15.112:0/3888256209 conn(0x7f5bbc0cfd50 msgr2=0x7f5bbc0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.184+0000 7f5bc17b2700 1 -- 172.21.15.112:0/3888256209 shutdown_connections 2023-08-27T11:26:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.184+0000 7f5bc17b2700 1 -- 172.21.15.112:0/3888256209 wait complete. 2023-08-27T11:26:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.185+0000 7f5bc17b2700 1 Processor -- start 2023-08-27T11:26:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.185+0000 7f5bc17b2700 1 -- start start 2023-08-27T11:26:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bc17b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bbc0cde60 0x7f5bbc1e41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bc17b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bbc0d23f0 0x7f5bbc158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bc17b2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bbc157b80 0x7f5bbc1594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bc17b2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc15ba90 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bc17b2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc145260 con 0x7f5bbc0cde60 2023-08-27T11:26:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bc17b2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc15aba0 con 0x7f5bbc157b80 2023-08-27T11:26:53.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bba7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bbc0d23f0 0x7f5bbc158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bba7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bbc0d23f0 0x7f5bbc158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33806/0 (socket says 172.21.15.112:33806) 2023-08-27T11:26:53.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bba7fc700 1 -- 172.21.15.112:0/1145555563 learned_addr learned my addr 172.21.15.112:0/1145555563 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:53.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bbaffd700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bbc0cde60 0x7f5bbc1e41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bba7fc700 1 -- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bbc157b80 msgr2=0x7f5bbc1594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bbb7fe700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bbc157b80 0x7f5bbc1594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bba7fc700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bbc157b80 0x7f5bbc1594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bba7fc700 1 -- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bbc0cde60 msgr2=0x7f5bbc1e41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bba7fc700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bbc0cde60 0x7f5bbc1e41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.186+0000 7f5bba7fc700 1 -- 172.21.15.112:0/1145555563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb4009b30 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.187+0000 7f5bba7fc700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bbc0d23f0 0x7f5bbc158fb0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000e840 tx=0x7f5bb000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:53.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.187+0000 7f5ba37fe700 1 -- 172.21.15.112:0/1145555563 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0013610 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.187+0000 7f5ba37fe700 1 -- 172.21.15.112:0/1145555563 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb0013770 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.187+0000 7f5bc17b2700 1 -- 172.21.15.112:0/1145555563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc15ae80 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.187+0000 7f5ba37fe700 1 -- 172.21.15.112:0/1145555563 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0013b00 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.187+0000 7f5bc17b2700 1 -- 172.21.15.112:0/1145555563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc15b440 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.188+0000 7f5bc17b2700 1 -- 172.21.15.112:0/1145555563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bbc067e90 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.188+0000 7f5ba37fe700 1 -- 172.21.15.112:0/1145555563 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f5bb0013c60 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.189+0000 7f5ba37fe700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ba806a440 0x7f5ba806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.189+0000 7f5ba37fe700 1 -- 172.21.15.112:0/1145555563 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5bb0088840 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.189+0000 7f5bbaffd700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ba806a440 0x7f5ba806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.189+0000 7f5bbaffd700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ba806a440 0x7f5ba806c900 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009a10 tx=0x7f5bac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:53.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.193+0000 7f5ba37fe700 1 -- 172.21.15.112:0/1145555563 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5bb00578f0 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.380+0000 7f5bc17b2700 1 -- 172.21.15.112:0/1145555563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5bbc067a20 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.381+0000 7f5ba37fe700 1 -- 172.21.15.112:0/1145555563 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5bb0009440 con 0x7f5bbc0d23f0 2023-08-27T11:26:53.383 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:26:53.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.383+0000 7f5bc17b2700 1 -- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ba806a440 msgr2=0x7f5ba806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.383+0000 7f5bc17b2700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ba806a440 0x7f5ba806c900 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009a10 tx=0x7f5bac008040 comp rx=0 tx=0).stop 2023-08-27T11:26:53.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.383+0000 7f5bc17b2700 1 -- 172.21.15.112:0/1145555563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bbc0d23f0 msgr2=0x7f5bbc158fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.383+0000 7f5bc17b2700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bbc0d23f0 0x7f5bbc158fb0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000e840 tx=0x7f5bb000ec00 comp rx=0 tx=0).stop 2023-08-27T11:26:53.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.383+0000 7f5bc17b2700 1 -- 172.21.15.112:0/1145555563 shutdown_connections 2023-08-27T11:26:53.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.383+0000 7f5bc17b2700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ba806a440 0x7f5ba806c900 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.383+0000 7f5bc17b2700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bbc157b80 0x7f5bbc1594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.383+0000 7f5bc17b2700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bbc0cde60 0x7f5bbc1e41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.383+0000 7f5bc17b2700 1 --2- 172.21.15.112:0/1145555563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bbc0d23f0 0x7f5bbc158fb0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.383+0000 7f5bc17b2700 1 -- 172.21.15.112:0/1145555563 >> 172.21.15.112:0/1145555563 conn(0x7f5bbc0cfd50 msgr2=0x7f5bbc0c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:53.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.383+0000 7f5bc17b2700 1 -- 172.21.15.112:0/1145555563 shutdown_connections 2023-08-27T11:26:53.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.383+0000 7f5bc17b2700 1 -- 172.21.15.112:0/1145555563 wait complete. 2023-08-27T11:26:53.397 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:26:53.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.515+0000 7ff71ddf7700 1 Processor -- start 2023-08-27T11:26:53.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.515+0000 7ff71ddf7700 1 -- start start 2023-08-27T11:26:53.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff71ddf7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7180b1400 0x7ff7180cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff71ddf7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff718161ce0 0x7ff71806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff71ddf7700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff718161850 con 0x7ff7180b17e0 2023-08-27T11:26:53.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff71ddf7700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7180cfc10 con 0x7ff718161ce0 2023-08-27T11:26:53.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff71ddf7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff718161200 con 0x7ff7180b1400 2023-08-27T11:26:53.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff716ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff7180b17e0 0x7ff71806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56428/0 (socket says 172.21.15.112:56428) 2023-08-27T11:26:53.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff716ffd700 1 -- 172.21.15.112:0/242532577 learned_addr learned my addr 172.21.15.112:0/242532577 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:53.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff717fff700 1 --2- 172.21.15.112:0/242532577 >> v2:172.21.15.138:3300/0 conn(0x7ff718161ce0 0x7ff71806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff7177fe700 1 --2- 172.21.15.112:0/242532577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7180b1400 0x7ff7180cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083530151 0 0) 0x7ff718161850 con 0x7ff7180b17e0 2023-08-27T11:26:53.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7000035b0 con 0x7ff7180b17e0 2023-08-27T11:26:53.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1331665486 0 0) 0x7ff7000035b0 con 0x7ff7180b17e0 2023-08-27T11:26:53.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7180b1400 msgr2=0x7ff7180cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:53.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff7167fc700 1 --2- 172.21.15.112:0/242532577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7180b1400 0x7ff7180cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 >> v2:172.21.15.138:3300/0 conn(0x7ff718161ce0 msgr2=0x7ff71806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:53.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff7167fc700 1 --2- 172.21.15.112:0/242532577 >> v2:172.21.15.138:3300/0 conn(0x7ff718161ce0 0x7ff71806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7181614e0 con 0x7ff7180b17e0 2023-08-27T11:26:53.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.516+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7ff708001670 con 0x7ff7180b17e0 2023-08-27T11:26:53.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 >> v1:172.21.15.112:6789/0 conn(0x7ff7180b17e0 legacy=0x7ff71806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7167fc700 1 --2- 172.21.15.112:0/242532577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff700003a00 0x7ff700003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7167fc700 1 --2- 172.21.15.112:0/242532577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7180b1400 0x7ff700004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7167fc700 1 --2- 172.21.15.112:0/242532577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff700004d90 0x7ff700005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7180cfc10 con 0x7ff700003a00 2023-08-27T11:26:53.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff718161200 con 0x7ff700004d90 2023-08-27T11:26:53.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70000ba10 con 0x7ff7180b1400 2023-08-27T11:26:53.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7177fe700 1 --2- 172.21.15.112:0/242532577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff700003a00 0x7ff700003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff716ffd700 1 --2- 172.21.15.112:0/242532577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7180b1400 0x7ff700004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff717fff700 1 --2- 172.21.15.112:0/242532577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff700004d90 0x7ff700005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7177fe700 1 -- 172.21.15.112:0/242532577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7180b1400 msgr2=0x7ff700004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:53.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7177fe700 1 --2- 172.21.15.112:0/242532577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7180b1400 0x7ff700004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7177fe700 1 -- 172.21.15.112:0/242532577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff700004d90 msgr2=0x7ff700005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:53.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7177fe700 1 --2- 172.21.15.112:0/242532577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff700004d90 0x7ff700005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7177fe700 1 -- 172.21.15.112:0/242532577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7000035b0 con 0x7ff700003a00 2023-08-27T11:26:53.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7177fe700 1 --2- 172.21.15.112:0/242532577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff700003a00 0x7ff700003e20 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009b60 tx=0x7ff700005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fc6e69bb7999e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:53.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c00e6c0 con 0x7ff700003a00 2023-08-27T11:26:53.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70c00e820 con 0x7ff700003a00 2023-08-27T11:26:53.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.517+0000 7ff7167fc700 1 -- 172.21.15.112:0/242532577 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c00eb90 con 0x7ff700003a00 2023-08-27T11:26:53.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.518+0000 7ff71ddf7700 1 -- 172.21.15.112:0/242532577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff700003a00 msgr2=0x7ff700003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.518+0000 7ff71ddf7700 1 --2- 172.21.15.112:0/242532577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff700003a00 0x7ff700003e20 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009b60 tx=0x7ff700005880 comp rx=0 tx=0).stop 2023-08-27T11:26:53.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.518+0000 7ff717fff700 1 -- 172.21.15.112:0/242532577 reap_dead start 2023-08-27T11:26:53.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.518+0000 7ff71ddf7700 1 -- 172.21.15.112:0/242532577 shutdown_connections 2023-08-27T11:26:53.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.518+0000 7ff71ddf7700 1 -- 172.21.15.112:0/242532577 >> 172.21.15.112:0/242532577 conn(0x7ff7180d0000 msgr2=0x7ff7180d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:53.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.518+0000 7ff71ddf7700 1 -- 172.21.15.112:0/242532577 shutdown_connections 2023-08-27T11:26:53.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.518+0000 7ff71ddf7700 1 -- 172.21.15.112:0/242532577 wait complete. 2023-08-27T11:26:53.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.519+0000 7ff71ddf7700 1 Processor -- start 2023-08-27T11:26:53.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.520+0000 7ff71ddf7700 1 -- start start 2023-08-27T11:26:53.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.520+0000 7ff71ddf7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff700003a00 0x7ff7180c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.520+0000 7ff71ddf7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff718151060 0x7ff7180c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.520+0000 7ff71ddf7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff71815b7b0 0x7ff71815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.520+0000 7ff71ddf7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71815c170 con 0x7ff71815b7b0 2023-08-27T11:26:53.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.520+0000 7ff71ddf7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7180c8b50 con 0x7ff700003a00 2023-08-27T11:26:53.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.520+0000 7ff71ddf7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7180c86c0 con 0x7ff718151060 2023-08-27T11:26:53.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff717fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff71815b7b0 0x7ff71815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff717fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff71815b7b0 0x7ff71815bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33826/0 (socket says 172.21.15.112:33826) 2023-08-27T11:26:53.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff717fff700 1 -- 172.21.15.112:0/2547836461 learned_addr learned my addr 172.21.15.112:0/2547836461 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:53.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff7177fe700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff700003a00 0x7ff7180c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff716ffd700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff718151060 0x7ff7180c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff717fff700 1 -- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff718151060 msgr2=0x7ff7180c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff717fff700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff718151060 0x7ff7180c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff717fff700 1 -- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff700003a00 msgr2=0x7ff7180c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff717fff700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff700003a00 0x7ff7180c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff717fff700 1 -- 172.21.15.112:0/2547836461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7180c7ab0 con 0x7ff71815b7b0 2023-08-27T11:26:53.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff7177fe700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff700003a00 0x7ff7180c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:53.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff716ffd700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff718151060 0x7ff7180c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:53.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff717fff700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff71815b7b0 0x7ff71815bc30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ff710007d10 tx=0x7ff7100114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:53.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff6fffff700 1 -- 172.21.15.112:0/2547836461 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71000e070 con 0x7ff71815b7b0 2023-08-27T11:26:53.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff6fffff700 1 -- 172.21.15.112:0/2547836461 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff710003c30 con 0x7ff71815b7b0 2023-08-27T11:26:53.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff71ddf7700 1 -- 172.21.15.112:0/2547836461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7180c7d90 con 0x7ff71815b7b0 2023-08-27T11:26:53.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff6fffff700 1 -- 172.21.15.112:0/2547836461 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710003e20 con 0x7ff71815b7b0 2023-08-27T11:26:53.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.521+0000 7ff71ddf7700 1 -- 172.21.15.112:0/2547836461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7180c8350 con 0x7ff71815b7b0 2023-08-27T11:26:53.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.522+0000 7ff71ddf7700 1 -- 172.21.15.112:0/2547836461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7180c8850 con 0x7ff71815b7b0 2023-08-27T11:26:53.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.523+0000 7ff6fffff700 1 -- 172.21.15.112:0/2547836461 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7ff710011a40 con 0x7ff71815b7b0 2023-08-27T11:26:53.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.523+0000 7ff6fffff700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff70406a440 0x7ff70406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.523+0000 7ff6fffff700 1 -- 172.21.15.112:0/2547836461 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff710088c60 con 0x7ff71815b7b0 2023-08-27T11:26:53.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.523+0000 7ff7177fe700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff70406a440 0x7ff70406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.524+0000 7ff7177fe700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff70406a440 0x7ff70406c900 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff70c000c00 tx=0x7ff70c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:53.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.527+0000 7ff6fffff700 1 -- 172.21.15.112:0/2547836461 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff710054180 con 0x7ff71815b7b0 2023-08-27T11:26:53.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.720+0000 7ff71ddf7700 1 -- 172.21.15.112:0/2547836461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff718067a20 con 0x7ff71815b7b0 2023-08-27T11:26:53.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.720+0000 7ff6fffff700 1 -- 172.21.15.112:0/2547836461 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v98) v1 ==== 84+0+126 (secure 0 0 0) 0x7ff71005be60 con 0x7ff71815b7b0 2023-08-27T11:26:53.723 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:53.723 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:26:53.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:53.723 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2023-08-27T11:26:53.724 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "rbd" 2023-08-27T11:26:53.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:26:53.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:26:53.725 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 2, 2023-08-27T11:26:53.725 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2023-08-27T11:26:53.725 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:26:53.725 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:26:53.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.722+0000 7ff71ddf7700 1 -- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff70406a440 msgr2=0x7ff70406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.722+0000 7ff71ddf7700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff70406a440 0x7ff70406c900 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff70c000c00 tx=0x7ff70c008030 comp rx=0 tx=0).stop 2023-08-27T11:26:53.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.722+0000 7ff71ddf7700 1 -- 172.21.15.112:0/2547836461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff71815b7b0 msgr2=0x7ff71815bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.722+0000 7ff71ddf7700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff71815b7b0 0x7ff71815bc30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ff710007d10 tx=0x7ff7100114b0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.722+0000 7ff71ddf7700 1 -- 172.21.15.112:0/2547836461 shutdown_connections 2023-08-27T11:26:53.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.722+0000 7ff71ddf7700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff70406a440 0x7ff70406c900 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.722+0000 7ff71ddf7700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff71815b7b0 0x7ff71815bc30 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.722+0000 7ff71ddf7700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff700003a00 0x7ff7180c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.722+0000 7ff71ddf7700 1 --2- 172.21.15.112:0/2547836461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff718151060 0x7ff7180c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.722+0000 7ff71ddf7700 1 -- 172.21.15.112:0/2547836461 >> 172.21.15.112:0/2547836461 conn(0x7ff7180d0000 msgr2=0x7ff71815fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:53.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.722+0000 7ff71ddf7700 1 -- 172.21.15.112:0/2547836461 shutdown_connections 2023-08-27T11:26:53.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.722+0000 7ff71ddf7700 1 -- 172.21.15.112:0/2547836461 wait complete. 2023-08-27T11:26:53.736 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:26:53.736 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:26:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.859+0000 7faf78de3700 1 Processor -- start 2023-08-27T11:26:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.859+0000 7faf78de3700 1 -- start start 2023-08-27T11:26:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.859+0000 7faf78de3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faf740b3de0 0x7faf7415e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf78de3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf74161390 con 0x7faf7406f790 2023-08-27T11:26:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf78de3700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf740b35f0 con 0x7faf74157ae0 2023-08-27T11:26:53.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf78de3700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf740b1aa0 con 0x7faf740b3de0 2023-08-27T11:26:53.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf72d9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7faf7406f790 0x7faf740b05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56448/0 (socket says 172.21.15.112:56448) 2023-08-27T11:26:53.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf72d9d700 1 -- 172.21.15.112:0/3686113033 learned_addr learned my addr 172.21.15.112:0/3686113033 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:53.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664831089 0 0) 0x7faf74161390 con 0x7faf7406f790 2023-08-27T11:26:53.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf71d9b700 1 --2- 172.21.15.112:0/3686113033 >> v2:172.21.15.138:3300/0 conn(0x7faf740b3de0 0x7faf7415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf600035b0 con 0x7faf7406f790 2023-08-27T11:26:53.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2694642829 0 0) 0x7faf600035b0 con 0x7faf7406f790 2023-08-27T11:26:53.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 >> v2:172.21.15.138:3300/0 conn(0x7faf740b3de0 msgr2=0x7faf7415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:53.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 --2- 172.21.15.112:0/3686113033 >> v2:172.21.15.138:3300/0 conn(0x7faf740b3de0 0x7faf7415e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 >> v1:172.21.15.138:6789/0 conn(0x7faf74157ae0 legacy=0x7faf74161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf740b4330 con 0x7faf7406f790 2023-08-27T11:26:53.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7faf6c0019f0 con 0x7faf7406f790 2023-08-27T11:26:53.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 >> v1:172.21.15.112:6789/0 conn(0x7faf7406f790 legacy=0x7faf740b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 --2- 172.21.15.112:0/3686113033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf60003a00 0x7faf60003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 --2- 172.21.15.112:0/3686113033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf60004850 0x7faf60004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 --2- 172.21.15.112:0/3686113033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf60005210 0x7faf6000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740b1aa0 con 0x7faf60004850 2023-08-27T11:26:53.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6000c0b0 con 0x7faf60005210 2023-08-27T11:26:53.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6000c270 con 0x7faf60003a00 2023-08-27T11:26:53.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf71d9b700 1 --2- 172.21.15.112:0/3686113033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf60004850 0x7faf60004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.860+0000 7faf7259c700 1 --2- 172.21.15.112:0/3686113033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf60003a00 0x7faf60003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf72d9d700 1 --2- 172.21.15.112:0/3686113033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf60005210 0x7faf6000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf71d9b700 1 -- 172.21.15.112:0/3686113033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf60003a00 msgr2=0x7faf60003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:53.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf71d9b700 1 --2- 172.21.15.112:0/3686113033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf60003a00 0x7faf60003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf71d9b700 1 -- 172.21.15.112:0/3686113033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf60005210 msgr2=0x7faf6000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:53.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf71d9b700 1 --2- 172.21.15.112:0/3686113033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf60005210 0x7faf6000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf71d9b700 1 -- 172.21.15.112:0/3686113033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf600035b0 con 0x7faf60004850 2023-08-27T11:26:53.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf72d9d700 1 --2- 172.21.15.112:0/3686113033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf60005210 0x7faf6000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:53.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf7259c700 1 --2- 172.21.15.112:0/3686113033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf60003a00 0x7faf60003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:53.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf72d9d700 1 -- 172.21.15.112:0/3686113033 reap_dead start 2023-08-27T11:26:53.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf71d9b700 1 --2- 172.21.15.112:0/3686113033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf60004850 0x7faf60004cd0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7faf64009b60 tx=0x7faf60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959acb1a74afaa29 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:53.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6400e6c0 con 0x7faf60004850 2023-08-27T11:26:53.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6400e820 con 0x7faf60004850 2023-08-27T11:26:53.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf7159a700 1 -- 172.21.15.112:0/3686113033 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6400eb90 con 0x7faf60004850 2023-08-27T11:26:53.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf78de3700 1 -- 172.21.15.112:0/3686113033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf60004850 msgr2=0x7faf60004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.861+0000 7faf78de3700 1 --2- 172.21.15.112:0/3686113033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf60004850 0x7faf60004cd0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7faf64009b60 tx=0x7faf60005880 comp rx=0 tx=0).stop 2023-08-27T11:26:53.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.862+0000 7faf78de3700 1 -- 172.21.15.112:0/3686113033 shutdown_connections 2023-08-27T11:26:53.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.862+0000 7faf78de3700 1 --2- 172.21.15.112:0/3686113033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf60004850 0x7faf60004cd0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.862+0000 7faf78de3700 1 -- 172.21.15.112:0/3686113033 >> 172.21.15.112:0/3686113033 conn(0x7faf740b1be0 msgr2=0x7faf740b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:53.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.862+0000 7faf78de3700 1 -- 172.21.15.112:0/3686113033 shutdown_connections 2023-08-27T11:26:53.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.862+0000 7faf78de3700 1 -- 172.21.15.112:0/3686113033 wait complete. 2023-08-27T11:26:53.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.863+0000 7faf78de3700 1 Processor -- start 2023-08-27T11:26:53.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.863+0000 7faf78de3700 1 -- start start 2023-08-27T11:26:53.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.864+0000 7faf78de3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf60004850 0x7faf740af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.864+0000 7faf78de3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf741f3fa0 0x7faf741f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.864+0000 7faf78de3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf741f5f20 0x7faf740cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.864+0000 7faf78de3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf741f5550 con 0x7faf741f3fa0 2023-08-27T11:26:53.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.864+0000 7faf78de3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740af5e0 con 0x7faf741f5f20 2023-08-27T11:26:53.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.864+0000 7faf78de3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf741f4940 con 0x7faf60004850 2023-08-27T11:26:53.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.864+0000 7faf71d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf741f3fa0 0x7faf741f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.864+0000 7faf71d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf741f3fa0 0x7faf741f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33844/0 (socket says 172.21.15.112:33844) 2023-08-27T11:26:53.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.864+0000 7faf71d9b700 1 -- 172.21.15.112:0/4133594342 learned_addr learned my addr 172.21.15.112:0/4133594342 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:53.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf72d9d700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf741f5f20 0x7faf740cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf7259c700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf60004850 0x7faf740af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf71d9b700 1 -- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf60004850 msgr2=0x7faf740af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf71d9b700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf60004850 0x7faf740af0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf71d9b700 1 -- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf741f5f20 msgr2=0x7faf740cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:53.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf71d9b700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf741f5f20 0x7faf740cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:53.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf71d9b700 1 -- 172.21.15.112:0/4133594342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf741f4c20 con 0x7faf741f3fa0 2023-08-27T11:26:53.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf72d9d700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf741f5f20 0x7faf740cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:53.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf71d9b700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf741f3fa0 0x7faf741f59e0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7faf6000c080 tx=0x7faf640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:53.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf6affd700 1 -- 172.21.15.112:0/4133594342 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6400e6c0 con 0x7faf741f3fa0 2023-08-27T11:26:53.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf6affd700 1 -- 172.21.15.112:0/4133594342 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf64003ad0 con 0x7faf741f3fa0 2023-08-27T11:26:53.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf78de3700 1 -- 172.21.15.112:0/4133594342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf741f4f00 con 0x7faf741f3fa0 2023-08-27T11:26:53.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf6affd700 1 -- 172.21.15.112:0/4133594342 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64003e80 con 0x7faf741f3fa0 2023-08-27T11:26:53.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.865+0000 7faf78de3700 1 -- 172.21.15.112:0/4133594342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf74160610 con 0x7faf741f3fa0 2023-08-27T11:26:53.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.866+0000 7faf78de3700 1 -- 172.21.15.112:0/4133594342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf74067a20 con 0x7faf741f3fa0 2023-08-27T11:26:53.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.866+0000 7faf6affd700 1 -- 172.21.15.112:0/4133594342 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7faf64013470 con 0x7faf741f3fa0 2023-08-27T11:26:53.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.867+0000 7faf6affd700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf5406a440 0x7faf5406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:53.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.867+0000 7faf6affd700 1 -- 172.21.15.112:0/4133594342 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faf6408a0a0 con 0x7faf741f3fa0 2023-08-27T11:26:53.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.867+0000 7faf7259c700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf5406a440 0x7faf5406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:53.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.867+0000 7faf7259c700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf5406a440 0x7faf5406c900 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7faf7404ffc0 tx=0x7faf5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:53.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:53.871+0000 7faf6affd700 1 -- 172.21.15.112:0/4133594342 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7faf64055880 con 0x7faf741f3fa0 2023-08-27T11:26:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.059+0000 7faf78de3700 1 -- 172.21.15.112:0/4133594342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7faf74160e60 con 0x7faf741f3fa0 2023-08-27T11:26:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.562+0000 7faf6affd700 1 -- 172.21.15.112:0/4133594342 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v99) v1 ==== 138+0+0 (secure 0 0 0) 0x7faf64058ea0 con 0x7faf741f3fa0 2023-08-27T11:26:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.565+0000 7faf78de3700 1 -- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf5406a440 msgr2=0x7faf5406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.565+0000 7faf78de3700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf5406a440 0x7faf5406c900 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7faf7404ffc0 tx=0x7faf5c009040 comp rx=0 tx=0).stop 2023-08-27T11:26:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.565+0000 7faf78de3700 1 -- 172.21.15.112:0/4133594342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf741f3fa0 msgr2=0x7faf741f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.565+0000 7faf78de3700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf741f3fa0 0x7faf741f59e0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7faf6000c080 tx=0x7faf640047c0 comp rx=0 tx=0).stop 2023-08-27T11:26:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.565+0000 7faf78de3700 1 -- 172.21.15.112:0/4133594342 shutdown_connections 2023-08-27T11:26:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.565+0000 7faf78de3700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faf5406a440 0x7faf5406c900 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.565+0000 7faf78de3700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf741f5f20 0x7faf740cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.565+0000 7faf78de3700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf60004850 0x7faf740af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.565+0000 7faf78de3700 1 --2- 172.21.15.112:0/4133594342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf741f3fa0 0x7faf741f59e0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.565+0000 7faf78de3700 1 -- 172.21.15.112:0/4133594342 >> 172.21.15.112:0/4133594342 conn(0x7faf740b1be0 msgr2=0x7faf74071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.566+0000 7faf78de3700 1 -- 172.21.15.112:0/4133594342 shutdown_connections 2023-08-27T11:26:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.566+0000 7faf78de3700 1 -- 172.21.15.112:0/4133594342 wait complete. 2023-08-27T11:26:54.571 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2023-08-27T11:26:54.581 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.708+0000 7efc3cff5700 1 Processor -- start 2023-08-27T11:26:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.708+0000 7efc3cff5700 1 -- start start 2023-08-27T11:26:54.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3cff5700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efc3806fea0 0x7efc380ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:54.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3cff5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc380ccad0 0x7efc3815b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:54.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3cff5700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc380b08e0 con 0x7efc381445b0 2023-08-27T11:26:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3cff5700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3815bc70 con 0x7efc3806fea0 2023-08-27T11:26:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3cff5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc380ccee0 con 0x7efc380ccad0 2023-08-27T11:26:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3659c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7efc381445b0 0x7efc3806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56464/0 (socket says 172.21.15.112:56464) 2023-08-27T11:26:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3659c700 1 -- 172.21.15.112:0/1763897024 learned_addr learned my addr 172.21.15.112:0/1763897024 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155135643 0 0) 0x7efc380b08e0 con 0x7efc381445b0 2023-08-27T11:26:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc200035b0 con 0x7efc381445b0 2023-08-27T11:26:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc35d9b700 1 --2- 172.21.15.112:0/1763897024 >> v2:172.21.15.138:3300/0 conn(0x7efc3806fea0 0x7efc380ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc36d9d700 1 --2- 172.21.15.112:0/1763897024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc380ccad0 0x7efc3815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2351228171 0 0) 0x7efc200035b0 con 0x7efc381445b0 2023-08-27T11:26:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc380ccad0 msgr2=0x7efc3815b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:54.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3559a700 1 --2- 172.21.15.112:0/1763897024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc380ccad0 0x7efc3815b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:54.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 >> v2:172.21.15.138:3300/0 conn(0x7efc3806fea0 msgr2=0x7efc380ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:54.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.709+0000 7efc3559a700 1 --2- 172.21.15.112:0/1763897024 >> v2:172.21.15.138:3300/0 conn(0x7efc3806fea0 0x7efc380ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:54.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc380b0b30 con 0x7efc381445b0 2023-08-27T11:26:54.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7efc2c001670 con 0x7efc381445b0 2023-08-27T11:26:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 >> v1:172.21.15.112:6789/0 conn(0x7efc381445b0 legacy=0x7efc3806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc3559a700 1 --2- 172.21.15.112:0/1763897024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc20003a00 0x7efc20003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc3559a700 1 --2- 172.21.15.112:0/1763897024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc380ccad0 0x7efc20004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc3559a700 1 --2- 172.21.15.112:0/1763897024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc20004d90 0x7efc20005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3815bc70 con 0x7efc20004d90 2023-08-27T11:26:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc380ccee0 con 0x7efc20003a00 2023-08-27T11:26:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2000ba10 con 0x7efc380ccad0 2023-08-27T11:26:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc36d9d700 1 --2- 172.21.15.112:0/1763897024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc20004d90 0x7efc20005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc3659c700 1 --2- 172.21.15.112:0/1763897024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc20003a00 0x7efc20003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:54.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc35d9b700 1 --2- 172.21.15.112:0/1763897024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc380ccad0 0x7efc20004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:54.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc36d9d700 1 -- 172.21.15.112:0/1763897024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc380ccad0 msgr2=0x7efc20004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:54.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc36d9d700 1 --2- 172.21.15.112:0/1763897024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc380ccad0 0x7efc20004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:54.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc36d9d700 1 -- 172.21.15.112:0/1763897024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc20003a00 msgr2=0x7efc20003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:54.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc36d9d700 1 --2- 172.21.15.112:0/1763897024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc20003a00 0x7efc20003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc36d9d700 1 -- 172.21.15.112:0/1763897024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200035b0 con 0x7efc20004d90 2023-08-27T11:26:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc36d9d700 1 --2- 172.21.15.112:0/1763897024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc20004d90 0x7efc20005210 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7efc30009fc0 tx=0x7efc20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b8364133049610 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3000ec30 con 0x7efc20004d90 2023-08-27T11:26:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3000f040 con 0x7efc20004d90 2023-08-27T11:26:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.710+0000 7efc3559a700 1 -- 172.21.15.112:0/1763897024 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30014480 con 0x7efc20004d90 2023-08-27T11:26:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.711+0000 7efc3cff5700 1 -- 172.21.15.112:0/1763897024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc20004d90 msgr2=0x7efc20005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.711+0000 7efc3cff5700 1 --2- 172.21.15.112:0/1763897024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc20004d90 0x7efc20005210 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7efc30009fc0 tx=0x7efc20005880 comp rx=0 tx=0).stop 2023-08-27T11:26:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.711+0000 7efc36d9d700 1 -- 172.21.15.112:0/1763897024 reap_dead start 2023-08-27T11:26:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.711+0000 7efc3cff5700 1 -- 172.21.15.112:0/1763897024 shutdown_connections 2023-08-27T11:26:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.711+0000 7efc3cff5700 1 -- 172.21.15.112:0/1763897024 >> 172.21.15.112:0/1763897024 conn(0x7efc380b1e90 msgr2=0x7efc380b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.711+0000 7efc3cff5700 1 -- 172.21.15.112:0/1763897024 shutdown_connections 2023-08-27T11:26:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.711+0000 7efc3cff5700 1 -- 172.21.15.112:0/1763897024 wait complete. 2023-08-27T11:26:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.712+0000 7efc3cff5700 1 Processor -- start 2023-08-27T11:26:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.712+0000 7efc3cff5700 1 -- start start 2023-08-27T11:26:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc3cff5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc20004d90 0x7efc3813fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc3cff5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc3815a750 0x7efc380ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc3cff5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc3815abd0 0x7efc380acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:54.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc3cff5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc380ad450 con 0x7efc3815a750 2023-08-27T11:26:54.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc3cff5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3813eb90 con 0x7efc3815abd0 2023-08-27T11:26:54.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc3cff5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3813f6d0 con 0x7efc20004d90 2023-08-27T11:26:54.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc35d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc3815a750 0x7efc380ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:54.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc35d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc3815a750 0x7efc380ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33864/0 (socket says 172.21.15.112:33864) 2023-08-27T11:26:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc35d9b700 1 -- 172.21.15.112:0/1250537795 learned_addr learned my addr 172.21.15.112:0/1250537795 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc3659c700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc20004d90 0x7efc3813fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc35d9b700 1 -- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc20004d90 msgr2=0x7efc3813fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc35d9b700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc20004d90 0x7efc3813fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc35d9b700 1 -- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc3815abd0 msgr2=0x7efc380acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc36d9d700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc3815abd0 0x7efc380acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.713+0000 7efc35d9b700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc3815abd0 0x7efc380acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.714+0000 7efc35d9b700 1 -- 172.21.15.112:0/1250537795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc380b5430 con 0x7efc3815a750 2023-08-27T11:26:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.714+0000 7efc35d9b700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc3815a750 0x7efc380ac890 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7efc28011a20 tx=0x7efc28011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.714+0000 7efc26ffd700 1 -- 172.21.15.112:0/1250537795 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2800fa00 con 0x7efc3815a750 2023-08-27T11:26:54.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.714+0000 7efc26ffd700 1 -- 172.21.15.112:0/1250537795 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2800fb60 con 0x7efc3815a750 2023-08-27T11:26:54.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.714+0000 7efc3cff5700 1 -- 172.21.15.112:0/1250537795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3813eda0 con 0x7efc3815a750 2023-08-27T11:26:54.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.714+0000 7efc26ffd700 1 -- 172.21.15.112:0/1250537795 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28014440 con 0x7efc3815a750 2023-08-27T11:26:54.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.714+0000 7efc3cff5700 1 -- 172.21.15.112:0/1250537795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3813f360 con 0x7efc3815a750 2023-08-27T11:26:54.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.715+0000 7efc3cff5700 1 -- 172.21.15.112:0/1250537795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc38067e90 con 0x7efc3815a750 2023-08-27T11:26:54.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.715+0000 7efc26ffd700 1 -- 172.21.15.112:0/1250537795 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7efc280145a0 con 0x7efc3815a750 2023-08-27T11:26:54.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.716+0000 7efc26ffd700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc1806a440 0x7efc1806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:54.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.716+0000 7efc26ffd700 1 -- 172.21.15.112:0/1250537795 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5580+0+0 (secure 0 0 0) 0x7efc28089070 con 0x7efc3815a750 2023-08-27T11:26:54.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.716+0000 7efc3659c700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc1806a440 0x7efc1806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:54.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.716+0000 7efc3659c700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc1806a440 0x7efc1806c900 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7efc2c000fb0 tx=0x7efc2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:54.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.720+0000 7efc26ffd700 1 -- 172.21.15.112:0/1250537795 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efc280545c0 con 0x7efc3815a750 2023-08-27T11:26:54.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:54.910+0000 7efc3cff5700 1 -- 172.21.15.112:0/1250537795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efc3813f840 con 0x7efc3815a750 2023-08-27T11:26:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.568+0000 7efc26ffd700 1 -- 172.21.15.112:0/1250537795 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v100) v1 ==== 170+0+0 (secure 0 0 0) 0x7efc28057be0 con 0x7efc3815a750 2023-08-27T11:26:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.572+0000 7efc3cff5700 1 -- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc1806a440 msgr2=0x7efc1806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.572+0000 7efc3cff5700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc1806a440 0x7efc1806c900 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7efc2c000fb0 tx=0x7efc2c009040 comp rx=0 tx=0).stop 2023-08-27T11:26:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.572+0000 7efc3cff5700 1 -- 172.21.15.112:0/1250537795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc3815a750 msgr2=0x7efc380ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.572+0000 7efc3cff5700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc3815a750 0x7efc380ac890 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7efc28011a20 tx=0x7efc28011de0 comp rx=0 tx=0).stop 2023-08-27T11:26:55.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.577+0000 7efc3cff5700 1 -- 172.21.15.112:0/1250537795 shutdown_connections 2023-08-27T11:26:55.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.577+0000 7efc3cff5700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc1806a440 0x7efc1806c900 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:55.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.577+0000 7efc3cff5700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc3815abd0 0x7efc380acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.577+0000 7efc3cff5700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc20004d90 0x7efc3813fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.578+0000 7efc3cff5700 1 --2- 172.21.15.112:0/1250537795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc3815a750 0x7efc380ac890 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.578+0000 7efc3cff5700 1 -- 172.21.15.112:0/1250537795 >> 172.21.15.112:0/1250537795 conn(0x7efc380b1e90 msgr2=0x7efc38073430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.579+0000 7efc3cff5700 1 -- 172.21.15.112:0/1250537795 shutdown_connections 2023-08-27T11:26:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.580+0000 7efc3cff5700 1 -- 172.21.15.112:0/1250537795 wait complete. 2023-08-27T11:26:55.584 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2023-08-27T11:26:55.597 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:26:55.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.718+0000 7f1efca7e700 1 Processor -- start 2023-08-27T11:26:55.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.718+0000 7f1efca7e700 1 -- start start 2023-08-27T11:26:55.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.719+0000 7f1efca7e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1ef81437c0 0x7f1ef80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:55.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.719+0000 7f1efca7e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ef806d5b0 0x7f1ef806da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:55.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.719+0000 7f1efca7e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef80cfd50 con 0x7f1ef8143ba0 2023-08-27T11:26:55.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.719+0000 7f1efca7e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef806df80 con 0x7f1ef81437c0 2023-08-27T11:26:55.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.719+0000 7f1efca7e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef80cf140 con 0x7f1ef806d5b0 2023-08-27T11:26:55.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.719+0000 7f1ef659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1ef81437c0 0x7f1ef80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:55.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.719+0000 7f1ef5d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1ef8143ba0 0x7f1ef80ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56484/0 (socket says 172.21.15.112:56484) 2023-08-27T11:26:55.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.719+0000 7f1ef5d9b700 1 -- 172.21.15.112:0/2749469002 learned_addr learned my addr 172.21.15.112:0/2749469002 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:55.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.720+0000 7f1ef6d9d700 1 --2- 172.21.15.112:0/2749469002 >> v2:172.21.15.138:3300/0 conn(0x7f1ef806d5b0 0x7f1ef806da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:55.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.720+0000 7f1ef659c700 1 -- 172.21.15.112:0/2749469002 >> v2:172.21.15.138:3300/0 conn(0x7f1ef806d5b0 msgr2=0x7f1ef806da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:55.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.720+0000 7f1ef659c700 1 --2- 172.21.15.112:0/2749469002 >> v2:172.21.15.138:3300/0 conn(0x7f1ef806d5b0 0x7f1ef806da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:55.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.720+0000 7f1ef659c700 1 -- 172.21.15.112:0/2749469002 >> v1:172.21.15.112:6789/0 conn(0x7f1ef8143ba0 legacy=0x7f1ef80ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:55.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.720+0000 7f1ef659c700 1 -- 172.21.15.112:0/2749469002 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef80cf420 con 0x7f1ef81437c0 2023-08-27T11:26:55.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.720+0000 7f1ef559a700 1 -- 172.21.15.112:0/2749469002 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229913091 0 0) 0x7f1ef80cfd50 con 0x7f1ef8143ba0 2023-08-27T11:26:55.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.720+0000 7f1ef659c700 1 --2- 172.21.15.112:0/2749469002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef81437c0 0x7f1ef80b4670 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80cfcc0 tx=0x7f1eec009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98728f6bb0842123 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:55.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.720+0000 7f1ef559a700 1 -- 172.21.15.112:0/2749469002 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec011830 con 0x7f1ef81437c0 2023-08-27T11:26:55.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.720+0000 7f1ef559a700 1 -- 172.21.15.112:0/2749469002 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec011990 con 0x7f1ef81437c0 2023-08-27T11:26:55.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.720+0000 7f1ef559a700 1 -- 172.21.15.112:0/2749469002 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec011d20 con 0x7f1ef81437c0 2023-08-27T11:26:55.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.720+0000 7f1efca7e700 1 -- 172.21.15.112:0/2749469002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef81437c0 msgr2=0x7f1ef80b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:55.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.720+0000 7f1efca7e700 1 --2- 172.21.15.112:0/2749469002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef81437c0 0x7f1ef80b4670 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80cfcc0 tx=0x7f1eec009b20 comp rx=0 tx=0).stop 2023-08-27T11:26:55.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.721+0000 7f1efca7e700 1 -- 172.21.15.112:0/2749469002 shutdown_connections 2023-08-27T11:26:55.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.721+0000 7f1efca7e700 1 --2- 172.21.15.112:0/2749469002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef81437c0 0x7f1ef80b4670 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:55.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.721+0000 7f1efca7e700 1 --2- 172.21.15.112:0/2749469002 >> v2:172.21.15.138:3300/0 conn(0x7f1ef806d5b0 0x7f1ef806da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:55.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.721+0000 7f1efca7e700 1 -- 172.21.15.112:0/2749469002 >> 172.21.15.112:0/2749469002 conn(0x7f1ef80b1e90 msgr2=0x7f1ef80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:55.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.721+0000 7f1efca7e700 1 -- 172.21.15.112:0/2749469002 shutdown_connections 2023-08-27T11:26:55.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.721+0000 7f1efca7e700 1 -- 172.21.15.112:0/2749469002 wait complete. 2023-08-27T11:26:55.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.722+0000 7f1efca7e700 1 Processor -- start 2023-08-27T11:26:55.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.722+0000 7f1efca7e700 1 -- start start 2023-08-27T11:26:55.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1efca7e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef806d5b0 0x7f1ef813b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1efca7e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef81437c0 0x7f1ef81391d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1efca7e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ef8143ba0 0x7f1ef806f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1efca7e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef81f6fd0 con 0x7f1ef81437c0 2023-08-27T11:26:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1efca7e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef813bb00 con 0x7f1ef8143ba0 2023-08-27T11:26:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1efca7e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef81f63c0 con 0x7f1ef806d5b0 2023-08-27T11:26:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ef5d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef81437c0 0x7f1ef81391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ef5d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef81437c0 0x7f1ef81391d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33878/0 (socket says 172.21.15.112:33878) 2023-08-27T11:26:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ef5d9b700 1 -- 172.21.15.112:0/1690899900 learned_addr learned my addr 172.21.15.112:0/1690899900 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ef6d9d700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ef8143ba0 0x7f1ef806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ef659c700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef806d5b0 0x7f1ef813b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:55.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ef5d9b700 1 -- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef806d5b0 msgr2=0x7f1ef813b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:55.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ef5d9b700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef806d5b0 0x7f1ef813b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:55.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ef5d9b700 1 -- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ef8143ba0 msgr2=0x7f1ef806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:55.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ef5d9b700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ef8143ba0 0x7f1ef806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:55.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ef5d9b700 1 -- 172.21.15.112:0/1690899900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec0094a0 con 0x7f1ef81437c0 2023-08-27T11:26:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ef5d9b700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef81437c0 0x7f1ef81391d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800c550 tx=0x7f1ee800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ee6ffd700 1 -- 172.21.15.112:0/1690899900 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8004c80 con 0x7f1ef81437c0 2023-08-27T11:26:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ee6ffd700 1 -- 172.21.15.112:0/1690899900 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee8004e00 con 0x7f1ef81437c0 2023-08-27T11:26:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1efca7e700 1 -- 172.21.15.112:0/1690899900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef81f66a0 con 0x7f1ef81437c0 2023-08-27T11:26:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1ee6ffd700 1 -- 172.21.15.112:0/1690899900 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8007310 con 0x7f1ef81437c0 2023-08-27T11:26:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.723+0000 7f1efca7e700 1 -- 172.21.15.112:0/1690899900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef81f6c60 con 0x7f1ef81437c0 2023-08-27T11:26:55.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.724+0000 7f1efca7e700 1 -- 172.21.15.112:0/1690899900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef815a970 con 0x7f1ef81437c0 2023-08-27T11:26:55.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.725+0000 7f1ee6ffd700 1 -- 172.21.15.112:0/1690899900 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f1ee8007470 con 0x7f1ef81437c0 2023-08-27T11:26:55.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.725+0000 7f1ee6ffd700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ed806a440 0x7f1ed806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:55.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.725+0000 7f1ee6ffd700 1 -- 172.21.15.112:0/1690899900 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f1ee8088630 con 0x7f1ef81437c0 2023-08-27T11:26:55.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.725+0000 7f1ef659c700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ed806a440 0x7f1ed806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:55.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.726+0000 7f1ef659c700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ed806a440 0x7f1ed806c900 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0090b0 tx=0x7f1eec008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:55.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.730+0000 7f1ee6ffd700 1 -- 172.21.15.112:0/1690899900 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1ef815a970 con 0x7f1ef81437c0 2023-08-27T11:26:55.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:55.963+0000 7f1efca7e700 1 -- 172.21.15.112:0/1690899900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1ef815a970 con 0x7f1ef81437c0 2023-08-27T11:26:56.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.568+0000 7f1ee6ffd700 1 -- 172.21.15.112:0/1690899900 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v63) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1ef815a970 con 0x7f1ef81437c0 2023-08-27T11:26:56.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.572+0000 7f1efca7e700 1 -- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ed806a440 msgr2=0x7f1ed806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:56.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.572+0000 7f1efca7e700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ed806a440 0x7f1ed806c900 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0090b0 tx=0x7f1eec008030 comp rx=0 tx=0).stop 2023-08-27T11:26:56.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.572+0000 7f1efca7e700 1 -- 172.21.15.112:0/1690899900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef81437c0 msgr2=0x7f1ef81391d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:56.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.572+0000 7f1efca7e700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef81437c0 0x7f1ef81391d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800c550 tx=0x7f1ee800c860 comp rx=0 tx=0).stop 2023-08-27T11:26:56.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.574+0000 7f1efca7e700 1 -- 172.21.15.112:0/1690899900 shutdown_connections 2023-08-27T11:26:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.574+0000 7f1efca7e700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1ed806a440 0x7f1ed806c900 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.574+0000 7f1efca7e700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ef8143ba0 0x7f1ef806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.574+0000 7f1efca7e700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef806d5b0 0x7f1ef813b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.574+0000 7f1efca7e700 1 --2- 172.21.15.112:0/1690899900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef81437c0 0x7f1ef81391d0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.574+0000 7f1efca7e700 1 -- 172.21.15.112:0/1690899900 >> 172.21.15.112:0/1690899900 conn(0x7f1ef80b1e90 msgr2=0x7f1ef8071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.575+0000 7f1efca7e700 1 -- 172.21.15.112:0/1690899900 shutdown_connections 2023-08-27T11:26:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.575+0000 7f1efca7e700 1 -- 172.21.15.112:0/1690899900 wait complete. 2023-08-27T11:26:56.579 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 17 and data pool 18 2023-08-27T11:26:56.590 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:56.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.709+0000 7f637e716700 1 Processor -- start 2023-08-27T11:26:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.709+0000 7f637e716700 1 -- start start 2023-08-27T11:26:56.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f637e716700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63780b1400 0x7f63780cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:56.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f637e716700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6378161850 con 0x7f6378161ce0 2023-08-27T11:26:56.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f637e716700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63780cfc10 con 0x7f63780b17e0 2023-08-27T11:26:56.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f637e716700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6378161200 con 0x7f63780b1400 2023-08-27T11:26:56.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f637ccb3700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6378161ce0 0x7f637806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56518/0 (socket says 172.21.15.112:56518) 2023-08-27T11:26:56.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f637ccb3700 1 -- 172.21.15.112:0/317374107 learned_addr learned my addr 172.21.15.112:0/317374107 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:56.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6377fff700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63780b1400 0x7f63780cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:56.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363866109 0 0) 0x7f6378161850 con 0x7f6378161ce0 2023-08-27T11:26:56.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63600035b0 con 0x7f6378161ce0 2023-08-27T11:26:56.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1959668089 0 0) 0x7f63600035b0 con 0x7f6378161ce0 2023-08-27T11:26:56.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63780b1400 msgr2=0x7f63780cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:56.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63780b1400 0x7f63780cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 >> v1:172.21.15.138:6789/0 conn(0x7f63780b17e0 legacy=0x7f637806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:56.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63781614e0 con 0x7f6378161ce0 2023-08-27T11:26:56.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f63700019f0 con 0x7f6378161ce0 2023-08-27T11:26:56.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 >> v1:172.21.15.112:6789/0 conn(0x7f6378161ce0 legacy=0x7f637806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:56.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6360003a30 0x7f6360003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:56.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63780b1400 0x7f6360004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:56.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6360004d90 0x7f6360005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:56.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6378161200 con 0x7f6360003a30 2023-08-27T11:26:56.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636000bac0 con 0x7f6360004d90 2023-08-27T11:26:56.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636000bc80 con 0x7f63780b1400 2023-08-27T11:26:56.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.710+0000 7f6377fff700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6360003a30 0x7f6360003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:56.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f637ccb3700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6360004d90 0x7f6360005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:56.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f63777fe700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63780b1400 0x7f6360004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:56.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f6377fff700 1 -- 172.21.15.112:0/317374107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63780b1400 msgr2=0x7f6360004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:56.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f6377fff700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63780b1400 0x7f6360004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f6377fff700 1 -- 172.21.15.112:0/317374107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6360004d90 msgr2=0x7f6360005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:56.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f6377fff700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6360004d90 0x7f6360005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f6377fff700 1 -- 172.21.15.112:0/317374107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600035b0 con 0x7f6360003a30 2023-08-27T11:26:56.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f637ccb3700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6360004d90 0x7f6360005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:56.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f63777fe700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63780b1400 0x7f6360004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:56.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f6377fff700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6360003a30 0x7f6360003e10 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f6368009b60 tx=0x7f6360005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe66058e3e595dc7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:56.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636800e6c0 con 0x7f6360003a30 2023-08-27T11:26:56.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636800e820 con 0x7f6360003a30 2023-08-27T11:26:56.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f6376ffd700 1 -- 172.21.15.112:0/317374107 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636800eb90 con 0x7f6360003a30 2023-08-27T11:26:56.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f637e716700 1 -- 172.21.15.112:0/317374107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6360003a30 msgr2=0x7f6360003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:56.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f637e716700 1 --2- 172.21.15.112:0/317374107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6360003a30 0x7f6360003e10 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f6368009b60 tx=0x7f6360005880 comp rx=0 tx=0).stop 2023-08-27T11:26:56.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f637ccb3700 1 -- 172.21.15.112:0/317374107 reap_dead start 2023-08-27T11:26:56.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.711+0000 7f637e716700 1 -- 172.21.15.112:0/317374107 shutdown_connections 2023-08-27T11:26:56.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.712+0000 7f637e716700 1 -- 172.21.15.112:0/317374107 >> 172.21.15.112:0/317374107 conn(0x7f63780d0000 msgr2=0x7f63780d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:56.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.712+0000 7f637e716700 1 -- 172.21.15.112:0/317374107 shutdown_connections 2023-08-27T11:26:56.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.712+0000 7f637e716700 1 -- 172.21.15.112:0/317374107 wait complete. 2023-08-27T11:26:56.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.713+0000 7f637e716700 1 Processor -- start 2023-08-27T11:26:56.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.713+0000 7f637e716700 1 -- start start 2023-08-27T11:26:56.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f637e716700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6360003a30 0x7f63780c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:56.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f637e716700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63780c5570 0x7f63781482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:56.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f637e716700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63781f65b0 0x7f63781f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:56.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f637e716700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63781f4290 con 0x7f6360003a30 2023-08-27T11:26:56.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f637e716700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63781f6f80 con 0x7f63780c5570 2023-08-27T11:26:56.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f637e716700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63780c5ce0 con 0x7f63781f65b0 2023-08-27T11:26:56.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f6377fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6360003a30 0x7f63780c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:56.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f6377fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6360003a30 0x7f63780c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33912/0 (socket says 172.21.15.112:33912) 2023-08-27T11:26:56.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f6377fff700 1 -- 172.21.15.112:0/3908512008 learned_addr learned my addr 172.21.15.112:0/3908512008 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:56.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f63777fe700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63780c5570 0x7f63781482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:56.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f637ccb3700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63781f65b0 0x7f63781f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:56.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f6377fff700 1 -- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63781f65b0 msgr2=0x7f63781f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:56.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f6377fff700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63781f65b0 0x7f63781f6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f6377fff700 1 -- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63780c5570 msgr2=0x7f63781482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:56.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f6377fff700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63780c5570 0x7f63781482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f6377fff700 1 -- 172.21.15.112:0/3908512008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63780c5fc0 con 0x7f6360003a30 2023-08-27T11:26:56.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f63777fe700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63780c5570 0x7f63781482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:56.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f6377fff700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6360003a30 0x7f63780c9f40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f63600047b0 tx=0x7f63680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:56.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f6374ff9700 1 -- 172.21.15.112:0/3908512008 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636800e6c0 con 0x7f6360003a30 2023-08-27T11:26:56.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f6374ff9700 1 -- 172.21.15.112:0/3908512008 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6368003ab0 con 0x7f6360003a30 2023-08-27T11:26:56.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f6374ff9700 1 -- 172.21.15.112:0/3908512008 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368003e60 con 0x7f6360003a30 2023-08-27T11:26:56.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f637e716700 1 -- 172.21.15.112:0/3908512008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63780c62a0 con 0x7f6360003a30 2023-08-27T11:26:56.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.714+0000 7f637e716700 1 -- 172.21.15.112:0/3908512008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63780c59f0 con 0x7f6360003a30 2023-08-27T11:26:56.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.715+0000 7f637e716700 1 -- 172.21.15.112:0/3908512008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6378067a20 con 0x7f6360003a30 2023-08-27T11:26:56.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.716+0000 7f6374ff9700 1 -- 172.21.15.112:0/3908512008 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f6368013470 con 0x7f6360003a30 2023-08-27T11:26:56.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.716+0000 7f6374ff9700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f636406a440 0x7f636406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:56.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.716+0000 7f6374ff9700 1 -- 172.21.15.112:0/3908512008 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f636808a350 con 0x7f6360003a30 2023-08-27T11:26:56.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.716+0000 7f63777fe700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f636406a440 0x7f636406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:56.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.717+0000 7f63777fe700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f636406a440 0x7f636406c900 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f636c007b50 tx=0x7f636c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:56.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.720+0000 7f6374ff9700 1 -- 172.21.15.112:0/3908512008 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f63680557e0 con 0x7f6360003a30 2023-08-27T11:26:56.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.909+0000 7f637e716700 1 -- 172.21.15.112:0/3908512008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63780b2aa0 con 0x7f6360003a30 2023-08-27T11:26:56.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.910+0000 7f6374ff9700 1 -- 172.21.15.112:0/3908512008 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+18340 (secure 0 0 0) 0x7f6368058e00 con 0x7f6360003a30 2023-08-27T11:26:56.912 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:56.912 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":101,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:55.964954+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:54.078457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:54.929983+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:56.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.912+0000 7f637e716700 1 -- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f636406a440 msgr2=0x7f636406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:56.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.912+0000 7f637e716700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f636406a440 0x7f636406c900 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f636c007b50 tx=0x7f636c009040 comp rx=0 tx=0).stop 2023-08-27T11:26:56.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.912+0000 7f637e716700 1 -- 172.21.15.112:0/3908512008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6360003a30 msgr2=0x7f63780c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:56.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.912+0000 7f637e716700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6360003a30 0x7f63780c9f40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f63600047b0 tx=0x7f63680047c0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.912+0000 7f637e716700 1 -- 172.21.15.112:0/3908512008 shutdown_connections 2023-08-27T11:26:56.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.912+0000 7f637e716700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f636406a440 0x7f636406c900 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.912+0000 7f637e716700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63781f65b0 0x7f63781f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.912+0000 7f637e716700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6360003a30 0x7f63780c9f40 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.912+0000 7f637e716700 1 --2- 172.21.15.112:0/3908512008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63780c5570 0x7f63781482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:56.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.912+0000 7f637e716700 1 -- 172.21.15.112:0/3908512008 >> 172.21.15.112:0/3908512008 conn(0x7f63780d0000 msgr2=0x7f63780ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:56.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.913+0000 7f637e716700 1 -- 172.21.15.112:0/3908512008 shutdown_connections 2023-08-27T11:26:56.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:56.913+0000 7f637e716700 1 -- 172.21.15.112:0/3908512008 wait complete. 2023-08-27T11:26:56.927 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:26:57.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.045+0000 7f3ff7c1b700 1 Processor -- start 2023-08-27T11:26:57.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.045+0000 7f3ff7c1b700 1 -- start start 2023-08-27T11:26:57.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.046+0000 7f3ff7c1b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3ff00b1400 0x7f3ff00cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:57.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.046+0000 7f3ff7c1b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3ff00b17e0 0x7f3ff006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:57.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.046+0000 7f3ff7c1b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff0161850 con 0x7f3ff0161ce0 2023-08-27T11:26:57.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.046+0000 7f3ff7c1b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff006e7d0 con 0x7f3ff00b1400 2023-08-27T11:26:57.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.046+0000 7f3ff7c1b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff00cfc10 con 0x7f3ff00b17e0 2023-08-27T11:26:57.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.046+0000 7f3ff59b7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3ff00b1400 0x7f3ff00cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:57.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.046+0000 7f3ff61b8700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3ff0161ce0 0x7f3ff006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56536/0 (socket says 172.21.15.112:56536) 2023-08-27T11:26:57.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.046+0000 7f3ff61b8700 1 -- 172.21.15.112:0/113238957 learned_addr learned my addr 172.21.15.112:0/113238957 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:57.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.046+0000 7f3ff51b6700 1 --2- 172.21.15.112:0/113238957 >> v2:172.21.15.138:3300/0 conn(0x7f3ff00b17e0 0x7f3ff006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:57.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.047+0000 7f3ff59b7700 1 -- 172.21.15.112:0/113238957 >> v2:172.21.15.138:3300/0 conn(0x7f3ff00b17e0 msgr2=0x7f3ff006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:57.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.047+0000 7f3ff59b7700 1 --2- 172.21.15.112:0/113238957 >> v2:172.21.15.138:3300/0 conn(0x7f3ff00b17e0 0x7f3ff006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.047+0000 7f3ff59b7700 1 -- 172.21.15.112:0/113238957 >> v1:172.21.15.112:6789/0 conn(0x7f3ff0161ce0 legacy=0x7f3ff006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:57.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.047+0000 7f3ff49b5700 1 -- 172.21.15.112:0/113238957 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100427746 0 0) 0x7f3ff0161850 con 0x7f3ff0161ce0 2023-08-27T11:26:57.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.047+0000 7f3ff59b7700 1 -- 172.21.15.112:0/113238957 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff0161200 con 0x7f3ff00b1400 2023-08-27T11:26:57.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.047+0000 7f3ff59b7700 1 --2- 172.21.15.112:0/113238957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00b1400 0x7f3ff00cf5a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00cc420 tx=0x7f3fe0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc5c01de27a3805 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:57.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.047+0000 7f3ff49b5700 1 -- 172.21.15.112:0/113238957 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0011830 con 0x7f3ff00b1400 2023-08-27T11:26:57.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.047+0000 7f3ff49b5700 1 -- 172.21.15.112:0/113238957 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe0011990 con 0x7f3ff00b1400 2023-08-27T11:26:57.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.047+0000 7f3ff49b5700 1 -- 172.21.15.112:0/113238957 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0011d20 con 0x7f3ff00b1400 2023-08-27T11:26:57.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.047+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/113238957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00b1400 msgr2=0x7f3ff00cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:57.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.047+0000 7f3ff7c1b700 1 --2- 172.21.15.112:0/113238957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00b1400 0x7f3ff00cf5a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00cc420 tx=0x7f3fe0009b20 comp rx=0 tx=0).stop 2023-08-27T11:26:57.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.048+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/113238957 shutdown_connections 2023-08-27T11:26:57.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.048+0000 7f3ff7c1b700 1 --2- 172.21.15.112:0/113238957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00b1400 0x7f3ff00cf5a0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.048+0000 7f3ff7c1b700 1 --2- 172.21.15.112:0/113238957 >> v2:172.21.15.138:3300/0 conn(0x7f3ff00b17e0 0x7f3ff006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.048+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/113238957 >> 172.21.15.112:0/113238957 conn(0x7f3ff00d0000 msgr2=0x7f3ff00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.048+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/113238957 shutdown_connections 2023-08-27T11:26:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.048+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/113238957 wait complete. 2023-08-27T11:26:57.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.049+0000 7f3ff7c1b700 1 Processor -- start 2023-08-27T11:26:57.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.049+0000 7f3ff7c1b700 1 -- start start 2023-08-27T11:26:57.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.049+0000 7f3ff7c1b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff00b1400 0x7f3ff01549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:57.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.049+0000 7f3ff7c1b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff00b17e0 0x7f3ff0150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:57.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff7c1b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff0161ce0 0x7f3ff00c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:57.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff7c1b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00c4320 con 0x7f3ff0161ce0 2023-08-27T11:26:57.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff7c1b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff0151330 con 0x7f3ff00b1400 2023-08-27T11:26:57.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff7c1b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00c5190 con 0x7f3ff00b17e0 2023-08-27T11:26:57.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff61b8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff0161ce0 0x7f3ff00c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:57.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff61b8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff0161ce0 0x7f3ff00c4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33930/0 (socket says 172.21.15.112:33930) 2023-08-27T11:26:57.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff61b8700 1 -- 172.21.15.112:0/1693433921 learned_addr learned my addr 172.21.15.112:0/1693433921 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:57.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff59b7700 1 --2- 172.21.15.112:0/1693433921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff00b1400 0x7f3ff01549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:57.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff61b8700 1 -- 172.21.15.112:0/1693433921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff00b17e0 msgr2=0x7f3ff0150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:57.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff61b8700 1 --2- 172.21.15.112:0/1693433921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff00b17e0 0x7f3ff0150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff61b8700 1 -- 172.21.15.112:0/1693433921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff00b1400 msgr2=0x7f3ff01549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:57.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff61b8700 1 --2- 172.21.15.112:0/1693433921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff00b1400 0x7f3ff01549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff61b8700 1 -- 172.21.15.112:0/1693433921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00094a0 con 0x7f3ff0161ce0 2023-08-27T11:26:57.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff61b8700 1 --2- 172.21.15.112:0/1693433921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff0161ce0 0x7f3ff00c4c50 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800cba0 tx=0x7f3fe800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:57.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3fe67fc700 1 -- 172.21.15.112:0/1693433921 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe800ab80 con 0x7f3ff0161ce0 2023-08-27T11:26:57.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3fe67fc700 1 -- 172.21.15.112:0/1693433921 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe800ad00 con 0x7f3ff0161ce0 2023-08-27T11:26:57.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/1693433921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff00c5430 con 0x7f3ff0161ce0 2023-08-27T11:26:57.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3fe67fc700 1 -- 172.21.15.112:0/1693433921 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8007310 con 0x7f3ff0161ce0 2023-08-27T11:26:57.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.050+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/1693433921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff00c3f40 con 0x7f3ff0161ce0 2023-08-27T11:26:57.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.051+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/1693433921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff00c4460 con 0x7f3ff0161ce0 2023-08-27T11:26:57.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.052+0000 7f3fe67fc700 1 -- 172.21.15.112:0/1693433921 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f3fe8007470 con 0x7f3ff0161ce0 2023-08-27T11:26:57.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.052+0000 7f3fe67fc700 1 --2- 172.21.15.112:0/1693433921 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fd406a440 0x7f3fd406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:57.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.052+0000 7f3fe67fc700 1 -- 172.21.15.112:0/1693433921 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3fe808c020 con 0x7f3ff0161ce0 2023-08-27T11:26:57.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.052+0000 7f3ff59b7700 1 --2- 172.21.15.112:0/1693433921 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fd406a440 0x7f3fd406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:57.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.053+0000 7f3ff59b7700 1 --2- 172.21.15.112:0/1693433921 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fd406a440 0x7f3fd406c900 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000c00 tx=0x7f3fe0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:57.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.056+0000 7f3fe67fc700 1 -- 172.21.15.112:0/1693433921 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3fe8053b70 con 0x7f3ff0161ce0 2023-08-27T11:26:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.246+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/1693433921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ff0067a20 con 0x7f3ff0161ce0 2023-08-27T11:26:57.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.247+0000 7f3fe67fc700 1 -- 172.21.15.112:0/1693433921 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+18340 (secure 0 0 0) 0x7f3fe8018020 con 0x7f3ff0161ce0 2023-08-27T11:26:57.249 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:57.249 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":101,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:55.964954+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:54.078457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:54.929983+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:26:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.249+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/1693433921 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fd406a440 msgr2=0x7f3fd406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.249+0000 7f3ff7c1b700 1 --2- 172.21.15.112:0/1693433921 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fd406a440 0x7f3fd406c900 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000c00 tx=0x7f3fe0008030 comp rx=0 tx=0).stop 2023-08-27T11:26:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.249+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/1693433921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff0161ce0 msgr2=0x7f3ff00c4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.249+0000 7f3ff7c1b700 1 --2- 172.21.15.112:0/1693433921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff0161ce0 0x7f3ff00c4c50 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800cba0 tx=0x7f3fe800cf60 comp rx=0 tx=0).stop 2023-08-27T11:26:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.249+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/1693433921 shutdown_connections 2023-08-27T11:26:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.249+0000 7f3ff7c1b700 1 --2- 172.21.15.112:0/1693433921 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fd406a440 0x7f3fd406c900 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.249+0000 7f3ff7c1b700 1 --2- 172.21.15.112:0/1693433921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff0161ce0 0x7f3ff00c4c50 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.249+0000 7f3ff7c1b700 1 --2- 172.21.15.112:0/1693433921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff00b1400 0x7f3ff01549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.249+0000 7f3ff7c1b700 1 --2- 172.21.15.112:0/1693433921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ff00b17e0 0x7f3ff0150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.249+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/1693433921 >> 172.21.15.112:0/1693433921 conn(0x7f3ff00d0000 msgr2=0x7f3ff015ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:57.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.249+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/1693433921 shutdown_connections 2023-08-27T11:26:57.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.249+0000 7f3ff7c1b700 1 -- 172.21.15.112:0/1693433921 wait complete. 2023-08-27T11:26:57.263 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:26:57.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.388+0000 7f6e68e5e700 1 Processor -- start 2023-08-27T11:26:57.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.388+0000 7f6e68e5e700 1 -- start start 2023-08-27T11:26:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e68e5e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6e64146bf0 0x7f6e640b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e68e5e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6e64146fd0 0x7f6e640cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e68e5e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e640d1500 con 0x7f6e640d0290 2023-08-27T11:26:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e68e5e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e640d0710 con 0x7f6e64146fd0 2023-08-27T11:26:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e68e5e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e640d09f0 con 0x7f6e64146bf0 2023-08-27T11:26:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e61d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6e64146fd0 0x7f6e640cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e61d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6e64146fd0 0x7f6e640cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33936/0 (socket says 172.21.15.112:33936) 2023-08-27T11:26:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e61d9b700 1 -- 172.21.15.112:0/4253193775 learned_addr learned my addr 172.21.15.112:0/4253193775 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e6259c700 1 --2- 172.21.15.112:0/4253193775 >> v2:172.21.15.138:3300/0 conn(0x7f6e64146bf0 0x7f6e640b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e61d9b700 1 -- 172.21.15.112:0/4253193775 >> v2:172.21.15.138:3300/0 conn(0x7f6e64146bf0 msgr2=0x7f6e640b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e61d9b700 1 --2- 172.21.15.112:0/4253193775 >> v2:172.21.15.138:3300/0 conn(0x7f6e64146bf0 0x7f6e640b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e61d9b700 1 -- 172.21.15.112:0/4253193775 >> v1:172.21.15.138:6789/0 conn(0x7f6e640d0290 legacy=0x7f6e64070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e61d9b700 1 -- 172.21.15.112:0/4253193775 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e640d0cd0 con 0x7f6e64146fd0 2023-08-27T11:26:57.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.389+0000 7f6e6259c700 1 --2- 172.21.15.112:0/4253193775 >> v2:172.21.15.138:3300/0 conn(0x7f6e64146bf0 0x7f6e640b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:57.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.390+0000 7f6e61d9b700 1 --2- 172.21.15.112:0/4253193775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64146fd0 0x7f6e640cfd50 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6e6406e8f0 tx=0x7f6e58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4f93e413b23ecc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:57.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.390+0000 7f6e6159a700 1 -- 172.21.15.112:0/4253193775 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58004230 con 0x7f6e64146fd0 2023-08-27T11:26:57.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.390+0000 7f6e6159a700 1 -- 172.21.15.112:0/4253193775 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e58004390 con 0x7f6e64146fd0 2023-08-27T11:26:57.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.390+0000 7f6e6159a700 1 -- 172.21.15.112:0/4253193775 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e580036a0 con 0x7f6e64146fd0 2023-08-27T11:26:57.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.390+0000 7f6e68e5e700 1 -- 172.21.15.112:0/4253193775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64146fd0 msgr2=0x7f6e640cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:57.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.390+0000 7f6e68e5e700 1 --2- 172.21.15.112:0/4253193775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64146fd0 0x7f6e640cfd50 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6e6406e8f0 tx=0x7f6e58009bd0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.390+0000 7f6e68e5e700 1 -- 172.21.15.112:0/4253193775 shutdown_connections 2023-08-27T11:26:57.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.390+0000 7f6e68e5e700 1 --2- 172.21.15.112:0/4253193775 >> v2:172.21.15.138:3300/0 conn(0x7f6e64146bf0 0x7f6e640b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.390+0000 7f6e68e5e700 1 --2- 172.21.15.112:0/4253193775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64146fd0 0x7f6e640cfd50 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.390+0000 7f6e68e5e700 1 -- 172.21.15.112:0/4253193775 >> 172.21.15.112:0/4253193775 conn(0x7f6e6406c450 msgr2=0x7f6e6406f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:57.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.390+0000 7f6e68e5e700 1 -- 172.21.15.112:0/4253193775 shutdown_connections 2023-08-27T11:26:57.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.390+0000 7f6e68e5e700 1 -- 172.21.15.112:0/4253193775 wait complete. 2023-08-27T11:26:57.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.392+0000 7f6e68e5e700 1 Processor -- start 2023-08-27T11:26:57.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.392+0000 7f6e68e5e700 1 -- start start 2023-08-27T11:26:57.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.392+0000 7f6e68e5e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e640d0290 0x7f6e6413c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:57.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e68e5e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e64146bf0 0x7f6e641f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:57.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e68e5e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64146fd0 0x7f6e6413d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:57.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e68e5e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6415c2a0 con 0x7f6e64146fd0 2023-08-27T11:26:57.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e68e5e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6413e2c0 con 0x7f6e640d0290 2023-08-27T11:26:57.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e68e5e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6413e5a0 con 0x7f6e64146bf0 2023-08-27T11:26:57.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e62d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64146fd0 0x7f6e6413d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:57.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e62d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64146fd0 0x7f6e6413d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33944/0 (socket says 172.21.15.112:33944) 2023-08-27T11:26:57.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e62d9d700 1 -- 172.21.15.112:0/3494169578 learned_addr learned my addr 172.21.15.112:0/3494169578 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:57.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e6259c700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e640d0290 0x7f6e6413c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:57.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e61d9b700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e64146bf0 0x7f6e641f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:57.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e62d9d700 1 -- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e64146bf0 msgr2=0x7f6e641f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:57.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e62d9d700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e64146bf0 0x7f6e641f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e62d9d700 1 -- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e640d0290 msgr2=0x7f6e6413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:57.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e62d9d700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e640d0290 0x7f6e6413c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:57.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e62d9d700 1 -- 172.21.15.112:0/3494169578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e58009550 con 0x7f6e64146fd0 2023-08-27T11:26:57.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e62d9d700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64146fd0 0x7f6e6413d6e0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00cc10 tx=0x7f6e5c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:57.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e52ffd700 1 -- 172.21.15.112:0/3494169578 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c00aa60 con 0x7f6e64146fd0 2023-08-27T11:26:57.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e52ffd700 1 -- 172.21.15.112:0/3494169578 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c00abc0 con 0x7f6e64146fd0 2023-08-27T11:26:57.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e68e5e700 1 -- 172.21.15.112:0/3494169578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6413e880 con 0x7f6e64146fd0 2023-08-27T11:26:57.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e52ffd700 1 -- 172.21.15.112:0/3494169578 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c012420 con 0x7f6e64146fd0 2023-08-27T11:26:57.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.393+0000 7f6e68e5e700 1 -- 172.21.15.112:0/3494169578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6413ee40 con 0x7f6e64146fd0 2023-08-27T11:26:57.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.394+0000 7f6e68e5e700 1 -- 172.21.15.112:0/3494169578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e64067e90 con 0x7f6e64146fd0 2023-08-27T11:26:57.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.395+0000 7f6e52ffd700 1 -- 172.21.15.112:0/3494169578 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f6e5c012580 con 0x7f6e64146fd0 2023-08-27T11:26:57.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.396+0000 7f6e52ffd700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6e4406a440 0x7f6e4406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:57.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.396+0000 7f6e52ffd700 1 -- 172.21.15.112:0/3494169578 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6e5c089850 con 0x7f6e64146fd0 2023-08-27T11:26:57.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.396+0000 7f6e6259c700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6e4406a440 0x7f6e4406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:57.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.396+0000 7f6e6259c700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6e4406a440 0x7f6e4406c900 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6e54006fd0 tx=0x7f6e54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:57.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.399+0000 7f6e52ffd700 1 -- 172.21.15.112:0/3494169578 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6e5c054c20 con 0x7f6e64146fd0 2023-08-27T11:26:57.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:57.635+0000 7f6e68e5e700 1 -- 172.21.15.112:0/3494169578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6e64067a20 con 0x7f6e64146fd0 2023-08-27T11:26:58.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.590+0000 7f6e52ffd700 1 -- 172.21.15.112:0/3494169578 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6e5c058240 con 0x7f6e64146fd0 2023-08-27T11:26:58.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.593+0000 7f6e68e5e700 1 -- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6e4406a440 msgr2=0x7f6e4406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:58.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.593+0000 7f6e68e5e700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6e4406a440 0x7f6e4406c900 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6e54006fd0 tx=0x7f6e54008040 comp rx=0 tx=0).stop 2023-08-27T11:26:58.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.593+0000 7f6e68e5e700 1 -- 172.21.15.112:0/3494169578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64146fd0 msgr2=0x7f6e6413d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:58.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.593+0000 7f6e68e5e700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64146fd0 0x7f6e6413d6e0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00cc10 tx=0x7f6e5c00cf20 comp rx=0 tx=0).stop 2023-08-27T11:26:58.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.593+0000 7f6e68e5e700 1 -- 172.21.15.112:0/3494169578 shutdown_connections 2023-08-27T11:26:58.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.593+0000 7f6e68e5e700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6e4406a440 0x7f6e4406c900 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:58.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.593+0000 7f6e68e5e700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64146fd0 0x7f6e6413d6e0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:58.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.593+0000 7f6e68e5e700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e640d0290 0x7f6e6413c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:58.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.593+0000 7f6e68e5e700 1 --2- 172.21.15.112:0/3494169578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e64146bf0 0x7f6e641f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:58.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.593+0000 7f6e68e5e700 1 -- 172.21.15.112:0/3494169578 >> 172.21.15.112:0/3494169578 conn(0x7f6e6406c450 msgr2=0x7f6e64145e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:58.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.593+0000 7f6e68e5e700 1 -- 172.21.15.112:0/3494169578 shutdown_connections 2023-08-27T11:26:58.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.593+0000 7f6e68e5e700 1 -- 172.21.15.112:0/3494169578 wait complete. 2023-08-27T11:26:58.610 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:26:58.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:26:58.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.738+0000 7f4814053700 1 Processor -- start 2023-08-27T11:26:58.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.739+0000 7f4814053700 1 -- start start 2023-08-27T11:26:58.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.739+0000 7f4814053700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f480c157840 0x7f480c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:58.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.739+0000 7f4814053700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f480c1617d0 0x7f480c0b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:58.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.739+0000 7f4814053700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f480c160ff0 con 0x7f480c157c20 2023-08-27T11:26:58.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.739+0000 7f4814053700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480c0d14e0 con 0x7f480c157840 2023-08-27T11:26:58.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.739+0000 7f4814053700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480c0d17c0 con 0x7f480c1617d0 2023-08-27T11:26:58.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.739+0000 7f4811def700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f480c157840 0x7f480c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:58.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.740+0000 7f4811def700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f480c157840 0x7f480c0d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33948/0 (socket says 172.21.15.112:33948) 2023-08-27T11:26:58.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.740+0000 7f4811def700 1 -- 172.21.15.112:0/910767257 learned_addr learned my addr 172.21.15.112:0/910767257 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:58.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.740+0000 7f48125f0700 1 --2- 172.21.15.112:0/910767257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f480c1617d0 0x7f480c0b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:58.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.740+0000 7f4811def700 1 -- 172.21.15.112:0/910767257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f480c1617d0 msgr2=0x7f480c0b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:58.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.740+0000 7f4811def700 1 --2- 172.21.15.112:0/910767257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f480c1617d0 0x7f480c0b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:58.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.740+0000 7f4811def700 1 -- 172.21.15.112:0/910767257 >> v1:172.21.15.138:6789/0 conn(0x7f480c157c20 legacy=0x7f480c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:58.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.740+0000 7f4811def700 1 -- 172.21.15.112:0/910767257 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c0d1aa0 con 0x7f480c157840 2023-08-27T11:26:58.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.740+0000 7f4811def700 1 --2- 172.21.15.112:0/910767257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480c157840 0x7f480c0d0df0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0098f0 tx=0x7f47fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f832d7e9c3540d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:58.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.740+0000 7f4810ded700 1 -- 172.21.15.112:0/910767257 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00b8a0 con 0x7f480c157840 2023-08-27T11:26:58.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.740+0000 7f4810ded700 1 -- 172.21.15.112:0/910767257 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc00ba00 con 0x7f480c157840 2023-08-27T11:26:58.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.740+0000 7f4810ded700 1 -- 172.21.15.112:0/910767257 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00bd90 con 0x7f480c157840 2023-08-27T11:26:58.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.741+0000 7f4814053700 1 -- 172.21.15.112:0/910767257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480c157840 msgr2=0x7f480c0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:58.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.741+0000 7f4814053700 1 --2- 172.21.15.112:0/910767257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480c157840 0x7f480c0d0df0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0098f0 tx=0x7f47fc009c00 comp rx=0 tx=0).stop 2023-08-27T11:26:58.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.741+0000 7f4814053700 1 -- 172.21.15.112:0/910767257 shutdown_connections 2023-08-27T11:26:58.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.741+0000 7f4814053700 1 --2- 172.21.15.112:0/910767257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f480c1617d0 0x7f480c0b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:58.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.741+0000 7f4814053700 1 --2- 172.21.15.112:0/910767257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480c157840 0x7f480c0d0df0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:58.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.741+0000 7f4814053700 1 -- 172.21.15.112:0/910767257 >> 172.21.15.112:0/910767257 conn(0x7f480c066010 msgr2=0x7f480c0663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:58.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.741+0000 7f4814053700 1 -- 172.21.15.112:0/910767257 shutdown_connections 2023-08-27T11:26:58.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.741+0000 7f4814053700 1 -- 172.21.15.112:0/910767257 wait complete. 2023-08-27T11:26:58.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.742+0000 7f4814053700 1 Processor -- start 2023-08-27T11:26:58.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.742+0000 7f4814053700 1 -- start start 2023-08-27T11:26:58.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4814053700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480c157840 0x7f480c153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:58.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4814053700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f480c157c20 0x7f480c0b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:58.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4814053700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f480c1617d0 0x7f480c1f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:58.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4814053700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c1f37d0 con 0x7f480c157840 2023-08-27T11:26:58.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4814053700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c1f3ab0 con 0x7f480c157c20 2023-08-27T11:26:58.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4814053700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480c1f3d90 con 0x7f480c1617d0 2023-08-27T11:26:58.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4811def700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480c157840 0x7f480c153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:58.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4811def700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480c157840 0x7f480c153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33954/0 (socket says 172.21.15.112:33954) 2023-08-27T11:26:58.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4811def700 1 -- 172.21.15.112:0/1819568151 learned_addr learned my addr 172.21.15.112:0/1819568151 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:58.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f48115ee700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f480c157c20 0x7f480c0b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:58.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f48125f0700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f480c1617d0 0x7f480c1f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:58.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4811def700 1 -- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f480c1617d0 msgr2=0x7f480c1f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:58.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4811def700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f480c1617d0 0x7f480c1f2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:58.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4811def700 1 -- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f480c157c20 msgr2=0x7f480c0b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:58.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4811def700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f480c157c20 0x7f480c0b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4811def700 1 -- 172.21.15.112:0/1819568151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc009550 con 0x7f480c157840 2023-08-27T11:26:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f48125f0700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f480c1617d0 0x7f480c1f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:26:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4811def700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480c157840 0x7f480c153e90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f47fc000c00 tx=0x7f47fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f48027fc700 1 -- 172.21.15.112:0/1819568151 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc004380 con 0x7f480c157840 2023-08-27T11:26:58.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f48027fc700 1 -- 172.21.15.112:0/1819568151 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc01f070 con 0x7f480c157840 2023-08-27T11:26:58.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4814053700 1 -- 172.21.15.112:0/1819568151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480c1f0bf0 con 0x7f480c157840 2023-08-27T11:26:58.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f48027fc700 1 -- 172.21.15.112:0/1819568151 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc0036a0 con 0x7f480c157840 2023-08-27T11:26:58.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.743+0000 7f4814053700 1 -- 172.21.15.112:0/1819568151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480c1f3450 con 0x7f480c157840 2023-08-27T11:26:58.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.744+0000 7f4814053700 1 -- 172.21.15.112:0/1819568151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480c1f1110 con 0x7f480c157840 2023-08-27T11:26:58.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.745+0000 7f48027fc700 1 -- 172.21.15.112:0/1819568151 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f47fc021030 con 0x7f480c157840 2023-08-27T11:26:58.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.745+0000 7f48027fc700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f47f006a440 0x7f47f006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:58.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.745+0000 7f48027fc700 1 -- 172.21.15.112:0/1819568151 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47fc089e20 con 0x7f480c157840 2023-08-27T11:26:58.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.745+0000 7f48115ee700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f47f006a440 0x7f47f006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:58.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.746+0000 7f48115ee700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f47f006a440 0x7f47f006c900 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f480c1543d0 tx=0x7f4808009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:58.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.750+0000 7f48027fc700 1 -- 172.21.15.112:0/1819568151 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f480c1f1110 con 0x7f480c157840 2023-08-27T11:26:58.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:58.981+0000 7f4814053700 1 -- 172.21.15.112:0/1819568151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f480c1f1110 con 0x7f480c157840 2023-08-27T11:26:59.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.592+0000 7f48027fc700 1 -- 172.21.15.112:0/1819568151 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) v1 ==== 122+0+0 (secure 0 0 0) 0x7f480c1f1110 con 0x7f480c157840 2023-08-27T11:26:59.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.594+0000 7f4814053700 1 -- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f47f006a440 msgr2=0x7f47f006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:59.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.594+0000 7f4814053700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f47f006a440 0x7f47f006c900 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f480c1543d0 tx=0x7f4808009040 comp rx=0 tx=0).stop 2023-08-27T11:26:59.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.594+0000 7f4814053700 1 -- 172.21.15.112:0/1819568151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480c157840 msgr2=0x7f480c153e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:59.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.594+0000 7f4814053700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480c157840 0x7f480c153e90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f47fc000c00 tx=0x7f47fc004750 comp rx=0 tx=0).stop 2023-08-27T11:26:59.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.594+0000 7f4814053700 1 -- 172.21.15.112:0/1819568151 shutdown_connections 2023-08-27T11:26:59.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.594+0000 7f4814053700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f47f006a440 0x7f47f006c900 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:59.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.594+0000 7f4814053700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f480c1617d0 0x7f480c1f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:59.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.594+0000 7f4814053700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480c157840 0x7f480c153e90 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:59.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.594+0000 7f4814053700 1 --2- 172.21.15.112:0/1819568151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f480c157c20 0x7f480c0b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:59.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.594+0000 7f4814053700 1 -- 172.21.15.112:0/1819568151 >> 172.21.15.112:0/1819568151 conn(0x7f480c066010 msgr2=0x7f480c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:59.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.594+0000 7f4814053700 1 -- 172.21.15.112:0/1819568151 shutdown_connections 2023-08-27T11:26:59.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.594+0000 7f4814053700 1 -- 172.21.15.112:0/1819568151 wait complete. 2023-08-27T11:26:59.609 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:26:59.610 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:26:59.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.731+0000 7fc4fa9fd700 1 Processor -- start 2023-08-27T11:26:59.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.732+0000 7fc4fa9fd700 1 -- start start 2023-08-27T11:26:59.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.732+0000 7fc4fa9fd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4f40b17e0 0x7fc4f40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:59.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.732+0000 7fc4fa9fd700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f4161850 con 0x7fc4f40b1400 2023-08-27T11:26:59.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.732+0000 7fc4fa9fd700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f40cfc10 con 0x7fc4f4161ce0 2023-08-27T11:26:59.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.732+0000 7fc4fa9fd700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f4161200 con 0x7fc4f40b17e0 2023-08-27T11:26:59.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.732+0000 7fc4f37fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4f40b17e0 0x7fc4f40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:59.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.732+0000 7fc4f3fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc4f40b1400 0x7fc4f406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56576/0 (socket says 172.21.15.112:56576) 2023-08-27T11:26:59.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.732+0000 7fc4f3fff700 1 -- 172.21.15.112:0/420713395 learned_addr learned my addr 172.21.15.112:0/420713395 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:26:59.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.733+0000 7fc4f37fe700 1 -- 172.21.15.112:0/420713395 >> v1:172.21.15.138:6789/0 conn(0x7fc4f4161ce0 legacy=0x7fc4f406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:26:59.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.733+0000 7fc4f37fe700 1 -- 172.21.15.112:0/420713395 >> v1:172.21.15.112:6789/0 conn(0x7fc4f40b1400 legacy=0x7fc4f406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:59.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.733+0000 7fc4f37fe700 1 -- 172.21.15.112:0/420713395 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f41614e0 con 0x7fc4f40b17e0 2023-08-27T11:26:59.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.733+0000 7fc4f2ffd700 1 -- 172.21.15.112:0/420713395 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368068133 0 0) 0x7fc4f4161850 con 0x7fc4f40b1400 2023-08-27T11:26:59.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.733+0000 7fc4f37fe700 1 --2- 172.21.15.112:0/420713395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f40b17e0 0x7fc4f40cf5a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009840 tx=0x7fc4e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a690df2e483cb2c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:59.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.733+0000 7fc4f2ffd700 1 -- 172.21.15.112:0/420713395 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8004230 con 0x7fc4f40b17e0 2023-08-27T11:26:59.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.733+0000 7fc4f2ffd700 1 -- 172.21.15.112:0/420713395 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e8004390 con 0x7fc4f40b17e0 2023-08-27T11:26:59.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.733+0000 7fc4f2ffd700 1 -- 172.21.15.112:0/420713395 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80036a0 con 0x7fc4f40b17e0 2023-08-27T11:26:59.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.733+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/420713395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f40b17e0 msgr2=0x7fc4f40cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:59.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.733+0000 7fc4fa9fd700 1 --2- 172.21.15.112:0/420713395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f40b17e0 0x7fc4f40cf5a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009840 tx=0x7fc4e8009b50 comp rx=0 tx=0).stop 2023-08-27T11:26:59.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.734+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/420713395 shutdown_connections 2023-08-27T11:26:59.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.734+0000 7fc4fa9fd700 1 --2- 172.21.15.112:0/420713395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f40b17e0 0x7fc4f40cf5a0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:59.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.734+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/420713395 >> 172.21.15.112:0/420713395 conn(0x7fc4f40d0000 msgr2=0x7fc4f40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:59.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.734+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/420713395 shutdown_connections 2023-08-27T11:26:59.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.734+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/420713395 wait complete. 2023-08-27T11:26:59.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.735+0000 7fc4fa9fd700 1 Processor -- start 2023-08-27T11:26:59.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.735+0000 7fc4fa9fd700 1 -- start start 2023-08-27T11:26:59.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.736+0000 7fc4fa9fd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4f40b1400 0x7fc4f4146140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:59.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.736+0000 7fc4fa9fd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4f40b17e0 0x7fc4f41521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:59.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.736+0000 7fc4fa9fd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f4161ce0 0x7fc4f4152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:59.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.736+0000 7fc4fa9fd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f40c58a0 con 0x7fc4f4161ce0 2023-08-27T11:26:59.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.736+0000 7fc4fa9fd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f41f2f30 con 0x7fc4f40b17e0 2023-08-27T11:26:59.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.736+0000 7fc4fa9fd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f41f2c40 con 0x7fc4f40b1400 2023-08-27T11:26:59.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.736+0000 7fc4f8f9a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f4161ce0 0x7fc4f4152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:59.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.736+0000 7fc4f8f9a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f4161ce0 0x7fc4f4152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33972/0 (socket says 172.21.15.112:33972) 2023-08-27T11:26:59.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.736+0000 7fc4f8f9a700 1 -- 172.21.15.112:0/3345504855 learned_addr learned my addr 172.21.15.112:0/3345504855 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:26:59.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.736+0000 7fc4f37fe700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4f40b17e0 0x7fc4f41521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:59.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.737+0000 7fc4f3fff700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4f40b1400 0x7fc4f4146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:59.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.737+0000 7fc4f8f9a700 1 -- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4f40b1400 msgr2=0x7fc4f4146140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:59.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.737+0000 7fc4f8f9a700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4f40b1400 0x7fc4f4146140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:59.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.737+0000 7fc4f8f9a700 1 -- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4f40b17e0 msgr2=0x7fc4f41521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:59.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.737+0000 7fc4f8f9a700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4f40b17e0 0x7fc4f41521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:59.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.737+0000 7fc4f8f9a700 1 -- 172.21.15.112:0/3345504855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e80094a0 con 0x7fc4f4161ce0 2023-08-27T11:26:59.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.737+0000 7fc4f3fff700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4f40b1400 0x7fc4f4146140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:26:59.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.737+0000 7fc4f8f9a700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f4161ce0 0x7fc4f4152720 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00b990 tx=0x7fc4ec00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:59.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.737+0000 7fc4f0ff9700 1 -- 172.21.15.112:0/3345504855 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec00c9d0 con 0x7fc4f4161ce0 2023-08-27T11:26:59.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.737+0000 7fc4f0ff9700 1 -- 172.21.15.112:0/3345504855 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec00cb30 con 0x7fc4f4161ce0 2023-08-27T11:26:59.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.737+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/3345504855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f40c86f0 con 0x7fc4f4161ce0 2023-08-27T11:26:59.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.737+0000 7fc4f0ff9700 1 -- 172.21.15.112:0/3345504855 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec014440 con 0x7fc4f4161ce0 2023-08-27T11:26:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.737+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/3345504855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f40c8cb0 con 0x7fc4f4161ce0 2023-08-27T11:26:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.738+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/3345504855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f41f30a0 con 0x7fc4f4161ce0 2023-08-27T11:26:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.738+0000 7fc4f0ff9700 1 -- 172.21.15.112:0/3345504855 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fc4ec006d50 con 0x7fc4f4161ce0 2023-08-27T11:26:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.739+0000 7fc4f0ff9700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc4e006a440 0x7fc4e006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:26:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.739+0000 7fc4f0ff9700 1 -- 172.21.15.112:0/3345504855 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc4ec0885f0 con 0x7fc4f4161ce0 2023-08-27T11:26:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.739+0000 7fc4f3fff700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc4e006a440 0x7fc4e006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:26:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.739+0000 7fc4f3fff700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc4e006a440 0x7fc4e006c900 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4000f60 tx=0x7fc4e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:26:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.743+0000 7fc4f0ff9700 1 -- 172.21.15.112:0/3345504855 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc4ec0538d0 con 0x7fc4f4161ce0 2023-08-27T11:26:59.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.976+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/3345504855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4f4067a20 con 0x7fc4f4161ce0 2023-08-27T11:26:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.976+0000 7fc4f0ff9700 1 -- 172.21.15.112:0/3345504855 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fc4ec056ef0 con 0x7fc4f4161ce0 2023-08-27T11:26:59.978 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:26:59.979 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/37643970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":37643970},{"type":"v1","addr":"172.21.15.112:6834","nonce":37643970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/568782348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":568782348},{"type":"v1","addr":"172.21.15.138:6836","nonce":568782348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1867471713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1867471713},{"type":"v1","addr":"172.21.15.138:6837","nonce":1867471713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:26:55.965268+0000","modified":"2023-08-27T11:26:59.589861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3180263819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3180263819},{"type":"v1","addr":"172.21.15.112:6835","nonce":3180263819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-27T11:26:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.978+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc4e006a440 msgr2=0x7fc4e006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.978+0000 7fc4fa9fd700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc4e006a440 0x7fc4e006c900 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4000f60 tx=0x7fc4e4009040 comp rx=0 tx=0).stop 2023-08-27T11:26:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.978+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/3345504855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f4161ce0 msgr2=0x7fc4f4152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:26:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.978+0000 7fc4fa9fd700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f4161ce0 0x7fc4f4152720 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00b990 tx=0x7fc4ec00bd50 comp rx=0 tx=0).stop 2023-08-27T11:26:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.978+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/3345504855 shutdown_connections 2023-08-27T11:26:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.978+0000 7fc4fa9fd700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc4e006a440 0x7fc4e006c900 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:59.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.978+0000 7fc4fa9fd700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f4161ce0 0x7fc4f4152720 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:59.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.978+0000 7fc4fa9fd700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4f40b1400 0x7fc4f4146140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:59.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.978+0000 7fc4fa9fd700 1 --2- 172.21.15.112:0/3345504855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4f40b17e0 0x7fc4f41521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:26:59.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.978+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/3345504855 >> 172.21.15.112:0/3345504855 conn(0x7fc4f40d0000 msgr2=0x7fc4f406f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:26:59.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.978+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/3345504855 shutdown_connections 2023-08-27T11:26:59.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:26:59.979+0000 7fc4fa9fd700 1 -- 172.21.15.112:0/3345504855 wait complete. 2023-08-27T11:26:59.983 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 67 2023-08-27T11:26:59.993 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.113+0000 7f86a18ce700 1 Processor -- start 2023-08-27T11:27:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.113+0000 7f86a18ce700 1 -- start start 2023-08-27T11:27:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.113+0000 7f86a18ce700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f869c06e860 0x7f869c15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.113+0000 7f86a18ce700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869c142fe0 con 0x7f869c140f00 2023-08-27T11:27:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.113+0000 7f86a18ce700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869c1432c0 con 0x7f869c0ad210 2023-08-27T11:27:00.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.113+0000 7f86a18ce700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869c143bf0 con 0x7f869c06e860 2023-08-27T11:27:00.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f869b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f869c140f00 0x7f869c141360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56594/0 (socket says 172.21.15.112:56594) 2023-08-27T11:27:00.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f869b7fe700 1 -- 172.21.15.112:0/3906098448 learned_addr learned my addr 172.21.15.112:0/3906098448 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:00.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f869a7fc700 1 --2- 172.21.15.112:0/3906098448 >> v2:172.21.15.138:3300/0 conn(0x7f869c06e860 0x7f869c15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654336339 0 0) 0x7f869c142fe0 con 0x7f869c140f00 2023-08-27T11:27:00.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86840035b0 con 0x7f869c140f00 2023-08-27T11:27:00.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3372471852 0 0) 0x7f86840035b0 con 0x7f869c140f00 2023-08-27T11:27:00.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 >> v2:172.21.15.138:3300/0 conn(0x7f869c06e860 msgr2=0x7f869c15e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:00.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 --2- 172.21.15.112:0/3906098448 >> v2:172.21.15.138:3300/0 conn(0x7f869c06e860 0x7f869c15e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 >> v1:172.21.15.138:6789/0 conn(0x7f869c0ad210 legacy=0x7f869c06e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c1435a0 con 0x7f869c140f00 2023-08-27T11:27:00.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f8694001670 con 0x7f869c140f00 2023-08-27T11:27:00.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 >> v1:172.21.15.112:6789/0 conn(0x7f869c140f00 legacy=0x7f869c141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 --2- 172.21.15.112:0/3906098448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8684003a00 0x7f8684003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 --2- 172.21.15.112:0/3906098448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8684004850 0x7f8684004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 --2- 172.21.15.112:0/3906098448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8684005210 0x7f868400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c143bf0 con 0x7f8684005210 2023-08-27T11:27:00.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868400c0b0 con 0x7f8684003a00 2023-08-27T11:27:00.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868400c270 con 0x7f8684004850 2023-08-27T11:27:00.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f869b7fe700 1 --2- 172.21.15.112:0/3906098448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8684005210 0x7f868400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f869affd700 1 --2- 172.21.15.112:0/3906098448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8684003a00 0x7f8684003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f869a7fc700 1 --2- 172.21.15.112:0/3906098448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8684004850 0x7f8684004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.114+0000 7f869b7fe700 1 -- 172.21.15.112:0/3906098448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8684004850 msgr2=0x7f8684004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:00.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f869b7fe700 1 --2- 172.21.15.112:0/3906098448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8684004850 0x7f8684004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f869b7fe700 1 -- 172.21.15.112:0/3906098448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8684003a00 msgr2=0x7f8684003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:00.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f869b7fe700 1 --2- 172.21.15.112:0/3906098448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8684003a00 0x7f8684003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f869b7fe700 1 -- 172.21.15.112:0/3906098448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840035b0 con 0x7f8684005210 2023-08-27T11:27:00.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f869a7fc700 1 --2- 172.21.15.112:0/3906098448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8684004850 0x7f8684004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:00.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f869b7fe700 1 -- 172.21.15.112:0/3906098448 reap_dead start 2023-08-27T11:27:00.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f869b7fe700 1 --2- 172.21.15.112:0/3906098448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8684005210 0x7f868400ba10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f869400c240 tx=0x7f8684005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae2d9089aa5247e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:00.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694007c10 con 0x7f8684005210 2023-08-27T11:27:00.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869400cde0 con 0x7f8684005210 2023-08-27T11:27:00.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f8699ffb700 1 -- 172.21.15.112:0/3906098448 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694005ba0 con 0x7f8684005210 2023-08-27T11:27:00.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f86a18ce700 1 -- 172.21.15.112:0/3906098448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8684005210 msgr2=0x7f868400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f86a18ce700 1 --2- 172.21.15.112:0/3906098448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8684005210 0x7f868400ba10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f869400c240 tx=0x7f8684005880 comp rx=0 tx=0).stop 2023-08-27T11:27:00.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f86a18ce700 1 -- 172.21.15.112:0/3906098448 shutdown_connections 2023-08-27T11:27:00.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f86a18ce700 1 --2- 172.21.15.112:0/3906098448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8684005210 0x7f868400ba10 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f86a18ce700 1 -- 172.21.15.112:0/3906098448 >> 172.21.15.112:0/3906098448 conn(0x7f869c06cd30 msgr2=0x7f869c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:00.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f86a18ce700 1 -- 172.21.15.112:0/3906098448 shutdown_connections 2023-08-27T11:27:00.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.115+0000 7f86a18ce700 1 -- 172.21.15.112:0/3906098448 wait complete. 2023-08-27T11:27:00.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.117+0000 7f86a18ce700 1 Processor -- start 2023-08-27T11:27:00.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.117+0000 7f86a18ce700 1 -- start start 2023-08-27T11:27:00.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.117+0000 7f86a18ce700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8684005210 0x7f869c0ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.117+0000 7f86a18ce700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f869c0cecf0 0x7f869c13bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.117+0000 7f86a18ce700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f869c13c470 0x7f869c1443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.117+0000 7f86a18ce700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c13d440 con 0x7f869c0cecf0 2023-08-27T11:27:00.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.117+0000 7f86a18ce700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0cf170 con 0x7f8684005210 2023-08-27T11:27:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.117+0000 7f86a18ce700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c143ed0 con 0x7f869c13c470 2023-08-27T11:27:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f869a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f869c0cecf0 0x7f869c13bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f869a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f869c0cecf0 0x7f869c13bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33990/0 (socket says 172.21.15.112:33990) 2023-08-27T11:27:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f869a7fc700 1 -- 172.21.15.112:0/653131294 learned_addr learned my addr 172.21.15.112:0/653131294 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f869affd700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8684005210 0x7f869c0ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f869b7fe700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f869c13c470 0x7f869c1443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f869a7fc700 1 -- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f869c13c470 msgr2=0x7f869c1443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f869a7fc700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f869c13c470 0x7f869c1443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f869a7fc700 1 -- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8684005210 msgr2=0x7f869c0ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f869a7fc700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8684005210 0x7f869c0ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f869a7fc700 1 -- 172.21.15.112:0/653131294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c0ad770 con 0x7f869c0cecf0 2023-08-27T11:27:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f869b7fe700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f869c13c470 0x7f869c1443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f869a7fc700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f869c0cecf0 0x7f869c13bf30 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f86900115d0 tx=0x7f8690011990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f8683fff700 1 -- 172.21.15.112:0/653131294 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690005c30 con 0x7f869c0cecf0 2023-08-27T11:27:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f8683fff700 1 -- 172.21.15.112:0/653131294 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8690005d90 con 0x7f869c0cecf0 2023-08-27T11:27:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f86a18ce700 1 -- 172.21.15.112:0/653131294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869c13c900 con 0x7f869c0cecf0 2023-08-27T11:27:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f8683fff700 1 -- 172.21.15.112:0/653131294 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690004240 con 0x7f869c0cecf0 2023-08-27T11:27:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.118+0000 7f86a18ce700 1 -- 172.21.15.112:0/653131294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869c13ce80 con 0x7f869c0cecf0 2023-08-27T11:27:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.119+0000 7f86a18ce700 1 -- 172.21.15.112:0/653131294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869c144200 con 0x7f869c0cecf0 2023-08-27T11:27:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.120+0000 7f8683fff700 1 -- 172.21.15.112:0/653131294 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f86900043a0 con 0x7f869c0cecf0 2023-08-27T11:27:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.120+0000 7f8683fff700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f868806a440 0x7f868806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.120+0000 7f8683fff700 1 -- 172.21.15.112:0/653131294 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8690088f20 con 0x7f869c0cecf0 2023-08-27T11:27:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.120+0000 7f869affd700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f868806a440 0x7f868806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.121+0000 7f869affd700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f868806a440 0x7f868806c900 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f869c0b2060 tx=0x7f868c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.124+0000 7f8683fff700 1 -- 172.21.15.112:0/653131294 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8690054290 con 0x7f869c0cecf0 2023-08-27T11:27:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.314+0000 7f86a18ce700 1 -- 172.21.15.112:0/653131294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f869c143d30 con 0x7f869c0cecf0 2023-08-27T11:27:00.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.315+0000 7f8683fff700 1 -- 172.21.15.112:0/653131294 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+18327 (secure 0 0 0) 0x7f869c143d30 con 0x7f869c0cecf0 2023-08-27T11:27:00.317 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:00.318 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":102,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:57.319281+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:54.078457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:54.929983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.318+0000 7f86a18ce700 1 -- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f868806a440 msgr2=0x7f868806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.318+0000 7f86a18ce700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f868806a440 0x7f868806c900 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f869c0b2060 tx=0x7f868c009040 comp rx=0 tx=0).stop 2023-08-27T11:27:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.318+0000 7f86a18ce700 1 -- 172.21.15.112:0/653131294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f869c0cecf0 msgr2=0x7f869c13bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.318+0000 7f86a18ce700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f869c0cecf0 0x7f869c13bf30 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f86900115d0 tx=0x7f8690011990 comp rx=0 tx=0).stop 2023-08-27T11:27:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.318+0000 7f86a18ce700 1 -- 172.21.15.112:0/653131294 shutdown_connections 2023-08-27T11:27:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.318+0000 7f86a18ce700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f868806a440 0x7f868806c900 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.318+0000 7f86a18ce700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f869c13c470 0x7f869c1443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.318+0000 7f86a18ce700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8684005210 0x7f869c0ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.318+0000 7f86a18ce700 1 --2- 172.21.15.112:0/653131294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f869c0cecf0 0x7f869c13bf30 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.318+0000 7f86a18ce700 1 -- 172.21.15.112:0/653131294 >> 172.21.15.112:0/653131294 conn(0x7f869c06cd30 msgr2=0x7f869c15fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:00.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.318+0000 7f86a18ce700 1 -- 172.21.15.112:0/653131294 shutdown_connections 2023-08-27T11:27:00.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.318+0000 7f86a18ce700 1 -- 172.21.15.112:0/653131294 wait complete. 2023-08-27T11:27:00.332 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:27:00.332 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:27:00.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.451+0000 7f5ef2a20700 1 Processor -- start 2023-08-27T11:27:00.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.451+0000 7f5ef2a20700 1 -- start start 2023-08-27T11:27:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.451+0000 7f5ef2a20700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5eec0b1400 0x7f5eec0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.451+0000 7f5ef2a20700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec161850 con 0x7f5eec161ce0 2023-08-27T11:27:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.451+0000 7f5ef2a20700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec0cfc10 con 0x7f5eec0b17e0 2023-08-27T11:27:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.451+0000 7f5ef2a20700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec161200 con 0x7f5eec0b1400 2023-08-27T11:27:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5ef0fbd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5eec161ce0 0x7f5eec06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56614/0 (socket says 172.21.15.112:56614) 2023-08-27T11:27:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5ef0fbd700 1 -- 172.21.15.112:0/472317637 learned_addr learned my addr 172.21.15.112:0/472317637 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222851724 0 0) 0x7f5eec161850 con 0x7f5eec161ce0 2023-08-27T11:27:00.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed40035b0 con 0x7f5eec161ce0 2023-08-27T11:27:00.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eebfff700 1 --2- 172.21.15.112:0/472317637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5eec0b1400 0x7f5eec0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 791757878 0 0) 0x7f5ed40035b0 con 0x7f5eec161ce0 2023-08-27T11:27:00.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5eec0b1400 msgr2=0x7f5eec0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:00.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 --2- 172.21.15.112:0/472317637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5eec0b1400 0x7f5eec0cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 >> v1:172.21.15.138:6789/0 conn(0x7f5eec0b17e0 legacy=0x7f5eec06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec1614e0 con 0x7f5eec161ce0 2023-08-27T11:27:00.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f5ee40019f0 con 0x7f5eec161ce0 2023-08-27T11:27:00.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 >> v1:172.21.15.112:6789/0 conn(0x7f5eec161ce0 legacy=0x7f5eec06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 --2- 172.21.15.112:0/472317637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4003a30 0x7f5ed4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 --2- 172.21.15.112:0/472317637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5eec0b1400 0x7f5ed4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 --2- 172.21.15.112:0/472317637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed4004d90 0x7f5ed4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec161200 con 0x7f5ed4003a30 2023-08-27T11:27:00.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed400bac0 con 0x7f5ed4004d90 2023-08-27T11:27:00.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed400bc80 con 0x7f5eec0b1400 2023-08-27T11:27:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eebfff700 1 --2- 172.21.15.112:0/472317637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4003a30 0x7f5ed4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eeb7fe700 1 --2- 172.21.15.112:0/472317637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5eec0b1400 0x7f5ed4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5ef0fbd700 1 --2- 172.21.15.112:0/472317637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed4004d90 0x7f5ed4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eebfff700 1 -- 172.21.15.112:0/472317637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5eec0b1400 msgr2=0x7f5ed4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eebfff700 1 --2- 172.21.15.112:0/472317637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5eec0b1400 0x7f5ed4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eebfff700 1 -- 172.21.15.112:0/472317637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed4004d90 msgr2=0x7f5ed4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eebfff700 1 --2- 172.21.15.112:0/472317637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed4004d90 0x7f5ed4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.452+0000 7f5eebfff700 1 -- 172.21.15.112:0/472317637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed40035b0 con 0x7f5ed4003a30 2023-08-27T11:27:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.453+0000 7f5eebfff700 1 --2- 172.21.15.112:0/472317637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4003a30 0x7f5ed4003e10 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009b60 tx=0x7f5ed4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7f3e94afd1d81f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.453+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000e6c0 con 0x7f5ed4003a30 2023-08-27T11:27:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.453+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee000e820 con 0x7f5ed4003a30 2023-08-27T11:27:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.453+0000 7f5eeaffd700 1 -- 172.21.15.112:0/472317637 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000eb90 con 0x7f5ed4003a30 2023-08-27T11:27:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.453+0000 7f5ef2a20700 1 -- 172.21.15.112:0/472317637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4003a30 msgr2=0x7f5ed4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.453+0000 7f5ef2a20700 1 --2- 172.21.15.112:0/472317637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4003a30 0x7f5ed4003e10 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009b60 tx=0x7f5ed4005880 comp rx=0 tx=0).stop 2023-08-27T11:27:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.453+0000 7f5ef0fbd700 1 -- 172.21.15.112:0/472317637 reap_dead start 2023-08-27T11:27:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.453+0000 7f5ef2a20700 1 -- 172.21.15.112:0/472317637 shutdown_connections 2023-08-27T11:27:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.453+0000 7f5ef2a20700 1 -- 172.21.15.112:0/472317637 >> 172.21.15.112:0/472317637 conn(0x7f5eec0d0000 msgr2=0x7f5eec0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.453+0000 7f5ef2a20700 1 -- 172.21.15.112:0/472317637 shutdown_connections 2023-08-27T11:27:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.453+0000 7f5ef2a20700 1 -- 172.21.15.112:0/472317637 wait complete. 2023-08-27T11:27:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.455+0000 7f5ef2a20700 1 Processor -- start 2023-08-27T11:27:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.455+0000 7f5ef2a20700 1 -- start start 2023-08-27T11:27:00.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.455+0000 7f5ef2a20700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ed4003a30 0x7f5eec1f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.455+0000 7f5ef2a20700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eec0c70d0 0x7f5eec0c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.455+0000 7f5ef2a20700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eec0cab40 0x7f5eec0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.455+0000 7f5ef2a20700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0cb530 con 0x7f5eec0cab40 2023-08-27T11:27:00.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.455+0000 7f5ef2a20700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0c7b70 con 0x7f5eec0c70d0 2023-08-27T11:27:00.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.455+0000 7f5ef2a20700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0c7e50 con 0x7f5ed4003a30 2023-08-27T11:27:00.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ef0fbd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eec0cab40 0x7f5eec0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ef0fbd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eec0cab40 0x7f5eec0caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34012/0 (socket says 172.21.15.112:34012) 2023-08-27T11:27:00.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ef0fbd700 1 -- 172.21.15.112:0/401133714 learned_addr learned my addr 172.21.15.112:0/401133714 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:00.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5eebfff700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ed4003a30 0x7f5eec1f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5eeb7fe700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eec0c70d0 0x7f5eec0c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ef0fbd700 1 -- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ed4003a30 msgr2=0x7f5eec1f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ef0fbd700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ed4003a30 0x7f5eec1f2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ef0fbd700 1 -- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eec0c70d0 msgr2=0x7f5eec0c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ef0fbd700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eec0c70d0 0x7f5eec0c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ef0fbd700 1 -- 172.21.15.112:0/401133714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec0c8130 con 0x7f5eec0cab40 2023-08-27T11:27:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ef0fbd700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eec0cab40 0x7f5eec0caff0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5eec1620c0 tx=0x7f5ee400e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ee8ff9700 1 -- 172.21.15.112:0/401133714 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee400eb00 con 0x7f5eec0cab40 2023-08-27T11:27:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ee8ff9700 1 -- 172.21.15.112:0/401133714 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee400ec60 con 0x7f5eec0cab40 2023-08-27T11:27:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ef2a20700 1 -- 172.21.15.112:0/401133714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec0c3b30 con 0x7f5eec0cab40 2023-08-27T11:27:00.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ee8ff9700 1 -- 172.21.15.112:0/401133714 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4013420 con 0x7f5eec0cab40 2023-08-27T11:27:00.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.456+0000 7f5ef2a20700 1 -- 172.21.15.112:0/401133714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec0c8e10 con 0x7f5eec0cab40 2023-08-27T11:27:00.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.457+0000 7f5ef2a20700 1 -- 172.21.15.112:0/401133714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eec0c4050 con 0x7f5eec0cab40 2023-08-27T11:27:00.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.458+0000 7f5ee8ff9700 1 -- 172.21.15.112:0/401133714 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f5ee4013580 con 0x7f5eec0cab40 2023-08-27T11:27:00.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.458+0000 7f5ee8ff9700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ed806a440 0x7f5ed806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.458+0000 7f5ee8ff9700 1 -- 172.21.15.112:0/401133714 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5ee4088ec0 con 0x7f5eec0cab40 2023-08-27T11:27:00.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.458+0000 7f5eebfff700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ed806a440 0x7f5ed806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.459+0000 7f5eebfff700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ed806a440 0x7f5ed806c900 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00090b0 tx=0x7f5ee001f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:00.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.462+0000 7f5ee8ff9700 1 -- 172.21.15.112:0/401133714 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5eec0c4050 con 0x7f5eec0cab40 2023-08-27T11:27:00.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.651+0000 7f5ef2a20700 1 -- 172.21.15.112:0/401133714 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5eec0c9390 con 0x7f5ed806a440 2023-08-27T11:27:00.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.661+0000 7f5ee8ff9700 1 -- 172.21.15.112:0/401133714 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312311 (secure 0 0 0) 0x7f5eec0c9390 con 0x7f5ed806a440 2023-08-27T11:27:00.666 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:00.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.664+0000 7f5ef2a20700 1 -- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ed806a440 msgr2=0x7f5ed806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.664+0000 7f5ef2a20700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ed806a440 0x7f5ed806c900 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00090b0 tx=0x7f5ee001f040 comp rx=0 tx=0).stop 2023-08-27T11:27:00.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.664+0000 7f5ef2a20700 1 -- 172.21.15.112:0/401133714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eec0cab40 msgr2=0x7f5eec0caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.664+0000 7f5ef2a20700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eec0cab40 0x7f5eec0caff0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5eec1620c0 tx=0x7f5ee400e440 comp rx=0 tx=0).stop 2023-08-27T11:27:00.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.664+0000 7f5ef2a20700 1 -- 172.21.15.112:0/401133714 shutdown_connections 2023-08-27T11:27:00.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.664+0000 7f5ef2a20700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5ed806a440 0x7f5ed806c900 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.664+0000 7f5ef2a20700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eec0cab40 0x7f5eec0caff0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.664+0000 7f5ef2a20700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ed4003a30 0x7f5eec1f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.664+0000 7f5ef2a20700 1 --2- 172.21.15.112:0/401133714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eec0c70d0 0x7f5eec0c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.664+0000 7f5ef2a20700 1 -- 172.21.15.112:0/401133714 >> 172.21.15.112:0/401133714 conn(0x7f5eec0d0000 msgr2=0x7f5eec0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:00.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.664+0000 7f5ef2a20700 1 -- 172.21.15.112:0/401133714 shutdown_connections 2023-08-27T11:27:00.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.664+0000 7f5ef2a20700 1 -- 172.21.15.112:0/401133714 wait complete. 2023-08-27T11:27:00.671 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:27:00.678 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2023-08-27T11:27:00.378056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260548,"kb_used_data":11044,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754714172,"statfs":{"total":773094113280,"available":772827312128,"internally_reserved":0,"allocated":11309056,"data_stored":5815299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.815594"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327969+0000","last_change":"2023-08-27T11:26:56.338200+0000","last_active":"2023-08-27T11:26:57.327969+0000","last_peered":"2023-08-27T11:26:57.327969+0000","last_clean":"2023-08-27T11:26:57.327969+0000","last_became_active":"2023-08-27T11:26:56.337893+0000","last_became_peered":"2023-08-27T11:26:56.337893+0000","last_unstale":"2023-08-27T11:26:57.327969+0000","last_undegraded":"2023-08-27T11:26:57.327969+0000","last_fullsized":"2023-08-27T11:26:57.327969+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:35.647340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.334148+0000","last_change":"2023-08-27T11:26:56.332252+0000","last_active":"2023-08-27T11:26:56.334148+0000","last_peered":"2023-08-27T11:26:56.334148+0000","last_clean":"2023-08-27T11:26:56.334148+0000","last_became_active":"2023-08-27T11:26:56.332007+0000","last_became_peered":"2023-08-27T11:26:56.332007+0000","last_unstale":"2023-08-27T11:26:56.334148+0000","last_undegraded":"2023-08-27T11:26:56.334148+0000","last_fullsized":"2023-08-27T11:26:56.334148+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:29:20.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.334196+0000","last_change":"2023-08-27T11:26:56.331504+0000","last_active":"2023-08-27T11:26:56.334196+0000","last_peered":"2023-08-27T11:26:56.334196+0000","last_clean":"2023-08-27T11:26:56.334196+0000","last_became_active":"2023-08-27T11:26:56.331123+0000","last_became_peered":"2023-08-27T11:26:56.331123+0000","last_unstale":"2023-08-27T11:26:56.334196+0000","last_undegraded":"2023-08-27T11:26:56.334196+0000","last_fullsized":"2023-08-27T11:26:56.334196+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:23.782267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344302+0000","last_change":"2023-08-27T11:26:56.341402+0000","last_active":"2023-08-27T11:26:56.344302+0000","last_peered":"2023-08-27T11:26:56.344302+0000","last_clean":"2023-08-27T11:26:56.344302+0000","last_became_active":"2023-08-27T11:26:56.341052+0000","last_became_peered":"2023-08-27T11:26:56.341052+0000","last_unstale":"2023-08-27T11:26:56.344302+0000","last_undegraded":"2023-08-27T11:26:56.344302+0000","last_fullsized":"2023-08-27T11:26:56.344302+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:50:24.630824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327517+0000","last_change":"2023-08-27T11:26:56.338947+0000","last_active":"2023-08-27T11:26:57.327517+0000","last_peered":"2023-08-27T11:26:57.327517+0000","last_clean":"2023-08-27T11:26:57.327517+0000","last_became_active":"2023-08-27T11:26:56.338641+0000","last_became_peered":"2023-08-27T11:26:56.338641+0000","last_unstale":"2023-08-27T11:26:57.327517+0000","last_undegraded":"2023-08-27T11:26:57.327517+0000","last_fullsized":"2023-08-27T11:26:57.327517+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:42:52.388238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322543+0000","last_change":"2023-08-27T11:26:56.329620+0000","last_active":"2023-08-27T11:26:57.322543+0000","last_peered":"2023-08-27T11:26:57.322543+0000","last_clean":"2023-08-27T11:26:57.322543+0000","last_became_active":"2023-08-27T11:26:56.329378+0000","last_became_peered":"2023-08-27T11:26:56.329378+0000","last_unstale":"2023-08-27T11:26:57.322543+0000","last_undegraded":"2023-08-27T11:26:57.322543+0000","last_fullsized":"2023-08-27T11:26:57.322543+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:04:42.408211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322419+0000","last_change":"2023-08-27T11:26:56.333654+0000","last_active":"2023-08-27T11:26:57.322419+0000","last_peered":"2023-08-27T11:26:57.322419+0000","last_clean":"2023-08-27T11:26:57.322419+0000","last_became_active":"2023-08-27T11:26:56.333402+0000","last_became_peered":"2023-08-27T11:26:56.333402+0000","last_unstale":"2023-08-27T11:26:57.322419+0000","last_undegraded":"2023-08-27T11:26:57.322419+0000","last_fullsized":"2023-08-27T11:26:57.322419+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:20:28.589177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.332638+0000","last_change":"2023-08-27T11:26:56.330853+0000","last_active":"2023-08-27T11:26:56.332638+0000","last_peered":"2023-08-27T11:26:56.332638+0000","last_clean":"2023-08-27T11:26:56.332638+0000","last_became_active":"2023-08-27T11:26:56.330373+0000","last_became_peered":"2023-08-27T11:26:56.330373+0000","last_unstale":"2023-08-27T11:26:56.332638+0000","last_undegraded":"2023-08-27T11:26:56.332638+0000","last_fullsized":"2023-08-27T11:26:56.332638+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:04:49.296681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339620+0000","last_change":"2023-08-27T11:26:56.335039+0000","last_active":"2023-08-27T11:26:56.339620+0000","last_peered":"2023-08-27T11:26:56.339620+0000","last_clean":"2023-08-27T11:26:56.339620+0000","last_became_active":"2023-08-27T11:26:56.333034+0000","last_became_peered":"2023-08-27T11:26:56.333034+0000","last_unstale":"2023-08-27T11:26:56.339620+0000","last_undegraded":"2023-08-27T11:26:56.339620+0000","last_fullsized":"2023-08-27T11:26:56.339620+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:13:47.340239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.337065+0000","last_change":"2023-08-27T11:26:56.333879+0000","last_active":"2023-08-27T11:26:56.337065+0000","last_peered":"2023-08-27T11:26:56.337065+0000","last_clean":"2023-08-27T11:26:56.337065+0000","last_became_active":"2023-08-27T11:26:56.333613+0000","last_became_peered":"2023-08-27T11:26:56.333613+0000","last_unstale":"2023-08-27T11:26:56.337065+0000","last_undegraded":"2023-08-27T11:26:56.337065+0000","last_fullsized":"2023-08-27T11:26:56.337065+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:07.580879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327775+0000","last_change":"2023-08-27T11:26:56.344921+0000","last_active":"2023-08-27T11:26:57.327775+0000","last_peered":"2023-08-27T11:26:57.327775+0000","last_clean":"2023-08-27T11:26:57.327775+0000","last_became_active":"2023-08-27T11:26:56.344655+0000","last_became_peered":"2023-08-27T11:26:56.344655+0000","last_unstale":"2023-08-27T11:26:57.327775+0000","last_undegraded":"2023-08-27T11:26:57.327775+0000","last_fullsized":"2023-08-27T11:26:57.327775+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:30:05.139797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.323064+0000","last_change":"2023-08-27T11:26:56.339849+0000","last_active":"2023-08-27T11:26:57.323064+0000","last_peered":"2023-08-27T11:26:57.323064+0000","last_clean":"2023-08-27T11:26:57.323064+0000","last_became_active":"2023-08-27T11:26:56.339417+0000","last_became_peered":"2023-08-27T11:26:56.339417+0000","last_unstale":"2023-08-27T11:26:57.323064+0000","last_undegraded":"2023-08-27T11:26:57.323064+0000","last_fullsized":"2023-08-27T11:26:57.323064+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:29:20.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.337044+0000","last_change":"2023-08-27T11:26:56.333995+0000","last_active":"2023-08-27T11:26:56.337044+0000","last_peered":"2023-08-27T11:26:56.337044+0000","last_clean":"2023-08-27T11:26:56.337044+0000","last_became_active":"2023-08-27T11:26:56.333536+0000","last_became_peered":"2023-08-27T11:26:56.333536+0000","last_unstale":"2023-08-27T11:26:56.337044+0000","last_undegraded":"2023-08-27T11:26:56.337044+0000","last_fullsized":"2023-08-27T11:26:56.337044+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:07:54.390021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.334883+0000","last_change":"2023-08-27T11:26:56.332765+0000","last_active":"2023-08-27T11:26:56.334883+0000","last_peered":"2023-08-27T11:26:56.334883+0000","last_clean":"2023-08-27T11:26:56.334883+0000","last_became_active":"2023-08-27T11:26:56.332321+0000","last_became_peered":"2023-08-27T11:26:56.332321+0000","last_unstale":"2023-08-27T11:26:56.334883+0000","last_undegraded":"2023-08-27T11:26:56.334883+0000","last_fullsized":"2023-08-27T11:26:56.334883+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:22:40.892753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327270+0000","last_change":"2023-08-27T11:26:56.342939+0000","last_active":"2023-08-27T11:26:57.327270+0000","last_peered":"2023-08-27T11:26:57.327270+0000","last_clean":"2023-08-27T11:26:57.327270+0000","last_became_active":"2023-08-27T11:26:56.342711+0000","last_became_peered":"2023-08-27T11:26:56.342711+0000","last_unstale":"2023-08-27T11:26:57.327270+0000","last_undegraded":"2023-08-27T11:26:57.327270+0000","last_fullsized":"2023-08-27T11:26:57.327270+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:40:09.000394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322572+0000","last_change":"2023-08-27T11:26:56.330300+0000","last_active":"2023-08-27T11:26:57.322572+0000","last_peered":"2023-08-27T11:26:57.322572+0000","last_clean":"2023-08-27T11:26:57.322572+0000","last_became_active":"2023-08-27T11:26:56.330041+0000","last_became_peered":"2023-08-27T11:26:56.330041+0000","last_unstale":"2023-08-27T11:26:57.322572+0000","last_undegraded":"2023-08-27T11:26:57.322572+0000","last_fullsized":"2023-08-27T11:26:57.322572+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:46:17.505423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339730+0000","last_change":"2023-08-27T11:26:56.338258+0000","last_active":"2023-08-27T11:26:56.339730+0000","last_peered":"2023-08-27T11:26:56.339730+0000","last_clean":"2023-08-27T11:26:56.339730+0000","last_became_active":"2023-08-27T11:26:56.337945+0000","last_became_peered":"2023-08-27T11:26:56.337945+0000","last_unstale":"2023-08-27T11:26:56.339730+0000","last_undegraded":"2023-08-27T11:26:56.339730+0000","last_fullsized":"2023-08-27T11:26:56.339730+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:57:11.690365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.332559+0000","last_change":"2023-08-27T11:26:56.330006+0000","last_active":"2023-08-27T11:26:56.332559+0000","last_peered":"2023-08-27T11:26:56.332559+0000","last_clean":"2023-08-27T11:26:56.332559+0000","last_became_active":"2023-08-27T11:26:56.329672+0000","last_became_peered":"2023-08-27T11:26:56.329672+0000","last_unstale":"2023-08-27T11:26:56.332559+0000","last_undegraded":"2023-08-27T11:26:56.332559+0000","last_fullsized":"2023-08-27T11:26:56.332559+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:57.070690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.329037+0000","last_change":"2023-08-27T11:26:56.338425+0000","last_active":"2023-08-27T11:26:57.329037+0000","last_peered":"2023-08-27T11:26:57.329037+0000","last_clean":"2023-08-27T11:26:57.329037+0000","last_became_active":"2023-08-27T11:26:56.334863+0000","last_became_peered":"2023-08-27T11:26:56.334863+0000","last_unstale":"2023-08-27T11:26:57.329037+0000","last_undegraded":"2023-08-27T11:26:57.329037+0000","last_fullsized":"2023-08-27T11:26:57.329037+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:14.305235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.338189+0000","last_change":"2023-08-27T11:26:56.336262+0000","last_active":"2023-08-27T11:26:56.338189+0000","last_peered":"2023-08-27T11:26:56.338189+0000","last_clean":"2023-08-27T11:26:56.338189+0000","last_became_active":"2023-08-27T11:26:56.335989+0000","last_became_peered":"2023-08-27T11:26:56.335989+0000","last_unstale":"2023-08-27T11:26:56.338189+0000","last_undegraded":"2023-08-27T11:26:56.338189+0000","last_fullsized":"2023-08-27T11:26:56.338189+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:17:12.187523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.340006+0000","last_change":"2023-08-27T11:26:56.334231+0000","last_active":"2023-08-27T11:26:56.340006+0000","last_peered":"2023-08-27T11:26:56.340006+0000","last_clean":"2023-08-27T11:26:56.340006+0000","last_became_active":"2023-08-27T11:26:56.333968+0000","last_became_peered":"2023-08-27T11:26:56.333968+0000","last_unstale":"2023-08-27T11:26:56.340006+0000","last_undegraded":"2023-08-27T11:26:56.340006+0000","last_fullsized":"2023-08-27T11:26:56.340006+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:27:42.625540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327575+0000","last_change":"2023-08-27T11:26:56.340517+0000","last_active":"2023-08-27T11:26:57.327575+0000","last_peered":"2023-08-27T11:26:57.327575+0000","last_clean":"2023-08-27T11:26:57.327575+0000","last_became_active":"2023-08-27T11:26:56.340269+0000","last_became_peered":"2023-08-27T11:26:56.340269+0000","last_unstale":"2023-08-27T11:26:57.327575+0000","last_undegraded":"2023-08-27T11:26:57.327575+0000","last_fullsized":"2023-08-27T11:26:57.327575+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:39.459063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339545+0000","last_change":"2023-08-27T11:26:56.332850+0000","last_active":"2023-08-27T11:26:56.339545+0000","last_peered":"2023-08-27T11:26:56.339545+0000","last_clean":"2023-08-27T11:26:56.339545+0000","last_became_active":"2023-08-27T11:26:56.332542+0000","last_became_peered":"2023-08-27T11:26:56.332542+0000","last_unstale":"2023-08-27T11:26:56.339545+0000","last_undegraded":"2023-08-27T11:26:56.339545+0000","last_fullsized":"2023-08-27T11:26:56.339545+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:26:35.955137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344625+0000","last_change":"2023-08-27T11:26:56.339443+0000","last_active":"2023-08-27T11:26:56.344625+0000","last_peered":"2023-08-27T11:26:56.344625+0000","last_clean":"2023-08-27T11:26:56.344625+0000","last_became_active":"2023-08-27T11:26:56.338997+0000","last_became_peered":"2023-08-27T11:26:56.338997+0000","last_unstale":"2023-08-27T11:26:56.344625+0000","last_undegraded":"2023-08-27T11:26:56.344625+0000","last_fullsized":"2023-08-27T11:26:56.344625+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:48:59.244598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344676+0000","last_change":"2023-08-27T11:26:56.341489+0000","last_active":"2023-08-27T11:26:56.344676+0000","last_peered":"2023-08-27T11:26:56.344676+0000","last_clean":"2023-08-27T11:26:56.344676+0000","last_became_active":"2023-08-27T11:26:56.340689+0000","last_became_peered":"2023-08-27T11:26:56.340689+0000","last_unstale":"2023-08-27T11:26:56.344676+0000","last_undegraded":"2023-08-27T11:26:56.344676+0000","last_fullsized":"2023-08-27T11:26:56.344676+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:44.284960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339212+0000","last_change":"2023-08-27T11:26:56.337136+0000","last_active":"2023-08-27T11:26:56.339212+0000","last_peered":"2023-08-27T11:26:56.339212+0000","last_clean":"2023-08-27T11:26:56.339212+0000","last_became_active":"2023-08-27T11:26:56.336870+0000","last_became_peered":"2023-08-27T11:26:56.336870+0000","last_unstale":"2023-08-27T11:26:56.339212+0000","last_undegraded":"2023-08-27T11:26:56.339212+0000","last_fullsized":"2023-08-27T11:26:56.339212+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:21:08.994604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.329196+0000","last_change":"2023-08-27T11:26:56.334680+0000","last_active":"2023-08-27T11:26:57.329196+0000","last_peered":"2023-08-27T11:26:57.329196+0000","last_clean":"2023-08-27T11:26:57.329196+0000","last_became_active":"2023-08-27T11:26:56.334385+0000","last_became_peered":"2023-08-27T11:26:56.334385+0000","last_unstale":"2023-08-27T11:26:57.329196+0000","last_undegraded":"2023-08-27T11:26:57.329196+0000","last_fullsized":"2023-08-27T11:26:57.329196+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:35.647340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322939+0000","last_change":"2023-08-27T11:26:56.335394+0000","last_active":"2023-08-27T11:26:57.322939+0000","last_peered":"2023-08-27T11:26:57.322939+0000","last_clean":"2023-08-27T11:26:57.322939+0000","last_became_active":"2023-08-27T11:26:56.335132+0000","last_became_peered":"2023-08-27T11:26:56.335132+0000","last_unstale":"2023-08-27T11:26:57.322939+0000","last_undegraded":"2023-08-27T11:26:57.322939+0000","last_fullsized":"2023-08-27T11:26:57.322939+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:43.704451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.332124+0000","last_change":"2023-08-27T11:26:56.330318+0000","last_active":"2023-08-27T11:26:56.332124+0000","last_peered":"2023-08-27T11:26:56.332124+0000","last_clean":"2023-08-27T11:26:56.332124+0000","last_became_active":"2023-08-27T11:26:56.330052+0000","last_became_peered":"2023-08-27T11:26:56.330052+0000","last_unstale":"2023-08-27T11:26:56.332124+0000","last_undegraded":"2023-08-27T11:26:56.332124+0000","last_fullsized":"2023-08-27T11:26:56.332124+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:04:37.323648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.343736+0000","last_change":"2023-08-27T11:26:56.341456+0000","last_active":"2023-08-27T11:26:56.343736+0000","last_peered":"2023-08-27T11:26:56.343736+0000","last_clean":"2023-08-27T11:26:56.343736+0000","last_became_active":"2023-08-27T11:26:56.341141+0000","last_became_peered":"2023-08-27T11:26:56.341141+0000","last_unstale":"2023-08-27T11:26:56.343736+0000","last_undegraded":"2023-08-27T11:26:56.343736+0000","last_fullsized":"2023-08-27T11:26:56.343736+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:06:26.167325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344484+0000","last_change":"2023-08-27T11:26:56.336729+0000","last_active":"2023-08-27T11:26:56.344484+0000","last_peered":"2023-08-27T11:26:56.344484+0000","last_clean":"2023-08-27T11:26:56.344484+0000","last_became_active":"2023-08-27T11:26:56.336466+0000","last_became_peered":"2023-08-27T11:26:56.336466+0000","last_unstale":"2023-08-27T11:26:56.344484+0000","last_undegraded":"2023-08-27T11:26:56.344484+0000","last_fullsized":"2023-08-27T11:26:56.344484+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:50:18.052365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344604+0000","last_change":"2023-08-27T11:26:56.339258+0000","last_active":"2023-08-27T11:26:56.344604+0000","last_peered":"2023-08-27T11:26:56.344604+0000","last_clean":"2023-08-27T11:26:56.344604+0000","last_became_active":"2023-08-27T11:26:56.338644+0000","last_became_peered":"2023-08-27T11:26:56.338644+0000","last_unstale":"2023-08-27T11:26:56.344604+0000","last_undegraded":"2023-08-27T11:26:56.344604+0000","last_fullsized":"2023-08-27T11:26:56.344604+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:26.089004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.336992+0000","last_change":"2023-08-27T11:26:56.334767+0000","last_active":"2023-08-27T11:26:56.336992+0000","last_peered":"2023-08-27T11:26:56.336992+0000","last_clean":"2023-08-27T11:26:56.336992+0000","last_became_active":"2023-08-27T11:26:56.334152+0000","last_became_peered":"2023-08-27T11:26:56.334152+0000","last_unstale":"2023-08-27T11:26:56.336992+0000","last_undegraded":"2023-08-27T11:26:56.336992+0000","last_fullsized":"2023-08-27T11:26:56.336992+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:48:20.020708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"101'4","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328846+0000","last_change":"2023-08-27T11:26:55.586454+0000","last_active":"2023-08-27T11:26:57.328846+0000","last_peered":"2023-08-27T11:26:57.328846+0000","last_clean":"2023-08-27T11:26:57.328846+0000","last_became_active":"2023-08-27T11:26:55.586152+0000","last_became_peered":"2023-08-27T11:26:55.586152+0000","last_unstale":"2023-08-27T11:26:57.328846+0000","last_undegraded":"2023-08-27T11:26:57.328846+0000","last_fullsized":"2023-08-27T11:26:57.328846+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:45:31.804998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.323060+0000","last_change":"2023-08-27T11:26:56.329075+0000","last_active":"2023-08-27T11:26:57.323060+0000","last_peered":"2023-08-27T11:26:57.323060+0000","last_clean":"2023-08-27T11:26:57.323060+0000","last_became_active":"2023-08-27T11:26:56.328805+0000","last_became_peered":"2023-08-27T11:26:56.328805+0000","last_unstale":"2023-08-27T11:26:57.323060+0000","last_undegraded":"2023-08-27T11:26:57.323060+0000","last_fullsized":"2023-08-27T11:26:57.323060+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:23.782267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"101'1","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.575053+0000","last_change":"2023-08-27T11:26:55.578778+0000","last_active":"2023-08-27T11:26:56.575053+0000","last_peered":"2023-08-27T11:26:56.575053+0000","last_clean":"2023-08-27T11:26:56.575053+0000","last_became_active":"2023-08-27T11:26:55.578526+0000","last_became_peered":"2023-08-27T11:26:55.578526+0000","last_unstale":"2023-08-27T11:26:56.575053+0000","last_undegraded":"2023-08-27T11:26:56.575053+0000","last_fullsized":"2023-08-27T11:26:56.575053+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:34:19.671570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344188+0000","last_change":"2023-08-27T11:26:56.335716+0000","last_active":"2023-08-27T11:26:56.344188+0000","last_peered":"2023-08-27T11:26:56.344188+0000","last_clean":"2023-08-27T11:26:56.344188+0000","last_became_active":"2023-08-27T11:26:56.335414+0000","last_became_peered":"2023-08-27T11:26:56.335414+0000","last_unstale":"2023-08-27T11:26:56.344188+0000","last_undegraded":"2023-08-27T11:26:56.344188+0000","last_fullsized":"2023-08-27T11:26:56.344188+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:00:17.644967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"101'2","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.578039+0000","last_change":"2023-08-27T11:26:55.587764+0000","last_active":"2023-08-27T11:26:56.578039+0000","last_peered":"2023-08-27T11:26:56.578039+0000","last_clean":"2023-08-27T11:26:56.578039+0000","last_became_active":"2023-08-27T11:26:55.587413+0000","last_became_peered":"2023-08-27T11:26:55.587413+0000","last_unstale":"2023-08-27T11:26:56.578039+0000","last_undegraded":"2023-08-27T11:26:56.578039+0000","last_fullsized":"2023-08-27T11:26:56.578039+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:41:16.545929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344277+0000","last_change":"2023-08-27T11:26:56.341218+0000","last_active":"2023-08-27T11:26:56.344277+0000","last_peered":"2023-08-27T11:26:56.344277+0000","last_clean":"2023-08-27T11:26:56.344277+0000","last_became_active":"2023-08-27T11:26:56.340907+0000","last_became_peered":"2023-08-27T11:26:56.340907+0000","last_unstale":"2023-08-27T11:26:56.344277+0000","last_undegraded":"2023-08-27T11:26:56.344277+0000","last_fullsized":"2023-08-27T11:26:56.344277+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:26:59.498363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"101'2","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.576506+0000","last_change":"2023-08-27T11:26:55.581671+0000","last_active":"2023-08-27T11:26:56.576506+0000","last_peered":"2023-08-27T11:26:56.576506+0000","last_clean":"2023-08-27T11:26:56.576506+0000","last_became_active":"2023-08-27T11:26:55.581391+0000","last_became_peered":"2023-08-27T11:26:55.581391+0000","last_unstale":"2023-08-27T11:26:56.576506+0000","last_undegraded":"2023-08-27T11:26:56.576506+0000","last_fullsized":"2023-08-27T11:26:56.576506+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:05:49.716981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.336464+0000","last_change":"2023-08-27T11:26:56.329286+0000","last_active":"2023-08-27T11:26:56.336464+0000","last_peered":"2023-08-27T11:26:56.336464+0000","last_clean":"2023-08-27T11:26:56.336464+0000","last_became_active":"2023-08-27T11:26:56.328972+0000","last_became_peered":"2023-08-27T11:26:56.328972+0000","last_unstale":"2023-08-27T11:26:56.336464+0000","last_undegraded":"2023-08-27T11:26:56.336464+0000","last_fullsized":"2023-08-27T11:26:56.336464+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:59:33.133043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"101'1","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327580+0000","last_change":"2023-08-27T11:26:55.582175+0000","last_active":"2023-08-27T11:26:57.327580+0000","last_peered":"2023-08-27T11:26:57.327580+0000","last_clean":"2023-08-27T11:26:57.327580+0000","last_became_active":"2023-08-27T11:26:55.581875+0000","last_became_peered":"2023-08-27T11:26:55.581875+0000","last_unstale":"2023-08-27T11:26:57.327580+0000","last_undegraded":"2023-08-27T11:26:57.327580+0000","last_fullsized":"2023-08-27T11:26:57.327580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:37:16.373316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327387+0000","last_change":"2023-08-27T11:26:56.341989+0000","last_active":"2023-08-27T11:26:57.327387+0000","last_peered":"2023-08-27T11:26:57.327387+0000","last_clean":"2023-08-27T11:26:57.327387+0000","last_became_active":"2023-08-27T11:26:56.341705+0000","last_became_peered":"2023-08-27T11:26:56.341705+0000","last_unstale":"2023-08-27T11:26:57.327387+0000","last_undegraded":"2023-08-27T11:26:57.327387+0000","last_fullsized":"2023-08-27T11:26:57.327387+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:34:28.655765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"101'1","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322545+0000","last_change":"2023-08-27T11:26:55.581626+0000","last_active":"2023-08-27T11:26:57.322545+0000","last_peered":"2023-08-27T11:26:57.322545+0000","last_clean":"2023-08-27T11:26:57.322545+0000","last_became_active":"2023-08-27T11:26:55.581370+0000","last_became_peered":"2023-08-27T11:26:55.581370+0000","last_unstale":"2023-08-27T11:26:57.322545+0000","last_undegraded":"2023-08-27T11:26:57.322545+0000","last_fullsized":"2023-08-27T11:26:57.322545+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:20:26.130428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.335985+0000","last_change":"2023-08-27T11:26:56.335006+0000","last_active":"2023-08-27T11:26:56.335985+0000","last_peered":"2023-08-27T11:26:56.335985+0000","last_clean":"2023-08-27T11:26:56.335985+0000","last_became_active":"2023-08-27T11:26:56.334692+0000","last_became_peered":"2023-08-27T11:26:56.334692+0000","last_unstale":"2023-08-27T11:26:56.335985+0000","last_undegraded":"2023-08-27T11:26:56.335985+0000","last_fullsized":"2023-08-27T11:26:56.335985+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:57:36.557530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.330710+0000","last_change":"2023-08-27T11:26:55.583745+0000","last_active":"2023-08-27T11:26:56.330710+0000","last_peered":"2023-08-27T11:26:56.330710+0000","last_clean":"2023-08-27T11:26:56.330710+0000","last_became_active":"2023-08-27T11:26:55.583446+0000","last_became_peered":"2023-08-27T11:26:55.583446+0000","last_unstale":"2023-08-27T11:26:56.330710+0000","last_undegraded":"2023-08-27T11:26:56.330710+0000","last_fullsized":"2023-08-27T11:26:56.330710+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:28:45.011961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.336599+0000","last_change":"2023-08-27T11:26:56.327884+0000","last_active":"2023-08-27T11:26:56.336599+0000","last_peered":"2023-08-27T11:26:56.336599+0000","last_clean":"2023-08-27T11:26:56.336599+0000","last_became_active":"2023-08-27T11:26:56.327633+0000","last_became_peered":"2023-08-27T11:26:56.327633+0000","last_unstale":"2023-08-27T11:26:56.336599+0000","last_undegraded":"2023-08-27T11:26:56.336599+0000","last_fullsized":"2023-08-27T11:26:56.336599+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:45:00.914901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322704+0000","last_change":"2023-08-27T11:26:55.584436+0000","last_active":"2023-08-27T11:26:57.322704+0000","last_peered":"2023-08-27T11:26:57.322704+0000","last_clean":"2023-08-27T11:26:57.322704+0000","last_became_active":"2023-08-27T11:26:55.584121+0000","last_became_peered":"2023-08-27T11:26:55.584121+0000","last_unstale":"2023-08-27T11:26:57.322704+0000","last_undegraded":"2023-08-27T11:26:57.322704+0000","last_fullsized":"2023-08-27T11:26:57.322704+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:53.593374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344373+0000","last_change":"2023-08-27T11:26:56.336443+0000","last_active":"2023-08-27T11:26:56.344373+0000","last_peered":"2023-08-27T11:26:56.344373+0000","last_clean":"2023-08-27T11:26:56.344373+0000","last_became_active":"2023-08-27T11:26:56.336159+0000","last_became_peered":"2023-08-27T11:26:56.336159+0000","last_unstale":"2023-08-27T11:26:56.344373+0000","last_undegraded":"2023-08-27T11:26:56.344373+0000","last_fullsized":"2023-08-27T11:26:56.344373+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:36:12.710448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"101'1","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.579478+0000","last_change":"2023-08-27T11:26:55.579157+0000","last_active":"2023-08-27T11:26:56.579478+0000","last_peered":"2023-08-27T11:26:56.579478+0000","last_clean":"2023-08-27T11:26:56.579478+0000","last_became_active":"2023-08-27T11:26:55.578848+0000","last_became_peered":"2023-08-27T11:26:55.578848+0000","last_unstale":"2023-08-27T11:26:56.579478+0000","last_undegraded":"2023-08-27T11:26:56.579478+0000","last_fullsized":"2023-08-27T11:26:56.579478+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:04:45.535534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327922+0000","last_change":"2023-08-27T11:26:56.337690+0000","last_active":"2023-08-27T11:26:57.327922+0000","last_peered":"2023-08-27T11:26:57.327922+0000","last_clean":"2023-08-27T11:26:57.327922+0000","last_became_active":"2023-08-27T11:26:56.337380+0000","last_became_peered":"2023-08-27T11:26:56.337380+0000","last_unstale":"2023-08-27T11:26:57.327922+0000","last_undegraded":"2023-08-27T11:26:57.327922+0000","last_fullsized":"2023-08-27T11:26:57.327922+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:06:42.811268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"101'2","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.577961+0000","last_change":"2023-08-27T11:26:55.580451+0000","last_active":"2023-08-27T11:26:56.577961+0000","last_peered":"2023-08-27T11:26:56.577961+0000","last_clean":"2023-08-27T11:26:56.577961+0000","last_became_active":"2023-08-27T11:26:55.580114+0000","last_became_peered":"2023-08-27T11:26:55.580114+0000","last_unstale":"2023-08-27T11:26:56.577961+0000","last_undegraded":"2023-08-27T11:26:56.577961+0000","last_fullsized":"2023-08-27T11:26:56.577961+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:45:42.879972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.323072+0000","last_change":"2023-08-27T11:26:56.331051+0000","last_active":"2023-08-27T11:26:57.323072+0000","last_peered":"2023-08-27T11:26:57.323072+0000","last_clean":"2023-08-27T11:26:57.323072+0000","last_became_active":"2023-08-27T11:26:56.330821+0000","last_became_peered":"2023-08-27T11:26:56.330821+0000","last_unstale":"2023-08-27T11:26:57.323072+0000","last_undegraded":"2023-08-27T11:26:57.323072+0000","last_fullsized":"2023-08-27T11:26:57.323072+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:00:27.371100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"101'1","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.578168+0000","last_change":"2023-08-27T11:26:55.587419+0000","last_active":"2023-08-27T11:26:56.578168+0000","last_peered":"2023-08-27T11:26:56.578168+0000","last_clean":"2023-08-27T11:26:56.578168+0000","last_became_active":"2023-08-27T11:26:55.586785+0000","last_became_peered":"2023-08-27T11:26:55.586785+0000","last_unstale":"2023-08-27T11:26:56.578168+0000","last_undegraded":"2023-08-27T11:26:56.578168+0000","last_fullsized":"2023-08-27T11:26:56.578168+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:23.921659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344504+0000","last_change":"2023-08-27T11:26:56.337097+0000","last_active":"2023-08-27T11:26:56.344504+0000","last_peered":"2023-08-27T11:26:56.344504+0000","last_clean":"2023-08-27T11:26:56.344504+0000","last_became_active":"2023-08-27T11:26:56.336480+0000","last_became_peered":"2023-08-27T11:26:56.336480+0000","last_unstale":"2023-08-27T11:26:56.344504+0000","last_undegraded":"2023-08-27T11:26:56.344504+0000","last_fullsized":"2023-08-27T11:26:56.344504+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:42:39.682934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.327636+0000","last_change":"2023-08-27T11:26:55.590251+0000","last_active":"2023-08-27T11:26:56.327636+0000","last_peered":"2023-08-27T11:26:56.327636+0000","last_clean":"2023-08-27T11:26:56.327636+0000","last_became_active":"2023-08-27T11:26:55.590020+0000","last_became_peered":"2023-08-27T11:26:55.590020+0000","last_unstale":"2023-08-27T11:26:56.327636+0000","last_undegraded":"2023-08-27T11:26:56.327636+0000","last_fullsized":"2023-08-27T11:26:56.327636+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:47:29.396825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322406+0000","last_change":"2023-08-27T11:26:56.331375+0000","last_active":"2023-08-27T11:26:57.322406+0000","last_peered":"2023-08-27T11:26:57.322406+0000","last_clean":"2023-08-27T11:26:57.322406+0000","last_became_active":"2023-08-27T11:26:56.331109+0000","last_became_peered":"2023-08-27T11:26:56.331109+0000","last_unstale":"2023-08-27T11:26:57.322406+0000","last_undegraded":"2023-08-27T11:26:57.322406+0000","last_fullsized":"2023-08-27T11:26:57.322406+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:31.087412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"101'2","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322785+0000","last_change":"2023-08-27T11:26:55.584910+0000","last_active":"2023-08-27T11:26:57.322785+0000","last_peered":"2023-08-27T11:26:57.322785+0000","last_clean":"2023-08-27T11:26:57.322785+0000","last_became_active":"2023-08-27T11:26:55.584623+0000","last_became_peered":"2023-08-27T11:26:55.584623+0000","last_unstale":"2023-08-27T11:26:57.322785+0000","last_undegraded":"2023-08-27T11:26:57.322785+0000","last_fullsized":"2023-08-27T11:26:57.322785+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:13:17.448970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339457+0000","last_change":"2023-08-27T11:26:56.337341+0000","last_active":"2023-08-27T11:26:56.339457+0000","last_peered":"2023-08-27T11:26:56.339457+0000","last_clean":"2023-08-27T11:26:56.339457+0000","last_became_active":"2023-08-27T11:26:56.337021+0000","last_became_peered":"2023-08-27T11:26:56.337021+0000","last_unstale":"2023-08-27T11:26:56.339457+0000","last_undegraded":"2023-08-27T11:26:56.339457+0000","last_fullsized":"2023-08-27T11:26:56.339457+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:59:38.308471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.325667+0000","last_change":"2023-08-27T11:26:55.588388+0000","last_active":"2023-08-27T11:26:56.325667+0000","last_peered":"2023-08-27T11:26:56.325667+0000","last_clean":"2023-08-27T11:26:56.325667+0000","last_became_active":"2023-08-27T11:26:55.588047+0000","last_became_peered":"2023-08-27T11:26:55.588047+0000","last_unstale":"2023-08-27T11:26:56.325667+0000","last_undegraded":"2023-08-27T11:26:56.325667+0000","last_fullsized":"2023-08-27T11:26:56.325667+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:25:36.557116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.334901+0000","last_change":"2023-08-27T11:26:56.332807+0000","last_active":"2023-08-27T11:26:56.334901+0000","last_peered":"2023-08-27T11:26:56.334901+0000","last_clean":"2023-08-27T11:26:56.334901+0000","last_became_active":"2023-08-27T11:26:56.332429+0000","last_became_peered":"2023-08-27T11:26:56.332429+0000","last_unstale":"2023-08-27T11:26:56.334901+0000","last_undegraded":"2023-08-27T11:26:56.334901+0000","last_fullsized":"2023-08-27T11:26:56.334901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:37:01.825078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.321372+0000","last_change":"2023-08-27T11:26:55.579967+0000","last_active":"2023-08-27T11:26:56.321372+0000","last_peered":"2023-08-27T11:26:56.321372+0000","last_clean":"2023-08-27T11:26:56.321372+0000","last_became_active":"2023-08-27T11:26:55.579711+0000","last_became_peered":"2023-08-27T11:26:55.579711+0000","last_unstale":"2023-08-27T11:26:56.321372+0000","last_undegraded":"2023-08-27T11:26:56.321372+0000","last_fullsized":"2023-08-27T11:26:56.321372+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:14.065624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344302+0000","last_change":"2023-08-27T11:26:56.336199+0000","last_active":"2023-08-27T11:26:56.344302+0000","last_peered":"2023-08-27T11:26:56.344302+0000","last_clean":"2023-08-27T11:26:56.344302+0000","last_became_active":"2023-08-27T11:26:56.335902+0000","last_became_peered":"2023-08-27T11:26:56.335902+0000","last_unstale":"2023-08-27T11:26:56.344302+0000","last_undegraded":"2023-08-27T11:26:56.344302+0000","last_fullsized":"2023-08-27T11:26:56.344302+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:03.648033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.325122+0000","last_change":"2023-08-27T11:26:55.587769+0000","last_active":"2023-08-27T11:26:56.325122+0000","last_peered":"2023-08-27T11:26:56.325122+0000","last_clean":"2023-08-27T11:26:56.325122+0000","last_became_active":"2023-08-27T11:26:55.587366+0000","last_became_peered":"2023-08-27T11:26:55.587366+0000","last_unstale":"2023-08-27T11:26:56.325122+0000","last_undegraded":"2023-08-27T11:26:56.325122+0000","last_fullsized":"2023-08-27T11:26:56.325122+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:00:35.636685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322900+0000","last_change":"2023-08-27T11:26:56.335078+0000","last_active":"2023-08-27T11:26:57.322900+0000","last_peered":"2023-08-27T11:26:57.322900+0000","last_clean":"2023-08-27T11:26:57.322900+0000","last_became_active":"2023-08-27T11:26:56.334811+0000","last_became_peered":"2023-08-27T11:26:56.334811+0000","last_unstale":"2023-08-27T11:26:57.322900+0000","last_undegraded":"2023-08-27T11:26:57.322900+0000","last_fullsized":"2023-08-27T11:26:57.322900+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:05.072625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"101'2","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.323345+0000","last_change":"2023-08-27T11:26:55.579468+0000","last_active":"2023-08-27T11:26:57.323345+0000","last_peered":"2023-08-27T11:26:57.323345+0000","last_clean":"2023-08-27T11:26:57.323345+0000","last_became_active":"2023-08-27T11:26:55.579211+0000","last_became_peered":"2023-08-27T11:26:55.579211+0000","last_unstale":"2023-08-27T11:26:57.323345+0000","last_undegraded":"2023-08-27T11:26:57.323345+0000","last_fullsized":"2023-08-27T11:26:57.323345+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:24:35.451519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.336495+0000","last_change":"2023-08-27T11:26:56.329332+0000","last_active":"2023-08-27T11:26:56.336495+0000","last_peered":"2023-08-27T11:26:56.336495+0000","last_clean":"2023-08-27T11:26:56.336495+0000","last_became_active":"2023-08-27T11:26:56.329030+0000","last_became_peered":"2023-08-27T11:26:56.329030+0000","last_unstale":"2023-08-27T11:26:56.336495+0000","last_undegraded":"2023-08-27T11:26:56.336495+0000","last_fullsized":"2023-08-27T11:26:56.336495+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:21.500768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328803+0000","last_change":"2023-08-27T11:26:55.579972+0000","last_active":"2023-08-27T11:26:57.328803+0000","last_peered":"2023-08-27T11:26:57.328803+0000","last_clean":"2023-08-27T11:26:57.328803+0000","last_became_active":"2023-08-27T11:26:55.579637+0000","last_became_peered":"2023-08-27T11:26:55.579637+0000","last_unstale":"2023-08-27T11:26:57.328803+0000","last_undegraded":"2023-08-27T11:26:57.328803+0000","last_fullsized":"2023-08-27T11:26:57.328803+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:44:41.838536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344316+0000","last_change":"2023-08-27T11:26:56.341267+0000","last_active":"2023-08-27T11:26:56.344316+0000","last_peered":"2023-08-27T11:26:56.344316+0000","last_clean":"2023-08-27T11:26:56.344316+0000","last_became_active":"2023-08-27T11:26:56.340982+0000","last_became_peered":"2023-08-27T11:26:56.340982+0000","last_unstale":"2023-08-27T11:26:56.344316+0000","last_undegraded":"2023-08-27T11:26:56.344316+0000","last_fullsized":"2023-08-27T11:26:56.344316+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:08:06.414855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.329345+0000","last_change":"2023-08-27T11:26:55.581127+0000","last_active":"2023-08-27T11:26:57.329345+0000","last_peered":"2023-08-27T11:26:57.329345+0000","last_clean":"2023-08-27T11:26:57.329345+0000","last_became_active":"2023-08-27T11:26:55.580654+0000","last_became_peered":"2023-08-27T11:26:55.580654+0000","last_unstale":"2023-08-27T11:26:57.329345+0000","last_undegraded":"2023-08-27T11:26:57.329345+0000","last_fullsized":"2023-08-27T11:26:57.329345+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:13:09.993091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.336412+0000","last_change":"2023-08-27T11:26:56.333736+0000","last_active":"2023-08-27T11:26:56.336412+0000","last_peered":"2023-08-27T11:26:56.336412+0000","last_clean":"2023-08-27T11:26:56.336412+0000","last_became_active":"2023-08-27T11:26:56.333480+0000","last_became_peered":"2023-08-27T11:26:56.333480+0000","last_unstale":"2023-08-27T11:26:56.336412+0000","last_undegraded":"2023-08-27T11:26:56.336412+0000","last_fullsized":"2023-08-27T11:26:56.336412+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:35:41.830099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.323047+0000","last_change":"2023-08-27T11:26:55.582915+0000","last_active":"2023-08-27T11:26:57.323047+0000","last_peered":"2023-08-27T11:26:57.323047+0000","last_clean":"2023-08-27T11:26:57.323047+0000","last_became_active":"2023-08-27T11:26:55.582556+0000","last_became_peered":"2023-08-27T11:26:55.582556+0000","last_unstale":"2023-08-27T11:26:57.323047+0000","last_undegraded":"2023-08-27T11:26:57.323047+0000","last_fullsized":"2023-08-27T11:26:57.323047+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:59:37.302724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322566+0000","last_change":"2023-08-27T11:26:56.339670+0000","last_active":"2023-08-27T11:26:57.322566+0000","last_peered":"2023-08-27T11:26:57.322566+0000","last_clean":"2023-08-27T11:26:57.322566+0000","last_became_active":"2023-08-27T11:26:56.339421+0000","last_became_peered":"2023-08-27T11:26:56.339421+0000","last_unstale":"2023-08-27T11:26:57.322566+0000","last_undegraded":"2023-08-27T11:26:57.322566+0000","last_fullsized":"2023-08-27T11:26:57.322566+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:55:23.934383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327626+0000","last_change":"2023-08-27T11:26:55.581587+0000","last_active":"2023-08-27T11:26:57.327626+0000","last_peered":"2023-08-27T11:26:57.327626+0000","last_clean":"2023-08-27T11:26:57.327626+0000","last_became_active":"2023-08-27T11:26:55.581259+0000","last_became_peered":"2023-08-27T11:26:55.581259+0000","last_unstale":"2023-08-27T11:26:57.327626+0000","last_undegraded":"2023-08-27T11:26:57.327626+0000","last_fullsized":"2023-08-27T11:26:57.327626+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:44:41.838536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.334100+0000","last_change":"2023-08-27T11:26:56.331407+0000","last_active":"2023-08-27T11:26:56.334100+0000","last_peered":"2023-08-27T11:26:56.334100+0000","last_clean":"2023-08-27T11:26:56.334100+0000","last_became_active":"2023-08-27T11:26:56.331157+0000","last_became_peered":"2023-08-27T11:26:56.331157+0000","last_unstale":"2023-08-27T11:26:56.334100+0000","last_undegraded":"2023-08-27T11:26:56.334100+0000","last_fullsized":"2023-08-27T11:26:56.334100+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:05.072625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327650+0000","last_change":"2023-08-27T11:26:55.584486+0000","last_active":"2023-08-27T11:26:57.327650+0000","last_peered":"2023-08-27T11:26:57.327650+0000","last_clean":"2023-08-27T11:26:57.327650+0000","last_became_active":"2023-08-27T11:26:55.584146+0000","last_became_peered":"2023-08-27T11:26:55.584146+0000","last_unstale":"2023-08-27T11:26:57.327650+0000","last_undegraded":"2023-08-27T11:26:57.327650+0000","last_fullsized":"2023-08-27T11:26:57.327650+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:45:31.804998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339847+0000","last_change":"2023-08-27T11:26:56.337314+0000","last_active":"2023-08-27T11:26:56.339847+0000","last_peered":"2023-08-27T11:26:56.339847+0000","last_clean":"2023-08-27T11:26:56.339847+0000","last_became_active":"2023-08-27T11:26:56.336826+0000","last_became_peered":"2023-08-27T11:26:56.336826+0000","last_unstale":"2023-08-27T11:26:56.339847+0000","last_undegraded":"2023-08-27T11:26:56.339847+0000","last_fullsized":"2023-08-27T11:26:56.339847+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:01:35.049959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322467+0000","last_change":"2023-08-27T11:26:55.580984+0000","last_active":"2023-08-27T11:26:57.322467+0000","last_peered":"2023-08-27T11:26:57.322467+0000","last_clean":"2023-08-27T11:26:57.322467+0000","last_became_active":"2023-08-27T11:26:55.580719+0000","last_became_peered":"2023-08-27T11:26:55.580719+0000","last_unstale":"2023-08-27T11:26:57.322467+0000","last_undegraded":"2023-08-27T11:26:57.322467+0000","last_fullsized":"2023-08-27T11:26:57.322467+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:17:23.104816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.337026+0000","last_change":"2023-08-27T11:26:56.333780+0000","last_active":"2023-08-27T11:26:56.337026+0000","last_peered":"2023-08-27T11:26:56.337026+0000","last_clean":"2023-08-27T11:26:56.337026+0000","last_became_active":"2023-08-27T11:26:56.333181+0000","last_became_peered":"2023-08-27T11:26:56.333181+0000","last_unstale":"2023-08-27T11:26:56.337026+0000","last_undegraded":"2023-08-27T11:26:56.337026+0000","last_fullsized":"2023-08-27T11:26:56.337026+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:49.527060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.325139+0000","last_change":"2023-08-27T11:26:55.586684+0000","last_active":"2023-08-27T11:26:56.325139+0000","last_peered":"2023-08-27T11:26:56.325139+0000","last_clean":"2023-08-27T11:26:56.325139+0000","last_became_active":"2023-08-27T11:26:55.586295+0000","last_became_peered":"2023-08-27T11:26:55.586295+0000","last_unstale":"2023-08-27T11:26:56.325139+0000","last_undegraded":"2023-08-27T11:26:56.325139+0000","last_fullsized":"2023-08-27T11:26:56.325139+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:44:01.667490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328578+0000","last_change":"2023-08-27T11:26:56.338204+0000","last_active":"2023-08-27T11:26:57.328578+0000","last_peered":"2023-08-27T11:26:57.328578+0000","last_clean":"2023-08-27T11:26:57.328578+0000","last_became_active":"2023-08-27T11:26:56.334760+0000","last_became_peered":"2023-08-27T11:26:56.334760+0000","last_unstale":"2023-08-27T11:26:57.328578+0000","last_undegraded":"2023-08-27T11:26:57.328578+0000","last_fullsized":"2023-08-27T11:26:57.328578+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:42:52.388238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328911+0000","last_change":"2023-08-27T11:26:55.580094+0000","last_active":"2023-08-27T11:26:57.328911+0000","last_peered":"2023-08-27T11:26:57.328911+0000","last_clean":"2023-08-27T11:26:57.328911+0000","last_became_active":"2023-08-27T11:26:55.579780+0000","last_became_peered":"2023-08-27T11:26:55.579780+0000","last_unstale":"2023-08-27T11:26:57.328911+0000","last_undegraded":"2023-08-27T11:26:57.328911+0000","last_fullsized":"2023-08-27T11:26:57.328911+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:30:04.134051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":187,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328915+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:26:57.328915+0000","last_peered":"2023-08-27T11:26:57.328915+0000","last_clean":"2023-08-27T11:26:57.328915+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:26:57.328915+0000","last_undegraded":"2023-08-27T11:26:57.328915+0000","last_fullsized":"2023-08-27T11:26:57.328915+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:56:45.554523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.342096+0000","last_change":"2023-08-27T11:26:56.333603+0000","last_active":"2023-08-27T11:26:56.342096+0000","last_peered":"2023-08-27T11:26:56.342096+0000","last_clean":"2023-08-27T11:26:56.342096+0000","last_became_active":"2023-08-27T11:26:56.333297+0000","last_became_peered":"2023-08-27T11:26:56.333297+0000","last_unstale":"2023-08-27T11:26:56.342096+0000","last_undegraded":"2023-08-27T11:26:56.342096+0000","last_fullsized":"2023-08-27T11:26:56.342096+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:27.152327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.329896+0000","last_change":"2023-08-27T11:26:55.583739+0000","last_active":"2023-08-27T11:26:56.329896+0000","last_peered":"2023-08-27T11:26:56.329896+0000","last_clean":"2023-08-27T11:26:56.329896+0000","last_became_active":"2023-08-27T11:26:55.583473+0000","last_became_peered":"2023-08-27T11:26:55.583473+0000","last_unstale":"2023-08-27T11:26:56.329896+0000","last_undegraded":"2023-08-27T11:26:56.329896+0000","last_fullsized":"2023-08-27T11:26:56.329896+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:06:25.014565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":187,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328730+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:26:57.328730+0000","last_peered":"2023-08-27T11:26:57.328730+0000","last_clean":"2023-08-27T11:26:57.328730+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:26:57.328730+0000","last_undegraded":"2023-08-27T11:26:57.328730+0000","last_fullsized":"2023-08-27T11:26:57.328730+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:27:20.122317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.343563+0000","last_change":"2023-08-27T11:26:56.341510+0000","last_active":"2023-08-27T11:26:56.343563+0000","last_peered":"2023-08-27T11:26:56.343563+0000","last_clean":"2023-08-27T11:26:56.343563+0000","last_became_active":"2023-08-27T11:26:56.341204+0000","last_became_peered":"2023-08-27T11:26:56.341204+0000","last_unstale":"2023-08-27T11:26:56.343563+0000","last_undegraded":"2023-08-27T11:26:56.343563+0000","last_fullsized":"2023-08-27T11:26:56.343563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:17:23.703806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.321238+0000","last_change":"2023-08-27T11:26:55.581210+0000","last_active":"2023-08-27T11:26:56.321238+0000","last_peered":"2023-08-27T11:26:56.321238+0000","last_clean":"2023-08-27T11:26:56.321238+0000","last_became_active":"2023-08-27T11:26:55.580944+0000","last_became_peered":"2023-08-27T11:26:55.580944+0000","last_unstale":"2023-08-27T11:26:56.321238+0000","last_undegraded":"2023-08-27T11:26:56.321238+0000","last_fullsized":"2023-08-27T11:26:56.321238+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:00:26.365353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":159,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.331068+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:26:56.331068+0000","last_peered":"2023-08-27T11:26:56.331068+0000","last_clean":"2023-08-27T11:26:56.331068+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:26:56.331068+0000","last_undegraded":"2023-08-27T11:26:56.331068+0000","last_fullsized":"2023-08-27T11:26:56.331068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:42:51.656620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339985+0000","last_change":"2023-08-27T11:26:56.333698+0000","last_active":"2023-08-27T11:26:56.339985+0000","last_peered":"2023-08-27T11:26:56.339985+0000","last_clean":"2023-08-27T11:26:56.339985+0000","last_became_active":"2023-08-27T11:26:56.333431+0000","last_became_peered":"2023-08-27T11:26:56.333431+0000","last_unstale":"2023-08-27T11:26:56.339985+0000","last_undegraded":"2023-08-27T11:26:56.339985+0000","last_fullsized":"2023-08-27T11:26:56.339985+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:32:09.535815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"101'8","reported_seq":19,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.582937+0000","last_change":"2023-08-27T11:26:55.585886+0000","last_active":"2023-08-27T11:26:56.582937+0000","last_peered":"2023-08-27T11:26:56.582937+0000","last_clean":"2023-08-27T11:26:56.582937+0000","last_became_active":"2023-08-27T11:26:55.585639+0000","last_became_peered":"2023-08-27T11:26:55.585639+0000","last_unstale":"2023-08-27T11:26:56.582937+0000","last_undegraded":"2023-08-27T11:26:56.582937+0000","last_fullsized":"2023-08-27T11:26:56.582937+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:53.593374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":157,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.325414+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:26:56.325414+0000","last_peered":"2023-08-27T11:26:56.325414+0000","last_clean":"2023-08-27T11:26:56.325414+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:26:56.325414+0000","last_undegraded":"2023-08-27T11:26:56.325414+0000","last_fullsized":"2023-08-27T11:26:56.325414+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:10:24.148761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.336425+0000","last_change":"2023-08-27T11:26:56.332815+0000","last_active":"2023-08-27T11:26:56.336425+0000","last_peered":"2023-08-27T11:26:56.336425+0000","last_clean":"2023-08-27T11:26:56.336425+0000","last_became_active":"2023-08-27T11:26:56.332539+0000","last_became_peered":"2023-08-27T11:26:56.332539+0000","last_unstale":"2023-08-27T11:26:56.336425+0000","last_undegraded":"2023-08-27T11:26:56.336425+0000","last_fullsized":"2023-08-27T11:26:56.336425+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:13:11.808107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.322917+0000","last_change":"2023-08-27T11:26:55.581749+0000","last_active":"2023-08-27T11:26:56.322917+0000","last_peered":"2023-08-27T11:26:56.322917+0000","last_clean":"2023-08-27T11:26:56.322917+0000","last_became_active":"2023-08-27T11:26:55.581397+0000","last_became_peered":"2023-08-27T11:26:55.581397+0000","last_unstale":"2023-08-27T11:26:56.322917+0000","last_undegraded":"2023-08-27T11:26:56.322917+0000","last_fullsized":"2023-08-27T11:26:56.322917+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:25.083257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":157,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.325772+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:26:56.325772+0000","last_peered":"2023-08-27T11:26:56.325772+0000","last_clean":"2023-08-27T11:26:56.325772+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:26:56.325772+0000","last_undegraded":"2023-08-27T11:26:56.325772+0000","last_fullsized":"2023-08-27T11:26:56.325772+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:25:18.614507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328665+0000","last_change":"2023-08-27T11:26:56.329794+0000","last_active":"2023-08-27T11:26:57.328665+0000","last_peered":"2023-08-27T11:26:57.328665+0000","last_clean":"2023-08-27T11:26:57.328665+0000","last_became_active":"2023-08-27T11:26:56.329504+0000","last_became_peered":"2023-08-27T11:26:56.329504+0000","last_unstale":"2023-08-27T11:26:57.328665+0000","last_undegraded":"2023-08-27T11:26:57.328665+0000","last_fullsized":"2023-08-27T11:26:57.328665+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:39.459063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"101'2","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.580085+0000","last_change":"2023-08-27T11:26:55.584519+0000","last_active":"2023-08-27T11:26:56.580085+0000","last_peered":"2023-08-27T11:26:56.580085+0000","last_clean":"2023-08-27T11:26:56.580085+0000","last_became_active":"2023-08-27T11:26:55.584238+0000","last_became_peered":"2023-08-27T11:26:55.584238+0000","last_unstale":"2023-08-27T11:26:56.580085+0000","last_undegraded":"2023-08-27T11:26:56.580085+0000","last_fullsized":"2023-08-27T11:26:56.580085+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:42.698705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":307,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.325269+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:26:56.325269+0000","last_peered":"2023-08-27T11:26:56.325269+0000","last_clean":"2023-08-27T11:26:56.325269+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:26:56.325269+0000","last_undegraded":"2023-08-27T11:26:56.325269+0000","last_fullsized":"2023-08-27T11:26:56.325269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:49:26.983333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":163,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.327342+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:26:56.327342+0000","last_peered":"2023-08-27T11:26:56.327342+0000","last_clean":"2023-08-27T11:26:56.327342+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:26:56.327342+0000","last_undegraded":"2023-08-27T11:26:56.327342+0000","last_fullsized":"2023-08-27T11:26:56.327342+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:38:30.057816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344527+0000","last_change":"2023-08-27T11:26:56.337315+0000","last_active":"2023-08-27T11:26:56.344527+0000","last_peered":"2023-08-27T11:26:56.344527+0000","last_clean":"2023-08-27T11:26:56.344527+0000","last_became_active":"2023-08-27T11:26:56.337056+0000","last_became_peered":"2023-08-27T11:26:56.337056+0000","last_unstale":"2023-08-27T11:26:56.344527+0000","last_undegraded":"2023-08-27T11:26:56.344527+0000","last_fullsized":"2023-08-27T11:26:56.344527+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:28:39.173763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"101'6","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.582876+0000","last_change":"2023-08-27T11:26:55.581149+0000","last_active":"2023-08-27T11:26:56.582876+0000","last_peered":"2023-08-27T11:26:56.582876+0000","last_clean":"2023-08-27T11:26:56.582876+0000","last_became_active":"2023-08-27T11:26:55.580849+0000","last_became_peered":"2023-08-27T11:26:55.580849+0000","last_unstale":"2023-08-27T11:26:56.582876+0000","last_undegraded":"2023-08-27T11:26:56.582876+0000","last_fullsized":"2023-08-27T11:26:56.582876+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:24:35.451519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":155,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.321613+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:26:56.321613+0000","last_peered":"2023-08-27T11:26:56.321613+0000","last_clean":"2023-08-27T11:26:56.321613+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:26:56.321613+0000","last_undegraded":"2023-08-27T11:26:56.321613+0000","last_fullsized":"2023-08-27T11:26:56.321613+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:35.245524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.333886+0000","last_change":"2023-08-27T11:26:56.330525+0000","last_active":"2023-08-27T11:26:56.333886+0000","last_peered":"2023-08-27T11:26:56.333886+0000","last_clean":"2023-08-27T11:26:56.333886+0000","last_became_active":"2023-08-27T11:26:56.330249+0000","last_became_peered":"2023-08-27T11:26:56.330249+0000","last_unstale":"2023-08-27T11:26:56.333886+0000","last_undegraded":"2023-08-27T11:26:56.333886+0000","last_fullsized":"2023-08-27T11:26:56.333886+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:04:48.846519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"101'3","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.578568+0000","last_change":"2023-08-27T11:26:55.586242+0000","last_active":"2023-08-27T11:26:56.578568+0000","last_peered":"2023-08-27T11:26:56.578568+0000","last_clean":"2023-08-27T11:26:56.578568+0000","last_became_active":"2023-08-27T11:26:55.585929+0000","last_became_peered":"2023-08-27T11:26:55.585929+0000","last_unstale":"2023-08-27T11:26:56.578568+0000","last_undegraded":"2023-08-27T11:26:56.578568+0000","last_fullsized":"2023-08-27T11:26:56.578568+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:09:48.090341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":177,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327920+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:26:57.327920+0000","last_peered":"2023-08-27T11:26:57.327920+0000","last_clean":"2023-08-27T11:26:57.327920+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:26:57.327920+0000","last_undegraded":"2023-08-27T11:26:57.327920+0000","last_fullsized":"2023-08-27T11:26:57.327920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:12:17.048490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574972,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1867776,"data_stored":1170032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.39400000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.55000000000000004,"15min":0.50800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":0.77500000000000002,"15min":0.872},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.253},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.53900000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.48699999999999999},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.33800000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.90200000000000002},"last":0.60999999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.57499999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.505},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.72699999999999998,"5min":0.72799999999999998,"15min":0.77400000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.41399999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.44}]},{"osd":6,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.55000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1261568,"data_stored":579470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51500000000000001,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.29099999999999998},"max":{"1min":0.626,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67300000000000004,"5min":0.86199999999999999,"15min":0.89800000000000002},"last":0.48099999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.316,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.80600000000000005,"15min":0.873},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.62,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.67700000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58299999999999996,"5min":0.74199999999999999,"15min":0.85899999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.65500000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.58199999999999996,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.70699999999999996,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.65200000000000002,"5min":0.84699999999999998,"15min":0.93799999999999994},"last":0.621}]},{"osd":4,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.70699999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.374},"max":{"1min":0.77800000000000002,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.76400000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.624,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.78100000000000003,"5min":0.875,"15min":0.875},"last":0.748}]},{"osd":7,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.748,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.34899999999999998},"max":{"1min":0.76700000000000002,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.82999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1855488,"data_stored":1169278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.308,"15min":0.59299999999999997},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.45800000000000002,"5min":0.502,"15min":11.971},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.89000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.51700000000000002,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.27300000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.625},"min":{"1min":0.23400000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.495,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.34300000000000003},"min":{"1min":0.246,"5min":0.224,"15min":0.224},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.58599999999999997},"last":0.317}]},{"osd":3,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.64100000000000001},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":10.99},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.373,"15min":0.92500000000000004},"min":{"1min":0.318,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.54700000000000004,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.38500000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.36099999999999999,"15min":0.33400000000000002},"max":{"1min":0.76900000000000002,"5min":0.90700000000000003,"15min":1.26},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.80700000000000005},"min":{"1min":0.434,"5min":0.37,"15min":0.29099999999999998},"max":{"1min":0.64500000000000002,"5min":0.77600000000000002,"15min":10.957000000000001},"last":0.50600000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":1.121},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.36199999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71499999999999997,"5min":0.88700000000000001,"15min":1.296},"last":0.60399999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.84299999999999997},"min":{"1min":0.44800000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":1.1379999999999999},"min":{"1min":0.435,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.74299999999999999,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.497}]},{"osd":7,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":1.1319999999999999},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84199999999999997,"15min":22.988},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.63500000000000001},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":1.327},"last":0.51600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574972,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":577549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.29699999999999999,"15min":0.311},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.33200000000000002,"15min":0.34999999999999998},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.42699999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.24399999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.628,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.33200000000000002,"15min":0.33900000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.35599999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.33700000000000002,"15min":0.34599999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.502,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.27500000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.54200000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.40100000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79100000000000004,"5min":0.92500000000000004,"15min":1.214},"last":0.628},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.56999999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.77500000000000002,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85399999999999998,"5min":0.90200000000000002,"15min":1.603},"last":0.65700000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.621},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.66800000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.74099999999999999,"5min":0.94199999999999995,"15min":1.383},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82699999999999996,"5min":0.88900000000000001,"15min":0.92000000000000004},"last":0.67900000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":578888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29799999999999999,"15min":0.30199999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40500000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.314,"5min":0.316,"15min":0.317},"min":{"1min":0.22700000000000001,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.629},"last":0.32200000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.317,"15min":0.32400000000000001},"min":{"1min":0.222,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.36499999999999999,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33800000000000002,"15min":0.34599999999999997},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47799999999999998,"5min":0.64200000000000002,"15min":0.749},"last":0.439}]},{"osd":3,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36099999999999999,"15min":0.38200000000000001},"min":{"1min":0.318,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41599999999999998,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.376,"15min":0.39800000000000002},"min":{"1min":0.30599999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.69499999999999995},"last":0.40999999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.68600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.433,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.76100000000000001,"5min":0.78400000000000003,"15min":0.879},"last":0.48499999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70699999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.40999999999999998,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.73399999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53000000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77200000000000002,"5min":0.82899999999999996,"15min":0.85299999999999998},"last":0.505},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.59399999999999997},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.65900000000000003,"5min":0.80500000000000005,"15min":0.83599999999999997},"last":0.59099999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.60899999999999999},"min":{"1min":0.438,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.626}]}]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1273856,"data_stored":580534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29699999999999999,"15min":0.29999999999999999},"min":{"1min":0.21299999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32100000000000001,"15min":0.312},"min":{"1min":0.23100000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.38400000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.33400000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32300000000000001,"15min":0.313},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.19500000000000001},"max":{"1min":0.46700000000000003,"5min":0.50600000000000001,"15min":0.60399999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":0.44900000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"last":0.41799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.28499999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.53300000000000003,"5min":0.57199999999999995,"15min":0.57599999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.52500000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.499}]},{"osd":4,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.28599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.78600000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.371,"15min":0.314},"max":{"1min":0.79100000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.83799999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.373},"max":{"1min":0.80400000000000005,"5min":1.091,"15min":1.091},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.61199999999999999},"min":{"1min":0.41699999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78400000000000003,"5min":1.111,"15min":1.111},"last":0.71399999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.61199999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74399999999999999,"5min":1.032,"15min":1.236},"last":0.748},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.61899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":580500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.755,"5min":0.997,"15min":0.997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.68700000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.752,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55600000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.51000000000000001},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.44800000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40699999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.85899999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75700000000000001,"5min":0.85999999999999999,"15min":0.879},"last":0.66100000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70599999999999996,"5min":0.78900000000000003,"15min":0.90600000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59599999999999997,"15min":0.623},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.92300000000000004},"last":0.67900000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.80600000000000005,"15min":0.82499999999999996},"last":0.622},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.89300000000000002},"last":0.63400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574972,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1265664,"data_stored":579048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.69699999999999995,"5min":0.78600000000000003,"15min":0.96899999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.64500000000000002,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.56799999999999995}]},{"osd":1,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.54200000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66700000000000004,"5min":0.72899999999999998,"15min":1.075},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.54800000000000004},"min":{"1min":0.36399999999999999,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.68899999999999995,"5min":0.78000000000000003,"15min":0.95699999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54100000000000004,"15min":0.55700000000000005},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.67600000000000005,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.55500000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.56299999999999994,"15min":0.58099999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73399999999999999,"5min":0.81000000000000005,"15min":1.03},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.35199999999999998},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":1.026},"last":0.443}]},{"osd":4,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.31,"15min":0.27000000000000002},"max":{"1min":0.72799999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.73399999999999999,"5min":0.86399999999999999,"15min":0.91400000000000003},"last":0.51400000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59199999999999997,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.28799999999999998},"max":{"1min":0.77700000000000002,"5min":0.84799999999999998,"15min":0.94199999999999995},"last":0.503}]},{"osd":7,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.61499999999999999,"15min":0.629},"min":{"1min":0.432,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.63900000000000001,"5min":0.90700000000000003,"15min":0.95899999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.31},"max":{"1min":0.69399999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.48699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:27:00.685 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:27:00.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.836+0000 7f89a8aec700 1 Processor -- start 2023-08-27T11:27:00.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.836+0000 7f89a8aec700 1 -- start start 2023-08-27T11:27:00.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.837+0000 7f89a8aec700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a40b17e0 0x7f89a40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.837+0000 7f89a8aec700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f89a4161ce0 0x7f89a406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.837+0000 7f89a8aec700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a4161850 con 0x7f89a40b1400 2023-08-27T11:27:00.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.837+0000 7f89a8aec700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a40cfc10 con 0x7f89a4161ce0 2023-08-27T11:27:00.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.837+0000 7f89a8aec700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a4161200 con 0x7f89a40b17e0 2023-08-27T11:27:00.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a259c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f89a40b1400 0x7f89a406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56630/0 (socket says 172.21.15.112:56630) 2023-08-27T11:27:00.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a259c700 1 -- 172.21.15.112:0/2884788894 learned_addr learned my addr 172.21.15.112:0/2884788894 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:00.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a2d9d700 1 --2- 172.21.15.112:0/2884788894 >> v2:172.21.15.138:3300/0 conn(0x7f89a4161ce0 0x7f89a406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a1d9b700 1 --2- 172.21.15.112:0/2884788894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a40b17e0 0x7f89a40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916733314 0 0) 0x7f89a4161850 con 0x7f89a40b1400 2023-08-27T11:27:00.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f898c0035b0 con 0x7f89a40b1400 2023-08-27T11:27:00.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2962028206 0 0) 0x7f898c0035b0 con 0x7f89a40b1400 2023-08-27T11:27:00.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a40b17e0 msgr2=0x7f89a40cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:00.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 --2- 172.21.15.112:0/2884788894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a40b17e0 0x7f89a40cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 >> v2:172.21.15.138:3300/0 conn(0x7f89a4161ce0 msgr2=0x7f89a406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 --2- 172.21.15.112:0/2884788894 >> v2:172.21.15.138:3300/0 conn(0x7f89a4161ce0 0x7f89a406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a41614e0 con 0x7f89a40b1400 2023-08-27T11:27:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f8998001670 con 0x7f89a40b1400 2023-08-27T11:27:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a1d9b700 1 --2- 172.21.15.112:0/2884788894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a40b17e0 0x7f89a40cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a2d9d700 1 --2- 172.21.15.112:0/2884788894 >> v2:172.21.15.138:3300/0 conn(0x7f89a4161ce0 0x7f89a406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:00.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 >> v1:172.21.15.112:6789/0 conn(0x7f89a40b1400 legacy=0x7f89a406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 --2- 172.21.15.112:0/2884788894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f898c003a00 0x7f898c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 --2- 172.21.15.112:0/2884788894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a40b17e0 0x7f898c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 --2- 172.21.15.112:0/2884788894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f898c004d90 0x7f898c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40cfc10 con 0x7f898c003a00 2023-08-27T11:27:00.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a4161200 con 0x7f898c004d90 2023-08-27T11:27:00.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c00ba10 con 0x7f89a40b17e0 2023-08-27T11:27:00.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a259c700 1 --2- 172.21.15.112:0/2884788894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f898c003a00 0x7f898c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a1d9b700 1 --2- 172.21.15.112:0/2884788894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a40b17e0 0x7f898c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.838+0000 7f89a2d9d700 1 --2- 172.21.15.112:0/2884788894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f898c004d90 0x7f898c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a259c700 1 -- 172.21.15.112:0/2884788894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a40b17e0 msgr2=0x7f898c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:00.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a259c700 1 --2- 172.21.15.112:0/2884788894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a40b17e0 0x7f898c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a259c700 1 -- 172.21.15.112:0/2884788894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f898c004d90 msgr2=0x7f898c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:00.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a259c700 1 --2- 172.21.15.112:0/2884788894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f898c004d90 0x7f898c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a259c700 1 -- 172.21.15.112:0/2884788894 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89940094a0 con 0x7f898c003a00 2023-08-27T11:27:00.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a259c700 1 --2- 172.21.15.112:0/2884788894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f898c003a00 0x7f898c003e20 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f899800c320 tx=0x7f898c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7495e73168a4e8a4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:00.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899800ce00 con 0x7f898c003a00 2023-08-27T11:27:00.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8998005960 con 0x7f898c003a00 2023-08-27T11:27:00.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a159a700 1 -- 172.21.15.112:0/2884788894 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998005d10 con 0x7f898c003a00 2023-08-27T11:27:00.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a8aec700 1 -- 172.21.15.112:0/2884788894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f898c003a00 msgr2=0x7f898c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a8aec700 1 --2- 172.21.15.112:0/2884788894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f898c003a00 0x7f898c003e20 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f899800c320 tx=0x7f898c005880 comp rx=0 tx=0).stop 2023-08-27T11:27:00.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a2d9d700 1 -- 172.21.15.112:0/2884788894 reap_dead start 2023-08-27T11:27:00.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a8aec700 1 -- 172.21.15.112:0/2884788894 shutdown_connections 2023-08-27T11:27:00.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a8aec700 1 -- 172.21.15.112:0/2884788894 >> 172.21.15.112:0/2884788894 conn(0x7f89a40d0000 msgr2=0x7f89a40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:00.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a8aec700 1 -- 172.21.15.112:0/2884788894 shutdown_connections 2023-08-27T11:27:00.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.839+0000 7f89a8aec700 1 -- 172.21.15.112:0/2884788894 wait complete. 2023-08-27T11:27:00.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.841+0000 7f89a8aec700 1 Processor -- start 2023-08-27T11:27:00.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.841+0000 7f89a8aec700 1 -- start start 2023-08-27T11:27:00.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.842+0000 7f89a8aec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f898c003a00 0x7f89a414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.842+0000 7f89a8aec700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a414c400 0x7f89a414d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.842+0000 7f89a8aec700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89a414fb80 0x7f89a4150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.842+0000 7f89a8aec700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a414db70 con 0x7f898c003a00 2023-08-27T11:27:00.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.842+0000 7f89a8aec700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a414ddc0 con 0x7f89a414fb80 2023-08-27T11:27:00.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.842+0000 7f89a8aec700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a414e0a0 con 0x7f89a414c400 2023-08-27T11:27:00.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.842+0000 7f89a259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f898c003a00 0x7f89a414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.842+0000 7f89a259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f898c003a00 0x7f89a414bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34030/0 (socket says 172.21.15.112:34030) 2023-08-27T11:27:00.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.842+0000 7f89a259c700 1 -- 172.21.15.112:0/2535061969 learned_addr learned my addr 172.21.15.112:0/2535061969 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:00.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.842+0000 7f89a1d9b700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a414c400 0x7f89a414d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.842+0000 7f89a2d9d700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89a414fb80 0x7f89a4150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.843+0000 7f89a259c700 1 -- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a414c400 msgr2=0x7f89a414d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.843+0000 7f89a259c700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a414c400 0x7f89a414d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.843+0000 7f89a259c700 1 -- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89a414fb80 msgr2=0x7f89a4150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:00.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.843+0000 7f89a259c700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89a414fb80 0x7f89a4150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:00.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.843+0000 7f89a259c700 1 -- 172.21.15.112:0/2535061969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a414e380 con 0x7f898c003a00 2023-08-27T11:27:00.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.843+0000 7f89a259c700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f898c003a00 0x7f89a414bcc0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8998001930 tx=0x7f8998001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:00.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.843+0000 7f8992ffd700 1 -- 172.21.15.112:0/2535061969 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899800ce70 con 0x7f898c003a00 2023-08-27T11:27:00.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.843+0000 7f8992ffd700 1 -- 172.21.15.112:0/2535061969 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8998004a80 con 0x7f898c003a00 2023-08-27T11:27:00.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.843+0000 7f89a8aec700 1 -- 172.21.15.112:0/2535061969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a40708d0 con 0x7f898c003a00 2023-08-27T11:27:00.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.843+0000 7f8992ffd700 1 -- 172.21.15.112:0/2535061969 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998004e30 con 0x7f898c003a00 2023-08-27T11:27:00.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.843+0000 7f89a8aec700 1 -- 172.21.15.112:0/2535061969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a414d7a0 con 0x7f898c003a00 2023-08-27T11:27:00.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.844+0000 7f89a8aec700 1 -- 172.21.15.112:0/2535061969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a4067a20 con 0x7f898c003a00 2023-08-27T11:27:00.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.844+0000 7f8992ffd700 1 -- 172.21.15.112:0/2535061969 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f89980070d0 con 0x7f898c003a00 2023-08-27T11:27:00.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.845+0000 7f8992ffd700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f898406a440 0x7f898406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:00.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.845+0000 7f8992ffd700 1 -- 172.21.15.112:0/2535061969 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8998089ef0 con 0x7f898c003a00 2023-08-27T11:27:00.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.845+0000 7f89a1d9b700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f898406a440 0x7f898406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:00.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.845+0000 7f89a1d9b700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f898406a440 0x7f898406c900 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f89a414c800 tx=0x7f8994008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:00.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:00.849+0000 7f8992ffd700 1 -- 172.21.15.112:0/2535061969 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8998055440 con 0x7f898c003a00 2023-08-27T11:27:01.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.034+0000 7f89a8aec700 1 -- 172.21.15.112:0/2535061969 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89a414db70 con 0x7f898406a440 2023-08-27T11:27:01.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.043+0000 7f8992ffd700 1 -- 172.21.15.112:0/2535061969 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312311 (secure 0 0 0) 0x7f89a414db70 con 0x7f898406a440 2023-08-27T11:27:01.048 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:01.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.046+0000 7f89a8aec700 1 -- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f898406a440 msgr2=0x7f898406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.046+0000 7f89a8aec700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f898406a440 0x7f898406c900 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f89a414c800 tx=0x7f8994008040 comp rx=0 tx=0).stop 2023-08-27T11:27:01.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.046+0000 7f89a8aec700 1 -- 172.21.15.112:0/2535061969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f898c003a00 msgr2=0x7f89a414bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.046+0000 7f89a8aec700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f898c003a00 0x7f89a414bcc0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8998001930 tx=0x7f8998001a00 comp rx=0 tx=0).stop 2023-08-27T11:27:01.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.046+0000 7f89a8aec700 1 -- 172.21.15.112:0/2535061969 shutdown_connections 2023-08-27T11:27:01.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.046+0000 7f89a8aec700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f898406a440 0x7f898406c900 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.046+0000 7f89a8aec700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89a414fb80 0x7f89a4150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.046+0000 7f89a8aec700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f898c003a00 0x7f89a414bcc0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.046+0000 7f89a8aec700 1 --2- 172.21.15.112:0/2535061969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89a414c400 0x7f89a414d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.046+0000 7f89a8aec700 1 -- 172.21.15.112:0/2535061969 >> 172.21.15.112:0/2535061969 conn(0x7f89a40d0000 msgr2=0x7f89a415fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.046+0000 7f89a8aec700 1 -- 172.21.15.112:0/2535061969 shutdown_connections 2023-08-27T11:27:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.046+0000 7f89a8aec700 1 -- 172.21.15.112:0/2535061969 wait complete. 2023-08-27T11:27:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:27:01.060 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2023-08-27T11:27:00.378056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260548,"kb_used_data":11044,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754714172,"statfs":{"total":773094113280,"available":772827312128,"internally_reserved":0,"allocated":11309056,"data_stored":5815299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.815594"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327969+0000","last_change":"2023-08-27T11:26:56.338200+0000","last_active":"2023-08-27T11:26:57.327969+0000","last_peered":"2023-08-27T11:26:57.327969+0000","last_clean":"2023-08-27T11:26:57.327969+0000","last_became_active":"2023-08-27T11:26:56.337893+0000","last_became_peered":"2023-08-27T11:26:56.337893+0000","last_unstale":"2023-08-27T11:26:57.327969+0000","last_undegraded":"2023-08-27T11:26:57.327969+0000","last_fullsized":"2023-08-27T11:26:57.327969+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:35.647340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.334148+0000","last_change":"2023-08-27T11:26:56.332252+0000","last_active":"2023-08-27T11:26:56.334148+0000","last_peered":"2023-08-27T11:26:56.334148+0000","last_clean":"2023-08-27T11:26:56.334148+0000","last_became_active":"2023-08-27T11:26:56.332007+0000","last_became_peered":"2023-08-27T11:26:56.332007+0000","last_unstale":"2023-08-27T11:26:56.334148+0000","last_undegraded":"2023-08-27T11:26:56.334148+0000","last_fullsized":"2023-08-27T11:26:56.334148+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:29:20.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.334196+0000","last_change":"2023-08-27T11:26:56.331504+0000","last_active":"2023-08-27T11:26:56.334196+0000","last_peered":"2023-08-27T11:26:56.334196+0000","last_clean":"2023-08-27T11:26:56.334196+0000","last_became_active":"2023-08-27T11:26:56.331123+0000","last_became_peered":"2023-08-27T11:26:56.331123+0000","last_unstale":"2023-08-27T11:26:56.334196+0000","last_undegraded":"2023-08-27T11:26:56.334196+0000","last_fullsized":"2023-08-27T11:26:56.334196+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:23.782267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344302+0000","last_change":"2023-08-27T11:26:56.341402+0000","last_active":"2023-08-27T11:26:56.344302+0000","last_peered":"2023-08-27T11:26:56.344302+0000","last_clean":"2023-08-27T11:26:56.344302+0000","last_became_active":"2023-08-27T11:26:56.341052+0000","last_became_peered":"2023-08-27T11:26:56.341052+0000","last_unstale":"2023-08-27T11:26:56.344302+0000","last_undegraded":"2023-08-27T11:26:56.344302+0000","last_fullsized":"2023-08-27T11:26:56.344302+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:50:24.630824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327517+0000","last_change":"2023-08-27T11:26:56.338947+0000","last_active":"2023-08-27T11:26:57.327517+0000","last_peered":"2023-08-27T11:26:57.327517+0000","last_clean":"2023-08-27T11:26:57.327517+0000","last_became_active":"2023-08-27T11:26:56.338641+0000","last_became_peered":"2023-08-27T11:26:56.338641+0000","last_unstale":"2023-08-27T11:26:57.327517+0000","last_undegraded":"2023-08-27T11:26:57.327517+0000","last_fullsized":"2023-08-27T11:26:57.327517+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:42:52.388238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322543+0000","last_change":"2023-08-27T11:26:56.329620+0000","last_active":"2023-08-27T11:26:57.322543+0000","last_peered":"2023-08-27T11:26:57.322543+0000","last_clean":"2023-08-27T11:26:57.322543+0000","last_became_active":"2023-08-27T11:26:56.329378+0000","last_became_peered":"2023-08-27T11:26:56.329378+0000","last_unstale":"2023-08-27T11:26:57.322543+0000","last_undegraded":"2023-08-27T11:26:57.322543+0000","last_fullsized":"2023-08-27T11:26:57.322543+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:04:42.408211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322419+0000","last_change":"2023-08-27T11:26:56.333654+0000","last_active":"2023-08-27T11:26:57.322419+0000","last_peered":"2023-08-27T11:26:57.322419+0000","last_clean":"2023-08-27T11:26:57.322419+0000","last_became_active":"2023-08-27T11:26:56.333402+0000","last_became_peered":"2023-08-27T11:26:56.333402+0000","last_unstale":"2023-08-27T11:26:57.322419+0000","last_undegraded":"2023-08-27T11:26:57.322419+0000","last_fullsized":"2023-08-27T11:26:57.322419+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:20:28.589177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.332638+0000","last_change":"2023-08-27T11:26:56.330853+0000","last_active":"2023-08-27T11:26:56.332638+0000","last_peered":"2023-08-27T11:26:56.332638+0000","last_clean":"2023-08-27T11:26:56.332638+0000","last_became_active":"2023-08-27T11:26:56.330373+0000","last_became_peered":"2023-08-27T11:26:56.330373+0000","last_unstale":"2023-08-27T11:26:56.332638+0000","last_undegraded":"2023-08-27T11:26:56.332638+0000","last_fullsized":"2023-08-27T11:26:56.332638+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:04:49.296681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339620+0000","last_change":"2023-08-27T11:26:56.335039+0000","last_active":"2023-08-27T11:26:56.339620+0000","last_peered":"2023-08-27T11:26:56.339620+0000","last_clean":"2023-08-27T11:26:56.339620+0000","last_became_active":"2023-08-27T11:26:56.333034+0000","last_became_peered":"2023-08-27T11:26:56.333034+0000","last_unstale":"2023-08-27T11:26:56.339620+0000","last_undegraded":"2023-08-27T11:26:56.339620+0000","last_fullsized":"2023-08-27T11:26:56.339620+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:13:47.340239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.337065+0000","last_change":"2023-08-27T11:26:56.333879+0000","last_active":"2023-08-27T11:26:56.337065+0000","last_peered":"2023-08-27T11:26:56.337065+0000","last_clean":"2023-08-27T11:26:56.337065+0000","last_became_active":"2023-08-27T11:26:56.333613+0000","last_became_peered":"2023-08-27T11:26:56.333613+0000","last_unstale":"2023-08-27T11:26:56.337065+0000","last_undegraded":"2023-08-27T11:26:56.337065+0000","last_fullsized":"2023-08-27T11:26:56.337065+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:07.580879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327775+0000","last_change":"2023-08-27T11:26:56.344921+0000","last_active":"2023-08-27T11:26:57.327775+0000","last_peered":"2023-08-27T11:26:57.327775+0000","last_clean":"2023-08-27T11:26:57.327775+0000","last_became_active":"2023-08-27T11:26:56.344655+0000","last_became_peered":"2023-08-27T11:26:56.344655+0000","last_unstale":"2023-08-27T11:26:57.327775+0000","last_undegraded":"2023-08-27T11:26:57.327775+0000","last_fullsized":"2023-08-27T11:26:57.327775+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:30:05.139797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.323064+0000","last_change":"2023-08-27T11:26:56.339849+0000","last_active":"2023-08-27T11:26:57.323064+0000","last_peered":"2023-08-27T11:26:57.323064+0000","last_clean":"2023-08-27T11:26:57.323064+0000","last_became_active":"2023-08-27T11:26:56.339417+0000","last_became_peered":"2023-08-27T11:26:56.339417+0000","last_unstale":"2023-08-27T11:26:57.323064+0000","last_undegraded":"2023-08-27T11:26:57.323064+0000","last_fullsized":"2023-08-27T11:26:57.323064+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:29:20.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.337044+0000","last_change":"2023-08-27T11:26:56.333995+0000","last_active":"2023-08-27T11:26:56.337044+0000","last_peered":"2023-08-27T11:26:56.337044+0000","last_clean":"2023-08-27T11:26:56.337044+0000","last_became_active":"2023-08-27T11:26:56.333536+0000","last_became_peered":"2023-08-27T11:26:56.333536+0000","last_unstale":"2023-08-27T11:26:56.337044+0000","last_undegraded":"2023-08-27T11:26:56.337044+0000","last_fullsized":"2023-08-27T11:26:56.337044+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:07:54.390021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.334883+0000","last_change":"2023-08-27T11:26:56.332765+0000","last_active":"2023-08-27T11:26:56.334883+0000","last_peered":"2023-08-27T11:26:56.334883+0000","last_clean":"2023-08-27T11:26:56.334883+0000","last_became_active":"2023-08-27T11:26:56.332321+0000","last_became_peered":"2023-08-27T11:26:56.332321+0000","last_unstale":"2023-08-27T11:26:56.334883+0000","last_undegraded":"2023-08-27T11:26:56.334883+0000","last_fullsized":"2023-08-27T11:26:56.334883+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:22:40.892753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327270+0000","last_change":"2023-08-27T11:26:56.342939+0000","last_active":"2023-08-27T11:26:57.327270+0000","last_peered":"2023-08-27T11:26:57.327270+0000","last_clean":"2023-08-27T11:26:57.327270+0000","last_became_active":"2023-08-27T11:26:56.342711+0000","last_became_peered":"2023-08-27T11:26:56.342711+0000","last_unstale":"2023-08-27T11:26:57.327270+0000","last_undegraded":"2023-08-27T11:26:57.327270+0000","last_fullsized":"2023-08-27T11:26:57.327270+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:40:09.000394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322572+0000","last_change":"2023-08-27T11:26:56.330300+0000","last_active":"2023-08-27T11:26:57.322572+0000","last_peered":"2023-08-27T11:26:57.322572+0000","last_clean":"2023-08-27T11:26:57.322572+0000","last_became_active":"2023-08-27T11:26:56.330041+0000","last_became_peered":"2023-08-27T11:26:56.330041+0000","last_unstale":"2023-08-27T11:26:57.322572+0000","last_undegraded":"2023-08-27T11:26:57.322572+0000","last_fullsized":"2023-08-27T11:26:57.322572+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:46:17.505423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339730+0000","last_change":"2023-08-27T11:26:56.338258+0000","last_active":"2023-08-27T11:26:56.339730+0000","last_peered":"2023-08-27T11:26:56.339730+0000","last_clean":"2023-08-27T11:26:56.339730+0000","last_became_active":"2023-08-27T11:26:56.337945+0000","last_became_peered":"2023-08-27T11:26:56.337945+0000","last_unstale":"2023-08-27T11:26:56.339730+0000","last_undegraded":"2023-08-27T11:26:56.339730+0000","last_fullsized":"2023-08-27T11:26:56.339730+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:57:11.690365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.332559+0000","last_change":"2023-08-27T11:26:56.330006+0000","last_active":"2023-08-27T11:26:56.332559+0000","last_peered":"2023-08-27T11:26:56.332559+0000","last_clean":"2023-08-27T11:26:56.332559+0000","last_became_active":"2023-08-27T11:26:56.329672+0000","last_became_peered":"2023-08-27T11:26:56.329672+0000","last_unstale":"2023-08-27T11:26:56.332559+0000","last_undegraded":"2023-08-27T11:26:56.332559+0000","last_fullsized":"2023-08-27T11:26:56.332559+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:57.070690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.329037+0000","last_change":"2023-08-27T11:26:56.338425+0000","last_active":"2023-08-27T11:26:57.329037+0000","last_peered":"2023-08-27T11:26:57.329037+0000","last_clean":"2023-08-27T11:26:57.329037+0000","last_became_active":"2023-08-27T11:26:56.334863+0000","last_became_peered":"2023-08-27T11:26:56.334863+0000","last_unstale":"2023-08-27T11:26:57.329037+0000","last_undegraded":"2023-08-27T11:26:57.329037+0000","last_fullsized":"2023-08-27T11:26:57.329037+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:14.305235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.338189+0000","last_change":"2023-08-27T11:26:56.336262+0000","last_active":"2023-08-27T11:26:56.338189+0000","last_peered":"2023-08-27T11:26:56.338189+0000","last_clean":"2023-08-27T11:26:56.338189+0000","last_became_active":"2023-08-27T11:26:56.335989+0000","last_became_peered":"2023-08-27T11:26:56.335989+0000","last_unstale":"2023-08-27T11:26:56.338189+0000","last_undegraded":"2023-08-27T11:26:56.338189+0000","last_fullsized":"2023-08-27T11:26:56.338189+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:17:12.187523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.340006+0000","last_change":"2023-08-27T11:26:56.334231+0000","last_active":"2023-08-27T11:26:56.340006+0000","last_peered":"2023-08-27T11:26:56.340006+0000","last_clean":"2023-08-27T11:26:56.340006+0000","last_became_active":"2023-08-27T11:26:56.333968+0000","last_became_peered":"2023-08-27T11:26:56.333968+0000","last_unstale":"2023-08-27T11:26:56.340006+0000","last_undegraded":"2023-08-27T11:26:56.340006+0000","last_fullsized":"2023-08-27T11:26:56.340006+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:27:42.625540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327575+0000","last_change":"2023-08-27T11:26:56.340517+0000","last_active":"2023-08-27T11:26:57.327575+0000","last_peered":"2023-08-27T11:26:57.327575+0000","last_clean":"2023-08-27T11:26:57.327575+0000","last_became_active":"2023-08-27T11:26:56.340269+0000","last_became_peered":"2023-08-27T11:26:56.340269+0000","last_unstale":"2023-08-27T11:26:57.327575+0000","last_undegraded":"2023-08-27T11:26:57.327575+0000","last_fullsized":"2023-08-27T11:26:57.327575+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:39.459063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339545+0000","last_change":"2023-08-27T11:26:56.332850+0000","last_active":"2023-08-27T11:26:56.339545+0000","last_peered":"2023-08-27T11:26:56.339545+0000","last_clean":"2023-08-27T11:26:56.339545+0000","last_became_active":"2023-08-27T11:26:56.332542+0000","last_became_peered":"2023-08-27T11:26:56.332542+0000","last_unstale":"2023-08-27T11:26:56.339545+0000","last_undegraded":"2023-08-27T11:26:56.339545+0000","last_fullsized":"2023-08-27T11:26:56.339545+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:26:35.955137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344625+0000","last_change":"2023-08-27T11:26:56.339443+0000","last_active":"2023-08-27T11:26:56.344625+0000","last_peered":"2023-08-27T11:26:56.344625+0000","last_clean":"2023-08-27T11:26:56.344625+0000","last_became_active":"2023-08-27T11:26:56.338997+0000","last_became_peered":"2023-08-27T11:26:56.338997+0000","last_unstale":"2023-08-27T11:26:56.344625+0000","last_undegraded":"2023-08-27T11:26:56.344625+0000","last_fullsized":"2023-08-27T11:26:56.344625+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:48:59.244598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344676+0000","last_change":"2023-08-27T11:26:56.341489+0000","last_active":"2023-08-27T11:26:56.344676+0000","last_peered":"2023-08-27T11:26:56.344676+0000","last_clean":"2023-08-27T11:26:56.344676+0000","last_became_active":"2023-08-27T11:26:56.340689+0000","last_became_peered":"2023-08-27T11:26:56.340689+0000","last_unstale":"2023-08-27T11:26:56.344676+0000","last_undegraded":"2023-08-27T11:26:56.344676+0000","last_fullsized":"2023-08-27T11:26:56.344676+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:44.284960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339212+0000","last_change":"2023-08-27T11:26:56.337136+0000","last_active":"2023-08-27T11:26:56.339212+0000","last_peered":"2023-08-27T11:26:56.339212+0000","last_clean":"2023-08-27T11:26:56.339212+0000","last_became_active":"2023-08-27T11:26:56.336870+0000","last_became_peered":"2023-08-27T11:26:56.336870+0000","last_unstale":"2023-08-27T11:26:56.339212+0000","last_undegraded":"2023-08-27T11:26:56.339212+0000","last_fullsized":"2023-08-27T11:26:56.339212+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:21:08.994604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.329196+0000","last_change":"2023-08-27T11:26:56.334680+0000","last_active":"2023-08-27T11:26:57.329196+0000","last_peered":"2023-08-27T11:26:57.329196+0000","last_clean":"2023-08-27T11:26:57.329196+0000","last_became_active":"2023-08-27T11:26:56.334385+0000","last_became_peered":"2023-08-27T11:26:56.334385+0000","last_unstale":"2023-08-27T11:26:57.329196+0000","last_undegraded":"2023-08-27T11:26:57.329196+0000","last_fullsized":"2023-08-27T11:26:57.329196+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:35.647340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322939+0000","last_change":"2023-08-27T11:26:56.335394+0000","last_active":"2023-08-27T11:26:57.322939+0000","last_peered":"2023-08-27T11:26:57.322939+0000","last_clean":"2023-08-27T11:26:57.322939+0000","last_became_active":"2023-08-27T11:26:56.335132+0000","last_became_peered":"2023-08-27T11:26:56.335132+0000","last_unstale":"2023-08-27T11:26:57.322939+0000","last_undegraded":"2023-08-27T11:26:57.322939+0000","last_fullsized":"2023-08-27T11:26:57.322939+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:43.704451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.332124+0000","last_change":"2023-08-27T11:26:56.330318+0000","last_active":"2023-08-27T11:26:56.332124+0000","last_peered":"2023-08-27T11:26:56.332124+0000","last_clean":"2023-08-27T11:26:56.332124+0000","last_became_active":"2023-08-27T11:26:56.330052+0000","last_became_peered":"2023-08-27T11:26:56.330052+0000","last_unstale":"2023-08-27T11:26:56.332124+0000","last_undegraded":"2023-08-27T11:26:56.332124+0000","last_fullsized":"2023-08-27T11:26:56.332124+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:04:37.323648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.343736+0000","last_change":"2023-08-27T11:26:56.341456+0000","last_active":"2023-08-27T11:26:56.343736+0000","last_peered":"2023-08-27T11:26:56.343736+0000","last_clean":"2023-08-27T11:26:56.343736+0000","last_became_active":"2023-08-27T11:26:56.341141+0000","last_became_peered":"2023-08-27T11:26:56.341141+0000","last_unstale":"2023-08-27T11:26:56.343736+0000","last_undegraded":"2023-08-27T11:26:56.343736+0000","last_fullsized":"2023-08-27T11:26:56.343736+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:06:26.167325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344484+0000","last_change":"2023-08-27T11:26:56.336729+0000","last_active":"2023-08-27T11:26:56.344484+0000","last_peered":"2023-08-27T11:26:56.344484+0000","last_clean":"2023-08-27T11:26:56.344484+0000","last_became_active":"2023-08-27T11:26:56.336466+0000","last_became_peered":"2023-08-27T11:26:56.336466+0000","last_unstale":"2023-08-27T11:26:56.344484+0000","last_undegraded":"2023-08-27T11:26:56.344484+0000","last_fullsized":"2023-08-27T11:26:56.344484+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:50:18.052365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344604+0000","last_change":"2023-08-27T11:26:56.339258+0000","last_active":"2023-08-27T11:26:56.344604+0000","last_peered":"2023-08-27T11:26:56.344604+0000","last_clean":"2023-08-27T11:26:56.344604+0000","last_became_active":"2023-08-27T11:26:56.338644+0000","last_became_peered":"2023-08-27T11:26:56.338644+0000","last_unstale":"2023-08-27T11:26:56.344604+0000","last_undegraded":"2023-08-27T11:26:56.344604+0000","last_fullsized":"2023-08-27T11:26:56.344604+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:26.089004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.336992+0000","last_change":"2023-08-27T11:26:56.334767+0000","last_active":"2023-08-27T11:26:56.336992+0000","last_peered":"2023-08-27T11:26:56.336992+0000","last_clean":"2023-08-27T11:26:56.336992+0000","last_became_active":"2023-08-27T11:26:56.334152+0000","last_became_peered":"2023-08-27T11:26:56.334152+0000","last_unstale":"2023-08-27T11:26:56.336992+0000","last_undegraded":"2023-08-27T11:26:56.336992+0000","last_fullsized":"2023-08-27T11:26:56.336992+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:48:20.020708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"101'4","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328846+0000","last_change":"2023-08-27T11:26:55.586454+0000","last_active":"2023-08-27T11:26:57.328846+0000","last_peered":"2023-08-27T11:26:57.328846+0000","last_clean":"2023-08-27T11:26:57.328846+0000","last_became_active":"2023-08-27T11:26:55.586152+0000","last_became_peered":"2023-08-27T11:26:55.586152+0000","last_unstale":"2023-08-27T11:26:57.328846+0000","last_undegraded":"2023-08-27T11:26:57.328846+0000","last_fullsized":"2023-08-27T11:26:57.328846+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:45:31.804998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.323060+0000","last_change":"2023-08-27T11:26:56.329075+0000","last_active":"2023-08-27T11:26:57.323060+0000","last_peered":"2023-08-27T11:26:57.323060+0000","last_clean":"2023-08-27T11:26:57.323060+0000","last_became_active":"2023-08-27T11:26:56.328805+0000","last_became_peered":"2023-08-27T11:26:56.328805+0000","last_unstale":"2023-08-27T11:26:57.323060+0000","last_undegraded":"2023-08-27T11:26:57.323060+0000","last_fullsized":"2023-08-27T11:26:57.323060+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:23.782267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"101'1","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.575053+0000","last_change":"2023-08-27T11:26:55.578778+0000","last_active":"2023-08-27T11:26:56.575053+0000","last_peered":"2023-08-27T11:26:56.575053+0000","last_clean":"2023-08-27T11:26:56.575053+0000","last_became_active":"2023-08-27T11:26:55.578526+0000","last_became_peered":"2023-08-27T11:26:55.578526+0000","last_unstale":"2023-08-27T11:26:56.575053+0000","last_undegraded":"2023-08-27T11:26:56.575053+0000","last_fullsized":"2023-08-27T11:26:56.575053+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:34:19.671570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344188+0000","last_change":"2023-08-27T11:26:56.335716+0000","last_active":"2023-08-27T11:26:56.344188+0000","last_peered":"2023-08-27T11:26:56.344188+0000","last_clean":"2023-08-27T11:26:56.344188+0000","last_became_active":"2023-08-27T11:26:56.335414+0000","last_became_peered":"2023-08-27T11:26:56.335414+0000","last_unstale":"2023-08-27T11:26:56.344188+0000","last_undegraded":"2023-08-27T11:26:56.344188+0000","last_fullsized":"2023-08-27T11:26:56.344188+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:00:17.644967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"101'2","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.578039+0000","last_change":"2023-08-27T11:26:55.587764+0000","last_active":"2023-08-27T11:26:56.578039+0000","last_peered":"2023-08-27T11:26:56.578039+0000","last_clean":"2023-08-27T11:26:56.578039+0000","last_became_active":"2023-08-27T11:26:55.587413+0000","last_became_peered":"2023-08-27T11:26:55.587413+0000","last_unstale":"2023-08-27T11:26:56.578039+0000","last_undegraded":"2023-08-27T11:26:56.578039+0000","last_fullsized":"2023-08-27T11:26:56.578039+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:41:16.545929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344277+0000","last_change":"2023-08-27T11:26:56.341218+0000","last_active":"2023-08-27T11:26:56.344277+0000","last_peered":"2023-08-27T11:26:56.344277+0000","last_clean":"2023-08-27T11:26:56.344277+0000","last_became_active":"2023-08-27T11:26:56.340907+0000","last_became_peered":"2023-08-27T11:26:56.340907+0000","last_unstale":"2023-08-27T11:26:56.344277+0000","last_undegraded":"2023-08-27T11:26:56.344277+0000","last_fullsized":"2023-08-27T11:26:56.344277+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:26:59.498363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"101'2","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.576506+0000","last_change":"2023-08-27T11:26:55.581671+0000","last_active":"2023-08-27T11:26:56.576506+0000","last_peered":"2023-08-27T11:26:56.576506+0000","last_clean":"2023-08-27T11:26:56.576506+0000","last_became_active":"2023-08-27T11:26:55.581391+0000","last_became_peered":"2023-08-27T11:26:55.581391+0000","last_unstale":"2023-08-27T11:26:56.576506+0000","last_undegraded":"2023-08-27T11:26:56.576506+0000","last_fullsized":"2023-08-27T11:26:56.576506+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:05:49.716981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.336464+0000","last_change":"2023-08-27T11:26:56.329286+0000","last_active":"2023-08-27T11:26:56.336464+0000","last_peered":"2023-08-27T11:26:56.336464+0000","last_clean":"2023-08-27T11:26:56.336464+0000","last_became_active":"2023-08-27T11:26:56.328972+0000","last_became_peered":"2023-08-27T11:26:56.328972+0000","last_unstale":"2023-08-27T11:26:56.336464+0000","last_undegraded":"2023-08-27T11:26:56.336464+0000","last_fullsized":"2023-08-27T11:26:56.336464+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:59:33.133043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"101'1","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327580+0000","last_change":"2023-08-27T11:26:55.582175+0000","last_active":"2023-08-27T11:26:57.327580+0000","last_peered":"2023-08-27T11:26:57.327580+0000","last_clean":"2023-08-27T11:26:57.327580+0000","last_became_active":"2023-08-27T11:26:55.581875+0000","last_became_peered":"2023-08-27T11:26:55.581875+0000","last_unstale":"2023-08-27T11:26:57.327580+0000","last_undegraded":"2023-08-27T11:26:57.327580+0000","last_fullsized":"2023-08-27T11:26:57.327580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:37:16.373316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327387+0000","last_change":"2023-08-27T11:26:56.341989+0000","last_active":"2023-08-27T11:26:57.327387+0000","last_peered":"2023-08-27T11:26:57.327387+0000","last_clean":"2023-08-27T11:26:57.327387+0000","last_became_active":"2023-08-27T11:26:56.341705+0000","last_became_peered":"2023-08-27T11:26:56.341705+0000","last_unstale":"2023-08-27T11:26:57.327387+0000","last_undegraded":"2023-08-27T11:26:57.327387+0000","last_fullsized":"2023-08-27T11:26:57.327387+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:34:28.655765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"101'1","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322545+0000","last_change":"2023-08-27T11:26:55.581626+0000","last_active":"2023-08-27T11:26:57.322545+0000","last_peered":"2023-08-27T11:26:57.322545+0000","last_clean":"2023-08-27T11:26:57.322545+0000","last_became_active":"2023-08-27T11:26:55.581370+0000","last_became_peered":"2023-08-27T11:26:55.581370+0000","last_unstale":"2023-08-27T11:26:57.322545+0000","last_undegraded":"2023-08-27T11:26:57.322545+0000","last_fullsized":"2023-08-27T11:26:57.322545+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:20:26.130428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.335985+0000","last_change":"2023-08-27T11:26:56.335006+0000","last_active":"2023-08-27T11:26:56.335985+0000","last_peered":"2023-08-27T11:26:56.335985+0000","last_clean":"2023-08-27T11:26:56.335985+0000","last_became_active":"2023-08-27T11:26:56.334692+0000","last_became_peered":"2023-08-27T11:26:56.334692+0000","last_unstale":"2023-08-27T11:26:56.335985+0000","last_undegraded":"2023-08-27T11:26:56.335985+0000","last_fullsized":"2023-08-27T11:26:56.335985+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:57:36.557530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.330710+0000","last_change":"2023-08-27T11:26:55.583745+0000","last_active":"2023-08-27T11:26:56.330710+0000","last_peered":"2023-08-27T11:26:56.330710+0000","last_clean":"2023-08-27T11:26:56.330710+0000","last_became_active":"2023-08-27T11:26:55.583446+0000","last_became_peered":"2023-08-27T11:26:55.583446+0000","last_unstale":"2023-08-27T11:26:56.330710+0000","last_undegraded":"2023-08-27T11:26:56.330710+0000","last_fullsized":"2023-08-27T11:26:56.330710+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:28:45.011961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.336599+0000","last_change":"2023-08-27T11:26:56.327884+0000","last_active":"2023-08-27T11:26:56.336599+0000","last_peered":"2023-08-27T11:26:56.336599+0000","last_clean":"2023-08-27T11:26:56.336599+0000","last_became_active":"2023-08-27T11:26:56.327633+0000","last_became_peered":"2023-08-27T11:26:56.327633+0000","last_unstale":"2023-08-27T11:26:56.336599+0000","last_undegraded":"2023-08-27T11:26:56.336599+0000","last_fullsized":"2023-08-27T11:26:56.336599+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:45:00.914901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322704+0000","last_change":"2023-08-27T11:26:55.584436+0000","last_active":"2023-08-27T11:26:57.322704+0000","last_peered":"2023-08-27T11:26:57.322704+0000","last_clean":"2023-08-27T11:26:57.322704+0000","last_became_active":"2023-08-27T11:26:55.584121+0000","last_became_peered":"2023-08-27T11:26:55.584121+0000","last_unstale":"2023-08-27T11:26:57.322704+0000","last_undegraded":"2023-08-27T11:26:57.322704+0000","last_fullsized":"2023-08-27T11:26:57.322704+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:53.593374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344373+0000","last_change":"2023-08-27T11:26:56.336443+0000","last_active":"2023-08-27T11:26:56.344373+0000","last_peered":"2023-08-27T11:26:56.344373+0000","last_clean":"2023-08-27T11:26:56.344373+0000","last_became_active":"2023-08-27T11:26:56.336159+0000","last_became_peered":"2023-08-27T11:26:56.336159+0000","last_unstale":"2023-08-27T11:26:56.344373+0000","last_undegraded":"2023-08-27T11:26:56.344373+0000","last_fullsized":"2023-08-27T11:26:56.344373+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:36:12.710448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"101'1","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.579478+0000","last_change":"2023-08-27T11:26:55.579157+0000","last_active":"2023-08-27T11:26:56.579478+0000","last_peered":"2023-08-27T11:26:56.579478+0000","last_clean":"2023-08-27T11:26:56.579478+0000","last_became_active":"2023-08-27T11:26:55.578848+0000","last_became_peered":"2023-08-27T11:26:55.578848+0000","last_unstale":"2023-08-27T11:26:56.579478+0000","last_undegraded":"2023-08-27T11:26:56.579478+0000","last_fullsized":"2023-08-27T11:26:56.579478+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:04:45.535534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327922+0000","last_change":"2023-08-27T11:26:56.337690+0000","last_active":"2023-08-27T11:26:57.327922+0000","last_peered":"2023-08-27T11:26:57.327922+0000","last_clean":"2023-08-27T11:26:57.327922+0000","last_became_active":"2023-08-27T11:26:56.337380+0000","last_became_peered":"2023-08-27T11:26:56.337380+0000","last_unstale":"2023-08-27T11:26:57.327922+0000","last_undegraded":"2023-08-27T11:26:57.327922+0000","last_fullsized":"2023-08-27T11:26:57.327922+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:06:42.811268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"101'2","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.577961+0000","last_change":"2023-08-27T11:26:55.580451+0000","last_active":"2023-08-27T11:26:56.577961+0000","last_peered":"2023-08-27T11:26:56.577961+0000","last_clean":"2023-08-27T11:26:56.577961+0000","last_became_active":"2023-08-27T11:26:55.580114+0000","last_became_peered":"2023-08-27T11:26:55.580114+0000","last_unstale":"2023-08-27T11:26:56.577961+0000","last_undegraded":"2023-08-27T11:26:56.577961+0000","last_fullsized":"2023-08-27T11:26:56.577961+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:45:42.879972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.323072+0000","last_change":"2023-08-27T11:26:56.331051+0000","last_active":"2023-08-27T11:26:57.323072+0000","last_peered":"2023-08-27T11:26:57.323072+0000","last_clean":"2023-08-27T11:26:57.323072+0000","last_became_active":"2023-08-27T11:26:56.330821+0000","last_became_peered":"2023-08-27T11:26:56.330821+0000","last_unstale":"2023-08-27T11:26:57.323072+0000","last_undegraded":"2023-08-27T11:26:57.323072+0000","last_fullsized":"2023-08-27T11:26:57.323072+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:00:27.371100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"101'1","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.578168+0000","last_change":"2023-08-27T11:26:55.587419+0000","last_active":"2023-08-27T11:26:56.578168+0000","last_peered":"2023-08-27T11:26:56.578168+0000","last_clean":"2023-08-27T11:26:56.578168+0000","last_became_active":"2023-08-27T11:26:55.586785+0000","last_became_peered":"2023-08-27T11:26:55.586785+0000","last_unstale":"2023-08-27T11:26:56.578168+0000","last_undegraded":"2023-08-27T11:26:56.578168+0000","last_fullsized":"2023-08-27T11:26:56.578168+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:23.921659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344504+0000","last_change":"2023-08-27T11:26:56.337097+0000","last_active":"2023-08-27T11:26:56.344504+0000","last_peered":"2023-08-27T11:26:56.344504+0000","last_clean":"2023-08-27T11:26:56.344504+0000","last_became_active":"2023-08-27T11:26:56.336480+0000","last_became_peered":"2023-08-27T11:26:56.336480+0000","last_unstale":"2023-08-27T11:26:56.344504+0000","last_undegraded":"2023-08-27T11:26:56.344504+0000","last_fullsized":"2023-08-27T11:26:56.344504+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:42:39.682934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.327636+0000","last_change":"2023-08-27T11:26:55.590251+0000","last_active":"2023-08-27T11:26:56.327636+0000","last_peered":"2023-08-27T11:26:56.327636+0000","last_clean":"2023-08-27T11:26:56.327636+0000","last_became_active":"2023-08-27T11:26:55.590020+0000","last_became_peered":"2023-08-27T11:26:55.590020+0000","last_unstale":"2023-08-27T11:26:56.327636+0000","last_undegraded":"2023-08-27T11:26:56.327636+0000","last_fullsized":"2023-08-27T11:26:56.327636+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:47:29.396825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322406+0000","last_change":"2023-08-27T11:26:56.331375+0000","last_active":"2023-08-27T11:26:57.322406+0000","last_peered":"2023-08-27T11:26:57.322406+0000","last_clean":"2023-08-27T11:26:57.322406+0000","last_became_active":"2023-08-27T11:26:56.331109+0000","last_became_peered":"2023-08-27T11:26:56.331109+0000","last_unstale":"2023-08-27T11:26:57.322406+0000","last_undegraded":"2023-08-27T11:26:57.322406+0000","last_fullsized":"2023-08-27T11:26:57.322406+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:31.087412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"101'2","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322785+0000","last_change":"2023-08-27T11:26:55.584910+0000","last_active":"2023-08-27T11:26:57.322785+0000","last_peered":"2023-08-27T11:26:57.322785+0000","last_clean":"2023-08-27T11:26:57.322785+0000","last_became_active":"2023-08-27T11:26:55.584623+0000","last_became_peered":"2023-08-27T11:26:55.584623+0000","last_unstale":"2023-08-27T11:26:57.322785+0000","last_undegraded":"2023-08-27T11:26:57.322785+0000","last_fullsized":"2023-08-27T11:26:57.322785+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:13:17.448970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339457+0000","last_change":"2023-08-27T11:26:56.337341+0000","last_active":"2023-08-27T11:26:56.339457+0000","last_peered":"2023-08-27T11:26:56.339457+0000","last_clean":"2023-08-27T11:26:56.339457+0000","last_became_active":"2023-08-27T11:26:56.337021+0000","last_became_peered":"2023-08-27T11:26:56.337021+0000","last_unstale":"2023-08-27T11:26:56.339457+0000","last_undegraded":"2023-08-27T11:26:56.339457+0000","last_fullsized":"2023-08-27T11:26:56.339457+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:59:38.308471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.325667+0000","last_change":"2023-08-27T11:26:55.588388+0000","last_active":"2023-08-27T11:26:56.325667+0000","last_peered":"2023-08-27T11:26:56.325667+0000","last_clean":"2023-08-27T11:26:56.325667+0000","last_became_active":"2023-08-27T11:26:55.588047+0000","last_became_peered":"2023-08-27T11:26:55.588047+0000","last_unstale":"2023-08-27T11:26:56.325667+0000","last_undegraded":"2023-08-27T11:26:56.325667+0000","last_fullsized":"2023-08-27T11:26:56.325667+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:25:36.557116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.334901+0000","last_change":"2023-08-27T11:26:56.332807+0000","last_active":"2023-08-27T11:26:56.334901+0000","last_peered":"2023-08-27T11:26:56.334901+0000","last_clean":"2023-08-27T11:26:56.334901+0000","last_became_active":"2023-08-27T11:26:56.332429+0000","last_became_peered":"2023-08-27T11:26:56.332429+0000","last_unstale":"2023-08-27T11:26:56.334901+0000","last_undegraded":"2023-08-27T11:26:56.334901+0000","last_fullsized":"2023-08-27T11:26:56.334901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:37:01.825078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.321372+0000","last_change":"2023-08-27T11:26:55.579967+0000","last_active":"2023-08-27T11:26:56.321372+0000","last_peered":"2023-08-27T11:26:56.321372+0000","last_clean":"2023-08-27T11:26:56.321372+0000","last_became_active":"2023-08-27T11:26:55.579711+0000","last_became_peered":"2023-08-27T11:26:55.579711+0000","last_unstale":"2023-08-27T11:26:56.321372+0000","last_undegraded":"2023-08-27T11:26:56.321372+0000","last_fullsized":"2023-08-27T11:26:56.321372+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:14.065624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344302+0000","last_change":"2023-08-27T11:26:56.336199+0000","last_active":"2023-08-27T11:26:56.344302+0000","last_peered":"2023-08-27T11:26:56.344302+0000","last_clean":"2023-08-27T11:26:56.344302+0000","last_became_active":"2023-08-27T11:26:56.335902+0000","last_became_peered":"2023-08-27T11:26:56.335902+0000","last_unstale":"2023-08-27T11:26:56.344302+0000","last_undegraded":"2023-08-27T11:26:56.344302+0000","last_fullsized":"2023-08-27T11:26:56.344302+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:03.648033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.325122+0000","last_change":"2023-08-27T11:26:55.587769+0000","last_active":"2023-08-27T11:26:56.325122+0000","last_peered":"2023-08-27T11:26:56.325122+0000","last_clean":"2023-08-27T11:26:56.325122+0000","last_became_active":"2023-08-27T11:26:55.587366+0000","last_became_peered":"2023-08-27T11:26:55.587366+0000","last_unstale":"2023-08-27T11:26:56.325122+0000","last_undegraded":"2023-08-27T11:26:56.325122+0000","last_fullsized":"2023-08-27T11:26:56.325122+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:00:35.636685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322900+0000","last_change":"2023-08-27T11:26:56.335078+0000","last_active":"2023-08-27T11:26:57.322900+0000","last_peered":"2023-08-27T11:26:57.322900+0000","last_clean":"2023-08-27T11:26:57.322900+0000","last_became_active":"2023-08-27T11:26:56.334811+0000","last_became_peered":"2023-08-27T11:26:56.334811+0000","last_unstale":"2023-08-27T11:26:57.322900+0000","last_undegraded":"2023-08-27T11:26:57.322900+0000","last_fullsized":"2023-08-27T11:26:57.322900+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:05.072625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"101'2","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.323345+0000","last_change":"2023-08-27T11:26:55.579468+0000","last_active":"2023-08-27T11:26:57.323345+0000","last_peered":"2023-08-27T11:26:57.323345+0000","last_clean":"2023-08-27T11:26:57.323345+0000","last_became_active":"2023-08-27T11:26:55.579211+0000","last_became_peered":"2023-08-27T11:26:55.579211+0000","last_unstale":"2023-08-27T11:26:57.323345+0000","last_undegraded":"2023-08-27T11:26:57.323345+0000","last_fullsized":"2023-08-27T11:26:57.323345+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:24:35.451519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.336495+0000","last_change":"2023-08-27T11:26:56.329332+0000","last_active":"2023-08-27T11:26:56.336495+0000","last_peered":"2023-08-27T11:26:56.336495+0000","last_clean":"2023-08-27T11:26:56.336495+0000","last_became_active":"2023-08-27T11:26:56.329030+0000","last_became_peered":"2023-08-27T11:26:56.329030+0000","last_unstale":"2023-08-27T11:26:56.336495+0000","last_undegraded":"2023-08-27T11:26:56.336495+0000","last_fullsized":"2023-08-27T11:26:56.336495+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:21.500768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328803+0000","last_change":"2023-08-27T11:26:55.579972+0000","last_active":"2023-08-27T11:26:57.328803+0000","last_peered":"2023-08-27T11:26:57.328803+0000","last_clean":"2023-08-27T11:26:57.328803+0000","last_became_active":"2023-08-27T11:26:55.579637+0000","last_became_peered":"2023-08-27T11:26:55.579637+0000","last_unstale":"2023-08-27T11:26:57.328803+0000","last_undegraded":"2023-08-27T11:26:57.328803+0000","last_fullsized":"2023-08-27T11:26:57.328803+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:44:41.838536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344316+0000","last_change":"2023-08-27T11:26:56.341267+0000","last_active":"2023-08-27T11:26:56.344316+0000","last_peered":"2023-08-27T11:26:56.344316+0000","last_clean":"2023-08-27T11:26:56.344316+0000","last_became_active":"2023-08-27T11:26:56.340982+0000","last_became_peered":"2023-08-27T11:26:56.340982+0000","last_unstale":"2023-08-27T11:26:56.344316+0000","last_undegraded":"2023-08-27T11:26:56.344316+0000","last_fullsized":"2023-08-27T11:26:56.344316+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:08:06.414855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.329345+0000","last_change":"2023-08-27T11:26:55.581127+0000","last_active":"2023-08-27T11:26:57.329345+0000","last_peered":"2023-08-27T11:26:57.329345+0000","last_clean":"2023-08-27T11:26:57.329345+0000","last_became_active":"2023-08-27T11:26:55.580654+0000","last_became_peered":"2023-08-27T11:26:55.580654+0000","last_unstale":"2023-08-27T11:26:57.329345+0000","last_undegraded":"2023-08-27T11:26:57.329345+0000","last_fullsized":"2023-08-27T11:26:57.329345+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:13:09.993091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.336412+0000","last_change":"2023-08-27T11:26:56.333736+0000","last_active":"2023-08-27T11:26:56.336412+0000","last_peered":"2023-08-27T11:26:56.336412+0000","last_clean":"2023-08-27T11:26:56.336412+0000","last_became_active":"2023-08-27T11:26:56.333480+0000","last_became_peered":"2023-08-27T11:26:56.333480+0000","last_unstale":"2023-08-27T11:26:56.336412+0000","last_undegraded":"2023-08-27T11:26:56.336412+0000","last_fullsized":"2023-08-27T11:26:56.336412+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:35:41.830099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.323047+0000","last_change":"2023-08-27T11:26:55.582915+0000","last_active":"2023-08-27T11:26:57.323047+0000","last_peered":"2023-08-27T11:26:57.323047+0000","last_clean":"2023-08-27T11:26:57.323047+0000","last_became_active":"2023-08-27T11:26:55.582556+0000","last_became_peered":"2023-08-27T11:26:55.582556+0000","last_unstale":"2023-08-27T11:26:57.323047+0000","last_undegraded":"2023-08-27T11:26:57.323047+0000","last_fullsized":"2023-08-27T11:26:57.323047+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:59:37.302724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322566+0000","last_change":"2023-08-27T11:26:56.339670+0000","last_active":"2023-08-27T11:26:57.322566+0000","last_peered":"2023-08-27T11:26:57.322566+0000","last_clean":"2023-08-27T11:26:57.322566+0000","last_became_active":"2023-08-27T11:26:56.339421+0000","last_became_peered":"2023-08-27T11:26:56.339421+0000","last_unstale":"2023-08-27T11:26:57.322566+0000","last_undegraded":"2023-08-27T11:26:57.322566+0000","last_fullsized":"2023-08-27T11:26:57.322566+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:55:23.934383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327626+0000","last_change":"2023-08-27T11:26:55.581587+0000","last_active":"2023-08-27T11:26:57.327626+0000","last_peered":"2023-08-27T11:26:57.327626+0000","last_clean":"2023-08-27T11:26:57.327626+0000","last_became_active":"2023-08-27T11:26:55.581259+0000","last_became_peered":"2023-08-27T11:26:55.581259+0000","last_unstale":"2023-08-27T11:26:57.327626+0000","last_undegraded":"2023-08-27T11:26:57.327626+0000","last_fullsized":"2023-08-27T11:26:57.327626+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:44:41.838536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.334100+0000","last_change":"2023-08-27T11:26:56.331407+0000","last_active":"2023-08-27T11:26:56.334100+0000","last_peered":"2023-08-27T11:26:56.334100+0000","last_clean":"2023-08-27T11:26:56.334100+0000","last_became_active":"2023-08-27T11:26:56.331157+0000","last_became_peered":"2023-08-27T11:26:56.331157+0000","last_unstale":"2023-08-27T11:26:56.334100+0000","last_undegraded":"2023-08-27T11:26:56.334100+0000","last_fullsized":"2023-08-27T11:26:56.334100+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:05.072625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327650+0000","last_change":"2023-08-27T11:26:55.584486+0000","last_active":"2023-08-27T11:26:57.327650+0000","last_peered":"2023-08-27T11:26:57.327650+0000","last_clean":"2023-08-27T11:26:57.327650+0000","last_became_active":"2023-08-27T11:26:55.584146+0000","last_became_peered":"2023-08-27T11:26:55.584146+0000","last_unstale":"2023-08-27T11:26:57.327650+0000","last_undegraded":"2023-08-27T11:26:57.327650+0000","last_fullsized":"2023-08-27T11:26:57.327650+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:45:31.804998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339847+0000","last_change":"2023-08-27T11:26:56.337314+0000","last_active":"2023-08-27T11:26:56.339847+0000","last_peered":"2023-08-27T11:26:56.339847+0000","last_clean":"2023-08-27T11:26:56.339847+0000","last_became_active":"2023-08-27T11:26:56.336826+0000","last_became_peered":"2023-08-27T11:26:56.336826+0000","last_unstale":"2023-08-27T11:26:56.339847+0000","last_undegraded":"2023-08-27T11:26:56.339847+0000","last_fullsized":"2023-08-27T11:26:56.339847+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:01:35.049959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.322467+0000","last_change":"2023-08-27T11:26:55.580984+0000","last_active":"2023-08-27T11:26:57.322467+0000","last_peered":"2023-08-27T11:26:57.322467+0000","last_clean":"2023-08-27T11:26:57.322467+0000","last_became_active":"2023-08-27T11:26:55.580719+0000","last_became_peered":"2023-08-27T11:26:55.580719+0000","last_unstale":"2023-08-27T11:26:57.322467+0000","last_undegraded":"2023-08-27T11:26:57.322467+0000","last_fullsized":"2023-08-27T11:26:57.322467+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:17:23.104816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.337026+0000","last_change":"2023-08-27T11:26:56.333780+0000","last_active":"2023-08-27T11:26:56.337026+0000","last_peered":"2023-08-27T11:26:56.337026+0000","last_clean":"2023-08-27T11:26:56.337026+0000","last_became_active":"2023-08-27T11:26:56.333181+0000","last_became_peered":"2023-08-27T11:26:56.333181+0000","last_unstale":"2023-08-27T11:26:56.337026+0000","last_undegraded":"2023-08-27T11:26:56.337026+0000","last_fullsized":"2023-08-27T11:26:56.337026+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:49.527060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.325139+0000","last_change":"2023-08-27T11:26:55.586684+0000","last_active":"2023-08-27T11:26:56.325139+0000","last_peered":"2023-08-27T11:26:56.325139+0000","last_clean":"2023-08-27T11:26:56.325139+0000","last_became_active":"2023-08-27T11:26:55.586295+0000","last_became_peered":"2023-08-27T11:26:55.586295+0000","last_unstale":"2023-08-27T11:26:56.325139+0000","last_undegraded":"2023-08-27T11:26:56.325139+0000","last_fullsized":"2023-08-27T11:26:56.325139+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:44:01.667490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328578+0000","last_change":"2023-08-27T11:26:56.338204+0000","last_active":"2023-08-27T11:26:57.328578+0000","last_peered":"2023-08-27T11:26:57.328578+0000","last_clean":"2023-08-27T11:26:57.328578+0000","last_became_active":"2023-08-27T11:26:56.334760+0000","last_became_peered":"2023-08-27T11:26:56.334760+0000","last_unstale":"2023-08-27T11:26:57.328578+0000","last_undegraded":"2023-08-27T11:26:57.328578+0000","last_fullsized":"2023-08-27T11:26:57.328578+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:42:52.388238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328911+0000","last_change":"2023-08-27T11:26:55.580094+0000","last_active":"2023-08-27T11:26:57.328911+0000","last_peered":"2023-08-27T11:26:57.328911+0000","last_clean":"2023-08-27T11:26:57.328911+0000","last_became_active":"2023-08-27T11:26:55.579780+0000","last_became_peered":"2023-08-27T11:26:55.579780+0000","last_unstale":"2023-08-27T11:26:57.328911+0000","last_undegraded":"2023-08-27T11:26:57.328911+0000","last_fullsized":"2023-08-27T11:26:57.328911+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:30:04.134051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":187,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328915+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:26:57.328915+0000","last_peered":"2023-08-27T11:26:57.328915+0000","last_clean":"2023-08-27T11:26:57.328915+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:26:57.328915+0000","last_undegraded":"2023-08-27T11:26:57.328915+0000","last_fullsized":"2023-08-27T11:26:57.328915+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:56:45.554523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.342096+0000","last_change":"2023-08-27T11:26:56.333603+0000","last_active":"2023-08-27T11:26:56.342096+0000","last_peered":"2023-08-27T11:26:56.342096+0000","last_clean":"2023-08-27T11:26:56.342096+0000","last_became_active":"2023-08-27T11:26:56.333297+0000","last_became_peered":"2023-08-27T11:26:56.333297+0000","last_unstale":"2023-08-27T11:26:56.342096+0000","last_undegraded":"2023-08-27T11:26:56.342096+0000","last_fullsized":"2023-08-27T11:26:56.342096+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:27.152327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.329896+0000","last_change":"2023-08-27T11:26:55.583739+0000","last_active":"2023-08-27T11:26:56.329896+0000","last_peered":"2023-08-27T11:26:56.329896+0000","last_clean":"2023-08-27T11:26:56.329896+0000","last_became_active":"2023-08-27T11:26:55.583473+0000","last_became_peered":"2023-08-27T11:26:55.583473+0000","last_unstale":"2023-08-27T11:26:56.329896+0000","last_undegraded":"2023-08-27T11:26:56.329896+0000","last_fullsized":"2023-08-27T11:26:56.329896+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:06:25.014565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":187,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328730+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:26:57.328730+0000","last_peered":"2023-08-27T11:26:57.328730+0000","last_clean":"2023-08-27T11:26:57.328730+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:26:57.328730+0000","last_undegraded":"2023-08-27T11:26:57.328730+0000","last_fullsized":"2023-08-27T11:26:57.328730+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:27:20.122317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.343563+0000","last_change":"2023-08-27T11:26:56.341510+0000","last_active":"2023-08-27T11:26:56.343563+0000","last_peered":"2023-08-27T11:26:56.343563+0000","last_clean":"2023-08-27T11:26:56.343563+0000","last_became_active":"2023-08-27T11:26:56.341204+0000","last_became_peered":"2023-08-27T11:26:56.341204+0000","last_unstale":"2023-08-27T11:26:56.343563+0000","last_undegraded":"2023-08-27T11:26:56.343563+0000","last_fullsized":"2023-08-27T11:26:56.343563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:17:23.703806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.321238+0000","last_change":"2023-08-27T11:26:55.581210+0000","last_active":"2023-08-27T11:26:56.321238+0000","last_peered":"2023-08-27T11:26:56.321238+0000","last_clean":"2023-08-27T11:26:56.321238+0000","last_became_active":"2023-08-27T11:26:55.580944+0000","last_became_peered":"2023-08-27T11:26:55.580944+0000","last_unstale":"2023-08-27T11:26:56.321238+0000","last_undegraded":"2023-08-27T11:26:56.321238+0000","last_fullsized":"2023-08-27T11:26:56.321238+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:00:26.365353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":159,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.331068+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:26:56.331068+0000","last_peered":"2023-08-27T11:26:56.331068+0000","last_clean":"2023-08-27T11:26:56.331068+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:26:56.331068+0000","last_undegraded":"2023-08-27T11:26:56.331068+0000","last_fullsized":"2023-08-27T11:26:56.331068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:42:51.656620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.339985+0000","last_change":"2023-08-27T11:26:56.333698+0000","last_active":"2023-08-27T11:26:56.339985+0000","last_peered":"2023-08-27T11:26:56.339985+0000","last_clean":"2023-08-27T11:26:56.339985+0000","last_became_active":"2023-08-27T11:26:56.333431+0000","last_became_peered":"2023-08-27T11:26:56.333431+0000","last_unstale":"2023-08-27T11:26:56.339985+0000","last_undegraded":"2023-08-27T11:26:56.339985+0000","last_fullsized":"2023-08-27T11:26:56.339985+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:32:09.535815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"101'8","reported_seq":19,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.582937+0000","last_change":"2023-08-27T11:26:55.585886+0000","last_active":"2023-08-27T11:26:56.582937+0000","last_peered":"2023-08-27T11:26:56.582937+0000","last_clean":"2023-08-27T11:26:56.582937+0000","last_became_active":"2023-08-27T11:26:55.585639+0000","last_became_peered":"2023-08-27T11:26:55.585639+0000","last_unstale":"2023-08-27T11:26:56.582937+0000","last_undegraded":"2023-08-27T11:26:56.582937+0000","last_fullsized":"2023-08-27T11:26:56.582937+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:53.593374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":157,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.325414+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:26:56.325414+0000","last_peered":"2023-08-27T11:26:56.325414+0000","last_clean":"2023-08-27T11:26:56.325414+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:26:56.325414+0000","last_undegraded":"2023-08-27T11:26:56.325414+0000","last_fullsized":"2023-08-27T11:26:56.325414+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:10:24.148761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.336425+0000","last_change":"2023-08-27T11:26:56.332815+0000","last_active":"2023-08-27T11:26:56.336425+0000","last_peered":"2023-08-27T11:26:56.336425+0000","last_clean":"2023-08-27T11:26:56.336425+0000","last_became_active":"2023-08-27T11:26:56.332539+0000","last_became_peered":"2023-08-27T11:26:56.332539+0000","last_unstale":"2023-08-27T11:26:56.336425+0000","last_undegraded":"2023-08-27T11:26:56.336425+0000","last_fullsized":"2023-08-27T11:26:56.336425+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:13:11.808107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.322917+0000","last_change":"2023-08-27T11:26:55.581749+0000","last_active":"2023-08-27T11:26:56.322917+0000","last_peered":"2023-08-27T11:26:56.322917+0000","last_clean":"2023-08-27T11:26:56.322917+0000","last_became_active":"2023-08-27T11:26:55.581397+0000","last_became_peered":"2023-08-27T11:26:55.581397+0000","last_unstale":"2023-08-27T11:26:56.322917+0000","last_undegraded":"2023-08-27T11:26:56.322917+0000","last_fullsized":"2023-08-27T11:26:56.322917+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:25.083257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":157,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.325772+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:26:56.325772+0000","last_peered":"2023-08-27T11:26:56.325772+0000","last_clean":"2023-08-27T11:26:56.325772+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:26:56.325772+0000","last_undegraded":"2023-08-27T11:26:56.325772+0000","last_fullsized":"2023-08-27T11:26:56.325772+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:25:18.614507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.328665+0000","last_change":"2023-08-27T11:26:56.329794+0000","last_active":"2023-08-27T11:26:57.328665+0000","last_peered":"2023-08-27T11:26:57.328665+0000","last_clean":"2023-08-27T11:26:57.328665+0000","last_became_active":"2023-08-27T11:26:56.329504+0000","last_became_peered":"2023-08-27T11:26:56.329504+0000","last_unstale":"2023-08-27T11:26:57.328665+0000","last_undegraded":"2023-08-27T11:26:57.328665+0000","last_fullsized":"2023-08-27T11:26:57.328665+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:39.459063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"101'2","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.580085+0000","last_change":"2023-08-27T11:26:55.584519+0000","last_active":"2023-08-27T11:26:56.580085+0000","last_peered":"2023-08-27T11:26:56.580085+0000","last_clean":"2023-08-27T11:26:56.580085+0000","last_became_active":"2023-08-27T11:26:55.584238+0000","last_became_peered":"2023-08-27T11:26:55.584238+0000","last_unstale":"2023-08-27T11:26:56.580085+0000","last_undegraded":"2023-08-27T11:26:56.580085+0000","last_fullsized":"2023-08-27T11:26:56.580085+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:42.698705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":307,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.325269+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:26:56.325269+0000","last_peered":"2023-08-27T11:26:56.325269+0000","last_clean":"2023-08-27T11:26:56.325269+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:26:56.325269+0000","last_undegraded":"2023-08-27T11:26:56.325269+0000","last_fullsized":"2023-08-27T11:26:56.325269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:49:26.983333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":163,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.327342+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:26:56.327342+0000","last_peered":"2023-08-27T11:26:56.327342+0000","last_clean":"2023-08-27T11:26:56.327342+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:26:56.327342+0000","last_undegraded":"2023-08-27T11:26:56.327342+0000","last_fullsized":"2023-08-27T11:26:56.327342+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:38:30.057816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.344527+0000","last_change":"2023-08-27T11:26:56.337315+0000","last_active":"2023-08-27T11:26:56.344527+0000","last_peered":"2023-08-27T11:26:56.344527+0000","last_clean":"2023-08-27T11:26:56.344527+0000","last_became_active":"2023-08-27T11:26:56.337056+0000","last_became_peered":"2023-08-27T11:26:56.337056+0000","last_unstale":"2023-08-27T11:26:56.344527+0000","last_undegraded":"2023-08-27T11:26:56.344527+0000","last_fullsized":"2023-08-27T11:26:56.344527+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:28:39.173763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"101'6","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.582876+0000","last_change":"2023-08-27T11:26:55.581149+0000","last_active":"2023-08-27T11:26:56.582876+0000","last_peered":"2023-08-27T11:26:56.582876+0000","last_clean":"2023-08-27T11:26:56.582876+0000","last_became_active":"2023-08-27T11:26:55.580849+0000","last_became_peered":"2023-08-27T11:26:55.580849+0000","last_unstale":"2023-08-27T11:26:56.582876+0000","last_undegraded":"2023-08-27T11:26:56.582876+0000","last_fullsized":"2023-08-27T11:26:56.582876+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:24:35.451519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":155,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.321613+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:26:56.321613+0000","last_peered":"2023-08-27T11:26:56.321613+0000","last_clean":"2023-08-27T11:26:56.321613+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:26:56.321613+0000","last_undegraded":"2023-08-27T11:26:56.321613+0000","last_fullsized":"2023-08-27T11:26:56.321613+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:35.245524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.333886+0000","last_change":"2023-08-27T11:26:56.330525+0000","last_active":"2023-08-27T11:26:56.333886+0000","last_peered":"2023-08-27T11:26:56.333886+0000","last_clean":"2023-08-27T11:26:56.333886+0000","last_became_active":"2023-08-27T11:26:56.330249+0000","last_became_peered":"2023-08-27T11:26:56.330249+0000","last_unstale":"2023-08-27T11:26:56.333886+0000","last_undegraded":"2023-08-27T11:26:56.333886+0000","last_fullsized":"2023-08-27T11:26:56.333886+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:55.561663+0000","last_clean_scrub_stamp":"2023-08-27T11:26:55.561663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:04:48.846519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"101'3","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-08-27T11:26:56.578568+0000","last_change":"2023-08-27T11:26:55.586242+0000","last_active":"2023-08-27T11:26:56.578568+0000","last_peered":"2023-08-27T11:26:56.578568+0000","last_clean":"2023-08-27T11:26:56.578568+0000","last_became_active":"2023-08-27T11:26:55.585929+0000","last_became_peered":"2023-08-27T11:26:55.585929+0000","last_unstale":"2023-08-27T11:26:56.578568+0000","last_undegraded":"2023-08-27T11:26:56.578568+0000","last_fullsized":"2023-08-27T11:26:56.578568+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:26:54.555916+0000","last_clean_scrub_stamp":"2023-08-27T11:26:54.555916+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:09:48.090341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":177,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-27T11:26:57.327920+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:26:57.327920+0000","last_peered":"2023-08-27T11:26:57.327920+0000","last_clean":"2023-08-27T11:26:57.327920+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:26:57.327920+0000","last_undegraded":"2023-08-27T11:26:57.327920+0000","last_fullsized":"2023-08-27T11:26:57.327920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:12:17.048490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574972,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1867776,"data_stored":1170032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.49299999999999999},"min":{"1min":0.33500000000000002,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.39400000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.55000000000000004,"15min":0.50800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":0.77500000000000002,"15min":0.872},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.253},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.53900000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.48699999999999999},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.33800000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.90200000000000002},"last":0.60999999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.57499999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.505},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.72699999999999998,"5min":0.72799999999999998,"15min":0.77400000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.41399999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.44}]},{"osd":6,"last update":"Sun Aug 27 11:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.55000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1261568,"data_stored":579470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51500000000000001,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.29099999999999998},"max":{"1min":0.626,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67300000000000004,"5min":0.86199999999999999,"15min":0.89800000000000002},"last":0.48099999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.316,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.80600000000000005,"15min":0.873},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.62,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.67700000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58299999999999996,"5min":0.74199999999999999,"15min":0.85899999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.65500000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.58199999999999996,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.70699999999999996,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.65200000000000002,"5min":0.84699999999999998,"15min":0.93799999999999994},"last":0.621}]},{"osd":4,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.70699999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.374},"max":{"1min":0.77800000000000002,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.76400000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.624,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.78100000000000003,"5min":0.875,"15min":0.875},"last":0.748}]},{"osd":7,"last update":"Sun Aug 27 11:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.748,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.34899999999999998},"max":{"1min":0.76700000000000002,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.82999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1855488,"data_stored":1169278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.308,"15min":0.59299999999999997},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.45800000000000002,"5min":0.502,"15min":11.971},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.89000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.51700000000000002,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.27300000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.625},"min":{"1min":0.23400000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.495,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.34300000000000003},"min":{"1min":0.246,"5min":0.224,"15min":0.224},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.58599999999999997},"last":0.317}]},{"osd":3,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.64100000000000001},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":10.99},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.373,"15min":0.92500000000000004},"min":{"1min":0.318,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.54700000000000004,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.38500000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.36099999999999999,"15min":0.33400000000000002},"max":{"1min":0.76900000000000002,"5min":0.90700000000000003,"15min":1.26},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.80700000000000005},"min":{"1min":0.434,"5min":0.37,"15min":0.29099999999999998},"max":{"1min":0.64500000000000002,"5min":0.77600000000000002,"15min":10.957000000000001},"last":0.50600000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":1.121},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.36199999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71499999999999997,"5min":0.88700000000000001,"15min":1.296},"last":0.60399999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.84299999999999997},"min":{"1min":0.44800000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":1.1379999999999999},"min":{"1min":0.435,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.74299999999999999,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.497}]},{"osd":7,"last update":"Sun Aug 27 11:26:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":1.1319999999999999},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84199999999999997,"15min":22.988},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.63500000000000001},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":1.327},"last":0.51600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574972,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":577549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.29699999999999999,"15min":0.311},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.33200000000000002,"15min":0.34999999999999998},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.42699999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.24399999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.628,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.33200000000000002,"15min":0.33900000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.35599999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.33700000000000002,"15min":0.34599999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.502,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.27500000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.54200000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.40100000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79100000000000004,"5min":0.92500000000000004,"15min":1.214},"last":0.628},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.56999999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.77500000000000002,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85399999999999998,"5min":0.90200000000000002,"15min":1.603},"last":0.65700000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.621},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.66800000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.74099999999999999,"5min":0.94199999999999995,"15min":1.383},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82699999999999996,"5min":0.88900000000000001,"15min":0.92000000000000004},"last":0.67900000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":578888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29799999999999999,"15min":0.30199999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40500000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.314,"5min":0.316,"15min":0.317},"min":{"1min":0.22700000000000001,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.629},"last":0.32200000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.317,"15min":0.32400000000000001},"min":{"1min":0.222,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.36499999999999999,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33800000000000002,"15min":0.34599999999999997},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47799999999999998,"5min":0.64200000000000002,"15min":0.749},"last":0.439}]},{"osd":3,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36099999999999999,"15min":0.38200000000000001},"min":{"1min":0.318,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41599999999999998,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.376,"15min":0.39800000000000002},"min":{"1min":0.30599999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.69499999999999995},"last":0.40999999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.68600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.433,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.76100000000000001,"5min":0.78400000000000003,"15min":0.879},"last":0.48499999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70699999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.40999999999999998,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.73399999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53000000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77200000000000002,"5min":0.82899999999999996,"15min":0.85299999999999998},"last":0.505},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.59399999999999997},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.65900000000000003,"5min":0.80500000000000005,"15min":0.83599999999999997},"last":0.59099999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.60899999999999999},"min":{"1min":0.438,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.626}]}]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1273856,"data_stored":580534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29699999999999999,"15min":0.29999999999999999},"min":{"1min":0.21299999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.40799999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32100000000000001,"15min":0.312},"min":{"1min":0.23100000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.38400000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.33400000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32300000000000001,"15min":0.313},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.19500000000000001},"max":{"1min":0.46700000000000003,"5min":0.50600000000000001,"15min":0.60399999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":0.44900000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"last":0.41799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.28499999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.53300000000000003,"5min":0.57199999999999995,"15min":0.57599999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.52500000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.499}]},{"osd":4,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.28599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.78600000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.371,"15min":0.314},"max":{"1min":0.79100000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.83799999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.373},"max":{"1min":0.80400000000000005,"5min":1.091,"15min":1.091},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.61199999999999999},"min":{"1min":0.41699999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78400000000000003,"5min":1.111,"15min":1.111},"last":0.71399999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.61199999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74399999999999999,"5min":1.032,"15min":1.236},"last":0.748},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.61899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":580500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.755,"5min":0.997,"15min":0.997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.68700000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.752,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55600000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.51000000000000001},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.44800000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40699999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.85899999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75700000000000001,"5min":0.85999999999999999,"15min":0.879},"last":0.66100000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70599999999999996,"5min":0.78900000000000003,"15min":0.90600000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59599999999999997,"15min":0.623},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.92300000000000004},"last":0.67900000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:26:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.80600000000000005,"15min":0.82499999999999996},"last":0.622},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.89300000000000002},"last":0.63400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574972,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1265664,"data_stored":579048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.69699999999999995,"5min":0.78600000000000003,"15min":0.96899999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.64500000000000002,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.56799999999999995}]},{"osd":1,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.54200000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66700000000000004,"5min":0.72899999999999998,"15min":1.075},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.54800000000000004},"min":{"1min":0.36399999999999999,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.68899999999999995,"5min":0.78000000000000003,"15min":0.95699999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54100000000000004,"15min":0.55700000000000005},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.67600000000000005,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.55500000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.56299999999999994,"15min":0.58099999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73399999999999999,"5min":0.81000000000000005,"15min":1.03},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.35199999999999998},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":1.026},"last":0.443}]},{"osd":4,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.31,"15min":0.27000000000000002},"max":{"1min":0.72799999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.73399999999999999,"5min":0.86399999999999999,"15min":0.91400000000000003},"last":0.51400000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59199999999999997,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.28799999999999998},"max":{"1min":0.77700000000000002,"5min":0.84799999999999998,"15min":0.94199999999999995},"last":0.503}]},{"osd":7,"last update":"Sun Aug 27 11:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.61499999999999999,"15min":0.629},"min":{"1min":0.432,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.63900000000000001,"5min":0.90700000000000003,"15min":0.95899999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.31},"max":{"1min":0.69399999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.48699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:27:01.067 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:27:01.067 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:27:01.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.226+0000 7f3eceb57700 1 Processor -- start 2023-08-27T11:27:01.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.226+0000 7f3eceb57700 1 -- start start 2023-08-27T11:27:01.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.226+0000 7f3eceb57700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3ec81437c0 0x7f3ec80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.226+0000 7f3eceb57700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ec8143ba0 0x7f3ec80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3eceb57700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec80cfd50 con 0x7f3ec806d5b0 2023-08-27T11:27:01.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3eceb57700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec80cf140 con 0x7f3ec81437c0 2023-08-27T11:27:01.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3eceb57700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec80cf420 con 0x7f3ec8143ba0 2023-08-27T11:27:01.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3ecc8f3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3ec81437c0 0x7f3ec80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:01.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3ecd0f4700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3ec806d5b0 0x7f3ec815dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56654/0 (socket says 172.21.15.112:56654) 2023-08-27T11:27:01.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3ecd0f4700 1 -- 172.21.15.112:0/3665401793 learned_addr learned my addr 172.21.15.112:0/3665401793 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:01.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3ec7fff700 1 --2- 172.21.15.112:0/3665401793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ec8143ba0 0x7f3ec80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:01.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3ecc8f3700 1 -- 172.21.15.112:0/3665401793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ec8143ba0 msgr2=0x7f3ec80ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:01.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3ecc8f3700 1 --2- 172.21.15.112:0/3665401793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ec8143ba0 0x7f3ec80ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3ecc8f3700 1 -- 172.21.15.112:0/3665401793 >> v1:172.21.15.112:6789/0 conn(0x7f3ec806d5b0 legacy=0x7f3ec815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3ec77fe700 1 -- 172.21.15.112:0/3665401793 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309838222 0 0) 0x7f3ec80cfd50 con 0x7f3ec806d5b0 2023-08-27T11:27:01.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3ecc8f3700 1 -- 172.21.15.112:0/3665401793 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec80cf700 con 0x7f3ec81437c0 2023-08-27T11:27:01.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3ecc8f3700 1 --2- 172.21.15.112:0/3665401793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec81437c0 0x7f3ec80b4670 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009840 tx=0x7f3ebc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400912da2746494c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:01.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3ec77fe700 1 -- 172.21.15.112:0/3665401793 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc011830 con 0x7f3ec81437c0 2023-08-27T11:27:01.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3ec77fe700 1 -- 172.21.15.112:0/3665401793 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ebc011990 con 0x7f3ec81437c0 2023-08-27T11:27:01.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.227+0000 7f3ec77fe700 1 -- 172.21.15.112:0/3665401793 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc011d20 con 0x7f3ec81437c0 2023-08-27T11:27:01.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.228+0000 7f3eceb57700 1 -- 172.21.15.112:0/3665401793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec81437c0 msgr2=0x7f3ec80b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.228+0000 7f3eceb57700 1 --2- 172.21.15.112:0/3665401793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec81437c0 0x7f3ec80b4670 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009840 tx=0x7f3ebc009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:01.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.228+0000 7f3eceb57700 1 -- 172.21.15.112:0/3665401793 shutdown_connections 2023-08-27T11:27:01.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.228+0000 7f3eceb57700 1 --2- 172.21.15.112:0/3665401793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec81437c0 0x7f3ec80b4670 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.228+0000 7f3eceb57700 1 --2- 172.21.15.112:0/3665401793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ec8143ba0 0x7f3ec80ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.228+0000 7f3eceb57700 1 -- 172.21.15.112:0/3665401793 >> 172.21.15.112:0/3665401793 conn(0x7f3ec80b1e90 msgr2=0x7f3ec80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:01.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.228+0000 7f3eceb57700 1 -- 172.21.15.112:0/3665401793 shutdown_connections 2023-08-27T11:27:01.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.228+0000 7f3eceb57700 1 -- 172.21.15.112:0/3665401793 wait complete. 2023-08-27T11:27:01.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.229+0000 7f3eceb57700 1 Processor -- start 2023-08-27T11:27:01.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.229+0000 7f3eceb57700 1 -- start start 2023-08-27T11:27:01.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3eceb57700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ec806d5b0 0x7f3ec813c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3eceb57700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ec81437c0 0x7f3ec81f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3eceb57700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8143ba0 0x7f3ec815cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3eceb57700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec813d000 con 0x7f3ec8143ba0 2023-08-27T11:27:01.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3eceb57700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec813ccf0 con 0x7f3ec806d5b0 2023-08-27T11:27:01.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3eceb57700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec81f3110 con 0x7f3ec81437c0 2023-08-27T11:27:01.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3ecd0f4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8143ba0 0x7f3ec815cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:01.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3ecd0f4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8143ba0 0x7f3ec815cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34046/0 (socket says 172.21.15.112:34046) 2023-08-27T11:27:01.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3ecd0f4700 1 -- 172.21.15.112:0/3236690632 learned_addr learned my addr 172.21.15.112:0/3236690632 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:01.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3ecd0f4700 1 -- 172.21.15.112:0/3236690632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ec81437c0 msgr2=0x7f3ec81f2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3ec7fff700 1 --2- 172.21.15.112:0/3236690632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ec81437c0 0x7f3ec81f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:01.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3ecd0f4700 1 --2- 172.21.15.112:0/3236690632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ec81437c0 0x7f3ec81f2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3ecd0f4700 1 -- 172.21.15.112:0/3236690632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ec806d5b0 msgr2=0x7f3ec813c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3ecd0f4700 1 --2- 172.21.15.112:0/3236690632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ec806d5b0 0x7f3ec813c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3ecd0f4700 1 -- 172.21.15.112:0/3236690632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc0094a0 con 0x7f3ec8143ba0 2023-08-27T11:27:01.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.230+0000 7f3ecd0f4700 1 --2- 172.21.15.112:0/3236690632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8143ba0 0x7f3ec815cce0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0002230 tx=0x7f3ec000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:01.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.231+0000 7f3ec57fa700 1 -- 172.21.15.112:0/3236690632 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec000ac20 con 0x7f3ec8143ba0 2023-08-27T11:27:01.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.231+0000 7f3ec57fa700 1 -- 172.21.15.112:0/3236690632 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec000ada0 con 0x7f3ec8143ba0 2023-08-27T11:27:01.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.231+0000 7f3eceb57700 1 -- 172.21.15.112:0/3236690632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec81f33b0 con 0x7f3ec8143ba0 2023-08-27T11:27:01.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.231+0000 7f3ec57fa700 1 -- 172.21.15.112:0/3236690632 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0007310 con 0x7f3ec8143ba0 2023-08-27T11:27:01.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.231+0000 7f3eceb57700 1 -- 172.21.15.112:0/3236690632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec81f3970 con 0x7f3ec8143ba0 2023-08-27T11:27:01.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.231+0000 7f3eceb57700 1 -- 172.21.15.112:0/3236690632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec813d140 con 0x7f3ec8143ba0 2023-08-27T11:27:01.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.232+0000 7f3ec57fa700 1 -- 172.21.15.112:0/3236690632 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f3ec0007470 con 0x7f3ec8143ba0 2023-08-27T11:27:01.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.232+0000 7f3ec57fa700 1 --2- 172.21.15.112:0/3236690632 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3eac06a440 0x7f3eac06c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.232+0000 7f3ec57fa700 1 -- 172.21.15.112:0/3236690632 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3ec00889f0 con 0x7f3ec8143ba0 2023-08-27T11:27:01.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.233+0000 7f3ecc8f3700 1 --2- 172.21.15.112:0/3236690632 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3eac06a440 0x7f3eac06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:01.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.233+0000 7f3ecc8f3700 1 --2- 172.21.15.112:0/3236690632 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3eac06a440 0x7f3eac06c900 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0090b0 tx=0x7f3ebc010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:01.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.237+0000 7f3ec57fa700 1 -- 172.21.15.112:0/3236690632 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3ec813d140 con 0x7f3ec8143ba0 2023-08-27T11:27:01.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.504+0000 7f3eceb57700 1 -- 172.21.15.112:0/3236690632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3ec813d140 con 0x7f3ec8143ba0 2023-08-27T11:27:01.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.507+0000 7f3ec57fa700 1 -- 172.21.15.112:0/3236690632 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3ec813d140 con 0x7f3ec8143ba0 2023-08-27T11:27:01.509 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-08-27T11:27:01.509 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA== 2023-08-27T11:27:01.509 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2023-08-27T11:27:01.510 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2023-08-27T11:27:01.510 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:27:01.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.509+0000 7f3eceb57700 1 -- 172.21.15.112:0/3236690632 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3eac06a440 msgr2=0x7f3eac06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.509+0000 7f3eceb57700 1 --2- 172.21.15.112:0/3236690632 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3eac06a440 0x7f3eac06c900 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0090b0 tx=0x7f3ebc010000 comp rx=0 tx=0).stop 2023-08-27T11:27:01.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.509+0000 7f3eceb57700 1 -- 172.21.15.112:0/3236690632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8143ba0 msgr2=0x7f3ec815cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.509+0000 7f3eceb57700 1 --2- 172.21.15.112:0/3236690632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8143ba0 0x7f3ec815cce0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0002230 tx=0x7f3ec000a060 comp rx=0 tx=0).stop 2023-08-27T11:27:01.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.510+0000 7f3eceb57700 1 -- 172.21.15.112:0/3236690632 shutdown_connections 2023-08-27T11:27:01.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.510+0000 7f3eceb57700 1 --2- 172.21.15.112:0/3236690632 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3eac06a440 0x7f3eac06c900 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.510+0000 7f3eceb57700 1 --2- 172.21.15.112:0/3236690632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8143ba0 0x7f3ec815cce0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.510+0000 7f3eceb57700 1 --2- 172.21.15.112:0/3236690632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ec806d5b0 0x7f3ec813c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.510+0000 7f3eceb57700 1 --2- 172.21.15.112:0/3236690632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ec81437c0 0x7f3ec81f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.510+0000 7f3eceb57700 1 -- 172.21.15.112:0/3236690632 >> 172.21.15.112:0/3236690632 conn(0x7f3ec80b1e90 msgr2=0x7f3ec80b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:01.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.510+0000 7f3eceb57700 1 -- 172.21.15.112:0/3236690632 shutdown_connections 2023-08-27T11:27:01.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.510+0000 7f3eceb57700 1 -- 172.21.15.112:0/3236690632 wait complete. 2023-08-27T11:27:01.514 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2023-08-27T11:27:01.523 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:01.557 INFO:tasks.ceph.mds.a.smithi112.stderr:2023-08-27T11:27:01.556+0000 7fb355c82700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:27:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.643+0000 7f39e75c5700 1 Processor -- start 2023-08-27T11:27:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.643+0000 7f39e75c5700 1 -- start start 2023-08-27T11:27:01.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39e75c5700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39e006dc20 0x7f39e00b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39e75c5700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e006f930 con 0x7f39e0160a00 2023-08-27T11:27:01.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39e75c5700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e006fc10 con 0x7f39e0146990 2023-08-27T11:27:01.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39e75c5700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e006fef0 con 0x7f39e006dc20 2023-08-27T11:27:01.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39e5361700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f39e0160a00 0x7f39e015dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56664/0 (socket says 172.21.15.112:56664) 2023-08-27T11:27:01.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39e5361700 1 -- 172.21.15.112:0/2057414358 learned_addr learned my addr 172.21.15.112:0/2057414358 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:01.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39e5b62700 1 --2- 172.21.15.112:0/2057414358 >> v2:172.21.15.138:3300/0 conn(0x7f39e006dc20 0x7f39e00b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:01.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597383832 0 0) 0x7f39e006f930 con 0x7f39e0160a00 2023-08-27T11:27:01.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39cc0035b0 con 0x7f39e0160a00 2023-08-27T11:27:01.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2238965245 0 0) 0x7f39cc0035b0 con 0x7f39e0160a00 2023-08-27T11:27:01.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 >> v2:172.21.15.138:3300/0 conn(0x7f39e006dc20 msgr2=0x7f39e00b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:01.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39d7fff700 1 --2- 172.21.15.112:0/2057414358 >> v2:172.21.15.138:3300/0 conn(0x7f39e006dc20 0x7f39e00b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 >> v1:172.21.15.138:6789/0 conn(0x7f39e0146990 legacy=0x7f39e006d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e00701d0 con 0x7f39e0160a00 2023-08-27T11:27:01.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.644+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f39dc001670 con 0x7f39e0160a00 2023-08-27T11:27:01.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 >> v1:172.21.15.112:6789/0 conn(0x7f39e0160a00 legacy=0x7f39e015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39d7fff700 1 --2- 172.21.15.112:0/2057414358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39cc003a50 0x7f39cc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39d7fff700 1 --2- 172.21.15.112:0/2057414358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39cc004850 0x7f39cc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39d7fff700 1 --2- 172.21.15.112:0/2057414358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39cc005210 0x7f39cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e006fef0 con 0x7f39cc005210 2023-08-27T11:27:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc00c110 con 0x7f39cc004850 2023-08-27T11:27:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc00c2d0 con 0x7f39cc003a50 2023-08-27T11:27:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39e5b62700 1 --2- 172.21.15.112:0/2057414358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39cc005210 0x7f39cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39e4b60700 1 --2- 172.21.15.112:0/2057414358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39cc004850 0x7f39cc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39e5361700 1 --2- 172.21.15.112:0/2057414358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39cc003a50 0x7f39cc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39e5b62700 1 -- 172.21.15.112:0/2057414358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39cc003a50 msgr2=0x7f39cc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39e5b62700 1 --2- 172.21.15.112:0/2057414358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39cc003a50 0x7f39cc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39e5b62700 1 -- 172.21.15.112:0/2057414358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39cc004850 msgr2=0x7f39cc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39e5b62700 1 --2- 172.21.15.112:0/2057414358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39cc004850 0x7f39cc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39e5b62700 1 -- 172.21.15.112:0/2057414358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc0035b0 con 0x7f39cc005210 2023-08-27T11:27:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39e5b62700 1 -- 172.21.15.112:0/2057414358 reap_dead start 2023-08-27T11:27:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39e5b62700 1 --2- 172.21.15.112:0/2057414358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39cc005210 0x7f39cc00ba10 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009fc0 tx=0x7f39cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1877e1e46e9ed server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8003a20 con 0x7f39cc005210 2023-08-27T11:27:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d800f040 con 0x7f39cc005210 2023-08-27T11:27:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.645+0000 7f39d7fff700 1 -- 172.21.15.112:0/2057414358 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8003de0 con 0x7f39cc005210 2023-08-27T11:27:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.646+0000 7f39e75c5700 1 -- 172.21.15.112:0/2057414358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39cc005210 msgr2=0x7f39cc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.646+0000 7f39e75c5700 1 --2- 172.21.15.112:0/2057414358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39cc005210 0x7f39cc00ba10 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009fc0 tx=0x7f39cc005880 comp rx=0 tx=0).stop 2023-08-27T11:27:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.646+0000 7f39e75c5700 1 -- 172.21.15.112:0/2057414358 shutdown_connections 2023-08-27T11:27:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.646+0000 7f39e75c5700 1 --2- 172.21.15.112:0/2057414358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39cc005210 0x7f39cc00ba10 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.646+0000 7f39e75c5700 1 -- 172.21.15.112:0/2057414358 >> 172.21.15.112:0/2057414358 conn(0x7f39e00cfe50 msgr2=0x7f39e00d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.646+0000 7f39e75c5700 1 -- 172.21.15.112:0/2057414358 shutdown_connections 2023-08-27T11:27:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.646+0000 7f39e75c5700 1 -- 172.21.15.112:0/2057414358 wait complete. 2023-08-27T11:27:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.647+0000 7f39e75c5700 1 Processor -- start 2023-08-27T11:27:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.647+0000 7f39e75c5700 1 -- start start 2023-08-27T11:27:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e75c5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39cc005210 0x7f39e015d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e75c5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39e013a480 0x7f39e00b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e75c5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e00b1030 0x7f39e0138780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e75c5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0137c60 con 0x7f39e00b1030 2023-08-27T11:27:01.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e75c5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00b14b0 con 0x7f39e013a480 2023-08-27T11:27:01.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e75c5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e013a900 con 0x7f39cc005210 2023-08-27T11:27:01.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e5b62700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e00b1030 0x7f39e0138780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:01.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e5b62700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e00b1030 0x7f39e0138780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34066/0 (socket says 172.21.15.112:34066) 2023-08-27T11:27:01.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e5b62700 1 -- 172.21.15.112:0/3432669874 learned_addr learned my addr 172.21.15.112:0/3432669874 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:01.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e5361700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39cc005210 0x7f39e015d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:01.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e4b60700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39e013a480 0x7f39e00b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e5b62700 1 -- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39cc005210 msgr2=0x7f39e015d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e5b62700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39cc005210 0x7f39e015d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e5b62700 1 -- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39e013a480 msgr2=0x7f39e00b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e5b62700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39e013a480 0x7f39e00b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e5b62700 1 -- 172.21.15.112:0/3432669874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e01393e0 con 0x7f39e00b1030 2023-08-27T11:27:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.648+0000 7f39e5b62700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e00b1030 0x7f39e0138780 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f39e0070be0 tx=0x7f39d8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.649+0000 7f39d5ffb700 1 -- 172.21.15.112:0/3432669874 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8007000 con 0x7f39e00b1030 2023-08-27T11:27:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.649+0000 7f39d5ffb700 1 -- 172.21.15.112:0/3432669874 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d8003e10 con 0x7f39e00b1030 2023-08-27T11:27:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.649+0000 7f39e75c5700 1 -- 172.21.15.112:0/3432669874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e01396c0 con 0x7f39e00b1030 2023-08-27T11:27:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.649+0000 7f39d5ffb700 1 -- 172.21.15.112:0/3432669874 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8004470 con 0x7f39e00b1030 2023-08-27T11:27:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.649+0000 7f39e75c5700 1 -- 172.21.15.112:0/3432669874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e0139c80 con 0x7f39e00b1030 2023-08-27T11:27:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.650+0000 7f39e75c5700 1 -- 172.21.15.112:0/3432669874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e0067a20 con 0x7f39e00b1030 2023-08-27T11:27:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.650+0000 7f39d5ffb700 1 -- 172.21.15.112:0/3432669874 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f39d80045d0 con 0x7f39e00b1030 2023-08-27T11:27:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.651+0000 7f39d5ffb700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39c406a470 0x7f39c406c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.651+0000 7f39d5ffb700 1 -- 172.21.15.112:0/3432669874 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39d8089c10 con 0x7f39e00b1030 2023-08-27T11:27:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.651+0000 7f39e5361700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39c406a470 0x7f39c406c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.651+0000 7f39e5361700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39c406a470 0x7f39c406c930 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f39dc000c00 tx=0x7f39dc000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.655+0000 7f39d5ffb700 1 -- 172.21.15.112:0/3432669874 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f39d8054e70 con 0x7f39e00b1030 2023-08-27T11:27:01.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.885+0000 7f39e75c5700 1 -- 172.21.15.112:0/3432669874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39e00b4180 con 0x7f39e00b1030 2023-08-27T11:27:01.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.886+0000 7f39d5ffb700 1 -- 172.21.15.112:0/3432669874 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f39d8018090 con 0x7f39e00b1030 2023-08-27T11:27:01.888 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:01.888 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/37643970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":37643970},{"type":"v1","addr":"172.21.15.112:6834","nonce":37643970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/568782348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":568782348},{"type":"v1","addr":"172.21.15.138:6836","nonce":568782348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1867471713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1867471713},{"type":"v1","addr":"172.21.15.138:6837","nonce":1867471713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:26:55.965268+0000","modified":"2023-08-27T11:26:59.589861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3180263819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3180263819},{"type":"v1","addr":"172.21.15.112:6835","nonce":3180263819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-27T11:27:01.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.888+0000 7f39e75c5700 1 -- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39c406a470 msgr2=0x7f39c406c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.888+0000 7f39e75c5700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39c406a470 0x7f39c406c930 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f39dc000c00 tx=0x7f39dc000ed0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.888+0000 7f39e75c5700 1 -- 172.21.15.112:0/3432669874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e00b1030 msgr2=0x7f39e0138780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:01.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.888+0000 7f39e75c5700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e00b1030 0x7f39e0138780 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f39e0070be0 tx=0x7f39d8009750 comp rx=0 tx=0).stop 2023-08-27T11:27:01.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.888+0000 7f39e75c5700 1 -- 172.21.15.112:0/3432669874 shutdown_connections 2023-08-27T11:27:01.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.888+0000 7f39e75c5700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f39c406a470 0x7f39c406c930 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.888+0000 7f39e75c5700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e00b1030 0x7f39e0138780 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.888+0000 7f39e75c5700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39cc005210 0x7f39e015d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.888+0000 7f39e75c5700 1 --2- 172.21.15.112:0/3432669874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39e013a480 0x7f39e00b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:01.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.888+0000 7f39e75c5700 1 -- 172.21.15.112:0/3432669874 >> 172.21.15.112:0/3432669874 conn(0x7f39e00cfe50 msgr2=0x7f39e00b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:01.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.888+0000 7f39e75c5700 1 -- 172.21.15.112:0/3432669874 shutdown_connections 2023-08-27T11:27:01.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:01.888+0000 7f39e75c5700 1 -- 172.21.15.112:0/3432669874 wait complete. 2023-08-27T11:27:01.892 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 67 2023-08-27T11:27:02.305 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:26:55.965268+0000', 'modified': '2023-08-27T11:26:59.589861+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7518}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7518': {'gid': 7518, 'name': 'a', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/3180263819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 3180263819}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3180263819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:27:02.305 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:27:02.306 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-27T11:27:02.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.427+0000 7fb944707700 1 Processor -- start 2023-08-27T11:27:02.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.427+0000 7fb944707700 1 -- start start 2023-08-27T11:27:02.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb944707700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb93c141420 0x7fb93c0aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb944707700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb93c0a9970 0x7fb93c06e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb944707700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb93c0ab440 con 0x7fb93c06e150 2023-08-27T11:27:02.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb944707700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb93c06eb40 con 0x7fb93c0a9970 2023-08-27T11:27:02.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb944707700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb93c15eed0 con 0x7fb93c141420 2023-08-27T11:27:02.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb942ca4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb93c0a9970 0x7fb93c06e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb942ca4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb93c0a9970 0x7fb93c06e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34074/0 (socket says 172.21.15.112:34074) 2023-08-27T11:27:02.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb942ca4700 1 -- 172.21.15.112:0/286590139 learned_addr learned my addr 172.21.15.112:0/286590139 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:02.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb942ca4700 1 -- 172.21.15.112:0/286590139 >> v2:172.21.15.138:3300/0 conn(0x7fb93c141420 msgr2=0x7fb93c0aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:02.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb942ca4700 1 --2- 172.21.15.112:0/286590139 >> v2:172.21.15.138:3300/0 conn(0x7fb93c141420 0x7fb93c0aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb942ca4700 1 -- 172.21.15.112:0/286590139 >> v1:172.21.15.138:6789/0 conn(0x7fb93c06e150 legacy=0x7fb93c0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:02.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb942ca4700 1 -- 172.21.15.112:0/286590139 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c15f1b0 con 0x7fb93c0a9970 2023-08-27T11:27:02.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb942ca4700 1 --2- 172.21.15.112:0/286590139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb93c0a9970 0x7fb93c06e600 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb93c06ed40 tx=0x7fb93400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efb7f4fe6d98107 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:02.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb9414a1700 1 -- 172.21.15.112:0/286590139 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93400e070 con 0x7fb93c0a9970 2023-08-27T11:27:02.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb9414a1700 1 -- 172.21.15.112:0/286590139 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb934003a20 con 0x7fb93c0a9970 2023-08-27T11:27:02.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.428+0000 7fb9414a1700 1 -- 172.21.15.112:0/286590139 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934003dd0 con 0x7fb93c0a9970 2023-08-27T11:27:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.429+0000 7fb944707700 1 -- 172.21.15.112:0/286590139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb93c0a9970 msgr2=0x7fb93c06e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.429+0000 7fb944707700 1 --2- 172.21.15.112:0/286590139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb93c0a9970 0x7fb93c06e600 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb93c06ed40 tx=0x7fb93400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.429+0000 7fb944707700 1 -- 172.21.15.112:0/286590139 shutdown_connections 2023-08-27T11:27:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.429+0000 7fb944707700 1 --2- 172.21.15.112:0/286590139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb93c0a9970 0x7fb93c06e600 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.429+0000 7fb944707700 1 --2- 172.21.15.112:0/286590139 >> v2:172.21.15.138:3300/0 conn(0x7fb93c141420 0x7fb93c0aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.429+0000 7fb944707700 1 -- 172.21.15.112:0/286590139 >> 172.21.15.112:0/286590139 conn(0x7fb93c06cd30 msgr2=0x7fb93c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.429+0000 7fb944707700 1 -- 172.21.15.112:0/286590139 shutdown_connections 2023-08-27T11:27:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.429+0000 7fb944707700 1 -- 172.21.15.112:0/286590139 wait complete. 2023-08-27T11:27:02.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.430+0000 7fb944707700 1 Processor -- start 2023-08-27T11:27:02.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.431+0000 7fb944707700 1 -- start start 2023-08-27T11:27:02.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.431+0000 7fb944707700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb93c06e150 0x7fb93c158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.431+0000 7fb944707700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb93c0a9970 0x7fb93c1583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb944707700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb93c141420 0x7fb93c13c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb944707700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c158910 con 0x7fb93c0a9970 2023-08-27T11:27:02.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb944707700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c13b530 con 0x7fb93c06e150 2023-08-27T11:27:02.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb944707700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c13b810 con 0x7fb93c141420 2023-08-27T11:27:02.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb941ca2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb93c0a9970 0x7fb93c1583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb941ca2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb93c0a9970 0x7fb93c1583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34078/0 (socket says 172.21.15.112:34078) 2023-08-27T11:27:02.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb941ca2700 1 -- 172.21.15.112:0/1061168684 learned_addr learned my addr 172.21.15.112:0/1061168684 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:02.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb9424a3700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb93c06e150 0x7fb93c158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb942ca4700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb93c141420 0x7fb93c13c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb941ca2700 1 -- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb93c141420 msgr2=0x7fb93c13c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb941ca2700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb93c141420 0x7fb93c13c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb941ca2700 1 -- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb93c06e150 msgr2=0x7fb93c158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb941ca2700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb93c06e150 0x7fb93c158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb941ca2700 1 -- 172.21.15.112:0/1061168684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb934009b30 con 0x7fb93c0a9970 2023-08-27T11:27:02.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb9424a3700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb93c06e150 0x7fb93c158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:02.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb941ca2700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb93c0a9970 0x7fb93c1583d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00c520 tx=0x7fb92c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:02.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb932ffd700 1 -- 172.21.15.112:0/1061168684 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0054e0 con 0x7fb93c0a9970 2023-08-27T11:27:02.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb932ffd700 1 -- 172.21.15.112:0/1061168684 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92c0036d0 con 0x7fb93c0a9970 2023-08-27T11:27:02.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb944707700 1 -- 172.21.15.112:0/1061168684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93c13baf0 con 0x7fb93c0a9970 2023-08-27T11:27:02.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb932ffd700 1 -- 172.21.15.112:0/1061168684 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c003a80 con 0x7fb93c0a9970 2023-08-27T11:27:02.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.432+0000 7fb944707700 1 -- 172.21.15.112:0/1061168684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c13c0b0 con 0x7fb93c0a9970 2023-08-27T11:27:02.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.434+0000 7fb932ffd700 1 -- 172.21.15.112:0/1061168684 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fb92c003be0 con 0x7fb93c0a9970 2023-08-27T11:27:02.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.434+0000 7fb932ffd700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb92006a440 0x7fb92006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.434+0000 7fb932ffd700 1 -- 172.21.15.112:0/1061168684 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb92c087f30 con 0x7fb93c0a9970 2023-08-27T11:27:02.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.434+0000 7fb9424a3700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb92006a440 0x7fb92006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.435+0000 7fb9424a3700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb92006a440 0x7fb92006c900 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb938006fd0 tx=0x7fb938008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.436+0000 7fb9508c9b80 1 Processor -- start 2023-08-27T11:27:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.436+0000 7fb9508c9b80 1 -- start start 2023-08-27T11:27:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.437+0000 7fb9508c9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31266320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.437+0000 7fb9508c9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31277960 0x559d31262b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.437+0000 7fb9508c9b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d312393d0 0x559d312630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.437+0000 7fb9508c9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d31194730 con 0x559d31265f40 2023-08-27T11:27:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.437+0000 7fb9508c9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d31192050 con 0x559d31277960 2023-08-27T11:27:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.437+0000 7fb9508c9b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d3118d940 con 0x559d312393d0 2023-08-27T11:27:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.437+0000 7fb9424a3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31266320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.437+0000 7fb9424a3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31266320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34084/0 (socket says 172.21.15.112:34084) 2023-08-27T11:27:02.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.437+0000 7fb9424a3700 1 -- 172.21.15.112:0/874977345 learned_addr learned my addr 172.21.15.112:0/874977345 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:02.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.437+0000 7fb941ca2700 1 --2- 172.21.15.112:0/874977345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31277960 0x559d31262b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.437+0000 7fb9424a3700 1 -- 172.21.15.112:0/874977345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d312393d0 msgr2=0x559d312630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:02.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.437+0000 7fb942ca4700 1 --2- 172.21.15.112:0/874977345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d312393d0 0x559d312630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb9424a3700 1 --2- 172.21.15.112:0/874977345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d312393d0 0x559d312630b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb9424a3700 1 -- 172.21.15.112:0/874977345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31277960 msgr2=0x559d31262b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:02.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb9424a3700 1 --2- 172.21.15.112:0/874977345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31277960 0x559d31262b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb9424a3700 1 -- 172.21.15.112:0/874977345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3100d5e0 con 0x559d31265f40 2023-08-27T11:27:02.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb941ca2700 1 --2- 172.21.15.112:0/874977345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31277960 0x559d31262b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:02.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb9424a3700 1 --2- 172.21.15.112:0/874977345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31266320 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb93800d780 tx=0x7fb938010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0399ed86d53a1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:02.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb91ffff700 1 -- 172.21.15.112:0/874977345 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938014640 con 0x559d31265f40 2023-08-27T11:27:02.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb91ffff700 1 -- 172.21.15.112:0/874977345 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9380147a0 con 0x559d31265f40 2023-08-27T11:27:02.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb91ffff700 1 -- 172.21.15.112:0/874977345 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938014b30 con 0x559d31265f40 2023-08-27T11:27:02.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb9508c9b80 1 -- 172.21.15.112:0/874977345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 msgr2=0x559d31266320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/874977345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31266320 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb93800d780 tx=0x7fb938010000 comp rx=0 tx=0).stop 2023-08-27T11:27:02.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb9508c9b80 1 -- 172.21.15.112:0/874977345 shutdown_connections 2023-08-27T11:27:02.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/874977345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d312393d0 0x559d312630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/874977345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31266320 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.438+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/874977345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31277960 0x559d31262b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.439+0000 7fb9508c9b80 1 -- 172.21.15.112:0/874977345 >> 172.21.15.112:0/874977345 conn(0x559d31199c30 msgr2=0x559d31280d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:02.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.439+0000 7fb9508c9b80 1 -- 172.21.15.112:0/874977345 shutdown_connections 2023-08-27T11:27:02.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.439+0000 7fb9508c9b80 1 -- 172.21.15.112:0/874977345 wait complete. 2023-08-27T11:27:02.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.439+0000 7fb9508c9b80 1 -- start start 2023-08-27T11:27:02.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.439+0000 7fb9508c9b80 1 Processor -- start 2023-08-27T11:27:02.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb9508c9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d312393d0 0x559d31266f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb9508c9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d3127fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb9508c9b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d31277960 0x559d31282f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb9508c9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3112bb60 con 0x559d31265f40 2023-08-27T11:27:02.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb9508c9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3112b860 con 0x559d312393d0 2023-08-27T11:27:02.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb9508c9b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d3100f8d0 con 0x559d31277960 2023-08-27T11:27:02.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb9508c9b80 10 client.0 _ll_register_callbacks cb 0x559d31129630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:27:02.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb941ca2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d3127fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb941ca2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d3127fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34092/0 (socket says 172.21.15.112:34092) 2023-08-27T11:27:02.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb941ca2700 1 -- 172.21.15.112:0/1893289984 learned_addr learned my addr 172.21.15.112:0/1893289984 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:02.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb942ca4700 1 --2- 172.21.15.112:0/1893289984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d31277960 0x559d31282f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb9424a3700 1 --2- 172.21.15.112:0/1893289984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d312393d0 0x559d31266f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb941ca2700 1 -- 172.21.15.112:0/1893289984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d31277960 msgr2=0x559d31282f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:02.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb941ca2700 1 --2- 172.21.15.112:0/1893289984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d31277960 0x559d31282f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb941ca2700 1 -- 172.21.15.112:0/1893289984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d312393d0 msgr2=0x559d31266f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:02.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.440+0000 7fb941ca2700 1 --2- 172.21.15.112:0/1893289984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d312393d0 0x559d31266f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb941ca2700 1 -- 172.21.15.112:0/1893289984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93800e040 con 0x559d31265f40 2023-08-27T11:27:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb941ca2700 1 --2- 172.21.15.112:0/1893289984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d3127fb10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb92c053460 tx=0x7fb92c056730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4fd7159e24cf1b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb91d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:27:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb91d7fa700 1 -- 172.21.15.112:0/1893289984 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c05ec70 con 0x559d31265f40 2023-08-27T11:27:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1893289984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d3122e4f0 con 0x559d31265f40 2023-08-27T11:27:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb91d7fa700 1 -- 172.21.15.112:0/1893289984 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb92c055ec0 con 0x559d31265f40 2023-08-27T11:27:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb91d7fa700 1 -- 172.21.15.112:0/1893289984 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb92c056080 con 0x559d31265f40 2023-08-27T11:27:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb9508c9b80 10 client.7674 fetch_fsmap learned FSMap version 67 2023-08-27T11:27:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1893289984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x559d311a5750 con 0x559d31265f40 2023-08-27T11:27:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb91d7fa700 1 -- 172.21.15.112:0/1893289984 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559d3122e4f0 con 0x559d31265f40 2023-08-27T11:27:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb91d7fa700 1 -- 172.21.15.112:0/1893289984 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb92c086030 con 0x559d31265f40 2023-08-27T11:27:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb9508c9b80 10 client.7674 fetch_fsmap finished waiting for FSMap version 67 2023-08-27T11:27:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb9508c9b80 10 client.7674 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-27T11:27:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb9508c9b80 20 client.7674 populate_metadata read hostname 'smithi112' 2023-08-27T11:27:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1893289984 >> [v2:172.21.15.112:6833/3180263819,v1:172.21.15.112:6835/3180263819] conn(0x559d31282b90 0x559d312834d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb9508c9b80 4 client.7674 mds_command: new command op to 7518 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:27:02.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1893289984 --> [v2:172.21.15.112:6833/3180263819,v1:172.21.15.112:6835/3180263819] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d31186920 con 0x559d31282b90 2023-08-27T11:27:02.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.441+0000 7fb9424a3700 1 --2- 172.21.15.112:0/1893289984 >> [v2:172.21.15.112:6833/3180263819,v1:172.21.15.112:6835/3180263819] conn(0x559d31282b90 0x559d312834d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb91d7fa700 10 client.7674 ms_handle_connect on v2:172.21.15.112:6833/3180263819 2023-08-27T11:27:02.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb9424a3700 1 --2- 172.21.15.112:0/1893289984 >> [v2:172.21.15.112:6833/3180263819,v1:172.21.15.112:6835/3180263819] conn(0x559d31282b90 0x559d312834d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32b2e27eb19ceb5e server_cookie=4533bfeffc2e98f5 in_seq=0 out_seq=0 2023-08-27T11:27:02.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb91d7fa700 1 -- 172.21.15.112:0/1893289984 <== mds.0 v2:172.21.15.112:6833/3180263819 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559d31186920 con 0x559d31282b90 2023-08-27T11:27:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb91d7fa700 10 client.7674 handle_command_reply: tid=0 2023-08-27T11:27:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb9508c9b80 1 client.7674 shutdown 2023-08-27T11:27:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1893289984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 msgr2=0x559d3127fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1893289984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d3127fb10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb92c053460 tx=0x7fb92c056730 comp rx=0 tx=0).stop 2023-08-27T11:27:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1893289984 shutdown_connections 2023-08-27T11:27:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1893289984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d31277960 0x559d31282f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1893289984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d312393d0 0x559d31266f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1893289984 >> [v2:172.21.15.112:6833/3180263819,v1:172.21.15.112:6835/3180263819] conn(0x559d31282b90 0x559d312834d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1893289984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d3127fb10 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb91d7fa700 0 client.7674 ms_handle_reset on v2:172.21.15.112:6833/3180263819 2023-08-27T11:27:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1893289984 >> 172.21.15.112:0/1893289984 conn(0x559d31199c30 msgr2=0x559d3123b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1893289984 shutdown_connections 2023-08-27T11:27:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.442+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1893289984 wait complete. 2023-08-27T11:27:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.443+0000 7fb9508c9b80 20 client.7674 trim_cache size 0 max 16384 2023-08-27T11:27:02.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.460+0000 7fb9508c9b80 1 Processor -- start 2023-08-27T11:27:02.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.460+0000 7fb9508c9b80 1 -- start start 2023-08-27T11:27:02.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb9508c9b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d312393d0 0x559d3123cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb9508c9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31266510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb9508c9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31277960 0x559d31268b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb9508c9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d31239200 con 0x559d31265f40 2023-08-27T11:27:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb9508c9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3123bfb0 con 0x559d31277960 2023-08-27T11:27:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb9508c9b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d31267930 con 0x559d312393d0 2023-08-27T11:27:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb941ca2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31266510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb941ca2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31266510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34100/0 (socket says 172.21.15.112:34100) 2023-08-27T11:27:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb941ca2700 1 -- 172.21.15.112:0/1172698126 learned_addr learned my addr 172.21.15.112:0/1172698126 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb942ca4700 1 --2- 172.21.15.112:0/1172698126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31277960 0x559d31268b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb9424a3700 1 --2- 172.21.15.112:0/1172698126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d312393d0 0x559d3123cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb941ca2700 1 -- 172.21.15.112:0/1172698126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d312393d0 msgr2=0x559d3123cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb941ca2700 1 --2- 172.21.15.112:0/1172698126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d312393d0 0x559d3123cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb941ca2700 1 -- 172.21.15.112:0/1172698126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31277960 msgr2=0x559d31268b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb941ca2700 1 --2- 172.21.15.112:0/1172698126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31277960 0x559d31268b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.461+0000 7fb941ca2700 1 -- 172.21.15.112:0/1172698126 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d30fb9d90 con 0x559d31265f40 2023-08-27T11:27:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.462+0000 7fb941ca2700 1 --2- 172.21.15.112:0/1172698126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31266510 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb92c056850 tx=0x7fb92c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b34926e645e55c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.462+0000 7fb930ff9700 1 -- 172.21.15.112:0/1172698126 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c05ec70 con 0x559d31265f40 2023-08-27T11:27:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.462+0000 7fb930ff9700 1 -- 172.21.15.112:0/1172698126 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92c00e040 con 0x559d31265f40 2023-08-27T11:27:02.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.462+0000 7fb930ff9700 1 -- 172.21.15.112:0/1172698126 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0560c0 con 0x559d31265f40 2023-08-27T11:27:02.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.462+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1172698126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 msgr2=0x559d31266510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.462+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1172698126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31266510 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb92c056850 tx=0x7fb92c014000 comp rx=0 tx=0).stop 2023-08-27T11:27:02.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.462+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1172698126 shutdown_connections 2023-08-27T11:27:02.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.462+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1172698126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31277960 0x559d31268b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.462+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1172698126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d312393d0 0x559d3123cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.462+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1172698126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31266510 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.462+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1172698126 >> 172.21.15.112:0/1172698126 conn(0x559d31199c30 msgr2=0x559d3127df90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.462+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1172698126 shutdown_connections 2023-08-27T11:27:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.462+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1172698126 wait complete. 2023-08-27T11:27:02.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.463+0000 7fb9508c9b80 1 -- start start 2023-08-27T11:27:02.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.463+0000 7fb9508c9b80 1 Processor -- start 2023-08-27T11:27:02.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb9508c9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d312393d0 0x559d31280ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb9508c9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31299610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb9508c9b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d31277960 0x559d31299b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb9508c9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d31269c10 con 0x559d31265f40 2023-08-27T11:27:02.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb9508c9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d31281880 con 0x559d312393d0 2023-08-27T11:27:02.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb9508c9b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d31266bd0 con 0x559d31277960 2023-08-27T11:27:02.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb9508c9b80 10 client.0 _ll_register_callbacks cb 0x559d31129630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:27:02.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb941ca2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31299610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb941ca2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31299610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34106/0 (socket says 172.21.15.112:34106) 2023-08-27T11:27:02.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb941ca2700 1 -- 172.21.15.112:0/1145239665 learned_addr learned my addr 172.21.15.112:0/1145239665 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:02.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb941ca2700 1 -- 172.21.15.112:0/1145239665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d31277960 msgr2=0x559d31299b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:02.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb941ca2700 1 --2- 172.21.15.112:0/1145239665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d31277960 0x559d31299b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb941ca2700 1 -- 172.21.15.112:0/1145239665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d312393d0 msgr2=0x559d31280ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:02.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb941ca2700 1 --2- 172.21.15.112:0/1145239665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d312393d0 0x559d31280ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb941ca2700 1 -- 172.21.15.112:0/1145239665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d311a5750 con 0x559d31265f40 2023-08-27T11:27:02.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb941ca2700 1 --2- 172.21.15.112:0/1145239665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31299610 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0056c0 tx=0x7fb92c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb09ad3e04759af7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:02.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb91d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:27:02.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb91d7fa700 1 -- 172.21.15.112:0/1145239665 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c00e040 con 0x559d31265f40 2023-08-27T11:27:02.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb91d7fa700 1 -- 172.21.15.112:0/1145239665 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb92c0560d0 con 0x559d31265f40 2023-08-27T11:27:02.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.464+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1145239665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d31147000 con 0x559d31265f40 2023-08-27T11:27:02.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb91d7fa700 1 -- 172.21.15.112:0/1145239665 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb92c056290 con 0x559d31265f40 2023-08-27T11:27:02.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb9508c9b80 10 client.7680 fetch_fsmap learned FSMap version 67 2023-08-27T11:27:02.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1145239665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x559d30fa2c10 con 0x559d31265f40 2023-08-27T11:27:02.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb91d7fa700 1 -- 172.21.15.112:0/1145239665 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559d31147000 con 0x559d31265f40 2023-08-27T11:27:02.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb91d7fa700 1 -- 172.21.15.112:0/1145239665 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb92c056f60 con 0x559d31265f40 2023-08-27T11:27:02.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb9508c9b80 10 client.7680 fetch_fsmap finished waiting for FSMap version 67 2023-08-27T11:27:02.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb9508c9b80 10 client.7680 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-27T11:27:02.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb9508c9b80 20 client.7680 populate_metadata read hostname 'smithi112' 2023-08-27T11:27:02.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1145239665 >> [v2:172.21.15.112:6833/3180263819,v1:172.21.15.112:6835/3180263819] conn(0x559d31269fb0 0x559d31282f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb9508c9b80 4 client.7680 mds_command: new command op to 7518 tid=0[{"prefix": "status"}] 2023-08-27T11:27:02.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1145239665 --> [v2:172.21.15.112:6833/3180263819,v1:172.21.15.112:6835/3180263819] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559d30fdaf20 con 0x559d31269fb0 2023-08-27T11:27:02.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb9424a3700 1 --2- 172.21.15.112:0/1145239665 >> [v2:172.21.15.112:6833/3180263819,v1:172.21.15.112:6835/3180263819] conn(0x559d31269fb0 0x559d31282f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb91d7fa700 10 client.7680 ms_handle_connect on v2:172.21.15.112:6833/3180263819 2023-08-27T11:27:02.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb9424a3700 1 --2- 172.21.15.112:0/1145239665 >> [v2:172.21.15.112:6833/3180263819,v1:172.21.15.112:6835/3180263819] conn(0x559d31269fb0 0x559d31282f70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49431226ffae8605 server_cookie=ccadd5dc1eb2c808 in_seq=0 out_seq=0 2023-08-27T11:27:02.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb91d7fa700 1 -- 172.21.15.112:0/1145239665 <== mds.0 v2:172.21.15.112:6833/3180263819 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x559d30fdaf20 con 0x559d31269fb0 2023-08-27T11:27:02.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb91d7fa700 10 client.7680 handle_command_reply: tid=0 2023-08-27T11:27:02.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.465+0000 7fb9508c9b80 1 client.7680 shutdown 2023-08-27T11:27:02.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1145239665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 msgr2=0x559d31299610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1145239665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31299610 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0056c0 tx=0x7fb92c001070 comp rx=0 tx=0).stop 2023-08-27T11:27:02.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1145239665 shutdown_connections 2023-08-27T11:27:02.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1145239665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559d31277960 0x559d31299b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1145239665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d312393d0 0x559d31280ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1145239665 >> [v2:172.21.15.112:6833/3180263819,v1:172.21.15.112:6835/3180263819] conn(0x559d31269fb0 0x559d31282f70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb9508c9b80 1 --2- 172.21.15.112:0/1145239665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d31265f40 0x559d31299610 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb91d7fa700 0 client.7680 ms_handle_reset on v2:172.21.15.112:6833/3180263819 2023-08-27T11:27:02.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1145239665 >> 172.21.15.112:0/1145239665 conn(0x559d31199c30 msgr2=0x559d3127f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:02.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1145239665 shutdown_connections 2023-08-27T11:27:02.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb9508c9b80 1 -- 172.21.15.112:0/1145239665 wait complete. 2023-08-27T11:27:02.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb9508c9b80 20 client.7680 trim_cache size 0 max 16384 2023-08-27T11:27:02.491 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:27:02.491 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e099a24f-c1cd-4f11-ac4f-ecacc10e5e98", 2023-08-27T11:27:02.492 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2023-08-27T11:27:02.492 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 7518, 2023-08-27T11:27:02.492 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2023-08-27T11:27:02.492 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2023-08-27T11:27:02.492 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2023-08-27T11:27:02.493 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 5.89857298, 2023-08-27T11:27:02.493 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 67, 2023-08-27T11:27:02.493 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 102, 2023-08-27T11:27:02.493 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 102, 2023-08-27T11:27:02.494 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 11.839173833 2023-08-27T11:27:02.494 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:27:02.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb944707700 1 -- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb92006a440 msgr2=0x7fb92006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb944707700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb92006a440 0x7fb92006c900 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb938006fd0 tx=0x7fb938008040 comp rx=0 tx=0).stop 2023-08-27T11:27:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb944707700 1 -- 172.21.15.112:0/1061168684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb93c0a9970 msgr2=0x7fb93c1583d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb944707700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb93c0a9970 0x7fb93c1583d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00c520 tx=0x7fb92c00c8e0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb944707700 1 -- 172.21.15.112:0/1061168684 shutdown_connections 2023-08-27T11:27:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb944707700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb92006a440 0x7fb92006c900 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb944707700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb93c141420 0x7fb93c13c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb944707700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb93c06e150 0x7fb93c158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb944707700 1 --2- 172.21.15.112:0/1061168684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb93c0a9970 0x7fb93c1583d0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb944707700 1 -- 172.21.15.112:0/1061168684 >> 172.21.15.112:0/1061168684 conn(0x7fb93c06cd30 msgr2=0x7fb93c15df80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb944707700 1 -- 172.21.15.112:0/1061168684 shutdown_connections 2023-08-27T11:27:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.466+0000 7fb944707700 1 -- 172.21.15.112:0/1061168684 wait complete. 2023-08-27T11:27:02.498 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:27:02.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.617+0000 7ffb50037700 1 Processor -- start 2023-08-27T11:27:02.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.617+0000 7ffb50037700 1 -- start start 2023-08-27T11:27:02.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.618+0000 7ffb50037700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb480ce220 0x7ffb480cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.618+0000 7ffb50037700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb480cc620 0x7ffb480cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.618+0000 7ffb50037700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb480ccab0 con 0x7ffb48152540 2023-08-27T11:27:02.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.618+0000 7ffb50037700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4815eee0 con 0x7ffb480cc620 2023-08-27T11:27:02.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.618+0000 7ffb50037700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4815f1c0 con 0x7ffb480ce220 2023-08-27T11:27:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.618+0000 7ffb4e5d4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb480cc620 0x7ffb480cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.618+0000 7ffb4ddd3700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ffb48152540 0x7ffb480cdb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56726/0 (socket says 172.21.15.112:56726) 2023-08-27T11:27:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.618+0000 7ffb4ddd3700 1 -- 172.21.15.112:0/1206092361 learned_addr learned my addr 172.21.15.112:0/1206092361 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.619+0000 7ffb4d5d2700 1 --2- 172.21.15.112:0/1206092361 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb480ce220 0x7ffb480cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.619+0000 7ffb4e5d4700 1 -- 172.21.15.112:0/1206092361 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb480ce220 msgr2=0x7ffb480cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.619+0000 7ffb4e5d4700 1 --2- 172.21.15.112:0/1206092361 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb480ce220 0x7ffb480cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.619+0000 7ffb4e5d4700 1 -- 172.21.15.112:0/1206092361 >> v1:172.21.15.112:6789/0 conn(0x7ffb48152540 legacy=0x7ffb480cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.619+0000 7ffb4e5d4700 1 -- 172.21.15.112:0/1206092361 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4815f4a0 con 0x7ffb480cc620 2023-08-27T11:27:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.619+0000 7ffb4cdd1700 1 -- 172.21.15.112:0/1206092361 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394709239 0 0) 0x7ffb480ccab0 con 0x7ffb48152540 2023-08-27T11:27:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.619+0000 7ffb4e5d4700 1 --2- 172.21.15.112:0/1206092361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480cc620 0x7ffb480cf700 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009d90 tx=0x7ffb4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d1c45da61ef38f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.619+0000 7ffb4cdd1700 1 -- 172.21.15.112:0/1206092361 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4000e070 con 0x7ffb480cc620 2023-08-27T11:27:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.619+0000 7ffb4cdd1700 1 -- 172.21.15.112:0/1206092361 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4000bbd0 con 0x7ffb480cc620 2023-08-27T11:27:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.619+0000 7ffb4cdd1700 1 -- 172.21.15.112:0/1206092361 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb40012430 con 0x7ffb480cc620 2023-08-27T11:27:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.620+0000 7ffb50037700 1 -- 172.21.15.112:0/1206092361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480cc620 msgr2=0x7ffb480cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.620+0000 7ffb50037700 1 --2- 172.21.15.112:0/1206092361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480cc620 0x7ffb480cf700 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009d90 tx=0x7ffb4000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:02.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.620+0000 7ffb50037700 1 -- 172.21.15.112:0/1206092361 shutdown_connections 2023-08-27T11:27:02.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.620+0000 7ffb50037700 1 --2- 172.21.15.112:0/1206092361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480cc620 0x7ffb480cf700 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.620+0000 7ffb50037700 1 --2- 172.21.15.112:0/1206092361 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb480ce220 0x7ffb480cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.620+0000 7ffb50037700 1 -- 172.21.15.112:0/1206092361 >> 172.21.15.112:0/1206092361 conn(0x7ffb4806f970 msgr2=0x7ffb4806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:02.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.620+0000 7ffb50037700 1 -- 172.21.15.112:0/1206092361 shutdown_connections 2023-08-27T11:27:02.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.620+0000 7ffb50037700 1 -- 172.21.15.112:0/1206092361 wait complete. 2023-08-27T11:27:02.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.621+0000 7ffb50037700 1 Processor -- start 2023-08-27T11:27:02.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.621+0000 7ffb50037700 1 -- start start 2023-08-27T11:27:02.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb50037700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb480cc620 0x7ffb481f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb50037700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480ce220 0x7ffb480b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb50037700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb48152540 0x7ffb480b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb50037700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb48070a90 con 0x7ffb480ce220 2023-08-27T11:27:02.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb50037700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb48071330 con 0x7ffb48152540 2023-08-27T11:27:02.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb50037700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb481f4100 con 0x7ffb480cc620 2023-08-27T11:27:02.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb4d5d2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480ce220 0x7ffb480b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb4d5d2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480ce220 0x7ffb480b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34118/0 (socket says 172.21.15.112:34118) 2023-08-27T11:27:02.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb4d5d2700 1 -- 172.21.15.112:0/3739061256 learned_addr learned my addr 172.21.15.112:0/3739061256 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:02.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb4e5d4700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb48152540 0x7ffb480b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb4ddd3700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb480cc620 0x7ffb481f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb4d5d2700 1 -- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb480cc620 msgr2=0x7ffb481f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb4d5d2700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb480cc620 0x7ffb481f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb4d5d2700 1 -- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb48152540 msgr2=0x7ffb480b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb4d5d2700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb48152540 0x7ffb480b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb4d5d2700 1 -- 172.21.15.112:0/3739061256 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb40009a70 con 0x7ffb480ce220 2023-08-27T11:27:02.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb4d5d2700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480ce220 0x7ffb480b0950 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00e840 tx=0x7ffb3c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:02.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb467fc700 1 -- 172.21.15.112:0/3739061256 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c013610 con 0x7ffb480ce220 2023-08-27T11:27:02.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb467fc700 1 -- 172.21.15.112:0/3739061256 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c013770 con 0x7ffb480ce220 2023-08-27T11:27:02.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.622+0000 7ffb50037700 1 -- 172.21.15.112:0/3739061256 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb48158450 con 0x7ffb480ce220 2023-08-27T11:27:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.623+0000 7ffb467fc700 1 -- 172.21.15.112:0/3739061256 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c013b00 con 0x7ffb480ce220 2023-08-27T11:27:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.623+0000 7ffb50037700 1 -- 172.21.15.112:0/3739061256 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb48070fc0 con 0x7ffb480ce220 2023-08-27T11:27:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.623+0000 7ffb50037700 1 -- 172.21.15.112:0/3739061256 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb480714a0 con 0x7ffb480ce220 2023-08-27T11:27:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.624+0000 7ffb467fc700 1 -- 172.21.15.112:0/3739061256 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7ffb3c013c60 con 0x7ffb480ce220 2023-08-27T11:27:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.624+0000 7ffb467fc700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb3406a3b0 0x7ffb3406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.624+0000 7ffb467fc700 1 -- 172.21.15.112:0/3739061256 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb3c088740 con 0x7ffb480ce220 2023-08-27T11:27:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.624+0000 7ffb4ddd3700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb3406a3b0 0x7ffb3406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.625+0000 7ffb4ddd3700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb3406a3b0 0x7ffb3406c870 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffb38000f60 tx=0x7ffb38007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.629+0000 7ffb467fc700 1 -- 172.21.15.112:0/3739061256 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffb3c057480 con 0x7ffb480ce220 2023-08-27T11:27:02.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.861+0000 7ffb50037700 1 -- 172.21.15.112:0/3739061256 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffb48067a20 con 0x7ffb480ce220 2023-08-27T11:27:02.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.862+0000 7ffb467fc700 1 -- 172.21.15.112:0/3739061256 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffb3c057010 con 0x7ffb480ce220 2023-08-27T11:27:02.864 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:27:02.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.864+0000 7ffb50037700 1 -- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb3406a3b0 msgr2=0x7ffb3406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.864+0000 7ffb50037700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb3406a3b0 0x7ffb3406c870 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffb38000f60 tx=0x7ffb38007c90 comp rx=0 tx=0).stop 2023-08-27T11:27:02.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.864+0000 7ffb50037700 1 -- 172.21.15.112:0/3739061256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480ce220 msgr2=0x7ffb480b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.864+0000 7ffb50037700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480ce220 0x7ffb480b0950 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00e840 tx=0x7ffb3c00ec00 comp rx=0 tx=0).stop 2023-08-27T11:27:02.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.864+0000 7ffb50037700 1 -- 172.21.15.112:0/3739061256 shutdown_connections 2023-08-27T11:27:02.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.864+0000 7ffb50037700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffb3406a3b0 0x7ffb3406c870 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.864+0000 7ffb50037700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb48152540 0x7ffb480b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.864+0000 7ffb50037700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb480cc620 0x7ffb481f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.864+0000 7ffb50037700 1 --2- 172.21.15.112:0/3739061256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480ce220 0x7ffb480b0950 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.864+0000 7ffb50037700 1 -- 172.21.15.112:0/3739061256 >> 172.21.15.112:0/3739061256 conn(0x7ffb4806f970 msgr2=0x7ffb480b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:02.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.864+0000 7ffb50037700 1 -- 172.21.15.112:0/3739061256 shutdown_connections 2023-08-27T11:27:02.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:02.864+0000 7ffb50037700 1 -- 172.21.15.112:0/3739061256 wait complete. 2023-08-27T11:27:02.877 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:27:02.878 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:27:02.878 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-08-27T11:27:02.879 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:27:02.879 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:02.879 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:27:02.880 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:27:02.880 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:27:02.880 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:27:02.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:02.881 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-08-27T11:27:02.899 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:27:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:27:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:27:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:27:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:02.900 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:02.901 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:02.901 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:02.901 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:02.901 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:02.901 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:27:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:27:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 41819sec preferred_lft 41819sec 2023-08-27T11:27:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8d6c:4fe6:6a84:8222/64 scope link noprefixroute 2023-08-27T11:27:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:02.903 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:27:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:27:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::d87c:a1ff:fe1d:b635/64 scope link 2023-08-27T11:27:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:02.904 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:27:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:27:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::cc14:79ff:feb3:452f/64 scope link 2023-08-27T11:27:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:02.905 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:27:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether e6:51:f0:78:04:02 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:27:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::e451:f0ff:fe78:402/64 scope link 2023-08-27T11:27:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:02.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:02.906 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:27:02.964 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-08-27T11:27:02.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:02.965 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-08-27T11:27:03.024 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-08-27T11:27:03.025 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:27:03.025 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:27:03.025 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:27:03.025 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:27:03.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:03.026 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:27:03.027 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:27:03.027 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:27:03.027 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:27:03.028 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:27:03.028 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:27:03.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:03.130 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:27:03.146 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:27:03.147 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:27:03.147 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:27:03.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:03.148 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-08-27T11:27:03.212 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:27:03.213 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:03.269 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:27:03.269 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:27:03.270 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:03.325 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:27:03.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:03.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:27:03.412 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:27:03.412 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:27:03.412 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:27:03.420 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:27:03.421 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:27:03.421 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:27:03.421 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:27:03.421 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:27:03.496 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:27:03.496 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:27:03.496 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:27:03.496 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:27:03.497 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:27:03.497 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:27:03.497 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:27:03.497 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:27:03.498 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:27:03.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:03.499 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:03.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:03.526 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:27:03.612 INFO:teuthology.orchestra.run.smithi112.stdout:client6976 2023-08-27T11:27:03.613 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:27:03.613 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:03.631 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:27:03.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.789+0000 7f8934272700 1 Processor -- start 2023-08-27T11:27:03.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.789+0000 7f8934272700 1 -- start start 2023-08-27T11:27:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.790+0000 7f8934272700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f892c0b1400 0x7f892c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.790+0000 7f8934272700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892c161850 con 0x7f892c0b17e0 2023-08-27T11:27:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.790+0000 7f8934272700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892c0cfc10 con 0x7f892c161ce0 2023-08-27T11:27:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.790+0000 7f8934272700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892c161200 con 0x7f892c0b1400 2023-08-27T11:27:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.790+0000 7f893200e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f892c0b1400 0x7f892c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.790+0000 7f893180d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f892c0b17e0 0x7f892c06e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56744/0 (socket says 172.21.15.112:56744) 2023-08-27T11:27:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.790+0000 7f893180d700 1 -- 172.21.15.112:0/38526566 learned_addr learned my addr 172.21.15.112:0/38526566 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.791+0000 7f893200e700 1 -- 172.21.15.112:0/38526566 >> v1:172.21.15.138:6789/0 conn(0x7f892c161ce0 legacy=0x7f892c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.791+0000 7f893200e700 1 -- 172.21.15.112:0/38526566 >> v1:172.21.15.112:6789/0 conn(0x7f892c0b17e0 legacy=0x7f892c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.791+0000 7f893200e700 1 -- 172.21.15.112:0/38526566 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c1614e0 con 0x7f892c0b1400 2023-08-27T11:27:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.791+0000 7f893200e700 1 --2- 172.21.15.112:0/38526566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f892c0b1400 0x7f892c0cf5a0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f891c009840 tx=0x7f891c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d137b23ffa53ad98 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.791+0000 7f893100c700 1 -- 172.21.15.112:0/38526566 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c004230 con 0x7f892c0b1400 2023-08-27T11:27:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.791+0000 7f893100c700 1 -- 172.21.15.112:0/38526566 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891c004390 con 0x7f892c0b1400 2023-08-27T11:27:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.791+0000 7f893100c700 1 -- 172.21.15.112:0/38526566 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c0036a0 con 0x7f892c0b1400 2023-08-27T11:27:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.791+0000 7f8934272700 1 -- 172.21.15.112:0/38526566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f892c0b1400 msgr2=0x7f892c0cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.791+0000 7f8934272700 1 --2- 172.21.15.112:0/38526566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f892c0b1400 0x7f892c0cf5a0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f891c009840 tx=0x7f891c009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.792+0000 7f8934272700 1 -- 172.21.15.112:0/38526566 shutdown_connections 2023-08-27T11:27:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.792+0000 7f8934272700 1 --2- 172.21.15.112:0/38526566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f892c0b1400 0x7f892c0cf5a0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.792+0000 7f8934272700 1 -- 172.21.15.112:0/38526566 >> 172.21.15.112:0/38526566 conn(0x7f892c0d0000 msgr2=0x7f892c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.792+0000 7f8934272700 1 -- 172.21.15.112:0/38526566 shutdown_connections 2023-08-27T11:27:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.792+0000 7f8934272700 1 -- 172.21.15.112:0/38526566 wait complete. 2023-08-27T11:27:03.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.793+0000 7f8934272700 1 Processor -- start 2023-08-27T11:27:03.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.793+0000 7f8934272700 1 -- start start 2023-08-27T11:27:03.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.793+0000 7f8934272700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f892c0b1400 0x7f892c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:03.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f8934272700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f892c0b17e0 0x7f892c143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:03.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f8934272700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f892c161ce0 0x7f892c06ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:03.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f8934272700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c0c8a30 con 0x7f892c0b1400 2023-08-27T11:27:03.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f8934272700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c0c8c60 con 0x7f892c161ce0 2023-08-27T11:27:03.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f8934272700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c0c8f40 con 0x7f892c0b17e0 2023-08-27T11:27:03.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f893200e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f892c0b1400 0x7f892c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:03.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f893200e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f892c0b1400 0x7f892c0c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34136/0 (socket says 172.21.15.112:34136) 2023-08-27T11:27:03.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f893200e700 1 -- 172.21.15.112:0/3205396431 learned_addr learned my addr 172.21.15.112:0/3205396431 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:03.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f893280f700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f892c161ce0 0x7f892c06ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f893180d700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f892c0b17e0 0x7f892c143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f893200e700 1 -- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f892c0b17e0 msgr2=0x7f892c143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f893200e700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f892c0b17e0 0x7f892c143d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f893200e700 1 -- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f892c161ce0 msgr2=0x7f892c06ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f893200e700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f892c161ce0 0x7f892c06ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f893200e700 1 -- 172.21.15.112:0/3205396431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c0094a0 con 0x7f892c0b1400 2023-08-27T11:27:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f893280f700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f892c161ce0 0x7f892c06ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f893200e700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f892c0b1400 0x7f892c0c9d00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f891c0090b0 tx=0x7f891c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f8922ffd700 1 -- 172.21.15.112:0/3205396431 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c011830 con 0x7f892c0b1400 2023-08-27T11:27:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f8922ffd700 1 -- 172.21.15.112:0/3205396431 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891c01e070 con 0x7f892c0b1400 2023-08-27T11:27:03.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f8922ffd700 1 -- 172.21.15.112:0/3205396431 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c011bb0 con 0x7f892c0b1400 2023-08-27T11:27:03.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f8934272700 1 -- 172.21.15.112:0/3205396431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892c0c9220 con 0x7f892c0b1400 2023-08-27T11:27:03.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.794+0000 7f8934272700 1 -- 172.21.15.112:0/3205396431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892c0c97e0 con 0x7f892c0b1400 2023-08-27T11:27:03.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.795+0000 7f8934272700 1 -- 172.21.15.112:0/3205396431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892c067a20 con 0x7f892c0b1400 2023-08-27T11:27:03.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.796+0000 7f8922ffd700 1 -- 172.21.15.112:0/3205396431 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f891c011d10 con 0x7f892c0b1400 2023-08-27T11:27:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.796+0000 7f8922ffd700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f891006a440 0x7f891006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.796+0000 7f8922ffd700 1 -- 172.21.15.112:0/3205396431 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f891c089da0 con 0x7f892c0b1400 2023-08-27T11:27:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.796+0000 7f893180d700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f891006a440 0x7f891006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.797+0000 7f893180d700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f891006a440 0x7f891006c900 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8928000f60 tx=0x7f8928009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:03.800+0000 7f8922ffd700 1 -- 172.21.15.112:0/3205396431 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f891c055110 con 0x7f892c0b1400 2023-08-27T11:27:04.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.030+0000 7f8934272700 1 -- 172.21.15.112:0/3205396431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f892c151ab0 con 0x7f892c0b1400 2023-08-27T11:27:04.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.031+0000 7f8922ffd700 1 -- 172.21.15.112:0/3205396431 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7f891c020070 con 0x7f892c0b1400 2023-08-27T11:27:04.032 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:27:04.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.032+0000 7f8934272700 1 -- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f891006a440 msgr2=0x7f891006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.032+0000 7f8934272700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f891006a440 0x7f891006c900 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8928000f60 tx=0x7f8928009040 comp rx=0 tx=0).stop 2023-08-27T11:27:04.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.032+0000 7f8934272700 1 -- 172.21.15.112:0/3205396431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f892c0b1400 msgr2=0x7f892c0c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.032+0000 7f8934272700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f892c0b1400 0x7f892c0c9d00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f891c0090b0 tx=0x7f891c004750 comp rx=0 tx=0).stop 2023-08-27T11:27:04.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.032+0000 7f8934272700 1 -- 172.21.15.112:0/3205396431 shutdown_connections 2023-08-27T11:27:04.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.032+0000 7f8934272700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f891006a440 0x7f891006c900 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.032+0000 7f8934272700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f892c161ce0 0x7f892c06ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.032+0000 7f8934272700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f892c0b1400 0x7f892c0c9d00 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.032+0000 7f8934272700 1 --2- 172.21.15.112:0/3205396431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f892c0b17e0 0x7f892c143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.032+0000 7f8934272700 1 -- 172.21.15.112:0/3205396431 >> 172.21.15.112:0/3205396431 conn(0x7f892c0d0000 msgr2=0x7f892c0cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:04.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.033+0000 7f8934272700 1 -- 172.21.15.112:0/3205396431 shutdown_connections 2023-08-27T11:27:04.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.033+0000 7f8934272700 1 -- 172.21.15.112:0/3205396431 wait complete. 2023-08-27T11:27:04.046 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:27:04.046 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:27:04.046 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2023-08-27T11:27:04.047 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-27T11:27:04.047 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:27:04.048 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:27:04.048 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:27:04.048 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:27:04.048 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:27:04.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:04.049 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2023-08-27T11:27:04.070 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:27:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:27:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:27:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:27:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:04.072 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:04.072 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:04.073 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:04.073 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:27:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:27:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 41834sec preferred_lft 41834sec 2023-08-27T11:27:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ac17:49c5:cb39:801a/64 scope link noprefixroute 2023-08-27T11:27:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:04.074 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:27:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:27:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c4b4:94ff:fe58:4e35/64 scope link 2023-08-27T11:27:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:04.075 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:27:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:27:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::5806:6aff:fe04:ef32/64 scope link 2023-08-27T11:27:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:04.076 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:27:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 8a:21:ba:15:69:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-27T11:27:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::8821:baff:fe15:6919/64 scope link 2023-08-27T11:27:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:04.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:04.078 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:27:04.141 INFO:teuthology.orchestra.run.smithi138.stdout:1 2023-08-27T11:27:04.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:04.142 DEBUG:teuthology.orchestra.run.smithi138:> route 2023-08-27T11:27:04.203 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2023-08-27T11:27:04.203 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:27:04.203 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:27:04.204 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:27:04.204 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:27:04.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:04.205 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:27:04.205 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:27:04.206 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:27:04.206 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:27:04.206 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:27:04.207 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:27:04.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:04.316 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:27:04.334 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-27T11:27:04.334 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-27T11:27:04.335 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:27:04.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:04.336 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2023-08-27T11:27:04.401 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:27:04.402 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-27T11:27:04.458 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:27:04.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:27:04.459 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:27:04.514 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:27:04.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:04.515 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:27:04.600 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:27:04.600 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:27:04.600 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:27:04.609 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:27:04.609 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:27:04.609 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:27:04.609 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:27:04.610 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:27:04.684 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:27:04.685 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:27:04.685 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:27:04.685 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:27:04.685 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:27:04.686 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:27:04.686 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:27:04.686 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:27:04.687 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:27:04.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:04.688 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:27:04.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:04.715 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:27:04.786 INFO:teuthology.orchestra.run.smithi138.stdout:client7701 2023-08-27T11:27:04.787 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:27:04.787 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:04.844 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-08-27T11:27:04.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.972+0000 7f0155403700 1 Processor -- start 2023-08-27T11:27:04.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.972+0000 7f0155403700 1 -- start start 2023-08-27T11:27:04.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.972+0000 7f0155403700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f01500ce830 0x7f015006bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.972+0000 7f0155403700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f015006c510 0x7f01500cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.972+0000 7f0155403700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0150161c20 con 0x7f01500cf780 2023-08-27T11:27:04.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.972+0000 7f0155403700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01500718b0 con 0x7f01500ce830 2023-08-27T11:27:04.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.972+0000 7f0155403700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0150161010 con 0x7f015006c510 2023-08-27T11:27:04.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.973+0000 7f014e7fc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f015006c510 0x7f01500cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.973+0000 7f014effd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f01500ce830 0x7f015006bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.973+0000 7f014f7fe700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f01500cf780 0x7f015015d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:36052/0 (socket says 172.21.15.112:36052) 2023-08-27T11:27:04.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.973+0000 7f014f7fe700 1 -- 172.21.15.112:0/1782141706 learned_addr learned my addr 172.21.15.112:0/1782141706 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:04.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.973+0000 7f014e7fc700 1 -- 172.21.15.112:0/1782141706 >> v2:172.21.15.138:3300/0 conn(0x7f01500ce830 msgr2=0x7f015006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:04.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.973+0000 7f014e7fc700 1 --2- 172.21.15.112:0/1782141706 >> v2:172.21.15.138:3300/0 conn(0x7f01500ce830 0x7f015006bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.973+0000 7f014e7fc700 1 -- 172.21.15.112:0/1782141706 >> v1:172.21.15.138:6789/0 conn(0x7f01500cf780 legacy=0x7f015015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.973+0000 7f014e7fc700 1 -- 172.21.15.112:0/1782141706 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015006c970 con 0x7f015006c510 2023-08-27T11:27:04.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.973+0000 7f014effd700 1 --2- 172.21.15.112:0/1782141706 >> v2:172.21.15.138:3300/0 conn(0x7f01500ce830 0x7f015006bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:04.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.973+0000 7f014e7fc700 1 --2- 172.21.15.112:0/1782141706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f015006c510 0x7f01500cf240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0140009840 tx=0x7f0140009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2de70e1c813b01e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:04.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.973+0000 7f014dffb700 1 -- 172.21.15.112:0/1782141706 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140011830 con 0x7f015006c510 2023-08-27T11:27:04.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.973+0000 7f014dffb700 1 -- 172.21.15.112:0/1782141706 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0140011990 con 0x7f015006c510 2023-08-27T11:27:04.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.973+0000 7f014dffb700 1 -- 172.21.15.112:0/1782141706 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140011d20 con 0x7f015006c510 2023-08-27T11:27:04.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.974+0000 7f0155403700 1 -- 172.21.15.112:0/1782141706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f015006c510 msgr2=0x7f01500cf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.974+0000 7f0155403700 1 --2- 172.21.15.112:0/1782141706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f015006c510 0x7f01500cf240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0140009840 tx=0x7f0140009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:04.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.974+0000 7f0155403700 1 -- 172.21.15.112:0/1782141706 shutdown_connections 2023-08-27T11:27:04.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.974+0000 7f0155403700 1 --2- 172.21.15.112:0/1782141706 >> v2:172.21.15.138:3300/0 conn(0x7f01500ce830 0x7f015006bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.974+0000 7f0155403700 1 --2- 172.21.15.112:0/1782141706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f015006c510 0x7f01500cf240 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.974+0000 7f0155403700 1 -- 172.21.15.112:0/1782141706 >> 172.21.15.112:0/1782141706 conn(0x7f0150066010 msgr2=0x7f0150066420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:04.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.974+0000 7f0155403700 1 -- 172.21.15.112:0/1782141706 shutdown_connections 2023-08-27T11:27:04.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.974+0000 7f0155403700 1 -- 172.21.15.112:0/1782141706 wait complete. 2023-08-27T11:27:04.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.975+0000 7f0155403700 1 Processor -- start 2023-08-27T11:27:04.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.975+0000 7f0155403700 1 -- start start 2023-08-27T11:27:04.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f0155403700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f015006c510 0x7f015014ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f0155403700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01500ce830 0x7f01500cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f0155403700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01500cf780 0x7f01500cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f0155403700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015014b390 con 0x7f01500cf780 2023-08-27T11:27:04.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f0155403700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01500cc770 con 0x7f015006c510 2023-08-27T11:27:04.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f0155403700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01500cca50 con 0x7f01500ce830 2023-08-27T11:27:04.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f014f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01500cf780 0x7f01500cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f014f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01500cf780 0x7f01500cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34154/0 (socket says 172.21.15.112:34154) 2023-08-27T11:27:04.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f014f7fe700 1 -- 172.21.15.112:0/3013527212 learned_addr learned my addr 172.21.15.112:0/3013527212 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:04.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f014e7fc700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01500ce830 0x7f01500cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f014effd700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f015006c510 0x7f015014ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f014f7fe700 1 -- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01500ce830 msgr2=0x7f01500cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f014f7fe700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01500ce830 0x7f01500cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f014f7fe700 1 -- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f015006c510 msgr2=0x7f015014ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f014f7fe700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f015006c510 0x7f015014ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.976+0000 7f014f7fe700 1 -- 172.21.15.112:0/3013527212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01380094a0 con 0x7f01500cf780 2023-08-27T11:27:04.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.977+0000 7f014f7fe700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01500cf780 0x7f01500cbd20 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f014800cbc0 tx=0x7f014800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:04.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.977+0000 7f01477fe700 1 -- 172.21.15.112:0/3013527212 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014800a9e0 con 0x7f01500cf780 2023-08-27T11:27:04.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.977+0000 7f01477fe700 1 -- 172.21.15.112:0/3013527212 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014800ab40 con 0x7f01500cf780 2023-08-27T11:27:04.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.977+0000 7f0155403700 1 -- 172.21.15.112:0/3013527212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01400094a0 con 0x7f01500cf780 2023-08-27T11:27:04.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.977+0000 7f01477fe700 1 -- 172.21.15.112:0/3013527212 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148012400 con 0x7f01500cf780 2023-08-27T11:27:04.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.977+0000 7f0155403700 1 -- 172.21.15.112:0/3013527212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01500cac00 con 0x7f01500cf780 2023-08-27T11:27:04.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.978+0000 7f0155403700 1 -- 172.21.15.112:0/3013527212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01500ca780 con 0x7f01500cf780 2023-08-27T11:27:04.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.978+0000 7f01477fe700 1 -- 172.21.15.112:0/3013527212 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f014801b3b0 con 0x7f01500cf780 2023-08-27T11:27:04.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.979+0000 7f01477fe700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f013006a440 0x7f013006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.979+0000 7f01477fe700 1 -- 172.21.15.112:0/3013527212 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f01480887a0 con 0x7f01500cf780 2023-08-27T11:27:04.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.979+0000 7f014effd700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f013006a440 0x7f013006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.979+0000 7f014effd700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f013006a440 0x7f013006c900 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0138009cd0 tx=0x7f0138009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:04.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:04.983+0000 7f01477fe700 1 -- 172.21.15.112:0/3013527212 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f01500ca780 con 0x7f01500cf780 2023-08-27T11:27:05.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.216+0000 7f0155403700 1 -- 172.21.15.112:0/3013527212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f01500ca780 con 0x7f01500cf780 2023-08-27T11:27:05.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.620+0000 7f01477fe700 1 -- 172.21.15.112:0/3013527212 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v68) v1 ==== 136+0+0 (secure 0 0 0) 0x7f01500ca780 con 0x7f01500cf780 2023-08-27T11:27:05.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.622+0000 7f0155403700 1 -- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f013006a440 msgr2=0x7f013006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.622+0000 7f0155403700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f013006a440 0x7f013006c900 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0138009cd0 tx=0x7f0138009210 comp rx=0 tx=0).stop 2023-08-27T11:27:05.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.622+0000 7f0155403700 1 -- 172.21.15.112:0/3013527212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01500cf780 msgr2=0x7f01500cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.622+0000 7f0155403700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01500cf780 0x7f01500cbd20 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f014800cbc0 tx=0x7f014800ced0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.622+0000 7f0155403700 1 -- 172.21.15.112:0/3013527212 shutdown_connections 2023-08-27T11:27:05.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.622+0000 7f0155403700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f013006a440 0x7f013006c900 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.622+0000 7f0155403700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01500cf780 0x7f01500cbd20 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.622+0000 7f0155403700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f015006c510 0x7f015014ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.622+0000 7f0155403700 1 --2- 172.21.15.112:0/3013527212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01500ce830 0x7f01500cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.622+0000 7f0155403700 1 -- 172.21.15.112:0/3013527212 >> 172.21.15.112:0/3013527212 conn(0x7f0150066010 msgr2=0x7f01500cd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:05.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.622+0000 7f0155403700 1 -- 172.21.15.112:0/3013527212 shutdown_connections 2023-08-27T11:27:05.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.622+0000 7f0155403700 1 -- 172.21.15.112:0/3013527212 wait complete. 2023-08-27T11:27:05.636 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:27:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.755+0000 7fd908a3a700 1 Processor -- start 2023-08-27T11:27:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.756+0000 7fd908a3a700 1 -- start start 2023-08-27T11:27:05.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.756+0000 7fd908a3a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd904143950 0x7fd9040cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.756+0000 7fd908a3a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd90406dcc0 0x7fd9040b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.756+0000 7fd908a3a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9040b3d70 con 0x7fd9040cdac0 2023-08-27T11:27:05.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd908a3a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9040b08e0 con 0x7fd90406dcc0 2023-08-27T11:27:05.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd908a3a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9040b3440 con 0x7fd904143950 2023-08-27T11:27:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd902d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd90406dcc0 0x7fd9040b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd901d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd9040cdac0 0x7fd90406d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56774/0 (socket says 172.21.15.112:56774) 2023-08-27T11:27:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd901d9b700 1 -- 172.21.15.112:0/550302815 learned_addr learned my addr 172.21.15.112:0/550302815 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd90259c700 1 --2- 172.21.15.112:0/550302815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd904143950 0x7fd9040cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd902d9d700 1 -- 172.21.15.112:0/550302815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd904143950 msgr2=0x7fd9040cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd902d9d700 1 --2- 172.21.15.112:0/550302815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd904143950 0x7fd9040cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd902d9d700 1 -- 172.21.15.112:0/550302815 >> v1:172.21.15.112:6789/0 conn(0x7fd9040cdac0 legacy=0x7fd90406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd902d9d700 1 -- 172.21.15.112:0/550302815 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9040b3720 con 0x7fd90406dcc0 2023-08-27T11:27:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd90159a700 1 -- 172.21.15.112:0/550302815 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956743311 0 0) 0x7fd9040b3d70 con 0x7fd9040cdac0 2023-08-27T11:27:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd902d9d700 1 --2- 172.21.15.112:0/550302815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd90406dcc0 0x7fd9040b4200 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009d90 tx=0x7fd8fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a560b05bc83483c6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:05.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd90159a700 1 -- 172.21.15.112:0/550302815 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc00e070 con 0x7fd90406dcc0 2023-08-27T11:27:05.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd90159a700 1 -- 172.21.15.112:0/550302815 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8fc00bbd0 con 0x7fd90406dcc0 2023-08-27T11:27:05.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.757+0000 7fd90159a700 1 -- 172.21.15.112:0/550302815 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc012430 con 0x7fd90406dcc0 2023-08-27T11:27:05.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.758+0000 7fd908a3a700 1 -- 172.21.15.112:0/550302815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd90406dcc0 msgr2=0x7fd9040b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.758+0000 7fd908a3a700 1 --2- 172.21.15.112:0/550302815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd90406dcc0 0x7fd9040b4200 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009d90 tx=0x7fd8fc00b330 comp rx=0 tx=0).stop 2023-08-27T11:27:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.758+0000 7fd908a3a700 1 -- 172.21.15.112:0/550302815 shutdown_connections 2023-08-27T11:27:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.758+0000 7fd908a3a700 1 --2- 172.21.15.112:0/550302815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd90406dcc0 0x7fd9040b4200 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.758+0000 7fd908a3a700 1 --2- 172.21.15.112:0/550302815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd904143950 0x7fd9040cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.758+0000 7fd908a3a700 1 -- 172.21.15.112:0/550302815 >> 172.21.15.112:0/550302815 conn(0x7fd9040cff80 msgr2=0x7fd9040d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.758+0000 7fd908a3a700 1 -- 172.21.15.112:0/550302815 shutdown_connections 2023-08-27T11:27:05.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.758+0000 7fd908a3a700 1 -- 172.21.15.112:0/550302815 wait complete. 2023-08-27T11:27:05.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.759+0000 7fd908a3a700 1 Processor -- start 2023-08-27T11:27:05.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.760+0000 7fd908a3a700 1 -- start start 2023-08-27T11:27:05.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.760+0000 7fd908a3a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd90406dcc0 0x7fd904158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.760+0000 7fd908a3a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9040cdac0 0x7fd90415c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.760+0000 7fd908a3a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd904143950 0x7fd90415c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.760+0000 7fd908a3a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90415b680 con 0x7fd9040cdac0 2023-08-27T11:27:05.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.760+0000 7fd908a3a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9041f0c90 con 0x7fd904143950 2023-08-27T11:27:05.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.760+0000 7fd908a3a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd904159130 con 0x7fd90406dcc0 2023-08-27T11:27:05.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd901d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9040cdac0 0x7fd90415c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd901d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9040cdac0 0x7fd90415c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34166/0 (socket says 172.21.15.112:34166) 2023-08-27T11:27:05.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd901d9b700 1 -- 172.21.15.112:0/3363158679 learned_addr learned my addr 172.21.15.112:0/3363158679 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:05.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd90259c700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd90406dcc0 0x7fd904158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd902d9d700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd904143950 0x7fd90415c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd901d9b700 1 -- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd90406dcc0 msgr2=0x7fd904158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd901d9b700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd90406dcc0 0x7fd904158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd901d9b700 1 -- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd904143950 msgr2=0x7fd90415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd901d9b700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd904143950 0x7fd90415c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd901d9b700 1 -- 172.21.15.112:0/3363158679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc009a70 con 0x7fd9040cdac0 2023-08-27T11:27:05.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd901d9b700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9040cdac0 0x7fd90415c420 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40011e0 tx=0x7fd8f400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:05.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd8faffd700 1 -- 172.21.15.112:0/3363158679 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4004c80 con 0x7fd9040cdac0 2023-08-27T11:27:05.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd8faffd700 1 -- 172.21.15.112:0/3363158679 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f4004e00 con 0x7fd9040cdac0 2023-08-27T11:27:05.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd908a3a700 1 -- 172.21.15.112:0/3363158679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90415ab00 con 0x7fd9040cdac0 2023-08-27T11:27:05.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd8faffd700 1 -- 172.21.15.112:0/3363158679 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4007310 con 0x7fd9040cdac0 2023-08-27T11:27:05.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.761+0000 7fd908a3a700 1 -- 172.21.15.112:0/3363158679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90415b0c0 con 0x7fd9040cdac0 2023-08-27T11:27:05.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.762+0000 7fd908a3a700 1 -- 172.21.15.112:0/3363158679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9041f0e00 con 0x7fd9040cdac0 2023-08-27T11:27:05.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.763+0000 7fd8faffd700 1 -- 172.21.15.112:0/3363158679 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fd8f4007470 con 0x7fd9040cdac0 2023-08-27T11:27:05.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.763+0000 7fd8faffd700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd8e406a440 0x7fd8e406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.763+0000 7fd8faffd700 1 -- 172.21.15.112:0/3363158679 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd8f4088860 con 0x7fd9040cdac0 2023-08-27T11:27:05.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.763+0000 7fd90259c700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd8e406a440 0x7fd8e406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.764+0000 7fd90259c700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd8e406a440 0x7fd8e406c900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00b9b0 tx=0x7fd8ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:05.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.768+0000 7fd8faffd700 1 -- 172.21.15.112:0/3363158679 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd9041f0e00 con 0x7fd9040cdac0 2023-08-27T11:27:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.982+0000 7fd908a3a700 1 -- 172.21.15.112:0/3363158679 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9040b3a00 con 0x7fd8e406a440 2023-08-27T11:27:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.985+0000 7fd8faffd700 1 -- 172.21.15.112:0/3363158679 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd8ec00be80 con 0x7fd8e406a440 2023-08-27T11:27:05.985 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:27:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs" 2023-08-27T11:27:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:05.986 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:27:05.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.986+0000 7fd908a3a700 1 -- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd8e406a440 msgr2=0x7fd8e406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.986+0000 7fd908a3a700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd8e406a440 0x7fd8e406c900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00b9b0 tx=0x7fd8ec008040 comp rx=0 tx=0).stop 2023-08-27T11:27:05.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.986+0000 7fd908a3a700 1 -- 172.21.15.112:0/3363158679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9040cdac0 msgr2=0x7fd90415c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.986+0000 7fd908a3a700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9040cdac0 0x7fd90415c420 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40011e0 tx=0x7fd8f400cae0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.986+0000 7fd908a3a700 1 -- 172.21.15.112:0/3363158679 shutdown_connections 2023-08-27T11:27:05.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.986+0000 7fd908a3a700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd8e406a440 0x7fd8e406c900 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.986+0000 7fd908a3a700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd904143950 0x7fd90415c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.986+0000 7fd908a3a700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd90406dcc0 0x7fd904158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.986+0000 7fd908a3a700 1 --2- 172.21.15.112:0/3363158679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9040cdac0 0x7fd90415c420 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.986+0000 7fd908a3a700 1 -- 172.21.15.112:0/3363158679 >> 172.21.15.112:0/3363158679 conn(0x7fd9040cff80 msgr2=0x7fd9041418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.986+0000 7fd908a3a700 1 -- 172.21.15.112:0/3363158679 shutdown_connections 2023-08-27T11:27:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:05.986+0000 7fd908a3a700 1 -- 172.21.15.112:0/3363158679 wait complete. 2023-08-27T11:27:06.000 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:27:06.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.121+0000 7f0d2f37d700 1 Processor -- start 2023-08-27T11:27:06.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.121+0000 7f0d2f37d700 1 -- start start 2023-08-27T11:27:06.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.122+0000 7f0d2f37d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0d281437c0 0x7f0d280b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.122+0000 7f0d2f37d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0d2806d5b0 0x7f0d2806da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.122+0000 7f0d2f37d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d280cfd50 con 0x7f0d28143ba0 2023-08-27T11:27:06.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.122+0000 7f0d2f37d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d2806df80 con 0x7f0d281437c0 2023-08-27T11:27:06.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.122+0000 7f0d2f37d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d280cf140 con 0x7f0d2806d5b0 2023-08-27T11:27:06.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.122+0000 7f0d2d119700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0d281437c0 0x7f0d280b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.122+0000 7f0d2d119700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0d281437c0 0x7f0d280b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34172/0 (socket says 172.21.15.112:34172) 2023-08-27T11:27:06.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.122+0000 7f0d2d119700 1 -- 172.21.15.112:0/3044435933 learned_addr learned my addr 172.21.15.112:0/3044435933 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:06.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.122+0000 7f0d2d91a700 1 --2- 172.21.15.112:0/3044435933 >> v2:172.21.15.138:3300/0 conn(0x7f0d2806d5b0 0x7f0d2806da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.122+0000 7f0d2d119700 1 -- 172.21.15.112:0/3044435933 >> v2:172.21.15.138:3300/0 conn(0x7f0d2806d5b0 msgr2=0x7f0d2806da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:06.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.122+0000 7f0d2d119700 1 --2- 172.21.15.112:0/3044435933 >> v2:172.21.15.138:3300/0 conn(0x7f0d2806d5b0 0x7f0d2806da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.122+0000 7f0d2d119700 1 -- 172.21.15.112:0/3044435933 >> v1:172.21.15.138:6789/0 conn(0x7f0d28143ba0 legacy=0x7f0d280ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:06.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.122+0000 7f0d2d119700 1 -- 172.21.15.112:0/3044435933 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d280cf420 con 0x7f0d281437c0 2023-08-27T11:27:06.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.123+0000 7f0d2d119700 1 --2- 172.21.15.112:0/3044435933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d281437c0 0x7f0d280b4670 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f0d280cfcc0 tx=0x7f0d1800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5136ca9483aec6d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:06.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.123+0000 7f0d1ffff700 1 -- 172.21.15.112:0/3044435933 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800dcf0 con 0x7f0d281437c0 2023-08-27T11:27:06.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.123+0000 7f0d1ffff700 1 -- 172.21.15.112:0/3044435933 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1800de50 con 0x7f0d281437c0 2023-08-27T11:27:06.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.123+0000 7f0d1ffff700 1 -- 172.21.15.112:0/3044435933 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18004250 con 0x7f0d281437c0 2023-08-27T11:27:06.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.123+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3044435933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d281437c0 msgr2=0x7f0d280b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.123+0000 7f0d2f37d700 1 --2- 172.21.15.112:0/3044435933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d281437c0 0x7f0d280b4670 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f0d280cfcc0 tx=0x7f0d1800be70 comp rx=0 tx=0).stop 2023-08-27T11:27:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.123+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3044435933 shutdown_connections 2023-08-27T11:27:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.123+0000 7f0d2f37d700 1 --2- 172.21.15.112:0/3044435933 >> v2:172.21.15.138:3300/0 conn(0x7f0d2806d5b0 0x7f0d2806da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.123+0000 7f0d2f37d700 1 --2- 172.21.15.112:0/3044435933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d281437c0 0x7f0d280b4670 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.123+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3044435933 >> 172.21.15.112:0/3044435933 conn(0x7f0d280b1e90 msgr2=0x7f0d280b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.123+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3044435933 shutdown_connections 2023-08-27T11:27:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.123+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3044435933 wait complete. 2023-08-27T11:27:06.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.125+0000 7f0d2f37d700 1 Processor -- start 2023-08-27T11:27:06.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.125+0000 7f0d2f37d700 1 -- start start 2023-08-27T11:27:06.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.125+0000 7f0d2f37d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d2806d5b0 0x7f0d280af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.125+0000 7f0d2f37d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d281437c0 0x7f0d280ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.125+0000 7f0d2f37d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d28143ba0 0x7f0d28137670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.125+0000 7f0d2f37d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d280ae390 con 0x7f0d281437c0 2023-08-27T11:27:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.125+0000 7f0d2f37d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d280ae630 con 0x7f0d28143ba0 2023-08-27T11:27:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.125+0000 7f0d2f37d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d280ae910 con 0x7f0d2806d5b0 2023-08-27T11:27:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.125+0000 7f0d2c918700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d281437c0 0x7f0d280ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.125+0000 7f0d2c918700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d281437c0 0x7f0d280ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34180/0 (socket says 172.21.15.112:34180) 2023-08-27T11:27:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.125+0000 7f0d2c918700 1 -- 172.21.15.112:0/3843535367 learned_addr learned my addr 172.21.15.112:0/3843535367 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d2d91a700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d28143ba0 0x7f0d28137670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d2d119700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d2806d5b0 0x7f0d280af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d2c918700 1 -- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d2806d5b0 msgr2=0x7f0d280af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d2c918700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d2806d5b0 0x7f0d280af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d2c918700 1 -- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d28143ba0 msgr2=0x7f0d28137670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d2c918700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d28143ba0 0x7f0d28137670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d2c918700 1 -- 172.21.15.112:0/3843535367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1800b7f0 con 0x7f0d281437c0 2023-08-27T11:27:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d2d119700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d2806d5b0 0x7f0d280af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d2d91a700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d28143ba0 0x7f0d28137670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d2c918700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d281437c0 0x7f0d280ade50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400b4a0 tx=0x7f0d2400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d1dffb700 1 -- 172.21.15.112:0/3843535367 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24011830 con 0x7f0d281437c0 2023-08-27T11:27:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d1dffb700 1 -- 172.21.15.112:0/3843535367 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d24011990 con 0x7f0d281437c0 2023-08-27T11:27:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3843535367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d280aebf0 con 0x7f0d281437c0 2023-08-27T11:27:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d1dffb700 1 -- 172.21.15.112:0/3843535367 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24011d40 con 0x7f0d281437c0 2023-08-27T11:27:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.126+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3843535367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d280af1b0 con 0x7f0d281437c0 2023-08-27T11:27:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.127+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3843535367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d28067e90 con 0x7f0d281437c0 2023-08-27T11:27:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.128+0000 7f0d1dffb700 1 -- 172.21.15.112:0/3843535367 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f0d2401a400 con 0x7f0d281437c0 2023-08-27T11:27:06.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.128+0000 7f0d1dffb700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0d0c06a440 0x7f0d0c06c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.128+0000 7f0d1dffb700 1 -- 172.21.15.112:0/3843535367 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d24088730 con 0x7f0d281437c0 2023-08-27T11:27:06.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.128+0000 7f0d2d119700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0d0c06a440 0x7f0d0c06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.129+0000 7f0d2d119700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0d0c06a440 0x7f0d0c06c900 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800beb0 tx=0x7f0d18012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:06.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.132+0000 7f0d1dffb700 1 -- 172.21.15.112:0/3843535367 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0d24053b50 con 0x7f0d281437c0 2023-08-27T11:27:06.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.323+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3843535367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0d280af3a0 con 0x7f0d281437c0 2023-08-27T11:27:06.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.324+0000 7f0d1dffb700 1 -- 172.21.15.112:0/3843535367 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0d24057170 con 0x7f0d281437c0 2023-08-27T11:27:06.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.326+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0d0c06a440 msgr2=0x7f0d0c06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.326+0000 7f0d2f37d700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0d0c06a440 0x7f0d0c06c900 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800beb0 tx=0x7f0d18012000 comp rx=0 tx=0).stop 2023-08-27T11:27:06.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.326+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3843535367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d281437c0 msgr2=0x7f0d280ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.326+0000 7f0d2f37d700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d281437c0 0x7f0d280ade50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400b4a0 tx=0x7f0d2400b860 comp rx=0 tx=0).stop 2023-08-27T11:27:06.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.326+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3843535367 shutdown_connections 2023-08-27T11:27:06.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.326+0000 7f0d2f37d700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0d0c06a440 0x7f0d0c06c900 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.326+0000 7f0d2f37d700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d28143ba0 0x7f0d28137670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.326+0000 7f0d2f37d700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d2806d5b0 0x7f0d280af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.326+0000 7f0d2f37d700 1 --2- 172.21.15.112:0/3843535367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d281437c0 0x7f0d280ade50 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.326+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3843535367 >> 172.21.15.112:0/3843535367 conn(0x7f0d280b1e90 msgr2=0x7f0d28141170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:06.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.326+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3843535367 shutdown_connections 2023-08-27T11:27:06.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.326+0000 7f0d2f37d700 1 -- 172.21.15.112:0/3843535367 wait complete. 2023-08-27T11:27:06.340 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-08-27T11:27:06.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.461+0000 7fd76ced0700 1 Processor -- start 2023-08-27T11:27:06.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.461+0000 7fd76ced0700 1 -- start start 2023-08-27T11:27:06.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.462+0000 7fd76ced0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd76815e190 0x7fd76815e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.462+0000 7fd76ced0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd7680cc980 0x7fd7680cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.462+0000 7fd76ced0700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7680b46c0 con 0x7fd768161dd0 2023-08-27T11:27:06.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.462+0000 7fd76ced0700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7680b44d0 con 0x7fd7680cc980 2023-08-27T11:27:06.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.462+0000 7fd76ced0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680cd350 con 0x7fd76815e190 2023-08-27T11:27:06.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.462+0000 7fd76659c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd76815e190 0x7fd76815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.462+0000 7fd766d9d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd7680cc980 0x7fd7680cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.462+0000 7fd76659c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd76815e190 0x7fd76815e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:45840/0 (socket says 172.21.15.112:45840) 2023-08-27T11:27:06.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.462+0000 7fd76659c700 1 -- 172.21.15.112:0/390649448 learned_addr learned my addr 172.21.15.112:0/390649448 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:06.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.463+0000 7fd766d9d700 1 -- 172.21.15.112:0/390649448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd76815e190 msgr2=0x7fd76815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:06.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.463+0000 7fd766d9d700 1 --2- 172.21.15.112:0/390649448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd76815e190 0x7fd76815e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.463+0000 7fd766d9d700 1 -- 172.21.15.112:0/390649448 >> v1:172.21.15.138:6789/0 conn(0x7fd768161dd0 legacy=0x7fd7680b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.463+0000 7fd766d9d700 1 -- 172.21.15.112:0/390649448 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7680b49a0 con 0x7fd7680cc980 2023-08-27T11:27:06.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.463+0000 7fd76559a700 1 -- 172.21.15.112:0/390649448 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611166111 0 0) 0x7fd7680b46c0 con 0x7fd768161dd0 2023-08-27T11:27:06.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.463+0000 7fd76659c700 1 --2- 172.21.15.112:0/390649448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd76815e190 0x7fd76815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:06.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.463+0000 7fd766d9d700 1 --2- 172.21.15.112:0/390649448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7680cc980 0x7fd7680cce10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd760009d90 tx=0x7fd76000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66e277b6a0bab008 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:06.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.463+0000 7fd76559a700 1 -- 172.21.15.112:0/390649448 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76000e070 con 0x7fd7680cc980 2023-08-27T11:27:06.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.463+0000 7fd76559a700 1 -- 172.21.15.112:0/390649448 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76000bb20 con 0x7fd7680cc980 2023-08-27T11:27:06.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.463+0000 7fd76559a700 1 -- 172.21.15.112:0/390649448 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd760012430 con 0x7fd7680cc980 2023-08-27T11:27:06.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.464+0000 7fd76ced0700 1 -- 172.21.15.112:0/390649448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7680cc980 msgr2=0x7fd7680cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.464+0000 7fd76ced0700 1 --2- 172.21.15.112:0/390649448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7680cc980 0x7fd7680cce10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd760009d90 tx=0x7fd76000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:06.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.464+0000 7fd76ced0700 1 -- 172.21.15.112:0/390649448 shutdown_connections 2023-08-27T11:27:06.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.464+0000 7fd76ced0700 1 --2- 172.21.15.112:0/390649448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7680cc980 0x7fd7680cce10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.464+0000 7fd76ced0700 1 --2- 172.21.15.112:0/390649448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd76815e190 0x7fd76815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.464+0000 7fd76ced0700 1 -- 172.21.15.112:0/390649448 >> 172.21.15.112:0/390649448 conn(0x7fd76806bbf0 msgr2=0x7fd7680cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:06.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.464+0000 7fd76ced0700 1 -- 172.21.15.112:0/390649448 shutdown_connections 2023-08-27T11:27:06.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.464+0000 7fd76ced0700 1 -- 172.21.15.112:0/390649448 wait complete. 2023-08-27T11:27:06.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.465+0000 7fd76ced0700 1 Processor -- start 2023-08-27T11:27:06.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.465+0000 7fd76ced0700 1 -- start start 2023-08-27T11:27:06.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd76ced0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7680cc980 0x7fd7680ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd76ced0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd76815e190 0x7fd76814f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd76ced0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768161dd0 0x7fd7680aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd76ced0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76806f6a0 con 0x7fd768161dd0 2023-08-27T11:27:06.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd76ced0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680acdd0 con 0x7fd7680cc980 2023-08-27T11:27:06.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd76ced0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7681f2fb0 con 0x7fd76815e190 2023-08-27T11:27:06.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd766d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768161dd0 0x7fd7680aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd766d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768161dd0 0x7fd7680aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34196/0 (socket says 172.21.15.112:34196) 2023-08-27T11:27:06.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd766d9d700 1 -- 172.21.15.112:0/426238341 learned_addr learned my addr 172.21.15.112:0/426238341 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:06.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd76659c700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7680cc980 0x7fd7680ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd765d9b700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd76815e190 0x7fd76814f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd766d9d700 1 -- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd76815e190 msgr2=0x7fd76814f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd766d9d700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd76815e190 0x7fd76814f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd766d9d700 1 -- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7680cc980 msgr2=0x7fd7680ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd766d9d700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7680cc980 0x7fd7680ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd766d9d700 1 -- 172.21.15.112:0/426238341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75c009550 con 0x7fd768161dd0 2023-08-27T11:27:06.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.466+0000 7fd766d9d700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768161dd0 0x7fd7680aead0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd7680d0fe0 tx=0x7fd760004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:06.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.467+0000 7fd756ffd700 1 -- 172.21.15.112:0/426238341 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd760012430 con 0x7fd768161dd0 2023-08-27T11:27:06.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.467+0000 7fd756ffd700 1 -- 172.21.15.112:0/426238341 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd760020070 con 0x7fd768161dd0 2023-08-27T11:27:06.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.467+0000 7fd756ffd700 1 -- 172.21.15.112:0/426238341 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd760004280 con 0x7fd768161dd0 2023-08-27T11:27:06.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.467+0000 7fd76ced0700 1 -- 172.21.15.112:0/426238341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd760009a70 con 0x7fd768161dd0 2023-08-27T11:27:06.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.467+0000 7fd76ced0700 1 -- 172.21.15.112:0/426238341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7681f3570 con 0x7fd768161dd0 2023-08-27T11:27:06.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.467+0000 7fd76ced0700 1 -- 172.21.15.112:0/426238341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd768066fa0 con 0x7fd768161dd0 2023-08-27T11:27:06.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.468+0000 7fd756ffd700 1 -- 172.21.15.112:0/426238341 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fd7600043e0 con 0x7fd768161dd0 2023-08-27T11:27:06.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.468+0000 7fd756ffd700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd74806a440 0x7fd74806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.468+0000 7fd756ffd700 1 -- 172.21.15.112:0/426238341 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd760022070 con 0x7fd768161dd0 2023-08-27T11:27:06.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.469+0000 7fd76659c700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd74806a440 0x7fd74806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.469+0000 7fd76659c700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd74806a440 0x7fd74806c900 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd7680b1900 tx=0x7fd75c0092c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:06.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.473+0000 7fd756ffd700 1 -- 172.21.15.112:0/426238341 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd7600559a0 con 0x7fd768161dd0 2023-08-27T11:27:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.692+0000 7fd76ced0700 1 -- 172.21.15.112:0/426238341 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7681615d0 con 0x7fd74806a440 2023-08-27T11:27:06.700 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:27:06.699+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:06.700 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:27:06.699+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:06.700 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:27:06.699+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:06.701 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:27:06.699+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:06.701 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:27:06.699+0000 7f2859c79700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:06.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.706+0000 7fd756ffd700 1 -- 172.21.15.112:0/426238341 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+473 (secure 0 0 0) 0x7fd75c009a00 con 0x7fd74806a440 2023-08-27T11:27:06.708 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:27:06.708 INFO:teuthology.orchestra.run.smithi112.stdout: "mon_addrs": [ 2023-08-27T11:27:06.708 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.112:6789", 2023-08-27T11:27:06.708 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.138:6789", 2023-08-27T11:27:06.709 INFO:teuthology.orchestra.run.smithi112.stdout: "172.21.15.138:6790" 2023-08-27T11:27:06.709 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2023-08-27T11:27:06.709 INFO:teuthology.orchestra.run.smithi112.stdout: "pools": { 2023-08-27T11:27:06.709 INFO:teuthology.orchestra.run.smithi112.stdout: "data": [ 2023-08-27T11:27:06.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:06.710 INFO:teuthology.orchestra.run.smithi112.stdout: "avail": 367084470272, 2023-08-27T11:27:06.710 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs_data", 2023-08-27T11:27:06.710 INFO:teuthology.orchestra.run.smithi112.stdout: "used": 0 2023-08-27T11:27:06.710 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:06.710 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2023-08-27T11:27:06.711 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": [ 2023-08-27T11:27:06.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:06.711 INFO:teuthology.orchestra.run.smithi112.stdout: "avail": 367084470272, 2023-08-27T11:27:06.711 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs_metadata", 2023-08-27T11:27:06.711 INFO:teuthology.orchestra.run.smithi112.stdout: "used": 65536 2023-08-27T11:27:06.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:06.712 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:27:06.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:06.712 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:27:06.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.708+0000 7fd76ced0700 1 -- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd74806a440 msgr2=0x7fd74806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.709+0000 7fd76ced0700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd74806a440 0x7fd74806c900 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd7680b1900 tx=0x7fd75c0092c0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.709+0000 7fd76ced0700 1 -- 172.21.15.112:0/426238341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768161dd0 msgr2=0x7fd7680aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.709+0000 7fd76ced0700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768161dd0 0x7fd7680aead0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd7680d0fe0 tx=0x7fd760004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.709+0000 7fd76ced0700 1 -- 172.21.15.112:0/426238341 shutdown_connections 2023-08-27T11:27:06.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.709+0000 7fd76ced0700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd74806a440 0x7fd74806c900 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.709+0000 7fd76ced0700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768161dd0 0x7fd7680aead0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.709+0000 7fd76ced0700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7680cc980 0x7fd7680ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.709+0000 7fd76ced0700 1 --2- 172.21.15.112:0/426238341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd76815e190 0x7fd76814f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.709+0000 7fd76ced0700 1 -- 172.21.15.112:0/426238341 >> 172.21.15.112:0/426238341 conn(0x7fd76806bbf0 msgr2=0x7fd7680b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.709+0000 7fd76ced0700 1 -- 172.21.15.112:0/426238341 shutdown_connections 2023-08-27T11:27:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:06.709+0000 7fd76ced0700 1 -- 172.21.15.112:0/426238341 wait complete. 2023-08-27T11:27:06.722 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:27:06.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:27:06.764 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:27:06.814 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:27:06.815 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:27:06.815 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:06.833 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:27:06.833 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:06.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:07.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.048+0000 7f9aeb5fd700 1 Processor -- start 2023-08-27T11:27:07.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.048+0000 7f9aeb5fd700 1 -- start start 2023-08-27T11:27:07.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9aeb5fd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9ae40b1400 0x7f9ae40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9aeb5fd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9ae40b17e0 0x7f9ae406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9aeb5fd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ae4161ce0 0x7f9ae406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9aeb5fd700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae4161850 con 0x7f9ae40b17e0 2023-08-27T11:27:07.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9aeb5fd700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae406e7d0 con 0x7f9ae40b1400 2023-08-27T11:27:07.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9aeb5fd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40cfc10 con 0x7f9ae4161ce0 2023-08-27T11:27:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9ae8b98700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9ae40b17e0 0x7f9ae406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9ae8b98700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9ae40b17e0 0x7f9ae406e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34202/0 (socket says 172.21.15.112:34202) 2023-08-27T11:27:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9ae8b98700 1 -- 172.21.15.112:0/3556260815 learned_addr learned my addr 172.21.15.112:0/3556260815 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9ae9b9a700 1 --2- 172.21.15.112:0/3556260815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ae4161ce0 0x7f9ae406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9ae9399700 1 --2- 172.21.15.112:0/3556260815 >> v2:172.21.15.138:3300/0 conn(0x7f9ae40b1400 0x7f9ae40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9ae8b98700 1 -- 172.21.15.112:0/3556260815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ae4161ce0 msgr2=0x7f9ae406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9ae8b98700 1 --2- 172.21.15.112:0/3556260815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ae4161ce0 0x7f9ae406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9ae8b98700 1 -- 172.21.15.112:0/3556260815 >> v2:172.21.15.138:3300/0 conn(0x7f9ae40b1400 msgr2=0x7f9ae40cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9ae8b98700 1 --2- 172.21.15.112:0/3556260815 >> v2:172.21.15.138:3300/0 conn(0x7f9ae40b1400 0x7f9ae40cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9ae8b98700 1 -- 172.21.15.112:0/3556260815 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae4161200 con 0x7f9ae40b17e0 2023-08-27T11:27:07.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9ae9b9a700 1 --2- 172.21.15.112:0/3556260815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ae4161ce0 0x7f9ae406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:07.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9ae9399700 1 --2- 172.21.15.112:0/3556260815 >> v2:172.21.15.138:3300/0 conn(0x7f9ae40b1400 0x7f9ae40cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:07.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.049+0000 7f9ae8b98700 1 --2- 172.21.15.112:0/3556260815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae40b17e0 0x7f9ae406e290 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4070370 tx=0x7f9ad4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ede8e2bcf4eaa6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.050+0000 7f9adbfff700 1 -- 172.21.15.112:0/3556260815 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400b870 con 0x7f9ae40b17e0 2023-08-27T11:27:07.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.050+0000 7f9adbfff700 1 -- 172.21.15.112:0/3556260815 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad400b9d0 con 0x7f9ae40b17e0 2023-08-27T11:27:07.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.050+0000 7f9adbfff700 1 -- 172.21.15.112:0/3556260815 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400bd60 con 0x7f9ae40b17e0 2023-08-27T11:27:07.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.050+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3556260815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae40b17e0 msgr2=0x7f9ae406e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.050+0000 7f9aeb5fd700 1 --2- 172.21.15.112:0/3556260815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae40b17e0 0x7f9ae406e290 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4070370 tx=0x7f9ad4009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.050+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3556260815 shutdown_connections 2023-08-27T11:27:07.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.050+0000 7f9aeb5fd700 1 --2- 172.21.15.112:0/3556260815 >> v2:172.21.15.138:3300/0 conn(0x7f9ae40b1400 0x7f9ae40cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.050+0000 7f9aeb5fd700 1 --2- 172.21.15.112:0/3556260815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ae4161ce0 0x7f9ae406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.050+0000 7f9aeb5fd700 1 --2- 172.21.15.112:0/3556260815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae40b17e0 0x7f9ae406e290 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.050+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3556260815 >> 172.21.15.112:0/3556260815 conn(0x7f9ae40d0000 msgr2=0x7f9ae40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:07.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.050+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3556260815 shutdown_connections 2023-08-27T11:27:07.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.050+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3556260815 wait complete. 2023-08-27T11:27:07.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.051+0000 7f9aeb5fd700 1 Processor -- start 2023-08-27T11:27:07.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.052+0000 7f9aeb5fd700 1 -- start start 2023-08-27T11:27:07.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.052+0000 7f9aeb5fd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae40b1400 0x7f9ae40c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.052+0000 7f9aeb5fd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ae40b17e0 0x7f9ae415e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.052+0000 7f9aeb5fd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae4161ce0 0x7f9ae4144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.052+0000 7f9aeb5fd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae415fda0 con 0x7f9ae4161ce0 2023-08-27T11:27:07.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.052+0000 7f9aeb5fd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae415f910 con 0x7f9ae40b1400 2023-08-27T11:27:07.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.052+0000 7f9aeb5fd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae415ed00 con 0x7f9ae40b17e0 2023-08-27T11:27:07.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.052+0000 7f9ae9b9a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae4161ce0 0x7f9ae4144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.052+0000 7f9ae9b9a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae4161ce0 0x7f9ae4144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34208/0 (socket says 172.21.15.112:34208) 2023-08-27T11:27:07.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.052+0000 7f9ae9b9a700 1 -- 172.21.15.112:0/3470307624 learned_addr learned my addr 172.21.15.112:0/3470307624 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:07.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.052+0000 7f9ae9399700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae40b1400 0x7f9ae40c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.052+0000 7f9ae8b98700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ae40b17e0 0x7f9ae415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.053+0000 7f9ae9b9a700 1 -- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ae40b17e0 msgr2=0x7f9ae415e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.053+0000 7f9ae9b9a700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ae40b17e0 0x7f9ae415e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.053+0000 7f9ae9b9a700 1 -- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae40b1400 msgr2=0x7f9ae40c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.053+0000 7f9ae9b9a700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae40b1400 0x7f9ae40c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.053+0000 7f9ae9b9a700 1 -- 172.21.15.112:0/3470307624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4009550 con 0x7f9ae4161ce0 2023-08-27T11:27:07.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.053+0000 7f9ae8b98700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ae40b17e0 0x7f9ae415e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:07.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.053+0000 7f9ae9b9a700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae4161ce0 0x7f9ae4144520 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00cd50 tx=0x7f9adc00a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.053+0000 7f9ad9ffb700 1 -- 172.21.15.112:0/3470307624 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc00cea0 con 0x7f9ae4161ce0 2023-08-27T11:27:07.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.053+0000 7f9ad9ffb700 1 -- 172.21.15.112:0/3470307624 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc00f480 con 0x7f9ae4161ce0 2023-08-27T11:27:07.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.053+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3470307624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae415efe0 con 0x7f9ae4161ce0 2023-08-27T11:27:07.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.053+0000 7f9ad9ffb700 1 -- 172.21.15.112:0/3470307624 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc00f810 con 0x7f9ae4161ce0 2023-08-27T11:27:07.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.053+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3470307624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae415f5a0 con 0x7f9ae4161ce0 2023-08-27T11:27:07.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.054+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3470307624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae415fa80 con 0x7f9ae4161ce0 2023-08-27T11:27:07.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.055+0000 7f9ad9ffb700 1 -- 172.21.15.112:0/3470307624 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f9adc00b040 con 0x7f9ae4161ce0 2023-08-27T11:27:07.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.055+0000 7f9ad9ffb700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9ac806a440 0x7f9ac806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.055+0000 7f9ad9ffb700 1 -- 172.21.15.112:0/3470307624 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9adc089320 con 0x7f9ae4161ce0 2023-08-27T11:27:07.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.055+0000 7f9ae9399700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9ac806a440 0x7f9ac806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.056+0000 7f9ae9399700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9ac806a440 0x7f9ac806c900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40c36d0 tx=0x7f9ae0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.059+0000 7f9ad9ffb700 1 -- 172.21.15.112:0/3470307624 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9adc0544b0 con 0x7f9ae4161ce0 2023-08-27T11:27:07.090 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:27:07.092 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:27:07.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.253+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3470307624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ae4067a20 con 0x7f9ae4161ce0 2023-08-27T11:27:07.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.254+0000 7f9ad9ffb700 1 -- 172.21.15.112:0/3470307624 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+18327 (secure 0 0 0) 0x7f9adc057ad0 con 0x7f9ae4161ce0 2023-08-27T11:27:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:07.256 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":102,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:57.319281+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:54.078457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:54.929983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:07.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.256+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9ac806a440 msgr2=0x7f9ac806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.256+0000 7f9aeb5fd700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9ac806a440 0x7f9ac806c900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40c36d0 tx=0x7f9ae0008040 comp rx=0 tx=0).stop 2023-08-27T11:27:07.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.256+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3470307624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae4161ce0 msgr2=0x7f9ae4144520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.256+0000 7f9aeb5fd700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae4161ce0 0x7f9ae4144520 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00cd50 tx=0x7f9adc00a560 comp rx=0 tx=0).stop 2023-08-27T11:27:07.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.256+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3470307624 shutdown_connections 2023-08-27T11:27:07.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.256+0000 7f9aeb5fd700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9ac806a440 0x7f9ac806c900 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.256+0000 7f9aeb5fd700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae4161ce0 0x7f9ae4144520 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.256+0000 7f9aeb5fd700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae40b1400 0x7f9ae40c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.256+0000 7f9aeb5fd700 1 --2- 172.21.15.112:0/3470307624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ae40b17e0 0x7f9ae415e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.256+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3470307624 >> 172.21.15.112:0/3470307624 conn(0x7f9ae40d0000 msgr2=0x7f9ae40d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:07.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.256+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3470307624 shutdown_connections 2023-08-27T11:27:07.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.256+0000 7f9aeb5fd700 1 -- 172.21.15.112:0/3470307624 wait complete. 2023-08-27T11:27:07.274 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:26:54.078457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:26:54.929983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:07.275 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.397+0000 7f5f5248d700 1 Processor -- start 2023-08-27T11:27:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.398+0000 7f5f5248d700 1 -- start start 2023-08-27T11:27:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.398+0000 7f5f5248d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5f4c0b3de0 0x7f5f4c06bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.398+0000 7f5f5248d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c06c510 0x7f5f4c0cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.398+0000 7f5f5248d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4c06c960 con 0x7f5f4c144b80 2023-08-27T11:27:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.398+0000 7f5f5248d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4c0b4b40 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.398+0000 7f5f5248d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0cef20 con 0x7f5f4c06c510 2023-08-27T11:27:07.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.399+0000 7f5f4b7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5f4c0b3de0 0x7f5f4c06bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.399+0000 7f5f4bfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5f4c144b80 0x7f5f4c0b6c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56828/0 (socket says 172.21.15.112:56828) 2023-08-27T11:27:07.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.399+0000 7f5f4bfff700 1 -- 172.21.15.112:0/2496807339 learned_addr learned my addr 172.21.15.112:0/2496807339 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:07.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.399+0000 7f5f4b7fe700 1 -- 172.21.15.112:0/2496807339 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c06c510 msgr2=0x7f5f4c0cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.399+0000 7f5f4b7fe700 1 --2- 172.21.15.112:0/2496807339 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c06c510 0x7f5f4c0cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.399+0000 7f5f4b7fe700 1 -- 172.21.15.112:0/2496807339 >> v1:172.21.15.112:6789/0 conn(0x7f5f4c144b80 legacy=0x7f5f4c0b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.399+0000 7f5f4b7fe700 1 -- 172.21.15.112:0/2496807339 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0b7310 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.399+0000 7f5f4affd700 1 -- 172.21.15.112:0/2496807339 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059242795 0 0) 0x7f5f40001110 con 0x7f5f4c144b80 2023-08-27T11:27:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.399+0000 7f5f4b7fe700 1 --2- 172.21.15.112:0/2496807339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c0b3de0 0x7f5f4c06bfd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009840 tx=0x7f5f3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8426cc608dcec87e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.399+0000 7f5f4affd700 1 -- 172.21.15.112:0/2496807339 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c011830 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.399+0000 7f5f4affd700 1 -- 172.21.15.112:0/2496807339 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c011990 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.399+0000 7f5f4affd700 1 -- 172.21.15.112:0/2496807339 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c011d20 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.400+0000 7f5f5248d700 1 -- 172.21.15.112:0/2496807339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c0b3de0 msgr2=0x7f5f4c06bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.400+0000 7f5f5248d700 1 --2- 172.21.15.112:0/2496807339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c0b3de0 0x7f5f4c06bfd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009840 tx=0x7f5f3c009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.400+0000 7f5f5248d700 1 -- 172.21.15.112:0/2496807339 shutdown_connections 2023-08-27T11:27:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.400+0000 7f5f5248d700 1 --2- 172.21.15.112:0/2496807339 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c06c510 0x7f5f4c0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.400+0000 7f5f5248d700 1 --2- 172.21.15.112:0/2496807339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c0b3de0 0x7f5f4c06bfd0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.400+0000 7f5f5248d700 1 -- 172.21.15.112:0/2496807339 >> 172.21.15.112:0/2496807339 conn(0x7f5f4c0cdc50 msgr2=0x7f5f4c0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.400+0000 7f5f5248d700 1 -- 172.21.15.112:0/2496807339 shutdown_connections 2023-08-27T11:27:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.400+0000 7f5f5248d700 1 -- 172.21.15.112:0/2496807339 wait complete. 2023-08-27T11:27:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.401+0000 7f5f5248d700 1 Processor -- start 2023-08-27T11:27:07.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.401+0000 7f5f5248d700 1 -- start start 2023-08-27T11:27:07.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f5248d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c06c510 0x7f5f4c1f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f5248d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c0b3de0 0x7f5f4c1f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f5248d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f4c144b80 0x7f5f4c070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f5248d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c071840 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f5248d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c071b20 con 0x7f5f4c144b80 2023-08-27T11:27:07.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f5248d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c071e00 con 0x7f5f4c06c510 2023-08-27T11:27:07.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f4b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c0b3de0 0x7f5f4c1f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f4b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c0b3de0 0x7f5f4c1f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34222/0 (socket says 172.21.15.112:34222) 2023-08-27T11:27:07.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f4b7fe700 1 -- 172.21.15.112:0/1239533113 learned_addr learned my addr 172.21.15.112:0/1239533113 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:07.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f4bfff700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c06c510 0x7f5f4c1f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f50a2a700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f4c144b80 0x7f5f4c070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f4b7fe700 1 -- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c06c510 msgr2=0x7f5f4c1f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f4b7fe700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c06c510 0x7f5f4c1f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f4b7fe700 1 -- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f4c144b80 msgr2=0x7f5f4c070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f4b7fe700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f4c144b80 0x7f5f4c070ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f4b7fe700 1 -- 172.21.15.112:0/1239533113 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c0094a0 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f4bfff700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c06c510 0x7f5f4c1f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.402+0000 7f5f4b7fe700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c0b3de0 0x7f5f4c1f66d0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c000c00 tx=0x7f5f3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.403+0000 7f5f48ff9700 1 -- 172.21.15.112:0/1239533113 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0036a0 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.403+0000 7f5f48ff9700 1 -- 172.21.15.112:0/1239533113 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c01f070 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.403+0000 7f5f5248d700 1 -- 172.21.15.112:0/1239533113 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c0720e0 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.403+0000 7f5f48ff9700 1 -- 172.21.15.112:0/1239533113 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c003a20 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.403+0000 7f5f5248d700 1 -- 172.21.15.112:0/1239533113 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c143760 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.404+0000 7f5f5248d700 1 -- 172.21.15.112:0/1239533113 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c0b5210 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.404+0000 7f5f48ff9700 1 -- 172.21.15.112:0/1239533113 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f5f3c003b80 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.405+0000 7f5f48ff9700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f3806a440 0x7f5f3806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.405+0000 7f5f48ff9700 1 -- 172.21.15.112:0/1239533113 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5f3c021070 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.405+0000 7f5f4bfff700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f3806a440 0x7f5f3806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.406+0000 7f5f4bfff700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f3806a440 0x7f5f3806c900 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5f40007fd0 tx=0x7f5f40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.409+0000 7f5f48ff9700 1 -- 172.21.15.112:0/1239533113 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5f3c055e90 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.598+0000 7f5f5248d700 1 -- 172.21.15.112:0/1239533113 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5f4c0b4550 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.598+0000 7f5f48ff9700 1 -- 172.21.15.112:0/1239533113 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v102) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5f3c0594b0 con 0x7f5f4c0b3de0 2023-08-27T11:27:07.600 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:27:07.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.600+0000 7f5f5248d700 1 -- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f3806a440 msgr2=0x7f5f3806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.600+0000 7f5f5248d700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f3806a440 0x7f5f3806c900 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5f40007fd0 tx=0x7f5f40009040 comp rx=0 tx=0).stop 2023-08-27T11:27:07.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.600+0000 7f5f5248d700 1 -- 172.21.15.112:0/1239533113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c0b3de0 msgr2=0x7f5f4c1f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.600+0000 7f5f5248d700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c0b3de0 0x7f5f4c1f66d0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c000c00 tx=0x7f5f3c004750 comp rx=0 tx=0).stop 2023-08-27T11:27:07.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.600+0000 7f5f5248d700 1 -- 172.21.15.112:0/1239533113 shutdown_connections 2023-08-27T11:27:07.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.600+0000 7f5f5248d700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f3806a440 0x7f5f3806c900 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.600+0000 7f5f5248d700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f4c144b80 0x7f5f4c070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.600+0000 7f5f5248d700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c06c510 0x7f5f4c1f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.600+0000 7f5f5248d700 1 --2- 172.21.15.112:0/1239533113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c0b3de0 0x7f5f4c1f66d0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.600+0000 7f5f5248d700 1 -- 172.21.15.112:0/1239533113 >> 172.21.15.112:0/1239533113 conn(0x7f5f4c0cdc50 msgr2=0x7f5f4c0b68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:07.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.600+0000 7f5f5248d700 1 -- 172.21.15.112:0/1239533113 shutdown_connections 2023-08-27T11:27:07.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.600+0000 7f5f5248d700 1 -- 172.21.15.112:0/1239533113 wait complete. 2023-08-27T11:27:07.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:07.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.736+0000 7fbf94414700 1 Processor -- start 2023-08-27T11:27:07.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.736+0000 7fbf94414700 1 -- start start 2023-08-27T11:27:07.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.737+0000 7fbf94414700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf8c146c20 0x7fbf8c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.737+0000 7fbf94414700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbf8c0b3440 0x7fbf8c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.737+0000 7fbf94414700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8c0b5070 con 0x7fbf8c15e6e0 2023-08-27T11:27:07.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.737+0000 7fbf94414700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf8c147000 con 0x7fbf8c0b3440 2023-08-27T11:27:07.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.737+0000 7fbf94414700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c1614a0 con 0x7fbf8c146c20 2023-08-27T11:27:07.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.737+0000 7fbf929b1700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbf8c15e6e0 0x7fbf8c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56846/0 (socket says 172.21.15.112:56846) 2023-08-27T11:27:07.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.737+0000 7fbf929b1700 1 -- 172.21.15.112:0/2234858957 learned_addr learned my addr 172.21.15.112:0/2234858957 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:07.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721670905 0 0) 0x7fbf84001490 con 0x7fbf8c15e6e0 2023-08-27T11:27:07.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf921b0700 1 --2- 172.21.15.112:0/2234858957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf8c146c20 0x7fbf8c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf780035b0 con 0x7fbf8c15e6e0 2023-08-27T11:27:07.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf919af700 1 --2- 172.21.15.112:0/2234858957 >> v2:172.21.15.138:3300/0 conn(0x7fbf8c0b3440 0x7fbf8c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1530673257 0 0) 0x7fbf780035b0 con 0x7fbf8c15e6e0 2023-08-27T11:27:07.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf8c146c20 msgr2=0x7fbf8c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:07.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 --2- 172.21.15.112:0/2234858957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf8c146c20 0x7fbf8c160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 >> v2:172.21.15.138:3300/0 conn(0x7fbf8c0b3440 msgr2=0x7fbf8c15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:07.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 --2- 172.21.15.112:0/2234858957 >> v2:172.21.15.138:3300/0 conn(0x7fbf8c0b3440 0x7fbf8c15e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c161740 con 0x7fbf8c15e6e0 2023-08-27T11:27:07.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fbf84001b80 con 0x7fbf8c15e6e0 2023-08-27T11:27:07.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 >> v1:172.21.15.112:6789/0 conn(0x7fbf8c15e6e0 legacy=0x7fbf8c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 --2- 172.21.15.112:0/2234858957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf78003a00 0x7fbf78003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 --2- 172.21.15.112:0/2234858957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf8c146c20 0x7fbf78004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 --2- 172.21.15.112:0/2234858957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf78004d90 0x7fbf78005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c147000 con 0x7fbf78003a00 2023-08-27T11:27:07.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c1614a0 con 0x7fbf78004d90 2023-08-27T11:27:07.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7800ba10 con 0x7fbf8c146c20 2023-08-27T11:27:07.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf921b0700 1 --2- 172.21.15.112:0/2234858957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf78003a00 0x7fbf78003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf919af700 1 --2- 172.21.15.112:0/2234858957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf8c146c20 0x7fbf78004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf929b1700 1 --2- 172.21.15.112:0/2234858957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf78004d90 0x7fbf78005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf921b0700 1 -- 172.21.15.112:0/2234858957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf8c146c20 msgr2=0x7fbf78004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:07.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf921b0700 1 --2- 172.21.15.112:0/2234858957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf8c146c20 0x7fbf78004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf921b0700 1 -- 172.21.15.112:0/2234858957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf78004d90 msgr2=0x7fbf78005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:07.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf921b0700 1 --2- 172.21.15.112:0/2234858957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf78004d90 0x7fbf78005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf921b0700 1 -- 172.21.15.112:0/2234858957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf780035b0 con 0x7fbf78003a00 2023-08-27T11:27:07.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf919af700 1 --2- 172.21.15.112:0/2234858957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf8c146c20 0x7fbf78004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:07.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.738+0000 7fbf921b0700 1 --2- 172.21.15.112:0/2234858957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf78003a00 0x7fbf78003e20 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009b60 tx=0x7fbf78005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9121c1ea3a276ee server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.739+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c00e6c0 con 0x7fbf78003a00 2023-08-27T11:27:07.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.739+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c00e820 con 0x7fbf78003a00 2023-08-27T11:27:07.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.739+0000 7fbf911ae700 1 -- 172.21.15.112:0/2234858957 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c00eb90 con 0x7fbf78003a00 2023-08-27T11:27:07.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.739+0000 7fbf94414700 1 -- 172.21.15.112:0/2234858957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf78003a00 msgr2=0x7fbf78003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.739+0000 7fbf94414700 1 --2- 172.21.15.112:0/2234858957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf78003a00 0x7fbf78003e20 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009b60 tx=0x7fbf78005880 comp rx=0 tx=0).stop 2023-08-27T11:27:07.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.739+0000 7fbf929b1700 1 -- 172.21.15.112:0/2234858957 reap_dead start 2023-08-27T11:27:07.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.739+0000 7fbf94414700 1 -- 172.21.15.112:0/2234858957 shutdown_connections 2023-08-27T11:27:07.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.739+0000 7fbf94414700 1 -- 172.21.15.112:0/2234858957 >> 172.21.15.112:0/2234858957 conn(0x7fbf8c0b1e90 msgr2=0x7fbf8c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:07.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.739+0000 7fbf94414700 1 -- 172.21.15.112:0/2234858957 shutdown_connections 2023-08-27T11:27:07.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.739+0000 7fbf94414700 1 -- 172.21.15.112:0/2234858957 wait complete. 2023-08-27T11:27:07.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.740+0000 7fbf94414700 1 Processor -- start 2023-08-27T11:27:07.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.740+0000 7fbf94414700 1 -- start start 2023-08-27T11:27:07.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf94414700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf78003a00 0x7fbf8c1f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf94414700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c144e50 0x7fbf8c1452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf94414700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf8c0ad6b0 0x7fbf8c0adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf94414700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c145810 con 0x7fbf8c144e50 2023-08-27T11:27:07.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf94414700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c1f3cd0 con 0x7fbf8c0ad6b0 2023-08-27T11:27:07.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf94414700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c0ae3e0 con 0x7fbf78003a00 2023-08-27T11:27:07.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf919af700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c144e50 0x7fbf8c1452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf919af700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c144e50 0x7fbf8c1452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34242/0 (socket says 172.21.15.112:34242) 2023-08-27T11:27:07.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf919af700 1 -- 172.21.15.112:0/2142383255 learned_addr learned my addr 172.21.15.112:0/2142383255 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:07.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf921b0700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf78003a00 0x7fbf8c1f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf919af700 1 -- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf78003a00 msgr2=0x7fbf8c1f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf929b1700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf8c0ad6b0 0x7fbf8c0adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf919af700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf78003a00 0x7fbf8c1f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf919af700 1 -- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf8c0ad6b0 msgr2=0x7fbf8c0adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf919af700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf8c0ad6b0 0x7fbf8c0adb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.741+0000 7fbf919af700 1 -- 172.21.15.112:0/2142383255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c0ae6c0 con 0x7fbf8c144e50 2023-08-27T11:27:07.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.742+0000 7fbf919af700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c144e50 0x7fbf8c1452d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fbf880118d0 tx=0x7fbf88011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.742+0000 7fbf82ffd700 1 -- 172.21.15.112:0/2142383255 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88004220 con 0x7fbf8c144e50 2023-08-27T11:27:07.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.742+0000 7fbf82ffd700 1 -- 172.21.15.112:0/2142383255 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf88004380 con 0x7fbf8c144e50 2023-08-27T11:27:07.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.742+0000 7fbf94414700 1 -- 172.21.15.112:0/2142383255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8c0ae9a0 con 0x7fbf8c144e50 2023-08-27T11:27:07.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.742+0000 7fbf82ffd700 1 -- 172.21.15.112:0/2142383255 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf880036a0 con 0x7fbf8c144e50 2023-08-27T11:27:07.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.742+0000 7fbf94414700 1 -- 172.21.15.112:0/2142383255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8c0ae080 con 0x7fbf8c144e50 2023-08-27T11:27:07.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.743+0000 7fbf94414700 1 -- 172.21.15.112:0/2142383255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf8c067e90 con 0x7fbf8c144e50 2023-08-27T11:27:07.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.743+0000 7fbf82ffd700 1 -- 172.21.15.112:0/2142383255 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fbf88003800 con 0x7fbf8c144e50 2023-08-27T11:27:07.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.743+0000 7fbf82ffd700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf7006a440 0x7fbf7006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.744+0000 7fbf82ffd700 1 -- 172.21.15.112:0/2142383255 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf88088ea0 con 0x7fbf8c144e50 2023-08-27T11:27:07.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.744+0000 7fbf921b0700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf7006a440 0x7fbf7006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.744+0000 7fbf921b0700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf7006a440 0x7fbf7006c900 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c000c00 tx=0x7fbf7c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.748+0000 7fbf82ffd700 1 -- 172.21.15.112:0/2142383255 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbf88054210 con 0x7fbf8c144e50 2023-08-27T11:27:07.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.936+0000 7fbf94414700 1 -- 172.21.15.112:0/2142383255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf8c067a20 con 0x7fbf8c144e50 2023-08-27T11:27:07.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.936+0000 7fbf82ffd700 1 -- 172.21.15.112:0/2142383255 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf880220a0 con 0x7fbf8c144e50 2023-08-27T11:27:07.939 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:27:07.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.938+0000 7fbf94414700 1 -- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf7006a440 msgr2=0x7fbf7006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.938+0000 7fbf94414700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf7006a440 0x7fbf7006c900 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c000c00 tx=0x7fbf7c008030 comp rx=0 tx=0).stop 2023-08-27T11:27:07.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.938+0000 7fbf94414700 1 -- 172.21.15.112:0/2142383255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c144e50 msgr2=0x7fbf8c1452d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.938+0000 7fbf94414700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c144e50 0x7fbf8c1452d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fbf880118d0 tx=0x7fbf88011c90 comp rx=0 tx=0).stop 2023-08-27T11:27:07.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.938+0000 7fbf94414700 1 -- 172.21.15.112:0/2142383255 shutdown_connections 2023-08-27T11:27:07.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.938+0000 7fbf94414700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf7006a440 0x7fbf7006c900 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.938+0000 7fbf94414700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf8c0ad6b0 0x7fbf8c0adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.938+0000 7fbf94414700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf78003a00 0x7fbf8c1f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.938+0000 7fbf94414700 1 --2- 172.21.15.112:0/2142383255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c144e50 0x7fbf8c1452d0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.938+0000 7fbf94414700 1 -- 172.21.15.112:0/2142383255 >> 172.21.15.112:0/2142383255 conn(0x7fbf8c0b1e90 msgr2=0x7fbf8c06fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:07.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.938+0000 7fbf94414700 1 -- 172.21.15.112:0/2142383255 shutdown_connections 2023-08-27T11:27:07.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:07.938+0000 7fbf94414700 1 -- 172.21.15.112:0/2142383255 wait complete. 2023-08-27T11:27:07.952 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:27:08.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.074+0000 7f08a643b700 1 Processor -- start 2023-08-27T11:27:08.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.074+0000 7f08a643b700 1 -- start start 2023-08-27T11:27:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.075+0000 7f08a643b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f08a00b3440 0x7f08a015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.075+0000 7f08a643b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08a015e6e0 0x7f08a006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.075+0000 7f08a643b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a00b5070 con 0x7f08a0146c20 2023-08-27T11:27:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.075+0000 7f08a643b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a0147000 con 0x7f08a015e6e0 2023-08-27T11:27:08.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.075+0000 7f08a643b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a0161740 con 0x7f08a00b3440 2023-08-27T11:27:08.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.075+0000 7f08a49d8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08a015e6e0 0x7f08a006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.075+0000 7f08a49d8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08a015e6e0 0x7f08a006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34252/0 (socket says 172.21.15.112:34252) 2023-08-27T11:27:08.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.075+0000 7f08a49d8700 1 -- 172.21.15.112:0/2444083554 learned_addr learned my addr 172.21.15.112:0/2444083554 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:08.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.075+0000 7f089f7fe700 1 --2- 172.21.15.112:0/2444083554 >> v2:172.21.15.138:3300/0 conn(0x7f08a00b3440 0x7f08a015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.075+0000 7f08a49d8700 1 -- 172.21.15.112:0/2444083554 >> v2:172.21.15.138:3300/0 conn(0x7f08a00b3440 msgr2=0x7f08a015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:08.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.075+0000 7f08a49d8700 1 --2- 172.21.15.112:0/2444083554 >> v2:172.21.15.138:3300/0 conn(0x7f08a00b3440 0x7f08a015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.075+0000 7f08a49d8700 1 -- 172.21.15.112:0/2444083554 >> v1:172.21.15.138:6789/0 conn(0x7f08a0146c20 legacy=0x7f08a0160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.075+0000 7f08a49d8700 1 -- 172.21.15.112:0/2444083554 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a0161a20 con 0x7f08a015e6e0 2023-08-27T11:27:08.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.076+0000 7f08a49d8700 1 --2- 172.21.15.112:0/2444083554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a015e6e0 0x7f08a006f790 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f08a00b3f00 tx=0x7f089800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3fdc1c581ba826 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.076+0000 7f089effd700 1 -- 172.21.15.112:0/2444083554 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089800e070 con 0x7f08a015e6e0 2023-08-27T11:27:08.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.076+0000 7f089effd700 1 -- 172.21.15.112:0/2444083554 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089800b9c0 con 0x7f08a015e6e0 2023-08-27T11:27:08.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.076+0000 7f089effd700 1 -- 172.21.15.112:0/2444083554 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089800bd70 con 0x7f08a015e6e0 2023-08-27T11:27:08.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.076+0000 7f08a643b700 1 -- 172.21.15.112:0/2444083554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a015e6e0 msgr2=0x7f08a006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.076+0000 7f08a643b700 1 --2- 172.21.15.112:0/2444083554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a015e6e0 0x7f08a006f790 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f08a00b3f00 tx=0x7f089800b330 comp rx=0 tx=0).stop 2023-08-27T11:27:08.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.076+0000 7f08a643b700 1 -- 172.21.15.112:0/2444083554 shutdown_connections 2023-08-27T11:27:08.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.076+0000 7f08a643b700 1 --2- 172.21.15.112:0/2444083554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a015e6e0 0x7f08a006f790 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.076+0000 7f08a643b700 1 --2- 172.21.15.112:0/2444083554 >> v2:172.21.15.138:3300/0 conn(0x7f08a00b3440 0x7f08a015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.076+0000 7f08a643b700 1 -- 172.21.15.112:0/2444083554 >> 172.21.15.112:0/2444083554 conn(0x7f08a00b1e90 msgr2=0x7f08a00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:08.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.076+0000 7f08a643b700 1 -- 172.21.15.112:0/2444083554 shutdown_connections 2023-08-27T11:27:08.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.076+0000 7f08a643b700 1 -- 172.21.15.112:0/2444083554 wait complete. 2023-08-27T11:27:08.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.077+0000 7f08a643b700 1 Processor -- start 2023-08-27T11:27:08.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.078+0000 7f08a643b700 1 -- start start 2023-08-27T11:27:08.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.078+0000 7f08a643b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08a00b3440 0x7f08a00aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.078+0000 7f08a643b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08a0146c20 0x7f08a015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.078+0000 7f08a643b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a015e6e0 0x7f08a00ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.078+0000 7f08a643b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a00aea20 con 0x7f08a015e6e0 2023-08-27T11:27:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.078+0000 7f08a643b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a00af380 con 0x7f08a0146c20 2023-08-27T11:27:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.078+0000 7f08a643b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a00adda0 con 0x7f08a00b3440 2023-08-27T11:27:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.078+0000 7f08a49d8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a015e6e0 0x7f08a00ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.078+0000 7f08a49d8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a015e6e0 0x7f08a00ad4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34258/0 (socket says 172.21.15.112:34258) 2023-08-27T11:27:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.078+0000 7f08a49d8700 1 -- 172.21.15.112:0/4204068743 learned_addr learned my addr 172.21.15.112:0/4204068743 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f089f7fe700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08a0146c20 0x7f08a015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f089ffff700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08a00b3440 0x7f08a00aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f08a49d8700 1 -- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08a00b3440 msgr2=0x7f08a00aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f08a49d8700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08a00b3440 0x7f08a00aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f08a49d8700 1 -- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08a0146c20 msgr2=0x7f08a015ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f08a49d8700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08a0146c20 0x7f08a015ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f08a49d8700 1 -- 172.21.15.112:0/4204068743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0898009b30 con 0x7f08a015e6e0 2023-08-27T11:27:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f089f7fe700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08a0146c20 0x7f08a015ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f089ffff700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08a00b3440 0x7f08a00aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f08a49d8700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a015e6e0 0x7f08a00ad4c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0898012000 tx=0x7f0898004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f089cff9700 1 -- 172.21.15.112:0/4204068743 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08980099b0 con 0x7f08a015e6e0 2023-08-27T11:27:08.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f089cff9700 1 -- 172.21.15.112:0/4204068743 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08980092e0 con 0x7f08a015e6e0 2023-08-27T11:27:08.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f08a643b700 1 -- 172.21.15.112:0/4204068743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a00ae080 con 0x7f08a015e6e0 2023-08-27T11:27:08.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f089cff9700 1 -- 172.21.15.112:0/4204068743 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089801b630 con 0x7f08a015e6e0 2023-08-27T11:27:08.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.079+0000 7f08a643b700 1 -- 172.21.15.112:0/4204068743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a00ae640 con 0x7f08a015e6e0 2023-08-27T11:27:08.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.080+0000 7f08a643b700 1 -- 172.21.15.112:0/4204068743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a0067e90 con 0x7f08a015e6e0 2023-08-27T11:27:08.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.080+0000 7f089cff9700 1 -- 172.21.15.112:0/4204068743 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7f089801b790 con 0x7f08a015e6e0 2023-08-27T11:27:08.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.081+0000 7f089cff9700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f088c06a440 0x7f088c06c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.081+0000 7f089cff9700 1 -- 172.21.15.112:0/4204068743 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f089808a0c0 con 0x7f08a015e6e0 2023-08-27T11:27:08.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.081+0000 7f089ffff700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f088c06a440 0x7f088c06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.082+0000 7f089ffff700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f088c06a440 0x7f088c06c900 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0894001710 tx=0x7f0894009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.085+0000 7f089cff9700 1 -- 172.21.15.112:0/4204068743 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0898055320 con 0x7f08a015e6e0 2023-08-27T11:27:08.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.271+0000 7f08a643b700 1 -- 172.21.15.112:0/4204068743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08a0067a20 con 0x7f08a015e6e0 2023-08-27T11:27:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.271+0000 7f089cff9700 1 -- 172.21.15.112:0/4204068743 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0898058940 con 0x7f08a015e6e0 2023-08-27T11:27:08.273 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:27:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.273+0000 7f08a643b700 1 -- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f088c06a440 msgr2=0x7f088c06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.273+0000 7f08a643b700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f088c06a440 0x7f088c06c900 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0894001710 tx=0x7f0894009040 comp rx=0 tx=0).stop 2023-08-27T11:27:08.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.273+0000 7f08a643b700 1 -- 172.21.15.112:0/4204068743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a015e6e0 msgr2=0x7f08a00ad4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.273+0000 7f08a643b700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a015e6e0 0x7f08a00ad4c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0898012000 tx=0x7f0898004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.273+0000 7f08a643b700 1 -- 172.21.15.112:0/4204068743 shutdown_connections 2023-08-27T11:27:08.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.273+0000 7f08a643b700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f088c06a440 0x7f088c06c900 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.273+0000 7f08a643b700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a015e6e0 0x7f08a00ad4c0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.273+0000 7f08a643b700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08a00b3440 0x7f08a00aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.273+0000 7f08a643b700 1 --2- 172.21.15.112:0/4204068743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08a0146c20 0x7f08a015ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.273+0000 7f08a643b700 1 -- 172.21.15.112:0/4204068743 >> 172.21.15.112:0/4204068743 conn(0x7f08a00b1e90 msgr2=0x7f08a0142a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:08.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.274+0000 7f08a643b700 1 -- 172.21.15.112:0/4204068743 shutdown_connections 2023-08-27T11:27:08.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.274+0000 7f08a643b700 1 -- 172.21.15.112:0/4204068743 wait complete. 2023-08-27T11:27:08.287 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:27:08.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.410+0000 7fac2d8a7700 1 Processor -- start 2023-08-27T11:27:08.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.410+0000 7fac2d8a7700 1 -- start start 2023-08-27T11:27:08.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac2d8a7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac280715c0 0x7fac280b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac2d8a7700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac280d1500 con 0x7fac28146bf0 2023-08-27T11:27:08.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac2d8a7700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac280d0460 con 0x7fac28146fd0 2023-08-27T11:27:08.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac2d8a7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac280d0710 con 0x7fac280715c0 2023-08-27T11:27:08.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac26ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fac28146bf0 0x7fac280cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56876/0 (socket says 172.21.15.112:56876) 2023-08-27T11:27:08.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac26ffd700 1 -- 172.21.15.112:0/1880375849 learned_addr learned my addr 172.21.15.112:0/1880375849 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:08.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac277fe700 1 --2- 172.21.15.112:0/1880375849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac280715c0 0x7fac280b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac25ffb700 1 -- 172.21.15.112:0/1880375849 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094171868 0 0) 0x7fac280d1500 con 0x7fac28146bf0 2023-08-27T11:27:08.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac25ffb700 1 -- 172.21.15.112:0/1880375849 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac100035b0 con 0x7fac28146bf0 2023-08-27T11:27:08.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac25ffb700 1 -- 172.21.15.112:0/1880375849 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1012638920 0 0) 0x7fac280d0460 con 0x7fac28146fd0 2023-08-27T11:27:08.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac25ffb700 1 -- 172.21.15.112:0/1880375849 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac280d1500 con 0x7fac28146fd0 2023-08-27T11:27:08.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac25ffb700 1 -- 172.21.15.112:0/1880375849 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1338535767 0 0) 0x7fac100035b0 con 0x7fac28146bf0 2023-08-27T11:27:08.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac277fe700 1 -- 172.21.15.112:0/1880375849 >> v1:172.21.15.138:6789/0 conn(0x7fac28146fd0 legacy=0x7fac28070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac277fe700 1 -- 172.21.15.112:0/1880375849 >> v1:172.21.15.112:6789/0 conn(0x7fac28146bf0 legacy=0x7fac280cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac277fe700 1 -- 172.21.15.112:0/1880375849 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac280d09f0 con 0x7fac280715c0 2023-08-27T11:27:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac277fe700 1 --2- 172.21.15.112:0/1880375849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac280715c0 0x7fac280b5db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fac20009dc0 tx=0x7fac2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff34bde1426d1f99 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac25ffb700 1 -- 172.21.15.112:0/1880375849 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2000e070 con 0x7fac280715c0 2023-08-27T11:27:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac25ffb700 1 -- 172.21.15.112:0/1880375849 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac20003a20 con 0x7fac280715c0 2023-08-27T11:27:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac25ffb700 1 -- 172.21.15.112:0/1880375849 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c001670 con 0x7fac280715c0 2023-08-27T11:27:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac2d8a7700 1 -- 172.21.15.112:0/1880375849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac280715c0 msgr2=0x7fac280b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac2d8a7700 1 --2- 172.21.15.112:0/1880375849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac280715c0 0x7fac280b5db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fac20009dc0 tx=0x7fac2000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac2d8a7700 1 -- 172.21.15.112:0/1880375849 shutdown_connections 2023-08-27T11:27:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac2d8a7700 1 --2- 172.21.15.112:0/1880375849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac280715c0 0x7fac280b5db0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac2d8a7700 1 -- 172.21.15.112:0/1880375849 >> 172.21.15.112:0/1880375849 conn(0x7fac2806c450 msgr2=0x7fac2806f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac2d8a7700 1 -- 172.21.15.112:0/1880375849 shutdown_connections 2023-08-27T11:27:08.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.411+0000 7fac2d8a7700 1 -- 172.21.15.112:0/1880375849 wait complete. 2023-08-27T11:27:08.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.413+0000 7fac2d8a7700 1 Processor -- start 2023-08-27T11:27:08.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.413+0000 7fac2d8a7700 1 -- start start 2023-08-27T11:27:08.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac2d8a7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac280715c0 0x7fac281604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac2d8a7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac28146bf0 0x7fac2814dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac2d8a7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac28146fd0 0x7fac2814e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac2d8a7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac280b7740 con 0x7fac28146fd0 2023-08-27T11:27:08.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac2d8a7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2813e540 con 0x7fac280715c0 2023-08-27T11:27:08.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac2d8a7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac280b7460 con 0x7fac28146bf0 2023-08-27T11:27:08.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac277fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac28146fd0 0x7fac2814e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac277fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac28146fd0 0x7fac2814e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34272/0 (socket says 172.21.15.112:34272) 2023-08-27T11:27:08.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac277fe700 1 -- 172.21.15.112:0/4218921009 learned_addr learned my addr 172.21.15.112:0/4218921009 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:08.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac26ffd700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac280715c0 0x7fac281604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac267fc700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac28146bf0 0x7fac2814dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac277fe700 1 -- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac28146bf0 msgr2=0x7fac2814dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac277fe700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac28146bf0 0x7fac2814dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac277fe700 1 -- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac280715c0 msgr2=0x7fac281604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac277fe700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac280715c0 0x7fac281604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac277fe700 1 -- 172.21.15.112:0/4218921009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac20009a70 con 0x7fac28146fd0 2023-08-27T11:27:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac277fe700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac28146fd0 0x7fac2814e1b0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fac20004000 tx=0x7fac20004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac0ffff700 1 -- 172.21.15.112:0/4218921009 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac200044c0 con 0x7fac28146fd0 2023-08-27T11:27:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac0ffff700 1 -- 172.21.15.112:0/4218921009 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac20004660 con 0x7fac28146fd0 2023-08-27T11:27:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac2d8a7700 1 -- 172.21.15.112:0/4218921009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2814bd50 con 0x7fac28146fd0 2023-08-27T11:27:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac0ffff700 1 -- 172.21.15.112:0/4218921009 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2001d470 con 0x7fac28146fd0 2023-08-27T11:27:08.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.414+0000 7fac2d8a7700 1 -- 172.21.15.112:0/4218921009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2814c310 con 0x7fac28146fd0 2023-08-27T11:27:08.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.415+0000 7fac2d8a7700 1 -- 172.21.15.112:0/4218921009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac28067e90 con 0x7fac28146fd0 2023-08-27T11:27:08.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.416+0000 7fac0ffff700 1 -- 172.21.15.112:0/4218921009 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) v1 ==== 88647+0+0 (secure 0 0 0) 0x7fac2001d5d0 con 0x7fac28146fd0 2023-08-27T11:27:08.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.416+0000 7fac0ffff700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fac1406a3b0 0x7fac1406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.416+0000 7fac0ffff700 1 -- 172.21.15.112:0/4218921009 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fac2008a730 con 0x7fac28146fd0 2023-08-27T11:27:08.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.416+0000 7fac26ffd700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fac1406a3b0 0x7fac1406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.417+0000 7fac26ffd700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fac1406a3b0 0x7fac1406c870 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fac1c001eb0 tx=0x7fac1c001d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.421+0000 7fac0ffff700 1 -- 172.21.15.112:0/4218921009 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fac20055b90 con 0x7fac28146fd0 2023-08-27T11:27:08.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.610+0000 7fac2d8a7700 1 -- 172.21.15.112:0/4218921009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac280b3b90 con 0x7fac28146fd0 2023-08-27T11:27:08.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.610+0000 7fac0ffff700 1 -- 172.21.15.112:0/4218921009 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac200591b0 con 0x7fac28146fd0 2023-08-27T11:27:08.612 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:27:08.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.612+0000 7fac2d8a7700 1 -- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fac1406a3b0 msgr2=0x7fac1406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.612+0000 7fac2d8a7700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fac1406a3b0 0x7fac1406c870 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fac1c001eb0 tx=0x7fac1c001d80 comp rx=0 tx=0).stop 2023-08-27T11:27:08.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.612+0000 7fac2d8a7700 1 -- 172.21.15.112:0/4218921009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac28146fd0 msgr2=0x7fac2814e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.612+0000 7fac2d8a7700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac28146fd0 0x7fac2814e1b0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fac20004000 tx=0x7fac20004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.612+0000 7fac2d8a7700 1 -- 172.21.15.112:0/4218921009 shutdown_connections 2023-08-27T11:27:08.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.612+0000 7fac2d8a7700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fac1406a3b0 0x7fac1406c870 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.612+0000 7fac2d8a7700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac28146fd0 0x7fac2814e1b0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.612+0000 7fac2d8a7700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac280715c0 0x7fac281604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.612+0000 7fac2d8a7700 1 --2- 172.21.15.112:0/4218921009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac28146bf0 0x7fac2814dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.612+0000 7fac2d8a7700 1 -- 172.21.15.112:0/4218921009 >> 172.21.15.112:0/4218921009 conn(0x7fac2806c450 msgr2=0x7fac280d1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:08.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.613+0000 7fac2d8a7700 1 -- 172.21.15.112:0/4218921009 shutdown_connections 2023-08-27T11:27:08.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.613+0000 7fac2d8a7700 1 -- 172.21.15.112:0/4218921009 wait complete. 2023-08-27T11:27:08.626 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:27:08.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.748+0000 7f57a99a9700 1 Processor -- start 2023-08-27T11:27:08.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.748+0000 7f57a99a9700 1 -- start start 2023-08-27T11:27:08.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a4157880 0x7f57a4160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f57a4161430 0x7f57a40d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f57a40d0c60 0x7f57a415d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a41618b0 con 0x7f57a40d0c60 2023-08-27T11:27:08.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a415f110 con 0x7f57a4161430 2023-08-27T11:27:08.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a40d1490 con 0x7f57a4157880 2023-08-27T11:27:08.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a37fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f57a40d0c60 0x7f57a415d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a37fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f57a40d0c60 0x7f57a415d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34278/0 (socket says 172.21.15.112:34278) 2023-08-27T11:27:08.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a37fe700 1 -- 172.21.15.112:0/2501507292 learned_addr learned my addr 172.21.15.112:0/2501507292 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:08.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a2ffd700 1 --2- 172.21.15.112:0/2501507292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a4157880 0x7f57a4160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a27fc700 1 --2- 172.21.15.112:0/2501507292 >> v2:172.21.15.138:3300/0 conn(0x7f57a4161430 0x7f57a40d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a37fe700 1 -- 172.21.15.112:0/2501507292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a4157880 msgr2=0x7f57a4160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:08.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a37fe700 1 --2- 172.21.15.112:0/2501507292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a4157880 0x7f57a4160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a37fe700 1 -- 172.21.15.112:0/2501507292 >> v2:172.21.15.138:3300/0 conn(0x7f57a4161430 msgr2=0x7f57a40d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:08.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a37fe700 1 --2- 172.21.15.112:0/2501507292 >> v2:172.21.15.138:3300/0 conn(0x7f57a4161430 0x7f57a40d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a37fe700 1 -- 172.21.15.112:0/2501507292 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a40d1770 con 0x7f57a40d0c60 2023-08-27T11:27:08.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a27fc700 1 --2- 172.21.15.112:0/2501507292 >> v2:172.21.15.138:3300/0 conn(0x7f57a4161430 0x7f57a40d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:08.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a2ffd700 1 --2- 172.21.15.112:0/2501507292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a4157880 0x7f57a4160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:08.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a37fe700 1 --2- 172.21.15.112:0/2501507292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a40d0c60 0x7f57a415d920 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f57a40720d0 tx=0x7f579c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3342338db1850747 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a1ffb700 1 -- 172.21.15.112:0/2501507292 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c00e070 con 0x7f57a40d0c60 2023-08-27T11:27:08.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a1ffb700 1 -- 172.21.15.112:0/2501507292 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c00b9c0 con 0x7f57a40d0c60 2023-08-27T11:27:08.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a1ffb700 1 -- 172.21.15.112:0/2501507292 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c00bd70 con 0x7f57a40d0c60 2023-08-27T11:27:08.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 -- 172.21.15.112:0/2501507292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a40d0c60 msgr2=0x7f57a415d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 --2- 172.21.15.112:0/2501507292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a40d0c60 0x7f57a415d920 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f57a40720d0 tx=0x7f579c00b330 comp rx=0 tx=0).stop 2023-08-27T11:27:08.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 -- 172.21.15.112:0/2501507292 shutdown_connections 2023-08-27T11:27:08.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 --2- 172.21.15.112:0/2501507292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a40d0c60 0x7f57a415d920 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 --2- 172.21.15.112:0/2501507292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a4157880 0x7f57a4160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 --2- 172.21.15.112:0/2501507292 >> v2:172.21.15.138:3300/0 conn(0x7f57a4161430 0x7f57a40d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 -- 172.21.15.112:0/2501507292 >> 172.21.15.112:0/2501507292 conn(0x7f57a406c880 msgr2=0x7f57a40717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:08.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 -- 172.21.15.112:0/2501507292 shutdown_connections 2023-08-27T11:27:08.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.750+0000 7f57a99a9700 1 -- 172.21.15.112:0/2501507292 wait complete. 2023-08-27T11:27:08.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.752+0000 7f57a99a9700 1 Processor -- start 2023-08-27T11:27:08.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.752+0000 7f57a99a9700 1 -- start start 2023-08-27T11:27:08.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a99a9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a40d0c60 0x7f57a41534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a99a9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a4157880 0x7f57a4154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a99a9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a4161430 0x7f57a4156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a99a9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a4154740 con 0x7f57a40d0c60 2023-08-27T11:27:08.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a99a9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a4153ac0 con 0x7f57a4157880 2023-08-27T11:27:08.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a99a9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a4153da0 con 0x7f57a4161430 2023-08-27T11:27:08.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a40d0c60 0x7f57a41534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a40d0c60 0x7f57a41534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34280/0 (socket says 172.21.15.112:34280) 2023-08-27T11:27:08.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a2ffd700 1 -- 172.21.15.112:0/276237459 learned_addr learned my addr 172.21.15.112:0/276237459 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:08.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a27fc700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a4157880 0x7f57a4154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a2ffd700 1 -- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a4161430 msgr2=0x7f57a4156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a37fe700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a4161430 0x7f57a4156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a2ffd700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a4161430 0x7f57a4156130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a2ffd700 1 -- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a4157880 msgr2=0x7f57a4154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a2ffd700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a4157880 0x7f57a4154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a2ffd700 1 -- 172.21.15.112:0/276237459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c009b30 con 0x7f57a40d0c60 2023-08-27T11:27:08.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a27fc700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a4157880 0x7f57a4154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:08.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.753+0000 7f57a2ffd700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a40d0c60 0x7f57a41534e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f579800d800 tx=0x7f579800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.754+0000 7f578bfff700 1 -- 172.21.15.112:0/276237459 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798004030 con 0x7f57a40d0c60 2023-08-27T11:27:08.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.754+0000 7f578bfff700 1 -- 172.21.15.112:0/276237459 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798004190 con 0x7f57a40d0c60 2023-08-27T11:27:08.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.754+0000 7f578bfff700 1 -- 172.21.15.112:0/276237459 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798004520 con 0x7f57a40d0c60 2023-08-27T11:27:08.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.754+0000 7f57a99a9700 1 -- 172.21.15.112:0/276237459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a4154080 con 0x7f57a40d0c60 2023-08-27T11:27:08.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.754+0000 7f57a99a9700 1 -- 172.21.15.112:0/276237459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a4156670 con 0x7f57a40d0c60 2023-08-27T11:27:08.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.754+0000 7f57a99a9700 1 -- 172.21.15.112:0/276237459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a4154880 con 0x7f57a40d0c60 2023-08-27T11:27:08.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.755+0000 7f578bfff700 1 -- 172.21.15.112:0/276237459 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f57980036a0 con 0x7f57a40d0c60 2023-08-27T11:27:08.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.755+0000 7f578bfff700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f579006a500 0x7f579006c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.755+0000 7f578bfff700 1 -- 172.21.15.112:0/276237459 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5798089150 con 0x7f57a40d0c60 2023-08-27T11:27:08.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.756+0000 7f57a27fc700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f579006a500 0x7f579006c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.756+0000 7f57a27fc700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f579006a500 0x7f579006c9c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5794009960 tx=0x7f5794008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.761+0000 7f578bfff700 1 -- 172.21.15.112:0/276237459 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f57a4154880 con 0x7f57a40d0c60 2023-08-27T11:27:08.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.966+0000 7f57a99a9700 1 -- 172.21.15.112:0/276237459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f57a4154880 con 0x7f57a40d0c60 2023-08-27T11:27:08.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.967+0000 7f578bfff700 1 -- 172.21.15.112:0/276237459 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v102) v1 ==== 80+0+0 (secure 0 0 0) 0x7f57a4154880 con 0x7f57a40d0c60 2023-08-27T11:27:08.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.968+0000 7f57a99a9700 1 -- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f579006a500 msgr2=0x7f579006c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.968+0000 7f57a99a9700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f579006a500 0x7f579006c9c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5794009960 tx=0x7f5794008040 comp rx=0 tx=0).stop 2023-08-27T11:27:08.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.968+0000 7f57a99a9700 1 -- 172.21.15.112:0/276237459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a40d0c60 msgr2=0x7f57a41534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.968+0000 7f57a99a9700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a40d0c60 0x7f57a41534e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f579800d800 tx=0x7f579800dbc0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.969+0000 7f57a99a9700 1 -- 172.21.15.112:0/276237459 shutdown_connections 2023-08-27T11:27:08.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.969+0000 7f57a99a9700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f579006a500 0x7f579006c9c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.969+0000 7f57a99a9700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a4161430 0x7f57a4156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.969+0000 7f57a99a9700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a40d0c60 0x7f57a41534e0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.969+0000 7f57a99a9700 1 --2- 172.21.15.112:0/276237459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a4157880 0x7f57a4154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.969+0000 7f57a99a9700 1 -- 172.21.15.112:0/276237459 >> 172.21.15.112:0/276237459 conn(0x7f57a406c880 msgr2=0x7f57a415cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:08.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.969+0000 7f57a99a9700 1 -- 172.21.15.112:0/276237459 shutdown_connections 2023-08-27T11:27:08.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:08.969+0000 7f57a99a9700 1 -- 172.21.15.112:0/276237459 wait complete. 2023-08-27T11:27:08.972 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:27:08.981 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:27:08.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:08.982 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:09.031 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:27:09.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:09.032 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:27:09.049 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:27:09.049 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:09.070 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:27:09.071 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:09.132 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:09.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.251+0000 7fbff359d700 1 Processor -- start 2023-08-27T11:27:09.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.251+0000 7fbff359d700 1 -- start start 2023-08-27T11:27:09.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.252+0000 7fbff359d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfec06adb0 0x7fbfec06b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.252+0000 7fbff359d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfec06b750 con 0x7fbfec157890 2023-08-27T11:27:09.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.252+0000 7fbff359d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfec0708a0 con 0x7fbfec15e980 2023-08-27T11:27:09.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.252+0000 7fbff359d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfec0cc7f0 con 0x7fbfec06adb0 2023-08-27T11:27:09.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.252+0000 7fbff1d9a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfec06adb0 0x7fbfec06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.252+0000 7fbff1d9a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfec06adb0 0x7fbfec06b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34290/0 (socket says 172.21.15.112:34290) 2023-08-27T11:27:09.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.252+0000 7fbff259b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbfec157890 0x7fbfec0715d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56902/0 (socket says 172.21.15.112:56902) 2023-08-27T11:27:09.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.252+0000 7fbff1d9a700 1 -- 172.21.15.112:0/1989082855 learned_addr learned my addr 172.21.15.112:0/1989082855 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:09.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.253+0000 7fbff1d9a700 1 -- 172.21.15.112:0/1989082855 >> v1:172.21.15.138:6789/0 conn(0x7fbfec15e980 legacy=0x7fbfec15edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:09.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.253+0000 7fbff1d9a700 1 -- 172.21.15.112:0/1989082855 >> v1:172.21.15.112:6789/0 conn(0x7fbfec157890 legacy=0x7fbfec0715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.253+0000 7fbff1d9a700 1 -- 172.21.15.112:0/1989082855 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec071140 con 0x7fbfec06adb0 2023-08-27T11:27:09.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.253+0000 7fbff1599700 1 -- 172.21.15.112:0/1989082855 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601096396 0 0) 0x7fbfec06b750 con 0x7fbfec157890 2023-08-27T11:27:09.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.253+0000 7fbff1d9a700 1 --2- 172.21.15.112:0/1989082855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfec06adb0 0x7fbfec06b210 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80098f0 tx=0x7fbfe8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c63c5c32338edc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:09.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.253+0000 7fbff1599700 1 -- 172.21.15.112:0/1989082855 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8004230 con 0x7fbfec06adb0 2023-08-27T11:27:09.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.253+0000 7fbff1599700 1 -- 172.21.15.112:0/1989082855 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe8004390 con 0x7fbfec06adb0 2023-08-27T11:27:09.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.253+0000 7fbff1599700 1 -- 172.21.15.112:0/1989082855 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe80036a0 con 0x7fbfec06adb0 2023-08-27T11:27:09.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.253+0000 7fbff359d700 1 -- 172.21.15.112:0/1989082855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfec06adb0 msgr2=0x7fbfec06b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.253+0000 7fbff359d700 1 --2- 172.21.15.112:0/1989082855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfec06adb0 0x7fbfec06b210 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80098f0 tx=0x7fbfe8009c00 comp rx=0 tx=0).stop 2023-08-27T11:27:09.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.253+0000 7fbff359d700 1 -- 172.21.15.112:0/1989082855 shutdown_connections 2023-08-27T11:27:09.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.253+0000 7fbff359d700 1 --2- 172.21.15.112:0/1989082855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfec06adb0 0x7fbfec06b210 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.253+0000 7fbff359d700 1 -- 172.21.15.112:0/1989082855 >> 172.21.15.112:0/1989082855 conn(0x7fbfec066010 msgr2=0x7fbfec0663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:09.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.254+0000 7fbff359d700 1 -- 172.21.15.112:0/1989082855 shutdown_connections 2023-08-27T11:27:09.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.254+0000 7fbff359d700 1 -- 172.21.15.112:0/1989082855 wait complete. 2023-08-27T11:27:09.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.255+0000 7fbff359d700 1 Processor -- start 2023-08-27T11:27:09.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.255+0000 7fbff359d700 1 -- start start 2023-08-27T11:27:09.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.255+0000 7fbff359d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfec06adb0 0x7fbfec1f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.255+0000 7fbff359d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfec157890 0x7fbfec1f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.255+0000 7fbff359d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfec15e980 0x7fbfec1527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.255+0000 7fbff359d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec15b0c0 con 0x7fbfec15e980 2023-08-27T11:27:09.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.255+0000 7fbff359d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec15b2d0 con 0x7fbfec06adb0 2023-08-27T11:27:09.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.255+0000 7fbff359d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec15b5b0 con 0x7fbfec157890 2023-08-27T11:27:09.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff2d9c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfec15e980 0x7fbfec1527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff2d9c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfec15e980 0x7fbfec1527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34298/0 (socket says 172.21.15.112:34298) 2023-08-27T11:27:09.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff2d9c700 1 -- 172.21.15.112:0/409765074 learned_addr learned my addr 172.21.15.112:0/409765074 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:09.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff259b700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfec06adb0 0x7fbfec1f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff1d9a700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfec157890 0x7fbfec1f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff2d9c700 1 -- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfec157890 msgr2=0x7fbfec1f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff2d9c700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfec157890 0x7fbfec1f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff2d9c700 1 -- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfec06adb0 msgr2=0x7fbfec1f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff2d9c700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfec06adb0 0x7fbfec1f3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff2d9c700 1 -- 172.21.15.112:0/409765074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe8009550 con 0x7fbfec15e980 2023-08-27T11:27:09.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff259b700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfec06adb0 0x7fbfec1f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:09.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff2d9c700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfec15e980 0x7fbfec1527e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400b990 tx=0x7fbfe400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:09.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbfe2ffd700 1 -- 172.21.15.112:0/409765074 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe400c9d0 con 0x7fbfec15e980 2023-08-27T11:27:09.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbfe2ffd700 1 -- 172.21.15.112:0/409765074 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe400cb30 con 0x7fbfec15e980 2023-08-27T11:27:09.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbfe2ffd700 1 -- 172.21.15.112:0/409765074 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4014440 con 0x7fbfec15e980 2023-08-27T11:27:09.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff359d700 1 -- 172.21.15.112:0/409765074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec15b890 con 0x7fbfec15e980 2023-08-27T11:27:09.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.256+0000 7fbff359d700 1 -- 172.21.15.112:0/409765074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec152d20 con 0x7fbfec15e980 2023-08-27T11:27:09.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.257+0000 7fbff359d700 1 -- 172.21.15.112:0/409765074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfec15bdb0 con 0x7fbfec15e980 2023-08-27T11:27:09.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.257+0000 7fbfe2ffd700 1 -- 172.21.15.112:0/409765074 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fbfe4006d50 con 0x7fbfec15e980 2023-08-27T11:27:09.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.258+0000 7fbfe2ffd700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfd006a4d0 0x7fbfd006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.258+0000 7fbfe2ffd700 1 -- 172.21.15.112:0/409765074 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbfe40887f0 con 0x7fbfec15e980 2023-08-27T11:27:09.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.258+0000 7fbff259b700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfd006a4d0 0x7fbfd006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.259+0000 7fbff259b700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfd006a4d0 0x7fbfd006c990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc000f60 tx=0x7fbfdc007cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:09.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.262+0000 7fbfe2ffd700 1 -- 172.21.15.112:0/409765074 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbfec15bdb0 con 0x7fbfec15e980 2023-08-27T11:27:09.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.452+0000 7fbff359d700 1 -- 172.21.15.112:0/409765074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfec15bdb0 con 0x7fbfec15e980 2023-08-27T11:27:09.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.453+0000 7fbfe2ffd700 1 -- 172.21.15.112:0/409765074 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+18327 (secure 0 0 0) 0x7fbfec15bdb0 con 0x7fbfec15e980 2023-08-27T11:27:09.454 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:09.455 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":102,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:57.319281+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:54.078457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:54.929983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:09.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.455+0000 7fbff359d700 1 -- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfd006a4d0 msgr2=0x7fbfd006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.455+0000 7fbff359d700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfd006a4d0 0x7fbfd006c990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc000f60 tx=0x7fbfdc007cc0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.455+0000 7fbff359d700 1 -- 172.21.15.112:0/409765074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfec15e980 msgr2=0x7fbfec1527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.455+0000 7fbff359d700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfec15e980 0x7fbfec1527e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400b990 tx=0x7fbfe400bd50 comp rx=0 tx=0).stop 2023-08-27T11:27:09.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.455+0000 7fbff359d700 1 -- 172.21.15.112:0/409765074 shutdown_connections 2023-08-27T11:27:09.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.455+0000 7fbff359d700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfd006a4d0 0x7fbfd006c990 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.455+0000 7fbff359d700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfec15e980 0x7fbfec1527e0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.455+0000 7fbff359d700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfec06adb0 0x7fbfec1f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.455+0000 7fbff359d700 1 --2- 172.21.15.112:0/409765074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfec157890 0x7fbfec1f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.456+0000 7fbff359d700 1 -- 172.21.15.112:0/409765074 >> 172.21.15.112:0/409765074 conn(0x7fbfec066010 msgr2=0x7fbfec073380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:09.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.456+0000 7fbff359d700 1 -- 172.21.15.112:0/409765074 shutdown_connections 2023-08-27T11:27:09.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.456+0000 7fbff359d700 1 -- 172.21.15.112:0/409765074 wait complete. 2023-08-27T11:27:09.468 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:26:54.078457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:26:54.929983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:09.469 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:09.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.592+0000 7f3ff13ae700 1 Processor -- start 2023-08-27T11:27:09.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.592+0000 7f3ff13ae700 1 -- start start 2023-08-27T11:27:09.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3ff13ae700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3fec146bf0 0x7f3fec0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3ff13ae700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fec146fd0 0x7f3fec0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3ff13ae700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3fec0d0290 0x7f3fec070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3ff13ae700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fec0d1500 con 0x7f3fec146bf0 2023-08-27T11:27:09.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3ff13ae700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fec0d0710 con 0x7f3fec0d0290 2023-08-27T11:27:09.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3ff13ae700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec0d09f0 con 0x7f3fec146fd0 2023-08-27T11:27:09.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3feaffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3fec146bf0 0x7f3fec0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3feaffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3fec146bf0 0x7f3fec0b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34302/0 (socket says 172.21.15.112:34302) 2023-08-27T11:27:09.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3feaffd700 1 -- 172.21.15.112:0/2125186462 learned_addr learned my addr 172.21.15.112:0/2125186462 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:09.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3fea7fc700 1 --2- 172.21.15.112:0/2125186462 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fec146fd0 0x7f3fec0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3feaffd700 1 -- 172.21.15.112:0/2125186462 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fec146fd0 msgr2=0x7f3fec0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:09.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3feaffd700 1 --2- 172.21.15.112:0/2125186462 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fec146fd0 0x7f3fec0cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3feb7fe700 1 --2- 172.21.15.112:0/2125186462 >> v2:172.21.15.138:3300/0 conn(0x7f3fec0d0290 0x7f3fec070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3feaffd700 1 -- 172.21.15.112:0/2125186462 >> v2:172.21.15.138:3300/0 conn(0x7f3fec0d0290 msgr2=0x7f3fec070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:09.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3feaffd700 1 --2- 172.21.15.112:0/2125186462 >> v2:172.21.15.138:3300/0 conn(0x7f3fec0d0290 0x7f3fec070eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3feaffd700 1 -- 172.21.15.112:0/2125186462 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec0d0cd0 con 0x7f3fec146bf0 2023-08-27T11:27:09.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3feb7fe700 1 --2- 172.21.15.112:0/2125186462 >> v2:172.21.15.138:3300/0 conn(0x7f3fec0d0290 0x7f3fec070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:09.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3feaffd700 1 --2- 172.21.15.112:0/2125186462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fec146bf0 0x7f3fec0b5db0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f3fec160f20 tx=0x7f3fe0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59beebe8a3e9cf02 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:09.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.593+0000 7f3fe9ffb700 1 -- 172.21.15.112:0/2125186462 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000b870 con 0x7f3fec146bf0 2023-08-27T11:27:09.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.594+0000 7f3fe9ffb700 1 -- 172.21.15.112:0/2125186462 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe000b9d0 con 0x7f3fec146bf0 2023-08-27T11:27:09.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.594+0000 7f3fe9ffb700 1 -- 172.21.15.112:0/2125186462 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000bd60 con 0x7f3fec146bf0 2023-08-27T11:27:09.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.594+0000 7f3ff13ae700 1 -- 172.21.15.112:0/2125186462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fec146bf0 msgr2=0x7f3fec0b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.594+0000 7f3ff13ae700 1 --2- 172.21.15.112:0/2125186462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fec146bf0 0x7f3fec0b5db0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f3fec160f20 tx=0x7f3fe0009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.594+0000 7f3ff13ae700 1 -- 172.21.15.112:0/2125186462 shutdown_connections 2023-08-27T11:27:09.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.594+0000 7f3ff13ae700 1 --2- 172.21.15.112:0/2125186462 >> v2:172.21.15.138:3300/0 conn(0x7f3fec0d0290 0x7f3fec070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.594+0000 7f3ff13ae700 1 --2- 172.21.15.112:0/2125186462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fec146bf0 0x7f3fec0b5db0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.594+0000 7f3ff13ae700 1 --2- 172.21.15.112:0/2125186462 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fec146fd0 0x7f3fec0cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.594+0000 7f3ff13ae700 1 -- 172.21.15.112:0/2125186462 >> 172.21.15.112:0/2125186462 conn(0x7f3fec06c450 msgr2=0x7f3fec06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:09.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.594+0000 7f3ff13ae700 1 -- 172.21.15.112:0/2125186462 shutdown_connections 2023-08-27T11:27:09.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.594+0000 7f3ff13ae700 1 -- 172.21.15.112:0/2125186462 wait complete. 2023-08-27T11:27:09.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.595+0000 7f3ff13ae700 1 Processor -- start 2023-08-27T11:27:09.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.595+0000 7f3ff13ae700 1 -- start start 2023-08-27T11:27:09.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.596+0000 7f3ff13ae700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fec0d0290 0x7f3fec0b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.596+0000 7f3ff13ae700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fec146bf0 0x7f3fec1f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.596+0000 7f3ff13ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fec146fd0 0x7f3fec0aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.596+0000 7f3ff13ae700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec1f38c0 con 0x7f3fec146fd0 2023-08-27T11:27:09.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.596+0000 7f3ff13ae700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec0ae8b0 con 0x7f3fec146bf0 2023-08-27T11:27:09.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.596+0000 7f3ff13ae700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec0b8a90 con 0x7f3fec0d0290 2023-08-27T11:27:09.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.596+0000 7f3feb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fec146fd0 0x7f3fec0aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.596+0000 7f3feb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fec146fd0 0x7f3fec0aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34312/0 (socket says 172.21.15.112:34312) 2023-08-27T11:27:09.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.596+0000 7f3feb7fe700 1 -- 172.21.15.112:0/3839656788 learned_addr learned my addr 172.21.15.112:0/3839656788 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:09.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.596+0000 7f3feaffd700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fec0d0290 0x7f3fec0b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.596+0000 7f3fea7fc700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fec146bf0 0x7f3fec1f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.596+0000 7f3feb7fe700 1 -- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fec0d0290 msgr2=0x7f3fec0b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.597+0000 7f3feb7fe700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fec0d0290 0x7f3fec0b8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.597+0000 7f3feb7fe700 1 -- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fec146bf0 msgr2=0x7f3fec1f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.597+0000 7f3feb7fe700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fec146bf0 0x7f3fec1f3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.597+0000 7f3feb7fe700 1 -- 172.21.15.112:0/3839656788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe0009550 con 0x7f3fec146fd0 2023-08-27T11:27:09.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.597+0000 7f3fea7fc700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fec146bf0 0x7f3fec1f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:09.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.597+0000 7f3feaffd700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fec0d0290 0x7f3fec0b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:09.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.597+0000 7f3feb7fe700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fec146fd0 0x7f3fec0aed40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400ce40 tx=0x7f3fe400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:09.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.597+0000 7f3fd37fe700 1 -- 172.21.15.112:0/3839656788 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400adb0 con 0x7f3fec146fd0 2023-08-27T11:27:09.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.597+0000 7f3fd37fe700 1 -- 172.21.15.112:0/3839656788 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe4015440 con 0x7f3fec146fd0 2023-08-27T11:27:09.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.597+0000 7f3ff13ae700 1 -- 172.21.15.112:0/3839656788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fec0adf80 con 0x7f3fec146fd0 2023-08-27T11:27:09.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.597+0000 7f3fd37fe700 1 -- 172.21.15.112:0/3839656788 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe40157f0 con 0x7f3fec146fd0 2023-08-27T11:27:09.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.597+0000 7f3ff13ae700 1 -- 172.21.15.112:0/3839656788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec0ae540 con 0x7f3fec146fd0 2023-08-27T11:27:09.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.598+0000 7f3ff13ae700 1 -- 172.21.15.112:0/3839656788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fec0aea60 con 0x7f3fec146fd0 2023-08-27T11:27:09.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.598+0000 7f3fd37fe700 1 -- 172.21.15.112:0/3839656788 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f3fe400d430 con 0x7f3fec146fd0 2023-08-27T11:27:09.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.598+0000 7f3fd37fe700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fd806a4d0 0x7f3fd806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.599+0000 7f3fd37fe700 1 -- 172.21.15.112:0/3839656788 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3fe40891c0 con 0x7f3fec146fd0 2023-08-27T11:27:09.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.599+0000 7f3feaffd700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fd806a4d0 0x7f3fd806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.599+0000 7f3feaffd700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fd806a4d0 0x7f3fd806c990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009130 tx=0x7f3fe0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:09.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.603+0000 7f3fd37fe700 1 -- 172.21.15.112:0/3839656788 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3fec0aea60 con 0x7f3fec146fd0 2023-08-27T11:27:09.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.791+0000 7f3ff13ae700 1 -- 172.21.15.112:0/3839656788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3fec067e90 con 0x7f3fec146fd0 2023-08-27T11:27:09.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.791+0000 7f3fd37fe700 1 -- 172.21.15.112:0/3839656788 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v102) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3fe40542d0 con 0x7f3fec146fd0 2023-08-27T11:27:09.792 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:27:09.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.793+0000 7f3ff13ae700 1 -- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fd806a4d0 msgr2=0x7f3fd806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.793+0000 7f3ff13ae700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fd806a4d0 0x7f3fd806c990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009130 tx=0x7f3fe0008030 comp rx=0 tx=0).stop 2023-08-27T11:27:09.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.793+0000 7f3ff13ae700 1 -- 172.21.15.112:0/3839656788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fec146fd0 msgr2=0x7f3fec0aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.793+0000 7f3ff13ae700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fec146fd0 0x7f3fec0aed40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400ce40 tx=0x7f3fe400cf20 comp rx=0 tx=0).stop 2023-08-27T11:27:09.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.793+0000 7f3ff13ae700 1 -- 172.21.15.112:0/3839656788 shutdown_connections 2023-08-27T11:27:09.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.793+0000 7f3ff13ae700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3fd806a4d0 0x7f3fd806c990 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.793+0000 7f3ff13ae700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fec146fd0 0x7f3fec0aed40 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.793+0000 7f3ff13ae700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fec0d0290 0x7f3fec0b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.793+0000 7f3ff13ae700 1 --2- 172.21.15.112:0/3839656788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fec146bf0 0x7f3fec1f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.793+0000 7f3ff13ae700 1 -- 172.21.15.112:0/3839656788 >> 172.21.15.112:0/3839656788 conn(0x7f3fec06c450 msgr2=0x7f3fec15f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:09.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.793+0000 7f3ff13ae700 1 -- 172.21.15.112:0/3839656788 shutdown_connections 2023-08-27T11:27:09.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.793+0000 7f3ff13ae700 1 -- 172.21.15.112:0/3839656788 wait complete. 2023-08-27T11:27:09.806 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:09.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.928+0000 7f7d017a0700 1 Processor -- start 2023-08-27T11:27:09.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.928+0000 7f7d017a0700 1 -- start start 2023-08-27T11:27:09.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.929+0000 7f7d017a0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7cfc0ccbe0 0x7f7cfc0cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.929+0000 7f7d017a0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc15f980 con 0x7f7cfc0d0940 2023-08-27T11:27:09.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.929+0000 7f7d017a0700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc0cde80 con 0x7f7cfc0d04e0 2023-08-27T11:27:09.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.929+0000 7f7d017a0700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc0ce160 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.929+0000 7f7d00f9f700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7cfc0d0940 0x7f7cfc15f270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56934/0 (socket says 172.21.15.112:56934) 2023-08-27T11:27:09.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.929+0000 7f7d00f9f700 1 -- 172.21.15.112:0/3334847872 learned_addr learned my addr 172.21.15.112:0/3334847872 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:09.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.929+0000 7f7cfbfff700 1 --2- 172.21.15.112:0/3334847872 >> v2:172.21.15.138:3300/0 conn(0x7f7cfc0ccbe0 0x7f7cfc0cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.929+0000 7f7cfaffd700 1 -- 172.21.15.112:0/3334847872 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850637134 0 0) 0x7f7cfc15f980 con 0x7f7cfc0d0940 2023-08-27T11:27:09.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.929+0000 7f7cfaffd700 1 -- 172.21.15.112:0/3334847872 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ce40035b0 con 0x7f7cfc0d0940 2023-08-27T11:27:09.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.929+0000 7f7cfaffd700 1 -- 172.21.15.112:0/3334847872 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1228502051 0 0) 0x7f7ce40035b0 con 0x7f7cfc0d0940 2023-08-27T11:27:09.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.929+0000 7f7cfbfff700 1 -- 172.21.15.112:0/3334847872 >> v1:172.21.15.138:6789/0 conn(0x7f7cfc0d04e0 legacy=0x7f7cfc15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.929+0000 7f7cfbfff700 1 -- 172.21.15.112:0/3334847872 >> v1:172.21.15.112:6789/0 conn(0x7f7cfc0d0940 legacy=0x7f7cfc15f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.929+0000 7f7cfbfff700 1 -- 172.21.15.112:0/3334847872 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0ce440 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.930+0000 7f7cfbfff700 1 --2- 172.21.15.112:0/3334847872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0ccbe0 0x7f7cfc0cef20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009840 tx=0x7f7cf0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fef1c9f56c40d91c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:09.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.930+0000 7f7cfaffd700 1 -- 172.21.15.112:0/3334847872 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40019f0 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.930+0000 7f7cfaffd700 1 -- 172.21.15.112:0/3334847872 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf0004030 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.930+0000 7f7cfaffd700 1 -- 172.21.15.112:0/3334847872 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf00043a0 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.930+0000 7f7d017a0700 1 -- 172.21.15.112:0/3334847872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0ccbe0 msgr2=0x7f7cfc0cef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.930+0000 7f7d017a0700 1 --2- 172.21.15.112:0/3334847872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0ccbe0 0x7f7cfc0cef20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009840 tx=0x7f7cf0009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:09.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.930+0000 7f7d017a0700 1 -- 172.21.15.112:0/3334847872 shutdown_connections 2023-08-27T11:27:09.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.930+0000 7f7d017a0700 1 --2- 172.21.15.112:0/3334847872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0ccbe0 0x7f7cfc0cef20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.930+0000 7f7d017a0700 1 -- 172.21.15.112:0/3334847872 >> 172.21.15.112:0/3334847872 conn(0x7f7cfc066010 msgr2=0x7f7cfc0663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:09.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.930+0000 7f7d017a0700 1 -- 172.21.15.112:0/3334847872 shutdown_connections 2023-08-27T11:27:09.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.930+0000 7f7d017a0700 1 -- 172.21.15.112:0/3334847872 wait complete. 2023-08-27T11:27:09.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.931+0000 7f7d017a0700 1 Processor -- start 2023-08-27T11:27:09.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.932+0000 7f7d017a0700 1 -- start start 2023-08-27T11:27:09.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.932+0000 7f7d017a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cfc0ccbe0 0x7f7cfc154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.932+0000 7f7d017a0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cfc0d04e0 0x7f7cfc15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.932+0000 7f7d017a0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0d0940 0x7f7cfc15f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.932+0000 7f7d017a0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc1f1190 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.932+0000 7f7d017a0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc154880 con 0x7f7cfc0d0940 2023-08-27T11:27:09.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.932+0000 7f7d017a0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc1f13e0 con 0x7f7cfc0d04e0 2023-08-27T11:27:09.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.932+0000 7f7cfbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cfc0ccbe0 0x7f7cfc154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.932+0000 7f7cfbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cfc0ccbe0 0x7f7cfc154030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34324/0 (socket says 172.21.15.112:34324) 2023-08-27T11:27:09.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.932+0000 7f7cfbfff700 1 -- 172.21.15.112:0/3708096666 learned_addr learned my addr 172.21.15.112:0/3708096666 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:09.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7d00f9f700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0d0940 0x7f7cfc15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7cfb7fe700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cfc0d04e0 0x7f7cfc15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7cfbfff700 1 -- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cfc0d04e0 msgr2=0x7f7cfc15eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7cfbfff700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cfc0d04e0 0x7f7cfc15eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7cfbfff700 1 -- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0d0940 msgr2=0x7f7cfc15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7cfbfff700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0d0940 0x7f7cfc15f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7cfbfff700 1 -- 172.21.15.112:0/3708096666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf00094a0 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7cfb7fe700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cfc0d04e0 0x7f7cfc15eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:09.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7d00f9f700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0d0940 0x7f7cfc15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:09.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7cfbfff700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cfc0ccbe0 0x7f7cfc154030 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0000c00 tx=0x7f7cf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:09.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7cf8ff9700 1 -- 172.21.15.112:0/3708096666 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0003df0 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7cf8ff9700 1 -- 172.21.15.112:0/3708096666 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf001f070 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7cf8ff9700 1 -- 172.21.15.112:0/3708096666 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0011870 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7d017a0700 1 -- 172.21.15.112:0/3708096666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc1f16c0 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.933+0000 7f7d017a0700 1 -- 172.21.15.112:0/3708096666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc154570 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.934+0000 7f7d017a0700 1 -- 172.21.15.112:0/3708096666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc061f50 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.934+0000 7f7cf8ff9700 1 -- 172.21.15.112:0/3708096666 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f7cf0021030 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.935+0000 7f7cf8ff9700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ce806a4d0 0x7f7ce806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.935+0000 7f7cf8ff9700 1 -- 172.21.15.112:0/3708096666 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7cf008a0a0 con 0x7f7cfc0ccbe0 2023-08-27T11:27:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.935+0000 7f7cfb7fe700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ce806a4d0 0x7f7ce806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.936+0000 7f7cfb7fe700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ce806a4d0 0x7f7ce806c990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0b3070 tx=0x7f7cec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:09.939+0000 7f7cf8ff9700 1 -- 172.21.15.112:0/3708096666 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7cf00552d0 con 0x7f7cfc0ccbe0 2023-08-27T11:27:10.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.126+0000 7f7d017a0700 1 -- 172.21.15.112:0/3708096666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7cfc0b29b0 con 0x7f7cfc0ccbe0 2023-08-27T11:27:10.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.126+0000 7f7cf8ff9700 1 -- 172.21.15.112:0/3708096666 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7cf00588f0 con 0x7f7cfc0ccbe0 2023-08-27T11:27:10.127 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:27:10.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.128+0000 7f7d017a0700 1 -- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ce806a4d0 msgr2=0x7f7ce806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.128+0000 7f7d017a0700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ce806a4d0 0x7f7ce806c990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0b3070 tx=0x7f7cec009040 comp rx=0 tx=0).stop 2023-08-27T11:27:10.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.128+0000 7f7d017a0700 1 -- 172.21.15.112:0/3708096666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cfc0ccbe0 msgr2=0x7f7cfc154030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.128+0000 7f7d017a0700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cfc0ccbe0 0x7f7cfc154030 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0000c00 tx=0x7f7cf0004750 comp rx=0 tx=0).stop 2023-08-27T11:27:10.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.128+0000 7f7d017a0700 1 -- 172.21.15.112:0/3708096666 shutdown_connections 2023-08-27T11:27:10.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.128+0000 7f7d017a0700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ce806a4d0 0x7f7ce806c990 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.128+0000 7f7d017a0700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0d0940 0x7f7cfc15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.128+0000 7f7d017a0700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cfc0ccbe0 0x7f7cfc154030 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.128+0000 7f7d017a0700 1 --2- 172.21.15.112:0/3708096666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cfc0d04e0 0x7f7cfc15eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.128+0000 7f7d017a0700 1 -- 172.21.15.112:0/3708096666 >> 172.21.15.112:0/3708096666 conn(0x7f7cfc066010 msgr2=0x7f7cfc0b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:10.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.129+0000 7f7d017a0700 1 -- 172.21.15.112:0/3708096666 shutdown_connections 2023-08-27T11:27:10.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.129+0000 7f7d017a0700 1 -- 172.21.15.112:0/3708096666 wait complete. 2023-08-27T11:27:10.141 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:27:10.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.264+0000 7fa65e048700 1 Processor -- start 2023-08-27T11:27:10.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.264+0000 7fa65e048700 1 -- start start 2023-08-27T11:27:10.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.264+0000 7fa65e048700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa6581437c0 0x7fa6580b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.264+0000 7fa65e048700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa658143ba0 0x7fa6580ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.264+0000 7fa65e048700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa65806d5b0 0x7fa65806da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.264+0000 7fa65e048700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65806df80 con 0x7fa6581437c0 2023-08-27T11:27:10.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.264+0000 7fa65e048700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6580cfd50 con 0x7fa658143ba0 2023-08-27T11:27:10.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.264+0000 7fa65e048700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6580cf140 con 0x7fa65806d5b0 2023-08-27T11:27:10.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa6577fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa6581437c0 0x7fa6580b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa6577fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa6581437c0 0x7fa6580b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34330/0 (socket says 172.21.15.112:34330) 2023-08-27T11:27:10.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa6577fe700 1 -- 172.21.15.112:0/215159882 learned_addr learned my addr 172.21.15.112:0/215159882 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:10.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa656ffd700 1 --2- 172.21.15.112:0/215159882 >> v2:172.21.15.138:3300/0 conn(0x7fa658143ba0 0x7fa6580ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa657fff700 1 --2- 172.21.15.112:0/215159882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa65806d5b0 0x7fa65806da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa6577fe700 1 -- 172.21.15.112:0/215159882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa65806d5b0 msgr2=0x7fa65806da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa6577fe700 1 --2- 172.21.15.112:0/215159882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa65806d5b0 0x7fa65806da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa6577fe700 1 -- 172.21.15.112:0/215159882 >> v2:172.21.15.138:3300/0 conn(0x7fa658143ba0 msgr2=0x7fa6580ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa6577fe700 1 --2- 172.21.15.112:0/215159882 >> v2:172.21.15.138:3300/0 conn(0x7fa658143ba0 0x7fa6580ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa6577fe700 1 -- 172.21.15.112:0/215159882 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6580cf420 con 0x7fa6581437c0 2023-08-27T11:27:10.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa6577fe700 1 --2- 172.21.15.112:0/215159882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6581437c0 0x7fa6580b4670 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fa65806e1a0 tx=0x7fa648009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f38f1ccea705db server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa6567fc700 1 -- 172.21.15.112:0/215159882 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64800b870 con 0x7fa6581437c0 2023-08-27T11:27:10.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa6567fc700 1 -- 172.21.15.112:0/215159882 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64800b9d0 con 0x7fa6581437c0 2023-08-27T11:27:10.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.265+0000 7fa6567fc700 1 -- 172.21.15.112:0/215159882 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64800bd60 con 0x7fa6581437c0 2023-08-27T11:27:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.266+0000 7fa65e048700 1 -- 172.21.15.112:0/215159882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6581437c0 msgr2=0x7fa6580b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.266+0000 7fa65e048700 1 --2- 172.21.15.112:0/215159882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6581437c0 0x7fa6580b4670 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fa65806e1a0 tx=0x7fa648009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.266+0000 7fa65e048700 1 -- 172.21.15.112:0/215159882 shutdown_connections 2023-08-27T11:27:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.266+0000 7fa65e048700 1 --2- 172.21.15.112:0/215159882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa65806d5b0 0x7fa65806da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.266+0000 7fa65e048700 1 --2- 172.21.15.112:0/215159882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6581437c0 0x7fa6580b4670 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.266+0000 7fa65e048700 1 --2- 172.21.15.112:0/215159882 >> v2:172.21.15.138:3300/0 conn(0x7fa658143ba0 0x7fa6580ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.266+0000 7fa65e048700 1 -- 172.21.15.112:0/215159882 >> 172.21.15.112:0/215159882 conn(0x7fa6580b1e90 msgr2=0x7fa6580b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:10.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.266+0000 7fa65e048700 1 -- 172.21.15.112:0/215159882 shutdown_connections 2023-08-27T11:27:10.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.266+0000 7fa65e048700 1 -- 172.21.15.112:0/215159882 wait complete. 2023-08-27T11:27:10.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.267+0000 7fa65e048700 1 Processor -- start 2023-08-27T11:27:10.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.267+0000 7fa65e048700 1 -- start start 2023-08-27T11:27:10.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa65e048700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa65806d5b0 0x7fa65813a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa65e048700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6581437c0 0x7fa6581591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa65e048700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa658143ba0 0x7fa658141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa65e048700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa658135210 con 0x7fa65806d5b0 2023-08-27T11:27:10.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa65e048700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa658134340 con 0x7fa6581437c0 2023-08-27T11:27:10.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa65e048700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa658134ec0 con 0x7fa658143ba0 2023-08-27T11:27:10.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa6577fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa65806d5b0 0x7fa65813a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa6577fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa65806d5b0 0x7fa65813a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34336/0 (socket says 172.21.15.112:34336) 2023-08-27T11:27:10.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa6577fe700 1 -- 172.21.15.112:0/443348019 learned_addr learned my addr 172.21.15.112:0/443348019 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:10.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa656ffd700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6581437c0 0x7fa6581591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa657fff700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa658143ba0 0x7fa658141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa6577fe700 1 -- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa658143ba0 msgr2=0x7fa658141c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa6577fe700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa658143ba0 0x7fa658141c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa6577fe700 1 -- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6581437c0 msgr2=0x7fa6581591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa6577fe700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6581437c0 0x7fa6581591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa6577fe700 1 -- 172.21.15.112:0/443348019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa648009550 con 0x7fa65806d5b0 2023-08-27T11:27:10.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.268+0000 7fa6577fe700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa65806d5b0 0x7fa65813a760 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fa65806e400 tx=0x7fa648004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.269+0000 7fa63ffff700 1 -- 172.21.15.112:0/443348019 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648004540 con 0x7fa65806d5b0 2023-08-27T11:27:10.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.269+0000 7fa63ffff700 1 -- 172.21.15.112:0/443348019 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6480036a0 con 0x7fa65806d5b0 2023-08-27T11:27:10.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.269+0000 7fa63ffff700 1 -- 172.21.15.112:0/443348019 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648003a50 con 0x7fa65806d5b0 2023-08-27T11:27:10.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.269+0000 7fa65e048700 1 -- 172.21.15.112:0/443348019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa658134590 con 0x7fa65806d5b0 2023-08-27T11:27:10.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.269+0000 7fa65e048700 1 -- 172.21.15.112:0/443348019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa658134b50 con 0x7fa65806d5b0 2023-08-27T11:27:10.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.270+0000 7fa65e048700 1 -- 172.21.15.112:0/443348019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa658067e90 con 0x7fa65806d5b0 2023-08-27T11:27:10.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.270+0000 7fa63ffff700 1 -- 172.21.15.112:0/443348019 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa648021070 con 0x7fa65806d5b0 2023-08-27T11:27:10.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.271+0000 7fa63ffff700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa64406a4d0 0x7fa64406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.271+0000 7fa63ffff700 1 -- 172.21.15.112:0/443348019 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa64808a620 con 0x7fa65806d5b0 2023-08-27T11:27:10.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.271+0000 7fa656ffd700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa64406a4d0 0x7fa64406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.271+0000 7fa656ffd700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa64406a4d0 0x7fa64406c990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa658142a70 tx=0x7fa64c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.274+0000 7fa63ffff700 1 -- 172.21.15.112:0/443348019 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa6480559d0 con 0x7fa65806d5b0 2023-08-27T11:27:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.465+0000 7fa65e048700 1 -- 172.21.15.112:0/443348019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa658067a20 con 0x7fa65806d5b0 2023-08-27T11:27:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.465+0000 7fa63ffff700 1 -- 172.21.15.112:0/443348019 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6480150a0 con 0x7fa65806d5b0 2023-08-27T11:27:10.466 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:27:10.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.467+0000 7fa65e048700 1 -- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa64406a4d0 msgr2=0x7fa64406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.467+0000 7fa65e048700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa64406a4d0 0x7fa64406c990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa658142a70 tx=0x7fa64c008040 comp rx=0 tx=0).stop 2023-08-27T11:27:10.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.467+0000 7fa65e048700 1 -- 172.21.15.112:0/443348019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa65806d5b0 msgr2=0x7fa65813a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.467+0000 7fa65e048700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa65806d5b0 0x7fa65813a760 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fa65806e400 tx=0x7fa648004750 comp rx=0 tx=0).stop 2023-08-27T11:27:10.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.467+0000 7fa65e048700 1 -- 172.21.15.112:0/443348019 shutdown_connections 2023-08-27T11:27:10.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.467+0000 7fa65e048700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa64406a4d0 0x7fa64406c990 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.467+0000 7fa65e048700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa658143ba0 0x7fa658141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.467+0000 7fa65e048700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa65806d5b0 0x7fa65813a760 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.467+0000 7fa65e048700 1 --2- 172.21.15.112:0/443348019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6581437c0 0x7fa6581591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.468+0000 7fa65e048700 1 -- 172.21.15.112:0/443348019 >> 172.21.15.112:0/443348019 conn(0x7fa6580b1e90 msgr2=0x7fa658071640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.468+0000 7fa65e048700 1 -- 172.21.15.112:0/443348019 shutdown_connections 2023-08-27T11:27:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.468+0000 7fa65e048700 1 -- 172.21.15.112:0/443348019 wait complete. 2023-08-27T11:27:10.480 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:27:10.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.604+0000 7f56d322e700 1 Processor -- start 2023-08-27T11:27:10.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.604+0000 7f56d322e700 1 -- start start 2023-08-27T11:27:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.604+0000 7f56d322e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc15b090 0x7f56cc15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.604+0000 7f56d322e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f56cc06adb0 0x7f56cc0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.604+0000 7f56d322e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc0c8230 con 0x7f56cc0cadd0 2023-08-27T11:27:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.604+0000 7f56d322e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56cc06b730 con 0x7f56cc06adb0 2023-08-27T11:27:10.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.604+0000 7f56d322e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc06b980 con 0x7f56cc15b090 2023-08-27T11:27:10.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cbfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f56cc0cadd0 0x7f56cc15f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56958/0 (socket says 172.21.15.112:56958) 2023-08-27T11:27:10.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cbfff700 1 -- 172.21.15.112:0/4080986766 learned_addr learned my addr 172.21.15.112:0/4080986766 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:10.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56d17cb700 1 --2- 172.21.15.112:0/4080986766 >> v2:172.21.15.138:3300/0 conn(0x7f56cc06adb0 0x7f56cc0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56d0fca700 1 --2- 172.21.15.112:0/4080986766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc15b090 0x7f56cc15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144335792 0 0) 0x7f56cc0c8230 con 0x7f56cc0cadd0 2023-08-27T11:27:10.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56bc0035b0 con 0x7f56cc0cadd0 2023-08-27T11:27:10.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2148018706 0 0) 0x7f56bc0035b0 con 0x7f56cc0cadd0 2023-08-27T11:27:10.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc15b090 msgr2=0x7f56cc15ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 --2- 172.21.15.112:0/4080986766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc15b090 0x7f56cc15ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 >> v2:172.21.15.138:3300/0 conn(0x7f56cc06adb0 msgr2=0x7f56cc0cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 --2- 172.21.15.112:0/4080986766 >> v2:172.21.15.138:3300/0 conn(0x7f56cc06adb0 0x7f56cc0cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc06bc60 con 0x7f56cc0cadd0 2023-08-27T11:27:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f56b8001670 con 0x7f56cc0cadd0 2023-08-27T11:27:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 >> v1:172.21.15.112:6789/0 conn(0x7f56cc0cadd0 legacy=0x7f56cc15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 --2- 172.21.15.112:0/4080986766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc15b090 0x7f56bc003a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 --2- 172.21.15.112:0/4080986766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56bc003ff0 0x7f56bc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 --2- 172.21.15.112:0/4080986766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc004d90 0x7f56bc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc06b730 con 0x7f56bc004d90 2023-08-27T11:27:10.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc06b980 con 0x7f56bc003ff0 2023-08-27T11:27:10.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc00ba40 con 0x7f56cc15b090 2023-08-27T11:27:10.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56d17cb700 1 --2- 172.21.15.112:0/4080986766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc004d90 0x7f56bc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56cbfff700 1 --2- 172.21.15.112:0/4080986766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56bc003ff0 0x7f56bc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.605+0000 7f56d0fca700 1 --2- 172.21.15.112:0/4080986766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc15b090 0x7f56bc003a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56d17cb700 1 -- 172.21.15.112:0/4080986766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc15b090 msgr2=0x7f56bc003a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56d17cb700 1 --2- 172.21.15.112:0/4080986766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc15b090 0x7f56bc003a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56d17cb700 1 -- 172.21.15.112:0/4080986766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56bc003ff0 msgr2=0x7f56bc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56d17cb700 1 --2- 172.21.15.112:0/4080986766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56bc003ff0 0x7f56bc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56d17cb700 1 -- 172.21.15.112:0/4080986766 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc0035b0 con 0x7f56bc004d90 2023-08-27T11:27:10.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56d17cb700 1 --2- 172.21.15.112:0/4080986766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc004d90 0x7f56bc005210 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009fc0 tx=0x7f56bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c5db77d883a1cc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c400ec30 con 0x7f56bc004d90 2023-08-27T11:27:10.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c400f040 con 0x7f56bc004d90 2023-08-27T11:27:10.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56cb7fe700 1 -- 172.21.15.112:0/4080986766 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4014480 con 0x7f56bc004d90 2023-08-27T11:27:10.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56d322e700 1 -- 172.21.15.112:0/4080986766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc004d90 msgr2=0x7f56bc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56d322e700 1 --2- 172.21.15.112:0/4080986766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc004d90 0x7f56bc005210 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009fc0 tx=0x7f56bc005880 comp rx=0 tx=0).stop 2023-08-27T11:27:10.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56d17cb700 1 -- 172.21.15.112:0/4080986766 reap_dead start 2023-08-27T11:27:10.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56d322e700 1 -- 172.21.15.112:0/4080986766 shutdown_connections 2023-08-27T11:27:10.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.606+0000 7f56d322e700 1 -- 172.21.15.112:0/4080986766 >> 172.21.15.112:0/4080986766 conn(0x7f56cc0d0fc0 msgr2=0x7f56cc0d13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:10.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.607+0000 7f56d322e700 1 -- 172.21.15.112:0/4080986766 shutdown_connections 2023-08-27T11:27:10.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.607+0000 7f56d322e700 1 -- 172.21.15.112:0/4080986766 wait complete. 2023-08-27T11:27:10.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.608+0000 7f56d322e700 1 Processor -- start 2023-08-27T11:27:10.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.608+0000 7f56d322e700 1 -- start start 2023-08-27T11:27:10.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.609+0000 7f56d322e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc0c98f0 0x7f56cc0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.609+0000 7f56d322e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc1f0540 0x7f56cc15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.609+0000 7f56d322e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56cc15c630 0x7f56cc143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.609+0000 7f56d322e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1f0b20 con 0x7f56cc15c630 2023-08-27T11:27:10.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.609+0000 7f56d322e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0ca210 con 0x7f56cc0c98f0 2023-08-27T11:27:10.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.609+0000 7f56d322e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc15cc60 con 0x7f56cc1f0540 2023-08-27T11:27:10.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.609+0000 7f56d17cb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56cc15c630 0x7f56cc143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.609+0000 7f56d17cb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56cc15c630 0x7f56cc143aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34358/0 (socket says 172.21.15.112:34358) 2023-08-27T11:27:10.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.609+0000 7f56d17cb700 1 -- 172.21.15.112:0/2426297482 learned_addr learned my addr 172.21.15.112:0/2426297482 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:10.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.609+0000 7f56d0fca700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc0c98f0 0x7f56cc0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.610+0000 7f56cbfff700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc1f0540 0x7f56cc15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.610+0000 7f56d17cb700 1 -- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc1f0540 msgr2=0x7f56cc15c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.610+0000 7f56d17cb700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc1f0540 0x7f56cc15c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.610+0000 7f56d17cb700 1 -- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc0c98f0 msgr2=0x7f56cc0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.610+0000 7f56d17cb700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc0c98f0 0x7f56cc0c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.610+0000 7f56d17cb700 1 -- 172.21.15.112:0/2426297482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc15cf40 con 0x7f56cc15c630 2023-08-27T11:27:10.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.610+0000 7f56d17cb700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56cc15c630 0x7f56cc143aa0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f56c4016000 tx=0x7f56c4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.610+0000 7f56c97fa700 1 -- 172.21.15.112:0/2426297482 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4007000 con 0x7f56cc15c630 2023-08-27T11:27:10.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.610+0000 7f56c97fa700 1 -- 172.21.15.112:0/2426297482 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c4015b30 con 0x7f56cc15c630 2023-08-27T11:27:10.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.610+0000 7f56d322e700 1 -- 172.21.15.112:0/2426297482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc15d220 con 0x7f56cc15c630 2023-08-27T11:27:10.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.610+0000 7f56c97fa700 1 -- 172.21.15.112:0/2426297482 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c401d420 con 0x7f56cc15c630 2023-08-27T11:27:10.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.610+0000 7f56d322e700 1 -- 172.21.15.112:0/2426297482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc15d500 con 0x7f56cc15c630 2023-08-27T11:27:10.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.611+0000 7f56d322e700 1 -- 172.21.15.112:0/2426297482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc061f50 con 0x7f56cc15c630 2023-08-27T11:27:10.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.611+0000 7f56c97fa700 1 -- 172.21.15.112:0/2426297482 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f56c401d580 con 0x7f56cc15c630 2023-08-27T11:27:10.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.612+0000 7f56c97fa700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f56b006a4d0 0x7f56b006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.612+0000 7f56c97fa700 1 -- 172.21.15.112:0/2426297482 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f56c408a520 con 0x7f56cc15c630 2023-08-27T11:27:10.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.612+0000 7f56d0fca700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f56b006a4d0 0x7f56b006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.612+0000 7f56d0fca700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f56b006a4d0 0x7f56b006c990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009aa0 tx=0x7f56c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.616+0000 7f56c97fa700 1 -- 172.21.15.112:0/2426297482 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f56c40556d0 con 0x7f56cc15c630 2023-08-27T11:27:10.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.808+0000 7f56d322e700 1 -- 172.21.15.112:0/2426297482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f56cc14ed60 con 0x7f56cc15c630 2023-08-27T11:27:10.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.808+0000 7f56c97fa700 1 -- 172.21.15.112:0/2426297482 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7f56c4058cf0 con 0x7f56cc15c630 2023-08-27T11:27:10.809 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:27:10.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.810+0000 7f56d322e700 1 -- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f56b006a4d0 msgr2=0x7f56b006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.810+0000 7f56d322e700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f56b006a4d0 0x7f56b006c990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009aa0 tx=0x7f56c0008040 comp rx=0 tx=0).stop 2023-08-27T11:27:10.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.810+0000 7f56d322e700 1 -- 172.21.15.112:0/2426297482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56cc15c630 msgr2=0x7f56cc143aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.810+0000 7f56d322e700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56cc15c630 0x7f56cc143aa0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f56c4016000 tx=0x7f56c4009750 comp rx=0 tx=0).stop 2023-08-27T11:27:10.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.810+0000 7f56d322e700 1 -- 172.21.15.112:0/2426297482 shutdown_connections 2023-08-27T11:27:10.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.810+0000 7f56d322e700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f56b006a4d0 0x7f56b006c990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.810+0000 7f56d322e700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56cc15c630 0x7f56cc143aa0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.810+0000 7f56d322e700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56cc0c98f0 0x7f56cc0c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.810+0000 7f56d322e700 1 --2- 172.21.15.112:0/2426297482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56cc1f0540 0x7f56cc15c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.810+0000 7f56d322e700 1 -- 172.21.15.112:0/2426297482 >> 172.21.15.112:0/2426297482 conn(0x7f56cc0d0fc0 msgr2=0x7f56cc0cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.810+0000 7f56d322e700 1 -- 172.21.15.112:0/2426297482 shutdown_connections 2023-08-27T11:27:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.810+0000 7f56d322e700 1 -- 172.21.15.112:0/2426297482 wait complete. 2023-08-27T11:27:10.823 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:27:10.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.943+0000 7faba4c8e700 1 Processor -- start 2023-08-27T11:27:10.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.943+0000 7faba4c8e700 1 -- start start 2023-08-27T11:27:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.944+0000 7faba4c8e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faba00ce750 0x7faba00ad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.944+0000 7faba4c8e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba0140810 con 0x7faba0140430 2023-08-27T11:27:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.944+0000 7faba4c8e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba015db70 con 0x7faba006e860 2023-08-27T11:27:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.944+0000 7faba4c8e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba015de50 con 0x7faba00ce750 2023-08-27T11:27:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.944+0000 7fab9e59c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7faba0140430 0x7faba006e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:56976/0 (socket says 172.21.15.112:56976) 2023-08-27T11:27:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.944+0000 7fab9e59c700 1 -- 172.21.15.112:0/1462774723 learned_addr learned my addr 172.21.15.112:0/1462774723 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9ed9d700 1 --2- 172.21.15.112:0/1462774723 >> v2:172.21.15.138:3300/0 conn(0x7faba00ce750 0x7faba00ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996109199 0 0) 0x7faba0140810 con 0x7faba0140430 2023-08-27T11:27:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab880035b0 con 0x7faba0140430 2023-08-27T11:27:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2542584464 0 0) 0x7fab880035b0 con 0x7faba0140430 2023-08-27T11:27:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 >> v2:172.21.15.138:3300/0 conn(0x7faba00ce750 msgr2=0x7faba00ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 --2- 172.21.15.112:0/1462774723 >> v2:172.21.15.138:3300/0 conn(0x7faba00ce750 0x7faba00ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 >> v1:172.21.15.138:6789/0 conn(0x7faba006e860 legacy=0x7faba00acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba015e130 con 0x7faba0140430 2023-08-27T11:27:10.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fab94001670 con 0x7faba0140430 2023-08-27T11:27:10.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 >> v1:172.21.15.112:6789/0 conn(0x7faba0140430 legacy=0x7faba006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 --2- 172.21.15.112:0/1462774723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab88003a00 0x7fab88003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 --2- 172.21.15.112:0/1462774723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab88004850 0x7fab88004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 --2- 172.21.15.112:0/1462774723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab88005210 0x7fab8800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba015de50 con 0x7fab88004850 2023-08-27T11:27:10.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8800c0b0 con 0x7fab88003a00 2023-08-27T11:27:10.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8800c270 con 0x7fab88005210 2023-08-27T11:27:10.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9dd9b700 1 --2- 172.21.15.112:0/1462774723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab88004850 0x7fab88004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9e59c700 1 --2- 172.21.15.112:0/1462774723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab88003a00 0x7fab88003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9ed9d700 1 --2- 172.21.15.112:0/1462774723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab88005210 0x7fab8800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9dd9b700 1 -- 172.21.15.112:0/1462774723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab88005210 msgr2=0x7fab8800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9dd9b700 1 --2- 172.21.15.112:0/1462774723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab88005210 0x7fab8800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9dd9b700 1 -- 172.21.15.112:0/1462774723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab88003a00 msgr2=0x7fab88003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9dd9b700 1 --2- 172.21.15.112:0/1462774723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab88003a00 0x7fab88003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9dd9b700 1 -- 172.21.15.112:0/1462774723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab880035b0 con 0x7fab88004850 2023-08-27T11:27:10.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9ed9d700 1 -- 172.21.15.112:0/1462774723 reap_dead start 2023-08-27T11:27:10.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.945+0000 7fab9e59c700 1 --2- 172.21.15.112:0/1462774723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab88003a00 0x7fab88003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:10.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.946+0000 7fab9dd9b700 1 --2- 172.21.15.112:0/1462774723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab88004850 0x7fab88004cd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fab90001820 tx=0x7fab88005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf9e28a952a1289 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.946+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab900012e0 con 0x7fab88004850 2023-08-27T11:27:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.946+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab900052d0 con 0x7fab88004850 2023-08-27T11:27:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.946+0000 7fab9d59a700 1 -- 172.21.15.112:0/1462774723 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab900036d0 con 0x7fab88004850 2023-08-27T11:27:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.946+0000 7faba4c8e700 1 -- 172.21.15.112:0/1462774723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab88004850 msgr2=0x7fab88004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.946+0000 7faba4c8e700 1 --2- 172.21.15.112:0/1462774723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab88004850 0x7fab88004cd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fab90001820 tx=0x7fab88005880 comp rx=0 tx=0).stop 2023-08-27T11:27:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.946+0000 7faba4c8e700 1 -- 172.21.15.112:0/1462774723 shutdown_connections 2023-08-27T11:27:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.946+0000 7faba4c8e700 1 --2- 172.21.15.112:0/1462774723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab88004850 0x7fab88004cd0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.946+0000 7faba4c8e700 1 -- 172.21.15.112:0/1462774723 >> 172.21.15.112:0/1462774723 conn(0x7faba006cd30 msgr2=0x7faba00717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.946+0000 7faba4c8e700 1 -- 172.21.15.112:0/1462774723 shutdown_connections 2023-08-27T11:27:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.946+0000 7faba4c8e700 1 -- 172.21.15.112:0/1462774723 wait complete. 2023-08-27T11:27:10.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.948+0000 7faba4c8e700 1 Processor -- start 2023-08-27T11:27:10.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.948+0000 7faba4c8e700 1 -- start start 2023-08-27T11:27:10.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.948+0000 7faba4c8e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab88004850 0x7faba01f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.948+0000 7faba4c8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faba0071230 0x7faba01596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.948+0000 7faba4c8e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faba006fcc0 0x7faba0070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.948+0000 7faba4c8e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba01f3ef0 con 0x7faba0071230 2023-08-27T11:27:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.948+0000 7faba4c8e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba01f3650 con 0x7fab88004850 2023-08-27T11:27:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.948+0000 7faba4c8e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba01f3930 con 0x7faba006fcc0 2023-08-27T11:27:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab9dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faba0071230 0x7faba01596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab9dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faba0071230 0x7faba01596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34376/0 (socket says 172.21.15.112:34376) 2023-08-27T11:27:10.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab9dd9b700 1 -- 172.21.15.112:0/724847777 learned_addr learned my addr 172.21.15.112:0/724847777 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:10.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab9e59c700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab88004850 0x7faba01f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab9ed9d700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faba006fcc0 0x7faba0070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab9dd9b700 1 -- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faba006fcc0 msgr2=0x7faba0070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab9dd9b700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faba006fcc0 0x7faba0070140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab9dd9b700 1 -- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab88004850 msgr2=0x7faba01f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab9dd9b700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab88004850 0x7faba01f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab9dd9b700 1 -- 172.21.15.112:0/724847777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba01f3c10 con 0x7faba0071230 2023-08-27T11:27:10.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab9dd9b700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faba0071230 0x7faba01596b0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fab90000c00 tx=0x7fab90001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab8effd700 1 -- 172.21.15.112:0/724847777 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab900012e0 con 0x7faba0071230 2023-08-27T11:27:10.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab8effd700 1 -- 172.21.15.112:0/724847777 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab900050d0 con 0x7faba0071230 2023-08-27T11:27:10.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7fab8effd700 1 -- 172.21.15.112:0/724847777 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9001d420 con 0x7faba0071230 2023-08-27T11:27:10.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7faba4c8e700 1 -- 172.21.15.112:0/724847777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba006f7a0 con 0x7faba0071230 2023-08-27T11:27:10.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.949+0000 7faba4c8e700 1 -- 172.21.15.112:0/724847777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba0159bf0 con 0x7faba0071230 2023-08-27T11:27:10.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.950+0000 7faba4c8e700 1 -- 172.21.15.112:0/724847777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba00cd4b0 con 0x7faba0071230 2023-08-27T11:27:10.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.950+0000 7fab8effd700 1 -- 172.21.15.112:0/724847777 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fab9001d580 con 0x7faba0071230 2023-08-27T11:27:10.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.951+0000 7fab8effd700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fab8006a4d0 0x7fab8006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.951+0000 7fab8effd700 1 -- 172.21.15.112:0/724847777 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fab90089480 con 0x7faba0071230 2023-08-27T11:27:10.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.951+0000 7fab9e59c700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fab8006a4d0 0x7fab8006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.951+0000 7fab9e59c700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fab8006a4d0 0x7fab8006c990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fab94001c90 tx=0x7fab94007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:10.955+0000 7fab8effd700 1 -- 172.21.15.112:0/724847777 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fab90054620 con 0x7faba0071230 2023-08-27T11:27:11.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.161+0000 7faba4c8e700 1 -- 172.21.15.112:0/724847777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faba006f600 con 0x7faba0071230 2023-08-27T11:27:11.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.161+0000 7fab8effd700 1 -- 172.21.15.112:0/724847777 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v102) v1 ==== 80+0+0 (secure 0 0 0) 0x7faba006f600 con 0x7faba0071230 2023-08-27T11:27:11.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.163+0000 7faba4c8e700 1 -- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fab8006a4d0 msgr2=0x7fab8006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.163+0000 7faba4c8e700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fab8006a4d0 0x7fab8006c990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fab94001c90 tx=0x7fab94007040 comp rx=0 tx=0).stop 2023-08-27T11:27:11.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.163+0000 7faba4c8e700 1 -- 172.21.15.112:0/724847777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faba0071230 msgr2=0x7faba01596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.163+0000 7faba4c8e700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faba0071230 0x7faba01596b0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fab90000c00 tx=0x7fab90001170 comp rx=0 tx=0).stop 2023-08-27T11:27:11.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.163+0000 7faba4c8e700 1 -- 172.21.15.112:0/724847777 shutdown_connections 2023-08-27T11:27:11.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.163+0000 7faba4c8e700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fab8006a4d0 0x7fab8006c990 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.163+0000 7faba4c8e700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faba006fcc0 0x7faba0070140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.163+0000 7faba4c8e700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab88004850 0x7faba01f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.163+0000 7faba4c8e700 1 --2- 172.21.15.112:0/724847777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faba0071230 0x7faba01596b0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.163+0000 7faba4c8e700 1 -- 172.21.15.112:0/724847777 >> 172.21.15.112:0/724847777 conn(0x7faba006cd30 msgr2=0x7faba00aca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:11.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.163+0000 7faba4c8e700 1 -- 172.21.15.112:0/724847777 shutdown_connections 2023-08-27T11:27:11.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.163+0000 7faba4c8e700 1 -- 172.21.15.112:0/724847777 wait complete. 2023-08-27T11:27:11.167 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:27:11.175 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-27T11:27:11.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:11.176 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-08-27T11:27:11.221 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:27:11.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:11.223 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-27T11:27:11.281 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:27:11.282 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:11.299 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:27:11.300 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:27:11.300 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:11.340 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:27:11.341 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:11.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.461+0000 7fc005c97700 1 Processor -- start 2023-08-27T11:27:11.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.461+0000 7fc005c97700 1 -- start start 2023-08-27T11:27:11.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fc005c97700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0000b1400 0x7fc0000cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fc005c97700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc0000b17e0 0x7fc00006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fc005c97700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc000161850 con 0x7fc000161ce0 2023-08-27T11:27:11.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fc005c97700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00006e7d0 con 0x7fc0000b17e0 2023-08-27T11:27:11.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fc005c97700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000cfc10 con 0x7fc0000b1400 2023-08-27T11:27:11.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fbffeffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc0000b17e0 0x7fc00006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fbffffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc000161ce0 0x7fc00006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57000/0 (socket says 172.21.15.112:57000) 2023-08-27T11:27:11.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fbffffff700 1 -- 172.21.15.112:0/603290245 learned_addr learned my addr 172.21.15.112:0/603290245 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:11.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fbfff7fe700 1 --2- 172.21.15.112:0/603290245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0000b1400 0x7fc0000cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fbffeffd700 1 -- 172.21.15.112:0/603290245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0000b1400 msgr2=0x7fc0000cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:11.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fbffeffd700 1 --2- 172.21.15.112:0/603290245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0000b1400 0x7fc0000cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fbffeffd700 1 -- 172.21.15.112:0/603290245 >> v1:172.21.15.112:6789/0 conn(0x7fc000161ce0 legacy=0x7fc00006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fbffeffd700 1 -- 172.21.15.112:0/603290245 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc000161200 con 0x7fc0000b17e0 2023-08-27T11:27:11.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fbffe7fc700 1 -- 172.21.15.112:0/603290245 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900208582 0 0) 0x7fc000161850 con 0x7fc000161ce0 2023-08-27T11:27:11.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fbffeffd700 1 --2- 172.21.15.112:0/603290245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0000b17e0 0x7fc00006e290 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009840 tx=0x7fbff4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2224cc7ab5a3e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:11.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fbffe7fc700 1 -- 172.21.15.112:0/603290245 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4011830 con 0x7fc0000b17e0 2023-08-27T11:27:11.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fbffe7fc700 1 -- 172.21.15.112:0/603290245 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4011990 con 0x7fc0000b17e0 2023-08-27T11:27:11.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.462+0000 7fbffe7fc700 1 -- 172.21.15.112:0/603290245 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4011d20 con 0x7fc0000b17e0 2023-08-27T11:27:11.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.463+0000 7fc005c97700 1 -- 172.21.15.112:0/603290245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0000b17e0 msgr2=0x7fc00006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.463+0000 7fc005c97700 1 --2- 172.21.15.112:0/603290245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0000b17e0 0x7fc00006e290 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009840 tx=0x7fbff4009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:11.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.463+0000 7fc005c97700 1 -- 172.21.15.112:0/603290245 shutdown_connections 2023-08-27T11:27:11.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.463+0000 7fc005c97700 1 --2- 172.21.15.112:0/603290245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0000b1400 0x7fc0000cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.463+0000 7fc005c97700 1 --2- 172.21.15.112:0/603290245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0000b17e0 0x7fc00006e290 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.463+0000 7fc005c97700 1 -- 172.21.15.112:0/603290245 >> 172.21.15.112:0/603290245 conn(0x7fc0000d0000 msgr2=0x7fc0000d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:11.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.463+0000 7fc005c97700 1 -- 172.21.15.112:0/603290245 shutdown_connections 2023-08-27T11:27:11.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.463+0000 7fc005c97700 1 -- 172.21.15.112:0/603290245 wait complete. 2023-08-27T11:27:11.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.464+0000 7fc005c97700 1 Processor -- start 2023-08-27T11:27:11.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.465+0000 7fc005c97700 1 -- start start 2023-08-27T11:27:11.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fc005c97700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0000b1400 0x7fc0001f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fc005c97700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0000b17e0 0x7fc000152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fc005c97700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc000161ce0 0x7fc0000c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fc005c97700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000c7420 con 0x7fc000161ce0 2023-08-27T11:27:11.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fc005c97700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000c70d0 con 0x7fc0000b17e0 2023-08-27T11:27:11.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fc005c97700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0001f6820 con 0x7fc0000b1400 2023-08-27T11:27:11.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbffffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc000161ce0 0x7fc0000c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbffffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc000161ce0 0x7fc0000c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34392/0 (socket says 172.21.15.112:34392) 2023-08-27T11:27:11.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbffffff700 1 -- 172.21.15.112:0/2970707940 learned_addr learned my addr 172.21.15.112:0/2970707940 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:11.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbfff7fe700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0000b1400 0x7fc0001f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbffeffd700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0000b17e0 0x7fc000152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbffffff700 1 -- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0000b1400 msgr2=0x7fc0001f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbffffff700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0000b1400 0x7fc0001f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbffffff700 1 -- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0000b17e0 msgr2=0x7fc000152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbffffff700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0000b17e0 0x7fc000152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbffffff700 1 -- 172.21.15.112:0/2970707940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40094a0 con 0x7fc000161ce0 2023-08-27T11:27:11.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbffffff700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc000161ce0 0x7fc0000c4d50 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fbff800cba0 tx=0x7fbff800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:11.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbfe7fff700 1 -- 172.21.15.112:0/2970707940 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff800aaa0 con 0x7fc000161ce0 2023-08-27T11:27:11.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbfe7fff700 1 -- 172.21.15.112:0/2970707940 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff800ac20 con 0x7fc000161ce0 2023-08-27T11:27:11.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fc005c97700 1 -- 172.21.15.112:0/2970707940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0001f6ac0 con 0x7fc000161ce0 2023-08-27T11:27:11.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fbfe7fff700 1 -- 172.21.15.112:0/2970707940 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff80070d0 con 0x7fc000161ce0 2023-08-27T11:27:11.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.466+0000 7fc005c97700 1 -- 172.21.15.112:0/2970707940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0001f7080 con 0x7fc000161ce0 2023-08-27T11:27:11.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.467+0000 7fc005c97700 1 -- 172.21.15.112:0/2970707940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0000c7560 con 0x7fc000161ce0 2023-08-27T11:27:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.468+0000 7fbfe7fff700 1 -- 172.21.15.112:0/2970707940 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fbff8007230 con 0x7fc000161ce0 2023-08-27T11:27:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.468+0000 7fbfe7fff700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfec06a4d0 0x7fbfec06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.468+0000 7fbfe7fff700 1 -- 172.21.15.112:0/2970707940 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbff80889c0 con 0x7fc000161ce0 2023-08-27T11:27:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.469+0000 7fbfff7fe700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfec06a4d0 0x7fbfec06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.469+0000 7fbfff7fe700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfec06a4d0 0x7fbfec06c990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbff0006b20 tx=0x7fbff0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.473+0000 7fbfe7fff700 1 -- 172.21.15.112:0/2970707940 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbff8053a90 con 0x7fc000161ce0 2023-08-27T11:27:11.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.709+0000 7fc005c97700 1 -- 172.21.15.112:0/2970707940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc00015f360 con 0x7fc000161ce0 2023-08-27T11:27:11.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.710+0000 7fbfe7fff700 1 -- 172.21.15.112:0/2970707940 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fbff8018090 con 0x7fc000161ce0 2023-08-27T11:27:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:11.711 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/37643970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":37643970},{"type":"v1","addr":"172.21.15.112:6834","nonce":37643970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/568782348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":568782348},{"type":"v1","addr":"172.21.15.138:6836","nonce":568782348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1867471713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1867471713},{"type":"v1","addr":"172.21.15.138:6837","nonce":1867471713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:26:55.965268+0000","modified":"2023-08-27T11:26:59.589861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3180263819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3180263819},{"type":"v1","addr":"172.21.15.112:6835","nonce":3180263819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-27T11:27:11.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.712+0000 7fc005c97700 1 -- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfec06a4d0 msgr2=0x7fbfec06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.712+0000 7fc005c97700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfec06a4d0 0x7fbfec06c990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbff0006b20 tx=0x7fbff0008040 comp rx=0 tx=0).stop 2023-08-27T11:27:11.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.712+0000 7fc005c97700 1 -- 172.21.15.112:0/2970707940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc000161ce0 msgr2=0x7fc0000c4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.712+0000 7fc005c97700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc000161ce0 0x7fc0000c4d50 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fbff800cba0 tx=0x7fbff800ceb0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.712+0000 7fc005c97700 1 -- 172.21.15.112:0/2970707940 shutdown_connections 2023-08-27T11:27:11.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.712+0000 7fc005c97700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbfec06a4d0 0x7fbfec06c990 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.712+0000 7fc005c97700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc000161ce0 0x7fc0000c4d50 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.712+0000 7fc005c97700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0000b1400 0x7fc0001f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.712+0000 7fc005c97700 1 --2- 172.21.15.112:0/2970707940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0000b17e0 0x7fc000152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.712+0000 7fc005c97700 1 -- 172.21.15.112:0/2970707940 >> 172.21.15.112:0/2970707940 conn(0x7fc0000d0000 msgr2=0x7fc000155830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.712+0000 7fc005c97700 1 -- 172.21.15.112:0/2970707940 shutdown_connections 2023-08-27T11:27:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.712+0000 7fc005c97700 1 -- 172.21.15.112:0/2970707940 wait complete. 2023-08-27T11:27:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 68 2023-08-27T11:27:11.724 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:11.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.844+0000 7f416f543700 1 Processor -- start 2023-08-27T11:27:11.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.845+0000 7f416f543700 1 -- start start 2023-08-27T11:27:11.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.845+0000 7f416f543700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4168161860 0x7f416806e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.845+0000 7f416f543700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f41680c35b0 0x7f41680ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.845+0000 7f416f543700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f416806f790 0x7f416806fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.845+0000 7f416f543700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4168070160 con 0x7f41680c35b0 2023-08-27T11:27:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.845+0000 7f416f543700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41680c3320 con 0x7f416806f790 2023-08-27T11:27:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.845+0000 7f416f543700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41680cdd00 con 0x7f4168161860 2023-08-27T11:27:11.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.845+0000 7f416cade700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f41680c35b0 0x7f41680ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.845+0000 7f416cade700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f41680c35b0 0x7f41680ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34396/0 (socket says 172.21.15.112:34396) 2023-08-27T11:27:11.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.845+0000 7f416cade700 1 -- 172.21.15.112:0/2310252141 learned_addr learned my addr 172.21.15.112:0/2310252141 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:11.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.846+0000 7f416dae0700 1 --2- 172.21.15.112:0/2310252141 >> v2:172.21.15.138:3300/0 conn(0x7f416806f790 0x7f416806fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.846+0000 7f416d2df700 1 --2- 172.21.15.112:0/2310252141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4168161860 0x7f416806e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.846+0000 7f416cade700 1 -- 172.21.15.112:0/2310252141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4168161860 msgr2=0x7f416806e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:11.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.846+0000 7f416cade700 1 --2- 172.21.15.112:0/2310252141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4168161860 0x7f416806e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.846+0000 7f416cade700 1 -- 172.21.15.112:0/2310252141 >> v2:172.21.15.138:3300/0 conn(0x7f416806f790 msgr2=0x7f416806fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:11.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.846+0000 7f416cade700 1 --2- 172.21.15.112:0/2310252141 >> v2:172.21.15.138:3300/0 conn(0x7f416806f790 0x7f416806fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.846+0000 7f416cade700 1 -- 172.21.15.112:0/2310252141 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41680cdfe0 con 0x7f41680c35b0 2023-08-27T11:27:11.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.846+0000 7f416dae0700 1 --2- 172.21.15.112:0/2310252141 >> v2:172.21.15.138:3300/0 conn(0x7f416806f790 0x7f416806fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:11.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.846+0000 7f416d2df700 1 --2- 172.21.15.112:0/2310252141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4168161860 0x7f416806e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:11.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.846+0000 7f416cade700 1 --2- 172.21.15.112:0/2310252141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41680c35b0 0x7f41680ca650 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f4168070870 tx=0x7f4164009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73827eeb2b031c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:11.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.846+0000 7f415ffff700 1 -- 172.21.15.112:0/2310252141 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416400b870 con 0x7f41680c35b0 2023-08-27T11:27:11.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.846+0000 7f415ffff700 1 -- 172.21.15.112:0/2310252141 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416400b9d0 con 0x7f41680c35b0 2023-08-27T11:27:11.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.846+0000 7f415ffff700 1 -- 172.21.15.112:0/2310252141 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416400bd60 con 0x7f41680c35b0 2023-08-27T11:27:11.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.847+0000 7f416f543700 1 -- 172.21.15.112:0/2310252141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41680c35b0 msgr2=0x7f41680ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.847+0000 7f416f543700 1 --2- 172.21.15.112:0/2310252141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41680c35b0 0x7f41680ca650 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f4168070870 tx=0x7f4164009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.847+0000 7f416f543700 1 -- 172.21.15.112:0/2310252141 shutdown_connections 2023-08-27T11:27:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.847+0000 7f416f543700 1 --2- 172.21.15.112:0/2310252141 >> v2:172.21.15.138:3300/0 conn(0x7f416806f790 0x7f416806fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.847+0000 7f416f543700 1 --2- 172.21.15.112:0/2310252141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4168161860 0x7f416806e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.847+0000 7f416f543700 1 --2- 172.21.15.112:0/2310252141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41680c35b0 0x7f41680ca650 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.847+0000 7f416f543700 1 -- 172.21.15.112:0/2310252141 >> 172.21.15.112:0/2310252141 conn(0x7f41680d0000 msgr2=0x7f41680d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.847+0000 7f416f543700 1 -- 172.21.15.112:0/2310252141 shutdown_connections 2023-08-27T11:27:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.847+0000 7f416f543700 1 -- 172.21.15.112:0/2310252141 wait complete. 2023-08-27T11:27:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.848+0000 7f416f543700 1 Processor -- start 2023-08-27T11:27:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.848+0000 7f416f543700 1 -- start start 2023-08-27T11:27:11.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416f543700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f416806f790 0x7f416814da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416f543700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41680c35b0 0x7f416814b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416f543700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4168161860 0x7f416815e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416f543700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41680b5290 con 0x7f4168161860 2023-08-27T11:27:11.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416f543700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416814e980 con 0x7f41680c35b0 2023-08-27T11:27:11.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416f543700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416814b6f0 con 0x7f416806f790 2023-08-27T11:27:11.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416dae0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4168161860 0x7f416815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416dae0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4168161860 0x7f416815e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34406/0 (socket says 172.21.15.112:34406) 2023-08-27T11:27:11.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416dae0700 1 -- 172.21.15.112:0/183082366 learned_addr learned my addr 172.21.15.112:0/183082366 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:11.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416d2df700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f416806f790 0x7f416814da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416cade700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41680c35b0 0x7f416814b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416dae0700 1 -- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f416806f790 msgr2=0x7f416814da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416dae0700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f416806f790 0x7f416814da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416dae0700 1 -- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41680c35b0 msgr2=0x7f416814b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416dae0700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41680c35b0 0x7f416814b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416dae0700 1 -- 172.21.15.112:0/183082366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4164009550 con 0x7f4168161860 2023-08-27T11:27:11.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416dae0700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4168161860 0x7f416815e570 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f416000cd40 tx=0x7f416000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:11.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f415dffb700 1 -- 172.21.15.112:0/183082366 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416000ce90 con 0x7f4168161860 2023-08-27T11:27:11.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f415dffb700 1 -- 172.21.15.112:0/183082366 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416000f480 con 0x7f4168161860 2023-08-27T11:27:11.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f415dffb700 1 -- 172.21.15.112:0/183082366 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416000f810 con 0x7f4168161860 2023-08-27T11:27:11.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.849+0000 7f416f543700 1 -- 172.21.15.112:0/183082366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416814e0e0 con 0x7f4168161860 2023-08-27T11:27:11.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.850+0000 7f416f543700 1 -- 172.21.15.112:0/183082366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416814e6a0 con 0x7f4168161860 2023-08-27T11:27:11.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.850+0000 7f416f543700 1 -- 172.21.15.112:0/183082366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4168067a20 con 0x7f4168161860 2023-08-27T11:27:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.851+0000 7f415dffb700 1 -- 172.21.15.112:0/183082366 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f416000b040 con 0x7f4168161860 2023-08-27T11:27:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.851+0000 7f415dffb700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f414c06a4d0 0x7f414c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.851+0000 7f415dffb700 1 -- 172.21.15.112:0/183082366 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f41600894c0 con 0x7f4168161860 2023-08-27T11:27:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.851+0000 7f416d2df700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f414c06a4d0 0x7f414c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.852+0000 7f416d2df700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f414c06a4d0 0x7f414c06c990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4158006fd0 tx=0x7f4158008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:11.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:11.855+0000 7f415dffb700 1 -- 172.21.15.112:0/183082366 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f41600544c0 con 0x7f4168161860 2023-08-27T11:27:12.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.046+0000 7f416f543700 1 -- 172.21.15.112:0/183082366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f416806dbd0 con 0x7f4168161860 2023-08-27T11:27:12.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.048+0000 7f415dffb700 1 -- 172.21.15.112:0/183082366 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+18327 (secure 0 0 0) 0x7f4160057ae0 con 0x7f4168161860 2023-08-27T11:27:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:12.049 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":102,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:57.319281+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:54.078457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:54.929983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.050+0000 7f416f543700 1 -- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f414c06a4d0 msgr2=0x7f414c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.050+0000 7f416f543700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f414c06a4d0 0x7f414c06c990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4158006fd0 tx=0x7f4158008040 comp rx=0 tx=0).stop 2023-08-27T11:27:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.050+0000 7f416f543700 1 -- 172.21.15.112:0/183082366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4168161860 msgr2=0x7f416815e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.050+0000 7f416f543700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4168161860 0x7f416815e570 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f416000cd40 tx=0x7f416000a560 comp rx=0 tx=0).stop 2023-08-27T11:27:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.050+0000 7f416f543700 1 -- 172.21.15.112:0/183082366 shutdown_connections 2023-08-27T11:27:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.050+0000 7f416f543700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f414c06a4d0 0x7f414c06c990 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.050+0000 7f416f543700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4168161860 0x7f416815e570 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.050+0000 7f416f543700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f416806f790 0x7f416814da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.050+0000 7f416f543700 1 --2- 172.21.15.112:0/183082366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41680c35b0 0x7f416814b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.050+0000 7f416f543700 1 -- 172.21.15.112:0/183082366 >> 172.21.15.112:0/183082366 conn(0x7f41680d0000 msgr2=0x7f4168070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:12.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.050+0000 7f416f543700 1 -- 172.21.15.112:0/183082366 shutdown_connections 2023-08-27T11:27:12.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.050+0000 7f416f543700 1 -- 172.21.15.112:0/183082366 wait complete. 2023-08-27T11:27:12.063 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:26:54.078457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:26:54.929983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:12.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:12.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.186+0000 7f4105b85700 1 Processor -- start 2023-08-27T11:27:12.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.186+0000 7f4105b85700 1 -- start start 2023-08-27T11:27:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.186+0000 7f4105b85700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4100146c20 0x7f4100160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.186+0000 7f4105b85700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41000b5070 con 0x7f410006f790 2023-08-27T11:27:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.186+0000 7f4105b85700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4100147000 con 0x7f41000b3440 2023-08-27T11:27:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.186+0000 7f4105b85700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41001614a0 con 0x7f4100146c20 2023-08-27T11:27:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f410006f790 0x7f41000b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57028/0 (socket says 172.21.15.112:57028) 2023-08-27T11:27:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fffff700 1 -- 172.21.15.112:0/457123770 learned_addr learned my addr 172.21.15.112:0/457123770 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40ff7fe700 1 --2- 172.21.15.112:0/457123770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4100146c20 0x7f4100160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029639323 0 0) 0x7f40f8001490 con 0x7f410006f790 2023-08-27T11:27:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40ec0035b0 con 0x7f410006f790 2023-08-27T11:27:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3091289949 0 0) 0x7f40ec0035b0 con 0x7f410006f790 2023-08-27T11:27:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4100146c20 msgr2=0x7f4100160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 --2- 172.21.15.112:0/457123770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4100146c20 0x7f4100160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 >> v1:172.21.15.138:6789/0 conn(0x7f41000b3440 legacy=0x7f410015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4100161740 con 0x7f410006f790 2023-08-27T11:27:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f40f8001b80 con 0x7f410006f790 2023-08-27T11:27:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 >> v1:172.21.15.112:6789/0 conn(0x7f410006f790 legacy=0x7f41000b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 --2- 172.21.15.112:0/457123770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec003a30 0x7f40ec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 --2- 172.21.15.112:0/457123770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec0043e0 0x7f40ec0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 --2- 172.21.15.112:0/457123770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4100146c20 0x7f40ec0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41001614a0 con 0x7f40ec0043e0 2023-08-27T11:27:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec00bac0 con 0x7f40ec003a30 2023-08-27T11:27:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec00bc80 con 0x7f4100146c20 2023-08-27T11:27:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40feffd700 1 --2- 172.21.15.112:0/457123770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec0043e0 0x7f40ec0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40ff7fe700 1 --2- 172.21.15.112:0/457123770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec003a30 0x7f40ec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40fffff700 1 --2- 172.21.15.112:0/457123770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4100146c20 0x7f40ec0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40feffd700 1 -- 172.21.15.112:0/457123770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4100146c20 msgr2=0x7f40ec0051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.187+0000 7f40feffd700 1 --2- 172.21.15.112:0/457123770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4100146c20 0x7f40ec0051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f40feffd700 1 -- 172.21.15.112:0/457123770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec003a30 msgr2=0x7f40ec003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f40feffd700 1 --2- 172.21.15.112:0/457123770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec003a30 0x7f40ec003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f40feffd700 1 -- 172.21.15.112:0/457123770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec0035b0 con 0x7f40ec0043e0 2023-08-27T11:27:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f40feffd700 1 --2- 172.21.15.112:0/457123770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec0043e0 0x7f40ec0048b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f40f00018d0 tx=0x7f40ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9923b15f8a84a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f000c690 con 0x7f40ec0043e0 2023-08-27T11:27:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f000c7f0 con 0x7f40ec0043e0 2023-08-27T11:27:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f40fe7fc700 1 -- 172.21.15.112:0/457123770 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f000cba0 con 0x7f40ec0043e0 2023-08-27T11:27:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f4105b85700 1 -- 172.21.15.112:0/457123770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec0043e0 msgr2=0x7f40ec0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f4105b85700 1 --2- 172.21.15.112:0/457123770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec0043e0 0x7f40ec0048b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f40f00018d0 tx=0x7f40ec005880 comp rx=0 tx=0).stop 2023-08-27T11:27:12.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f40fffff700 1 -- 172.21.15.112:0/457123770 reap_dead start 2023-08-27T11:27:12.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f4105b85700 1 -- 172.21.15.112:0/457123770 shutdown_connections 2023-08-27T11:27:12.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f4105b85700 1 -- 172.21.15.112:0/457123770 >> 172.21.15.112:0/457123770 conn(0x7f41000b1e90 msgr2=0x7f41000b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:12.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f4105b85700 1 -- 172.21.15.112:0/457123770 shutdown_connections 2023-08-27T11:27:12.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.188+0000 7f4105b85700 1 -- 172.21.15.112:0/457123770 wait complete. 2023-08-27T11:27:12.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.190+0000 7f4105b85700 1 Processor -- start 2023-08-27T11:27:12.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.190+0000 7f4105b85700 1 -- start start 2023-08-27T11:27:12.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.190+0000 7f4105b85700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec0043e0 0x7f41000c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f4105b85700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410015f490 0x7f410013a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f4105b85700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f410015f910 0x7f410013acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f4105b85700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41000c6f10 con 0x7f410015f490 2023-08-27T11:27:12.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f4105b85700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41000c71f0 con 0x7f40ec0043e0 2023-08-27T11:27:12.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f4105b85700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41000c74d0 con 0x7f410015f910 2023-08-27T11:27:12.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40feffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410015f490 0x7f410013a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40feffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410015f490 0x7f410013a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34424/0 (socket says 172.21.15.112:34424) 2023-08-27T11:27:12.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40feffd700 1 -- 172.21.15.112:0/3727670104 learned_addr learned my addr 172.21.15.112:0/3727670104 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:12.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40fffff700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f410015f910 0x7f410013acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40ff7fe700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec0043e0 0x7f41000c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40feffd700 1 -- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f410015f910 msgr2=0x7f410013acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40feffd700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f410015f910 0x7f410013acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40feffd700 1 -- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec0043e0 msgr2=0x7f41000c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40feffd700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec0043e0 0x7f41000c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40feffd700 1 -- 172.21.15.112:0/3727670104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41000c77b0 con 0x7f410015f490 2023-08-27T11:27:12.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40feffd700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410015f490 0x7f410013a7b0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f40f000c0e0 tx=0x7f40f0001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:12.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40f7fff700 1 -- 172.21.15.112:0/3727670104 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f000cb60 con 0x7f410015f490 2023-08-27T11:27:12.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40f7fff700 1 -- 172.21.15.112:0/3727670104 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f0021070 con 0x7f410015f490 2023-08-27T11:27:12.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f4105b85700 1 -- 172.21.15.112:0/3727670104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41000c7a90 con 0x7f410015f490 2023-08-27T11:27:12.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f40f7fff700 1 -- 172.21.15.112:0/3727670104 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f0001330 con 0x7f410015f490 2023-08-27T11:27:12.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.191+0000 7f4105b85700 1 -- 172.21.15.112:0/3727670104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41000c6220 con 0x7f410015f490 2023-08-27T11:27:12.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.192+0000 7f4105b85700 1 -- 172.21.15.112:0/3727670104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41000c7e00 con 0x7f410015f490 2023-08-27T11:27:12.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.193+0000 7f40f7fff700 1 -- 172.21.15.112:0/3727670104 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f40f0023030 con 0x7f410015f490 2023-08-27T11:27:12.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.193+0000 7f40f7fff700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f40e006a4d0 0x7f40e006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.193+0000 7f40f7fff700 1 -- 172.21.15.112:0/3727670104 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40f0089a10 con 0x7f410015f490 2023-08-27T11:27:12.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.193+0000 7f40ff7fe700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f40e006a4d0 0x7f40e006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.194+0000 7f40ff7fe700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f40e006a4d0 0x7f40e006c990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009c60 tx=0x7f40e8009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:12.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.198+0000 7f40f7fff700 1 -- 172.21.15.112:0/3727670104 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f40f0054cc0 con 0x7f410015f490 2023-08-27T11:27:12.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.388+0000 7f4105b85700 1 -- 172.21.15.112:0/3727670104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4100067e90 con 0x7f410015f490 2023-08-27T11:27:12.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.388+0000 7f40f7fff700 1 -- 172.21.15.112:0/3727670104 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v102) v1 ==== 92+0+10 (secure 0 0 0) 0x7f40f00582e0 con 0x7f410015f490 2023-08-27T11:27:12.390 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:27:12.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.390+0000 7f4105b85700 1 -- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f40e006a4d0 msgr2=0x7f40e006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.390+0000 7f4105b85700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f40e006a4d0 0x7f40e006c990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009c60 tx=0x7f40e8009210 comp rx=0 tx=0).stop 2023-08-27T11:27:12.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.390+0000 7f4105b85700 1 -- 172.21.15.112:0/3727670104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410015f490 msgr2=0x7f410013a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.390+0000 7f4105b85700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410015f490 0x7f410013a7b0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f40f000c0e0 tx=0x7f40f0001230 comp rx=0 tx=0).stop 2023-08-27T11:27:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.390+0000 7f4105b85700 1 -- 172.21.15.112:0/3727670104 shutdown_connections 2023-08-27T11:27:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.390+0000 7f4105b85700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f40e006a4d0 0x7f40e006c990 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.390+0000 7f4105b85700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f410015f910 0x7f410013acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.390+0000 7f4105b85700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec0043e0 0x7f41000c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.390+0000 7f4105b85700 1 --2- 172.21.15.112:0/3727670104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410015f490 0x7f410013a7b0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.390+0000 7f4105b85700 1 -- 172.21.15.112:0/3727670104 >> 172.21.15.112:0/3727670104 conn(0x7f41000b1e90 msgr2=0x7f4100160d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:12.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.390+0000 7f4105b85700 1 -- 172.21.15.112:0/3727670104 shutdown_connections 2023-08-27T11:27:12.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.390+0000 7f4105b85700 1 -- 172.21.15.112:0/3727670104 wait complete. 2023-08-27T11:27:12.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:12.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.525+0000 7f4877b12700 1 Processor -- start 2023-08-27T11:27:12.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.525+0000 7f4877b12700 1 -- start start 2023-08-27T11:27:12.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.526+0000 7f4877b12700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4870146c20 0x7f4870160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.526+0000 7f4877b12700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48700b3440 0x7f487015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.526+0000 7f4877b12700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48700b5070 con 0x7f487015e6e0 2023-08-27T11:27:12.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.526+0000 7f4877b12700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4870147000 con 0x7f48700b3440 2023-08-27T11:27:12.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.526+0000 7f4877b12700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48701614a0 con 0x7f4870146c20 2023-08-27T11:27:12.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.526+0000 7f48750ad700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48700b3440 0x7f487015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.526+0000 7f48750ad700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48700b3440 0x7f487015e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34432/0 (socket says 172.21.15.112:34432) 2023-08-27T11:27:12.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.526+0000 7f48750ad700 1 -- 172.21.15.112:0/978171846 learned_addr learned my addr 172.21.15.112:0/978171846 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:12.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.527+0000 7f48758ae700 1 --2- 172.21.15.112:0/978171846 >> v2:172.21.15.138:3300/0 conn(0x7f4870146c20 0x7f4870160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.527+0000 7f48750ad700 1 -- 172.21.15.112:0/978171846 >> v2:172.21.15.138:3300/0 conn(0x7f4870146c20 msgr2=0x7f4870160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:12.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.527+0000 7f48750ad700 1 --2- 172.21.15.112:0/978171846 >> v2:172.21.15.138:3300/0 conn(0x7f4870146c20 0x7f4870160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.527+0000 7f48750ad700 1 -- 172.21.15.112:0/978171846 >> v1:172.21.15.138:6789/0 conn(0x7f487015e6e0 legacy=0x7f487006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:12.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.527+0000 7f48750ad700 1 -- 172.21.15.112:0/978171846 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870161740 con 0x7f48700b3440 2023-08-27T11:27:12.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.527+0000 7f48750ad700 1 --2- 172.21.15.112:0/978171846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48700b3440 0x7f487015e1a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f48700b2d00 tx=0x7f486c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926e2d91352cc0a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:12.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.527+0000 7f48748ac700 1 -- 172.21.15.112:0/978171846 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c004230 con 0x7f48700b3440 2023-08-27T11:27:12.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.527+0000 7f48748ac700 1 -- 172.21.15.112:0/978171846 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c004390 con 0x7f48700b3440 2023-08-27T11:27:12.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.527+0000 7f48748ac700 1 -- 172.21.15.112:0/978171846 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c0036a0 con 0x7f48700b3440 2023-08-27T11:27:12.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.527+0000 7f4877b12700 1 -- 172.21.15.112:0/978171846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48700b3440 msgr2=0x7f487015e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.527+0000 7f4877b12700 1 --2- 172.21.15.112:0/978171846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48700b3440 0x7f487015e1a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f48700b2d00 tx=0x7f486c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.528+0000 7f4877b12700 1 -- 172.21.15.112:0/978171846 shutdown_connections 2023-08-27T11:27:12.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.528+0000 7f4877b12700 1 --2- 172.21.15.112:0/978171846 >> v2:172.21.15.138:3300/0 conn(0x7f4870146c20 0x7f4870160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.528+0000 7f4877b12700 1 --2- 172.21.15.112:0/978171846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48700b3440 0x7f487015e1a0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.528+0000 7f4877b12700 1 -- 172.21.15.112:0/978171846 >> 172.21.15.112:0/978171846 conn(0x7f48700b1e90 msgr2=0x7f48700b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:12.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.528+0000 7f4877b12700 1 -- 172.21.15.112:0/978171846 shutdown_connections 2023-08-27T11:27:12.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.528+0000 7f4877b12700 1 -- 172.21.15.112:0/978171846 wait complete. 2023-08-27T11:27:12.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.529+0000 7f4877b12700 1 Processor -- start 2023-08-27T11:27:12.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.529+0000 7f4877b12700 1 -- start start 2023-08-27T11:27:12.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.529+0000 7f4877b12700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48700b3440 0x7f48700ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.529+0000 7f4877b12700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4870146c20 0x7f487012efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.529+0000 7f4877b12700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487015e6e0 0x7f48701410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f4877b12700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487013e750 con 0x7f48700b3440 2023-08-27T11:27:12.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f4877b12700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48700ae380 con 0x7f487015e6e0 2023-08-27T11:27:12.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f4877b12700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487013e120 con 0x7f4870146c20 2023-08-27T11:27:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48758ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48700b3440 0x7f48700ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48758ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48700b3440 0x7f48700ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34436/0 (socket says 172.21.15.112:34436) 2023-08-27T11:27:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48758ae700 1 -- 172.21.15.112:0/602295963 learned_addr learned my addr 172.21.15.112:0/602295963 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48750ad700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4870146c20 0x7f487012efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48760af700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487015e6e0 0x7f48701410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48758ae700 1 -- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4870146c20 msgr2=0x7f487012efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48758ae700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4870146c20 0x7f487012efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48758ae700 1 -- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487015e6e0 msgr2=0x7f48701410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48758ae700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487015e6e0 0x7f48701410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48758ae700 1 -- 172.21.15.112:0/602295963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c009550 con 0x7f48700b3440 2023-08-27T11:27:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48758ae700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48700b3440 0x7f48700ade40 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f486000c7f0 tx=0x7f486000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48667fc700 1 -- 172.21.15.112:0/602295963 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860011800 con 0x7f48700b3440 2023-08-27T11:27:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48667fc700 1 -- 172.21.15.112:0/602295963 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4860011960 con 0x7f48700b3440 2023-08-27T11:27:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f4877b12700 1 -- 172.21.15.112:0/602295963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487012fa20 con 0x7f48700b3440 2023-08-27T11:27:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f48667fc700 1 -- 172.21.15.112:0/602295963 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860011d10 con 0x7f48700b3440 2023-08-27T11:27:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.530+0000 7f4877b12700 1 -- 172.21.15.112:0/602295963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487013e370 con 0x7f48700b3440 2023-08-27T11:27:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.531+0000 7f4877b12700 1 -- 172.21.15.112:0/602295963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4870067e90 con 0x7f48700b3440 2023-08-27T11:27:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.531+0000 7f48667fc700 1 -- 172.21.15.112:0/602295963 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4860018360 con 0x7f48700b3440 2023-08-27T11:27:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.532+0000 7f48667fc700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f485406a4d0 0x7f485406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.532+0000 7f48667fc700 1 -- 172.21.15.112:0/602295963 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4860088d20 con 0x7f48700b3440 2023-08-27T11:27:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.532+0000 7f48750ad700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f485406a4d0 0x7f485406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.532+0000 7f48750ad700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f485406a4d0 0x7f485406c990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f48700b2d00 tx=0x7f486c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.536+0000 7f48667fc700 1 -- 172.21.15.112:0/602295963 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4860053db0 con 0x7f48700b3440 2023-08-27T11:27:12.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.725+0000 7f4877b12700 1 -- 172.21.15.112:0/602295963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48700b3d20 con 0x7f48700b3440 2023-08-27T11:27:12.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.725+0000 7f48667fc700 1 -- 172.21.15.112:0/602295963 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48600573d0 con 0x7f48700b3440 2023-08-27T11:27:12.727 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:27:12.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.727+0000 7f4877b12700 1 -- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f485406a4d0 msgr2=0x7f485406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.727+0000 7f4877b12700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f485406a4d0 0x7f485406c990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f48700b2d00 tx=0x7f486c008030 comp rx=0 tx=0).stop 2023-08-27T11:27:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.727+0000 7f4877b12700 1 -- 172.21.15.112:0/602295963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48700b3440 msgr2=0x7f48700ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.727+0000 7f4877b12700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48700b3440 0x7f48700ade40 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f486000c7f0 tx=0x7f486000cbb0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.728+0000 7f4877b12700 1 -- 172.21.15.112:0/602295963 shutdown_connections 2023-08-27T11:27:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.728+0000 7f4877b12700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f485406a4d0 0x7f485406c990 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.728+0000 7f4877b12700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487015e6e0 0x7f48701410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.728+0000 7f4877b12700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48700b3440 0x7f48700ade40 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.728+0000 7f4877b12700 1 --2- 172.21.15.112:0/602295963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4870146c20 0x7f487012efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.728+0000 7f4877b12700 1 -- 172.21.15.112:0/602295963 >> 172.21.15.112:0/602295963 conn(0x7f48700b1e90 msgr2=0x7f4870160d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.728+0000 7f4877b12700 1 -- 172.21.15.112:0/602295963 shutdown_connections 2023-08-27T11:27:12.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.728+0000 7f4877b12700 1 -- 172.21.15.112:0/602295963 wait complete. 2023-08-27T11:27:12.742 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:27:12.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.864+0000 7f4fe7020700 1 Processor -- start 2023-08-27T11:27:12.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.864+0000 7f4fe7020700 1 -- start start 2023-08-27T11:27:12.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.865+0000 7f4fe7020700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fe0146c20 0x7f4fe0160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.865+0000 7f4fe7020700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4fe006f790 0x7f4fe006fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.865+0000 7f4fe7020700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe00b5070 con 0x7f4fe00b3440 2023-08-27T11:27:12.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.865+0000 7f4fe7020700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe0070160 con 0x7f4fe006f790 2023-08-27T11:27:12.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.865+0000 7f4fe7020700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe0147000 con 0x7f4fe0146c20 2023-08-27T11:27:12.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.865+0000 7f4fe55bd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4fe006f790 0x7f4fe006fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.865+0000 7f4fdffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4fe00b3440 0x7f4fe015e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57060/0 (socket says 172.21.15.112:57060) 2023-08-27T11:27:12.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.865+0000 7f4fdffff700 1 -- 172.21.15.112:0/2409790155 learned_addr learned my addr 172.21.15.112:0/2409790155 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:12.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.865+0000 7f4fe55bd700 1 -- 172.21.15.112:0/2409790155 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fe0146c20 msgr2=0x7f4fe0160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:12.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.865+0000 7f4fe55bd700 1 --2- 172.21.15.112:0/2409790155 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fe0146c20 0x7f4fe0160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.865+0000 7f4fe55bd700 1 -- 172.21.15.112:0/2409790155 >> v1:172.21.15.112:6789/0 conn(0x7f4fe00b3440 legacy=0x7f4fe015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.865+0000 7f4fe55bd700 1 -- 172.21.15.112:0/2409790155 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe01614a0 con 0x7f4fe006f790 2023-08-27T11:27:12.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.865+0000 7f4fdf7fe700 1 -- 172.21.15.112:0/2409790155 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604607677 0 0) 0x7f4fcc001110 con 0x7f4fe00b3440 2023-08-27T11:27:12.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.866+0000 7f4fe55bd700 1 --2- 172.21.15.112:0/2409790155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe006f790 0x7f4fe006fc20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009d90 tx=0x7f4fd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc82eead1dfee746 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:12.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.866+0000 7f4fdf7fe700 1 -- 172.21.15.112:0/2409790155 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd800e070 con 0x7f4fe006f790 2023-08-27T11:27:12.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.866+0000 7f4fdf7fe700 1 -- 172.21.15.112:0/2409790155 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd800bbd0 con 0x7f4fe006f790 2023-08-27T11:27:12.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.866+0000 7f4fdf7fe700 1 -- 172.21.15.112:0/2409790155 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8012430 con 0x7f4fe006f790 2023-08-27T11:27:12.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.866+0000 7f4fe7020700 1 -- 172.21.15.112:0/2409790155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe006f790 msgr2=0x7f4fe006fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.866+0000 7f4fe7020700 1 --2- 172.21.15.112:0/2409790155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe006f790 0x7f4fe006fc20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009d90 tx=0x7f4fd800b330 comp rx=0 tx=0).stop 2023-08-27T11:27:12.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.866+0000 7f4fe7020700 1 -- 172.21.15.112:0/2409790155 shutdown_connections 2023-08-27T11:27:12.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.866+0000 7f4fe7020700 1 --2- 172.21.15.112:0/2409790155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe006f790 0x7f4fe006fc20 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.866+0000 7f4fe7020700 1 --2- 172.21.15.112:0/2409790155 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fe0146c20 0x7f4fe0160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.866+0000 7f4fe7020700 1 -- 172.21.15.112:0/2409790155 >> 172.21.15.112:0/2409790155 conn(0x7f4fe00b1e90 msgr2=0x7f4fe00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:12.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.866+0000 7f4fe7020700 1 -- 172.21.15.112:0/2409790155 shutdown_connections 2023-08-27T11:27:12.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.866+0000 7f4fe7020700 1 -- 172.21.15.112:0/2409790155 wait complete. 2023-08-27T11:27:12.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.867+0000 7f4fe7020700 1 Processor -- start 2023-08-27T11:27:12.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.867+0000 7f4fe7020700 1 -- start start 2023-08-27T11:27:12.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.868+0000 7f4fe7020700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe006f790 0x7f4fe0142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.868+0000 7f4fe7020700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fe00b3440 0x7f4fe00c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.868+0000 7f4fe7020700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0146c20 0x7f4fe00c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.868+0000 7f4fe7020700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe015c610 con 0x7f4fe0146c20 2023-08-27T11:27:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.868+0000 7f4fe7020700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe015c180 con 0x7f4fe006f790 2023-08-27T11:27:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.868+0000 7f4fe7020700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe0143100 con 0x7f4fe00b3440 2023-08-27T11:27:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.868+0000 7f4fe55bd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0146c20 0x7f4fe00c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.868+0000 7f4fe55bd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0146c20 0x7f4fe00c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34454/0 (socket says 172.21.15.112:34454) 2023-08-27T11:27:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.868+0000 7f4fe55bd700 1 -- 172.21.15.112:0/4155683313 learned_addr learned my addr 172.21.15.112:0/4155683313 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.869+0000 7f4fdffff700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fe00b3440 0x7f4fe00c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.869+0000 7f4fe55bd700 1 -- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fe00b3440 msgr2=0x7f4fe00c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.869+0000 7f4fe55bd700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fe00b3440 0x7f4fe00c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.869+0000 7f4fe55bd700 1 -- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe006f790 msgr2=0x7f4fe0142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.869+0000 7f4fe4dbc700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe006f790 0x7f4fe0142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.869+0000 7f4fe55bd700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe006f790 0x7f4fe0142bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.869+0000 7f4fe55bd700 1 -- 172.21.15.112:0/4155683313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd8009a70 con 0x7f4fe0146c20 2023-08-27T11:27:12.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.869+0000 7f4fe55bd700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0146c20 0x7f4fe00c6d80 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800b370 tx=0x7f4fd8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.869+0000 7f4fdd7fa700 1 -- 172.21.15.112:0/4155683313 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8003ea0 con 0x7f4fe0146c20 2023-08-27T11:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.869+0000 7f4fdd7fa700 1 -- 172.21.15.112:0/4155683313 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd8020070 con 0x7f4fe0146c20 2023-08-27T11:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.869+0000 7f4fe7020700 1 -- 172.21.15.112:0/4155683313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe015b570 con 0x7f4fe0146c20 2023-08-27T11:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.869+0000 7f4fdd7fa700 1 -- 172.21.15.112:0/4155683313 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8004280 con 0x7f4fe0146c20 2023-08-27T11:27:12.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.869+0000 7f4fe7020700 1 -- 172.21.15.112:0/4155683313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe015bb30 con 0x7f4fe0146c20 2023-08-27T11:27:12.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.870+0000 7f4fe7020700 1 -- 172.21.15.112:0/4155683313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe0067e90 con 0x7f4fe0146c20 2023-08-27T11:27:12.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.870+0000 7f4fdd7fa700 1 -- 172.21.15.112:0/4155683313 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4fd80043e0 con 0x7f4fe0146c20 2023-08-27T11:27:12.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.871+0000 7f4fdd7fa700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4fc406a4d0 0x7f4fc406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.871+0000 7f4fdd7fa700 1 -- 172.21.15.112:0/4155683313 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4fd8022070 con 0x7f4fe0146c20 2023-08-27T11:27:12.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.871+0000 7f4fe4dbc700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4fc406a4d0 0x7f4fc406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.872+0000 7f4fe4dbc700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4fc406a4d0 0x7f4fc406c990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00c6700 tx=0x7f4fd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:12.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:12.875+0000 7f4fdd7fa700 1 -- 172.21.15.112:0/4155683313 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4fd808f070 con 0x7f4fe0146c20 2023-08-27T11:27:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.060+0000 7f4fe7020700 1 -- 172.21.15.112:0/4155683313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4fe0067a20 con 0x7f4fe0146c20 2023-08-27T11:27:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.061+0000 7f4fdd7fa700 1 -- 172.21.15.112:0/4155683313 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4fd8056110 con 0x7f4fe0146c20 2023-08-27T11:27:13.063 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:27:13.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.063+0000 7f4fe7020700 1 -- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4fc406a4d0 msgr2=0x7f4fc406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.063+0000 7f4fe7020700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4fc406a4d0 0x7f4fc406c990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00c6700 tx=0x7f4fd4008040 comp rx=0 tx=0).stop 2023-08-27T11:27:13.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.063+0000 7f4fe7020700 1 -- 172.21.15.112:0/4155683313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0146c20 msgr2=0x7f4fe00c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.063+0000 7f4fe7020700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0146c20 0x7f4fe00c6d80 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800b370 tx=0x7f4fd8004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.063+0000 7f4fe7020700 1 -- 172.21.15.112:0/4155683313 shutdown_connections 2023-08-27T11:27:13.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.063+0000 7f4fe7020700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4fc406a4d0 0x7f4fc406c990 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.063+0000 7f4fe7020700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0146c20 0x7f4fe00c6d80 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.063+0000 7f4fe7020700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe006f790 0x7f4fe0142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.063+0000 7f4fe7020700 1 --2- 172.21.15.112:0/4155683313 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fe00b3440 0x7f4fe00c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.063+0000 7f4fe7020700 1 -- 172.21.15.112:0/4155683313 >> 172.21.15.112:0/4155683313 conn(0x7f4fe00b1e90 msgr2=0x7f4fe0071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:13.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.063+0000 7f4fe7020700 1 -- 172.21.15.112:0/4155683313 shutdown_connections 2023-08-27T11:27:13.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.063+0000 7f4fe7020700 1 -- 172.21.15.112:0/4155683313 wait complete. 2023-08-27T11:27:13.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:27:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.198+0000 7f48f26bb700 1 Processor -- start 2023-08-27T11:27:13.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.198+0000 7f48f26bb700 1 -- start start 2023-08-27T11:27:13.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48f26bb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48ec146c20 0x7f48ec160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48f26bb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48ec06f790 0x7f48ec06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48f26bb700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec0b5070 con 0x7f48ec0b3440 2023-08-27T11:27:13.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48f26bb700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ec070160 con 0x7f48ec06f790 2023-08-27T11:27:13.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48f26bb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec147000 con 0x7f48ec146c20 2023-08-27T11:27:13.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48f0c58700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48ec06f790 0x7f48ec06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48f0c58700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48ec06f790 0x7f48ec06fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34462/0 (socket says 172.21.15.112:34462) 2023-08-27T11:27:13.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48f0c58700 1 -- 172.21.15.112:0/4070754038 learned_addr learned my addr 172.21.15.112:0/4070754038 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:13.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48ebfff700 1 --2- 172.21.15.112:0/4070754038 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48ec146c20 0x7f48ec160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48f0c58700 1 -- 172.21.15.112:0/4070754038 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48ec146c20 msgr2=0x7f48ec160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:13.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48f0c58700 1 --2- 172.21.15.112:0/4070754038 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48ec146c20 0x7f48ec160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48f0c58700 1 -- 172.21.15.112:0/4070754038 >> v1:172.21.15.138:6789/0 conn(0x7f48ec0b3440 legacy=0x7f48ec15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48f0c58700 1 -- 172.21.15.112:0/4070754038 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec1614a0 con 0x7f48ec06f790 2023-08-27T11:27:13.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.199+0000 7f48f0c58700 1 --2- 172.21.15.112:0/4070754038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48ec06f790 0x7f48ec06fc20 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009e50 tx=0x7f48e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98764e239c3128b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:13.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.200+0000 7f48eaffd700 1 -- 172.21.15.112:0/4070754038 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e400e070 con 0x7f48ec06f790 2023-08-27T11:27:13.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.200+0000 7f48eaffd700 1 -- 172.21.15.112:0/4070754038 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e4003a20 con 0x7f48ec06f790 2023-08-27T11:27:13.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.200+0000 7f48eaffd700 1 -- 172.21.15.112:0/4070754038 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4003dd0 con 0x7f48ec06f790 2023-08-27T11:27:13.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.200+0000 7f48f26bb700 1 -- 172.21.15.112:0/4070754038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48ec06f790 msgr2=0x7f48ec06fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.200+0000 7f48f26bb700 1 --2- 172.21.15.112:0/4070754038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48ec06f790 0x7f48ec06fc20 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009e50 tx=0x7f48e400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:13.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.200+0000 7f48f26bb700 1 -- 172.21.15.112:0/4070754038 shutdown_connections 2023-08-27T11:27:13.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.200+0000 7f48f26bb700 1 --2- 172.21.15.112:0/4070754038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48ec06f790 0x7f48ec06fc20 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.200+0000 7f48f26bb700 1 --2- 172.21.15.112:0/4070754038 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48ec146c20 0x7f48ec160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.200+0000 7f48f26bb700 1 -- 172.21.15.112:0/4070754038 >> 172.21.15.112:0/4070754038 conn(0x7f48ec0b1e90 msgr2=0x7f48ec0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:13.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.200+0000 7f48f26bb700 1 -- 172.21.15.112:0/4070754038 shutdown_connections 2023-08-27T11:27:13.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.200+0000 7f48f26bb700 1 -- 172.21.15.112:0/4070754038 wait complete. 2023-08-27T11:27:13.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.201+0000 7f48f26bb700 1 Processor -- start 2023-08-27T11:27:13.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.201+0000 7f48f26bb700 1 -- start start 2023-08-27T11:27:13.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.202+0000 7f48f26bb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48ec06f790 0x7f48ec0c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.202+0000 7f48f26bb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48ec0b3440 0x7f48ec15f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.202+0000 7f48f26bb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48ec146c20 0x7f48ec0ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.202+0000 7f48f26bb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0c6270 con 0x7f48ec06f790 2023-08-27T11:27:13.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.202+0000 7f48f26bb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0c55f0 con 0x7f48ec146c20 2023-08-27T11:27:13.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.202+0000 7f48f26bb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0c58d0 con 0x7f48ec0b3440 2023-08-27T11:27:13.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.202+0000 7f48ebfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48ec06f790 0x7f48ec0c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.202+0000 7f48ebfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48ec06f790 0x7f48ec0c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34464/0 (socket says 172.21.15.112:34464) 2023-08-27T11:27:13.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.202+0000 7f48ebfff700 1 -- 172.21.15.112:0/2990282227 learned_addr learned my addr 172.21.15.112:0/2990282227 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:13.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.202+0000 7f48f0c58700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48ec146c20 0x7f48ec0ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.202+0000 7f48eb7fe700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48ec0b3440 0x7f48ec15f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.203+0000 7f48ebfff700 1 -- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48ec0b3440 msgr2=0x7f48ec15f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.203+0000 7f48ebfff700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48ec0b3440 0x7f48ec15f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.203+0000 7f48ebfff700 1 -- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48ec146c20 msgr2=0x7f48ec0ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.203+0000 7f48ebfff700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48ec146c20 0x7f48ec0ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.203+0000 7f48ebfff700 1 -- 172.21.15.112:0/2990282227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e4009b30 con 0x7f48ec06f790 2023-08-27T11:27:13.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.203+0000 7f48eb7fe700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48ec0b3440 0x7f48ec15f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:13.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.203+0000 7f48ebfff700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48ec06f790 0x7f48ec0c6690 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f48e000d800 tx=0x7f48e000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:13.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.203+0000 7f48e8ff9700 1 -- 172.21.15.112:0/2990282227 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e00125f0 con 0x7f48ec06f790 2023-08-27T11:27:13.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.203+0000 7f48e8ff9700 1 -- 172.21.15.112:0/2990282227 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e0012750 con 0x7f48ec06f790 2023-08-27T11:27:13.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.203+0000 7f48f26bb700 1 -- 172.21.15.112:0/2990282227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec0c5bb0 con 0x7f48ec06f790 2023-08-27T11:27:13.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.203+0000 7f48e8ff9700 1 -- 172.21.15.112:0/2990282227 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e0012b00 con 0x7f48ec06f790 2023-08-27T11:27:13.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.203+0000 7f48f26bb700 1 -- 172.21.15.112:0/2990282227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec15fe80 con 0x7f48ec06f790 2023-08-27T11:27:13.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.204+0000 7f48f26bb700 1 -- 172.21.15.112:0/2990282227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec0c64e0 con 0x7f48ec06f790 2023-08-27T11:27:13.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.204+0000 7f48e8ff9700 1 -- 172.21.15.112:0/2990282227 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f48e0012c60 con 0x7f48ec06f790 2023-08-27T11:27:13.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.205+0000 7f48e8ff9700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48d806a4d0 0x7f48d806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.205+0000 7f48e8ff9700 1 -- 172.21.15.112:0/2990282227 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f48e0088f20 con 0x7f48ec06f790 2023-08-27T11:27:13.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.205+0000 7f48eb7fe700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48d806a4d0 0x7f48d806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.205+0000 7f48eb7fe700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48d806a4d0 0x7f48d806c990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f48ec1f5fb0 tx=0x7f48dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:13.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.209+0000 7f48e8ff9700 1 -- 172.21.15.112:0/2990282227 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f48e00540c0 con 0x7f48ec06f790 2023-08-27T11:27:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.398+0000 7f48f26bb700 1 -- 172.21.15.112:0/2990282227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48ec067e90 con 0x7f48ec06f790 2023-08-27T11:27:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.398+0000 7f48e8ff9700 1 -- 172.21.15.112:0/2990282227 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7f48e00576e0 con 0x7f48ec06f790 2023-08-27T11:27:13.400 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:27:13.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.400+0000 7f48f26bb700 1 -- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48d806a4d0 msgr2=0x7f48d806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.400+0000 7f48f26bb700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48d806a4d0 0x7f48d806c990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f48ec1f5fb0 tx=0x7f48dc009040 comp rx=0 tx=0).stop 2023-08-27T11:27:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.400+0000 7f48f26bb700 1 -- 172.21.15.112:0/2990282227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48ec06f790 msgr2=0x7f48ec0c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.400+0000 7f48f26bb700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48ec06f790 0x7f48ec0c6690 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f48e000d800 tx=0x7f48e000db10 comp rx=0 tx=0).stop 2023-08-27T11:27:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.400+0000 7f48f26bb700 1 -- 172.21.15.112:0/2990282227 shutdown_connections 2023-08-27T11:27:13.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.400+0000 7f48f26bb700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f48d806a4d0 0x7f48d806c990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.400+0000 7f48f26bb700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48ec146c20 0x7f48ec0ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.400+0000 7f48f26bb700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48ec06f790 0x7f48ec0c6690 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.400+0000 7f48f26bb700 1 --2- 172.21.15.112:0/2990282227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48ec0b3440 0x7f48ec15f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.400+0000 7f48f26bb700 1 -- 172.21.15.112:0/2990282227 >> 172.21.15.112:0/2990282227 conn(0x7f48ec0b1e90 msgr2=0x7f48ec145e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:13.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.400+0000 7f48f26bb700 1 -- 172.21.15.112:0/2990282227 shutdown_connections 2023-08-27T11:27:13.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.400+0000 7f48f26bb700 1 -- 172.21.15.112:0/2990282227 wait complete. 2023-08-27T11:27:13.413 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:13.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.532+0000 7fe0eecbd700 1 Processor -- start 2023-08-27T11:27:13.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.532+0000 7fe0eecbd700 1 -- start start 2023-08-27T11:27:13.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.533+0000 7fe0eecbd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe0e80b17e0 0x7fe0e80cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.533+0000 7fe0eecbd700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e8161850 con 0x7fe0e8161ce0 2023-08-27T11:27:13.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.533+0000 7fe0eecbd700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e80cfc10 con 0x7fe0e80b1400 2023-08-27T11:27:13.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.533+0000 7fe0eecbd700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e8161200 con 0x7fe0e80b17e0 2023-08-27T11:27:13.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.533+0000 7fe0ed25a700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe0e8161ce0 0x7fe0e806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57090/0 (socket says 172.21.15.112:57090) 2023-08-27T11:27:13.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.533+0000 7fe0ed25a700 1 -- 172.21.15.112:0/3628065576 learned_addr learned my addr 172.21.15.112:0/3628065576 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:13.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.533+0000 7fe0e7fff700 1 --2- 172.21.15.112:0/3628065576 >> v2:172.21.15.138:3300/0 conn(0x7fe0e80b17e0 0x7fe0e80cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.533+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527364950 0 0) 0x7fe0e8161850 con 0x7fe0e8161ce0 2023-08-27T11:27:13.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.533+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0d80035b0 con 0x7fe0e8161ce0 2023-08-27T11:27:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 728124127 0 0) 0x7fe0d80035b0 con 0x7fe0e8161ce0 2023-08-27T11:27:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 >> v2:172.21.15.138:3300/0 conn(0x7fe0e80b17e0 msgr2=0x7fe0e80cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 --2- 172.21.15.112:0/3628065576 >> v2:172.21.15.138:3300/0 conn(0x7fe0e80b17e0 0x7fe0e80cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 >> v1:172.21.15.138:6789/0 conn(0x7fe0e80b1400 legacy=0x7fe0e806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e81614e0 con 0x7fe0e8161ce0 2023-08-27T11:27:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fe0e00019f0 con 0x7fe0e8161ce0 2023-08-27T11:27:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 >> v1:172.21.15.112:6789/0 conn(0x7fe0e8161ce0 legacy=0x7fe0e806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 --2- 172.21.15.112:0/3628065576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d8003a00 0x7fe0d8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 --2- 172.21.15.112:0/3628065576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d8004850 0x7fe0d8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 --2- 172.21.15.112:0/3628065576 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d8005330 0x7fe0d800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e8161200 con 0x7fe0d8003a00 2023-08-27T11:27:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d800c200 con 0x7fe0d8004850 2023-08-27T11:27:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d800c3c0 con 0x7fe0d8005330 2023-08-27T11:27:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0eca59700 1 --2- 172.21.15.112:0/3628065576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d8003a00 0x7fe0d8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e7fff700 1 --2- 172.21.15.112:0/3628065576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d8004850 0x7fe0d8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0ed25a700 1 --2- 172.21.15.112:0/3628065576 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d8005330 0x7fe0d800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0eca59700 1 -- 172.21.15.112:0/3628065576 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d8005330 msgr2=0x7fe0d800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0eca59700 1 --2- 172.21.15.112:0/3628065576 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d8005330 0x7fe0d800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0eca59700 1 -- 172.21.15.112:0/3628065576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d8004850 msgr2=0x7fe0d8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0eca59700 1 --2- 172.21.15.112:0/3628065576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d8004850 0x7fe0d8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0eca59700 1 -- 172.21.15.112:0/3628065576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d80035b0 con 0x7fe0d8003a00 2023-08-27T11:27:13.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0ed25a700 1 --2- 172.21.15.112:0/3628065576 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d8005330 0x7fe0d800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:13.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0ed25a700 1 -- 172.21.15.112:0/3628065576 reap_dead start 2023-08-27T11:27:13.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0eca59700 1 --2- 172.21.15.112:0/3628065576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d8003a00 0x7fe0d8003e20 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc007fd0 tx=0x7fe0d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e355494a5b2bc8b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:13.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0052d0 con 0x7fe0d8003a00 2023-08-27T11:27:13.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc005430 con 0x7fe0d8003a00 2023-08-27T11:27:13.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.534+0000 7fe0e77fe700 1 -- 172.21.15.112:0/3628065576 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0038c0 con 0x7fe0d8003a00 2023-08-27T11:27:13.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.535+0000 7fe0eecbd700 1 -- 172.21.15.112:0/3628065576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d8003a00 msgr2=0x7fe0d8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.535+0000 7fe0eecbd700 1 --2- 172.21.15.112:0/3628065576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d8003a00 0x7fe0d8003e20 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc007fd0 tx=0x7fe0d8005880 comp rx=0 tx=0).stop 2023-08-27T11:27:13.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.535+0000 7fe0eecbd700 1 -- 172.21.15.112:0/3628065576 shutdown_connections 2023-08-27T11:27:13.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.535+0000 7fe0eecbd700 1 --2- 172.21.15.112:0/3628065576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d8003a00 0x7fe0d8003e20 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.535+0000 7fe0eecbd700 1 -- 172.21.15.112:0/3628065576 >> 172.21.15.112:0/3628065576 conn(0x7fe0e80d0000 msgr2=0x7fe0e80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.535+0000 7fe0eecbd700 1 -- 172.21.15.112:0/3628065576 shutdown_connections 2023-08-27T11:27:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.535+0000 7fe0eecbd700 1 -- 172.21.15.112:0/3628065576 wait complete. 2023-08-27T11:27:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.536+0000 7fe0eecbd700 1 Processor -- start 2023-08-27T11:27:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.536+0000 7fe0eecbd700 1 -- start start 2023-08-27T11:27:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eecbd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d8003a00 0x7fe0e80ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eecbd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0e80aeef0 0x7fe0e80ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eecbd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e81525d0 0x7fe0e80acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eecbd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e81488e0 con 0x7fe0d8003a00 2023-08-27T11:27:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eecbd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e80c6770 con 0x7fe0e81525d0 2023-08-27T11:27:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eecbd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e80af390 con 0x7fe0e80aeef0 2023-08-27T11:27:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eca59700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d8003a00 0x7fe0e80ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eca59700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d8003a00 0x7fe0e80ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34484/0 (socket says 172.21.15.112:34484) 2023-08-27T11:27:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eca59700 1 -- 172.21.15.112:0/448720047 learned_addr learned my addr 172.21.15.112:0/448720047 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eca59700 1 -- 172.21.15.112:0/448720047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0e80aeef0 msgr2=0x7fe0e80ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eca59700 1 --2- 172.21.15.112:0/448720047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0e80aeef0 0x7fe0e80ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0ed25a700 1 --2- 172.21.15.112:0/448720047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e81525d0 0x7fe0e80acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eca59700 1 -- 172.21.15.112:0/448720047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e81525d0 msgr2=0x7fe0e80acde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eca59700 1 --2- 172.21.15.112:0/448720047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e81525d0 0x7fe0e80acde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.537+0000 7fe0eca59700 1 -- 172.21.15.112:0/448720047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e80af670 con 0x7fe0d8003a00 2023-08-27T11:27:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.538+0000 7fe0eca59700 1 --2- 172.21.15.112:0/448720047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d8003a00 0x7fe0e80ae9b0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc015040 tx=0x7fe0dc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.538+0000 7fe0e57fa700 1 -- 172.21.15.112:0/448720047 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc001290 con 0x7fe0d8003a00 2023-08-27T11:27:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.538+0000 7fe0e57fa700 1 -- 172.21.15.112:0/448720047 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc006860 con 0x7fe0d8003a00 2023-08-27T11:27:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.538+0000 7fe0eecbd700 1 -- 172.21.15.112:0/448720047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e80af950 con 0x7fe0d8003a00 2023-08-27T11:27:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.538+0000 7fe0e57fa700 1 -- 172.21.15.112:0/448720047 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc006c10 con 0x7fe0d8003a00 2023-08-27T11:27:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.538+0000 7fe0eecbd700 1 -- 172.21.15.112:0/448720047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e80aff10 con 0x7fe0d8003a00 2023-08-27T11:27:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.539+0000 7fe0eecbd700 1 -- 172.21.15.112:0/448720047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e80b00a0 con 0x7fe0d8003a00 2023-08-27T11:27:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.539+0000 7fe0e57fa700 1 -- 172.21.15.112:0/448720047 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe0dc019040 con 0x7fe0d8003a00 2023-08-27T11:27:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.540+0000 7fe0e57fa700 1 --2- 172.21.15.112:0/448720047 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0cc06a4d0 0x7fe0cc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.540+0000 7fe0e57fa700 1 -- 172.21.15.112:0/448720047 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe0dc0897b0 con 0x7fe0d8003a00 2023-08-27T11:27:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.540+0000 7fe0e7fff700 1 --2- 172.21.15.112:0/448720047 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0cc06a4d0 0x7fe0cc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.540+0000 7fe0e7fff700 1 --2- 172.21.15.112:0/448720047 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0cc06a4d0 0x7fe0cc06c990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4006f10 tx=0x7fe0d4006e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.544+0000 7fe0e57fa700 1 -- 172.21.15.112:0/448720047 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe0dc054a70 con 0x7fe0d8003a00 2023-08-27T11:27:13.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.773+0000 7fe0eecbd700 1 -- 172.21.15.112:0/448720047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0e8067a20 con 0x7fe0d8003a00 2023-08-27T11:27:13.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.774+0000 7fe0e57fa700 1 -- 172.21.15.112:0/448720047 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fe0dc058090 con 0x7fe0d8003a00 2023-08-27T11:27:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:13.776 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/37643970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":37643970},{"type":"v1","addr":"172.21.15.112:6834","nonce":37643970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/568782348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":568782348},{"type":"v1","addr":"172.21.15.138:6836","nonce":568782348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1867471713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1867471713},{"type":"v1","addr":"172.21.15.138:6837","nonce":1867471713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:26:55.965268+0000","modified":"2023-08-27T11:26:59.589861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3180263819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3180263819},{"type":"v1","addr":"172.21.15.112:6835","nonce":3180263819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-27T11:27:13.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.776+0000 7fe0eecbd700 1 -- 172.21.15.112:0/448720047 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0cc06a4d0 msgr2=0x7fe0cc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.776+0000 7fe0eecbd700 1 --2- 172.21.15.112:0/448720047 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0cc06a4d0 0x7fe0cc06c990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4006f10 tx=0x7fe0d4006e80 comp rx=0 tx=0).stop 2023-08-27T11:27:13.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.776+0000 7fe0eecbd700 1 -- 172.21.15.112:0/448720047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d8003a00 msgr2=0x7fe0e80ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.776+0000 7fe0eecbd700 1 --2- 172.21.15.112:0/448720047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d8003a00 0x7fe0e80ae9b0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc015040 tx=0x7fe0dc001120 comp rx=0 tx=0).stop 2023-08-27T11:27:13.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.776+0000 7fe0eecbd700 1 -- 172.21.15.112:0/448720047 shutdown_connections 2023-08-27T11:27:13.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.776+0000 7fe0eecbd700 1 --2- 172.21.15.112:0/448720047 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0cc06a4d0 0x7fe0cc06c990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.776+0000 7fe0eecbd700 1 --2- 172.21.15.112:0/448720047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e81525d0 0x7fe0e80acde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.776+0000 7fe0eecbd700 1 --2- 172.21.15.112:0/448720047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d8003a00 0x7fe0e80ae9b0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.776+0000 7fe0eecbd700 1 --2- 172.21.15.112:0/448720047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0e80aeef0 0x7fe0e80ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.776+0000 7fe0eecbd700 1 -- 172.21.15.112:0/448720047 >> 172.21.15.112:0/448720047 conn(0x7fe0e80d0000 msgr2=0x7fe0e80cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:13.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.776+0000 7fe0eecbd700 1 -- 172.21.15.112:0/448720047 shutdown_connections 2023-08-27T11:27:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.776+0000 7fe0eecbd700 1 -- 172.21.15.112:0/448720047 wait complete. 2023-08-27T11:27:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 68 2023-08-27T11:27:13.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:13.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.908+0000 7feac2eaf700 1 Processor -- start 2023-08-27T11:27:13.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.908+0000 7feac2eaf700 1 -- start start 2023-08-27T11:27:13.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.909+0000 7feac2eaf700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feabc06e600 0x7feabc0a9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.909+0000 7feac2eaf700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feabc0cc200 con 0x7feabc143190 2023-08-27T11:27:13.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.909+0000 7feac2eaf700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feabc0cc4e0 con 0x7feabc06e150 2023-08-27T11:27:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.909+0000 7feac2eaf700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feabc0cc7c0 con 0x7feabc06e600 2023-08-27T11:27:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.909+0000 7feac0c4b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7feabc143190 0x7feabc0ceca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57106/0 (socket says 172.21.15.112:57106) 2023-08-27T11:27:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.909+0000 7feac0c4b700 1 -- 172.21.15.112:0/2015914255 learned_addr learned my addr 172.21.15.112:0/2015914255 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834707404 0 0) 0x7feabc0cc200 con 0x7feabc143190 2023-08-27T11:27:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feac144c700 1 --2- 172.21.15.112:0/2015914255 >> v2:172.21.15.138:3300/0 conn(0x7feabc06e600 0x7feabc0a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaac0035b0 con 0x7feabc143190 2023-08-27T11:27:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 675729414 0 0) 0x7feaac0035b0 con 0x7feabc143190 2023-08-27T11:27:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 >> v2:172.21.15.138:3300/0 conn(0x7feabc06e600 msgr2=0x7feabc0a9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 --2- 172.21.15.112:0/2015914255 >> v2:172.21.15.138:3300/0 conn(0x7feabc06e600 0x7feabc0a9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 >> v1:172.21.15.138:6789/0 conn(0x7feabc06e150 legacy=0x7feabc0a8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc0ccaa0 con 0x7feabc143190 2023-08-27T11:27:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7feab0001670 con 0x7feabc143190 2023-08-27T11:27:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 >> v1:172.21.15.112:6789/0 conn(0x7feabc143190 legacy=0x7feabc0ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 --2- 172.21.15.112:0/2015914255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feaac003a00 0x7feaac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 --2- 172.21.15.112:0/2015914255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaac004850 0x7feaac004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 --2- 172.21.15.112:0/2015914255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaac005210 0x7feaac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0cc7c0 con 0x7feaac005210 2023-08-27T11:27:13.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac00c0b0 con 0x7feaac004850 2023-08-27T11:27:13.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaac00c270 con 0x7feaac003a00 2023-08-27T11:27:13.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feac144c700 1 --2- 172.21.15.112:0/2015914255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaac005210 0x7feaac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feac0c4b700 1 --2- 172.21.15.112:0/2015914255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feaac003a00 0x7feaac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feabbfff700 1 --2- 172.21.15.112:0/2015914255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaac004850 0x7feaac004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feac144c700 1 -- 172.21.15.112:0/2015914255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feaac003a00 msgr2=0x7feaac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:13.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feac144c700 1 --2- 172.21.15.112:0/2015914255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feaac003a00 0x7feaac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feac144c700 1 -- 172.21.15.112:0/2015914255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaac004850 msgr2=0x7feaac004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:13.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feac144c700 1 --2- 172.21.15.112:0/2015914255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaac004850 0x7feaac004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feac144c700 1 -- 172.21.15.112:0/2015914255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaac0035b0 con 0x7feaac005210 2023-08-27T11:27:13.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feac144c700 1 -- 172.21.15.112:0/2015914255 reap_dead start 2023-08-27T11:27:13.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.910+0000 7feac144c700 1 --2- 172.21.15.112:0/2015914255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaac005210 0x7feaac00ba10 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7feab4009fc0 tx=0x7feaac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcb373dda0581cb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:13.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.911+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4003a20 con 0x7feaac005210 2023-08-27T11:27:13.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.911+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab400f040 con 0x7feaac005210 2023-08-27T11:27:13.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.911+0000 7feabb7fe700 1 -- 172.21.15.112:0/2015914255 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4003de0 con 0x7feaac005210 2023-08-27T11:27:13.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.911+0000 7feac2eaf700 1 -- 172.21.15.112:0/2015914255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaac005210 msgr2=0x7feaac00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.911+0000 7feac2eaf700 1 --2- 172.21.15.112:0/2015914255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaac005210 0x7feaac00ba10 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7feab4009fc0 tx=0x7feaac005880 comp rx=0 tx=0).stop 2023-08-27T11:27:13.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.911+0000 7feac2eaf700 1 -- 172.21.15.112:0/2015914255 shutdown_connections 2023-08-27T11:27:13.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.911+0000 7feac2eaf700 1 --2- 172.21.15.112:0/2015914255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaac005210 0x7feaac00ba10 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.911+0000 7feac2eaf700 1 -- 172.21.15.112:0/2015914255 >> 172.21.15.112:0/2015914255 conn(0x7feabc06cd30 msgr2=0x7feabc0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:13.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.911+0000 7feac2eaf700 1 -- 172.21.15.112:0/2015914255 shutdown_connections 2023-08-27T11:27:13.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.911+0000 7feac2eaf700 1 -- 172.21.15.112:0/2015914255 wait complete. 2023-08-27T11:27:13.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.912+0000 7feac2eaf700 1 Processor -- start 2023-08-27T11:27:13.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.912+0000 7feac2eaf700 1 -- start start 2023-08-27T11:27:13.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.913+0000 7feac2eaf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaac005210 0x7feabc13e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.913+0000 7feac2eaf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feabc0ab000 0x7feabc14c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.913+0000 7feac2eaf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc0ab480 0x7feabc153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.913+0000 7feac2eaf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc153200 con 0x7feabc0ab480 2023-08-27T11:27:13.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.913+0000 7feac2eaf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc14d090 con 0x7feaac005210 2023-08-27T11:27:13.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.913+0000 7feac2eaf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feabc153450 con 0x7feabc0ab000 2023-08-27T11:27:13.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.913+0000 7feac144c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc0ab480 0x7feabc153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.913+0000 7feac144c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc0ab480 0x7feabc153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34508/0 (socket says 172.21.15.112:34508) 2023-08-27T11:27:13.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.913+0000 7feac144c700 1 -- 172.21.15.112:0/3459128343 learned_addr learned my addr 172.21.15.112:0/3459128343 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:13.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.913+0000 7feac0c4b700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaac005210 0x7feabc13e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.914+0000 7feabbfff700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feabc0ab000 0x7feabc14c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.914+0000 7feac144c700 1 -- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feabc0ab000 msgr2=0x7feabc14c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.914+0000 7feac144c700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feabc0ab000 0x7feabc14c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.914+0000 7feac144c700 1 -- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaac005210 msgr2=0x7feabc13e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.914+0000 7feac144c700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaac005210 0x7feabc13e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.914+0000 7feac144c700 1 -- 172.21.15.112:0/3459128343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc153730 con 0x7feabc0ab480 2023-08-27T11:27:13.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.914+0000 7feac144c700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc0ab480 0x7feabc153f30 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7feabc0aa890 tx=0x7feab4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:13.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.914+0000 7feab97fa700 1 -- 172.21.15.112:0/3459128343 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4003e10 con 0x7feabc0ab480 2023-08-27T11:27:13.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.914+0000 7feab97fa700 1 -- 172.21.15.112:0/3459128343 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab4004230 con 0x7feabc0ab480 2023-08-27T11:27:13.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.914+0000 7feac2eaf700 1 -- 172.21.15.112:0/3459128343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feabc152e10 con 0x7feabc0ab480 2023-08-27T11:27:13.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.914+0000 7feab97fa700 1 -- 172.21.15.112:0/3459128343 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40045e0 con 0x7feabc0ab480 2023-08-27T11:27:13.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.914+0000 7feac2eaf700 1 -- 172.21.15.112:0/3459128343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feabc13ea20 con 0x7feabc0ab480 2023-08-27T11:27:13.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.915+0000 7feac2eaf700 1 -- 172.21.15.112:0/3459128343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feabc067a20 con 0x7feabc0ab480 2023-08-27T11:27:13.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.915+0000 7feab97fa700 1 -- 172.21.15.112:0/3459128343 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7feab4004740 con 0x7feabc0ab480 2023-08-27T11:27:13.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.916+0000 7feab97fa700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaa006a4d0 0x7feaa006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.916+0000 7feab97fa700 1 -- 172.21.15.112:0/3459128343 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feab4089ca0 con 0x7feabc0ab480 2023-08-27T11:27:13.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.916+0000 7feac0c4b700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaa006a4d0 0x7feaa006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.916+0000 7feac0c4b700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaa006a4d0 0x7feaa006c990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7feab0000c00 tx=0x7feab0000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:13.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:13.920+0000 7feab97fa700 1 -- 172.21.15.112:0/3459128343 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7feab4054cb0 con 0x7feabc0ab480 2023-08-27T11:27:14.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.109+0000 7feac2eaf700 1 -- 172.21.15.112:0/3459128343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feabc13ea20 con 0x7feabc0ab480 2023-08-27T11:27:14.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.111+0000 7feab97fa700 1 -- 172.21.15.112:0/3459128343 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+18327 (secure 0 0 0) 0x7feabc13ea20 con 0x7feabc0ab480 2023-08-27T11:27:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:14.113 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":102,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:57.319281+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:54.078457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:54.929983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:14.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.112+0000 7feac2eaf700 1 -- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaa006a4d0 msgr2=0x7feaa006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.112+0000 7feac2eaf700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaa006a4d0 0x7feaa006c990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7feab0000c00 tx=0x7feab0000ed0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.112+0000 7feac2eaf700 1 -- 172.21.15.112:0/3459128343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc0ab480 msgr2=0x7feabc153f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.112+0000 7feac2eaf700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc0ab480 0x7feabc153f30 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7feabc0aa890 tx=0x7feab4009750 comp rx=0 tx=0).stop 2023-08-27T11:27:14.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.113+0000 7feac2eaf700 1 -- 172.21.15.112:0/3459128343 shutdown_connections 2023-08-27T11:27:14.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.113+0000 7feac2eaf700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feaa006a4d0 0x7feaa006c990 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.113+0000 7feac2eaf700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feabc0ab480 0x7feabc153f30 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.113+0000 7feac2eaf700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaac005210 0x7feabc13e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.113+0000 7feac2eaf700 1 --2- 172.21.15.112:0/3459128343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feabc0ab000 0x7feabc14c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.113+0000 7feac2eaf700 1 -- 172.21.15.112:0/3459128343 >> 172.21.15.112:0/3459128343 conn(0x7feabc06cd30 msgr2=0x7feabc155260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:14.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.113+0000 7feac2eaf700 1 -- 172.21.15.112:0/3459128343 shutdown_connections 2023-08-27T11:27:14.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.113+0000 7feac2eaf700 1 -- 172.21.15.112:0/3459128343 wait complete. 2023-08-27T11:27:14.126 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:27:14.127 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:14.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.245+0000 7f6939c5f700 1 Processor -- start 2023-08-27T11:27:14.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.245+0000 7f6939c5f700 1 -- start start 2023-08-27T11:27:14.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.245+0000 7f6939c5f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6934160a00 0x7f69340b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.245+0000 7f6939c5f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f693406d510 0x7f693406d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.245+0000 7f6939c5f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f693406de50 con 0x7f6934146990 2023-08-27T11:27:14.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.245+0000 7f6939c5f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f693406f930 con 0x7f6934160a00 2023-08-27T11:27:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.245+0000 7f6939c5f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f693406fc10 con 0x7f693406d510 2023-08-27T11:27:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.246+0000 7f69337fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6934160a00 0x7f69340b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.246+0000 7f69337fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6934160a00 0x7f69340b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34512/0 (socket says 172.21.15.112:34512) 2023-08-27T11:27:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.246+0000 7f69337fe700 1 -- 172.21.15.112:0/3036194306 learned_addr learned my addr 172.21.15.112:0/3036194306 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.246+0000 7f6933fff700 1 --2- 172.21.15.112:0/3036194306 >> v2:172.21.15.138:3300/0 conn(0x7f693406d510 0x7f693406d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.246+0000 7f69337fe700 1 -- 172.21.15.112:0/3036194306 >> v2:172.21.15.138:3300/0 conn(0x7f693406d510 msgr2=0x7f693406d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:14.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.246+0000 7f69337fe700 1 --2- 172.21.15.112:0/3036194306 >> v2:172.21.15.138:3300/0 conn(0x7f693406d510 0x7f693406d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.246+0000 7f69337fe700 1 -- 172.21.15.112:0/3036194306 >> v1:172.21.15.138:6789/0 conn(0x7f6934146990 legacy=0x7f693415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:14.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.246+0000 7f69337fe700 1 -- 172.21.15.112:0/3036194306 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693406fef0 con 0x7f6934160a00 2023-08-27T11:27:14.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.246+0000 7f69337fe700 1 --2- 172.21.15.112:0/3036194306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6934160a00 0x7f69340b46d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f69340cfdf0 tx=0x7f6924009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3ad205d67b2efd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.246+0000 7f69327fc700 1 -- 172.21.15.112:0/3036194306 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924004230 con 0x7f6934160a00 2023-08-27T11:27:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.246+0000 7f69327fc700 1 -- 172.21.15.112:0/3036194306 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6924004390 con 0x7f6934160a00 2023-08-27T11:27:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.246+0000 7f69327fc700 1 -- 172.21.15.112:0/3036194306 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69240036a0 con 0x7f6934160a00 2023-08-27T11:27:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.247+0000 7f6939c5f700 1 -- 172.21.15.112:0/3036194306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6934160a00 msgr2=0x7f69340b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.247+0000 7f6939c5f700 1 --2- 172.21.15.112:0/3036194306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6934160a00 0x7f69340b46d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f69340cfdf0 tx=0x7f6924009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.247+0000 7f6939c5f700 1 -- 172.21.15.112:0/3036194306 shutdown_connections 2023-08-27T11:27:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.247+0000 7f6939c5f700 1 --2- 172.21.15.112:0/3036194306 >> v2:172.21.15.138:3300/0 conn(0x7f693406d510 0x7f693406d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.247+0000 7f6939c5f700 1 --2- 172.21.15.112:0/3036194306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6934160a00 0x7f69340b46d0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.247+0000 7f6939c5f700 1 -- 172.21.15.112:0/3036194306 >> 172.21.15.112:0/3036194306 conn(0x7f69340cfe50 msgr2=0x7f69340d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.247+0000 7f6939c5f700 1 -- 172.21.15.112:0/3036194306 shutdown_connections 2023-08-27T11:27:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.247+0000 7f6939c5f700 1 -- 172.21.15.112:0/3036194306 wait complete. 2023-08-27T11:27:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.248+0000 7f6939c5f700 1 Processor -- start 2023-08-27T11:27:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.248+0000 7f6939c5f700 1 -- start start 2023-08-27T11:27:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6939c5f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f693406d510 0x7f693413e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6939c5f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6934146990 0x7f6934140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6939c5f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6934160a00 0x7f6934140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6939c5f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69341412c0 con 0x7f6934146990 2023-08-27T11:27:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6939c5f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693413d8c0 con 0x7f6934160a00 2023-08-27T11:27:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6939c5f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693413dba0 con 0x7f693406d510 2023-08-27T11:27:14.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6932ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6934146990 0x7f6934140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6932ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6934146990 0x7f6934140840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34520/0 (socket says 172.21.15.112:34520) 2023-08-27T11:27:14.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6932ffd700 1 -- 172.21.15.112:0/1738246326 learned_addr learned my addr 172.21.15.112:0/1738246326 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:14.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6933fff700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6934160a00 0x7f6934140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f69337fe700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f693406d510 0x7f693413e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6932ffd700 1 -- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f693406d510 msgr2=0x7f693413e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6932ffd700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f693406d510 0x7f693413e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6932ffd700 1 -- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6934160a00 msgr2=0x7f6934140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6932ffd700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6934160a00 0x7f6934140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6932ffd700 1 -- 172.21.15.112:0/1738246326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6924009550 con 0x7f6934146990 2023-08-27T11:27:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6933fff700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6934160a00 0x7f6934140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:14.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f6932ffd700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6934146990 0x7f6934140840 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f691c00c4d0 tx=0x7f691c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:14.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.249+0000 7f692bfff700 1 -- 172.21.15.112:0/1738246326 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c011790 con 0x7f6934146990 2023-08-27T11:27:14.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.250+0000 7f692bfff700 1 -- 172.21.15.112:0/1738246326 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c0118f0 con 0x7f6934146990 2023-08-27T11:27:14.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.250+0000 7f6939c5f700 1 -- 172.21.15.112:0/1738246326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693413de80 con 0x7f6934146990 2023-08-27T11:27:14.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.250+0000 7f692bfff700 1 -- 172.21.15.112:0/1738246326 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c011ca0 con 0x7f6934146990 2023-08-27T11:27:14.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.250+0000 7f6939c5f700 1 -- 172.21.15.112:0/1738246326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693413e440 con 0x7f6934146990 2023-08-27T11:27:14.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.251+0000 7f6939c5f700 1 -- 172.21.15.112:0/1738246326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f693413e660 con 0x7f6934146990 2023-08-27T11:27:14.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.251+0000 7f692bfff700 1 -- 172.21.15.112:0/1738246326 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f691c01c460 con 0x7f6934146990 2023-08-27T11:27:14.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.251+0000 7f692bfff700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f691406a4d0 0x7f691406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.251+0000 7f692bfff700 1 -- 172.21.15.112:0/1738246326 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f691c088a80 con 0x7f6934146990 2023-08-27T11:27:14.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.252+0000 7f69337fe700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f691406a4d0 0x7f691406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.252+0000 7f69337fe700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f691406a4d0 0x7f691406c990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6924009130 tx=0x7f6924008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:14.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.256+0000 7f692bfff700 1 -- 172.21.15.112:0/1738246326 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f691c053c20 con 0x7f6934146990 2023-08-27T11:27:14.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.484+0000 7f6939c5f700 1 -- 172.21.15.112:0/1738246326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6934067a20 con 0x7f6934146990 2023-08-27T11:27:14.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.484+0000 7f692bfff700 1 -- 172.21.15.112:0/1738246326 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f691c057240 con 0x7f6934146990 2023-08-27T11:27:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:14.486 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/37643970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":37643970},{"type":"v1","addr":"172.21.15.112:6834","nonce":37643970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/568782348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":568782348},{"type":"v1","addr":"172.21.15.138:6836","nonce":568782348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1867471713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1867471713},{"type":"v1","addr":"172.21.15.138:6837","nonce":1867471713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:26:55.965268+0000","modified":"2023-08-27T11:26:59.589861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3180263819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3180263819},{"type":"v1","addr":"172.21.15.112:6835","nonce":3180263819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-27T11:27:14.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.486+0000 7f6939c5f700 1 -- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f691406a4d0 msgr2=0x7f691406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.486+0000 7f6939c5f700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f691406a4d0 0x7f691406c990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6924009130 tx=0x7f6924008030 comp rx=0 tx=0).stop 2023-08-27T11:27:14.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.486+0000 7f6939c5f700 1 -- 172.21.15.112:0/1738246326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6934146990 msgr2=0x7f6934140840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.486+0000 7f6939c5f700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6934146990 0x7f6934140840 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f691c00c4d0 tx=0x7f691c00c890 comp rx=0 tx=0).stop 2023-08-27T11:27:14.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.486+0000 7f6939c5f700 1 -- 172.21.15.112:0/1738246326 shutdown_connections 2023-08-27T11:27:14.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.486+0000 7f6939c5f700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f691406a4d0 0x7f691406c990 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.486+0000 7f6939c5f700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6934160a00 0x7f6934140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.486+0000 7f6939c5f700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f693406d510 0x7f693413e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.486+0000 7f6939c5f700 1 --2- 172.21.15.112:0/1738246326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6934146990 0x7f6934140840 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.486+0000 7f6939c5f700 1 -- 172.21.15.112:0/1738246326 >> 172.21.15.112:0/1738246326 conn(0x7f69340cfe50 msgr2=0x7f693415e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.486+0000 7f6939c5f700 1 -- 172.21.15.112:0/1738246326 shutdown_connections 2023-08-27T11:27:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.486+0000 7f6939c5f700 1 -- 172.21.15.112:0/1738246326 wait complete. 2023-08-27T11:27:14.491 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 68 2023-08-27T11:27:14.500 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:14.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.619+0000 7f5bdc0db700 1 Processor -- start 2023-08-27T11:27:14.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.619+0000 7f5bdc0db700 1 -- start start 2023-08-27T11:27:14.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bdc0db700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5bd406dcc0 0x7f5bd40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bdc0db700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5bd40b3440 0x7f5bd40b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bdc0db700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd415b540 con 0x7f5bd415a120 2023-08-27T11:27:14.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bdc0db700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd40af830 con 0x7f5bd406dcc0 2023-08-27T11:27:14.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bdc0db700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd40afb10 con 0x7f5bd40b3440 2023-08-27T11:27:14.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bd9676700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5bd406dcc0 0x7f5bd40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bd9e77700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5bd415a120 0x7f5bd406d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57140/0 (socket says 172.21.15.112:57140) 2023-08-27T11:27:14.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bd9e77700 1 -- 172.21.15.112:0/243256359 learned_addr learned my addr 172.21.15.112:0/243256359 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:14.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bda678700 1 --2- 172.21.15.112:0/243256359 >> v2:172.21.15.138:3300/0 conn(0x7f5bd40b3440 0x7f5bd40b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bd9676700 1 -- 172.21.15.112:0/243256359 >> v2:172.21.15.138:3300/0 conn(0x7f5bd40b3440 msgr2=0x7f5bd40b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:14.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bd9676700 1 --2- 172.21.15.112:0/243256359 >> v2:172.21.15.138:3300/0 conn(0x7f5bd40b3440 0x7f5bd40b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bd9676700 1 -- 172.21.15.112:0/243256359 >> v1:172.21.15.112:6789/0 conn(0x7f5bd415a120 legacy=0x7f5bd406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bd8e75700 1 -- 172.21.15.112:0/243256359 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278976480 0 0) 0x7f5bd415b540 con 0x7f5bd415a120 2023-08-27T11:27:14.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.620+0000 7f5bd9676700 1 -- 172.21.15.112:0/243256359 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd40afdf0 con 0x7f5bd406dcc0 2023-08-27T11:27:14.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.621+0000 7f5bd9676700 1 --2- 172.21.15.112:0/243256359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bd406dcc0 0x7f5bd40b05f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40b5cd0 tx=0x7f5bd0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb8d0073de4b6f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:14.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.621+0000 7f5bd8e75700 1 -- 172.21.15.112:0/243256359 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0011830 con 0x7f5bd406dcc0 2023-08-27T11:27:14.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.621+0000 7f5bd8e75700 1 -- 172.21.15.112:0/243256359 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd0011990 con 0x7f5bd406dcc0 2023-08-27T11:27:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.621+0000 7f5bd8e75700 1 -- 172.21.15.112:0/243256359 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0011d20 con 0x7f5bd406dcc0 2023-08-27T11:27:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.621+0000 7f5bdc0db700 1 -- 172.21.15.112:0/243256359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bd406dcc0 msgr2=0x7f5bd40b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.621+0000 7f5bdc0db700 1 --2- 172.21.15.112:0/243256359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bd406dcc0 0x7f5bd40b05f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40b5cd0 tx=0x7f5bd0009b20 comp rx=0 tx=0).stop 2023-08-27T11:27:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.621+0000 7f5bdc0db700 1 -- 172.21.15.112:0/243256359 shutdown_connections 2023-08-27T11:27:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.621+0000 7f5bdc0db700 1 --2- 172.21.15.112:0/243256359 >> v2:172.21.15.138:3300/0 conn(0x7f5bd40b3440 0x7f5bd40b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.621+0000 7f5bdc0db700 1 --2- 172.21.15.112:0/243256359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bd406dcc0 0x7f5bd40b05f0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.621+0000 7f5bdc0db700 1 -- 172.21.15.112:0/243256359 >> 172.21.15.112:0/243256359 conn(0x7f5bd40cff80 msgr2=0x7f5bd40d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.621+0000 7f5bdc0db700 1 -- 172.21.15.112:0/243256359 shutdown_connections 2023-08-27T11:27:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.621+0000 7f5bdc0db700 1 -- 172.21.15.112:0/243256359 wait complete. 2023-08-27T11:27:14.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.623+0000 7f5bdc0db700 1 Processor -- start 2023-08-27T11:27:14.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.623+0000 7f5bdc0db700 1 -- start start 2023-08-27T11:27:14.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.623+0000 7f5bdc0db700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bd406dcc0 0x7f5bd414ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.623+0000 7f5bdc0db700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bd40b3440 0x7f5bd41f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.623+0000 7f5bdc0db700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bd415a120 0x7f5bd406f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.623+0000 7f5bdc0db700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd414ef90 con 0x7f5bd40b3440 2023-08-27T11:27:14.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.623+0000 7f5bdc0db700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40c88d0 con 0x7f5bd415a120 2023-08-27T11:27:14.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.623+0000 7f5bdc0db700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40c8b70 con 0x7f5bd406dcc0 2023-08-27T11:27:14.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.623+0000 7f5bd9676700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bd40b3440 0x7f5bd41f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.623+0000 7f5bd9676700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bd40b3440 0x7f5bd41f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34534/0 (socket says 172.21.15.112:34534) 2023-08-27T11:27:14.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.623+0000 7f5bd9676700 1 -- 172.21.15.112:0/3517582964 learned_addr learned my addr 172.21.15.112:0/3517582964 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:14.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bda678700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bd415a120 0x7f5bd406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bd9e77700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bd406dcc0 0x7f5bd414ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bd9676700 1 -- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bd406dcc0 msgr2=0x7f5bd414ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bd9676700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bd406dcc0 0x7f5bd414ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bd9676700 1 -- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bd415a120 msgr2=0x7f5bd406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bd9676700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bd415a120 0x7f5bd406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bd9676700 1 -- 172.21.15.112:0/3517582964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd00094a0 con 0x7f5bd40b3440 2023-08-27T11:27:14.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bd9e77700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bd406dcc0 0x7f5bd414ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:14.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bd9676700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bd40b3440 0x7f5bd41f1650 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5bd00090b0 tx=0x7f5bd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:14.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bca7fc700 1 -- 172.21.15.112:0/3517582964 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00038a0 con 0x7f5bd40b3440 2023-08-27T11:27:14.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bca7fc700 1 -- 172.21.15.112:0/3517582964 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd001f070 con 0x7f5bd40b3440 2023-08-27T11:27:14.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bdc0db700 1 -- 172.21.15.112:0/3517582964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd40c8e50 con 0x7f5bd40b3440 2023-08-27T11:27:14.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bca7fc700 1 -- 172.21.15.112:0/3517582964 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0003c20 con 0x7f5bd40b3440 2023-08-27T11:27:14.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.624+0000 7f5bdc0db700 1 -- 172.21.15.112:0/3517582964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd40c9410 con 0x7f5bd40b3440 2023-08-27T11:27:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.625+0000 7f5bdc0db700 1 -- 172.21.15.112:0/3517582964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd4067a20 con 0x7f5bd40b3440 2023-08-27T11:27:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.625+0000 7f5bca7fc700 1 -- 172.21.15.112:0/3517582964 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5bd001a430 con 0x7f5bd40b3440 2023-08-27T11:27:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.626+0000 7f5bca7fc700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5bb806a4d0 0x7f5bb806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.626+0000 7f5bca7fc700 1 -- 172.21.15.112:0/3517582964 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5bd0021070 con 0x7f5bd40b3440 2023-08-27T11:27:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.626+0000 7f5bd9e77700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5bb806a4d0 0x7f5bb806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.627+0000 7f5bd9e77700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5bb806a4d0 0x7f5bb806c990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4000f60 tx=0x7f5bc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.630+0000 7f5bca7fc700 1 -- 172.21.15.112:0/3517582964 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5bd0055750 con 0x7f5bd40b3440 2023-08-27T11:27:14.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.862+0000 7f5bdc0db700 1 -- 172.21.15.112:0/3517582964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bd40b0590 con 0x7f5bd40b3440 2023-08-27T11:27:14.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.862+0000 7f5bca7fc700 1 -- 172.21.15.112:0/3517582964 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f5bd0058d70 con 0x7f5bd40b3440 2023-08-27T11:27:14.864 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:14.865 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/37643970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":37643970},{"type":"v1","addr":"172.21.15.112:6834","nonce":37643970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/568782348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":568782348},{"type":"v1","addr":"172.21.15.138:6836","nonce":568782348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1867471713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1867471713},{"type":"v1","addr":"172.21.15.138:6837","nonce":1867471713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:26:55.965268+0000","modified":"2023-08-27T11:26:59.589861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3180263819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3180263819},{"type":"v1","addr":"172.21.15.112:6835","nonce":3180263819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-27T11:27:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.864+0000 7f5bdc0db700 1 -- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5bb806a4d0 msgr2=0x7f5bb806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.864+0000 7f5bdc0db700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5bb806a4d0 0x7f5bb806c990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4000f60 tx=0x7f5bc4009040 comp rx=0 tx=0).stop 2023-08-27T11:27:14.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.864+0000 7f5bdc0db700 1 -- 172.21.15.112:0/3517582964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bd40b3440 msgr2=0x7f5bd41f1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.864+0000 7f5bdc0db700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bd40b3440 0x7f5bd41f1650 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5bd00090b0 tx=0x7f5bd0004750 comp rx=0 tx=0).stop 2023-08-27T11:27:14.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.864+0000 7f5bdc0db700 1 -- 172.21.15.112:0/3517582964 shutdown_connections 2023-08-27T11:27:14.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.864+0000 7f5bdc0db700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5bb806a4d0 0x7f5bb806c990 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.864+0000 7f5bdc0db700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bd415a120 0x7f5bd406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.864+0000 7f5bdc0db700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bd406dcc0 0x7f5bd414ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.864+0000 7f5bdc0db700 1 --2- 172.21.15.112:0/3517582964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bd40b3440 0x7f5bd41f1650 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.864+0000 7f5bdc0db700 1 -- 172.21.15.112:0/3517582964 >> 172.21.15.112:0/3517582964 conn(0x7f5bd40cff80 msgr2=0x7f5bd4073380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:14.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.864+0000 7f5bdc0db700 1 -- 172.21.15.112:0/3517582964 shutdown_connections 2023-08-27T11:27:14.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.864+0000 7f5bdc0db700 1 -- 172.21.15.112:0/3517582964 wait complete. 2023-08-27T11:27:14.870 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 68 2023-08-27T11:27:14.878 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:14.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.997+0000 7fb3eb668700 1 Processor -- start 2023-08-27T11:27:14.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.997+0000 7fb3eb668700 1 -- start start 2023-08-27T11:27:15.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3eb668700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e40ad210 0x7fb3e415e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3eb668700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb3e406e150 0x7fb3e406e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3eb668700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e406ead0 con 0x7fb3e4161050 2023-08-27T11:27:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3eb668700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e4143bf0 con 0x7fb3e406e150 2023-08-27T11:27:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3eb668700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e40add70 con 0x7fb3e40ad210 2023-08-27T11:27:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3e8c03700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb3e406e150 0x7fb3e406e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3e9404700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e40ad210 0x7fb3e415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3e9c05700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb3e4161050 0x7fb3e41614e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:36446/0 (socket says 172.21.15.112:36446) 2023-08-27T11:27:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3e8c03700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb3e406e150 0x7fb3e406e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:34966/0 (socket says 172.21.15.112:34966) 2023-08-27T11:27:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3e9c05700 1 -- 172.21.15.112:0/218720784 learned_addr learned my addr 172.21.15.112:0/218720784 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3e8c03700 1 -- 172.21.15.112:0/218720784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e40ad210 msgr2=0x7fb3e415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3e8c03700 1 --2- 172.21.15.112:0/218720784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e40ad210 0x7fb3e415e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3e8c03700 1 -- 172.21.15.112:0/218720784 >> v1:172.21.15.138:6789/0 conn(0x7fb3e4161050 legacy=0x7fb3e41614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3e8c03700 1 -- 172.21.15.112:0/218720784 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e4142fe0 con 0x7fb3e406e150 2023-08-27T11:27:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.998+0000 7fb3e9404700 1 --2- 172.21.15.112:0/218720784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e40ad210 0x7fb3e415e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.999+0000 7fb3e8c03700 1 --2- 172.21.15.112:0/218720784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e406e150 0x7fb3e406e590 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009820 tx=0x7fb3d4009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbca8da4ef0b2e83 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.999+0000 7fb3dbfff700 1 -- 172.21.15.112:0/218720784 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4005e30 con 0x7fb3e406e150 2023-08-27T11:27:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.999+0000 7fb3dbfff700 1 -- 172.21.15.112:0/218720784 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d4006030 con 0x7fb3e406e150 2023-08-27T11:27:15.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.999+0000 7fb3dbfff700 1 -- 172.21.15.112:0/218720784 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40063c0 con 0x7fb3e406e150 2023-08-27T11:27:15.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.999+0000 7fb3eb668700 1 -- 172.21.15.112:0/218720784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e406e150 msgr2=0x7fb3e406e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.999+0000 7fb3eb668700 1 --2- 172.21.15.112:0/218720784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e406e150 0x7fb3e406e590 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009820 tx=0x7fb3d4009b30 comp rx=0 tx=0).stop 2023-08-27T11:27:15.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.999+0000 7fb3eb668700 1 -- 172.21.15.112:0/218720784 shutdown_connections 2023-08-27T11:27:15.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.999+0000 7fb3eb668700 1 --2- 172.21.15.112:0/218720784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e40ad210 0x7fb3e415e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:14.999+0000 7fb3eb668700 1 --2- 172.21.15.112:0/218720784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e406e150 0x7fb3e406e590 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.000+0000 7fb3eb668700 1 -- 172.21.15.112:0/218720784 >> 172.21.15.112:0/218720784 conn(0x7fb3e406cd30 msgr2=0x7fb3e40717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.000+0000 7fb3eb668700 1 -- 172.21.15.112:0/218720784 shutdown_connections 2023-08-27T11:27:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.000+0000 7fb3eb668700 1 -- 172.21.15.112:0/218720784 wait complete. 2023-08-27T11:27:15.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.001+0000 7fb3eb668700 1 Processor -- start 2023-08-27T11:27:15.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.001+0000 7fb3eb668700 1 -- start start 2023-08-27T11:27:15.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.001+0000 7fb3eb668700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e406e150 0x7fb3e40ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.001+0000 7fb3eb668700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e40ad210 0x7fb3e41f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.001+0000 7fb3eb668700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e4161050 0x7fb3e40ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.001+0000 7fb3eb668700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e40cd2b0 con 0x7fb3e406e150 2023-08-27T11:27:15.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.001+0000 7fb3eb668700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e413ea40 con 0x7fb3e4161050 2023-08-27T11:27:15.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.001+0000 7fb3eb668700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e41f3b50 con 0x7fb3e40ad210 2023-08-27T11:27:15.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3e9404700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e406e150 0x7fb3e40ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3e9404700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e406e150 0x7fb3e40ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34544/0 (socket says 172.21.15.112:34544) 2023-08-27T11:27:15.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3e9404700 1 -- 172.21.15.112:0/3845647817 learned_addr learned my addr 172.21.15.112:0/3845647817 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:15.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3e8c03700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e40ad210 0x7fb3e41f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3e9c05700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e4161050 0x7fb3e40ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3e9404700 1 -- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e40ad210 msgr2=0x7fb3e41f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3e9404700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e40ad210 0x7fb3e41f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3e9404700 1 -- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e4161050 msgr2=0x7fb3e40ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3e9404700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e4161050 0x7fb3e40ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3e9404700 1 -- 172.21.15.112:0/3845647817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00094a0 con 0x7fb3e406e150 2023-08-27T11:27:15.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3e9c05700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e4161050 0x7fb3e40ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:15.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3e9404700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e406e150 0x7fb3e40ccd70 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000ec40 tx=0x7fb3e0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3d9ffb700 1 -- 172.21.15.112:0/3845647817 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e000c970 con 0x7fb3e406e150 2023-08-27T11:27:15.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3d9ffb700 1 -- 172.21.15.112:0/3845647817 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e000cad0 con 0x7fb3e406e150 2023-08-27T11:27:15.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3d9ffb700 1 -- 172.21.15.112:0/3845647817 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e000ce40 con 0x7fb3e406e150 2023-08-27T11:27:15.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3eb668700 1 -- 172.21.15.112:0/3845647817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d4009430 con 0x7fb3e406e150 2023-08-27T11:27:15.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.002+0000 7fb3eb668700 1 -- 172.21.15.112:0/3845647817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e41e69e0 con 0x7fb3e406e150 2023-08-27T11:27:15.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.003+0000 7fb3eb668700 1 -- 172.21.15.112:0/3845647817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e4067a20 con 0x7fb3e406e150 2023-08-27T11:27:15.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.004+0000 7fb3d9ffb700 1 -- 172.21.15.112:0/3845647817 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fb3e0013470 con 0x7fb3e406e150 2023-08-27T11:27:15.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.004+0000 7fb3d9ffb700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb3c806a4d0 0x7fb3c806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.004+0000 7fb3d9ffb700 1 -- 172.21.15.112:0/3845647817 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb3e0088db0 con 0x7fb3e406e150 2023-08-27T11:27:15.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.004+0000 7fb3e8c03700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb3c806a4d0 0x7fb3c806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.005+0000 7fb3e8c03700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb3c806a4d0 0x7fb3c806c990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4000c00 tx=0x7fb3d4008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.008+0000 7fb3d9ffb700 1 -- 172.21.15.112:0/3845647817 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb3e008d070 con 0x7fb3e406e150 2023-08-27T11:27:15.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.198+0000 7fb3eb668700 1 -- 172.21.15.112:0/3845647817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3e4156280 con 0x7fb3e406e150 2023-08-27T11:27:15.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.199+0000 7fb3d9ffb700 1 -- 172.21.15.112:0/3845647817 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+18327 (secure 0 0 0) 0x7fb3e00579e0 con 0x7fb3e406e150 2023-08-27T11:27:15.201 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:15.201 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":102,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:26:57.319281+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:26:54.078457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-27T11:26:54.929983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:15.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.201+0000 7fb3eb668700 1 -- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb3c806a4d0 msgr2=0x7fb3c806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.201+0000 7fb3eb668700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb3c806a4d0 0x7fb3c806c990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4000c00 tx=0x7fb3d4008070 comp rx=0 tx=0).stop 2023-08-27T11:27:15.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.201+0000 7fb3eb668700 1 -- 172.21.15.112:0/3845647817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e406e150 msgr2=0x7fb3e40ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.201+0000 7fb3eb668700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e406e150 0x7fb3e40ccd70 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000ec40 tx=0x7fb3e0009ec0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.201+0000 7fb3eb668700 1 -- 172.21.15.112:0/3845647817 shutdown_connections 2023-08-27T11:27:15.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.201+0000 7fb3eb668700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb3c806a4d0 0x7fb3c806c990 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.201+0000 7fb3eb668700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e4161050 0x7fb3e40ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.201+0000 7fb3eb668700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e406e150 0x7fb3e40ccd70 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.201+0000 7fb3eb668700 1 --2- 172.21.15.112:0/3845647817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e40ad210 0x7fb3e41f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.201+0000 7fb3eb668700 1 -- 172.21.15.112:0/3845647817 >> 172.21.15.112:0/3845647817 conn(0x7fb3e406cd30 msgr2=0x7fb3e41390c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:15.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.201+0000 7fb3eb668700 1 -- 172.21.15.112:0/3845647817 shutdown_connections 2023-08-27T11:27:15.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.201+0000 7fb3eb668700 1 -- 172.21.15.112:0/3845647817 wait complete. 2023-08-27T11:27:15.215 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:27:15.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.334+0000 7f7910e5f700 1 Processor -- start 2023-08-27T11:27:15.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.334+0000 7f7910e5f700 1 -- start start 2023-08-27T11:27:15.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.334+0000 7f7910e5f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f790c0b1400 0x7f790c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f7910e5f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f790c0b17e0 0x7f790c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f7910e5f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c161850 con 0x7f790c161ce0 2023-08-27T11:27:15.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f7910e5f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790c06e7d0 con 0x7f790c0b1400 2023-08-27T11:27:15.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f7910e5f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790c0cfc10 con 0x7f790c0b17e0 2023-08-27T11:27:15.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f790a59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f790c0b1400 0x7f790c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f790a59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f790c0b1400 0x7f790c0cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34552/0 (socket says 172.21.15.112:34552) 2023-08-27T11:27:15.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f790a59c700 1 -- 172.21.15.112:0/1594733996 learned_addr learned my addr 172.21.15.112:0/1594733996 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:15.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f790ad9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f790c161ce0 0x7f790c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57170/0 (socket says 172.21.15.112:57170) 2023-08-27T11:27:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f7909d9b700 1 --2- 172.21.15.112:0/1594733996 >> v2:172.21.15.138:3300/0 conn(0x7f790c0b17e0 0x7f790c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f790a59c700 1 -- 172.21.15.112:0/1594733996 >> v2:172.21.15.138:3300/0 conn(0x7f790c0b17e0 msgr2=0x7f790c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f790a59c700 1 --2- 172.21.15.112:0/1594733996 >> v2:172.21.15.138:3300/0 conn(0x7f790c0b17e0 0x7f790c06e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f790a59c700 1 -- 172.21.15.112:0/1594733996 >> v1:172.21.15.112:6789/0 conn(0x7f790c161ce0 legacy=0x7f790c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f790959a700 1 -- 172.21.15.112:0/1594733996 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346725903 0 0) 0x7f790c161850 con 0x7f790c161ce0 2023-08-27T11:27:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f790a59c700 1 -- 172.21.15.112:0/1594733996 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c161200 con 0x7f790c0b1400 2023-08-27T11:27:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f7909d9b700 1 --2- 172.21.15.112:0/1594733996 >> v2:172.21.15.138:3300/0 conn(0x7f790c0b17e0 0x7f790c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f790a59c700 1 --2- 172.21.15.112:0/1594733996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c0b1400 0x7f790c0cf5a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f790c0cc420 tx=0x7f78fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ea5318ec433a3c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f790959a700 1 -- 172.21.15.112:0/1594733996 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00b870 con 0x7f790c0b1400 2023-08-27T11:27:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f790959a700 1 -- 172.21.15.112:0/1594733996 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc00b9d0 con 0x7f790c0b1400 2023-08-27T11:27:15.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.335+0000 7f790959a700 1 -- 172.21.15.112:0/1594733996 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00bd60 con 0x7f790c0b1400 2023-08-27T11:27:15.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.336+0000 7f7910e5f700 1 -- 172.21.15.112:0/1594733996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c0b1400 msgr2=0x7f790c0cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.336+0000 7f7910e5f700 1 --2- 172.21.15.112:0/1594733996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c0b1400 0x7f790c0cf5a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f790c0cc420 tx=0x7f78fc009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.336+0000 7f7910e5f700 1 -- 172.21.15.112:0/1594733996 shutdown_connections 2023-08-27T11:27:15.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.336+0000 7f7910e5f700 1 --2- 172.21.15.112:0/1594733996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c0b1400 0x7f790c0cf5a0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.336+0000 7f7910e5f700 1 --2- 172.21.15.112:0/1594733996 >> v2:172.21.15.138:3300/0 conn(0x7f790c0b17e0 0x7f790c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.336+0000 7f7910e5f700 1 -- 172.21.15.112:0/1594733996 >> 172.21.15.112:0/1594733996 conn(0x7f790c0d0000 msgr2=0x7f790c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:15.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.336+0000 7f7910e5f700 1 -- 172.21.15.112:0/1594733996 shutdown_connections 2023-08-27T11:27:15.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.336+0000 7f7910e5f700 1 -- 172.21.15.112:0/1594733996 wait complete. 2023-08-27T11:27:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.337+0000 7f7910e5f700 1 Processor -- start 2023-08-27T11:27:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.337+0000 7f7910e5f700 1 -- start start 2023-08-27T11:27:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f7910e5f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f790c0b1400 0x7f790c144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f7910e5f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f790c0b17e0 0x7f790c1f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f7910e5f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c161ce0 0x7f790c0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f7910e5f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c1f7330 con 0x7f790c161ce0 2023-08-27T11:27:15.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f7910e5f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c144f60 con 0x7f790c0b1400 2023-08-27T11:27:15.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f7910e5f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790c1f6720 con 0x7f790c0b17e0 2023-08-27T11:27:15.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f790ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c161ce0 0x7f790c0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f790ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c161ce0 0x7f790c0cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34562/0 (socket says 172.21.15.112:34562) 2023-08-27T11:27:15.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f790ad9d700 1 -- 172.21.15.112:0/3427677822 learned_addr learned my addr 172.21.15.112:0/3427677822 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:15.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f790a59c700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f790c0b1400 0x7f790c144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f7909d9b700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f790c0b17e0 0x7f790c1f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f790ad9d700 1 -- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f790c0b17e0 msgr2=0x7f790c1f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f790ad9d700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f790c0b17e0 0x7f790c1f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f790ad9d700 1 -- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f790c0b1400 msgr2=0x7f790c144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f790ad9d700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f790c0b1400 0x7f790c144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f790ad9d700 1 -- 172.21.15.112:0/3427677822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc009550 con 0x7f790c161ce0 2023-08-27T11:27:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.338+0000 7f790a59c700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f790c0b1400 0x7f790c144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.339+0000 7f790ad9d700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c161ce0 0x7f790c0cb690 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f790400cb60 tx=0x7f790400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.339+0000 7f78faffd700 1 -- 172.21.15.112:0/3427677822 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790400ab50 con 0x7f790c161ce0 2023-08-27T11:27:15.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.339+0000 7f78faffd700 1 -- 172.21.15.112:0/3427677822 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790400acb0 con 0x7f790c161ce0 2023-08-27T11:27:15.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.339+0000 7f7910e5f700 1 -- 172.21.15.112:0/3427677822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790c1f6a00 con 0x7f790c161ce0 2023-08-27T11:27:15.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.339+0000 7f78faffd700 1 -- 172.21.15.112:0/3427677822 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904007310 con 0x7f790c161ce0 2023-08-27T11:27:15.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.339+0000 7f7910e5f700 1 -- 172.21.15.112:0/3427677822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790c1f6fc0 con 0x7f790c161ce0 2023-08-27T11:27:15.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.340+0000 7f7910e5f700 1 -- 172.21.15.112:0/3427677822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790c0c38e0 con 0x7f790c161ce0 2023-08-27T11:27:15.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.340+0000 7f78faffd700 1 -- 172.21.15.112:0/3427677822 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f7904007470 con 0x7f790c161ce0 2023-08-27T11:27:15.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.341+0000 7f78faffd700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f78ec06a4d0 0x7f78ec06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.341+0000 7f78faffd700 1 -- 172.21.15.112:0/3427677822 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7904088b00 con 0x7f790c161ce0 2023-08-27T11:27:15.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.341+0000 7f790a59c700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f78ec06a4d0 0x7f78ec06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.341+0000 7f790a59c700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f78ec06a4d0 0x7f78ec06c990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f78fc000c00 tx=0x7f78fc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.345+0000 7f78faffd700 1 -- 172.21.15.112:0/3427677822 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f790408ae60 con 0x7f790c161ce0 2023-08-27T11:27:15.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.575+0000 7f7910e5f700 1 -- 172.21.15.112:0/3427677822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f790c067a20 con 0x7f790c161ce0 2023-08-27T11:27:15.577 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:27:15.576+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:27:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.580+0000 7f78faffd700 1 -- 172.21.15.112:0/3427677822 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7904018090 con 0x7f790c161ce0 2023-08-27T11:27:15.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.582+0000 7f7910e5f700 1 -- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f78ec06a4d0 msgr2=0x7f78ec06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.582+0000 7f7910e5f700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f78ec06a4d0 0x7f78ec06c990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f78fc000c00 tx=0x7f78fc008030 comp rx=0 tx=0).stop 2023-08-27T11:27:15.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.582+0000 7f7910e5f700 1 -- 172.21.15.112:0/3427677822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c161ce0 msgr2=0x7f790c0cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.582+0000 7f7910e5f700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c161ce0 0x7f790c0cb690 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f790400cb60 tx=0x7f790400cf20 comp rx=0 tx=0).stop 2023-08-27T11:27:15.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.582+0000 7f7910e5f700 1 -- 172.21.15.112:0/3427677822 shutdown_connections 2023-08-27T11:27:15.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.582+0000 7f7910e5f700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f78ec06a4d0 0x7f78ec06c990 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.582+0000 7f7910e5f700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c161ce0 0x7f790c0cb690 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.582+0000 7f7910e5f700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f790c0b1400 0x7f790c144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.582+0000 7f7910e5f700 1 --2- 172.21.15.112:0/3427677822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f790c0b17e0 0x7f790c1f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.582+0000 7f7910e5f700 1 -- 172.21.15.112:0/3427677822 >> 172.21.15.112:0/3427677822 conn(0x7f790c0d0000 msgr2=0x7f790c155a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:15.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.582+0000 7f7910e5f700 1 -- 172.21.15.112:0/3427677822 shutdown_connections 2023-08-27T11:27:15.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.582+0000 7f7910e5f700 1 -- 172.21.15.112:0/3427677822 wait complete. 2023-08-27T11:27:15.587 INFO:tasks.ceph.mds.a.smithi112.stderr: -3421> 2023-08-27T11:27:01.556+0000 7fb355c82700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:27:15.588 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:27:15.597 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:27:15.612 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:27:15.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.721+0000 7f7582101700 1 Processor -- start 2023-08-27T11:27:15.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.721+0000 7f7582101700 1 -- start start 2023-08-27T11:27:15.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f7582101700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f757c157c60 0x7f757c160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f7582101700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f757c161c30 0x7f757c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f7582101700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c06e050 con 0x7f757c0aee50 2023-08-27T11:27:15.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f7582101700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757c0adce0 con 0x7f757c157c60 2023-08-27T11:27:15.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f7582101700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757c0ad340 con 0x7f757c161c30 2023-08-27T11:27:15.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f757b7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f757c157c60 0x7f757c160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f7572ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f757c0aee50 0x7f757c161520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57192/0 (socket says 172.21.15.112:57192) 2023-08-27T11:27:15.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f7572ffd700 1 -- 172.21.15.112:0/3051821256 learned_addr learned my addr 172.21.15.112:0/3051821256 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:15.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f757bfff700 1 --2- 172.21.15.112:0/3051821256 >> v2:172.21.15.138:3300/0 conn(0x7f757c161c30 0x7f757c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f757b7fe700 1 -- 172.21.15.112:0/3051821256 >> v2:172.21.15.138:3300/0 conn(0x7f757c161c30 msgr2=0x7f757c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:15.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f757b7fe700 1 --2- 172.21.15.112:0/3051821256 >> v2:172.21.15.138:3300/0 conn(0x7f757c161c30 0x7f757c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f757b7fe700 1 -- 172.21.15.112:0/3051821256 >> v1:172.21.15.112:6789/0 conn(0x7f757c0aee50 legacy=0x7f757c161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f757b7fe700 1 -- 172.21.15.112:0/3051821256 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0ad620 con 0x7f757c157c60 2023-08-27T11:27:15.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f757affd700 1 -- 172.21.15.112:0/3051821256 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737440180 0 0) 0x7f757c06e050 con 0x7f757c0aee50 2023-08-27T11:27:15.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f757b7fe700 1 --2- 172.21.15.112:0/3051821256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c157c60 0x7f757c160fe0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f757c0b2790 tx=0x7f756c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ef7df71ab02586 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.722+0000 7f757affd700 1 -- 172.21.15.112:0/3051821256 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c011830 con 0x7f757c157c60 2023-08-27T11:27:15.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.723+0000 7f757affd700 1 -- 172.21.15.112:0/3051821256 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c011990 con 0x7f757c157c60 2023-08-27T11:27:15.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.723+0000 7f757affd700 1 -- 172.21.15.112:0/3051821256 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c011d20 con 0x7f757c157c60 2023-08-27T11:27:15.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.723+0000 7f7582101700 1 -- 172.21.15.112:0/3051821256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c157c60 msgr2=0x7f757c160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.723+0000 7f7582101700 1 --2- 172.21.15.112:0/3051821256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c157c60 0x7f757c160fe0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f757c0b2790 tx=0x7f756c009b20 comp rx=0 tx=0).stop 2023-08-27T11:27:15.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.723+0000 7f7582101700 1 -- 172.21.15.112:0/3051821256 shutdown_connections 2023-08-27T11:27:15.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.723+0000 7f7582101700 1 --2- 172.21.15.112:0/3051821256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c157c60 0x7f757c160fe0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.723+0000 7f7582101700 1 --2- 172.21.15.112:0/3051821256 >> v2:172.21.15.138:3300/0 conn(0x7f757c161c30 0x7f757c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.723+0000 7f7582101700 1 -- 172.21.15.112:0/3051821256 >> 172.21.15.112:0/3051821256 conn(0x7f757c0b1380 msgr2=0x7f757c0b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:15.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.723+0000 7f7582101700 1 -- 172.21.15.112:0/3051821256 shutdown_connections 2023-08-27T11:27:15.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.723+0000 7f7582101700 1 -- 172.21.15.112:0/3051821256 wait complete. 2023-08-27T11:27:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.724+0000 7f7582101700 1 Processor -- start 2023-08-27T11:27:15.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.724+0000 7f7582101700 1 -- start start 2023-08-27T11:27:15.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.725+0000 7f7582101700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0aee50 0x7f757c1506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.725+0000 7f7582101700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c157c60 0x7f757c14ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.725+0000 7f7582101700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c161c30 0x7f757c14c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.725+0000 7f7582101700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c153800 con 0x7f757c161c30 2023-08-27T11:27:15.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.725+0000 7f7582101700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c14e9d0 con 0x7f757c0aee50 2023-08-27T11:27:15.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.725+0000 7f7582101700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c153140 con 0x7f757c157c60 2023-08-27T11:27:15.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.725+0000 7f757bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c161c30 0x7f757c14c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.725+0000 7f757bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c161c30 0x7f757c14c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34586/0 (socket says 172.21.15.112:34586) 2023-08-27T11:27:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.725+0000 7f757bfff700 1 -- 172.21.15.112:0/808932190 learned_addr learned my addr 172.21.15.112:0/808932190 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.725+0000 7f7572ffd700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c157c60 0x7f757c14ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.725+0000 7f757b7fe700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0aee50 0x7f757c1506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.725+0000 7f757bfff700 1 -- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c157c60 msgr2=0x7f757c14ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.725+0000 7f757bfff700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c157c60 0x7f757c14ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.726+0000 7f757bfff700 1 -- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0aee50 msgr2=0x7f757c1506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.726+0000 7f757bfff700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0aee50 0x7f757c1506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.726+0000 7f757bfff700 1 -- 172.21.15.112:0/808932190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c0094a0 con 0x7f757c161c30 2023-08-27T11:27:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.726+0000 7f757b7fe700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0aee50 0x7f757c1506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.726+0000 7f757bfff700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c161c30 0x7f757c14c510 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7574007e60 tx=0x7f757400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.726+0000 7f7578ff9700 1 -- 172.21.15.112:0/808932190 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757400ede0 con 0x7f757c161c30 2023-08-27T11:27:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.726+0000 7f7578ff9700 1 -- 172.21.15.112:0/808932190 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7574014420 con 0x7f757c161c30 2023-08-27T11:27:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.726+0000 7f7582101700 1 -- 172.21.15.112:0/808932190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c153420 con 0x7f757c161c30 2023-08-27T11:27:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.726+0000 7f7578ff9700 1 -- 172.21.15.112:0/808932190 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75740147d0 con 0x7f757c161c30 2023-08-27T11:27:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.726+0000 7f7582101700 1 -- 172.21.15.112:0/808932190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c14b3b0 con 0x7f757c161c30 2023-08-27T11:27:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.727+0000 7f7582101700 1 -- 172.21.15.112:0/808932190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c153940 con 0x7f757c161c30 2023-08-27T11:27:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.727+0000 7f7578ff9700 1 -- 172.21.15.112:0/808932190 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f7574014930 con 0x7f757c161c30 2023-08-27T11:27:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.728+0000 7f7578ff9700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f755c06a4d0 0x7f755c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.728+0000 7f7578ff9700 1 -- 172.21.15.112:0/808932190 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f75740893f0 con 0x7f757c161c30 2023-08-27T11:27:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.728+0000 7f757b7fe700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f755c06a4d0 0x7f755c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.728+0000 7f757b7fe700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f755c06a4d0 0x7f755c06c990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f756c0090b0 tx=0x7f756c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.732+0000 7f7578ff9700 1 -- 172.21.15.112:0/808932190 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f75740544f0 con 0x7f757c161c30 2023-08-27T11:27:15.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:15.965+0000 7f7582101700 1 -- 172.21.15.112:0/808932190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f757c067a20 con 0x7f757c161c30 2023-08-27T11:27:16.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.580+0000 7f7578ff9700 1 -- 172.21.15.112:0/808932190 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7574057b10 con 0x7f757c161c30 2023-08-27T11:27:16.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.583+0000 7f7582101700 1 -- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f755c06a4d0 msgr2=0x7f755c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:16.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.583+0000 7f7582101700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f755c06a4d0 0x7f755c06c990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f756c0090b0 tx=0x7f756c01c040 comp rx=0 tx=0).stop 2023-08-27T11:27:16.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.583+0000 7f7582101700 1 -- 172.21.15.112:0/808932190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c161c30 msgr2=0x7f757c14c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:16.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.583+0000 7f7582101700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c161c30 0x7f757c14c510 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7574007e60 tx=0x7f757400e3c0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.583+0000 7f7582101700 1 -- 172.21.15.112:0/808932190 shutdown_connections 2023-08-27T11:27:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.583+0000 7f7582101700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f755c06a4d0 0x7f755c06c990 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.583+0000 7f7582101700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c161c30 0x7f757c14c510 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.583+0000 7f7582101700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0aee50 0x7f757c1506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.583+0000 7f7582101700 1 --2- 172.21.15.112:0/808932190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c157c60 0x7f757c14ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.583+0000 7f7582101700 1 -- 172.21.15.112:0/808932190 >> 172.21.15.112:0/808932190 conn(0x7f757c0b1380 msgr2=0x7f757c14bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.583+0000 7f7582101700 1 -- 172.21.15.112:0/808932190 shutdown_connections 2023-08-27T11:27:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.583+0000 7f7582101700 1 -- 172.21.15.112:0/808932190 wait complete. 2023-08-27T11:27:16.598 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:27:16.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.762+0000 7f7f0b891700 1 Processor -- start 2023-08-27T11:27:16.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.762+0000 7f7f0b891700 1 -- start start 2023-08-27T11:27:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0b891700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7f04157ae0 0x7f7f0415e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0b891700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f040b3de0 0x7f7f04161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0b891700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7f04161d60 0x7f7f0406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0b891700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f040b1aa0 con 0x7f7f04157ae0 2023-08-27T11:27:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0b891700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f040b4bd0 con 0x7f7f04161d60 2023-08-27T11:27:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0b891700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f041610f0 con 0x7f7f040b3de0 2023-08-27T11:27:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0962d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7f04157ae0 0x7f7f0415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0962d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7f04157ae0 0x7f7f0415e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34594/0 (socket says 172.21.15.112:34594) 2023-08-27T11:27:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0962d700 1 -- 172.21.15.112:0/510027315 learned_addr learned my addr 172.21.15.112:0/510027315 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0962d700 1 -- 172.21.15.112:0/510027315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f040b3de0 msgr2=0x7f7f04161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f09e2e700 1 --2- 172.21.15.112:0/510027315 >> v2:172.21.15.138:3300/0 conn(0x7f7f04161d60 0x7f7f0406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0962d700 1 --2- 172.21.15.112:0/510027315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f040b3de0 0x7f7f04161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0962d700 1 -- 172.21.15.112:0/510027315 >> v2:172.21.15.138:3300/0 conn(0x7f7f04161d60 msgr2=0x7f7f0406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0962d700 1 --2- 172.21.15.112:0/510027315 >> v2:172.21.15.138:3300/0 conn(0x7f7f04161d60 0x7f7f0406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.763+0000 7f7f0962d700 1 -- 172.21.15.112:0/510027315 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f04161390 con 0x7f7f04157ae0 2023-08-27T11:27:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.764+0000 7f7f0962d700 1 --2- 172.21.15.112:0/510027315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f04157ae0 0x7f7f0415e580 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7f041472e0 tx=0x7f7ef4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ac23bc500e9081 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.764+0000 7f7efbfff700 1 -- 172.21.15.112:0/510027315 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400b870 con 0x7f7f04157ae0 2023-08-27T11:27:16.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.764+0000 7f7efbfff700 1 -- 172.21.15.112:0/510027315 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef400b9d0 con 0x7f7f04157ae0 2023-08-27T11:27:16.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.764+0000 7f7efbfff700 1 -- 172.21.15.112:0/510027315 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400bd60 con 0x7f7f04157ae0 2023-08-27T11:27:16.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.764+0000 7f7f0b891700 1 -- 172.21.15.112:0/510027315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f04157ae0 msgr2=0x7f7f0415e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:16.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.764+0000 7f7f0b891700 1 --2- 172.21.15.112:0/510027315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f04157ae0 0x7f7f0415e580 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7f041472e0 tx=0x7f7ef4009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.764+0000 7f7f0b891700 1 -- 172.21.15.112:0/510027315 shutdown_connections 2023-08-27T11:27:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.764+0000 7f7f0b891700 1 --2- 172.21.15.112:0/510027315 >> v2:172.21.15.138:3300/0 conn(0x7f7f04161d60 0x7f7f0406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.764+0000 7f7f0b891700 1 --2- 172.21.15.112:0/510027315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f04157ae0 0x7f7f0415e580 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.764+0000 7f7f0b891700 1 --2- 172.21.15.112:0/510027315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f040b3de0 0x7f7f04161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.764+0000 7f7f0b891700 1 -- 172.21.15.112:0/510027315 >> 172.21.15.112:0/510027315 conn(0x7f7f040b1be0 msgr2=0x7f7f040b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.764+0000 7f7f0b891700 1 -- 172.21.15.112:0/510027315 shutdown_connections 2023-08-27T11:27:16.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.764+0000 7f7f0b891700 1 -- 172.21.15.112:0/510027315 wait complete. 2023-08-27T11:27:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.765+0000 7f7f0b891700 1 Processor -- start 2023-08-27T11:27:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.766+0000 7f7f0b891700 1 -- start start 2023-08-27T11:27:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.766+0000 7f7f0b891700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f040b3de0 0x7f7f0415fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.766+0000 7f7f0b891700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f04157ae0 0x7f7f04156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.766+0000 7f7f0b891700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f04161d60 0x7f7f04156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.766+0000 7f7f0b891700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f041573e0 con 0x7f7f04161d60 2023-08-27T11:27:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.766+0000 7f7f0b891700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0415b830 con 0x7f7f04157ae0 2023-08-27T11:27:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.766+0000 7f7f0b891700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0415bb10 con 0x7f7f040b3de0 2023-08-27T11:27:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.766+0000 7f7f09e2e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f04161d60 0x7f7f04156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.766+0000 7f7f09e2e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f04161d60 0x7f7f04156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34600/0 (socket says 172.21.15.112:34600) 2023-08-27T11:27:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.766+0000 7f7f09e2e700 1 -- 172.21.15.112:0/4033822135 learned_addr learned my addr 172.21.15.112:0/4033822135 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.766+0000 7f7f08e2c700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f04157ae0 0x7f7f04156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.766+0000 7f7f0962d700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f040b3de0 0x7f7f0415fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:16.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.767+0000 7f7f09e2e700 1 -- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f040b3de0 msgr2=0x7f7f0415fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:16.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.767+0000 7f7f09e2e700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f040b3de0 0x7f7f0415fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.767+0000 7f7f09e2e700 1 -- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f04157ae0 msgr2=0x7f7f04156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:16.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.767+0000 7f7f09e2e700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f04157ae0 0x7f7f04156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.767+0000 7f7f09e2e700 1 -- 172.21.15.112:0/4033822135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef4009550 con 0x7f7f04161d60 2023-08-27T11:27:16.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.767+0000 7f7f09e2e700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f04161d60 0x7f7f04156ea0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00cc00 tx=0x7f7efc00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:16.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.767+0000 7f7ef9ffb700 1 -- 172.21.15.112:0/4033822135 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc003a20 con 0x7f7f04161d60 2023-08-27T11:27:16.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.767+0000 7f7ef9ffb700 1 -- 172.21.15.112:0/4033822135 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc003b80 con 0x7f7f04161d60 2023-08-27T11:27:16.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.767+0000 7f7f0b891700 1 -- 172.21.15.112:0/4033822135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0415bdf0 con 0x7f7f04161d60 2023-08-27T11:27:16.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.767+0000 7f7ef9ffb700 1 -- 172.21.15.112:0/4033822135 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc00a9d0 con 0x7f7f04161d60 2023-08-27T11:27:16.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.767+0000 7f7f0b891700 1 -- 172.21.15.112:0/4033822135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0415c5f0 con 0x7f7f04161d60 2023-08-27T11:27:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.768+0000 7f7f0b891700 1 -- 172.21.15.112:0/4033822135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f04067a20 con 0x7f7f04161d60 2023-08-27T11:27:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.768+0000 7f7ef9ffb700 1 -- 172.21.15.112:0/4033822135 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f7efc00ab30 con 0x7f7f04161d60 2023-08-27T11:27:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.769+0000 7f7ef9ffb700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ee806a4d0 0x7f7ee806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.769+0000 7f7ef9ffb700 1 -- 172.21.15.112:0/4033822135 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f7efc088df0 con 0x7f7f04161d60 2023-08-27T11:27:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.769+0000 7f7f0962d700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ee806a4d0 0x7f7ee806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.769+0000 7f7f0962d700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ee806a4d0 0x7f7ee806c990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4000c00 tx=0x7f7ef4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:16.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.773+0000 7f7ef9ffb700 1 -- 172.21.15.112:0/4033822135 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7efc053e20 con 0x7f7f04161d60 2023-08-27T11:27:16.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:16.961+0000 7f7f0b891700 1 -- 172.21.15.112:0/4033822135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7f040b48e0 con 0x7f7f04161d60 2023-08-27T11:27:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.668+0000 7f7ef9ffb700 1 -- 172.21.15.112:0/4033822135 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7efc057440 con 0x7f7f04161d60 2023-08-27T11:27:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.671+0000 7f7f0b891700 1 -- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ee806a4d0 msgr2=0x7f7ee806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.671+0000 7f7f0b891700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ee806a4d0 0x7f7ee806c990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4000c00 tx=0x7f7ef4008030 comp rx=0 tx=0).stop 2023-08-27T11:27:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.671+0000 7f7f0b891700 1 -- 172.21.15.112:0/4033822135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f04161d60 msgr2=0x7f7f04156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.671+0000 7f7f0b891700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f04161d60 0x7f7f04156ea0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00cc00 tx=0x7f7efc00cf10 comp rx=0 tx=0).stop 2023-08-27T11:27:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.671+0000 7f7f0b891700 1 -- 172.21.15.112:0/4033822135 shutdown_connections 2023-08-27T11:27:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.671+0000 7f7f0b891700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ee806a4d0 0x7f7ee806c990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.671+0000 7f7f0b891700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f04161d60 0x7f7f04156ea0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.671+0000 7f7f0b891700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f040b3de0 0x7f7f0415fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.671+0000 7f7f0b891700 1 --2- 172.21.15.112:0/4033822135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f04157ae0 0x7f7f04156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.671+0000 7f7f0b891700 1 -- 172.21.15.112:0/4033822135 >> 172.21.15.112:0/4033822135 conn(0x7f7f040b1be0 msgr2=0x7f7f0415db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.671+0000 7f7f0b891700 1 -- 172.21.15.112:0/4033822135 shutdown_connections 2023-08-27T11:27:17.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.672+0000 7f7f0b891700 1 -- 172.21.15.112:0/4033822135 wait complete. 2023-08-27T11:27:17.676 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:27:17.686 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:27:17.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.810+0000 7f24a7527700 1 Processor -- start 2023-08-27T11:27:17.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.810+0000 7f24a7527700 1 -- start start 2023-08-27T11:27:17.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a7527700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f24a0144a10 0x7f24a015eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a7527700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f24a0144df0 0x7f24a00b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a7527700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24a00cda40 0x7f24a00b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a7527700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a015fe10 con 0x7f24a0144a10 2023-08-27T11:27:17.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a7527700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a015f570 con 0x7f24a0144df0 2023-08-27T11:27:17.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a7527700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a015f850 con 0x7f24a00cda40 2023-08-27T11:27:17.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a52c3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f24a0144a10 0x7f24a015eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a52c3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f24a0144a10 0x7f24a015eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34604/0 (socket says 172.21.15.112:34604) 2023-08-27T11:27:17.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a52c3700 1 -- 172.21.15.112:0/1405270023 learned_addr learned my addr 172.21.15.112:0/1405270023 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:17.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a5ac4700 1 --2- 172.21.15.112:0/1405270023 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24a00cda40 0x7f24a00b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a4ac2700 1 --2- 172.21.15.112:0/1405270023 >> v2:172.21.15.138:3300/0 conn(0x7f24a0144df0 0x7f24a00b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a52c3700 1 -- 172.21.15.112:0/1405270023 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24a00cda40 msgr2=0x7f24a00b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:17.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a52c3700 1 --2- 172.21.15.112:0/1405270023 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24a00cda40 0x7f24a00b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a52c3700 1 -- 172.21.15.112:0/1405270023 >> v2:172.21.15.138:3300/0 conn(0x7f24a0144df0 msgr2=0x7f24a00b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a52c3700 1 --2- 172.21.15.112:0/1405270023 >> v2:172.21.15.138:3300/0 conn(0x7f24a0144df0 0x7f24a00b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a52c3700 1 -- 172.21.15.112:0/1405270023 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a015fb30 con 0x7f24a0144a10 2023-08-27T11:27:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.811+0000 7f24a52c3700 1 --2- 172.21.15.112:0/1405270023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a0144a10 0x7f24a015eee0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f24a006bd90 tx=0x7f249c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1ccb88887c7b0d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.812+0000 7f2497fff700 1 -- 172.21.15.112:0/1405270023 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c011830 con 0x7f24a0144a10 2023-08-27T11:27:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.812+0000 7f2497fff700 1 -- 172.21.15.112:0/1405270023 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249c011990 con 0x7f24a0144a10 2023-08-27T11:27:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.812+0000 7f2497fff700 1 -- 172.21.15.112:0/1405270023 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c011d20 con 0x7f24a0144a10 2023-08-27T11:27:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.812+0000 7f24a7527700 1 -- 172.21.15.112:0/1405270023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a0144a10 msgr2=0x7f24a015eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.812+0000 7f24a7527700 1 --2- 172.21.15.112:0/1405270023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a0144a10 0x7f24a015eee0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f24a006bd90 tx=0x7f249c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.812+0000 7f24a7527700 1 -- 172.21.15.112:0/1405270023 shutdown_connections 2023-08-27T11:27:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.812+0000 7f24a7527700 1 --2- 172.21.15.112:0/1405270023 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24a00cda40 0x7f24a00b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.812+0000 7f24a7527700 1 --2- 172.21.15.112:0/1405270023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a0144a10 0x7f24a015eee0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.812+0000 7f24a7527700 1 --2- 172.21.15.112:0/1405270023 >> v2:172.21.15.138:3300/0 conn(0x7f24a0144df0 0x7f24a00b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.812+0000 7f24a7527700 1 -- 172.21.15.112:0/1405270023 >> 172.21.15.112:0/1405270023 conn(0x7f24a006c8d0 msgr2=0x7f24a006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.812+0000 7f24a7527700 1 -- 172.21.15.112:0/1405270023 shutdown_connections 2023-08-27T11:27:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.812+0000 7f24a7527700 1 -- 172.21.15.112:0/1405270023 wait complete. 2023-08-27T11:27:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.814+0000 7f24a7527700 1 Processor -- start 2023-08-27T11:27:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.814+0000 7f24a7527700 1 -- start start 2023-08-27T11:27:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a7527700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24a00cda40 0x7f24a00c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a7527700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a0144a10 0x7f24a0140130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a7527700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24a0144df0 0x7f24a0140670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a7527700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a00c6fa0 con 0x7f24a0144a10 2023-08-27T11:27:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a7527700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a0140ca0 con 0x7f24a00cda40 2023-08-27T11:27:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a7527700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a0140f80 con 0x7f24a0144df0 2023-08-27T11:27:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a4ac2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a0144a10 0x7f24a0140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a4ac2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a0144a10 0x7f24a0140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34612/0 (socket says 172.21.15.112:34612) 2023-08-27T11:27:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a4ac2700 1 -- 172.21.15.112:0/4270756801 learned_addr learned my addr 172.21.15.112:0/4270756801 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a52c3700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24a00cda40 0x7f24a00c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a5ac4700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24a0144df0 0x7f24a0140670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a4ac2700 1 -- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24a0144df0 msgr2=0x7f24a0140670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a4ac2700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24a0144df0 0x7f24a0140670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a4ac2700 1 -- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24a00cda40 msgr2=0x7f24a00c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a4ac2700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24a00cda40 0x7f24a00c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a4ac2700 1 -- 172.21.15.112:0/4270756801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c009550 con 0x7f24a0144a10 2023-08-27T11:27:17.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a5ac4700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24a0144df0 0x7f24a0140670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:17.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a4ac2700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a0144a10 0x7f24a0140130 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f249000d800 tx=0x7f249000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:17.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f2495ffb700 1 -- 172.21.15.112:0/4270756801 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24900125f0 con 0x7f24a0144a10 2023-08-27T11:27:17.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f2495ffb700 1 -- 172.21.15.112:0/4270756801 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2490012750 con 0x7f24a0144a10 2023-08-27T11:27:17.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a7527700 1 -- 172.21.15.112:0/4270756801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a0141260 con 0x7f24a0144a10 2023-08-27T11:27:17.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f2495ffb700 1 -- 172.21.15.112:0/4270756801 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490012b00 con 0x7f24a0144a10 2023-08-27T11:27:17.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.815+0000 7f24a7527700 1 -- 172.21.15.112:0/4270756801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a0141820 con 0x7f24a0144a10 2023-08-27T11:27:17.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.816+0000 7f24a7527700 1 -- 172.21.15.112:0/4270756801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a01419e0 con 0x7f24a0144a10 2023-08-27T11:27:17.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.817+0000 7f2495ffb700 1 -- 172.21.15.112:0/4270756801 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f2490012c60 con 0x7f24a0144a10 2023-08-27T11:27:17.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.817+0000 7f2495ffb700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f248406a4d0 0x7f248406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.817+0000 7f2495ffb700 1 -- 172.21.15.112:0/4270756801 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f2490088dc0 con 0x7f24a0144a10 2023-08-27T11:27:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.817+0000 7f24a52c3700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f248406a4d0 0x7f248406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.818+0000 7f24a52c3700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f248406a4d0 0x7f248406c990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f249c009c10 tx=0x7f249c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:17.821+0000 7f2495ffb700 1 -- 172.21.15.112:0/4270756801 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2490054030 con 0x7f24a0144a10 2023-08-27T11:27:18.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.012+0000 7f24a7527700 1 -- 172.21.15.112:0/4270756801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f24a0067a20 con 0x7f24a0144a10 2023-08-27T11:27:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.674+0000 7f2495ffb700 1 -- 172.21.15.112:0/4270756801 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2490057650 con 0x7f24a0144a10 2023-08-27T11:27:18.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.678+0000 7f24a7527700 1 -- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f248406a4d0 msgr2=0x7f248406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.678+0000 7f24a7527700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f248406a4d0 0x7f248406c990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f249c009c10 tx=0x7f249c008030 comp rx=0 tx=0).stop 2023-08-27T11:27:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.678+0000 7f24a7527700 1 -- 172.21.15.112:0/4270756801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a0144a10 msgr2=0x7f24a0140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.678+0000 7f24a7527700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a0144a10 0x7f24a0140130 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f249000d800 tx=0x7f249000db10 comp rx=0 tx=0).stop 2023-08-27T11:27:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.678+0000 7f24a7527700 1 -- 172.21.15.112:0/4270756801 shutdown_connections 2023-08-27T11:27:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.678+0000 7f24a7527700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f248406a4d0 0x7f248406c990 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.678+0000 7f24a7527700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24a0144df0 0x7f24a0140670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.678+0000 7f24a7527700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24a00cda40 0x7f24a00c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.678+0000 7f24a7527700 1 --2- 172.21.15.112:0/4270756801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a0144a10 0x7f24a0140130 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.678+0000 7f24a7527700 1 -- 172.21.15.112:0/4270756801 >> 172.21.15.112:0/4270756801 conn(0x7f24a006c8d0 msgr2=0x7f24a00cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.678+0000 7f24a7527700 1 -- 172.21.15.112:0/4270756801 shutdown_connections 2023-08-27T11:27:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.678+0000 7f24a7527700 1 -- 172.21.15.112:0/4270756801 wait complete. 2023-08-27T11:27:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2023-08-27T11:27:18.694 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:27:18.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.820+0000 7f90b6fa1700 1 Processor -- start 2023-08-27T11:27:18.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.820+0000 7f90b6fa1700 1 -- start start 2023-08-27T11:27:18.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.821+0000 7f90b6fa1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b00cc980 0x7f90b00cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.821+0000 7f90b6fa1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b00b46c0 con 0x7f90b0161dd0 2023-08-27T11:27:18.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.821+0000 7f90b6fa1700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b00b49a0 con 0x7f90b015e190 2023-08-27T11:27:18.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.821+0000 7f90b6fa1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00b44d0 con 0x7f90b00cc980 2023-08-27T11:27:18.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.821+0000 7f90affff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f90b0161dd0 0x7f90b00b3dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57234/0 (socket says 172.21.15.112:57234) 2023-08-27T11:27:18.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.821+0000 7f90affff700 1 -- 172.21.15.112:0/2881456089 learned_addr learned my addr 172.21.15.112:0/2881456089 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:18.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.821+0000 7f90b553e700 1 --2- 172.21.15.112:0/2881456089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b00cc980 0x7f90b00cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.821+0000 7f90af7fe700 1 -- 172.21.15.112:0/2881456089 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614775153 0 0) 0x7f90b00b46c0 con 0x7f90b0161dd0 2023-08-27T11:27:18.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.821+0000 7f90af7fe700 1 -- 172.21.15.112:0/2881456089 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90a00035b0 con 0x7f90b0161dd0 2023-08-27T11:27:18.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.821+0000 7f90af7fe700 1 -- 172.21.15.112:0/2881456089 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731344964 0 0) 0x7f90b00b49a0 con 0x7f90b015e190 2023-08-27T11:27:18.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.821+0000 7f90b553e700 1 -- 172.21.15.112:0/2881456089 >> v1:172.21.15.138:6789/0 conn(0x7f90b015e190 legacy=0x7f90b015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.821+0000 7f90b553e700 1 -- 172.21.15.112:0/2881456089 >> v1:172.21.15.112:6789/0 conn(0x7f90b0161dd0 legacy=0x7f90b00b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.821+0000 7f90b553e700 1 -- 172.21.15.112:0/2881456089 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b01612f0 con 0x7f90b00cc980 2023-08-27T11:27:18.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.822+0000 7f90b553e700 1 --2- 172.21.15.112:0/2881456089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b00cc980 0x7f90b00cce00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009dc0 tx=0x7f90a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b5bfdbd7b6faefe server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:18.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.822+0000 7f90af7fe700 1 -- 172.21.15.112:0/2881456089 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a800e070 con 0x7f90b00cc980 2023-08-27T11:27:18.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.822+0000 7f90af7fe700 1 -- 172.21.15.112:0/2881456089 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a800bb20 con 0x7f90b00cc980 2023-08-27T11:27:18.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.822+0000 7f90af7fe700 1 -- 172.21.15.112:0/2881456089 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a800be90 con 0x7f90b00cc980 2023-08-27T11:27:18.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.822+0000 7f90b6fa1700 1 -- 172.21.15.112:0/2881456089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b00cc980 msgr2=0x7f90b00cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.822+0000 7f90b6fa1700 1 --2- 172.21.15.112:0/2881456089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b00cc980 0x7f90b00cce00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009dc0 tx=0x7f90a800b330 comp rx=0 tx=0).stop 2023-08-27T11:27:18.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.822+0000 7f90b6fa1700 1 -- 172.21.15.112:0/2881456089 shutdown_connections 2023-08-27T11:27:18.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.822+0000 7f90b6fa1700 1 --2- 172.21.15.112:0/2881456089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b00cc980 0x7f90b00cce00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.822+0000 7f90b6fa1700 1 -- 172.21.15.112:0/2881456089 >> 172.21.15.112:0/2881456089 conn(0x7f90b006bbf0 msgr2=0x7f90b00cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:18.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.822+0000 7f90b6fa1700 1 -- 172.21.15.112:0/2881456089 shutdown_connections 2023-08-27T11:27:18.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.822+0000 7f90b6fa1700 1 -- 172.21.15.112:0/2881456089 wait complete. 2023-08-27T11:27:18.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.823+0000 7f90b6fa1700 1 Processor -- start 2023-08-27T11:27:18.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.824+0000 7f90b6fa1700 1 -- start start 2023-08-27T11:27:18.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.824+0000 7f90b6fa1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b00cc980 0x7f90b0151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.824+0000 7f90b6fa1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b015e190 0x7f90b0141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.824+0000 7f90b6fa1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0161dd0 0x7f90b0150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.824+0000 7f90b6fa1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b0070b60 con 0x7f90b00cc980 2023-08-27T11:27:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.824+0000 7f90b6fa1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b0151b20 con 0x7f90b015e190 2023-08-27T11:27:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.824+0000 7f90b6fa1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b0151e00 con 0x7f90b0161dd0 2023-08-27T11:27:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.824+0000 7f90b4d3d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b00cc980 0x7f90b0151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90b4d3d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b00cc980 0x7f90b0151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34624/0 (socket says 172.21.15.112:34624) 2023-08-27T11:27:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90b4d3d700 1 -- 172.21.15.112:0/398584802 learned_addr learned my addr 172.21.15.112:0/398584802 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90b553e700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0161dd0 0x7f90b0150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90affff700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b015e190 0x7f90b0141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90b4d3d700 1 -- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0161dd0 msgr2=0x7f90b0150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90b4d3d700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0161dd0 0x7f90b0150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90b4d3d700 1 -- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b015e190 msgr2=0x7f90b0141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90b4d3d700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b015e190 0x7f90b0141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90b4d3d700 1 -- 172.21.15.112:0/398584802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a8009a70 con 0x7f90b00cc980 2023-08-27T11:27:18.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90b553e700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0161dd0 0x7f90b0150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:18.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90affff700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b015e190 0x7f90b0141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:18.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90b4d3d700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b00cc980 0x7f90b0151540 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f90a40011e0 tx=0x7f90a400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:18.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90ad7fa700 1 -- 172.21.15.112:0/398584802 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a40054e0 con 0x7f90b00cc980 2023-08-27T11:27:18.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90ad7fa700 1 -- 172.21.15.112:0/398584802 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a4003680 con 0x7f90b00cc980 2023-08-27T11:27:18.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90b6fa1700 1 -- 172.21.15.112:0/398584802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b00707f0 con 0x7f90b00cc980 2023-08-27T11:27:18.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90ad7fa700 1 -- 172.21.15.112:0/398584802 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4003a30 con 0x7f90b00cc980 2023-08-27T11:27:18.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.825+0000 7f90b6fa1700 1 -- 172.21.15.112:0/398584802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b00ccde0 con 0x7f90b00cc980 2023-08-27T11:27:18.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.826+0000 7f90b6fa1700 1 -- 172.21.15.112:0/398584802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b0066fa0 con 0x7f90b00cc980 2023-08-27T11:27:18.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.827+0000 7f90ad7fa700 1 -- 172.21.15.112:0/398584802 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f90a4003b90 con 0x7f90b00cc980 2023-08-27T11:27:18.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.827+0000 7f90ad7fa700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f909406a4d0 0x7f909406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.827+0000 7f90ad7fa700 1 -- 172.21.15.112:0/398584802 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f90a4087ee0 con 0x7f90b00cc980 2023-08-27T11:27:18.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.827+0000 7f90affff700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f909406a4d0 0x7f909406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.828+0000 7f90affff700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f909406a4d0 0x7f909406c990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f90b0156780 tx=0x7f909c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:18.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:18.831+0000 7f90ad7fa700 1 -- 172.21.15.112:0/398584802 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f90a40532f0 con 0x7f90b00cc980 2023-08-27T11:27:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.026+0000 7f90b6fa1700 1 -- 172.21.15.112:0/398584802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f90b0070650 con 0x7f90b00cc980 2023-08-27T11:27:19.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.030+0000 7f90ad7fa700 1 -- 172.21.15.112:0/398584802 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f90b0070650 con 0x7f90b00cc980 2023-08-27T11:27:19.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.032+0000 7f90b6fa1700 1 -- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f909406a4d0 msgr2=0x7f909406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.032+0000 7f90b6fa1700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f909406a4d0 0x7f909406c990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f90b0156780 tx=0x7f909c009040 comp rx=0 tx=0).stop 2023-08-27T11:27:19.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.032+0000 7f90b6fa1700 1 -- 172.21.15.112:0/398584802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b00cc980 msgr2=0x7f90b0151540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.032+0000 7f90b6fa1700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b00cc980 0x7f90b0151540 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f90a40011e0 tx=0x7f90a400cae0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.032+0000 7f90b6fa1700 1 -- 172.21.15.112:0/398584802 shutdown_connections 2023-08-27T11:27:19.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.032+0000 7f90b6fa1700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f909406a4d0 0x7f909406c990 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.032+0000 7f90b6fa1700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0161dd0 0x7f90b0150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.032+0000 7f90b6fa1700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b00cc980 0x7f90b0151540 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.032+0000 7f90b6fa1700 1 --2- 172.21.15.112:0/398584802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b015e190 0x7f90b0141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.032+0000 7f90b6fa1700 1 -- 172.21.15.112:0/398584802 >> 172.21.15.112:0/398584802 conn(0x7f90b006bbf0 msgr2=0x7f90b00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:19.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.032+0000 7f90b6fa1700 1 -- 172.21.15.112:0/398584802 shutdown_connections 2023-08-27T11:27:19.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.032+0000 7f90b6fa1700 1 -- 172.21.15.112:0/398584802 wait complete. 2023-08-27T11:27:19.045 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup' 2023-08-27T11:27:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.164+0000 7ff497dd3700 1 Processor -- start 2023-08-27T11:27:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.165+0000 7ff497dd3700 1 -- start start 2023-08-27T11:27:19.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.165+0000 7ff497dd3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4900b4960 0x7ff49015db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.165+0000 7ff497dd3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff4900b4dc0 0x7ff49015e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.165+0000 7ff497dd3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4900cfe70 con 0x7ff490143a60 2023-08-27T11:27:19.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.165+0000 7ff497dd3700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff490143e40 con 0x7ff4900b4dc0 2023-08-27T11:27:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.165+0000 7ff497dd3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900b5340 con 0x7ff4900b4960 2023-08-27T11:27:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.165+0000 7ff495b6f700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff490143a60 0x7ff4900cf760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57246/0 (socket says 172.21.15.112:57246) 2023-08-27T11:27:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.165+0000 7ff495b6f700 1 -- 172.21.15.112:0/830856884 learned_addr learned my addr 172.21.15.112:0/830856884 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff49536e700 1 --2- 172.21.15.112:0/830856884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4900b4960 0x7ff49015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff496370700 1 --2- 172.21.15.112:0/830856884 >> v2:172.21.15.138:3300/0 conn(0x7ff4900b4dc0 0x7ff49015e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1867370127 0 0) 0x7ff4900cfe70 con 0x7ff490143a60 2023-08-27T11:27:19.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff47c0057c0 con 0x7ff490143a60 2023-08-27T11:27:19.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1004810149 0 0) 0x7ff47c0057c0 con 0x7ff490143a60 2023-08-27T11:27:19.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4900b4960 msgr2=0x7ff49015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:19.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 --2- 172.21.15.112:0/830856884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4900b4960 0x7ff49015db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 >> v2:172.21.15.138:3300/0 conn(0x7ff4900b4dc0 msgr2=0x7ff49015e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:19.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 --2- 172.21.15.112:0/830856884 >> v2:172.21.15.138:3300/0 conn(0x7ff4900b4dc0 0x7ff49015e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4900ce9a0 con 0x7ff490143a60 2023-08-27T11:27:19.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7ff48c0017f0 con 0x7ff490143a60 2023-08-27T11:27:19.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 >> v1:172.21.15.112:6789/0 conn(0x7ff490143a60 legacy=0x7ff4900cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 --2- 172.21.15.112:0/830856884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4900b4960 0x7ff47c005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 --2- 172.21.15.112:0/830856884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47c006190 0x7ff47c006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 --2- 172.21.15.112:0/830856884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff47c006fa0 0x7ff47c007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff490143e40 con 0x7ff47c006fa0 2023-08-27T11:27:19.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900b5340 con 0x7ff47c006190 2023-08-27T11:27:19.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c00dc20 con 0x7ff4900b4960 2023-08-27T11:27:19.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff496370700 1 --2- 172.21.15.112:0/830856884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff47c006fa0 0x7ff47c007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff495b6f700 1 --2- 172.21.15.112:0/830856884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4900b4960 0x7ff47c005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff49536e700 1 --2- 172.21.15.112:0/830856884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47c006190 0x7ff47c006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff496370700 1 -- 172.21.15.112:0/830856884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4900b4960 msgr2=0x7ff47c005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:19.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff496370700 1 --2- 172.21.15.112:0/830856884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4900b4960 0x7ff47c005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff496370700 1 -- 172.21.15.112:0/830856884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47c006190 msgr2=0x7ff47c006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:19.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff496370700 1 --2- 172.21.15.112:0/830856884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47c006190 0x7ff47c006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.166+0000 7ff496370700 1 -- 172.21.15.112:0/830856884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c0057c0 con 0x7ff47c006fa0 2023-08-27T11:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.167+0000 7ff496370700 1 --2- 172.21.15.112:0/830856884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff47c006fa0 0x7ff47c007420 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7ff488009fc0 tx=0x7ff47c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d03a69d81528a81 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.167+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48800ebd0 con 0x7ff47c006fa0 2023-08-27T11:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.167+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48800f040 con 0x7ff47c006fa0 2023-08-27T11:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.167+0000 7ff494b6d700 1 -- 172.21.15.112:0/830856884 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488014480 con 0x7ff47c006fa0 2023-08-27T11:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.167+0000 7ff497dd3700 1 -- 172.21.15.112:0/830856884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff47c006fa0 msgr2=0x7ff47c007420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.167+0000 7ff497dd3700 1 --2- 172.21.15.112:0/830856884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff47c006fa0 0x7ff47c007420 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7ff488009fc0 tx=0x7ff47c007a90 comp rx=0 tx=0).stop 2023-08-27T11:27:19.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.167+0000 7ff496370700 1 -- 172.21.15.112:0/830856884 reap_dead start 2023-08-27T11:27:19.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.167+0000 7ff497dd3700 1 -- 172.21.15.112:0/830856884 shutdown_connections 2023-08-27T11:27:19.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.167+0000 7ff497dd3700 1 -- 172.21.15.112:0/830856884 >> 172.21.15.112:0/830856884 conn(0x7ff49006adb0 msgr2=0x7ff49006b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:19.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.167+0000 7ff497dd3700 1 -- 172.21.15.112:0/830856884 shutdown_connections 2023-08-27T11:27:19.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.167+0000 7ff497dd3700 1 -- 172.21.15.112:0/830856884 wait complete. 2023-08-27T11:27:19.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.168+0000 7ff497dd3700 1 Processor -- start 2023-08-27T11:27:19.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.169+0000 7ff497dd3700 1 -- start start 2023-08-27T11:27:19.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.169+0000 7ff497dd3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff47c006fa0 0x7ff49013fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.169+0000 7ff497dd3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4900c50f0 0x7ff4901f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.169+0000 7ff497dd3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff490157b90 0x7ff490158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.169+0000 7ff497dd3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff490156910 con 0x7ff47c006fa0 2023-08-27T11:27:19.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.169+0000 7ff497dd3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49013ff60 con 0x7ff490157b90 2023-08-27T11:27:19.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.169+0000 7ff497dd3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4901592d0 con 0x7ff4900c50f0 2023-08-27T11:27:19.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.169+0000 7ff495b6f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff47c006fa0 0x7ff49013fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.169+0000 7ff495b6f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff47c006fa0 0x7ff49013fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34644/0 (socket says 172.21.15.112:34644) 2023-08-27T11:27:19.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.169+0000 7ff495b6f700 1 -- 172.21.15.112:0/4293140683 learned_addr learned my addr 172.21.15.112:0/4293140683 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:19.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.169+0000 7ff496370700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff490157b90 0x7ff490158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.169+0000 7ff49536e700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4900c50f0 0x7ff4901f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.170+0000 7ff495b6f700 1 -- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4900c50f0 msgr2=0x7ff4901f2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.170+0000 7ff495b6f700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4900c50f0 0x7ff4901f2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.170+0000 7ff495b6f700 1 -- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff490157b90 msgr2=0x7ff490158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.170+0000 7ff495b6f700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff490157b90 0x7ff490158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.170+0000 7ff495b6f700 1 -- 172.21.15.112:0/4293140683 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff490156af0 con 0x7ff47c006fa0 2023-08-27T11:27:19.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.170+0000 7ff495b6f700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff47c006fa0 0x7ff49013fa20 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7ff48c007da0 tx=0x7ff48c001890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:19.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.170+0000 7ff4827fc700 1 -- 172.21.15.112:0/4293140683 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0072e0 con 0x7ff47c006fa0 2023-08-27T11:27:19.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.170+0000 7ff4827fc700 1 -- 172.21.15.112:0/4293140683 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c007440 con 0x7ff47c006fa0 2023-08-27T11:27:19.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.170+0000 7ff4827fc700 1 -- 172.21.15.112:0/4293140683 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c005830 con 0x7ff47c006fa0 2023-08-27T11:27:19.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.170+0000 7ff497dd3700 1 -- 172.21.15.112:0/4293140683 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff490156dd0 con 0x7ff47c006fa0 2023-08-27T11:27:19.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.170+0000 7ff497dd3700 1 -- 172.21.15.112:0/4293140683 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff490157390 con 0x7ff47c006fa0 2023-08-27T11:27:19.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.171+0000 7ff497dd3700 1 -- 172.21.15.112:0/4293140683 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff490067a20 con 0x7ff47c006fa0 2023-08-27T11:27:19.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.171+0000 7ff4827fc700 1 -- 172.21.15.112:0/4293140683 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7ff48c005990 con 0x7ff47c006fa0 2023-08-27T11:27:19.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.172+0000 7ff4827fc700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff47406a4d0 0x7ff47406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.172+0000 7ff4827fc700 1 -- 172.21.15.112:0/4293140683 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff48c088a00 con 0x7ff47c006fa0 2023-08-27T11:27:19.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.172+0000 7ff49536e700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff47406a4d0 0x7ff47406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.172+0000 7ff49536e700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff47406a4d0 0x7ff47406c990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff49015c3c0 tx=0x7ff484008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:19.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.176+0000 7ff4827fc700 1 -- 172.21.15.112:0/4293140683 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff48c053e10 con 0x7ff47c006fa0 2023-08-27T11:27:19.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.447+0000 7ff497dd3700 1 -- 172.21.15.112:0/4293140683 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]} v 0) v1 -- 0x7ff4900b3850 con 0x7ff47c006fa0 2023-08-27T11:27:19.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.677+0000 7ff4827fc700 1 -- 172.21.15.112:0/4293140683 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]}]=0 v451) v1 ==== 156+0+0 (secure 0 0 0) 0x7ff48c057430 con 0x7ff47c006fa0 2023-08-27T11:27:19.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.679+0000 7ff497dd3700 1 -- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff47406a4d0 msgr2=0x7ff47406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.679+0000 7ff497dd3700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff47406a4d0 0x7ff47406c990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff49015c3c0 tx=0x7ff484008040 comp rx=0 tx=0).stop 2023-08-27T11:27:19.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.679+0000 7ff497dd3700 1 -- 172.21.15.112:0/4293140683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff47c006fa0 msgr2=0x7ff49013fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.679+0000 7ff497dd3700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff47c006fa0 0x7ff49013fa20 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7ff48c007da0 tx=0x7ff48c001890 comp rx=0 tx=0).stop 2023-08-27T11:27:19.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.679+0000 7ff497dd3700 1 -- 172.21.15.112:0/4293140683 shutdown_connections 2023-08-27T11:27:19.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.679+0000 7ff497dd3700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff47406a4d0 0x7ff47406c990 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.679+0000 7ff497dd3700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff490157b90 0x7ff490158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.679+0000 7ff497dd3700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff47c006fa0 0x7ff49013fa20 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.679+0000 7ff497dd3700 1 --2- 172.21.15.112:0/4293140683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4900c50f0 0x7ff4901f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.679+0000 7ff497dd3700 1 -- 172.21.15.112:0/4293140683 >> 172.21.15.112:0/4293140683 conn(0x7ff49006adb0 msgr2=0x7ff49015db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:19.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.679+0000 7ff497dd3700 1 -- 172.21.15.112:0/4293140683 shutdown_connections 2023-08-27T11:27:19.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.679+0000 7ff497dd3700 1 -- 172.21.15.112:0/4293140683 wait complete. 2023-08-27T11:27:19.694 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info' command without subvolume group ... ok 2023-08-27T11:27:19.694 INFO:tasks.cephfs_test_runner:Starting test: test_volume_ls (tasks.cephfs.test_volumes.TestVolumes) That the existing and the newly created volumes can be listed and 2023-08-27T11:27:19.695 INFO:tasks.cephfs_test_runner:test_volume_ls (tasks.cephfs.test_volumes.TestVolumes) 2023-08-27T11:27:19.695 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls' 2023-08-27T11:27:19.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.818+0000 7f290a6ff700 1 Processor -- start 2023-08-27T11:27:19.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.819+0000 7f290a6ff700 1 -- start start 2023-08-27T11:27:19.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.819+0000 7f290a6ff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f29040b1400 0x7f29040cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.819+0000 7f290a6ff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904161ce0 0x7f290406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.819+0000 7f290a6ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2904161850 con 0x7f29040b17e0 2023-08-27T11:27:19.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.819+0000 7f290a6ff700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29040cfc10 con 0x7f29040b1400 2023-08-27T11:27:19.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.819+0000 7f290a6ff700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2904161200 con 0x7f2904161ce0 2023-08-27T11:27:19.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f29037fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f29040b17e0 0x7f290406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57268/0 (socket says 172.21.15.112:57268) 2023-08-27T11:27:19.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f29037fe700 1 -- 172.21.15.112:0/1410383358 learned_addr learned my addr 172.21.15.112:0/1410383358 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:19.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428708559 0 0) 0x7f2904161850 con 0x7f29040b17e0 2023-08-27T11:27:19.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28f00035b0 con 0x7f29040b17e0 2023-08-27T11:27:19.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2903fff700 1 --2- 172.21.15.112:0/1410383358 >> v2:172.21.15.138:3300/0 conn(0x7f29040b1400 0x7f29040cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2908c9c700 1 --2- 172.21.15.112:0/1410383358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904161ce0 0x7f290406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3453249218 0 0) 0x7f28f00035b0 con 0x7f29040b17e0 2023-08-27T11:27:19.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904161ce0 msgr2=0x7f290406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:19.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 --2- 172.21.15.112:0/1410383358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904161ce0 0x7f290406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 >> v2:172.21.15.138:3300/0 conn(0x7f29040b1400 msgr2=0x7f29040cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:19.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 --2- 172.21.15.112:0/1410383358 >> v2:172.21.15.138:3300/0 conn(0x7f29040b1400 0x7f29040cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29041614e0 con 0x7f29040b17e0 2023-08-27T11:27:19.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f28ec001670 con 0x7f29040b17e0 2023-08-27T11:27:19.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 >> v1:172.21.15.112:6789/0 conn(0x7f29040b17e0 legacy=0x7f290406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 --2- 172.21.15.112:0/1410383358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f0003a00 0x7f28f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 --2- 172.21.15.112:0/1410383358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904161ce0 0x7f28f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 --2- 172.21.15.112:0/1410383358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28f0004d90 0x7f28f0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29040cfc10 con 0x7f28f0003a00 2023-08-27T11:27:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2904161200 con 0x7f28f0004d90 2023-08-27T11:27:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f000bc50 con 0x7f2904161ce0 2023-08-27T11:27:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2903fff700 1 --2- 172.21.15.112:0/1410383358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f0003a00 0x7f28f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f29037fe700 1 --2- 172.21.15.112:0/1410383358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904161ce0 0x7f28f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2908c9c700 1 --2- 172.21.15.112:0/1410383358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28f0004d90 0x7f28f0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2903fff700 1 -- 172.21.15.112:0/1410383358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904161ce0 msgr2=0x7f28f0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:19.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2903fff700 1 --2- 172.21.15.112:0/1410383358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904161ce0 0x7f28f0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2903fff700 1 -- 172.21.15.112:0/1410383358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28f0004d90 msgr2=0x7f28f0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:19.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2903fff700 1 --2- 172.21.15.112:0/1410383358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28f0004d90 0x7f28f0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2903fff700 1 -- 172.21.15.112:0/1410383358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f00035b0 con 0x7f28f0003a00 2023-08-27T11:27:19.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.820+0000 7f2908c9c700 1 --2- 172.21.15.112:0/1410383358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28f0004d90 0x7f28f0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:19.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.821+0000 7f2903fff700 1 --2- 172.21.15.112:0/1410383358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f0003a00 0x7f28f0003e20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009890 tx=0x7f28f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a40810698ac3d9b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:19.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.821+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f400c070 con 0x7f28f0003a00 2023-08-27T11:27:19.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.821+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4013610 con 0x7f28f0003a00 2023-08-27T11:27:19.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.821+0000 7f2902ffd700 1 -- 172.21.15.112:0/1410383358 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f40139a0 con 0x7f28f0003a00 2023-08-27T11:27:19.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.821+0000 7f290a6ff700 1 -- 172.21.15.112:0/1410383358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f0003a00 msgr2=0x7f28f0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.821+0000 7f290a6ff700 1 --2- 172.21.15.112:0/1410383358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f0003a00 0x7f28f0003e20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009890 tx=0x7f28f0005880 comp rx=0 tx=0).stop 2023-08-27T11:27:19.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.821+0000 7f2908c9c700 1 -- 172.21.15.112:0/1410383358 reap_dead start 2023-08-27T11:27:19.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.821+0000 7f290a6ff700 1 -- 172.21.15.112:0/1410383358 shutdown_connections 2023-08-27T11:27:19.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.821+0000 7f290a6ff700 1 -- 172.21.15.112:0/1410383358 >> 172.21.15.112:0/1410383358 conn(0x7f29040d0000 msgr2=0x7f29040d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:19.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.821+0000 7f290a6ff700 1 -- 172.21.15.112:0/1410383358 shutdown_connections 2023-08-27T11:27:19.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.821+0000 7f290a6ff700 1 -- 172.21.15.112:0/1410383358 wait complete. 2023-08-27T11:27:19.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.822+0000 7f290a6ff700 1 Processor -- start 2023-08-27T11:27:19.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.823+0000 7f290a6ff700 1 -- start start 2023-08-27T11:27:19.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.823+0000 7f290a6ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f0003a00 0x7f29041f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.823+0000 7f290a6ff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29041604b0 0x7f29040c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.823+0000 7f290a6ff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2904155460 0x7f290415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.823+0000 7f290a6ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29041f3e90 con 0x7f28f0003a00 2023-08-27T11:27:19.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.823+0000 7f290a6ff700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29041f2e90 con 0x7f2904155460 2023-08-27T11:27:19.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.823+0000 7f290a6ff700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29040c4380 con 0x7f29041604b0 2023-08-27T11:27:19.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.823+0000 7f2903fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f0003a00 0x7f29041f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.823+0000 7f2903fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f0003a00 0x7f29041f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34664/0 (socket says 172.21.15.112:34664) 2023-08-27T11:27:19.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.823+0000 7f2903fff700 1 -- 172.21.15.112:0/1754996032 learned_addr learned my addr 172.21.15.112:0/1754996032 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:19.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.824+0000 7f2908c9c700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2904155460 0x7f290415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.824+0000 7f29037fe700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29041604b0 0x7f29040c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.824+0000 7f2903fff700 1 -- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29041604b0 msgr2=0x7f29040c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.824+0000 7f2903fff700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29041604b0 0x7f29040c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.824+0000 7f2903fff700 1 -- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2904155460 msgr2=0x7f290415eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.824+0000 7f2903fff700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2904155460 0x7f290415eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.824+0000 7f2903fff700 1 -- 172.21.15.112:0/1754996032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29040c4660 con 0x7f28f0003a00 2023-08-27T11:27:19.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.824+0000 7f2903fff700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f0003a00 0x7f29041f3950 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f28f4016040 tx=0x7f28f40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:19.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.824+0000 7f2900ff9700 1 -- 172.21.15.112:0/1754996032 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4013c50 con 0x7f28f0003a00 2023-08-27T11:27:19.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.824+0000 7f2900ff9700 1 -- 172.21.15.112:0/1754996032 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4013dd0 con 0x7f28f0003a00 2023-08-27T11:27:19.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.824+0000 7f290a6ff700 1 -- 172.21.15.112:0/1754996032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29041f3150 con 0x7f28f0003a00 2023-08-27T11:27:19.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.824+0000 7f2900ff9700 1 -- 172.21.15.112:0/1754996032 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f40038e0 con 0x7f28f0003a00 2023-08-27T11:27:19.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.824+0000 7f290a6ff700 1 -- 172.21.15.112:0/1754996032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29040c4070 con 0x7f28f0003a00 2023-08-27T11:27:19.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.825+0000 7f290a6ff700 1 -- 172.21.15.112:0/1754996032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2904067a20 con 0x7f28f0003a00 2023-08-27T11:27:19.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.825+0000 7f2900ff9700 1 -- 172.21.15.112:0/1754996032 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f28f4003a40 con 0x7f28f0003a00 2023-08-27T11:27:19.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.826+0000 7f2900ff9700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f28e406a4d0 0x7f28e406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.826+0000 7f2900ff9700 1 -- 172.21.15.112:0/1754996032 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f28f4089b00 con 0x7f28f0003a00 2023-08-27T11:27:19.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.826+0000 7f29037fe700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f28e406a4d0 0x7f28e406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.826+0000 7f29037fe700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f28e406a4d0 0x7f28e406c990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f28ec001120 tx=0x7f28ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:19.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:19.830+0000 7f2900ff9700 1 -- 172.21.15.112:0/1754996032 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f28f408be60 con 0x7f28f0003a00 2023-08-27T11:27:20.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.095+0000 7f290a6ff700 1 -- 172.21.15.112:0/1754996032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]} v 0) v1 -- 0x7f29040ca310 con 0x7f28f0003a00 2023-08-27T11:27:20.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.682+0000 7f2900ff9700 1 -- 172.21.15.112:0/1754996032 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]}]=0 v452) v1 ==== 134+0+0 (secure 0 0 0) 0x7f28f4054e90 con 0x7f28f0003a00 2023-08-27T11:27:20.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.684+0000 7f290a6ff700 1 -- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f28e406a4d0 msgr2=0x7f28e406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:20.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.684+0000 7f290a6ff700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f28e406a4d0 0x7f28e406c990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f28ec001120 tx=0x7f28ec009040 comp rx=0 tx=0).stop 2023-08-27T11:27:20.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.684+0000 7f290a6ff700 1 -- 172.21.15.112:0/1754996032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f0003a00 msgr2=0x7f29041f3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:20.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.684+0000 7f290a6ff700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f0003a00 0x7f29041f3950 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f28f4016040 tx=0x7f28f40047c0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.684+0000 7f290a6ff700 1 -- 172.21.15.112:0/1754996032 shutdown_connections 2023-08-27T11:27:20.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.684+0000 7f290a6ff700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f28e406a4d0 0x7f28e406c990 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.684+0000 7f290a6ff700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2904155460 0x7f290415eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.684+0000 7f290a6ff700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f0003a00 0x7f29041f3950 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.684+0000 7f290a6ff700 1 --2- 172.21.15.112:0/1754996032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29041604b0 0x7f29040c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.684+0000 7f290a6ff700 1 -- 172.21.15.112:0/1754996032 >> 172.21.15.112:0/1754996032 conn(0x7f29040d0000 msgr2=0x7f29040ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:20.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.684+0000 7f290a6ff700 1 -- 172.21.15.112:0/1754996032 shutdown_connections 2023-08-27T11:27:20.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.684+0000 7f290a6ff700 1 -- 172.21.15.112:0/1754996032 wait complete. 2023-08-27T11:27:20.698 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:20.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.817+0000 7f29377e2700 1 Processor -- start 2023-08-27T11:27:20.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.817+0000 7f29377e2700 1 -- start start 2023-08-27T11:27:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.818+0000 7f29377e2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2930160a00 0x7f29300b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.818+0000 7f29377e2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2930146990 0x7f293015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.818+0000 7f29377e2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f293006d510 0x7f293006d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.818+0000 7f29377e2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293006dec0 con 0x7f2930146990 2023-08-27T11:27:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.818+0000 7f29377e2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293015e140 con 0x7f293006d510 2023-08-27T11:27:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.818+0000 7f29377e2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293006f930 con 0x7f2930160a00 2023-08-27T11:27:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.818+0000 7f2934d7d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2930146990 0x7f293015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.818+0000 7f2934d7d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2930146990 0x7f293015dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34674/0 (socket says 172.21.15.112:34674) 2023-08-27T11:27:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.818+0000 7f2934d7d700 1 -- 172.21.15.112:0/2643622067 learned_addr learned my addr 172.21.15.112:0/2643622067 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.818+0000 7f293557e700 1 --2- 172.21.15.112:0/2643622067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2930160a00 0x7f29300b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.818+0000 7f2935d7f700 1 --2- 172.21.15.112:0/2643622067 >> v2:172.21.15.138:3300/0 conn(0x7f293006d510 0x7f293006d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.818+0000 7f2934d7d700 1 -- 172.21.15.112:0/2643622067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2930160a00 msgr2=0x7f29300b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.818+0000 7f2934d7d700 1 --2- 172.21.15.112:0/2643622067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2930160a00 0x7f29300b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f2934d7d700 1 -- 172.21.15.112:0/2643622067 >> v2:172.21.15.138:3300/0 conn(0x7f293006d510 msgr2=0x7f293006d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f2934d7d700 1 --2- 172.21.15.112:0/2643622067 >> v2:172.21.15.138:3300/0 conn(0x7f293006d510 0x7f293006d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f2934d7d700 1 -- 172.21.15.112:0/2643622067 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293006fc10 con 0x7f2930146990 2023-08-27T11:27:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f2934d7d700 1 --2- 172.21.15.112:0/2643622067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930146990 0x7f293015dc00 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f293006e180 tx=0x7f292c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91eed2bb64e13def server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f2927fff700 1 -- 172.21.15.112:0/2643622067 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c011830 con 0x7f2930146990 2023-08-27T11:27:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f2927fff700 1 -- 172.21.15.112:0/2643622067 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c011990 con 0x7f2930146990 2023-08-27T11:27:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f2927fff700 1 -- 172.21.15.112:0/2643622067 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c011d20 con 0x7f2930146990 2023-08-27T11:27:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f29377e2700 1 -- 172.21.15.112:0/2643622067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930146990 msgr2=0x7f293015dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f29377e2700 1 --2- 172.21.15.112:0/2643622067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930146990 0x7f293015dc00 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f293006e180 tx=0x7f292c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f29377e2700 1 -- 172.21.15.112:0/2643622067 shutdown_connections 2023-08-27T11:27:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f29377e2700 1 --2- 172.21.15.112:0/2643622067 >> v2:172.21.15.138:3300/0 conn(0x7f293006d510 0x7f293006d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f29377e2700 1 --2- 172.21.15.112:0/2643622067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2930160a00 0x7f29300b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f29377e2700 1 --2- 172.21.15.112:0/2643622067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930146990 0x7f293015dc00 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.819+0000 7f29377e2700 1 -- 172.21.15.112:0/2643622067 >> 172.21.15.112:0/2643622067 conn(0x7f29300cfe50 msgr2=0x7f29300d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.820+0000 7f29377e2700 1 -- 172.21.15.112:0/2643622067 shutdown_connections 2023-08-27T11:27:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.820+0000 7f29377e2700 1 -- 172.21.15.112:0/2643622067 wait complete. 2023-08-27T11:27:20.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.821+0000 7f29377e2700 1 Processor -- start 2023-08-27T11:27:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.821+0000 7f29377e2700 1 -- start start 2023-08-27T11:27:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.821+0000 7f29377e2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f293006d510 0x7f29301f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.821+0000 7f29377e2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930146990 0x7f29300ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.821+0000 7f29377e2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2930160a00 0x7f293015f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.821+0000 7f29377e2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29300ae8a0 con 0x7f2930146990 2023-08-27T11:27:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.821+0000 7f29377e2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29300ada10 con 0x7f293006d510 2023-08-27T11:27:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.821+0000 7f29377e2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29300adc20 con 0x7f2930160a00 2023-08-27T11:27:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f2934d7d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930146990 0x7f29300ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f2934d7d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930146990 0x7f29300ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34680/0 (socket says 172.21.15.112:34680) 2023-08-27T11:27:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f2934d7d700 1 -- 172.21.15.112:0/255465100 learned_addr learned my addr 172.21.15.112:0/255465100 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f293557e700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f293006d510 0x7f29301f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f2935d7f700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2930160a00 0x7f293015f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f2934d7d700 1 -- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2930160a00 msgr2=0x7f293015f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f2934d7d700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2930160a00 0x7f293015f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f2934d7d700 1 -- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f293006d510 msgr2=0x7f29301f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f2934d7d700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f293006d510 0x7f29301f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f2934d7d700 1 -- 172.21.15.112:0/255465100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c009550 con 0x7f2930146990 2023-08-27T11:27:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f2934d7d700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930146990 0x7f29300ceb90 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f292c000c00 tx=0x7f292c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f2925ffb700 1 -- 172.21.15.112:0/255465100 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c004240 con 0x7f2930146990 2023-08-27T11:27:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f2925ffb700 1 -- 172.21.15.112:0/255465100 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c01f070 con 0x7f2930146990 2023-08-27T11:27:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f29377e2700 1 -- 172.21.15.112:0/255465100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29300adf00 con 0x7f2930146990 2023-08-27T11:27:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f2925ffb700 1 -- 172.21.15.112:0/255465100 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c0036a0 con 0x7f2930146990 2023-08-27T11:27:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.822+0000 7f29377e2700 1 -- 172.21.15.112:0/255465100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29300ae4c0 con 0x7f2930146990 2023-08-27T11:27:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.823+0000 7f29377e2700 1 -- 172.21.15.112:0/255465100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29300ae9e0 con 0x7f2930146990 2023-08-27T11:27:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.824+0000 7f2925ffb700 1 -- 172.21.15.112:0/255465100 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f292c021030 con 0x7f2930146990 2023-08-27T11:27:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.824+0000 7f2925ffb700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f291406a4d0 0x7f291406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.824+0000 7f2925ffb700 1 -- 172.21.15.112:0/255465100 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f292c0898b0 con 0x7f2930146990 2023-08-27T11:27:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.824+0000 7f293557e700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f291406a4d0 0x7f291406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.825+0000 7f293557e700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f291406a4d0 0x7f291406c990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2920009b40 tx=0x7f2920009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:20.829+0000 7f2925ffb700 1 -- 172.21.15.112:0/255465100 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f29300ae9e0 con 0x7f2930146990 2023-08-27T11:27:21.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.019+0000 7f29377e2700 1 -- 172.21.15.112:0/255465100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29300ae9e0 con 0x7f2930146990 2023-08-27T11:27:21.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.020+0000 7f2925ffb700 1 -- 172.21.15.112:0/255465100 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+14746 (secure 0 0 0) 0x7f29300ae9e0 con 0x7f2930146990 2023-08-27T11:27:21.022 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:21.022 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":105,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:27:18.667812+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/3180263819":"2023-08-28T11:27:15.577350+0000","172.21.15.112:6833/3180263819":"2023-08-28T11:27:15.577350+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:21.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.022+0000 7f29377e2700 1 -- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f291406a4d0 msgr2=0x7f291406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.022+0000 7f29377e2700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f291406a4d0 0x7f291406c990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2920009b40 tx=0x7f2920009210 comp rx=0 tx=0).stop 2023-08-27T11:27:21.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.022+0000 7f29377e2700 1 -- 172.21.15.112:0/255465100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930146990 msgr2=0x7f29300ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.022+0000 7f29377e2700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930146990 0x7f29300ceb90 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f292c000c00 tx=0x7f292c004750 comp rx=0 tx=0).stop 2023-08-27T11:27:21.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.022+0000 7f29377e2700 1 -- 172.21.15.112:0/255465100 shutdown_connections 2023-08-27T11:27:21.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.022+0000 7f29377e2700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f291406a4d0 0x7f291406c990 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.022+0000 7f29377e2700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2930160a00 0x7f293015f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.022+0000 7f29377e2700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f293006d510 0x7f29301f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.022+0000 7f29377e2700 1 --2- 172.21.15.112:0/255465100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2930146990 0x7f29300ceb90 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.022+0000 7f29377e2700 1 -- 172.21.15.112:0/255465100 >> 172.21.15.112:0/255465100 conn(0x7f29300cfe50 msgr2=0x7f29300d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.023+0000 7f29377e2700 1 -- 172.21.15.112:0/255465100 shutdown_connections 2023-08-27T11:27:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.023+0000 7f29377e2700 1 -- 172.21.15.112:0/255465100 wait complete. 2023-08-27T11:27:21.036 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:21.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.158+0000 7f42b0d29700 1 Processor -- start 2023-08-27T11:27:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.158+0000 7f42b0d29700 1 -- start start 2023-08-27T11:27:21.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.159+0000 7f42b0d29700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ac143ba0 0x7f42ac0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.159+0000 7f42b0d29700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ac0cfd50 con 0x7f42ac06d5b0 2023-08-27T11:27:21.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.159+0000 7f42b0d29700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ac0cf140 con 0x7f42ac1437c0 2023-08-27T11:27:21.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.159+0000 7f42b0d29700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0cf420 con 0x7f42ac143ba0 2023-08-27T11:27:21.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42aad9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f42ac06d5b0 0x7f42ac15a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57304/0 (socket says 172.21.15.112:57304) 2023-08-27T11:27:21.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42aad9d700 1 -- 172.21.15.112:0/2573456886 learned_addr learned my addr 172.21.15.112:0/2573456886 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a9d9b700 1 --2- 172.21.15.112:0/2573456886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ac143ba0 0x7f42ac0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861971808 0 0) 0x7f42ac0cfd50 con 0x7f42ac06d5b0 2023-08-27T11:27:21.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42940035b0 con 0x7f42ac06d5b0 2023-08-27T11:27:21.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4178657859 0 0) 0x7f42940035b0 con 0x7f42ac06d5b0 2023-08-27T11:27:21.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ac143ba0 msgr2=0x7f42ac0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:21.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 --2- 172.21.15.112:0/2573456886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ac143ba0 0x7f42ac0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 >> v1:172.21.15.138:6789/0 conn(0x7f42ac1437c0 legacy=0x7f42ac0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac0cf700 con 0x7f42ac06d5b0 2023-08-27T11:27:21.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f42a40019f0 con 0x7f42ac06d5b0 2023-08-27T11:27:21.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 >> v1:172.21.15.112:6789/0 conn(0x7f42ac06d5b0 legacy=0x7f42ac15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 --2- 172.21.15.112:0/2573456886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ac143ba0 0x7f4294003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 --2- 172.21.15.112:0/2573456886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4294003f70 0x7f4294004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 --2- 172.21.15.112:0/2573456886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4294004d90 0x7f4294005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0cf420 con 0x7f4294004d90 2023-08-27T11:27:21.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429400ba70 con 0x7f4294003f70 2023-08-27T11:27:21.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429400bc30 con 0x7f42ac143ba0 2023-08-27T11:27:21.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42aad9d700 1 --2- 172.21.15.112:0/2573456886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4294004d90 0x7f4294005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42a9d9b700 1 --2- 172.21.15.112:0/2573456886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4294003f70 0x7f4294004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.160+0000 7f42aa59c700 1 --2- 172.21.15.112:0/2573456886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ac143ba0 0x7f4294003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42aad9d700 1 -- 172.21.15.112:0/2573456886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ac143ba0 msgr2=0x7f4294003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:21.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42aad9d700 1 --2- 172.21.15.112:0/2573456886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ac143ba0 0x7f4294003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42aad9d700 1 -- 172.21.15.112:0/2573456886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4294003f70 msgr2=0x7f4294004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:21.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42aad9d700 1 --2- 172.21.15.112:0/2573456886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4294003f70 0x7f4294004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42aad9d700 1 -- 172.21.15.112:0/2573456886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42940035b0 con 0x7f4294004d90 2023-08-27T11:27:21.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42aad9d700 1 --2- 172.21.15.112:0/2573456886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4294004d90 0x7f4294005210 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f42a400a3b0 tx=0x7f4294005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d110d7620d3f469 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4010690 con 0x7f4294004d90 2023-08-27T11:27:21.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a40107f0 con 0x7f4294004d90 2023-08-27T11:27:21.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42a959a700 1 -- 172.21.15.112:0/2573456886 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4010ba0 con 0x7f4294004d90 2023-08-27T11:27:21.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42b0d29700 1 -- 172.21.15.112:0/2573456886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4294004d90 msgr2=0x7f4294005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42b0d29700 1 --2- 172.21.15.112:0/2573456886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4294004d90 0x7f4294005210 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f42a400a3b0 tx=0x7f4294005880 comp rx=0 tx=0).stop 2023-08-27T11:27:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42aad9d700 1 -- 172.21.15.112:0/2573456886 reap_dead start 2023-08-27T11:27:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42b0d29700 1 -- 172.21.15.112:0/2573456886 shutdown_connections 2023-08-27T11:27:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42b0d29700 1 -- 172.21.15.112:0/2573456886 >> 172.21.15.112:0/2573456886 conn(0x7f42ac0b1e90 msgr2=0x7f42ac0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42b0d29700 1 -- 172.21.15.112:0/2573456886 shutdown_connections 2023-08-27T11:27:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.161+0000 7f42b0d29700 1 -- 172.21.15.112:0/2573456886 wait complete. 2023-08-27T11:27:21.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.163+0000 7f42b0d29700 1 Processor -- start 2023-08-27T11:27:21.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.163+0000 7f42b0d29700 1 -- start start 2023-08-27T11:27:21.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.163+0000 7f42b0d29700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4294004d90 0x7f42ac155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.163+0000 7f42b0d29700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ac156c90 0x7f42ac157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42b0d29700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ac0715d0 0x7f42ac071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42b0d29700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac071fc0 con 0x7f4294004d90 2023-08-27T11:27:21.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42b0d29700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac157710 con 0x7f42ac156c90 2023-08-27T11:27:21.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42b0d29700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac1eeaf0 con 0x7f42ac0715d0 2023-08-27T11:27:21.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42aa59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4294004d90 0x7f42ac155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42aa59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4294004d90 0x7f42ac155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34698/0 (socket says 172.21.15.112:34698) 2023-08-27T11:27:21.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42aa59c700 1 -- 172.21.15.112:0/2602362870 learned_addr learned my addr 172.21.15.112:0/2602362870 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:21.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42a9d9b700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ac156c90 0x7f42ac157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42aad9d700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ac0715d0 0x7f42ac071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42aa59c700 1 -- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ac0715d0 msgr2=0x7f42ac071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42aa59c700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ac0715d0 0x7f42ac071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42aa59c700 1 -- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ac156c90 msgr2=0x7f42ac157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42aa59c700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ac156c90 0x7f42ac157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42aa59c700 1 -- 172.21.15.112:0/2602362870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac1eed30 con 0x7f4294004d90 2023-08-27T11:27:21.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42a9d9b700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ac156c90 0x7f42ac157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:21.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42aa59c700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4294004d90 0x7f42ac155c30 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f42a000f870 tx=0x7f42a000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f429affd700 1 -- 172.21.15.112:0/2602362870 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a000fe70 con 0x7f4294004d90 2023-08-27T11:27:21.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f429affd700 1 -- 172.21.15.112:0/2602362870 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a0005320 con 0x7f4294004d90 2023-08-27T11:27:21.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f429affd700 1 -- 172.21.15.112:0/2602362870 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0006860 con 0x7f4294004d90 2023-08-27T11:27:21.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42b0d29700 1 -- 172.21.15.112:0/2602362870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ac1efab0 con 0x7f4294004d90 2023-08-27T11:27:21.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.164+0000 7f42b0d29700 1 -- 172.21.15.112:0/2602362870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac1ef2b0 con 0x7f4294004d90 2023-08-27T11:27:21.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.165+0000 7f42b0d29700 1 -- 172.21.15.112:0/2602362870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ac067e90 con 0x7f4294004d90 2023-08-27T11:27:21.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.166+0000 7f429affd700 1 -- 172.21.15.112:0/2602362870 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f42a00069e0 con 0x7f4294004d90 2023-08-27T11:27:21.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.166+0000 7f429affd700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f428c06a4d0 0x7f428c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.166+0000 7f429affd700 1 -- 172.21.15.112:0/2602362870 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f42a0087ed0 con 0x7f4294004d90 2023-08-27T11:27:21.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.166+0000 7f42a9d9b700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f428c06a4d0 0x7f428c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.167+0000 7f42a9d9b700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f428c06a4d0 0x7f428c06c990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f429c009c60 tx=0x7f429c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.170+0000 7f429affd700 1 -- 172.21.15.112:0/2602362870 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f42a0056e00 con 0x7f4294004d90 2023-08-27T11:27:21.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.359+0000 7f42b0d29700 1 -- 172.21.15.112:0/2602362870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f42ac067a20 con 0x7f4294004d90 2023-08-27T11:27:21.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.359+0000 7f429affd700 1 -- 172.21.15.112:0/2602362870 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v105) v1 ==== 92+0+10 (secure 0 0 0) 0x7f42a001a020 con 0x7f4294004d90 2023-08-27T11:27:21.362 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:27:21.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.361+0000 7f42b0d29700 1 -- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f428c06a4d0 msgr2=0x7f428c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.361+0000 7f42b0d29700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f428c06a4d0 0x7f428c06c990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f429c009c60 tx=0x7f429c008040 comp rx=0 tx=0).stop 2023-08-27T11:27:21.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.361+0000 7f42b0d29700 1 -- 172.21.15.112:0/2602362870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4294004d90 msgr2=0x7f42ac155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.361+0000 7f42b0d29700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4294004d90 0x7f42ac155c30 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f42a000f870 tx=0x7f42a000fc30 comp rx=0 tx=0).stop 2023-08-27T11:27:21.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.361+0000 7f42b0d29700 1 -- 172.21.15.112:0/2602362870 shutdown_connections 2023-08-27T11:27:21.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.361+0000 7f42b0d29700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f428c06a4d0 0x7f428c06c990 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.361+0000 7f42b0d29700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42ac0715d0 0x7f42ac071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.361+0000 7f42b0d29700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4294004d90 0x7f42ac155c30 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.361+0000 7f42b0d29700 1 --2- 172.21.15.112:0/2602362870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ac156c90 0x7f42ac157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.361+0000 7f42b0d29700 1 -- 172.21.15.112:0/2602362870 >> 172.21.15.112:0/2602362870 conn(0x7f42ac0b1e90 msgr2=0x7f42ac0b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:21.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.361+0000 7f42b0d29700 1 -- 172.21.15.112:0/2602362870 shutdown_connections 2023-08-27T11:27:21.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.361+0000 7f42b0d29700 1 -- 172.21.15.112:0/2602362870 wait complete. 2023-08-27T11:27:21.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:21.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.496+0000 7f4faf569700 1 Processor -- start 2023-08-27T11:27:21.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.497+0000 7f4faf569700 1 -- start start 2023-08-27T11:27:21.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4faf569700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4fa80d1a60 0x7f4fa815e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4faf569700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa815b370 con 0x7f4fa815af90 2023-08-27T11:27:21.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4faf569700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa80b6e70 con 0x7f4fa8071360 2023-08-27T11:27:21.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4faf569700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa80b7150 con 0x7f4fa80d1a60 2023-08-27T11:27:21.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4facb04700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4fa80d1a60 0x7f4fa815e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4fad305700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4fa815af90 0x7f4fa80b6680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57320/0 (socket says 172.21.15.112:57320) 2023-08-27T11:27:21.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4fad305700 1 -- 172.21.15.112:0/4086749290 learned_addr learned my addr 172.21.15.112:0/4086749290 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:21.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4facb04700 1 -- 172.21.15.112:0/4086749290 >> v1:172.21.15.138:6789/0 conn(0x7f4fa8071360 legacy=0x7f4fa8071760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:21.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4f9ffff700 1 -- 172.21.15.112:0/4086749290 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793652273 0 0) 0x7f4fa815b370 con 0x7f4fa815af90 2023-08-27T11:27:21.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4facb04700 1 -- 172.21.15.112:0/4086749290 >> v1:172.21.15.112:6789/0 conn(0x7f4fa815af90 legacy=0x7f4fa80b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4facb04700 1 -- 172.21.15.112:0/4086749290 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa815dc70 con 0x7f4fa80d1a60 2023-08-27T11:27:21.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4facb04700 1 --2- 172.21.15.112:0/4086749290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa80d1a60 0x7f4fa815e470 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009840 tx=0x7f4fa4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695581d4b428f937 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4f9ffff700 1 -- 172.21.15.112:0/4086749290 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4004230 con 0x7f4fa80d1a60 2023-08-27T11:27:21.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4f9ffff700 1 -- 172.21.15.112:0/4086749290 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa4004390 con 0x7f4fa80d1a60 2023-08-27T11:27:21.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.498+0000 7f4f9ffff700 1 -- 172.21.15.112:0/4086749290 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa40036a0 con 0x7f4fa80d1a60 2023-08-27T11:27:21.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.499+0000 7f4faf569700 1 -- 172.21.15.112:0/4086749290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa80d1a60 msgr2=0x7f4fa815e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.499+0000 7f4faf569700 1 --2- 172.21.15.112:0/4086749290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa80d1a60 0x7f4fa815e470 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009840 tx=0x7f4fa4009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:21.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.499+0000 7f4faf569700 1 -- 172.21.15.112:0/4086749290 shutdown_connections 2023-08-27T11:27:21.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.499+0000 7f4faf569700 1 --2- 172.21.15.112:0/4086749290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa80d1a60 0x7f4fa815e470 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.499+0000 7f4faf569700 1 -- 172.21.15.112:0/4086749290 >> 172.21.15.112:0/4086749290 conn(0x7f4fa8066010 msgr2=0x7f4fa80663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:21.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.499+0000 7f4faf569700 1 -- 172.21.15.112:0/4086749290 shutdown_connections 2023-08-27T11:27:21.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.499+0000 7f4faf569700 1 -- 172.21.15.112:0/4086749290 wait complete. 2023-08-27T11:27:21.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.500+0000 7f4faf569700 1 Processor -- start 2023-08-27T11:27:21.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.500+0000 7f4faf569700 1 -- start start 2023-08-27T11:27:21.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4faf569700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8071360 0x7f4fa815fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4faf569700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa80d1a60 0x7f4fa81603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4faf569700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa815af90 0x7f4fa80cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4faf569700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa80a8800 con 0x7f4fa8071360 2023-08-27T11:27:21.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4faf569700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa8160a40 con 0x7f4fa815af90 2023-08-27T11:27:21.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4faf569700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa80b3550 con 0x7f4fa80d1a60 2023-08-27T11:27:21.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4fad305700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8071360 0x7f4fa815fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4fad305700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8071360 0x7f4fa815fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34712/0 (socket says 172.21.15.112:34712) 2023-08-27T11:27:21.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4fad305700 1 -- 172.21.15.112:0/1956546870 learned_addr learned my addr 172.21.15.112:0/1956546870 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:21.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4fadb06700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa815af90 0x7f4fa80cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4facb04700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa80d1a60 0x7f4fa81603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4fad305700 1 -- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa80d1a60 msgr2=0x7f4fa81603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4fad305700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa80d1a60 0x7f4fa81603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4fad305700 1 -- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa815af90 msgr2=0x7f4fa80cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4fad305700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa815af90 0x7f4fa80cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4fad305700 1 -- 172.21.15.112:0/1956546870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa40094a0 con 0x7f4fa8071360 2023-08-27T11:27:21.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4facb04700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa80d1a60 0x7f4fa81603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:21.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4fad305700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8071360 0x7f4fa815fea0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800c520 tx=0x7f4f9800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.501+0000 7f4f9dffb700 1 -- 172.21.15.112:0/1956546870 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98004c80 con 0x7f4fa8071360 2023-08-27T11:27:21.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.502+0000 7f4f9dffb700 1 -- 172.21.15.112:0/1956546870 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f98004e20 con 0x7f4fa8071360 2023-08-27T11:27:21.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.502+0000 7f4faf569700 1 -- 172.21.15.112:0/1956546870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa80b3830 con 0x7f4fa8071360 2023-08-27T11:27:21.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.502+0000 7f4f9dffb700 1 -- 172.21.15.112:0/1956546870 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98007310 con 0x7f4fa8071360 2023-08-27T11:27:21.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.502+0000 7f4faf569700 1 -- 172.21.15.112:0/1956546870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa80b3df0 con 0x7f4fa8071360 2023-08-27T11:27:21.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.503+0000 7f4faf569700 1 -- 172.21.15.112:0/1956546870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa815d8e0 con 0x7f4fa8071360 2023-08-27T11:27:21.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.503+0000 7f4f9dffb700 1 -- 172.21.15.112:0/1956546870 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4f98007470 con 0x7f4fa8071360 2023-08-27T11:27:21.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.503+0000 7f4f9dffb700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4f8c06a4d0 0x7f4f8c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.503+0000 7f4f9dffb700 1 -- 172.21.15.112:0/1956546870 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4f98088410 con 0x7f4fa8071360 2023-08-27T11:27:21.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.504+0000 7f4facb04700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4f8c06a4d0 0x7f4f8c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.504+0000 7f4facb04700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4f8c06a4d0 0x7f4f8c06c990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009b90 tx=0x7f4fa4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.508+0000 7f4f9dffb700 1 -- 172.21.15.112:0/1956546870 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4f98053ac0 con 0x7f4fa8071360 2023-08-27T11:27:21.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.696+0000 7f4faf569700 1 -- 172.21.15.112:0/1956546870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4fa80b4010 con 0x7f4fa8071360 2023-08-27T11:27:21.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.696+0000 7f4f9dffb700 1 -- 172.21.15.112:0/1956546870 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f980570e0 con 0x7f4fa8071360 2023-08-27T11:27:21.698 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:27:21.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.698+0000 7f4faf569700 1 -- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4f8c06a4d0 msgr2=0x7f4f8c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.698+0000 7f4faf569700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4f8c06a4d0 0x7f4f8c06c990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009b90 tx=0x7f4fa4008030 comp rx=0 tx=0).stop 2023-08-27T11:27:21.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.698+0000 7f4faf569700 1 -- 172.21.15.112:0/1956546870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8071360 msgr2=0x7f4fa815fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.698+0000 7f4faf569700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8071360 0x7f4fa815fea0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800c520 tx=0x7f4f9800c8e0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.698+0000 7f4faf569700 1 -- 172.21.15.112:0/1956546870 shutdown_connections 2023-08-27T11:27:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.698+0000 7f4faf569700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4f8c06a4d0 0x7f4f8c06c990 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.698+0000 7f4faf569700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa815af90 0x7f4fa80cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.698+0000 7f4faf569700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8071360 0x7f4fa815fea0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.698+0000 7f4faf569700 1 --2- 172.21.15.112:0/1956546870 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa80d1a60 0x7f4fa81603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.698+0000 7f4faf569700 1 -- 172.21.15.112:0/1956546870 >> 172.21.15.112:0/1956546870 conn(0x7f4fa8066010 msgr2=0x7f4fa8159ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:21.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.698+0000 7f4faf569700 1 -- 172.21.15.112:0/1956546870 shutdown_connections 2023-08-27T11:27:21.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.698+0000 7f4faf569700 1 -- 172.21.15.112:0/1956546870 wait complete. 2023-08-27T11:27:21.711 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:27:21.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.832+0000 7effda501700 1 Processor -- start 2023-08-27T11:27:21.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.832+0000 7effda501700 1 -- start start 2023-08-27T11:27:21.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.833+0000 7effda501700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7effd4146c20 0x7effd4160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.833+0000 7effda501700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7effd40b3440 0x7effd415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.833+0000 7effda501700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd415e6e0 0x7effd406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.833+0000 7effda501700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd40b5070 con 0x7effd4146c20 2023-08-27T11:27:21.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.833+0000 7effda501700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd4147000 con 0x7effd40b3440 2023-08-27T11:27:21.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.833+0000 7effda501700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd41614a0 con 0x7effd415e6e0 2023-08-27T11:27:21.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.833+0000 7effd3fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7effd4146c20 0x7effd4160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.833+0000 7effd3fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7effd4146c20 0x7effd4160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34720/0 (socket says 172.21.15.112:34720) 2023-08-27T11:27:21.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.833+0000 7effd3fff700 1 -- 172.21.15.112:0/2878723289 learned_addr learned my addr 172.21.15.112:0/2878723289 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:21.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.833+0000 7effd3fff700 1 -- 172.21.15.112:0/2878723289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd415e6e0 msgr2=0x7effd406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:21.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.833+0000 7effd8a9e700 1 --2- 172.21.15.112:0/2878723289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd415e6e0 0x7effd406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.833+0000 7effd37fe700 1 --2- 172.21.15.112:0/2878723289 >> v2:172.21.15.138:3300/0 conn(0x7effd40b3440 0x7effd415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effd3fff700 1 --2- 172.21.15.112:0/2878723289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd415e6e0 0x7effd406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effd3fff700 1 -- 172.21.15.112:0/2878723289 >> v2:172.21.15.138:3300/0 conn(0x7effd40b3440 msgr2=0x7effd415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:21.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effd3fff700 1 --2- 172.21.15.112:0/2878723289 >> v2:172.21.15.138:3300/0 conn(0x7effd40b3440 0x7effd415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effd3fff700 1 -- 172.21.15.112:0/2878723289 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd4161740 con 0x7effd4146c20 2023-08-27T11:27:21.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effd8a9e700 1 --2- 172.21.15.112:0/2878723289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd415e6e0 0x7effd406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:21.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effd37fe700 1 --2- 172.21.15.112:0/2878723289 >> v2:172.21.15.138:3300/0 conn(0x7effd40b3440 0x7effd415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:21.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effd3fff700 1 --2- 172.21.15.112:0/2878723289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4146c20 0x7effd4160f60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7effd40b2d80 tx=0x7effbc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231974232bca226b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effd2ffd700 1 -- 172.21.15.112:0/2878723289 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc00b870 con 0x7effd4146c20 2023-08-27T11:27:21.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effd2ffd700 1 -- 172.21.15.112:0/2878723289 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc00b9d0 con 0x7effd4146c20 2023-08-27T11:27:21.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effd2ffd700 1 -- 172.21.15.112:0/2878723289 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc00bd60 con 0x7effd4146c20 2023-08-27T11:27:21.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effda501700 1 -- 172.21.15.112:0/2878723289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4146c20 msgr2=0x7effd4160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effda501700 1 --2- 172.21.15.112:0/2878723289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4146c20 0x7effd4160f60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7effd40b2d80 tx=0x7effbc009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effda501700 1 -- 172.21.15.112:0/2878723289 shutdown_connections 2023-08-27T11:27:21.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effda501700 1 --2- 172.21.15.112:0/2878723289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd415e6e0 0x7effd406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effda501700 1 --2- 172.21.15.112:0/2878723289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4146c20 0x7effd4160f60 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effda501700 1 --2- 172.21.15.112:0/2878723289 >> v2:172.21.15.138:3300/0 conn(0x7effd40b3440 0x7effd415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.834+0000 7effda501700 1 -- 172.21.15.112:0/2878723289 >> 172.21.15.112:0/2878723289 conn(0x7effd40b1e90 msgr2=0x7effd40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:21.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.835+0000 7effda501700 1 -- 172.21.15.112:0/2878723289 shutdown_connections 2023-08-27T11:27:21.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.835+0000 7effda501700 1 -- 172.21.15.112:0/2878723289 wait complete. 2023-08-27T11:27:21.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.836+0000 7effda501700 1 Processor -- start 2023-08-27T11:27:21.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.836+0000 7effda501700 1 -- start start 2023-08-27T11:27:21.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effda501700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd40b3440 0x7effd41f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effda501700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effd4146c20 0x7effd40c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effda501700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd415e6e0 0x7effd40c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effda501700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd41f4e30 con 0x7effd40b3440 2023-08-27T11:27:21.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effda501700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd41f3f80 con 0x7effd4146c20 2023-08-27T11:27:21.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effda501700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd41f5800 con 0x7effd415e6e0 2023-08-27T11:27:21.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd40b3440 0x7effd41f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd40b3440 0x7effd41f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34726/0 (socket says 172.21.15.112:34726) 2023-08-27T11:27:21.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd3fff700 1 -- 172.21.15.112:0/3775276703 learned_addr learned my addr 172.21.15.112:0/3775276703 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:21.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd37fe700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effd4146c20 0x7effd40c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd8a9e700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd415e6e0 0x7effd40c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd3fff700 1 -- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd415e6e0 msgr2=0x7effd40c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd3fff700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd415e6e0 0x7effd40c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd3fff700 1 -- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effd4146c20 msgr2=0x7effd40c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd3fff700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effd4146c20 0x7effd40c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd3fff700 1 -- 172.21.15.112:0/3775276703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc009550 con 0x7effd40b3440 2023-08-27T11:27:21.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd3fff700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd40b3440 0x7effd41f52c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7effbc014040 tx=0x7effbc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd0ff9700 1 -- 172.21.15.112:0/3775276703 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc004250 con 0x7effd40b3440 2023-08-27T11:27:21.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd0ff9700 1 -- 172.21.15.112:0/3775276703 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc01e070 con 0x7effd40b3440 2023-08-27T11:27:21.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effda501700 1 -- 172.21.15.112:0/3775276703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd41f4220 con 0x7effd40b3440 2023-08-27T11:27:21.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effd0ff9700 1 -- 172.21.15.112:0/3775276703 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc0036a0 con 0x7effd40b3440 2023-08-27T11:27:21.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.837+0000 7effda501700 1 -- 172.21.15.112:0/3775276703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd41f47e0 con 0x7effd40b3440 2023-08-27T11:27:21.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.838+0000 7effda501700 1 -- 172.21.15.112:0/3775276703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd41f5110 con 0x7effd40b3440 2023-08-27T11:27:21.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.839+0000 7effd0ff9700 1 -- 172.21.15.112:0/3775276703 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7effbc020030 con 0x7effd40b3440 2023-08-27T11:27:21.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.839+0000 7effd0ff9700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7effb406a4d0 0x7effb406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.839+0000 7effd0ff9700 1 -- 172.21.15.112:0/3775276703 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5296+0+0 (secure 0 0 0) 0x7effbc089900 con 0x7effd40b3440 2023-08-27T11:27:21.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.839+0000 7effd37fe700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7effb406a4d0 0x7effb406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.840+0000 7effd37fe700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7effb406a4d0 0x7effb406c990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7effd40b2900 tx=0x7effc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:21.843+0000 7effd0ff9700 1 -- 172.21.15.112:0/3775276703 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7effbc054e70 con 0x7effd40b3440 2023-08-27T11:27:22.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.032+0000 7effda501700 1 -- 172.21.15.112:0/3775276703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7effd4067e90 con 0x7effd40b3440 2023-08-27T11:27:22.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.036+0000 7effd0ff9700 1 -- 172.21.15.112:0/3775276703 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7effbc058490 con 0x7effd40b3440 2023-08-27T11:27:22.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.039+0000 7effda501700 1 -- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7effb406a4d0 msgr2=0x7effb406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.039+0000 7effda501700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7effb406a4d0 0x7effb406c990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7effd40b2900 tx=0x7effc4008040 comp rx=0 tx=0).stop 2023-08-27T11:27:22.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.039+0000 7effda501700 1 -- 172.21.15.112:0/3775276703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd40b3440 msgr2=0x7effd41f52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.039+0000 7effda501700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd40b3440 0x7effd41f52c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7effbc014040 tx=0x7effbc004750 comp rx=0 tx=0).stop 2023-08-27T11:27:22.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.039+0000 7effda501700 1 -- 172.21.15.112:0/3775276703 shutdown_connections 2023-08-27T11:27:22.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.039+0000 7effda501700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7effb406a4d0 0x7effb406c990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.039+0000 7effda501700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd415e6e0 0x7effd40c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.039+0000 7effda501700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd40b3440 0x7effd41f52c0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.039+0000 7effda501700 1 --2- 172.21.15.112:0/3775276703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effd4146c20 0x7effd40c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.039+0000 7effda501700 1 -- 172.21.15.112:0/3775276703 >> 172.21.15.112:0/3775276703 conn(0x7effd40b1e90 msgr2=0x7effd406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:22.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.039+0000 7effda501700 1 -- 172.21.15.112:0/3775276703 shutdown_connections 2023-08-27T11:27:22.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.039+0000 7effda501700 1 -- 172.21.15.112:0/3775276703 wait complete. 2023-08-27T11:27:22.052 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:27:22.053 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:27:22.088 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:27:22.138 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:27:22.138 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:27:22.139 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:22.156 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:27:22.156 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:27:22.157 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:22.176 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:27:22.176 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:27:22.177 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:22.213 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:27:22.213 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:27:22.213 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:22.234 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:27:22.235 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:27:22.235 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:22.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.371+0000 7f5aa7656700 1 Processor -- start 2023-08-27T11:27:22.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.371+0000 7f5aa7656700 1 -- start start 2023-08-27T11:27:22.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa7656700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5aa0160a00 0x7f5aa00b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa7656700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aa0146990 0x7f5aa015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa7656700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5aa006d510 0x7f5aa006d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa7656700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa006dec0 con 0x7f5aa0160a00 2023-08-27T11:27:22.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa7656700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa015e140 con 0x7f5aa006d510 2023-08-27T11:27:22.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa7656700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa006f930 con 0x7f5aa0146990 2023-08-27T11:27:22.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa53f2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5aa0160a00 0x7f5aa00b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa53f2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5aa0160a00 0x7f5aa00b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34734/0 (socket says 172.21.15.112:34734) 2023-08-27T11:27:22.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa53f2700 1 -- 172.21.15.112:0/1136171397 learned_addr learned my addr 172.21.15.112:0/1136171397 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:22.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa5bf3700 1 --2- 172.21.15.112:0/1136171397 >> v2:172.21.15.138:3300/0 conn(0x7f5aa006d510 0x7f5aa006d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa4bf1700 1 --2- 172.21.15.112:0/1136171397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aa0146990 0x7f5aa015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa53f2700 1 -- 172.21.15.112:0/1136171397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aa0146990 msgr2=0x7f5aa015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa53f2700 1 --2- 172.21.15.112:0/1136171397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aa0146990 0x7f5aa015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa53f2700 1 -- 172.21.15.112:0/1136171397 >> v2:172.21.15.138:3300/0 conn(0x7f5aa006d510 msgr2=0x7f5aa006d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa53f2700 1 --2- 172.21.15.112:0/1136171397 >> v2:172.21.15.138:3300/0 conn(0x7f5aa006d510 0x7f5aa006d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa53f2700 1 -- 172.21.15.112:0/1136171397 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa006fc10 con 0x7f5aa0160a00 2023-08-27T11:27:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.372+0000 7f5aa53f2700 1 --2- 172.21.15.112:0/1136171397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aa0160a00 0x7f5aa00b46d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f5aa006e180 tx=0x7f5a9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7581e66970b0cfa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.373+0000 7f5a97fff700 1 -- 172.21.15.112:0/1136171397 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c00b870 con 0x7f5aa0160a00 2023-08-27T11:27:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.373+0000 7f5a97fff700 1 -- 172.21.15.112:0/1136171397 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9c00b9d0 con 0x7f5aa0160a00 2023-08-27T11:27:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.373+0000 7f5a97fff700 1 -- 172.21.15.112:0/1136171397 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c00bd60 con 0x7f5aa0160a00 2023-08-27T11:27:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.373+0000 7f5aa7656700 1 -- 172.21.15.112:0/1136171397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aa0160a00 msgr2=0x7f5aa00b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.373+0000 7f5aa7656700 1 --2- 172.21.15.112:0/1136171397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aa0160a00 0x7f5aa00b46d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f5aa006e180 tx=0x7f5a9c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.373+0000 7f5aa7656700 1 -- 172.21.15.112:0/1136171397 shutdown_connections 2023-08-27T11:27:22.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.373+0000 7f5aa7656700 1 --2- 172.21.15.112:0/1136171397 >> v2:172.21.15.138:3300/0 conn(0x7f5aa006d510 0x7f5aa006d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.373+0000 7f5aa7656700 1 --2- 172.21.15.112:0/1136171397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aa0160a00 0x7f5aa00b46d0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.373+0000 7f5aa7656700 1 --2- 172.21.15.112:0/1136171397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aa0146990 0x7f5aa015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.373+0000 7f5aa7656700 1 -- 172.21.15.112:0/1136171397 >> 172.21.15.112:0/1136171397 conn(0x7f5aa00cfe50 msgr2=0x7f5aa00d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:22.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.373+0000 7f5aa7656700 1 -- 172.21.15.112:0/1136171397 shutdown_connections 2023-08-27T11:27:22.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.373+0000 7f5aa7656700 1 -- 172.21.15.112:0/1136171397 wait complete. 2023-08-27T11:27:22.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.374+0000 7f5aa7656700 1 Processor -- start 2023-08-27T11:27:22.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.374+0000 7f5aa7656700 1 -- start start 2023-08-27T11:27:22.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.375+0000 7f5aa7656700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aa006d510 0x7f5aa00ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.375+0000 7f5aa7656700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa0146990 0x7f5aa01f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.375+0000 7f5aa7656700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aa0160a00 0x7f5aa015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.375+0000 7f5aa7656700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00aee60 con 0x7f5aa0160a00 2023-08-27T11:27:22.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.375+0000 7f5aa7656700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa01f4220 con 0x7f5aa0146990 2023-08-27T11:27:22.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.375+0000 7f5aa7656700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa01f36f0 con 0x7f5aa006d510 2023-08-27T11:27:22.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.375+0000 7f5aa5bf3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aa0160a00 0x7f5aa015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.375+0000 7f5aa5bf3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aa0160a00 0x7f5aa015cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34744/0 (socket says 172.21.15.112:34744) 2023-08-27T11:27:22.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.375+0000 7f5aa5bf3700 1 -- 172.21.15.112:0/4124224097 learned_addr learned my addr 172.21.15.112:0/4124224097 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:22.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.375+0000 7f5aa4bf1700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa0146990 0x7f5aa01f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.376+0000 7f5aa5bf3700 1 -- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aa006d510 msgr2=0x7f5aa00ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.376+0000 7f5aa53f2700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aa006d510 0x7f5aa00ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.376+0000 7f5aa5bf3700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aa006d510 0x7f5aa00ae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.376+0000 7f5aa5bf3700 1 -- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa0146990 msgr2=0x7f5aa01f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.376+0000 7f5aa5bf3700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa0146990 0x7f5aa01f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.376+0000 7f5aa5bf3700 1 -- 172.21.15.112:0/4124224097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c009550 con 0x7f5aa0160a00 2023-08-27T11:27:22.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.376+0000 7f5aa4bf1700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa0146990 0x7f5aa01f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:22.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.376+0000 7f5aa5bf3700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aa0160a00 0x7f5aa015cf30 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5a98007e60 tx=0x7f5a9800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.376+0000 7f5a95ffb700 1 -- 172.21.15.112:0/4124224097 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800ede0 con 0x7f5aa0160a00 2023-08-27T11:27:22.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.376+0000 7f5a95ffb700 1 -- 172.21.15.112:0/4124224097 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a98014420 con 0x7f5aa0160a00 2023-08-27T11:27:22.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.376+0000 7f5aa7656700 1 -- 172.21.15.112:0/4124224097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa01f3980 con 0x7f5aa0160a00 2023-08-27T11:27:22.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.376+0000 7f5a95ffb700 1 -- 172.21.15.112:0/4124224097 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a980147d0 con 0x7f5aa0160a00 2023-08-27T11:27:22.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.376+0000 7f5aa7656700 1 -- 172.21.15.112:0/4124224097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa01f3f40 con 0x7f5aa0160a00 2023-08-27T11:27:22.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.377+0000 7f5aa7656700 1 -- 172.21.15.112:0/4124224097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa0067a20 con 0x7f5aa0160a00 2023-08-27T11:27:22.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.377+0000 7f5a95ffb700 1 -- 172.21.15.112:0/4124224097 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5a98014930 con 0x7f5aa0160a00 2023-08-27T11:27:22.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.378+0000 7f5a95ffb700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5a8406a4d0 0x7f5a8406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.378+0000 7f5a95ffb700 1 -- 172.21.15.112:0/4124224097 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5a980890b0 con 0x7f5aa0160a00 2023-08-27T11:27:22.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.378+0000 7f5aa53f2700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5a8406a4d0 0x7f5a8406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.378+0000 7f5aa53f2700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5a8406a4d0 0x7f5a8406c990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009c10 tx=0x7f5a9c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.382+0000 7f5a95ffb700 1 -- 172.21.15.112:0/4124224097 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5a98054550 con 0x7f5aa0160a00 2023-08-27T11:27:22.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.614+0000 7f5aa7656700 1 -- 172.21.15.112:0/4124224097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5aa006e740 con 0x7f5aa0160a00 2023-08-27T11:27:22.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.614+0000 7f5a95ffb700 1 -- 172.21.15.112:0/4124224097 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f5a98057b70 con 0x7f5aa0160a00 2023-08-27T11:27:22.617 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:22.617 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":70,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/37643970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":37643970},{"type":"v1","addr":"172.21.15.112:6834","nonce":37643970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/568782348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":568782348},{"type":"v1","addr":"172.21.15.138:6836","nonce":568782348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1867471713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1867471713},{"type":"v1","addr":"172.21.15.138:6837","nonce":1867471713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":7893,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2582091269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2582091269},{"type":"v1","addr":"172.21.15.112:6835","nonce":2582091269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[]} 2023-08-27T11:27:22.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.616+0000 7f5aa7656700 1 -- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5a8406a4d0 msgr2=0x7f5a8406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.616+0000 7f5aa7656700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5a8406a4d0 0x7f5a8406c990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009c10 tx=0x7f5a9c008030 comp rx=0 tx=0).stop 2023-08-27T11:27:22.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.616+0000 7f5aa7656700 1 -- 172.21.15.112:0/4124224097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aa0160a00 msgr2=0x7f5aa015cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.617+0000 7f5aa7656700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aa0160a00 0x7f5aa015cf30 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5a98007e60 tx=0x7f5a9800e3c0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.617+0000 7f5aa7656700 1 -- 172.21.15.112:0/4124224097 shutdown_connections 2023-08-27T11:27:22.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.617+0000 7f5aa7656700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5a8406a4d0 0x7f5a8406c990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.617+0000 7f5aa7656700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aa0160a00 0x7f5aa015cf30 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.617+0000 7f5aa7656700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aa006d510 0x7f5aa00ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.617+0000 7f5aa7656700 1 --2- 172.21.15.112:0/4124224097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa0146990 0x7f5aa01f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.617+0000 7f5aa7656700 1 -- 172.21.15.112:0/4124224097 >> 172.21.15.112:0/4124224097 conn(0x7f5aa00cfe50 msgr2=0x7f5aa0145d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:22.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.617+0000 7f5aa7656700 1 -- 172.21.15.112:0/4124224097 shutdown_connections 2023-08-27T11:27:22.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.617+0000 7f5aa7656700 1 -- 172.21.15.112:0/4124224097 wait complete. 2023-08-27T11:27:22.621 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 70 2023-08-27T11:27:22.630 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:27:22.631 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:27:22.631 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:27:22.631 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:27:22.631 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:27:22.632 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:27:22.632 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:27:22.632 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:27:22.633 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:27:22.633 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:27:22.633 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:27:22.633 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:27:22.634 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:27:22.634 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:27:22.634 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:27:22.635 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:27:22.639 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:27:22.640 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:27:22.641 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:27:22.641 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:27:22.646 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:27:22.646 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:27:22.647 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:27:22.648 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:27:22.683 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:27:22.684 DEBUG:teuthology.parallel:result is None 2023-08-27T11:27:22.684 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:27:22.685 DEBUG:teuthology.parallel:result is None 2023-08-27T11:27:22.688 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:27:22.689 DEBUG:teuthology.parallel:result is None 2023-08-27T11:27:22.689 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:27:22.690 DEBUG:teuthology.parallel:result is None 2023-08-27T11:27:22.691 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:27:22.762 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:27:22.763 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:27:22.773 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:27:22.774 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2023-08-27T11:27:22.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.816+0000 7fee14dc1700 1 Processor -- start 2023-08-27T11:27:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.816+0000 7fee14dc1700 1 -- start start 2023-08-27T11:27:22.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.817+0000 7fee14dc1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fee10159e30 0x7fee1006ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.817+0000 7fee14dc1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fee100cd490 0x7fee100cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.817+0000 7fee14dc1700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee100708a0 con 0x7fee1015a210 2023-08-27T11:27:22.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.817+0000 7fee14dc1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee100cde40 con 0x7fee10159e30 2023-08-27T11:27:22.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.817+0000 7fee14dc1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1013dfc0 con 0x7fee100cd490 2023-08-27T11:27:22.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.817+0000 7fee0e59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fee10159e30 0x7fee1006ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.817+0000 7fee0e59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fee10159e30 0x7fee1006ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34774/0 (socket says 172.21.15.112:34774) 2023-08-27T11:27:22.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.817+0000 7fee0e59c700 1 -- 172.21.15.112:0/381746449 learned_addr learned my addr 172.21.15.112:0/381746449 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.817+0000 7fee0ed9d700 1 --2- 172.21.15.112:0/381746449 >> v2:172.21.15.138:3300/0 conn(0x7fee100cd490 0x7fee100cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee0e59c700 1 -- 172.21.15.112:0/381746449 >> v2:172.21.15.138:3300/0 conn(0x7fee100cd490 msgr2=0x7fee100cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee0e59c700 1 --2- 172.21.15.112:0/381746449 >> v2:172.21.15.138:3300/0 conn(0x7fee100cd490 0x7fee100cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee0e59c700 1 -- 172.21.15.112:0/381746449 >> v1:172.21.15.138:6789/0 conn(0x7fee1015a210 legacy=0x7fee1013d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee0e59c700 1 -- 172.21.15.112:0/381746449 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee10070af0 con 0x7fee10159e30 2023-08-27T11:27:22.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee0e59c700 1 --2- 172.21.15.112:0/381746449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee10159e30 0x7fee1006ddb0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fee1006cd30 tx=0x7fee00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4334207719e4669 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee0d59a700 1 -- 172.21.15.112:0/381746449 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00004230 con 0x7fee10159e30 2023-08-27T11:27:22.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee0d59a700 1 -- 172.21.15.112:0/381746449 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee00004390 con 0x7fee10159e30 2023-08-27T11:27:22.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee0d59a700 1 -- 172.21.15.112:0/381746449 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee000036a0 con 0x7fee10159e30 2023-08-27T11:27:22.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee14dc1700 1 -- 172.21.15.112:0/381746449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee10159e30 msgr2=0x7fee1006ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee14dc1700 1 --2- 172.21.15.112:0/381746449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee10159e30 0x7fee1006ddb0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fee1006cd30 tx=0x7fee00009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee14dc1700 1 -- 172.21.15.112:0/381746449 shutdown_connections 2023-08-27T11:27:22.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee14dc1700 1 --2- 172.21.15.112:0/381746449 >> v2:172.21.15.138:3300/0 conn(0x7fee100cd490 0x7fee100cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee14dc1700 1 --2- 172.21.15.112:0/381746449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee10159e30 0x7fee1006ddb0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee14dc1700 1 -- 172.21.15.112:0/381746449 >> 172.21.15.112:0/381746449 conn(0x7fee100b1380 msgr2=0x7fee100b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee14dc1700 1 -- 172.21.15.112:0/381746449 shutdown_connections 2023-08-27T11:27:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.818+0000 7fee14dc1700 1 -- 172.21.15.112:0/381746449 wait complete. 2023-08-27T11:27:22.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.820+0000 7fee14dc1700 1 Processor -- start 2023-08-27T11:27:22.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.820+0000 7fee14dc1700 1 -- start start 2023-08-27T11:27:22.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee14dc1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee100cd490 0x7fee100ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee14dc1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee10159e30 0x7fee1013d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee14dc1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1015a210 0x7fee100ab570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee14dc1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee100aefa0 con 0x7fee1015a210 2023-08-27T11:27:22.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee14dc1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee100af280 con 0x7fee100cd490 2023-08-27T11:27:22.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee14dc1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee100af560 con 0x7fee10159e30 2023-08-27T11:27:22.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee0ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1015a210 0x7fee100ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee0e59c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee100cd490 0x7fee100ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee0ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1015a210 0x7fee100ab570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34782/0 (socket says 172.21.15.112:34782) 2023-08-27T11:27:22.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee0ed9d700 1 -- 172.21.15.112:0/3802303691 learned_addr learned my addr 172.21.15.112:0/3802303691 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:22.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee0e59c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee100cd490 0x7fee100ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:35204/0 (socket says 172.21.15.112:35204) 2023-08-27T11:27:22.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee0dd9b700 1 --2- 172.21.15.112:0/3802303691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee10159e30 0x7fee1013d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee0ed9d700 1 -- 172.21.15.112:0/3802303691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee10159e30 msgr2=0x7fee1013d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee0ed9d700 1 --2- 172.21.15.112:0/3802303691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee10159e30 0x7fee1013d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee0ed9d700 1 -- 172.21.15.112:0/3802303691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee100cd490 msgr2=0x7fee100ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee0ed9d700 1 --2- 172.21.15.112:0/3802303691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee100cd490 0x7fee100ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee0ed9d700 1 -- 172.21.15.112:0/3802303691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee00009550 con 0x7fee1015a210 2023-08-27T11:27:22.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee0ed9d700 1 --2- 172.21.15.112:0/3802303691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1015a210 0x7fee100ab570 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fee08007e40 tx=0x7fee0800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fedfeffd700 1 -- 172.21.15.112:0/3802303691 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0800ed30 con 0x7fee1015a210 2023-08-27T11:27:22.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fedfeffd700 1 -- 172.21.15.112:0/3802303691 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee08014420 con 0x7fee1015a210 2023-08-27T11:27:22.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fedfeffd700 1 -- 172.21.15.112:0/3802303691 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee080147d0 con 0x7fee1015a210 2023-08-27T11:27:22.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee14dc1700 1 -- 172.21.15.112:0/3802303691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee100af840 con 0x7fee1015a210 2023-08-27T11:27:22.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.821+0000 7fee14dc1700 1 -- 172.21.15.112:0/3802303691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1013d540 con 0x7fee1015a210 2023-08-27T11:27:22.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.822+0000 7fee14dc1700 1 -- 172.21.15.112:0/3802303691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee100af6a0 con 0x7fee1015a210 2023-08-27T11:27:22.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.823+0000 7fedfeffd700 1 -- 172.21.15.112:0/3802303691 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fee08014930 con 0x7fee1015a210 2023-08-27T11:27:22.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.823+0000 7fedfeffd700 1 --2- 172.21.15.112:0/3802303691 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedf006a4d0 0x7fedf006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.823+0000 7fedfeffd700 1 -- 172.21.15.112:0/3802303691 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fee08089000 con 0x7fee1015a210 2023-08-27T11:27:22.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.823+0000 7fee0e59c700 1 --2- 172.21.15.112:0/3802303691 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedf006a4d0 0x7fedf006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.824+0000 7fee0e59c700 1 --2- 172.21.15.112:0/3802303691 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedf006a4d0 0x7fedf006c990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fee00009130 tx=0x7fee00008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:22.827+0000 7fedfeffd700 1 -- 172.21.15.112:0/3802303691 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fee100af6a0 con 0x7fee1015a210 2023-08-27T11:27:23.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:23.031+0000 7fee14dc1700 1 -- 172.21.15.112:0/3802303691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fee100afd60 con 0x7fee1015a210 2023-08-27T11:27:24.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.055+0000 7fedfeffd700 1 -- 172.21.15.112:0/3802303691 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v110) v1 ==== 97+0+0 (secure 0 0 0) 0x7fee100afd60 con 0x7fee1015a210 2023-08-27T11:27:24.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.058+0000 7fee14dc1700 1 -- 172.21.15.112:0/3802303691 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedf006a4d0 msgr2=0x7fedf006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.058+0000 7fee14dc1700 1 --2- 172.21.15.112:0/3802303691 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedf006a4d0 0x7fedf006c990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fee00009130 tx=0x7fee00008030 comp rx=0 tx=0).stop 2023-08-27T11:27:24.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.058+0000 7fee14dc1700 1 -- 172.21.15.112:0/3802303691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1015a210 msgr2=0x7fee100ab570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.058+0000 7fee14dc1700 1 --2- 172.21.15.112:0/3802303691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1015a210 0x7fee100ab570 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fee08007e40 tx=0x7fee0800e3c0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.058+0000 7fee14dc1700 1 -- 172.21.15.112:0/3802303691 shutdown_connections 2023-08-27T11:27:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.058+0000 7fee14dc1700 1 --2- 172.21.15.112:0/3802303691 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fedf006a4d0 0x7fedf006c990 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.058+0000 7fee14dc1700 1 --2- 172.21.15.112:0/3802303691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1015a210 0x7fee100ab570 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.058+0000 7fee14dc1700 1 --2- 172.21.15.112:0/3802303691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee100cd490 0x7fee100ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.058+0000 7fee14dc1700 1 --2- 172.21.15.112:0/3802303691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee10159e30 0x7fee1013d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.058+0000 7fee14dc1700 1 -- 172.21.15.112:0/3802303691 >> 172.21.15.112:0/3802303691 conn(0x7fee100b1380 msgr2=0x7fee1015c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:24.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.058+0000 7fee14dc1700 1 -- 172.21.15.112:0/3802303691 shutdown_connections 2023-08-27T11:27:24.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.058+0000 7fee14dc1700 1 -- 172.21.15.112:0/3802303691 wait complete. 2023-08-27T11:27:24.062 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2023-08-27T11:27:24.074 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:27:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.196+0000 7f10caef2700 1 Processor -- start 2023-08-27T11:27:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.196+0000 7f10caef2700 1 -- start start 2023-08-27T11:27:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.197+0000 7f10caef2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10c40b17e0 0x7f10c40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.197+0000 7f10caef2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10c4161ce0 0x7f10c406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.197+0000 7f10caef2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c4161850 con 0x7f10c40b1400 2023-08-27T11:27:24.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.197+0000 7f10caef2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c40cfc10 con 0x7f10c4161ce0 2023-08-27T11:27:24.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.197+0000 7f10caef2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c4161200 con 0x7f10c40b17e0 2023-08-27T11:27:24.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.197+0000 7f10c8c8e700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f10c40b1400 0x7f10c406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57404/0 (socket says 172.21.15.112:57404) 2023-08-27T11:27:24.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.197+0000 7f10c8c8e700 1 -- 172.21.15.112:0/2780796713 learned_addr learned my addr 172.21.15.112:0/2780796713 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:24.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.197+0000 7f10c3fff700 1 --2- 172.21.15.112:0/2780796713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10c40b17e0 0x7f10c40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.197+0000 7f10c948f700 1 --2- 172.21.15.112:0/2780796713 >> v2:172.21.15.138:3300/0 conn(0x7f10c4161ce0 0x7f10c406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.197+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701397817 0 0) 0x7f10c4161850 con 0x7f10c40b1400 2023-08-27T11:27:24.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.197+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10b40035b0 con 0x7f10c40b1400 2023-08-27T11:27:24.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1410694798 0 0) 0x7f10b40035b0 con 0x7f10c40b1400 2023-08-27T11:27:24.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10c40b17e0 msgr2=0x7f10c40cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:24.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 --2- 172.21.15.112:0/2780796713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10c40b17e0 0x7f10c40cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 >> v2:172.21.15.138:3300/0 conn(0x7f10c4161ce0 msgr2=0x7f10c406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:24.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 --2- 172.21.15.112:0/2780796713 >> v2:172.21.15.138:3300/0 conn(0x7f10c4161ce0 0x7f10c406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c41614e0 con 0x7f10c40b1400 2023-08-27T11:27:24.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f10b8001670 con 0x7f10c40b1400 2023-08-27T11:27:24.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 >> v1:172.21.15.112:6789/0 conn(0x7f10c40b1400 legacy=0x7f10c406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 --2- 172.21.15.112:0/2780796713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b4003a00 0x7f10b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 --2- 172.21.15.112:0/2780796713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10b4004850 0x7f10b4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 --2- 172.21.15.112:0/2780796713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10c40b17e0 0x7f10b4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40cfc10 con 0x7f10b4003a00 2023-08-27T11:27:24.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c4161200 con 0x7f10b4004850 2023-08-27T11:27:24.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b400ba10 con 0x7f10c40b17e0 2023-08-27T11:27:24.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c8c8e700 1 --2- 172.21.15.112:0/2780796713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b4003a00 0x7f10b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c948f700 1 --2- 172.21.15.112:0/2780796713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10c40b17e0 0x7f10b4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c3fff700 1 --2- 172.21.15.112:0/2780796713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10b4004850 0x7f10b4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c8c8e700 1 -- 172.21.15.112:0/2780796713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10c40b17e0 msgr2=0x7f10b4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:24.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c8c8e700 1 --2- 172.21.15.112:0/2780796713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10c40b17e0 0x7f10b4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c8c8e700 1 -- 172.21.15.112:0/2780796713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10b4004850 msgr2=0x7f10b4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:24.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c8c8e700 1 --2- 172.21.15.112:0/2780796713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10b4004850 0x7f10b4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c8c8e700 1 -- 172.21.15.112:0/2780796713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b40035b0 con 0x7f10b4003a00 2023-08-27T11:27:24.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c8c8e700 1 --2- 172.21.15.112:0/2780796713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b4003a00 0x7f10b4003e20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f10b800c350 tx=0x7f10b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec0811d05baf06b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:24.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b800ce50 con 0x7f10b4003a00 2023-08-27T11:27:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b8005960 con 0x7f10b4003a00 2023-08-27T11:27:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.198+0000 7f10c37fe700 1 -- 172.21.15.112:0/2780796713 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8005d10 con 0x7f10b4003a00 2023-08-27T11:27:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.199+0000 7f10caef2700 1 -- 172.21.15.112:0/2780796713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b4003a00 msgr2=0x7f10b4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.199+0000 7f10caef2700 1 --2- 172.21.15.112:0/2780796713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b4003a00 0x7f10b4003e20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f10b800c350 tx=0x7f10b4005880 comp rx=0 tx=0).stop 2023-08-27T11:27:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.199+0000 7f10c948f700 1 -- 172.21.15.112:0/2780796713 reap_dead start 2023-08-27T11:27:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.199+0000 7f10caef2700 1 -- 172.21.15.112:0/2780796713 shutdown_connections 2023-08-27T11:27:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.199+0000 7f10caef2700 1 -- 172.21.15.112:0/2780796713 >> 172.21.15.112:0/2780796713 conn(0x7f10c40d0000 msgr2=0x7f10c40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.199+0000 7f10caef2700 1 -- 172.21.15.112:0/2780796713 shutdown_connections 2023-08-27T11:27:24.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.199+0000 7f10caef2700 1 -- 172.21.15.112:0/2780796713 wait complete. 2023-08-27T11:27:24.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.200+0000 7f10caef2700 1 Processor -- start 2023-08-27T11:27:24.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.200+0000 7f10caef2700 1 -- start start 2023-08-27T11:27:24.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.201+0000 7f10caef2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10b4003a00 0x7f10c414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.201+0000 7f10caef2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10c414c400 0x7f10c414d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.201+0000 7f10caef2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c414fb80 0x7f10c4150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.201+0000 7f10caef2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c414db70 con 0x7f10c414fb80 2023-08-27T11:27:24.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.201+0000 7f10caef2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c414ddc0 con 0x7f10c414c400 2023-08-27T11:27:24.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.201+0000 7f10caef2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c414e0a0 con 0x7f10b4003a00 2023-08-27T11:27:24.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.201+0000 7f10c948f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c414fb80 0x7f10c4150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.201+0000 7f10c948f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c414fb80 0x7f10c4150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34806/0 (socket says 172.21.15.112:34806) 2023-08-27T11:27:24.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.201+0000 7f10c948f700 1 -- 172.21.15.112:0/2042631717 learned_addr learned my addr 172.21.15.112:0/2042631717 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:24.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.202+0000 7f10c948f700 1 -- 172.21.15.112:0/2042631717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10b4003a00 msgr2=0x7f10c414bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.202+0000 7f10c948f700 1 --2- 172.21.15.112:0/2042631717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10b4003a00 0x7f10c414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.202+0000 7f10c948f700 1 -- 172.21.15.112:0/2042631717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10c414c400 msgr2=0x7f10c414d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.202+0000 7f10c948f700 1 --2- 172.21.15.112:0/2042631717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10c414c400 0x7f10c414d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.202+0000 7f10c948f700 1 -- 172.21.15.112:0/2042631717 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c414e380 con 0x7f10c414fb80 2023-08-27T11:27:24.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.202+0000 7f10c948f700 1 --2- 172.21.15.112:0/2042631717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c414fb80 0x7f10c4150010 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f10c406df40 tx=0x7f10bc011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:24.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.202+0000 7f10c17fa700 1 -- 172.21.15.112:0/2042631717 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc00e040 con 0x7f10c414fb80 2023-08-27T11:27:24.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.202+0000 7f10c17fa700 1 -- 172.21.15.112:0/2042631717 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10bc00f040 con 0x7f10c414fb80 2023-08-27T11:27:24.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.202+0000 7f10caef2700 1 -- 172.21.15.112:0/2042631717 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c40708d0 con 0x7f10c414fb80 2023-08-27T11:27:24.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.202+0000 7f10c17fa700 1 -- 172.21.15.112:0/2042631717 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc00fdd0 con 0x7f10c414fb80 2023-08-27T11:27:24.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.202+0000 7f10caef2700 1 -- 172.21.15.112:0/2042631717 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c414d7a0 con 0x7f10c414fb80 2023-08-27T11:27:24.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.203+0000 7f10caef2700 1 -- 172.21.15.112:0/2042631717 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c4070ac0 con 0x7f10c414fb80 2023-08-27T11:27:24.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.203+0000 7f10c17fa700 1 -- 172.21.15.112:0/2042631717 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f10bc015430 con 0x7f10c414fb80 2023-08-27T11:27:24.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.204+0000 7f10c17fa700 1 --2- 172.21.15.112:0/2042631717 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10a806a4d0 0x7f10a806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.204+0000 7f10c17fa700 1 -- 172.21.15.112:0/2042631717 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f10bc088f50 con 0x7f10c414fb80 2023-08-27T11:27:24.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.204+0000 7f10c8c8e700 1 --2- 172.21.15.112:0/2042631717 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10a806a4d0 0x7f10a806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.204+0000 7f10c8c8e700 1 --2- 172.21.15.112:0/2042631717 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10a806a4d0 0x7f10a806c990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f10b8000c00 tx=0x7f10b800f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:24.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.208+0000 7f10c17fa700 1 -- 172.21.15.112:0/2042631717 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f10bc054330 con 0x7f10c414fb80 2023-08-27T11:27:24.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.474+0000 7f10caef2700 1 -- 172.21.15.112:0/2042631717 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f10c4067a20 con 0x7f10c414fb80 2023-08-27T11:27:24.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.475+0000 7f10c17fa700 1 -- 172.21.15.112:0/2042631717 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f10bc057950 con 0x7f10c414fb80 2023-08-27T11:27:24.477 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:24.477 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:27:24.477 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2023-08-27T11:27:24.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.478 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2023-08-27T11:27:24.478 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk2bnkLBAA+vxcTeJzucKk9iSF/Zw3Lg==", 2023-08-27T11:27:24.478 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.478 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:27:24.478 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:27:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:27:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2023-08-27T11:27:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk3zD5OBAASVnwG6yn/YQzfYyeMakFpA==", 2023-08-27T11:27:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:27:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:27:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:27:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2023-08-27T11:27:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtki7wJNBAAsIGgDLM82sWNcKnCDlSBMw==", 2023-08-27T11:27:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:27:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:27:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:27:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2023-08-27T11:27:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkCsW/BBAAFTPiTOpBo1L+rLo9px+AjQ==", 2023-08-27T11:27:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:27:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:27:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:27:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2023-08-27T11:27:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB7MOtkR/cSHhAAR8e8VRd+e0ESJkqQDYWbEw==", 2023-08-27T11:27:24.486 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.486 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:24.486 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:27:24.486 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.486 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2023-08-27T11:27:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB+MOtksBF3HRAA1OIXiIkhgPDYpQyYPaWOXg==", 2023-08-27T11:27:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:27:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.489 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2023-08-27T11:27:24.489 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCBMOtkWsrdHBAAkfWbiE6wAFzpt9+BxRh8fw==", 2023-08-27T11:27:24.489 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.489 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:24.489 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:27:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2023-08-27T11:27:24.491 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCEMOtkIkiKHRAARDoh/FH9ZdqUlZZ29Mc/Dw==", 2023-08-27T11:27:24.491 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.491 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:24.491 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:27:24.491 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2023-08-27T11:27:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCKMOtkPgDzGxAA+U4kbwD0d4lS8aNPG2CgHw==", 2023-08-27T11:27:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:27:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2023-08-27T11:27:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCNMOtkKomSGxAAGAQUcIH2H/uyZIU8vlq1pw==", 2023-08-27T11:27:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:27:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2023-08-27T11:27:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCQMOtkHI2MGhAA7sgbi8JXO8P9wnVvMy3PqA==", 2023-08-27T11:27:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:27:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.497 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.497 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2023-08-27T11:27:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCTMOtkVHpiGhAA4C4YJMWbQ0mmGgB4i8G/AA==", 2023-08-27T11:27:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:27:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.499 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.499 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2023-08-27T11:27:24.499 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA==", 2023-08-27T11:27:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:27:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2023-08-27T11:27:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:27:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2023-08-27T11:27:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtk66DbDxAAWZ5StELux0lriJ2gEN9icg==", 2023-08-27T11:27:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.502 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:27:24.502 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:27:24.502 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:27:24.502 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:27:24.502 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2023-08-27T11:27:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkViKJDBAAF2ofwO+78er9OyFfn2bO+g==", 2023-08-27T11:27:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:27:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:27:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:27:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:27:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.505 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2023-08-27T11:27:24.505 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkwt+QFRAAowWE69VqeRhQz0896rxx9Q==", 2023-08-27T11:27:24.505 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.505 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:27:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:27:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:27:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:27:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.507 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2023-08-27T11:27:24.507 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkjDA+DxAAjmOn82AC2OMpKlAVCD6hlg==", 2023-08-27T11:27:24.507 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.507 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:27:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:27:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:27:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.509 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2023-08-27T11:27:24.509 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkXguHKRAALHwzk8diVXqsMvTKr+AzLQ==", 2023-08-27T11:27:24.509 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.509 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:27:24.509 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:27:24.510 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.510 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:24.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:24.510 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2023-08-27T11:27:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkrRjBJRAAq3ftmIh81LKadpliVUiJ6g==", 2023-08-27T11:27:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:27:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:27:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:27:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:27:24.512 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.512 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:24.512 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:27:24.512 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:27:24.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.477+0000 7f10caef2700 1 -- 172.21.15.112:0/2042631717 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10a806a4d0 msgr2=0x7f10a806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.477+0000 7f10caef2700 1 --2- 172.21.15.112:0/2042631717 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10a806a4d0 0x7f10a806c990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f10b8000c00 tx=0x7f10b800f000 comp rx=0 tx=0).stop 2023-08-27T11:27:24.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.477+0000 7f10caef2700 1 -- 172.21.15.112:0/2042631717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c414fb80 msgr2=0x7f10c4150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.477+0000 7f10caef2700 1 --2- 172.21.15.112:0/2042631717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c414fb80 0x7f10c4150010 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f10c406df40 tx=0x7f10bc011f80 comp rx=0 tx=0).stop 2023-08-27T11:27:24.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.477+0000 7f10caef2700 1 -- 172.21.15.112:0/2042631717 shutdown_connections 2023-08-27T11:27:24.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.477+0000 7f10caef2700 1 --2- 172.21.15.112:0/2042631717 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10a806a4d0 0x7f10a806c990 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.477+0000 7f10caef2700 1 --2- 172.21.15.112:0/2042631717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c414fb80 0x7f10c4150010 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.477+0000 7f10caef2700 1 --2- 172.21.15.112:0/2042631717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10b4003a00 0x7f10c414bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.477+0000 7f10caef2700 1 --2- 172.21.15.112:0/2042631717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10c414c400 0x7f10c414d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.477+0000 7f10caef2700 1 -- 172.21.15.112:0/2042631717 >> 172.21.15.112:0/2042631717 conn(0x7f10c40d0000 msgr2=0x7f10c415fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:24.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.477+0000 7f10caef2700 1 -- 172.21.15.112:0/2042631717 shutdown_connections 2023-08-27T11:27:24.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.477+0000 7f10caef2700 1 -- 172.21.15.112:0/2042631717 wait complete. 2023-08-27T11:27:24.516 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:24.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.636+0000 7f9f4678f700 1 Processor -- start 2023-08-27T11:27:24.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.636+0000 7f9f4678f700 1 -- start start 2023-08-27T11:27:24.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.636+0000 7f9f4678f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f40143a60 0x7f9f400cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.636+0000 7f9f4678f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f400b4960 0x7f9f4015db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.636+0000 7f9f4678f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f40143e40 con 0x7f9f400b4de0 2023-08-27T11:27:24.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.636+0000 7f9f4678f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f400b5340 con 0x7f9f400b4960 2023-08-27T11:27:24.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.636+0000 7f9f4678f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400ce9a0 con 0x7f9f40143a60 2023-08-27T11:27:24.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.637+0000 7f9f3ffff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f40143a60 0x7f9f400cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.637+0000 7f9f3f7fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f400b4960 0x7f9f4015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.637+0000 7f9f3f7fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f400b4960 0x7f9f4015db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:35238/0 (socket says 172.21.15.112:35238) 2023-08-27T11:27:24.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.637+0000 7f9f3ffff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f40143a60 0x7f9f400cf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:46464/0 (socket says 172.21.15.112:46464) 2023-08-27T11:27:24.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.637+0000 7f9f3f7fe700 1 -- 172.21.15.112:0/2229012526 learned_addr learned my addr 172.21.15.112:0/2229012526 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:24.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.637+0000 7f9f3f7fe700 1 -- 172.21.15.112:0/2229012526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f40143a60 msgr2=0x7f9f400cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:24.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.637+0000 7f9f3f7fe700 1 --2- 172.21.15.112:0/2229012526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f40143a60 0x7f9f400cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.637+0000 7f9f3f7fe700 1 -- 172.21.15.112:0/2229012526 >> v1:172.21.15.138:6789/0 conn(0x7f9f400b4de0 legacy=0x7f9f4015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.637+0000 7f9f3f7fe700 1 -- 172.21.15.112:0/2229012526 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f400cec80 con 0x7f9f400b4960 2023-08-27T11:27:24.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.637+0000 7f9f3ffff700 1 --2- 172.21.15.112:0/2229012526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f40143a60 0x7f9f400cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:24.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.637+0000 7f9f3f7fe700 1 --2- 172.21.15.112:0/2229012526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f400b4960 0x7f9f4015db80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006fd0 tx=0x7f9f30009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a3306662f1dffb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:24.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.637+0000 7f9f3effd700 1 -- 172.21.15.112:0/2229012526 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30005e30 con 0x7f9f400b4960 2023-08-27T11:27:24.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.638+0000 7f9f3effd700 1 -- 172.21.15.112:0/2229012526 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f30004030 con 0x7f9f400b4960 2023-08-27T11:27:24.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.638+0000 7f9f3effd700 1 -- 172.21.15.112:0/2229012526 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f300043e0 con 0x7f9f400b4960 2023-08-27T11:27:24.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.638+0000 7f9f4678f700 1 -- 172.21.15.112:0/2229012526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f400b4960 msgr2=0x7f9f4015db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.638+0000 7f9f4678f700 1 --2- 172.21.15.112:0/2229012526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f400b4960 0x7f9f4015db80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006fd0 tx=0x7f9f30009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.638+0000 7f9f4678f700 1 -- 172.21.15.112:0/2229012526 shutdown_connections 2023-08-27T11:27:24.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.638+0000 7f9f4678f700 1 --2- 172.21.15.112:0/2229012526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f40143a60 0x7f9f400cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.638+0000 7f9f4678f700 1 --2- 172.21.15.112:0/2229012526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f400b4960 0x7f9f4015db80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.638+0000 7f9f4678f700 1 -- 172.21.15.112:0/2229012526 >> 172.21.15.112:0/2229012526 conn(0x7f9f4006adb0 msgr2=0x7f9f4006b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:24.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.638+0000 7f9f4678f700 1 -- 172.21.15.112:0/2229012526 shutdown_connections 2023-08-27T11:27:24.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.638+0000 7f9f4678f700 1 -- 172.21.15.112:0/2229012526 wait complete. 2023-08-27T11:27:24.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.639+0000 7f9f4678f700 1 Processor -- start 2023-08-27T11:27:24.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.640+0000 7f9f4678f700 1 -- start start 2023-08-27T11:27:24.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.640+0000 7f9f4678f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f400b4960 0x7f9f40167080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.640+0000 7f9f4678f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f400b4de0 0x7f9f40154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.640+0000 7f9f4678f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f40143a60 0x7f9f40167810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.640+0000 7f9f4678f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f40141f80 con 0x7f9f400b4960 2023-08-27T11:27:24.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.640+0000 7f9f4678f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f40142220 con 0x7f9f400b4de0 2023-08-27T11:27:24.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.640+0000 7f9f4678f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f40142500 con 0x7f9f40143a60 2023-08-27T11:27:24.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.640+0000 7f9f3ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f400b4960 0x7f9f40167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.640+0000 7f9f3ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f400b4960 0x7f9f40167080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34816/0 (socket says 172.21.15.112:34816) 2023-08-27T11:27:24.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.640+0000 7f9f3ffff700 1 -- 172.21.15.112:0/2853880490 learned_addr learned my addr 172.21.15.112:0/2853880490 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:24.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f44d2c700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f40143a60 0x7f9f40167810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f3f7fe700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f400b4de0 0x7f9f40154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f3ffff700 1 -- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f40143a60 msgr2=0x7f9f40167810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f3ffff700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f40143a60 0x7f9f40167810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f3ffff700 1 -- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f400b4de0 msgr2=0x7f9f40154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f3ffff700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f400b4de0 0x7f9f40154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f3ffff700 1 -- 172.21.15.112:0/2853880490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f28009430 con 0x7f9f400b4960 2023-08-27T11:27:24.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f3f7fe700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f400b4de0 0x7f9f40154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:24.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f3ffff700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f400b4960 0x7f9f40167080 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800ebe0 tx=0x7f9f28009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:24.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f3cff9700 1 -- 172.21.15.112:0/2853880490 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f28005a20 con 0x7f9f400b4960 2023-08-27T11:27:24.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f3cff9700 1 -- 172.21.15.112:0/2853880490 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f28005b80 con 0x7f9f400b4960 2023-08-27T11:27:24.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f4678f700 1 -- 172.21.15.112:0/2853880490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f30009500 con 0x7f9f400b4960 2023-08-27T11:27:24.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f3cff9700 1 -- 172.21.15.112:0/2853880490 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2800c8b0 con 0x7f9f400b4960 2023-08-27T11:27:24.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.641+0000 7f9f4678f700 1 -- 172.21.15.112:0/2853880490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f40167d50 con 0x7f9f400b4960 2023-08-27T11:27:24.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.642+0000 7f9f4678f700 1 -- 172.21.15.112:0/2853880490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4013dec0 con 0x7f9f400b4960 2023-08-27T11:27:24.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.642+0000 7f9f3cff9700 1 -- 172.21.15.112:0/2853880490 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f9f2800ca10 con 0x7f9f400b4960 2023-08-27T11:27:24.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.643+0000 7f9f3cff9700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9f2006a4d0 0x7f9f2006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.643+0000 7f9f3cff9700 1 -- 172.21.15.112:0/2853880490 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9f28088b80 con 0x7f9f400b4960 2023-08-27T11:27:24.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.643+0000 7f9f3f7fe700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9f2006a4d0 0x7f9f2006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.643+0000 7f9f3f7fe700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9f2006a4d0 0x7f9f2006c990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006fd0 tx=0x7f9f30008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:24.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.647+0000 7f9f3cff9700 1 -- 172.21.15.112:0/2853880490 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9f28054070 con 0x7f9f400b4960 2023-08-27T11:27:24.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.837+0000 7f9f4678f700 1 -- 172.21.15.112:0/2853880490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f40142a20 con 0x7f9f400b4960 2023-08-27T11:27:24.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.838+0000 7f9f3cff9700 1 -- 172.21.15.112:0/2853880490 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+14615 (secure 0 0 0) 0x7f9f28020090 con 0x7f9f400b4960 2023-08-27T11:27:24.840 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:24.840 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":110,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:27:24.050054+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:24.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.840+0000 7f9f4678f700 1 -- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9f2006a4d0 msgr2=0x7f9f2006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.840+0000 7f9f4678f700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9f2006a4d0 0x7f9f2006c990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006fd0 tx=0x7f9f30008070 comp rx=0 tx=0).stop 2023-08-27T11:27:24.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.840+0000 7f9f4678f700 1 -- 172.21.15.112:0/2853880490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f400b4960 msgr2=0x7f9f40167080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.840+0000 7f9f4678f700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f400b4960 0x7f9f40167080 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800ebe0 tx=0x7f9f28009e00 comp rx=0 tx=0).stop 2023-08-27T11:27:24.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.840+0000 7f9f4678f700 1 -- 172.21.15.112:0/2853880490 shutdown_connections 2023-08-27T11:27:24.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.840+0000 7f9f4678f700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9f2006a4d0 0x7f9f2006c990 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.840+0000 7f9f4678f700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f40143a60 0x7f9f40167810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.840+0000 7f9f4678f700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f400b4960 0x7f9f40167080 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.840+0000 7f9f4678f700 1 --2- 172.21.15.112:0/2853880490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f400b4de0 0x7f9f40154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.840+0000 7f9f4678f700 1 -- 172.21.15.112:0/2853880490 >> 172.21.15.112:0/2853880490 conn(0x7f9f4006adb0 msgr2=0x7f9f4006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:24.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.840+0000 7f9f4678f700 1 -- 172.21.15.112:0/2853880490 shutdown_connections 2023-08-27T11:27:24.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.840+0000 7f9f4678f700 1 -- 172.21.15.112:0/2853880490 wait complete. 2023-08-27T11:27:24.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:24.855 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.976+0000 7fc0a76dd700 1 Processor -- start 2023-08-27T11:27:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.976+0000 7fc0a76dd700 1 -- start start 2023-08-27T11:27:24.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.977+0000 7fc0a76dd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0a0143ba0 0x7fc0a00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.977+0000 7fc0a76dd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc0a006d5b0 0x7fc0a006da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.977+0000 7fc0a76dd700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a00cfd50 con 0x7fc0a01437c0 2023-08-27T11:27:24.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.977+0000 7fc0a76dd700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a006df40 con 0x7fc0a006d5b0 2023-08-27T11:27:24.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.977+0000 7fc0a76dd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a00cf140 con 0x7fc0a0143ba0 2023-08-27T11:27:24.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.977+0000 7fc0a5c7a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc0a006d5b0 0x7fc0a006da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.977+0000 7fc0a5c7a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc0a006d5b0 0x7fc0a006da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34828/0 (socket says 172.21.15.112:34828) 2023-08-27T11:27:24.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.977+0000 7fc0a5c7a700 1 -- 172.21.15.112:0/516345372 learned_addr learned my addr 172.21.15.112:0/516345372 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:24.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.978+0000 7fc0a4c78700 1 --2- 172.21.15.112:0/516345372 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0a0143ba0 0x7fc0a00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.978+0000 7fc0a5c7a700 1 -- 172.21.15.112:0/516345372 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0a0143ba0 msgr2=0x7fc0a00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:24.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.978+0000 7fc0a5c7a700 1 --2- 172.21.15.112:0/516345372 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0a0143ba0 0x7fc0a00b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.978+0000 7fc0a5c7a700 1 -- 172.21.15.112:0/516345372 >> v1:172.21.15.138:6789/0 conn(0x7fc0a01437c0 legacy=0x7fc0a00ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.978+0000 7fc0a5c7a700 1 -- 172.21.15.112:0/516345372 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a00cf420 con 0x7fc0a006d5b0 2023-08-27T11:27:24.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.978+0000 7fc0a5c7a700 1 --2- 172.21.15.112:0/516345372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006d5b0 0x7fc0a006da00 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fc098009e50 tx=0x7fc09800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2341ba285342ef06 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:24.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.978+0000 7fc097fff700 1 -- 172.21.15.112:0/516345372 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800e070 con 0x7fc0a006d5b0 2023-08-27T11:27:24.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.978+0000 7fc097fff700 1 -- 172.21.15.112:0/516345372 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09800b9c0 con 0x7fc0a006d5b0 2023-08-27T11:27:24.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.978+0000 7fc097fff700 1 -- 172.21.15.112:0/516345372 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800bd70 con 0x7fc0a006d5b0 2023-08-27T11:27:24.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.979+0000 7fc0a76dd700 1 -- 172.21.15.112:0/516345372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006d5b0 msgr2=0x7fc0a006da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.979+0000 7fc0a76dd700 1 --2- 172.21.15.112:0/516345372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006d5b0 0x7fc0a006da00 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fc098009e50 tx=0x7fc09800b330 comp rx=0 tx=0).stop 2023-08-27T11:27:24.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.979+0000 7fc0a76dd700 1 -- 172.21.15.112:0/516345372 shutdown_connections 2023-08-27T11:27:24.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.979+0000 7fc0a76dd700 1 --2- 172.21.15.112:0/516345372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006d5b0 0x7fc0a006da00 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.979+0000 7fc0a76dd700 1 --2- 172.21.15.112:0/516345372 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0a0143ba0 0x7fc0a00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.979+0000 7fc0a76dd700 1 -- 172.21.15.112:0/516345372 >> 172.21.15.112:0/516345372 conn(0x7fc0a00b1e90 msgr2=0x7fc0a00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:24.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.979+0000 7fc0a76dd700 1 -- 172.21.15.112:0/516345372 shutdown_connections 2023-08-27T11:27:24.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.979+0000 7fc0a76dd700 1 -- 172.21.15.112:0/516345372 wait complete. 2023-08-27T11:27:24.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.980+0000 7fc0a76dd700 1 Processor -- start 2023-08-27T11:27:24.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.980+0000 7fc0a76dd700 1 -- start start 2023-08-27T11:27:24.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a76dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006d5b0 0x7fc0a0158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a76dd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0a01437c0 0x7fc0a0140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a76dd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0a0143ba0 0x7fc0a0141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a76dd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0139de0 con 0x7fc0a006d5b0 2023-08-27T11:27:24.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a76dd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0158d50 con 0x7fc0a01437c0 2023-08-27T11:27:24.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a76dd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a01391d0 con 0x7fc0a0143ba0 2023-08-27T11:27:24.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a5479700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006d5b0 0x7fc0a0158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a5479700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006d5b0 0x7fc0a0158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34830/0 (socket says 172.21.15.112:34830) 2023-08-27T11:27:24.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a5479700 1 -- 172.21.15.112:0/3146116694 learned_addr learned my addr 172.21.15.112:0/3146116694 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:24.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a5c7a700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0a0143ba0 0x7fc0a0141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a4c78700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0a01437c0 0x7fc0a0140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a5479700 1 -- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0a0143ba0 msgr2=0x7fc0a0141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a5479700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0a0143ba0 0x7fc0a0141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a5479700 1 -- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0a01437c0 msgr2=0x7fc0a0140bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a5479700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0a01437c0 0x7fc0a0140bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a5479700 1 -- 172.21.15.112:0/3146116694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc098009b30 con 0x7fc0a006d5b0 2023-08-27T11:27:24.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a5479700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006d5b0 0x7fc0a0158810 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc09000c4e0 tx=0x7fc09000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:24.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc095ffb700 1 -- 172.21.15.112:0/3146116694 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0900052d0 con 0x7fc0a006d5b0 2023-08-27T11:27:24.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc095ffb700 1 -- 172.21.15.112:0/3146116694 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc090005430 con 0x7fc0a006d5b0 2023-08-27T11:27:24.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc0a76dd700 1 -- 172.21.15.112:0/3146116694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a01394b0 con 0x7fc0a006d5b0 2023-08-27T11:27:24.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.981+0000 7fc095ffb700 1 -- 172.21.15.112:0/3146116694 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0900038c0 con 0x7fc0a006d5b0 2023-08-27T11:27:24.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.982+0000 7fc0a76dd700 1 -- 172.21.15.112:0/3146116694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a0139a70 con 0x7fc0a006d5b0 2023-08-27T11:27:24.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.982+0000 7fc0a76dd700 1 -- 172.21.15.112:0/3146116694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a015a970 con 0x7fc0a006d5b0 2023-08-27T11:27:24.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.983+0000 7fc095ffb700 1 -- 172.21.15.112:0/3146116694 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fc090003a20 con 0x7fc0a006d5b0 2023-08-27T11:27:24.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.983+0000 7fc095ffb700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc08406a500 0x7fc08406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.983+0000 7fc095ffb700 1 -- 172.21.15.112:0/3146116694 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc090087d50 con 0x7fc0a006d5b0 2023-08-27T11:27:24.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.984+0000 7fc0a4c78700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc08406a500 0x7fc08406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.984+0000 7fc0a4c78700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc08406a500 0x7fc08406c9c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc0a013d0f0 tx=0x7fc09c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:24.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:24.988+0000 7fc095ffb700 1 -- 172.21.15.112:0/3146116694 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc0a015a970 con 0x7fc0a006d5b0 2023-08-27T11:27:25.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.177+0000 7fc0a76dd700 1 -- 172.21.15.112:0/3146116694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc0a015a970 con 0x7fc0a006d5b0 2023-08-27T11:27:25.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.177+0000 7fc095ffb700 1 -- 172.21.15.112:0/3146116694 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v110) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc0a015a970 con 0x7fc0a006d5b0 2023-08-27T11:27:25.179 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:27:25.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.179+0000 7fc0a76dd700 1 -- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc08406a500 msgr2=0x7fc08406c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.179+0000 7fc0a76dd700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc08406a500 0x7fc08406c9c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc0a013d0f0 tx=0x7fc09c008040 comp rx=0 tx=0).stop 2023-08-27T11:27:25.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.179+0000 7fc0a76dd700 1 -- 172.21.15.112:0/3146116694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006d5b0 msgr2=0x7fc0a0158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.179+0000 7fc0a76dd700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006d5b0 0x7fc0a0158810 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc09000c4e0 tx=0x7fc09000c7f0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.179+0000 7fc0a76dd700 1 -- 172.21.15.112:0/3146116694 shutdown_connections 2023-08-27T11:27:25.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.179+0000 7fc0a76dd700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc08406a500 0x7fc08406c9c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.179+0000 7fc0a76dd700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0a0143ba0 0x7fc0a0141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.179+0000 7fc0a76dd700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0a006d5b0 0x7fc0a0158810 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.179+0000 7fc0a76dd700 1 --2- 172.21.15.112:0/3146116694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0a01437c0 0x7fc0a0140bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.179+0000 7fc0a76dd700 1 -- 172.21.15.112:0/3146116694 >> 172.21.15.112:0/3146116694 conn(0x7fc0a00b1e90 msgr2=0x7fc0a0070350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:25.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.179+0000 7fc0a76dd700 1 -- 172.21.15.112:0/3146116694 shutdown_connections 2023-08-27T11:27:25.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.179+0000 7fc0a76dd700 1 -- 172.21.15.112:0/3146116694 wait complete. 2023-08-27T11:27:25.193 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:25.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.314+0000 7fe3e9bcf700 1 Processor -- start 2023-08-27T11:27:25.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.314+0000 7fe3e9bcf700 1 -- start start 2023-08-27T11:27:25.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.315+0000 7fe3e9bcf700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3e4143ba0 0x7fe3e40b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.315+0000 7fe3e9bcf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3e406d5b0 0x7fe3e406da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.315+0000 7fe3e9bcf700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e40cfd50 con 0x7fe3e41437c0 2023-08-27T11:27:25.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.315+0000 7fe3e9bcf700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e406df80 con 0x7fe3e4143ba0 2023-08-27T11:27:25.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.315+0000 7fe3e9bcf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40cf140 con 0x7fe3e406d5b0 2023-08-27T11:27:25.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.315+0000 7fe3e2ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3e4143ba0 0x7fe3e40b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.315+0000 7fe3e2ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3e4143ba0 0x7fe3e40b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34840/0 (socket says 172.21.15.112:34840) 2023-08-27T11:27:25.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.315+0000 7fe3e2ffd700 1 -- 172.21.15.112:0/543180730 learned_addr learned my addr 172.21.15.112:0/543180730 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:25.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.316+0000 7fe3e3fff700 1 --2- 172.21.15.112:0/543180730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3e406d5b0 0x7fe3e406da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.316+0000 7fe3e2ffd700 1 -- 172.21.15.112:0/543180730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3e406d5b0 msgr2=0x7fe3e406da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:25.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.316+0000 7fe3e2ffd700 1 --2- 172.21.15.112:0/543180730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3e406d5b0 0x7fe3e406da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.316+0000 7fe3e2ffd700 1 -- 172.21.15.112:0/543180730 >> v1:172.21.15.138:6789/0 conn(0x7fe3e41437c0 legacy=0x7fe3e40ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.316+0000 7fe3e2ffd700 1 -- 172.21.15.112:0/543180730 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e40cf420 con 0x7fe3e4143ba0 2023-08-27T11:27:25.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.316+0000 7fe3e3fff700 1 --2- 172.21.15.112:0/543180730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3e406d5b0 0x7fe3e406da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:25.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.316+0000 7fe3e2ffd700 1 --2- 172.21.15.112:0/543180730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3e4143ba0 0x7fe3e40b4670 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400bb90 tx=0x7fe3d400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8305a5b5c9538 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:25.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.316+0000 7fe3e27fc700 1 -- 172.21.15.112:0/543180730 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400dd50 con 0x7fe3e4143ba0 2023-08-27T11:27:25.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.316+0000 7fe3e27fc700 1 -- 172.21.15.112:0/543180730 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d4010470 con 0x7fe3e4143ba0 2023-08-27T11:27:25.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.316+0000 7fe3e27fc700 1 -- 172.21.15.112:0/543180730 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4010800 con 0x7fe3e4143ba0 2023-08-27T11:27:25.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.316+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/543180730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3e4143ba0 msgr2=0x7fe3e40b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.316+0000 7fe3e9bcf700 1 --2- 172.21.15.112:0/543180730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3e4143ba0 0x7fe3e40b4670 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400bb90 tx=0x7fe3d400bea0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.317+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/543180730 shutdown_connections 2023-08-27T11:27:25.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.317+0000 7fe3e9bcf700 1 --2- 172.21.15.112:0/543180730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3e406d5b0 0x7fe3e406da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.317+0000 7fe3e9bcf700 1 --2- 172.21.15.112:0/543180730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3e4143ba0 0x7fe3e40b4670 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.317+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/543180730 >> 172.21.15.112:0/543180730 conn(0x7fe3e40b1e90 msgr2=0x7fe3e40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:25.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.317+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/543180730 shutdown_connections 2023-08-27T11:27:25.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.317+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/543180730 wait complete. 2023-08-27T11:27:25.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.318+0000 7fe3e9bcf700 1 Processor -- start 2023-08-27T11:27:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.318+0000 7fe3e9bcf700 1 -- start start 2023-08-27T11:27:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e9bcf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3e406d5b0 0x7fe3e4159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e9bcf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3e41437c0 0x7fe3e413a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e9bcf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3e4143ba0 0x7fe3e415cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e9bcf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4138c30 con 0x7fe3e406d5b0 2023-08-27T11:27:25.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e9bcf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4138f10 con 0x7fe3e41437c0 2023-08-27T11:27:25.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e9bcf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e41391f0 con 0x7fe3e4143ba0 2023-08-27T11:27:25.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3e406d5b0 0x7fe3e4159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3e406d5b0 0x7fe3e4159ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34844/0 (socket says 172.21.15.112:34844) 2023-08-27T11:27:25.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e37fe700 1 -- 172.21.15.112:0/306489420 learned_addr learned my addr 172.21.15.112:0/306489420 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:25.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e2ffd700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3e41437c0 0x7fe3e413a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e3fff700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3e4143ba0 0x7fe3e415cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e37fe700 1 -- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3e4143ba0 msgr2=0x7fe3e415cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e37fe700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3e4143ba0 0x7fe3e415cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e37fe700 1 -- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3e41437c0 msgr2=0x7fe3e413a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e37fe700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3e41437c0 0x7fe3e413a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e37fe700 1 -- 172.21.15.112:0/306489420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d400b7f0 con 0x7fe3e406d5b0 2023-08-27T11:27:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e2ffd700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3e41437c0 0x7fe3e413a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e37fe700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3e406d5b0 0x7fe3e4159ae0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800c4e0 tx=0x7fe3d800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3cbfff700 1 -- 172.21.15.112:0/306489420 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80052d0 con 0x7fe3e406d5b0 2023-08-27T11:27:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3cbfff700 1 -- 172.21.15.112:0/306489420 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d8005430 con 0x7fe3e406d5b0 2023-08-27T11:27:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/306489420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e413c8b0 con 0x7fe3e406d5b0 2023-08-27T11:27:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3cbfff700 1 -- 172.21.15.112:0/306489420 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80038c0 con 0x7fe3e406d5b0 2023-08-27T11:27:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.319+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/306489420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e413a950 con 0x7fe3e406d5b0 2023-08-27T11:27:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.320+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/306489420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e4067e90 con 0x7fe3e406d5b0 2023-08-27T11:27:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.321+0000 7fe3cbfff700 1 -- 172.21.15.112:0/306489420 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe3d8003a20 con 0x7fe3e406d5b0 2023-08-27T11:27:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.321+0000 7fe3cbfff700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3d006a4d0 0x7fe3d006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.321+0000 7fe3cbfff700 1 -- 172.21.15.112:0/306489420 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe3d8087e40 con 0x7fe3e406d5b0 2023-08-27T11:27:25.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.321+0000 7fe3e2ffd700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3d006a4d0 0x7fe3d006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.322+0000 7fe3e2ffd700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3d006a4d0 0x7fe3d006c990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400bee0 tx=0x7fe3d4013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:25.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.325+0000 7fe3cbfff700 1 -- 172.21.15.112:0/306489420 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe3d80532a0 con 0x7fe3e406d5b0 2023-08-27T11:27:25.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.513+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/306489420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe3e4067a20 con 0x7fe3e406d5b0 2023-08-27T11:27:25.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.513+0000 7fe3cbfff700 1 -- 172.21.15.112:0/306489420 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3d8017020 con 0x7fe3e406d5b0 2023-08-27T11:27:25.515 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:27:25.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.515+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3d006a4d0 msgr2=0x7fe3d006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.515+0000 7fe3e9bcf700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3d006a4d0 0x7fe3d006c990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400bee0 tx=0x7fe3d4013000 comp rx=0 tx=0).stop 2023-08-27T11:27:25.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.515+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/306489420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3e406d5b0 msgr2=0x7fe3e4159ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.515+0000 7fe3e9bcf700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3e406d5b0 0x7fe3e4159ae0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800c4e0 tx=0x7fe3d800c8a0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.515+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/306489420 shutdown_connections 2023-08-27T11:27:25.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.515+0000 7fe3e9bcf700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3d006a4d0 0x7fe3d006c990 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.515+0000 7fe3e9bcf700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3e4143ba0 0x7fe3e415cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.515+0000 7fe3e9bcf700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3e406d5b0 0x7fe3e4159ae0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.515+0000 7fe3e9bcf700 1 --2- 172.21.15.112:0/306489420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3e41437c0 0x7fe3e413a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.515+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/306489420 >> 172.21.15.112:0/306489420 conn(0x7fe3e40b1e90 msgr2=0x7fe3e406fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:25.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.515+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/306489420 shutdown_connections 2023-08-27T11:27:25.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.515+0000 7fe3e9bcf700 1 -- 172.21.15.112:0/306489420 wait complete. 2023-08-27T11:27:25.529 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:27:25.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.647+0000 7fdb088c4700 1 Processor -- start 2023-08-27T11:27:25.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.647+0000 7fdb088c4700 1 -- start start 2023-08-27T11:27:25.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.648+0000 7fdb088c4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdb040b1400 0x7fdb040cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.648+0000 7fdb088c4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb040b17e0 0x7fdb0406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.648+0000 7fdb088c4700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb04161850 con 0x7fdb04161ce0 2023-08-27T11:27:25.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.648+0000 7fdb088c4700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0406e7d0 con 0x7fdb040b1400 2023-08-27T11:27:25.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.648+0000 7fdb088c4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb040cfc10 con 0x7fdb040b17e0 2023-08-27T11:27:25.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb0259c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdb040b1400 0x7fdb040cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb0259c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdb040b1400 0x7fdb040cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:35278/0 (socket says 172.21.15.112:35278) 2023-08-27T11:27:25.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb0259c700 1 -- 172.21.15.112:0/553303956 learned_addr learned my addr 172.21.15.112:0/553303956 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:25.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb02d9d700 1 --1- 172.21.15.112:0/553303956 >> v1:172.21.15.138:6789/0 conn(0x7fdb04161ce0 0x7fdb0406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:36762/0 (socket says 172.21.15.112:36762) 2023-08-27T11:27:25.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb01d9b700 1 --2- 172.21.15.112:0/553303956 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb040b17e0 0x7fdb0406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb0259c700 1 -- 172.21.15.112:0/553303956 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb040b17e0 msgr2=0x7fdb0406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:25.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb0259c700 1 --2- 172.21.15.112:0/553303956 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb040b17e0 0x7fdb0406e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb0259c700 1 -- 172.21.15.112:0/553303956 >> v1:172.21.15.138:6789/0 conn(0x7fdb04161ce0 legacy=0x7fdb0406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb0159a700 1 -- 172.21.15.112:0/553303956 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 173858180 0 0) 0x7fdb04161850 con 0x7fdb04161ce0 2023-08-27T11:27:25.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb0259c700 1 -- 172.21.15.112:0/553303956 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb04161200 con 0x7fdb040b1400 2023-08-27T11:27:25.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb01d9b700 1 --2- 172.21.15.112:0/553303956 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb040b17e0 0x7fdb0406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:25.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb0259c700 1 --2- 172.21.15.112:0/553303956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb040b1400 0x7fdb040cf5a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80098f0 tx=0x7fdaf8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05a138372096521 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:25.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb0159a700 1 -- 172.21.15.112:0/553303956 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8011830 con 0x7fdb040b1400 2023-08-27T11:27:25.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb0159a700 1 -- 172.21.15.112:0/553303956 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf8011990 con 0x7fdb040b1400 2023-08-27T11:27:25.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.649+0000 7fdb0159a700 1 -- 172.21.15.112:0/553303956 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8011d20 con 0x7fdb040b1400 2023-08-27T11:27:25.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.650+0000 7fdb088c4700 1 -- 172.21.15.112:0/553303956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb040b1400 msgr2=0x7fdb040cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.650+0000 7fdb088c4700 1 --2- 172.21.15.112:0/553303956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb040b1400 0x7fdb040cf5a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80098f0 tx=0x7fdaf8009c00 comp rx=0 tx=0).stop 2023-08-27T11:27:25.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.650+0000 7fdb088c4700 1 -- 172.21.15.112:0/553303956 shutdown_connections 2023-08-27T11:27:25.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.650+0000 7fdb088c4700 1 --2- 172.21.15.112:0/553303956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb040b1400 0x7fdb040cf5a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.650+0000 7fdb088c4700 1 --2- 172.21.15.112:0/553303956 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb040b17e0 0x7fdb0406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.650+0000 7fdb088c4700 1 -- 172.21.15.112:0/553303956 >> 172.21.15.112:0/553303956 conn(0x7fdb040d0000 msgr2=0x7fdb040d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:25.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.650+0000 7fdb088c4700 1 -- 172.21.15.112:0/553303956 shutdown_connections 2023-08-27T11:27:25.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.650+0000 7fdb088c4700 1 -- 172.21.15.112:0/553303956 wait complete. 2023-08-27T11:27:25.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.651+0000 7fdb088c4700 1 Processor -- start 2023-08-27T11:27:25.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.651+0000 7fdb088c4700 1 -- start start 2023-08-27T11:27:25.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.652+0000 7fdb088c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb040b1400 0x7fdb0414a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.652+0000 7fdb088c4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb040b17e0 0x7fdb040c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.652+0000 7fdb088c4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb04161ce0 0x7fdb040c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.652+0000 7fdb088c4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb040c8b30 con 0x7fdb040b1400 2023-08-27T11:27:25.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.652+0000 7fdb088c4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb040c7eb0 con 0x7fdb04161ce0 2023-08-27T11:27:25.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.652+0000 7fdb088c4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb040c8190 con 0x7fdb040b17e0 2023-08-27T11:27:25.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.652+0000 7fdb0259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb040b1400 0x7fdb0414a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.652+0000 7fdb0259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb040b1400 0x7fdb0414a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34858/0 (socket says 172.21.15.112:34858) 2023-08-27T11:27:25.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.652+0000 7fdb0259c700 1 -- 172.21.15.112:0/2554563161 learned_addr learned my addr 172.21.15.112:0/2554563161 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:25.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdb01d9b700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb040b17e0 0x7fdb040c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdb0259c700 1 -- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb040b17e0 msgr2=0x7fdb040c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdb0259c700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb040b17e0 0x7fdb040c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdb02d9d700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb04161ce0 0x7fdb040c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdb0259c700 1 -- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb04161ce0 msgr2=0x7fdb040c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdb0259c700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb04161ce0 0x7fdb040c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdb0259c700 1 -- 172.21.15.112:0/2554563161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf40094a0 con 0x7fdb040b1400 2023-08-27T11:27:25.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdb02d9d700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb04161ce0 0x7fdb040c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:25.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdb0259c700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb040b1400 0x7fdb0414a3d0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009c40 tx=0x7fdaf8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:25.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdaf2ffd700 1 -- 172.21.15.112:0/2554563161 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80044b0 con 0x7fdb040b1400 2023-08-27T11:27:25.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdaf2ffd700 1 -- 172.21.15.112:0/2554563161 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf801f070 con 0x7fdb040b1400 2023-08-27T11:27:25.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdb088c4700 1 -- 172.21.15.112:0/2554563161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf8009550 con 0x7fdb040b1400 2023-08-27T11:27:25.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdaf2ffd700 1 -- 172.21.15.112:0/2554563161 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80038e0 con 0x7fdb040b1400 2023-08-27T11:27:25.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.653+0000 7fdb088c4700 1 -- 172.21.15.112:0/2554563161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb040c8750 con 0x7fdb040b1400 2023-08-27T11:27:25.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.654+0000 7fdb088c4700 1 -- 172.21.15.112:0/2554563161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb04067a20 con 0x7fdb040b1400 2023-08-27T11:27:25.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.654+0000 7fdaf2ffd700 1 -- 172.21.15.112:0/2554563161 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fdaf8021030 con 0x7fdb040b1400 2023-08-27T11:27:25.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.655+0000 7fdaf2ffd700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdae406a4d0 0x7fdae406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.655+0000 7fdaf2ffd700 1 -- 172.21.15.112:0/2554563161 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdaf8089cd0 con 0x7fdb040b1400 2023-08-27T11:27:25.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.655+0000 7fdb01d9b700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdae406a4d0 0x7fdae406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.655+0000 7fdb01d9b700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdae406a4d0 0x7fdae406c990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdb041f5ff0 tx=0x7fdaf4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:25.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.659+0000 7fdaf2ffd700 1 -- 172.21.15.112:0/2554563161 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdaf8055250 con 0x7fdb040b1400 2023-08-27T11:27:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.852+0000 7fdb088c4700 1 -- 172.21.15.112:0/2554563161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdb0406db60 con 0x7fdb040b1400 2023-08-27T11:27:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.853+0000 7fdaf2ffd700 1 -- 172.21.15.112:0/2554563161 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v110) v1 ==== 84+0+126 (secure 0 0 0) 0x7fdaf8058870 con 0x7fdb040b1400 2023-08-27T11:27:25.855 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:25.855 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:27:25.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:25.855 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2023-08-27T11:27:25.855 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "rbd" 2023-08-27T11:27:25.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:25.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:25.856 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 2, 2023-08-27T11:27:25.856 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2023-08-27T11:27:25.856 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:25.857 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:27:25.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.855+0000 7fdb088c4700 1 -- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdae406a4d0 msgr2=0x7fdae406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.855+0000 7fdb088c4700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdae406a4d0 0x7fdae406c990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdb041f5ff0 tx=0x7fdaf4008040 comp rx=0 tx=0).stop 2023-08-27T11:27:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.855+0000 7fdb088c4700 1 -- 172.21.15.112:0/2554563161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb040b1400 msgr2=0x7fdb0414a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.855+0000 7fdb088c4700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb040b1400 0x7fdb0414a3d0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009c40 tx=0x7fdaf8004750 comp rx=0 tx=0).stop 2023-08-27T11:27:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.855+0000 7fdb088c4700 1 -- 172.21.15.112:0/2554563161 shutdown_connections 2023-08-27T11:27:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.855+0000 7fdb088c4700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdae406a4d0 0x7fdae406c990 secure :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdb041f5ff0 tx=0x7fdaf4008040 comp rx=0 tx=0).stop 2023-08-27T11:27:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.855+0000 7fdb088c4700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb04161ce0 0x7fdb040c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.855+0000 7fdb088c4700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb040b1400 0x7fdb0414a3d0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.855+0000 7fdb088c4700 1 --2- 172.21.15.112:0/2554563161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb040b17e0 0x7fdb040c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.855+0000 7fdb088c4700 1 -- 172.21.15.112:0/2554563161 >> 172.21.15.112:0/2554563161 conn(0x7fdb040d0000 msgr2=0x7fdb0415eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.855+0000 7fdb088c4700 1 -- 172.21.15.112:0/2554563161 shutdown_connections 2023-08-27T11:27:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.855+0000 7fdb088c4700 1 -- 172.21.15.112:0/2554563161 wait complete. 2023-08-27T11:27:25.868 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:27:25.869 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:27:25.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.991+0000 7fd522532700 1 Processor -- start 2023-08-27T11:27:25.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.991+0000 7fd522532700 1 -- start start 2023-08-27T11:27:25.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.992+0000 7fd522532700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd51c0b3de0 0x7fd51c15e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.992+0000 7fd522532700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd51c06f790 0x7fd51c06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.992+0000 7fd522532700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51c161390 con 0x7fd51c157ae0 2023-08-27T11:27:25.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.992+0000 7fd522532700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51c0b1aa0 con 0x7fd51c0b3de0 2023-08-27T11:27:25.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.992+0000 7fd522532700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c070180 con 0x7fd51c06f790 2023-08-27T11:27:25.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.992+0000 7fd520acf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd51c06f790 0x7fd51c06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.992+0000 7fd51b7fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd51c0b3de0 0x7fd51c15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.992+0000 7fd51bfff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd51c157ae0 0x7fd51c161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:36772/0 (socket says 172.21.15.112:36772) 2023-08-27T11:27:25.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.992+0000 7fd51bfff700 1 -- 172.21.15.112:0/964017954 learned_addr learned my addr 172.21.15.112:0/964017954 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:25.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.993+0000 7fd520acf700 1 -- 172.21.15.112:0/964017954 >> v2:172.21.15.138:3300/0 conn(0x7fd51c0b3de0 msgr2=0x7fd51c15e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:25.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.993+0000 7fd520acf700 1 --2- 172.21.15.112:0/964017954 >> v2:172.21.15.138:3300/0 conn(0x7fd51c0b3de0 0x7fd51c15e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.993+0000 7fd520acf700 1 -- 172.21.15.112:0/964017954 >> v1:172.21.15.138:6789/0 conn(0x7fd51c157ae0 legacy=0x7fd51c161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.993+0000 7fd520acf700 1 -- 172.21.15.112:0/964017954 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c0b35f0 con 0x7fd51c06f790 2023-08-27T11:27:25.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.993+0000 7fd51b7fe700 1 --2- 172.21.15.112:0/964017954 >> v2:172.21.15.138:3300/0 conn(0x7fd51c0b3de0 0x7fd51c15e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:25.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.993+0000 7fd520acf700 1 --2- 172.21.15.112:0/964017954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd51c06f790 0x7fd51c06fc40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd514009d90 tx=0x7fd51400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5abf29540d6951f6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:25.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.993+0000 7fd51affd700 1 -- 172.21.15.112:0/964017954 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51400e070 con 0x7fd51c06f790 2023-08-27T11:27:25.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.993+0000 7fd51affd700 1 -- 172.21.15.112:0/964017954 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51400bb20 con 0x7fd51c06f790 2023-08-27T11:27:25.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.993+0000 7fd51affd700 1 -- 172.21.15.112:0/964017954 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514012430 con 0x7fd51c06f790 2023-08-27T11:27:25.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.994+0000 7fd522532700 1 -- 172.21.15.112:0/964017954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd51c06f790 msgr2=0x7fd51c06fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.994+0000 7fd522532700 1 --2- 172.21.15.112:0/964017954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd51c06f790 0x7fd51c06fc40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd514009d90 tx=0x7fd51400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:25.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.994+0000 7fd522532700 1 -- 172.21.15.112:0/964017954 shutdown_connections 2023-08-27T11:27:25.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.994+0000 7fd522532700 1 --2- 172.21.15.112:0/964017954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd51c06f790 0x7fd51c06fc40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.994+0000 7fd522532700 1 --2- 172.21.15.112:0/964017954 >> v2:172.21.15.138:3300/0 conn(0x7fd51c0b3de0 0x7fd51c15e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.994+0000 7fd522532700 1 -- 172.21.15.112:0/964017954 >> 172.21.15.112:0/964017954 conn(0x7fd51c0b1be0 msgr2=0x7fd51c0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:26.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.994+0000 7fd522532700 1 -- 172.21.15.112:0/964017954 shutdown_connections 2023-08-27T11:27:26.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.994+0000 7fd522532700 1 -- 172.21.15.112:0/964017954 wait complete. 2023-08-27T11:27:26.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.995+0000 7fd522532700 1 Processor -- start 2023-08-27T11:27:26.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.995+0000 7fd522532700 1 -- start start 2023-08-27T11:27:26.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd522532700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd51c06f790 0x7fd51c0aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd522532700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd51c0b3de0 0x7fd51c145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd522532700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd51c157ae0 0x7fd51c146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd522532700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c1f36e0 con 0x7fd51c157ae0 2023-08-27T11:27:26.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd522532700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c1f2b60 con 0x7fd51c0b3de0 2023-08-27T11:27:26.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd522532700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c1f2e40 con 0x7fd51c06f790 2023-08-27T11:27:26.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd520acf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd51c157ae0 0x7fd51c146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd51bfff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd51c06f790 0x7fd51c0aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd520acf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd51c157ae0 0x7fd51c146110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34874/0 (socket says 172.21.15.112:34874) 2023-08-27T11:27:26.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd520acf700 1 -- 172.21.15.112:0/3006234214 learned_addr learned my addr 172.21.15.112:0/3006234214 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:26.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd51b7fe700 1 --2- 172.21.15.112:0/3006234214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd51c0b3de0 0x7fd51c145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd520acf700 1 -- 172.21.15.112:0/3006234214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd51c06f790 msgr2=0x7fd51c0aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd520acf700 1 --2- 172.21.15.112:0/3006234214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd51c06f790 0x7fd51c0aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd520acf700 1 -- 172.21.15.112:0/3006234214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd51c0b3de0 msgr2=0x7fd51c145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd520acf700 1 --2- 172.21.15.112:0/3006234214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd51c0b3de0 0x7fd51c145bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd520acf700 1 -- 172.21.15.112:0/3006234214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c0094a0 con 0x7fd51c157ae0 2023-08-27T11:27:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd520acf700 1 --2- 172.21.15.112:0/3006234214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd51c157ae0 0x7fd51c146110 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd514014040 tx=0x7fd514004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.996+0000 7fd518ff9700 1 -- 172.21.15.112:0/3006234214 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514012430 con 0x7fd51c157ae0 2023-08-27T11:27:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.997+0000 7fd522532700 1 -- 172.21.15.112:0/3006234214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd514009a70 con 0x7fd51c157ae0 2023-08-27T11:27:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.997+0000 7fd518ff9700 1 -- 172.21.15.112:0/3006234214 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd514020070 con 0x7fd51c157ae0 2023-08-27T11:27:26.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.997+0000 7fd518ff9700 1 -- 172.21.15.112:0/3006234214 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514003ea0 con 0x7fd51c157ae0 2023-08-27T11:27:26.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.997+0000 7fd522532700 1 -- 172.21.15.112:0/3006234214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c1f3400 con 0x7fd51c157ae0 2023-08-27T11:27:26.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.998+0000 7fd522532700 1 -- 172.21.15.112:0/3006234214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51c067a20 con 0x7fd51c157ae0 2023-08-27T11:27:26.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.998+0000 7fd518ff9700 1 -- 172.21.15.112:0/3006234214 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fd514004040 con 0x7fd51c157ae0 2023-08-27T11:27:26.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.998+0000 7fd518ff9700 1 --2- 172.21.15.112:0/3006234214 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd50806a4d0 0x7fd50806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.998+0000 7fd518ff9700 1 -- 172.21.15.112:0/3006234214 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd514022070 con 0x7fd51c157ae0 2023-08-27T11:27:26.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.999+0000 7fd51bfff700 1 --2- 172.21.15.112:0/3006234214 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd50806a4d0 0x7fd50806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:25.999+0000 7fd51bfff700 1 --2- 172.21.15.112:0/3006234214 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd50806a4d0 0x7fd50806c990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd510005910 tx=0x7fd510009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.003+0000 7fd518ff9700 1 -- 172.21.15.112:0/3006234214 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd514055a90 con 0x7fd51c157ae0 2023-08-27T11:27:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.193+0000 7fd522532700 1 -- 172.21.15.112:0/3006234214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd51c15ad40 con 0x7fd51c157ae0 2023-08-27T11:27:26.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.698+0000 7fd518ff9700 1 -- 172.21.15.112:0/3006234214 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v111) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd5140590b0 con 0x7fd51c157ae0 2023-08-27T11:27:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.702+0000 7fd522532700 1 -- 172.21.15.112:0/3006234214 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd50806a4d0 msgr2=0x7fd50806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.702+0000 7fd522532700 1 --2- 172.21.15.112:0/3006234214 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd50806a4d0 0x7fd50806c990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd510005910 tx=0x7fd510009040 comp rx=0 tx=0).stop 2023-08-27T11:27:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.702+0000 7fd522532700 1 -- 172.21.15.112:0/3006234214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd51c157ae0 msgr2=0x7fd51c146110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.702+0000 7fd522532700 1 --2- 172.21.15.112:0/3006234214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd51c157ae0 0x7fd51c146110 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd514014040 tx=0x7fd514004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.702+0000 7fd522532700 1 -- 172.21.15.112:0/3006234214 shutdown_connections 2023-08-27T11:27:26.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.702+0000 7fd522532700 1 --2- 172.21.15.112:0/3006234214 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd50806a4d0 0x7fd50806c990 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.702+0000 7fd522532700 1 --2- 172.21.15.112:0/3006234214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd51c157ae0 0x7fd51c146110 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.702+0000 7fd522532700 1 --2- 172.21.15.112:0/3006234214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd51c06f790 0x7fd51c0aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.702+0000 7fd522532700 1 --2- 172.21.15.112:0/3006234214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd51c0b3de0 0x7fd51c145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.702+0000 7fd522532700 1 -- 172.21.15.112:0/3006234214 >> 172.21.15.112:0/3006234214 conn(0x7fd51c0b1be0 msgr2=0x7fd51c15fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:26.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.702+0000 7fd522532700 1 -- 172.21.15.112:0/3006234214 shutdown_connections 2023-08-27T11:27:26.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.702+0000 7fd522532700 1 -- 172.21.15.112:0/3006234214 wait complete. 2023-08-27T11:27:26.706 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2023-08-27T11:27:26.717 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:27:26.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.844+0000 7f5d40eba700 1 Processor -- start 2023-08-27T11:27:26.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.844+0000 7f5d40eba700 1 -- start start 2023-08-27T11:27:26.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.845+0000 7f5d40eba700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5d3c143ba0 0x7f5d3c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.845+0000 7f5d40eba700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d3c06d5b0 0x7f5d3c06da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.845+0000 7f5d40eba700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c0cfd50 con 0x7f5d3c1437c0 2023-08-27T11:27:26.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.845+0000 7f5d40eba700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3c06df80 con 0x7f5d3c143ba0 2023-08-27T11:27:26.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.845+0000 7f5d40eba700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0cf140 con 0x7f5d3c06d5b0 2023-08-27T11:27:26.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.845+0000 7f5d3ad9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d3c06d5b0 0x7f5d3c06da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.845+0000 7f5d39d9b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5d3c143ba0 0x7f5d3c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.845+0000 7f5d3ad9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d3c06d5b0 0x7f5d3c06da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:46536/0 (socket says 172.21.15.112:46536) 2023-08-27T11:27:26.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.845+0000 7f5d3ad9d700 1 -- 172.21.15.112:0/4184551533 learned_addr learned my addr 172.21.15.112:0/4184551533 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:26.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.845+0000 7f5d3a59c700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5d3c1437c0 0x7f5d3c0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:36784/0 (socket says 172.21.15.112:36784) 2023-08-27T11:27:26.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.846+0000 7f5d3ad9d700 1 -- 172.21.15.112:0/4184551533 >> v2:172.21.15.138:3300/0 conn(0x7f5d3c143ba0 msgr2=0x7f5d3c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:26.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.846+0000 7f5d3ad9d700 1 --2- 172.21.15.112:0/4184551533 >> v2:172.21.15.138:3300/0 conn(0x7f5d3c143ba0 0x7f5d3c0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.846+0000 7f5d3ad9d700 1 -- 172.21.15.112:0/4184551533 >> v1:172.21.15.138:6789/0 conn(0x7f5d3c1437c0 legacy=0x7f5d3c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.846+0000 7f5d3ad9d700 1 -- 172.21.15.112:0/4184551533 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0cf420 con 0x7f5d3c06d5b0 2023-08-27T11:27:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.846+0000 7f5d39d9b700 1 --2- 172.21.15.112:0/4184551533 >> v2:172.21.15.138:3300/0 conn(0x7f5d3c143ba0 0x7f5d3c0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.846+0000 7f5d3ad9d700 1 --2- 172.21.15.112:0/4184551533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d3c06d5b0 0x7f5d3c06da40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009e50 tx=0x7f5d3400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb764cd8b9880f35 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.846+0000 7f5d3959a700 1 -- 172.21.15.112:0/4184551533 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400e070 con 0x7f5d3c06d5b0 2023-08-27T11:27:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.846+0000 7f5d3959a700 1 -- 172.21.15.112:0/4184551533 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3400b910 con 0x7f5d3c06d5b0 2023-08-27T11:27:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.846+0000 7f5d3959a700 1 -- 172.21.15.112:0/4184551533 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400bcc0 con 0x7f5d3c06d5b0 2023-08-27T11:27:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.847+0000 7f5d40eba700 1 -- 172.21.15.112:0/4184551533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d3c06d5b0 msgr2=0x7f5d3c06da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.847+0000 7f5d40eba700 1 --2- 172.21.15.112:0/4184551533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d3c06d5b0 0x7f5d3c06da40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009e50 tx=0x7f5d3400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.847+0000 7f5d40eba700 1 -- 172.21.15.112:0/4184551533 shutdown_connections 2023-08-27T11:27:26.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.847+0000 7f5d40eba700 1 --2- 172.21.15.112:0/4184551533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d3c06d5b0 0x7f5d3c06da40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.847+0000 7f5d40eba700 1 --2- 172.21.15.112:0/4184551533 >> v2:172.21.15.138:3300/0 conn(0x7f5d3c143ba0 0x7f5d3c0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.847+0000 7f5d40eba700 1 -- 172.21.15.112:0/4184551533 >> 172.21.15.112:0/4184551533 conn(0x7f5d3c0b1e90 msgr2=0x7f5d3c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:26.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.847+0000 7f5d40eba700 1 -- 172.21.15.112:0/4184551533 shutdown_connections 2023-08-27T11:27:26.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.847+0000 7f5d40eba700 1 -- 172.21.15.112:0/4184551533 wait complete. 2023-08-27T11:27:26.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.848+0000 7f5d40eba700 1 Processor -- start 2023-08-27T11:27:26.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.848+0000 7f5d40eba700 1 -- start start 2023-08-27T11:27:26.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d40eba700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d3c06d5b0 0x7f5d3c139f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d40eba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d3c1437c0 0x7f5d3c13dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d40eba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d3c143ba0 0x7f5d3c137970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d40eba700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c15ada0 con 0x7f5d3c1437c0 2023-08-27T11:27:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d40eba700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c1f3ed0 con 0x7f5d3c143ba0 2023-08-27T11:27:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d40eba700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c13a520 con 0x7f5d3c06d5b0 2023-08-27T11:27:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d39d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d3c1437c0 0x7f5d3c13dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d39d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d3c1437c0 0x7f5d3c13dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34884/0 (socket says 172.21.15.112:34884) 2023-08-27T11:27:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d39d9b700 1 -- 172.21.15.112:0/2244008672 learned_addr learned my addr 172.21.15.112:0/2244008672 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d3a59c700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d3c06d5b0 0x7f5d3c139f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d3ad9d700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d3c143ba0 0x7f5d3c137970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d39d9b700 1 -- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d3c06d5b0 msgr2=0x7f5d3c139f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d39d9b700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d3c06d5b0 0x7f5d3c139f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d39d9b700 1 -- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d3c143ba0 msgr2=0x7f5d3c137970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.849+0000 7f5d39d9b700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d3c143ba0 0x7f5d3c137970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.850+0000 7f5d39d9b700 1 -- 172.21.15.112:0/2244008672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c0094a0 con 0x7f5d3c1437c0 2023-08-27T11:27:26.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.850+0000 7f5d3ad9d700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d3c143ba0 0x7f5d3c137970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:26.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.850+0000 7f5d39d9b700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d3c1437c0 0x7f5d3c13dd00 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00ebe0 tx=0x7f5d2c009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.850+0000 7f5d2affd700 1 -- 172.21.15.112:0/2244008672 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c00c8b0 con 0x7f5d3c1437c0 2023-08-27T11:27:26.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.850+0000 7f5d2affd700 1 -- 172.21.15.112:0/2244008672 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2c00ca10 con 0x7f5d3c1437c0 2023-08-27T11:27:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.850+0000 7f5d40eba700 1 -- 172.21.15.112:0/2244008672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d34009b30 con 0x7f5d3c1437c0 2023-08-27T11:27:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.850+0000 7f5d2affd700 1 -- 172.21.15.112:0/2244008672 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c00cda0 con 0x7f5d3c1437c0 2023-08-27T11:27:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.850+0000 7f5d40eba700 1 -- 172.21.15.112:0/2244008672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c15aa30 con 0x7f5d3c1437c0 2023-08-27T11:27:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.851+0000 7f5d40eba700 1 -- 172.21.15.112:0/2244008672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c067e90 con 0x7f5d3c1437c0 2023-08-27T11:27:26.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.852+0000 7f5d2affd700 1 -- 172.21.15.112:0/2244008672 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5d2c013470 con 0x7f5d3c1437c0 2023-08-27T11:27:26.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.852+0000 7f5d2affd700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5d1c06a4d0 0x7f5d1c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.852+0000 7f5d2affd700 1 -- 172.21.15.112:0/2244008672 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f5d2c088bd0 con 0x7f5d3c1437c0 2023-08-27T11:27:26.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.852+0000 7f5d3a59c700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5d1c06a4d0 0x7f5d1c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.853+0000 7f5d3a59c700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5d1c06a4d0 0x7f5d1c06c990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c070b60 tx=0x7f5d30007d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:26.857+0000 7f5d2affd700 1 -- 172.21.15.112:0/2244008672 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5d2c053f50 con 0x7f5d3c1437c0 2023-08-27T11:27:27.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.048+0000 7f5d40eba700 1 -- 172.21.15.112:0/2244008672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5d3c067a20 con 0x7f5d3c1437c0 2023-08-27T11:27:27.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.712+0000 7f5d2affd700 1 -- 172.21.15.112:0/2244008672 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v112) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5d2c057570 con 0x7f5d3c1437c0 2023-08-27T11:27:27.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.717+0000 7f5d40eba700 1 -- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5d1c06a4d0 msgr2=0x7f5d1c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:27.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.717+0000 7f5d40eba700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5d1c06a4d0 0x7f5d1c06c990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c070b60 tx=0x7f5d30007d70 comp rx=0 tx=0).stop 2023-08-27T11:27:27.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.717+0000 7f5d40eba700 1 -- 172.21.15.112:0/2244008672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d3c1437c0 msgr2=0x7f5d3c13dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:27.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.717+0000 7f5d40eba700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d3c1437c0 0x7f5d3c13dd00 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00ebe0 tx=0x7f5d2c009e20 comp rx=0 tx=0).stop 2023-08-27T11:27:27.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.717+0000 7f5d40eba700 1 -- 172.21.15.112:0/2244008672 shutdown_connections 2023-08-27T11:27:27.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.717+0000 7f5d40eba700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5d1c06a4d0 0x7f5d1c06c990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:27.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.717+0000 7f5d40eba700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d3c143ba0 0x7f5d3c137970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:27.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.717+0000 7f5d40eba700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d3c06d5b0 0x7f5d3c139f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:27.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.717+0000 7f5d40eba700 1 --2- 172.21.15.112:0/2244008672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d3c1437c0 0x7f5d3c13dd00 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:27.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.717+0000 7f5d40eba700 1 -- 172.21.15.112:0/2244008672 >> 172.21.15.112:0/2244008672 conn(0x7f5d3c0b1e90 msgr2=0x7f5d3c06fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:27.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.717+0000 7f5d40eba700 1 -- 172.21.15.112:0/2244008672 shutdown_connections 2023-08-27T11:27:27.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.718+0000 7f5d40eba700 1 -- 172.21.15.112:0/2244008672 wait complete. 2023-08-27T11:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2023-08-27T11:27:27.740 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:27:27.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.861+0000 7f2afd0be700 1 Processor -- start 2023-08-27T11:27:27.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.861+0000 7f2afd0be700 1 -- start start 2023-08-27T11:27:27.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.862+0000 7f2afd0be700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2af806e4c0 0x7f2af815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:27.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.862+0000 7f2afd0be700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af8072000 0x7f2af815cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:27.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.862+0000 7f2afd0be700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af815ca50 con 0x7f2af8158ed0 2023-08-27T11:27:27.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.862+0000 7f2afd0be700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af8072470 con 0x7f2af806e4c0 2023-08-27T11:27:27.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.862+0000 7f2afd0be700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af815be20 con 0x7f2af8072000 2023-08-27T11:27:27.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.862+0000 7f2af659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2af806e4c0 0x7f2af815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:27.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.862+0000 7f2af659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2af806e4c0 0x7f2af815b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34894/0 (socket says 172.21.15.112:34894) 2023-08-27T11:27:27.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.862+0000 7f2af659c700 1 -- 172.21.15.112:0/849972703 learned_addr learned my addr 172.21.15.112:0/849972703 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:27.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.863+0000 7f2af759e700 1 --2- 172.21.15.112:0/849972703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af8072000 0x7f2af815cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:27.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.863+0000 7f2af659c700 1 -- 172.21.15.112:0/849972703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af8072000 msgr2=0x7f2af815cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:27.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.863+0000 7f2af659c700 1 --2- 172.21.15.112:0/849972703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af8072000 0x7f2af815cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:27.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.863+0000 7f2af659c700 1 -- 172.21.15.112:0/849972703 >> v1:172.21.15.138:6789/0 conn(0x7f2af8158ed0 legacy=0x7f2af806ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:27.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.863+0000 7f2af659c700 1 -- 172.21.15.112:0/849972703 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af815c100 con 0x7f2af806e4c0 2023-08-27T11:27:27.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.863+0000 7f2af659c700 1 --2- 172.21.15.112:0/849972703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af806e4c0 0x7f2af815b7f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80098f0 tx=0x7f2ae8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43412b73ca51c2c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:27.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.863+0000 7f2af5d9b700 1 -- 172.21.15.112:0/849972703 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae800b8a0 con 0x7f2af806e4c0 2023-08-27T11:27:27.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.863+0000 7f2af5d9b700 1 -- 172.21.15.112:0/849972703 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae800ba00 con 0x7f2af806e4c0 2023-08-27T11:27:27.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.863+0000 7f2af5d9b700 1 -- 172.21.15.112:0/849972703 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae800bd90 con 0x7f2af806e4c0 2023-08-27T11:27:27.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.863+0000 7f2afd0be700 1 -- 172.21.15.112:0/849972703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af806e4c0 msgr2=0x7f2af815b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:27.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.863+0000 7f2afd0be700 1 --2- 172.21.15.112:0/849972703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af806e4c0 0x7f2af815b7f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80098f0 tx=0x7f2ae8009c00 comp rx=0 tx=0).stop 2023-08-27T11:27:27.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.864+0000 7f2afd0be700 1 -- 172.21.15.112:0/849972703 shutdown_connections 2023-08-27T11:27:27.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.864+0000 7f2afd0be700 1 --2- 172.21.15.112:0/849972703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af8072000 0x7f2af815cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:27.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.864+0000 7f2afd0be700 1 --2- 172.21.15.112:0/849972703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af806e4c0 0x7f2af815b7f0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:27.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.864+0000 7f2afd0be700 1 -- 172.21.15.112:0/849972703 >> 172.21.15.112:0/849972703 conn(0x7f2af80cc240 msgr2=0x7f2af80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:27.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.864+0000 7f2afd0be700 1 -- 172.21.15.112:0/849972703 shutdown_connections 2023-08-27T11:27:27.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.864+0000 7f2afd0be700 1 -- 172.21.15.112:0/849972703 wait complete. 2023-08-27T11:27:27.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.865+0000 7f2afd0be700 1 Processor -- start 2023-08-27T11:27:27.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.865+0000 7f2afd0be700 1 -- start start 2023-08-27T11:27:27.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2afd0be700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af806e4c0 0x7f2af8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:27.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2afd0be700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af8072000 0x7f2af8156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:27.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2afd0be700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8158ed0 0x7f2af8156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:27.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2afd0be700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80aa240 con 0x7f2af8072000 2023-08-27T11:27:27.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2afd0be700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af8156cb0 con 0x7f2af8158ed0 2023-08-27T11:27:27.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2afd0be700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af8158330 con 0x7f2af806e4c0 2023-08-27T11:27:27.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2af659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af8072000 0x7f2af8156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:27.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2af659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af8072000 0x7f2af8156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34900/0 (socket says 172.21.15.112:34900) 2023-08-27T11:27:27.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2af659c700 1 -- 172.21.15.112:0/3807957645 learned_addr learned my addr 172.21.15.112:0/3807957645 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:27.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2af759e700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8158ed0 0x7f2af8156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:27.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2af6d9d700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af806e4c0 0x7f2af8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:27.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2af659c700 1 -- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af806e4c0 msgr2=0x7f2af8071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:27.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2af659c700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af806e4c0 0x7f2af8071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:27.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2af659c700 1 -- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8158ed0 msgr2=0x7f2af8156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:27.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2af659c700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8158ed0 0x7f2af8156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:27.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2af659c700 1 -- 172.21.15.112:0/3807957645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae8009550 con 0x7f2af8072000 2023-08-27T11:27:27.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2af759e700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8158ed0 0x7f2af8156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2af6d9d700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af806e4c0 0x7f2af8071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2af659c700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af8072000 0x7f2af8156e90 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000c00 tx=0x7f2ae8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.866+0000 7f2adf7fe700 1 -- 172.21.15.112:0/3807957645 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8004480 con 0x7f2af8072000 2023-08-27T11:27:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.867+0000 7f2adf7fe700 1 -- 172.21.15.112:0/3807957645 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae801f070 con 0x7f2af8072000 2023-08-27T11:27:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.867+0000 7f2afd0be700 1 -- 172.21.15.112:0/3807957645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af81585d0 con 0x7f2af8072000 2023-08-27T11:27:27.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.867+0000 7f2adf7fe700 1 -- 172.21.15.112:0/3807957645 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80038c0 con 0x7f2af8072000 2023-08-27T11:27:27.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.867+0000 7f2afd0be700 1 -- 172.21.15.112:0/3807957645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af80aa480 con 0x7f2af8072000 2023-08-27T11:27:27.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.867+0000 7f2afd0be700 1 -- 172.21.15.112:0/3807957645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af8067e90 con 0x7f2af8072000 2023-08-27T11:27:27.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.868+0000 7f2adf7fe700 1 -- 172.21.15.112:0/3807957645 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f2ae8021030 con 0x7f2af8072000 2023-08-27T11:27:27.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.869+0000 7f2adf7fe700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ae406a4d0 0x7f2ae406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:27.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.869+0000 7f2adf7fe700 1 -- 172.21.15.112:0/3807957645 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f2ae808a360 con 0x7f2af8072000 2023-08-27T11:27:27.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.869+0000 7f2af6d9d700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ae406a4d0 0x7f2ae406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:27.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.869+0000 7f2af6d9d700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ae406a4d0 0x7f2ae406c990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2af80cc790 tx=0x7f2aec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:27.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:27.873+0000 7f2adf7fe700 1 -- 172.21.15.112:0/3807957645 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2ae8055770 con 0x7f2af8072000 2023-08-27T11:27:28.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.105+0000 7f2afd0be700 1 -- 172.21.15.112:0/3807957645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2af8067a20 con 0x7f2af8072000 2023-08-27T11:27:28.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.729+0000 7f2adf7fe700 1 -- 172.21.15.112:0/3807957645 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v72) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2ae8058d90 con 0x7f2af8072000 2023-08-27T11:27:28.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.732+0000 7f2afd0be700 1 -- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ae406a4d0 msgr2=0x7f2ae406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.732+0000 7f2afd0be700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ae406a4d0 0x7f2ae406c990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2af80cc790 tx=0x7f2aec009040 comp rx=0 tx=0).stop 2023-08-27T11:27:28.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.732+0000 7f2afd0be700 1 -- 172.21.15.112:0/3807957645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af8072000 msgr2=0x7f2af8156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.732+0000 7f2afd0be700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af8072000 0x7f2af8156e90 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000c00 tx=0x7f2ae8004750 comp rx=0 tx=0).stop 2023-08-27T11:27:28.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.732+0000 7f2afd0be700 1 -- 172.21.15.112:0/3807957645 shutdown_connections 2023-08-27T11:27:28.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.732+0000 7f2afd0be700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2ae406a4d0 0x7f2ae406c990 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.732+0000 7f2afd0be700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8158ed0 0x7f2af8156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.732+0000 7f2afd0be700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af806e4c0 0x7f2af8071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.732+0000 7f2afd0be700 1 --2- 172.21.15.112:0/3807957645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af8072000 0x7f2af8156e90 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.732+0000 7f2afd0be700 1 -- 172.21.15.112:0/3807957645 >> 172.21.15.112:0/3807957645 conn(0x7f2af80cc240 msgr2=0x7f2af80cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:28.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.733+0000 7f2afd0be700 1 -- 172.21.15.112:0/3807957645 shutdown_connections 2023-08-27T11:27:28.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.733+0000 7f2afd0be700 1 -- 172.21.15.112:0/3807957645 wait complete. 2023-08-27T11:27:28.737 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 19 and data pool 20 2023-08-27T11:27:28.752 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:28.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.873+0000 7fe425df9700 1 Processor -- start 2023-08-27T11:27:28.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.873+0000 7fe425df9700 1 -- start start 2023-08-27T11:27:28.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe425df9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe42006adb0 0x7fe42006b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:28.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe425df9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42015eb60 con 0x7fe4200cdbd0 2023-08-27T11:27:28.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe425df9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4200b02f0 con 0x7fe420151060 2023-08-27T11:27:28.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe425df9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42006b770 con 0x7fe42006adb0 2023-08-27T11:27:28.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe41ffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe4200cdbd0 0x7fe4200ce060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57524/0 (socket says 172.21.15.112:57524) 2023-08-27T11:27:28.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe41ffff700 1 -- 172.21.15.112:0/3217799636 learned_addr learned my addr 172.21.15.112:0/3217799636 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:28.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe41effd700 1 --2- 172.21.15.112:0/3217799636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe42006adb0 0x7fe42006b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:28.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe41e7fc700 1 -- 172.21.15.112:0/3217799636 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424628456 0 0) 0x7fe42015eb60 con 0x7fe4200cdbd0 2023-08-27T11:27:28.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe41e7fc700 1 -- 172.21.15.112:0/3217799636 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4080035b0 con 0x7fe4200cdbd0 2023-08-27T11:27:28.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe41e7fc700 1 -- 172.21.15.112:0/3217799636 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1659269448 0 0) 0x7fe4200b02f0 con 0x7fe420151060 2023-08-27T11:27:28.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe41e7fc700 1 -- 172.21.15.112:0/3217799636 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe42015eb60 con 0x7fe420151060 2023-08-27T11:27:28.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe41e7fc700 1 -- 172.21.15.112:0/3217799636 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3267395888 0 0) 0x7fe4080035b0 con 0x7fe4200cdbd0 2023-08-27T11:27:28.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe41effd700 1 -- 172.21.15.112:0/3217799636 >> v1:172.21.15.138:6789/0 conn(0x7fe420151060 legacy=0x7fe42015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe41effd700 1 -- 172.21.15.112:0/3217799636 >> v1:172.21.15.112:6789/0 conn(0x7fe4200cdbd0 legacy=0x7fe4200ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.874+0000 7fe41effd700 1 -- 172.21.15.112:0/3217799636 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4200b05d0 con 0x7fe42006adb0 2023-08-27T11:27:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.875+0000 7fe41effd700 1 --2- 172.21.15.112:0/3217799636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe42006adb0 0x7fe42006b230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe410009870 tx=0x7fe410009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b90b276233dfe2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.875+0000 7fe41e7fc700 1 -- 172.21.15.112:0/3217799636 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4180019f0 con 0x7fe42006adb0 2023-08-27T11:27:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.875+0000 7fe41e7fc700 1 -- 172.21.15.112:0/3217799636 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe410004030 con 0x7fe42006adb0 2023-08-27T11:27:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.875+0000 7fe41e7fc700 1 -- 172.21.15.112:0/3217799636 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100043c0 con 0x7fe42006adb0 2023-08-27T11:27:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.875+0000 7fe425df9700 1 -- 172.21.15.112:0/3217799636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe42006adb0 msgr2=0x7fe42006b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.875+0000 7fe425df9700 1 --2- 172.21.15.112:0/3217799636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe42006adb0 0x7fe42006b230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe410009870 tx=0x7fe410009b80 comp rx=0 tx=0).stop 2023-08-27T11:27:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.875+0000 7fe425df9700 1 -- 172.21.15.112:0/3217799636 shutdown_connections 2023-08-27T11:27:28.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.875+0000 7fe425df9700 1 --2- 172.21.15.112:0/3217799636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe42006adb0 0x7fe42006b230 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.875+0000 7fe425df9700 1 -- 172.21.15.112:0/3217799636 >> 172.21.15.112:0/3217799636 conn(0x7fe4200d0000 msgr2=0x7fe4200d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:28.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.875+0000 7fe425df9700 1 -- 172.21.15.112:0/3217799636 shutdown_connections 2023-08-27T11:27:28.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.875+0000 7fe425df9700 1 -- 172.21.15.112:0/3217799636 wait complete. 2023-08-27T11:27:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.877+0000 7fe425df9700 1 Processor -- start 2023-08-27T11:27:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.877+0000 7fe425df9700 1 -- start start 2023-08-27T11:27:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.877+0000 7fe425df9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe42006adb0 0x7fe42014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe425df9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4200cdbd0 0x7fe420070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe425df9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe420151060 0x7fe4200c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe425df9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe420159120 con 0x7fe4200cdbd0 2023-08-27T11:27:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe425df9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe420070e10 con 0x7fe420151060 2023-08-27T11:27:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe425df9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe420159370 con 0x7fe42006adb0 2023-08-27T11:27:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe41effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4200cdbd0 0x7fe420070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe41effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4200cdbd0 0x7fe420070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34912/0 (socket says 172.21.15.112:34912) 2023-08-27T11:27:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe41effd700 1 -- 172.21.15.112:0/3070049903 learned_addr learned my addr 172.21.15.112:0/3070049903 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe41ffff700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe420151060 0x7fe4200c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:28.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe41f7fe700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe42006adb0 0x7fe42014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:28.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe41effd700 1 -- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe42006adb0 msgr2=0x7fe42014f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe41effd700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe42006adb0 0x7fe42014f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe41effd700 1 -- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe420151060 msgr2=0x7fe4200c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe41effd700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe420151060 0x7fe4200c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe41effd700 1 -- 172.21.15.112:0/3070049903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4100094a0 con 0x7fe4200cdbd0 2023-08-27T11:27:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe41ffff700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe420151060 0x7fe4200c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe41effd700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4200cdbd0 0x7fe420070470 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe410000c00 tx=0x7fe410004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe407fff700 1 -- 172.21.15.112:0/3070049903 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410003d00 con 0x7fe4200cdbd0 2023-08-27T11:27:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe407fff700 1 -- 172.21.15.112:0/3070049903 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41001f070 con 0x7fe4200cdbd0 2023-08-27T11:27:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe425df9700 1 -- 172.21.15.112:0/3070049903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe420159650 con 0x7fe4200cdbd0 2023-08-27T11:27:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe407fff700 1 -- 172.21.15.112:0/3070049903 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410011870 con 0x7fe4200cdbd0 2023-08-27T11:27:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.878+0000 7fe425df9700 1 -- 172.21.15.112:0/3070049903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42014f920 con 0x7fe4200cdbd0 2023-08-27T11:27:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.879+0000 7fe425df9700 1 -- 172.21.15.112:0/3070049903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe420159b70 con 0x7fe4200cdbd0 2023-08-27T11:27:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.880+0000 7fe407fff700 1 -- 172.21.15.112:0/3070049903 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe410021030 con 0x7fe4200cdbd0 2023-08-27T11:27:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.880+0000 7fe407fff700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe40c06a4d0 0x7fe40c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.880+0000 7fe407fff700 1 -- 172.21.15.112:0/3070049903 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe410089db0 con 0x7fe4200cdbd0 2023-08-27T11:27:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.880+0000 7fe41f7fe700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe40c06a4d0 0x7fe40c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.881+0000 7fe41f7fe700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe40c06a4d0 0x7fe40c06c990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe4140011e0 tx=0x7fe414009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:28.884+0000 7fe407fff700 1 -- 172.21.15.112:0/3070049903 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe420159b70 con 0x7fe4200cdbd0 2023-08-27T11:27:29.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.072+0000 7fe425df9700 1 -- 172.21.15.112:0/3070049903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe420159b70 con 0x7fe4200cdbd0 2023-08-27T11:27:29.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.074+0000 7fe407fff700 1 -- 172.21.15.112:0/3070049903 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+18288 (secure 0 0 0) 0x7fe420159b70 con 0x7fe4200cdbd0 2023-08-27T11:27:29.076 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:29.076 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":113,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:27:28.106990+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:26.212595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:27.068136+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:29.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.076+0000 7fe425df9700 1 -- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe40c06a4d0 msgr2=0x7fe40c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.076+0000 7fe425df9700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe40c06a4d0 0x7fe40c06c990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe4140011e0 tx=0x7fe414009040 comp rx=0 tx=0).stop 2023-08-27T11:27:29.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.076+0000 7fe425df9700 1 -- 172.21.15.112:0/3070049903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4200cdbd0 msgr2=0x7fe420070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.076+0000 7fe425df9700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4200cdbd0 0x7fe420070470 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe410000c00 tx=0x7fe410004750 comp rx=0 tx=0).stop 2023-08-27T11:27:29.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.076+0000 7fe425df9700 1 -- 172.21.15.112:0/3070049903 shutdown_connections 2023-08-27T11:27:29.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.076+0000 7fe425df9700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe40c06a4d0 0x7fe40c06c990 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.076+0000 7fe425df9700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe420151060 0x7fe4200c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.076+0000 7fe425df9700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe42006adb0 0x7fe42014f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.076+0000 7fe425df9700 1 --2- 172.21.15.112:0/3070049903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4200cdbd0 0x7fe420070470 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.076+0000 7fe425df9700 1 -- 172.21.15.112:0/3070049903 >> 172.21.15.112:0/3070049903 conn(0x7fe4200d0000 msgr2=0x7fe42015de90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:29.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.076+0000 7fe425df9700 1 -- 172.21.15.112:0/3070049903 shutdown_connections 2023-08-27T11:27:29.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.076+0000 7fe425df9700 1 -- 172.21.15.112:0/3070049903 wait complete. 2023-08-27T11:27:29.090 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:29.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.209+0000 7f18f9eb1700 1 Processor -- start 2023-08-27T11:27:29.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.209+0000 7f18f9eb1700 1 -- start start 2023-08-27T11:27:29.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f9eb1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18f4141420 0x7f18f40aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f9eb1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f18f406e150 0x7f18f406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f9eb1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f18f40cc070 0x7f18f40cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f9eb1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f406eb10 con 0x7f18f406e150 2023-08-27T11:27:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f9eb1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f40ab440 con 0x7f18f40cc070 2023-08-27T11:27:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f9eb1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f415eed0 con 0x7f18f4141420 2023-08-27T11:27:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f2ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f18f406e150 0x7f18f406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f2ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f18f406e150 0x7f18f406e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34920/0 (socket says 172.21.15.112:34920) 2023-08-27T11:27:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f2ffd700 1 -- 172.21.15.112:0/3309558217 learned_addr learned my addr 172.21.15.112:0/3309558217 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f37fe700 1 --2- 172.21.15.112:0/3309558217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18f4141420 0x7f18f40aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f3fff700 1 --2- 172.21.15.112:0/3309558217 >> v2:172.21.15.138:3300/0 conn(0x7f18f40cc070 0x7f18f40cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f2ffd700 1 -- 172.21.15.112:0/3309558217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18f4141420 msgr2=0x7f18f40aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:29.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f2ffd700 1 --2- 172.21.15.112:0/3309558217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18f4141420 0x7f18f40aaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f2ffd700 1 -- 172.21.15.112:0/3309558217 >> v2:172.21.15.138:3300/0 conn(0x7f18f40cc070 msgr2=0x7f18f40cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:29.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f2ffd700 1 --2- 172.21.15.112:0/3309558217 >> v2:172.21.15.138:3300/0 conn(0x7f18f40cc070 0x7f18f40cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f2ffd700 1 -- 172.21.15.112:0/3309558217 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f415f1b0 con 0x7f18f406e150 2023-08-27T11:27:29.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.210+0000 7f18f3fff700 1 --2- 172.21.15.112:0/3309558217 >> v2:172.21.15.138:3300/0 conn(0x7f18f40cc070 0x7f18f40cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:29.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.211+0000 7f18f2ffd700 1 --2- 172.21.15.112:0/3309558217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18f406e150 0x7f18f406e5d0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f18f40aa6a0 tx=0x7f18e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1754969e8786fee6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:29.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.211+0000 7f18f27fc700 1 -- 172.21.15.112:0/3309558217 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4005e30 con 0x7f18f406e150 2023-08-27T11:27:29.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.211+0000 7f18f27fc700 1 -- 172.21.15.112:0/3309558217 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e4004030 con 0x7f18f406e150 2023-08-27T11:27:29.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.211+0000 7f18f27fc700 1 -- 172.21.15.112:0/3309558217 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e40043e0 con 0x7f18f406e150 2023-08-27T11:27:29.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.211+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3309558217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18f406e150 msgr2=0x7f18f406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.211+0000 7f18f9eb1700 1 --2- 172.21.15.112:0/3309558217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18f406e150 0x7f18f406e5d0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f18f40aa6a0 tx=0x7f18e4009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.211+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3309558217 shutdown_connections 2023-08-27T11:27:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.211+0000 7f18f9eb1700 1 --2- 172.21.15.112:0/3309558217 >> v2:172.21.15.138:3300/0 conn(0x7f18f40cc070 0x7f18f40cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.211+0000 7f18f9eb1700 1 --2- 172.21.15.112:0/3309558217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18f4141420 0x7f18f40aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.211+0000 7f18f9eb1700 1 --2- 172.21.15.112:0/3309558217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18f406e150 0x7f18f406e5d0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.211+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3309558217 >> 172.21.15.112:0/3309558217 conn(0x7f18f406cd30 msgr2=0x7f18f40717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:29.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.211+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3309558217 shutdown_connections 2023-08-27T11:27:29.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.211+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3309558217 wait complete. 2023-08-27T11:27:29.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.212+0000 7f18f9eb1700 1 Processor -- start 2023-08-27T11:27:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.213+0000 7f18f9eb1700 1 -- start start 2023-08-27T11:27:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.213+0000 7f18f9eb1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18f406e150 0x7f18f4143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.213+0000 7f18f9eb1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18f40cc070 0x7f18f41402c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.213+0000 7f18f9eb1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4141420 0x7f18f41593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.213+0000 7f18f9eb1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40ad390 con 0x7f18f40cc070 2023-08-27T11:27:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.213+0000 7f18f9eb1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40ad8e0 con 0x7f18f4141420 2023-08-27T11:27:29.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.213+0000 7f18f9eb1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40ac980 con 0x7f18f406e150 2023-08-27T11:27:29.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.213+0000 7f18f2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18f40cc070 0x7f18f41402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18f2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18f40cc070 0x7f18f41402c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34928/0 (socket says 172.21.15.112:34928) 2023-08-27T11:27:29.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18f2ffd700 1 -- 172.21.15.112:0/3891757557 learned_addr learned my addr 172.21.15.112:0/3891757557 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:29.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18f3fff700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4141420 0x7f18f41593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18f37fe700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18f406e150 0x7f18f4143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18f2ffd700 1 -- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18f406e150 msgr2=0x7f18f4143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18f2ffd700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18f406e150 0x7f18f4143a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18f2ffd700 1 -- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4141420 msgr2=0x7f18f41593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18f2ffd700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4141420 0x7f18f41593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18f2ffd700 1 -- 172.21.15.112:0/3891757557 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e4009500 con 0x7f18f40cc070 2023-08-27T11:27:29.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18f2ffd700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18f40cc070 0x7f18f41402c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f18f406ed70 tx=0x7f18e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18dbfff700 1 -- 172.21.15.112:0/3891757557 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4003c30 con 0x7f18f40cc070 2023-08-27T11:27:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18dbfff700 1 -- 172.21.15.112:0/3891757557 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e4003d90 con 0x7f18f40cc070 2023-08-27T11:27:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3891757557 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f40acc20 con 0x7f18f40cc070 2023-08-27T11:27:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18dbfff700 1 -- 172.21.15.112:0/3891757557 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e401c640 con 0x7f18f40cc070 2023-08-27T11:27:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.214+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3891757557 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f40ad570 con 0x7f18f40cc070 2023-08-27T11:27:29.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.215+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3891757557 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f40ada90 con 0x7f18f40cc070 2023-08-27T11:27:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.216+0000 7f18dbfff700 1 -- 172.21.15.112:0/3891757557 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f18e401c7a0 con 0x7f18f40cc070 2023-08-27T11:27:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.216+0000 7f18dbfff700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18e006a4d0 0x7f18e006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.216+0000 7f18dbfff700 1 -- 172.21.15.112:0/3891757557 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f18e4089b20 con 0x7f18f40cc070 2023-08-27T11:27:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.216+0000 7f18f37fe700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18e006a4d0 0x7f18e006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.217+0000 7f18f37fe700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18e006a4d0 0x7f18e006c990 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009ac0 tx=0x7f18e8009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:29.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.220+0000 7f18dbfff700 1 -- 172.21.15.112:0/3891757557 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f18f40ada90 con 0x7f18f40cc070 2023-08-27T11:27:29.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.409+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3891757557 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18f40ada90 con 0x7f18f40cc070 2023-08-27T11:27:29.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.410+0000 7f18dbfff700 1 -- 172.21.15.112:0/3891757557 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+18288 (secure 0 0 0) 0x7f18f40ada90 con 0x7f18f40cc070 2023-08-27T11:27:29.412 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:29.412 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":113,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:27:28.106990+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:26.212595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:27.068136+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:29.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.412+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18e006a4d0 msgr2=0x7f18e006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.412+0000 7f18f9eb1700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18e006a4d0 0x7f18e006c990 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009ac0 tx=0x7f18e8009210 comp rx=0 tx=0).stop 2023-08-27T11:27:29.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.412+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3891757557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18f40cc070 msgr2=0x7f18f41402c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.412+0000 7f18f9eb1700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18f40cc070 0x7f18f41402c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f18f406ed70 tx=0x7f18e4004750 comp rx=0 tx=0).stop 2023-08-27T11:27:29.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.412+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3891757557 shutdown_connections 2023-08-27T11:27:29.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.412+0000 7f18f9eb1700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18e006a4d0 0x7f18e006c990 secure :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009ac0 tx=0x7f18e8009210 comp rx=0 tx=0).stop 2023-08-27T11:27:29.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.412+0000 7f18f9eb1700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4141420 0x7f18f41593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.412+0000 7f18f9eb1700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18f406e150 0x7f18f4143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.412+0000 7f18f9eb1700 1 --2- 172.21.15.112:0/3891757557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18f40cc070 0x7f18f41402c0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.412+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3891757557 >> 172.21.15.112:0/3891757557 conn(0x7f18f406cd30 msgr2=0x7f18f40a9270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.412+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3891757557 shutdown_connections 2023-08-27T11:27:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.412+0000 7f18f9eb1700 1 -- 172.21.15.112:0/3891757557 wait complete. 2023-08-27T11:27:29.426 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:27:29.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.549+0000 7f147841f700 1 Processor -- start 2023-08-27T11:27:29.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.549+0000 7f147841f700 1 -- start start 2023-08-27T11:27:29.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f147841f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1470146fd0 0x7f14700b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f147841f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14700d1500 con 0x7f1470070eb0 2023-08-27T11:27:29.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f147841f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14700d0460 con 0x7f1470146bf0 2023-08-27T11:27:29.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f147841f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14700d0710 con 0x7f1470146fd0 2023-08-27T11:27:29.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f14759ba700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1470146fd0 0x7f14700b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f14769bc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1470070eb0 0x7f14700d1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57550/0 (socket says 172.21.15.112:57550) 2023-08-27T11:27:29.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f14769bc700 1 -- 172.21.15.112:0/3743436705 learned_addr learned my addr 172.21.15.112:0/3743436705 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:29.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f14759ba700 1 -- 172.21.15.112:0/3743436705 >> v1:172.21.15.138:6789/0 conn(0x7f1470146bf0 legacy=0x7f14700cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:29.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f14759ba700 1 -- 172.21.15.112:0/3743436705 >> v1:172.21.15.112:6789/0 conn(0x7f1470070eb0 legacy=0x7f14700d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f14751b9700 1 -- 172.21.15.112:0/3743436705 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2755552148 0 0) 0x7f14700d1500 con 0x7f1470070eb0 2023-08-27T11:27:29.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f14759ba700 1 -- 172.21.15.112:0/3743436705 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14700d09f0 con 0x7f1470146fd0 2023-08-27T11:27:29.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f14759ba700 1 --2- 172.21.15.112:0/3743436705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1470146fd0 0x7f14700b5db0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f1460009840 tx=0x7f1460009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b4f5f5802d6a09 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:29.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f14751b9700 1 -- 172.21.15.112:0/3743436705 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460004230 con 0x7f1470146fd0 2023-08-27T11:27:29.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f14751b9700 1 -- 172.21.15.112:0/3743436705 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1460004390 con 0x7f1470146fd0 2023-08-27T11:27:29.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.550+0000 7f14751b9700 1 -- 172.21.15.112:0/3743436705 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14600036a0 con 0x7f1470146fd0 2023-08-27T11:27:29.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.551+0000 7f147841f700 1 -- 172.21.15.112:0/3743436705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1470146fd0 msgr2=0x7f14700b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.551+0000 7f147841f700 1 --2- 172.21.15.112:0/3743436705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1470146fd0 0x7f14700b5db0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f1460009840 tx=0x7f1460009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:29.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.551+0000 7f147841f700 1 -- 172.21.15.112:0/3743436705 shutdown_connections 2023-08-27T11:27:29.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.551+0000 7f147841f700 1 --2- 172.21.15.112:0/3743436705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1470146fd0 0x7f14700b5db0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.551+0000 7f147841f700 1 -- 172.21.15.112:0/3743436705 >> 172.21.15.112:0/3743436705 conn(0x7f147006c450 msgr2=0x7f147006f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.551+0000 7f147841f700 1 -- 172.21.15.112:0/3743436705 shutdown_connections 2023-08-27T11:27:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.551+0000 7f147841f700 1 -- 172.21.15.112:0/3743436705 wait complete. 2023-08-27T11:27:29.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.552+0000 7f147841f700 1 Processor -- start 2023-08-27T11:27:29.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.552+0000 7f147841f700 1 -- start start 2023-08-27T11:27:29.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f147841f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1470070eb0 0x7f147015d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f147841f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1470146bf0 0x7f14701ea230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f147841f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1470146fd0 0x7f14701ea770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f147841f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147015d550 con 0x7f1470146bf0 2023-08-27T11:27:29.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f147841f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14701f4590 con 0x7f1470070eb0 2023-08-27T11:27:29.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f147841f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14700b70c0 con 0x7f1470146fd0 2023-08-27T11:27:29.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f14759ba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1470146bf0 0x7f14701ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f14759ba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1470146bf0 0x7f14701ea230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34940/0 (socket says 172.21.15.112:34940) 2023-08-27T11:27:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f14759ba700 1 -- 172.21.15.112:0/663575244 learned_addr learned my addr 172.21.15.112:0/663575244 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f14769bc700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1470146fd0 0x7f14701ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f14761bb700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1470070eb0 0x7f147015d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f14759ba700 1 -- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1470146fd0 msgr2=0x7f14701ea770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f14759ba700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1470146fd0 0x7f14701ea770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f14759ba700 1 -- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1470070eb0 msgr2=0x7f147015d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f14759ba700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1470070eb0 0x7f147015d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f14759ba700 1 -- 172.21.15.112:0/663575244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14600094a0 con 0x7f1470146bf0 2023-08-27T11:27:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.553+0000 7f14759ba700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1470146bf0 0x7f14701ea230 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f14600090b0 tx=0x7f14600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.554+0000 7f1466ffd700 1 -- 172.21.15.112:0/663575244 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460011630 con 0x7f1470146bf0 2023-08-27T11:27:29.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.554+0000 7f1466ffd700 1 -- 172.21.15.112:0/663575244 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1460011790 con 0x7f1470146bf0 2023-08-27T11:27:29.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.554+0000 7f147841f700 1 -- 172.21.15.112:0/663575244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14700b73a0 con 0x7f1470146bf0 2023-08-27T11:27:29.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.554+0000 7f1466ffd700 1 -- 172.21.15.112:0/663575244 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460011b40 con 0x7f1470146bf0 2023-08-27T11:27:29.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.554+0000 7f147841f700 1 -- 172.21.15.112:0/663575244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14700b7960 con 0x7f1470146bf0 2023-08-27T11:27:29.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.554+0000 7f147841f700 1 -- 172.21.15.112:0/663575244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14700d02d0 con 0x7f1470146bf0 2023-08-27T11:27:29.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.555+0000 7f1466ffd700 1 -- 172.21.15.112:0/663575244 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f1460011ca0 con 0x7f1470146bf0 2023-08-27T11:27:29.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.555+0000 7f1466ffd700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f145406a4d0 0x7f145406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.555+0000 7f1466ffd700 1 -- 172.21.15.112:0/663575244 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f146008a750 con 0x7f1470146bf0 2023-08-27T11:27:29.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.556+0000 7f14761bb700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f145406a4d0 0x7f145406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.556+0000 7f14761bb700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f145406a4d0 0x7f145406c990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f146c004f30 tx=0x7f146c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:29.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.560+0000 7f1466ffd700 1 -- 172.21.15.112:0/663575244 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1460055b10 con 0x7f1470146bf0 2023-08-27T11:27:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:29.798+0000 7f147841f700 1 -- 172.21.15.112:0/663575244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f14700d0cb0 con 0x7f1470146bf0 2023-08-27T11:27:30.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.743+0000 7f1466ffd700 1 -- 172.21.15.112:0/663575244 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) v1 ==== 118+0+0 (secure 0 0 0) 0x7f14700d0cb0 con 0x7f1470146bf0 2023-08-27T11:27:30.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.746+0000 7f147841f700 1 -- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f145406a4d0 msgr2=0x7f145406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.746+0000 7f147841f700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f145406a4d0 0x7f145406c990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f146c004f30 tx=0x7f146c009040 comp rx=0 tx=0).stop 2023-08-27T11:27:30.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.746+0000 7f147841f700 1 -- 172.21.15.112:0/663575244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1470146bf0 msgr2=0x7f14701ea230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.746+0000 7f147841f700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1470146bf0 0x7f14701ea230 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f14600090b0 tx=0x7f14600047c0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.746+0000 7f147841f700 1 -- 172.21.15.112:0/663575244 shutdown_connections 2023-08-27T11:27:30.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.746+0000 7f147841f700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f145406a4d0 0x7f145406c990 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.746+0000 7f147841f700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1470146fd0 0x7f14701ea770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.746+0000 7f147841f700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1470070eb0 0x7f147015d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.746+0000 7f147841f700 1 --2- 172.21.15.112:0/663575244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1470146bf0 0x7f14701ea230 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.746+0000 7f147841f700 1 -- 172.21.15.112:0/663575244 >> 172.21.15.112:0/663575244 conn(0x7f147006c450 msgr2=0x7f147006d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:30.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.746+0000 7f147841f700 1 -- 172.21.15.112:0/663575244 shutdown_connections 2023-08-27T11:27:30.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.746+0000 7f147841f700 1 -- 172.21.15.112:0/663575244 wait complete. 2023-08-27T11:27:30.762 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:27:30.762 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:27:30.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.888+0000 7f91e463c700 1 Processor -- start 2023-08-27T11:27:30.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.889+0000 7f91e463c700 1 -- start start 2023-08-27T11:27:30.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.889+0000 7f91e463c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f91dc0708a0 0x7f91dc0cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.889+0000 7f91e463c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91dc06bfd0 0x7f91dc06c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.889+0000 7f91e463c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91dc06c970 con 0x7f91dc1466c0 2023-08-27T11:27:30.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.889+0000 7f91e463c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91dc0719b0 con 0x7f91dc0708a0 2023-08-27T11:27:30.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.889+0000 7f91e463c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0d1030 con 0x7f91dc06bfd0 2023-08-27T11:27:30.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.890+0000 7f91e1bd7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f91dc0708a0 0x7f91dc0cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.890+0000 7f91e1bd7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f91dc0708a0 0x7f91dc0cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34948/0 (socket says 172.21.15.112:34948) 2023-08-27T11:27:30.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.890+0000 7f91e1bd7700 1 -- 172.21.15.112:0/2405757797 learned_addr learned my addr 172.21.15.112:0/2405757797 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:30.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.890+0000 7f91e2bd9700 1 --2- 172.21.15.112:0/2405757797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91dc06bfd0 0x7f91dc06c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.890+0000 7f91e1bd7700 1 -- 172.21.15.112:0/2405757797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91dc06bfd0 msgr2=0x7f91dc06c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.890+0000 7f91e1bd7700 1 --2- 172.21.15.112:0/2405757797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91dc06bfd0 0x7f91dc06c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.890+0000 7f91e1bd7700 1 -- 172.21.15.112:0/2405757797 >> v1:172.21.15.138:6789/0 conn(0x7f91dc1466c0 legacy=0x7f91dc0cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.890+0000 7f91e1bd7700 1 -- 172.21.15.112:0/2405757797 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc0cf230 con 0x7f91dc0708a0 2023-08-27T11:27:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.890+0000 7f91e1bd7700 1 --2- 172.21.15.112:0/2405757797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc0708a0 0x7f91dc0cecf0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009920 tx=0x7f91d8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f67a0e8db71b1c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.890+0000 7f91e13d6700 1 -- 172.21.15.112:0/2405757797 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8004230 con 0x7f91dc0708a0 2023-08-27T11:27:30.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.890+0000 7f91e13d6700 1 -- 172.21.15.112:0/2405757797 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8004390 con 0x7f91dc0708a0 2023-08-27T11:27:30.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.890+0000 7f91e13d6700 1 -- 172.21.15.112:0/2405757797 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8009e70 con 0x7f91dc0708a0 2023-08-27T11:27:30.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.891+0000 7f91e463c700 1 -- 172.21.15.112:0/2405757797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc0708a0 msgr2=0x7f91dc0cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.891+0000 7f91e463c700 1 --2- 172.21.15.112:0/2405757797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc0708a0 0x7f91dc0cecf0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009920 tx=0x7f91d8009c30 comp rx=0 tx=0).stop 2023-08-27T11:27:30.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.891+0000 7f91e463c700 1 -- 172.21.15.112:0/2405757797 shutdown_connections 2023-08-27T11:27:30.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.891+0000 7f91e463c700 1 --2- 172.21.15.112:0/2405757797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91dc06bfd0 0x7f91dc06c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.891+0000 7f91e463c700 1 --2- 172.21.15.112:0/2405757797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc0708a0 0x7f91dc0cecf0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.891+0000 7f91e463c700 1 -- 172.21.15.112:0/2405757797 >> 172.21.15.112:0/2405757797 conn(0x7f91dc0cdc50 msgr2=0x7f91dc0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:30.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.891+0000 7f91e463c700 1 -- 172.21.15.112:0/2405757797 shutdown_connections 2023-08-27T11:27:30.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.891+0000 7f91e463c700 1 -- 172.21.15.112:0/2405757797 wait complete. 2023-08-27T11:27:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.892+0000 7f91e463c700 1 Processor -- start 2023-08-27T11:27:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.892+0000 7f91e463c700 1 -- start start 2023-08-27T11:27:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e463c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc06bfd0 0x7f91dc13b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e463c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91dc1466c0 0x7f91dc13fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e463c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91dc154140 0x7f91dc1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e463c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc1402d0 con 0x7f91dc06bfd0 2023-08-27T11:27:30.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e463c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc154920 con 0x7f91dc1466c0 2023-08-27T11:27:30.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e463c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc154c00 con 0x7f91dc154140 2023-08-27T11:27:30.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e23d8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc06bfd0 0x7f91dc13b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e23d8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc06bfd0 0x7f91dc13b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34952/0 (socket says 172.21.15.112:34952) 2023-08-27T11:27:30.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e23d8700 1 -- 172.21.15.112:0/2176989651 learned_addr learned my addr 172.21.15.112:0/2176989651 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:30.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e23d8700 1 -- 172.21.15.112:0/2176989651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91dc154140 msgr2=0x7f91dc1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e23d8700 1 --2- 172.21.15.112:0/2176989651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91dc154140 0x7f91dc1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e23d8700 1 -- 172.21.15.112:0/2176989651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91dc1466c0 msgr2=0x7f91dc13fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e23d8700 1 --2- 172.21.15.112:0/2176989651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91dc1466c0 0x7f91dc13fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e23d8700 1 -- 172.21.15.112:0/2176989651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8009550 con 0x7f91dc06bfd0 2023-08-27T11:27:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e23d8700 1 --2- 172.21.15.112:0/2176989651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc06bfd0 0x7f91dc13b040 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00b4a0 tx=0x7f91cc00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91d2ffd700 1 -- 172.21.15.112:0/2176989651 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc011830 con 0x7f91dc06bfd0 2023-08-27T11:27:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91d2ffd700 1 -- 172.21.15.112:0/2176989651 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc011990 con 0x7f91dc06bfd0 2023-08-27T11:27:30.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e463c700 1 -- 172.21.15.112:0/2176989651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91dc154ee0 con 0x7f91dc06bfd0 2023-08-27T11:27:30.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91d2ffd700 1 -- 172.21.15.112:0/2176989651 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc011d40 con 0x7f91dc06bfd0 2023-08-27T11:27:30.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.893+0000 7f91e463c700 1 -- 172.21.15.112:0/2176989651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91dc1554a0 con 0x7f91dc06bfd0 2023-08-27T11:27:30.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.894+0000 7f91e463c700 1 -- 172.21.15.112:0/2176989651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91dc1556c0 con 0x7f91dc06bfd0 2023-08-27T11:27:30.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.895+0000 7f91d2ffd700 1 -- 172.21.15.112:0/2176989651 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f91cc01a400 con 0x7f91dc06bfd0 2023-08-27T11:27:30.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.895+0000 7f91d2ffd700 1 --2- 172.21.15.112:0/2176989651 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91c006a4d0 0x7f91c006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.895+0000 7f91d2ffd700 1 -- 172.21.15.112:0/2176989651 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f91cc088d40 con 0x7f91dc06bfd0 2023-08-27T11:27:30.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.896+0000 7f91e1bd7700 1 --2- 172.21.15.112:0/2176989651 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91c006a4d0 0x7f91c006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.896+0000 7f91e1bd7700 1 --2- 172.21.15.112:0/2176989651 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91c006a4d0 0x7f91c006c990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f91d8000c00 tx=0x7f91d8010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:30.899+0000 7f91d2ffd700 1 -- 172.21.15.112:0/2176989651 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f91cc053e10 con 0x7f91dc06bfd0 2023-08-27T11:27:31.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.131+0000 7f91e463c700 1 -- 172.21.15.112:0/2176989651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f91dc067a20 con 0x7f91dc06bfd0 2023-08-27T11:27:31.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.744+0000 7f91d2ffd700 1 -- 172.21.15.112:0/2176989651 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7f91cc057430 con 0x7f91dc06bfd0 2023-08-27T11:27:31.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.747+0000 7f91e463c700 1 -- 172.21.15.112:0/2176989651 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91c006a4d0 msgr2=0x7f91c006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.747+0000 7f91e463c700 1 --2- 172.21.15.112:0/2176989651 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91c006a4d0 0x7f91c006c990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f91d8000c00 tx=0x7f91d8010000 comp rx=0 tx=0).stop 2023-08-27T11:27:31.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.747+0000 7f91e463c700 1 -- 172.21.15.112:0/2176989651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc06bfd0 msgr2=0x7f91dc13b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.747+0000 7f91e463c700 1 --2- 172.21.15.112:0/2176989651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc06bfd0 0x7f91dc13b040 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00b4a0 tx=0x7f91cc00b860 comp rx=0 tx=0).stop 2023-08-27T11:27:31.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.747+0000 7f91e463c700 1 -- 172.21.15.112:0/2176989651 shutdown_connections 2023-08-27T11:27:31.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.747+0000 7f91e463c700 1 --2- 172.21.15.112:0/2176989651 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91c006a4d0 0x7f91c006c990 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.747+0000 7f91e463c700 1 --2- 172.21.15.112:0/2176989651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91dc154140 0x7f91dc1559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.747+0000 7f91e463c700 1 --2- 172.21.15.112:0/2176989651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc06bfd0 0x7f91dc13b040 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.747+0000 7f91e463c700 1 --2- 172.21.15.112:0/2176989651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91dc1466c0 0x7f91dc13fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.747+0000 7f91e463c700 1 -- 172.21.15.112:0/2176989651 >> 172.21.15.112:0/2176989651 conn(0x7f91dc0cdc50 msgr2=0x7f91dc145610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:31.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.747+0000 7f91e463c700 1 -- 172.21.15.112:0/2176989651 shutdown_connections 2023-08-27T11:27:31.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.747+0000 7f91e463c700 1 -- 172.21.15.112:0/2176989651 wait complete. 2023-08-27T11:27:31.763 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:27:31.763 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:31.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.887+0000 7f985877f700 1 Processor -- start 2023-08-27T11:27:31.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.887+0000 7f985877f700 1 -- start start 2023-08-27T11:27:31.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f985877f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f985015de40 0x7f985006e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f985877f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f985006e690 0x7f98500cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f985877f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985006eaf0 con 0x7f9850153e70 2023-08-27T11:27:31.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f985877f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985015e2d0 con 0x7f985015de40 2023-08-27T11:27:31.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f985877f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9850154250 con 0x7f985006e690 2023-08-27T11:27:31.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f9855d1a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f985015de40 0x7f985006e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f985651b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9850153e70 0x7f9850159e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57574/0 (socket says 172.21.15.112:57574) 2023-08-27T11:27:31.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f985651b700 1 -- 172.21.15.112:0/2714705672 learned_addr learned my addr 172.21.15.112:0/2714705672 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:31.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f9855d1a700 1 -- 172.21.15.112:0/2714705672 >> v2:172.21.15.138:3300/0 conn(0x7f985006e690 msgr2=0x7f98500cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:31.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f9855d1a700 1 --2- 172.21.15.112:0/2714705672 >> v2:172.21.15.138:3300/0 conn(0x7f985006e690 0x7f98500cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f9855d1a700 1 -- 172.21.15.112:0/2714705672 >> v1:172.21.15.112:6789/0 conn(0x7f9850153e70 legacy=0x7f9850159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f9855d1a700 1 -- 172.21.15.112:0/2714705672 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985015a970 con 0x7f985015de40 2023-08-27T11:27:31.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f9855519700 1 -- 172.21.15.112:0/2714705672 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618571154 0 0) 0x7f985006eaf0 con 0x7f9850153e70 2023-08-27T11:27:31.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f9855d1a700 1 --2- 172.21.15.112:0/2714705672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985015de40 0x7f985006e150 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9850072580 tx=0x7f9840009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6855a23dfac85330 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.888+0000 7f9855519700 1 -- 172.21.15.112:0/2714705672 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840011830 con 0x7f985015de40 2023-08-27T11:27:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.889+0000 7f9855519700 1 -- 172.21.15.112:0/2714705672 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9840011990 con 0x7f985015de40 2023-08-27T11:27:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.889+0000 7f9855519700 1 -- 172.21.15.112:0/2714705672 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840011d20 con 0x7f985015de40 2023-08-27T11:27:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.889+0000 7f985877f700 1 -- 172.21.15.112:0/2714705672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985015de40 msgr2=0x7f985006e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.889+0000 7f985877f700 1 --2- 172.21.15.112:0/2714705672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985015de40 0x7f985006e150 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9850072580 tx=0x7f9840009b20 comp rx=0 tx=0).stop 2023-08-27T11:27:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.889+0000 7f985877f700 1 -- 172.21.15.112:0/2714705672 shutdown_connections 2023-08-27T11:27:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.889+0000 7f985877f700 1 --2- 172.21.15.112:0/2714705672 >> v2:172.21.15.138:3300/0 conn(0x7f985006e690 0x7f98500cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.889+0000 7f985877f700 1 --2- 172.21.15.112:0/2714705672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985015de40 0x7f985006e150 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.889+0000 7f985877f700 1 -- 172.21.15.112:0/2714705672 >> 172.21.15.112:0/2714705672 conn(0x7f985006cd30 msgr2=0x7f98500717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.889+0000 7f985877f700 1 -- 172.21.15.112:0/2714705672 shutdown_connections 2023-08-27T11:27:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.889+0000 7f985877f700 1 -- 172.21.15.112:0/2714705672 wait complete. 2023-08-27T11:27:31.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.890+0000 7f985877f700 1 Processor -- start 2023-08-27T11:27:31.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.890+0000 7f985877f700 1 -- start start 2023-08-27T11:27:31.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985877f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985006e690 0x7f9850156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985877f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9850153e70 0x7f9850154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985877f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985015de40 0x7f985013f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985877f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9850154990 con 0x7f985006e690 2023-08-27T11:27:31.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985877f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9850070790 con 0x7f985015de40 2023-08-27T11:27:31.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985877f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985015b290 con 0x7f9850153e70 2023-08-27T11:27:31.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985651b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985006e690 0x7f9850156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985651b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985006e690 0x7f9850156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34966/0 (socket says 172.21.15.112:34966) 2023-08-27T11:27:31.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985651b700 1 -- 172.21.15.112:0/3579870148 learned_addr learned my addr 172.21.15.112:0/3579870148 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:31.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985651b700 1 -- 172.21.15.112:0/3579870148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9850153e70 msgr2=0x7f9850154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985651b700 1 --2- 172.21.15.112:0/3579870148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9850153e70 0x7f9850154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985651b700 1 -- 172.21.15.112:0/3579870148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985015de40 msgr2=0x7f985013f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985651b700 1 --2- 172.21.15.112:0/3579870148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985015de40 0x7f985013f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.891+0000 7f985651b700 1 -- 172.21.15.112:0/3579870148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98400094a0 con 0x7f985006e690 2023-08-27T11:27:31.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.892+0000 7f985651b700 1 --2- 172.21.15.112:0/3579870148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985006e690 0x7f9850156ef0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f984c0011e0 tx=0x7f984c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:31.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.892+0000 7f9846ffd700 1 -- 172.21.15.112:0/3579870148 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c004c80 con 0x7f985006e690 2023-08-27T11:27:31.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.892+0000 7f9846ffd700 1 -- 172.21.15.112:0/3579870148 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c004e00 con 0x7f985006e690 2023-08-27T11:27:31.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.892+0000 7f985877f700 1 -- 172.21.15.112:0/3579870148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985015b570 con 0x7f985006e690 2023-08-27T11:27:31.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.892+0000 7f9846ffd700 1 -- 172.21.15.112:0/3579870148 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c007310 con 0x7f985006e690 2023-08-27T11:27:31.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.892+0000 7f985877f700 1 -- 172.21.15.112:0/3579870148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985013fbe0 con 0x7f985006e690 2023-08-27T11:27:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.893+0000 7f985877f700 1 -- 172.21.15.112:0/3579870148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985015ba90 con 0x7f985006e690 2023-08-27T11:27:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.893+0000 7f9846ffd700 1 -- 172.21.15.112:0/3579870148 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f984c007470 con 0x7f985006e690 2023-08-27T11:27:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.893+0000 7f9846ffd700 1 --2- 172.21.15.112:0/3579870148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f983406a4d0 0x7f983406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.894+0000 7f9846ffd700 1 -- 172.21.15.112:0/3579870148 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f984c088940 con 0x7f985006e690 2023-08-27T11:27:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.894+0000 7f9855d1a700 1 --2- 172.21.15.112:0/3579870148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f983406a4d0 0x7f983406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.894+0000 7f9855d1a700 1 --2- 172.21.15.112:0/3579870148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f983406a4d0 0x7f983406c990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f98501590b0 tx=0x7f9840010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:31.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:31.898+0000 7f9846ffd700 1 -- 172.21.15.112:0/3579870148 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f984c053b60 con 0x7f985006e690 2023-08-27T11:27:32.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.127+0000 7f985877f700 1 -- 172.21.15.112:0/3579870148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98500748d0 con 0x7f985006e690 2023-08-27T11:27:32.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.127+0000 7f9846ffd700 1 -- 172.21.15.112:0/3579870148 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4889 (secure 0 0 0) 0x7f984c057180 con 0x7f985006e690 2023-08-27T11:27:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:32.130 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1795721853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1795721853},{"type":"v1","addr":"172.21.15.138:6836","nonce":1795721853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":7974,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3832579050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3832579050},{"type":"v1","addr":"172.21.15.112:6834","nonce":3832579050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4019485943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":4019485943},{"type":"v1","addr":"172.21.15.112:6835","nonce":4019485943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:27:28.107305+0000","modified":"2023-08-27T11:27:31.742912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7989},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7989":{"gid":7989,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/3245198376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3245198376},{"type":"v1","addr":"172.21.15.138:6837","nonce":3245198376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-27T11:27:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.129+0000 7f985877f700 1 -- 172.21.15.112:0/3579870148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f983406a4d0 msgr2=0x7f983406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.129+0000 7f985877f700 1 --2- 172.21.15.112:0/3579870148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f983406a4d0 0x7f983406c990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f98501590b0 tx=0x7f9840010000 comp rx=0 tx=0).stop 2023-08-27T11:27:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.129+0000 7f985877f700 1 -- 172.21.15.112:0/3579870148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985006e690 msgr2=0x7f9850156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.129+0000 7f985877f700 1 --2- 172.21.15.112:0/3579870148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985006e690 0x7f9850156ef0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f984c0011e0 tx=0x7f984c00cae0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.130+0000 7f985877f700 1 -- 172.21.15.112:0/3579870148 shutdown_connections 2023-08-27T11:27:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.130+0000 7f985877f700 1 --2- 172.21.15.112:0/3579870148 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f983406a4d0 0x7f983406c990 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.130+0000 7f985877f700 1 --2- 172.21.15.112:0/3579870148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985015de40 0x7f985013f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.130+0000 7f985877f700 1 --2- 172.21.15.112:0/3579870148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f985006e690 0x7f9850156ef0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.130+0000 7f985877f700 1 --2- 172.21.15.112:0/3579870148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9850153e70 0x7f9850154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.130+0000 7f985877f700 1 -- 172.21.15.112:0/3579870148 >> 172.21.15.112:0/3579870148 conn(0x7f985006cd30 msgr2=0x7f98501512d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.130+0000 7f985877f700 1 -- 172.21.15.112:0/3579870148 shutdown_connections 2023-08-27T11:27:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.130+0000 7f985877f700 1 -- 172.21.15.112:0/3579870148 wait complete. 2023-08-27T11:27:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 76 2023-08-27T11:27:32.143 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:32.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.261+0000 7f36c1736700 1 Processor -- start 2023-08-27T11:27:32.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.262+0000 7f36c1736700 1 -- start start 2023-08-27T11:27:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.262+0000 7f36c1736700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f36bc140700 0x7f36bc0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.262+0000 7f36c1736700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15e8c0 0x7f36bc142250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.262+0000 7f36c1736700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36bc161fd0 con 0x7f36bc0cf1b0 2023-08-27T11:27:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.262+0000 7f36c1736700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36bc0ce5b0 con 0x7f36bc140700 2023-08-27T11:27:32.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.262+0000 7f36c1736700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc15ed30 con 0x7f36bc15e8c0 2023-08-27T11:27:32.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36ba7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f36bc0cf1b0 0x7f36bc15e1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57588/0 (socket says 172.21.15.112:57588) 2023-08-27T11:27:32.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36ba7fc700 1 -- 172.21.15.112:0/3019408884 learned_addr learned my addr 172.21.15.112:0/3019408884 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:32.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36baffd700 1 --2- 172.21.15.112:0/3019408884 >> v2:172.21.15.138:3300/0 conn(0x7f36bc140700 0x7f36bc0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36bb7fe700 1 --2- 172.21.15.112:0/3019408884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15e8c0 0x7f36bc142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998118561 0 0) 0x7f36bc161fd0 con 0x7f36bc0cf1b0 2023-08-27T11:27:32.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36a40035b0 con 0x7f36bc0cf1b0 2023-08-27T11:27:32.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3820594010 0 0) 0x7f36a40035b0 con 0x7f36bc0cf1b0 2023-08-27T11:27:32.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15e8c0 msgr2=0x7f36bc142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:32.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 --2- 172.21.15.112:0/3019408884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15e8c0 0x7f36bc142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 >> v2:172.21.15.138:3300/0 conn(0x7f36bc140700 msgr2=0x7f36bc0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:32.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 --2- 172.21.15.112:0/3019408884 >> v2:172.21.15.138:3300/0 conn(0x7f36bc140700 0x7f36bc0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36bc1616a0 con 0x7f36bc0cf1b0 2023-08-27T11:27:32.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f36b0001670 con 0x7f36bc0cf1b0 2023-08-27T11:27:32.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 >> v1:172.21.15.112:6789/0 conn(0x7f36bc0cf1b0 legacy=0x7f36bc15e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 --2- 172.21.15.112:0/3019408884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36a4003a00 0x7f36a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 --2- 172.21.15.112:0/3019408884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004850 0x7f36a4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 --2- 172.21.15.112:0/3019408884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15e8c0 0x7f36a4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a40046a0 con 0x7f36a4003a00 2023-08-27T11:27:32.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a400ba40 con 0x7f36a4004850 2023-08-27T11:27:32.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a400bc00 con 0x7f36bc15e8c0 2023-08-27T11:27:32.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36baffd700 1 --2- 172.21.15.112:0/3019408884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36a4003a00 0x7f36a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36ba7fc700 1 --2- 172.21.15.112:0/3019408884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004850 0x7f36a4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.263+0000 7f36bb7fe700 1 --2- 172.21.15.112:0/3019408884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15e8c0 0x7f36a4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36baffd700 1 -- 172.21.15.112:0/3019408884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15e8c0 msgr2=0x7f36a4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:32.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36baffd700 1 --2- 172.21.15.112:0/3019408884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15e8c0 0x7f36a4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36baffd700 1 -- 172.21.15.112:0/3019408884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004850 msgr2=0x7f36a4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:32.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36baffd700 1 --2- 172.21.15.112:0/3019408884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004850 0x7f36a4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36baffd700 1 -- 172.21.15.112:0/3019408884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a40035b0 con 0x7f36a4003a00 2023-08-27T11:27:32.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36bb7fe700 1 --2- 172.21.15.112:0/3019408884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15e8c0 0x7f36a4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:32.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36baffd700 1 --2- 172.21.15.112:0/3019408884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36a4003a00 0x7f36a4003e20 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009b60 tx=0x7f36a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb5d289154e5c26 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:32.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00e6c0 con 0x7f36a4003a00 2023-08-27T11:27:32.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac00e820 con 0x7f36a4003a00 2023-08-27T11:27:32.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36b9ffb700 1 -- 172.21.15.112:0/3019408884 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00eb90 con 0x7f36a4003a00 2023-08-27T11:27:32.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36c1736700 1 -- 172.21.15.112:0/3019408884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36a4003a00 msgr2=0x7f36a4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36c1736700 1 --2- 172.21.15.112:0/3019408884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36a4003a00 0x7f36a4003e20 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009b60 tx=0x7f36a4005880 comp rx=0 tx=0).stop 2023-08-27T11:27:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36bb7fe700 1 -- 172.21.15.112:0/3019408884 reap_dead start 2023-08-27T11:27:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36c1736700 1 -- 172.21.15.112:0/3019408884 shutdown_connections 2023-08-27T11:27:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36c1736700 1 -- 172.21.15.112:0/3019408884 >> 172.21.15.112:0/3019408884 conn(0x7f36bc066010 msgr2=0x7f36bc0663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36c1736700 1 -- 172.21.15.112:0/3019408884 shutdown_connections 2023-08-27T11:27:32.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.264+0000 7f36c1736700 1 -- 172.21.15.112:0/3019408884 wait complete. 2023-08-27T11:27:32.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.266+0000 7f36c1736700 1 Processor -- start 2023-08-27T11:27:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.266+0000 7f36c1736700 1 -- start start 2023-08-27T11:27:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36c1736700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4003a00 0x7f36bc157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36c1736700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc134bf0 0x7f36bc1586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36c1736700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15b550 0x7f36bc0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36c1736700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc15a790 con 0x7f36bc134bf0 2023-08-27T11:27:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36c1736700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc15aa70 con 0x7f36a4003a00 2023-08-27T11:27:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36c1736700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc15ad50 con 0x7f36bc15b550 2023-08-27T11:27:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36ba7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc134bf0 0x7f36bc1586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36ba7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc134bf0 0x7f36bc1586f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34988/0 (socket says 172.21.15.112:34988) 2023-08-27T11:27:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36ba7fc700 1 -- 172.21.15.112:0/690860881 learned_addr learned my addr 172.21.15.112:0/690860881 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36baffd700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4003a00 0x7f36bc157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36bb7fe700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15b550 0x7f36bc0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36ba7fc700 1 -- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15b550 msgr2=0x7f36bc0cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36ba7fc700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15b550 0x7f36bc0cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36ba7fc700 1 -- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4003a00 msgr2=0x7f36bc157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36ba7fc700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4003a00 0x7f36bc157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36ba7fc700 1 -- 172.21.15.112:0/690860881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36bc15b030 con 0x7f36bc134bf0 2023-08-27T11:27:32.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36bb7fe700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15b550 0x7f36bc0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:32.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36ba7fc700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc134bf0 0x7f36bc1586f0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f36b0001aa0 tx=0x7f36b00075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:32.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36a37fe700 1 -- 172.21.15.112:0/690860881 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b000f660 con 0x7f36bc134bf0 2023-08-27T11:27:32.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36a37fe700 1 -- 172.21.15.112:0/690860881 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b000f7e0 con 0x7f36bc134bf0 2023-08-27T11:27:32.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36c1736700 1 -- 172.21.15.112:0/690860881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36bc15a3b0 con 0x7f36bc134bf0 2023-08-27T11:27:32.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36a37fe700 1 -- 172.21.15.112:0/690860881 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b000fb90 con 0x7f36bc134bf0 2023-08-27T11:27:32.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.267+0000 7f36c1736700 1 -- 172.21.15.112:0/690860881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc158e10 con 0x7f36bc134bf0 2023-08-27T11:27:32.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.268+0000 7f36c1736700 1 -- 172.21.15.112:0/690860881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36bc158420 con 0x7f36bc134bf0 2023-08-27T11:27:32.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.269+0000 7f36a37fe700 1 -- 172.21.15.112:0/690860881 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f36b000fcf0 con 0x7f36bc134bf0 2023-08-27T11:27:32.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.269+0000 7f36a37fe700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f36a806a4d0 0x7f36a806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.269+0000 7f36a37fe700 1 -- 172.21.15.112:0/690860881 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f36b0088a70 con 0x7f36bc134bf0 2023-08-27T11:27:32.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.269+0000 7f36baffd700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f36a806a4d0 0x7f36a806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.270+0000 7f36baffd700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f36a806a4d0 0x7f36a806c990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0090b0 tx=0x7f36ac01f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:32.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.274+0000 7f36a37fe700 1 -- 172.21.15.112:0/690860881 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f36b0053d20 con 0x7f36bc134bf0 2023-08-27T11:27:32.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.467+0000 7f36c1736700 1 -- 172.21.15.112:0/690860881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36bc15a8d0 con 0x7f36bc134bf0 2023-08-27T11:27:32.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.468+0000 7f36a37fe700 1 -- 172.21.15.112:0/690860881 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+18275 (secure 0 0 0) 0x7f36bc15a8d0 con 0x7f36bc134bf0 2023-08-27T11:27:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:32.470 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":114,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:27:29.710515+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:26.212595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:27.068136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:32.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.470+0000 7f36c1736700 1 -- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f36a806a4d0 msgr2=0x7f36a806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.470+0000 7f36c1736700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f36a806a4d0 0x7f36a806c990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0090b0 tx=0x7f36ac01f040 comp rx=0 tx=0).stop 2023-08-27T11:27:32.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.470+0000 7f36c1736700 1 -- 172.21.15.112:0/690860881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc134bf0 msgr2=0x7f36bc1586f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.470+0000 7f36c1736700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc134bf0 0x7f36bc1586f0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f36b0001aa0 tx=0x7f36b00075f0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.470+0000 7f36c1736700 1 -- 172.21.15.112:0/690860881 shutdown_connections 2023-08-27T11:27:32.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.470+0000 7f36c1736700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f36a806a4d0 0x7f36a806c990 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.470+0000 7f36c1736700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36bc15b550 0x7f36bc0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.470+0000 7f36c1736700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4003a00 0x7f36bc157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.470+0000 7f36c1736700 1 --2- 172.21.15.112:0/690860881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc134bf0 0x7f36bc1586f0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.470+0000 7f36c1736700 1 -- 172.21.15.112:0/690860881 >> 172.21.15.112:0/690860881 conn(0x7f36bc066010 msgr2=0x7f36bc15e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:32.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.470+0000 7f36c1736700 1 -- 172.21.15.112:0/690860881 shutdown_connections 2023-08-27T11:27:32.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.470+0000 7f36c1736700 1 -- 172.21.15.112:0/690860881 wait complete. 2023-08-27T11:27:32.484 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:27:32.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:27:32.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.603+0000 7f970c4d8700 1 Processor -- start 2023-08-27T11:27:32.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.603+0000 7f970c4d8700 1 -- start start 2023-08-27T11:27:32.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.604+0000 7f970c4d8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97040b1400 0x7f97040cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.604+0000 7f970c4d8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9704161850 con 0x7f9704161ce0 2023-08-27T11:27:32.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.604+0000 7f970c4d8700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97040cfc10 con 0x7f97040b17e0 2023-08-27T11:27:32.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.604+0000 7f970c4d8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9704161200 con 0x7f97040b1400 2023-08-27T11:27:32.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.604+0000 7f970aa75700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9704161ce0 0x7f970406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57610/0 (socket says 172.21.15.112:57610) 2023-08-27T11:27:32.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.604+0000 7f970aa75700 1 -- 172.21.15.112:0/2257065591 learned_addr learned my addr 172.21.15.112:0/2257065591 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:32.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f970a274700 1 --2- 172.21.15.112:0/2257065591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97040b1400 0x7f97040cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799532742 0 0) 0x7f9704161850 con 0x7f9704161ce0 2023-08-27T11:27:32.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f00035b0 con 0x7f9704161ce0 2023-08-27T11:27:32.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3273908560 0 0) 0x7f96f00035b0 con 0x7f9704161ce0 2023-08-27T11:27:32.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97040b1400 msgr2=0x7f97040cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:32.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 --2- 172.21.15.112:0/2257065591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97040b1400 0x7f97040cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 >> v1:172.21.15.138:6789/0 conn(0x7f97040b17e0 legacy=0x7f970406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97041614e0 con 0x7f9704161ce0 2023-08-27T11:27:32.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f96fc0019f0 con 0x7f9704161ce0 2023-08-27T11:27:32.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 >> v1:172.21.15.112:6789/0 conn(0x7f9704161ce0 legacy=0x7f970406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 --2- 172.21.15.112:0/2257065591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f0003a30 0x7f96f0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 --2- 172.21.15.112:0/2257065591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97040b1400 0x7f96f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 --2- 172.21.15.112:0/2257065591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96f0004d90 0x7f96f0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9704161200 con 0x7f96f0003a30 2023-08-27T11:27:32.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f000bb40 con 0x7f96f0004d90 2023-08-27T11:27:32.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f000bd00 con 0x7f97040b1400 2023-08-27T11:27:32.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f970a274700 1 --2- 172.21.15.112:0/2257065591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f0003a30 0x7f96f0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f9709a73700 1 --2- 172.21.15.112:0/2257065591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97040b1400 0x7f96f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f970aa75700 1 --2- 172.21.15.112:0/2257065591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96f0004d90 0x7f96f0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f970a274700 1 -- 172.21.15.112:0/2257065591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97040b1400 msgr2=0x7f96f0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:32.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f970a274700 1 --2- 172.21.15.112:0/2257065591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97040b1400 0x7f96f0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f970a274700 1 -- 172.21.15.112:0/2257065591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96f0004d90 msgr2=0x7f96f0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:32.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f970a274700 1 --2- 172.21.15.112:0/2257065591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96f0004d90 0x7f96f0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f970a274700 1 -- 172.21.15.112:0/2257065591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f00035b0 con 0x7f96f0003a30 2023-08-27T11:27:32.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.605+0000 7f970a274700 1 --2- 172.21.15.112:0/2257065591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f0003a30 0x7f96f0003e50 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009b60 tx=0x7f96f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602f9cc72afed16e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:32.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.606+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f400e6c0 con 0x7f96f0003a30 2023-08-27T11:27:32.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.606+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f400e820 con 0x7f96f0003a30 2023-08-27T11:27:32.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.606+0000 7f9709272700 1 -- 172.21.15.112:0/2257065591 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f400eb90 con 0x7f96f0003a30 2023-08-27T11:27:32.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.606+0000 7f970c4d8700 1 -- 172.21.15.112:0/2257065591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f0003a30 msgr2=0x7f96f0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.606+0000 7f970c4d8700 1 --2- 172.21.15.112:0/2257065591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f0003a30 0x7f96f0003e50 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009b60 tx=0x7f96f0005880 comp rx=0 tx=0).stop 2023-08-27T11:27:32.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.606+0000 7f970aa75700 1 -- 172.21.15.112:0/2257065591 reap_dead start 2023-08-27T11:27:32.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.606+0000 7f970c4d8700 1 -- 172.21.15.112:0/2257065591 shutdown_connections 2023-08-27T11:27:32.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.606+0000 7f970c4d8700 1 -- 172.21.15.112:0/2257065591 >> 172.21.15.112:0/2257065591 conn(0x7f97040d0000 msgr2=0x7f97040d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:32.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.606+0000 7f970c4d8700 1 -- 172.21.15.112:0/2257065591 shutdown_connections 2023-08-27T11:27:32.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.606+0000 7f970c4d8700 1 -- 172.21.15.112:0/2257065591 wait complete. 2023-08-27T11:27:32.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.607+0000 7f970c4d8700 1 Processor -- start 2023-08-27T11:27:32.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.607+0000 7f970c4d8700 1 -- start start 2023-08-27T11:27:32.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970c4d8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f0003a30 0x7f97041514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970c4d8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97040c8590 0x7f970415f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970c4d8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97041508b0 0x7f970415fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970c4d8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97041f30b0 con 0x7f96f0003a30 2023-08-27T11:27:32.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970c4d8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97041f3370 con 0x7f97040c8590 2023-08-27T11:27:32.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970c4d8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97041f3650 con 0x7f97041508b0 2023-08-27T11:27:32.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970a274700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f0003a30 0x7f97041514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970a274700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f0003a30 0x7f97041514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35004/0 (socket says 172.21.15.112:35004) 2023-08-27T11:27:32.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970a274700 1 -- 172.21.15.112:0/1222446815 learned_addr learned my addr 172.21.15.112:0/1222446815 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:32.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970a274700 1 -- 172.21.15.112:0/1222446815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97041508b0 msgr2=0x7f970415fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f9709a73700 1 --2- 172.21.15.112:0/1222446815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97040c8590 0x7f970415f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970a274700 1 --2- 172.21.15.112:0/1222446815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97041508b0 0x7f970415fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970a274700 1 -- 172.21.15.112:0/1222446815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97040c8590 msgr2=0x7f970415f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970a274700 1 --2- 172.21.15.112:0/1222446815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97040c8590 0x7f970415f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.608+0000 7f970a274700 1 -- 172.21.15.112:0/1222446815 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97041f3930 con 0x7f96f0003a30 2023-08-27T11:27:32.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.609+0000 7f970a274700 1 --2- 172.21.15.112:0/1222446815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f0003a30 0x7f97041514d0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f96f000bb10 tx=0x7f96f40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:32.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.609+0000 7f96faffd700 1 -- 172.21.15.112:0/1222446815 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f400e6c0 con 0x7f96f0003a30 2023-08-27T11:27:32.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.609+0000 7f96faffd700 1 -- 172.21.15.112:0/1222446815 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f4009ed0 con 0x7f96f0003a30 2023-08-27T11:27:32.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.609+0000 7f970c4d8700 1 -- 172.21.15.112:0/1222446815 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97041f3c10 con 0x7f96f0003a30 2023-08-27T11:27:32.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.609+0000 7f96faffd700 1 -- 172.21.15.112:0/1222446815 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4003d10 con 0x7f96f0003a30 2023-08-27T11:27:32.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.609+0000 7f970c4d8700 1 -- 172.21.15.112:0/1222446815 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97040716b0 con 0x7f96f0003a30 2023-08-27T11:27:32.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.610+0000 7f970c4d8700 1 -- 172.21.15.112:0/1222446815 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97041f3f80 con 0x7f96f0003a30 2023-08-27T11:27:32.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.610+0000 7f96faffd700 1 -- 172.21.15.112:0/1222446815 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f96f4013470 con 0x7f96f0003a30 2023-08-27T11:27:32.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.610+0000 7f96faffd700 1 --2- 172.21.15.112:0/1222446815 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f96e806a4d0 0x7f96e806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.611+0000 7f96faffd700 1 -- 172.21.15.112:0/1222446815 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f96f408a400 con 0x7f96f0003a30 2023-08-27T11:27:32.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.611+0000 7f9709a73700 1 --2- 172.21.15.112:0/1222446815 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f96e806a4d0 0x7f96e806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.611+0000 7f9709a73700 1 --2- 172.21.15.112:0/1222446815 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f96e806a4d0 0x7f96e806c990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f97041605f0 tx=0x7f9700009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:32.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.615+0000 7f96faffd700 1 -- 172.21.15.112:0/1222446815 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f96f40555a0 con 0x7f96f0003a30 2023-08-27T11:27:32.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.801+0000 7f970c4d8700 1 -- 172.21.15.112:0/1222446815 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97041f30b0 con 0x7f96e806a4d0 2023-08-27T11:27:32.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.810+0000 7f96faffd700 1 -- 172.21.15.112:0/1222446815 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312358 (secure 0 0 0) 0x7f97041f30b0 con 0x7f96e806a4d0 2023-08-27T11:27:32.816 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:32.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.814+0000 7f970c4d8700 1 -- 172.21.15.112:0/1222446815 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f96e806a4d0 msgr2=0x7f96e806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.814+0000 7f970c4d8700 1 --2- 172.21.15.112:0/1222446815 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f96e806a4d0 0x7f96e806c990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f97041605f0 tx=0x7f9700009040 comp rx=0 tx=0).stop 2023-08-27T11:27:32.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.814+0000 7f970c4d8700 1 -- 172.21.15.112:0/1222446815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f0003a30 msgr2=0x7f97041514d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.814+0000 7f970c4d8700 1 --2- 172.21.15.112:0/1222446815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f0003a30 0x7f97041514d0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f96f000bb10 tx=0x7f96f40047c0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.814+0000 7f970c4d8700 1 -- 172.21.15.112:0/1222446815 shutdown_connections 2023-08-27T11:27:32.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.814+0000 7f970c4d8700 1 --2- 172.21.15.112:0/1222446815 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f96e806a4d0 0x7f96e806c990 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.814+0000 7f970c4d8700 1 --2- 172.21.15.112:0/1222446815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97041508b0 0x7f970415fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.814+0000 7f970c4d8700 1 --2- 172.21.15.112:0/1222446815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f0003a30 0x7f97041514d0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.814+0000 7f970c4d8700 1 --2- 172.21.15.112:0/1222446815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97040c8590 0x7f970415f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.814+0000 7f970c4d8700 1 -- 172.21.15.112:0/1222446815 >> 172.21.15.112:0/1222446815 conn(0x7f97040d0000 msgr2=0x7f9704157950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:32.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.814+0000 7f970c4d8700 1 -- 172.21.15.112:0/1222446815 shutdown_connections 2023-08-27T11:27:32.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.814+0000 7f970c4d8700 1 -- 172.21.15.112:0/1222446815 wait complete. 2023-08-27T11:27:32.821 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:27:32.827 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2023-08-27T11:27:32.386136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261820,"kb_used_data":12356,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754712900,"statfs":{"total":773094113280,"available":772826009600,"internally_reserved":0,"allocated":12652544,"data_stored":6450467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.686352"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.714391+0000","last_change":"2023-08-27T11:27:28.721911+0000","last_active":"2023-08-27T11:27:29.714391+0000","last_peered":"2023-08-27T11:27:29.714391+0000","last_clean":"2023-08-27T11:27:29.714391+0000","last_became_active":"2023-08-27T11:27:28.721622+0000","last_became_peered":"2023-08-27T11:27:28.721622+0000","last_unstale":"2023-08-27T11:27:29.714391+0000","last_undegraded":"2023-08-27T11:27:29.714391+0000","last_fullsized":"2023-08-27T11:27:29.714391+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:34:15.793813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300616+0000","last_change":"2023-08-27T11:27:28.733803+0000","last_active":"2023-08-27T11:27:30.300616+0000","last_peered":"2023-08-27T11:27:30.300616+0000","last_clean":"2023-08-27T11:27:30.300616+0000","last_became_active":"2023-08-27T11:27:28.733521+0000","last_became_peered":"2023-08-27T11:27:28.733521+0000","last_unstale":"2023-08-27T11:27:30.300616+0000","last_undegraded":"2023-08-27T11:27:30.300616+0000","last_fullsized":"2023-08-27T11:27:30.300616+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:28:23.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715287+0000","last_change":"2023-08-27T11:27:28.722194+0000","last_active":"2023-08-27T11:27:29.715287+0000","last_peered":"2023-08-27T11:27:29.715287+0000","last_clean":"2023-08-27T11:27:29.715287+0000","last_became_active":"2023-08-27T11:27:28.721986+0000","last_became_peered":"2023-08-27T11:27:28.721986+0000","last_unstale":"2023-08-27T11:27:29.715287+0000","last_undegraded":"2023-08-27T11:27:29.715287+0000","last_fullsized":"2023-08-27T11:27:29.715287+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:24:26.197854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300303+0000","last_change":"2023-08-27T11:27:28.730569+0000","last_active":"2023-08-27T11:27:30.300303+0000","last_peered":"2023-08-27T11:27:30.300303+0000","last_clean":"2023-08-27T11:27:30.300303+0000","last_became_active":"2023-08-27T11:27:28.730273+0000","last_became_peered":"2023-08-27T11:27:28.730273+0000","last_unstale":"2023-08-27T11:27:30.300303+0000","last_undegraded":"2023-08-27T11:27:30.300303+0000","last_fullsized":"2023-08-27T11:27:30.300303+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:28:43.469961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300462+0000","last_change":"2023-08-27T11:27:28.721400+0000","last_active":"2023-08-27T11:27:30.300462+0000","last_peered":"2023-08-27T11:27:30.300462+0000","last_clean":"2023-08-27T11:27:30.300462+0000","last_became_active":"2023-08-27T11:27:28.720872+0000","last_became_peered":"2023-08-27T11:27:28.720872+0000","last_unstale":"2023-08-27T11:27:30.300462+0000","last_undegraded":"2023-08-27T11:27:30.300462+0000","last_fullsized":"2023-08-27T11:27:30.300462+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:37.403227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300847+0000","last_change":"2023-08-27T11:27:28.728027+0000","last_active":"2023-08-27T11:27:30.300847+0000","last_peered":"2023-08-27T11:27:30.300847+0000","last_clean":"2023-08-27T11:27:30.300847+0000","last_became_active":"2023-08-27T11:27:28.727728+0000","last_became_peered":"2023-08-27T11:27:28.727728+0000","last_unstale":"2023-08-27T11:27:30.300847+0000","last_undegraded":"2023-08-27T11:27:30.300847+0000","last_fullsized":"2023-08-27T11:27:30.300847+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:01:53.347873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300292+0000","last_change":"2023-08-27T11:27:28.744097+0000","last_active":"2023-08-27T11:27:30.300292+0000","last_peered":"2023-08-27T11:27:30.300292+0000","last_clean":"2023-08-27T11:27:30.300292+0000","last_became_active":"2023-08-27T11:27:28.740829+0000","last_became_peered":"2023-08-27T11:27:28.740829+0000","last_unstale":"2023-08-27T11:27:30.300292+0000","last_undegraded":"2023-08-27T11:27:30.300292+0000","last_fullsized":"2023-08-27T11:27:30.300292+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:23.555957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715856+0000","last_change":"2023-08-27T11:27:28.727125+0000","last_active":"2023-08-27T11:27:29.715856+0000","last_peered":"2023-08-27T11:27:29.715856+0000","last_clean":"2023-08-27T11:27:29.715856+0000","last_became_active":"2023-08-27T11:27:28.726861+0000","last_became_peered":"2023-08-27T11:27:28.726861+0000","last_unstale":"2023-08-27T11:27:29.715856+0000","last_undegraded":"2023-08-27T11:27:29.715856+0000","last_fullsized":"2023-08-27T11:27:29.715856+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:28:14.761948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301393+0000","last_change":"2023-08-27T11:27:28.733174+0000","last_active":"2023-08-27T11:27:30.301393+0000","last_peered":"2023-08-27T11:27:30.301393+0000","last_clean":"2023-08-27T11:27:30.301393+0000","last_became_active":"2023-08-27T11:27:28.732789+0000","last_became_peered":"2023-08-27T11:27:28.732789+0000","last_unstale":"2023-08-27T11:27:30.301393+0000","last_undegraded":"2023-08-27T11:27:30.301393+0000","last_fullsized":"2023-08-27T11:27:30.301393+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:10.830106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301435+0000","last_change":"2023-08-27T11:27:28.726784+0000","last_active":"2023-08-27T11:27:30.301435+0000","last_peered":"2023-08-27T11:27:30.301435+0000","last_clean":"2023-08-27T11:27:30.301435+0000","last_became_active":"2023-08-27T11:27:28.726515+0000","last_became_peered":"2023-08-27T11:27:28.726515+0000","last_unstale":"2023-08-27T11:27:30.301435+0000","last_undegraded":"2023-08-27T11:27:30.301435+0000","last_fullsized":"2023-08-27T11:27:30.301435+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:05.401079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300232+0000","last_change":"2023-08-27T11:27:28.736506+0000","last_active":"2023-08-27T11:27:30.300232+0000","last_peered":"2023-08-27T11:27:30.300232+0000","last_clean":"2023-08-27T11:27:30.300232+0000","last_became_active":"2023-08-27T11:27:28.736142+0000","last_became_peered":"2023-08-27T11:27:28.736142+0000","last_unstale":"2023-08-27T11:27:30.300232+0000","last_undegraded":"2023-08-27T11:27:30.300232+0000","last_fullsized":"2023-08-27T11:27:30.300232+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:38.106756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.714027+0000","last_change":"2023-08-27T11:27:28.722139+0000","last_active":"2023-08-27T11:27:29.714027+0000","last_peered":"2023-08-27T11:27:29.714027+0000","last_clean":"2023-08-27T11:27:29.714027+0000","last_became_active":"2023-08-27T11:27:28.721862+0000","last_became_peered":"2023-08-27T11:27:28.721862+0000","last_unstale":"2023-08-27T11:27:29.714027+0000","last_undegraded":"2023-08-27T11:27:29.714027+0000","last_fullsized":"2023-08-27T11:27:29.714027+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:16:03.259583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301068+0000","last_change":"2023-08-27T11:27:28.727583+0000","last_active":"2023-08-27T11:27:30.301068+0000","last_peered":"2023-08-27T11:27:30.301068+0000","last_clean":"2023-08-27T11:27:30.301068+0000","last_became_active":"2023-08-27T11:27:28.725833+0000","last_became_peered":"2023-08-27T11:27:28.725833+0000","last_unstale":"2023-08-27T11:27:30.301068+0000","last_undegraded":"2023-08-27T11:27:30.301068+0000","last_fullsized":"2023-08-27T11:27:30.301068+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:49:57.748777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.714124+0000","last_change":"2023-08-27T11:27:28.738604+0000","last_active":"2023-08-27T11:27:29.714124+0000","last_peered":"2023-08-27T11:27:29.714124+0000","last_clean":"2023-08-27T11:27:29.714124+0000","last_became_active":"2023-08-27T11:27:28.738248+0000","last_became_peered":"2023-08-27T11:27:28.738248+0000","last_unstale":"2023-08-27T11:27:29.714124+0000","last_undegraded":"2023-08-27T11:27:29.714124+0000","last_fullsized":"2023-08-27T11:27:29.714124+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:24:35.192028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713738+0000","last_change":"2023-08-27T11:27:28.743854+0000","last_active":"2023-08-27T11:27:29.713738+0000","last_peered":"2023-08-27T11:27:29.713738+0000","last_clean":"2023-08-27T11:27:29.713738+0000","last_became_active":"2023-08-27T11:27:28.743568+0000","last_became_peered":"2023-08-27T11:27:28.743568+0000","last_unstale":"2023-08-27T11:27:29.713738+0000","last_undegraded":"2023-08-27T11:27:29.713738+0000","last_fullsized":"2023-08-27T11:27:29.713738+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:58:54.705362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300900+0000","last_change":"2023-08-27T11:27:28.724622+0000","last_active":"2023-08-27T11:27:30.300900+0000","last_peered":"2023-08-27T11:27:30.300900+0000","last_clean":"2023-08-27T11:27:30.300900+0000","last_became_active":"2023-08-27T11:27:28.724379+0000","last_became_peered":"2023-08-27T11:27:28.724379+0000","last_unstale":"2023-08-27T11:27:30.300900+0000","last_undegraded":"2023-08-27T11:27:30.300900+0000","last_fullsized":"2023-08-27T11:27:30.300900+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:55:10.100708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301510+0000","last_change":"2023-08-27T11:27:28.722365+0000","last_active":"2023-08-27T11:27:30.301510+0000","last_peered":"2023-08-27T11:27:30.301510+0000","last_clean":"2023-08-27T11:27:30.301510+0000","last_became_active":"2023-08-27T11:27:28.722097+0000","last_became_peered":"2023-08-27T11:27:28.722097+0000","last_unstale":"2023-08-27T11:27:30.301510+0000","last_undegraded":"2023-08-27T11:27:30.301510+0000","last_fullsized":"2023-08-27T11:27:30.301510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:52.622388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300311+0000","last_change":"2023-08-27T11:27:28.723197+0000","last_active":"2023-08-27T11:27:30.300311+0000","last_peered":"2023-08-27T11:27:30.300311+0000","last_clean":"2023-08-27T11:27:30.300311+0000","last_became_active":"2023-08-27T11:27:28.722937+0000","last_became_peered":"2023-08-27T11:27:28.722937+0000","last_unstale":"2023-08-27T11:27:30.300311+0000","last_undegraded":"2023-08-27T11:27:30.300311+0000","last_fullsized":"2023-08-27T11:27:30.300311+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:32:23.412658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301392+0000","last_change":"2023-08-27T11:27:28.746158+0000","last_active":"2023-08-27T11:27:30.301392+0000","last_peered":"2023-08-27T11:27:30.301392+0000","last_clean":"2023-08-27T11:27:30.301392+0000","last_became_active":"2023-08-27T11:27:28.743787+0000","last_became_peered":"2023-08-27T11:27:28.743787+0000","last_unstale":"2023-08-27T11:27:30.301392+0000","last_undegraded":"2023-08-27T11:27:30.301392+0000","last_fullsized":"2023-08-27T11:27:30.301392+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:21:16.525932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300755+0000","last_change":"2023-08-27T11:27:28.741346+0000","last_active":"2023-08-27T11:27:30.300755+0000","last_peered":"2023-08-27T11:27:30.300755+0000","last_clean":"2023-08-27T11:27:30.300755+0000","last_became_active":"2023-08-27T11:27:28.741036+0000","last_became_peered":"2023-08-27T11:27:28.741036+0000","last_unstale":"2023-08-27T11:27:30.300755+0000","last_undegraded":"2023-08-27T11:27:30.300755+0000","last_fullsized":"2023-08-27T11:27:30.300755+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:36:16.575366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300446+0000","last_change":"2023-08-27T11:27:28.730040+0000","last_active":"2023-08-27T11:27:30.300446+0000","last_peered":"2023-08-27T11:27:30.300446+0000","last_clean":"2023-08-27T11:27:30.300446+0000","last_became_active":"2023-08-27T11:27:28.729763+0000","last_became_peered":"2023-08-27T11:27:28.729763+0000","last_unstale":"2023-08-27T11:27:30.300446+0000","last_undegraded":"2023-08-27T11:27:30.300446+0000","last_fullsized":"2023-08-27T11:27:30.300446+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:32:23.412658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713903+0000","last_change":"2023-08-27T11:27:28.726440+0000","last_active":"2023-08-27T11:27:29.713903+0000","last_peered":"2023-08-27T11:27:29.713903+0000","last_clean":"2023-08-27T11:27:29.713903+0000","last_became_active":"2023-08-27T11:27:28.726160+0000","last_became_peered":"2023-08-27T11:27:28.726160+0000","last_unstale":"2023-08-27T11:27:29.713903+0000","last_undegraded":"2023-08-27T11:27:29.713903+0000","last_fullsized":"2023-08-27T11:27:29.713903+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:32:44.006355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300266+0000","last_change":"2023-08-27T11:27:28.741507+0000","last_active":"2023-08-27T11:27:30.300266+0000","last_peered":"2023-08-27T11:27:30.300266+0000","last_clean":"2023-08-27T11:27:30.300266+0000","last_became_active":"2023-08-27T11:27:28.741182+0000","last_became_peered":"2023-08-27T11:27:28.741182+0000","last_unstale":"2023-08-27T11:27:30.300266+0000","last_undegraded":"2023-08-27T11:27:30.300266+0000","last_fullsized":"2023-08-27T11:27:30.300266+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:43:05.337653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300078+0000","last_change":"2023-08-27T11:27:28.726111+0000","last_active":"2023-08-27T11:27:30.300078+0000","last_peered":"2023-08-27T11:27:30.300078+0000","last_clean":"2023-08-27T11:27:30.300078+0000","last_became_active":"2023-08-27T11:27:28.725777+0000","last_became_peered":"2023-08-27T11:27:28.725777+0000","last_unstale":"2023-08-27T11:27:30.300078+0000","last_undegraded":"2023-08-27T11:27:30.300078+0000","last_fullsized":"2023-08-27T11:27:30.300078+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:07:14.947676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715241+0000","last_change":"2023-08-27T11:27:28.724936+0000","last_active":"2023-08-27T11:27:29.715241+0000","last_peered":"2023-08-27T11:27:29.715241+0000","last_clean":"2023-08-27T11:27:29.715241+0000","last_became_active":"2023-08-27T11:27:28.724673+0000","last_became_peered":"2023-08-27T11:27:28.724673+0000","last_unstale":"2023-08-27T11:27:29.715241+0000","last_undegraded":"2023-08-27T11:27:29.715241+0000","last_fullsized":"2023-08-27T11:27:29.715241+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:38:02.890484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715496+0000","last_change":"2023-08-27T11:27:28.740166+0000","last_active":"2023-08-27T11:27:29.715496+0000","last_peered":"2023-08-27T11:27:29.715496+0000","last_clean":"2023-08-27T11:27:29.715496+0000","last_became_active":"2023-08-27T11:27:28.739685+0000","last_became_peered":"2023-08-27T11:27:28.739685+0000","last_unstale":"2023-08-27T11:27:29.715496+0000","last_undegraded":"2023-08-27T11:27:29.715496+0000","last_fullsized":"2023-08-27T11:27:29.715496+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:41:35.903009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301299+0000","last_change":"2023-08-27T11:27:28.745245+0000","last_active":"2023-08-27T11:27:30.301299+0000","last_peered":"2023-08-27T11:27:30.301299+0000","last_clean":"2023-08-27T11:27:30.301299+0000","last_became_active":"2023-08-27T11:27:28.743764+0000","last_became_peered":"2023-08-27T11:27:28.743764+0000","last_unstale":"2023-08-27T11:27:30.301299+0000","last_undegraded":"2023-08-27T11:27:30.301299+0000","last_fullsized":"2023-08-27T11:27:30.301299+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:38:55.395148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300754+0000","last_change":"2023-08-27T11:27:28.723904+0000","last_active":"2023-08-27T11:27:30.300754+0000","last_peered":"2023-08-27T11:27:30.300754+0000","last_clean":"2023-08-27T11:27:30.300754+0000","last_became_active":"2023-08-27T11:27:28.723641+0000","last_became_peered":"2023-08-27T11:27:28.723641+0000","last_unstale":"2023-08-27T11:27:30.300754+0000","last_undegraded":"2023-08-27T11:27:30.300754+0000","last_fullsized":"2023-08-27T11:27:30.300754+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:37:08.574504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716985+0000","last_change":"2023-08-27T11:27:28.725708+0000","last_active":"2023-08-27T11:27:29.716985+0000","last_peered":"2023-08-27T11:27:29.716985+0000","last_clean":"2023-08-27T11:27:29.716985+0000","last_became_active":"2023-08-27T11:27:28.725458+0000","last_became_peered":"2023-08-27T11:27:28.725458+0000","last_unstale":"2023-08-27T11:27:29.716985+0000","last_undegraded":"2023-08-27T11:27:29.716985+0000","last_fullsized":"2023-08-27T11:27:29.716985+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:27.276966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715595+0000","last_change":"2023-08-27T11:27:28.724498+0000","last_active":"2023-08-27T11:27:29.715595+0000","last_peered":"2023-08-27T11:27:29.715595+0000","last_clean":"2023-08-27T11:27:29.715595+0000","last_became_active":"2023-08-27T11:27:28.724212+0000","last_became_peered":"2023-08-27T11:27:28.724212+0000","last_unstale":"2023-08-27T11:27:29.715595+0000","last_undegraded":"2023-08-27T11:27:29.715595+0000","last_fullsized":"2023-08-27T11:27:29.715595+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:38.342129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715960+0000","last_change":"2023-08-27T11:27:28.725778+0000","last_active":"2023-08-27T11:27:29.715960+0000","last_peered":"2023-08-27T11:27:29.715960+0000","last_clean":"2023-08-27T11:27:29.715960+0000","last_became_active":"2023-08-27T11:27:28.725503+0000","last_became_peered":"2023-08-27T11:27:28.725503+0000","last_unstale":"2023-08-27T11:27:29.715960+0000","last_undegraded":"2023-08-27T11:27:29.715960+0000","last_fullsized":"2023-08-27T11:27:29.715960+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:02:07.186367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300448+0000","last_change":"2023-08-27T11:27:28.724944+0000","last_active":"2023-08-27T11:27:30.300448+0000","last_peered":"2023-08-27T11:27:30.300448+0000","last_clean":"2023-08-27T11:27:30.300448+0000","last_became_active":"2023-08-27T11:27:28.724643+0000","last_became_peered":"2023-08-27T11:27:28.724643+0000","last_unstale":"2023-08-27T11:27:30.300448+0000","last_undegraded":"2023-08-27T11:27:30.300448+0000","last_fullsized":"2023-08-27T11:27:30.300448+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:09:06.818940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716013+0000","last_change":"2023-08-27T11:27:28.738972+0000","last_active":"2023-08-27T11:27:29.716013+0000","last_peered":"2023-08-27T11:27:29.716013+0000","last_clean":"2023-08-27T11:27:29.716013+0000","last_became_active":"2023-08-27T11:27:28.738673+0000","last_became_peered":"2023-08-27T11:27:28.738673+0000","last_unstale":"2023-08-27T11:27:29.716013+0000","last_undegraded":"2023-08-27T11:27:29.716013+0000","last_fullsized":"2023-08-27T11:27:29.716013+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:35:48.007661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"113'4","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715428+0000","last_change":"2023-08-27T11:27:27.726870+0000","last_active":"2023-08-27T11:27:29.715428+0000","last_peered":"2023-08-27T11:27:29.715428+0000","last_clean":"2023-08-27T11:27:29.715428+0000","last_became_active":"2023-08-27T11:27:27.726617+0000","last_became_peered":"2023-08-27T11:27:27.726617+0000","last_unstale":"2023-08-27T11:27:29.715428+0000","last_undegraded":"2023-08-27T11:27:29.715428+0000","last_fullsized":"2023-08-27T11:27:29.715428+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:53:42.044040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713784+0000","last_change":"2023-08-27T11:27:28.733353+0000","last_active":"2023-08-27T11:27:29.713784+0000","last_peered":"2023-08-27T11:27:29.713784+0000","last_clean":"2023-08-27T11:27:29.713784+0000","last_became_active":"2023-08-27T11:27:28.733045+0000","last_became_peered":"2023-08-27T11:27:28.733045+0000","last_unstale":"2023-08-27T11:27:29.713784+0000","last_undegraded":"2023-08-27T11:27:29.713784+0000","last_fullsized":"2023-08-27T11:27:29.713784+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:12:21.038437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"113'1","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300624+0000","last_change":"2023-08-27T11:27:27.727923+0000","last_active":"2023-08-27T11:27:30.300624+0000","last_peered":"2023-08-27T11:27:30.300624+0000","last_clean":"2023-08-27T11:27:30.300624+0000","last_became_active":"2023-08-27T11:27:27.727656+0000","last_became_peered":"2023-08-27T11:27:27.727656+0000","last_unstale":"2023-08-27T11:27:30.300624+0000","last_undegraded":"2023-08-27T11:27:30.300624+0000","last_fullsized":"2023-08-27T11:27:30.300624+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:08:25.007899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300668+0000","last_change":"2023-08-27T11:27:28.741777+0000","last_active":"2023-08-27T11:27:30.300668+0000","last_peered":"2023-08-27T11:27:30.300668+0000","last_clean":"2023-08-27T11:27:30.300668+0000","last_became_active":"2023-08-27T11:27:28.741194+0000","last_became_peered":"2023-08-27T11:27:28.741194+0000","last_unstale":"2023-08-27T11:27:30.300668+0000","last_undegraded":"2023-08-27T11:27:30.300668+0000","last_fullsized":"2023-08-27T11:27:30.300668+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:03.492761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"113'2","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301180+0000","last_change":"2023-08-27T11:27:27.733316+0000","last_active":"2023-08-27T11:27:30.301180+0000","last_peered":"2023-08-27T11:27:30.301180+0000","last_clean":"2023-08-27T11:27:30.301180+0000","last_became_active":"2023-08-27T11:27:27.732469+0000","last_became_peered":"2023-08-27T11:27:27.732469+0000","last_unstale":"2023-08-27T11:27:30.301180+0000","last_undegraded":"2023-08-27T11:27:30.301180+0000","last_fullsized":"2023-08-27T11:27:30.301180+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:27:35.126554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713943+0000","last_change":"2023-08-27T11:27:28.743781+0000","last_active":"2023-08-27T11:27:29.713943+0000","last_peered":"2023-08-27T11:27:29.713943+0000","last_clean":"2023-08-27T11:27:29.713943+0000","last_became_active":"2023-08-27T11:27:28.743484+0000","last_became_peered":"2023-08-27T11:27:28.743484+0000","last_unstale":"2023-08-27T11:27:29.713943+0000","last_undegraded":"2023-08-27T11:27:29.713943+0000","last_fullsized":"2023-08-27T11:27:29.713943+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:43:38.690558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"113'2","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300788+0000","last_change":"2023-08-27T11:27:27.732724+0000","last_active":"2023-08-27T11:27:30.300788+0000","last_peered":"2023-08-27T11:27:30.300788+0000","last_clean":"2023-08-27T11:27:30.300788+0000","last_became_active":"2023-08-27T11:27:27.732421+0000","last_became_peered":"2023-08-27T11:27:27.732421+0000","last_unstale":"2023-08-27T11:27:30.300788+0000","last_undegraded":"2023-08-27T11:27:30.300788+0000","last_fullsized":"2023-08-27T11:27:30.300788+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:29.547142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716503+0000","last_change":"2023-08-27T11:27:28.726152+0000","last_active":"2023-08-27T11:27:29.716503+0000","last_peered":"2023-08-27T11:27:29.716503+0000","last_clean":"2023-08-27T11:27:29.716503+0000","last_became_active":"2023-08-27T11:27:28.725589+0000","last_became_peered":"2023-08-27T11:27:28.725589+0000","last_unstale":"2023-08-27T11:27:29.716503+0000","last_undegraded":"2023-08-27T11:27:29.716503+0000","last_fullsized":"2023-08-27T11:27:29.716503+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:08:38.551263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"113'1","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716257+0000","last_change":"2023-08-27T11:27:27.735989+0000","last_active":"2023-08-27T11:27:29.716257+0000","last_peered":"2023-08-27T11:27:29.716257+0000","last_clean":"2023-08-27T11:27:29.716257+0000","last_became_active":"2023-08-27T11:27:27.735725+0000","last_became_peered":"2023-08-27T11:27:27.735725+0000","last_unstale":"2023-08-27T11:27:29.716257+0000","last_undegraded":"2023-08-27T11:27:29.716257+0000","last_fullsized":"2023-08-27T11:27:29.716257+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:41:48.682595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716585+0000","last_change":"2023-08-27T11:27:28.728523+0000","last_active":"2023-08-27T11:27:29.716585+0000","last_peered":"2023-08-27T11:27:29.716585+0000","last_clean":"2023-08-27T11:27:29.716585+0000","last_became_active":"2023-08-27T11:27:28.727638+0000","last_became_peered":"2023-08-27T11:27:28.727638+0000","last_unstale":"2023-08-27T11:27:29.716585+0000","last_undegraded":"2023-08-27T11:27:29.716585+0000","last_fullsized":"2023-08-27T11:27:29.716585+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:57.063813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"113'1","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300156+0000","last_change":"2023-08-27T11:27:27.718129+0000","last_active":"2023-08-27T11:27:30.300156+0000","last_peered":"2023-08-27T11:27:30.300156+0000","last_clean":"2023-08-27T11:27:30.300156+0000","last_became_active":"2023-08-27T11:27:27.717804+0000","last_became_peered":"2023-08-27T11:27:27.717804+0000","last_unstale":"2023-08-27T11:27:30.300156+0000","last_undegraded":"2023-08-27T11:27:30.300156+0000","last_fullsized":"2023-08-27T11:27:30.300156+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:28:42.464473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300435+0000","last_change":"2023-08-27T11:27:28.739529+0000","last_active":"2023-08-27T11:27:30.300435+0000","last_peered":"2023-08-27T11:27:30.300435+0000","last_clean":"2023-08-27T11:27:30.300435+0000","last_became_active":"2023-08-27T11:27:28.739083+0000","last_became_peered":"2023-08-27T11:27:28.739083+0000","last_unstale":"2023-08-27T11:27:30.300435+0000","last_undegraded":"2023-08-27T11:27:30.300435+0000","last_fullsized":"2023-08-27T11:27:30.300435+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:28:23.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300803+0000","last_change":"2023-08-27T11:27:27.729848+0000","last_active":"2023-08-27T11:27:30.300803+0000","last_peered":"2023-08-27T11:27:30.300803+0000","last_clean":"2023-08-27T11:27:30.300803+0000","last_became_active":"2023-08-27T11:27:27.729580+0000","last_became_peered":"2023-08-27T11:27:27.729580+0000","last_unstale":"2023-08-27T11:27:30.300803+0000","last_undegraded":"2023-08-27T11:27:30.300803+0000","last_fullsized":"2023-08-27T11:27:30.300803+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:50:19.805790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301045+0000","last_change":"2023-08-27T11:27:28.739636+0000","last_active":"2023-08-27T11:27:30.301045+0000","last_peered":"2023-08-27T11:27:30.301045+0000","last_clean":"2023-08-27T11:27:30.301045+0000","last_became_active":"2023-08-27T11:27:28.739331+0000","last_became_peered":"2023-08-27T11:27:28.739331+0000","last_unstale":"2023-08-27T11:27:30.301045+0000","last_undegraded":"2023-08-27T11:27:30.301045+0000","last_fullsized":"2023-08-27T11:27:30.301045+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:51:52.004048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.714974+0000","last_change":"2023-08-27T11:27:27.726696+0000","last_active":"2023-08-27T11:27:29.714974+0000","last_peered":"2023-08-27T11:27:29.714974+0000","last_clean":"2023-08-27T11:27:29.714974+0000","last_became_active":"2023-08-27T11:27:27.725404+0000","last_became_peered":"2023-08-27T11:27:27.725404+0000","last_unstale":"2023-08-27T11:27:29.714974+0000","last_undegraded":"2023-08-27T11:27:29.714974+0000","last_fullsized":"2023-08-27T11:27:29.714974+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:33:06.170408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715133+0000","last_change":"2023-08-27T11:27:28.728224+0000","last_active":"2023-08-27T11:27:29.715133+0000","last_peered":"2023-08-27T11:27:29.715133+0000","last_clean":"2023-08-27T11:27:29.715133+0000","last_became_active":"2023-08-27T11:27:28.727758+0000","last_became_peered":"2023-08-27T11:27:28.727758+0000","last_unstale":"2023-08-27T11:27:29.715133+0000","last_undegraded":"2023-08-27T11:27:29.715133+0000","last_fullsized":"2023-08-27T11:27:29.715133+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:14:19.476647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"113'1","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300761+0000","last_change":"2023-08-27T11:27:27.738134+0000","last_active":"2023-08-27T11:27:30.300761+0000","last_peered":"2023-08-27T11:27:30.300761+0000","last_clean":"2023-08-27T11:27:30.300761+0000","last_became_active":"2023-08-27T11:27:27.737875+0000","last_became_peered":"2023-08-27T11:27:27.737875+0000","last_unstale":"2023-08-27T11:27:30.300761+0000","last_undegraded":"2023-08-27T11:27:30.300761+0000","last_fullsized":"2023-08-27T11:27:30.300761+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:08:25.007899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301248+0000","last_change":"2023-08-27T11:27:28.740180+0000","last_active":"2023-08-27T11:27:30.301248+0000","last_peered":"2023-08-27T11:27:30.301248+0000","last_clean":"2023-08-27T11:27:30.301248+0000","last_became_active":"2023-08-27T11:27:28.739919+0000","last_became_peered":"2023-08-27T11:27:28.739919+0000","last_unstale":"2023-08-27T11:27:30.301248+0000","last_undegraded":"2023-08-27T11:27:30.301248+0000","last_fullsized":"2023-08-27T11:27:30.301248+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:40.770172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"113'2","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715561+0000","last_change":"2023-08-27T11:27:27.726126+0000","last_active":"2023-08-27T11:27:29.715561+0000","last_peered":"2023-08-27T11:27:29.715561+0000","last_clean":"2023-08-27T11:27:29.715561+0000","last_became_active":"2023-08-27T11:27:27.725842+0000","last_became_peered":"2023-08-27T11:27:27.725842+0000","last_unstale":"2023-08-27T11:27:29.715561+0000","last_undegraded":"2023-08-27T11:27:29.715561+0000","last_fullsized":"2023-08-27T11:27:29.715561+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:22:33.212837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301408+0000","last_change":"2023-08-27T11:27:28.740851+0000","last_active":"2023-08-27T11:27:30.301408+0000","last_peered":"2023-08-27T11:27:30.301408+0000","last_clean":"2023-08-27T11:27:30.301408+0000","last_became_active":"2023-08-27T11:27:28.740529+0000","last_became_peered":"2023-08-27T11:27:28.740529+0000","last_unstale":"2023-08-27T11:27:30.301408+0000","last_undegraded":"2023-08-27T11:27:30.301408+0000","last_fullsized":"2023-08-27T11:27:30.301408+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:34:16.231819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"113'1","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301237+0000","last_change":"2023-08-27T11:27:27.729967+0000","last_active":"2023-08-27T11:27:30.301237+0000","last_peered":"2023-08-27T11:27:30.301237+0000","last_clean":"2023-08-27T11:27:30.301237+0000","last_became_active":"2023-08-27T11:27:27.729709+0000","last_became_peered":"2023-08-27T11:27:27.729709+0000","last_unstale":"2023-08-27T11:27:30.301237+0000","last_undegraded":"2023-08-27T11:27:30.301237+0000","last_fullsized":"2023-08-27T11:27:30.301237+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:18:41.998717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300838+0000","last_change":"2023-08-27T11:27:28.727499+0000","last_active":"2023-08-27T11:27:30.300838+0000","last_peered":"2023-08-27T11:27:30.300838+0000","last_clean":"2023-08-27T11:27:30.300838+0000","last_became_active":"2023-08-27T11:27:28.727109+0000","last_became_peered":"2023-08-27T11:27:28.727109+0000","last_unstale":"2023-08-27T11:27:30.300838+0000","last_undegraded":"2023-08-27T11:27:30.300838+0000","last_fullsized":"2023-08-27T11:27:30.300838+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:22:20.257671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300285+0000","last_change":"2023-08-27T11:27:27.722235+0000","last_active":"2023-08-27T11:27:30.300285+0000","last_peered":"2023-08-27T11:27:30.300285+0000","last_clean":"2023-08-27T11:27:30.300285+0000","last_became_active":"2023-08-27T11:27:27.721970+0000","last_became_peered":"2023-08-27T11:27:27.721970+0000","last_unstale":"2023-08-27T11:27:30.300285+0000","last_undegraded":"2023-08-27T11:27:30.300285+0000","last_fullsized":"2023-08-27T11:27:30.300285+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:48:12.711894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301115+0000","last_change":"2023-08-27T11:27:28.723187+0000","last_active":"2023-08-27T11:27:30.301115+0000","last_peered":"2023-08-27T11:27:30.301115+0000","last_clean":"2023-08-27T11:27:30.301115+0000","last_became_active":"2023-08-27T11:27:28.722906+0000","last_became_peered":"2023-08-27T11:27:28.722906+0000","last_unstale":"2023-08-27T11:27:30.301115+0000","last_undegraded":"2023-08-27T11:27:30.301115+0000","last_fullsized":"2023-08-27T11:27:30.301115+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:39:59.704373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"113'2","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716324+0000","last_change":"2023-08-27T11:27:27.723983+0000","last_active":"2023-08-27T11:27:29.716324+0000","last_peered":"2023-08-27T11:27:29.716324+0000","last_clean":"2023-08-27T11:27:29.716324+0000","last_became_active":"2023-08-27T11:27:27.723680+0000","last_became_peered":"2023-08-27T11:27:27.723680+0000","last_unstale":"2023-08-27T11:27:29.716324+0000","last_undegraded":"2023-08-27T11:27:29.716324+0000","last_fullsized":"2023-08-27T11:27:29.716324+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:32:40.666735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713935+0000","last_change":"2023-08-27T11:27:28.746409+0000","last_active":"2023-08-27T11:27:29.713935+0000","last_peered":"2023-08-27T11:27:29.713935+0000","last_clean":"2023-08-27T11:27:29.713935+0000","last_became_active":"2023-08-27T11:27:28.746109+0000","last_became_peered":"2023-08-27T11:27:28.746109+0000","last_unstale":"2023-08-27T11:27:29.713935+0000","last_undegraded":"2023-08-27T11:27:29.713935+0000","last_fullsized":"2023-08-27T11:27:29.713935+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:40.531052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713994+0000","last_change":"2023-08-27T11:27:27.726812+0000","last_active":"2023-08-27T11:27:29.713994+0000","last_peered":"2023-08-27T11:27:29.713994+0000","last_clean":"2023-08-27T11:27:29.713994+0000","last_became_active":"2023-08-27T11:27:27.726561+0000","last_became_peered":"2023-08-27T11:27:27.726561+0000","last_unstale":"2023-08-27T11:27:29.713994+0000","last_undegraded":"2023-08-27T11:27:29.713994+0000","last_fullsized":"2023-08-27T11:27:29.713994+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:09:23.080018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300584+0000","last_change":"2023-08-27T11:27:28.727953+0000","last_active":"2023-08-27T11:27:30.300584+0000","last_peered":"2023-08-27T11:27:30.300584+0000","last_clean":"2023-08-27T11:27:30.300584+0000","last_became_active":"2023-08-27T11:27:28.727657+0000","last_became_peered":"2023-08-27T11:27:28.727657+0000","last_unstale":"2023-08-27T11:27:30.300584+0000","last_undegraded":"2023-08-27T11:27:30.300584+0000","last_fullsized":"2023-08-27T11:27:30.300584+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:18:07.138304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301053+0000","last_change":"2023-08-27T11:27:27.732818+0000","last_active":"2023-08-27T11:27:30.301053+0000","last_peered":"2023-08-27T11:27:30.301053+0000","last_clean":"2023-08-27T11:27:30.301053+0000","last_became_active":"2023-08-27T11:27:27.732544+0000","last_became_peered":"2023-08-27T11:27:27.732544+0000","last_unstale":"2023-08-27T11:27:30.301053+0000","last_undegraded":"2023-08-27T11:27:30.301053+0000","last_fullsized":"2023-08-27T11:27:30.301053+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:37.233701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300663+0000","last_change":"2023-08-27T11:27:28.736577+0000","last_active":"2023-08-27T11:27:30.300663+0000","last_peered":"2023-08-27T11:27:30.300663+0000","last_clean":"2023-08-27T11:27:30.300663+0000","last_became_active":"2023-08-27T11:27:28.736220+0000","last_became_peered":"2023-08-27T11:27:28.736220+0000","last_unstale":"2023-08-27T11:27:30.300663+0000","last_undegraded":"2023-08-27T11:27:30.300663+0000","last_fullsized":"2023-08-27T11:27:30.300663+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:37.403227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301116+0000","last_change":"2023-08-27T11:27:27.738107+0000","last_active":"2023-08-27T11:27:30.301116+0000","last_peered":"2023-08-27T11:27:30.301116+0000","last_clean":"2023-08-27T11:27:30.301116+0000","last_became_active":"2023-08-27T11:27:27.737815+0000","last_became_peered":"2023-08-27T11:27:27.737815+0000","last_unstale":"2023-08-27T11:27:30.301116+0000","last_undegraded":"2023-08-27T11:27:30.301116+0000","last_fullsized":"2023-08-27T11:27:30.301116+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:24:14.996980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301137+0000","last_change":"2023-08-27T11:27:28.739803+0000","last_active":"2023-08-27T11:27:30.301137+0000","last_peered":"2023-08-27T11:27:30.301137+0000","last_clean":"2023-08-27T11:27:30.301137+0000","last_became_active":"2023-08-27T11:27:28.739028+0000","last_became_peered":"2023-08-27T11:27:28.739028+0000","last_unstale":"2023-08-27T11:27:30.301137+0000","last_undegraded":"2023-08-27T11:27:30.301137+0000","last_fullsized":"2023-08-27T11:27:30.301137+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:43:05.337653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"113'2","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713980+0000","last_change":"2023-08-27T11:27:27.723989+0000","last_active":"2023-08-27T11:27:29.713980+0000","last_peered":"2023-08-27T11:27:29.713980+0000","last_clean":"2023-08-27T11:27:29.713980+0000","last_became_active":"2023-08-27T11:27:27.723612+0000","last_became_peered":"2023-08-27T11:27:27.723612+0000","last_unstale":"2023-08-27T11:27:29.713980+0000","last_undegraded":"2023-08-27T11:27:29.713980+0000","last_fullsized":"2023-08-27T11:27:29.713980+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:06:59.013539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715062+0000","last_change":"2023-08-27T11:27:28.727442+0000","last_active":"2023-08-27T11:27:29.715062+0000","last_peered":"2023-08-27T11:27:29.715062+0000","last_clean":"2023-08-27T11:27:29.715062+0000","last_became_active":"2023-08-27T11:27:28.727145+0000","last_became_peered":"2023-08-27T11:27:28.727145+0000","last_unstale":"2023-08-27T11:27:29.715062+0000","last_undegraded":"2023-08-27T11:27:29.715062+0000","last_fullsized":"2023-08-27T11:27:29.715062+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:03:40.178562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716885+0000","last_change":"2023-08-27T11:27:27.728604+0000","last_active":"2023-08-27T11:27:29.716885+0000","last_peered":"2023-08-27T11:27:29.716885+0000","last_clean":"2023-08-27T11:27:29.716885+0000","last_became_active":"2023-08-27T11:27:27.728297+0000","last_became_peered":"2023-08-27T11:27:27.728297+0000","last_unstale":"2023-08-27T11:27:29.716885+0000","last_undegraded":"2023-08-27T11:27:29.716885+0000","last_fullsized":"2023-08-27T11:27:29.716885+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:01.533491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301332+0000","last_change":"2023-08-27T11:27:28.745857+0000","last_active":"2023-08-27T11:27:30.301332+0000","last_peered":"2023-08-27T11:27:30.301332+0000","last_clean":"2023-08-27T11:27:30.301332+0000","last_became_active":"2023-08-27T11:27:28.745614+0000","last_became_peered":"2023-08-27T11:27:28.745614+0000","last_unstale":"2023-08-27T11:27:30.301332+0000","last_undegraded":"2023-08-27T11:27:30.301332+0000","last_fullsized":"2023-08-27T11:27:30.301332+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:16:04.566393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716593+0000","last_change":"2023-08-27T11:27:27.726039+0000","last_active":"2023-08-27T11:27:29.716593+0000","last_peered":"2023-08-27T11:27:29.716593+0000","last_clean":"2023-08-27T11:27:29.716593+0000","last_became_active":"2023-08-27T11:27:27.725769+0000","last_became_peered":"2023-08-27T11:27:27.725769+0000","last_unstale":"2023-08-27T11:27:29.716593+0000","last_undegraded":"2023-08-27T11:27:29.716593+0000","last_fullsized":"2023-08-27T11:27:29.716593+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:27:30.629283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300612+0000","last_change":"2023-08-27T11:27:28.724809+0000","last_active":"2023-08-27T11:27:30.300612+0000","last_peered":"2023-08-27T11:27:30.300612+0000","last_clean":"2023-08-27T11:27:30.300612+0000","last_became_active":"2023-08-27T11:27:28.724540+0000","last_became_peered":"2023-08-27T11:27:28.724540+0000","last_unstale":"2023-08-27T11:27:30.300612+0000","last_undegraded":"2023-08-27T11:27:30.300612+0000","last_fullsized":"2023-08-27T11:27:30.300612+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:01:58.599489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301062+0000","last_change":"2023-08-27T11:27:27.732726+0000","last_active":"2023-08-27T11:27:30.301062+0000","last_peered":"2023-08-27T11:27:30.301062+0000","last_clean":"2023-08-27T11:27:30.301062+0000","last_became_active":"2023-08-27T11:27:27.732460+0000","last_became_peered":"2023-08-27T11:27:27.732460+0000","last_unstale":"2023-08-27T11:27:30.301062+0000","last_undegraded":"2023-08-27T11:27:30.301062+0000","last_fullsized":"2023-08-27T11:27:30.301062+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:08:02.907856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301034+0000","last_change":"2023-08-27T11:27:28.723945+0000","last_active":"2023-08-27T11:27:30.301034+0000","last_peered":"2023-08-27T11:27:30.301034+0000","last_clean":"2023-08-27T11:27:30.301034+0000","last_became_active":"2023-08-27T11:27:28.723684+0000","last_became_peered":"2023-08-27T11:27:28.723684+0000","last_unstale":"2023-08-27T11:27:30.301034+0000","last_undegraded":"2023-08-27T11:27:30.301034+0000","last_fullsized":"2023-08-27T11:27:30.301034+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:16:01.763110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300357+0000","last_change":"2023-08-27T11:27:27.730150+0000","last_active":"2023-08-27T11:27:30.300357+0000","last_peered":"2023-08-27T11:27:30.300357+0000","last_clean":"2023-08-27T11:27:30.300357+0000","last_became_active":"2023-08-27T11:27:27.729865+0000","last_became_peered":"2023-08-27T11:27:27.729865+0000","last_unstale":"2023-08-27T11:27:30.300357+0000","last_undegraded":"2023-08-27T11:27:30.300357+0000","last_fullsized":"2023-08-27T11:27:30.300357+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:48:12.711894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300976+0000","last_change":"2023-08-27T11:27:28.722696+0000","last_active":"2023-08-27T11:27:30.300976+0000","last_peered":"2023-08-27T11:27:30.300976+0000","last_clean":"2023-08-27T11:27:30.300976+0000","last_became_active":"2023-08-27T11:27:28.722445+0000","last_became_peered":"2023-08-27T11:27:28.722445+0000","last_unstale":"2023-08-27T11:27:30.300976+0000","last_undegraded":"2023-08-27T11:27:30.300976+0000","last_fullsized":"2023-08-27T11:27:30.300976+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:17:52.218419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301026+0000","last_change":"2023-08-27T11:27:27.733453+0000","last_active":"2023-08-27T11:27:30.301026+0000","last_peered":"2023-08-27T11:27:30.301026+0000","last_clean":"2023-08-27T11:27:30.301026+0000","last_became_active":"2023-08-27T11:27:27.732510+0000","last_became_peered":"2023-08-27T11:27:27.732510+0000","last_unstale":"2023-08-27T11:27:30.301026+0000","last_undegraded":"2023-08-27T11:27:30.301026+0000","last_fullsized":"2023-08-27T11:27:30.301026+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:43:07.897186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301299+0000","last_change":"2023-08-27T11:27:28.739728+0000","last_active":"2023-08-27T11:27:30.301299+0000","last_peered":"2023-08-27T11:27:30.301299+0000","last_clean":"2023-08-27T11:27:30.301299+0000","last_became_active":"2023-08-27T11:27:28.739440+0000","last_became_peered":"2023-08-27T11:27:28.739440+0000","last_unstale":"2023-08-27T11:27:30.301299+0000","last_undegraded":"2023-08-27T11:27:30.301299+0000","last_fullsized":"2023-08-27T11:27:30.301299+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:29:15.252048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301182+0000","last_change":"2023-08-27T11:27:27.733485+0000","last_active":"2023-08-27T11:27:30.301182+0000","last_peered":"2023-08-27T11:27:30.301182+0000","last_clean":"2023-08-27T11:27:30.301182+0000","last_became_active":"2023-08-27T11:27:27.733239+0000","last_became_peered":"2023-08-27T11:27:27.733239+0000","last_unstale":"2023-08-27T11:27:30.301182+0000","last_undegraded":"2023-08-27T11:27:30.301182+0000","last_fullsized":"2023-08-27T11:27:30.301182+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:00:41.683486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301249+0000","last_change":"2023-08-27T11:27:28.746868+0000","last_active":"2023-08-27T11:27:30.301249+0000","last_peered":"2023-08-27T11:27:30.301249+0000","last_clean":"2023-08-27T11:27:30.301249+0000","last_became_active":"2023-08-27T11:27:28.746590+0000","last_became_peered":"2023-08-27T11:27:28.746590+0000","last_unstale":"2023-08-27T11:27:30.301249+0000","last_undegraded":"2023-08-27T11:27:30.301249+0000","last_fullsized":"2023-08-27T11:27:30.301249+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:14:02.387821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300505+0000","last_change":"2023-08-27T11:27:27.721893+0000","last_active":"2023-08-27T11:27:30.300505+0000","last_peered":"2023-08-27T11:27:30.300505+0000","last_clean":"2023-08-27T11:27:30.300505+0000","last_became_active":"2023-08-27T11:27:27.721626+0000","last_became_peered":"2023-08-27T11:27:27.721626+0000","last_unstale":"2023-08-27T11:27:30.300505+0000","last_undegraded":"2023-08-27T11:27:30.300505+0000","last_fullsized":"2023-08-27T11:27:30.300505+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:02.487272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715366+0000","last_change":"2023-08-27T11:27:28.728000+0000","last_active":"2023-08-27T11:27:29.715366+0000","last_peered":"2023-08-27T11:27:29.715366+0000","last_clean":"2023-08-27T11:27:29.715366+0000","last_became_active":"2023-08-27T11:27:28.726716+0000","last_became_peered":"2023-08-27T11:27:28.726716+0000","last_unstale":"2023-08-27T11:27:29.715366+0000","last_undegraded":"2023-08-27T11:27:29.715366+0000","last_fullsized":"2023-08-27T11:27:29.715366+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:28:07.254859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"113'3","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301393+0000","last_change":"2023-08-27T11:27:27.733703+0000","last_active":"2023-08-27T11:27:30.301393+0000","last_peered":"2023-08-27T11:27:30.301393+0000","last_clean":"2023-08-27T11:27:30.301393+0000","last_became_active":"2023-08-27T11:27:27.733428+0000","last_became_peered":"2023-08-27T11:27:27.733428+0000","last_unstale":"2023-08-27T11:27:30.301393+0000","last_undegraded":"2023-08-27T11:27:30.301393+0000","last_fullsized":"2023-08-27T11:27:30.301393+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:50:35.605909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":211,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715941+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:27:29.715941+0000","last_peered":"2023-08-27T11:27:29.715941+0000","last_clean":"2023-08-27T11:27:29.715941+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:27:29.715941+0000","last_undegraded":"2023-08-27T11:27:29.715941+0000","last_fullsized":"2023-08-27T11:27:29.715941+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:35:46.774169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.717067+0000","last_change":"2023-08-27T11:27:28.727375+0000","last_active":"2023-08-27T11:27:29.717067+0000","last_peered":"2023-08-27T11:27:29.717067+0000","last_clean":"2023-08-27T11:27:29.717067+0000","last_became_active":"2023-08-27T11:27:28.727057+0000","last_became_peered":"2023-08-27T11:27:28.727057+0000","last_unstale":"2023-08-27T11:27:29.717067+0000","last_undegraded":"2023-08-27T11:27:29.717067+0000","last_fullsized":"2023-08-27T11:27:29.717067+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:36:13.007592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"113'6","reported_seq":19,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300964+0000","last_change":"2023-08-27T11:27:27.733450+0000","last_active":"2023-08-27T11:27:30.300964+0000","last_peered":"2023-08-27T11:27:30.300964+0000","last_clean":"2023-08-27T11:27:30.300964+0000","last_became_active":"2023-08-27T11:27:27.733128+0000","last_became_peered":"2023-08-27T11:27:27.733128+0000","last_unstale":"2023-08-27T11:27:30.300964+0000","last_undegraded":"2023-08-27T11:27:30.300964+0000","last_fullsized":"2023-08-27T11:27:30.300964+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:27:17.357947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":211,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715676+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:27:29.715676+0000","last_peered":"2023-08-27T11:27:29.715676+0000","last_clean":"2023-08-27T11:27:29.715676+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:27:29.715676+0000","last_undegraded":"2023-08-27T11:27:29.715676+0000","last_fullsized":"2023-08-27T11:27:29.715676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:52:02.533670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300873+0000","last_change":"2023-08-27T11:27:28.722079+0000","last_active":"2023-08-27T11:27:30.300873+0000","last_peered":"2023-08-27T11:27:30.300873+0000","last_clean":"2023-08-27T11:27:30.300873+0000","last_became_active":"2023-08-27T11:27:28.721811+0000","last_became_peered":"2023-08-27T11:27:28.721811+0000","last_unstale":"2023-08-27T11:27:30.300873+0000","last_undegraded":"2023-08-27T11:27:30.300873+0000","last_fullsized":"2023-08-27T11:27:30.300873+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:09:31.693592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"113'2","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.714028+0000","last_change":"2023-08-27T11:27:27.725475+0000","last_active":"2023-08-27T11:27:29.714028+0000","last_peered":"2023-08-27T11:27:29.714028+0000","last_clean":"2023-08-27T11:27:29.714028+0000","last_became_active":"2023-08-27T11:27:27.725243+0000","last_became_peered":"2023-08-27T11:27:27.725243+0000","last_unstale":"2023-08-27T11:27:29.714028+0000","last_undegraded":"2023-08-27T11:27:29.714028+0000","last_fullsized":"2023-08-27T11:27:29.714028+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:30.027570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":180,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300443+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:27:30.300443+0000","last_peered":"2023-08-27T11:27:30.300443+0000","last_clean":"2023-08-27T11:27:30.300443+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:27:30.300443+0000","last_undegraded":"2023-08-27T11:27:30.300443+0000","last_fullsized":"2023-08-27T11:27:30.300443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:29:30.311219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715473+0000","last_change":"2023-08-27T11:27:28.722192+0000","last_active":"2023-08-27T11:27:29.715473+0000","last_peered":"2023-08-27T11:27:29.715473+0000","last_clean":"2023-08-27T11:27:29.715473+0000","last_became_active":"2023-08-27T11:27:28.721959+0000","last_became_peered":"2023-08-27T11:27:28.721959+0000","last_unstale":"2023-08-27T11:27:29.715473+0000","last_undegraded":"2023-08-27T11:27:29.715473+0000","last_fullsized":"2023-08-27T11:27:29.715473+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:38:58.901876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301171+0000","last_change":"2023-08-27T11:27:27.732793+0000","last_active":"2023-08-27T11:27:30.301171+0000","last_peered":"2023-08-27T11:27:30.301171+0000","last_clean":"2023-08-27T11:27:30.301171+0000","last_became_active":"2023-08-27T11:27:27.732489+0000","last_became_peered":"2023-08-27T11:27:27.732489+0000","last_unstale":"2023-08-27T11:27:30.301171+0000","last_undegraded":"2023-08-27T11:27:30.301171+0000","last_fullsized":"2023-08-27T11:27:30.301171+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:10:21.200976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":178,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300303+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:27:30.300303+0000","last_peered":"2023-08-27T11:27:30.300303+0000","last_clean":"2023-08-27T11:27:30.300303+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:27:30.300303+0000","last_undegraded":"2023-08-27T11:27:30.300303+0000","last_fullsized":"2023-08-27T11:27:30.300303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:52.610932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300995+0000","last_change":"2023-08-27T11:27:28.724456+0000","last_active":"2023-08-27T11:27:30.300995+0000","last_peered":"2023-08-27T11:27:30.300995+0000","last_clean":"2023-08-27T11:27:30.300995+0000","last_became_active":"2023-08-27T11:27:28.724165+0000","last_became_peered":"2023-08-27T11:27:28.724165+0000","last_unstale":"2023-08-27T11:27:30.300995+0000","last_undegraded":"2023-08-27T11:27:30.300995+0000","last_fullsized":"2023-08-27T11:27:30.300995+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:59:20.756731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"113'8","reported_seq":21,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300717+0000","last_change":"2023-08-27T11:27:27.730253+0000","last_active":"2023-08-27T11:27:30.300717+0000","last_peered":"2023-08-27T11:27:30.300717+0000","last_clean":"2023-08-27T11:27:30.300717+0000","last_became_active":"2023-08-27T11:27:27.730010+0000","last_became_peered":"2023-08-27T11:27:27.730010+0000","last_unstale":"2023-08-27T11:27:30.300717+0000","last_undegraded":"2023-08-27T11:27:30.300717+0000","last_fullsized":"2023-08-27T11:27:30.300717+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:06.778260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":178,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300966+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:27:30.300966+0000","last_peered":"2023-08-27T11:27:30.300966+0000","last_clean":"2023-08-27T11:27:30.300966+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:27:30.300966+0000","last_undegraded":"2023-08-27T11:27:30.300966+0000","last_fullsized":"2023-08-27T11:27:30.300966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:19:33.541372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301125+0000","last_change":"2023-08-27T11:27:28.745302+0000","last_active":"2023-08-27T11:27:30.301125+0000","last_peered":"2023-08-27T11:27:30.301125+0000","last_clean":"2023-08-27T11:27:30.301125+0000","last_became_active":"2023-08-27T11:27:28.743659+0000","last_became_peered":"2023-08-27T11:27:28.743659+0000","last_unstale":"2023-08-27T11:27:30.301125+0000","last_undegraded":"2023-08-27T11:27:30.301125+0000","last_fullsized":"2023-08-27T11:27:30.301125+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:53.775941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715164+0000","last_change":"2023-08-27T11:27:27.728183+0000","last_active":"2023-08-27T11:27:29.715164+0000","last_peered":"2023-08-27T11:27:29.715164+0000","last_clean":"2023-08-27T11:27:29.715164+0000","last_became_active":"2023-08-27T11:27:27.727887+0000","last_became_peered":"2023-08-27T11:27:27.727887+0000","last_unstale":"2023-08-27T11:27:29.715164+0000","last_undegraded":"2023-08-27T11:27:29.715164+0000","last_fullsized":"2023-08-27T11:27:29.715164+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:34.243069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":328,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300140+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:27:30.300140+0000","last_peered":"2023-08-27T11:27:30.300140+0000","last_clean":"2023-08-27T11:27:30.300140+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:27:30.300140+0000","last_undegraded":"2023-08-27T11:27:30.300140+0000","last_fullsized":"2023-08-27T11:27:30.300140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:48:51.726242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":184,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300800+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:27:30.300800+0000","last_peered":"2023-08-27T11:27:30.300800+0000","last_clean":"2023-08-27T11:27:30.300800+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:27:30.300800+0000","last_undegraded":"2023-08-27T11:27:30.300800+0000","last_fullsized":"2023-08-27T11:27:30.300800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:03:13.742093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300703+0000","last_change":"2023-08-27T11:27:28.727300+0000","last_active":"2023-08-27T11:27:30.300703+0000","last_peered":"2023-08-27T11:27:30.300703+0000","last_clean":"2023-08-27T11:27:30.300703+0000","last_became_active":"2023-08-27T11:27:28.727029+0000","last_became_peered":"2023-08-27T11:27:28.727029+0000","last_unstale":"2023-08-27T11:27:30.300703+0000","last_undegraded":"2023-08-27T11:27:30.300703+0000","last_fullsized":"2023-08-27T11:27:30.300703+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:46.441644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.714066+0000","last_change":"2023-08-27T11:27:27.725362+0000","last_active":"2023-08-27T11:27:29.714066+0000","last_peered":"2023-08-27T11:27:29.714066+0000","last_clean":"2023-08-27T11:27:29.714066+0000","last_became_active":"2023-08-27T11:27:27.725113+0000","last_became_peered":"2023-08-27T11:27:27.725113+0000","last_unstale":"2023-08-27T11:27:29.714066+0000","last_undegraded":"2023-08-27T11:27:29.714066+0000","last_fullsized":"2023-08-27T11:27:29.714066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:23:49.650754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":178,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300835+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:27:30.300835+0000","last_peered":"2023-08-27T11:27:30.300835+0000","last_clean":"2023-08-27T11:27:30.300835+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:27:30.300835+0000","last_undegraded":"2023-08-27T11:27:30.300835+0000","last_fullsized":"2023-08-27T11:27:30.300835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:33:36.130112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713726+0000","last_change":"2023-08-27T11:27:28.725207+0000","last_active":"2023-08-27T11:27:29.713726+0000","last_peered":"2023-08-27T11:27:29.713726+0000","last_clean":"2023-08-27T11:27:29.713726+0000","last_became_active":"2023-08-27T11:27:28.724890+0000","last_became_peered":"2023-08-27T11:27:28.724890+0000","last_unstale":"2023-08-27T11:27:29.713726+0000","last_undegraded":"2023-08-27T11:27:29.713726+0000","last_fullsized":"2023-08-27T11:27:29.713726+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:05.392066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300617+0000","last_change":"2023-08-27T11:27:27.728162+0000","last_active":"2023-08-27T11:27:30.300617+0000","last_peered":"2023-08-27T11:27:30.300617+0000","last_clean":"2023-08-27T11:27:30.300617+0000","last_became_active":"2023-08-27T11:27:27.727886+0000","last_became_peered":"2023-08-27T11:27:27.727886+0000","last_unstale":"2023-08-27T11:27:30.300617+0000","last_undegraded":"2023-08-27T11:27:30.300617+0000","last_fullsized":"2023-08-27T11:27:30.300617+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:18:06.132815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":202,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716638+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:27:29.716638+0000","last_peered":"2023-08-27T11:27:29.716638+0000","last_clean":"2023-08-27T11:27:29.716638+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:27:29.716638+0000","last_undegraded":"2023-08-27T11:27:29.716638+0000","last_fullsized":"2023-08-27T11:27:29.716638+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:40:27.374079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574979,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2035712,"data_stored":1250354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.35499999999999998,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.749,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.52500000000000002,"15min":0.501},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.66800000000000004,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.496}]},{"osd":1,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54800000000000004,"15min":0.51400000000000001},"min":{"1min":0.39100000000000001,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.73499999999999999,"5min":0.77500000000000002,"15min":0.872},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54200000000000004,"15min":0.52000000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.69399999999999995,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.54000000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51400000000000001,"15min":0.49099999999999999},"min":{"1min":0.34499999999999997,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.77400000000000002,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.45800000000000002,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.90200000000000002},"last":0.52600000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57499999999999996,"15min":0.55400000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77300000000000002,"5min":0.996,"15min":0.996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.60199999999999998},"min":{"1min":0.45900000000000002,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.79600000000000004,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.62}]},{"osd":4,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.504},"min":{"1min":0.39600000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.75,"5min":0.75,"15min":0.77400000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73799999999999999,"5min":1.224,"15min":1.224},"last":0.45000000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.38900000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.84799999999999998},"last":0.57499999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.48899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80600000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.39800000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80700000000000005,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.56599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":658956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.441,"5min":0.33500000000000002,"15min":0.29099999999999998},"max":{"1min":0.70299999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.54100000000000004},"min":{"1min":0.35899999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.89800000000000002},"last":0.80300000000000005}]},{"osd":1,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.316,"15min":0.312},"max":{"1min":0.67800000000000005,"5min":0.80600000000000005,"15min":0.873},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.751,"5min":0.75600000000000001,"15min":0.84399999999999997},"last":0.54700000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.42299999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.67200000000000004,"5min":0.72499999999999998,"15min":0.85899999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55600000000000005,"15min":0.57899999999999996},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.754,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.73099999999999998,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.48599999999999999,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.72799999999999998,"5min":0.80700000000000005,"15min":0.93799999999999994},"last":0.63200000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.28199999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.70699999999999996,"5min":0.71299999999999997,"15min":0.83099999999999996},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.374},"max":{"1min":0.81299999999999994,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.60799999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.35099999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.623,"15min":0.60399999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.33900000000000002},"max":{"1min":0.76400000000000001,"5min":0.78700000000000003,"15min":0.875},"last":0.46899999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.32200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78800000000000003,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.34899999999999998},"max":{"1min":0.82999999999999996,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.77900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574979,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2023424,"data_stored":1248722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.311,"15min":0.55400000000000005},"min":{"1min":0.19500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.42599999999999999,"5min":0.502,"15min":11.971},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.30099999999999999,"5min":0.33200000000000002,"15min":0.80800000000000005},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.39500000000000002,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.34399999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.34000000000000002,"15min":0.58299999999999996},"min":{"1min":0.27700000000000002,"5min":0.23400000000000001,"15min":0.221},"max":{"1min":0.45500000000000002,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.317,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.22800000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.38800000000000001,"5min":0.52000000000000002,"15min":0.58599999999999997},"last":0.371}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38800000000000001,"15min":0.60399999999999998},"min":{"1min":0.27500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.47399999999999998,"5min":0.58099999999999996,"15min":10.99},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.372,"15min":0.84499999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23499999999999999},"max":{"1min":0.45000000000000001,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.41699999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57399999999999995,"15min":0.59499999999999997},"min":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.33400000000000002},"max":{"1min":0.79500000000000004,"5min":0.90700000000000003,"15min":1.26},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.77300000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":1.0409999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36199999999999999},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.46300000000000002,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.71599999999999997,"5min":0.88700000000000001,"15min":1.296},"last":0.78100000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.80400000000000005},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.371},"max":{"1min":0.83599999999999997,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":1.0589999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81200000000000006,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.72999999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":1.0529999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.84199999999999997,"15min":22.988},"last":0.752},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.85899999999999999,"5min":0.86399999999999999,"15min":1.327},"last":0.66300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574979,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1429504,"data_stored":656979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.311},"min":{"1min":0.223,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.46200000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.435},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.33300000000000002,"15min":0.34899999999999998},"min":{"1min":0.253,"5min":0.223,"15min":0.223},"max":{"1min":0.5,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.34799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34100000000000003},"min":{"1min":0.251,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.44800000000000001,"5min":0.628,"15min":0.63200000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.33900000000000002},"min":{"1min":0.26600000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51000000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34599999999999997},"min":{"1min":0.247,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47099999999999997,"5min":0.50800000000000001,"15min":0.59599999999999997},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.371,"15min":0.373},"min":{"1min":0.30299999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.60599999999999998},"last":0.41299999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.85399999999999998,"15min":1.214},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.42999999999999999,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.88400000000000001,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.627}]},{"osd":5,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.41099999999999998,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.72599999999999998,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.40200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78500000000000003,"5min":0.85399999999999998,"15min":1.603},"last":0.67900000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89000000000000001,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.39900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.878,"15min":0.878},"last":0.71499999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.39900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.79,"5min":3.79,"15min":3.79},"last":0.69899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":658900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.29899999999999999,"15min":0.30199999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.45300000000000001,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.377},{"interface":"front","average":{"1min":0.30099999999999999,"5min":0.313,"15min":0.316},"min":{"1min":0.24099999999999999,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.36199999999999999,"5min":0.52500000000000002,"15min":0.629},"last":0.29799999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.31900000000000001,"15min":0.32300000000000001},"min":{"1min":0.23499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.40699999999999997,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.251},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33700000000000002,"15min":0.34499999999999997},"min":{"1min":0.26000000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.439,"5min":0.623,"15min":0.749},"last":0.34499999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.36699999999999999,"15min":0.38},"min":{"1min":0.245,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.44500000000000001,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38100000000000001,"15min":0.39600000000000002},"min":{"1min":0.33000000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.49199999999999999,"5min":0.57199999999999995,"15min":0.69499999999999995},"last":0.44800000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.375,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.76600000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.879},"last":0.41799999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72199999999999998,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.495},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.76100000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.42899999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73799999999999999,"5min":0.82899999999999996,"15min":0.85299999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.44,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78100000000000003,"5min":0.80500000000000005,"15min":0.83599999999999997},"last":0.47899999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.41799999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.747,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.41399999999999998,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.84899999999999998},"last":0.46200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":659966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.29699999999999999,"15min":0.30099999999999999},"min":{"1min":0.23699999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41399999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32000000000000001,"15min":0.314},"min":{"1min":0.26000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.41699999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.27800000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.316},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.19500000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.34200000000000003},"min":{"1min":0.27600000000000002,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":0.45200000000000001,"5min":0.52300000000000002,"15min":0.56299999999999994},"last":0.38300000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.29299999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.36299999999999999,"15min":0.36599999999999999},"min":{"1min":0.27500000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.50600000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.441}]},{"osd":4,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.438,"5min":0.33900000000000002,"15min":0.28599999999999998},"max":{"1min":0.753,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.5}]},{"osd":5,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.314},"max":{"1min":0.78500000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.83799999999999997,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.51100000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59799999999999998,"15min":0.60199999999999998},"min":{"1min":0.434,"5min":0.379,"15min":0.373},"max":{"1min":0.82799999999999996,"5min":1.091,"15min":1.091},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.71399999999999997,"5min":1.111,"15min":1.111},"last":0.47999999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.60799999999999998},"min":{"1min":0.39600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":1.032,"15min":1.236},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.41599999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85899999999999999,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.41099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":658284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51000000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64100000000000001,"5min":0.997,"15min":0.997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.497,"15min":0.496},"min":{"1min":0.36899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.79100000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40899999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50600000000000001,"15min":0.51500000000000001},"min":{"1min":0.34899999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.76500000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.34000000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.73099999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.498}]},{"osd":2,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69899999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52500000000000002,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69199999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.44500000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54500000000000004,"15min":0.53700000000000003},"min":{"1min":0.439,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.66600000000000004,"5min":0.78200000000000003,"15min":0.79400000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80600000000000005},"last":0.51600000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.49299999999999999,"15min":0.51500000000000001},"min":{"1min":0.32600000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68300000000000005,"5min":0.81100000000000005,"15min":0.85899999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.42399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75900000000000001,"5min":0.85999999999999999,"15min":0.879},"last":0.53300000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68799999999999994,"5min":0.78900000000000003,"15min":0.90600000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59399999999999997,"15min":0.621},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77000000000000002,"5min":0.79300000000000004,"15min":0.92300000000000004},"last":0.58199999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71499999999999997,"5min":0.80600000000000005,"15min":0.82499999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.60699999999999998},"min":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.71399999999999997,"5min":0.79200000000000004,"15min":0.89300000000000002},"last":0.629}]}]},{"osd":5,"up_from":13,"seq":55834574979,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1429504,"data_stored":658306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73899999999999999,"5min":0.753,"15min":0.96899999999999997},"last":0.377},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39300000000000002,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.68200000000000005,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65800000000000003,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68400000000000005,"5min":0.72899999999999998,"15min":1.075},"last":0.48499999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.33200000000000002},"max":{"1min":0.72099999999999997,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.41099999999999998,"5min":0.373,"15min":0.307},"max":{"1min":0.753,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.59599999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":1.03},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.35199999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":1.026},"last":0.55300000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.371,"5min":0.31,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.50700000000000001,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79600000000000004,"5min":0.83899999999999997,"15min":0.91400000000000003},"last":0.53700000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.79400000000000004,"5min":0.79700000000000004,"15min":0.88200000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":0.81499999999999995,"15min":0.94199999999999995},"last":0.44700000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.41499999999999998,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.95899999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.31},"max":{"1min":0.82299999999999995,"5min":0.85599999999999998,"15min":0.92000000000000004},"last":0.629}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:27:32.834 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:27:32.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.985+0000 7fb990a5a700 1 Processor -- start 2023-08-27T11:27:32.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.985+0000 7fb990a5a700 1 -- start start 2023-08-27T11:27:32.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.986+0000 7fb990a5a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb98c144570 0x7fb98c15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.986+0000 7fb990a5a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb98c0b3e50 0x7fb98c15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.986+0000 7fb990a5a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98c0b5ae0 con 0x7fb98c15f8a0 2023-08-27T11:27:32.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.986+0000 7fb990a5a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98c144950 con 0x7fb98c144570 2023-08-27T11:27:32.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.986+0000 7fb990a5a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c0b45f0 con 0x7fb98c0b3e50 2023-08-27T11:27:32.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb989d9b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb98c0b3e50 0x7fb98c15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb98a59c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb98c144570 0x7fb98c15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:32.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb989d9b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb98c0b3e50 0x7fb98c15f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:46666/0 (socket says 172.21.15.112:46666) 2023-08-27T11:27:32.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb98ad9d700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb98c15f8a0 0x7fb98c0ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:36920/0 (socket says 172.21.15.112:36920) 2023-08-27T11:27:32.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb989d9b700 1 -- 172.21.15.112:0/1195033267 learned_addr learned my addr 172.21.15.112:0/1195033267 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:32.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb98a59c700 1 -- 172.21.15.112:0/1195033267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb98c0b3e50 msgr2=0x7fb98c15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:32.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb98a59c700 1 --2- 172.21.15.112:0/1195033267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb98c0b3e50 0x7fb98c15f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb98a59c700 1 -- 172.21.15.112:0/1195033267 >> v1:172.21.15.138:6789/0 conn(0x7fb98c15f8a0 legacy=0x7fb98c0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb98a59c700 1 -- 172.21.15.112:0/1195033267 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c0b4310 con 0x7fb98c144570 2023-08-27T11:27:32.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb989d9b700 1 --2- 172.21.15.112:0/1195033267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb98c0b3e50 0x7fb98c15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:32.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb98a59c700 1 --2- 172.21.15.112:0/1195033267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98c144570 0x7fb98c15ee20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb980009840 tx=0x7fb980009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e069b026f3b6643 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:32.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb98959a700 1 -- 172.21.15.112:0/1195033267 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980011830 con 0x7fb98c144570 2023-08-27T11:27:32.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb98959a700 1 -- 172.21.15.112:0/1195033267 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb980011990 con 0x7fb98c144570 2023-08-27T11:27:32.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.987+0000 7fb98959a700 1 -- 172.21.15.112:0/1195033267 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980011d20 con 0x7fb98c144570 2023-08-27T11:27:32.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.988+0000 7fb990a5a700 1 -- 172.21.15.112:0/1195033267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98c144570 msgr2=0x7fb98c15ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:32.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.988+0000 7fb990a5a700 1 --2- 172.21.15.112:0/1195033267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98c144570 0x7fb98c15ee20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb980009840 tx=0x7fb980009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:32.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.988+0000 7fb990a5a700 1 -- 172.21.15.112:0/1195033267 shutdown_connections 2023-08-27T11:27:32.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.988+0000 7fb990a5a700 1 --2- 172.21.15.112:0/1195033267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98c144570 0x7fb98c15ee20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.988+0000 7fb990a5a700 1 --2- 172.21.15.112:0/1195033267 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb98c0b3e50 0x7fb98c15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:32.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.988+0000 7fb990a5a700 1 -- 172.21.15.112:0/1195033267 >> 172.21.15.112:0/1195033267 conn(0x7fb98c06f970 msgr2=0x7fb98c06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:32.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.988+0000 7fb990a5a700 1 -- 172.21.15.112:0/1195033267 shutdown_connections 2023-08-27T11:27:32.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.988+0000 7fb990a5a700 1 -- 172.21.15.112:0/1195033267 wait complete. 2023-08-27T11:27:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.989+0000 7fb990a5a700 1 Processor -- start 2023-08-27T11:27:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.989+0000 7fb990a5a700 1 -- start start 2023-08-27T11:27:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.990+0000 7fb990a5a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb98c0b3e50 0x7fb98c13f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.990+0000 7fb990a5a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb98c144570 0x7fb98c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.990+0000 7fb990a5a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98c15f8a0 0x7fb98c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.990+0000 7fb990a5a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c142da0 con 0x7fb98c0b3e50 2023-08-27T11:27:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.990+0000 7fb990a5a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c143080 con 0x7fb98c15f8a0 2023-08-27T11:27:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.990+0000 7fb990a5a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c143360 con 0x7fb98c144570 2023-08-27T11:27:32.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.990+0000 7fb98a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb98c0b3e50 0x7fb98c13f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.990+0000 7fb98a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb98c0b3e50 0x7fb98c13f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35016/0 (socket says 172.21.15.112:35016) 2023-08-27T11:27:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.990+0000 7fb98a59c700 1 -- 172.21.15.112:0/3038237536 learned_addr learned my addr 172.21.15.112:0/3038237536 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.990+0000 7fb989d9b700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb98c144570 0x7fb98c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.990+0000 7fb98ad9d700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98c15f8a0 0x7fb98c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.991+0000 7fb98a59c700 1 -- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb98c144570 msgr2=0x7fb98c15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.991+0000 7fb98a59c700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb98c144570 0x7fb98c15d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.991+0000 7fb98a59c700 1 -- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98c15f8a0 msgr2=0x7fb98c15de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.991+0000 7fb98a59c700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98c15f8a0 0x7fb98c15de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.991+0000 7fb98a59c700 1 -- 172.21.15.112:0/3038237536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c009550 con 0x7fb98c0b3e50 2023-08-27T11:27:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.991+0000 7fb98a59c700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb98c0b3e50 0x7fb98c13f9e0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb980009fd0 tx=0x7fb980004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.991+0000 7fb97affd700 1 -- 172.21.15.112:0/3038237536 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980004550 con 0x7fb98c0b3e50 2023-08-27T11:27:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.991+0000 7fb97affd700 1 -- 172.21.15.112:0/3038237536 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98001f070 con 0x7fb98c0b3e50 2023-08-27T11:27:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.991+0000 7fb990a5a700 1 -- 172.21.15.112:0/3038237536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9800094a0 con 0x7fb98c0b3e50 2023-08-27T11:27:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.991+0000 7fb97affd700 1 -- 172.21.15.112:0/3038237536 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9800038e0 con 0x7fb98c0b3e50 2023-08-27T11:27:33.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.991+0000 7fb990a5a700 1 -- 172.21.15.112:0/3038237536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98c0a8800 con 0x7fb98c0b3e50 2023-08-27T11:27:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.992+0000 7fb990a5a700 1 -- 172.21.15.112:0/3038237536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98c067a20 con 0x7fb98c0b3e50 2023-08-27T11:27:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.992+0000 7fb97affd700 1 -- 172.21.15.112:0/3038237536 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fb980003a40 con 0x7fb98c0b3e50 2023-08-27T11:27:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.993+0000 7fb97affd700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb96c06a4d0 0x7fb96c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.993+0000 7fb97affd700 1 -- 172.21.15.112:0/3038237536 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb980021070 con 0x7fb98c0b3e50 2023-08-27T11:27:33.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.993+0000 7fb989d9b700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb96c06a4d0 0x7fb96c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.994+0000 7fb989d9b700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb96c06a4d0 0x7fb96c06c990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb98c143e70 tx=0x7fb97c0092c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:32.997+0000 7fb97affd700 1 -- 172.21.15.112:0/3038237536 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb9800555a0 con 0x7fb98c0b3e50 2023-08-27T11:27:33.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.184+0000 7fb990a5a700 1 -- 172.21.15.112:0/3038237536 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb98c0cdf90 con 0x7fb96c06a4d0 2023-08-27T11:27:33.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.193+0000 7fb97affd700 1 -- 172.21.15.112:0/3038237536 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312358 (secure 0 0 0) 0x7fb97c009e00 con 0x7fb96c06a4d0 2023-08-27T11:27:33.198 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:33.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.196+0000 7fb990a5a700 1 -- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb96c06a4d0 msgr2=0x7fb96c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.196+0000 7fb990a5a700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb96c06a4d0 0x7fb96c06c990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb98c143e70 tx=0x7fb97c0092c0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.196+0000 7fb990a5a700 1 -- 172.21.15.112:0/3038237536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb98c0b3e50 msgr2=0x7fb98c13f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.196+0000 7fb990a5a700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb98c0b3e50 0x7fb98c13f9e0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb980009fd0 tx=0x7fb980004750 comp rx=0 tx=0).stop 2023-08-27T11:27:33.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.196+0000 7fb990a5a700 1 -- 172.21.15.112:0/3038237536 shutdown_connections 2023-08-27T11:27:33.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.196+0000 7fb990a5a700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb96c06a4d0 0x7fb96c06c990 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.196+0000 7fb990a5a700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98c15f8a0 0x7fb98c15de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.196+0000 7fb990a5a700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb98c0b3e50 0x7fb98c13f9e0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.196+0000 7fb990a5a700 1 --2- 172.21.15.112:0/3038237536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb98c144570 0x7fb98c15d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.196+0000 7fb990a5a700 1 -- 172.21.15.112:0/3038237536 >> 172.21.15.112:0/3038237536 conn(0x7fb98c06f970 msgr2=0x7fb98c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:33.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.197+0000 7fb990a5a700 1 -- 172.21.15.112:0/3038237536 shutdown_connections 2023-08-27T11:27:33.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.197+0000 7fb990a5a700 1 -- 172.21.15.112:0/3038237536 wait complete. 2023-08-27T11:27:33.204 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:27:33.210 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2023-08-27T11:27:32.386136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261820,"kb_used_data":12356,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754712900,"statfs":{"total":773094113280,"available":772826009600,"internally_reserved":0,"allocated":12652544,"data_stored":6450467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.686352"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.714391+0000","last_change":"2023-08-27T11:27:28.721911+0000","last_active":"2023-08-27T11:27:29.714391+0000","last_peered":"2023-08-27T11:27:29.714391+0000","last_clean":"2023-08-27T11:27:29.714391+0000","last_became_active":"2023-08-27T11:27:28.721622+0000","last_became_peered":"2023-08-27T11:27:28.721622+0000","last_unstale":"2023-08-27T11:27:29.714391+0000","last_undegraded":"2023-08-27T11:27:29.714391+0000","last_fullsized":"2023-08-27T11:27:29.714391+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:34:15.793813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300616+0000","last_change":"2023-08-27T11:27:28.733803+0000","last_active":"2023-08-27T11:27:30.300616+0000","last_peered":"2023-08-27T11:27:30.300616+0000","last_clean":"2023-08-27T11:27:30.300616+0000","last_became_active":"2023-08-27T11:27:28.733521+0000","last_became_peered":"2023-08-27T11:27:28.733521+0000","last_unstale":"2023-08-27T11:27:30.300616+0000","last_undegraded":"2023-08-27T11:27:30.300616+0000","last_fullsized":"2023-08-27T11:27:30.300616+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:28:23.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715287+0000","last_change":"2023-08-27T11:27:28.722194+0000","last_active":"2023-08-27T11:27:29.715287+0000","last_peered":"2023-08-27T11:27:29.715287+0000","last_clean":"2023-08-27T11:27:29.715287+0000","last_became_active":"2023-08-27T11:27:28.721986+0000","last_became_peered":"2023-08-27T11:27:28.721986+0000","last_unstale":"2023-08-27T11:27:29.715287+0000","last_undegraded":"2023-08-27T11:27:29.715287+0000","last_fullsized":"2023-08-27T11:27:29.715287+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:24:26.197854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300303+0000","last_change":"2023-08-27T11:27:28.730569+0000","last_active":"2023-08-27T11:27:30.300303+0000","last_peered":"2023-08-27T11:27:30.300303+0000","last_clean":"2023-08-27T11:27:30.300303+0000","last_became_active":"2023-08-27T11:27:28.730273+0000","last_became_peered":"2023-08-27T11:27:28.730273+0000","last_unstale":"2023-08-27T11:27:30.300303+0000","last_undegraded":"2023-08-27T11:27:30.300303+0000","last_fullsized":"2023-08-27T11:27:30.300303+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:28:43.469961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300462+0000","last_change":"2023-08-27T11:27:28.721400+0000","last_active":"2023-08-27T11:27:30.300462+0000","last_peered":"2023-08-27T11:27:30.300462+0000","last_clean":"2023-08-27T11:27:30.300462+0000","last_became_active":"2023-08-27T11:27:28.720872+0000","last_became_peered":"2023-08-27T11:27:28.720872+0000","last_unstale":"2023-08-27T11:27:30.300462+0000","last_undegraded":"2023-08-27T11:27:30.300462+0000","last_fullsized":"2023-08-27T11:27:30.300462+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:37.403227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300847+0000","last_change":"2023-08-27T11:27:28.728027+0000","last_active":"2023-08-27T11:27:30.300847+0000","last_peered":"2023-08-27T11:27:30.300847+0000","last_clean":"2023-08-27T11:27:30.300847+0000","last_became_active":"2023-08-27T11:27:28.727728+0000","last_became_peered":"2023-08-27T11:27:28.727728+0000","last_unstale":"2023-08-27T11:27:30.300847+0000","last_undegraded":"2023-08-27T11:27:30.300847+0000","last_fullsized":"2023-08-27T11:27:30.300847+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:01:53.347873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300292+0000","last_change":"2023-08-27T11:27:28.744097+0000","last_active":"2023-08-27T11:27:30.300292+0000","last_peered":"2023-08-27T11:27:30.300292+0000","last_clean":"2023-08-27T11:27:30.300292+0000","last_became_active":"2023-08-27T11:27:28.740829+0000","last_became_peered":"2023-08-27T11:27:28.740829+0000","last_unstale":"2023-08-27T11:27:30.300292+0000","last_undegraded":"2023-08-27T11:27:30.300292+0000","last_fullsized":"2023-08-27T11:27:30.300292+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:23.555957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715856+0000","last_change":"2023-08-27T11:27:28.727125+0000","last_active":"2023-08-27T11:27:29.715856+0000","last_peered":"2023-08-27T11:27:29.715856+0000","last_clean":"2023-08-27T11:27:29.715856+0000","last_became_active":"2023-08-27T11:27:28.726861+0000","last_became_peered":"2023-08-27T11:27:28.726861+0000","last_unstale":"2023-08-27T11:27:29.715856+0000","last_undegraded":"2023-08-27T11:27:29.715856+0000","last_fullsized":"2023-08-27T11:27:29.715856+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:28:14.761948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301393+0000","last_change":"2023-08-27T11:27:28.733174+0000","last_active":"2023-08-27T11:27:30.301393+0000","last_peered":"2023-08-27T11:27:30.301393+0000","last_clean":"2023-08-27T11:27:30.301393+0000","last_became_active":"2023-08-27T11:27:28.732789+0000","last_became_peered":"2023-08-27T11:27:28.732789+0000","last_unstale":"2023-08-27T11:27:30.301393+0000","last_undegraded":"2023-08-27T11:27:30.301393+0000","last_fullsized":"2023-08-27T11:27:30.301393+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:10.830106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301435+0000","last_change":"2023-08-27T11:27:28.726784+0000","last_active":"2023-08-27T11:27:30.301435+0000","last_peered":"2023-08-27T11:27:30.301435+0000","last_clean":"2023-08-27T11:27:30.301435+0000","last_became_active":"2023-08-27T11:27:28.726515+0000","last_became_peered":"2023-08-27T11:27:28.726515+0000","last_unstale":"2023-08-27T11:27:30.301435+0000","last_undegraded":"2023-08-27T11:27:30.301435+0000","last_fullsized":"2023-08-27T11:27:30.301435+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:05.401079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300232+0000","last_change":"2023-08-27T11:27:28.736506+0000","last_active":"2023-08-27T11:27:30.300232+0000","last_peered":"2023-08-27T11:27:30.300232+0000","last_clean":"2023-08-27T11:27:30.300232+0000","last_became_active":"2023-08-27T11:27:28.736142+0000","last_became_peered":"2023-08-27T11:27:28.736142+0000","last_unstale":"2023-08-27T11:27:30.300232+0000","last_undegraded":"2023-08-27T11:27:30.300232+0000","last_fullsized":"2023-08-27T11:27:30.300232+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:38.106756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.714027+0000","last_change":"2023-08-27T11:27:28.722139+0000","last_active":"2023-08-27T11:27:29.714027+0000","last_peered":"2023-08-27T11:27:29.714027+0000","last_clean":"2023-08-27T11:27:29.714027+0000","last_became_active":"2023-08-27T11:27:28.721862+0000","last_became_peered":"2023-08-27T11:27:28.721862+0000","last_unstale":"2023-08-27T11:27:29.714027+0000","last_undegraded":"2023-08-27T11:27:29.714027+0000","last_fullsized":"2023-08-27T11:27:29.714027+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:16:03.259583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301068+0000","last_change":"2023-08-27T11:27:28.727583+0000","last_active":"2023-08-27T11:27:30.301068+0000","last_peered":"2023-08-27T11:27:30.301068+0000","last_clean":"2023-08-27T11:27:30.301068+0000","last_became_active":"2023-08-27T11:27:28.725833+0000","last_became_peered":"2023-08-27T11:27:28.725833+0000","last_unstale":"2023-08-27T11:27:30.301068+0000","last_undegraded":"2023-08-27T11:27:30.301068+0000","last_fullsized":"2023-08-27T11:27:30.301068+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:49:57.748777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.714124+0000","last_change":"2023-08-27T11:27:28.738604+0000","last_active":"2023-08-27T11:27:29.714124+0000","last_peered":"2023-08-27T11:27:29.714124+0000","last_clean":"2023-08-27T11:27:29.714124+0000","last_became_active":"2023-08-27T11:27:28.738248+0000","last_became_peered":"2023-08-27T11:27:28.738248+0000","last_unstale":"2023-08-27T11:27:29.714124+0000","last_undegraded":"2023-08-27T11:27:29.714124+0000","last_fullsized":"2023-08-27T11:27:29.714124+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:24:35.192028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713738+0000","last_change":"2023-08-27T11:27:28.743854+0000","last_active":"2023-08-27T11:27:29.713738+0000","last_peered":"2023-08-27T11:27:29.713738+0000","last_clean":"2023-08-27T11:27:29.713738+0000","last_became_active":"2023-08-27T11:27:28.743568+0000","last_became_peered":"2023-08-27T11:27:28.743568+0000","last_unstale":"2023-08-27T11:27:29.713738+0000","last_undegraded":"2023-08-27T11:27:29.713738+0000","last_fullsized":"2023-08-27T11:27:29.713738+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:58:54.705362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300900+0000","last_change":"2023-08-27T11:27:28.724622+0000","last_active":"2023-08-27T11:27:30.300900+0000","last_peered":"2023-08-27T11:27:30.300900+0000","last_clean":"2023-08-27T11:27:30.300900+0000","last_became_active":"2023-08-27T11:27:28.724379+0000","last_became_peered":"2023-08-27T11:27:28.724379+0000","last_unstale":"2023-08-27T11:27:30.300900+0000","last_undegraded":"2023-08-27T11:27:30.300900+0000","last_fullsized":"2023-08-27T11:27:30.300900+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:55:10.100708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301510+0000","last_change":"2023-08-27T11:27:28.722365+0000","last_active":"2023-08-27T11:27:30.301510+0000","last_peered":"2023-08-27T11:27:30.301510+0000","last_clean":"2023-08-27T11:27:30.301510+0000","last_became_active":"2023-08-27T11:27:28.722097+0000","last_became_peered":"2023-08-27T11:27:28.722097+0000","last_unstale":"2023-08-27T11:27:30.301510+0000","last_undegraded":"2023-08-27T11:27:30.301510+0000","last_fullsized":"2023-08-27T11:27:30.301510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:52.622388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300311+0000","last_change":"2023-08-27T11:27:28.723197+0000","last_active":"2023-08-27T11:27:30.300311+0000","last_peered":"2023-08-27T11:27:30.300311+0000","last_clean":"2023-08-27T11:27:30.300311+0000","last_became_active":"2023-08-27T11:27:28.722937+0000","last_became_peered":"2023-08-27T11:27:28.722937+0000","last_unstale":"2023-08-27T11:27:30.300311+0000","last_undegraded":"2023-08-27T11:27:30.300311+0000","last_fullsized":"2023-08-27T11:27:30.300311+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:32:23.412658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301392+0000","last_change":"2023-08-27T11:27:28.746158+0000","last_active":"2023-08-27T11:27:30.301392+0000","last_peered":"2023-08-27T11:27:30.301392+0000","last_clean":"2023-08-27T11:27:30.301392+0000","last_became_active":"2023-08-27T11:27:28.743787+0000","last_became_peered":"2023-08-27T11:27:28.743787+0000","last_unstale":"2023-08-27T11:27:30.301392+0000","last_undegraded":"2023-08-27T11:27:30.301392+0000","last_fullsized":"2023-08-27T11:27:30.301392+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:21:16.525932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300755+0000","last_change":"2023-08-27T11:27:28.741346+0000","last_active":"2023-08-27T11:27:30.300755+0000","last_peered":"2023-08-27T11:27:30.300755+0000","last_clean":"2023-08-27T11:27:30.300755+0000","last_became_active":"2023-08-27T11:27:28.741036+0000","last_became_peered":"2023-08-27T11:27:28.741036+0000","last_unstale":"2023-08-27T11:27:30.300755+0000","last_undegraded":"2023-08-27T11:27:30.300755+0000","last_fullsized":"2023-08-27T11:27:30.300755+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:36:16.575366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300446+0000","last_change":"2023-08-27T11:27:28.730040+0000","last_active":"2023-08-27T11:27:30.300446+0000","last_peered":"2023-08-27T11:27:30.300446+0000","last_clean":"2023-08-27T11:27:30.300446+0000","last_became_active":"2023-08-27T11:27:28.729763+0000","last_became_peered":"2023-08-27T11:27:28.729763+0000","last_unstale":"2023-08-27T11:27:30.300446+0000","last_undegraded":"2023-08-27T11:27:30.300446+0000","last_fullsized":"2023-08-27T11:27:30.300446+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:32:23.412658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713903+0000","last_change":"2023-08-27T11:27:28.726440+0000","last_active":"2023-08-27T11:27:29.713903+0000","last_peered":"2023-08-27T11:27:29.713903+0000","last_clean":"2023-08-27T11:27:29.713903+0000","last_became_active":"2023-08-27T11:27:28.726160+0000","last_became_peered":"2023-08-27T11:27:28.726160+0000","last_unstale":"2023-08-27T11:27:29.713903+0000","last_undegraded":"2023-08-27T11:27:29.713903+0000","last_fullsized":"2023-08-27T11:27:29.713903+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:32:44.006355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300266+0000","last_change":"2023-08-27T11:27:28.741507+0000","last_active":"2023-08-27T11:27:30.300266+0000","last_peered":"2023-08-27T11:27:30.300266+0000","last_clean":"2023-08-27T11:27:30.300266+0000","last_became_active":"2023-08-27T11:27:28.741182+0000","last_became_peered":"2023-08-27T11:27:28.741182+0000","last_unstale":"2023-08-27T11:27:30.300266+0000","last_undegraded":"2023-08-27T11:27:30.300266+0000","last_fullsized":"2023-08-27T11:27:30.300266+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:43:05.337653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300078+0000","last_change":"2023-08-27T11:27:28.726111+0000","last_active":"2023-08-27T11:27:30.300078+0000","last_peered":"2023-08-27T11:27:30.300078+0000","last_clean":"2023-08-27T11:27:30.300078+0000","last_became_active":"2023-08-27T11:27:28.725777+0000","last_became_peered":"2023-08-27T11:27:28.725777+0000","last_unstale":"2023-08-27T11:27:30.300078+0000","last_undegraded":"2023-08-27T11:27:30.300078+0000","last_fullsized":"2023-08-27T11:27:30.300078+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:07:14.947676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715241+0000","last_change":"2023-08-27T11:27:28.724936+0000","last_active":"2023-08-27T11:27:29.715241+0000","last_peered":"2023-08-27T11:27:29.715241+0000","last_clean":"2023-08-27T11:27:29.715241+0000","last_became_active":"2023-08-27T11:27:28.724673+0000","last_became_peered":"2023-08-27T11:27:28.724673+0000","last_unstale":"2023-08-27T11:27:29.715241+0000","last_undegraded":"2023-08-27T11:27:29.715241+0000","last_fullsized":"2023-08-27T11:27:29.715241+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:38:02.890484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715496+0000","last_change":"2023-08-27T11:27:28.740166+0000","last_active":"2023-08-27T11:27:29.715496+0000","last_peered":"2023-08-27T11:27:29.715496+0000","last_clean":"2023-08-27T11:27:29.715496+0000","last_became_active":"2023-08-27T11:27:28.739685+0000","last_became_peered":"2023-08-27T11:27:28.739685+0000","last_unstale":"2023-08-27T11:27:29.715496+0000","last_undegraded":"2023-08-27T11:27:29.715496+0000","last_fullsized":"2023-08-27T11:27:29.715496+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:41:35.903009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301299+0000","last_change":"2023-08-27T11:27:28.745245+0000","last_active":"2023-08-27T11:27:30.301299+0000","last_peered":"2023-08-27T11:27:30.301299+0000","last_clean":"2023-08-27T11:27:30.301299+0000","last_became_active":"2023-08-27T11:27:28.743764+0000","last_became_peered":"2023-08-27T11:27:28.743764+0000","last_unstale":"2023-08-27T11:27:30.301299+0000","last_undegraded":"2023-08-27T11:27:30.301299+0000","last_fullsized":"2023-08-27T11:27:30.301299+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:38:55.395148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300754+0000","last_change":"2023-08-27T11:27:28.723904+0000","last_active":"2023-08-27T11:27:30.300754+0000","last_peered":"2023-08-27T11:27:30.300754+0000","last_clean":"2023-08-27T11:27:30.300754+0000","last_became_active":"2023-08-27T11:27:28.723641+0000","last_became_peered":"2023-08-27T11:27:28.723641+0000","last_unstale":"2023-08-27T11:27:30.300754+0000","last_undegraded":"2023-08-27T11:27:30.300754+0000","last_fullsized":"2023-08-27T11:27:30.300754+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:37:08.574504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716985+0000","last_change":"2023-08-27T11:27:28.725708+0000","last_active":"2023-08-27T11:27:29.716985+0000","last_peered":"2023-08-27T11:27:29.716985+0000","last_clean":"2023-08-27T11:27:29.716985+0000","last_became_active":"2023-08-27T11:27:28.725458+0000","last_became_peered":"2023-08-27T11:27:28.725458+0000","last_unstale":"2023-08-27T11:27:29.716985+0000","last_undegraded":"2023-08-27T11:27:29.716985+0000","last_fullsized":"2023-08-27T11:27:29.716985+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:27.276966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715595+0000","last_change":"2023-08-27T11:27:28.724498+0000","last_active":"2023-08-27T11:27:29.715595+0000","last_peered":"2023-08-27T11:27:29.715595+0000","last_clean":"2023-08-27T11:27:29.715595+0000","last_became_active":"2023-08-27T11:27:28.724212+0000","last_became_peered":"2023-08-27T11:27:28.724212+0000","last_unstale":"2023-08-27T11:27:29.715595+0000","last_undegraded":"2023-08-27T11:27:29.715595+0000","last_fullsized":"2023-08-27T11:27:29.715595+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:38.342129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715960+0000","last_change":"2023-08-27T11:27:28.725778+0000","last_active":"2023-08-27T11:27:29.715960+0000","last_peered":"2023-08-27T11:27:29.715960+0000","last_clean":"2023-08-27T11:27:29.715960+0000","last_became_active":"2023-08-27T11:27:28.725503+0000","last_became_peered":"2023-08-27T11:27:28.725503+0000","last_unstale":"2023-08-27T11:27:29.715960+0000","last_undegraded":"2023-08-27T11:27:29.715960+0000","last_fullsized":"2023-08-27T11:27:29.715960+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:02:07.186367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300448+0000","last_change":"2023-08-27T11:27:28.724944+0000","last_active":"2023-08-27T11:27:30.300448+0000","last_peered":"2023-08-27T11:27:30.300448+0000","last_clean":"2023-08-27T11:27:30.300448+0000","last_became_active":"2023-08-27T11:27:28.724643+0000","last_became_peered":"2023-08-27T11:27:28.724643+0000","last_unstale":"2023-08-27T11:27:30.300448+0000","last_undegraded":"2023-08-27T11:27:30.300448+0000","last_fullsized":"2023-08-27T11:27:30.300448+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:09:06.818940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716013+0000","last_change":"2023-08-27T11:27:28.738972+0000","last_active":"2023-08-27T11:27:29.716013+0000","last_peered":"2023-08-27T11:27:29.716013+0000","last_clean":"2023-08-27T11:27:29.716013+0000","last_became_active":"2023-08-27T11:27:28.738673+0000","last_became_peered":"2023-08-27T11:27:28.738673+0000","last_unstale":"2023-08-27T11:27:29.716013+0000","last_undegraded":"2023-08-27T11:27:29.716013+0000","last_fullsized":"2023-08-27T11:27:29.716013+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:35:48.007661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"113'4","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715428+0000","last_change":"2023-08-27T11:27:27.726870+0000","last_active":"2023-08-27T11:27:29.715428+0000","last_peered":"2023-08-27T11:27:29.715428+0000","last_clean":"2023-08-27T11:27:29.715428+0000","last_became_active":"2023-08-27T11:27:27.726617+0000","last_became_peered":"2023-08-27T11:27:27.726617+0000","last_unstale":"2023-08-27T11:27:29.715428+0000","last_undegraded":"2023-08-27T11:27:29.715428+0000","last_fullsized":"2023-08-27T11:27:29.715428+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:53:42.044040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713784+0000","last_change":"2023-08-27T11:27:28.733353+0000","last_active":"2023-08-27T11:27:29.713784+0000","last_peered":"2023-08-27T11:27:29.713784+0000","last_clean":"2023-08-27T11:27:29.713784+0000","last_became_active":"2023-08-27T11:27:28.733045+0000","last_became_peered":"2023-08-27T11:27:28.733045+0000","last_unstale":"2023-08-27T11:27:29.713784+0000","last_undegraded":"2023-08-27T11:27:29.713784+0000","last_fullsized":"2023-08-27T11:27:29.713784+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:12:21.038437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"113'1","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300624+0000","last_change":"2023-08-27T11:27:27.727923+0000","last_active":"2023-08-27T11:27:30.300624+0000","last_peered":"2023-08-27T11:27:30.300624+0000","last_clean":"2023-08-27T11:27:30.300624+0000","last_became_active":"2023-08-27T11:27:27.727656+0000","last_became_peered":"2023-08-27T11:27:27.727656+0000","last_unstale":"2023-08-27T11:27:30.300624+0000","last_undegraded":"2023-08-27T11:27:30.300624+0000","last_fullsized":"2023-08-27T11:27:30.300624+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:08:25.007899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300668+0000","last_change":"2023-08-27T11:27:28.741777+0000","last_active":"2023-08-27T11:27:30.300668+0000","last_peered":"2023-08-27T11:27:30.300668+0000","last_clean":"2023-08-27T11:27:30.300668+0000","last_became_active":"2023-08-27T11:27:28.741194+0000","last_became_peered":"2023-08-27T11:27:28.741194+0000","last_unstale":"2023-08-27T11:27:30.300668+0000","last_undegraded":"2023-08-27T11:27:30.300668+0000","last_fullsized":"2023-08-27T11:27:30.300668+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:03.492761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"113'2","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301180+0000","last_change":"2023-08-27T11:27:27.733316+0000","last_active":"2023-08-27T11:27:30.301180+0000","last_peered":"2023-08-27T11:27:30.301180+0000","last_clean":"2023-08-27T11:27:30.301180+0000","last_became_active":"2023-08-27T11:27:27.732469+0000","last_became_peered":"2023-08-27T11:27:27.732469+0000","last_unstale":"2023-08-27T11:27:30.301180+0000","last_undegraded":"2023-08-27T11:27:30.301180+0000","last_fullsized":"2023-08-27T11:27:30.301180+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:27:35.126554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713943+0000","last_change":"2023-08-27T11:27:28.743781+0000","last_active":"2023-08-27T11:27:29.713943+0000","last_peered":"2023-08-27T11:27:29.713943+0000","last_clean":"2023-08-27T11:27:29.713943+0000","last_became_active":"2023-08-27T11:27:28.743484+0000","last_became_peered":"2023-08-27T11:27:28.743484+0000","last_unstale":"2023-08-27T11:27:29.713943+0000","last_undegraded":"2023-08-27T11:27:29.713943+0000","last_fullsized":"2023-08-27T11:27:29.713943+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:43:38.690558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"113'2","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300788+0000","last_change":"2023-08-27T11:27:27.732724+0000","last_active":"2023-08-27T11:27:30.300788+0000","last_peered":"2023-08-27T11:27:30.300788+0000","last_clean":"2023-08-27T11:27:30.300788+0000","last_became_active":"2023-08-27T11:27:27.732421+0000","last_became_peered":"2023-08-27T11:27:27.732421+0000","last_unstale":"2023-08-27T11:27:30.300788+0000","last_undegraded":"2023-08-27T11:27:30.300788+0000","last_fullsized":"2023-08-27T11:27:30.300788+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:29.547142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716503+0000","last_change":"2023-08-27T11:27:28.726152+0000","last_active":"2023-08-27T11:27:29.716503+0000","last_peered":"2023-08-27T11:27:29.716503+0000","last_clean":"2023-08-27T11:27:29.716503+0000","last_became_active":"2023-08-27T11:27:28.725589+0000","last_became_peered":"2023-08-27T11:27:28.725589+0000","last_unstale":"2023-08-27T11:27:29.716503+0000","last_undegraded":"2023-08-27T11:27:29.716503+0000","last_fullsized":"2023-08-27T11:27:29.716503+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:08:38.551263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"113'1","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716257+0000","last_change":"2023-08-27T11:27:27.735989+0000","last_active":"2023-08-27T11:27:29.716257+0000","last_peered":"2023-08-27T11:27:29.716257+0000","last_clean":"2023-08-27T11:27:29.716257+0000","last_became_active":"2023-08-27T11:27:27.735725+0000","last_became_peered":"2023-08-27T11:27:27.735725+0000","last_unstale":"2023-08-27T11:27:29.716257+0000","last_undegraded":"2023-08-27T11:27:29.716257+0000","last_fullsized":"2023-08-27T11:27:29.716257+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:41:48.682595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716585+0000","last_change":"2023-08-27T11:27:28.728523+0000","last_active":"2023-08-27T11:27:29.716585+0000","last_peered":"2023-08-27T11:27:29.716585+0000","last_clean":"2023-08-27T11:27:29.716585+0000","last_became_active":"2023-08-27T11:27:28.727638+0000","last_became_peered":"2023-08-27T11:27:28.727638+0000","last_unstale":"2023-08-27T11:27:29.716585+0000","last_undegraded":"2023-08-27T11:27:29.716585+0000","last_fullsized":"2023-08-27T11:27:29.716585+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:57.063813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"113'1","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300156+0000","last_change":"2023-08-27T11:27:27.718129+0000","last_active":"2023-08-27T11:27:30.300156+0000","last_peered":"2023-08-27T11:27:30.300156+0000","last_clean":"2023-08-27T11:27:30.300156+0000","last_became_active":"2023-08-27T11:27:27.717804+0000","last_became_peered":"2023-08-27T11:27:27.717804+0000","last_unstale":"2023-08-27T11:27:30.300156+0000","last_undegraded":"2023-08-27T11:27:30.300156+0000","last_fullsized":"2023-08-27T11:27:30.300156+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:28:42.464473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300435+0000","last_change":"2023-08-27T11:27:28.739529+0000","last_active":"2023-08-27T11:27:30.300435+0000","last_peered":"2023-08-27T11:27:30.300435+0000","last_clean":"2023-08-27T11:27:30.300435+0000","last_became_active":"2023-08-27T11:27:28.739083+0000","last_became_peered":"2023-08-27T11:27:28.739083+0000","last_unstale":"2023-08-27T11:27:30.300435+0000","last_undegraded":"2023-08-27T11:27:30.300435+0000","last_fullsized":"2023-08-27T11:27:30.300435+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:28:23.709464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300803+0000","last_change":"2023-08-27T11:27:27.729848+0000","last_active":"2023-08-27T11:27:30.300803+0000","last_peered":"2023-08-27T11:27:30.300803+0000","last_clean":"2023-08-27T11:27:30.300803+0000","last_became_active":"2023-08-27T11:27:27.729580+0000","last_became_peered":"2023-08-27T11:27:27.729580+0000","last_unstale":"2023-08-27T11:27:30.300803+0000","last_undegraded":"2023-08-27T11:27:30.300803+0000","last_fullsized":"2023-08-27T11:27:30.300803+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:50:19.805790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301045+0000","last_change":"2023-08-27T11:27:28.739636+0000","last_active":"2023-08-27T11:27:30.301045+0000","last_peered":"2023-08-27T11:27:30.301045+0000","last_clean":"2023-08-27T11:27:30.301045+0000","last_became_active":"2023-08-27T11:27:28.739331+0000","last_became_peered":"2023-08-27T11:27:28.739331+0000","last_unstale":"2023-08-27T11:27:30.301045+0000","last_undegraded":"2023-08-27T11:27:30.301045+0000","last_fullsized":"2023-08-27T11:27:30.301045+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:51:52.004048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.714974+0000","last_change":"2023-08-27T11:27:27.726696+0000","last_active":"2023-08-27T11:27:29.714974+0000","last_peered":"2023-08-27T11:27:29.714974+0000","last_clean":"2023-08-27T11:27:29.714974+0000","last_became_active":"2023-08-27T11:27:27.725404+0000","last_became_peered":"2023-08-27T11:27:27.725404+0000","last_unstale":"2023-08-27T11:27:29.714974+0000","last_undegraded":"2023-08-27T11:27:29.714974+0000","last_fullsized":"2023-08-27T11:27:29.714974+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:33:06.170408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715133+0000","last_change":"2023-08-27T11:27:28.728224+0000","last_active":"2023-08-27T11:27:29.715133+0000","last_peered":"2023-08-27T11:27:29.715133+0000","last_clean":"2023-08-27T11:27:29.715133+0000","last_became_active":"2023-08-27T11:27:28.727758+0000","last_became_peered":"2023-08-27T11:27:28.727758+0000","last_unstale":"2023-08-27T11:27:29.715133+0000","last_undegraded":"2023-08-27T11:27:29.715133+0000","last_fullsized":"2023-08-27T11:27:29.715133+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:14:19.476647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"113'1","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300761+0000","last_change":"2023-08-27T11:27:27.738134+0000","last_active":"2023-08-27T11:27:30.300761+0000","last_peered":"2023-08-27T11:27:30.300761+0000","last_clean":"2023-08-27T11:27:30.300761+0000","last_became_active":"2023-08-27T11:27:27.737875+0000","last_became_peered":"2023-08-27T11:27:27.737875+0000","last_unstale":"2023-08-27T11:27:30.300761+0000","last_undegraded":"2023-08-27T11:27:30.300761+0000","last_fullsized":"2023-08-27T11:27:30.300761+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:08:25.007899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301248+0000","last_change":"2023-08-27T11:27:28.740180+0000","last_active":"2023-08-27T11:27:30.301248+0000","last_peered":"2023-08-27T11:27:30.301248+0000","last_clean":"2023-08-27T11:27:30.301248+0000","last_became_active":"2023-08-27T11:27:28.739919+0000","last_became_peered":"2023-08-27T11:27:28.739919+0000","last_unstale":"2023-08-27T11:27:30.301248+0000","last_undegraded":"2023-08-27T11:27:30.301248+0000","last_fullsized":"2023-08-27T11:27:30.301248+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:40.770172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"113'2","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715561+0000","last_change":"2023-08-27T11:27:27.726126+0000","last_active":"2023-08-27T11:27:29.715561+0000","last_peered":"2023-08-27T11:27:29.715561+0000","last_clean":"2023-08-27T11:27:29.715561+0000","last_became_active":"2023-08-27T11:27:27.725842+0000","last_became_peered":"2023-08-27T11:27:27.725842+0000","last_unstale":"2023-08-27T11:27:29.715561+0000","last_undegraded":"2023-08-27T11:27:29.715561+0000","last_fullsized":"2023-08-27T11:27:29.715561+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:22:33.212837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301408+0000","last_change":"2023-08-27T11:27:28.740851+0000","last_active":"2023-08-27T11:27:30.301408+0000","last_peered":"2023-08-27T11:27:30.301408+0000","last_clean":"2023-08-27T11:27:30.301408+0000","last_became_active":"2023-08-27T11:27:28.740529+0000","last_became_peered":"2023-08-27T11:27:28.740529+0000","last_unstale":"2023-08-27T11:27:30.301408+0000","last_undegraded":"2023-08-27T11:27:30.301408+0000","last_fullsized":"2023-08-27T11:27:30.301408+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:34:16.231819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"113'1","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301237+0000","last_change":"2023-08-27T11:27:27.729967+0000","last_active":"2023-08-27T11:27:30.301237+0000","last_peered":"2023-08-27T11:27:30.301237+0000","last_clean":"2023-08-27T11:27:30.301237+0000","last_became_active":"2023-08-27T11:27:27.729709+0000","last_became_peered":"2023-08-27T11:27:27.729709+0000","last_unstale":"2023-08-27T11:27:30.301237+0000","last_undegraded":"2023-08-27T11:27:30.301237+0000","last_fullsized":"2023-08-27T11:27:30.301237+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:18:41.998717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300838+0000","last_change":"2023-08-27T11:27:28.727499+0000","last_active":"2023-08-27T11:27:30.300838+0000","last_peered":"2023-08-27T11:27:30.300838+0000","last_clean":"2023-08-27T11:27:30.300838+0000","last_became_active":"2023-08-27T11:27:28.727109+0000","last_became_peered":"2023-08-27T11:27:28.727109+0000","last_unstale":"2023-08-27T11:27:30.300838+0000","last_undegraded":"2023-08-27T11:27:30.300838+0000","last_fullsized":"2023-08-27T11:27:30.300838+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:22:20.257671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300285+0000","last_change":"2023-08-27T11:27:27.722235+0000","last_active":"2023-08-27T11:27:30.300285+0000","last_peered":"2023-08-27T11:27:30.300285+0000","last_clean":"2023-08-27T11:27:30.300285+0000","last_became_active":"2023-08-27T11:27:27.721970+0000","last_became_peered":"2023-08-27T11:27:27.721970+0000","last_unstale":"2023-08-27T11:27:30.300285+0000","last_undegraded":"2023-08-27T11:27:30.300285+0000","last_fullsized":"2023-08-27T11:27:30.300285+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:48:12.711894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301115+0000","last_change":"2023-08-27T11:27:28.723187+0000","last_active":"2023-08-27T11:27:30.301115+0000","last_peered":"2023-08-27T11:27:30.301115+0000","last_clean":"2023-08-27T11:27:30.301115+0000","last_became_active":"2023-08-27T11:27:28.722906+0000","last_became_peered":"2023-08-27T11:27:28.722906+0000","last_unstale":"2023-08-27T11:27:30.301115+0000","last_undegraded":"2023-08-27T11:27:30.301115+0000","last_fullsized":"2023-08-27T11:27:30.301115+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:39:59.704373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"113'2","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716324+0000","last_change":"2023-08-27T11:27:27.723983+0000","last_active":"2023-08-27T11:27:29.716324+0000","last_peered":"2023-08-27T11:27:29.716324+0000","last_clean":"2023-08-27T11:27:29.716324+0000","last_became_active":"2023-08-27T11:27:27.723680+0000","last_became_peered":"2023-08-27T11:27:27.723680+0000","last_unstale":"2023-08-27T11:27:29.716324+0000","last_undegraded":"2023-08-27T11:27:29.716324+0000","last_fullsized":"2023-08-27T11:27:29.716324+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:32:40.666735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713935+0000","last_change":"2023-08-27T11:27:28.746409+0000","last_active":"2023-08-27T11:27:29.713935+0000","last_peered":"2023-08-27T11:27:29.713935+0000","last_clean":"2023-08-27T11:27:29.713935+0000","last_became_active":"2023-08-27T11:27:28.746109+0000","last_became_peered":"2023-08-27T11:27:28.746109+0000","last_unstale":"2023-08-27T11:27:29.713935+0000","last_undegraded":"2023-08-27T11:27:29.713935+0000","last_fullsized":"2023-08-27T11:27:29.713935+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:40.531052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713994+0000","last_change":"2023-08-27T11:27:27.726812+0000","last_active":"2023-08-27T11:27:29.713994+0000","last_peered":"2023-08-27T11:27:29.713994+0000","last_clean":"2023-08-27T11:27:29.713994+0000","last_became_active":"2023-08-27T11:27:27.726561+0000","last_became_peered":"2023-08-27T11:27:27.726561+0000","last_unstale":"2023-08-27T11:27:29.713994+0000","last_undegraded":"2023-08-27T11:27:29.713994+0000","last_fullsized":"2023-08-27T11:27:29.713994+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:09:23.080018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300584+0000","last_change":"2023-08-27T11:27:28.727953+0000","last_active":"2023-08-27T11:27:30.300584+0000","last_peered":"2023-08-27T11:27:30.300584+0000","last_clean":"2023-08-27T11:27:30.300584+0000","last_became_active":"2023-08-27T11:27:28.727657+0000","last_became_peered":"2023-08-27T11:27:28.727657+0000","last_unstale":"2023-08-27T11:27:30.300584+0000","last_undegraded":"2023-08-27T11:27:30.300584+0000","last_fullsized":"2023-08-27T11:27:30.300584+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:18:07.138304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301053+0000","last_change":"2023-08-27T11:27:27.732818+0000","last_active":"2023-08-27T11:27:30.301053+0000","last_peered":"2023-08-27T11:27:30.301053+0000","last_clean":"2023-08-27T11:27:30.301053+0000","last_became_active":"2023-08-27T11:27:27.732544+0000","last_became_peered":"2023-08-27T11:27:27.732544+0000","last_unstale":"2023-08-27T11:27:30.301053+0000","last_undegraded":"2023-08-27T11:27:30.301053+0000","last_fullsized":"2023-08-27T11:27:30.301053+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:37.233701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300663+0000","last_change":"2023-08-27T11:27:28.736577+0000","last_active":"2023-08-27T11:27:30.300663+0000","last_peered":"2023-08-27T11:27:30.300663+0000","last_clean":"2023-08-27T11:27:30.300663+0000","last_became_active":"2023-08-27T11:27:28.736220+0000","last_became_peered":"2023-08-27T11:27:28.736220+0000","last_unstale":"2023-08-27T11:27:30.300663+0000","last_undegraded":"2023-08-27T11:27:30.300663+0000","last_fullsized":"2023-08-27T11:27:30.300663+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:37.403227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301116+0000","last_change":"2023-08-27T11:27:27.738107+0000","last_active":"2023-08-27T11:27:30.301116+0000","last_peered":"2023-08-27T11:27:30.301116+0000","last_clean":"2023-08-27T11:27:30.301116+0000","last_became_active":"2023-08-27T11:27:27.737815+0000","last_became_peered":"2023-08-27T11:27:27.737815+0000","last_unstale":"2023-08-27T11:27:30.301116+0000","last_undegraded":"2023-08-27T11:27:30.301116+0000","last_fullsized":"2023-08-27T11:27:30.301116+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:24:14.996980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301137+0000","last_change":"2023-08-27T11:27:28.739803+0000","last_active":"2023-08-27T11:27:30.301137+0000","last_peered":"2023-08-27T11:27:30.301137+0000","last_clean":"2023-08-27T11:27:30.301137+0000","last_became_active":"2023-08-27T11:27:28.739028+0000","last_became_peered":"2023-08-27T11:27:28.739028+0000","last_unstale":"2023-08-27T11:27:30.301137+0000","last_undegraded":"2023-08-27T11:27:30.301137+0000","last_fullsized":"2023-08-27T11:27:30.301137+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:43:05.337653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"113'2","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713980+0000","last_change":"2023-08-27T11:27:27.723989+0000","last_active":"2023-08-27T11:27:29.713980+0000","last_peered":"2023-08-27T11:27:29.713980+0000","last_clean":"2023-08-27T11:27:29.713980+0000","last_became_active":"2023-08-27T11:27:27.723612+0000","last_became_peered":"2023-08-27T11:27:27.723612+0000","last_unstale":"2023-08-27T11:27:29.713980+0000","last_undegraded":"2023-08-27T11:27:29.713980+0000","last_fullsized":"2023-08-27T11:27:29.713980+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:06:59.013539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715062+0000","last_change":"2023-08-27T11:27:28.727442+0000","last_active":"2023-08-27T11:27:29.715062+0000","last_peered":"2023-08-27T11:27:29.715062+0000","last_clean":"2023-08-27T11:27:29.715062+0000","last_became_active":"2023-08-27T11:27:28.727145+0000","last_became_peered":"2023-08-27T11:27:28.727145+0000","last_unstale":"2023-08-27T11:27:29.715062+0000","last_undegraded":"2023-08-27T11:27:29.715062+0000","last_fullsized":"2023-08-27T11:27:29.715062+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:03:40.178562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716885+0000","last_change":"2023-08-27T11:27:27.728604+0000","last_active":"2023-08-27T11:27:29.716885+0000","last_peered":"2023-08-27T11:27:29.716885+0000","last_clean":"2023-08-27T11:27:29.716885+0000","last_became_active":"2023-08-27T11:27:27.728297+0000","last_became_peered":"2023-08-27T11:27:27.728297+0000","last_unstale":"2023-08-27T11:27:29.716885+0000","last_undegraded":"2023-08-27T11:27:29.716885+0000","last_fullsized":"2023-08-27T11:27:29.716885+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:01.533491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301332+0000","last_change":"2023-08-27T11:27:28.745857+0000","last_active":"2023-08-27T11:27:30.301332+0000","last_peered":"2023-08-27T11:27:30.301332+0000","last_clean":"2023-08-27T11:27:30.301332+0000","last_became_active":"2023-08-27T11:27:28.745614+0000","last_became_peered":"2023-08-27T11:27:28.745614+0000","last_unstale":"2023-08-27T11:27:30.301332+0000","last_undegraded":"2023-08-27T11:27:30.301332+0000","last_fullsized":"2023-08-27T11:27:30.301332+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:16:04.566393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716593+0000","last_change":"2023-08-27T11:27:27.726039+0000","last_active":"2023-08-27T11:27:29.716593+0000","last_peered":"2023-08-27T11:27:29.716593+0000","last_clean":"2023-08-27T11:27:29.716593+0000","last_became_active":"2023-08-27T11:27:27.725769+0000","last_became_peered":"2023-08-27T11:27:27.725769+0000","last_unstale":"2023-08-27T11:27:29.716593+0000","last_undegraded":"2023-08-27T11:27:29.716593+0000","last_fullsized":"2023-08-27T11:27:29.716593+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:27:30.629283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300612+0000","last_change":"2023-08-27T11:27:28.724809+0000","last_active":"2023-08-27T11:27:30.300612+0000","last_peered":"2023-08-27T11:27:30.300612+0000","last_clean":"2023-08-27T11:27:30.300612+0000","last_became_active":"2023-08-27T11:27:28.724540+0000","last_became_peered":"2023-08-27T11:27:28.724540+0000","last_unstale":"2023-08-27T11:27:30.300612+0000","last_undegraded":"2023-08-27T11:27:30.300612+0000","last_fullsized":"2023-08-27T11:27:30.300612+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:01:58.599489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301062+0000","last_change":"2023-08-27T11:27:27.732726+0000","last_active":"2023-08-27T11:27:30.301062+0000","last_peered":"2023-08-27T11:27:30.301062+0000","last_clean":"2023-08-27T11:27:30.301062+0000","last_became_active":"2023-08-27T11:27:27.732460+0000","last_became_peered":"2023-08-27T11:27:27.732460+0000","last_unstale":"2023-08-27T11:27:30.301062+0000","last_undegraded":"2023-08-27T11:27:30.301062+0000","last_fullsized":"2023-08-27T11:27:30.301062+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:08:02.907856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301034+0000","last_change":"2023-08-27T11:27:28.723945+0000","last_active":"2023-08-27T11:27:30.301034+0000","last_peered":"2023-08-27T11:27:30.301034+0000","last_clean":"2023-08-27T11:27:30.301034+0000","last_became_active":"2023-08-27T11:27:28.723684+0000","last_became_peered":"2023-08-27T11:27:28.723684+0000","last_unstale":"2023-08-27T11:27:30.301034+0000","last_undegraded":"2023-08-27T11:27:30.301034+0000","last_fullsized":"2023-08-27T11:27:30.301034+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:16:01.763110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300357+0000","last_change":"2023-08-27T11:27:27.730150+0000","last_active":"2023-08-27T11:27:30.300357+0000","last_peered":"2023-08-27T11:27:30.300357+0000","last_clean":"2023-08-27T11:27:30.300357+0000","last_became_active":"2023-08-27T11:27:27.729865+0000","last_became_peered":"2023-08-27T11:27:27.729865+0000","last_unstale":"2023-08-27T11:27:30.300357+0000","last_undegraded":"2023-08-27T11:27:30.300357+0000","last_fullsized":"2023-08-27T11:27:30.300357+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:48:12.711894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300976+0000","last_change":"2023-08-27T11:27:28.722696+0000","last_active":"2023-08-27T11:27:30.300976+0000","last_peered":"2023-08-27T11:27:30.300976+0000","last_clean":"2023-08-27T11:27:30.300976+0000","last_became_active":"2023-08-27T11:27:28.722445+0000","last_became_peered":"2023-08-27T11:27:28.722445+0000","last_unstale":"2023-08-27T11:27:30.300976+0000","last_undegraded":"2023-08-27T11:27:30.300976+0000","last_fullsized":"2023-08-27T11:27:30.300976+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:17:52.218419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301026+0000","last_change":"2023-08-27T11:27:27.733453+0000","last_active":"2023-08-27T11:27:30.301026+0000","last_peered":"2023-08-27T11:27:30.301026+0000","last_clean":"2023-08-27T11:27:30.301026+0000","last_became_active":"2023-08-27T11:27:27.732510+0000","last_became_peered":"2023-08-27T11:27:27.732510+0000","last_unstale":"2023-08-27T11:27:30.301026+0000","last_undegraded":"2023-08-27T11:27:30.301026+0000","last_fullsized":"2023-08-27T11:27:30.301026+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:43:07.897186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301299+0000","last_change":"2023-08-27T11:27:28.739728+0000","last_active":"2023-08-27T11:27:30.301299+0000","last_peered":"2023-08-27T11:27:30.301299+0000","last_clean":"2023-08-27T11:27:30.301299+0000","last_became_active":"2023-08-27T11:27:28.739440+0000","last_became_peered":"2023-08-27T11:27:28.739440+0000","last_unstale":"2023-08-27T11:27:30.301299+0000","last_undegraded":"2023-08-27T11:27:30.301299+0000","last_fullsized":"2023-08-27T11:27:30.301299+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:29:15.252048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301182+0000","last_change":"2023-08-27T11:27:27.733485+0000","last_active":"2023-08-27T11:27:30.301182+0000","last_peered":"2023-08-27T11:27:30.301182+0000","last_clean":"2023-08-27T11:27:30.301182+0000","last_became_active":"2023-08-27T11:27:27.733239+0000","last_became_peered":"2023-08-27T11:27:27.733239+0000","last_unstale":"2023-08-27T11:27:30.301182+0000","last_undegraded":"2023-08-27T11:27:30.301182+0000","last_fullsized":"2023-08-27T11:27:30.301182+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:00:41.683486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301249+0000","last_change":"2023-08-27T11:27:28.746868+0000","last_active":"2023-08-27T11:27:30.301249+0000","last_peered":"2023-08-27T11:27:30.301249+0000","last_clean":"2023-08-27T11:27:30.301249+0000","last_became_active":"2023-08-27T11:27:28.746590+0000","last_became_peered":"2023-08-27T11:27:28.746590+0000","last_unstale":"2023-08-27T11:27:30.301249+0000","last_undegraded":"2023-08-27T11:27:30.301249+0000","last_fullsized":"2023-08-27T11:27:30.301249+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:14:02.387821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300505+0000","last_change":"2023-08-27T11:27:27.721893+0000","last_active":"2023-08-27T11:27:30.300505+0000","last_peered":"2023-08-27T11:27:30.300505+0000","last_clean":"2023-08-27T11:27:30.300505+0000","last_became_active":"2023-08-27T11:27:27.721626+0000","last_became_peered":"2023-08-27T11:27:27.721626+0000","last_unstale":"2023-08-27T11:27:30.300505+0000","last_undegraded":"2023-08-27T11:27:30.300505+0000","last_fullsized":"2023-08-27T11:27:30.300505+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:02.487272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715366+0000","last_change":"2023-08-27T11:27:28.728000+0000","last_active":"2023-08-27T11:27:29.715366+0000","last_peered":"2023-08-27T11:27:29.715366+0000","last_clean":"2023-08-27T11:27:29.715366+0000","last_became_active":"2023-08-27T11:27:28.726716+0000","last_became_peered":"2023-08-27T11:27:28.726716+0000","last_unstale":"2023-08-27T11:27:29.715366+0000","last_undegraded":"2023-08-27T11:27:29.715366+0000","last_fullsized":"2023-08-27T11:27:29.715366+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:28:07.254859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"113'3","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301393+0000","last_change":"2023-08-27T11:27:27.733703+0000","last_active":"2023-08-27T11:27:30.301393+0000","last_peered":"2023-08-27T11:27:30.301393+0000","last_clean":"2023-08-27T11:27:30.301393+0000","last_became_active":"2023-08-27T11:27:27.733428+0000","last_became_peered":"2023-08-27T11:27:27.733428+0000","last_unstale":"2023-08-27T11:27:30.301393+0000","last_undegraded":"2023-08-27T11:27:30.301393+0000","last_fullsized":"2023-08-27T11:27:30.301393+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:50:35.605909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":211,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715941+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:27:29.715941+0000","last_peered":"2023-08-27T11:27:29.715941+0000","last_clean":"2023-08-27T11:27:29.715941+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:27:29.715941+0000","last_undegraded":"2023-08-27T11:27:29.715941+0000","last_fullsized":"2023-08-27T11:27:29.715941+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:35:46.774169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.717067+0000","last_change":"2023-08-27T11:27:28.727375+0000","last_active":"2023-08-27T11:27:29.717067+0000","last_peered":"2023-08-27T11:27:29.717067+0000","last_clean":"2023-08-27T11:27:29.717067+0000","last_became_active":"2023-08-27T11:27:28.727057+0000","last_became_peered":"2023-08-27T11:27:28.727057+0000","last_unstale":"2023-08-27T11:27:29.717067+0000","last_undegraded":"2023-08-27T11:27:29.717067+0000","last_fullsized":"2023-08-27T11:27:29.717067+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:36:13.007592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"113'6","reported_seq":19,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300964+0000","last_change":"2023-08-27T11:27:27.733450+0000","last_active":"2023-08-27T11:27:30.300964+0000","last_peered":"2023-08-27T11:27:30.300964+0000","last_clean":"2023-08-27T11:27:30.300964+0000","last_became_active":"2023-08-27T11:27:27.733128+0000","last_became_peered":"2023-08-27T11:27:27.733128+0000","last_unstale":"2023-08-27T11:27:30.300964+0000","last_undegraded":"2023-08-27T11:27:30.300964+0000","last_fullsized":"2023-08-27T11:27:30.300964+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:27:17.357947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":211,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715676+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:27:29.715676+0000","last_peered":"2023-08-27T11:27:29.715676+0000","last_clean":"2023-08-27T11:27:29.715676+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:27:29.715676+0000","last_undegraded":"2023-08-27T11:27:29.715676+0000","last_fullsized":"2023-08-27T11:27:29.715676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:52:02.533670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300873+0000","last_change":"2023-08-27T11:27:28.722079+0000","last_active":"2023-08-27T11:27:30.300873+0000","last_peered":"2023-08-27T11:27:30.300873+0000","last_clean":"2023-08-27T11:27:30.300873+0000","last_became_active":"2023-08-27T11:27:28.721811+0000","last_became_peered":"2023-08-27T11:27:28.721811+0000","last_unstale":"2023-08-27T11:27:30.300873+0000","last_undegraded":"2023-08-27T11:27:30.300873+0000","last_fullsized":"2023-08-27T11:27:30.300873+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:09:31.693592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"113'2","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.714028+0000","last_change":"2023-08-27T11:27:27.725475+0000","last_active":"2023-08-27T11:27:29.714028+0000","last_peered":"2023-08-27T11:27:29.714028+0000","last_clean":"2023-08-27T11:27:29.714028+0000","last_became_active":"2023-08-27T11:27:27.725243+0000","last_became_peered":"2023-08-27T11:27:27.725243+0000","last_unstale":"2023-08-27T11:27:29.714028+0000","last_undegraded":"2023-08-27T11:27:29.714028+0000","last_fullsized":"2023-08-27T11:27:29.714028+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:30.027570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":180,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300443+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:27:30.300443+0000","last_peered":"2023-08-27T11:27:30.300443+0000","last_clean":"2023-08-27T11:27:30.300443+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:27:30.300443+0000","last_undegraded":"2023-08-27T11:27:30.300443+0000","last_fullsized":"2023-08-27T11:27:30.300443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:29:30.311219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715473+0000","last_change":"2023-08-27T11:27:28.722192+0000","last_active":"2023-08-27T11:27:29.715473+0000","last_peered":"2023-08-27T11:27:29.715473+0000","last_clean":"2023-08-27T11:27:29.715473+0000","last_became_active":"2023-08-27T11:27:28.721959+0000","last_became_peered":"2023-08-27T11:27:28.721959+0000","last_unstale":"2023-08-27T11:27:29.715473+0000","last_undegraded":"2023-08-27T11:27:29.715473+0000","last_fullsized":"2023-08-27T11:27:29.715473+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:38:58.901876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301171+0000","last_change":"2023-08-27T11:27:27.732793+0000","last_active":"2023-08-27T11:27:30.301171+0000","last_peered":"2023-08-27T11:27:30.301171+0000","last_clean":"2023-08-27T11:27:30.301171+0000","last_became_active":"2023-08-27T11:27:27.732489+0000","last_became_peered":"2023-08-27T11:27:27.732489+0000","last_unstale":"2023-08-27T11:27:30.301171+0000","last_undegraded":"2023-08-27T11:27:30.301171+0000","last_fullsized":"2023-08-27T11:27:30.301171+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:10:21.200976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":178,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300303+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:27:30.300303+0000","last_peered":"2023-08-27T11:27:30.300303+0000","last_clean":"2023-08-27T11:27:30.300303+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:27:30.300303+0000","last_undegraded":"2023-08-27T11:27:30.300303+0000","last_fullsized":"2023-08-27T11:27:30.300303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:52.610932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300995+0000","last_change":"2023-08-27T11:27:28.724456+0000","last_active":"2023-08-27T11:27:30.300995+0000","last_peered":"2023-08-27T11:27:30.300995+0000","last_clean":"2023-08-27T11:27:30.300995+0000","last_became_active":"2023-08-27T11:27:28.724165+0000","last_became_peered":"2023-08-27T11:27:28.724165+0000","last_unstale":"2023-08-27T11:27:30.300995+0000","last_undegraded":"2023-08-27T11:27:30.300995+0000","last_fullsized":"2023-08-27T11:27:30.300995+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:59:20.756731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"113'8","reported_seq":21,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300717+0000","last_change":"2023-08-27T11:27:27.730253+0000","last_active":"2023-08-27T11:27:30.300717+0000","last_peered":"2023-08-27T11:27:30.300717+0000","last_clean":"2023-08-27T11:27:30.300717+0000","last_became_active":"2023-08-27T11:27:27.730010+0000","last_became_peered":"2023-08-27T11:27:27.730010+0000","last_unstale":"2023-08-27T11:27:30.300717+0000","last_undegraded":"2023-08-27T11:27:30.300717+0000","last_fullsized":"2023-08-27T11:27:30.300717+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:06.778260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":178,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300966+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:27:30.300966+0000","last_peered":"2023-08-27T11:27:30.300966+0000","last_clean":"2023-08-27T11:27:30.300966+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:27:30.300966+0000","last_undegraded":"2023-08-27T11:27:30.300966+0000","last_fullsized":"2023-08-27T11:27:30.300966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:19:33.541372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.301125+0000","last_change":"2023-08-27T11:27:28.745302+0000","last_active":"2023-08-27T11:27:30.301125+0000","last_peered":"2023-08-27T11:27:30.301125+0000","last_clean":"2023-08-27T11:27:30.301125+0000","last_became_active":"2023-08-27T11:27:28.743659+0000","last_became_peered":"2023-08-27T11:27:28.743659+0000","last_unstale":"2023-08-27T11:27:30.301125+0000","last_undegraded":"2023-08-27T11:27:30.301125+0000","last_fullsized":"2023-08-27T11:27:30.301125+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:53.775941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.715164+0000","last_change":"2023-08-27T11:27:27.728183+0000","last_active":"2023-08-27T11:27:29.715164+0000","last_peered":"2023-08-27T11:27:29.715164+0000","last_clean":"2023-08-27T11:27:29.715164+0000","last_became_active":"2023-08-27T11:27:27.727887+0000","last_became_peered":"2023-08-27T11:27:27.727887+0000","last_unstale":"2023-08-27T11:27:29.715164+0000","last_undegraded":"2023-08-27T11:27:29.715164+0000","last_fullsized":"2023-08-27T11:27:29.715164+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:34.243069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":328,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300140+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:27:30.300140+0000","last_peered":"2023-08-27T11:27:30.300140+0000","last_clean":"2023-08-27T11:27:30.300140+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:27:30.300140+0000","last_undegraded":"2023-08-27T11:27:30.300140+0000","last_fullsized":"2023-08-27T11:27:30.300140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:48:51.726242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":184,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300800+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:27:30.300800+0000","last_peered":"2023-08-27T11:27:30.300800+0000","last_clean":"2023-08-27T11:27:30.300800+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:27:30.300800+0000","last_undegraded":"2023-08-27T11:27:30.300800+0000","last_fullsized":"2023-08-27T11:27:30.300800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:03:13.742093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300703+0000","last_change":"2023-08-27T11:27:28.727300+0000","last_active":"2023-08-27T11:27:30.300703+0000","last_peered":"2023-08-27T11:27:30.300703+0000","last_clean":"2023-08-27T11:27:30.300703+0000","last_became_active":"2023-08-27T11:27:28.727029+0000","last_became_peered":"2023-08-27T11:27:28.727029+0000","last_unstale":"2023-08-27T11:27:30.300703+0000","last_undegraded":"2023-08-27T11:27:30.300703+0000","last_fullsized":"2023-08-27T11:27:30.300703+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:22:46.441644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.714066+0000","last_change":"2023-08-27T11:27:27.725362+0000","last_active":"2023-08-27T11:27:29.714066+0000","last_peered":"2023-08-27T11:27:29.714066+0000","last_clean":"2023-08-27T11:27:29.714066+0000","last_became_active":"2023-08-27T11:27:27.725113+0000","last_became_peered":"2023-08-27T11:27:27.725113+0000","last_unstale":"2023-08-27T11:27:29.714066+0000","last_undegraded":"2023-08-27T11:27:29.714066+0000","last_fullsized":"2023-08-27T11:27:29.714066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:23:49.650754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":178,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300835+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:27:30.300835+0000","last_peered":"2023-08-27T11:27:30.300835+0000","last_clean":"2023-08-27T11:27:30.300835+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:27:30.300835+0000","last_undegraded":"2023-08-27T11:27:30.300835+0000","last_fullsized":"2023-08-27T11:27:30.300835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:33:36.130112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.713726+0000","last_change":"2023-08-27T11:27:28.725207+0000","last_active":"2023-08-27T11:27:29.713726+0000","last_peered":"2023-08-27T11:27:29.713726+0000","last_clean":"2023-08-27T11:27:29.713726+0000","last_became_active":"2023-08-27T11:27:28.724890+0000","last_became_peered":"2023-08-27T11:27:28.724890+0000","last_unstale":"2023-08-27T11:27:29.713726+0000","last_undegraded":"2023-08-27T11:27:29.713726+0000","last_fullsized":"2023-08-27T11:27:29.713726+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:27.698071+0000","last_clean_scrub_stamp":"2023-08-27T11:27:27.698071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:05.392066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:30.300617+0000","last_change":"2023-08-27T11:27:27.728162+0000","last_active":"2023-08-27T11:27:30.300617+0000","last_peered":"2023-08-27T11:27:30.300617+0000","last_clean":"2023-08-27T11:27:30.300617+0000","last_became_active":"2023-08-27T11:27:27.727886+0000","last_became_peered":"2023-08-27T11:27:27.727886+0000","last_unstale":"2023-08-27T11:27:30.300617+0000","last_undegraded":"2023-08-27T11:27:30.300617+0000","last_fullsized":"2023-08-27T11:27:30.300617+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:26.692583+0000","last_clean_scrub_stamp":"2023-08-27T11:27:26.692583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:18:06.132815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":202,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-27T11:27:29.716638+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:27:29.716638+0000","last_peered":"2023-08-27T11:27:29.716638+0000","last_clean":"2023-08-27T11:27:29.716638+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:27:29.716638+0000","last_undegraded":"2023-08-27T11:27:29.716638+0000","last_fullsized":"2023-08-27T11:27:29.716638+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:40:27.374079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574979,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2035712,"data_stored":1250354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.35499999999999998,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.749,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.52500000000000002,"15min":0.501},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.66800000000000004,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.496}]},{"osd":1,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54800000000000004,"15min":0.51400000000000001},"min":{"1min":0.39100000000000001,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.73499999999999999,"5min":0.77500000000000002,"15min":0.872},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54200000000000004,"15min":0.52000000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.69399999999999995,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.54000000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51400000000000001,"15min":0.49099999999999999},"min":{"1min":0.34499999999999997,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.77400000000000002,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.45800000000000002,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.90200000000000002},"last":0.52600000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57499999999999996,"15min":0.55400000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77300000000000002,"5min":0.996,"15min":0.996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.60199999999999998},"min":{"1min":0.45900000000000002,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.79600000000000004,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.62}]},{"osd":4,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.504},"min":{"1min":0.39600000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.75,"5min":0.75,"15min":0.77400000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73799999999999999,"5min":1.224,"15min":1.224},"last":0.45000000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.38900000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.84799999999999998},"last":0.57499999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.48899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80600000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.39800000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80700000000000005,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.56599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":658956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.441,"5min":0.33500000000000002,"15min":0.29099999999999998},"max":{"1min":0.70299999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.54100000000000004},"min":{"1min":0.35899999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.89800000000000002},"last":0.80300000000000005}]},{"osd":1,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.316,"15min":0.312},"max":{"1min":0.67800000000000005,"5min":0.80600000000000005,"15min":0.873},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.751,"5min":0.75600000000000001,"15min":0.84399999999999997},"last":0.54700000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.42299999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.67200000000000004,"5min":0.72499999999999998,"15min":0.85899999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55600000000000005,"15min":0.57899999999999996},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.754,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.73099999999999998,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.48599999999999999,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.72799999999999998,"5min":0.80700000000000005,"15min":0.93799999999999994},"last":0.63200000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.28199999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.70699999999999996,"5min":0.71299999999999997,"15min":0.83099999999999996},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.374},"max":{"1min":0.81299999999999994,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.60799999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.35099999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.623,"15min":0.60399999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.33900000000000002},"max":{"1min":0.76400000000000001,"5min":0.78700000000000003,"15min":0.875},"last":0.46899999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.32200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78800000000000003,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.34899999999999998},"max":{"1min":0.82999999999999996,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.77900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574979,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2023424,"data_stored":1248722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.311,"15min":0.55400000000000005},"min":{"1min":0.19500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.42599999999999999,"5min":0.502,"15min":11.971},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.30099999999999999,"5min":0.33200000000000002,"15min":0.80800000000000005},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.39500000000000002,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.34399999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.34000000000000002,"15min":0.58299999999999996},"min":{"1min":0.27700000000000002,"5min":0.23400000000000001,"15min":0.221},"max":{"1min":0.45500000000000002,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.317,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.22800000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.38800000000000001,"5min":0.52000000000000002,"15min":0.58599999999999997},"last":0.371}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38800000000000001,"15min":0.60399999999999998},"min":{"1min":0.27500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.47399999999999998,"5min":0.58099999999999996,"15min":10.99},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.372,"15min":0.84499999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23499999999999999},"max":{"1min":0.45000000000000001,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.41699999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57399999999999995,"15min":0.59499999999999997},"min":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.33400000000000002},"max":{"1min":0.79500000000000004,"5min":0.90700000000000003,"15min":1.26},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.77300000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":1.0409999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36199999999999999},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.46300000000000002,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.71599999999999997,"5min":0.88700000000000001,"15min":1.296},"last":0.78100000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.80400000000000005},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.371},"max":{"1min":0.83599999999999997,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":1.0589999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81200000000000006,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.72999999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":1.0529999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.84199999999999997,"15min":22.988},"last":0.752},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.85899999999999999,"5min":0.86399999999999999,"15min":1.327},"last":0.66300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574979,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1429504,"data_stored":656979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.311},"min":{"1min":0.223,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.46200000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.435},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.33300000000000002,"15min":0.34899999999999998},"min":{"1min":0.253,"5min":0.223,"15min":0.223},"max":{"1min":0.5,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.34799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34100000000000003},"min":{"1min":0.251,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.44800000000000001,"5min":0.628,"15min":0.63200000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.33900000000000002},"min":{"1min":0.26600000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51000000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34599999999999997},"min":{"1min":0.247,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47099999999999997,"5min":0.50800000000000001,"15min":0.59599999999999997},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.371,"15min":0.373},"min":{"1min":0.30299999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.60599999999999998},"last":0.41299999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.85399999999999998,"15min":1.214},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.42999999999999999,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.88400000000000001,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.627}]},{"osd":5,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.41099999999999998,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.72599999999999998,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.40200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78500000000000003,"5min":0.85399999999999998,"15min":1.603},"last":0.67900000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89000000000000001,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.39900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.878,"15min":0.878},"last":0.71499999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.39900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.79,"5min":3.79,"15min":3.79},"last":0.69899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":658900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.29899999999999999,"15min":0.30199999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.45300000000000001,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.377},{"interface":"front","average":{"1min":0.30099999999999999,"5min":0.313,"15min":0.316},"min":{"1min":0.24099999999999999,"5min":0.222,"15min":0.19700000000000001},"max":{"1min":0.36199999999999999,"5min":0.52500000000000002,"15min":0.629},"last":0.29799999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.31900000000000001,"15min":0.32300000000000001},"min":{"1min":0.23499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.40699999999999997,"5min":0.51100000000000001,"15min":0.54400000000000004},"last":0.251},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33700000000000002,"15min":0.34499999999999997},"min":{"1min":0.26000000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.439,"5min":0.623,"15min":0.749},"last":0.34499999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.36699999999999999,"15min":0.38},"min":{"1min":0.245,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.44500000000000001,"5min":0.59499999999999997,"15min":0.73699999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38100000000000001,"15min":0.39600000000000002},"min":{"1min":0.33000000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.49199999999999999,"5min":0.57199999999999995,"15min":0.69499999999999995},"last":0.44800000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.375,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.76600000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.879},"last":0.41799999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72199999999999998,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.495},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.76100000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.42899999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73799999999999999,"5min":0.82899999999999996,"15min":0.85299999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.44,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78100000000000003,"5min":0.80500000000000005,"15min":0.83599999999999997},"last":0.47899999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.41799999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.747,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.41399999999999998,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.84899999999999998},"last":0.46200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":659966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.29699999999999999,"15min":0.30099999999999999},"min":{"1min":0.23699999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41399999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32000000000000001,"15min":0.314},"min":{"1min":0.26000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.41699999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.27800000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.316},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.19500000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.34200000000000003},"min":{"1min":0.27600000000000002,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":0.45200000000000001,"5min":0.52300000000000002,"15min":0.56299999999999994},"last":0.38300000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.29299999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.36299999999999999,"15min":0.36599999999999999},"min":{"1min":0.27500000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.50600000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.441}]},{"osd":4,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.438,"5min":0.33900000000000002,"15min":0.28599999999999998},"max":{"1min":0.753,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.5}]},{"osd":5,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.314},"max":{"1min":0.78500000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.83799999999999997,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.51100000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59799999999999998,"15min":0.60199999999999998},"min":{"1min":0.434,"5min":0.379,"15min":0.373},"max":{"1min":0.82799999999999996,"5min":1.091,"15min":1.091},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.71399999999999997,"5min":1.111,"15min":1.111},"last":0.47999999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.60799999999999998},"min":{"1min":0.39600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":1.032,"15min":1.236},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.41599999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85899999999999999,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.41099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":658284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51000000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64100000000000001,"5min":0.997,"15min":0.997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.497,"15min":0.496},"min":{"1min":0.36899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.79100000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40899999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50600000000000001,"15min":0.51500000000000001},"min":{"1min":0.34899999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.76500000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.34000000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.73099999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.498}]},{"osd":2,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69899999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52500000000000002,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69199999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.44500000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54500000000000004,"15min":0.53700000000000003},"min":{"1min":0.439,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.66600000000000004,"5min":0.78200000000000003,"15min":0.79400000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80600000000000005},"last":0.51600000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.49299999999999999,"15min":0.51500000000000001},"min":{"1min":0.32600000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68300000000000005,"5min":0.81100000000000005,"15min":0.85899999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.42399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75900000000000001,"5min":0.85999999999999999,"15min":0.879},"last":0.53300000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68799999999999994,"5min":0.78900000000000003,"15min":0.90600000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59399999999999997,"15min":0.621},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77000000000000002,"5min":0.79300000000000004,"15min":0.92300000000000004},"last":0.58199999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71499999999999997,"5min":0.80600000000000005,"15min":0.82499999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.60699999999999998},"min":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.71399999999999997,"5min":0.79200000000000004,"15min":0.89300000000000002},"last":0.629}]}]},{"osd":5,"up_from":13,"seq":55834574979,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1429504,"data_stored":658306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73899999999999999,"5min":0.753,"15min":0.96899999999999997},"last":0.377},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39300000000000002,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.68200000000000005,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65800000000000003,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68400000000000005,"5min":0.72899999999999998,"15min":1.075},"last":0.48499999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.33200000000000002},"max":{"1min":0.72099999999999997,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.41099999999999998,"5min":0.373,"15min":0.307},"max":{"1min":0.753,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.59599999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":1.03},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.35199999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":1.026},"last":0.55300000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.371,"5min":0.31,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.50700000000000001,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79600000000000004,"5min":0.83899999999999997,"15min":0.91400000000000003},"last":0.53700000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.79400000000000004,"5min":0.79700000000000004,"15min":0.88200000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":0.81499999999999995,"15min":0.94199999999999995},"last":0.44700000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.41499999999999998,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.95899999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.31},"max":{"1min":0.82299999999999995,"5min":0.85599999999999998,"15min":0.92000000000000004},"last":0.629}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:27:33.216 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:27:33.217 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:27:33.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.376+0000 7f04be01d700 1 Processor -- start 2023-08-27T11:27:33.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.376+0000 7f04be01d700 1 -- start start 2023-08-27T11:27:33.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.377+0000 7f04be01d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f04b80b3de0 0x7f04b806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:33.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.377+0000 7f04be01d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04b806c510 0x7f04b80cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:33.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.377+0000 7f04be01d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b806c960 con 0x7f04b8144b80 2023-08-27T11:27:33.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.377+0000 7f04be01d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b80b4b40 con 0x7f04b80b3de0 2023-08-27T11:27:33.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.377+0000 7f04be01d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b80cef20 con 0x7f04b806c510 2023-08-27T11:27:33.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.377+0000 7f04b6ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f04b80b3de0 0x7f04b806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.377+0000 7f04b77fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f04b8144b80 0x7f04b80b6c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57638/0 (socket says 172.21.15.112:57638) 2023-08-27T11:27:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.377+0000 7f04b77fe700 1 -- 172.21.15.112:0/3180216045 learned_addr learned my addr 172.21.15.112:0/3180216045 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04b6ffd700 1 -- 172.21.15.112:0/3180216045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04b806c510 msgr2=0x7f04b80cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04b6ffd700 1 --2- 172.21.15.112:0/3180216045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04b806c510 0x7f04b80cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04b6ffd700 1 -- 172.21.15.112:0/3180216045 >> v1:172.21.15.112:6789/0 conn(0x7f04b8144b80 legacy=0x7f04b80b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04b6ffd700 1 -- 172.21.15.112:0/3180216045 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b80b7310 con 0x7f04b80b3de0 2023-08-27T11:27:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04b67fc700 1 -- 172.21.15.112:0/3180216045 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508927336 0 0) 0x7f04a8001490 con 0x7f04b8144b80 2023-08-27T11:27:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04b6ffd700 1 --2- 172.21.15.112:0/3180216045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b80b3de0 0x7f04b806bfd0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009840 tx=0x7f04a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229c0601d82d27bb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04b67fc700 1 -- 172.21.15.112:0/3180216045 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0011830 con 0x7f04b80b3de0 2023-08-27T11:27:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04b67fc700 1 -- 172.21.15.112:0/3180216045 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a0011990 con 0x7f04b80b3de0 2023-08-27T11:27:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04b67fc700 1 -- 172.21.15.112:0/3180216045 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0011d20 con 0x7f04b80b3de0 2023-08-27T11:27:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04be01d700 1 -- 172.21.15.112:0/3180216045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b80b3de0 msgr2=0x7f04b806bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04be01d700 1 --2- 172.21.15.112:0/3180216045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b80b3de0 0x7f04b806bfd0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009840 tx=0x7f04a0009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:33.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04be01d700 1 -- 172.21.15.112:0/3180216045 shutdown_connections 2023-08-27T11:27:33.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04be01d700 1 --2- 172.21.15.112:0/3180216045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04b806c510 0x7f04b80cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04be01d700 1 --2- 172.21.15.112:0/3180216045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b80b3de0 0x7f04b806bfd0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04be01d700 1 -- 172.21.15.112:0/3180216045 >> 172.21.15.112:0/3180216045 conn(0x7f04b80cdc50 msgr2=0x7f04b80ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:33.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04be01d700 1 -- 172.21.15.112:0/3180216045 shutdown_connections 2023-08-27T11:27:33.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.378+0000 7f04be01d700 1 -- 172.21.15.112:0/3180216045 wait complete. 2023-08-27T11:27:33.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.380+0000 7f04be01d700 1 Processor -- start 2023-08-27T11:27:33.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.380+0000 7f04be01d700 1 -- start start 2023-08-27T11:27:33.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.380+0000 7f04be01d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b806c510 0x7f04b81f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:33.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.380+0000 7f04be01d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04b80b3de0 0x7f04b8158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:33.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.380+0000 7f04be01d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b8144b80 0x7f04b80c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:33.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.380+0000 7f04be01d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b81f2270 con 0x7f04b8144b80 2023-08-27T11:27:33.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.380+0000 7f04be01d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b80d02c0 con 0x7f04b806c510 2023-08-27T11:27:33.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04be01d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b8158bc0 con 0x7f04b80b3de0 2023-08-27T11:27:33.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b8144b80 0x7f04b80c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b8144b80 0x7f04b80c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35034/0 (socket says 172.21.15.112:35034) 2023-08-27T11:27:33.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04b7fff700 1 -- 172.21.15.112:0/1666417403 learned_addr learned my addr 172.21.15.112:0/1666417403 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:33.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04b6ffd700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04b80b3de0 0x7f04b8158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04b7fff700 1 -- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04b80b3de0 msgr2=0x7f04b8158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04b77fe700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b806c510 0x7f04b81f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04b7fff700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04b80b3de0 0x7f04b8158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04b7fff700 1 -- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b806c510 msgr2=0x7f04b81f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04b7fff700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b806c510 0x7f04b81f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04b7fff700 1 -- 172.21.15.112:0/1666417403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a00094a0 con 0x7f04b8144b80 2023-08-27T11:27:33.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04b7fff700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b8144b80 0x7f04b80c7ad0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f04b000b580 tx=0x7f04b000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:33.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04affff700 1 -- 172.21.15.112:0/1666417403 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0011830 con 0x7f04b8144b80 2023-08-27T11:27:33.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04affff700 1 -- 172.21.15.112:0/1666417403 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b0011990 con 0x7f04b8144b80 2023-08-27T11:27:33.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04be01d700 1 -- 172.21.15.112:0/1666417403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b81f2460 con 0x7f04b8144b80 2023-08-27T11:27:33.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04affff700 1 -- 172.21.15.112:0/1666417403 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0011d20 con 0x7f04b8144b80 2023-08-27T11:27:33.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.381+0000 7f04be01d700 1 -- 172.21.15.112:0/1666417403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b81f2a20 con 0x7f04b8144b80 2023-08-27T11:27:33.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.382+0000 7f04be01d700 1 -- 172.21.15.112:0/1666417403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b8067a20 con 0x7f04b8144b80 2023-08-27T11:27:33.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.383+0000 7f04affff700 1 -- 172.21.15.112:0/1666417403 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f04b001a400 con 0x7f04b8144b80 2023-08-27T11:27:33.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.383+0000 7f04affff700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f049806a4d0 0x7f049806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:33.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.383+0000 7f04affff700 1 -- 172.21.15.112:0/1666417403 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f04b0088d40 con 0x7f04b8144b80 2023-08-27T11:27:33.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.383+0000 7f04b77fe700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f049806a4d0 0x7f049806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.384+0000 7f04b77fe700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f049806a4d0 0x7f049806c990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f04a80012e0 tx=0x7f04a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:33.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.387+0000 7f04affff700 1 -- 172.21.15.112:0/1666417403 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f04b0053e50 con 0x7f04b8144b80 2023-08-27T11:27:33.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.653+0000 7f04be01d700 1 -- 172.21.15.112:0/1666417403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f04b80b0b70 con 0x7f04b8144b80 2023-08-27T11:27:33.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.657+0000 7f04affff700 1 -- 172.21.15.112:0/1666417403 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f04b00150e0 con 0x7f04b8144b80 2023-08-27T11:27:33.659 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-08-27T11:27:33.659 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA== 2023-08-27T11:27:33.659 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2023-08-27T11:27:33.660 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2023-08-27T11:27:33.660 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:27:33.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.660+0000 7f04be01d700 1 -- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f049806a4d0 msgr2=0x7f049806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.660+0000 7f04be01d700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f049806a4d0 0x7f049806c990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f04a80012e0 tx=0x7f04a8009040 comp rx=0 tx=0).stop 2023-08-27T11:27:33.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.660+0000 7f04be01d700 1 -- 172.21.15.112:0/1666417403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b8144b80 msgr2=0x7f04b80c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.660+0000 7f04be01d700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b8144b80 0x7f04b80c7ad0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f04b000b580 tx=0x7f04b000b940 comp rx=0 tx=0).stop 2023-08-27T11:27:33.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.660+0000 7f04be01d700 1 -- 172.21.15.112:0/1666417403 shutdown_connections 2023-08-27T11:27:33.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.660+0000 7f04be01d700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f049806a4d0 0x7f049806c990 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.660+0000 7f04be01d700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b8144b80 0x7f04b80c7ad0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.660+0000 7f04be01d700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b806c510 0x7f04b81f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.660+0000 7f04be01d700 1 --2- 172.21.15.112:0/1666417403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04b80b3de0 0x7f04b8158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.660+0000 7f04be01d700 1 -- 172.21.15.112:0/1666417403 >> 172.21.15.112:0/1666417403 conn(0x7f04b80cdc50 msgr2=0x7f04b80b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.660+0000 7f04be01d700 1 -- 172.21.15.112:0/1666417403 shutdown_connections 2023-08-27T11:27:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.660+0000 7f04be01d700 1 -- 172.21.15.112:0/1666417403 wait complete. 2023-08-27T11:27:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2023-08-27T11:27:33.674 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:33.696 INFO:tasks.ceph.mds.b.smithi138.stderr:2023-08-27T11:27:33.696+0000 7faa2a7a8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:27:33.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.792+0000 7fc90b8d5700 1 Processor -- start 2023-08-27T11:27:33.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.793+0000 7fc90b8d5700 1 -- start start 2023-08-27T11:27:33.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.793+0000 7fc90b8d5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc904066720 0x7fc90415e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:33.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.793+0000 7fc90b8d5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc9040cec70 0x7fc9040708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:33.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.793+0000 7fc90b8d5700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc904072250 con 0x7fc904146c00 2023-08-27T11:27:33.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.793+0000 7fc90b8d5700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9040d0b60 con 0x7fc9040cec70 2023-08-27T11:27:33.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.793+0000 7fc90b8d5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040724a0 con 0x7fc904066720 2023-08-27T11:27:33.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc909e72700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc9040cec70 0x7fc9040708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc909671700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc904146c00 0x7fc904066010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57652/0 (socket says 172.21.15.112:57652) 2023-08-27T11:27:33.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc909671700 1 -- 172.21.15.112:0/242832140 learned_addr learned my addr 172.21.15.112:0/242832140 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:33.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc908e70700 1 --2- 172.21.15.112:0/242832140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc904066720 0x7fc90415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc8f7fff700 1 -- 172.21.15.112:0/242832140 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361267799 0 0) 0x7fc904072250 con 0x7fc904146c00 2023-08-27T11:27:33.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc8f7fff700 1 -- 172.21.15.112:0/242832140 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8f00035b0 con 0x7fc904146c00 2023-08-27T11:27:33.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc909e72700 1 -- 172.21.15.112:0/242832140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc904066720 msgr2=0x7fc90415e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:33.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc909e72700 1 --2- 172.21.15.112:0/242832140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc904066720 0x7fc90415e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc909e72700 1 -- 172.21.15.112:0/242832140 >> v1:172.21.15.112:6789/0 conn(0x7fc904146c00 legacy=0x7fc904066010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc909e72700 1 -- 172.21.15.112:0/242832140 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc904072780 con 0x7fc9040cec70 2023-08-27T11:27:33.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc909e72700 1 --2- 172.21.15.112:0/242832140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9040cec70 0x7fc9040708a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009d90 tx=0x7fc8fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa3268201deffdd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:33.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc8f7fff700 1 -- 172.21.15.112:0/242832140 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc00e070 con 0x7fc9040cec70 2023-08-27T11:27:33.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc8f7fff700 1 -- 172.21.15.112:0/242832140 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc00bb20 con 0x7fc9040cec70 2023-08-27T11:27:33.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.794+0000 7fc8f7fff700 1 -- 172.21.15.112:0/242832140 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc012430 con 0x7fc9040cec70 2023-08-27T11:27:33.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.795+0000 7fc90b8d5700 1 -- 172.21.15.112:0/242832140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9040cec70 msgr2=0x7fc9040708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.795+0000 7fc90b8d5700 1 --2- 172.21.15.112:0/242832140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9040cec70 0x7fc9040708a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009d90 tx=0x7fc8fc00b330 comp rx=0 tx=0).stop 2023-08-27T11:27:33.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.795+0000 7fc90b8d5700 1 -- 172.21.15.112:0/242832140 shutdown_connections 2023-08-27T11:27:33.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.795+0000 7fc90b8d5700 1 --2- 172.21.15.112:0/242832140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9040cec70 0x7fc9040708a0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.795+0000 7fc90b8d5700 1 --2- 172.21.15.112:0/242832140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc904066720 0x7fc90415e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.795+0000 7fc90b8d5700 1 -- 172.21.15.112:0/242832140 >> 172.21.15.112:0/242832140 conn(0x7fc90406f970 msgr2=0x7fc90406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:33.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.795+0000 7fc90b8d5700 1 -- 172.21.15.112:0/242832140 shutdown_connections 2023-08-27T11:27:33.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.795+0000 7fc90b8d5700 1 -- 172.21.15.112:0/242832140 wait complete. 2023-08-27T11:27:33.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.796+0000 7fc90b8d5700 1 Processor -- start 2023-08-27T11:27:33.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.796+0000 7fc90b8d5700 1 -- start start 2023-08-27T11:27:33.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc90b8d5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc904066720 0x7fc904159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:33.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc90b8d5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9040cec70 0x7fc9040b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:33.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc90b8d5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc904146c00 0x7fc9041f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:33.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc90b8d5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040ce1f0 con 0x7fc904066720 2023-08-27T11:27:33.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc90b8d5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040ce4d0 con 0x7fc904146c00 2023-08-27T11:27:33.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc90b8d5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9041f2840 con 0x7fc9040cec70 2023-08-27T11:27:33.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc909671700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc904066720 0x7fc904159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc909671700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc904066720 0x7fc904159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35044/0 (socket says 172.21.15.112:35044) 2023-08-27T11:27:33.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc909671700 1 -- 172.21.15.112:0/3352523124 learned_addr learned my addr 172.21.15.112:0/3352523124 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:33.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc909671700 1 -- 172.21.15.112:0/3352523124 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9040cec70 msgr2=0x7fc9040b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc909671700 1 --2- 172.21.15.112:0/3352523124 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9040cec70 0x7fc9040b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc909e72700 1 --2- 172.21.15.112:0/3352523124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc904146c00 0x7fc9041f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc909671700 1 -- 172.21.15.112:0/3352523124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc904146c00 msgr2=0x7fc9041f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:33.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc909671700 1 --2- 172.21.15.112:0/3352523124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc904146c00 0x7fc9041f3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:33.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc909671700 1 -- 172.21.15.112:0/3352523124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc009a70 con 0x7fc904066720 2023-08-27T11:27:33.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.797+0000 7fc909671700 1 --2- 172.21.15.112:0/3352523124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc904066720 0x7fc904159c90 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800c700 tx=0x7fc8f800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:33.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.798+0000 7fc8f5ffb700 1 -- 172.21.15.112:0/3352523124 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8005d70 con 0x7fc904066720 2023-08-27T11:27:33.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.798+0000 7fc8f5ffb700 1 -- 172.21.15.112:0/3352523124 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f80070d0 con 0x7fc904066720 2023-08-27T11:27:33.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.798+0000 7fc90b8d5700 1 -- 172.21.15.112:0/3352523124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9041f2b20 con 0x7fc904066720 2023-08-27T11:27:33.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.798+0000 7fc8f5ffb700 1 -- 172.21.15.112:0/3352523124 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8007460 con 0x7fc904066720 2023-08-27T11:27:33.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.798+0000 7fc90b8d5700 1 -- 172.21.15.112:0/3352523124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9041f3860 con 0x7fc904066720 2023-08-27T11:27:33.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.799+0000 7fc90b8d5700 1 -- 172.21.15.112:0/3352523124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc904067a20 con 0x7fc904066720 2023-08-27T11:27:33.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.799+0000 7fc8f5ffb700 1 -- 172.21.15.112:0/3352523124 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fc8f80075c0 con 0x7fc904066720 2023-08-27T11:27:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.799+0000 7fc8f5ffb700 1 --2- 172.21.15.112:0/3352523124 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc8e806a4d0 0x7fc8e806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.800+0000 7fc8f5ffb700 1 -- 172.21.15.112:0/3352523124 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc8f80882a0 con 0x7fc904066720 2023-08-27T11:27:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.800+0000 7fc908e70700 1 --2- 172.21.15.112:0/3352523124 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc8e806a4d0 0x7fc8e806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.800+0000 7fc908e70700 1 --2- 172.21.15.112:0/3352523124 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc8e806a4d0 0x7fc8e806c990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc90413cd90 tx=0x7fc900008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:33.804+0000 7fc8f5ffb700 1 -- 172.21.15.112:0/3352523124 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc8f80533c0 con 0x7fc904066720 2023-08-27T11:27:34.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.036+0000 7fc90b8d5700 1 -- 172.21.15.112:0/3352523124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9040caa60 con 0x7fc904066720 2023-08-27T11:27:34.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.036+0000 7fc8f5ffb700 1 -- 172.21.15.112:0/3352523124 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4889 (secure 0 0 0) 0x7fc8f80569e0 con 0x7fc904066720 2023-08-27T11:27:34.038 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:34.038 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1795721853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1795721853},{"type":"v1","addr":"172.21.15.138:6836","nonce":1795721853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":7974,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3832579050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3832579050},{"type":"v1","addr":"172.21.15.112:6834","nonce":3832579050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4019485943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":4019485943},{"type":"v1","addr":"172.21.15.112:6835","nonce":4019485943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:27:28.107305+0000","modified":"2023-08-27T11:27:31.742912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7989},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7989":{"gid":7989,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/3245198376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3245198376},{"type":"v1","addr":"172.21.15.138:6837","nonce":3245198376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-27T11:27:34.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.038+0000 7fc90b8d5700 1 -- 172.21.15.112:0/3352523124 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc8e806a4d0 msgr2=0x7fc8e806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.038+0000 7fc90b8d5700 1 --2- 172.21.15.112:0/3352523124 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc8e806a4d0 0x7fc8e806c990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc90413cd90 tx=0x7fc900008040 comp rx=0 tx=0).stop 2023-08-27T11:27:34.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.038+0000 7fc90b8d5700 1 -- 172.21.15.112:0/3352523124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc904066720 msgr2=0x7fc904159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.038+0000 7fc90b8d5700 1 --2- 172.21.15.112:0/3352523124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc904066720 0x7fc904159c90 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800c700 tx=0x7fc8f800cac0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.038+0000 7fc90b8d5700 1 -- 172.21.15.112:0/3352523124 shutdown_connections 2023-08-27T11:27:34.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.038+0000 7fc90b8d5700 1 --2- 172.21.15.112:0/3352523124 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc8e806a4d0 0x7fc8e806c990 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.038+0000 7fc90b8d5700 1 --2- 172.21.15.112:0/3352523124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc904146c00 0x7fc9041f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.038+0000 7fc90b8d5700 1 --2- 172.21.15.112:0/3352523124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc904066720 0x7fc904159c90 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.038+0000 7fc90b8d5700 1 --2- 172.21.15.112:0/3352523124 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc9040cec70 0x7fc9040b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.038+0000 7fc90b8d5700 1 -- 172.21.15.112:0/3352523124 >> 172.21.15.112:0/3352523124 conn(0x7fc90406f970 msgr2=0x7fc9040d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:34.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.039+0000 7fc90b8d5700 1 -- 172.21.15.112:0/3352523124 shutdown_connections 2023-08-27T11:27:34.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.039+0000 7fc90b8d5700 1 -- 172.21.15.112:0/3352523124 wait complete. 2023-08-27T11:27:34.045 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 76 2023-08-27T11:27:34.455 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:27:28.107305+0000', 'modified': '2023-08-27T11:27:31.742912+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7989}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7989': {'gid': 7989, 'name': 'b', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6837/3245198376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6835', 'nonce': 3245198376}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 3245198376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:27:34.455 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:27:34.456 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-27T11:27:34.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.578+0000 7fa5dc8d2700 1 Processor -- start 2023-08-27T11:27:34.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.578+0000 7fa5dc8d2700 1 -- start start 2023-08-27T11:27:34.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.579+0000 7fa5dc8d2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa5d806fea0 0x7fa5d80b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.579+0000 7fa5dc8d2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d806e3a0 0x7fa5d80af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.579+0000 7fa5dc8d2700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d815c190 con 0x7fa5d8158e20 2023-08-27T11:27:34.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.579+0000 7fa5dc8d2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d80b1730 con 0x7fa5d806fea0 2023-08-27T11:27:34.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.579+0000 7fa5dc8d2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d8159200 con 0x7fa5d806e3a0 2023-08-27T11:27:34.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.580+0000 7fa5d5d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa5d806fea0 0x7fa5d80b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.580+0000 7fa5d5d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa5d806fea0 0x7fa5d80b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35054/0 (socket says 172.21.15.112:35054) 2023-08-27T11:27:34.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.580+0000 7fa5d5d9b700 1 -- 172.21.15.112:0/969088154 learned_addr learned my addr 172.21.15.112:0/969088154 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:34.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.580+0000 7fa5d5d9b700 1 -- 172.21.15.112:0/969088154 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d806e3a0 msgr2=0x7fa5d80af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:34.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.580+0000 7fa5d6d9d700 1 --2- 172.21.15.112:0/969088154 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d806e3a0 0x7fa5d80af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.580+0000 7fa5d5d9b700 1 --2- 172.21.15.112:0/969088154 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d806e3a0 0x7fa5d80af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.580+0000 7fa5d5d9b700 1 -- 172.21.15.112:0/969088154 >> v1:172.21.15.138:6789/0 conn(0x7fa5d8158e20 legacy=0x7fa5d806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:34.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.580+0000 7fa5d5d9b700 1 -- 172.21.15.112:0/969088154 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d80b1910 con 0x7fa5d806fea0 2023-08-27T11:27:34.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.580+0000 7fa5d5d9b700 1 --2- 172.21.15.112:0/969088154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5d806fea0 0x7fa5d80b11f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0098f0 tx=0x7fa5cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b8eb5807e2b96e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:34.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.580+0000 7fa5d559a700 1 -- 172.21.15.112:0/969088154 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc004230 con 0x7fa5d806fea0 2023-08-27T11:27:34.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.580+0000 7fa5d559a700 1 -- 172.21.15.112:0/969088154 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc004390 con 0x7fa5d806fea0 2023-08-27T11:27:34.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.580+0000 7fa5d559a700 1 -- 172.21.15.112:0/969088154 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0036a0 con 0x7fa5d806fea0 2023-08-27T11:27:34.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.581+0000 7fa5dc8d2700 1 -- 172.21.15.112:0/969088154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5d806fea0 msgr2=0x7fa5d80b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.581+0000 7fa5dc8d2700 1 --2- 172.21.15.112:0/969088154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5d806fea0 0x7fa5d80b11f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0098f0 tx=0x7fa5cc009c00 comp rx=0 tx=0).stop 2023-08-27T11:27:34.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.581+0000 7fa5dc8d2700 1 -- 172.21.15.112:0/969088154 shutdown_connections 2023-08-27T11:27:34.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.581+0000 7fa5dc8d2700 1 --2- 172.21.15.112:0/969088154 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d806e3a0 0x7fa5d80af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.581+0000 7fa5dc8d2700 1 --2- 172.21.15.112:0/969088154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5d806fea0 0x7fa5d80b11f0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.581+0000 7fa5dc8d2700 1 -- 172.21.15.112:0/969088154 >> 172.21.15.112:0/969088154 conn(0x7fa5d80d0000 msgr2=0x7fa5d80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:34.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.581+0000 7fa5dc8d2700 1 -- 172.21.15.112:0/969088154 shutdown_connections 2023-08-27T11:27:34.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.581+0000 7fa5dc8d2700 1 -- 172.21.15.112:0/969088154 wait complete. 2023-08-27T11:27:34.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.582+0000 7fa5dc8d2700 1 Processor -- start 2023-08-27T11:27:34.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.582+0000 7fa5dc8d2700 1 -- start start 2023-08-27T11:27:34.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5dc8d2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5d806e3a0 0x7fa5d80c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5dc8d2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5d806fea0 0x7fa5d80c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5dc8d2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d8158e20 0x7fa5d80c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5dc8d2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d81551e0 con 0x7fa5d806fea0 2023-08-27T11:27:34.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5dc8d2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d80c5210 con 0x7fa5d806e3a0 2023-08-27T11:27:34.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5dc8d2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d80c4c00 con 0x7fa5d8158e20 2023-08-27T11:27:34.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5d5d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5d806fea0 0x7fa5d80c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5d5d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5d806fea0 0x7fa5d80c56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35060/0 (socket says 172.21.15.112:35060) 2023-08-27T11:27:34.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5d5d9b700 1 -- 172.21.15.112:0/129994421 learned_addr learned my addr 172.21.15.112:0/129994421 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:34.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5d6d9d700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d8158e20 0x7fa5d80c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5d659c700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5d806e3a0 0x7fa5d80c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5d5d9b700 1 -- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d8158e20 msgr2=0x7fa5d80c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5d5d9b700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d8158e20 0x7fa5d80c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5d5d9b700 1 -- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5d806e3a0 msgr2=0x7fa5d80c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5d5d9b700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5d806e3a0 0x7fa5d80c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.583+0000 7fa5d5d9b700 1 -- 172.21.15.112:0/129994421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc009550 con 0x7fa5d806fea0 2023-08-27T11:27:34.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.584+0000 7fa5d5d9b700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5d806fea0 0x7fa5d80c56a0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00b0e0 tx=0x7fa5cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:34.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.584+0000 7fa5c6ffd700 1 -- 172.21.15.112:0/129994421 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc003e30 con 0x7fa5d806fea0 2023-08-27T11:27:34.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.584+0000 7fa5c6ffd700 1 -- 172.21.15.112:0/129994421 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc01f070 con 0x7fa5d806fea0 2023-08-27T11:27:34.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.584+0000 7fa5c6ffd700 1 -- 172.21.15.112:0/129994421 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc00b8a0 con 0x7fa5d806fea0 2023-08-27T11:27:34.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.584+0000 7fa5dc8d2700 1 -- 172.21.15.112:0/129994421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d8155430 con 0x7fa5d806fea0 2023-08-27T11:27:34.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.584+0000 7fa5dc8d2700 1 -- 172.21.15.112:0/129994421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d8155710 con 0x7fa5d806fea0 2023-08-27T11:27:34.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.585+0000 7fa5c6ffd700 1 -- 172.21.15.112:0/129994421 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa5cc021030 con 0x7fa5d806fea0 2023-08-27T11:27:34.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.585+0000 7fa5c6ffd700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa5b806a4d0 0x7fa5b806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.586+0000 7fa5c6ffd700 1 -- 172.21.15.112:0/129994421 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa5cc08a210 con 0x7fa5d806fea0 2023-08-27T11:27:34.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.586+0000 7fa5d659c700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa5b806a4d0 0x7fa5b806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.586+0000 7fa5d659c700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa5b806a4d0 0x7fa5b806c990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009870 tx=0x7fa5c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:34.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.588+0000 7fa5e8a94b80 1 Processor -- start 2023-08-27T11:27:34.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.588+0000 7fa5e8a94b80 1 -- start start 2023-08-27T11:27:34.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.588+0000 7fa5e8a94b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814e16f40 0x55c814e17320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.588+0000 7fa5e8a94b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e28960 0x55c814e13b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.588+0000 7fa5e8a94b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 0x55c814e140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.588+0000 7fa5e8a94b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c814d45730 con 0x55c814dea3d0 2023-08-27T11:27:34.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.588+0000 7fa5e8a94b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c814d43050 con 0x55c814e16f40 2023-08-27T11:27:34.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.588+0000 7fa5e8a94b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c814d3e940 con 0x55c814e28960 2023-08-27T11:27:34.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5d6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 0x55c814e140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5d6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 0x55c814e140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35070/0 (socket says 172.21.15.112:35070) 2023-08-27T11:27:34.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5d6d9d700 1 -- 172.21.15.112:0/633158543 learned_addr learned my addr 172.21.15.112:0/633158543 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:34.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5d5d9b700 1 --2- 172.21.15.112:0/633158543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e28960 0x55c814e13b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5d659c700 1 --2- 172.21.15.112:0/633158543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814e16f40 0x55c814e17320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5d6d9d700 1 -- 172.21.15.112:0/633158543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e28960 msgr2=0x55c814e13b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:34.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5d6d9d700 1 --2- 172.21.15.112:0/633158543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e28960 0x55c814e13b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5d6d9d700 1 -- 172.21.15.112:0/633158543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814e16f40 msgr2=0x55c814e17320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:34.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5d6d9d700 1 --2- 172.21.15.112:0/633158543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814e16f40 0x55c814e17320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5d6d9d700 1 -- 172.21.15.112:0/633158543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c814bbe5e0 con 0x55c814dea3d0 2023-08-27T11:27:34.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5d6d9d700 1 --2- 172.21.15.112:0/633158543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 0x55c814e140b0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e3f0 tx=0x7fa5d000e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2865f9c4dc4a6d09 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:34.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5b7fff700 1 -- 172.21.15.112:0/633158543 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0018430 con 0x55c814dea3d0 2023-08-27T11:27:34.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5b7fff700 1 -- 172.21.15.112:0/633158543 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d0013070 con 0x55c814dea3d0 2023-08-27T11:27:34.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.589+0000 7fa5b7fff700 1 -- 172.21.15.112:0/633158543 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0018790 con 0x55c814dea3d0 2023-08-27T11:27:34.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.590+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/633158543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 msgr2=0x55c814e140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.590+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/633158543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 0x55c814e140b0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e3f0 tx=0x7fa5d000e700 comp rx=0 tx=0).stop 2023-08-27T11:27:34.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.590+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/633158543 shutdown_connections 2023-08-27T11:27:34.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.590+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/633158543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 0x55c814e140b0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.590+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/633158543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814e16f40 0x55c814e17320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.590+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/633158543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e28960 0x55c814e13b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.590+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/633158543 >> 172.21.15.112:0/633158543 conn(0x55c814d4ac30 msgr2=0x55c814e31d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:34.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.590+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/633158543 shutdown_connections 2023-08-27T11:27:34.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.590+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/633158543 wait complete. 2023-08-27T11:27:34.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.591+0000 7fa5e8a94b80 1 -- start start 2023-08-27T11:27:34.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.591+0000 7fa5e8a94b80 1 Processor -- start 2023-08-27T11:27:34.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.591+0000 7fa5e8a94b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 0x55c814e0a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.591+0000 7fa5e8a94b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 0x55c814e14ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.591+0000 7fa5e8a94b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814e28960 0x55c814e12a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5e8a94b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c814cdcb60 con 0x55c814dea3d0 2023-08-27T11:27:34.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5e8a94b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c814cdc860 con 0x55c814e28960 2023-08-27T11:27:34.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5e8a94b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c814bc08d0 con 0x55c814e16f40 2023-08-27T11:27:34.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5e8a94b80 10 client.0 _ll_register_callbacks cb 0x55c814cda630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:27:34.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5d659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 0x55c814e0a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5d659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 0x55c814e0a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35072/0 (socket says 172.21.15.112:35072) 2023-08-27T11:27:34.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5d659c700 1 -- 172.21.15.112:0/752523083 learned_addr learned my addr 172.21.15.112:0/752523083 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:34.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5d6d9d700 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814e28960 0x55c814e12a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5d5d9b700 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 0x55c814e14ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5d659c700 1 -- 172.21.15.112:0/752523083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 msgr2=0x55c814e14ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5d659c700 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 0x55c814e14ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5d659c700 1 -- 172.21.15.112:0/752523083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814e28960 msgr2=0x55c814e12a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5d659c700 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814e28960 0x55c814e12a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5d659c700 1 -- 172.21.15.112:0/752523083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d0007c60 con 0x55c814dea3d0 2023-08-27T11:27:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5d6d9d700 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814e28960 0x55c814e12a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5d659c700 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 0x55c814e0a180 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8014ce0 tx=0x7fa5c8004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65700684a5b96018 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:27:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5b57fa700 1 -- 172.21.15.112:0/752523083 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8012b30 con 0x55c814dea3d0 2023-08-27T11:27:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5b57fa700 1 -- 172.21.15.112:0/752523083 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa5c8012c90 con 0x55c814dea3d0 2023-08-27T11:27:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/752523083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c814ddf4f0 con 0x55c814dea3d0 2023-08-27T11:27:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5b57fa700 1 -- 172.21.15.112:0/752523083 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5c8012e50 con 0x55c814dea3d0 2023-08-27T11:27:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5e8a94b80 10 client.8112 fetch_fsmap learned FSMap version 76 2023-08-27T11:27:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.592+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/752523083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55c814d56750 con 0x55c814dea3d0 2023-08-27T11:27:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.593+0000 7fa5b57fa700 1 -- 172.21.15.112:0/752523083 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c814ddf4f0 con 0x55c814dea3d0 2023-08-27T11:27:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.593+0000 7fa5b57fa700 1 -- 172.21.15.112:0/752523083 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa5c80059a0 con 0x55c814dea3d0 2023-08-27T11:27:34.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.593+0000 7fa5e8a94b80 10 client.8112 fetch_fsmap finished waiting for FSMap version 76 2023-08-27T11:27:34.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.593+0000 7fa5e8a94b80 10 client.8112 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-27T11:27:34.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.593+0000 7fa5e8a94b80 20 client.8112 populate_metadata read hostname 'smithi112' 2023-08-27T11:27:34.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.593+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.138:6835/3245198376,v1:172.21.15.138:6837/3245198376] conn(0x55c814e18980 0x55c814e18d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.593+0000 7fa5e8a94b80 4 client.8112 mds_command: new command op to 7989 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:27:34.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.593+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/752523083 --> [v2:172.21.15.138:6835/3245198376,v1:172.21.15.138:6837/3245198376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c814d37920 con 0x55c814e18980 2023-08-27T11:27:34.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.593+0000 7fa5d5d9b700 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.138:6835/3245198376,v1:172.21.15.138:6837/3245198376] conn(0x55c814e18980 0x55c814e18d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.593+0000 7fa5b57fa700 10 client.8112 ms_handle_connect on v2:172.21.15.138:6835/3245198376 2023-08-27T11:27:34.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.593+0000 7fa5d5d9b700 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.138:6835/3245198376,v1:172.21.15.138:6837/3245198376] conn(0x55c814e18980 0x55c814e18d60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3efd324445ea5936 server_cookie=799583bca7a39098 in_seq=0 out_seq=0 2023-08-27T11:27:34.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5b57fa700 1 -- 172.21.15.112:0/752523083 <== mds.0 v2:172.21.15.138:6835/3245198376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c814d37920 con 0x55c814e18980 2023-08-27T11:27:34.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5b57fa700 10 client.8112 handle_command_reply: tid=0 2023-08-27T11:27:34.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5e8a94b80 1 client.8112 shutdown 2023-08-27T11:27:34.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/752523083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 msgr2=0x55c814e0a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 0x55c814e0a180 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8014ce0 tx=0x7fa5c8004710 comp rx=0 tx=0).stop 2023-08-27T11:27:34.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/752523083 shutdown_connections 2023-08-27T11:27:34.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.138:6835/3245198376,v1:172.21.15.138:6837/3245198376] conn(0x55c814e18980 0x55c814e18d60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814e28960 0x55c814e12a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814dea3d0 0x55c814e0a180 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5b57fa700 0 client.8112 ms_handle_reset on v2:172.21.15.138:6835/3245198376 2023-08-27T11:27:34.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/752523083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 0x55c814e14ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/752523083 >> 172.21.15.112:0/752523083 conn(0x55c814d4ac30 msgr2=0x55c814e10a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:34.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/752523083 shutdown_connections 2023-08-27T11:27:34.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/752523083 wait complete. 2023-08-27T11:27:34.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.594+0000 7fa5e8a94b80 20 client.8112 trim_cache size 0 max 16384 2023-08-27T11:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.612+0000 7fa5e8a94b80 1 Processor -- start 2023-08-27T11:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.612+0000 7fa5e8a94b80 1 -- start start 2023-08-27T11:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5e8a94b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814dea3d0 0x55c814e19da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5e8a94b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 0x55c814de93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5e8a94b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 0x55c814e16540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5e8a94b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c814de3960 con 0x55c814e28960 2023-08-27T11:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5e8a94b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c814e11340 con 0x55c814dea3d0 2023-08-27T11:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5e8a94b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c814e17ef0 con 0x55c814e16f40 2023-08-27T11:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5d6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 0x55c814e16540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5d6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 0x55c814e16540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35082/0 (socket says 172.21.15.112:35082) 2023-08-27T11:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5d6d9d700 1 -- 172.21.15.112:0/167942000 learned_addr learned my addr 172.21.15.112:0/167942000 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5d659c700 1 --2- 172.21.15.112:0/167942000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814dea3d0 0x55c814e19da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5d5d9b700 1 --2- 172.21.15.112:0/167942000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 0x55c814de93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5d6d9d700 1 -- 172.21.15.112:0/167942000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 msgr2=0x55c814de93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5d6d9d700 1 --2- 172.21.15.112:0/167942000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 0x55c814de93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5d6d9d700 1 -- 172.21.15.112:0/167942000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814dea3d0 msgr2=0x55c814e19da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5d6d9d700 1 --2- 172.21.15.112:0/167942000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814dea3d0 0x55c814e19da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5d6d9d700 1 -- 172.21.15.112:0/167942000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c814b6ad90 con 0x55c814e28960 2023-08-27T11:27:34.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5d5d9b700 1 --2- 172.21.15.112:0/167942000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 0x55c814de93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:34.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5d659c700 1 --2- 172.21.15.112:0/167942000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814dea3d0 0x55c814e19da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:34.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.613+0000 7fa5d6d9d700 1 --2- 172.21.15.112:0/167942000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 0x55c814e16540 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e740 tx=0x7fa5d0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d6c9c2c55581c8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:34.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.614+0000 7fa5c4ff9700 1 -- 172.21.15.112:0/167942000 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000ee30 con 0x55c814e28960 2023-08-27T11:27:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.614+0000 7fa5c4ff9700 1 -- 172.21.15.112:0/167942000 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d0003a20 con 0x55c814e28960 2023-08-27T11:27:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.614+0000 7fa5c4ff9700 1 -- 172.21.15.112:0/167942000 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0003dd0 con 0x55c814e28960 2023-08-27T11:27:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.614+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/167942000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 msgr2=0x55c814e16540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.614+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/167942000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 0x55c814e16540 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e740 tx=0x7fa5d0011000 comp rx=0 tx=0).stop 2023-08-27T11:27:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.614+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/167942000 shutdown_connections 2023-08-27T11:27:34.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.614+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/167942000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 0x55c814e16540 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.614+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/167942000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814dea3d0 0x55c814e19da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.614+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/167942000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 0x55c814de93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.614+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/167942000 >> 172.21.15.112:0/167942000 conn(0x55c814d4ac30 msgr2=0x55c814e18d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:34.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.614+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/167942000 shutdown_connections 2023-08-27T11:27:34.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.614+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/167942000 wait complete. 2023-08-27T11:27:34.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.615+0000 7fa5e8a94b80 1 -- start start 2023-08-27T11:27:34.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.615+0000 7fa5e8a94b80 1 Processor -- start 2023-08-27T11:27:34.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5e8a94b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814dea3d0 0x55c814e32350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5e8a94b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 0x55c814e2b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5e8a94b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 0x55c814e4a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5e8a94b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c814e31e00 con 0x55c814e28960 2023-08-27T11:27:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5e8a94b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c814e27f80 con 0x55c814dea3d0 2023-08-27T11:27:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5e8a94b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c814e1ad00 con 0x55c814e16f40 2023-08-27T11:27:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5e8a94b80 10 client.0 _ll_register_callbacks cb 0x55c814ddf5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:27:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5d6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 0x55c814e4a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5d6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 0x55c814e4a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35088/0 (socket says 172.21.15.112:35088) 2023-08-27T11:27:34.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5d6d9d700 1 -- 172.21.15.112:0/3240268562 learned_addr learned my addr 172.21.15.112:0/3240268562 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:34.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5d5d9b700 1 --2- 172.21.15.112:0/3240268562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 0x55c814e2b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5d659c700 1 --2- 172.21.15.112:0/3240268562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814dea3d0 0x55c814e32350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5d6d9d700 1 -- 172.21.15.112:0/3240268562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 msgr2=0x55c814e2b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:34.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5d6d9d700 1 --2- 172.21.15.112:0/3240268562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 0x55c814e2b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5d6d9d700 1 -- 172.21.15.112:0/3240268562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814dea3d0 msgr2=0x55c814e32350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:34.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5d6d9d700 1 --2- 172.21.15.112:0/3240268562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814dea3d0 0x55c814e32350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5d6d9d700 1 -- 172.21.15.112:0/3240268562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d001c040 con 0x55c814e28960 2023-08-27T11:27:34.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5d6d9d700 1 --2- 172.21.15.112:0/3240268562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 0x55c814e4a9a0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0024040 tx=0x7fa5d0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0c032da11b8ed9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:34.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:27:34.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5b57fa700 1 -- 172.21.15.112:0/3240268562 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0003b90 con 0x55c814e28960 2023-08-27T11:27:34.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5b57fa700 1 -- 172.21.15.112:0/3240268562 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa5d0003cf0 con 0x55c814e28960 2023-08-27T11:27:34.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.616+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/3240268562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5d0013070 con 0x55c814e28960 2023-08-27T11:27:34.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5b57fa700 1 -- 172.21.15.112:0/3240268562 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5d0004030 con 0x55c814e28960 2023-08-27T11:27:34.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5e8a94b80 10 client.8118 fetch_fsmap learned FSMap version 76 2023-08-27T11:27:34.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/3240268562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55c814b53c10 con 0x55c814e28960 2023-08-27T11:27:34.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5b57fa700 1 -- 172.21.15.112:0/3240268562 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d0013070 con 0x55c814e28960 2023-08-27T11:27:34.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5b57fa700 1 -- 172.21.15.112:0/3240268562 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa5d0004190 con 0x55c814e28960 2023-08-27T11:27:34.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5e8a94b80 10 client.8118 fetch_fsmap finished waiting for FSMap version 76 2023-08-27T11:27:34.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5e8a94b80 10 client.8118 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-27T11:27:34.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5e8a94b80 20 client.8118 populate_metadata read hostname 'smithi112' 2023-08-27T11:27:34.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/3240268562 >> [v2:172.21.15.138:6835/3245198376,v1:172.21.15.138:6837/3245198376] conn(0x55c814e4b080 0x55c814e986c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5e8a94b80 4 client.8118 mds_command: new command op to 7989 tid=0[{"prefix": "status"}] 2023-08-27T11:27:34.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/3240268562 --> [v2:172.21.15.138:6835/3245198376,v1:172.21.15.138:6837/3245198376] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c814cf8000 con 0x55c814e4b080 2023-08-27T11:27:34.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5d659c700 1 --2- 172.21.15.112:0/3240268562 >> [v2:172.21.15.138:6835/3245198376,v1:172.21.15.138:6837/3245198376] conn(0x55c814e4b080 0x55c814e986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5b57fa700 10 client.8118 ms_handle_connect on v2:172.21.15.138:6835/3245198376 2023-08-27T11:27:34.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5d659c700 1 --2- 172.21.15.112:0/3240268562 >> [v2:172.21.15.138:6835/3245198376,v1:172.21.15.138:6837/3245198376] conn(0x55c814e4b080 0x55c814e986c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e673ff7ff94a1986 server_cookie=dab12c4060df3cf5 in_seq=0 out_seq=0 2023-08-27T11:27:34.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5b57fa700 1 -- 172.21.15.112:0/3240268562 <== mds.0 v2:172.21.15.138:6835/3245198376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55c814cf8000 con 0x55c814e4b080 2023-08-27T11:27:34.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.617+0000 7fa5b57fa700 10 client.8118 handle_command_reply: tid=0 2023-08-27T11:27:34.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5e8a94b80 1 client.8118 shutdown 2023-08-27T11:27:34.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/3240268562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 msgr2=0x55c814e4a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/3240268562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 0x55c814e4a9a0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0024040 tx=0x7fa5d0004b40 comp rx=0 tx=0).stop 2023-08-27T11:27:34.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/3240268562 shutdown_connections 2023-08-27T11:27:34.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/3240268562 >> [v2:172.21.15.138:6835/3245198376,v1:172.21.15.138:6837/3245198376] conn(0x55c814e4b080 0x55c814e986c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/3240268562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c814e28960 0x55c814e4a9a0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5b57fa700 0 client.8118 ms_handle_reset on v2:172.21.15.138:6835/3245198376 2023-08-27T11:27:34.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/3240268562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c814dea3d0 0x55c814e32350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5e8a94b80 1 --2- 172.21.15.112:0/3240268562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c814e16f40 0x55c814e2b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/3240268562 >> 172.21.15.112:0/3240268562 conn(0x55c814d4ac30 msgr2=0x55c814dea7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:34.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/3240268562 shutdown_connections 2023-08-27T11:27:34.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5e8a94b80 1 -- 172.21.15.112:0/3240268562 wait complete. 2023-08-27T11:27:34.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5e8a94b80 20 client.8118 trim_cache size 0 max 16384 2023-08-27T11:27:34.641 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:27:34.641 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e099a24f-c1cd-4f11-ac4f-ecacc10e5e98", 2023-08-27T11:27:34.641 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2023-08-27T11:27:34.641 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 7989, 2023-08-27T11:27:34.641 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2023-08-27T11:27:34.642 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2023-08-27T11:27:34.642 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2023-08-27T11:27:34.642 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 5.8915758589999996, 2023-08-27T11:27:34.642 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 76, 2023-08-27T11:27:34.642 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 114, 2023-08-27T11:27:34.643 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 114, 2023-08-27T11:27:34.643 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 11.843994378 2023-08-27T11:27:34.643 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:27:34.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5dc8d2700 1 -- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa5b806a4d0 msgr2=0x7fa5b806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5dc8d2700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa5b806a4d0 0x7fa5b806c990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009870 tx=0x7fa5c8008040 comp rx=0 tx=0).stop 2023-08-27T11:27:34.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5dc8d2700 1 -- 172.21.15.112:0/129994421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5d806fea0 msgr2=0x7fa5d80c56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5dc8d2700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5d806fea0 0x7fa5d80c56a0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00b0e0 tx=0x7fa5cc004750 comp rx=0 tx=0).stop 2023-08-27T11:27:34.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5dc8d2700 1 -- 172.21.15.112:0/129994421 shutdown_connections 2023-08-27T11:27:34.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5dc8d2700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa5b806a4d0 0x7fa5b806c990 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5dc8d2700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d8158e20 0x7fa5d80c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5dc8d2700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5d806e3a0 0x7fa5d80c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5dc8d2700 1 --2- 172.21.15.112:0/129994421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5d806fea0 0x7fa5d80c56a0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5dc8d2700 1 -- 172.21.15.112:0/129994421 >> 172.21.15.112:0/129994421 conn(0x7fa5d80d0000 msgr2=0x7fa5d806ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:34.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5dc8d2700 1 -- 172.21.15.112:0/129994421 shutdown_connections 2023-08-27T11:27:34.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.618+0000 7fa5dc8d2700 1 -- 172.21.15.112:0/129994421 wait complete. 2023-08-27T11:27:34.647 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:27:34.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.766+0000 7f59c4468700 1 Processor -- start 2023-08-27T11:27:34.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.766+0000 7f59c4468700 1 -- start start 2023-08-27T11:27:34.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.767+0000 7f59c4468700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f59bc144570 0x7f59bc15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.767+0000 7f59c4468700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59bc0b3e50 0x7f59bc15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.767+0000 7f59c4468700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f59bc15f8a0 0x7f59bc0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.767+0000 7f59c4468700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59bc144950 con 0x7f59bc15f8a0 2023-08-27T11:27:34.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.767+0000 7f59c4468700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59bc0b45f0 con 0x7f59bc144570 2023-08-27T11:27:34.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.767+0000 7f59c4468700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0b4310 con 0x7f59bc0b3e50 2023-08-27T11:27:34.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.767+0000 7f59c2a05700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f59bc15f8a0 0x7f59bc0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.767+0000 7f59c2a05700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f59bc15f8a0 0x7f59bc0ce790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35096/0 (socket says 172.21.15.112:35096) 2023-08-27T11:27:34.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.767+0000 7f59c2a05700 1 -- 172.21.15.112:0/3510972322 learned_addr learned my addr 172.21.15.112:0/3510972322 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:34.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.768+0000 7f59c1a03700 1 --2- 172.21.15.112:0/3510972322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59bc0b3e50 0x7f59bc15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.768+0000 7f59c2204700 1 --2- 172.21.15.112:0/3510972322 >> v2:172.21.15.138:3300/0 conn(0x7f59bc144570 0x7f59bc15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.768+0000 7f59c2a05700 1 -- 172.21.15.112:0/3510972322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59bc0b3e50 msgr2=0x7f59bc15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:34.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.768+0000 7f59c2a05700 1 --2- 172.21.15.112:0/3510972322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59bc0b3e50 0x7f59bc15f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.768+0000 7f59c2a05700 1 -- 172.21.15.112:0/3510972322 >> v2:172.21.15.138:3300/0 conn(0x7f59bc144570 msgr2=0x7f59bc15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:34.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.768+0000 7f59c2a05700 1 --2- 172.21.15.112:0/3510972322 >> v2:172.21.15.138:3300/0 conn(0x7f59bc144570 0x7f59bc15ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.768+0000 7f59c2a05700 1 -- 172.21.15.112:0/3510972322 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc0cdcb0 con 0x7f59bc15f8a0 2023-08-27T11:27:34.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.768+0000 7f59c2a05700 1 --2- 172.21.15.112:0/3510972322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59bc15f8a0 0x7f59bc0ce790 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f59bc06b7b0 tx=0x7f59b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451e0e0bdbe13f51 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:34.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.768+0000 7f59c1202700 1 -- 172.21.15.112:0/3510972322 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b400e070 con 0x7f59bc15f8a0 2023-08-27T11:27:34.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.768+0000 7f59c1202700 1 -- 172.21.15.112:0/3510972322 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b400b9c0 con 0x7f59bc15f8a0 2023-08-27T11:27:34.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.768+0000 7f59c1202700 1 -- 172.21.15.112:0/3510972322 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b400bd70 con 0x7f59bc15f8a0 2023-08-27T11:27:34.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.769+0000 7f59c4468700 1 -- 172.21.15.112:0/3510972322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59bc15f8a0 msgr2=0x7f59bc0ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.769+0000 7f59c4468700 1 --2- 172.21.15.112:0/3510972322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59bc15f8a0 0x7f59bc0ce790 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f59bc06b7b0 tx=0x7f59b400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:34.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.769+0000 7f59c4468700 1 -- 172.21.15.112:0/3510972322 shutdown_connections 2023-08-27T11:27:34.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.769+0000 7f59c4468700 1 --2- 172.21.15.112:0/3510972322 >> v2:172.21.15.138:3300/0 conn(0x7f59bc144570 0x7f59bc15ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.769+0000 7f59c4468700 1 --2- 172.21.15.112:0/3510972322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59bc15f8a0 0x7f59bc0ce790 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.769+0000 7f59c4468700 1 --2- 172.21.15.112:0/3510972322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59bc0b3e50 0x7f59bc15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.769+0000 7f59c4468700 1 -- 172.21.15.112:0/3510972322 >> 172.21.15.112:0/3510972322 conn(0x7f59bc06f970 msgr2=0x7f59bc06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:34.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.769+0000 7f59c4468700 1 -- 172.21.15.112:0/3510972322 shutdown_connections 2023-08-27T11:27:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.769+0000 7f59c4468700 1 -- 172.21.15.112:0/3510972322 wait complete. 2023-08-27T11:27:34.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.770+0000 7f59c4468700 1 Processor -- start 2023-08-27T11:27:34.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.770+0000 7f59c4468700 1 -- start start 2023-08-27T11:27:34.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c4468700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59bc0b3e50 0x7f59bc14e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c4468700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59bc144570 0x7f59bc14a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c4468700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59bc15f8a0 0x7f59bc14a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c4468700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc149f50 con 0x7f59bc0b3e50 2023-08-27T11:27:34.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c4468700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc149340 con 0x7f59bc144570 2023-08-27T11:27:34.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c4468700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc149620 con 0x7f59bc15f8a0 2023-08-27T11:27:34.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c2204700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59bc0b3e50 0x7f59bc14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c2204700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59bc0b3e50 0x7f59bc14e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35098/0 (socket says 172.21.15.112:35098) 2023-08-27T11:27:34.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c2204700 1 -- 172.21.15.112:0/2729441852 learned_addr learned my addr 172.21.15.112:0/2729441852 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:34.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c1a03700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59bc144570 0x7f59bc14a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c2a05700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59bc15f8a0 0x7f59bc14a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c2204700 1 -- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59bc15f8a0 msgr2=0x7f59bc14a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c2204700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59bc15f8a0 0x7f59bc14a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c2204700 1 -- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59bc144570 msgr2=0x7f59bc14a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:34.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c2204700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59bc144570 0x7f59bc14a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:34.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c2204700 1 -- 172.21.15.112:0/2729441852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b4009b30 con 0x7f59bc0b3e50 2023-08-27T11:27:34.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59c2204700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59bc0b3e50 0x7f59bc14e210 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f59b800e810 tx=0x7f59b800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:34.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.771+0000 7f59b2ffd700 1 -- 172.21.15.112:0/2729441852 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8013610 con 0x7f59bc0b3e50 2023-08-27T11:27:34.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.772+0000 7f59b2ffd700 1 -- 172.21.15.112:0/2729441852 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b8013770 con 0x7f59bc0b3e50 2023-08-27T11:27:34.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.772+0000 7f59c4468700 1 -- 172.21.15.112:0/2729441852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc149900 con 0x7f59bc0b3e50 2023-08-27T11:27:34.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.772+0000 7f59b2ffd700 1 -- 172.21.15.112:0/2729441852 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8013b20 con 0x7f59bc0b3e50 2023-08-27T11:27:34.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.772+0000 7f59c4468700 1 -- 172.21.15.112:0/2729441852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc14e750 con 0x7f59bc0b3e50 2023-08-27T11:27:34.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.773+0000 7f59c4468700 1 -- 172.21.15.112:0/2729441852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc067a20 con 0x7f59bc0b3e50 2023-08-27T11:27:34.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.773+0000 7f59b2ffd700 1 -- 172.21.15.112:0/2729441852 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f59b8013c80 con 0x7f59bc0b3e50 2023-08-27T11:27:34.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.773+0000 7f59b2ffd700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59a006a4d0 0x7f59a006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:34.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.774+0000 7f59b2ffd700 1 -- 172.21.15.112:0/2729441852 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f59b8088b20 con 0x7f59bc0b3e50 2023-08-27T11:27:34.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.774+0000 7f59c1a03700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59a006a4d0 0x7f59a006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:34.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.774+0000 7f59c1a03700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59a006a4d0 0x7f59a006c990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00c900 tx=0x7f59ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:34.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:34.778+0000 7f59b2ffd700 1 -- 172.21.15.112:0/2729441852 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f59b8053b30 con 0x7f59bc0b3e50 2023-08-27T11:27:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.008+0000 7f59c4468700 1 -- 172.21.15.112:0/2729441852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f59bc1593f0 con 0x7f59bc0b3e50 2023-08-27T11:27:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.009+0000 7f59b2ffd700 1 -- 172.21.15.112:0/2729441852 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) v1 ==== 53+0+73 (secure 0 0 0) 0x7f59b8057150 con 0x7f59bc0b3e50 2023-08-27T11:27:35.011 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:27:35.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.011+0000 7f59c4468700 1 -- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59a006a4d0 msgr2=0x7f59a006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.011+0000 7f59c4468700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59a006a4d0 0x7f59a006c990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00c900 tx=0x7f59ac008040 comp rx=0 tx=0).stop 2023-08-27T11:27:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.011+0000 7f59c4468700 1 -- 172.21.15.112:0/2729441852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59bc0b3e50 msgr2=0x7f59bc14e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.011+0000 7f59c4468700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59bc0b3e50 0x7f59bc14e210 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f59b800e810 tx=0x7f59b800ebd0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.011+0000 7f59c4468700 1 -- 172.21.15.112:0/2729441852 shutdown_connections 2023-08-27T11:27:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.011+0000 7f59c4468700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59a006a4d0 0x7f59a006c990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.011+0000 7f59c4468700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59bc15f8a0 0x7f59bc14a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.011+0000 7f59c4468700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59bc0b3e50 0x7f59bc14e210 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.011+0000 7f59c4468700 1 --2- 172.21.15.112:0/2729441852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59bc144570 0x7f59bc14a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.011+0000 7f59c4468700 1 -- 172.21.15.112:0/2729441852 >> 172.21.15.112:0/2729441852 conn(0x7f59bc06f970 msgr2=0x7f59bc141b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:35.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.011+0000 7f59c4468700 1 -- 172.21.15.112:0/2729441852 shutdown_connections 2023-08-27T11:27:35.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.011+0000 7f59c4468700 1 -- 172.21.15.112:0/2729441852 wait complete. 2023-08-27T11:27:35.024 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:27:35.025 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:27:35.025 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-08-27T11:27:35.026 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:27:35.026 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:35.026 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:27:35.026 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:27:35.027 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:27:35.027 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:27:35.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:35.028 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-08-27T11:27:35.045 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:27:35.045 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:27:35.046 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:27:35.046 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:35.046 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:27:35.046 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:35.047 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:35.047 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:35.047 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:35.047 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:35.047 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:35.048 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:35.048 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:27:35.048 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:35.048 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:27:35.048 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 41787sec preferred_lft 41787sec 2023-08-27T11:27:35.049 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8d6c:4fe6:6a84:8222/64 scope link noprefixroute 2023-08-27T11:27:35.049 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:35.049 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:27:35.049 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:35.049 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:27:35.050 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:35.050 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::d87c:a1ff:fe1d:b635/64 scope link 2023-08-27T11:27:35.050 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:35.050 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:27:35.050 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:27:35.051 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::cc14:79ff:feb3:452f/64 scope link 2023-08-27T11:27:35.051 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:35.051 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:27:35.051 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether e6:51:f0:78:04:02 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:27:35.051 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::e451:f0ff:fe78:402/64 scope link 2023-08-27T11:27:35.052 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:35.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:35.052 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:27:35.110 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-08-27T11:27:35.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:35.112 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-08-27T11:27:35.170 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-08-27T11:27:35.171 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:27:35.171 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:27:35.171 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:27:35.171 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:27:35.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:35.172 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:27:35.173 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:27:35.173 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:27:35.173 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:27:35.174 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:27:35.174 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:27:35.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:35.277 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:27:35.294 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:27:35.294 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:27:35.295 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:27:35.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:35.295 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-08-27T11:27:35.361 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:27:35.361 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:35.418 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:27:35.418 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:27:35.419 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:35.474 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:27:35.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:35.475 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:27:35.560 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:27:35.560 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:27:35.560 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:27:35.568 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:27:35.569 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:27:35.569 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:27:35.569 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:27:35.569 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:27:35.636 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:27:35.636 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:27:35.636 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:27:35.637 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:27:35.637 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:27:35.637 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:27:35.637 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:27:35.637 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:27:35.639 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:27:35.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:35.640 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:35.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:35.667 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:27:35.756 INFO:teuthology.orchestra.run.smithi112.stdout:client7303 2023-08-27T11:27:35.757 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:27:35.757 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:35.774 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:27:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.933+0000 7fbc7f7ed700 1 Processor -- start 2023-08-27T11:27:35.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.933+0000 7fbc7f7ed700 1 -- start start 2023-08-27T11:27:35.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc7f7ed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc78163f90 0x7fbc78166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc7f7ed700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc780b5ae0 con 0x7fbc78144570 2023-08-27T11:27:35.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc7f7ed700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc780b45f0 con 0x7fbc780b3e50 2023-08-27T11:27:35.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc7f7ed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc780cf7a0 con 0x7fbc78163f90 2023-08-27T11:27:35.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc7d589700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbc78144570 0x7fbc7815ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57724/0 (socket says 172.21.15.112:57724) 2023-08-27T11:27:35.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc7d589700 1 -- 172.21.15.112:0/2822701123 learned_addr learned my addr 172.21.15.112:0/2822701123 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:35.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234095608 0 0) 0x7fbc74001110 con 0x7fbc78144570 2023-08-27T11:27:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc7dd8a700 1 --2- 172.21.15.112:0/2822701123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc78163f90 0x7fbc78166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc640035b0 con 0x7fbc78144570 2023-08-27T11:27:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1150741688 0 0) 0x7fbc640035b0 con 0x7fbc78144570 2023-08-27T11:27:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc78163f90 msgr2=0x7fbc78166380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 --2- 172.21.15.112:0/2822701123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc78163f90 0x7fbc78166380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 >> v1:172.21.15.138:6789/0 conn(0x7fbc780b3e50 legacy=0x7fbc7815f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7815fc40 con 0x7fbc78144570 2023-08-27T11:27:35.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fbc74001800 con 0x7fbc78144570 2023-08-27T11:27:35.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 >> v1:172.21.15.112:6789/0 conn(0x7fbc78144570 legacy=0x7fbc7815ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 --2- 172.21.15.112:0/2822701123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc78163f90 0x7fbc64003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 --2- 172.21.15.112:0/2822701123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc64003fb0 0x7fbc64004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 --2- 172.21.15.112:0/2822701123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64004d90 0x7fbc64005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc780cf7a0 con 0x7fbc64003fb0 2023-08-27T11:27:35.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6400bba0 con 0x7fbc64004d90 2023-08-27T11:27:35.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6400bd60 con 0x7fbc78163f90 2023-08-27T11:27:35.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.934+0000 7fbc7cd88700 1 --2- 172.21.15.112:0/2822701123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc64003fb0 0x7fbc64004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc7dd8a700 1 --2- 172.21.15.112:0/2822701123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64004d90 0x7fbc64005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc7d589700 1 --2- 172.21.15.112:0/2822701123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc78163f90 0x7fbc64003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc7cd88700 1 -- 172.21.15.112:0/2822701123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc78163f90 msgr2=0x7fbc64003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:35.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc7cd88700 1 --2- 172.21.15.112:0/2822701123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc78163f90 0x7fbc64003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc7cd88700 1 -- 172.21.15.112:0/2822701123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64004d90 msgr2=0x7fbc64005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:35.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc7cd88700 1 --2- 172.21.15.112:0/2822701123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64004d90 0x7fbc64005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc7cd88700 1 -- 172.21.15.112:0/2822701123 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640035b0 con 0x7fbc64003fb0 2023-08-27T11:27:35.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc7d589700 1 --2- 172.21.15.112:0/2822701123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc78163f90 0x7fbc64003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:35.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc7cd88700 1 --2- 172.21.15.112:0/2822701123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc64003fb0 0x7fbc64004850 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fbc680017c0 tx=0x7fbc64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a167c4d1c1b8cbcf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:35.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc680052d0 con 0x7fbc64003fb0 2023-08-27T11:27:35.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc68005430 con 0x7fbc64003fb0 2023-08-27T11:27:35.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc6ffff700 1 -- 172.21.15.112:0/2822701123 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc680038c0 con 0x7fbc64003fb0 2023-08-27T11:27:35.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2822701123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc64003fb0 msgr2=0x7fbc64004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.935+0000 7fbc7f7ed700 1 --2- 172.21.15.112:0/2822701123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc64003fb0 0x7fbc64004850 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fbc680017c0 tx=0x7fbc64005880 comp rx=0 tx=0).stop 2023-08-27T11:27:35.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.936+0000 7fbc7dd8a700 1 -- 172.21.15.112:0/2822701123 reap_dead start 2023-08-27T11:27:35.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.936+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2822701123 shutdown_connections 2023-08-27T11:27:35.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.936+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2822701123 >> 172.21.15.112:0/2822701123 conn(0x7fbc7806f970 msgr2=0x7fbc7806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:35.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.936+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2822701123 shutdown_connections 2023-08-27T11:27:35.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.936+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2822701123 wait complete. 2023-08-27T11:27:35.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.937+0000 7fbc7f7ed700 1 Processor -- start 2023-08-27T11:27:35.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.937+0000 7fbc7f7ed700 1 -- start start 2023-08-27T11:27:35.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7f7ed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64003fb0 0x7fbc78155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7f7ed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc781408e0 0x7fbc7815d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7f7ed700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc7813a380 0x7fbc7813a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7f7ed700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7813ad40 con 0x7fbc781408e0 2023-08-27T11:27:35.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7f7ed700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc78140450 con 0x7fbc64003fb0 2023-08-27T11:27:35.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7f7ed700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7813f840 con 0x7fbc7813a380 2023-08-27T11:27:35.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7cd88700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc781408e0 0x7fbc7815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7cd88700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc781408e0 0x7fbc7815d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35124/0 (socket says 172.21.15.112:35124) 2023-08-27T11:27:35.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7cd88700 1 -- 172.21.15.112:0/2133495957 learned_addr learned my addr 172.21.15.112:0/2133495957 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:35.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7d589700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64003fb0 0x7fbc78155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7dd8a700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc7813a380 0x7fbc7813a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7cd88700 1 -- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc7813a380 msgr2=0x7fbc7813a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7cd88700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc7813a380 0x7fbc7813a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7cd88700 1 -- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64003fb0 msgr2=0x7fbc78155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7cd88700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64003fb0 0x7fbc78155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7cd88700 1 -- 172.21.15.112:0/2133495957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7813fb20 con 0x7fbc781408e0 2023-08-27T11:27:35.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7cd88700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc781408e0 0x7fbc7815d190 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fbc68006b10 tx=0x7fbc68001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc6dffb700 1 -- 172.21.15.112:0/2133495957 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68007240 con 0x7fbc781408e0 2023-08-27T11:27:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc6dffb700 1 -- 172.21.15.112:0/2133495957 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc680073a0 con 0x7fbc781408e0 2023-08-27T11:27:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2133495957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7813fe00 con 0x7fbc781408e0 2023-08-27T11:27:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.938+0000 7fbc6dffb700 1 -- 172.21.15.112:0/2133495957 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68001290 con 0x7fbc781408e0 2023-08-27T11:27:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.939+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2133495957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc78156370 con 0x7fbc781408e0 2023-08-27T11:27:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.939+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2133495957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc78067a20 con 0x7fbc781408e0 2023-08-27T11:27:35.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.940+0000 7fbc6dffb700 1 -- 172.21.15.112:0/2133495957 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fbc68003680 con 0x7fbc781408e0 2023-08-27T11:27:35.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.940+0000 7fbc6dffb700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbc5c06a4d0 0x7fbc5c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.940+0000 7fbc6dffb700 1 -- 172.21.15.112:0/2133495957 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbc68089a00 con 0x7fbc781408e0 2023-08-27T11:27:35.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.941+0000 7fbc7d589700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbc5c06a4d0 0x7fbc5c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.941+0000 7fbc7d589700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbc5c06a4d0 0x7fbc5c06c990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbc74000c00 tx=0x7fbc740012b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:35.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:35.945+0000 7fbc6dffb700 1 -- 172.21.15.112:0/2133495957 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbc68054c50 con 0x7fbc781408e0 2023-08-27T11:27:36.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.175+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2133495957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbc7813e700 con 0x7fbc781408e0 2023-08-27T11:27:36.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.175+0000 7fbc6dffb700 1 -- 172.21.15.112:0/2133495957 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbc68017060 con 0x7fbc781408e0 2023-08-27T11:27:36.177 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:27:36.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.177+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbc5c06a4d0 msgr2=0x7fbc5c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.177+0000 7fbc7f7ed700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbc5c06a4d0 0x7fbc5c06c990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbc74000c00 tx=0x7fbc740012b0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.177+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2133495957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc781408e0 msgr2=0x7fbc7815d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.177+0000 7fbc7f7ed700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc781408e0 0x7fbc7815d190 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fbc68006b10 tx=0x7fbc68001120 comp rx=0 tx=0).stop 2023-08-27T11:27:36.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.177+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2133495957 shutdown_connections 2023-08-27T11:27:36.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.177+0000 7fbc7f7ed700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbc5c06a4d0 0x7fbc5c06c990 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.177+0000 7fbc7f7ed700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc7813a380 0x7fbc7813a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.177+0000 7fbc7f7ed700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64003fb0 0x7fbc78155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.177+0000 7fbc7f7ed700 1 --2- 172.21.15.112:0/2133495957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc781408e0 0x7fbc7815d190 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.177+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2133495957 >> 172.21.15.112:0/2133495957 conn(0x7fbc7806f970 msgr2=0x7fbc7815aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.177+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2133495957 shutdown_connections 2023-08-27T11:27:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:36.177+0000 7fbc7f7ed700 1 -- 172.21.15.112:0/2133495957 wait complete. 2023-08-27T11:27:36.191 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:27:36.191 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:27:36.192 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2023-08-27T11:27:36.192 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-27T11:27:36.192 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:27:36.193 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:27:36.193 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:27:36.193 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:27:36.194 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:27:36.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:36.194 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2023-08-27T11:27:36.216 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:27:36.216 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:27:36.216 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:27:36.217 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:36.217 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:27:36.217 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:36.217 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:36.218 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:36.218 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:36.218 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:36.218 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:36.218 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:36.219 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:27:36.219 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:36.219 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:27:36.219 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 41802sec preferred_lft 41802sec 2023-08-27T11:27:36.219 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ac17:49c5:cb39:801a/64 scope link noprefixroute 2023-08-27T11:27:36.220 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:36.220 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:27:36.220 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:36.220 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:27:36.220 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:36.221 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c4b4:94ff:fe58:4e35/64 scope link 2023-08-27T11:27:36.221 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:36.221 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:27:36.221 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:27:36.221 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::5806:6aff:fe04:ef32/64 scope link 2023-08-27T11:27:36.222 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:36.222 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:27:36.222 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 8a:21:ba:15:69:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-27T11:27:36.222 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::8821:baff:fe15:6919/64 scope link 2023-08-27T11:27:36.222 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:36.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:36.223 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:27:36.287 INFO:teuthology.orchestra.run.smithi138.stdout:1 2023-08-27T11:27:36.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:36.290 DEBUG:teuthology.orchestra.run.smithi138:> route 2023-08-27T11:27:36.350 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2023-08-27T11:27:36.350 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:27:36.351 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:27:36.351 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:27:36.351 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:27:36.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:36.352 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:27:36.352 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:27:36.353 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:27:36.353 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:27:36.353 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:27:36.354 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:27:36.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:36.460 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:27:36.477 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-27T11:27:36.477 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-27T11:27:36.478 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:27:36.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:36.479 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2023-08-27T11:27:36.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:27:36.544 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-27T11:27:36.601 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:27:36.601 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:27:36.602 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:27:36.657 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:27:36.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:36.658 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:27:36.744 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:27:36.745 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:27:36.745 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:27:36.753 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:27:36.753 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:27:36.754 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:27:36.754 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:27:36.754 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:27:36.791 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:27:36.791 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:27:36.792 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:27:36.792 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:27:36.792 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:27:36.792 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:27:36.792 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:27:36.793 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:27:36.805 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:27:36.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:36.806 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:27:36.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:36.834 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:27:36.906 INFO:teuthology.orchestra.run.smithi138.stdout:client6557 2023-08-27T11:27:36.906 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:27:36.907 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:36.963 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-08-27T11:27:37.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.093+0000 7f98b402e700 1 Processor -- start 2023-08-27T11:27:37.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.094+0000 7f98b402e700 1 -- start start 2023-08-27T11:27:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.094+0000 7f98b402e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f98ac15b910 0x7f98ac0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.094+0000 7f98b402e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac0b6560 con 0x7f98ac06c6e0 2023-08-27T11:27:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.094+0000 7f98b402e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac0b6840 con 0x7f98ac144bd0 2023-08-27T11:27:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.094+0000 7f98b402e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ac0b6b20 con 0x7f98ac15b910 2023-08-27T11:27:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.094+0000 7f98b25cb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f98ac15b910 0x7f98ac0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.095+0000 7f98b15c9700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f98ac06c6e0 0x7f98ac0b57a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57744/0 (socket says 172.21.15.112:57744) 2023-08-27T11:27:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.095+0000 7f98b15c9700 1 -- 172.21.15.112:0/292860491 learned_addr learned my addr 172.21.15.112:0/292860491 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.095+0000 7f98b25cb700 1 -- 172.21.15.112:0/292860491 >> v1:172.21.15.138:6789/0 conn(0x7f98ac144bd0 legacy=0x7f98ac06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.095+0000 7f98b25cb700 1 -- 172.21.15.112:0/292860491 >> v1:172.21.15.112:6789/0 conn(0x7f98ac06c6e0 legacy=0x7f98ac0b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.095+0000 7f98b25cb700 1 -- 172.21.15.112:0/292860491 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac0cf050 con 0x7f98ac15b910 2023-08-27T11:27:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.095+0000 7f98b0dc8700 1 -- 172.21.15.112:0/292860491 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316372010 0 0) 0x7f98ac0b6560 con 0x7f98ac06c6e0 2023-08-27T11:27:37.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.095+0000 7f98b25cb700 1 --2- 172.21.15.112:0/292860491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98ac15b910 0x7f98ac0b5eb0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009d90 tx=0x7f98a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47618de18e09710 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:37.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.095+0000 7f98b0dc8700 1 -- 172.21.15.112:0/292860491 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a400e070 con 0x7f98ac15b910 2023-08-27T11:27:37.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.095+0000 7f98b0dc8700 1 -- 172.21.15.112:0/292860491 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a400bbd0 con 0x7f98ac15b910 2023-08-27T11:27:37.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.095+0000 7f98b0dc8700 1 -- 172.21.15.112:0/292860491 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4012430 con 0x7f98ac15b910 2023-08-27T11:27:37.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.096+0000 7f98b402e700 1 -- 172.21.15.112:0/292860491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98ac15b910 msgr2=0x7f98ac0b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:37.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.096+0000 7f98b402e700 1 --2- 172.21.15.112:0/292860491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98ac15b910 0x7f98ac0b5eb0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009d90 tx=0x7f98a400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.096+0000 7f98b402e700 1 -- 172.21.15.112:0/292860491 shutdown_connections 2023-08-27T11:27:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.096+0000 7f98b402e700 1 --2- 172.21.15.112:0/292860491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98ac15b910 0x7f98ac0b5eb0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.096+0000 7f98b402e700 1 -- 172.21.15.112:0/292860491 >> 172.21.15.112:0/292860491 conn(0x7f98ac066010 msgr2=0x7f98ac066420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.096+0000 7f98b402e700 1 -- 172.21.15.112:0/292860491 shutdown_connections 2023-08-27T11:27:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.096+0000 7f98b402e700 1 -- 172.21.15.112:0/292860491 wait complete. 2023-08-27T11:27:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.097+0000 7f98b402e700 1 Processor -- start 2023-08-27T11:27:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.097+0000 7f98b402e700 1 -- start start 2023-08-27T11:27:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b402e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac06c6e0 0x7f98ac143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b402e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98ac144bd0 0x7f98ac13e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b402e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ac15b910 0x7f98ac1427b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b402e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac143ec0 con 0x7f98ac144bd0 2023-08-27T11:27:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b402e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac1f4760 con 0x7f98ac15b910 2023-08-27T11:27:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b402e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac1f4a40 con 0x7f98ac06c6e0 2023-08-27T11:27:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b15c9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98ac144bd0 0x7f98ac13e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b15c9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98ac144bd0 0x7f98ac13e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35138/0 (socket says 172.21.15.112:35138) 2023-08-27T11:27:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b15c9700 1 -- 172.21.15.112:0/2371245881 learned_addr learned my addr 172.21.15.112:0/2371245881 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b1dca700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac06c6e0 0x7f98ac143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b25cb700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ac15b910 0x7f98ac1427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b15c9700 1 -- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac06c6e0 msgr2=0x7f98ac143980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b15c9700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac06c6e0 0x7f98ac143980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b15c9700 1 -- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ac15b910 msgr2=0x7f98ac1427b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b15c9700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ac15b910 0x7f98ac1427b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b15c9700 1 -- 172.21.15.112:0/2371245881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a4009a70 con 0x7f98ac144bd0 2023-08-27T11:27:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b15c9700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98ac144bd0 0x7f98ac13e470 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f989c00c520 tx=0x7f989c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98a27fc700 1 -- 172.21.15.112:0/2371245881 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c004c80 con 0x7f98ac144bd0 2023-08-27T11:27:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98a27fc700 1 -- 172.21.15.112:0/2371245881 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c004e00 con 0x7f98ac144bd0 2023-08-27T11:27:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b402e700 1 -- 172.21.15.112:0/2371245881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac1f4d20 con 0x7f98ac144bd0 2023-08-27T11:27:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98a27fc700 1 -- 172.21.15.112:0/2371245881 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c007310 con 0x7f98ac144bd0 2023-08-27T11:27:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.098+0000 7f98b402e700 1 -- 172.21.15.112:0/2371245881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac13e9b0 con 0x7f98ac144bd0 2023-08-27T11:27:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.099+0000 7f98b402e700 1 -- 172.21.15.112:0/2371245881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac1f0f00 con 0x7f98ac144bd0 2023-08-27T11:27:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.100+0000 7f98a27fc700 1 -- 172.21.15.112:0/2371245881 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f989c007470 con 0x7f98ac144bd0 2023-08-27T11:27:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.100+0000 7f98a27fc700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f989006a4d0 0x7f989006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.100+0000 7f98a27fc700 1 -- 172.21.15.112:0/2371245881 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f989c088760 con 0x7f98ac144bd0 2023-08-27T11:27:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.100+0000 7f98b1dca700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f989006a4d0 0x7f989006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.101+0000 7f98b1dca700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f989006a4d0 0x7f989006c990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f98ac13b870 tx=0x7f98a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:37.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.105+0000 7f98a27fc700 1 -- 172.21.15.112:0/2371245881 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f98ac1f0f00 con 0x7f98ac144bd0 2023-08-27T11:27:37.154 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:27:37.157 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:27:37.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:37.341+0000 7f98b402e700 1 -- 172.21.15.112:0/2371245881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f98ac1f0f00 con 0x7f98ac144bd0 2023-08-27T11:27:38.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.335+0000 7f98a27fc700 1 -- 172.21.15.112:0/2371245881 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v77) v1 ==== 136+0+0 (secure 0 0 0) 0x7f98ac1f0f00 con 0x7f98ac144bd0 2023-08-27T11:27:38.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.338+0000 7f98b402e700 1 -- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f989006a4d0 msgr2=0x7f989006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.338+0000 7f98b402e700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f989006a4d0 0x7f989006c990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f98ac13b870 tx=0x7f98a8009040 comp rx=0 tx=0).stop 2023-08-27T11:27:38.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.338+0000 7f98b402e700 1 -- 172.21.15.112:0/2371245881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98ac144bd0 msgr2=0x7f98ac13e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.338+0000 7f98b402e700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98ac144bd0 0x7f98ac13e470 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f989c00c520 tx=0x7f989c00c830 comp rx=0 tx=0).stop 2023-08-27T11:27:38.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.338+0000 7f98b402e700 1 -- 172.21.15.112:0/2371245881 shutdown_connections 2023-08-27T11:27:38.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.338+0000 7f98b402e700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f989006a4d0 0x7f989006c990 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.338+0000 7f98b402e700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ac15b910 0x7f98ac1427b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.338+0000 7f98b402e700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac06c6e0 0x7f98ac143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.338+0000 7f98b402e700 1 --2- 172.21.15.112:0/2371245881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98ac144bd0 0x7f98ac13e470 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.338+0000 7f98b402e700 1 -- 172.21.15.112:0/2371245881 >> 172.21.15.112:0/2371245881 conn(0x7f98ac066010 msgr2=0x7f98ac0b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:38.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.338+0000 7f98b402e700 1 -- 172.21.15.112:0/2371245881 shutdown_connections 2023-08-27T11:27:38.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.338+0000 7f98b402e700 1 -- 172.21.15.112:0/2371245881 wait complete. 2023-08-27T11:27:38.354 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:27:38.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.480+0000 7f4cf1bd2700 1 Processor -- start 2023-08-27T11:27:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.480+0000 7f4cf1bd2700 1 -- start start 2023-08-27T11:27:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.480+0000 7f4cf1bd2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4cec144570 0x7f4cec15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4cf1bd2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4cec0b3e50 0x7f4cec15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4cf1bd2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec15f8a0 0x7f4cec0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4cf1bd2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cec144950 con 0x7f4cec144570 2023-08-27T11:27:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4cf1bd2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cec0b45f0 con 0x7f4cec0b3e50 2023-08-27T11:27:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4cf1bd2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec0b4310 con 0x7f4cec15f8a0 2023-08-27T11:27:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4ceb7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4cec144570 0x7f4cec15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4ceb7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4cec144570 0x7f4cec15ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35144/0 (socket says 172.21.15.112:35144) 2023-08-27T11:27:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4ceb7fe700 1 -- 172.21.15.112:0/677636830 learned_addr learned my addr 172.21.15.112:0/677636830 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4ceaffd700 1 --2- 172.21.15.112:0/677636830 >> v2:172.21.15.138:3300/0 conn(0x7f4cec0b3e50 0x7f4cec15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4cebfff700 1 --2- 172.21.15.112:0/677636830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec15f8a0 0x7f4cec0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4ceb7fe700 1 -- 172.21.15.112:0/677636830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec15f8a0 msgr2=0x7f4cec0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4ceb7fe700 1 --2- 172.21.15.112:0/677636830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec15f8a0 0x7f4cec0ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4ceb7fe700 1 -- 172.21.15.112:0/677636830 >> v2:172.21.15.138:3300/0 conn(0x7f4cec0b3e50 msgr2=0x7f4cec15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4ceb7fe700 1 --2- 172.21.15.112:0/677636830 >> v2:172.21.15.138:3300/0 conn(0x7f4cec0b3e50 0x7f4cec15f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4ceb7fe700 1 -- 172.21.15.112:0/677636830 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec0cdcb0 con 0x7f4cec144570 2023-08-27T11:27:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4ceaffd700 1 --2- 172.21.15.112:0/677636830 >> v2:172.21.15.138:3300/0 conn(0x7f4cec0b3e50 0x7f4cec15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4ceb7fe700 1 --2- 172.21.15.112:0/677636830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cec144570 0x7f4cec15ee20 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0b33f0 tx=0x7f4cdc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9c938690c23e64 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4cea7fc700 1 -- 172.21.15.112:0/677636830 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc00b870 con 0x7f4cec144570 2023-08-27T11:27:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4cea7fc700 1 -- 172.21.15.112:0/677636830 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cdc00b9d0 con 0x7f4cec144570 2023-08-27T11:27:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.481+0000 7f4cea7fc700 1 -- 172.21.15.112:0/677636830 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc00bd60 con 0x7f4cec144570 2023-08-27T11:27:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.482+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/677636830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cec144570 msgr2=0x7f4cec15ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.482+0000 7f4cf1bd2700 1 --2- 172.21.15.112:0/677636830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cec144570 0x7f4cec15ee20 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0b33f0 tx=0x7f4cdc009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.482+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/677636830 shutdown_connections 2023-08-27T11:27:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.482+0000 7f4cf1bd2700 1 --2- 172.21.15.112:0/677636830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec15f8a0 0x7f4cec0ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.482+0000 7f4cf1bd2700 1 --2- 172.21.15.112:0/677636830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cec144570 0x7f4cec15ee20 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.482+0000 7f4cf1bd2700 1 --2- 172.21.15.112:0/677636830 >> v2:172.21.15.138:3300/0 conn(0x7f4cec0b3e50 0x7f4cec15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.482+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/677636830 >> 172.21.15.112:0/677636830 conn(0x7f4cec06f970 msgr2=0x7f4cec06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.482+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/677636830 shutdown_connections 2023-08-27T11:27:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.482+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/677636830 wait complete. 2023-08-27T11:27:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.483+0000 7f4cf1bd2700 1 Processor -- start 2023-08-27T11:27:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.483+0000 7f4cf1bd2700 1 -- start start 2023-08-27T11:27:38.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4cf1bd2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cec0b3e50 0x7f4cec14b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4cf1bd2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec144570 0x7f4cec1f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4cf1bd2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cec15f8a0 0x7f4cec13f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4cf1bd2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec1f45e0 con 0x7f4cec0b3e50 2023-08-27T11:27:38.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4cf1bd2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec14c830 con 0x7f4cec15f8a0 2023-08-27T11:27:38.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4cf1bd2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec149990 con 0x7f4cec144570 2023-08-27T11:27:38.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4ceb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cec0b3e50 0x7f4cec14b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4ceb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cec0b3e50 0x7f4cec14b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35150/0 (socket says 172.21.15.112:35150) 2023-08-27T11:27:38.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4ceb7fe700 1 -- 172.21.15.112:0/1042269057 learned_addr learned my addr 172.21.15.112:0/1042269057 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:38.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4cebfff700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cec15f8a0 0x7f4cec13f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4ceaffd700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec144570 0x7f4cec1f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4ceb7fe700 1 -- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec144570 msgr2=0x7f4cec1f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4ceb7fe700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec144570 0x7f4cec1f40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4ceb7fe700 1 -- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cec15f8a0 msgr2=0x7f4cec13f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4ceb7fe700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cec15f8a0 0x7f4cec13f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4ceb7fe700 1 -- 172.21.15.112:0/1042269057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc009550 con 0x7f4cec0b3e50 2023-08-27T11:27:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.484+0000 7f4cebfff700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cec15f8a0 0x7f4cec13f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.485+0000 7f4ceb7fe700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cec0b3e50 0x7f4cec14b8f0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc000c00 tx=0x7f4cdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.485+0000 7f4cd3fff700 1 -- 172.21.15.112:0/1042269057 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc004540 con 0x7f4cec0b3e50 2023-08-27T11:27:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.485+0000 7f4cd3fff700 1 -- 172.21.15.112:0/1042269057 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cdc0036a0 con 0x7f4cec0b3e50 2023-08-27T11:27:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.485+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/1042269057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cec14bf90 con 0x7f4cec0b3e50 2023-08-27T11:27:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.485+0000 7f4cd3fff700 1 -- 172.21.15.112:0/1042269057 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc003a50 con 0x7f4cec0b3e50 2023-08-27T11:27:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.485+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/1042269057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cec14c550 con 0x7f4cec0b3e50 2023-08-27T11:27:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.486+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/1042269057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cec067a20 con 0x7f4cec0b3e50 2023-08-27T11:27:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.486+0000 7f4cd3fff700 1 -- 172.21.15.112:0/1042269057 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4cdc021070 con 0x7f4cec0b3e50 2023-08-27T11:27:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.487+0000 7f4cd3fff700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4cd806a4d0 0x7f4cd806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.487+0000 7f4cd3fff700 1 -- 172.21.15.112:0/1042269057 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4cdc08a250 con 0x7f4cec0b3e50 2023-08-27T11:27:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.487+0000 7f4ceaffd700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4cd806a4d0 0x7f4cd806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.487+0000 7f4ceaffd700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4cd806a4d0 0x7f4cd806c990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4cec14ff70 tx=0x7f4ce0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.491+0000 7f4cd3fff700 1 -- 172.21.15.112:0/1042269057 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4cdc055500 con 0x7f4cec0b3e50 2023-08-27T11:27:38.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.705+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/1042269057 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cec0ce270 con 0x7f4cd806a4d0 2023-08-27T11:27:38.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.707+0000 7f4cd3fff700 1 -- 172.21.15.112:0/1042269057 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4ce0003d90 con 0x7f4cd806a4d0 2023-08-27T11:27:38.709 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:27:38.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:38.709 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs" 2023-08-27T11:27:38.709 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:38.710 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:27:38.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.709+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4cd806a4d0 msgr2=0x7f4cd806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.709+0000 7f4cf1bd2700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4cd806a4d0 0x7f4cd806c990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4cec14ff70 tx=0x7f4ce0008040 comp rx=0 tx=0).stop 2023-08-27T11:27:38.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.709+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/1042269057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cec0b3e50 msgr2=0x7f4cec14b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.709+0000 7f4cf1bd2700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cec0b3e50 0x7f4cec14b8f0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc000c00 tx=0x7f4cdc004750 comp rx=0 tx=0).stop 2023-08-27T11:27:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.709+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/1042269057 shutdown_connections 2023-08-27T11:27:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.709+0000 7f4cf1bd2700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4cd806a4d0 0x7f4cd806c990 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.709+0000 7f4cf1bd2700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cec15f8a0 0x7f4cec13f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.709+0000 7f4cf1bd2700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cec0b3e50 0x7f4cec14b8f0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.709+0000 7f4cf1bd2700 1 --2- 172.21.15.112:0/1042269057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec144570 0x7f4cec1f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.709+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/1042269057 >> 172.21.15.112:0/1042269057 conn(0x7f4cec06f970 msgr2=0x7f4cec15b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.709+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/1042269057 shutdown_connections 2023-08-27T11:27:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.709+0000 7f4cf1bd2700 1 -- 172.21.15.112:0/1042269057 wait complete. 2023-08-27T11:27:38.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:27:38.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.845+0000 7fda9e43a700 1 Processor -- start 2023-08-27T11:27:38.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.845+0000 7fda9e43a700 1 -- start start 2023-08-27T11:27:38.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.845+0000 7fda9e43a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fda980b4d00 0x7fda9806b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.845+0000 7fda9e43a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fda9806b880 0x7fda9815a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.845+0000 7fda9e43a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda9815b8d0 con 0x7fda9815ae00 2023-08-27T11:27:38.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.845+0000 7fda9e43a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9806bd30 con 0x7fda9806b880 2023-08-27T11:27:38.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.845+0000 7fda9e43a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9815b680 con 0x7fda980b4d00 2023-08-27T11:27:38.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.846+0000 7fda977fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fda9806b880 0x7fda9815a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.846+0000 7fda977fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fda9806b880 0x7fda9815a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35158/0 (socket says 172.21.15.112:35158) 2023-08-27T11:27:38.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.846+0000 7fda977fe700 1 -- 172.21.15.112:0/800956363 learned_addr learned my addr 172.21.15.112:0/800956363 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:38.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.846+0000 7fda977fe700 1 -- 172.21.15.112:0/800956363 >> v2:172.21.15.138:3300/0 conn(0x7fda980b4d00 msgr2=0x7fda9806b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:38.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.846+0000 7fda977fe700 1 --2- 172.21.15.112:0/800956363 >> v2:172.21.15.138:3300/0 conn(0x7fda980b4d00 0x7fda9806b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.846+0000 7fda977fe700 1 -- 172.21.15.112:0/800956363 >> v1:172.21.15.138:6789/0 conn(0x7fda9815ae00 legacy=0x7fda9806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:38.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.846+0000 7fda977fe700 1 -- 172.21.15.112:0/800956363 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9815ec10 con 0x7fda9806b880 2023-08-27T11:27:38.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.846+0000 7fda977fe700 1 --2- 172.21.15.112:0/800956363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9806b880 0x7fda9815a8c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fda9806c160 tx=0x7fda8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610adbf69bb4648a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:38.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.846+0000 7fda96ffd700 1 -- 172.21.15.112:0/800956363 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c004230 con 0x7fda9806b880 2023-08-27T11:27:38.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.846+0000 7fda96ffd700 1 -- 172.21.15.112:0/800956363 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8c004390 con 0x7fda9806b880 2023-08-27T11:27:38.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.846+0000 7fda96ffd700 1 -- 172.21.15.112:0/800956363 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c0036a0 con 0x7fda9806b880 2023-08-27T11:27:38.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.847+0000 7fda9e43a700 1 -- 172.21.15.112:0/800956363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9806b880 msgr2=0x7fda9815a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.847+0000 7fda9e43a700 1 --2- 172.21.15.112:0/800956363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9806b880 0x7fda9815a8c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fda9806c160 tx=0x7fda8c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.847+0000 7fda9e43a700 1 -- 172.21.15.112:0/800956363 shutdown_connections 2023-08-27T11:27:38.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.847+0000 7fda9e43a700 1 --2- 172.21.15.112:0/800956363 >> v2:172.21.15.138:3300/0 conn(0x7fda980b4d00 0x7fda9806b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.847+0000 7fda9e43a700 1 --2- 172.21.15.112:0/800956363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9806b880 0x7fda9815a8c0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.847+0000 7fda9e43a700 1 -- 172.21.15.112:0/800956363 >> 172.21.15.112:0/800956363 conn(0x7fda980d03e0 msgr2=0x7fda980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:38.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.847+0000 7fda9e43a700 1 -- 172.21.15.112:0/800956363 shutdown_connections 2023-08-27T11:27:38.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.847+0000 7fda9e43a700 1 -- 172.21.15.112:0/800956363 wait complete. 2023-08-27T11:27:38.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.848+0000 7fda9e43a700 1 Processor -- start 2023-08-27T11:27:38.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.848+0000 7fda9e43a700 1 -- start start 2023-08-27T11:27:38.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9e43a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9806b880 0x7fda98144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9e43a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda980b4d00 0x7fda98070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9e43a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9815ae00 0x7fda981f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9e43a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda981f3f60 con 0x7fda9815ae00 2023-08-27T11:27:38.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9e43a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda981f4520 con 0x7fda9806b880 2023-08-27T11:27:38.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9e43a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda980cb810 con 0x7fda980b4d00 2023-08-27T11:27:38.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9c9d7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9815ae00 0x7fda981f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9c9d7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9815ae00 0x7fda981f49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35166/0 (socket says 172.21.15.112:35166) 2023-08-27T11:27:38.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9c9d7700 1 -- 172.21.15.112:0/739737824 learned_addr learned my addr 172.21.15.112:0/739737824 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:38.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda97fff700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9806b880 0x7fda98144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda977fe700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda980b4d00 0x7fda98070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9c9d7700 1 -- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda980b4d00 msgr2=0x7fda98070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9c9d7700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda980b4d00 0x7fda98070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9c9d7700 1 -- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9806b880 msgr2=0x7fda98144b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9c9d7700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9806b880 0x7fda98144b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda9c9d7700 1 -- 172.21.15.112:0/739737824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c009550 con 0x7fda9815ae00 2023-08-27T11:27:38.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.849+0000 7fda97fff700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9806b880 0x7fda98144b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:38.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.850+0000 7fda9c9d7700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9815ae00 0x7fda981f49b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fda9000b990 tx=0x7fda9000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:38.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.850+0000 7fda94ff9700 1 -- 172.21.15.112:0/739737824 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9000c9d0 con 0x7fda9815ae00 2023-08-27T11:27:38.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.850+0000 7fda94ff9700 1 -- 172.21.15.112:0/739737824 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9000cb30 con 0x7fda9815ae00 2023-08-27T11:27:38.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.850+0000 7fda94ff9700 1 -- 172.21.15.112:0/739737824 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90014440 con 0x7fda9815ae00 2023-08-27T11:27:38.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.850+0000 7fda9e43a700 1 -- 172.21.15.112:0/739737824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda980cbaf0 con 0x7fda9815ae00 2023-08-27T11:27:38.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.850+0000 7fda9e43a700 1 -- 172.21.15.112:0/739737824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda980caf70 con 0x7fda9815ae00 2023-08-27T11:27:38.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.851+0000 7fda9e43a700 1 -- 172.21.15.112:0/739737824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9806e8f0 con 0x7fda9815ae00 2023-08-27T11:27:38.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.851+0000 7fda94ff9700 1 -- 172.21.15.112:0/739737824 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fda90006d50 con 0x7fda9815ae00 2023-08-27T11:27:38.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.851+0000 7fda94ff9700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fda8406a4d0 0x7fda8406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.852+0000 7fda94ff9700 1 -- 172.21.15.112:0/739737824 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fda90088760 con 0x7fda9815ae00 2023-08-27T11:27:38.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.852+0000 7fda97fff700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fda8406a4d0 0x7fda8406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.852+0000 7fda97fff700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fda8406a4d0 0x7fda8406c990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fda88007900 tx=0x7fda88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:38.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:38.856+0000 7fda94ff9700 1 -- 172.21.15.112:0/739737824 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fda90053900 con 0x7fda9815ae00 2023-08-27T11:27:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.047+0000 7fda9e43a700 1 -- 172.21.15.112:0/739737824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fda98067e90 con 0x7fda9815ae00 2023-08-27T11:27:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.047+0000 7fda94ff9700 1 -- 172.21.15.112:0/739737824 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fda90056f20 con 0x7fda9815ae00 2023-08-27T11:27:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.049+0000 7fda9e43a700 1 -- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fda8406a4d0 msgr2=0x7fda8406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.049+0000 7fda9e43a700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fda8406a4d0 0x7fda8406c990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fda88007900 tx=0x7fda88008040 comp rx=0 tx=0).stop 2023-08-27T11:27:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.049+0000 7fda9e43a700 1 -- 172.21.15.112:0/739737824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9815ae00 msgr2=0x7fda981f49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.049+0000 7fda9e43a700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9815ae00 0x7fda981f49b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fda9000b990 tx=0x7fda9000bd50 comp rx=0 tx=0).stop 2023-08-27T11:27:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.049+0000 7fda9e43a700 1 -- 172.21.15.112:0/739737824 shutdown_connections 2023-08-27T11:27:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.049+0000 7fda9e43a700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fda8406a4d0 0x7fda8406c990 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.049+0000 7fda9e43a700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9815ae00 0x7fda981f49b0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.049+0000 7fda9e43a700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9806b880 0x7fda98144b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.049+0000 7fda9e43a700 1 --2- 172.21.15.112:0/739737824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda980b4d00 0x7fda98070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.049+0000 7fda9e43a700 1 -- 172.21.15.112:0/739737824 >> 172.21.15.112:0/739737824 conn(0x7fda980d03e0 msgr2=0x7fda980d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:39.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.049+0000 7fda9e43a700 1 -- 172.21.15.112:0/739737824 shutdown_connections 2023-08-27T11:27:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.049+0000 7fda9e43a700 1 -- 172.21.15.112:0/739737824 wait complete. 2023-08-27T11:27:39.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:27:39.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.182+0000 7f87d4e7a700 1 Processor -- start 2023-08-27T11:27:39.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.182+0000 7f87d4e7a700 1 -- start start 2023-08-27T11:27:39.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.183+0000 7f87d4e7a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d0143a60 0x7f87d00cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.183+0000 7f87d4e7a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d0143e40 con 0x7f87d00b4960 2023-08-27T11:27:39.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.183+0000 7f87d4e7a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d00b5340 con 0x7f87d015e290 2023-08-27T11:27:39.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.183+0000 7f87d4e7a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00ce9a0 con 0x7f87d0143a60 2023-08-27T11:27:39.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cdd9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f87d00b4960 0x7f87d015db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57784/0 (socket says 172.21.15.112:57784) 2023-08-27T11:27:39.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cdd9b700 1 -- 172.21.15.112:0/1338956317 learned_addr learned my addr 172.21.15.112:0/1338956317 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:39.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87ce59c700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d0143a60 0x7f87d00cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144396794 0 0) 0x7f87d0143e40 con 0x7f87d00b4960 2023-08-27T11:27:39.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87b80035b0 con 0x7f87d00b4960 2023-08-27T11:27:39.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2850846035 0 0) 0x7f87b80035b0 con 0x7f87d00b4960 2023-08-27T11:27:39.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d0143a60 msgr2=0x7f87d00cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:39.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d0143a60 0x7f87d00cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 >> v1:172.21.15.138:6789/0 conn(0x7f87d015e290 legacy=0x7f87d015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d00cec80 con 0x7f87d00b4960 2023-08-27T11:27:39.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f87c00017f0 con 0x7f87d00b4960 2023-08-27T11:27:39.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 >> v1:172.21.15.112:6789/0 conn(0x7f87d00b4960 legacy=0x7f87d015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d0143a60 0x7f87b8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b8003f70 0x7f87b8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b8004d90 0x7f87b8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00ce9a0 con 0x7f87b8003f70 2023-08-27T11:27:39.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b800ba70 con 0x7f87b8004d90 2023-08-27T11:27:39.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b800de40 con 0x7f87d0143a60 2023-08-27T11:27:39.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cdd9b700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b8003f70 0x7f87b8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87ce59c700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d0143a60 0x7f87b8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87ced9d700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b8004d90 0x7f87b8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cdd9b700 1 -- 172.21.15.112:0/1338956317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d0143a60 msgr2=0x7f87b8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:39.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cdd9b700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d0143a60 0x7f87b8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cdd9b700 1 -- 172.21.15.112:0/1338956317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b8004d90 msgr2=0x7f87b8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cdd9b700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b8004d90 0x7f87b8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87cdd9b700 1 -- 172.21.15.112:0/1338956317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b80035b0 con 0x7f87b8003f70 2023-08-27T11:27:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87ce59c700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d0143a60 0x7f87b8003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.184+0000 7f87ced9d700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b8004d90 0x7f87b8005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:39.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.185+0000 7f87cdd9b700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b8003f70 0x7f87b8004850 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f87c000c240 tx=0x7f87b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70088720bfd860a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:39.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.185+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c000cd20 con 0x7f87b8003f70 2023-08-27T11:27:39.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.185+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c0007ba0 con 0x7f87b8003f70 2023-08-27T11:27:39.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.185+0000 7f87cd59a700 1 -- 172.21.15.112:0/1338956317 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c000cea0 con 0x7f87b8003f70 2023-08-27T11:27:39.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.185+0000 7f87d4e7a700 1 -- 172.21.15.112:0/1338956317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b8003f70 msgr2=0x7f87b8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.185+0000 7f87d4e7a700 1 --2- 172.21.15.112:0/1338956317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b8003f70 0x7f87b8004850 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f87c000c240 tx=0x7f87b8005880 comp rx=0 tx=0).stop 2023-08-27T11:27:39.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.185+0000 7f87ced9d700 1 -- 172.21.15.112:0/1338956317 reap_dead start 2023-08-27T11:27:39.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.185+0000 7f87d4e7a700 1 -- 172.21.15.112:0/1338956317 shutdown_connections 2023-08-27T11:27:39.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.185+0000 7f87d4e7a700 1 -- 172.21.15.112:0/1338956317 >> 172.21.15.112:0/1338956317 conn(0x7f87d006adb0 msgr2=0x7f87d006b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:39.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.185+0000 7f87d4e7a700 1 -- 172.21.15.112:0/1338956317 shutdown_connections 2023-08-27T11:27:39.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.185+0000 7f87d4e7a700 1 -- 172.21.15.112:0/1338956317 wait complete. 2023-08-27T11:27:39.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.187+0000 7f87d4e7a700 1 Processor -- start 2023-08-27T11:27:39.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.187+0000 7f87d4e7a700 1 -- start start 2023-08-27T11:27:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.187+0000 7f87d4e7a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b8003f70 0x7f87d00c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.187+0000 7f87d4e7a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d01f32b0 0x7f87d0159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.187+0000 7f87d4e7a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d013e630 0x7f87d013eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.187+0000 7f87d4e7a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d01f5c20 con 0x7f87d01f32b0 2023-08-27T11:27:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87d4e7a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00c5cb0 con 0x7f87b8003f70 2023-08-27T11:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87d4e7a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d013eff0 con 0x7f87d013e630 2023-08-27T11:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87cdd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d01f32b0 0x7f87d0159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87cdd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d01f32b0 0x7f87d0159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35182/0 (socket says 172.21.15.112:35182) 2023-08-27T11:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87cdd9b700 1 -- 172.21.15.112:0/2779471024 learned_addr learned my addr 172.21.15.112:0/2779471024 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87ce59c700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b8003f70 0x7f87d00c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87ced9d700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d013e630 0x7f87d013eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87cdd9b700 1 -- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d013e630 msgr2=0x7f87d013eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87cdd9b700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d013e630 0x7f87d013eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87cdd9b700 1 -- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b8003f70 msgr2=0x7f87d00c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87cdd9b700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b8003f70 0x7f87d00c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87cdd9b700 1 -- 172.21.15.112:0/2779471024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d00c59d0 con 0x7f87d01f32b0 2023-08-27T11:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87ce59c700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b8003f70 0x7f87d00c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87cdd9b700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d01f32b0 0x7f87d0159240 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f87c0000ef0 tx=0x7f87c0001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87beffd700 1 -- 172.21.15.112:0/2779471024 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c00074e0 con 0x7f87d01f32b0 2023-08-27T11:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87beffd700 1 -- 172.21.15.112:0/2779471024 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c0007640 con 0x7f87d01f32b0 2023-08-27T11:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87d4e7a700 1 -- 172.21.15.112:0/2779471024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d01f5e00 con 0x7f87d01f32b0 2023-08-27T11:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87beffd700 1 -- 172.21.15.112:0/2779471024 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0005a70 con 0x7f87d01f32b0 2023-08-27T11:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.188+0000 7f87d4e7a700 1 -- 172.21.15.112:0/2779471024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d01f63c0 con 0x7f87d01f32b0 2023-08-27T11:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.189+0000 7f87d4e7a700 1 -- 172.21.15.112:0/2779471024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d01f6550 con 0x7f87d01f32b0 2023-08-27T11:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.190+0000 7f87beffd700 1 -- 172.21.15.112:0/2779471024 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f87c0019030 con 0x7f87d01f32b0 2023-08-27T11:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.190+0000 7f87beffd700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87b006a4d0 0x7f87b006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.190+0000 7f87beffd700 1 -- 172.21.15.112:0/2779471024 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f87c008a320 con 0x7f87d01f32b0 2023-08-27T11:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.190+0000 7f87ce59c700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87b006a4d0 0x7f87b006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.191+0000 7f87ce59c700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87b006a4d0 0x7f87b006c990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009bd0 tx=0x7f87c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.195+0000 7f87beffd700 1 -- 172.21.15.112:0/2779471024 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f87c0055440 con 0x7f87d01f32b0 2023-08-27T11:27:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.412+0000 7f87d4e7a700 1 -- 172.21.15.112:0/2779471024 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f87d00cf240 con 0x7f87b006a4d0 2023-08-27T11:27:39.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.414+0000 7f87beffd700 1 -- 172.21.15.112:0/2779471024 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f87c4009820 con 0x7f87b006a4d0 2023-08-27T11:27:39.415 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:27:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs" 2023-08-27T11:27:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:39.416 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:27:39.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.416+0000 7f87d4e7a700 1 -- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87b006a4d0 msgr2=0x7f87b006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.416+0000 7f87d4e7a700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87b006a4d0 0x7f87b006c990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009bd0 tx=0x7f87c4008040 comp rx=0 tx=0).stop 2023-08-27T11:27:39.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.416+0000 7f87d4e7a700 1 -- 172.21.15.112:0/2779471024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d01f32b0 msgr2=0x7f87d0159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.416+0000 7f87d4e7a700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d01f32b0 0x7f87d0159240 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f87c0000ef0 tx=0x7f87c0001b80 comp rx=0 tx=0).stop 2023-08-27T11:27:39.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.416+0000 7f87d4e7a700 1 -- 172.21.15.112:0/2779471024 shutdown_connections 2023-08-27T11:27:39.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.416+0000 7f87d4e7a700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f87b006a4d0 0x7f87b006c990 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.416+0000 7f87d4e7a700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87d013e630 0x7f87d013eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.416+0000 7f87d4e7a700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b8003f70 0x7f87d00c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.416+0000 7f87d4e7a700 1 --2- 172.21.15.112:0/2779471024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d01f32b0 0x7f87d0159240 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.416+0000 7f87d4e7a700 1 -- 172.21.15.112:0/2779471024 >> 172.21.15.112:0/2779471024 conn(0x7f87d006adb0 msgr2=0x7f87d0151a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:39.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.416+0000 7f87d4e7a700 1 -- 172.21.15.112:0/2779471024 shutdown_connections 2023-08-27T11:27:39.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.416+0000 7f87d4e7a700 1 -- 172.21.15.112:0/2779471024 wait complete. 2023-08-27T11:27:39.429 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000519093 2023-08-27T11:27:39.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.550+0000 7f2746a34700 1 Processor -- start 2023-08-27T11:27:39.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.550+0000 7f2746a34700 1 -- start start 2023-08-27T11:27:39.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.551+0000 7f2746a34700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27400cd070 0x7f274015a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.551+0000 7f2746a34700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f274015ea00 0x7f274015aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.551+0000 7f2746a34700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274015b510 con 0x7f2740071980 2023-08-27T11:27:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.551+0000 7f2746a34700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274015e5e0 con 0x7f274015ea00 2023-08-27T11:27:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.551+0000 7f2746a34700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274015df20 con 0x7f27400cd070 2023-08-27T11:27:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.551+0000 7f2744fd1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f274015ea00 0x7f274015aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.551+0000 7f2744fd1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f274015ea00 0x7f274015aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35192/0 (socket says 172.21.15.112:35192) 2023-08-27T11:27:39.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.551+0000 7f2744fd1700 1 -- 172.21.15.112:0/2260475319 learned_addr learned my addr 172.21.15.112:0/2260475319 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:39.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f273f7fe700 1 --2- 172.21.15.112:0/2260475319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27400cd070 0x7f274015a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f2744fd1700 1 -- 172.21.15.112:0/2260475319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27400cd070 msgr2=0x7f274015a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:39.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f2744fd1700 1 --2- 172.21.15.112:0/2260475319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27400cd070 0x7f274015a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f2744fd1700 1 -- 172.21.15.112:0/2260475319 >> v1:172.21.15.138:6789/0 conn(0x7f2740071980 legacy=0x7f27400cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:39.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f2744fd1700 1 -- 172.21.15.112:0/2260475319 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274015e200 con 0x7f274015ea00 2023-08-27T11:27:39.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f2744fd1700 1 --2- 172.21.15.112:0/2260475319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f274015ea00 0x7f274015aca0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f2738009e50 tx=0x7f273800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879ff13e0cb4221 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:39.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f273effd700 1 -- 172.21.15.112:0/2260475319 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273800e070 con 0x7f274015ea00 2023-08-27T11:27:39.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f273effd700 1 -- 172.21.15.112:0/2260475319 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2738003a20 con 0x7f274015ea00 2023-08-27T11:27:39.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f273effd700 1 -- 172.21.15.112:0/2260475319 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738003dd0 con 0x7f274015ea00 2023-08-27T11:27:39.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f2746a34700 1 -- 172.21.15.112:0/2260475319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f274015ea00 msgr2=0x7f274015aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f2746a34700 1 --2- 172.21.15.112:0/2260475319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f274015ea00 0x7f274015aca0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f2738009e50 tx=0x7f273800b330 comp rx=0 tx=0).stop 2023-08-27T11:27:39.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f2746a34700 1 -- 172.21.15.112:0/2260475319 shutdown_connections 2023-08-27T11:27:39.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f2746a34700 1 --2- 172.21.15.112:0/2260475319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f274015ea00 0x7f274015aca0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.552+0000 7f2746a34700 1 --2- 172.21.15.112:0/2260475319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27400cd070 0x7f274015a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.553+0000 7f2746a34700 1 -- 172.21.15.112:0/2260475319 >> 172.21.15.112:0/2260475319 conn(0x7f274006bbf0 msgr2=0x7f27400cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:39.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.553+0000 7f2746a34700 1 -- 172.21.15.112:0/2260475319 shutdown_connections 2023-08-27T11:27:39.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.553+0000 7f2746a34700 1 -- 172.21.15.112:0/2260475319 wait complete. 2023-08-27T11:27:39.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.554+0000 7f2746a34700 1 Processor -- start 2023-08-27T11:27:39.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.554+0000 7f2746a34700 1 -- start start 2023-08-27T11:27:39.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.554+0000 7f2746a34700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2740071980 0x7f27401506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f2746a34700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27400cd070 0x7f27401398a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f2746a34700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f274015ea00 0x7f274014c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f2746a34700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27401510a0 con 0x7f2740071980 2023-08-27T11:27:39.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f2746a34700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2740139de0 con 0x7f27400cd070 2023-08-27T11:27:39.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f2746a34700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2740150dc0 con 0x7f274015ea00 2023-08-27T11:27:39.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f273ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2740071980 0x7f27401506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f273ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2740071980 0x7f27401506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35194/0 (socket says 172.21.15.112:35194) 2023-08-27T11:27:39.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f273ffff700 1 -- 172.21.15.112:0/843608208 learned_addr learned my addr 172.21.15.112:0/843608208 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:39.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f2744fd1700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f274015ea00 0x7f274014c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f273f7fe700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27400cd070 0x7f27401398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f273ffff700 1 -- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f274015ea00 msgr2=0x7f274014c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f273ffff700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f274015ea00 0x7f274014c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f273ffff700 1 -- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27400cd070 msgr2=0x7f27401398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f273ffff700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27400cd070 0x7f27401398a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f273ffff700 1 -- 172.21.15.112:0/843608208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2738009b30 con 0x7f2740071980 2023-08-27T11:27:39.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f273ffff700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2740071980 0x7f27401506f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f273400c4d0 tx=0x7f273400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:39.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f273cff9700 1 -- 172.21.15.112:0/843608208 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2734011790 con 0x7f2740071980 2023-08-27T11:27:39.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f273cff9700 1 -- 172.21.15.112:0/843608208 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27340118f0 con 0x7f2740071980 2023-08-27T11:27:39.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f2746a34700 1 -- 172.21.15.112:0/843608208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2740158550 con 0x7f2740071980 2023-08-27T11:27:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f273cff9700 1 -- 172.21.15.112:0/843608208 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2734011ca0 con 0x7f2740071980 2023-08-27T11:27:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.555+0000 7f2746a34700 1 -- 172.21.15.112:0/843608208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2740158b10 con 0x7f2740071980 2023-08-27T11:27:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.556+0000 7f2746a34700 1 -- 172.21.15.112:0/843608208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2740066fa0 con 0x7f2740071980 2023-08-27T11:27:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.557+0000 7f273cff9700 1 -- 172.21.15.112:0/843608208 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f273401c460 con 0x7f2740071980 2023-08-27T11:27:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.557+0000 7f273cff9700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f272c06a4d0 0x7f272c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.557+0000 7f273cff9700 1 -- 172.21.15.112:0/843608208 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f27340889b0 con 0x7f2740071980 2023-08-27T11:27:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.557+0000 7f273f7fe700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f272c06a4d0 0x7f272c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.558+0000 7f273f7fe700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f272c06a4d0 0x7f272c06c990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f27400af7f0 tx=0x7f2730008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.562+0000 7f273cff9700 1 -- 172.21.15.112:0/843608208 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2734053ac0 con 0x7f2740071980 2023-08-27T11:27:39.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:39.780+0000 7f2746a34700 1 -- 172.21.15.112:0/843608208 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000519093", "target": ["mon-mgr", ""]}) v1 -- 0x7f274015b1e0 con 0x7f272c06a4d0 2023-08-27T11:27:42.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.397+0000 7f273cff9700 1 -- 172.21.15.112:0/843608208 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f273000c040 con 0x7f272c06a4d0 2023-08-27T11:27:42.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.400+0000 7f2746a34700 1 -- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f272c06a4d0 msgr2=0x7f272c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:42.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.400+0000 7f2746a34700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f272c06a4d0 0x7f272c06c990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f27400af7f0 tx=0x7f2730008040 comp rx=0 tx=0).stop 2023-08-27T11:27:42.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.400+0000 7f2746a34700 1 -- 172.21.15.112:0/843608208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2740071980 msgr2=0x7f27401506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:42.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.400+0000 7f2746a34700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2740071980 0x7f27401506f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f273400c4d0 tx=0x7f273400c7e0 comp rx=0 tx=0).stop 2023-08-27T11:27:42.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.400+0000 7f2746a34700 1 -- 172.21.15.112:0/843608208 shutdown_connections 2023-08-27T11:27:42.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.400+0000 7f2746a34700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f272c06a4d0 0x7f272c06c990 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:42.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.400+0000 7f2746a34700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f274015ea00 0x7f274014c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:42.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.400+0000 7f2746a34700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2740071980 0x7f27401506f0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:42.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.400+0000 7f2746a34700 1 --2- 172.21.15.112:0/843608208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27400cd070 0x7f27401398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:42.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.400+0000 7f2746a34700 1 -- 172.21.15.112:0/843608208 >> 172.21.15.112:0/843608208 conn(0x7f274006bbf0 msgr2=0x7f27400cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:42.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.400+0000 7f2746a34700 1 -- 172.21.15.112:0/843608208 shutdown_connections 2023-08-27T11:27:42.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.400+0000 7f2746a34700 1 -- 172.21.15.112:0/843608208 wait complete. 2023-08-27T11:27:42.405 INFO:teuthology.orchestra.run.smithi112.stderr:Volume created successfully (no MDS daemons created) 2023-08-27T11:27:42.416 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000519094 2023-08-27T11:27:42.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.540+0000 7fcda120f700 1 Processor -- start 2023-08-27T11:27:42.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.540+0000 7fcda120f700 1 -- start start 2023-08-27T11:27:42.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.541+0000 7fcda120f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcd9c0cd070 0x7fcd9c15a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:42.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.541+0000 7fcda120f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9c15b510 con 0x7fcd9c071980 2023-08-27T11:27:42.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.541+0000 7fcda120f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9c15df20 con 0x7fcd9c15ea00 2023-08-27T11:27:42.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.541+0000 7fcda120f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9c15e5e0 con 0x7fcd9c0cd070 2023-08-27T11:27:42.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.541+0000 7fcd9a59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcd9c0cd070 0x7fcd9c15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:42.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.541+0000 7fcd9a59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcd9c0cd070 0x7fcd9c15a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35220/0 (socket says 172.21.15.112:35220) 2023-08-27T11:27:42.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.541+0000 7fcd9ad9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fcd9c071980 0x7fcd9c0cc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57832/0 (socket says 172.21.15.112:57832) 2023-08-27T11:27:42.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.541+0000 7fcd9a59c700 1 -- 172.21.15.112:0/3405757488 learned_addr learned my addr 172.21.15.112:0/3405757488 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:42.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.541+0000 7fcd9a59c700 1 -- 172.21.15.112:0/3405757488 >> v1:172.21.15.138:6789/0 conn(0x7fcd9c15ea00 legacy=0x7fcd9c15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:42.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.541+0000 7fcd9a59c700 1 -- 172.21.15.112:0/3405757488 >> v1:172.21.15.112:6789/0 conn(0x7fcd9c071980 legacy=0x7fcd9c0cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:42.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.541+0000 7fcd9a59c700 1 -- 172.21.15.112:0/3405757488 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c15e200 con 0x7fcd9c0cd070 2023-08-27T11:27:42.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.541+0000 7fcd9a59c700 1 --2- 172.21.15.112:0/3405757488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd9c0cd070 0x7fcd9c15a760 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00bb90 tx=0x7fcd8c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca003a3282023f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:42.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.542+0000 7fcd99d9b700 1 -- 172.21.15.112:0/3405757488 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00dd50 con 0x7fcd9c0cd070 2023-08-27T11:27:42.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.542+0000 7fcd99d9b700 1 -- 172.21.15.112:0/3405757488 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c010470 con 0x7fcd9c0cd070 2023-08-27T11:27:42.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.542+0000 7fcd99d9b700 1 -- 172.21.15.112:0/3405757488 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c010800 con 0x7fcd9c0cd070 2023-08-27T11:27:42.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.542+0000 7fcda120f700 1 -- 172.21.15.112:0/3405757488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd9c0cd070 msgr2=0x7fcd9c15a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:42.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.542+0000 7fcda120f700 1 --2- 172.21.15.112:0/3405757488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd9c0cd070 0x7fcd9c15a760 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00bb90 tx=0x7fcd8c00bea0 comp rx=0 tx=0).stop 2023-08-27T11:27:42.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.542+0000 7fcda120f700 1 -- 172.21.15.112:0/3405757488 shutdown_connections 2023-08-27T11:27:42.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.542+0000 7fcda120f700 1 --2- 172.21.15.112:0/3405757488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd9c0cd070 0x7fcd9c15a760 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:42.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.542+0000 7fcda120f700 1 -- 172.21.15.112:0/3405757488 >> 172.21.15.112:0/3405757488 conn(0x7fcd9c06bbf0 msgr2=0x7fcd9c0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:42.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.542+0000 7fcda120f700 1 -- 172.21.15.112:0/3405757488 shutdown_connections 2023-08-27T11:27:42.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.542+0000 7fcda120f700 1 -- 172.21.15.112:0/3405757488 wait complete. 2023-08-27T11:27:42.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.544+0000 7fcda120f700 1 Processor -- start 2023-08-27T11:27:42.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.544+0000 7fcda120f700 1 -- start start 2023-08-27T11:27:42.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.544+0000 7fcda120f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd9c071980 0x7fcd9c15b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:42.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.544+0000 7fcda120f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd9c0cd070 0x7fcd9c1ee100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:42.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.544+0000 7fcda120f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd9c15ea00 0x7fcd9c0b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:42.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.544+0000 7fcda120f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c15db20 con 0x7fcd9c071980 2023-08-27T11:27:42.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.544+0000 7fcda120f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c15d840 con 0x7fcd9c15ea00 2023-08-27T11:27:42.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.544+0000 7fcda120f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c1f20d0 con 0x7fcd9c0cd070 2023-08-27T11:27:42.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd9ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd9c071980 0x7fcd9c15b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:42.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd9ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd9c071980 0x7fcd9c15b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35224/0 (socket says 172.21.15.112:35224) 2023-08-27T11:27:42.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd9ad9d700 1 -- 172.21.15.112:0/4232954733 learned_addr learned my addr 172.21.15.112:0/4232954733 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:42.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd9b59e700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd9c15ea00 0x7fcd9c0b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:42.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd9a59c700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd9c0cd070 0x7fcd9c1ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:42.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd9ad9d700 1 -- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd9c0cd070 msgr2=0x7fcd9c1ee100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:42.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd9ad9d700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd9c0cd070 0x7fcd9c1ee100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:42.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd9ad9d700 1 -- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd9c15ea00 msgr2=0x7fcd9c0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:42.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd9ad9d700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd9c15ea00 0x7fcd9c0b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:42.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd9ad9d700 1 -- 172.21.15.112:0/4232954733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c00b7f0 con 0x7fcd9c071980 2023-08-27T11:27:42.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd9b59e700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd9c15ea00 0x7fcd9c0b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:42.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd9ad9d700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd9c071980 0x7fcd9c15b750 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000c520 tx=0x7fcd9000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:42.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd837fe700 1 -- 172.21.15.112:0/4232954733 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90005cb0 con 0x7fcd9c071980 2023-08-27T11:27:42.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd837fe700 1 -- 172.21.15.112:0/4232954733 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd90005e10 con 0x7fcd9c071980 2023-08-27T11:27:42.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcda120f700 1 -- 172.21.15.112:0/4232954733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9c1f23b0 con 0x7fcd9c071980 2023-08-27T11:27:42.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcd837fe700 1 -- 172.21.15.112:0/4232954733 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd900072d0 con 0x7fcd9c071980 2023-08-27T11:27:42.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.545+0000 7fcda120f700 1 -- 172.21.15.112:0/4232954733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c1f2bb0 con 0x7fcd9c071980 2023-08-27T11:27:42.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.546+0000 7fcda120f700 1 -- 172.21.15.112:0/4232954733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c066fa0 con 0x7fcd9c071980 2023-08-27T11:27:42.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.547+0000 7fcd837fe700 1 -- 172.21.15.112:0/4232954733 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fcd90007430 con 0x7fcd9c071980 2023-08-27T11:27:42.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.547+0000 7fcd837fe700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcd8806a4d0 0x7fcd8806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:42.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.547+0000 7fcd837fe700 1 -- 172.21.15.112:0/4232954733 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6960+0+0 (secure 0 0 0) 0x7fcd90088380 con 0x7fcd9c071980 2023-08-27T11:27:42.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.547+0000 7fcd9a59c700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcd8806a4d0 0x7fcd8806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:42.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.548+0000 7fcd9a59c700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcd8806a4d0 0x7fcd8806c990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c06e900 tx=0x7fcd8c013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:42.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.552+0000 7fcd837fe700 1 -- 172.21.15.112:0/4232954733 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcd90053250 con 0x7fcd9c071980 2023-08-27T11:27:42.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:42.768+0000 7fcda120f700 1 -- 172.21.15.112:0/4232954733 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000519094", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd9c06f790 con 0x7fcd8806a4d0 2023-08-27T11:27:43.695 INFO:tasks.ceph.mds.a.smithi112.stderr:2023-08-27T11:27:43.693+0000 7f95694c3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:27:45.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.380+0000 7fcd837fe700 1 -- 172.21.15.112:0/4232954733 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7fcd8c00deb0 con 0x7fcd8806a4d0 2023-08-27T11:27:45.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.385+0000 7fcda120f700 1 -- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcd8806a4d0 msgr2=0x7fcd8806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.385+0000 7fcda120f700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcd8806a4d0 0x7fcd8806c990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c06e900 tx=0x7fcd8c013000 comp rx=0 tx=0).stop 2023-08-27T11:27:45.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.385+0000 7fcda120f700 1 -- 172.21.15.112:0/4232954733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd9c071980 msgr2=0x7fcd9c15b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.385+0000 7fcda120f700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd9c071980 0x7fcd9c15b750 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000c520 tx=0x7fcd9000c8e0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.386+0000 7fcda120f700 1 -- 172.21.15.112:0/4232954733 shutdown_connections 2023-08-27T11:27:45.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.386+0000 7fcda120f700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcd8806a4d0 0x7fcd8806c990 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.387+0000 7fcda120f700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd9c15ea00 0x7fcd9c0b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.387+0000 7fcda120f700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd9c071980 0x7fcd9c15b750 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.387+0000 7fcda120f700 1 --2- 172.21.15.112:0/4232954733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd9c0cd070 0x7fcd9c1ee100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.387+0000 7fcda120f700 1 -- 172.21.15.112:0/4232954733 >> 172.21.15.112:0/4232954733 conn(0x7fcd9c06bbf0 msgr2=0x7fcd9c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:45.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.387+0000 7fcda120f700 1 -- 172.21.15.112:0/4232954733 shutdown_connections 2023-08-27T11:27:45.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.388+0000 7fcda120f700 1 -- 172.21.15.112:0/4232954733 wait complete. 2023-08-27T11:27:45.391 INFO:teuthology.orchestra.run.smithi112.stderr:Volume created successfully (no MDS daemons created) 2023-08-27T11:27:45.405 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:27:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.535+0000 7f81f891b700 1 Processor -- start 2023-08-27T11:27:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.535+0000 7f81f891b700 1 -- start start 2023-08-27T11:27:45.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f891b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f81f4144570 0x7f81f415ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f891b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f40b4ad0 0x7f81f40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f891b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f40b45f0 con 0x7f81f40b3e50 2023-08-27T11:27:45.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f891b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f40cf7a0 con 0x7f81f4144570 2023-08-27T11:27:45.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f891b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f4144950 con 0x7f81f40b4ad0 2023-08-27T11:27:45.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f2d9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f40b4ad0 0x7f81f40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f259c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f81f4144570 0x7f81f415ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f2d9d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f40b4ad0 0x7f81f40b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:46902/0 (socket says 172.21.15.112:46902) 2023-08-27T11:27:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f1d9b700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f81f40b3e50 0x7f81f415f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:37150/0 (socket says 172.21.15.112:37150) 2023-08-27T11:27:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f2d9d700 1 -- 172.21.15.112:0/954818201 learned_addr learned my addr 172.21.15.112:0/954818201 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:45.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f259c700 1 -- 172.21.15.112:0/954818201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f40b4ad0 msgr2=0x7f81f40b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:45.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f259c700 1 --2- 172.21.15.112:0/954818201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f40b4ad0 0x7f81f40b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f259c700 1 -- 172.21.15.112:0/954818201 >> v1:172.21.15.138:6789/0 conn(0x7f81f40b3e50 legacy=0x7f81f415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f259c700 1 -- 172.21.15.112:0/954818201 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f40b4310 con 0x7f81f4144570 2023-08-27T11:27:45.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.536+0000 7f81f2d9d700 1 --2- 172.21.15.112:0/954818201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f40b4ad0 0x7f81f40b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:45.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.537+0000 7f81f259c700 1 --2- 172.21.15.112:0/954818201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f4144570 0x7f81f415ee20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009870 tx=0x7f81e4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=983b073235c25a7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:45.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.537+0000 7f81f159a700 1 -- 172.21.15.112:0/954818201 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4011830 con 0x7f81f4144570 2023-08-27T11:27:45.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.537+0000 7f81f159a700 1 -- 172.21.15.112:0/954818201 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e4011990 con 0x7f81f4144570 2023-08-27T11:27:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.537+0000 7f81f159a700 1 -- 172.21.15.112:0/954818201 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4011d20 con 0x7f81f4144570 2023-08-27T11:27:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.537+0000 7f81f891b700 1 -- 172.21.15.112:0/954818201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f4144570 msgr2=0x7f81f415ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.537+0000 7f81f891b700 1 --2- 172.21.15.112:0/954818201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f4144570 0x7f81f415ee20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009870 tx=0x7f81e4009b80 comp rx=0 tx=0).stop 2023-08-27T11:27:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.537+0000 7f81f891b700 1 -- 172.21.15.112:0/954818201 shutdown_connections 2023-08-27T11:27:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.537+0000 7f81f891b700 1 --2- 172.21.15.112:0/954818201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f40b4ad0 0x7f81f40b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.537+0000 7f81f891b700 1 --2- 172.21.15.112:0/954818201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f4144570 0x7f81f415ee20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.537+0000 7f81f891b700 1 -- 172.21.15.112:0/954818201 >> 172.21.15.112:0/954818201 conn(0x7f81f406f970 msgr2=0x7f81f406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:45.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.537+0000 7f81f891b700 1 -- 172.21.15.112:0/954818201 shutdown_connections 2023-08-27T11:27:45.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.537+0000 7f81f891b700 1 -- 172.21.15.112:0/954818201 wait complete. 2023-08-27T11:27:45.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.539+0000 7f81f891b700 1 Processor -- start 2023-08-27T11:27:45.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.539+0000 7f81f891b700 1 -- start start 2023-08-27T11:27:45.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.539+0000 7f81f891b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f40b3e50 0x7f81f415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.539+0000 7f81f891b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f40b4ad0 0x7f81f40cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.539+0000 7f81f891b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81f4144570 0x7f81f4142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.539+0000 7f81f891b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f41f2c40 con 0x7f81f4144570 2023-08-27T11:27:45.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.539+0000 7f81f891b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f40cd010 con 0x7f81f40b4ad0 2023-08-27T11:27:45.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.539+0000 7f81f891b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f40ccd30 con 0x7f81f40b3e50 2023-08-27T11:27:45.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81f2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81f4144570 0x7f81f4142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81f2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81f4144570 0x7f81f4142d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35254/0 (socket says 172.21.15.112:35254) 2023-08-27T11:27:45.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81f2d9d700 1 -- 172.21.15.112:0/1543795908 learned_addr learned my addr 172.21.15.112:0/1543795908 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:45.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81f1d9b700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f40b4ad0 0x7f81f40cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81f259c700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f40b3e50 0x7f81f415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81f2d9d700 1 -- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f40b3e50 msgr2=0x7f81f415a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81f2d9d700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f40b3e50 0x7f81f415a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81f2d9d700 1 -- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f40b4ad0 msgr2=0x7f81f40cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81f2d9d700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f40b4ad0 0x7f81f40cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81f2d9d700 1 -- 172.21.15.112:0/1543795908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec009b30 con 0x7f81f4144570 2023-08-27T11:27:45.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81f2d9d700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81f4144570 0x7f81f4142d50 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00efc0 tx=0x7f81ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:45.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81e2ffd700 1 -- 172.21.15.112:0/1543795908 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec018070 con 0x7f81f4144570 2023-08-27T11:27:45.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81e2ffd700 1 -- 172.21.15.112:0/1543795908 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec00f040 con 0x7f81f4144570 2023-08-27T11:27:45.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81f891b700 1 -- 172.21.15.112:0/1543795908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e40094a0 con 0x7f81f4144570 2023-08-27T11:27:45.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81e2ffd700 1 -- 172.21.15.112:0/1543795908 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec003c20 con 0x7f81f4144570 2023-08-27T11:27:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.540+0000 7f81f891b700 1 -- 172.21.15.112:0/1543795908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f4070d20 con 0x7f81f4144570 2023-08-27T11:27:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.541+0000 7f81f891b700 1 -- 172.21.15.112:0/1543795908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f40708a0 con 0x7f81f4144570 2023-08-27T11:27:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.542+0000 7f81e2ffd700 1 -- 172.21.15.112:0/1543795908 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f81ec00c880 con 0x7f81f4144570 2023-08-27T11:27:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.542+0000 7f81e2ffd700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f81d406a4d0 0x7f81d406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.542+0000 7f81e2ffd700 1 -- 172.21.15.112:0/1543795908 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 7858+0+0 (secure 0 0 0) 0x7f81ec089640 con 0x7f81f4144570 2023-08-27T11:27:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.542+0000 7f81f259c700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f81d406a4d0 0x7f81d406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.543+0000 7f81f259c700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f81d406a4d0 0x7f81d406c990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f81e40090b0 tx=0x7f81e401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.547+0000 7f81e2ffd700 1 -- 172.21.15.112:0/1543795908 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f81f40708a0 con 0x7f81f4144570 2023-08-27T11:27:45.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.763+0000 7f81f891b700 1 -- 172.21.15.112:0/1543795908 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f81f40cdf90 con 0x7f81d406a4d0 2023-08-27T11:27:45.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.765+0000 7f81e2ffd700 1 -- 172.21.15.112:0/1543795908 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+151 (secure 0 0 0) 0x7f81e4011b80 con 0x7f81d406a4d0 2023-08-27T11:27:45.766 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:27:45.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:45.767 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs" 2023-08-27T11:27:45.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:45.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:45.767 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volume_0000000000519093" 2023-08-27T11:27:45.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:27:45.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:27:45.768 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volume_0000000000519094" 2023-08-27T11:27:45.768 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:27:45.769 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:27:45.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.767+0000 7f81f891b700 1 -- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f81d406a4d0 msgr2=0x7f81d406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.767+0000 7f81f891b700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f81d406a4d0 0x7f81d406c990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f81e40090b0 tx=0x7f81e401c040 comp rx=0 tx=0).stop 2023-08-27T11:27:45.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.767+0000 7f81f891b700 1 -- 172.21.15.112:0/1543795908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81f4144570 msgr2=0x7f81f4142d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.767+0000 7f81f891b700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81f4144570 0x7f81f4142d50 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00efc0 tx=0x7f81ec00c3a0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.767+0000 7f81f891b700 1 -- 172.21.15.112:0/1543795908 shutdown_connections 2023-08-27T11:27:45.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.767+0000 7f81f891b700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f81d406a4d0 0x7f81d406c990 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.767+0000 7f81f891b700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81f4144570 0x7f81f4142d50 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.767+0000 7f81f891b700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f40b3e50 0x7f81f415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.767+0000 7f81f891b700 1 --2- 172.21.15.112:0/1543795908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f40b4ad0 0x7f81f40cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.767+0000 7f81f891b700 1 -- 172.21.15.112:0/1543795908 >> 172.21.15.112:0/1543795908 conn(0x7f81f406f970 msgr2=0x7f81f41e3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:45.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.767+0000 7f81f891b700 1 -- 172.21.15.112:0/1543795908 shutdown_connections 2023-08-27T11:27:45.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.767+0000 7f81f891b700 1 -- 172.21.15.112:0/1543795908 wait complete. 2023-08-27T11:27:45.781 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000519093 --yes-i-really-mean-it 2023-08-27T11:27:45.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.902+0000 7f42e3162700 1 Processor -- start 2023-08-27T11:27:45.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.902+0000 7f42e3162700 1 -- start start 2023-08-27T11:27:45.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.903+0000 7f42e3162700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f42dc0b3de0 0x7f42dc06bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.903+0000 7f42e3162700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42dc0cef20 con 0x7f42dc06c510 2023-08-27T11:27:45.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.903+0000 7f42e3162700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42dc0b7310 con 0x7f42dc144b80 2023-08-27T11:27:45.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.903+0000 7f42e3162700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42dc0b4b40 con 0x7f42dc0b3de0 2023-08-27T11:27:45.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.903+0000 7f42e16ff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f42dc06c510 0x7f42dc0cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57876/0 (socket says 172.21.15.112:57876) 2023-08-27T11:27:45.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.903+0000 7f42e16ff700 1 -- 172.21.15.112:0/3161584360 learned_addr learned my addr 172.21.15.112:0/3161584360 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:45.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.903+0000 7f42dbfff700 1 --2- 172.21.15.112:0/3161584360 >> v2:172.21.15.138:3300/0 conn(0x7f42dc0b3de0 0x7f42dc06bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.903+0000 7f42db7fe700 1 -- 172.21.15.112:0/3161584360 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898908967 0 0) 0x7f42dc0cef20 con 0x7f42dc06c510 2023-08-27T11:27:45.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.903+0000 7f42db7fe700 1 -- 172.21.15.112:0/3161584360 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42cc0035b0 con 0x7f42dc06c510 2023-08-27T11:27:45.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.903+0000 7f42dbfff700 1 -- 172.21.15.112:0/3161584360 >> v1:172.21.15.138:6789/0 conn(0x7f42dc144b80 legacy=0x7f42dc0b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.903+0000 7f42db7fe700 1 -- 172.21.15.112:0/3161584360 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3533092355 0 0) 0x7f42cc0035b0 con 0x7f42dc06c510 2023-08-27T11:27:45.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.903+0000 7f42dbfff700 1 -- 172.21.15.112:0/3161584360 >> v1:172.21.15.112:6789/0 conn(0x7f42dc06c510 legacy=0x7f42dc0cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.903+0000 7f42dbfff700 1 -- 172.21.15.112:0/3161584360 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc06c910 con 0x7f42dc0b3de0 2023-08-27T11:27:45.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.904+0000 7f42dbfff700 1 --2- 172.21.15.112:0/3161584360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42dc0b3de0 0x7f42dc06bfd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009840 tx=0x7f42c8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42c406359bd8770c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:45.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.904+0000 7f42db7fe700 1 -- 172.21.15.112:0/3161584360 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d40019f0 con 0x7f42dc0b3de0 2023-08-27T11:27:45.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.904+0000 7f42db7fe700 1 -- 172.21.15.112:0/3161584360 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c8004030 con 0x7f42dc0b3de0 2023-08-27T11:27:45.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.904+0000 7f42db7fe700 1 -- 172.21.15.112:0/3161584360 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c80043a0 con 0x7f42dc0b3de0 2023-08-27T11:27:45.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.904+0000 7f42e3162700 1 -- 172.21.15.112:0/3161584360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42dc0b3de0 msgr2=0x7f42dc06bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.904+0000 7f42e3162700 1 --2- 172.21.15.112:0/3161584360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42dc0b3de0 0x7f42dc06bfd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009840 tx=0x7f42c8009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:45.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.904+0000 7f42e3162700 1 -- 172.21.15.112:0/3161584360 shutdown_connections 2023-08-27T11:27:45.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.904+0000 7f42e3162700 1 --2- 172.21.15.112:0/3161584360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42dc0b3de0 0x7f42dc06bfd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.904+0000 7f42e3162700 1 -- 172.21.15.112:0/3161584360 >> 172.21.15.112:0/3161584360 conn(0x7f42dc0cdc50 msgr2=0x7f42dc0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:45.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.904+0000 7f42e3162700 1 -- 172.21.15.112:0/3161584360 shutdown_connections 2023-08-27T11:27:45.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.904+0000 7f42e3162700 1 -- 172.21.15.112:0/3161584360 wait complete. 2023-08-27T11:27:45.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.905+0000 7f42e3162700 1 Processor -- start 2023-08-27T11:27:45.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.905+0000 7f42e3162700 1 -- start start 2023-08-27T11:27:45.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.906+0000 7f42e3162700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42dc06c510 0x7f42dc1f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.906+0000 7f42e3162700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42dc0b3de0 0x7f42dc13d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.906+0000 7f42e3162700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42dc144b80 0x7f42dc0c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.906+0000 7f42e3162700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc1f4150 con 0x7f42dc06c510 2023-08-27T11:27:45.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.906+0000 7f42e3162700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc15a4c0 con 0x7f42dc0b3de0 2023-08-27T11:27:45.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.906+0000 7f42e3162700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0c84f0 con 0x7f42dc144b80 2023-08-27T11:27:45.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.906+0000 7f42e0efe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42dc06c510 0x7f42dc1f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.906+0000 7f42e0efe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42dc06c510 0x7f42dc1f1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35266/0 (socket says 172.21.15.112:35266) 2023-08-27T11:27:45.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.906+0000 7f42e0efe700 1 -- 172.21.15.112:0/2699787693 learned_addr learned my addr 172.21.15.112:0/2699787693 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:45.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42dbfff700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42dc0b3de0 0x7f42dc13d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42e0efe700 1 -- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42dc144b80 msgr2=0x7f42dc0c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42e16ff700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42dc144b80 0x7f42dc0c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42e0efe700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42dc144b80 0x7f42dc0c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42e0efe700 1 -- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42dc0b3de0 msgr2=0x7f42dc13d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42e0efe700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42dc0b3de0 0x7f42dc13d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42e0efe700 1 -- 172.21.15.112:0/2699787693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c80094a0 con 0x7f42dc06c510 2023-08-27T11:27:45.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42dbfff700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42dc0b3de0 0x7f42dc13d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:45.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42e0efe700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42dc06c510 0x7f42dc1f1f00 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f42d000c430 tx=0x7f42d000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:45.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42d97fa700 1 -- 172.21.15.112:0/2699787693 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00052d0 con 0x7f42dc06c510 2023-08-27T11:27:45.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42d97fa700 1 -- 172.21.15.112:0/2699787693 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d0005450 con 0x7f42dc06c510 2023-08-27T11:27:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42e3162700 1 -- 172.21.15.112:0/2699787693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc159e70 con 0x7f42dc06c510 2023-08-27T11:27:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42d97fa700 1 -- 172.21.15.112:0/2699787693 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0006aa0 con 0x7f42dc06c510 2023-08-27T11:27:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.907+0000 7f42e3162700 1 -- 172.21.15.112:0/2699787693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc0c87d0 con 0x7f42dc06c510 2023-08-27T11:27:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.908+0000 7f42e3162700 1 -- 172.21.15.112:0/2699787693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc074890 con 0x7f42dc06c510 2023-08-27T11:27:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.908+0000 7f42d97fa700 1 -- 172.21.15.112:0/2699787693 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f42d0003680 con 0x7f42dc06c510 2023-08-27T11:27:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.909+0000 7f42d97fa700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f42c006a4d0 0x7f42c006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.909+0000 7f42d97fa700 1 -- 172.21.15.112:0/2699787693 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 7858+0+0 (secure 0 0 0) 0x7f42d00885f0 con 0x7f42dc06c510 2023-08-27T11:27:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.909+0000 7f42dbfff700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f42c006a4d0 0x7f42c006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.909+0000 7f42dbfff700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f42c006a4d0 0x7f42c006c990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0b2660 tx=0x7f42c8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:45.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:45.913+0000 7f42d97fa700 1 -- 172.21.15.112:0/2699787693 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f42d0052ff0 con 0x7f42dc06c510 2023-08-27T11:27:46.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:46.127+0000 7f42e3162700 1 -- 172.21.15.112:0/2699787693 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000519093", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f42dc0b4580 con 0x7f42c006a4d0 2023-08-27T11:27:46.133 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:27:46.133+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:27:46.138 INFO:tasks.ceph.mds.a.smithi112.stderr: -72> 2023-08-27T11:27:43.693+0000 7f95694c3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:27:46.160 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:27:48.695 INFO:tasks.ceph.mds.c.smithi112.stderr:2023-08-27T11:27:48.695+0000 7fd24a370700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:27:49.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.161+0000 7f42d97fa700 1 -- 172.21.15.112:0/2699787693 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7f42c8004340 con 0x7f42c006a4d0 2023-08-27T11:27:49.162 INFO:teuthology.orchestra.run.smithi112.stdout:metadata pool: cephfs.volume_0000000000519093.meta data pool: ['cephfs.volume_0000000000519093.data'] removed 2023-08-27T11:27:49.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.164+0000 7f42e3162700 1 -- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f42c006a4d0 msgr2=0x7f42c006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.164+0000 7f42e3162700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f42c006a4d0 0x7f42c006c990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0b2660 tx=0x7f42c8008030 comp rx=0 tx=0).stop 2023-08-27T11:27:49.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.164+0000 7f42e3162700 1 -- 172.21.15.112:0/2699787693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42dc06c510 msgr2=0x7f42dc1f1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.164+0000 7f42e3162700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42dc06c510 0x7f42dc1f1f00 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f42d000c430 tx=0x7f42d000c7f0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.164+0000 7f42e3162700 1 -- 172.21.15.112:0/2699787693 shutdown_connections 2023-08-27T11:27:49.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.164+0000 7f42e3162700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f42c006a4d0 0x7f42c006c990 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.164+0000 7f42e3162700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42dc144b80 0x7f42dc0c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.164+0000 7f42e3162700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42dc06c510 0x7f42dc1f1f00 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.164+0000 7f42e3162700 1 --2- 172.21.15.112:0/2699787693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42dc0b3de0 0x7f42dc13d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.164+0000 7f42e3162700 1 -- 172.21.15.112:0/2699787693 >> 172.21.15.112:0/2699787693 conn(0x7f42dc0cdc50 msgr2=0x7f42dc15e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:49.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.164+0000 7f42e3162700 1 -- 172.21.15.112:0/2699787693 shutdown_connections 2023-08-27T11:27:49.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.164+0000 7f42e3162700 1 -- 172.21.15.112:0/2699787693 wait complete. 2023-08-27T11:27:49.180 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000519094 --yes-i-really-mean-it 2023-08-27T11:27:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.309+0000 7f101599f700 1 Processor -- start 2023-08-27T11:27:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.309+0000 7f101599f700 1 -- start start 2023-08-27T11:27:49.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f101599f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10100b3440 0x7f101015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f101599f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f101015e6e0 0x7f101006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f101599f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10100b5070 con 0x7f1010146c20 2023-08-27T11:27:49.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f101599f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1010147000 con 0x7f101015e6e0 2023-08-27T11:27:49.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f101599f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1010161740 con 0x7f10100b3440 2023-08-27T11:27:49.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f100f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f101015e6e0 0x7f101006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f100effd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1010146c20 0x7f1010160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57900/0 (socket says 172.21.15.112:57900) 2023-08-27T11:27:49.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f100effd700 1 -- 172.21.15.112:0/1264624599 learned_addr learned my addr 172.21.15.112:0/1264624599 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:49.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f100e7fc700 1 --2- 172.21.15.112:0/1264624599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10100b3440 0x7f101015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f100f7fe700 1 -- 172.21.15.112:0/1264624599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10100b3440 msgr2=0x7f101015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:49.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f100f7fe700 1 --2- 172.21.15.112:0/1264624599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10100b3440 0x7f101015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f100f7fe700 1 -- 172.21.15.112:0/1264624599 >> v1:172.21.15.112:6789/0 conn(0x7f1010146c20 legacy=0x7f1010160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f100f7fe700 1 -- 172.21.15.112:0/1264624599 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1010161a20 con 0x7f101015e6e0 2023-08-27T11:27:49.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f100dffb700 1 -- 172.21.15.112:0/1264624599 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2877275387 0 0) 0x7f1004001110 con 0x7f1010146c20 2023-08-27T11:27:49.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f100f7fe700 1 --2- 172.21.15.112:0/1264624599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101015e6e0 0x7f101006f790 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f1008009d90 tx=0x7f100800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c707ed2096a5898a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:49.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f100dffb700 1 -- 172.21.15.112:0/1264624599 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100800e070 con 0x7f101015e6e0 2023-08-27T11:27:49.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f100dffb700 1 -- 172.21.15.112:0/1264624599 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100800bbd0 con 0x7f101015e6e0 2023-08-27T11:27:49.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.310+0000 7f100dffb700 1 -- 172.21.15.112:0/1264624599 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1008012430 con 0x7f101015e6e0 2023-08-27T11:27:49.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.311+0000 7f101599f700 1 -- 172.21.15.112:0/1264624599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101015e6e0 msgr2=0x7f101006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.311+0000 7f101599f700 1 --2- 172.21.15.112:0/1264624599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101015e6e0 0x7f101006f790 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f1008009d90 tx=0x7f100800b330 comp rx=0 tx=0).stop 2023-08-27T11:27:49.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.311+0000 7f101599f700 1 -- 172.21.15.112:0/1264624599 shutdown_connections 2023-08-27T11:27:49.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.311+0000 7f101599f700 1 --2- 172.21.15.112:0/1264624599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101015e6e0 0x7f101006f790 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.311+0000 7f101599f700 1 --2- 172.21.15.112:0/1264624599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10100b3440 0x7f101015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.311+0000 7f101599f700 1 -- 172.21.15.112:0/1264624599 >> 172.21.15.112:0/1264624599 conn(0x7f10100b1e90 msgr2=0x7f10100b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:49.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.311+0000 7f101599f700 1 -- 172.21.15.112:0/1264624599 shutdown_connections 2023-08-27T11:27:49.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.311+0000 7f101599f700 1 -- 172.21.15.112:0/1264624599 wait complete. 2023-08-27T11:27:49.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.312+0000 7f101599f700 1 Processor -- start 2023-08-27T11:27:49.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.312+0000 7f101599f700 1 -- start start 2023-08-27T11:27:49.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f101599f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10100b3440 0x7f10101603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f101599f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1010146c20 0x7f10101f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f101599f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f101015e6e0 0x7f101015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f101599f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101015f9a0 con 0x7f10100b3440 2023-08-27T11:27:49.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f101599f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1010160920 con 0x7f1010146c20 2023-08-27T11:27:49.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f101599f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1010146260 con 0x7f101015e6e0 2023-08-27T11:27:49.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f100effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10100b3440 0x7f10101603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f100effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10100b3440 0x7f10101603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35292/0 (socket says 172.21.15.112:35292) 2023-08-27T11:27:49.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f100effd700 1 -- 172.21.15.112:0/106360063 learned_addr learned my addr 172.21.15.112:0/106360063 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:49.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f100e7fc700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1010146c20 0x7f10101f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f100f7fe700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f101015e6e0 0x7f101015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f100effd700 1 -- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f101015e6e0 msgr2=0x7f101015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f100effd700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f101015e6e0 0x7f101015a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f100effd700 1 -- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1010146c20 msgr2=0x7f10101f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f100effd700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1010146c20 0x7f10101f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f100effd700 1 -- 172.21.15.112:0/106360063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1008009a70 con 0x7f10100b3440 2023-08-27T11:27:49.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f100f7fe700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f101015e6e0 0x7f101015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:49.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.313+0000 7f100effd700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10100b3440 0x7f10101603e0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f100400c730 tx=0x7f100400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:49.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.314+0000 7f0ff7fff700 1 -- 172.21.15.112:0/106360063 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004004c80 con 0x7f10100b3440 2023-08-27T11:27:49.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.314+0000 7f0ff7fff700 1 -- 172.21.15.112:0/106360063 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1004004e00 con 0x7f10100b3440 2023-08-27T11:27:49.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.314+0000 7f101599f700 1 -- 172.21.15.112:0/106360063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101015f3e0 con 0x7f10100b3440 2023-08-27T11:27:49.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.314+0000 7f0ff7fff700 1 -- 172.21.15.112:0/106360063 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004007310 con 0x7f10100b3440 2023-08-27T11:27:49.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.314+0000 7f101599f700 1 -- 172.21.15.112:0/106360063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10101f6780 con 0x7f10100b3440 2023-08-27T11:27:49.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.315+0000 7f101599f700 1 -- 172.21.15.112:0/106360063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101013eb50 con 0x7f10100b3440 2023-08-27T11:27:49.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.315+0000 7f0ff7fff700 1 -- 172.21.15.112:0/106360063 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f1004007470 con 0x7f10100b3440 2023-08-27T11:27:49.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.316+0000 7f0ff7fff700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0ffc06a4d0 0x7f0ffc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.316+0000 7f0ff7fff700 1 -- 172.21.15.112:0/106360063 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 7046+0+0 (secure 0 0 0) 0x7f1004088b50 con 0x7f10100b3440 2023-08-27T11:27:49.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.316+0000 7f100e7fc700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0ffc06a4d0 0x7f0ffc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.316+0000 7f100e7fc700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0ffc06a4d0 0x7f0ffc06c990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f101006bcc0 tx=0x7f1000008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:49.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.320+0000 7f0ff7fff700 1 -- 172.21.15.112:0/106360063 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f101013eb50 con 0x7f10100b3440 2023-08-27T11:27:49.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:49.535+0000 7f101599f700 1 -- 172.21.15.112:0/106360063 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000519094", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f1010161d00 con 0x7f0ffc06a4d0 2023-08-27T11:27:49.541 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:27:49.541+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:27:49.546 INFO:tasks.ceph.mds.c.smithi112.stderr: -39> 2023-08-27T11:27:48.695+0000 7fd24a370700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:27:49.569 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:27:52.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.556+0000 7f0ff7fff700 1 -- 172.21.15.112:0/106360063 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7f10000095c0 con 0x7f0ffc06a4d0 2023-08-27T11:27:52.558 INFO:teuthology.orchestra.run.smithi112.stdout:metadata pool: cephfs.volume_0000000000519094.meta data pool: ['cephfs.volume_0000000000519094.data'] removed 2023-08-27T11:27:52.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.558+0000 7f101599f700 1 -- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0ffc06a4d0 msgr2=0x7f0ffc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.558+0000 7f101599f700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0ffc06a4d0 0x7f0ffc06c990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f101006bcc0 tx=0x7f1000008040 comp rx=0 tx=0).stop 2023-08-27T11:27:52.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.558+0000 7f101599f700 1 -- 172.21.15.112:0/106360063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10100b3440 msgr2=0x7f10101603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.558+0000 7f101599f700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10100b3440 0x7f10101603e0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f100400c730 tx=0x7f100400caf0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.559+0000 7f101599f700 1 -- 172.21.15.112:0/106360063 shutdown_connections 2023-08-27T11:27:52.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.559+0000 7f101599f700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0ffc06a4d0 0x7f0ffc06c990 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.559+0000 7f101599f700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f101015e6e0 0x7f101015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.559+0000 7f101599f700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10100b3440 0x7f10101603e0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.559+0000 7f101599f700 1 --2- 172.21.15.112:0/106360063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1010146c20 0x7f10101f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.559+0000 7f101599f700 1 -- 172.21.15.112:0/106360063 >> 172.21.15.112:0/106360063 conn(0x7f10100b1e90 msgr2=0x7f101006d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:52.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.559+0000 7f101599f700 1 -- 172.21.15.112:0/106360063 shutdown_connections 2023-08-27T11:27:52.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.559+0000 7f101599f700 1 -- 172.21.15.112:0/106360063 wait complete. 2023-08-27T11:27:52.575 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:27:52.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:27:52.628 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:27:52.675 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:27:52.675 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:27:52.676 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:52.694 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:27:52.695 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:52.754 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:52.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.915+0000 7f244adb4700 1 Processor -- start 2023-08-27T11:27:52.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.915+0000 7f244adb4700 1 -- start start 2023-08-27T11:27:52.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.916+0000 7f244adb4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2444144570 0x7f244415ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.916+0000 7f244adb4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f24440b3e50 0x7f244415f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.916+0000 7f244adb4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24440b5ae0 con 0x7f244415f8a0 2023-08-27T11:27:52.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.916+0000 7f244adb4700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244415fcf0 con 0x7f24440b3e50 2023-08-27T11:27:52.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.916+0000 7f244adb4700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2444144950 con 0x7f2444144570 2023-08-27T11:27:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.916+0000 7f2443fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f24440b3e50 0x7f244415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.916+0000 7f2449351700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f244415f8a0 0x7f24440ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57938/0 (socket says 172.21.15.112:57938) 2023-08-27T11:27:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.916+0000 7f2449351700 1 -- 172.21.15.112:0/2966350984 learned_addr learned my addr 172.21.15.112:0/2966350984 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.917+0000 7f2448b50700 1 --2- 172.21.15.112:0/2966350984 >> v2:172.21.15.138:3300/0 conn(0x7f2444144570 0x7f244415ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.917+0000 7f2443fff700 1 -- 172.21.15.112:0/2966350984 >> v2:172.21.15.138:3300/0 conn(0x7f2444144570 msgr2=0x7f244415ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.917+0000 7f2443fff700 1 --2- 172.21.15.112:0/2966350984 >> v2:172.21.15.138:3300/0 conn(0x7f2444144570 0x7f244415ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.917+0000 7f2443fff700 1 -- 172.21.15.112:0/2966350984 >> v1:172.21.15.112:6789/0 conn(0x7f244415f8a0 legacy=0x7f24440ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.917+0000 7f24437fe700 1 -- 172.21.15.112:0/2966350984 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098251256 0 0) 0x7f243c001490 con 0x7f244415f8a0 2023-08-27T11:27:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.917+0000 7f2443fff700 1 -- 172.21.15.112:0/2966350984 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440b45f0 con 0x7f24440b3e50 2023-08-27T11:27:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.917+0000 7f2443fff700 1 --2- 172.21.15.112:0/2966350984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24440b3e50 0x7f244415f360 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f244406b7b0 tx=0x7f2430009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36d7edf259632d2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:52.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.917+0000 7f24437fe700 1 -- 172.21.15.112:0/2966350984 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430011830 con 0x7f24440b3e50 2023-08-27T11:27:52.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.917+0000 7f24437fe700 1 -- 172.21.15.112:0/2966350984 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430011990 con 0x7f24440b3e50 2023-08-27T11:27:52.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.917+0000 7f24437fe700 1 -- 172.21.15.112:0/2966350984 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430011d20 con 0x7f24440b3e50 2023-08-27T11:27:52.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.918+0000 7f244adb4700 1 -- 172.21.15.112:0/2966350984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24440b3e50 msgr2=0x7f244415f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.918+0000 7f244adb4700 1 --2- 172.21.15.112:0/2966350984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24440b3e50 0x7f244415f360 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f244406b7b0 tx=0x7f2430009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:52.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.918+0000 7f244adb4700 1 -- 172.21.15.112:0/2966350984 shutdown_connections 2023-08-27T11:27:52.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.918+0000 7f244adb4700 1 --2- 172.21.15.112:0/2966350984 >> v2:172.21.15.138:3300/0 conn(0x7f2444144570 0x7f244415ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.918+0000 7f244adb4700 1 --2- 172.21.15.112:0/2966350984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24440b3e50 0x7f244415f360 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.918+0000 7f244adb4700 1 -- 172.21.15.112:0/2966350984 >> 172.21.15.112:0/2966350984 conn(0x7f244406f970 msgr2=0x7f244406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:52.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.918+0000 7f244adb4700 1 -- 172.21.15.112:0/2966350984 shutdown_connections 2023-08-27T11:27:52.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.918+0000 7f244adb4700 1 -- 172.21.15.112:0/2966350984 wait complete. 2023-08-27T11:27:52.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.919+0000 7f244adb4700 1 Processor -- start 2023-08-27T11:27:52.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.919+0000 7f244adb4700 1 -- start start 2023-08-27T11:27:52.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.920+0000 7f244adb4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24440b3e50 0x7f2444140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.920+0000 7f244adb4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2444144570 0x7f24440c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.920+0000 7f244adb4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244415f8a0 0x7f244413d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.920+0000 7f244adb4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440c8ba0 con 0x7f244415f8a0 2023-08-27T11:27:52.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.920+0000 7f244adb4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440c6bf0 con 0x7f24440b3e50 2023-08-27T11:27:52.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.920+0000 7f244adb4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24440c6ed0 con 0x7f2444144570 2023-08-27T11:27:52.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f2449351700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244415f8a0 0x7f244413d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f2449351700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244415f8a0 0x7f244413d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35330/0 (socket says 172.21.15.112:35330) 2023-08-27T11:27:52.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f2449351700 1 -- 172.21.15.112:0/972424910 learned_addr learned my addr 172.21.15.112:0/972424910 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:52.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f2448b50700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24440b3e50 0x7f2444140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f2443fff700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2444144570 0x7f24440c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f2449351700 1 -- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2444144570 msgr2=0x7f24440c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f2449351700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2444144570 0x7f24440c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f2449351700 1 -- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24440b3e50 msgr2=0x7f2444140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f2449351700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24440b3e50 0x7f2444140e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f2449351700 1 -- 172.21.15.112:0/972424910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24300094a0 con 0x7f244415f8a0 2023-08-27T11:27:52.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f2449351700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244415f8a0 0x7f244413d340 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f243c00cb60 tx=0x7f243c00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:52.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f24417fa700 1 -- 172.21.15.112:0/972424910 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c00aaa0 con 0x7f244415f8a0 2023-08-27T11:27:52.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f24417fa700 1 -- 172.21.15.112:0/972424910 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c00ac00 con 0x7f244415f8a0 2023-08-27T11:27:52.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f244adb4700 1 -- 172.21.15.112:0/972424910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24440c71b0 con 0x7f244415f8a0 2023-08-27T11:27:52.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f24417fa700 1 -- 172.21.15.112:0/972424910 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c0070d0 con 0x7f244415f8a0 2023-08-27T11:27:52.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.921+0000 7f244adb4700 1 -- 172.21.15.112:0/972424910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24440cd160 con 0x7f244415f8a0 2023-08-27T11:27:52.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.922+0000 7f244adb4700 1 -- 172.21.15.112:0/972424910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2444067a20 con 0x7f244415f8a0 2023-08-27T11:27:52.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.923+0000 7f24417fa700 1 -- 172.21.15.112:0/972424910 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f243c007230 con 0x7f244415f8a0 2023-08-27T11:27:52.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.923+0000 7f24417fa700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f242806a4d0 0x7f242806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.923+0000 7f24417fa700 1 -- 172.21.15.112:0/972424910 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f243c088800 con 0x7f244415f8a0 2023-08-27T11:27:52.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.923+0000 7f2448b50700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f242806a4d0 0x7f242806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.924+0000 7f2448b50700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f242806a4d0 0x7f242806c990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2438006b20 tx=0x7f2438008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:52.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:52.927+0000 7f24417fa700 1 -- 172.21.15.112:0/972424910 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f243c053960 con 0x7f244415f8a0 2023-08-27T11:27:53.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.119+0000 7f244adb4700 1 -- 172.21.15.112:0/972424910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24440b5ae0 con 0x7f244415f8a0 2023-08-27T11:27:53.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.120+0000 7f24417fa700 1 -- 172.21.15.112:0/972424910 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+18538 (secure 0 0 0) 0x7f24440b5ae0 con 0x7f244415f8a0 2023-08-27T11:27:53.122 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:53.122 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":127,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:27:52.546502+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:26.212595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:27.068136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6832/3832579050":"2023-08-28T11:27:49.540827+0000","172.21.15.112:6834/3832579050":"2023-08-28T11:27:49.540827+0000","172.21.15.112:6835/4019485943":"2023-08-28T11:27:46.133510+0000","172.21.15.112:6833/4019485943":"2023-08-28T11:27:46.133510+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:53.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.122+0000 7f244adb4700 1 -- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f242806a4d0 msgr2=0x7f242806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.122+0000 7f244adb4700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f242806a4d0 0x7f242806c990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2438006b20 tx=0x7f2438008040 comp rx=0 tx=0).stop 2023-08-27T11:27:53.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.122+0000 7f244adb4700 1 -- 172.21.15.112:0/972424910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244415f8a0 msgr2=0x7f244413d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.122+0000 7f244adb4700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244415f8a0 0x7f244413d340 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f243c00cb60 tx=0x7f243c00ce70 comp rx=0 tx=0).stop 2023-08-27T11:27:53.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.122+0000 7f244adb4700 1 -- 172.21.15.112:0/972424910 shutdown_connections 2023-08-27T11:27:53.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.122+0000 7f244adb4700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f242806a4d0 0x7f242806c990 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.122+0000 7f244adb4700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244415f8a0 0x7f244413d340 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.122+0000 7f244adb4700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24440b3e50 0x7f2444140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.122+0000 7f244adb4700 1 --2- 172.21.15.112:0/972424910 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2444144570 0x7f24440c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.122+0000 7f244adb4700 1 -- 172.21.15.112:0/972424910 >> 172.21.15.112:0/972424910 conn(0x7f244406f970 msgr2=0x7f244415a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:53.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.122+0000 7f244adb4700 1 -- 172.21.15.112:0/972424910 shutdown_connections 2023-08-27T11:27:53.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.123+0000 7f244adb4700 1 -- 172.21.15.112:0/972424910 wait complete. 2023-08-27T11:27:53.136 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:27:26.212595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:27:27.068136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:53.137 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:53.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.258+0000 7fb6f6389700 1 Processor -- start 2023-08-27T11:27:53.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.258+0000 7fb6f6389700 1 -- start start 2023-08-27T11:27:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.259+0000 7fb6f6389700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f0152130 0x7fb6f015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.259+0000 7fb6f6389700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb6f00b3cf0 0x7fb6f00b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.259+0000 7fb6f6389700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f015f750 con 0x7fb6f015cd60 2023-08-27T11:27:53.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.259+0000 7fb6f6389700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f00b46c0 con 0x7fb6f00b3cf0 2023-08-27T11:27:53.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.259+0000 7fb6f6389700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f015fa30 con 0x7fb6f0152130 2023-08-27T11:27:53.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.259+0000 7fb6f4926700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb6f00b3cf0 0x7fb6f00b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.259+0000 7fb6f4926700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb6f00b3cf0 0x7fb6f00b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35336/0 (socket says 172.21.15.112:35336) 2023-08-27T11:27:53.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.259+0000 7fb6f4926700 1 -- 172.21.15.112:0/1772817389 learned_addr learned my addr 172.21.15.112:0/1772817389 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.260+0000 7fb6f4926700 1 -- 172.21.15.112:0/1772817389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f0152130 msgr2=0x7fb6f015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.260+0000 7fb6effff700 1 --2- 172.21.15.112:0/1772817389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f0152130 0x7fb6f015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.260+0000 7fb6f4926700 1 --2- 172.21.15.112:0/1772817389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f0152130 0x7fb6f015c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.260+0000 7fb6f4926700 1 -- 172.21.15.112:0/1772817389 >> v1:172.21.15.138:6789/0 conn(0x7fb6f015cd60 legacy=0x7fb6f015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.260+0000 7fb6f4926700 1 -- 172.21.15.112:0/1772817389 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f00b60a0 con 0x7fb6f00b3cf0 2023-08-27T11:27:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.260+0000 7fb6effff700 1 --2- 172.21.15.112:0/1772817389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f0152130 0x7fb6f015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.260+0000 7fb6f4926700 1 --2- 172.21.15.112:0/1772817389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00b3cf0 0x7fb6f00b4180 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009e50 tx=0x7fb6e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dbe8c48db08fac server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.260+0000 7fb6eeffd700 1 -- 172.21.15.112:0/1772817389 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800e070 con 0x7fb6f00b3cf0 2023-08-27T11:27:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.260+0000 7fb6eeffd700 1 -- 172.21.15.112:0/1772817389 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e800b9c0 con 0x7fb6f00b3cf0 2023-08-27T11:27:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.260+0000 7fb6eeffd700 1 -- 172.21.15.112:0/1772817389 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800bd70 con 0x7fb6f00b3cf0 2023-08-27T11:27:53.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.260+0000 7fb6f6389700 1 -- 172.21.15.112:0/1772817389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00b3cf0 msgr2=0x7fb6f00b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.260+0000 7fb6f6389700 1 --2- 172.21.15.112:0/1772817389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00b3cf0 0x7fb6f00b4180 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009e50 tx=0x7fb6e800b330 comp rx=0 tx=0).stop 2023-08-27T11:27:53.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.261+0000 7fb6f6389700 1 -- 172.21.15.112:0/1772817389 shutdown_connections 2023-08-27T11:27:53.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.261+0000 7fb6f6389700 1 --2- 172.21.15.112:0/1772817389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00b3cf0 0x7fb6f00b4180 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.261+0000 7fb6f6389700 1 --2- 172.21.15.112:0/1772817389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f0152130 0x7fb6f015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.261+0000 7fb6f6389700 1 -- 172.21.15.112:0/1772817389 >> 172.21.15.112:0/1772817389 conn(0x7fb6f0066010 msgr2=0x7fb6f00663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:53.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.261+0000 7fb6f6389700 1 -- 172.21.15.112:0/1772817389 shutdown_connections 2023-08-27T11:27:53.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.261+0000 7fb6f6389700 1 -- 172.21.15.112:0/1772817389 wait complete. 2023-08-27T11:27:53.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.262+0000 7fb6f6389700 1 Processor -- start 2023-08-27T11:27:53.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.262+0000 7fb6f6389700 1 -- start start 2023-08-27T11:27:53.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f6389700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00b3cf0 0x7fb6f0139680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f6389700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f0152130 0x7fb6f013c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f6389700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f015cd60 0x7fb6f014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f6389700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f00b27c0 con 0x7fb6f015cd60 2023-08-27T11:27:53.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f6389700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f0139bc0 con 0x7fb6f00b3cf0 2023-08-27T11:27:53.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f6389700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f01f4a20 con 0x7fb6f0152130 2023-08-27T11:27:53.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f4926700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f015cd60 0x7fb6f014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f4926700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f015cd60 0x7fb6f014eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35342/0 (socket says 172.21.15.112:35342) 2023-08-27T11:27:53.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f4926700 1 -- 172.21.15.112:0/1878340871 learned_addr learned my addr 172.21.15.112:0/1878340871 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:53.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6effff700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00b3cf0 0x7fb6f0139680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6ef7fe700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f0152130 0x7fb6f013c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f4926700 1 -- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f0152130 msgr2=0x7fb6f013c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f4926700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f0152130 0x7fb6f013c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f4926700 1 -- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00b3cf0 msgr2=0x7fb6f0139680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f4926700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00b3cf0 0x7fb6f0139680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f4926700 1 -- 172.21.15.112:0/1878340871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e8009b30 con 0x7fb6f015cd60 2023-08-27T11:27:53.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f4926700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f015cd60 0x7fb6f014eae0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800b730 tx=0x7fb6e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6ecff9700 1 -- 172.21.15.112:0/1878340871 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8004830 con 0x7fb6f015cd60 2023-08-27T11:27:53.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6ecff9700 1 -- 172.21.15.112:0/1878340871 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e80092e0 con 0x7fb6f015cd60 2023-08-27T11:27:53.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f6389700 1 -- 172.21.15.112:0/1878340871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f013cda0 con 0x7fb6f015cd60 2023-08-27T11:27:53.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6ecff9700 1 -- 172.21.15.112:0/1878340871 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e801c680 con 0x7fb6f015cd60 2023-08-27T11:27:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.263+0000 7fb6f6389700 1 -- 172.21.15.112:0/1878340871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f01f4080 con 0x7fb6f015cd60 2023-08-27T11:27:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.264+0000 7fb6f6389700 1 -- 172.21.15.112:0/1878340871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f0067a20 con 0x7fb6f015cd60 2023-08-27T11:27:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.265+0000 7fb6ecff9700 1 -- 172.21.15.112:0/1878340871 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fb6e801c7e0 con 0x7fb6f015cd60 2023-08-27T11:27:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.265+0000 7fb6ecff9700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6dc06a4d0 0x7fb6dc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.265+0000 7fb6ecff9700 1 -- 172.21.15.112:0/1878340871 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fb6e808aa60 con 0x7fb6f015cd60 2023-08-27T11:27:53.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.265+0000 7fb6effff700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6dc06a4d0 0x7fb6dc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.266+0000 7fb6effff700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6dc06a4d0 0x7fb6dc06c990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4006fd0 tx=0x7fb6e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.269+0000 7fb6ecff9700 1 -- 172.21.15.112:0/1878340871 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb6e8055a40 con 0x7fb6f015cd60 2023-08-27T11:27:53.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.455+0000 7fb6f6389700 1 -- 172.21.15.112:0/1878340871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb6f015d530 con 0x7fb6f015cd60 2023-08-27T11:27:53.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.455+0000 7fb6ecff9700 1 -- 172.21.15.112:0/1878340871 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v127) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb6e8059060 con 0x7fb6f015cd60 2023-08-27T11:27:53.457 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:27:53.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.457+0000 7fb6f6389700 1 -- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6dc06a4d0 msgr2=0x7fb6dc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.457+0000 7fb6f6389700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6dc06a4d0 0x7fb6dc06c990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4006fd0 tx=0x7fb6e4008040 comp rx=0 tx=0).stop 2023-08-27T11:27:53.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.457+0000 7fb6f6389700 1 -- 172.21.15.112:0/1878340871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f015cd60 msgr2=0x7fb6f014eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.457+0000 7fb6f6389700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f015cd60 0x7fb6f014eae0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800b730 tx=0x7fb6e8004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.457+0000 7fb6f6389700 1 -- 172.21.15.112:0/1878340871 shutdown_connections 2023-08-27T11:27:53.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.457+0000 7fb6f6389700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb6dc06a4d0 0x7fb6dc06c990 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.457+0000 7fb6f6389700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f015cd60 0x7fb6f014eae0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.457+0000 7fb6f6389700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00b3cf0 0x7fb6f0139680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.457+0000 7fb6f6389700 1 --2- 172.21.15.112:0/1878340871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f0152130 0x7fb6f013c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.457+0000 7fb6f6389700 1 -- 172.21.15.112:0/1878340871 >> 172.21.15.112:0/1878340871 conn(0x7fb6f0066010 msgr2=0x7fb6f015de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:53.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.457+0000 7fb6f6389700 1 -- 172.21.15.112:0/1878340871 shutdown_connections 2023-08-27T11:27:53.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.457+0000 7fb6f6389700 1 -- 172.21.15.112:0/1878340871 wait complete. 2023-08-27T11:27:53.470 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:53.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.596+0000 7f0eafd5e700 1 Processor -- start 2023-08-27T11:27:53.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.596+0000 7f0eafd5e700 1 -- start start 2023-08-27T11:27:53.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eafd5e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0ea815b7f0 0x7f0ea8072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eafd5e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea815ca50 con 0x7f0ea806e4c0 2023-08-27T11:27:53.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eafd5e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea815be20 con 0x7f0ea8158ed0 2023-08-27T11:27:53.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eafd5e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea815bc20 con 0x7f0ea815b7f0 2023-08-27T11:27:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0ead2f9700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0ea806e4c0 0x7f0ea815cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57960/0 (socket says 172.21.15.112:57960) 2023-08-27T11:27:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0ead2f9700 1 -- 172.21.15.112:0/3730637777 learned_addr learned my addr 172.21.15.112:0/3730637777 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eae2fb700 1 --2- 172.21.15.112:0/3730637777 >> v2:172.21.15.112:3300/0 conn(0x7f0ea815b7f0 0x7f0ea8072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eacaf8700 1 -- 172.21.15.112:0/3730637777 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861193522 0 0) 0x7f0ea815ca50 con 0x7f0ea806e4c0 2023-08-27T11:27:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eacaf8700 1 -- 172.21.15.112:0/3730637777 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e940035b0 con 0x7f0ea806e4c0 2023-08-27T11:27:53.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eae2fb700 1 -- 172.21.15.112:0/3730637777 >> v1:172.21.15.138:6789/0 conn(0x7f0ea8158ed0 legacy=0x7f0ea806ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eae2fb700 1 -- 172.21.15.112:0/3730637777 >> v1:172.21.15.112:6789/0 conn(0x7f0ea806e4c0 legacy=0x7f0ea815cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eae2fb700 1 -- 172.21.15.112:0/3730637777 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea815c100 con 0x7f0ea815b7f0 2023-08-27T11:27:53.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eae2fb700 1 --2- 172.21.15.112:0/3730637777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea815b7f0 0x7f0ea8072000 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009d90 tx=0x7f0ea000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06a1a2eaa151545 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eacaf8700 1 -- 172.21.15.112:0/3730637777 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea000e070 con 0x7f0ea815b7f0 2023-08-27T11:27:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eacaf8700 1 -- 172.21.15.112:0/3730637777 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea000bb20 con 0x7f0ea815b7f0 2023-08-27T11:27:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.597+0000 7f0eacaf8700 1 -- 172.21.15.112:0/3730637777 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0012430 con 0x7f0ea815b7f0 2023-08-27T11:27:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.598+0000 7f0eafd5e700 1 -- 172.21.15.112:0/3730637777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea815b7f0 msgr2=0x7f0ea8072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.598+0000 7f0eafd5e700 1 --2- 172.21.15.112:0/3730637777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea815b7f0 0x7f0ea8072000 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009d90 tx=0x7f0ea000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.598+0000 7f0eafd5e700 1 -- 172.21.15.112:0/3730637777 shutdown_connections 2023-08-27T11:27:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.598+0000 7f0eafd5e700 1 --2- 172.21.15.112:0/3730637777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea815b7f0 0x7f0ea8072000 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.598+0000 7f0eafd5e700 1 -- 172.21.15.112:0/3730637777 >> 172.21.15.112:0/3730637777 conn(0x7f0ea80cc240 msgr2=0x7f0ea80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.598+0000 7f0eafd5e700 1 -- 172.21.15.112:0/3730637777 shutdown_connections 2023-08-27T11:27:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.598+0000 7f0eafd5e700 1 -- 172.21.15.112:0/3730637777 wait complete. 2023-08-27T11:27:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.599+0000 7f0eafd5e700 1 Processor -- start 2023-08-27T11:27:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.599+0000 7f0eafd5e700 1 -- start start 2023-08-27T11:27:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eafd5e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ea806e4c0 0x7f0ea81f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eafd5e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea8158ed0 0x7f0ea8071070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eafd5e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea815b7f0 0x7f0ea8157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eafd5e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea813f3b0 con 0x7f0ea815b7f0 2023-08-27T11:27:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eafd5e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea806fb00 con 0x7f0ea806e4c0 2023-08-27T11:27:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eafd5e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea806fde0 con 0x7f0ea8158ed0 2023-08-27T11:27:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eae2fb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea815b7f0 0x7f0ea8157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eae2fb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea815b7f0 0x7f0ea8157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35356/0 (socket says 172.21.15.112:35356) 2023-08-27T11:27:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eae2fb700 1 -- 172.21.15.112:0/2515538670 learned_addr learned my addr 172.21.15.112:0/2515538670 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eadafa700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ea806e4c0 0x7f0ea81f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eae2fb700 1 -- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea8158ed0 msgr2=0x7f0ea8071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0ead2f9700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea8158ed0 0x7f0ea8071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eae2fb700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea8158ed0 0x7f0ea8071070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eae2fb700 1 -- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ea806e4c0 msgr2=0x7f0ea81f1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eae2fb700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ea806e4c0 0x7f0ea81f1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eae2fb700 1 -- 172.21.15.112:0/2515538670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea0009a70 con 0x7f0ea815b7f0 2023-08-27T11:27:53.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.600+0000 7f0eadafa700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ea806e4c0 0x7f0ea81f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:53.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.601+0000 7f0eae2fb700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea815b7f0 0x7f0ea8157e20 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f0ea806ac50 tx=0x7f0ea0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.601+0000 7f0e9e7fc700 1 -- 172.21.15.112:0/2515538670 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0012430 con 0x7f0ea815b7f0 2023-08-27T11:27:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.601+0000 7f0e9e7fc700 1 -- 172.21.15.112:0/2515538670 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea0020070 con 0x7f0ea815b7f0 2023-08-27T11:27:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.601+0000 7f0e9e7fc700 1 -- 172.21.15.112:0/2515538670 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0004280 con 0x7f0ea815b7f0 2023-08-27T11:27:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.601+0000 7f0eafd5e700 1 -- 172.21.15.112:0/2515538670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea80700c0 con 0x7f0ea815b7f0 2023-08-27T11:27:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.601+0000 7f0eafd5e700 1 -- 172.21.15.112:0/2515538670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea8070680 con 0x7f0ea815b7f0 2023-08-27T11:27:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.602+0000 7f0eafd5e700 1 -- 172.21.15.112:0/2515538670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea8067e90 con 0x7f0ea815b7f0 2023-08-27T11:27:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.602+0000 7f0e9e7fc700 1 -- 172.21.15.112:0/2515538670 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f0ea00043e0 con 0x7f0ea815b7f0 2023-08-27T11:27:53.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.602+0000 7f0e9e7fc700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e8c06a4d0 0x7f0e8c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.603+0000 7f0e9e7fc700 1 -- 172.21.15.112:0/2515538670 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f0ea0022070 con 0x7f0ea815b7f0 2023-08-27T11:27:53.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.603+0000 7f0eadafa700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e8c06a4d0 0x7f0e8c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.603+0000 7f0eadafa700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e8c06a4d0 0x7f0e8c06c990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4000f60 tx=0x7f0ea4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.607+0000 7f0e9e7fc700 1 -- 172.21.15.112:0/2515538670 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0ea0055b40 con 0x7f0ea815b7f0 2023-08-27T11:27:53.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.795+0000 7f0eafd5e700 1 -- 172.21.15.112:0/2515538670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ea813d230 con 0x7f0ea815b7f0 2023-08-27T11:27:53.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.795+0000 7f0e9e7fc700 1 -- 172.21.15.112:0/2515538670 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ea002c020 con 0x7f0ea815b7f0 2023-08-27T11:27:53.797 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:27:53.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.797+0000 7f0eafd5e700 1 -- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e8c06a4d0 msgr2=0x7f0e8c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.797+0000 7f0eafd5e700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e8c06a4d0 0x7f0e8c06c990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4000f60 tx=0x7f0ea4009040 comp rx=0 tx=0).stop 2023-08-27T11:27:53.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.797+0000 7f0eafd5e700 1 -- 172.21.15.112:0/2515538670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea815b7f0 msgr2=0x7f0ea8157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.797+0000 7f0eafd5e700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea815b7f0 0x7f0ea8157e20 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f0ea806ac50 tx=0x7f0ea0004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.797+0000 7f0eafd5e700 1 -- 172.21.15.112:0/2515538670 shutdown_connections 2023-08-27T11:27:53.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.797+0000 7f0eafd5e700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e8c06a4d0 0x7f0e8c06c990 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.797+0000 7f0eafd5e700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ea815b7f0 0x7f0ea8157e20 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.797+0000 7f0eafd5e700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ea806e4c0 0x7f0ea81f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.797+0000 7f0eafd5e700 1 --2- 172.21.15.112:0/2515538670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ea8158ed0 0x7f0ea8071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.797+0000 7f0eafd5e700 1 -- 172.21.15.112:0/2515538670 >> 172.21.15.112:0/2515538670 conn(0x7f0ea80cc240 msgr2=0x7f0ea80cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:53.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.797+0000 7f0eafd5e700 1 -- 172.21.15.112:0/2515538670 shutdown_connections 2023-08-27T11:27:53.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.797+0000 7f0eafd5e700 1 -- 172.21.15.112:0/2515538670 wait complete. 2023-08-27T11:27:53.811 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:27:53.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.932+0000 7f8f68af2700 1 Processor -- start 2023-08-27T11:27:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.932+0000 7f8f68af2700 1 -- start start 2023-08-27T11:27:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.933+0000 7f8f68af2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8f640b1cf0 0x7f8f6415c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.933+0000 7f8f68af2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f640b20d0 0x7f8f640719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.933+0000 7f8f68af2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6406ccc0 con 0x7f8f6415ee70 2023-08-27T11:27:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.933+0000 7f8f68af2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6415c840 con 0x7f8f640b1cf0 2023-08-27T11:27:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.933+0000 7f8f68af2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640aab00 con 0x7f8f640b20d0 2023-08-27T11:27:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.933+0000 7f8f61d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8f6415ee70 0x7f8f640a8860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57974/0 (socket says 172.21.15.112:57974) 2023-08-27T11:27:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.933+0000 7f8f61d9b700 1 -- 172.21.15.112:0/283998175 learned_addr learned my addr 172.21.15.112:0/283998175 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.933+0000 7f8f6259c700 1 --2- 172.21.15.112:0/283998175 >> v2:172.21.15.138:3300/0 conn(0x7f8f640b1cf0 0x7f8f6415c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.933+0000 7f8f62d9d700 1 --2- 172.21.15.112:0/283998175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f640b20d0 0x7f8f640719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.933+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594648490 0 0) 0x7f8f6406ccc0 con 0x7f8f6415ee70 2023-08-27T11:27:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.933+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f4c0035b0 con 0x7f8f6415ee70 2023-08-27T11:27:53.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1477030429 0 0) 0x7f8f4c0035b0 con 0x7f8f6415ee70 2023-08-27T11:27:53.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f640b20d0 msgr2=0x7f8f640719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:53.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 --2- 172.21.15.112:0/283998175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f640b20d0 0x7f8f640719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 >> v2:172.21.15.138:3300/0 conn(0x7f8f640b1cf0 msgr2=0x7f8f6415c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:53.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 --2- 172.21.15.112:0/283998175 >> v2:172.21.15.138:3300/0 conn(0x7f8f640b1cf0 0x7f8f6415c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6415f350 con 0x7f8f6415ee70 2023-08-27T11:27:53.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f8f54001670 con 0x7f8f6415ee70 2023-08-27T11:27:53.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 >> v1:172.21.15.112:6789/0 conn(0x7f8f6415ee70 legacy=0x7f8f640a8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 --2- 172.21.15.112:0/283998175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f640b20d0 0x7f8f4c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 --2- 172.21.15.112:0/283998175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c003f80 0x7f8f4c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 --2- 172.21.15.112:0/283998175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4c004d90 0x7f8f4c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6415c840 con 0x7f8f4c003f80 2023-08-27T11:27:53.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640aab00 con 0x7f8f4c004d90 2023-08-27T11:27:53.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c00ba10 con 0x7f8f640b20d0 2023-08-27T11:27:53.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f61d9b700 1 --2- 172.21.15.112:0/283998175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c003f80 0x7f8f4c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6259c700 1 --2- 172.21.15.112:0/283998175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f640b20d0 0x7f8f4c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f62d9d700 1 --2- 172.21.15.112:0/283998175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4c004d90 0x7f8f4c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f61d9b700 1 -- 172.21.15.112:0/283998175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f640b20d0 msgr2=0x7f8f4c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:53.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f61d9b700 1 --2- 172.21.15.112:0/283998175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f640b20d0 0x7f8f4c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f61d9b700 1 -- 172.21.15.112:0/283998175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4c004d90 msgr2=0x7f8f4c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:53.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f61d9b700 1 --2- 172.21.15.112:0/283998175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4c004d90 0x7f8f4c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f61d9b700 1 -- 172.21.15.112:0/283998175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c0035b0 con 0x7f8f4c003f80 2023-08-27T11:27:53.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f62d9d700 1 --2- 172.21.15.112:0/283998175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4c004d90 0x7f8f4c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:53.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f61d9b700 1 --2- 172.21.15.112:0/283998175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c003f80 0x7f8f4c004850 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400c320 tx=0x7f8f4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d60e5e013a9d53 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5400ce00 con 0x7f8f4c003f80 2023-08-27T11:27:53.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f54005960 con 0x7f8f4c003f80 2023-08-27T11:27:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.934+0000 7f8f6159a700 1 -- 172.21.15.112:0/283998175 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54005d10 con 0x7f8f4c003f80 2023-08-27T11:27:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.935+0000 7f8f68af2700 1 -- 172.21.15.112:0/283998175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c003f80 msgr2=0x7f8f4c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.935+0000 7f8f68af2700 1 --2- 172.21.15.112:0/283998175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c003f80 0x7f8f4c004850 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400c320 tx=0x7f8f4c005880 comp rx=0 tx=0).stop 2023-08-27T11:27:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.935+0000 7f8f62d9d700 1 -- 172.21.15.112:0/283998175 reap_dead start 2023-08-27T11:27:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.935+0000 7f8f68af2700 1 -- 172.21.15.112:0/283998175 shutdown_connections 2023-08-27T11:27:53.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.935+0000 7f8f68af2700 1 -- 172.21.15.112:0/283998175 >> 172.21.15.112:0/283998175 conn(0x7f8f640cfae0 msgr2=0x7f8f640cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:53.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.935+0000 7f8f68af2700 1 -- 172.21.15.112:0/283998175 shutdown_connections 2023-08-27T11:27:53.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.935+0000 7f8f68af2700 1 -- 172.21.15.112:0/283998175 wait complete. 2023-08-27T11:27:53.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.937+0000 7f8f68af2700 1 Processor -- start 2023-08-27T11:27:53.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.937+0000 7f8f68af2700 1 -- start start 2023-08-27T11:27:53.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f68af2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4c003f80 0x7f8f64154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f68af2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6413cb10 0x7f8f6415e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f68af2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f641f0850 0x7f8f641f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f68af2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f64137590 con 0x7f8f6413cb10 2023-08-27T11:27:53.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f68af2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6415e700 con 0x7f8f641f0850 2023-08-27T11:27:53.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f68af2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f64136800 con 0x7f8f4c003f80 2023-08-27T11:27:53.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f61d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6413cb10 0x7f8f6415e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f61d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6413cb10 0x7f8f6415e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35370/0 (socket says 172.21.15.112:35370) 2023-08-27T11:27:53.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f61d9b700 1 -- 172.21.15.112:0/26770074 learned_addr learned my addr 172.21.15.112:0/26770074 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:53.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f6259c700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4c003f80 0x7f8f64154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f62d9d700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f641f0850 0x7f8f641f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f61d9b700 1 -- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4c003f80 msgr2=0x7f8f64154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f61d9b700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4c003f80 0x7f8f64154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f61d9b700 1 -- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f641f0850 msgr2=0x7f8f641f0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f61d9b700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f641f0850 0x7f8f641f0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f61d9b700 1 -- 172.21.15.112:0/26770074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f64136ae0 con 0x7f8f6413cb10 2023-08-27T11:27:53.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f61d9b700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6413cb10 0x7f8f6415e1c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f54001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f52ffd700 1 -- 172.21.15.112:0/26770074 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5400ce50 con 0x7f8f6413cb10 2023-08-27T11:27:53.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f52ffd700 1 -- 172.21.15.112:0/26770074 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f54004a80 con 0x7f8f6413cb10 2023-08-27T11:27:53.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f68af2700 1 -- 172.21.15.112:0/26770074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f64137220 con 0x7f8f6413cb10 2023-08-27T11:27:53.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f52ffd700 1 -- 172.21.15.112:0/26770074 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54004e30 con 0x7f8f6413cb10 2023-08-27T11:27:53.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.938+0000 7f8f68af2700 1 -- 172.21.15.112:0/26770074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6413b830 con 0x7f8f6413cb10 2023-08-27T11:27:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.940+0000 7f8f68af2700 1 -- 172.21.15.112:0/26770074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f64067a20 con 0x7f8f6413cb10 2023-08-27T11:27:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.940+0000 7f8f52ffd700 1 -- 172.21.15.112:0/26770074 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f8f540070d0 con 0x7f8f6413cb10 2023-08-27T11:27:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.940+0000 7f8f52ffd700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f4406a4d0 0x7f8f4406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.940+0000 7f8f52ffd700 1 -- 172.21.15.112:0/26770074 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f8f5408a160 con 0x7f8f6413cb10 2023-08-27T11:27:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.940+0000 7f8f6259c700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f4406a4d0 0x7f8f4406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.941+0000 7f8f6259c700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f4406a4d0 0x7f8f4406c990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009cc0 tx=0x7f8f580091c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:53.945+0000 7f8f52ffd700 1 -- 172.21.15.112:0/26770074 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8f54055430 con 0x7f8f6413cb10 2023-08-27T11:27:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.134+0000 7f8f68af2700 1 -- 172.21.15.112:0/26770074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f640a8d50 con 0x7f8f6413cb10 2023-08-27T11:27:54.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.134+0000 7f8f52ffd700 1 -- 172.21.15.112:0/26770074 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f54007da0 con 0x7f8f6413cb10 2023-08-27T11:27:54.136 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:27:54.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.136+0000 7f8f68af2700 1 -- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f4406a4d0 msgr2=0x7f8f4406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.136+0000 7f8f68af2700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f4406a4d0 0x7f8f4406c990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009cc0 tx=0x7f8f580091c0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.136+0000 7f8f68af2700 1 -- 172.21.15.112:0/26770074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6413cb10 msgr2=0x7f8f6415e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.136+0000 7f8f68af2700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6413cb10 0x7f8f6415e1c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f54001a00 comp rx=0 tx=0).stop 2023-08-27T11:27:54.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.136+0000 7f8f68af2700 1 -- 172.21.15.112:0/26770074 shutdown_connections 2023-08-27T11:27:54.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.136+0000 7f8f68af2700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f4406a4d0 0x7f8f4406c990 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.136+0000 7f8f68af2700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f641f0850 0x7f8f641f0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.136+0000 7f8f68af2700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4c003f80 0x7f8f64154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.136+0000 7f8f68af2700 1 --2- 172.21.15.112:0/26770074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6413cb10 0x7f8f6415e1c0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.136+0000 7f8f68af2700 1 -- 172.21.15.112:0/26770074 >> 172.21.15.112:0/26770074 conn(0x7f8f640cfae0 msgr2=0x7f8f6415b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:54.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.136+0000 7f8f68af2700 1 -- 172.21.15.112:0/26770074 shutdown_connections 2023-08-27T11:27:54.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.136+0000 7f8f68af2700 1 -- 172.21.15.112:0/26770074 wait complete. 2023-08-27T11:27:54.150 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:27:54.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.271+0000 7f7f7206f700 1 Processor -- start 2023-08-27T11:27:54.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.271+0000 7f7f7206f700 1 -- start start 2023-08-27T11:27:54.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f7206f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7f6c146c20 0x7f7f6c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f7206f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7f6c0b3440 0x7f7f6c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f7206f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6c0b5070 con 0x7f7f6c15e6e0 2023-08-27T11:27:54.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f7206f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f6c147000 con 0x7f7f6c0b3440 2023-08-27T11:27:54.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f7206f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f6c1614a0 con 0x7f7f6c146c20 2023-08-27T11:27:54.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f6affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7f6c0b3440 0x7f7f6c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f6affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7f6c0b3440 0x7f7f6c15e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35378/0 (socket says 172.21.15.112:35378) 2023-08-27T11:27:54.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f6affd700 1 -- 172.21.15.112:0/157889815 learned_addr learned my addr 172.21.15.112:0/157889815 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:54.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f6b7fe700 1 --2- 172.21.15.112:0/157889815 >> v2:172.21.15.138:3300/0 conn(0x7f7f6c146c20 0x7f7f6c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f6affd700 1 -- 172.21.15.112:0/157889815 >> v2:172.21.15.138:3300/0 conn(0x7f7f6c146c20 msgr2=0x7f7f6c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:54.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f6affd700 1 --2- 172.21.15.112:0/157889815 >> v2:172.21.15.138:3300/0 conn(0x7f7f6c146c20 0x7f7f6c160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f6affd700 1 -- 172.21.15.112:0/157889815 >> v1:172.21.15.138:6789/0 conn(0x7f7f6c15e6e0 legacy=0x7f7f6c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f6affd700 1 -- 172.21.15.112:0/157889815 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c161740 con 0x7f7f6c0b3440 2023-08-27T11:27:54.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f6b7fe700 1 --2- 172.21.15.112:0/157889815 >> v2:172.21.15.138:3300/0 conn(0x7f7f6c146c20 0x7f7f6c160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:54.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.272+0000 7f7f6affd700 1 --2- 172.21.15.112:0/157889815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6c0b3440 0x7f7f6c15e1a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0b2d00 tx=0x7f7f5c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad59d03111a563b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:54.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.273+0000 7f7f6a7fc700 1 -- 172.21.15.112:0/157889815 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00b870 con 0x7f7f6c0b3440 2023-08-27T11:27:54.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.273+0000 7f7f6a7fc700 1 -- 172.21.15.112:0/157889815 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c00b9d0 con 0x7f7f6c0b3440 2023-08-27T11:27:54.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.273+0000 7f7f6a7fc700 1 -- 172.21.15.112:0/157889815 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00bd60 con 0x7f7f6c0b3440 2023-08-27T11:27:54.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.273+0000 7f7f7206f700 1 -- 172.21.15.112:0/157889815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6c0b3440 msgr2=0x7f7f6c15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.273+0000 7f7f7206f700 1 --2- 172.21.15.112:0/157889815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6c0b3440 0x7f7f6c15e1a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0b2d00 tx=0x7f7f5c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.273+0000 7f7f7206f700 1 -- 172.21.15.112:0/157889815 shutdown_connections 2023-08-27T11:27:54.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.273+0000 7f7f7206f700 1 --2- 172.21.15.112:0/157889815 >> v2:172.21.15.138:3300/0 conn(0x7f7f6c146c20 0x7f7f6c160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.273+0000 7f7f7206f700 1 --2- 172.21.15.112:0/157889815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6c0b3440 0x7f7f6c15e1a0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.273+0000 7f7f7206f700 1 -- 172.21.15.112:0/157889815 >> 172.21.15.112:0/157889815 conn(0x7f7f6c0b1e90 msgr2=0x7f7f6c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:54.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.273+0000 7f7f7206f700 1 -- 172.21.15.112:0/157889815 shutdown_connections 2023-08-27T11:27:54.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.273+0000 7f7f7206f700 1 -- 172.21.15.112:0/157889815 wait complete. 2023-08-27T11:27:54.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.274+0000 7f7f7206f700 1 Processor -- start 2023-08-27T11:27:54.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.275+0000 7f7f7206f700 1 -- start start 2023-08-27T11:27:54.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.275+0000 7f7f7206f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6c0b3440 0x7f7f6c0c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.275+0000 7f7f7206f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f6c146c20 0x7f7f6c145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.275+0000 7f7f7206f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f6c15e6e0 0x7f7f6c1457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.275+0000 7f7f7206f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c1f6f50 con 0x7f7f6c0b3440 2023-08-27T11:27:54.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.275+0000 7f7f7206f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c141120 con 0x7f7f6c146c20 2023-08-27T11:27:54.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.275+0000 7f7f7206f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0c57d0 con 0x7f7f6c15e6e0 2023-08-27T11:27:54.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.275+0000 7f7f6b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6c0b3440 0x7f7f6c0c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.275+0000 7f7f6b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6c0b3440 0x7f7f6c0c50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35382/0 (socket says 172.21.15.112:35382) 2023-08-27T11:27:54.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.275+0000 7f7f6b7fe700 1 -- 172.21.15.112:0/3771759932 learned_addr learned my addr 172.21.15.112:0/3771759932 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:54.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.276+0000 7f7f6affd700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f6c146c20 0x7f7f6c145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.276+0000 7f7f6bfff700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f6c15e6e0 0x7f7f6c1457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.276+0000 7f7f6b7fe700 1 -- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f6c15e6e0 msgr2=0x7f7f6c1457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.276+0000 7f7f6b7fe700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f6c15e6e0 0x7f7f6c1457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.276+0000 7f7f6b7fe700 1 -- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f6c146c20 msgr2=0x7f7f6c145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.276+0000 7f7f6b7fe700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f6c146c20 0x7f7f6c145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.276+0000 7f7f6b7fe700 1 -- 172.21.15.112:0/3771759932 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c009550 con 0x7f7f6c0b3440 2023-08-27T11:27:54.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.276+0000 7f7f6b7fe700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6c0b3440 0x7f7f6c0c50f0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400e810 tx=0x7f7f5400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:54.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.276+0000 7f7f63fff700 1 -- 172.21.15.112:0/3771759932 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54004230 con 0x7f7f6c0b3440 2023-08-27T11:27:54.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.276+0000 7f7f63fff700 1 -- 172.21.15.112:0/3771759932 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f54004390 con 0x7f7f6c0b3440 2023-08-27T11:27:54.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.276+0000 7f7f7206f700 1 -- 172.21.15.112:0/3771759932 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c1f6190 con 0x7f7f6c0b3440 2023-08-27T11:27:54.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.276+0000 7f7f63fff700 1 -- 172.21.15.112:0/3771759932 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f540036a0 con 0x7f7f6c0b3440 2023-08-27T11:27:54.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.276+0000 7f7f7206f700 1 -- 172.21.15.112:0/3771759932 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c1f6750 con 0x7f7f6c0b3440 2023-08-27T11:27:54.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.277+0000 7f7f7206f700 1 -- 172.21.15.112:0/3771759932 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6c067e90 con 0x7f7f6c0b3440 2023-08-27T11:27:54.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.277+0000 7f7f63fff700 1 -- 172.21.15.112:0/3771759932 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f7f54003800 con 0x7f7f6c0b3440 2023-08-27T11:27:54.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.278+0000 7f7f63fff700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f4c06a4d0 0x7f7f4c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.278+0000 7f7f63fff700 1 -- 172.21.15.112:0/3771759932 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f7f54088920 con 0x7f7f6c0b3440 2023-08-27T11:27:54.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.278+0000 7f7f6affd700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f4c06a4d0 0x7f7f4c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.278+0000 7f7f6affd700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f4c06a4d0 0x7f7f4c06c990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0b2d00 tx=0x7f7f5c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:54.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.282+0000 7f7f63fff700 1 -- 172.21.15.112:0/3771759932 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7f54053890 con 0x7f7f6c0b3440 2023-08-27T11:27:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.473+0000 7f7f7206f700 1 -- 172.21.15.112:0/3771759932 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f6c067a20 con 0x7f7f6c0b3440 2023-08-27T11:27:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.473+0000 7f7f63fff700 1 -- 172.21.15.112:0/3771759932 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f540200a0 con 0x7f7f6c0b3440 2023-08-27T11:27:54.475 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:27:54.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.475+0000 7f7f7206f700 1 -- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f4c06a4d0 msgr2=0x7f7f4c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.475+0000 7f7f7206f700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f4c06a4d0 0x7f7f4c06c990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0b2d00 tx=0x7f7f5c008030 comp rx=0 tx=0).stop 2023-08-27T11:27:54.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.475+0000 7f7f7206f700 1 -- 172.21.15.112:0/3771759932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6c0b3440 msgr2=0x7f7f6c0c50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.475+0000 7f7f7206f700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6c0b3440 0x7f7f6c0c50f0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400e810 tx=0x7f7f5400ebd0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.475+0000 7f7f7206f700 1 -- 172.21.15.112:0/3771759932 shutdown_connections 2023-08-27T11:27:54.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.475+0000 7f7f7206f700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7f4c06a4d0 0x7f7f4c06c990 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.475+0000 7f7f7206f700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f6c15e6e0 0x7f7f6c1457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.475+0000 7f7f7206f700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6c0b3440 0x7f7f6c0c50f0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.475+0000 7f7f7206f700 1 --2- 172.21.15.112:0/3771759932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f6c146c20 0x7f7f6c145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.475+0000 7f7f7206f700 1 -- 172.21.15.112:0/3771759932 >> 172.21.15.112:0/3771759932 conn(0x7f7f6c0b1e90 msgr2=0x7f7f6c06b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:54.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.475+0000 7f7f7206f700 1 -- 172.21.15.112:0/3771759932 shutdown_connections 2023-08-27T11:27:54.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.475+0000 7f7f7206f700 1 -- 172.21.15.112:0/3771759932 wait complete. 2023-08-27T11:27:54.489 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:27:54.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.611+0000 7f4f63a99700 1 Processor -- start 2023-08-27T11:27:54.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.611+0000 7f4f63a99700 1 -- start start 2023-08-27T11:27:54.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.612+0000 7f4f63a99700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4f5c143a60 0x7f4f5c0cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.612+0000 7f4f63a99700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f5c0b4960 0x7f4f5c15db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.612+0000 7f4f63a99700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f5c143e40 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.612+0000 7f4f63a99700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f5c0b5340 con 0x7f4f5c143a60 2023-08-27T11:27:54.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.612+0000 7f4f63a99700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c0ce9a0 con 0x7f4f5c0b4960 2023-08-27T11:27:54.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.612+0000 7f4f61835700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4f5c143a60 0x7f4f5c0cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.612+0000 7f4f62036700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4f5c0b4dc0 0x7f4f5c15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58006/0 (socket says 172.21.15.112:58006) 2023-08-27T11:27:54.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.612+0000 7f4f62036700 1 -- 172.21.15.112:0/24558635 learned_addr learned my addr 172.21.15.112:0/24558635 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:54.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f61034700 1 --2- 172.21.15.112:0/24558635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f5c0b4960 0x7f4f5c15db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f61835700 1 -- 172.21.15.112:0/24558635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f5c0b4960 msgr2=0x7f4f5c15db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:54.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f61835700 1 --2- 172.21.15.112:0/24558635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f5c0b4960 0x7f4f5c15db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f61835700 1 -- 172.21.15.112:0/24558635 >> v1:172.21.15.112:6789/0 conn(0x7f4f5c0b4dc0 legacy=0x7f4f5c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f61835700 1 -- 172.21.15.112:0/24558635 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c0cec80 con 0x7f4f5c143a60 2023-08-27T11:27:54.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f60833700 1 -- 172.21.15.112:0/24558635 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138465671 0 0) 0x7f4f5c143e40 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f61034700 1 --2- 172.21.15.112:0/24558635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f5c0b4960 0x7f4f5c15db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:54.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f61835700 1 --2- 172.21.15.112:0/24558635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c143a60 0x7f4f5c0cf760 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009820 tx=0x7f4f58009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c10e2daa0f8b748 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:54.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f60833700 1 -- 172.21.15.112:0/24558635 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58005e30 con 0x7f4f5c143a60 2023-08-27T11:27:54.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f60833700 1 -- 172.21.15.112:0/24558635 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f58006030 con 0x7f4f5c143a60 2023-08-27T11:27:54.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f60833700 1 -- 172.21.15.112:0/24558635 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f580063e0 con 0x7f4f5c143a60 2023-08-27T11:27:54.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f63a99700 1 -- 172.21.15.112:0/24558635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c143a60 msgr2=0x7f4f5c0cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f63a99700 1 --2- 172.21.15.112:0/24558635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c143a60 0x7f4f5c0cf760 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009820 tx=0x7f4f58009b30 comp rx=0 tx=0).stop 2023-08-27T11:27:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f63a99700 1 -- 172.21.15.112:0/24558635 shutdown_connections 2023-08-27T11:27:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f63a99700 1 --2- 172.21.15.112:0/24558635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c143a60 0x7f4f5c0cf760 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f63a99700 1 --2- 172.21.15.112:0/24558635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f5c0b4960 0x7f4f5c15db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.613+0000 7f4f63a99700 1 -- 172.21.15.112:0/24558635 >> 172.21.15.112:0/24558635 conn(0x7f4f5c06adb0 msgr2=0x7f4f5c06b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.614+0000 7f4f63a99700 1 -- 172.21.15.112:0/24558635 shutdown_connections 2023-08-27T11:27:54.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.614+0000 7f4f63a99700 1 -- 172.21.15.112:0/24558635 wait complete. 2023-08-27T11:27:54.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.615+0000 7f4f63a99700 1 Processor -- start 2023-08-27T11:27:54.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.615+0000 7f4f63a99700 1 -- start start 2023-08-27T11:27:54.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f63a99700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f5c0b4960 0x7f4f5c13efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f63a99700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c0b4dc0 0x7f4f5c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f63a99700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f5c143a60 0x7f4f5c15f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f63a99700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c071fe0 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f63a99700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c13e230 con 0x7f4f5c0b4960 2023-08-27T11:27:54.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f63a99700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c13e510 con 0x7f4f5c143a60 2023-08-27T11:27:54.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f61034700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c0b4dc0 0x7f4f5c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f61034700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c0b4dc0 0x7f4f5c15ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35396/0 (socket says 172.21.15.112:35396) 2023-08-27T11:27:54.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f61034700 1 -- 172.21.15.112:0/1671200308 learned_addr learned my addr 172.21.15.112:0/1671200308 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:54.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f61835700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f5c0b4960 0x7f4f5c13efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f62036700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f5c143a60 0x7f4f5c15f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f61034700 1 -- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f5c143a60 msgr2=0x7f4f5c15f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f61034700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f5c143a60 0x7f4f5c15f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f61034700 1 -- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f5c0b4960 msgr2=0x7f4f5c13efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f61034700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f5c0b4960 0x7f4f5c13efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f61034700 1 -- 172.21.15.112:0/1671200308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f58009430 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.616+0000 7f4f61835700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f5c0b4960 0x7f4f5c13efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:54.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.617+0000 7f4f61034700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c0b4dc0 0x7f4f5c15ec90 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c7f0 tx=0x7f4f4c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:54.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.617+0000 7f4f527fc700 1 -- 172.21.15.112:0/1671200308 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c005c20 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.617+0000 7f4f527fc700 1 -- 172.21.15.112:0/1671200308 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c005d80 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.617+0000 7f4f63a99700 1 -- 172.21.15.112:0/1671200308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5c13e7f0 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.617+0000 7f4f527fc700 1 -- 172.21.15.112:0/1671200308 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c011820 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.617+0000 7f4f63a99700 1 -- 172.21.15.112:0/1671200308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5c138df0 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.618+0000 7f4f63a99700 1 -- 172.21.15.112:0/1671200308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f5c067a20 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.618+0000 7f4f527fc700 1 -- 172.21.15.112:0/1671200308 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4f4c011980 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.618+0000 7f4f527fc700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4f4006a4d0 0x7f4f4006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.619+0000 7f4f527fc700 1 -- 172.21.15.112:0/1671200308 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f4f4c088cb0 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.619+0000 7f4f61835700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4f4006a4d0 0x7f4f4006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.619+0000 7f4f61835700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4f4006a4d0 0x7f4f4006c990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4f58013040 tx=0x7f4f58008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:54.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.623+0000 7f4f527fc700 1 -- 172.21.15.112:0/1671200308 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4f4c057600 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.826+0000 7f4f63a99700 1 -- 172.21.15.112:0/1671200308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4f5c165d00 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.827+0000 7f4f527fc700 1 -- 172.21.15.112:0/1671200308 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v127) v1 ==== 80+0+248 (secure 0 0 0) 0x7f4f4c01e070 con 0x7f4f5c0b4dc0 2023-08-27T11:27:54.829 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6832/3832579050 2023-08-28T11:27:49.540827+0000 2023-08-27T11:27:54.829 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6834/3832579050 2023-08-28T11:27:49.540827+0000 2023-08-27T11:27:54.829 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6835/4019485943 2023-08-28T11:27:46.133510+0000 2023-08-27T11:27:54.829 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6833/4019485943 2023-08-28T11:27:46.133510+0000 2023-08-27T11:27:54.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.829+0000 7f4f63a99700 1 -- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4f4006a4d0 msgr2=0x7f4f4006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.829+0000 7f4f63a99700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4f4006a4d0 0x7f4f4006c990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4f58013040 tx=0x7f4f58008070 comp rx=0 tx=0).stop 2023-08-27T11:27:54.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.829+0000 7f4f63a99700 1 -- 172.21.15.112:0/1671200308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c0b4dc0 msgr2=0x7f4f5c15ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.829+0000 7f4f63a99700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c0b4dc0 0x7f4f5c15ec90 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c7f0 tx=0x7f4f4c00cbb0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.829+0000 7f4f63a99700 1 -- 172.21.15.112:0/1671200308 shutdown_connections 2023-08-27T11:27:54.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.829+0000 7f4f63a99700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4f4006a4d0 0x7f4f4006c990 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.829+0000 7f4f63a99700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4f5c143a60 0x7f4f5c15f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.829+0000 7f4f63a99700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f5c0b4960 0x7f4f5c13efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.829+0000 7f4f63a99700 1 --2- 172.21.15.112:0/1671200308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c0b4dc0 0x7f4f5c15ec90 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.829+0000 7f4f63a99700 1 -- 172.21.15.112:0/1671200308 >> 172.21.15.112:0/1671200308 conn(0x7f4f5c06adb0 msgr2=0x7f4f5c15bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:54.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.829+0000 7f4f63a99700 1 -- 172.21.15.112:0/1671200308 shutdown_connections 2023-08-27T11:27:54.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:54.829+0000 7f4f63a99700 1 -- 172.21.15.112:0/1671200308 wait complete. 2023-08-27T11:27:54.833 INFO:teuthology.orchestra.run.smithi112.stderr:listed 4 entries 2023-08-27T11:27:54.842 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:27:54.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:54.843 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:54.883 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:27:54.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:54.884 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:27:54.941 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:27:54.941 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:54.963 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:27:54.963 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:55.024 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:55.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.142+0000 7fe036e6c700 1 Processor -- start 2023-08-27T11:27:55.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.142+0000 7fe036e6c700 1 -- start start 2023-08-27T11:27:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe036e6c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe0300b1400 0x7fe0300cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe036e6c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe0300b17e0 0x7fe03006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe036e6c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe030161850 con 0x7fe030161ce0 2023-08-27T11:27:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe036e6c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03006e7d0 con 0x7fe0300b1400 2023-08-27T11:27:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe036e6c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0300cfc10 con 0x7fe0300b17e0 2023-08-27T11:27:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe034c08700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe0300b1400 0x7fe0300cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe035409700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe030161ce0 0x7fe03006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58018/0 (socket says 172.21.15.112:58018) 2023-08-27T11:27:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe035409700 1 -- 172.21.15.112:0/626933615 learned_addr learned my addr 172.21.15.112:0/626933615 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe02ffff700 1 --2- 172.21.15.112:0/626933615 >> v2:172.21.15.138:3300/0 conn(0x7fe0300b17e0 0x7fe03006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe034c08700 1 -- 172.21.15.112:0/626933615 >> v2:172.21.15.138:3300/0 conn(0x7fe0300b17e0 msgr2=0x7fe03006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:55.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe034c08700 1 --2- 172.21.15.112:0/626933615 >> v2:172.21.15.138:3300/0 conn(0x7fe0300b17e0 0x7fe03006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe034c08700 1 -- 172.21.15.112:0/626933615 >> v1:172.21.15.112:6789/0 conn(0x7fe030161ce0 legacy=0x7fe03006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe034c08700 1 -- 172.21.15.112:0/626933615 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe030161200 con 0x7fe0300b1400 2023-08-27T11:27:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe02f7fe700 1 -- 172.21.15.112:0/626933615 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830323976 0 0) 0x7fe030161850 con 0x7fe030161ce0 2023-08-27T11:27:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe034c08700 1 --2- 172.21.15.112:0/626933615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0300b1400 0x7fe0300cf5a0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fe0300cc420 tx=0x7fe024009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b2fa3f3dcd9959 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe02f7fe700 1 -- 172.21.15.112:0/626933615 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024011830 con 0x7fe0300b1400 2023-08-27T11:27:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe02f7fe700 1 -- 172.21.15.112:0/626933615 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe024011990 con 0x7fe0300b1400 2023-08-27T11:27:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.143+0000 7fe02f7fe700 1 -- 172.21.15.112:0/626933615 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024011d20 con 0x7fe0300b1400 2023-08-27T11:27:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.144+0000 7fe036e6c700 1 -- 172.21.15.112:0/626933615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0300b1400 msgr2=0x7fe0300cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.144+0000 7fe036e6c700 1 --2- 172.21.15.112:0/626933615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0300b1400 0x7fe0300cf5a0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fe0300cc420 tx=0x7fe024009b20 comp rx=0 tx=0).stop 2023-08-27T11:27:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.144+0000 7fe036e6c700 1 -- 172.21.15.112:0/626933615 shutdown_connections 2023-08-27T11:27:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.144+0000 7fe036e6c700 1 --2- 172.21.15.112:0/626933615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0300b1400 0x7fe0300cf5a0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.144+0000 7fe036e6c700 1 --2- 172.21.15.112:0/626933615 >> v2:172.21.15.138:3300/0 conn(0x7fe0300b17e0 0x7fe03006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.144+0000 7fe036e6c700 1 -- 172.21.15.112:0/626933615 >> 172.21.15.112:0/626933615 conn(0x7fe0300d0000 msgr2=0x7fe0300d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.144+0000 7fe036e6c700 1 -- 172.21.15.112:0/626933615 shutdown_connections 2023-08-27T11:27:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.144+0000 7fe036e6c700 1 -- 172.21.15.112:0/626933615 wait complete. 2023-08-27T11:27:55.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.146+0000 7fe036e6c700 1 Processor -- start 2023-08-27T11:27:55.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.146+0000 7fe036e6c700 1 -- start start 2023-08-27T11:27:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe036e6c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0300b1400 0x7fe0300c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe036e6c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0300b17e0 0x7fe030071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe036e6c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe030161ce0 0x7fe0300c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe036e6c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300c6230 con 0x7fe0300b17e0 2023-08-27T11:27:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe036e6c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300c5380 con 0x7fe030161ce0 2023-08-27T11:27:55.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe036e6c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300721e0 con 0x7fe0300b1400 2023-08-27T11:27:55.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe02ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0300b17e0 0x7fe030071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe02ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0300b17e0 0x7fe030071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35408/0 (socket says 172.21.15.112:35408) 2023-08-27T11:27:55.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe02ffff700 1 -- 172.21.15.112:0/4059078996 learned_addr learned my addr 172.21.15.112:0/4059078996 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:55.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe034c08700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0300b1400 0x7fe0300c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe035409700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe030161ce0 0x7fe0300c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe02ffff700 1 -- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0300b1400 msgr2=0x7fe0300c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe02ffff700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0300b1400 0x7fe0300c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe02ffff700 1 -- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe030161ce0 msgr2=0x7fe0300c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe02ffff700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe030161ce0 0x7fe0300c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe02ffff700 1 -- 172.21.15.112:0/4059078996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0240094a0 con 0x7fe0300b17e0 2023-08-27T11:27:55.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe034c08700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0300b1400 0x7fe0300c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:55.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe035409700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe030161ce0 0x7fe0300c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:55.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe02ffff700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0300b17e0 0x7fe030071ca0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00e810 tx=0x7fe01c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:55.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe02d7fa700 1 -- 172.21.15.112:0/4059078996 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c013610 con 0x7fe0300b17e0 2023-08-27T11:27:55.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe02d7fa700 1 -- 172.21.15.112:0/4059078996 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c013770 con 0x7fe0300b17e0 2023-08-27T11:27:55.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe036e6c700 1 -- 172.21.15.112:0/4059078996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0300c55b0 con 0x7fe0300b17e0 2023-08-27T11:27:55.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.147+0000 7fe02d7fa700 1 -- 172.21.15.112:0/4059078996 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c013b00 con 0x7fe0300b17e0 2023-08-27T11:27:55.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.148+0000 7fe036e6c700 1 -- 172.21.15.112:0/4059078996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0300c5b70 con 0x7fe0300b17e0 2023-08-27T11:27:55.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.148+0000 7fe036e6c700 1 -- 172.21.15.112:0/4059078996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0300c6370 con 0x7fe0300b17e0 2023-08-27T11:27:55.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.149+0000 7fe02d7fa700 1 -- 172.21.15.112:0/4059078996 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe01c013c60 con 0x7fe0300b17e0 2023-08-27T11:27:55.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.149+0000 7fe02d7fa700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe01406a500 0x7fe01406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.149+0000 7fe02d7fa700 1 -- 172.21.15.112:0/4059078996 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fe01c088a60 con 0x7fe0300b17e0 2023-08-27T11:27:55.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.149+0000 7fe034c08700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe01406a500 0x7fe01406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.150+0000 7fe034c08700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe01406a500 0x7fe01406c9c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe0240090b0 tx=0x7fe024008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:55.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.154+0000 7fe02d7fa700 1 -- 172.21.15.112:0/4059078996 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe0300c6370 con 0x7fe0300b17e0 2023-08-27T11:27:55.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.344+0000 7fe036e6c700 1 -- 172.21.15.112:0/4059078996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0300c6370 con 0x7fe0300b17e0 2023-08-27T11:27:55.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.346+0000 7fe02d7fa700 1 -- 172.21.15.112:0/4059078996 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+18538 (secure 0 0 0) 0x7fe0300c6370 con 0x7fe0300b17e0 2023-08-27T11:27:55.347 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:55.348 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":127,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:27:52.546502+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:26.212595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:27.068136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6832/3832579050":"2023-08-28T11:27:49.540827+0000","172.21.15.112:6834/3832579050":"2023-08-28T11:27:49.540827+0000","172.21.15.112:6835/4019485943":"2023-08-28T11:27:46.133510+0000","172.21.15.112:6833/4019485943":"2023-08-28T11:27:46.133510+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:55.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.348+0000 7fe036e6c700 1 -- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe01406a500 msgr2=0x7fe01406c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.348+0000 7fe036e6c700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe01406a500 0x7fe01406c9c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe0240090b0 tx=0x7fe024008030 comp rx=0 tx=0).stop 2023-08-27T11:27:55.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.348+0000 7fe036e6c700 1 -- 172.21.15.112:0/4059078996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0300b17e0 msgr2=0x7fe030071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.348+0000 7fe036e6c700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0300b17e0 0x7fe030071ca0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00e810 tx=0x7fe01c00ebd0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.348+0000 7fe036e6c700 1 -- 172.21.15.112:0/4059078996 shutdown_connections 2023-08-27T11:27:55.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.348+0000 7fe036e6c700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe01406a500 0x7fe01406c9c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.348+0000 7fe036e6c700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe030161ce0 0x7fe0300c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.348+0000 7fe036e6c700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0300b1400 0x7fe0300c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.348+0000 7fe036e6c700 1 --2- 172.21.15.112:0/4059078996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0300b17e0 0x7fe030071ca0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.348+0000 7fe036e6c700 1 -- 172.21.15.112:0/4059078996 >> 172.21.15.112:0/4059078996 conn(0x7fe0300d0000 msgr2=0x7fe0300cb040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:55.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.348+0000 7fe036e6c700 1 -- 172.21.15.112:0/4059078996 shutdown_connections 2023-08-27T11:27:55.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.348+0000 7fe036e6c700 1 -- 172.21.15.112:0/4059078996 wait complete. 2023-08-27T11:27:55.362 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:27:26.212595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:27:27.068136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:55.363 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.484+0000 7fe156afc700 1 Processor -- start 2023-08-27T11:27:55.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.484+0000 7fe156afc700 1 -- start start 2023-08-27T11:27:55.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe156afc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1501445b0 0x7fe1500ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe156afc700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1500b08e0 con 0x7fe1500ccad0 2023-08-27T11:27:55.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe156afc700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1500b0b30 con 0x7fe15006fea0 2023-08-27T11:27:55.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe156afc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500ccf50 con 0x7fe1501445b0 2023-08-27T11:27:55.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14ffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe1500ccad0 0x7fe15006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58030/0 (socket says 172.21.15.112:58030) 2023-08-27T11:27:55.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14ffff700 1 -- 172.21.15.112:0/1771164423 learned_addr learned my addr 172.21.15.112:0/1771164423 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:55.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe154898700 1 --2- 172.21.15.112:0/1771164423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1501445b0 0x7fe1500ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495216832 0 0) 0x7fe1500b08e0 con 0x7fe1500ccad0 2023-08-27T11:27:55.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1400035b0 con 0x7fe1500ccad0 2023-08-27T11:27:55.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 980563676 0 0) 0x7fe1400035b0 con 0x7fe1500ccad0 2023-08-27T11:27:55.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1501445b0 msgr2=0x7fe1500ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:55.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 --2- 172.21.15.112:0/1771164423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1501445b0 0x7fe1500ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 >> v1:172.21.15.138:6789/0 conn(0x7fe15006fea0 legacy=0x7fe15015ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1500b0e10 con 0x7fe1500ccad0 2023-08-27T11:27:55.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fe13c001670 con 0x7fe1500ccad0 2023-08-27T11:27:55.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 >> v1:172.21.15.112:6789/0 conn(0x7fe1500ccad0 legacy=0x7fe15006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 --2- 172.21.15.112:0/1771164423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe140003a30 0x7fe140003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 --2- 172.21.15.112:0/1771164423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1400043e0 0x7fe1400048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 --2- 172.21.15.112:0/1771164423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1501445b0 0x7fe140005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500ccf50 con 0x7fe1400043e0 2023-08-27T11:27:55.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14000ba10 con 0x7fe140003a30 2023-08-27T11:27:55.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14000bbd0 con 0x7fe1501445b0 2023-08-27T11:27:55.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.485+0000 7fe14ffff700 1 --2- 172.21.15.112:0/1771164423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1400043e0 0x7fe1400048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.486+0000 7fe154898700 1 --2- 172.21.15.112:0/1771164423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe140003a30 0x7fe140003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.486+0000 7fe155099700 1 --2- 172.21.15.112:0/1771164423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1501445b0 0x7fe140005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.486+0000 7fe14ffff700 1 -- 172.21.15.112:0/1771164423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1501445b0 msgr2=0x7fe140005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:55.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.486+0000 7fe14ffff700 1 --2- 172.21.15.112:0/1771164423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1501445b0 0x7fe140005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.486+0000 7fe14ffff700 1 -- 172.21.15.112:0/1771164423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe140003a30 msgr2=0x7fe140003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:55.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.486+0000 7fe14ffff700 1 --2- 172.21.15.112:0/1771164423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe140003a30 0x7fe140003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.486+0000 7fe14ffff700 1 -- 172.21.15.112:0/1771164423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1400035b0 con 0x7fe1400043e0 2023-08-27T11:27:55.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.486+0000 7fe14ffff700 1 --2- 172.21.15.112:0/1771164423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1400043e0 0x7fe1400048b0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00c2f0 tx=0x7fe140005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92e2512e452dfc1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:55.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.486+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c00ce30 con 0x7fe1400043e0 2023-08-27T11:27:55.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.486+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c005960 con 0x7fe1400043e0 2023-08-27T11:27:55.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.486+0000 7fe14f7fe700 1 -- 172.21.15.112:0/1771164423 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c005d10 con 0x7fe1400043e0 2023-08-27T11:27:55.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.486+0000 7fe156afc700 1 -- 172.21.15.112:0/1771164423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1400043e0 msgr2=0x7fe1400048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.487+0000 7fe156afc700 1 --2- 172.21.15.112:0/1771164423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1400043e0 0x7fe1400048b0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00c2f0 tx=0x7fe140005880 comp rx=0 tx=0).stop 2023-08-27T11:27:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.487+0000 7fe155099700 1 -- 172.21.15.112:0/1771164423 reap_dead start 2023-08-27T11:27:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.487+0000 7fe156afc700 1 -- 172.21.15.112:0/1771164423 shutdown_connections 2023-08-27T11:27:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.487+0000 7fe156afc700 1 -- 172.21.15.112:0/1771164423 >> 172.21.15.112:0/1771164423 conn(0x7fe1500b1e90 msgr2=0x7fe1500b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.487+0000 7fe156afc700 1 -- 172.21.15.112:0/1771164423 shutdown_connections 2023-08-27T11:27:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.487+0000 7fe156afc700 1 -- 172.21.15.112:0/1771164423 wait complete. 2023-08-27T11:27:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.488+0000 7fe156afc700 1 Processor -- start 2023-08-27T11:27:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.488+0000 7fe156afc700 1 -- start start 2023-08-27T11:27:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe156afc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1400043e0 0x7fe150140660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe156afc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1500aea30 0x7fe1501f0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe156afc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1501f06a0 0x7fe1501f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe156afc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe150140ed0 con 0x7fe1501f06a0 2023-08-27T11:27:55.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe156afc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1501411b0 con 0x7fe1500aea30 2023-08-27T11:27:55.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe156afc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe150141490 con 0x7fe1400043e0 2023-08-27T11:27:55.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe155099700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1501f06a0 0x7fe1501f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe155099700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1501f06a0 0x7fe1501f40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35430/0 (socket says 172.21.15.112:35430) 2023-08-27T11:27:55.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe155099700 1 -- 172.21.15.112:0/2988551892 learned_addr learned my addr 172.21.15.112:0/2988551892 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:55.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe14ffff700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1500aea30 0x7fe1501f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe154898700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1400043e0 0x7fe150140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe155099700 1 -- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1400043e0 msgr2=0x7fe150140660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe155099700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1400043e0 0x7fe150140660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe155099700 1 -- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1500aea30 msgr2=0x7fe1501f0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe155099700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1500aea30 0x7fe1501f0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe155099700 1 -- 172.21.15.112:0/2988551892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe150141770 con 0x7fe1501f06a0 2023-08-27T11:27:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe155099700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1501f06a0 0x7fe1501f40c0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fe148001520 tx=0x7fe14800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe14d7fa700 1 -- 172.21.15.112:0/2988551892 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148003c10 con 0x7fe1501f06a0 2023-08-27T11:27:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe14d7fa700 1 -- 172.21.15.112:0/2988551892 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe148003d70 con 0x7fe1501f06a0 2023-08-27T11:27:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe156afc700 1 -- 172.21.15.112:0/2988551892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1501f0c60 con 0x7fe1501f06a0 2023-08-27T11:27:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe14d7fa700 1 -- 172.21.15.112:0/2988551892 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14800c950 con 0x7fe1501f06a0 2023-08-27T11:27:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.489+0000 7fe156afc700 1 -- 172.21.15.112:0/2988551892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1501f4600 con 0x7fe1501f06a0 2023-08-27T11:27:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.490+0000 7fe156afc700 1 -- 172.21.15.112:0/2988551892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe150067e90 con 0x7fe1501f06a0 2023-08-27T11:27:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.491+0000 7fe14d7fa700 1 -- 172.21.15.112:0/2988551892 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe14800cab0 con 0x7fe1501f06a0 2023-08-27T11:27:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.491+0000 7fe14d7fa700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe13406a4d0 0x7fe13406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.492+0000 7fe14d7fa700 1 -- 172.21.15.112:0/2988551892 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fe148088e30 con 0x7fe1501f06a0 2023-08-27T11:27:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.492+0000 7fe154898700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe13406a4d0 0x7fe13406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.492+0000 7fe154898700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe13406a4d0 0x7fe13406c990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe144009ba0 tx=0x7fe144008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.496+0000 7fe14d7fa700 1 -- 172.21.15.112:0/2988551892 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe148053f20 con 0x7fe1501f06a0 2023-08-27T11:27:55.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.685+0000 7fe156afc700 1 -- 172.21.15.112:0/2988551892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe150067a20 con 0x7fe1501f06a0 2023-08-27T11:27:55.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.686+0000 7fe14d7fa700 1 -- 172.21.15.112:0/2988551892 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v127) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe148057540 con 0x7fe1501f06a0 2023-08-27T11:27:55.688 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:27:55.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.688+0000 7fe156afc700 1 -- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe13406a4d0 msgr2=0x7fe13406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.688+0000 7fe156afc700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe13406a4d0 0x7fe13406c990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe144009ba0 tx=0x7fe144008040 comp rx=0 tx=0).stop 2023-08-27T11:27:55.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.688+0000 7fe156afc700 1 -- 172.21.15.112:0/2988551892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1501f06a0 msgr2=0x7fe1501f40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.688+0000 7fe156afc700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1501f06a0 0x7fe1501f40c0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fe148001520 tx=0x7fe14800ef20 comp rx=0 tx=0).stop 2023-08-27T11:27:55.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.688+0000 7fe156afc700 1 -- 172.21.15.112:0/2988551892 shutdown_connections 2023-08-27T11:27:55.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.688+0000 7fe156afc700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe13406a4d0 0x7fe13406c990 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.688+0000 7fe156afc700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1501f06a0 0x7fe1501f40c0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.688+0000 7fe156afc700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1400043e0 0x7fe150140660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.688+0000 7fe156afc700 1 --2- 172.21.15.112:0/2988551892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1500aea30 0x7fe1501f0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.688+0000 7fe156afc700 1 -- 172.21.15.112:0/2988551892 >> 172.21.15.112:0/2988551892 conn(0x7fe1500b1e90 msgr2=0x7fe150143950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:55.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.688+0000 7fe156afc700 1 -- 172.21.15.112:0/2988551892 shutdown_connections 2023-08-27T11:27:55.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.688+0000 7fe156afc700 1 -- 172.21.15.112:0/2988551892 wait complete. 2023-08-27T11:27:55.701 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:55.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.824+0000 7f4be0e77700 1 Processor -- start 2023-08-27T11:27:55.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.824+0000 7f4be0e77700 1 -- start start 2023-08-27T11:27:55.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.825+0000 7f4be0e77700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bdc13d400 0x7f4bdc15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.825+0000 7f4be0e77700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4bdc0cd2e0 0x7f4bdc06e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.825+0000 7f4be0e77700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bdc06eb30 con 0x7f4bdc06e700 2023-08-27T11:27:55.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.825+0000 7f4be0e77700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bdc0a84f0 con 0x7f4bdc0cd2e0 2023-08-27T11:27:55.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.825+0000 7f4be0e77700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc15b700 con 0x7f4bdc13d400 2023-08-27T11:27:55.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.825+0000 7f4bd9d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4bdc0cd2e0 0x7f4bdc06e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.825+0000 7f4bdad9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4bdc06e700 0x7f4bdc13fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58050/0 (socket says 172.21.15.112:58050) 2023-08-27T11:27:55.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.825+0000 7f4bdad9d700 1 -- 172.21.15.112:0/166889855 learned_addr learned my addr 172.21.15.112:0/166889855 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:55.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.825+0000 7f4bda59c700 1 --2- 172.21.15.112:0/166889855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bdc13d400 0x7f4bdc15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.826+0000 7f4bd959a700 1 -- 172.21.15.112:0/166889855 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888962302 0 0) 0x7f4bdc06eb30 con 0x7f4bdc06e700 2023-08-27T11:27:55.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.826+0000 7f4bd959a700 1 -- 172.21.15.112:0/166889855 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc40035b0 con 0x7f4bdc06e700 2023-08-27T11:27:55.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.826+0000 7f4bd9d9b700 1 -- 172.21.15.112:0/166889855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bdc13d400 msgr2=0x7f4bdc15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:55.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.826+0000 7f4bd9d9b700 1 --2- 172.21.15.112:0/166889855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bdc13d400 0x7f4bdc15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.826+0000 7f4bd9d9b700 1 -- 172.21.15.112:0/166889855 >> v1:172.21.15.112:6789/0 conn(0x7f4bdc06e700 legacy=0x7f4bdc13fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.826+0000 7f4bd9d9b700 1 -- 172.21.15.112:0/166889855 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc0cd8a0 con 0x7f4bdc0cd2e0 2023-08-27T11:27:55.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.826+0000 7f4bd9d9b700 1 --2- 172.21.15.112:0/166889855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bdc0cd2e0 0x7f4bdc06e1c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009840 tx=0x7f4bcc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb774a881bdc99e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:55.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.826+0000 7f4bd959a700 1 -- 172.21.15.112:0/166889855 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc011830 con 0x7f4bdc0cd2e0 2023-08-27T11:27:55.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.826+0000 7f4bd959a700 1 -- 172.21.15.112:0/166889855 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bcc011990 con 0x7f4bdc0cd2e0 2023-08-27T11:27:55.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.826+0000 7f4bd959a700 1 -- 172.21.15.112:0/166889855 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc011d20 con 0x7f4bdc0cd2e0 2023-08-27T11:27:55.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.826+0000 7f4be0e77700 1 -- 172.21.15.112:0/166889855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bdc0cd2e0 msgr2=0x7f4bdc06e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.826+0000 7f4be0e77700 1 --2- 172.21.15.112:0/166889855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bdc0cd2e0 0x7f4bdc06e1c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009840 tx=0x7f4bcc009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:55.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.827+0000 7f4be0e77700 1 -- 172.21.15.112:0/166889855 shutdown_connections 2023-08-27T11:27:55.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.827+0000 7f4be0e77700 1 --2- 172.21.15.112:0/166889855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bdc13d400 0x7f4bdc15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.827+0000 7f4be0e77700 1 --2- 172.21.15.112:0/166889855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bdc0cd2e0 0x7f4bdc06e1c0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.827+0000 7f4be0e77700 1 -- 172.21.15.112:0/166889855 >> 172.21.15.112:0/166889855 conn(0x7f4bdc06cda0 msgr2=0x7f4bdc0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:55.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.827+0000 7f4be0e77700 1 -- 172.21.15.112:0/166889855 shutdown_connections 2023-08-27T11:27:55.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.827+0000 7f4be0e77700 1 -- 172.21.15.112:0/166889855 wait complete. 2023-08-27T11:27:55.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.828+0000 7f4be0e77700 1 Processor -- start 2023-08-27T11:27:55.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.828+0000 7f4be0e77700 1 -- start start 2023-08-27T11:27:55.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4be0e77700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bdc06e700 0x7f4bdc150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4be0e77700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bdc0cd2e0 0x7f4bdc13aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4be0e77700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bdc13d400 0x7f4bdc13b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4be0e77700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc153370 con 0x7f4bdc13d400 2023-08-27T11:27:55.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4be0e77700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc156ba0 con 0x7f4bdc0cd2e0 2023-08-27T11:27:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4be0e77700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc152a40 con 0x7f4bdc06e700 2023-08-27T11:27:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bdad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bdc13d400 0x7f4bdc13b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bdad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bdc13d400 0x7f4bdc13b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35442/0 (socket says 172.21.15.112:35442) 2023-08-27T11:27:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bdad9d700 1 -- 172.21.15.112:0/159973390 learned_addr learned my addr 172.21.15.112:0/159973390 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bda59c700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bdc06e700 0x7f4bdc150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bd9d9b700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bdc0cd2e0 0x7f4bdc13aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bdad9d700 1 -- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bdc06e700 msgr2=0x7f4bdc150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bdad9d700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bdc06e700 0x7f4bdc150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bdad9d700 1 -- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bdc0cd2e0 msgr2=0x7f4bdc13aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bdad9d700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bdc0cd2e0 0x7f4bdc13aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bdad9d700 1 -- 172.21.15.112:0/159973390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc0094a0 con 0x7f4bdc13d400 2023-08-27T11:27:55.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bdad9d700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bdc13d400 0x7f4bdc13b020 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400caa0 tx=0x7f4bd400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:55.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bcaffd700 1 -- 172.21.15.112:0/159973390 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd400a9e0 con 0x7f4bdc13d400 2023-08-27T11:27:55.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bcaffd700 1 -- 172.21.15.112:0/159973390 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd400ab40 con 0x7f4bdc13d400 2023-08-27T11:27:55.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4be0e77700 1 -- 172.21.15.112:0/159973390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bdc152d20 con 0x7f4bdc13d400 2023-08-27T11:27:55.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4bcaffd700 1 -- 172.21.15.112:0/159973390 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4012400 con 0x7f4bdc13d400 2023-08-27T11:27:55.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.829+0000 7f4be0e77700 1 -- 172.21.15.112:0/159973390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bdc150c70 con 0x7f4bdc13d400 2023-08-27T11:27:55.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.830+0000 7f4be0e77700 1 -- 172.21.15.112:0/159973390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bdc150e60 con 0x7f4bdc13d400 2023-08-27T11:27:55.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.831+0000 7f4bcaffd700 1 -- 172.21.15.112:0/159973390 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4bd401b3b0 con 0x7f4bdc13d400 2023-08-27T11:27:55.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.831+0000 7f4bcaffd700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bbc06a4d0 0x7f4bbc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.831+0000 7f4bcaffd700 1 -- 172.21.15.112:0/159973390 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f4bd40888a0 con 0x7f4bdc13d400 2023-08-27T11:27:55.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.832+0000 7f4bda59c700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bbc06a4d0 0x7f4bbc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.832+0000 7f4bda59c700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bbc06a4d0 0x7f4bbc06c990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0006b20 tx=0x7f4bd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:55.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:55.836+0000 7f4bcaffd700 1 -- 172.21.15.112:0/159973390 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4bd4053a50 con 0x7f4bdc13d400 2023-08-27T11:27:56.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.023+0000 7f4be0e77700 1 -- 172.21.15.112:0/159973390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bdc067a20 con 0x7f4bdc13d400 2023-08-27T11:27:56.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.024+0000 7f4bcaffd700 1 -- 172.21.15.112:0/159973390 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4bd4057070 con 0x7f4bdc13d400 2023-08-27T11:27:56.026 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:27:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.026+0000 7f4be0e77700 1 -- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bbc06a4d0 msgr2=0x7f4bbc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.026+0000 7f4be0e77700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bbc06a4d0 0x7f4bbc06c990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0006b20 tx=0x7f4bd0008040 comp rx=0 tx=0).stop 2023-08-27T11:27:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.026+0000 7f4be0e77700 1 -- 172.21.15.112:0/159973390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bdc13d400 msgr2=0x7f4bdc13b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.026+0000 7f4be0e77700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bdc13d400 0x7f4bdc13b020 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400caa0 tx=0x7f4bd400cdb0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.026+0000 7f4be0e77700 1 -- 172.21.15.112:0/159973390 shutdown_connections 2023-08-27T11:27:56.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.026+0000 7f4be0e77700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bbc06a4d0 0x7f4bbc06c990 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.026+0000 7f4be0e77700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bdc13d400 0x7f4bdc13b020 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.026+0000 7f4be0e77700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bdc06e700 0x7f4bdc150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.026+0000 7f4be0e77700 1 --2- 172.21.15.112:0/159973390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bdc0cd2e0 0x7f4bdc13aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.026+0000 7f4be0e77700 1 -- 172.21.15.112:0/159973390 >> 172.21.15.112:0/159973390 conn(0x7f4bdc06cda0 msgr2=0x7f4bdc0a9500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:56.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.026+0000 7f4be0e77700 1 -- 172.21.15.112:0/159973390 shutdown_connections 2023-08-27T11:27:56.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.026+0000 7f4be0e77700 1 -- 172.21.15.112:0/159973390 wait complete. 2023-08-27T11:27:56.039 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:27:56.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.161+0000 7f805cb79700 1 Processor -- start 2023-08-27T11:27:56.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.161+0000 7f805cb79700 1 -- start start 2023-08-27T11:27:56.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.162+0000 7f805cb79700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f805806e4c0 0x7f8058164da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.162+0000 7f805cb79700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805815eb20 con 0x7f805815e690 2023-08-27T11:27:56.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.162+0000 7f805cb79700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805815dc40 con 0x7f805813f370 2023-08-27T11:27:56.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.162+0000 7f805cb79700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805815b110 con 0x7f805806e4c0 2023-08-27T11:27:56.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.162+0000 7f8055d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f805815e690 0x7f805806ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58060/0 (socket says 172.21.15.112:58060) 2023-08-27T11:27:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.162+0000 7f8055d9b700 1 -- 172.21.15.112:0/1084942185 learned_addr learned my addr 172.21.15.112:0/1084942185 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.162+0000 7f8056d9d700 1 --2- 172.21.15.112:0/1084942185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f805806e4c0 0x7f8058164da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.162+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802306358 0 0) 0x7f805815eb20 con 0x7f805815e690 2023-08-27T11:27:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80400035b0 con 0x7f805815e690 2023-08-27T11:27:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312904091 0 0) 0x7f805815dc40 con 0x7f805813f370 2023-08-27T11:27:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f805815eb20 con 0x7f805813f370 2023-08-27T11:27:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 447172506 0 0) 0x7f80400035b0 con 0x7f805815e690 2023-08-27T11:27:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f805806e4c0 msgr2=0x7f8058164da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 --2- 172.21.15.112:0/1084942185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f805806e4c0 0x7f8058164da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 >> v1:172.21.15.138:6789/0 conn(0x7f805813f370 legacy=0x7f805815a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805815b3f0 con 0x7f805815e690 2023-08-27T11:27:56.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f804c001670 con 0x7f805815e690 2023-08-27T11:27:56.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 >> v1:172.21.15.112:6789/0 conn(0x7f805815e690 legacy=0x7f805806ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 --2- 172.21.15.112:0/1084942185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f805806e4c0 0x7f8040003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 --2- 172.21.15.112:0/1084942185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040004170 0x7f8040004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 --2- 172.21.15.112:0/1084942185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8040004d90 0x7f8040005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805815b110 con 0x7f8040004d90 2023-08-27T11:27:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804000bb50 con 0x7f8040004170 2023-08-27T11:27:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804000bd10 con 0x7f805806e4c0 2023-08-27T11:27:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f8056d9d700 1 --2- 172.21.15.112:0/1084942185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8040004d90 0x7f8040005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805659c700 1 --2- 172.21.15.112:0/1084942185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f805806e4c0 0x7f8040003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f8055d9b700 1 --2- 172.21.15.112:0/1084942185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040004170 0x7f8040004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f8056d9d700 1 -- 172.21.15.112:0/1084942185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f805806e4c0 msgr2=0x7f8040003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:56.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f8056d9d700 1 --2- 172.21.15.112:0/1084942185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f805806e4c0 0x7f8040003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f8056d9d700 1 -- 172.21.15.112:0/1084942185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040004170 msgr2=0x7f8040004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:56.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f8056d9d700 1 --2- 172.21.15.112:0/1084942185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040004170 0x7f8040004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f8056d9d700 1 -- 172.21.15.112:0/1084942185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80400035b0 con 0x7f8040004d90 2023-08-27T11:27:56.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f8056d9d700 1 --2- 172.21.15.112:0/1084942185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8040004d90 0x7f8040005210 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f8050009f90 tx=0x7f8040005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab611356372cacc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:56.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.163+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805000ebd0 con 0x7f8040004d90 2023-08-27T11:27:56.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.164+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805000f040 con 0x7f8040004d90 2023-08-27T11:27:56.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.164+0000 7f805559a700 1 -- 172.21.15.112:0/1084942185 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8050014480 con 0x7f8040004d90 2023-08-27T11:27:56.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.164+0000 7f805cb79700 1 -- 172.21.15.112:0/1084942185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8040004d90 msgr2=0x7f8040005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.164+0000 7f805cb79700 1 --2- 172.21.15.112:0/1084942185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8040004d90 0x7f8040005210 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f8050009f90 tx=0x7f8040005880 comp rx=0 tx=0).stop 2023-08-27T11:27:56.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.164+0000 7f8056d9d700 1 -- 172.21.15.112:0/1084942185 reap_dead start 2023-08-27T11:27:56.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.164+0000 7f805cb79700 1 -- 172.21.15.112:0/1084942185 shutdown_connections 2023-08-27T11:27:56.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.164+0000 7f805cb79700 1 -- 172.21.15.112:0/1084942185 >> 172.21.15.112:0/1084942185 conn(0x7f80580cc240 msgr2=0x7f80580cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:56.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.164+0000 7f805cb79700 1 -- 172.21.15.112:0/1084942185 shutdown_connections 2023-08-27T11:27:56.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.164+0000 7f805cb79700 1 -- 172.21.15.112:0/1084942185 wait complete. 2023-08-27T11:27:56.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.165+0000 7f805cb79700 1 Processor -- start 2023-08-27T11:27:56.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.165+0000 7f805cb79700 1 -- start start 2023-08-27T11:27:56.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.166+0000 7f805cb79700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040004d90 0x7f805813dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.166+0000 7f805cb79700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8058155d80 0x7f80580ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.166+0000 7f805cb79700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80580ae660 0x7f8058151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.166+0000 7f805cb79700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8058156aa0 con 0x7f8058155d80 2023-08-27T11:27:56.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.166+0000 7f805cb79700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8058156200 con 0x7f8040004d90 2023-08-27T11:27:56.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.166+0000 7f805cb79700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80581564e0 con 0x7f80580ae660 2023-08-27T11:27:56.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.166+0000 7f8055d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8058155d80 0x7f80580ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.166+0000 7f8055d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8058155d80 0x7f80580ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35460/0 (socket says 172.21.15.112:35460) 2023-08-27T11:27:56.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.166+0000 7f8055d9b700 1 -- 172.21.15.112:0/1463368019 learned_addr learned my addr 172.21.15.112:0/1463368019 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:56.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.167+0000 7f8055d9b700 1 -- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80580ae660 msgr2=0x7f8058151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.167+0000 7f8056d9d700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80580ae660 0x7f8058151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.167+0000 7f805659c700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040004d90 0x7f805813dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.167+0000 7f8055d9b700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80580ae660 0x7f8058151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.167+0000 7f8055d9b700 1 -- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040004d90 msgr2=0x7f805813dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.167+0000 7f8055d9b700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040004d90 0x7f805813dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.167+0000 7f8055d9b700 1 -- 172.21.15.112:0/1463368019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80581567c0 con 0x7f8058155d80 2023-08-27T11:27:56.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.167+0000 7f8055d9b700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8058155d80 0x7f80580ae120 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f804c007290 tx=0x7f804c0075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:56.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.167+0000 7f8046ffd700 1 -- 172.21.15.112:0/1463368019 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c00f6c0 con 0x7f8058155d80 2023-08-27T11:27:56.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.167+0000 7f8046ffd700 1 -- 172.21.15.112:0/1463368019 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c00f860 con 0x7f8058155d80 2023-08-27T11:27:56.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.167+0000 7f805cb79700 1 -- 172.21.15.112:0/1463368019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80581f2a60 con 0x7f8058155d80 2023-08-27T11:27:56.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.167+0000 7f8046ffd700 1 -- 172.21.15.112:0/1463368019 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c00fc10 con 0x7f8058155d80 2023-08-27T11:27:56.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.167+0000 7f805cb79700 1 -- 172.21.15.112:0/1463368019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8058151a90 con 0x7f8058155d80 2023-08-27T11:27:56.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.168+0000 7f805cb79700 1 -- 172.21.15.112:0/1463368019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8058067e90 con 0x7f8058155d80 2023-08-27T11:27:56.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.168+0000 7f8046ffd700 1 -- 172.21.15.112:0/1463368019 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f804c00fd70 con 0x7f8058155d80 2023-08-27T11:27:56.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.169+0000 7f8046ffd700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f803806a4d0 0x7f803806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.169+0000 7f8046ffd700 1 -- 172.21.15.112:0/1463368019 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f804c088be0 con 0x7f8058155d80 2023-08-27T11:27:56.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.169+0000 7f805659c700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f803806a4d0 0x7f803806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.169+0000 7f805659c700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f803806a4d0 0x7f803806c990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8048007fd0 tx=0x7f8048009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:56.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.173+0000 7f8046ffd700 1 -- 172.21.15.112:0/1463368019 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f804c053de0 con 0x7f8058155d80 2023-08-27T11:27:56.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.363+0000 7f805cb79700 1 -- 172.21.15.112:0/1463368019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8058067a20 con 0x7f8058155d80 2023-08-27T11:27:56.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.364+0000 7f8046ffd700 1 -- 172.21.15.112:0/1463368019 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f804c057400 con 0x7f8058155d80 2023-08-27T11:27:56.366 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:27:56.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.366+0000 7f805cb79700 1 -- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f803806a4d0 msgr2=0x7f803806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.366+0000 7f805cb79700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f803806a4d0 0x7f803806c990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8048007fd0 tx=0x7f8048009040 comp rx=0 tx=0).stop 2023-08-27T11:27:56.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.366+0000 7f805cb79700 1 -- 172.21.15.112:0/1463368019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8058155d80 msgr2=0x7f80580ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.366+0000 7f805cb79700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8058155d80 0x7f80580ae120 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f804c007290 tx=0x7f804c0075a0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.366+0000 7f805cb79700 1 -- 172.21.15.112:0/1463368019 shutdown_connections 2023-08-27T11:27:56.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.366+0000 7f805cb79700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f803806a4d0 0x7f803806c990 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.366+0000 7f805cb79700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80580ae660 0x7f8058151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.366+0000 7f805cb79700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8040004d90 0x7f805813dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.366+0000 7f805cb79700 1 --2- 172.21.15.112:0/1463368019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8058155d80 0x7f80580ae120 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.366+0000 7f805cb79700 1 -- 172.21.15.112:0/1463368019 >> 172.21.15.112:0/1463368019 conn(0x7f80580cc240 msgr2=0x7f805813f750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:56.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.366+0000 7f805cb79700 1 -- 172.21.15.112:0/1463368019 shutdown_connections 2023-08-27T11:27:56.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.366+0000 7f805cb79700 1 -- 172.21.15.112:0/1463368019 wait complete. 2023-08-27T11:27:56.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:27:56.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.501+0000 7fc55a6ce700 1 Processor -- start 2023-08-27T11:27:56.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.501+0000 7fc55a6ce700 1 -- start start 2023-08-27T11:27:56.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.502+0000 7fc55a6ce700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc554158ed0 0x7fc55415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.502+0000 7fc55a6ce700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc55406e4c0 0x7fc55415cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.502+0000 7fc55a6ce700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55415ca50 con 0x7fc554072000 2023-08-27T11:27:56.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.502+0000 7fc55a6ce700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc554072490 con 0x7fc55406e4c0 2023-08-27T11:27:56.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.502+0000 7fc55a6ce700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55415be20 con 0x7fc554158ed0 2023-08-27T11:27:56.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.502+0000 7fc558c6b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc55406e4c0 0x7fc55415cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.502+0000 7fc5537fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc554072000 0x7fc55406ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58080/0 (socket says 172.21.15.112:58080) 2023-08-27T11:27:56.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.502+0000 7fc5537fe700 1 -- 172.21.15.112:0/3213575120 learned_addr learned my addr 172.21.15.112:0/3213575120 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:56.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc553fff700 1 --2- 172.21.15.112:0/3213575120 >> v2:172.21.15.138:3300/0 conn(0x7fc554158ed0 0x7fc55415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc558c6b700 1 -- 172.21.15.112:0/3213575120 >> v2:172.21.15.138:3300/0 conn(0x7fc554158ed0 msgr2=0x7fc55415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:56.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc558c6b700 1 --2- 172.21.15.112:0/3213575120 >> v2:172.21.15.138:3300/0 conn(0x7fc554158ed0 0x7fc55415b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc558c6b700 1 -- 172.21.15.112:0/3213575120 >> v1:172.21.15.112:6789/0 conn(0x7fc554072000 legacy=0x7fc55406ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc558c6b700 1 -- 172.21.15.112:0/3213575120 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55415c100 con 0x7fc55406e4c0 2023-08-27T11:27:56.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc552ffd700 1 -- 172.21.15.112:0/3213575120 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097128282 0 0) 0x7fc55415ca50 con 0x7fc554072000 2023-08-27T11:27:56.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc558c6b700 1 --2- 172.21.15.112:0/3213575120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc55406e4c0 0x7fc55415cee0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc5540687a0 tx=0x7fc54c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af89f478d5fd573a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:56.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc552ffd700 1 -- 172.21.15.112:0/3213575120 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c00e070 con 0x7fc55406e4c0 2023-08-27T11:27:56.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc552ffd700 1 -- 172.21.15.112:0/3213575120 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c00bbd0 con 0x7fc55406e4c0 2023-08-27T11:27:56.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc552ffd700 1 -- 172.21.15.112:0/3213575120 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c012430 con 0x7fc55406e4c0 2023-08-27T11:27:56.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc55a6ce700 1 -- 172.21.15.112:0/3213575120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc55406e4c0 msgr2=0x7fc55415cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc55a6ce700 1 --2- 172.21.15.112:0/3213575120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc55406e4c0 0x7fc55415cee0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc5540687a0 tx=0x7fc54c00b330 comp rx=0 tx=0).stop 2023-08-27T11:27:56.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc55a6ce700 1 -- 172.21.15.112:0/3213575120 shutdown_connections 2023-08-27T11:27:56.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc55a6ce700 1 --2- 172.21.15.112:0/3213575120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc55406e4c0 0x7fc55415cee0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc55a6ce700 1 --2- 172.21.15.112:0/3213575120 >> v2:172.21.15.138:3300/0 conn(0x7fc554158ed0 0x7fc55415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc55a6ce700 1 -- 172.21.15.112:0/3213575120 >> 172.21.15.112:0/3213575120 conn(0x7fc5540cc240 msgr2=0x7fc5540cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:56.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc55a6ce700 1 -- 172.21.15.112:0/3213575120 shutdown_connections 2023-08-27T11:27:56.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.503+0000 7fc55a6ce700 1 -- 172.21.15.112:0/3213575120 wait complete. 2023-08-27T11:27:56.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.505+0000 7fc55a6ce700 1 Processor -- start 2023-08-27T11:27:56.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.505+0000 7fc55a6ce700 1 -- start start 2023-08-27T11:27:56.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.505+0000 7fc55a6ce700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc55406e4c0 0x7fc5541e2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.505+0000 7fc55a6ce700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc554072000 0x7fc5541e3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc55a6ce700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc554158ed0 0x7fc5541f0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc55a6ce700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541e35d0 con 0x7fc554072000 2023-08-27T11:27:56.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc55a6ce700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541e3880 con 0x7fc554158ed0 2023-08-27T11:27:56.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc55a6ce700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541e3b60 con 0x7fc55406e4c0 2023-08-27T11:27:56.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc5537fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc554072000 0x7fc5541e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc5537fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc554072000 0x7fc5541e3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35474/0 (socket says 172.21.15.112:35474) 2023-08-27T11:27:56.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc5537fe700 1 -- 172.21.15.112:0/953713253 learned_addr learned my addr 172.21.15.112:0/953713253 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:56.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc553fff700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc55406e4c0 0x7fc5541e2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc558c6b700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc554158ed0 0x7fc5541f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc5537fe700 1 -- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc55406e4c0 msgr2=0x7fc5541e2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc5537fe700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc55406e4c0 0x7fc5541e2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc5537fe700 1 -- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc554158ed0 msgr2=0x7fc5541f0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc5537fe700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc554158ed0 0x7fc5541f0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc5537fe700 1 -- 172.21.15.112:0/953713253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c009a70 con 0x7fc554072000 2023-08-27T11:27:56.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc553fff700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc55406e4c0 0x7fc5541e2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:56.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc558c6b700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc554158ed0 0x7fc5541f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:56.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc5537fe700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc554072000 0x7fc5541e3090 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fc5440011e0 tx=0x7fc54400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:56.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc550ff9700 1 -- 172.21.15.112:0/953713253 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544004c80 con 0x7fc554072000 2023-08-27T11:27:56.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc550ff9700 1 -- 172.21.15.112:0/953713253 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc544004e00 con 0x7fc554072000 2023-08-27T11:27:56.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc55a6ce700 1 -- 172.21.15.112:0/953713253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5541e3e40 con 0x7fc554072000 2023-08-27T11:27:56.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc550ff9700 1 -- 172.21.15.112:0/953713253 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544007310 con 0x7fc554072000 2023-08-27T11:27:56.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.506+0000 7fc55a6ce700 1 -- 172.21.15.112:0/953713253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc554158920 con 0x7fc554072000 2023-08-27T11:27:56.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.507+0000 7fc55a6ce700 1 -- 172.21.15.112:0/953713253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5541e4360 con 0x7fc554072000 2023-08-27T11:27:56.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.508+0000 7fc550ff9700 1 -- 172.21.15.112:0/953713253 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fc544007470 con 0x7fc554072000 2023-08-27T11:27:56.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.508+0000 7fc550ff9700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc53406a4d0 0x7fc53406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.508+0000 7fc550ff9700 1 -- 172.21.15.112:0/953713253 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fc5440889d0 con 0x7fc554072000 2023-08-27T11:27:56.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.508+0000 7fc553fff700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc53406a4d0 0x7fc53406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.509+0000 7fc553fff700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc53406a4d0 0x7fc53406c990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc53c006fd0 tx=0x7fc53c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:56.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.513+0000 7fc550ff9700 1 -- 172.21.15.112:0/953713253 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc5541e4360 con 0x7fc554072000 2023-08-27T11:27:56.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.702+0000 7fc55a6ce700 1 -- 172.21.15.112:0/953713253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5541e4360 con 0x7fc554072000 2023-08-27T11:27:56.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.702+0000 7fc550ff9700 1 -- 172.21.15.112:0/953713253 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc5541e4360 con 0x7fc554072000 2023-08-27T11:27:56.704 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:27:56.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.704+0000 7fc55a6ce700 1 -- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc53406a4d0 msgr2=0x7fc53406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.704+0000 7fc55a6ce700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc53406a4d0 0x7fc53406c990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc53c006fd0 tx=0x7fc53c008040 comp rx=0 tx=0).stop 2023-08-27T11:27:56.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.704+0000 7fc55a6ce700 1 -- 172.21.15.112:0/953713253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc554072000 msgr2=0x7fc5541e3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.704+0000 7fc55a6ce700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc554072000 0x7fc5541e3090 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fc5440011e0 tx=0x7fc54400cae0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.705+0000 7fc55a6ce700 1 -- 172.21.15.112:0/953713253 shutdown_connections 2023-08-27T11:27:56.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.705+0000 7fc55a6ce700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc53406a4d0 0x7fc53406c990 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.705+0000 7fc55a6ce700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc554158ed0 0x7fc5541f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.705+0000 7fc55a6ce700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc55406e4c0 0x7fc5541e2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.705+0000 7fc55a6ce700 1 --2- 172.21.15.112:0/953713253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc554072000 0x7fc5541e3090 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.705+0000 7fc55a6ce700 1 -- 172.21.15.112:0/953713253 >> 172.21.15.112:0/953713253 conn(0x7fc5540cc240 msgr2=0x7fc5540cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.705+0000 7fc55a6ce700 1 -- 172.21.15.112:0/953713253 shutdown_connections 2023-08-27T11:27:56.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.705+0000 7fc55a6ce700 1 -- 172.21.15.112:0/953713253 wait complete. 2023-08-27T11:27:56.718 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:27:56.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.838+0000 7fa0be8c3700 1 Processor -- start 2023-08-27T11:27:56.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.838+0000 7fa0be8c3700 1 -- start start 2023-08-27T11:27:56.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0be8c3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0b806dcc0 0x7fa0b80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0be8c3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa0b80b3440 0x7fa0b80b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0be8c3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b815b540 con 0x7fa0b815a120 2023-08-27T11:27:56.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0be8c3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b80af830 con 0x7fa0b806dcc0 2023-08-27T11:27:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0be8c3700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b80afb10 con 0x7fa0b80b3440 2023-08-27T11:27:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0b77fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0b806dcc0 0x7fa0b80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0b7fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa0b815a120 0x7fa0b806d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58094/0 (socket says 172.21.15.112:58094) 2023-08-27T11:27:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0b7fff700 1 -- 172.21.15.112:0/1028650938 learned_addr learned my addr 172.21.15.112:0/1028650938 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0bce60700 1 --2- 172.21.15.112:0/1028650938 >> v2:172.21.15.138:3300/0 conn(0x7fa0b80b3440 0x7fa0b80b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0b77fe700 1 -- 172.21.15.112:0/1028650938 >> v2:172.21.15.138:3300/0 conn(0x7fa0b80b3440 msgr2=0x7fa0b80b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0b77fe700 1 --2- 172.21.15.112:0/1028650938 >> v2:172.21.15.138:3300/0 conn(0x7fa0b80b3440 0x7fa0b80b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0b77fe700 1 -- 172.21.15.112:0/1028650938 >> v1:172.21.15.112:6789/0 conn(0x7fa0b815a120 legacy=0x7fa0b806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0b77fe700 1 -- 172.21.15.112:0/1028650938 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b80afdf0 con 0x7fa0b806dcc0 2023-08-27T11:27:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0b6ffd700 1 -- 172.21.15.112:0/1028650938 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413937616 0 0) 0x7fa0b815b540 con 0x7fa0b815a120 2023-08-27T11:27:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.839+0000 7fa0b77fe700 1 --2- 172.21.15.112:0/1028650938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b806dcc0 0x7fa0b80b05f0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80b5cd0 tx=0x7fa0a8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94798e951f1bd9e9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.840+0000 7fa0b6ffd700 1 -- 172.21.15.112:0/1028650938 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8011830 con 0x7fa0b806dcc0 2023-08-27T11:27:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.840+0000 7fa0b6ffd700 1 -- 172.21.15.112:0/1028650938 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a8011990 con 0x7fa0b806dcc0 2023-08-27T11:27:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.840+0000 7fa0b6ffd700 1 -- 172.21.15.112:0/1028650938 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8011d20 con 0x7fa0b806dcc0 2023-08-27T11:27:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.840+0000 7fa0be8c3700 1 -- 172.21.15.112:0/1028650938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b806dcc0 msgr2=0x7fa0b80b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.840+0000 7fa0be8c3700 1 --2- 172.21.15.112:0/1028650938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b806dcc0 0x7fa0b80b05f0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80b5cd0 tx=0x7fa0a8009b20 comp rx=0 tx=0).stop 2023-08-27T11:27:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.840+0000 7fa0be8c3700 1 -- 172.21.15.112:0/1028650938 shutdown_connections 2023-08-27T11:27:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.840+0000 7fa0be8c3700 1 --2- 172.21.15.112:0/1028650938 >> v2:172.21.15.138:3300/0 conn(0x7fa0b80b3440 0x7fa0b80b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.840+0000 7fa0be8c3700 1 --2- 172.21.15.112:0/1028650938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b806dcc0 0x7fa0b80b05f0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.840+0000 7fa0be8c3700 1 -- 172.21.15.112:0/1028650938 >> 172.21.15.112:0/1028650938 conn(0x7fa0b80cff80 msgr2=0x7fa0b80d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.840+0000 7fa0be8c3700 1 -- 172.21.15.112:0/1028650938 shutdown_connections 2023-08-27T11:27:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.840+0000 7fa0be8c3700 1 -- 172.21.15.112:0/1028650938 wait complete. 2023-08-27T11:27:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.841+0000 7fa0be8c3700 1 Processor -- start 2023-08-27T11:27:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.842+0000 7fa0be8c3700 1 -- start start 2023-08-27T11:27:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.842+0000 7fa0be8c3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b806dcc0 0x7fa0b81520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.842+0000 7fa0be8c3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0b80b3440 0x7fa0b814d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.842+0000 7fa0be8c3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0b815a120 0x7fa0b8154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.842+0000 7fa0be8c3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b81536e0 con 0x7fa0b806dcc0 2023-08-27T11:27:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.842+0000 7fa0be8c3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80c9610 con 0x7fa0b80b3440 2023-08-27T11:27:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.842+0000 7fa0be8c3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b8153950 con 0x7fa0b815a120 2023-08-27T11:27:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.842+0000 7fa0b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b806dcc0 0x7fa0b81520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.842+0000 7fa0b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b806dcc0 0x7fa0b81520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35486/0 (socket says 172.21.15.112:35486) 2023-08-27T11:27:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.842+0000 7fa0b7fff700 1 -- 172.21.15.112:0/2505892618 learned_addr learned my addr 172.21.15.112:0/2505892618 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.842+0000 7fa0bce60700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0b815a120 0x7fa0b8154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.843+0000 7fa0b77fe700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0b80b3440 0x7fa0b814d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.843+0000 7fa0b7fff700 1 -- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0b815a120 msgr2=0x7fa0b8154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.843+0000 7fa0b7fff700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0b815a120 0x7fa0b8154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.843+0000 7fa0b7fff700 1 -- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0b80b3440 msgr2=0x7fa0b814d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.843+0000 7fa0b7fff700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0b80b3440 0x7fa0b814d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.843+0000 7fa0b7fff700 1 -- 172.21.15.112:0/2505892618 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a80094a0 con 0x7fa0b806dcc0 2023-08-27T11:27:56.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.843+0000 7fa0bce60700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0b815a120 0x7fa0b8154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:56.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.843+0000 7fa0b7fff700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b806dcc0 0x7fa0b81520b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00c550 tx=0x7fa0ac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:56.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.843+0000 7fa0b4ff9700 1 -- 172.21.15.112:0/2505892618 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac004c80 con 0x7fa0b806dcc0 2023-08-27T11:27:56.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.843+0000 7fa0b4ff9700 1 -- 172.21.15.112:0/2505892618 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac004e00 con 0x7fa0b806dcc0 2023-08-27T11:27:56.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.843+0000 7fa0be8c3700 1 -- 172.21.15.112:0/2505892618 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b8153c30 con 0x7fa0b806dcc0 2023-08-27T11:27:56.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.843+0000 7fa0b4ff9700 1 -- 172.21.15.112:0/2505892618 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac007310 con 0x7fa0b806dcc0 2023-08-27T11:27:56.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.843+0000 7fa0be8c3700 1 -- 172.21.15.112:0/2505892618 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b814de00 con 0x7fa0b806dcc0 2023-08-27T11:27:56.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.844+0000 7fa0be8c3700 1 -- 172.21.15.112:0/2505892618 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b8067a20 con 0x7fa0b806dcc0 2023-08-27T11:27:56.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.844+0000 7fa0b4ff9700 1 -- 172.21.15.112:0/2505892618 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa0ac007470 con 0x7fa0b806dcc0 2023-08-27T11:27:56.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.845+0000 7fa0b4ff9700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa0a406a4d0 0x7fa0a406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.845+0000 7fa0b4ff9700 1 -- 172.21.15.112:0/2505892618 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fa0ac0887c0 con 0x7fa0b806dcc0 2023-08-27T11:27:56.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.845+0000 7fa0b77fe700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa0a406a4d0 0x7fa0a406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.845+0000 7fa0b77fe700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa0a406a4d0 0x7fa0a406c990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80090b0 tx=0x7fa0a8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:56.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:56.849+0000 7fa0b4ff9700 1 -- 172.21.15.112:0/2505892618 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa0ac053ac0 con 0x7fa0b806dcc0 2023-08-27T11:27:57.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.052+0000 7fa0be8c3700 1 -- 172.21.15.112:0/2505892618 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa0b80b1190 con 0x7fa0b806dcc0 2023-08-27T11:27:57.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.053+0000 7fa0b4ff9700 1 -- 172.21.15.112:0/2505892618 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v127) v1 ==== 80+0+248 (secure 0 0 0) 0x7fa0ac0570e0 con 0x7fa0b806dcc0 2023-08-27T11:27:57.055 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6832/3832579050 2023-08-28T11:27:49.540827+0000 2023-08-27T11:27:57.055 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6834/3832579050 2023-08-28T11:27:49.540827+0000 2023-08-27T11:27:57.055 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6835/4019485943 2023-08-28T11:27:46.133510+0000 2023-08-27T11:27:57.055 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6833/4019485943 2023-08-28T11:27:46.133510+0000 2023-08-27T11:27:57.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.055+0000 7fa0be8c3700 1 -- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa0a406a4d0 msgr2=0x7fa0a406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.055+0000 7fa0be8c3700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa0a406a4d0 0x7fa0a406c990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80090b0 tx=0x7fa0a8008030 comp rx=0 tx=0).stop 2023-08-27T11:27:57.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.055+0000 7fa0be8c3700 1 -- 172.21.15.112:0/2505892618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b806dcc0 msgr2=0x7fa0b81520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.055+0000 7fa0be8c3700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b806dcc0 0x7fa0b81520b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00c550 tx=0x7fa0ac00c910 comp rx=0 tx=0).stop 2023-08-27T11:27:57.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.055+0000 7fa0be8c3700 1 -- 172.21.15.112:0/2505892618 shutdown_connections 2023-08-27T11:27:57.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.055+0000 7fa0be8c3700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa0a406a4d0 0x7fa0a406c990 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.055+0000 7fa0be8c3700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0b815a120 0x7fa0b8154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.055+0000 7fa0be8c3700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b806dcc0 0x7fa0b81520b0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.055+0000 7fa0be8c3700 1 --2- 172.21.15.112:0/2505892618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0b80b3440 0x7fa0b814d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.055+0000 7fa0be8c3700 1 -- 172.21.15.112:0/2505892618 >> 172.21.15.112:0/2505892618 conn(0x7fa0b80cff80 msgr2=0x7fa0b806e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:57.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.055+0000 7fa0be8c3700 1 -- 172.21.15.112:0/2505892618 shutdown_connections 2023-08-27T11:27:57.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.055+0000 7fa0be8c3700 1 -- 172.21.15.112:0/2505892618 wait complete. 2023-08-27T11:27:57.059 INFO:teuthology.orchestra.run.smithi112.stderr:listed 4 entries 2023-08-27T11:27:57.069 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-27T11:27:57.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:57.069 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-08-27T11:27:57.116 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:27:57.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:57.117 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-27T11:27:57.136 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:27:57.136 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:57.153 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:27:57.154 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:27:57.154 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:57.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:27:57.195 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:57.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.314+0000 7fe84b83b700 1 Processor -- start 2023-08-27T11:27:57.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.314+0000 7fe84b83b700 1 -- start start 2023-08-27T11:27:57.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.315+0000 7fe84b83b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe84415d400 0x7fe8440717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.315+0000 7fe84b83b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84415d7e0 con 0x7fe8440cf5d0 2023-08-27T11:27:57.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.315+0000 7fe84b83b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8440cbde0 con 0x7fe844071cf0 2023-08-27T11:27:57.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.315+0000 7fe84b83b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8440d0300 con 0x7fe84415d400 2023-08-27T11:27:57.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.315+0000 7fe8495d7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe84415d400 0x7fe8440717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.315+0000 7fe849dd8700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe8440cf5d0 0x7fe8440cc5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58112/0 (socket says 172.21.15.112:58112) 2023-08-27T11:27:57.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.315+0000 7fe849dd8700 1 -- 172.21.15.112:0/3005223472 learned_addr learned my addr 172.21.15.112:0/3005223472 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:57.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.315+0000 7fe8495d7700 1 -- 172.21.15.112:0/3005223472 >> v1:172.21.15.138:6789/0 conn(0x7fe844071cf0 legacy=0x7fe8440d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:57.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.315+0000 7fe8495d7700 1 -- 172.21.15.112:0/3005223472 >> v1:172.21.15.112:6789/0 conn(0x7fe8440cf5d0 legacy=0x7fe8440cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.315+0000 7fe83bfff700 1 -- 172.21.15.112:0/3005223472 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474574529 0 0) 0x7fe84415d7e0 con 0x7fe8440cf5d0 2023-08-27T11:27:57.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.315+0000 7fe8495d7700 1 -- 172.21.15.112:0/3005223472 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8440cfa60 con 0x7fe84415d400 2023-08-27T11:27:57.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.316+0000 7fe8495d7700 1 --2- 172.21.15.112:0/3005223472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe84415d400 0x7fe8440717b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fe834009840 tx=0x7fe834009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f130f599c5f9aac server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.316+0000 7fe83bfff700 1 -- 172.21.15.112:0/3005223472 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834004230 con 0x7fe84415d400 2023-08-27T11:27:57.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.316+0000 7fe83bfff700 1 -- 172.21.15.112:0/3005223472 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe834004390 con 0x7fe84415d400 2023-08-27T11:27:57.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.316+0000 7fe83bfff700 1 -- 172.21.15.112:0/3005223472 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8340036a0 con 0x7fe84415d400 2023-08-27T11:27:57.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.316+0000 7fe84b83b700 1 -- 172.21.15.112:0/3005223472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe84415d400 msgr2=0x7fe8440717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.316+0000 7fe84b83b700 1 --2- 172.21.15.112:0/3005223472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe84415d400 0x7fe8440717b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fe834009840 tx=0x7fe834009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:57.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.316+0000 7fe84b83b700 1 -- 172.21.15.112:0/3005223472 shutdown_connections 2023-08-27T11:27:57.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.316+0000 7fe84b83b700 1 --2- 172.21.15.112:0/3005223472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe84415d400 0x7fe8440717b0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.316+0000 7fe84b83b700 1 -- 172.21.15.112:0/3005223472 >> 172.21.15.112:0/3005223472 conn(0x7fe84406adb0 msgr2=0x7fe84406b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:57.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.316+0000 7fe84b83b700 1 -- 172.21.15.112:0/3005223472 shutdown_connections 2023-08-27T11:27:57.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.316+0000 7fe84b83b700 1 -- 172.21.15.112:0/3005223472 wait complete. 2023-08-27T11:27:57.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.318+0000 7fe84b83b700 1 Processor -- start 2023-08-27T11:27:57.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.318+0000 7fe84b83b700 1 -- start start 2023-08-27T11:27:57.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.318+0000 7fe84b83b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe844071cf0 0x7fe8440cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.318+0000 7fe84b83b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84415d400 0x7fe8440c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.318+0000 7fe84b83b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8441f2630 0x7fe8441f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.318+0000 7fe84b83b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8441f3690 con 0x7fe844071cf0 2023-08-27T11:27:57.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.318+0000 7fe84b83b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8441f30d0 con 0x7fe84415d400 2023-08-27T11:27:57.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.318+0000 7fe84b83b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8441f33b0 con 0x7fe8441f2630 2023-08-27T11:27:57.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.318+0000 7fe8495d7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe844071cf0 0x7fe8440cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.318+0000 7fe8495d7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe844071cf0 0x7fe8440cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35500/0 (socket says 172.21.15.112:35500) 2023-08-27T11:27:57.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.318+0000 7fe8495d7700 1 -- 172.21.15.112:0/2130602114 learned_addr learned my addr 172.21.15.112:0/2130602114 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:57.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe849dd8700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8441f2630 0x7fe8441f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe848dd6700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84415d400 0x7fe8440c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe8495d7700 1 -- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8441f2630 msgr2=0x7fe8441f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe8495d7700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8441f2630 0x7fe8441f2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe8495d7700 1 -- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84415d400 msgr2=0x7fe8440c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe8495d7700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84415d400 0x7fe8440c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe8495d7700 1 -- 172.21.15.112:0/2130602114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8340094a0 con 0x7fe844071cf0 2023-08-27T11:27:57.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe849dd8700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8441f2630 0x7fe8441f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:57.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe848dd6700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84415d400 0x7fe8440c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:57.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe8495d7700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe844071cf0 0x7fe8440cae50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe8340090b0 tx=0x7fe8340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe839ffb700 1 -- 172.21.15.112:0/2130602114 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834011830 con 0x7fe844071cf0 2023-08-27T11:27:57.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe839ffb700 1 -- 172.21.15.112:0/2130602114 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe834011990 con 0x7fe844071cf0 2023-08-27T11:27:57.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe84b83b700 1 -- 172.21.15.112:0/2130602114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8440c82d0 con 0x7fe844071cf0 2023-08-27T11:27:57.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe839ffb700 1 -- 172.21.15.112:0/2130602114 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834011d40 con 0x7fe844071cf0 2023-08-27T11:27:57.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.319+0000 7fe84b83b700 1 -- 172.21.15.112:0/2130602114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8440cb390 con 0x7fe844071cf0 2023-08-27T11:27:57.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.320+0000 7fe84b83b700 1 -- 172.21.15.112:0/2130602114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8440c8130 con 0x7fe844071cf0 2023-08-27T11:27:57.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.321+0000 7fe839ffb700 1 -- 172.21.15.112:0/2130602114 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe83401b440 con 0x7fe844071cf0 2023-08-27T11:27:57.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.321+0000 7fe839ffb700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe82806a4d0 0x7fe82806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.321+0000 7fe839ffb700 1 -- 172.21.15.112:0/2130602114 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fe83408a5e0 con 0x7fe844071cf0 2023-08-27T11:27:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.321+0000 7fe848dd6700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe82806a4d0 0x7fe82806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.322+0000 7fe848dd6700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe82806a4d0 0x7fe82806c990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe8440ce1b0 tx=0x7fe840009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.325+0000 7fe839ffb700 1 -- 172.21.15.112:0/2130602114 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe8440c8130 con 0x7fe844071cf0 2023-08-27T11:27:57.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.556+0000 7fe84b83b700 1 -- 172.21.15.112:0/2130602114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8440c87f0 con 0x7fe844071cf0 2023-08-27T11:27:57.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.557+0000 7fe839ffb700 1 -- 172.21.15.112:0/2130602114 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4889 (secure 0 0 0) 0x7fe8440c87f0 con 0x7fe844071cf0 2023-08-27T11:27:57.559 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:57.559 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1795721853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1795721853},{"type":"v1","addr":"172.21.15.138:6836","nonce":1795721853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":8184,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2658811115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2658811115},{"type":"v1","addr":"172.21.15.112:6835","nonce":2658811115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":8199,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3536640393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3536640393},{"type":"v1","addr":"172.21.15.112:6834","nonce":3536640393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:27:28.107305+0000","modified":"2023-08-27T11:27:31.742912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7989},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7989":{"gid":7989,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/3245198376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3245198376},{"type":"v1","addr":"172.21.15.138:6837","nonce":3245198376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-27T11:27:57.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.559+0000 7fe84b83b700 1 -- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe82806a4d0 msgr2=0x7fe82806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.559+0000 7fe84b83b700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe82806a4d0 0x7fe82806c990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe8440ce1b0 tx=0x7fe840009040 comp rx=0 tx=0).stop 2023-08-27T11:27:57.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.559+0000 7fe84b83b700 1 -- 172.21.15.112:0/2130602114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe844071cf0 msgr2=0x7fe8440cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.559+0000 7fe84b83b700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe844071cf0 0x7fe8440cae50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe8340090b0 tx=0x7fe8340047c0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.559+0000 7fe84b83b700 1 -- 172.21.15.112:0/2130602114 shutdown_connections 2023-08-27T11:27:57.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.559+0000 7fe84b83b700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe82806a4d0 0x7fe82806c990 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.559+0000 7fe84b83b700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8441f2630 0x7fe8441f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.559+0000 7fe84b83b700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe844071cf0 0x7fe8440cae50 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.559+0000 7fe84b83b700 1 --2- 172.21.15.112:0/2130602114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84415d400 0x7fe8440c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.559+0000 7fe84b83b700 1 -- 172.21.15.112:0/2130602114 >> 172.21.15.112:0/2130602114 conn(0x7fe84406adb0 msgr2=0x7fe8441514f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:57.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.559+0000 7fe84b83b700 1 -- 172.21.15.112:0/2130602114 shutdown_connections 2023-08-27T11:27:57.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.559+0000 7fe84b83b700 1 -- 172.21.15.112:0/2130602114 wait complete. 2023-08-27T11:27:57.564 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 86 2023-08-27T11:27:57.574 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:57.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.692+0000 7fcb8d19f700 1 Processor -- start 2023-08-27T11:27:57.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.692+0000 7fcb8d19f700 1 -- start start 2023-08-27T11:27:57.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.693+0000 7fcb8d19f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb880cea00 0x7fcb880b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.693+0000 7fcb8d19f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcb88155770 0x7fcb8806bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.693+0000 7fcb8d19f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb880b6040 con 0x7fcb88155390 2023-08-27T11:27:57.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.693+0000 7fcb8d19f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb880b6320 con 0x7fcb88155770 2023-08-27T11:27:57.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.693+0000 7fcb8d19f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880b6600 con 0x7fcb880cea00 2023-08-27T11:27:57.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.693+0000 7fcb8c99e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcb88155770 0x7fcb8806bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.693+0000 7fcb8c99e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcb88155770 0x7fcb8806bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35512/0 (socket says 172.21.15.112:35512) 2023-08-27T11:27:57.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.693+0000 7fcb8c99e700 1 -- 172.21.15.112:0/1854542513 learned_addr learned my addr 172.21.15.112:0/1854542513 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:57.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.694+0000 7fcb877fe700 1 --2- 172.21.15.112:0/1854542513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb880cea00 0x7fcb880b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.694+0000 7fcb8c99e700 1 -- 172.21.15.112:0/1854542513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb880cea00 msgr2=0x7fcb880b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:57.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.694+0000 7fcb8c99e700 1 --2- 172.21.15.112:0/1854542513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb880cea00 0x7fcb880b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.694+0000 7fcb8c99e700 1 -- 172.21.15.112:0/1854542513 >> v1:172.21.15.138:6789/0 conn(0x7fcb88155390 legacy=0x7fcb8815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.694+0000 7fcb8c99e700 1 -- 172.21.15.112:0/1854542513 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb880b68e0 con 0x7fcb88155770 2023-08-27T11:27:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.694+0000 7fcb8c99e700 1 --2- 172.21.15.112:0/1854542513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb88155770 0x7fcb8806bdc0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009e50 tx=0x7fcb8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e5930b2e9d7711 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.694+0000 7fcb86ffd700 1 -- 172.21.15.112:0/1854542513 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000e070 con 0x7fcb88155770 2023-08-27T11:27:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.694+0000 7fcb86ffd700 1 -- 172.21.15.112:0/1854542513 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8000b9c0 con 0x7fcb88155770 2023-08-27T11:27:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.694+0000 7fcb86ffd700 1 -- 172.21.15.112:0/1854542513 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000bd70 con 0x7fcb88155770 2023-08-27T11:27:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.694+0000 7fcb8d19f700 1 -- 172.21.15.112:0/1854542513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb88155770 msgr2=0x7fcb8806bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.694+0000 7fcb8d19f700 1 --2- 172.21.15.112:0/1854542513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb88155770 0x7fcb8806bdc0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009e50 tx=0x7fcb8000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.695+0000 7fcb8d19f700 1 -- 172.21.15.112:0/1854542513 shutdown_connections 2023-08-27T11:27:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.695+0000 7fcb8d19f700 1 --2- 172.21.15.112:0/1854542513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb88155770 0x7fcb8806bdc0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.695+0000 7fcb8d19f700 1 --2- 172.21.15.112:0/1854542513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb880cea00 0x7fcb880b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.695+0000 7fcb8d19f700 1 -- 172.21.15.112:0/1854542513 >> 172.21.15.112:0/1854542513 conn(0x7fcb880666d0 msgr2=0x7fcb880d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.695+0000 7fcb8d19f700 1 -- 172.21.15.112:0/1854542513 shutdown_connections 2023-08-27T11:27:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.695+0000 7fcb8d19f700 1 -- 172.21.15.112:0/1854542513 wait complete. 2023-08-27T11:27:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.696+0000 7fcb8d19f700 1 Processor -- start 2023-08-27T11:27:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.696+0000 7fcb8d19f700 1 -- start start 2023-08-27T11:27:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb8d19f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb880cea00 0x7fcb880c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb8d19f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb88155390 0x7fcb8814e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb8d19f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88155770 0x7fcb880c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb8d19f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb881f3770 con 0x7fcb880cea00 2023-08-27T11:27:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb8d19f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880709b0 con 0x7fcb88155770 2023-08-27T11:27:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb8d19f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb88070c90 con 0x7fcb88155390 2023-08-27T11:27:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb87fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb880cea00 0x7fcb880c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb87fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb880cea00 0x7fcb880c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35514/0 (socket says 172.21.15.112:35514) 2023-08-27T11:27:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb87fff700 1 -- 172.21.15.112:0/3705851909 learned_addr learned my addr 172.21.15.112:0/3705851909 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb877fe700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb88155390 0x7fcb8814e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb87fff700 1 -- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb88155390 msgr2=0x7fcb8814e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb8c99e700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88155770 0x7fcb880c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb87fff700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb88155390 0x7fcb8814e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb87fff700 1 -- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88155770 msgr2=0x7fcb880c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb87fff700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88155770 0x7fcb880c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb87fff700 1 -- 172.21.15.112:0/3705851909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb80009b30 con 0x7fcb880cea00 2023-08-27T11:27:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.697+0000 7fcb877fe700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb88155390 0x7fcb8814e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.698+0000 7fcb87fff700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb880cea00 0x7fcb880c6100 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c4e0 tx=0x7fcb7800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.698+0000 7fcb84ff9700 1 -- 172.21.15.112:0/3705851909 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb780052d0 con 0x7fcb880cea00 2023-08-27T11:27:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.698+0000 7fcb84ff9700 1 -- 172.21.15.112:0/3705851909 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb78005430 con 0x7fcb880cea00 2023-08-27T11:27:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.698+0000 7fcb8d19f700 1 -- 172.21.15.112:0/3705851909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb88070f70 con 0x7fcb880cea00 2023-08-27T11:27:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.698+0000 7fcb84ff9700 1 -- 172.21.15.112:0/3705851909 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb780038c0 con 0x7fcb880cea00 2023-08-27T11:27:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.698+0000 7fcb8d19f700 1 -- 172.21.15.112:0/3705851909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb880c6640 con 0x7fcb880cea00 2023-08-27T11:27:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.699+0000 7fcb8d19f700 1 -- 172.21.15.112:0/3705851909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb880c6800 con 0x7fcb880cea00 2023-08-27T11:27:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.699+0000 7fcb84ff9700 1 -- 172.21.15.112:0/3705851909 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fcb78003a20 con 0x7fcb880cea00 2023-08-27T11:27:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.699+0000 7fcb84ff9700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcb6806a4d0 0x7fcb6806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.700+0000 7fcb84ff9700 1 -- 172.21.15.112:0/3705851909 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fcb780881a0 con 0x7fcb880cea00 2023-08-27T11:27:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.700+0000 7fcb877fe700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcb6806a4d0 0x7fcb6806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.700+0000 7fcb877fe700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcb6806a4d0 0x7fcb6806c990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcb881542c0 tx=0x7fcb70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.704+0000 7fcb84ff9700 1 -- 172.21.15.112:0/3705851909 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcb78053290 con 0x7fcb880cea00 2023-08-27T11:27:57.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.893+0000 7fcb8d19f700 1 -- 172.21.15.112:0/3705851909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb880d0900 con 0x7fcb880cea00 2023-08-27T11:27:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.894+0000 7fcb84ff9700 1 -- 172.21.15.112:0/3705851909 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+18538 (secure 0 0 0) 0x7fcb780568b0 con 0x7fcb880cea00 2023-08-27T11:27:57.896 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:57.896 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":127,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:27:52.546502+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:26.212595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:27.068136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6832/3832579050":"2023-08-28T11:27:49.540827+0000","172.21.15.112:6834/3832579050":"2023-08-28T11:27:49.540827+0000","172.21.15.112:6835/4019485943":"2023-08-28T11:27:46.133510+0000","172.21.15.112:6833/4019485943":"2023-08-28T11:27:46.133510+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:57.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.896+0000 7fcb8d19f700 1 -- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcb6806a4d0 msgr2=0x7fcb6806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.896+0000 7fcb8d19f700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcb6806a4d0 0x7fcb6806c990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcb881542c0 tx=0x7fcb70008040 comp rx=0 tx=0).stop 2023-08-27T11:27:57.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.896+0000 7fcb8d19f700 1 -- 172.21.15.112:0/3705851909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb880cea00 msgr2=0x7fcb880c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.896+0000 7fcb8d19f700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb880cea00 0x7fcb880c6100 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c4e0 tx=0x7fcb7800c8a0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.896+0000 7fcb8d19f700 1 -- 172.21.15.112:0/3705851909 shutdown_connections 2023-08-27T11:27:57.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.896+0000 7fcb8d19f700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcb6806a4d0 0x7fcb6806c990 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.896+0000 7fcb8d19f700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88155770 0x7fcb880c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.896+0000 7fcb8d19f700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb880cea00 0x7fcb880c6100 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.896+0000 7fcb8d19f700 1 --2- 172.21.15.112:0/3705851909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb88155390 0x7fcb8814e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.896+0000 7fcb8d19f700 1 -- 172.21.15.112:0/3705851909 >> 172.21.15.112:0/3705851909 conn(0x7fcb880666d0 msgr2=0x7fcb880b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.896+0000 7fcb8d19f700 1 -- 172.21.15.112:0/3705851909 shutdown_connections 2023-08-27T11:27:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:57.896+0000 7fcb8d19f700 1 -- 172.21.15.112:0/3705851909 wait complete. 2023-08-27T11:27:57.910 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:27:26.212595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:27:27.068136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:57.911 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:58.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.032+0000 7f024d110700 1 Processor -- start 2023-08-27T11:27:58.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.032+0000 7f024d110700 1 -- start start 2023-08-27T11:27:58.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024d110700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f02480b3de0 0x7f024806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024d110700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f024806c510 0x7f02480cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024d110700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02480cef20 con 0x7f0248144b80 2023-08-27T11:27:58.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024d110700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02480b4b40 con 0x7f024806c510 2023-08-27T11:27:58.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024d110700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02480b7310 con 0x7f02480b3de0 2023-08-27T11:27:58.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024759e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f024806c510 0x7f02480cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024759e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f024806c510 0x7f02480cf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35522/0 (socket says 172.21.15.112:35522) 2023-08-27T11:27:58.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024759e700 1 -- 172.21.15.112:0/2215010343 learned_addr learned my addr 172.21.15.112:0/2215010343 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:58.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024659c700 1 --2- 172.21.15.112:0/2215010343 >> v2:172.21.15.138:3300/0 conn(0x7f02480b3de0 0x7f024806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024759e700 1 -- 172.21.15.112:0/2215010343 >> v2:172.21.15.138:3300/0 conn(0x7f02480b3de0 msgr2=0x7f024806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:58.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024759e700 1 --2- 172.21.15.112:0/2215010343 >> v2:172.21.15.138:3300/0 conn(0x7f02480b3de0 0x7f024806bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024759e700 1 -- 172.21.15.112:0/2215010343 >> v1:172.21.15.138:6789/0 conn(0x7f0248144b80 legacy=0x7f02480b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024759e700 1 -- 172.21.15.112:0/2215010343 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024806c910 con 0x7f024806c510 2023-08-27T11:27:58.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.033+0000 7f024759e700 1 --2- 172.21.15.112:0/2215010343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f024806c510 0x7f02480cf3b0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f02480d0d50 tx=0x7f024000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12bf0c6f2e96b9f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:58.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.034+0000 7f0245d9b700 1 -- 172.21.15.112:0/2215010343 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024000e070 con 0x7f024806c510 2023-08-27T11:27:58.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.034+0000 7f0245d9b700 1 -- 172.21.15.112:0/2215010343 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024000b910 con 0x7f024806c510 2023-08-27T11:27:58.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.034+0000 7f0245d9b700 1 -- 172.21.15.112:0/2215010343 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024000bcc0 con 0x7f024806c510 2023-08-27T11:27:58.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.034+0000 7f024d110700 1 -- 172.21.15.112:0/2215010343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f024806c510 msgr2=0x7f02480cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.034+0000 7f024d110700 1 --2- 172.21.15.112:0/2215010343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f024806c510 0x7f02480cf3b0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f02480d0d50 tx=0x7f024000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:58.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.034+0000 7f024d110700 1 -- 172.21.15.112:0/2215010343 shutdown_connections 2023-08-27T11:27:58.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.034+0000 7f024d110700 1 --2- 172.21.15.112:0/2215010343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f024806c510 0x7f02480cf3b0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.034+0000 7f024d110700 1 --2- 172.21.15.112:0/2215010343 >> v2:172.21.15.138:3300/0 conn(0x7f02480b3de0 0x7f024806bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.034+0000 7f024d110700 1 -- 172.21.15.112:0/2215010343 >> 172.21.15.112:0/2215010343 conn(0x7f02480cdc50 msgr2=0x7f02480ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:58.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.034+0000 7f024d110700 1 -- 172.21.15.112:0/2215010343 shutdown_connections 2023-08-27T11:27:58.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.034+0000 7f024d110700 1 -- 172.21.15.112:0/2215010343 wait complete. 2023-08-27T11:27:58.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.036+0000 7f024d110700 1 Processor -- start 2023-08-27T11:27:58.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.036+0000 7f024d110700 1 -- start start 2023-08-27T11:27:58.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024d110700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f024806c510 0x7f0248071000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024d110700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02480b3de0 0x7f02481571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024d110700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248144b80 0x7f02481f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024d110700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02480728d0 con 0x7f0248144b80 2023-08-27T11:27:58.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024d110700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0248072440 con 0x7f02480b3de0 2023-08-27T11:27:58.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024d110700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0248071830 con 0x7f024806c510 2023-08-27T11:27:58.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248144b80 0x7f02481f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248144b80 0x7f02481f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35532/0 (socket says 172.21.15.112:35532) 2023-08-27T11:27:58.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024759e700 1 -- 172.21.15.112:0/1333651984 learned_addr learned my addr 172.21.15.112:0/1333651984 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:58.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024659c700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02480b3de0 0x7f02481571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f0246d9d700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f024806c510 0x7f0248071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024759e700 1 -- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f024806c510 msgr2=0x7f0248071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024759e700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f024806c510 0x7f0248071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024759e700 1 -- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02480b3de0 msgr2=0x7f02481571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024759e700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02480b3de0 0x7f02481571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.037+0000 7f024759e700 1 -- 172.21.15.112:0/1333651984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0240009b30 con 0x7f0248144b80 2023-08-27T11:27:58.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.038+0000 7f024759e700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248144b80 0x7f02481f63b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f02480741d0 tx=0x7f0240004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:58.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.038+0000 7f022f7fe700 1 -- 172.21.15.112:0/1333651984 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400099b0 con 0x7f0248144b80 2023-08-27T11:27:58.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.038+0000 7f022f7fe700 1 -- 172.21.15.112:0/1333651984 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02400092e0 con 0x7f0248144b80 2023-08-27T11:27:58.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.038+0000 7f024d110700 1 -- 172.21.15.112:0/1333651984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0248071b10 con 0x7f0248144b80 2023-08-27T11:27:58.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.038+0000 7f022f7fe700 1 -- 172.21.15.112:0/1333651984 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024001c640 con 0x7f0248144b80 2023-08-27T11:27:58.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.038+0000 7f024d110700 1 -- 172.21.15.112:0/1333651984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02480720d0 con 0x7f0248144b80 2023-08-27T11:27:58.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.039+0000 7f024d110700 1 -- 172.21.15.112:0/1333651984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0248072290 con 0x7f0248144b80 2023-08-27T11:27:58.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.039+0000 7f022f7fe700 1 -- 172.21.15.112:0/1333651984 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f024001c7a0 con 0x7f0248144b80 2023-08-27T11:27:58.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.040+0000 7f022f7fe700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f023406a4d0 0x7f023406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.040+0000 7f022f7fe700 1 -- 172.21.15.112:0/1333651984 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f024008ab30 con 0x7f0248144b80 2023-08-27T11:27:58.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.040+0000 7f0246d9d700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f023406a4d0 0x7f023406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.040+0000 7f0246d9d700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f023406a4d0 0x7f023406c990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f023c000f60 tx=0x7f023c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:58.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.044+0000 7f022f7fe700 1 -- 172.21.15.112:0/1333651984 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0240055b10 con 0x7f0248144b80 2023-08-27T11:27:58.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.232+0000 7f024d110700 1 -- 172.21.15.112:0/1333651984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f02480725b0 con 0x7f0248144b80 2023-08-27T11:27:58.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.232+0000 7f022f7fe700 1 -- 172.21.15.112:0/1333651984 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v127) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0240059130 con 0x7f0248144b80 2023-08-27T11:27:58.234 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:27:58.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.234+0000 7f024d110700 1 -- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f023406a4d0 msgr2=0x7f023406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.234+0000 7f024d110700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f023406a4d0 0x7f023406c990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f023c000f60 tx=0x7f023c009040 comp rx=0 tx=0).stop 2023-08-27T11:27:58.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.234+0000 7f024d110700 1 -- 172.21.15.112:0/1333651984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248144b80 msgr2=0x7f02481f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.234+0000 7f024d110700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248144b80 0x7f02481f63b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f02480741d0 tx=0x7f0240004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.234+0000 7f024d110700 1 -- 172.21.15.112:0/1333651984 shutdown_connections 2023-08-27T11:27:58.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.234+0000 7f024d110700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f023406a4d0 0x7f023406c990 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.234+0000 7f024d110700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248144b80 0x7f02481f63b0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.234+0000 7f024d110700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f024806c510 0x7f0248071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.234+0000 7f024d110700 1 --2- 172.21.15.112:0/1333651984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02480b3de0 0x7f02481571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.234+0000 7f024d110700 1 -- 172.21.15.112:0/1333651984 >> 172.21.15.112:0/1333651984 conn(0x7f02480cdc50 msgr2=0x7f02480b68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.234+0000 7f024d110700 1 -- 172.21.15.112:0/1333651984 shutdown_connections 2023-08-27T11:27:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.234+0000 7f024d110700 1 -- 172.21.15.112:0/1333651984 wait complete. 2023-08-27T11:27:58.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:58.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.369+0000 7f7515e86700 1 Processor -- start 2023-08-27T11:27:58.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.369+0000 7f7515e86700 1 -- start start 2023-08-27T11:27:58.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.370+0000 7f7515e86700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75100715c0 0x7f7510161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.370+0000 7f7515e86700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751006ce30 con 0x7f75100d1580 2023-08-27T11:27:58.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.370+0000 7f7515e86700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751006c870 con 0x7f7510146b30 2023-08-27T11:27:58.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.370+0000 7f7515e86700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751006cb50 con 0x7f75100715c0 2023-08-27T11:27:58.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.370+0000 7f750ffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f75100d1580 0x7f751015eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58154/0 (socket says 172.21.15.112:58154) 2023-08-27T11:27:58.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.370+0000 7f750ffff700 1 -- 172.21.15.112:0/3365040633 learned_addr learned my addr 172.21.15.112:0/3365040633 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:58.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.370+0000 7f750effd700 1 --2- 172.21.15.112:0/3365040633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75100715c0 0x7f7510161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.370+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4188128330 0 0) 0x7f751006ce30 con 0x7f75100d1580 2023-08-27T11:27:58.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.370+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74fc0035b0 con 0x7f75100d1580 2023-08-27T11:27:58.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1604610174 0 0) 0x7f74fc0035b0 con 0x7f75100d1580 2023-08-27T11:27:58.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75100715c0 msgr2=0x7f7510161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:58.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 --2- 172.21.15.112:0/3365040633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75100715c0 0x7f7510161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 >> v1:172.21.15.138:6789/0 conn(0x7f7510146b30 legacy=0x7f751006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7510161020 con 0x7f75100d1580 2023-08-27T11:27:58.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f75080019f0 con 0x7f75100d1580 2023-08-27T11:27:58.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 >> v1:172.21.15.112:6789/0 conn(0x7f75100d1580 legacy=0x7f751015eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 --2- 172.21.15.112:0/3365040633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75100715c0 0x7f74fc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 --2- 172.21.15.112:0/3365040633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc003fb0 0x7f74fc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 --2- 172.21.15.112:0/3365040633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74fc004d90 0x7f74fc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751006cb50 con 0x7f74fc003fb0 2023-08-27T11:27:58.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc00bba0 con 0x7f74fc004d90 2023-08-27T11:27:58.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc00bd60 con 0x7f75100715c0 2023-08-27T11:27:58.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750effd700 1 --2- 172.21.15.112:0/3365040633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc003fb0 0x7f74fc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750f7fe700 1 --2- 172.21.15.112:0/3365040633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75100715c0 0x7f74fc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750ffff700 1 --2- 172.21.15.112:0/3365040633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74fc004d90 0x7f74fc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750effd700 1 -- 172.21.15.112:0/3365040633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75100715c0 msgr2=0x7f74fc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:58.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750effd700 1 --2- 172.21.15.112:0/3365040633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75100715c0 0x7f74fc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750effd700 1 -- 172.21.15.112:0/3365040633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74fc004d90 msgr2=0x7f74fc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:58.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750effd700 1 --2- 172.21.15.112:0/3365040633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74fc004d90 0x7f74fc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750effd700 1 -- 172.21.15.112:0/3365040633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74fc0035b0 con 0x7f74fc003fb0 2023-08-27T11:27:58.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750ffff700 1 --2- 172.21.15.112:0/3365040633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74fc004d90 0x7f74fc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:58.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750effd700 1 --2- 172.21.15.112:0/3365040633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc003fb0 0x7f74fc004850 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7500009b30 tx=0x7f74fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409b2fe0965d0143 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:58.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750000e6c0 con 0x7f74fc003fb0 2023-08-27T11:27:58.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750000e820 con 0x7f74fc003fb0 2023-08-27T11:27:58.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.371+0000 7f750e7fc700 1 -- 172.21.15.112:0/3365040633 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750000eb90 con 0x7f74fc003fb0 2023-08-27T11:27:58.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.372+0000 7f7515e86700 1 -- 172.21.15.112:0/3365040633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc003fb0 msgr2=0x7f74fc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.372+0000 7f7515e86700 1 --2- 172.21.15.112:0/3365040633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc003fb0 0x7f74fc004850 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7500009b30 tx=0x7f74fc005880 comp rx=0 tx=0).stop 2023-08-27T11:27:58.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.372+0000 7f750ffff700 1 -- 172.21.15.112:0/3365040633 reap_dead start 2023-08-27T11:27:58.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.372+0000 7f7515e86700 1 -- 172.21.15.112:0/3365040633 shutdown_connections 2023-08-27T11:27:58.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.372+0000 7f7515e86700 1 -- 172.21.15.112:0/3365040633 >> 172.21.15.112:0/3365040633 conn(0x7f751006b1d0 msgr2=0x7f751006b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:58.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.372+0000 7f7515e86700 1 -- 172.21.15.112:0/3365040633 shutdown_connections 2023-08-27T11:27:58.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.372+0000 7f7515e86700 1 -- 172.21.15.112:0/3365040633 wait complete. 2023-08-27T11:27:58.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.373+0000 7f7515e86700 1 Processor -- start 2023-08-27T11:27:58.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.373+0000 7f7515e86700 1 -- start start 2023-08-27T11:27:58.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.374+0000 7f7515e86700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74fc003fb0 0x7f75100caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.374+0000 7f7515e86700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75100cb420 0x7f75100cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.374+0000 7f7515e86700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75100cce60 0x7f75101f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.374+0000 7f7515e86700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75100cb8a0 con 0x7f75100cce60 2023-08-27T11:27:58.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.374+0000 7f7515e86700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75100cd2f0 con 0x7f74fc003fb0 2023-08-27T11:27:58.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.374+0000 7f7515e86700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75100cba90 con 0x7f75100cb420 2023-08-27T11:27:58.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.374+0000 7f750ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75100cce60 0x7f75101f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.374+0000 7f750ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75100cce60 0x7f75101f27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35552/0 (socket says 172.21.15.112:35552) 2023-08-27T11:27:58.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.374+0000 7f750ffff700 1 -- 172.21.15.112:0/2450902344 learned_addr learned my addr 172.21.15.112:0/2450902344 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:58.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.374+0000 7f750effd700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75100cb420 0x7f75100cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.375+0000 7f750f7fe700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74fc003fb0 0x7f75100caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.375+0000 7f750ffff700 1 -- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75100cb420 msgr2=0x7f75100cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.375+0000 7f750ffff700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75100cb420 0x7f75100cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.375+0000 7f750ffff700 1 -- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74fc003fb0 msgr2=0x7f75100caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.375+0000 7f750ffff700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74fc003fb0 0x7f75100caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.375+0000 7f750ffff700 1 -- 172.21.15.112:0/2450902344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75100b4520 con 0x7f75100cce60 2023-08-27T11:27:58.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.375+0000 7f750ffff700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75100cce60 0x7f75101f27d0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f7510072260 tx=0x7f7508007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:58.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.375+0000 7f7507fff700 1 -- 172.21.15.112:0/2450902344 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750800ec60 con 0x7f75100cce60 2023-08-27T11:27:58.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.375+0000 7f7507fff700 1 -- 172.21.15.112:0/2450902344 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750800edc0 con 0x7f75100cce60 2023-08-27T11:27:58.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.375+0000 7f7507fff700 1 -- 172.21.15.112:0/2450902344 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508013660 con 0x7f75100cce60 2023-08-27T11:27:58.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.375+0000 7f7515e86700 1 -- 172.21.15.112:0/2450902344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75100b4800 con 0x7f75100cce60 2023-08-27T11:27:58.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.375+0000 7f7515e86700 1 -- 172.21.15.112:0/2450902344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75100b4dc0 con 0x7f75100cce60 2023-08-27T11:27:58.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.376+0000 7f7515e86700 1 -- 172.21.15.112:0/2450902344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7510067a20 con 0x7f75100cce60 2023-08-27T11:27:58.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.376+0000 7f7507fff700 1 -- 172.21.15.112:0/2450902344 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f75080137c0 con 0x7f75100cce60 2023-08-27T11:27:58.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.377+0000 7f7507fff700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f74f006a4d0 0x7f74f006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.377+0000 7f7507fff700 1 -- 172.21.15.112:0/2450902344 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f7508089350 con 0x7f75100cce60 2023-08-27T11:27:58.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.377+0000 7f750f7fe700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f74f006a4d0 0x7f74f006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.377+0000 7f750f7fe700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f74f006a4d0 0x7f74f006c990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f74f8007fd0 tx=0x7f74f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:58.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.381+0000 7f7507fff700 1 -- 172.21.15.112:0/2450902344 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7508054460 con 0x7f75100cce60 2023-08-27T11:27:58.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.571+0000 7f7515e86700 1 -- 172.21.15.112:0/2450902344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75100b07c0 con 0x7f75100cce60 2023-08-27T11:27:58.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.572+0000 7f7507fff700 1 -- 172.21.15.112:0/2450902344 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7508057a80 con 0x7f75100cce60 2023-08-27T11:27:58.574 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:27:58.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.573+0000 7f7515e86700 1 -- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f74f006a4d0 msgr2=0x7f74f006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.573+0000 7f7515e86700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f74f006a4d0 0x7f74f006c990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f74f8007fd0 tx=0x7f74f8009040 comp rx=0 tx=0).stop 2023-08-27T11:27:58.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.573+0000 7f7515e86700 1 -- 172.21.15.112:0/2450902344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75100cce60 msgr2=0x7f75101f27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.573+0000 7f7515e86700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75100cce60 0x7f75101f27d0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f7510072260 tx=0x7f7508007c90 comp rx=0 tx=0).stop 2023-08-27T11:27:58.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.574+0000 7f7515e86700 1 -- 172.21.15.112:0/2450902344 shutdown_connections 2023-08-27T11:27:58.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.574+0000 7f7515e86700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f74f006a4d0 0x7f74f006c990 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.574+0000 7f7515e86700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75100cce60 0x7f75101f27d0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.574+0000 7f7515e86700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74fc003fb0 0x7f75100caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.574+0000 7f7515e86700 1 --2- 172.21.15.112:0/2450902344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75100cb420 0x7f75100cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.574+0000 7f7515e86700 1 -- 172.21.15.112:0/2450902344 >> 172.21.15.112:0/2450902344 conn(0x7f751006b1d0 msgr2=0x7f75100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:58.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.574+0000 7f7515e86700 1 -- 172.21.15.112:0/2450902344 shutdown_connections 2023-08-27T11:27:58.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.574+0000 7f7515e86700 1 -- 172.21.15.112:0/2450902344 wait complete. 2023-08-27T11:27:58.587 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:27:58.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.709+0000 7f5de9ed8700 1 Processor -- start 2023-08-27T11:27:58.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.709+0000 7f5de9ed8700 1 -- start start 2023-08-27T11:27:58.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de9ed8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5de4144b80 0x7f5de40b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de9ed8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5de406c6e0 0x7f5de40cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de9ed8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de40cef20 con 0x7f5de40b3de0 2023-08-27T11:27:58.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de9ed8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de40b4b40 con 0x7f5de406c6e0 2023-08-27T11:27:58.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de9ed8700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de40b42a0 con 0x7f5de4144b80 2023-08-27T11:27:58.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de3fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5de406c6e0 0x7f5de40cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de2ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5de40b3de0 0x7f5de406bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58172/0 (socket says 172.21.15.112:58172) 2023-08-27T11:27:58.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de2ffd700 1 -- 172.21.15.112:0/3887578586 learned_addr learned my addr 172.21.15.112:0/3887578586 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:58.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de37fe700 1 --2- 172.21.15.112:0/3887578586 >> v2:172.21.15.138:3300/0 conn(0x7f5de4144b80 0x7f5de40b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de3fff700 1 -- 172.21.15.112:0/3887578586 >> v2:172.21.15.138:3300/0 conn(0x7f5de4144b80 msgr2=0x7f5de40b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:58.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de3fff700 1 --2- 172.21.15.112:0/3887578586 >> v2:172.21.15.138:3300/0 conn(0x7f5de4144b80 0x7f5de40b6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de3fff700 1 -- 172.21.15.112:0/3887578586 >> v1:172.21.15.112:6789/0 conn(0x7f5de40b3de0 legacy=0x7f5de406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de3fff700 1 -- 172.21.15.112:0/3887578586 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de40b4580 con 0x7f5de406c6e0 2023-08-27T11:27:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.710+0000 7f5de3fff700 1 --2- 172.21.15.112:0/3887578586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de406c6e0 0x7f5de40cf3b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f5de40695a0 tx=0x7f5ddc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171b5b76adb6aec1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.711+0000 7f5de27fc700 1 -- 172.21.15.112:0/3887578586 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc00e070 con 0x7f5de406c6e0 2023-08-27T11:27:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.711+0000 7f5de27fc700 1 -- 172.21.15.112:0/3887578586 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ddc00bbd0 con 0x7f5de406c6e0 2023-08-27T11:27:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.711+0000 7f5de27fc700 1 -- 172.21.15.112:0/3887578586 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc012430 con 0x7f5de406c6e0 2023-08-27T11:27:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.711+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3887578586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de406c6e0 msgr2=0x7f5de40cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.711+0000 7f5de9ed8700 1 --2- 172.21.15.112:0/3887578586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de406c6e0 0x7f5de40cf3b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f5de40695a0 tx=0x7f5ddc00b330 comp rx=0 tx=0).stop 2023-08-27T11:27:58.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.711+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3887578586 shutdown_connections 2023-08-27T11:27:58.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.711+0000 7f5de9ed8700 1 --2- 172.21.15.112:0/3887578586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de406c6e0 0x7f5de40cf3b0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.711+0000 7f5de9ed8700 1 --2- 172.21.15.112:0/3887578586 >> v2:172.21.15.138:3300/0 conn(0x7f5de4144b80 0x7f5de40b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.711+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3887578586 >> 172.21.15.112:0/3887578586 conn(0x7f5de40cdc50 msgr2=0x7f5de40ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:58.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.711+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3887578586 shutdown_connections 2023-08-27T11:27:58.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.711+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3887578586 wait complete. 2023-08-27T11:27:58.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.712+0000 7f5de9ed8700 1 Processor -- start 2023-08-27T11:27:58.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.713+0000 7f5de9ed8700 1 -- start start 2023-08-27T11:27:58.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.713+0000 7f5de9ed8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de406c6e0 0x7f5de40cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.713+0000 7f5de9ed8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de40b3de0 0x7f5de40708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.713+0000 7f5de9ed8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5de4144b80 0x7f5de41436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.713+0000 7f5de9ed8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de4157f30 con 0x7f5de406c6e0 2023-08-27T11:27:58.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.713+0000 7f5de9ed8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de4071340 con 0x7f5de4144b80 2023-08-27T11:27:58.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.713+0000 7f5de9ed8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de4071620 con 0x7f5de40b3de0 2023-08-27T11:27:58.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.713+0000 7f5de37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de406c6e0 0x7f5de40cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.713+0000 7f5de37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de406c6e0 0x7f5de40cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35562/0 (socket says 172.21.15.112:35562) 2023-08-27T11:27:58.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.713+0000 7f5de37fe700 1 -- 172.21.15.112:0/3696642170 learned_addr learned my addr 172.21.15.112:0/3696642170 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:58.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.713+0000 7f5de2ffd700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de40b3de0 0x7f5de40708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5de3fff700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5de4144b80 0x7f5de41436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5de37fe700 1 -- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de40b3de0 msgr2=0x7f5de40708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5de37fe700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de40b3de0 0x7f5de40708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5de37fe700 1 -- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5de4144b80 msgr2=0x7f5de41436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5de37fe700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5de4144b80 0x7f5de41436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5de37fe700 1 -- 172.21.15.112:0/3696642170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc009a70 con 0x7f5de406c6e0 2023-08-27T11:27:58.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5de3fff700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5de4144b80 0x7f5de41436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:58.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5de2ffd700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de40b3de0 0x7f5de40708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:58.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5de37fe700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de406c6e0 0x7f5de40cbf50 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400d800 tx=0x7f5dd400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:58.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5dcbfff700 1 -- 172.21.15.112:0/3696642170 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40125f0 con 0x7f5de406c6e0 2023-08-27T11:27:58.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5dcbfff700 1 -- 172.21.15.112:0/3696642170 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd4012750 con 0x7f5de406c6e0 2023-08-27T11:27:58.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3696642170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de4071900 con 0x7f5de406c6e0 2023-08-27T11:27:58.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5dcbfff700 1 -- 172.21.15.112:0/3696642170 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4012b00 con 0x7f5de406c6e0 2023-08-27T11:27:58.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.714+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3696642170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de4071ec0 con 0x7f5de406c6e0 2023-08-27T11:27:58.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.715+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3696642170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de4072230 con 0x7f5de406c6e0 2023-08-27T11:27:58.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.715+0000 7f5dcbfff700 1 -- 172.21.15.112:0/3696642170 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5dd4012c60 con 0x7f5de406c6e0 2023-08-27T11:27:58.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.716+0000 7f5dcbfff700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5dd006a4d0 0x7f5dd006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.716+0000 7f5dcbfff700 1 -- 172.21.15.112:0/3696642170 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f5dd4089100 con 0x7f5de406c6e0 2023-08-27T11:27:58.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.716+0000 7f5de2ffd700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5dd006a4d0 0x7f5dd006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.717+0000 7f5de2ffd700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5dd006a4d0 0x7f5dd006c990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8007fd0 tx=0x7f5dd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:58.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.720+0000 7f5dcbfff700 1 -- 172.21.15.112:0/3696642170 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5dd4054160 con 0x7f5de406c6e0 2023-08-27T11:27:58.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.909+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3696642170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5de40686a0 con 0x7f5de406c6e0 2023-08-27T11:27:58.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.909+0000 7f5dcbfff700 1 -- 172.21.15.112:0/3696642170 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5dd4057780 con 0x7f5de406c6e0 2023-08-27T11:27:58.912 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:27:58.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.911+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5dd006a4d0 msgr2=0x7f5dd006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.911+0000 7f5de9ed8700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5dd006a4d0 0x7f5dd006c990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8007fd0 tx=0x7f5dd8009040 comp rx=0 tx=0).stop 2023-08-27T11:27:58.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.911+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3696642170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de406c6e0 msgr2=0x7f5de40cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.911+0000 7f5de9ed8700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de406c6e0 0x7f5de40cbf50 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400d800 tx=0x7f5dd400dbc0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.911+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3696642170 shutdown_connections 2023-08-27T11:27:58.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.911+0000 7f5de9ed8700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5dd006a4d0 0x7f5dd006c990 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.912+0000 7f5de9ed8700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5de4144b80 0x7f5de41436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.912+0000 7f5de9ed8700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de406c6e0 0x7f5de40cbf50 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.912+0000 7f5de9ed8700 1 --2- 172.21.15.112:0/3696642170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de40b3de0 0x7f5de40708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.912+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3696642170 >> 172.21.15.112:0/3696642170 conn(0x7f5de40cdc50 msgr2=0x7f5de40b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:58.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.912+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3696642170 shutdown_connections 2023-08-27T11:27:58.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:58.912+0000 7f5de9ed8700 1 -- 172.21.15.112:0/3696642170 wait complete. 2023-08-27T11:27:58.925 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:27:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.047+0000 7f4b112b0700 1 Processor -- start 2023-08-27T11:27:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.047+0000 7f4b112b0700 1 -- start start 2023-08-27T11:27:59.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.047+0000 7f4b112b0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c0ce210 0x7f4b0c15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.047+0000 7f4b112b0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4b0c0ce5f0 0x7f4b0c0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.047+0000 7f4b112b0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b0c0cb500 con 0x7f4b0c06c470 2023-08-27T11:27:59.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.047+0000 7f4b112b0700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b0c15ded0 con 0x7f4b0c0ce5f0 2023-08-27T11:27:59.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.047+0000 7f4b112b0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c0cdb50 con 0x7f4b0c0ce210 2023-08-27T11:27:59.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.047+0000 7f4b0a7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4b0c0ce5f0 0x7f4b0c0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.047+0000 7f4b0b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4b0c06c470 0x7f4b0c0c99e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58188/0 (socket says 172.21.15.112:58188) 2023-08-27T11:27:59.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.048+0000 7f4b0b7fe700 1 -- 172.21.15.112:0/3853113366 learned_addr learned my addr 172.21.15.112:0/3853113366 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:27:59.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.048+0000 7f4b0a7fc700 1 -- 172.21.15.112:0/3853113366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c0ce210 msgr2=0x7f4b0c15aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:59.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.048+0000 7f4b0affd700 1 --2- 172.21.15.112:0/3853113366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c0ce210 0x7f4b0c15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.048+0000 7f4b0a7fc700 1 --2- 172.21.15.112:0/3853113366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c0ce210 0x7f4b0c15aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.048+0000 7f4b09ffb700 1 -- 172.21.15.112:0/3853113366 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2706591409 0 0) 0x7f4b0c0cb500 con 0x7f4b0c06c470 2023-08-27T11:27:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.048+0000 7f4b0a7fc700 1 -- 172.21.15.112:0/3853113366 >> v1:172.21.15.112:6789/0 conn(0x7f4b0c06c470 legacy=0x7f4b0c0c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.048+0000 7f4b0a7fc700 1 -- 172.21.15.112:0/3853113366 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c0cb7b0 con 0x7f4b0c0ce5f0 2023-08-27T11:27:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.048+0000 7f4b0affd700 1 --2- 172.21.15.112:0/3853113366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c0ce210 0x7f4b0c15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.048+0000 7f4b0a7fc700 1 --2- 172.21.15.112:0/3853113366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c0ce5f0 0x7f4b0c0cafc0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009840 tx=0x7f4afc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcc306bb9cdb3d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.048+0000 7f4b09ffb700 1 -- 172.21.15.112:0/3853113366 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc011830 con 0x7f4b0c0ce5f0 2023-08-27T11:27:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.048+0000 7f4b09ffb700 1 -- 172.21.15.112:0/3853113366 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc011990 con 0x7f4b0c0ce5f0 2023-08-27T11:27:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.048+0000 7f4b09ffb700 1 -- 172.21.15.112:0/3853113366 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc011d20 con 0x7f4b0c0ce5f0 2023-08-27T11:27:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.049+0000 7f4b112b0700 1 -- 172.21.15.112:0/3853113366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c0ce5f0 msgr2=0x7f4b0c0cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.049+0000 7f4b112b0700 1 --2- 172.21.15.112:0/3853113366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c0ce5f0 0x7f4b0c0cafc0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009840 tx=0x7f4afc009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.049+0000 7f4b112b0700 1 -- 172.21.15.112:0/3853113366 shutdown_connections 2023-08-27T11:27:59.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.049+0000 7f4b112b0700 1 --2- 172.21.15.112:0/3853113366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c0ce210 0x7f4b0c15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.049+0000 7f4b112b0700 1 --2- 172.21.15.112:0/3853113366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c0ce5f0 0x7f4b0c0cafc0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.049+0000 7f4b112b0700 1 -- 172.21.15.112:0/3853113366 >> 172.21.15.112:0/3853113366 conn(0x7f4b0c0cef70 msgr2=0x7f4b0c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:59.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.049+0000 7f4b112b0700 1 -- 172.21.15.112:0/3853113366 shutdown_connections 2023-08-27T11:27:59.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.049+0000 7f4b112b0700 1 -- 172.21.15.112:0/3853113366 wait complete. 2023-08-27T11:27:59.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.050+0000 7f4b112b0700 1 Processor -- start 2023-08-27T11:27:59.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.050+0000 7f4b112b0700 1 -- start start 2023-08-27T11:27:59.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b112b0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c06c470 0x7f4b0c1efa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b112b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c0ce210 0x7f4b0c0a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b112b0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c0ce5f0 0x7f4b0c147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b112b0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c1effd0 con 0x7f4b0c0ce210 2023-08-27T11:27:59.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b112b0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c0a8d10 con 0x7f4b0c0ce5f0 2023-08-27T11:27:59.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b112b0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c0a8ff0 con 0x7f4b0c06c470 2023-08-27T11:27:59.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b0a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c0ce210 0x7f4b0c0a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b0a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c0ce210 0x7f4b0c0a8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35578/0 (socket says 172.21.15.112:35578) 2023-08-27T11:27:59.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b0a7fc700 1 -- 172.21.15.112:0/651020665 learned_addr learned my addr 172.21.15.112:0/651020665 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:59.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b0affd700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c06c470 0x7f4b0c1efa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b0b7fe700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c0ce5f0 0x7f4b0c147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b0a7fc700 1 -- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c06c470 msgr2=0x7f4b0c1efa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b0a7fc700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c06c470 0x7f4b0c1efa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b0a7fc700 1 -- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c0ce5f0 msgr2=0x7f4b0c147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b0a7fc700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c0ce5f0 0x7f4b0c147670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b0a7fc700 1 -- 172.21.15.112:0/651020665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc0094a0 con 0x7f4b0c0ce210 2023-08-27T11:27:59.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b0b7fe700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c0ce5f0 0x7f4b0c147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:59.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.051+0000 7f4b0a7fc700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c0ce210 0x7f4b0c0a8480 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4afc000c00 tx=0x7f4afc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:59.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.052+0000 7f4b037fe700 1 -- 172.21.15.112:0/651020665 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc0036a0 con 0x7f4b0c0ce210 2023-08-27T11:27:59.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.052+0000 7f4b037fe700 1 -- 172.21.15.112:0/651020665 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc01e070 con 0x7f4b0c0ce210 2023-08-27T11:27:59.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.052+0000 7f4b112b0700 1 -- 172.21.15.112:0/651020665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0c0a92d0 con 0x7f4b0c0ce210 2023-08-27T11:27:59.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.052+0000 7f4b037fe700 1 -- 172.21.15.112:0/651020665 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc003a40 con 0x7f4b0c0ce210 2023-08-27T11:27:59.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.052+0000 7f4b112b0700 1 -- 172.21.15.112:0/651020665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0c0a9890 con 0x7f4b0c0ce210 2023-08-27T11:27:59.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.052+0000 7f4b112b0700 1 -- 172.21.15.112:0/651020665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0c067a20 con 0x7f4b0c0ce210 2023-08-27T11:27:59.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.053+0000 7f4b037fe700 1 -- 172.21.15.112:0/651020665 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4afc003ba0 con 0x7f4b0c0ce210 2023-08-27T11:27:59.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.053+0000 7f4b037fe700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4aec06a4d0 0x7f4aec06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.053+0000 7f4b037fe700 1 -- 172.21.15.112:0/651020665 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f4afc089ce0 con 0x7f4b0c0ce210 2023-08-27T11:27:59.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.054+0000 7f4b0affd700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4aec06a4d0 0x7f4aec06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.054+0000 7f4b0affd700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4aec06a4d0 0x7f4aec06c990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4af4006fd0 tx=0x7f4af4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:59.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.058+0000 7f4b037fe700 1 -- 172.21.15.112:0/651020665 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4afc054ee0 con 0x7f4b0c0ce210 2023-08-27T11:27:59.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.247+0000 7f4b112b0700 1 -- 172.21.15.112:0/651020665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b0c0b2d10 con 0x7f4b0c0ce210 2023-08-27T11:27:59.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.248+0000 7f4b037fe700 1 -- 172.21.15.112:0/651020665 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4afc0150a0 con 0x7f4b0c0ce210 2023-08-27T11:27:59.249 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:27:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.250+0000 7f4b112b0700 1 -- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4aec06a4d0 msgr2=0x7f4aec06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.250+0000 7f4b112b0700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4aec06a4d0 0x7f4aec06c990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4af4006fd0 tx=0x7f4af4008040 comp rx=0 tx=0).stop 2023-08-27T11:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.250+0000 7f4b112b0700 1 -- 172.21.15.112:0/651020665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c0ce210 msgr2=0x7f4b0c0a8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.250+0000 7f4b112b0700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c0ce210 0x7f4b0c0a8480 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4afc000c00 tx=0x7f4afc004750 comp rx=0 tx=0).stop 2023-08-27T11:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.250+0000 7f4b112b0700 1 -- 172.21.15.112:0/651020665 shutdown_connections 2023-08-27T11:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.250+0000 7f4b112b0700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4aec06a4d0 0x7f4aec06c990 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.250+0000 7f4b112b0700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c0ce5f0 0x7f4b0c147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.250+0000 7f4b112b0700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c06c470 0x7f4b0c1efa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.250+0000 7f4b112b0700 1 --2- 172.21.15.112:0/651020665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c0ce210 0x7f4b0c0a8480 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.250+0000 7f4b112b0700 1 -- 172.21.15.112:0/651020665 >> 172.21.15.112:0/651020665 conn(0x7f4b0c0cef70 msgr2=0x7f4b0c158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.250+0000 7f4b112b0700 1 -- 172.21.15.112:0/651020665 shutdown_connections 2023-08-27T11:27:59.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.250+0000 7f4b112b0700 1 -- 172.21.15.112:0/651020665 wait complete. 2023-08-27T11:27:59.264 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:59.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.383+0000 7f4e76a96700 1 Processor -- start 2023-08-27T11:27:59.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.383+0000 7f4e76a96700 1 -- start start 2023-08-27T11:27:59.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e76a96700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700b1400 0x7f4e700cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e76a96700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4e70161ce0 0x7f4e7006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e76a96700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e70161850 con 0x7f4e700b17e0 2023-08-27T11:27:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e76a96700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e700cfc10 con 0x7f4e70161ce0 2023-08-27T11:27:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e76a96700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e70161200 con 0x7f4e700b1400 2023-08-27T11:27:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e74832700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700b1400 0x7f4e700cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e75033700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4e70161ce0 0x7f4e7006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e74832700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700b1400 0x7f4e700cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:47238/0 (socket says 172.21.15.112:47238) 2023-08-27T11:27:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e74832700 1 -- 172.21.15.112:0/1614418796 learned_addr learned my addr 172.21.15.112:0/1614418796 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e6ffff700 1 --1- 172.21.15.112:0/1614418796 >> v1:172.21.15.138:6789/0 conn(0x7f4e700b17e0 0x7f4e7006e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:37490/0 (socket says 172.21.15.112:37490) 2023-08-27T11:27:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e74832700 1 -- 172.21.15.112:0/1614418796 >> v2:172.21.15.138:3300/0 conn(0x7f4e70161ce0 msgr2=0x7f4e7006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e74832700 1 --2- 172.21.15.112:0/1614418796 >> v2:172.21.15.138:3300/0 conn(0x7f4e70161ce0 0x7f4e7006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e74832700 1 -- 172.21.15.112:0/1614418796 >> v1:172.21.15.138:6789/0 conn(0x7f4e700b17e0 legacy=0x7f4e7006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.384+0000 7f4e74832700 1 -- 172.21.15.112:0/1614418796 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e701614e0 con 0x7f4e700b1400 2023-08-27T11:27:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.385+0000 7f4e74832700 1 --2- 172.21.15.112:0/1614418796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700b1400 0x7f4e700cf5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e640098f0 tx=0x7f4e64009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aef58f643b5e5f8f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.385+0000 7f4e6f7fe700 1 -- 172.21.15.112:0/1614418796 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64011830 con 0x7f4e700b1400 2023-08-27T11:27:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.385+0000 7f4e6f7fe700 1 -- 172.21.15.112:0/1614418796 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e64011990 con 0x7f4e700b1400 2023-08-27T11:27:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.385+0000 7f4e6f7fe700 1 -- 172.21.15.112:0/1614418796 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64011d20 con 0x7f4e700b1400 2023-08-27T11:27:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.385+0000 7f4e76a96700 1 -- 172.21.15.112:0/1614418796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700b1400 msgr2=0x7f4e700cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.385+0000 7f4e76a96700 1 --2- 172.21.15.112:0/1614418796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700b1400 0x7f4e700cf5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e640098f0 tx=0x7f4e64009c00 comp rx=0 tx=0).stop 2023-08-27T11:27:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.385+0000 7f4e76a96700 1 -- 172.21.15.112:0/1614418796 shutdown_connections 2023-08-27T11:27:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.385+0000 7f4e76a96700 1 --2- 172.21.15.112:0/1614418796 >> v2:172.21.15.138:3300/0 conn(0x7f4e70161ce0 0x7f4e7006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.385+0000 7f4e76a96700 1 --2- 172.21.15.112:0/1614418796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700b1400 0x7f4e700cf5a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.385+0000 7f4e76a96700 1 -- 172.21.15.112:0/1614418796 >> 172.21.15.112:0/1614418796 conn(0x7f4e700d0000 msgr2=0x7f4e700d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:59.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.385+0000 7f4e76a96700 1 -- 172.21.15.112:0/1614418796 shutdown_connections 2023-08-27T11:27:59.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.385+0000 7f4e76a96700 1 -- 172.21.15.112:0/1614418796 wait complete. 2023-08-27T11:27:59.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.387+0000 7f4e76a96700 1 Processor -- start 2023-08-27T11:27:59.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.387+0000 7f4e76a96700 1 -- start start 2023-08-27T11:27:59.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.387+0000 7f4e76a96700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b1400 0x7f4e700c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.387+0000 7f4e76a96700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700b17e0 0x7f4e700c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.387+0000 7f4e76a96700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e70161ce0 0x7f4e7006ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.387+0000 7f4e76a96700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700ca300 con 0x7f4e700b1400 2023-08-27T11:27:59.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.387+0000 7f4e76a96700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700c8c20 con 0x7f4e70161ce0 2023-08-27T11:27:59.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.387+0000 7f4e76a96700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700c7b80 con 0x7f4e700b17e0 2023-08-27T11:27:59.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e74832700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b1400 0x7f4e700c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e74832700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b1400 0x7f4e700c60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35590/0 (socket says 172.21.15.112:35590) 2023-08-27T11:27:59.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e74832700 1 -- 172.21.15.112:0/4242709761 learned_addr learned my addr 172.21.15.112:0/4242709761 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:59.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e6ffff700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700b17e0 0x7f4e700c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e75033700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e70161ce0 0x7f4e7006ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e74832700 1 -- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700b17e0 msgr2=0x7f4e700c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e74832700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700b17e0 0x7f4e700c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e74832700 1 -- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e70161ce0 msgr2=0x7f4e7006ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e74832700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e70161ce0 0x7f4e7006ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e74832700 1 -- 172.21.15.112:0/4242709761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e64009550 con 0x7f4e700b1400 2023-08-27T11:27:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e75033700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e70161ce0 0x7f4e7006ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e74832700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b1400 0x7f4e700c60e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000c00 tx=0x7f4e64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e6d7fa700 1 -- 172.21.15.112:0/4242709761 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64004480 con 0x7f4e700b1400 2023-08-27T11:27:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e6d7fa700 1 -- 172.21.15.112:0/4242709761 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6401f070 con 0x7f4e700b1400 2023-08-27T11:27:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e76a96700 1 -- 172.21.15.112:0/4242709761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e700c7e60 con 0x7f4e700b1400 2023-08-27T11:27:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e6d7fa700 1 -- 172.21.15.112:0/4242709761 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e640038c0 con 0x7f4e700b1400 2023-08-27T11:27:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.388+0000 7f4e76a96700 1 -- 172.21.15.112:0/4242709761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e700c8420 con 0x7f4e700b1400 2023-08-27T11:27:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.389+0000 7f4e76a96700 1 -- 172.21.15.112:0/4242709761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e70067a20 con 0x7f4e700b1400 2023-08-27T11:27:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.390+0000 7f4e6d7fa700 1 -- 172.21.15.112:0/4242709761 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4e64021030 con 0x7f4e700b1400 2023-08-27T11:27:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.390+0000 7f4e6d7fa700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e5406a4d0 0x7f4e5406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.390+0000 7f4e6d7fa700 1 -- 172.21.15.112:0/4242709761 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f4e6408a020 con 0x7f4e700b1400 2023-08-27T11:27:59.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.390+0000 7f4e6ffff700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e5406a4d0 0x7f4e5406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.391+0000 7f4e6ffff700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e5406a4d0 0x7f4e5406c990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4e700cb0b0 tx=0x7f4e5c007ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:59.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.395+0000 7f4e6d7fa700 1 -- 172.21.15.112:0/4242709761 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4e64055220 con 0x7f4e700b1400 2023-08-27T11:27:59.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.629+0000 7f4e76a96700 1 -- 172.21.15.112:0/4242709761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e7006db60 con 0x7f4e700b1400 2023-08-27T11:27:59.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.629+0000 7f4e6d7fa700 1 -- 172.21.15.112:0/4242709761 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4889 (secure 0 0 0) 0x7f4e64015090 con 0x7f4e700b1400 2023-08-27T11:27:59.631 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:59.631 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1795721853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1795721853},{"type":"v1","addr":"172.21.15.138:6836","nonce":1795721853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":8184,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2658811115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2658811115},{"type":"v1","addr":"172.21.15.112:6835","nonce":2658811115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":8199,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3536640393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3536640393},{"type":"v1","addr":"172.21.15.112:6834","nonce":3536640393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:27:28.107305+0000","modified":"2023-08-27T11:27:31.742912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7989},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7989":{"gid":7989,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/3245198376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3245198376},{"type":"v1","addr":"172.21.15.138:6837","nonce":3245198376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-27T11:27:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.631+0000 7f4e76a96700 1 -- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e5406a4d0 msgr2=0x7f4e5406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.631+0000 7f4e76a96700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e5406a4d0 0x7f4e5406c990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4e700cb0b0 tx=0x7f4e5c007ed0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.631+0000 7f4e76a96700 1 -- 172.21.15.112:0/4242709761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b1400 msgr2=0x7f4e700c60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.631+0000 7f4e76a96700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b1400 0x7f4e700c60e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000c00 tx=0x7f4e64004750 comp rx=0 tx=0).stop 2023-08-27T11:27:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.631+0000 7f4e76a96700 1 -- 172.21.15.112:0/4242709761 shutdown_connections 2023-08-27T11:27:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.631+0000 7f4e76a96700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e5406a4d0 0x7f4e5406c990 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.631+0000 7f4e76a96700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e70161ce0 0x7f4e7006ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.631+0000 7f4e76a96700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b1400 0x7f4e700c60e0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.631+0000 7f4e76a96700 1 --2- 172.21.15.112:0/4242709761 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e700b17e0 0x7f4e700c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.631+0000 7f4e76a96700 1 -- 172.21.15.112:0/4242709761 >> 172.21.15.112:0/4242709761 conn(0x7f4e700d0000 msgr2=0x7f4e700d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.631+0000 7f4e76a96700 1 -- 172.21.15.112:0/4242709761 shutdown_connections 2023-08-27T11:27:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.631+0000 7f4e76a96700 1 -- 172.21.15.112:0/4242709761 wait complete. 2023-08-27T11:27:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 86 2023-08-27T11:27:59.648 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:59.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.768+0000 7fbe7d572700 1 Processor -- start 2023-08-27T11:27:59.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.769+0000 7fbe7d572700 1 -- start start 2023-08-27T11:27:59.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.769+0000 7fbe7d572700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe78146be0 0x7fbe780b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.769+0000 7fbe7d572700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbe781619b0 0x7fbe780b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.769+0000 7fbe7d572700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbe780b52a0 0x7fbe7815d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.769+0000 7fbe7d572700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe780b3980 con 0x7fbe780b52a0 2023-08-27T11:27:59.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.769+0000 7fbe7d572700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe780b5730 con 0x7fbe781619b0 2023-08-27T11:27:59.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.769+0000 7fbe7d572700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe78161e30 con 0x7fbe78146be0 2023-08-27T11:27:59.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.769+0000 7fbe777fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbe780b52a0 0x7fbe7815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.769+0000 7fbe777fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbe780b52a0 0x7fbe7815d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35602/0 (socket says 172.21.15.112:35602) 2023-08-27T11:27:59.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.770+0000 7fbe777fe700 1 -- 172.21.15.112:0/889292335 learned_addr learned my addr 172.21.15.112:0/889292335 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.770+0000 7fbe767fc700 1 --2- 172.21.15.112:0/889292335 >> v2:172.21.15.138:3300/0 conn(0x7fbe781619b0 0x7fbe780b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.770+0000 7fbe76ffd700 1 --2- 172.21.15.112:0/889292335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe78146be0 0x7fbe780b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.770+0000 7fbe777fe700 1 -- 172.21.15.112:0/889292335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe78146be0 msgr2=0x7fbe780b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.770+0000 7fbe777fe700 1 --2- 172.21.15.112:0/889292335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe78146be0 0x7fbe780b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.770+0000 7fbe777fe700 1 -- 172.21.15.112:0/889292335 >> v2:172.21.15.138:3300/0 conn(0x7fbe781619b0 msgr2=0x7fbe780b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:59.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.770+0000 7fbe777fe700 1 --2- 172.21.15.112:0/889292335 >> v2:172.21.15.138:3300/0 conn(0x7fbe781619b0 0x7fbe780b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.770+0000 7fbe777fe700 1 -- 172.21.15.112:0/889292335 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe780b3b70 con 0x7fbe780b52a0 2023-08-27T11:27:59.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.770+0000 7fbe777fe700 1 --2- 172.21.15.112:0/889292335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe780b52a0 0x7fbe7815d930 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fbe780720a0 tx=0x7fbe7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31df23a519be7326 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:59.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.770+0000 7fbe75ffb700 1 -- 172.21.15.112:0/889292335 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7000e070 con 0x7fbe780b52a0 2023-08-27T11:27:59.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.770+0000 7fbe75ffb700 1 -- 172.21.15.112:0/889292335 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7000b9c0 con 0x7fbe780b52a0 2023-08-27T11:27:59.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.770+0000 7fbe75ffb700 1 -- 172.21.15.112:0/889292335 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7000bd70 con 0x7fbe780b52a0 2023-08-27T11:27:59.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.771+0000 7fbe7d572700 1 -- 172.21.15.112:0/889292335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe780b52a0 msgr2=0x7fbe7815d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.771+0000 7fbe7d572700 1 --2- 172.21.15.112:0/889292335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe780b52a0 0x7fbe7815d930 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fbe780720a0 tx=0x7fbe7000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:59.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.771+0000 7fbe7d572700 1 -- 172.21.15.112:0/889292335 shutdown_connections 2023-08-27T11:27:59.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.771+0000 7fbe7d572700 1 --2- 172.21.15.112:0/889292335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe780b52a0 0x7fbe7815d930 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.771+0000 7fbe7d572700 1 --2- 172.21.15.112:0/889292335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe78146be0 0x7fbe780b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.771+0000 7fbe7d572700 1 --2- 172.21.15.112:0/889292335 >> v2:172.21.15.138:3300/0 conn(0x7fbe781619b0 0x7fbe780b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.771+0000 7fbe7d572700 1 -- 172.21.15.112:0/889292335 >> 172.21.15.112:0/889292335 conn(0x7fbe7806c880 msgr2=0x7fbe780717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:59.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.771+0000 7fbe7d572700 1 -- 172.21.15.112:0/889292335 shutdown_connections 2023-08-27T11:27:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.771+0000 7fbe7d572700 1 -- 172.21.15.112:0/889292335 wait complete. 2023-08-27T11:27:59.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.772+0000 7fbe7d572700 1 Processor -- start 2023-08-27T11:27:59.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.772+0000 7fbe7d572700 1 -- start start 2023-08-27T11:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe7d572700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe780b52a0 0x7fbe781f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe7d572700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe78146be0 0x7fbe781f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe7d572700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe781619b0 0x7fbe7813cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe7d572700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe781f09f0 con 0x7fbe781619b0 2023-08-27T11:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe7d572700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe781f3f20 con 0x7fbe78146be0 2023-08-27T11:27:59.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe7d572700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7813b950 con 0x7fbe780b52a0 2023-08-27T11:27:59.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe781619b0 0x7fbe7813cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe781619b0 0x7fbe7813cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35608/0 (socket says 172.21.15.112:35608) 2023-08-27T11:27:59.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe777fe700 1 -- 172.21.15.112:0/1382535460 learned_addr learned my addr 172.21.15.112:0/1382535460 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:27:59.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe767fc700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe78146be0 0x7fbe781f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe76ffd700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe780b52a0 0x7fbe781f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe777fe700 1 -- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe780b52a0 msgr2=0x7fbe781f04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe777fe700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe780b52a0 0x7fbe781f04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe777fe700 1 -- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe78146be0 msgr2=0x7fbe781f39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe777fe700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe78146be0 0x7fbe781f39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe777fe700 1 -- 172.21.15.112:0/1382535460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe70009b30 con 0x7fbe781619b0 2023-08-27T11:27:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe767fc700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe78146be0 0x7fbe781f39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe76ffd700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe780b52a0 0x7fbe781f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe777fe700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe781619b0 0x7fbe7813cf20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000b370 tx=0x7fbe70004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe5f7fe700 1 -- 172.21.15.112:0/1382535460 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70004390 con 0x7fbe781619b0 2023-08-27T11:27:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.773+0000 7fbe5f7fe700 1 -- 172.21.15.112:0/1382535460 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe70004510 con 0x7fbe781619b0 2023-08-27T11:27:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.774+0000 7fbe7d572700 1 -- 172.21.15.112:0/1382535460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7813bc30 con 0x7fbe781619b0 2023-08-27T11:27:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.774+0000 7fbe5f7fe700 1 -- 172.21.15.112:0/1382535460 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe700048c0 con 0x7fbe781619b0 2023-08-27T11:27:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.774+0000 7fbe7d572700 1 -- 172.21.15.112:0/1382535460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7813c1f0 con 0x7fbe781619b0 2023-08-27T11:27:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.774+0000 7fbe7d572700 1 -- 172.21.15.112:0/1382535460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe78067a20 con 0x7fbe781619b0 2023-08-27T11:27:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.775+0000 7fbe5f7fe700 1 -- 172.21.15.112:0/1382535460 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fbe7001b430 con 0x7fbe781619b0 2023-08-27T11:27:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.775+0000 7fbe5f7fe700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbe6406a4d0 0x7fbe6406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.775+0000 7fbe5f7fe700 1 -- 172.21.15.112:0/1382535460 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fbe7008a190 con 0x7fbe781619b0 2023-08-27T11:27:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.776+0000 7fbe76ffd700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbe6406a4d0 0x7fbe6406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.776+0000 7fbe76ffd700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbe6406a4d0 0x7fbe6406c990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbe780b7300 tx=0x7fbe6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.780+0000 7fbe5f7fe700 1 -- 172.21.15.112:0/1382535460 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbe700551a0 con 0x7fbe781619b0 2023-08-27T11:27:59.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.969+0000 7fbe7d572700 1 -- 172.21.15.112:0/1382535460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe780b3760 con 0x7fbe781619b0 2023-08-27T11:27:59.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.970+0000 7fbe5f7fe700 1 -- 172.21.15.112:0/1382535460 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+18538 (secure 0 0 0) 0x7fbe700587c0 con 0x7fbe781619b0 2023-08-27T11:27:59.972 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:27:59.972 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":127,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:27:52.546502+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:26.212595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:27.068136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6832/3832579050":"2023-08-28T11:27:49.540827+0000","172.21.15.112:6834/3832579050":"2023-08-28T11:27:49.540827+0000","172.21.15.112:6835/4019485943":"2023-08-28T11:27:46.133510+0000","172.21.15.112:6833/4019485943":"2023-08-28T11:27:46.133510+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.972+0000 7fbe7d572700 1 -- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbe6406a4d0 msgr2=0x7fbe6406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.972+0000 7fbe7d572700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbe6406a4d0 0x7fbe6406c990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbe780b7300 tx=0x7fbe6c008040 comp rx=0 tx=0).stop 2023-08-27T11:27:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.972+0000 7fbe7d572700 1 -- 172.21.15.112:0/1382535460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe781619b0 msgr2=0x7fbe7813cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.972+0000 7fbe7d572700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe781619b0 0x7fbe7813cf20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000b370 tx=0x7fbe70004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.973+0000 7fbe7d572700 1 -- 172.21.15.112:0/1382535460 shutdown_connections 2023-08-27T11:27:59.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.973+0000 7fbe7d572700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbe6406a4d0 0x7fbe6406c990 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.973+0000 7fbe7d572700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe781619b0 0x7fbe7813cf20 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.973+0000 7fbe7d572700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe780b52a0 0x7fbe781f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.973+0000 7fbe7d572700 1 --2- 172.21.15.112:0/1382535460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe78146be0 0x7fbe781f39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.973+0000 7fbe7d572700 1 -- 172.21.15.112:0/1382535460 >> 172.21.15.112:0/1382535460 conn(0x7fbe7806c880 msgr2=0x7fbe780b7540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:59.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.973+0000 7fbe7d572700 1 -- 172.21.15.112:0/1382535460 shutdown_connections 2023-08-27T11:27:59.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:27:59.973+0000 7fbe7d572700 1 -- 172.21.15.112:0/1382535460 wait complete. 2023-08-27T11:27:59.986 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:27:59.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:00.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.105+0000 7f535c4ab700 1 Processor -- start 2023-08-27T11:28:00.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.106+0000 7f535c4ab700 1 -- start start 2023-08-27T11:28:00.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f535c4ab700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f53540b5470 0x7f53540b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f535c4ab700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53540b3b70 con 0x7f53541619b0 2023-08-27T11:28:00.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f535c4ab700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53540b3e50 con 0x7f5354146be0 2023-08-27T11:28:00.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f535c4ab700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53540b3980 con 0x7f53540b5470 2023-08-27T11:28:00.112 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f535aa48700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f53541619b0 0x7f53540cfa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58230/0 (socket says 172.21.15.112:58230) 2023-08-27T11:28:00.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f535aa48700 1 -- 172.21.15.112:0/243397720 learned_addr learned my addr 172.21.15.112:0/243397720 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:00.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359a46700 1 --2- 172.21.15.112:0/243397720 >> v2:172.21.15.138:3300/0 conn(0x7f53540b5470 0x7f53540b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234731981 0 0) 0x7f53540b3b70 con 0x7f53541619b0 2023-08-27T11:28:00.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f533c0035b0 con 0x7f53541619b0 2023-08-27T11:28:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1612692665 0 0) 0x7f533c0035b0 con 0x7f53541619b0 2023-08-27T11:28:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 >> v2:172.21.15.138:3300/0 conn(0x7f53540b5470 msgr2=0x7f53540b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 --2- 172.21.15.112:0/243397720 >> v2:172.21.15.138:3300/0 conn(0x7f53540b5470 0x7f53540b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 >> v1:172.21.15.138:6789/0 conn(0x7f5354146be0 legacy=0x7f53540b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53540b4130 con 0x7f53541619b0 2023-08-27T11:28:00.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f5344001670 con 0x7f53541619b0 2023-08-27T11:28:00.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 >> v1:172.21.15.112:6789/0 conn(0x7f53541619b0 legacy=0x7f53540cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 --2- 172.21.15.112:0/243397720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f533c003a00 0x7f533c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 --2- 172.21.15.112:0/243397720 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f533c004850 0x7f533c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 --2- 172.21.15.112:0/243397720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c005210 0x7f533c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c00c080 con 0x7f533c005210 2023-08-27T11:28:00.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c00c1f0 con 0x7f533c003a00 2023-08-27T11:28:00.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533c00c3b0 con 0x7f533c004850 2023-08-27T11:28:00.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.107+0000 7f535aa48700 1 --2- 172.21.15.112:0/243397720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c005210 0x7f533c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.108+0000 7f535a247700 1 --2- 172.21.15.112:0/243397720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f533c003a00 0x7f533c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.108+0000 7f5359a46700 1 --2- 172.21.15.112:0/243397720 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f533c004850 0x7f533c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.108+0000 7f535aa48700 1 -- 172.21.15.112:0/243397720 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f533c004850 msgr2=0x7f533c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.108+0000 7f535aa48700 1 --2- 172.21.15.112:0/243397720 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f533c004850 0x7f533c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.108+0000 7f535aa48700 1 -- 172.21.15.112:0/243397720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f533c003a00 msgr2=0x7f533c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.108+0000 7f535aa48700 1 --2- 172.21.15.112:0/243397720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f533c003a00 0x7f533c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.108+0000 7f535aa48700 1 -- 172.21.15.112:0/243397720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c0035b0 con 0x7f533c005210 2023-08-27T11:28:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.108+0000 7f535aa48700 1 -- 172.21.15.112:0/243397720 reap_dead start 2023-08-27T11:28:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.108+0000 7f535aa48700 1 --2- 172.21.15.112:0/243397720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c005210 0x7f533c00ba10 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f534400c240 tx=0x7f533c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9566d94ad116550 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.108+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344007c10 con 0x7f533c005210 2023-08-27T11:28:00.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.108+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534400cde0 con 0x7f533c005210 2023-08-27T11:28:00.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.108+0000 7f5359245700 1 -- 172.21.15.112:0/243397720 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344005ba0 con 0x7f533c005210 2023-08-27T11:28:00.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.109+0000 7f535c4ab700 1 -- 172.21.15.112:0/243397720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c005210 msgr2=0x7f533c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.109+0000 7f535c4ab700 1 --2- 172.21.15.112:0/243397720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c005210 0x7f533c00ba10 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f534400c240 tx=0x7f533c005880 comp rx=0 tx=0).stop 2023-08-27T11:28:00.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.109+0000 7f535c4ab700 1 -- 172.21.15.112:0/243397720 shutdown_connections 2023-08-27T11:28:00.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.109+0000 7f535c4ab700 1 --2- 172.21.15.112:0/243397720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c005210 0x7f533c00ba10 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.109+0000 7f535c4ab700 1 -- 172.21.15.112:0/243397720 >> 172.21.15.112:0/243397720 conn(0x7f535406c880 msgr2=0x7f53540717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:00.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.109+0000 7f535c4ab700 1 -- 172.21.15.112:0/243397720 shutdown_connections 2023-08-27T11:28:00.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.109+0000 7f535c4ab700 1 -- 172.21.15.112:0/243397720 wait complete. 2023-08-27T11:28:00.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.110+0000 7f535c4ab700 1 Processor -- start 2023-08-27T11:28:00.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.110+0000 7f535c4ab700 1 -- start start 2023-08-27T11:28:00.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.111+0000 7f535c4ab700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f533c005210 0x7f535415cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.111+0000 7f535c4ab700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f535413b6e0 0x7f5354160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.111+0000 7f535c4ab700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535413cf00 0x7f535413d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.111+0000 7f535c4ab700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535413d8c0 con 0x7f535413cf00 2023-08-27T11:28:00.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.111+0000 7f535c4ab700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535413bc50 con 0x7f533c005210 2023-08-27T11:28:00.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.111+0000 7f535c4ab700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535413bf30 con 0x7f535413b6e0 2023-08-27T11:28:00.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.111+0000 7f535aa48700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535413cf00 0x7f535413d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.111+0000 7f535aa48700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535413cf00 0x7f535413d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35628/0 (socket says 172.21.15.112:35628) 2023-08-27T11:28:00.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.111+0000 7f535aa48700 1 -- 172.21.15.112:0/1807512475 learned_addr learned my addr 172.21.15.112:0/1807512475 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:00.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.112+0000 7f535a247700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f533c005210 0x7f535415cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.112+0000 7f5359a46700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f535413b6e0 0x7f5354160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.112+0000 7f535aa48700 1 -- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f535413b6e0 msgr2=0x7f5354160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.112+0000 7f535aa48700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f535413b6e0 0x7f5354160400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.112+0000 7f535aa48700 1 -- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f533c005210 msgr2=0x7f535415cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.112+0000 7f535aa48700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f533c005210 0x7f535415cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.112+0000 7f535aa48700 1 -- 172.21.15.112:0/1807512475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535413c210 con 0x7f535413cf00 2023-08-27T11:28:00.127 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.112+0000 7f535aa48700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535413cf00 0x7f535413d380 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f53540b9630 tx=0x7f5344001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.112+0000 7f5352ffd700 1 -- 172.21.15.112:0/1807512475 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344001640 con 0x7f535413cf00 2023-08-27T11:28:00.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.112+0000 7f5352ffd700 1 -- 172.21.15.112:0/1807512475 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5344000ed0 con 0x7f535413cf00 2023-08-27T11:28:00.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.112+0000 7f535c4ab700 1 -- 172.21.15.112:0/1807512475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535413c4f0 con 0x7f535413cf00 2023-08-27T11:28:00.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.112+0000 7f5352ffd700 1 -- 172.21.15.112:0/1807512475 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344007310 con 0x7f535413cf00 2023-08-27T11:28:00.128 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.112+0000 7f535c4ab700 1 -- 172.21.15.112:0/1807512475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5354160940 con 0x7f535413cf00 2023-08-27T11:28:00.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.113+0000 7f535c4ab700 1 -- 172.21.15.112:0/1807512475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535415d9a0 con 0x7f535413cf00 2023-08-27T11:28:00.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.113+0000 7f5352ffd700 1 -- 172.21.15.112:0/1807512475 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5344007470 con 0x7f535413cf00 2023-08-27T11:28:00.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.114+0000 7f5352ffd700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f534006a4d0 0x7f534006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.114+0000 7f5352ffd700 1 -- 172.21.15.112:0/1807512475 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f5344089490 con 0x7f535413cf00 2023-08-27T11:28:00.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.114+0000 7f535a247700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f534006a4d0 0x7f534006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.114+0000 7f535a247700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f534006a4d0 0x7f534006c990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f534c000f80 tx=0x7f534c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.118+0000 7f5352ffd700 1 -- 172.21.15.112:0/1807512475 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f535415d9a0 con 0x7f535413cf00 2023-08-27T11:28:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.348+0000 7f535c4ab700 1 -- 172.21.15.112:0/1807512475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f535415d9a0 con 0x7f535413cf00 2023-08-27T11:28:00.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.348+0000 7f5352ffd700 1 -- 172.21.15.112:0/1807512475 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4889 (secure 0 0 0) 0x7f535415d9a0 con 0x7f535413cf00 2023-08-27T11:28:00.350 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:00.350 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1795721853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1795721853},{"type":"v1","addr":"172.21.15.138:6836","nonce":1795721853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":8184,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2658811115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2658811115},{"type":"v1","addr":"172.21.15.112:6835","nonce":2658811115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":8199,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3536640393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3536640393},{"type":"v1","addr":"172.21.15.112:6834","nonce":3536640393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:27:28.107305+0000","modified":"2023-08-27T11:27:31.742912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7989},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7989":{"gid":7989,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/3245198376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3245198376},{"type":"v1","addr":"172.21.15.138:6837","nonce":3245198376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-27T11:28:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.350+0000 7f535c4ab700 1 -- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f534006a4d0 msgr2=0x7f534006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.350+0000 7f535c4ab700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f534006a4d0 0x7f534006c990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f534c000f80 tx=0x7f534c009040 comp rx=0 tx=0).stop 2023-08-27T11:28:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.350+0000 7f535c4ab700 1 -- 172.21.15.112:0/1807512475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535413cf00 msgr2=0x7f535413d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.350+0000 7f535c4ab700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535413cf00 0x7f535413d380 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f53540b9630 tx=0x7f5344001a00 comp rx=0 tx=0).stop 2023-08-27T11:28:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.350+0000 7f535c4ab700 1 -- 172.21.15.112:0/1807512475 shutdown_connections 2023-08-27T11:28:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.350+0000 7f535c4ab700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f534006a4d0 0x7f534006c990 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.350+0000 7f535c4ab700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535413cf00 0x7f535413d380 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.350+0000 7f535c4ab700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f533c005210 0x7f535415cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.350+0000 7f535c4ab700 1 --2- 172.21.15.112:0/1807512475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f535413b6e0 0x7f5354160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.350+0000 7f535c4ab700 1 -- 172.21.15.112:0/1807512475 >> 172.21.15.112:0/1807512475 conn(0x7f535406c880 msgr2=0x7f53540b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.350+0000 7f535c4ab700 1 -- 172.21.15.112:0/1807512475 shutdown_connections 2023-08-27T11:28:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.350+0000 7f535c4ab700 1 -- 172.21.15.112:0/1807512475 wait complete. 2023-08-27T11:28:00.355 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 86 2023-08-27T11:28:00.364 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:00.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.482+0000 7ffb0b860700 1 Processor -- start 2023-08-27T11:28:00.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.482+0000 7ffb0b860700 1 -- start start 2023-08-27T11:28:00.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffb0b860700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb04146990 0x7ffb040b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffb0b860700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0406f930 con 0x7ffb0406d510 2023-08-27T11:28:00.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffb0b860700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0406fc10 con 0x7ffb04160a00 2023-08-27T11:28:00.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffb0b860700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0406fef0 con 0x7ffb04146990 2023-08-27T11:28:00.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffb09dfd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ffb0406d510 0x7ffb04161400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58250/0 (socket says 172.21.15.112:58250) 2023-08-27T11:28:00.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffb09dfd700 1 -- 172.21.15.112:0/4050153910 learned_addr learned my addr 172.21.15.112:0/4050153910 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:00.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffb08dfb700 1 --2- 172.21.15.112:0/4050153910 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb04146990 0x7ffb040b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffafbfff700 1 -- 172.21.15.112:0/4050153910 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466529902 0 0) 0x7ffb0406f930 con 0x7ffb0406d510 2023-08-27T11:28:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffafbfff700 1 -- 172.21.15.112:0/4050153910 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaf00035b0 con 0x7ffb0406d510 2023-08-27T11:28:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffafbfff700 1 -- 172.21.15.112:0/4050153910 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676082363 0 0) 0x7ffb0406fc10 con 0x7ffb04160a00 2023-08-27T11:28:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffafbfff700 1 -- 172.21.15.112:0/4050153910 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb0406f930 con 0x7ffb04160a00 2023-08-27T11:28:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffafbfff700 1 -- 172.21.15.112:0/4050153910 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 994284692 0 0) 0x7ffaf00035b0 con 0x7ffb0406d510 2023-08-27T11:28:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffb08dfb700 1 -- 172.21.15.112:0/4050153910 >> v1:172.21.15.138:6789/0 conn(0x7ffb04160a00 legacy=0x7ffb0415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffb08dfb700 1 -- 172.21.15.112:0/4050153910 >> v1:172.21.15.112:6789/0 conn(0x7ffb0406d510 legacy=0x7ffb04161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.483+0000 7ffb08dfb700 1 -- 172.21.15.112:0/4050153910 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb040701d0 con 0x7ffb04146990 2023-08-27T11:28:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.484+0000 7ffb08dfb700 1 --2- 172.21.15.112:0/4050153910 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb04146990 0x7ffb040b46d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009870 tx=0x7ffaf4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b00241ea0870dc8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.484+0000 7ffafbfff700 1 -- 172.21.15.112:0/4050153910 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0019f0 con 0x7ffb04146990 2023-08-27T11:28:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.484+0000 7ffafbfff700 1 -- 172.21.15.112:0/4050153910 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf4004030 con 0x7ffb04146990 2023-08-27T11:28:00.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.484+0000 7ffafbfff700 1 -- 172.21.15.112:0/4050153910 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40043c0 con 0x7ffb04146990 2023-08-27T11:28:00.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.484+0000 7ffb0b860700 1 -- 172.21.15.112:0/4050153910 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb04146990 msgr2=0x7ffb040b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.484+0000 7ffb0b860700 1 --2- 172.21.15.112:0/4050153910 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb04146990 0x7ffb040b46d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009870 tx=0x7ffaf4009b80 comp rx=0 tx=0).stop 2023-08-27T11:28:00.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.484+0000 7ffb0b860700 1 -- 172.21.15.112:0/4050153910 shutdown_connections 2023-08-27T11:28:00.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.484+0000 7ffb0b860700 1 --2- 172.21.15.112:0/4050153910 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb04146990 0x7ffb040b46d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.484+0000 7ffb0b860700 1 -- 172.21.15.112:0/4050153910 >> 172.21.15.112:0/4050153910 conn(0x7ffb040cfe50 msgr2=0x7ffb040d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:00.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.485+0000 7ffb0b860700 1 -- 172.21.15.112:0/4050153910 shutdown_connections 2023-08-27T11:28:00.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.485+0000 7ffb0b860700 1 -- 172.21.15.112:0/4050153910 wait complete. 2023-08-27T11:28:00.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.486+0000 7ffb0b860700 1 Processor -- start 2023-08-27T11:28:00.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.486+0000 7ffb0b860700 1 -- start start 2023-08-27T11:28:00.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.486+0000 7ffb0b860700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0406d510 0x7ffb040b9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.486+0000 7ffb0b860700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb04146990 0x7ffb04142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.486+0000 7ffb0b860700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb04160a00 0x7ffb040b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb0b860700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040b2100 con 0x7ffb0406d510 2023-08-27T11:28:00.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb0b860700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040b23e0 con 0x7ffb04160a00 2023-08-27T11:28:00.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb0b860700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040b26c0 con 0x7ffb04146990 2023-08-27T11:28:00.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb095fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0406d510 0x7ffb040b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb095fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0406d510 0x7ffb040b9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35638/0 (socket says 172.21.15.112:35638) 2023-08-27T11:28:00.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb095fc700 1 -- 172.21.15.112:0/3947520865 learned_addr learned my addr 172.21.15.112:0/3947520865 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:00.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb08dfb700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb04146990 0x7ffb04142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb09dfd700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb04160a00 0x7ffb040b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb095fc700 1 -- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb04146990 msgr2=0x7ffb04142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb095fc700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb04146990 0x7ffb04142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb095fc700 1 -- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb04160a00 msgr2=0x7ffb040b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb095fc700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb04160a00 0x7ffb040b7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb095fc700 1 -- 172.21.15.112:0/3947520865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf40094a0 con 0x7ffb0406d510 2023-08-27T11:28:00.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb09dfd700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb04160a00 0x7ffb040b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:00.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb095fc700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0406d510 0x7ffb040b9290 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000c500 tx=0x7ffb0000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffaf9ffb700 1 -- 172.21.15.112:0/3947520865 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb000052d0 con 0x7ffb0406d510 2023-08-27T11:28:00.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffaf9ffb700 1 -- 172.21.15.112:0/3947520865 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb00005450 con 0x7ffb0406d510 2023-08-27T11:28:00.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffaf9ffb700 1 -- 172.21.15.112:0/3947520865 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb000038c0 con 0x7ffb0406d510 2023-08-27T11:28:00.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb0b860700 1 -- 172.21.15.112:0/3947520865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb040b29a0 con 0x7ffb0406d510 2023-08-27T11:28:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.487+0000 7ffb0b860700 1 -- 172.21.15.112:0/3947520865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb04143340 con 0x7ffb0406d510 2023-08-27T11:28:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.488+0000 7ffb0b860700 1 -- 172.21.15.112:0/3947520865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb040ac890 con 0x7ffb0406d510 2023-08-27T11:28:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.489+0000 7ffaf9ffb700 1 -- 172.21.15.112:0/3947520865 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7ffb00003a20 con 0x7ffb0406d510 2023-08-27T11:28:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.489+0000 7ffaf9ffb700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffae806a4d0 0x7ffae806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.489+0000 7ffaf9ffb700 1 -- 172.21.15.112:0/3947520865 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7ffb000880f0 con 0x7ffb0406d510 2023-08-27T11:28:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.489+0000 7ffb08dfb700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffae806a4d0 0x7ffae806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.490+0000 7ffb08dfb700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffae806a4d0 0x7ffae806c990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40097c0 tx=0x7ffaf4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.493+0000 7ffaf9ffb700 1 -- 172.21.15.112:0/3947520865 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffb000531e0 con 0x7ffb0406d510 2023-08-27T11:28:00.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.724+0000 7ffb0b860700 1 -- 172.21.15.112:0/3947520865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb04067a20 con 0x7ffb0406d510 2023-08-27T11:28:00.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.725+0000 7ffaf9ffb700 1 -- 172.21.15.112:0/3947520865 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4889 (secure 0 0 0) 0x7ffb00056800 con 0x7ffb0406d510 2023-08-27T11:28:00.727 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:00.727 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1795721853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1795721853},{"type":"v1","addr":"172.21.15.138:6836","nonce":1795721853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":8184,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2658811115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2658811115},{"type":"v1","addr":"172.21.15.112:6835","nonce":2658811115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":8199,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3536640393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3536640393},{"type":"v1","addr":"172.21.15.112:6834","nonce":3536640393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:27:28.107305+0000","modified":"2023-08-27T11:27:31.742912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7989},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7989":{"gid":7989,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/3245198376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3245198376},{"type":"v1","addr":"172.21.15.138:6837","nonce":3245198376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-27T11:28:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.727+0000 7ffb0b860700 1 -- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffae806a4d0 msgr2=0x7ffae806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.727+0000 7ffb0b860700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffae806a4d0 0x7ffae806c990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40097c0 tx=0x7ffaf4008030 comp rx=0 tx=0).stop 2023-08-27T11:28:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.727+0000 7ffb0b860700 1 -- 172.21.15.112:0/3947520865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0406d510 msgr2=0x7ffb040b9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.727+0000 7ffb0b860700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0406d510 0x7ffb040b9290 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000c500 tx=0x7ffb0000c810 comp rx=0 tx=0).stop 2023-08-27T11:28:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.727+0000 7ffb0b860700 1 -- 172.21.15.112:0/3947520865 shutdown_connections 2023-08-27T11:28:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.727+0000 7ffb0b860700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ffae806a4d0 0x7ffae806c990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.727+0000 7ffb0b860700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb04160a00 0x7ffb040b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.727+0000 7ffb0b860700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0406d510 0x7ffb040b9290 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.727+0000 7ffb0b860700 1 --2- 172.21.15.112:0/3947520865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb04146990 0x7ffb04142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.727+0000 7ffb0b860700 1 -- 172.21.15.112:0/3947520865 >> 172.21.15.112:0/3947520865 conn(0x7ffb040cfe50 msgr2=0x7ffb040d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.727+0000 7ffb0b860700 1 -- 172.21.15.112:0/3947520865 shutdown_connections 2023-08-27T11:28:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.727+0000 7ffb0b860700 1 -- 172.21.15.112:0/3947520865 wait complete. 2023-08-27T11:28:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 86 2023-08-27T11:28:00.741 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:00.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.861+0000 7f91df69a700 1 Processor -- start 2023-08-27T11:28:00.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.861+0000 7f91df69a700 1 -- start start 2023-08-27T11:28:00.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91df69a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f91d80b17e0 0x7f91d80cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91df69a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d8161850 con 0x7f91d80b1400 2023-08-27T11:28:00.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91df69a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d80cfc10 con 0x7f91d8161ce0 2023-08-27T11:28:00.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91df69a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d8161200 con 0x7f91d80b17e0 2023-08-27T11:28:00.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91dd436700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f91d80b1400 0x7f91d806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58260/0 (socket says 172.21.15.112:58260) 2023-08-27T11:28:00.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91dd436700 1 -- 172.21.15.112:0/1879265105 learned_addr learned my addr 172.21.15.112:0/1879265105 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:00.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229151152 0 0) 0x7f91d8161850 con 0x7f91d80b1400 2023-08-27T11:28:00.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c40035b0 con 0x7f91d80b1400 2023-08-27T11:28:00.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91dcc35700 1 --2- 172.21.15.112:0/1879265105 >> v2:172.21.15.138:3300/0 conn(0x7f91d80b17e0 0x7f91d80cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3505932454 0 0) 0x7f91c40035b0 con 0x7f91d80b1400 2023-08-27T11:28:00.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 >> v2:172.21.15.138:3300/0 conn(0x7f91d80b17e0 msgr2=0x7f91d80cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:00.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 --2- 172.21.15.112:0/1879265105 >> v2:172.21.15.138:3300/0 conn(0x7f91d80b17e0 0x7f91d80cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 >> v1:172.21.15.138:6789/0 conn(0x7f91d8161ce0 legacy=0x7f91d806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d81614e0 con 0x7f91d80b1400 2023-08-27T11:28:00.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f91d4001670 con 0x7f91d80b1400 2023-08-27T11:28:00.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 >> v1:172.21.15.112:6789/0 conn(0x7f91d80b1400 legacy=0x7f91d806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 --2- 172.21.15.112:0/1879265105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91c4003a00 0x7f91c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 --2- 172.21.15.112:0/1879265105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91c4004850 0x7f91c4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 --2- 172.21.15.112:0/1879265105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c4005330 0x7f91c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8161200 con 0x7f91c4003a00 2023-08-27T11:28:00.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c400c200 con 0x7f91c4005330 2023-08-27T11:28:00.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.862+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c400c3c0 con 0x7f91c4004850 2023-08-27T11:28:00.871 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91dd436700 1 --2- 172.21.15.112:0/1879265105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91c4003a00 0x7f91c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91ddc37700 1 --2- 172.21.15.112:0/1879265105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c4005330 0x7f91c400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91dcc35700 1 --2- 172.21.15.112:0/1879265105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91c4004850 0x7f91c4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91dd436700 1 -- 172.21.15.112:0/1879265105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91c4004850 msgr2=0x7f91c4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:00.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91dd436700 1 --2- 172.21.15.112:0/1879265105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91c4004850 0x7f91c4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91dd436700 1 -- 172.21.15.112:0/1879265105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c4005330 msgr2=0x7f91c400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:00.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91dd436700 1 --2- 172.21.15.112:0/1879265105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c4005330 0x7f91c400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91dd436700 1 -- 172.21.15.112:0/1879265105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c40035b0 con 0x7f91c4003a00 2023-08-27T11:28:00.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91ddc37700 1 --2- 172.21.15.112:0/1879265105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c4005330 0x7f91c400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:00.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91ddc37700 1 -- 172.21.15.112:0/1879265105 reap_dead start 2023-08-27T11:28:00.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91dd436700 1 --2- 172.21.15.112:0/1879265105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91c4003a00 0x7f91c4003e20 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f91d400c210 tx=0x7f91c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9b54950b2f53bd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4007c10 con 0x7f91c4003a00 2023-08-27T11:28:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d400cdd0 con 0x7f91c4003a00 2023-08-27T11:28:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.863+0000 7f91cffff700 1 -- 172.21.15.112:0/1879265105 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4005ba0 con 0x7f91c4003a00 2023-08-27T11:28:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.864+0000 7f91df69a700 1 -- 172.21.15.112:0/1879265105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91c4003a00 msgr2=0x7f91c4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.864+0000 7f91df69a700 1 --2- 172.21.15.112:0/1879265105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91c4003a00 0x7f91c4003e20 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f91d400c210 tx=0x7f91c4005880 comp rx=0 tx=0).stop 2023-08-27T11:28:00.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.864+0000 7f91df69a700 1 -- 172.21.15.112:0/1879265105 shutdown_connections 2023-08-27T11:28:00.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.864+0000 7f91df69a700 1 --2- 172.21.15.112:0/1879265105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91c4003a00 0x7f91c4003e20 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.864+0000 7f91df69a700 1 -- 172.21.15.112:0/1879265105 >> 172.21.15.112:0/1879265105 conn(0x7f91d80d0000 msgr2=0x7f91d80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:00.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.864+0000 7f91df69a700 1 -- 172.21.15.112:0/1879265105 shutdown_connections 2023-08-27T11:28:00.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.864+0000 7f91df69a700 1 -- 172.21.15.112:0/1879265105 wait complete. 2023-08-27T11:28:00.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.865+0000 7f91df69a700 1 Processor -- start 2023-08-27T11:28:00.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.865+0000 7f91df69a700 1 -- start start 2023-08-27T11:28:00.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91df69a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c4003a00 0x7f91d814c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91df69a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d80713c0 0x7f91d8071840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91df69a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d814f770 0x7f91d814fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91df69a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8071d80 con 0x7f91d80713c0 2023-08-27T11:28:00.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91df69a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d80c8ab0 con 0x7f91c4003a00 2023-08-27T11:28:00.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91df69a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8150110 con 0x7f91d814f770 2023-08-27T11:28:00.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91dcc35700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d80713c0 0x7f91d8071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91dcc35700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d80713c0 0x7f91d8071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35660/0 (socket says 172.21.15.112:35660) 2023-08-27T11:28:00.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91dcc35700 1 -- 172.21.15.112:0/3430293167 learned_addr learned my addr 172.21.15.112:0/3430293167 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:00.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91dd436700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c4003a00 0x7f91d814c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91ddc37700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d814f770 0x7f91d814fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91dcc35700 1 -- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d814f770 msgr2=0x7f91d814fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91dcc35700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d814f770 0x7f91d814fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91dcc35700 1 -- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c4003a00 msgr2=0x7f91d814c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91dcc35700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c4003a00 0x7f91d814c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91dcc35700 1 -- 172.21.15.112:0/3430293167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d814d400 con 0x7f91d80713c0 2023-08-27T11:28:00.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91ddc37700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d814f770 0x7f91d814fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:00.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91dcc35700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d80713c0 0x7f91d8071840 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f91c8011680 tx=0x7f91c8011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91cdffb700 1 -- 172.21.15.112:0/3430293167 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8004220 con 0x7f91d80713c0 2023-08-27T11:28:00.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91cdffb700 1 -- 172.21.15.112:0/3430293167 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c8004380 con 0x7f91d80713c0 2023-08-27T11:28:00.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91df69a700 1 -- 172.21.15.112:0/3430293167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d80c8180 con 0x7f91d80713c0 2023-08-27T11:28:00.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.866+0000 7f91cdffb700 1 -- 172.21.15.112:0/3430293167 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c80036a0 con 0x7f91d80713c0 2023-08-27T11:28:00.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.867+0000 7f91df69a700 1 -- 172.21.15.112:0/3430293167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d80c8740 con 0x7f91d80713c0 2023-08-27T11:28:00.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.867+0000 7f91df69a700 1 -- 172.21.15.112:0/3430293167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d80c8c60 con 0x7f91d80713c0 2023-08-27T11:28:00.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.868+0000 7f91cdffb700 1 -- 172.21.15.112:0/3430293167 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f91c8003800 con 0x7f91d80713c0 2023-08-27T11:28:00.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.868+0000 7f91cdffb700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91bc06a4d0 0x7f91bc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.869+0000 7f91cdffb700 1 -- 172.21.15.112:0/3430293167 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f91c8088e00 con 0x7f91d80713c0 2023-08-27T11:28:00.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.869+0000 7f91dd436700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91bc06a4d0 0x7f91bc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.869+0000 7f91dd436700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91bc06a4d0 0x7f91bc06c990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f91d4000c00 tx=0x7f91d4007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:00.873+0000 7f91cdffb700 1 -- 172.21.15.112:0/3430293167 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f91d80c8c60 con 0x7f91d80713c0 2023-08-27T11:28:01.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.063+0000 7f91df69a700 1 -- 172.21.15.112:0/3430293167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91d80c8c20 con 0x7f91d80713c0 2023-08-27T11:28:01.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.065+0000 7f91cdffb700 1 -- 172.21.15.112:0/3430293167 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+18538 (secure 0 0 0) 0x7f91c8057900 con 0x7f91d80713c0 2023-08-27T11:28:01.067 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:01.067 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":127,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:27:52.546502+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:26.212595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:27.068136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6832/3832579050":"2023-08-28T11:27:49.540827+0000","172.21.15.112:6834/3832579050":"2023-08-28T11:27:49.540827+0000","172.21.15.112:6835/4019485943":"2023-08-28T11:27:46.133510+0000","172.21.15.112:6833/4019485943":"2023-08-28T11:27:46.133510+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:01.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.067+0000 7f91df69a700 1 -- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91bc06a4d0 msgr2=0x7f91bc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.067+0000 7f91df69a700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91bc06a4d0 0x7f91bc06c990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f91d4000c00 tx=0x7f91d4007000 comp rx=0 tx=0).stop 2023-08-27T11:28:01.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.067+0000 7f91df69a700 1 -- 172.21.15.112:0/3430293167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d80713c0 msgr2=0x7f91d8071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.067+0000 7f91df69a700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d80713c0 0x7f91d8071840 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f91c8011680 tx=0x7f91c8011a40 comp rx=0 tx=0).stop 2023-08-27T11:28:01.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.067+0000 7f91df69a700 1 -- 172.21.15.112:0/3430293167 shutdown_connections 2023-08-27T11:28:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.067+0000 7f91df69a700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f91bc06a4d0 0x7f91bc06c990 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.067+0000 7f91df69a700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d814f770 0x7f91d814fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.067+0000 7f91df69a700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c4003a00 0x7f91d814c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.067+0000 7f91df69a700 1 --2- 172.21.15.112:0/3430293167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d80713c0 0x7f91d8071840 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.067+0000 7f91df69a700 1 -- 172.21.15.112:0/3430293167 >> 172.21.15.112:0/3430293167 conn(0x7f91d80d0000 msgr2=0x7f91d806f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.067+0000 7f91df69a700 1 -- 172.21.15.112:0/3430293167 shutdown_connections 2023-08-27T11:28:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.067+0000 7f91df69a700 1 -- 172.21.15.112:0/3430293167 wait complete. 2023-08-27T11:28:01.081 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:28:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.200+0000 7fe47fdb7700 1 Processor -- start 2023-08-27T11:28:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.201+0000 7fe47fdb7700 1 -- start start 2023-08-27T11:28:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47fdb7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe47806dcc0 0x7fe4780b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47fdb7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe4780b3440 0x7fe4780b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47fdb7700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47815b540 con 0x7fe47815a120 2023-08-27T11:28:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47fdb7700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4780af830 con 0x7fe47806dcc0 2023-08-27T11:28:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47fdb7700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4780afb10 con 0x7fe4780b3440 2023-08-27T11:28:01.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47d352700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe47806dcc0 0x7fe4780b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47db53700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe47815a120 0x7fe47806d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58280/0 (socket says 172.21.15.112:58280) 2023-08-27T11:28:01.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47db53700 1 -- 172.21.15.112:0/2623230958 learned_addr learned my addr 172.21.15.112:0/2623230958 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:01.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47d352700 1 -- 172.21.15.112:0/2623230958 >> v2:172.21.15.138:3300/0 conn(0x7fe4780b3440 msgr2=0x7fe4780b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:01.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47d352700 1 --2- 172.21.15.112:0/2623230958 >> v2:172.21.15.138:3300/0 conn(0x7fe4780b3440 0x7fe4780b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47d352700 1 -- 172.21.15.112:0/2623230958 >> v1:172.21.15.112:6789/0 conn(0x7fe47815a120 legacy=0x7fe47806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47d352700 1 -- 172.21.15.112:0/2623230958 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4780afdf0 con 0x7fe47806dcc0 2023-08-27T11:28:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47cb51700 1 -- 172.21.15.112:0/2623230958 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108009183 0 0) 0x7fe47815b540 con 0x7fe47815a120 2023-08-27T11:28:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47d352700 1 --2- 172.21.15.112:0/2623230958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe47806dcc0 0x7fe4780b05f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fe4780b5cd0 tx=0x7fe474009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8553cfc2711dd7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47cb51700 1 -- 172.21.15.112:0/2623230958 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47400f8a0 con 0x7fe47806dcc0 2023-08-27T11:28:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47cb51700 1 -- 172.21.15.112:0/2623230958 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47400fa00 con 0x7fe47806dcc0 2023-08-27T11:28:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.202+0000 7fe47cb51700 1 -- 172.21.15.112:0/2623230958 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47400fd90 con 0x7fe47806dcc0 2023-08-27T11:28:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.203+0000 7fe47fdb7700 1 -- 172.21.15.112:0/2623230958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe47806dcc0 msgr2=0x7fe4780b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.203+0000 7fe47fdb7700 1 --2- 172.21.15.112:0/2623230958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe47806dcc0 0x7fe4780b05f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fe4780b5cd0 tx=0x7fe474009b20 comp rx=0 tx=0).stop 2023-08-27T11:28:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.203+0000 7fe47fdb7700 1 -- 172.21.15.112:0/2623230958 shutdown_connections 2023-08-27T11:28:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.203+0000 7fe47fdb7700 1 --2- 172.21.15.112:0/2623230958 >> v2:172.21.15.138:3300/0 conn(0x7fe4780b3440 0x7fe4780b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.203+0000 7fe47fdb7700 1 --2- 172.21.15.112:0/2623230958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe47806dcc0 0x7fe4780b05f0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.203+0000 7fe47fdb7700 1 -- 172.21.15.112:0/2623230958 >> 172.21.15.112:0/2623230958 conn(0x7fe4780cff80 msgr2=0x7fe4780d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.203+0000 7fe47fdb7700 1 -- 172.21.15.112:0/2623230958 shutdown_connections 2023-08-27T11:28:01.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.203+0000 7fe47fdb7700 1 -- 172.21.15.112:0/2623230958 wait complete. 2023-08-27T11:28:01.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.204+0000 7fe47fdb7700 1 Processor -- start 2023-08-27T11:28:01.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.204+0000 7fe47fdb7700 1 -- start start 2023-08-27T11:28:01.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47fdb7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe47806dcc0 0x7fe478141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47fdb7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4780b3440 0x7fe478157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47fdb7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe47815a120 0x7fe47813d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47fdb7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4781f1b50 con 0x7fe47806dcc0 2023-08-27T11:28:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47fdb7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4781415d0 con 0x7fe4780b3440 2023-08-27T11:28:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47fdb7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4781f1d60 con 0x7fe47815a120 2023-08-27T11:28:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47db53700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe47806dcc0 0x7fe478141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47db53700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe47806dcc0 0x7fe478141090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35672/0 (socket says 172.21.15.112:35672) 2023-08-27T11:28:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47db53700 1 -- 172.21.15.112:0/1932557326 learned_addr learned my addr 172.21.15.112:0/1932557326 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47d352700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4780b3440 0x7fe478157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47db53700 1 -- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe47815a120 msgr2=0x7fe47813d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47e354700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe47815a120 0x7fe47813d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47db53700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe47815a120 0x7fe47813d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47db53700 1 -- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4780b3440 msgr2=0x7fe478157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47db53700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4780b3440 0x7fe478157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47db53700 1 -- 172.21.15.112:0/1932557326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4740094a0 con 0x7fe47806dcc0 2023-08-27T11:28:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47d352700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4780b3440 0x7fe478157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:01.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47e354700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe47815a120 0x7fe47813d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:01.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe47db53700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe47806dcc0 0x7fe478141090 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c550 tx=0x7fe46c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:01.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.205+0000 7fe46a7fc700 1 -- 172.21.15.112:0/1932557326 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c004c80 con 0x7fe47806dcc0 2023-08-27T11:28:01.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.206+0000 7fe46a7fc700 1 -- 172.21.15.112:0/1932557326 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c004e00 con 0x7fe47806dcc0 2023-08-27T11:28:01.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.206+0000 7fe47fdb7700 1 -- 172.21.15.112:0/1932557326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4781f2040 con 0x7fe47806dcc0 2023-08-27T11:28:01.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.206+0000 7fe46a7fc700 1 -- 172.21.15.112:0/1932557326 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c007310 con 0x7fe47806dcc0 2023-08-27T11:28:01.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.206+0000 7fe47fdb7700 1 -- 172.21.15.112:0/1932557326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4781f2600 con 0x7fe47806dcc0 2023-08-27T11:28:01.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.206+0000 7fe47fdb7700 1 -- 172.21.15.112:0/1932557326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe478067a20 con 0x7fe47806dcc0 2023-08-27T11:28:01.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.207+0000 7fe46a7fc700 1 -- 172.21.15.112:0/1932557326 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe46c007470 con 0x7fe47806dcc0 2023-08-27T11:28:01.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.207+0000 7fe46a7fc700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe45c06a4d0 0x7fe45c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.207+0000 7fe46a7fc700 1 -- 172.21.15.112:0/1932557326 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fe46c0889c0 con 0x7fe47806dcc0 2023-08-27T11:28:01.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.208+0000 7fe47d352700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe45c06a4d0 0x7fe45c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.208+0000 7fe47d352700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe45c06a4d0 0x7fe45c06c990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe47814d360 tx=0x7fe47401b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.212+0000 7fe46a7fc700 1 -- 172.21.15.112:0/1932557326 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe46c053b30 con 0x7fe47806dcc0 2023-08-27T11:28:01.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.446+0000 7fe47fdb7700 1 -- 172.21.15.112:0/1932557326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe4781598f0 con 0x7fe47806dcc0 2023-08-27T11:28:01.448 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:28:01.447+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:28:01.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.450+0000 7fe46a7fc700 1 -- 172.21.15.112:0/1932557326 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe46c057150 con 0x7fe47806dcc0 2023-08-27T11:28:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.452+0000 7fe47fdb7700 1 -- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe45c06a4d0 msgr2=0x7fe45c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.452+0000 7fe47fdb7700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe45c06a4d0 0x7fe45c06c990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe47814d360 tx=0x7fe47401b040 comp rx=0 tx=0).stop 2023-08-27T11:28:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.452+0000 7fe47fdb7700 1 -- 172.21.15.112:0/1932557326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe47806dcc0 msgr2=0x7fe478141090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.452+0000 7fe47fdb7700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe47806dcc0 0x7fe478141090 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c550 tx=0x7fe46c00c910 comp rx=0 tx=0).stop 2023-08-27T11:28:01.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.452+0000 7fe47fdb7700 1 -- 172.21.15.112:0/1932557326 shutdown_connections 2023-08-27T11:28:01.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.452+0000 7fe47fdb7700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe45c06a4d0 0x7fe45c06c990 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.452+0000 7fe47fdb7700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe47815a120 0x7fe47813d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.452+0000 7fe47fdb7700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe47806dcc0 0x7fe478141090 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.452+0000 7fe47fdb7700 1 --2- 172.21.15.112:0/1932557326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4780b3440 0x7fe478157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.452+0000 7fe47fdb7700 1 -- 172.21.15.112:0/1932557326 >> 172.21.15.112:0/1932557326 conn(0x7fe4780cff80 msgr2=0x7fe478156050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:01.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.452+0000 7fe47fdb7700 1 -- 172.21.15.112:0/1932557326 shutdown_connections 2023-08-27T11:28:01.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.452+0000 7fe47fdb7700 1 -- 172.21.15.112:0/1932557326 wait complete. 2023-08-27T11:28:01.457 INFO:tasks.ceph.mds.b.smithi138.stderr: -3986> 2023-08-27T11:27:33.696+0000 7faa2a7a8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:28:01.457 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:28:01.466 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:28:01.487 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:28:01.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.587+0000 7f8984574700 1 Processor -- start 2023-08-27T11:28:01.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.587+0000 7f8984574700 1 -- start start 2023-08-27T11:28:01.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.588+0000 7f8984574700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897c15b2b0 0x7f897c15f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.588+0000 7f8984574700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f897c0cbe00 0x7f897c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.588+0000 7f8984574700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c15f370 con 0x7f897c0cf130 2023-08-27T11:28:01.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.588+0000 7f8984574700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c071c20 con 0x7f897c0cbe00 2023-08-27T11:28:01.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.588+0000 7f8984574700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c071660 con 0x7f897c15b2b0 2023-08-27T11:28:01.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f8982310700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897c15b2b0 0x7f897c15f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f8981b0f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f897c0cbe00 0x7f897c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f8982310700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897c15b2b0 0x7f897c15f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:47338/0 (socket says 172.21.15.112:47338) 2023-08-27T11:28:01.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f8982310700 1 -- 172.21.15.112:0/2744664685 learned_addr learned my addr 172.21.15.112:0/2744664685 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:01.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f8981b0f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f897c0cbe00 0x7f897c070cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:36106/0 (socket says 172.21.15.112:36106) 2023-08-27T11:28:01.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f8982b11700 1 --1- 172.21.15.112:0/2744664685 >> v1:172.21.15.138:6789/0 conn(0x7f897c0cf130 0x7f897c0cf5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:37586/0 (socket says 172.21.15.112:37586) 2023-08-27T11:28:01.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f8982310700 1 -- 172.21.15.112:0/2744664685 >> v2:172.21.15.138:3300/0 conn(0x7f897c0cbe00 msgr2=0x7f897c070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:01.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f8982310700 1 --2- 172.21.15.112:0/2744664685 >> v2:172.21.15.138:3300/0 conn(0x7f897c0cbe00 0x7f897c070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f8982310700 1 -- 172.21.15.112:0/2744664685 >> v1:172.21.15.138:6789/0 conn(0x7f897c0cf130 legacy=0x7f897c0cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f8982310700 1 -- 172.21.15.112:0/2744664685 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c071940 con 0x7f897c15b2b0 2023-08-27T11:28:01.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f8981b0f700 1 --2- 172.21.15.112:0/2744664685 >> v2:172.21.15.138:3300/0 conn(0x7f897c0cbe00 0x7f897c070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:01.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f8982310700 1 --2- 172.21.15.112:0/2744664685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897c15b2b0 0x7f897c15f800 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f896c0098f0 tx=0x7f896c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8d9b6033714e041 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:01.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f898130e700 1 -- 172.21.15.112:0/2744664685 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c011830 con 0x7f897c15b2b0 2023-08-27T11:28:01.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f898130e700 1 -- 172.21.15.112:0/2744664685 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896c011990 con 0x7f897c15b2b0 2023-08-27T11:28:01.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.589+0000 7f898130e700 1 -- 172.21.15.112:0/2744664685 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c011d20 con 0x7f897c15b2b0 2023-08-27T11:28:01.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.590+0000 7f8984574700 1 -- 172.21.15.112:0/2744664685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897c15b2b0 msgr2=0x7f897c15f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.590+0000 7f8984574700 1 --2- 172.21.15.112:0/2744664685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897c15b2b0 0x7f897c15f800 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f896c0098f0 tx=0x7f896c009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:01.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.590+0000 7f8984574700 1 -- 172.21.15.112:0/2744664685 shutdown_connections 2023-08-27T11:28:01.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.590+0000 7f8984574700 1 --2- 172.21.15.112:0/2744664685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897c15b2b0 0x7f897c15f800 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.590+0000 7f8984574700 1 --2- 172.21.15.112:0/2744664685 >> v2:172.21.15.138:3300/0 conn(0x7f897c0cbe00 0x7f897c070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.590+0000 7f8984574700 1 -- 172.21.15.112:0/2744664685 >> 172.21.15.112:0/2744664685 conn(0x7f897c066720 msgr2=0x7f897c06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:01.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.590+0000 7f8984574700 1 -- 172.21.15.112:0/2744664685 shutdown_connections 2023-08-27T11:28:01.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.590+0000 7f8984574700 1 -- 172.21.15.112:0/2744664685 wait complete. 2023-08-27T11:28:01.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.591+0000 7f8984574700 1 Processor -- start 2023-08-27T11:28:01.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.591+0000 7f8984574700 1 -- start start 2023-08-27T11:28:01.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8984574700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f897c0cbe00 0x7f897c15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8984574700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897c0cf130 0x7f897c0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8984574700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897c15b2b0 0x7f897c15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8984574700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c15dbd0 con 0x7f897c0cf130 2023-08-27T11:28:01.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8984574700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c15b690 con 0x7f897c0cbe00 2023-08-27T11:28:01.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8984574700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c15b8a0 con 0x7f897c15b2b0 2023-08-27T11:28:01.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8981b0f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897c0cf130 0x7f897c0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8981b0f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897c0cf130 0x7f897c0cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35688/0 (socket says 172.21.15.112:35688) 2023-08-27T11:28:01.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8981b0f700 1 -- 172.21.15.112:0/540929522 learned_addr learned my addr 172.21.15.112:0/540929522 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:01.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8982310700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f897c0cbe00 0x7f897c15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8982b11700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897c15b2b0 0x7f897c15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8981b0f700 1 -- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897c15b2b0 msgr2=0x7f897c15cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8981b0f700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897c15b2b0 0x7f897c15cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8981b0f700 1 -- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f897c0cbe00 msgr2=0x7f897c15c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8981b0f700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f897c0cbe00 0x7f897c15c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8981b0f700 1 -- 172.21.15.112:0/540929522 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89780094a0 con 0x7f897c0cf130 2023-08-27T11:28:01.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.592+0000 7f8981b0f700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897c0cf130 0x7f897c0cf780 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f897800ebe0 tx=0x7f897800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:01.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.593+0000 7f8972ffd700 1 -- 172.21.15.112:0/540929522 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978009760 con 0x7f897c0cf130 2023-08-27T11:28:01.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.593+0000 7f8972ffd700 1 -- 172.21.15.112:0/540929522 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897800c850 con 0x7f897c0cf130 2023-08-27T11:28:01.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.593+0000 7f8984574700 1 -- 172.21.15.112:0/540929522 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896c009550 con 0x7f897c0cf130 2023-08-27T11:28:01.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.593+0000 7f8972ffd700 1 -- 172.21.15.112:0/540929522 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897800cbe0 con 0x7f897c0cf130 2023-08-27T11:28:01.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.593+0000 7f8984574700 1 -- 172.21.15.112:0/540929522 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c15c140 con 0x7f897c0cf130 2023-08-27T11:28:01.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.594+0000 7f8984574700 1 -- 172.21.15.112:0/540929522 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c067a20 con 0x7f897c0cf130 2023-08-27T11:28:01.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.594+0000 7f8972ffd700 1 -- 172.21.15.112:0/540929522 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f897800cd40 con 0x7f897c0cf130 2023-08-27T11:28:01.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.594+0000 7f8972ffd700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f896006a4d0 0x7f896006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.594+0000 7f8972ffd700 1 -- 172.21.15.112:0/540929522 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6320+0+0 (secure 0 0 0) 0x7f8978089060 con 0x7f897c0cf130 2023-08-27T11:28:01.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.595+0000 7f8982310700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f896006a4d0 0x7f896006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.595+0000 7f8982310700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f896006a4d0 0x7f896006c990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f896c009c40 tx=0x7f896c01d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:01.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.599+0000 7f8972ffd700 1 -- 172.21.15.112:0/540929522 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f89780bb990 con 0x7f897c0cf130 2023-08-27T11:28:01.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:01.834+0000 7f8984574700 1 -- 172.21.15.112:0/540929522 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f897c0d0110 con 0x7f897c0cf130 2023-08-27T11:28:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.451+0000 7f8972ffd700 1 -- 172.21.15.112:0/540929522 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v88) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8978054100 con 0x7f897c0cf130 2023-08-27T11:28:02.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.454+0000 7f8984574700 1 -- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f896006a4d0 msgr2=0x7f896006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:02.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.454+0000 7f8984574700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f896006a4d0 0x7f896006c990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f896c009c40 tx=0x7f896c01d040 comp rx=0 tx=0).stop 2023-08-27T11:28:02.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.454+0000 7f8984574700 1 -- 172.21.15.112:0/540929522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897c0cf130 msgr2=0x7f897c0cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:02.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.454+0000 7f8984574700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897c0cf130 0x7f897c0cf780 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f897800ebe0 tx=0x7f897800eef0 comp rx=0 tx=0).stop 2023-08-27T11:28:02.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.454+0000 7f8984574700 1 -- 172.21.15.112:0/540929522 shutdown_connections 2023-08-27T11:28:02.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.454+0000 7f8984574700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f896006a4d0 0x7f896006c990 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:02.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.454+0000 7f8984574700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897c15b2b0 0x7f897c15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:02.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.454+0000 7f8984574700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f897c0cbe00 0x7f897c15c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:02.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.454+0000 7f8984574700 1 --2- 172.21.15.112:0/540929522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897c0cf130 0x7f897c0cf780 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:02.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.454+0000 7f8984574700 1 -- 172.21.15.112:0/540929522 >> 172.21.15.112:0/540929522 conn(0x7f897c066720 msgr2=0x7f897c0b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:02.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.454+0000 7f8984574700 1 -- 172.21.15.112:0/540929522 shutdown_connections 2023-08-27T11:28:02.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.454+0000 7f8984574700 1 -- 172.21.15.112:0/540929522 wait complete. 2023-08-27T11:28:02.468 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:28:02.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.591+0000 7f2b288e0700 1 Processor -- start 2023-08-27T11:28:02.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.591+0000 7f2b288e0700 1 -- start start 2023-08-27T11:28:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b288e0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b24157ae0 0x7f2b2415e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b288e0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2b2406f790 0x7f2b2406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b288e0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b24161390 con 0x7f2b240b3de0 2023-08-27T11:28:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b288e0700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b240b1aa0 con 0x7f2b2406f790 2023-08-27T11:28:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b288e0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b24070180 con 0x7f2b24157ae0 2023-08-27T11:28:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b22d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2b2406f790 0x7f2b2406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b22d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2b2406f790 0x7f2b2406fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35696/0 (socket says 172.21.15.112:35696) 2023-08-27T11:28:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b22d9d700 1 -- 172.21.15.112:0/1668821550 learned_addr learned my addr 172.21.15.112:0/1668821550 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b21d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2b240b3de0 0x7f2b24161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58308/0 (socket says 172.21.15.112:58308) 2023-08-27T11:28:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b2259c700 1 --2- 172.21.15.112:0/1668821550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b24157ae0 0x7f2b2415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b22d9d700 1 -- 172.21.15.112:0/1668821550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b24157ae0 msgr2=0x7f2b2415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b22d9d700 1 --2- 172.21.15.112:0/1668821550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b24157ae0 0x7f2b2415e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b22d9d700 1 -- 172.21.15.112:0/1668821550 >> v1:172.21.15.112:6789/0 conn(0x7f2b240b3de0 legacy=0x7f2b24161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b22d9d700 1 -- 172.21.15.112:0/1668821550 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b240b35f0 con 0x7f2b2406f790 2023-08-27T11:28:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.592+0000 7f2b2259c700 1 --2- 172.21.15.112:0/1668821550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b24157ae0 0x7f2b2415e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.593+0000 7f2b22d9d700 1 --2- 172.21.15.112:0/1668821550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b2406f790 0x7f2b2406fc40 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009e50 tx=0x7f2b1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc71fad44c511402 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.593+0000 7f2b2159a700 1 -- 172.21.15.112:0/1668821550 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00e070 con 0x7f2b2406f790 2023-08-27T11:28:02.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.593+0000 7f2b2159a700 1 -- 172.21.15.112:0/1668821550 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c00b9c0 con 0x7f2b2406f790 2023-08-27T11:28:02.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.593+0000 7f2b2159a700 1 -- 172.21.15.112:0/1668821550 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00bd70 con 0x7f2b2406f790 2023-08-27T11:28:02.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.593+0000 7f2b288e0700 1 -- 172.21.15.112:0/1668821550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b2406f790 msgr2=0x7f2b2406fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:02.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.593+0000 7f2b288e0700 1 --2- 172.21.15.112:0/1668821550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b2406f790 0x7f2b2406fc40 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009e50 tx=0x7f2b1c00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:02.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.593+0000 7f2b288e0700 1 -- 172.21.15.112:0/1668821550 shutdown_connections 2023-08-27T11:28:02.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.593+0000 7f2b288e0700 1 --2- 172.21.15.112:0/1668821550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b2406f790 0x7f2b2406fc40 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:02.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.593+0000 7f2b288e0700 1 --2- 172.21.15.112:0/1668821550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b24157ae0 0x7f2b2415e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:02.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.593+0000 7f2b288e0700 1 -- 172.21.15.112:0/1668821550 >> 172.21.15.112:0/1668821550 conn(0x7f2b240b1be0 msgr2=0x7f2b240b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:02.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.593+0000 7f2b288e0700 1 -- 172.21.15.112:0/1668821550 shutdown_connections 2023-08-27T11:28:02.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.593+0000 7f2b288e0700 1 -- 172.21.15.112:0/1668821550 wait complete. 2023-08-27T11:28:02.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.595+0000 7f2b288e0700 1 Processor -- start 2023-08-27T11:28:02.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.595+0000 7f2b288e0700 1 -- start start 2023-08-27T11:28:02.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.595+0000 7f2b288e0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b2406f790 0x7f2b2415f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:02.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.595+0000 7f2b288e0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b240b3de0 0x7f2b241f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:02.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.595+0000 7f2b288e0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b24157ae0 0x7f2b241f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:02.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.595+0000 7f2b288e0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2415dfa0 con 0x7f2b240b3de0 2023-08-27T11:28:02.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.595+0000 7f2b288e0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2415e280 con 0x7f2b2406f790 2023-08-27T11:28:02.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.595+0000 7f2b288e0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2415e560 con 0x7f2b24157ae0 2023-08-27T11:28:02.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b21d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b240b3de0 0x7f2b241f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:02.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b21d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b240b3de0 0x7f2b241f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35700/0 (socket says 172.21.15.112:35700) 2023-08-27T11:28:02.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b21d9b700 1 -- 172.21.15.112:0/1421862269 learned_addr learned my addr 172.21.15.112:0/1421862269 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:02.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b22d9d700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b24157ae0 0x7f2b241f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:02.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b2259c700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b2406f790 0x7f2b2415f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:02.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b21d9b700 1 -- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b24157ae0 msgr2=0x7f2b241f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:02.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b21d9b700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b24157ae0 0x7f2b241f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:02.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b21d9b700 1 -- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b2406f790 msgr2=0x7f2b2415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:02.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b21d9b700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b2406f790 0x7f2b2415f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:02.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b21d9b700 1 -- 172.21.15.112:0/1421862269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c009b30 con 0x7f2b240b3de0 2023-08-27T11:28:02.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b21d9b700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b240b3de0 0x7f2b241f3f30 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400c4f0 tx=0x7f2b1400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:02.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b12ffd700 1 -- 172.21.15.112:0/1421862269 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14005cb0 con 0x7f2b240b3de0 2023-08-27T11:28:02.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b12ffd700 1 -- 172.21.15.112:0/1421862269 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b14005e10 con 0x7f2b240b3de0 2023-08-27T11:28:02.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b288e0700 1 -- 172.21.15.112:0/1421862269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2415e840 con 0x7f2b240b3de0 2023-08-27T11:28:02.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b12ffd700 1 -- 172.21.15.112:0/1421862269 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b140072d0 con 0x7f2b240b3de0 2023-08-27T11:28:02.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.596+0000 7f2b288e0700 1 -- 172.21.15.112:0/1421862269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2415f580 con 0x7f2b240b3de0 2023-08-27T11:28:02.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.597+0000 7f2b288e0700 1 -- 172.21.15.112:0/1421862269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2415ed60 con 0x7f2b240b3de0 2023-08-27T11:28:02.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.598+0000 7f2b12ffd700 1 -- 172.21.15.112:0/1421862269 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f2b14007430 con 0x7f2b240b3de0 2023-08-27T11:28:02.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.598+0000 7f2b12ffd700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b0406a4d0 0x7f2b0406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:02.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.598+0000 7f2b12ffd700 1 -- 172.21.15.112:0/1421862269 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6320+0+0 (secure 0 0 0) 0x7f2b14088150 con 0x7f2b240b3de0 2023-08-27T11:28:02.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.598+0000 7f2b2259c700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b0406a4d0 0x7f2b0406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:02.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.599+0000 7f2b2259c700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b0406a4d0 0x7f2b0406c990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2b18006fd0 tx=0x7f2b18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:02.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.602+0000 7f2b12ffd700 1 -- 172.21.15.112:0/1421862269 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2b2415ed60 con 0x7f2b240b3de0 2023-08-27T11:28:02.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:02.791+0000 7f2b288e0700 1 -- 172.21.15.112:0/1421862269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2b2415ed60 con 0x7f2b240b3de0 2023-08-27T11:28:03.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.586+0000 7f2b12ffd700 1 -- 172.21.15.112:0/1421862269 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2b2415ed60 con 0x7f2b240b3de0 2023-08-27T11:28:03.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.589+0000 7f2b288e0700 1 -- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b0406a4d0 msgr2=0x7f2b0406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.589+0000 7f2b288e0700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b0406a4d0 0x7f2b0406c990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2b18006fd0 tx=0x7f2b18008040 comp rx=0 tx=0).stop 2023-08-27T11:28:03.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.589+0000 7f2b288e0700 1 -- 172.21.15.112:0/1421862269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b240b3de0 msgr2=0x7f2b241f3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.589+0000 7f2b288e0700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b240b3de0 0x7f2b241f3f30 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400c4f0 tx=0x7f2b1400c8b0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.589+0000 7f2b288e0700 1 -- 172.21.15.112:0/1421862269 shutdown_connections 2023-08-27T11:28:03.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.589+0000 7f2b288e0700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b0406a4d0 0x7f2b0406c990 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.589+0000 7f2b288e0700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b24157ae0 0x7f2b241f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.589+0000 7f2b288e0700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b2406f790 0x7f2b2415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.589+0000 7f2b288e0700 1 --2- 172.21.15.112:0/1421862269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b240b3de0 0x7f2b241f3f30 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.589+0000 7f2b288e0700 1 -- 172.21.15.112:0/1421862269 >> 172.21.15.112:0/1421862269 conn(0x7f2b240b1be0 msgr2=0x7f2b24145810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:03.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.590+0000 7f2b288e0700 1 -- 172.21.15.112:0/1421862269 shutdown_connections 2023-08-27T11:28:03.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.590+0000 7f2b288e0700 1 -- 172.21.15.112:0/1421862269 wait complete. 2023-08-27T11:28:03.594 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:28:03.607 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:28:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.733+0000 7f2b359e0700 1 Processor -- start 2023-08-27T11:28:03.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.733+0000 7f2b359e0700 1 -- start start 2023-08-27T11:28:03.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b359e0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b30146d50 0x7f2b300b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b359e0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2b300b5750 0x7f2b300d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b359e0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b300b7f60 con 0x7f2b300b69b0 2023-08-27T11:28:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b359e0700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b300b6ec0 con 0x7f2b300b5750 2023-08-27T11:28:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b359e0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b300b71a0 con 0x7f2b30146d50 2023-08-27T11:28:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b2f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2b300b5750 0x7f2b300d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b2f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2b300b5750 0x7f2b300d0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35710/0 (socket says 172.21.15.112:35710) 2023-08-27T11:28:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b2e7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2b300b69b0 0x7f2b300b5040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58320/0 (socket says 172.21.15.112:58320) 2023-08-27T11:28:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b2f7fe700 1 -- 172.21.15.112:0/1359773040 learned_addr learned my addr 172.21.15.112:0/1359773040 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b2effd700 1 --2- 172.21.15.112:0/1359773040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b30146d50 0x7f2b300b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b2f7fe700 1 -- 172.21.15.112:0/1359773040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b30146d50 msgr2=0x7f2b300b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b2f7fe700 1 --2- 172.21.15.112:0/1359773040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b30146d50 0x7f2b300b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b2f7fe700 1 -- 172.21.15.112:0/1359773040 >> v1:172.21.15.112:6789/0 conn(0x7f2b300b69b0 legacy=0x7f2b300b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b2f7fe700 1 -- 172.21.15.112:0/1359773040 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b300b7480 con 0x7f2b300b5750 2023-08-27T11:28:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b2dffb700 1 -- 172.21.15.112:0/1359773040 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200421946 0 0) 0x7f2b300b7f60 con 0x7f2b300b69b0 2023-08-27T11:28:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.734+0000 7f2b2f7fe700 1 --2- 172.21.15.112:0/1359773040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b300b5750 0x7f2b300d0cb0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009e50 tx=0x7f2b2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fb2d2060b5676b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.735+0000 7f2b2dffb700 1 -- 172.21.15.112:0/1359773040 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2800e070 con 0x7f2b300b5750 2023-08-27T11:28:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.735+0000 7f2b2dffb700 1 -- 172.21.15.112:0/1359773040 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2800b9c0 con 0x7f2b300b5750 2023-08-27T11:28:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.735+0000 7f2b2dffb700 1 -- 172.21.15.112:0/1359773040 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2800bd70 con 0x7f2b300b5750 2023-08-27T11:28:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.735+0000 7f2b359e0700 1 -- 172.21.15.112:0/1359773040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b300b5750 msgr2=0x7f2b300d0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.735+0000 7f2b359e0700 1 --2- 172.21.15.112:0/1359773040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b300b5750 0x7f2b300d0cb0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009e50 tx=0x7f2b2800b330 comp rx=0 tx=0).stop 2023-08-27T11:28:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.735+0000 7f2b359e0700 1 -- 172.21.15.112:0/1359773040 shutdown_connections 2023-08-27T11:28:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.735+0000 7f2b359e0700 1 --2- 172.21.15.112:0/1359773040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b300b5750 0x7f2b300d0cb0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.735+0000 7f2b359e0700 1 --2- 172.21.15.112:0/1359773040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b30146d50 0x7f2b300b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.735+0000 7f2b359e0700 1 -- 172.21.15.112:0/1359773040 >> 172.21.15.112:0/1359773040 conn(0x7f2b3006d330 msgr2=0x7f2b300717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.735+0000 7f2b359e0700 1 -- 172.21.15.112:0/1359773040 shutdown_connections 2023-08-27T11:28:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.735+0000 7f2b359e0700 1 -- 172.21.15.112:0/1359773040 wait complete. 2023-08-27T11:28:03.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.737+0000 7f2b359e0700 1 Processor -- start 2023-08-27T11:28:03.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.737+0000 7f2b359e0700 1 -- start start 2023-08-27T11:28:03.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b359e0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b300b5750 0x7f2b3015c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b359e0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b300b69b0 0x7f2b300d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b359e0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b30146d50 0x7f2b301456f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b359e0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3015f850 con 0x7f2b30146d50 2023-08-27T11:28:03.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b359e0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b301443b0 con 0x7f2b300b69b0 2023-08-27T11:28:03.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b359e0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b30144650 con 0x7f2b300b5750 2023-08-27T11:28:03.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b2f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b30146d50 0x7f2b301456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b2f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b30146d50 0x7f2b301456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35716/0 (socket says 172.21.15.112:35716) 2023-08-27T11:28:03.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b2f7fe700 1 -- 172.21.15.112:0/1504687470 learned_addr learned my addr 172.21.15.112:0/1504687470 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:03.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b2effd700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b300b5750 0x7f2b3015c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b2e7fc700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b300b69b0 0x7f2b300d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b2f7fe700 1 -- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b300b5750 msgr2=0x7f2b3015c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b2f7fe700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b300b5750 0x7f2b3015c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b2f7fe700 1 -- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b300b69b0 msgr2=0x7f2b300d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b2f7fe700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b300b69b0 0x7f2b300d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b2f7fe700 1 -- 172.21.15.112:0/1504687470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b28009b30 con 0x7f2b30146d50 2023-08-27T11:28:03.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b2e7fc700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b300b69b0 0x7f2b300d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:03.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b2f7fe700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b30146d50 0x7f2b301456f0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800b730 tx=0x7f2b28004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.738+0000 7f2b17fff700 1 -- 172.21.15.112:0/1504687470 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b280099b0 con 0x7f2b30146d50 2023-08-27T11:28:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.739+0000 7f2b17fff700 1 -- 172.21.15.112:0/1504687470 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b280092e0 con 0x7f2b30146d50 2023-08-27T11:28:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.739+0000 7f2b359e0700 1 -- 172.21.15.112:0/1504687470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b30144930 con 0x7f2b30146d50 2023-08-27T11:28:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.739+0000 7f2b17fff700 1 -- 172.21.15.112:0/1504687470 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2801c660 con 0x7f2b30146d50 2023-08-27T11:28:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.739+0000 7f2b359e0700 1 -- 172.21.15.112:0/1504687470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b30144ef0 con 0x7f2b30146d50 2023-08-27T11:28:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.740+0000 7f2b359e0700 1 -- 172.21.15.112:0/1504687470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b30067a20 con 0x7f2b30146d50 2023-08-27T11:28:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.740+0000 7f2b17fff700 1 -- 172.21.15.112:0/1504687470 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f2b2801c7c0 con 0x7f2b30146d50 2023-08-27T11:28:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.740+0000 7f2b17fff700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b1c06a4d0 0x7f2b1c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.740+0000 7f2b17fff700 1 -- 172.21.15.112:0/1504687470 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f2b2808a9d0 con 0x7f2b30146d50 2023-08-27T11:28:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.741+0000 7f2b2effd700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b1c06a4d0 0x7f2b1c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.741+0000 7f2b2effd700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b1c06a4d0 0x7f2b1c06c990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400b9b0 tx=0x7f2b24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.745+0000 7f2b17fff700 1 -- 172.21.15.112:0/1504687470 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2b28055c30 con 0x7f2b30146d50 2023-08-27T11:28:03.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:03.931+0000 7f2b359e0700 1 -- 172.21.15.112:0/1504687470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2b300b2db0 con 0x7f2b30146d50 2023-08-27T11:28:04.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.599+0000 7f2b17fff700 1 -- 172.21.15.112:0/1504687470 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2b28059250 con 0x7f2b30146d50 2023-08-27T11:28:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.602+0000 7f2b359e0700 1 -- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b1c06a4d0 msgr2=0x7f2b1c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.602+0000 7f2b359e0700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b1c06a4d0 0x7f2b1c06c990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400b9b0 tx=0x7f2b24008040 comp rx=0 tx=0).stop 2023-08-27T11:28:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.602+0000 7f2b359e0700 1 -- 172.21.15.112:0/1504687470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b30146d50 msgr2=0x7f2b301456f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.602+0000 7f2b359e0700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b30146d50 0x7f2b301456f0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800b730 tx=0x7f2b28004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.602+0000 7f2b359e0700 1 -- 172.21.15.112:0/1504687470 shutdown_connections 2023-08-27T11:28:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.602+0000 7f2b359e0700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b1c06a4d0 0x7f2b1c06c990 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.602+0000 7f2b359e0700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b30146d50 0x7f2b301456f0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.602+0000 7f2b359e0700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b300b5750 0x7f2b3015c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.602+0000 7f2b359e0700 1 --2- 172.21.15.112:0/1504687470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b300b69b0 0x7f2b300d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.602+0000 7f2b359e0700 1 -- 172.21.15.112:0/1504687470 >> 172.21.15.112:0/1504687470 conn(0x7f2b3006d330 msgr2=0x7f2b30071840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:04.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.602+0000 7f2b359e0700 1 -- 172.21.15.112:0/1504687470 shutdown_connections 2023-08-27T11:28:04.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.602+0000 7f2b359e0700 1 -- 172.21.15.112:0/1504687470 wait complete. 2023-08-27T11:28:04.608 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2023-08-27T11:28:04.617 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:28:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.749+0000 7f0b36641700 1 Processor -- start 2023-08-27T11:28:04.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.749+0000 7f0b36641700 1 -- start start 2023-08-27T11:28:04.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.749+0000 7f0b36641700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b30161dd0 0x7f0b300b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b36641700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0b300cc980 0x7f0b300ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b36641700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b300b46c0 con 0x7f0b3015e190 2023-08-27T11:28:04.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b36641700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b300cd330 con 0x7f0b300cc980 2023-08-27T11:28:04.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b36641700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300b49a0 con 0x7f0b30161dd0 2023-08-27T11:28:04.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2ffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0b3015e190 0x7f0b3015e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58334/0 (socket says 172.21.15.112:58334) 2023-08-27T11:28:04.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2ffff700 1 -- 172.21.15.112:0/778860755 learned_addr learned my addr 172.21.15.112:0/778860755 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:04.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605010138 0 0) 0x7f0b300b46c0 con 0x7f0b3015e190 2023-08-27T11:28:04.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b180035b0 con 0x7f0b3015e190 2023-08-27T11:28:04.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2f7fe700 1 --2- 172.21.15.112:0/778860755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b30161dd0 0x7f0b300b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b34bde700 1 --2- 172.21.15.112:0/778860755 >> v2:172.21.15.138:3300/0 conn(0x7f0b300cc980 0x7f0b300ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1126530498 0 0) 0x7f0b180035b0 con 0x7f0b3015e190 2023-08-27T11:28:04.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b30161dd0 msgr2=0x7f0b300b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:04.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 --2- 172.21.15.112:0/778860755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b30161dd0 0x7f0b300b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 >> v2:172.21.15.138:3300/0 conn(0x7f0b300cc980 msgr2=0x7f0b300ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:04.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 --2- 172.21.15.112:0/778860755 >> v2:172.21.15.138:3300/0 conn(0x7f0b300cc980 0x7f0b300ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b301612f0 con 0x7f0b3015e190 2023-08-27T11:28:04.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f0b20001670 con 0x7f0b3015e190 2023-08-27T11:28:04.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 >> v1:172.21.15.112:6789/0 conn(0x7f0b3015e190 legacy=0x7f0b3015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 --2- 172.21.15.112:0/778860755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b18003a00 0x7f0b18003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 --2- 172.21.15.112:0/778860755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b30161dd0 0x7f0b18004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 --2- 172.21.15.112:0/778860755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b18004d90 0x7f0b18005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300cd330 con 0x7f0b18004d90 2023-08-27T11:28:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300b49a0 con 0x7f0b18003a00 2023-08-27T11:28:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b18014220 con 0x7f0b30161dd0 2023-08-27T11:28:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b34bde700 1 --2- 172.21.15.112:0/778860755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b18004d90 0x7f0b18005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.750+0000 7f0b2ffff700 1 --2- 172.21.15.112:0/778860755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b18003a00 0x7f0b18003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.751+0000 7f0b2f7fe700 1 --2- 172.21.15.112:0/778860755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b30161dd0 0x7f0b18004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.751+0000 7f0b34bde700 1 -- 172.21.15.112:0/778860755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b30161dd0 msgr2=0x7f0b18004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.751+0000 7f0b34bde700 1 --2- 172.21.15.112:0/778860755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b30161dd0 0x7f0b18004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.751+0000 7f0b34bde700 1 -- 172.21.15.112:0/778860755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b18003a00 msgr2=0x7f0b18003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.751+0000 7f0b34bde700 1 --2- 172.21.15.112:0/778860755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b18003a00 0x7f0b18003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.751+0000 7f0b34bde700 1 -- 172.21.15.112:0/778860755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b180035b0 con 0x7f0b18004d90 2023-08-27T11:28:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.751+0000 7f0b34bde700 1 --2- 172.21.15.112:0/778860755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b18004d90 0x7f0b18005210 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800e170 tx=0x7f0b18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9defef0371123a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.751+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2800ec90 con 0x7f0b18004d90 2023-08-27T11:28:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.751+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2800f040 con 0x7f0b18004d90 2023-08-27T11:28:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.751+0000 7f0b2effd700 1 -- 172.21.15.112:0/778860755 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28014480 con 0x7f0b18004d90 2023-08-27T11:28:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.751+0000 7f0b36641700 1 -- 172.21.15.112:0/778860755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b18004d90 msgr2=0x7f0b18005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.751+0000 7f0b36641700 1 --2- 172.21.15.112:0/778860755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b18004d90 0x7f0b18005210 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800e170 tx=0x7f0b18005880 comp rx=0 tx=0).stop 2023-08-27T11:28:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.751+0000 7f0b34bde700 1 -- 172.21.15.112:0/778860755 reap_dead start 2023-08-27T11:28:04.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.752+0000 7f0b36641700 1 -- 172.21.15.112:0/778860755 shutdown_connections 2023-08-27T11:28:04.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.752+0000 7f0b36641700 1 -- 172.21.15.112:0/778860755 >> 172.21.15.112:0/778860755 conn(0x7f0b3006bbf0 msgr2=0x7f0b300cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:04.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.752+0000 7f0b36641700 1 -- 172.21.15.112:0/778860755 shutdown_connections 2023-08-27T11:28:04.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.752+0000 7f0b36641700 1 -- 172.21.15.112:0/778860755 wait complete. 2023-08-27T11:28:04.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.753+0000 7f0b36641700 1 Processor -- start 2023-08-27T11:28:04.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.753+0000 7f0b36641700 1 -- start start 2023-08-27T11:28:04.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b36641700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b3015ece0 0x7f0b3015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b36641700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b3006fff0 0x7f0b30070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b36641700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b3013f460 0x7f0b3013f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b36641700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b301582b0 con 0x7f0b3013f460 2023-08-27T11:28:04.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b36641700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3013fe20 con 0x7f0b3015ece0 2023-08-27T11:28:04.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b36641700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b301402f0 con 0x7f0b3006fff0 2023-08-27T11:28:04.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b34bde700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b3013f460 0x7f0b3013f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b34bde700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b3013f460 0x7f0b3013f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35736/0 (socket says 172.21.15.112:35736) 2023-08-27T11:28:04.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b34bde700 1 -- 172.21.15.112:0/463064469 learned_addr learned my addr 172.21.15.112:0/463064469 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:04.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b2ffff700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b3015ece0 0x7f0b3015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b2f7fe700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b3006fff0 0x7f0b30070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b34bde700 1 -- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b3006fff0 msgr2=0x7f0b30070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b34bde700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b3006fff0 0x7f0b30070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b34bde700 1 -- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b3015ece0 msgr2=0x7f0b3015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b34bde700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b3015ece0 0x7f0b3015f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b34bde700 1 -- 172.21.15.112:0/463064469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3015f600 con 0x7f0b3013f460 2023-08-27T11:28:04.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b34bde700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b3013f460 0x7f0b3013f8e0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800e760 tx=0x7f0b28004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:04.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b2cff9700 1 -- 172.21.15.112:0/463064469 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28007000 con 0x7f0b3013f460 2023-08-27T11:28:04.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b2cff9700 1 -- 172.21.15.112:0/463064469 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2800ee80 con 0x7f0b3013f460 2023-08-27T11:28:04.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b36641700 1 -- 172.21.15.112:0/463064469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b30070b80 con 0x7f0b3013f460 2023-08-27T11:28:04.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b2cff9700 1 -- 172.21.15.112:0/463064469 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28003a20 con 0x7f0b3013f460 2023-08-27T11:28:04.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.754+0000 7f0b36641700 1 -- 172.21.15.112:0/463064469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b30071140 con 0x7f0b3013f460 2023-08-27T11:28:04.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.755+0000 7f0b36641700 1 -- 172.21.15.112:0/463064469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b30066fa0 con 0x7f0b3013f460 2023-08-27T11:28:04.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.756+0000 7f0b2cff9700 1 -- 172.21.15.112:0/463064469 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f0b28003b80 con 0x7f0b3013f460 2023-08-27T11:28:04.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.756+0000 7f0b2cff9700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b1c06a4d0 0x7f0b1c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.756+0000 7f0b2cff9700 1 -- 172.21.15.112:0/463064469 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f0b28089fa0 con 0x7f0b3013f460 2023-08-27T11:28:04.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.756+0000 7f0b2ffff700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b1c06a4d0 0x7f0b1c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.757+0000 7f0b2ffff700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b1c06a4d0 0x7f0b1c06c990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0b200048d0 tx=0x7f0b20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:04.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.761+0000 7f0b2cff9700 1 -- 172.21.15.112:0/463064469 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0b280553a0 con 0x7f0b3013f460 2023-08-27T11:28:04.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.950+0000 7f0b36641700 1 -- 172.21.15.112:0/463064469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0b30067a20 con 0x7f0b3013f460 2023-08-27T11:28:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.953+0000 7f0b2cff9700 1 -- 172.21.15.112:0/463064469 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0b280589c0 con 0x7f0b3013f460 2023-08-27T11:28:04.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.956+0000 7f0b36641700 1 -- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b1c06a4d0 msgr2=0x7f0b1c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.956+0000 7f0b36641700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b1c06a4d0 0x7f0b1c06c990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0b200048d0 tx=0x7f0b20009040 comp rx=0 tx=0).stop 2023-08-27T11:28:04.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.956+0000 7f0b36641700 1 -- 172.21.15.112:0/463064469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b3013f460 msgr2=0x7f0b3013f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.956+0000 7f0b36641700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b3013f460 0x7f0b3013f8e0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800e760 tx=0x7f0b28004b40 comp rx=0 tx=0).stop 2023-08-27T11:28:04.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.956+0000 7f0b36641700 1 -- 172.21.15.112:0/463064469 shutdown_connections 2023-08-27T11:28:04.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.956+0000 7f0b36641700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0b1c06a4d0 0x7f0b1c06c990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.956+0000 7f0b36641700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b3013f460 0x7f0b3013f8e0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.956+0000 7f0b36641700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b3015ece0 0x7f0b3015f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.956+0000 7f0b36641700 1 --2- 172.21.15.112:0/463064469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b3006fff0 0x7f0b30070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.956+0000 7f0b36641700 1 -- 172.21.15.112:0/463064469 >> 172.21.15.112:0/463064469 conn(0x7f0b3006bbf0 msgr2=0x7f0b300b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:04.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.956+0000 7f0b36641700 1 -- 172.21.15.112:0/463064469 shutdown_connections 2023-08-27T11:28:04.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:04.956+0000 7f0b36641700 1 -- 172.21.15.112:0/463064469 wait complete. 2023-08-27T11:28:04.970 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls' 2023-08-27T11:28:05.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.088+0000 7fd99eacd700 1 Processor -- start 2023-08-27T11:28:05.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.088+0000 7fd99eacd700 1 -- start start 2023-08-27T11:28:05.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.089+0000 7fd99eacd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd9980ca740 0x7fd9980b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.089+0000 7fd99eacd700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9980cf2a0 con 0x7fd99815ae80 2023-08-27T11:28:05.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.089+0000 7fd99eacd700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9980cf580 con 0x7fd9980d0560 2023-08-27T11:28:05.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.089+0000 7fd99eacd700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9980b0c50 con 0x7fd9980ca740 2023-08-27T11:28:05.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.089+0000 7fd997fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd9980ca740 0x7fd9980b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.089+0000 7fd997fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd9980ca740 0x7fd9980b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35742/0 (socket says 172.21.15.112:35742) 2023-08-27T11:28:05.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.089+0000 7fd997fff700 1 -- 172.21.15.112:0/3639199306 learned_addr learned my addr 172.21.15.112:0/3639199306 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.089+0000 7fd99c869700 1 --1- 172.21.15.112:0/3639199306 >> v1:172.21.15.112:6789/0 conn(0x7fd99815ae80 0x7fd9980708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58354/0 (socket says 172.21.15.112:58354) 2023-08-27T11:28:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.090+0000 7fd997fff700 1 -- 172.21.15.112:0/3639199306 >> v1:172.21.15.138:6789/0 conn(0x7fd9980d0560 legacy=0x7fd998160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.090+0000 7fd997fff700 1 -- 172.21.15.112:0/3639199306 >> v1:172.21.15.112:6789/0 conn(0x7fd99815ae80 legacy=0x7fd9980708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.090+0000 7fd997fff700 1 -- 172.21.15.112:0/3639199306 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9980cfa80 con 0x7fd9980ca740 2023-08-27T11:28:05.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.090+0000 7fd9977fe700 1 -- 172.21.15.112:0/3639199306 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861347962 0 0) 0x7fd9980cf2a0 con 0x7fd99815ae80 2023-08-27T11:28:05.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.090+0000 7fd997fff700 1 --2- 172.21.15.112:0/3639199306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9980ca740 0x7fd9980b0710 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd9840098f0 tx=0x7fd984009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dfb8b40c31d7f9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:05.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.090+0000 7fd9977fe700 1 -- 172.21.15.112:0/3639199306 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984004230 con 0x7fd9980ca740 2023-08-27T11:28:05.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.090+0000 7fd9977fe700 1 -- 172.21.15.112:0/3639199306 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd984004390 con 0x7fd9980ca740 2023-08-27T11:28:05.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.090+0000 7fd9977fe700 1 -- 172.21.15.112:0/3639199306 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9840036a0 con 0x7fd9980ca740 2023-08-27T11:28:05.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.090+0000 7fd99eacd700 1 -- 172.21.15.112:0/3639199306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9980ca740 msgr2=0x7fd9980b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.090+0000 7fd99eacd700 1 --2- 172.21.15.112:0/3639199306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9980ca740 0x7fd9980b0710 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd9840098f0 tx=0x7fd984009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:05.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.091+0000 7fd99eacd700 1 -- 172.21.15.112:0/3639199306 shutdown_connections 2023-08-27T11:28:05.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.091+0000 7fd99eacd700 1 --2- 172.21.15.112:0/3639199306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9980ca740 0x7fd9980b0710 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.091+0000 7fd99eacd700 1 -- 172.21.15.112:0/3639199306 >> 172.21.15.112:0/3639199306 conn(0x7fd99806f970 msgr2=0x7fd99806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:05.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.091+0000 7fd99eacd700 1 -- 172.21.15.112:0/3639199306 shutdown_connections 2023-08-27T11:28:05.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.091+0000 7fd99eacd700 1 -- 172.21.15.112:0/3639199306 wait complete. 2023-08-27T11:28:05.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.092+0000 7fd99eacd700 1 Processor -- start 2023-08-27T11:28:05.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.092+0000 7fd99eacd700 1 -- start start 2023-08-27T11:28:05.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd99eacd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9980ca740 0x7fd99815e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd99eacd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9980d0560 0x7fd998160520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd99eacd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd99815ae80 0x7fd99815c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd99eacd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd998145150 con 0x7fd9980d0560 2023-08-27T11:28:05.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd99eacd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd998144280 con 0x7fd99815ae80 2023-08-27T11:28:05.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd99eacd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9981444d0 con 0x7fd9980ca740 2023-08-27T11:28:05.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd997fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9980d0560 0x7fd998160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd997fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9980d0560 0x7fd998160520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35748/0 (socket says 172.21.15.112:35748) 2023-08-27T11:28:05.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd997fff700 1 -- 172.21.15.112:0/3502246632 learned_addr learned my addr 172.21.15.112:0/3502246632 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:05.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd99c869700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9980ca740 0x7fd99815e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd99d06a700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd99815ae80 0x7fd99815c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd997fff700 1 -- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9980ca740 msgr2=0x7fd99815e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd997fff700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9980ca740 0x7fd99815e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd997fff700 1 -- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd99815ae80 msgr2=0x7fd99815c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd997fff700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd99815ae80 0x7fd99815c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd997fff700 1 -- 172.21.15.112:0/3502246632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd984009550 con 0x7fd9980d0560 2023-08-27T11:28:05.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd997fff700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9980d0560 0x7fd998160520 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fd9980727f0 tx=0x7fd984004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:05.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd9957fa700 1 -- 172.21.15.112:0/3502246632 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98400b660 con 0x7fd9980d0560 2023-08-27T11:28:05.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd9957fa700 1 -- 172.21.15.112:0/3502246632 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98400b7c0 con 0x7fd9980d0560 2023-08-27T11:28:05.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd99eacd700 1 -- 172.21.15.112:0/3502246632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9981447b0 con 0x7fd9980d0560 2023-08-27T11:28:05.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd9957fa700 1 -- 172.21.15.112:0/3502246632 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98400bb50 con 0x7fd9980d0560 2023-08-27T11:28:05.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.093+0000 7fd99eacd700 1 -- 172.21.15.112:0/3502246632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd998144d70 con 0x7fd9980d0560 2023-08-27T11:28:05.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.094+0000 7fd99eacd700 1 -- 172.21.15.112:0/3502246632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd998145290 con 0x7fd9980d0560 2023-08-27T11:28:05.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.095+0000 7fd9957fa700 1 -- 172.21.15.112:0/3502246632 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fd984021070 con 0x7fd9980d0560 2023-08-27T11:28:05.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.095+0000 7fd9957fa700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd97c06a4d0 0x7fd97c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.095+0000 7fd9957fa700 1 -- 172.21.15.112:0/3502246632 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fd98408a300 con 0x7fd9980d0560 2023-08-27T11:28:05.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.095+0000 7fd99c869700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd97c06a4d0 0x7fd97c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.096+0000 7fd99c869700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd97c06a4d0 0x7fd97c06c990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd98c007fd0 tx=0x7fd98c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:05.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.099+0000 7fd9957fa700 1 -- 172.21.15.112:0/3502246632 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd998145290 con 0x7fd9980d0560 2023-08-27T11:28:05.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.369+0000 7fd99eacd700 1 -- 172.21.15.112:0/3502246632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]} v 0) v1 -- 0x7fd998145290 con 0x7fd9980d0560 2023-08-27T11:28:05.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.597+0000 7fd9957fa700 1 -- 172.21.15.112:0/3502246632 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]}]=0 v495) v1 ==== 131+0+0 (secure 0 0 0) 0x7fd998145290 con 0x7fd9980d0560 2023-08-27T11:28:05.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.599+0000 7fd99eacd700 1 -- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd97c06a4d0 msgr2=0x7fd97c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.599+0000 7fd99eacd700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd97c06a4d0 0x7fd97c06c990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd98c007fd0 tx=0x7fd98c009040 comp rx=0 tx=0).stop 2023-08-27T11:28:05.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.599+0000 7fd99eacd700 1 -- 172.21.15.112:0/3502246632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9980d0560 msgr2=0x7fd998160520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.599+0000 7fd99eacd700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9980d0560 0x7fd998160520 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fd9980727f0 tx=0x7fd984004750 comp rx=0 tx=0).stop 2023-08-27T11:28:05.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.599+0000 7fd99eacd700 1 -- 172.21.15.112:0/3502246632 shutdown_connections 2023-08-27T11:28:05.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.599+0000 7fd99eacd700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd97c06a4d0 0x7fd97c06c990 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.599+0000 7fd99eacd700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd99815ae80 0x7fd99815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.599+0000 7fd99eacd700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9980ca740 0x7fd99815e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.599+0000 7fd99eacd700 1 --2- 172.21.15.112:0/3502246632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9980d0560 0x7fd998160520 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.599+0000 7fd99eacd700 1 -- 172.21.15.112:0/3502246632 >> 172.21.15.112:0/3502246632 conn(0x7fd99806f970 msgr2=0x7fd998155ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.599+0000 7fd99eacd700 1 -- 172.21.15.112:0/3502246632 shutdown_connections 2023-08-27T11:28:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.599+0000 7fd99eacd700 1 -- 172.21.15.112:0/3502246632 wait complete. 2023-08-27T11:28:05.613 INFO:tasks.cephfs_test_runner:That the existing and the newly created volumes can be listed and ... ok 2023-08-27T11:28:05.613 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename (tasks.cephfs.test_volumes.TestVolumes) That volume, its file system and pools, can be renamed. 2023-08-27T11:28:05.614 INFO:tasks.cephfs_test_runner:test_volume_rename (tasks.cephfs.test_volumes.TestVolumes) 2023-08-27T11:28:05.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename' 2023-08-27T11:28:05.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.733+0000 7fcf12219700 1 Processor -- start 2023-08-27T11:28:05.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.733+0000 7fcf12219700 1 -- start start 2023-08-27T11:28:05.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf12219700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcf0c1404c0 0x7fcf0c142c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf12219700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcf0c0ad310 0x7fcf0c0cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf12219700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c0cce30 0x7fcf0c140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf12219700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0c1408a0 con 0x7fcf0c0ad310 2023-08-27T11:28:05.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf12219700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0c143490 con 0x7fcf0c1404c0 2023-08-27T11:28:05.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf12219700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c143770 con 0x7fcf0c0cce30 2023-08-27T11:28:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf0affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcf0c0ad310 0x7fcf0c0cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf0affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcf0c0ad310 0x7fcf0c0cc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35754/0 (socket says 172.21.15.112:35754) 2023-08-27T11:28:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf0affd700 1 -- 172.21.15.112:0/4022986424 learned_addr learned my addr 172.21.15.112:0/4022986424 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf0b7fe700 1 --2- 172.21.15.112:0/4022986424 >> v2:172.21.15.138:3300/0 conn(0x7fcf0c1404c0 0x7fcf0c142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf0bfff700 1 --2- 172.21.15.112:0/4022986424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c0cce30 0x7fcf0c140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf0affd700 1 -- 172.21.15.112:0/4022986424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c0cce30 msgr2=0x7fcf0c140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf0affd700 1 --2- 172.21.15.112:0/4022986424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c0cce30 0x7fcf0c140bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf0affd700 1 -- 172.21.15.112:0/4022986424 >> v2:172.21.15.138:3300/0 conn(0x7fcf0c1404c0 msgr2=0x7fcf0c142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf0affd700 1 --2- 172.21.15.112:0/4022986424 >> v2:172.21.15.138:3300/0 conn(0x7fcf0c1404c0 0x7fcf0c142c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf0affd700 1 -- 172.21.15.112:0/4022986424 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c143a50 con 0x7fcf0c0ad310 2023-08-27T11:28:05.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf0bfff700 1 --2- 172.21.15.112:0/4022986424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c0cce30 0x7fcf0c140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:05.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.734+0000 7fcf0b7fe700 1 --2- 172.21.15.112:0/4022986424 >> v2:172.21.15.138:3300/0 conn(0x7fcf0c1404c0 0x7fcf0c142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:05.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.735+0000 7fcf0affd700 1 --2- 172.21.15.112:0/4022986424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf0c0ad310 0x7fcf0c0cc8f0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c04e8f0 tx=0x7fcefc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfca1d007d48519 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:05.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.735+0000 7fcf0a7fc700 1 -- 172.21.15.112:0/4022986424 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc005e30 con 0x7fcf0c0ad310 2023-08-27T11:28:05.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.735+0000 7fcf0a7fc700 1 -- 172.21.15.112:0/4022986424 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcefc004030 con 0x7fcf0c0ad310 2023-08-27T11:28:05.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.735+0000 7fcf0a7fc700 1 -- 172.21.15.112:0/4022986424 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0043e0 con 0x7fcf0c0ad310 2023-08-27T11:28:05.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.735+0000 7fcf12219700 1 -- 172.21.15.112:0/4022986424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf0c0ad310 msgr2=0x7fcf0c0cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.735+0000 7fcf12219700 1 --2- 172.21.15.112:0/4022986424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf0c0ad310 0x7fcf0c0cc8f0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c04e8f0 tx=0x7fcefc009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.735+0000 7fcf12219700 1 -- 172.21.15.112:0/4022986424 shutdown_connections 2023-08-27T11:28:05.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.735+0000 7fcf12219700 1 --2- 172.21.15.112:0/4022986424 >> v2:172.21.15.138:3300/0 conn(0x7fcf0c1404c0 0x7fcf0c142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.735+0000 7fcf12219700 1 --2- 172.21.15.112:0/4022986424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c0cce30 0x7fcf0c140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.735+0000 7fcf12219700 1 --2- 172.21.15.112:0/4022986424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf0c0ad310 0x7fcf0c0cc8f0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.735+0000 7fcf12219700 1 -- 172.21.15.112:0/4022986424 >> 172.21.15.112:0/4022986424 conn(0x7fcf0c06cd30 msgr2=0x7fcf0c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:05.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.735+0000 7fcf12219700 1 -- 172.21.15.112:0/4022986424 shutdown_connections 2023-08-27T11:28:05.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.735+0000 7fcf12219700 1 -- 172.21.15.112:0/4022986424 wait complete. 2023-08-27T11:28:05.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.737+0000 7fcf12219700 1 Processor -- start 2023-08-27T11:28:05.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.737+0000 7fcf12219700 1 -- start start 2023-08-27T11:28:05.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf12219700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf0c0ad310 0x7fcf0c13edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf12219700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c0cce30 0x7fcf0c15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf12219700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf0c1404c0 0x7fcf0c15aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf12219700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c15aa50 con 0x7fcf0c1404c0 2023-08-27T11:28:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf12219700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c159e40 con 0x7fcf0c0ad310 2023-08-27T11:28:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf12219700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c15a120 con 0x7fcf0c0cce30 2023-08-27T11:28:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf0bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf0c1404c0 0x7fcf0c15aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf0bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf0c1404c0 0x7fcf0c15aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35764/0 (socket says 172.21.15.112:35764) 2023-08-27T11:28:05.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf0bfff700 1 -- 172.21.15.112:0/3824896044 learned_addr learned my addr 172.21.15.112:0/3824896044 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:05.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf0bfff700 1 -- 172.21.15.112:0/3824896044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c0cce30 msgr2=0x7fcf0c15d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf0bfff700 1 --2- 172.21.15.112:0/3824896044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c0cce30 0x7fcf0c15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf0b7fe700 1 --2- 172.21.15.112:0/3824896044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf0c0ad310 0x7fcf0c13edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf0bfff700 1 -- 172.21.15.112:0/3824896044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf0c0ad310 msgr2=0x7fcf0c13edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf0bfff700 1 --2- 172.21.15.112:0/3824896044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf0c0ad310 0x7fcf0c13edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.738+0000 7fcf0bfff700 1 -- 172.21.15.112:0/3824896044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc009500 con 0x7fcf0c1404c0 2023-08-27T11:28:05.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.739+0000 7fcf0bfff700 1 --2- 172.21.15.112:0/3824896044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf0c1404c0 0x7fcf0c15aee0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fcf04007db0 tx=0x7fcf0400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:05.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.739+0000 7fcef3fff700 1 -- 172.21.15.112:0/3824896044 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0400ddd0 con 0x7fcf0c1404c0 2023-08-27T11:28:05.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.739+0000 7fcef3fff700 1 -- 172.21.15.112:0/3824896044 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0400f040 con 0x7fcf0c1404c0 2023-08-27T11:28:05.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.739+0000 7fcf12219700 1 -- 172.21.15.112:0/3824896044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0c15a400 con 0x7fcf0c1404c0 2023-08-27T11:28:05.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.739+0000 7fcef3fff700 1 -- 172.21.15.112:0/3824896044 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf04003c60 con 0x7fcf0c1404c0 2023-08-27T11:28:05.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.739+0000 7fcf12219700 1 -- 172.21.15.112:0/3824896044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0c13f310 con 0x7fcf0c1404c0 2023-08-27T11:28:05.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.740+0000 7fcf12219700 1 -- 172.21.15.112:0/3824896044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0c15abc0 con 0x7fcf0c1404c0 2023-08-27T11:28:05.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.740+0000 7fcef3fff700 1 -- 172.21.15.112:0/3824896044 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fcf04004030 con 0x7fcf0c1404c0 2023-08-27T11:28:05.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.741+0000 7fcef3fff700 1 --2- 172.21.15.112:0/3824896044 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcef806a4d0 0x7fcef806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.741+0000 7fcef3fff700 1 -- 172.21.15.112:0/3824896044 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fcf04088c30 con 0x7fcf0c1404c0 2023-08-27T11:28:05.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.741+0000 7fcf0b7fe700 1 --2- 172.21.15.112:0/3824896044 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcef806a4d0 0x7fcef806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.741+0000 7fcf0b7fe700 1 --2- 172.21.15.112:0/3824896044 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcef806a4d0 0x7fcef806c990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009930 tx=0x7fcf00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:05.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:05.745+0000 7fcef3fff700 1 -- 172.21.15.112:0/3824896044 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcf04053e80 con 0x7fcf0c1404c0 2023-08-27T11:28:06.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.010+0000 7fcf12219700 1 -- 172.21.15.112:0/3824896044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]} v 0) v1 -- 0x7fcf0c0cc070 con 0x7fcf0c1404c0 2023-08-27T11:28:06.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.599+0000 7fcef3fff700 1 -- 172.21.15.112:0/3824896044 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]}]=0 v496) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcf040574a0 con 0x7fcf0c1404c0 2023-08-27T11:28:06.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.602+0000 7fcf12219700 1 -- 172.21.15.112:0/3824896044 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcef806a4d0 msgr2=0x7fcef806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.602+0000 7fcf12219700 1 --2- 172.21.15.112:0/3824896044 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcef806a4d0 0x7fcef806c990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009930 tx=0x7fcf00008040 comp rx=0 tx=0).stop 2023-08-27T11:28:06.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.602+0000 7fcf12219700 1 -- 172.21.15.112:0/3824896044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf0c1404c0 msgr2=0x7fcf0c15aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.602+0000 7fcf12219700 1 --2- 172.21.15.112:0/3824896044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf0c1404c0 0x7fcf0c15aee0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fcf04007db0 tx=0x7fcf0400d3b0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.602+0000 7fcf12219700 1 -- 172.21.15.112:0/3824896044 shutdown_connections 2023-08-27T11:28:06.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.602+0000 7fcf12219700 1 --2- 172.21.15.112:0/3824896044 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcef806a4d0 0x7fcef806c990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.602+0000 7fcf12219700 1 --2- 172.21.15.112:0/3824896044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf0c1404c0 0x7fcf0c15aee0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.602+0000 7fcf12219700 1 --2- 172.21.15.112:0/3824896044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf0c0ad310 0x7fcf0c13edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.602+0000 7fcf12219700 1 --2- 172.21.15.112:0/3824896044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c0cce30 0x7fcf0c15d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.602+0000 7fcf12219700 1 -- 172.21.15.112:0/3824896044 >> 172.21.15.112:0/3824896044 conn(0x7fcf0c06cd30 msgr2=0x7fcf0c144350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:06.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.602+0000 7fcf12219700 1 -- 172.21.15.112:0/3824896044 shutdown_connections 2023-08-27T11:28:06.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.602+0000 7fcf12219700 1 -- 172.21.15.112:0/3824896044 wait complete. 2023-08-27T11:28:06.618 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:06.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.742+0000 7f48360de700 1 Processor -- start 2023-08-27T11:28:06.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.743+0000 7f48360de700 1 -- start start 2023-08-27T11:28:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.743+0000 7f48360de700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f483006adb0 0x7f483006b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.743+0000 7f48360de700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48300cdbd0 0x7f48300ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.743+0000 7f48360de700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483015eb60 con 0x7f4830151060 2023-08-27T11:28:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.743+0000 7f48360de700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f483006b770 con 0x7f48300cdbd0 2023-08-27T11:28:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.743+0000 7f48360de700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48300b02f0 con 0x7f483006adb0 2023-08-27T11:28:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482f7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4830151060 0x7f483015e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58382/0 (socket says 172.21.15.112:58382) 2023-08-27T11:28:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482f7fe700 1 -- 172.21.15.112:0/4113302793 learned_addr learned my addr 172.21.15.112:0/4113302793 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482ffff700 1 --2- 172.21.15.112:0/4113302793 >> v2:172.21.15.112:3300/0 conn(0x7f48300cdbd0 0x7f48300ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482effd700 1 --2- 172.21.15.112:0/4113302793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f483006adb0 0x7f483006b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482ffff700 1 -- 172.21.15.112:0/4113302793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f483006adb0 msgr2=0x7f483006b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:06.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482ffff700 1 --2- 172.21.15.112:0/4113302793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f483006adb0 0x7f483006b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482ffff700 1 -- 172.21.15.112:0/4113302793 >> v1:172.21.15.112:6789/0 conn(0x7f4830151060 legacy=0x7f483015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482ffff700 1 -- 172.21.15.112:0/4113302793 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48300b05d0 con 0x7f48300cdbd0 2023-08-27T11:28:06.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482e7fc700 1 -- 172.21.15.112:0/4113302793 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1324007285 0 0) 0x7f483015eb60 con 0x7f4830151060 2023-08-27T11:28:06.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482effd700 1 --2- 172.21.15.112:0/4113302793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f483006adb0 0x7f483006b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482ffff700 1 --2- 172.21.15.112:0/4113302793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48300cdbd0 0x7f48300ce060 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f4828009d90 tx=0x7f482800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa72b3d997345e0d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482e7fc700 1 -- 172.21.15.112:0/4113302793 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482800e070 con 0x7f48300cdbd0 2023-08-27T11:28:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482e7fc700 1 -- 172.21.15.112:0/4113302793 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482800bbd0 con 0x7f48300cdbd0 2023-08-27T11:28:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.744+0000 7f482e7fc700 1 -- 172.21.15.112:0/4113302793 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4828012430 con 0x7f48300cdbd0 2023-08-27T11:28:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.745+0000 7f48360de700 1 -- 172.21.15.112:0/4113302793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48300cdbd0 msgr2=0x7f48300ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.745+0000 7f48360de700 1 --2- 172.21.15.112:0/4113302793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48300cdbd0 0x7f48300ce060 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f4828009d90 tx=0x7f482800b330 comp rx=0 tx=0).stop 2023-08-27T11:28:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.745+0000 7f48360de700 1 -- 172.21.15.112:0/4113302793 shutdown_connections 2023-08-27T11:28:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.745+0000 7f48360de700 1 --2- 172.21.15.112:0/4113302793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48300cdbd0 0x7f48300ce060 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.745+0000 7f48360de700 1 --2- 172.21.15.112:0/4113302793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f483006adb0 0x7f483006b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.745+0000 7f48360de700 1 -- 172.21.15.112:0/4113302793 >> 172.21.15.112:0/4113302793 conn(0x7f48300d0000 msgr2=0x7f48300d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.745+0000 7f48360de700 1 -- 172.21.15.112:0/4113302793 shutdown_connections 2023-08-27T11:28:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.745+0000 7f48360de700 1 -- 172.21.15.112:0/4113302793 wait complete. 2023-08-27T11:28:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.746+0000 7f48360de700 1 Processor -- start 2023-08-27T11:28:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.746+0000 7f48360de700 1 -- start start 2023-08-27T11:28:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f48360de700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483006adb0 0x7f4830153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f48360de700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48300cdbd0 0x7f48301503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f48360de700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4830151060 0x7f48300c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f48360de700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48301538c0 con 0x7f48300cdbd0 2023-08-27T11:28:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f48360de700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483015b7f0 con 0x7f483006adb0 2023-08-27T11:28:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f48360de700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4830150910 con 0x7f4830151060 2023-08-27T11:28:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f482effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48300cdbd0 0x7f48301503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f482effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48300cdbd0 0x7f48301503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35776/0 (socket says 172.21.15.112:35776) 2023-08-27T11:28:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f482effd700 1 -- 172.21.15.112:0/1983585771 learned_addr learned my addr 172.21.15.112:0/1983585771 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f482effd700 1 -- 172.21.15.112:0/1983585771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4830151060 msgr2=0x7f48300c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f482effd700 1 --2- 172.21.15.112:0/1983585771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4830151060 0x7f48300c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f482effd700 1 -- 172.21.15.112:0/1983585771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483006adb0 msgr2=0x7f4830153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f482effd700 1 --2- 172.21.15.112:0/1983585771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483006adb0 0x7f4830153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f482effd700 1 -- 172.21.15.112:0/1983585771 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4828009a70 con 0x7f48300cdbd0 2023-08-27T11:28:06.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f482effd700 1 --2- 172.21.15.112:0/1983585771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48300cdbd0 0x7f48301503d0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f481800c820 tx=0x7f481800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:06.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f4827fff700 1 -- 172.21.15.112:0/1983585771 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818011800 con 0x7f48300cdbd0 2023-08-27T11:28:06.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f4827fff700 1 -- 172.21.15.112:0/1983585771 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4818011960 con 0x7f48300cdbd0 2023-08-27T11:28:06.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.747+0000 7f48360de700 1 -- 172.21.15.112:0/1983585771 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4830153af0 con 0x7f48300cdbd0 2023-08-27T11:28:06.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.748+0000 7f4827fff700 1 -- 172.21.15.112:0/1983585771 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818011d10 con 0x7f48300cdbd0 2023-08-27T11:28:06.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.748+0000 7f48360de700 1 -- 172.21.15.112:0/1983585771 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48301540b0 con 0x7f48300cdbd0 2023-08-27T11:28:06.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.749+0000 7f48360de700 1 -- 172.21.15.112:0/1983585771 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4830067a20 con 0x7f48300cdbd0 2023-08-27T11:28:06.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.749+0000 7f4827fff700 1 -- 172.21.15.112:0/1983585771 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4818018360 con 0x7f48300cdbd0 2023-08-27T11:28:06.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.749+0000 7f4827fff700 1 --2- 172.21.15.112:0/1983585771 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f481006a4d0 0x7f481006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:06.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.749+0000 7f4827fff700 1 -- 172.21.15.112:0/1983585771 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f4818088b70 con 0x7f48300cdbd0 2023-08-27T11:28:06.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.750+0000 7f482f7fe700 1 --2- 172.21.15.112:0/1983585771 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f481006a4d0 0x7f481006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:06.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.750+0000 7f482f7fe700 1 --2- 172.21.15.112:0/1983585771 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f481006a4d0 0x7f481006c990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4820001610 tx=0x7f48200079f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:06.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.754+0000 7f4827fff700 1 -- 172.21.15.112:0/1983585771 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4818057860 con 0x7f48300cdbd0 2023-08-27T11:28:06.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.942+0000 7f48360de700 1 -- 172.21.15.112:0/1983585771 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48300c7610 con 0x7f48300cdbd0 2023-08-27T11:28:06.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.943+0000 7f4827fff700 1 -- 172.21.15.112:0/1983585771 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+15010 (secure 0 0 0) 0x7f48180573f0 con 0x7f48300cdbd0 2023-08-27T11:28:06.945 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:06.945 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":130,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:04.585571+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6835/3245198376":"2023-08-28T11:28:01.448008+0000","172.21.15.112:6832/3832579050":"2023-08-28T11:27:49.540827+0000","172.21.15.138:6837/3245198376":"2023-08-28T11:28:01.448008+0000","172.21.15.112:6834/3832579050":"2023-08-28T11:27:49.540827+0000","172.21.15.112:6835/4019485943":"2023-08-28T11:27:46.133510+0000","172.21.15.112:6833/4019485943":"2023-08-28T11:27:46.133510+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:06.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.946+0000 7f48360de700 1 -- 172.21.15.112:0/1983585771 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f481006a4d0 msgr2=0x7f481006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.946+0000 7f48360de700 1 --2- 172.21.15.112:0/1983585771 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f481006a4d0 0x7f481006c990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4820001610 tx=0x7f48200079f0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.946+0000 7f48360de700 1 -- 172.21.15.112:0/1983585771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48300cdbd0 msgr2=0x7f48301503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.946+0000 7f48360de700 1 --2- 172.21.15.112:0/1983585771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48300cdbd0 0x7f48301503d0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f481800c820 tx=0x7f481800cbe0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.946+0000 7f48360de700 1 -- 172.21.15.112:0/1983585771 shutdown_connections 2023-08-27T11:28:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.946+0000 7f48360de700 1 --2- 172.21.15.112:0/1983585771 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f481006a4d0 0x7f481006c990 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.946+0000 7f48360de700 1 --2- 172.21.15.112:0/1983585771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4830151060 0x7f48300c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.946+0000 7f48360de700 1 --2- 172.21.15.112:0/1983585771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483006adb0 0x7f4830153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.946+0000 7f48360de700 1 --2- 172.21.15.112:0/1983585771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48300cdbd0 0x7f48301503d0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.946+0000 7f48360de700 1 -- 172.21.15.112:0/1983585771 >> 172.21.15.112:0/1983585771 conn(0x7f48300d0000 msgr2=0x7f483006d480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:06.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.946+0000 7f48360de700 1 -- 172.21.15.112:0/1983585771 shutdown_connections 2023-08-27T11:28:06.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:06.946+0000 7f48360de700 1 -- 172.21.15.112:0/1983585771 wait complete. 2023-08-27T11:28:06.959 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:28:06.960 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:28:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.081+0000 7feadba7e700 1 Processor -- start 2023-08-27T11:28:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.082+0000 7feadba7e700 1 -- start start 2023-08-27T11:28:07.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.082+0000 7feadba7e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fead4143ba0 0x7fead40b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.082+0000 7feadba7e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead40cf410 con 0x7fead41437c0 2023-08-27T11:28:07.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.082+0000 7feadba7e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead40b09e0 con 0x7fead406dc90 2023-08-27T11:28:07.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.082+0000 7feadba7e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead40ceae0 con 0x7fead4143ba0 2023-08-27T11:28:07.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.083+0000 7fead981a700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fead41437c0 0x7fead406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58396/0 (socket says 172.21.15.112:58396) 2023-08-27T11:28:07.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.083+0000 7fead981a700 1 -- 172.21.15.112:0/89199654 learned_addr learned my addr 172.21.15.112:0/89199654 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:07.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.083+0000 7feada01b700 1 --2- 172.21.15.112:0/89199654 >> v2:172.21.15.138:3300/0 conn(0x7fead4143ba0 0x7fead40b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.083+0000 7fead8818700 1 -- 172.21.15.112:0/89199654 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200613119 0 0) 0x7fead40cf410 con 0x7fead41437c0 2023-08-27T11:28:07.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.083+0000 7fead8818700 1 -- 172.21.15.112:0/89199654 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feac00035b0 con 0x7fead41437c0 2023-08-27T11:28:07.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.083+0000 7fead8818700 1 -- 172.21.15.112:0/89199654 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 689244666 0 0) 0x7feac00035b0 con 0x7fead41437c0 2023-08-27T11:28:07.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.083+0000 7feada01b700 1 -- 172.21.15.112:0/89199654 >> v1:172.21.15.138:6789/0 conn(0x7fead406dc90 legacy=0x7fead40afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.083+0000 7feada01b700 1 -- 172.21.15.112:0/89199654 >> v1:172.21.15.112:6789/0 conn(0x7fead41437c0 legacy=0x7fead406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.083+0000 7feada01b700 1 -- 172.21.15.112:0/89199654 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead40cedc0 con 0x7fead4143ba0 2023-08-27T11:28:07.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.083+0000 7feada01b700 1 --2- 172.21.15.112:0/89199654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fead4143ba0 0x7fead40b03c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feacc009dc0 tx=0x7feacc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1f232cd60049b9f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:07.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.083+0000 7fead8818700 1 -- 172.21.15.112:0/89199654 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc00e070 con 0x7fead4143ba0 2023-08-27T11:28:07.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.083+0000 7fead8818700 1 -- 172.21.15.112:0/89199654 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feacc003a20 con 0x7fead4143ba0 2023-08-27T11:28:07.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.083+0000 7fead8818700 1 -- 172.21.15.112:0/89199654 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4001670 con 0x7fead4143ba0 2023-08-27T11:28:07.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.084+0000 7feadba7e700 1 -- 172.21.15.112:0/89199654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fead4143ba0 msgr2=0x7fead40b03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.084+0000 7feadba7e700 1 --2- 172.21.15.112:0/89199654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fead4143ba0 0x7fead40b03c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feacc009dc0 tx=0x7feacc00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:07.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.084+0000 7feadba7e700 1 -- 172.21.15.112:0/89199654 shutdown_connections 2023-08-27T11:28:07.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.084+0000 7feadba7e700 1 --2- 172.21.15.112:0/89199654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fead4143ba0 0x7fead40b03c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.084+0000 7feadba7e700 1 -- 172.21.15.112:0/89199654 >> 172.21.15.112:0/89199654 conn(0x7fead40b1e90 msgr2=0x7fead40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:07.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.084+0000 7feadba7e700 1 -- 172.21.15.112:0/89199654 shutdown_connections 2023-08-27T11:28:07.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.084+0000 7feadba7e700 1 -- 172.21.15.112:0/89199654 wait complete. 2023-08-27T11:28:07.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.085+0000 7feadba7e700 1 Processor -- start 2023-08-27T11:28:07.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.085+0000 7feadba7e700 1 -- start start 2023-08-27T11:28:07.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7feadba7e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fead406dc90 0x7fead4151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7feadba7e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fead41437c0 0x7fead4158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7feadba7e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fead4143ba0 0x7fead41594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7feadba7e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead41553b0 con 0x7fead406dc90 2023-08-27T11:28:07.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7feadba7e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead4155690 con 0x7fead4143ba0 2023-08-27T11:28:07.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7feadba7e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead4155970 con 0x7fead41437c0 2023-08-27T11:28:07.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7fead981a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fead406dc90 0x7fead4151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7fead981a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fead406dc90 0x7fead4151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35788/0 (socket says 172.21.15.112:35788) 2023-08-27T11:28:07.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7fead981a700 1 -- 172.21.15.112:0/3172877670 learned_addr learned my addr 172.21.15.112:0/3172877670 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:07.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7feada01b700 1 --2- 172.21.15.112:0/3172877670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fead4143ba0 0x7fead41594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7fead981a700 1 -- 172.21.15.112:0/3172877670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fead41437c0 msgr2=0x7fead4158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7fead981a700 1 --2- 172.21.15.112:0/3172877670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fead41437c0 0x7fead4158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7fead981a700 1 -- 172.21.15.112:0/3172877670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fead4143ba0 msgr2=0x7fead41594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7fead981a700 1 --2- 172.21.15.112:0/3172877670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fead4143ba0 0x7fead41594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7fead981a700 1 -- 172.21.15.112:0/3172877670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc009a70 con 0x7fead406dc90 2023-08-27T11:28:07.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7fead981a700 1 --2- 172.21.15.112:0/3172877670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fead406dc90 0x7fead4151880 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7feac400c750 tx=0x7feac400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:07.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.086+0000 7feaca7fc700 1 -- 172.21.15.112:0/3172877670 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4004c80 con 0x7fead406dc90 2023-08-27T11:28:07.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.087+0000 7feaca7fc700 1 -- 172.21.15.112:0/3172877670 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac4004e00 con 0x7fead406dc90 2023-08-27T11:28:07.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.087+0000 7feaca7fc700 1 -- 172.21.15.112:0/3172877670 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4007310 con 0x7fead406dc90 2023-08-27T11:28:07.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.087+0000 7feadba7e700 1 -- 172.21.15.112:0/3172877670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead4155c50 con 0x7fead406dc90 2023-08-27T11:28:07.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.087+0000 7feadba7e700 1 -- 172.21.15.112:0/3172877670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead4159a20 con 0x7fead406dc90 2023-08-27T11:28:07.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.087+0000 7feadba7e700 1 -- 172.21.15.112:0/3172877670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead41571e0 con 0x7fead406dc90 2023-08-27T11:28:07.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.088+0000 7feaca7fc700 1 -- 172.21.15.112:0/3172877670 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7feac4007470 con 0x7fead406dc90 2023-08-27T11:28:07.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.088+0000 7feaca7fc700 1 --2- 172.21.15.112:0/3172877670 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feab806a4d0 0x7feab806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.088+0000 7feaca7fc700 1 -- 172.21.15.112:0/3172877670 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5468+0+0 (secure 0 0 0) 0x7feac4087c70 con 0x7fead406dc90 2023-08-27T11:28:07.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.089+0000 7fead9019700 1 --2- 172.21.15.112:0/3172877670 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feab806a4d0 0x7feab806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.089+0000 7fead9019700 1 --2- 172.21.15.112:0/3172877670 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feab806a4d0 0x7feab806c990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fead0000f60 tx=0x7fead0007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:07.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.093+0000 7feaca7fc700 1 -- 172.21.15.112:0/3172877670 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fead41571e0 con 0x7fead406dc90 2023-08-27T11:28:07.233 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:28:07.238 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:28:07.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.285+0000 7feadba7e700 1 -- 172.21.15.112:0/3172877670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fead41575c0 con 0x7fead406dc90 2023-08-27T11:28:07.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.285+0000 7feaca7fc700 1 -- 172.21.15.112:0/3172877670 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v130) v1 ==== 92+0+10 (secure 0 0 0) 0x7fead41575c0 con 0x7fead406dc90 2023-08-27T11:28:07.287 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:28:07.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.287+0000 7feadba7e700 1 -- 172.21.15.112:0/3172877670 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feab806a4d0 msgr2=0x7feab806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.287+0000 7feadba7e700 1 --2- 172.21.15.112:0/3172877670 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feab806a4d0 0x7feab806c990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fead0000f60 tx=0x7fead0007a40 comp rx=0 tx=0).stop 2023-08-27T11:28:07.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.287+0000 7feadba7e700 1 -- 172.21.15.112:0/3172877670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fead406dc90 msgr2=0x7fead4151880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.287+0000 7feadba7e700 1 --2- 172.21.15.112:0/3172877670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fead406dc90 0x7fead4151880 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7feac400c750 tx=0x7feac400cb10 comp rx=0 tx=0).stop 2023-08-27T11:28:07.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.287+0000 7feadba7e700 1 -- 172.21.15.112:0/3172877670 shutdown_connections 2023-08-27T11:28:07.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.287+0000 7feadba7e700 1 --2- 172.21.15.112:0/3172877670 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feab806a4d0 0x7feab806c990 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.287+0000 7feadba7e700 1 --2- 172.21.15.112:0/3172877670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fead4143ba0 0x7fead41594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.287+0000 7feadba7e700 1 --2- 172.21.15.112:0/3172877670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fead406dc90 0x7fead4151880 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.287+0000 7feadba7e700 1 --2- 172.21.15.112:0/3172877670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fead41437c0 0x7fead4158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.287+0000 7feadba7e700 1 -- 172.21.15.112:0/3172877670 >> 172.21.15.112:0/3172877670 conn(0x7fead40b1e90 msgr2=0x7fead40b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:07.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.287+0000 7feadba7e700 1 -- 172.21.15.112:0/3172877670 shutdown_connections 2023-08-27T11:28:07.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.287+0000 7feadba7e700 1 -- 172.21.15.112:0/3172877670 wait complete. 2023-08-27T11:28:07.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:28:07.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.463+0000 7f2c8cbd9700 1 Processor -- start 2023-08-27T11:28:07.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.463+0000 7f2c8cbd9700 1 -- start start 2023-08-27T11:28:07.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c8cbd9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c881437c0 0x7f2c880b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c8cbd9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2c8806d5b0 0x7f2c8806da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c8cbd9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c880cfd50 con 0x7f2c88143ba0 2023-08-27T11:28:07.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c8cbd9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8806df40 con 0x7f2c8806d5b0 2023-08-27T11:28:07.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c8cbd9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880cf140 con 0x7f2c881437c0 2023-08-27T11:28:07.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c86d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2c8806d5b0 0x7f2c8806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c85d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2c88143ba0 0x7f2c880ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58412/0 (socket says 172.21.15.112:58412) 2023-08-27T11:28:07.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c85d9b700 1 -- 172.21.15.112:0/3288458916 learned_addr learned my addr 172.21.15.112:0/3288458916 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:07.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c8659c700 1 --2- 172.21.15.112:0/3288458916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c881437c0 0x7f2c880b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c86d9d700 1 -- 172.21.15.112:0/3288458916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c881437c0 msgr2=0x7f2c880b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:07.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c86d9d700 1 --2- 172.21.15.112:0/3288458916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c881437c0 0x7f2c880b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c86d9d700 1 -- 172.21.15.112:0/3288458916 >> v1:172.21.15.112:6789/0 conn(0x7f2c88143ba0 legacy=0x7f2c880ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c86d9d700 1 -- 172.21.15.112:0/3288458916 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c880cf420 con 0x7f2c8806d5b0 2023-08-27T11:28:07.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c8559a700 1 -- 172.21.15.112:0/3288458916 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2517130433 0 0) 0x7f2c880cfd50 con 0x7f2c88143ba0 2023-08-27T11:28:07.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c8659c700 1 --2- 172.21.15.112:0/3288458916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c881437c0 0x7f2c880b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:07.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.464+0000 7f2c86d9d700 1 --2- 172.21.15.112:0/3288458916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8806d5b0 0x7f2c8806da00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009d90 tx=0x7f2c8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e99d7ad71d777e0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:07.470 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.465+0000 7f2c8559a700 1 -- 172.21.15.112:0/3288458916 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8000e070 con 0x7f2c8806d5b0 2023-08-27T11:28:07.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.465+0000 7f2c8559a700 1 -- 172.21.15.112:0/3288458916 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8000bbd0 con 0x7f2c8806d5b0 2023-08-27T11:28:07.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.465+0000 7f2c8559a700 1 -- 172.21.15.112:0/3288458916 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80012430 con 0x7f2c8806d5b0 2023-08-27T11:28:07.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.465+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/3288458916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8806d5b0 msgr2=0x7f2c8806da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.465+0000 7f2c8cbd9700 1 --2- 172.21.15.112:0/3288458916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8806d5b0 0x7f2c8806da00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009d90 tx=0x7f2c8000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:07.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.465+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/3288458916 shutdown_connections 2023-08-27T11:28:07.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.465+0000 7f2c8cbd9700 1 --2- 172.21.15.112:0/3288458916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8806d5b0 0x7f2c8806da00 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.465+0000 7f2c8cbd9700 1 --2- 172.21.15.112:0/3288458916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c881437c0 0x7f2c880b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.465+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/3288458916 >> 172.21.15.112:0/3288458916 conn(0x7f2c880b1e90 msgr2=0x7f2c880b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:07.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.465+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/3288458916 shutdown_connections 2023-08-27T11:28:07.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.465+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/3288458916 wait complete. 2023-08-27T11:28:07.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.467+0000 7f2c8cbd9700 1 Processor -- start 2023-08-27T11:28:07.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.467+0000 7f2c8cbd9700 1 -- start start 2023-08-27T11:28:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.467+0000 7f2c8cbd9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c8806d5b0 0x7f2c880c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.467+0000 7f2c8cbd9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c881437c0 0x7f2c8813a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.467+0000 7f2c8cbd9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c88143ba0 0x7f2c880b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.467+0000 7f2c8cbd9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880c5900 con 0x7f2c88143ba0 2023-08-27T11:28:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.467+0000 7f2c8cbd9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88071fc0 con 0x7f2c881437c0 2023-08-27T11:28:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.467+0000 7f2c8cbd9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8813c630 con 0x7f2c8806d5b0 2023-08-27T11:28:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c86d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c88143ba0 0x7f2c880b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c86d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c88143ba0 0x7f2c880b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35806/0 (socket says 172.21.15.112:35806) 2023-08-27T11:28:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c86d9d700 1 -- 172.21.15.112:0/625703679 learned_addr learned my addr 172.21.15.112:0/625703679 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c8659c700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c8806d5b0 0x7f2c880c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c86d9d700 1 -- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c8806d5b0 msgr2=0x7f2c880c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c86d9d700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c8806d5b0 0x7f2c880c50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c86d9d700 1 -- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c881437c0 msgr2=0x7f2c8813a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c85d9b700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c881437c0 0x7f2c8813a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c86d9d700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c881437c0 0x7f2c8813a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c86d9d700 1 -- 172.21.15.112:0/625703679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c80009a70 con 0x7f2c88143ba0 2023-08-27T11:28:07.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c86d9d700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c88143ba0 0x7f2c880b4be0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f2c8806e400 tx=0x7f2c80004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:07.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c7effd700 1 -- 172.21.15.112:0/625703679 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80012430 con 0x7f2c88143ba0 2023-08-27T11:28:07.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c7effd700 1 -- 172.21.15.112:0/625703679 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c80020070 con 0x7f2c88143ba0 2023-08-27T11:28:07.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/625703679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c880c5630 con 0x7f2c88143ba0 2023-08-27T11:28:07.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c7effd700 1 -- 172.21.15.112:0/625703679 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80004280 con 0x7f2c88143ba0 2023-08-27T11:28:07.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.468+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/625703679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c880724b0 con 0x7f2c88143ba0 2023-08-27T11:28:07.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.469+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/625703679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8813c7e0 con 0x7f2c88143ba0 2023-08-27T11:28:07.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.470+0000 7f2c7effd700 1 -- 172.21.15.112:0/625703679 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f2c800043e0 con 0x7f2c88143ba0 2023-08-27T11:28:07.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.470+0000 7f2c7effd700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c6806a4d0 0x7f2c6806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.470+0000 7f2c7effd700 1 -- 172.21.15.112:0/625703679 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f2c80022070 con 0x7f2c88143ba0 2023-08-27T11:28:07.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.470+0000 7f2c8659c700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c6806a4d0 0x7f2c6806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.471+0000 7f2c8659c700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c6806a4d0 0x7f2c6806c990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009930 tx=0x7f2c70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:07.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.474+0000 7f2c7effd700 1 -- 172.21.15.112:0/625703679 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2c8813c7e0 con 0x7f2c88143ba0 2023-08-27T11:28:07.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.664+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/625703679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c88067e90 con 0x7f2c88143ba0 2023-08-27T11:28:07.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.664+0000 7f2c7effd700 1 -- 172.21.15.112:0/625703679 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c80056390 con 0x7f2c88143ba0 2023-08-27T11:28:07.666 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:28:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.666+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c6806a4d0 msgr2=0x7f2c6806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.666+0000 7f2c8cbd9700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c6806a4d0 0x7f2c6806c990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009930 tx=0x7f2c70008040 comp rx=0 tx=0).stop 2023-08-27T11:28:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.666+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/625703679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c88143ba0 msgr2=0x7f2c880b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.666+0000 7f2c8cbd9700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c88143ba0 0x7f2c880b4be0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f2c8806e400 tx=0x7f2c80004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.666+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/625703679 shutdown_connections 2023-08-27T11:28:07.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.666+0000 7f2c8cbd9700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2c6806a4d0 0x7f2c6806c990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.666+0000 7f2c8cbd9700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c88143ba0 0x7f2c880b4be0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.666+0000 7f2c8cbd9700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c8806d5b0 0x7f2c880c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.666+0000 7f2c8cbd9700 1 --2- 172.21.15.112:0/625703679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c881437c0 0x7f2c8813a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.666+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/625703679 >> 172.21.15.112:0/625703679 conn(0x7f2c880b1e90 msgr2=0x7f2c881598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.667+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/625703679 shutdown_connections 2023-08-27T11:28:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.667+0000 7f2c8cbd9700 1 -- 172.21.15.112:0/625703679 wait complete. 2023-08-27T11:28:07.680 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:28:07.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.802+0000 7f5de9d4b700 1 Processor -- start 2023-08-27T11:28:07.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.802+0000 7f5de9d4b700 1 -- start start 2023-08-27T11:28:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.803+0000 7f5de9d4b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de4144900 0x7f5de40b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.803+0000 7f5de9d4b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de415fd60 con 0x7f5de40cca90 2023-08-27T11:28:07.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.803+0000 7f5de9d4b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de415ee90 con 0x7f5de406f790 2023-08-27T11:28:07.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.803+0000 7f5de9d4b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de40b11d0 con 0x7f5de4144900 2023-08-27T11:28:07.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.803+0000 7f5de2ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5de40cca90 0x7f5de415f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58426/0 (socket says 172.21.15.112:58426) 2023-08-27T11:28:07.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.803+0000 7f5de2ffd700 1 -- 172.21.15.112:0/4002946247 learned_addr learned my addr 172.21.15.112:0/4002946247 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:07.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.803+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 674623483 0 0) 0x7f5de415fd60 con 0x7f5de40cca90 2023-08-27T11:28:07.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.803+0000 7f5de37fe700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de4144900 0x7f5de40b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.803+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dd00035b0 con 0x7f5de40cca90 2023-08-27T11:28:07.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.803+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3333520992 0 0) 0x7f5dd00035b0 con 0x7f5de40cca90 2023-08-27T11:28:07.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.803+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de4144900 msgr2=0x7f5de40b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.803+0000 7f5de27fc700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de4144900 0x7f5de40b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.803+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 >> v1:172.21.15.138:6789/0 conn(0x7f5de406f790 legacy=0x7f5de415b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de40b14b0 con 0x7f5de40cca90 2023-08-27T11:28:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f5dd4001690 con 0x7f5de40cca90 2023-08-27T11:28:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 >> v1:172.21.15.112:6789/0 conn(0x7f5de40cca90 legacy=0x7f5de415f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de27fc700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd0003a30 0x7f5dd0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de27fc700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de4144900 0x7f5dd0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de27fc700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd0004d90 0x7f5dd0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de40b11d0 con 0x7f5dd0003a30 2023-08-27T11:28:07.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd000bac0 con 0x7f5dd0004d90 2023-08-27T11:28:07.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd000bc80 con 0x7f5de4144900 2023-08-27T11:28:07.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de37fe700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd0003a30 0x7f5dd0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de2ffd700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de4144900 0x7f5dd0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de3fff700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd0004d90 0x7f5dd0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de37fe700 1 -- 172.21.15.112:0/4002946247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de4144900 msgr2=0x7f5dd0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:07.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de37fe700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de4144900 0x7f5dd0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de37fe700 1 -- 172.21.15.112:0/4002946247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd0004d90 msgr2=0x7f5dd0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:07.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de37fe700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd0004d90 0x7f5dd0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de37fe700 1 -- 172.21.15.112:0/4002946247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd00035b0 con 0x7f5dd0003a30 2023-08-27T11:28:07.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de2ffd700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de4144900 0x7f5dd0004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:07.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de3fff700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd0004d90 0x7f5dd0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de37fe700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd0003a30 0x7f5dd0003e50 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009b30 tx=0x7f5dd0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b32ce275c3038a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc00e6c0 con 0x7f5dd0003a30 2023-08-27T11:28:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc00e820 con 0x7f5dd0003a30 2023-08-27T11:28:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.804+0000 7f5de27fc700 1 -- 172.21.15.112:0/4002946247 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc00eb90 con 0x7f5dd0003a30 2023-08-27T11:28:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.805+0000 7f5de9d4b700 1 -- 172.21.15.112:0/4002946247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd0003a30 msgr2=0x7f5dd0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.805+0000 7f5de9d4b700 1 --2- 172.21.15.112:0/4002946247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd0003a30 0x7f5dd0003e50 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009b30 tx=0x7f5dd0005880 comp rx=0 tx=0).stop 2023-08-27T11:28:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.805+0000 7f5de3fff700 1 -- 172.21.15.112:0/4002946247 reap_dead start 2023-08-27T11:28:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.805+0000 7f5de9d4b700 1 -- 172.21.15.112:0/4002946247 shutdown_connections 2023-08-27T11:28:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.805+0000 7f5de9d4b700 1 -- 172.21.15.112:0/4002946247 >> 172.21.15.112:0/4002946247 conn(0x7f5de40b1e90 msgr2=0x7f5de40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.805+0000 7f5de9d4b700 1 -- 172.21.15.112:0/4002946247 shutdown_connections 2023-08-27T11:28:07.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.805+0000 7f5de9d4b700 1 -- 172.21.15.112:0/4002946247 wait complete. 2023-08-27T11:28:07.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.806+0000 7f5de9d4b700 1 Processor -- start 2023-08-27T11:28:07.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.806+0000 7f5de9d4b700 1 -- start start 2023-08-27T11:28:07.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de9d4b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd0003a30 0x7f5de41f10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de9d4b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40adeb0 0x7f5de413ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de9d4b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de406d4b0 0x7f5de406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de9d4b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de41f1890 con 0x7f5de40adeb0 2023-08-27T11:28:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de9d4b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de406de80 con 0x7f5dd0003a30 2023-08-27T11:28:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de9d4b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de40ae670 con 0x7f5de406d4b0 2023-08-27T11:28:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40adeb0 0x7f5de413ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40adeb0 0x7f5de413ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35822/0 (socket says 172.21.15.112:35822) 2023-08-27T11:28:07.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de2ffd700 1 -- 172.21.15.112:0/3444883546 learned_addr learned my addr 172.21.15.112:0/3444883546 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:07.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de3fff700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de406d4b0 0x7f5de406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de37fe700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd0003a30 0x7f5de41f10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de2ffd700 1 -- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de406d4b0 msgr2=0x7f5de406d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de2ffd700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de406d4b0 0x7f5de406d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de2ffd700 1 -- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd0003a30 msgr2=0x7f5de41f10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de2ffd700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd0003a30 0x7f5de41f10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de2ffd700 1 -- 172.21.15.112:0/3444883546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de40ae950 con 0x7f5de40adeb0 2023-08-27T11:28:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de37fe700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd0003a30 0x7f5de41f10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de3fff700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de406d4b0 0x7f5de406d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.807+0000 7f5de2ffd700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40adeb0 0x7f5de413ea20 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400e8c0 tx=0x7f5dd400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.808+0000 7f5ddbfff700 1 -- 172.21.15.112:0/3444883546 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400c7e0 con 0x7f5de40adeb0 2023-08-27T11:28:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.808+0000 7f5ddbfff700 1 -- 172.21.15.112:0/3444883546 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd400c940 con 0x7f5de40adeb0 2023-08-27T11:28:07.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.808+0000 7f5de9d4b700 1 -- 172.21.15.112:0/3444883546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de40aec30 con 0x7f5de40adeb0 2023-08-27T11:28:07.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.808+0000 7f5ddbfff700 1 -- 172.21.15.112:0/3444883546 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400ccd0 con 0x7f5de40adeb0 2023-08-27T11:28:07.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.808+0000 7f5de9d4b700 1 -- 172.21.15.112:0/3444883546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de40ae330 con 0x7f5de40adeb0 2023-08-27T11:28:07.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.809+0000 7f5de9d4b700 1 -- 172.21.15.112:0/3444883546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de4067e90 con 0x7f5de40adeb0 2023-08-27T11:28:07.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.809+0000 7f5ddbfff700 1 -- 172.21.15.112:0/3444883546 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5dd401d460 con 0x7f5de40adeb0 2023-08-27T11:28:07.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.809+0000 7f5ddbfff700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5dc406a4d0 0x7f5dc406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.810+0000 7f5ddbfff700 1 -- 172.21.15.112:0/3444883546 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f5dd4088840 con 0x7f5de40adeb0 2023-08-27T11:28:07.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.810+0000 7f5de37fe700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5dc406a4d0 0x7f5dc406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.810+0000 7f5de37fe700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5dc406a4d0 0x7f5dc406c990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0090b0 tx=0x7f5dcc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:07.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:07.814+0000 7f5ddbfff700 1 -- 172.21.15.112:0/3444883546 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5dd4053c30 con 0x7f5de40adeb0 2023-08-27T11:28:08.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.001+0000 7f5de9d4b700 1 -- 172.21.15.112:0/3444883546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5de4067a20 con 0x7f5de40adeb0 2023-08-27T11:28:08.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.005+0000 7f5ddbfff700 1 -- 172.21.15.112:0/3444883546 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5dd4057250 con 0x7f5de40adeb0 2023-08-27T11:28:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.007+0000 7f5de9d4b700 1 -- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5dc406a4d0 msgr2=0x7f5dc406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.007+0000 7f5de9d4b700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5dc406a4d0 0x7f5dc406c990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0090b0 tx=0x7f5dcc008030 comp rx=0 tx=0).stop 2023-08-27T11:28:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.007+0000 7f5de9d4b700 1 -- 172.21.15.112:0/3444883546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40adeb0 msgr2=0x7f5de413ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.007+0000 7f5de9d4b700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40adeb0 0x7f5de413ea20 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400e8c0 tx=0x7f5dd400ec80 comp rx=0 tx=0).stop 2023-08-27T11:28:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.007+0000 7f5de9d4b700 1 -- 172.21.15.112:0/3444883546 shutdown_connections 2023-08-27T11:28:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.007+0000 7f5de9d4b700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5dc406a4d0 0x7f5dc406c990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.007+0000 7f5de9d4b700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5de406d4b0 0x7f5de406d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.007+0000 7f5de9d4b700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd0003a30 0x7f5de41f10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.007+0000 7f5de9d4b700 1 --2- 172.21.15.112:0/3444883546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40adeb0 0x7f5de413ea20 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.007+0000 7f5de9d4b700 1 -- 172.21.15.112:0/3444883546 >> 172.21.15.112:0/3444883546 conn(0x7f5de40b1e90 msgr2=0x7f5de415b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.007+0000 7f5de9d4b700 1 -- 172.21.15.112:0/3444883546 shutdown_connections 2023-08-27T11:28:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.007+0000 7f5de9d4b700 1 -- 172.21.15.112:0/3444883546 wait complete. 2023-08-27T11:28:08.020 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:28:08.021 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:28:08.062 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:28:08.108 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:28:08.109 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:28:08.109 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:08.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:28:08.128 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:28:08.128 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:08.148 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:28:08.149 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:28:08.149 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:08.184 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:28:08.184 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:28:08.185 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:08.207 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:28:08.207 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:28:08.208 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:08.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.343+0000 7f877cc4f700 1 Processor -- start 2023-08-27T11:28:08.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.343+0000 7f877cc4f700 1 -- start start 2023-08-27T11:28:08.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.343+0000 7f877cc4f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87780b1400 0x7f87780cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.343+0000 7f877cc4f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f87780b17e0 0x7f877806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f877cc4f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8778161ce0 0x7f877806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f877cc4f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8778161850 con 0x7f8778161ce0 2023-08-27T11:28:08.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f877cc4f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877806e7d0 con 0x7f87780b17e0 2023-08-27T11:28:08.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f877cc4f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87780cfc10 con 0x7f87780b1400 2023-08-27T11:28:08.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f8776d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8778161ce0 0x7f877806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f8776d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8778161ce0 0x7f877806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35834/0 (socket says 172.21.15.112:35834) 2023-08-27T11:28:08.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f8776d9d700 1 -- 172.21.15.112:0/3868933678 learned_addr learned my addr 172.21.15.112:0/3868933678 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:08.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f877659c700 1 --2- 172.21.15.112:0/3868933678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87780b1400 0x7f87780cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f8775d9b700 1 --2- 172.21.15.112:0/3868933678 >> v2:172.21.15.138:3300/0 conn(0x7f87780b17e0 0x7f877806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f8776d9d700 1 -- 172.21.15.112:0/3868933678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87780b1400 msgr2=0x7f87780cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:08.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f8776d9d700 1 --2- 172.21.15.112:0/3868933678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87780b1400 0x7f87780cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f8776d9d700 1 -- 172.21.15.112:0/3868933678 >> v2:172.21.15.138:3300/0 conn(0x7f87780b17e0 msgr2=0x7f877806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:08.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f8776d9d700 1 --2- 172.21.15.112:0/3868933678 >> v2:172.21.15.138:3300/0 conn(0x7f87780b17e0 0x7f877806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f8776d9d700 1 -- 172.21.15.112:0/3868933678 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8778161200 con 0x7f8778161ce0 2023-08-27T11:28:08.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f8776d9d700 1 --2- 172.21.15.112:0/3868933678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8778161ce0 0x7f877806f790 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f87780d0db0 tx=0x7f877000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314e0b613062a2e9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:08.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f877559a700 1 -- 172.21.15.112:0/3868933678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877000e070 con 0x7f8778161ce0 2023-08-27T11:28:08.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f877559a700 1 -- 172.21.15.112:0/3868933678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877000b9c0 con 0x7f8778161ce0 2023-08-27T11:28:08.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.344+0000 7f877559a700 1 -- 172.21.15.112:0/3868933678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877000bd70 con 0x7f8778161ce0 2023-08-27T11:28:08.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.345+0000 7f877cc4f700 1 -- 172.21.15.112:0/3868933678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8778161ce0 msgr2=0x7f877806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.345+0000 7f877cc4f700 1 --2- 172.21.15.112:0/3868933678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8778161ce0 0x7f877806f790 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f87780d0db0 tx=0x7f877000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:08.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.345+0000 7f877cc4f700 1 -- 172.21.15.112:0/3868933678 shutdown_connections 2023-08-27T11:28:08.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.345+0000 7f877cc4f700 1 --2- 172.21.15.112:0/3868933678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8778161ce0 0x7f877806f790 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.345+0000 7f877cc4f700 1 --2- 172.21.15.112:0/3868933678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87780b1400 0x7f87780cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.345+0000 7f877cc4f700 1 --2- 172.21.15.112:0/3868933678 >> v2:172.21.15.138:3300/0 conn(0x7f87780b17e0 0x7f877806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.345+0000 7f877cc4f700 1 -- 172.21.15.112:0/3868933678 >> 172.21.15.112:0/3868933678 conn(0x7f87780d0000 msgr2=0x7f87780d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:08.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.345+0000 7f877cc4f700 1 -- 172.21.15.112:0/3868933678 shutdown_connections 2023-08-27T11:28:08.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.345+0000 7f877cc4f700 1 -- 172.21.15.112:0/3868933678 wait complete. 2023-08-27T11:28:08.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.347+0000 7f877cc4f700 1 Processor -- start 2023-08-27T11:28:08.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.347+0000 7f877cc4f700 1 -- start start 2023-08-27T11:28:08.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f877cc4f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87780b1400 0x7f87780c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f877cc4f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87780b17e0 0x7f87780c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f877cc4f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8778161ce0 0x7f8778143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f877cc4f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8778148080 con 0x7f87780b17e0 2023-08-27T11:28:08.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f877cc4f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87780c7de0 con 0x7f87780b1400 2023-08-27T11:28:08.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f877cc4f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87780c7b00 con 0x7f8778161ce0 2023-08-27T11:28:08.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8775d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87780b17e0 0x7f87780c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8775d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87780b17e0 0x7f87780c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35838/0 (socket says 172.21.15.112:35838) 2023-08-27T11:28:08.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8775d9b700 1 -- 172.21.15.112:0/3574386289 learned_addr learned my addr 172.21.15.112:0/3574386289 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:08.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f877659c700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87780b1400 0x7f87780c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8776d9d700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8778161ce0 0x7f8778143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8775d9b700 1 -- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8778161ce0 msgr2=0x7f8778143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8775d9b700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8778161ce0 0x7f8778143f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8775d9b700 1 -- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87780b1400 msgr2=0x7f87780c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8775d9b700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87780b1400 0x7f87780c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8775d9b700 1 -- 172.21.15.112:0/3574386289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8770009b30 con 0x7f87780b17e0 2023-08-27T11:28:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8776d9d700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8778161ce0 0x7f8778143f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8775d9b700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87780b17e0 0x7f87780c70d0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f876c00e810 tx=0x7f876c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8766ffd700 1 -- 172.21.15.112:0/3574386289 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c013610 con 0x7f87780b17e0 2023-08-27T11:28:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8766ffd700 1 -- 172.21.15.112:0/3574386289 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c013770 con 0x7f87780b17e0 2023-08-27T11:28:08.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f877cc4f700 1 -- 172.21.15.112:0/3574386289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8778144d30 con 0x7f87780b17e0 2023-08-27T11:28:08.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f8766ffd700 1 -- 172.21.15.112:0/3574386289 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c013b20 con 0x7f87780b17e0 2023-08-27T11:28:08.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.348+0000 7f877cc4f700 1 -- 172.21.15.112:0/3574386289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87781452f0 con 0x7f87780b17e0 2023-08-27T11:28:08.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.349+0000 7f877cc4f700 1 -- 172.21.15.112:0/3574386289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8778144b90 con 0x7f87780b17e0 2023-08-27T11:28:08.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.350+0000 7f8766ffd700 1 -- 172.21.15.112:0/3574386289 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f876c013c80 con 0x7f87780b17e0 2023-08-27T11:28:08.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.350+0000 7f8766ffd700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f875806a4d0 0x7f875806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.350+0000 7f8766ffd700 1 -- 172.21.15.112:0/3574386289 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f876c088770 con 0x7f87780b17e0 2023-08-27T11:28:08.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.350+0000 7f877659c700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f875806a4d0 0x7f875806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.351+0000 7f877659c700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f875806a4d0 0x7f875806c990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f87781f5ee0 tx=0x7f8768008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.355+0000 7f8766ffd700 1 -- 172.21.15.112:0/3574386289 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8778144b90 con 0x7f87780b17e0 2023-08-27T11:28:08.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.586+0000 7f877cc4f700 1 -- 172.21.15.112:0/3574386289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8778144b90 con 0x7f87780b17e0 2023-08-27T11:28:08.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.586+0000 7f8766ffd700 1 -- 172.21.15.112:0/3574386289 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f876c021e90 con 0x7f87780b17e0 2023-08-27T11:28:08.588 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:08.588 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":88,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1795721853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1795721853},{"type":"v1","addr":"172.21.15.138:6836","nonce":1795721853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":7486,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2533379552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2533379552},{"type":"v1","addr":"172.21.15.138:6837","nonce":2533379552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":8184,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2658811115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2658811115},{"type":"v1","addr":"172.21.15.112:6835","nonce":2658811115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":8199,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3536640393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3536640393},{"type":"v1","addr":"172.21.15.112:6834","nonce":3536640393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[]} 2023-08-27T11:28:08.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.588+0000 7f877cc4f700 1 -- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f875806a4d0 msgr2=0x7f875806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.588+0000 7f877cc4f700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f875806a4d0 0x7f875806c990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f87781f5ee0 tx=0x7f8768008040 comp rx=0 tx=0).stop 2023-08-27T11:28:08.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.588+0000 7f877cc4f700 1 -- 172.21.15.112:0/3574386289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87780b17e0 msgr2=0x7f87780c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.588+0000 7f877cc4f700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87780b17e0 0x7f87780c70d0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f876c00e810 tx=0x7f876c00eb20 comp rx=0 tx=0).stop 2023-08-27T11:28:08.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.588+0000 7f877cc4f700 1 -- 172.21.15.112:0/3574386289 shutdown_connections 2023-08-27T11:28:08.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.588+0000 7f877cc4f700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f875806a4d0 0x7f875806c990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.588+0000 7f877cc4f700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8778161ce0 0x7f8778143f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.588+0000 7f877cc4f700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87780b1400 0x7f87780c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.588+0000 7f877cc4f700 1 --2- 172.21.15.112:0/3574386289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87780b17e0 0x7f87780c70d0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.588+0000 7f877cc4f700 1 -- 172.21.15.112:0/3574386289 >> 172.21.15.112:0/3574386289 conn(0x7f87780d0000 msgr2=0x7f87781589f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:08.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.588+0000 7f877cc4f700 1 -- 172.21.15.112:0/3574386289 shutdown_connections 2023-08-27T11:28:08.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.588+0000 7f877cc4f700 1 -- 172.21.15.112:0/3574386289 wait complete. 2023-08-27T11:28:08.592 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 88 2023-08-27T11:28:08.602 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:28:08.602 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:28:08.602 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:28:08.603 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:08.603 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:28:08.603 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:28:08.604 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:28:08.604 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:08.605 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:28:08.605 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:28:08.605 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:28:08.606 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:08.606 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:28:08.606 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:28:08.607 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:28:08.607 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:08.611 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:28:08.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:28:08.613 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:28:08.613 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:28:08.615 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:28:08.616 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:28:08.617 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:28:08.617 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:28:08.655 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:28:08.656 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:08.657 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:28:08.657 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:08.659 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:28:08.659 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:08.661 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:28:08.661 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:08.662 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:28:08.734 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:28:08.735 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:28:08.742 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:28:08.743 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2023-08-27T11:28:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.788+0000 7f504e113700 1 Processor -- start 2023-08-27T11:28:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.788+0000 7f504e113700 1 -- start start 2023-08-27T11:28:08.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.789+0000 7f504e113700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f50480cf170 0x7f50480b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.789+0000 7f504e113700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50480cfb40 con 0x7f50481578e0 2023-08-27T11:28:08.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.789+0000 7f504e113700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50480cfde0 con 0x7f5048160f90 2023-08-27T11:28:08.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.789+0000 7f504e113700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50480d00c0 con 0x7f50480cf170 2023-08-27T11:28:08.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.789+0000 7f5046ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f50480cf170 0x7f50480b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.789+0000 7f50477fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f50481578e0 0x7f50480d08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58480/0 (socket says 172.21.15.112:58480) 2023-08-27T11:28:08.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.789+0000 7f50477fe700 1 -- 172.21.15.112:0/4259740779 learned_addr learned my addr 172.21.15.112:0/4259740779 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.789+0000 7f5046ffd700 1 -- 172.21.15.112:0/4259740779 >> v1:172.21.15.138:6789/0 conn(0x7f5048160f90 legacy=0x7f50480708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.789+0000 7f5046ffd700 1 -- 172.21.15.112:0/4259740779 >> v1:172.21.15.112:6789/0 conn(0x7f50481578e0 legacy=0x7f50480d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.789+0000 7f5046ffd700 1 -- 172.21.15.112:0/4259740779 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50480d03a0 con 0x7f50480cf170 2023-08-27T11:28:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.789+0000 7f5046ffd700 1 --2- 172.21.15.112:0/4259740779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50480cf170 0x7f50480b0710 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f5038009840 tx=0x7f5038009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65c8eb573cad8cd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.789+0000 7f50467fc700 1 -- 172.21.15.112:0/4259740779 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038004230 con 0x7f50480cf170 2023-08-27T11:28:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.790+0000 7f50467fc700 1 -- 172.21.15.112:0/4259740779 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5038004390 con 0x7f50480cf170 2023-08-27T11:28:08.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.790+0000 7f50467fc700 1 -- 172.21.15.112:0/4259740779 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50380036a0 con 0x7f50480cf170 2023-08-27T11:28:08.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.790+0000 7f504e113700 1 -- 172.21.15.112:0/4259740779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50480cf170 msgr2=0x7f50480b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.790+0000 7f504e113700 1 --2- 172.21.15.112:0/4259740779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50480cf170 0x7f50480b0710 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f5038009840 tx=0x7f5038009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:08.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.790+0000 7f504e113700 1 -- 172.21.15.112:0/4259740779 shutdown_connections 2023-08-27T11:28:08.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.790+0000 7f504e113700 1 --2- 172.21.15.112:0/4259740779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50480cf170 0x7f50480b0710 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.790+0000 7f504e113700 1 -- 172.21.15.112:0/4259740779 >> 172.21.15.112:0/4259740779 conn(0x7f504806f970 msgr2=0x7f504806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.790+0000 7f504e113700 1 -- 172.21.15.112:0/4259740779 shutdown_connections 2023-08-27T11:28:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.790+0000 7f504e113700 1 -- 172.21.15.112:0/4259740779 wait complete. 2023-08-27T11:28:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.791+0000 7f504e113700 1 Processor -- start 2023-08-27T11:28:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.792+0000 7f504e113700 1 -- start start 2023-08-27T11:28:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.792+0000 7f504e113700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50480cf170 0x7f504814e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.792+0000 7f504e113700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50481578e0 0x7f50480cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.792+0000 7f504e113700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5048160f90 0x7f5048065fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.792+0000 7f504e113700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504815a0e0 con 0x7f5048160f90 2023-08-27T11:28:08.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.792+0000 7f504e113700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504814e860 con 0x7f50481578e0 2023-08-27T11:28:08.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.792+0000 7f504e113700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5048159d90 con 0x7f50480cf170 2023-08-27T11:28:08.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.792+0000 7f5047fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5048160f90 0x7f5048065fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.792+0000 7f5047fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5048160f90 0x7f5048065fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35876/0 (socket says 172.21.15.112:35876) 2023-08-27T11:28:08.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.792+0000 7f5047fff700 1 -- 172.21.15.112:0/3733989283 learned_addr learned my addr 172.21.15.112:0/3733989283 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:08.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.793+0000 7f50477fe700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50480cf170 0x7f504814e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.793+0000 7f5046ffd700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50481578e0 0x7f50480cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.793+0000 7f5047fff700 1 -- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50480cf170 msgr2=0x7f504814e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.793+0000 7f5047fff700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50480cf170 0x7f504814e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.793+0000 7f5047fff700 1 -- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50481578e0 msgr2=0x7f50480cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.793+0000 7f5047fff700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50481578e0 0x7f50480cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.793+0000 7f5047fff700 1 -- 172.21.15.112:0/3733989283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50380094a0 con 0x7f5048160f90 2023-08-27T11:28:08.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.793+0000 7f5047fff700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5048160f90 0x7f5048065fe0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f504000b990 tx=0x7f504000bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:08.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.793+0000 7f502ffff700 1 -- 172.21.15.112:0/3733989283 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504000d890 con 0x7f5048160f90 2023-08-27T11:28:08.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.793+0000 7f502ffff700 1 -- 172.21.15.112:0/3733989283 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504000d9f0 con 0x7f5048160f90 2023-08-27T11:28:08.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.793+0000 7f504e113700 1 -- 172.21.15.112:0/3733989283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5048157e40 con 0x7f5048160f90 2023-08-27T11:28:08.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.793+0000 7f502ffff700 1 -- 172.21.15.112:0/3733989283 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504000dda0 con 0x7f5048160f90 2023-08-27T11:28:08.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.793+0000 7f504e113700 1 -- 172.21.15.112:0/3733989283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5048159460 con 0x7f5048160f90 2023-08-27T11:28:08.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.794+0000 7f504e113700 1 -- 172.21.15.112:0/3733989283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5048159f00 con 0x7f5048160f90 2023-08-27T11:28:08.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.794+0000 7f502ffff700 1 -- 172.21.15.112:0/3733989283 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5040006d50 con 0x7f5048160f90 2023-08-27T11:28:08.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.795+0000 7f502ffff700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f503406a4d0 0x7f503406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.795+0000 7f502ffff700 1 -- 172.21.15.112:0/3733989283 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f5040088a10 con 0x7f5048160f90 2023-08-27T11:28:08.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.795+0000 7f50477fe700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f503406a4d0 0x7f503406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.796+0000 7f50477fe700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f503406a4d0 0x7f503406c990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f503c000f60 tx=0x7f503c007f10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:08.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:08.799+0000 7f502ffff700 1 -- 172.21.15.112:0/3733989283 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5040053c60 con 0x7f5048160f90 2023-08-27T11:28:09.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:09.004+0000 7f504e113700 1 -- 172.21.15.112:0/3733989283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5048067a20 con 0x7f5048160f90 2023-08-27T11:28:10.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.022+0000 7f502ffff700 1 -- 172.21.15.112:0/3733989283 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v135) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5040057280 con 0x7f5048160f90 2023-08-27T11:28:10.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.025+0000 7f504e113700 1 -- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f503406a4d0 msgr2=0x7f503406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.025+0000 7f504e113700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f503406a4d0 0x7f503406c990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f503c000f60 tx=0x7f503c007f10 comp rx=0 tx=0).stop 2023-08-27T11:28:10.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.025+0000 7f504e113700 1 -- 172.21.15.112:0/3733989283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5048160f90 msgr2=0x7f5048065fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.025+0000 7f504e113700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5048160f90 0x7f5048065fe0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f504000b990 tx=0x7f504000bca0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.025+0000 7f504e113700 1 -- 172.21.15.112:0/3733989283 shutdown_connections 2023-08-27T11:28:10.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.025+0000 7f504e113700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f503406a4d0 0x7f503406c990 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.025+0000 7f504e113700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5048160f90 0x7f5048065fe0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.025+0000 7f504e113700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50480cf170 0x7f504814e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.025+0000 7f504e113700 1 --2- 172.21.15.112:0/3733989283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50481578e0 0x7f50480cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.025+0000 7f504e113700 1 -- 172.21.15.112:0/3733989283 >> 172.21.15.112:0/3733989283 conn(0x7f504806f970 msgr2=0x7f50480d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:10.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.025+0000 7f504e113700 1 -- 172.21.15.112:0/3733989283 shutdown_connections 2023-08-27T11:28:10.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.025+0000 7f504e113700 1 -- 172.21.15.112:0/3733989283 wait complete. 2023-08-27T11:28:10.029 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2023-08-27T11:28:10.040 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:28:10.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.162+0000 7f58ef701700 1 Processor -- start 2023-08-27T11:28:10.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.162+0000 7f58ef701700 1 -- start start 2023-08-27T11:28:10.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.163+0000 7f58ef701700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f58e8151060 0x7f58e815e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.163+0000 7f58ef701700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f58e806b4c0 0x7f58e80cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.163+0000 7f58ef701700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e80b02f0 con 0x7f58e815e990 2023-08-27T11:28:10.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.163+0000 7f58ef701700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e80b05d0 con 0x7f58e8151060 2023-08-27T11:28:10.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.163+0000 7f58ef701700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e80b08b0 con 0x7f58e806b4c0 2023-08-27T11:28:10.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.163+0000 7f58ed49d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f58e8151060 0x7f58e815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.163+0000 7f58ecc9c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f58e815e990 0x7f58e806adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58498/0 (socket says 172.21.15.112:58498) 2023-08-27T11:28:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.163+0000 7f58ecc9c700 1 -- 172.21.15.112:0/3637941678 learned_addr learned my addr 172.21.15.112:0/3637941678 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.163+0000 7f58ed49d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f58e8151060 0x7f58e815e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35884/0 (socket says 172.21.15.112:35884) 2023-08-27T11:28:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58edc9e700 1 --2- 172.21.15.112:0/3637941678 >> v2:172.21.15.138:3300/0 conn(0x7f58e806b4c0 0x7f58e80cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58ed49d700 1 -- 172.21.15.112:0/3637941678 >> v2:172.21.15.138:3300/0 conn(0x7f58e806b4c0 msgr2=0x7f58e80cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58ed49d700 1 --2- 172.21.15.112:0/3637941678 >> v2:172.21.15.138:3300/0 conn(0x7f58e806b4c0 0x7f58e80cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58ed49d700 1 -- 172.21.15.112:0/3637941678 >> v1:172.21.15.112:6789/0 conn(0x7f58e815e990 legacy=0x7f58e806adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58ed49d700 1 -- 172.21.15.112:0/3637941678 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80b0b90 con 0x7f58e8151060 2023-08-27T11:28:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58dffff700 1 -- 172.21.15.112:0/3637941678 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368335859 0 0) 0x7f58e80b02f0 con 0x7f58e815e990 2023-08-27T11:28:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58ed49d700 1 --2- 172.21.15.112:0/3637941678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e8151060 0x7f58e815e450 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f58e80c7050 tx=0x7f58d8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa48cd271a0a56c9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:10.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58dffff700 1 -- 172.21.15.112:0/3637941678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d8011830 con 0x7f58e8151060 2023-08-27T11:28:10.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58dffff700 1 -- 172.21.15.112:0/3637941678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d8011990 con 0x7f58e8151060 2023-08-27T11:28:10.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58dffff700 1 -- 172.21.15.112:0/3637941678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d8011d20 con 0x7f58e8151060 2023-08-27T11:28:10.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58ef701700 1 -- 172.21.15.112:0/3637941678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e8151060 msgr2=0x7f58e815e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58ef701700 1 --2- 172.21.15.112:0/3637941678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e8151060 0x7f58e815e450 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f58e80c7050 tx=0x7f58d8009b20 comp rx=0 tx=0).stop 2023-08-27T11:28:10.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58ef701700 1 -- 172.21.15.112:0/3637941678 shutdown_connections 2023-08-27T11:28:10.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58ef701700 1 --2- 172.21.15.112:0/3637941678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e8151060 0x7f58e815e450 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.164+0000 7f58ef701700 1 --2- 172.21.15.112:0/3637941678 >> v2:172.21.15.138:3300/0 conn(0x7f58e806b4c0 0x7f58e80cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.165+0000 7f58ef701700 1 -- 172.21.15.112:0/3637941678 >> 172.21.15.112:0/3637941678 conn(0x7f58e80d0000 msgr2=0x7f58e80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:10.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.165+0000 7f58ef701700 1 -- 172.21.15.112:0/3637941678 shutdown_connections 2023-08-27T11:28:10.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.165+0000 7f58ef701700 1 -- 172.21.15.112:0/3637941678 wait complete. 2023-08-27T11:28:10.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.166+0000 7f58ef701700 1 Processor -- start 2023-08-27T11:28:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.166+0000 7f58ef701700 1 -- start start 2023-08-27T11:28:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.166+0000 7f58ef701700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e806b4c0 0x7f58e80c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.166+0000 7f58ef701700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58e8151060 0x7f58e814db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ef701700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58e815e990 0x7f58e8071a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ef701700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80c59b0 con 0x7f58e806b4c0 2023-08-27T11:28:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ef701700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e81f4250 con 0x7f58e8151060 2023-08-27T11:28:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ef701700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e81f4530 con 0x7f58e815e990 2023-08-27T11:28:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ed49d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e806b4c0 0x7f58e80c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ed49d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e806b4c0 0x7f58e80c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35890/0 (socket says 172.21.15.112:35890) 2023-08-27T11:28:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ed49d700 1 -- 172.21.15.112:0/998824400 learned_addr learned my addr 172.21.15.112:0/998824400 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58edc9e700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58e815e990 0x7f58e8071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ed49d700 1 -- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58e815e990 msgr2=0x7f58e8071a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ecc9c700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58e8151060 0x7f58e814db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ed49d700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58e815e990 0x7f58e8071a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ed49d700 1 -- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58e8151060 msgr2=0x7f58e814db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ed49d700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58e8151060 0x7f58e814db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ed49d700 1 -- 172.21.15.112:0/998824400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d80094a0 con 0x7f58e806b4c0 2023-08-27T11:28:10.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ed49d700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e806b4c0 0x7f58e80c53d0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f58d8000c00 tx=0x7f58d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:10.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ddffb700 1 -- 172.21.15.112:0/998824400 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d80038a0 con 0x7f58e806b4c0 2023-08-27T11:28:10.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ddffb700 1 -- 172.21.15.112:0/998824400 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d801f070 con 0x7f58e806b4c0 2023-08-27T11:28:10.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ef701700 1 -- 172.21.15.112:0/998824400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e81f4810 con 0x7f58e806b4c0 2023-08-27T11:28:10.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ddffb700 1 -- 172.21.15.112:0/998824400 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d8003c60 con 0x7f58e806b4c0 2023-08-27T11:28:10.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.167+0000 7f58ef701700 1 -- 172.21.15.112:0/998824400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e81f5010 con 0x7f58e806b4c0 2023-08-27T11:28:10.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.168+0000 7f58ef701700 1 -- 172.21.15.112:0/998824400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e81f4d30 con 0x7f58e806b4c0 2023-08-27T11:28:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.169+0000 7f58ddffb700 1 -- 172.21.15.112:0/998824400 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f58d801a430 con 0x7f58e806b4c0 2023-08-27T11:28:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.169+0000 7f58ddffb700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f58cc06a4d0 0x7f58cc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.169+0000 7f58ddffb700 1 -- 172.21.15.112:0/998824400 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f58d8021070 con 0x7f58e806b4c0 2023-08-27T11:28:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.169+0000 7f58ecc9c700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f58cc06a4d0 0x7f58cc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.170+0000 7f58ecc9c700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f58cc06a4d0 0x7f58cc06c990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f58e80714d0 tx=0x7f58e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:10.182 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.173+0000 7f58ddffb700 1 -- 172.21.15.112:0/998824400 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f58e81f4d30 con 0x7f58e806b4c0 2023-08-27T11:28:10.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.439+0000 7f58ef701700 1 -- 172.21.15.112:0/998824400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f58e81f4d30 con 0x7f58e806b4c0 2023-08-27T11:28:10.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.440+0000 7f58ddffb700 1 -- 172.21.15.112:0/998824400 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f58e81f4d30 con 0x7f58e806b4c0 2023-08-27T11:28:10.442 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:10.442 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:28:10.442 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2023-08-27T11:28:10.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.442 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2023-08-27T11:28:10.443 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk2bnkLBAA+vxcTeJzucKk9iSF/Zw3Lg==", 2023-08-27T11:28:10.443 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.443 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:10.443 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:28:10.443 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:28:10.444 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.444 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.444 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2023-08-27T11:28:10.445 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk3zD5OBAASVnwG6yn/YQzfYyeMakFpA==", 2023-08-27T11:28:10.445 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.445 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:10.445 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:28:10.445 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:28:10.445 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.446 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.446 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2023-08-27T11:28:10.446 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtki7wJNBAAsIGgDLM82sWNcKnCDlSBMw==", 2023-08-27T11:28:10.447 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.447 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:10.447 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:28:10.447 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:28:10.447 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.448 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.448 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2023-08-27T11:28:10.448 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkCsW/BBAAFTPiTOpBo1L+rLo9px+AjQ==", 2023-08-27T11:28:10.448 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.449 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:10.449 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:28:10.449 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:28:10.449 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.449 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.450 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2023-08-27T11:28:10.450 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB7MOtkR/cSHhAAR8e8VRd+e0ESJkqQDYWbEw==", 2023-08-27T11:28:10.450 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.451 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:10.451 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:10.451 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.452 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2023-08-27T11:28:10.452 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB+MOtksBF3HRAA1OIXiIkhgPDYpQyYPaWOXg==", 2023-08-27T11:28:10.452 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.453 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:10.453 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:10.453 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.453 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.454 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2023-08-27T11:28:10.454 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCBMOtkWsrdHBAAkfWbiE6wAFzpt9+BxRh8fw==", 2023-08-27T11:28:10.454 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.454 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:10.455 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:10.455 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.455 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.455 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2023-08-27T11:28:10.456 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCEMOtkIkiKHRAARDoh/FH9ZdqUlZZ29Mc/Dw==", 2023-08-27T11:28:10.456 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.456 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:10.456 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:10.456 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.456 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.457 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2023-08-27T11:28:10.457 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCKMOtkPgDzGxAA+U4kbwD0d4lS8aNPG2CgHw==", 2023-08-27T11:28:10.457 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.458 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:10.458 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:10.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.458 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.459 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2023-08-27T11:28:10.459 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCNMOtkKomSGxAAGAQUcIH2H/uyZIU8vlq1pw==", 2023-08-27T11:28:10.459 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.459 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:10.459 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:10.460 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.460 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.460 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2023-08-27T11:28:10.461 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCQMOtkHI2MGhAA7sgbi8JXO8P9wnVvMy3PqA==", 2023-08-27T11:28:10.461 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.461 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:10.461 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:10.461 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.461 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.462 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2023-08-27T11:28:10.462 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCTMOtkVHpiGhAA4C4YJMWbQ0mmGgB4i8G/AA==", 2023-08-27T11:28:10.463 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.463 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:10.463 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:10.463 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.463 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.464 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.464 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2023-08-27T11:28:10.464 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA==", 2023-08-27T11:28:10.464 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.464 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:10.465 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2023-08-27T11:28:10.465 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:28:10.465 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.466 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2023-08-27T11:28:10.466 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtk66DbDxAAWZ5StELux0lriJ2gEN9icg==", 2023-08-27T11:28:10.466 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.466 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:10.466 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:28:10.466 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:28:10.467 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:28:10.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.467 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2023-08-27T11:28:10.467 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkViKJDBAAF2ofwO+78er9OyFfn2bO+g==", 2023-08-27T11:28:10.468 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:10.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:28:10.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:28:10.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:28:10.469 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.469 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2023-08-27T11:28:10.469 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkwt+QFRAAowWE69VqeRhQz0896rxx9Q==", 2023-08-27T11:28:10.469 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.470 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:10.470 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:28:10.470 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:28:10.470 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:28:10.470 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.471 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2023-08-27T11:28:10.471 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkjDA+DxAAjmOn82AC2OMpKlAVCD6hlg==", 2023-08-27T11:28:10.471 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.472 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:28:10.472 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:28:10.472 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:28:10.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.472 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.473 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2023-08-27T11:28:10.473 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkXguHKRAALHwzk8diVXqsMvTKr+AzLQ==", 2023-08-27T11:28:10.473 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.473 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:28:10.474 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:28:10.474 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.474 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:10.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:10.475 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2023-08-27T11:28:10.475 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkrRjBJRAAq3ftmIh81LKadpliVUiJ6g==", 2023-08-27T11:28:10.475 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:10.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:28:10.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:28:10.476 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:10.476 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.476 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:10.476 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:28:10.476 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:28:10.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.442+0000 7f58ef701700 1 -- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f58cc06a4d0 msgr2=0x7f58cc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.442+0000 7f58ef701700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f58cc06a4d0 0x7f58cc06c990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f58e80714d0 tx=0x7f58e4009040 comp rx=0 tx=0).stop 2023-08-27T11:28:10.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.442+0000 7f58ef701700 1 -- 172.21.15.112:0/998824400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e806b4c0 msgr2=0x7f58e80c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.442+0000 7f58ef701700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e806b4c0 0x7f58e80c53d0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f58d8000c00 tx=0x7f58d8004750 comp rx=0 tx=0).stop 2023-08-27T11:28:10.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.442+0000 7f58ef701700 1 -- 172.21.15.112:0/998824400 shutdown_connections 2023-08-27T11:28:10.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.442+0000 7f58ef701700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f58cc06a4d0 0x7f58cc06c990 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.442+0000 7f58ef701700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58e815e990 0x7f58e8071a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.442+0000 7f58ef701700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e806b4c0 0x7f58e80c53d0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.442+0000 7f58ef701700 1 --2- 172.21.15.112:0/998824400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58e8151060 0x7f58e814db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.442+0000 7f58ef701700 1 -- 172.21.15.112:0/998824400 >> 172.21.15.112:0/998824400 conn(0x7f58e80d0000 msgr2=0x7f58e80cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:10.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.442+0000 7f58ef701700 1 -- 172.21.15.112:0/998824400 shutdown_connections 2023-08-27T11:28:10.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.442+0000 7f58ef701700 1 -- 172.21.15.112:0/998824400 wait complete. 2023-08-27T11:28:10.481 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:10.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.600+0000 7f3002be8700 1 Processor -- start 2023-08-27T11:28:10.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.600+0000 7f3002be8700 1 -- start start 2023-08-27T11:28:10.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.600+0000 7f3002be8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2ffc146f90 0x7f2ffc0708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.600+0000 7f3002be8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2ffc072230 0x7f2ffc0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.600+0000 7f3002be8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ffc1610d0 con 0x7f2ffc146bb0 2023-08-27T11:28:10.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f3002be8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ffc0727b0 con 0x7f2ffc146f90 2023-08-27T11:28:10.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f3002be8700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ffc0d0a40 con 0x7f2ffc072230 2023-08-27T11:28:10.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f2ffbfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2ffc146f90 0x7f2ffc0708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f3000984700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2ffc146bb0 0x7f2ffc15e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58512/0 (socket says 172.21.15.112:58512) 2023-08-27T11:28:10.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f3000984700 1 -- 172.21.15.112:0/2097714268 learned_addr learned my addr 172.21.15.112:0/2097714268 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:10.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f3001185700 1 --2- 172.21.15.112:0/2097714268 >> v2:172.21.15.138:3300/0 conn(0x7f2ffc072230 0x7f2ffc0cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f2ffbfff700 1 -- 172.21.15.112:0/2097714268 >> v2:172.21.15.138:3300/0 conn(0x7f2ffc072230 msgr2=0x7f2ffc0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:10.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f2ffbfff700 1 --2- 172.21.15.112:0/2097714268 >> v2:172.21.15.138:3300/0 conn(0x7f2ffc072230 0x7f2ffc0cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f2ffbfff700 1 -- 172.21.15.112:0/2097714268 >> v1:172.21.15.112:6789/0 conn(0x7f2ffc146bb0 legacy=0x7f2ffc15e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f2ffbfff700 1 -- 172.21.15.112:0/2097714268 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc0d0d20 con 0x7f2ffc146f90 2023-08-27T11:28:10.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f2ffbfff700 1 --2- 172.21.15.112:0/2097714268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc146f90 0x7f2ffc0708a0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0d01f0 tx=0x7f2fe8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206d89faed45b678 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:10.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f2ffb7fe700 1 -- 172.21.15.112:0/2097714268 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8011830 con 0x7f2ffc146f90 2023-08-27T11:28:10.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f2ffb7fe700 1 -- 172.21.15.112:0/2097714268 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe8011990 con 0x7f2ffc146f90 2023-08-27T11:28:10.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.601+0000 7f2ffb7fe700 1 -- 172.21.15.112:0/2097714268 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8011d20 con 0x7f2ffc146f90 2023-08-27T11:28:10.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.602+0000 7f3002be8700 1 -- 172.21.15.112:0/2097714268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc146f90 msgr2=0x7f2ffc0708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.602+0000 7f3002be8700 1 --2- 172.21.15.112:0/2097714268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc146f90 0x7f2ffc0708a0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0d01f0 tx=0x7f2fe8009b20 comp rx=0 tx=0).stop 2023-08-27T11:28:10.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.602+0000 7f3002be8700 1 -- 172.21.15.112:0/2097714268 shutdown_connections 2023-08-27T11:28:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.602+0000 7f3002be8700 1 --2- 172.21.15.112:0/2097714268 >> v2:172.21.15.138:3300/0 conn(0x7f2ffc072230 0x7f2ffc0cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.602+0000 7f3002be8700 1 --2- 172.21.15.112:0/2097714268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc146f90 0x7f2ffc0708a0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.602+0000 7f3002be8700 1 -- 172.21.15.112:0/2097714268 >> 172.21.15.112:0/2097714268 conn(0x7f2ffc06f970 msgr2=0x7f2ffc06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.602+0000 7f3002be8700 1 -- 172.21.15.112:0/2097714268 shutdown_connections 2023-08-27T11:28:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.602+0000 7f3002be8700 1 -- 172.21.15.112:0/2097714268 wait complete. 2023-08-27T11:28:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.603+0000 7f3002be8700 1 Processor -- start 2023-08-27T11:28:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.603+0000 7f3002be8700 1 -- start start 2023-08-27T11:28:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f3002be8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ffc072230 0x7f2ffc13bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f3002be8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc146bb0 0x7f2ffc0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f3002be8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ffc146f90 0x7f2ffc0cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f3002be8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc140840 con 0x7f2ffc146bb0 2023-08-27T11:28:10.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f3002be8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc13c1d0 con 0x7f2ffc072230 2023-08-27T11:28:10.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f3002be8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc0ce380 con 0x7f2ffc146f90 2023-08-27T11:28:10.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f2ffbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc146bb0 0x7f2ffc0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f2ffbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc146bb0 0x7f2ffc0cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35906/0 (socket says 172.21.15.112:35906) 2023-08-27T11:28:10.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f2ffbfff700 1 -- 172.21.15.112:0/24005537 learned_addr learned my addr 172.21.15.112:0/24005537 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:10.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f3000984700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ffc072230 0x7f2ffc13bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f3001185700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ffc146f90 0x7f2ffc0cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f2ffbfff700 1 -- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ffc146f90 msgr2=0x7f2ffc0cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f2ffbfff700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ffc146f90 0x7f2ffc0cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f2ffbfff700 1 -- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ffc072230 msgr2=0x7f2ffc13bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f2ffbfff700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ffc072230 0x7f2ffc13bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f2ffbfff700 1 -- 172.21.15.112:0/24005537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe80094a0 con 0x7f2ffc146bb0 2023-08-27T11:28:10.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f3000984700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ffc072230 0x7f2ffc13bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:10.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f2ffbfff700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc146bb0 0x7f2ffc0cb490 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8000c00 tx=0x7f2fe8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:10.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.604+0000 7f2ff97fa700 1 -- 172.21.15.112:0/24005537 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80036a0 con 0x7f2ffc146bb0 2023-08-27T11:28:10.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.605+0000 7f2ff97fa700 1 -- 172.21.15.112:0/24005537 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe801f070 con 0x7f2ffc146bb0 2023-08-27T11:28:10.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.605+0000 7f3002be8700 1 -- 172.21.15.112:0/24005537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ffc0ce0a0 con 0x7f2ffc146bb0 2023-08-27T11:28:10.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.605+0000 7f2ff97fa700 1 -- 172.21.15.112:0/24005537 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8003a60 con 0x7f2ffc146bb0 2023-08-27T11:28:10.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.605+0000 7f3002be8700 1 -- 172.21.15.112:0/24005537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc13c7c0 con 0x7f2ffc146bb0 2023-08-27T11:28:10.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.605+0000 7f3002be8700 1 -- 172.21.15.112:0/24005537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ffc067a20 con 0x7f2ffc146bb0 2023-08-27T11:28:10.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.606+0000 7f2ff97fa700 1 -- 172.21.15.112:0/24005537 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f2fe8003bc0 con 0x7f2ffc146bb0 2023-08-27T11:28:10.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.606+0000 7f2ff97fa700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2fe006a4d0 0x7f2fe006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.606+0000 7f2ff97fa700 1 -- 172.21.15.112:0/24005537 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2fe8021070 con 0x7f2ffc146bb0 2023-08-27T11:28:10.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.607+0000 7f3000984700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2fe006a4d0 0x7f2fe006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.607+0000 7f3000984700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2fe006a4d0 0x7f2fe006c990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0005910 tx=0x7f2ff0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:10.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.611+0000 7f2ff97fa700 1 -- 172.21.15.112:0/24005537 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2fe80bc990 con 0x7f2ffc146bb0 2023-08-27T11:28:10.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.801+0000 7f3002be8700 1 -- 172.21.15.112:0/24005537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ffc0caa40 con 0x7f2ffc146bb0 2023-08-27T11:28:10.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.802+0000 7f2ff97fa700 1 -- 172.21.15.112:0/24005537 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+14615 (secure 0 0 0) 0x7f2fe8015090 con 0x7f2ffc146bb0 2023-08-27T11:28:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:10.804 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":135,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:10.018113+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:10.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.804+0000 7f3002be8700 1 -- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2fe006a4d0 msgr2=0x7f2fe006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.804+0000 7f3002be8700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2fe006a4d0 0x7f2fe006c990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0005910 tx=0x7f2ff0009040 comp rx=0 tx=0).stop 2023-08-27T11:28:10.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.804+0000 7f3002be8700 1 -- 172.21.15.112:0/24005537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc146bb0 msgr2=0x7f2ffc0cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.804+0000 7f3002be8700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc146bb0 0x7f2ffc0cb490 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8000c00 tx=0x7f2fe8004750 comp rx=0 tx=0).stop 2023-08-27T11:28:10.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.804+0000 7f3002be8700 1 -- 172.21.15.112:0/24005537 shutdown_connections 2023-08-27T11:28:10.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.804+0000 7f3002be8700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2fe006a4d0 0x7f2fe006c990 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.804+0000 7f3002be8700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ffc146f90 0x7f2ffc0cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.804+0000 7f3002be8700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ffc072230 0x7f2ffc13bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.804+0000 7f3002be8700 1 --2- 172.21.15.112:0/24005537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc146bb0 0x7f2ffc0cb490 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.805+0000 7f3002be8700 1 -- 172.21.15.112:0/24005537 >> 172.21.15.112:0/24005537 conn(0x7f2ffc06f970 msgr2=0x7f2ffc15f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:10.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.805+0000 7f3002be8700 1 -- 172.21.15.112:0/24005537 shutdown_connections 2023-08-27T11:28:10.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.805+0000 7f3002be8700 1 -- 172.21.15.112:0/24005537 wait complete. 2023-08-27T11:28:10.819 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:28:10.819 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:28:10.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.941+0000 7fa3f1440700 1 Processor -- start 2023-08-27T11:28:10.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.941+0000 7fa3f1440700 1 -- start start 2023-08-27T11:28:10.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3f1440700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3ec0715c0 0x7fa3ec0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3f1440700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec0d1500 con 0x7fa3ec146bf0 2023-08-27T11:28:10.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3f1440700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec0d0460 con 0x7fa3ec146fd0 2023-08-27T11:28:10.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3f1440700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ec0d0710 con 0x7fa3ec0715c0 2023-08-27T11:28:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3eaffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa3ec146bf0 0x7fa3ec0cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58526/0 (socket says 172.21.15.112:58526) 2023-08-27T11:28:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3eaffd700 1 -- 172.21.15.112:0/4029547433 learned_addr learned my addr 172.21.15.112:0/4029547433 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951589693 0 0) 0x7fa3ec0d1500 con 0x7fa3ec146bf0 2023-08-27T11:28:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3d40035b0 con 0x7fa3ec146bf0 2023-08-27T11:28:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3eb7fe700 1 --2- 172.21.15.112:0/4029547433 >> v2:172.21.15.138:3300/0 conn(0x7fa3ec0715c0 0x7fa3ec0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4060794328 0 0) 0x7fa3d40035b0 con 0x7fa3ec146bf0 2023-08-27T11:28:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 >> v2:172.21.15.138:3300/0 conn(0x7fa3ec0715c0 msgr2=0x7fa3ec0b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 --2- 172.21.15.112:0/4029547433 >> v2:172.21.15.138:3300/0 conn(0x7fa3ec0715c0 0x7fa3ec0b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 >> v1:172.21.15.138:6789/0 conn(0x7fa3ec146fd0 legacy=0x7fa3ec070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec0d09f0 con 0x7fa3ec146bf0 2023-08-27T11:28:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fa3e0001670 con 0x7fa3ec146bf0 2023-08-27T11:28:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 >> v1:172.21.15.112:6789/0 conn(0x7fa3ec146bf0 legacy=0x7fa3ec0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 --2- 172.21.15.112:0/4029547433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3d4003a00 0x7fa3d4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 --2- 172.21.15.112:0/4029547433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d4004850 0x7fa3d4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 --2- 172.21.15.112:0/4029547433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3d4005210 0x7fa3d400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec0d0710 con 0x7fa3d4004850 2023-08-27T11:28:10.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d400c1b0 con 0x7fa3d4003a00 2023-08-27T11:28:10.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.942+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d400c370 con 0x7fa3d4005210 2023-08-27T11:28:10.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3ea7fc700 1 --2- 172.21.15.112:0/4029547433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d4004850 0x7fa3d4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3eaffd700 1 --2- 172.21.15.112:0/4029547433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3d4003a00 0x7fa3d4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3eb7fe700 1 --2- 172.21.15.112:0/4029547433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3d4005210 0x7fa3d400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3ea7fc700 1 -- 172.21.15.112:0/4029547433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3d4005210 msgr2=0x7fa3d400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:10.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3ea7fc700 1 --2- 172.21.15.112:0/4029547433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3d4005210 0x7fa3d400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3ea7fc700 1 -- 172.21.15.112:0/4029547433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3d4003a00 msgr2=0x7fa3d4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:10.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3ea7fc700 1 --2- 172.21.15.112:0/4029547433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3d4003a00 0x7fa3d4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3eb7fe700 1 -- 172.21.15.112:0/4029547433 reap_dead start 2023-08-27T11:28:10.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3ea7fc700 1 -- 172.21.15.112:0/4029547433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d40035b0 con 0x7fa3d4004850 2023-08-27T11:28:10.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3eaffd700 1 --2- 172.21.15.112:0/4029547433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3d4003a00 0x7fa3d4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:10.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3ea7fc700 1 --2- 172.21.15.112:0/4029547433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d4004850 0x7fa3d4004cd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0017c0 tx=0x7fa3d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6d254f5ba4d76 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:10.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0052d0 con 0x7fa3d4004850 2023-08-27T11:28:10.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc005430 con 0x7fa3d4004850 2023-08-27T11:28:10.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.943+0000 7fa3e9ffb700 1 -- 172.21.15.112:0/4029547433 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0038c0 con 0x7fa3d4004850 2023-08-27T11:28:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.944+0000 7fa3f1440700 1 -- 172.21.15.112:0/4029547433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d4004850 msgr2=0x7fa3d4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.944+0000 7fa3f1440700 1 --2- 172.21.15.112:0/4029547433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d4004850 0x7fa3d4004cd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0017c0 tx=0x7fa3d4005880 comp rx=0 tx=0).stop 2023-08-27T11:28:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.944+0000 7fa3f1440700 1 -- 172.21.15.112:0/4029547433 shutdown_connections 2023-08-27T11:28:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.944+0000 7fa3f1440700 1 --2- 172.21.15.112:0/4029547433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d4004850 0x7fa3d4004cd0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.944+0000 7fa3f1440700 1 -- 172.21.15.112:0/4029547433 >> 172.21.15.112:0/4029547433 conn(0x7fa3ec06c450 msgr2=0x7fa3ec06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.944+0000 7fa3f1440700 1 -- 172.21.15.112:0/4029547433 shutdown_connections 2023-08-27T11:28:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.944+0000 7fa3f1440700 1 -- 172.21.15.112:0/4029547433 wait complete. 2023-08-27T11:28:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.945+0000 7fa3f1440700 1 Processor -- start 2023-08-27T11:28:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.945+0000 7fa3f1440700 1 -- start start 2023-08-27T11:28:10.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3f1440700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d4004850 0x7fa3ec0b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3f1440700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ec1f3020 0x7fa3ec0aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3f1440700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3ec1f34a0 0x7fa3ec0af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3f1440700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec0b7610 con 0x7fa3d4004850 2023-08-27T11:28:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3f1440700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec0b78f0 con 0x7fa3ec1f3020 2023-08-27T11:28:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3f1440700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec0b7bd0 con 0x7fa3ec1f34a0 2023-08-27T11:28:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3eaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d4004850 0x7fa3ec0b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3eaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d4004850 0x7fa3ec0b6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35924/0 (socket says 172.21.15.112:35924) 2023-08-27T11:28:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3eaffd700 1 -- 172.21.15.112:0/2936236116 learned_addr learned my addr 172.21.15.112:0/2936236116 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:10.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3eb7fe700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3ec1f34a0 0x7fa3ec0af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3ea7fc700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ec1f3020 0x7fa3ec0aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3eaffd700 1 -- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3ec1f34a0 msgr2=0x7fa3ec0af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3eaffd700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3ec1f34a0 0x7fa3ec0af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3eaffd700 1 -- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ec1f3020 msgr2=0x7fa3ec0aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3eaffd700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ec1f3020 0x7fa3ec0aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3eaffd700 1 -- 172.21.15.112:0/2936236116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec0b7eb0 con 0x7fa3d4004850 2023-08-27T11:28:10.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3eaffd700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d4004850 0x7fa3ec0b6dc0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0000fb0 tx=0x7fa3e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:10.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3d37fe700 1 -- 172.21.15.112:0/2936236116 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e000c570 con 0x7fa3d4004850 2023-08-27T11:28:10.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3d37fe700 1 -- 172.21.15.112:0/2936236116 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e000c6d0 con 0x7fa3d4004850 2023-08-27T11:28:10.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3f1440700 1 -- 172.21.15.112:0/2936236116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec0b8190 con 0x7fa3d4004850 2023-08-27T11:28:10.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3d37fe700 1 -- 172.21.15.112:0/2936236116 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e000ca60 con 0x7fa3d4004850 2023-08-27T11:28:10.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.946+0000 7fa3f1440700 1 -- 172.21.15.112:0/2936236116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec0b86b0 con 0x7fa3d4004850 2023-08-27T11:28:10.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.947+0000 7fa3f1440700 1 -- 172.21.15.112:0/2936236116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec067e90 con 0x7fa3d4004850 2023-08-27T11:28:10.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.948+0000 7fa3d37fe700 1 -- 172.21.15.112:0/2936236116 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa3e000cbc0 con 0x7fa3d4004850 2023-08-27T11:28:10.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.948+0000 7fa3d37fe700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa3d806a500 0x7fa3d806c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.948+0000 7fa3d37fe700 1 -- 172.21.15.112:0/2936236116 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa3e00886f0 con 0x7fa3d4004850 2023-08-27T11:28:10.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.948+0000 7fa3ea7fc700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa3d806a500 0x7fa3d806c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.949+0000 7fa3ea7fc700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa3d806a500 0x7fa3d806c9c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc005040 tx=0x7fa3dc009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:10.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:10.952+0000 7fa3d37fe700 1 -- 172.21.15.112:0/2936236116 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa3e0053b50 con 0x7fa3d4004850 2023-08-27T11:28:11.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.142+0000 7fa3f1440700 1 -- 172.21.15.112:0/2936236116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa3ec067a20 con 0x7fa3d4004850 2023-08-27T11:28:11.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.142+0000 7fa3d37fe700 1 -- 172.21.15.112:0/2936236116 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v135) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa3e0057170 con 0x7fa3d4004850 2023-08-27T11:28:11.144 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:28:11.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.144+0000 7fa3f1440700 1 -- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa3d806a500 msgr2=0x7fa3d806c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.144+0000 7fa3f1440700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa3d806a500 0x7fa3d806c9c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc005040 tx=0x7fa3dc009030 comp rx=0 tx=0).stop 2023-08-27T11:28:11.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.144+0000 7fa3f1440700 1 -- 172.21.15.112:0/2936236116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d4004850 msgr2=0x7fa3ec0b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.144+0000 7fa3f1440700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d4004850 0x7fa3ec0b6dc0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0000fb0 tx=0x7fa3e000ee80 comp rx=0 tx=0).stop 2023-08-27T11:28:11.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.144+0000 7fa3f1440700 1 -- 172.21.15.112:0/2936236116 shutdown_connections 2023-08-27T11:28:11.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.144+0000 7fa3f1440700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa3d806a500 0x7fa3d806c9c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.144+0000 7fa3f1440700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3ec1f34a0 0x7fa3ec0af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.144+0000 7fa3f1440700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d4004850 0x7fa3ec0b6dc0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.144+0000 7fa3f1440700 1 --2- 172.21.15.112:0/2936236116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ec1f3020 0x7fa3ec0aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.144+0000 7fa3f1440700 1 -- 172.21.15.112:0/2936236116 >> 172.21.15.112:0/2936236116 conn(0x7fa3ec06c450 msgr2=0x7fa3ec145ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:11.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.144+0000 7fa3f1440700 1 -- 172.21.15.112:0/2936236116 shutdown_connections 2023-08-27T11:28:11.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.144+0000 7fa3f1440700 1 -- 172.21.15.112:0/2936236116 wait complete. 2023-08-27T11:28:11.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:28:11.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.280+0000 7f6610f7c700 1 Processor -- start 2023-08-27T11:28:11.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.280+0000 7f6610f7c700 1 -- start start 2023-08-27T11:28:11.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.281+0000 7f6610f7c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660c1437c0 0x7f660c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.281+0000 7f6610f7c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f660c143ba0 0x7f660c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.281+0000 7f6610f7c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660c0cfd50 con 0x7f660c06d5b0 2023-08-27T11:28:11.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.281+0000 7f6610f7c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660c0cf140 con 0x7f660c143ba0 2023-08-27T11:28:11.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.281+0000 7f6610f7c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660c0cf420 con 0x7f660c1437c0 2023-08-27T11:28:11.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.281+0000 7f6609d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f660c143ba0 0x7f660c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.281+0000 7f6609d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f660c143ba0 0x7f660c0ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35932/0 (socket says 172.21.15.112:35932) 2023-08-27T11:28:11.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.281+0000 7f6609d9b700 1 -- 172.21.15.112:0/2622701833 learned_addr learned my addr 172.21.15.112:0/2622701833 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:11.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.282+0000 7f660a59c700 1 --2- 172.21.15.112:0/2622701833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660c1437c0 0x7f660c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.282+0000 7f6609d9b700 1 -- 172.21.15.112:0/2622701833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660c1437c0 msgr2=0x7f660c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:11.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.282+0000 7f6609d9b700 1 --2- 172.21.15.112:0/2622701833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660c1437c0 0x7f660c0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.282+0000 7f6609d9b700 1 -- 172.21.15.112:0/2622701833 >> v1:172.21.15.138:6789/0 conn(0x7f660c06d5b0 legacy=0x7f660c15dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:11.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.282+0000 7f6609d9b700 1 -- 172.21.15.112:0/2622701833 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c0cf700 con 0x7f660c143ba0 2023-08-27T11:28:11.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.282+0000 7f6609d9b700 1 --2- 172.21.15.112:0/2622701833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f660c143ba0 0x7f660c0ce860 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00bb90 tx=0x7f65fc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4836790ebfeefdc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.282+0000 7f660959a700 1 -- 172.21.15.112:0/2622701833 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00dd10 con 0x7f660c143ba0 2023-08-27T11:28:11.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.282+0000 7f660959a700 1 -- 172.21.15.112:0/2622701833 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc00de70 con 0x7f660c143ba0 2023-08-27T11:28:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.282+0000 7f660959a700 1 -- 172.21.15.112:0/2622701833 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc004230 con 0x7f660c143ba0 2023-08-27T11:28:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.282+0000 7f6610f7c700 1 -- 172.21.15.112:0/2622701833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f660c143ba0 msgr2=0x7f660c0ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.282+0000 7f6610f7c700 1 --2- 172.21.15.112:0/2622701833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f660c143ba0 0x7f660c0ce860 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00bb90 tx=0x7f65fc00bea0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.283+0000 7f6610f7c700 1 -- 172.21.15.112:0/2622701833 shutdown_connections 2023-08-27T11:28:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.283+0000 7f6610f7c700 1 --2- 172.21.15.112:0/2622701833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660c1437c0 0x7f660c0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.283+0000 7f6610f7c700 1 --2- 172.21.15.112:0/2622701833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f660c143ba0 0x7f660c0ce860 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.283+0000 7f6610f7c700 1 -- 172.21.15.112:0/2622701833 >> 172.21.15.112:0/2622701833 conn(0x7f660c0b1e90 msgr2=0x7f660c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.283+0000 7f6610f7c700 1 -- 172.21.15.112:0/2622701833 shutdown_connections 2023-08-27T11:28:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.283+0000 7f6610f7c700 1 -- 172.21.15.112:0/2622701833 wait complete. 2023-08-27T11:28:11.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.284+0000 7f6610f7c700 1 Processor -- start 2023-08-27T11:28:11.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.284+0000 7f6610f7c700 1 -- start start 2023-08-27T11:28:11.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f6610f7c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f660c06d5b0 0x7f660c15a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f6610f7c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660c1437c0 0x7f660c1f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f6610f7c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f660c143ba0 0x7f660c1ede50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f6610f7c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c13d6f0 con 0x7f660c143ba0 2023-08-27T11:28:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f6610f7c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c138c40 con 0x7f660c06d5b0 2023-08-27T11:28:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f6610f7c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660c138f20 con 0x7f660c1437c0 2023-08-27T11:28:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f660ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f660c143ba0 0x7f660c1ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f660ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f660c143ba0 0x7f660c1ede50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35940/0 (socket says 172.21.15.112:35940) 2023-08-27T11:28:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f660ad9d700 1 -- 172.21.15.112:0/978788855 learned_addr learned my addr 172.21.15.112:0/978788855 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f660a59c700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f660c06d5b0 0x7f660c15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f6609d9b700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660c1437c0 0x7f660c1f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f660ad9d700 1 -- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660c1437c0 msgr2=0x7f660c1f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f660ad9d700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660c1437c0 0x7f660c1f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f660ad9d700 1 -- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f660c06d5b0 msgr2=0x7f660c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f660ad9d700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f660c06d5b0 0x7f660c15a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f660ad9d700 1 -- 172.21.15.112:0/978788855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc00b7f0 con 0x7f660c143ba0 2023-08-27T11:28:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f660ad9d700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f660c143ba0 0x7f660c1ede50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f660400cc10 tx=0x7f660400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f65faffd700 1 -- 172.21.15.112:0/978788855 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660400aa60 con 0x7f660c143ba0 2023-08-27T11:28:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f65faffd700 1 -- 172.21.15.112:0/978788855 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660400abc0 con 0x7f660c143ba0 2023-08-27T11:28:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f6610f7c700 1 -- 172.21.15.112:0/978788855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c139200 con 0x7f660c143ba0 2023-08-27T11:28:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f65faffd700 1 -- 172.21.15.112:0/978788855 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604012420 con 0x7f660c143ba0 2023-08-27T11:28:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.285+0000 7f6610f7c700 1 -- 172.21.15.112:0/978788855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660c1f2780 con 0x7f660c143ba0 2023-08-27T11:28:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.286+0000 7f6610f7c700 1 -- 172.21.15.112:0/978788855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660c138aa0 con 0x7f660c143ba0 2023-08-27T11:28:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.287+0000 7f65faffd700 1 -- 172.21.15.112:0/978788855 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f6604012580 con 0x7f660c143ba0 2023-08-27T11:28:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.287+0000 7f65faffd700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65ec06a4d0 0x7f65ec06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.287+0000 7f65faffd700 1 -- 172.21.15.112:0/978788855 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6604088b60 con 0x7f660c143ba0 2023-08-27T11:28:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.287+0000 7f660a59c700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65ec06a4d0 0x7f65ec06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.288+0000 7f660a59c700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65ec06a4d0 0x7f65ec06c990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f660c13a9b0 tx=0x7f6600008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.292+0000 7f65faffd700 1 -- 172.21.15.112:0/978788855 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f660c138aa0 con 0x7f660c143ba0 2023-08-27T11:28:11.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.480+0000 7f6610f7c700 1 -- 172.21.15.112:0/978788855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f660c138aa0 con 0x7f660c143ba0 2023-08-27T11:28:11.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.480+0000 7f65faffd700 1 -- 172.21.15.112:0/978788855 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) v1 ==== 93+0+10 (secure 0 0 0) 0x7f660c138aa0 con 0x7f660c143ba0 2023-08-27T11:28:11.482 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:28:11.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.482+0000 7f6610f7c700 1 -- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65ec06a4d0 msgr2=0x7f65ec06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.482+0000 7f6610f7c700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65ec06a4d0 0x7f65ec06c990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f660c13a9b0 tx=0x7f6600008040 comp rx=0 tx=0).stop 2023-08-27T11:28:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.482+0000 7f6610f7c700 1 -- 172.21.15.112:0/978788855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f660c143ba0 msgr2=0x7f660c1ede50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.482+0000 7f6610f7c700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f660c143ba0 0x7f660c1ede50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f660400cc10 tx=0x7f660400cf20 comp rx=0 tx=0).stop 2023-08-27T11:28:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.482+0000 7f6610f7c700 1 -- 172.21.15.112:0/978788855 shutdown_connections 2023-08-27T11:28:11.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.482+0000 7f6610f7c700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f65ec06a4d0 0x7f65ec06c990 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.482+0000 7f6610f7c700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f660c143ba0 0x7f660c1ede50 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.482+0000 7f6610f7c700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f660c06d5b0 0x7f660c15a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.482+0000 7f6610f7c700 1 --2- 172.21.15.112:0/978788855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660c1437c0 0x7f660c1f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.482+0000 7f6610f7c700 1 -- 172.21.15.112:0/978788855 >> 172.21.15.112:0/978788855 conn(0x7f660c0b1e90 msgr2=0x7f660c1410c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:11.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.482+0000 7f6610f7c700 1 -- 172.21.15.112:0/978788855 shutdown_connections 2023-08-27T11:28:11.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.482+0000 7f6610f7c700 1 -- 172.21.15.112:0/978788855 wait complete. 2023-08-27T11:28:11.496 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:28:11.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.615+0000 7fc6a8144700 1 Processor -- start 2023-08-27T11:28:11.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.616+0000 7fc6a8144700 1 -- start start 2023-08-27T11:28:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.616+0000 7fc6a8144700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc6a006fea0 0x7fc6a00cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.616+0000 7fc6a8144700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a0161850 con 0x7fc6a00b17e0 2023-08-27T11:28:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.616+0000 7fc6a8144700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a00cfc10 con 0x7fc6a00b1400 2023-08-27T11:28:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.616+0000 7fc6a8144700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a0161200 con 0x7fc6a006fea0 2023-08-27T11:28:11.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.616+0000 7fc6a56df700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc6a00b17e0 0x7fc6a006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58558/0 (socket says 172.21.15.112:58558) 2023-08-27T11:28:11.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.616+0000 7fc6a56df700 1 -- 172.21.15.112:0/4169125649 learned_addr learned my addr 172.21.15.112:0/4169125649 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:11.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.616+0000 7fc6a66e1700 1 --2- 172.21.15.112:0/4169125649 >> v2:172.21.15.112:3300/0 conn(0x7fc6a006fea0 0x7fc6a00cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.617+0000 7fc6a66e1700 1 -- 172.21.15.112:0/4169125649 >> v1:172.21.15.138:6789/0 conn(0x7fc6a00b1400 legacy=0x7fc6a006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.617+0000 7fc6a66e1700 1 -- 172.21.15.112:0/4169125649 >> v1:172.21.15.112:6789/0 conn(0x7fc6a00b17e0 legacy=0x7fc6a006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.617+0000 7fc6a66e1700 1 -- 172.21.15.112:0/4169125649 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a01614e0 con 0x7fc6a006fea0 2023-08-27T11:28:11.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.617+0000 7fc6a66e1700 1 --2- 172.21.15.112:0/4169125649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a006fea0 0x7fc6a00cf5a0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fc698009d90 tx=0x7fc69800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5975d2f127357c1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.619+0000 7fc6a4ede700 1 -- 172.21.15.112:0/4169125649 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69800e070 con 0x7fc6a006fea0 2023-08-27T11:28:11.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.619+0000 7fc6a4ede700 1 -- 172.21.15.112:0/4169125649 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69800bbd0 con 0x7fc6a006fea0 2023-08-27T11:28:11.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.619+0000 7fc6a4ede700 1 -- 172.21.15.112:0/4169125649 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc698012430 con 0x7fc6a006fea0 2023-08-27T11:28:11.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.619+0000 7fc6a8144700 1 -- 172.21.15.112:0/4169125649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a006fea0 msgr2=0x7fc6a00cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.619+0000 7fc6a8144700 1 --2- 172.21.15.112:0/4169125649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a006fea0 0x7fc6a00cf5a0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fc698009d90 tx=0x7fc69800b330 comp rx=0 tx=0).stop 2023-08-27T11:28:11.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.619+0000 7fc6a8144700 1 -- 172.21.15.112:0/4169125649 shutdown_connections 2023-08-27T11:28:11.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.619+0000 7fc6a8144700 1 --2- 172.21.15.112:0/4169125649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a006fea0 0x7fc6a00cf5a0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.619+0000 7fc6a8144700 1 -- 172.21.15.112:0/4169125649 >> 172.21.15.112:0/4169125649 conn(0x7fc6a00d0000 msgr2=0x7fc6a00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:11.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.619+0000 7fc6a8144700 1 -- 172.21.15.112:0/4169125649 shutdown_connections 2023-08-27T11:28:11.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.619+0000 7fc6a8144700 1 -- 172.21.15.112:0/4169125649 wait complete. 2023-08-27T11:28:11.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.621+0000 7fc6a8144700 1 Processor -- start 2023-08-27T11:28:11.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.621+0000 7fc6a8144700 1 -- start start 2023-08-27T11:28:11.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.621+0000 7fc6a8144700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6a006fea0 0x7fc6a01f0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.621+0000 7fc6a8144700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a00b1400 0x7fc6a00c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.621+0000 7fc6a8144700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a00b17e0 0x7fc6a01f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.621+0000 7fc6a8144700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a01f3640 con 0x7fc6a00b1400 2023-08-27T11:28:11.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.621+0000 7fc6a8144700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00c8180 con 0x7fc6a00b17e0 2023-08-27T11:28:11.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.621+0000 7fc6a8144700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00c8460 con 0x7fc6a006fea0 2023-08-27T11:28:11.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.621+0000 7fc6a56df700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a00b1400 0x7fc6a00c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6a56df700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a00b1400 0x7fc6a00c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35950/0 (socket says 172.21.15.112:35950) 2023-08-27T11:28:11.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6a56df700 1 -- 172.21.15.112:0/2565248367 learned_addr learned my addr 172.21.15.112:0/2565248367 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:11.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6a66e1700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a00b17e0 0x7fc6a01f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6a5ee0700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6a006fea0 0x7fc6a01f0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6a56df700 1 -- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6a006fea0 msgr2=0x7fc6a01f0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6a56df700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6a006fea0 0x7fc6a01f0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6a56df700 1 -- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a00b17e0 msgr2=0x7fc6a01f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6a56df700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a00b17e0 0x7fc6a01f3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6a56df700 1 -- 172.21.15.112:0/2565248367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc698009a70 con 0x7fc6a00b1400 2023-08-27T11:28:11.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6a5ee0700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6a006fea0 0x7fc6a01f0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:11.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6a56df700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a00b1400 0x7fc6a00c7950 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c450 tx=0x7fc69c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6967fc700 1 -- 172.21.15.112:0/2565248367 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c005d70 con 0x7fc6a00b1400 2023-08-27T11:28:11.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6967fc700 1 -- 172.21.15.112:0/2565248367 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69c0070d0 con 0x7fc6a00b1400 2023-08-27T11:28:11.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6a8144700 1 -- 172.21.15.112:0/2565248367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a00c8740 con 0x7fc6a00b1400 2023-08-27T11:28:11.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6967fc700 1 -- 172.21.15.112:0/2565248367 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c007440 con 0x7fc6a00b1400 2023-08-27T11:28:11.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.622+0000 7fc6a8144700 1 -- 172.21.15.112:0/2565248367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a00c7e90 con 0x7fc6a00b1400 2023-08-27T11:28:11.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.623+0000 7fc6a8144700 1 -- 172.21.15.112:0/2565248367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a01f37f0 con 0x7fc6a00b1400 2023-08-27T11:28:11.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.623+0000 7fc6967fc700 1 -- 172.21.15.112:0/2565248367 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fc69c0075a0 con 0x7fc6a00b1400 2023-08-27T11:28:11.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.624+0000 7fc6967fc700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc68406a4d0 0x7fc68406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.624+0000 7fc6967fc700 1 -- 172.21.15.112:0/2565248367 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc69c087bf0 con 0x7fc6a00b1400 2023-08-27T11:28:11.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.624+0000 7fc6a5ee0700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc68406a4d0 0x7fc68406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.625+0000 7fc6a5ee0700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc68406a4d0 0x7fc68406c990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc6a01579c0 tx=0x7fc690009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.629+0000 7fc6967fc700 1 -- 172.21.15.112:0/2565248367 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc6a01f37f0 con 0x7fc6a00b1400 2023-08-27T11:28:11.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.817+0000 7fc6a8144700 1 -- 172.21.15.112:0/2565248367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc6a01f37f0 con 0x7fc6a00b1400 2023-08-27T11:28:11.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.818+0000 7fc6967fc700 1 -- 172.21.15.112:0/2565248367 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v135) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc69c017090 con 0x7fc6a00b1400 2023-08-27T11:28:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:11.820 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:28:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2023-08-27T11:28:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "rbd" 2023-08-27T11:28:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 2, 2023-08-27T11:28:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2023-08-27T11:28:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:11.822 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:28:11.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.820+0000 7fc6a8144700 1 -- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc68406a4d0 msgr2=0x7fc68406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.820+0000 7fc6a8144700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc68406a4d0 0x7fc68406c990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc6a01579c0 tx=0x7fc690009040 comp rx=0 tx=0).stop 2023-08-27T11:28:11.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.820+0000 7fc6a8144700 1 -- 172.21.15.112:0/2565248367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a00b1400 msgr2=0x7fc6a00c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.820+0000 7fc6a8144700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a00b1400 0x7fc6a00c7950 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c450 tx=0x7fc69c00c760 comp rx=0 tx=0).stop 2023-08-27T11:28:11.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.820+0000 7fc6a8144700 1 -- 172.21.15.112:0/2565248367 shutdown_connections 2023-08-27T11:28:11.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.820+0000 7fc6a8144700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc68406a4d0 0x7fc68406c990 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.820+0000 7fc6a8144700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6a00b17e0 0x7fc6a01f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.820+0000 7fc6a8144700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6a006fea0 0x7fc6a01f0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.820+0000 7fc6a8144700 1 --2- 172.21.15.112:0/2565248367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a00b1400 0x7fc6a00c7950 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.820+0000 7fc6a8144700 1 -- 172.21.15.112:0/2565248367 >> 172.21.15.112:0/2565248367 conn(0x7fc6a00d0000 msgr2=0x7fc6a015eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:11.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.820+0000 7fc6a8144700 1 -- 172.21.15.112:0/2565248367 shutdown_connections 2023-08-27T11:28:11.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.820+0000 7fc6a8144700 1 -- 172.21.15.112:0/2565248367 wait complete. 2023-08-27T11:28:11.834 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:28:11.834 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:28:11.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.957+0000 7fc559ee9700 1 Processor -- start 2023-08-27T11:28:11.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.957+0000 7fc559ee9700 1 -- start start 2023-08-27T11:28:11.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc559ee9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5540ce8b0 0x7fc55406adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc559ee9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc55406e120 0x7fc55406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc559ee9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5540ce280 con 0x7fc55415a8d0 2023-08-27T11:28:11.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc559ee9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55415b800 con 0x7fc5540ce8b0 2023-08-27T11:28:11.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc559ee9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55415b520 con 0x7fc55406e120 2023-08-27T11:28:11.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc5537fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5540ce8b0 0x7fc55406adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc553fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc55415a8d0 0x7fc55415dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58574/0 (socket says 172.21.15.112:58574) 2023-08-27T11:28:11.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc553fff700 1 -- 172.21.15.112:0/2994553606 learned_addr learned my addr 172.21.15.112:0/2994553606 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:11.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc5537fe700 1 -- 172.21.15.112:0/2994553606 >> v2:172.21.15.138:3300/0 conn(0x7fc55406e120 msgr2=0x7fc55406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:11.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc5537fe700 1 --2- 172.21.15.112:0/2994553606 >> v2:172.21.15.138:3300/0 conn(0x7fc55406e120 0x7fc55406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc5537fe700 1 -- 172.21.15.112:0/2994553606 >> v1:172.21.15.112:6789/0 conn(0x7fc55415a8d0 legacy=0x7fc55415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc5537fe700 1 -- 172.21.15.112:0/2994553606 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55406d640 con 0x7fc5540ce8b0 2023-08-27T11:28:11.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc5527fc700 1 -- 172.21.15.112:0/2994553606 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342116104 0 0) 0x7fc5540ce280 con 0x7fc55415a8d0 2023-08-27T11:28:11.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc5537fe700 1 --2- 172.21.15.112:0/2994553606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5540ce8b0 0x7fc55406adb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fc5540cdb00 tx=0x7fc548009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e0c0b4cd95be51 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.958+0000 7fc5527fc700 1 -- 172.21.15.112:0/2994553606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548011830 con 0x7fc5540ce8b0 2023-08-27T11:28:11.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.959+0000 7fc5527fc700 1 -- 172.21.15.112:0/2994553606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc548011990 con 0x7fc5540ce8b0 2023-08-27T11:28:11.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.959+0000 7fc5527fc700 1 -- 172.21.15.112:0/2994553606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548011d20 con 0x7fc5540ce8b0 2023-08-27T11:28:11.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.959+0000 7fc559ee9700 1 -- 172.21.15.112:0/2994553606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5540ce8b0 msgr2=0x7fc55406adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.959+0000 7fc559ee9700 1 --2- 172.21.15.112:0/2994553606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5540ce8b0 0x7fc55406adb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fc5540cdb00 tx=0x7fc548009b20 comp rx=0 tx=0).stop 2023-08-27T11:28:11.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.959+0000 7fc559ee9700 1 -- 172.21.15.112:0/2994553606 shutdown_connections 2023-08-27T11:28:11.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.959+0000 7fc559ee9700 1 --2- 172.21.15.112:0/2994553606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5540ce8b0 0x7fc55406adb0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.959+0000 7fc559ee9700 1 --2- 172.21.15.112:0/2994553606 >> v2:172.21.15.138:3300/0 conn(0x7fc55406e120 0x7fc55406b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.959+0000 7fc559ee9700 1 -- 172.21.15.112:0/2994553606 >> 172.21.15.112:0/2994553606 conn(0x7fc5540ccef0 msgr2=0x7fc5540cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:11.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.959+0000 7fc559ee9700 1 -- 172.21.15.112:0/2994553606 shutdown_connections 2023-08-27T11:28:11.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.959+0000 7fc559ee9700 1 -- 172.21.15.112:0/2994553606 wait complete. 2023-08-27T11:28:11.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.960+0000 7fc559ee9700 1 Processor -- start 2023-08-27T11:28:11.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.960+0000 7fc559ee9700 1 -- start start 2023-08-27T11:28:11.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.961+0000 7fc559ee9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc55406e120 0x7fc554157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.961+0000 7fc559ee9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5540ce8b0 0x7fc554159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.961+0000 7fc559ee9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc55415a8d0 0x7fc55415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.961+0000 7fc559ee9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55415caa0 con 0x7fc5540ce8b0 2023-08-27T11:28:11.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.961+0000 7fc559ee9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55415be90 con 0x7fc55415a8d0 2023-08-27T11:28:11.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.961+0000 7fc559ee9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55415c170 con 0x7fc55406e120 2023-08-27T11:28:11.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.961+0000 7fc552ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5540ce8b0 0x7fc554159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.961+0000 7fc552ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5540ce8b0 0x7fc554159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35964/0 (socket says 172.21.15.112:35964) 2023-08-27T11:28:11.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.961+0000 7fc552ffd700 1 -- 172.21.15.112:0/53928948 learned_addr learned my addr 172.21.15.112:0/53928948 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:11.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.962+0000 7fc5537fe700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc55406e120 0x7fc554157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.962+0000 7fc553fff700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc55415a8d0 0x7fc55415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.962+0000 7fc552ffd700 1 -- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc55406e120 msgr2=0x7fc554157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.962+0000 7fc552ffd700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc55406e120 0x7fc554157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.962+0000 7fc552ffd700 1 -- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc55415a8d0 msgr2=0x7fc55415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.962+0000 7fc552ffd700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc55415a8d0 0x7fc55415a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.962+0000 7fc552ffd700 1 -- 172.21.15.112:0/53928948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480094a0 con 0x7fc5540ce8b0 2023-08-27T11:28:11.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.962+0000 7fc552ffd700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5540ce8b0 0x7fc554159bb0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fc54400d920 tx=0x7fc54400dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.962+0000 7fc53bfff700 1 -- 172.21.15.112:0/53928948 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54400ec50 con 0x7fc5540ce8b0 2023-08-27T11:28:11.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.962+0000 7fc53bfff700 1 -- 172.21.15.112:0/53928948 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54400edb0 con 0x7fc5540ce8b0 2023-08-27T11:28:11.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.962+0000 7fc559ee9700 1 -- 172.21.15.112:0/53928948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55415c450 con 0x7fc5540ce8b0 2023-08-27T11:28:11.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.962+0000 7fc53bfff700 1 -- 172.21.15.112:0/53928948 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544016680 con 0x7fc5540ce8b0 2023-08-27T11:28:11.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.962+0000 7fc559ee9700 1 -- 172.21.15.112:0/53928948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5541575c0 con 0x7fc5540ce8b0 2023-08-27T11:28:11.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.963+0000 7fc559ee9700 1 -- 172.21.15.112:0/53928948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5540cf000 con 0x7fc5540ce8b0 2023-08-27T11:28:11.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.963+0000 7fc53bfff700 1 -- 172.21.15.112:0/53928948 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fc54401f3b0 con 0x7fc5540ce8b0 2023-08-27T11:28:11.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.964+0000 7fc53bfff700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc54006a4a0 0x7fc54006c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.964+0000 7fc53bfff700 1 -- 172.21.15.112:0/53928948 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc54408a800 con 0x7fc5540ce8b0 2023-08-27T11:28:11.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.964+0000 7fc5537fe700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc54006a4a0 0x7fc54006c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.965+0000 7fc5537fe700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc54006a4a0 0x7fc54006c960 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc5480090b0 tx=0x7fc548008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:11.968+0000 7fc53bfff700 1 -- 172.21.15.112:0/53928948 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc544055c20 con 0x7fc5540ce8b0 2023-08-27T11:28:12.158 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.156+0000 7fc559ee9700 1 -- 172.21.15.112:0/53928948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc554067a20 con 0x7fc5540ce8b0 2023-08-27T11:28:12.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.629+0000 7fc53bfff700 1 -- 172.21.15.112:0/53928948 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v136) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc544059240 con 0x7fc5540ce8b0 2023-08-27T11:28:12.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.632+0000 7fc559ee9700 1 -- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc54006a4a0 msgr2=0x7fc54006c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.632+0000 7fc559ee9700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc54006a4a0 0x7fc54006c960 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc5480090b0 tx=0x7fc548008030 comp rx=0 tx=0).stop 2023-08-27T11:28:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.632+0000 7fc559ee9700 1 -- 172.21.15.112:0/53928948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5540ce8b0 msgr2=0x7fc554159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.632+0000 7fc559ee9700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5540ce8b0 0x7fc554159bb0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fc54400d920 tx=0x7fc54400dce0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.633+0000 7fc559ee9700 1 -- 172.21.15.112:0/53928948 shutdown_connections 2023-08-27T11:28:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.633+0000 7fc559ee9700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc54006a4a0 0x7fc54006c960 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.633+0000 7fc559ee9700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc55415a8d0 0x7fc55415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.633+0000 7fc559ee9700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc55406e120 0x7fc554157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.633+0000 7fc559ee9700 1 --2- 172.21.15.112:0/53928948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5540ce8b0 0x7fc554159bb0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.633+0000 7fc559ee9700 1 -- 172.21.15.112:0/53928948 >> 172.21.15.112:0/53928948 conn(0x7fc5540ccef0 msgr2=0x7fc554158460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.633+0000 7fc559ee9700 1 -- 172.21.15.112:0/53928948 shutdown_connections 2023-08-27T11:28:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.633+0000 7fc559ee9700 1 -- 172.21.15.112:0/53928948 wait complete. 2023-08-27T11:28:12.638 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2023-08-27T11:28:12.647 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:28:12.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.774+0000 7f4c1b7b6700 1 Processor -- start 2023-08-27T11:28:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.774+0000 7f4c1b7b6700 1 -- start start 2023-08-27T11:28:12.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c1b7b6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4c141437c0 0x7f4c140b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c1b7b6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c1406d5b0 0x7f4c1406da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c1b7b6700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c140cfd50 con 0x7f4c14143ba0 2023-08-27T11:28:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c1b7b6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1406df80 con 0x7f4c141437c0 2023-08-27T11:28:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c1b7b6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c140cf140 con 0x7f4c1406d5b0 2023-08-27T11:28:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c19552700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4c141437c0 0x7f4c140b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c19552700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4c141437c0 0x7f4c140b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35970/0 (socket says 172.21.15.112:35970) 2023-08-27T11:28:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c19552700 1 -- 172.21.15.112:0/4023849645 learned_addr learned my addr 172.21.15.112:0/4023849645 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:12.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c19d53700 1 --2- 172.21.15.112:0/4023849645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c1406d5b0 0x7f4c1406da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c19552700 1 -- 172.21.15.112:0/4023849645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c1406d5b0 msgr2=0x7f4c1406da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:12.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c19552700 1 --2- 172.21.15.112:0/4023849645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c1406d5b0 0x7f4c1406da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c19552700 1 -- 172.21.15.112:0/4023849645 >> v1:172.21.15.138:6789/0 conn(0x7f4c14143ba0 legacy=0x7f4c140ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:12.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c19552700 1 -- 172.21.15.112:0/4023849645 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c140cf420 con 0x7f4c141437c0 2023-08-27T11:28:12.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c19d53700 1 --2- 172.21.15.112:0/4023849645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c1406d5b0 0x7f4c1406da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:12.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c19552700 1 --2- 172.21.15.112:0/4023849645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c141437c0 0x7f4c140b4670 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f4c100098f0 tx=0x7f4c10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1847a802afb0746d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:12.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c0bfff700 1 -- 172.21.15.112:0/4023849645 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10004230 con 0x7f4c141437c0 2023-08-27T11:28:12.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c0bfff700 1 -- 172.21.15.112:0/4023849645 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c10004390 con 0x7f4c141437c0 2023-08-27T11:28:12.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.775+0000 7f4c0bfff700 1 -- 172.21.15.112:0/4023849645 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c100036a0 con 0x7f4c141437c0 2023-08-27T11:28:12.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.776+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/4023849645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c141437c0 msgr2=0x7f4c140b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.776+0000 7f4c1b7b6700 1 --2- 172.21.15.112:0/4023849645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c141437c0 0x7f4c140b4670 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f4c100098f0 tx=0x7f4c10009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:12.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.776+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/4023849645 shutdown_connections 2023-08-27T11:28:12.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.776+0000 7f4c1b7b6700 1 --2- 172.21.15.112:0/4023849645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c1406d5b0 0x7f4c1406da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.776+0000 7f4c1b7b6700 1 --2- 172.21.15.112:0/4023849645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c141437c0 0x7f4c140b4670 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.776+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/4023849645 >> 172.21.15.112:0/4023849645 conn(0x7f4c140b1e90 msgr2=0x7f4c140b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:12.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.776+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/4023849645 shutdown_connections 2023-08-27T11:28:12.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.776+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/4023849645 wait complete. 2023-08-27T11:28:12.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.777+0000 7f4c1b7b6700 1 Processor -- start 2023-08-27T11:28:12.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.777+0000 7f4c1b7b6700 1 -- start start 2023-08-27T11:28:12.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c1b7b6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c1406d5b0 0x7f4c141332a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c1b7b6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c141437c0 0x7f4c14159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c1b7b6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c14143ba0 0x7f4c140708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c1b7b6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14136190 con 0x7f4c1406d5b0 2023-08-27T11:28:12.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c1b7b6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c140ca200 con 0x7f4c141437c0 2023-08-27T11:28:12.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c1b7b6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c140ca4e0 con 0x7f4c14143ba0 2023-08-27T11:28:12.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c19552700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c1406d5b0 0x7f4c141332a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c19552700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c1406d5b0 0x7f4c141332a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35974/0 (socket says 172.21.15.112:35974) 2023-08-27T11:28:12.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c19552700 1 -- 172.21.15.112:0/349903531 learned_addr learned my addr 172.21.15.112:0/349903531 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:12.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c18d51700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c141437c0 0x7f4c14159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c19d53700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c14143ba0 0x7f4c140708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c19552700 1 -- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c14143ba0 msgr2=0x7f4c140708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c19552700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c14143ba0 0x7f4c140708d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c19552700 1 -- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c141437c0 msgr2=0x7f4c14159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c19552700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c141437c0 0x7f4c14159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c19552700 1 -- 172.21.15.112:0/349903531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c10009550 con 0x7f4c1406d5b0 2023-08-27T11:28:12.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.778+0000 7f4c18d51700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c141437c0 0x7f4c14159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:12.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.779+0000 7f4c19552700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c1406d5b0 0x7f4c141332a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009130 tx=0x7f4c10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:12.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.779+0000 7f4c09ffb700 1 -- 172.21.15.112:0/349903531 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10003e30 con 0x7f4c1406d5b0 2023-08-27T11:28:12.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.779+0000 7f4c09ffb700 1 -- 172.21.15.112:0/349903531 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1001f070 con 0x7f4c1406d5b0 2023-08-27T11:28:12.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.779+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/349903531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c140ca7c0 con 0x7f4c1406d5b0 2023-08-27T11:28:12.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.779+0000 7f4c09ffb700 1 -- 172.21.15.112:0/349903531 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1000b8a0 con 0x7f4c1406d5b0 2023-08-27T11:28:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.779+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/349903531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c140cad80 con 0x7f4c1406d5b0 2023-08-27T11:28:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.780+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/349903531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c14067e90 con 0x7f4c1406d5b0 2023-08-27T11:28:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.780+0000 7f4c09ffb700 1 -- 172.21.15.112:0/349903531 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4c10021030 con 0x7f4c1406d5b0 2023-08-27T11:28:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.781+0000 7f4c09ffb700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bf806a500 0x7f4bf806c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.781+0000 7f4c09ffb700 1 -- 172.21.15.112:0/349903531 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f4c1008a0b0 con 0x7f4c1406d5b0 2023-08-27T11:28:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.781+0000 7f4c18d51700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bf806a500 0x7f4bf806c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.781+0000 7f4c18d51700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bf806a500 0x7f4bf806c9c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4c141f5bd0 tx=0x7f4c04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.785+0000 7f4c09ffb700 1 -- 172.21.15.112:0/349903531 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4c100554b0 con 0x7f4c1406d5b0 2023-08-27T11:28:12.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:12.973+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/349903531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4c14067e90 con 0x7f4c1406d5b0 2023-08-27T11:28:13.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.631+0000 7f4c09ffb700 1 -- 172.21.15.112:0/349903531 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v137) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4c10058ad0 con 0x7f4c1406d5b0 2023-08-27T11:28:13.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.635+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bf806a500 msgr2=0x7f4bf806c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.635+0000 7f4c1b7b6700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bf806a500 0x7f4bf806c9c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4c141f5bd0 tx=0x7f4c04008040 comp rx=0 tx=0).stop 2023-08-27T11:28:13.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.635+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/349903531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c1406d5b0 msgr2=0x7f4c141332a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.635+0000 7f4c1b7b6700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c1406d5b0 0x7f4c141332a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009130 tx=0x7f4c10004750 comp rx=0 tx=0).stop 2023-08-27T11:28:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.636+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/349903531 shutdown_connections 2023-08-27T11:28:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.636+0000 7f4c1b7b6700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bf806a500 0x7f4bf806c9c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.636+0000 7f4c1b7b6700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c14143ba0 0x7f4c140708d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.636+0000 7f4c1b7b6700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c1406d5b0 0x7f4c141332a0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.636+0000 7f4c1b7b6700 1 --2- 172.21.15.112:0/349903531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c141437c0 0x7f4c14159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.636+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/349903531 >> 172.21.15.112:0/349903531 conn(0x7f4c140b1e90 msgr2=0x7f4c140b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:13.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.637+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/349903531 shutdown_connections 2023-08-27T11:28:13.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.637+0000 7f4c1b7b6700 1 -- 172.21.15.112:0/349903531 wait complete. 2023-08-27T11:28:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2023-08-27T11:28:13.662 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:28:13.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.786+0000 7fe8e083a700 1 Processor -- start 2023-08-27T11:28:13.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.786+0000 7fe8e083a700 1 -- start start 2023-08-27T11:28:13.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.787+0000 7fe8e083a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8dc144b80 0x7fe8dc0b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:13.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.787+0000 7fe8e083a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe8dc06c6e0 0x7fe8dc0cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:13.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.787+0000 7fe8e083a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8dc0cef20 con 0x7fe8dc0b3de0 2023-08-27T11:28:13.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.787+0000 7fe8e083a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc0b4b40 con 0x7fe8dc06c6e0 2023-08-27T11:28:13.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.787+0000 7fe8e083a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0b42a0 con 0x7fe8dc144b80 2023-08-27T11:28:13.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.787+0000 7fe8dad9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe8dc06c6e0 0x7fe8dc0cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:13.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.787+0000 7fe8d9d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe8dc0b3de0 0x7fe8dc06bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58596/0 (socket says 172.21.15.112:58596) 2023-08-27T11:28:13.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.787+0000 7fe8d9d9b700 1 -- 172.21.15.112:0/3171505882 learned_addr learned my addr 172.21.15.112:0/3171505882 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:13.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.788+0000 7fe8dad9d700 1 -- 172.21.15.112:0/3171505882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8dc144b80 msgr2=0x7fe8dc0b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:13.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.788+0000 7fe8dad9d700 1 --2- 172.21.15.112:0/3171505882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8dc144b80 0x7fe8dc0b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.788+0000 7fe8dad9d700 1 -- 172.21.15.112:0/3171505882 >> v1:172.21.15.112:6789/0 conn(0x7fe8dc0b3de0 legacy=0x7fe8dc06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.788+0000 7fe8dad9d700 1 -- 172.21.15.112:0/3171505882 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc0b4580 con 0x7fe8dc06c6e0 2023-08-27T11:28:13.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.788+0000 7fe8d959a700 1 -- 172.21.15.112:0/3171505882 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762874373 0 0) 0x7fe8dc0cef20 con 0x7fe8dc0b3de0 2023-08-27T11:28:13.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.788+0000 7fe8dad9d700 1 --2- 172.21.15.112:0/3171505882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8dc06c6e0 0x7fe8dc0cf3b0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009d90 tx=0x7fe8d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9385a18b064d360b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:13.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.788+0000 7fe8d959a700 1 -- 172.21.15.112:0/3171505882 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400e070 con 0x7fe8dc06c6e0 2023-08-27T11:28:13.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.788+0000 7fe8d959a700 1 -- 172.21.15.112:0/3171505882 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d400bbd0 con 0x7fe8dc06c6e0 2023-08-27T11:28:13.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.788+0000 7fe8d959a700 1 -- 172.21.15.112:0/3171505882 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4012430 con 0x7fe8dc06c6e0 2023-08-27T11:28:13.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.788+0000 7fe8e083a700 1 -- 172.21.15.112:0/3171505882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8dc06c6e0 msgr2=0x7fe8dc0cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.788+0000 7fe8e083a700 1 --2- 172.21.15.112:0/3171505882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8dc06c6e0 0x7fe8dc0cf3b0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009d90 tx=0x7fe8d400b330 comp rx=0 tx=0).stop 2023-08-27T11:28:13.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.789+0000 7fe8e083a700 1 -- 172.21.15.112:0/3171505882 shutdown_connections 2023-08-27T11:28:13.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.789+0000 7fe8e083a700 1 --2- 172.21.15.112:0/3171505882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8dc06c6e0 0x7fe8dc0cf3b0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.789+0000 7fe8e083a700 1 --2- 172.21.15.112:0/3171505882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8dc144b80 0x7fe8dc0b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.789+0000 7fe8e083a700 1 -- 172.21.15.112:0/3171505882 >> 172.21.15.112:0/3171505882 conn(0x7fe8dc0cdc50 msgr2=0x7fe8dc0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:13.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.789+0000 7fe8e083a700 1 -- 172.21.15.112:0/3171505882 shutdown_connections 2023-08-27T11:28:13.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.789+0000 7fe8e083a700 1 -- 172.21.15.112:0/3171505882 wait complete. 2023-08-27T11:28:13.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.790+0000 7fe8e083a700 1 Processor -- start 2023-08-27T11:28:13.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.790+0000 7fe8e083a700 1 -- start start 2023-08-27T11:28:13.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.791+0000 7fe8e083a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8dc06c6e0 0x7fe8dc143190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:13.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.791+0000 7fe8e083a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8dc0b3de0 0x7fe8dc0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:13.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.791+0000 7fe8e083a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8dc144b80 0x7fe8dc159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:13.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.791+0000 7fe8e083a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0c9ed0 con 0x7fe8dc144b80 2023-08-27T11:28:13.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.791+0000 7fe8e083a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc158ac0 con 0x7fe8dc0b3de0 2023-08-27T11:28:13.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.791+0000 7fe8e083a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc158da0 con 0x7fe8dc06c6e0 2023-08-27T11:28:13.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.791+0000 7fe8dad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8dc144b80 0x7fe8dc159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:13.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.791+0000 7fe8dad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8dc144b80 0x7fe8dc159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35990/0 (socket says 172.21.15.112:35990) 2023-08-27T11:28:13.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.791+0000 7fe8dad9d700 1 -- 172.21.15.112:0/558376211 learned_addr learned my addr 172.21.15.112:0/558376211 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:13.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.791+0000 7fe8da59c700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8dc06c6e0 0x7fe8dc143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:13.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.791+0000 7fe8d9d9b700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8dc0b3de0 0x7fe8dc0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:13.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.792+0000 7fe8dad9d700 1 -- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8dc06c6e0 msgr2=0x7fe8dc143190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.792+0000 7fe8dad9d700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8dc06c6e0 0x7fe8dc143190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.792+0000 7fe8dad9d700 1 -- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8dc0b3de0 msgr2=0x7fe8dc0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.792+0000 7fe8dad9d700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8dc0b3de0 0x7fe8dc0cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.792+0000 7fe8dad9d700 1 -- 172.21.15.112:0/558376211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d4009a70 con 0x7fe8dc144b80 2023-08-27T11:28:13.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.792+0000 7fe8da59c700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8dc06c6e0 0x7fe8dc143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:13.803 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.792+0000 7fe8d9d9b700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8dc0b3de0 0x7fe8dc0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:13.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.792+0000 7fe8dad9d700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8dc144b80 0x7fe8dc159b60 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4013000 tx=0x7fe8d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:13.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.792+0000 7fe8caffd700 1 -- 172.21.15.112:0/558376211 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4012430 con 0x7fe8dc144b80 2023-08-27T11:28:13.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.792+0000 7fe8caffd700 1 -- 172.21.15.112:0/558376211 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4020070 con 0x7fe8dc144b80 2023-08-27T11:28:13.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.792+0000 7fe8e083a700 1 -- 172.21.15.112:0/558376211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc159080 con 0x7fe8dc144b80 2023-08-27T11:28:13.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.792+0000 7fe8caffd700 1 -- 172.21.15.112:0/558376211 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4004280 con 0x7fe8dc144b80 2023-08-27T11:28:13.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.792+0000 7fe8e083a700 1 -- 172.21.15.112:0/558376211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc159640 con 0x7fe8dc144b80 2023-08-27T11:28:13.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.793+0000 7fe8e083a700 1 -- 172.21.15.112:0/558376211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8dc067a20 con 0x7fe8dc144b80 2023-08-27T11:28:13.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.794+0000 7fe8caffd700 1 -- 172.21.15.112:0/558376211 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe8d40043e0 con 0x7fe8dc144b80 2023-08-27T11:28:13.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.794+0000 7fe8caffd700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe8bc06a4d0 0x7fe8bc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:13.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.794+0000 7fe8caffd700 1 -- 172.21.15.112:0/558376211 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fe8d4022070 con 0x7fe8dc144b80 2023-08-27T11:28:13.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.794+0000 7fe8da59c700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe8bc06a4d0 0x7fe8bc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:13.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.795+0000 7fe8da59c700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe8bc06a4d0 0x7fe8bc06c990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc006fd0 tx=0x7fe8cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:13.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:13.798+0000 7fe8caffd700 1 -- 172.21.15.112:0/558376211 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe8d4056390 con 0x7fe8dc144b80 2023-08-27T11:28:14.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.030+0000 7fe8e083a700 1 -- 172.21.15.112:0/558376211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe8dc0b0b70 con 0x7fe8dc144b80 2023-08-27T11:28:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.634+0000 7fe8caffd700 1 -- 172.21.15.112:0/558376211 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v90) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe8d402c0c0 con 0x7fe8dc144b80 2023-08-27T11:28:14.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.640+0000 7fe8e083a700 1 -- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe8bc06a4d0 msgr2=0x7fe8bc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:14.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.640+0000 7fe8e083a700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe8bc06a4d0 0x7fe8bc06c990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc006fd0 tx=0x7fe8cc008040 comp rx=0 tx=0).stop 2023-08-27T11:28:14.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.640+0000 7fe8e083a700 1 -- 172.21.15.112:0/558376211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8dc144b80 msgr2=0x7fe8dc159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:14.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.640+0000 7fe8e083a700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8dc144b80 0x7fe8dc159b60 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4013000 tx=0x7fe8d4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:14.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.643+0000 7fe8e083a700 1 -- 172.21.15.112:0/558376211 shutdown_connections 2023-08-27T11:28:14.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.643+0000 7fe8e083a700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe8bc06a4d0 0x7fe8bc06c990 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:14.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.643+0000 7fe8e083a700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8dc144b80 0x7fe8dc159b60 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:14.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.643+0000 7fe8e083a700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8dc06c6e0 0x7fe8dc143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:14.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.643+0000 7fe8e083a700 1 --2- 172.21.15.112:0/558376211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8dc0b3de0 0x7fe8dc0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:14.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.643+0000 7fe8e083a700 1 -- 172.21.15.112:0/558376211 >> 172.21.15.112:0/558376211 conn(0x7fe8dc0cdc50 msgr2=0x7fe8dc143d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:14.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.646+0000 7fe8e083a700 1 -- 172.21.15.112:0/558376211 shutdown_connections 2023-08-27T11:28:14.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.647+0000 7fe8e083a700 1 -- 172.21.15.112:0/558376211 wait complete. 2023-08-27T11:28:14.658 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 25 and data pool 26 2023-08-27T11:28:14.673 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:14.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.793+0000 7f6899cf6700 1 Processor -- start 2023-08-27T11:28:14.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.793+0000 7f6899cf6700 1 -- start start 2023-08-27T11:28:14.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.794+0000 7f6899cf6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f689406b230 0x7f68940ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:14.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.794+0000 7f6899cf6700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689415eb60 con 0x7f6894151060 2023-08-27T11:28:14.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.794+0000 7f6899cf6700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68940b02f0 con 0x7f689406adb0 2023-08-27T11:28:14.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.794+0000 7f6899cf6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68940b05d0 con 0x7f689406b230 2023-08-27T11:28:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.794+0000 7f6893fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f689406b230 0x7f68940ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.794+0000 7f68937fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6894151060 0x7f689415e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58608/0 (socket says 172.21.15.112:58608) 2023-08-27T11:28:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.794+0000 7f68937fe700 1 -- 172.21.15.112:0/2505201708 learned_addr learned my addr 172.21.15.112:0/2505201708 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.794+0000 7f6893fff700 1 -- 172.21.15.112:0/2505201708 >> v1:172.21.15.138:6789/0 conn(0x7f689406adb0 legacy=0x7f68940cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.794+0000 7f6893fff700 1 -- 172.21.15.112:0/2505201708 >> v1:172.21.15.112:6789/0 conn(0x7f6894151060 legacy=0x7f689415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.794+0000 7f6893fff700 1 -- 172.21.15.112:0/2505201708 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68940b08b0 con 0x7f689406b230 2023-08-27T11:28:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.794+0000 7f68927fc700 1 -- 172.21.15.112:0/2505201708 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203625001 0 0) 0x7f689415eb60 con 0x7f6894151060 2023-08-27T11:28:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.795+0000 7f6893fff700 1 --2- 172.21.15.112:0/2505201708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f689406b230 0x7f68940ce2e0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f688c009d90 tx=0x7f688c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fe15041e35fa9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.795+0000 7f68927fc700 1 -- 172.21.15.112:0/2505201708 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c00e070 con 0x7f689406b230 2023-08-27T11:28:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.795+0000 7f68927fc700 1 -- 172.21.15.112:0/2505201708 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688c00bb90 con 0x7f689406b230 2023-08-27T11:28:14.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.795+0000 7f68927fc700 1 -- 172.21.15.112:0/2505201708 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c003a20 con 0x7f689406b230 2023-08-27T11:28:14.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.795+0000 7f6899cf6700 1 -- 172.21.15.112:0/2505201708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f689406b230 msgr2=0x7f68940ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:14.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.795+0000 7f6899cf6700 1 --2- 172.21.15.112:0/2505201708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f689406b230 0x7f68940ce2e0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f688c009d90 tx=0x7f688c00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:14.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.795+0000 7f6899cf6700 1 -- 172.21.15.112:0/2505201708 shutdown_connections 2023-08-27T11:28:14.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.795+0000 7f6899cf6700 1 --2- 172.21.15.112:0/2505201708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f689406b230 0x7f68940ce2e0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:14.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.795+0000 7f6899cf6700 1 -- 172.21.15.112:0/2505201708 >> 172.21.15.112:0/2505201708 conn(0x7f68940d0000 msgr2=0x7f68940d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:14.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.795+0000 7f6899cf6700 1 -- 172.21.15.112:0/2505201708 shutdown_connections 2023-08-27T11:28:14.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.795+0000 7f6899cf6700 1 -- 172.21.15.112:0/2505201708 wait complete. 2023-08-27T11:28:14.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.796+0000 7f6899cf6700 1 Processor -- start 2023-08-27T11:28:14.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.797+0000 7f6899cf6700 1 -- start start 2023-08-27T11:28:14.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.797+0000 7f6899cf6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f689406adb0 0x7f689414db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:14.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.797+0000 7f6899cf6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f689406b230 0x7f6894159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:14.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.797+0000 7f6899cf6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6894151060 0x7f68940702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:14.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.797+0000 7f6899cf6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689406f600 con 0x7f689406b230 2023-08-27T11:28:14.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.797+0000 7f6899cf6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689414e0d0 con 0x7f689406adb0 2023-08-27T11:28:14.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.797+0000 7f6899cf6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689406e200 con 0x7f6894151060 2023-08-27T11:28:14.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.797+0000 7f6892ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f689406b230 0x7f6894159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:14.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f6892ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f689406b230 0x7f6894159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36002/0 (socket says 172.21.15.112:36002) 2023-08-27T11:28:14.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f6892ffd700 1 -- 172.21.15.112:0/388338768 learned_addr learned my addr 172.21.15.112:0/388338768 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:14.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f68937fe700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f689406adb0 0x7f689414db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:14.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f6893fff700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6894151060 0x7f68940702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:14.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f6892ffd700 1 -- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6894151060 msgr2=0x7f68940702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:14.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f6892ffd700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6894151060 0x7f68940702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:14.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f6892ffd700 1 -- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f689406adb0 msgr2=0x7f689414db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:14.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f6892ffd700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f689406adb0 0x7f689414db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:14.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f6892ffd700 1 -- 172.21.15.112:0/388338768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c009a70 con 0x7f689406b230 2023-08-27T11:28:14.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f68937fe700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f689406adb0 0x7f689414db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:14.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f6892ffd700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f689406b230 0x7f6894159b30 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f688800b470 tx=0x7f688800b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f687bfff700 1 -- 172.21.15.112:0/388338768 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888011830 con 0x7f689406b230 2023-08-27T11:28:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f687bfff700 1 -- 172.21.15.112:0/388338768 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6888011990 con 0x7f689406b230 2023-08-27T11:28:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f6899cf6700 1 -- 172.21.15.112:0/388338768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689406f810 con 0x7f689406b230 2023-08-27T11:28:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f687bfff700 1 -- 172.21.15.112:0/388338768 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888011d40 con 0x7f689406b230 2023-08-27T11:28:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.798+0000 7f6899cf6700 1 -- 172.21.15.112:0/388338768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689406fdd0 con 0x7f689406b230 2023-08-27T11:28:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.799+0000 7f6899cf6700 1 -- 172.21.15.112:0/388338768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6894067a20 con 0x7f689406b230 2023-08-27T11:28:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.800+0000 7f687bfff700 1 -- 172.21.15.112:0/388338768 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f688801a400 con 0x7f689406b230 2023-08-27T11:28:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.800+0000 7f687bfff700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f688006a4d0 0x7f688006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.800+0000 7f687bfff700 1 -- 172.21.15.112:0/388338768 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6888088da0 con 0x7f689406b230 2023-08-27T11:28:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.800+0000 7f68937fe700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f688006a4d0 0x7f688006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:14.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.801+0000 7f68937fe700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f688006a4d0 0x7f688006c990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6884000f60 tx=0x7f6884007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:14.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.804+0000 7f687bfff700 1 -- 172.21.15.112:0/388338768 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6888053e20 con 0x7f689406b230 2023-08-27T11:28:14.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.995+0000 7f6899cf6700 1 -- 172.21.15.112:0/388338768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68940cddc0 con 0x7f689406b230 2023-08-27T11:28:14.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.996+0000 7f687bfff700 1 -- 172.21.15.112:0/388338768 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18276 (secure 0 0 0) 0x7f6888015070 con 0x7f689406b230 2023-08-27T11:28:14.998 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:14.998 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":138,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:14.031762+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:12.176044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:12.992996+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:15.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.999+0000 7f6899cf6700 1 -- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f688006a4d0 msgr2=0x7f688006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:15.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.999+0000 7f6899cf6700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f688006a4d0 0x7f688006c990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6884000f60 tx=0x7f6884007d30 comp rx=0 tx=0).stop 2023-08-27T11:28:15.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.999+0000 7f6899cf6700 1 -- 172.21.15.112:0/388338768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f689406b230 msgr2=0x7f6894159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:15.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.999+0000 7f6899cf6700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f689406b230 0x7f6894159b30 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f688800b470 tx=0x7f688800b830 comp rx=0 tx=0).stop 2023-08-27T11:28:15.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.999+0000 7f6899cf6700 1 -- 172.21.15.112:0/388338768 shutdown_connections 2023-08-27T11:28:15.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.999+0000 7f6899cf6700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f688006a4d0 0x7f688006c990 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.999+0000 7f6899cf6700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6894151060 0x7f68940702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.999+0000 7f6899cf6700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f689406adb0 0x7f689414db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.999+0000 7f6899cf6700 1 --2- 172.21.15.112:0/388338768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f689406b230 0x7f6894159b30 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.999+0000 7f6899cf6700 1 -- 172.21.15.112:0/388338768 >> 172.21.15.112:0/388338768 conn(0x7f68940d0000 msgr2=0x7f689414ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.999+0000 7f6899cf6700 1 -- 172.21.15.112:0/388338768 shutdown_connections 2023-08-27T11:28:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:14.999+0000 7f6899cf6700 1 -- 172.21.15.112:0/388338768 wait complete. 2023-08-27T11:28:15.013 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:15.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.133+0000 7f6281e13700 1 Processor -- start 2023-08-27T11:28:15.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.133+0000 7f6281e13700 1 -- start start 2023-08-27T11:28:15.137 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.133+0000 7f6281e13700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f627c144570 0x7f627c15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:15.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.133+0000 7f6281e13700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f627c0b3e50 0x7f627c15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:15.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.133+0000 7f6281e13700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627c0b5ae0 con 0x7f627c15f8a0 2023-08-27T11:28:15.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.133+0000 7f6281e13700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627c144950 con 0x7f627c0b3e50 2023-08-27T11:28:15.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.133+0000 7f6281e13700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627c0b45f0 con 0x7f627c144570 2023-08-27T11:28:15.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.134+0000 7f627affd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f627c0b3e50 0x7f627c15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:15.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.134+0000 7f627b7fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f627c144570 0x7f627c15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:15.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.134+0000 7f627bfff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f627c15f8a0 0x7f627c0ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:37918/0 (socket says 172.21.15.112:37918) 2023-08-27T11:28:15.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.134+0000 7f627bfff700 1 -- 172.21.15.112:0/181309001 learned_addr learned my addr 172.21.15.112:0/181309001 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:15.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.134+0000 7f627b7fe700 1 -- 172.21.15.112:0/181309001 >> v2:172.21.15.138:3300/0 conn(0x7f627c0b3e50 msgr2=0x7f627c15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:15.139 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.134+0000 7f627b7fe700 1 --2- 172.21.15.112:0/181309001 >> v2:172.21.15.138:3300/0 conn(0x7f627c0b3e50 0x7f627c15f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.134+0000 7f627b7fe700 1 -- 172.21.15.112:0/181309001 >> v1:172.21.15.138:6789/0 conn(0x7f627c15f8a0 legacy=0x7f627c0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:15.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.134+0000 7f627b7fe700 1 -- 172.21.15.112:0/181309001 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c0b4310 con 0x7f627c144570 2023-08-27T11:28:15.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.134+0000 7f627affd700 1 --2- 172.21.15.112:0/181309001 >> v2:172.21.15.138:3300/0 conn(0x7f627c0b3e50 0x7f627c15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:15.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.134+0000 7f627b7fe700 1 --2- 172.21.15.112:0/181309001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f627c144570 0x7f627c15ee20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f626c009840 tx=0x7f626c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef945acdab5d2fc9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:15.140 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.134+0000 7f627a7fc700 1 -- 172.21.15.112:0/181309001 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c011830 con 0x7f627c144570 2023-08-27T11:28:15.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.134+0000 7f627a7fc700 1 -- 172.21.15.112:0/181309001 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626c011990 con 0x7f627c144570 2023-08-27T11:28:15.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.134+0000 7f627a7fc700 1 -- 172.21.15.112:0/181309001 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c011d20 con 0x7f627c144570 2023-08-27T11:28:15.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.135+0000 7f6281e13700 1 -- 172.21.15.112:0/181309001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f627c144570 msgr2=0x7f627c15ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:15.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.135+0000 7f6281e13700 1 --2- 172.21.15.112:0/181309001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f627c144570 0x7f627c15ee20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f626c009840 tx=0x7f626c009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:15.141 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.135+0000 7f6281e13700 1 -- 172.21.15.112:0/181309001 shutdown_connections 2023-08-27T11:28:15.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.135+0000 7f6281e13700 1 --2- 172.21.15.112:0/181309001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f627c144570 0x7f627c15ee20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.135+0000 7f6281e13700 1 --2- 172.21.15.112:0/181309001 >> v2:172.21.15.138:3300/0 conn(0x7f627c0b3e50 0x7f627c15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.135+0000 7f6281e13700 1 -- 172.21.15.112:0/181309001 >> 172.21.15.112:0/181309001 conn(0x7f627c06f970 msgr2=0x7f627c06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:15.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.135+0000 7f6281e13700 1 -- 172.21.15.112:0/181309001 shutdown_connections 2023-08-27T11:28:15.142 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.135+0000 7f6281e13700 1 -- 172.21.15.112:0/181309001 wait complete. 2023-08-27T11:28:15.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.136+0000 7f6281e13700 1 Processor -- start 2023-08-27T11:28:15.145 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.136+0000 7f6281e13700 1 -- start start 2023-08-27T11:28:15.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.137+0000 7f6281e13700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f627c0b3e50 0x7f627c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:15.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.137+0000 7f6281e13700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f627c144570 0x7f627c0aa860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:15.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.137+0000 7f6281e13700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f627c15f8a0 0x7f627c0cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:15.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.137+0000 7f6281e13700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c070f60 con 0x7f627c15f8a0 2023-08-27T11:28:15.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.137+0000 7f6281e13700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c0aada0 con 0x7f627c0b3e50 2023-08-27T11:28:15.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.137+0000 7f6281e13700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627c072140 con 0x7f627c144570 2023-08-27T11:28:15.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.137+0000 7f627bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f627c15f8a0 0x7f627c0cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:15.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.137+0000 7f627bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f627c15f8a0 0x7f627c0cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36018/0 (socket says 172.21.15.112:36018) 2023-08-27T11:28:15.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.137+0000 7f627bfff700 1 -- 172.21.15.112:0/1687546537 learned_addr learned my addr 172.21.15.112:0/1687546537 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:15.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.137+0000 7f627affd700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f627c144570 0x7f627c0aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:15.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.137+0000 7f627b7fe700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f627c0b3e50 0x7f627c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:15.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.138+0000 7f627bfff700 1 -- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f627c144570 msgr2=0x7f627c0aa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:15.149 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.138+0000 7f627bfff700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f627c144570 0x7f627c0aa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.138+0000 7f627bfff700 1 -- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f627c0b3e50 msgr2=0x7f627c071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.138+0000 7f627bfff700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f627c0b3e50 0x7f627c071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.138+0000 7f627bfff700 1 -- 172.21.15.112:0/1687546537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62700094a0 con 0x7f627c15f8a0 2023-08-27T11:28:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.138+0000 7f627bfff700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f627c15f8a0 0x7f627c0cc380 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f627400ca80 tx=0x7f627400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.138+0000 7f6263fff700 1 -- 172.21.15.112:0/1687546537 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627400a9e0 con 0x7f627c15f8a0 2023-08-27T11:28:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.138+0000 7f6263fff700 1 -- 172.21.15.112:0/1687546537 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627400ab40 con 0x7f627c15f8a0 2023-08-27T11:28:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.138+0000 7f6281e13700 1 -- 172.21.15.112:0/1687546537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626c0094a0 con 0x7f627c15f8a0 2023-08-27T11:28:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.138+0000 7f6263fff700 1 -- 172.21.15.112:0/1687546537 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6274012400 con 0x7f627c15f8a0 2023-08-27T11:28:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.138+0000 7f6281e13700 1 -- 172.21.15.112:0/1687546537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c1f3870 con 0x7f627c15f8a0 2023-08-27T11:28:15.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.139+0000 7f6281e13700 1 -- 172.21.15.112:0/1687546537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627c0710a0 con 0x7f627c15f8a0 2023-08-27T11:28:15.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.139+0000 7f6263fff700 1 -- 172.21.15.112:0/1687546537 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f627401b3b0 con 0x7f627c15f8a0 2023-08-27T11:28:15.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.140+0000 7f6263fff700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f626806a4d0 0x7f626806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:15.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.140+0000 7f6263fff700 1 -- 172.21.15.112:0/1687546537 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6274088540 con 0x7f627c15f8a0 2023-08-27T11:28:15.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.140+0000 7f627b7fe700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f626806a4d0 0x7f626806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:15.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.140+0000 7f627b7fe700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f626806a4d0 0x7f626806c990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f626c000c00 tx=0x7f626c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:15.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.144+0000 7f6263fff700 1 -- 172.21.15.112:0/1687546537 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f627c0710a0 con 0x7f627c15f8a0 2023-08-27T11:28:15.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.335+0000 7f6281e13700 1 -- 172.21.15.112:0/1687546537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f627c0b5ae0 con 0x7f627c15f8a0 2023-08-27T11:28:15.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.336+0000 7f6263fff700 1 -- 172.21.15.112:0/1687546537 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18276 (secure 0 0 0) 0x7f627c0b5ae0 con 0x7f627c15f8a0 2023-08-27T11:28:15.338 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:15.338 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":138,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:14.031762+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:12.176044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:12.992996+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.338+0000 7f6281e13700 1 -- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f626806a4d0 msgr2=0x7f626806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.338+0000 7f6281e13700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f626806a4d0 0x7f626806c990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f626c000c00 tx=0x7f626c008030 comp rx=0 tx=0).stop 2023-08-27T11:28:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.338+0000 7f6281e13700 1 -- 172.21.15.112:0/1687546537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f627c15f8a0 msgr2=0x7f627c0cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.338+0000 7f6281e13700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f627c15f8a0 0x7f627c0cc380 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f627400ca80 tx=0x7f627400cd90 comp rx=0 tx=0).stop 2023-08-27T11:28:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.338+0000 7f6281e13700 1 -- 172.21.15.112:0/1687546537 shutdown_connections 2023-08-27T11:28:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.338+0000 7f6281e13700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f626806a4d0 0x7f626806c990 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.338+0000 7f6281e13700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f627c15f8a0 0x7f627c0cc380 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.338+0000 7f6281e13700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f627c0b3e50 0x7f627c071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.338+0000 7f6281e13700 1 --2- 172.21.15.112:0/1687546537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f627c144570 0x7f627c0aa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.338+0000 7f6281e13700 1 -- 172.21.15.112:0/1687546537 >> 172.21.15.112:0/1687546537 conn(0x7f627c06f970 msgr2=0x7f627c15d740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:15.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.338+0000 7f6281e13700 1 -- 172.21.15.112:0/1687546537 shutdown_connections 2023-08-27T11:28:15.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.338+0000 7f6281e13700 1 -- 172.21.15.112:0/1687546537 wait complete. 2023-08-27T11:28:15.352 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:28:15.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.474+0000 7fb2fb7fd700 1 Processor -- start 2023-08-27T11:28:15.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.474+0000 7fb2fb7fd700 1 -- start start 2023-08-27T11:28:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2fb7fd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb2f4158ed0 0x7fb2f415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2fb7fd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb2f4072000 0x7fb2f406ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2fb7fd700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f406e740 con 0x7fb2f406e2f0 2023-08-27T11:28:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2fb7fd700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f415ca50 con 0x7fb2f4158ed0 2023-08-27T11:28:15.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2fb7fd700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f4072490 con 0x7fb2f4072000 2023-08-27T11:28:15.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2f9599700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb2f4158ed0 0x7fb2f415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:15.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2f9599700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb2f4158ed0 0x7fb2f415b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36022/0 (socket says 172.21.15.112:36022) 2023-08-27T11:28:15.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2f9599700 1 -- 172.21.15.112:0/457468014 learned_addr learned my addr 172.21.15.112:0/457468014 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:15.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2f8d98700 1 --2- 172.21.15.112:0/457468014 >> v2:172.21.15.138:3300/0 conn(0x7fb2f4072000 0x7fb2f406ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2f9599700 1 -- 172.21.15.112:0/457468014 >> v2:172.21.15.138:3300/0 conn(0x7fb2f4072000 msgr2=0x7fb2f406ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2f9599700 1 --2- 172.21.15.112:0/457468014 >> v2:172.21.15.138:3300/0 conn(0x7fb2f4072000 0x7fb2f406ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2f9599700 1 -- 172.21.15.112:0/457468014 >> v1:172.21.15.138:6789/0 conn(0x7fb2f406e2f0 legacy=0x7fb2f415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2f9599700 1 -- 172.21.15.112:0/457468014 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f415be20 con 0x7fb2f4158ed0 2023-08-27T11:28:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2f9599700 1 --2- 172.21.15.112:0/457468014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f4158ed0 0x7fb2f415b7f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4068590 tx=0x7fb2f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfcb5c23c0fd49b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:15.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2ebfff700 1 -- 172.21.15.112:0/457468014 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000b870 con 0x7fb2f4158ed0 2023-08-27T11:28:15.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2ebfff700 1 -- 172.21.15.112:0/457468014 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f000b9d0 con 0x7fb2f4158ed0 2023-08-27T11:28:15.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.475+0000 7fb2ebfff700 1 -- 172.21.15.112:0/457468014 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000bd60 con 0x7fb2f4158ed0 2023-08-27T11:28:15.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.476+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/457468014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f4158ed0 msgr2=0x7fb2f415b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:15.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.476+0000 7fb2fb7fd700 1 --2- 172.21.15.112:0/457468014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f4158ed0 0x7fb2f415b7f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4068590 tx=0x7fb2f0009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.476+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/457468014 shutdown_connections 2023-08-27T11:28:15.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.476+0000 7fb2fb7fd700 1 --2- 172.21.15.112:0/457468014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f4158ed0 0x7fb2f415b7f0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.476+0000 7fb2fb7fd700 1 --2- 172.21.15.112:0/457468014 >> v2:172.21.15.138:3300/0 conn(0x7fb2f4072000 0x7fb2f406ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.476+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/457468014 >> 172.21.15.112:0/457468014 conn(0x7fb2f40cc240 msgr2=0x7fb2f40cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:15.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.476+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/457468014 shutdown_connections 2023-08-27T11:28:15.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.476+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/457468014 wait complete. 2023-08-27T11:28:15.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.478+0000 7fb2fb7fd700 1 Processor -- start 2023-08-27T11:28:15.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.478+0000 7fb2fb7fd700 1 -- start start 2023-08-27T11:28:15.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.478+0000 7fb2fb7fd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f406e2f0 0x7fb2f41e53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:15.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2fb7fd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f4072000 0x7fb2f41e2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:15.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2fb7fd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2f4158ed0 0x7fb2f4157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:15.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2fb7fd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f41e3980 con 0x7fb2f4072000 2023-08-27T11:28:15.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2fb7fd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f41e5920 con 0x7fb2f406e2f0 2023-08-27T11:28:15.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2fb7fd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f41e2a80 con 0x7fb2f4158ed0 2023-08-27T11:28:15.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2f8d98700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f4072000 0x7fb2f41e2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:15.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2f8d98700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f4072000 0x7fb2f41e2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36030/0 (socket says 172.21.15.112:36030) 2023-08-27T11:28:15.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2f8d98700 1 -- 172.21.15.112:0/1146579016 learned_addr learned my addr 172.21.15.112:0/1146579016 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:15.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2f9d9a700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2f4158ed0 0x7fb2f4157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2f9599700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f406e2f0 0x7fb2f41e53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2f8d98700 1 -- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2f4158ed0 msgr2=0x7fb2f4157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2f8d98700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2f4158ed0 0x7fb2f4157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2f8d98700 1 -- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f406e2f0 msgr2=0x7fb2f41e53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2f8d98700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f406e2f0 0x7fb2f41e53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2f8d98700 1 -- 172.21.15.112:0/1146579016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f0009550 con 0x7fb2f4072000 2023-08-27T11:28:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2f8d98700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f4072000 0x7fb2f41e2540 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e840 tx=0x7fb2e400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2e9ffb700 1 -- 172.21.15.112:0/1146579016 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4013610 con 0x7fb2f4072000 2023-08-27T11:28:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2e9ffb700 1 -- 172.21.15.112:0/1146579016 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e4013770 con 0x7fb2f4072000 2023-08-27T11:28:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/1146579016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f41e2d00 con 0x7fb2f4072000 2023-08-27T11:28:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2e9ffb700 1 -- 172.21.15.112:0/1146579016 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4013b20 con 0x7fb2f4072000 2023-08-27T11:28:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.479+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/1146579016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f41e32c0 con 0x7fb2f4072000 2023-08-27T11:28:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.480+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/1146579016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f4067e90 con 0x7fb2f4072000 2023-08-27T11:28:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.481+0000 7fb2e9ffb700 1 -- 172.21.15.112:0/1146579016 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fb2e4013c80 con 0x7fb2f4072000 2023-08-27T11:28:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.481+0000 7fb2e9ffb700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb2d806a4d0 0x7fb2d806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.481+0000 7fb2e9ffb700 1 -- 172.21.15.112:0/1146579016 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2e4088a20 con 0x7fb2f4072000 2023-08-27T11:28:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.482+0000 7fb2f9599700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb2d806a4d0 0x7fb2d806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.482+0000 7fb2f9599700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb2d806a4d0 0x7fb2d806c990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0014040 tx=0x7fb2f0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.486+0000 7fb2e9ffb700 1 -- 172.21.15.112:0/1146579016 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb2e4053a70 con 0x7fb2f4072000 2023-08-27T11:28:15.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:15.721+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/1146579016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb2f4067a20 con 0x7fb2f4072000 2023-08-27T11:28:16.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.666+0000 7fb2e9ffb700 1 -- 172.21.15.112:0/1146579016 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v93) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb2e4057090 con 0x7fb2f4072000 2023-08-27T11:28:16.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.669+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb2d806a4d0 msgr2=0x7fb2d806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:16.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.669+0000 7fb2fb7fd700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb2d806a4d0 0x7fb2d806c990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0014040 tx=0x7fb2f0008030 comp rx=0 tx=0).stop 2023-08-27T11:28:16.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.669+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/1146579016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f4072000 msgr2=0x7fb2f41e2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:16.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.669+0000 7fb2fb7fd700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f4072000 0x7fb2f41e2540 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e840 tx=0x7fb2e400ec00 comp rx=0 tx=0).stop 2023-08-27T11:28:16.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.670+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/1146579016 shutdown_connections 2023-08-27T11:28:16.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.670+0000 7fb2fb7fd700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb2d806a4d0 0x7fb2d806c990 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.670+0000 7fb2fb7fd700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2f4158ed0 0x7fb2f4157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.670+0000 7fb2fb7fd700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f406e2f0 0x7fb2f41e53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.670+0000 7fb2fb7fd700 1 --2- 172.21.15.112:0/1146579016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f4072000 0x7fb2f41e2540 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.670+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/1146579016 >> 172.21.15.112:0/1146579016 conn(0x7fb2f40cc240 msgr2=0x7fb2f4140d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:16.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.670+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/1146579016 shutdown_connections 2023-08-27T11:28:16.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.670+0000 7fb2fb7fd700 1 -- 172.21.15.112:0/1146579016 wait complete. 2023-08-27T11:28:16.687 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:28:16.688 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:28:16.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.815+0000 7f196a9fb700 1 Processor -- start 2023-08-27T11:28:16.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.815+0000 7f196a9fb700 1 -- start start 2023-08-27T11:28:16.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.816+0000 7f196a9fb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1964158ed0 0x7f196415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:16.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.816+0000 7f196a9fb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1964072000 0x7f196406ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:16.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.816+0000 7f196a9fb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f196406e2f0 0x7f196415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:16.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.816+0000 7f196a9fb700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196406e780 con 0x7f1964158ed0 2023-08-27T11:28:16.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.816+0000 7f196a9fb700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196415ca50 con 0x7f1964072000 2023-08-27T11:28:16.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.816+0000 7f196a9fb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1964072490 con 0x7f196406e2f0 2023-08-27T11:28:16.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.816+0000 7f1963fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1964158ed0 0x7f196415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:16.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.816+0000 7f1963fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1964158ed0 0x7f196415b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36038/0 (socket says 172.21.15.112:36038) 2023-08-27T11:28:16.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.816+0000 7f1963fff700 1 -- 172.21.15.112:0/148711896 learned_addr learned my addr 172.21.15.112:0/148711896 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:16.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.816+0000 7f19637fe700 1 --2- 172.21.15.112:0/148711896 >> v2:172.21.15.138:3300/0 conn(0x7f1964072000 0x7f196406ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:16.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.816+0000 7f1968f98700 1 --2- 172.21.15.112:0/148711896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f196406e2f0 0x7f196415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:16.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f1963fff700 1 -- 172.21.15.112:0/148711896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f196406e2f0 msgr2=0x7f196415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:16.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f1963fff700 1 --2- 172.21.15.112:0/148711896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f196406e2f0 0x7f196415eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f1963fff700 1 -- 172.21.15.112:0/148711896 >> v2:172.21.15.138:3300/0 conn(0x7f1964072000 msgr2=0x7f196406ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:16.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f1963fff700 1 --2- 172.21.15.112:0/148711896 >> v2:172.21.15.138:3300/0 conn(0x7f1964072000 0x7f196406ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f1963fff700 1 -- 172.21.15.112:0/148711896 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196415be20 con 0x7f1964158ed0 2023-08-27T11:28:16.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f1968f98700 1 --2- 172.21.15.112:0/148711896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f196406e2f0 0x7f196415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:16.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f1963fff700 1 --2- 172.21.15.112:0/148711896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1964158ed0 0x7f196415b7f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f1964068590 tx=0x7f1958009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b92eee168d0955 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:16.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f1962ffd700 1 -- 172.21.15.112:0/148711896 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958011830 con 0x7f1964158ed0 2023-08-27T11:28:16.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f1962ffd700 1 -- 172.21.15.112:0/148711896 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1958011990 con 0x7f1964158ed0 2023-08-27T11:28:16.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f1962ffd700 1 -- 172.21.15.112:0/148711896 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958011d20 con 0x7f1964158ed0 2023-08-27T11:28:16.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f196a9fb700 1 -- 172.21.15.112:0/148711896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1964158ed0 msgr2=0x7f196415b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:16.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f196a9fb700 1 --2- 172.21.15.112:0/148711896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1964158ed0 0x7f196415b7f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f1964068590 tx=0x7f1958009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f196a9fb700 1 -- 172.21.15.112:0/148711896 shutdown_connections 2023-08-27T11:28:16.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f196a9fb700 1 --2- 172.21.15.112:0/148711896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f196406e2f0 0x7f196415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f196a9fb700 1 --2- 172.21.15.112:0/148711896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1964158ed0 0x7f196415b7f0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f196a9fb700 1 --2- 172.21.15.112:0/148711896 >> v2:172.21.15.138:3300/0 conn(0x7f1964072000 0x7f196406ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f196a9fb700 1 -- 172.21.15.112:0/148711896 >> 172.21.15.112:0/148711896 conn(0x7f19640cc240 msgr2=0x7f19640cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:16.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.817+0000 7f196a9fb700 1 -- 172.21.15.112:0/148711896 shutdown_connections 2023-08-27T11:28:16.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.818+0000 7f196a9fb700 1 -- 172.21.15.112:0/148711896 wait complete. 2023-08-27T11:28:16.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.819+0000 7f196a9fb700 1 Processor -- start 2023-08-27T11:28:16.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.819+0000 7f196a9fb700 1 -- start start 2023-08-27T11:28:16.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.819+0000 7f196a9fb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f196406e2f0 0x7f19641555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:16.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.819+0000 7f196a9fb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1964072000 0x7f19641e5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:16.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.819+0000 7f196a9fb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1964158ed0 0x7f19640a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:16.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.819+0000 7f196a9fb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19640aa890 con 0x7f196406e2f0 2023-08-27T11:28:16.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.819+0000 7f196a9fb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19641e4fe0 con 0x7f1964072000 2023-08-27T11:28:16.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f196a9fb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19641e4150 con 0x7f1964158ed0 2023-08-27T11:28:16.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f1963fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f196406e2f0 0x7f19641555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:16.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f1963fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f196406e2f0 0x7f19641555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36040/0 (socket says 172.21.15.112:36040) 2023-08-27T11:28:16.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f1963fff700 1 -- 172.21.15.112:0/672780133 learned_addr learned my addr 172.21.15.112:0/672780133 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:16.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f1968f98700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1964158ed0 0x7f19640a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:16.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f19637fe700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1964072000 0x7f19641e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:16.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f1963fff700 1 -- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1964158ed0 msgr2=0x7f19640a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:16.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f1963fff700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1964158ed0 0x7f19640a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f1963fff700 1 -- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1964072000 msgr2=0x7f19641e5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:16.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f1963fff700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1964072000 0x7f19641e5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f1963fff700 1 -- 172.21.15.112:0/672780133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1958009550 con 0x7f196406e2f0 2023-08-27T11:28:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f1963fff700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f196406e2f0 0x7f19641555d0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1958014040 tx=0x7f1958004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f1960ff9700 1 -- 172.21.15.112:0/672780133 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958004450 con 0x7f196406e2f0 2023-08-27T11:28:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f1960ff9700 1 -- 172.21.15.112:0/672780133 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195801e070 con 0x7f196406e2f0 2023-08-27T11:28:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f196a9fb700 1 -- 172.21.15.112:0/672780133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1964155b10 con 0x7f196406e2f0 2023-08-27T11:28:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f1960ff9700 1 -- 172.21.15.112:0/672780133 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19580038c0 con 0x7f196406e2f0 2023-08-27T11:28:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.820+0000 7f196a9fb700 1 -- 172.21.15.112:0/672780133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19641e46b0 con 0x7f196406e2f0 2023-08-27T11:28:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.821+0000 7f196a9fb700 1 -- 172.21.15.112:0/672780133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19641e52c0 con 0x7f196406e2f0 2023-08-27T11:28:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.822+0000 7f1960ff9700 1 -- 172.21.15.112:0/672780133 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f1958003a20 con 0x7f196406e2f0 2023-08-27T11:28:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.822+0000 7f1960ff9700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f195006a4d0 0x7f195006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.822+0000 7f1960ff9700 1 -- 172.21.15.112:0/672780133 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1958089780 con 0x7f196406e2f0 2023-08-27T11:28:16.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.822+0000 7f19637fe700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f195006a4d0 0x7f195006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:16.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.823+0000 7f19637fe700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f195006a4d0 0x7f195006c990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f19640b07f0 tx=0x7f1954009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:16.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:16.826+0000 7f1960ff9700 1 -- 172.21.15.112:0/672780133 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1958054b30 con 0x7f196406e2f0 2023-08-27T11:28:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.060+0000 7f196a9fb700 1 -- 172.21.15.112:0/672780133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1964067e90 con 0x7f196406e2f0 2023-08-27T11:28:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.668+0000 7f1960ff9700 1 -- 172.21.15.112:0/672780133 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v94) v1 ==== 122+0+0 (secure 0 0 0) 0x7f19580200c0 con 0x7f196406e2f0 2023-08-27T11:28:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.671+0000 7f196a9fb700 1 -- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f195006a4d0 msgr2=0x7f195006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.671+0000 7f196a9fb700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f195006a4d0 0x7f195006c990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f19640b07f0 tx=0x7f1954009210 comp rx=0 tx=0).stop 2023-08-27T11:28:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.671+0000 7f196a9fb700 1 -- 172.21.15.112:0/672780133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f196406e2f0 msgr2=0x7f19641555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.671+0000 7f196a9fb700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f196406e2f0 0x7f19641555d0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1958014040 tx=0x7f1958004750 comp rx=0 tx=0).stop 2023-08-27T11:28:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.671+0000 7f196a9fb700 1 -- 172.21.15.112:0/672780133 shutdown_connections 2023-08-27T11:28:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.671+0000 7f196a9fb700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f195006a4d0 0x7f195006c990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.671+0000 7f196a9fb700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1964158ed0 0x7f19640a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.671+0000 7f196a9fb700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f196406e2f0 0x7f19641555d0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.671+0000 7f196a9fb700 1 --2- 172.21.15.112:0/672780133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1964072000 0x7f19641e5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.671+0000 7f196a9fb700 1 -- 172.21.15.112:0/672780133 >> 172.21.15.112:0/672780133 conn(0x7f19640cc240 msgr2=0x7f196413ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.671+0000 7f196a9fb700 1 -- 172.21.15.112:0/672780133 shutdown_connections 2023-08-27T11:28:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.671+0000 7f196a9fb700 1 -- 172.21.15.112:0/672780133 wait complete. 2023-08-27T11:28:17.687 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:28:17.688 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:17.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.812+0000 7f5f67639700 1 Processor -- start 2023-08-27T11:28:17.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.812+0000 7f5f67639700 1 -- start start 2023-08-27T11:28:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f67639700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f6006adb0 0x7f5f6006b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f67639700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f6015eb60 con 0x7f5f600cdbd0 2023-08-27T11:28:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f67639700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f600b02f0 con 0x7f5f60151060 2023-08-27T11:28:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f67639700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6006b770 con 0x7f5f6006adb0 2023-08-27T11:28:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f65bd6700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5f600cdbd0 0x7f5f600ce020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58666/0 (socket says 172.21.15.112:58666) 2023-08-27T11:28:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f65bd6700 1 -- 172.21.15.112:0/1188610238 learned_addr learned my addr 172.21.15.112:0/1188610238 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123944579 0 0) 0x7f5f6015eb60 con 0x7f5f600cdbd0 2023-08-27T11:28:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f64bd4700 1 --2- 172.21.15.112:0/1188610238 >> v2:172.21.15.138:3300/0 conn(0x7f5f6006adb0 0x7f5f6006b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f4c0035b0 con 0x7f5f600cdbd0 2023-08-27T11:28:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3801869684 0 0) 0x7f5f4c0035b0 con 0x7f5f600cdbd0 2023-08-27T11:28:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 >> v2:172.21.15.138:3300/0 conn(0x7f5f6006adb0 msgr2=0x7f5f6006b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 --2- 172.21.15.112:0/1188610238 >> v2:172.21.15.138:3300/0 conn(0x7f5f6006adb0 0x7f5f6006b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 >> v1:172.21.15.138:6789/0 conn(0x7f5f60151060 legacy=0x7f5f6015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f600b05d0 con 0x7f5f600cdbd0 2023-08-27T11:28:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f5f580019f0 con 0x7f5f600cdbd0 2023-08-27T11:28:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 >> v1:172.21.15.112:6789/0 conn(0x7f5f600cdbd0 legacy=0x7f5f600ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 --2- 172.21.15.112:0/1188610238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c003a00 0x7f5f4c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 --2- 172.21.15.112:0/1188610238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f4c004850 0x7f5f4c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 --2- 172.21.15.112:0/1188610238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c005210 0x7f5f4c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6006b770 con 0x7f5f4c003a00 2023-08-27T11:28:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c00e2c0 con 0x7f5f4c004850 2023-08-27T11:28:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c00e480 con 0x7f5f4c005210 2023-08-27T11:28:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.813+0000 7f5f653d5700 1 --2- 172.21.15.112:0/1188610238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c003a00 0x7f5f4c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.814+0000 7f5f64bd4700 1 --2- 172.21.15.112:0/1188610238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f4c004850 0x7f5f4c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.814+0000 7f5f65bd6700 1 --2- 172.21.15.112:0/1188610238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c005210 0x7f5f4c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.814+0000 7f5f653d5700 1 -- 172.21.15.112:0/1188610238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c005210 msgr2=0x7f5f4c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.814+0000 7f5f653d5700 1 --2- 172.21.15.112:0/1188610238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c005210 0x7f5f4c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.814+0000 7f5f653d5700 1 -- 172.21.15.112:0/1188610238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f4c004850 msgr2=0x7f5f4c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.814+0000 7f5f653d5700 1 --2- 172.21.15.112:0/1188610238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f4c004850 0x7f5f4c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.814+0000 7f5f653d5700 1 -- 172.21.15.112:0/1188610238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0035b0 con 0x7f5f4c003a00 2023-08-27T11:28:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.814+0000 7f5f65bd6700 1 -- 172.21.15.112:0/1188610238 reap_dead start 2023-08-27T11:28:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.814+0000 7f5f64bd4700 1 --2- 172.21.15.112:0/1188610238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f4c004850 0x7f5f4c004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.814+0000 7f5f653d5700 1 --2- 172.21.15.112:0/1188610238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c003a00 0x7f5f4c003e20 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0017c0 tx=0x7f5f4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827c98e37a42eda7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.814+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c015c20 con 0x7f5f4c003a00 2023-08-27T11:28:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.814+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5c015d80 con 0x7f5f4c003a00 2023-08-27T11:28:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.814+0000 7f5f57fff700 1 -- 172.21.15.112:0/1188610238 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c005310 con 0x7f5f4c003a00 2023-08-27T11:28:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.815+0000 7f5f67639700 1 -- 172.21.15.112:0/1188610238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c003a00 msgr2=0x7f5f4c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.815+0000 7f5f67639700 1 --2- 172.21.15.112:0/1188610238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c003a00 0x7f5f4c003e20 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0017c0 tx=0x7f5f4c005880 comp rx=0 tx=0).stop 2023-08-27T11:28:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.815+0000 7f5f67639700 1 -- 172.21.15.112:0/1188610238 shutdown_connections 2023-08-27T11:28:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.815+0000 7f5f67639700 1 --2- 172.21.15.112:0/1188610238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f4c003a00 0x7f5f4c003e20 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.815+0000 7f5f67639700 1 -- 172.21.15.112:0/1188610238 >> 172.21.15.112:0/1188610238 conn(0x7f5f600d0000 msgr2=0x7f5f600d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.815+0000 7f5f67639700 1 -- 172.21.15.112:0/1188610238 shutdown_connections 2023-08-27T11:28:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.815+0000 7f5f67639700 1 -- 172.21.15.112:0/1188610238 wait complete. 2023-08-27T11:28:17.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.816+0000 7f5f67639700 1 Processor -- start 2023-08-27T11:28:17.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.816+0000 7f5f67639700 1 -- start start 2023-08-27T11:28:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.817+0000 7f5f67639700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c003a00 0x7f5f600c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.817+0000 7f5f67639700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f60152310 0x7f5f600c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.817+0000 7f5f67639700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f6013fb20 0x7f5f6013ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.817+0000 7f5f67639700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f600c70d0 con 0x7f5f60152310 2023-08-27T11:28:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.817+0000 7f5f67639700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f600c73b0 con 0x7f5f6013fb20 2023-08-27T11:28:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.817+0000 7f5f67639700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f600c7690 con 0x7f5f4c003a00 2023-08-27T11:28:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.817+0000 7f5f64bd4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f60152310 0x7f5f600c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.817+0000 7f5f64bd4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f60152310 0x7f5f600c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36060/0 (socket says 172.21.15.112:36060) 2023-08-27T11:28:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.817+0000 7f5f64bd4700 1 -- 172.21.15.112:0/3299923417 learned_addr learned my addr 172.21.15.112:0/3299923417 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.818+0000 7f5f653d5700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c003a00 0x7f5f600c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.818+0000 7f5f65bd6700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f6013fb20 0x7f5f6013ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.818+0000 7f5f64bd4700 1 -- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c003a00 msgr2=0x7f5f600c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.818+0000 7f5f64bd4700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c003a00 0x7f5f600c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.818+0000 7f5f64bd4700 1 -- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f6013fb20 msgr2=0x7f5f6013ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.818+0000 7f5f64bd4700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f6013fb20 0x7f5f6013ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.818+0000 7f5f64bd4700 1 -- 172.21.15.112:0/3299923417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f600c7970 con 0x7f5f60152310 2023-08-27T11:28:17.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.818+0000 7f5f64bd4700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f60152310 0x7f5f600c98a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f5f50011590 tx=0x7f5f50011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:17.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.818+0000 7f5f55ffb700 1 -- 172.21.15.112:0/3299923417 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f50004220 con 0x7f5f60152310 2023-08-27T11:28:17.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.818+0000 7f5f55ffb700 1 -- 172.21.15.112:0/3299923417 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f50004380 con 0x7f5f60152310 2023-08-27T11:28:17.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.818+0000 7f5f67639700 1 -- 172.21.15.112:0/3299923417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f600c7c50 con 0x7f5f60152310 2023-08-27T11:28:17.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.818+0000 7f5f55ffb700 1 -- 172.21.15.112:0/3299923417 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f500036a0 con 0x7f5f60152310 2023-08-27T11:28:17.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.818+0000 7f5f67639700 1 -- 172.21.15.112:0/3299923417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f601406f0 con 0x7f5f60152310 2023-08-27T11:28:17.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.819+0000 7f5f67639700 1 -- 172.21.15.112:0/3299923417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f600c7fc0 con 0x7f5f60152310 2023-08-27T11:28:17.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.819+0000 7f5f55ffb700 1 -- 172.21.15.112:0/3299923417 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5f50003800 con 0x7f5f60152310 2023-08-27T11:28:17.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.820+0000 7f5f55ffb700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f4406a4d0 0x7f5f4406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.820+0000 7f5f55ffb700 1 -- 172.21.15.112:0/3299923417 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5f50088ef0 con 0x7f5f60152310 2023-08-27T11:28:17.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.820+0000 7f5f653d5700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f4406a4d0 0x7f5f4406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.821+0000 7f5f653d5700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f4406a4d0 0x7f5f4406c990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5f600c6b50 tx=0x7f5f5c019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:17.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:17.824+0000 7f5f55ffb700 1 -- 172.21.15.112:0/3299923417 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5f50054090 con 0x7f5f60152310 2023-08-27T11:28:18.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.054+0000 7f5f67639700 1 -- 172.21.15.112:0/3299923417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f60140970 con 0x7f5f60152310 2023-08-27T11:28:18.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.054+0000 7f5f55ffb700 1 -- 172.21.15.112:0/3299923417 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4885 (secure 0 0 0) 0x7f5f50022090 con 0x7f5f60152310 2023-08-27T11:28:18.056 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:18.057 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/739970856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":739970856},{"type":"v1","addr":"172.21.15.138:6837","nonce":739970856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3233069521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3233069521},{"type":"v1","addr":"172.21.15.112:6834","nonce":3233069521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2559241026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2559241026},{"type":"v1","addr":"172.21.15.112:6835","nonce":2559241026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:14.032146+0000","modified":"2023-08-27T11:28:17.665929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6836/529571603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":529571603},{"type":"v1","addr":"172.21.15.138:6836","nonce":529571603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-08-27T11:28:18.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.057+0000 7f5f67639700 1 -- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f4406a4d0 msgr2=0x7f5f4406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.057+0000 7f5f67639700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f4406a4d0 0x7f5f4406c990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5f600c6b50 tx=0x7f5f5c019000 comp rx=0 tx=0).stop 2023-08-27T11:28:18.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.057+0000 7f5f67639700 1 -- 172.21.15.112:0/3299923417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f60152310 msgr2=0x7f5f600c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.057+0000 7f5f67639700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f60152310 0x7f5f600c98a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f5f50011590 tx=0x7f5f50011950 comp rx=0 tx=0).stop 2023-08-27T11:28:18.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.057+0000 7f5f67639700 1 -- 172.21.15.112:0/3299923417 shutdown_connections 2023-08-27T11:28:18.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.057+0000 7f5f67639700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f4406a4d0 0x7f5f4406c990 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.057+0000 7f5f67639700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f6013fb20 0x7f5f6013ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.057+0000 7f5f67639700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f4c003a00 0x7f5f600c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.057+0000 7f5f67639700 1 --2- 172.21.15.112:0/3299923417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f60152310 0x7f5f600c98a0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.057+0000 7f5f67639700 1 -- 172.21.15.112:0/3299923417 >> 172.21.15.112:0/3299923417 conn(0x7f5f600d0000 msgr2=0x7f5f600cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.057+0000 7f5f67639700 1 -- 172.21.15.112:0/3299923417 shutdown_connections 2023-08-27T11:28:18.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.057+0000 7f5f67639700 1 -- 172.21.15.112:0/3299923417 wait complete. 2023-08-27T11:28:18.061 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 94 2023-08-27T11:28:18.070 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:18.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.188+0000 7fa2e8e55700 1 Processor -- start 2023-08-27T11:28:18.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.188+0000 7fa2e8e55700 1 -- start start 2023-08-27T11:28:18.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.189+0000 7fa2e8e55700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2e40719a0 0x7fa2e4071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.189+0000 7fa2e8e55700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2e413ebc0 0x7fa2e413f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.189+0000 7fa2e8e55700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e415ed80 con 0x7fa2e40cedc0 2023-08-27T11:28:18.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.189+0000 7fa2e8e55700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e4072360 con 0x7fa2e413ebc0 2023-08-27T11:28:18.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.189+0000 7fa2e8e55700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e415f020 con 0x7fa2e40719a0 2023-08-27T11:28:18.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.189+0000 7fa2e259c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa2e40cedc0 0x7fa2e415c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58682/0 (socket says 172.21.15.112:58682) 2023-08-27T11:28:18.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.189+0000 7fa2e259c700 1 -- 172.21.15.112:0/882304656 learned_addr learned my addr 172.21.15.112:0/882304656 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:18.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.189+0000 7fa2e1d9b700 1 --2- 172.21.15.112:0/882304656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2e40719a0 0x7fa2e4071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.189+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1793267797 0 0) 0x7fa2e415ed80 con 0x7fa2e40cedc0 2023-08-27T11:28:18.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.189+0000 7fa2e2d9d700 1 --2- 172.21.15.112:0/882304656 >> v2:172.21.15.138:3300/0 conn(0x7fa2e413ebc0 0x7fa2e413f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.189+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2cc0035b0 con 0x7fa2e40cedc0 2023-08-27T11:28:18.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2733173137 0 0) 0x7fa2cc0035b0 con 0x7fa2e40cedc0 2023-08-27T11:28:18.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2e40719a0 msgr2=0x7fa2e4071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 --2- 172.21.15.112:0/882304656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2e40719a0 0x7fa2e4071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 >> v2:172.21.15.138:3300/0 conn(0x7fa2e413ebc0 msgr2=0x7fa2e413f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 --2- 172.21.15.112:0/882304656 >> v2:172.21.15.138:3300/0 conn(0x7fa2e413ebc0 0x7fa2e413f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e415f300 con 0x7fa2e40cedc0 2023-08-27T11:28:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fa2d8001670 con 0x7fa2e40cedc0 2023-08-27T11:28:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 >> v1:172.21.15.112:6789/0 conn(0x7fa2e40cedc0 legacy=0x7fa2e415c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 --2- 172.21.15.112:0/882304656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2cc003a00 0x7fa2cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 --2- 172.21.15.112:0/882304656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2cc004850 0x7fa2cc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 --2- 172.21.15.112:0/882304656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2e40719a0 0x7fa2cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e4072360 con 0x7fa2cc003a00 2023-08-27T11:28:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e415f020 con 0x7fa2cc004850 2023-08-27T11:28:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc00ba10 con 0x7fa2e40719a0 2023-08-27T11:28:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e259c700 1 --2- 172.21.15.112:0/882304656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2cc003a00 0x7fa2cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e1d9b700 1 --2- 172.21.15.112:0/882304656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2cc004850 0x7fa2cc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e2d9d700 1 --2- 172.21.15.112:0/882304656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2e40719a0 0x7fa2cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e259c700 1 -- 172.21.15.112:0/882304656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2e40719a0 msgr2=0x7fa2cc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e259c700 1 --2- 172.21.15.112:0/882304656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2e40719a0 0x7fa2cc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e259c700 1 -- 172.21.15.112:0/882304656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2cc004850 msgr2=0x7fa2cc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e259c700 1 --2- 172.21.15.112:0/882304656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2cc004850 0x7fa2cc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e259c700 1 -- 172.21.15.112:0/882304656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0035b0 con 0x7fa2cc003a00 2023-08-27T11:28:18.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e2d9d700 1 --2- 172.21.15.112:0/882304656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2e40719a0 0x7fa2cc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:18.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e259c700 1 --2- 172.21.15.112:0/882304656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2cc003a00 0x7fa2cc003e20 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800c320 tx=0x7fa2cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dffedf602c2084f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800ce60 con 0x7fa2cc003a00 2023-08-27T11:28:18.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d8005960 con 0x7fa2cc003a00 2023-08-27T11:28:18.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.190+0000 7fa2e159a700 1 -- 172.21.15.112:0/882304656 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8005d10 con 0x7fa2cc003a00 2023-08-27T11:28:18.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.191+0000 7fa2e8e55700 1 -- 172.21.15.112:0/882304656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2cc003a00 msgr2=0x7fa2cc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.191+0000 7fa2e8e55700 1 --2- 172.21.15.112:0/882304656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2cc003a00 0x7fa2cc003e20 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800c320 tx=0x7fa2cc005880 comp rx=0 tx=0).stop 2023-08-27T11:28:18.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.191+0000 7fa2e2d9d700 1 -- 172.21.15.112:0/882304656 reap_dead start 2023-08-27T11:28:18.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.191+0000 7fa2e8e55700 1 -- 172.21.15.112:0/882304656 shutdown_connections 2023-08-27T11:28:18.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.191+0000 7fa2e8e55700 1 -- 172.21.15.112:0/882304656 >> 172.21.15.112:0/882304656 conn(0x7fa2e40cf750 msgr2=0x7fa2e40cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.191+0000 7fa2e8e55700 1 -- 172.21.15.112:0/882304656 shutdown_connections 2023-08-27T11:28:18.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.191+0000 7fa2e8e55700 1 -- 172.21.15.112:0/882304656 wait complete. 2023-08-27T11:28:18.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.192+0000 7fa2e8e55700 1 Processor -- start 2023-08-27T11:28:18.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.192+0000 7fa2e8e55700 1 -- start start 2023-08-27T11:28:18.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e8e55700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2cc003a00 0x7fa2e4155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e8e55700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2e40b03b0 0x7fa2e413e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e8e55700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2e41f6ea0 0x7fa2e40ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e8e55700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e40b10e0 con 0x7fa2e40b03b0 2023-08-27T11:28:18.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e8e55700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e40b0b20 con 0x7fa2e41f6ea0 2023-08-27T11:28:18.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e8e55700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e40b0e00 con 0x7fa2cc003a00 2023-08-27T11:28:18.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e1d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2e40b03b0 0x7fa2e413e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e1d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2e40b03b0 0x7fa2e413e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36082/0 (socket says 172.21.15.112:36082) 2023-08-27T11:28:18.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e1d9b700 1 -- 172.21.15.112:0/3630430901 learned_addr learned my addr 172.21.15.112:0/3630430901 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:18.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e2d9d700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2e41f6ea0 0x7fa2e40ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e259c700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2cc003a00 0x7fa2e4155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e1d9b700 1 -- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2cc003a00 msgr2=0x7fa2e4155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e1d9b700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2cc003a00 0x7fa2e4155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e1d9b700 1 -- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2e41f6ea0 msgr2=0x7fa2e40ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e1d9b700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2e41f6ea0 0x7fa2e40ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e1d9b700 1 -- 172.21.15.112:0/3630430901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e41f66a0 con 0x7fa2e40b03b0 2023-08-27T11:28:18.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.193+0000 7fa2e2d9d700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2e41f6ea0 0x7fa2e40ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:18.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.194+0000 7fa2e1d9b700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2e40b03b0 0x7fa2e413e1d0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40119a0 tx=0x7fa2d4011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.194+0000 7fa2d2ffd700 1 -- 172.21.15.112:0/3630430901 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400f940 con 0x7fa2e40b03b0 2023-08-27T11:28:18.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.194+0000 7fa2d2ffd700 1 -- 172.21.15.112:0/3630430901 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d400faa0 con 0x7fa2e40b03b0 2023-08-27T11:28:18.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.194+0000 7fa2e8e55700 1 -- 172.21.15.112:0/3630430901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e41f6980 con 0x7fa2e40b03b0 2023-08-27T11:28:18.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.194+0000 7fa2d2ffd700 1 -- 172.21.15.112:0/3630430901 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400fe30 con 0x7fa2e40b03b0 2023-08-27T11:28:18.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.194+0000 7fa2e8e55700 1 -- 172.21.15.112:0/3630430901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e413e710 con 0x7fa2e40b03b0 2023-08-27T11:28:18.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.195+0000 7fa2e8e55700 1 -- 172.21.15.112:0/3630430901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e4067a20 con 0x7fa2e40b03b0 2023-08-27T11:28:18.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.195+0000 7fa2d2ffd700 1 -- 172.21.15.112:0/3630430901 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa2d4014440 con 0x7fa2e40b03b0 2023-08-27T11:28:18.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.196+0000 7fa2d2ffd700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa2c406a4d0 0x7fa2c406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.196+0000 7fa2d2ffd700 1 -- 172.21.15.112:0/3630430901 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa2d4089320 con 0x7fa2e40b03b0 2023-08-27T11:28:18.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.196+0000 7fa2e259c700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa2c406a4d0 0x7fa2c406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.196+0000 7fa2e259c700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa2c406a4d0 0x7fa2c406c990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8008fd0 tx=0x7fa2d800a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.200+0000 7fa2d2ffd700 1 -- 172.21.15.112:0/3630430901 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa2d4058000 con 0x7fa2e40b03b0 2023-08-27T11:28:18.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.389+0000 7fa2e8e55700 1 -- 172.21.15.112:0/3630430901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2e415f890 con 0x7fa2e40b03b0 2023-08-27T11:28:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.390+0000 7fa2d2ffd700 1 -- 172.21.15.112:0/3630430901 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+18263 (secure 0 0 0) 0x7fa2d4057b90 con 0x7fa2e40b03b0 2023-08-27T11:28:18.392 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:18.392 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":139,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:15.631983+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:12.176044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:12.992996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:18.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.392+0000 7fa2e8e55700 1 -- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa2c406a4d0 msgr2=0x7fa2c406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.392+0000 7fa2e8e55700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa2c406a4d0 0x7fa2c406c990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8008fd0 tx=0x7fa2d800a000 comp rx=0 tx=0).stop 2023-08-27T11:28:18.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.392+0000 7fa2e8e55700 1 -- 172.21.15.112:0/3630430901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2e40b03b0 msgr2=0x7fa2e413e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.392+0000 7fa2e8e55700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2e40b03b0 0x7fa2e413e1d0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40119a0 tx=0x7fa2d4011d60 comp rx=0 tx=0).stop 2023-08-27T11:28:18.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.392+0000 7fa2e8e55700 1 -- 172.21.15.112:0/3630430901 shutdown_connections 2023-08-27T11:28:18.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.392+0000 7fa2e8e55700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa2c406a4d0 0x7fa2c406c990 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.392+0000 7fa2e8e55700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2e41f6ea0 0x7fa2e40ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.392+0000 7fa2e8e55700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2cc003a00 0x7fa2e4155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.392+0000 7fa2e8e55700 1 --2- 172.21.15.112:0/3630430901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2e40b03b0 0x7fa2e413e1d0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.392+0000 7fa2e8e55700 1 -- 172.21.15.112:0/3630430901 >> 172.21.15.112:0/3630430901 conn(0x7fa2e40cf750 msgr2=0x7fa2e4157a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.392+0000 7fa2e8e55700 1 -- 172.21.15.112:0/3630430901 shutdown_connections 2023-08-27T11:28:18.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.392+0000 7fa2e8e55700 1 -- 172.21.15.112:0/3630430901 wait complete. 2023-08-27T11:28:18.406 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:28:18.407 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:18.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.525+0000 7f00851fc700 1 Processor -- start 2023-08-27T11:28:18.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.525+0000 7f00851fc700 1 -- start start 2023-08-27T11:28:18.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f00851fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0080146d00 0x7f00800b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f00851fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f00800b3980 0x7f00800ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f00851fc700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00801470e0 con 0x7f00800ad970 2023-08-27T11:28:18.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f00851fc700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00800aeb70 con 0x7f00800b3980 2023-08-27T11:28:18.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f00851fc700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00800adf60 con 0x7f0080146d00 2023-08-27T11:28:18.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007e59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f00800b3980 0x7f00800ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007f59e700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f00800ad970 0x7f00800af000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58706/0 (socket says 172.21.15.112:58706) 2023-08-27T11:28:18.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007f59e700 1 -- 172.21.15.112:0/2835602779 learned_addr learned my addr 172.21.15.112:0/2835602779 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:18.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007ed9d700 1 --2- 172.21.15.112:0/2835602779 >> v2:172.21.15.138:3300/0 conn(0x7f0080146d00 0x7f00800b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007e59c700 1 -- 172.21.15.112:0/2835602779 >> v2:172.21.15.138:3300/0 conn(0x7f0080146d00 msgr2=0x7f00800b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007e59c700 1 --2- 172.21.15.112:0/2835602779 >> v2:172.21.15.138:3300/0 conn(0x7f0080146d00 0x7f00800b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007e59c700 1 -- 172.21.15.112:0/2835602779 >> v1:172.21.15.112:6789/0 conn(0x7f00800ad970 legacy=0x7f00800af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007e59c700 1 -- 172.21.15.112:0/2835602779 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00800ae240 con 0x7f00800b3980 2023-08-27T11:28:18.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007dd9b700 1 -- 172.21.15.112:0/2835602779 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055904573 0 0) 0x7f00801470e0 con 0x7f00800ad970 2023-08-27T11:28:18.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007ed9d700 1 --2- 172.21.15.112:0/2835602779 >> v2:172.21.15.138:3300/0 conn(0x7f0080146d00 0x7f00800b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:18.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007e59c700 1 --2- 172.21.15.112:0/2835602779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00800b3980 0x7f00800ad430 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0080070420 tx=0x7f0074009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41a7b595cc9de6d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007dd9b700 1 -- 172.21.15.112:0/2835602779 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074005e30 con 0x7f00800b3980 2023-08-27T11:28:18.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007dd9b700 1 -- 172.21.15.112:0/2835602779 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0074006030 con 0x7f00800b3980 2023-08-27T11:28:18.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.526+0000 7f007dd9b700 1 -- 172.21.15.112:0/2835602779 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00740063c0 con 0x7f00800b3980 2023-08-27T11:28:18.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.527+0000 7f00851fc700 1 -- 172.21.15.112:0/2835602779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00800b3980 msgr2=0x7f00800ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.527+0000 7f00851fc700 1 --2- 172.21.15.112:0/2835602779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00800b3980 0x7f00800ad430 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0080070420 tx=0x7f0074009b00 comp rx=0 tx=0).stop 2023-08-27T11:28:18.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.527+0000 7f00851fc700 1 -- 172.21.15.112:0/2835602779 shutdown_connections 2023-08-27T11:28:18.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.527+0000 7f00851fc700 1 --2- 172.21.15.112:0/2835602779 >> v2:172.21.15.138:3300/0 conn(0x7f0080146d00 0x7f00800b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.527+0000 7f00851fc700 1 --2- 172.21.15.112:0/2835602779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00800b3980 0x7f00800ad430 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.527+0000 7f00851fc700 1 -- 172.21.15.112:0/2835602779 >> 172.21.15.112:0/2835602779 conn(0x7f008006adb0 msgr2=0x7f008006b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.527+0000 7f00851fc700 1 -- 172.21.15.112:0/2835602779 shutdown_connections 2023-08-27T11:28:18.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.527+0000 7f00851fc700 1 -- 172.21.15.112:0/2835602779 wait complete. 2023-08-27T11:28:18.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.528+0000 7f00851fc700 1 Processor -- start 2023-08-27T11:28:18.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.528+0000 7f00851fc700 1 -- start start 2023-08-27T11:28:18.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.529+0000 7f00851fc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00800ad970 0x7f00800ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.529+0000 7f00851fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00800b3980 0x7f008015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.529+0000 7f00851fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0080146d00 0x7f00800b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.529+0000 7f00851fc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008015c2c0 con 0x7f00800b3980 2023-08-27T11:28:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.529+0000 7f00851fc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008015c4b0 con 0x7f0080146d00 2023-08-27T11:28:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.529+0000 7f00851fc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00800cddd0 con 0x7f00800ad970 2023-08-27T11:28:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.529+0000 7f007e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00800b3980 0x7f008015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.529+0000 7f007e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00800b3980 0x7f008015bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36096/0 (socket says 172.21.15.112:36096) 2023-08-27T11:28:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.529+0000 7f007e59c700 1 -- 172.21.15.112:0/1905619826 learned_addr learned my addr 172.21.15.112:0/1905619826 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.529+0000 7f007ed9d700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00800ad970 0x7f00800ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.529+0000 7f007f59e700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0080146d00 0x7f00800b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.530+0000 7f007e59c700 1 -- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00800ad970 msgr2=0x7f00800ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.530+0000 7f007e59c700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00800ad970 0x7f00800ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.530+0000 7f007e59c700 1 -- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0080146d00 msgr2=0x7f00800b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.530+0000 7f007e59c700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0080146d00 0x7f00800b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.530+0000 7f007e59c700 1 -- 172.21.15.112:0/1905619826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0074009430 con 0x7f00800b3980 2023-08-27T11:28:18.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.530+0000 7f007f59e700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0080146d00 0x7f00800b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:18.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.530+0000 7f007e59c700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00800b3980 0x7f008015bd80 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0074009b40 tx=0x7f0074004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.530+0000 7f00677fe700 1 -- 172.21.15.112:0/1905619826 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074006960 con 0x7f00800b3980 2023-08-27T11:28:18.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.530+0000 7f00677fe700 1 -- 172.21.15.112:0/1905619826 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0074020070 con 0x7f00800b3980 2023-08-27T11:28:18.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.530+0000 7f00677fe700 1 -- 172.21.15.112:0/1905619826 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074004280 con 0x7f00800b3980 2023-08-27T11:28:18.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.530+0000 7f00851fc700 1 -- 172.21.15.112:0/1905619826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00800ce0b0 con 0x7f00800b3980 2023-08-27T11:28:18.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.530+0000 7f00851fc700 1 -- 172.21.15.112:0/1905619826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00800ce670 con 0x7f00800b3980 2023-08-27T11:28:18.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.531+0000 7f00851fc700 1 -- 172.21.15.112:0/1905619826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0080067a20 con 0x7f00800b3980 2023-08-27T11:28:18.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.531+0000 7f00677fe700 1 -- 172.21.15.112:0/1905619826 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f0074022030 con 0x7f00800b3980 2023-08-27T11:28:18.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.532+0000 7f00677fe700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f006c06a4d0 0x7f006c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.532+0000 7f00677fe700 1 -- 172.21.15.112:0/1905619826 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f007408a0d0 con 0x7f00800b3980 2023-08-27T11:28:18.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.532+0000 7f007ed9d700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f006c06a4d0 0x7f006c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.532+0000 7f007ed9d700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f006c06a4d0 0x7f006c06c990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f007000c960 tx=0x7f0070008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.536+0000 7f00677fe700 1 -- 172.21.15.112:0/1905619826 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f00740bc990 con 0x7f00800b3980 2023-08-27T11:28:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.726+0000 7f00851fc700 1 -- 172.21.15.112:0/1905619826 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00800ae800 con 0x7f006c06a4d0 2023-08-27T11:28:18.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.735+0000 7f00677fe700 1 -- 172.21.15.112:0/1905619826 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312235 (secure 0 0 0) 0x7f007000cdd0 con 0x7f006c06a4d0 2023-08-27T11:28:18.741 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:18.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.739+0000 7f00851fc700 1 -- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f006c06a4d0 msgr2=0x7f006c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.739+0000 7f00851fc700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f006c06a4d0 0x7f006c06c990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f007000c960 tx=0x7f0070008040 comp rx=0 tx=0).stop 2023-08-27T11:28:18.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.739+0000 7f00851fc700 1 -- 172.21.15.112:0/1905619826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00800b3980 msgr2=0x7f008015bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.739+0000 7f00851fc700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00800b3980 0x7f008015bd80 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0074009b40 tx=0x7f0074004750 comp rx=0 tx=0).stop 2023-08-27T11:28:18.744 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.739+0000 7f00851fc700 1 -- 172.21.15.112:0/1905619826 shutdown_connections 2023-08-27T11:28:18.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.739+0000 7f00851fc700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f006c06a4d0 0x7f006c06c990 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.739+0000 7f00851fc700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0080146d00 0x7f00800b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.739+0000 7f00851fc700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00800ad970 0x7f00800ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.739+0000 7f00851fc700 1 --2- 172.21.15.112:0/1905619826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00800b3980 0x7f008015bd80 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.745 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.739+0000 7f00851fc700 1 -- 172.21.15.112:0/1905619826 >> 172.21.15.112:0/1905619826 conn(0x7f008006adb0 msgr2=0x7f008006d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.739+0000 7f00851fc700 1 -- 172.21.15.112:0/1905619826 shutdown_connections 2023-08-27T11:28:18.746 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.739+0000 7f00851fc700 1 -- 172.21.15.112:0/1905619826 wait complete. 2023-08-27T11:28:18.746 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:28:18.753 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2023-08-27T11:28:18.399206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264380,"kb_used_data":14924,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754710340,"statfs":{"total":773094113280,"available":772823388160,"internally_reserved":0,"allocated":15282176,"data_stored":7854609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.774615"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171725+0000","last_change":"2023-08-27T11:28:14.653632+0000","last_active":"2023-08-27T11:28:16.171725+0000","last_peered":"2023-08-27T11:28:16.171725+0000","last_clean":"2023-08-27T11:28:16.171725+0000","last_became_active":"2023-08-27T11:28:14.653202+0000","last_became_peered":"2023-08-27T11:28:14.653202+0000","last_unstale":"2023-08-27T11:28:16.171725+0000","last_undegraded":"2023-08-27T11:28:16.171725+0000","last_fullsized":"2023-08-27T11:28:16.171725+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:47:54.569076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171956+0000","last_change":"2023-08-27T11:28:14.650120+0000","last_active":"2023-08-27T11:28:16.171956+0000","last_peered":"2023-08-27T11:28:16.171956+0000","last_clean":"2023-08-27T11:28:16.171956+0000","last_became_active":"2023-08-27T11:28:14.649837+0000","last_became_peered":"2023-08-27T11:28:14.649837+0000","last_unstale":"2023-08-27T11:28:16.171956+0000","last_undegraded":"2023-08-27T11:28:16.171956+0000","last_fullsized":"2023-08-27T11:28:16.171956+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:34:09.267774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640135+0000","last_change":"2023-08-27T11:28:14.661362+0000","last_active":"2023-08-27T11:28:15.640135+0000","last_peered":"2023-08-27T11:28:15.640135+0000","last_clean":"2023-08-27T11:28:15.640135+0000","last_became_active":"2023-08-27T11:28:14.661023+0000","last_became_peered":"2023-08-27T11:28:14.661023+0000","last_unstale":"2023-08-27T11:28:15.640135+0000","last_undegraded":"2023-08-27T11:28:15.640135+0000","last_fullsized":"2023-08-27T11:28:15.640135+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:03.434668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673076+0000","last_change":"2023-08-27T11:28:14.650040+0000","last_active":"2023-08-27T11:28:16.673076+0000","last_peered":"2023-08-27T11:28:16.673076+0000","last_clean":"2023-08-27T11:28:16.673076+0000","last_became_active":"2023-08-27T11:28:14.649135+0000","last_became_peered":"2023-08-27T11:28:14.649135+0000","last_unstale":"2023-08-27T11:28:16.673076+0000","last_undegraded":"2023-08-27T11:28:16.673076+0000","last_fullsized":"2023-08-27T11:28:16.673076+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:45:51.828640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635138+0000","last_change":"2023-08-27T11:28:14.656675+0000","last_active":"2023-08-27T11:28:15.635138+0000","last_peered":"2023-08-27T11:28:15.635138+0000","last_clean":"2023-08-27T11:28:15.635138+0000","last_became_active":"2023-08-27T11:28:14.655942+0000","last_became_peered":"2023-08-27T11:28:14.655942+0000","last_unstale":"2023-08-27T11:28:15.635138+0000","last_undegraded":"2023-08-27T11:28:15.635138+0000","last_fullsized":"2023-08-27T11:28:15.635138+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:50.800979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635060+0000","last_change":"2023-08-27T11:28:14.654403+0000","last_active":"2023-08-27T11:28:15.635060+0000","last_peered":"2023-08-27T11:28:15.635060+0000","last_clean":"2023-08-27T11:28:15.635060+0000","last_became_active":"2023-08-27T11:28:14.654094+0000","last_became_peered":"2023-08-27T11:28:14.654094+0000","last_unstale":"2023-08-27T11:28:15.635060+0000","last_undegraded":"2023-08-27T11:28:15.635060+0000","last_fullsized":"2023-08-27T11:28:15.635060+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:03.434668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673852+0000","last_change":"2023-08-27T11:28:14.653279+0000","last_active":"2023-08-27T11:28:16.673852+0000","last_peered":"2023-08-27T11:28:16.673852+0000","last_clean":"2023-08-27T11:28:16.673852+0000","last_became_active":"2023-08-27T11:28:14.652988+0000","last_became_peered":"2023-08-27T11:28:14.652988+0000","last_unstale":"2023-08-27T11:28:16.673852+0000","last_undegraded":"2023-08-27T11:28:16.673852+0000","last_fullsized":"2023-08-27T11:28:16.673852+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:24:00.946418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672986+0000","last_change":"2023-08-27T11:28:14.667142+0000","last_active":"2023-08-27T11:28:16.672986+0000","last_peered":"2023-08-27T11:28:16.672986+0000","last_clean":"2023-08-27T11:28:16.672986+0000","last_became_active":"2023-08-27T11:28:14.665039+0000","last_became_peered":"2023-08-27T11:28:14.665039+0000","last_unstale":"2023-08-27T11:28:16.672986+0000","last_undegraded":"2023-08-27T11:28:16.672986+0000","last_fullsized":"2023-08-27T11:28:16.672986+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:07:10.121767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171779+0000","last_change":"2023-08-27T11:28:14.653472+0000","last_active":"2023-08-27T11:28:16.171779+0000","last_peered":"2023-08-27T11:28:16.171779+0000","last_clean":"2023-08-27T11:28:16.171779+0000","last_became_active":"2023-08-27T11:28:14.652816+0000","last_became_peered":"2023-08-27T11:28:14.652816+0000","last_unstale":"2023-08-27T11:28:16.171779+0000","last_undegraded":"2023-08-27T11:28:16.171779+0000","last_fullsized":"2023-08-27T11:28:16.171779+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:40.732259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635173+0000","last_change":"2023-08-27T11:28:14.657163+0000","last_active":"2023-08-27T11:28:15.635173+0000","last_peered":"2023-08-27T11:28:15.635173+0000","last_clean":"2023-08-27T11:28:15.635173+0000","last_became_active":"2023-08-27T11:28:14.656659+0000","last_became_peered":"2023-08-27T11:28:14.656659+0000","last_unstale":"2023-08-27T11:28:15.635173+0000","last_undegraded":"2023-08-27T11:28:15.635173+0000","last_fullsized":"2023-08-27T11:28:15.635173+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:55:02.903945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916393+0000","last_change":"2023-08-27T11:28:14.662167+0000","last_active":"2023-08-27T11:28:15.916393+0000","last_peered":"2023-08-27T11:28:15.916393+0000","last_clean":"2023-08-27T11:28:15.916393+0000","last_became_active":"2023-08-27T11:28:14.661899+0000","last_became_peered":"2023-08-27T11:28:14.661899+0000","last_unstale":"2023-08-27T11:28:15.916393+0000","last_undegraded":"2023-08-27T11:28:15.916393+0000","last_fullsized":"2023-08-27T11:28:15.916393+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:25:27.533398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673228+0000","last_change":"2023-08-27T11:28:14.658540+0000","last_active":"2023-08-27T11:28:16.673228+0000","last_peered":"2023-08-27T11:28:16.673228+0000","last_clean":"2023-08-27T11:28:16.673228+0000","last_became_active":"2023-08-27T11:28:14.658229+0000","last_became_peered":"2023-08-27T11:28:14.658229+0000","last_unstale":"2023-08-27T11:28:16.673228+0000","last_undegraded":"2023-08-27T11:28:16.673228+0000","last_fullsized":"2023-08-27T11:28:16.673228+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:28:39.113843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641013+0000","last_change":"2023-08-27T11:28:14.666082+0000","last_active":"2023-08-27T11:28:15.641013+0000","last_peered":"2023-08-27T11:28:15.641013+0000","last_clean":"2023-08-27T11:28:15.641013+0000","last_became_active":"2023-08-27T11:28:14.665656+0000","last_became_peered":"2023-08-27T11:28:14.665656+0000","last_unstale":"2023-08-27T11:28:15.641013+0000","last_undegraded":"2023-08-27T11:28:15.641013+0000","last_fullsized":"2023-08-27T11:28:15.641013+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:03.556396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672935+0000","last_change":"2023-08-27T11:28:14.653903+0000","last_active":"2023-08-27T11:28:16.672935+0000","last_peered":"2023-08-27T11:28:16.672935+0000","last_clean":"2023-08-27T11:28:16.672935+0000","last_became_active":"2023-08-27T11:28:14.653657+0000","last_became_peered":"2023-08-27T11:28:14.653657+0000","last_unstale":"2023-08-27T11:28:16.672935+0000","last_undegraded":"2023-08-27T11:28:16.672935+0000","last_fullsized":"2023-08-27T11:28:16.672935+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:41:51.876480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171599+0000","last_change":"2023-08-27T11:28:14.643766+0000","last_active":"2023-08-27T11:28:16.171599+0000","last_peered":"2023-08-27T11:28:16.171599+0000","last_clean":"2023-08-27T11:28:16.171599+0000","last_became_active":"2023-08-27T11:28:14.643212+0000","last_became_peered":"2023-08-27T11:28:14.643212+0000","last_unstale":"2023-08-27T11:28:16.171599+0000","last_undegraded":"2023-08-27T11:28:16.171599+0000","last_fullsized":"2023-08-27T11:28:16.171599+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:20:04.295796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.634913+0000","last_change":"2023-08-27T11:28:14.657418+0000","last_active":"2023-08-27T11:28:15.634913+0000","last_peered":"2023-08-27T11:28:15.634913+0000","last_clean":"2023-08-27T11:28:15.634913+0000","last_became_active":"2023-08-27T11:28:14.656144+0000","last_became_peered":"2023-08-27T11:28:14.656144+0000","last_unstale":"2023-08-27T11:28:15.634913+0000","last_undegraded":"2023-08-27T11:28:15.634913+0000","last_fullsized":"2023-08-27T11:28:15.634913+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:27:29.886325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635482+0000","last_change":"2023-08-27T11:28:14.658801+0000","last_active":"2023-08-27T11:28:15.635482+0000","last_peered":"2023-08-27T11:28:15.635482+0000","last_clean":"2023-08-27T11:28:15.635482+0000","last_became_active":"2023-08-27T11:28:14.658561+0000","last_became_peered":"2023-08-27T11:28:14.658561+0000","last_unstale":"2023-08-27T11:28:15.635482+0000","last_undegraded":"2023-08-27T11:28:15.635482+0000","last_fullsized":"2023-08-27T11:28:15.635482+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:57:19.740295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641378+0000","last_change":"2023-08-27T11:28:14.655039+0000","last_active":"2023-08-27T11:28:15.641378+0000","last_peered":"2023-08-27T11:28:15.641378+0000","last_clean":"2023-08-27T11:28:15.641378+0000","last_became_active":"2023-08-27T11:28:14.654781+0000","last_became_peered":"2023-08-27T11:28:14.654781+0000","last_unstale":"2023-08-27T11:28:15.641378+0000","last_undegraded":"2023-08-27T11:28:15.641378+0000","last_fullsized":"2023-08-27T11:28:15.641378+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:28:40.318425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641466+0000","last_change":"2023-08-27T11:28:14.650908+0000","last_active":"2023-08-27T11:28:15.641466+0000","last_peered":"2023-08-27T11:28:15.641466+0000","last_clean":"2023-08-27T11:28:15.641466+0000","last_became_active":"2023-08-27T11:28:14.647101+0000","last_became_peered":"2023-08-27T11:28:14.647101+0000","last_unstale":"2023-08-27T11:28:15.641466+0000","last_undegraded":"2023-08-27T11:28:15.641466+0000","last_fullsized":"2023-08-27T11:28:15.641466+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:55:00.113382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635082+0000","last_change":"2023-08-27T11:28:14.653232+0000","last_active":"2023-08-27T11:28:15.635082+0000","last_peered":"2023-08-27T11:28:15.635082+0000","last_clean":"2023-08-27T11:28:15.635082+0000","last_became_active":"2023-08-27T11:28:14.652576+0000","last_became_peered":"2023-08-27T11:28:14.652576+0000","last_unstale":"2023-08-27T11:28:15.635082+0000","last_undegraded":"2023-08-27T11:28:15.635082+0000","last_fullsized":"2023-08-27T11:28:15.635082+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:55:02.771821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171934+0000","last_change":"2023-08-27T11:28:14.647657+0000","last_active":"2023-08-27T11:28:16.171934+0000","last_peered":"2023-08-27T11:28:16.171934+0000","last_clean":"2023-08-27T11:28:16.171934+0000","last_became_active":"2023-08-27T11:28:14.647381+0000","last_became_peered":"2023-08-27T11:28:14.647381+0000","last_unstale":"2023-08-27T11:28:16.171934+0000","last_undegraded":"2023-08-27T11:28:16.171934+0000","last_fullsized":"2023-08-27T11:28:16.171934+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:27:29.886325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171982+0000","last_change":"2023-08-27T11:28:14.648423+0000","last_active":"2023-08-27T11:28:16.171982+0000","last_peered":"2023-08-27T11:28:16.171982+0000","last_clean":"2023-08-27T11:28:16.171982+0000","last_became_active":"2023-08-27T11:28:14.647791+0000","last_became_peered":"2023-08-27T11:28:14.647791+0000","last_unstale":"2023-08-27T11:28:16.171982+0000","last_undegraded":"2023-08-27T11:28:16.171982+0000","last_fullsized":"2023-08-27T11:28:16.171982+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:55:02.771821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672963+0000","last_change":"2023-08-27T11:28:14.664163+0000","last_active":"2023-08-27T11:28:16.672963+0000","last_peered":"2023-08-27T11:28:16.672963+0000","last_clean":"2023-08-27T11:28:16.672963+0000","last_became_active":"2023-08-27T11:28:14.663861+0000","last_became_peered":"2023-08-27T11:28:14.663861+0000","last_unstale":"2023-08-27T11:28:16.672963+0000","last_undegraded":"2023-08-27T11:28:16.672963+0000","last_fullsized":"2023-08-27T11:28:16.672963+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:46:04.177457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171476+0000","last_change":"2023-08-27T11:28:14.658603+0000","last_active":"2023-08-27T11:28:16.171476+0000","last_peered":"2023-08-27T11:28:16.171476+0000","last_clean":"2023-08-27T11:28:16.171476+0000","last_became_active":"2023-08-27T11:28:14.658245+0000","last_became_peered":"2023-08-27T11:28:14.658245+0000","last_unstale":"2023-08-27T11:28:16.171476+0000","last_undegraded":"2023-08-27T11:28:16.171476+0000","last_fullsized":"2023-08-27T11:28:16.171476+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:26.993320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673142+0000","last_change":"2023-08-27T11:28:14.659617+0000","last_active":"2023-08-27T11:28:16.673142+0000","last_peered":"2023-08-27T11:28:16.673142+0000","last_clean":"2023-08-27T11:28:16.673142+0000","last_became_active":"2023-08-27T11:28:14.659303+0000","last_became_peered":"2023-08-27T11:28:14.659303+0000","last_unstale":"2023-08-27T11:28:16.673142+0000","last_undegraded":"2023-08-27T11:28:16.673142+0000","last_fullsized":"2023-08-27T11:28:16.673142+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:53:16.248207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916596+0000","last_change":"2023-08-27T11:28:14.656752+0000","last_active":"2023-08-27T11:28:15.916596+0000","last_peered":"2023-08-27T11:28:15.916596+0000","last_clean":"2023-08-27T11:28:15.916596+0000","last_became_active":"2023-08-27T11:28:14.656474+0000","last_became_peered":"2023-08-27T11:28:14.656474+0000","last_unstale":"2023-08-27T11:28:15.916596+0000","last_undegraded":"2023-08-27T11:28:15.916596+0000","last_fullsized":"2023-08-27T11:28:15.916596+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:09:54.600326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673353+0000","last_change":"2023-08-27T11:28:14.664807+0000","last_active":"2023-08-27T11:28:16.673353+0000","last_peered":"2023-08-27T11:28:16.673353+0000","last_clean":"2023-08-27T11:28:16.673353+0000","last_became_active":"2023-08-27T11:28:14.664469+0000","last_became_peered":"2023-08-27T11:28:14.664469+0000","last_unstale":"2023-08-27T11:28:16.673353+0000","last_undegraded":"2023-08-27T11:28:16.673353+0000","last_fullsized":"2023-08-27T11:28:16.673353+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:30:03.114552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635222+0000","last_change":"2023-08-27T11:28:14.654267+0000","last_active":"2023-08-27T11:28:15.635222+0000","last_peered":"2023-08-27T11:28:15.635222+0000","last_clean":"2023-08-27T11:28:15.635222+0000","last_became_active":"2023-08-27T11:28:14.653971+0000","last_became_peered":"2023-08-27T11:28:14.653971+0000","last_unstale":"2023-08-27T11:28:15.635222+0000","last_undegraded":"2023-08-27T11:28:15.635222+0000","last_fullsized":"2023-08-27T11:28:15.635222+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:07:29.840522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916342+0000","last_change":"2023-08-27T11:28:14.661421+0000","last_active":"2023-08-27T11:28:15.916342+0000","last_peered":"2023-08-27T11:28:15.916342+0000","last_clean":"2023-08-27T11:28:15.916342+0000","last_became_active":"2023-08-27T11:28:14.661064+0000","last_became_peered":"2023-08-27T11:28:14.661064+0000","last_unstale":"2023-08-27T11:28:15.916342+0000","last_undegraded":"2023-08-27T11:28:15.916342+0000","last_fullsized":"2023-08-27T11:28:15.916342+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:02:50.171479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916205+0000","last_change":"2023-08-27T11:28:14.653898+0000","last_active":"2023-08-27T11:28:15.916205+0000","last_peered":"2023-08-27T11:28:15.916205+0000","last_clean":"2023-08-27T11:28:15.916205+0000","last_became_active":"2023-08-27T11:28:14.653430+0000","last_became_peered":"2023-08-27T11:28:14.653430+0000","last_unstale":"2023-08-27T11:28:15.916205+0000","last_undegraded":"2023-08-27T11:28:15.916205+0000","last_fullsized":"2023-08-27T11:28:15.916205+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:57:08.260806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.674628+0000","last_change":"2023-08-27T11:28:14.647766+0000","last_active":"2023-08-27T11:28:16.674628+0000","last_peered":"2023-08-27T11:28:16.674628+0000","last_clean":"2023-08-27T11:28:16.674628+0000","last_became_active":"2023-08-27T11:28:14.647276+0000","last_became_peered":"2023-08-27T11:28:14.647276+0000","last_unstale":"2023-08-27T11:28:16.674628+0000","last_undegraded":"2023-08-27T11:28:16.674628+0000","last_fullsized":"2023-08-27T11:28:16.674628+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:52.480549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.634949+0000","last_change":"2023-08-27T11:28:14.657238+0000","last_active":"2023-08-27T11:28:15.634949+0000","last_peered":"2023-08-27T11:28:15.634949+0000","last_clean":"2023-08-27T11:28:15.634949+0000","last_became_active":"2023-08-27T11:28:14.656196+0000","last_became_peered":"2023-08-27T11:28:14.656196+0000","last_unstale":"2023-08-27T11:28:15.634949+0000","last_undegraded":"2023-08-27T11:28:15.634949+0000","last_fullsized":"2023-08-27T11:28:15.634949+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:34:09.267774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673343+0000","last_change":"2023-08-27T11:28:14.664187+0000","last_active":"2023-08-27T11:28:16.673343+0000","last_peered":"2023-08-27T11:28:16.673343+0000","last_clean":"2023-08-27T11:28:16.673343+0000","last_became_active":"2023-08-27T11:28:14.663654+0000","last_became_peered":"2023-08-27T11:28:14.663654+0000","last_unstale":"2023-08-27T11:28:16.673343+0000","last_undegraded":"2023-08-27T11:28:16.673343+0000","last_fullsized":"2023-08-27T11:28:16.673343+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:30.035009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635518+0000","last_change":"2023-08-27T11:28:13.650131+0000","last_active":"2023-08-27T11:28:15.635518+0000","last_peered":"2023-08-27T11:28:15.635518+0000","last_clean":"2023-08-27T11:28:15.635518+0000","last_became_active":"2023-08-27T11:28:13.649845+0000","last_became_peered":"2023-08-27T11:28:13.649845+0000","last_unstale":"2023-08-27T11:28:15.635518+0000","last_undegraded":"2023-08-27T11:28:15.635518+0000","last_fullsized":"2023-08-27T11:28:15.635518+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:35:46.387936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635352+0000","last_change":"2023-08-27T11:28:14.654305+0000","last_active":"2023-08-27T11:28:15.635352+0000","last_peered":"2023-08-27T11:28:15.635352+0000","last_clean":"2023-08-27T11:28:15.635352+0000","last_became_active":"2023-08-27T11:28:14.654047+0000","last_became_peered":"2023-08-27T11:28:14.654047+0000","last_unstale":"2023-08-27T11:28:15.635352+0000","last_undegraded":"2023-08-27T11:28:15.635352+0000","last_fullsized":"2023-08-27T11:28:15.635352+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:48:53.806730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171845+0000","last_change":"2023-08-27T11:28:13.650587+0000","last_active":"2023-08-27T11:28:16.171845+0000","last_peered":"2023-08-27T11:28:16.171845+0000","last_clean":"2023-08-27T11:28:16.171845+0000","last_became_active":"2023-08-27T11:28:13.650325+0000","last_became_peered":"2023-08-27T11:28:13.650325+0000","last_unstale":"2023-08-27T11:28:16.171845+0000","last_undegraded":"2023-08-27T11:28:16.171845+0000","last_fullsized":"2023-08-27T11:28:16.171845+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:08:28.011180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641379+0000","last_change":"2023-08-27T11:28:14.652957+0000","last_active":"2023-08-27T11:28:15.641379+0000","last_peered":"2023-08-27T11:28:15.641379+0000","last_clean":"2023-08-27T11:28:15.641379+0000","last_became_active":"2023-08-27T11:28:14.652648+0000","last_became_peered":"2023-08-27T11:28:14.652648+0000","last_unstale":"2023-08-27T11:28:15.641379+0000","last_undegraded":"2023-08-27T11:28:15.641379+0000","last_fullsized":"2023-08-27T11:28:15.641379+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:04:56.941782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"138'3","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672711+0000","last_change":"2023-08-27T11:28:13.651281+0000","last_active":"2023-08-27T11:28:16.672711+0000","last_peered":"2023-08-27T11:28:16.672711+0000","last_clean":"2023-08-27T11:28:16.672711+0000","last_became_active":"2023-08-27T11:28:13.651022+0000","last_became_peered":"2023-08-27T11:28:13.651022+0000","last_unstale":"2023-08-27T11:28:16.672711+0000","last_undegraded":"2023-08-27T11:28:16.672711+0000","last_fullsized":"2023-08-27T11:28:16.672711+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:54:01.513256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171881+0000","last_change":"2023-08-27T11:28:14.650678+0000","last_active":"2023-08-27T11:28:16.171881+0000","last_peered":"2023-08-27T11:28:16.171881+0000","last_clean":"2023-08-27T11:28:16.171881+0000","last_became_active":"2023-08-27T11:28:14.649718+0000","last_became_peered":"2023-08-27T11:28:14.649718+0000","last_unstale":"2023-08-27T11:28:16.171881+0000","last_undegraded":"2023-08-27T11:28:16.171881+0000","last_fullsized":"2023-08-27T11:28:16.171881+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:42:24.963359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"138'6","reported_seq":19,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641636+0000","last_change":"2023-08-27T11:28:13.646100+0000","last_active":"2023-08-27T11:28:15.641636+0000","last_peered":"2023-08-27T11:28:15.641636+0000","last_clean":"2023-08-27T11:28:15.641636+0000","last_became_active":"2023-08-27T11:28:13.645775+0000","last_became_peered":"2023-08-27T11:28:13.645775+0000","last_unstale":"2023-08-27T11:28:15.641636+0000","last_undegraded":"2023-08-27T11:28:15.641636+0000","last_fullsized":"2023-08-27T11:28:15.641636+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:56.475941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672505+0000","last_change":"2023-08-27T11:28:14.658845+0000","last_active":"2023-08-27T11:28:16.672505+0000","last_peered":"2023-08-27T11:28:16.672505+0000","last_clean":"2023-08-27T11:28:16.672505+0000","last_became_active":"2023-08-27T11:28:14.658570+0000","last_became_peered":"2023-08-27T11:28:14.658570+0000","last_unstale":"2023-08-27T11:28:16.672505+0000","last_undegraded":"2023-08-27T11:28:16.672505+0000","last_fullsized":"2023-08-27T11:28:16.672505+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:23:34.459876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"138'2","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641345+0000","last_change":"2023-08-27T11:28:13.646623+0000","last_active":"2023-08-27T11:28:15.641345+0000","last_peered":"2023-08-27T11:28:15.641345+0000","last_clean":"2023-08-27T11:28:15.641345+0000","last_became_active":"2023-08-27T11:28:13.646053+0000","last_became_peered":"2023-08-27T11:28:13.646053+0000","last_unstale":"2023-08-27T11:28:15.641345+0000","last_undegraded":"2023-08-27T11:28:15.641345+0000","last_fullsized":"2023-08-27T11:28:15.641345+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:02:08.085302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.634877+0000","last_change":"2023-08-27T11:28:14.653655+0000","last_active":"2023-08-27T11:28:15.634877+0000","last_peered":"2023-08-27T11:28:15.634877+0000","last_clean":"2023-08-27T11:28:15.634877+0000","last_became_active":"2023-08-27T11:28:14.653374+0000","last_became_peered":"2023-08-27T11:28:14.653374+0000","last_unstale":"2023-08-27T11:28:15.634877+0000","last_undegraded":"2023-08-27T11:28:15.634877+0000","last_fullsized":"2023-08-27T11:28:15.634877+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:28:44.942493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.172145+0000","last_change":"2023-08-27T11:28:13.648211+0000","last_active":"2023-08-27T11:28:16.172145+0000","last_peered":"2023-08-27T11:28:16.172145+0000","last_clean":"2023-08-27T11:28:16.172145+0000","last_became_active":"2023-08-27T11:28:13.647926+0000","last_became_peered":"2023-08-27T11:28:13.647926+0000","last_unstale":"2023-08-27T11:28:16.172145+0000","last_undegraded":"2023-08-27T11:28:16.172145+0000","last_fullsized":"2023-08-27T11:28:16.172145+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:30:50.057457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641278+0000","last_change":"2023-08-27T11:28:14.664867+0000","last_active":"2023-08-27T11:28:15.641278+0000","last_peered":"2023-08-27T11:28:15.641278+0000","last_clean":"2023-08-27T11:28:15.641278+0000","last_became_active":"2023-08-27T11:28:14.664589+0000","last_became_peered":"2023-08-27T11:28:14.664589+0000","last_unstale":"2023-08-27T11:28:15.641278+0000","last_undegraded":"2023-08-27T11:28:15.641278+0000","last_fullsized":"2023-08-27T11:28:15.641278+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:40:33.267545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672922+0000","last_change":"2023-08-27T11:28:13.644841+0000","last_active":"2023-08-27T11:28:16.672922+0000","last_peered":"2023-08-27T11:28:16.672922+0000","last_clean":"2023-08-27T11:28:16.672922+0000","last_became_active":"2023-08-27T11:28:13.644589+0000","last_became_peered":"2023-08-27T11:28:13.644589+0000","last_unstale":"2023-08-27T11:28:16.672922+0000","last_undegraded":"2023-08-27T11:28:16.672922+0000","last_fullsized":"2023-08-27T11:28:16.672922+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:16:40.302148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641417+0000","last_change":"2023-08-27T11:28:14.660190+0000","last_active":"2023-08-27T11:28:15.641417+0000","last_peered":"2023-08-27T11:28:15.641417+0000","last_clean":"2023-08-27T11:28:15.641417+0000","last_became_active":"2023-08-27T11:28:14.659884+0000","last_became_peered":"2023-08-27T11:28:14.659884+0000","last_unstale":"2023-08-27T11:28:15.641417+0000","last_undegraded":"2023-08-27T11:28:15.641417+0000","last_fullsized":"2023-08-27T11:28:15.641417+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:13:53.734648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"138'8","reported_seq":21,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916488+0000","last_change":"2023-08-27T11:28:13.647523+0000","last_active":"2023-08-27T11:28:15.916488+0000","last_peered":"2023-08-27T11:28:15.916488+0000","last_clean":"2023-08-27T11:28:15.916488+0000","last_became_active":"2023-08-27T11:28:13.647233+0000","last_became_peered":"2023-08-27T11:28:13.647233+0000","last_unstale":"2023-08-27T11:28:15.916488+0000","last_undegraded":"2023-08-27T11:28:15.916488+0000","last_fullsized":"2023-08-27T11:28:15.916488+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:02:09.845015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673037+0000","last_change":"2023-08-27T11:28:14.641836+0000","last_active":"2023-08-27T11:28:16.673037+0000","last_peered":"2023-08-27T11:28:16.673037+0000","last_clean":"2023-08-27T11:28:16.673037+0000","last_became_active":"2023-08-27T11:28:14.641570+0000","last_became_peered":"2023-08-27T11:28:14.641570+0000","last_unstale":"2023-08-27T11:28:16.673037+0000","last_undegraded":"2023-08-27T11:28:16.673037+0000","last_fullsized":"2023-08-27T11:28:16.673037+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:27:26.727128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"138'1","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916633+0000","last_change":"2023-08-27T11:28:13.652658+0000","last_active":"2023-08-27T11:28:15.916633+0000","last_peered":"2023-08-27T11:28:15.916633+0000","last_clean":"2023-08-27T11:28:15.916633+0000","last_became_active":"2023-08-27T11:28:13.652329+0000","last_became_peered":"2023-08-27T11:28:13.652329+0000","last_unstale":"2023-08-27T11:28:15.916633+0000","last_undegraded":"2023-08-27T11:28:15.916633+0000","last_fullsized":"2023-08-27T11:28:15.916633+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:47:49.956438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635361+0000","last_change":"2023-08-27T11:28:14.656548+0000","last_active":"2023-08-27T11:28:15.635361+0000","last_peered":"2023-08-27T11:28:15.635361+0000","last_clean":"2023-08-27T11:28:15.635361+0000","last_became_active":"2023-08-27T11:28:14.656044+0000","last_became_peered":"2023-08-27T11:28:14.656044+0000","last_unstale":"2023-08-27T11:28:15.635361+0000","last_undegraded":"2023-08-27T11:28:15.635361+0000","last_fullsized":"2023-08-27T11:28:15.635361+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:54.650454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"138'2","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672792+0000","last_change":"2023-08-27T11:28:13.644465+0000","last_active":"2023-08-27T11:28:16.672792+0000","last_peered":"2023-08-27T11:28:16.672792+0000","last_clean":"2023-08-27T11:28:16.672792+0000","last_became_active":"2023-08-27T11:28:13.644135+0000","last_became_peered":"2023-08-27T11:28:13.644135+0000","last_unstale":"2023-08-27T11:28:16.672792+0000","last_undegraded":"2023-08-27T11:28:16.672792+0000","last_fullsized":"2023-08-27T11:28:16.672792+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:57:22.012755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916693+0000","last_change":"2023-08-27T11:28:14.656696+0000","last_active":"2023-08-27T11:28:15.916693+0000","last_peered":"2023-08-27T11:28:15.916693+0000","last_clean":"2023-08-27T11:28:15.916693+0000","last_became_active":"2023-08-27T11:28:14.656419+0000","last_became_peered":"2023-08-27T11:28:14.656419+0000","last_unstale":"2023-08-27T11:28:15.916693+0000","last_undegraded":"2023-08-27T11:28:15.916693+0000","last_fullsized":"2023-08-27T11:28:15.916693+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:36:41.090504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"138'1","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640294+0000","last_change":"2023-08-27T11:28:13.648731+0000","last_active":"2023-08-27T11:28:15.640294+0000","last_peered":"2023-08-27T11:28:15.640294+0000","last_clean":"2023-08-27T11:28:15.640294+0000","last_became_active":"2023-08-27T11:28:13.648503+0000","last_became_peered":"2023-08-27T11:28:13.648503+0000","last_unstale":"2023-08-27T11:28:15.640294+0000","last_undegraded":"2023-08-27T11:28:15.640294+0000","last_fullsized":"2023-08-27T11:28:15.640294+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:08.667418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171931+0000","last_change":"2023-08-27T11:28:14.659900+0000","last_active":"2023-08-27T11:28:16.171931+0000","last_peered":"2023-08-27T11:28:16.171931+0000","last_clean":"2023-08-27T11:28:16.171931+0000","last_became_active":"2023-08-27T11:28:14.659598+0000","last_became_peered":"2023-08-27T11:28:14.659598+0000","last_unstale":"2023-08-27T11:28:16.171931+0000","last_undegraded":"2023-08-27T11:28:16.171931+0000","last_fullsized":"2023-08-27T11:28:16.171931+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:28:44.942493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916440+0000","last_change":"2023-08-27T11:28:13.647955+0000","last_active":"2023-08-27T11:28:15.916440+0000","last_peered":"2023-08-27T11:28:15.916440+0000","last_clean":"2023-08-27T11:28:15.916440+0000","last_became_active":"2023-08-27T11:28:13.647696+0000","last_became_peered":"2023-08-27T11:28:13.647696+0000","last_unstale":"2023-08-27T11:28:15.916440+0000","last_undegraded":"2023-08-27T11:28:15.916440+0000","last_fullsized":"2023-08-27T11:28:15.916440+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:09:26.898187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171472+0000","last_change":"2023-08-27T11:28:14.660921+0000","last_active":"2023-08-27T11:28:16.171472+0000","last_peered":"2023-08-27T11:28:16.171472+0000","last_clean":"2023-08-27T11:28:16.171472+0000","last_became_active":"2023-08-27T11:28:14.660473+0000","last_became_peered":"2023-08-27T11:28:14.660473+0000","last_unstale":"2023-08-27T11:28:16.171472+0000","last_undegraded":"2023-08-27T11:28:16.171472+0000","last_fullsized":"2023-08-27T11:28:16.171472+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:20:20.806596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"138'2","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641535+0000","last_change":"2023-08-27T11:28:13.650453+0000","last_active":"2023-08-27T11:28:15.641535+0000","last_peered":"2023-08-27T11:28:15.641535+0000","last_clean":"2023-08-27T11:28:15.641535+0000","last_became_active":"2023-08-27T11:28:13.650244+0000","last_became_peered":"2023-08-27T11:28:13.650244+0000","last_unstale":"2023-08-27T11:28:15.641535+0000","last_undegraded":"2023-08-27T11:28:15.641535+0000","last_fullsized":"2023-08-27T11:28:15.641535+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:00.535127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672779+0000","last_change":"2023-08-27T11:28:14.647979+0000","last_active":"2023-08-27T11:28:16.672779+0000","last_peered":"2023-08-27T11:28:16.672779+0000","last_clean":"2023-08-27T11:28:16.672779+0000","last_became_active":"2023-08-27T11:28:14.642687+0000","last_became_peered":"2023-08-27T11:28:14.642687+0000","last_unstale":"2023-08-27T11:28:16.672779+0000","last_undegraded":"2023-08-27T11:28:16.672779+0000","last_fullsized":"2023-08-27T11:28:16.672779+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:05.224224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673262+0000","last_change":"2023-08-27T11:28:13.647815+0000","last_active":"2023-08-27T11:28:16.673262+0000","last_peered":"2023-08-27T11:28:16.673262+0000","last_clean":"2023-08-27T11:28:16.673262+0000","last_became_active":"2023-08-27T11:28:13.647390+0000","last_became_peered":"2023-08-27T11:28:13.647390+0000","last_unstale":"2023-08-27T11:28:16.673262+0000","last_undegraded":"2023-08-27T11:28:16.673262+0000","last_fullsized":"2023-08-27T11:28:16.673262+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:45.914735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672801+0000","last_change":"2023-08-27T11:28:14.649394+0000","last_active":"2023-08-27T11:28:16.672801+0000","last_peered":"2023-08-27T11:28:16.672801+0000","last_clean":"2023-08-27T11:28:16.672801+0000","last_became_active":"2023-08-27T11:28:14.649110+0000","last_became_peered":"2023-08-27T11:28:14.649110+0000","last_unstale":"2023-08-27T11:28:16.672801+0000","last_undegraded":"2023-08-27T11:28:16.672801+0000","last_fullsized":"2023-08-27T11:28:16.672801+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:00:04.157887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.674016+0000","last_change":"2023-08-27T11:28:13.648193+0000","last_active":"2023-08-27T11:28:16.674016+0000","last_peered":"2023-08-27T11:28:16.674016+0000","last_clean":"2023-08-27T11:28:16.674016+0000","last_became_active":"2023-08-27T11:28:13.647916+0000","last_became_peered":"2023-08-27T11:28:13.647916+0000","last_unstale":"2023-08-27T11:28:16.674016+0000","last_undegraded":"2023-08-27T11:28:16.674016+0000","last_fullsized":"2023-08-27T11:28:16.674016+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:52:37.401119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635197+0000","last_change":"2023-08-27T11:28:14.656981+0000","last_active":"2023-08-27T11:28:15.635197+0000","last_peered":"2023-08-27T11:28:15.635197+0000","last_clean":"2023-08-27T11:28:15.635197+0000","last_became_active":"2023-08-27T11:28:14.656410+0000","last_became_peered":"2023-08-27T11:28:14.656410+0000","last_unstale":"2023-08-27T11:28:15.635197+0000","last_undegraded":"2023-08-27T11:28:15.635197+0000","last_fullsized":"2023-08-27T11:28:15.635197+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:09.674358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.639915+0000","last_change":"2023-08-27T11:28:13.644528+0000","last_active":"2023-08-27T11:28:15.639915+0000","last_peered":"2023-08-27T11:28:15.639915+0000","last_clean":"2023-08-27T11:28:15.639915+0000","last_became_active":"2023-08-27T11:28:13.644225+0000","last_became_peered":"2023-08-27T11:28:13.644225+0000","last_unstale":"2023-08-27T11:28:15.639915+0000","last_undegraded":"2023-08-27T11:28:15.639915+0000","last_fullsized":"2023-08-27T11:28:15.639915+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:34:08.260834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672827+0000","last_change":"2023-08-27T11:28:14.648888+0000","last_active":"2023-08-27T11:28:16.672827+0000","last_peered":"2023-08-27T11:28:16.672827+0000","last_clean":"2023-08-27T11:28:16.672827+0000","last_became_active":"2023-08-27T11:28:14.648593+0000","last_became_peered":"2023-08-27T11:28:14.648593+0000","last_unstale":"2023-08-27T11:28:16.672827+0000","last_undegraded":"2023-08-27T11:28:16.672827+0000","last_fullsized":"2023-08-27T11:28:16.672827+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:02:45.668209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"138'2","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673652+0000","last_change":"2023-08-27T11:28:13.652305+0000","last_active":"2023-08-27T11:28:16.673652+0000","last_peered":"2023-08-27T11:28:16.673652+0000","last_clean":"2023-08-27T11:28:16.673652+0000","last_became_active":"2023-08-27T11:28:13.652034+0000","last_became_peered":"2023-08-27T11:28:13.652034+0000","last_unstale":"2023-08-27T11:28:16.673652+0000","last_undegraded":"2023-08-27T11:28:16.673652+0000","last_fullsized":"2023-08-27T11:28:16.673652+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:03:30.384686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171729+0000","last_change":"2023-08-27T11:28:14.645680+0000","last_active":"2023-08-27T11:28:16.171729+0000","last_peered":"2023-08-27T11:28:16.171729+0000","last_clean":"2023-08-27T11:28:16.171729+0000","last_became_active":"2023-08-27T11:28:14.643431+0000","last_became_peered":"2023-08-27T11:28:14.643431+0000","last_unstale":"2023-08-27T11:28:16.171729+0000","last_undegraded":"2023-08-27T11:28:16.171729+0000","last_fullsized":"2023-08-27T11:28:16.171729+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:24:04.698188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673628+0000","last_change":"2023-08-27T11:28:13.648441+0000","last_active":"2023-08-27T11:28:16.673628+0000","last_peered":"2023-08-27T11:28:16.673628+0000","last_clean":"2023-08-27T11:28:16.673628+0000","last_became_active":"2023-08-27T11:28:13.648177+0000","last_became_peered":"2023-08-27T11:28:13.648177+0000","last_unstale":"2023-08-27T11:28:16.673628+0000","last_undegraded":"2023-08-27T11:28:16.673628+0000","last_fullsized":"2023-08-27T11:28:16.673628+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:45:23.429246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673436+0000","last_change":"2023-08-27T11:28:14.666855+0000","last_active":"2023-08-27T11:28:16.673436+0000","last_peered":"2023-08-27T11:28:16.673436+0000","last_clean":"2023-08-27T11:28:16.673436+0000","last_became_active":"2023-08-27T11:28:14.665012+0000","last_became_peered":"2023-08-27T11:28:14.665012+0000","last_unstale":"2023-08-27T11:28:16.673436+0000","last_undegraded":"2023-08-27T11:28:16.673436+0000","last_fullsized":"2023-08-27T11:28:16.673436+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:24.301561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635473+0000","last_change":"2023-08-27T11:28:13.646624+0000","last_active":"2023-08-27T11:28:15.635473+0000","last_peered":"2023-08-27T11:28:15.635473+0000","last_clean":"2023-08-27T11:28:15.635473+0000","last_became_active":"2023-08-27T11:28:13.646207+0000","last_became_peered":"2023-08-27T11:28:13.646207+0000","last_unstale":"2023-08-27T11:28:15.635473+0000","last_undegraded":"2023-08-27T11:28:15.635473+0000","last_fullsized":"2023-08-27T11:28:15.635473+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:53.054545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673287+0000","last_change":"2023-08-27T11:28:14.648725+0000","last_active":"2023-08-27T11:28:16.673287+0000","last_peered":"2023-08-27T11:28:16.673287+0000","last_clean":"2023-08-27T11:28:16.673287+0000","last_became_active":"2023-08-27T11:28:14.648299+0000","last_became_peered":"2023-08-27T11:28:14.648299+0000","last_unstale":"2023-08-27T11:28:16.673287+0000","last_undegraded":"2023-08-27T11:28:16.673287+0000","last_fullsized":"2023-08-27T11:28:16.673287+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:07:59.656326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635632+0000","last_change":"2023-08-27T11:28:13.648441+0000","last_active":"2023-08-27T11:28:15.635632+0000","last_peered":"2023-08-27T11:28:15.635632+0000","last_clean":"2023-08-27T11:28:15.635632+0000","last_became_active":"2023-08-27T11:28:13.648179+0000","last_became_peered":"2023-08-27T11:28:13.648179+0000","last_unstale":"2023-08-27T11:28:15.635632+0000","last_undegraded":"2023-08-27T11:28:15.635632+0000","last_fullsized":"2023-08-27T11:28:15.635632+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:21:35.536688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.639913+0000","last_change":"2023-08-27T11:28:14.650435+0000","last_active":"2023-08-27T11:28:15.639913+0000","last_peered":"2023-08-27T11:28:15.639913+0000","last_clean":"2023-08-27T11:28:15.639913+0000","last_became_active":"2023-08-27T11:28:14.650137+0000","last_became_peered":"2023-08-27T11:28:14.650137+0000","last_unstale":"2023-08-27T11:28:15.639913+0000","last_undegraded":"2023-08-27T11:28:15.639913+0000","last_fullsized":"2023-08-27T11:28:15.639913+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:27:29.886325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673059+0000","last_change":"2023-08-27T11:28:13.645381+0000","last_active":"2023-08-27T11:28:16.673059+0000","last_peered":"2023-08-27T11:28:16.673059+0000","last_clean":"2023-08-27T11:28:16.673059+0000","last_became_active":"2023-08-27T11:28:13.645065+0000","last_became_peered":"2023-08-27T11:28:13.645065+0000","last_unstale":"2023-08-27T11:28:16.673059+0000","last_undegraded":"2023-08-27T11:28:16.673059+0000","last_fullsized":"2023-08-27T11:28:16.673059+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:40:03.185204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641407+0000","last_change":"2023-08-27T11:28:14.666004+0000","last_active":"2023-08-27T11:28:15.641407+0000","last_peered":"2023-08-27T11:28:15.641407+0000","last_clean":"2023-08-27T11:28:15.641407+0000","last_became_active":"2023-08-27T11:28:14.665530+0000","last_became_peered":"2023-08-27T11:28:14.665530+0000","last_unstale":"2023-08-27T11:28:15.641407+0000","last_undegraded":"2023-08-27T11:28:15.641407+0000","last_fullsized":"2023-08-27T11:28:15.641407+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:27:57.013012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673140+0000","last_change":"2023-08-27T11:28:13.652345+0000","last_active":"2023-08-27T11:28:16.673140+0000","last_peered":"2023-08-27T11:28:16.673140+0000","last_clean":"2023-08-27T11:28:16.673140+0000","last_became_active":"2023-08-27T11:28:13.652061+0000","last_became_peered":"2023-08-27T11:28:13.652061+0000","last_unstale":"2023-08-27T11:28:16.673140+0000","last_undegraded":"2023-08-27T11:28:16.673140+0000","last_fullsized":"2023-08-27T11:28:16.673140+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:57:54.060241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.172060+0000","last_change":"2023-08-27T11:28:14.660024+0000","last_active":"2023-08-27T11:28:16.172060+0000","last_peered":"2023-08-27T11:28:16.172060+0000","last_clean":"2023-08-27T11:28:16.172060+0000","last_became_active":"2023-08-27T11:28:14.659767+0000","last_became_peered":"2023-08-27T11:28:14.659767+0000","last_unstale":"2023-08-27T11:28:16.172060+0000","last_undegraded":"2023-08-27T11:28:16.172060+0000","last_fullsized":"2023-08-27T11:28:16.172060+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:50.800979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.674171+0000","last_change":"2023-08-27T11:28:13.638797+0000","last_active":"2023-08-27T11:28:16.674171+0000","last_peered":"2023-08-27T11:28:16.674171+0000","last_clean":"2023-08-27T11:28:16.674171+0000","last_became_active":"2023-08-27T11:28:13.638526+0000","last_became_peered":"2023-08-27T11:28:13.638526+0000","last_unstale":"2023-08-27T11:28:16.674171+0000","last_undegraded":"2023-08-27T11:28:16.674171+0000","last_fullsized":"2023-08-27T11:28:16.674171+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:44.619300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673010+0000","last_change":"2023-08-27T11:28:14.663595+0000","last_active":"2023-08-27T11:28:16.673010+0000","last_peered":"2023-08-27T11:28:16.673010+0000","last_clean":"2023-08-27T11:28:16.673010+0000","last_became_active":"2023-08-27T11:28:14.663269+0000","last_became_peered":"2023-08-27T11:28:14.663269+0000","last_unstale":"2023-08-27T11:28:16.673010+0000","last_undegraded":"2023-08-27T11:28:16.673010+0000","last_fullsized":"2023-08-27T11:28:16.673010+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:28:31.283625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640292+0000","last_change":"2023-08-27T11:28:13.638139+0000","last_active":"2023-08-27T11:28:15.640292+0000","last_peered":"2023-08-27T11:28:15.640292+0000","last_clean":"2023-08-27T11:28:15.640292+0000","last_became_active":"2023-08-27T11:28:13.637703+0000","last_became_peered":"2023-08-27T11:28:13.637703+0000","last_unstale":"2023-08-27T11:28:15.640292+0000","last_undegraded":"2023-08-27T11:28:15.640292+0000","last_fullsized":"2023-08-27T11:28:15.640292+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:07:28.833582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673521+0000","last_change":"2023-08-27T11:28:14.663944+0000","last_active":"2023-08-27T11:28:16.673521+0000","last_peered":"2023-08-27T11:28:16.673521+0000","last_clean":"2023-08-27T11:28:16.673521+0000","last_became_active":"2023-08-27T11:28:14.663230+0000","last_became_peered":"2023-08-27T11:28:14.663230+0000","last_unstale":"2023-08-27T11:28:16.673521+0000","last_undegraded":"2023-08-27T11:28:16.673521+0000","last_fullsized":"2023-08-27T11:28:16.673521+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:15:26.373192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"138'4","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673526+0000","last_change":"2023-08-27T11:28:13.649633+0000","last_active":"2023-08-27T11:28:16.673526+0000","last_peered":"2023-08-27T11:28:16.673526+0000","last_clean":"2023-08-27T11:28:16.673526+0000","last_became_active":"2023-08-27T11:28:13.649362+0000","last_became_peered":"2023-08-27T11:28:13.649362+0000","last_unstale":"2023-08-27T11:28:16.673526+0000","last_undegraded":"2023-08-27T11:28:16.673526+0000","last_fullsized":"2023-08-27T11:28:16.673526+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:25:32.461248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":262,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640206+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:28:15.640206+0000","last_peered":"2023-08-27T11:28:15.640206+0000","last_clean":"2023-08-27T11:28:15.640206+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:28:15.640206+0000","last_undegraded":"2023-08-27T11:28:15.640206+0000","last_fullsized":"2023-08-27T11:28:15.640206+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:43:47.585658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673237+0000","last_change":"2023-08-27T11:28:14.653814+0000","last_active":"2023-08-27T11:28:16.673237+0000","last_peered":"2023-08-27T11:28:16.673237+0000","last_clean":"2023-08-27T11:28:16.673237+0000","last_became_active":"2023-08-27T11:28:14.652929+0000","last_became_peered":"2023-08-27T11:28:14.652929+0000","last_unstale":"2023-08-27T11:28:16.673237+0000","last_undegraded":"2023-08-27T11:28:16.673237+0000","last_fullsized":"2023-08-27T11:28:16.673237+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:40:04.831481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"138'1","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635185+0000","last_change":"2023-08-27T11:28:13.646599+0000","last_active":"2023-08-27T11:28:15.635185+0000","last_peered":"2023-08-27T11:28:15.635185+0000","last_clean":"2023-08-27T11:28:15.635185+0000","last_became_active":"2023-08-27T11:28:13.646235+0000","last_became_peered":"2023-08-27T11:28:13.646235+0000","last_unstale":"2023-08-27T11:28:15.635185+0000","last_undegraded":"2023-08-27T11:28:15.635185+0000","last_fullsized":"2023-08-27T11:28:15.635185+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:30:50.057457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":262,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640145+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:28:15.640145+0000","last_peered":"2023-08-27T11:28:15.640145+0000","last_clean":"2023-08-27T11:28:15.640145+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:28:15.640145+0000","last_undegraded":"2023-08-27T11:28:15.640145+0000","last_fullsized":"2023-08-27T11:28:15.640145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:48:35.482692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.172073+0000","last_change":"2023-08-27T11:28:14.643550+0000","last_active":"2023-08-27T11:28:16.172073+0000","last_peered":"2023-08-27T11:28:16.172073+0000","last_clean":"2023-08-27T11:28:16.172073+0000","last_became_active":"2023-08-27T11:28:14.643204+0000","last_became_peered":"2023-08-27T11:28:14.643204+0000","last_unstale":"2023-08-27T11:28:16.172073+0000","last_undegraded":"2023-08-27T11:28:16.172073+0000","last_fullsized":"2023-08-27T11:28:16.172073+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:55:02.903945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641244+0000","last_change":"2023-08-27T11:28:13.644990+0000","last_active":"2023-08-27T11:28:15.641244+0000","last_peered":"2023-08-27T11:28:15.641244+0000","last_clean":"2023-08-27T11:28:15.641244+0000","last_became_active":"2023-08-27T11:28:13.644707+0000","last_became_peered":"2023-08-27T11:28:13.644707+0000","last_unstale":"2023-08-27T11:28:15.641244+0000","last_undegraded":"2023-08-27T11:28:15.641244+0000","last_fullsized":"2023-08-27T11:28:15.641244+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:46.405496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":224,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916097+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:28:15.916097+0000","last_peered":"2023-08-27T11:28:15.916097+0000","last_clean":"2023-08-27T11:28:15.916097+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:28:15.916097+0000","last_undegraded":"2023-08-27T11:28:15.916097+0000","last_fullsized":"2023-08-27T11:28:15.916097+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:43:38.425879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640278+0000","last_change":"2023-08-27T11:28:14.661294+0000","last_active":"2023-08-27T11:28:15.640278+0000","last_peered":"2023-08-27T11:28:15.640278+0000","last_clean":"2023-08-27T11:28:15.640278+0000","last_became_active":"2023-08-27T11:28:14.660937+0000","last_became_peered":"2023-08-27T11:28:14.660937+0000","last_unstale":"2023-08-27T11:28:15.640278+0000","last_undegraded":"2023-08-27T11:28:15.640278+0000","last_fullsized":"2023-08-27T11:28:15.640278+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:30:51.064396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672907+0000","last_change":"2023-08-27T11:28:13.645934+0000","last_active":"2023-08-27T11:28:16.672907+0000","last_peered":"2023-08-27T11:28:16.672907+0000","last_clean":"2023-08-27T11:28:16.672907+0000","last_became_active":"2023-08-27T11:28:13.645672+0000","last_became_peered":"2023-08-27T11:28:13.645672+0000","last_unstale":"2023-08-27T11:28:16.672907+0000","last_undegraded":"2023-08-27T11:28:16.672907+0000","last_fullsized":"2023-08-27T11:28:16.672907+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:23:15.755614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":220,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673076+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:28:16.673076+0000","last_peered":"2023-08-27T11:28:16.673076+0000","last_clean":"2023-08-27T11:28:16.673076+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:28:16.673076+0000","last_undegraded":"2023-08-27T11:28:16.673076+0000","last_fullsized":"2023-08-27T11:28:16.673076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:32.388513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171668+0000","last_change":"2023-08-27T11:28:14.646006+0000","last_active":"2023-08-27T11:28:16.171668+0000","last_peered":"2023-08-27T11:28:16.171668+0000","last_clean":"2023-08-27T11:28:16.171668+0000","last_became_active":"2023-08-27T11:28:14.645708+0000","last_became_peered":"2023-08-27T11:28:14.645708+0000","last_unstale":"2023-08-27T11:28:16.171668+0000","last_undegraded":"2023-08-27T11:28:16.171668+0000","last_fullsized":"2023-08-27T11:28:16.171668+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:35:31.159606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"138'1","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673394+0000","last_change":"2023-08-27T11:28:13.650359+0000","last_active":"2023-08-27T11:28:16.673394+0000","last_peered":"2023-08-27T11:28:16.673394+0000","last_clean":"2023-08-27T11:28:16.673394+0000","last_became_active":"2023-08-27T11:28:13.650102+0000","last_became_peered":"2023-08-27T11:28:13.650102+0000","last_unstale":"2023-08-27T11:28:16.673394+0000","last_undegraded":"2023-08-27T11:28:16.673394+0000","last_fullsized":"2023-08-27T11:28:16.673394+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:56:03.277328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":220,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672904+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:28:16.672904+0000","last_peered":"2023-08-27T11:28:16.672904+0000","last_clean":"2023-08-27T11:28:16.672904+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:28:16.672904+0000","last_undegraded":"2023-08-27T11:28:16.672904+0000","last_fullsized":"2023-08-27T11:28:16.672904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:09:49.885071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640920+0000","last_change":"2023-08-27T11:28:14.657479+0000","last_active":"2023-08-27T11:28:15.640920+0000","last_peered":"2023-08-27T11:28:15.640920+0000","last_clean":"2023-08-27T11:28:15.640920+0000","last_became_active":"2023-08-27T11:28:14.657095+0000","last_became_peered":"2023-08-27T11:28:14.657095+0000","last_unstale":"2023-08-27T11:28:15.640920+0000","last_undegraded":"2023-08-27T11:28:15.640920+0000","last_fullsized":"2023-08-27T11:28:15.640920+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:00:09.041184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"138'1","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635274+0000","last_change":"2023-08-27T11:28:13.646539+0000","last_active":"2023-08-27T11:28:15.635274+0000","last_peered":"2023-08-27T11:28:15.635274+0000","last_clean":"2023-08-27T11:28:15.635274+0000","last_became_active":"2023-08-27T11:28:13.646217+0000","last_became_peered":"2023-08-27T11:28:13.646217+0000","last_unstale":"2023-08-27T11:28:15.635274+0000","last_undegraded":"2023-08-27T11:28:15.635274+0000","last_fullsized":"2023-08-27T11:28:15.635274+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:42:08.449313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":370,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673014+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:28:16.673014+0000","last_peered":"2023-08-27T11:28:16.673014+0000","last_clean":"2023-08-27T11:28:16.673014+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:28:16.673014+0000","last_undegraded":"2023-08-27T11:28:16.673014+0000","last_fullsized":"2023-08-27T11:28:16.673014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:11:01.893595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":230,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171914+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:28:16.171914+0000","last_peered":"2023-08-27T11:28:16.171914+0000","last_clean":"2023-08-27T11:28:16.171914+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:28:16.171914+0000","last_undegraded":"2023-08-27T11:28:16.171914+0000","last_fullsized":"2023-08-27T11:28:16.171914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:39:48.116381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640161+0000","last_change":"2023-08-27T11:28:14.654482+0000","last_active":"2023-08-27T11:28:15.640161+0000","last_peered":"2023-08-27T11:28:15.640161+0000","last_clean":"2023-08-27T11:28:15.640161+0000","last_became_active":"2023-08-27T11:28:14.654067+0000","last_became_peered":"2023-08-27T11:28:14.654067+0000","last_unstale":"2023-08-27T11:28:15.640161+0000","last_undegraded":"2023-08-27T11:28:15.640161+0000","last_fullsized":"2023-08-27T11:28:15.640161+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:55:02.771821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"138'2","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641515+0000","last_change":"2023-08-27T11:28:13.645061+0000","last_active":"2023-08-27T11:28:15.641515+0000","last_peered":"2023-08-27T11:28:15.641515+0000","last_clean":"2023-08-27T11:28:15.641515+0000","last_became_active":"2023-08-27T11:28:13.644717+0000","last_became_peered":"2023-08-27T11:28:13.644717+0000","last_unstale":"2023-08-27T11:28:15.641515+0000","last_undegraded":"2023-08-27T11:28:15.641515+0000","last_fullsized":"2023-08-27T11:28:15.641515+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:55:27.308469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":224,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635349+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:28:15.635349+0000","last_peered":"2023-08-27T11:28:15.635349+0000","last_clean":"2023-08-27T11:28:15.635349+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:28:15.635349+0000","last_undegraded":"2023-08-27T11:28:15.635349+0000","last_fullsized":"2023-08-27T11:28:15.635349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:45:26.488760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916627+0000","last_change":"2023-08-27T11:28:14.655078+0000","last_active":"2023-08-27T11:28:15.916627+0000","last_peered":"2023-08-27T11:28:15.916627+0000","last_clean":"2023-08-27T11:28:15.916627+0000","last_became_active":"2023-08-27T11:28:14.654406+0000","last_became_peered":"2023-08-27T11:28:14.654406+0000","last_unstale":"2023-08-27T11:28:15.916627+0000","last_undegraded":"2023-08-27T11:28:15.916627+0000","last_fullsized":"2023-08-27T11:28:15.916627+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:36:29.030283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"138'2","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916441+0000","last_change":"2023-08-27T11:28:13.645173+0000","last_active":"2023-08-27T11:28:15.916441+0000","last_peered":"2023-08-27T11:28:15.916441+0000","last_clean":"2023-08-27T11:28:15.916441+0000","last_became_active":"2023-08-27T11:28:13.644874+0000","last_became_peered":"2023-08-27T11:28:13.644874+0000","last_unstale":"2023-08-27T11:28:15.916441+0000","last_undegraded":"2023-08-27T11:28:15.916441+0000","last_fullsized":"2023-08-27T11:28:15.916441+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:36:44.633556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":250,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641553+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:28:15.641553+0000","last_peered":"2023-08-27T11:28:15.641553+0000","last_clean":"2023-08-27T11:28:15.641553+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:28:15.641553+0000","last_undegraded":"2023-08-27T11:28:15.641553+0000","last_fullsized":"2023-08-27T11:28:15.641553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:37:28.710975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2338816,"data_stored":1419180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.44,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.72799999999999998,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.39800000000000002,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.75600000000000001,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.51900000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55700000000000005,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.872},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54000000000000004,"15min":0.52500000000000002},"min":{"1min":0.374,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.47199999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.497},"min":{"1min":0.377,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.72399999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.437},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.46100000000000002,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.90200000000000002},"last":0.57199999999999995}]},{"osd":3,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.443,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77700000000000002,"5min":0.996,"15min":0.996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.48899999999999999,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.76400000000000001,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.66500000000000004}]},{"osd":4,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.496,"15min":0.504},"min":{"1min":0.35899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.42399999999999999,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.73199999999999998,"5min":1.224,"15min":1.224},"last":0.59899999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.78100000000000003,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73199999999999998,"5min":0.78400000000000003,"15min":0.84799999999999998},"last":0.63300000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84499999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.47299999999999998,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.65100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1761280,"data_stored":835535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.29099999999999998},"max":{"1min":0.73399999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.374},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80300000000000005,"5min":0.83799999999999997,"15min":0.89800000000000002},"last":0.54000000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.873},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.84399999999999997},"last":0.47899999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.85899999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.69699999999999995,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.52400000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57599999999999996,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84499999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.80200000000000005,"5min":0.80700000000000005,"15min":0.93799999999999994},"last":0.504}]},{"osd":4,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.753,"5min":0.753,"15min":0.83099999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.374},"max":{"1min":0.81399999999999995,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.61199999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.875},"last":0.625}]},{"osd":7,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.34899999999999998},"max":{"1min":0.82999999999999996,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.59699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574988,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33476,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338364,"statfs":{"total":96636764160,"available":96602484736,"internally_reserved":0,"allocated":2351104,"data_stored":1425833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.311,"15min":0.55400000000000005},"min":{"1min":0.19500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.42599999999999999,"5min":0.502,"15min":11.971},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.30099999999999999,"5min":0.33200000000000002,"15min":0.80800000000000005},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.39500000000000002,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.34000000000000002,"15min":0.58299999999999996},"min":{"1min":0.27700000000000002,"5min":0.23400000000000001,"15min":0.221},"max":{"1min":0.45500000000000002,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.317,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.22800000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.38800000000000001,"5min":0.52000000000000002,"15min":0.58599999999999997},"last":0.52200000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38800000000000001,"15min":0.60399999999999998},"min":{"1min":0.27500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.47399999999999998,"5min":0.58099999999999996,"15min":10.99},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.372,"15min":0.84499999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23499999999999999},"max":{"1min":0.45000000000000001,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.44700000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57399999999999995,"15min":0.59499999999999997},"min":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.33400000000000002},"max":{"1min":0.79500000000000004,"5min":0.90700000000000003,"15min":1.26},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.77300000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.69999999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":1.0409999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36199999999999999},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.46300000000000002,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.71599999999999997,"5min":0.88700000000000001,"15min":1.296},"last":0.67900000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.80400000000000005},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.371},"max":{"1min":0.83599999999999997,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":1.0589999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81200000000000006,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.71399999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":1.0529999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.84199999999999997,"15min":22.988},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.85899999999999999,"5min":0.86399999999999999,"15min":1.327},"last":0.72799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1757184,"data_stored":828179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.311},"min":{"1min":0.223,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.46200000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.33300000000000002,"15min":0.34899999999999998},"min":{"1min":0.253,"5min":0.223,"15min":0.223},"max":{"1min":0.5,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.38100000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34100000000000003},"min":{"1min":0.251,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.44800000000000001,"5min":0.628,"15min":0.63200000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.33900000000000002},"min":{"1min":0.26600000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51000000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.28799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34599999999999997},"min":{"1min":0.247,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47099999999999997,"5min":0.50800000000000001,"15min":0.59599999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.371,"15min":0.373},"min":{"1min":0.30299999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.60599999999999998},"last":0.35999999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.85399999999999998,"15min":1.214},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.42999999999999999,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.88400000000000001,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.41999999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.41099999999999998,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.72599999999999998,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.40200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78500000000000003,"5min":0.85399999999999998,"15min":1.603},"last":0.503}]},{"osd":6,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89000000000000001,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.39900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.878,"15min":0.878},"last":0.61699999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.39900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.79,"5min":3.79,"15min":3.79},"last":0.57399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1761280,"data_stored":836047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.30199999999999999},"min":{"1min":0.22800000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.377,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.31900000000000001,"15min":0.318},"min":{"1min":0.251,"5min":0.22700000000000001,"15min":0.19700000000000001},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.629},"last":0.32900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.315,"15min":0.32200000000000001},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.40799999999999997,"5min":0.50800000000000001,"15min":0.54400000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.34100000000000003,"15min":0.34699999999999998},"min":{"1min":0.29999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.749},"last":0.38700000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.379},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.73699999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.39500000000000002},"min":{"1min":0.33600000000000002,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":0.53500000000000003,"5min":0.55100000000000005,"15min":0.69499999999999995},"last":0.36799999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.41799999999999998,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.879},"last":0.55000000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.47199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.42899999999999999,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.52600000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.46000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.47899999999999998,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.83599999999999997},"last":0.53800000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.46899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.5},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.56200000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1765376,"data_stored":835625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.29799999999999999,"15min":0.30299999999999999},"min":{"1min":0.221,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.435,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.252},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.318},"min":{"1min":0.26500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.28999999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.33800000000000002,"15min":0.32200000000000001},"min":{"1min":0.24399999999999999,"5min":0.22700000000000001,"15min":0.19500000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.36399999999999999,"15min":0.35199999999999998},"min":{"1min":0.27500000000000002,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":0.32600000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.376,"15min":0.36599999999999999},"min":{"1min":0.27400000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.373,"15min":0.36899999999999999},"min":{"1min":0.28699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.375}]},{"osd":4,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.55200000000000005},"min":{"1min":0.41099999999999998,"5min":0.33900000000000002,"15min":0.28599999999999998},"max":{"1min":0.80700000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.47999999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55800000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.38,"15min":0.314},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.58799999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.44700000000000001,"5min":0.379,"15min":0.373},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.60999999999999999},"min":{"1min":0.47999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":1.0069999999999999,"15min":1.111},"last":0.77600000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58499999999999996,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":1.032,"15min":1.236},"last":0.755},{"interface":"front","average":{"1min":0.625,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1773568,"data_stored":837111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.33400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.501},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.40899999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55300000000000005}]},{"osd":1,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.41499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68999999999999995,"5min":0.746,"15min":0.76500000000000001},"last":0.308},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.44,"5min":0.308,"15min":0.308},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.35399999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.35099999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69199999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.38600000000000001,"5min":0.32400000000000001,"15min":0.32200000000000001},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.56899999999999995}]},{"osd":3,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56499999999999995,"15min":0.54200000000000004},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73699999999999999,"5min":0.80300000000000005,"15min":0.80600000000000005},"last":0.58299999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.51600000000000001,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60399999999999998,"15min":0.59499999999999997},"min":{"1min":0.46200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.61299999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.38100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73999999999999999,"5min":0.78900000000000003,"15min":0.90600000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.62,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.60099999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.47399999999999998,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.89300000000000002},"last":0.626}]}]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1773568,"data_stored":837099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73899999999999999,"5min":0.753,"15min":0.96899999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39300000000000002,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.68200000000000005,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.66300000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65800000000000003,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68400000000000005,"5min":0.72899999999999998,"15min":1.075},"last":0.68000000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.33200000000000002},"max":{"1min":0.72099999999999997,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.41099999999999998,"5min":0.373,"15min":0.307},"max":{"1min":0.753,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.747}]},{"osd":3,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":1.03},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.35199999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":1.026},"last":0.77700000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.371,"5min":0.31,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.50700000000000001,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79600000000000004,"5min":0.83899999999999997,"15min":0.91400000000000003},"last":0.70599999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.79400000000000004,"5min":0.79700000000000004,"15min":0.88200000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":0.81499999999999995,"15min":0.94199999999999995},"last":0.72599999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.41499999999999998,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.95899999999999996},"last":0.495},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.31},"max":{"1min":0.82299999999999995,"5min":0.85599999999999998,"15min":0.92000000000000004},"last":0.51900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:18.760 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:18.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.911+0000 7fe9640af700 1 Processor -- start 2023-08-27T11:28:18.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.911+0000 7fe9640af700 1 -- start start 2023-08-27T11:28:18.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.912+0000 7fe9640af700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe95c0b17e0 0x7fe95c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.912+0000 7fe9640af700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c161850 con 0x7fe95c161ce0 2023-08-27T11:28:18.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.912+0000 7fe9640af700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c0cfc10 con 0x7fe95c0b1400 2023-08-27T11:28:18.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.912+0000 7fe9640af700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c161200 con 0x7fe95c0b17e0 2023-08-27T11:28:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.912+0000 7fe96264c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe95c161ce0 0x7fe95c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58720/0 (socket says 172.21.15.112:58720) 2023-08-27T11:28:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.912+0000 7fe96264c700 1 -- 172.21.15.112:0/1786873139 learned_addr learned my addr 172.21.15.112:0/1786873139 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.913+0000 7fe96164a700 1 --2- 172.21.15.112:0/1786873139 >> v2:172.21.15.138:3300/0 conn(0x7fe95c0b17e0 0x7fe95c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.913+0000 7fe960e49700 1 -- 172.21.15.112:0/1786873139 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031545578 0 0) 0x7fe95c161850 con 0x7fe95c161ce0 2023-08-27T11:28:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.913+0000 7fe960e49700 1 -- 172.21.15.112:0/1786873139 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9480035b0 con 0x7fe95c161ce0 2023-08-27T11:28:18.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.913+0000 7fe960e49700 1 -- 172.21.15.112:0/1786873139 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645782512 0 0) 0x7fe95c0cfc10 con 0x7fe95c0b1400 2023-08-27T11:28:18.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.913+0000 7fe960e49700 1 -- 172.21.15.112:0/1786873139 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe95c161850 con 0x7fe95c0b1400 2023-08-27T11:28:18.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.913+0000 7fe960e49700 1 -- 172.21.15.112:0/1786873139 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2940960393 0 0) 0x7fe9480035b0 con 0x7fe95c161ce0 2023-08-27T11:28:18.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.913+0000 7fe96164a700 1 -- 172.21.15.112:0/1786873139 >> v1:172.21.15.138:6789/0 conn(0x7fe95c0b1400 legacy=0x7fe95c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.913+0000 7fe96164a700 1 -- 172.21.15.112:0/1786873139 >> v1:172.21.15.112:6789/0 conn(0x7fe95c161ce0 legacy=0x7fe95c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.913+0000 7fe96164a700 1 -- 172.21.15.112:0/1786873139 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c1614e0 con 0x7fe95c0b17e0 2023-08-27T11:28:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.913+0000 7fe96164a700 1 --2- 172.21.15.112:0/1786873139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c0b17e0 0x7fe95c0cf5a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe958009840 tx=0x7fe958009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=841574afa8ebdc42 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.913+0000 7fe960e49700 1 -- 172.21.15.112:0/1786873139 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9540019f0 con 0x7fe95c0b17e0 2023-08-27T11:28:18.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.913+0000 7fe960e49700 1 -- 172.21.15.112:0/1786873139 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe958004030 con 0x7fe95c0b17e0 2023-08-27T11:28:18.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.913+0000 7fe960e49700 1 -- 172.21.15.112:0/1786873139 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9580043c0 con 0x7fe95c0b17e0 2023-08-27T11:28:18.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.914+0000 7fe9640af700 1 -- 172.21.15.112:0/1786873139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c0b17e0 msgr2=0x7fe95c0cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.914+0000 7fe9640af700 1 --2- 172.21.15.112:0/1786873139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c0b17e0 0x7fe95c0cf5a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe958009840 tx=0x7fe958009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:18.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.914+0000 7fe9640af700 1 -- 172.21.15.112:0/1786873139 shutdown_connections 2023-08-27T11:28:18.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.914+0000 7fe9640af700 1 --2- 172.21.15.112:0/1786873139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c0b17e0 0x7fe95c0cf5a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.914+0000 7fe9640af700 1 -- 172.21.15.112:0/1786873139 >> 172.21.15.112:0/1786873139 conn(0x7fe95c0d0000 msgr2=0x7fe95c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.914+0000 7fe9640af700 1 -- 172.21.15.112:0/1786873139 shutdown_connections 2023-08-27T11:28:18.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.914+0000 7fe9640af700 1 -- 172.21.15.112:0/1786873139 wait complete. 2023-08-27T11:28:18.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.915+0000 7fe9640af700 1 Processor -- start 2023-08-27T11:28:18.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.915+0000 7fe9640af700 1 -- start start 2023-08-27T11:28:18.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe9640af700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c0b1400 0x7fe95c1f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe9640af700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95c0b17e0 0x7fe95c1f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe9640af700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe95c161ce0 0x7fe95c0c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe9640af700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c4cd0 con 0x7fe95c0b17e0 2023-08-27T11:28:18.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe9640af700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c7200 con 0x7fe95c0b1400 2023-08-27T11:28:18.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe9640af700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c4430 con 0x7fe95c161ce0 2023-08-27T11:28:18.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe96164a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95c0b17e0 0x7fe95c1f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe96164a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95c0b17e0 0x7fe95c1f5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36110/0 (socket says 172.21.15.112:36110) 2023-08-27T11:28:18.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe96164a700 1 -- 172.21.15.112:0/868731767 learned_addr learned my addr 172.21.15.112:0/868731767 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:18.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe961e4b700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c0b1400 0x7fe95c1f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe96264c700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe95c161ce0 0x7fe95c0c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe96164a700 1 -- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe95c161ce0 msgr2=0x7fe95c0c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe96164a700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe95c161ce0 0x7fe95c0c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe96164a700 1 -- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c0b1400 msgr2=0x7fe95c1f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe96164a700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c0b1400 0x7fe95c1f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe96164a700 1 -- 172.21.15.112:0/868731767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9580094a0 con 0x7fe95c0b17e0 2023-08-27T11:28:18.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe961e4b700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c0b1400 0x7fe95c1f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:18.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe96264c700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe95c161ce0 0x7fe95c0c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:18.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe96164a700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95c0b17e0 0x7fe95c1f5960 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fe9580090b0 tx=0x7fe958004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe9527fc700 1 -- 172.21.15.112:0/868731767 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958011830 con 0x7fe95c0b17e0 2023-08-27T11:28:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.916+0000 7fe9527fc700 1 -- 172.21.15.112:0/868731767 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95801f070 con 0x7fe95c0b17e0 2023-08-27T11:28:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.917+0000 7fe9640af700 1 -- 172.21.15.112:0/868731767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c0c4710 con 0x7fe95c0b17e0 2023-08-27T11:28:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.917+0000 7fe9527fc700 1 -- 172.21.15.112:0/868731767 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958011bf0 con 0x7fe95c0b17e0 2023-08-27T11:28:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.917+0000 7fe9640af700 1 -- 172.21.15.112:0/868731767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c1f5ea0 con 0x7fe95c0b17e0 2023-08-27T11:28:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.917+0000 7fe9640af700 1 -- 172.21.15.112:0/868731767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c0c73c0 con 0x7fe95c0b17e0 2023-08-27T11:28:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.918+0000 7fe9527fc700 1 -- 172.21.15.112:0/868731767 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe958011d50 con 0x7fe95c0b17e0 2023-08-27T11:28:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.918+0000 7fe9527fc700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe94006a4d0 0x7fe94006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.918+0000 7fe9527fc700 1 -- 172.21.15.112:0/868731767 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe958021070 con 0x7fe95c0b17e0 2023-08-27T11:28:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.918+0000 7fe961e4b700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe94006a4d0 0x7fe94006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.919+0000 7fe961e4b700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe94006a4d0 0x7fe94006c990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0011e0 tx=0x7fe94c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:18.922+0000 7fe9527fc700 1 -- 172.21.15.112:0/868731767 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe9580556c0 con 0x7fe95c0b17e0 2023-08-27T11:28:19.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.105+0000 7fe9640af700 1 -- 172.21.15.112:0/868731767 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe95c153bf0 con 0x7fe94006a4d0 2023-08-27T11:28:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.115+0000 7fe9527fc700 1 -- 172.21.15.112:0/868731767 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312235 (secure 0 0 0) 0x7fe94c001cf0 con 0x7fe94006a4d0 2023-08-27T11:28:19.120 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:19.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.118+0000 7fe9640af700 1 -- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe94006a4d0 msgr2=0x7fe94006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.118+0000 7fe9640af700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe94006a4d0 0x7fe94006c990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0011e0 tx=0x7fe94c009040 comp rx=0 tx=0).stop 2023-08-27T11:28:19.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.118+0000 7fe9640af700 1 -- 172.21.15.112:0/868731767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95c0b17e0 msgr2=0x7fe95c1f5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.118+0000 7fe9640af700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95c0b17e0 0x7fe95c1f5960 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fe9580090b0 tx=0x7fe958004750 comp rx=0 tx=0).stop 2023-08-27T11:28:19.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.118+0000 7fe9640af700 1 -- 172.21.15.112:0/868731767 shutdown_connections 2023-08-27T11:28:19.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.118+0000 7fe9640af700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe94006a4d0 0x7fe94006c990 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.118+0000 7fe9640af700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe95c161ce0 0x7fe95c0c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.118+0000 7fe9640af700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c0b1400 0x7fe95c1f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.118+0000 7fe9640af700 1 --2- 172.21.15.112:0/868731767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95c0b17e0 0x7fe95c1f5960 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.118+0000 7fe9640af700 1 -- 172.21.15.112:0/868731767 >> 172.21.15.112:0/868731767 conn(0x7fe95c0d0000 msgr2=0x7fe95c1532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.118+0000 7fe9640af700 1 -- 172.21.15.112:0/868731767 shutdown_connections 2023-08-27T11:28:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.118+0000 7fe9640af700 1 -- 172.21.15.112:0/868731767 wait complete. 2023-08-27T11:28:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:28:19.131 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2023-08-27T11:28:18.399206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264380,"kb_used_data":14924,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754710340,"statfs":{"total":773094113280,"available":772823388160,"internally_reserved":0,"allocated":15282176,"data_stored":7854609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.774615"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171725+0000","last_change":"2023-08-27T11:28:14.653632+0000","last_active":"2023-08-27T11:28:16.171725+0000","last_peered":"2023-08-27T11:28:16.171725+0000","last_clean":"2023-08-27T11:28:16.171725+0000","last_became_active":"2023-08-27T11:28:14.653202+0000","last_became_peered":"2023-08-27T11:28:14.653202+0000","last_unstale":"2023-08-27T11:28:16.171725+0000","last_undegraded":"2023-08-27T11:28:16.171725+0000","last_fullsized":"2023-08-27T11:28:16.171725+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:47:54.569076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171956+0000","last_change":"2023-08-27T11:28:14.650120+0000","last_active":"2023-08-27T11:28:16.171956+0000","last_peered":"2023-08-27T11:28:16.171956+0000","last_clean":"2023-08-27T11:28:16.171956+0000","last_became_active":"2023-08-27T11:28:14.649837+0000","last_became_peered":"2023-08-27T11:28:14.649837+0000","last_unstale":"2023-08-27T11:28:16.171956+0000","last_undegraded":"2023-08-27T11:28:16.171956+0000","last_fullsized":"2023-08-27T11:28:16.171956+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:34:09.267774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640135+0000","last_change":"2023-08-27T11:28:14.661362+0000","last_active":"2023-08-27T11:28:15.640135+0000","last_peered":"2023-08-27T11:28:15.640135+0000","last_clean":"2023-08-27T11:28:15.640135+0000","last_became_active":"2023-08-27T11:28:14.661023+0000","last_became_peered":"2023-08-27T11:28:14.661023+0000","last_unstale":"2023-08-27T11:28:15.640135+0000","last_undegraded":"2023-08-27T11:28:15.640135+0000","last_fullsized":"2023-08-27T11:28:15.640135+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:03.434668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673076+0000","last_change":"2023-08-27T11:28:14.650040+0000","last_active":"2023-08-27T11:28:16.673076+0000","last_peered":"2023-08-27T11:28:16.673076+0000","last_clean":"2023-08-27T11:28:16.673076+0000","last_became_active":"2023-08-27T11:28:14.649135+0000","last_became_peered":"2023-08-27T11:28:14.649135+0000","last_unstale":"2023-08-27T11:28:16.673076+0000","last_undegraded":"2023-08-27T11:28:16.673076+0000","last_fullsized":"2023-08-27T11:28:16.673076+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:45:51.828640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635138+0000","last_change":"2023-08-27T11:28:14.656675+0000","last_active":"2023-08-27T11:28:15.635138+0000","last_peered":"2023-08-27T11:28:15.635138+0000","last_clean":"2023-08-27T11:28:15.635138+0000","last_became_active":"2023-08-27T11:28:14.655942+0000","last_became_peered":"2023-08-27T11:28:14.655942+0000","last_unstale":"2023-08-27T11:28:15.635138+0000","last_undegraded":"2023-08-27T11:28:15.635138+0000","last_fullsized":"2023-08-27T11:28:15.635138+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:50.800979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635060+0000","last_change":"2023-08-27T11:28:14.654403+0000","last_active":"2023-08-27T11:28:15.635060+0000","last_peered":"2023-08-27T11:28:15.635060+0000","last_clean":"2023-08-27T11:28:15.635060+0000","last_became_active":"2023-08-27T11:28:14.654094+0000","last_became_peered":"2023-08-27T11:28:14.654094+0000","last_unstale":"2023-08-27T11:28:15.635060+0000","last_undegraded":"2023-08-27T11:28:15.635060+0000","last_fullsized":"2023-08-27T11:28:15.635060+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:03.434668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673852+0000","last_change":"2023-08-27T11:28:14.653279+0000","last_active":"2023-08-27T11:28:16.673852+0000","last_peered":"2023-08-27T11:28:16.673852+0000","last_clean":"2023-08-27T11:28:16.673852+0000","last_became_active":"2023-08-27T11:28:14.652988+0000","last_became_peered":"2023-08-27T11:28:14.652988+0000","last_unstale":"2023-08-27T11:28:16.673852+0000","last_undegraded":"2023-08-27T11:28:16.673852+0000","last_fullsized":"2023-08-27T11:28:16.673852+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:24:00.946418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672986+0000","last_change":"2023-08-27T11:28:14.667142+0000","last_active":"2023-08-27T11:28:16.672986+0000","last_peered":"2023-08-27T11:28:16.672986+0000","last_clean":"2023-08-27T11:28:16.672986+0000","last_became_active":"2023-08-27T11:28:14.665039+0000","last_became_peered":"2023-08-27T11:28:14.665039+0000","last_unstale":"2023-08-27T11:28:16.672986+0000","last_undegraded":"2023-08-27T11:28:16.672986+0000","last_fullsized":"2023-08-27T11:28:16.672986+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:07:10.121767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171779+0000","last_change":"2023-08-27T11:28:14.653472+0000","last_active":"2023-08-27T11:28:16.171779+0000","last_peered":"2023-08-27T11:28:16.171779+0000","last_clean":"2023-08-27T11:28:16.171779+0000","last_became_active":"2023-08-27T11:28:14.652816+0000","last_became_peered":"2023-08-27T11:28:14.652816+0000","last_unstale":"2023-08-27T11:28:16.171779+0000","last_undegraded":"2023-08-27T11:28:16.171779+0000","last_fullsized":"2023-08-27T11:28:16.171779+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:40.732259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635173+0000","last_change":"2023-08-27T11:28:14.657163+0000","last_active":"2023-08-27T11:28:15.635173+0000","last_peered":"2023-08-27T11:28:15.635173+0000","last_clean":"2023-08-27T11:28:15.635173+0000","last_became_active":"2023-08-27T11:28:14.656659+0000","last_became_peered":"2023-08-27T11:28:14.656659+0000","last_unstale":"2023-08-27T11:28:15.635173+0000","last_undegraded":"2023-08-27T11:28:15.635173+0000","last_fullsized":"2023-08-27T11:28:15.635173+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:55:02.903945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916393+0000","last_change":"2023-08-27T11:28:14.662167+0000","last_active":"2023-08-27T11:28:15.916393+0000","last_peered":"2023-08-27T11:28:15.916393+0000","last_clean":"2023-08-27T11:28:15.916393+0000","last_became_active":"2023-08-27T11:28:14.661899+0000","last_became_peered":"2023-08-27T11:28:14.661899+0000","last_unstale":"2023-08-27T11:28:15.916393+0000","last_undegraded":"2023-08-27T11:28:15.916393+0000","last_fullsized":"2023-08-27T11:28:15.916393+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:25:27.533398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673228+0000","last_change":"2023-08-27T11:28:14.658540+0000","last_active":"2023-08-27T11:28:16.673228+0000","last_peered":"2023-08-27T11:28:16.673228+0000","last_clean":"2023-08-27T11:28:16.673228+0000","last_became_active":"2023-08-27T11:28:14.658229+0000","last_became_peered":"2023-08-27T11:28:14.658229+0000","last_unstale":"2023-08-27T11:28:16.673228+0000","last_undegraded":"2023-08-27T11:28:16.673228+0000","last_fullsized":"2023-08-27T11:28:16.673228+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:28:39.113843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641013+0000","last_change":"2023-08-27T11:28:14.666082+0000","last_active":"2023-08-27T11:28:15.641013+0000","last_peered":"2023-08-27T11:28:15.641013+0000","last_clean":"2023-08-27T11:28:15.641013+0000","last_became_active":"2023-08-27T11:28:14.665656+0000","last_became_peered":"2023-08-27T11:28:14.665656+0000","last_unstale":"2023-08-27T11:28:15.641013+0000","last_undegraded":"2023-08-27T11:28:15.641013+0000","last_fullsized":"2023-08-27T11:28:15.641013+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:03.556396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672935+0000","last_change":"2023-08-27T11:28:14.653903+0000","last_active":"2023-08-27T11:28:16.672935+0000","last_peered":"2023-08-27T11:28:16.672935+0000","last_clean":"2023-08-27T11:28:16.672935+0000","last_became_active":"2023-08-27T11:28:14.653657+0000","last_became_peered":"2023-08-27T11:28:14.653657+0000","last_unstale":"2023-08-27T11:28:16.672935+0000","last_undegraded":"2023-08-27T11:28:16.672935+0000","last_fullsized":"2023-08-27T11:28:16.672935+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:41:51.876480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171599+0000","last_change":"2023-08-27T11:28:14.643766+0000","last_active":"2023-08-27T11:28:16.171599+0000","last_peered":"2023-08-27T11:28:16.171599+0000","last_clean":"2023-08-27T11:28:16.171599+0000","last_became_active":"2023-08-27T11:28:14.643212+0000","last_became_peered":"2023-08-27T11:28:14.643212+0000","last_unstale":"2023-08-27T11:28:16.171599+0000","last_undegraded":"2023-08-27T11:28:16.171599+0000","last_fullsized":"2023-08-27T11:28:16.171599+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:20:04.295796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.634913+0000","last_change":"2023-08-27T11:28:14.657418+0000","last_active":"2023-08-27T11:28:15.634913+0000","last_peered":"2023-08-27T11:28:15.634913+0000","last_clean":"2023-08-27T11:28:15.634913+0000","last_became_active":"2023-08-27T11:28:14.656144+0000","last_became_peered":"2023-08-27T11:28:14.656144+0000","last_unstale":"2023-08-27T11:28:15.634913+0000","last_undegraded":"2023-08-27T11:28:15.634913+0000","last_fullsized":"2023-08-27T11:28:15.634913+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:27:29.886325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635482+0000","last_change":"2023-08-27T11:28:14.658801+0000","last_active":"2023-08-27T11:28:15.635482+0000","last_peered":"2023-08-27T11:28:15.635482+0000","last_clean":"2023-08-27T11:28:15.635482+0000","last_became_active":"2023-08-27T11:28:14.658561+0000","last_became_peered":"2023-08-27T11:28:14.658561+0000","last_unstale":"2023-08-27T11:28:15.635482+0000","last_undegraded":"2023-08-27T11:28:15.635482+0000","last_fullsized":"2023-08-27T11:28:15.635482+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:57:19.740295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641378+0000","last_change":"2023-08-27T11:28:14.655039+0000","last_active":"2023-08-27T11:28:15.641378+0000","last_peered":"2023-08-27T11:28:15.641378+0000","last_clean":"2023-08-27T11:28:15.641378+0000","last_became_active":"2023-08-27T11:28:14.654781+0000","last_became_peered":"2023-08-27T11:28:14.654781+0000","last_unstale":"2023-08-27T11:28:15.641378+0000","last_undegraded":"2023-08-27T11:28:15.641378+0000","last_fullsized":"2023-08-27T11:28:15.641378+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:28:40.318425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641466+0000","last_change":"2023-08-27T11:28:14.650908+0000","last_active":"2023-08-27T11:28:15.641466+0000","last_peered":"2023-08-27T11:28:15.641466+0000","last_clean":"2023-08-27T11:28:15.641466+0000","last_became_active":"2023-08-27T11:28:14.647101+0000","last_became_peered":"2023-08-27T11:28:14.647101+0000","last_unstale":"2023-08-27T11:28:15.641466+0000","last_undegraded":"2023-08-27T11:28:15.641466+0000","last_fullsized":"2023-08-27T11:28:15.641466+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:55:00.113382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635082+0000","last_change":"2023-08-27T11:28:14.653232+0000","last_active":"2023-08-27T11:28:15.635082+0000","last_peered":"2023-08-27T11:28:15.635082+0000","last_clean":"2023-08-27T11:28:15.635082+0000","last_became_active":"2023-08-27T11:28:14.652576+0000","last_became_peered":"2023-08-27T11:28:14.652576+0000","last_unstale":"2023-08-27T11:28:15.635082+0000","last_undegraded":"2023-08-27T11:28:15.635082+0000","last_fullsized":"2023-08-27T11:28:15.635082+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:55:02.771821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171934+0000","last_change":"2023-08-27T11:28:14.647657+0000","last_active":"2023-08-27T11:28:16.171934+0000","last_peered":"2023-08-27T11:28:16.171934+0000","last_clean":"2023-08-27T11:28:16.171934+0000","last_became_active":"2023-08-27T11:28:14.647381+0000","last_became_peered":"2023-08-27T11:28:14.647381+0000","last_unstale":"2023-08-27T11:28:16.171934+0000","last_undegraded":"2023-08-27T11:28:16.171934+0000","last_fullsized":"2023-08-27T11:28:16.171934+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:27:29.886325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171982+0000","last_change":"2023-08-27T11:28:14.648423+0000","last_active":"2023-08-27T11:28:16.171982+0000","last_peered":"2023-08-27T11:28:16.171982+0000","last_clean":"2023-08-27T11:28:16.171982+0000","last_became_active":"2023-08-27T11:28:14.647791+0000","last_became_peered":"2023-08-27T11:28:14.647791+0000","last_unstale":"2023-08-27T11:28:16.171982+0000","last_undegraded":"2023-08-27T11:28:16.171982+0000","last_fullsized":"2023-08-27T11:28:16.171982+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:55:02.771821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672963+0000","last_change":"2023-08-27T11:28:14.664163+0000","last_active":"2023-08-27T11:28:16.672963+0000","last_peered":"2023-08-27T11:28:16.672963+0000","last_clean":"2023-08-27T11:28:16.672963+0000","last_became_active":"2023-08-27T11:28:14.663861+0000","last_became_peered":"2023-08-27T11:28:14.663861+0000","last_unstale":"2023-08-27T11:28:16.672963+0000","last_undegraded":"2023-08-27T11:28:16.672963+0000","last_fullsized":"2023-08-27T11:28:16.672963+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:46:04.177457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171476+0000","last_change":"2023-08-27T11:28:14.658603+0000","last_active":"2023-08-27T11:28:16.171476+0000","last_peered":"2023-08-27T11:28:16.171476+0000","last_clean":"2023-08-27T11:28:16.171476+0000","last_became_active":"2023-08-27T11:28:14.658245+0000","last_became_peered":"2023-08-27T11:28:14.658245+0000","last_unstale":"2023-08-27T11:28:16.171476+0000","last_undegraded":"2023-08-27T11:28:16.171476+0000","last_fullsized":"2023-08-27T11:28:16.171476+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:26.993320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673142+0000","last_change":"2023-08-27T11:28:14.659617+0000","last_active":"2023-08-27T11:28:16.673142+0000","last_peered":"2023-08-27T11:28:16.673142+0000","last_clean":"2023-08-27T11:28:16.673142+0000","last_became_active":"2023-08-27T11:28:14.659303+0000","last_became_peered":"2023-08-27T11:28:14.659303+0000","last_unstale":"2023-08-27T11:28:16.673142+0000","last_undegraded":"2023-08-27T11:28:16.673142+0000","last_fullsized":"2023-08-27T11:28:16.673142+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:53:16.248207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916596+0000","last_change":"2023-08-27T11:28:14.656752+0000","last_active":"2023-08-27T11:28:15.916596+0000","last_peered":"2023-08-27T11:28:15.916596+0000","last_clean":"2023-08-27T11:28:15.916596+0000","last_became_active":"2023-08-27T11:28:14.656474+0000","last_became_peered":"2023-08-27T11:28:14.656474+0000","last_unstale":"2023-08-27T11:28:15.916596+0000","last_undegraded":"2023-08-27T11:28:15.916596+0000","last_fullsized":"2023-08-27T11:28:15.916596+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:09:54.600326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673353+0000","last_change":"2023-08-27T11:28:14.664807+0000","last_active":"2023-08-27T11:28:16.673353+0000","last_peered":"2023-08-27T11:28:16.673353+0000","last_clean":"2023-08-27T11:28:16.673353+0000","last_became_active":"2023-08-27T11:28:14.664469+0000","last_became_peered":"2023-08-27T11:28:14.664469+0000","last_unstale":"2023-08-27T11:28:16.673353+0000","last_undegraded":"2023-08-27T11:28:16.673353+0000","last_fullsized":"2023-08-27T11:28:16.673353+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:30:03.114552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635222+0000","last_change":"2023-08-27T11:28:14.654267+0000","last_active":"2023-08-27T11:28:15.635222+0000","last_peered":"2023-08-27T11:28:15.635222+0000","last_clean":"2023-08-27T11:28:15.635222+0000","last_became_active":"2023-08-27T11:28:14.653971+0000","last_became_peered":"2023-08-27T11:28:14.653971+0000","last_unstale":"2023-08-27T11:28:15.635222+0000","last_undegraded":"2023-08-27T11:28:15.635222+0000","last_fullsized":"2023-08-27T11:28:15.635222+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:07:29.840522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916342+0000","last_change":"2023-08-27T11:28:14.661421+0000","last_active":"2023-08-27T11:28:15.916342+0000","last_peered":"2023-08-27T11:28:15.916342+0000","last_clean":"2023-08-27T11:28:15.916342+0000","last_became_active":"2023-08-27T11:28:14.661064+0000","last_became_peered":"2023-08-27T11:28:14.661064+0000","last_unstale":"2023-08-27T11:28:15.916342+0000","last_undegraded":"2023-08-27T11:28:15.916342+0000","last_fullsized":"2023-08-27T11:28:15.916342+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:02:50.171479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916205+0000","last_change":"2023-08-27T11:28:14.653898+0000","last_active":"2023-08-27T11:28:15.916205+0000","last_peered":"2023-08-27T11:28:15.916205+0000","last_clean":"2023-08-27T11:28:15.916205+0000","last_became_active":"2023-08-27T11:28:14.653430+0000","last_became_peered":"2023-08-27T11:28:14.653430+0000","last_unstale":"2023-08-27T11:28:15.916205+0000","last_undegraded":"2023-08-27T11:28:15.916205+0000","last_fullsized":"2023-08-27T11:28:15.916205+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:57:08.260806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.674628+0000","last_change":"2023-08-27T11:28:14.647766+0000","last_active":"2023-08-27T11:28:16.674628+0000","last_peered":"2023-08-27T11:28:16.674628+0000","last_clean":"2023-08-27T11:28:16.674628+0000","last_became_active":"2023-08-27T11:28:14.647276+0000","last_became_peered":"2023-08-27T11:28:14.647276+0000","last_unstale":"2023-08-27T11:28:16.674628+0000","last_undegraded":"2023-08-27T11:28:16.674628+0000","last_fullsized":"2023-08-27T11:28:16.674628+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:52.480549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.634949+0000","last_change":"2023-08-27T11:28:14.657238+0000","last_active":"2023-08-27T11:28:15.634949+0000","last_peered":"2023-08-27T11:28:15.634949+0000","last_clean":"2023-08-27T11:28:15.634949+0000","last_became_active":"2023-08-27T11:28:14.656196+0000","last_became_peered":"2023-08-27T11:28:14.656196+0000","last_unstale":"2023-08-27T11:28:15.634949+0000","last_undegraded":"2023-08-27T11:28:15.634949+0000","last_fullsized":"2023-08-27T11:28:15.634949+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:34:09.267774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673343+0000","last_change":"2023-08-27T11:28:14.664187+0000","last_active":"2023-08-27T11:28:16.673343+0000","last_peered":"2023-08-27T11:28:16.673343+0000","last_clean":"2023-08-27T11:28:16.673343+0000","last_became_active":"2023-08-27T11:28:14.663654+0000","last_became_peered":"2023-08-27T11:28:14.663654+0000","last_unstale":"2023-08-27T11:28:16.673343+0000","last_undegraded":"2023-08-27T11:28:16.673343+0000","last_fullsized":"2023-08-27T11:28:16.673343+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:30.035009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635518+0000","last_change":"2023-08-27T11:28:13.650131+0000","last_active":"2023-08-27T11:28:15.635518+0000","last_peered":"2023-08-27T11:28:15.635518+0000","last_clean":"2023-08-27T11:28:15.635518+0000","last_became_active":"2023-08-27T11:28:13.649845+0000","last_became_peered":"2023-08-27T11:28:13.649845+0000","last_unstale":"2023-08-27T11:28:15.635518+0000","last_undegraded":"2023-08-27T11:28:15.635518+0000","last_fullsized":"2023-08-27T11:28:15.635518+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:35:46.387936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635352+0000","last_change":"2023-08-27T11:28:14.654305+0000","last_active":"2023-08-27T11:28:15.635352+0000","last_peered":"2023-08-27T11:28:15.635352+0000","last_clean":"2023-08-27T11:28:15.635352+0000","last_became_active":"2023-08-27T11:28:14.654047+0000","last_became_peered":"2023-08-27T11:28:14.654047+0000","last_unstale":"2023-08-27T11:28:15.635352+0000","last_undegraded":"2023-08-27T11:28:15.635352+0000","last_fullsized":"2023-08-27T11:28:15.635352+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:48:53.806730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171845+0000","last_change":"2023-08-27T11:28:13.650587+0000","last_active":"2023-08-27T11:28:16.171845+0000","last_peered":"2023-08-27T11:28:16.171845+0000","last_clean":"2023-08-27T11:28:16.171845+0000","last_became_active":"2023-08-27T11:28:13.650325+0000","last_became_peered":"2023-08-27T11:28:13.650325+0000","last_unstale":"2023-08-27T11:28:16.171845+0000","last_undegraded":"2023-08-27T11:28:16.171845+0000","last_fullsized":"2023-08-27T11:28:16.171845+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:08:28.011180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641379+0000","last_change":"2023-08-27T11:28:14.652957+0000","last_active":"2023-08-27T11:28:15.641379+0000","last_peered":"2023-08-27T11:28:15.641379+0000","last_clean":"2023-08-27T11:28:15.641379+0000","last_became_active":"2023-08-27T11:28:14.652648+0000","last_became_peered":"2023-08-27T11:28:14.652648+0000","last_unstale":"2023-08-27T11:28:15.641379+0000","last_undegraded":"2023-08-27T11:28:15.641379+0000","last_fullsized":"2023-08-27T11:28:15.641379+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:04:56.941782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"138'3","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672711+0000","last_change":"2023-08-27T11:28:13.651281+0000","last_active":"2023-08-27T11:28:16.672711+0000","last_peered":"2023-08-27T11:28:16.672711+0000","last_clean":"2023-08-27T11:28:16.672711+0000","last_became_active":"2023-08-27T11:28:13.651022+0000","last_became_peered":"2023-08-27T11:28:13.651022+0000","last_unstale":"2023-08-27T11:28:16.672711+0000","last_undegraded":"2023-08-27T11:28:16.672711+0000","last_fullsized":"2023-08-27T11:28:16.672711+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:54:01.513256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171881+0000","last_change":"2023-08-27T11:28:14.650678+0000","last_active":"2023-08-27T11:28:16.171881+0000","last_peered":"2023-08-27T11:28:16.171881+0000","last_clean":"2023-08-27T11:28:16.171881+0000","last_became_active":"2023-08-27T11:28:14.649718+0000","last_became_peered":"2023-08-27T11:28:14.649718+0000","last_unstale":"2023-08-27T11:28:16.171881+0000","last_undegraded":"2023-08-27T11:28:16.171881+0000","last_fullsized":"2023-08-27T11:28:16.171881+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:42:24.963359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"138'6","reported_seq":19,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641636+0000","last_change":"2023-08-27T11:28:13.646100+0000","last_active":"2023-08-27T11:28:15.641636+0000","last_peered":"2023-08-27T11:28:15.641636+0000","last_clean":"2023-08-27T11:28:15.641636+0000","last_became_active":"2023-08-27T11:28:13.645775+0000","last_became_peered":"2023-08-27T11:28:13.645775+0000","last_unstale":"2023-08-27T11:28:15.641636+0000","last_undegraded":"2023-08-27T11:28:15.641636+0000","last_fullsized":"2023-08-27T11:28:15.641636+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:56.475941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672505+0000","last_change":"2023-08-27T11:28:14.658845+0000","last_active":"2023-08-27T11:28:16.672505+0000","last_peered":"2023-08-27T11:28:16.672505+0000","last_clean":"2023-08-27T11:28:16.672505+0000","last_became_active":"2023-08-27T11:28:14.658570+0000","last_became_peered":"2023-08-27T11:28:14.658570+0000","last_unstale":"2023-08-27T11:28:16.672505+0000","last_undegraded":"2023-08-27T11:28:16.672505+0000","last_fullsized":"2023-08-27T11:28:16.672505+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:23:34.459876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"138'2","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641345+0000","last_change":"2023-08-27T11:28:13.646623+0000","last_active":"2023-08-27T11:28:15.641345+0000","last_peered":"2023-08-27T11:28:15.641345+0000","last_clean":"2023-08-27T11:28:15.641345+0000","last_became_active":"2023-08-27T11:28:13.646053+0000","last_became_peered":"2023-08-27T11:28:13.646053+0000","last_unstale":"2023-08-27T11:28:15.641345+0000","last_undegraded":"2023-08-27T11:28:15.641345+0000","last_fullsized":"2023-08-27T11:28:15.641345+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:02:08.085302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.634877+0000","last_change":"2023-08-27T11:28:14.653655+0000","last_active":"2023-08-27T11:28:15.634877+0000","last_peered":"2023-08-27T11:28:15.634877+0000","last_clean":"2023-08-27T11:28:15.634877+0000","last_became_active":"2023-08-27T11:28:14.653374+0000","last_became_peered":"2023-08-27T11:28:14.653374+0000","last_unstale":"2023-08-27T11:28:15.634877+0000","last_undegraded":"2023-08-27T11:28:15.634877+0000","last_fullsized":"2023-08-27T11:28:15.634877+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:28:44.942493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.172145+0000","last_change":"2023-08-27T11:28:13.648211+0000","last_active":"2023-08-27T11:28:16.172145+0000","last_peered":"2023-08-27T11:28:16.172145+0000","last_clean":"2023-08-27T11:28:16.172145+0000","last_became_active":"2023-08-27T11:28:13.647926+0000","last_became_peered":"2023-08-27T11:28:13.647926+0000","last_unstale":"2023-08-27T11:28:16.172145+0000","last_undegraded":"2023-08-27T11:28:16.172145+0000","last_fullsized":"2023-08-27T11:28:16.172145+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:30:50.057457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641278+0000","last_change":"2023-08-27T11:28:14.664867+0000","last_active":"2023-08-27T11:28:15.641278+0000","last_peered":"2023-08-27T11:28:15.641278+0000","last_clean":"2023-08-27T11:28:15.641278+0000","last_became_active":"2023-08-27T11:28:14.664589+0000","last_became_peered":"2023-08-27T11:28:14.664589+0000","last_unstale":"2023-08-27T11:28:15.641278+0000","last_undegraded":"2023-08-27T11:28:15.641278+0000","last_fullsized":"2023-08-27T11:28:15.641278+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:40:33.267545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672922+0000","last_change":"2023-08-27T11:28:13.644841+0000","last_active":"2023-08-27T11:28:16.672922+0000","last_peered":"2023-08-27T11:28:16.672922+0000","last_clean":"2023-08-27T11:28:16.672922+0000","last_became_active":"2023-08-27T11:28:13.644589+0000","last_became_peered":"2023-08-27T11:28:13.644589+0000","last_unstale":"2023-08-27T11:28:16.672922+0000","last_undegraded":"2023-08-27T11:28:16.672922+0000","last_fullsized":"2023-08-27T11:28:16.672922+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:16:40.302148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641417+0000","last_change":"2023-08-27T11:28:14.660190+0000","last_active":"2023-08-27T11:28:15.641417+0000","last_peered":"2023-08-27T11:28:15.641417+0000","last_clean":"2023-08-27T11:28:15.641417+0000","last_became_active":"2023-08-27T11:28:14.659884+0000","last_became_peered":"2023-08-27T11:28:14.659884+0000","last_unstale":"2023-08-27T11:28:15.641417+0000","last_undegraded":"2023-08-27T11:28:15.641417+0000","last_fullsized":"2023-08-27T11:28:15.641417+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:13:53.734648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"138'8","reported_seq":21,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916488+0000","last_change":"2023-08-27T11:28:13.647523+0000","last_active":"2023-08-27T11:28:15.916488+0000","last_peered":"2023-08-27T11:28:15.916488+0000","last_clean":"2023-08-27T11:28:15.916488+0000","last_became_active":"2023-08-27T11:28:13.647233+0000","last_became_peered":"2023-08-27T11:28:13.647233+0000","last_unstale":"2023-08-27T11:28:15.916488+0000","last_undegraded":"2023-08-27T11:28:15.916488+0000","last_fullsized":"2023-08-27T11:28:15.916488+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:02:09.845015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673037+0000","last_change":"2023-08-27T11:28:14.641836+0000","last_active":"2023-08-27T11:28:16.673037+0000","last_peered":"2023-08-27T11:28:16.673037+0000","last_clean":"2023-08-27T11:28:16.673037+0000","last_became_active":"2023-08-27T11:28:14.641570+0000","last_became_peered":"2023-08-27T11:28:14.641570+0000","last_unstale":"2023-08-27T11:28:16.673037+0000","last_undegraded":"2023-08-27T11:28:16.673037+0000","last_fullsized":"2023-08-27T11:28:16.673037+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:27:26.727128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"138'1","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916633+0000","last_change":"2023-08-27T11:28:13.652658+0000","last_active":"2023-08-27T11:28:15.916633+0000","last_peered":"2023-08-27T11:28:15.916633+0000","last_clean":"2023-08-27T11:28:15.916633+0000","last_became_active":"2023-08-27T11:28:13.652329+0000","last_became_peered":"2023-08-27T11:28:13.652329+0000","last_unstale":"2023-08-27T11:28:15.916633+0000","last_undegraded":"2023-08-27T11:28:15.916633+0000","last_fullsized":"2023-08-27T11:28:15.916633+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:47:49.956438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635361+0000","last_change":"2023-08-27T11:28:14.656548+0000","last_active":"2023-08-27T11:28:15.635361+0000","last_peered":"2023-08-27T11:28:15.635361+0000","last_clean":"2023-08-27T11:28:15.635361+0000","last_became_active":"2023-08-27T11:28:14.656044+0000","last_became_peered":"2023-08-27T11:28:14.656044+0000","last_unstale":"2023-08-27T11:28:15.635361+0000","last_undegraded":"2023-08-27T11:28:15.635361+0000","last_fullsized":"2023-08-27T11:28:15.635361+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:54.650454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"138'2","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672792+0000","last_change":"2023-08-27T11:28:13.644465+0000","last_active":"2023-08-27T11:28:16.672792+0000","last_peered":"2023-08-27T11:28:16.672792+0000","last_clean":"2023-08-27T11:28:16.672792+0000","last_became_active":"2023-08-27T11:28:13.644135+0000","last_became_peered":"2023-08-27T11:28:13.644135+0000","last_unstale":"2023-08-27T11:28:16.672792+0000","last_undegraded":"2023-08-27T11:28:16.672792+0000","last_fullsized":"2023-08-27T11:28:16.672792+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:57:22.012755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916693+0000","last_change":"2023-08-27T11:28:14.656696+0000","last_active":"2023-08-27T11:28:15.916693+0000","last_peered":"2023-08-27T11:28:15.916693+0000","last_clean":"2023-08-27T11:28:15.916693+0000","last_became_active":"2023-08-27T11:28:14.656419+0000","last_became_peered":"2023-08-27T11:28:14.656419+0000","last_unstale":"2023-08-27T11:28:15.916693+0000","last_undegraded":"2023-08-27T11:28:15.916693+0000","last_fullsized":"2023-08-27T11:28:15.916693+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:36:41.090504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"138'1","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640294+0000","last_change":"2023-08-27T11:28:13.648731+0000","last_active":"2023-08-27T11:28:15.640294+0000","last_peered":"2023-08-27T11:28:15.640294+0000","last_clean":"2023-08-27T11:28:15.640294+0000","last_became_active":"2023-08-27T11:28:13.648503+0000","last_became_peered":"2023-08-27T11:28:13.648503+0000","last_unstale":"2023-08-27T11:28:15.640294+0000","last_undegraded":"2023-08-27T11:28:15.640294+0000","last_fullsized":"2023-08-27T11:28:15.640294+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:08.667418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171931+0000","last_change":"2023-08-27T11:28:14.659900+0000","last_active":"2023-08-27T11:28:16.171931+0000","last_peered":"2023-08-27T11:28:16.171931+0000","last_clean":"2023-08-27T11:28:16.171931+0000","last_became_active":"2023-08-27T11:28:14.659598+0000","last_became_peered":"2023-08-27T11:28:14.659598+0000","last_unstale":"2023-08-27T11:28:16.171931+0000","last_undegraded":"2023-08-27T11:28:16.171931+0000","last_fullsized":"2023-08-27T11:28:16.171931+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:28:44.942493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916440+0000","last_change":"2023-08-27T11:28:13.647955+0000","last_active":"2023-08-27T11:28:15.916440+0000","last_peered":"2023-08-27T11:28:15.916440+0000","last_clean":"2023-08-27T11:28:15.916440+0000","last_became_active":"2023-08-27T11:28:13.647696+0000","last_became_peered":"2023-08-27T11:28:13.647696+0000","last_unstale":"2023-08-27T11:28:15.916440+0000","last_undegraded":"2023-08-27T11:28:15.916440+0000","last_fullsized":"2023-08-27T11:28:15.916440+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:09:26.898187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171472+0000","last_change":"2023-08-27T11:28:14.660921+0000","last_active":"2023-08-27T11:28:16.171472+0000","last_peered":"2023-08-27T11:28:16.171472+0000","last_clean":"2023-08-27T11:28:16.171472+0000","last_became_active":"2023-08-27T11:28:14.660473+0000","last_became_peered":"2023-08-27T11:28:14.660473+0000","last_unstale":"2023-08-27T11:28:16.171472+0000","last_undegraded":"2023-08-27T11:28:16.171472+0000","last_fullsized":"2023-08-27T11:28:16.171472+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:20:20.806596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"138'2","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641535+0000","last_change":"2023-08-27T11:28:13.650453+0000","last_active":"2023-08-27T11:28:15.641535+0000","last_peered":"2023-08-27T11:28:15.641535+0000","last_clean":"2023-08-27T11:28:15.641535+0000","last_became_active":"2023-08-27T11:28:13.650244+0000","last_became_peered":"2023-08-27T11:28:13.650244+0000","last_unstale":"2023-08-27T11:28:15.641535+0000","last_undegraded":"2023-08-27T11:28:15.641535+0000","last_fullsized":"2023-08-27T11:28:15.641535+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:27:00.535127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672779+0000","last_change":"2023-08-27T11:28:14.647979+0000","last_active":"2023-08-27T11:28:16.672779+0000","last_peered":"2023-08-27T11:28:16.672779+0000","last_clean":"2023-08-27T11:28:16.672779+0000","last_became_active":"2023-08-27T11:28:14.642687+0000","last_became_peered":"2023-08-27T11:28:14.642687+0000","last_unstale":"2023-08-27T11:28:16.672779+0000","last_undegraded":"2023-08-27T11:28:16.672779+0000","last_fullsized":"2023-08-27T11:28:16.672779+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:05.224224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673262+0000","last_change":"2023-08-27T11:28:13.647815+0000","last_active":"2023-08-27T11:28:16.673262+0000","last_peered":"2023-08-27T11:28:16.673262+0000","last_clean":"2023-08-27T11:28:16.673262+0000","last_became_active":"2023-08-27T11:28:13.647390+0000","last_became_peered":"2023-08-27T11:28:13.647390+0000","last_unstale":"2023-08-27T11:28:16.673262+0000","last_undegraded":"2023-08-27T11:28:16.673262+0000","last_fullsized":"2023-08-27T11:28:16.673262+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:45.914735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672801+0000","last_change":"2023-08-27T11:28:14.649394+0000","last_active":"2023-08-27T11:28:16.672801+0000","last_peered":"2023-08-27T11:28:16.672801+0000","last_clean":"2023-08-27T11:28:16.672801+0000","last_became_active":"2023-08-27T11:28:14.649110+0000","last_became_peered":"2023-08-27T11:28:14.649110+0000","last_unstale":"2023-08-27T11:28:16.672801+0000","last_undegraded":"2023-08-27T11:28:16.672801+0000","last_fullsized":"2023-08-27T11:28:16.672801+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:00:04.157887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.674016+0000","last_change":"2023-08-27T11:28:13.648193+0000","last_active":"2023-08-27T11:28:16.674016+0000","last_peered":"2023-08-27T11:28:16.674016+0000","last_clean":"2023-08-27T11:28:16.674016+0000","last_became_active":"2023-08-27T11:28:13.647916+0000","last_became_peered":"2023-08-27T11:28:13.647916+0000","last_unstale":"2023-08-27T11:28:16.674016+0000","last_undegraded":"2023-08-27T11:28:16.674016+0000","last_fullsized":"2023-08-27T11:28:16.674016+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:52:37.401119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635197+0000","last_change":"2023-08-27T11:28:14.656981+0000","last_active":"2023-08-27T11:28:15.635197+0000","last_peered":"2023-08-27T11:28:15.635197+0000","last_clean":"2023-08-27T11:28:15.635197+0000","last_became_active":"2023-08-27T11:28:14.656410+0000","last_became_peered":"2023-08-27T11:28:14.656410+0000","last_unstale":"2023-08-27T11:28:15.635197+0000","last_undegraded":"2023-08-27T11:28:15.635197+0000","last_fullsized":"2023-08-27T11:28:15.635197+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:09.674358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.639915+0000","last_change":"2023-08-27T11:28:13.644528+0000","last_active":"2023-08-27T11:28:15.639915+0000","last_peered":"2023-08-27T11:28:15.639915+0000","last_clean":"2023-08-27T11:28:15.639915+0000","last_became_active":"2023-08-27T11:28:13.644225+0000","last_became_peered":"2023-08-27T11:28:13.644225+0000","last_unstale":"2023-08-27T11:28:15.639915+0000","last_undegraded":"2023-08-27T11:28:15.639915+0000","last_fullsized":"2023-08-27T11:28:15.639915+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:34:08.260834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672827+0000","last_change":"2023-08-27T11:28:14.648888+0000","last_active":"2023-08-27T11:28:16.672827+0000","last_peered":"2023-08-27T11:28:16.672827+0000","last_clean":"2023-08-27T11:28:16.672827+0000","last_became_active":"2023-08-27T11:28:14.648593+0000","last_became_peered":"2023-08-27T11:28:14.648593+0000","last_unstale":"2023-08-27T11:28:16.672827+0000","last_undegraded":"2023-08-27T11:28:16.672827+0000","last_fullsized":"2023-08-27T11:28:16.672827+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:02:45.668209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"138'2","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673652+0000","last_change":"2023-08-27T11:28:13.652305+0000","last_active":"2023-08-27T11:28:16.673652+0000","last_peered":"2023-08-27T11:28:16.673652+0000","last_clean":"2023-08-27T11:28:16.673652+0000","last_became_active":"2023-08-27T11:28:13.652034+0000","last_became_peered":"2023-08-27T11:28:13.652034+0000","last_unstale":"2023-08-27T11:28:16.673652+0000","last_undegraded":"2023-08-27T11:28:16.673652+0000","last_fullsized":"2023-08-27T11:28:16.673652+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:03:30.384686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171729+0000","last_change":"2023-08-27T11:28:14.645680+0000","last_active":"2023-08-27T11:28:16.171729+0000","last_peered":"2023-08-27T11:28:16.171729+0000","last_clean":"2023-08-27T11:28:16.171729+0000","last_became_active":"2023-08-27T11:28:14.643431+0000","last_became_peered":"2023-08-27T11:28:14.643431+0000","last_unstale":"2023-08-27T11:28:16.171729+0000","last_undegraded":"2023-08-27T11:28:16.171729+0000","last_fullsized":"2023-08-27T11:28:16.171729+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:24:04.698188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673628+0000","last_change":"2023-08-27T11:28:13.648441+0000","last_active":"2023-08-27T11:28:16.673628+0000","last_peered":"2023-08-27T11:28:16.673628+0000","last_clean":"2023-08-27T11:28:16.673628+0000","last_became_active":"2023-08-27T11:28:13.648177+0000","last_became_peered":"2023-08-27T11:28:13.648177+0000","last_unstale":"2023-08-27T11:28:16.673628+0000","last_undegraded":"2023-08-27T11:28:16.673628+0000","last_fullsized":"2023-08-27T11:28:16.673628+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:45:23.429246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673436+0000","last_change":"2023-08-27T11:28:14.666855+0000","last_active":"2023-08-27T11:28:16.673436+0000","last_peered":"2023-08-27T11:28:16.673436+0000","last_clean":"2023-08-27T11:28:16.673436+0000","last_became_active":"2023-08-27T11:28:14.665012+0000","last_became_peered":"2023-08-27T11:28:14.665012+0000","last_unstale":"2023-08-27T11:28:16.673436+0000","last_undegraded":"2023-08-27T11:28:16.673436+0000","last_fullsized":"2023-08-27T11:28:16.673436+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:24.301561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635473+0000","last_change":"2023-08-27T11:28:13.646624+0000","last_active":"2023-08-27T11:28:15.635473+0000","last_peered":"2023-08-27T11:28:15.635473+0000","last_clean":"2023-08-27T11:28:15.635473+0000","last_became_active":"2023-08-27T11:28:13.646207+0000","last_became_peered":"2023-08-27T11:28:13.646207+0000","last_unstale":"2023-08-27T11:28:15.635473+0000","last_undegraded":"2023-08-27T11:28:15.635473+0000","last_fullsized":"2023-08-27T11:28:15.635473+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:53.054545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673287+0000","last_change":"2023-08-27T11:28:14.648725+0000","last_active":"2023-08-27T11:28:16.673287+0000","last_peered":"2023-08-27T11:28:16.673287+0000","last_clean":"2023-08-27T11:28:16.673287+0000","last_became_active":"2023-08-27T11:28:14.648299+0000","last_became_peered":"2023-08-27T11:28:14.648299+0000","last_unstale":"2023-08-27T11:28:16.673287+0000","last_undegraded":"2023-08-27T11:28:16.673287+0000","last_fullsized":"2023-08-27T11:28:16.673287+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:07:59.656326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635632+0000","last_change":"2023-08-27T11:28:13.648441+0000","last_active":"2023-08-27T11:28:15.635632+0000","last_peered":"2023-08-27T11:28:15.635632+0000","last_clean":"2023-08-27T11:28:15.635632+0000","last_became_active":"2023-08-27T11:28:13.648179+0000","last_became_peered":"2023-08-27T11:28:13.648179+0000","last_unstale":"2023-08-27T11:28:15.635632+0000","last_undegraded":"2023-08-27T11:28:15.635632+0000","last_fullsized":"2023-08-27T11:28:15.635632+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:21:35.536688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.639913+0000","last_change":"2023-08-27T11:28:14.650435+0000","last_active":"2023-08-27T11:28:15.639913+0000","last_peered":"2023-08-27T11:28:15.639913+0000","last_clean":"2023-08-27T11:28:15.639913+0000","last_became_active":"2023-08-27T11:28:14.650137+0000","last_became_peered":"2023-08-27T11:28:14.650137+0000","last_unstale":"2023-08-27T11:28:15.639913+0000","last_undegraded":"2023-08-27T11:28:15.639913+0000","last_fullsized":"2023-08-27T11:28:15.639913+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:27:29.886325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673059+0000","last_change":"2023-08-27T11:28:13.645381+0000","last_active":"2023-08-27T11:28:16.673059+0000","last_peered":"2023-08-27T11:28:16.673059+0000","last_clean":"2023-08-27T11:28:16.673059+0000","last_became_active":"2023-08-27T11:28:13.645065+0000","last_became_peered":"2023-08-27T11:28:13.645065+0000","last_unstale":"2023-08-27T11:28:16.673059+0000","last_undegraded":"2023-08-27T11:28:16.673059+0000","last_fullsized":"2023-08-27T11:28:16.673059+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:40:03.185204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641407+0000","last_change":"2023-08-27T11:28:14.666004+0000","last_active":"2023-08-27T11:28:15.641407+0000","last_peered":"2023-08-27T11:28:15.641407+0000","last_clean":"2023-08-27T11:28:15.641407+0000","last_became_active":"2023-08-27T11:28:14.665530+0000","last_became_peered":"2023-08-27T11:28:14.665530+0000","last_unstale":"2023-08-27T11:28:15.641407+0000","last_undegraded":"2023-08-27T11:28:15.641407+0000","last_fullsized":"2023-08-27T11:28:15.641407+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:27:57.013012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673140+0000","last_change":"2023-08-27T11:28:13.652345+0000","last_active":"2023-08-27T11:28:16.673140+0000","last_peered":"2023-08-27T11:28:16.673140+0000","last_clean":"2023-08-27T11:28:16.673140+0000","last_became_active":"2023-08-27T11:28:13.652061+0000","last_became_peered":"2023-08-27T11:28:13.652061+0000","last_unstale":"2023-08-27T11:28:16.673140+0000","last_undegraded":"2023-08-27T11:28:16.673140+0000","last_fullsized":"2023-08-27T11:28:16.673140+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:57:54.060241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.172060+0000","last_change":"2023-08-27T11:28:14.660024+0000","last_active":"2023-08-27T11:28:16.172060+0000","last_peered":"2023-08-27T11:28:16.172060+0000","last_clean":"2023-08-27T11:28:16.172060+0000","last_became_active":"2023-08-27T11:28:14.659767+0000","last_became_peered":"2023-08-27T11:28:14.659767+0000","last_unstale":"2023-08-27T11:28:16.172060+0000","last_undegraded":"2023-08-27T11:28:16.172060+0000","last_fullsized":"2023-08-27T11:28:16.172060+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:50.800979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.674171+0000","last_change":"2023-08-27T11:28:13.638797+0000","last_active":"2023-08-27T11:28:16.674171+0000","last_peered":"2023-08-27T11:28:16.674171+0000","last_clean":"2023-08-27T11:28:16.674171+0000","last_became_active":"2023-08-27T11:28:13.638526+0000","last_became_peered":"2023-08-27T11:28:13.638526+0000","last_unstale":"2023-08-27T11:28:16.674171+0000","last_undegraded":"2023-08-27T11:28:16.674171+0000","last_fullsized":"2023-08-27T11:28:16.674171+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:44.619300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673010+0000","last_change":"2023-08-27T11:28:14.663595+0000","last_active":"2023-08-27T11:28:16.673010+0000","last_peered":"2023-08-27T11:28:16.673010+0000","last_clean":"2023-08-27T11:28:16.673010+0000","last_became_active":"2023-08-27T11:28:14.663269+0000","last_became_peered":"2023-08-27T11:28:14.663269+0000","last_unstale":"2023-08-27T11:28:16.673010+0000","last_undegraded":"2023-08-27T11:28:16.673010+0000","last_fullsized":"2023-08-27T11:28:16.673010+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:28:31.283625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640292+0000","last_change":"2023-08-27T11:28:13.638139+0000","last_active":"2023-08-27T11:28:15.640292+0000","last_peered":"2023-08-27T11:28:15.640292+0000","last_clean":"2023-08-27T11:28:15.640292+0000","last_became_active":"2023-08-27T11:28:13.637703+0000","last_became_peered":"2023-08-27T11:28:13.637703+0000","last_unstale":"2023-08-27T11:28:15.640292+0000","last_undegraded":"2023-08-27T11:28:15.640292+0000","last_fullsized":"2023-08-27T11:28:15.640292+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:07:28.833582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673521+0000","last_change":"2023-08-27T11:28:14.663944+0000","last_active":"2023-08-27T11:28:16.673521+0000","last_peered":"2023-08-27T11:28:16.673521+0000","last_clean":"2023-08-27T11:28:16.673521+0000","last_became_active":"2023-08-27T11:28:14.663230+0000","last_became_peered":"2023-08-27T11:28:14.663230+0000","last_unstale":"2023-08-27T11:28:16.673521+0000","last_undegraded":"2023-08-27T11:28:16.673521+0000","last_fullsized":"2023-08-27T11:28:16.673521+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:15:26.373192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"138'4","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673526+0000","last_change":"2023-08-27T11:28:13.649633+0000","last_active":"2023-08-27T11:28:16.673526+0000","last_peered":"2023-08-27T11:28:16.673526+0000","last_clean":"2023-08-27T11:28:16.673526+0000","last_became_active":"2023-08-27T11:28:13.649362+0000","last_became_peered":"2023-08-27T11:28:13.649362+0000","last_unstale":"2023-08-27T11:28:16.673526+0000","last_undegraded":"2023-08-27T11:28:16.673526+0000","last_fullsized":"2023-08-27T11:28:16.673526+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:25:32.461248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":262,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640206+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:28:15.640206+0000","last_peered":"2023-08-27T11:28:15.640206+0000","last_clean":"2023-08-27T11:28:15.640206+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:28:15.640206+0000","last_undegraded":"2023-08-27T11:28:15.640206+0000","last_fullsized":"2023-08-27T11:28:15.640206+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:43:47.585658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673237+0000","last_change":"2023-08-27T11:28:14.653814+0000","last_active":"2023-08-27T11:28:16.673237+0000","last_peered":"2023-08-27T11:28:16.673237+0000","last_clean":"2023-08-27T11:28:16.673237+0000","last_became_active":"2023-08-27T11:28:14.652929+0000","last_became_peered":"2023-08-27T11:28:14.652929+0000","last_unstale":"2023-08-27T11:28:16.673237+0000","last_undegraded":"2023-08-27T11:28:16.673237+0000","last_fullsized":"2023-08-27T11:28:16.673237+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:40:04.831481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"138'1","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635185+0000","last_change":"2023-08-27T11:28:13.646599+0000","last_active":"2023-08-27T11:28:15.635185+0000","last_peered":"2023-08-27T11:28:15.635185+0000","last_clean":"2023-08-27T11:28:15.635185+0000","last_became_active":"2023-08-27T11:28:13.646235+0000","last_became_peered":"2023-08-27T11:28:13.646235+0000","last_unstale":"2023-08-27T11:28:15.635185+0000","last_undegraded":"2023-08-27T11:28:15.635185+0000","last_fullsized":"2023-08-27T11:28:15.635185+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:30:50.057457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":262,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640145+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:28:15.640145+0000","last_peered":"2023-08-27T11:28:15.640145+0000","last_clean":"2023-08-27T11:28:15.640145+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:28:15.640145+0000","last_undegraded":"2023-08-27T11:28:15.640145+0000","last_fullsized":"2023-08-27T11:28:15.640145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:48:35.482692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.172073+0000","last_change":"2023-08-27T11:28:14.643550+0000","last_active":"2023-08-27T11:28:16.172073+0000","last_peered":"2023-08-27T11:28:16.172073+0000","last_clean":"2023-08-27T11:28:16.172073+0000","last_became_active":"2023-08-27T11:28:14.643204+0000","last_became_peered":"2023-08-27T11:28:14.643204+0000","last_unstale":"2023-08-27T11:28:16.172073+0000","last_undegraded":"2023-08-27T11:28:16.172073+0000","last_fullsized":"2023-08-27T11:28:16.172073+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:55:02.903945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641244+0000","last_change":"2023-08-27T11:28:13.644990+0000","last_active":"2023-08-27T11:28:15.641244+0000","last_peered":"2023-08-27T11:28:15.641244+0000","last_clean":"2023-08-27T11:28:15.641244+0000","last_became_active":"2023-08-27T11:28:13.644707+0000","last_became_peered":"2023-08-27T11:28:13.644707+0000","last_unstale":"2023-08-27T11:28:15.641244+0000","last_undegraded":"2023-08-27T11:28:15.641244+0000","last_fullsized":"2023-08-27T11:28:15.641244+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:46.405496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":224,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916097+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:28:15.916097+0000","last_peered":"2023-08-27T11:28:15.916097+0000","last_clean":"2023-08-27T11:28:15.916097+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:28:15.916097+0000","last_undegraded":"2023-08-27T11:28:15.916097+0000","last_fullsized":"2023-08-27T11:28:15.916097+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:43:38.425879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640278+0000","last_change":"2023-08-27T11:28:14.661294+0000","last_active":"2023-08-27T11:28:15.640278+0000","last_peered":"2023-08-27T11:28:15.640278+0000","last_clean":"2023-08-27T11:28:15.640278+0000","last_became_active":"2023-08-27T11:28:14.660937+0000","last_became_peered":"2023-08-27T11:28:14.660937+0000","last_unstale":"2023-08-27T11:28:15.640278+0000","last_undegraded":"2023-08-27T11:28:15.640278+0000","last_fullsized":"2023-08-27T11:28:15.640278+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:30:51.064396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672907+0000","last_change":"2023-08-27T11:28:13.645934+0000","last_active":"2023-08-27T11:28:16.672907+0000","last_peered":"2023-08-27T11:28:16.672907+0000","last_clean":"2023-08-27T11:28:16.672907+0000","last_became_active":"2023-08-27T11:28:13.645672+0000","last_became_peered":"2023-08-27T11:28:13.645672+0000","last_unstale":"2023-08-27T11:28:16.672907+0000","last_undegraded":"2023-08-27T11:28:16.672907+0000","last_fullsized":"2023-08-27T11:28:16.672907+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:23:15.755614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":220,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673076+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:28:16.673076+0000","last_peered":"2023-08-27T11:28:16.673076+0000","last_clean":"2023-08-27T11:28:16.673076+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:28:16.673076+0000","last_undegraded":"2023-08-27T11:28:16.673076+0000","last_fullsized":"2023-08-27T11:28:16.673076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:32.388513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171668+0000","last_change":"2023-08-27T11:28:14.646006+0000","last_active":"2023-08-27T11:28:16.171668+0000","last_peered":"2023-08-27T11:28:16.171668+0000","last_clean":"2023-08-27T11:28:16.171668+0000","last_became_active":"2023-08-27T11:28:14.645708+0000","last_became_peered":"2023-08-27T11:28:14.645708+0000","last_unstale":"2023-08-27T11:28:16.171668+0000","last_undegraded":"2023-08-27T11:28:16.171668+0000","last_fullsized":"2023-08-27T11:28:16.171668+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:35:31.159606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"138'1","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673394+0000","last_change":"2023-08-27T11:28:13.650359+0000","last_active":"2023-08-27T11:28:16.673394+0000","last_peered":"2023-08-27T11:28:16.673394+0000","last_clean":"2023-08-27T11:28:16.673394+0000","last_became_active":"2023-08-27T11:28:13.650102+0000","last_became_peered":"2023-08-27T11:28:13.650102+0000","last_unstale":"2023-08-27T11:28:16.673394+0000","last_undegraded":"2023-08-27T11:28:16.673394+0000","last_fullsized":"2023-08-27T11:28:16.673394+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:56:03.277328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":220,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.672904+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:28:16.672904+0000","last_peered":"2023-08-27T11:28:16.672904+0000","last_clean":"2023-08-27T11:28:16.672904+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:28:16.672904+0000","last_undegraded":"2023-08-27T11:28:16.672904+0000","last_fullsized":"2023-08-27T11:28:16.672904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:09:49.885071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640920+0000","last_change":"2023-08-27T11:28:14.657479+0000","last_active":"2023-08-27T11:28:15.640920+0000","last_peered":"2023-08-27T11:28:15.640920+0000","last_clean":"2023-08-27T11:28:15.640920+0000","last_became_active":"2023-08-27T11:28:14.657095+0000","last_became_peered":"2023-08-27T11:28:14.657095+0000","last_unstale":"2023-08-27T11:28:15.640920+0000","last_undegraded":"2023-08-27T11:28:15.640920+0000","last_fullsized":"2023-08-27T11:28:15.640920+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:00:09.041184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"138'1","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635274+0000","last_change":"2023-08-27T11:28:13.646539+0000","last_active":"2023-08-27T11:28:15.635274+0000","last_peered":"2023-08-27T11:28:15.635274+0000","last_clean":"2023-08-27T11:28:15.635274+0000","last_became_active":"2023-08-27T11:28:13.646217+0000","last_became_peered":"2023-08-27T11:28:13.646217+0000","last_unstale":"2023-08-27T11:28:15.635274+0000","last_undegraded":"2023-08-27T11:28:15.635274+0000","last_fullsized":"2023-08-27T11:28:15.635274+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:42:08.449313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":370,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.673014+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:28:16.673014+0000","last_peered":"2023-08-27T11:28:16.673014+0000","last_clean":"2023-08-27T11:28:16.673014+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:28:16.673014+0000","last_undegraded":"2023-08-27T11:28:16.673014+0000","last_fullsized":"2023-08-27T11:28:16.673014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:11:01.893595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":230,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:16.171914+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:28:16.171914+0000","last_peered":"2023-08-27T11:28:16.171914+0000","last_clean":"2023-08-27T11:28:16.171914+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:28:16.171914+0000","last_undegraded":"2023-08-27T11:28:16.171914+0000","last_fullsized":"2023-08-27T11:28:16.171914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:39:48.116381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.640161+0000","last_change":"2023-08-27T11:28:14.654482+0000","last_active":"2023-08-27T11:28:15.640161+0000","last_peered":"2023-08-27T11:28:15.640161+0000","last_clean":"2023-08-27T11:28:15.640161+0000","last_became_active":"2023-08-27T11:28:14.654067+0000","last_became_peered":"2023-08-27T11:28:14.654067+0000","last_unstale":"2023-08-27T11:28:15.640161+0000","last_undegraded":"2023-08-27T11:28:15.640161+0000","last_fullsized":"2023-08-27T11:28:15.640161+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:55:02.771821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"138'2","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641515+0000","last_change":"2023-08-27T11:28:13.645061+0000","last_active":"2023-08-27T11:28:15.641515+0000","last_peered":"2023-08-27T11:28:15.641515+0000","last_clean":"2023-08-27T11:28:15.641515+0000","last_became_active":"2023-08-27T11:28:13.644717+0000","last_became_peered":"2023-08-27T11:28:13.644717+0000","last_unstale":"2023-08-27T11:28:15.641515+0000","last_undegraded":"2023-08-27T11:28:15.641515+0000","last_fullsized":"2023-08-27T11:28:15.641515+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:55:27.308469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":224,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.635349+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:28:15.635349+0000","last_peered":"2023-08-27T11:28:15.635349+0000","last_clean":"2023-08-27T11:28:15.635349+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:28:15.635349+0000","last_undegraded":"2023-08-27T11:28:15.635349+0000","last_fullsized":"2023-08-27T11:28:15.635349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:45:26.488760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916627+0000","last_change":"2023-08-27T11:28:14.655078+0000","last_active":"2023-08-27T11:28:15.916627+0000","last_peered":"2023-08-27T11:28:15.916627+0000","last_clean":"2023-08-27T11:28:15.916627+0000","last_became_active":"2023-08-27T11:28:14.654406+0000","last_became_peered":"2023-08-27T11:28:14.654406+0000","last_unstale":"2023-08-27T11:28:15.916627+0000","last_undegraded":"2023-08-27T11:28:15.916627+0000","last_fullsized":"2023-08-27T11:28:15.916627+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:13.623205+0000","last_clean_scrub_stamp":"2023-08-27T11:28:13.623205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:36:29.030283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"138'2","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.916441+0000","last_change":"2023-08-27T11:28:13.645173+0000","last_active":"2023-08-27T11:28:15.916441+0000","last_peered":"2023-08-27T11:28:15.916441+0000","last_clean":"2023-08-27T11:28:15.916441+0000","last_became_active":"2023-08-27T11:28:13.644874+0000","last_became_peered":"2023-08-27T11:28:13.644874+0000","last_unstale":"2023-08-27T11:28:15.916441+0000","last_undegraded":"2023-08-27T11:28:15.916441+0000","last_fullsized":"2023-08-27T11:28:15.916441+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:12.616265+0000","last_clean_scrub_stamp":"2023-08-27T11:28:12.616265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:36:44.633556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":250,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-08-27T11:28:15.641553+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:28:15.641553+0000","last_peered":"2023-08-27T11:28:15.641553+0000","last_clean":"2023-08-27T11:28:15.641553+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:28:15.641553+0000","last_undegraded":"2023-08-27T11:28:15.641553+0000","last_fullsized":"2023-08-27T11:28:15.641553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:37:28.710975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2338816,"data_stored":1419180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.44,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.72799999999999998,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.39800000000000002,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.75600000000000001,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.51900000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55700000000000005,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.872},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54000000000000004,"15min":0.52500000000000002},"min":{"1min":0.374,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.47199999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.497},"min":{"1min":0.377,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.72399999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.437},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.46100000000000002,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.90200000000000002},"last":0.57199999999999995}]},{"osd":3,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.443,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77700000000000002,"5min":0.996,"15min":0.996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.48899999999999999,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.76400000000000001,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.66500000000000004}]},{"osd":4,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.496,"15min":0.504},"min":{"1min":0.35899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.42399999999999999,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.73199999999999998,"5min":1.224,"15min":1.224},"last":0.59899999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.78100000000000003,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73199999999999998,"5min":0.78400000000000003,"15min":0.84799999999999998},"last":0.63300000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84499999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.47299999999999998,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.65100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1761280,"data_stored":835535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.29099999999999998},"max":{"1min":0.73399999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.374},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80300000000000005,"5min":0.83799999999999997,"15min":0.89800000000000002},"last":0.54000000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.873},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.84399999999999997},"last":0.47899999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.85899999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.69699999999999995,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.52400000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57599999999999996,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84499999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.80200000000000005,"5min":0.80700000000000005,"15min":0.93799999999999994},"last":0.504}]},{"osd":4,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.753,"5min":0.753,"15min":0.83099999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.374},"max":{"1min":0.81399999999999995,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.61199999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.875},"last":0.625}]},{"osd":7,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.34899999999999998},"max":{"1min":0.82999999999999996,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.59699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574988,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33476,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338364,"statfs":{"total":96636764160,"available":96602484736,"internally_reserved":0,"allocated":2351104,"data_stored":1425833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.311,"15min":0.55400000000000005},"min":{"1min":0.19500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.42599999999999999,"5min":0.502,"15min":11.971},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.30099999999999999,"5min":0.33200000000000002,"15min":0.80800000000000005},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.39500000000000002,"5min":0.63800000000000001,"15min":23.045999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.34000000000000002,"15min":0.58299999999999996},"min":{"1min":0.27700000000000002,"5min":0.23400000000000001,"15min":0.221},"max":{"1min":0.45500000000000002,"5min":0.52300000000000002,"15min":11.928000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.317,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.22800000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.38800000000000001,"5min":0.52000000000000002,"15min":0.58599999999999997},"last":0.52200000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38800000000000001,"15min":0.60399999999999998},"min":{"1min":0.27500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.47399999999999998,"5min":0.58099999999999996,"15min":10.99},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.372,"15min":0.84499999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23499999999999999},"max":{"1min":0.45000000000000001,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.44700000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57399999999999995,"15min":0.59499999999999997},"min":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.33400000000000002},"max":{"1min":0.79500000000000004,"5min":0.90700000000000003,"15min":1.26},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.77300000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.69999999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":1.0409999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36199999999999999},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.46300000000000002,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.71599999999999997,"5min":0.88700000000000001,"15min":1.296},"last":0.67900000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.80400000000000005},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.371},"max":{"1min":0.83599999999999997,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":1.0589999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81200000000000006,"5min":0.83999999999999997,"15min":22.952000000000002},"last":0.71399999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":1.0529999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.84199999999999997,"15min":22.988},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.85899999999999999,"5min":0.86399999999999999,"15min":1.327},"last":0.72799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1757184,"data_stored":828179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.311},"min":{"1min":0.223,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.46200000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.33300000000000002,"15min":0.34899999999999998},"min":{"1min":0.253,"5min":0.223,"15min":0.223},"max":{"1min":0.5,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.38100000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34100000000000003},"min":{"1min":0.251,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.44800000000000001,"5min":0.628,"15min":0.63200000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.33900000000000002},"min":{"1min":0.26600000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51000000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.28799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34599999999999997},"min":{"1min":0.247,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47099999999999997,"5min":0.50800000000000001,"15min":0.59599999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.371,"15min":0.373},"min":{"1min":0.30299999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.60599999999999998},"last":0.35999999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.85399999999999998,"15min":1.214},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.42999999999999999,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.88400000000000001,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.41999999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.41099999999999998,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.72599999999999998,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.40200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78500000000000003,"5min":0.85399999999999998,"15min":1.603},"last":0.503}]},{"osd":6,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89000000000000001,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.39900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.878,"15min":0.878},"last":0.61699999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.39900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.79,"5min":3.79,"15min":3.79},"last":0.57399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1761280,"data_stored":836047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.30199999999999999},"min":{"1min":0.22800000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.377,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.31900000000000001,"15min":0.318},"min":{"1min":0.251,"5min":0.22700000000000001,"15min":0.19700000000000001},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.629},"last":0.32900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.315,"15min":0.32200000000000001},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.40799999999999997,"5min":0.50800000000000001,"15min":0.54400000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.34100000000000003,"15min":0.34699999999999998},"min":{"1min":0.29999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.749},"last":0.38700000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.379},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.73699999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.39500000000000002},"min":{"1min":0.33600000000000002,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":0.53500000000000003,"5min":0.55100000000000005,"15min":0.69499999999999995},"last":0.36799999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.41799999999999998,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.879},"last":0.55000000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.47199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.42899999999999999,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.52600000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.46000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.47899999999999998,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.83599999999999997},"last":0.53800000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.46899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.5},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.56200000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1765376,"data_stored":835625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.29799999999999999,"15min":0.30299999999999999},"min":{"1min":0.221,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.435,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.252},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.318},"min":{"1min":0.26500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.28999999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.33800000000000002,"15min":0.32200000000000001},"min":{"1min":0.24399999999999999,"5min":0.22700000000000001,"15min":0.19500000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.36399999999999999,"15min":0.35199999999999998},"min":{"1min":0.27500000000000002,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":0.32600000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.376,"15min":0.36599999999999999},"min":{"1min":0.27400000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.373,"15min":0.36899999999999999},"min":{"1min":0.28699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.375}]},{"osd":4,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.55200000000000005},"min":{"1min":0.41099999999999998,"5min":0.33900000000000002,"15min":0.28599999999999998},"max":{"1min":0.80700000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.47999999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55800000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.38,"15min":0.314},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.58799999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.44700000000000001,"5min":0.379,"15min":0.373},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.60999999999999999},"min":{"1min":0.47999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":1.0069999999999999,"15min":1.111},"last":0.77600000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58499999999999996,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":1.032,"15min":1.236},"last":0.755},{"interface":"front","average":{"1min":0.625,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1773568,"data_stored":837111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.33400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.501},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.40899999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55300000000000005}]},{"osd":1,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.41499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68999999999999995,"5min":0.746,"15min":0.76500000000000001},"last":0.308},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.44,"5min":0.308,"15min":0.308},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.35399999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.35099999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69199999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.38600000000000001,"5min":0.32400000000000001,"15min":0.32200000000000001},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.56899999999999995}]},{"osd":3,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56499999999999995,"15min":0.54200000000000004},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73699999999999999,"5min":0.80300000000000005,"15min":0.80600000000000005},"last":0.58299999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.51600000000000001,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60399999999999998,"15min":0.59499999999999997},"min":{"1min":0.46200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.61299999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.38100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73999999999999999,"5min":0.78900000000000003,"15min":0.90600000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.62,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.60099999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.47399999999999998,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.89300000000000002},"last":0.626}]}]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1773568,"data_stored":837099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73899999999999999,"5min":0.753,"15min":0.96899999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39300000000000002,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.68200000000000005,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.66300000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65800000000000003,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68400000000000005,"5min":0.72899999999999998,"15min":1.075},"last":0.68000000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.33200000000000002},"max":{"1min":0.72099999999999997,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.41099999999999998,"5min":0.373,"15min":0.307},"max":{"1min":0.753,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.747}]},{"osd":3,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":1.03},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.35199999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":1.026},"last":0.77700000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.371,"5min":0.31,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.50700000000000001,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.79600000000000004,"5min":0.83899999999999997,"15min":0.91400000000000003},"last":0.70599999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.79400000000000004,"5min":0.79700000000000004,"15min":0.88200000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":0.81499999999999995,"15min":0.94199999999999995},"last":0.72599999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.41499999999999998,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.95899999999999996},"last":0.495},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.31},"max":{"1min":0.82299999999999995,"5min":0.85599999999999998,"15min":0.92000000000000004},"last":0.51900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:19.138 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:28:19.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:28:19.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.298+0000 7f847d51e700 1 Processor -- start 2023-08-27T11:28:19.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.298+0000 7f847d51e700 1 -- start start 2023-08-27T11:28:19.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.298+0000 7f847d51e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478157980 0x7f84780d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.298+0000 7f847d51e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f84780b6680 0x7f84780b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f847d51e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8478160ff0 con 0x7f84780cef20 2023-08-27T11:28:19.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f847d51e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8478161340 con 0x7f84780b6680 2023-08-27T11:28:19.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f847d51e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84780b6ff0 con 0x7f8478157980 2023-08-27T11:28:19.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f84767fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f84780cef20 0x7f8478161760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58730/0 (socket says 172.21.15.112:58730) 2023-08-27T11:28:19.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f84767fc700 1 -- 172.21.15.112:0/1898189060 learned_addr learned my addr 172.21.15.112:0/1898189060 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:19.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f84777fe700 1 --2- 172.21.15.112:0/1898189060 >> v2:172.21.15.138:3300/0 conn(0x7f84780b6680 0x7f84780b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8476ffd700 1 --2- 172.21.15.112:0/1898189060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478157980 0x7f84780d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395716385 0 0) 0x7f8478160ff0 con 0x7f84780cef20 2023-08-27T11:28:19.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84600035b0 con 0x7f84780cef20 2023-08-27T11:28:19.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2409484637 0 0) 0x7f84600035b0 con 0x7f84780cef20 2023-08-27T11:28:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478157980 msgr2=0x7f84780d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 --2- 172.21.15.112:0/1898189060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478157980 0x7f84780d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 >> v2:172.21.15.138:3300/0 conn(0x7f84780b6680 msgr2=0x7f84780b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 --2- 172.21.15.112:0/1898189060 >> v2:172.21.15.138:3300/0 conn(0x7f84780b6680 0x7f84780b6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8478161e70 con 0x7f84780cef20 2023-08-27T11:28:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f8468001670 con 0x7f84780cef20 2023-08-27T11:28:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 >> v1:172.21.15.112:6789/0 conn(0x7f84780cef20 legacy=0x7f8478161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 --2- 172.21.15.112:0/1898189060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478157980 0x7f8460003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 --2- 172.21.15.112:0/1898189060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8460003f80 0x7f8460004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 --2- 172.21.15.112:0/1898189060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460004d90 0x7f8460005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8478161340 con 0x7f8460004d90 2023-08-27T11:28:19.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84780b6ff0 con 0x7f8460003f80 2023-08-27T11:28:19.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846000baf0 con 0x7f8478157980 2023-08-27T11:28:19.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f84777fe700 1 --2- 172.21.15.112:0/1898189060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460004d90 0x7f8460005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f8476ffd700 1 --2- 172.21.15.112:0/1898189060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478157980 0x7f8460003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.299+0000 7f84767fc700 1 --2- 172.21.15.112:0/1898189060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8460003f80 0x7f8460004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f84777fe700 1 -- 172.21.15.112:0/1898189060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478157980 msgr2=0x7f8460003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:19.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f84777fe700 1 --2- 172.21.15.112:0/1898189060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478157980 0x7f8460003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f84777fe700 1 -- 172.21.15.112:0/1898189060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8460003f80 msgr2=0x7f8460004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:19.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f84777fe700 1 --2- 172.21.15.112:0/1898189060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8460003f80 0x7f8460004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f84777fe700 1 -- 172.21.15.112:0/1898189060 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84600035b0 con 0x7f8460004d90 2023-08-27T11:28:19.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f84767fc700 1 --2- 172.21.15.112:0/1898189060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8460003f80 0x7f8460004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:19.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f84777fe700 1 --2- 172.21.15.112:0/1898189060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460004d90 0x7f8460005210 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f8470009fc0 tx=0x7f8460005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b69090c0c1564e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:19.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847000ec30 con 0x7f8460004d90 2023-08-27T11:28:19.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847000f040 con 0x7f8460004d90 2023-08-27T11:28:19.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f8475ffb700 1 -- 172.21.15.112:0/1898189060 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470014480 con 0x7f8460004d90 2023-08-27T11:28:19.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f847d51e700 1 -- 172.21.15.112:0/1898189060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460004d90 msgr2=0x7f8460005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f847d51e700 1 --2- 172.21.15.112:0/1898189060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460004d90 0x7f8460005210 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f8470009fc0 tx=0x7f8460005880 comp rx=0 tx=0).stop 2023-08-27T11:28:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f84777fe700 1 -- 172.21.15.112:0/1898189060 reap_dead start 2023-08-27T11:28:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f847d51e700 1 -- 172.21.15.112:0/1898189060 shutdown_connections 2023-08-27T11:28:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f847d51e700 1 -- 172.21.15.112:0/1898189060 >> 172.21.15.112:0/1898189060 conn(0x7f8478066010 msgr2=0x7f84780663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f847d51e700 1 -- 172.21.15.112:0/1898189060 shutdown_connections 2023-08-27T11:28:19.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.300+0000 7f847d51e700 1 -- 172.21.15.112:0/1898189060 wait complete. 2023-08-27T11:28:19.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.302+0000 7f847d51e700 1 Processor -- start 2023-08-27T11:28:19.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.302+0000 7f847d51e700 1 -- start start 2023-08-27T11:28:19.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.302+0000 7f847d51e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84780b5d10 0x7f84781561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.302+0000 7f847d51e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478156730 0x7f8478157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.302+0000 7f847d51e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84781582a0 0x7f84781f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f847d51e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8478158750 con 0x7f84781582a0 2023-08-27T11:28:19.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f847d51e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8478146670 con 0x7f84780b5d10 2023-08-27T11:28:19.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f847d51e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8478146950 con 0x7f8478156730 2023-08-27T11:28:19.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f84777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84781582a0 0x7f84781f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f84777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84781582a0 0x7f84781f5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36132/0 (socket says 172.21.15.112:36132) 2023-08-27T11:28:19.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f84777fe700 1 -- 172.21.15.112:0/3725358708 learned_addr learned my addr 172.21.15.112:0/3725358708 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:19.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f84767fc700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478156730 0x7f8478157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f8476ffd700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84780b5d10 0x7f84781561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f84777fe700 1 -- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478156730 msgr2=0x7f8478157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f84777fe700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478156730 0x7f8478157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f84777fe700 1 -- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84780b5d10 msgr2=0x7f84781561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f84777fe700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84780b5d10 0x7f84781561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f84777fe700 1 -- 172.21.15.112:0/3725358708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8478146c30 con 0x7f84781582a0 2023-08-27T11:28:19.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f8476ffd700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84780b5d10 0x7f84781561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:19.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f84767fc700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478156730 0x7f8478157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:19.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f84777fe700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84781582a0 0x7f84781f5c00 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8470016000 tx=0x7f8470009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:19.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f845f7fe700 1 -- 172.21.15.112:0/3725358708 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470007000 con 0x7f84781582a0 2023-08-27T11:28:19.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f845f7fe700 1 -- 172.21.15.112:0/3725358708 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8470015b30 con 0x7f84781582a0 2023-08-27T11:28:19.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f845f7fe700 1 -- 172.21.15.112:0/3725358708 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847001d420 con 0x7f84781582a0 2023-08-27T11:28:19.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f847d51e700 1 -- 172.21.15.112:0/3725358708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8478146f10 con 0x7f84781582a0 2023-08-27T11:28:19.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.303+0000 7f847d51e700 1 -- 172.21.15.112:0/3725358708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84781474d0 con 0x7f84781582a0 2023-08-27T11:28:19.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.304+0000 7f847d51e700 1 -- 172.21.15.112:0/3725358708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8478067a20 con 0x7f84781582a0 2023-08-27T11:28:19.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.305+0000 7f845f7fe700 1 -- 172.21.15.112:0/3725358708 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f847001d580 con 0x7f84781582a0 2023-08-27T11:28:19.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.305+0000 7f845f7fe700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f846406a4d0 0x7f846406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.305+0000 7f845f7fe700 1 -- 172.21.15.112:0/3725358708 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f847008a4d0 con 0x7f84781582a0 2023-08-27T11:28:19.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.305+0000 7f8476ffd700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f846406a4d0 0x7f846406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.306+0000 7f8476ffd700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f846406a4d0 0x7f846406c990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f846c009b70 tx=0x7f846c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:19.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.309+0000 7f845f7fe700 1 -- 172.21.15.112:0/3725358708 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8470055680 con 0x7f84781582a0 2023-08-27T11:28:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.574+0000 7f847d51e700 1 -- 172.21.15.112:0/3725358708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f84780d1a00 con 0x7f84781582a0 2023-08-27T11:28:19.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.577+0000 7f845f7fe700 1 -- 172.21.15.112:0/3725358708 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8470058ca0 con 0x7f84781582a0 2023-08-27T11:28:19.579 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-08-27T11:28:19.579 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA== 2023-08-27T11:28:19.579 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2023-08-27T11:28:19.579 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2023-08-27T11:28:19.580 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:28:19.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.579+0000 7f847d51e700 1 -- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f846406a4d0 msgr2=0x7f846406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.579+0000 7f847d51e700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f846406a4d0 0x7f846406c990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f846c009b70 tx=0x7f846c008040 comp rx=0 tx=0).stop 2023-08-27T11:28:19.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.579+0000 7f847d51e700 1 -- 172.21.15.112:0/3725358708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84781582a0 msgr2=0x7f84781f5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.579+0000 7f847d51e700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84781582a0 0x7f84781f5c00 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8470016000 tx=0x7f8470009750 comp rx=0 tx=0).stop 2023-08-27T11:28:19.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.579+0000 7f847d51e700 1 -- 172.21.15.112:0/3725358708 shutdown_connections 2023-08-27T11:28:19.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.579+0000 7f847d51e700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f846406a4d0 0x7f846406c990 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.579+0000 7f847d51e700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84781582a0 0x7f84781f5c00 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.579+0000 7f847d51e700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84780b5d10 0x7f84781561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.579+0000 7f847d51e700 1 --2- 172.21.15.112:0/3725358708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8478156730 0x7f8478157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.579+0000 7f847d51e700 1 -- 172.21.15.112:0/3725358708 >> 172.21.15.112:0/3725358708 conn(0x7f8478066010 msgr2=0x7f847806bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:19.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.580+0000 7f847d51e700 1 -- 172.21.15.112:0/3725358708 shutdown_connections 2023-08-27T11:28:19.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.580+0000 7f847d51e700 1 -- 172.21.15.112:0/3725358708 wait complete. 2023-08-27T11:28:19.584 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2023-08-27T11:28:19.593 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:19.610 INFO:tasks.ceph.mds.d.smithi138.stderr:2023-08-27T11:28:19.608+0000 7f8a3bdef700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:28:19.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.713+0000 7f16b42b9700 1 Processor -- start 2023-08-27T11:28:19.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.713+0000 7f16b42b9700 1 -- start start 2023-08-27T11:28:19.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b42b9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16ac0b1400 0x7f16ac0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b42b9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ac161850 con 0x7f16ac161ce0 2023-08-27T11:28:19.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b42b9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ac0cfc10 con 0x7f16ac0b17e0 2023-08-27T11:28:19.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b42b9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac161200 con 0x7f16ac0b1400 2023-08-27T11:28:19.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b2856700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f16ac161ce0 0x7f16ac06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58752/0 (socket says 172.21.15.112:58752) 2023-08-27T11:28:19.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b2856700 1 -- 172.21.15.112:0/1720894374 learned_addr learned my addr 172.21.15.112:0/1720894374 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:19.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b2055700 1 --2- 172.21.15.112:0/1720894374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16ac0b1400 0x7f16ac0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788863131 0 0) 0x7f16ac161850 con 0x7f16ac161ce0 2023-08-27T11:28:19.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16980035b0 con 0x7f16ac161ce0 2023-08-27T11:28:19.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1906774499 0 0) 0x7f16980035b0 con 0x7f16ac161ce0 2023-08-27T11:28:19.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16ac0b1400 msgr2=0x7f16ac0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:19.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b1053700 1 --2- 172.21.15.112:0/1720894374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16ac0b1400 0x7f16ac0cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 >> v1:172.21.15.138:6789/0 conn(0x7f16ac0b17e0 legacy=0x7f16ac06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac1614e0 con 0x7f16ac161ce0 2023-08-27T11:28:19.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f16a40019f0 con 0x7f16ac161ce0 2023-08-27T11:28:19.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 >> v1:172.21.15.112:6789/0 conn(0x7f16ac161ce0 legacy=0x7f16ac06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.714+0000 7f16b1053700 1 --2- 172.21.15.112:0/1720894374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1698003a30 0x7f1698003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b1053700 1 --2- 172.21.15.112:0/1720894374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16ac0b1400 0x7f1698004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b1053700 1 --2- 172.21.15.112:0/1720894374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1698004d90 0x7f1698005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac161200 con 0x7f1698004d90 2023-08-27T11:28:19.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169800bb40 con 0x7f1698003a30 2023-08-27T11:28:19.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169800bd00 con 0x7f16ac0b1400 2023-08-27T11:28:19.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b2856700 1 --2- 172.21.15.112:0/1720894374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1698004d90 0x7f1698005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b2055700 1 --2- 172.21.15.112:0/1720894374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1698003a30 0x7f1698003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b1854700 1 --2- 172.21.15.112:0/1720894374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16ac0b1400 0x7f1698004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b2856700 1 -- 172.21.15.112:0/1720894374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16ac0b1400 msgr2=0x7f1698004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b2856700 1 --2- 172.21.15.112:0/1720894374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16ac0b1400 0x7f1698004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b2856700 1 -- 172.21.15.112:0/1720894374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1698003a30 msgr2=0x7f1698003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b2856700 1 --2- 172.21.15.112:0/1720894374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1698003a30 0x7f1698003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b2856700 1 -- 172.21.15.112:0/1720894374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16980035b0 con 0x7f1698004d90 2023-08-27T11:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b2856700 1 --2- 172.21.15.112:0/1720894374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1698004d90 0x7f1698005210 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f16a400a830 tx=0x7f1698005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ba202bce4543c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a400a2f0 con 0x7f1698004d90 2023-08-27T11:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a4012600 con 0x7f1698004d90 2023-08-27T11:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.715+0000 7f16b1053700 1 -- 172.21.15.112:0/1720894374 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a40129b0 con 0x7f1698004d90 2023-08-27T11:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.716+0000 7f16b42b9700 1 -- 172.21.15.112:0/1720894374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1698004d90 msgr2=0x7f1698005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.716+0000 7f16b42b9700 1 --2- 172.21.15.112:0/1720894374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1698004d90 0x7f1698005210 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f16a400a830 tx=0x7f1698005880 comp rx=0 tx=0).stop 2023-08-27T11:28:19.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.716+0000 7f16b2856700 1 -- 172.21.15.112:0/1720894374 reap_dead start 2023-08-27T11:28:19.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.716+0000 7f16b42b9700 1 -- 172.21.15.112:0/1720894374 shutdown_connections 2023-08-27T11:28:19.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.716+0000 7f16b42b9700 1 -- 172.21.15.112:0/1720894374 >> 172.21.15.112:0/1720894374 conn(0x7f16ac0d0000 msgr2=0x7f16ac0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:19.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.716+0000 7f16b42b9700 1 -- 172.21.15.112:0/1720894374 shutdown_connections 2023-08-27T11:28:19.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.716+0000 7f16b42b9700 1 -- 172.21.15.112:0/1720894374 wait complete. 2023-08-27T11:28:19.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.717+0000 7f16b42b9700 1 Processor -- start 2023-08-27T11:28:19.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.717+0000 7f16b42b9700 1 -- start start 2023-08-27T11:28:19.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b42b9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1698004d90 0x7f16ac1f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b42b9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16ac1f7600 0x7f16ac14fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b42b9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16ac0cabd0 0x7f16ac0cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b42b9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac0c75c0 con 0x7f16ac1f7600 2023-08-27T11:28:19.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b42b9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac0cb5a0 con 0x7f1698004d90 2023-08-27T11:28:19.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b42b9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac070eb0 con 0x7f16ac0cabd0 2023-08-27T11:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b1854700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16ac1f7600 0x7f16ac14fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b1854700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16ac1f7600 0x7f16ac14fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36150/0 (socket says 172.21.15.112:36150) 2023-08-27T11:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b1854700 1 -- 172.21.15.112:0/3046590757 learned_addr learned my addr 172.21.15.112:0/3046590757 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b2856700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16ac0cabd0 0x7f16ac0cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b2055700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1698004d90 0x7f16ac1f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b1854700 1 -- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16ac0cabd0 msgr2=0x7f16ac0cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b1854700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16ac0cabd0 0x7f16ac0cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b1854700 1 -- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1698004d90 msgr2=0x7f16ac1f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b1854700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1698004d90 0x7f16ac1f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b1854700 1 -- 172.21.15.112:0/3046590757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac1f3c50 con 0x7f16ac1f7600 2023-08-27T11:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b1854700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16ac1f7600 0x7f16ac14fe20 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f169c00f7f0 tx=0x7f169c00fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16a2ffd700 1 -- 172.21.15.112:0/3046590757 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0052e0 con 0x7f16ac1f7600 2023-08-27T11:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16a2ffd700 1 -- 172.21.15.112:0/3046590757 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c005440 con 0x7f16ac1f7600 2023-08-27T11:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16b42b9700 1 -- 172.21.15.112:0/3046590757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ac1f3f30 con 0x7f16ac1f7600 2023-08-27T11:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.718+0000 7f16a2ffd700 1 -- 172.21.15.112:0/3046590757 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0038c0 con 0x7f16ac1f7600 2023-08-27T11:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.719+0000 7f16b42b9700 1 -- 172.21.15.112:0/3046590757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ac1f7a80 con 0x7f16ac1f7600 2023-08-27T11:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.719+0000 7f16b42b9700 1 -- 172.21.15.112:0/3046590757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ac0c7700 con 0x7f16ac1f7600 2023-08-27T11:28:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.720+0000 7f16a2ffd700 1 -- 172.21.15.112:0/3046590757 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f169c003a20 con 0x7f16ac1f7600 2023-08-27T11:28:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.720+0000 7f16a2ffd700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f169006a500 0x7f169006c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.720+0000 7f16a2ffd700 1 -- 172.21.15.112:0/3046590757 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f169c088870 con 0x7f16ac1f7600 2023-08-27T11:28:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.720+0000 7f16b2055700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f169006a500 0x7f169006c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.721+0000 7f16b2055700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f169006a500 0x7f169006c9c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009c60 tx=0x7f16a8009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:19.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.725+0000 7f16a2ffd700 1 -- 172.21.15.112:0/3046590757 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f16ac0c7700 con 0x7f16ac1f7600 2023-08-27T11:28:19.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.955+0000 7f16b42b9700 1 -- 172.21.15.112:0/3046590757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16ac0c7700 con 0x7f16ac1f7600 2023-08-27T11:28:19.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.955+0000 7f16a2ffd700 1 -- 172.21.15.112:0/3046590757 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4885 (secure 0 0 0) 0x7f16ac0c7700 con 0x7f16ac1f7600 2023-08-27T11:28:19.957 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:19.957 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/739970856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":739970856},{"type":"v1","addr":"172.21.15.138:6837","nonce":739970856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3233069521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3233069521},{"type":"v1","addr":"172.21.15.112:6834","nonce":3233069521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2559241026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2559241026},{"type":"v1","addr":"172.21.15.112:6835","nonce":2559241026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:14.032146+0000","modified":"2023-08-27T11:28:17.665929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6836/529571603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":529571603},{"type":"v1","addr":"172.21.15.138:6836","nonce":529571603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-08-27T11:28:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.957+0000 7f16b42b9700 1 -- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f169006a500 msgr2=0x7f169006c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.957+0000 7f16b42b9700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f169006a500 0x7f169006c9c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009c60 tx=0x7f16a8009210 comp rx=0 tx=0).stop 2023-08-27T11:28:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.957+0000 7f16b42b9700 1 -- 172.21.15.112:0/3046590757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16ac1f7600 msgr2=0x7f16ac14fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.957+0000 7f16b42b9700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16ac1f7600 0x7f16ac14fe20 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f169c00f7f0 tx=0x7f169c00fb00 comp rx=0 tx=0).stop 2023-08-27T11:28:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.957+0000 7f16b42b9700 1 -- 172.21.15.112:0/3046590757 shutdown_connections 2023-08-27T11:28:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.957+0000 7f16b42b9700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f169006a500 0x7f169006c9c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.957+0000 7f16b42b9700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16ac0cabd0 0x7f16ac0cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.957+0000 7f16b42b9700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1698004d90 0x7f16ac1f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.957+0000 7f16b42b9700 1 --2- 172.21.15.112:0/3046590757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16ac1f7600 0x7f16ac14fe20 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.957+0000 7f16b42b9700 1 -- 172.21.15.112:0/3046590757 >> 172.21.15.112:0/3046590757 conn(0x7f16ac0d0000 msgr2=0x7f16ac15f030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.957+0000 7f16b42b9700 1 -- 172.21.15.112:0/3046590757 shutdown_connections 2023-08-27T11:28:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:19.957+0000 7f16b42b9700 1 -- 172.21.15.112:0/3046590757 wait complete. 2023-08-27T11:28:19.962 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 94 2023-08-27T11:28:20.373 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:28:14.032146+0000', 'modified': '2023-08-27T11:28:17.665929+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8472': {'gid': 8472, 'name': 'd', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6836/529571603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 529571603}, {'type': 'v1', 'addr': '172.21.15.138:6836', 'nonce': 529571603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:28:20.374 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:28:20.374 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-08-27T11:28:20.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.496+0000 7f0db1c6e700 1 Processor -- start 2023-08-27T11:28:20.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.496+0000 7f0db1c6e700 1 -- start start 2023-08-27T11:28:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0db1c6e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0dac06adb0 0x7f0dac06b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0db1c6e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0dac0cdbd0 0x7f0dac0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0db1c6e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dac15eb60 con 0x7f0dac151060 2023-08-27T11:28:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0db1c6e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dac06b770 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0db1c6e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0b02f0 con 0x7f0dac06adb0 2023-08-27T11:28:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0dabfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0dac0cdbd0 0x7f0dac0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0dabfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0dac0cdbd0 0x7f0dac0ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36160/0 (socket says 172.21.15.112:36160) 2023-08-27T11:28:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0dabfff700 1 -- 172.21.15.112:0/3167504705 learned_addr learned my addr 172.21.15.112:0/3167504705 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0daaffd700 1 --2- 172.21.15.112:0/3167504705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0dac06adb0 0x7f0dac06b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0dabfff700 1 -- 172.21.15.112:0/3167504705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0dac06adb0 msgr2=0x7f0dac06b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:20.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0dabfff700 1 --2- 172.21.15.112:0/3167504705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0dac06adb0 0x7f0dac06b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0dabfff700 1 -- 172.21.15.112:0/3167504705 >> v1:172.21.15.138:6789/0 conn(0x7f0dac151060 legacy=0x7f0dac15e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0dabfff700 1 -- 172.21.15.112:0/3167504705 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac0b05d0 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0daaffd700 1 --2- 172.21.15.112:0/3167504705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0dac06adb0 0x7f0dac06b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:20.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.497+0000 7f0dabfff700 1 --2- 172.21.15.112:0/3167504705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dac0cdbd0 0x7f0dac0ce060 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009e50 tx=0x7f0da400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc956810f796fa1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:20.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.498+0000 7f0daa7fc700 1 -- 172.21.15.112:0/3167504705 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da400e070 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.498+0000 7f0daa7fc700 1 -- 172.21.15.112:0/3167504705 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da400b9c0 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.498+0000 7f0daa7fc700 1 -- 172.21.15.112:0/3167504705 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da400bd70 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.498+0000 7f0db1c6e700 1 -- 172.21.15.112:0/3167504705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dac0cdbd0 msgr2=0x7f0dac0ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.498+0000 7f0db1c6e700 1 --2- 172.21.15.112:0/3167504705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dac0cdbd0 0x7f0dac0ce060 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009e50 tx=0x7f0da400b330 comp rx=0 tx=0).stop 2023-08-27T11:28:20.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.498+0000 7f0db1c6e700 1 -- 172.21.15.112:0/3167504705 shutdown_connections 2023-08-27T11:28:20.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.498+0000 7f0db1c6e700 1 --2- 172.21.15.112:0/3167504705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dac0cdbd0 0x7f0dac0ce060 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.498+0000 7f0db1c6e700 1 --2- 172.21.15.112:0/3167504705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0dac06adb0 0x7f0dac06b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.498+0000 7f0db1c6e700 1 -- 172.21.15.112:0/3167504705 >> 172.21.15.112:0/3167504705 conn(0x7f0dac0d0000 msgr2=0x7f0dac0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.498+0000 7f0db1c6e700 1 -- 172.21.15.112:0/3167504705 shutdown_connections 2023-08-27T11:28:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.498+0000 7f0db1c6e700 1 -- 172.21.15.112:0/3167504705 wait complete. 2023-08-27T11:28:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.499+0000 7f0db1c6e700 1 Processor -- start 2023-08-27T11:28:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.500+0000 7f0db1c6e700 1 -- start start 2023-08-27T11:28:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.500+0000 7f0db1c6e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac06adb0 0x7f0dac153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.500+0000 7f0db1c6e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dac0cdbd0 0x7f0dac14dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.500+0000 7f0db1c6e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0dac151060 0x7f0dac159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.500+0000 7f0db1c6e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac15b7f0 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.500+0000 7f0db1c6e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac153950 con 0x7f0dac06adb0 2023-08-27T11:28:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.500+0000 7f0db1c6e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac153c30 con 0x7f0dac151060 2023-08-27T11:28:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.500+0000 7f0daaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dac0cdbd0 0x7f0dac14dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.500+0000 7f0daaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dac0cdbd0 0x7f0dac14dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36162/0 (socket says 172.21.15.112:36162) 2023-08-27T11:28:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.500+0000 7f0daaffd700 1 -- 172.21.15.112:0/1076490800 learned_addr learned my addr 172.21.15.112:0/1076490800 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0dabfff700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0dac151060 0x7f0dac159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0dab7fe700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac06adb0 0x7f0dac153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0daaffd700 1 -- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0dac151060 msgr2=0x7f0dac159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0daaffd700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0dac151060 0x7f0dac159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0daaffd700 1 -- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac06adb0 msgr2=0x7f0dac153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0daaffd700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac06adb0 0x7f0dac153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0daaffd700 1 -- 172.21.15.112:0/1076490800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da4009b30 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0dab7fe700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac06adb0 0x7f0dac153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0daaffd700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dac0cdbd0 0x7f0dac14dfa0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00e810 tx=0x7f0d9c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0da3fff700 1 -- 172.21.15.112:0/1076490800 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00c020 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0da3fff700 1 -- 172.21.15.112:0/1076490800 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c004250 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0db1c6e700 1 -- 172.21.15.112:0/1076490800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dac153f10 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0da3fff700 1 -- 172.21.15.112:0/1076490800 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0036a0 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.501+0000 7f0db1c6e700 1 -- 172.21.15.112:0/1076490800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dac14ea60 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.505+0000 7f0dbde30b80 1 Processor -- start 2023-08-27T11:28:20.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.505+0000 7f0dbde30b80 1 -- start start 2023-08-27T11:28:20.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0da3fff700 1 -- 172.21.15.112:0/1076490800 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f0d9c003800 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0dbde30b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 0x559e1fd2e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0dbde30b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd3f960 0x559e1fd2ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0dbde30b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 0x559e1fd2b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0dbde30b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1fc5c730 con 0x559e1fd013d0 2023-08-27T11:28:20.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0dbde30b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1fc5a050 con 0x559e1fd3f960 2023-08-27T11:28:20.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0dbde30b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e1fc55940 con 0x559e1fd2df40 2023-08-27T11:28:20.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0daaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 0x559e1fd2b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0daaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 0x559e1fd2b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36172/0 (socket says 172.21.15.112:36172) 2023-08-27T11:28:20.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0daaffd700 1 -- 172.21.15.112:0/1410491571 learned_addr learned my addr 172.21.15.112:0/1410491571 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:20.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0dabfff700 1 --2- 172.21.15.112:0/1410491571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 0x559e1fd2e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0dab7fe700 1 --2- 172.21.15.112:0/1410491571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd3f960 0x559e1fd2ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0daaffd700 1 -- 172.21.15.112:0/1410491571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 msgr2=0x559e1fd2e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:20.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0daaffd700 1 --2- 172.21.15.112:0/1410491571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 0x559e1fd2e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0daaffd700 1 -- 172.21.15.112:0/1410491571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd3f960 msgr2=0x559e1fd2ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:20.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0da3fff700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0d8c06a4d0 0x7f0d8c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0daaffd700 1 --2- 172.21.15.112:0/1410491571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd3f960 0x559e1fd2ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0daaffd700 1 -- 172.21.15.112:0/1410491571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e1fad55e0 con 0x559e1fd013d0 2023-08-27T11:28:20.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0da3fff700 1 -- 172.21.15.112:0/1076490800 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d9c088740 con 0x7f0dac0cdbd0 2023-08-27T11:28:20.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0daaffd700 1 --2- 172.21.15.112:0/1410491571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 0x559e1fd2b0b0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c086530 tx=0x7f0d9c053840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf54b159c53fb72d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:20.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0dabfff700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0d8c06a4d0 0x7f0d8c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0da17fa700 1 -- 172.21.15.112:0/1410491571 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c053120 con 0x559e1fd013d0 2023-08-27T11:28:20.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0da17fa700 1 -- 172.21.15.112:0/1410491571 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c057040 con 0x559e1fd013d0 2023-08-27T11:28:20.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.506+0000 7f0da17fa700 1 -- 172.21.15.112:0/1410491571 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c056e60 con 0x559e1fd013d0 2023-08-27T11:28:20.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.507+0000 7f0dabfff700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0d8c06a4d0 0x7f0d8c06c990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0da400b370 tx=0x7f0da4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:20.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.507+0000 7f0dbde30b80 1 -- 172.21.15.112:0/1410491571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 msgr2=0x559e1fd2b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.507+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/1410491571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 0x559e1fd2b0b0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c086530 tx=0x7f0d9c053840 comp rx=0 tx=0).stop 2023-08-27T11:28:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.507+0000 7f0dbde30b80 1 -- 172.21.15.112:0/1410491571 shutdown_connections 2023-08-27T11:28:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.507+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/1410491571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 0x559e1fd2b0b0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.507+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/1410491571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 0x559e1fd2e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.507+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/1410491571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd3f960 0x559e1fd2ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.507+0000 7f0dbde30b80 1 -- 172.21.15.112:0/1410491571 >> 172.21.15.112:0/1410491571 conn(0x559e1fc61c30 msgr2=0x559e1fd48d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.507+0000 7f0dbde30b80 1 -- 172.21.15.112:0/1410491571 shutdown_connections 2023-08-27T11:28:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.507+0000 7f0dbde30b80 1 -- 172.21.15.112:0/1410491571 wait complete. 2023-08-27T11:28:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.508+0000 7f0dbde30b80 1 -- start start 2023-08-27T11:28:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.508+0000 7f0dbde30b80 1 Processor -- start 2023-08-27T11:28:20.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0dbde30b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd013d0 0x559e1fd5bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0dbde30b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd2df40 0x559e1fd48b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0dbde30b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd3f960 0x559e1fd490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0dbde30b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1fbf3b60 con 0x559e1fd2df40 2023-08-27T11:28:20.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0dbde30b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1fbf3860 con 0x559e1fd013d0 2023-08-27T11:28:20.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0dbde30b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e1fad78d0 con 0x559e1fd3f960 2023-08-27T11:28:20.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0dbde30b80 10 client.0 _ll_register_callbacks cb 0x559e1fbf1630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:28:20.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0daaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd2df40 0x559e1fd48b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0daaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd2df40 0x559e1fd48b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36178/0 (socket says 172.21.15.112:36178) 2023-08-27T11:28:20.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0daaffd700 1 -- 172.21.15.112:0/864964257 learned_addr learned my addr 172.21.15.112:0/864964257 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:20.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0dab7fe700 1 --2- 172.21.15.112:0/864964257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd013d0 0x559e1fd5bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0dabfff700 1 --2- 172.21.15.112:0/864964257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd3f960 0x559e1fd490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0daaffd700 1 -- 172.21.15.112:0/864964257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd3f960 msgr2=0x559e1fd490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:20.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0daaffd700 1 --2- 172.21.15.112:0/864964257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd3f960 0x559e1fd490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0daaffd700 1 -- 172.21.15.112:0/864964257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd013d0 msgr2=0x559e1fd5bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:20.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0daaffd700 1 --2- 172.21.15.112:0/864964257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd013d0 0x559e1fd5bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0daaffd700 1 -- 172.21.15.112:0/864964257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dac14ea60 con 0x559e1fd2df40 2023-08-27T11:28:20.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0daaffd700 1 --2- 172.21.15.112:0/864964257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd2df40 0x559e1fd48b70 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c057490 tx=0x7f0d9c08acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d51956fd9440e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:20.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0d8a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:28:20.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0d8a7fc700 1 -- 172.21.15.112:0/864964257 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c053500 con 0x559e1fd2df40 2023-08-27T11:28:20.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0d8a7fc700 1 -- 172.21.15.112:0/864964257 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d9c057b90 con 0x559e1fd2df40 2023-08-27T11:28:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0dbde30b80 1 -- 172.21.15.112:0/864964257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e1fcf64f0 con 0x559e1fd2df40 2023-08-27T11:28:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0d8a7fc700 1 -- 172.21.15.112:0/864964257 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d9c053120 con 0x559e1fd2df40 2023-08-27T11:28:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0dbde30b80 10 client.8616 fetch_fsmap learned FSMap version 94 2023-08-27T11:28:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.509+0000 7f0dbde30b80 1 -- 172.21.15.112:0/864964257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x559e1fc6d750 con 0x559e1fd2df40 2023-08-27T11:28:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.510+0000 7f0d8a7fc700 1 -- 172.21.15.112:0/864964257 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559e1fcf64f0 con 0x559e1fd2df40 2023-08-27T11:28:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.510+0000 7f0d8a7fc700 1 -- 172.21.15.112:0/864964257 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 94) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0d9c0112f0 con 0x559e1fd2df40 2023-08-27T11:28:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.510+0000 7f0dbde30b80 10 client.8616 fetch_fsmap finished waiting for FSMap version 94 2023-08-27T11:28:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.510+0000 7f0dbde30b80 10 client.8616 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-27T11:28:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.510+0000 7f0dbde30b80 20 client.8616 populate_metadata read hostname 'smithi112' 2023-08-27T11:28:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.510+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/864964257 >> [v2:172.21.15.138:6834/529571603,v1:172.21.15.138:6836/529571603] conn(0x559e1fd4b710 0x559e1fd4baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.510+0000 7f0dbde30b80 4 client.8616 mds_command: new command op to 8472 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:28:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.510+0000 7f0dbde30b80 1 -- 172.21.15.112:0/864964257 --> [v2:172.21.15.138:6834/529571603,v1:172.21.15.138:6836/529571603] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e1fc4e920 con 0x559e1fd4b710 2023-08-27T11:28:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.510+0000 7f0dab7fe700 1 --2- 172.21.15.112:0/864964257 >> [v2:172.21.15.138:6834/529571603,v1:172.21.15.138:6836/529571603] conn(0x559e1fd4b710 0x559e1fd4baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.510+0000 7f0d8a7fc700 10 client.8616 ms_handle_connect on v2:172.21.15.138:6834/529571603 2023-08-27T11:28:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.510+0000 7f0dab7fe700 1 --2- 172.21.15.112:0/864964257 >> [v2:172.21.15.138:6834/529571603,v1:172.21.15.138:6836/529571603] conn(0x559e1fd4b710 0x559e1fd4baf0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc6bf1ce33f46640 server_cookie=4947c3e9e800b168 in_seq=0 out_seq=0 2023-08-27T11:28:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.511+0000 7f0d8a7fc700 1 -- 172.21.15.112:0/864964257 <== mds.0 v2:172.21.15.138:6834/529571603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559e1fc4e920 con 0x559e1fd4b710 2023-08-27T11:28:20.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.511+0000 7f0d8a7fc700 10 client.8616 handle_command_reply: tid=0 2023-08-27T11:28:20.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.511+0000 7f0dbde30b80 1 client.8616 shutdown 2023-08-27T11:28:20.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.511+0000 7f0dbde30b80 1 -- 172.21.15.112:0/864964257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd2df40 msgr2=0x559e1fd48b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.511+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/864964257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd2df40 0x559e1fd48b70 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c057490 tx=0x7f0d9c08acd0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.511+0000 7f0dbde30b80 1 -- 172.21.15.112:0/864964257 shutdown_connections 2023-08-27T11:28:20.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.511+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/864964257 >> [v2:172.21.15.138:6834/529571603,v1:172.21.15.138:6836/529571603] conn(0x559e1fd4b710 0x559e1fd4baf0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.511+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/864964257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd3f960 0x559e1fd490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.511+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/864964257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd013d0 0x559e1fd5bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.511+0000 7f0d8a7fc700 0 client.8616 ms_handle_reset on v2:172.21.15.138:6834/529571603 2023-08-27T11:28:20.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.511+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/864964257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd2df40 0x559e1fd48b70 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.511+0000 7f0dbde30b80 1 -- 172.21.15.112:0/864964257 >> 172.21.15.112:0/864964257 conn(0x559e1fc61c30 msgr2=0x559e1fd48200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:20.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.512+0000 7f0dbde30b80 1 -- 172.21.15.112:0/864964257 shutdown_connections 2023-08-27T11:28:20.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.512+0000 7f0dbde30b80 1 -- 172.21.15.112:0/864964257 wait complete. 2023-08-27T11:28:20.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.512+0000 7f0dbde30b80 20 client.8616 trim_cache size 0 max 16384 2023-08-27T11:28:20.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.529+0000 7f0dbde30b80 1 Processor -- start 2023-08-27T11:28:20.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.529+0000 7f0dbde30b80 1 -- start start 2023-08-27T11:28:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.530+0000 7f0dbde30b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd013d0 0x559e1fd48d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.530+0000 7f0dbde30b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 0x559e1fd5b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.530+0000 7f0dbde30b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd3f960 0x559e1fd5bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.530+0000 7f0dbde30b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1fd01e80 con 0x559e1fd3f960 2023-08-27T11:28:20.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.530+0000 7f0dbde30b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1fd29b50 con 0x559e1fd013d0 2023-08-27T11:28:20.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.530+0000 7f0dbde30b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e1fd021e0 con 0x559e1fd2df40 2023-08-27T11:28:20.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.530+0000 7f0dabfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd3f960 0x559e1fd5bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.530+0000 7f0dabfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd3f960 0x559e1fd5bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36188/0 (socket says 172.21.15.112:36188) 2023-08-27T11:28:20.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.530+0000 7f0dabfff700 1 -- 172.21.15.112:0/3262452906 learned_addr learned my addr 172.21.15.112:0/3262452906 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.531+0000 7f0dabfff700 1 -- 172.21.15.112:0/3262452906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 msgr2=0x559e1fd5b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.531+0000 7f0dabfff700 1 --2- 172.21.15.112:0/3262452906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 0x559e1fd5b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.531+0000 7f0dabfff700 1 -- 172.21.15.112:0/3262452906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd013d0 msgr2=0x559e1fd48d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.531+0000 7f0dabfff700 1 --2- 172.21.15.112:0/3262452906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd013d0 0x559e1fd48d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.531+0000 7f0dabfff700 1 -- 172.21.15.112:0/3262452906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e1fa81d90 con 0x559e1fd3f960 2023-08-27T11:28:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.531+0000 7f0dabfff700 1 --2- 172.21.15.112:0/3262452906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd3f960 0x559e1fd5bce0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f0da4012000 tx=0x7f0da4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f766dab5283f03 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.531+0000 7f0da1ffb700 1 -- 172.21.15.112:0/3262452906 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da402e070 con 0x559e1fd3f960 2023-08-27T11:28:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.531+0000 7f0da1ffb700 1 -- 172.21.15.112:0/3262452906 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da4003e00 con 0x559e1fd3f960 2023-08-27T11:28:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.531+0000 7f0da1ffb700 1 -- 172.21.15.112:0/3262452906 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da4004620 con 0x559e1fd3f960 2023-08-27T11:28:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.531+0000 7f0dbde30b80 1 -- 172.21.15.112:0/3262452906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd3f960 msgr2=0x559e1fd5bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.532+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/3262452906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd3f960 0x559e1fd5bce0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f0da4012000 tx=0x7f0da4010000 comp rx=0 tx=0).stop 2023-08-27T11:28:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.532+0000 7f0dbde30b80 1 -- 172.21.15.112:0/3262452906 shutdown_connections 2023-08-27T11:28:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.532+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/3262452906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd3f960 0x559e1fd5bce0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.532+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/3262452906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd013d0 0x559e1fd48d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.532+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/3262452906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 0x559e1fd5b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.532+0000 7f0dbde30b80 1 -- 172.21.15.112:0/3262452906 >> 172.21.15.112:0/3262452906 conn(0x559e1fc61c30 msgr2=0x559e1fd48200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.532+0000 7f0dbde30b80 1 -- 172.21.15.112:0/3262452906 shutdown_connections 2023-08-27T11:28:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.532+0000 7f0dbde30b80 1 -- 172.21.15.112:0/3262452906 wait complete. 2023-08-27T11:28:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.532+0000 7f0dbde30b80 1 -- start start 2023-08-27T11:28:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.532+0000 7f0dbde30b80 1 Processor -- start 2023-08-27T11:28:20.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dbde30b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 0x559e1fd5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dbde30b80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 0x559e1fd57470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dbde30b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd3f960 0x559e1fd579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dbde30b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1fdb0730 con 0x559e1fd013d0 2023-08-27T11:28:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dbde30b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1fcfd430 con 0x559e1fd3f960 2023-08-27T11:28:20.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dbde30b80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e1fcfd960 con 0x559e1fd2df40 2023-08-27T11:28:20.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dbde30b80 10 client.0 _ll_register_callbacks cb 0x559e1fbf1630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:28:20.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dab7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 0x559e1fd5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dab7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 0x559e1fd5ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36190/0 (socket says 172.21.15.112:36190) 2023-08-27T11:28:20.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dab7fe700 1 -- 172.21.15.112:0/1987313233 learned_addr learned my addr 172.21.15.112:0/1987313233 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:20.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0daaffd700 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 0x559e1fd57470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dab7fe700 1 -- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 msgr2=0x559e1fd57470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:20.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dabfff700 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd3f960 0x559e1fd579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dab7fe700 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 0x559e1fd57470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dab7fe700 1 -- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd3f960 msgr2=0x559e1fd579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dab7fe700 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd3f960 0x559e1fd579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dab7fe700 1 -- 172.21.15.112:0/1987313233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dac0b05d0 con 0x559e1fd013d0 2023-08-27T11:28:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0daaffd700 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 0x559e1fd57470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.533+0000 7f0dabfff700 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd3f960 0x559e1fd579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0dab7fe700 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 0x559e1fd5ab00 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0d08d0 tx=0x7f0d94001d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749a97a3504a3f9d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0d8a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:28:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0d8a7fc700 1 -- 172.21.15.112:0/1987313233 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94005310 con 0x559e1fd013d0 2023-08-27T11:28:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0d8a7fc700 1 -- 172.21.15.112:0/1987313233 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d94005470 con 0x559e1fd013d0 2023-08-27T11:28:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0dbde30b80 1 -- 172.21.15.112:0/1987313233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d9c057040 con 0x559e1fd013d0 2023-08-27T11:28:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0d8a7fc700 1 -- 172.21.15.112:0/1987313233 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d94007040 con 0x559e1fd013d0 2023-08-27T11:28:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0dbde30b80 10 client.8622 fetch_fsmap learned FSMap version 94 2023-08-27T11:28:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0dbde30b80 1 -- 172.21.15.112:0/1987313233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x559e1fa6ac10 con 0x559e1fd013d0 2023-08-27T11:28:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0d8a7fc700 1 -- 172.21.15.112:0/1987313233 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c057040 con 0x559e1fd013d0 2023-08-27T11:28:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0d8a7fc700 1 -- 172.21.15.112:0/1987313233 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 94) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0d9401e070 con 0x559e1fd013d0 2023-08-27T11:28:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0dbde30b80 10 client.8622 fetch_fsmap finished waiting for FSMap version 94 2023-08-27T11:28:20.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0dbde30b80 10 client.8622 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-27T11:28:20.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0dbde30b80 20 client.8622 populate_metadata read hostname 'smithi112' 2023-08-27T11:28:20.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:6834/529571603,v1:172.21.15.138:6836/529571603] conn(0x559e1fd4b710 0x559e1fdaf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0dbde30b80 4 client.8622 mds_command: new command op to 8472 tid=0[{"prefix": "status"}] 2023-08-27T11:28:20.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0dbde30b80 1 -- 172.21.15.112:0/1987313233 --> [v2:172.21.15.138:6834/529571603,v1:172.21.15.138:6836/529571603] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559e1fc0f000 con 0x559e1fd4b710 2023-08-27T11:28:20.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.534+0000 7f0daaffd700 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:6834/529571603,v1:172.21.15.138:6836/529571603] conn(0x559e1fd4b710 0x559e1fdaf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0d8a7fc700 10 client.8622 ms_handle_connect on v2:172.21.15.138:6834/529571603 2023-08-27T11:28:20.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0daaffd700 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:6834/529571603,v1:172.21.15.138:6836/529571603] conn(0x559e1fd4b710 0x559e1fdaf310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63beab5d66cfce70 server_cookie=5a8a9d891ab049b8 in_seq=0 out_seq=0 2023-08-27T11:28:20.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0d8a7fc700 1 -- 172.21.15.112:0/1987313233 <== mds.0 v2:172.21.15.138:6834/529571603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x559e1fc0f000 con 0x559e1fd4b710 2023-08-27T11:28:20.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0d8a7fc700 10 client.8622 handle_command_reply: tid=0 2023-08-27T11:28:20.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0dbde30b80 1 client.8622 shutdown 2023-08-27T11:28:20.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0dbde30b80 1 -- 172.21.15.112:0/1987313233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 msgr2=0x559e1fd5ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 0x559e1fd5ab00 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0d08d0 tx=0x7f0d94001d50 comp rx=0 tx=0).stop 2023-08-27T11:28:20.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0dbde30b80 1 -- 172.21.15.112:0/1987313233 shutdown_connections 2023-08-27T11:28:20.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e1fd3f960 0x559e1fd579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e1fd013d0 0x559e1fd5ab00 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:6834/529571603,v1:172.21.15.138:6836/529571603] conn(0x559e1fd4b710 0x559e1fdaf310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0dbde30b80 1 --2- 172.21.15.112:0/1987313233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559e1fd2df40 0x559e1fd57470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0d8a7fc700 0 client.8622 ms_handle_reset on v2:172.21.15.138:6834/529571603 2023-08-27T11:28:20.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0dbde30b80 1 -- 172.21.15.112:0/1987313233 >> 172.21.15.112:0/1987313233 conn(0x559e1fc61c30 msgr2=0x559e1fd04a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:20.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0dbde30b80 1 -- 172.21.15.112:0/1987313233 shutdown_connections 2023-08-27T11:28:20.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0dbde30b80 1 -- 172.21.15.112:0/1987313233 wait complete. 2023-08-27T11:28:20.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.535+0000 7f0dbde30b80 20 client.8622 trim_cache size 0 max 16384 2023-08-27T11:28:20.566 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:28:20.566 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e099a24f-c1cd-4f11-ac4f-ecacc10e5e98", 2023-08-27T11:28:20.567 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2023-08-27T11:28:20.567 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 8472, 2023-08-27T11:28:20.567 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2023-08-27T11:28:20.567 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2023-08-27T11:28:20.567 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2023-08-27T11:28:20.567 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 5.9037879020000004, 2023-08-27T11:28:20.568 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 94, 2023-08-27T11:28:20.568 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 139, 2023-08-27T11:28:20.568 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 139, 2023-08-27T11:28:20.568 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 11.792531740999999 2023-08-27T11:28:20.568 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:28:20.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.536+0000 7f0db1c6e700 1 -- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0d8c06a4d0 msgr2=0x7f0d8c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.536+0000 7f0db1c6e700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0d8c06a4d0 0x7f0d8c06c990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0da400b370 tx=0x7f0da4009000 comp rx=0 tx=0).stop 2023-08-27T11:28:20.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.536+0000 7f0db1c6e700 1 -- 172.21.15.112:0/1076490800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dac0cdbd0 msgr2=0x7f0dac14dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.536+0000 7f0db1c6e700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dac0cdbd0 0x7f0dac14dfa0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00e810 tx=0x7f0d9c00ebd0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.536+0000 7f0db1c6e700 1 -- 172.21.15.112:0/1076490800 shutdown_connections 2023-08-27T11:28:20.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.536+0000 7f0db1c6e700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0d8c06a4d0 0x7f0d8c06c990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.536+0000 7f0db1c6e700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0dac151060 0x7f0dac159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.536+0000 7f0db1c6e700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac06adb0 0x7f0dac153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.536+0000 7f0db1c6e700 1 --2- 172.21.15.112:0/1076490800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dac0cdbd0 0x7f0dac14dfa0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.536+0000 7f0db1c6e700 1 -- 172.21.15.112:0/1076490800 >> 172.21.15.112:0/1076490800 conn(0x7f0dac0d0000 msgr2=0x7f0dac15de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:20.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.536+0000 7f0db1c6e700 1 -- 172.21.15.112:0/1076490800 shutdown_connections 2023-08-27T11:28:20.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.536+0000 7f0db1c6e700 1 -- 172.21.15.112:0/1076490800 wait complete. 2023-08-27T11:28:20.573 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:28:20.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.692+0000 7fc1827bd700 1 Processor -- start 2023-08-27T11:28:20.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.692+0000 7fc1827bd700 1 -- start start 2023-08-27T11:28:20.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc1827bd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c141420 0x7fc17c0aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc1827bd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc17c06e150 0x7fc17c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc1827bd700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17c0ab440 con 0x7fc17c0cc070 2023-08-27T11:28:20.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc1827bd700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17c06eb10 con 0x7fc17c06e150 2023-08-27T11:28:20.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc1827bd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c15eed0 con 0x7fc17c141420 2023-08-27T11:28:20.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc17b7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc17c06e150 0x7fc17c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc180d5a700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc17c0cc070 0x7fc17c0cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58816/0 (socket says 172.21.15.112:58816) 2023-08-27T11:28:20.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc17b7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc17c06e150 0x7fc17c06e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36198/0 (socket says 172.21.15.112:36198) 2023-08-27T11:28:20.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc180d5a700 1 -- 172.21.15.112:0/172198231 learned_addr learned my addr 172.21.15.112:0/172198231 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:20.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc17bfff700 1 --2- 172.21.15.112:0/172198231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c141420 0x7fc17c0aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc17b7fe700 1 -- 172.21.15.112:0/172198231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c141420 msgr2=0x7fc17c0aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:20.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc17b7fe700 1 --2- 172.21.15.112:0/172198231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c141420 0x7fc17c0aaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc17b7fe700 1 -- 172.21.15.112:0/172198231 >> v1:172.21.15.112:6789/0 conn(0x7fc17c0cc070 legacy=0x7fc17c0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc17b7fe700 1 -- 172.21.15.112:0/172198231 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c15f1b0 con 0x7fc17c06e150 2023-08-27T11:28:20.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.693+0000 7fc17affd700 1 -- 172.21.15.112:0/172198231 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123879605 0 0) 0x7fc17c0ab440 con 0x7fc17c0cc070 2023-08-27T11:28:20.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.694+0000 7fc17b7fe700 1 --2- 172.21.15.112:0/172198231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c06e150 0x7fc17c06e5d0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009820 tx=0x7fc16c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6d1e2181f7313a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:20.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.694+0000 7fc17affd700 1 -- 172.21.15.112:0/172198231 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c005e30 con 0x7fc17c06e150 2023-08-27T11:28:20.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.694+0000 7fc17affd700 1 -- 172.21.15.112:0/172198231 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c006030 con 0x7fc17c06e150 2023-08-27T11:28:20.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.694+0000 7fc17affd700 1 -- 172.21.15.112:0/172198231 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c0063e0 con 0x7fc17c06e150 2023-08-27T11:28:20.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.694+0000 7fc1827bd700 1 -- 172.21.15.112:0/172198231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c06e150 msgr2=0x7fc17c06e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.694+0000 7fc1827bd700 1 --2- 172.21.15.112:0/172198231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c06e150 0x7fc17c06e5d0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009820 tx=0x7fc16c009b30 comp rx=0 tx=0).stop 2023-08-27T11:28:20.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.694+0000 7fc1827bd700 1 -- 172.21.15.112:0/172198231 shutdown_connections 2023-08-27T11:28:20.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.694+0000 7fc1827bd700 1 --2- 172.21.15.112:0/172198231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c141420 0x7fc17c0aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.694+0000 7fc1827bd700 1 --2- 172.21.15.112:0/172198231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c06e150 0x7fc17c06e5d0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.694+0000 7fc1827bd700 1 -- 172.21.15.112:0/172198231 >> 172.21.15.112:0/172198231 conn(0x7fc17c06cd30 msgr2=0x7fc17c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:20.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.694+0000 7fc1827bd700 1 -- 172.21.15.112:0/172198231 shutdown_connections 2023-08-27T11:28:20.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.694+0000 7fc1827bd700 1 -- 172.21.15.112:0/172198231 wait complete. 2023-08-27T11:28:20.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.695+0000 7fc1827bd700 1 Processor -- start 2023-08-27T11:28:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.696+0000 7fc1827bd700 1 -- start start 2023-08-27T11:28:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.696+0000 7fc1827bd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c06e150 0x7fc17c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.696+0000 7fc1827bd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c0cc070 0x7fc17c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.696+0000 7fc1827bd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c141420 0x7fc17c06d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.696+0000 7fc1827bd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0add80 con 0x7fc17c0cc070 2023-08-27T11:28:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.696+0000 7fc1827bd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c15b000 con 0x7fc17c141420 2023-08-27T11:28:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.696+0000 7fc1827bd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c1426c0 con 0x7fc17c06e150 2023-08-27T11:28:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.696+0000 7fc17b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c0cc070 0x7fc17c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.696+0000 7fc17b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c0cc070 0x7fc17c06d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36206/0 (socket says 172.21.15.112:36206) 2023-08-27T11:28:20.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.696+0000 7fc17b7fe700 1 -- 172.21.15.112:0/81911799 learned_addr learned my addr 172.21.15.112:0/81911799 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:20.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc17bfff700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c06e150 0x7fc17c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc17b7fe700 1 -- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c06e150 msgr2=0x7fc17c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc180d5a700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c141420 0x7fc17c06d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc17b7fe700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c06e150 0x7fc17c15aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc17b7fe700 1 -- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c141420 msgr2=0x7fc17c06d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc17b7fe700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c141420 0x7fc17c06d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc17b7fe700 1 -- 172.21.15.112:0/81911799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c009430 con 0x7fc17c0cc070 2023-08-27T11:28:20.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc17bfff700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c06e150 0x7fc17c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:20.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc17b7fe700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c0cc070 0x7fc17c06d110 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc17c06ed70 tx=0x7fc16c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:20.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc178ff9700 1 -- 172.21.15.112:0/81911799 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c025630 con 0x7fc17c0cc070 2023-08-27T11:28:20.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc178ff9700 1 -- 172.21.15.112:0/81911799 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c020070 con 0x7fc17c0cc070 2023-08-27T11:28:20.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc1827bd700 1 -- 172.21.15.112:0/81911799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17c1429a0 con 0x7fc17c0cc070 2023-08-27T11:28:20.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc178ff9700 1 -- 172.21.15.112:0/81911799 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c0259d0 con 0x7fc17c0cc070 2023-08-27T11:28:20.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.697+0000 7fc1827bd700 1 -- 172.21.15.112:0/81911799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17c142f60 con 0x7fc17c0cc070 2023-08-27T11:28:20.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.698+0000 7fc1827bd700 1 -- 172.21.15.112:0/81911799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17c0adef0 con 0x7fc17c0cc070 2023-08-27T11:28:20.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.699+0000 7fc178ff9700 1 -- 172.21.15.112:0/81911799 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fc16c022030 con 0x7fc17c0cc070 2023-08-27T11:28:20.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.699+0000 7fc178ff9700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16806a4d0 0x7fc16806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.699+0000 7fc178ff9700 1 -- 172.21.15.112:0/81911799 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc16c08a600 con 0x7fc17c0cc070 2023-08-27T11:28:20.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.699+0000 7fc17bfff700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16806a4d0 0x7fc16806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.700+0000 7fc17bfff700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16806a4d0 0x7fc16806c990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc17c141920 tx=0x7fc170008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:20.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.703+0000 7fc178ff9700 1 -- 172.21.15.112:0/81911799 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc16c0558f0 con 0x7fc17c0cc070 2023-08-27T11:28:20.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.937+0000 7fc1827bd700 1 -- 172.21.15.112:0/81911799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc17c067a20 con 0x7fc17c0cc070 2023-08-27T11:28:20.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.937+0000 7fc178ff9700 1 -- 172.21.15.112:0/81911799 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc16c00b3e0 con 0x7fc17c0cc070 2023-08-27T11:28:20.939 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:28:20.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.939+0000 7fc1827bd700 1 -- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16806a4d0 msgr2=0x7fc16806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.939+0000 7fc1827bd700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16806a4d0 0x7fc16806c990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc17c141920 tx=0x7fc170008040 comp rx=0 tx=0).stop 2023-08-27T11:28:20.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.939+0000 7fc1827bd700 1 -- 172.21.15.112:0/81911799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c0cc070 msgr2=0x7fc17c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.939+0000 7fc1827bd700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c0cc070 0x7fc17c06d110 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc17c06ed70 tx=0x7fc16c004750 comp rx=0 tx=0).stop 2023-08-27T11:28:20.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.939+0000 7fc1827bd700 1 -- 172.21.15.112:0/81911799 shutdown_connections 2023-08-27T11:28:20.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.939+0000 7fc1827bd700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16806a4d0 0x7fc16806c990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.939+0000 7fc1827bd700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c141420 0x7fc17c06d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.939+0000 7fc1827bd700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c06e150 0x7fc17c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.939+0000 7fc1827bd700 1 --2- 172.21.15.112:0/81911799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17c0cc070 0x7fc17c06d110 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.939+0000 7fc1827bd700 1 -- 172.21.15.112:0/81911799 >> 172.21.15.112:0/81911799 conn(0x7fc17c06cd30 msgr2=0x7fc17c15e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:20.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.939+0000 7fc1827bd700 1 -- 172.21.15.112:0/81911799 shutdown_connections 2023-08-27T11:28:20.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:20.939+0000 7fc1827bd700 1 -- 172.21.15.112:0/81911799 wait complete. 2023-08-27T11:28:20.953 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:28:20.953 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:28:20.953 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-08-27T11:28:20.954 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:28:20.954 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:20.954 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:28:20.955 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:28:20.955 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:28:20.955 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:28:20.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:20.956 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-08-27T11:28:20.974 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:28:20.974 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:28:20.974 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:28:20.975 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:20.975 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:28:20.975 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:20.975 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:20.975 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:20.975 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:20.976 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:20.976 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:20.976 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:20.976 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:28:20.976 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:20.977 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:28:20.977 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 41742sec preferred_lft 41742sec 2023-08-27T11:28:20.977 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8d6c:4fe6:6a84:8222/64 scope link noprefixroute 2023-08-27T11:28:20.977 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:20.977 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:28:20.978 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:20.978 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:28:20.978 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:20.978 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::d87c:a1ff:fe1d:b635/64 scope link 2023-08-27T11:28:20.978 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:20.979 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:28:20.979 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:28:20.979 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::cc14:79ff:feb3:452f/64 scope link 2023-08-27T11:28:20.979 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:20.979 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:28:20.980 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether e6:51:f0:78:04:02 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:28:20.980 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::e451:f0ff:fe78:402/64 scope link 2023-08-27T11:28:20.980 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:20.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:20.981 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:28:21.039 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-08-27T11:28:21.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:21.041 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-08-27T11:28:21.099 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-08-27T11:28:21.100 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:28:21.100 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:28:21.100 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:28:21.101 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:28:21.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:21.102 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:28:21.103 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:28:21.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:28:21.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:28:21.104 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:28:21.104 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:28:21.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:21.204 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:28:21.220 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:28:21.221 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:28:21.221 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:28:21.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:21.222 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-08-27T11:28:21.287 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:28:21.288 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:21.344 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:28:21.344 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:28:21.345 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:21.400 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:28:21.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:21.401 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:28:21.485 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:28:21.485 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:28:21.485 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:28:21.494 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:28:21.494 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:28:21.494 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:28:21.494 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:28:21.495 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:28:21.561 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:28:21.561 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:28:21.561 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:28:21.561 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:28:21.562 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:28:21.562 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:28:21.562 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:28:21.562 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:28:21.563 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:28:21.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:21.564 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:21.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:21.592 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:28:21.683 INFO:teuthology.orchestra.run.smithi112.stdout:client6809 2023-08-27T11:28:21.684 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:28:21.684 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:21.701 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:28:21.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.860+0000 7fcf42f8f700 1 Processor -- start 2023-08-27T11:28:21.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.860+0000 7fcf42f8f700 1 -- start start 2023-08-27T11:28:21.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.860+0000 7fcf42f8f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf3c0b1400 0x7fcf3c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.860+0000 7fcf42f8f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3c161850 con 0x7fcf3c0b17e0 2023-08-27T11:28:21.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.860+0000 7fcf42f8f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3c0cfc10 con 0x7fcf3c161ce0 2023-08-27T11:28:21.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.860+0000 7fcf42f8f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c161200 con 0x7fcf3c0b1400 2023-08-27T11:28:21.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3bfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fcf3c0b17e0 0x7fcf3c06e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58832/0 (socket says 172.21.15.112:58832) 2023-08-27T11:28:21.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3bfff700 1 -- 172.21.15.112:0/3994900567 learned_addr learned my addr 172.21.15.112:0/3994900567 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:21.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf40d2b700 1 --2- 172.21.15.112:0/3994900567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf3c0b1400 0x7fcf3c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152689288 0 0) 0x7fcf3c161850 con 0x7fcf3c0b17e0 2023-08-27T11:28:21.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf2c0035b0 con 0x7fcf3c0b17e0 2023-08-27T11:28:21.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190103936 0 0) 0x7fcf3c0cfc10 con 0x7fcf3c161ce0 2023-08-27T11:28:21.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf3c161850 con 0x7fcf3c161ce0 2023-08-27T11:28:21.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 786642405 0 0) 0x7fcf2c0035b0 con 0x7fcf3c0b17e0 2023-08-27T11:28:21.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf3c0b1400 msgr2=0x7fcf3c0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:21.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 --2- 172.21.15.112:0/3994900567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf3c0b1400 0x7fcf3c0cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 >> v1:172.21.15.138:6789/0 conn(0x7fcf3c161ce0 legacy=0x7fcf3c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c1614e0 con 0x7fcf3c0b17e0 2023-08-27T11:28:21.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fcf28001670 con 0x7fcf3c0b17e0 2023-08-27T11:28:21.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 >> v1:172.21.15.112:6789/0 conn(0x7fcf3c0b17e0 legacy=0x7fcf3c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 --2- 172.21.15.112:0/3994900567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf2c003bc0 0x7fcf2c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 --2- 172.21.15.112:0/3994900567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf3c0b1400 0x7fcf2c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 --2- 172.21.15.112:0/3994900567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf2c004d90 0x7fcf2c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c161200 con 0x7fcf2c003bc0 2023-08-27T11:28:21.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c00bba0 con 0x7fcf2c004d90 2023-08-27T11:28:21.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c00bd30 con 0x7fcf3c0b1400 2023-08-27T11:28:21.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf40d2b700 1 --2- 172.21.15.112:0/3994900567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf2c003bc0 0x7fcf2c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf3bfff700 1 --2- 172.21.15.112:0/3994900567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf3c0b1400 0x7fcf2c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.861+0000 7fcf4152c700 1 --2- 172.21.15.112:0/3994900567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf2c004d90 0x7fcf2c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf40d2b700 1 -- 172.21.15.112:0/3994900567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf3c0b1400 msgr2=0x7fcf2c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:21.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf40d2b700 1 --2- 172.21.15.112:0/3994900567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf3c0b1400 0x7fcf2c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf40d2b700 1 -- 172.21.15.112:0/3994900567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf2c004d90 msgr2=0x7fcf2c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:21.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf40d2b700 1 --2- 172.21.15.112:0/3994900567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf2c004d90 0x7fcf2c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf40d2b700 1 -- 172.21.15.112:0/3994900567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c0035b0 con 0x7fcf2c003bc0 2023-08-27T11:28:21.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf3bfff700 1 --2- 172.21.15.112:0/3994900567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf3c0b1400 0x7fcf2c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:21.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf40d2b700 1 --2- 172.21.15.112:0/3994900567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf2c003bc0 0x7fcf2c003fc0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009b90 tx=0x7fcf2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae56b1bec221066 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:21.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000e6c0 con 0x7fcf2c003bc0 2023-08-27T11:28:21.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3000e820 con 0x7fcf2c003bc0 2023-08-27T11:28:21.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf3b7fe700 1 -- 172.21.15.112:0/3994900567 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000eb90 con 0x7fcf2c003bc0 2023-08-27T11:28:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3994900567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf2c003bc0 msgr2=0x7fcf2c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf42f8f700 1 --2- 172.21.15.112:0/3994900567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf2c003bc0 0x7fcf2c003fc0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009b90 tx=0x7fcf2c005880 comp rx=0 tx=0).stop 2023-08-27T11:28:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf4152c700 1 -- 172.21.15.112:0/3994900567 reap_dead start 2023-08-27T11:28:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3994900567 shutdown_connections 2023-08-27T11:28:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.862+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3994900567 >> 172.21.15.112:0/3994900567 conn(0x7fcf3c0d0000 msgr2=0x7fcf3c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:21.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.863+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3994900567 shutdown_connections 2023-08-27T11:28:21.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.863+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3994900567 wait complete. 2023-08-27T11:28:21.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.864+0000 7fcf42f8f700 1 Processor -- start 2023-08-27T11:28:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.864+0000 7fcf42f8f700 1 -- start start 2023-08-27T11:28:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf42f8f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf2c003bc0 0x7fcf3c0c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf42f8f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf3c0c9210 0x7fcf3c0c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf42f8f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf3c0c9690 0x7fcf3c0c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf42f8f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c154850 con 0x7fcf2c003bc0 2023-08-27T11:28:21.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf42f8f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c0c4680 con 0x7fcf3c0c9210 2023-08-27T11:28:21.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf42f8f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c0c4960 con 0x7fcf3c0c9690 2023-08-27T11:28:21.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf40d2b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf2c003bc0 0x7fcf3c0c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf40d2b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf2c003bc0 0x7fcf3c0c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36228/0 (socket says 172.21.15.112:36228) 2023-08-27T11:28:21.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf40d2b700 1 -- 172.21.15.112:0/3620291260 learned_addr learned my addr 172.21.15.112:0/3620291260 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:21.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf40d2b700 1 -- 172.21.15.112:0/3620291260 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf3c0c9690 msgr2=0x7fcf3c0c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf40d2b700 1 --2- 172.21.15.112:0/3620291260 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf3c0c9690 0x7fcf3c0c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf40d2b700 1 -- 172.21.15.112:0/3620291260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf3c0c9210 msgr2=0x7fcf3c0c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf3bfff700 1 --2- 172.21.15.112:0/3620291260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf3c0c9210 0x7fcf3c0c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf40d2b700 1 --2- 172.21.15.112:0/3620291260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf3c0c9210 0x7fcf3c0c3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf40d2b700 1 -- 172.21.15.112:0/3620291260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c0c4c40 con 0x7fcf2c003bc0 2023-08-27T11:28:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf40d2b700 1 --2- 172.21.15.112:0/3620291260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf2c003bc0 0x7fcf3c0c5700 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fcf300090b0 tx=0x7fcf300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf397fa700 1 -- 172.21.15.112:0/3620291260 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000e6c0 con 0x7fcf2c003bc0 2023-08-27T11:28:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf397fa700 1 -- 172.21.15.112:0/3620291260 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf300038b0 con 0x7fcf2c003bc0 2023-08-27T11:28:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf397fa700 1 -- 172.21.15.112:0/3620291260 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30003c60 con 0x7fcf2c003bc0 2023-08-27T11:28:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3620291260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3c0c4f20 con 0x7fcf2c003bc0 2023-08-27T11:28:21.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.865+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3620291260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3c0c5c40 con 0x7fcf2c003bc0 2023-08-27T11:28:21.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.866+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3620291260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3c067a20 con 0x7fcf2c003bc0 2023-08-27T11:28:21.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.867+0000 7fcf397fa700 1 -- 172.21.15.112:0/3620291260 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fcf30013470 con 0x7fcf2c003bc0 2023-08-27T11:28:21.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.867+0000 7fcf397fa700 1 --2- 172.21.15.112:0/3620291260 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcf2006a4d0 0x7fcf2006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.867+0000 7fcf397fa700 1 -- 172.21.15.112:0/3620291260 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcf3008a4d0 con 0x7fcf2c003bc0 2023-08-27T11:28:21.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.867+0000 7fcf3bfff700 1 --2- 172.21.15.112:0/3620291260 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcf2006a4d0 0x7fcf2006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.868+0000 7fcf3bfff700 1 --2- 172.21.15.112:0/3620291260 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcf2006a4d0 0x7fcf2006c990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fcf28001b80 tx=0x7fcf28000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:21.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:21.871+0000 7fcf397fa700 1 -- 172.21.15.112:0/3620291260 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcf30055670 con 0x7fcf2c003bc0 2023-08-27T11:28:22.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.102+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3620291260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcf3c160710 con 0x7fcf2c003bc0 2023-08-27T11:28:22.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.102+0000 7fcf397fa700 1 -- 172.21.15.112:0/3620291260 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcf30058c90 con 0x7fcf2c003bc0 2023-08-27T11:28:22.104 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:28:22.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.104+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3620291260 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcf2006a4d0 msgr2=0x7fcf2006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:22.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.104+0000 7fcf42f8f700 1 --2- 172.21.15.112:0/3620291260 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcf2006a4d0 0x7fcf2006c990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fcf28001b80 tx=0x7fcf28000ed0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.104+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3620291260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf2c003bc0 msgr2=0x7fcf3c0c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:22.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.104+0000 7fcf42f8f700 1 --2- 172.21.15.112:0/3620291260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf2c003bc0 0x7fcf3c0c5700 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fcf300090b0 tx=0x7fcf300047c0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.105+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3620291260 shutdown_connections 2023-08-27T11:28:22.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.105+0000 7fcf42f8f700 1 --2- 172.21.15.112:0/3620291260 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcf2006a4d0 0x7fcf2006c990 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.105+0000 7fcf42f8f700 1 --2- 172.21.15.112:0/3620291260 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf3c0c9690 0x7fcf3c0c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.105+0000 7fcf42f8f700 1 --2- 172.21.15.112:0/3620291260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf2c003bc0 0x7fcf3c0c5700 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.105+0000 7fcf42f8f700 1 --2- 172.21.15.112:0/3620291260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf3c0c9210 0x7fcf3c0c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.105+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3620291260 >> 172.21.15.112:0/3620291260 conn(0x7fcf3c0d0000 msgr2=0x7fcf3c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:22.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.105+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3620291260 shutdown_connections 2023-08-27T11:28:22.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:22.105+0000 7fcf42f8f700 1 -- 172.21.15.112:0/3620291260 wait complete. 2023-08-27T11:28:22.118 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:28:22.118 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:28:22.119 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2023-08-27T11:28:22.119 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-27T11:28:22.119 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:28:22.120 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:28:22.120 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:28:22.120 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:28:22.121 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:28:22.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:22.122 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2023-08-27T11:28:22.143 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:28:22.143 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:28:22.143 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:28:22.144 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:22.144 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:28:22.144 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:22.144 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:22.144 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:22.145 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:22.145 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:22.145 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:22.145 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:22.146 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:28:22.146 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:22.146 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:28:22.146 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 41757sec preferred_lft 41757sec 2023-08-27T11:28:22.147 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ac17:49c5:cb39:801a/64 scope link noprefixroute 2023-08-27T11:28:22.147 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:22.147 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:28:22.148 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:22.148 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:28:22.148 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:22.149 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c4b4:94ff:fe58:4e35/64 scope link 2023-08-27T11:28:22.149 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:22.149 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:28:22.150 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:28:22.150 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::5806:6aff:fe04:ef32/64 scope link 2023-08-27T11:28:22.150 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:22.151 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:28:22.151 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 8a:21:ba:15:69:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-27T11:28:22.151 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::8821:baff:fe15:6919/64 scope link 2023-08-27T11:28:22.152 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:22.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:22.153 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:28:22.215 INFO:teuthology.orchestra.run.smithi138.stdout:1 2023-08-27T11:28:22.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:22.216 DEBUG:teuthology.orchestra.run.smithi138:> route 2023-08-27T11:28:22.277 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2023-08-27T11:28:22.277 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:28:22.278 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:28:22.278 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:28:22.278 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:28:22.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:22.279 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:28:22.279 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:28:22.280 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:28:22.280 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:28:22.280 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:28:22.281 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:28:22.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:22.389 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:28:22.406 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-27T11:28:22.407 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-27T11:28:22.407 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:28:22.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:22.408 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2023-08-27T11:28:22.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:28:22.475 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-27T11:28:22.532 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:28:22.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:28:22.533 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:28:22.589 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:28:22.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:22.590 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:28:22.676 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:28:22.676 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:28:22.676 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:28:22.684 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:28:22.685 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:28:22.685 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:28:22.685 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:28:22.685 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:28:22.760 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:28:22.760 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:28:22.760 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:28:22.761 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:28:22.761 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:28:22.761 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:28:22.761 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:28:22.761 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:28:22.762 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:28:22.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:22.763 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:28:22.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:22.789 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:28:22.860 INFO:teuthology.orchestra.run.smithi138.stdout:client8640 2023-08-27T11:28:22.861 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:28:22.861 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:22.917 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-08-27T11:28:23.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.042+0000 7f38f45c4700 1 Processor -- start 2023-08-27T11:28:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.042+0000 7f38f45c4700 1 -- start start 2023-08-27T11:28:23.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f45c4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f38ec144bd0 0x7f38ec15b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:23.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f45c4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f38ec0b5eb0 0x7f38ec06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:23.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f45c4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec0b6560 con 0x7f38ec06bfd0 2023-08-27T11:28:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f45c4700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec06c9c0 con 0x7f38ec144bd0 2023-08-27T11:28:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f45c4700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec0b6340 con 0x7f38ec0b5eb0 2023-08-27T11:28:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f2360700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f38ec144bd0 0x7f38ec15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f1b5f700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f38ec06bfd0 0x7f38ec0b57a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58850/0 (socket says 172.21.15.112:58850) 2023-08-27T11:28:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f1b5f700 1 -- 172.21.15.112:0/3708384937 learned_addr learned my addr 172.21.15.112:0/3708384937 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f2b61700 1 --2- 172.21.15.112:0/3708384937 >> v2:172.21.15.138:3300/0 conn(0x7f38ec0b5eb0 0x7f38ec06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f2360700 1 -- 172.21.15.112:0/3708384937 >> v2:172.21.15.138:3300/0 conn(0x7f38ec0b5eb0 msgr2=0x7f38ec06c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f2360700 1 --2- 172.21.15.112:0/3708384937 >> v2:172.21.15.138:3300/0 conn(0x7f38ec0b5eb0 0x7f38ec06c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f2360700 1 -- 172.21.15.112:0/3708384937 >> v1:172.21.15.112:6789/0 conn(0x7f38ec06bfd0 legacy=0x7f38ec0b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f135e700 1 -- 172.21.15.112:0/3708384937 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350312520 0 0) 0x7f38ec0b6560 con 0x7f38ec06bfd0 2023-08-27T11:28:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f2360700 1 -- 172.21.15.112:0/3708384937 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec0b6840 con 0x7f38ec144bd0 2023-08-27T11:28:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.043+0000 7f38f2360700 1 --2- 172.21.15.112:0/3708384937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec144bd0 0x7f38ec15b910 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f38ec066b30 tx=0x7f38dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107cd3ba246db454 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.044+0000 7f38f135e700 1 -- 172.21.15.112:0/3708384937 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc011830 con 0x7f38ec144bd0 2023-08-27T11:28:23.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.044+0000 7f38f135e700 1 -- 172.21.15.112:0/3708384937 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc011990 con 0x7f38ec144bd0 2023-08-27T11:28:23.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.044+0000 7f38f135e700 1 -- 172.21.15.112:0/3708384937 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc011d20 con 0x7f38ec144bd0 2023-08-27T11:28:23.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.044+0000 7f38f45c4700 1 -- 172.21.15.112:0/3708384937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec144bd0 msgr2=0x7f38ec15b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:23.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.044+0000 7f38f45c4700 1 --2- 172.21.15.112:0/3708384937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec144bd0 0x7f38ec15b910 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f38ec066b30 tx=0x7f38dc009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:23.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.044+0000 7f38f45c4700 1 -- 172.21.15.112:0/3708384937 shutdown_connections 2023-08-27T11:28:23.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.044+0000 7f38f45c4700 1 --2- 172.21.15.112:0/3708384937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec144bd0 0x7f38ec15b910 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:23.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.044+0000 7f38f45c4700 1 --2- 172.21.15.112:0/3708384937 >> v2:172.21.15.138:3300/0 conn(0x7f38ec0b5eb0 0x7f38ec06c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:23.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.044+0000 7f38f45c4700 1 -- 172.21.15.112:0/3708384937 >> 172.21.15.112:0/3708384937 conn(0x7f38ec066010 msgr2=0x7f38ec066420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:23.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.044+0000 7f38f45c4700 1 -- 172.21.15.112:0/3708384937 shutdown_connections 2023-08-27T11:28:23.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.044+0000 7f38f45c4700 1 -- 172.21.15.112:0/3708384937 wait complete. 2023-08-27T11:28:23.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.045+0000 7f38f45c4700 1 Processor -- start 2023-08-27T11:28:23.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.045+0000 7f38f45c4700 1 -- start start 2023-08-27T11:28:23.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f45c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec06bfd0 0x7f38ec13aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:23.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f45c4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38ec0b5eb0 0x7f38ec141990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:23.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f45c4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38ec144bd0 0x7f38ec141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:23.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f45c4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec072300 con 0x7f38ec06bfd0 2023-08-27T11:28:23.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f45c4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0716f0 con 0x7f38ec144bd0 2023-08-27T11:28:23.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f45c4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0719d0 con 0x7f38ec0b5eb0 2023-08-27T11:28:23.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f2360700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec06bfd0 0x7f38ec13aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:23.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f2360700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec06bfd0 0x7f38ec13aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36242/0 (socket says 172.21.15.112:36242) 2023-08-27T11:28:23.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f2360700 1 -- 172.21.15.112:0/2314620436 learned_addr learned my addr 172.21.15.112:0/2314620436 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:23.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f2b61700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38ec144bd0 0x7f38ec141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f1b5f700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38ec0b5eb0 0x7f38ec141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f2360700 1 -- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38ec0b5eb0 msgr2=0x7f38ec141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f2360700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38ec0b5eb0 0x7f38ec141990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f2360700 1 -- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38ec144bd0 msgr2=0x7f38ec141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:23.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f2360700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38ec144bd0 0x7f38ec141ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:23.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f2360700 1 -- 172.21.15.112:0/2314620436 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc0094a0 con 0x7f38ec06bfd0 2023-08-27T11:28:23.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.046+0000 7f38f1b5f700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38ec0b5eb0 0x7f38ec141990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:23.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.047+0000 7f38f2360700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec06bfd0 0x7f38ec13aac0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0090b0 tx=0x7f38dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:23.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.047+0000 7f38e2ffd700 1 -- 172.21.15.112:0/2314620436 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc01aa10 con 0x7f38ec06bfd0 2023-08-27T11:28:23.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.047+0000 7f38e2ffd700 1 -- 172.21.15.112:0/2314620436 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc01ab70 con 0x7f38ec06bfd0 2023-08-27T11:28:23.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.047+0000 7f38f45c4700 1 -- 172.21.15.112:0/2314620436 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec071cb0 con 0x7f38ec06bfd0 2023-08-27T11:28:23.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.047+0000 7f38e2ffd700 1 -- 172.21.15.112:0/2314620436 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0243f0 con 0x7f38ec06bfd0 2023-08-27T11:28:23.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.047+0000 7f38f45c4700 1 -- 172.21.15.112:0/2314620436 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec13b000 con 0x7f38ec06bfd0 2023-08-27T11:28:23.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.048+0000 7f38f45c4700 1 -- 172.21.15.112:0/2314620436 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec067a20 con 0x7f38ec06bfd0 2023-08-27T11:28:23.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.048+0000 7f38e2ffd700 1 -- 172.21.15.112:0/2314620436 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f38dc0193b0 con 0x7f38ec06bfd0 2023-08-27T11:28:23.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.049+0000 7f38e2ffd700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f38d006a4d0 0x7f38d006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:23.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.049+0000 7f38e2ffd700 1 -- 172.21.15.112:0/2314620436 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38dc08a530 con 0x7f38ec06bfd0 2023-08-27T11:28:23.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.049+0000 7f38f1b5f700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f38d006a4d0 0x7f38d006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:23.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.050+0000 7f38f1b5f700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f38d006a4d0 0x7f38d006c990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f38ec140d50 tx=0x7f38e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:23.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.053+0000 7f38e2ffd700 1 -- 172.21.15.112:0/2314620436 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f38dc055750 con 0x7f38ec06bfd0 2023-08-27T11:28:23.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.286+0000 7f38f45c4700 1 -- 172.21.15.112:0/2314620436 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f38ec066770 con 0x7f38ec06bfd0 2023-08-27T11:28:23.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.698+0000 7f38e2ffd700 1 -- 172.21.15.112:0/2314620436 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v95) v1 ==== 136+0+0 (secure 0 0 0) 0x7f38dc058d70 con 0x7f38ec06bfd0 2023-08-27T11:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.700+0000 7f38f45c4700 1 -- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f38d006a4d0 msgr2=0x7f38d006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.700+0000 7f38f45c4700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f38d006a4d0 0x7f38d006c990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f38ec140d50 tx=0x7f38e8009040 comp rx=0 tx=0).stop 2023-08-27T11:28:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.700+0000 7f38f45c4700 1 -- 172.21.15.112:0/2314620436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec06bfd0 msgr2=0x7f38ec13aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.700+0000 7f38f45c4700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec06bfd0 0x7f38ec13aac0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0090b0 tx=0x7f38dc004750 comp rx=0 tx=0).stop 2023-08-27T11:28:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.701+0000 7f38f45c4700 1 -- 172.21.15.112:0/2314620436 shutdown_connections 2023-08-27T11:28:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.701+0000 7f38f45c4700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f38d006a4d0 0x7f38d006c990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.701+0000 7f38f45c4700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38ec144bd0 0x7f38ec141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.701+0000 7f38f45c4700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec06bfd0 0x7f38ec13aac0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.701+0000 7f38f45c4700 1 --2- 172.21.15.112:0/2314620436 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38ec0b5eb0 0x7f38ec141990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.701+0000 7f38f45c4700 1 -- 172.21.15.112:0/2314620436 >> 172.21.15.112:0/2314620436 conn(0x7f38ec066010 msgr2=0x7f38ec15e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.701+0000 7f38f45c4700 1 -- 172.21.15.112:0/2314620436 shutdown_connections 2023-08-27T11:28:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.701+0000 7f38f45c4700 1 -- 172.21.15.112:0/2314620436 wait complete. 2023-08-27T11:28:23.714 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:28:23.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.834+0000 7fb455ff9700 1 Processor -- start 2023-08-27T11:28:23.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.834+0000 7fb455ff9700 1 -- start start 2023-08-27T11:28:23.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.834+0000 7fb455ff9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb450163f90 0x7fb450166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:23.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.834+0000 7fb455ff9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4500b5ae0 con 0x7fb450144570 2023-08-27T11:28:23.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.834+0000 7fb455ff9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4500b45f0 con 0x7fb4500b3e50 2023-08-27T11:28:23.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.835+0000 7fb455ff9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4500cf7a0 con 0x7fb450163f90 2023-08-27T11:28:23.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.835+0000 7fb44ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb450163f90 0x7fb450166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:23.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.835+0000 7fb44f7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb450144570 0x7fb45015ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58864/0 (socket says 172.21.15.112:58864) 2023-08-27T11:28:23.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.835+0000 7fb44f7fe700 1 -- 172.21.15.112:0/3641149938 learned_addr learned my addr 172.21.15.112:0/3641149938 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:23.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.835+0000 7fb44ffff700 1 -- 172.21.15.112:0/3641149938 >> v1:172.21.15.138:6789/0 conn(0x7fb4500b3e50 legacy=0x7fb45015f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:23.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.835+0000 7fb44ffff700 1 -- 172.21.15.112:0/3641149938 >> v1:172.21.15.112:6789/0 conn(0x7fb450144570 legacy=0x7fb45015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:23.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.835+0000 7fb44ffff700 1 -- 172.21.15.112:0/3641149938 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45015fc40 con 0x7fb450163f90 2023-08-27T11:28:23.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.835+0000 7fb44e7fc700 1 -- 172.21.15.112:0/3641149938 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686708976 0 0) 0x7fb440001110 con 0x7fb450144570 2023-08-27T11:28:23.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.835+0000 7fb44ffff700 1 --2- 172.21.15.112:0/3641149938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb450163f90 0x7fb450166380 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fb448009d90 tx=0x7fb44800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b08cda3f1c81371 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:23.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.835+0000 7fb44e7fc700 1 -- 172.21.15.112:0/3641149938 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44800e070 con 0x7fb450163f90 2023-08-27T11:28:23.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.835+0000 7fb44e7fc700 1 -- 172.21.15.112:0/3641149938 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44800bb90 con 0x7fb450163f90 2023-08-27T11:28:23.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.835+0000 7fb44e7fc700 1 -- 172.21.15.112:0/3641149938 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448003a20 con 0x7fb450163f90 2023-08-27T11:28:23.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.836+0000 7fb455ff9700 1 -- 172.21.15.112:0/3641149938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb450163f90 msgr2=0x7fb450166380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:23.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.836+0000 7fb455ff9700 1 --2- 172.21.15.112:0/3641149938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb450163f90 0x7fb450166380 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fb448009d90 tx=0x7fb44800b330 comp rx=0 tx=0).stop 2023-08-27T11:28:23.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.836+0000 7fb455ff9700 1 -- 172.21.15.112:0/3641149938 shutdown_connections 2023-08-27T11:28:23.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.836+0000 7fb455ff9700 1 --2- 172.21.15.112:0/3641149938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb450163f90 0x7fb450166380 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:23.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.836+0000 7fb455ff9700 1 -- 172.21.15.112:0/3641149938 >> 172.21.15.112:0/3641149938 conn(0x7fb45006f970 msgr2=0x7fb45006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:23.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.836+0000 7fb455ff9700 1 -- 172.21.15.112:0/3641149938 shutdown_connections 2023-08-27T11:28:23.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.836+0000 7fb455ff9700 1 -- 172.21.15.112:0/3641149938 wait complete. 2023-08-27T11:28:23.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.837+0000 7fb455ff9700 1 Processor -- start 2023-08-27T11:28:23.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.837+0000 7fb455ff9700 1 -- start start 2023-08-27T11:28:23.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb455ff9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b3e50 0x7fb450157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:23.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb455ff9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb450144570 0x7fb450161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:23.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb455ff9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb450163f90 0x7fb4501409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:23.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb455ff9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb450158170 con 0x7fb450163f90 2023-08-27T11:28:23.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb455ff9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45015c780 con 0x7fb4500b3e50 2023-08-27T11:28:23.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb455ff9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb450160160 con 0x7fb450144570 2023-08-27T11:28:23.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb44ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb450163f90 0x7fb4501409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:23.847 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb44ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb450163f90 0x7fb4501409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36260/0 (socket says 172.21.15.112:36260) 2023-08-27T11:28:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb44ffff700 1 -- 172.21.15.112:0/2138145090 learned_addr learned my addr 172.21.15.112:0/2138145090 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb44f7fe700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b3e50 0x7fb450157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb44effd700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb450144570 0x7fb450161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb44ffff700 1 -- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb450144570 msgr2=0x7fb450161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb44ffff700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb450144570 0x7fb450161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb44ffff700 1 -- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b3e50 msgr2=0x7fb450157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:23.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb44ffff700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b3e50 0x7fb450157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:23.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb44ffff700 1 -- 172.21.15.112:0/2138145090 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb448009a70 con 0x7fb450163f90 2023-08-27T11:28:23.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.838+0000 7fb44ffff700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb450163f90 0x7fb4501409c0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fb45006b920 tx=0x7fb448004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:23.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.839+0000 7fb437fff700 1 -- 172.21.15.112:0/2138145090 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448004970 con 0x7fb450163f90 2023-08-27T11:28:23.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.839+0000 7fb437fff700 1 -- 172.21.15.112:0/2138145090 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb448012630 con 0x7fb450163f90 2023-08-27T11:28:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.839+0000 7fb437fff700 1 -- 172.21.15.112:0/2138145090 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4480129e0 con 0x7fb450163f90 2023-08-27T11:28:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.839+0000 7fb455ff9700 1 -- 172.21.15.112:0/2138145090 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb450160440 con 0x7fb450163f90 2023-08-27T11:28:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.839+0000 7fb455ff9700 1 -- 172.21.15.112:0/2138145090 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb450160a00 con 0x7fb450163f90 2023-08-27T11:28:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.839+0000 7fb455ff9700 1 -- 172.21.15.112:0/2138145090 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb450067a20 con 0x7fb450163f90 2023-08-27T11:28:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.840+0000 7fb437fff700 1 -- 172.21.15.112:0/2138145090 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fb448012b40 con 0x7fb450163f90 2023-08-27T11:28:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.840+0000 7fb437fff700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb43c06a500 0x7fb43c06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.840+0000 7fb437fff700 1 -- 172.21.15.112:0/2138145090 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb44808ad90 con 0x7fb450163f90 2023-08-27T11:28:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.841+0000 7fb44f7fe700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb43c06a500 0x7fb43c06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.841+0000 7fb44f7fe700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb43c06a500 0x7fb43c06c9c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb440007e80 tx=0x7fb440007e10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:23.845+0000 7fb437fff700 1 -- 172.21.15.112:0/2138145090 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb4480598b0 con 0x7fb450163f90 2023-08-27T11:28:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.061+0000 7fb455ff9700 1 -- 172.21.15.112:0/2138145090 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4500b4310 con 0x7fb43c06a500 2023-08-27T11:28:24.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.064+0000 7fb437fff700 1 -- 172.21.15.112:0/2138145090 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb440004a70 con 0x7fb43c06a500 2023-08-27T11:28:24.065 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:28:24.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:24.065 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs" 2023-08-27T11:28:24.066 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:24.066 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:28:24.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.065+0000 7fb455ff9700 1 -- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb43c06a500 msgr2=0x7fb43c06c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:24.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.065+0000 7fb455ff9700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb43c06a500 0x7fb43c06c9c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb440007e80 tx=0x7fb440007e10 comp rx=0 tx=0).stop 2023-08-27T11:28:24.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.065+0000 7fb455ff9700 1 -- 172.21.15.112:0/2138145090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb450163f90 msgr2=0x7fb4501409c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:24.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.065+0000 7fb455ff9700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb450163f90 0x7fb4501409c0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fb45006b920 tx=0x7fb448004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.065+0000 7fb455ff9700 1 -- 172.21.15.112:0/2138145090 shutdown_connections 2023-08-27T11:28:24.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.065+0000 7fb455ff9700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb43c06a500 0x7fb43c06c9c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.065+0000 7fb455ff9700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb450163f90 0x7fb4501409c0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.065+0000 7fb455ff9700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b3e50 0x7fb450157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.065+0000 7fb455ff9700 1 --2- 172.21.15.112:0/2138145090 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb450144570 0x7fb450161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.065+0000 7fb455ff9700 1 -- 172.21.15.112:0/2138145090 >> 172.21.15.112:0/2138145090 conn(0x7fb45006f970 msgr2=0x7fb45015f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:24.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.065+0000 7fb455ff9700 1 -- 172.21.15.112:0/2138145090 shutdown_connections 2023-08-27T11:28:24.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.065+0000 7fb455ff9700 1 -- 172.21.15.112:0/2138145090 wait complete. 2023-08-27T11:28:24.080 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:28:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.200+0000 7f5137592700 1 Processor -- start 2023-08-27T11:28:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.201+0000 7f5137592700 1 -- start start 2023-08-27T11:28:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.201+0000 7f5137592700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5130146c20 0x7f5130160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.201+0000 7f5137592700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f513006f790 0x7f513006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.201+0000 7f5137592700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51300b5070 con 0x7f51300b3440 2023-08-27T11:28:24.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.201+0000 7f5137592700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5130070180 con 0x7f513006f790 2023-08-27T11:28:24.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.201+0000 7f5137592700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5130147000 con 0x7f5130146c20 2023-08-27T11:28:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.202+0000 7f5135b2f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f513006f790 0x7f513006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.202+0000 7f5135b2f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f513006f790 0x7f513006fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36268/0 (socket says 172.21.15.112:36268) 2023-08-27T11:28:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.202+0000 7f5135b2f700 1 -- 172.21.15.112:0/3265264090 learned_addr learned my addr 172.21.15.112:0/3265264090 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.202+0000 7f513532e700 1 --2- 172.21.15.112:0/3265264090 >> v2:172.21.15.138:3300/0 conn(0x7f5130146c20 0x7f5130160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.202+0000 7f5135b2f700 1 -- 172.21.15.112:0/3265264090 >> v2:172.21.15.138:3300/0 conn(0x7f5130146c20 msgr2=0x7f5130160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.202+0000 7f5135b2f700 1 --2- 172.21.15.112:0/3265264090 >> v2:172.21.15.138:3300/0 conn(0x7f5130146c20 0x7f5130160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.202+0000 7f5135b2f700 1 -- 172.21.15.112:0/3265264090 >> v1:172.21.15.138:6789/0 conn(0x7f51300b3440 legacy=0x7f513015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:24.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.202+0000 7f5135b2f700 1 -- 172.21.15.112:0/3265264090 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51301614a0 con 0x7f513006f790 2023-08-27T11:28:24.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.202+0000 7f5135b2f700 1 --2- 172.21.15.112:0/3265264090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513006f790 0x7f513006fc40 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f51300b2d00 tx=0x7f512800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfe50401a97543c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:24.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.202+0000 7f5127fff700 1 -- 172.21.15.112:0/3265264090 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800e070 con 0x7f513006f790 2023-08-27T11:28:24.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.202+0000 7f5127fff700 1 -- 172.21.15.112:0/3265264090 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512800b9c0 con 0x7f513006f790 2023-08-27T11:28:24.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.202+0000 7f5127fff700 1 -- 172.21.15.112:0/3265264090 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800bd70 con 0x7f513006f790 2023-08-27T11:28:24.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.203+0000 7f5137592700 1 -- 172.21.15.112:0/3265264090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513006f790 msgr2=0x7f513006fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:24.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.203+0000 7f5137592700 1 --2- 172.21.15.112:0/3265264090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513006f790 0x7f513006fc40 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f51300b2d00 tx=0x7f512800b330 comp rx=0 tx=0).stop 2023-08-27T11:28:24.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.203+0000 7f5137592700 1 -- 172.21.15.112:0/3265264090 shutdown_connections 2023-08-27T11:28:24.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.203+0000 7f5137592700 1 --2- 172.21.15.112:0/3265264090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513006f790 0x7f513006fc40 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.203+0000 7f5137592700 1 --2- 172.21.15.112:0/3265264090 >> v2:172.21.15.138:3300/0 conn(0x7f5130146c20 0x7f5130160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.203+0000 7f5137592700 1 -- 172.21.15.112:0/3265264090 >> 172.21.15.112:0/3265264090 conn(0x7f51300b1e90 msgr2=0x7f51300b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:24.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.203+0000 7f5137592700 1 -- 172.21.15.112:0/3265264090 shutdown_connections 2023-08-27T11:28:24.209 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.203+0000 7f5137592700 1 -- 172.21.15.112:0/3265264090 wait complete. 2023-08-27T11:28:24.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.204+0000 7f5137592700 1 Processor -- start 2023-08-27T11:28:24.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.204+0000 7f5137592700 1 -- start start 2023-08-27T11:28:24.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f5137592700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513006f790 0x7f513013fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f5137592700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51300b3440 0x7f51301f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f5137592700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5130146c20 0x7f51300ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f5137592700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51300b1760 con 0x7f513006f790 2023-08-27T11:28:24.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f5137592700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513013ff60 con 0x7f51300b3440 2023-08-27T11:28:24.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f5137592700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5130141850 con 0x7f5130146c20 2023-08-27T11:28:24.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f513532e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513006f790 0x7f513013fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f513532e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513006f790 0x7f513013fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36270/0 (socket says 172.21.15.112:36270) 2023-08-27T11:28:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f513532e700 1 -- 172.21.15.112:0/2259494196 learned_addr learned my addr 172.21.15.112:0/2259494196 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f5134b2d700 1 --2- 172.21.15.112:0/2259494196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51300b3440 0x7f51301f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f513532e700 1 -- 172.21.15.112:0/2259494196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5130146c20 msgr2=0x7f51300ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f513532e700 1 --2- 172.21.15.112:0/2259494196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5130146c20 0x7f51300ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f513532e700 1 -- 172.21.15.112:0/2259494196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51300b3440 msgr2=0x7f51301f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f513532e700 1 --2- 172.21.15.112:0/2259494196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51300b3440 0x7f51301f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f513532e700 1 -- 172.21.15.112:0/2259494196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5128009b30 con 0x7f513006f790 2023-08-27T11:28:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f513532e700 1 --2- 172.21.15.112:0/2259494196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513006f790 0x7f513013fa20 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f512000d800 tx=0x7f512000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f5125ffb700 1 -- 172.21.15.112:0/2259494196 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51200125f0 con 0x7f513006f790 2023-08-27T11:28:24.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f5125ffb700 1 -- 172.21.15.112:0/2259494196 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5120012750 con 0x7f513006f790 2023-08-27T11:28:24.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f5137592700 1 -- 172.21.15.112:0/2259494196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51300c5220 con 0x7f513006f790 2023-08-27T11:28:24.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.205+0000 7f5125ffb700 1 -- 172.21.15.112:0/2259494196 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120012b00 con 0x7f513006f790 2023-08-27T11:28:24.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.206+0000 7f5137592700 1 -- 172.21.15.112:0/2259494196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51300c57a0 con 0x7f513006f790 2023-08-27T11:28:24.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.207+0000 7f5137592700 1 -- 172.21.15.112:0/2259494196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5130067e90 con 0x7f513006f790 2023-08-27T11:28:24.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.207+0000 7f5125ffb700 1 -- 172.21.15.112:0/2259494196 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5120012c60 con 0x7f513006f790 2023-08-27T11:28:24.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.207+0000 7f5125ffb700 1 --2- 172.21.15.112:0/2259494196 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f511406a4d0 0x7f511406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.207+0000 7f5125ffb700 1 -- 172.21.15.112:0/2259494196 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5120088ff0 con 0x7f513006f790 2023-08-27T11:28:24.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.207+0000 7f5134b2d700 1 --2- 172.21.15.112:0/2259494196 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f511406a4d0 0x7f511406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:24.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.208+0000 7f5134b2d700 1 --2- 172.21.15.112:0/2259494196 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f511406a4d0 0x7f511406c990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5130141bf0 tx=0x7f512c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:24.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.212+0000 7f5125ffb700 1 -- 172.21.15.112:0/2259494196 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5120054190 con 0x7f513006f790 2023-08-27T11:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.401+0000 7f5137592700 1 -- 172.21.15.112:0/2259494196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5130067a20 con 0x7f513006f790 2023-08-27T11:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.401+0000 7f5125ffb700 1 -- 172.21.15.112:0/2259494196 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f51200577b0 con 0x7f513006f790 2023-08-27T11:28:24.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.403+0000 7f5137592700 1 -- 172.21.15.112:0/2259494196 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f511406a4d0 msgr2=0x7f511406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:24.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.403+0000 7f5137592700 1 --2- 172.21.15.112:0/2259494196 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f511406a4d0 0x7f511406c990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5130141bf0 tx=0x7f512c009040 comp rx=0 tx=0).stop 2023-08-27T11:28:24.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.403+0000 7f5137592700 1 -- 172.21.15.112:0/2259494196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513006f790 msgr2=0x7f513013fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:24.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.403+0000 7f5137592700 1 --2- 172.21.15.112:0/2259494196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513006f790 0x7f513013fa20 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f512000d800 tx=0x7f512000dbc0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.403+0000 7f5137592700 1 -- 172.21.15.112:0/2259494196 shutdown_connections 2023-08-27T11:28:24.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.403+0000 7f5137592700 1 --2- 172.21.15.112:0/2259494196 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f511406a4d0 0x7f511406c990 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.403+0000 7f5137592700 1 --2- 172.21.15.112:0/2259494196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5130146c20 0x7f51300ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.403+0000 7f5137592700 1 --2- 172.21.15.112:0/2259494196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513006f790 0x7f513013fa20 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.403+0000 7f5137592700 1 --2- 172.21.15.112:0/2259494196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51300b3440 0x7f51301f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.403+0000 7f5137592700 1 -- 172.21.15.112:0/2259494196 >> 172.21.15.112:0/2259494196 conn(0x7f51300b1e90 msgr2=0x7f51300709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.404+0000 7f5137592700 1 -- 172.21.15.112:0/2259494196 shutdown_connections 2023-08-27T11:28:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.404+0000 7f5137592700 1 -- 172.21.15.112:0/2259494196 wait complete. 2023-08-27T11:28:24.417 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:28:24.418 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:24.435 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:28:24.436 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:24.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:24.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.650+0000 7f110c485700 1 Processor -- start 2023-08-27T11:28:24.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.651+0000 7f110c485700 1 -- start start 2023-08-27T11:28:24.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.651+0000 7f110c485700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11040b1400 0x7f11040cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.651+0000 7f110c485700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f11040b17e0 0x7f110406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.651+0000 7f110c485700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1104161ce0 0x7f110406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.651+0000 7f110c485700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1104161850 con 0x7f1104161ce0 2023-08-27T11:28:24.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.651+0000 7f110c485700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110406e7d0 con 0x7f11040b17e0 2023-08-27T11:28:24.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.651+0000 7f110c485700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11040cfc10 con 0x7f11040b1400 2023-08-27T11:28:24.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f110aa22700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1104161ce0 0x7f110406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:24.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f110aa22700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1104161ce0 0x7f110406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36278/0 (socket says 172.21.15.112:36278) 2023-08-27T11:28:24.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f110aa22700 1 -- 172.21.15.112:0/1607188397 learned_addr learned my addr 172.21.15.112:0/1607188397 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:24.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f1109a20700 1 --2- 172.21.15.112:0/1607188397 >> v2:172.21.15.138:3300/0 conn(0x7f11040b17e0 0x7f110406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:24.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f110aa22700 1 -- 172.21.15.112:0/1607188397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11040b1400 msgr2=0x7f11040cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:24.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f110aa22700 1 --2- 172.21.15.112:0/1607188397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11040b1400 0x7f11040cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f110aa22700 1 -- 172.21.15.112:0/1607188397 >> v2:172.21.15.138:3300/0 conn(0x7f11040b17e0 msgr2=0x7f110406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:24.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f110aa22700 1 --2- 172.21.15.112:0/1607188397 >> v2:172.21.15.138:3300/0 conn(0x7f11040b17e0 0x7f110406e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f110aa22700 1 -- 172.21.15.112:0/1607188397 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1104161200 con 0x7f1104161ce0 2023-08-27T11:28:24.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f1109a20700 1 --2- 172.21.15.112:0/1607188397 >> v2:172.21.15.138:3300/0 conn(0x7f11040b17e0 0x7f110406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:24.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f110aa22700 1 --2- 172.21.15.112:0/1607188397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1104161ce0 0x7f110406f790 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f11040d0db0 tx=0x7f10fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96de437858c890a1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:24.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f110921f700 1 -- 172.21.15.112:0/1607188397 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00e070 con 0x7f1104161ce0 2023-08-27T11:28:24.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f110921f700 1 -- 172.21.15.112:0/1607188397 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc00b9c0 con 0x7f1104161ce0 2023-08-27T11:28:24.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.652+0000 7f110921f700 1 -- 172.21.15.112:0/1607188397 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00bd70 con 0x7f1104161ce0 2023-08-27T11:28:24.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.653+0000 7f110c485700 1 -- 172.21.15.112:0/1607188397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1104161ce0 msgr2=0x7f110406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:24.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.653+0000 7f110c485700 1 --2- 172.21.15.112:0/1607188397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1104161ce0 0x7f110406f790 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f11040d0db0 tx=0x7f10fc00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:24.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.653+0000 7f110c485700 1 -- 172.21.15.112:0/1607188397 shutdown_connections 2023-08-27T11:28:24.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.653+0000 7f110c485700 1 --2- 172.21.15.112:0/1607188397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1104161ce0 0x7f110406f790 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.653+0000 7f110c485700 1 --2- 172.21.15.112:0/1607188397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11040b1400 0x7f11040cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.653+0000 7f110c485700 1 --2- 172.21.15.112:0/1607188397 >> v2:172.21.15.138:3300/0 conn(0x7f11040b17e0 0x7f110406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.653+0000 7f110c485700 1 -- 172.21.15.112:0/1607188397 >> 172.21.15.112:0/1607188397 conn(0x7f11040d0000 msgr2=0x7f11040d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:24.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.653+0000 7f110c485700 1 -- 172.21.15.112:0/1607188397 shutdown_connections 2023-08-27T11:28:24.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.653+0000 7f110c485700 1 -- 172.21.15.112:0/1607188397 wait complete. 2023-08-27T11:28:24.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.654+0000 7f110c485700 1 Processor -- start 2023-08-27T11:28:24.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.654+0000 7f110c485700 1 -- start start 2023-08-27T11:28:24.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110c485700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11040b1400 0x7f1104154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110c485700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11040b17e0 0x7f11040b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110c485700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1104161ce0 0x7f11040c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110c485700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11041606d0 con 0x7f1104161ce0 2023-08-27T11:28:24.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110c485700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1104155450 con 0x7f11040b17e0 2023-08-27T11:28:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110c485700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110415e9f0 con 0x7f11040b1400 2023-08-27T11:28:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110aa22700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1104161ce0 0x7f11040c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110aa22700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1104161ce0 0x7f11040c3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36286/0 (socket says 172.21.15.112:36286) 2023-08-27T11:28:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110aa22700 1 -- 172.21.15.112:0/4219439156 learned_addr learned my addr 172.21.15.112:0/4219439156 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110a221700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11040b1400 0x7f1104154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:24.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f1109a20700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11040b17e0 0x7f11040b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:24.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110aa22700 1 -- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11040b1400 msgr2=0x7f1104154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:24.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110aa22700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11040b1400 0x7f1104154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110aa22700 1 -- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11040b17e0 msgr2=0x7f11040b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:24.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110aa22700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11040b17e0 0x7f11040b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110aa22700 1 -- 172.21.15.112:0/4219439156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc009b30 con 0x7f1104161ce0 2023-08-27T11:28:24.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f1109a20700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11040b17e0 0x7f11040b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:24.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f110aa22700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1104161ce0 0x7f11040c3b30 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f11040d1100 tx=0x7f10fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.655+0000 7f10faffd700 1 -- 172.21.15.112:0/4219439156 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc004310 con 0x7f1104161ce0 2023-08-27T11:28:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.656+0000 7f10faffd700 1 -- 172.21.15.112:0/4219439156 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc004470 con 0x7f1104161ce0 2023-08-27T11:28:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.656+0000 7f110c485700 1 -- 172.21.15.112:0/4219439156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11040b0700 con 0x7f1104161ce0 2023-08-27T11:28:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.656+0000 7f10faffd700 1 -- 172.21.15.112:0/4219439156 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc004820 con 0x7f1104161ce0 2023-08-27T11:28:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.656+0000 7f110c485700 1 -- 172.21.15.112:0/4219439156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11041f6e80 con 0x7f1104161ce0 2023-08-27T11:28:24.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.657+0000 7f110c485700 1 -- 172.21.15.112:0/4219439156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110415ebc0 con 0x7f1104161ce0 2023-08-27T11:28:24.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.657+0000 7f10faffd700 1 -- 172.21.15.112:0/4219439156 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f10fc01c440 con 0x7f1104161ce0 2023-08-27T11:28:24.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.657+0000 7f10faffd700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10e806a4d0 0x7f10e806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.657+0000 7f10faffd700 1 -- 172.21.15.112:0/4219439156 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f10fc08a5c0 con 0x7f1104161ce0 2023-08-27T11:28:24.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.658+0000 7f110a221700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10e806a4d0 0x7f10e806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:24.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.658+0000 7f110a221700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10e806a4d0 0x7f10e806c990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009a10 tx=0x7f10f4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:24.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.662+0000 7f10faffd700 1 -- 172.21.15.112:0/4219439156 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f110415ebc0 con 0x7f1104161ce0 2023-08-27T11:28:24.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.853+0000 7f110c485700 1 -- 172.21.15.112:0/4219439156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f110415ebc0 con 0x7f1104161ce0 2023-08-27T11:28:24.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.854+0000 7f10faffd700 1 -- 172.21.15.112:0/4219439156 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+18263 (secure 0 0 0) 0x7f110415ebc0 con 0x7f1104161ce0 2023-08-27T11:28:24.856 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:24.856 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":139,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:15.631983+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:12.176044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:12.992996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:24.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.856+0000 7f110c485700 1 -- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10e806a4d0 msgr2=0x7f10e806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:24.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.856+0000 7f110c485700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10e806a4d0 0x7f10e806c990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009a10 tx=0x7f10f4009190 comp rx=0 tx=0).stop 2023-08-27T11:28:24.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.856+0000 7f110c485700 1 -- 172.21.15.112:0/4219439156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1104161ce0 msgr2=0x7f11040c3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:24.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.856+0000 7f110c485700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1104161ce0 0x7f11040c3b30 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f11040d1100 tx=0x7f10fc004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.856+0000 7f110c485700 1 -- 172.21.15.112:0/4219439156 shutdown_connections 2023-08-27T11:28:24.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.856+0000 7f110c485700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f10e806a4d0 0x7f10e806c990 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.856+0000 7f110c485700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1104161ce0 0x7f11040c3b30 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.856+0000 7f110c485700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11040b1400 0x7f1104154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.856+0000 7f110c485700 1 --2- 172.21.15.112:0/4219439156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11040b17e0 0x7f11040b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:24.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.856+0000 7f110c485700 1 -- 172.21.15.112:0/4219439156 >> 172.21.15.112:0/4219439156 conn(0x7f11040d0000 msgr2=0x7f1104071560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:24.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.856+0000 7f110c485700 1 -- 172.21.15.112:0/4219439156 shutdown_connections 2023-08-27T11:28:24.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.856+0000 7f110c485700 1 -- 172.21.15.112:0/4219439156 wait complete. 2023-08-27T11:28:24.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:28:12.176044+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:28:12.992996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:28:24.871 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:28:24.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.992+0000 7f3ad5ca4700 1 Processor -- start 2023-08-27T11:28:24.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.992+0000 7f3ad5ca4700 1 -- start start 2023-08-27T11:28:24.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.993+0000 7f3ad5ca4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ad0146c20 0x7f3ad0160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.993+0000 7f3ad5ca4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3ad00b3440 0x7f3ad015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:24.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.993+0000 7f3ad5ca4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad00b5070 con 0x7f3ad015e6e0 2023-08-27T11:28:24.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.993+0000 7f3ad5ca4700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad0147000 con 0x7f3ad00b3440 2023-08-27T11:28:24.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.993+0000 7f3ad5ca4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad01614a0 con 0x7f3ad0146c20 2023-08-27T11:28:24.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.993+0000 7f3acffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3ad015e6e0 0x7f3ad006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58908/0 (socket says 172.21.15.112:58908) 2023-08-27T11:28:24.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.993+0000 7f3acffff700 1 -- 172.21.15.112:0/696939601 learned_addr learned my addr 172.21.15.112:0/696939601 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:24.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3acf7fe700 1 --2- 172.21.15.112:0/696939601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ad0146c20 0x7f3ad0160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:24.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3aceffd700 1 --2- 172.21.15.112:0/696939601 >> v2:172.21.15.138:3300/0 conn(0x7f3ad00b3440 0x7f3ad015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186149640 0 0) 0x7f3ac8001490 con 0x7f3ad015e6e0 2023-08-27T11:28:25.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ab80035b0 con 0x7f3ad015e6e0 2023-08-27T11:28:25.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 564406984 0 0) 0x7f3ab80035b0 con 0x7f3ad015e6e0 2023-08-27T11:28:25.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ad0146c20 msgr2=0x7f3ad0160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:25.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 --2- 172.21.15.112:0/696939601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ad0146c20 0x7f3ad0160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 >> v2:172.21.15.138:3300/0 conn(0x7f3ad00b3440 msgr2=0x7f3ad015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 --2- 172.21.15.112:0/696939601 >> v2:172.21.15.138:3300/0 conn(0x7f3ad00b3440 0x7f3ad015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad0161740 con 0x7f3ad015e6e0 2023-08-27T11:28:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f3ac8001b80 con 0x7f3ad015e6e0 2023-08-27T11:28:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 >> v1:172.21.15.112:6789/0 conn(0x7f3ad015e6e0 legacy=0x7f3ad006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 --2- 172.21.15.112:0/696939601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab8003a00 0x7f3ab8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 --2- 172.21.15.112:0/696939601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ad0146c20 0x7f3ab8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 --2- 172.21.15.112:0/696939601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab8004d90 0x7f3ab8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0147000 con 0x7f3ab8004d90 2023-08-27T11:28:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad01614a0 con 0x7f3ab8003a00 2023-08-27T11:28:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab800ba10 con 0x7f3ad0146c20 2023-08-27T11:28:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3acffff700 1 --2- 172.21.15.112:0/696939601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab8004d90 0x7f3ab8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3acf7fe700 1 --2- 172.21.15.112:0/696939601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab8003a00 0x7f3ab8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3aceffd700 1 --2- 172.21.15.112:0/696939601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ad0146c20 0x7f3ab8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3acffff700 1 -- 172.21.15.112:0/696939601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ad0146c20 msgr2=0x7f3ab8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3acffff700 1 --2- 172.21.15.112:0/696939601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ad0146c20 0x7f3ab8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3acffff700 1 -- 172.21.15.112:0/696939601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab8003a00 msgr2=0x7f3ab8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:25.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3acffff700 1 --2- 172.21.15.112:0/696939601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab8003a00 0x7f3ab8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3acffff700 1 -- 172.21.15.112:0/696939601 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab80035b0 con 0x7f3ab8004d90 2023-08-27T11:28:25.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.994+0000 7f3acf7fe700 1 --2- 172.21.15.112:0/696939601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab8003a00 0x7f3ab8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:25.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.995+0000 7f3acffff700 1 --2- 172.21.15.112:0/696939601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab8004d90 0x7f3ab8005210 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800a830 tx=0x7f3ab8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546cb1b3865b9595 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.995+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac800a4c0 con 0x7f3ab8004d90 2023-08-27T11:28:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.995+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac8012600 con 0x7f3ab8004d90 2023-08-27T11:28:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.995+0000 7f3ace7fc700 1 -- 172.21.15.112:0/696939601 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac80129b0 con 0x7f3ab8004d90 2023-08-27T11:28:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.995+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/696939601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab8004d90 msgr2=0x7f3ab8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.995+0000 7f3ad5ca4700 1 --2- 172.21.15.112:0/696939601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab8004d90 0x7f3ab8005210 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800a830 tx=0x7f3ab8005880 comp rx=0 tx=0).stop 2023-08-27T11:28:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.995+0000 7f3acffff700 1 -- 172.21.15.112:0/696939601 reap_dead start 2023-08-27T11:28:25.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.995+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/696939601 shutdown_connections 2023-08-27T11:28:25.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.995+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/696939601 >> 172.21.15.112:0/696939601 conn(0x7f3ad00b1e90 msgr2=0x7f3ad00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:25.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.995+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/696939601 shutdown_connections 2023-08-27T11:28:25.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.995+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/696939601 wait complete. 2023-08-27T11:28:25.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.996+0000 7f3ad5ca4700 1 Processor -- start 2023-08-27T11:28:25.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.997+0000 7f3ad5ca4700 1 -- start start 2023-08-27T11:28:25.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.997+0000 7f3ad5ca4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ad01407e0 0x7f3ad01444c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.997+0000 7f3ad5ca4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ad0140bc0 0x7f3ad00b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.997+0000 7f3ad5ca4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ad00b0d70 0x7f3ad0144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.997+0000 7f3ad5ca4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad01440a0 con 0x7f3ad01407e0 2023-08-27T11:28:25.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.997+0000 7f3ad5ca4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad00b1200 con 0x7f3ad0140bc0 2023-08-27T11:28:25.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.997+0000 7f3ad5ca4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0143420 con 0x7f3ad00b0d70 2023-08-27T11:28:25.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.997+0000 7f3acf7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ad01407e0 0x7f3ad01444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.997+0000 7f3acf7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ad01407e0 0x7f3ad01444c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36302/0 (socket says 172.21.15.112:36302) 2023-08-27T11:28:25.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.997+0000 7f3acf7fe700 1 -- 172.21.15.112:0/2690522699 learned_addr learned my addr 172.21.15.112:0/2690522699 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:25.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.998+0000 7f3aceffd700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ad0140bc0 0x7f3ad00b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.998+0000 7f3acffff700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ad00b0d70 0x7f3ad0144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.998+0000 7f3acf7fe700 1 -- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ad00b0d70 msgr2=0x7f3ad0144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.998+0000 7f3acf7fe700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ad00b0d70 0x7f3ad0144e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.998+0000 7f3acf7fe700 1 -- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ad0140bc0 msgr2=0x7f3ad00b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.998+0000 7f3acf7fe700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ad0140bc0 0x7f3ad00b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.998+0000 7f3acf7fe700 1 -- 172.21.15.112:0/2690522699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad0143700 con 0x7f3ad01407e0 2023-08-27T11:28:25.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.998+0000 7f3acf7fe700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ad01407e0 0x7f3ad01444c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4010850 tx=0x7f3ac4010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:25.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.998+0000 7f3ab7fff700 1 -- 172.21.15.112:0/2690522699 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4004020 con 0x7f3ad01407e0 2023-08-27T11:28:25.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.998+0000 7f3ab7fff700 1 -- 172.21.15.112:0/2690522699 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac4004180 con 0x7f3ad01407e0 2023-08-27T11:28:25.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.998+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/2690522699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad01439e0 con 0x7f3ad01407e0 2023-08-27T11:28:25.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.998+0000 7f3ab7fff700 1 -- 172.21.15.112:0/2690522699 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4004510 con 0x7f3ad01407e0 2023-08-27T11:28:25.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.998+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/2690522699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad0144a00 con 0x7f3ad01407e0 2023-08-27T11:28:25.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.999+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/2690522699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad0067e90 con 0x7f3ad01407e0 2023-08-27T11:28:25.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:24.999+0000 7f3ab7fff700 1 -- 172.21.15.112:0/2690522699 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f3ac40036a0 con 0x7f3ad01407e0 2023-08-27T11:28:25.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.000+0000 7f3ab7fff700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3abc06a4d0 0x7f3abc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.000+0000 7f3ab7fff700 1 -- 172.21.15.112:0/2690522699 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3ac4088e10 con 0x7f3ad01407e0 2023-08-27T11:28:25.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.000+0000 7f3aceffd700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3abc06a4d0 0x7f3abc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.000+0000 7f3aceffd700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3abc06a4d0 0x7f3abc06c990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0141000 tx=0x7f3ac0009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:25.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.004+0000 7f3ab7fff700 1 -- 172.21.15.112:0/2690522699 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3ac408c030 con 0x7f3ad01407e0 2023-08-27T11:28:25.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.191+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/2690522699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3ad0144310 con 0x7f3ad01407e0 2023-08-27T11:28:25.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.192+0000 7f3ab7fff700 1 -- 172.21.15.112:0/2690522699 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v139) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3ac4057a40 con 0x7f3ad01407e0 2023-08-27T11:28:25.194 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:28:25.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.193+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3abc06a4d0 msgr2=0x7f3abc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.193+0000 7f3ad5ca4700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3abc06a4d0 0x7f3abc06c990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0141000 tx=0x7f3ac0009210 comp rx=0 tx=0).stop 2023-08-27T11:28:25.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.194+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/2690522699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ad01407e0 msgr2=0x7f3ad01444c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.194+0000 7f3ad5ca4700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ad01407e0 0x7f3ad01444c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4010850 tx=0x7f3ac4010b60 comp rx=0 tx=0).stop 2023-08-27T11:28:25.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.194+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/2690522699 shutdown_connections 2023-08-27T11:28:25.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.194+0000 7f3ad5ca4700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3abc06a4d0 0x7f3abc06c990 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.194+0000 7f3ad5ca4700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ad00b0d70 0x7f3ad0144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.194+0000 7f3ad5ca4700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ad01407e0 0x7f3ad01444c0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.194+0000 7f3ad5ca4700 1 --2- 172.21.15.112:0/2690522699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ad0140bc0 0x7f3ad00b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.194+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/2690522699 >> 172.21.15.112:0/2690522699 conn(0x7f3ad00b1e90 msgr2=0x7f3ad01412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:25.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.194+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/2690522699 shutdown_connections 2023-08-27T11:28:25.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.194+0000 7f3ad5ca4700 1 -- 172.21.15.112:0/2690522699 wait complete. 2023-08-27T11:28:25.207 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:28:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.328+0000 7faeba091700 1 Processor -- start 2023-08-27T11:28:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.328+0000 7faeba091700 1 -- start start 2023-08-27T11:28:25.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.329+0000 7faeba091700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faeb41417d0 0x7faeb4072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.329+0000 7faeba091700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb415ca40 con 0x7faeb406e020 2023-08-27T11:28:25.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.329+0000 7faeba091700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb4072540 con 0x7faeb415b880 2023-08-27T11:28:25.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.329+0000 7faeba091700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb40665b0 con 0x7faeb41417d0 2023-08-27T11:28:25.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.329+0000 7faeb2ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faeb41417d0 0x7faeb4072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.329+0000 7faeb37fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7faeb406e020 0x7faeb41410c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58924/0 (socket says 172.21.15.112:58924) 2023-08-27T11:28:25.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.329+0000 7faeb37fe700 1 -- 172.21.15.112:0/260740662 learned_addr learned my addr 172.21.15.112:0/260740662 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:25.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.329+0000 7faeb2ffd700 1 -- 172.21.15.112:0/260740662 >> v1:172.21.15.138:6789/0 conn(0x7faeb415b880 legacy=0x7faeb415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:25.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.329+0000 7faeb2ffd700 1 -- 172.21.15.112:0/260740662 >> v1:172.21.15.112:6789/0 conn(0x7faeb406e020 legacy=0x7faeb41410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.329+0000 7faeb2ffd700 1 -- 172.21.15.112:0/260740662 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb415bdf0 con 0x7faeb41417d0 2023-08-27T11:28:25.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.329+0000 7faeb27fc700 1 -- 172.21.15.112:0/260740662 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745324536 0 0) 0x7faeb415ca40 con 0x7faeb406e020 2023-08-27T11:28:25.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.330+0000 7faeb2ffd700 1 --2- 172.21.15.112:0/260740662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeb41417d0 0x7faeb4072000 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7faea4009840 tx=0x7faea4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a725b70c1d94c61c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:25.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.330+0000 7faeb27fc700 1 -- 172.21.15.112:0/260740662 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4004230 con 0x7faeb41417d0 2023-08-27T11:28:25.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.330+0000 7faeb27fc700 1 -- 172.21.15.112:0/260740662 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea4004390 con 0x7faeb41417d0 2023-08-27T11:28:25.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.330+0000 7faeb27fc700 1 -- 172.21.15.112:0/260740662 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea40036a0 con 0x7faeb41417d0 2023-08-27T11:28:25.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.330+0000 7faeba091700 1 -- 172.21.15.112:0/260740662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeb41417d0 msgr2=0x7faeb4072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.330+0000 7faeba091700 1 --2- 172.21.15.112:0/260740662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeb41417d0 0x7faeb4072000 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7faea4009840 tx=0x7faea4009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:25.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.330+0000 7faeba091700 1 -- 172.21.15.112:0/260740662 shutdown_connections 2023-08-27T11:28:25.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.330+0000 7faeba091700 1 --2- 172.21.15.112:0/260740662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeb41417d0 0x7faeb4072000 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.330+0000 7faeba091700 1 -- 172.21.15.112:0/260740662 >> 172.21.15.112:0/260740662 conn(0x7faeb40cc240 msgr2=0x7faeb40cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:25.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.330+0000 7faeba091700 1 -- 172.21.15.112:0/260740662 shutdown_connections 2023-08-27T11:28:25.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.330+0000 7faeba091700 1 -- 172.21.15.112:0/260740662 wait complete. 2023-08-27T11:28:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.331+0000 7faeba091700 1 Processor -- start 2023-08-27T11:28:25.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.332+0000 7faeba091700 1 -- start start 2023-08-27T11:28:25.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.332+0000 7faeba091700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faeb406e020 0x7faeb41f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.332+0000 7faeba091700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faeb41417d0 0x7faeb4070930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.332+0000 7faeba091700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeb415b880 0x7faeb40cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.332+0000 7faeba091700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb40c2050 con 0x7faeb415b880 2023-08-27T11:28:25.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.332+0000 7faeba091700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4071280 con 0x7faeb406e020 2023-08-27T11:28:25.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.332+0000 7faeba091700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb406f830 con 0x7faeb41417d0 2023-08-27T11:28:25.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.332+0000 7faeb3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeb415b880 0x7faeb40cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.332+0000 7faeb3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeb415b880 0x7faeb40cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36320/0 (socket says 172.21.15.112:36320) 2023-08-27T11:28:25.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.332+0000 7faeb3fff700 1 -- 172.21.15.112:0/2928744719 learned_addr learned my addr 172.21.15.112:0/2928744719 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:25.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7faeb37fe700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faeb406e020 0x7faeb41f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7faeb2ffd700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faeb41417d0 0x7faeb4070930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7faeb3fff700 1 -- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faeb41417d0 msgr2=0x7faeb4070930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7faeb3fff700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faeb41417d0 0x7faeb4070930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7faeb3fff700 1 -- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faeb406e020 msgr2=0x7faeb41f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7faeb3fff700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faeb406e020 0x7faeb41f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7faeb3fff700 1 -- 172.21.15.112:0/2928744719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea40094a0 con 0x7faeb415b880 2023-08-27T11:28:25.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7faeb37fe700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faeb406e020 0x7faeb41f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:25.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7faeb3fff700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeb415b880 0x7faeb40cb9f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7faeac00b9c0 tx=0x7faeac00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:25.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7fae9bfff700 1 -- 172.21.15.112:0/2928744719 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac00c9d0 con 0x7faeb415b880 2023-08-27T11:28:25.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7fae9bfff700 1 -- 172.21.15.112:0/2928744719 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac00cb30 con 0x7faeb415b880 2023-08-27T11:28:25.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7faeba091700 1 -- 172.21.15.112:0/2928744719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb41f64a0 con 0x7faeb415b880 2023-08-27T11:28:25.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7fae9bfff700 1 -- 172.21.15.112:0/2928744719 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac014440 con 0x7faeb415b880 2023-08-27T11:28:25.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.333+0000 7faeba091700 1 -- 172.21.15.112:0/2928744719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb41f6a60 con 0x7faeb415b880 2023-08-27T11:28:25.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.334+0000 7faeba091700 1 -- 172.21.15.112:0/2928744719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb4067e90 con 0x7faeb415b880 2023-08-27T11:28:25.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.334+0000 7fae9bfff700 1 -- 172.21.15.112:0/2928744719 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7faeac006d50 con 0x7faeb415b880 2023-08-27T11:28:25.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.335+0000 7fae9bfff700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faea006a4d0 0x7faea006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.335+0000 7fae9bfff700 1 -- 172.21.15.112:0/2928744719 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faeac0887a0 con 0x7faeb415b880 2023-08-27T11:28:25.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.335+0000 7faeb37fe700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faea006a4d0 0x7faea006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.336+0000 7faeb37fe700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faea006a4d0 0x7faea006c990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7faea8000f60 tx=0x7faea8007ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:25.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.340+0000 7fae9bfff700 1 -- 172.21.15.112:0/2928744719 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7faeac0538b0 con 0x7faeb415b880 2023-08-27T11:28:25.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.526+0000 7faeba091700 1 -- 172.21.15.112:0/2928744719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faeb4067a20 con 0x7faeb415b880 2023-08-27T11:28:25.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.526+0000 7fae9bfff700 1 -- 172.21.15.112:0/2928744719 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7faeac056ed0 con 0x7faeb415b880 2023-08-27T11:28:25.528 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:28:25.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.528+0000 7faeba091700 1 -- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faea006a4d0 msgr2=0x7faea006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.528+0000 7faeba091700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faea006a4d0 0x7faea006c990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7faea8000f60 tx=0x7faea8007ca0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.528+0000 7faeba091700 1 -- 172.21.15.112:0/2928744719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeb415b880 msgr2=0x7faeb40cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.528+0000 7faeba091700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeb415b880 0x7faeb40cb9f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7faeac00b9c0 tx=0x7faeac00bd80 comp rx=0 tx=0).stop 2023-08-27T11:28:25.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.528+0000 7faeba091700 1 -- 172.21.15.112:0/2928744719 shutdown_connections 2023-08-27T11:28:25.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.528+0000 7faeba091700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faea006a4d0 0x7faea006c990 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.528+0000 7faeba091700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeb415b880 0x7faeb40cb9f0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.528+0000 7faeba091700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faeb406e020 0x7faeb41f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.528+0000 7faeba091700 1 --2- 172.21.15.112:0/2928744719 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faeb41417d0 0x7faeb4070930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.528+0000 7faeba091700 1 -- 172.21.15.112:0/2928744719 >> 172.21.15.112:0/2928744719 conn(0x7faeb40cc240 msgr2=0x7faeb415abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:25.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.528+0000 7faeba091700 1 -- 172.21.15.112:0/2928744719 shutdown_connections 2023-08-27T11:28:25.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.528+0000 7faeba091700 1 -- 172.21.15.112:0/2928744719 wait complete. 2023-08-27T11:28:25.542 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:28:25.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.664+0000 7f8aa1966700 1 Processor -- start 2023-08-27T11:28:25.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.664+0000 7f8aa1966700 1 -- start start 2023-08-27T11:28:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.665+0000 7f8aa1966700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8a9c15b000 0x7f8a9c0c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.665+0000 7f8aa1966700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a9c0ce680 0x7f8a9c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.665+0000 7f8aa1966700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a9c15b3e0 con 0x7f8a9c15d920 2023-08-27T11:28:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.665+0000 7f8aa1966700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9c15e940 con 0x7f8a9c15b000 2023-08-27T11:28:25.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.665+0000 7f8aa1966700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c15ddc0 con 0x7f8a9c0ce680 2023-08-27T11:28:25.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.665+0000 7f8a9a7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8a9c15d920 0x7f8a9c160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58940/0 (socket says 172.21.15.112:58940) 2023-08-27T11:28:25.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.665+0000 7f8a9a7fc700 1 -- 172.21.15.112:0/3334097687 learned_addr learned my addr 172.21.15.112:0/3334097687 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:25.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a9b7fe700 1 --2- 172.21.15.112:0/3334097687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a9c0ce680 0x7f8a9c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a9affd700 1 --2- 172.21.15.112:0/3334097687 >> v2:172.21.15.138:3300/0 conn(0x7f8a9c15b000 0x7f8a9c0c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268532728 0 0) 0x7f8a9c15b3e0 con 0x7f8a9c15d920 2023-08-27T11:28:25.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a840035b0 con 0x7f8a9c15d920 2023-08-27T11:28:25.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2204335120 0 0) 0x7f8a840035b0 con 0x7f8a9c15d920 2023-08-27T11:28:25.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a9c0ce680 msgr2=0x7f8a9c0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:25.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 --2- 172.21.15.112:0/3334097687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a9c0ce680 0x7f8a9c0ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 >> v2:172.21.15.138:3300/0 conn(0x7f8a9c15b000 msgr2=0x7f8a9c0c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:25.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 --2- 172.21.15.112:0/3334097687 >> v2:172.21.15.138:3300/0 conn(0x7f8a9c15b000 0x7f8a9c0c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c15e0a0 con 0x7f8a9c15d920 2023-08-27T11:28:25.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f8a8c001670 con 0x7f8a9c15d920 2023-08-27T11:28:25.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 >> v1:172.21.15.112:6789/0 conn(0x7f8a9c15d920 legacy=0x7f8a9c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 --2- 172.21.15.112:0/3334097687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a84003a00 0x7f8a84003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 --2- 172.21.15.112:0/3334097687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a84004850 0x7f8a84004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 --2- 172.21.15.112:0/3334097687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a9c0ce680 0x7f8a84005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c15e940 con 0x7f8a84004850 2023-08-27T11:28:25.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c15ddc0 con 0x7f8a84003a00 2023-08-27T11:28:25.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8400bac0 con 0x7f8a9c0ce680 2023-08-27T11:28:25.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a9a7fc700 1 --2- 172.21.15.112:0/3334097687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a84004850 0x7f8a84004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a9affd700 1 --2- 172.21.15.112:0/3334097687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a84003a00 0x7f8a84003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a9b7fe700 1 --2- 172.21.15.112:0/3334097687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a9c0ce680 0x7f8a84005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a9a7fc700 1 -- 172.21.15.112:0/3334097687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a9c0ce680 msgr2=0x7f8a84005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:25.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a9a7fc700 1 --2- 172.21.15.112:0/3334097687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a9c0ce680 0x7f8a84005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a9a7fc700 1 -- 172.21.15.112:0/3334097687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a84003a00 msgr2=0x7f8a84003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:25.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a9a7fc700 1 --2- 172.21.15.112:0/3334097687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a84003a00 0x7f8a84003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a9a7fc700 1 -- 172.21.15.112:0/3334097687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a840035b0 con 0x7f8a84004850 2023-08-27T11:28:25.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a9affd700 1 --2- 172.21.15.112:0/3334097687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a84003a00 0x7f8a84003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:25.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.666+0000 7f8a9a7fc700 1 --2- 172.21.15.112:0/3334097687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a84004850 0x7f8a84004cd0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00c370 tx=0x7f8a84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37485263fd49a05 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:25.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.667+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c007c60 con 0x7f8a84004850 2023-08-27T11:28:25.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.667+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c005960 con 0x7f8a84004850 2023-08-27T11:28:25.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.667+0000 7f8a99ffb700 1 -- 172.21.15.112:0/3334097687 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c005d10 con 0x7f8a84004850 2023-08-27T11:28:25.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.667+0000 7f8aa1966700 1 -- 172.21.15.112:0/3334097687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a84004850 msgr2=0x7f8a84004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.667+0000 7f8aa1966700 1 --2- 172.21.15.112:0/3334097687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a84004850 0x7f8a84004cd0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00c370 tx=0x7f8a84005880 comp rx=0 tx=0).stop 2023-08-27T11:28:25.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.667+0000 7f8a9b7fe700 1 -- 172.21.15.112:0/3334097687 reap_dead start 2023-08-27T11:28:25.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.667+0000 7f8aa1966700 1 -- 172.21.15.112:0/3334097687 shutdown_connections 2023-08-27T11:28:25.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.667+0000 7f8aa1966700 1 -- 172.21.15.112:0/3334097687 >> 172.21.15.112:0/3334097687 conn(0x7f8a9c0cf750 msgr2=0x7f8a9c0cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:25.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.667+0000 7f8aa1966700 1 -- 172.21.15.112:0/3334097687 shutdown_connections 2023-08-27T11:28:25.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.667+0000 7f8aa1966700 1 -- 172.21.15.112:0/3334097687 wait complete. 2023-08-27T11:28:25.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.668+0000 7f8aa1966700 1 Processor -- start 2023-08-27T11:28:25.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.669+0000 7f8aa1966700 1 -- start start 2023-08-27T11:28:25.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.669+0000 7f8aa1966700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a84004850 0x7f8a9c0c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.669+0000 7f8aa1966700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c0c98b0 0x7f8a9c146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.669+0000 7f8aa1966700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a9c15c470 0x7f8a9c160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.669+0000 7f8aa1966700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c147590 con 0x7f8a9c0c98b0 2023-08-27T11:28:25.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.669+0000 7f8aa1966700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c146910 con 0x7f8a84004850 2023-08-27T11:28:25.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.669+0000 7f8aa1966700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c146bf0 con 0x7f8a9c15c470 2023-08-27T11:28:25.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.669+0000 7f8a9a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c0c98b0 0x7f8a9c146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.669+0000 7f8a9a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c0c98b0 0x7f8a9c146170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36338/0 (socket says 172.21.15.112:36338) 2023-08-27T11:28:25.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.669+0000 7f8a9a7fc700 1 -- 172.21.15.112:0/2935592007 learned_addr learned my addr 172.21.15.112:0/2935592007 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:25.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.670+0000 7f8a9b7fe700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a9c15c470 0x7f8a9c160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.670+0000 7f8a9affd700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a84004850 0x7f8a9c0c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.670+0000 7f8a9a7fc700 1 -- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a9c15c470 msgr2=0x7f8a9c160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.670+0000 7f8a9a7fc700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a9c15c470 0x7f8a9c160fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.670+0000 7f8a9a7fc700 1 -- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a84004850 msgr2=0x7f8a9c0c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.670+0000 7f8a9a7fc700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a84004850 0x7f8a9c0c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.670+0000 7f8a9a7fc700 1 -- 172.21.15.112:0/2935592007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c146ed0 con 0x7f8a9c0c98b0 2023-08-27T11:28:25.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.670+0000 7f8a9a7fc700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c0c98b0 0x7f8a9c146170 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400ba10 tx=0x7f8a8c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:25.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.670+0000 7f8a83fff700 1 -- 172.21.15.112:0/2935592007 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c004870 con 0x7f8a9c0c98b0 2023-08-27T11:28:25.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.670+0000 7f8a83fff700 1 -- 172.21.15.112:0/2935592007 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c0049d0 con 0x7f8a9c0c98b0 2023-08-27T11:28:25.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.670+0000 7f8aa1966700 1 -- 172.21.15.112:0/2935592007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9c1471b0 con 0x7f8a9c0c98b0 2023-08-27T11:28:25.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.670+0000 7f8a83fff700 1 -- 172.21.15.112:0/2935592007 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c004d80 con 0x7f8a9c0c98b0 2023-08-27T11:28:25.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.670+0000 7f8aa1966700 1 -- 172.21.15.112:0/2935592007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9c15cb00 con 0x7f8a9c0c98b0 2023-08-27T11:28:25.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.671+0000 7f8aa1966700 1 -- 172.21.15.112:0/2935592007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9c067a20 con 0x7f8a9c0c98b0 2023-08-27T11:28:25.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.671+0000 7f8a83fff700 1 -- 172.21.15.112:0/2935592007 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f8a8c0070d0 con 0x7f8a9c0c98b0 2023-08-27T11:28:25.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.672+0000 7f8a83fff700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a8806a4d0 0x7f8a8806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:25.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.672+0000 7f8a83fff700 1 -- 172.21.15.112:0/2935592007 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8a8c08a150 con 0x7f8a9c0c98b0 2023-08-27T11:28:25.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.672+0000 7f8a9affd700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a8806a4d0 0x7f8a8806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:25.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.672+0000 7f8a9affd700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a8806a4d0 0x7f8a8806c990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009b10 tx=0x7f8a90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:25.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.676+0000 7f8a83fff700 1 -- 172.21.15.112:0/2935592007 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8a8c0552f0 con 0x7f8a9c0c98b0 2023-08-27T11:28:25.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.863+0000 7f8aa1966700 1 -- 172.21.15.112:0/2935592007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a9c0cffe0 con 0x7f8a9c0c98b0 2023-08-27T11:28:25.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.863+0000 7f8a83fff700 1 -- 172.21.15.112:0/2935592007 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a8c058910 con 0x7f8a9c0c98b0 2023-08-27T11:28:25.865 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:28:25.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.865+0000 7f8aa1966700 1 -- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a8806a4d0 msgr2=0x7f8a8806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.865+0000 7f8aa1966700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a8806a4d0 0x7f8a8806c990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009b10 tx=0x7f8a90008040 comp rx=0 tx=0).stop 2023-08-27T11:28:25.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.865+0000 7f8aa1966700 1 -- 172.21.15.112:0/2935592007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c0c98b0 msgr2=0x7f8a9c146170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:25.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.865+0000 7f8aa1966700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c0c98b0 0x7f8a9c146170 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400ba10 tx=0x7f8a8c000fe0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.865+0000 7f8aa1966700 1 -- 172.21.15.112:0/2935592007 shutdown_connections 2023-08-27T11:28:25.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.865+0000 7f8aa1966700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8a8806a4d0 0x7f8a8806c990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.865+0000 7f8aa1966700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a9c15c470 0x7f8a9c160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.865+0000 7f8aa1966700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a84004850 0x7f8a9c0c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.865+0000 7f8aa1966700 1 --2- 172.21.15.112:0/2935592007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c0c98b0 0x7f8a9c146170 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:25.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.866+0000 7f8aa1966700 1 -- 172.21.15.112:0/2935592007 >> 172.21.15.112:0/2935592007 conn(0x7f8a9c0cf750 msgr2=0x7f8a9c0ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:25.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.866+0000 7f8aa1966700 1 -- 172.21.15.112:0/2935592007 shutdown_connections 2023-08-27T11:28:25.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:25.866+0000 7f8aa1966700 1 -- 172.21.15.112:0/2935592007 wait complete. 2023-08-27T11:28:25.879 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:28:26.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.001+0000 7fb04b71f700 1 Processor -- start 2023-08-27T11:28:26.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.001+0000 7fb04b71f700 1 -- start start 2023-08-27T11:28:26.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb04b71f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb0440b3440 0x7fb04415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb04b71f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb04415e6e0 0x7fb04406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb04b71f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0440b5070 con 0x7fb044146c20 2023-08-27T11:28:26.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb04b71f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb044147000 con 0x7fb04415e6e0 2023-08-27T11:28:26.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb04b71f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb044161740 con 0x7fb0440b3440 2023-08-27T11:28:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb049cbc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb04415e6e0 0x7fb04406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb0494bb700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb044146c20 0x7fb044160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58958/0 (socket says 172.21.15.112:58958) 2023-08-27T11:28:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb0494bb700 1 -- 172.21.15.112:0/1093081080 learned_addr learned my addr 172.21.15.112:0/1093081080 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb048cba700 1 --2- 172.21.15.112:0/1093081080 >> v2:172.21.15.138:3300/0 conn(0x7fb0440b3440 0x7fb04415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb049cbc700 1 -- 172.21.15.112:0/1093081080 >> v2:172.21.15.138:3300/0 conn(0x7fb0440b3440 msgr2=0x7fb04415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:26.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb049cbc700 1 --2- 172.21.15.112:0/1093081080 >> v2:172.21.15.138:3300/0 conn(0x7fb0440b3440 0x7fb04415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb049cbc700 1 -- 172.21.15.112:0/1093081080 >> v1:172.21.15.112:6789/0 conn(0x7fb044146c20 legacy=0x7fb044160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb049cbc700 1 -- 172.21.15.112:0/1093081080 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb044161a20 con 0x7fb04415e6e0 2023-08-27T11:28:26.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb03bfff700 1 -- 172.21.15.112:0/1093081080 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823806466 0 0) 0x7fb040001110 con 0x7fb044146c20 2023-08-27T11:28:26.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb049cbc700 1 --2- 172.21.15.112:0/1093081080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04415e6e0 0x7fb04406f790 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb0440b3f00 tx=0x7fb03c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6568346da31bdb0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb03bfff700 1 -- 172.21.15.112:0/1093081080 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00e070 con 0x7fb04415e6e0 2023-08-27T11:28:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb03bfff700 1 -- 172.21.15.112:0/1093081080 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c00bbd0 con 0x7fb04415e6e0 2023-08-27T11:28:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb03bfff700 1 -- 172.21.15.112:0/1093081080 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c012430 con 0x7fb04415e6e0 2023-08-27T11:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb04b71f700 1 -- 172.21.15.112:0/1093081080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04415e6e0 msgr2=0x7fb04406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb04b71f700 1 --2- 172.21.15.112:0/1093081080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04415e6e0 0x7fb04406f790 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb0440b3f00 tx=0x7fb03c00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb04b71f700 1 -- 172.21.15.112:0/1093081080 shutdown_connections 2023-08-27T11:28:26.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb04b71f700 1 --2- 172.21.15.112:0/1093081080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04415e6e0 0x7fb04406f790 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb04b71f700 1 --2- 172.21.15.112:0/1093081080 >> v2:172.21.15.138:3300/0 conn(0x7fb0440b3440 0x7fb04415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb04b71f700 1 -- 172.21.15.112:0/1093081080 >> 172.21.15.112:0/1093081080 conn(0x7fb0440b1e90 msgr2=0x7fb0440b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:26.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb04b71f700 1 -- 172.21.15.112:0/1093081080 shutdown_connections 2023-08-27T11:28:26.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.003+0000 7fb04b71f700 1 -- 172.21.15.112:0/1093081080 wait complete. 2023-08-27T11:28:26.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.004+0000 7fb04b71f700 1 Processor -- start 2023-08-27T11:28:26.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.004+0000 7fb04b71f700 1 -- start start 2023-08-27T11:28:26.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb04b71f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0440b3440 0x7fb04415c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb04b71f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb044146c20 0x7fb04413db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb04b71f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb04415e6e0 0x7fb0440ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb04b71f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440adc30 con 0x7fb0440b3440 2023-08-27T11:28:26.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb04b71f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04415f6a0 con 0x7fb044146c20 2023-08-27T11:28:26.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb04b71f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04415f980 con 0x7fb04415e6e0 2023-08-27T11:28:26.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb0494bb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0440b3440 0x7fb04415c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb0494bb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0440b3440 0x7fb04415c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36350/0 (socket says 172.21.15.112:36350) 2023-08-27T11:28:26.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb0494bb700 1 -- 172.21.15.112:0/2946087570 learned_addr learned my addr 172.21.15.112:0/2946087570 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:26.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb048cba700 1 --2- 172.21.15.112:0/2946087570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb044146c20 0x7fb04413db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb0494bb700 1 -- 172.21.15.112:0/2946087570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb04415e6e0 msgr2=0x7fb0440ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb0494bb700 1 --2- 172.21.15.112:0/2946087570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb04415e6e0 0x7fb0440ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb0494bb700 1 -- 172.21.15.112:0/2946087570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb044146c20 msgr2=0x7fb04413db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb0494bb700 1 --2- 172.21.15.112:0/2946087570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb044146c20 0x7fb04413db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb0494bb700 1 -- 172.21.15.112:0/2946087570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c009a70 con 0x7fb0440b3440 2023-08-27T11:28:26.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.005+0000 7fb0494bb700 1 --2- 172.21.15.112:0/2946087570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0440b3440 0x7fb04415c790 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fb04000c730 tx=0x7fb04000caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:26.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.006+0000 7fb039ffb700 1 -- 172.21.15.112:0/2946087570 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040004c80 con 0x7fb0440b3440 2023-08-27T11:28:26.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.006+0000 7fb039ffb700 1 -- 172.21.15.112:0/2946087570 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb040004e00 con 0x7fb0440b3440 2023-08-27T11:28:26.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.006+0000 7fb039ffb700 1 -- 172.21.15.112:0/2946087570 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040007310 con 0x7fb0440b3440 2023-08-27T11:28:26.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.006+0000 7fb04b71f700 1 -- 172.21.15.112:0/2946087570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04415fc60 con 0x7fb0440b3440 2023-08-27T11:28:26.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.006+0000 7fb04b71f700 1 -- 172.21.15.112:0/2946087570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb044160220 con 0x7fb0440b3440 2023-08-27T11:28:26.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.007+0000 7fb04b71f700 1 -- 172.21.15.112:0/2946087570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb044160440 con 0x7fb0440b3440 2023-08-27T11:28:26.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.007+0000 7fb039ffb700 1 -- 172.21.15.112:0/2946087570 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fb040007470 con 0x7fb0440b3440 2023-08-27T11:28:26.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.007+0000 7fb039ffb700 1 --2- 172.21.15.112:0/2946087570 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb02806a4d0 0x7fb02806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.008+0000 7fb039ffb700 1 -- 172.21.15.112:0/2946087570 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb040088760 con 0x7fb0440b3440 2023-08-27T11:28:26.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.008+0000 7fb048cba700 1 --2- 172.21.15.112:0/2946087570 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb02806a4d0 0x7fb02806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.008+0000 7fb048cba700 1 --2- 172.21.15.112:0/2946087570 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb02806a4d0 0x7fb02806c990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb0440b4b70 tx=0x7fb034008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:26.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.013+0000 7fb039ffb700 1 -- 172.21.15.112:0/2946087570 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb040053980 con 0x7fb0440b3440 2023-08-27T11:28:26.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.201+0000 7fb04b71f700 1 -- 172.21.15.112:0/2946087570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb044067e90 con 0x7fb0440b3440 2023-08-27T11:28:26.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.201+0000 7fb039ffb700 1 -- 172.21.15.112:0/2946087570 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb040056fa0 con 0x7fb0440b3440 2023-08-27T11:28:26.203 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:28:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.203+0000 7fb04b71f700 1 -- 172.21.15.112:0/2946087570 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb02806a4d0 msgr2=0x7fb02806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.203+0000 7fb04b71f700 1 --2- 172.21.15.112:0/2946087570 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb02806a4d0 0x7fb02806c990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb0440b4b70 tx=0x7fb034008040 comp rx=0 tx=0).stop 2023-08-27T11:28:26.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.203+0000 7fb04b71f700 1 -- 172.21.15.112:0/2946087570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0440b3440 msgr2=0x7fb04415c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.203+0000 7fb04b71f700 1 --2- 172.21.15.112:0/2946087570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0440b3440 0x7fb04415c790 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fb04000c730 tx=0x7fb04000caf0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.203+0000 7fb04b71f700 1 -- 172.21.15.112:0/2946087570 shutdown_connections 2023-08-27T11:28:26.205 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.203+0000 7fb04b71f700 1 --2- 172.21.15.112:0/2946087570 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb02806a4d0 0x7fb02806c990 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.203+0000 7fb04b71f700 1 --2- 172.21.15.112:0/2946087570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb04415e6e0 0x7fb0440ad6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.203+0000 7fb04b71f700 1 --2- 172.21.15.112:0/2946087570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0440b3440 0x7fb04415c790 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.203+0000 7fb04b71f700 1 --2- 172.21.15.112:0/2946087570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb044146c20 0x7fb04413db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.203+0000 7fb04b71f700 1 -- 172.21.15.112:0/2946087570 >> 172.21.15.112:0/2946087570 conn(0x7fb0440b1e90 msgr2=0x7fb04406ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:26.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.204+0000 7fb04b71f700 1 -- 172.21.15.112:0/2946087570 shutdown_connections 2023-08-27T11:28:26.206 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.204+0000 7fb04b71f700 1 -- 172.21.15.112:0/2946087570 wait complete. 2023-08-27T11:28:26.216 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:28:26.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.339+0000 7f66615c6700 1 Processor -- start 2023-08-27T11:28:26.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.339+0000 7f66615c6700 1 -- start start 2023-08-27T11:28:26.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.339+0000 7f66615c6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f665c0b5470 0x7f665c0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.339+0000 7f66615c6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c1619b0 0x7f665c0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.339+0000 7f66615c6700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c161e40 con 0x7f665c146be0 2023-08-27T11:28:26.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.339+0000 7f66615c6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f665c0b3980 con 0x7f665c0b5470 2023-08-27T11:28:26.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.339+0000 7f66615c6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0b3b70 con 0x7f665c1619b0 2023-08-27T11:28:26.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.339+0000 7f665a7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f665c0b5470 0x7f665c0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.340+0000 7f665a7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f665c0b5470 0x7f665c0b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36360/0 (socket says 172.21.15.112:36360) 2023-08-27T11:28:26.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.340+0000 7f665a7fc700 1 -- 172.21.15.112:0/1530304995 learned_addr learned my addr 172.21.15.112:0/1530304995 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:26.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.340+0000 7f665b7fe700 1 --2- 172.21.15.112:0/1530304995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c1619b0 0x7f665c0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.340+0000 7f665a7fc700 1 -- 172.21.15.112:0/1530304995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c1619b0 msgr2=0x7f665c0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:26.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.340+0000 7f665a7fc700 1 --2- 172.21.15.112:0/1530304995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c1619b0 0x7f665c0cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.340+0000 7f665a7fc700 1 -- 172.21.15.112:0/1530304995 >> v1:172.21.15.138:6789/0 conn(0x7f665c146be0 legacy=0x7f665c0b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.340+0000 7f665a7fc700 1 -- 172.21.15.112:0/1530304995 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c0b3e50 con 0x7f665c0b5470 2023-08-27T11:28:26.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.340+0000 7f665a7fc700 1 --2- 172.21.15.112:0/1530304995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c0b5470 0x7f665c0b3440 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f66500098f0 tx=0x7f6650009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66a58b806268514 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:26.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.340+0000 7f6659ffb700 1 -- 172.21.15.112:0/1530304995 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6650011830 con 0x7f665c0b5470 2023-08-27T11:28:26.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.340+0000 7f6659ffb700 1 -- 172.21.15.112:0/1530304995 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6650011990 con 0x7f665c0b5470 2023-08-27T11:28:26.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.340+0000 7f6659ffb700 1 -- 172.21.15.112:0/1530304995 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6650011d20 con 0x7f665c0b5470 2023-08-27T11:28:26.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.341+0000 7f66615c6700 1 -- 172.21.15.112:0/1530304995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c0b5470 msgr2=0x7f665c0b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.341+0000 7f66615c6700 1 --2- 172.21.15.112:0/1530304995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c0b5470 0x7f665c0b3440 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f66500098f0 tx=0x7f6650009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:26.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.341+0000 7f66615c6700 1 -- 172.21.15.112:0/1530304995 shutdown_connections 2023-08-27T11:28:26.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.341+0000 7f66615c6700 1 --2- 172.21.15.112:0/1530304995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c1619b0 0x7f665c0cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.341+0000 7f66615c6700 1 --2- 172.21.15.112:0/1530304995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c0b5470 0x7f665c0b3440 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.341+0000 7f66615c6700 1 -- 172.21.15.112:0/1530304995 >> 172.21.15.112:0/1530304995 conn(0x7f665c06c880 msgr2=0x7f665c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:26.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.341+0000 7f66615c6700 1 -- 172.21.15.112:0/1530304995 shutdown_connections 2023-08-27T11:28:26.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.341+0000 7f66615c6700 1 -- 172.21.15.112:0/1530304995 wait complete. 2023-08-27T11:28:26.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.342+0000 7f66615c6700 1 Processor -- start 2023-08-27T11:28:26.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.342+0000 7f66615c6700 1 -- start start 2023-08-27T11:28:26.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f66615c6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c0b5470 0x7f665c0b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f66615c6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c146be0 0x7f665c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f66615c6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c1619b0 0x7f665c0cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f66615c6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0b0bb0 con 0x7f665c0b5470 2023-08-27T11:28:26.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f66615c6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c1f72a0 con 0x7f665c146be0 2023-08-27T11:28:26.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f66615c6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c1f6720 con 0x7f665c1619b0 2023-08-27T11:28:26.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f665affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c0b5470 0x7f665c0b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f665affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c0b5470 0x7f665c0b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36364/0 (socket says 172.21.15.112:36364) 2023-08-27T11:28:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f665affd700 1 -- 172.21.15.112:0/1152828807 learned_addr learned my addr 172.21.15.112:0/1152828807 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f665b7fe700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c1619b0 0x7f665c0cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f665a7fc700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c146be0 0x7f665c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f665affd700 1 -- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c1619b0 msgr2=0x7f665c0cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f665affd700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c1619b0 0x7f665c0cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f665affd700 1 -- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c146be0 msgr2=0x7f665c0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f665affd700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c146be0 0x7f665c0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f665affd700 1 -- 172.21.15.112:0/1152828807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6650009550 con 0x7f665c0b5470 2023-08-27T11:28:26.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f665b7fe700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c1619b0 0x7f665c0cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:26.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.343+0000 7f665affd700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c0b5470 0x7f665c0b1f80 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c4e0 tx=0x7f664c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:26.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.344+0000 7f66437fe700 1 -- 172.21.15.112:0/1152828807 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c0052d0 con 0x7f665c0b5470 2023-08-27T11:28:26.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.344+0000 7f66437fe700 1 -- 172.21.15.112:0/1152828807 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c005430 con 0x7f665c0b5470 2023-08-27T11:28:26.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.344+0000 7f66615c6700 1 -- 172.21.15.112:0/1152828807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c1f6a00 con 0x7f665c0b5470 2023-08-27T11:28:26.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.344+0000 7f66437fe700 1 -- 172.21.15.112:0/1152828807 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c0038c0 con 0x7f665c0b5470 2023-08-27T11:28:26.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.344+0000 7f66615c6700 1 -- 172.21.15.112:0/1152828807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c1f6fc0 con 0x7f665c0b5470 2023-08-27T11:28:26.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.344+0000 7f66615c6700 1 -- 172.21.15.112:0/1152828807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665c067a20 con 0x7f665c0b5470 2023-08-27T11:28:26.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.345+0000 7f66437fe700 1 -- 172.21.15.112:0/1152828807 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f664c003a20 con 0x7f665c0b5470 2023-08-27T11:28:26.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.345+0000 7f66437fe700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664806a4d0 0x7f664806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.345+0000 7f66437fe700 1 -- 172.21.15.112:0/1152828807 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f664c0881b0 con 0x7f665c0b5470 2023-08-27T11:28:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.346+0000 7f665a7fc700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664806a4d0 0x7f664806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.346+0000 7f665a7fc700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664806a4d0 0x7f664806c990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6650009c40 tx=0x7f6650008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.350+0000 7f66437fe700 1 -- 172.21.15.112:0/1152828807 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f664c053290 con 0x7f665c0b5470 2023-08-27T11:28:26.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.554+0000 7f66615c6700 1 -- 172.21.15.112:0/1152828807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f665c0b5250 con 0x7f665c0b5470 2023-08-27T11:28:26.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.554+0000 7f66437fe700 1 -- 172.21.15.112:0/1152828807 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v139) v1 ==== 80+0+0 (secure 0 0 0) 0x7f664c0568b0 con 0x7f665c0b5470 2023-08-27T11:28:26.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.556+0000 7f66615c6700 1 -- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664806a4d0 msgr2=0x7f664806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.556+0000 7f66615c6700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664806a4d0 0x7f664806c990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6650009c40 tx=0x7f6650008030 comp rx=0 tx=0).stop 2023-08-27T11:28:26.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.556+0000 7f66615c6700 1 -- 172.21.15.112:0/1152828807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c0b5470 msgr2=0x7f665c0b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.556+0000 7f66615c6700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c0b5470 0x7f665c0b1f80 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c4e0 tx=0x7f664c00c8a0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.556+0000 7f66615c6700 1 -- 172.21.15.112:0/1152828807 shutdown_connections 2023-08-27T11:28:26.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.556+0000 7f66615c6700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f664806a4d0 0x7f664806c990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.556+0000 7f66615c6700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f665c1619b0 0x7f665c0cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.556+0000 7f66615c6700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f665c0b5470 0x7f665c0b1f80 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.556+0000 7f66615c6700 1 --2- 172.21.15.112:0/1152828807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c146be0 0x7f665c0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.556+0000 7f66615c6700 1 -- 172.21.15.112:0/1152828807 >> 172.21.15.112:0/1152828807 conn(0x7f665c06c880 msgr2=0x7f665c145050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:26.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.556+0000 7f66615c6700 1 -- 172.21.15.112:0/1152828807 shutdown_connections 2023-08-27T11:28:26.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.556+0000 7f66615c6700 1 -- 172.21.15.112:0/1152828807 wait complete. 2023-08-27T11:28:26.560 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:28:26.569 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:28:26.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:26.570 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:26.614 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:28:26.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:26.616 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:28:26.671 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:28:26.672 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:26.693 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:28:26.694 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:26.754 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.874+0000 7fe0ba40e700 1 Processor -- start 2023-08-27T11:28:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.874+0000 7fe0ba40e700 1 -- start start 2023-08-27T11:28:26.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.874+0000 7fe0ba40e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe0b406adb0 0x7fe0b406b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.874+0000 7fe0ba40e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe0b40cdbd0 0x7fe0b40ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.874+0000 7fe0ba40e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b415eb60 con 0x7fe0b4151060 2023-08-27T11:28:26.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.874+0000 7fe0ba40e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b406b770 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.874+0000 7fe0ba40e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b40b02f0 con 0x7fe0b406adb0 2023-08-27T11:28:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.875+0000 7fe0b89ab700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe0b40cdbd0 0x7fe0b40ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.875+0000 7fe0b3fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe0b4151060 0x7fe0b415e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58986/0 (socket says 172.21.15.112:58986) 2023-08-27T11:28:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.875+0000 7fe0b3fff700 1 -- 172.21.15.112:0/3599305356 learned_addr learned my addr 172.21.15.112:0/3599305356 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.875+0000 7fe0b89ab700 1 -- 172.21.15.112:0/3599305356 >> v2:172.21.15.138:3300/0 conn(0x7fe0b406adb0 msgr2=0x7fe0b406b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.875+0000 7fe0b89ab700 1 --2- 172.21.15.112:0/3599305356 >> v2:172.21.15.138:3300/0 conn(0x7fe0b406adb0 0x7fe0b406b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.875+0000 7fe0b89ab700 1 -- 172.21.15.112:0/3599305356 >> v1:172.21.15.112:6789/0 conn(0x7fe0b4151060 legacy=0x7fe0b415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.875+0000 7fe0b89ab700 1 -- 172.21.15.112:0/3599305356 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b40b05d0 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.875+0000 7fe0b2ffd700 1 -- 172.21.15.112:0/3599305356 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084707577 0 0) 0x7fe0b415eb60 con 0x7fe0b4151060 2023-08-27T11:28:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.875+0000 7fe0b89ab700 1 --2- 172.21.15.112:0/3599305356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b40cdbd0 0x7fe0b40ce020 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fe0b406ccb0 tx=0x7fe0ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae535b0c8bb2771e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:26.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.875+0000 7fe0b2ffd700 1 -- 172.21.15.112:0/3599305356 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00e070 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.875+0000 7fe0b2ffd700 1 -- 172.21.15.112:0/3599305356 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac00bbd0 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.875+0000 7fe0b2ffd700 1 -- 172.21.15.112:0/3599305356 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac012430 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.876+0000 7fe0ba40e700 1 -- 172.21.15.112:0/3599305356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b40cdbd0 msgr2=0x7fe0b40ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.876+0000 7fe0ba40e700 1 --2- 172.21.15.112:0/3599305356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b40cdbd0 0x7fe0b40ce020 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fe0b406ccb0 tx=0x7fe0ac00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:26.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.876+0000 7fe0ba40e700 1 -- 172.21.15.112:0/3599305356 shutdown_connections 2023-08-27T11:28:26.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.876+0000 7fe0ba40e700 1 --2- 172.21.15.112:0/3599305356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b40cdbd0 0x7fe0b40ce020 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.876+0000 7fe0ba40e700 1 --2- 172.21.15.112:0/3599305356 >> v2:172.21.15.138:3300/0 conn(0x7fe0b406adb0 0x7fe0b406b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.876+0000 7fe0ba40e700 1 -- 172.21.15.112:0/3599305356 >> 172.21.15.112:0/3599305356 conn(0x7fe0b40d0000 msgr2=0x7fe0b40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:26.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.876+0000 7fe0ba40e700 1 -- 172.21.15.112:0/3599305356 shutdown_connections 2023-08-27T11:28:26.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.876+0000 7fe0ba40e700 1 -- 172.21.15.112:0/3599305356 wait complete. 2023-08-27T11:28:26.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.877+0000 7fe0ba40e700 1 Processor -- start 2023-08-27T11:28:26.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.877+0000 7fe0ba40e700 1 -- start start 2023-08-27T11:28:26.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0ba40e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0b406adb0 0x7fe0b414e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0ba40e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b40cdbd0 0x7fe0b4153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0ba40e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0b4151060 0x7fe0b40c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0ba40e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b4152580 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0ba40e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b4152860 con 0x7fe0b4151060 2023-08-27T11:28:26.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0ba40e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b4152b40 con 0x7fe0b406adb0 2023-08-27T11:28:26.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0b37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b40cdbd0 0x7fe0b4153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0b37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b40cdbd0 0x7fe0b4153620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36380/0 (socket says 172.21.15.112:36380) 2023-08-27T11:28:26.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0b37fe700 1 -- 172.21.15.112:0/282232847 learned_addr learned my addr 172.21.15.112:0/282232847 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:26.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0b3fff700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0b406adb0 0x7fe0b414e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0b37fe700 1 -- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0b406adb0 msgr2=0x7fe0b414e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0b89ab700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0b4151060 0x7fe0b40c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0b37fe700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0b406adb0 0x7fe0b414e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0b37fe700 1 -- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0b4151060 msgr2=0x7fe0b40c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:26.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0b37fe700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0b4151060 0x7fe0b40c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0b37fe700 1 -- 172.21.15.112:0/282232847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac009a70 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0b37fe700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b40cdbd0 0x7fe0b4153620 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800b4a0 tx=0x7fe0a800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:26.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.878+0000 7fe0b0ff9700 1 -- 172.21.15.112:0/282232847 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8011830 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.879+0000 7fe0b0ff9700 1 -- 172.21.15.112:0/282232847 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a8011990 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.879+0000 7fe0b0ff9700 1 -- 172.21.15.112:0/282232847 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8011d40 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.879+0000 7fe0ba40e700 1 -- 172.21.15.112:0/282232847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b4152e20 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.879+0000 7fe0ba40e700 1 -- 172.21.15.112:0/282232847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b414ece0 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.879+0000 7fe0ba40e700 1 -- 172.21.15.112:0/282232847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b4153340 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.880+0000 7fe0b0ff9700 1 -- 172.21.15.112:0/282232847 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe0a801a400 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.880+0000 7fe0b0ff9700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0a006a4d0 0x7fe0a006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:26.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.880+0000 7fe0b0ff9700 1 -- 172.21.15.112:0/282232847 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe0a8088d00 con 0x7fe0b40cdbd0 2023-08-27T11:28:26.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.880+0000 7fe0b3fff700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0a006a4d0 0x7fe0a006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:26.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.881+0000 7fe0b3fff700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0a006a4d0 0x7fe0a006c990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4000f60 tx=0x7fe0a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:26.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:26.884+0000 7fe0b0ff9700 1 -- 172.21.15.112:0/282232847 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe0b4153340 con 0x7fe0b40cdbd0 2023-08-27T11:28:27.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.073+0000 7fe0ba40e700 1 -- 172.21.15.112:0/282232847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0b4153340 con 0x7fe0b40cdbd0 2023-08-27T11:28:27.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.074+0000 7fe0b0ff9700 1 -- 172.21.15.112:0/282232847 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+18263 (secure 0 0 0) 0x7fe0b4153340 con 0x7fe0b40cdbd0 2023-08-27T11:28:27.075 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:27.075 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":139,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:15.631983+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:12.176044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:12.992996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.076+0000 7fe0ba40e700 1 -- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0a006a4d0 msgr2=0x7fe0a006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.077+0000 7fe0ba40e700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0a006a4d0 0x7fe0a006c990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4000f60 tx=0x7fe0a4009040 comp rx=0 tx=0).stop 2023-08-27T11:28:27.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.077+0000 7fe0ba40e700 1 -- 172.21.15.112:0/282232847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b40cdbd0 msgr2=0x7fe0b4153620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.077+0000 7fe0ba40e700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b40cdbd0 0x7fe0b4153620 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800b4a0 tx=0x7fe0a800b860 comp rx=0 tx=0).stop 2023-08-27T11:28:27.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.077+0000 7fe0ba40e700 1 -- 172.21.15.112:0/282232847 shutdown_connections 2023-08-27T11:28:27.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.077+0000 7fe0ba40e700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0a006a4d0 0x7fe0a006c990 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.077+0000 7fe0ba40e700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0b4151060 0x7fe0b40c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.077+0000 7fe0ba40e700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0b406adb0 0x7fe0b414e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.077+0000 7fe0ba40e700 1 --2- 172.21.15.112:0/282232847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b40cdbd0 0x7fe0b4153620 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.077+0000 7fe0ba40e700 1 -- 172.21.15.112:0/282232847 >> 172.21.15.112:0/282232847 conn(0x7fe0b40d0000 msgr2=0x7fe0b40cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:27.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.077+0000 7fe0ba40e700 1 -- 172.21.15.112:0/282232847 shutdown_connections 2023-08-27T11:28:27.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.077+0000 7fe0ba40e700 1 -- 172.21.15.112:0/282232847 wait complete. 2023-08-27T11:28:27.089 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:28:12.176044+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:28:12.992996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:28:27.090 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:28:27.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.212+0000 7f7e08516700 1 Processor -- start 2023-08-27T11:28:27.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.212+0000 7f7e08516700 1 -- start start 2023-08-27T11:28:27.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.213+0000 7f7e08516700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0015af90 0x7f7e0015e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.213+0000 7f7e08516700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7e00071360 0x7f7e000717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.213+0000 7f7e08516700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e00071d10 con 0x7f7e000d1a60 2023-08-27T11:28:27.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.213+0000 7f7e08516700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0015b370 con 0x7f7e00071360 2023-08-27T11:28:27.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.213+0000 7f7e08516700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e000b6e70 con 0x7f7e0015af90 2023-08-27T11:28:27.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e062b2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0015af90 0x7f7e0015e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e06ab3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7e00071360 0x7f7e000717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e06ab3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7e00071360 0x7f7e000717d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:36816/0 (socket says 172.21.15.112:36816) 2023-08-27T11:28:27.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e062b2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0015af90 0x7f7e0015e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:48040/0 (socket says 172.21.15.112:48040) 2023-08-27T11:28:27.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e05ab1700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f7e000d1a60 0x7f7e000b6680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:38292/0 (socket says 172.21.15.112:38292) 2023-08-27T11:28:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e06ab3700 1 -- 172.21.15.112:0/2488733564 learned_addr learned my addr 172.21.15.112:0/2488733564 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e062b2700 1 -- 172.21.15.112:0/2488733564 >> v2:172.21.15.138:3300/0 conn(0x7f7e00071360 msgr2=0x7f7e000717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e062b2700 1 --2- 172.21.15.112:0/2488733564 >> v2:172.21.15.138:3300/0 conn(0x7f7e00071360 0x7f7e000717d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e062b2700 1 -- 172.21.15.112:0/2488733564 >> v1:172.21.15.138:6789/0 conn(0x7f7e000d1a60 legacy=0x7f7e000b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e062b2700 1 -- 172.21.15.112:0/2488733564 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e000b7150 con 0x7f7e0015af90 2023-08-27T11:28:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e06ab3700 1 --2- 172.21.15.112:0/2488733564 >> v2:172.21.15.138:3300/0 conn(0x7f7e00071360 0x7f7e000717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e062b2700 1 --2- 172.21.15.112:0/2488733564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0015af90 0x7f7e0015e470 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009840 tx=0x7f7df0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2997f24546e604e8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e052b0700 1 -- 172.21.15.112:0/2488733564 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df0011830 con 0x7f7e0015af90 2023-08-27T11:28:27.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e052b0700 1 -- 172.21.15.112:0/2488733564 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df0011990 con 0x7f7e0015af90 2023-08-27T11:28:27.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.214+0000 7f7e052b0700 1 -- 172.21.15.112:0/2488733564 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df0011d20 con 0x7f7e0015af90 2023-08-27T11:28:27.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.215+0000 7f7e08516700 1 -- 172.21.15.112:0/2488733564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0015af90 msgr2=0x7f7e0015e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.215+0000 7f7e08516700 1 --2- 172.21.15.112:0/2488733564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0015af90 0x7f7e0015e470 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009840 tx=0x7f7df0009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:27.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.215+0000 7f7e08516700 1 -- 172.21.15.112:0/2488733564 shutdown_connections 2023-08-27T11:28:27.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.215+0000 7f7e08516700 1 --2- 172.21.15.112:0/2488733564 >> v2:172.21.15.138:3300/0 conn(0x7f7e00071360 0x7f7e000717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.215+0000 7f7e08516700 1 --2- 172.21.15.112:0/2488733564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0015af90 0x7f7e0015e470 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.215+0000 7f7e08516700 1 -- 172.21.15.112:0/2488733564 >> 172.21.15.112:0/2488733564 conn(0x7f7e00066010 msgr2=0x7f7e000663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:27.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.215+0000 7f7e08516700 1 -- 172.21.15.112:0/2488733564 shutdown_connections 2023-08-27T11:28:27.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.215+0000 7f7e08516700 1 -- 172.21.15.112:0/2488733564 wait complete. 2023-08-27T11:28:27.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.216+0000 7f7e08516700 1 Processor -- start 2023-08-27T11:28:27.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.216+0000 7f7e08516700 1 -- start start 2023-08-27T11:28:27.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e08516700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e00071360 0x7f7e0014b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e08516700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e000d1a60 0x7f7e000cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e08516700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e0015af90 0x7f7e0015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e08516700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0014b5e0 con 0x7f7e000d1a60 2023-08-27T11:28:27.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e08516700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e000cc180 con 0x7f7e0015af90 2023-08-27T11:28:27.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e08516700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e000b4870 con 0x7f7e00071360 2023-08-27T11:28:27.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e05ab1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e000d1a60 0x7f7e000cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e05ab1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e000d1a60 0x7f7e000cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36396/0 (socket says 172.21.15.112:36396) 2023-08-27T11:28:27.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e05ab1700 1 -- 172.21.15.112:0/4091684006 learned_addr learned my addr 172.21.15.112:0/4091684006 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:27.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e062b2700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e00071360 0x7f7e0014b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e06ab3700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e0015af90 0x7f7e0015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e05ab1700 1 -- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e00071360 msgr2=0x7f7e0014b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e05ab1700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e00071360 0x7f7e0014b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e05ab1700 1 -- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e0015af90 msgr2=0x7f7e0015c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e05ab1700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e0015af90 0x7f7e0015c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e05ab1700 1 -- 172.21.15.112:0/4091684006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df8009b30 con 0x7f7e000d1a60 2023-08-27T11:28:27.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.217+0000 7f7e06ab3700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e0015af90 0x7f7e0015c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:27.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.218+0000 7f7e05ab1700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e000d1a60 0x7f7e000cb670 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00c430 tx=0x7f7dfc00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:27.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.218+0000 7f7df6ffd700 1 -- 172.21.15.112:0/4091684006 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc005cb0 con 0x7f7e000d1a60 2023-08-27T11:28:27.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.218+0000 7f7df6ffd700 1 -- 172.21.15.112:0/4091684006 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc005e10 con 0x7f7e000d1a60 2023-08-27T11:28:27.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.218+0000 7f7e08516700 1 -- 172.21.15.112:0/4091684006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df00094a0 con 0x7f7e000d1a60 2023-08-27T11:28:27.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.218+0000 7f7df6ffd700 1 -- 172.21.15.112:0/4091684006 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0072d0 con 0x7f7e000d1a60 2023-08-27T11:28:27.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.218+0000 7f7e08516700 1 -- 172.21.15.112:0/4091684006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e000b4e30 con 0x7f7e000d1a60 2023-08-27T11:28:27.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.219+0000 7f7e08516700 1 -- 172.21.15.112:0/4091684006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e00067a20 con 0x7f7e000d1a60 2023-08-27T11:28:27.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.219+0000 7f7df6ffd700 1 -- 172.21.15.112:0/4091684006 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f7dfc007430 con 0x7f7e000d1a60 2023-08-27T11:28:27.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.220+0000 7f7df6ffd700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7de406a4d0 0x7f7de406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.220+0000 7f7df6ffd700 1 -- 172.21.15.112:0/4091684006 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7dfc087e60 con 0x7f7e000d1a60 2023-08-27T11:28:27.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.220+0000 7f7e062b2700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7de406a4d0 0x7f7de406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.220+0000 7f7e062b2700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7de406a4d0 0x7f7de406c990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7df0000c00 tx=0x7f7df0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:27.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.224+0000 7f7df6ffd700 1 -- 172.21.15.112:0/4091684006 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7dfc0530b0 con 0x7f7e000d1a60 2023-08-27T11:28:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.414+0000 7f7e08516700 1 -- 172.21.15.112:0/4091684006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7e000b4ff0 con 0x7f7e000d1a60 2023-08-27T11:28:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.414+0000 7f7df6ffd700 1 -- 172.21.15.112:0/4091684006 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v139) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7dfc0170a0 con 0x7f7e000d1a60 2023-08-27T11:28:27.415 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:28:27.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.416+0000 7f7e08516700 1 -- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7de406a4d0 msgr2=0x7f7de406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.416+0000 7f7e08516700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7de406a4d0 0x7f7de406c990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7df0000c00 tx=0x7f7df0008030 comp rx=0 tx=0).stop 2023-08-27T11:28:27.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.416+0000 7f7e08516700 1 -- 172.21.15.112:0/4091684006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e000d1a60 msgr2=0x7f7e000cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.416+0000 7f7e08516700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e000d1a60 0x7f7e000cb670 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00c430 tx=0x7f7dfc00c740 comp rx=0 tx=0).stop 2023-08-27T11:28:27.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.416+0000 7f7e08516700 1 -- 172.21.15.112:0/4091684006 shutdown_connections 2023-08-27T11:28:27.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.416+0000 7f7e08516700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7de406a4d0 0x7f7de406c990 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.416+0000 7f7e08516700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e0015af90 0x7f7e0015c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.416+0000 7f7e08516700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e00071360 0x7f7e0014b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.416+0000 7f7e08516700 1 --2- 172.21.15.112:0/4091684006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e000d1a60 0x7f7e000cb670 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.416+0000 7f7e08516700 1 -- 172.21.15.112:0/4091684006 >> 172.21.15.112:0/4091684006 conn(0x7f7e00066010 msgr2=0x7f7e000b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:27.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.416+0000 7f7e08516700 1 -- 172.21.15.112:0/4091684006 shutdown_connections 2023-08-27T11:28:27.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.416+0000 7f7e08516700 1 -- 172.21.15.112:0/4091684006 wait complete. 2023-08-27T11:28:27.428 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:28:27.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.551+0000 7fd4b2b2b700 1 Processor -- start 2023-08-27T11:28:27.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.551+0000 7fd4b2b2b700 1 -- start start 2023-08-27T11:28:27.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b2b2b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4ac154800 0x7fd4ac0b1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b2b2b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd4ac15e4d0 0x7fd4ac0b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b2b2b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac154be0 con 0x7fd4ac0b1950 2023-08-27T11:28:27.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b2b2b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac0b2310 con 0x7fd4ac15e4d0 2023-08-27T11:28:27.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b2b2b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac15e960 con 0x7fd4ac154800 2023-08-27T11:28:27.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b08c7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4ac154800 0x7fd4ac0b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b10c8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd4ac15e4d0 0x7fd4ac0b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b08c7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4ac154800 0x7fd4ac0b1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:48054/0 (socket says 172.21.15.112:48054) 2023-08-27T11:28:27.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4abfff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd4ac0b1950 0x7fd4ac15ddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:38306/0 (socket says 172.21.15.112:38306) 2023-08-27T11:28:27.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b08c7700 1 -- 172.21.15.112:0/774744723 learned_addr learned my addr 172.21.15.112:0/774744723 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:27.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b08c7700 1 -- 172.21.15.112:0/774744723 >> v2:172.21.15.138:3300/0 conn(0x7fd4ac15e4d0 msgr2=0x7fd4ac0b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:27.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b08c7700 1 --2- 172.21.15.112:0/774744723 >> v2:172.21.15.138:3300/0 conn(0x7fd4ac15e4d0 0x7fd4ac0b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b08c7700 1 -- 172.21.15.112:0/774744723 >> v1:172.21.15.138:6789/0 conn(0x7fd4ac0b1950 legacy=0x7fd4ac15ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b08c7700 1 -- 172.21.15.112:0/774744723 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac0b0260 con 0x7fd4ac154800 2023-08-27T11:28:27.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.552+0000 7fd4b10c8700 1 --2- 172.21.15.112:0/774744723 >> v2:172.21.15.138:3300/0 conn(0x7fd4ac15e4d0 0x7fd4ac0b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:27.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.553+0000 7fd4b08c7700 1 --2- 172.21.15.112:0/774744723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4ac154800 0x7fd4ac0b1000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00098f0 tx=0x7fd4a0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bbf7ae3613d448f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:27.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.553+0000 7fd4ab7fe700 1 -- 172.21.15.112:0/774744723 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0011830 con 0x7fd4ac154800 2023-08-27T11:28:27.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.553+0000 7fd4ab7fe700 1 -- 172.21.15.112:0/774744723 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a0011990 con 0x7fd4ac154800 2023-08-27T11:28:27.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.553+0000 7fd4ab7fe700 1 -- 172.21.15.112:0/774744723 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0011d20 con 0x7fd4ac154800 2023-08-27T11:28:27.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.553+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/774744723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4ac154800 msgr2=0x7fd4ac0b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.553+0000 7fd4b2b2b700 1 --2- 172.21.15.112:0/774744723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4ac154800 0x7fd4ac0b1000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00098f0 tx=0x7fd4a0009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:27.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.553+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/774744723 shutdown_connections 2023-08-27T11:28:27.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.553+0000 7fd4b2b2b700 1 --2- 172.21.15.112:0/774744723 >> v2:172.21.15.138:3300/0 conn(0x7fd4ac15e4d0 0x7fd4ac0b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.553+0000 7fd4b2b2b700 1 --2- 172.21.15.112:0/774744723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4ac154800 0x7fd4ac0b1000 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.553+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/774744723 >> 172.21.15.112:0/774744723 conn(0x7fd4ac0cca80 msgr2=0x7fd4ac0cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:27.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.553+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/774744723 shutdown_connections 2023-08-27T11:28:27.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.553+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/774744723 wait complete. 2023-08-27T11:28:27.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.554+0000 7fd4b2b2b700 1 Processor -- start 2023-08-27T11:28:27.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.555+0000 7fd4b2b2b700 1 -- start start 2023-08-27T11:28:27.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.555+0000 7fd4b2b2b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4ac0b1950 0x7fd4ac156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.555+0000 7fd4b2b2b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4ac154800 0x7fd4ac154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.555+0000 7fd4b2b2b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4ac15e4d0 0x7fd4ac155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.555+0000 7fd4b2b2b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac14af20 con 0x7fd4ac0b1950 2023-08-27T11:28:27.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.555+0000 7fd4b2b2b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac14a610 con 0x7fd4ac15e4d0 2023-08-27T11:28:27.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.555+0000 7fd4b2b2b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0c56d0 con 0x7fd4ac154800 2023-08-27T11:28:27.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4b08c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4ac0b1950 0x7fd4ac156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4b08c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4ac0b1950 0x7fd4ac156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36406/0 (socket says 172.21.15.112:36406) 2023-08-27T11:28:27.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4b08c7700 1 -- 172.21.15.112:0/1138394237 learned_addr learned my addr 172.21.15.112:0/1138394237 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:27.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4abfff700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4ac154800 0x7fd4ac154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4b10c8700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4ac15e4d0 0x7fd4ac155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4b08c7700 1 -- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4ac154800 msgr2=0x7fd4ac154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4b08c7700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4ac154800 0x7fd4ac154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4b08c7700 1 -- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4ac15e4d0 msgr2=0x7fd4ac155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4b08c7700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4ac15e4d0 0x7fd4ac155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4b08c7700 1 -- 172.21.15.112:0/1138394237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a4009a70 con 0x7fd4ac0b1950 2023-08-27T11:28:27.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4b08c7700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4ac0b1950 0x7fd4ac156350 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0000c00 tx=0x7fd4a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:27.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4a97fa700 1 -- 172.21.15.112:0/1138394237 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00043c0 con 0x7fd4ac0b1950 2023-08-27T11:28:27.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4a97fa700 1 -- 172.21.15.112:0/1138394237 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a001f070 con 0x7fd4ac0b1950 2023-08-27T11:28:27.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/1138394237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a0009550 con 0x7fd4ac0b1950 2023-08-27T11:28:27.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4a97fa700 1 -- 172.21.15.112:0/1138394237 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00036a0 con 0x7fd4ac0b1950 2023-08-27T11:28:27.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.556+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/1138394237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac14a8d0 con 0x7fd4ac0b1950 2023-08-27T11:28:27.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.557+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/1138394237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ac067a20 con 0x7fd4ac0b1950 2023-08-27T11:28:27.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.558+0000 7fd4a97fa700 1 -- 172.21.15.112:0/1138394237 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fd4a0021030 con 0x7fd4ac0b1950 2023-08-27T11:28:27.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.558+0000 7fd4a97fa700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd49006a4d0 0x7fd49006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.558+0000 7fd4a97fa700 1 -- 172.21.15.112:0/1138394237 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd4a008d120 con 0x7fd4ac0b1950 2023-08-27T11:28:27.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.558+0000 7fd4abfff700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd49006a4d0 0x7fd49006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.558+0000 7fd4abfff700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd49006a4d0 0x7fd49006c990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0c5e50 tx=0x7fd498007ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:27.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.562+0000 7fd4a97fa700 1 -- 172.21.15.112:0/1138394237 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd4a00551a0 con 0x7fd4ac0b1950 2023-08-27T11:28:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.750+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/1138394237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4ac15d700 con 0x7fd4ac0b1950 2023-08-27T11:28:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.751+0000 7fd4a97fa700 1 -- 172.21.15.112:0/1138394237 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4a00587c0 con 0x7fd4ac0b1950 2023-08-27T11:28:27.752 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:28:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.753+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd49006a4d0 msgr2=0x7fd49006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.753+0000 7fd4b2b2b700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd49006a4d0 0x7fd49006c990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0c5e50 tx=0x7fd498007ed0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.753+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/1138394237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4ac0b1950 msgr2=0x7fd4ac156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.753+0000 7fd4b2b2b700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4ac0b1950 0x7fd4ac156350 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0000c00 tx=0x7fd4a0004750 comp rx=0 tx=0).stop 2023-08-27T11:28:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.753+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/1138394237 shutdown_connections 2023-08-27T11:28:27.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.753+0000 7fd4b2b2b700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd49006a4d0 0x7fd49006c990 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.753+0000 7fd4b2b2b700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4ac15e4d0 0x7fd4ac155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.753+0000 7fd4b2b2b700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4ac0b1950 0x7fd4ac156350 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.753+0000 7fd4b2b2b700 1 --2- 172.21.15.112:0/1138394237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4ac154800 0x7fd4ac154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.753+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/1138394237 >> 172.21.15.112:0/1138394237 conn(0x7fd4ac0cca80 msgr2=0x7fd4ac0cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:27.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.753+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/1138394237 shutdown_connections 2023-08-27T11:28:27.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.753+0000 7fd4b2b2b700 1 -- 172.21.15.112:0/1138394237 wait complete. 2023-08-27T11:28:27.765 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:28:27.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.888+0000 7fa216f7c700 1 Processor -- start 2023-08-27T11:28:27.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.888+0000 7fa216f7c700 1 -- start start 2023-08-27T11:28:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.889+0000 7fa216f7c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa210146bf0 0x7fa2100b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.889+0000 7fa216f7c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa210070eb0 0x7fa210071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.889+0000 7fa216f7c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2100d1500 con 0x7fa210146fd0 2023-08-27T11:28:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.889+0000 7fa216f7c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2100718a0 con 0x7fa210070eb0 2023-08-27T11:28:27.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.889+0000 7fa216f7c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100d0460 con 0x7fa210146bf0 2023-08-27T11:28:27.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.890+0000 7fa214d18700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa210146bf0 0x7fa2100b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.890+0000 7fa215519700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa210070eb0 0x7fa210071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.890+0000 7fa20ffff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa210146fd0 0x7fa2100cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:38322/0 (socket says 172.21.15.112:38322) 2023-08-27T11:28:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.890+0000 7fa20ffff700 1 -- 172.21.15.112:0/422970214 learned_addr learned my addr 172.21.15.112:0/422970214 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.890+0000 7fa214d18700 1 -- 172.21.15.112:0/422970214 >> v2:172.21.15.138:3300/0 conn(0x7fa210070eb0 msgr2=0x7fa210071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.890+0000 7fa214d18700 1 --2- 172.21.15.112:0/422970214 >> v2:172.21.15.138:3300/0 conn(0x7fa210070eb0 0x7fa210071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.890+0000 7fa214d18700 1 -- 172.21.15.112:0/422970214 >> v1:172.21.15.138:6789/0 conn(0x7fa210146fd0 legacy=0x7fa2100cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.890+0000 7fa214d18700 1 -- 172.21.15.112:0/422970214 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100d0710 con 0x7fa210146bf0 2023-08-27T11:28:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.890+0000 7fa214d18700 1 --2- 172.21.15.112:0/422970214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa210146bf0 0x7fa2100b5db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa204009840 tx=0x7fa204009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c559e4de0c30172 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.890+0000 7fa20f7fe700 1 -- 172.21.15.112:0/422970214 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204011830 con 0x7fa210146bf0 2023-08-27T11:28:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.890+0000 7fa20f7fe700 1 -- 172.21.15.112:0/422970214 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa204011990 con 0x7fa210146bf0 2023-08-27T11:28:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.890+0000 7fa20f7fe700 1 -- 172.21.15.112:0/422970214 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204011d20 con 0x7fa210146bf0 2023-08-27T11:28:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.891+0000 7fa216f7c700 1 -- 172.21.15.112:0/422970214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa210146bf0 msgr2=0x7fa2100b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.891+0000 7fa216f7c700 1 --2- 172.21.15.112:0/422970214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa210146bf0 0x7fa2100b5db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa204009840 tx=0x7fa204009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.891+0000 7fa216f7c700 1 -- 172.21.15.112:0/422970214 shutdown_connections 2023-08-27T11:28:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.891+0000 7fa216f7c700 1 --2- 172.21.15.112:0/422970214 >> v2:172.21.15.138:3300/0 conn(0x7fa210070eb0 0x7fa210071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.891+0000 7fa216f7c700 1 --2- 172.21.15.112:0/422970214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa210146bf0 0x7fa2100b5db0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.891+0000 7fa216f7c700 1 -- 172.21.15.112:0/422970214 >> 172.21.15.112:0/422970214 conn(0x7fa21006c450 msgr2=0x7fa21006f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.891+0000 7fa216f7c700 1 -- 172.21.15.112:0/422970214 shutdown_connections 2023-08-27T11:28:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.891+0000 7fa216f7c700 1 -- 172.21.15.112:0/422970214 wait complete. 2023-08-27T11:28:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.892+0000 7fa216f7c700 1 Processor -- start 2023-08-27T11:28:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.892+0000 7fa216f7c700 1 -- start start 2023-08-27T11:28:27.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.893+0000 7fa216f7c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa210070eb0 0x7fa21013e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.893+0000 7fa216f7c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa210146bf0 0x7fa2101f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.893+0000 7fa216f7c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa210146fd0 0x7fa21015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.893+0000 7fa216f7c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2101f71a0 con 0x7fa210146fd0 2023-08-27T11:28:27.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.893+0000 7fa216f7c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21013d6e0 con 0x7fa210146bf0 2023-08-27T11:28:27.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.893+0000 7fa216f7c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21013efd0 con 0x7fa210070eb0 2023-08-27T11:28:27.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.893+0000 7fa215519700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa210146fd0 0x7fa21015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.893+0000 7fa215519700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa210146fd0 0x7fa21015a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36424/0 (socket says 172.21.15.112:36424) 2023-08-27T11:28:27.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.893+0000 7fa215519700 1 -- 172.21.15.112:0/2684134763 learned_addr learned my addr 172.21.15.112:0/2684134763 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:27.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.893+0000 7fa214d18700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa210070eb0 0x7fa21013e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.893+0000 7fa20ffff700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa210146bf0 0x7fa2101f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.893+0000 7fa215519700 1 -- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa210070eb0 msgr2=0x7fa21013e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.893+0000 7fa215519700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa210070eb0 0x7fa21013e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.894+0000 7fa215519700 1 -- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa210146bf0 msgr2=0x7fa2101f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.894+0000 7fa215519700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa210146bf0 0x7fa2101f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.894+0000 7fa215519700 1 -- 172.21.15.112:0/2684134763 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2040094a0 con 0x7fa210146fd0 2023-08-27T11:28:27.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.894+0000 7fa214d18700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa210070eb0 0x7fa21013e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:27.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.894+0000 7fa20ffff700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa210146bf0 0x7fa2101f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:27.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.894+0000 7fa215519700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa210146fd0 0x7fa21015a090 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fa20800efc0 tx=0x7fa20800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:27.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.894+0000 7fa20d7fa700 1 -- 172.21.15.112:0/2684134763 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208009ba0 con 0x7fa210146fd0 2023-08-27T11:28:27.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.894+0000 7fa20d7fa700 1 -- 172.21.15.112:0/2684134763 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20800f040 con 0x7fa210146fd0 2023-08-27T11:28:27.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.894+0000 7fa216f7c700 1 -- 172.21.15.112:0/2684134763 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21013f2b0 con 0x7fa210146fd0 2023-08-27T11:28:27.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.894+0000 7fa20d7fa700 1 -- 172.21.15.112:0/2684134763 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208014480 con 0x7fa210146fd0 2023-08-27T11:28:27.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.894+0000 7fa216f7c700 1 -- 172.21.15.112:0/2684134763 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21013dc80 con 0x7fa210146fd0 2023-08-27T11:28:27.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.895+0000 7fa216f7c700 1 -- 172.21.15.112:0/2684134763 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa210067e90 con 0x7fa210146fd0 2023-08-27T11:28:27.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.895+0000 7fa20d7fa700 1 -- 172.21.15.112:0/2684134763 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa2080145e0 con 0x7fa210146fd0 2023-08-27T11:28:27.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.896+0000 7fa20d7fa700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1f406a4d0 0x7fa1f406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.896+0000 7fa20d7fa700 1 -- 172.21.15.112:0/2684134763 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa2080896d0 con 0x7fa210146fd0 2023-08-27T11:28:27.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.896+0000 7fa214d18700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1f406a4d0 0x7fa1f406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.896+0000 7fa214d18700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1f406a4d0 0x7fa1f406c990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa204000c00 tx=0x7fa204008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:27.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:27.900+0000 7fa20d7fa700 1 -- 172.21.15.112:0/2684134763 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa2080546d0 con 0x7fa210146fd0 2023-08-27T11:28:28.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.087+0000 7fa216f7c700 1 -- 172.21.15.112:0/2684134763 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa210067a20 con 0x7fa210146fd0 2023-08-27T11:28:28.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.087+0000 7fa20d7fa700 1 -- 172.21.15.112:0/2684134763 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa208057cf0 con 0x7fa210146fd0 2023-08-27T11:28:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:28:28.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.089+0000 7fa216f7c700 1 -- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1f406a4d0 msgr2=0x7fa1f406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.089+0000 7fa216f7c700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1f406a4d0 0x7fa1f406c990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa204000c00 tx=0x7fa204008030 comp rx=0 tx=0).stop 2023-08-27T11:28:28.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.090+0000 7fa216f7c700 1 -- 172.21.15.112:0/2684134763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa210146fd0 msgr2=0x7fa21015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.090+0000 7fa216f7c700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa210146fd0 0x7fa21015a090 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fa20800efc0 tx=0x7fa20800c3a0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.090+0000 7fa216f7c700 1 -- 172.21.15.112:0/2684134763 shutdown_connections 2023-08-27T11:28:28.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.090+0000 7fa216f7c700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa1f406a4d0 0x7fa1f406c990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.090+0000 7fa216f7c700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa210146fd0 0x7fa21015a090 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.090+0000 7fa216f7c700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa210070eb0 0x7fa21013e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.090+0000 7fa216f7c700 1 --2- 172.21.15.112:0/2684134763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa210146bf0 0x7fa2101f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.090+0000 7fa216f7c700 1 -- 172.21.15.112:0/2684134763 >> 172.21.15.112:0/2684134763 conn(0x7fa21006c450 msgr2=0x7fa21006f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:28.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.090+0000 7fa216f7c700 1 -- 172.21.15.112:0/2684134763 shutdown_connections 2023-08-27T11:28:28.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.090+0000 7fa216f7c700 1 -- 172.21.15.112:0/2684134763 wait complete. 2023-08-27T11:28:28.102 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:28:28.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.224+0000 7ff79d41a700 1 Processor -- start 2023-08-27T11:28:28.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.224+0000 7ff79d41a700 1 -- start start 2023-08-27T11:28:28.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff79d41a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7980ceb90 0x7ff7980721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff79d41a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff79806f790 0x7ff7980c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff79d41a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7981580b0 con 0x7ff798157cd0 2023-08-27T11:28:28.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff79d41a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff798070dd0 con 0x7ff79806f790 2023-08-27T11:28:28.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff79d41a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79815a680 con 0x7ff7980ceb90 2023-08-27T11:28:28.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff796ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff798157cd0 0x7ff7980ce480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59042/0 (socket says 172.21.15.112:59042) 2023-08-27T11:28:28.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff796ffd700 1 -- 172.21.15.112:0/1231256240 learned_addr learned my addr 172.21.15.112:0/1231256240 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:28.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff7967fc700 1 --2- 172.21.15.112:0/1231256240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7980ceb90 0x7ff7980721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548912049 0 0) 0x7ff7981580b0 con 0x7ff798157cd0 2023-08-27T11:28:28.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7800035b0 con 0x7ff798157cd0 2023-08-27T11:28:28.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff7977fe700 1 --2- 172.21.15.112:0/1231256240 >> v2:172.21.15.138:3300/0 conn(0x7ff79806f790 0x7ff7980c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1970316020 0 0) 0x7ff7800035b0 con 0x7ff798157cd0 2023-08-27T11:28:28.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7980ceb90 msgr2=0x7ff7980721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:28.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 --2- 172.21.15.112:0/1231256240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7980ceb90 0x7ff7980721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 >> v2:172.21.15.138:3300/0 conn(0x7ff79806f790 msgr2=0x7ff7980c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:28.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 --2- 172.21.15.112:0/1231256240 >> v2:172.21.15.138:3300/0 conn(0x7ff79806f790 0x7ff7980c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79815a940 con 0x7ff798157cd0 2023-08-27T11:28:28.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7ff788001670 con 0x7ff798157cd0 2023-08-27T11:28:28.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 >> v1:172.21.15.112:6789/0 conn(0x7ff798157cd0 legacy=0x7ff7980ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 --2- 172.21.15.112:0/1231256240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff780003a70 0x7ff780003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 --2- 172.21.15.112:0/1231256240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff780004850 0x7ff780004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 --2- 172.21.15.112:0/1231256240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7980ceb90 0x7ff780005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff798070dd0 con 0x7ff780004850 2023-08-27T11:28:28.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79815a680 con 0x7ff780003a70 2023-08-27T11:28:28.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78000ba10 con 0x7ff7980ceb90 2023-08-27T11:28:28.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.226+0000 7ff7967fc700 1 --2- 172.21.15.112:0/1231256240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff780004850 0x7ff780004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff796ffd700 1 --2- 172.21.15.112:0/1231256240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff780003a70 0x7ff780003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff7977fe700 1 --2- 172.21.15.112:0/1231256240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7980ceb90 0x7ff780005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff7967fc700 1 -- 172.21.15.112:0/1231256240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7980ceb90 msgr2=0x7ff780005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:28.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff7967fc700 1 --2- 172.21.15.112:0/1231256240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7980ceb90 0x7ff780005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff7967fc700 1 -- 172.21.15.112:0/1231256240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff780003a70 msgr2=0x7ff780003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:28.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff7967fc700 1 --2- 172.21.15.112:0/1231256240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff780003a70 0x7ff780003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff7967fc700 1 -- 172.21.15.112:0/1231256240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800035b0 con 0x7ff780004850 2023-08-27T11:28:28.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff7967fc700 1 --2- 172.21.15.112:0/1231256240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff780004850 0x7ff780004cd0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009b90 tx=0x7ff780005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b4b7e9aa5d233 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:28.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c00e6c0 con 0x7ff780004850 2023-08-27T11:28:28.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c00e820 con 0x7ff780004850 2023-08-27T11:28:28.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff795ffb700 1 -- 172.21.15.112:0/1231256240 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c00eb90 con 0x7ff780004850 2023-08-27T11:28:28.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff79d41a700 1 -- 172.21.15.112:0/1231256240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff780004850 msgr2=0x7ff780004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff79d41a700 1 --2- 172.21.15.112:0/1231256240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff780004850 0x7ff780004cd0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009b90 tx=0x7ff780005880 comp rx=0 tx=0).stop 2023-08-27T11:28:28.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff7977fe700 1 -- 172.21.15.112:0/1231256240 reap_dead start 2023-08-27T11:28:28.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff79d41a700 1 -- 172.21.15.112:0/1231256240 shutdown_connections 2023-08-27T11:28:28.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.227+0000 7ff79d41a700 1 -- 172.21.15.112:0/1231256240 >> 172.21.15.112:0/1231256240 conn(0x7ff7980cc220 msgr2=0x7ff7980cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:28.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.228+0000 7ff79d41a700 1 -- 172.21.15.112:0/1231256240 shutdown_connections 2023-08-27T11:28:28.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.228+0000 7ff79d41a700 1 -- 172.21.15.112:0/1231256240 wait complete. 2023-08-27T11:28:28.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.229+0000 7ff79d41a700 1 Processor -- start 2023-08-27T11:28:28.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.229+0000 7ff79d41a700 1 -- start start 2023-08-27T11:28:28.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff79d41a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff780004850 0x7ff798148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff79d41a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff79814b0e0 0x7ff79814b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff79d41a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff79815dbe0 0x7ff79815e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff79d41a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79815e5d0 con 0x7ff79814b0e0 2023-08-27T11:28:28.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff79d41a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980c4090 con 0x7ff780004850 2023-08-27T11:28:28.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff79d41a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff798148d50 con 0x7ff79815dbe0 2023-08-27T11:28:28.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff7967fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff79814b0e0 0x7ff79814b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff7967fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff79814b0e0 0x7ff79814b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36442/0 (socket says 172.21.15.112:36442) 2023-08-27T11:28:28.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff7967fc700 1 -- 172.21.15.112:0/802712084 learned_addr learned my addr 172.21.15.112:0/802712084 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:28.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff796ffd700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff780004850 0x7ff798148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff7977fe700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff79815dbe0 0x7ff79815e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff7967fc700 1 -- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff79815dbe0 msgr2=0x7ff79815e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff7967fc700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff79815dbe0 0x7ff79815e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff7967fc700 1 -- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff780004850 msgr2=0x7ff798148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff7967fc700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff780004850 0x7ff798148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff7967fc700 1 -- 172.21.15.112:0/802712084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79814bc50 con 0x7ff79814b0e0 2023-08-27T11:28:28.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff7967fc700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff79814b0e0 0x7ff79814b560 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ff780004710 tx=0x7ff78c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:28.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff77f7fe700 1 -- 172.21.15.112:0/802712084 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c00e6c0 con 0x7ff79814b0e0 2023-08-27T11:28:28.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff77f7fe700 1 -- 172.21.15.112:0/802712084 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c0038a0 con 0x7ff79814b0e0 2023-08-27T11:28:28.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff79d41a700 1 -- 172.21.15.112:0/802712084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79814bf30 con 0x7ff79814b0e0 2023-08-27T11:28:28.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff77f7fe700 1 -- 172.21.15.112:0/802712084 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c003c50 con 0x7ff79814b0e0 2023-08-27T11:28:28.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.230+0000 7ff79d41a700 1 -- 172.21.15.112:0/802712084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7980c4f30 con 0x7ff79814b0e0 2023-08-27T11:28:28.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.231+0000 7ff79d41a700 1 -- 172.21.15.112:0/802712084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff798061f50 con 0x7ff79814b0e0 2023-08-27T11:28:28.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.232+0000 7ff77f7fe700 1 -- 172.21.15.112:0/802712084 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7ff78c013470 con 0x7ff79814b0e0 2023-08-27T11:28:28.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.232+0000 7ff77f7fe700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff78406a4d0 0x7ff78406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.232+0000 7ff77f7fe700 1 -- 172.21.15.112:0/802712084 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff78c08a310 con 0x7ff79814b0e0 2023-08-27T11:28:28.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.232+0000 7ff796ffd700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff78406a4d0 0x7ff78406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.233+0000 7ff796ffd700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff78406a4d0 0x7ff78406c990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff788000fb0 tx=0x7ff788007980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:28.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.237+0000 7ff77f7fe700 1 -- 172.21.15.112:0/802712084 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff78c055590 con 0x7ff79814b0e0 2023-08-27T11:28:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.424+0000 7ff79d41a700 1 -- 172.21.15.112:0/802712084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff798071170 con 0x7ff79814b0e0 2023-08-27T11:28:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.424+0000 7ff77f7fe700 1 -- 172.21.15.112:0/802712084 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff78c058bb0 con 0x7ff79814b0e0 2023-08-27T11:28:28.425 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:28:28.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.426+0000 7ff79d41a700 1 -- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff78406a4d0 msgr2=0x7ff78406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.426+0000 7ff79d41a700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff78406a4d0 0x7ff78406c990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff788000fb0 tx=0x7ff788007980 comp rx=0 tx=0).stop 2023-08-27T11:28:28.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.426+0000 7ff79d41a700 1 -- 172.21.15.112:0/802712084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff79814b0e0 msgr2=0x7ff79814b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.426+0000 7ff79d41a700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff79814b0e0 0x7ff79814b560 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ff780004710 tx=0x7ff78c0047c0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.426+0000 7ff79d41a700 1 -- 172.21.15.112:0/802712084 shutdown_connections 2023-08-27T11:28:28.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.426+0000 7ff79d41a700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff78406a4d0 0x7ff78406c990 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.426+0000 7ff79d41a700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff79815dbe0 0x7ff79815e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.426+0000 7ff79d41a700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff780004850 0x7ff798148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.426+0000 7ff79d41a700 1 --2- 172.21.15.112:0/802712084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff79814b0e0 0x7ff79814b560 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.426+0000 7ff79d41a700 1 -- 172.21.15.112:0/802712084 >> 172.21.15.112:0/802712084 conn(0x7ff7980cc220 msgr2=0x7ff798070000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.426+0000 7ff79d41a700 1 -- 172.21.15.112:0/802712084 shutdown_connections 2023-08-27T11:28:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.426+0000 7ff79d41a700 1 -- 172.21.15.112:0/802712084 wait complete. 2023-08-27T11:28:28.438 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:28:28.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.559+0000 7fe60f02c700 1 Processor -- start 2023-08-27T11:28:28.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.560+0000 7fe60f02c700 1 -- start start 2023-08-27T11:28:28.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.560+0000 7fe60f02c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe608151060 0x7fe60815e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.560+0000 7fe60f02c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe60806b4c0 0x7fe6080cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.560+0000 7fe60f02c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6080b02f0 con 0x7fe60815e990 2023-08-27T11:28:28.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.560+0000 7fe60f02c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6080b05d0 con 0x7fe60806b4c0 2023-08-27T11:28:28.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.560+0000 7fe60f02c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6080b08b0 con 0x7fe608151060 2023-08-27T11:28:28.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.560+0000 7fe60d5c9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe60806b4c0 0x7fe6080cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.560+0000 7fe607fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe60815e990 0x7fe60806adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59064/0 (socket says 172.21.15.112:59064) 2023-08-27T11:28:28.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.560+0000 7fe607fff700 1 -- 172.21.15.112:0/2531802114 learned_addr learned my addr 172.21.15.112:0/2531802114 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:28.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.561+0000 7fe60d5c9700 1 -- 172.21.15.112:0/2531802114 >> v2:172.21.15.138:3300/0 conn(0x7fe608151060 msgr2=0x7fe60815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:28.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.561+0000 7fe60cdc8700 1 --2- 172.21.15.112:0/2531802114 >> v2:172.21.15.138:3300/0 conn(0x7fe608151060 0x7fe60815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.561+0000 7fe60d5c9700 1 --2- 172.21.15.112:0/2531802114 >> v2:172.21.15.138:3300/0 conn(0x7fe608151060 0x7fe60815e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.561+0000 7fe60d5c9700 1 -- 172.21.15.112:0/2531802114 >> v1:172.21.15.112:6789/0 conn(0x7fe60815e990 legacy=0x7fe60806adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.561+0000 7fe6077fe700 1 -- 172.21.15.112:0/2531802114 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249642038 0 0) 0x7fe6080b02f0 con 0x7fe60815e990 2023-08-27T11:28:28.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.561+0000 7fe60d5c9700 1 -- 172.21.15.112:0/2531802114 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6080b0b90 con 0x7fe60806b4c0 2023-08-27T11:28:28.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.561+0000 7fe60d5c9700 1 --2- 172.21.15.112:0/2531802114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60806b4c0 0x7fe6080cdbd0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fe60806e8c0 tx=0x7fe60000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359d5af8850a2fa0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:28.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.561+0000 7fe6077fe700 1 -- 172.21.15.112:0/2531802114 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60000e070 con 0x7fe60806b4c0 2023-08-27T11:28:28.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.561+0000 7fe6077fe700 1 -- 172.21.15.112:0/2531802114 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60000bbd0 con 0x7fe60806b4c0 2023-08-27T11:28:28.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.561+0000 7fe6077fe700 1 -- 172.21.15.112:0/2531802114 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600012430 con 0x7fe60806b4c0 2023-08-27T11:28:28.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.561+0000 7fe60f02c700 1 -- 172.21.15.112:0/2531802114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60806b4c0 msgr2=0x7fe6080cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.561+0000 7fe60f02c700 1 --2- 172.21.15.112:0/2531802114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60806b4c0 0x7fe6080cdbd0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fe60806e8c0 tx=0x7fe60000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:28.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.562+0000 7fe60f02c700 1 -- 172.21.15.112:0/2531802114 shutdown_connections 2023-08-27T11:28:28.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.562+0000 7fe60f02c700 1 --2- 172.21.15.112:0/2531802114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60806b4c0 0x7fe6080cdbd0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.562+0000 7fe60f02c700 1 --2- 172.21.15.112:0/2531802114 >> v2:172.21.15.138:3300/0 conn(0x7fe608151060 0x7fe60815e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.562+0000 7fe60f02c700 1 -- 172.21.15.112:0/2531802114 >> 172.21.15.112:0/2531802114 conn(0x7fe6080d0000 msgr2=0x7fe6080d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:28.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.562+0000 7fe60f02c700 1 -- 172.21.15.112:0/2531802114 shutdown_connections 2023-08-27T11:28:28.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.562+0000 7fe60f02c700 1 -- 172.21.15.112:0/2531802114 wait complete. 2023-08-27T11:28:28.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.563+0000 7fe60f02c700 1 Processor -- start 2023-08-27T11:28:28.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.563+0000 7fe60f02c700 1 -- start start 2023-08-27T11:28:28.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60f02c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60806b4c0 0x7fe6080c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60f02c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe608151060 0x7fe608159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60f02c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60815e990 0x7fe60814fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60f02c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6081500b0 con 0x7fe60806b4c0 2023-08-27T11:28:28.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60f02c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080c4480 con 0x7fe60815e990 2023-08-27T11:28:28.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60f02c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080c4720 con 0x7fe608151060 2023-08-27T11:28:28.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60cdc8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60806b4c0 0x7fe6080c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60cdc8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60806b4c0 0x7fe6080c3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36454/0 (socket says 172.21.15.112:36454) 2023-08-27T11:28:28.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60cdc8700 1 -- 172.21.15.112:0/1083538075 learned_addr learned my addr 172.21.15.112:0/1083538075 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:28.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe607fff700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe608151060 0x7fe608159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60d5c9700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60815e990 0x7fe60814fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60cdc8700 1 -- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe608151060 msgr2=0x7fe608159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60cdc8700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe608151060 0x7fe608159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60cdc8700 1 -- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60815e990 msgr2=0x7fe60814fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60cdc8700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60815e990 0x7fe60814fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60cdc8700 1 -- 172.21.15.112:0/1083538075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe600009a70 con 0x7fe60806b4c0 2023-08-27T11:28:28.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60cdc8700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60806b4c0 0x7fe6080c3d90 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00c7f0 tx=0x7fe5fc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe6057fa700 1 -- 172.21.15.112:0/1083538075 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc011800 con 0x7fe60806b4c0 2023-08-27T11:28:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe6057fa700 1 -- 172.21.15.112:0/1083538075 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc011960 con 0x7fe60806b4c0 2023-08-27T11:28:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60f02c700 1 -- 172.21.15.112:0/1083538075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6080c4a00 con 0x7fe60806b4c0 2023-08-27T11:28:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe6057fa700 1 -- 172.21.15.112:0/1083538075 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc011cf0 con 0x7fe60806b4c0 2023-08-27T11:28:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.564+0000 7fe60f02c700 1 -- 172.21.15.112:0/1083538075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6080c9810 con 0x7fe60806b4c0 2023-08-27T11:28:28.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.565+0000 7fe60f02c700 1 -- 172.21.15.112:0/1083538075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe608067a20 con 0x7fe60806b4c0 2023-08-27T11:28:28.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.566+0000 7fe6057fa700 1 -- 172.21.15.112:0/1083538075 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe5fc018360 con 0x7fe60806b4c0 2023-08-27T11:28:28.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.566+0000 7fe6057fa700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe5ec06a4d0 0x7fe5ec06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.566+0000 7fe6057fa700 1 -- 172.21.15.112:0/1083538075 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe5fc088e10 con 0x7fe60806b4c0 2023-08-27T11:28:28.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.566+0000 7fe607fff700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe5ec06a4d0 0x7fe5ec06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.567+0000 7fe607fff700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe5ec06a4d0 0x7fe5ec06c990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe60815a3d0 tx=0x7fe5f4007be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:28.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.570+0000 7fe6057fa700 1 -- 172.21.15.112:0/1083538075 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe5fc08c030 con 0x7fe60806b4c0 2023-08-27T11:28:28.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.777+0000 7fe60f02c700 1 -- 172.21.15.112:0/1083538075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe60815a120 con 0x7fe60806b4c0 2023-08-27T11:28:28.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.777+0000 7fe6057fa700 1 -- 172.21.15.112:0/1083538075 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v139) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe5fc0578b0 con 0x7fe60806b4c0 2023-08-27T11:28:28.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.779+0000 7fe60f02c700 1 -- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe5ec06a4d0 msgr2=0x7fe5ec06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.779+0000 7fe60f02c700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe5ec06a4d0 0x7fe5ec06c990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe60815a3d0 tx=0x7fe5f4007be0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.779+0000 7fe60f02c700 1 -- 172.21.15.112:0/1083538075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60806b4c0 msgr2=0x7fe6080c3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.779+0000 7fe60f02c700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60806b4c0 0x7fe6080c3d90 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00c7f0 tx=0x7fe5fc00cbb0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.779+0000 7fe60f02c700 1 -- 172.21.15.112:0/1083538075 shutdown_connections 2023-08-27T11:28:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.779+0000 7fe60f02c700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe5ec06a4d0 0x7fe5ec06c990 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.779+0000 7fe60f02c700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60815e990 0x7fe60814fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.779+0000 7fe60f02c700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60806b4c0 0x7fe6080c3d90 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.779+0000 7fe60f02c700 1 --2- 172.21.15.112:0/1083538075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe608151060 0x7fe608159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.779+0000 7fe60f02c700 1 -- 172.21.15.112:0/1083538075 >> 172.21.15.112:0/1083538075 conn(0x7fe6080d0000 msgr2=0x7fe6080d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.779+0000 7fe60f02c700 1 -- 172.21.15.112:0/1083538075 shutdown_connections 2023-08-27T11:28:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:28.779+0000 7fe60f02c700 1 -- 172.21.15.112:0/1083538075 wait complete. 2023-08-27T11:28:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:28:28.792 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-27T11:28:28.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:28.793 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-08-27T11:28:28.837 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:28:28.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:28.838 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-27T11:28:28.897 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:28:28.897 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:28.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:28:28.916 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:28:28.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:28.917 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-27T11:28:28.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:28:28.973 INFO:teuthology.orchestra.run.smithi112.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-27T11:28:28.973 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:28:28.974 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:28.993 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:28:28.994 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:28:28.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:28.995 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-27T11:28:29.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:28:29.053 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-27T11:28:29.054 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000561463 --yes-i-really-mean-it 2023-08-27T11:28:29.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.178+0000 7f7b5e560700 1 Processor -- start 2023-08-27T11:28:29.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.178+0000 7f7b5e560700 1 -- start start 2023-08-27T11:28:29.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.178+0000 7f7b5e560700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7b580ce8b0 0x7f7b5806adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.178+0000 7f7b5e560700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5815afe0 0x7f7b5806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.178+0000 7f7b5e560700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b580ce280 con 0x7f7b5806e120 2023-08-27T11:28:29.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b5e560700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5815b800 con 0x7f7b580ce8b0 2023-08-27T11:28:29.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b5e560700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5815b520 con 0x7f7b5815afe0 2023-08-27T11:28:29.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b577fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7b5806e120 0x7f7b5815a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59076/0 (socket says 172.21.15.112:59076) 2023-08-27T11:28:29.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b577fe700 1 -- 172.21.15.112:0/4157285063 learned_addr learned my addr 172.21.15.112:0/4157285063 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:29.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b5cafd700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5815afe0 0x7f7b5806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b57fff700 1 --2- 172.21.15.112:0/4157285063 >> v2:172.21.15.138:3300/0 conn(0x7f7b580ce8b0 0x7f7b5806adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568031683 0 0) 0x7f7b580ce280 con 0x7f7b5806e120 2023-08-27T11:28:29.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b400035b0 con 0x7f7b5806e120 2023-08-27T11:28:29.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1191345580 0 0) 0x7f7b400035b0 con 0x7f7b5806e120 2023-08-27T11:28:29.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5815afe0 msgr2=0x7f7b5806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:29.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5815afe0 0x7f7b5806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 >> v2:172.21.15.138:3300/0 conn(0x7f7b580ce8b0 msgr2=0x7f7b5806adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 --2- 172.21.15.112:0/4157285063 >> v2:172.21.15.138:3300/0 conn(0x7f7b580ce8b0 0x7f7b5806adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5806d640 con 0x7f7b5806e120 2023-08-27T11:28:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f7b4c001670 con 0x7f7b5806e120 2023-08-27T11:28:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b57fff700 1 --2- 172.21.15.112:0/4157285063 >> v2:172.21.15.138:3300/0 conn(0x7f7b580ce8b0 0x7f7b5806adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:29.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b5cafd700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5815afe0 0x7f7b5806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:29.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 >> v1:172.21.15.112:6789/0 conn(0x7f7b5806e120 legacy=0x7f7b5815a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b40003a00 0x7f7b40003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5815afe0 0x7f7b40004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b40004d90 0x7f7b40005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5815b800 con 0x7f7b40003a00 2023-08-27T11:28:29.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5815b520 con 0x7f7b40004d90 2023-08-27T11:28:29.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4000ba10 con 0x7f7b5815afe0 2023-08-27T11:28:29.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b57fff700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b40003a00 0x7f7b40003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b577fe700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5815afe0 0x7f7b40004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.179+0000 7f7b5cafd700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b40004d90 0x7f7b40005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b57fff700 1 -- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5815afe0 msgr2=0x7f7b40004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:29.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b57fff700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5815afe0 0x7f7b40004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b57fff700 1 -- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b40004d90 msgr2=0x7f7b40005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:29.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b57fff700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b40004d90 0x7f7b40005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b57fff700 1 -- 172.21.15.112:0/4157285063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b50009a70 con 0x7f7b40003a00 2023-08-27T11:28:29.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b577fe700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b5815afe0 0x7f7b40004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:29.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b5cafd700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b40004d90 0x7f7b40005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:29.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b57fff700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b40003a00 0x7f7b40003e20 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009d30 tx=0x7f7b40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4675bafe9cb997f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:29.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48009e80 con 0x7f7b40003a00 2023-08-27T11:28:29.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4800e9a0 con 0x7f7b40003a00 2023-08-27T11:28:29.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b56ffd700 1 -- 172.21.15.112:0/4157285063 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4800ed30 con 0x7f7b40003a00 2023-08-27T11:28:29.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b5e560700 1 -- 172.21.15.112:0/4157285063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b40003a00 msgr2=0x7f7b40003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b5e560700 1 --2- 172.21.15.112:0/4157285063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b40003a00 0x7f7b40003e20 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009d30 tx=0x7f7b40005880 comp rx=0 tx=0).stop 2023-08-27T11:28:29.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b5cafd700 1 -- 172.21.15.112:0/4157285063 reap_dead start 2023-08-27T11:28:29.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b5e560700 1 -- 172.21.15.112:0/4157285063 shutdown_connections 2023-08-27T11:28:29.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.180+0000 7f7b5e560700 1 -- 172.21.15.112:0/4157285063 >> 172.21.15.112:0/4157285063 conn(0x7f7b580ccef0 msgr2=0x7f7b580cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:29.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.181+0000 7f7b5e560700 1 -- 172.21.15.112:0/4157285063 shutdown_connections 2023-08-27T11:28:29.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.181+0000 7f7b5e560700 1 -- 172.21.15.112:0/4157285063 wait complete. 2023-08-27T11:28:29.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.182+0000 7f7b5e560700 1 Processor -- start 2023-08-27T11:28:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.182+0000 7f7b5e560700 1 -- start start 2023-08-27T11:28:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b5e560700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b40003a00 0x7f7b58158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b5e560700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b580af050 0x7f7b581558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b5e560700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b58155df0 0x7f7b580a85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b5e560700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b581562a0 con 0x7f7b580af050 2023-08-27T11:28:29.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b5e560700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b580a9340 con 0x7f7b40003a00 2023-08-27T11:28:29.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b5e560700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b580ae570 con 0x7f7b58155df0 2023-08-27T11:28:29.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b577fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b580af050 0x7f7b581558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b577fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b580af050 0x7f7b581558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36476/0 (socket says 172.21.15.112:36476) 2023-08-27T11:28:29.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b577fe700 1 -- 172.21.15.112:0/2615416131 learned_addr learned my addr 172.21.15.112:0/2615416131 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:29.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b57fff700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b40003a00 0x7f7b58158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b5cafd700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b58155df0 0x7f7b580a85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b577fe700 1 -- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b58155df0 msgr2=0x7f7b580a85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b577fe700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b58155df0 0x7f7b580a85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b577fe700 1 -- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b40003a00 msgr2=0x7f7b58158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b577fe700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b40003a00 0x7f7b58158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b577fe700 1 -- 172.21.15.112:0/2615416131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b580ae850 con 0x7f7b580af050 2023-08-27T11:28:29.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b5cafd700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b58155df0 0x7f7b580a85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:29.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b577fe700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b580af050 0x7f7b581558b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00eaf0 tx=0x7f7b4c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:29.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b54ff9700 1 -- 172.21.15.112:0/2615416131 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c00c780 con 0x7f7b580af050 2023-08-27T11:28:29.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b54ff9700 1 -- 172.21.15.112:0/2615416131 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c00c8e0 con 0x7f7b580af050 2023-08-27T11:28:29.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b5e560700 1 -- 172.21.15.112:0/2615416131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b580aeb30 con 0x7f7b580af050 2023-08-27T11:28:29.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b54ff9700 1 -- 172.21.15.112:0/2615416131 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c00cc50 con 0x7f7b580af050 2023-08-27T11:28:29.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.183+0000 7f7b5e560700 1 -- 172.21.15.112:0/2615416131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b58158c70 con 0x7f7b580af050 2023-08-27T11:28:29.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.184+0000 7f7b5e560700 1 -- 172.21.15.112:0/2615416131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b580aed50 con 0x7f7b580af050 2023-08-27T11:28:29.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.185+0000 7f7b54ff9700 1 -- 172.21.15.112:0/2615416131 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f7b4c01d460 con 0x7f7b580af050 2023-08-27T11:28:29.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.185+0000 7f7b54ff9700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b4406a4d0 0x7f7b4406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.185+0000 7f7b54ff9700 1 -- 172.21.15.112:0/2615416131 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7b4c088b70 con 0x7f7b580af050 2023-08-27T11:28:29.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.185+0000 7f7b57fff700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b4406a4d0 0x7f7b4406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.186+0000 7f7b57fff700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b4406a4d0 0x7f7b4406c990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7b48000c00 tx=0x7f7b48008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:29.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.189+0000 7f7b54ff9700 1 -- 172.21.15.112:0/2615416131 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7b4c053c80 con 0x7f7b580af050 2023-08-27T11:28:29.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:29.405+0000 7f7b5e560700 1 -- 172.21.15.112:0/2615416131 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000561463", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7b5806dc00 con 0x7f7b4406a4d0 2023-08-27T11:28:31.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.733+0000 7f7b54ff9700 1 -- 172.21.15.112:0/2615416131 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+56 (secure 0 0 0) 0x7f7b48022040 con 0x7f7b4406a4d0 2023-08-27T11:28:31.735 INFO:teuthology.orchestra.run.smithi112.stdout:FS volume 'cephfs' renamed to 'volume_0000000000561463' 2023-08-27T11:28:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.737+0000 7f7b5e560700 1 -- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b4406a4d0 msgr2=0x7f7b4406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.737+0000 7f7b5e560700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b4406a4d0 0x7f7b4406c990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7b48000c00 tx=0x7f7b48008030 comp rx=0 tx=0).stop 2023-08-27T11:28:31.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.737+0000 7f7b5e560700 1 -- 172.21.15.112:0/2615416131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b580af050 msgr2=0x7f7b581558b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.737+0000 7f7b5e560700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b580af050 0x7f7b581558b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00eaf0 tx=0x7f7b4c00ee00 comp rx=0 tx=0).stop 2023-08-27T11:28:31.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.737+0000 7f7b5e560700 1 -- 172.21.15.112:0/2615416131 shutdown_connections 2023-08-27T11:28:31.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.737+0000 7f7b5e560700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b4406a4d0 0x7f7b4406c990 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.737+0000 7f7b5e560700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b58155df0 0x7f7b580a85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.737+0000 7f7b5e560700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b40003a00 0x7f7b58158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.737+0000 7f7b5e560700 1 --2- 172.21.15.112:0/2615416131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b580af050 0x7f7b581558b0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.737+0000 7f7b5e560700 1 -- 172.21.15.112:0/2615416131 >> 172.21.15.112:0/2615416131 conn(0x7f7b580ccef0 msgr2=0x7f7b58156ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.737+0000 7f7b5e560700 1 -- 172.21.15.112:0/2615416131 shutdown_connections 2023-08-27T11:28:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.737+0000 7f7b5e560700 1 -- 172.21.15.112:0/2615416131 wait complete. 2023-08-27T11:28:31.753 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:28:31.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.878+0000 7f8e6cab3700 1 Processor -- start 2023-08-27T11:28:31.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.878+0000 7f8e6cab3700 1 -- start start 2023-08-27T11:28:31.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.879+0000 7f8e6cab3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8e680ad430 0x7f8e680ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.882 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.879+0000 7f8e6cab3700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8e680af000 0x7f8e680af450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.879+0000 7f8e6cab3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e681470e0 con 0x7f8e68146d00 2023-08-27T11:28:31.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.879+0000 7f8e6cab3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e680af990 con 0x7f8e680ad430 2023-08-27T11:28:31.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.879+0000 7f8e6cab3700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e680aeb70 con 0x7f8e680af000 2023-08-27T11:28:31.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.880+0000 7f8e65d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8e680ad430 0x7f8e680ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.880+0000 7f8e6659c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8e68146d00 0x7f8e680b3440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59096/0 (socket says 172.21.15.112:59096) 2023-08-27T11:28:31.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.880+0000 7f8e6659c700 1 -- 172.21.15.112:0/2642363829 learned_addr learned my addr 172.21.15.112:0/2642363829 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:31.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.880+0000 7f8e65d9b700 1 -- 172.21.15.112:0/2642363829 >> v2:172.21.15.138:3300/0 conn(0x7f8e680af000 msgr2=0x7f8e680af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:31.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.880+0000 7f8e65d9b700 1 --2- 172.21.15.112:0/2642363829 >> v2:172.21.15.138:3300/0 conn(0x7f8e680af000 0x7f8e680af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.880+0000 7f8e65d9b700 1 -- 172.21.15.112:0/2642363829 >> v1:172.21.15.112:6789/0 conn(0x7f8e68146d00 legacy=0x7f8e680b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.880+0000 7f8e65d9b700 1 -- 172.21.15.112:0/2642363829 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e680adf60 con 0x7f8e680ad430 2023-08-27T11:28:31.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.880+0000 7f8e6559a700 1 -- 172.21.15.112:0/2642363829 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712552549 0 0) 0x7f8e681470e0 con 0x7f8e68146d00 2023-08-27T11:28:31.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.880+0000 7f8e65d9b700 1 --2- 172.21.15.112:0/2642363829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e680ad430 0x7f8e680ad8e0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8e680cffd0 tx=0x7f8e50009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ddd60d0aab4023 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:31.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.880+0000 7f8e6559a700 1 -- 172.21.15.112:0/2642363829 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e50005e30 con 0x7f8e680ad430 2023-08-27T11:28:31.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.880+0000 7f8e6559a700 1 -- 172.21.15.112:0/2642363829 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e50006030 con 0x7f8e680ad430 2023-08-27T11:28:31.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.880+0000 7f8e6559a700 1 -- 172.21.15.112:0/2642363829 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e500063c0 con 0x7f8e680ad430 2023-08-27T11:28:31.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.881+0000 7f8e6cab3700 1 -- 172.21.15.112:0/2642363829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e680ad430 msgr2=0x7f8e680ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.881+0000 7f8e6cab3700 1 --2- 172.21.15.112:0/2642363829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e680ad430 0x7f8e680ad8e0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8e680cffd0 tx=0x7f8e50009b00 comp rx=0 tx=0).stop 2023-08-27T11:28:31.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.881+0000 7f8e6cab3700 1 -- 172.21.15.112:0/2642363829 shutdown_connections 2023-08-27T11:28:31.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.881+0000 7f8e6cab3700 1 --2- 172.21.15.112:0/2642363829 >> v2:172.21.15.138:3300/0 conn(0x7f8e680af000 0x7f8e680af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.881+0000 7f8e6cab3700 1 --2- 172.21.15.112:0/2642363829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e680ad430 0x7f8e680ad8e0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.881+0000 7f8e6cab3700 1 -- 172.21.15.112:0/2642363829 >> 172.21.15.112:0/2642363829 conn(0x7f8e6806adb0 msgr2=0x7f8e6806b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:31.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.881+0000 7f8e6cab3700 1 -- 172.21.15.112:0/2642363829 shutdown_connections 2023-08-27T11:28:31.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.881+0000 7f8e6cab3700 1 -- 172.21.15.112:0/2642363829 wait complete. 2023-08-27T11:28:31.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.882+0000 7f8e6cab3700 1 Processor -- start 2023-08-27T11:28:31.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.882+0000 7f8e6cab3700 1 -- start start 2023-08-27T11:28:31.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e6cab3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e680ad430 0x7f8e6815cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e6cab3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e680af000 0x7f8e68070b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e6cab3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e68146d00 0x7f8e68071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e6cab3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6815d060 con 0x7f8e680af000 2023-08-27T11:28:31.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e6cab3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e680ce1d0 con 0x7f8e68146d00 2023-08-27T11:28:31.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e6cab3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e680ce4b0 con 0x7f8e680ad430 2023-08-27T11:28:31.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e65d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e680af000 0x7f8e68070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e65d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e680af000 0x7f8e68070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36490/0 (socket says 172.21.15.112:36490) 2023-08-27T11:28:31.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e65d9b700 1 -- 172.21.15.112:0/121092608 learned_addr learned my addr 172.21.15.112:0/121092608 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:31.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e6659c700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e680ad430 0x7f8e6815cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e66d9d700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e68146d00 0x7f8e68071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e65d9b700 1 -- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e680ad430 msgr2=0x7f8e6815cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e65d9b700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e680ad430 0x7f8e6815cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e65d9b700 1 -- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e68146d00 msgr2=0x7f8e68071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e65d9b700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e68146d00 0x7f8e68071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e65d9b700 1 -- 172.21.15.112:0/121092608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e50009430 con 0x7f8e680af000 2023-08-27T11:28:31.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e66d9d700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e68146d00 0x7f8e68071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:31.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e6659c700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e680ad430 0x7f8e6815cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.883+0000 7f8e65d9b700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e680af000 0x7f8e68070b50 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f8e50000c00 tx=0x7f8e50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.884+0000 7f8e5effd700 1 -- 172.21.15.112:0/121092608 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e50004240 con 0x7f8e680af000 2023-08-27T11:28:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.884+0000 7f8e5effd700 1 -- 172.21.15.112:0/121092608 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e50020070 con 0x7f8e680af000 2023-08-27T11:28:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.884+0000 7f8e5effd700 1 -- 172.21.15.112:0/121092608 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e500045e0 con 0x7f8e680af000 2023-08-27T11:28:31.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.884+0000 7f8e6cab3700 1 -- 172.21.15.112:0/121092608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e680ce790 con 0x7f8e680af000 2023-08-27T11:28:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.884+0000 7f8e6cab3700 1 -- 172.21.15.112:0/121092608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e680ced50 con 0x7f8e680af000 2023-08-27T11:28:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.885+0000 7f8e6cab3700 1 -- 172.21.15.112:0/121092608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e680b1480 con 0x7f8e680af000 2023-08-27T11:28:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.885+0000 7f8e5effd700 1 -- 172.21.15.112:0/121092608 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f8e50022030 con 0x7f8e680af000 2023-08-27T11:28:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.885+0000 7f8e5effd700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8e4806a4d0 0x7f8e4806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.885+0000 7f8e5effd700 1 -- 172.21.15.112:0/121092608 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7f8e5008a390 con 0x7f8e680af000 2023-08-27T11:28:31.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.886+0000 7f8e6659c700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8e4806a4d0 0x7f8e4806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.886+0000 7f8e6659c700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8e4806a4d0 0x7f8e4806c990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8e580016e0 tx=0x7f8e58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:31.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:31.890+0000 7f8e5effd700 1 -- 172.21.15.112:0/121092608 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8e50055710 con 0x7f8e680af000 2023-08-27T11:28:32.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.107+0000 7f8e6cab3700 1 -- 172.21.15.112:0/121092608 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e680ae520 con 0x7f8e4806a4d0 2023-08-27T11:28:32.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.109+0000 7f8e5effd700 1 -- 172.21.15.112:0/121092608 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7f8e580013c0 con 0x7f8e4806a4d0 2023-08-27T11:28:32.110 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:28:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volume_0000000000561463" 2023-08-27T11:28:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:32.111 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:28:32.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.111+0000 7f8e6cab3700 1 -- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8e4806a4d0 msgr2=0x7f8e4806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.111+0000 7f8e6cab3700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8e4806a4d0 0x7f8e4806c990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8e580016e0 tx=0x7f8e58009040 comp rx=0 tx=0).stop 2023-08-27T11:28:32.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.111+0000 7f8e6cab3700 1 -- 172.21.15.112:0/121092608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e680af000 msgr2=0x7f8e68070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.111+0000 7f8e6cab3700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e680af000 0x7f8e68070b50 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f8e50000c00 tx=0x7f8e50004750 comp rx=0 tx=0).stop 2023-08-27T11:28:32.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.111+0000 7f8e6cab3700 1 -- 172.21.15.112:0/121092608 shutdown_connections 2023-08-27T11:28:32.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.111+0000 7f8e6cab3700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8e4806a4d0 0x7f8e4806c990 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.111+0000 7f8e6cab3700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e68146d00 0x7f8e68071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.111+0000 7f8e6cab3700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e680ad430 0x7f8e6815cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.111+0000 7f8e6cab3700 1 --2- 172.21.15.112:0/121092608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e680af000 0x7f8e68070b50 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.111+0000 7f8e6cab3700 1 -- 172.21.15.112:0/121092608 >> 172.21.15.112:0/121092608 conn(0x7f8e6806adb0 msgr2=0x7f8e68137830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:32.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.111+0000 7f8e6cab3700 1 -- 172.21.15.112:0/121092608 shutdown_connections 2023-08-27T11:28:32.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.111+0000 7f8e6cab3700 1 -- 172.21.15.112:0/121092608 wait complete. 2023-08-27T11:28:32.124 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:32.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.243+0000 7fd1c6829700 1 Processor -- start 2023-08-27T11:28:32.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.243+0000 7fd1c6829700 1 -- start start 2023-08-27T11:28:32.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1c6829700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1c0141b60 0x7fd1c00cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1c6829700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd1c0144a40 0x7fd1c00cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1c6829700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd1c00cc8e0 0x7fd1c006bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1c6829700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c00ccd50 con 0x7fd1c0144a40 2023-08-27T11:28:32.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1c6829700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c015f3d0 con 0x7fd1c00cc8e0 2023-08-27T11:28:32.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1c6829700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c015f6b0 con 0x7fd1c0141b60 2023-08-27T11:28:32.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1bf7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd1c0144a40 0x7fd1c00cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1bf7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd1c0144a40 0x7fd1c00cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36498/0 (socket says 172.21.15.112:36498) 2023-08-27T11:28:32.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1bf7fe700 1 -- 172.21.15.112:0/2584979102 learned_addr learned my addr 172.21.15.112:0/2584979102 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:32.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1bffff700 1 --2- 172.21.15.112:0/2584979102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1c0141b60 0x7fd1c00cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1c4dc6700 1 --2- 172.21.15.112:0/2584979102 >> v2:172.21.15.138:3300/0 conn(0x7fd1c00cc8e0 0x7fd1c006bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1bf7fe700 1 -- 172.21.15.112:0/2584979102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1c0141b60 msgr2=0x7fd1c00cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:32.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1bf7fe700 1 --2- 172.21.15.112:0/2584979102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1c0141b60 0x7fd1c00cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1bf7fe700 1 -- 172.21.15.112:0/2584979102 >> v2:172.21.15.138:3300/0 conn(0x7fd1c00cc8e0 msgr2=0x7fd1c006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:32.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1bf7fe700 1 --2- 172.21.15.112:0/2584979102 >> v2:172.21.15.138:3300/0 conn(0x7fd1c00cc8e0 0x7fd1c006bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1bf7fe700 1 -- 172.21.15.112:0/2584979102 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c015f990 con 0x7fd1c0144a40 2023-08-27T11:28:32.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1c4dc6700 1 --2- 172.21.15.112:0/2584979102 >> v2:172.21.15.138:3300/0 conn(0x7fd1c00cc8e0 0x7fd1c006bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:32.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.244+0000 7fd1bffff700 1 --2- 172.21.15.112:0/2584979102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1c0141b60 0x7fd1c00cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:32.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.245+0000 7fd1bf7fe700 1 --2- 172.21.15.112:0/2584979102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c0144a40 0x7fd1c00cc3a0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00cbf40 tx=0x7fd1b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62d80a1f1fcdce3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:32.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.245+0000 7fd1beffd700 1 -- 172.21.15.112:0/2584979102 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b000b870 con 0x7fd1c0144a40 2023-08-27T11:28:32.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.245+0000 7fd1beffd700 1 -- 172.21.15.112:0/2584979102 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b000b9d0 con 0x7fd1c0144a40 2023-08-27T11:28:32.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.245+0000 7fd1beffd700 1 -- 172.21.15.112:0/2584979102 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b000bd60 con 0x7fd1c0144a40 2023-08-27T11:28:32.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.245+0000 7fd1c6829700 1 -- 172.21.15.112:0/2584979102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c0144a40 msgr2=0x7fd1c00cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.245+0000 7fd1c6829700 1 --2- 172.21.15.112:0/2584979102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c0144a40 0x7fd1c00cc3a0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00cbf40 tx=0x7fd1b0009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.245+0000 7fd1c6829700 1 -- 172.21.15.112:0/2584979102 shutdown_connections 2023-08-27T11:28:32.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.245+0000 7fd1c6829700 1 --2- 172.21.15.112:0/2584979102 >> v2:172.21.15.138:3300/0 conn(0x7fd1c00cc8e0 0x7fd1c006bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.245+0000 7fd1c6829700 1 --2- 172.21.15.112:0/2584979102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1c0141b60 0x7fd1c00cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.245+0000 7fd1c6829700 1 --2- 172.21.15.112:0/2584979102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c0144a40 0x7fd1c00cc3a0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.245+0000 7fd1c6829700 1 -- 172.21.15.112:0/2584979102 >> 172.21.15.112:0/2584979102 conn(0x7fd1c0066010 msgr2=0x7fd1c00663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:32.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.245+0000 7fd1c6829700 1 -- 172.21.15.112:0/2584979102 shutdown_connections 2023-08-27T11:28:32.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.245+0000 7fd1c6829700 1 -- 172.21.15.112:0/2584979102 wait complete. 2023-08-27T11:28:32.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.246+0000 7fd1c6829700 1 Processor -- start 2023-08-27T11:28:32.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.247+0000 7fd1c6829700 1 -- start start 2023-08-27T11:28:32.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.247+0000 7fd1c6829700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1c00cc8e0 0x7fd1c013f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.247+0000 7fd1c6829700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c0141b60 0x7fd1c01406f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.247+0000 7fd1c6829700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1c0144a40 0x7fd1c01e79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.247+0000 7fd1c6829700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c015cb30 con 0x7fd1c0141b60 2023-08-27T11:28:32.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.247+0000 7fd1c6829700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c015cdf0 con 0x7fd1c0144a40 2023-08-27T11:28:32.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.247+0000 7fd1c6829700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c01411a0 con 0x7fd1c00cc8e0 2023-08-27T11:28:32.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.247+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c0141b60 0x7fd1c01406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c0141b60 0x7fd1c01406f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36506/0 (socket says 172.21.15.112:36506) 2023-08-27T11:28:32.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1bf7fe700 1 -- 172.21.15.112:0/2457693043 learned_addr learned my addr 172.21.15.112:0/2457693043 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:32.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1bffff700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1c00cc8e0 0x7fd1c013f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1c4dc6700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1c0144a40 0x7fd1c01e79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1bf7fe700 1 -- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1c00cc8e0 msgr2=0x7fd1c013f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1bf7fe700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1c00cc8e0 0x7fd1c013f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1bf7fe700 1 -- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1c0144a40 msgr2=0x7fd1c01e79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1bf7fe700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1c0144a40 0x7fd1c01e79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1bf7fe700 1 -- 172.21.15.112:0/2457693043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b0009550 con 0x7fd1c0141b60 2023-08-27T11:28:32.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1bffff700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1c00cc8e0 0x7fd1c013f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:32.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1bf7fe700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c0141b60 0x7fd1c01406f0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009c10 tx=0x7fd1b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:32.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1bcff9700 1 -- 172.21.15.112:0/2457693043 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0004590 con 0x7fd1c0141b60 2023-08-27T11:28:32.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1bcff9700 1 -- 172.21.15.112:0/2457693043 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b00036a0 con 0x7fd1c0141b60 2023-08-27T11:28:32.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1c6829700 1 -- 172.21.15.112:0/2457693043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c00b61a0 con 0x7fd1c0141b60 2023-08-27T11:28:32.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1bcff9700 1 -- 172.21.15.112:0/2457693043 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0003a50 con 0x7fd1c0141b60 2023-08-27T11:28:32.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.248+0000 7fd1c6829700 1 -- 172.21.15.112:0/2457693043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c00b6760 con 0x7fd1c0141b60 2023-08-27T11:28:32.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.249+0000 7fd1c6829700 1 -- 172.21.15.112:0/2457693043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c015cf60 con 0x7fd1c0141b60 2023-08-27T11:28:32.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.250+0000 7fd1bcff9700 1 -- 172.21.15.112:0/2457693043 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fd1b0003bb0 con 0x7fd1c0141b60 2023-08-27T11:28:32.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.250+0000 7fd1bcff9700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd1ac06a520 0x7fd1ac06c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.250+0000 7fd1bcff9700 1 -- 172.21.15.112:0/2457693043 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7fd1b0089e70 con 0x7fd1c0141b60 2023-08-27T11:28:32.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.250+0000 7fd1bffff700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd1ac06a520 0x7fd1ac06c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.251+0000 7fd1bffff700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd1ac06a520 0x7fd1ac06c9e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40099f0 tx=0x7fd1b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:32.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.254+0000 7fd1bcff9700 1 -- 172.21.15.112:0/2457693043 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd1b00551d0 con 0x7fd1c0141b60 2023-08-27T11:28:32.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.489+0000 7fd1c6829700 1 -- 172.21.15.112:0/2457693043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1c0067a20 con 0x7fd1c0141b60 2023-08-27T11:28:32.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.489+0000 7fd1bcff9700 1 -- 172.21.15.112:0/2457693043 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4902 (secure 0 0 0) 0x7fd1b0003e60 con 0x7fd1c0141b60 2023-08-27T11:28:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:32.492 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/739970856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":739970856},{"type":"v1","addr":"172.21.15.138:6837","nonce":739970856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3233069521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3233069521},{"type":"v1","addr":"172.21.15.112:6834","nonce":3233069521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2559241026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2559241026},{"type":"v1","addr":"172.21.15.112:6835","nonce":2559241026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:14.032146+0000","modified":"2023-08-27T11:28:29.716445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6836/529571603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":529571603},{"type":"v1","addr":"172.21.15.138:6836","nonce":529571603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"volume_0000000000561463","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-08-27T11:28:32.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.491+0000 7fd1c6829700 1 -- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd1ac06a520 msgr2=0x7fd1ac06c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.491+0000 7fd1c6829700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd1ac06a520 0x7fd1ac06c9e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40099f0 tx=0x7fd1b4008040 comp rx=0 tx=0).stop 2023-08-27T11:28:32.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.491+0000 7fd1c6829700 1 -- 172.21.15.112:0/2457693043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c0141b60 msgr2=0x7fd1c01406f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.491+0000 7fd1c6829700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c0141b60 0x7fd1c01406f0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009c10 tx=0x7fd1b0004750 comp rx=0 tx=0).stop 2023-08-27T11:28:32.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.492+0000 7fd1c6829700 1 -- 172.21.15.112:0/2457693043 shutdown_connections 2023-08-27T11:28:32.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.492+0000 7fd1c6829700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd1ac06a520 0x7fd1ac06c9e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.492+0000 7fd1c6829700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1c0144a40 0x7fd1c01e79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.492+0000 7fd1c6829700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1c00cc8e0 0x7fd1c013f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.492+0000 7fd1c6829700 1 --2- 172.21.15.112:0/2457693043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c0141b60 0x7fd1c01406f0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.492+0000 7fd1c6829700 1 -- 172.21.15.112:0/2457693043 >> 172.21.15.112:0/2457693043 conn(0x7fd1c0066010 msgr2=0x7fd1c00721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:32.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.492+0000 7fd1c6829700 1 -- 172.21.15.112:0/2457693043 shutdown_connections 2023-08-27T11:28:32.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.492+0000 7fd1c6829700 1 -- 172.21.15.112:0/2457693043 wait complete. 2023-08-27T11:28:32.496 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 96 2023-08-27T11:28:32.505 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:32.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.624+0000 7f96491bd700 1 Processor -- start 2023-08-27T11:28:32.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.624+0000 7f96491bd700 1 -- start start 2023-08-27T11:28:32.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.624+0000 7f96491bd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f96440b17e0 0x7f96440cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.624+0000 7f96491bd700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9644161850 con 0x7f96440b1400 2023-08-27T11:28:32.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.624+0000 7f96491bd700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96440cfc10 con 0x7f9644161ce0 2023-08-27T11:28:32.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.624+0000 7f96491bd700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9644161200 con 0x7f96440b17e0 2023-08-27T11:28:32.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.625+0000 7f9642d9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f96440b1400 0x7f964406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59124/0 (socket says 172.21.15.112:59124) 2023-08-27T11:28:32.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.625+0000 7f9642d9d700 1 -- 172.21.15.112:0/927106355 learned_addr learned my addr 172.21.15.112:0/927106355 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:32.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.625+0000 7f964259c700 1 --2- 172.21.15.112:0/927106355 >> v2:172.21.15.138:3300/0 conn(0x7f96440b17e0 0x7f96440cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.625+0000 7f9641d9b700 1 -- 172.21.15.112:0/927106355 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941532414 0 0) 0x7f9644161850 con 0x7f96440b1400 2023-08-27T11:28:32.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.625+0000 7f9641d9b700 1 -- 172.21.15.112:0/927106355 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f962c0035b0 con 0x7f96440b1400 2023-08-27T11:28:32.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.625+0000 7f964259c700 1 -- 172.21.15.112:0/927106355 >> v1:172.21.15.138:6789/0 conn(0x7f9644161ce0 legacy=0x7f964406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.625+0000 7f964259c700 1 -- 172.21.15.112:0/927106355 >> v1:172.21.15.112:6789/0 conn(0x7f96440b1400 legacy=0x7f964406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.625+0000 7f9641d9b700 1 -- 172.21.15.112:0/927106355 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 778069715 0 0) 0x7f962c0035b0 con 0x7f96440b1400 2023-08-27T11:28:32.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.625+0000 7f964259c700 1 -- 172.21.15.112:0/927106355 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96441614e0 con 0x7f96440b17e0 2023-08-27T11:28:32.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.625+0000 7f964259c700 1 --2- 172.21.15.112:0/927106355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96440b17e0 0x7f96440cf5a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9638009840 tx=0x7f9638009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=186530a4d524d95f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:32.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.625+0000 7f9641d9b700 1 -- 172.21.15.112:0/927106355 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634001670 con 0x7f96440b17e0 2023-08-27T11:28:32.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.625+0000 7f9641d9b700 1 -- 172.21.15.112:0/927106355 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9638004030 con 0x7f96440b17e0 2023-08-27T11:28:32.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.625+0000 7f9641d9b700 1 -- 172.21.15.112:0/927106355 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96380043a0 con 0x7f96440b17e0 2023-08-27T11:28:32.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.626+0000 7f96491bd700 1 -- 172.21.15.112:0/927106355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96440b17e0 msgr2=0x7f96440cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.626+0000 7f96491bd700 1 --2- 172.21.15.112:0/927106355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96440b17e0 0x7f96440cf5a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9638009840 tx=0x7f9638009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:32.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.626+0000 7f96491bd700 1 -- 172.21.15.112:0/927106355 shutdown_connections 2023-08-27T11:28:32.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.626+0000 7f96491bd700 1 --2- 172.21.15.112:0/927106355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96440b17e0 0x7f96440cf5a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.626+0000 7f96491bd700 1 -- 172.21.15.112:0/927106355 >> 172.21.15.112:0/927106355 conn(0x7f96440d0000 msgr2=0x7f96440d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:32.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.626+0000 7f96491bd700 1 -- 172.21.15.112:0/927106355 shutdown_connections 2023-08-27T11:28:32.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.626+0000 7f96491bd700 1 -- 172.21.15.112:0/927106355 wait complete. 2023-08-27T11:28:32.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.627+0000 7f96491bd700 1 Processor -- start 2023-08-27T11:28:32.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.627+0000 7f96491bd700 1 -- start start 2023-08-27T11:28:32.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.628+0000 7f96491bd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b1400 0x7f9644144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.628+0000 7f96491bd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96440b17e0 0x7f96440c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.628+0000 7f96491bd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9644161ce0 0x7f96440cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.628+0000 7f96491bd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96440ad6a0 con 0x7f96440b17e0 2023-08-27T11:28:32.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.628+0000 7f96491bd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96440aca20 con 0x7f9644161ce0 2023-08-27T11:28:32.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.628+0000 7f96491bd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96440acd00 con 0x7f96440b1400 2023-08-27T11:28:32.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.628+0000 7f964259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96440b17e0 0x7f96440c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.628+0000 7f964259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96440b17e0 0x7f96440c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36518/0 (socket says 172.21.15.112:36518) 2023-08-27T11:28:32.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.628+0000 7f964259c700 1 -- 172.21.15.112:0/3270155099 learned_addr learned my addr 172.21.15.112:0/3270155099 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:32.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f9642d9d700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b1400 0x7f9644144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f964359e700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9644161ce0 0x7f96440cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f964259c700 1 -- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b1400 msgr2=0x7f9644144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f964259c700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b1400 0x7f9644144d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f964259c700 1 -- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9644161ce0 msgr2=0x7f96440cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f964259c700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9644161ce0 0x7f96440cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f964259c700 1 -- 172.21.15.112:0/3270155099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96380094a0 con 0x7f96440b17e0 2023-08-27T11:28:32.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f9642d9d700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b1400 0x7f9644144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:32.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f964259c700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96440b17e0 0x7f96440c5ea0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f96380090b0 tx=0x7f9638004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:32.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f962b7fe700 1 -- 172.21.15.112:0/3270155099 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638011630 con 0x7f96440b17e0 2023-08-27T11:28:32.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f962b7fe700 1 -- 172.21.15.112:0/3270155099 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963801f070 con 0x7f96440b17e0 2023-08-27T11:28:32.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f96491bd700 1 -- 172.21.15.112:0/3270155099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96440acfe0 con 0x7f96440b17e0 2023-08-27T11:28:32.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f962b7fe700 1 -- 172.21.15.112:0/3270155099 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96380119d0 con 0x7f96440b17e0 2023-08-27T11:28:32.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.629+0000 7f96491bd700 1 -- 172.21.15.112:0/3270155099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96440c63e0 con 0x7f96440b17e0 2023-08-27T11:28:32.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.630+0000 7f96491bd700 1 -- 172.21.15.112:0/3270155099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96440ad7e0 con 0x7f96440b17e0 2023-08-27T11:28:32.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.630+0000 7f962b7fe700 1 -- 172.21.15.112:0/3270155099 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f9638011b30 con 0x7f96440b17e0 2023-08-27T11:28:32.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.631+0000 7f962b7fe700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f963006a4d0 0x7f963006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.631+0000 7f962b7fe700 1 -- 172.21.15.112:0/3270155099 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7f9638021070 con 0x7f96440b17e0 2023-08-27T11:28:32.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.631+0000 7f9642d9d700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f963006a4d0 0x7f963006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.631+0000 7f9642d9d700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f963006a4d0 0x7f963006c990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9634007960 tx=0x7f9634009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:32.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.635+0000 7f962b7fe700 1 -- 172.21.15.112:0/3270155099 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9638055750 con 0x7f96440b17e0 2023-08-27T11:28:32.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.829+0000 7f96491bd700 1 -- 172.21.15.112:0/3270155099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9644067a20 con 0x7f96440b17e0 2023-08-27T11:28:32.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.830+0000 7f962b7fe700 1 -- 172.21.15.112:0/3270155099 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+18341 (secure 0 0 0) 0x7f9638006cc0 con 0x7f96440b17e0 2023-08-27T11:28:32.832 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:32.832 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":142,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:31.728440+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs.volume_0000000000561463.meta","create_time":"2023-08-27T11:28:12.176044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000561463"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs.volume_0000000000561463.data","create_time":"2023-08-27T11:28:12.992996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000561463"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:32.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.832+0000 7f96491bd700 1 -- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f963006a4d0 msgr2=0x7f963006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.832+0000 7f96491bd700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f963006a4d0 0x7f963006c990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9634007960 tx=0x7f9634009040 comp rx=0 tx=0).stop 2023-08-27T11:28:32.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.832+0000 7f96491bd700 1 -- 172.21.15.112:0/3270155099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96440b17e0 msgr2=0x7f96440c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.832+0000 7f96491bd700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96440b17e0 0x7f96440c5ea0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f96380090b0 tx=0x7f9638004750 comp rx=0 tx=0).stop 2023-08-27T11:28:32.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.832+0000 7f96491bd700 1 -- 172.21.15.112:0/3270155099 shutdown_connections 2023-08-27T11:28:32.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.832+0000 7f96491bd700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f963006a4d0 0x7f963006c990 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.832+0000 7f96491bd700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9644161ce0 0x7f96440cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.832+0000 7f96491bd700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b1400 0x7f9644144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.832+0000 7f96491bd700 1 --2- 172.21.15.112:0/3270155099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96440b17e0 0x7f96440c5ea0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.832+0000 7f96491bd700 1 -- 172.21.15.112:0/3270155099 >> 172.21.15.112:0/3270155099 conn(0x7f96440d0000 msgr2=0x7f96441531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:32.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.832+0000 7f96491bd700 1 -- 172.21.15.112:0/3270155099 shutdown_connections 2023-08-27T11:28:32.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:32.832+0000 7f96491bd700 1 -- 172.21.15.112:0/3270155099 wait complete. 2023-08-27T11:28:32.846 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:28:32.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:28:32.888 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:28:32.937 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:28:32.937 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:28:32.938 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:32.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:28:32.955 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:28:32.955 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:32.975 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:28:32.975 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:28:32.976 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:33.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:28:33.013 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:28:33.013 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:33.034 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:28:33.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:33.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.170+0000 7fa9176b1700 1 Processor -- start 2023-08-27T11:28:33.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.170+0000 7fa9176b1700 1 -- start start 2023-08-27T11:28:33.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.171+0000 7fa9176b1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9100b1400 0x7fa9100cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.171+0000 7fa9176b1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa9100b17e0 0x7fa91006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.171+0000 7fa9176b1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa910161850 con 0x7fa910161ce0 2023-08-27T11:28:33.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.171+0000 7fa9176b1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91006e7d0 con 0x7fa9100b1400 2023-08-27T11:28:33.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.171+0000 7fa9176b1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9100cfc10 con 0x7fa9100b17e0 2023-08-27T11:28:33.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.171+0000 7fa91544d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9100b1400 0x7fa9100cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.171+0000 7fa915c4e700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa910161ce0 0x7fa91006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59142/0 (socket says 172.21.15.112:59142) 2023-08-27T11:28:33.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.171+0000 7fa91544d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9100b1400 0x7fa9100cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36524/0 (socket says 172.21.15.112:36524) 2023-08-27T11:28:33.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.171+0000 7fa915c4e700 1 -- 172.21.15.112:0/4213659940 learned_addr learned my addr 172.21.15.112:0/4213659940 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:33.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa914c4c700 1 --2- 172.21.15.112:0/4213659940 >> v2:172.21.15.138:3300/0 conn(0x7fa9100b17e0 0x7fa91006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa91544d700 1 -- 172.21.15.112:0/4213659940 >> v2:172.21.15.138:3300/0 conn(0x7fa9100b17e0 msgr2=0x7fa91006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:33.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa91544d700 1 --2- 172.21.15.112:0/4213659940 >> v2:172.21.15.138:3300/0 conn(0x7fa9100b17e0 0x7fa91006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa91544d700 1 -- 172.21.15.112:0/4213659940 >> v1:172.21.15.112:6789/0 conn(0x7fa910161ce0 legacy=0x7fa91006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa907fff700 1 -- 172.21.15.112:0/4213659940 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608889603 0 0) 0x7fa910161850 con 0x7fa910161ce0 2023-08-27T11:28:33.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa91544d700 1 -- 172.21.15.112:0/4213659940 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa910161200 con 0x7fa9100b1400 2023-08-27T11:28:33.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa914c4c700 1 --2- 172.21.15.112:0/4213659940 >> v2:172.21.15.138:3300/0 conn(0x7fa9100b17e0 0x7fa91006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:33.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa91544d700 1 --2- 172.21.15.112:0/4213659940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9100b1400 0x7fa9100cf5a0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fa9100cc420 tx=0x7fa900009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33692194289328d0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:33.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa907fff700 1 -- 172.21.15.112:0/4213659940 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900011830 con 0x7fa9100b1400 2023-08-27T11:28:33.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa907fff700 1 -- 172.21.15.112:0/4213659940 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa900011990 con 0x7fa9100b1400 2023-08-27T11:28:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa907fff700 1 -- 172.21.15.112:0/4213659940 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900011d20 con 0x7fa9100b1400 2023-08-27T11:28:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa9176b1700 1 -- 172.21.15.112:0/4213659940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9100b1400 msgr2=0x7fa9100cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa9176b1700 1 --2- 172.21.15.112:0/4213659940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9100b1400 0x7fa9100cf5a0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fa9100cc420 tx=0x7fa900009b20 comp rx=0 tx=0).stop 2023-08-27T11:28:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa9176b1700 1 -- 172.21.15.112:0/4213659940 shutdown_connections 2023-08-27T11:28:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa9176b1700 1 --2- 172.21.15.112:0/4213659940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9100b1400 0x7fa9100cf5a0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa9176b1700 1 --2- 172.21.15.112:0/4213659940 >> v2:172.21.15.138:3300/0 conn(0x7fa9100b17e0 0x7fa91006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.172+0000 7fa9176b1700 1 -- 172.21.15.112:0/4213659940 >> 172.21.15.112:0/4213659940 conn(0x7fa9100d0000 msgr2=0x7fa9100d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:33.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.173+0000 7fa9176b1700 1 -- 172.21.15.112:0/4213659940 shutdown_connections 2023-08-27T11:28:33.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.173+0000 7fa9176b1700 1 -- 172.21.15.112:0/4213659940 wait complete. 2023-08-27T11:28:33.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.174+0000 7fa9176b1700 1 Processor -- start 2023-08-27T11:28:33.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.174+0000 7fa9176b1700 1 -- start start 2023-08-27T11:28:33.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.174+0000 7fa9176b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9100b1400 0x7fa9100c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.174+0000 7fa9176b1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9100b17e0 0x7fa9101f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.174+0000 7fa9176b1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa910161ce0 0x7fa91006ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.174+0000 7fa9176b1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100c4c90 con 0x7fa9100b1400 2023-08-27T11:28:33.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.174+0000 7fa9176b1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100c4080 con 0x7fa910161ce0 2023-08-27T11:28:33.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa9176b1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100c4360 con 0x7fa9100b17e0 2023-08-27T11:28:33.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa91544d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9100b1400 0x7fa9100c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa91544d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9100b1400 0x7fa9100c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36530/0 (socket says 172.21.15.112:36530) 2023-08-27T11:28:33.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa91544d700 1 -- 172.21.15.112:0/504486570 learned_addr learned my addr 172.21.15.112:0/504486570 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:33.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa914c4c700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9100b17e0 0x7fa9101f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa915c4e700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa910161ce0 0x7fa91006ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa91544d700 1 -- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9100b17e0 msgr2=0x7fa9101f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa91544d700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9100b17e0 0x7fa9101f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa91544d700 1 -- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa910161ce0 msgr2=0x7fa91006ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa91544d700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa910161ce0 0x7fa91006ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa91544d700 1 -- 172.21.15.112:0/504486570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9000094a0 con 0x7fa9100b1400 2023-08-27T11:28:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa914c4c700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9100b17e0 0x7fa9101f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa91544d700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9100b1400 0x7fa9100c5950 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fa900000c00 tx=0x7fa900004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa905ffb700 1 -- 172.21.15.112:0/504486570 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9000038a0 con 0x7fa9100b1400 2023-08-27T11:28:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa905ffb700 1 -- 172.21.15.112:0/504486570 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90001f070 con 0x7fa9100b1400 2023-08-27T11:28:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa905ffb700 1 -- 172.21.15.112:0/504486570 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900003c40 con 0x7fa9100b1400 2023-08-27T11:28:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa9176b1700 1 -- 172.21.15.112:0/504486570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9100c4640 con 0x7fa9100b1400 2023-08-27T11:28:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.175+0000 7fa9176b1700 1 -- 172.21.15.112:0/504486570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9100c5e90 con 0x7fa9100b1400 2023-08-27T11:28:33.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.176+0000 7fa9176b1700 1 -- 172.21.15.112:0/504486570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa910067a20 con 0x7fa9100b1400 2023-08-27T11:28:33.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.177+0000 7fa905ffb700 1 -- 172.21.15.112:0/504486570 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa900026480 con 0x7fa9100b1400 2023-08-27T11:28:33.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.177+0000 7fa905ffb700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa8f406a4d0 0x7fa8f406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.177+0000 7fa905ffb700 1 -- 172.21.15.112:0/504486570 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7fa900021070 con 0x7fa9100b1400 2023-08-27T11:28:33.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.177+0000 7fa914c4c700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa8f406a4d0 0x7fa8f406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.178+0000 7fa914c4c700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa8f406a4d0 0x7fa8f406c990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa910155a90 tx=0x7fa90c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:33.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.181+0000 7fa905ffb700 1 -- 172.21.15.112:0/504486570 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa900055910 con 0x7fa9100b1400 2023-08-27T11:28:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.411+0000 7fa9176b1700 1 -- 172.21.15.112:0/504486570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa91006db60 con 0x7fa9100b1400 2023-08-27T11:28:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.412+0000 7fa905ffb700 1 -- 172.21.15.112:0/504486570 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4902 (secure 0 0 0) 0x7fa900058f30 con 0x7fa9100b1400 2023-08-27T11:28:33.414 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:33.414 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/739970856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":739970856},{"type":"v1","addr":"172.21.15.138:6837","nonce":739970856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3233069521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3233069521},{"type":"v1","addr":"172.21.15.112:6834","nonce":3233069521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2559241026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2559241026},{"type":"v1","addr":"172.21.15.112:6835","nonce":2559241026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:14.032146+0000","modified":"2023-08-27T11:28:29.716445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6836/529571603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":529571603},{"type":"v1","addr":"172.21.15.138:6836","nonce":529571603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"volume_0000000000561463","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-08-27T11:28:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.414+0000 7fa9176b1700 1 -- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa8f406a4d0 msgr2=0x7fa8f406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.414+0000 7fa9176b1700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa8f406a4d0 0x7fa8f406c990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa910155a90 tx=0x7fa90c008040 comp rx=0 tx=0).stop 2023-08-27T11:28:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.414+0000 7fa9176b1700 1 -- 172.21.15.112:0/504486570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9100b1400 msgr2=0x7fa9100c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.414+0000 7fa9176b1700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9100b1400 0x7fa9100c5950 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fa900000c00 tx=0x7fa900004750 comp rx=0 tx=0).stop 2023-08-27T11:28:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.414+0000 7fa9176b1700 1 -- 172.21.15.112:0/504486570 shutdown_connections 2023-08-27T11:28:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.414+0000 7fa9176b1700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa8f406a4d0 0x7fa8f406c990 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.414+0000 7fa9176b1700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa910161ce0 0x7fa91006ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.414+0000 7fa9176b1700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9100b1400 0x7fa9100c5950 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.414+0000 7fa9176b1700 1 --2- 172.21.15.112:0/504486570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9100b17e0 0x7fa9101f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.414+0000 7fa9176b1700 1 -- 172.21.15.112:0/504486570 >> 172.21.15.112:0/504486570 conn(0x7fa9100d0000 msgr2=0x7fa9100cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:33.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.414+0000 7fa9176b1700 1 -- 172.21.15.112:0/504486570 shutdown_connections 2023-08-27T11:28:33.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.414+0000 7fa9176b1700 1 -- 172.21.15.112:0/504486570 wait complete. 2023-08-27T11:28:33.418 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 96 2023-08-27T11:28:33.428 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:33.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.547+0000 7f5f7e928700 1 Processor -- start 2023-08-27T11:28:33.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.547+0000 7f5f7e928700 1 -- start start 2023-08-27T11:28:33.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.548+0000 7f5f7e928700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5f78158e20 0x7f5f780b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.548+0000 7f5f7e928700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f7806fea0 0x7f5f780af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.548+0000 7f5f7e928700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7815c190 con 0x7f5f7806e3a0 2023-08-27T11:28:33.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.548+0000 7f5f7e928700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f780b1730 con 0x7f5f78158e20 2023-08-27T11:28:33.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.548+0000 7f5f7e928700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f78159200 con 0x7f5f7806fea0 2023-08-27T11:28:33.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.548+0000 7f5f77fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5f78158e20 0x7f5f780b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.548+0000 7f5f77fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5f78158e20 0x7f5f780b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36538/0 (socket says 172.21.15.112:36538) 2023-08-27T11:28:33.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.548+0000 7f5f77fff700 1 -- 172.21.15.112:0/2328380175 learned_addr learned my addr 172.21.15.112:0/2328380175 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:33.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f7cec5700 1 --2- 172.21.15.112:0/2328380175 >> v2:172.21.15.138:3300/0 conn(0x7f5f7806fea0 0x7f5f780af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f77fff700 1 -- 172.21.15.112:0/2328380175 >> v2:172.21.15.138:3300/0 conn(0x7f5f7806fea0 msgr2=0x7f5f780af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:33.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f77fff700 1 --2- 172.21.15.112:0/2328380175 >> v2:172.21.15.138:3300/0 conn(0x7f5f7806fea0 0x7f5f780af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f77fff700 1 -- 172.21.15.112:0/2328380175 >> v1:172.21.15.138:6789/0 conn(0x7f5f7806e3a0 legacy=0x7f5f7806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:33.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f77fff700 1 -- 172.21.15.112:0/2328380175 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f780b1910 con 0x7f5f78158e20 2023-08-27T11:28:33.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f77fff700 1 --2- 172.21.15.112:0/2328380175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f78158e20 0x7f5f780b11f0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f5f7806dba0 tx=0x7f5f6c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768ad0ed4d0baefe server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:33.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f76ffd700 1 -- 172.21.15.112:0/2328380175 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004230 con 0x7f5f78158e20 2023-08-27T11:28:33.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f76ffd700 1 -- 172.21.15.112:0/2328380175 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c004390 con 0x7f5f78158e20 2023-08-27T11:28:33.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f76ffd700 1 -- 172.21.15.112:0/2328380175 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c0036a0 con 0x7f5f78158e20 2023-08-27T11:28:33.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f7e928700 1 -- 172.21.15.112:0/2328380175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f78158e20 msgr2=0x7f5f780b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f7e928700 1 --2- 172.21.15.112:0/2328380175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f78158e20 0x7f5f780b11f0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f5f7806dba0 tx=0x7f5f6c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f7e928700 1 -- 172.21.15.112:0/2328380175 shutdown_connections 2023-08-27T11:28:33.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f7e928700 1 --2- 172.21.15.112:0/2328380175 >> v2:172.21.15.138:3300/0 conn(0x7f5f7806fea0 0x7f5f780af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f7e928700 1 --2- 172.21.15.112:0/2328380175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f78158e20 0x7f5f780b11f0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f7e928700 1 -- 172.21.15.112:0/2328380175 >> 172.21.15.112:0/2328380175 conn(0x7f5f780d0000 msgr2=0x7f5f780d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:33.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.549+0000 7f5f7e928700 1 -- 172.21.15.112:0/2328380175 shutdown_connections 2023-08-27T11:28:33.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.550+0000 7f5f7e928700 1 -- 172.21.15.112:0/2328380175 wait complete. 2023-08-27T11:28:33.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.551+0000 7f5f7e928700 1 Processor -- start 2023-08-27T11:28:33.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.551+0000 7f5f7e928700 1 -- start start 2023-08-27T11:28:33.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.551+0000 7f5f7e928700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7806e3a0 0x7f5f78071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.551+0000 7f5f7e928700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f7806fea0 0x7f5f780c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.551+0000 7f5f7e928700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f78158e20 0x7f5f780c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f7e928700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f78153d80 con 0x7f5f7806e3a0 2023-08-27T11:28:33.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f7e928700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f78158020 con 0x7f5f7806fea0 2023-08-27T11:28:33.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f7e928700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f781584b0 con 0x7f5f78158e20 2023-08-27T11:28:33.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f77fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7806e3a0 0x7f5f78071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f77fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7806e3a0 0x7f5f78071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36542/0 (socket says 172.21.15.112:36542) 2023-08-27T11:28:33.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f77fff700 1 -- 172.21.15.112:0/73370056 learned_addr learned my addr 172.21.15.112:0/73370056 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:33.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f7cec5700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f78158e20 0x7f5f780c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f777fe700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f7806fea0 0x7f5f780c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f77fff700 1 -- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f78158e20 msgr2=0x7f5f780c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f77fff700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f78158e20 0x7f5f780c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f77fff700 1 -- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f7806fea0 msgr2=0x7f5f780c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f77fff700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f7806fea0 0x7f5f780c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f77fff700 1 -- 172.21.15.112:0/73370056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c009550 con 0x7f5f7806e3a0 2023-08-27T11:28:33.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f7cec5700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f78158e20 0x7f5f780c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:33.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f777fe700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f7806fea0 0x7f5f780c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:33.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f77fff700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7806e3a0 0x7f5f78071ae0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f5f780d0f90 tx=0x7f5f6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:33.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f74ff9700 1 -- 172.21.15.112:0/73370056 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c011630 con 0x7f5f7806e3a0 2023-08-27T11:28:33.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f74ff9700 1 -- 172.21.15.112:0/73370056 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c011790 con 0x7f5f7806e3a0 2023-08-27T11:28:33.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f74ff9700 1 -- 172.21.15.112:0/73370056 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c011b40 con 0x7f5f7806e3a0 2023-08-27T11:28:33.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f7e928700 1 -- 172.21.15.112:0/73370056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f781576f0 con 0x7f5f7806e3a0 2023-08-27T11:28:33.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.552+0000 7f5f7e928700 1 -- 172.21.15.112:0/73370056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f78157cb0 con 0x7f5f7806e3a0 2023-08-27T11:28:33.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.553+0000 7f5f7e928700 1 -- 172.21.15.112:0/73370056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f78157e40 con 0x7f5f7806e3a0 2023-08-27T11:28:33.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.554+0000 7f5f74ff9700 1 -- 172.21.15.112:0/73370056 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5f6c011ca0 con 0x7f5f7806e3a0 2023-08-27T11:28:33.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.554+0000 7f5f74ff9700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f6406a4d0 0x7f5f6406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.554+0000 7f5f74ff9700 1 -- 172.21.15.112:0/73370056 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7f5f6c08a3b0 con 0x7f5f7806e3a0 2023-08-27T11:28:33.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.554+0000 7f5f777fe700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f6406a4d0 0x7f5f6406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.555+0000 7f5f777fe700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f6406a4d0 0x7f5f6406c990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5f78159790 tx=0x7f5f68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:33.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.559+0000 7f5f74ff9700 1 -- 172.21.15.112:0/73370056 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5f6c055500 con 0x7f5f7806e3a0 2023-08-27T11:28:33.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.749+0000 7f5f7e928700 1 -- 172.21.15.112:0/73370056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f7806e7e0 con 0x7f5f7806e3a0 2023-08-27T11:28:33.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.750+0000 7f5f74ff9700 1 -- 172.21.15.112:0/73370056 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+18341 (secure 0 0 0) 0x7f5f7806e7e0 con 0x7f5f7806e3a0 2023-08-27T11:28:33.752 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:33.753 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":142,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:31.728440+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs.volume_0000000000561463.meta","create_time":"2023-08-27T11:28:12.176044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000561463"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs.volume_0000000000561463.data","create_time":"2023-08-27T11:28:12.992996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000561463"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:33.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.752+0000 7f5f7e928700 1 -- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f6406a4d0 msgr2=0x7f5f6406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.752+0000 7f5f7e928700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f6406a4d0 0x7f5f6406c990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5f78159790 tx=0x7f5f68009040 comp rx=0 tx=0).stop 2023-08-27T11:28:33.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.752+0000 7f5f7e928700 1 -- 172.21.15.112:0/73370056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7806e3a0 msgr2=0x7f5f78071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.752+0000 7f5f7e928700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7806e3a0 0x7f5f78071ae0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f5f780d0f90 tx=0x7f5f6c004750 comp rx=0 tx=0).stop 2023-08-27T11:28:33.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.752+0000 7f5f7e928700 1 -- 172.21.15.112:0/73370056 shutdown_connections 2023-08-27T11:28:33.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.752+0000 7f5f7e928700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5f6406a4d0 0x7f5f6406c990 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.752+0000 7f5f7e928700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f78158e20 0x7f5f780c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.752+0000 7f5f7e928700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7806e3a0 0x7f5f78071ae0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.752+0000 7f5f7e928700 1 --2- 172.21.15.112:0/73370056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f7806fea0 0x7f5f780c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.752+0000 7f5f7e928700 1 -- 172.21.15.112:0/73370056 >> 172.21.15.112:0/73370056 conn(0x7f5f780d0000 msgr2=0x7f5f7815e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:33.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.752+0000 7f5f7e928700 1 -- 172.21.15.112:0/73370056 shutdown_connections 2023-08-27T11:28:33.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.752+0000 7f5f7e928700 1 -- 172.21.15.112:0/73370056 wait complete. 2023-08-27T11:28:33.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs.volume_0000000000561463.meta', 'create_time': '2023-08-27T11:28:12.176044+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000561463'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs.volume_0000000000561463.data', 'create_time': '2023-08-27T11:28:12.992996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000561463'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:28:33.767 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:28:33.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.888+0000 7f138bfdc700 1 Processor -- start 2023-08-27T11:28:33.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.888+0000 7f138bfdc700 1 -- start start 2023-08-27T11:28:33.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138bfdc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13841437c0 0x7f13840b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138bfdc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1384143ba0 0x7f13840ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138bfdc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f138406d5b0 0x7f138406da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138bfdc700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13840cfd50 con 0x7f138406d5b0 2023-08-27T11:28:33.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138bfdc700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138406df40 con 0x7f1384143ba0 2023-08-27T11:28:33.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138bfdc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13840cf140 con 0x7f13841437c0 2023-08-27T11:28:33.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138a579700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f138406d5b0 0x7f138406da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138a579700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f138406d5b0 0x7f138406da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36554/0 (socket says 172.21.15.112:36554) 2023-08-27T11:28:33.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138a579700 1 -- 172.21.15.112:0/3144511243 learned_addr learned my addr 172.21.15.112:0/3144511243 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:33.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f1389d78700 1 --2- 172.21.15.112:0/3144511243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13841437c0 0x7f13840b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138a579700 1 -- 172.21.15.112:0/3144511243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13841437c0 msgr2=0x7f13840b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:33.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138a579700 1 --2- 172.21.15.112:0/3144511243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13841437c0 0x7f13840b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138a579700 1 -- 172.21.15.112:0/3144511243 >> v2:172.21.15.138:3300/0 conn(0x7f1384143ba0 msgr2=0x7f13840ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:33.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f1389577700 1 --2- 172.21.15.112:0/3144511243 >> v2:172.21.15.138:3300/0 conn(0x7f1384143ba0 0x7f13840ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138a579700 1 --2- 172.21.15.112:0/3144511243 >> v2:172.21.15.138:3300/0 conn(0x7f1384143ba0 0x7f13840ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.889+0000 7f138a579700 1 -- 172.21.15.112:0/3144511243 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13840cf420 con 0x7f138406d5b0 2023-08-27T11:28:33.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.890+0000 7f138a579700 1 --2- 172.21.15.112:0/3144511243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138406d5b0 0x7f138406da00 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f13840b2db0 tx=0x7f137c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fc7e98b7c74cb9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:33.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.890+0000 7f1388d76700 1 -- 172.21.15.112:0/3144511243 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00e070 con 0x7f138406d5b0 2023-08-27T11:28:33.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.890+0000 7f1388d76700 1 -- 172.21.15.112:0/3144511243 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c00b9c0 con 0x7f138406d5b0 2023-08-27T11:28:33.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.890+0000 7f1388d76700 1 -- 172.21.15.112:0/3144511243 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00bd70 con 0x7f138406d5b0 2023-08-27T11:28:33.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.890+0000 7f138bfdc700 1 -- 172.21.15.112:0/3144511243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138406d5b0 msgr2=0x7f138406da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.890+0000 7f138bfdc700 1 --2- 172.21.15.112:0/3144511243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138406d5b0 0x7f138406da00 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f13840b2db0 tx=0x7f137c00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:33.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.890+0000 7f138bfdc700 1 -- 172.21.15.112:0/3144511243 shutdown_connections 2023-08-27T11:28:33.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.890+0000 7f138bfdc700 1 --2- 172.21.15.112:0/3144511243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138406d5b0 0x7f138406da00 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.890+0000 7f138bfdc700 1 --2- 172.21.15.112:0/3144511243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13841437c0 0x7f13840b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.890+0000 7f138bfdc700 1 --2- 172.21.15.112:0/3144511243 >> v2:172.21.15.138:3300/0 conn(0x7f1384143ba0 0x7f13840ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.890+0000 7f138bfdc700 1 -- 172.21.15.112:0/3144511243 >> 172.21.15.112:0/3144511243 conn(0x7f13840b1e90 msgr2=0x7f13840b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:33.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.890+0000 7f138bfdc700 1 -- 172.21.15.112:0/3144511243 shutdown_connections 2023-08-27T11:28:33.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.890+0000 7f138bfdc700 1 -- 172.21.15.112:0/3144511243 wait complete. 2023-08-27T11:28:33.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.892+0000 7f138bfdc700 1 Processor -- start 2023-08-27T11:28:33.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.892+0000 7f138bfdc700 1 -- start start 2023-08-27T11:28:33.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.892+0000 7f138bfdc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138406d5b0 0x7f13841378a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.892+0000 7f138bfdc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13841437c0 0x7f138413bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.892+0000 7f138bfdc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1384143ba0 0x7f1384070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.892+0000 7f138bfdc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138413dc80 con 0x7f138406d5b0 2023-08-27T11:28:33.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.892+0000 7f138bfdc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138413df60 con 0x7f13841437c0 2023-08-27T11:28:33.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.892+0000 7f138bfdc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138413e240 con 0x7f1384143ba0 2023-08-27T11:28:33.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.892+0000 7f1389d78700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138406d5b0 0x7f13841378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.892+0000 7f1389d78700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138406d5b0 0x7f13841378a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36556/0 (socket says 172.21.15.112:36556) 2023-08-27T11:28:33.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.892+0000 7f1389d78700 1 -- 172.21.15.112:0/1912330193 learned_addr learned my addr 172.21.15.112:0/1912330193 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:33.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.893+0000 7f1389d78700 1 -- 172.21.15.112:0/1912330193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1384143ba0 msgr2=0x7f1384070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.893+0000 7f1389d78700 1 --2- 172.21.15.112:0/1912330193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1384143ba0 0x7f1384070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.893+0000 7f1389d78700 1 -- 172.21.15.112:0/1912330193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13841437c0 msgr2=0x7f138413bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.893+0000 7f1389d78700 1 --2- 172.21.15.112:0/1912330193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13841437c0 0x7f138413bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.893+0000 7f1389d78700 1 -- 172.21.15.112:0/1912330193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c009b30 con 0x7f138406d5b0 2023-08-27T11:28:33.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.893+0000 7f1389d78700 1 --2- 172.21.15.112:0/1912330193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138406d5b0 0x7f13841378a0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f137400c7f0 tx=0x7f137400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:33.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.893+0000 7f137a7fc700 1 -- 172.21.15.112:0/1912330193 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374011800 con 0x7f138406d5b0 2023-08-27T11:28:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.893+0000 7f137a7fc700 1 -- 172.21.15.112:0/1912330193 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1374011960 con 0x7f138406d5b0 2023-08-27T11:28:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.893+0000 7f137a7fc700 1 -- 172.21.15.112:0/1912330193 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374011d10 con 0x7f138406d5b0 2023-08-27T11:28:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.893+0000 7f138bfdc700 1 -- 172.21.15.112:0/1912330193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138413e520 con 0x7f138406d5b0 2023-08-27T11:28:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.893+0000 7f138bfdc700 1 -- 172.21.15.112:0/1912330193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138413d8a0 con 0x7f138406d5b0 2023-08-27T11:28:33.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.894+0000 7f138bfdc700 1 -- 172.21.15.112:0/1912330193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138413e760 con 0x7f138406d5b0 2023-08-27T11:28:33.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.894+0000 7f137a7fc700 1 -- 172.21.15.112:0/1912330193 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f1374018360 con 0x7f138406d5b0 2023-08-27T11:28:33.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.895+0000 7f137a7fc700 1 --2- 172.21.15.112:0/1912330193 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f136806a4d0 0x7f136806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.895+0000 7f137a7fc700 1 -- 172.21.15.112:0/1912330193 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7f1374088e00 con 0x7f138406d5b0 2023-08-27T11:28:33.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.895+0000 7f1389577700 1 --2- 172.21.15.112:0/1912330193 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f136806a4d0 0x7f136806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.895+0000 7f1389577700 1 --2- 172.21.15.112:0/1912330193 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f136806a4d0 0x7f136806c990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f13841598d0 tx=0x7f1380008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:33.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:33.899+0000 7f137a7fc700 1 -- 172.21.15.112:0/1912330193 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f138413e760 con 0x7f138406d5b0 2023-08-27T11:28:34.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.087+0000 7f138bfdc700 1 -- 172.21.15.112:0/1912330193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f138413ddc0 con 0x7f138406d5b0 2023-08-27T11:28:34.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.088+0000 7f137a7fc700 1 -- 172.21.15.112:0/1912330193 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v142) v1 ==== 92+0+10 (secure 0 0 0) 0x7f138413ddc0 con 0x7f138406d5b0 2023-08-27T11:28:34.090 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:28:34.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.090+0000 7f138bfdc700 1 -- 172.21.15.112:0/1912330193 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f136806a4d0 msgr2=0x7f136806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.090+0000 7f138bfdc700 1 --2- 172.21.15.112:0/1912330193 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f136806a4d0 0x7f136806c990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f13841598d0 tx=0x7f1380008040 comp rx=0 tx=0).stop 2023-08-27T11:28:34.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.090+0000 7f138bfdc700 1 -- 172.21.15.112:0/1912330193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138406d5b0 msgr2=0x7f13841378a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.090+0000 7f138bfdc700 1 --2- 172.21.15.112:0/1912330193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138406d5b0 0x7f13841378a0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f137400c7f0 tx=0x7f137400cbb0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.090+0000 7f138bfdc700 1 -- 172.21.15.112:0/1912330193 shutdown_connections 2023-08-27T11:28:34.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.090+0000 7f138bfdc700 1 --2- 172.21.15.112:0/1912330193 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f136806a4d0 0x7f136806c990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.090+0000 7f138bfdc700 1 --2- 172.21.15.112:0/1912330193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1384143ba0 0x7f1384070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.090+0000 7f138bfdc700 1 --2- 172.21.15.112:0/1912330193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138406d5b0 0x7f13841378a0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.090+0000 7f138bfdc700 1 --2- 172.21.15.112:0/1912330193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13841437c0 0x7f138413bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.090+0000 7f138bfdc700 1 -- 172.21.15.112:0/1912330193 >> 172.21.15.112:0/1912330193 conn(0x7f13840b1e90 msgr2=0x7f1384072000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:34.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.090+0000 7f138bfdc700 1 -- 172.21.15.112:0/1912330193 shutdown_connections 2023-08-27T11:28:34.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.090+0000 7f138bfdc700 1 -- 172.21.15.112:0/1912330193 wait complete. 2023-08-27T11:28:34.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:28:34.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.224+0000 7f26698f0700 1 Processor -- start 2023-08-27T11:28:34.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.224+0000 7f26698f0700 1 -- start start 2023-08-27T11:28:34.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.225+0000 7f26698f0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2664157b80 0x7f26640d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.225+0000 7f26698f0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26640ce030 0x7f26640706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.225+0000 7f26698f0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26640d2850 con 0x7f26640d23f0 2023-08-27T11:28:34.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.225+0000 7f26698f0700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266415ddd0 con 0x7f2664157b80 2023-08-27T11:28:34.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.225+0000 7f26698f0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26640cf0f0 con 0x7f26640ce030 2023-08-27T11:28:34.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.225+0000 7f2662ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2664157b80 0x7f26640d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.225+0000 7f2662ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2664157b80 0x7f26640d1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36564/0 (socket says 172.21.15.112:36564) 2023-08-27T11:28:34.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.225+0000 7f26627fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f26640d23f0 0x7f26640cd920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59178/0 (socket says 172.21.15.112:59178) 2023-08-27T11:28:34.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.225+0000 7f2662ffd700 1 -- 172.21.15.112:0/2133817904 learned_addr learned my addr 172.21.15.112:0/2133817904 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:34.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.226+0000 7f26637fe700 1 --2- 172.21.15.112:0/2133817904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26640ce030 0x7f26640706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.226+0000 7f2662ffd700 1 -- 172.21.15.112:0/2133817904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26640ce030 msgr2=0x7f26640706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:34.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.226+0000 7f2662ffd700 1 --2- 172.21.15.112:0/2133817904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26640ce030 0x7f26640706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.226+0000 7f2662ffd700 1 -- 172.21.15.112:0/2133817904 >> v1:172.21.15.112:6789/0 conn(0x7f26640d23f0 legacy=0x7f26640cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.226+0000 7f2662ffd700 1 -- 172.21.15.112:0/2133817904 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26640ce570 con 0x7f2664157b80 2023-08-27T11:28:34.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.226+0000 7f2662ffd700 1 --2- 172.21.15.112:0/2133817904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2664157b80 0x7f26640d1eb0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f26580098f0 tx=0x7f2658009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25e8dd951edbf77 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.226+0000 7f2661ffb700 1 -- 172.21.15.112:0/2133817904 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265800b8a0 con 0x7f2664157b80 2023-08-27T11:28:34.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.226+0000 7f2661ffb700 1 -- 172.21.15.112:0/2133817904 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265800ba00 con 0x7f2664157b80 2023-08-27T11:28:34.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.226+0000 7f2661ffb700 1 -- 172.21.15.112:0/2133817904 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265800bd90 con 0x7f2664157b80 2023-08-27T11:28:34.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.226+0000 7f26698f0700 1 -- 172.21.15.112:0/2133817904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2664157b80 msgr2=0x7f26640d1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.226+0000 7f26698f0700 1 --2- 172.21.15.112:0/2133817904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2664157b80 0x7f26640d1eb0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f26580098f0 tx=0x7f2658009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:34.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.227+0000 7f26698f0700 1 -- 172.21.15.112:0/2133817904 shutdown_connections 2023-08-27T11:28:34.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.227+0000 7f26698f0700 1 --2- 172.21.15.112:0/2133817904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26640ce030 0x7f26640706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.227+0000 7f26698f0700 1 --2- 172.21.15.112:0/2133817904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2664157b80 0x7f26640d1eb0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.227+0000 7f26698f0700 1 -- 172.21.15.112:0/2133817904 >> 172.21.15.112:0/2133817904 conn(0x7f26640cfd50 msgr2=0x7f26640d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:34.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.227+0000 7f26698f0700 1 -- 172.21.15.112:0/2133817904 shutdown_connections 2023-08-27T11:28:34.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.227+0000 7f26698f0700 1 -- 172.21.15.112:0/2133817904 wait complete. 2023-08-27T11:28:34.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.228+0000 7f26698f0700 1 Processor -- start 2023-08-27T11:28:34.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.228+0000 7f26698f0700 1 -- start start 2023-08-27T11:28:34.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.229+0000 7f26698f0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26640ce030 0x7f26641f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.229+0000 7f26698f0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26640d23f0 0x7f26641f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.229+0000 7f26698f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2664157b80 0x7f2664151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.229+0000 7f26698f0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26641f6f10 con 0x7f26640ce030 2023-08-27T11:28:34.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.229+0000 7f26698f0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2664153b60 con 0x7f2664157b80 2023-08-27T11:28:34.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.229+0000 7f26698f0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2664155b40 con 0x7f26640d23f0 2023-08-27T11:28:34.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.229+0000 7f2662ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26640ce030 0x7f26641f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.229+0000 7f2662ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26640ce030 0x7f26641f29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36570/0 (socket says 172.21.15.112:36570) 2023-08-27T11:28:34.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.229+0000 7f2662ffd700 1 -- 172.21.15.112:0/2746579985 learned_addr learned my addr 172.21.15.112:0/2746579985 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:34.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.229+0000 7f26627fc700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26640d23f0 0x7f26641f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.229+0000 7f26637fe700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2664157b80 0x7f2664151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.230+0000 7f2662ffd700 1 -- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26640d23f0 msgr2=0x7f26641f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.230+0000 7f2662ffd700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26640d23f0 0x7f26641f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.230+0000 7f2662ffd700 1 -- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2664157b80 msgr2=0x7f2664151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.230+0000 7f2662ffd700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2664157b80 0x7f2664151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.230+0000 7f2662ffd700 1 -- 172.21.15.112:0/2746579985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2658009550 con 0x7f26640ce030 2023-08-27T11:28:34.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.230+0000 7f26637fe700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2664157b80 0x7f2664151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:34.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.230+0000 7f2662ffd700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26640ce030 0x7f26641f29f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f2658009c40 tx=0x7f2658004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.230+0000 7f264bfff700 1 -- 172.21.15.112:0/2746579985 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2658004590 con 0x7f26640ce030 2023-08-27T11:28:34.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.230+0000 7f264bfff700 1 -- 172.21.15.112:0/2746579985 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265801e070 con 0x7f26640ce030 2023-08-27T11:28:34.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.230+0000 7f26698f0700 1 -- 172.21.15.112:0/2746579985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26641f6c30 con 0x7f26640ce030 2023-08-27T11:28:34.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.230+0000 7f264bfff700 1 -- 172.21.15.112:0/2746579985 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26580038e0 con 0x7f26640ce030 2023-08-27T11:28:34.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.230+0000 7f26698f0700 1 -- 172.21.15.112:0/2746579985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2664153780 con 0x7f26640ce030 2023-08-27T11:28:34.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.231+0000 7f26698f0700 1 -- 172.21.15.112:0/2746579985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2664153300 con 0x7f26640ce030 2023-08-27T11:28:34.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.231+0000 7f264bfff700 1 -- 172.21.15.112:0/2746579985 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f2658003a40 con 0x7f26640ce030 2023-08-27T11:28:34.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.232+0000 7f264bfff700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f265006a4d0 0x7f265006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.232+0000 7f264bfff700 1 -- 172.21.15.112:0/2746579985 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7f2658020070 con 0x7f26640ce030 2023-08-27T11:28:34.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.232+0000 7f26627fc700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f265006a4d0 0x7f265006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.232+0000 7f26627fc700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f265006a4d0 0x7f265006c990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f26641555c0 tx=0x7f2654007ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.236+0000 7f264bfff700 1 -- 172.21.15.112:0/2746579985 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2664153300 con 0x7f26640ce030 2023-08-27T11:28:34.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.421+0000 7f26698f0700 1 -- 172.21.15.112:0/2746579985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2664153300 con 0x7f26640ce030 2023-08-27T11:28:34.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.422+0000 7f264bfff700 1 -- 172.21.15.112:0/2746579985 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v142) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2664153300 con 0x7f26640ce030 2023-08-27T11:28:34.424 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:28:34.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.423+0000 7f26698f0700 1 -- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f265006a4d0 msgr2=0x7f265006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.423+0000 7f26698f0700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f265006a4d0 0x7f265006c990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f26641555c0 tx=0x7f2654007ed0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.423+0000 7f26698f0700 1 -- 172.21.15.112:0/2746579985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26640ce030 msgr2=0x7f26641f29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.423+0000 7f26698f0700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26640ce030 0x7f26641f29f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f2658009c40 tx=0x7f2658004750 comp rx=0 tx=0).stop 2023-08-27T11:28:34.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.424+0000 7f26698f0700 1 -- 172.21.15.112:0/2746579985 shutdown_connections 2023-08-27T11:28:34.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.424+0000 7f26698f0700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f265006a4d0 0x7f265006c990 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.424+0000 7f26698f0700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2664157b80 0x7f2664151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.424+0000 7f26698f0700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26640ce030 0x7f26641f29f0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.424+0000 7f26698f0700 1 --2- 172.21.15.112:0/2746579985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26640d23f0 0x7f26641f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.424+0000 7f26698f0700 1 -- 172.21.15.112:0/2746579985 >> 172.21.15.112:0/2746579985 conn(0x7f26640cfd50 msgr2=0x7f2664070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:34.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.424+0000 7f26698f0700 1 -- 172.21.15.112:0/2746579985 shutdown_connections 2023-08-27T11:28:34.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.424+0000 7f26698f0700 1 -- 172.21.15.112:0/2746579985 wait complete. 2023-08-27T11:28:34.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000561463.meta pg_num 2023-08-27T11:28:34.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.558+0000 7fd7ce03c700 1 Processor -- start 2023-08-27T11:28:34.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.558+0000 7fd7ce03c700 1 -- start start 2023-08-27T11:28:34.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.558+0000 7fd7ce03c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7c80b3440 0x7fd7c815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.558+0000 7fd7ce03c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd7c815e6e0 0x7fd7c806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7ce03c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c80b5070 con 0x7fd7c8146c20 2023-08-27T11:28:34.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7ce03c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c8147000 con 0x7fd7c815e6e0 2023-08-27T11:28:34.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7ce03c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c8161740 con 0x7fd7c80b3440 2023-08-27T11:28:34.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7c7fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd7c815e6e0 0x7fd7c806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7c7fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd7c815e6e0 0x7fd7c806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36582/0 (socket says 172.21.15.112:36582) 2023-08-27T11:28:34.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7c7fff700 1 -- 172.21.15.112:0/4219046778 learned_addr learned my addr 172.21.15.112:0/4219046778 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:34.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7c6ffd700 1 --2- 172.21.15.112:0/4219046778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7c80b3440 0x7fd7c815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7c7fff700 1 -- 172.21.15.112:0/4219046778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7c80b3440 msgr2=0x7fd7c815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:34.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7c7fff700 1 --2- 172.21.15.112:0/4219046778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7c80b3440 0x7fd7c815e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7c7fff700 1 -- 172.21.15.112:0/4219046778 >> v1:172.21.15.138:6789/0 conn(0x7fd7c8146c20 legacy=0x7fd7c8160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7c7fff700 1 -- 172.21.15.112:0/4219046778 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c8161a20 con 0x7fd7c815e6e0 2023-08-27T11:28:34.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7c7fff700 1 --2- 172.21.15.112:0/4219046778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7c815e6e0 0x7fd7c806f790 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009e50 tx=0x7fd7c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4f1fda362cc6d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7c67fc700 1 -- 172.21.15.112:0/4219046778 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000e070 con 0x7fd7c815e6e0 2023-08-27T11:28:34.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7c67fc700 1 -- 172.21.15.112:0/4219046778 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c000b9c0 con 0x7fd7c815e6e0 2023-08-27T11:28:34.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.559+0000 7fd7c67fc700 1 -- 172.21.15.112:0/4219046778 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000bd70 con 0x7fd7c815e6e0 2023-08-27T11:28:34.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.560+0000 7fd7ce03c700 1 -- 172.21.15.112:0/4219046778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7c815e6e0 msgr2=0x7fd7c806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.560+0000 7fd7ce03c700 1 --2- 172.21.15.112:0/4219046778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7c815e6e0 0x7fd7c806f790 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009e50 tx=0x7fd7c000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:34.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.560+0000 7fd7ce03c700 1 -- 172.21.15.112:0/4219046778 shutdown_connections 2023-08-27T11:28:34.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.560+0000 7fd7ce03c700 1 --2- 172.21.15.112:0/4219046778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7c815e6e0 0x7fd7c806f790 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.560+0000 7fd7ce03c700 1 --2- 172.21.15.112:0/4219046778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7c80b3440 0x7fd7c815e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.560+0000 7fd7ce03c700 1 -- 172.21.15.112:0/4219046778 >> 172.21.15.112:0/4219046778 conn(0x7fd7c80b1e90 msgr2=0x7fd7c80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:34.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.560+0000 7fd7ce03c700 1 -- 172.21.15.112:0/4219046778 shutdown_connections 2023-08-27T11:28:34.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.560+0000 7fd7ce03c700 1 -- 172.21.15.112:0/4219046778 wait complete. 2023-08-27T11:28:34.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.561+0000 7fd7ce03c700 1 Processor -- start 2023-08-27T11:28:34.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.562+0000 7fd7ce03c700 1 -- start start 2023-08-27T11:28:34.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.562+0000 7fd7ce03c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7c80b3440 0x7fd7c8140880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.562+0000 7fd7ce03c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7c8146c20 0x7fd7c813bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.562+0000 7fd7ce03c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7c815e6e0 0x7fd7c80ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.562+0000 7fd7ce03c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81f3030 con 0x7fd7c80b3440 2023-08-27T11:28:34.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.562+0000 7fd7ce03c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81f3310 con 0x7fd7c8146c20 2023-08-27T11:28:34.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.562+0000 7fd7ce03c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81f35f0 con 0x7fd7c815e6e0 2023-08-27T11:28:34.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7c77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7c80b3440 0x7fd7c8140880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7c77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7c80b3440 0x7fd7c8140880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36584/0 (socket says 172.21.15.112:36584) 2023-08-27T11:28:34.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7c77fe700 1 -- 172.21.15.112:0/1530140065 learned_addr learned my addr 172.21.15.112:0/1530140065 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:34.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7c7fff700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7c815e6e0 0x7fd7c80ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7c6ffd700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7c8146c20 0x7fd7c813bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7c77fe700 1 -- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7c815e6e0 msgr2=0x7fd7c80ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7c77fe700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7c815e6e0 0x7fd7c80ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7c77fe700 1 -- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7c8146c20 msgr2=0x7fd7c813bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7c77fe700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7c8146c20 0x7fd7c813bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7c77fe700 1 -- 172.21.15.112:0/1530140065 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c0009b30 con 0x7fd7c80b3440 2023-08-27T11:28:34.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7c77fe700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7c80b3440 0x7fd7c8140880 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800c4e0 tx=0x7fd7b800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7affff700 1 -- 172.21.15.112:0/1530140065 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80052d0 con 0x7fd7c80b3440 2023-08-27T11:28:34.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7affff700 1 -- 172.21.15.112:0/1530140065 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b8005430 con 0x7fd7c80b3440 2023-08-27T11:28:34.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7ce03c700 1 -- 172.21.15.112:0/1530140065 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c81f38d0 con 0x7fd7c80b3440 2023-08-27T11:28:34.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7affff700 1 -- 172.21.15.112:0/1530140065 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80038c0 con 0x7fd7c80b3440 2023-08-27T11:28:34.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.563+0000 7fd7ce03c700 1 -- 172.21.15.112:0/1530140065 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c813c520 con 0x7fd7c80b3440 2023-08-27T11:28:34.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.564+0000 7fd7ce03c700 1 -- 172.21.15.112:0/1530140065 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c8067e90 con 0x7fd7c80b3440 2023-08-27T11:28:34.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.565+0000 7fd7affff700 1 -- 172.21.15.112:0/1530140065 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fd7b8003a20 con 0x7fd7c80b3440 2023-08-27T11:28:34.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.565+0000 7fd7affff700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd7b406a4d0 0x7fd7b406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.565+0000 7fd7affff700 1 -- 172.21.15.112:0/1530140065 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7fd7b8088040 con 0x7fd7c80b3440 2023-08-27T11:28:34.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.565+0000 7fd7c6ffd700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd7b406a4d0 0x7fd7b406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.566+0000 7fd7c6ffd700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd7b406a4d0 0x7fd7b406c990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80af3a0 tx=0x7fd7bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.569+0000 7fd7affff700 1 -- 172.21.15.112:0/1530140065 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd7b80ba990 con 0x7fd7c80b3440 2023-08-27T11:28:34.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.759+0000 7fd7ce03c700 1 -- 172.21.15.112:0/1530140065 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000561463.meta", "var": "pg_num"} v 0) v1 -- 0x7fd7c80b5070 con 0x7fd7c80b3440 2023-08-27T11:28:34.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.759+0000 7fd7affff700 1 -- 172.21.15.112:0/1530140065 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000561463.meta", "var": "pg_num"}]=0 v142) v1 ==== 124+0+11 (secure 0 0 0) 0x7fd7c80b5070 con 0x7fd7c80b3440 2023-08-27T11:28:34.761 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:28:34.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.761+0000 7fd7ce03c700 1 -- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd7b406a4d0 msgr2=0x7fd7b406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.761+0000 7fd7ce03c700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd7b406a4d0 0x7fd7b406c990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80af3a0 tx=0x7fd7bc008040 comp rx=0 tx=0).stop 2023-08-27T11:28:34.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.761+0000 7fd7ce03c700 1 -- 172.21.15.112:0/1530140065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7c80b3440 msgr2=0x7fd7c8140880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.761+0000 7fd7ce03c700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7c80b3440 0x7fd7c8140880 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800c4e0 tx=0x7fd7b800c8a0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.761+0000 7fd7ce03c700 1 -- 172.21.15.112:0/1530140065 shutdown_connections 2023-08-27T11:28:34.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.761+0000 7fd7ce03c700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd7b406a4d0 0x7fd7b406c990 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.761+0000 7fd7ce03c700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7c815e6e0 0x7fd7c80ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.761+0000 7fd7ce03c700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7c80b3440 0x7fd7c8140880 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.761+0000 7fd7ce03c700 1 --2- 172.21.15.112:0/1530140065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7c8146c20 0x7fd7c813bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.761+0000 7fd7ce03c700 1 -- 172.21.15.112:0/1530140065 >> 172.21.15.112:0/1530140065 conn(0x7fd7c80b1e90 msgr2=0x7fd7c8145d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.761+0000 7fd7ce03c700 1 -- 172.21.15.112:0/1530140065 shutdown_connections 2023-08-27T11:28:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.761+0000 7fd7ce03c700 1 -- 172.21.15.112:0/1530140065 wait complete. 2023-08-27T11:28:34.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000561463.data pg_num 2023-08-27T11:28:34.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.897+0000 7f81ee826700 1 Processor -- start 2023-08-27T11:28:34.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.897+0000 7f81ee826700 1 -- start start 2023-08-27T11:28:34.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.898+0000 7f81ee826700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f81e8146b60 0x7f81e806ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.898+0000 7f81ee826700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81e806e2f0 0x7f81e80acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.898+0000 7f81ee826700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e8160fa0 con 0x7f81e80ad000 2023-08-27T11:28:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.898+0000 7f81ee826700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e8161b70 con 0x7f81e8146b60 2023-08-27T11:28:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.898+0000 7f81ee826700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8143ad0 con 0x7f81e806e2f0 2023-08-27T11:28:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.898+0000 7f81e7fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f81e8146b60 0x7f81e806ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.898+0000 7f81ecdc3700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f81e80ad000 0x7f81e8143f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59210/0 (socket says 172.21.15.112:59210) 2023-08-27T11:28:34.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.898+0000 7f81e7fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f81e8146b60 0x7f81e806ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36592/0 (socket says 172.21.15.112:36592) 2023-08-27T11:28:34.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.898+0000 7f81ecdc3700 1 -- 172.21.15.112:0/982476692 learned_addr learned my addr 172.21.15.112:0/982476692 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:34.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.898+0000 7f81e77fe700 1 --2- 172.21.15.112:0/982476692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81e806e2f0 0x7f81e80acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81e7fff700 1 -- 172.21.15.112:0/982476692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81e806e2f0 msgr2=0x7f81e80acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81e7fff700 1 --2- 172.21.15.112:0/982476692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81e806e2f0 0x7f81e80acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81e7fff700 1 -- 172.21.15.112:0/982476692 >> v1:172.21.15.112:6789/0 conn(0x7f81e80ad000 legacy=0x7f81e8143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81e7fff700 1 -- 172.21.15.112:0/982476692 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e806e750 con 0x7f81e8146b60 2023-08-27T11:28:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81e6ffd700 1 -- 172.21.15.112:0/982476692 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957882879 0 0) 0x7f81e8160fa0 con 0x7f81e80ad000 2023-08-27T11:28:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81e7fff700 1 --2- 172.21.15.112:0/982476692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81e8146b60 0x7f81e806ddb0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009840 tx=0x7f81d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdff69011e9fc17 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81e6ffd700 1 -- 172.21.15.112:0/982476692 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0011830 con 0x7f81e8146b60 2023-08-27T11:28:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81e6ffd700 1 -- 172.21.15.112:0/982476692 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d0011990 con 0x7f81e8146b60 2023-08-27T11:28:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81e6ffd700 1 -- 172.21.15.112:0/982476692 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0011d20 con 0x7f81e8146b60 2023-08-27T11:28:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81ee826700 1 -- 172.21.15.112:0/982476692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81e8146b60 msgr2=0x7f81e806ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81ee826700 1 --2- 172.21.15.112:0/982476692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81e8146b60 0x7f81e806ddb0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009840 tx=0x7f81d0009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:34.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81ee826700 1 -- 172.21.15.112:0/982476692 shutdown_connections 2023-08-27T11:28:34.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81ee826700 1 --2- 172.21.15.112:0/982476692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81e8146b60 0x7f81e806ddb0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81ee826700 1 --2- 172.21.15.112:0/982476692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81e806e2f0 0x7f81e80acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.899+0000 7f81ee826700 1 -- 172.21.15.112:0/982476692 >> 172.21.15.112:0/982476692 conn(0x7f81e80cc240 msgr2=0x7f81e80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:34.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.900+0000 7f81ee826700 1 -- 172.21.15.112:0/982476692 shutdown_connections 2023-08-27T11:28:34.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.900+0000 7f81ee826700 1 -- 172.21.15.112:0/982476692 wait complete. 2023-08-27T11:28:34.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.901+0000 7f81ee826700 1 Processor -- start 2023-08-27T11:28:34.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.901+0000 7f81ee826700 1 -- start start 2023-08-27T11:28:34.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81ee826700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81e806e2f0 0x7f81e80723f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81ee826700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81e80ad000 0x7f81e815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81ee826700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81e8146b60 0x7f81e8143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81ee826700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e813fbd0 con 0x7f81e806e2f0 2023-08-27T11:28:34.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81ee826700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8144150 con 0x7f81e80ad000 2023-08-27T11:28:34.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81ee826700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e80717c0 con 0x7f81e8146b60 2023-08-27T11:28:34.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81e7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81e806e2f0 0x7f81e80723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81e7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81e806e2f0 0x7f81e80723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36598/0 (socket says 172.21.15.112:36598) 2023-08-27T11:28:34.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81e7fff700 1 -- 172.21.15.112:0/1829048769 learned_addr learned my addr 172.21.15.112:0/1829048769 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:34.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81e7fff700 1 -- 172.21.15.112:0/1829048769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81e8146b60 msgr2=0x7f81e8143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81e7fff700 1 --2- 172.21.15.112:0/1829048769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81e8146b60 0x7f81e8143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81e7fff700 1 -- 172.21.15.112:0/1829048769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81e80ad000 msgr2=0x7f81e815ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81e7fff700 1 --2- 172.21.15.112:0/1829048769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81e80ad000 0x7f81e815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.902+0000 7f81e7fff700 1 -- 172.21.15.112:0/1829048769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d00094a0 con 0x7f81e806e2f0 2023-08-27T11:28:34.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.903+0000 7f81e7fff700 1 --2- 172.21.15.112:0/1829048769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81e806e2f0 0x7f81e80723f0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f81d0000c00 tx=0x7f81d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.903+0000 7f81e4ff9700 1 -- 172.21.15.112:0/1829048769 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d00036a0 con 0x7f81e806e2f0 2023-08-27T11:28:34.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.903+0000 7f81e4ff9700 1 -- 172.21.15.112:0/1829048769 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d001e070 con 0x7f81e806e2f0 2023-08-27T11:28:34.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.903+0000 7f81ee826700 1 -- 172.21.15.112:0/1829048769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e813f2a0 con 0x7f81e806e2f0 2023-08-27T11:28:34.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.903+0000 7f81e4ff9700 1 -- 172.21.15.112:0/1829048769 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0003a60 con 0x7f81e806e2f0 2023-08-27T11:28:34.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.903+0000 7f81ee826700 1 -- 172.21.15.112:0/1829048769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e813f860 con 0x7f81e806e2f0 2023-08-27T11:28:34.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.904+0000 7f81ee826700 1 -- 172.21.15.112:0/1829048769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e8067e90 con 0x7f81e806e2f0 2023-08-27T11:28:34.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.904+0000 7f81e4ff9700 1 -- 172.21.15.112:0/1829048769 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f81d0003bc0 con 0x7f81e806e2f0 2023-08-27T11:28:34.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.904+0000 7f81e4ff9700 1 --2- 172.21.15.112:0/1829048769 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f81c806a4d0 0x7f81c806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.905+0000 7f81e4ff9700 1 -- 172.21.15.112:0/1829048769 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7f81d0089bc0 con 0x7f81e806e2f0 2023-08-27T11:28:34.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.905+0000 7f81e77fe700 1 --2- 172.21.15.112:0/1829048769 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f81c806a4d0 0x7f81c806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.905+0000 7f81e77fe700 1 --2- 172.21.15.112:0/1829048769 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f81c806a4d0 0x7f81c806c990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f81e815b270 tx=0x7f81d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:34.909+0000 7f81e4ff9700 1 -- 172.21.15.112:0/1829048769 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f81d0054e20 con 0x7f81e806e2f0 2023-08-27T11:28:35.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.097+0000 7f81ee826700 1 -- 172.21.15.112:0/1829048769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000561463.data", "var": "pg_num"} v 0) v1 -- 0x7f81e8067a20 con 0x7f81e806e2f0 2023-08-27T11:28:35.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.097+0000 7f81e4ff9700 1 -- 172.21.15.112:0/1829048769 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000561463.data", "var": "pg_num"}]=0 v142) v1 ==== 124+0+11 (secure 0 0 0) 0x7f81d0015020 con 0x7f81e806e2f0 2023-08-27T11:28:35.099 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:28:35.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.099+0000 7f81ee826700 1 -- 172.21.15.112:0/1829048769 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f81c806a4d0 msgr2=0x7f81c806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.099+0000 7f81ee826700 1 --2- 172.21.15.112:0/1829048769 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f81c806a4d0 0x7f81c806c990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f81e815b270 tx=0x7f81d8008040 comp rx=0 tx=0).stop 2023-08-27T11:28:35.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.099+0000 7f81ee826700 1 -- 172.21.15.112:0/1829048769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81e806e2f0 msgr2=0x7f81e80723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.099+0000 7f81ee826700 1 --2- 172.21.15.112:0/1829048769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81e806e2f0 0x7f81e80723f0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f81d0000c00 tx=0x7f81d0004750 comp rx=0 tx=0).stop 2023-08-27T11:28:35.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.100+0000 7f81ee826700 1 -- 172.21.15.112:0/1829048769 shutdown_connections 2023-08-27T11:28:35.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.100+0000 7f81ee826700 1 --2- 172.21.15.112:0/1829048769 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f81c806a4d0 0x7f81c806c990 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.100+0000 7f81ee826700 1 --2- 172.21.15.112:0/1829048769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81e8146b60 0x7f81e8143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.100+0000 7f81ee826700 1 --2- 172.21.15.112:0/1829048769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81e806e2f0 0x7f81e80723f0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.100+0000 7f81ee826700 1 --2- 172.21.15.112:0/1829048769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81e80ad000 0x7f81e815ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.100+0000 7f81ee826700 1 -- 172.21.15.112:0/1829048769 >> 172.21.15.112:0/1829048769 conn(0x7f81e80cc240 msgr2=0x7f81e815eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:35.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.100+0000 7f81ee826700 1 -- 172.21.15.112:0/1829048769 shutdown_connections 2023-08-27T11:28:35.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.100+0000 7f81ee826700 1 -- 172.21.15.112:0/1829048769 wait complete. 2023-08-27T11:28:35.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:35.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.232+0000 7fc222bf6700 1 Processor -- start 2023-08-27T11:28:35.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.232+0000 7fc222bf6700 1 -- start start 2023-08-27T11:28:35.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.233+0000 7fc222bf6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc21c15ae80 0x7fc21c0b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.233+0000 7fc222bf6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc21c0ca740 0x7fc21c0708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.233+0000 7fc222bf6700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21c0cf2a0 con 0x7fc21c070de0 2023-08-27T11:28:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.233+0000 7fc222bf6700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21c0b0c50 con 0x7fc21c15ae80 2023-08-27T11:28:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.233+0000 7fc222bf6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c0cf580 con 0x7fc21c0ca740 2023-08-27T11:28:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.233+0000 7fc21bfff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc21c0ca740 0x7fc21c0708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.233+0000 7fc220992700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc21c15ae80 0x7fc21c0b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.233+0000 7fc221193700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc21c070de0 0x7fc21c0d0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:38514/0 (socket says 172.21.15.112:38514) 2023-08-27T11:28:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.233+0000 7fc221193700 1 -- 172.21.15.112:0/2861266297 learned_addr learned my addr 172.21.15.112:0/2861266297 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.234+0000 7fc21bfff700 1 -- 172.21.15.112:0/2861266297 >> v2:172.21.15.138:3300/0 conn(0x7fc21c15ae80 msgr2=0x7fc21c0b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.234+0000 7fc21bfff700 1 --2- 172.21.15.112:0/2861266297 >> v2:172.21.15.138:3300/0 conn(0x7fc21c15ae80 0x7fc21c0b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.234+0000 7fc21bfff700 1 -- 172.21.15.112:0/2861266297 >> v1:172.21.15.138:6789/0 conn(0x7fc21c070de0 legacy=0x7fc21c0d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.234+0000 7fc21bfff700 1 -- 172.21.15.112:0/2861266297 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c0cfa80 con 0x7fc21c0ca740 2023-08-27T11:28:35.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.234+0000 7fc21b7fe700 1 -- 172.21.15.112:0/2861266297 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560123491 0 0) 0x7fc21c0cf2a0 con 0x7fc21c070de0 2023-08-27T11:28:35.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.234+0000 7fc220992700 1 --2- 172.21.15.112:0/2861266297 >> v2:172.21.15.138:3300/0 conn(0x7fc21c15ae80 0x7fc21c0b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:35.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.234+0000 7fc21bfff700 1 --2- 172.21.15.112:0/2861266297 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc21c0ca740 0x7fc21c0708a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc208009840 tx=0x7fc208009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6723ff293c3521d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.234+0000 7fc21b7fe700 1 -- 172.21.15.112:0/2861266297 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208011830 con 0x7fc21c0ca740 2023-08-27T11:28:35.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.234+0000 7fc21b7fe700 1 -- 172.21.15.112:0/2861266297 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208011990 con 0x7fc21c0ca740 2023-08-27T11:28:35.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.234+0000 7fc21b7fe700 1 -- 172.21.15.112:0/2861266297 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208011d20 con 0x7fc21c0ca740 2023-08-27T11:28:35.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.235+0000 7fc222bf6700 1 -- 172.21.15.112:0/2861266297 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc21c0ca740 msgr2=0x7fc21c0708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.235+0000 7fc222bf6700 1 --2- 172.21.15.112:0/2861266297 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc21c0ca740 0x7fc21c0708a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc208009840 tx=0x7fc208009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:35.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.235+0000 7fc222bf6700 1 -- 172.21.15.112:0/2861266297 shutdown_connections 2023-08-27T11:28:35.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.235+0000 7fc222bf6700 1 --2- 172.21.15.112:0/2861266297 >> v2:172.21.15.138:3300/0 conn(0x7fc21c15ae80 0x7fc21c0b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.235+0000 7fc222bf6700 1 --2- 172.21.15.112:0/2861266297 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc21c0ca740 0x7fc21c0708a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.235+0000 7fc222bf6700 1 -- 172.21.15.112:0/2861266297 >> 172.21.15.112:0/2861266297 conn(0x7fc21c06f970 msgr2=0x7fc21c06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:35.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.235+0000 7fc222bf6700 1 -- 172.21.15.112:0/2861266297 shutdown_connections 2023-08-27T11:28:35.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.235+0000 7fc222bf6700 1 -- 172.21.15.112:0/2861266297 wait complete. 2023-08-27T11:28:35.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.236+0000 7fc222bf6700 1 Processor -- start 2023-08-27T11:28:35.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.236+0000 7fc222bf6700 1 -- start start 2023-08-27T11:28:35.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc222bf6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc21c070de0 0x7fc21c1f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc222bf6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc21c0ca740 0x7fc21c0715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc222bf6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc21c15ae80 0x7fc21c1f3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc222bf6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c1f6a30 con 0x7fc21c0ca740 2023-08-27T11:28:35.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc222bf6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c1f6d10 con 0x7fc21c15ae80 2023-08-27T11:28:35.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc222bf6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c1f6ff0 con 0x7fc21c070de0 2023-08-27T11:28:35.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc21bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc21c0ca740 0x7fc21c0715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc21bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc21c0ca740 0x7fc21c0715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36614/0 (socket says 172.21.15.112:36614) 2023-08-27T11:28:35.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc21bfff700 1 -- 172.21.15.112:0/3635255335 learned_addr learned my addr 172.21.15.112:0/3635255335 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:35.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc220992700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc21c070de0 0x7fc21c1f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc221193700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc21c15ae80 0x7fc21c1f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc21bfff700 1 -- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc21c070de0 msgr2=0x7fc21c1f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc21bfff700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc21c070de0 0x7fc21c1f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc21bfff700 1 -- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc21c15ae80 msgr2=0x7fc21c1f3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc21bfff700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc21c15ae80 0x7fc21c1f3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc21bfff700 1 -- 172.21.15.112:0/3635255335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2100094a0 con 0x7fc21c0ca740 2023-08-27T11:28:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc221193700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc21c15ae80 0x7fc21c1f3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:35.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.237+0000 7fc21bfff700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc21c0ca740 0x7fc21c0715d0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fc2080090b0 tx=0x7fc208004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.238+0000 7fc2197fa700 1 -- 172.21.15.112:0/3635255335 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080045d0 con 0x7fc21c0ca740 2023-08-27T11:28:35.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.238+0000 7fc2197fa700 1 -- 172.21.15.112:0/3635255335 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20801f070 con 0x7fc21c0ca740 2023-08-27T11:28:35.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.238+0000 7fc2197fa700 1 -- 172.21.15.112:0/3635255335 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080038e0 con 0x7fc21c0ca740 2023-08-27T11:28:35.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.238+0000 7fc222bf6700 1 -- 172.21.15.112:0/3635255335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2080094a0 con 0x7fc21c0ca740 2023-08-27T11:28:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.238+0000 7fc222bf6700 1 -- 172.21.15.112:0/3635255335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c1f2bd0 con 0x7fc21c0ca740 2023-08-27T11:28:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.238+0000 7fc222bf6700 1 -- 172.21.15.112:0/3635255335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21c1f2d60 con 0x7fc21c0ca740 2023-08-27T11:28:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.239+0000 7fc2197fa700 1 -- 172.21.15.112:0/3635255335 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fc208003a40 con 0x7fc21c0ca740 2023-08-27T11:28:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.239+0000 7fc2197fa700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc20006a4d0 0x7fc20006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.239+0000 7fc2197fa700 1 -- 172.21.15.112:0/3635255335 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7fc208021070 con 0x7fc21c0ca740 2023-08-27T11:28:35.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.240+0000 7fc220992700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc20006a4d0 0x7fc20006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.240+0000 7fc220992700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc20006a4d0 0x7fc20006c990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc210009cd0 tx=0x7fc210009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.244+0000 7fc2197fa700 1 -- 172.21.15.112:0/3635255335 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc21c1f2d60 con 0x7fc21c0ca740 2023-08-27T11:28:35.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.476+0000 7fc222bf6700 1 -- 172.21.15.112:0/3635255335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc21c1f2d60 con 0x7fc21c0ca740 2023-08-27T11:28:35.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.477+0000 7fc2197fa700 1 -- 172.21.15.112:0/3635255335 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4902 (secure 0 0 0) 0x7fc21c1f2d60 con 0x7fc21c0ca740 2023-08-27T11:28:35.479 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:35.479 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/739970856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":739970856},{"type":"v1","addr":"172.21.15.138:6837","nonce":739970856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3233069521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3233069521},{"type":"v1","addr":"172.21.15.112:6834","nonce":3233069521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2559241026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2559241026},{"type":"v1","addr":"172.21.15.112:6835","nonce":2559241026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:14.032146+0000","modified":"2023-08-27T11:28:29.716445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6836/529571603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":529571603},{"type":"v1","addr":"172.21.15.138:6836","nonce":529571603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"volume_0000000000561463","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-08-27T11:28:35.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.479+0000 7fc222bf6700 1 -- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc20006a4d0 msgr2=0x7fc20006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.479+0000 7fc222bf6700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc20006a4d0 0x7fc20006c990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc210009cd0 tx=0x7fc210009210 comp rx=0 tx=0).stop 2023-08-27T11:28:35.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.479+0000 7fc222bf6700 1 -- 172.21.15.112:0/3635255335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc21c0ca740 msgr2=0x7fc21c0715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.479+0000 7fc222bf6700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc21c0ca740 0x7fc21c0715d0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fc2080090b0 tx=0x7fc208004750 comp rx=0 tx=0).stop 2023-08-27T11:28:35.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.479+0000 7fc222bf6700 1 -- 172.21.15.112:0/3635255335 shutdown_connections 2023-08-27T11:28:35.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.479+0000 7fc222bf6700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc20006a4d0 0x7fc20006c990 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.479+0000 7fc222bf6700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc21c15ae80 0x7fc21c1f3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.479+0000 7fc222bf6700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc21c070de0 0x7fc21c1f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.479+0000 7fc222bf6700 1 --2- 172.21.15.112:0/3635255335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc21c0ca740 0x7fc21c0715d0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.479+0000 7fc222bf6700 1 -- 172.21.15.112:0/3635255335 >> 172.21.15.112:0/3635255335 conn(0x7fc21c06f970 msgr2=0x7fc21c0b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:35.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.479+0000 7fc222bf6700 1 -- 172.21.15.112:0/3635255335 shutdown_connections 2023-08-27T11:28:35.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.479+0000 7fc222bf6700 1 -- 172.21.15.112:0/3635255335 wait complete. 2023-08-27T11:28:35.484 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 96 2023-08-27T11:28:35.493 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:35.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.611+0000 7f7bb46b9700 1 Processor -- start 2023-08-27T11:28:35.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.611+0000 7f7bb46b9700 1 -- start start 2023-08-27T11:28:35.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.612+0000 7f7bb46b9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7bac1466c0 0x7f7bac06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.612+0000 7f7bb46b9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7bac0b4760 0x7f7bac06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.612+0000 7f7bb46b9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac161030 con 0x7f7bac146bb0 2023-08-27T11:28:35.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.612+0000 7f7bb46b9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bac0b57e0 con 0x7f7bac0b4760 2023-08-27T11:28:35.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.612+0000 7f7bb46b9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bac161310 con 0x7f7bac1466c0 2023-08-27T11:28:35.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.612+0000 7f7bb1c54700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7bac0b4760 0x7f7bac06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.612+0000 7f7bb1c54700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7bac0b4760 0x7f7bac06fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36620/0 (socket says 172.21.15.112:36620) 2023-08-27T11:28:35.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.612+0000 7f7bb1c54700 1 -- 172.21.15.112:0/1592475826 learned_addr learned my addr 172.21.15.112:0/1592475826 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:35.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.613+0000 7f7bb2455700 1 --2- 172.21.15.112:0/1592475826 >> v2:172.21.15.138:3300/0 conn(0x7f7bac1466c0 0x7f7bac06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.613+0000 7f7bb1c54700 1 -- 172.21.15.112:0/1592475826 >> v2:172.21.15.138:3300/0 conn(0x7f7bac1466c0 msgr2=0x7f7bac06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:35.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.613+0000 7f7bb1c54700 1 --2- 172.21.15.112:0/1592475826 >> v2:172.21.15.138:3300/0 conn(0x7f7bac1466c0 0x7f7bac06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.613+0000 7f7bb1c54700 1 -- 172.21.15.112:0/1592475826 >> v1:172.21.15.138:6789/0 conn(0x7f7bac146bb0 legacy=0x7f7bac06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.613+0000 7f7bb1c54700 1 -- 172.21.15.112:0/1592475826 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac1615f0 con 0x7f7bac0b4760 2023-08-27T11:28:35.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.613+0000 7f7bb1c54700 1 --2- 172.21.15.112:0/1592475826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bac0b4760 0x7f7bac06fcd0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0d0e70 tx=0x7f7ba8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5183a0e66b63d057 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.613+0000 7f7bb1453700 1 -- 172.21.15.112:0/1592475826 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800b870 con 0x7f7bac0b4760 2023-08-27T11:28:35.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.613+0000 7f7bb1453700 1 -- 172.21.15.112:0/1592475826 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba800b9d0 con 0x7f7bac0b4760 2023-08-27T11:28:35.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.613+0000 7f7bb1453700 1 -- 172.21.15.112:0/1592475826 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800bd60 con 0x7f7bac0b4760 2023-08-27T11:28:35.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.613+0000 7f7bb46b9700 1 -- 172.21.15.112:0/1592475826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bac0b4760 msgr2=0x7f7bac06fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.613+0000 7f7bb46b9700 1 --2- 172.21.15.112:0/1592475826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bac0b4760 0x7f7bac06fcd0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0d0e70 tx=0x7f7ba8009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.614+0000 7f7bb46b9700 1 -- 172.21.15.112:0/1592475826 shutdown_connections 2023-08-27T11:28:35.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.614+0000 7f7bb46b9700 1 --2- 172.21.15.112:0/1592475826 >> v2:172.21.15.138:3300/0 conn(0x7f7bac1466c0 0x7f7bac06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.614+0000 7f7bb46b9700 1 --2- 172.21.15.112:0/1592475826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bac0b4760 0x7f7bac06fcd0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.614+0000 7f7bb46b9700 1 -- 172.21.15.112:0/1592475826 >> 172.21.15.112:0/1592475826 conn(0x7f7bac0cff80 msgr2=0x7f7bac0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:35.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.614+0000 7f7bb46b9700 1 -- 172.21.15.112:0/1592475826 shutdown_connections 2023-08-27T11:28:35.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.614+0000 7f7bb46b9700 1 -- 172.21.15.112:0/1592475826 wait complete. 2023-08-27T11:28:35.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.615+0000 7f7bb46b9700 1 Processor -- start 2023-08-27T11:28:35.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.615+0000 7f7bb46b9700 1 -- start start 2023-08-27T11:28:35.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.615+0000 7f7bb46b9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7bac0b4760 0x7f7bac13c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.615+0000 7f7bb46b9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bac1466c0 0x7f7bac0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.615+0000 7f7bb46b9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bac146bb0 0x7f7bac142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.615+0000 7f7bb46b9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac13e200 con 0x7f7bac1466c0 2023-08-27T11:28:35.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.615+0000 7f7bb46b9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac13e4e0 con 0x7f7bac146bb0 2023-08-27T11:28:35.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.615+0000 7f7bb46b9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac13e7c0 con 0x7f7bac0b4760 2023-08-27T11:28:35.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb1c54700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bac1466c0 0x7f7bac0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb1c54700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bac1466c0 0x7f7bac0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36628/0 (socket says 172.21.15.112:36628) 2023-08-27T11:28:35.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb1c54700 1 -- 172.21.15.112:0/2846952808 learned_addr learned my addr 172.21.15.112:0/2846952808 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:35.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb2455700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7bac0b4760 0x7f7bac13c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb2c56700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bac146bb0 0x7f7bac142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb1c54700 1 -- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7bac0b4760 msgr2=0x7f7bac13c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb1c54700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7bac0b4760 0x7f7bac13c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb1c54700 1 -- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bac146bb0 msgr2=0x7f7bac142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb1c54700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bac146bb0 0x7f7bac142e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb1c54700 1 -- 172.21.15.112:0/2846952808 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba8009550 con 0x7f7bac1466c0 2023-08-27T11:28:35.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb2c56700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bac146bb0 0x7f7bac142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:35.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb2455700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7bac0b4760 0x7f7bac13c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:35.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb1c54700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bac1466c0 0x7f7bac0ac890 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8000c00 tx=0x7f7ba8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7ba2ffd700 1 -- 172.21.15.112:0/2846952808 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8004520 con 0x7f7bac1466c0 2023-08-27T11:28:35.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7ba2ffd700 1 -- 172.21.15.112:0/2846952808 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba801e070 con 0x7f7bac1466c0 2023-08-27T11:28:35.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb46b9700 1 -- 172.21.15.112:0/2846952808 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bac13eaa0 con 0x7f7bac1466c0 2023-08-27T11:28:35.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7ba2ffd700 1 -- 172.21.15.112:0/2846952808 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80038a0 con 0x7f7bac1466c0 2023-08-27T11:28:35.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.616+0000 7f7bb46b9700 1 -- 172.21.15.112:0/2846952808 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bac13cf00 con 0x7f7bac1466c0 2023-08-27T11:28:35.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.617+0000 7f7bb46b9700 1 -- 172.21.15.112:0/2846952808 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bac13efc0 con 0x7f7bac1466c0 2023-08-27T11:28:35.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.618+0000 7f7ba2ffd700 1 -- 172.21.15.112:0/2846952808 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f7ba8003a00 con 0x7f7bac1466c0 2023-08-27T11:28:35.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.618+0000 7f7ba2ffd700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b9006a500 0x7f7b9006c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.618+0000 7f7ba2ffd700 1 -- 172.21.15.112:0/2846952808 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7f7ba8020070 con 0x7f7bac1466c0 2023-08-27T11:28:35.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.618+0000 7f7bb2455700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b9006a500 0x7f7b9006c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.619+0000 7f7bb2455700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b9006a500 0x7f7b9006c9c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0099f0 tx=0x7f7b9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.622+0000 7f7ba2ffd700 1 -- 172.21.15.112:0/2846952808 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7bac13efc0 con 0x7f7bac1466c0 2023-08-27T11:28:35.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.813+0000 7f7bb46b9700 1 -- 172.21.15.112:0/2846952808 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bac13efc0 con 0x7f7bac1466c0 2023-08-27T11:28:35.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.814+0000 7f7ba2ffd700 1 -- 172.21.15.112:0/2846952808 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+18341 (secure 0 0 0) 0x7f7bac13efc0 con 0x7f7bac1466c0 2023-08-27T11:28:35.816 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:35.816 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":142,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:31.728440+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs.volume_0000000000561463.meta","create_time":"2023-08-27T11:28:12.176044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000561463"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs.volume_0000000000561463.data","create_time":"2023-08-27T11:28:12.992996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000561463"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:35.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.816+0000 7f7bb46b9700 1 -- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b9006a500 msgr2=0x7f7b9006c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.816+0000 7f7bb46b9700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b9006a500 0x7f7b9006c9c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0099f0 tx=0x7f7b9c008040 comp rx=0 tx=0).stop 2023-08-27T11:28:35.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.816+0000 7f7bb46b9700 1 -- 172.21.15.112:0/2846952808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bac1466c0 msgr2=0x7f7bac0ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.816+0000 7f7bb46b9700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bac1466c0 0x7f7bac0ac890 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8000c00 tx=0x7f7ba8004750 comp rx=0 tx=0).stop 2023-08-27T11:28:35.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.816+0000 7f7bb46b9700 1 -- 172.21.15.112:0/2846952808 shutdown_connections 2023-08-27T11:28:35.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.816+0000 7f7bb46b9700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7b9006a500 0x7f7b9006c9c0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.816+0000 7f7bb46b9700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bac146bb0 0x7f7bac142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.816+0000 7f7bb46b9700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7bac0b4760 0x7f7bac13c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.816+0000 7f7bb46b9700 1 --2- 172.21.15.112:0/2846952808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bac1466c0 0x7f7bac0ac890 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.816+0000 7f7bb46b9700 1 -- 172.21.15.112:0/2846952808 >> 172.21.15.112:0/2846952808 conn(0x7f7bac0cff80 msgr2=0x7f7bac15f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:35.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.816+0000 7f7bb46b9700 1 -- 172.21.15.112:0/2846952808 shutdown_connections 2023-08-27T11:28:35.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.816+0000 7f7bb46b9700 1 -- 172.21.15.112:0/2846952808 wait complete. 2023-08-27T11:28:35.830 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000561463 and related pools 2023-08-27T11:28:35.830 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:35.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.948+0000 7fddee51c700 1 Processor -- start 2023-08-27T11:28:35.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.948+0000 7fddee51c700 1 -- start start 2023-08-27T11:28:35.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.949+0000 7fddee51c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdde81404c0 0x7fdde8142c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.949+0000 7fddee51c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde81408a0 con 0x7fdde8140bb0 2023-08-27T11:28:35.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.949+0000 7fddee51c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde8143490 con 0x7fdde80ad310 2023-08-27T11:28:35.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.949+0000 7fddee51c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde8143770 con 0x7fdde81404c0 2023-08-27T11:28:35.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.949+0000 7fdde7fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdde81404c0 0x7fdde8142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.949+0000 7fddecab9700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fdde8140bb0 0x7fdde806e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59252/0 (socket says 172.21.15.112:59252) 2023-08-27T11:28:35.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.949+0000 7fddecab9700 1 -- 172.21.15.112:0/281350395 learned_addr learned my addr 172.21.15.112:0/281350395 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:35.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.950+0000 7fdde7fff700 1 -- 172.21.15.112:0/281350395 >> v1:172.21.15.138:6789/0 conn(0x7fdde80ad310 legacy=0x7fdde80cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:35.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.950+0000 7fdde7fff700 1 -- 172.21.15.112:0/281350395 >> v1:172.21.15.112:6789/0 conn(0x7fdde8140bb0 legacy=0x7fdde806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.950+0000 7fdde7fff700 1 -- 172.21.15.112:0/281350395 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde8143a50 con 0x7fdde81404c0 2023-08-27T11:28:35.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.950+0000 7fdde6ffd700 1 -- 172.21.15.112:0/281350395 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240489835 0 0) 0x7fdde81408a0 con 0x7fdde8140bb0 2023-08-27T11:28:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.950+0000 7fdde7fff700 1 --2- 172.21.15.112:0/281350395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde81404c0 0x7fdde8142c10 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009840 tx=0x7fddd0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa33898db24d3fd4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.950+0000 7fdde6ffd700 1 -- 172.21.15.112:0/281350395 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0004230 con 0x7fdde81404c0 2023-08-27T11:28:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.950+0000 7fdde6ffd700 1 -- 172.21.15.112:0/281350395 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd0004390 con 0x7fdde81404c0 2023-08-27T11:28:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.950+0000 7fdde6ffd700 1 -- 172.21.15.112:0/281350395 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd00036a0 con 0x7fdde81404c0 2023-08-27T11:28:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.951+0000 7fddee51c700 1 -- 172.21.15.112:0/281350395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde81404c0 msgr2=0x7fdde8142c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.951+0000 7fddee51c700 1 --2- 172.21.15.112:0/281350395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde81404c0 0x7fdde8142c10 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009840 tx=0x7fddd0009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.951+0000 7fddee51c700 1 -- 172.21.15.112:0/281350395 shutdown_connections 2023-08-27T11:28:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.951+0000 7fddee51c700 1 --2- 172.21.15.112:0/281350395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde81404c0 0x7fdde8142c10 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.951+0000 7fddee51c700 1 -- 172.21.15.112:0/281350395 >> 172.21.15.112:0/281350395 conn(0x7fdde806cd30 msgr2=0x7fdde80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.951+0000 7fddee51c700 1 -- 172.21.15.112:0/281350395 shutdown_connections 2023-08-27T11:28:35.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.951+0000 7fddee51c700 1 -- 172.21.15.112:0/281350395 wait complete. 2023-08-27T11:28:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.952+0000 7fddee51c700 1 Processor -- start 2023-08-27T11:28:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.952+0000 7fddee51c700 1 -- start start 2023-08-27T11:28:35.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fddee51c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde80ad310 0x7fdde813c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fddee51c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdde81404c0 0x7fdde8143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fddee51c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde8140bb0 0x7fdde8160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fddee51c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde813db30 con 0x7fdde80ad310 2023-08-27T11:28:35.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fddee51c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde813dd20 con 0x7fdde8140bb0 2023-08-27T11:28:35.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fddee51c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde81608c0 con 0x7fdde81404c0 2023-08-27T11:28:35.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fdde7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde80ad310 0x7fdde813c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fdde7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde80ad310 0x7fdde813c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36640/0 (socket says 172.21.15.112:36640) 2023-08-27T11:28:35.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fdde7fff700 1 -- 172.21.15.112:0/612463104 learned_addr learned my addr 172.21.15.112:0/612463104 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:35.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fdde77fe700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdde81404c0 0x7fdde8143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fddecab9700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde8140bb0 0x7fdde8160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fdde7fff700 1 -- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdde81404c0 msgr2=0x7fdde8143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fdde7fff700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdde81404c0 0x7fdde8143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fdde7fff700 1 -- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde8140bb0 msgr2=0x7fdde8160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fdde7fff700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde8140bb0 0x7fdde8160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fdde7fff700 1 -- 172.21.15.112:0/612463104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd00094a0 con 0x7fdde80ad310 2023-08-27T11:28:35.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fdde77fe700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdde81404c0 0x7fdde8143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:35.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fdde7fff700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde80ad310 0x7fdde813c690 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fdde806eeb0 tx=0x7fddd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fdde4ff9700 1 -- 172.21.15.112:0/612463104 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0011630 con 0x7fdde80ad310 2023-08-27T11:28:35.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fdde4ff9700 1 -- 172.21.15.112:0/612463104 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd001e070 con 0x7fdde80ad310 2023-08-27T11:28:35.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.953+0000 7fddee51c700 1 -- 172.21.15.112:0/612463104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde8144510 con 0x7fdde80ad310 2023-08-27T11:28:35.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.954+0000 7fdde4ff9700 1 -- 172.21.15.112:0/612463104 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd00119b0 con 0x7fdde80ad310 2023-08-27T11:28:35.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.954+0000 7fddee51c700 1 -- 172.21.15.112:0/612463104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde81447d0 con 0x7fdde80ad310 2023-08-27T11:28:35.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.955+0000 7fddee51c700 1 -- 172.21.15.112:0/612463104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde80cc070 con 0x7fdde80ad310 2023-08-27T11:28:35.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.955+0000 7fdde4ff9700 1 -- 172.21.15.112:0/612463104 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fddd0011b10 con 0x7fdde80ad310 2023-08-27T11:28:35.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.955+0000 7fdde4ff9700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fddc806a4d0 0x7fddc806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.955+0000 7fdde4ff9700 1 -- 172.21.15.112:0/612463104 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7fddd0089f20 con 0x7fdde80ad310 2023-08-27T11:28:35.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.955+0000 7fdde77fe700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fddc806a4d0 0x7fddc806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.956+0000 7fdde77fe700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fddc806a4d0 0x7fddc806c990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fddd8007fd0 tx=0x7fddd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:35.960+0000 7fdde4ff9700 1 -- 172.21.15.112:0/612463104 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fddd0055180 con 0x7fdde80ad310 2023-08-27T11:28:36.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.190+0000 7fddee51c700 1 -- 172.21.15.112:0/612463104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdde8067a20 con 0x7fdde80ad310 2023-08-27T11:28:36.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.191+0000 7fdde4ff9700 1 -- 172.21.15.112:0/612463104 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4902 (secure 0 0 0) 0x7fddd0020090 con 0x7fdde80ad310 2023-08-27T11:28:36.193 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:36.193 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/739970856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":739970856},{"type":"v1","addr":"172.21.15.138:6837","nonce":739970856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3233069521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3233069521},{"type":"v1","addr":"172.21.15.112:6834","nonce":3233069521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2559241026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2559241026},{"type":"v1","addr":"172.21.15.112:6835","nonce":2559241026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:14.032146+0000","modified":"2023-08-27T11:28:29.716445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6836/529571603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":529571603},{"type":"v1","addr":"172.21.15.138:6836","nonce":529571603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"volume_0000000000561463","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-08-27T11:28:36.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.193+0000 7fddee51c700 1 -- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fddc806a4d0 msgr2=0x7fddc806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.193+0000 7fddee51c700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fddc806a4d0 0x7fddc806c990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fddd8007fd0 tx=0x7fddd8009040 comp rx=0 tx=0).stop 2023-08-27T11:28:36.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.193+0000 7fddee51c700 1 -- 172.21.15.112:0/612463104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde80ad310 msgr2=0x7fdde813c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.193+0000 7fddee51c700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde80ad310 0x7fdde813c690 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fdde806eeb0 tx=0x7fddd0004750 comp rx=0 tx=0).stop 2023-08-27T11:28:36.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.193+0000 7fddee51c700 1 -- 172.21.15.112:0/612463104 shutdown_connections 2023-08-27T11:28:36.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.193+0000 7fddee51c700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fddc806a4d0 0x7fddc806c990 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.193+0000 7fddee51c700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde8140bb0 0x7fdde8160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.193+0000 7fddee51c700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde80ad310 0x7fdde813c690 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.193+0000 7fddee51c700 1 --2- 172.21.15.112:0/612463104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdde81404c0 0x7fdde8143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.193+0000 7fddee51c700 1 -- 172.21.15.112:0/612463104 >> 172.21.15.112:0/612463104 conn(0x7fdde806cd30 msgr2=0x7fdde813f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:36.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.193+0000 7fddee51c700 1 -- 172.21.15.112:0/612463104 shutdown_connections 2023-08-27T11:28:36.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.193+0000 7fddee51c700 1 -- 172.21.15.112:0/612463104 wait complete. 2023-08-27T11:28:36.197 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 96 2023-08-27T11:28:36.207 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.325+0000 7f056fd05700 1 Processor -- start 2023-08-27T11:28:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.326+0000 7f056fd05700 1 -- start start 2023-08-27T11:28:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.326+0000 7f056fd05700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0568151060 0x7f056815e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.326+0000 7f056fd05700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f056815e990 0x7f056806adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.326+0000 7f056fd05700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f056806b2f0 0x7f056806d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056fd05700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056806b780 con 0x7f056806b2f0 2023-08-27T11:28:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056fd05700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05680b02f0 con 0x7f0568151060 2023-08-27T11:28:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056fd05700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05680b05d0 con 0x7f056815e990 2023-08-27T11:28:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056e2a2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f056806b2f0 0x7f056806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056e2a2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f056806b2f0 0x7f056806d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36652/0 (socket says 172.21.15.112:36652) 2023-08-27T11:28:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056e2a2700 1 -- 172.21.15.112:0/1437531424 learned_addr learned my addr 172.21.15.112:0/1437531424 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056d2a0700 1 --2- 172.21.15.112:0/1437531424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f056815e990 0x7f056806adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056daa1700 1 --2- 172.21.15.112:0/1437531424 >> v2:172.21.15.138:3300/0 conn(0x7f0568151060 0x7f056815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056e2a2700 1 -- 172.21.15.112:0/1437531424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f056815e990 msgr2=0x7f056806adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:36.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056e2a2700 1 --2- 172.21.15.112:0/1437531424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f056815e990 0x7f056806adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056e2a2700 1 -- 172.21.15.112:0/1437531424 >> v2:172.21.15.138:3300/0 conn(0x7f0568151060 msgr2=0x7f056815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:36.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056e2a2700 1 --2- 172.21.15.112:0/1437531424 >> v2:172.21.15.138:3300/0 conn(0x7f0568151060 0x7f056815e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056e2a2700 1 -- 172.21.15.112:0/1437531424 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05680b08b0 con 0x7f056806b2f0 2023-08-27T11:28:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056daa1700 1 --2- 172.21.15.112:0/1437531424 >> v2:172.21.15.138:3300/0 conn(0x7f0568151060 0x7f056815e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056e2a2700 1 --2- 172.21.15.112:0/1437531424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f056806b2f0 0x7f056806d580 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f056806e8c0 tx=0x7f056000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3ba8f686b7b796 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056ca9f700 1 -- 172.21.15.112:0/1437531424 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056000e070 con 0x7f056806b2f0 2023-08-27T11:28:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056ca9f700 1 -- 172.21.15.112:0/1437531424 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056000b910 con 0x7f056806b2f0 2023-08-27T11:28:36.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.327+0000 7f056ca9f700 1 -- 172.21.15.112:0/1437531424 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056000bcc0 con 0x7f056806b2f0 2023-08-27T11:28:36.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.328+0000 7f056fd05700 1 -- 172.21.15.112:0/1437531424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f056806b2f0 msgr2=0x7f056806d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.328+0000 7f056fd05700 1 --2- 172.21.15.112:0/1437531424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f056806b2f0 0x7f056806d580 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f056806e8c0 tx=0x7f056000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:36.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.328+0000 7f056fd05700 1 -- 172.21.15.112:0/1437531424 shutdown_connections 2023-08-27T11:28:36.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.328+0000 7f056fd05700 1 --2- 172.21.15.112:0/1437531424 >> v2:172.21.15.138:3300/0 conn(0x7f0568151060 0x7f056815e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.328+0000 7f056fd05700 1 --2- 172.21.15.112:0/1437531424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f056806b2f0 0x7f056806d580 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.328+0000 7f056fd05700 1 --2- 172.21.15.112:0/1437531424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f056815e990 0x7f056806adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.328+0000 7f056fd05700 1 -- 172.21.15.112:0/1437531424 >> 172.21.15.112:0/1437531424 conn(0x7f05680d0000 msgr2=0x7f05680d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:36.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.328+0000 7f056fd05700 1 -- 172.21.15.112:0/1437531424 shutdown_connections 2023-08-27T11:28:36.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.328+0000 7f056fd05700 1 -- 172.21.15.112:0/1437531424 wait complete. 2023-08-27T11:28:36.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.329+0000 7f056fd05700 1 Processor -- start 2023-08-27T11:28:36.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.329+0000 7f056fd05700 1 -- start start 2023-08-27T11:28:36.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056fd05700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f056806b2f0 0x7f0568070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056fd05700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0568151060 0x7f056814d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056fd05700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f056815e990 0x7f05680719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056fd05700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05680711c0 con 0x7f056806b2f0 2023-08-27T11:28:36.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056fd05700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056806fae0 con 0x7f0568151060 2023-08-27T11:28:36.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056fd05700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056806fdc0 con 0x7f056815e990 2023-08-27T11:28:36.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056daa1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f056806b2f0 0x7f0568070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056daa1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f056806b2f0 0x7f0568070c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36654/0 (socket says 172.21.15.112:36654) 2023-08-27T11:28:36.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056daa1700 1 -- 172.21.15.112:0/3189757060 learned_addr learned my addr 172.21.15.112:0/3189757060 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:36.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056d2a0700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0568151060 0x7f056814d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056e2a2700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f056815e990 0x7f05680719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056daa1700 1 -- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f056815e990 msgr2=0x7f05680719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056daa1700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f056815e990 0x7f05680719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056daa1700 1 -- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0568151060 msgr2=0x7f056814d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056daa1700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0568151060 0x7f056814d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056daa1700 1 -- 172.21.15.112:0/3189757060 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0560009b30 con 0x7f056806b2f0 2023-08-27T11:28:36.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056daa1700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f056806b2f0 0x7f0568070c80 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f055800d800 tx=0x7f055800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:36.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f055e7fc700 1 -- 172.21.15.112:0/3189757060 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558004030 con 0x7f056806b2f0 2023-08-27T11:28:36.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f055e7fc700 1 -- 172.21.15.112:0/3189757060 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0558004190 con 0x7f056806b2f0 2023-08-27T11:28:36.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056fd05700 1 -- 172.21.15.112:0/3189757060 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05680700a0 con 0x7f056806b2f0 2023-08-27T11:28:36.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f055e7fc700 1 -- 172.21.15.112:0/3189757060 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558004520 con 0x7f056806b2f0 2023-08-27T11:28:36.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.330+0000 7f056fd05700 1 -- 172.21.15.112:0/3189757060 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0568070660 con 0x7f056806b2f0 2023-08-27T11:28:36.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.331+0000 7f056fd05700 1 -- 172.21.15.112:0/3189757060 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05680ac950 con 0x7f056806b2f0 2023-08-27T11:28:36.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.332+0000 7f055e7fc700 1 -- 172.21.15.112:0/3189757060 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f05580036a0 con 0x7f056806b2f0 2023-08-27T11:28:36.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.332+0000 7f055e7fc700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f054c06a4d0 0x7f054c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.332+0000 7f055e7fc700 1 -- 172.21.15.112:0/3189757060 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7f0558089340 con 0x7f056806b2f0 2023-08-27T11:28:36.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.332+0000 7f056d2a0700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f054c06a4d0 0x7f054c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.333+0000 7f056d2a0700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f054c06a4d0 0x7f054c06c990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0568150d10 tx=0x7f0564008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:36.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.336+0000 7f055e7fc700 1 -- 172.21.15.112:0/3189757060 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0558057e90 con 0x7f056806b2f0 2023-08-27T11:28:36.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.566+0000 7f056fd05700 1 -- 172.21.15.112:0/3189757060 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05680ac740 con 0x7f056806b2f0 2023-08-27T11:28:36.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.567+0000 7f055e7fc700 1 -- 172.21.15.112:0/3189757060 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4902 (secure 0 0 0) 0x7f0558057a20 con 0x7f056806b2f0 2023-08-27T11:28:36.569 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:36.569 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/739970856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":739970856},{"type":"v1","addr":"172.21.15.138:6837","nonce":739970856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3233069521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3233069521},{"type":"v1","addr":"172.21.15.112:6834","nonce":3233069521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2559241026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2559241026},{"type":"v1","addr":"172.21.15.112:6835","nonce":2559241026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:14.032146+0000","modified":"2023-08-27T11:28:29.716445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6836/529571603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":529571603},{"type":"v1","addr":"172.21.15.138:6836","nonce":529571603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"volume_0000000000561463","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-08-27T11:28:36.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.569+0000 7f056fd05700 1 -- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f054c06a4d0 msgr2=0x7f054c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.569+0000 7f056fd05700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f054c06a4d0 0x7f054c06c990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0568150d10 tx=0x7f0564008040 comp rx=0 tx=0).stop 2023-08-27T11:28:36.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.569+0000 7f056fd05700 1 -- 172.21.15.112:0/3189757060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f056806b2f0 msgr2=0x7f0568070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.569+0000 7f056fd05700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f056806b2f0 0x7f0568070c80 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f055800d800 tx=0x7f055800dbc0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.569+0000 7f056fd05700 1 -- 172.21.15.112:0/3189757060 shutdown_connections 2023-08-27T11:28:36.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.569+0000 7f056fd05700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f054c06a4d0 0x7f054c06c990 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.569+0000 7f056fd05700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f056815e990 0x7f05680719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.569+0000 7f056fd05700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f056806b2f0 0x7f0568070c80 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.569+0000 7f056fd05700 1 --2- 172.21.15.112:0/3189757060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0568151060 0x7f056814d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.569+0000 7f056fd05700 1 -- 172.21.15.112:0/3189757060 >> 172.21.15.112:0/3189757060 conn(0x7f05680d0000 msgr2=0x7f05680cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:36.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.569+0000 7f056fd05700 1 -- 172.21.15.112:0/3189757060 shutdown_connections 2023-08-27T11:28:36.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.569+0000 7f056fd05700 1 -- 172.21.15.112:0/3189757060 wait complete. 2023-08-27T11:28:36.574 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 96 2023-08-27T11:28:36.583 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:36.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.701+0000 7efc074a9700 1 Processor -- start 2023-08-27T11:28:36.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.702+0000 7efc074a9700 1 -- start start 2023-08-27T11:28:36.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efc074a9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc00066450 0x7efc000701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efc074a9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efc0006e150 0x7efc000706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efc074a9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc000ad580 con 0x7efc0015d490 2023-08-27T11:28:36.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efc074a9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc000ad260 con 0x7efc0006e150 2023-08-27T11:28:36.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efc074a9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0006f790 con 0x7efc00066450 2023-08-27T11:28:36.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efc05245700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7efc0015d490 0x7efc0015d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59276/0 (socket says 172.21.15.112:59276) 2023-08-27T11:28:36.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efc05245700 1 -- 172.21.15.112:0/742296556 learned_addr learned my addr 172.21.15.112:0/742296556 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:36.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efc05a46700 1 --2- 172.21.15.112:0/742296556 >> v2:172.21.15.138:3300/0 conn(0x7efc0006e150 0x7efc000706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efc04a44700 1 --2- 172.21.15.112:0/742296556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc00066450 0x7efc000701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986758774 0 0) 0x7efc000ad580 con 0x7efc0015d490 2023-08-27T11:28:36.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbec0035b0 con 0x7efc0015d490 2023-08-27T11:28:36.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1727669602 0 0) 0x7efbec0035b0 con 0x7efc0015d490 2023-08-27T11:28:36.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc00066450 msgr2=0x7efc000701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:36.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 --2- 172.21.15.112:0/742296556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc00066450 0x7efc000701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 >> v2:172.21.15.138:3300/0 conn(0x7efc0006e150 msgr2=0x7efc000706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:36.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 --2- 172.21.15.112:0/742296556 >> v2:172.21.15.138:3300/0 conn(0x7efc0006e150 0x7efc000706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0006fa70 con 0x7efc0015d490 2023-08-27T11:28:36.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7efbf0001670 con 0x7efc0015d490 2023-08-27T11:28:36.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 >> v1:172.21.15.112:6789/0 conn(0x7efc0015d490 legacy=0x7efc0015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 --2- 172.21.15.112:0/742296556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbec003a00 0x7efbec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 --2- 172.21.15.112:0/742296556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbec004850 0x7efbec004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 --2- 172.21.15.112:0/742296556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc00066450 0x7efbec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc000ad260 con 0x7efbec004850 2023-08-27T11:28:36.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0006f790 con 0x7efbec003a00 2023-08-27T11:28:36.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbec00ba10 con 0x7efc00066450 2023-08-27T11:28:36.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.703+0000 7efc04a44700 1 --2- 172.21.15.112:0/742296556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbec004850 0x7efbec004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efc05245700 1 --2- 172.21.15.112:0/742296556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbec003a00 0x7efbec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efc05a46700 1 --2- 172.21.15.112:0/742296556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc00066450 0x7efbec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efc04a44700 1 -- 172.21.15.112:0/742296556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc00066450 msgr2=0x7efbec005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:36.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efc04a44700 1 --2- 172.21.15.112:0/742296556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc00066450 0x7efbec005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efc04a44700 1 -- 172.21.15.112:0/742296556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbec003a00 msgr2=0x7efbec003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:36.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efc04a44700 1 --2- 172.21.15.112:0/742296556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbec003a00 0x7efbec003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efc04a44700 1 -- 172.21.15.112:0/742296556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbec0035b0 con 0x7efbec004850 2023-08-27T11:28:36.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efc04a44700 1 --2- 172.21.15.112:0/742296556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbec004850 0x7efbec004cd0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009b60 tx=0x7efbec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4a5f02b221d3c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:36.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc00e6c0 con 0x7efbec004850 2023-08-27T11:28:36.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc00e820 con 0x7efbec004850 2023-08-27T11:28:36.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efbf7fff700 1 -- 172.21.15.112:0/742296556 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc00eb90 con 0x7efbec004850 2023-08-27T11:28:36.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efc074a9700 1 -- 172.21.15.112:0/742296556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbec004850 msgr2=0x7efbec004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efc074a9700 1 --2- 172.21.15.112:0/742296556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbec004850 0x7efbec004cd0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009b60 tx=0x7efbec005880 comp rx=0 tx=0).stop 2023-08-27T11:28:36.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.704+0000 7efc05a46700 1 -- 172.21.15.112:0/742296556 reap_dead start 2023-08-27T11:28:36.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.705+0000 7efc074a9700 1 -- 172.21.15.112:0/742296556 shutdown_connections 2023-08-27T11:28:36.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.705+0000 7efc074a9700 1 -- 172.21.15.112:0/742296556 >> 172.21.15.112:0/742296556 conn(0x7efc000b1380 msgr2=0x7efc000b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:36.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.705+0000 7efc074a9700 1 -- 172.21.15.112:0/742296556 shutdown_connections 2023-08-27T11:28:36.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.705+0000 7efc074a9700 1 -- 172.21.15.112:0/742296556 wait complete. 2023-08-27T11:28:36.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.706+0000 7efc074a9700 1 Processor -- start 2023-08-27T11:28:36.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.706+0000 7efc074a9700 1 -- start start 2023-08-27T11:28:36.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.706+0000 7efc074a9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbec004850 0x7efc00154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.706+0000 7efc074a9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc001f40c0 0x7efc001f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc074a9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc001f4540 0x7efc001f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc074a9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc001e3050 con 0x7efc001f40c0 2023-08-27T11:28:36.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc074a9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc001f6ff0 con 0x7efc001f4540 2023-08-27T11:28:36.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc074a9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc001e32a0 con 0x7efbec004850 2023-08-27T11:28:36.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc04a44700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc001f40c0 0x7efc001f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc04a44700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc001f40c0 0x7efc001f61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36674/0 (socket says 172.21.15.112:36674) 2023-08-27T11:28:36.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc04a44700 1 -- 172.21.15.112:0/1203010955 learned_addr learned my addr 172.21.15.112:0/1203010955 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:36.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc05245700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbec004850 0x7efc00154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc05a46700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc001f4540 0x7efc001f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc04a44700 1 -- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbec004850 msgr2=0x7efc00154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc04a44700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbec004850 0x7efc00154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc04a44700 1 -- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc001f4540 msgr2=0x7efc001f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc04a44700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc001f4540 0x7efc001f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc04a44700 1 -- 172.21.15.112:0/1203010955 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc001e3580 con 0x7efc001f40c0 2023-08-27T11:28:36.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc04a44700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc001f40c0 0x7efc001f61c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7efbec004740 tx=0x7efbfc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:36.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efbf5ffb700 1 -- 172.21.15.112:0/1203010955 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc00e6c0 con 0x7efc001f40c0 2023-08-27T11:28:36.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efbf5ffb700 1 -- 172.21.15.112:0/1203010955 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc003ab0 con 0x7efc001f40c0 2023-08-27T11:28:36.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc074a9700 1 -- 172.21.15.112:0/1203010955 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc001e3050 con 0x7efc001f40c0 2023-08-27T11:28:36.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efbf5ffb700 1 -- 172.21.15.112:0/1203010955 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc003e60 con 0x7efc001f40c0 2023-08-27T11:28:36.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.707+0000 7efc074a9700 1 -- 172.21.15.112:0/1203010955 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc001e3b40 con 0x7efc001f40c0 2023-08-27T11:28:36.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.708+0000 7efc074a9700 1 -- 172.21.15.112:0/1203010955 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc001e3eb0 con 0x7efc001f40c0 2023-08-27T11:28:36.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.709+0000 7efbf5ffb700 1 -- 172.21.15.112:0/1203010955 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7efbfc013470 con 0x7efc001f40c0 2023-08-27T11:28:36.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.709+0000 7efbf5ffb700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efbe406a4d0 0x7efbe406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.709+0000 7efbf5ffb700 1 -- 172.21.15.112:0/1203010955 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7efbfc08a4d0 con 0x7efc001f40c0 2023-08-27T11:28:36.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.709+0000 7efc05245700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efbe406a4d0 0x7efbe406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.710+0000 7efc05245700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efbe406a4d0 0x7efbe406c990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efbf0001cf0 tx=0x7efbf0007980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:36.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.714+0000 7efbf5ffb700 1 -- 172.21.15.112:0/1203010955 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efbfc055620 con 0x7efc001f40c0 2023-08-27T11:28:36.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.908+0000 7efc074a9700 1 -- 172.21.15.112:0/1203010955 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc00067a20 con 0x7efc001f40c0 2023-08-27T11:28:36.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.909+0000 7efbf5ffb700 1 -- 172.21.15.112:0/1203010955 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+18341 (secure 0 0 0) 0x7efbfc058c40 con 0x7efc001f40c0 2023-08-27T11:28:36.911 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":142,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:31.728440+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs.volume_0000000000561463.meta","create_time":"2023-08-27T11:28:12.176044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000561463"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs.volume_0000000000561463.data","create_time":"2023-08-27T11:28:12.992996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000561463"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:36.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.911+0000 7efc074a9700 1 -- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efbe406a4d0 msgr2=0x7efbe406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.911+0000 7efc074a9700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efbe406a4d0 0x7efbe406c990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efbf0001cf0 tx=0x7efbf0007980 comp rx=0 tx=0).stop 2023-08-27T11:28:36.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.911+0000 7efc074a9700 1 -- 172.21.15.112:0/1203010955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc001f40c0 msgr2=0x7efc001f61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.911+0000 7efc074a9700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc001f40c0 0x7efc001f61c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7efbec004740 tx=0x7efbfc0047c0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.911+0000 7efc074a9700 1 -- 172.21.15.112:0/1203010955 shutdown_connections 2023-08-27T11:28:36.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.911+0000 7efc074a9700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efbe406a4d0 0x7efbe406c990 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.911+0000 7efc074a9700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc001f4540 0x7efc001f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.911+0000 7efc074a9700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbec004850 0x7efc00154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.911+0000 7efc074a9700 1 --2- 172.21.15.112:0/1203010955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc001f40c0 0x7efc001f61c0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.911+0000 7efc074a9700 1 -- 172.21.15.112:0/1203010955 >> 172.21.15.112:0/1203010955 conn(0x7efc000b1380 msgr2=0x7efc001553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:36.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.911+0000 7efc074a9700 1 -- 172.21.15.112:0/1203010955 shutdown_connections 2023-08-27T11:28:36.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:36.911+0000 7efc074a9700 1 -- 172.21.15.112:0/1203010955 wait complete. 2023-08-27T11:28:36.925 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000561463 2023-08-27T11:28:37.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.044+0000 7fa0a2e15700 1 Processor -- start 2023-08-27T11:28:37.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.045+0000 7fa0a2e15700 1 -- start start 2023-08-27T11:28:37.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.045+0000 7fa0a2e15700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa09c0b17e0 0x7fa09c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.045+0000 7fa0a2e15700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa09c161ce0 0x7fa09c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.045+0000 7fa0a2e15700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09c161850 con 0x7fa09c0b1400 2023-08-27T11:28:37.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.045+0000 7fa0a2e15700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09c0cfc10 con 0x7fa09c0b17e0 2023-08-27T11:28:37.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.045+0000 7fa0a2e15700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09c161200 con 0x7fa09c161ce0 2023-08-27T11:28:37.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.045+0000 7fa09bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa09c0b17e0 0x7fa09c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.046+0000 7fa0a0bb1700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa09c0b1400 0x7fa09c06e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59294/0 (socket says 172.21.15.112:59294) 2023-08-27T11:28:37.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.046+0000 7fa0a0bb1700 1 -- 172.21.15.112:0/1973278279 learned_addr learned my addr 172.21.15.112:0/1973278279 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:37.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.046+0000 7fa0a13b2700 1 --2- 172.21.15.112:0/1973278279 >> v2:172.21.15.138:3300/0 conn(0x7fa09c161ce0 0x7fa09c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.046+0000 7fa09bfff700 1 -- 172.21.15.112:0/1973278279 >> v2:172.21.15.138:3300/0 conn(0x7fa09c161ce0 msgr2=0x7fa09c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:37.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.046+0000 7fa09bfff700 1 --2- 172.21.15.112:0/1973278279 >> v2:172.21.15.138:3300/0 conn(0x7fa09c161ce0 0x7fa09c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.046+0000 7fa09bfff700 1 -- 172.21.15.112:0/1973278279 >> v1:172.21.15.112:6789/0 conn(0x7fa09c0b1400 legacy=0x7fa09c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.046+0000 7fa09bfff700 1 -- 172.21.15.112:0/1973278279 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c1614e0 con 0x7fa09c0b17e0 2023-08-27T11:28:37.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.046+0000 7fa09b7fe700 1 -- 172.21.15.112:0/1973278279 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236275851 0 0) 0x7fa09c161850 con 0x7fa09c0b1400 2023-08-27T11:28:37.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.046+0000 7fa09bfff700 1 --2- 172.21.15.112:0/1973278279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa09c0b17e0 0x7fa09c0cf5a0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fa09c06df10 tx=0x7fa088009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3144f4ec0744460e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:37.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.046+0000 7fa09b7fe700 1 -- 172.21.15.112:0/1973278279 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088011830 con 0x7fa09c0b17e0 2023-08-27T11:28:37.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.046+0000 7fa09b7fe700 1 -- 172.21.15.112:0/1973278279 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa088011990 con 0x7fa09c0b17e0 2023-08-27T11:28:37.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.046+0000 7fa09b7fe700 1 -- 172.21.15.112:0/1973278279 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088011d20 con 0x7fa09c0b17e0 2023-08-27T11:28:37.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.047+0000 7fa0a2e15700 1 -- 172.21.15.112:0/1973278279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa09c0b17e0 msgr2=0x7fa09c0cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.047+0000 7fa0a2e15700 1 --2- 172.21.15.112:0/1973278279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa09c0b17e0 0x7fa09c0cf5a0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fa09c06df10 tx=0x7fa088009b20 comp rx=0 tx=0).stop 2023-08-27T11:28:37.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.047+0000 7fa0a2e15700 1 -- 172.21.15.112:0/1973278279 shutdown_connections 2023-08-27T11:28:37.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.047+0000 7fa0a2e15700 1 --2- 172.21.15.112:0/1973278279 >> v2:172.21.15.138:3300/0 conn(0x7fa09c161ce0 0x7fa09c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.047+0000 7fa0a2e15700 1 --2- 172.21.15.112:0/1973278279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa09c0b17e0 0x7fa09c0cf5a0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.047+0000 7fa0a2e15700 1 -- 172.21.15.112:0/1973278279 >> 172.21.15.112:0/1973278279 conn(0x7fa09c0d0000 msgr2=0x7fa09c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:37.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.047+0000 7fa0a2e15700 1 -- 172.21.15.112:0/1973278279 shutdown_connections 2023-08-27T11:28:37.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.047+0000 7fa0a2e15700 1 -- 172.21.15.112:0/1973278279 wait complete. 2023-08-27T11:28:37.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.048+0000 7fa0a2e15700 1 Processor -- start 2023-08-27T11:28:37.056 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.048+0000 7fa0a2e15700 1 -- start start 2023-08-27T11:28:37.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a2e15700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa09c0b1400 0x7fa09c0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a2e15700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa09c0b17e0 0x7fa09c0c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a2e15700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa09c161ce0 0x7fa09c0c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a2e15700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0c9ae0 con 0x7fa09c161ce0 2023-08-27T11:28:37.057 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a2e15700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0c8c30 con 0x7fa09c0b17e0 2023-08-27T11:28:37.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a2e15700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0c8e60 con 0x7fa09c0b1400 2023-08-27T11:28:37.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a13b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa09c161ce0 0x7fa09c0c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a13b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa09c161ce0 0x7fa09c0c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36690/0 (socket says 172.21.15.112:36690) 2023-08-27T11:28:37.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a13b2700 1 -- 172.21.15.112:0/3162378251 learned_addr learned my addr 172.21.15.112:0/3162378251 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:37.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a0bb1700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa09c0b1400 0x7fa09c0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa09bfff700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa09c0b17e0 0x7fa09c0c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a13b2700 1 -- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa09c0b1400 msgr2=0x7fa09c0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a13b2700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa09c0b1400 0x7fa09c0c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a13b2700 1 -- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa09c0b17e0 msgr2=0x7fa09c0c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a13b2700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa09c0b17e0 0x7fa09c0c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a13b2700 1 -- 172.21.15.112:0/3162378251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0880094a0 con 0x7fa09c161ce0 2023-08-27T11:28:37.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.049+0000 7fa0a13b2700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa09c161ce0 0x7fa09c0c6230 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fa094007e40 tx=0x7fa09400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:37.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.050+0000 7fa0997fa700 1 -- 172.21.15.112:0/3162378251 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09400eea0 con 0x7fa09c161ce0 2023-08-27T11:28:37.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.050+0000 7fa0997fa700 1 -- 172.21.15.112:0/3162378251 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa094014420 con 0x7fa09c161ce0 2023-08-27T11:28:37.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.050+0000 7fa0a2e15700 1 -- 172.21.15.112:0/3162378251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c0c9140 con 0x7fa09c161ce0 2023-08-27T11:28:37.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.050+0000 7fa0997fa700 1 -- 172.21.15.112:0/3162378251 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0940147b0 con 0x7fa09c161ce0 2023-08-27T11:28:37.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.050+0000 7fa0a2e15700 1 -- 172.21.15.112:0/3162378251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c0c9700 con 0x7fa09c161ce0 2023-08-27T11:28:37.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.050+0000 7fa0a2e15700 1 -- 172.21.15.112:0/3162378251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09c0c9c20 con 0x7fa09c161ce0 2023-08-27T11:28:37.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.051+0000 7fa0997fa700 1 -- 172.21.15.112:0/3162378251 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa094014910 con 0x7fa09c161ce0 2023-08-27T11:28:37.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.051+0000 7fa0997fa700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa08006a4d0 0x7fa08006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.051+0000 7fa0997fa700 1 -- 172.21.15.112:0/3162378251 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6140+0+0 (secure 0 0 0) 0x7fa094089520 con 0x7fa09c161ce0 2023-08-27T11:28:37.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.052+0000 7fa0a0bb1700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa08006a4d0 0x7fa08006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.052+0000 7fa0a0bb1700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa08006a4d0 0x7fa08006c990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa090000f60 tx=0x7fa090007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:37.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.056+0000 7fa0997fa700 1 -- 172.21.15.112:0/3162378251 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa09c0c9c20 con 0x7fa09c161ce0 2023-08-27T11:28:37.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.288+0000 7fa0a2e15700 1 -- 172.21.15.112:0/3162378251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000561463"} v 0) v1 -- 0x7fa09c0c9c20 con 0x7fa09c161ce0 2023-08-27T11:28:37.290 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:28:37.289+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:28:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.293+0000 7fa0997fa700 1 -- 172.21.15.112:0/3162378251 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000561463"}]=0 volume_0000000000561463 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) v1 ==== 195+0+0 (secure 0 0 0) 0x7fa09c0c9c20 con 0x7fa09c161ce0 2023-08-27T11:28:37.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.295+0000 7fa0a2e15700 1 -- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa08006a4d0 msgr2=0x7fa08006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.295+0000 7fa0a2e15700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa08006a4d0 0x7fa08006c990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa090000f60 tx=0x7fa090007c90 comp rx=0 tx=0).stop 2023-08-27T11:28:37.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.295+0000 7fa0a2e15700 1 -- 172.21.15.112:0/3162378251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa09c161ce0 msgr2=0x7fa09c0c6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.295+0000 7fa0a2e15700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa09c161ce0 0x7fa09c0c6230 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fa094007e40 tx=0x7fa09400e3c0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.295+0000 7fa0a2e15700 1 -- 172.21.15.112:0/3162378251 shutdown_connections 2023-08-27T11:28:37.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.295+0000 7fa0a2e15700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa08006a4d0 0x7fa08006c990 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.295+0000 7fa0a2e15700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa09c161ce0 0x7fa09c0c6230 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.295+0000 7fa0a2e15700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa09c0b1400 0x7fa09c0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.295+0000 7fa0a2e15700 1 --2- 172.21.15.112:0/3162378251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa09c0b17e0 0x7fa09c0c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.295+0000 7fa0a2e15700 1 -- 172.21.15.112:0/3162378251 >> 172.21.15.112:0/3162378251 conn(0x7fa09c0d0000 msgr2=0x7fa09c1578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:37.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.295+0000 7fa0a2e15700 1 -- 172.21.15.112:0/3162378251 shutdown_connections 2023-08-27T11:28:37.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.295+0000 7fa0a2e15700 1 -- 172.21.15.112:0/3162378251 wait complete. 2023-08-27T11:28:37.300 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:28:37.300 INFO:tasks.ceph.mds.d.smithi138.stderr: -3288> 2023-08-27T11:28:19.608+0000 7f8a3bdef700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:28:37.301 INFO:teuthology.orchestra.run.smithi112.stderr:volume_0000000000561463 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:28:37.303 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:28:37.309 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000561463 --yes-i-really-mean-it 2023-08-27T11:28:37.328 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2023-08-27T11:28:37.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.470+0000 7f2d76298700 1 Processor -- start 2023-08-27T11:28:37.471 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.470+0000 7f2d76298700 1 -- start start 2023-08-27T11:28:37.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.470+0000 7f2d76298700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2d701447a0 0x7f2d700cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.470+0000 7f2d76298700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2d70066720 0x7f2d700b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.470+0000 7f2d76298700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d700d00a0 con 0x7f2d700d0920 2023-08-27T11:28:37.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.470+0000 7f2d76298700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d700b3670 con 0x7f2d701447a0 2023-08-27T11:28:37.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.470+0000 7f2d76298700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d70144b80 con 0x7f2d70066720 2023-08-27T11:28:37.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.471+0000 7f2d6ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2d701447a0 0x7f2d700cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.471+0000 7f2d6ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2d701447a0 0x7f2d700cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36694/0 (socket says 172.21.15.112:36694) 2023-08-27T11:28:37.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.471+0000 7f2d6ffff700 1 -- 172.21.15.112:0/1962225786 learned_addr learned my addr 172.21.15.112:0/1962225786 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:37.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.471+0000 7f2d74835700 1 --2- 172.21.15.112:0/1962225786 >> v2:172.21.15.138:3300/0 conn(0x7f2d70066720 0x7f2d700b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.471+0000 7f2d6ffff700 1 -- 172.21.15.112:0/1962225786 >> v2:172.21.15.138:3300/0 conn(0x7f2d70066720 msgr2=0x7f2d700b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:37.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.471+0000 7f2d6ffff700 1 --2- 172.21.15.112:0/1962225786 >> v2:172.21.15.138:3300/0 conn(0x7f2d70066720 0x7f2d700b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.471+0000 7f2d6ffff700 1 -- 172.21.15.112:0/1962225786 >> v1:172.21.15.138:6789/0 conn(0x7f2d700d0920 legacy=0x7f2d70066010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.471+0000 7f2d6ffff700 1 -- 172.21.15.112:0/1962225786 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700b3ef0 con 0x7f2d701447a0 2023-08-27T11:28:37.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.471+0000 7f2d6ffff700 1 --2- 172.21.15.112:0/1962225786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d701447a0 0x7f2d700cfa70 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f2d700cffb0 tx=0x7f2d60009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1d64e5bf22439f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:37.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.471+0000 7f2d6effd700 1 -- 172.21.15.112:0/1962225786 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60011830 con 0x7f2d701447a0 2023-08-27T11:28:37.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.471+0000 7f2d6effd700 1 -- 172.21.15.112:0/1962225786 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d60011990 con 0x7f2d701447a0 2023-08-27T11:28:37.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.471+0000 7f2d6effd700 1 -- 172.21.15.112:0/1962225786 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60011d20 con 0x7f2d701447a0 2023-08-27T11:28:37.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.472+0000 7f2d76298700 1 -- 172.21.15.112:0/1962225786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d701447a0 msgr2=0x7f2d700cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.472+0000 7f2d76298700 1 --2- 172.21.15.112:0/1962225786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d701447a0 0x7f2d700cfa70 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f2d700cffb0 tx=0x7f2d60009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.472+0000 7f2d76298700 1 -- 172.21.15.112:0/1962225786 shutdown_connections 2023-08-27T11:28:37.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.472+0000 7f2d76298700 1 --2- 172.21.15.112:0/1962225786 >> v2:172.21.15.138:3300/0 conn(0x7f2d70066720 0x7f2d700b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.472+0000 7f2d76298700 1 --2- 172.21.15.112:0/1962225786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d701447a0 0x7f2d700cfa70 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.472+0000 7f2d76298700 1 -- 172.21.15.112:0/1962225786 >> 172.21.15.112:0/1962225786 conn(0x7f2d7006f790 msgr2=0x7f2d7006fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:37.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.472+0000 7f2d76298700 1 -- 172.21.15.112:0/1962225786 shutdown_connections 2023-08-27T11:28:37.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.472+0000 7f2d76298700 1 -- 172.21.15.112:0/1962225786 wait complete. 2023-08-27T11:28:37.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.473+0000 7f2d76298700 1 Processor -- start 2023-08-27T11:28:37.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.474+0000 7f2d76298700 1 -- start start 2023-08-27T11:28:37.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d76298700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d70066720 0x7f2d701f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d76298700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d700d0920 0x7f2d700afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d76298700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d701447a0 0x7f2d700b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d76298700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701f4670 con 0x7f2d70066720 2023-08-27T11:28:37.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d76298700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701f48a0 con 0x7f2d700d0920 2023-08-27T11:28:37.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d76298700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701f4b80 con 0x7f2d701447a0 2023-08-27T11:28:37.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d6ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d70066720 0x7f2d701f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d6ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d70066720 0x7f2d701f4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36698/0 (socket says 172.21.15.112:36698) 2023-08-27T11:28:37.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d6ffff700 1 -- 172.21.15.112:0/1586392723 learned_addr learned my addr 172.21.15.112:0/1586392723 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:37.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d6f7fe700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d700d0920 0x7f2d700afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d74835700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d701447a0 0x7f2d700b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d6ffff700 1 -- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d701447a0 msgr2=0x7f2d700b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d6ffff700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d701447a0 0x7f2d700b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d6ffff700 1 -- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d700d0920 msgr2=0x7f2d700afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d6ffff700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d700d0920 0x7f2d700afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d6ffff700 1 -- 172.21.15.112:0/1586392723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d60009550 con 0x7f2d70066720 2023-08-27T11:28:37.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d6f7fe700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d700d0920 0x7f2d700afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:37.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d6ffff700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d70066720 0x7f2d701f4130 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009c10 tx=0x7f2d60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:37.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.475+0000 7f2d6cff9700 1 -- 172.21.15.112:0/1586392723 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60004590 con 0x7f2d70066720 2023-08-27T11:28:37.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.476+0000 7f2d6cff9700 1 -- 172.21.15.112:0/1586392723 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6001e070 con 0x7f2d70066720 2023-08-27T11:28:37.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.476+0000 7f2d76298700 1 -- 172.21.15.112:0/1586392723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d701f4e60 con 0x7f2d70066720 2023-08-27T11:28:37.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.476+0000 7f2d6cff9700 1 -- 172.21.15.112:0/1586392723 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d600038e0 con 0x7f2d70066720 2023-08-27T11:28:37.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.476+0000 7f2d76298700 1 -- 172.21.15.112:0/1586392723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d700b00f0 con 0x7f2d70066720 2023-08-27T11:28:37.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.476+0000 7f2d76298700 1 -- 172.21.15.112:0/1586392723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d700b28b0 con 0x7f2d70066720 2023-08-27T11:28:37.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.477+0000 7f2d6cff9700 1 -- 172.21.15.112:0/1586392723 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f2d60003a40 con 0x7f2d70066720 2023-08-27T11:28:37.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.477+0000 7f2d6cff9700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2d5c06a4d0 0x7f2d5c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.477+0000 7f2d6cff9700 1 -- 172.21.15.112:0/1586392723 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6226+0+0 (secure 0 0 0) 0x7f2d60089a70 con 0x7f2d70066720 2023-08-27T11:28:37.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.477+0000 7f2d6f7fe700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2d5c06a4d0 0x7f2d5c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.478+0000 7f2d6f7fe700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2d5c06a4d0 0x7f2d5c06c990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2d700b6790 tx=0x7f2d64009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:37.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.482+0000 7f2d6cff9700 1 -- 172.21.15.112:0/1586392723 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2d60054c70 con 0x7f2d70066720 2023-08-27T11:28:37.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:37.716+0000 7f2d76298700 1 -- 172.21.15.112:0/1586392723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000561463", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2d70070710 con 0x7f2d70066720 2023-08-27T11:28:38.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.342+0000 7f2d6cff9700 1 -- 172.21.15.112:0/1586392723 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000561463", "yes_i_really_mean_it": true}]=0 v98) v1 ==== 121+0+0 (secure 0 0 0) 0x7f2d600200c0 con 0x7f2d70066720 2023-08-27T11:28:38.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.344+0000 7f2d76298700 1 -- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2d5c06a4d0 msgr2=0x7f2d5c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:38.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.344+0000 7f2d76298700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2d5c06a4d0 0x7f2d5c06c990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2d700b6790 tx=0x7f2d64009040 comp rx=0 tx=0).stop 2023-08-27T11:28:38.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.344+0000 7f2d76298700 1 -- 172.21.15.112:0/1586392723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d70066720 msgr2=0x7f2d701f4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:38.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.344+0000 7f2d76298700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d70066720 0x7f2d701f4130 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009c10 tx=0x7f2d60004750 comp rx=0 tx=0).stop 2023-08-27T11:28:38.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.345+0000 7f2d76298700 1 -- 172.21.15.112:0/1586392723 shutdown_connections 2023-08-27T11:28:38.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.345+0000 7f2d76298700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2d5c06a4d0 0x7f2d5c06c990 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.345+0000 7f2d76298700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d701447a0 0x7f2d700b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.345+0000 7f2d76298700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d70066720 0x7f2d701f4130 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.345+0000 7f2d76298700 1 --2- 172.21.15.112:0/1586392723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d700d0920 0x7f2d700afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.345+0000 7f2d76298700 1 -- 172.21.15.112:0/1586392723 >> 172.21.15.112:0/1586392723 conn(0x7f2d7006f790 msgr2=0x7f2d7006b600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:38.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.345+0000 7f2d76298700 1 -- 172.21.15.112:0/1586392723 shutdown_connections 2023-08-27T11:28:38.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.345+0000 7f2d76298700 1 -- 172.21.15.112:0/1586392723 wait complete. 2023-08-27T11:28:38.359 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000561463.meta cephfs.volume_0000000000561463.meta --yes-i-really-really-mean-it 2023-08-27T11:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.484+0000 7f5c00891700 1 Processor -- start 2023-08-27T11:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.484+0000 7f5c00891700 1 -- start start 2023-08-27T11:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.485+0000 7f5c00891700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5bfc143dc0 0x7f5bfc0b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.485+0000 7f5c00891700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bfc0b0b10 con 0x7f5bfc0b1420 2023-08-27T11:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.485+0000 7f5c00891700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bfc06d850 con 0x7f5bfc1439e0 2023-08-27T11:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.485+0000 7f5c00891700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bfc15b460 con 0x7f5bfc143dc0 2023-08-27T11:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.485+0000 7f5bf9d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5bfc143dc0 0x7f5bfc0b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.485+0000 7f5bfad9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5bfc0b1420 0x7f5bfc15a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59324/0 (socket says 172.21.15.112:59324) 2023-08-27T11:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.485+0000 7f5bf9d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5bfc143dc0 0x7f5bfc0b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36708/0 (socket says 172.21.15.112:36708) 2023-08-27T11:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.485+0000 7f5bfad9d700 1 -- 172.21.15.112:0/3073706412 learned_addr learned my addr 172.21.15.112:0/3073706412 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.485+0000 7f5bf9d9b700 1 -- 172.21.15.112:0/3073706412 >> v1:172.21.15.138:6789/0 conn(0x7f5bfc1439e0 legacy=0x7f5bfc0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.485+0000 7f5bf9d9b700 1 -- 172.21.15.112:0/3073706412 >> v1:172.21.15.112:6789/0 conn(0x7f5bfc0b1420 legacy=0x7f5bfc15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.485+0000 7f5bf959a700 1 -- 172.21.15.112:0/3073706412 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415054234 0 0) 0x7f5bfc0b0b10 con 0x7f5bfc0b1420 2023-08-27T11:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.486+0000 7f5bf9d9b700 1 -- 172.21.15.112:0/3073706412 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc15b740 con 0x7f5bfc143dc0 2023-08-27T11:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.486+0000 7f5bf9d9b700 1 --2- 172.21.15.112:0/3073706412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc143dc0 0x7f5bfc0b05d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009840 tx=0x7f5bec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386d54db42079721 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.486+0000 7f5bf959a700 1 -- 172.21.15.112:0/3073706412 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec011830 con 0x7f5bfc143dc0 2023-08-27T11:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.486+0000 7f5bf959a700 1 -- 172.21.15.112:0/3073706412 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bec011990 con 0x7f5bfc143dc0 2023-08-27T11:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.486+0000 7f5bf959a700 1 -- 172.21.15.112:0/3073706412 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec011d20 con 0x7f5bfc143dc0 2023-08-27T11:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.486+0000 7f5c00891700 1 -- 172.21.15.112:0/3073706412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc143dc0 msgr2=0x7f5bfc0b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.486+0000 7f5c00891700 1 --2- 172.21.15.112:0/3073706412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc143dc0 0x7f5bfc0b05d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009840 tx=0x7f5bec009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.486+0000 7f5c00891700 1 -- 172.21.15.112:0/3073706412 shutdown_connections 2023-08-27T11:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.486+0000 7f5c00891700 1 --2- 172.21.15.112:0/3073706412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc143dc0 0x7f5bfc0b05d0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.486+0000 7f5c00891700 1 -- 172.21.15.112:0/3073706412 >> 172.21.15.112:0/3073706412 conn(0x7f5bfc0b1be0 msgr2=0x7f5bfc0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:38.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.486+0000 7f5c00891700 1 -- 172.21.15.112:0/3073706412 shutdown_connections 2023-08-27T11:28:38.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.486+0000 7f5c00891700 1 -- 172.21.15.112:0/3073706412 wait complete. 2023-08-27T11:28:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.488+0000 7f5c00891700 1 Processor -- start 2023-08-27T11:28:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.488+0000 7f5c00891700 1 -- start start 2023-08-27T11:28:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.488+0000 7f5c00891700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc0b1420 0x7f5bfc0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5c00891700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bfc1439e0 0x7f5bfc0cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5c00891700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bfc1f4320 0x7f5bfc13ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5c00891700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc0af310 con 0x7f5bfc0b1420 2023-08-27T11:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5c00891700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc1f0ce0 con 0x7f5bfc1f4320 2023-08-27T11:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5c00891700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc1f4820 con 0x7f5bfc1439e0 2023-08-27T11:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5bfa59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc0b1420 0x7f5bfc0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5bfa59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc0b1420 0x7f5bfc0aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36712/0 (socket says 172.21.15.112:36712) 2023-08-27T11:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5bfa59c700 1 -- 172.21.15.112:0/940854923 learned_addr learned my addr 172.21.15.112:0/940854923 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5bf9d9b700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bfc1439e0 0x7f5bfc0cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5bfad9d700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bfc1f4320 0x7f5bfc13ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5bfa59c700 1 -- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bfc1439e0 msgr2=0x7f5bfc0cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5bfa59c700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bfc1439e0 0x7f5bfc0cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5bfa59c700 1 -- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bfc1f4320 msgr2=0x7f5bfc13ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5bfa59c700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bfc1f4320 0x7f5bfc13ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5bfa59c700 1 -- 172.21.15.112:0/940854923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec0094a0 con 0x7f5bfc0b1420 2023-08-27T11:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5bfa59c700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc0b1420 0x7f5bfc0aedd0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000c4e0 tx=0x7f5bf000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5beaffd700 1 -- 172.21.15.112:0/940854923 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf00052d0 con 0x7f5bfc0b1420 2023-08-27T11:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5beaffd700 1 -- 172.21.15.112:0/940854923 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf0005430 con 0x7f5bfc0b1420 2023-08-27T11:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5beaffd700 1 -- 172.21.15.112:0/940854923 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf00038c0 con 0x7f5bfc0b1420 2023-08-27T11:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5c00891700 1 -- 172.21.15.112:0/940854923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bfc1f4b00 con 0x7f5bfc0b1420 2023-08-27T11:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.489+0000 7f5c00891700 1 -- 172.21.15.112:0/940854923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bfc1f50c0 con 0x7f5bfc0b1420 2023-08-27T11:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.490+0000 7f5c00891700 1 -- 172.21.15.112:0/940854923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bfc1f5280 con 0x7f5bfc0b1420 2023-08-27T11:28:38.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.491+0000 7f5beaffd700 1 -- 172.21.15.112:0/940854923 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5bf0003a20 con 0x7f5bfc0b1420 2023-08-27T11:28:38.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.491+0000 7f5beaffd700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5bdc06a4d0 0x7f5bdc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:38.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.491+0000 7f5beaffd700 1 -- 172.21.15.112:0/940854923 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6226+0+0 (secure 0 0 0) 0x7f5bf0088240 con 0x7f5bfc0b1420 2023-08-27T11:28:38.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.491+0000 7f5bf9d9b700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5bdc06a4d0 0x7f5bdc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:38.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.492+0000 7f5bf9d9b700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5bdc06a4d0 0x7f5bdc06c990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009840 tx=0x7f5bec01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:38.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.495+0000 7f5beaffd700 1 -- 172.21.15.112:0/940854923 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5bf00532a0 con 0x7f5bfc0b1420 2023-08-27T11:28:38.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:38.688+0000 7f5c00891700 1 -- 172.21.15.112:0/940854923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000561463.meta", "pool2": "cephfs.volume_0000000000561463.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5bfc067a20 con 0x7f5bfc0b1420 2023-08-27T11:28:39.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.351+0000 7f5beaffd700 1 -- 172.21.15.112:0/940854923 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000561463.meta", "pool2": "cephfs.volume_0000000000561463.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000561463.meta' removed v144) v1 ==== 241+0+0 (secure 0 0 0) 0x7f5bf00568c0 con 0x7f5bfc0b1420 2023-08-27T11:28:39.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.354+0000 7f5c00891700 1 -- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5bdc06a4d0 msgr2=0x7f5bdc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.354+0000 7f5c00891700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5bdc06a4d0 0x7f5bdc06c990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009840 tx=0x7f5bec01c040 comp rx=0 tx=0).stop 2023-08-27T11:28:39.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.354+0000 7f5c00891700 1 -- 172.21.15.112:0/940854923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc0b1420 msgr2=0x7f5bfc0aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.354+0000 7f5c00891700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc0b1420 0x7f5bfc0aedd0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000c4e0 tx=0x7f5bf000c8a0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.354+0000 7f5c00891700 1 -- 172.21.15.112:0/940854923 shutdown_connections 2023-08-27T11:28:39.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.354+0000 7f5c00891700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5bdc06a4d0 0x7f5bdc06c990 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.354+0000 7f5c00891700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bfc1f4320 0x7f5bfc13ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.354+0000 7f5c00891700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc0b1420 0x7f5bfc0aedd0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.354+0000 7f5c00891700 1 --2- 172.21.15.112:0/940854923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5bfc1439e0 0x7f5bfc0cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.354+0000 7f5c00891700 1 -- 172.21.15.112:0/940854923 >> 172.21.15.112:0/940854923 conn(0x7f5bfc0b1be0 msgr2=0x7f5bfc159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:39.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.354+0000 7f5c00891700 1 -- 172.21.15.112:0/940854923 shutdown_connections 2023-08-27T11:28:39.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.354+0000 7f5c00891700 1 -- 172.21.15.112:0/940854923 wait complete. 2023-08-27T11:28:39.359 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs.volume_0000000000561463.meta' removed 2023-08-27T11:28:39.372 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000561463.data cephfs.volume_0000000000561463.data --yes-i-really-really-mean-it 2023-08-27T11:28:39.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.496+0000 7fa424f58700 1 Processor -- start 2023-08-27T11:28:39.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.496+0000 7fa424f58700 1 -- start start 2023-08-27T11:28:39.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.497+0000 7fa424f58700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4200b3de0 0x7fa42015e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.497+0000 7fa424f58700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa420161390 con 0x7fa420157ae0 2023-08-27T11:28:39.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.497+0000 7fa424f58700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4200b35f0 con 0x7fa42006f790 2023-08-27T11:28:39.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.497+0000 7fa424f58700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200b1aa0 con 0x7fa4200b3de0 2023-08-27T11:28:39.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.497+0000 7fa41e59c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa420157ae0 0x7fa420161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59334/0 (socket says 172.21.15.112:59334) 2023-08-27T11:28:39.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.497+0000 7fa41e59c700 1 -- 172.21.15.112:0/3212165293 learned_addr learned my addr 172.21.15.112:0/3212165293 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:39.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.497+0000 7fa41dd9b700 1 --2- 172.21.15.112:0/3212165293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4200b3de0 0x7fa42015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.497+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401526131 0 0) 0x7fa420161390 con 0x7fa420157ae0 2023-08-27T11:28:39.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.497+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4080035b0 con 0x7fa420157ae0 2023-08-27T11:28:39.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3938508186 0 0) 0x7fa4080035b0 con 0x7fa420157ae0 2023-08-27T11:28:39.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4200b3de0 msgr2=0x7fa42015e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:39.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 --2- 172.21.15.112:0/3212165293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4200b3de0 0x7fa42015e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 >> v1:172.21.15.138:6789/0 conn(0x7fa42006f790 legacy=0x7fa4200b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4200b4330 con 0x7fa420157ae0 2023-08-27T11:28:39.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fa414001670 con 0x7fa420157ae0 2023-08-27T11:28:39.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 >> v1:172.21.15.112:6789/0 conn(0x7fa420157ae0 legacy=0x7fa420161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 --2- 172.21.15.112:0/3212165293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa408003a30 0x7fa408003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 --2- 172.21.15.112:0/3212165293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4200b3de0 0x7fa408004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 --2- 172.21.15.112:0/3212165293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa408004d90 0x7fa408005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200b1aa0 con 0x7fa408003a30 2023-08-27T11:28:39.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40800bac0 con 0x7fa408004d90 2023-08-27T11:28:39.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40800bc80 con 0x7fa4200b3de0 2023-08-27T11:28:39.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41e59c700 1 --2- 172.21.15.112:0/3212165293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa408003a30 0x7fa408003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41dd9b700 1 --2- 172.21.15.112:0/3212165293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4200b3de0 0x7fa408004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41ed9d700 1 --2- 172.21.15.112:0/3212165293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa408004d90 0x7fa408005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41e59c700 1 -- 172.21.15.112:0/3212165293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4200b3de0 msgr2=0x7fa408004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:39.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41e59c700 1 --2- 172.21.15.112:0/3212165293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4200b3de0 0x7fa408004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41e59c700 1 -- 172.21.15.112:0/3212165293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa408004d90 msgr2=0x7fa408005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:39.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41e59c700 1 --2- 172.21.15.112:0/3212165293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa408004d90 0x7fa408005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41e59c700 1 -- 172.21.15.112:0/3212165293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4080035b0 con 0x7fa408003a30 2023-08-27T11:28:39.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41e59c700 1 --2- 172.21.15.112:0/3212165293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa408003a30 0x7fa408003e50 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fa41400c270 tx=0x7fa408005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e12d3bad4043a2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:39.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414007c10 con 0x7fa408003a30 2023-08-27T11:28:39.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41400cdb0 con 0x7fa408003a30 2023-08-27T11:28:39.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.498+0000 7fa41d59a700 1 -- 172.21.15.112:0/3212165293 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414005ba0 con 0x7fa408003a30 2023-08-27T11:28:39.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.499+0000 7fa424f58700 1 -- 172.21.15.112:0/3212165293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa408003a30 msgr2=0x7fa408003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.499+0000 7fa424f58700 1 --2- 172.21.15.112:0/3212165293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa408003a30 0x7fa408003e50 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fa41400c270 tx=0x7fa408005880 comp rx=0 tx=0).stop 2023-08-27T11:28:39.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.499+0000 7fa41ed9d700 1 -- 172.21.15.112:0/3212165293 reap_dead start 2023-08-27T11:28:39.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.499+0000 7fa424f58700 1 -- 172.21.15.112:0/3212165293 shutdown_connections 2023-08-27T11:28:39.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.499+0000 7fa424f58700 1 -- 172.21.15.112:0/3212165293 >> 172.21.15.112:0/3212165293 conn(0x7fa4200b1be0 msgr2=0x7fa4200b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:39.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.499+0000 7fa424f58700 1 -- 172.21.15.112:0/3212165293 shutdown_connections 2023-08-27T11:28:39.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.499+0000 7fa424f58700 1 -- 172.21.15.112:0/3212165293 wait complete. 2023-08-27T11:28:39.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.500+0000 7fa424f58700 1 Processor -- start 2023-08-27T11:28:39.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.500+0000 7fa424f58700 1 -- start start 2023-08-27T11:28:39.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.501+0000 7fa424f58700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa408003a30 0x7fa420070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.501+0000 7fa424f58700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4201f2890 0x7fa4200d0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.501+0000 7fa424f58700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4200d0570 0x7fa4200ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.501+0000 7fa424f58700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200ae8c0 con 0x7fa4200d0570 2023-08-27T11:28:39.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.501+0000 7fa424f58700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa420158bd0 con 0x7fa4201f2890 2023-08-27T11:28:39.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.501+0000 7fa424f58700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42013b340 con 0x7fa408003a30 2023-08-27T11:28:39.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.501+0000 7fa41ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4200d0570 0x7fa4200ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.501+0000 7fa41ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4200d0570 0x7fa4200ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36736/0 (socket says 172.21.15.112:36736) 2023-08-27T11:28:39.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.501+0000 7fa41ed9d700 1 -- 172.21.15.112:0/3971362487 learned_addr learned my addr 172.21.15.112:0/3971362487 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.501+0000 7fa41e59c700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa408003a30 0x7fa420070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.501+0000 7fa41dd9b700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4201f2890 0x7fa4200d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.502+0000 7fa41ed9d700 1 -- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa408003a30 msgr2=0x7fa420070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.502+0000 7fa41ed9d700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa408003a30 0x7fa420070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.502+0000 7fa41ed9d700 1 -- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4201f2890 msgr2=0x7fa4200d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.502+0000 7fa41ed9d700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4201f2890 0x7fa4200d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.502+0000 7fa41ed9d700 1 -- 172.21.15.112:0/3971362487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4200adc40 con 0x7fa4200d0570 2023-08-27T11:28:39.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.502+0000 7fa41ed9d700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4200d0570 0x7fa4200ac890 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fa42006e7c0 tx=0x7fa41800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:39.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.502+0000 7fa40effd700 1 -- 172.21.15.112:0/3971362487 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418003c10 con 0x7fa4200d0570 2023-08-27T11:28:39.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.502+0000 7fa40effd700 1 -- 172.21.15.112:0/3971362487 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa418003d90 con 0x7fa4200d0570 2023-08-27T11:28:39.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.502+0000 7fa424f58700 1 -- 172.21.15.112:0/3971362487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4200adf20 con 0x7fa4200d0570 2023-08-27T11:28:39.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.502+0000 7fa40effd700 1 -- 172.21.15.112:0/3971362487 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418005310 con 0x7fa4200d0570 2023-08-27T11:28:39.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.502+0000 7fa424f58700 1 -- 172.21.15.112:0/3971362487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4200ae4e0 con 0x7fa4200d0570 2023-08-27T11:28:39.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.503+0000 7fa424f58700 1 -- 172.21.15.112:0/3971362487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4200ae6a0 con 0x7fa4200d0570 2023-08-27T11:28:39.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.503+0000 7fa40effd700 1 -- 172.21.15.112:0/3971362487 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa418005470 con 0x7fa4200d0570 2023-08-27T11:28:39.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.504+0000 7fa40effd700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa40006a4d0 0x7fa40006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.504+0000 7fa40effd700 1 -- 172.21.15.112:0/3971362487 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5751+0+0 (secure 0 0 0) 0x7fa418088b40 con 0x7fa4200d0570 2023-08-27T11:28:39.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.504+0000 7fa41e59c700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa40006a4d0 0x7fa40006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.504+0000 7fa41e59c700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa40006a4d0 0x7fa40006c990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa414001930 tx=0x7fa414007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:39.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.508+0000 7fa40effd700 1 -- 172.21.15.112:0/3971362487 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa418053e10 con 0x7fa4200d0570 2023-08-27T11:28:39.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:39.696+0000 7fa424f58700 1 -- 172.21.15.112:0/3971362487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000561463.data", "pool2": "cephfs.volume_0000000000561463.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa420067a20 con 0x7fa4200d0570 2023-08-27T11:28:40.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.359+0000 7fa40effd700 1 -- 172.21.15.112:0/3971362487 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000561463.data", "pool2": "cephfs.volume_0000000000561463.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000561463.data' removed v145) v1 ==== 241+0+0 (secure 0 0 0) 0x7fa41801a310 con 0x7fa4200d0570 2023-08-27T11:28:40.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.362+0000 7fa424f58700 1 -- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa40006a4d0 msgr2=0x7fa40006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.362+0000 7fa424f58700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa40006a4d0 0x7fa40006c990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa414001930 tx=0x7fa414007000 comp rx=0 tx=0).stop 2023-08-27T11:28:40.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.362+0000 7fa424f58700 1 -- 172.21.15.112:0/3971362487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4200d0570 msgr2=0x7fa4200ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.362+0000 7fa424f58700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4200d0570 0x7fa4200ac890 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fa42006e7c0 tx=0x7fa41800ee80 comp rx=0 tx=0).stop 2023-08-27T11:28:40.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.362+0000 7fa424f58700 1 -- 172.21.15.112:0/3971362487 shutdown_connections 2023-08-27T11:28:40.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.362+0000 7fa424f58700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa40006a4d0 0x7fa40006c990 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.362+0000 7fa424f58700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4200d0570 0x7fa4200ac890 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.362+0000 7fa424f58700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa408003a30 0x7fa420070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.362+0000 7fa424f58700 1 --2- 172.21.15.112:0/3971362487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4201f2890 0x7fa4200d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.362+0000 7fa424f58700 1 -- 172.21.15.112:0/3971362487 >> 172.21.15.112:0/3971362487 conn(0x7fa4200b1be0 msgr2=0x7fa4200b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:40.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.362+0000 7fa424f58700 1 -- 172.21.15.112:0/3971362487 shutdown_connections 2023-08-27T11:28:40.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.362+0000 7fa424f58700 1 -- 172.21.15.112:0/3971362487 wait complete. 2023-08-27T11:28:40.366 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs.volume_0000000000561463.data' removed 2023-08-27T11:28:40.378 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:28:40.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.502+0000 7fb3fd3d1700 1 Processor -- start 2023-08-27T11:28:40.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.502+0000 7fb3fd3d1700 1 -- start start 2023-08-27T11:28:40.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3fd3d1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f8146c30 0x7fb3f806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3fd3d1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f80b3ee0 con 0x7fb3f806fea0 2023-08-27T11:28:40.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3fd3d1700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f8070500 con 0x7fb3f80b4c10 2023-08-27T11:28:40.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3fd3d1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8070310 con 0x7fb3f8146c30 2023-08-27T11:28:40.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f77fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb3f806fea0 0x7fb3f80713f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59358/0 (socket says 172.21.15.112:59358) 2023-08-27T11:28:40.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f77fe700 1 -- 172.21.15.112:0/3985260117 learned_addr learned my addr 172.21.15.112:0/3985260117 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:40.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f6ffd700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f8146c30 0x7fb3f806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175339993 0 0) 0x7fb3f80b3ee0 con 0x7fb3f806fea0 2023-08-27T11:28:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3e00035b0 con 0x7fb3f806fea0 2023-08-27T11:28:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399978802 0 0) 0x7fb3f8070500 con 0x7fb3f80b4c10 2023-08-27T11:28:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3f80b3ee0 con 0x7fb3f80b4c10 2023-08-27T11:28:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3338155418 0 0) 0x7fb3e00035b0 con 0x7fb3f806fea0 2023-08-27T11:28:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f8146c30 msgr2=0x7fb3f806d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:40.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f8146c30 0x7fb3f806d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 >> v1:172.21.15.138:6789/0 conn(0x7fb3f80b4c10 legacy=0x7fb3f806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f80b4130 con 0x7fb3f806fea0 2023-08-27T11:28:40.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fb3f00019f0 con 0x7fb3f806fea0 2023-08-27T11:28:40.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 >> v1:172.21.15.112:6789/0 conn(0x7fb3f806fea0 legacy=0x7fb3f80713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f8146c30 0x7fb3e0003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e0004120 0x7fb3e0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e0004d90 0x7fb3e0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e000bac0 con 0x7fb3e0004120 2023-08-27T11:28:40.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e000bc30 con 0x7fb3e0004d90 2023-08-27T11:28:40.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.503+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e000bdf0 con 0x7fb3f8146c30 2023-08-27T11:28:40.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f67fc700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e0004120 0x7fb3e0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f6ffd700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f8146c30 0x7fb3e0003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f77fe700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e0004d90 0x7fb3e0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f67fc700 1 -- 172.21.15.112:0/3985260117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f8146c30 msgr2=0x7fb3e0003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:40.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f67fc700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f8146c30 0x7fb3e0003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f67fc700 1 -- 172.21.15.112:0/3985260117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e0004d90 msgr2=0x7fb3e0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:40.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f67fc700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e0004d90 0x7fb3e0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f67fc700 1 -- 172.21.15.112:0/3985260117 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00035b0 con 0x7fb3e0004120 2023-08-27T11:28:40.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f6ffd700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f8146c30 0x7fb3e0003bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:40.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f77fe700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e0004d90 0x7fb3e0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:40.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f67fc700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e0004120 0x7fb3e0004850 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec001940 tx=0x7fb3e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aadd748ab6d004f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:40.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec001430 con 0x7fb3e0004120 2023-08-27T11:28:40.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ec001d70 con 0x7fb3e0004120 2023-08-27T11:28:40.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.504+0000 7fb3f5ffb700 1 -- 172.21.15.112:0/3985260117 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec005510 con 0x7fb3e0004120 2023-08-27T11:28:40.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.505+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/3985260117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e0004120 msgr2=0x7fb3e0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.505+0000 7fb3fd3d1700 1 --2- 172.21.15.112:0/3985260117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e0004120 0x7fb3e0004850 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec001940 tx=0x7fb3e0005880 comp rx=0 tx=0).stop 2023-08-27T11:28:40.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.505+0000 7fb3f77fe700 1 -- 172.21.15.112:0/3985260117 reap_dead start 2023-08-27T11:28:40.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.505+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/3985260117 shutdown_connections 2023-08-27T11:28:40.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.505+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/3985260117 >> 172.21.15.112:0/3985260117 conn(0x7fb3f80cffa0 msgr2=0x7fb3f80d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:40.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.505+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/3985260117 shutdown_connections 2023-08-27T11:28:40.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.505+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/3985260117 wait complete. 2023-08-27T11:28:40.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.506+0000 7fb3fd3d1700 1 Processor -- start 2023-08-27T11:28:40.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.506+0000 7fb3fd3d1700 1 -- start start 2023-08-27T11:28:40.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.507+0000 7fb3fd3d1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e0004120 0x7fb3f81459d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.507+0000 7fb3fd3d1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3f80ae680 0x7fb3f815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.507+0000 7fb3fd3d1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f8160170 0x7fb3f81f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.507+0000 7fb3fd3d1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81f6de0 con 0x7fb3e0004120 2023-08-27T11:28:40.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.507+0000 7fb3fd3d1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81605f0 con 0x7fb3f80ae680 2023-08-27T11:28:40.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.507+0000 7fb3fd3d1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81f6b00 con 0x7fb3f8160170 2023-08-27T11:28:40.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.507+0000 7fb3f6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e0004120 0x7fb3f81459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.507+0000 7fb3f6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e0004120 0x7fb3f81459d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36752/0 (socket says 172.21.15.112:36752) 2023-08-27T11:28:40.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.507+0000 7fb3f6ffd700 1 -- 172.21.15.112:0/1437819652 learned_addr learned my addr 172.21.15.112:0/1437819652 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:40.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.508+0000 7fb3f6ffd700 1 -- 172.21.15.112:0/1437819652 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f8160170 msgr2=0x7fb3f81f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.508+0000 7fb3f6ffd700 1 --2- 172.21.15.112:0/1437819652 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f8160170 0x7fb3f81f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.508+0000 7fb3f6ffd700 1 -- 172.21.15.112:0/1437819652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3f80ae680 msgr2=0x7fb3f815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.508+0000 7fb3f6ffd700 1 --2- 172.21.15.112:0/1437819652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3f80ae680 0x7fb3f815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.508+0000 7fb3f6ffd700 1 -- 172.21.15.112:0/1437819652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f8144ef0 con 0x7fb3e0004120 2023-08-27T11:28:40.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.508+0000 7fb3f6ffd700 1 --2- 172.21.15.112:0/1437819652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e0004120 0x7fb3f81459d0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8011850 tx=0x7fb3e8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:40.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.508+0000 7fb3df7fe700 1 -- 172.21.15.112:0/1437819652 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8004220 con 0x7fb3e0004120 2023-08-27T11:28:40.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.508+0000 7fb3df7fe700 1 -- 172.21.15.112:0/1437819652 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e8004380 con 0x7fb3e0004120 2023-08-27T11:28:40.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.508+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/1437819652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f81451d0 con 0x7fb3e0004120 2023-08-27T11:28:40.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.508+0000 7fb3df7fe700 1 -- 172.21.15.112:0/1437819652 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80036a0 con 0x7fb3e0004120 2023-08-27T11:28:40.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.508+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/1437819652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f80aeb00 con 0x7fb3e0004120 2023-08-27T11:28:40.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.509+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/1437819652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f8145820 con 0x7fb3e0004120 2023-08-27T11:28:40.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.509+0000 7fb3df7fe700 1 -- 172.21.15.112:0/1437819652 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fb3e8003800 con 0x7fb3e0004120 2023-08-27T11:28:40.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.510+0000 7fb3df7fe700 1 --2- 172.21.15.112:0/1437819652 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb3e406a4d0 0x7fb3e406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.510+0000 7fb3df7fe700 1 -- 172.21.15.112:0/1437819652 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb3e8088c30 con 0x7fb3e0004120 2023-08-27T11:28:40.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.510+0000 7fb3f67fc700 1 --2- 172.21.15.112:0/1437819652 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb3e406a4d0 0x7fb3e406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.510+0000 7fb3f67fc700 1 --2- 172.21.15.112:0/1437819652 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb3e406a4d0 0x7fb3e406c990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8160960 tx=0x7fb3ec009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:40.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.514+0000 7fb3df7fe700 1 -- 172.21.15.112:0/1437819652 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb3e80540d0 con 0x7fb3e0004120 2023-08-27T11:28:40.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.701+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/1437819652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb3f8144d50 con 0x7fb3e0004120 2023-08-27T11:28:40.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.705+0000 7fb3df7fe700 1 -- 172.21.15.112:0/1437819652 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb3f8144d50 con 0x7fb3e0004120 2023-08-27T11:28:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.707+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/1437819652 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb3e406a4d0 msgr2=0x7fb3e406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.707+0000 7fb3fd3d1700 1 --2- 172.21.15.112:0/1437819652 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb3e406a4d0 0x7fb3e406c990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8160960 tx=0x7fb3ec009030 comp rx=0 tx=0).stop 2023-08-27T11:28:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.707+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/1437819652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e0004120 msgr2=0x7fb3f81459d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.707+0000 7fb3fd3d1700 1 --2- 172.21.15.112:0/1437819652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e0004120 0x7fb3f81459d0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8011850 tx=0x7fb3e8011c10 comp rx=0 tx=0).stop 2023-08-27T11:28:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.707+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/1437819652 shutdown_connections 2023-08-27T11:28:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.707+0000 7fb3fd3d1700 1 --2- 172.21.15.112:0/1437819652 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb3e406a4d0 0x7fb3e406c990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.707+0000 7fb3fd3d1700 1 --2- 172.21.15.112:0/1437819652 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f8160170 0x7fb3f81f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.707+0000 7fb3fd3d1700 1 --2- 172.21.15.112:0/1437819652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e0004120 0x7fb3f81459d0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.707+0000 7fb3fd3d1700 1 --2- 172.21.15.112:0/1437819652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3f80ae680 0x7fb3f815fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.707+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/1437819652 >> 172.21.15.112:0/1437819652 conn(0x7fb3f80cffa0 msgr2=0x7fb3f80cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.707+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/1437819652 shutdown_connections 2023-08-27T11:28:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.707+0000 7fb3fd3d1700 1 -- 172.21.15.112:0/1437819652 wait complete. 2023-08-27T11:28:40.721 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename' 2023-08-27T11:28:40.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.843+0000 7f42293e0700 1 Processor -- start 2023-08-27T11:28:40.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.843+0000 7f42293e0700 1 -- start start 2023-08-27T11:28:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.844+0000 7f42293e0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f422406dc20 0x7f42240b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.844+0000 7f42293e0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422406f930 con 0x7f4224146990 2023-08-27T11:28:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.844+0000 7f42293e0700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422406fc10 con 0x7f4224160a00 2023-08-27T11:28:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.844+0000 7f42293e0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422406fef0 con 0x7f422406dc20 2023-08-27T11:28:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.844+0000 7f42227fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4224146990 0x7f422406d510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59376/0 (socket says 172.21.15.112:59376) 2023-08-27T11:28:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.844+0000 7f42227fc700 1 -- 172.21.15.112:0/1404051300 learned_addr learned my addr 172.21.15.112:0/1404051300 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.844+0000 7f42237fe700 1 --2- 172.21.15.112:0/1404051300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f422406dc20 0x7f42240b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.844+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3829170631 0 0) 0x7f422406f930 con 0x7f4224146990 2023-08-27T11:28:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.844+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f420c0035b0 con 0x7f4224146990 2023-08-27T11:28:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.844+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953460666 0 0) 0x7f422406fc10 con 0x7f4224160a00 2023-08-27T11:28:40.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.844+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f422406f930 con 0x7f4224160a00 2023-08-27T11:28:40.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1541979094 0 0) 0x7f420c0035b0 con 0x7f4224146990 2023-08-27T11:28:40.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f422406dc20 msgr2=0x7f42240b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:40.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 --2- 172.21.15.112:0/1404051300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f422406dc20 0x7f42240b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 >> v1:172.21.15.138:6789/0 conn(0x7f4224160a00 legacy=0x7f422415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42240701d0 con 0x7f4224146990 2023-08-27T11:28:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f4214001670 con 0x7f4224146990 2023-08-27T11:28:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 >> v1:172.21.15.112:6789/0 conn(0x7f4224146990 legacy=0x7f422406d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 --2- 172.21.15.112:0/1404051300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f422406dc20 0x7f420c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 --2- 172.21.15.112:0/1404051300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c004120 0x7f420c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 --2- 172.21.15.112:0/1404051300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420c004d90 0x7f420c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422406fef0 con 0x7f420c004120 2023-08-27T11:28:40.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c00baf0 con 0x7f420c004d90 2023-08-27T11:28:40.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c00bcb0 con 0x7f422406dc20 2023-08-27T11:28:40.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f42227fc700 1 --2- 172.21.15.112:0/1404051300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c004120 0x7f420c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f42237fe700 1 --2- 172.21.15.112:0/1404051300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420c004d90 0x7f420c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4222ffd700 1 --2- 172.21.15.112:0/1404051300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f422406dc20 0x7f420c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f42227fc700 1 -- 172.21.15.112:0/1404051300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f422406dc20 msgr2=0x7f420c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:40.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f42227fc700 1 --2- 172.21.15.112:0/1404051300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f422406dc20 0x7f420c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f42227fc700 1 -- 172.21.15.112:0/1404051300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420c004d90 msgr2=0x7f420c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:40.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f42227fc700 1 --2- 172.21.15.112:0/1404051300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420c004d90 0x7f420c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f42227fc700 1 -- 172.21.15.112:0/1404051300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c0035b0 con 0x7f420c004120 2023-08-27T11:28:40.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4222ffd700 1 --2- 172.21.15.112:0/1404051300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f422406dc20 0x7f420c003bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:40.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f42227fc700 1 --2- 172.21.15.112:0/1404051300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c004120 0x7f420c004850 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f421400c210 tx=0x7f420c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335850da88e7e6a0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:40.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421400cdb0 con 0x7f420c004120 2023-08-27T11:28:40.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4214001af0 con 0x7f420c004120 2023-08-27T11:28:40.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.845+0000 7f4221ffb700 1 -- 172.21.15.112:0/1404051300 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4214005ba0 con 0x7f420c004120 2023-08-27T11:28:40.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.846+0000 7f42293e0700 1 -- 172.21.15.112:0/1404051300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c004120 msgr2=0x7f420c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.846+0000 7f42293e0700 1 --2- 172.21.15.112:0/1404051300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c004120 0x7f420c004850 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f421400c210 tx=0x7f420c005880 comp rx=0 tx=0).stop 2023-08-27T11:28:40.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.846+0000 7f42237fe700 1 -- 172.21.15.112:0/1404051300 reap_dead start 2023-08-27T11:28:40.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.846+0000 7f42293e0700 1 -- 172.21.15.112:0/1404051300 shutdown_connections 2023-08-27T11:28:40.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.846+0000 7f42293e0700 1 -- 172.21.15.112:0/1404051300 >> 172.21.15.112:0/1404051300 conn(0x7f42240cfe50 msgr2=0x7f42240d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:40.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.846+0000 7f42293e0700 1 -- 172.21.15.112:0/1404051300 shutdown_connections 2023-08-27T11:28:40.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.846+0000 7f42293e0700 1 -- 172.21.15.112:0/1404051300 wait complete. 2023-08-27T11:28:40.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.847+0000 7f42293e0700 1 Processor -- start 2023-08-27T11:28:40.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.847+0000 7f42293e0700 1 -- start start 2023-08-27T11:28:40.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f42293e0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c004120 0x7f42241f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f42293e0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42241f5b10 0x7f42241f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f42293e0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42241f31e0 0x7f42240b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f42293e0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42241f42e0 con 0x7f420c004120 2023-08-27T11:28:40.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f42293e0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42240b88f0 con 0x7f42241f31e0 2023-08-27T11:28:40.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f42293e0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42241f3660 con 0x7f42241f5b10 2023-08-27T11:28:40.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f4222ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c004120 0x7f42241f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f4222ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c004120 0x7f42241f3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36776/0 (socket says 172.21.15.112:36776) 2023-08-27T11:28:40.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f4222ffd700 1 -- 172.21.15.112:0/3490988080 learned_addr learned my addr 172.21.15.112:0/3490988080 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:40.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f42227fc700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42241f5b10 0x7f42241f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f42237fe700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42241f31e0 0x7f42240b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f4222ffd700 1 -- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42241f5b10 msgr2=0x7f42241f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f4222ffd700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42241f5b10 0x7f42241f2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f4222ffd700 1 -- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42241f31e0 msgr2=0x7f42240b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f4222ffd700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42241f31e0 0x7f42240b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f4222ffd700 1 -- 172.21.15.112:0/3490988080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42241413a0 con 0x7f420c004120 2023-08-27T11:28:40.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f42237fe700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42241f31e0 0x7f42240b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:40.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.848+0000 7f4222ffd700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c004120 0x7f42241f3da0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f421800e780 tx=0x7f421800eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:40.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.849+0000 7f420b7fe700 1 -- 172.21.15.112:0/3490988080 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4218017070 con 0x7f420c004120 2023-08-27T11:28:40.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.849+0000 7f420b7fe700 1 -- 172.21.15.112:0/3490988080 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42180052e0 con 0x7f420c004120 2023-08-27T11:28:40.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.849+0000 7f42293e0700 1 -- 172.21.15.112:0/3490988080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4224141680 con 0x7f420c004120 2023-08-27T11:28:40.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.849+0000 7f420b7fe700 1 -- 172.21.15.112:0/3490988080 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4218005450 con 0x7f420c004120 2023-08-27T11:28:40.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.849+0000 7f42293e0700 1 -- 172.21.15.112:0/3490988080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4224141c40 con 0x7f420c004120 2023-08-27T11:28:40.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.850+0000 7f42293e0700 1 -- 172.21.15.112:0/3490988080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4224067a20 con 0x7f420c004120 2023-08-27T11:28:40.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.850+0000 7f420b7fe700 1 -- 172.21.15.112:0/3490988080 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4218006860 con 0x7f420c004120 2023-08-27T11:28:40.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.850+0000 7f420b7fe700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f421006a4d0 0x7f421006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.850+0000 7f420b7fe700 1 -- 172.21.15.112:0/3490988080 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f42180880b0 con 0x7f420c004120 2023-08-27T11:28:40.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.851+0000 7f42227fc700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f421006a4d0 0x7f421006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.851+0000 7f42227fc700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f421006a4d0 0x7f421006c990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4214000c00 tx=0x7f421400a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:40.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:40.855+0000 7f420b7fe700 1 -- 172.21.15.112:0/3490988080 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f42180534d0 con 0x7f420c004120 2023-08-27T11:28:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.121+0000 7f42293e0700 1 -- 172.21.15.112:0/3490988080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]} v 0) v1 -- 0x7f4224070480 con 0x7f420c004120 2023-08-27T11:28:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.368+0000 7f420b7fe700 1 -- 172.21.15.112:0/3490988080 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]}]=0 v530) v1 ==== 135+0+0 (secure 0 0 0) 0x7f4218075070 con 0x7f420c004120 2023-08-27T11:28:41.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.370+0000 7f42293e0700 1 -- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f421006a4d0 msgr2=0x7f421006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:41.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.370+0000 7f42293e0700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f421006a4d0 0x7f421006c990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4214000c00 tx=0x7f421400a000 comp rx=0 tx=0).stop 2023-08-27T11:28:41.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.370+0000 7f42293e0700 1 -- 172.21.15.112:0/3490988080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c004120 msgr2=0x7f42241f3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:41.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.370+0000 7f42293e0700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c004120 0x7f42241f3da0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f421800e780 tx=0x7f421800eb40 comp rx=0 tx=0).stop 2023-08-27T11:28:41.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.370+0000 7f42293e0700 1 -- 172.21.15.112:0/3490988080 shutdown_connections 2023-08-27T11:28:41.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.370+0000 7f42293e0700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f421006a4d0 0x7f421006c990 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.370+0000 7f42293e0700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42241f31e0 0x7f42240b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.370+0000 7f42293e0700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c004120 0x7f42241f3da0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.370+0000 7f42293e0700 1 --2- 172.21.15.112:0/3490988080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42241f5b10 0x7f42241f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.370+0000 7f42293e0700 1 -- 172.21.15.112:0/3490988080 >> 172.21.15.112:0/3490988080 conn(0x7f42240cfe50 msgr2=0x7f422415f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:41.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.370+0000 7f42293e0700 1 -- 172.21.15.112:0/3490988080 shutdown_connections 2023-08-27T11:28:41.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.370+0000 7f42293e0700 1 -- 172.21.15.112:0/3490988080 wait complete. 2023-08-27T11:28:41.384 INFO:tasks.cephfs_test_runner:That volume, its file system and pools, can be renamed. ... ok 2023-08-27T11:28:41.384 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_fails_without_confirmation_flag (tasks.cephfs.test_volumes.TestVolumes) That renaming volume fails without --yes-i-really-mean-it flag. 2023-08-27T11:28:41.385 INFO:tasks.cephfs_test_runner:test_volume_rename_fails_without_confirmation_flag (tasks.cephfs.test_volumes.TestVolumes) 2023-08-27T11:28:41.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag' 2023-08-27T11:28:41.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.503+0000 7f7171e6b700 1 Processor -- start 2023-08-27T11:28:41.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.503+0000 7f7171e6b700 1 -- start start 2023-08-27T11:28:41.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.504+0000 7f7171e6b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f716c161600 0x7f716c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.504+0000 7f7171e6b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f716c0d0c60 0x7f716c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.504+0000 7f7171e6b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716c15f110 con 0x7f716c157880 2023-08-27T11:28:41.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.504+0000 7f7171e6b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716c0d1490 con 0x7f716c161600 2023-08-27T11:28:41.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.504+0000 7f7171e6b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0d1770 con 0x7f716c0d0c60 2023-08-27T11:28:41.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.504+0000 7f716b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f716c157880 0x7f716c160ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59394/0 (socket says 172.21.15.112:59394) 2023-08-27T11:28:41.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.504+0000 7f716b7fe700 1 -- 172.21.15.112:0/1882854019 learned_addr learned my addr 172.21.15.112:0/1882854019 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:41.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883130630 0 0) 0x7f716c15f110 con 0x7f716c157880 2023-08-27T11:28:41.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71540035b0 con 0x7f716c157880 2023-08-27T11:28:41.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716bfff700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f716c0d0c60 0x7f716c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716affd700 1 --2- 172.21.15.112:0/1882854019 >> v2:172.21.15.138:3300/0 conn(0x7f716c161600 0x7f716c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2692211112 0 0) 0x7f71540035b0 con 0x7f716c157880 2023-08-27T11:28:41.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f716c0d0c60 msgr2=0x7f716c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:41.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f716c0d0c60 0x7f716c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 >> v2:172.21.15.138:3300/0 conn(0x7f716c161600 msgr2=0x7f716c0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:41.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 --2- 172.21.15.112:0/1882854019 >> v2:172.21.15.138:3300/0 conn(0x7f716c161600 0x7f716c0d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c15eac0 con 0x7f716c157880 2023-08-27T11:28:41.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f7160001670 con 0x7f716c157880 2023-08-27T11:28:41.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 >> v1:172.21.15.112:6789/0 conn(0x7f716c157880 legacy=0x7f716c160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:41.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7154003a00 0x7f7154003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7154004850 0x7f7154004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f716c0d0c60 0x7f7154005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0d1490 con 0x7f7154003a00 2023-08-27T11:28:41.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0d1770 con 0x7f7154004850 2023-08-27T11:28:41.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715400ba10 con 0x7f716c0d0c60 2023-08-27T11:28:41.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716b7fe700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7154003a00 0x7f7154003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716affd700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7154004850 0x7f7154004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716bfff700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f716c0d0c60 0x7f7154005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716b7fe700 1 -- 172.21.15.112:0/1882854019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f716c0d0c60 msgr2=0x7f7154005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:41.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716b7fe700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f716c0d0c60 0x7f7154005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716b7fe700 1 -- 172.21.15.112:0/1882854019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7154004850 msgr2=0x7f7154004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:41.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716b7fe700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7154004850 0x7f7154004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716b7fe700 1 -- 172.21.15.112:0/1882854019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71540035b0 con 0x7f7154003a00 2023-08-27T11:28:41.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716bfff700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f716c0d0c60 0x7f7154005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:41.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716affd700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7154004850 0x7f7154004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:41.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.505+0000 7f716b7fe700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7154003a00 0x7f7154003e20 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f716000c370 tx=0x7f7154005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e94d83801c6765 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:41.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.506+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160007c60 con 0x7f7154003a00 2023-08-27T11:28:41.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.506+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7160005960 con 0x7f7154003a00 2023-08-27T11:28:41.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.506+0000 7f716a7fc700 1 -- 172.21.15.112:0/1882854019 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160005d10 con 0x7f7154003a00 2023-08-27T11:28:41.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.506+0000 7f7171e6b700 1 -- 172.21.15.112:0/1882854019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7154003a00 msgr2=0x7f7154003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:41.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.506+0000 7f7171e6b700 1 --2- 172.21.15.112:0/1882854019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7154003a00 0x7f7154003e20 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f716000c370 tx=0x7f7154005880 comp rx=0 tx=0).stop 2023-08-27T11:28:41.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.506+0000 7f716bfff700 1 -- 172.21.15.112:0/1882854019 reap_dead start 2023-08-27T11:28:41.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.506+0000 7f7171e6b700 1 -- 172.21.15.112:0/1882854019 shutdown_connections 2023-08-27T11:28:41.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.506+0000 7f7171e6b700 1 -- 172.21.15.112:0/1882854019 >> 172.21.15.112:0/1882854019 conn(0x7f716c06c880 msgr2=0x7f716c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:41.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.506+0000 7f7171e6b700 1 -- 172.21.15.112:0/1882854019 shutdown_connections 2023-08-27T11:28:41.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.506+0000 7f7171e6b700 1 -- 172.21.15.112:0/1882854019 wait complete. 2023-08-27T11:28:41.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.507+0000 7f7171e6b700 1 Processor -- start 2023-08-27T11:28:41.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.508+0000 7f7171e6b700 1 -- start start 2023-08-27T11:28:41.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.508+0000 7f7171e6b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7154003a00 0x7f716c1f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.508+0000 7f7171e6b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716c1525c0 0x7f716c152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.508+0000 7f7171e6b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f716c1f2da0 0x7f716c1f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.508+0000 7f7171e6b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c152f80 con 0x7f716c1f2da0 2023-08-27T11:28:41.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.508+0000 7f7171e6b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c1f28a0 con 0x7f716c1525c0 2023-08-27T11:28:41.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.508+0000 7f7171e6b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c140650 con 0x7f7154003a00 2023-08-27T11:28:41.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f716bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f716c1f2da0 0x7f716c1f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f716bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f716c1f2da0 0x7f716c1f3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36796/0 (socket says 172.21.15.112:36796) 2023-08-27T11:28:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f716bfff700 1 -- 172.21.15.112:0/2053910387 learned_addr learned my addr 172.21.15.112:0/2053910387 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f716affd700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716c1525c0 0x7f716c152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f716b7fe700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7154003a00 0x7f716c1f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f716bfff700 1 -- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7154003a00 msgr2=0x7f716c1f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f716bfff700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7154003a00 0x7f716c1f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f716bfff700 1 -- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716c1525c0 msgr2=0x7f716c152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:41.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f716bfff700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716c1525c0 0x7f716c152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f716bfff700 1 -- 172.21.15.112:0/2053910387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c1428f0 con 0x7f716c1f2da0 2023-08-27T11:28:41.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f716b7fe700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7154003a00 0x7f716c1f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:41.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f716bfff700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f716c1f2da0 0x7f716c1f3230 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f716c0b2cb0 tx=0x7f716400f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:41.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f7153fff700 1 -- 172.21.15.112:0/2053910387 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716400fd60 con 0x7f716c1f2da0 2023-08-27T11:28:41.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f7153fff700 1 -- 172.21.15.112:0/2053910387 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7164011040 con 0x7f716c1f2da0 2023-08-27T11:28:41.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f7171e6b700 1 -- 172.21.15.112:0/2053910387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c142bd0 con 0x7f716c1f2da0 2023-08-27T11:28:41.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f7153fff700 1 -- 172.21.15.112:0/2053910387 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164017620 con 0x7f716c1f2da0 2023-08-27T11:28:41.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.509+0000 7f7171e6b700 1 -- 172.21.15.112:0/2053910387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c143190 con 0x7f716c1f2da0 2023-08-27T11:28:41.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.510+0000 7f7171e6b700 1 -- 172.21.15.112:0/2053910387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c067a20 con 0x7f716c1f2da0 2023-08-27T11:28:41.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.511+0000 7f7153fff700 1 -- 172.21.15.112:0/2053910387 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f71640203b0 con 0x7f716c1f2da0 2023-08-27T11:28:41.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.511+0000 7f7153fff700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f715806a4d0 0x7f715806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.511+0000 7f7153fff700 1 -- 172.21.15.112:0/2053910387 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7164088b60 con 0x7f716c1f2da0 2023-08-27T11:28:41.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.511+0000 7f716b7fe700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f715806a4d0 0x7f715806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.512+0000 7f716b7fe700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f715806a4d0 0x7f715806c990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7160015040 tx=0x7f716000a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:41.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.516+0000 7f7153fff700 1 -- 172.21.15.112:0/2053910387 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7164053ef0 con 0x7f716c1f2da0 2023-08-27T11:28:41.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:41.787+0000 7f7171e6b700 1 -- 172.21.15.112:0/2053910387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]} v 0) v1 -- 0x7f716c1609e0 con 0x7f716c1f2da0 2023-08-27T11:28:42.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.370+0000 7f7153fff700 1 -- 172.21.15.112:0/2053910387 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]}]=0 v531) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7164057510 con 0x7f716c1f2da0 2023-08-27T11:28:42.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.373+0000 7f7171e6b700 1 -- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f715806a4d0 msgr2=0x7f715806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.373+0000 7f7171e6b700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f715806a4d0 0x7f715806c990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7160015040 tx=0x7f716000a000 comp rx=0 tx=0).stop 2023-08-27T11:28:42.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.373+0000 7f7171e6b700 1 -- 172.21.15.112:0/2053910387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f716c1f2da0 msgr2=0x7f716c1f3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.373+0000 7f7171e6b700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f716c1f2da0 0x7f716c1f3230 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f716c0b2cb0 tx=0x7f716400f3b0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.373+0000 7f7171e6b700 1 -- 172.21.15.112:0/2053910387 shutdown_connections 2023-08-27T11:28:42.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.373+0000 7f7171e6b700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f715806a4d0 0x7f715806c990 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.373+0000 7f7171e6b700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f716c1f2da0 0x7f716c1f3230 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.373+0000 7f7171e6b700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7154003a00 0x7f716c1f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.373+0000 7f7171e6b700 1 --2- 172.21.15.112:0/2053910387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716c1525c0 0x7f716c152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.373+0000 7f7171e6b700 1 -- 172.21.15.112:0/2053910387 >> 172.21.15.112:0/2053910387 conn(0x7f716c06c880 msgr2=0x7f716c0d1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.373+0000 7f7171e6b700 1 -- 172.21.15.112:0/2053910387 shutdown_connections 2023-08-27T11:28:42.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.373+0000 7f7171e6b700 1 -- 172.21.15.112:0/2053910387 wait complete. 2023-08-27T11:28:42.387 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:42.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.507+0000 7f1aa4855700 1 Processor -- start 2023-08-27T11:28:42.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.507+0000 7f1aa4855700 1 -- start start 2023-08-27T11:28:42.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.508+0000 7f1aa4855700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1aa00b1400 0x7f1aa00cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.508+0000 7f1aa4855700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aa00b17e0 0x7f1aa006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.508+0000 7f1aa4855700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa0161850 con 0x7f1aa0161ce0 2023-08-27T11:28:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.508+0000 7f1aa4855700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa006e7d0 con 0x7f1aa00b1400 2023-08-27T11:28:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.508+0000 7f1aa4855700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00cfc10 con 0x7f1aa00b17e0 2023-08-27T11:28:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9e59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1aa00b1400 0x7f1aa00cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9e59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1aa00b1400 0x7f1aa00cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36800/0 (socket says 172.21.15.112:36800) 2023-08-27T11:28:42.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9ed9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1aa0161ce0 0x7f1aa006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59416/0 (socket says 172.21.15.112:59416) 2023-08-27T11:28:42.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9e59c700 1 -- 172.21.15.112:0/2766494389 learned_addr learned my addr 172.21.15.112:0/2766494389 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:42.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9dd9b700 1 --2- 172.21.15.112:0/2766494389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aa00b17e0 0x7f1aa006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9e59c700 1 -- 172.21.15.112:0/2766494389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aa00b17e0 msgr2=0x7f1aa006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:42.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9e59c700 1 --2- 172.21.15.112:0/2766494389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aa00b17e0 0x7f1aa006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9e59c700 1 -- 172.21.15.112:0/2766494389 >> v1:172.21.15.112:6789/0 conn(0x7f1aa0161ce0 legacy=0x7f1aa006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9e59c700 1 -- 172.21.15.112:0/2766494389 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa0161200 con 0x7f1aa00b1400 2023-08-27T11:28:42.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9dd9b700 1 --2- 172.21.15.112:0/2766494389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aa00b17e0 0x7f1aa006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:42.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9e59c700 1 --2- 172.21.15.112:0/2766494389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa00b1400 0x7f1aa00cf5a0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f1a940098f0 tx=0x7f1a94009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12c0440bce940b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9d59a700 1 -- 172.21.15.112:0/2766494389 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400b8a0 con 0x7f1aa00b1400 2023-08-27T11:28:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9d59a700 1 -- 172.21.15.112:0/2766494389 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9400ba00 con 0x7f1aa00b1400 2023-08-27T11:28:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.509+0000 7f1a9d59a700 1 -- 172.21.15.112:0/2766494389 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400bd90 con 0x7f1aa00b1400 2023-08-27T11:28:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.510+0000 7f1aa4855700 1 -- 172.21.15.112:0/2766494389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa00b1400 msgr2=0x7f1aa00cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.510+0000 7f1aa4855700 1 --2- 172.21.15.112:0/2766494389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa00b1400 0x7f1aa00cf5a0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f1a940098f0 tx=0x7f1a94009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.510+0000 7f1aa4855700 1 -- 172.21.15.112:0/2766494389 shutdown_connections 2023-08-27T11:28:42.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.510+0000 7f1aa4855700 1 --2- 172.21.15.112:0/2766494389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa00b1400 0x7f1aa00cf5a0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.510+0000 7f1aa4855700 1 --2- 172.21.15.112:0/2766494389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aa00b17e0 0x7f1aa006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.510+0000 7f1aa4855700 1 -- 172.21.15.112:0/2766494389 >> 172.21.15.112:0/2766494389 conn(0x7f1aa00d0000 msgr2=0x7f1aa00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:42.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.510+0000 7f1aa4855700 1 -- 172.21.15.112:0/2766494389 shutdown_connections 2023-08-27T11:28:42.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.510+0000 7f1aa4855700 1 -- 172.21.15.112:0/2766494389 wait complete. 2023-08-27T11:28:42.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.512+0000 7f1aa4855700 1 Processor -- start 2023-08-27T11:28:42.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.512+0000 7f1aa4855700 1 -- start start 2023-08-27T11:28:42.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.512+0000 7f1aa4855700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aa00b1400 0x7f1aa00c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.512+0000 7f1aa4855700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1aa00b17e0 0x7f1aa014cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.512+0000 7f1aa4855700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa0161ce0 0x7f1aa00c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1aa4855700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa014d080 con 0x7f1aa0161ce0 2023-08-27T11:28:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1aa4855700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00c79b0 con 0x7f1aa00b17e0 2023-08-27T11:28:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1aa4855700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00c7c90 con 0x7f1aa00b1400 2023-08-27T11:28:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a9ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa0161ce0 0x7f1aa00c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a9ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa0161ce0 0x7f1aa00c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36806/0 (socket says 172.21.15.112:36806) 2023-08-27T11:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a9ed9d700 1 -- 172.21.15.112:0/3521628250 learned_addr learned my addr 172.21.15.112:0/3521628250 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a9e59c700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aa00b1400 0x7f1aa00c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a9dd9b700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1aa00b17e0 0x7f1aa014cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a9ed9d700 1 -- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aa00b1400 msgr2=0x7f1aa00c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a9ed9d700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aa00b1400 0x7f1aa00c8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a9ed9d700 1 -- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1aa00b17e0 msgr2=0x7f1aa014cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a9ed9d700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1aa00b17e0 0x7f1aa014cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a9ed9d700 1 -- 172.21.15.112:0/3521628250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a94009550 con 0x7f1aa0161ce0 2023-08-27T11:28:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a9ed9d700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa0161ce0 0x7f1aa00c70d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800cbc0 tx=0x7f1a9800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a8effd700 1 -- 172.21.15.112:0/3521628250 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9800ab50 con 0x7f1aa0161ce0 2023-08-27T11:28:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a8effd700 1 -- 172.21.15.112:0/3521628250 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9800acb0 con 0x7f1aa0161ce0 2023-08-27T11:28:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1aa4855700 1 -- 172.21.15.112:0/3521628250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa00c7f70 con 0x7f1aa0161ce0 2023-08-27T11:28:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1a8effd700 1 -- 172.21.15.112:0/3521628250 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a98007270 con 0x7f1aa0161ce0 2023-08-27T11:28:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.513+0000 7f1aa4855700 1 -- 172.21.15.112:0/3521628250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa00c8530 con 0x7f1aa0161ce0 2023-08-27T11:28:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.514+0000 7f1aa4855700 1 -- 172.21.15.112:0/3521628250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa00c8750 con 0x7f1aa0161ce0 2023-08-27T11:28:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.515+0000 7f1a8effd700 1 -- 172.21.15.112:0/3521628250 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f1a980073d0 con 0x7f1aa0161ce0 2023-08-27T11:28:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.515+0000 7f1a8effd700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1a8006a4d0 0x7f1a8006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.515+0000 7f1a8effd700 1 -- 172.21.15.112:0/3521628250 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1a980884c0 con 0x7f1aa0161ce0 2023-08-27T11:28:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.515+0000 7f1a9e59c700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1a8006a4d0 0x7f1a8006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.516+0000 7f1a9e59c700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1a8006a4d0 0x7f1a8006c990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009c40 tx=0x7f1a94008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.519+0000 7f1a8effd700 1 -- 172.21.15.112:0/3521628250 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1a98053b10 con 0x7f1aa0161ce0 2023-08-27T11:28:42.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.709+0000 7f1aa4855700 1 -- 172.21.15.112:0/3521628250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1aa0067a20 con 0x7f1aa0161ce0 2023-08-27T11:28:42.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.710+0000 7f1a8effd700 1 -- 172.21.15.112:0/3521628250 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+14744 (secure 0 0 0) 0x7f1a98057130 con 0x7f1aa0161ce0 2023-08-27T11:28:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":145,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:40.356379+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6836/529571603":"2023-08-28T11:28:37.289838+0000","172.21.15.138:6834/529571603":"2023-08-28T11:28:37.289838+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:42.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.712+0000 7f1aa4855700 1 -- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1a8006a4d0 msgr2=0x7f1a8006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.712+0000 7f1aa4855700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1a8006a4d0 0x7f1a8006c990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009c40 tx=0x7f1a94008030 comp rx=0 tx=0).stop 2023-08-27T11:28:42.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.712+0000 7f1aa4855700 1 -- 172.21.15.112:0/3521628250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa0161ce0 msgr2=0x7f1aa00c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.712+0000 7f1aa4855700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa0161ce0 0x7f1aa00c70d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800cbc0 tx=0x7f1a9800cf80 comp rx=0 tx=0).stop 2023-08-27T11:28:42.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.712+0000 7f1aa4855700 1 -- 172.21.15.112:0/3521628250 shutdown_connections 2023-08-27T11:28:42.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.712+0000 7f1aa4855700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1a8006a4d0 0x7f1a8006c990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.712+0000 7f1aa4855700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa0161ce0 0x7f1aa00c70d0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.712+0000 7f1aa4855700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aa00b1400 0x7f1aa00c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.712+0000 7f1aa4855700 1 --2- 172.21.15.112:0/3521628250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1aa00b17e0 0x7f1aa014cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.712+0000 7f1aa4855700 1 -- 172.21.15.112:0/3521628250 >> 172.21.15.112:0/3521628250 conn(0x7f1aa00d0000 msgr2=0x7f1aa015eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:42.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.712+0000 7f1aa4855700 1 -- 172.21.15.112:0/3521628250 shutdown_connections 2023-08-27T11:28:42.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.712+0000 7f1aa4855700 1 -- 172.21.15.112:0/3521628250 wait complete. 2023-08-27T11:28:42.726 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:28:42.727 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:28:42.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.848+0000 7f0488503700 1 Processor -- start 2023-08-27T11:28:42.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.848+0000 7f0488503700 1 -- start start 2023-08-27T11:28:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.849+0000 7f0488503700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f048006f790 0x7f048006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.849+0000 7f0488503700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048015fd60 con 0x7f0480144900 2023-08-27T11:28:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.849+0000 7f0488503700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048015ee90 con 0x7f04800cca90 2023-08-27T11:28:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.849+0000 7f0488503700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0480070140 con 0x7f048006f790 2023-08-27T11:28:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.849+0000 7f048629f700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0480144900 0x7f04800b09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59426/0 (socket says 172.21.15.112:59426) 2023-08-27T11:28:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.849+0000 7f048629f700 1 -- 172.21.15.112:0/2482971696 learned_addr learned my addr 172.21.15.112:0/2482971696 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.849+0000 7f0486aa0700 1 --2- 172.21.15.112:0/2482971696 >> v2:172.21.15.138:3300/0 conn(0x7f048006f790 0x7f048006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.849+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271214371 0 0) 0x7f048015fd60 con 0x7f0480144900 2023-08-27T11:28:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.849+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f046c0035b0 con 0x7f0480144900 2023-08-27T11:28:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432916586 0 0) 0x7f048015ee90 con 0x7f04800cca90 2023-08-27T11:28:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f048015fd60 con 0x7f04800cca90 2023-08-27T11:28:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3676321448 0 0) 0x7f046c0035b0 con 0x7f0480144900 2023-08-27T11:28:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 >> v2:172.21.15.138:3300/0 conn(0x7f048006f790 msgr2=0x7f048006fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 --2- 172.21.15.112:0/2482971696 >> v2:172.21.15.138:3300/0 conn(0x7f048006f790 0x7f048006fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 >> v1:172.21.15.138:6789/0 conn(0x7f04800cca90 legacy=0x7f048015f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04800b11d0 con 0x7f0480144900 2023-08-27T11:28:42.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f0470001670 con 0x7f0480144900 2023-08-27T11:28:42.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f0486aa0700 1 --2- 172.21.15.112:0/2482971696 >> v2:172.21.15.138:3300/0 conn(0x7f048006f790 0x7f048006fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:42.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 >> v1:172.21.15.112:6789/0 conn(0x7f0480144900 legacy=0x7f04800b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 --2- 172.21.15.112:0/2482971696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046c003b90 0x7f046c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 --2- 172.21.15.112:0/2482971696 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f046c004850 0x7f046c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 --2- 172.21.15.112:0/2482971696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046c005210 0x7f046c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0480070140 con 0x7f046c005210 2023-08-27T11:28:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c00c090 con 0x7f046c003b90 2023-08-27T11:28:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046c00c250 con 0x7f046c004850 2023-08-27T11:28:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f0486aa0700 1 --2- 172.21.15.112:0/2482971696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046c005210 0x7f046c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048629f700 1 --2- 172.21.15.112:0/2482971696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046c003b90 0x7f046c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f0485a9e700 1 --2- 172.21.15.112:0/2482971696 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f046c004850 0x7f046c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f0486aa0700 1 -- 172.21.15.112:0/2482971696 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f046c004850 msgr2=0x7f046c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:42.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f0486aa0700 1 --2- 172.21.15.112:0/2482971696 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f046c004850 0x7f046c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f0486aa0700 1 -- 172.21.15.112:0/2482971696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046c003b90 msgr2=0x7f046c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:42.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f0486aa0700 1 --2- 172.21.15.112:0/2482971696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046c003b90 0x7f046c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f0486aa0700 1 -- 172.21.15.112:0/2482971696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c0035b0 con 0x7f046c005210 2023-08-27T11:28:42.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f0486aa0700 1 -- 172.21.15.112:0/2482971696 reap_dead start 2023-08-27T11:28:42.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f0486aa0700 1 --2- 172.21.15.112:0/2482971696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046c005210 0x7f046c00ba10 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f047800e350 tx=0x7f046c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63133b997b44c51 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047800edf0 con 0x7f046c005210 2023-08-27T11:28:42.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047800f040 con 0x7f046c005210 2023-08-27T11:28:42.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.850+0000 7f048529d700 1 -- 172.21.15.112:0/2482971696 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04780146c0 con 0x7f046c005210 2023-08-27T11:28:42.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.851+0000 7f0488503700 1 -- 172.21.15.112:0/2482971696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046c005210 msgr2=0x7f046c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.851+0000 7f0488503700 1 --2- 172.21.15.112:0/2482971696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046c005210 0x7f046c00ba10 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f047800e350 tx=0x7f046c005880 comp rx=0 tx=0).stop 2023-08-27T11:28:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.851+0000 7f0488503700 1 -- 172.21.15.112:0/2482971696 shutdown_connections 2023-08-27T11:28:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.851+0000 7f0488503700 1 --2- 172.21.15.112:0/2482971696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046c005210 0x7f046c00ba10 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.851+0000 7f0488503700 1 -- 172.21.15.112:0/2482971696 >> 172.21.15.112:0/2482971696 conn(0x7f04800b1e90 msgr2=0x7f04800b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.851+0000 7f0488503700 1 -- 172.21.15.112:0/2482971696 shutdown_connections 2023-08-27T11:28:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.851+0000 7f0488503700 1 -- 172.21.15.112:0/2482971696 wait complete. 2023-08-27T11:28:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.852+0000 7f0488503700 1 Processor -- start 2023-08-27T11:28:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.852+0000 7f0488503700 1 -- start start 2023-08-27T11:28:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f0488503700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046c005210 0x7f048013e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f0488503700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f048015e1e0 0x7f0480071a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f0488503700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0480071f60 0x7f048015c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f0488503700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04800723e0 con 0x7f046c005210 2023-08-27T11:28:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f0488503700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04800a9370 con 0x7f048015e1e0 2023-08-27T11:28:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f0488503700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048013e9e0 con 0x7f0480071f60 2023-08-27T11:28:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f048629f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046c005210 0x7f048013e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f048629f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046c005210 0x7f048013e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36824/0 (socket says 172.21.15.112:36824) 2023-08-27T11:28:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f048629f700 1 -- 172.21.15.112:0/2341642220 learned_addr learned my addr 172.21.15.112:0/2341642220 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f0486aa0700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0480071f60 0x7f048015c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f0485a9e700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f048015e1e0 0x7f0480071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f048629f700 1 -- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0480071f60 msgr2=0x7f048015c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f048629f700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0480071f60 0x7f048015c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f048629f700 1 -- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f048015e1e0 msgr2=0x7f0480071a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f048629f700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f048015e1e0 0x7f0480071a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f048629f700 1 -- 172.21.15.112:0/2341642220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04800a8480 con 0x7f046c005210 2023-08-27T11:28:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.853+0000 7f0485a9e700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f048015e1e0 0x7f0480071a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.854+0000 7f048629f700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046c005210 0x7f048013e4a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f0470007370 tx=0x7f0470007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.854+0000 7f0476ffd700 1 -- 172.21.15.112:0/2341642220 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04700077a0 con 0x7f046c005210 2023-08-27T11:28:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.854+0000 7f0476ffd700 1 -- 172.21.15.112:0/2341642220 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047000f870 con 0x7f046c005210 2023-08-27T11:28:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.854+0000 7f0488503700 1 -- 172.21.15.112:0/2341642220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04800a8760 con 0x7f046c005210 2023-08-27T11:28:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.854+0000 7f0476ffd700 1 -- 172.21.15.112:0/2341642220 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047000fc20 con 0x7f046c005210 2023-08-27T11:28:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.854+0000 7f0488503700 1 -- 172.21.15.112:0/2341642220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04800a8d20 con 0x7f046c005210 2023-08-27T11:28:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.855+0000 7f0488503700 1 -- 172.21.15.112:0/2341642220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0480067e90 con 0x7f046c005210 2023-08-27T11:28:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.855+0000 7f0476ffd700 1 -- 172.21.15.112:0/2341642220 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f047000fd80 con 0x7f046c005210 2023-08-27T11:28:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.855+0000 7f0476ffd700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f046406a500 0x7f046406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.855+0000 7f0476ffd700 1 -- 172.21.15.112:0/2341642220 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0470088890 con 0x7f046c005210 2023-08-27T11:28:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.856+0000 7f0485a9e700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f046406a500 0x7f046406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.856+0000 7f0485a9e700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f046406a500 0x7f046406c9c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0480141e20 tx=0x7f047c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:42.860+0000 7f0476ffd700 1 -- 172.21.15.112:0/2341642220 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0470053db0 con 0x7f046c005210 2023-08-27T11:28:43.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.045+0000 7f0488503700 1 -- 172.21.15.112:0/2341642220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0480067a20 con 0x7f046c005210 2023-08-27T11:28:43.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.046+0000 7f0476ffd700 1 -- 172.21.15.112:0/2341642220 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v145) v1 ==== 92+0+10 (secure 0 0 0) 0x7f04700573d0 con 0x7f046c005210 2023-08-27T11:28:43.048 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:28:43.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.048+0000 7f0488503700 1 -- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f046406a500 msgr2=0x7f046406c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.048+0000 7f0488503700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f046406a500 0x7f046406c9c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0480141e20 tx=0x7f047c009040 comp rx=0 tx=0).stop 2023-08-27T11:28:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.048+0000 7f0488503700 1 -- 172.21.15.112:0/2341642220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046c005210 msgr2=0x7f048013e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.048+0000 7f0488503700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046c005210 0x7f048013e4a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f0470007370 tx=0x7f0470007680 comp rx=0 tx=0).stop 2023-08-27T11:28:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.048+0000 7f0488503700 1 -- 172.21.15.112:0/2341642220 shutdown_connections 2023-08-27T11:28:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.048+0000 7f0488503700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f046406a500 0x7f046406c9c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.048+0000 7f0488503700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0480071f60 0x7f048015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.048+0000 7f0488503700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046c005210 0x7f048013e4a0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.048+0000 7f0488503700 1 --2- 172.21.15.112:0/2341642220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f048015e1e0 0x7f0480071a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.048+0000 7f0488503700 1 -- 172.21.15.112:0/2341642220 >> 172.21.15.112:0/2341642220 conn(0x7f04800b1e90 msgr2=0x7f048015abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:43.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.048+0000 7f0488503700 1 -- 172.21.15.112:0/2341642220 shutdown_connections 2023-08-27T11:28:43.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.048+0000 7f0488503700 1 -- 172.21.15.112:0/2341642220 wait complete. 2023-08-27T11:28:43.061 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:28:43.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.183+0000 7f71b151b700 1 Processor -- start 2023-08-27T11:28:43.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.183+0000 7f71b151b700 1 -- start start 2023-08-27T11:28:43.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.183+0000 7f71b151b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f71ac0ce210 0x7f71ac15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.183+0000 7f71b151b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71ac0ce5f0 0x7f71ac0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71b151b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ac0cb500 con 0x7f71ac0b1ff0 2023-08-27T11:28:43.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71b151b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac15ded0 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71b151b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac0cb7b0 con 0x7f71ac0ce210 2023-08-27T11:28:43.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71aa7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71ac0ce5f0 0x7f71ac0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71aa7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71ac0ce5f0 0x7f71ac0cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36832/0 (socket says 172.21.15.112:36832) 2023-08-27T11:28:43.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71aa7fc700 1 -- 172.21.15.112:0/4212312588 learned_addr learned my addr 172.21.15.112:0/4212312588 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:43.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71aaffd700 1 --2- 172.21.15.112:0/4212312588 >> v2:172.21.15.138:3300/0 conn(0x7f71ac0ce210 0x7f71ac15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71aa7fc700 1 -- 172.21.15.112:0/4212312588 >> v2:172.21.15.138:3300/0 conn(0x7f71ac0ce210 msgr2=0x7f71ac15aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:43.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71aa7fc700 1 --2- 172.21.15.112:0/4212312588 >> v2:172.21.15.138:3300/0 conn(0x7f71ac0ce210 0x7f71ac15aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71aa7fc700 1 -- 172.21.15.112:0/4212312588 >> v1:172.21.15.138:6789/0 conn(0x7f71ac0b1ff0 legacy=0x7f71ac06c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71aa7fc700 1 -- 172.21.15.112:0/4212312588 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac15d530 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71aa7fc700 1 --2- 172.21.15.112:0/4212312588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ac0ce5f0 0x7f71ac0cafc0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0cfa80 tx=0x7f71a000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b144b605671e17a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:43.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71a9ffb700 1 -- 172.21.15.112:0/4212312588 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a000dd30 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71a9ffb700 1 -- 172.21.15.112:0/4212312588 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a000de90 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.184+0000 7f71a9ffb700 1 -- 172.21.15.112:0/4212312588 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0010690 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.185+0000 7f71b151b700 1 -- 172.21.15.112:0/4212312588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ac0ce5f0 msgr2=0x7f71ac0cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.185+0000 7f71b151b700 1 --2- 172.21.15.112:0/4212312588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ac0ce5f0 0x7f71ac0cafc0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0cfa80 tx=0x7f71a000be70 comp rx=0 tx=0).stop 2023-08-27T11:28:43.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.185+0000 7f71b151b700 1 -- 172.21.15.112:0/4212312588 shutdown_connections 2023-08-27T11:28:43.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.185+0000 7f71b151b700 1 --2- 172.21.15.112:0/4212312588 >> v2:172.21.15.138:3300/0 conn(0x7f71ac0ce210 0x7f71ac15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.185+0000 7f71b151b700 1 --2- 172.21.15.112:0/4212312588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ac0ce5f0 0x7f71ac0cafc0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.185+0000 7f71b151b700 1 -- 172.21.15.112:0/4212312588 >> 172.21.15.112:0/4212312588 conn(0x7f71ac0cef70 msgr2=0x7f71ac0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.185+0000 7f71b151b700 1 -- 172.21.15.112:0/4212312588 shutdown_connections 2023-08-27T11:28:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.185+0000 7f71b151b700 1 -- 172.21.15.112:0/4212312588 wait complete. 2023-08-27T11:28:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.186+0000 7f71b151b700 1 Processor -- start 2023-08-27T11:28:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.186+0000 7f71b151b700 1 -- start start 2023-08-27T11:28:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.187+0000 7f71b151b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71ac0b1ff0 0x7f71ac150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.187+0000 7f71b151b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71ac0ce210 0x7f71ac14d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.187+0000 7f71b151b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ac0ce5f0 0x7f71ac154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.187+0000 7f71b151b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac1508c0 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.187+0000 7f71b151b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac151220 con 0x7f71ac0b1ff0 2023-08-27T11:28:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.187+0000 7f71b151b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac15b140 con 0x7f71ac0ce210 2023-08-27T11:28:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.187+0000 7f71ab7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ac0ce5f0 0x7f71ac154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.187+0000 7f71ab7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ac0ce5f0 0x7f71ac154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36840/0 (socket says 172.21.15.112:36840) 2023-08-27T11:28:43.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.187+0000 7f71ab7fe700 1 -- 172.21.15.112:0/4097086185 learned_addr learned my addr 172.21.15.112:0/4097086185 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:43.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.187+0000 7f71aaffd700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71ac0b1ff0 0x7f71ac150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.187+0000 7f71aa7fc700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71ac0ce210 0x7f71ac14d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.187+0000 7f71ab7fe700 1 -- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71ac0ce210 msgr2=0x7f71ac14d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.187+0000 7f71ab7fe700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71ac0ce210 0x7f71ac14d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.188+0000 7f71ab7fe700 1 -- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71ac0b1ff0 msgr2=0x7f71ac150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.188+0000 7f71ab7fe700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71ac0b1ff0 0x7f71ac150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.188+0000 7f71ab7fe700 1 -- 172.21.15.112:0/4097086185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a000b7f0 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.188+0000 7f71ab7fe700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ac0ce5f0 0x7f71ac154cd0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f71a400cb90 tx=0x7f71a400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:43.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.188+0000 7f71937fe700 1 -- 172.21.15.112:0/4097086185 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4003c50 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.188+0000 7f71937fe700 1 -- 172.21.15.112:0/4097086185 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a4003dd0 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.188+0000 7f71b151b700 1 -- 172.21.15.112:0/4097086185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac150290 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.188+0000 7f71937fe700 1 -- 172.21.15.112:0/4097086185 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a400aa80 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.188+0000 7f71b151b700 1 -- 172.21.15.112:0/4097086185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac1504e0 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.189+0000 7f71b151b700 1 -- 172.21.15.112:0/4097086185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac067a20 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.190+0000 7f71937fe700 1 -- 172.21.15.112:0/4097086185 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f71a400abe0 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.190+0000 7f71937fe700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f719806a4d0 0x7f719806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.190+0000 7f71937fe700 1 -- 172.21.15.112:0/4097086185 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f71a4089310 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.190+0000 7f71aaffd700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f719806a4d0 0x7f719806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.191+0000 7f71aaffd700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f719806a4d0 0x7f719806c990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f719c006b20 tx=0x7f719c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:43.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.193+0000 7f71937fe700 1 -- 172.21.15.112:0/4097086185 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f71a40547f0 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.383+0000 7f71b151b700 1 -- 172.21.15.112:0/4097086185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71ac0b2d10 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.383+0000 7f71937fe700 1 -- 172.21.15.112:0/4097086185 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71a4057e10 con 0x7f71ac0ce5f0 2023-08-27T11:28:43.385 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:28:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.385+0000 7f71b151b700 1 -- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f719806a4d0 msgr2=0x7f719806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.385+0000 7f71b151b700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f719806a4d0 0x7f719806c990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f719c006b20 tx=0x7f719c008040 comp rx=0 tx=0).stop 2023-08-27T11:28:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.385+0000 7f71b151b700 1 -- 172.21.15.112:0/4097086185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ac0ce5f0 msgr2=0x7f71ac154cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.385+0000 7f71b151b700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ac0ce5f0 0x7f71ac154cd0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f71a400cb90 tx=0x7f71a400cea0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.385+0000 7f71b151b700 1 -- 172.21.15.112:0/4097086185 shutdown_connections 2023-08-27T11:28:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.385+0000 7f71b151b700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f719806a4d0 0x7f719806c990 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.385+0000 7f71b151b700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ac0ce5f0 0x7f71ac154cd0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.385+0000 7f71b151b700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71ac0b1ff0 0x7f71ac150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.385+0000 7f71b151b700 1 --2- 172.21.15.112:0/4097086185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71ac0ce210 0x7f71ac14d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.385+0000 7f71b151b700 1 -- 172.21.15.112:0/4097086185 >> 172.21.15.112:0/4097086185 conn(0x7f71ac0cef70 msgr2=0x7f71ac0aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:43.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.385+0000 7f71b151b700 1 -- 172.21.15.112:0/4097086185 shutdown_connections 2023-08-27T11:28:43.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.385+0000 7f71b151b700 1 -- 172.21.15.112:0/4097086185 wait complete. 2023-08-27T11:28:43.399 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:28:43.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.520+0000 7fd467470700 1 Processor -- start 2023-08-27T11:28:43.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.521+0000 7fd467470700 1 -- start start 2023-08-27T11:28:43.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.521+0000 7fd467470700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd460146c20 0x7fd460160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.521+0000 7fd467470700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd46006f790 0x7fd46006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.521+0000 7fd467470700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4600b5070 con 0x7fd4600b3440 2023-08-27T11:28:43.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.521+0000 7fd467470700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd460070180 con 0x7fd460146c20 2023-08-27T11:28:43.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.521+0000 7fd467470700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd460147000 con 0x7fd46006f790 2023-08-27T11:28:43.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.522+0000 7fd46520c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd460146c20 0x7fd460160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.522+0000 7fd464a0b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd4600b3440 0x7fd46015e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59458/0 (socket says 172.21.15.112:59458) 2023-08-27T11:28:43.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.522+0000 7fd464a0b700 1 -- 172.21.15.112:0/2199795235 learned_addr learned my addr 172.21.15.112:0/2199795235 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:43.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.522+0000 7fd465a0d700 1 --2- 172.21.15.112:0/2199795235 >> v2:172.21.15.138:3300/0 conn(0x7fd46006f790 0x7fd46006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.522+0000 7fd46520c700 1 -- 172.21.15.112:0/2199795235 >> v2:172.21.15.138:3300/0 conn(0x7fd46006f790 msgr2=0x7fd46006fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:43.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.522+0000 7fd46520c700 1 --2- 172.21.15.112:0/2199795235 >> v2:172.21.15.138:3300/0 conn(0x7fd46006f790 0x7fd46006fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.522+0000 7fd46520c700 1 -- 172.21.15.112:0/2199795235 >> v1:172.21.15.112:6789/0 conn(0x7fd4600b3440 legacy=0x7fd46015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.522+0000 7fd457fff700 1 -- 172.21.15.112:0/2199795235 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419374529 0 0) 0x7fd45c001110 con 0x7fd4600b3440 2023-08-27T11:28:43.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.522+0000 7fd46520c700 1 -- 172.21.15.112:0/2199795235 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4601614a0 con 0x7fd460146c20 2023-08-27T11:28:43.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.522+0000 7fd46520c700 1 --2- 172.21.15.112:0/2199795235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd460146c20 0x7fd460160f60 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd4600b1c80 tx=0x7fd450009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c325edaed1ae179 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:43.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.522+0000 7fd457fff700 1 -- 172.21.15.112:0/2199795235 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450011830 con 0x7fd460146c20 2023-08-27T11:28:43.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.522+0000 7fd457fff700 1 -- 172.21.15.112:0/2199795235 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd450011990 con 0x7fd460146c20 2023-08-27T11:28:43.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.522+0000 7fd457fff700 1 -- 172.21.15.112:0/2199795235 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450011d20 con 0x7fd460146c20 2023-08-27T11:28:43.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.523+0000 7fd467470700 1 -- 172.21.15.112:0/2199795235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd460146c20 msgr2=0x7fd460160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.523+0000 7fd467470700 1 --2- 172.21.15.112:0/2199795235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd460146c20 0x7fd460160f60 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd4600b1c80 tx=0x7fd450009b20 comp rx=0 tx=0).stop 2023-08-27T11:28:43.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.523+0000 7fd467470700 1 -- 172.21.15.112:0/2199795235 shutdown_connections 2023-08-27T11:28:43.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.523+0000 7fd467470700 1 --2- 172.21.15.112:0/2199795235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd460146c20 0x7fd460160f60 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.523+0000 7fd467470700 1 --2- 172.21.15.112:0/2199795235 >> v2:172.21.15.138:3300/0 conn(0x7fd46006f790 0x7fd46006fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.523+0000 7fd467470700 1 -- 172.21.15.112:0/2199795235 >> 172.21.15.112:0/2199795235 conn(0x7fd4600b1e90 msgr2=0x7fd4600b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:43.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.523+0000 7fd467470700 1 -- 172.21.15.112:0/2199795235 shutdown_connections 2023-08-27T11:28:43.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.523+0000 7fd467470700 1 -- 172.21.15.112:0/2199795235 wait complete. 2023-08-27T11:28:43.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.524+0000 7fd467470700 1 Processor -- start 2023-08-27T11:28:43.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.524+0000 7fd467470700 1 -- start start 2023-08-27T11:28:43.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.525+0000 7fd467470700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd46006f790 0x7fd460140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.525+0000 7fd467470700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b3440 0x7fd4601f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.525+0000 7fd467470700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460146c20 0x7fd4600c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.525+0000 7fd467470700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd460142840 con 0x7fd4600b3440 2023-08-27T11:28:43.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.525+0000 7fd467470700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4601f5850 con 0x7fd46006f790 2023-08-27T11:28:43.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.525+0000 7fd467470700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4601f5b30 con 0x7fd460146c20 2023-08-27T11:28:43.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.525+0000 7fd464a0b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b3440 0x7fd4601f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.525+0000 7fd464a0b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b3440 0x7fd4601f4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36852/0 (socket says 172.21.15.112:36852) 2023-08-27T11:28:43.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.525+0000 7fd464a0b700 1 -- 172.21.15.112:0/34438292 learned_addr learned my addr 172.21.15.112:0/34438292 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:43.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.526+0000 7fd464a0b700 1 -- 172.21.15.112:0/34438292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460146c20 msgr2=0x7fd4600c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.526+0000 7fd464a0b700 1 --2- 172.21.15.112:0/34438292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460146c20 0x7fd4600c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.526+0000 7fd464a0b700 1 -- 172.21.15.112:0/34438292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd46006f790 msgr2=0x7fd460140aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.526+0000 7fd46520c700 1 --2- 172.21.15.112:0/34438292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd46006f790 0x7fd460140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.526+0000 7fd464a0b700 1 --2- 172.21.15.112:0/34438292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd46006f790 0x7fd460140aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.526+0000 7fd464a0b700 1 -- 172.21.15.112:0/34438292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4500094a0 con 0x7fd4600b3440 2023-08-27T11:28:43.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.526+0000 7fd464a0b700 1 --2- 172.21.15.112:0/34438292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b3440 0x7fd4601f4f00 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00c550 tx=0x7fd45c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:43.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.526+0000 7fd455ffb700 1 -- 172.21.15.112:0/34438292 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c004c80 con 0x7fd4600b3440 2023-08-27T11:28:43.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.526+0000 7fd455ffb700 1 -- 172.21.15.112:0/34438292 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c004e00 con 0x7fd4600b3440 2023-08-27T11:28:43.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.526+0000 7fd455ffb700 1 -- 172.21.15.112:0/34438292 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c007310 con 0x7fd4600b3440 2023-08-27T11:28:43.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.526+0000 7fd467470700 1 -- 172.21.15.112:0/34438292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4601f5e10 con 0x7fd4600b3440 2023-08-27T11:28:43.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.526+0000 7fd467470700 1 -- 172.21.15.112:0/34438292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4601f5440 con 0x7fd4600b3440 2023-08-27T11:28:43.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.527+0000 7fd467470700 1 -- 172.21.15.112:0/34438292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd460067e90 con 0x7fd4600b3440 2023-08-27T11:28:43.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.527+0000 7fd455ffb700 1 -- 172.21.15.112:0/34438292 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fd45c007470 con 0x7fd4600b3440 2023-08-27T11:28:43.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.528+0000 7fd455ffb700 1 --2- 172.21.15.112:0/34438292 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd44406a4d0 0x7fd44406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.528+0000 7fd455ffb700 1 -- 172.21.15.112:0/34438292 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd45c088430 con 0x7fd4600b3440 2023-08-27T11:28:43.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.528+0000 7fd46520c700 1 --2- 172.21.15.112:0/34438292 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd44406a4d0 0x7fd44406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.529+0000 7fd46520c700 1 --2- 172.21.15.112:0/34438292 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd44406a4d0 0x7fd44406c990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd450000c00 tx=0x7fd450008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:43.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.532+0000 7fd455ffb700 1 -- 172.21.15.112:0/34438292 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd45c0539e0 con 0x7fd4600b3440 2023-08-27T11:28:43.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.721+0000 7fd467470700 1 -- 172.21.15.112:0/34438292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd4600b5070 con 0x7fd4600b3440 2023-08-27T11:28:43.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.727+0000 7fd455ffb700 1 -- 172.21.15.112:0/34438292 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd4600b5070 con 0x7fd4600b3440 2023-08-27T11:28:43.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.729+0000 7fd467470700 1 -- 172.21.15.112:0/34438292 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd44406a4d0 msgr2=0x7fd44406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.729+0000 7fd467470700 1 --2- 172.21.15.112:0/34438292 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd44406a4d0 0x7fd44406c990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd450000c00 tx=0x7fd450008030 comp rx=0 tx=0).stop 2023-08-27T11:28:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.729+0000 7fd467470700 1 -- 172.21.15.112:0/34438292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b3440 msgr2=0x7fd4601f4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.729+0000 7fd467470700 1 --2- 172.21.15.112:0/34438292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b3440 0x7fd4601f4f00 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00c550 tx=0x7fd45c00c910 comp rx=0 tx=0).stop 2023-08-27T11:28:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.729+0000 7fd467470700 1 -- 172.21.15.112:0/34438292 shutdown_connections 2023-08-27T11:28:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.729+0000 7fd467470700 1 --2- 172.21.15.112:0/34438292 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd44406a4d0 0x7fd44406c990 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.729+0000 7fd467470700 1 --2- 172.21.15.112:0/34438292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460146c20 0x7fd4600c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.729+0000 7fd467470700 1 --2- 172.21.15.112:0/34438292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd46006f790 0x7fd460140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.729+0000 7fd467470700 1 --2- 172.21.15.112:0/34438292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b3440 0x7fd4601f4f00 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.729+0000 7fd467470700 1 -- 172.21.15.112:0/34438292 >> 172.21.15.112:0/34438292 conn(0x7fd4600b1e90 msgr2=0x7fd4601413d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:43.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.729+0000 7fd467470700 1 -- 172.21.15.112:0/34438292 shutdown_connections 2023-08-27T11:28:43.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:43.729+0000 7fd467470700 1 -- 172.21.15.112:0/34438292 wait complete. 2023-08-27T11:28:43.743 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:28:43.743 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:28:43.783 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:28:43.828 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:28:43.828 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:28:43.829 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:43.846 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:28:43.847 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:28:43.847 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:43.866 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:28:43.867 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:28:43.867 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:43.903 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:28:43.903 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:28:43.903 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:43.924 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:28:43.925 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:28:43.925 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:44.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.061+0000 7f8dc81fb700 1 Processor -- start 2023-08-27T11:28:44.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.061+0000 7f8dc81fb700 1 -- start start 2023-08-27T11:28:44.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc81fb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8dc0163f90 0x7f8dc0166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc81fb700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc00b5ae0 con 0x7f8dc0144570 2023-08-27T11:28:44.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc81fb700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc00b45f0 con 0x7f8dc00b3e50 2023-08-27T11:28:44.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc81fb700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc00cf7a0 con 0x7f8dc0163f90 2023-08-27T11:28:44.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc6798700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8dc0163f90 0x7f8dc0166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc6798700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8dc0163f90 0x7f8dc0166380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36862/0 (socket says 172.21.15.112:36862) 2023-08-27T11:28:44.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc5f97700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8dc0144570 0x7f8dc015ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59472/0 (socket says 172.21.15.112:59472) 2023-08-27T11:28:44.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc6798700 1 -- 172.21.15.112:0/1721238262 learned_addr learned my addr 172.21.15.112:0/1721238262 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:44.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc6798700 1 -- 172.21.15.112:0/1721238262 >> v1:172.21.15.138:6789/0 conn(0x7f8dc00b3e50 legacy=0x7f8dc015f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:44.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc6798700 1 -- 172.21.15.112:0/1721238262 >> v1:172.21.15.112:6789/0 conn(0x7f8dc0144570 legacy=0x7f8dc015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc6798700 1 -- 172.21.15.112:0/1721238262 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc015fc40 con 0x7f8dc0163f90 2023-08-27T11:28:44.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc4f95700 1 -- 172.21.15.112:0/1721238262 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629637791 0 0) 0x7f8dbc001110 con 0x7f8dc0144570 2023-08-27T11:28:44.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc6798700 1 --2- 172.21.15.112:0/1721238262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0163f90 0x7f8dc0166380 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009e50 tx=0x7f8db800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af3de3a5537f2c6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:44.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc4f95700 1 -- 172.21.15.112:0/1721238262 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db800e070 con 0x7f8dc0163f90 2023-08-27T11:28:44.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc4f95700 1 -- 172.21.15.112:0/1721238262 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db8003a20 con 0x7f8dc0163f90 2023-08-27T11:28:44.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.062+0000 7f8dc4f95700 1 -- 172.21.15.112:0/1721238262 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8003dd0 con 0x7f8dc0163f90 2023-08-27T11:28:44.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.063+0000 7f8dc81fb700 1 -- 172.21.15.112:0/1721238262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0163f90 msgr2=0x7f8dc0166380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.063+0000 7f8dc81fb700 1 --2- 172.21.15.112:0/1721238262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0163f90 0x7f8dc0166380 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009e50 tx=0x7f8db800b330 comp rx=0 tx=0).stop 2023-08-27T11:28:44.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.063+0000 7f8dc81fb700 1 -- 172.21.15.112:0/1721238262 shutdown_connections 2023-08-27T11:28:44.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.063+0000 7f8dc81fb700 1 --2- 172.21.15.112:0/1721238262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0163f90 0x7f8dc0166380 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.063+0000 7f8dc81fb700 1 -- 172.21.15.112:0/1721238262 >> 172.21.15.112:0/1721238262 conn(0x7f8dc006f970 msgr2=0x7f8dc006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:44.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.063+0000 7f8dc81fb700 1 -- 172.21.15.112:0/1721238262 shutdown_connections 2023-08-27T11:28:44.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.063+0000 7f8dc81fb700 1 -- 172.21.15.112:0/1721238262 wait complete. 2023-08-27T11:28:44.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.064+0000 7f8dc81fb700 1 Processor -- start 2023-08-27T11:28:44.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.064+0000 7f8dc81fb700 1 -- start start 2023-08-27T11:28:44.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc81fb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8dc00b3e50 0x7f8dc0137320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc81fb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0144570 0x7f8dc0143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc81fb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dc0163f90 0x7f8dc0143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc81fb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc01416b0 con 0x7f8dc0144570 2023-08-27T11:28:44.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc81fb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00708a0 con 0x7f8dc0163f90 2023-08-27T11:28:44.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc81fb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00718b0 con 0x7f8dc00b3e50 2023-08-27T11:28:44.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc5796700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0144570 0x7f8dc0143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc5796700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0144570 0x7f8dc0143250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36866/0 (socket says 172.21.15.112:36866) 2023-08-27T11:28:44.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc5796700 1 -- 172.21.15.112:0/4197159106 learned_addr learned my addr 172.21.15.112:0/4197159106 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:44.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc6798700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dc0163f90 0x7f8dc0143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc5f97700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8dc00b3e50 0x7f8dc0137320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc5796700 1 -- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8dc00b3e50 msgr2=0x7f8dc0137320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc5796700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8dc00b3e50 0x7f8dc0137320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc5796700 1 -- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dc0163f90 msgr2=0x7f8dc0143790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc5796700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dc0163f90 0x7f8dc0143790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc5796700 1 -- 172.21.15.112:0/4197159106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db8009b30 con 0x7f8dc0144570 2023-08-27T11:28:44.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc5f97700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8dc00b3e50 0x7f8dc0137320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:44.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.065+0000 7f8dc5796700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0144570 0x7f8dc0143250 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c4d0 tx=0x7f8db000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:44.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.066+0000 7f8db67fc700 1 -- 172.21.15.112:0/4197159106 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0011790 con 0x7f8dc0144570 2023-08-27T11:28:44.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.066+0000 7f8db67fc700 1 -- 172.21.15.112:0/4197159106 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db00118f0 con 0x7f8dc0144570 2023-08-27T11:28:44.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.066+0000 7f8db67fc700 1 -- 172.21.15.112:0/4197159106 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0011ca0 con 0x7f8dc0144570 2023-08-27T11:28:44.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.066+0000 7f8dc81fb700 1 -- 172.21.15.112:0/4197159106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc0070af0 con 0x7f8dc0144570 2023-08-27T11:28:44.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.066+0000 7f8dc81fb700 1 -- 172.21.15.112:0/4197159106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc00710b0 con 0x7f8dc0144570 2023-08-27T11:28:44.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.067+0000 7f8dc81fb700 1 -- 172.21.15.112:0/4197159106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc0067a20 con 0x7f8dc0144570 2023-08-27T11:28:44.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.067+0000 7f8db67fc700 1 -- 172.21.15.112:0/4197159106 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f8db001c460 con 0x7f8dc0144570 2023-08-27T11:28:44.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.067+0000 7f8db67fc700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8da406a4d0 0x7f8da406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.068+0000 7f8db67fc700 1 -- 172.21.15.112:0/4197159106 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8db0088780 con 0x7f8dc0144570 2023-08-27T11:28:44.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.068+0000 7f8dc5f97700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8da406a4d0 0x7f8da406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.068+0000 7f8dc5f97700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8da406a4d0 0x7f8da406c990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc007fd0 tx=0x7f8dbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:44.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.072+0000 7f8db67fc700 1 -- 172.21.15.112:0/4197159106 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8db0053c20 con 0x7f8dc0144570 2023-08-27T11:28:44.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.304+0000 7f8dc81fb700 1 -- 172.21.15.112:0/4197159106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8dc00b6f90 con 0x7f8dc0144570 2023-08-27T11:28:44.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.305+0000 7f8db67fc700 1 -- 172.21.15.112:0/4197159106 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f8db0057240 con 0x7f8dc0144570 2023-08-27T11:28:44.306 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:44.307 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":98,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/739970856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":739970856},{"type":"v1","addr":"172.21.15.138:6837","nonce":739970856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":6929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1107384711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1107384711},{"type":"v1","addr":"172.21.15.138:6836","nonce":1107384711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3233069521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3233069521},{"type":"v1","addr":"172.21.15.112:6834","nonce":3233069521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2559241026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2559241026},{"type":"v1","addr":"172.21.15.112:6835","nonce":2559241026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[]} 2023-08-27T11:28:44.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.307+0000 7f8dc81fb700 1 -- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8da406a4d0 msgr2=0x7f8da406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.307+0000 7f8dc81fb700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8da406a4d0 0x7f8da406c990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc007fd0 tx=0x7f8dbc009040 comp rx=0 tx=0).stop 2023-08-27T11:28:44.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.307+0000 7f8dc81fb700 1 -- 172.21.15.112:0/4197159106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0144570 msgr2=0x7f8dc0143250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.307+0000 7f8dc81fb700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0144570 0x7f8dc0143250 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c4d0 tx=0x7f8db000c890 comp rx=0 tx=0).stop 2023-08-27T11:28:44.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.307+0000 7f8dc81fb700 1 -- 172.21.15.112:0/4197159106 shutdown_connections 2023-08-27T11:28:44.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.307+0000 7f8dc81fb700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8da406a4d0 0x7f8da406c990 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.307+0000 7f8dc81fb700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dc0163f90 0x7f8dc0143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.307+0000 7f8dc81fb700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8dc00b3e50 0x7f8dc0137320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.307+0000 7f8dc81fb700 1 --2- 172.21.15.112:0/4197159106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0144570 0x7f8dc0143250 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.307+0000 7f8dc81fb700 1 -- 172.21.15.112:0/4197159106 >> 172.21.15.112:0/4197159106 conn(0x7f8dc006f970 msgr2=0x7f8dc0141bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:44.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.307+0000 7f8dc81fb700 1 -- 172.21.15.112:0/4197159106 shutdown_connections 2023-08-27T11:28:44.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.307+0000 7f8dc81fb700 1 -- 172.21.15.112:0/4197159106 wait complete. 2023-08-27T11:28:44.311 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 98 2023-08-27T11:28:44.320 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:28:44.321 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:28:44.321 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:28:44.321 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:44.322 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:28:44.322 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:28:44.322 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:28:44.323 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:44.323 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:28:44.323 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:28:44.323 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:28:44.324 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:44.324 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:28:44.324 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:28:44.325 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:28:44.325 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:44.329 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:28:44.330 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:28:44.331 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:28:44.332 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:28:44.334 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:28:44.335 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:28:44.336 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:28:44.336 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:28:44.373 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:28:44.373 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:44.374 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:28:44.374 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:44.377 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:28:44.377 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:44.378 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:28:44.379 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:44.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:28:44.451 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:28:44.451 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:28:44.463 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:28:44.463 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2023-08-27T11:28:44.463 INFO:tasks.ceph.mds.b.smithi138.stderr:2023-08-27T11:28:44.462+0000 7fc8f0df5700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-08-27T11:28:44.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.505+0000 7f665553e700 1 Processor -- start 2023-08-27T11:28:44.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.506+0000 7f665553e700 1 -- start start 2023-08-27T11:28:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.506+0000 7f665553e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6650066720 0x7f665015e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.506+0000 7f665553e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66500cec70 0x7f66500708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.506+0000 7f665553e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6650072250 con 0x7f6650146c00 2023-08-27T11:28:44.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.506+0000 7f665553e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66500d0b60 con 0x7f6650066720 2023-08-27T11:28:44.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.506+0000 7f665553e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66500724a0 con 0x7f66500cec70 2023-08-27T11:28:44.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.506+0000 7f664e7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6650066720 0x7f665015e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.507+0000 7f664effd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6650146c00 0x7f6650066010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59510/0 (socket says 172.21.15.112:59510) 2023-08-27T11:28:44.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.507+0000 7f664e7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6650066720 0x7f665015e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36896/0 (socket says 172.21.15.112:36896) 2023-08-27T11:28:44.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.507+0000 7f664effd700 1 -- 172.21.15.112:0/1408939707 learned_addr learned my addr 172.21.15.112:0/1408939707 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:44.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.507+0000 7f664f7fe700 1 --2- 172.21.15.112:0/1408939707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66500cec70 0x7f66500708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.507+0000 7f664e7fc700 1 -- 172.21.15.112:0/1408939707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66500cec70 msgr2=0x7f66500708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:44.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.507+0000 7f664e7fc700 1 --2- 172.21.15.112:0/1408939707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66500cec70 0x7f66500708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.507+0000 7f664e7fc700 1 -- 172.21.15.112:0/1408939707 >> v1:172.21.15.112:6789/0 conn(0x7f6650146c00 legacy=0x7f6650066010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.507+0000 7f664dffb700 1 -- 172.21.15.112:0/1408939707 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798828399 0 0) 0x7f6650072250 con 0x7f6650146c00 2023-08-27T11:28:44.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.507+0000 7f664e7fc700 1 -- 172.21.15.112:0/1408939707 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6650072780 con 0x7f6650066720 2023-08-27T11:28:44.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.507+0000 7f664e7fc700 1 --2- 172.21.15.112:0/1408939707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6650066720 0x7f665015e550 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f6640009840 tx=0x7f6640009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e180aaa486873a7f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.507+0000 7f664dffb700 1 -- 172.21.15.112:0/1408939707 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640011830 con 0x7f6650066720 2023-08-27T11:28:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.507+0000 7f664dffb700 1 -- 172.21.15.112:0/1408939707 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6640011990 con 0x7f6650066720 2023-08-27T11:28:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.507+0000 7f664dffb700 1 -- 172.21.15.112:0/1408939707 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640011d20 con 0x7f6650066720 2023-08-27T11:28:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.508+0000 7f665553e700 1 -- 172.21.15.112:0/1408939707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6650066720 msgr2=0x7f665015e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.508+0000 7f665553e700 1 --2- 172.21.15.112:0/1408939707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6650066720 0x7f665015e550 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f6640009840 tx=0x7f6640009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:44.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.508+0000 7f665553e700 1 -- 172.21.15.112:0/1408939707 shutdown_connections 2023-08-27T11:28:44.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.508+0000 7f665553e700 1 --2- 172.21.15.112:0/1408939707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66500cec70 0x7f66500708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.508+0000 7f665553e700 1 --2- 172.21.15.112:0/1408939707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6650066720 0x7f665015e550 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.508+0000 7f665553e700 1 -- 172.21.15.112:0/1408939707 >> 172.21.15.112:0/1408939707 conn(0x7f665006f970 msgr2=0x7f665006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:44.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.508+0000 7f665553e700 1 -- 172.21.15.112:0/1408939707 shutdown_connections 2023-08-27T11:28:44.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.508+0000 7f665553e700 1 -- 172.21.15.112:0/1408939707 wait complete. 2023-08-27T11:28:44.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.509+0000 7f665553e700 1 Processor -- start 2023-08-27T11:28:44.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.509+0000 7f665553e700 1 -- start start 2023-08-27T11:28:44.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f665553e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6650066720 0x7f66501f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f665553e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66500cec70 0x7f66500cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f665553e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6650146c00 0x7f665015fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f665553e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66501f5cd0 con 0x7f6650146c00 2023-08-27T11:28:44.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f665553e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6650144520 con 0x7f6650066720 2023-08-27T11:28:44.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f665553e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6650145820 con 0x7f66500cec70 2023-08-27T11:28:44.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f664f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6650146c00 0x7f665015fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f664f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6650146c00 0x7f665015fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36902/0 (socket says 172.21.15.112:36902) 2023-08-27T11:28:44.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f664f7fe700 1 -- 172.21.15.112:0/1112958167 learned_addr learned my addr 172.21.15.112:0/1112958167 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:44.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f664effd700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6650066720 0x7f66501f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f664e7fc700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66500cec70 0x7f66500cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f664f7fe700 1 -- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66500cec70 msgr2=0x7f66500cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f664f7fe700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66500cec70 0x7f66500cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f664f7fe700 1 -- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6650066720 msgr2=0x7f66501f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f664f7fe700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6650066720 0x7f66501f5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f664f7fe700 1 -- 172.21.15.112:0/1112958167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66400094a0 con 0x7f6650146c00 2023-08-27T11:28:44.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f664f7fe700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6650146c00 0x7f665015fd60 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f6648009dd0 tx=0x7f664800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:44.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f66377fe700 1 -- 172.21.15.112:0/1112958167 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664800cd40 con 0x7f6650146c00 2023-08-27T11:28:44.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f66377fe700 1 -- 172.21.15.112:0/1112958167 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664800f040 con 0x7f6650146c00 2023-08-27T11:28:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f665553e700 1 -- 172.21.15.112:0/1112958167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6650142390 con 0x7f6650146c00 2023-08-27T11:28:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.510+0000 7f66377fe700 1 -- 172.21.15.112:0/1112958167 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66480146c0 con 0x7f6650146c00 2023-08-27T11:28:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.511+0000 7f665553e700 1 -- 172.21.15.112:0/1112958167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6650145d50 con 0x7f6650146c00 2023-08-27T11:28:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.512+0000 7f665553e700 1 -- 172.21.15.112:0/1112958167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66500cf970 con 0x7f6650146c00 2023-08-27T11:28:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.512+0000 7f66377fe700 1 -- 172.21.15.112:0/1112958167 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f6648014820 con 0x7f6650146c00 2023-08-27T11:28:44.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.512+0000 7f66377fe700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f663c06a4d0 0x7f663c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.512+0000 7f66377fe700 1 -- 172.21.15.112:0/1112958167 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f66480896e0 con 0x7f6650146c00 2023-08-27T11:28:44.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.513+0000 7f664effd700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f663c06a4d0 0x7f663c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.513+0000 7f664effd700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f663c06a4d0 0x7f663c06c990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6644000f60 tx=0x7f6644007be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:44.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.517+0000 7f66377fe700 1 -- 172.21.15.112:0/1112958167 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f66480549e0 con 0x7f6650146c00 2023-08-27T11:28:44.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:44.722+0000 7f665553e700 1 -- 172.21.15.112:0/1112958167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6650142670 con 0x7f6650146c00 2023-08-27T11:28:45.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.386+0000 7f66377fe700 1 -- 172.21.15.112:0/1112958167 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v147) v1 ==== 97+0+0 (secure 0 0 0) 0x7f664800cea0 con 0x7f6650146c00 2023-08-27T11:28:45.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.389+0000 7f665553e700 1 -- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f663c06a4d0 msgr2=0x7f663c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.389+0000 7f665553e700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f663c06a4d0 0x7f663c06c990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6644000f60 tx=0x7f6644007be0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.389+0000 7f665553e700 1 -- 172.21.15.112:0/1112958167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6650146c00 msgr2=0x7f665015fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.389+0000 7f665553e700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6650146c00 0x7f665015fd60 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f6648009dd0 tx=0x7f664800c3a0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.389+0000 7f665553e700 1 -- 172.21.15.112:0/1112958167 shutdown_connections 2023-08-27T11:28:45.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.389+0000 7f665553e700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f663c06a4d0 0x7f663c06c990 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.389+0000 7f665553e700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6650146c00 0x7f665015fd60 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.389+0000 7f665553e700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6650066720 0x7f66501f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.389+0000 7f665553e700 1 --2- 172.21.15.112:0/1112958167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66500cec70 0x7f66500cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.389+0000 7f665553e700 1 -- 172.21.15.112:0/1112958167 >> 172.21.15.112:0/1112958167 conn(0x7f665006f970 msgr2=0x7f665015ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:45.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.389+0000 7f665553e700 1 -- 172.21.15.112:0/1112958167 shutdown_connections 2023-08-27T11:28:45.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.389+0000 7f665553e700 1 -- 172.21.15.112:0/1112958167 wait complete. 2023-08-27T11:28:45.393 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2023-08-27T11:28:45.405 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:28:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.529+0000 7f8cba939700 1 Processor -- start 2023-08-27T11:28:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.529+0000 7f8cba939700 1 -- start start 2023-08-27T11:28:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.529+0000 7f8cba939700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cb406b230 0x7f8cb40ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.529+0000 7f8cba939700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb415eb60 con 0x7f8cb406adb0 2023-08-27T11:28:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.529+0000 7f8cba939700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb406b6c0 con 0x7f8cb4151060 2023-08-27T11:28:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.529+0000 7f8cba939700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb40b02f0 con 0x7f8cb406b230 2023-08-27T11:28:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb37fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8cb406adb0 0x7f8cb40cdbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59524/0 (socket says 172.21.15.112:59524) 2023-08-27T11:28:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb37fe700 1 -- 172.21.15.112:0/2548460937 learned_addr learned my addr 172.21.15.112:0/2548460937 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb8ed6700 1 --2- 172.21.15.112:0/2548460937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cb406b230 0x7f8cb40ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226050677 0 0) 0x7f8cb415eb60 con 0x7f8cb406adb0 2023-08-27T11:28:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c9c0035b0 con 0x7f8cb406adb0 2023-08-27T11:28:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2537407106 0 0) 0x7f8c9c0035b0 con 0x7f8cb406adb0 2023-08-27T11:28:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cb406b230 msgr2=0x7f8cb40ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 --2- 172.21.15.112:0/2548460937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cb406b230 0x7f8cb40ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 >> v1:172.21.15.138:6789/0 conn(0x7f8cb4151060 legacy=0x7f8cb415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb40b05d0 con 0x7f8cb406adb0 2023-08-27T11:28:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f8ca8001670 con 0x7f8cb406adb0 2023-08-27T11:28:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 >> v1:172.21.15.112:6789/0 conn(0x7f8cb406adb0 legacy=0x7f8cb40cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 --2- 172.21.15.112:0/2548460937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c9c003a30 0x7f8c9c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 --2- 172.21.15.112:0/2548460937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c9c004390 0x7f8c9c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 --2- 172.21.15.112:0/2548460937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c9c005070 0x7f8c9c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb40b02f0 con 0x7f8c9c005070 2023-08-27T11:28:45.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c00bf80 con 0x7f8c9c004390 2023-08-27T11:28:45.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c00c140 con 0x7f8c9c003a30 2023-08-27T11:28:45.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb8ed6700 1 --2- 172.21.15.112:0/2548460937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cb406b230 0x7f8cb40ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:45.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb8ed6700 1 --2- 172.21.15.112:0/2548460937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c9c005070 0x7f8c9c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb3fff700 1 --2- 172.21.15.112:0/2548460937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c9c003a30 0x7f8c9c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.530+0000 7f8cb37fe700 1 --2- 172.21.15.112:0/2548460937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c9c004390 0x7f8c9c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cb8ed6700 1 -- 172.21.15.112:0/2548460937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c9c003a30 msgr2=0x7f8c9c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cb8ed6700 1 --2- 172.21.15.112:0/2548460937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c9c003a30 0x7f8c9c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cb8ed6700 1 -- 172.21.15.112:0/2548460937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c9c004390 msgr2=0x7f8c9c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cb8ed6700 1 --2- 172.21.15.112:0/2548460937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c9c004390 0x7f8c9c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cb8ed6700 1 -- 172.21.15.112:0/2548460937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c0035b0 con 0x7f8c9c005070 2023-08-27T11:28:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cb8ed6700 1 --2- 172.21.15.112:0/2548460937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c9c005070 0x7f8c9c00ba10 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f8cb406b610 tx=0x7f8c9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fe6e0641dc9fd6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac003c20 con 0x7f8c9c005070 2023-08-27T11:28:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac00f040 con 0x7f8c9c005070 2023-08-27T11:28:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cb2ffd700 1 -- 172.21.15.112:0/2548460937 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00ea60 con 0x7f8c9c005070 2023-08-27T11:28:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cba939700 1 -- 172.21.15.112:0/2548460937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c9c005070 msgr2=0x7f8c9c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cba939700 1 --2- 172.21.15.112:0/2548460937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c9c005070 0x7f8c9c00ba10 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f8cb406b610 tx=0x7f8c9c005880 comp rx=0 tx=0).stop 2023-08-27T11:28:45.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cb8ed6700 1 -- 172.21.15.112:0/2548460937 reap_dead start 2023-08-27T11:28:45.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cba939700 1 -- 172.21.15.112:0/2548460937 shutdown_connections 2023-08-27T11:28:45.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cba939700 1 -- 172.21.15.112:0/2548460937 >> 172.21.15.112:0/2548460937 conn(0x7f8cb40d0000 msgr2=0x7f8cb40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:45.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cba939700 1 -- 172.21.15.112:0/2548460937 shutdown_connections 2023-08-27T11:28:45.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.531+0000 7f8cba939700 1 -- 172.21.15.112:0/2548460937 wait complete. 2023-08-27T11:28:45.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.533+0000 7f8cba939700 1 Processor -- start 2023-08-27T11:28:45.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.533+0000 7f8cba939700 1 -- start start 2023-08-27T11:28:45.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.533+0000 7f8cba939700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cb4153e20 0x7f8cb4142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cba939700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb415de90 0x7f8cb415e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cba939700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cb41f5d60 0x7f8cb41f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cba939700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb415e850 con 0x7f8cb415de90 2023-08-27T11:28:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cba939700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb414eea0 con 0x7f8cb41f5d60 2023-08-27T11:28:45.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cba939700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb414ea10 con 0x7f8cb4153e20 2023-08-27T11:28:45.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb415de90 0x7f8cb415e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb415de90 0x7f8cb415e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36922/0 (socket says 172.21.15.112:36922) 2023-08-27T11:28:45.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb37fe700 1 -- 172.21.15.112:0/196887166 learned_addr learned my addr 172.21.15.112:0/196887166 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:45.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb8ed6700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cb41f5d60 0x7f8cb41f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb3fff700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cb4153e20 0x7f8cb4142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb37fe700 1 -- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cb4153e20 msgr2=0x7f8cb4142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb37fe700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cb4153e20 0x7f8cb4142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb37fe700 1 -- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cb41f5d60 msgr2=0x7f8cb41f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb37fe700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cb41f5d60 0x7f8cb41f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb37fe700 1 -- 172.21.15.112:0/196887166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb4142dc0 con 0x7f8cb415de90 2023-08-27T11:28:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb37fe700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb415de90 0x7f8cb415e310 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8007300 tx=0x7f8ca8007610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb0ff9700 1 -- 172.21.15.112:0/196887166 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca800f660 con 0x7f8cb415de90 2023-08-27T11:28:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb0ff9700 1 -- 172.21.15.112:0/196887166 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca800f7c0 con 0x7f8cb415de90 2023-08-27T11:28:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cba939700 1 -- 172.21.15.112:0/196887166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb414de00 con 0x7f8cb415de90 2023-08-27T11:28:45.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cb0ff9700 1 -- 172.21.15.112:0/196887166 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca800fb70 con 0x7f8cb415de90 2023-08-27T11:28:45.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.534+0000 7f8cba939700 1 -- 172.21.15.112:0/196887166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb414e3c0 con 0x7f8cb415de90 2023-08-27T11:28:45.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.535+0000 7f8cba939700 1 -- 172.21.15.112:0/196887166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb414ebc0 con 0x7f8cb415de90 2023-08-27T11:28:45.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.536+0000 7f8cb0ff9700 1 -- 172.21.15.112:0/196887166 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f8ca800fcd0 con 0x7f8cb415de90 2023-08-27T11:28:45.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.536+0000 7f8cb0ff9700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8ca006a4d0 0x7f8ca006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.536+0000 7f8cb0ff9700 1 -- 172.21.15.112:0/196887166 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8ca8088780 con 0x7f8cb415de90 2023-08-27T11:28:45.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.536+0000 7f8cb3fff700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8ca006a4d0 0x7f8ca006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.537+0000 7f8cb3fff700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8ca006a4d0 0x7f8ca006c990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40019c0 tx=0x7f8ca4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:45.552 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.540+0000 7f8cb0ff9700 1 -- 172.21.15.112:0/196887166 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8cb414ebc0 con 0x7f8cb415de90 2023-08-27T11:28:45.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.809+0000 7f8cba939700 1 -- 172.21.15.112:0/196887166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8cb414eb80 con 0x7f8cb415de90 2023-08-27T11:28:45.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.809+0000 7f8cb0ff9700 1 -- 172.21.15.112:0/196887166 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8ca8057810 con 0x7f8cb415de90 2023-08-27T11:28:45.811 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:45.812 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:28:45.812 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2023-08-27T11:28:45.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.812 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2023-08-27T11:28:45.812 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk2bnkLBAA+vxcTeJzucKk9iSF/Zw3Lg==", 2023-08-27T11:28:45.813 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.813 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:45.813 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:28:45.813 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:28:45.813 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.814 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.814 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2023-08-27T11:28:45.814 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk3zD5OBAASVnwG6yn/YQzfYyeMakFpA==", 2023-08-27T11:28:45.815 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.815 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:45.815 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:28:45.815 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:28:45.815 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.816 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.816 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2023-08-27T11:28:45.816 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtki7wJNBAAsIGgDLM82sWNcKnCDlSBMw==", 2023-08-27T11:28:45.817 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.817 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:45.817 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:28:45.817 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:28:45.818 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.818 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.818 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2023-08-27T11:28:45.819 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkCsW/BBAAFTPiTOpBo1L+rLo9px+AjQ==", 2023-08-27T11:28:45.819 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.819 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:45.819 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:28:45.819 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:28:45.820 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.820 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.821 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2023-08-27T11:28:45.821 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB7MOtkR/cSHhAAR8e8VRd+e0ESJkqQDYWbEw==", 2023-08-27T11:28:45.821 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.821 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.821 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.822 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.822 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.822 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2023-08-27T11:28:45.823 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB+MOtksBF3HRAA1OIXiIkhgPDYpQyYPaWOXg==", 2023-08-27T11:28:45.823 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.823 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.823 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.823 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.824 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.824 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2023-08-27T11:28:45.824 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCBMOtkWsrdHBAAkfWbiE6wAFzpt9+BxRh8fw==", 2023-08-27T11:28:45.824 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.825 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.825 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.825 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.825 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.826 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2023-08-27T11:28:45.826 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCEMOtkIkiKHRAARDoh/FH9ZdqUlZZ29Mc/Dw==", 2023-08-27T11:28:45.826 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.826 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.827 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.827 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.829 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.830 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2023-08-27T11:28:45.830 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCKMOtkPgDzGxAA+U4kbwD0d4lS8aNPG2CgHw==", 2023-08-27T11:28:45.830 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.830 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.830 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.831 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.831 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.831 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2023-08-27T11:28:45.832 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCNMOtkKomSGxAAGAQUcIH2H/uyZIU8vlq1pw==", 2023-08-27T11:28:45.832 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.832 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.832 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.833 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.833 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2023-08-27T11:28:45.833 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCQMOtkHI2MGhAA7sgbi8JXO8P9wnVvMy3PqA==", 2023-08-27T11:28:45.834 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.834 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.834 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.834 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.834 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.835 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2023-08-27T11:28:45.835 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCTMOtkVHpiGhAA4C4YJMWbQ0mmGgB4i8G/AA==", 2023-08-27T11:28:45.835 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.835 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.836 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.836 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.836 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.836 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.837 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2023-08-27T11:28:45.837 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA==", 2023-08-27T11:28:45.837 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.837 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:45.838 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2023-08-27T11:28:45.838 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:28:45.838 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.838 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.838 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2023-08-27T11:28:45.839 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtk66DbDxAAWZ5StELux0lriJ2gEN9icg==", 2023-08-27T11:28:45.839 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.839 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:45.839 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:28:45.839 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:28:45.840 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:28:45.840 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.840 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.840 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2023-08-27T11:28:45.840 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkViKJDBAAF2ofwO+78er9OyFfn2bO+g==", 2023-08-27T11:28:45.841 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.841 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:45.841 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:28:45.841 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:28:45.842 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:28:45.842 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.842 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2023-08-27T11:28:45.842 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkwt+QFRAAowWE69VqeRhQz0896rxx9Q==", 2023-08-27T11:28:45.843 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.843 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:28:45.843 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:28:45.843 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:28:45.844 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:28:45.844 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.844 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.844 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2023-08-27T11:28:45.844 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkjDA+DxAAjmOn82AC2OMpKlAVCD6hlg==", 2023-08-27T11:28:45.845 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.845 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:28:45.845 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:28:45.845 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:28:45.845 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.846 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.846 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.846 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2023-08-27T11:28:45.846 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkXguHKRAALHwzk8diVXqsMvTKr+AzLQ==", 2023-08-27T11:28:45.847 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.847 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:28:45.847 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:28:45.847 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.847 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:45.848 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:45.848 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2023-08-27T11:28:45.848 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkrRjBJRAAq3ftmIh81LKadpliVUiJ6g==", 2023-08-27T11:28:45.848 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:28:45.849 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:28:45.849 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:28:45.849 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:28:45.849 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.849 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:45.850 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:28:45.850 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:28:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.811+0000 7f8cba939700 1 -- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8ca006a4d0 msgr2=0x7f8ca006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.811+0000 7f8cba939700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8ca006a4d0 0x7f8ca006c990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40019c0 tx=0x7f8ca4009040 comp rx=0 tx=0).stop 2023-08-27T11:28:45.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.811+0000 7f8cba939700 1 -- 172.21.15.112:0/196887166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb415de90 msgr2=0x7f8cb415e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.811+0000 7f8cba939700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb415de90 0x7f8cb415e310 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8007300 tx=0x7f8ca8007610 comp rx=0 tx=0).stop 2023-08-27T11:28:45.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.811+0000 7f8cba939700 1 -- 172.21.15.112:0/196887166 shutdown_connections 2023-08-27T11:28:45.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.811+0000 7f8cba939700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8ca006a4d0 0x7f8ca006c990 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.811+0000 7f8cba939700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cb41f5d60 0x7f8cb41f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.811+0000 7f8cba939700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cb4153e20 0x7f8cb4142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.811+0000 7f8cba939700 1 --2- 172.21.15.112:0/196887166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb415de90 0x7f8cb415e310 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.811+0000 7f8cba939700 1 -- 172.21.15.112:0/196887166 >> 172.21.15.112:0/196887166 conn(0x7f8cb40d0000 msgr2=0x7f8cb415be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.811+0000 7f8cba939700 1 -- 172.21.15.112:0/196887166 shutdown_connections 2023-08-27T11:28:45.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.811+0000 7f8cba939700 1 -- 172.21.15.112:0/196887166 wait complete. 2023-08-27T11:28:45.854 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.972+0000 7fb837d94700 1 Processor -- start 2023-08-27T11:28:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.973+0000 7fb837d94700 1 -- start start 2023-08-27T11:28:45.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.973+0000 7fb837d94700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb83013f990 0x7fb8300cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.973+0000 7fb837d94700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb8300cd270 0x7fb8300cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.973+0000 7fb837d94700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83013ff50 con 0x7fb83006e150 2023-08-27T11:28:45.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.973+0000 7fb837d94700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8300cecb0 con 0x7fb8300cd270 2023-08-27T11:28:45.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.973+0000 7fb837d94700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8300cf570 con 0x7fb83013f990 2023-08-27T11:28:45.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.973+0000 7fb836331700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb8300cd270 0x7fb8300cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.974+0000 7fb836331700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb8300cd270 0x7fb8300cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36932/0 (socket says 172.21.15.112:36932) 2023-08-27T11:28:45.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.974+0000 7fb836331700 1 -- 172.21.15.112:0/3922606449 learned_addr learned my addr 172.21.15.112:0/3922606449 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:45.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.974+0000 7fb83532f700 1 --2- 172.21.15.112:0/3922606449 >> v2:172.21.15.138:3300/0 conn(0x7fb83013f990 0x7fb8300cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.974+0000 7fb836331700 1 -- 172.21.15.112:0/3922606449 >> v2:172.21.15.138:3300/0 conn(0x7fb83013f990 msgr2=0x7fb8300cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:45.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.974+0000 7fb836331700 1 --2- 172.21.15.112:0/3922606449 >> v2:172.21.15.138:3300/0 conn(0x7fb83013f990 0x7fb8300cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.974+0000 7fb836331700 1 -- 172.21.15.112:0/3922606449 >> v1:172.21.15.138:6789/0 conn(0x7fb83006e150 legacy=0x7fb83006e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:45.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.974+0000 7fb836331700 1 -- 172.21.15.112:0/3922606449 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb830140230 con 0x7fb8300cd270 2023-08-27T11:28:45.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.974+0000 7fb83532f700 1 --2- 172.21.15.112:0/3922606449 >> v2:172.21.15.138:3300/0 conn(0x7fb83013f990 0x7fb8300cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:45.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.974+0000 7fb836331700 1 --2- 172.21.15.112:0/3922606449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8300cd270 0x7fb8300cd6a0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fb83006ee60 tx=0x7fb82800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f8a827ebeed6de server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:45.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.974+0000 7fb834b2e700 1 -- 172.21.15.112:0/3922606449 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82800e070 con 0x7fb8300cd270 2023-08-27T11:28:45.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.974+0000 7fb834b2e700 1 -- 172.21.15.112:0/3922606449 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb828003a20 con 0x7fb8300cd270 2023-08-27T11:28:45.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.974+0000 7fb834b2e700 1 -- 172.21.15.112:0/3922606449 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828003dd0 con 0x7fb8300cd270 2023-08-27T11:28:45.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.975+0000 7fb837d94700 1 -- 172.21.15.112:0/3922606449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8300cd270 msgr2=0x7fb8300cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.975+0000 7fb837d94700 1 --2- 172.21.15.112:0/3922606449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8300cd270 0x7fb8300cd6a0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fb83006ee60 tx=0x7fb82800b330 comp rx=0 tx=0).stop 2023-08-27T11:28:45.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.975+0000 7fb837d94700 1 -- 172.21.15.112:0/3922606449 shutdown_connections 2023-08-27T11:28:45.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.975+0000 7fb837d94700 1 --2- 172.21.15.112:0/3922606449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8300cd270 0x7fb8300cd6a0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.975+0000 7fb837d94700 1 --2- 172.21.15.112:0/3922606449 >> v2:172.21.15.138:3300/0 conn(0x7fb83013f990 0x7fb8300cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.975+0000 7fb837d94700 1 -- 172.21.15.112:0/3922606449 >> 172.21.15.112:0/3922606449 conn(0x7fb83006cd30 msgr2=0x7fb8300717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:45.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.975+0000 7fb837d94700 1 -- 172.21.15.112:0/3922606449 shutdown_connections 2023-08-27T11:28:45.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.975+0000 7fb837d94700 1 -- 172.21.15.112:0/3922606449 wait complete. 2023-08-27T11:28:45.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.976+0000 7fb837d94700 1 Processor -- start 2023-08-27T11:28:45.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.976+0000 7fb837d94700 1 -- start start 2023-08-27T11:28:45.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb837d94700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb83006e150 0x7fb8300b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb837d94700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8300cd270 0x7fb8300ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb837d94700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb83013f990 0x7fb83014e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb837d94700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300acd80 con 0x7fb8300cd270 2023-08-27T11:28:45.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb837d94700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83014e810 con 0x7fb83013f990 2023-08-27T11:28:45.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb837d94700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb830070f80 con 0x7fb83006e150 2023-08-27T11:28:45.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb83532f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8300cd270 0x7fb8300ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb83532f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8300cd270 0x7fb8300ac840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36936/0 (socket says 172.21.15.112:36936) 2023-08-27T11:28:45.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb83532f700 1 -- 172.21.15.112:0/921918232 learned_addr learned my addr 172.21.15.112:0/921918232 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:45.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb836331700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb83013f990 0x7fb83014e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb835b30700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb83006e150 0x7fb8300b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb83532f700 1 -- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb83006e150 msgr2=0x7fb8300b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb83532f700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb83006e150 0x7fb8300b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb83532f700 1 -- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb83013f990 msgr2=0x7fb83014e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb83532f700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb83013f990 0x7fb83014e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb83532f700 1 -- 172.21.15.112:0/921918232 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb828009b30 con 0x7fb8300cd270 2023-08-27T11:28:45.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb83532f700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8300cd270 0x7fb8300ac840 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00c7f0 tx=0x7fb82c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:45.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb8267fc700 1 -- 172.21.15.112:0/921918232 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c005c20 con 0x7fb8300cd270 2023-08-27T11:28:45.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb8267fc700 1 -- 172.21.15.112:0/921918232 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c005d80 con 0x7fb8300cd270 2023-08-27T11:28:45.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb837d94700 1 -- 172.21.15.112:0/921918232 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb830070650 con 0x7fb8300cd270 2023-08-27T11:28:45.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb8267fc700 1 -- 172.21.15.112:0/921918232 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c011820 con 0x7fb8300cd270 2023-08-27T11:28:45.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.977+0000 7fb837d94700 1 -- 172.21.15.112:0/921918232 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb830070c10 con 0x7fb8300cd270 2023-08-27T11:28:45.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.978+0000 7fb837d94700 1 -- 172.21.15.112:0/921918232 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb830067a20 con 0x7fb8300cd270 2023-08-27T11:28:45.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.979+0000 7fb8267fc700 1 -- 172.21.15.112:0/921918232 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fb82c011980 con 0x7fb8300cd270 2023-08-27T11:28:45.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.979+0000 7fb8267fc700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb81406a4d0 0x7fb81406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.979+0000 7fb8267fc700 1 -- 172.21.15.112:0/921918232 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb82c088600 con 0x7fb8300cd270 2023-08-27T11:28:45.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.979+0000 7fb835b30700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb81406a4d0 0x7fb81406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.980+0000 7fb835b30700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb81406a4d0 0x7fb81406c990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb820007fd0 tx=0x7fb820007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:45.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:45.983+0000 7fb8267fc700 1 -- 172.21.15.112:0/921918232 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb82c057470 con 0x7fb8300cd270 2023-08-27T11:28:46.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.174+0000 7fb837d94700 1 -- 172.21.15.112:0/921918232 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8300a8c70 con 0x7fb8300cd270 2023-08-27T11:28:46.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.175+0000 7fb8267fc700 1 -- 172.21.15.112:0/921918232 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+14615 (secure 0 0 0) 0x7fb82c057000 con 0x7fb8300cd270 2023-08-27T11:28:46.176 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:46.176 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":149,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:45.398909+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:46.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.177+0000 7fb837d94700 1 -- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb81406a4d0 msgr2=0x7fb81406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.177+0000 7fb837d94700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb81406a4d0 0x7fb81406c990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb820007fd0 tx=0x7fb820007ec0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.177+0000 7fb837d94700 1 -- 172.21.15.112:0/921918232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8300cd270 msgr2=0x7fb8300ac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.177+0000 7fb837d94700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8300cd270 0x7fb8300ac840 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00c7f0 tx=0x7fb82c00cb00 comp rx=0 tx=0).stop 2023-08-27T11:28:46.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.177+0000 7fb837d94700 1 -- 172.21.15.112:0/921918232 shutdown_connections 2023-08-27T11:28:46.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.177+0000 7fb837d94700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb81406a4d0 0x7fb81406c990 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.177+0000 7fb837d94700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb83013f990 0x7fb83014e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.177+0000 7fb837d94700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb83006e150 0x7fb8300b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.177+0000 7fb837d94700 1 --2- 172.21.15.112:0/921918232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8300cd270 0x7fb8300ac840 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.177+0000 7fb837d94700 1 -- 172.21.15.112:0/921918232 >> 172.21.15.112:0/921918232 conn(0x7fb83006cd30 msgr2=0x7fb8300aa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:46.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.177+0000 7fb837d94700 1 -- 172.21.15.112:0/921918232 shutdown_connections 2023-08-27T11:28:46.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.177+0000 7fb837d94700 1 -- 172.21.15.112:0/921918232 wait complete. 2023-08-27T11:28:46.191 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:28:46.191 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:28:46.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.312+0000 7fe8dba15700 1 Processor -- start 2023-08-27T11:28:46.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.313+0000 7fe8dba15700 1 -- start start 2023-08-27T11:28:46.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.313+0000 7fe8dba15700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8d40b3440 0x7fe8d415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.313+0000 7fe8dba15700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d40b5070 con 0x7fe8d4146c20 2023-08-27T11:28:46.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.313+0000 7fe8dba15700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d4147000 con 0x7fe8d415e6e0 2023-08-27T11:28:46.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.313+0000 7fe8dba15700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4161740 con 0x7fe8d40b3440 2023-08-27T11:28:46.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8d97b1700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe8d4146c20 0x7fe8d4160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59556/0 (socket says 172.21.15.112:59556) 2023-08-27T11:28:46.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8d97b1700 1 -- 172.21.15.112:0/3184304236 learned_addr learned my addr 172.21.15.112:0/3184304236 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:46.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8d8fb0700 1 --2- 172.21.15.112:0/3184304236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8d40b3440 0x7fe8d415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8cbfff700 1 -- 172.21.15.112:0/3184304236 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448884042 0 0) 0x7fe8c4001110 con 0x7fe8d4146c20 2023-08-27T11:28:46.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8cbfff700 1 -- 172.21.15.112:0/3184304236 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8c00035b0 con 0x7fe8d4146c20 2023-08-27T11:28:46.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8cbfff700 1 -- 172.21.15.112:0/3184304236 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919665661 0 0) 0x7fe8d4147000 con 0x7fe8d415e6e0 2023-08-27T11:28:46.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8cbfff700 1 -- 172.21.15.112:0/3184304236 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8c4001110 con 0x7fe8d415e6e0 2023-08-27T11:28:46.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8d8fb0700 1 -- 172.21.15.112:0/3184304236 >> v1:172.21.15.138:6789/0 conn(0x7fe8d415e6e0 legacy=0x7fe8d406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8d8fb0700 1 -- 172.21.15.112:0/3184304236 >> v1:172.21.15.112:6789/0 conn(0x7fe8d4146c20 legacy=0x7fe8d4160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8d8fb0700 1 -- 172.21.15.112:0/3184304236 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d4161a20 con 0x7fe8d40b3440 2023-08-27T11:28:46.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8d8fb0700 1 --2- 172.21.15.112:0/3184304236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8d40b3440 0x7fe8d415e1a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009870 tx=0x7fe8d0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b3d50d6401885b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8cbfff700 1 -- 172.21.15.112:0/3184304236 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0011830 con 0x7fe8d40b3440 2023-08-27T11:28:46.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8cbfff700 1 -- 172.21.15.112:0/3184304236 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d0011990 con 0x7fe8d40b3440 2023-08-27T11:28:46.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.314+0000 7fe8cbfff700 1 -- 172.21.15.112:0/3184304236 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0011d00 con 0x7fe8d40b3440 2023-08-27T11:28:46.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.315+0000 7fe8dba15700 1 -- 172.21.15.112:0/3184304236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8d40b3440 msgr2=0x7fe8d415e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.315+0000 7fe8dba15700 1 --2- 172.21.15.112:0/3184304236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8d40b3440 0x7fe8d415e1a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009870 tx=0x7fe8d0009b80 comp rx=0 tx=0).stop 2023-08-27T11:28:46.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.315+0000 7fe8dba15700 1 -- 172.21.15.112:0/3184304236 shutdown_connections 2023-08-27T11:28:46.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.315+0000 7fe8dba15700 1 --2- 172.21.15.112:0/3184304236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8d40b3440 0x7fe8d415e1a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.315+0000 7fe8dba15700 1 -- 172.21.15.112:0/3184304236 >> 172.21.15.112:0/3184304236 conn(0x7fe8d40b1e90 msgr2=0x7fe8d40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:46.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.315+0000 7fe8dba15700 1 -- 172.21.15.112:0/3184304236 shutdown_connections 2023-08-27T11:28:46.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.315+0000 7fe8dba15700 1 -- 172.21.15.112:0/3184304236 wait complete. 2023-08-27T11:28:46.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.316+0000 7fe8dba15700 1 Processor -- start 2023-08-27T11:28:46.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.316+0000 7fe8dba15700 1 -- start start 2023-08-27T11:28:46.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8dba15700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8d40b3440 0x7fe8d4140f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8dba15700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d4146c20 0x7fe8d4145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8dba15700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8d415e6e0 0x7fe8d41457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8dba15700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d415d450 con 0x7fe8d415e6e0 2023-08-27T11:28:46.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8dba15700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d415c840 con 0x7fe8d4146c20 2023-08-27T11:28:46.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8dba15700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d415cb20 con 0x7fe8d40b3440 2023-08-27T11:28:46.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8d9fb2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8d415e6e0 0x7fe8d41457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8d9fb2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8d415e6e0 0x7fe8d41457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36952/0 (socket says 172.21.15.112:36952) 2023-08-27T11:28:46.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8d9fb2700 1 -- 172.21.15.112:0/1928720742 learned_addr learned my addr 172.21.15.112:0/1928720742 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:46.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8d97b1700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8d40b3440 0x7fe8d4140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8d8fb0700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d4146c20 0x7fe8d4145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8d9fb2700 1 -- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8d40b3440 msgr2=0x7fe8d4140f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8d9fb2700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8d40b3440 0x7fe8d4140f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8d9fb2700 1 -- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d4146c20 msgr2=0x7fe8d4145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8d9fb2700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d4146c20 0x7fe8d4145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.317+0000 7fe8d9fb2700 1 -- 172.21.15.112:0/1928720742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d00094a0 con 0x7fe8d415e6e0 2023-08-27T11:28:46.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.318+0000 7fe8d9fb2700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8d415e6e0 0x7fe8d41457a0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00cbc0 tx=0x7fe8cc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.318+0000 7fe8c9ffb700 1 -- 172.21.15.112:0/1928720742 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc003c00 con 0x7fe8d415e6e0 2023-08-27T11:28:46.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.318+0000 7fe8c9ffb700 1 -- 172.21.15.112:0/1928720742 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc003d60 con 0x7fe8d415e6e0 2023-08-27T11:28:46.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.318+0000 7fe8dba15700 1 -- 172.21.15.112:0/1928720742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d415ce00 con 0x7fe8d415e6e0 2023-08-27T11:28:46.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.318+0000 7fe8c9ffb700 1 -- 172.21.15.112:0/1928720742 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00aaa0 con 0x7fe8d415e6e0 2023-08-27T11:28:46.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.318+0000 7fe8dba15700 1 -- 172.21.15.112:0/1928720742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d40ad4c0 con 0x7fe8d415e6e0 2023-08-27T11:28:46.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.319+0000 7fe8dba15700 1 -- 172.21.15.112:0/1928720742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d4067e90 con 0x7fe8d415e6e0 2023-08-27T11:28:46.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.319+0000 7fe8c9ffb700 1 -- 172.21.15.112:0/1928720742 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe8cc00ac00 con 0x7fe8d415e6e0 2023-08-27T11:28:46.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.319+0000 7fe8c9ffb700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe8b806a4d0 0x7fe8b806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.320+0000 7fe8c9ffb700 1 -- 172.21.15.112:0/1928720742 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe8cc088910 con 0x7fe8d415e6e0 2023-08-27T11:28:46.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.320+0000 7fe8d97b1700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe8b806a4d0 0x7fe8b806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.320+0000 7fe8d97b1700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe8b806a4d0 0x7fe8b806c990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4007fd0 tx=0x7fe8c4007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.324+0000 7fe8c9ffb700 1 -- 172.21.15.112:0/1928720742 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe8cc053dc0 con 0x7fe8d415e6e0 2023-08-27T11:28:46.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.516+0000 7fe8dba15700 1 -- 172.21.15.112:0/1928720742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe8d4067a20 con 0x7fe8d415e6e0 2023-08-27T11:28:46.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.516+0000 7fe8c9ffb700 1 -- 172.21.15.112:0/1928720742 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v149) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe8cc0573e0 con 0x7fe8d415e6e0 2023-08-27T11:28:46.518 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:28:46.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.518+0000 7fe8dba15700 1 -- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe8b806a4d0 msgr2=0x7fe8b806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.518+0000 7fe8dba15700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe8b806a4d0 0x7fe8b806c990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4007fd0 tx=0x7fe8c4007dc0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.518+0000 7fe8dba15700 1 -- 172.21.15.112:0/1928720742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8d415e6e0 msgr2=0x7fe8d41457a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.518+0000 7fe8dba15700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8d415e6e0 0x7fe8d41457a0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00cbc0 tx=0x7fe8cc00ced0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.518+0000 7fe8dba15700 1 -- 172.21.15.112:0/1928720742 shutdown_connections 2023-08-27T11:28:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.518+0000 7fe8dba15700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe8b806a4d0 0x7fe8b806c990 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.518+0000 7fe8dba15700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8d415e6e0 0x7fe8d41457a0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.518+0000 7fe8dba15700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8d40b3440 0x7fe8d4140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.518+0000 7fe8dba15700 1 --2- 172.21.15.112:0/1928720742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d4146c20 0x7fe8d4145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.518+0000 7fe8dba15700 1 -- 172.21.15.112:0/1928720742 >> 172.21.15.112:0/1928720742 conn(0x7fe8d40b1e90 msgr2=0x7fe8d406d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.518+0000 7fe8dba15700 1 -- 172.21.15.112:0/1928720742 shutdown_connections 2023-08-27T11:28:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.518+0000 7fe8dba15700 1 -- 172.21.15.112:0/1928720742 wait complete. 2023-08-27T11:28:46.532 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:28:46.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.654+0000 7f8f4ade9700 1 Processor -- start 2023-08-27T11:28:46.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.654+0000 7f8f4ade9700 1 -- start start 2023-08-27T11:28:46.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f4ade9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4415c930 0x7f8f4415cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f4ade9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8f440b6930 0x7f8f440b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f4ade9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f440b64a0 con 0x7f8f44158da0 2023-08-27T11:28:46.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f4ade9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f44159180 con 0x7f8f440b6930 2023-08-27T11:28:46.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f4ade9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4415f820 con 0x7f8f4415c930 2023-08-27T11:28:46.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f49386700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8f440b6930 0x7f8f440b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f48b85700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8f44158da0 0x7f8f4415ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59572/0 (socket says 172.21.15.112:59572) 2023-08-27T11:28:46.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f48b85700 1 -- 172.21.15.112:0/2682562384 learned_addr learned my addr 172.21.15.112:0/2682562384 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:46.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f43fff700 1 --2- 172.21.15.112:0/2682562384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4415c930 0x7f8f4415cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f49386700 1 -- 172.21.15.112:0/2682562384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4415c930 msgr2=0x7f8f4415cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:46.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f49386700 1 --2- 172.21.15.112:0/2682562384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4415c930 0x7f8f4415cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f49386700 1 -- 172.21.15.112:0/2682562384 >> v1:172.21.15.112:6789/0 conn(0x7f8f44158da0 legacy=0x7f8f4415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f49386700 1 -- 172.21.15.112:0/2682562384 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4415fb00 con 0x7f8f440b6930 2023-08-27T11:28:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f437fe700 1 -- 172.21.15.112:0/2682562384 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229712193 0 0) 0x7f8f440b64a0 con 0x7f8f44158da0 2023-08-27T11:28:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f43fff700 1 --2- 172.21.15.112:0/2682562384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4415c930 0x7f8f4415cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f49386700 1 --2- 172.21.15.112:0/2682562384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f440b6930 0x7f8f440b6dc0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009d90 tx=0x7f8f3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7daf53548c545ebd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f437fe700 1 -- 172.21.15.112:0/2682562384 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00e070 con 0x7f8f440b6930 2023-08-27T11:28:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f437fe700 1 -- 172.21.15.112:0/2682562384 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c00bbd0 con 0x7f8f440b6930 2023-08-27T11:28:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.656+0000 7f8f437fe700 1 -- 172.21.15.112:0/2682562384 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c012430 con 0x7f8f440b6930 2023-08-27T11:28:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.657+0000 7f8f4ade9700 1 -- 172.21.15.112:0/2682562384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f440b6930 msgr2=0x7f8f440b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.657+0000 7f8f4ade9700 1 --2- 172.21.15.112:0/2682562384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f440b6930 0x7f8f440b6dc0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009d90 tx=0x7f8f3c00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.657+0000 7f8f4ade9700 1 -- 172.21.15.112:0/2682562384 shutdown_connections 2023-08-27T11:28:46.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.657+0000 7f8f4ade9700 1 --2- 172.21.15.112:0/2682562384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f440b6930 0x7f8f440b6dc0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.657+0000 7f8f4ade9700 1 --2- 172.21.15.112:0/2682562384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4415c930 0x7f8f4415cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.657+0000 7f8f4ade9700 1 -- 172.21.15.112:0/2682562384 >> 172.21.15.112:0/2682562384 conn(0x7f8f44066010 msgr2=0x7f8f440663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:46.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.657+0000 7f8f4ade9700 1 -- 172.21.15.112:0/2682562384 shutdown_connections 2023-08-27T11:28:46.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.657+0000 7f8f4ade9700 1 -- 172.21.15.112:0/2682562384 wait complete. 2023-08-27T11:28:46.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.658+0000 7f8f4ade9700 1 Processor -- start 2023-08-27T11:28:46.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.658+0000 7f8f4ade9700 1 -- start start 2023-08-27T11:28:46.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f4ade9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f440b6930 0x7f8f4414d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f4ade9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f44158da0 0x7f8f4414cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f4ade9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4415c930 0x7f8f440b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f4ade9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44155210 con 0x7f8f440b6930 2023-08-27T11:28:46.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f4ade9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4415bce0 con 0x7f8f44158da0 2023-08-27T11:28:46.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f4ade9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4414dcd0 con 0x7f8f4415c930 2023-08-27T11:28:46.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f48b85700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f440b6930 0x7f8f4414d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f48b85700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f440b6930 0x7f8f4414d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36964/0 (socket says 172.21.15.112:36964) 2023-08-27T11:28:46.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f48b85700 1 -- 172.21.15.112:0/341316556 learned_addr learned my addr 172.21.15.112:0/341316556 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:46.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f49386700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4415c930 0x7f8f440b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f43fff700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f44158da0 0x7f8f4414cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f48b85700 1 -- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4415c930 msgr2=0x7f8f440b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f48b85700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4415c930 0x7f8f440b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f48b85700 1 -- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f44158da0 msgr2=0x7f8f4414cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f48b85700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f44158da0 0x7f8f4414cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f48b85700 1 -- 172.21.15.112:0/341316556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c009a70 con 0x7f8f440b6930 2023-08-27T11:28:46.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f49386700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4415c930 0x7f8f440b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:46.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.659+0000 7f8f43fff700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f44158da0 0x7f8f4414cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:46.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.660+0000 7f8f48b85700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f440b6930 0x7f8f4414d640 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f8f380011e0 tx=0x7f8f3800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.660+0000 7f8f417fa700 1 -- 172.21.15.112:0/341316556 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38004c80 con 0x7f8f440b6930 2023-08-27T11:28:46.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.660+0000 7f8f417fa700 1 -- 172.21.15.112:0/341316556 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f38004e00 con 0x7f8f440b6930 2023-08-27T11:28:46.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.660+0000 7f8f4ade9700 1 -- 172.21.15.112:0/341316556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4414dfb0 con 0x7f8f440b6930 2023-08-27T11:28:46.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.660+0000 7f8f417fa700 1 -- 172.21.15.112:0/341316556 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38007310 con 0x7f8f440b6930 2023-08-27T11:28:46.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.660+0000 7f8f4ade9700 1 -- 172.21.15.112:0/341316556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4414e570 con 0x7f8f440b6930 2023-08-27T11:28:46.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.661+0000 7f8f4ade9700 1 -- 172.21.15.112:0/341316556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f44155410 con 0x7f8f440b6930 2023-08-27T11:28:46.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.661+0000 7f8f417fa700 1 -- 172.21.15.112:0/341316556 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f8f38007470 con 0x7f8f440b6930 2023-08-27T11:28:46.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.661+0000 7f8f417fa700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f2806a4d0 0x7f8f2806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.661+0000 7f8f417fa700 1 -- 172.21.15.112:0/341316556 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8f38088660 con 0x7f8f440b6930 2023-08-27T11:28:46.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.662+0000 7f8f43fff700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f2806a4d0 0x7f8f2806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.662+0000 7f8f43fff700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f2806a4d0 0x7f8f2806c990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8f441f35d0 tx=0x7f8f30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.666+0000 7f8f417fa700 1 -- 172.21.15.112:0/341316556 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8f38053b40 con 0x7f8f440b6930 2023-08-27T11:28:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.852+0000 7f8f4ade9700 1 -- 172.21.15.112:0/341316556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f4414e760 con 0x7f8f440b6930 2023-08-27T11:28:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.853+0000 7f8f417fa700 1 -- 172.21.15.112:0/341316556 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f38057160 con 0x7f8f440b6930 2023-08-27T11:28:46.854 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:28:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.854+0000 7f8f4ade9700 1 -- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f2806a4d0 msgr2=0x7f8f2806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.854+0000 7f8f4ade9700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f2806a4d0 0x7f8f2806c990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8f441f35d0 tx=0x7f8f30008040 comp rx=0 tx=0).stop 2023-08-27T11:28:46.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.854+0000 7f8f4ade9700 1 -- 172.21.15.112:0/341316556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f440b6930 msgr2=0x7f8f4414d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.854+0000 7f8f4ade9700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f440b6930 0x7f8f4414d640 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f8f380011e0 tx=0x7f8f3800cae0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.855+0000 7f8f4ade9700 1 -- 172.21.15.112:0/341316556 shutdown_connections 2023-08-27T11:28:46.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.855+0000 7f8f4ade9700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f2806a4d0 0x7f8f2806c990 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.855+0000 7f8f4ade9700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f4415c930 0x7f8f440b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.855+0000 7f8f4ade9700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f440b6930 0x7f8f4414d640 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.855+0000 7f8f4ade9700 1 --2- 172.21.15.112:0/341316556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f44158da0 0x7f8f4414cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.855+0000 7f8f4ade9700 1 -- 172.21.15.112:0/341316556 >> 172.21.15.112:0/341316556 conn(0x7f8f44066010 msgr2=0x7f8f441538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.855+0000 7f8f4ade9700 1 -- 172.21.15.112:0/341316556 shutdown_connections 2023-08-27T11:28:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.855+0000 7f8f4ade9700 1 -- 172.21.15.112:0/341316556 wait complete. 2023-08-27T11:28:46.868 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:28:46.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.986+0000 7f543bde2700 1 Processor -- start 2023-08-27T11:28:46.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.986+0000 7f543bde2700 1 -- start start 2023-08-27T11:28:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.987+0000 7f543bde2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5434144a40 0x7f54340cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.987+0000 7f543bde2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f54340cc8e0 0x7f54340cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.987+0000 7f543bde2700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543415f3d0 con 0x7f5434141b60 2023-08-27T11:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.987+0000 7f543bde2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54340ccd50 con 0x7f54340cc8e0 2023-08-27T11:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.987+0000 7f543bde2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543415f6b0 con 0x7f5434144a40 2023-08-27T11:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.988+0000 7f543a37f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f54340cc8e0 0x7f54340cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.988+0000 7f543937d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5434144a40 0x7f54340cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.988+0000 7f5439b7e700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5434141b60 0x7f54340cb6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:38878/0 (socket says 172.21.15.112:38878) 2023-08-27T11:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.988+0000 7f5439b7e700 1 -- 172.21.15.112:0/297009459 learned_addr learned my addr 172.21.15.112:0/297009459 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.988+0000 7f543a37f700 1 -- 172.21.15.112:0/297009459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5434144a40 msgr2=0x7f54340cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.988+0000 7f543a37f700 1 --2- 172.21.15.112:0/297009459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5434144a40 0x7f54340cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.988+0000 7f543a37f700 1 -- 172.21.15.112:0/297009459 >> v1:172.21.15.138:6789/0 conn(0x7f5434141b60 legacy=0x7f54340cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.988+0000 7f543a37f700 1 -- 172.21.15.112:0/297009459 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543415f990 con 0x7f54340cc8e0 2023-08-27T11:28:46.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.988+0000 7f543937d700 1 --2- 172.21.15.112:0/297009459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5434144a40 0x7f54340cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:46.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.988+0000 7f543a37f700 1 --2- 172.21.15.112:0/297009459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54340cc8e0 0x7f54340cf410 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5424009840 tx=0x7f5424009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e97c86a8382cff server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.988+0000 7f5438b7c700 1 -- 172.21.15.112:0/297009459 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424011830 con 0x7f54340cc8e0 2023-08-27T11:28:46.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.988+0000 7f5438b7c700 1 -- 172.21.15.112:0/297009459 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5424011990 con 0x7f54340cc8e0 2023-08-27T11:28:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.988+0000 7f5438b7c700 1 -- 172.21.15.112:0/297009459 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424011d20 con 0x7f54340cc8e0 2023-08-27T11:28:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.989+0000 7f543bde2700 1 -- 172.21.15.112:0/297009459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54340cc8e0 msgr2=0x7f54340cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.989+0000 7f543bde2700 1 --2- 172.21.15.112:0/297009459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54340cc8e0 0x7f54340cf410 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5424009840 tx=0x7f5424009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.989+0000 7f543bde2700 1 -- 172.21.15.112:0/297009459 shutdown_connections 2023-08-27T11:28:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.989+0000 7f543bde2700 1 --2- 172.21.15.112:0/297009459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54340cc8e0 0x7f54340cf410 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.989+0000 7f543bde2700 1 --2- 172.21.15.112:0/297009459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5434144a40 0x7f54340cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.989+0000 7f543bde2700 1 -- 172.21.15.112:0/297009459 >> 172.21.15.112:0/297009459 conn(0x7f5434066010 msgr2=0x7f54340663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:46.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.989+0000 7f543bde2700 1 -- 172.21.15.112:0/297009459 shutdown_connections 2023-08-27T11:28:46.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.989+0000 7f543bde2700 1 -- 172.21.15.112:0/297009459 wait complete. 2023-08-27T11:28:46.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.990+0000 7f543bde2700 1 Processor -- start 2023-08-27T11:28:47.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.990+0000 7f543bde2700 1 -- start start 2023-08-27T11:28:47.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f543bde2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54340cc8e0 0x7f54340b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f543bde2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5434141b60 0x7f543413ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f543bde2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5434144a40 0x7f54340c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f543bde2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54340c73c0 con 0x7f54340cc8e0 2023-08-27T11:28:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f543bde2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54340b5bb0 con 0x7f5434141b60 2023-08-27T11:28:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f543bde2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54340b6730 con 0x7f5434144a40 2023-08-27T11:28:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f5439b7e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54340cc8e0 0x7f54340b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f5439b7e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54340cc8e0 0x7f54340b6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36978/0 (socket says 172.21.15.112:36978) 2023-08-27T11:28:47.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f5439b7e700 1 -- 172.21.15.112:0/444662679 learned_addr learned my addr 172.21.15.112:0/444662679 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:47.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f543a37f700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5434144a40 0x7f54340c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f543937d700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5434141b60 0x7f543413ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f5439b7e700 1 -- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5434144a40 msgr2=0x7f54340c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f5439b7e700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5434144a40 0x7f54340c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f5439b7e700 1 -- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5434141b60 msgr2=0x7f543413ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f5439b7e700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5434141b60 0x7f543413ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f5439b7e700 1 -- 172.21.15.112:0/444662679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54280094a0 con 0x7f54340cc8e0 2023-08-27T11:28:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f543937d700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5434141b60 0x7f543413ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f543a37f700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5434144a40 0x7f54340c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f5439b7e700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54340cc8e0 0x7f54340b6bc0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f542c00cbf0 tx=0x7f542c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f54327fc700 1 -- 172.21.15.112:0/444662679 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c00ab20 con 0x7f54340cc8e0 2023-08-27T11:28:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f54327fc700 1 -- 172.21.15.112:0/444662679 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c00ac80 con 0x7f54340cc8e0 2023-08-27T11:28:47.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f543bde2700 1 -- 172.21.15.112:0/444662679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54240094a0 con 0x7f54340cc8e0 2023-08-27T11:28:47.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.991+0000 7f54327fc700 1 -- 172.21.15.112:0/444662679 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c012400 con 0x7f54340cc8e0 2023-08-27T11:28:47.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.992+0000 7f543bde2700 1 -- 172.21.15.112:0/444662679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54340b63c0 con 0x7f54340cc8e0 2023-08-27T11:28:47.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.992+0000 7f543bde2700 1 -- 172.21.15.112:0/444662679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54340b6a10 con 0x7f54340cc8e0 2023-08-27T11:28:47.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.993+0000 7f54327fc700 1 -- 172.21.15.112:0/444662679 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f542c01b3b0 con 0x7f54340cc8e0 2023-08-27T11:28:47.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.993+0000 7f54327fc700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f542006a4d0 0x7f542006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.993+0000 7f54327fc700 1 -- 172.21.15.112:0/444662679 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f542c088520 con 0x7f54340cc8e0 2023-08-27T11:28:47.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.994+0000 7f543937d700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f542006a4d0 0x7f542006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.994+0000 7f543937d700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f542006a4d0 0x7f542006c990 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f543415e9a0 tx=0x7f5428008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:47.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:46.998+0000 7f54327fc700 1 -- 172.21.15.112:0/444662679 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f542c053ab0 con 0x7f54340cc8e0 2023-08-27T11:28:47.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.190+0000 7f543bde2700 1 -- 172.21.15.112:0/444662679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5434067a20 con 0x7f54340cc8e0 2023-08-27T11:28:47.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.190+0000 7f54327fc700 1 -- 172.21.15.112:0/444662679 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v149) v1 ==== 84+0+126 (secure 0 0 0) 0x7f542c0570d0 con 0x7f54340cc8e0 2023-08-27T11:28:47.192 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:47.192 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:28:47.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:47.193 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2023-08-27T11:28:47.193 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "rbd" 2023-08-27T11:28:47.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:28:47.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:47.194 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 2, 2023-08-27T11:28:47.194 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2023-08-27T11:28:47.194 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:47.195 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:28:47.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.192+0000 7f543bde2700 1 -- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f542006a4d0 msgr2=0x7f542006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.192+0000 7f543bde2700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f542006a4d0 0x7f542006c990 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f543415e9a0 tx=0x7f5428008040 comp rx=0 tx=0).stop 2023-08-27T11:28:47.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.192+0000 7f543bde2700 1 -- 172.21.15.112:0/444662679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54340cc8e0 msgr2=0x7f54340b6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.192+0000 7f543bde2700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54340cc8e0 0x7f54340b6bc0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f542c00cbf0 tx=0x7f542c00a060 comp rx=0 tx=0).stop 2023-08-27T11:28:47.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.192+0000 7f543bde2700 1 -- 172.21.15.112:0/444662679 shutdown_connections 2023-08-27T11:28:47.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.192+0000 7f543bde2700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f542006a4d0 0x7f542006c990 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.192+0000 7f543bde2700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5434144a40 0x7f54340c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.192+0000 7f543bde2700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54340cc8e0 0x7f54340b6bc0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.192+0000 7f543bde2700 1 --2- 172.21.15.112:0/444662679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5434141b60 0x7f543413ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.192+0000 7f543bde2700 1 -- 172.21.15.112:0/444662679 >> 172.21.15.112:0/444662679 conn(0x7f5434066010 msgr2=0x7f54340b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:47.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.192+0000 7f543bde2700 1 -- 172.21.15.112:0/444662679 shutdown_connections 2023-08-27T11:28:47.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.192+0000 7f543bde2700 1 -- 172.21.15.112:0/444662679 wait complete. 2023-08-27T11:28:47.205 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:28:47.206 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:28:47.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.329+0000 7f315c414700 1 Processor -- start 2023-08-27T11:28:47.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.329+0000 7f315c414700 1 -- start start 2023-08-27T11:28:47.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f315c414700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31540b3de0 0x7f315415e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f315c414700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f315406f790 0x7f315406fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f315c414700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3154161390 con 0x7f3154157ae0 2023-08-27T11:28:47.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f315c414700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31540b1aa0 con 0x7f315406f790 2023-08-27T11:28:47.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f315c414700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3154070160 con 0x7f31540b3de0 2023-08-27T11:28:47.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f315a9b1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f315406f790 0x7f315406fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f31599af700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31540b3de0 0x7f315415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f315a1b0700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3154157ae0 0x7f3154161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:38892/0 (socket says 172.21.15.112:38892) 2023-08-27T11:28:47.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f315a1b0700 1 -- 172.21.15.112:0/3947938075 learned_addr learned my addr 172.21.15.112:0/3947938075 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:47.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f31591ae700 1 -- 172.21.15.112:0/3947938075 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913382826 0 0) 0x7f3154161390 con 0x7f3154157ae0 2023-08-27T11:28:47.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f31591ae700 1 -- 172.21.15.112:0/3947938075 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31400035b0 con 0x7f3154157ae0 2023-08-27T11:28:47.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f315a9b1700 1 -- 172.21.15.112:0/3947938075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31540b3de0 msgr2=0x7f315415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:47.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f315a9b1700 1 --2- 172.21.15.112:0/3947938075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31540b3de0 0x7f315415e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f315a9b1700 1 -- 172.21.15.112:0/3947938075 >> v1:172.21.15.138:6789/0 conn(0x7f3154157ae0 legacy=0x7f3154161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.330+0000 7f315a9b1700 1 -- 172.21.15.112:0/3947938075 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31540b35f0 con 0x7f315406f790 2023-08-27T11:28:47.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.331+0000 7f315a9b1700 1 --2- 172.21.15.112:0/3947938075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f315406f790 0x7f315406fc20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f314c009d90 tx=0x7f314c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bab0632439bb22bd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:47.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.331+0000 7f31591ae700 1 -- 172.21.15.112:0/3947938075 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c00e070 con 0x7f315406f790 2023-08-27T11:28:47.336 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.331+0000 7f31591ae700 1 -- 172.21.15.112:0/3947938075 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c00bb20 con 0x7f315406f790 2023-08-27T11:28:47.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.331+0000 7f31591ae700 1 -- 172.21.15.112:0/3947938075 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c012430 con 0x7f315406f790 2023-08-27T11:28:47.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.331+0000 7f315c414700 1 -- 172.21.15.112:0/3947938075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f315406f790 msgr2=0x7f315406fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.331+0000 7f315c414700 1 --2- 172.21.15.112:0/3947938075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f315406f790 0x7f315406fc20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f314c009d90 tx=0x7f314c00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:47.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.331+0000 7f315c414700 1 -- 172.21.15.112:0/3947938075 shutdown_connections 2023-08-27T11:28:47.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.331+0000 7f315c414700 1 --2- 172.21.15.112:0/3947938075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f315406f790 0x7f315406fc20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.331+0000 7f315c414700 1 --2- 172.21.15.112:0/3947938075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31540b3de0 0x7f315415e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.331+0000 7f315c414700 1 -- 172.21.15.112:0/3947938075 >> 172.21.15.112:0/3947938075 conn(0x7f31540b1be0 msgr2=0x7f31540b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:47.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.331+0000 7f315c414700 1 -- 172.21.15.112:0/3947938075 shutdown_connections 2023-08-27T11:28:47.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.331+0000 7f315c414700 1 -- 172.21.15.112:0/3947938075 wait complete. 2023-08-27T11:28:47.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.332+0000 7f315c414700 1 Processor -- start 2023-08-27T11:28:47.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.333+0000 7f315c414700 1 -- start start 2023-08-27T11:28:47.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.333+0000 7f315c414700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f315406f790 0x7f315413ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.333+0000 7f315c414700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31540b3de0 0x7f315415f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.333+0000 7f315c414700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3154157ae0 0x7f3154141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.333+0000 7f315c414700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3154160260 con 0x7f3154157ae0 2023-08-27T11:28:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.333+0000 7f315c414700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31540af0e0 con 0x7f31540b3de0 2023-08-27T11:28:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.333+0000 7f315c414700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315413f230 con 0x7f315406f790 2023-08-27T11:28:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.333+0000 7f315a9b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3154157ae0 0x7f3154141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.333+0000 7f315a9b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3154157ae0 0x7f3154141470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36996/0 (socket says 172.21.15.112:36996) 2023-08-27T11:28:47.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.333+0000 7f315a9b1700 1 -- 172.21.15.112:0/3983057566 learned_addr learned my addr 172.21.15.112:0/3983057566 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:47.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.333+0000 7f315a1b0700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f315406f790 0x7f315413ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.334+0000 7f31599af700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31540b3de0 0x7f315415f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.334+0000 7f315a9b1700 1 -- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f315406f790 msgr2=0x7f315413ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.334+0000 7f315a9b1700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f315406f790 0x7f315413ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.334+0000 7f315a9b1700 1 -- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31540b3de0 msgr2=0x7f315415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.334+0000 7f315a9b1700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31540b3de0 0x7f315415f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.334+0000 7f315a9b1700 1 -- 172.21.15.112:0/3983057566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c009a70 con 0x7f3154157ae0 2023-08-27T11:28:47.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.334+0000 7f315a9b1700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3154157ae0 0x7f3154141470 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f315406e930 tx=0x7f314c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:47.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.334+0000 7f314affd700 1 -- 172.21.15.112:0/3983057566 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c012430 con 0x7f3154157ae0 2023-08-27T11:28:47.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.334+0000 7f314affd700 1 -- 172.21.15.112:0/3983057566 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c020070 con 0x7f3154157ae0 2023-08-27T11:28:47.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.334+0000 7f315c414700 1 -- 172.21.15.112:0/3983057566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31540ae040 con 0x7f3154157ae0 2023-08-27T11:28:47.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.334+0000 7f314affd700 1 -- 172.21.15.112:0/3983057566 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c003e10 con 0x7f3154157ae0 2023-08-27T11:28:47.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.334+0000 7f315c414700 1 -- 172.21.15.112:0/3983057566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31540ae600 con 0x7f3154157ae0 2023-08-27T11:28:47.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.335+0000 7f315c414700 1 -- 172.21.15.112:0/3983057566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3154067a20 con 0x7f3154157ae0 2023-08-27T11:28:47.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.335+0000 7f314affd700 1 -- 172.21.15.112:0/3983057566 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f314c004040 con 0x7f3154157ae0 2023-08-27T11:28:47.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.336+0000 7f314affd700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f313806a4d0 0x7f313806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.336+0000 7f314affd700 1 -- 172.21.15.112:0/3983057566 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f314c022070 con 0x7f3154157ae0 2023-08-27T11:28:47.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.336+0000 7f315a1b0700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f313806a4d0 0x7f313806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.336+0000 7f315a1b0700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f313806a4d0 0x7f313806c990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3144007fd0 tx=0x7f3144007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:47.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.340+0000 7f314affd700 1 -- 172.21.15.112:0/3983057566 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f314c055db0 con 0x7f3154157ae0 2023-08-27T11:28:47.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:47.532+0000 7f315c414700 1 -- 172.21.15.112:0/3983057566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3154160e60 con 0x7f3154157ae0 2023-08-27T11:28:48.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.400+0000 7f314affd700 1 -- 172.21.15.112:0/3983057566 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v150) v1 ==== 138+0+0 (secure 0 0 0) 0x7f314c0593d0 con 0x7f3154157ae0 2023-08-27T11:28:48.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.404+0000 7f315c414700 1 -- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f313806a4d0 msgr2=0x7f313806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:48.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.404+0000 7f315c414700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f313806a4d0 0x7f313806c990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3144007fd0 tx=0x7f3144007dc0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.404+0000 7f315c414700 1 -- 172.21.15.112:0/3983057566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3154157ae0 msgr2=0x7f3154141470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:48.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.404+0000 7f315c414700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3154157ae0 0x7f3154141470 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f315406e930 tx=0x7f314c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.405+0000 7f315c414700 1 -- 172.21.15.112:0/3983057566 shutdown_connections 2023-08-27T11:28:48.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.405+0000 7f315c414700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f313806a4d0 0x7f313806c990 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.405+0000 7f315c414700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3154157ae0 0x7f3154141470 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.405+0000 7f315c414700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f315406f790 0x7f315413ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.405+0000 7f315c414700 1 --2- 172.21.15.112:0/3983057566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31540b3de0 0x7f315415f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.405+0000 7f315c414700 1 -- 172.21.15.112:0/3983057566 >> 172.21.15.112:0/3983057566 conn(0x7f31540b1be0 msgr2=0x7f31541464b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:48.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.405+0000 7f315c414700 1 -- 172.21.15.112:0/3983057566 shutdown_connections 2023-08-27T11:28:48.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.405+0000 7f315c414700 1 -- 172.21.15.112:0/3983057566 wait complete. 2023-08-27T11:28:48.409 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2023-08-27T11:28:48.423 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:28:48.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.553+0000 7fbea41d2700 1 Processor -- start 2023-08-27T11:28:48.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.554+0000 7fbea41d2700 1 -- start start 2023-08-27T11:28:48.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.554+0000 7fbea41d2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c1466c0 0x7fbe9c0cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:48.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.554+0000 7fbea41d2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbe9c0708a0 0x7fbe9c0cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:48.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.554+0000 7fbea41d2700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9c0cf230 con 0x7fbe9c0cb510 2023-08-27T11:28:48.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.554+0000 7fbea41d2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9c0719b0 con 0x7fbe9c0708a0 2023-08-27T11:28:48.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.554+0000 7fbea41d2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0d1030 con 0x7fbe9c1466c0 2023-08-27T11:28:48.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.555+0000 7fbea176d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbe9c0708a0 0x7fbe9c0cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:48.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.555+0000 7fbea176d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbe9c0708a0 0x7fbe9c0cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37000/0 (socket says 172.21.15.112:37000) 2023-08-27T11:28:48.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.555+0000 7fbea176d700 1 -- 172.21.15.112:0/1583972202 learned_addr learned my addr 172.21.15.112:0/1583972202 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:48.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.555+0000 7fbea1f6e700 1 --2- 172.21.15.112:0/1583972202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c1466c0 0x7fbe9c0cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:48.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.555+0000 7fbea176d700 1 -- 172.21.15.112:0/1583972202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c1466c0 msgr2=0x7fbe9c0cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:48.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.555+0000 7fbea176d700 1 --2- 172.21.15.112:0/1583972202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c1466c0 0x7fbe9c0cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.555+0000 7fbea176d700 1 -- 172.21.15.112:0/1583972202 >> v1:172.21.15.138:6789/0 conn(0x7fbe9c0cb510 legacy=0x7fbe9c06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:48.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.555+0000 7fbea176d700 1 -- 172.21.15.112:0/1583972202 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c147000 con 0x7fbe9c0708a0 2023-08-27T11:28:48.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.555+0000 7fbea1f6e700 1 --2- 172.21.15.112:0/1583972202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c1466c0 0x7fbe9c0cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:48.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.555+0000 7fbea176d700 1 --2- 172.21.15.112:0/1583972202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe9c0708a0 0x7fbe9c0cafd0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009920 tx=0x7fbe98009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16181154b3e9a7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:48.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.555+0000 7fbea0f6c700 1 -- 172.21.15.112:0/1583972202 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe98004230 con 0x7fbe9c0708a0 2023-08-27T11:28:48.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.555+0000 7fbea0f6c700 1 -- 172.21.15.112:0/1583972202 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe98004390 con 0x7fbe9c0708a0 2023-08-27T11:28:48.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.555+0000 7fbea0f6c700 1 -- 172.21.15.112:0/1583972202 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe98009e70 con 0x7fbe9c0708a0 2023-08-27T11:28:48.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.556+0000 7fbea41d2700 1 -- 172.21.15.112:0/1583972202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe9c0708a0 msgr2=0x7fbe9c0cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:48.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.556+0000 7fbea41d2700 1 --2- 172.21.15.112:0/1583972202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe9c0708a0 0x7fbe9c0cafd0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009920 tx=0x7fbe98009c30 comp rx=0 tx=0).stop 2023-08-27T11:28:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.556+0000 7fbea41d2700 1 -- 172.21.15.112:0/1583972202 shutdown_connections 2023-08-27T11:28:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.556+0000 7fbea41d2700 1 --2- 172.21.15.112:0/1583972202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c1466c0 0x7fbe9c0cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.556+0000 7fbea41d2700 1 --2- 172.21.15.112:0/1583972202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe9c0708a0 0x7fbe9c0cafd0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.556+0000 7fbea41d2700 1 -- 172.21.15.112:0/1583972202 >> 172.21.15.112:0/1583972202 conn(0x7fbe9c0cdc50 msgr2=0x7fbe9c0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.556+0000 7fbea41d2700 1 -- 172.21.15.112:0/1583972202 shutdown_connections 2023-08-27T11:28:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.556+0000 7fbea41d2700 1 -- 172.21.15.112:0/1583972202 wait complete. 2023-08-27T11:28:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.557+0000 7fbea41d2700 1 Processor -- start 2023-08-27T11:28:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.557+0000 7fbea41d2700 1 -- start start 2023-08-27T11:28:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea41d2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c0cb510 0x7fbe9c1f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea41d2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe9c1466c0 0x7fbe9c1de840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea41d2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe9c1e7470 0x7fbe9c0b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea41d2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c1e5b50 con 0x7fbe9c1466c0 2023-08-27T11:28:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea41d2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c1e5da0 con 0x7fbe9c1e7470 2023-08-27T11:28:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea41d2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c1e6080 con 0x7fbe9c0cb510 2023-08-27T11:28:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea176d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe9c1466c0 0x7fbe9c1de840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea176d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe9c1466c0 0x7fbe9c1de840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37008/0 (socket says 172.21.15.112:37008) 2023-08-27T11:28:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea176d700 1 -- 172.21.15.112:0/991518428 learned_addr learned my addr 172.21.15.112:0/991518428 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea276f700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe9c1e7470 0x7fbe9c0b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea1f6e700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c0cb510 0x7fbe9c1f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea176d700 1 -- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c0cb510 msgr2=0x7fbe9c1f6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:48.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea176d700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c0cb510 0x7fbe9c1f6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea176d700 1 -- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe9c1e7470 msgr2=0x7fbe9c0b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:48.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea176d700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe9c1e7470 0x7fbe9c0b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea176d700 1 -- 172.21.15.112:0/991518428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe98009550 con 0x7fbe9c1466c0 2023-08-27T11:28:48.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea276f700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe9c1e7470 0x7fbe9c0b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:48.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.558+0000 7fbea176d700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe9c1466c0 0x7fbe9c1de840 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009870 tx=0x7fbe98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:48.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.559+0000 7fbe927fc700 1 -- 172.21.15.112:0/991518428 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9800bd00 con 0x7fbe9c1466c0 2023-08-27T11:28:48.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.559+0000 7fbe927fc700 1 -- 172.21.15.112:0/991518428 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9801f070 con 0x7fbe9c1466c0 2023-08-27T11:28:48.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.559+0000 7fbea41d2700 1 -- 172.21.15.112:0/991518428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c1e6360 con 0x7fbe9c1466c0 2023-08-27T11:28:48.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.559+0000 7fbe927fc700 1 -- 172.21.15.112:0/991518428 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe980038a0 con 0x7fbe9c1466c0 2023-08-27T11:28:48.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.559+0000 7fbea41d2700 1 -- 172.21.15.112:0/991518428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c1e6920 con 0x7fbe9c1466c0 2023-08-27T11:28:48.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.560+0000 7fbea41d2700 1 -- 172.21.15.112:0/991518428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9c067a20 con 0x7fbe9c1466c0 2023-08-27T11:28:48.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.560+0000 7fbe927fc700 1 -- 172.21.15.112:0/991518428 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fbe98021030 con 0x7fbe9c1466c0 2023-08-27T11:28:48.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.560+0000 7fbe927fc700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbe8006a4d0 0x7fbe8006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:48.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.561+0000 7fbe927fc700 1 -- 172.21.15.112:0/991518428 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fbe9808a040 con 0x7fbe9c1466c0 2023-08-27T11:28:48.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.561+0000 7fbea1f6e700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbe8006a4d0 0x7fbe8006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:48.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.561+0000 7fbea1f6e700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbe8006a4d0 0x7fbe8006c990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c006fd0 tx=0x7fbe8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:48.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.565+0000 7fbe927fc700 1 -- 172.21.15.112:0/991518428 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbe980bc990 con 0x7fbe9c1466c0 2023-08-27T11:28:48.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:48.753+0000 7fbea41d2700 1 -- 172.21.15.112:0/991518428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbe9c067a20 con 0x7fbe9c1466c0 2023-08-27T11:28:49.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.408+0000 7fbe927fc700 1 -- 172.21.15.112:0/991518428 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v151) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbe980554d0 con 0x7fbe9c1466c0 2023-08-27T11:28:49.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.413+0000 7fbea41d2700 1 -- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbe8006a4d0 msgr2=0x7fbe8006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:49.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.413+0000 7fbea41d2700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbe8006a4d0 0x7fbe8006c990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c006fd0 tx=0x7fbe8c008040 comp rx=0 tx=0).stop 2023-08-27T11:28:49.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.413+0000 7fbea41d2700 1 -- 172.21.15.112:0/991518428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe9c1466c0 msgr2=0x7fbe9c1de840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.413+0000 7fbea41d2700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe9c1466c0 0x7fbe9c1de840 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009870 tx=0x7fbe98004750 comp rx=0 tx=0).stop 2023-08-27T11:28:49.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.417+0000 7fbea41d2700 1 -- 172.21.15.112:0/991518428 shutdown_connections 2023-08-27T11:28:49.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.417+0000 7fbea41d2700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbe8006a4d0 0x7fbe8006c990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.417+0000 7fbea41d2700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe9c1e7470 0x7fbe9c0b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.417+0000 7fbea41d2700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c0cb510 0x7fbe9c1f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.417+0000 7fbea41d2700 1 --2- 172.21.15.112:0/991518428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe9c1466c0 0x7fbe9c1de840 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.417+0000 7fbea41d2700 1 -- 172.21.15.112:0/991518428 >> 172.21.15.112:0/991518428 conn(0x7fbe9c0cdc50 msgr2=0x7fbe9c143d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:49.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.417+0000 7fbea41d2700 1 -- 172.21.15.112:0/991518428 shutdown_connections 2023-08-27T11:28:49.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.417+0000 7fbea41d2700 1 -- 172.21.15.112:0/991518428 wait complete. 2023-08-27T11:28:49.457 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2023-08-27T11:28:49.457 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:28:49.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.580+0000 7fdbdf4b6700 1 Processor -- start 2023-08-27T11:28:49.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.580+0000 7fdbdf4b6700 1 -- start start 2023-08-27T11:28:49.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.581+0000 7fdbdf4b6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdbd81437c0 0x7fdbd80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:49.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.581+0000 7fdbdf4b6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbd8143ba0 0x7fdbd80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:49.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.581+0000 7fdbdf4b6700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd80cfd50 con 0x7fdbd806d5b0 2023-08-27T11:28:49.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.581+0000 7fdbdf4b6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd80cf140 con 0x7fdbd81437c0 2023-08-27T11:28:49.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.581+0000 7fdbdf4b6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd80cf420 con 0x7fdbd8143ba0 2023-08-27T11:28:49.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.581+0000 7fdbdd252700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdbd81437c0 0x7fdbd80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:49.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.581+0000 7fdbdd252700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdbd81437c0 0x7fdbd80b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37014/0 (socket says 172.21.15.112:37014) 2023-08-27T11:28:49.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.581+0000 7fdbdd252700 1 -- 172.21.15.112:0/1846293841 learned_addr learned my addr 172.21.15.112:0/1846293841 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:49.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.581+0000 7fdbdda53700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fdbd806d5b0 0x7fdbd815dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59630/0 (socket says 172.21.15.112:59630) 2023-08-27T11:28:49.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.581+0000 7fdbdca51700 1 --2- 172.21.15.112:0/1846293841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbd8143ba0 0x7fdbd80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:49.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.581+0000 7fdbdd252700 1 -- 172.21.15.112:0/1846293841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbd8143ba0 msgr2=0x7fdbd80ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:49.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.581+0000 7fdbdd252700 1 --2- 172.21.15.112:0/1846293841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbd8143ba0 0x7fdbd80ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbdd252700 1 -- 172.21.15.112:0/1846293841 >> v1:172.21.15.112:6789/0 conn(0x7fdbd806d5b0 legacy=0x7fdbd815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:49.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbdd252700 1 -- 172.21.15.112:0/1846293841 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd80cf700 con 0x7fdbd81437c0 2023-08-27T11:28:49.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbcffff700 1 -- 172.21.15.112:0/1846293841 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379454539 0 0) 0x7fdbd80cfd50 con 0x7fdbd806d5b0 2023-08-27T11:28:49.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbdd252700 1 --2- 172.21.15.112:0/1846293841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbd81437c0 0x7fdbd80b4670 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80098f0 tx=0x7fdbc8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebdab34f16ffd81 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:49.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbcffff700 1 -- 172.21.15.112:0/1846293841 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc800b8a0 con 0x7fdbd81437c0 2023-08-27T11:28:49.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbcffff700 1 -- 172.21.15.112:0/1846293841 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc800ba00 con 0x7fdbd81437c0 2023-08-27T11:28:49.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbcffff700 1 -- 172.21.15.112:0/1846293841 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc800bd90 con 0x7fdbd81437c0 2023-08-27T11:28:49.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/1846293841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbd81437c0 msgr2=0x7fdbd80b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:49.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbdf4b6700 1 --2- 172.21.15.112:0/1846293841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbd81437c0 0x7fdbd80b4670 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80098f0 tx=0x7fdbc8009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:49.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/1846293841 shutdown_connections 2023-08-27T11:28:49.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbdf4b6700 1 --2- 172.21.15.112:0/1846293841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbd81437c0 0x7fdbd80b4670 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbdf4b6700 1 --2- 172.21.15.112:0/1846293841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbd8143ba0 0x7fdbd80ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/1846293841 >> 172.21.15.112:0/1846293841 conn(0x7fdbd80b1e90 msgr2=0x7fdbd80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:49.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/1846293841 shutdown_connections 2023-08-27T11:28:49.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.582+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/1846293841 wait complete. 2023-08-27T11:28:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.584+0000 7fdbdf4b6700 1 Processor -- start 2023-08-27T11:28:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.584+0000 7fdbdf4b6700 1 -- start start 2023-08-27T11:28:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.584+0000 7fdbdf4b6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbd806d5b0 0x7fdbd8159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.584+0000 7fdbdf4b6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbd81437c0 0x7fdbd813ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.584+0000 7fdbdf4b6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbd8143ba0 0x7fdbd81580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.584+0000 7fdbdf4b6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd813da90 con 0x7fdbd806d5b0 2023-08-27T11:28:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.584+0000 7fdbdf4b6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd81ee7a0 con 0x7fdbd8143ba0 2023-08-27T11:28:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.584+0000 7fdbdf4b6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd813f280 con 0x7fdbd81437c0 2023-08-27T11:28:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.584+0000 7fdbdd252700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbd806d5b0 0x7fdbd8159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.584+0000 7fdbdd252700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbd806d5b0 0x7fdbd8159ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37018/0 (socket says 172.21.15.112:37018) 2023-08-27T11:28:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.584+0000 7fdbdd252700 1 -- 172.21.15.112:0/558846544 learned_addr learned my addr 172.21.15.112:0/558846544 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.585+0000 7fdbdd252700 1 -- 172.21.15.112:0/558846544 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbd81437c0 msgr2=0x7fdbd813ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.585+0000 7fdbdd252700 1 --2- 172.21.15.112:0/558846544 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbd81437c0 0x7fdbd813ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.585+0000 7fdbdd252700 1 -- 172.21.15.112:0/558846544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbd8143ba0 msgr2=0x7fdbd81580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.585+0000 7fdbdd252700 1 --2- 172.21.15.112:0/558846544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbd8143ba0 0x7fdbd81580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.585+0000 7fdbdd252700 1 -- 172.21.15.112:0/558846544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc8009550 con 0x7fdbd806d5b0 2023-08-27T11:28:49.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.585+0000 7fdbdd252700 1 --2- 172.21.15.112:0/558846544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbd806d5b0 0x7fdbd8159ae0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009c40 tx=0x7fdbc8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:49.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.585+0000 7fdbcdffb700 1 -- 172.21.15.112:0/558846544 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8004380 con 0x7fdbd806d5b0 2023-08-27T11:28:49.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.585+0000 7fdbcdffb700 1 -- 172.21.15.112:0/558846544 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc801f070 con 0x7fdbd806d5b0 2023-08-27T11:28:49.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.585+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/558846544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd813dca0 con 0x7fdbd806d5b0 2023-08-27T11:28:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.585+0000 7fdbcdffb700 1 -- 172.21.15.112:0/558846544 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80036a0 con 0x7fdbd806d5b0 2023-08-27T11:28:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.585+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/558846544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd813e260 con 0x7fdbd806d5b0 2023-08-27T11:28:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.586+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/558846544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd8067e90 con 0x7fdbd806d5b0 2023-08-27T11:28:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.587+0000 7fdbcdffb700 1 -- 172.21.15.112:0/558846544 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fdbc8021030 con 0x7fdbd806d5b0 2023-08-27T11:28:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.587+0000 7fdbcdffb700 1 --2- 172.21.15.112:0/558846544 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdbbc06a4d0 0x7fdbbc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.587+0000 7fdbcdffb700 1 -- 172.21.15.112:0/558846544 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fdbc8089fb0 con 0x7fdbd806d5b0 2023-08-27T11:28:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.587+0000 7fdbdca51700 1 --2- 172.21.15.112:0/558846544 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdbbc06a4d0 0x7fdbbc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.588+0000 7fdbdca51700 1 --2- 172.21.15.112:0/558846544 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdbbc06a4d0 0x7fdbbc06c990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdbd813d2a0 tx=0x7fdbd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.592+0000 7fdbcdffb700 1 -- 172.21.15.112:0/558846544 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdbc8058d50 con 0x7fdbd806d5b0 2023-08-27T11:28:49.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:49.823+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/558846544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdbd8067a20 con 0x7fdbd806d5b0 2023-08-27T11:28:50.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.408+0000 7fdbcdffb700 1 -- 172.21.15.112:0/558846544 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v101) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdbc80150c0 con 0x7fdbd806d5b0 2023-08-27T11:28:50.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.413+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/558846544 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdbbc06a4d0 msgr2=0x7fdbbc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.413+0000 7fdbdf4b6700 1 --2- 172.21.15.112:0/558846544 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdbbc06a4d0 0x7fdbbc06c990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdbd813d2a0 tx=0x7fdbd4008040 comp rx=0 tx=0).stop 2023-08-27T11:28:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.413+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/558846544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbd806d5b0 msgr2=0x7fdbd8159ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.413+0000 7fdbdf4b6700 1 --2- 172.21.15.112:0/558846544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbd806d5b0 0x7fdbd8159ae0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009c40 tx=0x7fdbc8004750 comp rx=0 tx=0).stop 2023-08-27T11:28:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.413+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/558846544 shutdown_connections 2023-08-27T11:28:50.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.413+0000 7fdbdf4b6700 1 --2- 172.21.15.112:0/558846544 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdbbc06a4d0 0x7fdbbc06c990 secure :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdbd813d2a0 tx=0x7fdbd4008040 comp rx=0 tx=0).stop 2023-08-27T11:28:50.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.413+0000 7fdbdf4b6700 1 --2- 172.21.15.112:0/558846544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbd8143ba0 0x7fdbd81580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.413+0000 7fdbdf4b6700 1 --2- 172.21.15.112:0/558846544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbd806d5b0 0x7fdbd8159ae0 secure :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009c40 tx=0x7fdbc8004750 comp rx=0 tx=0).stop 2023-08-27T11:28:50.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.413+0000 7fdbdf4b6700 1 --2- 172.21.15.112:0/558846544 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdbd81437c0 0x7fdbd813ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.413+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/558846544 >> 172.21.15.112:0/558846544 conn(0x7fdbd80b1e90 msgr2=0x7fdbd8071a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.415+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/558846544 shutdown_connections 2023-08-27T11:28:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.415+0000 7fdbdf4b6700 1 -- 172.21.15.112:0/558846544 wait complete. 2023-08-27T11:28:50.425 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 27 and data pool 28 2023-08-27T11:28:50.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:50.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.557+0000 7fe08d314700 1 Processor -- start 2023-08-27T11:28:50.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.557+0000 7fe08d314700 1 -- start start 2023-08-27T11:28:50.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe08d314700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe0880d1d00 0x7fe08815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe08d314700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815c120 0x7fe088144210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe08d314700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe08815ee80 con 0x7fe0880b3d30 2023-08-27T11:28:50.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe08d314700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08815fb00 con 0x7fe0880d1d00 2023-08-27T11:28:50.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe08d314700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880b5690 con 0x7fe08815c120 2023-08-27T11:28:50.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe086ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe0880b3d30 0x7fe0880d15f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59652/0 (socket says 172.21.15.112:59652) 2023-08-27T11:28:50.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe086ffd700 1 -- 172.21.15.112:0/2374831623 learned_addr learned my addr 172.21.15.112:0/2374831623 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:50.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe0867fc700 1 --2- 172.21.15.112:0/2374831623 >> v2:172.21.15.138:3300/0 conn(0x7fe0880d1d00 0x7fe08815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe0877fe700 1 --2- 172.21.15.112:0/2374831623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815c120 0x7fe088144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485719938 0 0) 0x7fe08815ee80 con 0x7fe0880b3d30 2023-08-27T11:28:50.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0700035b0 con 0x7fe0880b3d30 2023-08-27T11:28:50.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 865314071 0 0) 0x7fe0700035b0 con 0x7fe0880b3d30 2023-08-27T11:28:50.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815c120 msgr2=0x7fe088144210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:50.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe085ffb700 1 --2- 172.21.15.112:0/2374831623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815c120 0x7fe088144210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 >> v2:172.21.15.138:3300/0 conn(0x7fe0880d1d00 msgr2=0x7fe08815bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:50.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe085ffb700 1 --2- 172.21.15.112:0/2374831623 >> v2:172.21.15.138:3300/0 conn(0x7fe0880d1d00 0x7fe08815bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08815f160 con 0x7fe0880b3d30 2023-08-27T11:28:50.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fe07c001670 con 0x7fe0880b3d30 2023-08-27T11:28:50.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.558+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 >> v1:172.21.15.112:6789/0 conn(0x7fe0880b3d30 legacy=0x7fe0880d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe085ffb700 1 --2- 172.21.15.112:0/2374831623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe070003a00 0x7fe070003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe085ffb700 1 --2- 172.21.15.112:0/2374831623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe070004850 0x7fe070004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe085ffb700 1 --2- 172.21.15.112:0/2374831623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815c120 0x7fe070005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08815fb00 con 0x7fe070003a00 2023-08-27T11:28:50.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07000ba10 con 0x7fe070004850 2023-08-27T11:28:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07000bb50 con 0x7fe08815c120 2023-08-27T11:28:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe086ffd700 1 --2- 172.21.15.112:0/2374831623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe070003a00 0x7fe070003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe0867fc700 1 --2- 172.21.15.112:0/2374831623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe070004850 0x7fe070004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe0877fe700 1 --2- 172.21.15.112:0/2374831623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815c120 0x7fe070005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe086ffd700 1 -- 172.21.15.112:0/2374831623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815c120 msgr2=0x7fe070005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe086ffd700 1 --2- 172.21.15.112:0/2374831623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815c120 0x7fe070005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe086ffd700 1 -- 172.21.15.112:0/2374831623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe070004850 msgr2=0x7fe070004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:50.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe086ffd700 1 --2- 172.21.15.112:0/2374831623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe070004850 0x7fe070004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe086ffd700 1 -- 172.21.15.112:0/2374831623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0700035b0 con 0x7fe070003a00 2023-08-27T11:28:50.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe0877fe700 1 --2- 172.21.15.112:0/2374831623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815c120 0x7fe070005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:50.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe086ffd700 1 --2- 172.21.15.112:0/2374831623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe070003a00 0x7fe070003e20 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00c350 tx=0x7fe070005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d88e25fb56a06b4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00ce50 con 0x7fe070003a00 2023-08-27T11:28:50.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c005960 con 0x7fe070003a00 2023-08-27T11:28:50.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.559+0000 7fe085ffb700 1 -- 172.21.15.112:0/2374831623 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c005d10 con 0x7fe070003a00 2023-08-27T11:28:50.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.560+0000 7fe08d314700 1 -- 172.21.15.112:0/2374831623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe070003a00 msgr2=0x7fe070003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.560+0000 7fe08d314700 1 --2- 172.21.15.112:0/2374831623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe070003a00 0x7fe070003e20 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00c350 tx=0x7fe070005880 comp rx=0 tx=0).stop 2023-08-27T11:28:50.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.560+0000 7fe0877fe700 1 -- 172.21.15.112:0/2374831623 reap_dead start 2023-08-27T11:28:50.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.560+0000 7fe08d314700 1 -- 172.21.15.112:0/2374831623 shutdown_connections 2023-08-27T11:28:50.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.560+0000 7fe08d314700 1 -- 172.21.15.112:0/2374831623 >> 172.21.15.112:0/2374831623 conn(0x7fe08806b1d0 msgr2=0x7fe08806b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:50.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.560+0000 7fe08d314700 1 -- 172.21.15.112:0/2374831623 shutdown_connections 2023-08-27T11:28:50.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.560+0000 7fe08d314700 1 -- 172.21.15.112:0/2374831623 wait complete. 2023-08-27T11:28:50.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.561+0000 7fe08d314700 1 Processor -- start 2023-08-27T11:28:50.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.561+0000 7fe08d314700 1 -- start start 2023-08-27T11:28:50.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe08d314700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe070003a00 0x7fe088158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe08d314700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe088133560 0x7fe0881339e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe08d314700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0880cf8e0 0x7fe0880cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe08d314700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880cd030 con 0x7fe0880cf8e0 2023-08-27T11:28:50.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe08d314700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880d02d0 con 0x7fe088133560 2023-08-27T11:28:50.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe08d314700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe088131a00 con 0x7fe070003a00 2023-08-27T11:28:50.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe0877fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0880cf8e0 0x7fe0880cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe0877fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0880cf8e0 0x7fe0880cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37054/0 (socket says 172.21.15.112:37054) 2023-08-27T11:28:50.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe0877fe700 1 -- 172.21.15.112:0/262501419 learned_addr learned my addr 172.21.15.112:0/262501419 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:50.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe0867fc700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe088133560 0x7fe0881339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe086ffd700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe070003a00 0x7fe088158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe0877fe700 1 -- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe070003a00 msgr2=0x7fe088158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe0877fe700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe070003a00 0x7fe088158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe0877fe700 1 -- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe088133560 msgr2=0x7fe0881339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe0877fe700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe088133560 0x7fe0881339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe0877fe700 1 -- 172.21.15.112:0/262501419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0881316b0 con 0x7fe0880cf8e0 2023-08-27T11:28:50.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe0877fe700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0880cf8e0 0x7fe0880cfd90 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fe0880d07a0 tx=0x7fe0800104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe06f7fe700 1 -- 172.21.15.112:0/262501419 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08001b070 con 0x7fe0880cf8e0 2023-08-27T11:28:50.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.562+0000 7fe06f7fe700 1 -- 172.21.15.112:0/262501419 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08000f040 con 0x7fe0880cf8e0 2023-08-27T11:28:50.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.563+0000 7fe06f7fe700 1 -- 172.21.15.112:0/262501419 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08000e050 con 0x7fe0880cf8e0 2023-08-27T11:28:50.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.563+0000 7fe08d314700 1 -- 172.21.15.112:0/262501419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe088134010 con 0x7fe0880cf8e0 2023-08-27T11:28:50.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.563+0000 7fe08d314700 1 -- 172.21.15.112:0/262501419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe088159050 con 0x7fe0880cf8e0 2023-08-27T11:28:50.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.563+0000 7fe08d314700 1 -- 172.21.15.112:0/262501419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe088134530 con 0x7fe0880cf8e0 2023-08-27T11:28:50.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.564+0000 7fe06f7fe700 1 -- 172.21.15.112:0/262501419 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe080015440 con 0x7fe0880cf8e0 2023-08-27T11:28:50.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.564+0000 7fe06f7fe700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe07406a4d0 0x7fe07406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.565+0000 7fe06f7fe700 1 -- 172.21.15.112:0/262501419 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe080088e20 con 0x7fe0880cf8e0 2023-08-27T11:28:50.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.565+0000 7fe086ffd700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe07406a4d0 0x7fe07406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.565+0000 7fe086ffd700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe07406a4d0 0x7fe07406c990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe07c008fd0 tx=0x7fe07c00a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.569+0000 7fe06f7fe700 1 -- 172.21.15.112:0/262501419 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe080053de0 con 0x7fe0880cf8e0 2023-08-27T11:28:50.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.762+0000 7fe08d314700 1 -- 172.21.15.112:0/262501419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe08815f720 con 0x7fe0880cf8e0 2023-08-27T11:28:50.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.763+0000 7fe06f7fe700 1 -- 172.21.15.112:0/262501419 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+18340 (secure 0 0 0) 0x7fe080057400 con 0x7fe0880cf8e0 2023-08-27T11:28:50.764 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:50.765 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":152,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:49.824477+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.551922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.772403+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:50.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.765+0000 7fe08d314700 1 -- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe07406a4d0 msgr2=0x7fe07406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.765+0000 7fe08d314700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe07406a4d0 0x7fe07406c990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe07c008fd0 tx=0x7fe07c00a000 comp rx=0 tx=0).stop 2023-08-27T11:28:50.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.765+0000 7fe08d314700 1 -- 172.21.15.112:0/262501419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0880cf8e0 msgr2=0x7fe0880cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.765+0000 7fe08d314700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0880cf8e0 0x7fe0880cfd90 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fe0880d07a0 tx=0x7fe0800104a0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.765+0000 7fe08d314700 1 -- 172.21.15.112:0/262501419 shutdown_connections 2023-08-27T11:28:50.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.765+0000 7fe08d314700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe07406a4d0 0x7fe07406c990 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.765+0000 7fe08d314700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0880cf8e0 0x7fe0880cfd90 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.765+0000 7fe08d314700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe070003a00 0x7fe088158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.765+0000 7fe08d314700 1 --2- 172.21.15.112:0/262501419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe088133560 0x7fe0881339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.765+0000 7fe08d314700 1 -- 172.21.15.112:0/262501419 >> 172.21.15.112:0/262501419 conn(0x7fe08806b1d0 msgr2=0x7fe0880d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:50.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.765+0000 7fe08d314700 1 -- 172.21.15.112:0/262501419 shutdown_connections 2023-08-27T11:28:50.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.765+0000 7fe08d314700 1 -- 172.21.15.112:0/262501419 wait complete. 2023-08-27T11:28:50.779 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:50.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.897+0000 7fabc6f7e700 1 Processor -- start 2023-08-27T11:28:50.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.897+0000 7fabc6f7e700 1 -- start start 2023-08-27T11:28:50.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.898+0000 7fabc6f7e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fabc0144570 0x7fabc015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.898+0000 7fabc6f7e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fabc00b3e50 0x7fabc015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.898+0000 7fabc6f7e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc00b5ae0 con 0x7fabc015f8a0 2023-08-27T11:28:50.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.898+0000 7fabc6f7e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc015fcf0 con 0x7fabc00b3e50 2023-08-27T11:28:50.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.898+0000 7fabc6f7e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc0144950 con 0x7fabc0144570 2023-08-27T11:28:50.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.898+0000 7fabbffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fabc00b3e50 0x7fabc015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.898+0000 7fabbffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fabc00b3e50 0x7fabc015f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37062/0 (socket says 172.21.15.112:37062) 2023-08-27T11:28:50.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.898+0000 7fabc551b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fabc015f8a0 0x7fabc00ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59674/0 (socket says 172.21.15.112:59674) 2023-08-27T11:28:50.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.898+0000 7fabbffff700 1 -- 172.21.15.112:0/1702189231 learned_addr learned my addr 172.21.15.112:0/1702189231 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:50.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.899+0000 7fabc4d1a700 1 --2- 172.21.15.112:0/1702189231 >> v2:172.21.15.138:3300/0 conn(0x7fabc0144570 0x7fabc015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.899+0000 7fabbffff700 1 -- 172.21.15.112:0/1702189231 >> v2:172.21.15.138:3300/0 conn(0x7fabc0144570 msgr2=0x7fabc015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:50.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.899+0000 7fabbffff700 1 --2- 172.21.15.112:0/1702189231 >> v2:172.21.15.138:3300/0 conn(0x7fabc0144570 0x7fabc015ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.899+0000 7fabbffff700 1 -- 172.21.15.112:0/1702189231 >> v1:172.21.15.112:6789/0 conn(0x7fabc015f8a0 legacy=0x7fabc00ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.899+0000 7fabbf7fe700 1 -- 172.21.15.112:0/1702189231 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547985884 0 0) 0x7fabb8001490 con 0x7fabc015f8a0 2023-08-27T11:28:50.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.899+0000 7fabbffff700 1 -- 172.21.15.112:0/1702189231 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc00b45f0 con 0x7fabc00b3e50 2023-08-27T11:28:50.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.899+0000 7fabbffff700 1 --2- 172.21.15.112:0/1702189231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc00b3e50 0x7fabc015f360 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fabc006b7b0 tx=0x7fabac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a727a5759c7569 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.899+0000 7fabbf7fe700 1 -- 172.21.15.112:0/1702189231 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac00b860 con 0x7fabc00b3e50 2023-08-27T11:28:50.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.899+0000 7fabbf7fe700 1 -- 172.21.15.112:0/1702189231 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac00b9c0 con 0x7fabc00b3e50 2023-08-27T11:28:50.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.899+0000 7fabbf7fe700 1 -- 172.21.15.112:0/1702189231 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac00bd50 con 0x7fabc00b3e50 2023-08-27T11:28:50.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.900+0000 7fabc6f7e700 1 -- 172.21.15.112:0/1702189231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc00b3e50 msgr2=0x7fabc015f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.900+0000 7fabc6f7e700 1 --2- 172.21.15.112:0/1702189231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc00b3e50 0x7fabc015f360 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fabc006b7b0 tx=0x7fabac009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:50.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.900+0000 7fabc6f7e700 1 -- 172.21.15.112:0/1702189231 shutdown_connections 2023-08-27T11:28:50.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.900+0000 7fabc6f7e700 1 --2- 172.21.15.112:0/1702189231 >> v2:172.21.15.138:3300/0 conn(0x7fabc0144570 0x7fabc015ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.900+0000 7fabc6f7e700 1 --2- 172.21.15.112:0/1702189231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc00b3e50 0x7fabc015f360 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.900+0000 7fabc6f7e700 1 -- 172.21.15.112:0/1702189231 >> 172.21.15.112:0/1702189231 conn(0x7fabc006f970 msgr2=0x7fabc006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:50.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.900+0000 7fabc6f7e700 1 -- 172.21.15.112:0/1702189231 shutdown_connections 2023-08-27T11:28:50.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.900+0000 7fabc6f7e700 1 -- 172.21.15.112:0/1702189231 wait complete. 2023-08-27T11:28:50.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.901+0000 7fabc6f7e700 1 Processor -- start 2023-08-27T11:28:50.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.901+0000 7fabc6f7e700 1 -- start start 2023-08-27T11:28:50.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.902+0000 7fabc6f7e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc00b3e50 0x7fabc0140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.902+0000 7fabc6f7e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc0144570 0x7fabc00c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.902+0000 7fabc6f7e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc015f8a0 0x7fabc013d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.902+0000 7fabc6f7e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00c8ba0 con 0x7fabc015f8a0 2023-08-27T11:28:50.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.902+0000 7fabc6f7e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00c6bf0 con 0x7fabc00b3e50 2023-08-27T11:28:50.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.902+0000 7fabc6f7e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00c6ed0 con 0x7fabc0144570 2023-08-27T11:28:50.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.902+0000 7fabc551b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc015f8a0 0x7fabc013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.902+0000 7fabc551b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc015f8a0 0x7fabc013d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37068/0 (socket says 172.21.15.112:37068) 2023-08-27T11:28:50.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.902+0000 7fabc551b700 1 -- 172.21.15.112:0/908909779 learned_addr learned my addr 172.21.15.112:0/908909779 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:50.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.903+0000 7fabc4d1a700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc00b3e50 0x7fabc0140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.903+0000 7fabbffff700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc0144570 0x7fabc00c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.903+0000 7fabc551b700 1 -- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc0144570 msgr2=0x7fabc00c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.903+0000 7fabc551b700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc0144570 0x7fabc00c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.903+0000 7fabc551b700 1 -- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc00b3e50 msgr2=0x7fabc0140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.903+0000 7fabc551b700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc00b3e50 0x7fabc0140e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.903+0000 7fabc551b700 1 -- 172.21.15.112:0/908909779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac009550 con 0x7fabc015f8a0 2023-08-27T11:28:50.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.903+0000 7fabc551b700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc015f8a0 0x7fabc013d340 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fabb800cb60 tx=0x7fabb800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.903+0000 7fabbd7fa700 1 -- 172.21.15.112:0/908909779 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb800ab50 con 0x7fabc015f8a0 2023-08-27T11:28:50.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.903+0000 7fabbd7fa700 1 -- 172.21.15.112:0/908909779 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb800acb0 con 0x7fabc015f8a0 2023-08-27T11:28:50.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.903+0000 7fabc6f7e700 1 -- 172.21.15.112:0/908909779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc00c71b0 con 0x7fabc015f8a0 2023-08-27T11:28:50.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.903+0000 7fabbd7fa700 1 -- 172.21.15.112:0/908909779 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8007310 con 0x7fabc015f8a0 2023-08-27T11:28:50.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.903+0000 7fabc6f7e700 1 -- 172.21.15.112:0/908909779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc00cd160 con 0x7fabc015f8a0 2023-08-27T11:28:50.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.904+0000 7fabc6f7e700 1 -- 172.21.15.112:0/908909779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc0067a20 con 0x7fabc015f8a0 2023-08-27T11:28:50.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.904+0000 7fabbd7fa700 1 -- 172.21.15.112:0/908909779 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fabb8007470 con 0x7fabc015f8a0 2023-08-27T11:28:50.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.905+0000 7fabbd7fa700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faba406a4d0 0x7faba406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.905+0000 7fabbd7fa700 1 -- 172.21.15.112:0/908909779 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fabb8088860 con 0x7fabc015f8a0 2023-08-27T11:28:50.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.905+0000 7fabc4d1a700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faba406a4d0 0x7faba406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.906+0000 7fabc4d1a700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faba406a4d0 0x7faba406c990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009930 tx=0x7fabb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:50.909+0000 7fabbd7fa700 1 -- 172.21.15.112:0/908909779 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fabb80539f0 con 0x7fabc015f8a0 2023-08-27T11:28:51.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.099+0000 7fabc6f7e700 1 -- 172.21.15.112:0/908909779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabc00b5ae0 con 0x7fabc015f8a0 2023-08-27T11:28:51.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.100+0000 7fabbd7fa700 1 -- 172.21.15.112:0/908909779 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+18340 (secure 0 0 0) 0x7fabc00b5ae0 con 0x7fabc015f8a0 2023-08-27T11:28:51.102 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:51.103 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":152,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:49.824477+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.551922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.772403+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:51.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.103+0000 7fabc6f7e700 1 -- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faba406a4d0 msgr2=0x7faba406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:51.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.103+0000 7fabc6f7e700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faba406a4d0 0x7faba406c990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009930 tx=0x7fabb4008040 comp rx=0 tx=0).stop 2023-08-27T11:28:51.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.103+0000 7fabc6f7e700 1 -- 172.21.15.112:0/908909779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc015f8a0 msgr2=0x7fabc013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:51.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.103+0000 7fabc6f7e700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc015f8a0 0x7fabc013d340 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fabb800cb60 tx=0x7fabb800cf20 comp rx=0 tx=0).stop 2023-08-27T11:28:51.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.103+0000 7fabc6f7e700 1 -- 172.21.15.112:0/908909779 shutdown_connections 2023-08-27T11:28:51.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.103+0000 7fabc6f7e700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7faba406a4d0 0x7faba406c990 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:51.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.103+0000 7fabc6f7e700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc015f8a0 0x7fabc013d340 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:51.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.103+0000 7fabc6f7e700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc00b3e50 0x7fabc0140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:51.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.103+0000 7fabc6f7e700 1 --2- 172.21.15.112:0/908909779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fabc0144570 0x7fabc00c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:51.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.103+0000 7fabc6f7e700 1 -- 172.21.15.112:0/908909779 >> 172.21.15.112:0/908909779 conn(0x7fabc006f970 msgr2=0x7fabc015a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:51.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.103+0000 7fabc6f7e700 1 -- 172.21.15.112:0/908909779 shutdown_connections 2023-08-27T11:28:51.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.103+0000 7fabc6f7e700 1 -- 172.21.15.112:0/908909779 wait complete. 2023-08-27T11:28:51.117 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:28:51.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.239+0000 7f2fee258700 1 Processor -- start 2023-08-27T11:28:51.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.239+0000 7f2fee258700 1 -- start start 2023-08-27T11:28:51.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fee258700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2fe80715c0 0x7f2fe80b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:51.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fee258700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe80d1500 con 0x7f2fe8146bf0 2023-08-27T11:28:51.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fee258700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe80d0460 con 0x7f2fe8146fd0 2023-08-27T11:28:51.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fee258700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe80d0710 con 0x7f2fe80715c0 2023-08-27T11:28:51.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fe77fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2fe8146bf0 0x7f2fe80cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59686/0 (socket says 172.21.15.112:59686) 2023-08-27T11:28:51.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fe77fe700 1 -- 172.21.15.112:0/1983190031 learned_addr learned my addr 172.21.15.112:0/1983190031 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:51.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fe7fff700 1 --2- 172.21.15.112:0/1983190031 >> v2:172.21.15.138:3300/0 conn(0x7f2fe80715c0 0x7f2fe80b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:51.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157534356 0 0) 0x7f2fe80d1500 con 0x7f2fe8146bf0 2023-08-27T11:28:51.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fd00035b0 con 0x7f2fe8146bf0 2023-08-27T11:28:51.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1825644581 0 0) 0x7f2fd00035b0 con 0x7f2fe8146bf0 2023-08-27T11:28:51.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 >> v2:172.21.15.138:3300/0 conn(0x7f2fe80715c0 msgr2=0x7f2fe80b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:51.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fe67fc700 1 --2- 172.21.15.112:0/1983190031 >> v2:172.21.15.138:3300/0 conn(0x7f2fe80715c0 0x7f2fe80b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:51.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 >> v1:172.21.15.138:6789/0 conn(0x7f2fe8146fd0 legacy=0x7f2fe8070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:51.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe80d09f0 con 0x7f2fe8146bf0 2023-08-27T11:28:51.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.240+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f2fdc001670 con 0x7f2fe8146bf0 2023-08-27T11:28:51.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 >> v1:172.21.15.112:6789/0 conn(0x7f2fe8146bf0 legacy=0x7f2fe80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:51.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe67fc700 1 --2- 172.21.15.112:0/1983190031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fd0003a00 0x7f2fd0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:51.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe67fc700 1 --2- 172.21.15.112:0/1983190031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd0004850 0x7f2fd0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:51.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe67fc700 1 --2- 172.21.15.112:0/1983190031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fd0005210 0x7f2fd000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:51.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe80d0710 con 0x7f2fd0003a00 2023-08-27T11:28:51.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd000c1b0 con 0x7f2fd0004850 2023-08-27T11:28:51.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd000c370 con 0x7f2fd0005210 2023-08-27T11:28:51.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe77fe700 1 --2- 172.21.15.112:0/1983190031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fd0003a00 0x7f2fd0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:51.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe6ffd700 1 --2- 172.21.15.112:0/1983190031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd0004850 0x7f2fd0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:51.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe7fff700 1 --2- 172.21.15.112:0/1983190031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fd0005210 0x7f2fd000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:51.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe77fe700 1 -- 172.21.15.112:0/1983190031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fd0005210 msgr2=0x7f2fd000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:51.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe77fe700 1 --2- 172.21.15.112:0/1983190031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fd0005210 0x7f2fd000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:51.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe77fe700 1 -- 172.21.15.112:0/1983190031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd0004850 msgr2=0x7f2fd0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:51.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe77fe700 1 --2- 172.21.15.112:0/1983190031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd0004850 0x7f2fd0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:51.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe77fe700 1 -- 172.21.15.112:0/1983190031 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd00035b0 con 0x7f2fd0003a00 2023-08-27T11:28:51.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe7fff700 1 --2- 172.21.15.112:0/1983190031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fd0005210 0x7f2fd000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:51.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe7fff700 1 -- 172.21.15.112:0/1983190031 reap_dead start 2023-08-27T11:28:51.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe77fe700 1 --2- 172.21.15.112:0/1983190031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fd0003a00 0x7f2fd0003e20 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00c240 tx=0x7f2fd0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba989f4609dc20b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:51.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc007c10 con 0x7f2fd0003a00 2023-08-27T11:28:51.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc00ce00 con 0x7f2fd0003a00 2023-08-27T11:28:51.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.241+0000 7f2fe67fc700 1 -- 172.21.15.112:0/1983190031 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc005ba0 con 0x7f2fd0003a00 2023-08-27T11:28:51.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.242+0000 7f2fee258700 1 -- 172.21.15.112:0/1983190031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fd0003a00 msgr2=0x7f2fd0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:51.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.242+0000 7f2fee258700 1 --2- 172.21.15.112:0/1983190031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fd0003a00 0x7f2fd0003e20 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00c240 tx=0x7f2fd0005880 comp rx=0 tx=0).stop 2023-08-27T11:28:51.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.242+0000 7f2fee258700 1 -- 172.21.15.112:0/1983190031 shutdown_connections 2023-08-27T11:28:51.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.242+0000 7f2fee258700 1 --2- 172.21.15.112:0/1983190031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fd0003a00 0x7f2fd0003e20 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.242+0000 7f2fee258700 1 -- 172.21.15.112:0/1983190031 >> 172.21.15.112:0/1983190031 conn(0x7f2fe806c450 msgr2=0x7f2fe806f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.242+0000 7f2fee258700 1 -- 172.21.15.112:0/1983190031 shutdown_connections 2023-08-27T11:28:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.242+0000 7f2fee258700 1 -- 172.21.15.112:0/1983190031 wait complete. 2023-08-27T11:28:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.243+0000 7f2fee258700 1 Processor -- start 2023-08-27T11:28:51.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.243+0000 7f2fee258700 1 -- start start 2023-08-27T11:28:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fee258700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd0003a00 0x7f2fe815c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fee258700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fe80b0d00 0x7f2fe80b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fee258700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe80af7c0 0x7f2fe80afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fee258700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe80b1700 con 0x7f2fe80af7c0 2023-08-27T11:28:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fee258700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe80ae450 con 0x7f2fd0003a00 2023-08-27T11:28:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fee258700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe80ae720 con 0x7f2fe80b0d00 2023-08-27T11:28:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fe7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe80af7c0 0x7f2fe80afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fe7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe80af7c0 0x7f2fe80afc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37088/0 (socket says 172.21.15.112:37088) 2023-08-27T11:28:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fe7fff700 1 -- 172.21.15.112:0/3799453537 learned_addr learned my addr 172.21.15.112:0/3799453537 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fe77fe700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd0003a00 0x7f2fe815c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fe6ffd700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fe80b0d00 0x7f2fe80b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fe7fff700 1 -- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fe80b0d00 msgr2=0x7f2fe80b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fe7fff700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fe80b0d00 0x7f2fe80b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:51.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fe7fff700 1 -- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd0003a00 msgr2=0x7f2fe815c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:51.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fe7fff700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd0003a00 0x7f2fe815c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:51.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fe7fff700 1 -- 172.21.15.112:0/3799453537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe80aea00 con 0x7f2fe80af7c0 2023-08-27T11:28:51.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fe6ffd700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fe80b0d00 0x7f2fe80b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:51.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.244+0000 7f2fe7fff700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe80af7c0 0x7f2fe80afc50 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80676d0 tx=0x7f2fe00104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:51.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.245+0000 7f2fcffff700 1 -- 172.21.15.112:0/3799453537 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe001b070 con 0x7f2fe80af7c0 2023-08-27T11:28:51.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.245+0000 7f2fcffff700 1 -- 172.21.15.112:0/3799453537 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe000f040 con 0x7f2fe80af7c0 2023-08-27T11:28:51.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.245+0000 7f2fee258700 1 -- 172.21.15.112:0/3799453537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe80aece0 con 0x7f2fe80af7c0 2023-08-27T11:28:51.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.245+0000 7f2fcffff700 1 -- 172.21.15.112:0/3799453537 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe000e050 con 0x7f2fe80af7c0 2023-08-27T11:28:51.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.245+0000 7f2fee258700 1 -- 172.21.15.112:0/3799453537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe80af2a0 con 0x7f2fe80af7c0 2023-08-27T11:28:51.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.245+0000 7f2fee258700 1 -- 172.21.15.112:0/3799453537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe8067e90 con 0x7f2fe80af7c0 2023-08-27T11:28:51.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.246+0000 7f2fcffff700 1 -- 172.21.15.112:0/3799453537 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f2fe0003a20 con 0x7f2fe80af7c0 2023-08-27T11:28:51.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.246+0000 7f2fcffff700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2fd406a4d0 0x7f2fd406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:51.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.246+0000 7f2fcffff700 1 -- 172.21.15.112:0/3799453537 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2fe0089060 con 0x7f2fe80af7c0 2023-08-27T11:28:51.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.247+0000 7f2fe77fe700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2fd406a4d0 0x7f2fd406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:51.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.247+0000 7f2fe77fe700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2fd406a4d0 0x7f2fd406c990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc000c00 tx=0x7f2fdc007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:51.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.251+0000 7f2fcffff700 1 -- 172.21.15.112:0/3799453537 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2fe0054070 con 0x7f2fe80af7c0 2023-08-27T11:28:51.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:51.486+0000 7f2fee258700 1 -- 172.21.15.112:0/3799453537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2fe80d1750 con 0x7f2fe80af7c0 2023-08-27T11:28:52.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.433+0000 7f2fcffff700 1 -- 172.21.15.112:0/3799453537 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v103) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2fe0057690 con 0x7f2fe80af7c0 2023-08-27T11:28:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.436+0000 7f2fee258700 1 -- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2fd406a4d0 msgr2=0x7f2fd406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.436+0000 7f2fee258700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2fd406a4d0 0x7f2fd406c990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc000c00 tx=0x7f2fdc007000 comp rx=0 tx=0).stop 2023-08-27T11:28:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.436+0000 7f2fee258700 1 -- 172.21.15.112:0/3799453537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe80af7c0 msgr2=0x7f2fe80afc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.436+0000 7f2fee258700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe80af7c0 0x7f2fe80afc50 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80676d0 tx=0x7f2fe00104a0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.436+0000 7f2fee258700 1 -- 172.21.15.112:0/3799453537 shutdown_connections 2023-08-27T11:28:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.436+0000 7f2fee258700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2fd406a4d0 0x7f2fd406c990 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.436+0000 7f2fee258700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe80af7c0 0x7f2fe80afc50 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.436+0000 7f2fee258700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd0003a00 0x7f2fe815c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.436+0000 7f2fee258700 1 --2- 172.21.15.112:0/3799453537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fe80b0d00 0x7f2fe80b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.436+0000 7f2fee258700 1 -- 172.21.15.112:0/3799453537 >> 172.21.15.112:0/3799453537 conn(0x7f2fe806c450 msgr2=0x7f2fe815d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:52.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.436+0000 7f2fee258700 1 -- 172.21.15.112:0/3799453537 shutdown_connections 2023-08-27T11:28:52.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.436+0000 7f2fee258700 1 -- 172.21.15.112:0/3799453537 wait complete. 2023-08-27T11:28:52.451 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:28:52.452 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:28:52.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.578+0000 7fe995216700 1 Processor -- start 2023-08-27T11:28:52.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.578+0000 7fe995216700 1 -- start start 2023-08-27T11:28:52.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.579+0000 7fe995216700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe99006dcc0 0x7fe9900b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:52.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.579+0000 7fe995216700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9900cfd50 con 0x7fe9901437c0 2023-08-27T11:28:52.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.579+0000 7fe995216700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9900cf140 con 0x7fe990143ba0 2023-08-27T11:28:52.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.579+0000 7fe995216700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9900cf420 con 0x7fe99006dcc0 2023-08-27T11:28:52.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.579+0000 7fe98f59e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe99006dcc0 0x7fe9900b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:52.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.579+0000 7fe98ed9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe9901437c0 0x7fe9900ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59706/0 (socket says 172.21.15.112:59706) 2023-08-27T11:28:52.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.579+0000 7fe98ed9d700 1 -- 172.21.15.112:0/1260249598 learned_addr learned my addr 172.21.15.112:0/1260249598 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:52.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe98f59e700 1 -- 172.21.15.112:0/1260249598 >> v1:172.21.15.138:6789/0 conn(0x7fe990143ba0 legacy=0x7fe99006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:52.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe98f59e700 1 -- 172.21.15.112:0/1260249598 >> v1:172.21.15.112:6789/0 conn(0x7fe9901437c0 legacy=0x7fe9900ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:52.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe98dd9b700 1 -- 172.21.15.112:0/1260249598 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334204571 0 0) 0x7fe9900cfd50 con 0x7fe9901437c0 2023-08-27T11:28:52.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe98f59e700 1 -- 172.21.15.112:0/1260249598 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9900cf700 con 0x7fe99006dcc0 2023-08-27T11:28:52.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe98f59e700 1 --2- 172.21.15.112:0/1260249598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99006dcc0 0x7fe9900b4670 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fe988009d90 tx=0x7fe98800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc30f106b19330b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:52.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe98dd9b700 1 -- 172.21.15.112:0/1260249598 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98800e070 con 0x7fe99006dcc0 2023-08-27T11:28:52.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe98dd9b700 1 -- 172.21.15.112:0/1260249598 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98800bb90 con 0x7fe99006dcc0 2023-08-27T11:28:52.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe98dd9b700 1 -- 172.21.15.112:0/1260249598 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988003a20 con 0x7fe99006dcc0 2023-08-27T11:28:52.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe995216700 1 -- 172.21.15.112:0/1260249598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99006dcc0 msgr2=0x7fe9900b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:52.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe995216700 1 --2- 172.21.15.112:0/1260249598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99006dcc0 0x7fe9900b4670 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fe988009d90 tx=0x7fe98800b330 comp rx=0 tx=0).stop 2023-08-27T11:28:52.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe995216700 1 -- 172.21.15.112:0/1260249598 shutdown_connections 2023-08-27T11:28:52.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe995216700 1 --2- 172.21.15.112:0/1260249598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99006dcc0 0x7fe9900b4670 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe995216700 1 -- 172.21.15.112:0/1260249598 >> 172.21.15.112:0/1260249598 conn(0x7fe9900b1e90 msgr2=0x7fe9900b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:52.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe995216700 1 -- 172.21.15.112:0/1260249598 shutdown_connections 2023-08-27T11:28:52.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.580+0000 7fe995216700 1 -- 172.21.15.112:0/1260249598 wait complete. 2023-08-27T11:28:52.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.582+0000 7fe995216700 1 Processor -- start 2023-08-27T11:28:52.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.582+0000 7fe995216700 1 -- start start 2023-08-27T11:28:52.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe995216700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe99006dcc0 0x7fe99015cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:52.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe995216700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9901437c0 0x7fe9900b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:52.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe995216700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe990143ba0 0x7fe990132320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:52.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe995216700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901e4d00 con 0x7fe990143ba0 2023-08-27T11:28:52.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe995216700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99015d140 con 0x7fe9901437c0 2023-08-27T11:28:52.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe995216700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901351f0 con 0x7fe99006dcc0 2023-08-27T11:28:52.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe98f59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe990143ba0 0x7fe990132320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:52.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe98f59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe990143ba0 0x7fe990132320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37102/0 (socket says 172.21.15.112:37102) 2023-08-27T11:28:52.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe98f59e700 1 -- 172.21.15.112:0/3110811412 learned_addr learned my addr 172.21.15.112:0/3110811412 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:52.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe98ed9d700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe99006dcc0 0x7fe99015cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:52.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe98e59c700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9901437c0 0x7fe9900b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:52.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe98f59e700 1 -- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe99006dcc0 msgr2=0x7fe99015cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:52.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe98f59e700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe99006dcc0 0x7fe99015cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe98f59e700 1 -- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9901437c0 msgr2=0x7fe9900b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:52.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe98f59e700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9901437c0 0x7fe9900b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe98f59e700 1 -- 172.21.15.112:0/3110811412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe988009a70 con 0x7fe990143ba0 2023-08-27T11:28:52.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe98f59e700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe990143ba0 0x7fe990132320 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fe99006e400 tx=0x7fe988004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:52.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe9777fe700 1 -- 172.21.15.112:0/3110811412 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988012630 con 0x7fe990143ba0 2023-08-27T11:28:52.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe9777fe700 1 -- 172.21.15.112:0/3110811412 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9880127b0 con 0x7fe990143ba0 2023-08-27T11:28:52.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe995216700 1 -- 172.21.15.112:0/3110811412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9901e4080 con 0x7fe990143ba0 2023-08-27T11:28:52.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.583+0000 7fe9777fe700 1 -- 172.21.15.112:0/3110811412 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988012b60 con 0x7fe990143ba0 2023-08-27T11:28:52.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.584+0000 7fe995216700 1 -- 172.21.15.112:0/3110811412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9901e4640 con 0x7fe990143ba0 2023-08-27T11:28:52.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.584+0000 7fe995216700 1 -- 172.21.15.112:0/3110811412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9901e4e40 con 0x7fe990143ba0 2023-08-27T11:28:52.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.585+0000 7fe9777fe700 1 -- 172.21.15.112:0/3110811412 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe988012cc0 con 0x7fe990143ba0 2023-08-27T11:28:52.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.585+0000 7fe9777fe700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe97c06a4d0 0x7fe97c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:52.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.585+0000 7fe9777fe700 1 -- 172.21.15.112:0/3110811412 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe98808b2f0 con 0x7fe990143ba0 2023-08-27T11:28:52.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.585+0000 7fe98ed9d700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe97c06a4d0 0x7fe97c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:52.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.586+0000 7fe98ed9d700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe97c06a4d0 0x7fe97c06c990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe980007fd0 tx=0x7fe980007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:52.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.590+0000 7fe9777fe700 1 -- 172.21.15.112:0/3110811412 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe9901e4e40 con 0x7fe990143ba0 2023-08-27T11:28:52.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:52.824+0000 7fe995216700 1 -- 172.21.15.112:0/3110811412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe9901e4e40 con 0x7fe990143ba0 2023-08-27T11:28:53.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.436+0000 7fe9777fe700 1 -- 172.21.15.112:0/3110811412 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v104) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe9901e4e40 con 0x7fe990143ba0 2023-08-27T11:28:53.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.439+0000 7fe995216700 1 -- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe97c06a4d0 msgr2=0x7fe97c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.439+0000 7fe995216700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe97c06a4d0 0x7fe97c06c990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe980007fd0 tx=0x7fe980007de0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.439+0000 7fe995216700 1 -- 172.21.15.112:0/3110811412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe990143ba0 msgr2=0x7fe990132320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.439+0000 7fe995216700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe990143ba0 0x7fe990132320 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fe99006e400 tx=0x7fe988004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.439+0000 7fe995216700 1 -- 172.21.15.112:0/3110811412 shutdown_connections 2023-08-27T11:28:53.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.439+0000 7fe995216700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe97c06a4d0 0x7fe97c06c990 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.439+0000 7fe995216700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe990143ba0 0x7fe990132320 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.439+0000 7fe995216700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe99006dcc0 0x7fe99015cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.439+0000 7fe995216700 1 --2- 172.21.15.112:0/3110811412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9901437c0 0x7fe9900b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.439+0000 7fe995216700 1 -- 172.21.15.112:0/3110811412 >> 172.21.15.112:0/3110811412 conn(0x7fe9900b1e90 msgr2=0x7fe990142a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:53.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.439+0000 7fe995216700 1 -- 172.21.15.112:0/3110811412 shutdown_connections 2023-08-27T11:28:53.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.439+0000 7fe995216700 1 -- 172.21.15.112:0/3110811412 wait complete. 2023-08-27T11:28:53.454 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:28:53.455 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:53.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.578+0000 7f0e49f75700 1 Processor -- start 2023-08-27T11:28:53.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.578+0000 7f0e49f75700 1 -- start start 2023-08-27T11:28:53.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e49f75700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0e440ad430 0x7f0e440ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e49f75700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e440af000 0x7f0e440af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e49f75700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e441470e0 con 0x7f0e44146d00 2023-08-27T11:28:53.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e49f75700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e440aeb70 con 0x7f0e440ad430 2023-08-27T11:28:53.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e49f75700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440adf60 con 0x7f0e440af000 2023-08-27T11:28:53.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e42ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0e440ad430 0x7f0e440ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e437fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0e44146d00 0x7f0e440b3440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59720/0 (socket says 172.21.15.112:59720) 2023-08-27T11:28:53.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e42ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0e440ad430 0x7f0e440ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37108/0 (socket says 172.21.15.112:37108) 2023-08-27T11:28:53.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e437fe700 1 -- 172.21.15.112:0/3469575334 learned_addr learned my addr 172.21.15.112:0/3469575334 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:53.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e42ffd700 1 -- 172.21.15.112:0/3469575334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e440af000 msgr2=0x7f0e440af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:53.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e42ffd700 1 --2- 172.21.15.112:0/3469575334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e440af000 0x7f0e440af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e42ffd700 1 -- 172.21.15.112:0/3469575334 >> v1:172.21.15.112:6789/0 conn(0x7f0e44146d00 legacy=0x7f0e440b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e42ffd700 1 -- 172.21.15.112:0/3469575334 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e440ae240 con 0x7f0e440ad430 2023-08-27T11:28:53.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e427fc700 1 -- 172.21.15.112:0/3469575334 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2193527199 0 0) 0x7f0e441470e0 con 0x7f0e44146d00 2023-08-27T11:28:53.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.579+0000 7f0e42ffd700 1 --2- 172.21.15.112:0/3469575334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e440ad430 0x7f0e440ad8e0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009820 tx=0x7f0e38009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f866845643823c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:53.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.580+0000 7f0e427fc700 1 -- 172.21.15.112:0/3469575334 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38005e30 con 0x7f0e440ad430 2023-08-27T11:28:53.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.580+0000 7f0e427fc700 1 -- 172.21.15.112:0/3469575334 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e38006030 con 0x7f0e440ad430 2023-08-27T11:28:53.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.580+0000 7f0e427fc700 1 -- 172.21.15.112:0/3469575334 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380063e0 con 0x7f0e440ad430 2023-08-27T11:28:53.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.580+0000 7f0e49f75700 1 -- 172.21.15.112:0/3469575334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e440ad430 msgr2=0x7f0e440ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.580+0000 7f0e49f75700 1 --2- 172.21.15.112:0/3469575334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e440ad430 0x7f0e440ad8e0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009820 tx=0x7f0e38009b30 comp rx=0 tx=0).stop 2023-08-27T11:28:53.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.580+0000 7f0e49f75700 1 -- 172.21.15.112:0/3469575334 shutdown_connections 2023-08-27T11:28:53.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.580+0000 7f0e49f75700 1 --2- 172.21.15.112:0/3469575334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e440af000 0x7f0e440af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.580+0000 7f0e49f75700 1 --2- 172.21.15.112:0/3469575334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e440ad430 0x7f0e440ad8e0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.580+0000 7f0e49f75700 1 -- 172.21.15.112:0/3469575334 >> 172.21.15.112:0/3469575334 conn(0x7f0e4406adb0 msgr2=0x7f0e4406b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:53.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.580+0000 7f0e49f75700 1 -- 172.21.15.112:0/3469575334 shutdown_connections 2023-08-27T11:28:53.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.580+0000 7f0e49f75700 1 -- 172.21.15.112:0/3469575334 wait complete. 2023-08-27T11:28:53.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.581+0000 7f0e49f75700 1 Processor -- start 2023-08-27T11:28:53.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.582+0000 7f0e49f75700 1 -- start start 2023-08-27T11:28:53.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.582+0000 7f0e49f75700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e440ad430 0x7f0e4415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.582+0000 7f0e49f75700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e440af000 0x7f0e44160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.582+0000 7f0e49f75700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e44146d00 0x7f0e441f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.582+0000 7f0e49f75700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e44160240 con 0x7f0e440ad430 2023-08-27T11:28:53.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.582+0000 7f0e49f75700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4415cc10 con 0x7f0e440af000 2023-08-27T11:28:53.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.582+0000 7f0e49f75700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e44160ba0 con 0x7f0e44146d00 2023-08-27T11:28:53.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.582+0000 7f0e437fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e440ad430 0x7f0e4415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.582+0000 7f0e437fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e440ad430 0x7f0e4415c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37112/0 (socket says 172.21.15.112:37112) 2023-08-27T11:28:53.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.582+0000 7f0e437fe700 1 -- 172.21.15.112:0/4207379305 learned_addr learned my addr 172.21.15.112:0/4207379305 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:53.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e43fff700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e44146d00 0x7f0e441f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e42ffd700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e440af000 0x7f0e44160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e437fe700 1 -- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e44146d00 msgr2=0x7f0e441f67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e437fe700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e44146d00 0x7f0e441f67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e437fe700 1 -- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e440af000 msgr2=0x7f0e44160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e437fe700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e440af000 0x7f0e44160660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e437fe700 1 -- 172.21.15.112:0/4207379305 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e38009430 con 0x7f0e440ad430 2023-08-27T11:28:53.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e43fff700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e44146d00 0x7f0e441f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:53.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e42ffd700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e440af000 0x7f0e44160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:53.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e437fe700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e440ad430 0x7f0e4415c6d0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400c4d0 tx=0x7f0e3400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:53.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e2bfff700 1 -- 172.21.15.112:0/4207379305 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e340074e0 con 0x7f0e440ad430 2023-08-27T11:28:53.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e2bfff700 1 -- 172.21.15.112:0/4207379305 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e34007660 con 0x7f0e440ad430 2023-08-27T11:28:53.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e49f75700 1 -- 172.21.15.112:0/4207379305 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e44134a40 con 0x7f0e440ad430 2023-08-27T11:28:53.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e2bfff700 1 -- 172.21.15.112:0/4207379305 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34005a70 con 0x7f0e440ad430 2023-08-27T11:28:53.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.583+0000 7f0e49f75700 1 -- 172.21.15.112:0/4207379305 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e441f6d50 con 0x7f0e440ad430 2023-08-27T11:28:53.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.584+0000 7f0e49f75700 1 -- 172.21.15.112:0/4207379305 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e441604b0 con 0x7f0e440ad430 2023-08-27T11:28:53.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.584+0000 7f0e2bfff700 1 -- 172.21.15.112:0/4207379305 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f0e34005bd0 con 0x7f0e440ad430 2023-08-27T11:28:53.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.585+0000 7f0e2bfff700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e3006a4d0 0x7f0e3006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.585+0000 7f0e2bfff700 1 -- 172.21.15.112:0/4207379305 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e340880a0 con 0x7f0e440ad430 2023-08-27T11:28:53.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.585+0000 7f0e42ffd700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e3006a4d0 0x7f0e3006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.586+0000 7f0e42ffd700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e3006a4d0 0x7f0e3006c990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0e38013040 tx=0x7f0e38008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:53.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.589+0000 7f0e2bfff700 1 -- 172.21.15.112:0/4207379305 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0e34053240 con 0x7f0e440ad430 2023-08-27T11:28:53.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.819+0000 7f0e49f75700 1 -- 172.21.15.112:0/4207379305 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e44067a20 con 0x7f0e440ad430 2023-08-27T11:28:53.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.819+0000 7f0e2bfff700 1 -- 172.21.15.112:0/4207379305 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4891 (secure 0 0 0) 0x7f0e34056860 con 0x7f0e440ad430 2023-08-27T11:28:53.821 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:53.821 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6980,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/877756033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":877756033},{"type":"v1","addr":"172.21.15.138:6837","nonce":877756033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6983,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2624496097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2624496097},{"type":"v1","addr":"172.21.15.138:6836","nonce":2624496097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":8940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2687911818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2687911818},{"type":"v1","addr":"172.21.15.112:6834","nonce":2687911818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:49.824795+0000","modified":"2023-08-27T11:28:53.433019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8943":{"gid":8943,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/577891555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":577891555},{"type":"v1","addr":"172.21.15.112:6835","nonce":577891555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-08-27T11:28:53.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.821+0000 7f0e49f75700 1 -- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e3006a4d0 msgr2=0x7f0e3006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.821+0000 7f0e49f75700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e3006a4d0 0x7f0e3006c990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0e38013040 tx=0x7f0e38008070 comp rx=0 tx=0).stop 2023-08-27T11:28:53.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.821+0000 7f0e49f75700 1 -- 172.21.15.112:0/4207379305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e440ad430 msgr2=0x7f0e4415c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.821+0000 7f0e49f75700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e440ad430 0x7f0e4415c6d0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400c4d0 tx=0x7f0e3400c890 comp rx=0 tx=0).stop 2023-08-27T11:28:53.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.822+0000 7f0e49f75700 1 -- 172.21.15.112:0/4207379305 shutdown_connections 2023-08-27T11:28:53.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.822+0000 7f0e49f75700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e3006a4d0 0x7f0e3006c990 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.822+0000 7f0e49f75700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e44146d00 0x7f0e441f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.822+0000 7f0e49f75700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e440ad430 0x7f0e4415c6d0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.822+0000 7f0e49f75700 1 --2- 172.21.15.112:0/4207379305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e440af000 0x7f0e44160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.822+0000 7f0e49f75700 1 -- 172.21.15.112:0/4207379305 >> 172.21.15.112:0/4207379305 conn(0x7f0e4406adb0 msgr2=0x7f0e44140740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.822+0000 7f0e49f75700 1 -- 172.21.15.112:0/4207379305 shutdown_connections 2023-08-27T11:28:53.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.822+0000 7f0e49f75700 1 -- 172.21.15.112:0/4207379305 wait complete. 2023-08-27T11:28:53.826 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 104 2023-08-27T11:28:53.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:53.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.953+0000 7fd466d2e700 1 Processor -- start 2023-08-27T11:28:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.954+0000 7fd466d2e700 1 -- start start 2023-08-27T11:28:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.954+0000 7fd466d2e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd4600b17e0 0x7fd4600cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.954+0000 7fd466d2e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460161ce0 0x7fd46006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.954+0000 7fd466d2e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd460161850 con 0x7fd4600b1400 2023-08-27T11:28:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.954+0000 7fd466d2e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4600cfc10 con 0x7fd4600b17e0 2023-08-27T11:28:53.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.954+0000 7fd466d2e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd460161200 con 0x7fd460161ce0 2023-08-27T11:28:53.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.955+0000 7fd45ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd4600b17e0 0x7fd4600cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.955+0000 7fd45ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd4600b17e0 0x7fd4600cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37122/0 (socket says 172.21.15.112:37122) 2023-08-27T11:28:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.955+0000 7fd45ffff700 1 -- 172.21.15.112:0/1233505886 learned_addr learned my addr 172.21.15.112:0/1233505886 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.955+0000 7fd4652cb700 1 --2- 172.21.15.112:0/1233505886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460161ce0 0x7fd46006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.955+0000 7fd45ffff700 1 -- 172.21.15.112:0/1233505886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460161ce0 msgr2=0x7fd46006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.955+0000 7fd45ffff700 1 --2- 172.21.15.112:0/1233505886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460161ce0 0x7fd46006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.955+0000 7fd45ffff700 1 -- 172.21.15.112:0/1233505886 >> v1:172.21.15.138:6789/0 conn(0x7fd4600b1400 legacy=0x7fd46006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.955+0000 7fd45ffff700 1 -- 172.21.15.112:0/1233505886 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4601614e0 con 0x7fd4600b17e0 2023-08-27T11:28:53.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.955+0000 7fd45ffff700 1 --2- 172.21.15.112:0/1233505886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b17e0 0x7fd4600cf5a0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0098f0 tx=0x7fd44c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bf25caf79b3c7f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:53.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.955+0000 7fd45f7fe700 1 -- 172.21.15.112:0/1233505886 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00b8a0 con 0x7fd4600b17e0 2023-08-27T11:28:53.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.955+0000 7fd45f7fe700 1 -- 172.21.15.112:0/1233505886 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c00ba00 con 0x7fd4600b17e0 2023-08-27T11:28:53.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.955+0000 7fd45f7fe700 1 -- 172.21.15.112:0/1233505886 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00bd90 con 0x7fd4600b17e0 2023-08-27T11:28:53.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.956+0000 7fd466d2e700 1 -- 172.21.15.112:0/1233505886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b17e0 msgr2=0x7fd4600cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.956+0000 7fd466d2e700 1 --2- 172.21.15.112:0/1233505886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b17e0 0x7fd4600cf5a0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0098f0 tx=0x7fd44c009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.956+0000 7fd466d2e700 1 -- 172.21.15.112:0/1233505886 shutdown_connections 2023-08-27T11:28:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.956+0000 7fd466d2e700 1 --2- 172.21.15.112:0/1233505886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460161ce0 0x7fd46006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.956+0000 7fd466d2e700 1 --2- 172.21.15.112:0/1233505886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b17e0 0x7fd4600cf5a0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.956+0000 7fd466d2e700 1 -- 172.21.15.112:0/1233505886 >> 172.21.15.112:0/1233505886 conn(0x7fd4600d0000 msgr2=0x7fd4600d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.956+0000 7fd466d2e700 1 -- 172.21.15.112:0/1233505886 shutdown_connections 2023-08-27T11:28:53.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.956+0000 7fd466d2e700 1 -- 172.21.15.112:0/1233505886 wait complete. 2023-08-27T11:28:53.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.957+0000 7fd466d2e700 1 Processor -- start 2023-08-27T11:28:53.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.957+0000 7fd466d2e700 1 -- start start 2023-08-27T11:28:53.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd466d2e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4600b1400 0x7fd4601474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd466d2e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b17e0 0x7fd4600c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd466d2e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460161ce0 0x7fd460147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd466d2e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600c7270 con 0x7fd4600b17e0 2023-08-27T11:28:53.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd466d2e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600c7550 con 0x7fd4600b1400 2023-08-27T11:28:53.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd466d2e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600c7830 con 0x7fd460161ce0 2023-08-27T11:28:53.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd45ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b17e0 0x7fd4600c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd45ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b17e0 0x7fd4600c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37126/0 (socket says 172.21.15.112:37126) 2023-08-27T11:28:53.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd45ffff700 1 -- 172.21.15.112:0/1283402242 learned_addr learned my addr 172.21.15.112:0/1283402242 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:53.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd464aca700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4600b1400 0x7fd4601474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd4652cb700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460161ce0 0x7fd460147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd45ffff700 1 -- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460161ce0 msgr2=0x7fd460147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd45ffff700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460161ce0 0x7fd460147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd45ffff700 1 -- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4600b1400 msgr2=0x7fd4601474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd45ffff700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4600b1400 0x7fd4601474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd45ffff700 1 -- 172.21.15.112:0/1283402242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c009550 con 0x7fd4600b17e0 2023-08-27T11:28:53.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd45ffff700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b17e0 0x7fd4600c8310 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fd44c000c00 tx=0x7fd44c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:53.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd45d7fa700 1 -- 172.21.15.112:0/1283402242 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c004480 con 0x7fd4600b17e0 2023-08-27T11:28:53.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd45d7fa700 1 -- 172.21.15.112:0/1283402242 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c01f070 con 0x7fd4600b17e0 2023-08-27T11:28:53.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.958+0000 7fd466d2e700 1 -- 172.21.15.112:0/1283402242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4600c7b10 con 0x7fd4600b17e0 2023-08-27T11:28:53.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.959+0000 7fd45d7fa700 1 -- 172.21.15.112:0/1283402242 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0038c0 con 0x7fd4600b17e0 2023-08-27T11:28:53.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.959+0000 7fd466d2e700 1 -- 172.21.15.112:0/1283402242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4601479e0 con 0x7fd4600b17e0 2023-08-27T11:28:53.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.959+0000 7fd466d2e700 1 -- 172.21.15.112:0/1283402242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd460067a20 con 0x7fd4600b17e0 2023-08-27T11:28:53.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.960+0000 7fd45d7fa700 1 -- 172.21.15.112:0/1283402242 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fd44c021030 con 0x7fd4600b17e0 2023-08-27T11:28:53.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.960+0000 7fd45d7fa700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd44406a4d0 0x7fd44406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.960+0000 7fd45d7fa700 1 -- 172.21.15.112:0/1283402242 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd44c08a080 con 0x7fd4600b17e0 2023-08-27T11:28:53.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.960+0000 7fd464aca700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd44406a4d0 0x7fd44406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.961+0000 7fd464aca700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd44406a4d0 0x7fd44406c990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd4600c8f00 tx=0x7fd454009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:53.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:53.965+0000 7fd45d7fa700 1 -- 172.21.15.112:0/1283402242 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd44c0bc990 con 0x7fd4600b17e0 2023-08-27T11:28:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.154+0000 7fd466d2e700 1 -- 172.21.15.112:0/1283402242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4600cb3c0 con 0x7fd4600b17e0 2023-08-27T11:28:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.155+0000 7fd45d7fa700 1 -- 172.21.15.112:0/1283402242 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+18327 (secure 0 0 0) 0x7fd44c055450 con 0x7fd4600b17e0 2023-08-27T11:28:54.157 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:54.158 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":153,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:51.405765+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.551922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.772403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.157+0000 7fd466d2e700 1 -- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd44406a4d0 msgr2=0x7fd44406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.157+0000 7fd466d2e700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd44406a4d0 0x7fd44406c990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd4600c8f00 tx=0x7fd454009040 comp rx=0 tx=0).stop 2023-08-27T11:28:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.157+0000 7fd466d2e700 1 -- 172.21.15.112:0/1283402242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b17e0 msgr2=0x7fd4600c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.157+0000 7fd466d2e700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b17e0 0x7fd4600c8310 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fd44c000c00 tx=0x7fd44c004750 comp rx=0 tx=0).stop 2023-08-27T11:28:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.157+0000 7fd466d2e700 1 -- 172.21.15.112:0/1283402242 shutdown_connections 2023-08-27T11:28:54.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.157+0000 7fd466d2e700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd44406a4d0 0x7fd44406c990 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.157+0000 7fd466d2e700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd460161ce0 0x7fd460147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.157+0000 7fd466d2e700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4600b1400 0x7fd4601474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.157+0000 7fd466d2e700 1 --2- 172.21.15.112:0/1283402242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4600b17e0 0x7fd4600c8310 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.157+0000 7fd466d2e700 1 -- 172.21.15.112:0/1283402242 >> 172.21.15.112:0/1283402242 conn(0x7fd4600d0000 msgr2=0x7fd46015ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:54.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.157+0000 7fd466d2e700 1 -- 172.21.15.112:0/1283402242 shutdown_connections 2023-08-27T11:28:54.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.157+0000 7fd466d2e700 1 -- 172.21.15.112:0/1283402242 wait complete. 2023-08-27T11:28:54.171 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:28:54.172 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:54.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.290+0000 7f4592d70700 1 Processor -- start 2023-08-27T11:28:54.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.291+0000 7f4592d70700 1 -- start start 2023-08-27T11:28:54.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.291+0000 7f4592d70700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f458c140530 0x7f458c0ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.291+0000 7f4592d70700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c140910 con 0x7f458c06e150 2023-08-27T11:28:54.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.291+0000 7f4592d70700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c143b30 con 0x7f458c0cd3e0 2023-08-27T11:28:54.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.291+0000 7f4592d70700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c161dd0 con 0x7f458c140530 2023-08-27T11:28:54.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.291+0000 7f458bfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f458c06e150 0x7f458c160fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59748/0 (socket says 172.21.15.112:59748) 2023-08-27T11:28:54.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.291+0000 7f458bfff700 1 -- 172.21.15.112:0/1612401380 learned_addr learned my addr 172.21.15.112:0/1612401380 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:54.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.292+0000 7f4590b0c700 1 --2- 172.21.15.112:0/1612401380 >> v2:172.21.15.138:3300/0 conn(0x7f458c140530 0x7f458c0ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.292+0000 7f458b7fe700 1 -- 172.21.15.112:0/1612401380 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001706404 0 0) 0x7f4574001110 con 0x7f458c06e150 2023-08-27T11:28:54.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.292+0000 7f458b7fe700 1 -- 172.21.15.112:0/1612401380 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45780035b0 con 0x7f458c06e150 2023-08-27T11:28:54.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.292+0000 7f458b7fe700 1 -- 172.21.15.112:0/1612401380 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914291962 0 0) 0x7f458c143b30 con 0x7f458c0cd3e0 2023-08-27T11:28:54.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.292+0000 7f458b7fe700 1 -- 172.21.15.112:0/1612401380 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4574001110 con 0x7f458c0cd3e0 2023-08-27T11:28:54.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.292+0000 7f4590b0c700 1 -- 172.21.15.112:0/1612401380 >> v1:172.21.15.138:6789/0 conn(0x7f458c0cd3e0 legacy=0x7f458c1616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.292+0000 7f4590b0c700 1 -- 172.21.15.112:0/1612401380 >> v1:172.21.15.112:6789/0 conn(0x7f458c06e150 legacy=0x7f458c160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.292+0000 7f458b7fe700 1 -- 172.21.15.112:0/1612401380 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2880002649 0 0) 0x7f45780035b0 con 0x7f458c06e150 2023-08-27T11:28:54.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.292+0000 7f4590b0c700 1 -- 172.21.15.112:0/1612401380 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c142c40 con 0x7f458c140530 2023-08-27T11:28:54.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.292+0000 7f4590b0c700 1 --2- 172.21.15.112:0/1612401380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c140530 0x7f458c0ad090 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4584009d90 tx=0x7f458400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfe300bfe103eda4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.292+0000 7f458b7fe700 1 -- 172.21.15.112:0/1612401380 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458400e070 con 0x7f458c140530 2023-08-27T11:28:54.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.292+0000 7f458b7fe700 1 -- 172.21.15.112:0/1612401380 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4584003a20 con 0x7f458c140530 2023-08-27T11:28:54.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.292+0000 7f458b7fe700 1 -- 172.21.15.112:0/1612401380 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574001800 con 0x7f458c140530 2023-08-27T11:28:54.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.293+0000 7f4592d70700 1 -- 172.21.15.112:0/1612401380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c140530 msgr2=0x7f458c0ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.293+0000 7f4592d70700 1 --2- 172.21.15.112:0/1612401380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c140530 0x7f458c0ad090 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4584009d90 tx=0x7f458400b330 comp rx=0 tx=0).stop 2023-08-27T11:28:54.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.293+0000 7f4592d70700 1 -- 172.21.15.112:0/1612401380 shutdown_connections 2023-08-27T11:28:54.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.293+0000 7f4592d70700 1 --2- 172.21.15.112:0/1612401380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c140530 0x7f458c0ad090 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.293+0000 7f4592d70700 1 -- 172.21.15.112:0/1612401380 >> 172.21.15.112:0/1612401380 conn(0x7f458c06cd30 msgr2=0x7f458c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:54.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.293+0000 7f4592d70700 1 -- 172.21.15.112:0/1612401380 shutdown_connections 2023-08-27T11:28:54.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.293+0000 7f4592d70700 1 -- 172.21.15.112:0/1612401380 wait complete. 2023-08-27T11:28:54.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.294+0000 7f4592d70700 1 Processor -- start 2023-08-27T11:28:54.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.294+0000 7f4592d70700 1 -- start start 2023-08-27T11:28:54.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f4592d70700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c06e150 0x7f458c071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f4592d70700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f458c0cd3e0 0x7f458c1f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f4592d70700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f458c140530 0x7f458c1f2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f4592d70700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c138600 con 0x7f458c140530 2023-08-27T11:28:54.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f4592d70700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c1f6810 con 0x7f458c06e150 2023-08-27T11:28:54.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f4592d70700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c1f2bb0 con 0x7f458c0cd3e0 2023-08-27T11:28:54.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f459130d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f458c140530 0x7f458c1f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f459130d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f458c140530 0x7f458c1f2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37142/0 (socket says 172.21.15.112:37142) 2023-08-27T11:28:54.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f459130d700 1 -- 172.21.15.112:0/3670015787 learned_addr learned my addr 172.21.15.112:0/3670015787 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:54.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f4590b0c700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c06e150 0x7f458c071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f458bfff700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f458c0cd3e0 0x7f458c1f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f459130d700 1 -- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f458c0cd3e0 msgr2=0x7f458c1f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f459130d700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f458c0cd3e0 0x7f458c1f62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f459130d700 1 -- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c06e150 msgr2=0x7f458c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f459130d700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c06e150 0x7f458c071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.295+0000 7f459130d700 1 -- 172.21.15.112:0/3670015787 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4584009a70 con 0x7f458c140530 2023-08-27T11:28:54.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.296+0000 7f4590b0c700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c06e150 0x7f458c071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:54.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.296+0000 7f459130d700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f458c140530 0x7f458c1f2670 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f457c00c490 tx=0x7f457c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.296+0000 7f45897fa700 1 -- 172.21.15.112:0/3670015787 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c0052d0 con 0x7f458c140530 2023-08-27T11:28:54.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.296+0000 7f45897fa700 1 -- 172.21.15.112:0/3670015787 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457c005430 con 0x7f458c140530 2023-08-27T11:28:54.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.296+0000 7f4592d70700 1 -- 172.21.15.112:0/3670015787 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c1f6ab0 con 0x7f458c140530 2023-08-27T11:28:54.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.296+0000 7f45897fa700 1 -- 172.21.15.112:0/3670015787 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c0038c0 con 0x7f458c140530 2023-08-27T11:28:54.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.296+0000 7f4592d70700 1 -- 172.21.15.112:0/3670015787 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c1f7070 con 0x7f458c140530 2023-08-27T11:28:54.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.297+0000 7f4592d70700 1 -- 172.21.15.112:0/3670015787 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c067a20 con 0x7f458c140530 2023-08-27T11:28:54.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.297+0000 7f45897fa700 1 -- 172.21.15.112:0/3670015787 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f457c003a20 con 0x7f458c140530 2023-08-27T11:28:54.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.297+0000 7f45897fa700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f456c06a4d0 0x7f456c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.298+0000 7f45897fa700 1 -- 172.21.15.112:0/3670015787 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f457c088110 con 0x7f458c140530 2023-08-27T11:28:54.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.298+0000 7f4590b0c700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f456c06a4d0 0x7f456c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.298+0000 7f4590b0c700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f456c06a4d0 0x7f456c06c990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4584000f80 tx=0x7f4584011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.302+0000 7f45897fa700 1 -- 172.21.15.112:0/3670015787 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f457c0532b0 con 0x7f458c140530 2023-08-27T11:28:54.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.490+0000 7f4592d70700 1 -- 172.21.15.112:0/3670015787 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f458c143200 con 0x7f456c06a4d0 2023-08-27T11:28:54.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.499+0000 7f45897fa700 1 -- 172.21.15.112:0/3670015787 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312309 (secure 0 0 0) 0x7f4584003d80 con 0x7f456c06a4d0 2023-08-27T11:28:54.504 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:54.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.502+0000 7f4592d70700 1 -- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f456c06a4d0 msgr2=0x7f456c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.502+0000 7f4592d70700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f456c06a4d0 0x7f456c06c990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4584000f80 tx=0x7f4584011000 comp rx=0 tx=0).stop 2023-08-27T11:28:54.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.502+0000 7f4592d70700 1 -- 172.21.15.112:0/3670015787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f458c140530 msgr2=0x7f458c1f2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.502+0000 7f4592d70700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f458c140530 0x7f458c1f2670 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f457c00c490 tx=0x7f457c00c850 comp rx=0 tx=0).stop 2023-08-27T11:28:54.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.502+0000 7f4592d70700 1 -- 172.21.15.112:0/3670015787 shutdown_connections 2023-08-27T11:28:54.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.502+0000 7f4592d70700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f456c06a4d0 0x7f456c06c990 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.502+0000 7f4592d70700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f458c140530 0x7f458c1f2670 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.502+0000 7f4592d70700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c06e150 0x7f458c071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.502+0000 7f4592d70700 1 --2- 172.21.15.112:0/3670015787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f458c0cd3e0 0x7f458c1f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.502+0000 7f4592d70700 1 -- 172.21.15.112:0/3670015787 >> 172.21.15.112:0/3670015787 conn(0x7f458c06cd30 msgr2=0x7f458c15d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:54.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.502+0000 7f4592d70700 1 -- 172.21.15.112:0/3670015787 shutdown_connections 2023-08-27T11:28:54.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.502+0000 7f4592d70700 1 -- 172.21.15.112:0/3670015787 wait complete. 2023-08-27T11:28:54.510 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:28:54.516 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2023-08-27T11:28:54.438291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265780,"kb_used_data":16308,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754708940,"statfs":{"total":773094113280,"available":772821954560,"internally_reserved":0,"allocated":16699392,"data_stored":8538599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.028679"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320596+0000","last_change":"2023-08-27T11:28:50.426272+0000","last_active":"2023-08-27T11:28:52.320596+0000","last_peered":"2023-08-27T11:28:52.320596+0000","last_clean":"2023-08-27T11:28:52.320596+0000","last_became_active":"2023-08-27T11:28:50.425977+0000","last_became_peered":"2023-08-27T11:28:50.425977+0000","last_unstale":"2023-08-27T11:28:52.320596+0000","last_undegraded":"2023-08-27T11:28:52.320596+0000","last_fullsized":"2023-08-27T11:28:52.320596+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:32:01.714311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413612+0000","last_change":"2023-08-27T11:28:50.423791+0000","last_active":"2023-08-27T11:28:51.413612+0000","last_peered":"2023-08-27T11:28:51.413612+0000","last_clean":"2023-08-27T11:28:51.413612+0000","last_became_active":"2023-08-27T11:28:50.423541+0000","last_became_peered":"2023-08-27T11:28:50.423541+0000","last_unstale":"2023-08-27T11:28:51.413612+0000","last_undegraded":"2023-08-27T11:28:51.413612+0000","last_fullsized":"2023-08-27T11:28:51.413612+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:23.585726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076493+0000","last_change":"2023-08-27T11:28:50.429104+0000","last_active":"2023-08-27T11:28:52.076493+0000","last_peered":"2023-08-27T11:28:52.076493+0000","last_clean":"2023-08-27T11:28:52.076493+0000","last_became_active":"2023-08-27T11:28:50.428891+0000","last_became_peered":"2023-08-27T11:28:50.428891+0000","last_unstale":"2023-08-27T11:28:52.076493+0000","last_undegraded":"2023-08-27T11:28:52.076493+0000","last_fullsized":"2023-08-27T11:28:52.076493+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:13:04.498709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412355+0000","last_change":"2023-08-27T11:28:50.421808+0000","last_active":"2023-08-27T11:28:51.412355+0000","last_peered":"2023-08-27T11:28:51.412355+0000","last_clean":"2023-08-27T11:28:51.412355+0000","last_became_active":"2023-08-27T11:28:50.420778+0000","last_became_peered":"2023-08-27T11:28:50.420778+0000","last_unstale":"2023-08-27T11:28:51.412355+0000","last_undegraded":"2023-08-27T11:28:51.412355+0000","last_fullsized":"2023-08-27T11:28:51.412355+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:40.391553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435787+0000","last_change":"2023-08-27T11:28:50.432586+0000","last_active":"2023-08-27T11:28:50.435787+0000","last_peered":"2023-08-27T11:28:50.435787+0000","last_clean":"2023-08-27T11:28:50.435787+0000","last_became_active":"2023-08-27T11:28:50.432334+0000","last_became_peered":"2023-08-27T11:28:50.432334+0000","last_unstale":"2023-08-27T11:28:50.435787+0000","last_undegraded":"2023-08-27T11:28:50.435787+0000","last_fullsized":"2023-08-27T11:28:50.435787+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:55:50.393935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412202+0000","last_change":"2023-08-27T11:28:50.429089+0000","last_active":"2023-08-27T11:28:51.412202+0000","last_peered":"2023-08-27T11:28:51.412202+0000","last_clean":"2023-08-27T11:28:51.412202+0000","last_became_active":"2023-08-27T11:28:50.428814+0000","last_became_peered":"2023-08-27T11:28:50.428814+0000","last_unstale":"2023-08-27T11:28:51.412202+0000","last_undegraded":"2023-08-27T11:28:51.412202+0000","last_fullsized":"2023-08-27T11:28:51.412202+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:56.436880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435811+0000","last_change":"2023-08-27T11:28:50.432231+0000","last_active":"2023-08-27T11:28:50.435811+0000","last_peered":"2023-08-27T11:28:50.435811+0000","last_clean":"2023-08-27T11:28:50.435811+0000","last_became_active":"2023-08-27T11:28:50.431793+0000","last_became_peered":"2023-08-27T11:28:50.431793+0000","last_unstale":"2023-08-27T11:28:50.435811+0000","last_undegraded":"2023-08-27T11:28:50.435811+0000","last_fullsized":"2023-08-27T11:28:50.435811+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:42:42.911403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413533+0000","last_change":"2023-08-27T11:28:50.419372+0000","last_active":"2023-08-27T11:28:51.413533+0000","last_peered":"2023-08-27T11:28:51.413533+0000","last_clean":"2023-08-27T11:28:51.413533+0000","last_became_active":"2023-08-27T11:28:50.419084+0000","last_became_peered":"2023-08-27T11:28:50.419084+0000","last_unstale":"2023-08-27T11:28:51.413533+0000","last_undegraded":"2023-08-27T11:28:51.413533+0000","last_fullsized":"2023-08-27T11:28:51.413533+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:55:00.360756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.427772+0000","last_change":"2023-08-27T11:28:50.426775+0000","last_active":"2023-08-27T11:28:50.427772+0000","last_peered":"2023-08-27T11:28:50.427772+0000","last_clean":"2023-08-27T11:28:50.427772+0000","last_became_active":"2023-08-27T11:28:50.426521+0000","last_became_peered":"2023-08-27T11:28:50.426521+0000","last_unstale":"2023-08-27T11:28:50.427772+0000","last_undegraded":"2023-08-27T11:28:50.427772+0000","last_fullsized":"2023-08-27T11:28:50.427772+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:41:57.204734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414199+0000","last_change":"2023-08-27T11:28:50.427188+0000","last_active":"2023-08-27T11:28:51.414199+0000","last_peered":"2023-08-27T11:28:51.414199+0000","last_clean":"2023-08-27T11:28:51.414199+0000","last_became_active":"2023-08-27T11:28:50.426870+0000","last_became_peered":"2023-08-27T11:28:50.426870+0000","last_unstale":"2023-08-27T11:28:51.414199+0000","last_undegraded":"2023-08-27T11:28:51.414199+0000","last_fullsized":"2023-08-27T11:28:51.414199+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:20:57.365468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076481+0000","last_change":"2023-08-27T11:28:50.422020+0000","last_active":"2023-08-27T11:28:52.076481+0000","last_peered":"2023-08-27T11:28:52.076481+0000","last_clean":"2023-08-27T11:28:52.076481+0000","last_became_active":"2023-08-27T11:28:50.421491+0000","last_became_peered":"2023-08-27T11:28:50.421491+0000","last_unstale":"2023-08-27T11:28:52.076481+0000","last_undegraded":"2023-08-27T11:28:52.076481+0000","last_fullsized":"2023-08-27T11:28:52.076481+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:24:54.543099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.434609+0000","last_change":"2023-08-27T11:28:50.429548+0000","last_active":"2023-08-27T11:28:50.434609+0000","last_peered":"2023-08-27T11:28:50.434609+0000","last_clean":"2023-08-27T11:28:50.434609+0000","last_became_active":"2023-08-27T11:28:50.429304+0000","last_became_peered":"2023-08-27T11:28:50.429304+0000","last_unstale":"2023-08-27T11:28:50.434609+0000","last_undegraded":"2023-08-27T11:28:50.434609+0000","last_fullsized":"2023-08-27T11:28:50.434609+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:53:22.814103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412477+0000","last_change":"2023-08-27T11:28:50.433446+0000","last_active":"2023-08-27T11:28:51.412477+0000","last_peered":"2023-08-27T11:28:51.412477+0000","last_clean":"2023-08-27T11:28:51.412477+0000","last_became_active":"2023-08-27T11:28:50.433172+0000","last_became_peered":"2023-08-27T11:28:50.433172+0000","last_unstale":"2023-08-27T11:28:51.412477+0000","last_undegraded":"2023-08-27T11:28:51.412477+0000","last_fullsized":"2023-08-27T11:28:51.412477+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:52:22.634803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413701+0000","last_change":"2023-08-27T11:28:50.426603+0000","last_active":"2023-08-27T11:28:51.413701+0000","last_peered":"2023-08-27T11:28:51.413701+0000","last_clean":"2023-08-27T11:28:51.413701+0000","last_became_active":"2023-08-27T11:28:50.426346+0000","last_became_peered":"2023-08-27T11:28:50.426346+0000","last_unstale":"2023-08-27T11:28:51.413701+0000","last_undegraded":"2023-08-27T11:28:51.413701+0000","last_fullsized":"2023-08-27T11:28:51.413701+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:03:42.841698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.429244+0000","last_change":"2023-08-27T11:28:50.418544+0000","last_active":"2023-08-27T11:28:50.429244+0000","last_peered":"2023-08-27T11:28:50.429244+0000","last_clean":"2023-08-27T11:28:50.429244+0000","last_became_active":"2023-08-27T11:28:50.418180+0000","last_became_peered":"2023-08-27T11:28:50.418180+0000","last_unstale":"2023-08-27T11:28:50.429244+0000","last_undegraded":"2023-08-27T11:28:50.429244+0000","last_fullsized":"2023-08-27T11:28:50.429244+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:03:04.318857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076414+0000","last_change":"2023-08-27T11:28:50.427135+0000","last_active":"2023-08-27T11:28:52.076414+0000","last_peered":"2023-08-27T11:28:52.076414+0000","last_clean":"2023-08-27T11:28:52.076414+0000","last_became_active":"2023-08-27T11:28:50.426858+0000","last_became_peered":"2023-08-27T11:28:50.426858+0000","last_unstale":"2023-08-27T11:28:52.076414+0000","last_undegraded":"2023-08-27T11:28:52.076414+0000","last_fullsized":"2023-08-27T11:28:52.076414+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:43.579659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.424895+0000","last_change":"2023-08-27T11:28:50.418834+0000","last_active":"2023-08-27T11:28:50.424895+0000","last_peered":"2023-08-27T11:28:50.424895+0000","last_clean":"2023-08-27T11:28:50.424895+0000","last_became_active":"2023-08-27T11:28:50.418051+0000","last_became_peered":"2023-08-27T11:28:50.418051+0000","last_unstale":"2023-08-27T11:28:50.424895+0000","last_undegraded":"2023-08-27T11:28:50.424895+0000","last_fullsized":"2023-08-27T11:28:50.424895+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:29:18.564808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.424871+0000","last_change":"2023-08-27T11:28:50.417739+0000","last_active":"2023-08-27T11:28:50.424871+0000","last_peered":"2023-08-27T11:28:50.424871+0000","last_clean":"2023-08-27T11:28:50.424871+0000","last_became_active":"2023-08-27T11:28:50.417419+0000","last_became_peered":"2023-08-27T11:28:50.417419+0000","last_unstale":"2023-08-27T11:28:50.424871+0000","last_undegraded":"2023-08-27T11:28:50.424871+0000","last_fullsized":"2023-08-27T11:28:50.424871+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:54:37.318106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414333+0000","last_change":"2023-08-27T11:28:50.414165+0000","last_active":"2023-08-27T11:28:51.414333+0000","last_peered":"2023-08-27T11:28:51.414333+0000","last_clean":"2023-08-27T11:28:51.414333+0000","last_became_active":"2023-08-27T11:28:50.413894+0000","last_became_peered":"2023-08-27T11:28:50.413894+0000","last_unstale":"2023-08-27T11:28:51.414333+0000","last_undegraded":"2023-08-27T11:28:51.414333+0000","last_fullsized":"2023-08-27T11:28:51.414333+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:10:31.264961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414149+0000","last_change":"2023-08-27T11:28:50.419559+0000","last_active":"2023-08-27T11:28:51.414149+0000","last_peered":"2023-08-27T11:28:51.414149+0000","last_clean":"2023-08-27T11:28:51.414149+0000","last_became_active":"2023-08-27T11:28:50.418776+0000","last_became_peered":"2023-08-27T11:28:50.418776+0000","last_unstale":"2023-08-27T11:28:51.414149+0000","last_undegraded":"2023-08-27T11:28:51.414149+0000","last_fullsized":"2023-08-27T11:28:51.414149+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:05.835425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413852+0000","last_change":"2023-08-27T11:28:50.427983+0000","last_active":"2023-08-27T11:28:51.413852+0000","last_peered":"2023-08-27T11:28:51.413852+0000","last_clean":"2023-08-27T11:28:51.413852+0000","last_became_active":"2023-08-27T11:28:50.427673+0000","last_became_peered":"2023-08-27T11:28:50.427673+0000","last_unstale":"2023-08-27T11:28:51.413852+0000","last_undegraded":"2023-08-27T11:28:51.413852+0000","last_fullsized":"2023-08-27T11:28:51.413852+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:41:13.034838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413924+0000","last_change":"2023-08-27T11:28:50.426888+0000","last_active":"2023-08-27T11:28:51.413924+0000","last_peered":"2023-08-27T11:28:51.413924+0000","last_clean":"2023-08-27T11:28:51.413924+0000","last_became_active":"2023-08-27T11:28:50.424404+0000","last_became_peered":"2023-08-27T11:28:50.424404+0000","last_unstale":"2023-08-27T11:28:51.413924+0000","last_undegraded":"2023-08-27T11:28:51.413924+0000","last_fullsized":"2023-08-27T11:28:51.413924+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:25:14.196474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413407+0000","last_change":"2023-08-27T11:28:50.424164+0000","last_active":"2023-08-27T11:28:51.413407+0000","last_peered":"2023-08-27T11:28:51.413407+0000","last_clean":"2023-08-27T11:28:51.413407+0000","last_became_active":"2023-08-27T11:28:50.423858+0000","last_became_peered":"2023-08-27T11:28:50.423858+0000","last_unstale":"2023-08-27T11:28:51.413407+0000","last_undegraded":"2023-08-27T11:28:51.413407+0000","last_fullsized":"2023-08-27T11:28:51.413407+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:57.141092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435181+0000","last_change":"2023-08-27T11:28:50.430081+0000","last_active":"2023-08-27T11:28:50.435181+0000","last_peered":"2023-08-27T11:28:50.435181+0000","last_clean":"2023-08-27T11:28:50.435181+0000","last_became_active":"2023-08-27T11:28:50.429768+0000","last_became_peered":"2023-08-27T11:28:50.429768+0000","last_unstale":"2023-08-27T11:28:50.435181+0000","last_undegraded":"2023-08-27T11:28:50.435181+0000","last_fullsized":"2023-08-27T11:28:50.435181+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:00:03.880826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.424912+0000","last_change":"2023-08-27T11:28:50.418988+0000","last_active":"2023-08-27T11:28:50.424912+0000","last_peered":"2023-08-27T11:28:50.424912+0000","last_clean":"2023-08-27T11:28:50.424912+0000","last_became_active":"2023-08-27T11:28:50.418413+0000","last_became_peered":"2023-08-27T11:28:50.418413+0000","last_unstale":"2023-08-27T11:28:50.424912+0000","last_undegraded":"2023-08-27T11:28:50.424912+0000","last_fullsized":"2023-08-27T11:28:50.424912+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:06:21.397831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320327+0000","last_change":"2023-08-27T11:28:50.425372+0000","last_active":"2023-08-27T11:28:52.320327+0000","last_peered":"2023-08-27T11:28:52.320327+0000","last_clean":"2023-08-27T11:28:52.320327+0000","last_became_active":"2023-08-27T11:28:50.425076+0000","last_became_peered":"2023-08-27T11:28:50.425076+0000","last_unstale":"2023-08-27T11:28:52.320327+0000","last_undegraded":"2023-08-27T11:28:52.320327+0000","last_fullsized":"2023-08-27T11:28:52.320327+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:19:13.216408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076634+0000","last_change":"2023-08-27T11:28:50.421871+0000","last_active":"2023-08-27T11:28:52.076634+0000","last_peered":"2023-08-27T11:28:52.076634+0000","last_clean":"2023-08-27T11:28:52.076634+0000","last_became_active":"2023-08-27T11:28:50.421179+0000","last_became_peered":"2023-08-27T11:28:50.421179+0000","last_unstale":"2023-08-27T11:28:52.076634+0000","last_undegraded":"2023-08-27T11:28:52.076634+0000","last_fullsized":"2023-08-27T11:28:52.076634+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:01:42.017950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.434630+0000","last_change":"2023-08-27T11:28:50.430196+0000","last_active":"2023-08-27T11:28:50.434630+0000","last_peered":"2023-08-27T11:28:50.434630+0000","last_clean":"2023-08-27T11:28:50.434630+0000","last_became_active":"2023-08-27T11:28:50.429783+0000","last_became_peered":"2023-08-27T11:28:50.429783+0000","last_unstale":"2023-08-27T11:28:50.434630+0000","last_undegraded":"2023-08-27T11:28:50.434630+0000","last_fullsized":"2023-08-27T11:28:50.434630+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:19.316875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.419081+0000","last_change":"2023-08-27T11:28:50.414443+0000","last_active":"2023-08-27T11:28:50.419081+0000","last_peered":"2023-08-27T11:28:50.419081+0000","last_clean":"2023-08-27T11:28:50.419081+0000","last_became_active":"2023-08-27T11:28:50.414190+0000","last_became_peered":"2023-08-27T11:28:50.414190+0000","last_unstale":"2023-08-27T11:28:50.419081+0000","last_undegraded":"2023-08-27T11:28:50.419081+0000","last_fullsized":"2023-08-27T11:28:50.419081+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:09:13.623657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.431982+0000","last_change":"2023-08-27T11:28:50.426704+0000","last_active":"2023-08-27T11:28:50.431982+0000","last_peered":"2023-08-27T11:28:50.431982+0000","last_clean":"2023-08-27T11:28:50.431982+0000","last_became_active":"2023-08-27T11:28:50.426432+0000","last_became_peered":"2023-08-27T11:28:50.426432+0000","last_unstale":"2023-08-27T11:28:50.431982+0000","last_undegraded":"2023-08-27T11:28:50.431982+0000","last_fullsized":"2023-08-27T11:28:50.431982+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:40.667390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412134+0000","last_change":"2023-08-27T11:28:50.426515+0000","last_active":"2023-08-27T11:28:51.412134+0000","last_peered":"2023-08-27T11:28:51.412134+0000","last_clean":"2023-08-27T11:28:51.412134+0000","last_became_active":"2023-08-27T11:28:50.426217+0000","last_became_peered":"2023-08-27T11:28:50.426217+0000","last_unstale":"2023-08-27T11:28:51.412134+0000","last_undegraded":"2023-08-27T11:28:51.412134+0000","last_fullsized":"2023-08-27T11:28:51.412134+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:55:25.282686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412404+0000","last_change":"2023-08-27T11:28:50.422133+0000","last_active":"2023-08-27T11:28:51.412404+0000","last_peered":"2023-08-27T11:28:51.412404+0000","last_clean":"2023-08-27T11:28:51.412404+0000","last_became_active":"2023-08-27T11:28:50.421864+0000","last_became_peered":"2023-08-27T11:28:50.421864+0000","last_unstale":"2023-08-27T11:28:51.412404+0000","last_undegraded":"2023-08-27T11:28:51.412404+0000","last_fullsized":"2023-08-27T11:28:51.412404+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:14:30.582313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.428894+0000","last_change":"2023-08-27T11:28:50.419387+0000","last_active":"2023-08-27T11:28:50.428894+0000","last_peered":"2023-08-27T11:28:50.428894+0000","last_clean":"2023-08-27T11:28:50.428894+0000","last_became_active":"2023-08-27T11:28:50.419084+0000","last_became_peered":"2023-08-27T11:28:50.419084+0000","last_unstale":"2023-08-27T11:28:50.428894+0000","last_undegraded":"2023-08-27T11:28:50.428894+0000","last_fullsized":"2023-08-27T11:28:50.428894+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:16:14.370020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.417910+0000","last_change":"2023-08-27T11:28:49.418410+0000","last_active":"2023-08-27T11:28:50.417910+0000","last_peered":"2023-08-27T11:28:50.417910+0000","last_clean":"2023-08-27T11:28:50.417910+0000","last_became_active":"2023-08-27T11:28:49.418118+0000","last_became_peered":"2023-08-27T11:28:49.418118+0000","last_unstale":"2023-08-27T11:28:50.417910+0000","last_undegraded":"2023-08-27T11:28:50.417910+0000","last_fullsized":"2023-08-27T11:28:50.417910+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:30:12.138330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076523+0000","last_change":"2023-08-27T11:28:50.430755+0000","last_active":"2023-08-27T11:28:52.076523+0000","last_peered":"2023-08-27T11:28:52.076523+0000","last_clean":"2023-08-27T11:28:52.076523+0000","last_became_active":"2023-08-27T11:28:50.430503+0000","last_became_peered":"2023-08-27T11:28:50.430503+0000","last_unstale":"2023-08-27T11:28:52.076523+0000","last_undegraded":"2023-08-27T11:28:52.076523+0000","last_fullsized":"2023-08-27T11:28:52.076523+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:40:52.014836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412880+0000","last_change":"2023-08-27T11:28:49.430522+0000","last_active":"2023-08-27T11:28:51.412880+0000","last_peered":"2023-08-27T11:28:51.412880+0000","last_clean":"2023-08-27T11:28:51.412880+0000","last_became_active":"2023-08-27T11:28:49.430189+0000","last_became_peered":"2023-08-27T11:28:49.430189+0000","last_unstale":"2023-08-27T11:28:51.412880+0000","last_undegraded":"2023-08-27T11:28:51.412880+0000","last_fullsized":"2023-08-27T11:28:51.412880+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:45.562389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.427717+0000","last_change":"2023-08-27T11:28:50.426137+0000","last_active":"2023-08-27T11:28:50.427717+0000","last_peered":"2023-08-27T11:28:50.427717+0000","last_clean":"2023-08-27T11:28:50.427717+0000","last_became_active":"2023-08-27T11:28:50.425879+0000","last_became_peered":"2023-08-27T11:28:50.425879+0000","last_unstale":"2023-08-27T11:28:50.427717+0000","last_undegraded":"2023-08-27T11:28:50.427717+0000","last_fullsized":"2023-08-27T11:28:50.427717+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:19:54.933140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"152'3","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.430085+0000","last_change":"2023-08-27T11:28:49.422279+0000","last_active":"2023-08-27T11:28:50.430085+0000","last_peered":"2023-08-27T11:28:50.430085+0000","last_clean":"2023-08-27T11:28:50.430085+0000","last_became_active":"2023-08-27T11:28:49.422002+0000","last_became_peered":"2023-08-27T11:28:49.422002+0000","last_unstale":"2023-08-27T11:28:50.430085+0000","last_undegraded":"2023-08-27T11:28:50.430085+0000","last_fullsized":"2023-08-27T11:28:50.430085+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:55.438217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435770+0000","last_change":"2023-08-27T11:28:50.432294+0000","last_active":"2023-08-27T11:28:50.435770+0000","last_peered":"2023-08-27T11:28:50.435770+0000","last_clean":"2023-08-27T11:28:50.435770+0000","last_became_active":"2023-08-27T11:28:50.431507+0000","last_became_peered":"2023-08-27T11:28:50.431507+0000","last_unstale":"2023-08-27T11:28:50.435770+0000","last_undegraded":"2023-08-27T11:28:50.435770+0000","last_fullsized":"2023-08-27T11:28:50.435770+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:31:34.560849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"152'6","reported_seq":19,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412540+0000","last_change":"2023-08-27T11:28:49.430593+0000","last_active":"2023-08-27T11:28:51.412540+0000","last_peered":"2023-08-27T11:28:51.412540+0000","last_clean":"2023-08-27T11:28:51.412540+0000","last_became_active":"2023-08-27T11:28:49.430185+0000","last_became_peered":"2023-08-27T11:28:49.430185+0000","last_unstale":"2023-08-27T11:28:51.412540+0000","last_undegraded":"2023-08-27T11:28:51.412540+0000","last_fullsized":"2023-08-27T11:28:51.412540+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:41:42.671975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413584+0000","last_change":"2023-08-27T11:28:50.430732+0000","last_active":"2023-08-27T11:28:51.413584+0000","last_peered":"2023-08-27T11:28:51.413584+0000","last_clean":"2023-08-27T11:28:51.413584+0000","last_became_active":"2023-08-27T11:28:50.430469+0000","last_became_peered":"2023-08-27T11:28:50.430469+0000","last_unstale":"2023-08-27T11:28:51.413584+0000","last_undegraded":"2023-08-27T11:28:51.413584+0000","last_fullsized":"2023-08-27T11:28:51.413584+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:03:22.319458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"152'2","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413664+0000","last_change":"2023-08-27T11:28:49.421568+0000","last_active":"2023-08-27T11:28:51.413664+0000","last_peered":"2023-08-27T11:28:51.413664+0000","last_clean":"2023-08-27T11:28:51.413664+0000","last_became_active":"2023-08-27T11:28:49.421390+0000","last_became_peered":"2023-08-27T11:28:49.421390+0000","last_unstale":"2023-08-27T11:28:51.413664+0000","last_undegraded":"2023-08-27T11:28:51.413664+0000","last_fullsized":"2023-08-27T11:28:51.413664+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:30.430662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076657+0000","last_change":"2023-08-27T11:28:50.421728+0000","last_active":"2023-08-27T11:28:52.076657+0000","last_peered":"2023-08-27T11:28:52.076657+0000","last_clean":"2023-08-27T11:28:52.076657+0000","last_became_active":"2023-08-27T11:28:50.420814+0000","last_became_peered":"2023-08-27T11:28:50.420814+0000","last_unstale":"2023-08-27T11:28:52.076657+0000","last_undegraded":"2023-08-27T11:28:52.076657+0000","last_fullsized":"2023-08-27T11:28:52.076657+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:15.652339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076403+0000","last_change":"2023-08-27T11:28:49.420502+0000","last_active":"2023-08-27T11:28:52.076403+0000","last_peered":"2023-08-27T11:28:52.076403+0000","last_clean":"2023-08-27T11:28:52.076403+0000","last_became_active":"2023-08-27T11:28:49.420253+0000","last_became_peered":"2023-08-27T11:28:49.420253+0000","last_unstale":"2023-08-27T11:28:52.076403+0000","last_undegraded":"2023-08-27T11:28:52.076403+0000","last_fullsized":"2023-08-27T11:28:52.076403+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:05:57.560435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.432920+0000","last_change":"2023-08-27T11:28:50.432278+0000","last_active":"2023-08-27T11:28:50.432920+0000","last_peered":"2023-08-27T11:28:50.432920+0000","last_clean":"2023-08-27T11:28:50.432920+0000","last_became_active":"2023-08-27T11:28:50.431984+0000","last_became_peered":"2023-08-27T11:28:50.431984+0000","last_unstale":"2023-08-27T11:28:50.432920+0000","last_undegraded":"2023-08-27T11:28:50.432920+0000","last_fullsized":"2023-08-27T11:28:50.432920+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:15:52.341802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.407697+0000","last_change":"2023-08-27T11:28:49.425113+0000","last_active":"2023-08-27T11:28:50.407697+0000","last_peered":"2023-08-27T11:28:50.407697+0000","last_clean":"2023-08-27T11:28:50.407697+0000","last_became_active":"2023-08-27T11:28:49.424860+0000","last_became_peered":"2023-08-27T11:28:49.424860+0000","last_unstale":"2023-08-27T11:28:50.407697+0000","last_undegraded":"2023-08-27T11:28:50.407697+0000","last_fullsized":"2023-08-27T11:28:50.407697+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:12:38.264868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412725+0000","last_change":"2023-08-27T11:28:50.423760+0000","last_active":"2023-08-27T11:28:51.412725+0000","last_peered":"2023-08-27T11:28:51.412725+0000","last_clean":"2023-08-27T11:28:51.412725+0000","last_became_active":"2023-08-27T11:28:50.423466+0000","last_became_peered":"2023-08-27T11:28:50.423466+0000","last_unstale":"2023-08-27T11:28:51.412725+0000","last_undegraded":"2023-08-27T11:28:51.412725+0000","last_fullsized":"2023-08-27T11:28:51.412725+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:31:08.006587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"152'8","reported_seq":19,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435230+0000","last_change":"2023-08-27T11:28:49.423618+0000","last_active":"2023-08-27T11:28:50.435230+0000","last_peered":"2023-08-27T11:28:50.435230+0000","last_clean":"2023-08-27T11:28:50.435230+0000","last_became_active":"2023-08-27T11:28:49.423160+0000","last_became_peered":"2023-08-27T11:28:49.423160+0000","last_unstale":"2023-08-27T11:28:50.435230+0000","last_undegraded":"2023-08-27T11:28:50.435230+0000","last_fullsized":"2023-08-27T11:28:50.435230+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:56.773129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413529+0000","last_change":"2023-08-27T11:28:50.428616+0000","last_active":"2023-08-27T11:28:51.413529+0000","last_peered":"2023-08-27T11:28:51.413529+0000","last_clean":"2023-08-27T11:28:51.413529+0000","last_became_active":"2023-08-27T11:28:50.428002+0000","last_became_peered":"2023-08-27T11:28:50.428002+0000","last_unstale":"2023-08-27T11:28:51.413529+0000","last_undegraded":"2023-08-27T11:28:51.413529+0000","last_fullsized":"2023-08-27T11:28:51.413529+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:23:39.038605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"152'1","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320661+0000","last_change":"2023-08-27T11:28:49.423762+0000","last_active":"2023-08-27T11:28:52.320661+0000","last_peered":"2023-08-27T11:28:52.320661+0000","last_clean":"2023-08-27T11:28:52.320661+0000","last_became_active":"2023-08-27T11:28:49.423480+0000","last_became_peered":"2023-08-27T11:28:49.423480+0000","last_unstale":"2023-08-27T11:28:52.320661+0000","last_undegraded":"2023-08-27T11:28:52.320661+0000","last_fullsized":"2023-08-27T11:28:52.320661+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:58:55.021392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320165+0000","last_change":"2023-08-27T11:28:50.427403+0000","last_active":"2023-08-27T11:28:52.320165+0000","last_peered":"2023-08-27T11:28:52.320165+0000","last_clean":"2023-08-27T11:28:52.320165+0000","last_became_active":"2023-08-27T11:28:50.425494+0000","last_became_peered":"2023-08-27T11:28:50.425494+0000","last_unstale":"2023-08-27T11:28:52.320165+0000","last_undegraded":"2023-08-27T11:28:52.320165+0000","last_fullsized":"2023-08-27T11:28:52.320165+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:04:25.612634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"152'2","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320499+0000","last_change":"2023-08-27T11:28:49.427032+0000","last_active":"2023-08-27T11:28:52.320499+0000","last_peered":"2023-08-27T11:28:52.320499+0000","last_clean":"2023-08-27T11:28:52.320499+0000","last_became_active":"2023-08-27T11:28:49.426675+0000","last_became_peered":"2023-08-27T11:28:49.426675+0000","last_unstale":"2023-08-27T11:28:52.320499+0000","last_undegraded":"2023-08-27T11:28:52.320499+0000","last_fullsized":"2023-08-27T11:28:52.320499+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:15:30.794231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320352+0000","last_change":"2023-08-27T11:28:50.429724+0000","last_active":"2023-08-27T11:28:52.320352+0000","last_peered":"2023-08-27T11:28:52.320352+0000","last_clean":"2023-08-27T11:28:52.320352+0000","last_became_active":"2023-08-27T11:28:50.429322+0000","last_became_peered":"2023-08-27T11:28:50.429322+0000","last_unstale":"2023-08-27T11:28:52.320352+0000","last_undegraded":"2023-08-27T11:28:52.320352+0000","last_fullsized":"2023-08-27T11:28:52.320352+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:03.500060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"152'1","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413792+0000","last_change":"2023-08-27T11:28:49.426757+0000","last_active":"2023-08-27T11:28:51.413792+0000","last_peered":"2023-08-27T11:28:51.413792+0000","last_clean":"2023-08-27T11:28:51.413792+0000","last_became_active":"2023-08-27T11:28:49.426454+0000","last_became_peered":"2023-08-27T11:28:49.426454+0000","last_unstale":"2023-08-27T11:28:51.413792+0000","last_undegraded":"2023-08-27T11:28:51.413792+0000","last_fullsized":"2023-08-27T11:28:51.413792+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:17:32.469376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413837+0000","last_change":"2023-08-27T11:28:50.421401+0000","last_active":"2023-08-27T11:28:51.413837+0000","last_peered":"2023-08-27T11:28:51.413837+0000","last_clean":"2023-08-27T11:28:51.413837+0000","last_became_active":"2023-08-27T11:28:50.421132+0000","last_became_peered":"2023-08-27T11:28:50.421132+0000","last_unstale":"2023-08-27T11:28:51.413837+0000","last_undegraded":"2023-08-27T11:28:51.413837+0000","last_fullsized":"2023-08-27T11:28:51.413837+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:05:36.872167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.421782+0000","last_change":"2023-08-27T11:28:49.427420+0000","last_active":"2023-08-27T11:28:50.421782+0000","last_peered":"2023-08-27T11:28:50.421782+0000","last_clean":"2023-08-27T11:28:50.421782+0000","last_became_active":"2023-08-27T11:28:49.426995+0000","last_became_peered":"2023-08-27T11:28:49.426995+0000","last_unstale":"2023-08-27T11:28:50.421782+0000","last_undegraded":"2023-08-27T11:28:50.421782+0000","last_fullsized":"2023-08-27T11:28:50.421782+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:45:12.351029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413727+0000","last_change":"2023-08-27T11:28:50.425756+0000","last_active":"2023-08-27T11:28:51.413727+0000","last_peered":"2023-08-27T11:28:51.413727+0000","last_clean":"2023-08-27T11:28:51.413727+0000","last_became_active":"2023-08-27T11:28:50.425479+0000","last_became_peered":"2023-08-27T11:28:50.425479+0000","last_unstale":"2023-08-27T11:28:51.413727+0000","last_undegraded":"2023-08-27T11:28:51.413727+0000","last_fullsized":"2023-08-27T11:28:51.413727+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:46:13.434745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"152'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.430198+0000","last_change":"2023-08-27T11:28:49.419592+0000","last_active":"2023-08-27T11:28:50.430198+0000","last_peered":"2023-08-27T11:28:50.430198+0000","last_clean":"2023-08-27T11:28:50.430198+0000","last_became_active":"2023-08-27T11:28:49.419334+0000","last_became_peered":"2023-08-27T11:28:49.419334+0000","last_unstale":"2023-08-27T11:28:50.430198+0000","last_undegraded":"2023-08-27T11:28:50.430198+0000","last_fullsized":"2023-08-27T11:28:50.430198+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:59.764142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412832+0000","last_change":"2023-08-27T11:28:50.429405+0000","last_active":"2023-08-27T11:28:51.412832+0000","last_peered":"2023-08-27T11:28:51.412832+0000","last_clean":"2023-08-27T11:28:51.412832+0000","last_became_active":"2023-08-27T11:28:50.428635+0000","last_became_peered":"2023-08-27T11:28:50.428635+0000","last_unstale":"2023-08-27T11:28:51.412832+0000","last_undegraded":"2023-08-27T11:28:51.412832+0000","last_fullsized":"2023-08-27T11:28:51.412832+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:01:26.641877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413714+0000","last_change":"2023-08-27T11:28:49.417329+0000","last_active":"2023-08-27T11:28:51.413714+0000","last_peered":"2023-08-27T11:28:51.413714+0000","last_clean":"2023-08-27T11:28:51.413714+0000","last_became_active":"2023-08-27T11:28:49.417066+0000","last_became_peered":"2023-08-27T11:28:49.417066+0000","last_unstale":"2023-08-27T11:28:51.413714+0000","last_undegraded":"2023-08-27T11:28:51.413714+0000","last_fullsized":"2023-08-27T11:28:51.413714+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:45:03.177105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.427623+0000","last_change":"2023-08-27T11:28:50.424960+0000","last_active":"2023-08-27T11:28:50.427623+0000","last_peered":"2023-08-27T11:28:50.427623+0000","last_clean":"2023-08-27T11:28:50.427623+0000","last_became_active":"2023-08-27T11:28:50.424595+0000","last_became_peered":"2023-08-27T11:28:50.424595+0000","last_unstale":"2023-08-27T11:28:50.427623+0000","last_undegraded":"2023-08-27T11:28:50.427623+0000","last_fullsized":"2023-08-27T11:28:50.427623+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:55:09.192293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.410026+0000","last_change":"2023-08-27T11:28:49.420731+0000","last_active":"2023-08-27T11:28:50.410026+0000","last_peered":"2023-08-27T11:28:50.410026+0000","last_clean":"2023-08-27T11:28:50.410026+0000","last_became_active":"2023-08-27T11:28:49.420454+0000","last_became_peered":"2023-08-27T11:28:49.420454+0000","last_unstale":"2023-08-27T11:28:50.410026+0000","last_undegraded":"2023-08-27T11:28:50.410026+0000","last_fullsized":"2023-08-27T11:28:50.410026+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:20:40.703868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414495+0000","last_change":"2023-08-27T11:28:50.433625+0000","last_active":"2023-08-27T11:28:51.414495+0000","last_peered":"2023-08-27T11:28:51.414495+0000","last_clean":"2023-08-27T11:28:51.414495+0000","last_became_active":"2023-08-27T11:28:50.433361+0000","last_became_peered":"2023-08-27T11:28:50.433361+0000","last_unstale":"2023-08-27T11:28:51.414495+0000","last_undegraded":"2023-08-27T11:28:51.414495+0000","last_fullsized":"2023-08-27T11:28:51.414495+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:52:51.436030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320152+0000","last_change":"2023-08-27T11:28:49.421223+0000","last_active":"2023-08-27T11:28:52.320152+0000","last_peered":"2023-08-27T11:28:52.320152+0000","last_clean":"2023-08-27T11:28:52.320152+0000","last_became_active":"2023-08-27T11:28:49.420945+0000","last_became_peered":"2023-08-27T11:28:49.420945+0000","last_unstale":"2023-08-27T11:28:52.320152+0000","last_undegraded":"2023-08-27T11:28:52.320152+0000","last_fullsized":"2023-08-27T11:28:52.320152+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:03:06.795445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413732+0000","last_change":"2023-08-27T11:28:50.430556+0000","last_active":"2023-08-27T11:28:51.413732+0000","last_peered":"2023-08-27T11:28:51.413732+0000","last_clean":"2023-08-27T11:28:51.413732+0000","last_became_active":"2023-08-27T11:28:50.428358+0000","last_became_peered":"2023-08-27T11:28:50.428358+0000","last_unstale":"2023-08-27T11:28:51.413732+0000","last_undegraded":"2023-08-27T11:28:51.413732+0000","last_fullsized":"2023-08-27T11:28:51.413732+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:59:14.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"152'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.436244+0000","last_change":"2023-08-27T11:28:49.422807+0000","last_active":"2023-08-27T11:28:50.436244+0000","last_peered":"2023-08-27T11:28:50.436244+0000","last_clean":"2023-08-27T11:28:50.436244+0000","last_became_active":"2023-08-27T11:28:49.422560+0000","last_became_peered":"2023-08-27T11:28:49.422560+0000","last_unstale":"2023-08-27T11:28:50.436244+0000","last_undegraded":"2023-08-27T11:28:50.436244+0000","last_fullsized":"2023-08-27T11:28:50.436244+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:22:23.805738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076413+0000","last_change":"2023-08-27T11:28:50.418010+0000","last_active":"2023-08-27T11:28:52.076413+0000","last_peered":"2023-08-27T11:28:52.076413+0000","last_clean":"2023-08-27T11:28:52.076413+0000","last_became_active":"2023-08-27T11:28:50.417757+0000","last_became_peered":"2023-08-27T11:28:50.417757+0000","last_unstale":"2023-08-27T11:28:52.076413+0000","last_undegraded":"2023-08-27T11:28:52.076413+0000","last_fullsized":"2023-08-27T11:28:52.076413+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:18:14.857419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414261+0000","last_change":"2023-08-27T11:28:49.417726+0000","last_active":"2023-08-27T11:28:51.414261+0000","last_peered":"2023-08-27T11:28:51.414261+0000","last_clean":"2023-08-27T11:28:51.414261+0000","last_became_active":"2023-08-27T11:28:49.417390+0000","last_became_peered":"2023-08-27T11:28:49.417390+0000","last_unstale":"2023-08-27T11:28:51.414261+0000","last_undegraded":"2023-08-27T11:28:51.414261+0000","last_fullsized":"2023-08-27T11:28:51.414261+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:20:08.781333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435586+0000","last_change":"2023-08-27T11:28:50.432498+0000","last_active":"2023-08-27T11:28:50.435586+0000","last_peered":"2023-08-27T11:28:50.435586+0000","last_clean":"2023-08-27T11:28:50.435586+0000","last_became_active":"2023-08-27T11:28:50.432034+0000","last_became_peered":"2023-08-27T11:28:50.432034+0000","last_unstale":"2023-08-27T11:28:50.435586+0000","last_undegraded":"2023-08-27T11:28:50.435586+0000","last_fullsized":"2023-08-27T11:28:50.435586+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:37:46.190181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.409355+0000","last_change":"2023-08-27T11:28:49.424803+0000","last_active":"2023-08-27T11:28:50.409355+0000","last_peered":"2023-08-27T11:28:50.409355+0000","last_clean":"2023-08-27T11:28:50.409355+0000","last_became_active":"2023-08-27T11:28:49.424512+0000","last_became_peered":"2023-08-27T11:28:49.424512+0000","last_unstale":"2023-08-27T11:28:50.409355+0000","last_undegraded":"2023-08-27T11:28:50.409355+0000","last_fullsized":"2023-08-27T11:28:50.409355+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:33:57.191788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320165+0000","last_change":"2023-08-27T11:28:50.423533+0000","last_active":"2023-08-27T11:28:52.320165+0000","last_peered":"2023-08-27T11:28:52.320165+0000","last_clean":"2023-08-27T11:28:52.320165+0000","last_became_active":"2023-08-27T11:28:50.423252+0000","last_became_peered":"2023-08-27T11:28:50.423252+0000","last_unstale":"2023-08-27T11:28:52.320165+0000","last_undegraded":"2023-08-27T11:28:52.320165+0000","last_fullsized":"2023-08-27T11:28:52.320165+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:44:52.638610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320143+0000","last_change":"2023-08-27T11:28:49.425297+0000","last_active":"2023-08-27T11:28:52.320143+0000","last_peered":"2023-08-27T11:28:52.320143+0000","last_clean":"2023-08-27T11:28:52.320143+0000","last_became_active":"2023-08-27T11:28:49.424837+0000","last_became_peered":"2023-08-27T11:28:49.424837+0000","last_unstale":"2023-08-27T11:28:52.320143+0000","last_undegraded":"2023-08-27T11:28:52.320143+0000","last_fullsized":"2023-08-27T11:28:52.320143+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:59:15.798185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414061+0000","last_change":"2023-08-27T11:28:50.420363+0000","last_active":"2023-08-27T11:28:51.414061+0000","last_peered":"2023-08-27T11:28:51.414061+0000","last_clean":"2023-08-27T11:28:51.414061+0000","last_became_active":"2023-08-27T11:28:50.418825+0000","last_became_peered":"2023-08-27T11:28:50.418825+0000","last_unstale":"2023-08-27T11:28:51.414061+0000","last_undegraded":"2023-08-27T11:28:51.414061+0000","last_fullsized":"2023-08-27T11:28:51.414061+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:33:14.575682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414017+0000","last_change":"2023-08-27T11:28:49.417180+0000","last_active":"2023-08-27T11:28:51.414017+0000","last_peered":"2023-08-27T11:28:51.414017+0000","last_clean":"2023-08-27T11:28:51.414017+0000","last_became_active":"2023-08-27T11:28:49.416732+0000","last_became_peered":"2023-08-27T11:28:49.416732+0000","last_unstale":"2023-08-27T11:28:51.414017+0000","last_undegraded":"2023-08-27T11:28:51.414017+0000","last_fullsized":"2023-08-27T11:28:51.414017+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:56.320705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412822+0000","last_change":"2023-08-27T11:28:50.430756+0000","last_active":"2023-08-27T11:28:51.412822+0000","last_peered":"2023-08-27T11:28:51.412822+0000","last_clean":"2023-08-27T11:28:51.412822+0000","last_became_active":"2023-08-27T11:28:50.430504+0000","last_became_peered":"2023-08-27T11:28:50.430504+0000","last_unstale":"2023-08-27T11:28:51.412822+0000","last_undegraded":"2023-08-27T11:28:51.412822+0000","last_fullsized":"2023-08-27T11:28:51.412822+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:27:33.047800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.408583+0000","last_change":"2023-08-27T11:28:49.418290+0000","last_active":"2023-08-27T11:28:50.408583+0000","last_peered":"2023-08-27T11:28:50.408583+0000","last_clean":"2023-08-27T11:28:50.408583+0000","last_became_active":"2023-08-27T11:28:49.417576+0000","last_became_peered":"2023-08-27T11:28:49.417576+0000","last_unstale":"2023-08-27T11:28:50.408583+0000","last_undegraded":"2023-08-27T11:28:50.408583+0000","last_fullsized":"2023-08-27T11:28:50.408583+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:56:19.689714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412822+0000","last_change":"2023-08-27T11:28:50.432391+0000","last_active":"2023-08-27T11:28:51.412822+0000","last_peered":"2023-08-27T11:28:51.412822+0000","last_clean":"2023-08-27T11:28:51.412822+0000","last_became_active":"2023-08-27T11:28:50.432027+0000","last_became_peered":"2023-08-27T11:28:50.432027+0000","last_unstale":"2023-08-27T11:28:51.412822+0000","last_undegraded":"2023-08-27T11:28:51.412822+0000","last_fullsized":"2023-08-27T11:28:51.412822+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:54:33.889906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320334+0000","last_change":"2023-08-27T11:28:49.425777+0000","last_active":"2023-08-27T11:28:52.320334+0000","last_peered":"2023-08-27T11:28:52.320334+0000","last_clean":"2023-08-27T11:28:52.320334+0000","last_became_active":"2023-08-27T11:28:49.425435+0000","last_became_peered":"2023-08-27T11:28:49.425435+0000","last_unstale":"2023-08-27T11:28:52.320334+0000","last_undegraded":"2023-08-27T11:28:52.320334+0000","last_fullsized":"2023-08-27T11:28:52.320334+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:38:39.524177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076692+0000","last_change":"2023-08-27T11:28:50.428682+0000","last_active":"2023-08-27T11:28:52.076692+0000","last_peered":"2023-08-27T11:28:52.076692+0000","last_clean":"2023-08-27T11:28:52.076692+0000","last_became_active":"2023-08-27T11:28:50.428428+0000","last_became_peered":"2023-08-27T11:28:50.428428+0000","last_unstale":"2023-08-27T11:28:52.076692+0000","last_undegraded":"2023-08-27T11:28:52.076692+0000","last_fullsized":"2023-08-27T11:28:52.076692+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:39:54.608535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.419695+0000","last_change":"2023-08-27T11:28:49.423875+0000","last_active":"2023-08-27T11:28:50.419695+0000","last_peered":"2023-08-27T11:28:50.419695+0000","last_clean":"2023-08-27T11:28:50.419695+0000","last_became_active":"2023-08-27T11:28:49.423545+0000","last_became_peered":"2023-08-27T11:28:49.423545+0000","last_unstale":"2023-08-27T11:28:50.419695+0000","last_undegraded":"2023-08-27T11:28:50.419695+0000","last_fullsized":"2023-08-27T11:28:50.419695+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:58:32.509833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413910+0000","last_change":"2023-08-27T11:28:50.433738+0000","last_active":"2023-08-27T11:28:51.413910+0000","last_peered":"2023-08-27T11:28:51.413910+0000","last_clean":"2023-08-27T11:28:51.413910+0000","last_became_active":"2023-08-27T11:28:50.433455+0000","last_became_peered":"2023-08-27T11:28:50.433455+0000","last_unstale":"2023-08-27T11:28:51.413910+0000","last_undegraded":"2023-08-27T11:28:51.413910+0000","last_fullsized":"2023-08-27T11:28:51.413910+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:26:04.520477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"152'2","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413622+0000","last_change":"2023-08-27T11:28:49.422456+0000","last_active":"2023-08-27T11:28:51.413622+0000","last_peered":"2023-08-27T11:28:51.413622+0000","last_clean":"2023-08-27T11:28:51.413622+0000","last_became_active":"2023-08-27T11:28:49.422214+0000","last_became_peered":"2023-08-27T11:28:49.422214+0000","last_unstale":"2023-08-27T11:28:51.413622+0000","last_undegraded":"2023-08-27T11:28:51.413622+0000","last_fullsized":"2023-08-27T11:28:51.413622+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:27:30.543860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":290,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414466+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:28:51.414466+0000","last_peered":"2023-08-27T11:28:51.414466+0000","last_clean":"2023-08-27T11:28:51.414466+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:28:51.414466+0000","last_undegraded":"2023-08-27T11:28:51.414466+0000","last_fullsized":"2023-08-27T11:28:51.414466+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:47:51.352733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435751+0000","last_change":"2023-08-27T11:28:50.432082+0000","last_active":"2023-08-27T11:28:50.435751+0000","last_peered":"2023-08-27T11:28:50.435751+0000","last_clean":"2023-08-27T11:28:50.435751+0000","last_became_active":"2023-08-27T11:28:50.430391+0000","last_became_peered":"2023-08-27T11:28:50.430391+0000","last_unstale":"2023-08-27T11:28:50.435751+0000","last_undegraded":"2023-08-27T11:28:50.435751+0000","last_fullsized":"2023-08-27T11:28:50.435751+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:17.009060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"152'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.425811+0000","last_change":"2023-08-27T11:28:49.423748+0000","last_active":"2023-08-27T11:28:50.425811+0000","last_peered":"2023-08-27T11:28:50.425811+0000","last_clean":"2023-08-27T11:28:50.425811+0000","last_became_active":"2023-08-27T11:28:49.423436+0000","last_became_peered":"2023-08-27T11:28:49.423436+0000","last_unstale":"2023-08-27T11:28:50.425811+0000","last_undegraded":"2023-08-27T11:28:50.425811+0000","last_fullsized":"2023-08-27T11:28:50.425811+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:34:57.916663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":290,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414363+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:28:51.414363+0000","last_peered":"2023-08-27T11:28:51.414363+0000","last_clean":"2023-08-27T11:28:51.414363+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:28:51.414363+0000","last_undegraded":"2023-08-27T11:28:51.414363+0000","last_fullsized":"2023-08-27T11:28:51.414363+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:35:40.185824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.430197+0000","last_change":"2023-08-27T11:28:50.424133+0000","last_active":"2023-08-27T11:28:50.430197+0000","last_peered":"2023-08-27T11:28:50.430197+0000","last_clean":"2023-08-27T11:28:50.430197+0000","last_became_active":"2023-08-27T11:28:50.423846+0000","last_became_peered":"2023-08-27T11:28:50.423846+0000","last_unstale":"2023-08-27T11:28:50.430197+0000","last_undegraded":"2023-08-27T11:28:50.430197+0000","last_fullsized":"2023-08-27T11:28:50.430197+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:44:54.808280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"152'1","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412418+0000","last_change":"2023-08-27T11:28:49.428066+0000","last_active":"2023-08-27T11:28:51.412418+0000","last_peered":"2023-08-27T11:28:51.412418+0000","last_clean":"2023-08-27T11:28:51.412418+0000","last_became_active":"2023-08-27T11:28:49.427780+0000","last_became_peered":"2023-08-27T11:28:49.427780+0000","last_unstale":"2023-08-27T11:28:51.412418+0000","last_undegraded":"2023-08-27T11:28:51.412418+0000","last_fullsized":"2023-08-27T11:28:51.412418+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:13.146471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":250,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320354+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:28:52.320354+0000","last_peered":"2023-08-27T11:28:52.320354+0000","last_clean":"2023-08-27T11:28:52.320354+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:28:52.320354+0000","last_undegraded":"2023-08-27T11:28:52.320354+0000","last_fullsized":"2023-08-27T11:28:52.320354+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:16:50.455378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320275+0000","last_change":"2023-08-27T11:28:50.425521+0000","last_active":"2023-08-27T11:28:52.320275+0000","last_peered":"2023-08-27T11:28:52.320275+0000","last_clean":"2023-08-27T11:28:52.320275+0000","last_became_active":"2023-08-27T11:28:50.425256+0000","last_became_peered":"2023-08-27T11:28:50.425256+0000","last_unstale":"2023-08-27T11:28:52.320275+0000","last_undegraded":"2023-08-27T11:28:52.320275+0000","last_fullsized":"2023-08-27T11:28:52.320275+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:11.183530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"152'1","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.428290+0000","last_change":"2023-08-27T11:28:49.422723+0000","last_active":"2023-08-27T11:28:50.428290+0000","last_peered":"2023-08-27T11:28:50.428290+0000","last_clean":"2023-08-27T11:28:50.428290+0000","last_became_active":"2023-08-27T11:28:49.418557+0000","last_became_peered":"2023-08-27T11:28:49.418557+0000","last_unstale":"2023-08-27T11:28:50.428290+0000","last_undegraded":"2023-08-27T11:28:50.428290+0000","last_fullsized":"2023-08-27T11:28:50.428290+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:22:32.303217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":244,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412405+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:28:51.412405+0000","last_peered":"2023-08-27T11:28:51.412405+0000","last_clean":"2023-08-27T11:28:51.412405+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:28:51.412405+0000","last_undegraded":"2023-08-27T11:28:51.412405+0000","last_fullsized":"2023-08-27T11:28:51.412405+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:24:45.940962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.424875+0000","last_change":"2023-08-27T11:28:50.418664+0000","last_active":"2023-08-27T11:28:50.424875+0000","last_peered":"2023-08-27T11:28:50.424875+0000","last_clean":"2023-08-27T11:28:50.424875+0000","last_became_active":"2023-08-27T11:28:50.417708+0000","last_became_peered":"2023-08-27T11:28:50.417708+0000","last_unstale":"2023-08-27T11:28:50.424875+0000","last_undegraded":"2023-08-27T11:28:50.424875+0000","last_fullsized":"2023-08-27T11:28:50.424875+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:55:24.639054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076337+0000","last_change":"2023-08-27T11:28:49.426796+0000","last_active":"2023-08-27T11:28:52.076337+0000","last_peered":"2023-08-27T11:28:52.076337+0000","last_clean":"2023-08-27T11:28:52.076337+0000","last_became_active":"2023-08-27T11:28:49.426531+0000","last_became_peered":"2023-08-27T11:28:49.426531+0000","last_unstale":"2023-08-27T11:28:52.076337+0000","last_undegraded":"2023-08-27T11:28:52.076337+0000","last_fullsized":"2023-08-27T11:28:52.076337+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:08:12.522978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":244,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412250+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:28:51.412250+0000","last_peered":"2023-08-27T11:28:51.412250+0000","last_clean":"2023-08-27T11:28:51.412250+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:28:51.412250+0000","last_undegraded":"2023-08-27T11:28:51.412250+0000","last_fullsized":"2023-08-27T11:28:51.412250+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:44.401650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.432792+0000","last_change":"2023-08-27T11:28:50.432456+0000","last_active":"2023-08-27T11:28:50.432792+0000","last_peered":"2023-08-27T11:28:50.432792+0000","last_clean":"2023-08-27T11:28:50.432792+0000","last_became_active":"2023-08-27T11:28:50.432092+0000","last_became_peered":"2023-08-27T11:28:50.432092+0000","last_unstale":"2023-08-27T11:28:50.432792+0000","last_undegraded":"2023-08-27T11:28:50.432792+0000","last_fullsized":"2023-08-27T11:28:50.432792+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:16:51.033471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412051+0000","last_change":"2023-08-27T11:28:49.428023+0000","last_active":"2023-08-27T11:28:51.412051+0000","last_peered":"2023-08-27T11:28:51.412051+0000","last_clean":"2023-08-27T11:28:51.412051+0000","last_became_active":"2023-08-27T11:28:49.427710+0000","last_became_peered":"2023-08-27T11:28:49.427710+0000","last_unstale":"2023-08-27T11:28:51.412051+0000","last_undegraded":"2023-08-27T11:28:51.412051+0000","last_fullsized":"2023-08-27T11:28:51.412051+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:51.965584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":394,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412025+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:28:51.412025+0000","last_peered":"2023-08-27T11:28:51.412025+0000","last_clean":"2023-08-27T11:28:51.412025+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:28:51.412025+0000","last_undegraded":"2023-08-27T11:28:51.412025+0000","last_fullsized":"2023-08-27T11:28:51.412025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:37.921403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":256,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076502+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:28:52.076502+0000","last_peered":"2023-08-27T11:28:52.076502+0000","last_clean":"2023-08-27T11:28:52.076502+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:28:52.076502+0000","last_undegraded":"2023-08-27T11:28:52.076502+0000","last_fullsized":"2023-08-27T11:28:52.076502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:31:38.716999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.319973+0000","last_change":"2023-08-27T11:28:50.427447+0000","last_active":"2023-08-27T11:28:52.319973+0000","last_peered":"2023-08-27T11:28:52.319973+0000","last_clean":"2023-08-27T11:28:52.319973+0000","last_became_active":"2023-08-27T11:28:50.427177+0000","last_became_peered":"2023-08-27T11:28:50.427177+0000","last_unstale":"2023-08-27T11:28:52.319973+0000","last_undegraded":"2023-08-27T11:28:52.319973+0000","last_fullsized":"2023-08-27T11:28:52.319973+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:42.832759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"152'1","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320503+0000","last_change":"2023-08-27T11:28:49.424332+0000","last_active":"2023-08-27T11:28:52.320503+0000","last_peered":"2023-08-27T11:28:52.320503+0000","last_clean":"2023-08-27T11:28:52.320503+0000","last_became_active":"2023-08-27T11:28:49.424070+0000","last_became_peered":"2023-08-27T11:28:49.424070+0000","last_unstale":"2023-08-27T11:28:52.320503+0000","last_undegraded":"2023-08-27T11:28:52.320503+0000","last_fullsized":"2023-08-27T11:28:52.320503+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:09:04.126197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":246,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413751+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:28:51.413751+0000","last_peered":"2023-08-27T11:28:51.413751+0000","last_clean":"2023-08-27T11:28:51.413751+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:28:51.413751+0000","last_undegraded":"2023-08-27T11:28:51.413751+0000","last_fullsized":"2023-08-27T11:28:51.413751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:04:06.142501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413582+0000","last_change":"2023-08-27T11:28:50.426265+0000","last_active":"2023-08-27T11:28:51.413582+0000","last_peered":"2023-08-27T11:28:51.413582+0000","last_clean":"2023-08-27T11:28:51.413582+0000","last_became_active":"2023-08-27T11:28:50.425861+0000","last_became_peered":"2023-08-27T11:28:50.425861+0000","last_unstale":"2023-08-27T11:28:51.413582+0000","last_undegraded":"2023-08-27T11:28:51.413582+0000","last_fullsized":"2023-08-27T11:28:51.413582+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:02:11.046625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"152'4","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435013+0000","last_change":"2023-08-27T11:28:49.422431+0000","last_active":"2023-08-27T11:28:50.435013+0000","last_peered":"2023-08-27T11:28:50.435013+0000","last_clean":"2023-08-27T11:28:50.435013+0000","last_became_active":"2023-08-27T11:28:49.420423+0000","last_became_peered":"2023-08-27T11:28:49.420423+0000","last_unstale":"2023-08-27T11:28:50.435013+0000","last_undegraded":"2023-08-27T11:28:50.435013+0000","last_fullsized":"2023-08-27T11:28:50.435013+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:38:16.718705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":274,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.409448+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:28:50.409448+0000","last_peered":"2023-08-27T11:28:50.409448+0000","last_clean":"2023-08-27T11:28:50.409448+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:28:50.409448+0000","last_undegraded":"2023-08-27T11:28:50.409448+0000","last_fullsized":"2023-08-27T11:28:50.409448+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:01:16.849818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574995,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33668,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338172,"statfs":{"total":96636764160,"available":96602288128,"internally_reserved":0,"allocated":2547712,"data_stored":1514677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.44,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.72799999999999998,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.39800000000000002,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.75600000000000001,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.46899999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55700000000000005,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.872},"last":0.499},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54000000000000004,"15min":0.52500000000000002},"min":{"1min":0.374,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.53500000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.497},"min":{"1min":0.377,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.72399999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.46100000000000002,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.90200000000000002},"last":0.42999999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.443,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77700000000000002,"5min":0.996,"15min":0.996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.48899999999999999,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.76400000000000001,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.628}]},{"osd":4,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.496,"15min":0.504},"min":{"1min":0.35899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.42399999999999999,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.73199999999999998,"5min":1.224,"15min":1.224},"last":0.38100000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.78100000000000003,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73199999999999998,"5min":0.78400000000000003,"15min":0.84799999999999998},"last":0.41299999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84499999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.47299999999999998,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1953792,"data_stored":924821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.29099999999999998},"max":{"1min":0.73399999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.435},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80300000000000005,"5min":0.83799999999999997,"15min":0.89800000000000002},"last":0.54300000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.873},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.84399999999999997},"last":0.497}]},{"osd":2,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.85899999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.69699999999999995,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.52300000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57599999999999996,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84499999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.80200000000000005,"5min":0.80700000000000005,"15min":0.93799999999999994},"last":0.45200000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.753,"5min":0.753,"15min":0.83099999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.374},"max":{"1min":0.81399999999999995,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.57799999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.875},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.34899999999999998},"max":{"1min":0.82999999999999996,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.51100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33652,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338188,"statfs":{"total":96636764160,"available":96602304512,"internally_reserved":0,"allocated":2531328,"data_stored":1512995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.312,"15min":0.51400000000000001},"min":{"1min":0.221,"5min":0.191,"15min":0.191},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":11.971},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.72799999999999998},"min":{"1min":0.24299999999999999,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":23.045999999999999},"last":0.30399999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.54200000000000004},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.221},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":11.928000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.316,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.224},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.58599999999999997},"last":0.26200000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39000000000000001,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":10.99},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.76500000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49299999999999999,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.42799999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.41499999999999998,"5min":0.40400000000000003,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.90700000000000003,"15min":1.26},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.74099999999999999},"min":{"1min":0.44900000000000001,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.81899999999999995,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.96399999999999997},"min":{"1min":0.46700000000000003,"5min":0.40999999999999998,"15min":0.36199999999999999},"max":{"1min":0.76200000000000001,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.88700000000000001,"15min":1.296},"last":0.52600000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.76700000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.371},"max":{"1min":0.77800000000000002,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60099999999999998,"15min":0.98199999999999998},"min":{"1min":0.437,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":22.952000000000002},"last":0.61799999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59599999999999997,"15min":0.97799999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.84199999999999997,"15min":22.988},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.627},"min":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.83799999999999997,"5min":0.86399999999999999,"15min":1.327},"last":0.627}]}]},{"osd":0,"up_from":13,"seq":55834574995,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":914703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.316,"15min":0.311},"min":{"1min":0.224,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.46899999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34000000000000002,"15min":0.34799999999999998},"min":{"1min":0.26600000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.501,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.27400000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.35899999999999999,"15min":0.34399999999999997},"min":{"1min":0.26000000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.59899999999999998,"5min":0.628,"15min":0.63200000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.33900000000000002},"min":{"1min":0.26000000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.627,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.34200000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.34599999999999997},"min":{"1min":0.23499999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.59599999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.376},"min":{"1min":0.33700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.35599999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.56000000000000005},"min":{"1min":0.38400000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.85399999999999998,"15min":1.214},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.77500000000000002,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.51500000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.434,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.78300000000000003,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58999999999999997,"15min":0.59699999999999998},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76700000000000002,"5min":0.85399999999999998,"15min":1.603},"last":0.59499999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.503,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.75800000000000001,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.41799999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.878,"15min":0.878},"last":0.505}]},{"osd":7,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.64600000000000002},"min":{"1min":0.47299999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.74199999999999999,"5min":3.79,"15min":3.79},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1929216,"data_stored":916064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.30199999999999999},"min":{"1min":0.22800000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.377,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.31900000000000001,"15min":0.318},"min":{"1min":0.251,"5min":0.22700000000000001,"15min":0.19700000000000001},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.629},"last":0.35499999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.315,"15min":0.32200000000000001},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.40799999999999997,"5min":0.50800000000000001,"15min":0.54400000000000004},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.34100000000000003,"15min":0.34699999999999998},"min":{"1min":0.29999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.749},"last":0.34100000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.379},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.73699999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.39500000000000002},"min":{"1min":0.33600000000000002,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":0.53500000000000003,"5min":0.55100000000000005,"15min":0.69499999999999995},"last":0.38700000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.41799999999999998,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.879},"last":0.39900000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.47199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.42899999999999999,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.40999999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.46000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.47899999999999998,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.83599999999999997},"last":0.504}]},{"osd":7,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.46899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.49199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574996,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1949696,"data_stored":922753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.29799999999999999,"15min":0.30299999999999999},"min":{"1min":0.221,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.435,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.318},"min":{"1min":0.26500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.27600000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.33800000000000002,"15min":0.32200000000000001},"min":{"1min":0.24399999999999999,"5min":0.22700000000000001,"15min":0.19500000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.36399999999999999,"15min":0.35199999999999998},"min":{"1min":0.27500000000000002,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":0.436}]},{"osd":2,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.376,"15min":0.36599999999999999},"min":{"1min":0.27400000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.373,"15min":0.36899999999999999},"min":{"1min":0.28699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.40600000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.55200000000000005},"min":{"1min":0.41099999999999998,"5min":0.33900000000000002,"15min":0.28599999999999998},"max":{"1min":0.80700000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.47999999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55100000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.38,"15min":0.314},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.44700000000000001,"5min":0.379,"15min":0.373},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.60999999999999999},"min":{"1min":0.47999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":1.0069999999999999,"15min":1.111},"last":0.57899999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58499999999999996,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":1.032,"15min":1.236},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574996,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1929216,"data_stored":916556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.33400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.40899999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.78700000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.41499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68999999999999995,"5min":0.746,"15min":0.76500000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.44,"5min":0.308,"15min":0.308},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.75}]},{"osd":2,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.35099999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69199999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.38600000000000001,"5min":0.32400000000000001,"15min":0.32200000000000001},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.80100000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56499999999999995,"15min":0.54200000000000004},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73699999999999999,"5min":0.80300000000000005,"15min":0.80600000000000005},"last":0.76700000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.51600000000000001,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60399999999999998,"15min":0.59499999999999997},"min":{"1min":0.46200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.57699999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.38100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73999999999999999,"5min":0.78900000000000003,"15min":0.90600000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.62,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.70399999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.47399999999999998,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.89300000000000002},"last":0.55700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574995,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1933312,"data_stored":916030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.318,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.96899999999999997},"last":0.499},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39000000000000001,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.71499999999999997,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.623}]},{"osd":1,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55300000000000005,"15min":0.53700000000000003},"min":{"1min":0.41699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81000000000000005,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52500000000000002,"15min":0.53700000000000003},"min":{"1min":0.38300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68000000000000005,"5min":0.72899999999999998,"15min":1.075},"last":0.67500000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.69199999999999995,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55400000000000005,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.373,"15min":0.307},"max":{"1min":0.747,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.78900000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.57899999999999996},"min":{"1min":0.41899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":1.03},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.58999999999999997},"min":{"1min":0.50600000000000001,"5min":0.40699999999999997,"15min":0.35199999999999998},"max":{"1min":0.77700000000000002,"5min":0.77800000000000002,"15min":1.026},"last":0.73999999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.31,"15min":0.27000000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.86899999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.91400000000000003},"last":0.39800000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.42899999999999999,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.77600000000000002,"5min":0.79400000000000004,"15min":0.88200000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.44700000000000001,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.94199999999999995},"last":0.43099999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.95899999999999996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64200000000000002,"15min":0.65000000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.92000000000000004},"last":0.69999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:54.558 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:54.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.678+0000 7f0c23894700 1 Processor -- start 2023-08-27T11:28:54.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.678+0000 7f0c23894700 1 -- start start 2023-08-27T11:28:54.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c23894700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0c1c151060 0x7f0c1c15e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c23894700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c1c15e990 0x7f0c1c06adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c23894700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1c0b02f0 con 0x7f0c1c06b2f0 2023-08-27T11:28:54.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c23894700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1c06b780 con 0x7f0c1c151060 2023-08-27T11:28:54.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c23894700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0b05d0 con 0x7f0c1c15e990 2023-08-27T11:28:54.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c21630700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0c1c151060 0x7f0c1c15e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c21630700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0c1c151060 0x7f0c1c15e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37150/0 (socket says 172.21.15.112:37150) 2023-08-27T11:28:54.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c21630700 1 -- 172.21.15.112:0/3761761061 learned_addr learned my addr 172.21.15.112:0/3761761061 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:54.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c20e2f700 1 --2- 172.21.15.112:0/3761761061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c1c15e990 0x7f0c1c06adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c21630700 1 -- 172.21.15.112:0/3761761061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c1c15e990 msgr2=0x7f0c1c06adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:54.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c21630700 1 --2- 172.21.15.112:0/3761761061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c1c15e990 0x7f0c1c06adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c21630700 1 -- 172.21.15.112:0/3761761061 >> v1:172.21.15.138:6789/0 conn(0x7f0c1c06b2f0 legacy=0x7f0c1c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:54.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c21630700 1 -- 172.21.15.112:0/3761761061 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c0b08b0 con 0x7f0c1c151060 2023-08-27T11:28:54.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c20e2f700 1 --2- 172.21.15.112:0/3761761061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c1c15e990 0x7f0c1c06adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:54.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c21630700 1 --2- 172.21.15.112:0/3761761061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1c151060 0x7f0c1c15e450 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f0c180098f0 tx=0x7f0c18009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0a3d233641458 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.679+0000 7f0c13fff700 1 -- 172.21.15.112:0/3761761061 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800b8a0 con 0x7f0c1c151060 2023-08-27T11:28:54.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.680+0000 7f0c13fff700 1 -- 172.21.15.112:0/3761761061 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1800ba00 con 0x7f0c1c151060 2023-08-27T11:28:54.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.680+0000 7f0c13fff700 1 -- 172.21.15.112:0/3761761061 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800bd90 con 0x7f0c1c151060 2023-08-27T11:28:54.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.680+0000 7f0c23894700 1 -- 172.21.15.112:0/3761761061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1c151060 msgr2=0x7f0c1c15e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.680+0000 7f0c23894700 1 --2- 172.21.15.112:0/3761761061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1c151060 0x7f0c1c15e450 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f0c180098f0 tx=0x7f0c18009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:54.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.680+0000 7f0c23894700 1 -- 172.21.15.112:0/3761761061 shutdown_connections 2023-08-27T11:28:54.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.680+0000 7f0c23894700 1 --2- 172.21.15.112:0/3761761061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1c151060 0x7f0c1c15e450 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.680+0000 7f0c23894700 1 --2- 172.21.15.112:0/3761761061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c1c15e990 0x7f0c1c06adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.680+0000 7f0c23894700 1 -- 172.21.15.112:0/3761761061 >> 172.21.15.112:0/3761761061 conn(0x7f0c1c0d0000 msgr2=0x7f0c1c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:54.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.680+0000 7f0c23894700 1 -- 172.21.15.112:0/3761761061 shutdown_connections 2023-08-27T11:28:54.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.680+0000 7f0c23894700 1 -- 172.21.15.112:0/3761761061 wait complete. 2023-08-27T11:28:54.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.681+0000 7f0c23894700 1 Processor -- start 2023-08-27T11:28:54.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.681+0000 7f0c23894700 1 -- start start 2023-08-27T11:28:54.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c23894700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c1c06b2f0 0x7f0c1c1499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c23894700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1c151060 0x7f0c1c149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c23894700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c1c15e990 0x7f0c1c070d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c23894700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0c5780 con 0x7f0c1c151060 2023-08-27T11:28:54.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c23894700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c070890 con 0x7f0c1c15e990 2023-08-27T11:28:54.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c23894700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c06ffa0 con 0x7f0c1c06b2f0 2023-08-27T11:28:54.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c20e2f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1c151060 0x7f0c1c149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c20e2f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1c151060 0x7f0c1c149ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37158/0 (socket says 172.21.15.112:37158) 2023-08-27T11:28:54.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c20e2f700 1 -- 172.21.15.112:0/62247411 learned_addr learned my addr 172.21.15.112:0/62247411 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:54.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c21e31700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c1c15e990 0x7f0c1c070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c21630700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c1c06b2f0 0x7f0c1c1499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c20e2f700 1 -- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c1c06b2f0 msgr2=0x7f0c1c1499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c20e2f700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c1c06b2f0 0x7f0c1c1499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c20e2f700 1 -- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c1c15e990 msgr2=0x7f0c1c070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c20e2f700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c1c15e990 0x7f0c1c070d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c20e2f700 1 -- 172.21.15.112:0/62247411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c18009550 con 0x7f0c1c151060 2023-08-27T11:28:54.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.682+0000 7f0c21e31700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c1c15e990 0x7f0c1c070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:54.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.683+0000 7f0c20e2f700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1c151060 0x7f0c1c149ef0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c780 tx=0x7f0c0c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.683+0000 7f0c11ffb700 1 -- 172.21.15.112:0/62247411 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c011800 con 0x7f0c1c151060 2023-08-27T11:28:54.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.683+0000 7f0c11ffb700 1 -- 172.21.15.112:0/62247411 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c011960 con 0x7f0c1c151060 2023-08-27T11:28:54.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.683+0000 7f0c23894700 1 -- 172.21.15.112:0/62247411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1c0c54a0 con 0x7f0c1c151060 2023-08-27T11:28:54.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.683+0000 7f0c11ffb700 1 -- 172.21.15.112:0/62247411 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c011cd0 con 0x7f0c1c151060 2023-08-27T11:28:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.683+0000 7f0c23894700 1 -- 172.21.15.112:0/62247411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1c070520 con 0x7f0c1c151060 2023-08-27T11:28:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.684+0000 7f0c23894700 1 -- 172.21.15.112:0/62247411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1c067a20 con 0x7f0c1c151060 2023-08-27T11:28:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.684+0000 7f0c11ffb700 1 -- 172.21.15.112:0/62247411 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f0c0c021350 con 0x7f0c1c151060 2023-08-27T11:28:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.685+0000 7f0c11ffb700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0c0006a4d0 0x7f0c0006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.685+0000 7f0c11ffb700 1 -- 172.21.15.112:0/62247411 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f0c0c088e20 con 0x7f0c1c151060 2023-08-27T11:28:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.685+0000 7f0c21630700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0c0006a4d0 0x7f0c0006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.685+0000 7f0c21630700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0c0006a4d0 0x7f0c0006c990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0c18009c40 tx=0x7f0c18008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.689+0000 7f0c11ffb700 1 -- 172.21.15.112:0/62247411 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0c0c053d40 con 0x7f0c1c151060 2023-08-27T11:28:54.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.874+0000 7f0c23894700 1 -- 172.21.15.112:0/62247411 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c1c0b0e70 con 0x7f0c0006a4d0 2023-08-27T11:28:54.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.883+0000 7f0c11ffb700 1 -- 172.21.15.112:0/62247411 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312309 (secure 0 0 0) 0x7f0c1800bbf0 con 0x7f0c0006a4d0 2023-08-27T11:28:54.888 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.886+0000 7f0c23894700 1 -- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0c0006a4d0 msgr2=0x7f0c0006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.886+0000 7f0c23894700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0c0006a4d0 0x7f0c0006c990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0c18009c40 tx=0x7f0c18008030 comp rx=0 tx=0).stop 2023-08-27T11:28:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.886+0000 7f0c23894700 1 -- 172.21.15.112:0/62247411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1c151060 msgr2=0x7f0c1c149ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.886+0000 7f0c23894700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1c151060 0x7f0c1c149ef0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c780 tx=0x7f0c0c00cb40 comp rx=0 tx=0).stop 2023-08-27T11:28:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.886+0000 7f0c23894700 1 -- 172.21.15.112:0/62247411 shutdown_connections 2023-08-27T11:28:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.886+0000 7f0c23894700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0c0006a4d0 0x7f0c0006c990 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.886+0000 7f0c23894700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c1c15e990 0x7f0c1c070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.886+0000 7f0c23894700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c1c06b2f0 0x7f0c1c1499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.886+0000 7f0c23894700 1 --2- 172.21.15.112:0/62247411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1c151060 0x7f0c1c149ef0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.886+0000 7f0c23894700 1 -- 172.21.15.112:0/62247411 >> 172.21.15.112:0/62247411 conn(0x7f0c1c0d0000 msgr2=0x7f0c1c0cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.887+0000 7f0c23894700 1 -- 172.21.15.112:0/62247411 shutdown_connections 2023-08-27T11:28:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:54.887+0000 7f0c23894700 1 -- 172.21.15.112:0/62247411 wait complete. 2023-08-27T11:28:54.894 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:28:54.900 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2023-08-27T11:28:54.438291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265780,"kb_used_data":16308,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754708940,"statfs":{"total":773094113280,"available":772821954560,"internally_reserved":0,"allocated":16699392,"data_stored":8538599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.028679"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320596+0000","last_change":"2023-08-27T11:28:50.426272+0000","last_active":"2023-08-27T11:28:52.320596+0000","last_peered":"2023-08-27T11:28:52.320596+0000","last_clean":"2023-08-27T11:28:52.320596+0000","last_became_active":"2023-08-27T11:28:50.425977+0000","last_became_peered":"2023-08-27T11:28:50.425977+0000","last_unstale":"2023-08-27T11:28:52.320596+0000","last_undegraded":"2023-08-27T11:28:52.320596+0000","last_fullsized":"2023-08-27T11:28:52.320596+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:32:01.714311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413612+0000","last_change":"2023-08-27T11:28:50.423791+0000","last_active":"2023-08-27T11:28:51.413612+0000","last_peered":"2023-08-27T11:28:51.413612+0000","last_clean":"2023-08-27T11:28:51.413612+0000","last_became_active":"2023-08-27T11:28:50.423541+0000","last_became_peered":"2023-08-27T11:28:50.423541+0000","last_unstale":"2023-08-27T11:28:51.413612+0000","last_undegraded":"2023-08-27T11:28:51.413612+0000","last_fullsized":"2023-08-27T11:28:51.413612+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:23.585726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076493+0000","last_change":"2023-08-27T11:28:50.429104+0000","last_active":"2023-08-27T11:28:52.076493+0000","last_peered":"2023-08-27T11:28:52.076493+0000","last_clean":"2023-08-27T11:28:52.076493+0000","last_became_active":"2023-08-27T11:28:50.428891+0000","last_became_peered":"2023-08-27T11:28:50.428891+0000","last_unstale":"2023-08-27T11:28:52.076493+0000","last_undegraded":"2023-08-27T11:28:52.076493+0000","last_fullsized":"2023-08-27T11:28:52.076493+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:13:04.498709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412355+0000","last_change":"2023-08-27T11:28:50.421808+0000","last_active":"2023-08-27T11:28:51.412355+0000","last_peered":"2023-08-27T11:28:51.412355+0000","last_clean":"2023-08-27T11:28:51.412355+0000","last_became_active":"2023-08-27T11:28:50.420778+0000","last_became_peered":"2023-08-27T11:28:50.420778+0000","last_unstale":"2023-08-27T11:28:51.412355+0000","last_undegraded":"2023-08-27T11:28:51.412355+0000","last_fullsized":"2023-08-27T11:28:51.412355+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:40.391553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435787+0000","last_change":"2023-08-27T11:28:50.432586+0000","last_active":"2023-08-27T11:28:50.435787+0000","last_peered":"2023-08-27T11:28:50.435787+0000","last_clean":"2023-08-27T11:28:50.435787+0000","last_became_active":"2023-08-27T11:28:50.432334+0000","last_became_peered":"2023-08-27T11:28:50.432334+0000","last_unstale":"2023-08-27T11:28:50.435787+0000","last_undegraded":"2023-08-27T11:28:50.435787+0000","last_fullsized":"2023-08-27T11:28:50.435787+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:55:50.393935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412202+0000","last_change":"2023-08-27T11:28:50.429089+0000","last_active":"2023-08-27T11:28:51.412202+0000","last_peered":"2023-08-27T11:28:51.412202+0000","last_clean":"2023-08-27T11:28:51.412202+0000","last_became_active":"2023-08-27T11:28:50.428814+0000","last_became_peered":"2023-08-27T11:28:50.428814+0000","last_unstale":"2023-08-27T11:28:51.412202+0000","last_undegraded":"2023-08-27T11:28:51.412202+0000","last_fullsized":"2023-08-27T11:28:51.412202+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:56.436880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435811+0000","last_change":"2023-08-27T11:28:50.432231+0000","last_active":"2023-08-27T11:28:50.435811+0000","last_peered":"2023-08-27T11:28:50.435811+0000","last_clean":"2023-08-27T11:28:50.435811+0000","last_became_active":"2023-08-27T11:28:50.431793+0000","last_became_peered":"2023-08-27T11:28:50.431793+0000","last_unstale":"2023-08-27T11:28:50.435811+0000","last_undegraded":"2023-08-27T11:28:50.435811+0000","last_fullsized":"2023-08-27T11:28:50.435811+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:42:42.911403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413533+0000","last_change":"2023-08-27T11:28:50.419372+0000","last_active":"2023-08-27T11:28:51.413533+0000","last_peered":"2023-08-27T11:28:51.413533+0000","last_clean":"2023-08-27T11:28:51.413533+0000","last_became_active":"2023-08-27T11:28:50.419084+0000","last_became_peered":"2023-08-27T11:28:50.419084+0000","last_unstale":"2023-08-27T11:28:51.413533+0000","last_undegraded":"2023-08-27T11:28:51.413533+0000","last_fullsized":"2023-08-27T11:28:51.413533+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:55:00.360756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.427772+0000","last_change":"2023-08-27T11:28:50.426775+0000","last_active":"2023-08-27T11:28:50.427772+0000","last_peered":"2023-08-27T11:28:50.427772+0000","last_clean":"2023-08-27T11:28:50.427772+0000","last_became_active":"2023-08-27T11:28:50.426521+0000","last_became_peered":"2023-08-27T11:28:50.426521+0000","last_unstale":"2023-08-27T11:28:50.427772+0000","last_undegraded":"2023-08-27T11:28:50.427772+0000","last_fullsized":"2023-08-27T11:28:50.427772+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:41:57.204734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414199+0000","last_change":"2023-08-27T11:28:50.427188+0000","last_active":"2023-08-27T11:28:51.414199+0000","last_peered":"2023-08-27T11:28:51.414199+0000","last_clean":"2023-08-27T11:28:51.414199+0000","last_became_active":"2023-08-27T11:28:50.426870+0000","last_became_peered":"2023-08-27T11:28:50.426870+0000","last_unstale":"2023-08-27T11:28:51.414199+0000","last_undegraded":"2023-08-27T11:28:51.414199+0000","last_fullsized":"2023-08-27T11:28:51.414199+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:20:57.365468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076481+0000","last_change":"2023-08-27T11:28:50.422020+0000","last_active":"2023-08-27T11:28:52.076481+0000","last_peered":"2023-08-27T11:28:52.076481+0000","last_clean":"2023-08-27T11:28:52.076481+0000","last_became_active":"2023-08-27T11:28:50.421491+0000","last_became_peered":"2023-08-27T11:28:50.421491+0000","last_unstale":"2023-08-27T11:28:52.076481+0000","last_undegraded":"2023-08-27T11:28:52.076481+0000","last_fullsized":"2023-08-27T11:28:52.076481+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:24:54.543099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.434609+0000","last_change":"2023-08-27T11:28:50.429548+0000","last_active":"2023-08-27T11:28:50.434609+0000","last_peered":"2023-08-27T11:28:50.434609+0000","last_clean":"2023-08-27T11:28:50.434609+0000","last_became_active":"2023-08-27T11:28:50.429304+0000","last_became_peered":"2023-08-27T11:28:50.429304+0000","last_unstale":"2023-08-27T11:28:50.434609+0000","last_undegraded":"2023-08-27T11:28:50.434609+0000","last_fullsized":"2023-08-27T11:28:50.434609+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:53:22.814103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412477+0000","last_change":"2023-08-27T11:28:50.433446+0000","last_active":"2023-08-27T11:28:51.412477+0000","last_peered":"2023-08-27T11:28:51.412477+0000","last_clean":"2023-08-27T11:28:51.412477+0000","last_became_active":"2023-08-27T11:28:50.433172+0000","last_became_peered":"2023-08-27T11:28:50.433172+0000","last_unstale":"2023-08-27T11:28:51.412477+0000","last_undegraded":"2023-08-27T11:28:51.412477+0000","last_fullsized":"2023-08-27T11:28:51.412477+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:52:22.634803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413701+0000","last_change":"2023-08-27T11:28:50.426603+0000","last_active":"2023-08-27T11:28:51.413701+0000","last_peered":"2023-08-27T11:28:51.413701+0000","last_clean":"2023-08-27T11:28:51.413701+0000","last_became_active":"2023-08-27T11:28:50.426346+0000","last_became_peered":"2023-08-27T11:28:50.426346+0000","last_unstale":"2023-08-27T11:28:51.413701+0000","last_undegraded":"2023-08-27T11:28:51.413701+0000","last_fullsized":"2023-08-27T11:28:51.413701+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:03:42.841698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.429244+0000","last_change":"2023-08-27T11:28:50.418544+0000","last_active":"2023-08-27T11:28:50.429244+0000","last_peered":"2023-08-27T11:28:50.429244+0000","last_clean":"2023-08-27T11:28:50.429244+0000","last_became_active":"2023-08-27T11:28:50.418180+0000","last_became_peered":"2023-08-27T11:28:50.418180+0000","last_unstale":"2023-08-27T11:28:50.429244+0000","last_undegraded":"2023-08-27T11:28:50.429244+0000","last_fullsized":"2023-08-27T11:28:50.429244+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:03:04.318857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076414+0000","last_change":"2023-08-27T11:28:50.427135+0000","last_active":"2023-08-27T11:28:52.076414+0000","last_peered":"2023-08-27T11:28:52.076414+0000","last_clean":"2023-08-27T11:28:52.076414+0000","last_became_active":"2023-08-27T11:28:50.426858+0000","last_became_peered":"2023-08-27T11:28:50.426858+0000","last_unstale":"2023-08-27T11:28:52.076414+0000","last_undegraded":"2023-08-27T11:28:52.076414+0000","last_fullsized":"2023-08-27T11:28:52.076414+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:43.579659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.424895+0000","last_change":"2023-08-27T11:28:50.418834+0000","last_active":"2023-08-27T11:28:50.424895+0000","last_peered":"2023-08-27T11:28:50.424895+0000","last_clean":"2023-08-27T11:28:50.424895+0000","last_became_active":"2023-08-27T11:28:50.418051+0000","last_became_peered":"2023-08-27T11:28:50.418051+0000","last_unstale":"2023-08-27T11:28:50.424895+0000","last_undegraded":"2023-08-27T11:28:50.424895+0000","last_fullsized":"2023-08-27T11:28:50.424895+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:29:18.564808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.424871+0000","last_change":"2023-08-27T11:28:50.417739+0000","last_active":"2023-08-27T11:28:50.424871+0000","last_peered":"2023-08-27T11:28:50.424871+0000","last_clean":"2023-08-27T11:28:50.424871+0000","last_became_active":"2023-08-27T11:28:50.417419+0000","last_became_peered":"2023-08-27T11:28:50.417419+0000","last_unstale":"2023-08-27T11:28:50.424871+0000","last_undegraded":"2023-08-27T11:28:50.424871+0000","last_fullsized":"2023-08-27T11:28:50.424871+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:54:37.318106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414333+0000","last_change":"2023-08-27T11:28:50.414165+0000","last_active":"2023-08-27T11:28:51.414333+0000","last_peered":"2023-08-27T11:28:51.414333+0000","last_clean":"2023-08-27T11:28:51.414333+0000","last_became_active":"2023-08-27T11:28:50.413894+0000","last_became_peered":"2023-08-27T11:28:50.413894+0000","last_unstale":"2023-08-27T11:28:51.414333+0000","last_undegraded":"2023-08-27T11:28:51.414333+0000","last_fullsized":"2023-08-27T11:28:51.414333+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:10:31.264961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414149+0000","last_change":"2023-08-27T11:28:50.419559+0000","last_active":"2023-08-27T11:28:51.414149+0000","last_peered":"2023-08-27T11:28:51.414149+0000","last_clean":"2023-08-27T11:28:51.414149+0000","last_became_active":"2023-08-27T11:28:50.418776+0000","last_became_peered":"2023-08-27T11:28:50.418776+0000","last_unstale":"2023-08-27T11:28:51.414149+0000","last_undegraded":"2023-08-27T11:28:51.414149+0000","last_fullsized":"2023-08-27T11:28:51.414149+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:05.835425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413852+0000","last_change":"2023-08-27T11:28:50.427983+0000","last_active":"2023-08-27T11:28:51.413852+0000","last_peered":"2023-08-27T11:28:51.413852+0000","last_clean":"2023-08-27T11:28:51.413852+0000","last_became_active":"2023-08-27T11:28:50.427673+0000","last_became_peered":"2023-08-27T11:28:50.427673+0000","last_unstale":"2023-08-27T11:28:51.413852+0000","last_undegraded":"2023-08-27T11:28:51.413852+0000","last_fullsized":"2023-08-27T11:28:51.413852+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:41:13.034838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413924+0000","last_change":"2023-08-27T11:28:50.426888+0000","last_active":"2023-08-27T11:28:51.413924+0000","last_peered":"2023-08-27T11:28:51.413924+0000","last_clean":"2023-08-27T11:28:51.413924+0000","last_became_active":"2023-08-27T11:28:50.424404+0000","last_became_peered":"2023-08-27T11:28:50.424404+0000","last_unstale":"2023-08-27T11:28:51.413924+0000","last_undegraded":"2023-08-27T11:28:51.413924+0000","last_fullsized":"2023-08-27T11:28:51.413924+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:25:14.196474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413407+0000","last_change":"2023-08-27T11:28:50.424164+0000","last_active":"2023-08-27T11:28:51.413407+0000","last_peered":"2023-08-27T11:28:51.413407+0000","last_clean":"2023-08-27T11:28:51.413407+0000","last_became_active":"2023-08-27T11:28:50.423858+0000","last_became_peered":"2023-08-27T11:28:50.423858+0000","last_unstale":"2023-08-27T11:28:51.413407+0000","last_undegraded":"2023-08-27T11:28:51.413407+0000","last_fullsized":"2023-08-27T11:28:51.413407+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:57.141092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435181+0000","last_change":"2023-08-27T11:28:50.430081+0000","last_active":"2023-08-27T11:28:50.435181+0000","last_peered":"2023-08-27T11:28:50.435181+0000","last_clean":"2023-08-27T11:28:50.435181+0000","last_became_active":"2023-08-27T11:28:50.429768+0000","last_became_peered":"2023-08-27T11:28:50.429768+0000","last_unstale":"2023-08-27T11:28:50.435181+0000","last_undegraded":"2023-08-27T11:28:50.435181+0000","last_fullsized":"2023-08-27T11:28:50.435181+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:00:03.880826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.424912+0000","last_change":"2023-08-27T11:28:50.418988+0000","last_active":"2023-08-27T11:28:50.424912+0000","last_peered":"2023-08-27T11:28:50.424912+0000","last_clean":"2023-08-27T11:28:50.424912+0000","last_became_active":"2023-08-27T11:28:50.418413+0000","last_became_peered":"2023-08-27T11:28:50.418413+0000","last_unstale":"2023-08-27T11:28:50.424912+0000","last_undegraded":"2023-08-27T11:28:50.424912+0000","last_fullsized":"2023-08-27T11:28:50.424912+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:06:21.397831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320327+0000","last_change":"2023-08-27T11:28:50.425372+0000","last_active":"2023-08-27T11:28:52.320327+0000","last_peered":"2023-08-27T11:28:52.320327+0000","last_clean":"2023-08-27T11:28:52.320327+0000","last_became_active":"2023-08-27T11:28:50.425076+0000","last_became_peered":"2023-08-27T11:28:50.425076+0000","last_unstale":"2023-08-27T11:28:52.320327+0000","last_undegraded":"2023-08-27T11:28:52.320327+0000","last_fullsized":"2023-08-27T11:28:52.320327+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:19:13.216408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076634+0000","last_change":"2023-08-27T11:28:50.421871+0000","last_active":"2023-08-27T11:28:52.076634+0000","last_peered":"2023-08-27T11:28:52.076634+0000","last_clean":"2023-08-27T11:28:52.076634+0000","last_became_active":"2023-08-27T11:28:50.421179+0000","last_became_peered":"2023-08-27T11:28:50.421179+0000","last_unstale":"2023-08-27T11:28:52.076634+0000","last_undegraded":"2023-08-27T11:28:52.076634+0000","last_fullsized":"2023-08-27T11:28:52.076634+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:01:42.017950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.434630+0000","last_change":"2023-08-27T11:28:50.430196+0000","last_active":"2023-08-27T11:28:50.434630+0000","last_peered":"2023-08-27T11:28:50.434630+0000","last_clean":"2023-08-27T11:28:50.434630+0000","last_became_active":"2023-08-27T11:28:50.429783+0000","last_became_peered":"2023-08-27T11:28:50.429783+0000","last_unstale":"2023-08-27T11:28:50.434630+0000","last_undegraded":"2023-08-27T11:28:50.434630+0000","last_fullsized":"2023-08-27T11:28:50.434630+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:19.316875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.419081+0000","last_change":"2023-08-27T11:28:50.414443+0000","last_active":"2023-08-27T11:28:50.419081+0000","last_peered":"2023-08-27T11:28:50.419081+0000","last_clean":"2023-08-27T11:28:50.419081+0000","last_became_active":"2023-08-27T11:28:50.414190+0000","last_became_peered":"2023-08-27T11:28:50.414190+0000","last_unstale":"2023-08-27T11:28:50.419081+0000","last_undegraded":"2023-08-27T11:28:50.419081+0000","last_fullsized":"2023-08-27T11:28:50.419081+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:09:13.623657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.431982+0000","last_change":"2023-08-27T11:28:50.426704+0000","last_active":"2023-08-27T11:28:50.431982+0000","last_peered":"2023-08-27T11:28:50.431982+0000","last_clean":"2023-08-27T11:28:50.431982+0000","last_became_active":"2023-08-27T11:28:50.426432+0000","last_became_peered":"2023-08-27T11:28:50.426432+0000","last_unstale":"2023-08-27T11:28:50.431982+0000","last_undegraded":"2023-08-27T11:28:50.431982+0000","last_fullsized":"2023-08-27T11:28:50.431982+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:40.667390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412134+0000","last_change":"2023-08-27T11:28:50.426515+0000","last_active":"2023-08-27T11:28:51.412134+0000","last_peered":"2023-08-27T11:28:51.412134+0000","last_clean":"2023-08-27T11:28:51.412134+0000","last_became_active":"2023-08-27T11:28:50.426217+0000","last_became_peered":"2023-08-27T11:28:50.426217+0000","last_unstale":"2023-08-27T11:28:51.412134+0000","last_undegraded":"2023-08-27T11:28:51.412134+0000","last_fullsized":"2023-08-27T11:28:51.412134+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:55:25.282686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412404+0000","last_change":"2023-08-27T11:28:50.422133+0000","last_active":"2023-08-27T11:28:51.412404+0000","last_peered":"2023-08-27T11:28:51.412404+0000","last_clean":"2023-08-27T11:28:51.412404+0000","last_became_active":"2023-08-27T11:28:50.421864+0000","last_became_peered":"2023-08-27T11:28:50.421864+0000","last_unstale":"2023-08-27T11:28:51.412404+0000","last_undegraded":"2023-08-27T11:28:51.412404+0000","last_fullsized":"2023-08-27T11:28:51.412404+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:14:30.582313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.428894+0000","last_change":"2023-08-27T11:28:50.419387+0000","last_active":"2023-08-27T11:28:50.428894+0000","last_peered":"2023-08-27T11:28:50.428894+0000","last_clean":"2023-08-27T11:28:50.428894+0000","last_became_active":"2023-08-27T11:28:50.419084+0000","last_became_peered":"2023-08-27T11:28:50.419084+0000","last_unstale":"2023-08-27T11:28:50.428894+0000","last_undegraded":"2023-08-27T11:28:50.428894+0000","last_fullsized":"2023-08-27T11:28:50.428894+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:16:14.370020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.417910+0000","last_change":"2023-08-27T11:28:49.418410+0000","last_active":"2023-08-27T11:28:50.417910+0000","last_peered":"2023-08-27T11:28:50.417910+0000","last_clean":"2023-08-27T11:28:50.417910+0000","last_became_active":"2023-08-27T11:28:49.418118+0000","last_became_peered":"2023-08-27T11:28:49.418118+0000","last_unstale":"2023-08-27T11:28:50.417910+0000","last_undegraded":"2023-08-27T11:28:50.417910+0000","last_fullsized":"2023-08-27T11:28:50.417910+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:30:12.138330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076523+0000","last_change":"2023-08-27T11:28:50.430755+0000","last_active":"2023-08-27T11:28:52.076523+0000","last_peered":"2023-08-27T11:28:52.076523+0000","last_clean":"2023-08-27T11:28:52.076523+0000","last_became_active":"2023-08-27T11:28:50.430503+0000","last_became_peered":"2023-08-27T11:28:50.430503+0000","last_unstale":"2023-08-27T11:28:52.076523+0000","last_undegraded":"2023-08-27T11:28:52.076523+0000","last_fullsized":"2023-08-27T11:28:52.076523+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:40:52.014836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412880+0000","last_change":"2023-08-27T11:28:49.430522+0000","last_active":"2023-08-27T11:28:51.412880+0000","last_peered":"2023-08-27T11:28:51.412880+0000","last_clean":"2023-08-27T11:28:51.412880+0000","last_became_active":"2023-08-27T11:28:49.430189+0000","last_became_peered":"2023-08-27T11:28:49.430189+0000","last_unstale":"2023-08-27T11:28:51.412880+0000","last_undegraded":"2023-08-27T11:28:51.412880+0000","last_fullsized":"2023-08-27T11:28:51.412880+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:45.562389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.427717+0000","last_change":"2023-08-27T11:28:50.426137+0000","last_active":"2023-08-27T11:28:50.427717+0000","last_peered":"2023-08-27T11:28:50.427717+0000","last_clean":"2023-08-27T11:28:50.427717+0000","last_became_active":"2023-08-27T11:28:50.425879+0000","last_became_peered":"2023-08-27T11:28:50.425879+0000","last_unstale":"2023-08-27T11:28:50.427717+0000","last_undegraded":"2023-08-27T11:28:50.427717+0000","last_fullsized":"2023-08-27T11:28:50.427717+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:19:54.933140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"152'3","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.430085+0000","last_change":"2023-08-27T11:28:49.422279+0000","last_active":"2023-08-27T11:28:50.430085+0000","last_peered":"2023-08-27T11:28:50.430085+0000","last_clean":"2023-08-27T11:28:50.430085+0000","last_became_active":"2023-08-27T11:28:49.422002+0000","last_became_peered":"2023-08-27T11:28:49.422002+0000","last_unstale":"2023-08-27T11:28:50.430085+0000","last_undegraded":"2023-08-27T11:28:50.430085+0000","last_fullsized":"2023-08-27T11:28:50.430085+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:55.438217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435770+0000","last_change":"2023-08-27T11:28:50.432294+0000","last_active":"2023-08-27T11:28:50.435770+0000","last_peered":"2023-08-27T11:28:50.435770+0000","last_clean":"2023-08-27T11:28:50.435770+0000","last_became_active":"2023-08-27T11:28:50.431507+0000","last_became_peered":"2023-08-27T11:28:50.431507+0000","last_unstale":"2023-08-27T11:28:50.435770+0000","last_undegraded":"2023-08-27T11:28:50.435770+0000","last_fullsized":"2023-08-27T11:28:50.435770+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:31:34.560849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"152'6","reported_seq":19,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412540+0000","last_change":"2023-08-27T11:28:49.430593+0000","last_active":"2023-08-27T11:28:51.412540+0000","last_peered":"2023-08-27T11:28:51.412540+0000","last_clean":"2023-08-27T11:28:51.412540+0000","last_became_active":"2023-08-27T11:28:49.430185+0000","last_became_peered":"2023-08-27T11:28:49.430185+0000","last_unstale":"2023-08-27T11:28:51.412540+0000","last_undegraded":"2023-08-27T11:28:51.412540+0000","last_fullsized":"2023-08-27T11:28:51.412540+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:41:42.671975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413584+0000","last_change":"2023-08-27T11:28:50.430732+0000","last_active":"2023-08-27T11:28:51.413584+0000","last_peered":"2023-08-27T11:28:51.413584+0000","last_clean":"2023-08-27T11:28:51.413584+0000","last_became_active":"2023-08-27T11:28:50.430469+0000","last_became_peered":"2023-08-27T11:28:50.430469+0000","last_unstale":"2023-08-27T11:28:51.413584+0000","last_undegraded":"2023-08-27T11:28:51.413584+0000","last_fullsized":"2023-08-27T11:28:51.413584+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:03:22.319458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"152'2","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413664+0000","last_change":"2023-08-27T11:28:49.421568+0000","last_active":"2023-08-27T11:28:51.413664+0000","last_peered":"2023-08-27T11:28:51.413664+0000","last_clean":"2023-08-27T11:28:51.413664+0000","last_became_active":"2023-08-27T11:28:49.421390+0000","last_became_peered":"2023-08-27T11:28:49.421390+0000","last_unstale":"2023-08-27T11:28:51.413664+0000","last_undegraded":"2023-08-27T11:28:51.413664+0000","last_fullsized":"2023-08-27T11:28:51.413664+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:30.430662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076657+0000","last_change":"2023-08-27T11:28:50.421728+0000","last_active":"2023-08-27T11:28:52.076657+0000","last_peered":"2023-08-27T11:28:52.076657+0000","last_clean":"2023-08-27T11:28:52.076657+0000","last_became_active":"2023-08-27T11:28:50.420814+0000","last_became_peered":"2023-08-27T11:28:50.420814+0000","last_unstale":"2023-08-27T11:28:52.076657+0000","last_undegraded":"2023-08-27T11:28:52.076657+0000","last_fullsized":"2023-08-27T11:28:52.076657+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:15.652339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076403+0000","last_change":"2023-08-27T11:28:49.420502+0000","last_active":"2023-08-27T11:28:52.076403+0000","last_peered":"2023-08-27T11:28:52.076403+0000","last_clean":"2023-08-27T11:28:52.076403+0000","last_became_active":"2023-08-27T11:28:49.420253+0000","last_became_peered":"2023-08-27T11:28:49.420253+0000","last_unstale":"2023-08-27T11:28:52.076403+0000","last_undegraded":"2023-08-27T11:28:52.076403+0000","last_fullsized":"2023-08-27T11:28:52.076403+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:05:57.560435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.432920+0000","last_change":"2023-08-27T11:28:50.432278+0000","last_active":"2023-08-27T11:28:50.432920+0000","last_peered":"2023-08-27T11:28:50.432920+0000","last_clean":"2023-08-27T11:28:50.432920+0000","last_became_active":"2023-08-27T11:28:50.431984+0000","last_became_peered":"2023-08-27T11:28:50.431984+0000","last_unstale":"2023-08-27T11:28:50.432920+0000","last_undegraded":"2023-08-27T11:28:50.432920+0000","last_fullsized":"2023-08-27T11:28:50.432920+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:15:52.341802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.407697+0000","last_change":"2023-08-27T11:28:49.425113+0000","last_active":"2023-08-27T11:28:50.407697+0000","last_peered":"2023-08-27T11:28:50.407697+0000","last_clean":"2023-08-27T11:28:50.407697+0000","last_became_active":"2023-08-27T11:28:49.424860+0000","last_became_peered":"2023-08-27T11:28:49.424860+0000","last_unstale":"2023-08-27T11:28:50.407697+0000","last_undegraded":"2023-08-27T11:28:50.407697+0000","last_fullsized":"2023-08-27T11:28:50.407697+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:12:38.264868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412725+0000","last_change":"2023-08-27T11:28:50.423760+0000","last_active":"2023-08-27T11:28:51.412725+0000","last_peered":"2023-08-27T11:28:51.412725+0000","last_clean":"2023-08-27T11:28:51.412725+0000","last_became_active":"2023-08-27T11:28:50.423466+0000","last_became_peered":"2023-08-27T11:28:50.423466+0000","last_unstale":"2023-08-27T11:28:51.412725+0000","last_undegraded":"2023-08-27T11:28:51.412725+0000","last_fullsized":"2023-08-27T11:28:51.412725+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:31:08.006587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"152'8","reported_seq":19,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435230+0000","last_change":"2023-08-27T11:28:49.423618+0000","last_active":"2023-08-27T11:28:50.435230+0000","last_peered":"2023-08-27T11:28:50.435230+0000","last_clean":"2023-08-27T11:28:50.435230+0000","last_became_active":"2023-08-27T11:28:49.423160+0000","last_became_peered":"2023-08-27T11:28:49.423160+0000","last_unstale":"2023-08-27T11:28:50.435230+0000","last_undegraded":"2023-08-27T11:28:50.435230+0000","last_fullsized":"2023-08-27T11:28:50.435230+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:56.773129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413529+0000","last_change":"2023-08-27T11:28:50.428616+0000","last_active":"2023-08-27T11:28:51.413529+0000","last_peered":"2023-08-27T11:28:51.413529+0000","last_clean":"2023-08-27T11:28:51.413529+0000","last_became_active":"2023-08-27T11:28:50.428002+0000","last_became_peered":"2023-08-27T11:28:50.428002+0000","last_unstale":"2023-08-27T11:28:51.413529+0000","last_undegraded":"2023-08-27T11:28:51.413529+0000","last_fullsized":"2023-08-27T11:28:51.413529+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:23:39.038605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"152'1","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320661+0000","last_change":"2023-08-27T11:28:49.423762+0000","last_active":"2023-08-27T11:28:52.320661+0000","last_peered":"2023-08-27T11:28:52.320661+0000","last_clean":"2023-08-27T11:28:52.320661+0000","last_became_active":"2023-08-27T11:28:49.423480+0000","last_became_peered":"2023-08-27T11:28:49.423480+0000","last_unstale":"2023-08-27T11:28:52.320661+0000","last_undegraded":"2023-08-27T11:28:52.320661+0000","last_fullsized":"2023-08-27T11:28:52.320661+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:58:55.021392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320165+0000","last_change":"2023-08-27T11:28:50.427403+0000","last_active":"2023-08-27T11:28:52.320165+0000","last_peered":"2023-08-27T11:28:52.320165+0000","last_clean":"2023-08-27T11:28:52.320165+0000","last_became_active":"2023-08-27T11:28:50.425494+0000","last_became_peered":"2023-08-27T11:28:50.425494+0000","last_unstale":"2023-08-27T11:28:52.320165+0000","last_undegraded":"2023-08-27T11:28:52.320165+0000","last_fullsized":"2023-08-27T11:28:52.320165+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:04:25.612634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"152'2","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320499+0000","last_change":"2023-08-27T11:28:49.427032+0000","last_active":"2023-08-27T11:28:52.320499+0000","last_peered":"2023-08-27T11:28:52.320499+0000","last_clean":"2023-08-27T11:28:52.320499+0000","last_became_active":"2023-08-27T11:28:49.426675+0000","last_became_peered":"2023-08-27T11:28:49.426675+0000","last_unstale":"2023-08-27T11:28:52.320499+0000","last_undegraded":"2023-08-27T11:28:52.320499+0000","last_fullsized":"2023-08-27T11:28:52.320499+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:15:30.794231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320352+0000","last_change":"2023-08-27T11:28:50.429724+0000","last_active":"2023-08-27T11:28:52.320352+0000","last_peered":"2023-08-27T11:28:52.320352+0000","last_clean":"2023-08-27T11:28:52.320352+0000","last_became_active":"2023-08-27T11:28:50.429322+0000","last_became_peered":"2023-08-27T11:28:50.429322+0000","last_unstale":"2023-08-27T11:28:52.320352+0000","last_undegraded":"2023-08-27T11:28:52.320352+0000","last_fullsized":"2023-08-27T11:28:52.320352+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:03.500060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"152'1","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413792+0000","last_change":"2023-08-27T11:28:49.426757+0000","last_active":"2023-08-27T11:28:51.413792+0000","last_peered":"2023-08-27T11:28:51.413792+0000","last_clean":"2023-08-27T11:28:51.413792+0000","last_became_active":"2023-08-27T11:28:49.426454+0000","last_became_peered":"2023-08-27T11:28:49.426454+0000","last_unstale":"2023-08-27T11:28:51.413792+0000","last_undegraded":"2023-08-27T11:28:51.413792+0000","last_fullsized":"2023-08-27T11:28:51.413792+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:17:32.469376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413837+0000","last_change":"2023-08-27T11:28:50.421401+0000","last_active":"2023-08-27T11:28:51.413837+0000","last_peered":"2023-08-27T11:28:51.413837+0000","last_clean":"2023-08-27T11:28:51.413837+0000","last_became_active":"2023-08-27T11:28:50.421132+0000","last_became_peered":"2023-08-27T11:28:50.421132+0000","last_unstale":"2023-08-27T11:28:51.413837+0000","last_undegraded":"2023-08-27T11:28:51.413837+0000","last_fullsized":"2023-08-27T11:28:51.413837+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:05:36.872167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.421782+0000","last_change":"2023-08-27T11:28:49.427420+0000","last_active":"2023-08-27T11:28:50.421782+0000","last_peered":"2023-08-27T11:28:50.421782+0000","last_clean":"2023-08-27T11:28:50.421782+0000","last_became_active":"2023-08-27T11:28:49.426995+0000","last_became_peered":"2023-08-27T11:28:49.426995+0000","last_unstale":"2023-08-27T11:28:50.421782+0000","last_undegraded":"2023-08-27T11:28:50.421782+0000","last_fullsized":"2023-08-27T11:28:50.421782+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:45:12.351029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413727+0000","last_change":"2023-08-27T11:28:50.425756+0000","last_active":"2023-08-27T11:28:51.413727+0000","last_peered":"2023-08-27T11:28:51.413727+0000","last_clean":"2023-08-27T11:28:51.413727+0000","last_became_active":"2023-08-27T11:28:50.425479+0000","last_became_peered":"2023-08-27T11:28:50.425479+0000","last_unstale":"2023-08-27T11:28:51.413727+0000","last_undegraded":"2023-08-27T11:28:51.413727+0000","last_fullsized":"2023-08-27T11:28:51.413727+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:46:13.434745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"152'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.430198+0000","last_change":"2023-08-27T11:28:49.419592+0000","last_active":"2023-08-27T11:28:50.430198+0000","last_peered":"2023-08-27T11:28:50.430198+0000","last_clean":"2023-08-27T11:28:50.430198+0000","last_became_active":"2023-08-27T11:28:49.419334+0000","last_became_peered":"2023-08-27T11:28:49.419334+0000","last_unstale":"2023-08-27T11:28:50.430198+0000","last_undegraded":"2023-08-27T11:28:50.430198+0000","last_fullsized":"2023-08-27T11:28:50.430198+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:59.764142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412832+0000","last_change":"2023-08-27T11:28:50.429405+0000","last_active":"2023-08-27T11:28:51.412832+0000","last_peered":"2023-08-27T11:28:51.412832+0000","last_clean":"2023-08-27T11:28:51.412832+0000","last_became_active":"2023-08-27T11:28:50.428635+0000","last_became_peered":"2023-08-27T11:28:50.428635+0000","last_unstale":"2023-08-27T11:28:51.412832+0000","last_undegraded":"2023-08-27T11:28:51.412832+0000","last_fullsized":"2023-08-27T11:28:51.412832+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:01:26.641877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413714+0000","last_change":"2023-08-27T11:28:49.417329+0000","last_active":"2023-08-27T11:28:51.413714+0000","last_peered":"2023-08-27T11:28:51.413714+0000","last_clean":"2023-08-27T11:28:51.413714+0000","last_became_active":"2023-08-27T11:28:49.417066+0000","last_became_peered":"2023-08-27T11:28:49.417066+0000","last_unstale":"2023-08-27T11:28:51.413714+0000","last_undegraded":"2023-08-27T11:28:51.413714+0000","last_fullsized":"2023-08-27T11:28:51.413714+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:45:03.177105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.427623+0000","last_change":"2023-08-27T11:28:50.424960+0000","last_active":"2023-08-27T11:28:50.427623+0000","last_peered":"2023-08-27T11:28:50.427623+0000","last_clean":"2023-08-27T11:28:50.427623+0000","last_became_active":"2023-08-27T11:28:50.424595+0000","last_became_peered":"2023-08-27T11:28:50.424595+0000","last_unstale":"2023-08-27T11:28:50.427623+0000","last_undegraded":"2023-08-27T11:28:50.427623+0000","last_fullsized":"2023-08-27T11:28:50.427623+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:55:09.192293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.410026+0000","last_change":"2023-08-27T11:28:49.420731+0000","last_active":"2023-08-27T11:28:50.410026+0000","last_peered":"2023-08-27T11:28:50.410026+0000","last_clean":"2023-08-27T11:28:50.410026+0000","last_became_active":"2023-08-27T11:28:49.420454+0000","last_became_peered":"2023-08-27T11:28:49.420454+0000","last_unstale":"2023-08-27T11:28:50.410026+0000","last_undegraded":"2023-08-27T11:28:50.410026+0000","last_fullsized":"2023-08-27T11:28:50.410026+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:20:40.703868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414495+0000","last_change":"2023-08-27T11:28:50.433625+0000","last_active":"2023-08-27T11:28:51.414495+0000","last_peered":"2023-08-27T11:28:51.414495+0000","last_clean":"2023-08-27T11:28:51.414495+0000","last_became_active":"2023-08-27T11:28:50.433361+0000","last_became_peered":"2023-08-27T11:28:50.433361+0000","last_unstale":"2023-08-27T11:28:51.414495+0000","last_undegraded":"2023-08-27T11:28:51.414495+0000","last_fullsized":"2023-08-27T11:28:51.414495+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:52:51.436030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320152+0000","last_change":"2023-08-27T11:28:49.421223+0000","last_active":"2023-08-27T11:28:52.320152+0000","last_peered":"2023-08-27T11:28:52.320152+0000","last_clean":"2023-08-27T11:28:52.320152+0000","last_became_active":"2023-08-27T11:28:49.420945+0000","last_became_peered":"2023-08-27T11:28:49.420945+0000","last_unstale":"2023-08-27T11:28:52.320152+0000","last_undegraded":"2023-08-27T11:28:52.320152+0000","last_fullsized":"2023-08-27T11:28:52.320152+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:03:06.795445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413732+0000","last_change":"2023-08-27T11:28:50.430556+0000","last_active":"2023-08-27T11:28:51.413732+0000","last_peered":"2023-08-27T11:28:51.413732+0000","last_clean":"2023-08-27T11:28:51.413732+0000","last_became_active":"2023-08-27T11:28:50.428358+0000","last_became_peered":"2023-08-27T11:28:50.428358+0000","last_unstale":"2023-08-27T11:28:51.413732+0000","last_undegraded":"2023-08-27T11:28:51.413732+0000","last_fullsized":"2023-08-27T11:28:51.413732+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:59:14.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"152'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.436244+0000","last_change":"2023-08-27T11:28:49.422807+0000","last_active":"2023-08-27T11:28:50.436244+0000","last_peered":"2023-08-27T11:28:50.436244+0000","last_clean":"2023-08-27T11:28:50.436244+0000","last_became_active":"2023-08-27T11:28:49.422560+0000","last_became_peered":"2023-08-27T11:28:49.422560+0000","last_unstale":"2023-08-27T11:28:50.436244+0000","last_undegraded":"2023-08-27T11:28:50.436244+0000","last_fullsized":"2023-08-27T11:28:50.436244+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:22:23.805738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076413+0000","last_change":"2023-08-27T11:28:50.418010+0000","last_active":"2023-08-27T11:28:52.076413+0000","last_peered":"2023-08-27T11:28:52.076413+0000","last_clean":"2023-08-27T11:28:52.076413+0000","last_became_active":"2023-08-27T11:28:50.417757+0000","last_became_peered":"2023-08-27T11:28:50.417757+0000","last_unstale":"2023-08-27T11:28:52.076413+0000","last_undegraded":"2023-08-27T11:28:52.076413+0000","last_fullsized":"2023-08-27T11:28:52.076413+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:18:14.857419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414261+0000","last_change":"2023-08-27T11:28:49.417726+0000","last_active":"2023-08-27T11:28:51.414261+0000","last_peered":"2023-08-27T11:28:51.414261+0000","last_clean":"2023-08-27T11:28:51.414261+0000","last_became_active":"2023-08-27T11:28:49.417390+0000","last_became_peered":"2023-08-27T11:28:49.417390+0000","last_unstale":"2023-08-27T11:28:51.414261+0000","last_undegraded":"2023-08-27T11:28:51.414261+0000","last_fullsized":"2023-08-27T11:28:51.414261+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:20:08.781333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435586+0000","last_change":"2023-08-27T11:28:50.432498+0000","last_active":"2023-08-27T11:28:50.435586+0000","last_peered":"2023-08-27T11:28:50.435586+0000","last_clean":"2023-08-27T11:28:50.435586+0000","last_became_active":"2023-08-27T11:28:50.432034+0000","last_became_peered":"2023-08-27T11:28:50.432034+0000","last_unstale":"2023-08-27T11:28:50.435586+0000","last_undegraded":"2023-08-27T11:28:50.435586+0000","last_fullsized":"2023-08-27T11:28:50.435586+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:37:46.190181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.409355+0000","last_change":"2023-08-27T11:28:49.424803+0000","last_active":"2023-08-27T11:28:50.409355+0000","last_peered":"2023-08-27T11:28:50.409355+0000","last_clean":"2023-08-27T11:28:50.409355+0000","last_became_active":"2023-08-27T11:28:49.424512+0000","last_became_peered":"2023-08-27T11:28:49.424512+0000","last_unstale":"2023-08-27T11:28:50.409355+0000","last_undegraded":"2023-08-27T11:28:50.409355+0000","last_fullsized":"2023-08-27T11:28:50.409355+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:33:57.191788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320165+0000","last_change":"2023-08-27T11:28:50.423533+0000","last_active":"2023-08-27T11:28:52.320165+0000","last_peered":"2023-08-27T11:28:52.320165+0000","last_clean":"2023-08-27T11:28:52.320165+0000","last_became_active":"2023-08-27T11:28:50.423252+0000","last_became_peered":"2023-08-27T11:28:50.423252+0000","last_unstale":"2023-08-27T11:28:52.320165+0000","last_undegraded":"2023-08-27T11:28:52.320165+0000","last_fullsized":"2023-08-27T11:28:52.320165+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:44:52.638610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320143+0000","last_change":"2023-08-27T11:28:49.425297+0000","last_active":"2023-08-27T11:28:52.320143+0000","last_peered":"2023-08-27T11:28:52.320143+0000","last_clean":"2023-08-27T11:28:52.320143+0000","last_became_active":"2023-08-27T11:28:49.424837+0000","last_became_peered":"2023-08-27T11:28:49.424837+0000","last_unstale":"2023-08-27T11:28:52.320143+0000","last_undegraded":"2023-08-27T11:28:52.320143+0000","last_fullsized":"2023-08-27T11:28:52.320143+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:59:15.798185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414061+0000","last_change":"2023-08-27T11:28:50.420363+0000","last_active":"2023-08-27T11:28:51.414061+0000","last_peered":"2023-08-27T11:28:51.414061+0000","last_clean":"2023-08-27T11:28:51.414061+0000","last_became_active":"2023-08-27T11:28:50.418825+0000","last_became_peered":"2023-08-27T11:28:50.418825+0000","last_unstale":"2023-08-27T11:28:51.414061+0000","last_undegraded":"2023-08-27T11:28:51.414061+0000","last_fullsized":"2023-08-27T11:28:51.414061+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:33:14.575682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414017+0000","last_change":"2023-08-27T11:28:49.417180+0000","last_active":"2023-08-27T11:28:51.414017+0000","last_peered":"2023-08-27T11:28:51.414017+0000","last_clean":"2023-08-27T11:28:51.414017+0000","last_became_active":"2023-08-27T11:28:49.416732+0000","last_became_peered":"2023-08-27T11:28:49.416732+0000","last_unstale":"2023-08-27T11:28:51.414017+0000","last_undegraded":"2023-08-27T11:28:51.414017+0000","last_fullsized":"2023-08-27T11:28:51.414017+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:56.320705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412822+0000","last_change":"2023-08-27T11:28:50.430756+0000","last_active":"2023-08-27T11:28:51.412822+0000","last_peered":"2023-08-27T11:28:51.412822+0000","last_clean":"2023-08-27T11:28:51.412822+0000","last_became_active":"2023-08-27T11:28:50.430504+0000","last_became_peered":"2023-08-27T11:28:50.430504+0000","last_unstale":"2023-08-27T11:28:51.412822+0000","last_undegraded":"2023-08-27T11:28:51.412822+0000","last_fullsized":"2023-08-27T11:28:51.412822+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:27:33.047800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.408583+0000","last_change":"2023-08-27T11:28:49.418290+0000","last_active":"2023-08-27T11:28:50.408583+0000","last_peered":"2023-08-27T11:28:50.408583+0000","last_clean":"2023-08-27T11:28:50.408583+0000","last_became_active":"2023-08-27T11:28:49.417576+0000","last_became_peered":"2023-08-27T11:28:49.417576+0000","last_unstale":"2023-08-27T11:28:50.408583+0000","last_undegraded":"2023-08-27T11:28:50.408583+0000","last_fullsized":"2023-08-27T11:28:50.408583+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:56:19.689714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412822+0000","last_change":"2023-08-27T11:28:50.432391+0000","last_active":"2023-08-27T11:28:51.412822+0000","last_peered":"2023-08-27T11:28:51.412822+0000","last_clean":"2023-08-27T11:28:51.412822+0000","last_became_active":"2023-08-27T11:28:50.432027+0000","last_became_peered":"2023-08-27T11:28:50.432027+0000","last_unstale":"2023-08-27T11:28:51.412822+0000","last_undegraded":"2023-08-27T11:28:51.412822+0000","last_fullsized":"2023-08-27T11:28:51.412822+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:54:33.889906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320334+0000","last_change":"2023-08-27T11:28:49.425777+0000","last_active":"2023-08-27T11:28:52.320334+0000","last_peered":"2023-08-27T11:28:52.320334+0000","last_clean":"2023-08-27T11:28:52.320334+0000","last_became_active":"2023-08-27T11:28:49.425435+0000","last_became_peered":"2023-08-27T11:28:49.425435+0000","last_unstale":"2023-08-27T11:28:52.320334+0000","last_undegraded":"2023-08-27T11:28:52.320334+0000","last_fullsized":"2023-08-27T11:28:52.320334+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:38:39.524177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076692+0000","last_change":"2023-08-27T11:28:50.428682+0000","last_active":"2023-08-27T11:28:52.076692+0000","last_peered":"2023-08-27T11:28:52.076692+0000","last_clean":"2023-08-27T11:28:52.076692+0000","last_became_active":"2023-08-27T11:28:50.428428+0000","last_became_peered":"2023-08-27T11:28:50.428428+0000","last_unstale":"2023-08-27T11:28:52.076692+0000","last_undegraded":"2023-08-27T11:28:52.076692+0000","last_fullsized":"2023-08-27T11:28:52.076692+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:39:54.608535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.419695+0000","last_change":"2023-08-27T11:28:49.423875+0000","last_active":"2023-08-27T11:28:50.419695+0000","last_peered":"2023-08-27T11:28:50.419695+0000","last_clean":"2023-08-27T11:28:50.419695+0000","last_became_active":"2023-08-27T11:28:49.423545+0000","last_became_peered":"2023-08-27T11:28:49.423545+0000","last_unstale":"2023-08-27T11:28:50.419695+0000","last_undegraded":"2023-08-27T11:28:50.419695+0000","last_fullsized":"2023-08-27T11:28:50.419695+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:58:32.509833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413910+0000","last_change":"2023-08-27T11:28:50.433738+0000","last_active":"2023-08-27T11:28:51.413910+0000","last_peered":"2023-08-27T11:28:51.413910+0000","last_clean":"2023-08-27T11:28:51.413910+0000","last_became_active":"2023-08-27T11:28:50.433455+0000","last_became_peered":"2023-08-27T11:28:50.433455+0000","last_unstale":"2023-08-27T11:28:51.413910+0000","last_undegraded":"2023-08-27T11:28:51.413910+0000","last_fullsized":"2023-08-27T11:28:51.413910+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:26:04.520477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"152'2","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413622+0000","last_change":"2023-08-27T11:28:49.422456+0000","last_active":"2023-08-27T11:28:51.413622+0000","last_peered":"2023-08-27T11:28:51.413622+0000","last_clean":"2023-08-27T11:28:51.413622+0000","last_became_active":"2023-08-27T11:28:49.422214+0000","last_became_peered":"2023-08-27T11:28:49.422214+0000","last_unstale":"2023-08-27T11:28:51.413622+0000","last_undegraded":"2023-08-27T11:28:51.413622+0000","last_fullsized":"2023-08-27T11:28:51.413622+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:27:30.543860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":290,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414466+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:28:51.414466+0000","last_peered":"2023-08-27T11:28:51.414466+0000","last_clean":"2023-08-27T11:28:51.414466+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:28:51.414466+0000","last_undegraded":"2023-08-27T11:28:51.414466+0000","last_fullsized":"2023-08-27T11:28:51.414466+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:47:51.352733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435751+0000","last_change":"2023-08-27T11:28:50.432082+0000","last_active":"2023-08-27T11:28:50.435751+0000","last_peered":"2023-08-27T11:28:50.435751+0000","last_clean":"2023-08-27T11:28:50.435751+0000","last_became_active":"2023-08-27T11:28:50.430391+0000","last_became_peered":"2023-08-27T11:28:50.430391+0000","last_unstale":"2023-08-27T11:28:50.435751+0000","last_undegraded":"2023-08-27T11:28:50.435751+0000","last_fullsized":"2023-08-27T11:28:50.435751+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:17.009060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"152'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.425811+0000","last_change":"2023-08-27T11:28:49.423748+0000","last_active":"2023-08-27T11:28:50.425811+0000","last_peered":"2023-08-27T11:28:50.425811+0000","last_clean":"2023-08-27T11:28:50.425811+0000","last_became_active":"2023-08-27T11:28:49.423436+0000","last_became_peered":"2023-08-27T11:28:49.423436+0000","last_unstale":"2023-08-27T11:28:50.425811+0000","last_undegraded":"2023-08-27T11:28:50.425811+0000","last_fullsized":"2023-08-27T11:28:50.425811+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:34:57.916663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":290,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.414363+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:28:51.414363+0000","last_peered":"2023-08-27T11:28:51.414363+0000","last_clean":"2023-08-27T11:28:51.414363+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:28:51.414363+0000","last_undegraded":"2023-08-27T11:28:51.414363+0000","last_fullsized":"2023-08-27T11:28:51.414363+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:35:40.185824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.430197+0000","last_change":"2023-08-27T11:28:50.424133+0000","last_active":"2023-08-27T11:28:50.430197+0000","last_peered":"2023-08-27T11:28:50.430197+0000","last_clean":"2023-08-27T11:28:50.430197+0000","last_became_active":"2023-08-27T11:28:50.423846+0000","last_became_peered":"2023-08-27T11:28:50.423846+0000","last_unstale":"2023-08-27T11:28:50.430197+0000","last_undegraded":"2023-08-27T11:28:50.430197+0000","last_fullsized":"2023-08-27T11:28:50.430197+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:44:54.808280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"152'1","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412418+0000","last_change":"2023-08-27T11:28:49.428066+0000","last_active":"2023-08-27T11:28:51.412418+0000","last_peered":"2023-08-27T11:28:51.412418+0000","last_clean":"2023-08-27T11:28:51.412418+0000","last_became_active":"2023-08-27T11:28:49.427780+0000","last_became_peered":"2023-08-27T11:28:49.427780+0000","last_unstale":"2023-08-27T11:28:51.412418+0000","last_undegraded":"2023-08-27T11:28:51.412418+0000","last_fullsized":"2023-08-27T11:28:51.412418+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:13.146471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":250,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320354+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:28:52.320354+0000","last_peered":"2023-08-27T11:28:52.320354+0000","last_clean":"2023-08-27T11:28:52.320354+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:28:52.320354+0000","last_undegraded":"2023-08-27T11:28:52.320354+0000","last_fullsized":"2023-08-27T11:28:52.320354+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:16:50.455378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320275+0000","last_change":"2023-08-27T11:28:50.425521+0000","last_active":"2023-08-27T11:28:52.320275+0000","last_peered":"2023-08-27T11:28:52.320275+0000","last_clean":"2023-08-27T11:28:52.320275+0000","last_became_active":"2023-08-27T11:28:50.425256+0000","last_became_peered":"2023-08-27T11:28:50.425256+0000","last_unstale":"2023-08-27T11:28:52.320275+0000","last_undegraded":"2023-08-27T11:28:52.320275+0000","last_fullsized":"2023-08-27T11:28:52.320275+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:11.183530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"152'1","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.428290+0000","last_change":"2023-08-27T11:28:49.422723+0000","last_active":"2023-08-27T11:28:50.428290+0000","last_peered":"2023-08-27T11:28:50.428290+0000","last_clean":"2023-08-27T11:28:50.428290+0000","last_became_active":"2023-08-27T11:28:49.418557+0000","last_became_peered":"2023-08-27T11:28:49.418557+0000","last_unstale":"2023-08-27T11:28:50.428290+0000","last_undegraded":"2023-08-27T11:28:50.428290+0000","last_fullsized":"2023-08-27T11:28:50.428290+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:22:32.303217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":244,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412405+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:28:51.412405+0000","last_peered":"2023-08-27T11:28:51.412405+0000","last_clean":"2023-08-27T11:28:51.412405+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:28:51.412405+0000","last_undegraded":"2023-08-27T11:28:51.412405+0000","last_fullsized":"2023-08-27T11:28:51.412405+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:24:45.940962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.424875+0000","last_change":"2023-08-27T11:28:50.418664+0000","last_active":"2023-08-27T11:28:50.424875+0000","last_peered":"2023-08-27T11:28:50.424875+0000","last_clean":"2023-08-27T11:28:50.424875+0000","last_became_active":"2023-08-27T11:28:50.417708+0000","last_became_peered":"2023-08-27T11:28:50.417708+0000","last_unstale":"2023-08-27T11:28:50.424875+0000","last_undegraded":"2023-08-27T11:28:50.424875+0000","last_fullsized":"2023-08-27T11:28:50.424875+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:55:24.639054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076337+0000","last_change":"2023-08-27T11:28:49.426796+0000","last_active":"2023-08-27T11:28:52.076337+0000","last_peered":"2023-08-27T11:28:52.076337+0000","last_clean":"2023-08-27T11:28:52.076337+0000","last_became_active":"2023-08-27T11:28:49.426531+0000","last_became_peered":"2023-08-27T11:28:49.426531+0000","last_unstale":"2023-08-27T11:28:52.076337+0000","last_undegraded":"2023-08-27T11:28:52.076337+0000","last_fullsized":"2023-08-27T11:28:52.076337+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:08:12.522978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":244,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412250+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:28:51.412250+0000","last_peered":"2023-08-27T11:28:51.412250+0000","last_clean":"2023-08-27T11:28:51.412250+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:28:51.412250+0000","last_undegraded":"2023-08-27T11:28:51.412250+0000","last_fullsized":"2023-08-27T11:28:51.412250+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:44.401650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.432792+0000","last_change":"2023-08-27T11:28:50.432456+0000","last_active":"2023-08-27T11:28:50.432792+0000","last_peered":"2023-08-27T11:28:50.432792+0000","last_clean":"2023-08-27T11:28:50.432792+0000","last_became_active":"2023-08-27T11:28:50.432092+0000","last_became_peered":"2023-08-27T11:28:50.432092+0000","last_unstale":"2023-08-27T11:28:50.432792+0000","last_undegraded":"2023-08-27T11:28:50.432792+0000","last_fullsized":"2023-08-27T11:28:50.432792+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:16:51.033471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412051+0000","last_change":"2023-08-27T11:28:49.428023+0000","last_active":"2023-08-27T11:28:51.412051+0000","last_peered":"2023-08-27T11:28:51.412051+0000","last_clean":"2023-08-27T11:28:51.412051+0000","last_became_active":"2023-08-27T11:28:49.427710+0000","last_became_peered":"2023-08-27T11:28:49.427710+0000","last_unstale":"2023-08-27T11:28:51.412051+0000","last_undegraded":"2023-08-27T11:28:51.412051+0000","last_fullsized":"2023-08-27T11:28:51.412051+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:51.965584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":394,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.412025+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:28:51.412025+0000","last_peered":"2023-08-27T11:28:51.412025+0000","last_clean":"2023-08-27T11:28:51.412025+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:28:51.412025+0000","last_undegraded":"2023-08-27T11:28:51.412025+0000","last_fullsized":"2023-08-27T11:28:51.412025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:37.921403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":256,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.076502+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:28:52.076502+0000","last_peered":"2023-08-27T11:28:52.076502+0000","last_clean":"2023-08-27T11:28:52.076502+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:28:52.076502+0000","last_undegraded":"2023-08-27T11:28:52.076502+0000","last_fullsized":"2023-08-27T11:28:52.076502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:31:38.716999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.319973+0000","last_change":"2023-08-27T11:28:50.427447+0000","last_active":"2023-08-27T11:28:52.319973+0000","last_peered":"2023-08-27T11:28:52.319973+0000","last_clean":"2023-08-27T11:28:52.319973+0000","last_became_active":"2023-08-27T11:28:50.427177+0000","last_became_peered":"2023-08-27T11:28:50.427177+0000","last_unstale":"2023-08-27T11:28:52.319973+0000","last_undegraded":"2023-08-27T11:28:52.319973+0000","last_fullsized":"2023-08-27T11:28:52.319973+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:42.832759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"152'1","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:52.320503+0000","last_change":"2023-08-27T11:28:49.424332+0000","last_active":"2023-08-27T11:28:52.320503+0000","last_peered":"2023-08-27T11:28:52.320503+0000","last_clean":"2023-08-27T11:28:52.320503+0000","last_became_active":"2023-08-27T11:28:49.424070+0000","last_became_peered":"2023-08-27T11:28:49.424070+0000","last_unstale":"2023-08-27T11:28:52.320503+0000","last_undegraded":"2023-08-27T11:28:52.320503+0000","last_fullsized":"2023-08-27T11:28:52.320503+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:09:04.126197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":246,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413751+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:28:51.413751+0000","last_peered":"2023-08-27T11:28:51.413751+0000","last_clean":"2023-08-27T11:28:51.413751+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:28:51.413751+0000","last_undegraded":"2023-08-27T11:28:51.413751+0000","last_fullsized":"2023-08-27T11:28:51.413751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:04:06.142501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.413582+0000","last_change":"2023-08-27T11:28:50.426265+0000","last_active":"2023-08-27T11:28:51.413582+0000","last_peered":"2023-08-27T11:28:51.413582+0000","last_clean":"2023-08-27T11:28:51.413582+0000","last_became_active":"2023-08-27T11:28:50.425861+0000","last_became_peered":"2023-08-27T11:28:50.425861+0000","last_unstale":"2023-08-27T11:28:51.413582+0000","last_undegraded":"2023-08-27T11:28:51.413582+0000","last_fullsized":"2023-08-27T11:28:51.413582+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.397316+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.397316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:02:11.046625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"152'4","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.435013+0000","last_change":"2023-08-27T11:28:49.422431+0000","last_active":"2023-08-27T11:28:50.435013+0000","last_peered":"2023-08-27T11:28:50.435013+0000","last_clean":"2023-08-27T11:28:50.435013+0000","last_became_active":"2023-08-27T11:28:49.420423+0000","last_became_peered":"2023-08-27T11:28:49.420423+0000","last_unstale":"2023-08-27T11:28:50.435013+0000","last_undegraded":"2023-08-27T11:28:50.435013+0000","last_fullsized":"2023-08-27T11:28:50.435013+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.387317+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.387317+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:38:16.718705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":274,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-08-27T11:28:50.409448+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:28:50.409448+0000","last_peered":"2023-08-27T11:28:50.409448+0000","last_clean":"2023-08-27T11:28:50.409448+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:28:50.409448+0000","last_undegraded":"2023-08-27T11:28:50.409448+0000","last_fullsized":"2023-08-27T11:28:50.409448+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:01:16.849818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574995,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33668,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338172,"statfs":{"total":96636764160,"available":96602288128,"internally_reserved":0,"allocated":2547712,"data_stored":1514677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.44,"5min":0.28899999999999998,"15min":0.22700000000000001},"max":{"1min":0.72799999999999998,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.39800000000000002,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.75600000000000001,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.46899999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55700000000000005,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.872},"last":0.499},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54000000000000004,"15min":0.52500000000000002},"min":{"1min":0.374,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.53500000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.497},"min":{"1min":0.377,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.72399999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.46100000000000002,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.90200000000000002},"last":0.42999999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.443,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77700000000000002,"5min":0.996,"15min":0.996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.48899999999999999,"5min":0.38600000000000001,"15min":0.34399999999999997},"max":{"1min":0.76400000000000001,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.628}]},{"osd":4,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.496,"15min":0.504},"min":{"1min":0.35899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.42399999999999999,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.73199999999999998,"5min":1.224,"15min":1.224},"last":0.38100000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.78100000000000003,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73199999999999998,"5min":0.78400000000000003,"15min":0.84799999999999998},"last":0.41299999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84499999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.47299999999999998,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1953792,"data_stored":924821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.29099999999999998},"max":{"1min":0.73399999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.435},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80300000000000005,"5min":0.83799999999999997,"15min":0.89800000000000002},"last":0.54300000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.873},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.84399999999999997},"last":0.497}]},{"osd":2,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.85899999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.69699999999999995,"5min":0.81100000000000005,"15min":2.0830000000000002},"last":0.52300000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57599999999999996,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84499999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.80200000000000005,"5min":0.80700000000000005,"15min":0.93799999999999994},"last":0.45200000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.753,"5min":0.753,"15min":0.83099999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.374},"max":{"1min":0.81399999999999995,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.57799999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.875},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Aug 27 11:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.34899999999999998},"max":{"1min":0.82999999999999996,"5min":0.83999999999999997,"15min":0.91400000000000003},"last":0.51100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33652,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338188,"statfs":{"total":96636764160,"available":96602304512,"internally_reserved":0,"allocated":2531328,"data_stored":1512995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.312,"15min":0.51400000000000001},"min":{"1min":0.221,"5min":0.191,"15min":0.191},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":11.971},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.72799999999999998},"min":{"1min":0.24299999999999999,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":23.045999999999999},"last":0.30399999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.54200000000000004},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.221},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":11.928000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.316,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.224},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.58599999999999997},"last":0.26200000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39000000000000001,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":10.99},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.76500000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49299999999999999,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.42799999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.41499999999999998,"5min":0.40400000000000003,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.90700000000000003,"15min":1.26},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.74099999999999999},"min":{"1min":0.44900000000000001,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.81899999999999995,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.96399999999999997},"min":{"1min":0.46700000000000003,"5min":0.40999999999999998,"15min":0.36199999999999999},"max":{"1min":0.76200000000000001,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.88700000000000001,"15min":1.296},"last":0.52600000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.76700000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.371},"max":{"1min":0.77800000000000002,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60099999999999998,"15min":0.98199999999999998},"min":{"1min":0.437,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":22.952000000000002},"last":0.61799999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59599999999999997,"15min":0.97799999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.84199999999999997,"15min":22.988},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.627},"min":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.83799999999999997,"5min":0.86399999999999999,"15min":1.327},"last":0.627}]}]},{"osd":0,"up_from":13,"seq":55834574995,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":914703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.316,"15min":0.311},"min":{"1min":0.224,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.46899999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34000000000000002,"15min":0.34799999999999998},"min":{"1min":0.26600000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.501,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.27400000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.35899999999999999,"15min":0.34399999999999997},"min":{"1min":0.26000000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.59899999999999998,"5min":0.628,"15min":0.63200000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.33900000000000002},"min":{"1min":0.26000000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.627,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.34200000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.34599999999999997},"min":{"1min":0.23499999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.59599999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.376},"min":{"1min":0.33700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.35599999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.56000000000000005},"min":{"1min":0.38400000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.85399999999999998,"15min":1.214},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.77500000000000002,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.51500000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.434,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.78300000000000003,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58999999999999997,"15min":0.59699999999999998},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76700000000000002,"5min":0.85399999999999998,"15min":1.603},"last":0.59499999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.503,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.75800000000000001,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.41799999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.878,"15min":0.878},"last":0.505}]},{"osd":7,"last update":"Sun Aug 27 11:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.64600000000000002},"min":{"1min":0.47299999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.74199999999999999,"5min":3.79,"15min":3.79},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1929216,"data_stored":916064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.30199999999999999},"min":{"1min":0.22800000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.377,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.31900000000000001,"15min":0.318},"min":{"1min":0.251,"5min":0.22700000000000001,"15min":0.19700000000000001},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.629},"last":0.35499999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.315,"15min":0.32200000000000001},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.40799999999999997,"5min":0.50800000000000001,"15min":0.54400000000000004},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.34100000000000003,"15min":0.34699999999999998},"min":{"1min":0.29999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.749},"last":0.34100000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.379},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.73699999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.39500000000000002},"min":{"1min":0.33600000000000002,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":0.53500000000000003,"5min":0.55100000000000005,"15min":0.69499999999999995},"last":0.38700000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.41799999999999998,"5min":0.371,"15min":0.36699999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.879},"last":0.39900000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.47199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.42899999999999999,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.40999999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.46000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.47899999999999998,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.83599999999999997},"last":0.504}]},{"osd":7,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.46899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.49199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574996,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1949696,"data_stored":922753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.29799999999999999,"15min":0.30299999999999999},"min":{"1min":0.221,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.435,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.318},"min":{"1min":0.26500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.27600000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.33800000000000002,"15min":0.32200000000000001},"min":{"1min":0.24399999999999999,"5min":0.22700000000000001,"15min":0.19500000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.36399999999999999,"15min":0.35199999999999998},"min":{"1min":0.27500000000000002,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":0.436}]},{"osd":2,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.376,"15min":0.36599999999999999},"min":{"1min":0.27400000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.373,"15min":0.36899999999999999},"min":{"1min":0.28699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.40600000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.55200000000000005},"min":{"1min":0.41099999999999998,"5min":0.33900000000000002,"15min":0.28599999999999998},"max":{"1min":0.80700000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.47999999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55100000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.38,"15min":0.314},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.44700000000000001,"5min":0.379,"15min":0.373},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.60999999999999999},"min":{"1min":0.47999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":1.0069999999999999,"15min":1.111},"last":0.57899999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58499999999999996,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":1.032,"15min":1.236},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574996,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1929216,"data_stored":916556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.33400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.40899999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.78700000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.41499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68999999999999995,"5min":0.746,"15min":0.76500000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.44,"5min":0.308,"15min":0.308},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.75}]},{"osd":2,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.35099999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69199999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.38600000000000001,"5min":0.32400000000000001,"15min":0.32200000000000001},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.80100000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56499999999999995,"15min":0.54200000000000004},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73699999999999999,"5min":0.80300000000000005,"15min":0.80600000000000005},"last":0.76700000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.51600000000000001,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60399999999999998,"15min":0.59499999999999997},"min":{"1min":0.46200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.57699999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.38100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73999999999999999,"5min":0.78900000000000003,"15min":0.90600000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.62,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.70399999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.47399999999999998,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.89300000000000002},"last":0.55700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574995,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1933312,"data_stored":916030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.318,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.96899999999999997},"last":0.499},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39000000000000001,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.71499999999999997,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.623}]},{"osd":1,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55300000000000005,"15min":0.53700000000000003},"min":{"1min":0.41699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81000000000000005,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52500000000000002,"15min":0.53700000000000003},"min":{"1min":0.38300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68000000000000005,"5min":0.72899999999999998,"15min":1.075},"last":0.67500000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.69199999999999995,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55400000000000005,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.373,"15min":0.307},"max":{"1min":0.747,"5min":0.77600000000000002,"15min":1.0580000000000001},"last":0.78900000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.57899999999999996},"min":{"1min":0.41899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":1.03},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.58999999999999997},"min":{"1min":0.50600000000000001,"5min":0.40699999999999997,"15min":0.35199999999999998},"max":{"1min":0.77700000000000002,"5min":0.77800000000000002,"15min":1.026},"last":0.73999999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.31,"15min":0.27000000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.86899999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.91400000000000003},"last":0.39800000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.42899999999999999,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.77600000000000002,"5min":0.79400000000000004,"15min":0.88200000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.44700000000000001,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.94199999999999995},"last":0.43099999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.95899999999999996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64200000000000002,"15min":0.65000000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.92000000000000004},"last":0.69999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:54.907 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:28:54.907 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:28:55.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.067+0000 7f457af67700 1 Processor -- start 2023-08-27T11:28:55.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.067+0000 7f457af67700 1 -- start start 2023-08-27T11:28:55.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f457af67700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4574146c20 0x7f4574160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f457af67700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f457406f790 0x7f457406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f457af67700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45740b5070 con 0x7f45740b3440 2023-08-27T11:28:55.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f457af67700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4574070180 con 0x7f457406f790 2023-08-27T11:28:55.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f457af67700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4574147000 con 0x7f4574146c20 2023-08-27T11:28:55.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f4579504700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f457406f790 0x7f457406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f4579504700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f457406f790 0x7f457406fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37168/0 (socket says 172.21.15.112:37168) 2023-08-27T11:28:55.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f4573fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f45740b3440 0x7f457415e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59780/0 (socket says 172.21.15.112:59780) 2023-08-27T11:28:55.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f4579504700 1 -- 172.21.15.112:0/2125093333 learned_addr learned my addr 172.21.15.112:0/2125093333 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:55.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f4579504700 1 -- 172.21.15.112:0/2125093333 >> v2:172.21.15.138:3300/0 conn(0x7f4574146c20 msgr2=0x7f4574160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:55.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f4578d03700 1 --2- 172.21.15.112:0/2125093333 >> v2:172.21.15.138:3300/0 conn(0x7f4574146c20 0x7f4574160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f4579504700 1 --2- 172.21.15.112:0/2125093333 >> v2:172.21.15.138:3300/0 conn(0x7f4574146c20 0x7f4574160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f45737fe700 1 -- 172.21.15.112:0/2125093333 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2193220436 0 0) 0x7f4560001110 con 0x7f45740b3440 2023-08-27T11:28:55.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f4579504700 1 -- 172.21.15.112:0/2125093333 >> v1:172.21.15.112:6789/0 conn(0x7f45740b3440 legacy=0x7f457415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f4579504700 1 -- 172.21.15.112:0/2125093333 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45741614a0 con 0x7f457406f790 2023-08-27T11:28:55.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f4579504700 1 --2- 172.21.15.112:0/2125093333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457406f790 0x7f457406fc40 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f45740b2d00 tx=0x7f456c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791354cf63fe76eb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:55.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f45737fe700 1 -- 172.21.15.112:0/2125093333 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c00e070 con 0x7f457406f790 2023-08-27T11:28:55.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f45737fe700 1 -- 172.21.15.112:0/2125093333 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c00b9c0 con 0x7f457406f790 2023-08-27T11:28:55.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.068+0000 7f45737fe700 1 -- 172.21.15.112:0/2125093333 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c00bd70 con 0x7f457406f790 2023-08-27T11:28:55.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.069+0000 7f457af67700 1 -- 172.21.15.112:0/2125093333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457406f790 msgr2=0x7f457406fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.069+0000 7f457af67700 1 --2- 172.21.15.112:0/2125093333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457406f790 0x7f457406fc40 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f45740b2d00 tx=0x7f456c00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:55.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.069+0000 7f457af67700 1 -- 172.21.15.112:0/2125093333 shutdown_connections 2023-08-27T11:28:55.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.069+0000 7f457af67700 1 --2- 172.21.15.112:0/2125093333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457406f790 0x7f457406fc40 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.069+0000 7f457af67700 1 --2- 172.21.15.112:0/2125093333 >> v2:172.21.15.138:3300/0 conn(0x7f4574146c20 0x7f4574160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.069+0000 7f457af67700 1 -- 172.21.15.112:0/2125093333 >> 172.21.15.112:0/2125093333 conn(0x7f45740b1e90 msgr2=0x7f45740b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:55.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.069+0000 7f457af67700 1 -- 172.21.15.112:0/2125093333 shutdown_connections 2023-08-27T11:28:55.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.069+0000 7f457af67700 1 -- 172.21.15.112:0/2125093333 wait complete. 2023-08-27T11:28:55.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.070+0000 7f457af67700 1 Processor -- start 2023-08-27T11:28:55.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.070+0000 7f457af67700 1 -- start start 2023-08-27T11:28:55.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.071+0000 7f457af67700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457406f790 0x7f45741f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.071+0000 7f457af67700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45740b3440 0x7f45740c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.071+0000 7f457af67700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4574146c20 0x7f457415f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.071+0000 7f457af67700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45740c77a0 con 0x7f457406f790 2023-08-27T11:28:55.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.071+0000 7f457af67700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45740c6b90 con 0x7f4574146c20 2023-08-27T11:28:55.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.071+0000 7f457af67700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45740c6e70 con 0x7f45740b3440 2023-08-27T11:28:55.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.071+0000 7f4578d03700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457406f790 0x7f45741f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.071+0000 7f4578d03700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457406f790 0x7f45741f5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37170/0 (socket says 172.21.15.112:37170) 2023-08-27T11:28:55.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.071+0000 7f4578d03700 1 -- 172.21.15.112:0/615779742 learned_addr learned my addr 172.21.15.112:0/615779742 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:55.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.071+0000 7f4578d03700 1 -- 172.21.15.112:0/615779742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45740b3440 msgr2=0x7f45740c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.071+0000 7f4578d03700 1 --2- 172.21.15.112:0/615779742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45740b3440 0x7f45740c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.071+0000 7f4578d03700 1 -- 172.21.15.112:0/615779742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4574146c20 msgr2=0x7f457415f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.071+0000 7f4578d03700 1 --2- 172.21.15.112:0/615779742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4574146c20 0x7f457415f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.072+0000 7f4578d03700 1 -- 172.21.15.112:0/615779742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c009b30 con 0x7f457406f790 2023-08-27T11:28:55.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.072+0000 7f4578d03700 1 --2- 172.21.15.112:0/615779742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457406f790 0x7f45741f5650 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f456800c7f0 tx=0x7f456800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:55.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.072+0000 7f45717fa700 1 -- 172.21.15.112:0/615779742 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568011800 con 0x7f457406f790 2023-08-27T11:28:55.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.072+0000 7f45717fa700 1 -- 172.21.15.112:0/615779742 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4568011960 con 0x7f457406f790 2023-08-27T11:28:55.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.072+0000 7f457af67700 1 -- 172.21.15.112:0/615779742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45740c7150 con 0x7f457406f790 2023-08-27T11:28:55.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.072+0000 7f45717fa700 1 -- 172.21.15.112:0/615779742 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568011cf0 con 0x7f457406f790 2023-08-27T11:28:55.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.072+0000 7f457af67700 1 -- 172.21.15.112:0/615779742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45740c6760 con 0x7f457406f790 2023-08-27T11:28:55.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.073+0000 7f457af67700 1 -- 172.21.15.112:0/615779742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4574067e90 con 0x7f457406f790 2023-08-27T11:28:55.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.073+0000 7f45717fa700 1 -- 172.21.15.112:0/615779742 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4568018360 con 0x7f457406f790 2023-08-27T11:28:55.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.074+0000 7f45717fa700 1 --2- 172.21.15.112:0/615779742 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f455806a4d0 0x7f455806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.074+0000 7f45717fa700 1 -- 172.21.15.112:0/615779742 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f4568088d10 con 0x7f457406f790 2023-08-27T11:28:55.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.074+0000 7f4573fff700 1 --2- 172.21.15.112:0/615779742 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f455806a4d0 0x7f455806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.074+0000 7f4573fff700 1 --2- 172.21.15.112:0/615779742 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f455806a4d0 0x7f455806c990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f45740b38f0 tx=0x7f4560001310 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:55.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.078+0000 7f45717fa700 1 -- 172.21.15.112:0/615779742 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f456808e020 con 0x7f457406f790 2023-08-27T11:28:55.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.344+0000 7f457af67700 1 -- 172.21.15.112:0/615779742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4574067a20 con 0x7f457406f790 2023-08-27T11:28:55.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.348+0000 7f45717fa700 1 -- 172.21.15.112:0/615779742 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7f456801f0b0 con 0x7f457406f790 2023-08-27T11:28:55.349 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-08-27T11:28:55.350 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA== 2023-08-27T11:28:55.350 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2023-08-27T11:28:55.350 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2023-08-27T11:28:55.350 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:28:55.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.350+0000 7f457af67700 1 -- 172.21.15.112:0/615779742 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f455806a4d0 msgr2=0x7f455806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.350+0000 7f457af67700 1 --2- 172.21.15.112:0/615779742 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f455806a4d0 0x7f455806c990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f45740b38f0 tx=0x7f4560001310 comp rx=0 tx=0).stop 2023-08-27T11:28:55.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.350+0000 7f457af67700 1 -- 172.21.15.112:0/615779742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457406f790 msgr2=0x7f45741f5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.350+0000 7f457af67700 1 --2- 172.21.15.112:0/615779742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457406f790 0x7f45741f5650 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f456800c7f0 tx=0x7f456800cbb0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.350+0000 7f457af67700 1 -- 172.21.15.112:0/615779742 shutdown_connections 2023-08-27T11:28:55.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.350+0000 7f457af67700 1 --2- 172.21.15.112:0/615779742 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f455806a4d0 0x7f455806c990 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.350+0000 7f457af67700 1 --2- 172.21.15.112:0/615779742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4574146c20 0x7f457415f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.350+0000 7f457af67700 1 --2- 172.21.15.112:0/615779742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457406f790 0x7f45741f5650 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.350+0000 7f457af67700 1 --2- 172.21.15.112:0/615779742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45740b3440 0x7f45740c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.350+0000 7f457af67700 1 -- 172.21.15.112:0/615779742 >> 172.21.15.112:0/615779742 conn(0x7f45740b1e90 msgr2=0x7f4574071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:55.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.350+0000 7f457af67700 1 -- 172.21.15.112:0/615779742 shutdown_connections 2023-08-27T11:28:55.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.350+0000 7f457af67700 1 -- 172.21.15.112:0/615779742 wait complete. 2023-08-27T11:28:55.354 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2023-08-27T11:28:55.363 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:55.386 INFO:tasks.ceph.mds.c.smithi112.stderr:2023-08-27T11:28:55.385+0000 7f9fcbf38700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:28:55.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.486+0000 7fe6ed92d700 1 Processor -- start 2023-08-27T11:28:55.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.486+0000 7fe6ed92d700 1 -- start start 2023-08-27T11:28:55.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6ed92d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6e806fea0 0x7fe6e80b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6ed92d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe6e806e3a0 0x7fe6e80af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6ed92d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e815c190 con 0x7fe6e8158e20 2023-08-27T11:28:55.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6ed92d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e80b1730 con 0x7fe6e806e3a0 2023-08-27T11:28:55.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6ed92d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e8159200 con 0x7fe6e806fea0 2023-08-27T11:28:55.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6e77fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe6e806e3a0 0x7fe6e80af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6e6ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe6e8158e20 0x7fe6e806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59790/0 (socket says 172.21.15.112:59790) 2023-08-27T11:28:55.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6e6ffd700 1 -- 172.21.15.112:0/1337177598 learned_addr learned my addr 172.21.15.112:0/1337177598 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:55.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6e67fc700 1 --2- 172.21.15.112:0/1337177598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6e806fea0 0x7fe6e80b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6e77fe700 1 -- 172.21.15.112:0/1337177598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6e806fea0 msgr2=0x7fe6e80b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:55.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6e77fe700 1 --2- 172.21.15.112:0/1337177598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6e806fea0 0x7fe6e80b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6e77fe700 1 -- 172.21.15.112:0/1337177598 >> v1:172.21.15.112:6789/0 conn(0x7fe6e8158e20 legacy=0x7fe6e806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6e5ffb700 1 -- 172.21.15.112:0/1337177598 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388836624 0 0) 0x7fe6e815c190 con 0x7fe6e8158e20 2023-08-27T11:28:55.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6e77fe700 1 -- 172.21.15.112:0/1337177598 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e80b1910 con 0x7fe6e806e3a0 2023-08-27T11:28:55.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6e67fc700 1 --2- 172.21.15.112:0/1337177598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6e806fea0 0x7fe6e80b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:55.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.487+0000 7fe6e77fe700 1 --2- 172.21.15.112:0/1337177598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e806e3a0 0x7fe6e80af9c0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009d90 tx=0x7fe6e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad7ddaa58d608a0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:55.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.488+0000 7fe6e5ffb700 1 -- 172.21.15.112:0/1337177598 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e000e070 con 0x7fe6e806e3a0 2023-08-27T11:28:55.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.488+0000 7fe6e5ffb700 1 -- 172.21.15.112:0/1337177598 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e000bbd0 con 0x7fe6e806e3a0 2023-08-27T11:28:55.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.488+0000 7fe6e5ffb700 1 -- 172.21.15.112:0/1337177598 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0012430 con 0x7fe6e806e3a0 2023-08-27T11:28:55.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.488+0000 7fe6ed92d700 1 -- 172.21.15.112:0/1337177598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e806e3a0 msgr2=0x7fe6e80af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.488+0000 7fe6ed92d700 1 --2- 172.21.15.112:0/1337177598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e806e3a0 0x7fe6e80af9c0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009d90 tx=0x7fe6e000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:55.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.488+0000 7fe6ed92d700 1 -- 172.21.15.112:0/1337177598 shutdown_connections 2023-08-27T11:28:55.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.488+0000 7fe6ed92d700 1 --2- 172.21.15.112:0/1337177598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e806e3a0 0x7fe6e80af9c0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.488+0000 7fe6ed92d700 1 --2- 172.21.15.112:0/1337177598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6e806fea0 0x7fe6e80b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.488+0000 7fe6ed92d700 1 -- 172.21.15.112:0/1337177598 >> 172.21.15.112:0/1337177598 conn(0x7fe6e80d0000 msgr2=0x7fe6e80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:55.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.488+0000 7fe6ed92d700 1 -- 172.21.15.112:0/1337177598 shutdown_connections 2023-08-27T11:28:55.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.488+0000 7fe6ed92d700 1 -- 172.21.15.112:0/1337177598 wait complete. 2023-08-27T11:28:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.489+0000 7fe6ed92d700 1 Processor -- start 2023-08-27T11:28:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.489+0000 7fe6ed92d700 1 -- start start 2023-08-27T11:28:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.490+0000 7fe6ed92d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e806e3a0 0x7fe6e8151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.490+0000 7fe6ed92d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e806fea0 0x7fe6e80cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.490+0000 7fe6ed92d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6e8158e20 0x7fe6e80c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.490+0000 7fe6ed92d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80cdbd0 con 0x7fe6e806e3a0 2023-08-27T11:28:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.490+0000 7fe6ed92d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80cde70 con 0x7fe6e806fea0 2023-08-27T11:28:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.490+0000 7fe6ed92d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80ce150 con 0x7fe6e8158e20 2023-08-27T11:28:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.490+0000 7fe6e6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e806e3a0 0x7fe6e8151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.490+0000 7fe6e6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e806e3a0 0x7fe6e8151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37182/0 (socket says 172.21.15.112:37182) 2023-08-27T11:28:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.490+0000 7fe6e6ffd700 1 -- 172.21.15.112:0/2556141429 learned_addr learned my addr 172.21.15.112:0/2556141429 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.490+0000 7fe6e6ffd700 1 -- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6e8158e20 msgr2=0x7fe6e80c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.490+0000 7fe6e77fe700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6e8158e20 0x7fe6e80c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.491+0000 7fe6e6ffd700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6e8158e20 0x7fe6e80c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.491+0000 7fe6e6ffd700 1 -- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e806fea0 msgr2=0x7fe6e80cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.491+0000 7fe6e67fc700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e806fea0 0x7fe6e80cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.491+0000 7fe6e6ffd700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e806fea0 0x7fe6e80cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.491+0000 7fe6e6ffd700 1 -- 172.21.15.112:0/2556141429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e0009a70 con 0x7fe6e806e3a0 2023-08-27T11:28:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.491+0000 7fe6e6ffd700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e806e3a0 0x7fe6e8151060 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0011e0 tx=0x7fe6dc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.491+0000 7fe6cffff700 1 -- 172.21.15.112:0/2556141429 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc004c80 con 0x7fe6e806e3a0 2023-08-27T11:28:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.491+0000 7fe6cffff700 1 -- 172.21.15.112:0/2556141429 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6dc004e00 con 0x7fe6e806e3a0 2023-08-27T11:28:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.491+0000 7fe6ed92d700 1 -- 172.21.15.112:0/2556141429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e80ce430 con 0x7fe6e806e3a0 2023-08-27T11:28:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.491+0000 7fe6cffff700 1 -- 172.21.15.112:0/2556141429 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc007310 con 0x7fe6e806e3a0 2023-08-27T11:28:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.491+0000 7fe6ed92d700 1 -- 172.21.15.112:0/2556141429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e80ce9f0 con 0x7fe6e806e3a0 2023-08-27T11:28:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.492+0000 7fe6ed92d700 1 -- 172.21.15.112:0/2556141429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e80cec10 con 0x7fe6e806e3a0 2023-08-27T11:28:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.492+0000 7fe6cffff700 1 -- 172.21.15.112:0/2556141429 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe6dc007470 con 0x7fe6e806e3a0 2023-08-27T11:28:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.493+0000 7fe6cffff700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe6d406a4d0 0x7fe6d406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.493+0000 7fe6cffff700 1 -- 172.21.15.112:0/2556141429 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fe6dc088a20 con 0x7fe6e806e3a0 2023-08-27T11:28:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.493+0000 7fe6e67fc700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe6d406a4d0 0x7fe6d406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.493+0000 7fe6e67fc700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe6d406a4d0 0x7fe6d406c990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80c6b50 tx=0x7fe6d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:55.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.497+0000 7fe6cffff700 1 -- 172.21.15.112:0/2556141429 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe6dc053bb0 con 0x7fe6e806e3a0 2023-08-27T11:28:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.731+0000 7fe6ed92d700 1 -- 172.21.15.112:0/2556141429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6e8067a20 con 0x7fe6e806e3a0 2023-08-27T11:28:55.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.732+0000 7fe6cffff700 1 -- 172.21.15.112:0/2556141429 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4891 (secure 0 0 0) 0x7fe6dc088030 con 0x7fe6e806e3a0 2023-08-27T11:28:55.733 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:28:55.734 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6980,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/877756033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":877756033},{"type":"v1","addr":"172.21.15.138:6837","nonce":877756033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6983,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2624496097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2624496097},{"type":"v1","addr":"172.21.15.138:6836","nonce":2624496097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":8940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2687911818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2687911818},{"type":"v1","addr":"172.21.15.112:6834","nonce":2687911818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:49.824795+0000","modified":"2023-08-27T11:28:53.433019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8943":{"gid":8943,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/577891555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":577891555},{"type":"v1","addr":"172.21.15.112:6835","nonce":577891555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-08-27T11:28:55.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.734+0000 7fe6ed92d700 1 -- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe6d406a4d0 msgr2=0x7fe6d406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.734+0000 7fe6ed92d700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe6d406a4d0 0x7fe6d406c990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80c6b50 tx=0x7fe6d8008040 comp rx=0 tx=0).stop 2023-08-27T11:28:55.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.734+0000 7fe6ed92d700 1 -- 172.21.15.112:0/2556141429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e806e3a0 msgr2=0x7fe6e8151060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.734+0000 7fe6ed92d700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e806e3a0 0x7fe6e8151060 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0011e0 tx=0x7fe6dc00cae0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.734+0000 7fe6ed92d700 1 -- 172.21.15.112:0/2556141429 shutdown_connections 2023-08-27T11:28:55.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.734+0000 7fe6ed92d700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe6d406a4d0 0x7fe6d406c990 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.734+0000 7fe6ed92d700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6e8158e20 0x7fe6e80c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.734+0000 7fe6ed92d700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e806e3a0 0x7fe6e8151060 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.734+0000 7fe6ed92d700 1 --2- 172.21.15.112:0/2556141429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e806fea0 0x7fe6e80cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.734+0000 7fe6ed92d700 1 -- 172.21.15.112:0/2556141429 >> 172.21.15.112:0/2556141429 conn(0x7fe6e80d0000 msgr2=0x7fe6e80709a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:55.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.734+0000 7fe6ed92d700 1 -- 172.21.15.112:0/2556141429 shutdown_connections 2023-08-27T11:28:55.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:55.734+0000 7fe6ed92d700 1 -- 172.21.15.112:0/2556141429 wait complete. 2023-08-27T11:28:55.738 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 104 2023-08-27T11:28:56.150 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:28:49.824795+0000', 'modified': '2023-08-27T11:28:53.433019+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8943}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8943': {'gid': 8943, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/577891555', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 577891555}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 577891555}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:28:56.151 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:28:56.151 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-27T11:28:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.271+0000 7f7a1073d700 1 Processor -- start 2023-08-27T11:28:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.271+0000 7f7a1073d700 1 -- start start 2023-08-27T11:28:56.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.272+0000 7f7a1073d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7a08144b80 0x7f7a080b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.272+0000 7f7a1073d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a080cef20 con 0x7f7a080b3de0 2023-08-27T11:28:56.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.272+0000 7f7a1073d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a080b42a0 con 0x7f7a0806c6e0 2023-08-27T11:28:56.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.272+0000 7f7a1073d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a080b4b40 con 0x7f7a08144b80 2023-08-27T11:28:56.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.272+0000 7f7a0dcd8700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7a080b3de0 0x7f7a0806bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59804/0 (socket says 172.21.15.112:59804) 2023-08-27T11:28:56.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.272+0000 7f7a0dcd8700 1 -- 172.21.15.112:0/3828241242 learned_addr learned my addr 172.21.15.112:0/3828241242 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:56.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717337410 0 0) 0x7f7a080cef20 con 0x7f7a080b3de0 2023-08-27T11:28:56.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/3828241242 >> v2:172.21.15.138:3300/0 conn(0x7f7a08144b80 0x7f7a080b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f40035b0 con 0x7f7a080b3de0 2023-08-27T11:28:56.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 337076097 0 0) 0x7f79f40035b0 con 0x7f7a080b3de0 2023-08-27T11:28:56.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 >> v2:172.21.15.138:3300/0 conn(0x7f7a08144b80 msgr2=0x7f7a080b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:56.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 --2- 172.21.15.112:0/3828241242 >> v2:172.21.15.138:3300/0 conn(0x7f7a08144b80 0x7f7a080b6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 >> v1:172.21.15.138:6789/0 conn(0x7f7a0806c6e0 legacy=0x7f7a080cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a080b4580 con 0x7f7a080b3de0 2023-08-27T11:28:56.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f7a04001670 con 0x7f7a080b3de0 2023-08-27T11:28:56.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 >> v1:172.21.15.112:6789/0 conn(0x7f7a080b3de0 legacy=0x7f7a0806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 --2- 172.21.15.112:0/3828241242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4003a00 0x7f79f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 --2- 172.21.15.112:0/3828241242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79f4004850 0x7f79f4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 --2- 172.21.15.112:0/3828241242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79f4005330 0x7f79f400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a080b4b40 con 0x7f79f4003a00 2023-08-27T11:28:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f400c200 con 0x7f79f4004850 2023-08-27T11:28:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f400c3c0 con 0x7f79f4005330 2023-08-27T11:28:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/3828241242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4003a00 0x7f79f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0dcd8700 1 --2- 172.21.15.112:0/3828241242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79f4004850 0x7f79f4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0ecda700 1 --2- 172.21.15.112:0/3828241242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79f4005330 0x7f79f400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/3828241242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79f4005330 msgr2=0x7f79f400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/3828241242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79f4005330 0x7f79f400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/3828241242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79f4004850 msgr2=0x7f79f4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/3828241242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79f4004850 0x7f79f4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/3828241242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f40035b0 con 0x7f79f4003a00 2023-08-27T11:28:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0ecda700 1 -- 172.21.15.112:0/3828241242 reap_dead start 2023-08-27T11:28:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.273+0000 7f7a0dcd8700 1 --2- 172.21.15.112:0/3828241242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79f4004850 0x7f79f4004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:56.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.274+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/3828241242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4003a00 0x7f79f4003e20 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009860 tx=0x7f79f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7d6248db8b8b04 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:56.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.274+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f800c070 con 0x7f79f4003a00 2023-08-27T11:28:56.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.274+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8013610 con 0x7f79f4003a00 2023-08-27T11:28:56.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.274+0000 7f7a0d4d7700 1 -- 172.21.15.112:0/3828241242 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f80139c0 con 0x7f79f4003a00 2023-08-27T11:28:56.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.274+0000 7f7a1073d700 1 -- 172.21.15.112:0/3828241242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4003a00 msgr2=0x7f79f4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.274+0000 7f7a1073d700 1 --2- 172.21.15.112:0/3828241242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4003a00 0x7f79f4003e20 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009860 tx=0x7f79f4005880 comp rx=0 tx=0).stop 2023-08-27T11:28:56.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.274+0000 7f7a1073d700 1 -- 172.21.15.112:0/3828241242 shutdown_connections 2023-08-27T11:28:56.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.274+0000 7f7a1073d700 1 --2- 172.21.15.112:0/3828241242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4003a00 0x7f79f4003e20 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.274+0000 7f7a1073d700 1 -- 172.21.15.112:0/3828241242 >> 172.21.15.112:0/3828241242 conn(0x7f7a080cdc50 msgr2=0x7f7a080ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:56.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.274+0000 7f7a1073d700 1 -- 172.21.15.112:0/3828241242 shutdown_connections 2023-08-27T11:28:56.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.274+0000 7f7a1073d700 1 -- 172.21.15.112:0/3828241242 wait complete. 2023-08-27T11:28:56.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.276+0000 7f7a1073d700 1 Processor -- start 2023-08-27T11:28:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.276+0000 7f7a1073d700 1 -- start start 2023-08-27T11:28:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a1073d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4003a00 0x7f7a080c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a1073d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a081f0770 0x7f7a081f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a1073d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a08159660 0x7f7a080ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a1073d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a080cca10 con 0x7f79f4003a00 2023-08-27T11:28:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a1073d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a081f52c0 con 0x7f7a08159660 2023-08-27T11:28:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a1073d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a081f0bf0 con 0x7f7a081f0770 2023-08-27T11:28:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a0e4d9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4003a00 0x7f7a080c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a0e4d9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4003a00 0x7f7a080c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37200/0 (socket says 172.21.15.112:37200) 2023-08-27T11:28:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/1744802361 learned_addr learned my addr 172.21.15.112:0/1744802361 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a0dcd8700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a081f0770 0x7f7a081f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a0ecda700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a08159660 0x7f7a080ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a081f0770 msgr2=0x7f7a081f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a081f0770 0x7f7a081f4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a08159660 msgr2=0x7f7a080ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a08159660 0x7f7a080ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/1744802361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a080cc0e0 con 0x7f79f4003a00 2023-08-27T11:28:56.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.277+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4003a00 0x7f7a080c73c0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f79f400c1d0 tx=0x7f79f80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:56.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.278+0000 7f79feffd700 1 -- 172.21.15.112:0/1744802361 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8004030 con 0x7f79f4003a00 2023-08-27T11:28:56.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.278+0000 7f79feffd700 1 -- 172.21.15.112:0/1744802361 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f80041b0 con 0x7f79f4003a00 2023-08-27T11:28:56.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.278+0000 7f7a1073d700 1 -- 172.21.15.112:0/1744802361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a080cc3c0 con 0x7f79f4003a00 2023-08-27T11:28:56.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.278+0000 7f79feffd700 1 -- 172.21.15.112:0/1744802361 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8004560 con 0x7f79f4003a00 2023-08-27T11:28:56.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.278+0000 7f7a1073d700 1 -- 172.21.15.112:0/1744802361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a080cbdd0 con 0x7f79f4003a00 2023-08-27T11:28:56.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.279+0000 7f79feffd700 1 -- 172.21.15.112:0/1744802361 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f79f80036a0 con 0x7f79f4003a00 2023-08-27T11:28:56.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.279+0000 7f79feffd700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f79ec06a4d0 0x7f79ec06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.280+0000 7f79feffd700 1 -- 172.21.15.112:0/1744802361 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f79f8089840 con 0x7f79f4003a00 2023-08-27T11:28:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.280+0000 7f7a0dcd8700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f79ec06a4d0 0x7f79ec06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.280+0000 7f7a0dcd8700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f79ec06a4d0 0x7f79ec06c990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7a04001c90 tx=0x7f7a04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.281+0000 7f7a1c8ffb80 1 Processor -- start 2023-08-27T11:28:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.282+0000 7f7a1c8ffb80 1 -- start start 2023-08-27T11:28:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.282+0000 7f7a1c8ffb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 0x5621d9f4c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.282+0000 7f7a1c8ffb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 0x5621d9f48060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.282+0000 7f7a1c8ffb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9f73520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.282+0000 7f7a1c8ffb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d9ea5730 con 0x5621d9f4c1b0 2023-08-27T11:28:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.282+0000 7f7a1c8ffb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d9ea3050 con 0x5621d9f4a3d0 2023-08-27T11:28:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.282+0000 7f7a1c8ffb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621d9e9e940 con 0x5621d9f931d0 2023-08-27T11:28:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.282+0000 7f7a0e4d9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 0x5621d9f4c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.282+0000 7f7a0e4d9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 0x5621d9f4c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37208/0 (socket says 172.21.15.112:37208) 2023-08-27T11:28:56.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.283+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/31546311 learned_addr learned my addr 172.21.15.112:0/31546311 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:56.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.283+0000 7f7a0ecda700 1 --2- 172.21.15.112:0/31546311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9f73520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.283+0000 7f7a0dcd8700 1 --2- 172.21.15.112:0/31546311 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 0x5621d9f48060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.283+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/31546311 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 msgr2=0x5621d9f48060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:56.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.283+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/31546311 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 0x5621d9f48060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.283+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/31546311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 msgr2=0x5621d9f73520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:56.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.283+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/31546311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9f73520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.283+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/31546311 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621d9d1e5e0 con 0x5621d9f4c1b0 2023-08-27T11:28:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.283+0000 7f7a0ecda700 1 --2- 172.21.15.112:0/31546311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9f73520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.283+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/31546311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 0x5621d9f4c590 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f79f8054a30 tx=0x7f79f805c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c714db2f3833c548 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.283+0000 7f79ebfff700 1 -- 172.21.15.112:0/31546311 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f805c120 con 0x5621d9f4c1b0 2023-08-27T11:28:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.283+0000 7f79ebfff700 1 -- 172.21.15.112:0/31546311 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8056730 con 0x5621d9f4c1b0 2023-08-27T11:28:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.283+0000 7f79ebfff700 1 -- 172.21.15.112:0/31546311 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f802d560 con 0x5621d9f4c1b0 2023-08-27T11:28:56.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.284+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/31546311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 msgr2=0x5621d9f4c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.284+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/31546311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 0x5621d9f4c590 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f79f8054a30 tx=0x7f79f805c2f0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.284+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/31546311 shutdown_connections 2023-08-27T11:28:56.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.284+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/31546311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9f73520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.284+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/31546311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 0x5621d9f4c590 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.284+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/31546311 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 0x5621d9f48060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.284+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/31546311 >> 172.21.15.112:0/31546311 conn(0x5621d9eaac30 msgr2=0x5621d9f91d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:56.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.284+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/31546311 shutdown_connections 2023-08-27T11:28:56.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.284+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/31546311 wait complete. 2023-08-27T11:28:56.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.284+0000 7f7a1c8ffb80 1 -- start start 2023-08-27T11:28:56.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.284+0000 7f7a1c8ffb80 1 Processor -- start 2023-08-27T11:28:56.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.285+0000 7f7a1c8ffb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9f72200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.285+0000 7f7a1c8ffb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 0x5621d9f79ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.285+0000 7f7a1c8ffb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 0x5621d9f77ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.285+0000 7f7a1c8ffb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d9e3cb60 con 0x5621d9f4c1b0 2023-08-27T11:28:56.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.285+0000 7f7a1c8ffb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d9e3c860 con 0x5621d9f4a3d0 2023-08-27T11:28:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.285+0000 7f7a1c8ffb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621d9d208d0 con 0x5621d9f931d0 2023-08-27T11:28:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.285+0000 7f7a1c8ffb80 10 client.0 _ll_register_callbacks cb 0x5621d9e3a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:28:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.285+0000 7f7a0dcd8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 0x5621d9f79ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.285+0000 7f7a0dcd8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 0x5621d9f79ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37216/0 (socket says 172.21.15.112:37216) 2023-08-27T11:28:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.285+0000 7f7a0dcd8700 1 -- 172.21.15.112:0/77397552 learned_addr learned my addr 172.21.15.112:0/77397552 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.285+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9f72200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a0ecda700 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 0x5621d9f77ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a0dcd8700 1 -- 172.21.15.112:0/77397552 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 msgr2=0x5621d9f77ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:56.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a0dcd8700 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 0x5621d9f77ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a0dcd8700 1 -- 172.21.15.112:0/77397552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 msgr2=0x5621d9f72200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:56.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a0dcd8700 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9f72200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a0dcd8700 1 -- 172.21.15.112:0/77397552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a080cbdd0 con 0x5621d9f4c1b0 2023-08-27T11:28:56.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a0ecda700 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 0x5621d9f77ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:56.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a0dcd8700 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 0x5621d9f79ca0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f7a080643a0 tx=0x7f7a04001c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d691d16da454febc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:56.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f79e97fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:28:56.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f79e97fa700 1 -- 172.21.15.112:0/77397552 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04010620 con 0x5621d9f4c1b0 2023-08-27T11:28:56.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f79e97fa700 1 -- 172.21.15.112:0/77397552 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f7a0401c070 con 0x5621d9f4c1b0 2023-08-27T11:28:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/77397552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621d9e97920 con 0x5621d9f4c1b0 2023-08-27T11:28:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f79e97fa700 1 -- 172.21.15.112:0/77397552 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a0402d3b0 con 0x5621d9f4c1b0 2023-08-27T11:28:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a1c8ffb80 10 client.9102 fetch_fsmap learned FSMap version 104 2023-08-27T11:28:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/77397552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x5621d9eb6750 con 0x5621d9f4c1b0 2023-08-27T11:28:56.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f79e97fa700 1 -- 172.21.15.112:0/77397552 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5621d9e97920 con 0x5621d9f4c1b0 2023-08-27T11:28:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f79e97fa700 1 -- 172.21.15.112:0/77397552 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 104) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7a0402d510 con 0x5621d9f4c1b0 2023-08-27T11:28:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a1c8ffb80 10 client.9102 fetch_fsmap finished waiting for FSMap version 104 2023-08-27T11:28:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a1c8ffb80 10 client.9102 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:28:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a1c8ffb80 20 client.9102 populate_metadata read hostname 'smithi112' 2023-08-27T11:28:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.112:6833/577891555,v1:172.21.15.112:6835/577891555] conn(0x5621d9f990a0 0x5621d9f8a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a1c8ffb80 4 client.9102 mds_command: new command op to 8943 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:28:56.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.286+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/77397552 --> [v2:172.21.15.112:6833/577891555,v1:172.21.15.112:6835/577891555] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621d9e58000 con 0x5621d9f990a0 2023-08-27T11:28:56.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.287+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.112:6833/577891555,v1:172.21.15.112:6835/577891555] conn(0x5621d9f990a0 0x5621d9f8a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.287+0000 7f79e97fa700 10 client.9102 ms_handle_connect on v2:172.21.15.112:6833/577891555 2023-08-27T11:28:56.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.287+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.112:6833/577891555,v1:172.21.15.112:6835/577891555] conn(0x5621d9f990a0 0x5621d9f8a0a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57108fb7a4561318 server_cookie=30cdc2b7661ee4e7 in_seq=0 out_seq=0 2023-08-27T11:28:56.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.287+0000 7f79e97fa700 1 -- 172.21.15.112:0/77397552 <== mds.0 v2:172.21.15.112:6833/577891555 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5621d9e58000 con 0x5621d9f990a0 2023-08-27T11:28:56.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.287+0000 7f79e97fa700 10 client.9102 handle_command_reply: tid=0 2023-08-27T11:28:56.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.287+0000 7f7a1c8ffb80 1 client.9102 shutdown 2023-08-27T11:28:56.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.287+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/77397552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 msgr2=0x5621d9f79ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.288+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 0x5621d9f79ca0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f7a080643a0 tx=0x7f7a04001c00 comp rx=0 tx=0).stop 2023-08-27T11:28:56.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.288+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/77397552 shutdown_connections 2023-08-27T11:28:56.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.288+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.112:6833/577891555,v1:172.21.15.112:6835/577891555] conn(0x5621d9f990a0 0x5621d9f8a0a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.288+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 0x5621d9f77ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.288+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9f72200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.288+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/77397552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4c1b0 0x5621d9f79ca0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.288+0000 7f79e97fa700 0 client.9102 ms_handle_reset on v2:172.21.15.112:6833/577891555 2023-08-27T11:28:56.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.288+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/77397552 >> 172.21.15.112:0/77397552 conn(0x5621d9eaac30 msgr2=0x5621d9f4ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:56.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.288+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/77397552 shutdown_connections 2023-08-27T11:28:56.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.288+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/77397552 wait complete. 2023-08-27T11:28:56.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.288+0000 7f7a1c8ffb80 20 client.9102 trim_cache size 0 max 16384 2023-08-27T11:28:56.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.305+0000 7f7a1c8ffb80 1 Processor -- start 2023-08-27T11:28:56.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.305+0000 7f7a1c8ffb80 1 -- start start 2023-08-27T11:28:56.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a1c8ffb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4a3d0 0x5621d9f92060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a1c8ffb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4c1b0 0x5621d9f72010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a1c8ffb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 0x5621d9f72550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a1c8ffb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d9f79630 con 0x5621d9f4a3d0 2023-08-27T11:28:56.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a1c8ffb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d9f91dc0 con 0x5621d9f4c1b0 2023-08-27T11:28:56.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a1c8ffb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621d9f781d0 con 0x5621d9f931d0 2023-08-27T11:28:56.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a0e4d9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4a3d0 0x5621d9f92060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a0e4d9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4a3d0 0x5621d9f92060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37222/0 (socket says 172.21.15.112:37222) 2023-08-27T11:28:56.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/1417368458 learned_addr learned my addr 172.21.15.112:0/1417368458 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:56.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/1417368458 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 msgr2=0x5621d9f72550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:56.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/1417368458 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 0x5621d9f72550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/1417368458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4c1b0 msgr2=0x5621d9f72010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:56.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/1417368458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4c1b0 0x5621d9f72010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.306+0000 7f7a0e4d9700 1 -- 172.21.15.112:0/1417368458 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621d9ccad90 con 0x5621d9f4a3d0 2023-08-27T11:28:56.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.307+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/1417368458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4a3d0 0x5621d9f92060 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f79f8054a30 tx=0x7f79f8089000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c0f65285d625a3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:56.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.307+0000 7f79fcff9700 1 -- 172.21.15.112:0/1417368458 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8056730 con 0x5621d9f4a3d0 2023-08-27T11:28:56.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.307+0000 7f79fcff9700 1 -- 172.21.15.112:0/1417368458 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f80568d0 con 0x5621d9f4a3d0 2023-08-27T11:28:56.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.307+0000 7f79fcff9700 1 -- 172.21.15.112:0/1417368458 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8057c20 con 0x5621d9f4a3d0 2023-08-27T11:28:56.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.307+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/1417368458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4a3d0 msgr2=0x5621d9f92060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.307+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/1417368458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4a3d0 0x5621d9f92060 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f79f8054a30 tx=0x7f79f8089000 comp rx=0 tx=0).stop 2023-08-27T11:28:56.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.307+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/1417368458 shutdown_connections 2023-08-27T11:28:56.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.307+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/1417368458 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f931d0 0x5621d9f72550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.307+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/1417368458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f4a3d0 0x5621d9f92060 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.307+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/1417368458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4c1b0 0x5621d9f72010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.307+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/1417368458 >> 172.21.15.112:0/1417368458 conn(0x5621d9eaac30 msgr2=0x5621d9fb6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:56.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.307+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/1417368458 shutdown_connections 2023-08-27T11:28:56.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.307+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/1417368458 wait complete. 2023-08-27T11:28:56.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.308+0000 7f7a1c8ffb80 1 -- start start 2023-08-27T11:28:56.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.308+0000 7f7a1c8ffb80 1 Processor -- start 2023-08-27T11:28:56.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a1c8ffb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9ffb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a1c8ffb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f4c1b0 0x5621d9faf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a1c8ffb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f931d0 0x5621d9f955c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a1c8ffb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d9f45dd0 con 0x5621d9f931d0 2023-08-27T11:28:56.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a1c8ffb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d9f73c50 con 0x5621d9f4a3d0 2023-08-27T11:28:56.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a1c8ffb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621d9f45b60 con 0x5621d9f4c1b0 2023-08-27T11:28:56.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a1c8ffb80 10 client.0 _ll_register_callbacks cb 0x5621d9e3a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:28:56.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a0ecda700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f931d0 0x5621d9f955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a0ecda700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f931d0 0x5621d9f955c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37232/0 (socket says 172.21.15.112:37232) 2023-08-27T11:28:56.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a0ecda700 1 -- 172.21.15.112:0/1761181845 learned_addr learned my addr 172.21.15.112:0/1761181845 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:56.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9ffb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a0dcd8700 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f4c1b0 0x5621d9faf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a0ecda700 1 -- 172.21.15.112:0/1761181845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f4c1b0 msgr2=0x5621d9faf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:56.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a0ecda700 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f4c1b0 0x5621d9faf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a0ecda700 1 -- 172.21.15.112:0/1761181845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 msgr2=0x5621d9ffb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:56.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a0ecda700 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9ffb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a0ecda700 1 -- 172.21.15.112:0/1761181845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621d9d1e5e0 con 0x5621d9f931d0 2023-08-27T11:28:56.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9ffb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:56.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a0dcd8700 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f4c1b0 0x5621d9faf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:56.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f7a0ecda700 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f931d0 0x5621d9f955c0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f7a0806ef90 tx=0x7f7a000148a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c94266912b6712 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:56.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f79e97fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:28:56.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.309+0000 7f79e97fa700 1 -- 172.21.15.112:0/1761181845 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00003ae0 con 0x5621d9f931d0 2023-08-27T11:28:56.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f79e97fa700 1 -- 172.21.15.112:0/1761181845 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f7a00003c40 con 0x5621d9f931d0 2023-08-27T11:28:56.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/1761181845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621d9f49cb0 con 0x5621d9f931d0 2023-08-27T11:28:56.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f79e97fa700 1 -- 172.21.15.112:0/1761181845 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a00003e00 con 0x5621d9f931d0 2023-08-27T11:28:56.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f7a1c8ffb80 10 client.9108 fetch_fsmap learned FSMap version 104 2023-08-27T11:28:56.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/1761181845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x5621d9cb3c10 con 0x5621d9f931d0 2023-08-27T11:28:56.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f79e97fa700 1 -- 172.21.15.112:0/1761181845 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5621d9f49cb0 con 0x5621d9f931d0 2023-08-27T11:28:56.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f79e97fa700 1 -- 172.21.15.112:0/1761181845 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 104) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7a00005110 con 0x5621d9f931d0 2023-08-27T11:28:56.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f7a1c8ffb80 10 client.9108 fetch_fsmap finished waiting for FSMap version 104 2023-08-27T11:28:56.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f7a1c8ffb80 10 client.9108 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:28:56.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f7a1c8ffb80 20 client.9108 populate_metadata read hostname 'smithi112' 2023-08-27T11:28:56.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.112:6833/577891555,v1:172.21.15.112:6835/577891555] conn(0x5621d9fff7c0 0x5621d9f70b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f7a1c8ffb80 4 client.9108 mds_command: new command op to 8943 tid=0[{"prefix": "status"}] 2023-08-27T11:28:56.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/1761181845 --> [v2:172.21.15.112:6833/577891555,v1:172.21.15.112:6835/577891555] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5621d9f3f4f0 con 0x5621d9fff7c0 2023-08-27T11:28:56.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.112:6833/577891555,v1:172.21.15.112:6835/577891555] conn(0x5621d9fff7c0 0x5621d9f70b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f79e97fa700 10 client.9108 ms_handle_connect on v2:172.21.15.112:6833/577891555 2023-08-27T11:28:56.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f7a0e4d9700 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.112:6833/577891555,v1:172.21.15.112:6835/577891555] conn(0x5621d9fff7c0 0x5621d9f70b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c8b7c1820d2c784 server_cookie=a401de7ddd673e32 in_seq=0 out_seq=0 2023-08-27T11:28:56.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f79e97fa700 1 -- 172.21.15.112:0/1761181845 <== mds.0 v2:172.21.15.112:6833/577891555 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x5621d9f3f4f0 con 0x5621d9fff7c0 2023-08-27T11:28:56.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.310+0000 7f79e97fa700 10 client.9108 handle_command_reply: tid=0 2023-08-27T11:28:56.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1c8ffb80 1 client.9108 shutdown 2023-08-27T11:28:56.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/1761181845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f931d0 msgr2=0x5621d9f955c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f931d0 0x5621d9f955c0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f7a0806ef90 tx=0x7f7a000148a0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/1761181845 shutdown_connections 2023-08-27T11:28:56.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.112:6833/577891555,v1:172.21.15.112:6835/577891555] conn(0x5621d9fff7c0 0x5621d9f70b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621d9f931d0 0x5621d9f955c0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d9f4a3d0 0x5621d9ffb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f79e97fa700 0 client.9108 ms_handle_reset on v2:172.21.15.112:6833/577891555 2023-08-27T11:28:56.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1c8ffb80 1 --2- 172.21.15.112:0/1761181845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621d9f4c1b0 0x5621d9faf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/1761181845 >> 172.21.15.112:0/1761181845 conn(0x5621d9eaac30 msgr2=0x5621d9f91fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:56.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/1761181845 shutdown_connections 2023-08-27T11:28:56.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1c8ffb80 1 -- 172.21.15.112:0/1761181845 wait complete. 2023-08-27T11:28:56.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1c8ffb80 20 client.9108 trim_cache size 0 max 16384 2023-08-27T11:28:56.336 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:28:56.336 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e099a24f-c1cd-4f11-ac4f-ecacc10e5e98", 2023-08-27T11:28:56.336 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2023-08-27T11:28:56.336 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 8943, 2023-08-27T11:28:56.336 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2023-08-27T11:28:56.336 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2023-08-27T11:28:56.337 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2023-08-27T11:28:56.337 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 5.9058062290000004, 2023-08-27T11:28:56.337 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 104, 2023-08-27T11:28:56.337 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 155, 2023-08-27T11:28:56.337 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 153, 2023-08-27T11:28:56.338 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 11.859054879 2023-08-27T11:28:56.338 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:28:56.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1073d700 1 -- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f79ec06a4d0 msgr2=0x7f79ec06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1073d700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f79ec06a4d0 0x7f79ec06c990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7a04001c90 tx=0x7f7a04009040 comp rx=0 tx=0).stop 2023-08-27T11:28:56.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1073d700 1 -- 172.21.15.112:0/1744802361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4003a00 msgr2=0x7f7a080c73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1073d700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4003a00 0x7f7a080c73c0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f79f400c1d0 tx=0x7f79f80047c0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1073d700 1 -- 172.21.15.112:0/1744802361 shutdown_connections 2023-08-27T11:28:56.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1073d700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f79ec06a4d0 0x7f79ec06c990 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1073d700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a08159660 0x7f7a080ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1073d700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4003a00 0x7f7a080c73c0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1073d700 1 --2- 172.21.15.112:0/1744802361 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a081f0770 0x7f7a081f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1073d700 1 -- 172.21.15.112:0/1744802361 >> 172.21.15.112:0/1744802361 conn(0x7f7a080cdc50 msgr2=0x7f7a080c6170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:56.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1073d700 1 -- 172.21.15.112:0/1744802361 shutdown_connections 2023-08-27T11:28:56.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.311+0000 7f7a1073d700 1 -- 172.21.15.112:0/1744802361 wait complete. 2023-08-27T11:28:56.342 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:28:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.461+0000 7f73d8dc6700 1 Processor -- start 2023-08-27T11:28:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.461+0000 7f73d8dc6700 1 -- start start 2023-08-27T11:28:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d8dc6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f73d40af6a0 0x7f73d40b5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d8dc6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f73d4144cf0 0x7f73d40b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d8dc6700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d40cf5d0 con 0x7f73d4144910 2023-08-27T11:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d8dc6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d40ce740 con 0x7f73d4144cf0 2023-08-27T11:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d8dc6700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d4071550 con 0x7f73d40af6a0 2023-08-27T11:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d2d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f73d4144cf0 0x7f73d40b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d259c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f73d4144910 0x7f73d40721f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59850/0 (socket says 172.21.15.112:59850) 2023-08-27T11:28:56.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d259c700 1 -- 172.21.15.112:0/134973953 learned_addr learned my addr 172.21.15.112:0/134973953 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:56.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d1d9b700 1 --2- 172.21.15.112:0/134973953 >> v2:172.21.15.138:3300/0 conn(0x7f73d40af6a0 0x7f73d40b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d2d9d700 1 -- 172.21.15.112:0/134973953 >> v2:172.21.15.138:3300/0 conn(0x7f73d40af6a0 msgr2=0x7f73d40b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:56.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d2d9d700 1 --2- 172.21.15.112:0/134973953 >> v2:172.21.15.138:3300/0 conn(0x7f73d40af6a0 0x7f73d40b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d2d9d700 1 -- 172.21.15.112:0/134973953 >> v1:172.21.15.112:6789/0 conn(0x7f73d4144910 legacy=0x7f73d40721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d2d9d700 1 -- 172.21.15.112:0/134973953 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d40ce9c0 con 0x7f73d4144cf0 2023-08-27T11:28:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d2d9d700 1 --2- 172.21.15.112:0/134973953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4144cf0 0x7f73d40b5b90 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f73d415ece0 tx=0x7f73cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c21a68e39f623bc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d159a700 1 -- 172.21.15.112:0/134973953 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00e070 con 0x7f73d4144cf0 2023-08-27T11:28:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d159a700 1 -- 172.21.15.112:0/134973953 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc00bbd0 con 0x7f73d4144cf0 2023-08-27T11:28:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.462+0000 7f73d159a700 1 -- 172.21.15.112:0/134973953 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc012430 con 0x7f73d4144cf0 2023-08-27T11:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.463+0000 7f73d8dc6700 1 -- 172.21.15.112:0/134973953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4144cf0 msgr2=0x7f73d40b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.463+0000 7f73d8dc6700 1 --2- 172.21.15.112:0/134973953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4144cf0 0x7f73d40b5b90 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f73d415ece0 tx=0x7f73cc00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.463+0000 7f73d8dc6700 1 -- 172.21.15.112:0/134973953 shutdown_connections 2023-08-27T11:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.463+0000 7f73d8dc6700 1 --2- 172.21.15.112:0/134973953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4144cf0 0x7f73d40b5b90 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.463+0000 7f73d8dc6700 1 --2- 172.21.15.112:0/134973953 >> v2:172.21.15.138:3300/0 conn(0x7f73d40af6a0 0x7f73d40b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.463+0000 7f73d8dc6700 1 -- 172.21.15.112:0/134973953 >> 172.21.15.112:0/134973953 conn(0x7f73d40b1c20 msgr2=0x7f73d40b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.463+0000 7f73d8dc6700 1 -- 172.21.15.112:0/134973953 shutdown_connections 2023-08-27T11:28:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.463+0000 7f73d8dc6700 1 -- 172.21.15.112:0/134973953 wait complete. 2023-08-27T11:28:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.465+0000 7f73d8dc6700 1 Processor -- start 2023-08-27T11:28:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.465+0000 7f73d8dc6700 1 -- start start 2023-08-27T11:28:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d8dc6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4144910 0x7f73d41f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d8dc6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73d4144cf0 0x7f73d4071c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d8dc6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73d415e430 0x7f73d40721b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d8dc6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d41566d0 con 0x7f73d4144910 2023-08-27T11:28:56.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d8dc6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d413a670 con 0x7f73d4144cf0 2023-08-27T11:28:56.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d8dc6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d41f1ac0 con 0x7f73d415e430 2023-08-27T11:28:56.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4144910 0x7f73d41f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.473 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4144910 0x7f73d41f1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37242/0 (socket says 172.21.15.112:37242) 2023-08-27T11:28:56.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d259c700 1 -- 172.21.15.112:0/1955117309 learned_addr learned my addr 172.21.15.112:0/1955117309 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:56.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d1d9b700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73d4144cf0 0x7f73d4071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d2d9d700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73d415e430 0x7f73d40721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d259c700 1 -- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73d415e430 msgr2=0x7f73d40721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.474 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d259c700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73d415e430 0x7f73d40721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d259c700 1 -- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73d4144cf0 msgr2=0x7f73d4071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d259c700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73d4144cf0 0x7f73d4071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d259c700 1 -- 172.21.15.112:0/1955117309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc009a70 con 0x7f73d4144910 2023-08-27T11:28:56.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d259c700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4144910 0x7f73d41f1d90 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f73c400c520 tx=0x7f73c400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:56.475 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73c2ffd700 1 -- 172.21.15.112:0/1955117309 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4005d70 con 0x7f73d4144910 2023-08-27T11:28:56.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73c2ffd700 1 -- 172.21.15.112:0/1955117309 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c40070d0 con 0x7f73d4144910 2023-08-27T11:28:56.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d8dc6700 1 -- 172.21.15.112:0/1955117309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d41f1500 con 0x7f73d4144910 2023-08-27T11:28:56.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73c2ffd700 1 -- 172.21.15.112:0/1955117309 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4007440 con 0x7f73d4144910 2023-08-27T11:28:56.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.466+0000 7f73d8dc6700 1 -- 172.21.15.112:0/1955117309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d413a880 con 0x7f73d4144910 2023-08-27T11:28:56.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.467+0000 7f73d8dc6700 1 -- 172.21.15.112:0/1955117309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d4061f50 con 0x7f73d4144910 2023-08-27T11:28:56.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.468+0000 7f73c2ffd700 1 -- 172.21.15.112:0/1955117309 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f73c40075a0 con 0x7f73d4144910 2023-08-27T11:28:56.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.468+0000 7f73c2ffd700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73b406a4d0 0x7f73b406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:56.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.468+0000 7f73c2ffd700 1 -- 172.21.15.112:0/1955117309 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f73c4087fd0 con 0x7f73d4144910 2023-08-27T11:28:56.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.468+0000 7f73d1d9b700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73b406a4d0 0x7f73b406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:56.477 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.469+0000 7f73d1d9b700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73b406a4d0 0x7f73b406c990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f73d413c240 tx=0x7f73c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:56.478 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.473+0000 7f73c2ffd700 1 -- 172.21.15.112:0/1955117309 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f73c4053130 con 0x7f73d4144910 2023-08-27T11:28:56.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.703+0000 7f73d8dc6700 1 -- 172.21.15.112:0/1955117309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f73d406d800 con 0x7f73d4144910 2023-08-27T11:28:56.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.704+0000 7f73c2ffd700 1 -- 172.21.15.112:0/1955117309 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v104) v1 ==== 53+0+73 (secure 0 0 0) 0x7f73c4017060 con 0x7f73d4144910 2023-08-27T11:28:56.706 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:28:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.706+0000 7f73d8dc6700 1 -- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73b406a4d0 msgr2=0x7f73b406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.706+0000 7f73d8dc6700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73b406a4d0 0x7f73b406c990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f73d413c240 tx=0x7f73c8008040 comp rx=0 tx=0).stop 2023-08-27T11:28:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.706+0000 7f73d8dc6700 1 -- 172.21.15.112:0/1955117309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4144910 msgr2=0x7f73d41f1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.706+0000 7f73d8dc6700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4144910 0x7f73d41f1d90 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f73c400c520 tx=0x7f73c400c830 comp rx=0 tx=0).stop 2023-08-27T11:28:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.706+0000 7f73d8dc6700 1 -- 172.21.15.112:0/1955117309 shutdown_connections 2023-08-27T11:28:56.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.706+0000 7f73d8dc6700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f73b406a4d0 0x7f73b406c990 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.706+0000 7f73d8dc6700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73d415e430 0x7f73d40721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.706+0000 7f73d8dc6700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4144910 0x7f73d41f1d90 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.706+0000 7f73d8dc6700 1 --2- 172.21.15.112:0/1955117309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73d4144cf0 0x7f73d4071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:56.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.706+0000 7f73d8dc6700 1 -- 172.21.15.112:0/1955117309 >> 172.21.15.112:0/1955117309 conn(0x7f73d40b1c20 msgr2=0x7f73d406dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:56.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.706+0000 7f73d8dc6700 1 -- 172.21.15.112:0/1955117309 shutdown_connections 2023-08-27T11:28:56.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:56.706+0000 7f73d8dc6700 1 -- 172.21.15.112:0/1955117309 wait complete. 2023-08-27T11:28:56.719 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:28:56.720 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:28:56.720 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-08-27T11:28:56.720 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:28:56.720 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:56.721 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:28:56.721 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:28:56.721 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:28:56.722 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:28:56.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:56.723 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-08-27T11:28:56.740 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:28:56.740 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:28:56.740 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:28:56.741 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:56.741 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:28:56.742 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:56.742 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:56.742 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:56.743 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:56.743 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:56.743 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:56.743 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:56.744 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:28:56.744 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:56.744 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:28:56.744 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 41706sec preferred_lft 41706sec 2023-08-27T11:28:56.745 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8d6c:4fe6:6a84:8222/64 scope link noprefixroute 2023-08-27T11:28:56.745 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:56.745 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:28:56.746 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:56.746 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:28:56.746 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:56.746 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::d87c:a1ff:fe1d:b635/64 scope link 2023-08-27T11:28:56.747 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:56.747 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:28:56.747 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:28:56.747 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::cc14:79ff:feb3:452f/64 scope link 2023-08-27T11:28:56.748 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:56.748 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:28:56.748 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether e6:51:f0:78:04:02 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:28:56.748 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::e451:f0ff:fe78:402/64 scope link 2023-08-27T11:28:56.748 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:56.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:56.749 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:28:56.805 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-08-27T11:28:56.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:56.807 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-08-27T11:28:56.865 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-08-27T11:28:56.866 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:28:56.866 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:28:56.866 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:28:56.866 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:28:56.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:56.867 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:28:56.867 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:28:56.868 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:28:56.868 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:28:56.869 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:28:56.869 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:28:56.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:56.970 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:28:56.986 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:28:56.986 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:28:56.987 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:28:56.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:56.988 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-08-27T11:28:57.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:28:57.053 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:57.110 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:28:57.110 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:28:57.111 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:57.166 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:28:57.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:57.167 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:28:57.252 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:28:57.252 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:28:57.252 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:28:57.260 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:28:57.261 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:28:57.261 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:28:57.261 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:28:57.261 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:28:57.335 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:28:57.335 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:28:57.336 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:28:57.336 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:28:57.336 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:28:57.336 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:28:57.336 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:28:57.337 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:28:57.338 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:28:57.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:57.338 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:57.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:57.365 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:28:57.447 INFO:teuthology.orchestra.run.smithi112.stdout:client7996 2023-08-27T11:28:57.448 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:28:57.448 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:57.504 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:28:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.663+0000 7f50c9bdb700 1 Processor -- start 2023-08-27T11:28:57.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.663+0000 7f50c9bdb700 1 -- start start 2023-08-27T11:28:57.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c9bdb700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f50c40b1400 0x7f50c40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:57.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c9bdb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50c40b17e0 0x7f50c406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:57.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c9bdb700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c4161850 con 0x7f50c4161ce0 2023-08-27T11:28:57.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c9bdb700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c406e7d0 con 0x7f50c40b1400 2023-08-27T11:28:57.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c9bdb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40cfc10 con 0x7f50c40b17e0 2023-08-27T11:28:57.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c2ffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50c40b17e0 0x7f50c406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:57.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c37fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f50c40b1400 0x7f50c40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:57.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c2ffd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50c40b17e0 0x7f50c406e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:48910/0 (socket says 172.21.15.112:48910) 2023-08-27T11:28:57.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c2ffd700 1 -- 172.21.15.112:0/3579373937 learned_addr learned my addr 172.21.15.112:0/3579373937 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:57.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c3fff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f50c4161ce0 0x7f50c406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:39164/0 (socket says 172.21.15.112:39164) 2023-08-27T11:28:57.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c37fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f50c40b1400 0x7f50c40cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:37680/0 (socket says 172.21.15.112:37680) 2023-08-27T11:28:57.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c27fc700 1 -- 172.21.15.112:0/3579373937 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480314010 0 0) 0x7f50c4161850 con 0x7f50c4161ce0 2023-08-27T11:28:57.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c27fc700 1 -- 172.21.15.112:0/3579373937 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50ac0035b0 con 0x7f50c4161ce0 2023-08-27T11:28:57.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c2ffd700 1 -- 172.21.15.112:0/3579373937 >> v2:172.21.15.138:3300/0 conn(0x7f50c40b1400 msgr2=0x7f50c40cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:57.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c2ffd700 1 --2- 172.21.15.112:0/3579373937 >> v2:172.21.15.138:3300/0 conn(0x7f50c40b1400 0x7f50c40cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:57.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c2ffd700 1 -- 172.21.15.112:0/3579373937 >> v1:172.21.15.138:6789/0 conn(0x7f50c4161ce0 legacy=0x7f50c406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c2ffd700 1 -- 172.21.15.112:0/3579373937 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c4161200 con 0x7f50c40b17e0 2023-08-27T11:28:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.664+0000 7f50c37fe700 1 --2- 172.21.15.112:0/3579373937 >> v2:172.21.15.138:3300/0 conn(0x7f50c40b1400 0x7f50c40cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.665+0000 7f50c2ffd700 1 --2- 172.21.15.112:0/3579373937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50c40b17e0 0x7f50c406e290 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f50b80098f0 tx=0x7f50b8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=738d9cad5e1dd16d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.665+0000 7f50c27fc700 1 -- 172.21.15.112:0/3579373937 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8011830 con 0x7f50c40b17e0 2023-08-27T11:28:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.665+0000 7f50c27fc700 1 -- 172.21.15.112:0/3579373937 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b8011990 con 0x7f50c40b17e0 2023-08-27T11:28:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.665+0000 7f50c27fc700 1 -- 172.21.15.112:0/3579373937 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8011d00 con 0x7f50c40b17e0 2023-08-27T11:28:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.665+0000 7f50c9bdb700 1 -- 172.21.15.112:0/3579373937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50c40b17e0 msgr2=0x7f50c406e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.665+0000 7f50c9bdb700 1 --2- 172.21.15.112:0/3579373937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50c40b17e0 0x7f50c406e290 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f50b80098f0 tx=0x7f50b8009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.665+0000 7f50c9bdb700 1 -- 172.21.15.112:0/3579373937 shutdown_connections 2023-08-27T11:28:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.665+0000 7f50c9bdb700 1 --2- 172.21.15.112:0/3579373937 >> v2:172.21.15.138:3300/0 conn(0x7f50c40b1400 0x7f50c40cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.665+0000 7f50c9bdb700 1 --2- 172.21.15.112:0/3579373937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50c40b17e0 0x7f50c406e290 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.665+0000 7f50c9bdb700 1 -- 172.21.15.112:0/3579373937 >> 172.21.15.112:0/3579373937 conn(0x7f50c40d0000 msgr2=0x7f50c40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.665+0000 7f50c9bdb700 1 -- 172.21.15.112:0/3579373937 shutdown_connections 2023-08-27T11:28:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.665+0000 7f50c9bdb700 1 -- 172.21.15.112:0/3579373937 wait complete. 2023-08-27T11:28:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.666+0000 7f50c9bdb700 1 Processor -- start 2023-08-27T11:28:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.667+0000 7f50c9bdb700 1 -- start start 2023-08-27T11:28:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.667+0000 7f50c9bdb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50c40b1400 0x7f50c40acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.667+0000 7f50c9bdb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c40b17e0 0x7f50c4145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.667+0000 7f50c9bdb700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50c4161ce0 0x7f50c40c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.667+0000 7f50c9bdb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40c70d0 con 0x7f50c40b17e0 2023-08-27T11:28:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.667+0000 7f50c9bdb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40c7370 con 0x7f50c40b1400 2023-08-27T11:28:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.667+0000 7f50c9bdb700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c41460c0 con 0x7f50c4161ce0 2023-08-27T11:28:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50c2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c40b17e0 0x7f50c4145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50c2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c40b17e0 0x7f50c4145b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37262/0 (socket says 172.21.15.112:37262) 2023-08-27T11:28:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50c2ffd700 1 -- 172.21.15.112:0/1035748817 learned_addr learned my addr 172.21.15.112:0/1035748817 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50c37fe700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50c40b1400 0x7f50c40acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50c3fff700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50c4161ce0 0x7f50c40c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50c2ffd700 1 -- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50c4161ce0 msgr2=0x7f50c40c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50c2ffd700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50c4161ce0 0x7f50c40c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50c2ffd700 1 -- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50c40b1400 msgr2=0x7f50c40acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50c2ffd700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50c40b1400 0x7f50c40acc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50c2ffd700 1 -- 172.21.15.112:0/1035748817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b40094a0 con 0x7f50c40b17e0 2023-08-27T11:28:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50c2ffd700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c40b17e0 0x7f50c4145b80 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009c40 tx=0x7f50b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50abfff700 1 -- 172.21.15.112:0/1035748817 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b80044d0 con 0x7f50c40b17e0 2023-08-27T11:28:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50abfff700 1 -- 172.21.15.112:0/1035748817 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b80036a0 con 0x7f50c40b17e0 2023-08-27T11:28:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50abfff700 1 -- 172.21.15.112:0/1035748817 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8003a30 con 0x7f50c40b17e0 2023-08-27T11:28:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50c9bdb700 1 -- 172.21.15.112:0/1035748817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b8009550 con 0x7f50c40b17e0 2023-08-27T11:28:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.668+0000 7f50c9bdb700 1 -- 172.21.15.112:0/1035748817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c4146640 con 0x7f50c40b17e0 2023-08-27T11:28:57.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.669+0000 7f50c9bdb700 1 -- 172.21.15.112:0/1035748817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c4067a20 con 0x7f50c40b17e0 2023-08-27T11:28:57.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.670+0000 7f50abfff700 1 -- 172.21.15.112:0/1035748817 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f50b8021070 con 0x7f50c40b17e0 2023-08-27T11:28:57.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.670+0000 7f50abfff700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f50b006a4d0 0x7f50b006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:57.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.670+0000 7f50abfff700 1 -- 172.21.15.112:0/1035748817 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f50b8089ff0 con 0x7f50c40b17e0 2023-08-27T11:28:57.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.670+0000 7f50c37fe700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f50b006a4d0 0x7f50b006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:57.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.671+0000 7f50c37fe700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f50b006a4d0 0x7f50b006c990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009d70 tx=0x7f50b4009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:57.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.674+0000 7f50abfff700 1 -- 172.21.15.112:0/1035748817 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f50b80551c0 con 0x7f50c40b17e0 2023-08-27T11:28:57.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.907+0000 7f50c9bdb700 1 -- 172.21.15.112:0/1035748817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f50c406db60 con 0x7f50c40b17e0 2023-08-27T11:28:57.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.908+0000 7f50abfff700 1 -- 172.21.15.112:0/1035748817 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v104) v1 ==== 53+0+73 (secure 0 0 0) 0x7f50b8075e60 con 0x7f50c40b17e0 2023-08-27T11:28:57.910 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:28:57.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.910+0000 7f50c9bdb700 1 -- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f50b006a4d0 msgr2=0x7f50b006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:57.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.910+0000 7f50c9bdb700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f50b006a4d0 0x7f50b006c990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009d70 tx=0x7f50b4009210 comp rx=0 tx=0).stop 2023-08-27T11:28:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.910+0000 7f50c9bdb700 1 -- 172.21.15.112:0/1035748817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c40b17e0 msgr2=0x7f50c4145b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.910+0000 7f50c9bdb700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c40b17e0 0x7f50c4145b80 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009c40 tx=0x7f50b8004750 comp rx=0 tx=0).stop 2023-08-27T11:28:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.910+0000 7f50c9bdb700 1 -- 172.21.15.112:0/1035748817 shutdown_connections 2023-08-27T11:28:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.910+0000 7f50c9bdb700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f50b006a4d0 0x7f50b006c990 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.910+0000 7f50c9bdb700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50c4161ce0 0x7f50c40c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.910+0000 7f50c9bdb700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50c40b1400 0x7f50c40acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.910+0000 7f50c9bdb700 1 --2- 172.21.15.112:0/1035748817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50c40b17e0 0x7f50c4145b80 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.910+0000 7f50c9bdb700 1 -- 172.21.15.112:0/1035748817 >> 172.21.15.112:0/1035748817 conn(0x7f50c40d0000 msgr2=0x7f50c40cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.910+0000 7f50c9bdb700 1 -- 172.21.15.112:0/1035748817 shutdown_connections 2023-08-27T11:28:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:57.910+0000 7f50c9bdb700 1 -- 172.21.15.112:0/1035748817 wait complete. 2023-08-27T11:28:57.923 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:28:57.923 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:28:57.924 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2023-08-27T11:28:57.924 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-27T11:28:57.924 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:28:57.925 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:28:57.925 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:28:57.925 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:28:57.926 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:28:57.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:57.926 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2023-08-27T11:28:57.947 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:28:57.948 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:28:57.948 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:28:57.948 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:57.948 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:28:57.949 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:57.949 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:57.949 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:57.949 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:57.949 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:57.950 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:57.950 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:57.950 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:28:57.950 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:57.950 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:28:57.951 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 41721sec preferred_lft 41721sec 2023-08-27T11:28:57.951 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ac17:49c5:cb39:801a/64 scope link noprefixroute 2023-08-27T11:28:57.951 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:57.951 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:28:57.951 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:57.952 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:28:57.952 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:57.952 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c4b4:94ff:fe58:4e35/64 scope link 2023-08-27T11:28:57.952 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:57.952 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:28:57.953 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:28:57.953 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::5806:6aff:fe04:ef32/64 scope link 2023-08-27T11:28:57.953 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:57.953 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:28:57.953 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 8a:21:ba:15:69:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-27T11:28:57.954 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::8821:baff:fe15:6919/64 scope link 2023-08-27T11:28:57.954 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:57.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:57.955 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:28:58.017 INFO:teuthology.orchestra.run.smithi138.stdout:1 2023-08-27T11:28:58.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:58.020 DEBUG:teuthology.orchestra.run.smithi138:> route 2023-08-27T11:28:58.080 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2023-08-27T11:28:58.080 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:28:58.080 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:28:58.080 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:28:58.081 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:28:58.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:58.081 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:28:58.082 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:28:58.082 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:28:58.083 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:28:58.083 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:28:58.083 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:28:58.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:58.190 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:28:58.207 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-27T11:28:58.207 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-27T11:28:58.208 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:28:58.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:58.208 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2023-08-27T11:28:58.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:28:58.275 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-27T11:28:58.332 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:28:58.332 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:28:58.333 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:28:58.389 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:28:58.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:58.390 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:28:58.474 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:28:58.475 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:28:58.475 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:28:58.483 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:28:58.483 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:28:58.483 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:28:58.484 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:28:58.484 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:28:58.509 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:28:58.509 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:28:58.509 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:28:58.509 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:28:58.510 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:28:58.510 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:28:58.510 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:28:58.510 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:28:58.515 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:28:58.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:58.516 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:28:58.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:58.545 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:28:58.616 INFO:teuthology.orchestra.run.smithi138.stdout:client8008 2023-08-27T11:28:58.617 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:28:58.617 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:58.674 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-08-27T11:28:58.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.803+0000 7f9dbc2a8700 1 Processor -- start 2023-08-27T11:28:58.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.803+0000 7f9dbc2a8700 1 -- start start 2023-08-27T11:28:58.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9dbc2a8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9db4157ae0 0x7f9db415e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:58.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9dbc2a8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db4161390 con 0x7f9db406f790 2023-08-27T11:28:58.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9dbc2a8700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db40b35f0 con 0x7f9db40b3de0 2023-08-27T11:28:58.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9dbc2a8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db40b1aa0 con 0x7f9db4157ae0 2023-08-27T11:28:58.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9dba044700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9db4157ae0 0x7f9db415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:58.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9dba845700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9db406f790 0x7f9db40b05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59886/0 (socket says 172.21.15.112:59886) 2023-08-27T11:28:58.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9dba845700 1 -- 172.21.15.112:0/82893426 learned_addr learned my addr 172.21.15.112:0/82893426 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:28:58.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9dba044700 1 -- 172.21.15.112:0/82893426 >> v1:172.21.15.138:6789/0 conn(0x7f9db40b3de0 legacy=0x7f9db4161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:58.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9dba044700 1 -- 172.21.15.112:0/82893426 >> v1:172.21.15.112:6789/0 conn(0x7f9db406f790 legacy=0x7f9db40b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:58.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9dba044700 1 -- 172.21.15.112:0/82893426 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db40b4330 con 0x7f9db4157ae0 2023-08-27T11:28:58.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9db9042700 1 -- 172.21.15.112:0/82893426 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900688003 0 0) 0x7f9db4161390 con 0x7f9db406f790 2023-08-27T11:28:58.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9dba044700 1 --2- 172.21.15.112:0/82893426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db4157ae0 0x7f9db415e580 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009840 tx=0x7f9db0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50b394893916f5a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:58.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9db9042700 1 -- 172.21.15.112:0/82893426 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0011830 con 0x7f9db4157ae0 2023-08-27T11:28:58.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9db9042700 1 -- 172.21.15.112:0/82893426 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db0011990 con 0x7f9db4157ae0 2023-08-27T11:28:58.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.804+0000 7f9db9042700 1 -- 172.21.15.112:0/82893426 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0011d20 con 0x7f9db4157ae0 2023-08-27T11:28:58.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.805+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/82893426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db4157ae0 msgr2=0x7f9db415e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:58.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.805+0000 7f9dbc2a8700 1 --2- 172.21.15.112:0/82893426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db4157ae0 0x7f9db415e580 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009840 tx=0x7f9db0009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:58.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.805+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/82893426 shutdown_connections 2023-08-27T11:28:58.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.805+0000 7f9dbc2a8700 1 --2- 172.21.15.112:0/82893426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db4157ae0 0x7f9db415e580 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:58.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.805+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/82893426 >> 172.21.15.112:0/82893426 conn(0x7f9db40b1be0 msgr2=0x7f9db40b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:58.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.805+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/82893426 shutdown_connections 2023-08-27T11:28:58.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.805+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/82893426 wait complete. 2023-08-27T11:28:58.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.806+0000 7f9dbc2a8700 1 Processor -- start 2023-08-27T11:28:58.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.806+0000 7f9dbc2a8700 1 -- start start 2023-08-27T11:28:58.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dbc2a8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db406f790 0x7f9db4160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:58.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dbc2a8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db40b3de0 0x7f9db415ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:58.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dbc2a8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db4157ae0 0x7f9db415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:58.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dbc2a8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db41574c0 con 0x7f9db406f790 2023-08-27T11:28:58.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dbc2a8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db41f2ba0 con 0x7f9db4157ae0 2023-08-27T11:28:58.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dbc2a8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db41f2e80 con 0x7f9db40b3de0 2023-08-27T11:28:58.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dba044700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db406f790 0x7f9db4160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:58.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dba044700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db406f790 0x7f9db4160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37274/0 (socket says 172.21.15.112:37274) 2023-08-27T11:28:58.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dba044700 1 -- 172.21.15.112:0/2372915372 learned_addr learned my addr 172.21.15.112:0/2372915372 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:58.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dba845700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db4157ae0 0x7f9db415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:58.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9db9843700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db40b3de0 0x7f9db415ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:58.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dba044700 1 -- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db40b3de0 msgr2=0x7f9db415ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:58.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dba044700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db40b3de0 0x7f9db415ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:58.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dba044700 1 -- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db4157ae0 msgr2=0x7f9db415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:58.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dba044700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db4157ae0 0x7f9db415cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:58.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9dba044700 1 -- 172.21.15.112:0/2372915372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db00094a0 con 0x7f9db406f790 2023-08-27T11:28:58.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.807+0000 7f9db9843700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db40b3de0 0x7f9db415ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:58.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.808+0000 7f9dba044700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db406f790 0x7f9db4160230 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f9db00090b0 tx=0x7f9db0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:58.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.808+0000 7f9daaffd700 1 -- 172.21.15.112:0/2372915372 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db001a850 con 0x7f9db406f790 2023-08-27T11:28:58.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.808+0000 7f9daaffd700 1 -- 172.21.15.112:0/2372915372 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db001a9b0 con 0x7f9db406f790 2023-08-27T11:28:58.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.808+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/2372915372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db41f3160 con 0x7f9db406f790 2023-08-27T11:28:58.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.808+0000 7f9daaffd700 1 -- 172.21.15.112:0/2372915372 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db001ad20 con 0x7f9db406f790 2023-08-27T11:28:58.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.808+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/2372915372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db41f3720 con 0x7f9db406f790 2023-08-27T11:28:58.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.809+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/2372915372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db40adec0 con 0x7f9db406f790 2023-08-27T11:28:58.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.809+0000 7f9daaffd700 1 -- 172.21.15.112:0/2372915372 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f9db0022070 con 0x7f9db406f790 2023-08-27T11:28:58.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.809+0000 7f9daaffd700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d9806a4d0 0x7f9d9806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:58.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.810+0000 7f9daaffd700 1 -- 172.21.15.112:0/2372915372 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f9db0089e10 con 0x7f9db406f790 2023-08-27T11:28:58.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.810+0000 7f9db9843700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d9806a4d0 0x7f9d9806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:58.820 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.810+0000 7f9db9843700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d9806a4d0 0x7f9d9806c990 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9da4000f60 tx=0x7f9da4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:58.821 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:58.814+0000 7f9daaffd700 1 -- 172.21.15.112:0/2372915372 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9db008d030 con 0x7f9db406f790 2023-08-27T11:28:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.046+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/2372915372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9db4067a20 con 0x7f9db406f790 2023-08-27T11:28:59.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.459+0000 7f9daaffd700 1 -- 172.21.15.112:0/2372915372 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v105) v1 ==== 136+0+0 (secure 0 0 0) 0x7f9db0055000 con 0x7f9db406f790 2023-08-27T11:28:59.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.461+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d9806a4d0 msgr2=0x7f9d9806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.461+0000 7f9dbc2a8700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d9806a4d0 0x7f9d9806c990 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9da4000f60 tx=0x7f9da4009040 comp rx=0 tx=0).stop 2023-08-27T11:28:59.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.461+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/2372915372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db406f790 msgr2=0x7f9db4160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.461+0000 7f9dbc2a8700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db406f790 0x7f9db4160230 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f9db00090b0 tx=0x7f9db0004750 comp rx=0 tx=0).stop 2023-08-27T11:28:59.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.462+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/2372915372 shutdown_connections 2023-08-27T11:28:59.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.462+0000 7f9dbc2a8700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9d9806a4d0 0x7f9d9806c990 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.462+0000 7f9dbc2a8700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db4157ae0 0x7f9db415cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.462+0000 7f9dbc2a8700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db406f790 0x7f9db4160230 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.462+0000 7f9dbc2a8700 1 --2- 172.21.15.112:0/2372915372 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db40b3de0 0x7f9db415ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.462+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/2372915372 >> 172.21.15.112:0/2372915372 conn(0x7f9db40b1be0 msgr2=0x7f9db415b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.462+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/2372915372 shutdown_connections 2023-08-27T11:28:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.462+0000 7f9dbc2a8700 1 -- 172.21.15.112:0/2372915372 wait complete. 2023-08-27T11:28:59.476 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:28:59.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.600+0000 7f8996bc0700 1 Processor -- start 2023-08-27T11:28:59.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.600+0000 7f8996bc0700 1 -- start start 2023-08-27T11:28:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.601+0000 7f8996bc0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8990144570 0x7f899015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.601+0000 7f8996bc0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89900b3e50 0x7f899015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.601+0000 7f8996bc0700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89900b5ae0 con 0x7f899015f8a0 2023-08-27T11:28:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.601+0000 7f8996bc0700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8990144950 con 0x7f8990144570 2023-08-27T11:28:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.601+0000 7f8996bc0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89900b45f0 con 0x7f89900b3e50 2023-08-27T11:28:59.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.601+0000 7f899495c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8990144570 0x7f899015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.601+0000 7f899495c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8990144570 0x7f899015ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:37708/0 (socket says 172.21.15.112:37708) 2023-08-27T11:28:59.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.601+0000 7f899495c700 1 -- 172.21.15.112:0/2550392782 learned_addr learned my addr 172.21.15.112:0/2550392782 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:59.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.601+0000 7f898ffff700 1 --2- 172.21.15.112:0/2550392782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89900b3e50 0x7f899015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.601+0000 7f899495c700 1 -- 172.21.15.112:0/2550392782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89900b3e50 msgr2=0x7f899015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:59.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.601+0000 7f899495c700 1 --2- 172.21.15.112:0/2550392782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89900b3e50 0x7f899015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.601+0000 7f899495c700 1 -- 172.21.15.112:0/2550392782 >> v1:172.21.15.138:6789/0 conn(0x7f899015f8a0 legacy=0x7f89900ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.601+0000 7f899495c700 1 -- 172.21.15.112:0/2550392782 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89900b4310 con 0x7f8990144570 2023-08-27T11:28:59.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.602+0000 7f899495c700 1 --2- 172.21.15.112:0/2550392782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8990144570 0x7f899015ee20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f89840098f0 tx=0x7f8984009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81d871cfcad06536 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.602+0000 7f898f7fe700 1 -- 172.21.15.112:0/2550392782 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984011830 con 0x7f8990144570 2023-08-27T11:28:59.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.602+0000 7f898f7fe700 1 -- 172.21.15.112:0/2550392782 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8984011990 con 0x7f8990144570 2023-08-27T11:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.602+0000 7f898f7fe700 1 -- 172.21.15.112:0/2550392782 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984011d20 con 0x7f8990144570 2023-08-27T11:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.602+0000 7f8996bc0700 1 -- 172.21.15.112:0/2550392782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8990144570 msgr2=0x7f899015ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.602+0000 7f8996bc0700 1 --2- 172.21.15.112:0/2550392782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8990144570 0x7f899015ee20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f89840098f0 tx=0x7f8984009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.602+0000 7f8996bc0700 1 -- 172.21.15.112:0/2550392782 shutdown_connections 2023-08-27T11:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.602+0000 7f8996bc0700 1 --2- 172.21.15.112:0/2550392782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8990144570 0x7f899015ee20 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.602+0000 7f8996bc0700 1 --2- 172.21.15.112:0/2550392782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89900b3e50 0x7f899015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.602+0000 7f8996bc0700 1 -- 172.21.15.112:0/2550392782 >> 172.21.15.112:0/2550392782 conn(0x7f899006f970 msgr2=0x7f899006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.602+0000 7f8996bc0700 1 -- 172.21.15.112:0/2550392782 shutdown_connections 2023-08-27T11:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.602+0000 7f8996bc0700 1 -- 172.21.15.112:0/2550392782 wait complete. 2023-08-27T11:28:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.604+0000 7f8996bc0700 1 Processor -- start 2023-08-27T11:28:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.604+0000 7f8996bc0700 1 -- start start 2023-08-27T11:28:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.604+0000 7f8996bc0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89900b3e50 0x7f89900ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.604+0000 7f8996bc0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8990144570 0x7f899015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.604+0000 7f8996bc0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f899015f8a0 0x7f89900c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.604+0000 7f8996bc0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89900cd310 con 0x7f899015f8a0 2023-08-27T11:28:59.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.604+0000 7f8996bc0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899015a5c0 con 0x7f89900b3e50 2023-08-27T11:28:59.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.604+0000 7f8996bc0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89900cbd30 con 0x7f8990144570 2023-08-27T11:28:59.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f899515d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f899015f8a0 0x7f89900c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f899515d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f899015f8a0 0x7f89900c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37290/0 (socket says 172.21.15.112:37290) 2023-08-27T11:28:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f899515d700 1 -- 172.21.15.112:0/3173465187 learned_addr learned my addr 172.21.15.112:0/3173465187 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f899495c700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89900b3e50 0x7f89900ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f898ffff700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8990144570 0x7f899015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f899515d700 1 -- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8990144570 msgr2=0x7f899015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f899515d700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8990144570 0x7f899015a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f899515d700 1 -- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89900b3e50 msgr2=0x7f89900ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f899515d700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89900b3e50 0x7f89900ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f899515d700 1 -- 172.21.15.112:0/3173465187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8984009550 con 0x7f899015f8a0 2023-08-27T11:28:59.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f899495c700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89900b3e50 0x7f89900ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:59.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f898ffff700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8990144570 0x7f899015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:59.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f899515d700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f899015f8a0 0x7f89900c9e90 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f898800cb50 tx=0x7f898800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f898d7fa700 1 -- 172.21.15.112:0/3173465187 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988003c00 con 0x7f899015f8a0 2023-08-27T11:28:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f898d7fa700 1 -- 172.21.15.112:0/3173465187 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8988003d60 con 0x7f899015f8a0 2023-08-27T11:28:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f8996bc0700 1 -- 172.21.15.112:0/3173465187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89900cc010 con 0x7f899015f8a0 2023-08-27T11:28:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f898d7fa700 1 -- 172.21.15.112:0/3173465187 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898800ab70 con 0x7f899015f8a0 2023-08-27T11:28:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.605+0000 7f8996bc0700 1 -- 172.21.15.112:0/3173465187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89900cc5d0 con 0x7f899015f8a0 2023-08-27T11:28:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.606+0000 7f8996bc0700 1 -- 172.21.15.112:0/3173465187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8990067a20 con 0x7f899015f8a0 2023-08-27T11:28:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.607+0000 7f898d7fa700 1 -- 172.21.15.112:0/3173465187 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f898800acd0 con 0x7f899015f8a0 2023-08-27T11:28:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.607+0000 7f898d7fa700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f897406a4d0 0x7f897406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.607+0000 7f898d7fa700 1 -- 172.21.15.112:0/3173465187 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f8988088b50 con 0x7f899015f8a0 2023-08-27T11:28:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.607+0000 7f899495c700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f897406a4d0 0x7f897406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.608+0000 7f899495c700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f897406a4d0 0x7f897406c990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8984009c40 tx=0x7f8984008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.611+0000 7f898d7fa700 1 -- 172.21.15.112:0/3173465187 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8988053c90 con 0x7f899015f8a0 2023-08-27T11:28:59.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.827+0000 7f8996bc0700 1 -- 172.21.15.112:0/3173465187 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f89900cdf90 con 0x7f897406a4d0 2023-08-27T11:28:59.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.829+0000 7f898d7fa700 1 -- 172.21.15.112:0/3173465187 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f8984011c80 con 0x7f897406a4d0 2023-08-27T11:28:59.831 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:28:59.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:28:59.831 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs" 2023-08-27T11:28:59.832 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:28:59.832 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:28:59.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.831+0000 7f8996bc0700 1 -- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f897406a4d0 msgr2=0x7f897406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.831+0000 7f8996bc0700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f897406a4d0 0x7f897406c990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8984009c40 tx=0x7f8984008030 comp rx=0 tx=0).stop 2023-08-27T11:28:59.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.831+0000 7f8996bc0700 1 -- 172.21.15.112:0/3173465187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f899015f8a0 msgr2=0x7f89900c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.831+0000 7f8996bc0700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f899015f8a0 0x7f89900c9e90 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f898800cb50 tx=0x7f898800cf10 comp rx=0 tx=0).stop 2023-08-27T11:28:59.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.832+0000 7f8996bc0700 1 -- 172.21.15.112:0/3173465187 shutdown_connections 2023-08-27T11:28:59.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.832+0000 7f8996bc0700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f897406a4d0 0x7f897406c990 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.832+0000 7f8996bc0700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f899015f8a0 0x7f89900c9e90 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.832+0000 7f8996bc0700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89900b3e50 0x7f89900ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.832+0000 7f8996bc0700 1 --2- 172.21.15.112:0/3173465187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8990144570 0x7f899015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.832+0000 7f8996bc0700 1 -- 172.21.15.112:0/3173465187 >> 172.21.15.112:0/3173465187 conn(0x7f899006f970 msgr2=0x7f8990140fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:59.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.832+0000 7f8996bc0700 1 -- 172.21.15.112:0/3173465187 shutdown_connections 2023-08-27T11:28:59.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.832+0000 7f8996bc0700 1 -- 172.21.15.112:0/3173465187 wait complete. 2023-08-27T11:28:59.845 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:28:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.967+0000 7f179ce50700 1 Processor -- start 2023-08-27T11:28:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.967+0000 7f179ce50700 1 -- start start 2023-08-27T11:28:59.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f179ce50700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1798146c20 0x7f1798160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f179ce50700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f17980b3440 0x7f179815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f179ce50700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f179815e6e0 0x7f179806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f179ce50700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17980b5070 con 0x7f17980b3440 2023-08-27T11:28:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f179ce50700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1798147000 con 0x7f179815e6e0 2023-08-27T11:28:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f179ce50700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17981614a0 con 0x7f1798146c20 2023-08-27T11:28:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f1795d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f17980b3440 0x7f179815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f1795d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f17980b3440 0x7f179815e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37296/0 (socket says 172.21.15.112:37296) 2023-08-27T11:28:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f1795d9b700 1 -- 172.21.15.112:0/476212147 learned_addr learned my addr 172.21.15.112:0/476212147 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f1796d9d700 1 --2- 172.21.15.112:0/476212147 >> v2:172.21.15.138:3300/0 conn(0x7f179815e6e0 0x7f179806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f179659c700 1 --2- 172.21.15.112:0/476212147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1798146c20 0x7f1798160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f1795d9b700 1 -- 172.21.15.112:0/476212147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1798146c20 msgr2=0x7f1798160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f1795d9b700 1 --2- 172.21.15.112:0/476212147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1798146c20 0x7f1798160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.968+0000 7f1795d9b700 1 -- 172.21.15.112:0/476212147 >> v2:172.21.15.138:3300/0 conn(0x7f179815e6e0 msgr2=0x7f179806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f1795d9b700 1 --2- 172.21.15.112:0/476212147 >> v2:172.21.15.138:3300/0 conn(0x7f179815e6e0 0x7f179806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f1795d9b700 1 -- 172.21.15.112:0/476212147 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1798161740 con 0x7f17980b3440 2023-08-27T11:28:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f1795d9b700 1 --2- 172.21.15.112:0/476212147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17980b3440 0x7f179815e1a0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f17980b1c80 tx=0x7f1788009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fd046e4c6a4525 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f179559a700 1 -- 172.21.15.112:0/476212147 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178800b870 con 0x7f17980b3440 2023-08-27T11:28:59.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f179559a700 1 -- 172.21.15.112:0/476212147 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178800b9d0 con 0x7f17980b3440 2023-08-27T11:28:59.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f179559a700 1 -- 172.21.15.112:0/476212147 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178800bd60 con 0x7f17980b3440 2023-08-27T11:28:59.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f179ce50700 1 -- 172.21.15.112:0/476212147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17980b3440 msgr2=0x7f179815e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f179ce50700 1 --2- 172.21.15.112:0/476212147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17980b3440 0x7f179815e1a0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f17980b1c80 tx=0x7f1788009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f179ce50700 1 -- 172.21.15.112:0/476212147 shutdown_connections 2023-08-27T11:28:59.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f179ce50700 1 --2- 172.21.15.112:0/476212147 >> v2:172.21.15.138:3300/0 conn(0x7f179815e6e0 0x7f179806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f179ce50700 1 --2- 172.21.15.112:0/476212147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1798146c20 0x7f1798160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f179ce50700 1 --2- 172.21.15.112:0/476212147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17980b3440 0x7f179815e1a0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f179ce50700 1 -- 172.21.15.112:0/476212147 >> 172.21.15.112:0/476212147 conn(0x7f17980b1e90 msgr2=0x7f17980b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:59.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f179ce50700 1 -- 172.21.15.112:0/476212147 shutdown_connections 2023-08-27T11:28:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.969+0000 7f179ce50700 1 -- 172.21.15.112:0/476212147 wait complete. 2023-08-27T11:28:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.971+0000 7f179ce50700 1 Processor -- start 2023-08-27T11:28:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.971+0000 7f179ce50700 1 -- start start 2023-08-27T11:28:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.971+0000 7f179ce50700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17980b3440 0x7f179815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.971+0000 7f179ce50700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1798146c20 0x7f1798143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f179ce50700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179815e6e0 0x7f1798143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f179ce50700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179815b000 con 0x7f1798146c20 2023-08-27T11:28:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f179ce50700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17981442d0 con 0x7f179815e6e0 2023-08-27T11:28:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f179ce50700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17981445b0 con 0x7f17980b3440 2023-08-27T11:28:59.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1795d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1798146c20 0x7f1798143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1795d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1798146c20 0x7f1798143530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37302/0 (socket says 172.21.15.112:37302) 2023-08-27T11:28:59.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1795d9b700 1 -- 172.21.15.112:0/2075920106 learned_addr learned my addr 172.21.15.112:0/2075920106 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:28:59.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f179659c700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17980b3440 0x7f179815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1796d9d700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179815e6e0 0x7f1798143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1795d9b700 1 -- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17980b3440 msgr2=0x7f179815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1795d9b700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17980b3440 0x7f179815aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1795d9b700 1 -- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179815e6e0 msgr2=0x7f1798143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1795d9b700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179815e6e0 0x7f1798143a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1795d9b700 1 -- 172.21.15.112:0/2075920106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1788009550 con 0x7f1798146c20 2023-08-27T11:28:59.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1796d9d700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179815e6e0 0x7f1798143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:59.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1795d9b700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1798146c20 0x7f1798143530 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f1788000c00 tx=0x7f1788004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1786ffd700 1 -- 172.21.15.112:0/2075920106 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788004520 con 0x7f1798146c20 2023-08-27T11:28:59.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1786ffd700 1 -- 172.21.15.112:0/2075920106 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178801e070 con 0x7f1798146c20 2023-08-27T11:28:59.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f179ce50700 1 -- 172.21.15.112:0/2075920106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1798144890 con 0x7f1798146c20 2023-08-27T11:28:59.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f1786ffd700 1 -- 172.21.15.112:0/2075920106 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17880038c0 con 0x7f1798146c20 2023-08-27T11:28:59.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.972+0000 7f179ce50700 1 -- 172.21.15.112:0/2075920106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1798144e50 con 0x7f1798146c20 2023-08-27T11:28:59.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.973+0000 7f179ce50700 1 -- 172.21.15.112:0/2075920106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1798067e90 con 0x7f1798146c20 2023-08-27T11:28:59.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.974+0000 7f1786ffd700 1 -- 172.21.15.112:0/2075920106 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f1788003a20 con 0x7f1798146c20 2023-08-27T11:28:59.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.974+0000 7f1786ffd700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f177806a4d0 0x7f177806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.974+0000 7f1786ffd700 1 -- 172.21.15.112:0/2075920106 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f1788020070 con 0x7f1798146c20 2023-08-27T11:28:59.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.974+0000 7f179659c700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f177806a4d0 0x7f177806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.975+0000 7f179659c700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f177806a4d0 0x7f177806c990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f178c00c900 tx=0x7f178c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:28:59.978+0000 7f1786ffd700 1 -- 172.21.15.112:0/2075920106 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f17880553b0 con 0x7f1798146c20 2023-08-27T11:29:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.167+0000 7f179ce50700 1 -- 172.21.15.112:0/2075920106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1798067a20 con 0x7f1798146c20 2023-08-27T11:29:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.167+0000 7f1786ffd700 1 -- 172.21.15.112:0/2075920106 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f17880150c0 con 0x7f1798146c20 2023-08-27T11:29:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.169+0000 7f179ce50700 1 -- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f177806a4d0 msgr2=0x7f177806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.169+0000 7f179ce50700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f177806a4d0 0x7f177806c990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f178c00c900 tx=0x7f178c008040 comp rx=0 tx=0).stop 2023-08-27T11:29:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.169+0000 7f179ce50700 1 -- 172.21.15.112:0/2075920106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1798146c20 msgr2=0x7f1798143530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.169+0000 7f179ce50700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1798146c20 0x7f1798143530 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f1788000c00 tx=0x7f1788004750 comp rx=0 tx=0).stop 2023-08-27T11:29:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.169+0000 7f179ce50700 1 -- 172.21.15.112:0/2075920106 shutdown_connections 2023-08-27T11:29:00.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.169+0000 7f179ce50700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f177806a4d0 0x7f177806c990 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.169+0000 7f179ce50700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179815e6e0 0x7f1798143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.169+0000 7f179ce50700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f17980b3440 0x7f179815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.169+0000 7f179ce50700 1 --2- 172.21.15.112:0/2075920106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1798146c20 0x7f1798143530 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.169+0000 7f179ce50700 1 -- 172.21.15.112:0/2075920106 >> 172.21.15.112:0/2075920106 conn(0x7f17980b1e90 msgr2=0x7f17980b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:00.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.169+0000 7f179ce50700 1 -- 172.21.15.112:0/2075920106 shutdown_connections 2023-08-27T11:29:00.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.169+0000 7f179ce50700 1 -- 172.21.15.112:0/2075920106 wait complete. 2023-08-27T11:29:00.183 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000303197 2023-08-27T11:29:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.304+0000 7f773aaa7700 1 Processor -- start 2023-08-27T11:29:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.304+0000 7f773aaa7700 1 -- start start 2023-08-27T11:29:00.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.305+0000 7f773aaa7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77340b1350 0x7f773406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.305+0000 7f773aaa7700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f773415d7d0 0x7f773406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.305+0000 7f773aaa7700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77340b0c30 con 0x7f773415d3f0 2023-08-27T11:29:00.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.305+0000 7f773aaa7700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77340add00 con 0x7f773415d7d0 2023-08-27T11:29:00.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.305+0000 7f773aaa7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77340b3a10 con 0x7f77340b1350 2023-08-27T11:29:00.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.305+0000 7f7733fff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77340b1350 0x7f773406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.306+0000 7f7739044700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f773415d7d0 0x7f773406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.306+0000 7f7739044700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f773415d7d0 0x7f773406fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:37738/0 (socket says 172.21.15.112:37738) 2023-08-27T11:29:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.306+0000 7f7738843700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f773415d3f0 0x7f77340ad5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:39214/0 (socket says 172.21.15.112:39214) 2023-08-27T11:29:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.306+0000 7f7739044700 1 -- 172.21.15.112:0/3223871100 learned_addr learned my addr 172.21.15.112:0/3223871100 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.306+0000 7f7733fff700 1 -- 172.21.15.112:0/3223871100 >> v2:172.21.15.138:3300/0 conn(0x7f773415d7d0 msgr2=0x7f773406fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.306+0000 7f7733fff700 1 --2- 172.21.15.112:0/3223871100 >> v2:172.21.15.138:3300/0 conn(0x7f773415d7d0 0x7f773406fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.306+0000 7f7733fff700 1 -- 172.21.15.112:0/3223871100 >> v1:172.21.15.138:6789/0 conn(0x7f773415d3f0 legacy=0x7f77340ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.306+0000 7f7733fff700 1 -- 172.21.15.112:0/3223871100 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7734161d60 con 0x7f77340b1350 2023-08-27T11:29:00.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.306+0000 7f7739044700 1 --2- 172.21.15.112:0/3223871100 >> v2:172.21.15.138:3300/0 conn(0x7f773415d7d0 0x7f773406fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:00.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.306+0000 7f7733fff700 1 --2- 172.21.15.112:0/3223871100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77340b1350 0x7f773406f790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7720009840 tx=0x7f7720009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24fa9cf1eec1dd2f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.306+0000 7f77337fe700 1 -- 172.21.15.112:0/3223871100 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720011830 con 0x7f77340b1350 2023-08-27T11:29:00.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.306+0000 7f77337fe700 1 -- 172.21.15.112:0/3223871100 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7720011990 con 0x7f77340b1350 2023-08-27T11:29:00.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.306+0000 7f77337fe700 1 -- 172.21.15.112:0/3223871100 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720011d20 con 0x7f77340b1350 2023-08-27T11:29:00.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.307+0000 7f773aaa7700 1 -- 172.21.15.112:0/3223871100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77340b1350 msgr2=0x7f773406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.307+0000 7f773aaa7700 1 --2- 172.21.15.112:0/3223871100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77340b1350 0x7f773406f790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7720009840 tx=0x7f7720009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.307+0000 7f773aaa7700 1 -- 172.21.15.112:0/3223871100 shutdown_connections 2023-08-27T11:29:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.307+0000 7f773aaa7700 1 --2- 172.21.15.112:0/3223871100 >> v2:172.21.15.138:3300/0 conn(0x7f773415d7d0 0x7f773406fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.307+0000 7f773aaa7700 1 --2- 172.21.15.112:0/3223871100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77340b1350 0x7f773406f790 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.307+0000 7f773aaa7700 1 -- 172.21.15.112:0/3223871100 >> 172.21.15.112:0/3223871100 conn(0x7f77340b1e90 msgr2=0x7f77340b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.307+0000 7f773aaa7700 1 -- 172.21.15.112:0/3223871100 shutdown_connections 2023-08-27T11:29:00.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.307+0000 7f773aaa7700 1 -- 172.21.15.112:0/3223871100 wait complete. 2023-08-27T11:29:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.308+0000 7f773aaa7700 1 Processor -- start 2023-08-27T11:29:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.308+0000 7f773aaa7700 1 -- start start 2023-08-27T11:29:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.309+0000 7f773aaa7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77340b1350 0x7f7734150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.309+0000 7f773aaa7700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f773415d3f0 0x7f7734157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.309+0000 7f773aaa7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f773415d7d0 0x7f7734158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.309+0000 7f773aaa7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77341f3ad0 con 0x7f773415d7d0 2023-08-27T11:29:00.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.309+0000 7f773aaa7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773415edd0 con 0x7f77340b1350 2023-08-27T11:29:00.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.309+0000 7f773aaa7700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773415f0b0 con 0x7f773415d3f0 2023-08-27T11:29:00.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.309+0000 7f7739044700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f773415d7d0 0x7f7734158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.309+0000 7f7739044700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f773415d7d0 0x7f7734158000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37316/0 (socket says 172.21.15.112:37316) 2023-08-27T11:29:00.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.309+0000 7f7739044700 1 -- 172.21.15.112:0/4073081452 learned_addr learned my addr 172.21.15.112:0/4073081452 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:00.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.309+0000 7f7733fff700 1 --2- 172.21.15.112:0/4073081452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f773415d3f0 0x7f7734157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.309+0000 7f7738843700 1 --2- 172.21.15.112:0/4073081452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77340b1350 0x7f7734150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.310+0000 7f7739044700 1 -- 172.21.15.112:0/4073081452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f773415d3f0 msgr2=0x7f7734157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.310+0000 7f7739044700 1 --2- 172.21.15.112:0/4073081452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f773415d3f0 0x7f7734157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.310+0000 7f7739044700 1 -- 172.21.15.112:0/4073081452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77340b1350 msgr2=0x7f7734150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.310+0000 7f7739044700 1 --2- 172.21.15.112:0/4073081452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77340b1350 0x7f7734150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.310+0000 7f7739044700 1 -- 172.21.15.112:0/4073081452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c009b30 con 0x7f773415d7d0 2023-08-27T11:29:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.310+0000 7f7733fff700 1 --2- 172.21.15.112:0/4073081452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f773415d3f0 0x7f7734157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.310+0000 7f7738843700 1 --2- 172.21.15.112:0/4073081452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77340b1350 0x7f7734150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.310+0000 7f7739044700 1 --2- 172.21.15.112:0/4073081452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f773415d7d0 0x7f7734158000 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f772c00efc0 tx=0x7f772c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.310+0000 7f77317fa700 1 -- 172.21.15.112:0/4073081452 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c018070 con 0x7f773415d7d0 2023-08-27T11:29:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.310+0000 7f77317fa700 1 -- 172.21.15.112:0/4073081452 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c00f040 con 0x7f773415d7d0 2023-08-27T11:29:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.310+0000 7f773aaa7700 1 -- 172.21.15.112:0/4073081452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77200094a0 con 0x7f773415d7d0 2023-08-27T11:29:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.310+0000 7f77317fa700 1 -- 172.21.15.112:0/4073081452 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c003c40 con 0x7f773415d7d0 2023-08-27T11:29:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.310+0000 7f773aaa7700 1 -- 172.21.15.112:0/4073081452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7734150e90 con 0x7f773415d7d0 2023-08-27T11:29:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.311+0000 7f773aaa7700 1 -- 172.21.15.112:0/4073081452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7734151020 con 0x7f773415d7d0 2023-08-27T11:29:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.311+0000 7f77317fa700 1 -- 172.21.15.112:0/4073081452 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f772c00c9f0 con 0x7f773415d7d0 2023-08-27T11:29:00.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.312+0000 7f77317fa700 1 --2- 172.21.15.112:0/4073081452 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f771806a4d0 0x7f771806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.312+0000 7f77317fa700 1 -- 172.21.15.112:0/4073081452 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f772c089080 con 0x7f773415d7d0 2023-08-27T11:29:00.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.312+0000 7f7738843700 1 --2- 172.21.15.112:0/4073081452 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f771806a4d0 0x7f771806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.312+0000 7f7738843700 1 --2- 172.21.15.112:0/4073081452 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f771806a4d0 0x7f771806c990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7728000f60 tx=0x7f7728007dd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.316+0000 7f77317fa700 1 -- 172.21.15.112:0/4073081452 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f772c0bb990 con 0x7f773415d7d0 2023-08-27T11:29:00.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.532+0000 7f773aaa7700 1 -- 172.21.15.112:0/4073081452 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000303197", "target": ["mon-mgr", ""]}) v1 -- 0x7f77340b3f90 con 0x7f771806a4d0 2023-08-27T11:29:00.535 INFO:tasks.ceph.mgr.x.smithi138.stderr:2023-08-27T11:29:00.534+0000 7f2859c79700 -1 mgr.server reply reply (1) Operation not permitted WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue. 2023-08-27T11:29:00.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.534+0000 7f77317fa700 1 -- 172.21.15.112:0/4073081452 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: -1 WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue.) v1 ==== 283+0+0 (secure 0 0 0) 0x7f7728013420 con 0x7f771806a4d0 2023-08-27T11:29:00.537 INFO:teuthology.orchestra.run.smithi112.stderr:Error EPERM: WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue. 2023-08-27T11:29:00.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:29:00.541 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:29:00.542 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:29:00.581 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:29:00.632 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:29:00.632 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:29:00.633 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:00.650 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:29:00.651 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:00.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:00.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.866+0000 7f451b0d6700 1 Processor -- start 2023-08-27T11:29:00.868 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.866+0000 7f451b0d6700 1 -- start start 2023-08-27T11:29:00.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.867+0000 7f451b0d6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f451406dcc0 0x7f45140b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.867+0000 7f451b0d6700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f451415b540 con 0x7f451415a120 2023-08-27T11:29:00.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.867+0000 7f451b0d6700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45140af830 con 0x7f45140b3440 2023-08-27T11:29:00.872 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.867+0000 7f451b0d6700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45140afb10 con 0x7f451406dcc0 2023-08-27T11:29:00.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.867+0000 7f4518e72700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f451415a120 0x7f451406d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59936/0 (socket says 172.21.15.112:59936) 2023-08-27T11:29:00.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.867+0000 7f4518e72700 1 -- 172.21.15.112:0/520086502 learned_addr learned my addr 172.21.15.112:0/520086502 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:00.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f4513fff700 1 --2- 172.21.15.112:0/520086502 >> v2:172.21.15.138:3300/0 conn(0x7f451406dcc0 0x7f45140b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792483993 0 0) 0x7f451415b540 con 0x7f451415a120 2023-08-27T11:29:00.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45040035b0 con 0x7f451415a120 2023-08-27T11:29:00.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759517513 0 0) 0x7f45140af830 con 0x7f45140b3440 2023-08-27T11:29:00.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f451415b540 con 0x7f45140b3440 2023-08-27T11:29:00.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2371761812 0 0) 0x7f45040035b0 con 0x7f451415a120 2023-08-27T11:29:00.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 >> v2:172.21.15.138:3300/0 conn(0x7f451406dcc0 msgr2=0x7f45140b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 --2- 172.21.15.112:0/520086502 >> v2:172.21.15.138:3300/0 conn(0x7f451406dcc0 0x7f45140b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 >> v1:172.21.15.138:6789/0 conn(0x7f45140b3440 legacy=0x7f45140b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45140afdf0 con 0x7f451415a120 2023-08-27T11:29:00.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f4508001670 con 0x7f451415a120 2023-08-27T11:29:00.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 >> v1:172.21.15.112:6789/0 conn(0x7f451415a120 legacy=0x7f451406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 --2- 172.21.15.112:0/520086502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4504003b90 0x7f4504003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 --2- 172.21.15.112:0/520086502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4504004850 0x7f4504004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 --2- 172.21.15.112:0/520086502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4504005210 0x7f450400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45140afb10 con 0x7f4504004850 2023-08-27T11:29:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450400c090 con 0x7f4504003b90 2023-08-27T11:29:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450400c250 con 0x7f4504005210 2023-08-27T11:29:00.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f4513fff700 1 --2- 172.21.15.112:0/520086502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4504004850 0x7f4504004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f4518e72700 1 --2- 172.21.15.112:0/520086502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4504003b90 0x7f4504003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f4519673700 1 --2- 172.21.15.112:0/520086502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4504005210 0x7f450400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f4513fff700 1 -- 172.21.15.112:0/520086502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4504005210 msgr2=0x7f450400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f4513fff700 1 --2- 172.21.15.112:0/520086502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4504005210 0x7f450400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f4513fff700 1 -- 172.21.15.112:0/520086502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4504003b90 msgr2=0x7f4504003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f4513fff700 1 --2- 172.21.15.112:0/520086502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4504003b90 0x7f4504003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f4513fff700 1 -- 172.21.15.112:0/520086502 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040035b0 con 0x7f4504004850 2023-08-27T11:29:00.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f4519673700 1 --2- 172.21.15.112:0/520086502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4504005210 0x7f450400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:00.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.868+0000 7f4519673700 1 -- 172.21.15.112:0/520086502 reap_dead start 2023-08-27T11:29:00.878 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.869+0000 7f4513fff700 1 --2- 172.21.15.112:0/520086502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4504004850 0x7f4504004cd0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4500009b60 tx=0x7f4504005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5b7adb2e519bb4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.869+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450000e6c0 con 0x7f4504004850 2023-08-27T11:29:00.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.869+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450000e820 con 0x7f4504004850 2023-08-27T11:29:00.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.869+0000 7f45137fe700 1 -- 172.21.15.112:0/520086502 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450000eb90 con 0x7f4504004850 2023-08-27T11:29:00.879 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.869+0000 7f451b0d6700 1 -- 172.21.15.112:0/520086502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4504004850 msgr2=0x7f4504004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.869+0000 7f451b0d6700 1 --2- 172.21.15.112:0/520086502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4504004850 0x7f4504004cd0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4500009b60 tx=0x7f4504005880 comp rx=0 tx=0).stop 2023-08-27T11:29:00.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.869+0000 7f451b0d6700 1 -- 172.21.15.112:0/520086502 shutdown_connections 2023-08-27T11:29:00.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.869+0000 7f451b0d6700 1 --2- 172.21.15.112:0/520086502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4504004850 0x7f4504004cd0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.869+0000 7f451b0d6700 1 -- 172.21.15.112:0/520086502 >> 172.21.15.112:0/520086502 conn(0x7f45140cff80 msgr2=0x7f45140d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:00.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.869+0000 7f451b0d6700 1 -- 172.21.15.112:0/520086502 shutdown_connections 2023-08-27T11:29:00.880 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.869+0000 7f451b0d6700 1 -- 172.21.15.112:0/520086502 wait complete. 2023-08-27T11:29:00.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.870+0000 7f451b0d6700 1 Processor -- start 2023-08-27T11:29:00.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.871+0000 7f451b0d6700 1 -- start start 2023-08-27T11:29:00.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.871+0000 7f451b0d6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4504004850 0x7f451414aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.871+0000 7f451b0d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45141f18f0 0x7f45140b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.871+0000 7f451b0d6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45140b2160 0x7f45140ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.871+0000 7f451b0d6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451414b000 con 0x7f45141f18f0 2023-08-27T11:29:00.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.871+0000 7f451b0d6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45140b25e0 con 0x7f45140b2160 2023-08-27T11:29:00.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.871+0000 7f451b0d6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45140c9100 con 0x7f4504004850 2023-08-27T11:29:00.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f4513fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45141f18f0 0x7f45140b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f4513fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45141f18f0 0x7f45140b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37336/0 (socket says 172.21.15.112:37336) 2023-08-27T11:29:00.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f4513fff700 1 -- 172.21.15.112:0/1294195787 learned_addr learned my addr 172.21.15.112:0/1294195787 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:00.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f4518e72700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4504004850 0x7f451414aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f4519673700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45140b2160 0x7f45140ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f4513fff700 1 -- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4504004850 msgr2=0x7f451414aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f4513fff700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4504004850 0x7f451414aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f4513fff700 1 -- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45140b2160 msgr2=0x7f45140ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f4513fff700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45140b2160 0x7f45140ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f4513fff700 1 -- 172.21.15.112:0/1294195787 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45140c84f0 con 0x7f45141f18f0 2023-08-27T11:29:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f4518e72700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4504004850 0x7f451414aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f4519673700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45140b2160 0x7f45140ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f4513fff700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45141f18f0 0x7f45140b1c20 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f450400c060 tx=0x7f45000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f45117fa700 1 -- 172.21.15.112:0/1294195787 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450000e6c0 con 0x7f45141f18f0 2023-08-27T11:29:00.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f45117fa700 1 -- 172.21.15.112:0/1294195787 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4500003ad0 con 0x7f45141f18f0 2023-08-27T11:29:00.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f45117fa700 1 -- 172.21.15.112:0/1294195787 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500003e80 con 0x7f45141f18f0 2023-08-27T11:29:00.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f451b0d6700 1 -- 172.21.15.112:0/1294195787 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45140c87d0 con 0x7f45141f18f0 2023-08-27T11:29:00.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.872+0000 7f451b0d6700 1 -- 172.21.15.112:0/1294195787 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45140c8d90 con 0x7f45141f18f0 2023-08-27T11:29:00.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.873+0000 7f451b0d6700 1 -- 172.21.15.112:0/1294195787 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4514067a20 con 0x7f45141f18f0 2023-08-27T11:29:00.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.873+0000 7f45117fa700 1 -- 172.21.15.112:0/1294195787 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4500013470 con 0x7f45141f18f0 2023-08-27T11:29:00.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.874+0000 7f45117fa700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f44f806a4d0 0x7f44f806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.874+0000 7f45117fa700 1 -- 172.21.15.112:0/1294195787 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f450008a4d0 con 0x7f45141f18f0 2023-08-27T11:29:00.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.874+0000 7f4518e72700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f44f806a4d0 0x7f44f806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.874+0000 7f4518e72700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f44f806a4d0 0x7f44f806c990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4508001eb0 tx=0x7f4508009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:00.878+0000 7f45117fa700 1 -- 172.21.15.112:0/1294195787 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4500055730 con 0x7f45141f18f0 2023-08-27T11:29:01.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.069+0000 7f451b0d6700 1 -- 172.21.15.112:0/1294195787 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f451406d5b0 con 0x7f45141f18f0 2023-08-27T11:29:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.070+0000 7f45117fa700 1 -- 172.21.15.112:0/1294195787 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+18412 (secure 0 0 0) 0x7f4500058d50 con 0x7f45141f18f0 2023-08-27T11:29:01.072 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:01.072 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":155,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:55.437349+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.551922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.772403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.7","mappings":[{"from":6,"to":4}]},{"pgid":"28.1a","mappings":[{"from":6,"to":4}]},{"pgid":"28.1d","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.072+0000 7f451b0d6700 1 -- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f44f806a4d0 msgr2=0x7f44f806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.072+0000 7f451b0d6700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f44f806a4d0 0x7f44f806c990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4508001eb0 tx=0x7f4508009040 comp rx=0 tx=0).stop 2023-08-27T11:29:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.072+0000 7f451b0d6700 1 -- 172.21.15.112:0/1294195787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45141f18f0 msgr2=0x7f45140b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.072+0000 7f451b0d6700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45141f18f0 0x7f45140b1c20 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f450400c060 tx=0x7f45000047c0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.072+0000 7f451b0d6700 1 -- 172.21.15.112:0/1294195787 shutdown_connections 2023-08-27T11:29:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.072+0000 7f451b0d6700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f44f806a4d0 0x7f44f806c990 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.072+0000 7f451b0d6700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45140b2160 0x7f45140ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.072+0000 7f451b0d6700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4504004850 0x7f451414aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.072+0000 7f451b0d6700 1 --2- 172.21.15.112:0/1294195787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45141f18f0 0x7f45140b1c20 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.072+0000 7f451b0d6700 1 -- 172.21.15.112:0/1294195787 >> 172.21.15.112:0/1294195787 conn(0x7f45140cff80 msgr2=0x7f4514070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:01.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.072+0000 7f451b0d6700 1 -- 172.21.15.112:0/1294195787 shutdown_connections 2023-08-27T11:29:01.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.072+0000 7f451b0d6700 1 -- 172.21.15.112:0/1294195787 wait complete. 2023-08-27T11:29:01.086 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:28:47.551922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '152', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:28:48.772403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:29:01.086 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:29:01.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.208+0000 7f296893c700 1 Processor -- start 2023-08-27T11:29:01.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.208+0000 7f296893c700 1 -- start start 2023-08-27T11:29:01.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.209+0000 7f296893c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f296406dcc0 0x7f29640b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.209+0000 7f296893c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29640cfd50 con 0x7f2964143ba0 2023-08-27T11:29:01.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.209+0000 7f296893c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29640cf140 con 0x7f29641437c0 2023-08-27T11:29:01.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.209+0000 7f296893c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29640cf420 con 0x7f296406dcc0 2023-08-27T11:29:01.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.209+0000 7f2961d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2964143ba0 0x7f296406d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59958/0 (socket says 172.21.15.112:59958) 2023-08-27T11:29:01.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.209+0000 7f2961d9b700 1 -- 172.21.15.112:0/4150046389 learned_addr learned my addr 172.21.15.112:0/4150046389 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:01.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.210+0000 7f2962d9d700 1 --2- 172.21.15.112:0/4150046389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f296406dcc0 0x7f29640b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.210+0000 7f296159a700 1 -- 172.21.15.112:0/4150046389 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704373643 0 0) 0x7f29640cfd50 con 0x7f2964143ba0 2023-08-27T11:29:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.210+0000 7f296159a700 1 -- 172.21.15.112:0/4150046389 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f294c0035b0 con 0x7f2964143ba0 2023-08-27T11:29:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.210+0000 7f296159a700 1 -- 172.21.15.112:0/4150046389 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594145877 0 0) 0x7f29640cf140 con 0x7f29641437c0 2023-08-27T11:29:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.210+0000 7f296159a700 1 -- 172.21.15.112:0/4150046389 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29640cfd50 con 0x7f29641437c0 2023-08-27T11:29:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.210+0000 7f2962d9d700 1 -- 172.21.15.112:0/4150046389 >> v1:172.21.15.138:6789/0 conn(0x7f29641437c0 legacy=0x7f29640ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.210+0000 7f2962d9d700 1 -- 172.21.15.112:0/4150046389 >> v1:172.21.15.112:6789/0 conn(0x7f2964143ba0 legacy=0x7f296406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.210+0000 7f2962d9d700 1 -- 172.21.15.112:0/4150046389 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29640cf700 con 0x7f296406dcc0 2023-08-27T11:29:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.210+0000 7f2962d9d700 1 --2- 172.21.15.112:0/4150046389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f296406dcc0 0x7f29640b4670 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f295c009dc0 tx=0x7f295c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18f332c8a949e459 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.210+0000 7f296159a700 1 -- 172.21.15.112:0/4150046389 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c00e070 con 0x7f296406dcc0 2023-08-27T11:29:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.210+0000 7f296159a700 1 -- 172.21.15.112:0/4150046389 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c00bb20 con 0x7f296406dcc0 2023-08-27T11:29:01.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.210+0000 7f296159a700 1 -- 172.21.15.112:0/4150046389 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c00be90 con 0x7f296406dcc0 2023-08-27T11:29:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.211+0000 7f296893c700 1 -- 172.21.15.112:0/4150046389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f296406dcc0 msgr2=0x7f29640b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.211+0000 7f296893c700 1 --2- 172.21.15.112:0/4150046389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f296406dcc0 0x7f29640b4670 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f295c009dc0 tx=0x7f295c00b330 comp rx=0 tx=0).stop 2023-08-27T11:29:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.211+0000 7f296893c700 1 -- 172.21.15.112:0/4150046389 shutdown_connections 2023-08-27T11:29:01.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.211+0000 7f296893c700 1 --2- 172.21.15.112:0/4150046389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f296406dcc0 0x7f29640b4670 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.211+0000 7f296893c700 1 -- 172.21.15.112:0/4150046389 >> 172.21.15.112:0/4150046389 conn(0x7f29640b1e90 msgr2=0x7f29640b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:01.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.211+0000 7f296893c700 1 -- 172.21.15.112:0/4150046389 shutdown_connections 2023-08-27T11:29:01.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.211+0000 7f296893c700 1 -- 172.21.15.112:0/4150046389 wait complete. 2023-08-27T11:29:01.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.212+0000 7f296893c700 1 Processor -- start 2023-08-27T11:29:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.212+0000 7f296893c700 1 -- start start 2023-08-27T11:29:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f296893c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f296406dcc0 0x7f29641592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f296893c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29641437c0 0x7f2964070650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f296893c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2964143ba0 0x7f296415cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f296893c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964138d40 con 0x7f29641437c0 2023-08-27T11:29:01.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f296893c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964159f70 con 0x7f296406dcc0 2023-08-27T11:29:01.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f296893c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2964070ce0 con 0x7f2964143ba0 2023-08-27T11:29:01.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f2961d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29641437c0 0x7f2964070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f2961d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29641437c0 0x7f2964070650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37348/0 (socket says 172.21.15.112:37348) 2023-08-27T11:29:01.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f2961d9b700 1 -- 172.21.15.112:0/3378036141 learned_addr learned my addr 172.21.15.112:0/3378036141 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:01.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f296259c700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f296406dcc0 0x7f29641592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f2962d9d700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2964143ba0 0x7f296415cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f2961d9b700 1 -- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2964143ba0 msgr2=0x7f296415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f2961d9b700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2964143ba0 0x7f296415cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f2961d9b700 1 -- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f296406dcc0 msgr2=0x7f29641592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f2961d9b700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f296406dcc0 0x7f29641592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f2961d9b700 1 -- 172.21.15.112:0/3378036141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c009a70 con 0x7f29641437c0 2023-08-27T11:29:01.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.213+0000 7f296259c700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f296406dcc0 0x7f29641592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:01.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.214+0000 7f2961d9b700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29641437c0 0x7f2964070650 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f2954001eb0 tx=0x7f295400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:01.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.214+0000 7f2952ffd700 1 -- 172.21.15.112:0/3378036141 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954005d70 con 0x7f29641437c0 2023-08-27T11:29:01.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.214+0000 7f2952ffd700 1 -- 172.21.15.112:0/3378036141 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29540070d0 con 0x7f29641437c0 2023-08-27T11:29:01.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.214+0000 7f296893c700 1 -- 172.21.15.112:0/3378036141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2964070fc0 con 0x7f29641437c0 2023-08-27T11:29:01.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.214+0000 7f2952ffd700 1 -- 172.21.15.112:0/3378036141 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954007460 con 0x7f29641437c0 2023-08-27T11:29:01.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.214+0000 7f296893c700 1 -- 172.21.15.112:0/3378036141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2964138f90 con 0x7f29641437c0 2023-08-27T11:29:01.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.215+0000 7f296893c700 1 -- 172.21.15.112:0/3378036141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2964067e90 con 0x7f29641437c0 2023-08-27T11:29:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.216+0000 7f2952ffd700 1 -- 172.21.15.112:0/3378036141 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f29540075c0 con 0x7f29641437c0 2023-08-27T11:29:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.216+0000 7f2952ffd700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f294406a4d0 0x7f294406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.216+0000 7f2952ffd700 1 -- 172.21.15.112:0/3378036141 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f2954088680 con 0x7f29641437c0 2023-08-27T11:29:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.216+0000 7f296259c700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f294406a4d0 0x7f294406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.217+0000 7f296259c700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f294406a4d0 0x7f294406c990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f29640cc8a0 tx=0x7f2958009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.220+0000 7f2952ffd700 1 -- 172.21.15.112:0/3378036141 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f29540537c0 con 0x7f29641437c0 2023-08-27T11:29:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.410+0000 7f296893c700 1 -- 172.21.15.112:0/3378036141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2964067a20 con 0x7f29641437c0 2023-08-27T11:29:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.410+0000 7f2952ffd700 1 -- 172.21.15.112:0/3378036141 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v155) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2954056de0 con 0x7f29641437c0 2023-08-27T11:29:01.412 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:29:01.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.412+0000 7f296893c700 1 -- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f294406a4d0 msgr2=0x7f294406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.412+0000 7f296893c700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f294406a4d0 0x7f294406c990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f29640cc8a0 tx=0x7f2958009040 comp rx=0 tx=0).stop 2023-08-27T11:29:01.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.412+0000 7f296893c700 1 -- 172.21.15.112:0/3378036141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29641437c0 msgr2=0x7f2964070650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.412+0000 7f296893c700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29641437c0 0x7f2964070650 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f2954001eb0 tx=0x7f295400c860 comp rx=0 tx=0).stop 2023-08-27T11:29:01.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.412+0000 7f296893c700 1 -- 172.21.15.112:0/3378036141 shutdown_connections 2023-08-27T11:29:01.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.412+0000 7f296893c700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f294406a4d0 0x7f294406c990 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.412+0000 7f296893c700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2964143ba0 0x7f296415cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.412+0000 7f296893c700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f296406dcc0 0x7f29641592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.412+0000 7f296893c700 1 --2- 172.21.15.112:0/3378036141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29641437c0 0x7f2964070650 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.412+0000 7f296893c700 1 -- 172.21.15.112:0/3378036141 >> 172.21.15.112:0/3378036141 conn(0x7f29640b1e90 msgr2=0x7f296413d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.412+0000 7f296893c700 1 -- 172.21.15.112:0/3378036141 shutdown_connections 2023-08-27T11:29:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.412+0000 7f296893c700 1 -- 172.21.15.112:0/3378036141 wait complete. 2023-08-27T11:29:01.426 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:29:01.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.548+0000 7f692b0de700 1 Processor -- start 2023-08-27T11:29:01.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.548+0000 7f692b0de700 1 -- start start 2023-08-27T11:29:01.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.549+0000 7f692b0de700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6924146c20 0x7f6924160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.549+0000 7f692b0de700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69240b3440 0x7f692415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.549+0000 7f692b0de700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f692415e6e0 0x7f692406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.549+0000 7f692b0de700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69240b5070 con 0x7f692415e6e0 2023-08-27T11:29:01.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.549+0000 7f692b0de700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6924147000 con 0x7f6924146c20 2023-08-27T11:29:01.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.549+0000 7f692b0de700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69241614a0 con 0x7f69240b3440 2023-08-27T11:29:01.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.549+0000 7f692967b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f692415e6e0 0x7f692406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.549+0000 7f692967b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f692415e6e0 0x7f692406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37358/0 (socket says 172.21.15.112:37358) 2023-08-27T11:29:01.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.549+0000 7f692967b700 1 -- 172.21.15.112:0/2769387598 learned_addr learned my addr 172.21.15.112:0/2769387598 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:01.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.549+0000 7f6923fff700 1 --2- 172.21.15.112:0/2769387598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69240b3440 0x7f692415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.549+0000 7f6928e7a700 1 --2- 172.21.15.112:0/2769387598 >> v2:172.21.15.138:3300/0 conn(0x7f6924146c20 0x7f6924160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692967b700 1 -- 172.21.15.112:0/2769387598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69240b3440 msgr2=0x7f692415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:01.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692967b700 1 --2- 172.21.15.112:0/2769387598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69240b3440 0x7f692415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692967b700 1 -- 172.21.15.112:0/2769387598 >> v2:172.21.15.138:3300/0 conn(0x7f6924146c20 msgr2=0x7f6924160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692967b700 1 --2- 172.21.15.112:0/2769387598 >> v2:172.21.15.138:3300/0 conn(0x7f6924146c20 0x7f6924160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692967b700 1 -- 172.21.15.112:0/2769387598 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6924161740 con 0x7f692415e6e0 2023-08-27T11:29:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692967b700 1 --2- 172.21.15.112:0/2769387598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692415e6e0 0x7f692406f790 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f69240b2d00 tx=0x7f691c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2c258b65fa9295 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f69237fe700 1 -- 172.21.15.112:0/2769387598 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c00e070 con 0x7f692415e6e0 2023-08-27T11:29:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f69237fe700 1 -- 172.21.15.112:0/2769387598 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c00b9c0 con 0x7f692415e6e0 2023-08-27T11:29:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f69237fe700 1 -- 172.21.15.112:0/2769387598 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c00bd70 con 0x7f692415e6e0 2023-08-27T11:29:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692b0de700 1 -- 172.21.15.112:0/2769387598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692415e6e0 msgr2=0x7f692406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692b0de700 1 --2- 172.21.15.112:0/2769387598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692415e6e0 0x7f692406f790 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f69240b2d00 tx=0x7f691c00b330 comp rx=0 tx=0).stop 2023-08-27T11:29:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692b0de700 1 -- 172.21.15.112:0/2769387598 shutdown_connections 2023-08-27T11:29:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692b0de700 1 --2- 172.21.15.112:0/2769387598 >> v2:172.21.15.138:3300/0 conn(0x7f6924146c20 0x7f6924160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692b0de700 1 --2- 172.21.15.112:0/2769387598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692415e6e0 0x7f692406f790 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692b0de700 1 --2- 172.21.15.112:0/2769387598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69240b3440 0x7f692415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692b0de700 1 -- 172.21.15.112:0/2769387598 >> 172.21.15.112:0/2769387598 conn(0x7f69240b1e90 msgr2=0x7f69240b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692b0de700 1 -- 172.21.15.112:0/2769387598 shutdown_connections 2023-08-27T11:29:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.550+0000 7f692b0de700 1 -- 172.21.15.112:0/2769387598 wait complete. 2023-08-27T11:29:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.552+0000 7f692b0de700 1 Processor -- start 2023-08-27T11:29:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.552+0000 7f692b0de700 1 -- start start 2023-08-27T11:29:01.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.552+0000 7f692b0de700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69240b3440 0x7f6924141040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f692b0de700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6924146c20 0x7f6924133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f692b0de700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f692415e6e0 0x7f69240aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f692b0de700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6924140ce0 con 0x7f6924146c20 2023-08-27T11:29:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f692b0de700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6924141580 con 0x7f69240b3440 2023-08-27T11:29:01.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f692b0de700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69241344b0 con 0x7f692415e6e0 2023-08-27T11:29:01.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f6923fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6924146c20 0x7f6924133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f6923fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6924146c20 0x7f6924133f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37362/0 (socket says 172.21.15.112:37362) 2023-08-27T11:29:01.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f6923fff700 1 -- 172.21.15.112:0/3776368280 learned_addr learned my addr 172.21.15.112:0/3776368280 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:01.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f692967b700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f692415e6e0 0x7f69240aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f6928e7a700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69240b3440 0x7f6924141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f6923fff700 1 -- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f692415e6e0 msgr2=0x7f69240aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f6923fff700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f692415e6e0 0x7f69240aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f6923fff700 1 -- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69240b3440 msgr2=0x7f6924141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f6923fff700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69240b3440 0x7f6924141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f6923fff700 1 -- 172.21.15.112:0/3776368280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c009b30 con 0x7f6924146c20 2023-08-27T11:29:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f6923fff700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6924146c20 0x7f6924133f70 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f691000d800 tx=0x7f691000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f69217fa700 1 -- 172.21.15.112:0/3776368280 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69100125f0 con 0x7f6924146c20 2023-08-27T11:29:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f69217fa700 1 -- 172.21.15.112:0/3776368280 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6910012750 con 0x7f6924146c20 2023-08-27T11:29:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f692b0de700 1 -- 172.21.15.112:0/3776368280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6924140a00 con 0x7f6924146c20 2023-08-27T11:29:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f69217fa700 1 -- 172.21.15.112:0/3776368280 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6910012ae0 con 0x7f6924146c20 2023-08-27T11:29:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.553+0000 7f692b0de700 1 -- 172.21.15.112:0/3776368280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6924144620 con 0x7f6924146c20 2023-08-27T11:29:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.554+0000 7f692b0de700 1 -- 172.21.15.112:0/3776368280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6924067e90 con 0x7f6924146c20 2023-08-27T11:29:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.555+0000 7f69217fa700 1 -- 172.21.15.112:0/3776368280 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f6910012c40 con 0x7f6924146c20 2023-08-27T11:29:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.555+0000 7f69217fa700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f690806a4d0 0x7f690806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.555+0000 7f69217fa700 1 -- 172.21.15.112:0/3776368280 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f6910088c40 con 0x7f6924146c20 2023-08-27T11:29:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.555+0000 7f6928e7a700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f690806a4d0 0x7f690806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.556+0000 7f6928e7a700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f690806a4d0 0x7f690806c990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6918006b20 tx=0x7f6918008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:01.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.559+0000 7f69217fa700 1 -- 172.21.15.112:0/3776368280 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6910053db0 con 0x7f6924146c20 2023-08-27T11:29:01.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.750+0000 7f692b0de700 1 -- 172.21.15.112:0/3776368280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f69241619f0 con 0x7f6924146c20 2023-08-27T11:29:01.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.750+0000 7f69217fa700 1 -- 172.21.15.112:0/3776368280 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v155) v1 ==== 93+0+10 (secure 0 0 0) 0x7f69100573d0 con 0x7f6924146c20 2023-08-27T11:29:01.752 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:29:01.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.753+0000 7f692b0de700 1 -- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f690806a4d0 msgr2=0x7f690806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.753+0000 7f692b0de700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f690806a4d0 0x7f690806c990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6918006b20 tx=0x7f6918008040 comp rx=0 tx=0).stop 2023-08-27T11:29:01.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.753+0000 7f692b0de700 1 -- 172.21.15.112:0/3776368280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6924146c20 msgr2=0x7f6924133f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.753+0000 7f692b0de700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6924146c20 0x7f6924133f70 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f691000d800 tx=0x7f691000db10 comp rx=0 tx=0).stop 2023-08-27T11:29:01.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.753+0000 7f692b0de700 1 -- 172.21.15.112:0/3776368280 shutdown_connections 2023-08-27T11:29:01.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.753+0000 7f692b0de700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f690806a4d0 0x7f690806c990 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.753+0000 7f692b0de700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f692415e6e0 0x7f69240aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.753+0000 7f692b0de700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69240b3440 0x7f6924141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.753+0000 7f692b0de700 1 --2- 172.21.15.112:0/3776368280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6924146c20 0x7f6924133f70 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.753+0000 7f692b0de700 1 -- 172.21.15.112:0/3776368280 >> 172.21.15.112:0/3776368280 conn(0x7f69240b1e90 msgr2=0x7f692406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:01.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.753+0000 7f692b0de700 1 -- 172.21.15.112:0/3776368280 shutdown_connections 2023-08-27T11:29:01.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.753+0000 7f692b0de700 1 -- 172.21.15.112:0/3776368280 wait complete. 2023-08-27T11:29:01.766 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:29:01.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.887+0000 7fa22f2a9700 1 Processor -- start 2023-08-27T11:29:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.887+0000 7fa22f2a9700 1 -- start start 2023-08-27T11:29:01.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa22f2a9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2280ce030 0x7fa2281683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa22f2a9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2280cf0f0 con 0x7fa2280d25c0 2023-08-27T11:29:01.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa22f2a9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2280ce570 con 0x7fa228157b80 2023-08-27T11:29:01.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa22f2a9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22815ddd0 con 0x7fa2280ce030 2023-08-27T11:29:01.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa22c844700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa2280d25c0 0x7fa2280cd920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59982/0 (socket says 172.21.15.112:59982) 2023-08-27T11:29:01.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa22c844700 1 -- 172.21.15.112:0/26419076 learned_addr learned my addr 172.21.15.112:0/26419076 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:01.893 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa22d846700 1 --2- 172.21.15.112:0/26419076 >> v2:172.21.15.138:3300/0 conn(0x7fa2280ce030 0x7fa2281683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496381272 0 0) 0x7fa2280cf0f0 con 0x7fa2280d25c0 2023-08-27T11:29:01.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2140035b0 con 0x7fa2280d25c0 2023-08-27T11:29:01.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3661913739 0 0) 0x7fa2140035b0 con 0x7fa2280d25c0 2023-08-27T11:29:01.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 >> v2:172.21.15.138:3300/0 conn(0x7fa2280ce030 msgr2=0x7fa2281683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:01.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa21ffff700 1 --2- 172.21.15.112:0/26419076 >> v2:172.21.15.138:3300/0 conn(0x7fa2280ce030 0x7fa2281683b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 >> v1:172.21.15.138:6789/0 conn(0x7fa228157b80 legacy=0x7fa2280d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2280ce850 con 0x7fa2280d25c0 2023-08-27T11:29:01.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.888+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fa218001670 con 0x7fa2280d25c0 2023-08-27T11:29:01.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 >> v1:172.21.15.112:6789/0 conn(0x7fa2280d25c0 legacy=0x7fa2280cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa21ffff700 1 --2- 172.21.15.112:0/26419076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa214003a00 0x7fa214003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa21ffff700 1 --2- 172.21.15.112:0/26419076 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa214004850 0x7fa214004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa21ffff700 1 --2- 172.21.15.112:0/26419076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa214005210 0x7fa21400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22815ddd0 con 0x7fa214003a00 2023-08-27T11:29:01.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21400c0b0 con 0x7fa214005210 2023-08-27T11:29:01.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21400c270 con 0x7fa214004850 2023-08-27T11:29:01.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa22d045700 1 --2- 172.21.15.112:0/26419076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa214003a00 0x7fa214003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa22d846700 1 --2- 172.21.15.112:0/26419076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa214005210 0x7fa21400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa22c844700 1 --2- 172.21.15.112:0/26419076 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa214004850 0x7fa214004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa22d045700 1 -- 172.21.15.112:0/26419076 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa214004850 msgr2=0x7fa214004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:01.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa22d045700 1 --2- 172.21.15.112:0/26419076 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa214004850 0x7fa214004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa22d045700 1 -- 172.21.15.112:0/26419076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa214005210 msgr2=0x7fa21400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:01.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa22d045700 1 --2- 172.21.15.112:0/26419076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa214005210 0x7fa21400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa22d846700 1 -- 172.21.15.112:0/26419076 reap_dead start 2023-08-27T11:29:01.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa22d045700 1 -- 172.21.15.112:0/26419076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2140035b0 con 0x7fa214003a00 2023-08-27T11:29:01.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa22d045700 1 --2- 172.21.15.112:0/26419076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa214003a00 0x7fa214003e20 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fa2240017c0 tx=0x7fa214005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50eb799463497c07 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:01.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240052d0 con 0x7fa214003a00 2023-08-27T11:29:01.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa224005450 con 0x7fa214003a00 2023-08-27T11:29:01.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.889+0000 7fa21ffff700 1 -- 172.21.15.112:0/26419076 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240038c0 con 0x7fa214003a00 2023-08-27T11:29:01.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.890+0000 7fa22f2a9700 1 -- 172.21.15.112:0/26419076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa214003a00 msgr2=0x7fa214003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.890+0000 7fa22f2a9700 1 --2- 172.21.15.112:0/26419076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa214003a00 0x7fa214003e20 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fa2240017c0 tx=0x7fa214005880 comp rx=0 tx=0).stop 2023-08-27T11:29:01.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.890+0000 7fa22f2a9700 1 -- 172.21.15.112:0/26419076 shutdown_connections 2023-08-27T11:29:01.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.890+0000 7fa22f2a9700 1 --2- 172.21.15.112:0/26419076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa214003a00 0x7fa214003e20 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.890+0000 7fa22f2a9700 1 -- 172.21.15.112:0/26419076 >> 172.21.15.112:0/26419076 conn(0x7fa2280cfd50 msgr2=0x7fa2280d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:01.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.890+0000 7fa22f2a9700 1 -- 172.21.15.112:0/26419076 shutdown_connections 2023-08-27T11:29:01.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.890+0000 7fa22f2a9700 1 -- 172.21.15.112:0/26419076 wait complete. 2023-08-27T11:29:01.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.891+0000 7fa22f2a9700 1 Processor -- start 2023-08-27T11:29:01.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.891+0000 7fa22f2a9700 1 -- start start 2023-08-27T11:29:01.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22f2a9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa214003a00 0x7fa22814ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22f2a9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa22815cec0 0x7fa2281f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22f2a9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2281f61b0 0x7fa2280c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22f2a9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa228153700 con 0x7fa214003a00 2023-08-27T11:29:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22f2a9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa228153340 con 0x7fa22815cec0 2023-08-27T11:29:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22f2a9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2281f6630 con 0x7fa2281f61b0 2023-08-27T11:29:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22d045700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa214003a00 0x7fa22814ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22d045700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa214003a00 0x7fa22814ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37382/0 (socket says 172.21.15.112:37382) 2023-08-27T11:29:01.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22d045700 1 -- 172.21.15.112:0/1835240736 learned_addr learned my addr 172.21.15.112:0/1835240736 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:01.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22d846700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2281f61b0 0x7fa2280c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22c844700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa22815cec0 0x7fa2281f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22d045700 1 -- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2281f61b0 msgr2=0x7fa2280c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22d045700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2281f61b0 0x7fa2280c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22d045700 1 -- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa22815cec0 msgr2=0x7fa2281f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22d045700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa22815cec0 0x7fa2281f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22d045700 1 -- 172.21.15.112:0/1835240736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2281f6910 con 0x7fa214003a00 2023-08-27T11:29:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22d846700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2281f61b0 0x7fa2280c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22c844700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa22815cec0 0x7fa2281f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.892+0000 7fa22d045700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa214003a00 0x7fa22814ed90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa224000c00 tx=0x7fa224001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.893+0000 7fa21dffb700 1 -- 172.21.15.112:0/1835240736 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224001290 con 0x7fa214003a00 2023-08-27T11:29:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.893+0000 7fa21dffb700 1 -- 172.21.15.112:0/1835240736 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa224007040 con 0x7fa214003a00 2023-08-27T11:29:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.893+0000 7fa22f2a9700 1 -- 172.21.15.112:0/1835240736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22814f2d0 con 0x7fa214003a00 2023-08-27T11:29:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.893+0000 7fa21dffb700 1 -- 172.21.15.112:0/1835240736 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240073f0 con 0x7fa214003a00 2023-08-27T11:29:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.893+0000 7fa22f2a9700 1 -- 172.21.15.112:0/1835240736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2280c8c50 con 0x7fa214003a00 2023-08-27T11:29:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.894+0000 7fa22f2a9700 1 -- 172.21.15.112:0/1835240736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa228067e90 con 0x7fa214003a00 2023-08-27T11:29:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.894+0000 7fa21dffb700 1 -- 172.21.15.112:0/1835240736 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa224007550 con 0x7fa214003a00 2023-08-27T11:29:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.894+0000 7fa21dffb700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa20c06a4d0 0x7fa20c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.895+0000 7fa21dffb700 1 -- 172.21.15.112:0/1835240736 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fa2240895f0 con 0x7fa214003a00 2023-08-27T11:29:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.895+0000 7fa22c844700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa20c06a4d0 0x7fa20c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:01.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.895+0000 7fa22c844700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa20c06a4d0 0x7fa20c06c990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa22815caf0 tx=0x7fa218009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:01.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:01.899+0000 7fa21dffb700 1 -- 172.21.15.112:0/1835240736 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa224054730 con 0x7fa214003a00 2023-08-27T11:29:02.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.086+0000 7fa22f2a9700 1 -- 172.21.15.112:0/1835240736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa228067a20 con 0x7fa214003a00 2023-08-27T11:29:02.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.087+0000 7fa21dffb700 1 -- 172.21.15.112:0/1835240736 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v155) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa224057d50 con 0x7fa214003a00 2023-08-27T11:29:02.089 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:29:02.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.089+0000 7fa22f2a9700 1 -- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa20c06a4d0 msgr2=0x7fa20c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.089+0000 7fa22f2a9700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa20c06a4d0 0x7fa20c06c990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa22815caf0 tx=0x7fa218009040 comp rx=0 tx=0).stop 2023-08-27T11:29:02.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.089+0000 7fa22f2a9700 1 -- 172.21.15.112:0/1835240736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa214003a00 msgr2=0x7fa22814ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.089+0000 7fa22f2a9700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa214003a00 0x7fa22814ed90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa224000c00 tx=0x7fa224001120 comp rx=0 tx=0).stop 2023-08-27T11:29:02.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.089+0000 7fa22f2a9700 1 -- 172.21.15.112:0/1835240736 shutdown_connections 2023-08-27T11:29:02.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.089+0000 7fa22f2a9700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa20c06a4d0 0x7fa20c06c990 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.089+0000 7fa22f2a9700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2281f61b0 0x7fa2280c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.089+0000 7fa22f2a9700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa214003a00 0x7fa22814ed90 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.089+0000 7fa22f2a9700 1 --2- 172.21.15.112:0/1835240736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa22815cec0 0x7fa2281f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.089+0000 7fa22f2a9700 1 -- 172.21.15.112:0/1835240736 >> 172.21.15.112:0/1835240736 conn(0x7fa2280cfd50 msgr2=0x7fa2280cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:02.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.089+0000 7fa22f2a9700 1 -- 172.21.15.112:0/1835240736 shutdown_connections 2023-08-27T11:29:02.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.089+0000 7fa22f2a9700 1 -- 172.21.15.112:0/1835240736 wait complete. 2023-08-27T11:29:02.102 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:29:02.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.224+0000 7f6b65b1c700 1 Processor -- start 2023-08-27T11:29:02.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.224+0000 7f6b65b1c700 1 -- start start 2023-08-27T11:29:02.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.225+0000 7f6b65b1c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b60146c20 0x7f6b60160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.225+0000 7f6b65b1c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b600b5070 con 0x7f6b600b3440 2023-08-27T11:29:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.225+0000 7f6b65b1c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b60147000 con 0x7f6b6006f790 2023-08-27T11:29:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.225+0000 7f6b65b1c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b601614a0 con 0x7f6b60146c20 2023-08-27T11:29:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.225+0000 7f6b5effd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6b600b3440 0x7f6b6015e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60004/0 (socket says 172.21.15.112:60004) 2023-08-27T11:29:02.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.225+0000 7f6b5effd700 1 -- 172.21.15.112:0/1986055205 learned_addr learned my addr 172.21.15.112:0/1986055205 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:02.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.225+0000 7f6b5f7fe700 1 --2- 172.21.15.112:0/1986055205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b60146c20 0x7f6b60160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.225+0000 7f6b5e7fc700 1 -- 172.21.15.112:0/1986055205 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213075292 0 0) 0x7f6b50001110 con 0x7f6b600b3440 2023-08-27T11:29:02.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.225+0000 7f6b5e7fc700 1 -- 172.21.15.112:0/1986055205 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b4c0035b0 con 0x7f6b600b3440 2023-08-27T11:29:02.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.226+0000 7f6b5e7fc700 1 -- 172.21.15.112:0/1986055205 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998232536 0 0) 0x7f6b60147000 con 0x7f6b6006f790 2023-08-27T11:29:02.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.226+0000 7f6b5e7fc700 1 -- 172.21.15.112:0/1986055205 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b50001110 con 0x7f6b6006f790 2023-08-27T11:29:02.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.226+0000 7f6b5f7fe700 1 -- 172.21.15.112:0/1986055205 >> v1:172.21.15.138:6789/0 conn(0x7f6b6006f790 legacy=0x7f6b600b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.226+0000 7f6b5f7fe700 1 -- 172.21.15.112:0/1986055205 >> v1:172.21.15.112:6789/0 conn(0x7f6b600b3440 legacy=0x7f6b6015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.226+0000 7f6b5f7fe700 1 -- 172.21.15.112:0/1986055205 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b60161740 con 0x7f6b60146c20 2023-08-27T11:29:02.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.226+0000 7f6b5f7fe700 1 --2- 172.21.15.112:0/1986055205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b60146c20 0x7f6b60160f60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009870 tx=0x7f6b48009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5caceecec8325a7f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:02.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.226+0000 7f6b5e7fc700 1 -- 172.21.15.112:0/1986055205 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48011830 con 0x7f6b60146c20 2023-08-27T11:29:02.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.226+0000 7f6b5e7fc700 1 -- 172.21.15.112:0/1986055205 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b48011990 con 0x7f6b60146c20 2023-08-27T11:29:02.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.226+0000 7f6b5e7fc700 1 -- 172.21.15.112:0/1986055205 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48011d20 con 0x7f6b60146c20 2023-08-27T11:29:02.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.226+0000 7f6b65b1c700 1 -- 172.21.15.112:0/1986055205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b60146c20 msgr2=0x7f6b60160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.226+0000 7f6b65b1c700 1 --2- 172.21.15.112:0/1986055205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b60146c20 0x7f6b60160f60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009870 tx=0x7f6b48009b80 comp rx=0 tx=0).stop 2023-08-27T11:29:02.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.227+0000 7f6b65b1c700 1 -- 172.21.15.112:0/1986055205 shutdown_connections 2023-08-27T11:29:02.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.227+0000 7f6b65b1c700 1 --2- 172.21.15.112:0/1986055205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b60146c20 0x7f6b60160f60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.227+0000 7f6b65b1c700 1 -- 172.21.15.112:0/1986055205 >> 172.21.15.112:0/1986055205 conn(0x7f6b600b1e90 msgr2=0x7f6b600b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:02.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.227+0000 7f6b65b1c700 1 -- 172.21.15.112:0/1986055205 shutdown_connections 2023-08-27T11:29:02.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.227+0000 7f6b65b1c700 1 -- 172.21.15.112:0/1986055205 wait complete. 2023-08-27T11:29:02.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.228+0000 7f6b65b1c700 1 Processor -- start 2023-08-27T11:29:02.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.228+0000 7f6b65b1c700 1 -- start start 2023-08-27T11:29:02.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.229+0000 7f6b65b1c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b6006f790 0x7f6b60141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.229+0000 7f6b65b1c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b600b3440 0x7f6b60142f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.229+0000 7f6b65b1c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b60146c20 0x7f6b600ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.229+0000 7f6b65b1c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b601445c0 con 0x7f6b600b3440 2023-08-27T11:29:02.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.229+0000 7f6b65b1c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b601439b0 con 0x7f6b60146c20 2023-08-27T11:29:02.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.229+0000 7f6b65b1c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b60143c90 con 0x7f6b6006f790 2023-08-27T11:29:02.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.229+0000 7f6b5effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b600b3440 0x7f6b60142f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.229+0000 7f6b5effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b600b3440 0x7f6b60142f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37396/0 (socket says 172.21.15.112:37396) 2023-08-27T11:29:02.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.229+0000 7f6b5effd700 1 -- 172.21.15.112:0/3301790122 learned_addr learned my addr 172.21.15.112:0/3301790122 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:02.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.230+0000 7f6b5f7fe700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b6006f790 0x7f6b60141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.230+0000 7f6b5ffff700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b60146c20 0x7f6b600ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.230+0000 7f6b5effd700 1 -- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b6006f790 msgr2=0x7f6b60141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.230+0000 7f6b5effd700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b6006f790 0x7f6b60141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.230+0000 7f6b5effd700 1 -- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b60146c20 msgr2=0x7f6b600ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.230+0000 7f6b5effd700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b60146c20 0x7f6b600ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.230+0000 7f6b5effd700 1 -- 172.21.15.112:0/3301790122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b480094a0 con 0x7f6b600b3440 2023-08-27T11:29:02.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.230+0000 7f6b5effd700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b600b3440 0x7f6b60142f00 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f6b50001eb0 tx=0x7f6b5000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:02.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.230+0000 7f6b57fff700 1 -- 172.21.15.112:0/3301790122 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50005d70 con 0x7f6b600b3440 2023-08-27T11:29:02.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.230+0000 7f6b57fff700 1 -- 172.21.15.112:0/3301790122 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b500070d0 con 0x7f6b600b3440 2023-08-27T11:29:02.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.230+0000 7f6b65b1c700 1 -- 172.21.15.112:0/3301790122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b60143f70 con 0x7f6b600b3440 2023-08-27T11:29:02.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.230+0000 7f6b57fff700 1 -- 172.21.15.112:0/3301790122 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50007460 con 0x7f6b600b3440 2023-08-27T11:29:02.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.230+0000 7f6b65b1c700 1 -- 172.21.15.112:0/3301790122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b60141670 con 0x7f6b600b3440 2023-08-27T11:29:02.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.231+0000 7f6b65b1c700 1 -- 172.21.15.112:0/3301790122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b601f26f0 con 0x7f6b600b3440 2023-08-27T11:29:02.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.231+0000 7f6b57fff700 1 -- 172.21.15.112:0/3301790122 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f6b500075c0 con 0x7f6b600b3440 2023-08-27T11:29:02.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.232+0000 7f6b57fff700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6b4006a4d0 0x7f6b4006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.232+0000 7f6b57fff700 1 -- 172.21.15.112:0/3301790122 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f6b500881c0 con 0x7f6b600b3440 2023-08-27T11:29:02.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.232+0000 7f6b5f7fe700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6b4006a4d0 0x7f6b4006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.233+0000 7f6b5f7fe700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6b4006a4d0 0x7f6b4006c990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6b480090b0 tx=0x7f6b4801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:02.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.236+0000 7f6b57fff700 1 -- 172.21.15.112:0/3301790122 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6b601f26f0 con 0x7f6b600b3440 2023-08-27T11:29:02.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.426+0000 7f6b65b1c700 1 -- 172.21.15.112:0/3301790122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b600b5070 con 0x7f6b600b3440 2023-08-27T11:29:02.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.426+0000 7f6b57fff700 1 -- 172.21.15.112:0/3301790122 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v155) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b600b5070 con 0x7f6b600b3440 2023-08-27T11:29:02.428 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:29:02.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.428+0000 7f6b65b1c700 1 -- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6b4006a4d0 msgr2=0x7f6b4006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.428+0000 7f6b65b1c700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6b4006a4d0 0x7f6b4006c990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6b480090b0 tx=0x7f6b4801c040 comp rx=0 tx=0).stop 2023-08-27T11:29:02.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.428+0000 7f6b65b1c700 1 -- 172.21.15.112:0/3301790122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b600b3440 msgr2=0x7f6b60142f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.428+0000 7f6b65b1c700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b600b3440 0x7f6b60142f00 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f6b50001eb0 tx=0x7f6b5000c9e0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.428+0000 7f6b65b1c700 1 -- 172.21.15.112:0/3301790122 shutdown_connections 2023-08-27T11:29:02.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.428+0000 7f6b65b1c700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6b4006a4d0 0x7f6b4006c990 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.428+0000 7f6b65b1c700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b60146c20 0x7f6b600ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.428+0000 7f6b65b1c700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b6006f790 0x7f6b60141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.428+0000 7f6b65b1c700 1 --2- 172.21.15.112:0/3301790122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b600b3440 0x7f6b60142f00 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.428+0000 7f6b65b1c700 1 -- 172.21.15.112:0/3301790122 >> 172.21.15.112:0/3301790122 conn(0x7f6b600b1e90 msgr2=0x7f6b600720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:02.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.428+0000 7f6b65b1c700 1 -- 172.21.15.112:0/3301790122 shutdown_connections 2023-08-27T11:29:02.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.428+0000 7f6b65b1c700 1 -- 172.21.15.112:0/3301790122 wait complete. 2023-08-27T11:29:02.442 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:29:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.561+0000 7fbf2712f700 1 Processor -- start 2023-08-27T11:29:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.561+0000 7fbf2712f700 1 -- start start 2023-08-27T11:29:02.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.562+0000 7fbf2712f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20141280 0x7fbf2006e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.562+0000 7fbf2712f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbf2015f6a0 0x7fbf2013e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.562+0000 7fbf2712f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2015fce0 con 0x7fbf2006e690 2023-08-27T11:29:02.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.562+0000 7fbf2712f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2006eb10 con 0x7fbf2015f6a0 2023-08-27T11:29:02.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.562+0000 7fbf2712f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2015b8f0 con 0x7fbf20141280 2023-08-27T11:29:02.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.562+0000 7fbf1ffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbf2006e690 0x7fbf2015ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60016/0 (socket says 172.21.15.112:60016) 2023-08-27T11:29:02.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.562+0000 7fbf1ffff700 1 -- 172.21.15.112:0/3660341850 learned_addr learned my addr 172.21.15.112:0/3660341850 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:02.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903092204 0 0) 0x7fbf2015fce0 con 0x7fbf2006e690 2023-08-27T11:29:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf256cc700 1 --2- 172.21.15.112:0/3660341850 >> v2:172.21.15.138:3300/0 conn(0x7fbf2015f6a0 0x7fbf2013e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf24ecb700 1 --2- 172.21.15.112:0/3660341850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20141280 0x7fbf2006e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf100035b0 con 0x7fbf2006e690 2023-08-27T11:29:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 851255384 0 0) 0x7fbf100035b0 con 0x7fbf2006e690 2023-08-27T11:29:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20141280 msgr2=0x7fbf2006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 --2- 172.21.15.112:0/3660341850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20141280 0x7fbf2006e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 >> v2:172.21.15.138:3300/0 conn(0x7fbf2015f6a0 msgr2=0x7fbf2013e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 --2- 172.21.15.112:0/3660341850 >> v2:172.21.15.138:3300/0 conn(0x7fbf2015f6a0 0x7fbf2013e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2015bbd0 con 0x7fbf2006e690 2023-08-27T11:29:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fbf0c001670 con 0x7fbf2006e690 2023-08-27T11:29:02.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 >> v1:172.21.15.112:6789/0 conn(0x7fbf2006e690 legacy=0x7fbf2015ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 --2- 172.21.15.112:0/3660341850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf10003a00 0x7fbf10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 --2- 172.21.15.112:0/3660341850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf10004850 0x7fbf10004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 --2- 172.21.15.112:0/3660341850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20141280 0x7fbf10005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2006eb10 con 0x7fbf10003a00 2023-08-27T11:29:02.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2015b8f0 con 0x7fbf10004850 2023-08-27T11:29:02.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1000ba10 con 0x7fbf20141280 2023-08-27T11:29:02.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf24ecb700 1 --2- 172.21.15.112:0/3660341850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf10003a00 0x7fbf10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf1ffff700 1 --2- 172.21.15.112:0/3660341850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf10004850 0x7fbf10004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf256cc700 1 --2- 172.21.15.112:0/3660341850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20141280 0x7fbf10005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf24ecb700 1 -- 172.21.15.112:0/3660341850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20141280 msgr2=0x7fbf10005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:02.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf24ecb700 1 --2- 172.21.15.112:0/3660341850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20141280 0x7fbf10005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf24ecb700 1 -- 172.21.15.112:0/3660341850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf10004850 msgr2=0x7fbf10004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:02.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf24ecb700 1 --2- 172.21.15.112:0/3660341850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf10004850 0x7fbf10004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf24ecb700 1 -- 172.21.15.112:0/3660341850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100035b0 con 0x7fbf10003a00 2023-08-27T11:29:02.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.563+0000 7fbf256cc700 1 --2- 172.21.15.112:0/3660341850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf20141280 0x7fbf10005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:02.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.564+0000 7fbf24ecb700 1 --2- 172.21.15.112:0/3660341850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf10003a00 0x7fbf10003e20 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009b60 tx=0x7fbf10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f4907cb2a432f5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:02.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.564+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400e6c0 con 0x7fbf10003a00 2023-08-27T11:29:02.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.564+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1400e820 con 0x7fbf10003a00 2023-08-27T11:29:02.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.564+0000 7fbf1f7fe700 1 -- 172.21.15.112:0/3660341850 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400eb90 con 0x7fbf10003a00 2023-08-27T11:29:02.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.564+0000 7fbf2712f700 1 -- 172.21.15.112:0/3660341850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf10003a00 msgr2=0x7fbf10003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.564+0000 7fbf2712f700 1 --2- 172.21.15.112:0/3660341850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf10003a00 0x7fbf10003e20 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009b60 tx=0x7fbf10005880 comp rx=0 tx=0).stop 2023-08-27T11:29:02.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.564+0000 7fbf256cc700 1 -- 172.21.15.112:0/3660341850 reap_dead start 2023-08-27T11:29:02.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.564+0000 7fbf2712f700 1 -- 172.21.15.112:0/3660341850 shutdown_connections 2023-08-27T11:29:02.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.564+0000 7fbf2712f700 1 -- 172.21.15.112:0/3660341850 >> 172.21.15.112:0/3660341850 conn(0x7fbf2006cd30 msgr2=0x7fbf200717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:02.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.564+0000 7fbf2712f700 1 -- 172.21.15.112:0/3660341850 shutdown_connections 2023-08-27T11:29:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.564+0000 7fbf2712f700 1 -- 172.21.15.112:0/3660341850 wait complete. 2023-08-27T11:29:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.565+0000 7fbf2712f700 1 Processor -- start 2023-08-27T11:29:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.566+0000 7fbf2712f700 1 -- start start 2023-08-27T11:29:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.566+0000 7fbf2712f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf10003a00 0x7fbf200aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.566+0000 7fbf2712f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf200af630 0x7fbf200b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.566+0000 7fbf2712f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf200b1530 0x7fbf20132590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.566+0000 7fbf2712f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200b19b0 con 0x7fbf10003a00 2023-08-27T11:29:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.566+0000 7fbf2712f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf20131300 con 0x7fbf200b1530 2023-08-27T11:29:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.566+0000 7fbf2712f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf201314f0 con 0x7fbf200af630 2023-08-27T11:29:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.566+0000 7fbf24ecb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf10003a00 0x7fbf200aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.566+0000 7fbf24ecb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf10003a00 0x7fbf200aca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37412/0 (socket says 172.21.15.112:37412) 2023-08-27T11:29:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.566+0000 7fbf24ecb700 1 -- 172.21.15.112:0/1912962750 learned_addr learned my addr 172.21.15.112:0/1912962750 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.567+0000 7fbf24ecb700 1 -- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf200af630 msgr2=0x7fbf200b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.567+0000 7fbf1ffff700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf200af630 0x7fbf200b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.567+0000 7fbf24ecb700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf200af630 0x7fbf200b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.567+0000 7fbf256cc700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf200b1530 0x7fbf20132590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.567+0000 7fbf24ecb700 1 -- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf200b1530 msgr2=0x7fbf20132590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.567+0000 7fbf24ecb700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf200b1530 0x7fbf20132590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.567+0000 7fbf24ecb700 1 -- 172.21.15.112:0/1912962750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf201317d0 con 0x7fbf10003a00 2023-08-27T11:29:02.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.567+0000 7fbf24ecb700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf10003a00 0x7fbf200aca60 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fbf100046a0 tx=0x7fbf140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:02.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.567+0000 7fbf1d7fa700 1 -- 172.21.15.112:0/1912962750 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14021070 con 0x7fbf10003a00 2023-08-27T11:29:02.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.567+0000 7fbf1d7fa700 1 -- 172.21.15.112:0/1912962750 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf140045d0 con 0x7fbf10003a00 2023-08-27T11:29:02.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.567+0000 7fbf2712f700 1 -- 172.21.15.112:0/1912962750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf20131ab0 con 0x7fbf10003a00 2023-08-27T11:29:02.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.567+0000 7fbf1d7fa700 1 -- 172.21.15.112:0/1912962750 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400e6c0 con 0x7fbf10003a00 2023-08-27T11:29:02.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.567+0000 7fbf2712f700 1 -- 172.21.15.112:0/1912962750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf20132070 con 0x7fbf10003a00 2023-08-27T11:29:02.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.568+0000 7fbf2712f700 1 -- 172.21.15.112:0/1912962750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf20132230 con 0x7fbf10003a00 2023-08-27T11:29:02.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.568+0000 7fbf1d7fa700 1 -- 172.21.15.112:0/1912962750 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fbf14025070 con 0x7fbf10003a00 2023-08-27T11:29:02.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.569+0000 7fbf1d7fa700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf0406a4d0 0x7fbf0406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.569+0000 7fbf1d7fa700 1 -- 172.21.15.112:0/1912962750 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fbf1408a060 con 0x7fbf10003a00 2023-08-27T11:29:02.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.569+0000 7fbf1ffff700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf0406a4d0 0x7fbf0406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.570+0000 7fbf1ffff700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf0406a4d0 0x7fbf0406c990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c000fb0 tx=0x7fbf0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:02.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.573+0000 7fbf1d7fa700 1 -- 172.21.15.112:0/1912962750 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbf140bc990 con 0x7fbf10003a00 2023-08-27T11:29:02.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.776+0000 7fbf2712f700 1 -- 172.21.15.112:0/1912962750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbf20067a20 con 0x7fbf10003a00 2023-08-27T11:29:02.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.777+0000 7fbf1d7fa700 1 -- 172.21.15.112:0/1912962750 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v155) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbf140552c0 con 0x7fbf10003a00 2023-08-27T11:29:02.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.779+0000 7fbf2712f700 1 -- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf0406a4d0 msgr2=0x7fbf0406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.779+0000 7fbf2712f700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf0406a4d0 0x7fbf0406c990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c000fb0 tx=0x7fbf0c009040 comp rx=0 tx=0).stop 2023-08-27T11:29:02.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.779+0000 7fbf2712f700 1 -- 172.21.15.112:0/1912962750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf10003a00 msgr2=0x7fbf200aca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.779+0000 7fbf2712f700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf10003a00 0x7fbf200aca60 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fbf100046a0 tx=0x7fbf140047c0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.779+0000 7fbf2712f700 1 -- 172.21.15.112:0/1912962750 shutdown_connections 2023-08-27T11:29:02.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.779+0000 7fbf2712f700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbf0406a4d0 0x7fbf0406c990 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.779+0000 7fbf2712f700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf200b1530 0x7fbf20132590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.779+0000 7fbf2712f700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf10003a00 0x7fbf200aca60 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.779+0000 7fbf2712f700 1 --2- 172.21.15.112:0/1912962750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf200af630 0x7fbf200b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.779+0000 7fbf2712f700 1 -- 172.21.15.112:0/1912962750 >> 172.21.15.112:0/1912962750 conn(0x7fbf2006cd30 msgr2=0x7fbf200a8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:02.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.779+0000 7fbf2712f700 1 -- 172.21.15.112:0/1912962750 shutdown_connections 2023-08-27T11:29:02.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:02.779+0000 7fbf2712f700 1 -- 172.21.15.112:0/1912962750 wait complete. 2023-08-27T11:29:02.783 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:29:02.792 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:29:02.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:02.793 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:29:02.846 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:29:02.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:02.847 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:29:02.864 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:29:02.864 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:02.886 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:29:02.886 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:02.946 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:03.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.064+0000 7fac3fab5700 1 Processor -- start 2023-08-27T11:29:03.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.065+0000 7fac3fab5700 1 -- start start 2023-08-27T11:29:03.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.065+0000 7fac3fab5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac380b1400 0x7fac380cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.065+0000 7fac3fab5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac380b17e0 0x7fac3806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.065+0000 7fac3fab5700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac38161850 con 0x7fac38161ce0 2023-08-27T11:29:03.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.065+0000 7fac3fab5700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac3806e7d0 con 0x7fac380b17e0 2023-08-27T11:29:03.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.065+0000 7fac3fab5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac380cfc10 con 0x7fac380b1400 2023-08-27T11:29:03.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.065+0000 7fac3d050700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac380b17e0 0x7fac3806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.065+0000 7fac3d050700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac380b17e0 0x7fac3806e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37422/0 (socket says 172.21.15.112:37422) 2023-08-27T11:29:03.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.065+0000 7fac3d050700 1 -- 172.21.15.112:0/1700706718 learned_addr learned my addr 172.21.15.112:0/1700706718 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:03.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.066+0000 7fac3d050700 1 -- 172.21.15.112:0/1700706718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac380b1400 msgr2=0x7fac380cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:03.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.066+0000 7fac3d050700 1 --2- 172.21.15.112:0/1700706718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac380b1400 0x7fac380cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.066+0000 7fac3d050700 1 -- 172.21.15.112:0/1700706718 >> v1:172.21.15.138:6789/0 conn(0x7fac38161ce0 legacy=0x7fac3806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:03.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.066+0000 7fac3d050700 1 -- 172.21.15.112:0/1700706718 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac38161200 con 0x7fac380b17e0 2023-08-27T11:29:03.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.066+0000 7fac3d050700 1 --2- 172.21.15.112:0/1700706718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac380b17e0 0x7fac3806e290 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fac340098f0 tx=0x7fac34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479a692cb6757c68 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:03.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.066+0000 7fac3c84f700 1 -- 172.21.15.112:0/1700706718 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34004230 con 0x7fac380b17e0 2023-08-27T11:29:03.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.066+0000 7fac3c84f700 1 -- 172.21.15.112:0/1700706718 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac34004390 con 0x7fac380b17e0 2023-08-27T11:29:03.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.066+0000 7fac3c84f700 1 -- 172.21.15.112:0/1700706718 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac340036a0 con 0x7fac380b17e0 2023-08-27T11:29:03.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.067+0000 7fac3fab5700 1 -- 172.21.15.112:0/1700706718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac380b17e0 msgr2=0x7fac3806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.067+0000 7fac3fab5700 1 --2- 172.21.15.112:0/1700706718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac380b17e0 0x7fac3806e290 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fac340098f0 tx=0x7fac34009c00 comp rx=0 tx=0).stop 2023-08-27T11:29:03.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.067+0000 7fac3fab5700 1 -- 172.21.15.112:0/1700706718 shutdown_connections 2023-08-27T11:29:03.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.067+0000 7fac3fab5700 1 --2- 172.21.15.112:0/1700706718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac380b1400 0x7fac380cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.067+0000 7fac3fab5700 1 --2- 172.21.15.112:0/1700706718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac380b17e0 0x7fac3806e290 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.067+0000 7fac3fab5700 1 -- 172.21.15.112:0/1700706718 >> 172.21.15.112:0/1700706718 conn(0x7fac380d0000 msgr2=0x7fac380d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:03.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.067+0000 7fac3fab5700 1 -- 172.21.15.112:0/1700706718 shutdown_connections 2023-08-27T11:29:03.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.067+0000 7fac3fab5700 1 -- 172.21.15.112:0/1700706718 wait complete. 2023-08-27T11:29:03.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.068+0000 7fac3fab5700 1 Processor -- start 2023-08-27T11:29:03.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.068+0000 7fac3fab5700 1 -- start start 2023-08-27T11:29:03.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.069+0000 7fac3fab5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac380b1400 0x7fac380c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.069+0000 7fac3fab5700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac380b17e0 0x7fac381581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.069+0000 7fac3fab5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac38161ce0 0x7fac381f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.069+0000 7fac3fab5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac381f3fb0 con 0x7fac380b1400 2023-08-27T11:29:03.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.069+0000 7fac3fab5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac38157170 con 0x7fac38161ce0 2023-08-27T11:29:03.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.069+0000 7fac3fab5700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac381573f0 con 0x7fac380b17e0 2023-08-27T11:29:03.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.069+0000 7fac3d851700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac380b1400 0x7fac380c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.069+0000 7fac3d851700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac380b1400 0x7fac380c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37426/0 (socket says 172.21.15.112:37426) 2023-08-27T11:29:03.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.069+0000 7fac3d851700 1 -- 172.21.15.112:0/2892298738 learned_addr learned my addr 172.21.15.112:0/2892298738 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:03.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.070+0000 7fac3d851700 1 -- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac380b17e0 msgr2=0x7fac381581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.070+0000 7fac3d851700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac380b17e0 0x7fac381581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.070+0000 7fac3d851700 1 -- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac38161ce0 msgr2=0x7fac381f6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.070+0000 7fac3e052700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac38161ce0 0x7fac381f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.070+0000 7fac3d851700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac38161ce0 0x7fac381f6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.070+0000 7fac3d851700 1 -- 172.21.15.112:0/2892298738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac34009550 con 0x7fac380b1400 2023-08-27T11:29:03.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.070+0000 7fac3e052700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac38161ce0 0x7fac381f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:03.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.070+0000 7fac3d851700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac380b1400 0x7fac380c9b80 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00b530 tx=0x7fac2c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:03.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.070+0000 7fac2a7fc700 1 -- 172.21.15.112:0/2892298738 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c011830 con 0x7fac380b1400 2023-08-27T11:29:03.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.070+0000 7fac2a7fc700 1 -- 172.21.15.112:0/2892298738 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2c0119b0 con 0x7fac380b1400 2023-08-27T11:29:03.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.070+0000 7fac3fab5700 1 -- 172.21.15.112:0/2892298738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac381576d0 con 0x7fac380b1400 2023-08-27T11:29:03.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.070+0000 7fac2a7fc700 1 -- 172.21.15.112:0/2892298738 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c011d60 con 0x7fac380b1400 2023-08-27T11:29:03.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.070+0000 7fac3fab5700 1 -- 172.21.15.112:0/2892298738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac38157c90 con 0x7fac380b1400 2023-08-27T11:29:03.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.071+0000 7fac3fab5700 1 -- 172.21.15.112:0/2892298738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac38157e80 con 0x7fac380b1400 2023-08-27T11:29:03.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.071+0000 7fac2a7fc700 1 -- 172.21.15.112:0/2892298738 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fac2c01a400 con 0x7fac380b1400 2023-08-27T11:29:03.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.072+0000 7fac2a7fc700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fac1c06a4d0 0x7fac1c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.072+0000 7fac2a7fc700 1 -- 172.21.15.112:0/2892298738 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fac2c088d80 con 0x7fac380b1400 2023-08-27T11:29:03.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.072+0000 7fac3d050700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fac1c06a4d0 0x7fac1c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.072+0000 7fac3d050700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fac1c06a4d0 0x7fac1c06c990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fac380ce4c0 tx=0x7fac34010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:03.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.076+0000 7fac2a7fc700 1 -- 172.21.15.112:0/2892298738 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fac2c053e30 con 0x7fac380b1400 2023-08-27T11:29:03.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.266+0000 7fac3fab5700 1 -- 172.21.15.112:0/2892298738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac381614b0 con 0x7fac380b1400 2023-08-27T11:29:03.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.267+0000 7fac2a7fc700 1 -- 172.21.15.112:0/2892298738 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+18412 (secure 0 0 0) 0x7fac2c015090 con 0x7fac380b1400 2023-08-27T11:29:03.269 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:03.269 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":155,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:55.437349+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.551922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.772403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.7","mappings":[{"from":6,"to":4}]},{"pgid":"28.1a","mappings":[{"from":6,"to":4}]},{"pgid":"28.1d","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:03.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.269+0000 7fac3fab5700 1 -- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fac1c06a4d0 msgr2=0x7fac1c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.269+0000 7fac3fab5700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fac1c06a4d0 0x7fac1c06c990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fac380ce4c0 tx=0x7fac34010000 comp rx=0 tx=0).stop 2023-08-27T11:29:03.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.269+0000 7fac3fab5700 1 -- 172.21.15.112:0/2892298738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac380b1400 msgr2=0x7fac380c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.269+0000 7fac3fab5700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac380b1400 0x7fac380c9b80 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00b530 tx=0x7fac2c00b8f0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.269+0000 7fac3fab5700 1 -- 172.21.15.112:0/2892298738 shutdown_connections 2023-08-27T11:29:03.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.269+0000 7fac3fab5700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fac1c06a4d0 0x7fac1c06c990 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.269+0000 7fac3fab5700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac38161ce0 0x7fac381f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.269+0000 7fac3fab5700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac380b1400 0x7fac380c9b80 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.269+0000 7fac3fab5700 1 --2- 172.21.15.112:0/2892298738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac380b17e0 0x7fac381581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.269+0000 7fac3fab5700 1 -- 172.21.15.112:0/2892298738 >> 172.21.15.112:0/2892298738 conn(0x7fac380d0000 msgr2=0x7fac3815eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:03.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.269+0000 7fac3fab5700 1 -- 172.21.15.112:0/2892298738 shutdown_connections 2023-08-27T11:29:03.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.269+0000 7fac3fab5700 1 -- 172.21.15.112:0/2892298738 wait complete. 2023-08-27T11:29:03.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:28:47.551922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '152', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:28:48.772403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:29:03.283 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:29:03.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.404+0000 7fcd09053700 1 Processor -- start 2023-08-27T11:29:03.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.404+0000 7fcd09053700 1 -- start start 2023-08-27T11:29:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.405+0000 7fcd09053700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd04146c20 0x7fcd04160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.405+0000 7fcd09053700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd040b5070 con 0x7fcd0406f790 2023-08-27T11:29:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.405+0000 7fcd09053700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd04147000 con 0x7fcd040b3440 2023-08-27T11:29:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.405+0000 7fcd09053700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd041614a0 con 0x7fcd04146c20 2023-08-27T11:29:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.405+0000 7fcd0359e700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fcd0406f790 0x7fcd040b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60050/0 (socket says 172.21.15.112:60050) 2023-08-27T11:29:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.405+0000 7fcd0359e700 1 -- 172.21.15.112:0/3496266351 learned_addr learned my addr 172.21.15.112:0/3496266351 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.405+0000 7fcd02d9d700 1 --2- 172.21.15.112:0/3496266351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd04146c20 0x7fcd04160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.405+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405581504 0 0) 0x7fccfc001490 con 0x7fcd0406f790 2023-08-27T11:29:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccec0035b0 con 0x7fcd0406f790 2023-08-27T11:29:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3040603945 0 0) 0x7fccec0035b0 con 0x7fcd0406f790 2023-08-27T11:29:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd04146c20 msgr2=0x7fcd04160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 --2- 172.21.15.112:0/3496266351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd04146c20 0x7fcd04160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 >> v1:172.21.15.138:6789/0 conn(0x7fcd040b3440 legacy=0x7fcd0415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd04161740 con 0x7fcd0406f790 2023-08-27T11:29:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fccfc001b80 con 0x7fcd0406f790 2023-08-27T11:29:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 >> v1:172.21.15.112:6789/0 conn(0x7fcd0406f790 legacy=0x7fcd040b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 --2- 172.21.15.112:0/3496266351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccec003a30 0x7fccec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 --2- 172.21.15.112:0/3496266351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccec0043e0 0x7fccec0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 --2- 172.21.15.112:0/3496266351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd04146c20 0x7fccec0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd041614a0 con 0x7fccec0043e0 2023-08-27T11:29:03.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec00bac0 con 0x7fccec003a30 2023-08-27T11:29:03.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec00bc80 con 0x7fcd04146c20 2023-08-27T11:29:03.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd0259c700 1 --2- 172.21.15.112:0/3496266351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccec0043e0 0x7fccec0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd02d9d700 1 --2- 172.21.15.112:0/3496266351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccec003a30 0x7fccec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd0359e700 1 --2- 172.21.15.112:0/3496266351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd04146c20 0x7fccec0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd0259c700 1 -- 172.21.15.112:0/3496266351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd04146c20 msgr2=0x7fccec0051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:03.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd0259c700 1 --2- 172.21.15.112:0/3496266351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd04146c20 0x7fccec0051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd0259c700 1 -- 172.21.15.112:0/3496266351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccec003a30 msgr2=0x7fccec003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:03.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd0259c700 1 --2- 172.21.15.112:0/3496266351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccec003a30 0x7fccec003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd0259c700 1 -- 172.21.15.112:0/3496266351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec0035b0 con 0x7fccec0043e0 2023-08-27T11:29:03.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd0259c700 1 --2- 172.21.15.112:0/3496266351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccec0043e0 0x7fccec0048b0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fccf40018d0 tx=0x7fccec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ef5a3c3e6074df server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:03.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf400c690 con 0x7fccec0043e0 2023-08-27T11:29:03.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf400c7f0 con 0x7fccec0043e0 2023-08-27T11:29:03.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.406+0000 7fcd01d9b700 1 -- 172.21.15.112:0/3496266351 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf400cba0 con 0x7fccec0043e0 2023-08-27T11:29:03.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.407+0000 7fcd09053700 1 -- 172.21.15.112:0/3496266351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccec0043e0 msgr2=0x7fccec0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.407+0000 7fcd09053700 1 --2- 172.21.15.112:0/3496266351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccec0043e0 0x7fccec0048b0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fccf40018d0 tx=0x7fccec005880 comp rx=0 tx=0).stop 2023-08-27T11:29:03.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.407+0000 7fcd0359e700 1 -- 172.21.15.112:0/3496266351 reap_dead start 2023-08-27T11:29:03.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.407+0000 7fcd09053700 1 -- 172.21.15.112:0/3496266351 shutdown_connections 2023-08-27T11:29:03.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.407+0000 7fcd09053700 1 -- 172.21.15.112:0/3496266351 >> 172.21.15.112:0/3496266351 conn(0x7fcd040b1e90 msgr2=0x7fcd040b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:03.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.407+0000 7fcd09053700 1 -- 172.21.15.112:0/3496266351 shutdown_connections 2023-08-27T11:29:03.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.407+0000 7fcd09053700 1 -- 172.21.15.112:0/3496266351 wait complete. 2023-08-27T11:29:03.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.409+0000 7fcd09053700 1 Processor -- start 2023-08-27T11:29:03.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.409+0000 7fcd09053700 1 -- start start 2023-08-27T11:29:03.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.409+0000 7fcd09053700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccec0043e0 0x7fcd0415f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.409+0000 7fcd09053700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd0415a840 0x7fcd0415acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.409+0000 7fcd09053700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd040c8f30 0x7fcd040c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.409+0000 7fcd09053700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd041f2840 con 0x7fcd040c8f30 2023-08-27T11:29:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.409+0000 7fcd09053700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0415b200 con 0x7fcd0415a840 2023-08-27T11:29:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.409+0000 7fcd09053700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040c6890 con 0x7fccec0043e0 2023-08-27T11:29:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.409+0000 7fcd0359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd040c8f30 0x7fcd040c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.409+0000 7fcd0359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd040c8f30 0x7fcd040c93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37450/0 (socket says 172.21.15.112:37450) 2023-08-27T11:29:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.409+0000 7fcd0359e700 1 -- 172.21.15.112:0/2872841086 learned_addr learned my addr 172.21.15.112:0/2872841086 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcd02d9d700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccec0043e0 0x7fcd0415f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcd0259c700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd0415a840 0x7fcd0415acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcd0359e700 1 -- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccec0043e0 msgr2=0x7fcd0415f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcd0359e700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccec0043e0 0x7fcd0415f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcd0359e700 1 -- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd0415a840 msgr2=0x7fcd0415acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcd0359e700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd0415a840 0x7fcd0415acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcd0359e700 1 -- 172.21.15.112:0/2872841086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd040c9bf0 con 0x7fcd040c8f30 2023-08-27T11:29:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcd02d9d700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccec0043e0 0x7fcd0415f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcd0259c700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd0415a840 0x7fcd0415acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcd0359e700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd040c8f30 0x7fcd040c93b0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fcd040b2d30 tx=0x7fccfc007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcceb7fe700 1 -- 172.21.15.112:0/2872841086 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc007a50 con 0x7fcd040c8f30 2023-08-27T11:29:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcceb7fe700 1 -- 172.21.15.112:0/2872841086 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc0133f0 con 0x7fcd040c8f30 2023-08-27T11:29:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcd09053700 1 -- 172.21.15.112:0/2872841086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd040c6b30 con 0x7fcd040c8f30 2023-08-27T11:29:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcceb7fe700 1 -- 172.21.15.112:0/2872841086 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0137a0 con 0x7fcd040c8f30 2023-08-27T11:29:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.410+0000 7fcd09053700 1 -- 172.21.15.112:0/2872841086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd040c70f0 con 0x7fcd040c8f30 2023-08-27T11:29:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.411+0000 7fcd09053700 1 -- 172.21.15.112:0/2872841086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd04067e90 con 0x7fcd040c8f30 2023-08-27T11:29:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.411+0000 7fcceb7fe700 1 -- 172.21.15.112:0/2872841086 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fccfc013900 con 0x7fcd040c8f30 2023-08-27T11:29:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.412+0000 7fcceb7fe700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fccf006a4d0 0x7fccf006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.412+0000 7fcceb7fe700 1 -- 172.21.15.112:0/2872841086 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fccfc0893d0 con 0x7fcd040c8f30 2023-08-27T11:29:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.412+0000 7fcd02d9d700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fccf006a4d0 0x7fccf006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.412+0000 7fcd02d9d700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fccf006a4d0 0x7fccf006c990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009ae0 tx=0x7fccf8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.416+0000 7fcceb7fe700 1 -- 172.21.15.112:0/2872841086 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fccfc054480 con 0x7fcd040c8f30 2023-08-27T11:29:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.606+0000 7fcd09053700 1 -- 172.21.15.112:0/2872841086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcd040b5070 con 0x7fcd040c8f30 2023-08-27T11:29:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.607+0000 7fcceb7fe700 1 -- 172.21.15.112:0/2872841086 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v155) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcd040b5070 con 0x7fcd040c8f30 2023-08-27T11:29:03.609 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:29:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.609+0000 7fcd09053700 1 -- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fccf006a4d0 msgr2=0x7fccf006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.609+0000 7fcd09053700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fccf006a4d0 0x7fccf006c990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009ae0 tx=0x7fccf8008040 comp rx=0 tx=0).stop 2023-08-27T11:29:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.609+0000 7fcd09053700 1 -- 172.21.15.112:0/2872841086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd040c8f30 msgr2=0x7fcd040c93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.609+0000 7fcd09053700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd040c8f30 0x7fcd040c93b0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fcd040b2d30 tx=0x7fccfc007340 comp rx=0 tx=0).stop 2023-08-27T11:29:03.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.609+0000 7fcd09053700 1 -- 172.21.15.112:0/2872841086 shutdown_connections 2023-08-27T11:29:03.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.609+0000 7fcd09053700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fccf006a4d0 0x7fccf006c990 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.609+0000 7fcd09053700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd040c8f30 0x7fcd040c93b0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.609+0000 7fcd09053700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccec0043e0 0x7fcd0415f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.609+0000 7fcd09053700 1 --2- 172.21.15.112:0/2872841086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd0415a840 0x7fcd0415acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.609+0000 7fcd09053700 1 -- 172.21.15.112:0/2872841086 >> 172.21.15.112:0/2872841086 conn(0x7fcd040b1e90 msgr2=0x7fcd04070750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:03.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.609+0000 7fcd09053700 1 -- 172.21.15.112:0/2872841086 shutdown_connections 2023-08-27T11:29:03.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.609+0000 7fcd09053700 1 -- 172.21.15.112:0/2872841086 wait complete. 2023-08-27T11:29:03.623 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:29:03.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.745+0000 7fc188af2700 1 Processor -- start 2023-08-27T11:29:03.747 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.745+0000 7fc188af2700 1 -- start start 2023-08-27T11:29:03.749 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.746+0000 7fc188af2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc184153be0 0x7fc18406e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.746+0000 7fc188af2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc184153fc0 0x7fc18406e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.746+0000 7fc188af2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc1840cdf50 0x7fc1840ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.746+0000 7fc188af2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1840cf520 con 0x7fc1840cdf50 2023-08-27T11:29:03.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.746+0000 7fc188af2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1840cec80 con 0x7fc184153be0 2023-08-27T11:29:03.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.746+0000 7fc188af2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840cef60 con 0x7fc184153fc0 2023-08-27T11:29:03.750 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.746+0000 7fc182d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc1840cdf50 0x7fc1840ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.746+0000 7fc182d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc1840cdf50 0x7fc1840ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37458/0 (socket says 172.21.15.112:37458) 2023-08-27T11:29:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.746+0000 7fc182d9d700 1 -- 172.21.15.112:0/1736145998 learned_addr learned my addr 172.21.15.112:0/1736145998 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.747+0000 7fc181d9b700 1 --2- 172.21.15.112:0/1736145998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc184153fc0 0x7fc18406e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.747+0000 7fc18259c700 1 --2- 172.21.15.112:0/1736145998 >> v2:172.21.15.138:3300/0 conn(0x7fc184153be0 0x7fc18406e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.747+0000 7fc182d9d700 1 -- 172.21.15.112:0/1736145998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc184153fc0 msgr2=0x7fc18406e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.747+0000 7fc182d9d700 1 --2- 172.21.15.112:0/1736145998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc184153fc0 0x7fc18406e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.747+0000 7fc182d9d700 1 -- 172.21.15.112:0/1736145998 >> v2:172.21.15.138:3300/0 conn(0x7fc184153be0 msgr2=0x7fc18406e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.747+0000 7fc182d9d700 1 --2- 172.21.15.112:0/1736145998 >> v2:172.21.15.138:3300/0 conn(0x7fc184153be0 0x7fc18406e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.747+0000 7fc182d9d700 1 -- 172.21.15.112:0/1736145998 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1840cf240 con 0x7fc1840cdf50 2023-08-27T11:29:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.747+0000 7fc182d9d700 1 --2- 172.21.15.112:0/1736145998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1840cdf50 0x7fc1840ce3c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fc184072010 tx=0x7fc17c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f16bb13fe864d4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.747+0000 7fc18159a700 1 -- 172.21.15.112:0/1736145998 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c010070 con 0x7fc1840cdf50 2023-08-27T11:29:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.747+0000 7fc18159a700 1 -- 172.21.15.112:0/1736145998 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c00dd90 con 0x7fc1840cdf50 2023-08-27T11:29:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.747+0000 7fc18159a700 1 -- 172.21.15.112:0/1736145998 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00b310 con 0x7fc1840cdf50 2023-08-27T11:29:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.747+0000 7fc188af2700 1 -- 172.21.15.112:0/1736145998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1840cdf50 msgr2=0x7fc1840ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.747+0000 7fc188af2700 1 --2- 172.21.15.112:0/1736145998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1840cdf50 0x7fc1840ce3c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fc184072010 tx=0x7fc17c00d430 comp rx=0 tx=0).stop 2023-08-27T11:29:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.748+0000 7fc188af2700 1 -- 172.21.15.112:0/1736145998 shutdown_connections 2023-08-27T11:29:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.748+0000 7fc188af2700 1 --2- 172.21.15.112:0/1736145998 >> v2:172.21.15.138:3300/0 conn(0x7fc184153be0 0x7fc18406e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.748+0000 7fc188af2700 1 --2- 172.21.15.112:0/1736145998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1840cdf50 0x7fc1840ce3c0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.748+0000 7fc188af2700 1 --2- 172.21.15.112:0/1736145998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc184153fc0 0x7fc18406e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.748+0000 7fc188af2700 1 -- 172.21.15.112:0/1736145998 >> 172.21.15.112:0/1736145998 conn(0x7fc18406cda0 msgr2=0x7fc1840717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:03.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.748+0000 7fc188af2700 1 -- 172.21.15.112:0/1736145998 shutdown_connections 2023-08-27T11:29:03.755 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.748+0000 7fc188af2700 1 -- 172.21.15.112:0/1736145998 wait complete. 2023-08-27T11:29:03.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.749+0000 7fc188af2700 1 Processor -- start 2023-08-27T11:29:03.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.749+0000 7fc188af2700 1 -- start start 2023-08-27T11:29:03.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc188af2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1840cdf50 0x7fc18415c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc188af2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc184153be0 0x7fc18415c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc188af2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc184153fc0 0x7fc1841ee210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc188af2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1841f15a0 con 0x7fc184153fc0 2023-08-27T11:29:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc188af2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1841ef550 con 0x7fc184153be0 2023-08-27T11:29:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc188af2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1841ef810 con 0x7fc1840cdf50 2023-08-27T11:29:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc182d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc184153fc0 0x7fc1841ee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc182d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc184153fc0 0x7fc1841ee210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37464/0 (socket says 172.21.15.112:37464) 2023-08-27T11:29:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc182d9d700 1 -- 172.21.15.112:0/793028799 learned_addr learned my addr 172.21.15.112:0/793028799 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc181d9b700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc184153be0 0x7fc18415c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc18259c700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1840cdf50 0x7fc18415c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc182d9d700 1 -- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1840cdf50 msgr2=0x7fc18415c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc182d9d700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1840cdf50 0x7fc18415c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc182d9d700 1 -- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc184153be0 msgr2=0x7fc18415c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc182d9d700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc184153be0 0x7fc18415c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc182d9d700 1 -- 172.21.15.112:0/793028799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c00bd40 con 0x7fc184153fc0 2023-08-27T11:29:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc18259c700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1840cdf50 0x7fc18415c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc181d9b700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc184153be0 0x7fc18415c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc182d9d700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc184153fc0 0x7fc1841ee210 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fc1840d0850 tx=0x7fc17c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc172ffd700 1 -- 172.21.15.112:0/793028799 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00bbc0 con 0x7fc184153fc0 2023-08-27T11:29:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc172ffd700 1 -- 172.21.15.112:0/793028799 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c00b4c0 con 0x7fc184153fc0 2023-08-27T11:29:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc188af2700 1 -- 172.21.15.112:0/793028799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1841efaf0 con 0x7fc184153fc0 2023-08-27T11:29:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc172ffd700 1 -- 172.21.15.112:0/793028799 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c004280 con 0x7fc184153fc0 2023-08-27T11:29:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.750+0000 7fc188af2700 1 -- 172.21.15.112:0/793028799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18415cb30 con 0x7fc184153fc0 2023-08-27T11:29:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.751+0000 7fc188af2700 1 -- 172.21.15.112:0/793028799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1841f0010 con 0x7fc184153fc0 2023-08-27T11:29:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.752+0000 7fc172ffd700 1 -- 172.21.15.112:0/793028799 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fc17c0043e0 con 0x7fc184153fc0 2023-08-27T11:29:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.752+0000 7fc172ffd700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16406a4d0 0x7fc16406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.752+0000 7fc172ffd700 1 -- 172.21.15.112:0/793028799 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fc17c08caa0 con 0x7fc184153fc0 2023-08-27T11:29:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.752+0000 7fc18259c700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16406a4d0 0x7fc16406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.753+0000 7fc18259c700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16406a4d0 0x7fc16406c990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc17800aa90 tx=0x7fc178008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.756+0000 7fc172ffd700 1 -- 172.21.15.112:0/793028799 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc1841f0010 con 0x7fc184153fc0 2023-08-27T11:29:03.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.944+0000 7fc188af2700 1 -- 172.21.15.112:0/793028799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1841f0010 con 0x7fc184153fc0 2023-08-27T11:29:03.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.945+0000 7fc172ffd700 1 -- 172.21.15.112:0/793028799 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v155) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc17c022020 con 0x7fc184153fc0 2023-08-27T11:29:03.947 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:29:03.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.947+0000 7fc188af2700 1 -- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16406a4d0 msgr2=0x7fc16406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.947+0000 7fc188af2700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16406a4d0 0x7fc16406c990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc17800aa90 tx=0x7fc178008040 comp rx=0 tx=0).stop 2023-08-27T11:29:03.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.947+0000 7fc188af2700 1 -- 172.21.15.112:0/793028799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc184153fc0 msgr2=0x7fc1841ee210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.947+0000 7fc188af2700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc184153fc0 0x7fc1841ee210 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fc1840d0850 tx=0x7fc17c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.947+0000 7fc188af2700 1 -- 172.21.15.112:0/793028799 shutdown_connections 2023-08-27T11:29:03.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.947+0000 7fc188af2700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc16406a4d0 0x7fc16406c990 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.947+0000 7fc188af2700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc184153fc0 0x7fc1841ee210 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.947+0000 7fc188af2700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1840cdf50 0x7fc18415c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.947+0000 7fc188af2700 1 --2- 172.21.15.112:0/793028799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc184153be0 0x7fc18415c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.947+0000 7fc188af2700 1 -- 172.21.15.112:0/793028799 >> 172.21.15.112:0/793028799 conn(0x7fc18406cda0 msgr2=0x7fc18406d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:03.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.947+0000 7fc188af2700 1 -- 172.21.15.112:0/793028799 shutdown_connections 2023-08-27T11:29:03.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:03.947+0000 7fc188af2700 1 -- 172.21.15.112:0/793028799 wait complete. 2023-08-27T11:29:03.960 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:29:04.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.082+0000 7f362642f700 1 Processor -- start 2023-08-27T11:29:04.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.082+0000 7f362642f700 1 -- start start 2023-08-27T11:29:04.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.082+0000 7f362642f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f36200b3440 0x7f362015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.082+0000 7f362642f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f362015e6e0 0x7f362006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.082+0000 7f362642f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36200b5070 con 0x7f3620146c20 2023-08-27T11:29:04.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.082+0000 7f362642f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3620147000 con 0x7f36200b3440 2023-08-27T11:29:04.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.082+0000 7f362642f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3620161740 con 0x7f362015e6e0 2023-08-27T11:29:04.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.083+0000 7f361f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f36200b3440 0x7f362015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.083+0000 7f361f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f36200b3440 0x7f362015e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37470/0 (socket says 172.21.15.112:37470) 2023-08-27T11:29:04.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.083+0000 7f361f7fe700 1 -- 172.21.15.112:0/3229242371 learned_addr learned my addr 172.21.15.112:0/3229242371 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:04.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.083+0000 7f36249cc700 1 --2- 172.21.15.112:0/3229242371 >> v2:172.21.15.138:3300/0 conn(0x7f362015e6e0 0x7f362006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.083+0000 7f361f7fe700 1 -- 172.21.15.112:0/3229242371 >> v2:172.21.15.138:3300/0 conn(0x7f362015e6e0 msgr2=0x7f362006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:04.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.083+0000 7f361f7fe700 1 --2- 172.21.15.112:0/3229242371 >> v2:172.21.15.138:3300/0 conn(0x7f362015e6e0 0x7f362006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.083+0000 7f361f7fe700 1 -- 172.21.15.112:0/3229242371 >> v1:172.21.15.138:6789/0 conn(0x7f3620146c20 legacy=0x7f3620160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:04.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.083+0000 7f361f7fe700 1 -- 172.21.15.112:0/3229242371 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3620161a20 con 0x7f36200b3440 2023-08-27T11:29:04.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.083+0000 7f361f7fe700 1 --2- 172.21.15.112:0/3229242371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36200b3440 0x7f362015e1a0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f36200b38c0 tx=0x7f3608009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372073128a4476b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:04.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.083+0000 7f361effd700 1 -- 172.21.15.112:0/3229242371 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608004230 con 0x7f36200b3440 2023-08-27T11:29:04.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.083+0000 7f361effd700 1 -- 172.21.15.112:0/3229242371 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3608004390 con 0x7f36200b3440 2023-08-27T11:29:04.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.083+0000 7f361effd700 1 -- 172.21.15.112:0/3229242371 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36080036a0 con 0x7f36200b3440 2023-08-27T11:29:04.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.084+0000 7f362642f700 1 -- 172.21.15.112:0/3229242371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36200b3440 msgr2=0x7f362015e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.084+0000 7f362642f700 1 --2- 172.21.15.112:0/3229242371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36200b3440 0x7f362015e1a0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f36200b38c0 tx=0x7f3608009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.084+0000 7f362642f700 1 -- 172.21.15.112:0/3229242371 shutdown_connections 2023-08-27T11:29:04.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.084+0000 7f362642f700 1 --2- 172.21.15.112:0/3229242371 >> v2:172.21.15.138:3300/0 conn(0x7f362015e6e0 0x7f362006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.084+0000 7f362642f700 1 --2- 172.21.15.112:0/3229242371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36200b3440 0x7f362015e1a0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.084+0000 7f362642f700 1 -- 172.21.15.112:0/3229242371 >> 172.21.15.112:0/3229242371 conn(0x7f36200b1e90 msgr2=0x7f36200b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:04.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.084+0000 7f362642f700 1 -- 172.21.15.112:0/3229242371 shutdown_connections 2023-08-27T11:29:04.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.084+0000 7f362642f700 1 -- 172.21.15.112:0/3229242371 wait complete. 2023-08-27T11:29:04.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.085+0000 7f362642f700 1 Processor -- start 2023-08-27T11:29:04.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.085+0000 7f362642f700 1 -- start start 2023-08-27T11:29:04.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f362642f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36200b3440 0x7f36201407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f362642f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3620146c20 0x7f3620071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f362642f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362015e6e0 0x7f36201f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f362642f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200aec80 con 0x7f3620146c20 2023-08-27T11:29:04.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f362642f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200ae070 con 0x7f362015e6e0 2023-08-27T11:29:04.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f362642f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36200ae350 con 0x7f36200b3440 2023-08-27T11:29:04.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f361f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3620146c20 0x7f3620071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f361f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3620146c20 0x7f3620071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37476/0 (socket says 172.21.15.112:37476) 2023-08-27T11:29:04.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f361f7fe700 1 -- 172.21.15.112:0/253497905 learned_addr learned my addr 172.21.15.112:0/253497905 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:04.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f361ffff700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36200b3440 0x7f36201407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f36249cc700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362015e6e0 0x7f36201f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f361f7fe700 1 -- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36200b3440 msgr2=0x7f36201407c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f361f7fe700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36200b3440 0x7f36201407c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f361f7fe700 1 -- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362015e6e0 msgr2=0x7f36201f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f361f7fe700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362015e6e0 0x7f36201f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f361f7fe700 1 -- 172.21.15.112:0/253497905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3608009550 con 0x7f3620146c20 2023-08-27T11:29:04.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.086+0000 7f361f7fe700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3620146c20 0x7f3620071330 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f362006e8e0 tx=0x7f3608004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:04.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.087+0000 7f361cff9700 1 -- 172.21.15.112:0/253497905 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608003d80 con 0x7f3620146c20 2023-08-27T11:29:04.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.087+0000 7f361cff9700 1 -- 172.21.15.112:0/253497905 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360801f070 con 0x7f3620146c20 2023-08-27T11:29:04.097 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.087+0000 7f361cff9700 1 -- 172.21.15.112:0/253497905 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608011870 con 0x7f3620146c20 2023-08-27T11:29:04.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.087+0000 7f362642f700 1 -- 172.21.15.112:0/253497905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36200ae630 con 0x7f3620146c20 2023-08-27T11:29:04.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.087+0000 7f362642f700 1 -- 172.21.15.112:0/253497905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362015b310 con 0x7f3620146c20 2023-08-27T11:29:04.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.087+0000 7f362642f700 1 -- 172.21.15.112:0/253497905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3620067e90 con 0x7f3620146c20 2023-08-27T11:29:04.098 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.088+0000 7f361cff9700 1 -- 172.21.15.112:0/253497905 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f3608021030 con 0x7f3620146c20 2023-08-27T11:29:04.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.088+0000 7f361cff9700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f360006a4d0 0x7f360006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.088+0000 7f361cff9700 1 -- 172.21.15.112:0/253497905 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f360808a0b0 con 0x7f3620146c20 2023-08-27T11:29:04.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.089+0000 7f361ffff700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f360006a4d0 0x7f360006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.089+0000 7f361ffff700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f360006a4d0 0x7f360006c990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3610004f30 tx=0x7f3610009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:04.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.093+0000 7f361cff9700 1 -- 172.21.15.112:0/253497905 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f36080553b0 con 0x7f3620146c20 2023-08-27T11:29:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.283+0000 7f362642f700 1 -- 172.21.15.112:0/253497905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36200b5030 con 0x7f3620146c20 2023-08-27T11:29:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.284+0000 7f361cff9700 1 -- 172.21.15.112:0/253497905 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v155) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36080589d0 con 0x7f3620146c20 2023-08-27T11:29:04.285 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:29:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.286+0000 7f362642f700 1 -- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f360006a4d0 msgr2=0x7f360006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.286+0000 7f362642f700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f360006a4d0 0x7f360006c990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3610004f30 tx=0x7f3610009040 comp rx=0 tx=0).stop 2023-08-27T11:29:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.286+0000 7f362642f700 1 -- 172.21.15.112:0/253497905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3620146c20 msgr2=0x7f3620071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.286+0000 7f362642f700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3620146c20 0x7f3620071330 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f362006e8e0 tx=0x7f3608004750 comp rx=0 tx=0).stop 2023-08-27T11:29:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.286+0000 7f362642f700 1 -- 172.21.15.112:0/253497905 shutdown_connections 2023-08-27T11:29:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.286+0000 7f362642f700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f360006a4d0 0x7f360006c990 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.286+0000 7f362642f700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362015e6e0 0x7f36201f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.286+0000 7f362642f700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36200b3440 0x7f36201407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.286+0000 7f362642f700 1 --2- 172.21.15.112:0/253497905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3620146c20 0x7f3620071330 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.286+0000 7f362642f700 1 -- 172.21.15.112:0/253497905 >> 172.21.15.112:0/253497905 conn(0x7f36200b1e90 msgr2=0x7f362006d910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.286+0000 7f362642f700 1 -- 172.21.15.112:0/253497905 shutdown_connections 2023-08-27T11:29:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.286+0000 7f362642f700 1 -- 172.21.15.112:0/253497905 wait complete. 2023-08-27T11:29:04.299 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:29:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.421+0000 7f18d7309700 1 Processor -- start 2023-08-27T11:29:04.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.421+0000 7f18d7309700 1 -- start start 2023-08-27T11:29:04.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.422+0000 7f18d7309700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f18d00b4d00 0x7f18d006b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.422+0000 7f18d7309700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f18d006b880 0x7f18d015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.422+0000 7f18d7309700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d015ae00 0x7f18d006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.422+0000 7f18d7309700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d006bd30 con 0x7f18d00b4d00 2023-08-27T11:29:04.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.422+0000 7f18d7309700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d015b8d0 con 0x7f18d006b880 2023-08-27T11:29:04.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.422+0000 7f18d7309700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d015b680 con 0x7f18d015ae00 2023-08-27T11:29:04.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.422+0000 7f18d50a5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f18d00b4d00 0x7f18d006b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.422+0000 7f18d50a5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f18d00b4d00 0x7f18d006b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37482/0 (socket says 172.21.15.112:37482) 2023-08-27T11:29:04.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.422+0000 7f18d50a5700 1 -- 172.21.15.112:0/3875940036 learned_addr learned my addr 172.21.15.112:0/3875940036 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:04.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.422+0000 7f18d58a6700 1 --2- 172.21.15.112:0/3875940036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d015ae00 0x7f18d006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18d48a4700 1 --2- 172.21.15.112:0/3875940036 >> v2:172.21.15.138:3300/0 conn(0x7f18d006b880 0x7f18d015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18d50a5700 1 -- 172.21.15.112:0/3875940036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d015ae00 msgr2=0x7f18d006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:04.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18d50a5700 1 --2- 172.21.15.112:0/3875940036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d015ae00 0x7f18d006d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18d50a5700 1 -- 172.21.15.112:0/3875940036 >> v2:172.21.15.138:3300/0 conn(0x7f18d006b880 msgr2=0x7f18d015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:04.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18d50a5700 1 --2- 172.21.15.112:0/3875940036 >> v2:172.21.15.138:3300/0 conn(0x7f18d006b880 0x7f18d015a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18d50a5700 1 -- 172.21.15.112:0/3875940036 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d015ec10 con 0x7f18d00b4d00 2023-08-27T11:29:04.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18d48a4700 1 --2- 172.21.15.112:0/3875940036 >> v2:172.21.15.138:3300/0 conn(0x7f18d006b880 0x7f18d015a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:04.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18d58a6700 1 --2- 172.21.15.112:0/3875940036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d015ae00 0x7f18d006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:04.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18d50a5700 1 --2- 172.21.15.112:0/3875940036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d00b4d00 0x7f18d006b340 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f18d00b2fa0 tx=0x7f18c000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11724ac7074dc612 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:04.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18c7fff700 1 -- 172.21.15.112:0/3875940036 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000dd30 con 0x7f18d00b4d00 2023-08-27T11:29:04.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18c7fff700 1 -- 172.21.15.112:0/3875940036 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c000de90 con 0x7f18d00b4d00 2023-08-27T11:29:04.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18c7fff700 1 -- 172.21.15.112:0/3875940036 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0010690 con 0x7f18d00b4d00 2023-08-27T11:29:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18d7309700 1 -- 172.21.15.112:0/3875940036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d00b4d00 msgr2=0x7f18d006b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.423+0000 7f18d7309700 1 --2- 172.21.15.112:0/3875940036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d00b4d00 0x7f18d006b340 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f18d00b2fa0 tx=0x7f18c000be70 comp rx=0 tx=0).stop 2023-08-27T11:29:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.424+0000 7f18d7309700 1 -- 172.21.15.112:0/3875940036 shutdown_connections 2023-08-27T11:29:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.424+0000 7f18d7309700 1 --2- 172.21.15.112:0/3875940036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d015ae00 0x7f18d006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.424+0000 7f18d7309700 1 --2- 172.21.15.112:0/3875940036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d00b4d00 0x7f18d006b340 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.424+0000 7f18d7309700 1 --2- 172.21.15.112:0/3875940036 >> v2:172.21.15.138:3300/0 conn(0x7f18d006b880 0x7f18d015a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.424+0000 7f18d7309700 1 -- 172.21.15.112:0/3875940036 >> 172.21.15.112:0/3875940036 conn(0x7f18d00d03e0 msgr2=0x7f18d00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:04.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.424+0000 7f18d7309700 1 -- 172.21.15.112:0/3875940036 shutdown_connections 2023-08-27T11:29:04.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.424+0000 7f18d7309700 1 -- 172.21.15.112:0/3875940036 wait complete. 2023-08-27T11:29:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.425+0000 7f18d7309700 1 Processor -- start 2023-08-27T11:29:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.425+0000 7f18d7309700 1 -- start start 2023-08-27T11:29:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d7309700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d006b880 0x7f18d006f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d7309700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00b4d00 0x7f18d0072180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d7309700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d015ae00 0x7f18d01f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d7309700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00cac90 con 0x7f18d006b880 2023-08-27T11:29:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d7309700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00726c0 con 0x7f18d015ae00 2023-08-27T11:29:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d7309700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d0071120 con 0x7f18d00b4d00 2023-08-27T11:29:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d50a5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d006b880 0x7f18d006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d50a5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d006b880 0x7f18d006f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37486/0 (socket says 172.21.15.112:37486) 2023-08-27T11:29:04.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d50a5700 1 -- 172.21.15.112:0/3003793066 learned_addr learned my addr 172.21.15.112:0/3003793066 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:04.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d58a6700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d015ae00 0x7f18d01f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d48a4700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00b4d00 0x7f18d0072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d50a5700 1 -- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00b4d00 msgr2=0x7f18d0072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d50a5700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00b4d00 0x7f18d0072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d50a5700 1 -- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d015ae00 msgr2=0x7f18d01f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d50a5700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d015ae00 0x7f18d01f33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d50a5700 1 -- 172.21.15.112:0/3003793066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c000b7f0 con 0x7f18d006b880 2023-08-27T11:29:04.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d58a6700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d015ae00 0x7f18d01f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:04.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d48a4700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00b4d00 0x7f18d0072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:04.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d50a5700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d006b880 0x7f18d006f600 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f18c000beb0 tx=0x7f18c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18c5ffb700 1 -- 172.21.15.112:0/3003793066 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0003da0 con 0x7f18d006b880 2023-08-27T11:29:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18c5ffb700 1 -- 172.21.15.112:0/3003793066 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c0010470 con 0x7f18d006b880 2023-08-27T11:29:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18c5ffb700 1 -- 172.21.15.112:0/3003793066 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0010820 con 0x7f18d006b880 2023-08-27T11:29:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d7309700 1 -- 172.21.15.112:0/3003793066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d01f3910 con 0x7f18d006b880 2023-08-27T11:29:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.426+0000 7f18d7309700 1 -- 172.21.15.112:0/3003793066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d00716a0 con 0x7f18d006b880 2023-08-27T11:29:04.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.427+0000 7f18d7309700 1 -- 172.21.15.112:0/3003793066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d006e8f0 con 0x7f18d006b880 2023-08-27T11:29:04.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.429+0000 7f18c5ffb700 1 -- 172.21.15.112:0/3003793066 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f18c0019400 con 0x7f18d006b880 2023-08-27T11:29:04.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.429+0000 7f18c5ffb700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18b406a4d0 0x7f18b406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.429+0000 7f18c5ffb700 1 -- 172.21.15.112:0/3003793066 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f18c008a8f0 con 0x7f18d006b880 2023-08-27T11:29:04.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.429+0000 7f18d48a4700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18b406a4d0 0x7f18b406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.430+0000 7f18d48a4700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18b406a4d0 0x7f18b406c990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f18d006fb40 tx=0x7f18cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:04.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.432+0000 7f18c5ffb700 1 -- 172.21.15.112:0/3003793066 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f18c0055a20 con 0x7f18d006b880 2023-08-27T11:29:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.625+0000 7f18d7309700 1 -- 172.21.15.112:0/3003793066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f18d0067e90 con 0x7f18d006b880 2023-08-27T11:29:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.626+0000 7f18c5ffb700 1 -- 172.21.15.112:0/3003793066 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v155) v1 ==== 100+0+11 (secure 0 0 0) 0x7f18c0059040 con 0x7f18d006b880 2023-08-27T11:29:04.627 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:29:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.628+0000 7f18d7309700 1 -- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18b406a4d0 msgr2=0x7f18b406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.628+0000 7f18d7309700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18b406a4d0 0x7f18b406c990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f18d006fb40 tx=0x7f18cc008040 comp rx=0 tx=0).stop 2023-08-27T11:29:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.628+0000 7f18d7309700 1 -- 172.21.15.112:0/3003793066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d006b880 msgr2=0x7f18d006f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.628+0000 7f18d7309700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d006b880 0x7f18d006f600 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f18c000beb0 tx=0x7f18c0004750 comp rx=0 tx=0).stop 2023-08-27T11:29:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.628+0000 7f18d7309700 1 -- 172.21.15.112:0/3003793066 shutdown_connections 2023-08-27T11:29:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.628+0000 7f18d7309700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f18b406a4d0 0x7f18b406c990 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.628+0000 7f18d7309700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d015ae00 0x7f18d01f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.628+0000 7f18d7309700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d006b880 0x7f18d006f600 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.628+0000 7f18d7309700 1 --2- 172.21.15.112:0/3003793066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18d00b4d00 0x7f18d0072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.628+0000 7f18d7309700 1 -- 172.21.15.112:0/3003793066 >> 172.21.15.112:0/3003793066 conn(0x7f18d00d03e0 msgr2=0x7f18d015cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.628+0000 7f18d7309700 1 -- 172.21.15.112:0/3003793066 shutdown_connections 2023-08-27T11:29:04.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.628+0000 7f18d7309700 1 -- 172.21.15.112:0/3003793066 wait complete. 2023-08-27T11:29:04.641 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:29:04.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c9996e700 1 Processor -- start 2023-08-27T11:29:04.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c9996e700 1 -- start start 2023-08-27T11:29:04.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c9996e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6c9413f910 0x7f6c940cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c9996e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6c940ce090 0x7f6c9406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c9996e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c9406cb00 con 0x7f6c9406ff70 2023-08-27T11:29:04.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c9996e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c940b2580 con 0x7f6c940ce090 2023-08-27T11:29:04.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c9996e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c940707a0 con 0x7f6c9413f910 2023-08-27T11:29:04.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c927fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6c940ce090 0x7f6c9406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c927fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6c940ce090 0x7f6c9406fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37496/0 (socket says 172.21.15.112:37496) 2023-08-27T11:29:04.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c927fc700 1 -- 172.21.15.112:0/4203012243 learned_addr learned my addr 172.21.15.112:0/4203012243 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:04.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c92ffd700 1 --2- 172.21.15.112:0/4203012243 >> v2:172.21.15.138:3300/0 conn(0x7f6c9413f910 0x7f6c940cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c927fc700 1 -- 172.21.15.112:0/4203012243 >> v2:172.21.15.138:3300/0 conn(0x7f6c9413f910 msgr2=0x7f6c940cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:04.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c927fc700 1 --2- 172.21.15.112:0/4203012243 >> v2:172.21.15.138:3300/0 conn(0x7f6c9413f910 0x7f6c940cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c927fc700 1 -- 172.21.15.112:0/4203012243 >> v1:172.21.15.138:6789/0 conn(0x7f6c9406ff70 legacy=0x7f6c940b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:04.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c927fc700 1 -- 172.21.15.112:0/4203012243 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c94070a80 con 0x7f6c940ce090 2023-08-27T11:29:04.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c927fc700 1 --2- 172.21.15.112:0/4203012243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c940ce090 0x7f6c9406fa30 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f6c9415d1f0 tx=0x7f6c8800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279915ff488284a0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:04.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c91ffb700 1 -- 172.21.15.112:0/4203012243 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8800dcf0 con 0x7f6c940ce090 2023-08-27T11:29:04.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c91ffb700 1 -- 172.21.15.112:0/4203012243 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8800de50 con 0x7f6c940ce090 2023-08-27T11:29:04.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.762+0000 7f6c91ffb700 1 -- 172.21.15.112:0/4203012243 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88004250 con 0x7f6c940ce090 2023-08-27T11:29:04.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.763+0000 7f6c9996e700 1 -- 172.21.15.112:0/4203012243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c940ce090 msgr2=0x7f6c9406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.763+0000 7f6c9996e700 1 --2- 172.21.15.112:0/4203012243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c940ce090 0x7f6c9406fa30 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f6c9415d1f0 tx=0x7f6c8800be70 comp rx=0 tx=0).stop 2023-08-27T11:29:04.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.763+0000 7f6c9996e700 1 -- 172.21.15.112:0/4203012243 shutdown_connections 2023-08-27T11:29:04.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.763+0000 7f6c9996e700 1 --2- 172.21.15.112:0/4203012243 >> v2:172.21.15.138:3300/0 conn(0x7f6c9413f910 0x7f6c940cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.763+0000 7f6c9996e700 1 --2- 172.21.15.112:0/4203012243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c940ce090 0x7f6c9406fa30 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.763+0000 7f6c9996e700 1 -- 172.21.15.112:0/4203012243 >> 172.21.15.112:0/4203012243 conn(0x7f6c940b0bf0 msgr2=0x7f6c940b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:04.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.763+0000 7f6c9996e700 1 -- 172.21.15.112:0/4203012243 shutdown_connections 2023-08-27T11:29:04.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.763+0000 7f6c9996e700 1 -- 172.21.15.112:0/4203012243 wait complete. 2023-08-27T11:29:04.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.765+0000 7f6c9996e700 1 Processor -- start 2023-08-27T11:29:04.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.766+0000 7f6c9996e700 1 -- start start 2023-08-27T11:29:04.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.766+0000 7f6c9996e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c9406ff70 0x7f6c94159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.766+0000 7f6c9996e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c940ce090 0x7f6c9413bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.766+0000 7f6c9996e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c9413f910 0x7f6c9413c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.766+0000 7f6c9996e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94136950 con 0x7f6c940ce090 2023-08-27T11:29:04.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.766+0000 7f6c9996e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9413c750 con 0x7f6c9406ff70 2023-08-27T11:29:04.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.766+0000 7f6c9996e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94135fb0 con 0x7f6c9413f910 2023-08-27T11:29:04.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c927fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c940ce090 0x7f6c9413bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c927fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c940ce090 0x7f6c9413bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37502/0 (socket says 172.21.15.112:37502) 2023-08-27T11:29:04.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c927fc700 1 -- 172.21.15.112:0/1706966724 learned_addr learned my addr 172.21.15.112:0/1706966724 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:04.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c92ffd700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c9406ff70 0x7f6c94159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c937fe700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c9413f910 0x7f6c9413c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c927fc700 1 -- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c9413f910 msgr2=0x7f6c9413c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c927fc700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c9413f910 0x7f6c9413c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c927fc700 1 -- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c9406ff70 msgr2=0x7f6c94159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c927fc700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c9406ff70 0x7f6c94159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c927fc700 1 -- 172.21.15.112:0/1706966724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8800b7f0 con 0x7f6c940ce090 2023-08-27T11:29:04.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c927fc700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c940ce090 0x7f6c9413bbb0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f6c88000c00 tx=0x7f6c88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:04.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c7bfff700 1 -- 172.21.15.112:0/1706966724 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88020070 con 0x7f6c940ce090 2023-08-27T11:29:04.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c7bfff700 1 -- 172.21.15.112:0/1706966724 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c88004030 con 0x7f6c940ce090 2023-08-27T11:29:04.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c9996e700 1 -- 172.21.15.112:0/1706966724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c94136290 con 0x7f6c940ce090 2023-08-27T11:29:04.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c7bfff700 1 -- 172.21.15.112:0/1706966724 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88010470 con 0x7f6c940ce090 2023-08-27T11:29:04.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.767+0000 7f6c9996e700 1 -- 172.21.15.112:0/1706966724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c94159590 con 0x7f6c940ce090 2023-08-27T11:29:04.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.768+0000 7f6c9996e700 1 -- 172.21.15.112:0/1706966724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c94136a90 con 0x7f6c940ce090 2023-08-27T11:29:04.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.769+0000 7f6c7bfff700 1 -- 172.21.15.112:0/1706966724 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f6c8802d3f0 con 0x7f6c940ce090 2023-08-27T11:29:04.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.769+0000 7f6c7bfff700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6c8006a4d0 0x7f6c8006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.769+0000 7f6c7bfff700 1 -- 172.21.15.112:0/1706966724 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f6c8808c110 con 0x7f6c940ce090 2023-08-27T11:29:04.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.769+0000 7f6c92ffd700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6c8006a4d0 0x7f6c8006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.769+0000 7f6c92ffd700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6c8006a4d0 0x7f6c8006c990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6c84006b20 tx=0x7f6c84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:04.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.773+0000 7f6c7bfff700 1 -- 172.21.15.112:0/1706966724 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6c94136a90 con 0x7f6c940ce090 2023-08-27T11:29:04.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.977+0000 7f6c9996e700 1 -- 172.21.15.112:0/1706966724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6c94136a90 con 0x7f6c940ce090 2023-08-27T11:29:04.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.977+0000 7f6c7bfff700 1 -- 172.21.15.112:0/1706966724 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v155) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6c94136a90 con 0x7f6c940ce090 2023-08-27T11:29:04.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.979+0000 7f6c9996e700 1 -- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6c8006a4d0 msgr2=0x7f6c8006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.979+0000 7f6c9996e700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6c8006a4d0 0x7f6c8006c990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6c84006b20 tx=0x7f6c84008040 comp rx=0 tx=0).stop 2023-08-27T11:29:04.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.979+0000 7f6c9996e700 1 -- 172.21.15.112:0/1706966724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c940ce090 msgr2=0x7f6c9413bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.979+0000 7f6c9996e700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c940ce090 0x7f6c9413bbb0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f6c88000c00 tx=0x7f6c88004750 comp rx=0 tx=0).stop 2023-08-27T11:29:04.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.979+0000 7f6c9996e700 1 -- 172.21.15.112:0/1706966724 shutdown_connections 2023-08-27T11:29:04.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.979+0000 7f6c9996e700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6c8006a4d0 0x7f6c8006c990 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.979+0000 7f6c9996e700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c9413f910 0x7f6c9413c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.979+0000 7f6c9996e700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c9406ff70 0x7f6c94159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.979+0000 7f6c9996e700 1 --2- 172.21.15.112:0/1706966724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c940ce090 0x7f6c9413bbb0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.979+0000 7f6c9996e700 1 -- 172.21.15.112:0/1706966724 >> 172.21.15.112:0/1706966724 conn(0x7f6c940b0bf0 msgr2=0x7f6c9413fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:04.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.979+0000 7f6c9996e700 1 -- 172.21.15.112:0/1706966724 shutdown_connections 2023-08-27T11:29:04.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:04.979+0000 7f6c9996e700 1 -- 172.21.15.112:0/1706966724 wait complete. 2023-08-27T11:29:04.982 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:29:04.991 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-27T11:29:04.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:04.992 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-08-27T11:29:05.040 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:29:05.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:05.041 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-27T11:29:05.060 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:29:05.061 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:05.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:29:05.078 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:29:05.079 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:05.120 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:29:05.120 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:05.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.240+0000 7fc698e46700 1 Processor -- start 2023-08-27T11:29:05.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.240+0000 7fc698e46700 1 -- start start 2023-08-27T11:29:05.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc698e46700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc6940b17e0 0x7fc6940cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc698e46700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc694161850 con 0x7fc6940b1400 2023-08-27T11:29:05.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc698e46700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6940cfc10 con 0x7fc694161ce0 2023-08-27T11:29:05.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc698e46700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc694161200 con 0x7fc6940b17e0 2023-08-27T11:29:05.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69259c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc6940b1400 0x7fc69406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60122/0 (socket says 172.21.15.112:60122) 2023-08-27T11:29:05.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69259c700 1 -- 172.21.15.112:0/1681095264 learned_addr learned my addr 172.21.15.112:0/1681095264 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:05.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc691d9b700 1 --2- 172.21.15.112:0/1681095264 >> v2:172.21.15.138:3300/0 conn(0x7fc6940b17e0 0x7fc6940cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554735609 0 0) 0x7fc694161850 con 0x7fc6940b1400 2023-08-27T11:29:05.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc67c0035b0 con 0x7fc6940b1400 2023-08-27T11:29:05.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2896441181 0 0) 0x7fc67c0035b0 con 0x7fc6940b1400 2023-08-27T11:29:05.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 >> v2:172.21.15.138:3300/0 conn(0x7fc6940b17e0 msgr2=0x7fc6940cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:05.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 --2- 172.21.15.112:0/1681095264 >> v2:172.21.15.138:3300/0 conn(0x7fc6940b17e0 0x7fc6940cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 >> v1:172.21.15.138:6789/0 conn(0x7fc694161ce0 legacy=0x7fc69406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6941614e0 con 0x7fc6940b1400 2023-08-27T11:29:05.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fc688001670 con 0x7fc6940b1400 2023-08-27T11:29:05.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 >> v1:172.21.15.112:6789/0 conn(0x7fc6940b1400 legacy=0x7fc69406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 --2- 172.21.15.112:0/1681095264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc67c003a00 0x7fc67c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 --2- 172.21.15.112:0/1681095264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc67c004850 0x7fc67c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 --2- 172.21.15.112:0/1681095264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc67c005330 0x7fc67c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc694161200 con 0x7fc67c003a00 2023-08-27T11:29:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c00c200 con 0x7fc67c004850 2023-08-27T11:29:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c00c3c0 con 0x7fc67c005330 2023-08-27T11:29:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.241+0000 7fc69259c700 1 --2- 172.21.15.112:0/1681095264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc67c003a00 0x7fc67c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc691d9b700 1 --2- 172.21.15.112:0/1681095264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc67c004850 0x7fc67c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc692d9d700 1 --2- 172.21.15.112:0/1681095264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc67c005330 0x7fc67c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc69259c700 1 -- 172.21.15.112:0/1681095264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc67c005330 msgr2=0x7fc67c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc69259c700 1 --2- 172.21.15.112:0/1681095264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc67c005330 0x7fc67c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc69259c700 1 -- 172.21.15.112:0/1681095264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc67c004850 msgr2=0x7fc67c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc69259c700 1 --2- 172.21.15.112:0/1681095264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc67c004850 0x7fc67c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc69259c700 1 -- 172.21.15.112:0/1681095264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c0035b0 con 0x7fc67c003a00 2023-08-27T11:29:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc692d9d700 1 -- 172.21.15.112:0/1681095264 reap_dead start 2023-08-27T11:29:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc691d9b700 1 --2- 172.21.15.112:0/1681095264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc67c004850 0x7fc67c004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc69259c700 1 --2- 172.21.15.112:0/1681095264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc67c003a00 0x7fc67c003e20 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fc68800c210 tx=0x7fc67c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72398e697884e468 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:05.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688007c10 con 0x7fc67c003a00 2023-08-27T11:29:05.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68800cdb0 con 0x7fc67c003a00 2023-08-27T11:29:05.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc69159a700 1 -- 172.21.15.112:0/1681095264 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688005ba0 con 0x7fc67c003a00 2023-08-27T11:29:05.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc698e46700 1 -- 172.21.15.112:0/1681095264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc67c003a00 msgr2=0x7fc67c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.242+0000 7fc698e46700 1 --2- 172.21.15.112:0/1681095264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc67c003a00 0x7fc67c003e20 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fc68800c210 tx=0x7fc67c005880 comp rx=0 tx=0).stop 2023-08-27T11:29:05.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.243+0000 7fc698e46700 1 -- 172.21.15.112:0/1681095264 shutdown_connections 2023-08-27T11:29:05.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.243+0000 7fc698e46700 1 --2- 172.21.15.112:0/1681095264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc67c003a00 0x7fc67c003e20 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.243+0000 7fc698e46700 1 -- 172.21.15.112:0/1681095264 >> 172.21.15.112:0/1681095264 conn(0x7fc6940d0000 msgr2=0x7fc6940d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:05.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.243+0000 7fc698e46700 1 -- 172.21.15.112:0/1681095264 shutdown_connections 2023-08-27T11:29:05.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.243+0000 7fc698e46700 1 -- 172.21.15.112:0/1681095264 wait complete. 2023-08-27T11:29:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.244+0000 7fc698e46700 1 Processor -- start 2023-08-27T11:29:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.244+0000 7fc698e46700 1 -- start start 2023-08-27T11:29:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc698e46700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc67c003a00 0x7fc6940c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc698e46700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc69406ad20 0x7fc69406b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc698e46700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6941f6320 0x7fc6941f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc698e46700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69406b6e0 con 0x7fc69406ad20 2023-08-27T11:29:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc698e46700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940af2b0 con 0x7fc67c003a00 2023-08-27T11:29:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc698e46700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940ca360 con 0x7fc6941f6320 2023-08-27T11:29:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc691d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc69406ad20 0x7fc69406b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc691d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc69406ad20 0x7fc69406b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37522/0 (socket says 172.21.15.112:37522) 2023-08-27T11:29:05.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc691d9b700 1 -- 172.21.15.112:0/132354819 learned_addr learned my addr 172.21.15.112:0/132354819 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:05.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc69259c700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc67c003a00 0x7fc6940c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc692d9d700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6941f6320 0x7fc6941f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc691d9b700 1 -- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6941f6320 msgr2=0x7fc6941f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc691d9b700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6941f6320 0x7fc6941f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc691d9b700 1 -- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc67c003a00 msgr2=0x7fc6940c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc691d9b700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc67c003a00 0x7fc6940c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc691d9b700 1 -- 172.21.15.112:0/132354819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6940ae980 con 0x7fc69406ad20 2023-08-27T11:29:05.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc691d9b700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc69406ad20 0x7fc69406b1a0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fc684011580 tx=0x7fc684011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:05.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc682ffd700 1 -- 172.21.15.112:0/132354819 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684004220 con 0x7fc69406ad20 2023-08-27T11:29:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc682ffd700 1 -- 172.21.15.112:0/132354819 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc684004380 con 0x7fc69406ad20 2023-08-27T11:29:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc698e46700 1 -- 172.21.15.112:0/132354819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6940aec60 con 0x7fc69406ad20 2023-08-27T11:29:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc682ffd700 1 -- 172.21.15.112:0/132354819 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6840036a0 con 0x7fc69406ad20 2023-08-27T11:29:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.245+0000 7fc698e46700 1 -- 172.21.15.112:0/132354819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6941f7090 con 0x7fc69406ad20 2023-08-27T11:29:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.246+0000 7fc698e46700 1 -- 172.21.15.112:0/132354819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc694067a20 con 0x7fc69406ad20 2023-08-27T11:29:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.247+0000 7fc682ffd700 1 -- 172.21.15.112:0/132354819 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fc684003800 con 0x7fc69406ad20 2023-08-27T11:29:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.247+0000 7fc682ffd700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc67406a4d0 0x7fc67406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.247+0000 7fc682ffd700 1 -- 172.21.15.112:0/132354819 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fc684088e40 con 0x7fc69406ad20 2023-08-27T11:29:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.247+0000 7fc69259c700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc67406a4d0 0x7fc67406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.248+0000 7fc69259c700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc67406a4d0 0x7fc67406c990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc688000c00 tx=0x7fc688007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.251+0000 7fc682ffd700 1 -- 172.21.15.112:0/132354819 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc684053ef0 con 0x7fc69406ad20 2023-08-27T11:29:05.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.485+0000 7fc698e46700 1 -- 172.21.15.112:0/132354819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc69406db60 con 0x7fc69406ad20 2023-08-27T11:29:05.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.485+0000 7fc682ffd700 1 -- 172.21.15.112:0/132354819 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4891 (secure 0 0 0) 0x7fc684022090 con 0x7fc69406ad20 2023-08-27T11:29:05.486 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:05.486 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":105,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6980,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/877756033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":877756033},{"type":"v1","addr":"172.21.15.138:6837","nonce":877756033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6983,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2624496097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2624496097},{"type":"v1","addr":"172.21.15.138:6836","nonce":2624496097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":8940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2687911818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2687911818},{"type":"v1","addr":"172.21.15.112:6834","nonce":2687911818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:49.824795+0000","modified":"2023-08-27T11:28:53.433019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8943":{"gid":8943,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/577891555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":577891555},{"type":"v1","addr":"172.21.15.112:6835","nonce":577891555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-08-27T11:29:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.487+0000 7fc698e46700 1 -- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc67406a4d0 msgr2=0x7fc67406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.487+0000 7fc698e46700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc67406a4d0 0x7fc67406c990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc688000c00 tx=0x7fc688007000 comp rx=0 tx=0).stop 2023-08-27T11:29:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.487+0000 7fc698e46700 1 -- 172.21.15.112:0/132354819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc69406ad20 msgr2=0x7fc69406b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.487+0000 7fc698e46700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc69406ad20 0x7fc69406b1a0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fc684011580 tx=0x7fc684011890 comp rx=0 tx=0).stop 2023-08-27T11:29:05.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.487+0000 7fc698e46700 1 -- 172.21.15.112:0/132354819 shutdown_connections 2023-08-27T11:29:05.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.487+0000 7fc698e46700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc67406a4d0 0x7fc67406c990 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.487+0000 7fc698e46700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6941f6320 0x7fc6941f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.487+0000 7fc698e46700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc67c003a00 0x7fc6940c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.487+0000 7fc698e46700 1 --2- 172.21.15.112:0/132354819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc69406ad20 0x7fc69406b1a0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.487+0000 7fc698e46700 1 -- 172.21.15.112:0/132354819 >> 172.21.15.112:0/132354819 conn(0x7fc6940d0000 msgr2=0x7fc69415feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:05.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.487+0000 7fc698e46700 1 -- 172.21.15.112:0/132354819 shutdown_connections 2023-08-27T11:29:05.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.487+0000 7fc698e46700 1 -- 172.21.15.112:0/132354819 wait complete. 2023-08-27T11:29:05.490 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 105 2023-08-27T11:29:05.500 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:05.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.619+0000 7f6116754700 1 Processor -- start 2023-08-27T11:29:05.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.619+0000 7f6116754700 1 -- start start 2023-08-27T11:29:05.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.620+0000 7f6116754700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61100b17e0 0x7f61100cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.620+0000 7f6116754700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6110161850 con 0x7f61100b1400 2023-08-27T11:29:05.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.620+0000 7f6116754700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61100cfc10 con 0x7f6110161ce0 2023-08-27T11:29:05.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.620+0000 7f6116754700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6110161200 con 0x7f61100b17e0 2023-08-27T11:29:05.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.620+0000 7f610ffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f61100b1400 0x7f611006e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60142/0 (socket says 172.21.15.112:60142) 2023-08-27T11:29:05.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.620+0000 7f610ffff700 1 -- 172.21.15.112:0/803289767 learned_addr learned my addr 172.21.15.112:0/803289767 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:05.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.620+0000 7f610f7fe700 1 --2- 172.21.15.112:0/803289767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61100b17e0 0x7f61100cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.620+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474020261 0 0) 0x7f6110161850 con 0x7f61100b1400 2023-08-27T11:29:05.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.620+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60f80035b0 con 0x7f61100b1400 2023-08-27T11:29:05.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3949876658 0 0) 0x7f60f80035b0 con 0x7f61100b1400 2023-08-27T11:29:05.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61100b17e0 msgr2=0x7f61100cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:05.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 --2- 172.21.15.112:0/803289767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61100b17e0 0x7f61100cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 >> v1:172.21.15.138:6789/0 conn(0x7f6110161ce0 legacy=0x7f611006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61101614e0 con 0x7f61100b1400 2023-08-27T11:29:05.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f6104001670 con 0x7f61100b1400 2023-08-27T11:29:05.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 >> v1:172.21.15.112:6789/0 conn(0x7f61100b1400 legacy=0x7f611006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 --2- 172.21.15.112:0/803289767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60f8003a30 0x7f60f8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 --2- 172.21.15.112:0/803289767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61100b17e0 0x7f60f8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 --2- 172.21.15.112:0/803289767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60f8004d90 0x7f60f8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6110161200 con 0x7f60f8003a30 2023-08-27T11:29:05.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f800bb40 con 0x7f60f8004d90 2023-08-27T11:29:05.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f800bd00 con 0x7f61100b17e0 2023-08-27T11:29:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610ffff700 1 --2- 172.21.15.112:0/803289767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60f8003a30 0x7f60f8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f6114cf1700 1 --2- 172.21.15.112:0/803289767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60f8004d90 0x7f60f8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610f7fe700 1 --2- 172.21.15.112:0/803289767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61100b17e0 0x7f60f8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610ffff700 1 -- 172.21.15.112:0/803289767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61100b17e0 msgr2=0x7f60f8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:05.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610ffff700 1 --2- 172.21.15.112:0/803289767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61100b17e0 0x7f60f8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610ffff700 1 -- 172.21.15.112:0/803289767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60f8004d90 msgr2=0x7f60f8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:05.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610ffff700 1 --2- 172.21.15.112:0/803289767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60f8004d90 0x7f60f8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610ffff700 1 -- 172.21.15.112:0/803289767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f80035b0 con 0x7f60f8003a30 2023-08-27T11:29:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610ffff700 1 --2- 172.21.15.112:0/803289767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60f8003a30 0x7f60f8003e50 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f610400c240 tx=0x7f60f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319cf867aa7d7a9b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6104007c10 con 0x7f60f8003a30 2023-08-27T11:29:05.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610400cd40 con 0x7f60f8003a30 2023-08-27T11:29:05.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.621+0000 7f610effd700 1 -- 172.21.15.112:0/803289767 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6104005ba0 con 0x7f60f8003a30 2023-08-27T11:29:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.622+0000 7f6116754700 1 -- 172.21.15.112:0/803289767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60f8003a30 msgr2=0x7f60f8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.622+0000 7f6116754700 1 --2- 172.21.15.112:0/803289767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60f8003a30 0x7f60f8003e50 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f610400c240 tx=0x7f60f8005880 comp rx=0 tx=0).stop 2023-08-27T11:29:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.622+0000 7f6114cf1700 1 -- 172.21.15.112:0/803289767 reap_dead start 2023-08-27T11:29:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.622+0000 7f6116754700 1 -- 172.21.15.112:0/803289767 shutdown_connections 2023-08-27T11:29:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.622+0000 7f6116754700 1 -- 172.21.15.112:0/803289767 >> 172.21.15.112:0/803289767 conn(0x7f61100d0000 msgr2=0x7f61100d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.622+0000 7f6116754700 1 -- 172.21.15.112:0/803289767 shutdown_connections 2023-08-27T11:29:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.622+0000 7f6116754700 1 -- 172.21.15.112:0/803289767 wait complete. 2023-08-27T11:29:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.623+0000 7f6116754700 1 Processor -- start 2023-08-27T11:29:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.623+0000 7f6116754700 1 -- start start 2023-08-27T11:29:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f6116754700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60f8003a30 0x7f6110150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f6116754700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100c92f0 0x7f61100c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f6116754700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61100c72c0 0x7f61100c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f6116754700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61100c7c80 con 0x7f60f8003a30 2023-08-27T11:29:05.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f6116754700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61100ca350 con 0x7f61100c92f0 2023-08-27T11:29:05.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f6116754700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61100c9d90 con 0x7f61100c72c0 2023-08-27T11:29:05.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f610ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60f8003a30 0x7f6110150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f610ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60f8003a30 0x7f6110150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37540/0 (socket says 172.21.15.112:37540) 2023-08-27T11:29:05.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f610ffff700 1 -- 172.21.15.112:0/3648139920 learned_addr learned my addr 172.21.15.112:0/3648139920 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:05.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f6114cf1700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61100c72c0 0x7f61100c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f610f7fe700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100c92f0 0x7f61100c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f610ffff700 1 -- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61100c72c0 msgr2=0x7f61100c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f610ffff700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61100c72c0 0x7f61100c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f610ffff700 1 -- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100c92f0 msgr2=0x7f61100c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f610ffff700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100c92f0 0x7f61100c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f610ffff700 1 -- 172.21.15.112:0/3648139920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61100ca070 con 0x7f60f8003a30 2023-08-27T11:29:05.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.624+0000 7f610f7fe700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100c92f0 0x7f61100c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:05.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.625+0000 7f610ffff700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60f8003a30 0x7f6110150530 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f610400c830 tx=0x7f6104001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:05.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.625+0000 7f610cff9700 1 -- 172.21.15.112:0/3648139920 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6104001630 con 0x7f60f8003a30 2023-08-27T11:29:05.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.625+0000 7f610cff9700 1 -- 172.21.15.112:0/3648139920 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61040070d0 con 0x7f60f8003a30 2023-08-27T11:29:05.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.625+0000 7f610cff9700 1 -- 172.21.15.112:0/3648139920 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6104007480 con 0x7f60f8003a30 2023-08-27T11:29:05.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.625+0000 7f6116754700 1 -- 172.21.15.112:0/3648139920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6110150df0 con 0x7f60f8003a30 2023-08-27T11:29:05.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.625+0000 7f6116754700 1 -- 172.21.15.112:0/3648139920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6110150a70 con 0x7f60f8003a30 2023-08-27T11:29:05.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.626+0000 7f6116754700 1 -- 172.21.15.112:0/3648139920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6110151310 con 0x7f60f8003a30 2023-08-27T11:29:05.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.626+0000 7f610cff9700 1 -- 172.21.15.112:0/3648139920 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f6104019030 con 0x7f60f8003a30 2023-08-27T11:29:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.627+0000 7f610cff9700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f60fc06a4d0 0x7f60fc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.627+0000 7f610cff9700 1 -- 172.21.15.112:0/3648139920 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f6104089f10 con 0x7f60f8003a30 2023-08-27T11:29:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.627+0000 7f610f7fe700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f60fc06a4d0 0x7f60fc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.627+0000 7f610f7fe700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f60fc06a4d0 0x7f60fc06c990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6110071df0 tx=0x7f6100008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:05.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.631+0000 7f610cff9700 1 -- 172.21.15.112:0/3648139920 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6110151310 con 0x7f60f8003a30 2023-08-27T11:29:05.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.822+0000 7f6116754700 1 -- 172.21.15.112:0/3648139920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6110151310 con 0x7f60f8003a30 2023-08-27T11:29:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.823+0000 7f610cff9700 1 -- 172.21.15.112:0/3648139920 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+18412 (secure 0 0 0) 0x7f6110151310 con 0x7f60f8003a30 2023-08-27T11:29:05.824 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:05.824 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":155,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:55.437349+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.551922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.772403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.7","mappings":[{"from":6,"to":4}]},{"pgid":"28.1a","mappings":[{"from":6,"to":4}]},{"pgid":"28.1d","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:05.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.825+0000 7f6116754700 1 -- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f60fc06a4d0 msgr2=0x7f60fc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.825+0000 7f6116754700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f60fc06a4d0 0x7f60fc06c990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6110071df0 tx=0x7f6100008040 comp rx=0 tx=0).stop 2023-08-27T11:29:05.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.825+0000 7f6116754700 1 -- 172.21.15.112:0/3648139920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60f8003a30 msgr2=0x7f6110150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.825+0000 7f6116754700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60f8003a30 0x7f6110150530 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f610400c830 tx=0x7f6104001a00 comp rx=0 tx=0).stop 2023-08-27T11:29:05.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.825+0000 7f6116754700 1 -- 172.21.15.112:0/3648139920 shutdown_connections 2023-08-27T11:29:05.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.825+0000 7f6116754700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f60fc06a4d0 0x7f60fc06c990 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.825+0000 7f6116754700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61100c72c0 0x7f61100c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.825+0000 7f6116754700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60f8003a30 0x7f6110150530 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.825+0000 7f6116754700 1 --2- 172.21.15.112:0/3648139920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100c92f0 0x7f61100c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.825+0000 7f6116754700 1 -- 172.21.15.112:0/3648139920 >> 172.21.15.112:0/3648139920 conn(0x7f61100d0000 msgr2=0x7f6110156a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:05.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.825+0000 7f6116754700 1 -- 172.21.15.112:0/3648139920 shutdown_connections 2023-08-27T11:29:05.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.825+0000 7f6116754700 1 -- 172.21.15.112:0/3648139920 wait complete. 2023-08-27T11:29:05.838 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:28:47.551922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '152', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:28:48.772403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:29:05.839 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:29:05.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.961+0000 7f7c5de42700 1 Processor -- start 2023-08-27T11:29:05.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.961+0000 7f7c5de42700 1 -- start start 2023-08-27T11:29:05.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.962+0000 7f7c5de42700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7c58146b60 0x7f7c5806ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.962+0000 7f7c5de42700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c580ad1d0 0x7f7c5815d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.962+0000 7f7c5de42700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c58160fa0 con 0x7f7c5806e2f0 2023-08-27T11:29:05.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.962+0000 7f7c5de42700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c58161b70 con 0x7f7c58146b60 2023-08-27T11:29:05.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.962+0000 7f7c5de42700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c58143ad0 con 0x7f7c580ad1d0 2023-08-27T11:29:05.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.962+0000 7f7c56ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7c5806e2f0 0x7f7c580acac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60166/0 (socket says 172.21.15.112:60166) 2023-08-27T11:29:05.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.962+0000 7f7c56ffd700 1 -- 172.21.15.112:0/3278731475 learned_addr learned my addr 172.21.15.112:0/3278731475 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:05.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.962+0000 7f7c57fff700 1 --2- 172.21.15.112:0/3278731475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c580ad1d0 0x7f7c5815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.962+0000 7f7c577fe700 1 --2- 172.21.15.112:0/3278731475 >> v2:172.21.15.138:3300/0 conn(0x7f7c58146b60 0x7f7c5806ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.962+0000 7f7c567fc700 1 -- 172.21.15.112:0/3278731475 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76014947 0 0) 0x7f7c58160fa0 con 0x7f7c5806e2f0 2023-08-27T11:29:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.962+0000 7f7c567fc700 1 -- 172.21.15.112:0/3278731475 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c400035b0 con 0x7f7c5806e2f0 2023-08-27T11:29:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c57fff700 1 -- 172.21.15.112:0/3278731475 >> v2:172.21.15.138:3300/0 conn(0x7f7c58146b60 msgr2=0x7f7c5806ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c57fff700 1 --2- 172.21.15.112:0/3278731475 >> v2:172.21.15.138:3300/0 conn(0x7f7c58146b60 0x7f7c5806ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c57fff700 1 -- 172.21.15.112:0/3278731475 >> v1:172.21.15.112:6789/0 conn(0x7f7c5806e2f0 legacy=0x7f7c580acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c57fff700 1 -- 172.21.15.112:0/3278731475 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5806e750 con 0x7f7c580ad1d0 2023-08-27T11:29:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c577fe700 1 --2- 172.21.15.112:0/3278731475 >> v2:172.21.15.138:3300/0 conn(0x7f7c58146b60 0x7f7c5806ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c57fff700 1 --2- 172.21.15.112:0/3278731475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c580ad1d0 0x7f7c5815d930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009d90 tx=0x7f7c5000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d769dbf1704b2dc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c567fc700 1 -- 172.21.15.112:0/3278731475 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000e070 con 0x7f7c580ad1d0 2023-08-27T11:29:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c567fc700 1 -- 172.21.15.112:0/3278731475 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5000bb20 con 0x7f7c580ad1d0 2023-08-27T11:29:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c567fc700 1 -- 172.21.15.112:0/3278731475 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000be90 con 0x7f7c580ad1d0 2023-08-27T11:29:05.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c5de42700 1 -- 172.21.15.112:0/3278731475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c580ad1d0 msgr2=0x7f7c5815d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c5de42700 1 --2- 172.21.15.112:0/3278731475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c580ad1d0 0x7f7c5815d930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009d90 tx=0x7f7c5000b330 comp rx=0 tx=0).stop 2023-08-27T11:29:05.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c5de42700 1 -- 172.21.15.112:0/3278731475 shutdown_connections 2023-08-27T11:29:05.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c5de42700 1 --2- 172.21.15.112:0/3278731475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c580ad1d0 0x7f7c5815d930 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c5de42700 1 --2- 172.21.15.112:0/3278731475 >> v2:172.21.15.138:3300/0 conn(0x7f7c58146b60 0x7f7c5806ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.963+0000 7f7c5de42700 1 -- 172.21.15.112:0/3278731475 >> 172.21.15.112:0/3278731475 conn(0x7f7c580cc240 msgr2=0x7f7c580cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.964+0000 7f7c5de42700 1 -- 172.21.15.112:0/3278731475 shutdown_connections 2023-08-27T11:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.964+0000 7f7c5de42700 1 -- 172.21.15.112:0/3278731475 wait complete. 2023-08-27T11:29:05.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.965+0000 7f7c5de42700 1 Processor -- start 2023-08-27T11:29:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.965+0000 7f7c5de42700 1 -- start start 2023-08-27T11:29:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c5de42700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c5806e2f0 0x7f7c58156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c5de42700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c580ad1d0 0x7f7c580cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c5de42700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c58146b60 0x7f7c580cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c5de42700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c58156fe0 con 0x7f7c58146b60 2023-08-27T11:29:05.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c5de42700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c58158b60 con 0x7f7c580ad1d0 2023-08-27T11:29:05.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c5de42700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c581f3480 con 0x7f7c5806e2f0 2023-08-27T11:29:05.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c57fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c58146b60 0x7f7c580cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c57fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c58146b60 0x7f7c580cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37558/0 (socket says 172.21.15.112:37558) 2023-08-27T11:29:05.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c57fff700 1 -- 172.21.15.112:0/101277233 learned_addr learned my addr 172.21.15.112:0/101277233 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c577fe700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c5806e2f0 0x7f7c58156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c56ffd700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c580ad1d0 0x7f7c580cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c57fff700 1 -- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c5806e2f0 msgr2=0x7f7c58156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c57fff700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c5806e2f0 0x7f7c58156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c57fff700 1 -- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c580ad1d0 msgr2=0x7f7c580cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:05.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c57fff700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c580ad1d0 0x7f7c580cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:05.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c57fff700 1 -- 172.21.15.112:0/101277233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4c0094a0 con 0x7f7c58146b60 2023-08-27T11:29:05.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c57fff700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c58146b60 0x7f7c580cdcd0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f7c50014040 tx=0x7f7c50004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:05.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c3ffff700 1 -- 172.21.15.112:0/101277233 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50004800 con 0x7f7c58146b60 2023-08-27T11:29:05.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c5de42700 1 -- 172.21.15.112:0/101277233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c50009a70 con 0x7f7c58146b60 2023-08-27T11:29:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c3ffff700 1 -- 172.21.15.112:0/101277233 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c50020070 con 0x7f7c58146b60 2023-08-27T11:29:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c3ffff700 1 -- 172.21.15.112:0/101277233 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500276c0 con 0x7f7c58146b60 2023-08-27T11:29:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.966+0000 7f7c5de42700 1 -- 172.21.15.112:0/101277233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c581f3180 con 0x7f7c58146b60 2023-08-27T11:29:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.967+0000 7f7c5de42700 1 -- 172.21.15.112:0/101277233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c58067e90 con 0x7f7c58146b60 2023-08-27T11:29:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.968+0000 7f7c3ffff700 1 -- 172.21.15.112:0/101277233 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f7c50012430 con 0x7f7c58146b60 2023-08-27T11:29:05.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.968+0000 7f7c3ffff700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c4406a4d0 0x7f7c4406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:05.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.968+0000 7f7c3ffff700 1 -- 172.21.15.112:0/101277233 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f7c50022070 con 0x7f7c58146b60 2023-08-27T11:29:05.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.968+0000 7f7c577fe700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c4406a4d0 0x7f7c4406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:05.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.969+0000 7f7c577fe700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c4406a4d0 0x7f7c4406c990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7c58145e00 tx=0x7f7c4c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:05.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:05.972+0000 7f7c3ffff700 1 -- 172.21.15.112:0/101277233 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7c500559a0 con 0x7f7c58146b60 2023-08-27T11:29:06.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.159+0000 7f7c5de42700 1 -- 172.21.15.112:0/101277233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7c58067a20 con 0x7f7c58146b60 2023-08-27T11:29:06.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.160+0000 7f7c3ffff700 1 -- 172.21.15.112:0/101277233 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v155) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7c50058fc0 con 0x7f7c58146b60 2023-08-27T11:29:06.161 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:29:06.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.162+0000 7f7c5de42700 1 -- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c4406a4d0 msgr2=0x7f7c4406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.162+0000 7f7c5de42700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c4406a4d0 0x7f7c4406c990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7c58145e00 tx=0x7f7c4c009210 comp rx=0 tx=0).stop 2023-08-27T11:29:06.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.162+0000 7f7c5de42700 1 -- 172.21.15.112:0/101277233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c58146b60 msgr2=0x7f7c580cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.162+0000 7f7c5de42700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c58146b60 0x7f7c580cdcd0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f7c50014040 tx=0x7f7c50004ad0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.162+0000 7f7c5de42700 1 -- 172.21.15.112:0/101277233 shutdown_connections 2023-08-27T11:29:06.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.162+0000 7f7c5de42700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7c4406a4d0 0x7f7c4406c990 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.162+0000 7f7c5de42700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c58146b60 0x7f7c580cdcd0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.162+0000 7f7c5de42700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c5806e2f0 0x7f7c58156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.162+0000 7f7c5de42700 1 --2- 172.21.15.112:0/101277233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c580ad1d0 0x7f7c580cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.162+0000 7f7c5de42700 1 -- 172.21.15.112:0/101277233 >> 172.21.15.112:0/101277233 conn(0x7f7c580cc240 msgr2=0x7f7c581431b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:06.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.162+0000 7f7c5de42700 1 -- 172.21.15.112:0/101277233 shutdown_connections 2023-08-27T11:29:06.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.162+0000 7f7c5de42700 1 -- 172.21.15.112:0/101277233 wait complete. 2023-08-27T11:29:06.174 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:29:06.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.296+0000 7f123b934700 1 Processor -- start 2023-08-27T11:29:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.296+0000 7f123b934700 1 -- start start 2023-08-27T11:29:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f123b934700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f123415d300 0x7f123406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f123b934700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f123406d580 0x7f123406da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f123b934700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f12340b4be0 0x7f12340b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f123b934700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12340ae4f0 con 0x7f123406d580 2023-08-27T11:29:06.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f123b934700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123406df40 con 0x7f12340b4be0 2023-08-27T11:29:06.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f123b934700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12340b55b0 con 0x7f123415d300 2023-08-27T11:29:06.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f1238ecf700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f123406d580 0x7f123406da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f1238ecf700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f123406d580 0x7f123406da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37562/0 (socket says 172.21.15.112:37562) 2023-08-27T11:29:06.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f1238ecf700 1 -- 172.21.15.112:0/3949463409 learned_addr learned my addr 172.21.15.112:0/3949463409 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:06.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f1239ed1700 1 --2- 172.21.15.112:0/3949463409 >> v2:172.21.15.138:3300/0 conn(0x7f12340b4be0 0x7f12340b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f1238ecf700 1 -- 172.21.15.112:0/3949463409 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f123415d300 msgr2=0x7f123406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f1238ecf700 1 --2- 172.21.15.112:0/3949463409 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f123415d300 0x7f123406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f1238ecf700 1 -- 172.21.15.112:0/3949463409 >> v2:172.21.15.138:3300/0 conn(0x7f12340b4be0 msgr2=0x7f12340b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f1238ecf700 1 --2- 172.21.15.112:0/3949463409 >> v2:172.21.15.138:3300/0 conn(0x7f12340b4be0 0x7f12340b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.297+0000 7f1238ecf700 1 -- 172.21.15.112:0/3949463409 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12340ad8e0 con 0x7f123406d580 2023-08-27T11:29:06.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.298+0000 7f1238ecf700 1 --2- 172.21.15.112:0/3949463409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f123406d580 0x7f123406da00 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f12340d0a90 tx=0x7f1224009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6369631faa6bebb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:06.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.298+0000 7f122bfff700 1 -- 172.21.15.112:0/3949463409 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122400b870 con 0x7f123406d580 2023-08-27T11:29:06.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.298+0000 7f122bfff700 1 -- 172.21.15.112:0/3949463409 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122400b9d0 con 0x7f123406d580 2023-08-27T11:29:06.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.298+0000 7f122bfff700 1 -- 172.21.15.112:0/3949463409 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122400bd60 con 0x7f123406d580 2023-08-27T11:29:06.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.298+0000 7f123b934700 1 -- 172.21.15.112:0/3949463409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f123406d580 msgr2=0x7f123406da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.298+0000 7f123b934700 1 --2- 172.21.15.112:0/3949463409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f123406d580 0x7f123406da00 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f12340d0a90 tx=0x7f1224009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.298+0000 7f123b934700 1 -- 172.21.15.112:0/3949463409 shutdown_connections 2023-08-27T11:29:06.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.298+0000 7f123b934700 1 --2- 172.21.15.112:0/3949463409 >> v2:172.21.15.138:3300/0 conn(0x7f12340b4be0 0x7f12340b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.298+0000 7f123b934700 1 --2- 172.21.15.112:0/3949463409 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f123415d300 0x7f123406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.298+0000 7f123b934700 1 --2- 172.21.15.112:0/3949463409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f123406d580 0x7f123406da00 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.298+0000 7f123b934700 1 -- 172.21.15.112:0/3949463409 >> 172.21.15.112:0/3949463409 conn(0x7f1234066400 msgr2=0x7f12340cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:06.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.298+0000 7f123b934700 1 -- 172.21.15.112:0/3949463409 shutdown_connections 2023-08-27T11:29:06.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.299+0000 7f123b934700 1 -- 172.21.15.112:0/3949463409 wait complete. 2023-08-27T11:29:06.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.300+0000 7f123b934700 1 Processor -- start 2023-08-27T11:29:06.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.300+0000 7f123b934700 1 -- start start 2023-08-27T11:29:06.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.300+0000 7f123b934700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f123406d580 0x7f123414ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.300+0000 7f123b934700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12340b4be0 0x7f123415e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.300+0000 7f123b934700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f123415d300 0x7f123415ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.300+0000 7f123b934700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123414f130 con 0x7f12340b4be0 2023-08-27T11:29:06.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.300+0000 7f123b934700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1234159210 con 0x7f123406d580 2023-08-27T11:29:06.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.300+0000 7f123b934700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12341594f0 con 0x7f123415d300 2023-08-27T11:29:06.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f1238ecf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12340b4be0 0x7f123415e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f1238ecf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12340b4be0 0x7f123415e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37568/0 (socket says 172.21.15.112:37568) 2023-08-27T11:29:06.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f1238ecf700 1 -- 172.21.15.112:0/785339146 learned_addr learned my addr 172.21.15.112:0/785339146 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:06.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f1238ecf700 1 -- 172.21.15.112:0/785339146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f123415d300 msgr2=0x7f123415ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f1238ecf700 1 --2- 172.21.15.112:0/785339146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f123415d300 0x7f123415ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f1238ecf700 1 -- 172.21.15.112:0/785339146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f123406d580 msgr2=0x7f123414ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f1238ecf700 1 --2- 172.21.15.112:0/785339146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f123406d580 0x7f123414ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f1238ecf700 1 -- 172.21.15.112:0/785339146 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1224009550 con 0x7f12340b4be0 2023-08-27T11:29:06.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f1238ecf700 1 --2- 172.21.15.112:0/785339146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12340b4be0 0x7f123415e730 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f12340d0c00 tx=0x7f1224004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:06.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f1229ffb700 1 -- 172.21.15.112:0/785339146 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224004310 con 0x7f12340b4be0 2023-08-27T11:29:06.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f1229ffb700 1 -- 172.21.15.112:0/785339146 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122401f070 con 0x7f12340b4be0 2023-08-27T11:29:06.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f123b934700 1 -- 172.21.15.112:0/785339146 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12341597d0 con 0x7f12340b4be0 2023-08-27T11:29:06.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f1229ffb700 1 -- 172.21.15.112:0/785339146 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12240036a0 con 0x7f12340b4be0 2023-08-27T11:29:06.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.301+0000 7f123b934700 1 -- 172.21.15.112:0/785339146 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1234159d90 con 0x7f12340b4be0 2023-08-27T11:29:06.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.302+0000 7f123b934700 1 -- 172.21.15.112:0/785339146 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1234159f80 con 0x7f12340b4be0 2023-08-27T11:29:06.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.303+0000 7f1229ffb700 1 -- 172.21.15.112:0/785339146 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f1224021030 con 0x7f12340b4be0 2023-08-27T11:29:06.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.303+0000 7f1229ffb700 1 --2- 172.21.15.112:0/785339146 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f121806a4d0 0x7f121806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.303+0000 7f1229ffb700 1 -- 172.21.15.112:0/785339146 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f122408a160 con 0x7f12340b4be0 2023-08-27T11:29:06.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.303+0000 7f12396d0700 1 --2- 172.21.15.112:0/785339146 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f121806a4d0 0x7f121806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.304+0000 7f12396d0700 1 --2- 172.21.15.112:0/785339146 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f121806a4d0 0x7f121806c990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1234156060 tx=0x7f1230006c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:06.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.307+0000 7f1229ffb700 1 -- 172.21.15.112:0/785339146 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f12240554d0 con 0x7f12340b4be0 2023-08-27T11:29:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.499+0000 7f123b934700 1 -- 172.21.15.112:0/785339146 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f12340ade70 con 0x7f12340b4be0 2023-08-27T11:29:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.500+0000 7f1229ffb700 1 -- 172.21.15.112:0/785339146 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v155) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1224058af0 con 0x7f12340b4be0 2023-08-27T11:29:06.501 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:29:06.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.502+0000 7f123b934700 1 -- 172.21.15.112:0/785339146 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f121806a4d0 msgr2=0x7f121806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.502+0000 7f123b934700 1 --2- 172.21.15.112:0/785339146 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f121806a4d0 0x7f121806c990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1234156060 tx=0x7f1230006c80 comp rx=0 tx=0).stop 2023-08-27T11:29:06.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.502+0000 7f123b934700 1 -- 172.21.15.112:0/785339146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12340b4be0 msgr2=0x7f123415e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.502+0000 7f123b934700 1 --2- 172.21.15.112:0/785339146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12340b4be0 0x7f123415e730 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f12340d0c00 tx=0x7f1224004750 comp rx=0 tx=0).stop 2023-08-27T11:29:06.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.502+0000 7f123b934700 1 -- 172.21.15.112:0/785339146 shutdown_connections 2023-08-27T11:29:06.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.502+0000 7f123b934700 1 --2- 172.21.15.112:0/785339146 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f121806a4d0 0x7f121806c990 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.502+0000 7f123b934700 1 --2- 172.21.15.112:0/785339146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f123415d300 0x7f123415ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.502+0000 7f123b934700 1 --2- 172.21.15.112:0/785339146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f123406d580 0x7f123414ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.502+0000 7f123b934700 1 --2- 172.21.15.112:0/785339146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12340b4be0 0x7f123415e730 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.502+0000 7f123b934700 1 -- 172.21.15.112:0/785339146 >> 172.21.15.112:0/785339146 conn(0x7f1234066400 msgr2=0x7f123415fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:06.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.502+0000 7f123b934700 1 -- 172.21.15.112:0/785339146 shutdown_connections 2023-08-27T11:29:06.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.502+0000 7f123b934700 1 -- 172.21.15.112:0/785339146 wait complete. 2023-08-27T11:29:06.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:29:06.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.637+0000 7f38537f1700 1 Processor -- start 2023-08-27T11:29:06.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.637+0000 7f38537f1700 1 -- start start 2023-08-27T11:29:06.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f38537f1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f384c06dcc0 0x7f384c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f38537f1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384c0cfd50 con 0x7f384c1437c0 2023-08-27T11:29:06.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f38537f1700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384c0cf140 con 0x7f384c143ba0 2023-08-27T11:29:06.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f38537f1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384c0cf420 con 0x7f384c06dcc0 2023-08-27T11:29:06.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f385158d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f384c1437c0 0x7f384c0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60190/0 (socket says 172.21.15.112:60190) 2023-08-27T11:29:06.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f385158d700 1 -- 172.21.15.112:0/3397404053 learned_addr learned my addr 172.21.15.112:0/3397404053 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:06.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f3851d8e700 1 --2- 172.21.15.112:0/3397404053 >> v2:172.21.15.138:3300/0 conn(0x7f384c06dcc0 0x7f384c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2767389941 0 0) 0x7f384c0cfd50 con 0x7f384c1437c0 2023-08-27T11:29:06.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38380035b0 con 0x7f384c1437c0 2023-08-27T11:29:06.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1716505499 0 0) 0x7f38380035b0 con 0x7f384c1437c0 2023-08-27T11:29:06.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 >> v2:172.21.15.138:3300/0 conn(0x7f384c06dcc0 msgr2=0x7f384c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:06.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f3843fff700 1 --2- 172.21.15.112:0/3397404053 >> v2:172.21.15.138:3300/0 conn(0x7f384c06dcc0 0x7f384c0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 >> v1:172.21.15.138:6789/0 conn(0x7f384c143ba0 legacy=0x7f384c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c0cf700 con 0x7f384c1437c0 2023-08-27T11:29:06.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f3848001670 con 0x7f384c1437c0 2023-08-27T11:29:06.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 >> v1:172.21.15.112:6789/0 conn(0x7f384c1437c0 legacy=0x7f384c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f3843fff700 1 --2- 172.21.15.112:0/3397404053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3838003a00 0x7f3838003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f3843fff700 1 --2- 172.21.15.112:0/3397404053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3838004850 0x7f3838004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.638+0000 7f3843fff700 1 --2- 172.21.15.112:0/3397404053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3838005210 0x7f383800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c0cf420 con 0x7f3838004850 2023-08-27T11:29:06.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383800c0b0 con 0x7f3838003a00 2023-08-27T11:29:06.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383800c270 con 0x7f3838005210 2023-08-27T11:29:06.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3850d8c700 1 --2- 172.21.15.112:0/3397404053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3838004850 0x7f3838004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f385158d700 1 --2- 172.21.15.112:0/3397404053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3838003a00 0x7f3838003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3851d8e700 1 --2- 172.21.15.112:0/3397404053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3838005210 0x7f383800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3850d8c700 1 -- 172.21.15.112:0/3397404053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3838005210 msgr2=0x7f383800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:06.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3850d8c700 1 --2- 172.21.15.112:0/3397404053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3838005210 0x7f383800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3850d8c700 1 -- 172.21.15.112:0/3397404053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3838003a00 msgr2=0x7f3838003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:06.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3850d8c700 1 --2- 172.21.15.112:0/3397404053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3838003a00 0x7f3838003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3850d8c700 1 -- 172.21.15.112:0/3397404053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380035b0 con 0x7f3838004850 2023-08-27T11:29:06.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3851d8e700 1 -- 172.21.15.112:0/3397404053 reap_dead start 2023-08-27T11:29:06.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3850d8c700 1 --2- 172.21.15.112:0/3397404053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3838004850 0x7f3838004cd0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f383c0017c0 tx=0x7f3838005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00aac0fb92562ed server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:06.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c0052d0 con 0x7f3838004850 2023-08-27T11:29:06.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c005430 con 0x7f3838004850 2023-08-27T11:29:06.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.639+0000 7f3843fff700 1 -- 172.21.15.112:0/3397404053 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c0038c0 con 0x7f3838004850 2023-08-27T11:29:06.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.640+0000 7f38537f1700 1 -- 172.21.15.112:0/3397404053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3838004850 msgr2=0x7f3838004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.640+0000 7f38537f1700 1 --2- 172.21.15.112:0/3397404053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3838004850 0x7f3838004cd0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f383c0017c0 tx=0x7f3838005880 comp rx=0 tx=0).stop 2023-08-27T11:29:06.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.640+0000 7f38537f1700 1 -- 172.21.15.112:0/3397404053 shutdown_connections 2023-08-27T11:29:06.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.640+0000 7f38537f1700 1 --2- 172.21.15.112:0/3397404053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3838004850 0x7f3838004cd0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.640+0000 7f38537f1700 1 -- 172.21.15.112:0/3397404053 >> 172.21.15.112:0/3397404053 conn(0x7f384c0b1e90 msgr2=0x7f384c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:06.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.640+0000 7f38537f1700 1 -- 172.21.15.112:0/3397404053 shutdown_connections 2023-08-27T11:29:06.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.640+0000 7f38537f1700 1 -- 172.21.15.112:0/3397404053 wait complete. 2023-08-27T11:29:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.641+0000 7f38537f1700 1 Processor -- start 2023-08-27T11:29:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.641+0000 7f38537f1700 1 -- start start 2023-08-27T11:29:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.642+0000 7f38537f1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3838004850 0x7f384c15ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.642+0000 7f38537f1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f384c137220 0x7f384c135800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.642+0000 7f38537f1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f384c141d90 0x7f384c135d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.642+0000 7f38537f1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c15a1d0 con 0x7f3838004850 2023-08-27T11:29:06.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.642+0000 7f38537f1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c133170 con 0x7f384c141d90 2023-08-27T11:29:06.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.642+0000 7f38537f1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384c132280 con 0x7f384c137220 2023-08-27T11:29:06.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.642+0000 7f385158d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3838004850 0x7f384c15ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.642+0000 7f385158d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3838004850 0x7f384c15ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37588/0 (socket says 172.21.15.112:37588) 2023-08-27T11:29:06.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.642+0000 7f385158d700 1 -- 172.21.15.112:0/3923368398 learned_addr learned my addr 172.21.15.112:0/3923368398 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:06.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.643+0000 7f3851d8e700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f384c141d90 0x7f384c135d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.643+0000 7f385158d700 1 -- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f384c137220 msgr2=0x7f384c135800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.643+0000 7f3850d8c700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f384c137220 0x7f384c135800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.643+0000 7f385158d700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f384c137220 0x7f384c135800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.643+0000 7f385158d700 1 -- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f384c141d90 msgr2=0x7f384c135d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.643+0000 7f385158d700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f384c141d90 0x7f384c135d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.643+0000 7f385158d700 1 -- 172.21.15.112:0/3923368398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c132560 con 0x7f3838004850 2023-08-27T11:29:06.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.643+0000 7f385158d700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3838004850 0x7f384c15ac00 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f38480072c0 tx=0x7f3848007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:06.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.643+0000 7f3841ffb700 1 -- 172.21.15.112:0/3923368398 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480014c0 con 0x7f3838004850 2023-08-27T11:29:06.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.643+0000 7f3841ffb700 1 -- 172.21.15.112:0/3923368398 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384800f8d0 con 0x7f3838004850 2023-08-27T11:29:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.643+0000 7f3841ffb700 1 -- 172.21.15.112:0/3923368398 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384800fc80 con 0x7f3838004850 2023-08-27T11:29:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.643+0000 7f38537f1700 1 -- 172.21.15.112:0/3923368398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384c132840 con 0x7f3838004850 2023-08-27T11:29:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.643+0000 7f38537f1700 1 -- 172.21.15.112:0/3923368398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384c132e00 con 0x7f3838004850 2023-08-27T11:29:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.644+0000 7f38537f1700 1 -- 172.21.15.112:0/3923368398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384c132f90 con 0x7f3838004850 2023-08-27T11:29:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.645+0000 7f3841ffb700 1 -- 172.21.15.112:0/3923368398 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f384801d460 con 0x7f3838004850 2023-08-27T11:29:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.645+0000 7f3841ffb700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f383006a4d0 0x7f383006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.645+0000 7f3841ffb700 1 -- 172.21.15.112:0/3923368398 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f3848088c50 con 0x7f3838004850 2023-08-27T11:29:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.645+0000 7f3850d8c700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f383006a4d0 0x7f383006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.646+0000 7f3850d8c700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f383006a4d0 0x7f383006c990 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f384c139e70 tx=0x7f383c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.649+0000 7f3841ffb700 1 -- 172.21.15.112:0/3923368398 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3848053ea0 con 0x7f3838004850 2023-08-27T11:29:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.838+0000 7f38537f1700 1 -- 172.21.15.112:0/3923368398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f384c067e90 con 0x7f3838004850 2023-08-27T11:29:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.838+0000 7f3841ffb700 1 -- 172.21.15.112:0/3923368398 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v155) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38480574c0 con 0x7f3838004850 2023-08-27T11:29:06.839 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:29:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.840+0000 7f38537f1700 1 -- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f383006a4d0 msgr2=0x7f383006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.840+0000 7f38537f1700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f383006a4d0 0x7f383006c990 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f384c139e70 tx=0x7f383c009030 comp rx=0 tx=0).stop 2023-08-27T11:29:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.840+0000 7f38537f1700 1 -- 172.21.15.112:0/3923368398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3838004850 msgr2=0x7f384c15ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.840+0000 7f38537f1700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3838004850 0x7f384c15ac00 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f38480072c0 tx=0x7f3848007680 comp rx=0 tx=0).stop 2023-08-27T11:29:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.840+0000 7f38537f1700 1 -- 172.21.15.112:0/3923368398 shutdown_connections 2023-08-27T11:29:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.840+0000 7f38537f1700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f383006a4d0 0x7f383006c990 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.840+0000 7f38537f1700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f384c141d90 0x7f384c135d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.840+0000 7f38537f1700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3838004850 0x7f384c15ac00 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.840+0000 7f38537f1700 1 --2- 172.21.15.112:0/3923368398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f384c137220 0x7f384c135800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.840+0000 7f38537f1700 1 -- 172.21.15.112:0/3923368398 >> 172.21.15.112:0/3923368398 conn(0x7f384c0b1e90 msgr2=0x7f384c0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.840+0000 7f38537f1700 1 -- 172.21.15.112:0/3923368398 shutdown_connections 2023-08-27T11:29:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.840+0000 7f38537f1700 1 -- 172.21.15.112:0/3923368398 wait complete. 2023-08-27T11:29:06.852 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:29:06.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.975+0000 7f764943a700 1 Processor -- start 2023-08-27T11:29:06.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.975+0000 7f764943a700 1 -- start start 2023-08-27T11:29:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.975+0000 7f764943a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f764406d060 0x7f76440ca860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.975+0000 7f764943a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f764406d440 0x7f76440b2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.975+0000 7f764943a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76440b2a50 con 0x7f7644159c70 2023-08-27T11:29:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f764943a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76440caf40 con 0x7f764406d060 2023-08-27T11:29:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f764943a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76440cb220 con 0x7f764406d440 2023-08-27T11:29:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f7642ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f764406d060 0x7f76440ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f76437fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7644159c70 0x7f764415a0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60214/0 (socket says 172.21.15.112:60214) 2023-08-27T11:29:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f76437fe700 1 -- 172.21.15.112:0/2916550701 learned_addr learned my addr 172.21.15.112:0/2916550701 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f76427fc700 1 --2- 172.21.15.112:0/2916550701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f764406d440 0x7f76440b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f7642ffd700 1 -- 172.21.15.112:0/2916550701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f764406d440 msgr2=0x7f76440b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f7642ffd700 1 --2- 172.21.15.112:0/2916550701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f764406d440 0x7f76440b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f7642ffd700 1 -- 172.21.15.112:0/2916550701 >> v1:172.21.15.112:6789/0 conn(0x7f7644159c70 legacy=0x7f764415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f7642ffd700 1 -- 172.21.15.112:0/2916550701 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76440cb500 con 0x7f764406d060 2023-08-27T11:29:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f76427fc700 1 --2- 172.21.15.112:0/2916550701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f764406d440 0x7f76440b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f7641ffb700 1 -- 172.21.15.112:0/2916550701 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421554479 0 0) 0x7f76440b2a50 con 0x7f7644159c70 2023-08-27T11:29:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f7642ffd700 1 --2- 172.21.15.112:0/2916550701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764406d060 0x7f76440ca860 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f7634009840 tx=0x7f7634009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676849c041e7160 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f7641ffb700 1 -- 172.21.15.112:0/2916550701 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634011830 con 0x7f764406d060 2023-08-27T11:29:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f7641ffb700 1 -- 172.21.15.112:0/2916550701 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7634011990 con 0x7f764406d060 2023-08-27T11:29:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.976+0000 7f7641ffb700 1 -- 172.21.15.112:0/2916550701 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634011d20 con 0x7f764406d060 2023-08-27T11:29:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.977+0000 7f764943a700 1 -- 172.21.15.112:0/2916550701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764406d060 msgr2=0x7f76440ca860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.977+0000 7f764943a700 1 --2- 172.21.15.112:0/2916550701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764406d060 0x7f76440ca860 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f7634009840 tx=0x7f7634009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:06.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.977+0000 7f764943a700 1 -- 172.21.15.112:0/2916550701 shutdown_connections 2023-08-27T11:29:06.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.977+0000 7f764943a700 1 --2- 172.21.15.112:0/2916550701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764406d060 0x7f76440ca860 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.977+0000 7f764943a700 1 --2- 172.21.15.112:0/2916550701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f764406d440 0x7f76440b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.977+0000 7f764943a700 1 -- 172.21.15.112:0/2916550701 >> 172.21.15.112:0/2916550701 conn(0x7f76440cef70 msgr2=0x7f76440cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.977+0000 7f764943a700 1 -- 172.21.15.112:0/2916550701 shutdown_connections 2023-08-27T11:29:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.977+0000 7f764943a700 1 -- 172.21.15.112:0/2916550701 wait complete. 2023-08-27T11:29:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.978+0000 7f764943a700 1 Processor -- start 2023-08-27T11:29:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.978+0000 7f764943a700 1 -- start start 2023-08-27T11:29:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f764943a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764406d060 0x7f7644147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f764943a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f764406d440 0x7f7644148390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f764943a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644159c70 0x7f76441488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f764943a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7644152c70 con 0x7f7644159c70 2023-08-27T11:29:06.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f764943a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764415c3e0 con 0x7f764406d060 2023-08-27T11:29:06.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f764943a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7644152060 con 0x7f764406d440 2023-08-27T11:29:06.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f76437fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644159c70 0x7f76441488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f76437fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644159c70 0x7f76441488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37606/0 (socket says 172.21.15.112:37606) 2023-08-27T11:29:06.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f76437fe700 1 -- 172.21.15.112:0/1892243833 learned_addr learned my addr 172.21.15.112:0/1892243833 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:06.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f76427fc700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f764406d440 0x7f7644148390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f7642ffd700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764406d060 0x7f7644147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f76437fe700 1 -- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f764406d440 msgr2=0x7f7644148390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f76437fe700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f764406d440 0x7f7644148390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f76437fe700 1 -- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764406d060 msgr2=0x7f7644147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:06.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f76437fe700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764406d060 0x7f7644147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:06.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f76437fe700 1 -- 172.21.15.112:0/1892243833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76340094a0 con 0x7f7644159c70 2023-08-27T11:29:06.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.979+0000 7f76437fe700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644159c70 0x7f76441488d0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f763c00cb10 tx=0x7f763c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:06.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.980+0000 7f762b7fe700 1 -- 172.21.15.112:0/1892243833 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c00ab00 con 0x7f7644159c70 2023-08-27T11:29:06.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.980+0000 7f762b7fe700 1 -- 172.21.15.112:0/1892243833 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c00ac80 con 0x7f7644159c70 2023-08-27T11:29:06.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.980+0000 7f764943a700 1 -- 172.21.15.112:0/1892243833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7644152340 con 0x7f7644159c70 2023-08-27T11:29:06.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.980+0000 7f762b7fe700 1 -- 172.21.15.112:0/1892243833 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c0070d0 con 0x7f7644159c70 2023-08-27T11:29:06.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.980+0000 7f764943a700 1 -- 172.21.15.112:0/1892243833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7644152900 con 0x7f7644159c70 2023-08-27T11:29:06.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.981+0000 7f764943a700 1 -- 172.21.15.112:0/1892243833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764415ae50 con 0x7f7644159c70 2023-08-27T11:29:06.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.981+0000 7f762b7fe700 1 -- 172.21.15.112:0/1892243833 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f763c007230 con 0x7f7644159c70 2023-08-27T11:29:06.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.981+0000 7f762b7fe700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f763006a4d0 0x7f763006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:06.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.981+0000 7f762b7fe700 1 -- 172.21.15.112:0/1892243833 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f763c088a90 con 0x7f7644159c70 2023-08-27T11:29:06.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.982+0000 7f7642ffd700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f763006a4d0 0x7f763006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:06.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.982+0000 7f7642ffd700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f763006a4d0 0x7f763006c990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f76340090b0 tx=0x7f7634008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:06.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:06.986+0000 7f762b7fe700 1 -- 172.21.15.112:0/1892243833 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f764415ae50 con 0x7f7644159c70 2023-08-27T11:29:07.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.175+0000 7f764943a700 1 -- 172.21.15.112:0/1892243833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f764415ae50 con 0x7f7644159c70 2023-08-27T11:29:07.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.176+0000 7f762b7fe700 1 -- 172.21.15.112:0/1892243833 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v155) v1 ==== 100+0+11 (secure 0 0 0) 0x7f764415ae50 con 0x7f7644159c70 2023-08-27T11:29:07.177 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:29:07.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.178+0000 7f764943a700 1 -- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f763006a4d0 msgr2=0x7f763006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:07.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.178+0000 7f764943a700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f763006a4d0 0x7f763006c990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f76340090b0 tx=0x7f7634008030 comp rx=0 tx=0).stop 2023-08-27T11:29:07.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.178+0000 7f764943a700 1 -- 172.21.15.112:0/1892243833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644159c70 msgr2=0x7f76441488d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:07.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.178+0000 7f764943a700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644159c70 0x7f76441488d0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f763c00cb10 tx=0x7f763c00ced0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.178+0000 7f764943a700 1 -- 172.21.15.112:0/1892243833 shutdown_connections 2023-08-27T11:29:07.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.178+0000 7f764943a700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f763006a4d0 0x7f763006c990 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.178+0000 7f764943a700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644159c70 0x7f76441488d0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.178+0000 7f764943a700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764406d060 0x7f7644147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.178+0000 7f764943a700 1 --2- 172.21.15.112:0/1892243833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f764406d440 0x7f7644148390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.178+0000 7f764943a700 1 -- 172.21.15.112:0/1892243833 >> 172.21.15.112:0/1892243833 conn(0x7f76440cef70 msgr2=0x7f76440c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.178+0000 7f764943a700 1 -- 172.21.15.112:0/1892243833 shutdown_connections 2023-08-27T11:29:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.178+0000 7f764943a700 1 -- 172.21.15.112:0/1892243833 wait complete. 2023-08-27T11:29:07.190 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:07.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.309+0000 7f26d1ba1700 1 Processor -- start 2023-08-27T11:29:07.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.309+0000 7f26d1ba1700 1 -- start start 2023-08-27T11:29:07.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26d1ba1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f26cc1466c0 0x7f26cc06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:07.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26d1ba1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f26cc0b15c0 0x7f26cc06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:07.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26d1ba1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26cc161030 con 0x7f26cc0b4760 2023-08-27T11:29:07.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26d1ba1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26cc0b57e0 con 0x7f26cc0b15c0 2023-08-27T11:29:07.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26d1ba1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26cc161310 con 0x7f26cc1466c0 2023-08-27T11:29:07.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26cbfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f26cc0b15c0 0x7f26cc06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:07.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26caffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f26cc0b4760 0x7f26cc06d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60224/0 (socket says 172.21.15.112:60224) 2023-08-27T11:29:07.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26cbfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f26cc0b15c0 0x7f26cc06fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37614/0 (socket says 172.21.15.112:37614) 2023-08-27T11:29:07.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26caffd700 1 -- 172.21.15.112:0/552538010 learned_addr learned my addr 172.21.15.112:0/552538010 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:07.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26cb7fe700 1 --2- 172.21.15.112:0/552538010 >> v2:172.21.15.138:3300/0 conn(0x7f26cc1466c0 0x7f26cc06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:07.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26cbfff700 1 -- 172.21.15.112:0/552538010 >> v2:172.21.15.138:3300/0 conn(0x7f26cc1466c0 msgr2=0x7f26cc06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:07.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26cbfff700 1 --2- 172.21.15.112:0/552538010 >> v2:172.21.15.138:3300/0 conn(0x7f26cc1466c0 0x7f26cc06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26cbfff700 1 -- 172.21.15.112:0/552538010 >> v1:172.21.15.112:6789/0 conn(0x7f26cc0b4760 legacy=0x7f26cc06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26ca7fc700 1 -- 172.21.15.112:0/552538010 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064433367 0 0) 0x7f26cc161030 con 0x7f26cc0b4760 2023-08-27T11:29:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.310+0000 7f26cbfff700 1 -- 172.21.15.112:0/552538010 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc1615f0 con 0x7f26cc0b15c0 2023-08-27T11:29:07.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.311+0000 7f26cbfff700 1 --2- 172.21.15.112:0/552538010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26cc0b15c0 0x7f26cc06fcd0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0d0e70 tx=0x7f26c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb4152c08e5fd90 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:07.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.311+0000 7f26ca7fc700 1 -- 172.21.15.112:0/552538010 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c400e070 con 0x7f26cc0b15c0 2023-08-27T11:29:07.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.311+0000 7f26ca7fc700 1 -- 172.21.15.112:0/552538010 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c400bbd0 con 0x7f26cc0b15c0 2023-08-27T11:29:07.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.311+0000 7f26ca7fc700 1 -- 172.21.15.112:0/552538010 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4012430 con 0x7f26cc0b15c0 2023-08-27T11:29:07.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.311+0000 7f26d1ba1700 1 -- 172.21.15.112:0/552538010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26cc0b15c0 msgr2=0x7f26cc06fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:07.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.311+0000 7f26d1ba1700 1 --2- 172.21.15.112:0/552538010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26cc0b15c0 0x7f26cc06fcd0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0d0e70 tx=0x7f26c400b330 comp rx=0 tx=0).stop 2023-08-27T11:29:07.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.311+0000 7f26d1ba1700 1 -- 172.21.15.112:0/552538010 shutdown_connections 2023-08-27T11:29:07.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.311+0000 7f26d1ba1700 1 --2- 172.21.15.112:0/552538010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26cc0b15c0 0x7f26cc06fcd0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.311+0000 7f26d1ba1700 1 --2- 172.21.15.112:0/552538010 >> v2:172.21.15.138:3300/0 conn(0x7f26cc1466c0 0x7f26cc06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.311+0000 7f26d1ba1700 1 -- 172.21.15.112:0/552538010 >> 172.21.15.112:0/552538010 conn(0x7f26cc0cff80 msgr2=0x7f26cc0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:07.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.311+0000 7f26d1ba1700 1 -- 172.21.15.112:0/552538010 shutdown_connections 2023-08-27T11:29:07.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.311+0000 7f26d1ba1700 1 -- 172.21.15.112:0/552538010 wait complete. 2023-08-27T11:29:07.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.312+0000 7f26d1ba1700 1 Processor -- start 2023-08-27T11:29:07.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.313+0000 7f26d1ba1700 1 -- start start 2023-08-27T11:29:07.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.313+0000 7f26d1ba1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26cc0b15c0 0x7f26cc0af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:07.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.313+0000 7f26d1ba1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0b4760 0x7f26cc1337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:07.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.313+0000 7f26d1ba1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26cc1466c0 0x7f26cc133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:07.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.313+0000 7f26d1ba1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc134230 con 0x7f26cc1466c0 2023-08-27T11:29:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.313+0000 7f26d1ba1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc1f62d0 con 0x7f26cc0b15c0 2023-08-27T11:29:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.313+0000 7f26d1ba1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc144c10 con 0x7f26cc0b4760 2023-08-27T11:29:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.313+0000 7f26cbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26cc1466c0 0x7f26cc133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26cbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26cc1466c0 0x7f26cc133cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37620/0 (socket says 172.21.15.112:37620) 2023-08-27T11:29:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26cbfff700 1 -- 172.21.15.112:0/3874568443 learned_addr learned my addr 172.21.15.112:0/3874568443 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26cb7fe700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26cc0b15c0 0x7f26cc0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26caffd700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0b4760 0x7f26cc1337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26cbfff700 1 -- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0b4760 msgr2=0x7f26cc1337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26cbfff700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0b4760 0x7f26cc1337b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26cbfff700 1 -- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26cc0b15c0 msgr2=0x7f26cc0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26cbfff700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26cc0b15c0 0x7f26cc0af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26cbfff700 1 -- 172.21.15.112:0/3874568443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c4009a70 con 0x7f26cc1466c0 2023-08-27T11:29:07.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26caffd700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0b4760 0x7f26cc1337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:07.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26cbfff700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26cc1466c0 0x7f26cc133cf0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f26c400b730 tx=0x7f26c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:07.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26c3fff700 1 -- 172.21.15.112:0/3874568443 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4012430 con 0x7f26cc1466c0 2023-08-27T11:29:07.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26c3fff700 1 -- 172.21.15.112:0/3874568443 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c4020070 con 0x7f26cc1466c0 2023-08-27T11:29:07.328 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26d1ba1700 1 -- 172.21.15.112:0/3874568443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26cc144ef0 con 0x7f26cc1466c0 2023-08-27T11:29:07.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26c3fff700 1 -- 172.21.15.112:0/3874568443 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4004280 con 0x7f26cc1466c0 2023-08-27T11:29:07.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.314+0000 7f26d1ba1700 1 -- 172.21.15.112:0/3874568443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26cc1454b0 con 0x7f26cc1466c0 2023-08-27T11:29:07.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.315+0000 7f26d1ba1700 1 -- 172.21.15.112:0/3874568443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26cc067a20 con 0x7f26cc1466c0 2023-08-27T11:29:07.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.315+0000 7f26c3fff700 1 -- 172.21.15.112:0/3874568443 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f26c40043e0 con 0x7f26cc1466c0 2023-08-27T11:29:07.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.316+0000 7f26c3fff700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f26ac06a4d0 0x7f26ac06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:07.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.316+0000 7f26c3fff700 1 -- 172.21.15.112:0/3874568443 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f26c4022070 con 0x7f26cc1466c0 2023-08-27T11:29:07.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.316+0000 7f26cb7fe700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f26ac06a4d0 0x7f26ac06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:07.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.317+0000 7f26cb7fe700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f26ac06a4d0 0x7f26ac06c990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f26cc145e80 tx=0x7f26bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:07.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.320+0000 7f26c3fff700 1 -- 172.21.15.112:0/3874568443 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f26c4055db0 con 0x7f26cc1466c0 2023-08-27T11:29:07.365 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:29:07.368 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:29:07.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.555+0000 7f26d1ba1700 1 -- 172.21.15.112:0/3874568443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26cc0cf650 con 0x7f26cc1466c0 2023-08-27T11:29:07.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.556+0000 7f26c3fff700 1 -- 172.21.15.112:0/3874568443 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4891 (secure 0 0 0) 0x7f26c40593d0 con 0x7f26cc1466c0 2023-08-27T11:29:07.557 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:07.557 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":105,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6980,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/877756033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":877756033},{"type":"v1","addr":"172.21.15.138:6837","nonce":877756033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6983,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2624496097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2624496097},{"type":"v1","addr":"172.21.15.138:6836","nonce":2624496097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":8940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2687911818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2687911818},{"type":"v1","addr":"172.21.15.112:6834","nonce":2687911818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:49.824795+0000","modified":"2023-08-27T11:28:53.433019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8943":{"gid":8943,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/577891555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":577891555},{"type":"v1","addr":"172.21.15.112:6835","nonce":577891555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-08-27T11:29:07.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.558+0000 7f26d1ba1700 1 -- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f26ac06a4d0 msgr2=0x7f26ac06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:07.560 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.558+0000 7f26d1ba1700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f26ac06a4d0 0x7f26ac06c990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f26cc145e80 tx=0x7f26bc008040 comp rx=0 tx=0).stop 2023-08-27T11:29:07.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.558+0000 7f26d1ba1700 1 -- 172.21.15.112:0/3874568443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26cc1466c0 msgr2=0x7f26cc133cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:07.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.558+0000 7f26d1ba1700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26cc1466c0 0x7f26cc133cf0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f26c400b730 tx=0x7f26c4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.558+0000 7f26d1ba1700 1 -- 172.21.15.112:0/3874568443 shutdown_connections 2023-08-27T11:29:07.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.558+0000 7f26d1ba1700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f26ac06a4d0 0x7f26ac06c990 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.558+0000 7f26d1ba1700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26cc1466c0 0x7f26cc133cf0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.558+0000 7f26d1ba1700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26cc0b15c0 0x7f26cc0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.558+0000 7f26d1ba1700 1 --2- 172.21.15.112:0/3874568443 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0b4760 0x7f26cc1337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.558+0000 7f26d1ba1700 1 -- 172.21.15.112:0/3874568443 >> 172.21.15.112:0/3874568443 conn(0x7f26cc0cff80 msgr2=0x7f26cc155390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.558+0000 7f26d1ba1700 1 -- 172.21.15.112:0/3874568443 shutdown_connections 2023-08-27T11:29:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.558+0000 7f26d1ba1700 1 -- 172.21.15.112:0/3874568443 wait complete. 2023-08-27T11:29:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 105 2023-08-27T11:29:07.571 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:07.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.690+0000 7f7f022ff700 1 Processor -- start 2023-08-27T11:29:07.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.690+0000 7f7f022ff700 1 -- start start 2023-08-27T11:29:07.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7f022ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7efc144570 0x7f7efc15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:07.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7f022ff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7efc0b3e50 0x7f7efc15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:07.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7f022ff700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7efc0b5ae0 con 0x7f7efc15f8a0 2023-08-27T11:29:07.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7f022ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7efc144950 con 0x7f7efc144570 2023-08-27T11:29:07.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7f022ff700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0b45f0 con 0x7f7efc0b3e50 2023-08-27T11:29:07.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7efbfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7efc144570 0x7f7efc15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:07.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7efbfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7efc144570 0x7f7efc15ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37624/0 (socket says 172.21.15.112:37624) 2023-08-27T11:29:07.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7efbfff700 1 -- 172.21.15.112:0/998902442 learned_addr learned my addr 172.21.15.112:0/998902442 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:07.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7efbfff700 1 -- 172.21.15.112:0/998902442 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7efc0b3e50 msgr2=0x7f7efc15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:07.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7efb7fe700 1 --2- 172.21.15.112:0/998902442 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7efc0b3e50 0x7f7efc15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:07.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7efbfff700 1 --2- 172.21.15.112:0/998902442 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7efc0b3e50 0x7f7efc15f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7efbfff700 1 -- 172.21.15.112:0/998902442 >> v1:172.21.15.138:6789/0 conn(0x7f7efc15f8a0 legacy=0x7f7efc0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:07.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7efbfff700 1 -- 172.21.15.112:0/998902442 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc0b4310 con 0x7f7efc144570 2023-08-27T11:29:07.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.691+0000 7f7efbfff700 1 --2- 172.21.15.112:0/998902442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7efc144570 0x7f7efc15ee20 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00098f0 tx=0x7f7ef0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2404d0425ff6a152 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:07.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.692+0000 7f7efaffd700 1 -- 172.21.15.112:0/998902442 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0004230 con 0x7f7efc144570 2023-08-27T11:29:07.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.692+0000 7f7efaffd700 1 -- 172.21.15.112:0/998902442 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef0004390 con 0x7f7efc144570 2023-08-27T11:29:07.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.692+0000 7f7efaffd700 1 -- 172.21.15.112:0/998902442 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00036a0 con 0x7f7efc144570 2023-08-27T11:29:07.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.692+0000 7f7f022ff700 1 -- 172.21.15.112:0/998902442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7efc144570 msgr2=0x7f7efc15ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:07.696 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.692+0000 7f7f022ff700 1 --2- 172.21.15.112:0/998902442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7efc144570 0x7f7efc15ee20 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00098f0 tx=0x7f7ef0009c00 comp rx=0 tx=0).stop 2023-08-27T11:29:07.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.692+0000 7f7f022ff700 1 -- 172.21.15.112:0/998902442 shutdown_connections 2023-08-27T11:29:07.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.692+0000 7f7f022ff700 1 --2- 172.21.15.112:0/998902442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7efc144570 0x7f7efc15ee20 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.692+0000 7f7f022ff700 1 --2- 172.21.15.112:0/998902442 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7efc0b3e50 0x7f7efc15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.692+0000 7f7f022ff700 1 -- 172.21.15.112:0/998902442 >> 172.21.15.112:0/998902442 conn(0x7f7efc06f970 msgr2=0x7f7efc06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:07.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.692+0000 7f7f022ff700 1 -- 172.21.15.112:0/998902442 shutdown_connections 2023-08-27T11:29:07.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.692+0000 7f7f022ff700 1 -- 172.21.15.112:0/998902442 wait complete. 2023-08-27T11:29:07.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.693+0000 7f7f022ff700 1 Processor -- start 2023-08-27T11:29:07.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.694+0000 7f7f022ff700 1 -- start start 2023-08-27T11:29:07.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.694+0000 7f7f022ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7efc0b3e50 0x7f7efc1f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:07.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.694+0000 7f7f022ff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7efc144570 0x7f7efc0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:07.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.694+0000 7f7f022ff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7efc15f8a0 0x7f7efc1f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:07.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.694+0000 7f7f022ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc1f0c00 con 0x7f7efc0b3e50 2023-08-27T11:29:07.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.694+0000 7f7f022ff700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0cb380 con 0x7f7efc144570 2023-08-27T11:29:07.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.694+0000 7f7f022ff700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0c71c0 con 0x7f7efc15f8a0 2023-08-27T11:29:07.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.694+0000 7f7efbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7efc0b3e50 0x7f7efc1f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:07.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.694+0000 7f7efbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7efc0b3e50 0x7f7efc1f36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37632/0 (socket says 172.21.15.112:37632) 2023-08-27T11:29:07.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.694+0000 7f7efbfff700 1 -- 172.21.15.112:0/4188325373 learned_addr learned my addr 172.21.15.112:0/4188325373 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:07.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.694+0000 7f7efb7fe700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7efc144570 0x7f7efc0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:07.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7f0089c700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7efc15f8a0 0x7f7efc1f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:07.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7efbfff700 1 -- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7efc15f8a0 msgr2=0x7f7efc1f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:07.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7efbfff700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7efc15f8a0 0x7f7efc1f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7efbfff700 1 -- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7efc144570 msgr2=0x7f7efc0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:07.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7efbfff700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7efc144570 0x7f7efc0c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7efbfff700 1 -- 172.21.15.112:0/4188325373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef0009550 con 0x7f7efc0b3e50 2023-08-27T11:29:07.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7efb7fe700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7efc144570 0x7f7efc0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:07.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7f0089c700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7efc15f8a0 0x7f7efc1f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:07.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7efbfff700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7efc0b3e50 0x7f7efc1f36f0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009130 tx=0x7f7ef0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:07.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7ef8ff9700 1 -- 172.21.15.112:0/4188325373 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0003e70 con 0x7f7efc0b3e50 2023-08-27T11:29:07.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7ef8ff9700 1 -- 172.21.15.112:0/4188325373 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef001f070 con 0x7f7efc0b3e50 2023-08-27T11:29:07.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7f022ff700 1 -- 172.21.15.112:0/4188325373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc0c74a0 con 0x7f7efc0b3e50 2023-08-27T11:29:07.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7ef8ff9700 1 -- 172.21.15.112:0/4188325373 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef000b8a0 con 0x7f7efc0b3e50 2023-08-27T11:29:07.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.695+0000 7f7f022ff700 1 -- 172.21.15.112:0/4188325373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc0c7a60 con 0x7f7efc0b3e50 2023-08-27T11:29:07.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.696+0000 7f7f022ff700 1 -- 172.21.15.112:0/4188325373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc067a20 con 0x7f7efc0b3e50 2023-08-27T11:29:07.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.696+0000 7f7ef8ff9700 1 -- 172.21.15.112:0/4188325373 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f7ef0021030 con 0x7f7efc0b3e50 2023-08-27T11:29:07.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.697+0000 7f7ef8ff9700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ee806a4d0 0x7f7ee806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:07.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.697+0000 7f7ef8ff9700 1 -- 172.21.15.112:0/4188325373 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f7ef008a170 con 0x7f7efc0b3e50 2023-08-27T11:29:07.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.697+0000 7f7efb7fe700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ee806a4d0 0x7f7ee806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:07.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.697+0000 7f7efb7fe700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ee806a4d0 0x7f7ee806c990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0cb8b0 tx=0x7f7eec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:07.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.701+0000 7f7ef8ff9700 1 -- 172.21.15.112:0/4188325373 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7ef00553c0 con 0x7f7efc0b3e50 2023-08-27T11:29:07.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.890+0000 7f7f022ff700 1 -- 172.21.15.112:0/4188325373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7efc0b5ae0 con 0x7f7efc0b3e50 2023-08-27T11:29:07.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.891+0000 7f7ef8ff9700 1 -- 172.21.15.112:0/4188325373 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+18412 (secure 0 0 0) 0x7f7efc0b5ae0 con 0x7f7efc0b3e50 2023-08-27T11:29:07.892 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:07.892 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":155,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:55.437349+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.551922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.772403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.7","mappings":[{"from":6,"to":4}]},{"pgid":"28.1a","mappings":[{"from":6,"to":4}]},{"pgid":"28.1d","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:07.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.893+0000 7f7f022ff700 1 -- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ee806a4d0 msgr2=0x7f7ee806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:07.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.893+0000 7f7f022ff700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ee806a4d0 0x7f7ee806c990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0cb8b0 tx=0x7f7eec008040 comp rx=0 tx=0).stop 2023-08-27T11:29:07.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.893+0000 7f7f022ff700 1 -- 172.21.15.112:0/4188325373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7efc0b3e50 msgr2=0x7f7efc1f36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:07.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.893+0000 7f7f022ff700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7efc0b3e50 0x7f7efc1f36f0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009130 tx=0x7f7ef0004750 comp rx=0 tx=0).stop 2023-08-27T11:29:07.894 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.893+0000 7f7f022ff700 1 -- 172.21.15.112:0/4188325373 shutdown_connections 2023-08-27T11:29:07.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.893+0000 7f7f022ff700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7ee806a4d0 0x7f7ee806c990 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.893+0000 7f7f022ff700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7efc15f8a0 0x7f7efc1f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.893+0000 7f7f022ff700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7efc0b3e50 0x7f7efc1f36f0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.893+0000 7f7f022ff700 1 --2- 172.21.15.112:0/4188325373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7efc144570 0x7f7efc0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:07.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.893+0000 7f7f022ff700 1 -- 172.21.15.112:0/4188325373 >> 172.21.15.112:0/4188325373 conn(0x7f7efc06f970 msgr2=0x7f7efc13fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:07.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.893+0000 7f7f022ff700 1 -- 172.21.15.112:0/4188325373 shutdown_connections 2023-08-27T11:29:07.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:07.893+0000 7f7f022ff700 1 -- 172.21.15.112:0/4188325373 wait complete. 2023-08-27T11:29:07.906 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:29:07.908 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:08.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.027+0000 7efe1eadc700 1 Processor -- start 2023-08-27T11:29:08.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.027+0000 7efe1eadc700 1 -- start start 2023-08-27T11:29:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.028+0000 7efe1eadc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe180b17e0 0x7efe180cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.028+0000 7efe1eadc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efe18161ce0 0x7efe1806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.028+0000 7efe1eadc700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe18161850 con 0x7efe180b1400 2023-08-27T11:29:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.028+0000 7efe1eadc700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe180cfc10 con 0x7efe18161ce0 2023-08-27T11:29:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.028+0000 7efe1eadc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe18161200 con 0x7efe180b17e0 2023-08-27T11:29:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.028+0000 7efe1d079700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efe18161ce0 0x7efe1806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.028+0000 7efe1d079700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efe18161ce0 0x7efe1806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37640/0 (socket says 172.21.15.112:37640) 2023-08-27T11:29:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.028+0000 7efe1c878700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7efe180b1400 0x7efe1806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60252/0 (socket says 172.21.15.112:60252) 2023-08-27T11:29:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.028+0000 7efe1d079700 1 -- 172.21.15.112:0/3636925099 learned_addr learned my addr 172.21.15.112:0/3636925099 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.028+0000 7efe17fff700 1 --2- 172.21.15.112:0/3636925099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe180b17e0 0x7efe180cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe1d079700 1 -- 172.21.15.112:0/3636925099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe180b17e0 msgr2=0x7efe180cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe1d079700 1 --2- 172.21.15.112:0/3636925099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe180b17e0 0x7efe180cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe1d079700 1 -- 172.21.15.112:0/3636925099 >> v1:172.21.15.112:6789/0 conn(0x7efe180b1400 legacy=0x7efe1806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe177fe700 1 -- 172.21.15.112:0/3636925099 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623725421 0 0) 0x7efe18161850 con 0x7efe180b1400 2023-08-27T11:29:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe1d079700 1 -- 172.21.15.112:0/3636925099 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe181614e0 con 0x7efe18161ce0 2023-08-27T11:29:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe1d079700 1 --2- 172.21.15.112:0/3636925099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18161ce0 0x7efe1806f790 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7efe10009e50 tx=0x7efe1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd37d556e454a06 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe177fe700 1 -- 172.21.15.112:0/3636925099 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1000e070 con 0x7efe18161ce0 2023-08-27T11:29:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe177fe700 1 -- 172.21.15.112:0/3636925099 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1000b910 con 0x7efe18161ce0 2023-08-27T11:29:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe177fe700 1 -- 172.21.15.112:0/3636925099 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1000bcc0 con 0x7efe18161ce0 2023-08-27T11:29:08.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe1eadc700 1 -- 172.21.15.112:0/3636925099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18161ce0 msgr2=0x7efe1806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe1eadc700 1 --2- 172.21.15.112:0/3636925099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18161ce0 0x7efe1806f790 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7efe10009e50 tx=0x7efe1000b330 comp rx=0 tx=0).stop 2023-08-27T11:29:08.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe1eadc700 1 -- 172.21.15.112:0/3636925099 shutdown_connections 2023-08-27T11:29:08.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe1eadc700 1 --2- 172.21.15.112:0/3636925099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18161ce0 0x7efe1806f790 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe1eadc700 1 --2- 172.21.15.112:0/3636925099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe180b17e0 0x7efe180cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe1eadc700 1 -- 172.21.15.112:0/3636925099 >> 172.21.15.112:0/3636925099 conn(0x7efe180d0000 msgr2=0x7efe180d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:08.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe1eadc700 1 -- 172.21.15.112:0/3636925099 shutdown_connections 2023-08-27T11:29:08.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.029+0000 7efe1eadc700 1 -- 172.21.15.112:0/3636925099 wait complete. 2023-08-27T11:29:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.031+0000 7efe1eadc700 1 Processor -- start 2023-08-27T11:29:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.031+0000 7efe1eadc700 1 -- start start 2023-08-27T11:29:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.032+0000 7efe1eadc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe180b1400 0x7efe180c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.032+0000 7efe1eadc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe180b17e0 0x7efe1815db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.032+0000 7efe1eadc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe18161ce0 0x7efe180c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.032+0000 7efe1eadc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe180c45d0 con 0x7efe180b17e0 2023-08-27T11:29:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.032+0000 7efe1eadc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe18145310 con 0x7efe18161ce0 2023-08-27T11:29:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.032+0000 7efe1eadc700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe180c5d40 con 0x7efe180b1400 2023-08-27T11:29:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.032+0000 7efe17fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe180b17e0 0x7efe1815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.032+0000 7efe17fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe180b17e0 0x7efe1815db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37644/0 (socket says 172.21.15.112:37644) 2023-08-27T11:29:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.032+0000 7efe17fff700 1 -- 172.21.15.112:0/3303132859 learned_addr learned my addr 172.21.15.112:0/3303132859 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.032+0000 7efe1c878700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe180b1400 0x7efe180c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.032+0000 7efe1d079700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe18161ce0 0x7efe180c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.033+0000 7efe17fff700 1 -- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe180b1400 msgr2=0x7efe180c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.033+0000 7efe17fff700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe180b1400 0x7efe180c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.033+0000 7efe17fff700 1 -- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe18161ce0 msgr2=0x7efe180c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.033+0000 7efe17fff700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe18161ce0 0x7efe180c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.033+0000 7efe17fff700 1 -- 172.21.15.112:0/3303132859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe10009b30 con 0x7efe180b17e0 2023-08-27T11:29:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.033+0000 7efe17fff700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe180b17e0 0x7efe1815db70 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7efe0400e9d0 tx=0x7efe0400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.033+0000 7efe157fa700 1 -- 172.21.15.112:0/3303132859 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04009730 con 0x7efe180b17e0 2023-08-27T11:29:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.033+0000 7efe157fa700 1 -- 172.21.15.112:0/3303132859 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0400c850 con 0x7efe180b17e0 2023-08-27T11:29:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.033+0000 7efe1eadc700 1 -- 172.21.15.112:0/3303132859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe180c6020 con 0x7efe180b17e0 2023-08-27T11:29:08.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.033+0000 7efe157fa700 1 -- 172.21.15.112:0/3303132859 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400cbe0 con 0x7efe180b17e0 2023-08-27T11:29:08.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.033+0000 7efe1eadc700 1 -- 172.21.15.112:0/3303132859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe180c6d60 con 0x7efe180b17e0 2023-08-27T11:29:08.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.034+0000 7efe1eadc700 1 -- 172.21.15.112:0/3303132859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe18067a20 con 0x7efe180b17e0 2023-08-27T11:29:08.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.034+0000 7efe157fa700 1 -- 172.21.15.112:0/3303132859 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7efe0400cd40 con 0x7efe180b17e0 2023-08-27T11:29:08.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.035+0000 7efe157fa700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efdfc06a4d0 0x7efdfc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.035+0000 7efe157fa700 1 -- 172.21.15.112:0/3303132859 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7efe04088de0 con 0x7efe180b17e0 2023-08-27T11:29:08.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.035+0000 7efe1c878700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efdfc06a4d0 0x7efdfc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.035+0000 7efe1c878700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efdfc06a4d0 0x7efdfc06c990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7efe0c000f60 tx=0x7efe0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:08.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.039+0000 7efe157fa700 1 -- 172.21.15.112:0/3303132859 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efe040579b0 con 0x7efe180b17e0 2023-08-27T11:29:08.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.270+0000 7efe1eadc700 1 -- 172.21.15.112:0/3303132859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe180ca330 con 0x7efe180b17e0 2023-08-27T11:29:08.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.270+0000 7efe157fa700 1 -- 172.21.15.112:0/3303132859 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4891 (secure 0 0 0) 0x7efe04057540 con 0x7efe180b17e0 2023-08-27T11:29:08.272 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:08.272 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":105,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6980,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/877756033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":877756033},{"type":"v1","addr":"172.21.15.138:6837","nonce":877756033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6983,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2624496097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2624496097},{"type":"v1","addr":"172.21.15.138:6836","nonce":2624496097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":8940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2687911818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2687911818},{"type":"v1","addr":"172.21.15.112:6834","nonce":2687911818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:49.824795+0000","modified":"2023-08-27T11:28:53.433019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8943":{"gid":8943,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/577891555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":577891555},{"type":"v1","addr":"172.21.15.112:6835","nonce":577891555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-08-27T11:29:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.272+0000 7efe1eadc700 1 -- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efdfc06a4d0 msgr2=0x7efdfc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.272+0000 7efe1eadc700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efdfc06a4d0 0x7efdfc06c990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7efe0c000f60 tx=0x7efe0c009040 comp rx=0 tx=0).stop 2023-08-27T11:29:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.272+0000 7efe1eadc700 1 -- 172.21.15.112:0/3303132859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe180b17e0 msgr2=0x7efe1815db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.272+0000 7efe1eadc700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe180b17e0 0x7efe1815db70 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7efe0400e9d0 tx=0x7efe0400ed90 comp rx=0 tx=0).stop 2023-08-27T11:29:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.272+0000 7efe1eadc700 1 -- 172.21.15.112:0/3303132859 shutdown_connections 2023-08-27T11:29:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.272+0000 7efe1eadc700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efdfc06a4d0 0x7efdfc06c990 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.272+0000 7efe1eadc700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe18161ce0 0x7efe180c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.272+0000 7efe1eadc700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe180b1400 0x7efe180c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.272+0000 7efe1eadc700 1 --2- 172.21.15.112:0/3303132859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe180b17e0 0x7efe1815db70 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.272+0000 7efe1eadc700 1 -- 172.21.15.112:0/3303132859 >> 172.21.15.112:0/3303132859 conn(0x7efe180d0000 msgr2=0x7efe180d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.272+0000 7efe1eadc700 1 -- 172.21.15.112:0/3303132859 shutdown_connections 2023-08-27T11:29:08.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.272+0000 7efe1eadc700 1 -- 172.21.15.112:0/3303132859 wait complete. 2023-08-27T11:29:08.276 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 105 2023-08-27T11:29:08.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:08.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.404+0000 7fea8e92d700 1 Processor -- start 2023-08-27T11:29:08.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.404+0000 7fea8e92d700 1 -- start start 2023-08-27T11:29:08.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea8e92d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fea880b1400 0x7fea880cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea8e92d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fea880b17e0 0x7fea8806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea8e92d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea88161ce0 0x7fea8806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea8e92d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea88161850 con 0x7fea880b17e0 2023-08-27T11:29:08.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea8e92d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea8806e7d0 con 0x7fea880b1400 2023-08-27T11:29:08.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea8e92d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea880cfc10 con 0x7fea88161ce0 2023-08-27T11:29:08.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea877fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fea880b17e0 0x7fea8806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea877fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fea880b17e0 0x7fea8806e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37652/0 (socket says 172.21.15.112:37652) 2023-08-27T11:29:08.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea877fe700 1 -- 172.21.15.112:0/1265487103 learned_addr learned my addr 172.21.15.112:0/1265487103 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:08.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea8ceca700 1 --2- 172.21.15.112:0/1265487103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea88161ce0 0x7fea8806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea87fff700 1 --2- 172.21.15.112:0/1265487103 >> v2:172.21.15.138:3300/0 conn(0x7fea880b1400 0x7fea880cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea877fe700 1 -- 172.21.15.112:0/1265487103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea88161ce0 msgr2=0x7fea8806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:08.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea877fe700 1 --2- 172.21.15.112:0/1265487103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea88161ce0 0x7fea8806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea877fe700 1 -- 172.21.15.112:0/1265487103 >> v2:172.21.15.138:3300/0 conn(0x7fea880b1400 msgr2=0x7fea880cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:08.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea877fe700 1 --2- 172.21.15.112:0/1265487103 >> v2:172.21.15.138:3300/0 conn(0x7fea880b1400 0x7fea880cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea877fe700 1 -- 172.21.15.112:0/1265487103 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea88161200 con 0x7fea880b17e0 2023-08-27T11:29:08.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea87fff700 1 --2- 172.21.15.112:0/1265487103 >> v2:172.21.15.138:3300/0 conn(0x7fea880b1400 0x7fea880cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:08.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea8ceca700 1 --2- 172.21.15.112:0/1265487103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea88161ce0 0x7fea8806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:08.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.405+0000 7fea877fe700 1 --2- 172.21.15.112:0/1265487103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea880b17e0 0x7fea8806e290 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fea88070370 tx=0x7fea78009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed4067c889c48d2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:08.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.406+0000 7fea86ffd700 1 -- 172.21.15.112:0/1265487103 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7800b870 con 0x7fea880b17e0 2023-08-27T11:29:08.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.406+0000 7fea86ffd700 1 -- 172.21.15.112:0/1265487103 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7800b9d0 con 0x7fea880b17e0 2023-08-27T11:29:08.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.406+0000 7fea86ffd700 1 -- 172.21.15.112:0/1265487103 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7800bd60 con 0x7fea880b17e0 2023-08-27T11:29:08.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.406+0000 7fea8e92d700 1 -- 172.21.15.112:0/1265487103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea880b17e0 msgr2=0x7fea8806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.406+0000 7fea8e92d700 1 --2- 172.21.15.112:0/1265487103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea880b17e0 0x7fea8806e290 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fea88070370 tx=0x7fea78009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.406+0000 7fea8e92d700 1 -- 172.21.15.112:0/1265487103 shutdown_connections 2023-08-27T11:29:08.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.406+0000 7fea8e92d700 1 --2- 172.21.15.112:0/1265487103 >> v2:172.21.15.138:3300/0 conn(0x7fea880b1400 0x7fea880cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.406+0000 7fea8e92d700 1 --2- 172.21.15.112:0/1265487103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea88161ce0 0x7fea8806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.406+0000 7fea8e92d700 1 --2- 172.21.15.112:0/1265487103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea880b17e0 0x7fea8806e290 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.406+0000 7fea8e92d700 1 -- 172.21.15.112:0/1265487103 >> 172.21.15.112:0/1265487103 conn(0x7fea880d0000 msgr2=0x7fea880d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:08.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.406+0000 7fea8e92d700 1 -- 172.21.15.112:0/1265487103 shutdown_connections 2023-08-27T11:29:08.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.406+0000 7fea8e92d700 1 -- 172.21.15.112:0/1265487103 wait complete. 2023-08-27T11:29:08.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.407+0000 7fea8e92d700 1 Processor -- start 2023-08-27T11:29:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.408+0000 7fea8e92d700 1 -- start start 2023-08-27T11:29:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.408+0000 7fea8e92d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea880b1400 0x7fea880c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.408+0000 7fea8e92d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea880b17e0 0x7fea8806ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.408+0000 7fea8e92d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88161ce0 0x7fea8806b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.408+0000 7fea8e92d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea880c92f0 con 0x7fea88161ce0 2023-08-27T11:29:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.408+0000 7fea8e92d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea880c7e70 con 0x7fea880b17e0 2023-08-27T11:29:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.408+0000 7fea8e92d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea880ca310 con 0x7fea880b1400 2023-08-27T11:29:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.408+0000 7fea8ceca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88161ce0 0x7fea8806b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.408+0000 7fea8ceca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88161ce0 0x7fea8806b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37660/0 (socket says 172.21.15.112:37660) 2023-08-27T11:29:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.408+0000 7fea8ceca700 1 -- 172.21.15.112:0/1676121234 learned_addr learned my addr 172.21.15.112:0/1676121234 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:08.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.409+0000 7fea87fff700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea880b1400 0x7fea880c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.409+0000 7fea877fe700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea880b17e0 0x7fea8806ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.409+0000 7fea8ceca700 1 -- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea880b1400 msgr2=0x7fea880c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.409+0000 7fea8ceca700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea880b1400 0x7fea880c2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.409+0000 7fea8ceca700 1 -- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea880b17e0 msgr2=0x7fea8806ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.409+0000 7fea8ceca700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea880b17e0 0x7fea8806ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.409+0000 7fea8ceca700 1 -- 172.21.15.112:0/1676121234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea78009550 con 0x7fea88161ce0 2023-08-27T11:29:08.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.409+0000 7fea8ceca700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88161ce0 0x7fea8806b260 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fea80007df0 tx=0x7fea8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:08.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.409+0000 7fea84ff9700 1 -- 172.21.15.112:0/1676121234 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8000ddd0 con 0x7fea88161ce0 2023-08-27T11:29:08.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.409+0000 7fea84ff9700 1 -- 172.21.15.112:0/1676121234 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8000f040 con 0x7fea88161ce0 2023-08-27T11:29:08.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.409+0000 7fea84ff9700 1 -- 172.21.15.112:0/1676121234 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80003c60 con 0x7fea88161ce0 2023-08-27T11:29:08.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.409+0000 7fea8e92d700 1 -- 172.21.15.112:0/1676121234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea880c9570 con 0x7fea88161ce0 2023-08-27T11:29:08.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.409+0000 7fea8e92d700 1 -- 172.21.15.112:0/1676121234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea880c9b30 con 0x7fea88161ce0 2023-08-27T11:29:08.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.410+0000 7fea8e92d700 1 -- 172.21.15.112:0/1676121234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea880ca180 con 0x7fea88161ce0 2023-08-27T11:29:08.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.410+0000 7fea84ff9700 1 -- 172.21.15.112:0/1676121234 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fea80004030 con 0x7fea88161ce0 2023-08-27T11:29:08.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.411+0000 7fea84ff9700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fea7406a4d0 0x7fea7406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.411+0000 7fea84ff9700 1 -- 172.21.15.112:0/1676121234 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fea80088f80 con 0x7fea88161ce0 2023-08-27T11:29:08.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.411+0000 7fea87fff700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fea7406a4d0 0x7fea7406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.411+0000 7fea87fff700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fea7406a4d0 0x7fea7406c990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fea8815e450 tx=0x7fea7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:08.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.415+0000 7fea84ff9700 1 -- 172.21.15.112:0/1676121234 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fea80053fb0 con 0x7fea88161ce0 2023-08-27T11:29:08.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.647+0000 7fea8e92d700 1 -- 172.21.15.112:0/1676121234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea88067a20 con 0x7fea88161ce0 2023-08-27T11:29:08.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.648+0000 7fea84ff9700 1 -- 172.21.15.112:0/1676121234 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4891 (secure 0 0 0) 0x7fea800575d0 con 0x7fea88161ce0 2023-08-27T11:29:08.650 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:08.650 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":105,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6980,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/877756033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":877756033},{"type":"v1","addr":"172.21.15.138:6837","nonce":877756033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6983,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2624496097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2624496097},{"type":"v1","addr":"172.21.15.138:6836","nonce":2624496097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":8940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2687911818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2687911818},{"type":"v1","addr":"172.21.15.112:6834","nonce":2687911818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:49.824795+0000","modified":"2023-08-27T11:28:53.433019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8943":{"gid":8943,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/577891555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":577891555},{"type":"v1","addr":"172.21.15.112:6835","nonce":577891555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-08-27T11:29:08.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.650+0000 7fea8e92d700 1 -- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fea7406a4d0 msgr2=0x7fea7406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.650+0000 7fea8e92d700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fea7406a4d0 0x7fea7406c990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fea8815e450 tx=0x7fea7c008040 comp rx=0 tx=0).stop 2023-08-27T11:29:08.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.650+0000 7fea8e92d700 1 -- 172.21.15.112:0/1676121234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88161ce0 msgr2=0x7fea8806b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.650+0000 7fea8e92d700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88161ce0 0x7fea8806b260 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fea80007df0 tx=0x7fea8000d3b0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.650+0000 7fea8e92d700 1 -- 172.21.15.112:0/1676121234 shutdown_connections 2023-08-27T11:29:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.650+0000 7fea8e92d700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fea7406a4d0 0x7fea7406c990 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.650+0000 7fea8e92d700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88161ce0 0x7fea8806b260 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.650+0000 7fea8e92d700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea880b1400 0x7fea880c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.650+0000 7fea8e92d700 1 --2- 172.21.15.112:0/1676121234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea880b17e0 0x7fea8806ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.650+0000 7fea8e92d700 1 -- 172.21.15.112:0/1676121234 >> 172.21.15.112:0/1676121234 conn(0x7fea880d0000 msgr2=0x7fea880d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:08.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.650+0000 7fea8e92d700 1 -- 172.21.15.112:0/1676121234 shutdown_connections 2023-08-27T11:29:08.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.650+0000 7fea8e92d700 1 -- 172.21.15.112:0/1676121234 wait complete. 2023-08-27T11:29:08.654 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 105 2023-08-27T11:29:08.663 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:08.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.782+0000 7f6b17b11700 1 Processor -- start 2023-08-27T11:29:08.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.782+0000 7f6b17b11700 1 -- start start 2023-08-27T11:29:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.783+0000 7f6b17b11700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6b100cddb0 0x7f6b1015f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.783+0000 7f6b17b11700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6b100ce190 0x7f6b1015b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.783+0000 7f6b17b11700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1015eed0 con 0x7f6b1015bdc0 2023-08-27T11:29:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.783+0000 7f6b17b11700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1015f500 con 0x7f6b100cddb0 2023-08-27T11:29:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.783+0000 7f6b17b11700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1015c250 con 0x7f6b100ce190 2023-08-27T11:29:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.783+0000 7f6b158ad700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6b100cddb0 0x7f6b1015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.783+0000 7f6b158ad700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6b100cddb0 0x7f6b1015f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37664/0 (socket says 172.21.15.112:37664) 2023-08-27T11:29:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.783+0000 7f6b158ad700 1 -- 172.21.15.112:0/3387947236 learned_addr learned my addr 172.21.15.112:0/3387947236 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.783+0000 7f6b150ac700 1 --2- 172.21.15.112:0/3387947236 >> v2:172.21.15.138:3300/0 conn(0x7f6b100ce190 0x7f6b1015b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.784+0000 7f6b158ad700 1 -- 172.21.15.112:0/3387947236 >> v2:172.21.15.138:3300/0 conn(0x7f6b100ce190 msgr2=0x7f6b1015b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.784+0000 7f6b158ad700 1 --2- 172.21.15.112:0/3387947236 >> v2:172.21.15.138:3300/0 conn(0x7f6b100ce190 0x7f6b1015b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.784+0000 7f6b158ad700 1 -- 172.21.15.112:0/3387947236 >> v1:172.21.15.138:6789/0 conn(0x7f6b1015bdc0 legacy=0x7f6b100cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.784+0000 7f6b158ad700 1 -- 172.21.15.112:0/3387947236 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1015fe60 con 0x7f6b100cddb0 2023-08-27T11:29:08.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.784+0000 7f6b158ad700 1 --2- 172.21.15.112:0/3387947236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b100cddb0 0x7f6b1015f920 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f6b100ce940 tx=0x7f6b00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94463b434ca30fa6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:08.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.784+0000 7f6b148ab700 1 -- 172.21.15.112:0/3387947236 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0000b870 con 0x7f6b100cddb0 2023-08-27T11:29:08.789 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.784+0000 7f6b148ab700 1 -- 172.21.15.112:0/3387947236 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0000b9d0 con 0x7f6b100cddb0 2023-08-27T11:29:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.784+0000 7f6b148ab700 1 -- 172.21.15.112:0/3387947236 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0000bd60 con 0x7f6b100cddb0 2023-08-27T11:29:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.784+0000 7f6b17b11700 1 -- 172.21.15.112:0/3387947236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b100cddb0 msgr2=0x7f6b1015f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.785+0000 7f6b17b11700 1 --2- 172.21.15.112:0/3387947236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b100cddb0 0x7f6b1015f920 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f6b100ce940 tx=0x7f6b00009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.785+0000 7f6b17b11700 1 -- 172.21.15.112:0/3387947236 shutdown_connections 2023-08-27T11:29:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.785+0000 7f6b17b11700 1 --2- 172.21.15.112:0/3387947236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b100cddb0 0x7f6b1015f920 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.785+0000 7f6b17b11700 1 --2- 172.21.15.112:0/3387947236 >> v2:172.21.15.138:3300/0 conn(0x7f6b100ce190 0x7f6b1015b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.785+0000 7f6b17b11700 1 -- 172.21.15.112:0/3387947236 >> 172.21.15.112:0/3387947236 conn(0x7f6b1006f970 msgr2=0x7f6b1006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:08.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.785+0000 7f6b17b11700 1 -- 172.21.15.112:0/3387947236 shutdown_connections 2023-08-27T11:29:08.791 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.785+0000 7f6b17b11700 1 -- 172.21.15.112:0/3387947236 wait complete. 2023-08-27T11:29:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.786+0000 7f6b17b11700 1 Processor -- start 2023-08-27T11:29:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.786+0000 7f6b17b11700 1 -- start start 2023-08-27T11:29:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.786+0000 7f6b17b11700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b100cddb0 0x7f6b100b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b17b11700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b100ce190 0x7f6b10145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b17b11700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b1015bdc0 0x7f6b1014f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b17b11700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b10146150 con 0x7f6b100ce190 2023-08-27T11:29:08.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b17b11700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100726d0 con 0x7f6b100cddb0 2023-08-27T11:29:08.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b17b11700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1014e8a0 con 0x7f6b1015bdc0 2023-08-27T11:29:08.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b150ac700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b100ce190 0x7f6b10145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b150ac700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b100ce190 0x7f6b10145c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37670/0 (socket says 172.21.15.112:37670) 2023-08-27T11:29:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b150ac700 1 -- 172.21.15.112:0/1768695011 learned_addr learned my addr 172.21.15.112:0/1768695011 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b158ad700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b100cddb0 0x7f6b100b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b160ae700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b1015bdc0 0x7f6b1014f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b150ac700 1 -- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b1015bdc0 msgr2=0x7f6b1014f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b150ac700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b1015bdc0 0x7f6b1014f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b150ac700 1 -- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b100cddb0 msgr2=0x7f6b100b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b150ac700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b100cddb0 0x7f6b100b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b150ac700 1 -- 172.21.15.112:0/1768695011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b00009550 con 0x7f6b100ce190 2023-08-27T11:29:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b158ad700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b100cddb0 0x7f6b100b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b160ae700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b1015bdc0 0x7f6b1014f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b150ac700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b100ce190 0x7f6b10145c10 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e810 tx=0x7f6b0c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b067fc700 1 -- 172.21.15.112:0/1768695011 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c004230 con 0x7f6b100ce190 2023-08-27T11:29:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b067fc700 1 -- 172.21.15.112:0/1768695011 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c004390 con 0x7f6b100ce190 2023-08-27T11:29:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b17b11700 1 -- 172.21.15.112:0/1768695011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1014eb80 con 0x7f6b100ce190 2023-08-27T11:29:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b067fc700 1 -- 172.21.15.112:0/1768695011 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0036a0 con 0x7f6b100ce190 2023-08-27T11:29:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.787+0000 7f6b17b11700 1 -- 172.21.15.112:0/1768695011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1014f8c0 con 0x7f6b100ce190 2023-08-27T11:29:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.788+0000 7f6b17b11700 1 -- 172.21.15.112:0/1768695011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1014f0a0 con 0x7f6b100ce190 2023-08-27T11:29:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.789+0000 7f6b067fc700 1 -- 172.21.15.112:0/1768695011 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f6b0c003800 con 0x7f6b100ce190 2023-08-27T11:29:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.789+0000 7f6b067fc700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6af406a4d0 0x7f6af406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:08.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.789+0000 7f6b067fc700 1 -- 172.21.15.112:0/1768695011 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f6b0c088ca0 con 0x7f6b100ce190 2023-08-27T11:29:08.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.789+0000 7f6b158ad700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6af406a4d0 0x7f6af406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:08.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.790+0000 7f6b158ad700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6af406a4d0 0x7f6af406c990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6b00000c00 tx=0x7f6b00008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:08.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.793+0000 7f6b067fc700 1 -- 172.21.15.112:0/1768695011 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6b1014f0a0 con 0x7f6b100ce190 2023-08-27T11:29:08.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.981+0000 7f6b17b11700 1 -- 172.21.15.112:0/1768695011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b1014f0a0 con 0x7f6b100ce190 2023-08-27T11:29:08.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.982+0000 7f6b067fc700 1 -- 172.21.15.112:0/1768695011 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+18412 (secure 0 0 0) 0x7f6b1014f0a0 con 0x7f6b100ce190 2023-08-27T11:29:08.984 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:08.984 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":155,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:28:55.437349+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.551922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.772403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.7","mappings":[{"from":6,"to":4}]},{"pgid":"28.1a","mappings":[{"from":6,"to":4}]},{"pgid":"28.1d","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:08.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.984+0000 7f6b17b11700 1 -- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6af406a4d0 msgr2=0x7f6af406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.984+0000 7f6b17b11700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6af406a4d0 0x7f6af406c990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6b00000c00 tx=0x7f6b00008030 comp rx=0 tx=0).stop 2023-08-27T11:29:08.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.984+0000 7f6b17b11700 1 -- 172.21.15.112:0/1768695011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b100ce190 msgr2=0x7f6b10145c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:08.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.984+0000 7f6b17b11700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b100ce190 0x7f6b10145c10 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e810 tx=0x7f6b0c00ebd0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.984+0000 7f6b17b11700 1 -- 172.21.15.112:0/1768695011 shutdown_connections 2023-08-27T11:29:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.984+0000 7f6b17b11700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6af406a4d0 0x7f6af406c990 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.984+0000 7f6b17b11700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b1015bdc0 0x7f6b1014f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.984+0000 7f6b17b11700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b100cddb0 0x7f6b100b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.984+0000 7f6b17b11700 1 --2- 172.21.15.112:0/1768695011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b100ce190 0x7f6b10145c10 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.984+0000 7f6b17b11700 1 -- 172.21.15.112:0/1768695011 >> 172.21.15.112:0/1768695011 conn(0x7f6b1006f970 msgr2=0x7f6b1015c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.984+0000 7f6b17b11700 1 -- 172.21.15.112:0/1768695011 shutdown_connections 2023-08-27T11:29:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:08.984+0000 7f6b17b11700 1 -- 172.21.15.112:0/1768695011 wait complete. 2023-08-27T11:29:08.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:29:09.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.117+0000 7fe36c87c700 1 Processor -- start 2023-08-27T11:29:09.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.117+0000 7fe36c87c700 1 -- start start 2023-08-27T11:29:09.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe36c87c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3680b3e50 0x7fe36815f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe36c87c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe3680b4ad0 0x7fe3680b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe36c87c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3680b45f0 con 0x7fe368144570 2023-08-27T11:29:09.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe36c87c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3680cf7a0 con 0x7fe3680b3e50 2023-08-27T11:29:09.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe36c87c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe368144950 con 0x7fe3680b4ad0 2023-08-27T11:29:09.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe365d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3680b3e50 0x7fe36815f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.122 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe36659c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe368144570 0x7fe36815ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60290/0 (socket says 172.21.15.112:60290) 2023-08-27T11:29:09.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe36659c700 1 -- 172.21.15.112:0/3906971674 learned_addr learned my addr 172.21.15.112:0/3906971674 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:09.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe366d9d700 1 --2- 172.21.15.112:0/3906971674 >> v2:172.21.15.138:3300/0 conn(0x7fe3680b4ad0 0x7fe3680b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe36559a700 1 -- 172.21.15.112:0/3906971674 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102771358 0 0) 0x7fe3680b45f0 con 0x7fe368144570 2023-08-27T11:29:09.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe36559a700 1 -- 172.21.15.112:0/3906971674 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3500035b0 con 0x7fe368144570 2023-08-27T11:29:09.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe365d9b700 1 -- 172.21.15.112:0/3906971674 >> v2:172.21.15.138:3300/0 conn(0x7fe3680b4ad0 msgr2=0x7fe3680b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:09.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe365d9b700 1 --2- 172.21.15.112:0/3906971674 >> v2:172.21.15.138:3300/0 conn(0x7fe3680b4ad0 0x7fe3680b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe365d9b700 1 -- 172.21.15.112:0/3906971674 >> v1:172.21.15.112:6789/0 conn(0x7fe368144570 legacy=0x7fe36815ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.118+0000 7fe365d9b700 1 -- 172.21.15.112:0/3906971674 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3680b4310 con 0x7fe3680b3e50 2023-08-27T11:29:09.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.119+0000 7fe365d9b700 1 --2- 172.21.15.112:0/3906971674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3680b3e50 0x7fe36815f530 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fe36806b760 tx=0x7fe358009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49032191b54a057 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:09.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.119+0000 7fe36559a700 1 -- 172.21.15.112:0/3906971674 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358011830 con 0x7fe3680b3e50 2023-08-27T11:29:09.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.119+0000 7fe36559a700 1 -- 172.21.15.112:0/3906971674 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe358011990 con 0x7fe3680b3e50 2023-08-27T11:29:09.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.119+0000 7fe36559a700 1 -- 172.21.15.112:0/3906971674 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358011d20 con 0x7fe3680b3e50 2023-08-27T11:29:09.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.119+0000 7fe36c87c700 1 -- 172.21.15.112:0/3906971674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3680b3e50 msgr2=0x7fe36815f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.119+0000 7fe36c87c700 1 --2- 172.21.15.112:0/3906971674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3680b3e50 0x7fe36815f530 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fe36806b760 tx=0x7fe358009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:09.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.119+0000 7fe36c87c700 1 -- 172.21.15.112:0/3906971674 shutdown_connections 2023-08-27T11:29:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.119+0000 7fe36c87c700 1 --2- 172.21.15.112:0/3906971674 >> v2:172.21.15.138:3300/0 conn(0x7fe3680b4ad0 0x7fe3680b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.119+0000 7fe36c87c700 1 --2- 172.21.15.112:0/3906971674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3680b3e50 0x7fe36815f530 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.119+0000 7fe36c87c700 1 -- 172.21.15.112:0/3906971674 >> 172.21.15.112:0/3906971674 conn(0x7fe36806f970 msgr2=0x7fe36806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.119+0000 7fe36c87c700 1 -- 172.21.15.112:0/3906971674 shutdown_connections 2023-08-27T11:29:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.119+0000 7fe36c87c700 1 -- 172.21.15.112:0/3906971674 wait complete. 2023-08-27T11:29:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.121+0000 7fe36c87c700 1 Processor -- start 2023-08-27T11:29:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.121+0000 7fe36c87c700 1 -- start start 2023-08-27T11:29:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.121+0000 7fe36c87c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3680b3e50 0x7fe36813cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.121+0000 7fe36c87c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3680b4ad0 0x7fe36815b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.121+0000 7fe36c87c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe368144570 0x7fe36815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.121+0000 7fe36c87c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36815c380 con 0x7fe3680b4ad0 2023-08-27T11:29:09.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.121+0000 7fe36c87c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36815c660 con 0x7fe3680b3e50 2023-08-27T11:29:09.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.121+0000 7fe36c87c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36815c940 con 0x7fe368144570 2023-08-27T11:29:09.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe365d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3680b4ad0 0x7fe36815b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.130 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe365d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3680b4ad0 0x7fe36815b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37684/0 (socket says 172.21.15.112:37684) 2023-08-27T11:29:09.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe365d9b700 1 -- 172.21.15.112:0/3161900070 learned_addr learned my addr 172.21.15.112:0/3161900070 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:09.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe36659c700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3680b3e50 0x7fe36813cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe366d9d700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe368144570 0x7fe36815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe365d9b700 1 -- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe368144570 msgr2=0x7fe36815a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe365d9b700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe368144570 0x7fe36815a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe365d9b700 1 -- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3680b3e50 msgr2=0x7fe36813cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe365d9b700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3680b3e50 0x7fe36813cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe365d9b700 1 -- 172.21.15.112:0/3161900070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3580094a0 con 0x7fe3680b4ad0 2023-08-27T11:29:09.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe366d9d700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe368144570 0x7fe36815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:09.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe365d9b700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3680b4ad0 0x7fe36815b8c0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fe3580097c0 tx=0x7fe358004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:09.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe356ffd700 1 -- 172.21.15.112:0/3161900070 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3580036a0 con 0x7fe3680b4ad0 2023-08-27T11:29:09.132 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe356ffd700 1 -- 172.21.15.112:0/3161900070 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35801e070 con 0x7fe3680b4ad0 2023-08-27T11:29:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe36c87c700 1 -- 172.21.15.112:0/3161900070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36815cc20 con 0x7fe3680b4ad0 2023-08-27T11:29:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe356ffd700 1 -- 172.21.15.112:0/3161900070 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358003a60 con 0x7fe3680b4ad0 2023-08-27T11:29:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.122+0000 7fe36c87c700 1 -- 172.21.15.112:0/3161900070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36813d4f0 con 0x7fe3680b4ad0 2023-08-27T11:29:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.123+0000 7fe36c87c700 1 -- 172.21.15.112:0/3161900070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe368067a20 con 0x7fe3680b4ad0 2023-08-27T11:29:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.124+0000 7fe356ffd700 1 -- 172.21.15.112:0/3161900070 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe358003bc0 con 0x7fe3680b4ad0 2023-08-27T11:29:09.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.124+0000 7fe356ffd700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe34806a4d0 0x7fe34806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.124+0000 7fe356ffd700 1 -- 172.21.15.112:0/3161900070 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fe358089660 con 0x7fe3680b4ad0 2023-08-27T11:29:09.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.125+0000 7fe36659c700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe34806a4d0 0x7fe34806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.125+0000 7fe36659c700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe34806a4d0 0x7fe34806c990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe35c007fd0 tx=0x7fe35c007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:09.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.129+0000 7fe356ffd700 1 -- 172.21.15.112:0/3161900070 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe3580548b0 con 0x7fe3680b4ad0 2023-08-27T11:29:09.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.360+0000 7fe36c87c700 1 -- 172.21.15.112:0/3161900070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe36815b0b0 con 0x7fe3680b4ad0 2023-08-27T11:29:09.362 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:29:09.361+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:29:09.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.365+0000 7fe356ffd700 1 -- 172.21.15.112:0/3161900070 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe358057ed0 con 0x7fe3680b4ad0 2023-08-27T11:29:09.368 INFO:tasks.ceph.mds.c.smithi112.stderr: -3185> 2023-08-27T11:28:55.385+0000 7f9fcbf38700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:29:09.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.369+0000 7fe36c87c700 1 -- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe34806a4d0 msgr2=0x7fe34806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.369+0000 7fe36c87c700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe34806a4d0 0x7fe34806c990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe35c007fd0 tx=0x7fe35c007dc0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.369+0000 7fe36c87c700 1 -- 172.21.15.112:0/3161900070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3680b4ad0 msgr2=0x7fe36815b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.369+0000 7fe36c87c700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3680b4ad0 0x7fe36815b8c0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fe3580097c0 tx=0x7fe358004750 comp rx=0 tx=0).stop 2023-08-27T11:29:09.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.369+0000 7fe36c87c700 1 -- 172.21.15.112:0/3161900070 shutdown_connections 2023-08-27T11:29:09.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.369+0000 7fe36c87c700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe34806a4d0 0x7fe34806c990 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.369+0000 7fe36c87c700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe368144570 0x7fe36815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.369+0000 7fe36c87c700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3680b3e50 0x7fe36813cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.369+0000 7fe36c87c700 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3680b4ad0 0x7fe36815b8c0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.369+0000 7fe36c87c700 1 -- 172.21.15.112:0/3161900070 >> 172.21.15.112:0/3161900070 conn(0x7fe36806f970 msgr2=0x7fe3681427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:09.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.369+0000 7fe36c87c700 1 -- 172.21.15.112:0/3161900070 shutdown_connections 2023-08-27T11:29:09.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.369+0000 7fe36c87c700 1 -- 172.21.15.112:0/3161900070 wait complete. 2023-08-27T11:29:09.374 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:29:09.386 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:29:09.399 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:29:09.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.509+0000 7facf65e8700 1 Processor -- start 2023-08-27T11:29:09.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.509+0000 7facf65e8700 1 -- start start 2023-08-27T11:29:09.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facf65e8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7facf015e190 0x7facf015e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facf65e8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7facf0161dd0 0x7facf00b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facf65e8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf00cc980 0x7facf00cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facf65e8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf00cd340 con 0x7facf0161dd0 2023-08-27T11:29:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facf65e8700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf00b46c0 con 0x7facf015e190 2023-08-27T11:29:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facf65e8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf00b49a0 con 0x7facf00cc980 2023-08-27T11:29:09.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facef7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7facf0161dd0 0x7facf00b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facef7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7facf0161dd0 0x7facf00b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37704/0 (socket says 172.21.15.112:37704) 2023-08-27T11:29:09.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facef7fe700 1 -- 172.21.15.112:0/1392217585 learned_addr learned my addr 172.21.15.112:0/1392217585 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:09.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7faceffff700 1 --2- 172.21.15.112:0/1392217585 >> v2:172.21.15.138:3300/0 conn(0x7facf015e190 0x7facf015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facf4b85700 1 --2- 172.21.15.112:0/1392217585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf00cc980 0x7facf00cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facef7fe700 1 -- 172.21.15.112:0/1392217585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf00cc980 msgr2=0x7facf00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facef7fe700 1 --2- 172.21.15.112:0/1392217585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf00cc980 0x7facf00cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facef7fe700 1 -- 172.21.15.112:0/1392217585 >> v2:172.21.15.138:3300/0 conn(0x7facf015e190 msgr2=0x7facf015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facef7fe700 1 --2- 172.21.15.112:0/1392217585 >> v2:172.21.15.138:3300/0 conn(0x7facf015e190 0x7facf015e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facef7fe700 1 -- 172.21.15.112:0/1392217585 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf01612f0 con 0x7facf0161dd0 2023-08-27T11:29:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.510+0000 7facef7fe700 1 --2- 172.21.15.112:0/1392217585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf0161dd0 0x7facf00b3dc0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7facf00d0fb0 tx=0x7face0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e62bfc110ac35ba server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.511+0000 7faceeffd700 1 -- 172.21.15.112:0/1392217585 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0011830 con 0x7facf0161dd0 2023-08-27T11:29:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.511+0000 7faceeffd700 1 -- 172.21.15.112:0/1392217585 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face0011990 con 0x7facf0161dd0 2023-08-27T11:29:09.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.511+0000 7faceeffd700 1 -- 172.21.15.112:0/1392217585 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0011d20 con 0x7facf0161dd0 2023-08-27T11:29:09.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.511+0000 7facf65e8700 1 -- 172.21.15.112:0/1392217585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf0161dd0 msgr2=0x7facf00b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.511+0000 7facf65e8700 1 --2- 172.21.15.112:0/1392217585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf0161dd0 0x7facf00b3dc0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7facf00d0fb0 tx=0x7face0009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.511+0000 7facf65e8700 1 -- 172.21.15.112:0/1392217585 shutdown_connections 2023-08-27T11:29:09.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.511+0000 7facf65e8700 1 --2- 172.21.15.112:0/1392217585 >> v2:172.21.15.138:3300/0 conn(0x7facf015e190 0x7facf015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.511+0000 7facf65e8700 1 --2- 172.21.15.112:0/1392217585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf00cc980 0x7facf00cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.511+0000 7facf65e8700 1 --2- 172.21.15.112:0/1392217585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf0161dd0 0x7facf00b3dc0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.511+0000 7facf65e8700 1 -- 172.21.15.112:0/1392217585 >> 172.21.15.112:0/1392217585 conn(0x7facf006bbf0 msgr2=0x7facf00cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.511+0000 7facf65e8700 1 -- 172.21.15.112:0/1392217585 shutdown_connections 2023-08-27T11:29:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.511+0000 7facf65e8700 1 -- 172.21.15.112:0/1392217585 wait complete. 2023-08-27T11:29:09.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.513+0000 7facf65e8700 1 Processor -- start 2023-08-27T11:29:09.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.513+0000 7facf65e8700 1 -- start start 2023-08-27T11:29:09.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facf65e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf00cc980 0x7facf014e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facf65e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf015e190 0x7facf006f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facf65e8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf0161dd0 0x7facf00cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facf65e8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf014ed00 con 0x7facf015e190 2023-08-27T11:29:09.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facf65e8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf006ff30 con 0x7facf00cc980 2023-08-27T11:29:09.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facf65e8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf0070210 con 0x7facf0161dd0 2023-08-27T11:29:09.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facef7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf015e190 0x7facf006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facef7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf015e190 0x7facf006f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37710/0 (socket says 172.21.15.112:37710) 2023-08-27T11:29:09.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facef7fe700 1 -- 172.21.15.112:0/3415097731 learned_addr learned my addr 172.21.15.112:0/3415097731 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:09.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7faceffff700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf00cc980 0x7facf014e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facf4b85700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf0161dd0 0x7facf00cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facef7fe700 1 -- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf0161dd0 msgr2=0x7facf00cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facef7fe700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf0161dd0 0x7facf00cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facef7fe700 1 -- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf00cc980 msgr2=0x7facf014e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facef7fe700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf00cc980 0x7facf014e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facef7fe700 1 -- 172.21.15.112:0/3415097731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face0009550 con 0x7facf015e190 2023-08-27T11:29:09.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facef7fe700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf015e190 0x7facf006f600 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7face0009c10 tx=0x7face0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:09.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facecff9700 1 -- 172.21.15.112:0/3415097731 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0004450 con 0x7facf015e190 2023-08-27T11:29:09.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facecff9700 1 -- 172.21.15.112:0/3415097731 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face001e070 con 0x7facf015e190 2023-08-27T11:29:09.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facf65e8700 1 -- 172.21.15.112:0/3415097731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf00704f0 con 0x7facf015e190 2023-08-27T11:29:09.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.514+0000 7facecff9700 1 -- 172.21.15.112:0/3415097731 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face00038e0 con 0x7facf015e190 2023-08-27T11:29:09.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.515+0000 7facf65e8700 1 -- 172.21.15.112:0/3415097731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf0070f80 con 0x7facf015e190 2023-08-27T11:29:09.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.516+0000 7facf65e8700 1 -- 172.21.15.112:0/3415097731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf0066fa0 con 0x7facf015e190 2023-08-27T11:29:09.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.516+0000 7facecff9700 1 -- 172.21.15.112:0/3415097731 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7face0003a40 con 0x7facf015e190 2023-08-27T11:29:09.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.516+0000 7facecff9700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7facdc06a4d0 0x7facdc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.516+0000 7facecff9700 1 -- 172.21.15.112:0/3415097731 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6235+0+0 (secure 0 0 0) 0x7face00897e0 con 0x7facf015e190 2023-08-27T11:29:09.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.516+0000 7faceffff700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7facdc06a4d0 0x7facdc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.517+0000 7faceffff700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7facdc06a4d0 0x7facdc06c990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7face4009be0 tx=0x7face4009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:09.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.520+0000 7facecff9700 1 -- 172.21.15.112:0/3415097731 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7face0054ae0 con 0x7facf015e190 2023-08-27T11:29:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:09.752+0000 7facf65e8700 1 -- 172.21.15.112:0/3415097731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7facf0071170 con 0x7facf015e190 2023-08-27T11:29:10.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.366+0000 7facecff9700 1 -- 172.21.15.112:0/3415097731 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v107) v1 ==== 104+0+0 (secure 0 0 0) 0x7face00200a0 con 0x7facf015e190 2023-08-27T11:29:10.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.369+0000 7facf65e8700 1 -- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7facdc06a4d0 msgr2=0x7facdc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:10.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.369+0000 7facf65e8700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7facdc06a4d0 0x7facdc06c990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7face4009be0 tx=0x7face4009210 comp rx=0 tx=0).stop 2023-08-27T11:29:10.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.369+0000 7facf65e8700 1 -- 172.21.15.112:0/3415097731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf015e190 msgr2=0x7facf006f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:10.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.369+0000 7facf65e8700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf015e190 0x7facf006f600 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7face0009c10 tx=0x7face0004750 comp rx=0 tx=0).stop 2023-08-27T11:29:10.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.369+0000 7facf65e8700 1 -- 172.21.15.112:0/3415097731 shutdown_connections 2023-08-27T11:29:10.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.369+0000 7facf65e8700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7facdc06a4d0 0x7facdc06c990 secure :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7face4009be0 tx=0x7face4009210 comp rx=0 tx=0).stop 2023-08-27T11:29:10.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.369+0000 7facf65e8700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf0161dd0 0x7facf00cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:10.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.369+0000 7facf65e8700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf00cc980 0x7facf014e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:10.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.369+0000 7facf65e8700 1 --2- 172.21.15.112:0/3415097731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf015e190 0x7facf006f600 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:10.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.369+0000 7facf65e8700 1 -- 172.21.15.112:0/3415097731 >> 172.21.15.112:0/3415097731 conn(0x7facf006bbf0 msgr2=0x7facf00ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:10.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.369+0000 7facf65e8700 1 -- 172.21.15.112:0/3415097731 shutdown_connections 2023-08-27T11:29:10.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.369+0000 7facf65e8700 1 -- 172.21.15.112:0/3415097731 wait complete. 2023-08-27T11:29:10.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:29:10.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.556+0000 7f9287e1e700 1 Processor -- start 2023-08-27T11:29:10.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.556+0000 7f9287e1e700 1 -- start start 2023-08-27T11:29:10.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9287e1e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92800b3de0 0x7f928015e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:10.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9287e1e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9280161390 con 0x7f9280157ae0 2023-08-27T11:29:10.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9287e1e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92800b35f0 con 0x7f928006f790 2023-08-27T11:29:10.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9287e1e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92800b1aa0 con 0x7f92800b3de0 2023-08-27T11:29:10.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9285bba700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9280157ae0 0x7f9280161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60332/0 (socket says 172.21.15.112:60332) 2023-08-27T11:29:10.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9285bba700 1 -- 172.21.15.112:0/2413855606 learned_addr learned my addr 172.21.15.112:0/2413855606 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:10.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f92853b9700 1 --2- 172.21.15.112:0/2413855606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92800b3de0 0x7f928015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:10.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981321925 0 0) 0x7f9280161390 con 0x7f9280157ae0 2023-08-27T11:29:10.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f926c0035b0 con 0x7f9280157ae0 2023-08-27T11:29:10.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640452333 0 0) 0x7f92800b35f0 con 0x7f928006f790 2023-08-27T11:29:10.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9280161390 con 0x7f928006f790 2023-08-27T11:29:10.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3406836765 0 0) 0x7f926c0035b0 con 0x7f9280157ae0 2023-08-27T11:29:10.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92800b3de0 msgr2=0x7f928015e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:10.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9284bb8700 1 --2- 172.21.15.112:0/2413855606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92800b3de0 0x7f928015e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:10.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 >> v1:172.21.15.138:6789/0 conn(0x7f928006f790 legacy=0x7f92800b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:10.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92800b4330 con 0x7f9280157ae0 2023-08-27T11:29:10.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f9270001670 con 0x7f9280157ae0 2023-08-27T11:29:10.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 >> v1:172.21.15.112:6789/0 conn(0x7f9280157ae0 legacy=0x7f9280161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:10.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9284bb8700 1 --2- 172.21.15.112:0/2413855606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f926c003bc0 0x7f926c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:10.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.557+0000 7f9284bb8700 1 --2- 172.21.15.112:0/2413855606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f926c004850 0x7f926c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:10.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f9284bb8700 1 --2- 172.21.15.112:0/2413855606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f926c005210 0x7f926c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:10.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92800b1aa0 con 0x7f926c005210 2023-08-27T11:29:10.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c00c000 con 0x7f926c004850 2023-08-27T11:29:10.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c00c1c0 con 0x7f926c003bc0 2023-08-27T11:29:10.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f92853b9700 1 --2- 172.21.15.112:0/2413855606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92800b3de0 0x7f928015e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:10.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f92863bb700 1 --2- 172.21.15.112:0/2413855606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f926c005210 0x7f926c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:10.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f9285bba700 1 --2- 172.21.15.112:0/2413855606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f926c003bc0 0x7f926c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:10.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f92853b9700 1 --2- 172.21.15.112:0/2413855606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f926c004850 0x7f926c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:10.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f92863bb700 1 -- 172.21.15.112:0/2413855606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f926c003bc0 msgr2=0x7f926c003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:10.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f92863bb700 1 --2- 172.21.15.112:0/2413855606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f926c003bc0 0x7f926c003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:10.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f92863bb700 1 -- 172.21.15.112:0/2413855606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f926c004850 msgr2=0x7f926c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:10.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f92863bb700 1 --2- 172.21.15.112:0/2413855606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f926c004850 0x7f926c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:10.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f92863bb700 1 -- 172.21.15.112:0/2413855606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c0035b0 con 0x7f926c005210 2023-08-27T11:29:10.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f92863bb700 1 --2- 172.21.15.112:0/2413855606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f926c005210 0x7f926c00ba10 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f927800c2f0 tx=0x7f926c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d670281563ac4c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:10.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278001f30 con 0x7f926c005210 2023-08-27T11:29:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927800ce90 con 0x7f926c005210 2023-08-27T11:29:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.558+0000 7f9284bb8700 1 -- 172.21.15.112:0/2413855606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278003c40 con 0x7f926c005210 2023-08-27T11:29:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.559+0000 7f9287e1e700 1 -- 172.21.15.112:0/2413855606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f926c005210 msgr2=0x7f926c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.559+0000 7f9287e1e700 1 --2- 172.21.15.112:0/2413855606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f926c005210 0x7f926c00ba10 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f927800c2f0 tx=0x7f926c005880 comp rx=0 tx=0).stop 2023-08-27T11:29:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.559+0000 7f92863bb700 1 -- 172.21.15.112:0/2413855606 reap_dead start 2023-08-27T11:29:10.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.559+0000 7f9287e1e700 1 -- 172.21.15.112:0/2413855606 shutdown_connections 2023-08-27T11:29:10.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.559+0000 7f9287e1e700 1 -- 172.21.15.112:0/2413855606 >> 172.21.15.112:0/2413855606 conn(0x7f92800b1be0 msgr2=0x7f92800b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:10.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.559+0000 7f9287e1e700 1 -- 172.21.15.112:0/2413855606 shutdown_connections 2023-08-27T11:29:10.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.559+0000 7f9287e1e700 1 -- 172.21.15.112:0/2413855606 wait complete. 2023-08-27T11:29:10.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.560+0000 7f9287e1e700 1 Processor -- start 2023-08-27T11:29:10.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.560+0000 7f9287e1e700 1 -- start start 2023-08-27T11:29:10.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9287e1e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92800ce6d0 0x7f928015f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:10.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9287e1e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92800ceab0 0x7f92801f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:10.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9287e1e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92801f2d20 0x7f928013b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:10.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9287e1e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928015f720 con 0x7f92800ce6d0 2023-08-27T11:29:10.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9287e1e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92800acdc0 con 0x7f92800ceab0 2023-08-27T11:29:10.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9287e1e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92801f7260 con 0x7f92801f2d20 2023-08-27T11:29:10.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9285bba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92800ce6d0 0x7f928015f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:10.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9285bba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92800ce6d0 0x7f928015f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37730/0 (socket says 172.21.15.112:37730) 2023-08-27T11:29:10.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9285bba700 1 -- 172.21.15.112:0/3515648098 learned_addr learned my addr 172.21.15.112:0/3515648098 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:10.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f92863bb700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92801f2d20 0x7f928013b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:10.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f92853b9700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92800ceab0 0x7f92801f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:10.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9285bba700 1 -- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92801f2d20 msgr2=0x7f928013b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9285bba700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92801f2d20 0x7f928013b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9285bba700 1 -- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92800ceab0 msgr2=0x7f92801f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9285bba700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92800ceab0 0x7f92801f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9285bba700 1 -- 172.21.15.112:0/3515648098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92801f32c0 con 0x7f92800ce6d0 2023-08-27T11:29:10.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9285bba700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92800ce6d0 0x7f928015f1e0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f92700072f0 tx=0x7f92700076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:10.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f92767fc700 1 -- 172.21.15.112:0/3515648098 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927000f870 con 0x7f92800ce6d0 2023-08-27T11:29:10.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f92767fc700 1 -- 172.21.15.112:0/3515648098 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927000f9d0 con 0x7f92800ce6d0 2023-08-27T11:29:10.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9287e1e700 1 -- 172.21.15.112:0/3515648098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92801f35a0 con 0x7f92800ce6d0 2023-08-27T11:29:10.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f92767fc700 1 -- 172.21.15.112:0/3515648098 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927000fd60 con 0x7f92800ce6d0 2023-08-27T11:29:10.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.561+0000 7f9287e1e700 1 -- 172.21.15.112:0/3515648098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92801f3b60 con 0x7f92800ce6d0 2023-08-27T11:29:10.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.562+0000 7f9287e1e700 1 -- 172.21.15.112:0/3515648098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92800acf50 con 0x7f92800ce6d0 2023-08-27T11:29:10.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.563+0000 7f92767fc700 1 -- 172.21.15.112:0/3515648098 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f927001d460 con 0x7f92800ce6d0 2023-08-27T11:29:10.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.563+0000 7f92767fc700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f926406a4d0 0x7f926406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:10.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.563+0000 7f92767fc700 1 -- 172.21.15.112:0/3515648098 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f9270088e90 con 0x7f92800ce6d0 2023-08-27T11:29:10.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.563+0000 7f92853b9700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f926406a4d0 0x7f926406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:10.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.564+0000 7f92853b9700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f926406a4d0 0x7f926406c990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f92800af1d0 tx=0x7f927c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:10.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.567+0000 7f92767fc700 1 -- 172.21.15.112:0/3515648098 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9270054000 con 0x7f92800ce6d0 2023-08-27T11:29:10.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:10.759+0000 7f9287e1e700 1 -- 172.21.15.112:0/3515648098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f92801f3d20 con 0x7f92800ce6d0 2023-08-27T11:29:11.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.487+0000 7f92767fc700 1 -- 172.21.15.112:0/3515648098 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v157) v1 ==== 181+0+0 (secure 0 0 0) 0x7f92801f3d20 con 0x7f92800ce6d0 2023-08-27T11:29:11.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.491+0000 7f9287e1e700 1 -- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f926406a4d0 msgr2=0x7f926406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:11.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.491+0000 7f9287e1e700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f926406a4d0 0x7f926406c990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f92800af1d0 tx=0x7f927c008040 comp rx=0 tx=0).stop 2023-08-27T11:29:11.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.491+0000 7f9287e1e700 1 -- 172.21.15.112:0/3515648098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92800ce6d0 msgr2=0x7f928015f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:11.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.491+0000 7f9287e1e700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92800ce6d0 0x7f928015f1e0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f92700072f0 tx=0x7f92700076b0 comp rx=0 tx=0).stop 2023-08-27T11:29:11.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.491+0000 7f9287e1e700 1 -- 172.21.15.112:0/3515648098 shutdown_connections 2023-08-27T11:29:11.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.491+0000 7f9287e1e700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f926406a4d0 0x7f926406c990 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:11.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.491+0000 7f9287e1e700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92801f2d20 0x7f928013b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:11.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.491+0000 7f9287e1e700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92800ce6d0 0x7f928015f1e0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:11.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.491+0000 7f9287e1e700 1 --2- 172.21.15.112:0/3515648098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92800ceab0 0x7f92801f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:11.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.491+0000 7f9287e1e700 1 -- 172.21.15.112:0/3515648098 >> 172.21.15.112:0/3515648098 conn(0x7f92800b1be0 msgr2=0x7f92800cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:11.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.491+0000 7f9287e1e700 1 -- 172.21.15.112:0/3515648098 shutdown_connections 2023-08-27T11:29:11.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.491+0000 7f9287e1e700 1 -- 172.21.15.112:0/3515648098 wait complete. 2023-08-27T11:29:11.495 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:29:11.505 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:29:11.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.629+0000 7f9840872700 1 Processor -- start 2023-08-27T11:29:11.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.629+0000 7f9840872700 1 -- start start 2023-08-27T11:29:11.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.630+0000 7f9840872700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c157ae0 0x7f983c15e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:11.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.630+0000 7f9840872700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f983c0b3de0 0x7f983c161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:11.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.630+0000 7f9840872700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983c161390 con 0x7f983c161d60 2023-08-27T11:29:11.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.630+0000 7f9840872700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983c0b1aa0 con 0x7f983c0b3de0 2023-08-27T11:29:11.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.630+0000 7f9840872700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0b4bd0 con 0x7f983c157ae0 2023-08-27T11:29:11.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.630+0000 7f983ad9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f983c161d60 0x7f983c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60356/0 (socket says 172.21.15.112:60356) 2023-08-27T11:29:11.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.630+0000 7f983ad9d700 1 -- 172.21.15.112:0/1429869150 learned_addr learned my addr 172.21.15.112:0/1429869150 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:11.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.630+0000 7f983a59c700 1 --2- 172.21.15.112:0/1429869150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c157ae0 0x7f983c15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:11.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.630+0000 7f9839d9b700 1 --2- 172.21.15.112:0/1429869150 >> v2:172.21.15.138:3300/0 conn(0x7f983c0b3de0 0x7f983c161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:11.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.630+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2637952473 0 0) 0x7f983c161390 con 0x7f983c161d60 2023-08-27T11:29:11.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.630+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98240035b0 con 0x7f983c161d60 2023-08-27T11:29:11.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1848287794 0 0) 0x7f98240035b0 con 0x7f983c161d60 2023-08-27T11:29:11.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c157ae0 msgr2=0x7f983c15e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:11.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 --2- 172.21.15.112:0/1429869150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c157ae0 0x7f983c15e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:11.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 >> v2:172.21.15.138:3300/0 conn(0x7f983c0b3de0 msgr2=0x7f983c161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:11.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 --2- 172.21.15.112:0/1429869150 >> v2:172.21.15.138:3300/0 conn(0x7f983c0b3de0 0x7f983c161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:11.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c0b35f0 con 0x7f983c161d60 2023-08-27T11:29:11.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f98340019f0 con 0x7f983c161d60 2023-08-27T11:29:11.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 >> v1:172.21.15.112:6789/0 conn(0x7f983c161d60 legacy=0x7f983c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:11.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 --2- 172.21.15.112:0/1429869150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c157ae0 0x7f9824003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:11.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 --2- 172.21.15.112:0/1429869150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9824003f80 0x7f9824004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:11.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 --2- 172.21.15.112:0/1429869150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9824004d90 0x7f9824005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:11.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0b1aa0 con 0x7f9824004d90 2023-08-27T11:29:11.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0b4bd0 con 0x7f9824003f80 2023-08-27T11:29:11.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982400ba10 con 0x7f983c157ae0 2023-08-27T11:29:11.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983ad9d700 1 --2- 172.21.15.112:0/1429869150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9824004d90 0x7f9824005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:11.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983a59c700 1 --2- 172.21.15.112:0/1429869150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c157ae0 0x7f9824003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:11.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f9839d9b700 1 --2- 172.21.15.112:0/1429869150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9824003f80 0x7f9824004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:11.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983ad9d700 1 -- 172.21.15.112:0/1429869150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c157ae0 msgr2=0x7f9824003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:11.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983ad9d700 1 --2- 172.21.15.112:0/1429869150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c157ae0 0x7f9824003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:11.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983ad9d700 1 -- 172.21.15.112:0/1429869150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9824003f80 msgr2=0x7f9824004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:11.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983ad9d700 1 --2- 172.21.15.112:0/1429869150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9824003f80 0x7f9824004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:11.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983ad9d700 1 -- 172.21.15.112:0/1429869150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98240035b0 con 0x7f9824004d90 2023-08-27T11:29:11.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983ad9d700 1 --2- 172.21.15.112:0/1429869150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9824004d90 0x7f9824005210 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f983c06e680 tx=0x7f9824005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d2de07169417d6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:11.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.631+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983400a320 con 0x7f9824004d90 2023-08-27T11:29:11.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.632+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9834012600 con 0x7f9824004d90 2023-08-27T11:29:11.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.632+0000 7f983959a700 1 -- 172.21.15.112:0/1429869150 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98340129b0 con 0x7f9824004d90 2023-08-27T11:29:11.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.632+0000 7f9840872700 1 -- 172.21.15.112:0/1429869150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9824004d90 msgr2=0x7f9824005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:11.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.632+0000 7f9840872700 1 --2- 172.21.15.112:0/1429869150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9824004d90 0x7f9824005210 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f983c06e680 tx=0x7f9824005880 comp rx=0 tx=0).stop 2023-08-27T11:29:11.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.632+0000 7f983ad9d700 1 -- 172.21.15.112:0/1429869150 reap_dead start 2023-08-27T11:29:11.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.632+0000 7f9840872700 1 -- 172.21.15.112:0/1429869150 shutdown_connections 2023-08-27T11:29:11.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.632+0000 7f9840872700 1 -- 172.21.15.112:0/1429869150 >> 172.21.15.112:0/1429869150 conn(0x7f983c0b1be0 msgr2=0x7f983c0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:11.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.632+0000 7f9840872700 1 -- 172.21.15.112:0/1429869150 shutdown_connections 2023-08-27T11:29:11.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.632+0000 7f9840872700 1 -- 172.21.15.112:0/1429869150 wait complete. 2023-08-27T11:29:11.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.633+0000 7f9840872700 1 Processor -- start 2023-08-27T11:29:11.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.634+0000 7f9840872700 1 -- start start 2023-08-27T11:29:11.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.634+0000 7f9840872700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9824004d90 0x7f983c13b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:11.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.634+0000 7f9840872700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c15fcf0 0x7f983c0adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:11.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.634+0000 7f9840872700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f983c0ae210 0x7f983c0ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:11.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.634+0000 7f9840872700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c15e2a0 con 0x7f9824004d90 2023-08-27T11:29:11.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.634+0000 7f9840872700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c15e580 con 0x7f983c0ae210 2023-08-27T11:29:11.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.634+0000 7f9840872700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983c15e860 con 0x7f983c15fcf0 2023-08-27T11:29:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.634+0000 7f983a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9824004d90 0x7f983c13b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.634+0000 7f983a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9824004d90 0x7f983c13b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37752/0 (socket says 172.21.15.112:37752) 2023-08-27T11:29:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.634+0000 7f983a59c700 1 -- 172.21.15.112:0/2108915699 learned_addr learned my addr 172.21.15.112:0/2108915699 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f9839d9b700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c15fcf0 0x7f983c0adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f983ad9d700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f983c0ae210 0x7f983c0ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:11.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f983a59c700 1 -- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c15fcf0 msgr2=0x7f983c0adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:11.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f983a59c700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c15fcf0 0x7f983c0adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:11.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f983a59c700 1 -- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f983c0ae210 msgr2=0x7f983c0ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:11.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f983a59c700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f983c0ae210 0x7f983c0ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f983a59c700 1 -- 172.21.15.112:0/2108915699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c15eb40 con 0x7f9824004d90 2023-08-27T11:29:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f983ad9d700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f983c0ae210 0x7f983c0ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f9839d9b700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c15fcf0 0x7f983c0adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f983a59c700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9824004d90 0x7f983c13b8e0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f9830011aa0 tx=0x7f9830011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f982affd700 1 -- 172.21.15.112:0/2108915699 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983000f980 con 0x7f9824004d90 2023-08-27T11:29:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f982affd700 1 -- 172.21.15.112:0/2108915699 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983000fae0 con 0x7f9824004d90 2023-08-27T11:29:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f9840872700 1 -- 172.21.15.112:0/2108915699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983c15ee20 con 0x7f9824004d90 2023-08-27T11:29:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f982affd700 1 -- 172.21.15.112:0/2108915699 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983000fe90 con 0x7f9824004d90 2023-08-27T11:29:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.635+0000 7f9840872700 1 -- 172.21.15.112:0/2108915699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983c0ac890 con 0x7f9824004d90 2023-08-27T11:29:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.636+0000 7f9840872700 1 -- 172.21.15.112:0/2108915699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983c067a20 con 0x7f9824004d90 2023-08-27T11:29:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.636+0000 7f982affd700 1 -- 172.21.15.112:0/2108915699 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f9830014440 con 0x7f9824004d90 2023-08-27T11:29:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.637+0000 7f982affd700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f981c06a4d0 0x7f981c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.637+0000 7f982affd700 1 -- 172.21.15.112:0/2108915699 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5797+0+0 (secure 0 0 0) 0x7f98300893c0 con 0x7f9824004d90 2023-08-27T11:29:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.637+0000 7f9839d9b700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f981c06a4d0 0x7f981c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.638+0000 7f9839d9b700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f981c06a4d0 0x7f981c06c990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f983c0727c0 tx=0x7f982c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.641+0000 7f982affd700 1 -- 172.21.15.112:0/2108915699 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9830058120 con 0x7f9824004d90 2023-08-27T11:29:11.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:11.829+0000 7f9840872700 1 -- 172.21.15.112:0/2108915699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f983c160e60 con 0x7f9824004d90 2023-08-27T11:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.506+0000 7f982affd700 1 -- 172.21.15.112:0/2108915699 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v158) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9830057cb0 con 0x7f9824004d90 2023-08-27T11:29:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.509+0000 7f9840872700 1 -- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f981c06a4d0 msgr2=0x7f981c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:12.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.509+0000 7f9840872700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f981c06a4d0 0x7f981c06c990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f983c0727c0 tx=0x7f982c008040 comp rx=0 tx=0).stop 2023-08-27T11:29:12.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.509+0000 7f9840872700 1 -- 172.21.15.112:0/2108915699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9824004d90 msgr2=0x7f983c13b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:12.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.509+0000 7f9840872700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9824004d90 0x7f983c13b8e0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f9830011aa0 tx=0x7f9830011e60 comp rx=0 tx=0).stop 2023-08-27T11:29:12.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.509+0000 7f9840872700 1 -- 172.21.15.112:0/2108915699 shutdown_connections 2023-08-27T11:29:12.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.509+0000 7f9840872700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f981c06a4d0 0x7f981c06c990 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:12.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.509+0000 7f9840872700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f983c0ae210 0x7f983c0ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:12.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.509+0000 7f9840872700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9824004d90 0x7f983c13b8e0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:12.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.509+0000 7f9840872700 1 --2- 172.21.15.112:0/2108915699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f983c15fcf0 0x7f983c0adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:12.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.509+0000 7f9840872700 1 -- 172.21.15.112:0/2108915699 >> 172.21.15.112:0/2108915699 conn(0x7f983c0b1be0 msgr2=0x7f983c0d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:12.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.509+0000 7f9840872700 1 -- 172.21.15.112:0/2108915699 shutdown_connections 2023-08-27T11:29:12.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.509+0000 7f9840872700 1 -- 172.21.15.112:0/2108915699 wait complete. 2023-08-27T11:29:12.513 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2023-08-27T11:29:12.525 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:29:12.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.648+0000 7fcd6a7d6700 1 Processor -- start 2023-08-27T11:29:12.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.648+0000 7fcd6a7d6700 1 -- start start 2023-08-27T11:29:12.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd6a7d6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcd640cfa90 0x7fcd640b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:12.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd6a7d6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcd6406e1d0 0x7fcd640b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:12.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd6a7d6700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6406e770 con 0x7fcd640b5270 2023-08-27T11:29:12.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd6a7d6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd641444b0 con 0x7fcd6406e1d0 2023-08-27T11:29:12.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd6a7d6700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6415b910 con 0x7fcd640cfa90 2023-08-27T11:29:12.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd637fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcd6406e1d0 0x7fcd640b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:12.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd68d73700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fcd640b5270 0x7fcd6406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60376/0 (socket says 172.21.15.112:60376) 2023-08-27T11:29:12.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd68d73700 1 -- 172.21.15.112:0/2575171777 learned_addr learned my addr 172.21.15.112:0/2575171777 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:12.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd63fff700 1 --2- 172.21.15.112:0/2575171777 >> v2:172.21.15.138:3300/0 conn(0x7fcd640cfa90 0x7fcd640b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd637fe700 1 -- 172.21.15.112:0/2575171777 >> v2:172.21.15.138:3300/0 conn(0x7fcd640cfa90 msgr2=0x7fcd640b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd637fe700 1 --2- 172.21.15.112:0/2575171777 >> v2:172.21.15.138:3300/0 conn(0x7fcd640cfa90 0x7fcd640b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd637fe700 1 -- 172.21.15.112:0/2575171777 >> v1:172.21.15.112:6789/0 conn(0x7fcd640b5270 legacy=0x7fcd6406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd637fe700 1 -- 172.21.15.112:0/2575171777 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd64144e10 con 0x7fcd6406e1d0 2023-08-27T11:29:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd62ffd700 1 -- 172.21.15.112:0/2575171777 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87375596 0 0) 0x7fcd6406e770 con 0x7fcd640b5270 2023-08-27T11:29:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.649+0000 7fcd637fe700 1 --2- 172.21.15.112:0/2575171777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd6406e1d0 0x7fcd640b4d30 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fcd64070450 tx=0x7fcd58009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde7ee66d40d12e2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.650+0000 7fcd62ffd700 1 -- 172.21.15.112:0/2575171777 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58011830 con 0x7fcd6406e1d0 2023-08-27T11:29:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.650+0000 7fcd62ffd700 1 -- 172.21.15.112:0/2575171777 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd58011990 con 0x7fcd6406e1d0 2023-08-27T11:29:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.650+0000 7fcd62ffd700 1 -- 172.21.15.112:0/2575171777 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58011d20 con 0x7fcd6406e1d0 2023-08-27T11:29:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.650+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/2575171777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd6406e1d0 msgr2=0x7fcd640b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.650+0000 7fcd6a7d6700 1 --2- 172.21.15.112:0/2575171777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd6406e1d0 0x7fcd640b4d30 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fcd64070450 tx=0x7fcd58009b20 comp rx=0 tx=0).stop 2023-08-27T11:29:12.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.650+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/2575171777 shutdown_connections 2023-08-27T11:29:12.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.650+0000 7fcd6a7d6700 1 --2- 172.21.15.112:0/2575171777 >> v2:172.21.15.138:3300/0 conn(0x7fcd640cfa90 0x7fcd640b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:12.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.650+0000 7fcd6a7d6700 1 --2- 172.21.15.112:0/2575171777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd6406e1d0 0x7fcd640b4d30 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:12.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.650+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/2575171777 >> 172.21.15.112:0/2575171777 conn(0x7fcd6406bbf0 msgr2=0x7fcd640cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:12.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.650+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/2575171777 shutdown_connections 2023-08-27T11:29:12.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.650+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/2575171777 wait complete. 2023-08-27T11:29:12.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.651+0000 7fcd6a7d6700 1 Processor -- start 2023-08-27T11:29:12.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.652+0000 7fcd6a7d6700 1 -- start start 2023-08-27T11:29:12.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.652+0000 7fcd6a7d6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd6406e1d0 0x7fcd641f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:12.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.652+0000 7fcd6a7d6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd640b5270 0x7fcd64157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:12.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.652+0000 7fcd6a7d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd641582c0 0x7fcd640c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:12.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.652+0000 7fcd6a7d6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd641f7130 con 0x7fcd641582c0 2023-08-27T11:29:12.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.652+0000 7fcd6a7d6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd64070c70 con 0x7fcd6406e1d0 2023-08-27T11:29:12.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.652+0000 7fcd6a7d6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd640cbf40 con 0x7fcd640b5270 2023-08-27T11:29:12.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.652+0000 7fcd68d73700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd641582c0 0x7fcd640c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:12.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.652+0000 7fcd68d73700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd641582c0 0x7fcd640c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37768/0 (socket says 172.21.15.112:37768) 2023-08-27T11:29:12.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.652+0000 7fcd68d73700 1 -- 172.21.15.112:0/4200440899 learned_addr learned my addr 172.21.15.112:0/4200440899 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:12.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.653+0000 7fcd68d73700 1 -- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd640b5270 msgr2=0x7fcd64157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:12.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.653+0000 7fcd68d73700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd640b5270 0x7fcd64157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.653+0000 7fcd68d73700 1 -- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd6406e1d0 msgr2=0x7fcd641f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.653+0000 7fcd63fff700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd6406e1d0 0x7fcd641f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.653+0000 7fcd68d73700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd6406e1d0 0x7fcd641f6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.653+0000 7fcd68d73700 1 -- 172.21.15.112:0/4200440899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd580094a0 con 0x7fcd641582c0 2023-08-27T11:29:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.653+0000 7fcd63fff700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd6406e1d0 0x7fcd641f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:12.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.653+0000 7fcd68d73700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd641582c0 0x7fcd640c9500 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00cc70 tx=0x7fcd5c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:12.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.653+0000 7fcd60ff9700 1 -- 172.21.15.112:0/4200440899 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c00ada0 con 0x7fcd641582c0 2023-08-27T11:29:12.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.653+0000 7fcd60ff9700 1 -- 172.21.15.112:0/4200440899 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c00ce70 con 0x7fcd641582c0 2023-08-27T11:29:12.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.653+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/4200440899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd64070940 con 0x7fcd641582c0 2023-08-27T11:29:12.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.653+0000 7fcd60ff9700 1 -- 172.21.15.112:0/4200440899 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c0072f0 con 0x7fcd641582c0 2023-08-27T11:29:12.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.653+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/4200440899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd640cbc60 con 0x7fcd641582c0 2023-08-27T11:29:12.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.654+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/4200440899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd64066fa0 con 0x7fcd641582c0 2023-08-27T11:29:12.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.654+0000 7fcd60ff9700 1 -- 172.21.15.112:0/4200440899 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fcd5c007450 con 0x7fcd641582c0 2023-08-27T11:29:12.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.655+0000 7fcd60ff9700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcd5006a4d0 0x7fcd5006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:12.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.655+0000 7fcd60ff9700 1 -- 172.21.15.112:0/4200440899 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fcd5c088850 con 0x7fcd641582c0 2023-08-27T11:29:12.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.655+0000 7fcd63fff700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcd5006a4d0 0x7fcd5006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:12.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.655+0000 7fcd63fff700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcd5006a4d0 0x7fcd5006c990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcd54006fd0 tx=0x7fcd54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:12.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.659+0000 7fcd60ff9700 1 -- 172.21.15.112:0/4200440899 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcd5c053cf0 con 0x7fcd641582c0 2023-08-27T11:29:12.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.848+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/4200440899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcd64067a20 con 0x7fcd641582c0 2023-08-27T11:29:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.851+0000 7fcd60ff9700 1 -- 172.21.15.112:0/4200440899 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcd5c057310 con 0x7fcd641582c0 2023-08-27T11:29:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.853+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcd5006a4d0 msgr2=0x7fcd5006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.853+0000 7fcd6a7d6700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcd5006a4d0 0x7fcd5006c990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcd54006fd0 tx=0x7fcd54008040 comp rx=0 tx=0).stop 2023-08-27T11:29:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.853+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/4200440899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd641582c0 msgr2=0x7fcd640c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.853+0000 7fcd6a7d6700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd641582c0 0x7fcd640c9500 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00cc70 tx=0x7fcd5c00a340 comp rx=0 tx=0).stop 2023-08-27T11:29:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.854+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/4200440899 shutdown_connections 2023-08-27T11:29:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.854+0000 7fcd6a7d6700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcd5006a4d0 0x7fcd5006c990 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.854+0000 7fcd6a7d6700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd641582c0 0x7fcd640c9500 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.854+0000 7fcd6a7d6700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd6406e1d0 0x7fcd641f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.854+0000 7fcd6a7d6700 1 --2- 172.21.15.112:0/4200440899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd640b5270 0x7fcd64157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.854+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/4200440899 >> 172.21.15.112:0/4200440899 conn(0x7fcd6406bbf0 msgr2=0x7fcd6415bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.854+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/4200440899 shutdown_connections 2023-08-27T11:29:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.854+0000 7fcd6a7d6700 1 -- 172.21.15.112:0/4200440899 wait complete. 2023-08-27T11:29:12.867 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag' 2023-08-27T11:29:12.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.991+0000 7f99bed8a700 1 Processor -- start 2023-08-27T11:29:12.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.992+0000 7f99bed8a700 1 -- start start 2023-08-27T11:29:13.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.992+0000 7f99bed8a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f99b80ce220 0x7f99b80cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.992+0000 7f99bed8a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b80cca70 con 0x7f99b80cc620 2023-08-27T11:29:13.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.992+0000 7f99bed8a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b815eee0 con 0x7f99b8152540 2023-08-27T11:29:13.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.992+0000 7f99bed8a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b815f1c0 con 0x7f99b80ce220 2023-08-27T11:29:13.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99bd327700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f99b80cc620 0x7f99b80cf700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60390/0 (socket says 172.21.15.112:60390) 2023-08-27T11:29:13.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99bd327700 1 -- 172.21.15.112:0/1279035239 learned_addr learned my addr 172.21.15.112:0/1279035239 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:13.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986856711 0 0) 0x7f99b80cca70 con 0x7f99b80cc620 2023-08-27T11:29:13.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99a80035b0 con 0x7f99b80cc620 2023-08-27T11:29:13.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b7fff700 1 --2- 172.21.15.112:0/1279035239 >> v2:172.21.15.138:3300/0 conn(0x7f99b80ce220 0x7f99b80cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:13.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1767785282 0 0) 0x7f99a80035b0 con 0x7f99b80cc620 2023-08-27T11:29:13.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 >> v2:172.21.15.138:3300/0 conn(0x7f99b80ce220 msgr2=0x7f99b80cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:13.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 --2- 172.21.15.112:0/1279035239 >> v2:172.21.15.138:3300/0 conn(0x7f99b80ce220 0x7f99b80cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 >> v1:172.21.15.138:6789/0 conn(0x7f99b8152540 legacy=0x7f99b80cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b815f4a0 con 0x7f99b80cc620 2023-08-27T11:29:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f99b00019f0 con 0x7f99b80cc620 2023-08-27T11:29:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b7fff700 1 --2- 172.21.15.112:0/1279035239 >> v2:172.21.15.138:3300/0 conn(0x7f99b80ce220 0x7f99b80cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 >> v1:172.21.15.112:6789/0 conn(0x7f99b80cc620 legacy=0x7f99b80cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 --2- 172.21.15.112:0/1279035239 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a8003a00 0x7f99a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 --2- 172.21.15.112:0/1279035239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99a8004850 0x7f99a8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 --2- 172.21.15.112:0/1279035239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a8005210 0x7f99a800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b815f1c0 con 0x7f99a8005210 2023-08-27T11:29:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a800c160 con 0x7f99a8004850 2023-08-27T11:29:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a800c320 con 0x7f99a8003a00 2023-08-27T11:29:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99bd327700 1 --2- 172.21.15.112:0/1279035239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a8005210 0x7f99a800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99b7fff700 1 --2- 172.21.15.112:0/1279035239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99a8004850 0x7f99a8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.993+0000 7f99bcb26700 1 --2- 172.21.15.112:0/1279035239 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a8003a00 0x7f99a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bd327700 1 -- 172.21.15.112:0/1279035239 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a8003a00 msgr2=0x7f99a8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bd327700 1 --2- 172.21.15.112:0/1279035239 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a8003a00 0x7f99a8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bd327700 1 -- 172.21.15.112:0/1279035239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99a8004850 msgr2=0x7f99a8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bd327700 1 --2- 172.21.15.112:0/1279035239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99a8004850 0x7f99a8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bd327700 1 -- 172.21.15.112:0/1279035239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a80035b0 con 0x7f99a8005210 2023-08-27T11:29:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bcb26700 1 --2- 172.21.15.112:0/1279035239 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a8003a00 0x7f99a8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bd327700 1 -- 172.21.15.112:0/1279035239 reap_dead start 2023-08-27T11:29:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bd327700 1 --2- 172.21.15.112:0/1279035239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a8005210 0x7f99a800ba10 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f99b000a830 tx=0x7f99a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f775db04daf1c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b000a2f0 con 0x7f99a8005210 2023-08-27T11:29:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b0012620 con 0x7f99a8005210 2023-08-27T11:29:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99b77fe700 1 -- 172.21.15.112:0/1279035239 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b00129d0 con 0x7f99a8005210 2023-08-27T11:29:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bed8a700 1 -- 172.21.15.112:0/1279035239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a8005210 msgr2=0x7f99a800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bed8a700 1 --2- 172.21.15.112:0/1279035239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a8005210 0x7f99a800ba10 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f99b000a830 tx=0x7f99a8005880 comp rx=0 tx=0).stop 2023-08-27T11:29:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bed8a700 1 -- 172.21.15.112:0/1279035239 shutdown_connections 2023-08-27T11:29:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bed8a700 1 --2- 172.21.15.112:0/1279035239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a8005210 0x7f99a800ba10 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bed8a700 1 -- 172.21.15.112:0/1279035239 >> 172.21.15.112:0/1279035239 conn(0x7f99b806f970 msgr2=0x7f99b806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bed8a700 1 -- 172.21.15.112:0/1279035239 shutdown_connections 2023-08-27T11:29:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.994+0000 7f99bed8a700 1 -- 172.21.15.112:0/1279035239 wait complete. 2023-08-27T11:29:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.996+0000 7f99bed8a700 1 Processor -- start 2023-08-27T11:29:13.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.996+0000 7f99bed8a700 1 -- start start 2023-08-27T11:29:13.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99bed8a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99a8005210 0x7f99b8158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99bed8a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b815ab60 0x7f99b815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99bed8a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99b814bac0 0x7f99b814bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99bed8a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b814c490 con 0x7f99b815ab60 2023-08-27T11:29:13.012 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99bed8a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b815d2b0 con 0x7f99a8005210 2023-08-27T11:29:13.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99bed8a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b815b120 con 0x7f99b814bac0 2023-08-27T11:29:13.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b815ab60 0x7f99b815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:13.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b815ab60 0x7f99b815cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37786/0 (socket says 172.21.15.112:37786) 2023-08-27T11:29:13.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99b7fff700 1 -- 172.21.15.112:0/1811048221 learned_addr learned my addr 172.21.15.112:0/1811048221 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:13.013 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99b7fff700 1 -- 172.21.15.112:0/1811048221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99b814bac0 msgr2=0x7f99b814bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:13.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99b7fff700 1 --2- 172.21.15.112:0/1811048221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99b814bac0 0x7f99b814bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99b7fff700 1 -- 172.21.15.112:0/1811048221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99a8005210 msgr2=0x7f99b8158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:13.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99b7fff700 1 --2- 172.21.15.112:0/1811048221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99a8005210 0x7f99b8158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.997+0000 7f99b7fff700 1 -- 172.21.15.112:0/1811048221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8159510 con 0x7f99b815ab60 2023-08-27T11:29:13.014 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.998+0000 7f99b7fff700 1 --2- 172.21.15.112:0/1811048221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b815ab60 0x7f99b815cd70 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009660 tx=0x7f99a4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:13.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.998+0000 7f99b57fa700 1 -- 172.21.15.112:0/1811048221 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4015940 con 0x7f99b815ab60 2023-08-27T11:29:13.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.998+0000 7f99b57fa700 1 -- 172.21.15.112:0/1811048221 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a4015ac0 con 0x7f99b815ab60 2023-08-27T11:29:13.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.998+0000 7f99bed8a700 1 -- 172.21.15.112:0/1811048221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b81597f0 con 0x7f99b815ab60 2023-08-27T11:29:13.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.998+0000 7f99b57fa700 1 -- 172.21.15.112:0/1811048221 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4015e70 con 0x7f99b815ab60 2023-08-27T11:29:13.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.998+0000 7f99bed8a700 1 -- 172.21.15.112:0/1811048221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b8159db0 con 0x7f99b815ab60 2023-08-27T11:29:13.015 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.999+0000 7f99bed8a700 1 -- 172.21.15.112:0/1811048221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b815f760 con 0x7f99b815ab60 2023-08-27T11:29:13.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:12.999+0000 7f99b57fa700 1 -- 172.21.15.112:0/1811048221 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f99a40143d0 con 0x7f99b815ab60 2023-08-27T11:29:13.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.000+0000 7f99b57fa700 1 --2- 172.21.15.112:0/1811048221 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f999c06a4d0 0x7f999c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.000+0000 7f99b57fa700 1 -- 172.21.15.112:0/1811048221 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f99a4088c90 con 0x7f99b815ab60 2023-08-27T11:29:13.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.000+0000 7f99bcb26700 1 --2- 172.21.15.112:0/1811048221 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f999c06a4d0 0x7f999c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:13.016 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.000+0000 7f99bcb26700 1 --2- 172.21.15.112:0/1811048221 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f999c06a4d0 0x7f999c06c990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f99ac001590 tx=0x7f99ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:13.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.004+0000 7f99b57fa700 1 -- 172.21.15.112:0/1811048221 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f99b815f760 con 0x7f99b815ab60 2023-08-27T11:29:13.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.273+0000 7f99bed8a700 1 -- 172.21.15.112:0/1811048221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]} v 0) v1 -- 0x7f99b815f760 con 0x7f99b815ab60 2023-08-27T11:29:13.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.502+0000 7f99b57fa700 1 -- 172.21.15.112:0/1811048221 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]}]=0 v562) v1 ==== 167+0+0 (secure 0 0 0) 0x7f99b815f760 con 0x7f99b815ab60 2023-08-27T11:29:13.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.504+0000 7f99bed8a700 1 -- 172.21.15.112:0/1811048221 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f999c06a4d0 msgr2=0x7f999c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:13.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.504+0000 7f99bed8a700 1 --2- 172.21.15.112:0/1811048221 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f999c06a4d0 0x7f999c06c990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f99ac001590 tx=0x7f99ac009040 comp rx=0 tx=0).stop 2023-08-27T11:29:13.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.504+0000 7f99bed8a700 1 -- 172.21.15.112:0/1811048221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b815ab60 msgr2=0x7f99b815cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:13.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.504+0000 7f99bed8a700 1 --2- 172.21.15.112:0/1811048221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b815ab60 0x7f99b815cd70 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009660 tx=0x7f99a4009a20 comp rx=0 tx=0).stop 2023-08-27T11:29:13.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.504+0000 7f99bed8a700 1 -- 172.21.15.112:0/1811048221 shutdown_connections 2023-08-27T11:29:13.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.504+0000 7f99bed8a700 1 --2- 172.21.15.112:0/1811048221 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f999c06a4d0 0x7f999c06c990 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.504+0000 7f99bed8a700 1 --2- 172.21.15.112:0/1811048221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99b814bac0 0x7f99b814bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.504+0000 7f99bed8a700 1 --2- 172.21.15.112:0/1811048221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99a8005210 0x7f99b8158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.504+0000 7f99bed8a700 1 --2- 172.21.15.112:0/1811048221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b815ab60 0x7f99b815cd70 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.504+0000 7f99bed8a700 1 -- 172.21.15.112:0/1811048221 >> 172.21.15.112:0/1811048221 conn(0x7f99b806f970 msgr2=0x7f99b8066870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.504+0000 7f99bed8a700 1 -- 172.21.15.112:0/1811048221 shutdown_connections 2023-08-27T11:29:13.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.504+0000 7f99bed8a700 1 -- 172.21.15.112:0/1811048221 wait complete. 2023-08-27T11:29:13.518 INFO:tasks.cephfs_test_runner:That renaming volume fails without --yes-i-really-mean-it flag. ... ok 2023-08-27T11:29:13.518 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_for_more_than_one_data_pool (tasks.cephfs.test_volumes.TestVolumes) That renaming a volume with more than one data pool does not change 2023-08-27T11:29:13.518 INFO:tasks.cephfs_test_runner:test_volume_rename_for_more_than_one_data_pool (tasks.cephfs.test_volumes.TestVolumes) 2023-08-27T11:29:13.519 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool' 2023-08-27T11:29:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.637+0000 7fb473a87700 1 Processor -- start 2023-08-27T11:29:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.637+0000 7fb473a87700 1 -- start start 2023-08-27T11:29:13.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb473a87700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb46c06fea0 0x7fb46c0b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb473a87700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46c0b4760 0x7fb46c0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb473a87700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46c161030 con 0x7fb46c1466c0 2023-08-27T11:29:13.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb473a87700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c0b57e0 con 0x7fb46c06fea0 2023-08-27T11:29:13.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb473a87700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c161310 con 0x7fb46c0b4760 2023-08-27T11:29:13.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb471022700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb46c06fea0 0x7fb46c0b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:13.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb471823700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb46c1466c0 0x7fb46c06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60406/0 (socket says 172.21.15.112:60406) 2023-08-27T11:29:13.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb471823700 1 -- 172.21.15.112:0/2576197876 learned_addr learned my addr 172.21.15.112:0/2576197876 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:13.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb472024700 1 --2- 172.21.15.112:0/2576197876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46c0b4760 0x7fb46c0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:13.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb471022700 1 -- 172.21.15.112:0/2576197876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46c0b4760 msgr2=0x7fb46c0b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:13.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb471022700 1 --2- 172.21.15.112:0/2576197876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46c0b4760 0x7fb46c0b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb471022700 1 -- 172.21.15.112:0/2576197876 >> v1:172.21.15.112:6789/0 conn(0x7fb46c1466c0 legacy=0x7fb46c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:13.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb471022700 1 -- 172.21.15.112:0/2576197876 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c1615f0 con 0x7fb46c06fea0 2023-08-27T11:29:13.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.638+0000 7fb470821700 1 -- 172.21.15.112:0/2576197876 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557293254 0 0) 0x7fb46c161030 con 0x7fb46c1466c0 2023-08-27T11:29:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.639+0000 7fb471022700 1 --2- 172.21.15.112:0/2576197876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb46c06fea0 0x7fb46c0b05b0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fb468009840 tx=0x7fb468009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf877e35b2c2ef43 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.639+0000 7fb470821700 1 -- 172.21.15.112:0/2576197876 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468011830 con 0x7fb46c06fea0 2023-08-27T11:29:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.639+0000 7fb470821700 1 -- 172.21.15.112:0/2576197876 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb468011990 con 0x7fb46c06fea0 2023-08-27T11:29:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.639+0000 7fb470821700 1 -- 172.21.15.112:0/2576197876 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468011d20 con 0x7fb46c06fea0 2023-08-27T11:29:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.639+0000 7fb473a87700 1 -- 172.21.15.112:0/2576197876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb46c06fea0 msgr2=0x7fb46c0b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:13.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.639+0000 7fb473a87700 1 --2- 172.21.15.112:0/2576197876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb46c06fea0 0x7fb46c0b05b0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fb468009840 tx=0x7fb468009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:13.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.639+0000 7fb473a87700 1 -- 172.21.15.112:0/2576197876 shutdown_connections 2023-08-27T11:29:13.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.639+0000 7fb473a87700 1 --2- 172.21.15.112:0/2576197876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46c0b4760 0x7fb46c0b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.639+0000 7fb473a87700 1 --2- 172.21.15.112:0/2576197876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb46c06fea0 0x7fb46c0b05b0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.639+0000 7fb473a87700 1 -- 172.21.15.112:0/2576197876 >> 172.21.15.112:0/2576197876 conn(0x7fb46c0cff80 msgr2=0x7fb46c0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.639+0000 7fb473a87700 1 -- 172.21.15.112:0/2576197876 shutdown_connections 2023-08-27T11:29:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.639+0000 7fb473a87700 1 -- 172.21.15.112:0/2576197876 wait complete. 2023-08-27T11:29:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.641+0000 7fb473a87700 1 Processor -- start 2023-08-27T11:29:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.641+0000 7fb473a87700 1 -- start start 2023-08-27T11:29:13.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.641+0000 7fb473a87700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb46c06fea0 0x7fb46c1f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.641+0000 7fb473a87700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46c0b4760 0x7fb46c13c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.641+0000 7fb473a87700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb46c1466c0 0x7fb46c13ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.641+0000 7fb473a87700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c13cfc0 con 0x7fb46c06fea0 2023-08-27T11:29:13.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.641+0000 7fb473a87700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0ad120 con 0x7fb46c1466c0 2023-08-27T11:29:13.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.641+0000 7fb473a87700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c1434b0 con 0x7fb46c0b4760 2023-08-27T11:29:13.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.641+0000 7fb471823700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb46c06fea0 0x7fb46c1f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:13.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.641+0000 7fb471823700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb46c06fea0 0x7fb46c1f62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37798/0 (socket says 172.21.15.112:37798) 2023-08-27T11:29:13.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.641+0000 7fb471823700 1 -- 172.21.15.112:0/3863452845 learned_addr learned my addr 172.21.15.112:0/3863452845 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:13.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.641+0000 7fb472024700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb46c1466c0 0x7fb46c13ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:13.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.642+0000 7fb471022700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46c0b4760 0x7fb46c13c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:13.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.642+0000 7fb471823700 1 -- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46c0b4760 msgr2=0x7fb46c13c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:13.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.642+0000 7fb471823700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46c0b4760 0x7fb46c13c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.642+0000 7fb471823700 1 -- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb46c1466c0 msgr2=0x7fb46c13ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:13.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.642+0000 7fb471823700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb46c1466c0 0x7fb46c13ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:13.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.642+0000 7fb471823700 1 -- 172.21.15.112:0/3863452845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4680094a0 con 0x7fb46c06fea0 2023-08-27T11:29:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.642+0000 7fb471823700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb46c06fea0 0x7fb46c1f62b0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0011e0 tx=0x7fb45c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.642+0000 7fb4627fc700 1 -- 172.21.15.112:0/3863452845 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c004c80 con 0x7fb46c06fea0 2023-08-27T11:29:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.642+0000 7fb4627fc700 1 -- 172.21.15.112:0/3863452845 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c004e00 con 0x7fb46c06fea0 2023-08-27T11:29:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.642+0000 7fb4627fc700 1 -- 172.21.15.112:0/3863452845 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c007310 con 0x7fb46c06fea0 2023-08-27T11:29:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.642+0000 7fb473a87700 1 -- 172.21.15.112:0/3863452845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46c143790 con 0x7fb46c06fea0 2023-08-27T11:29:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.642+0000 7fb473a87700 1 -- 172.21.15.112:0/3863452845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c0ad610 con 0x7fb46c06fea0 2023-08-27T11:29:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.643+0000 7fb473a87700 1 -- 172.21.15.112:0/3863452845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46c0ad830 con 0x7fb46c06fea0 2023-08-27T11:29:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.643+0000 7fb4627fc700 1 -- 172.21.15.112:0/3863452845 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fb45c007470 con 0x7fb46c06fea0 2023-08-27T11:29:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.644+0000 7fb4627fc700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb45006a4d0 0x7fb45006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.644+0000 7fb4627fc700 1 -- 172.21.15.112:0/3863452845 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb45c088410 con 0x7fb46c06fea0 2023-08-27T11:29:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.644+0000 7fb471022700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb45006a4d0 0x7fb45006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.644+0000 7fb471022700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb45006a4d0 0x7fb45006c990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb4680090b0 tx=0x7fb46801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.648+0000 7fb4627fc700 1 -- 172.21.15.112:0/3863452845 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb45c053a30 con 0x7fb46c06fea0 2023-08-27T11:29:13.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:13.912+0000 7fb473a87700 1 -- 172.21.15.112:0/3863452845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]} v 0) v1 -- 0x7fb46c067a20 con 0x7fb46c06fea0 2023-08-27T11:29:14.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.507+0000 7fb4627fc700 1 -- 172.21.15.112:0/3863452845 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]}]=0 v563) v1 ==== 166+0+0 (secure 0 0 0) 0x7fb45c057050 con 0x7fb46c06fea0 2023-08-27T11:29:14.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.510+0000 7fb473a87700 1 -- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb45006a4d0 msgr2=0x7fb45006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:14.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.510+0000 7fb473a87700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb45006a4d0 0x7fb45006c990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb4680090b0 tx=0x7fb46801c040 comp rx=0 tx=0).stop 2023-08-27T11:29:14.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.510+0000 7fb473a87700 1 -- 172.21.15.112:0/3863452845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb46c06fea0 msgr2=0x7fb46c1f62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:14.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.510+0000 7fb473a87700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb46c06fea0 0x7fb46c1f62b0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0011e0 tx=0x7fb45c00ca30 comp rx=0 tx=0).stop 2023-08-27T11:29:14.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.510+0000 7fb473a87700 1 -- 172.21.15.112:0/3863452845 shutdown_connections 2023-08-27T11:29:14.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.510+0000 7fb473a87700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb45006a4d0 0x7fb45006c990 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.510+0000 7fb473a87700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb46c1466c0 0x7fb46c13ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.510+0000 7fb473a87700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb46c06fea0 0x7fb46c1f62b0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.510+0000 7fb473a87700 1 --2- 172.21.15.112:0/3863452845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46c0b4760 0x7fb46c13c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.510+0000 7fb473a87700 1 -- 172.21.15.112:0/3863452845 >> 172.21.15.112:0/3863452845 conn(0x7fb46c0cff80 msgr2=0x7fb46c15e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:14.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.510+0000 7fb473a87700 1 -- 172.21.15.112:0/3863452845 shutdown_connections 2023-08-27T11:29:14.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.510+0000 7fb473a87700 1 -- 172.21.15.112:0/3863452845 wait complete. 2023-08-27T11:29:14.526 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:14.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.650+0000 7f9726f92700 1 Processor -- start 2023-08-27T11:29:14.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.650+0000 7f9726f92700 1 -- start start 2023-08-27T11:29:14.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.650+0000 7f9726f92700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f97200b17e0 0x7f97200cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:14.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.650+0000 7f9726f92700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9720161ce0 0x7f972006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:14.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.650+0000 7f9726f92700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9720161850 con 0x7f97200b1400 2023-08-27T11:29:14.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.650+0000 7f9726f92700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97200cfc10 con 0x7f97200b17e0 2023-08-27T11:29:14.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.650+0000 7f9726f92700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9720161200 con 0x7f9720161ce0 2023-08-27T11:29:14.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.651+0000 7f971ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f97200b17e0 0x7f97200cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:14.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.651+0000 7f9724d2e700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f97200b1400 0x7f972006e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60420/0 (socket says 172.21.15.112:60420) 2023-08-27T11:29:14.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.651+0000 7f971ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f97200b17e0 0x7f97200cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37808/0 (socket says 172.21.15.112:37808) 2023-08-27T11:29:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.651+0000 7f9724d2e700 1 -- 172.21.15.112:0/1910997972 learned_addr learned my addr 172.21.15.112:0/1910997972 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.651+0000 7f972552f700 1 --2- 172.21.15.112:0/1910997972 >> v2:172.21.15.138:3300/0 conn(0x7f9720161ce0 0x7f972006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.651+0000 7f971ffff700 1 -- 172.21.15.112:0/1910997972 >> v2:172.21.15.138:3300/0 conn(0x7f9720161ce0 msgr2=0x7f972006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.651+0000 7f971ffff700 1 --2- 172.21.15.112:0/1910997972 >> v2:172.21.15.138:3300/0 conn(0x7f9720161ce0 0x7f972006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.651+0000 7f971ffff700 1 -- 172.21.15.112:0/1910997972 >> v1:172.21.15.112:6789/0 conn(0x7f97200b1400 legacy=0x7f972006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.651+0000 7f971ffff700 1 -- 172.21.15.112:0/1910997972 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97201614e0 con 0x7f97200b17e0 2023-08-27T11:29:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.651+0000 7f971ffff700 1 --2- 172.21.15.112:0/1910997972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97200b17e0 0x7f97200cf5a0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f972006df10 tx=0x7f970c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1bc15e9dfbef29 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.651+0000 7f971f7fe700 1 -- 172.21.15.112:0/1910997972 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c011830 con 0x7f97200b17e0 2023-08-27T11:29:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.651+0000 7f971f7fe700 1 -- 172.21.15.112:0/1910997972 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c011990 con 0x7f97200b17e0 2023-08-27T11:29:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.651+0000 7f971f7fe700 1 -- 172.21.15.112:0/1910997972 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c011d20 con 0x7f97200b17e0 2023-08-27T11:29:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.652+0000 7f9726f92700 1 -- 172.21.15.112:0/1910997972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97200b17e0 msgr2=0x7f97200cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.652+0000 7f9726f92700 1 --2- 172.21.15.112:0/1910997972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97200b17e0 0x7f97200cf5a0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f972006df10 tx=0x7f970c009b20 comp rx=0 tx=0).stop 2023-08-27T11:29:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.652+0000 7f9726f92700 1 -- 172.21.15.112:0/1910997972 shutdown_connections 2023-08-27T11:29:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.652+0000 7f9726f92700 1 --2- 172.21.15.112:0/1910997972 >> v2:172.21.15.138:3300/0 conn(0x7f9720161ce0 0x7f972006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.652+0000 7f9726f92700 1 --2- 172.21.15.112:0/1910997972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97200b17e0 0x7f97200cf5a0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.652+0000 7f9726f92700 1 -- 172.21.15.112:0/1910997972 >> 172.21.15.112:0/1910997972 conn(0x7f97200d0000 msgr2=0x7f97200d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.652+0000 7f9726f92700 1 -- 172.21.15.112:0/1910997972 shutdown_connections 2023-08-27T11:29:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.652+0000 7f9726f92700 1 -- 172.21.15.112:0/1910997972 wait complete. 2023-08-27T11:29:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.653+0000 7f9726f92700 1 Processor -- start 2023-08-27T11:29:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.653+0000 7f9726f92700 1 -- start start 2023-08-27T11:29:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.654+0000 7f9726f92700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97200b1400 0x7f97200b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.654+0000 7f9726f92700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97200b17e0 0x7f97200ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.654+0000 7f9726f92700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9720161ce0 0x7f97200ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.654+0000 7f9726f92700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97200b0760 con 0x7f97200b17e0 2023-08-27T11:29:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.654+0000 7f9726f92700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9720145670 con 0x7f9720161ce0 2023-08-27T11:29:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.654+0000 7f9726f92700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9720145950 con 0x7f97200b1400 2023-08-27T11:29:14.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.654+0000 7f971ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97200b17e0 0x7f97200ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:14.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f971ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97200b17e0 0x7f97200ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37814/0 (socket says 172.21.15.112:37814) 2023-08-27T11:29:14.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f971ffff700 1 -- 172.21.15.112:0/1965002382 learned_addr learned my addr 172.21.15.112:0/1965002382 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:14.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f972552f700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9720161ce0 0x7f97200ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:14.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f9724d2e700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97200b1400 0x7f97200b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:14.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f971ffff700 1 -- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97200b1400 msgr2=0x7f97200b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:14.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f971ffff700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97200b1400 0x7f97200b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f971ffff700 1 -- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9720161ce0 msgr2=0x7f97200ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f971ffff700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9720161ce0 0x7f97200ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f971ffff700 1 -- 172.21.15.112:0/1965002382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c0094a0 con 0x7f97200b17e0 2023-08-27T11:29:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f972552f700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9720161ce0 0x7f97200ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f971ffff700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97200b17e0 0x7f97200ae060 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f970c009b60 tx=0x7f970c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:14.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f971d7fa700 1 -- 172.21.15.112:0/1965002382 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c0036a0 con 0x7f97200b17e0 2023-08-27T11:29:14.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f971d7fa700 1 -- 172.21.15.112:0/1965002382 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c01f070 con 0x7f97200b17e0 2023-08-27T11:29:14.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f9726f92700 1 -- 172.21.15.112:0/1965002382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9720145c30 con 0x7f97200b17e0 2023-08-27T11:29:14.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f971d7fa700 1 -- 172.21.15.112:0/1965002382 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c003a60 con 0x7f97200b17e0 2023-08-27T11:29:14.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.655+0000 7f9726f92700 1 -- 172.21.15.112:0/1965002382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97201461f0 con 0x7f97200b17e0 2023-08-27T11:29:14.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.656+0000 7f9726f92700 1 -- 172.21.15.112:0/1965002382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9720067a20 con 0x7f97200b17e0 2023-08-27T11:29:14.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.657+0000 7f971d7fa700 1 -- 172.21.15.112:0/1965002382 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f970c003bc0 con 0x7f97200b17e0 2023-08-27T11:29:14.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.657+0000 7f971d7fa700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f970406a4d0 0x7f970406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:14.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.657+0000 7f971d7fa700 1 -- 172.21.15.112:0/1965002382 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f970c021070 con 0x7f97200b17e0 2023-08-27T11:29:14.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.657+0000 7f9724d2e700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f970406a4d0 0x7f970406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:14.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.658+0000 7f9724d2e700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f970406a4d0 0x7f970406c990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9714005910 tx=0x7f9714009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:14.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.662+0000 7f971d7fa700 1 -- 172.21.15.112:0/1965002382 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f970c0557a0 con 0x7f97200b17e0 2023-08-27T11:29:14.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.849+0000 7f9726f92700 1 -- 172.21.15.112:0/1965002382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f972006db60 con 0x7f97200b17e0 2023-08-27T11:29:14.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.850+0000 7f971d7fa700 1 -- 172.21.15.112:0/1965002382 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+14744 (secure 0 0 0) 0x7f970c058dc0 con 0x7f97200b17e0 2023-08-27T11:29:14.852 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:14.852 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":158,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:12.495409+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/577891555":"2023-08-28T11:29:09.362171+0000","172.21.15.112:6833/577891555":"2023-08-28T11:29:09.362171+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:14.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.852+0000 7f9726f92700 1 -- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f970406a4d0 msgr2=0x7f970406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:14.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.852+0000 7f9726f92700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f970406a4d0 0x7f970406c990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9714005910 tx=0x7f9714009040 comp rx=0 tx=0).stop 2023-08-27T11:29:14.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.852+0000 7f9726f92700 1 -- 172.21.15.112:0/1965002382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97200b17e0 msgr2=0x7f97200ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:14.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.852+0000 7f9726f92700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97200b17e0 0x7f97200ae060 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f970c009b60 tx=0x7f970c004750 comp rx=0 tx=0).stop 2023-08-27T11:29:14.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.852+0000 7f9726f92700 1 -- 172.21.15.112:0/1965002382 shutdown_connections 2023-08-27T11:29:14.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.852+0000 7f9726f92700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f970406a4d0 0x7f970406c990 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.852+0000 7f9726f92700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9720161ce0 0x7f97200ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.852+0000 7f9726f92700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97200b1400 0x7f97200b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.852+0000 7f9726f92700 1 --2- 172.21.15.112:0/1965002382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97200b17e0 0x7f97200ae060 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.852+0000 7f9726f92700 1 -- 172.21.15.112:0/1965002382 >> 172.21.15.112:0/1965002382 conn(0x7f97200d0000 msgr2=0x7f97200ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:14.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.852+0000 7f9726f92700 1 -- 172.21.15.112:0/1965002382 shutdown_connections 2023-08-27T11:29:14.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.852+0000 7f9726f92700 1 -- 172.21.15.112:0/1965002382 wait complete. 2023-08-27T11:29:14.866 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:29:14.867 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:29:14.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.987+0000 7f1526b6b700 1 Processor -- start 2023-08-27T11:29:14.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.987+0000 7f1526b6b700 1 -- start start 2023-08-27T11:29:14.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.988+0000 7f1526b6b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15200d1a60 0x7f152015e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:14.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.988+0000 7f1526b6b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1520071360 0x7f15200717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.988+0000 7f1526b6b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1520071d10 con 0x7f152015af90 2023-08-27T11:29:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.988+0000 7f1526b6b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152015b370 con 0x7f1520071360 2023-08-27T11:29:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.988+0000 7f1526b6b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15200b6e70 con 0x7f15200d1a60 2023-08-27T11:29:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.988+0000 7f1525108700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1520071360 0x7f15200717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.988+0000 7f1524907700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f152015af90 0x7f15200b6680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60434/0 (socket says 172.21.15.112:60434) 2023-08-27T11:29:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.988+0000 7f1524907700 1 -- 172.21.15.112:0/3533805513 learned_addr learned my addr 172.21.15.112:0/3533805513 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:14.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.989+0000 7f1525108700 1 -- 172.21.15.112:0/3533805513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15200d1a60 msgr2=0x7f152015e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:14.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.989+0000 7f151ffff700 1 --2- 172.21.15.112:0/3533805513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15200d1a60 0x7f152015e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:14.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.989+0000 7f1525108700 1 --2- 172.21.15.112:0/3533805513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15200d1a60 0x7f152015e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.989+0000 7f151f7fe700 1 -- 172.21.15.112:0/3533805513 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236805213 0 0) 0x7f1520071d10 con 0x7f152015af90 2023-08-27T11:29:14.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.989+0000 7f1525108700 1 -- 172.21.15.112:0/3533805513 >> v1:172.21.15.112:6789/0 conn(0x7f152015af90 legacy=0x7f15200b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:14.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.989+0000 7f1525108700 1 -- 172.21.15.112:0/3533805513 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15200b7150 con 0x7f1520071360 2023-08-27T11:29:14.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.989+0000 7f1525108700 1 --2- 172.21.15.112:0/3533805513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1520071360 0x7f15200717d0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f1518009d90 tx=0x7f151800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b163bd66d0c337ec server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:14.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.989+0000 7f151f7fe700 1 -- 172.21.15.112:0/3533805513 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151800e070 con 0x7f1520071360 2023-08-27T11:29:14.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.989+0000 7f151f7fe700 1 -- 172.21.15.112:0/3533805513 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151800bbd0 con 0x7f1520071360 2023-08-27T11:29:14.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.989+0000 7f151f7fe700 1 -- 172.21.15.112:0/3533805513 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518012430 con 0x7f1520071360 2023-08-27T11:29:14.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.990+0000 7f1526b6b700 1 -- 172.21.15.112:0/3533805513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1520071360 msgr2=0x7f15200717d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:14.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.990+0000 7f1526b6b700 1 --2- 172.21.15.112:0/3533805513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1520071360 0x7f15200717d0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f1518009d90 tx=0x7f151800b330 comp rx=0 tx=0).stop 2023-08-27T11:29:14.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.990+0000 7f1526b6b700 1 -- 172.21.15.112:0/3533805513 shutdown_connections 2023-08-27T11:29:14.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.990+0000 7f1526b6b700 1 --2- 172.21.15.112:0/3533805513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1520071360 0x7f15200717d0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.990+0000 7f1526b6b700 1 --2- 172.21.15.112:0/3533805513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15200d1a60 0x7f152015e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:14.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.990+0000 7f1526b6b700 1 -- 172.21.15.112:0/3533805513 >> 172.21.15.112:0/3533805513 conn(0x7f1520066010 msgr2=0x7f15200663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:14.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.990+0000 7f1526b6b700 1 -- 172.21.15.112:0/3533805513 shutdown_connections 2023-08-27T11:29:14.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.990+0000 7f1526b6b700 1 -- 172.21.15.112:0/3533805513 wait complete. 2023-08-27T11:29:14.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.991+0000 7f1526b6b700 1 Processor -- start 2023-08-27T11:29:14.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.991+0000 7f1526b6b700 1 -- start start 2023-08-27T11:29:14.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1526b6b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1520071360 0x7f1520150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:14.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1526b6b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15200d1a60 0x7f152015e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:14.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1526b6b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152015af90 0x7f152015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:14.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1526b6b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1520150ec0 con 0x7f1520071360 2023-08-27T11:29:15.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1526b6b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152015fd00 con 0x7f15200d1a60 2023-08-27T11:29:15.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1526b6b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152015f460 con 0x7f152015af90 2023-08-27T11:29:15.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1524907700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1520071360 0x7f1520150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1524907700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1520071360 0x7f1520150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37826/0 (socket says 172.21.15.112:37826) 2023-08-27T11:29:15.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1524907700 1 -- 172.21.15.112:0/2112120571 learned_addr learned my addr 172.21.15.112:0/2112120571 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:15.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f151ffff700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15200d1a60 0x7f152015e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1525108700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152015af90 0x7f152015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1524907700 1 -- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152015af90 msgr2=0x7f152015ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1524907700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152015af90 0x7f152015ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1524907700 1 -- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15200d1a60 msgr2=0x7f152015e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1524907700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15200d1a60 0x7f152015e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1524907700 1 -- 172.21.15.112:0/2112120571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518009a70 con 0x7f1520071360 2023-08-27T11:29:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f151ffff700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15200d1a60 0x7f152015e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1525108700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152015af90 0x7f152015ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1524907700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1520071360 0x7f1520150980 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f151400c550 tx=0x7f151400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f151d7fa700 1 -- 172.21.15.112:0/2112120571 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514004c80 con 0x7f1520071360 2023-08-27T11:29:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f151d7fa700 1 -- 172.21.15.112:0/2112120571 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1514004e00 con 0x7f1520071360 2023-08-27T11:29:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f151d7fa700 1 -- 172.21.15.112:0/2112120571 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514007310 con 0x7f1520071360 2023-08-27T11:29:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1526b6b700 1 -- 172.21.15.112:0/2112120571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152015f740 con 0x7f1520071360 2023-08-27T11:29:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.992+0000 7f1526b6b700 1 -- 172.21.15.112:0/2112120571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15201e5ce0 con 0x7f1520071360 2023-08-27T11:29:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.993+0000 7f1526b6b700 1 -- 172.21.15.112:0/2112120571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1520067a20 con 0x7f1520071360 2023-08-27T11:29:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.994+0000 7f151d7fa700 1 -- 172.21.15.112:0/2112120571 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f1514007470 con 0x7f1520071360 2023-08-27T11:29:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.994+0000 7f151d7fa700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f150406a4d0 0x7f150406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.994+0000 7f151d7fa700 1 -- 172.21.15.112:0/2112120571 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1514088470 con 0x7f1520071360 2023-08-27T11:29:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.994+0000 7f151ffff700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f150406a4d0 0x7f150406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.995+0000 7f151ffff700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f150406a4d0 0x7f150406c990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1520154d50 tx=0x7f150c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:15.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:14.998+0000 7f151d7fa700 1 -- 172.21.15.112:0/2112120571 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1514053b20 con 0x7f1520071360 2023-08-27T11:29:15.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.187+0000 7f1526b6b700 1 -- 172.21.15.112:0/2112120571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f15201596c0 con 0x7f1520071360 2023-08-27T11:29:15.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.188+0000 7f151d7fa700 1 -- 172.21.15.112:0/2112120571 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v158) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1514057140 con 0x7f1520071360 2023-08-27T11:29:15.190 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:29:15.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.190+0000 7f1526b6b700 1 -- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f150406a4d0 msgr2=0x7f150406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.190+0000 7f1526b6b700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f150406a4d0 0x7f150406c990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1520154d50 tx=0x7f150c008040 comp rx=0 tx=0).stop 2023-08-27T11:29:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.190+0000 7f1526b6b700 1 -- 172.21.15.112:0/2112120571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1520071360 msgr2=0x7f1520150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.190+0000 7f1526b6b700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1520071360 0x7f1520150980 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f151400c550 tx=0x7f151400c910 comp rx=0 tx=0).stop 2023-08-27T11:29:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.190+0000 7f1526b6b700 1 -- 172.21.15.112:0/2112120571 shutdown_connections 2023-08-27T11:29:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.190+0000 7f1526b6b700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f150406a4d0 0x7f150406c990 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.190+0000 7f1526b6b700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f152015af90 0x7f152015ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.190+0000 7f1526b6b700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1520071360 0x7f1520150980 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.190+0000 7f1526b6b700 1 --2- 172.21.15.112:0/2112120571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15200d1a60 0x7f152015e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.190+0000 7f1526b6b700 1 -- 172.21.15.112:0/2112120571 >> 172.21.15.112:0/2112120571 conn(0x7f1520066010 msgr2=0x7f15200ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:15.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.190+0000 7f1526b6b700 1 -- 172.21.15.112:0/2112120571 shutdown_connections 2023-08-27T11:29:15.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.190+0000 7f1526b6b700 1 -- 172.21.15.112:0/2112120571 wait complete. 2023-08-27T11:29:15.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:29:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.324+0000 7fe3cab58700 1 Processor -- start 2023-08-27T11:29:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.324+0000 7fe3cab58700 1 -- start start 2023-08-27T11:29:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.325+0000 7fe3cab58700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3c4146c20 0x7fe3c4160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.325+0000 7fe3cab58700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe3c40b3440 0x7fe3c415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.325+0000 7fe3cab58700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3c415e6e0 0x7fe3c406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.325+0000 7fe3cab58700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c40b5070 con 0x7fe3c4146c20 2023-08-27T11:29:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.325+0000 7fe3cab58700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c4147000 con 0x7fe3c40b3440 2023-08-27T11:29:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.325+0000 7fe3cab58700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c41614a0 con 0x7fe3c415e6e0 2023-08-27T11:29:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.325+0000 7fe3c88f4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3c4146c20 0x7fe3c4160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.325+0000 7fe3c88f4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3c4146c20 0x7fe3c4160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37834/0 (socket says 172.21.15.112:37834) 2023-08-27T11:29:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.325+0000 7fe3c88f4700 1 -- 172.21.15.112:0/2222681250 learned_addr learned my addr 172.21.15.112:0/2222681250 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:15.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3c3fff700 1 --2- 172.21.15.112:0/2222681250 >> v2:172.21.15.138:3300/0 conn(0x7fe3c40b3440 0x7fe3c415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3c90f5700 1 --2- 172.21.15.112:0/2222681250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3c415e6e0 0x7fe3c406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3c88f4700 1 -- 172.21.15.112:0/2222681250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3c415e6e0 msgr2=0x7fe3c406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:15.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3c88f4700 1 --2- 172.21.15.112:0/2222681250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3c415e6e0 0x7fe3c406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.331 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3c88f4700 1 -- 172.21.15.112:0/2222681250 >> v2:172.21.15.138:3300/0 conn(0x7fe3c40b3440 msgr2=0x7fe3c415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:15.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3c88f4700 1 --2- 172.21.15.112:0/2222681250 >> v2:172.21.15.138:3300/0 conn(0x7fe3c40b3440 0x7fe3c415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3c88f4700 1 -- 172.21.15.112:0/2222681250 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c4161740 con 0x7fe3c4146c20 2023-08-27T11:29:15.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3c90f5700 1 --2- 172.21.15.112:0/2222681250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3c415e6e0 0x7fe3c406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:15.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3c3fff700 1 --2- 172.21.15.112:0/2222681250 >> v2:172.21.15.138:3300/0 conn(0x7fe3c40b3440 0x7fe3c415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:15.332 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3c88f4700 1 --2- 172.21.15.112:0/2222681250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c4146c20 0x7fe3c4160f60 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40b2d80 tx=0x7fe3b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3dab842c9b8432 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:15.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3c37fe700 1 -- 172.21.15.112:0/2222681250 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b800b870 con 0x7fe3c4146c20 2023-08-27T11:29:15.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3c37fe700 1 -- 172.21.15.112:0/2222681250 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b800b9d0 con 0x7fe3c4146c20 2023-08-27T11:29:15.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3c37fe700 1 -- 172.21.15.112:0/2222681250 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b800bd60 con 0x7fe3c4146c20 2023-08-27T11:29:15.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3cab58700 1 -- 172.21.15.112:0/2222681250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c4146c20 msgr2=0x7fe3c4160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.333 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.326+0000 7fe3cab58700 1 --2- 172.21.15.112:0/2222681250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c4146c20 0x7fe3c4160f60 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40b2d80 tx=0x7fe3b8009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.327+0000 7fe3cab58700 1 -- 172.21.15.112:0/2222681250 shutdown_connections 2023-08-27T11:29:15.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.327+0000 7fe3cab58700 1 --2- 172.21.15.112:0/2222681250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3c415e6e0 0x7fe3c406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.327+0000 7fe3cab58700 1 --2- 172.21.15.112:0/2222681250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c4146c20 0x7fe3c4160f60 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.327+0000 7fe3cab58700 1 --2- 172.21.15.112:0/2222681250 >> v2:172.21.15.138:3300/0 conn(0x7fe3c40b3440 0x7fe3c415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.334 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.327+0000 7fe3cab58700 1 -- 172.21.15.112:0/2222681250 >> 172.21.15.112:0/2222681250 conn(0x7fe3c40b1e90 msgr2=0x7fe3c40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:15.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.327+0000 7fe3cab58700 1 -- 172.21.15.112:0/2222681250 shutdown_connections 2023-08-27T11:29:15.335 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.327+0000 7fe3cab58700 1 -- 172.21.15.112:0/2222681250 wait complete. 2023-08-27T11:29:15.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.328+0000 7fe3cab58700 1 Processor -- start 2023-08-27T11:29:15.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.328+0000 7fe3cab58700 1 -- start start 2023-08-27T11:29:15.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.329+0000 7fe3cab58700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c40b3440 0x7fe3c41f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.329+0000 7fe3cab58700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3c4146c20 0x7fe3c40c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.329+0000 7fe3cab58700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c415e6e0 0x7fe3c40c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.338 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.329+0000 7fe3cab58700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c41f4e30 con 0x7fe3c40b3440 2023-08-27T11:29:15.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.329+0000 7fe3cab58700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c41f3f80 con 0x7fe3c415e6e0 2023-08-27T11:29:15.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.329+0000 7fe3cab58700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c41f5800 con 0x7fe3c4146c20 2023-08-27T11:29:15.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.329+0000 7fe3c88f4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c40b3440 0x7fe3c41f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.329+0000 7fe3c88f4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c40b3440 0x7fe3c41f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37840/0 (socket says 172.21.15.112:37840) 2023-08-27T11:29:15.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.329+0000 7fe3c88f4700 1 -- 172.21.15.112:0/3718261970 learned_addr learned my addr 172.21.15.112:0/3718261970 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:15.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.330+0000 7fe3c88f4700 1 -- 172.21.15.112:0/3718261970 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3c4146c20 msgr2=0x7fe3c40c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.330+0000 7fe3c88f4700 1 --2- 172.21.15.112:0/3718261970 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3c4146c20 0x7fe3c40c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.330+0000 7fe3c88f4700 1 -- 172.21.15.112:0/3718261970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c415e6e0 msgr2=0x7fe3c40c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.330+0000 7fe3c88f4700 1 --2- 172.21.15.112:0/3718261970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c415e6e0 0x7fe3c40c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.330+0000 7fe3c88f4700 1 -- 172.21.15.112:0/3718261970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b8009550 con 0x7fe3c40b3440 2023-08-27T11:29:15.340 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.330+0000 7fe3c88f4700 1 --2- 172.21.15.112:0/3718261970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c40b3440 0x7fe3c41f52c0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8014040 tx=0x7fe3b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.330+0000 7fe3c17fa700 1 -- 172.21.15.112:0/3718261970 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8004520 con 0x7fe3c40b3440 2023-08-27T11:29:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.330+0000 7fe3c17fa700 1 -- 172.21.15.112:0/3718261970 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b801e070 con 0x7fe3c40b3440 2023-08-27T11:29:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.330+0000 7fe3cab58700 1 -- 172.21.15.112:0/3718261970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c41f4220 con 0x7fe3c40b3440 2023-08-27T11:29:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.330+0000 7fe3c17fa700 1 -- 172.21.15.112:0/3718261970 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b80038e0 con 0x7fe3c40b3440 2023-08-27T11:29:15.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.330+0000 7fe3cab58700 1 -- 172.21.15.112:0/3718261970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c41f47e0 con 0x7fe3c40b3440 2023-08-27T11:29:15.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.331+0000 7fe3cab58700 1 -- 172.21.15.112:0/3718261970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c41f5110 con 0x7fe3c40b3440 2023-08-27T11:29:15.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.331+0000 7fe3c17fa700 1 -- 172.21.15.112:0/3718261970 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe3b8003a40 con 0x7fe3c40b3440 2023-08-27T11:29:15.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.332+0000 7fe3c17fa700 1 --2- 172.21.15.112:0/3718261970 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3a806a4d0 0x7fe3a806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.332+0000 7fe3c17fa700 1 -- 172.21.15.112:0/3718261970 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe3b8020070 con 0x7fe3c40b3440 2023-08-27T11:29:15.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.332+0000 7fe3c3fff700 1 --2- 172.21.15.112:0/3718261970 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3a806a4d0 0x7fe3a806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.332+0000 7fe3c3fff700 1 --2- 172.21.15.112:0/3718261970 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3a806a4d0 0x7fe3a806c990 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0006ba0 tx=0x7fe3b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:15.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.336+0000 7fe3c17fa700 1 -- 172.21.15.112:0/3718261970 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe3b8055420 con 0x7fe3c40b3440 2023-08-27T11:29:15.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.524+0000 7fe3cab58700 1 -- 172.21.15.112:0/3718261970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe3c4067e90 con 0x7fe3c40b3440 2023-08-27T11:29:15.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.524+0000 7fe3c17fa700 1 -- 172.21.15.112:0/3718261970 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3b80150a0 con 0x7fe3c40b3440 2023-08-27T11:29:15.526 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:29:15.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.527+0000 7fe3cab58700 1 -- 172.21.15.112:0/3718261970 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3a806a4d0 msgr2=0x7fe3a806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.527+0000 7fe3cab58700 1 --2- 172.21.15.112:0/3718261970 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3a806a4d0 0x7fe3a806c990 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0006ba0 tx=0x7fe3b0008040 comp rx=0 tx=0).stop 2023-08-27T11:29:15.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.527+0000 7fe3cab58700 1 -- 172.21.15.112:0/3718261970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c40b3440 msgr2=0x7fe3c41f52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.527+0000 7fe3cab58700 1 --2- 172.21.15.112:0/3718261970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c40b3440 0x7fe3c41f52c0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8014040 tx=0x7fe3b8004750 comp rx=0 tx=0).stop 2023-08-27T11:29:15.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.527+0000 7fe3cab58700 1 -- 172.21.15.112:0/3718261970 shutdown_connections 2023-08-27T11:29:15.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.527+0000 7fe3cab58700 1 --2- 172.21.15.112:0/3718261970 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3a806a4d0 0x7fe3a806c990 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.527+0000 7fe3cab58700 1 --2- 172.21.15.112:0/3718261970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c415e6e0 0x7fe3c40c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.527+0000 7fe3cab58700 1 --2- 172.21.15.112:0/3718261970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c40b3440 0x7fe3c41f52c0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.527+0000 7fe3cab58700 1 --2- 172.21.15.112:0/3718261970 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3c4146c20 0x7fe3c40c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.527+0000 7fe3cab58700 1 -- 172.21.15.112:0/3718261970 >> 172.21.15.112:0/3718261970 conn(0x7fe3c40b1e90 msgr2=0x7fe3c406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:15.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.527+0000 7fe3cab58700 1 -- 172.21.15.112:0/3718261970 shutdown_connections 2023-08-27T11:29:15.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.527+0000 7fe3cab58700 1 -- 172.21.15.112:0/3718261970 wait complete. 2023-08-27T11:29:15.540 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:29:15.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.663+0000 7f4d1595a700 1 Processor -- start 2023-08-27T11:29:15.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.663+0000 7f4d1595a700 1 -- start start 2023-08-27T11:29:15.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.663+0000 7f4d1595a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d1006d060 0x7f4d100ca860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.663+0000 7f4d1595a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4d1015a380 0x7f4d100b2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.663+0000 7f4d1595a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d100b2a50 con 0x7f4d1006d440 2023-08-27T11:29:15.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.663+0000 7f4d1595a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d100caf40 con 0x7f4d1015a380 2023-08-27T11:29:15.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.663+0000 7f4d1595a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100cb220 con 0x7f4d1006d060 2023-08-27T11:29:15.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.664+0000 7f4d0f7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4d1015a380 0x7f4d100b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.664+0000 7f4d0e7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4d1006d440 0x7f4d10159c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60462/0 (socket says 172.21.15.112:60462) 2023-08-27T11:29:15.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.664+0000 7f4d0e7fc700 1 -- 172.21.15.112:0/1953507687 learned_addr learned my addr 172.21.15.112:0/1953507687 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:15.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.664+0000 7f4d0effd700 1 --2- 172.21.15.112:0/1953507687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d1006d060 0x7f4d100ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.664+0000 7f4d0f7fe700 1 -- 172.21.15.112:0/1953507687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d1006d060 msgr2=0x7f4d100ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:15.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.664+0000 7f4d0f7fe700 1 --2- 172.21.15.112:0/1953507687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d1006d060 0x7f4d100ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.664+0000 7f4d0f7fe700 1 -- 172.21.15.112:0/1953507687 >> v1:172.21.15.112:6789/0 conn(0x7f4d1006d440 legacy=0x7f4d10159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.664+0000 7f4d0f7fe700 1 -- 172.21.15.112:0/1953507687 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d100cb500 con 0x7f4d1015a380 2023-08-27T11:29:15.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.664+0000 7f4d0dffb700 1 -- 172.21.15.112:0/1953507687 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075988266 0 0) 0x7f4d100b2a50 con 0x7f4d1006d440 2023-08-27T11:29:15.669 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.664+0000 7f4d0f7fe700 1 --2- 172.21.15.112:0/1953507687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d1015a380 0x7f4d100b2510 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009d90 tx=0x7f4d0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adc671b2c7a54a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:15.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.664+0000 7f4d0dffb700 1 -- 172.21.15.112:0/1953507687 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800e070 con 0x7f4d1015a380 2023-08-27T11:29:15.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.664+0000 7f4d0dffb700 1 -- 172.21.15.112:0/1953507687 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0800bbd0 con 0x7f4d1015a380 2023-08-27T11:29:15.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.664+0000 7f4d0dffb700 1 -- 172.21.15.112:0/1953507687 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08012430 con 0x7f4d1015a380 2023-08-27T11:29:15.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.665+0000 7f4d1595a700 1 -- 172.21.15.112:0/1953507687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d1015a380 msgr2=0x7f4d100b2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.665+0000 7f4d1595a700 1 --2- 172.21.15.112:0/1953507687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d1015a380 0x7f4d100b2510 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009d90 tx=0x7f4d0800b330 comp rx=0 tx=0).stop 2023-08-27T11:29:15.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.665+0000 7f4d1595a700 1 -- 172.21.15.112:0/1953507687 shutdown_connections 2023-08-27T11:29:15.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.665+0000 7f4d1595a700 1 --2- 172.21.15.112:0/1953507687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d1015a380 0x7f4d100b2510 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.665+0000 7f4d1595a700 1 --2- 172.21.15.112:0/1953507687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d1006d060 0x7f4d100ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.665+0000 7f4d1595a700 1 -- 172.21.15.112:0/1953507687 >> 172.21.15.112:0/1953507687 conn(0x7f4d100cef70 msgr2=0x7f4d100cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:15.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.665+0000 7f4d1595a700 1 -- 172.21.15.112:0/1953507687 shutdown_connections 2023-08-27T11:29:15.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.665+0000 7f4d1595a700 1 -- 172.21.15.112:0/1953507687 wait complete. 2023-08-27T11:29:15.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.666+0000 7f4d1595a700 1 Processor -- start 2023-08-27T11:29:15.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.666+0000 7f4d1595a700 1 -- start start 2023-08-27T11:29:15.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d1595a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d1006d060 0x7f4d1014b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d1595a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d1006d440 0x7f4d10152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d1595a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1015a380 0x7f4d100afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d1595a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1014c0d0 con 0x7f4d1006d060 2023-08-27T11:29:15.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d1595a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1014c3b0 con 0x7f4d1015a380 2023-08-27T11:29:15.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d1595a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1014c690 con 0x7f4d1006d440 2023-08-27T11:29:15.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d0effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d1006d060 0x7f4d1014b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d0effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d1006d060 0x7f4d1014b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37854/0 (socket says 172.21.15.112:37854) 2023-08-27T11:29:15.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d0effd700 1 -- 172.21.15.112:0/3152683202 learned_addr learned my addr 172.21.15.112:0/3152683202 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:15.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d0f7fe700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1015a380 0x7f4d100afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d0effd700 1 -- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d1006d440 msgr2=0x7f4d10152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d0e7fc700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d1006d440 0x7f4d10152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d0effd700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d1006d440 0x7f4d10152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d0effd700 1 -- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1015a380 msgr2=0x7f4d100afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d0effd700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1015a380 0x7f4d100afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d0effd700 1 -- 172.21.15.112:0/3152683202 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d08009a70 con 0x7f4d1006d060 2023-08-27T11:29:15.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d0f7fe700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1015a380 0x7f4d100afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:15.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d0e7fc700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d1006d440 0x7f4d10152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:15.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.667+0000 7f4d0effd700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d1006d060 0x7f4d1014b5f0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000e810 tx=0x7f4d0000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:15.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.668+0000 7f4cf7fff700 1 -- 172.21.15.112:0/3152683202 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00013610 con 0x7f4d1006d060 2023-08-27T11:29:15.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.668+0000 7f4cf7fff700 1 -- 172.21.15.112:0/3152683202 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d00013770 con 0x7f4d1006d060 2023-08-27T11:29:15.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.668+0000 7f4d1595a700 1 -- 172.21.15.112:0/3152683202 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1014c970 con 0x7f4d1006d060 2023-08-27T11:29:15.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.668+0000 7f4cf7fff700 1 -- 172.21.15.112:0/3152683202 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00013b20 con 0x7f4d1006d060 2023-08-27T11:29:15.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.668+0000 7f4d1595a700 1 -- 172.21.15.112:0/3152683202 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d10152f60 con 0x7f4d1006d060 2023-08-27T11:29:15.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.668+0000 7f4d1595a700 1 -- 172.21.15.112:0/3152683202 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d10067a20 con 0x7f4d1006d060 2023-08-27T11:29:15.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.669+0000 7f4cf7fff700 1 -- 172.21.15.112:0/3152683202 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4d00013c80 con 0x7f4d1006d060 2023-08-27T11:29:15.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.669+0000 7f4cf7fff700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4cfc06a4d0 0x7f4cfc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:15.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.669+0000 7f4cf7fff700 1 -- 172.21.15.112:0/3152683202 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4d000887a0 con 0x7f4d1006d060 2023-08-27T11:29:15.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.670+0000 7f4d0e7fc700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4cfc06a4d0 0x7f4cfc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:15.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.670+0000 7f4d0e7fc700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4cfc06a4d0 0x7f4cfc06c990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4d101f2ac0 tx=0x7f4d04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:15.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.674+0000 7f4cf7fff700 1 -- 172.21.15.112:0/3152683202 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4d00053ab0 con 0x7f4d1006d060 2023-08-27T11:29:15.869 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.868+0000 7f4d1595a700 1 -- 172.21.15.112:0/3152683202 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4d1006fe70 con 0x7f4d1006d060 2023-08-27T11:29:15.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.872+0000 7f4cf7fff700 1 -- 172.21.15.112:0/3152683202 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4d000570d0 con 0x7f4d1006d060 2023-08-27T11:29:15.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.874+0000 7f4d1595a700 1 -- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4cfc06a4d0 msgr2=0x7f4cfc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.874+0000 7f4d1595a700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4cfc06a4d0 0x7f4cfc06c990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4d101f2ac0 tx=0x7f4d04009040 comp rx=0 tx=0).stop 2023-08-27T11:29:15.875 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.874+0000 7f4d1595a700 1 -- 172.21.15.112:0/3152683202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d1006d060 msgr2=0x7f4d1014b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:15.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.874+0000 7f4d1595a700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d1006d060 0x7f4d1014b5f0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000e810 tx=0x7f4d0000ebd0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.874+0000 7f4d1595a700 1 -- 172.21.15.112:0/3152683202 shutdown_connections 2023-08-27T11:29:15.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.874+0000 7f4d1595a700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4cfc06a4d0 0x7f4cfc06c990 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.874+0000 7f4d1595a700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1015a380 0x7f4d100afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.876 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.874+0000 7f4d1595a700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d1006d060 0x7f4d1014b5f0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.874+0000 7f4d1595a700 1 --2- 172.21.15.112:0/3152683202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d1006d440 0x7f4d10152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.874+0000 7f4d1595a700 1 -- 172.21.15.112:0/3152683202 >> 172.21.15.112:0/3152683202 conn(0x7f4d100cef70 msgr2=0x7f4d10157b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.874+0000 7f4d1595a700 1 -- 172.21.15.112:0/3152683202 shutdown_connections 2023-08-27T11:29:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:15.874+0000 7f4d1595a700 1 -- 172.21.15.112:0/3152683202 wait complete. 2023-08-27T11:29:15.887 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:29:15.887 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:29:15.933 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:29:15.979 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:29:15.979 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:29:15.979 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:15.997 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:29:15.997 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:29:15.997 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:16.017 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:29:16.017 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:29:16.018 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:16.053 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:29:16.054 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:29:16.054 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:16.075 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:29:16.076 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:29:16.076 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:16.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.212+0000 7f67d997a700 1 Processor -- start 2023-08-27T11:29:16.213 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.212+0000 7f67d997a700 1 -- start start 2023-08-27T11:29:16.215 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.212+0000 7f67d997a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d415d490 0x7f67d415d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:16.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.212+0000 7f67d997a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f67d4066450 0x7f67d415ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:16.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.212+0000 7f67d997a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d40ad580 con 0x7f67d406e150 2023-08-27T11:29:16.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.212+0000 7f67d997a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d40ad260 con 0x7f67d4066450 2023-08-27T11:29:16.216 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.212+0000 7f67d997a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d406f790 con 0x7f67d415d490 2023-08-27T11:29:16.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.213+0000 7f67d27fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f67d4066450 0x7f67d415ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:16.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.213+0000 7f67d27fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f67d4066450 0x7f67d415ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37866/0 (socket says 172.21.15.112:37866) 2023-08-27T11:29:16.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.213+0000 7f67d27fc700 1 -- 172.21.15.112:0/3016404338 learned_addr learned my addr 172.21.15.112:0/3016404338 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:16.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.213+0000 7f67d27fc700 1 -- 172.21.15.112:0/3016404338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d415d490 msgr2=0x7f67d415d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:16.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.213+0000 7f67d27fc700 1 --2- 172.21.15.112:0/3016404338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d415d490 0x7f67d415d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.213+0000 7f67d27fc700 1 -- 172.21.15.112:0/3016404338 >> v1:172.21.15.138:6789/0 conn(0x7f67d406e150 legacy=0x7f67d40701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:16.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.213+0000 7f67d27fc700 1 -- 172.21.15.112:0/3016404338 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d406fa70 con 0x7f67d4066450 2023-08-27T11:29:16.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.213+0000 7f67d27fc700 1 --2- 172.21.15.112:0/3016404338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d4066450 0x7f67d415ddb0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f67c80098f0 tx=0x7f67c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9080b9da2452a6c8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:16.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.213+0000 7f67d1ffb700 1 -- 172.21.15.112:0/3016404338 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8004230 con 0x7f67d4066450 2023-08-27T11:29:16.218 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.213+0000 7f67d1ffb700 1 -- 172.21.15.112:0/3016404338 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c8004390 con 0x7f67d4066450 2023-08-27T11:29:16.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.213+0000 7f67d1ffb700 1 -- 172.21.15.112:0/3016404338 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80036a0 con 0x7f67d4066450 2023-08-27T11:29:16.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.214+0000 7f67d997a700 1 -- 172.21.15.112:0/3016404338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d4066450 msgr2=0x7f67d415ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:16.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.214+0000 7f67d997a700 1 --2- 172.21.15.112:0/3016404338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d4066450 0x7f67d415ddb0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f67c80098f0 tx=0x7f67c8009c00 comp rx=0 tx=0).stop 2023-08-27T11:29:16.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.214+0000 7f67d997a700 1 -- 172.21.15.112:0/3016404338 shutdown_connections 2023-08-27T11:29:16.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.214+0000 7f67d997a700 1 --2- 172.21.15.112:0/3016404338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d415d490 0x7f67d415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.219 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.214+0000 7f67d997a700 1 --2- 172.21.15.112:0/3016404338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d4066450 0x7f67d415ddb0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.214+0000 7f67d997a700 1 -- 172.21.15.112:0/3016404338 >> 172.21.15.112:0/3016404338 conn(0x7f67d40b1380 msgr2=0x7f67d40b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.214+0000 7f67d997a700 1 -- 172.21.15.112:0/3016404338 shutdown_connections 2023-08-27T11:29:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.214+0000 7f67d997a700 1 -- 172.21.15.112:0/3016404338 wait complete. 2023-08-27T11:29:16.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.215+0000 7f67d997a700 1 Processor -- start 2023-08-27T11:29:16.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.215+0000 7f67d997a700 1 -- start start 2023-08-27T11:29:16.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d997a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d4066450 0x7f67d40b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:16.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d997a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d406e150 0x7f67d414e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:16.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d997a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d415d490 0x7f67d415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:16.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d997a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d414e730 con 0x7f67d406e150 2023-08-27T11:29:16.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d997a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d415dbe0 con 0x7f67d4066450 2023-08-27T11:29:16.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d997a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d415dec0 con 0x7f67d415d490 2023-08-27T11:29:16.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d406e150 0x7f67d414e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:16.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d406e150 0x7f67d414e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37870/0 (socket says 172.21.15.112:37870) 2023-08-27T11:29:16.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d27fc700 1 -- 172.21.15.112:0/1374697341 learned_addr learned my addr 172.21.15.112:0/1374697341 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:16.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d2ffd700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d4066450 0x7f67d40b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:16.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d37fe700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d415d490 0x7f67d415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:16.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d27fc700 1 -- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d415d490 msgr2=0x7f67d415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:16.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d27fc700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d415d490 0x7f67d415ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d27fc700 1 -- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d4066450 msgr2=0x7f67d40b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:16.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d27fc700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d4066450 0x7f67d40b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d27fc700 1 -- 172.21.15.112:0/1374697341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c8009550 con 0x7f67d406e150 2023-08-27T11:29:16.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d37fe700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d415d490 0x7f67d415ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:16.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67d27fc700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d406e150 0x7f67d414e1f0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f67c8004000 tx=0x7f67c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:16.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.216+0000 7f67bbfff700 1 -- 172.21.15.112:0/1374697341 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8003e70 con 0x7f67d406e150 2023-08-27T11:29:16.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.217+0000 7f67bbfff700 1 -- 172.21.15.112:0/1374697341 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c801f070 con 0x7f67d406e150 2023-08-27T11:29:16.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.217+0000 7f67d997a700 1 -- 172.21.15.112:0/1374697341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d415e1a0 con 0x7f67d406e150 2023-08-27T11:29:16.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.217+0000 7f67bbfff700 1 -- 172.21.15.112:0/1374697341 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c800b8a0 con 0x7f67d406e150 2023-08-27T11:29:16.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.217+0000 7f67d997a700 1 -- 172.21.15.112:0/1374697341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d415e760 con 0x7f67d406e150 2023-08-27T11:29:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.217+0000 7f67d997a700 1 -- 172.21.15.112:0/1374697341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d4067a20 con 0x7f67d406e150 2023-08-27T11:29:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.218+0000 7f67bbfff700 1 -- 172.21.15.112:0/1374697341 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f67c8021030 con 0x7f67d406e150 2023-08-27T11:29:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.218+0000 7f67bbfff700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f67c006a4d0 0x7f67c006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.219+0000 7f67bbfff700 1 -- 172.21.15.112:0/1374697341 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f67c808a050 con 0x7f67d406e150 2023-08-27T11:29:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.219+0000 7f67d2ffd700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f67c006a4d0 0x7f67c006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.219+0000 7f67d2ffd700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f67c006a4d0 0x7f67c006c990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f67c400a810 tx=0x7f67c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:16.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.223+0000 7f67bbfff700 1 -- 172.21.15.112:0/1374697341 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f67c80bc990 con 0x7f67d406e150 2023-08-27T11:29:16.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.456+0000 7f67d997a700 1 -- 172.21.15.112:0/1374697341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67d4067a20 con 0x7f67d406e150 2023-08-27T11:29:16.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.457+0000 7f67bbfff700 1 -- 172.21.15.112:0/1374697341 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+3713 (secure 0 0 0) 0x7f67c8055600 con 0x7f67d406e150 2023-08-27T11:29:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:16.459 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":107,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6980,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/877756033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":877756033},{"type":"v1","addr":"172.21.15.138:6837","nonce":877756033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6983,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2624496097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2624496097},{"type":"v1","addr":"172.21.15.138:6836","nonce":2624496097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":8940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2687911818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2687911818},{"type":"v1","addr":"172.21.15.112:6834","nonce":2687911818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9303,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3867187984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3867187984},{"type":"v1","addr":"172.21.15.112:6835","nonce":3867187984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107}],"filesystems":[]} 2023-08-27T11:29:16.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.459+0000 7f67d997a700 1 -- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f67c006a4d0 msgr2=0x7f67c006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:16.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.459+0000 7f67d997a700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f67c006a4d0 0x7f67c006c990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f67c400a810 tx=0x7f67c4008040 comp rx=0 tx=0).stop 2023-08-27T11:29:16.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.459+0000 7f67d997a700 1 -- 172.21.15.112:0/1374697341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d406e150 msgr2=0x7f67d414e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:16.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.459+0000 7f67d997a700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d406e150 0x7f67d414e1f0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f67c8004000 tx=0x7f67c8004750 comp rx=0 tx=0).stop 2023-08-27T11:29:16.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.459+0000 7f67d997a700 1 -- 172.21.15.112:0/1374697341 shutdown_connections 2023-08-27T11:29:16.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.459+0000 7f67d997a700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f67c006a4d0 0x7f67c006c990 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.459+0000 7f67d997a700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67d415d490 0x7f67d415ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.459+0000 7f67d997a700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d4066450 0x7f67d40b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.459+0000 7f67d997a700 1 --2- 172.21.15.112:0/1374697341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67d406e150 0x7f67d414e1f0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.459+0000 7f67d997a700 1 -- 172.21.15.112:0/1374697341 >> 172.21.15.112:0/1374697341 conn(0x7f67d40b1380 msgr2=0x7f67d415c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:16.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.459+0000 7f67d997a700 1 -- 172.21.15.112:0/1374697341 shutdown_connections 2023-08-27T11:29:16.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.459+0000 7f67d997a700 1 -- 172.21.15.112:0/1374697341 wait complete. 2023-08-27T11:29:16.464 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 107 2023-08-27T11:29:16.473 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:29:16.473 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:29:16.473 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:29:16.474 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:29:16.474 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:29:16.474 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:29:16.475 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:29:16.475 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:29:16.475 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:29:16.475 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:29:16.476 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:29:16.476 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:29:16.476 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:29:16.476 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:29:16.477 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:29:16.477 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:29:16.482 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:29:16.482 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:29:16.483 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:29:16.483 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:29:16.487 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:29:16.487 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:29:16.488 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:29:16.488 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:29:16.524 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:29:16.525 DEBUG:teuthology.parallel:result is None 2023-08-27T11:29:16.526 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:29:16.526 DEBUG:teuthology.parallel:result is None 2023-08-27T11:29:16.530 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:29:16.530 DEBUG:teuthology.parallel:result is None 2023-08-27T11:29:16.531 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:29:16.531 DEBUG:teuthology.parallel:result is None 2023-08-27T11:29:16.532 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:29:16.603 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:29:16.604 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:29:16.619 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:29:16.619 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2023-08-27T11:29:16.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.658+0000 7ff87ac9e700 1 Processor -- start 2023-08-27T11:29:16.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.658+0000 7ff87ac9e700 1 -- start start 2023-08-27T11:29:16.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff87ac9e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff874144570 0x7ff87415ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:16.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff87ac9e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff8740b3e50 0x7ff87415f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:16.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff87ac9e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff87415f8a0 0x7ff8740ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:16.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff87ac9e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff874144950 con 0x7ff874144570 2023-08-27T11:29:16.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff87ac9e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8740b45f0 con 0x7ff8740b3e50 2023-08-27T11:29:16.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff87ac9e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740b4310 con 0x7ff87415f8a0 2023-08-27T11:29:16.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff878a3a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff874144570 0x7ff87415ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:16.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff878a3a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff874144570 0x7ff87415ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37900/0 (socket says 172.21.15.112:37900) 2023-08-27T11:29:16.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff878a3a700 1 -- 172.21.15.112:0/872364538 learned_addr learned my addr 172.21.15.112:0/872364538 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:16.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff873fff700 1 --2- 172.21.15.112:0/872364538 >> v2:172.21.15.138:3300/0 conn(0x7ff8740b3e50 0x7ff87415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:16.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff87923b700 1 --2- 172.21.15.112:0/872364538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff87415f8a0 0x7ff8740ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:16.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff878a3a700 1 -- 172.21.15.112:0/872364538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff87415f8a0 msgr2=0x7ff8740ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:16.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff878a3a700 1 --2- 172.21.15.112:0/872364538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff87415f8a0 0x7ff8740ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff878a3a700 1 -- 172.21.15.112:0/872364538 >> v2:172.21.15.138:3300/0 conn(0x7ff8740b3e50 msgr2=0x7ff87415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:16.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff878a3a700 1 --2- 172.21.15.112:0/872364538 >> v2:172.21.15.138:3300/0 conn(0x7ff8740b3e50 0x7ff87415f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff878a3a700 1 -- 172.21.15.112:0/872364538 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8740cdcb0 con 0x7ff874144570 2023-08-27T11:29:16.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff878a3a700 1 --2- 172.21.15.112:0/872364538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff874144570 0x7ff87415ee20 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7ff8740b33f0 tx=0x7ff868009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb07f098406cb4af server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:16.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff8737fe700 1 -- 172.21.15.112:0/872364538 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868011830 con 0x7ff874144570 2023-08-27T11:29:16.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff8737fe700 1 -- 172.21.15.112:0/872364538 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff868011990 con 0x7ff874144570 2023-08-27T11:29:16.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.659+0000 7ff8737fe700 1 -- 172.21.15.112:0/872364538 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868011d20 con 0x7ff874144570 2023-08-27T11:29:16.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.660+0000 7ff87ac9e700 1 -- 172.21.15.112:0/872364538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff874144570 msgr2=0x7ff87415ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:16.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.660+0000 7ff87ac9e700 1 --2- 172.21.15.112:0/872364538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff874144570 0x7ff87415ee20 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7ff8740b33f0 tx=0x7ff868009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.660+0000 7ff87ac9e700 1 -- 172.21.15.112:0/872364538 shutdown_connections 2023-08-27T11:29:16.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.660+0000 7ff87ac9e700 1 --2- 172.21.15.112:0/872364538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff87415f8a0 0x7ff8740ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.660+0000 7ff87ac9e700 1 --2- 172.21.15.112:0/872364538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff874144570 0x7ff87415ee20 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.660+0000 7ff87ac9e700 1 --2- 172.21.15.112:0/872364538 >> v2:172.21.15.138:3300/0 conn(0x7ff8740b3e50 0x7ff87415f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.660+0000 7ff87ac9e700 1 -- 172.21.15.112:0/872364538 >> 172.21.15.112:0/872364538 conn(0x7ff87406f970 msgr2=0x7ff87406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:16.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.660+0000 7ff87ac9e700 1 -- 172.21.15.112:0/872364538 shutdown_connections 2023-08-27T11:29:16.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.660+0000 7ff87ac9e700 1 -- 172.21.15.112:0/872364538 wait complete. 2023-08-27T11:29:16.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.661+0000 7ff87ac9e700 1 Processor -- start 2023-08-27T11:29:16.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.661+0000 7ff87ac9e700 1 -- start start 2023-08-27T11:29:16.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.662+0000 7ff87ac9e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8740b3e50 0x7ff8740c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:16.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.662+0000 7ff87ac9e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff874144570 0x7ff8740c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:16.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.662+0000 7ff87ac9e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff87415f8a0 0x7ff87415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:16.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.662+0000 7ff87ac9e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff874141090 con 0x7ff87415f8a0 2023-08-27T11:29:16.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.662+0000 7ff87ac9e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740c8bf0 con 0x7ff874144570 2023-08-27T11:29:16.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.662+0000 7ff87ac9e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740c8950 con 0x7ff8740b3e50 2023-08-27T11:29:16.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.662+0000 7ff87923b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff87415f8a0 0x7ff87415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:16.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.662+0000 7ff87923b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff87415f8a0 0x7ff87415aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37910/0 (socket says 172.21.15.112:37910) 2023-08-27T11:29:16.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.662+0000 7ff87923b700 1 -- 172.21.15.112:0/2257877174 learned_addr learned my addr 172.21.15.112:0/2257877174 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:16.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.662+0000 7ff878a3a700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8740b3e50 0x7ff8740c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:16.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.662+0000 7ff873fff700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff874144570 0x7ff8740c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:16.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.663+0000 7ff87923b700 1 -- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8740b3e50 msgr2=0x7ff8740c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:16.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.663+0000 7ff87923b700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8740b3e50 0x7ff8740c6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.663+0000 7ff87923b700 1 -- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff874144570 msgr2=0x7ff8740c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:16.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.663+0000 7ff87923b700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff874144570 0x7ff8740c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:16.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.663+0000 7ff87923b700 1 -- 172.21.15.112:0/2257877174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff868009550 con 0x7ff87415f8a0 2023-08-27T11:29:16.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.663+0000 7ff87923b700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff87415f8a0 0x7ff87415aa30 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7ff86c007e60 tx=0x7ff86c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:16.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.663+0000 7ff8717fa700 1 -- 172.21.15.112:0/2257877174 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c00ed30 con 0x7ff87415f8a0 2023-08-27T11:29:16.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.663+0000 7ff8717fa700 1 -- 172.21.15.112:0/2257877174 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c014420 con 0x7ff87415f8a0 2023-08-27T11:29:16.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.663+0000 7ff8717fa700 1 -- 172.21.15.112:0/2257877174 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0147d0 con 0x7ff87415f8a0 2023-08-27T11:29:16.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.663+0000 7ff87ac9e700 1 -- 172.21.15.112:0/2257877174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff874140760 con 0x7ff87415f8a0 2023-08-27T11:29:16.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.663+0000 7ff87ac9e700 1 -- 172.21.15.112:0/2257877174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff874140d20 con 0x7ff87415f8a0 2023-08-27T11:29:16.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.664+0000 7ff87ac9e700 1 -- 172.21.15.112:0/2257877174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff874141200 con 0x7ff87415f8a0 2023-08-27T11:29:16.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.664+0000 7ff8717fa700 1 -- 172.21.15.112:0/2257877174 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7ff86c014930 con 0x7ff87415f8a0 2023-08-27T11:29:16.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.665+0000 7ff8717fa700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff85806a4d0 0x7ff85806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:16.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.665+0000 7ff8717fa700 1 -- 172.21.15.112:0/2257877174 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff86c088fe0 con 0x7ff87415f8a0 2023-08-27T11:29:16.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.665+0000 7ff878a3a700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff85806a4d0 0x7ff85806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:16.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.665+0000 7ff878a3a700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff85806a4d0 0x7ff85806c990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff868009c10 tx=0x7ff868008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:16.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.669+0000 7ff8717fa700 1 -- 172.21.15.112:0/2257877174 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff86c0544a0 con 0x7ff87415f8a0 2023-08-27T11:29:16.689 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-08-27T11:29:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:16.873+0000 7ff87ac9e700 1 -- 172.21.15.112:0/2257877174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff874067a20 con 0x7ff87415f8a0 2023-08-27T11:29:18.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.366+0000 7ff8717fa700 1 -- 172.21.15.112:0/2257877174 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v163) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff86c057ac0 con 0x7ff87415f8a0 2023-08-27T11:29:18.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.369+0000 7ff87ac9e700 1 -- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff85806a4d0 msgr2=0x7ff85806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:18.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.369+0000 7ff87ac9e700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff85806a4d0 0x7ff85806c990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff868009c10 tx=0x7ff868008030 comp rx=0 tx=0).stop 2023-08-27T11:29:18.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.369+0000 7ff87ac9e700 1 -- 172.21.15.112:0/2257877174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff87415f8a0 msgr2=0x7ff87415aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:18.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.369+0000 7ff87ac9e700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff87415f8a0 0x7ff87415aa30 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7ff86c007e60 tx=0x7ff86c00e3c0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.369+0000 7ff87ac9e700 1 -- 172.21.15.112:0/2257877174 shutdown_connections 2023-08-27T11:29:18.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.369+0000 7ff87ac9e700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff85806a4d0 0x7ff85806c990 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.369+0000 7ff87ac9e700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff87415f8a0 0x7ff87415aa30 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.369+0000 7ff87ac9e700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8740b3e50 0x7ff8740c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.369+0000 7ff87ac9e700 1 --2- 172.21.15.112:0/2257877174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff874144570 0x7ff8740c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.369+0000 7ff87ac9e700 1 -- 172.21.15.112:0/2257877174 >> 172.21.15.112:0/2257877174 conn(0x7ff87406f970 msgr2=0x7ff87413e870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:18.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.369+0000 7ff87ac9e700 1 -- 172.21.15.112:0/2257877174 shutdown_connections 2023-08-27T11:29:18.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.369+0000 7ff87ac9e700 1 -- 172.21.15.112:0/2257877174 wait complete. 2023-08-27T11:29:18.373 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2023-08-27T11:29:18.386 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:29:18.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.509+0000 7f86d4227700 1 Processor -- start 2023-08-27T11:29:18.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.509+0000 7f86d4227700 1 -- start start 2023-08-27T11:29:18.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.510+0000 7f86d4227700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f86cc144080 0x7f86cc0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:18.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.510+0000 7f86d4227700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f86cc0b5d60 0x7f86cc15ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:18.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.510+0000 7f86d4227700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86cc15fc00 con 0x7f86cc144690 2023-08-27T11:29:18.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.510+0000 7f86d4227700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86cc15b660 con 0x7f86cc0b5d60 2023-08-27T11:29:18.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.510+0000 7f86d4227700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86cc15f080 con 0x7f86cc144080 2023-08-27T11:29:18.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.510+0000 7f86d27c4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f86cc0b5d60 0x7f86cc15ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:18.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.510+0000 7f86d17c2700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f86cc144690 0x7f86cc0b5650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60532/0 (socket says 172.21.15.112:60532) 2023-08-27T11:29:18.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.510+0000 7f86d17c2700 1 -- 172.21.15.112:0/3619868038 learned_addr learned my addr 172.21.15.112:0/3619868038 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:18.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.511+0000 7f86d27c4700 1 -- 172.21.15.112:0/3619868038 >> v2:172.21.15.138:3300/0 conn(0x7f86cc144080 msgr2=0x7f86cc0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:18.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.511+0000 7f86d27c4700 1 --2- 172.21.15.112:0/3619868038 >> v2:172.21.15.138:3300/0 conn(0x7f86cc144080 0x7f86cc0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.511+0000 7f86d27c4700 1 -- 172.21.15.112:0/3619868038 >> v1:172.21.15.112:6789/0 conn(0x7f86cc144690 legacy=0x7f86cc0b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:18.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.511+0000 7f86d27c4700 1 -- 172.21.15.112:0/3619868038 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc15f360 con 0x7f86cc0b5d60 2023-08-27T11:29:18.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.511+0000 7f86d0fc1700 1 -- 172.21.15.112:0/3619868038 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717463700 0 0) 0x7f86cc15fc00 con 0x7f86cc144690 2023-08-27T11:29:18.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.511+0000 7f86d27c4700 1 --2- 172.21.15.112:0/3619868038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86cc0b5d60 0x7f86cc15ba50 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0707c0 tx=0x7f86c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa38f6d45ac8912 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:18.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.511+0000 7f86d0fc1700 1 -- 172.21.15.112:0/3619868038 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c400e070 con 0x7f86cc0b5d60 2023-08-27T11:29:18.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.511+0000 7f86d0fc1700 1 -- 172.21.15.112:0/3619868038 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c400bbd0 con 0x7f86cc0b5d60 2023-08-27T11:29:18.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.511+0000 7f86d0fc1700 1 -- 172.21.15.112:0/3619868038 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4012430 con 0x7f86cc0b5d60 2023-08-27T11:29:18.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.512+0000 7f86d4227700 1 -- 172.21.15.112:0/3619868038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86cc0b5d60 msgr2=0x7f86cc15ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:18.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.512+0000 7f86d4227700 1 --2- 172.21.15.112:0/3619868038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86cc0b5d60 0x7f86cc15ba50 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0707c0 tx=0x7f86c400b330 comp rx=0 tx=0).stop 2023-08-27T11:29:18.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.512+0000 7f86d4227700 1 -- 172.21.15.112:0/3619868038 shutdown_connections 2023-08-27T11:29:18.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.512+0000 7f86d4227700 1 --2- 172.21.15.112:0/3619868038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86cc0b5d60 0x7f86cc15ba50 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.512+0000 7f86d4227700 1 --2- 172.21.15.112:0/3619868038 >> v2:172.21.15.138:3300/0 conn(0x7f86cc144080 0x7f86cc0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.512+0000 7f86d4227700 1 -- 172.21.15.112:0/3619868038 >> 172.21.15.112:0/3619868038 conn(0x7f86cc0cff80 msgr2=0x7f86cc0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:18.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.512+0000 7f86d4227700 1 -- 172.21.15.112:0/3619868038 shutdown_connections 2023-08-27T11:29:18.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.512+0000 7f86d4227700 1 -- 172.21.15.112:0/3619868038 wait complete. 2023-08-27T11:29:18.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.513+0000 7f86d4227700 1 Processor -- start 2023-08-27T11:29:18.522 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.513+0000 7f86d4227700 1 -- start start 2023-08-27T11:29:18.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.514+0000 7f86d4227700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86cc0b5d60 0x7f86cc0c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:18.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.514+0000 7f86d4227700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86cc144080 0x7f86cc140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:18.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.514+0000 7f86d4227700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86cc144690 0x7f86cc150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:18.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.514+0000 7f86d4227700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc13f560 con 0x7f86cc144690 2023-08-27T11:29:18.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.514+0000 7f86d4227700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc13e990 con 0x7f86cc0b5d60 2023-08-27T11:29:18.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.514+0000 7f86d4227700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc141440 con 0x7f86cc144080 2023-08-27T11:29:18.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.514+0000 7f86d27c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86cc144690 0x7f86cc150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:18.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.514+0000 7f86d27c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86cc144690 0x7f86cc150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37928/0 (socket says 172.21.15.112:37928) 2023-08-27T11:29:18.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.514+0000 7f86d27c4700 1 -- 172.21.15.112:0/3961751360 learned_addr learned my addr 172.21.15.112:0/3961751360 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:18.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.514+0000 7f86d1fc3700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86cc0b5d60 0x7f86cc0c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:18.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.514+0000 7f86d17c2700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86cc144080 0x7f86cc140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:18.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.515+0000 7f86d27c4700 1 -- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86cc144080 msgr2=0x7f86cc140f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:18.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.515+0000 7f86d27c4700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86cc144080 0x7f86cc140f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.515+0000 7f86d27c4700 1 -- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86cc0b5d60 msgr2=0x7f86cc0c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:18.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.515+0000 7f86d27c4700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86cc0b5d60 0x7f86cc0c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.515+0000 7f86d27c4700 1 -- 172.21.15.112:0/3961751360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c4009a70 con 0x7f86cc144690 2023-08-27T11:29:18.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.515+0000 7f86d17c2700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86cc144080 0x7f86cc140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:18.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.515+0000 7f86d27c4700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86cc144690 0x7f86cc150320 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f86c400b370 tx=0x7f86c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:18.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.516+0000 7f86c27fc700 1 -- 172.21.15.112:0/3961751360 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4012430 con 0x7f86cc144690 2023-08-27T11:29:18.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.516+0000 7f86d4227700 1 -- 172.21.15.112:0/3961751360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86cc13ec30 con 0x7f86cc144690 2023-08-27T11:29:18.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.516+0000 7f86d4227700 1 -- 172.21.15.112:0/3961751360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86cc13f1f0 con 0x7f86cc144690 2023-08-27T11:29:18.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.516+0000 7f86c27fc700 1 -- 172.21.15.112:0/3961751360 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c4020070 con 0x7f86cc144690 2023-08-27T11:29:18.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.516+0000 7f86c27fc700 1 -- 172.21.15.112:0/3961751360 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c40098f0 con 0x7f86cc144690 2023-08-27T11:29:18.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.517+0000 7f86d4227700 1 -- 172.21.15.112:0/3961751360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b40052f0 con 0x7f86cc144690 2023-08-27T11:29:18.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.517+0000 7f86c27fc700 1 -- 172.21.15.112:0/3961751360 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f86c4004240 con 0x7f86cc144690 2023-08-27T11:29:18.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.518+0000 7f86c27fc700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f86b006a4d0 0x7f86b006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:18.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.518+0000 7f86c27fc700 1 -- 172.21.15.112:0/3961751360 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f86c4022070 con 0x7f86cc144690 2023-08-27T11:29:18.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.518+0000 7f86d1fc3700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f86b006a4d0 0x7f86b006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:18.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.518+0000 7f86d1fc3700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f86b006a4d0 0x7f86b006c990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f86cc13d4e0 tx=0x7f86bc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:18.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.522+0000 7f86c27fc700 1 -- 172.21.15.112:0/3961751360 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f86c4056390 con 0x7f86cc144690 2023-08-27T11:29:18.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.794+0000 7f86d4227700 1 -- 172.21.15.112:0/3961751360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f86b4005870 con 0x7f86cc144690 2023-08-27T11:29:18.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.794+0000 7f86c27fc700 1 -- 172.21.15.112:0/3961751360 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f86c40599b0 con 0x7f86cc144690 2023-08-27T11:29:18.796 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:18.797 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:29:18.797 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2023-08-27T11:29:18.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.798 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2023-08-27T11:29:18.798 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk2bnkLBAA+vxcTeJzucKk9iSF/Zw3Lg==", 2023-08-27T11:29:18.798 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.798 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:29:18.798 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:29:18.799 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:29:18.799 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.799 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.800 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2023-08-27T11:29:18.800 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk3zD5OBAASVnwG6yn/YQzfYyeMakFpA==", 2023-08-27T11:29:18.800 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.800 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:29:18.800 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:29:18.801 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:29:18.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.801 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.802 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2023-08-27T11:29:18.802 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtki7wJNBAAsIGgDLM82sWNcKnCDlSBMw==", 2023-08-27T11:29:18.802 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.802 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:29:18.803 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:29:18.803 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:29:18.803 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.803 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.804 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.804 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2023-08-27T11:29:18.804 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkCsW/BBAAFTPiTOpBo1L+rLo9px+AjQ==", 2023-08-27T11:29:18.804 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.805 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:29:18.805 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:29:18.805 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:29:18.805 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.806 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.806 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2023-08-27T11:29:18.806 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB7MOtkR/cSHhAAR8e8VRd+e0ESJkqQDYWbEw==", 2023-08-27T11:29:18.807 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.807 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:29:18.807 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:29:18.807 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.807 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.808 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.808 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2023-08-27T11:29:18.808 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB+MOtksBF3HRAA1OIXiIkhgPDYpQyYPaWOXg==", 2023-08-27T11:29:18.809 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.809 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:29:18.809 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:29:18.809 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.809 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.810 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2023-08-27T11:29:18.810 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCBMOtkWsrdHBAAkfWbiE6wAFzpt9+BxRh8fw==", 2023-08-27T11:29:18.810 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.811 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:29:18.811 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:29:18.811 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.811 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.812 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2023-08-27T11:29:18.812 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCEMOtkIkiKHRAARDoh/FH9ZdqUlZZ29Mc/Dw==", 2023-08-27T11:29:18.812 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.812 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:29:18.813 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:29:18.813 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.813 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.814 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2023-08-27T11:29:18.814 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCKMOtkPgDzGxAA+U4kbwD0d4lS8aNPG2CgHw==", 2023-08-27T11:29:18.814 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.814 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:29:18.814 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:29:18.815 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.815 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.816 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2023-08-27T11:29:18.816 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCNMOtkKomSGxAAGAQUcIH2H/uyZIU8vlq1pw==", 2023-08-27T11:29:18.816 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.816 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:29:18.817 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:29:18.817 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.817 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.818 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2023-08-27T11:29:18.818 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCQMOtkHI2MGhAA7sgbi8JXO8P9wnVvMy3PqA==", 2023-08-27T11:29:18.818 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.818 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:29:18.819 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:29:18.819 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.819 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.820 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2023-08-27T11:29:18.820 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCTMOtkVHpiGhAA4C4YJMWbQ0mmGgB4i8G/AA==", 2023-08-27T11:29:18.820 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.820 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:29:18.820 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:29:18.821 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.821 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.821 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2023-08-27T11:29:18.822 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA==", 2023-08-27T11:29:18.822 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.822 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:29:18.822 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2023-08-27T11:29:18.822 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:29:18.823 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.823 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2023-08-27T11:29:18.823 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtk66DbDxAAWZ5StELux0lriJ2gEN9icg==", 2023-08-27T11:29:18.824 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.824 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:29:18.824 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:29:18.824 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:29:18.824 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:29:18.825 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.825 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2023-08-27T11:29:18.825 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkViKJDBAAF2ofwO+78er9OyFfn2bO+g==", 2023-08-27T11:29:18.826 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.826 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:29:18.826 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:29:18.826 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:29:18.826 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:29:18.827 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.827 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2023-08-27T11:29:18.827 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkwt+QFRAAowWE69VqeRhQz0896rxx9Q==", 2023-08-27T11:29:18.828 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.828 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:29:18.828 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:29:18.828 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:29:18.828 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:29:18.828 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.829 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2023-08-27T11:29:18.829 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkjDA+DxAAjmOn82AC2OMpKlAVCD6hlg==", 2023-08-27T11:29:18.829 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.830 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:29:18.830 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:29:18.830 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:29:18.830 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.830 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.831 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2023-08-27T11:29:18.831 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkXguHKRAALHwzk8diVXqsMvTKr+AzLQ==", 2023-08-27T11:29:18.831 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.831 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:29:18.832 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:29:18.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.832 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:18.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:18.833 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2023-08-27T11:29:18.833 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkrRjBJRAAq3ftmIh81LKadpliVUiJ6g==", 2023-08-27T11:29:18.833 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:29:18.833 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:29:18.833 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:29:18.834 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:29:18.834 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.834 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:18.834 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:29:18.834 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:29:18.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.796+0000 7f86d4227700 1 -- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f86b006a4d0 msgr2=0x7f86b006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:18.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.796+0000 7f86d4227700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f86b006a4d0 0x7f86b006c990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f86cc13d4e0 tx=0x7f86bc009190 comp rx=0 tx=0).stop 2023-08-27T11:29:18.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.796+0000 7f86d4227700 1 -- 172.21.15.112:0/3961751360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86cc144690 msgr2=0x7f86cc150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:18.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.796+0000 7f86d4227700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86cc144690 0x7f86cc150320 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f86c400b370 tx=0x7f86c4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.796+0000 7f86d4227700 1 -- 172.21.15.112:0/3961751360 shutdown_connections 2023-08-27T11:29:18.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.796+0000 7f86d4227700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f86b006a4d0 0x7f86b006c990 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.796+0000 7f86d4227700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86cc144690 0x7f86cc150320 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.796+0000 7f86d4227700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86cc0b5d60 0x7f86cc0c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.796+0000 7f86d4227700 1 --2- 172.21.15.112:0/3961751360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86cc144080 0x7f86cc140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.796+0000 7f86d4227700 1 -- 172.21.15.112:0/3961751360 >> 172.21.15.112:0/3961751360 conn(0x7f86cc0cff80 msgr2=0x7f86cc0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:18.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.796+0000 7f86d4227700 1 -- 172.21.15.112:0/3961751360 shutdown_connections 2023-08-27T11:29:18.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.796+0000 7f86d4227700 1 -- 172.21.15.112:0/3961751360 wait complete. 2023-08-27T11:29:18.839 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:18.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.957+0000 7f5ba19f9700 1 Processor -- start 2023-08-27T11:29:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.957+0000 7f5ba19f9700 1 -- start start 2023-08-27T11:29:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.958+0000 7f5ba19f9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5b9c0b3e50 0x7f5b9c15f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.958+0000 7f5ba19f9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b9c0b4ad0 0x7f5b9c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.958+0000 7f5ba19f9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b9c0b45f0 con 0x7f5b9c144570 2023-08-27T11:29:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.958+0000 7f5ba19f9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9c0cf7a0 con 0x7f5b9c0b3e50 2023-08-27T11:29:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.958+0000 7f5ba19f9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c144950 con 0x7f5b9c0b4ad0 2023-08-27T11:29:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b9b7fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b9c0b4ad0 0x7f5b9c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b9b7fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b9c0b4ad0 0x7f5b9c0b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:49590/0 (socket says 172.21.15.112:49590) 2023-08-27T11:29:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b9b7fe700 1 -- 172.21.15.112:0/332476879 learned_addr learned my addr 172.21.15.112:0/332476879 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b9a7fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5b9c0b3e50 0x7f5b9c15f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b9affd700 1 --1- 172.21.15.112:0/332476879 >> v1:172.21.15.138:6789/0 conn(0x7f5b9c144570 0x7f5b9c15ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:39838/0 (socket says 172.21.15.112:39838) 2023-08-27T11:29:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b99ffb700 1 -- 172.21.15.112:0/332476879 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528339726 0 0) 0x7f5b9c0b45f0 con 0x7f5b9c144570 2023-08-27T11:29:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b99ffb700 1 -- 172.21.15.112:0/332476879 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b840035b0 con 0x7f5b9c144570 2023-08-27T11:29:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b9b7fe700 1 -- 172.21.15.112:0/332476879 >> v2:172.21.15.138:3300/0 conn(0x7f5b9c0b3e50 msgr2=0x7f5b9c15f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:18.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b9b7fe700 1 --2- 172.21.15.112:0/332476879 >> v2:172.21.15.138:3300/0 conn(0x7f5b9c0b3e50 0x7f5b9c15f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b9b7fe700 1 -- 172.21.15.112:0/332476879 >> v1:172.21.15.138:6789/0 conn(0x7f5b9c144570 legacy=0x7f5b9c15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:18.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b9b7fe700 1 -- 172.21.15.112:0/332476879 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c0b4310 con 0x7f5b9c0b4ad0 2023-08-27T11:29:18.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b9a7fc700 1 --2- 172.21.15.112:0/332476879 >> v2:172.21.15.138:3300/0 conn(0x7f5b9c0b3e50 0x7f5b9c15f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:18.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b9b7fe700 1 --2- 172.21.15.112:0/332476879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b9c0b4ad0 0x7f5b9c0b4f60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009e50 tx=0x7f5b9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5ca5e8bbae9f82 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:18.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b99ffb700 1 -- 172.21.15.112:0/332476879 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400e070 con 0x7f5b9c0b4ad0 2023-08-27T11:29:18.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b99ffb700 1 -- 172.21.15.112:0/332476879 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9400b910 con 0x7f5b9c0b4ad0 2023-08-27T11:29:18.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.959+0000 7f5b99ffb700 1 -- 172.21.15.112:0/332476879 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400bc80 con 0x7f5b9c0b4ad0 2023-08-27T11:29:18.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.960+0000 7f5ba19f9700 1 -- 172.21.15.112:0/332476879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b9c0b4ad0 msgr2=0x7f5b9c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:18.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.960+0000 7f5ba19f9700 1 --2- 172.21.15.112:0/332476879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b9c0b4ad0 0x7f5b9c0b4f60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009e50 tx=0x7f5b9400b330 comp rx=0 tx=0).stop 2023-08-27T11:29:18.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.960+0000 7f5ba19f9700 1 -- 172.21.15.112:0/332476879 shutdown_connections 2023-08-27T11:29:18.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.960+0000 7f5ba19f9700 1 --2- 172.21.15.112:0/332476879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b9c0b4ad0 0x7f5b9c0b4f60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.960+0000 7f5ba19f9700 1 --2- 172.21.15.112:0/332476879 >> v2:172.21.15.138:3300/0 conn(0x7f5b9c0b3e50 0x7f5b9c15f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.960+0000 7f5ba19f9700 1 -- 172.21.15.112:0/332476879 >> 172.21.15.112:0/332476879 conn(0x7f5b9c06f970 msgr2=0x7f5b9c06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:18.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.960+0000 7f5ba19f9700 1 -- 172.21.15.112:0/332476879 shutdown_connections 2023-08-27T11:29:18.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.960+0000 7f5ba19f9700 1 -- 172.21.15.112:0/332476879 wait complete. 2023-08-27T11:29:18.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.961+0000 7f5ba19f9700 1 Processor -- start 2023-08-27T11:29:18.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.961+0000 7f5ba19f9700 1 -- start start 2023-08-27T11:29:18.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5ba19f9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b9c0b3e50 0x7f5b9c0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:18.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5ba19f9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b9c0b4ad0 0x7f5b9c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:18.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5ba19f9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b9c144570 0x7f5b9c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:18.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5ba19f9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c0b2580 con 0x7f5b9c144570 2023-08-27T11:29:18.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5ba19f9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c1f2960 con 0x7f5b9c0b4ad0 2023-08-27T11:29:18.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5ba19f9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c0ca580 con 0x7f5b9c0b3e50 2023-08-27T11:29:18.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5b9b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b9c144570 0x7f5b9c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:18.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5b9b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b9c144570 0x7f5b9c15de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37942/0 (socket says 172.21.15.112:37942) 2023-08-27T11:29:18.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5b9b7fe700 1 -- 172.21.15.112:0/1649231864 learned_addr learned my addr 172.21.15.112:0/1649231864 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:18.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5b9a7fc700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b9c0b4ad0 0x7f5b9c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:18.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5b9affd700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b9c0b3e50 0x7f5b9c0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:18.985 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5b9b7fe700 1 -- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b9c0b3e50 msgr2=0x7f5b9c0b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:18.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5b9b7fe700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b9c0b3e50 0x7f5b9c0b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5b9b7fe700 1 -- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b9c0b4ad0 msgr2=0x7f5b9c15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:18.986 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.962+0000 7f5b9b7fe700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b9c0b4ad0 0x7f5b9c15d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:18.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.963+0000 7f5b9b7fe700 1 -- 172.21.15.112:0/1649231864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c0094a0 con 0x7f5b9c144570 2023-08-27T11:29:18.987 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.963+0000 7f5b9b7fe700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b9c144570 0x7f5b9c15de40 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400b680 tx=0x7f5b94004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:18.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.963+0000 7f5b83fff700 1 -- 172.21.15.112:0/1649231864 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b94004450 con 0x7f5b9c144570 2023-08-27T11:29:18.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.963+0000 7f5b83fff700 1 -- 172.21.15.112:0/1649231864 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b940045d0 con 0x7f5b9c144570 2023-08-27T11:29:18.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.963+0000 7f5ba19f9700 1 -- 172.21.15.112:0/1649231864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b94009b30 con 0x7f5b9c144570 2023-08-27T11:29:18.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.963+0000 7f5b83fff700 1 -- 172.21.15.112:0/1649231864 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9401c440 con 0x7f5b9c144570 2023-08-27T11:29:18.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.963+0000 7f5ba19f9700 1 -- 172.21.15.112:0/1649231864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9c0cab40 con 0x7f5b9c144570 2023-08-27T11:29:18.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.964+0000 7f5ba19f9700 1 -- 172.21.15.112:0/1649231864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9c1f2af0 con 0x7f5b9c144570 2023-08-27T11:29:18.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.964+0000 7f5b83fff700 1 -- 172.21.15.112:0/1649231864 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5b9401c5a0 con 0x7f5b9c144570 2023-08-27T11:29:18.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.965+0000 7f5b83fff700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b8806a4d0 0x7f5b8806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:18.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.965+0000 7f5b83fff700 1 -- 172.21.15.112:0/1649231864 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5b9408a6e0 con 0x7f5b9c144570 2023-08-27T11:29:18.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.965+0000 7f5b9affd700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b8806a4d0 0x7f5b8806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:18.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.965+0000 7f5b9affd700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b8806a4d0 0x7f5b8806c990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5b90000f60 tx=0x7f5b90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:18.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:18.969+0000 7f5b83fff700 1 -- 172.21.15.112:0/1649231864 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5b94055a40 con 0x7f5b9c144570 2023-08-27T11:29:19.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.159+0000 7f5ba19f9700 1 -- 172.21.15.112:0/1649231864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b9c143830 con 0x7f5b9c144570 2023-08-27T11:29:19.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.160+0000 7f5b83fff700 1 -- 172.21.15.112:0/1649231864 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+14615 (secure 0 0 0) 0x7f5b94059060 con 0x7f5b9c144570 2023-08-27T11:29:19.162 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:19.162 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":163,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:18.362222+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:19.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.162+0000 7f5ba19f9700 1 -- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b8806a4d0 msgr2=0x7f5b8806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.162+0000 7f5ba19f9700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b8806a4d0 0x7f5b8806c990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5b90000f60 tx=0x7f5b90009040 comp rx=0 tx=0).stop 2023-08-27T11:29:19.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.162+0000 7f5ba19f9700 1 -- 172.21.15.112:0/1649231864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b9c144570 msgr2=0x7f5b9c15de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.162+0000 7f5ba19f9700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b9c144570 0x7f5b9c15de40 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400b680 tx=0x7f5b94004ad0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.162+0000 7f5ba19f9700 1 -- 172.21.15.112:0/1649231864 shutdown_connections 2023-08-27T11:29:19.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.162+0000 7f5ba19f9700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f5b8806a4d0 0x7f5b8806c990 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.162+0000 7f5ba19f9700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b9c144570 0x7f5b9c15de40 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.162+0000 7f5ba19f9700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b9c0b3e50 0x7f5b9c0b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.162+0000 7f5ba19f9700 1 --2- 172.21.15.112:0/1649231864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b9c0b4ad0 0x7f5b9c15d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.162+0000 7f5ba19f9700 1 -- 172.21.15.112:0/1649231864 >> 172.21.15.112:0/1649231864 conn(0x7f5b9c06f970 msgr2=0x7f5b9c0cef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.163+0000 7f5ba19f9700 1 -- 172.21.15.112:0/1649231864 shutdown_connections 2023-08-27T11:29:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.163+0000 7f5ba19f9700 1 -- 172.21.15.112:0/1649231864 wait complete. 2023-08-27T11:29:19.176 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:29:19.177 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:29:19.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.299+0000 7fa6e9e25700 1 Processor -- start 2023-08-27T11:29:19.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.299+0000 7fa6e9e25700 1 -- start start 2023-08-27T11:29:19.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e9e25700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6e4146c20 0x7fa6e4160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e9e25700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e406f790 0x7fa6e406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e9e25700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e40b5070 con 0x7fa6e40b3440 2023-08-27T11:29:19.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e9e25700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e4070180 con 0x7fa6e4146c20 2023-08-27T11:29:19.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e9e25700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e4147000 con 0x7fa6e406f790 2023-08-27T11:29:19.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e2ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa6e40b3440 0x7fa6e415e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60562/0 (socket says 172.21.15.112:60562) 2023-08-27T11:29:19.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e2ffd700 1 -- 172.21.15.112:0/2056761172 learned_addr learned my addr 172.21.15.112:0/2056761172 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:19.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e37fe700 1 --2- 172.21.15.112:0/2056761172 >> v2:172.21.15.138:3300/0 conn(0x7fa6e4146c20 0x7fa6e4160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e3fff700 1 --2- 172.21.15.112:0/2056761172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e406f790 0x7fa6e406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e27fc700 1 -- 172.21.15.112:0/2056761172 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876213445 0 0) 0x7fa6d4001110 con 0x7fa6e40b3440 2023-08-27T11:29:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e27fc700 1 -- 172.21.15.112:0/2056761172 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6cc0035b0 con 0x7fa6e40b3440 2023-08-27T11:29:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e37fe700 1 -- 172.21.15.112:0/2056761172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e406f790 msgr2=0x7fa6e406fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e37fe700 1 --2- 172.21.15.112:0/2056761172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e406f790 0x7fa6e406fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e37fe700 1 -- 172.21.15.112:0/2056761172 >> v1:172.21.15.112:6789/0 conn(0x7fa6e40b3440 legacy=0x7fa6e415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e37fe700 1 -- 172.21.15.112:0/2056761172 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e41614a0 con 0x7fa6e4146c20 2023-08-27T11:29:19.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.300+0000 7fa6e37fe700 1 --2- 172.21.15.112:0/2056761172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e4146c20 0x7fa6e4160f60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009840 tx=0x7fa6d8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b57d72cda82a4e5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:19.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.301+0000 7fa6e27fc700 1 -- 172.21.15.112:0/2056761172 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8011830 con 0x7fa6e4146c20 2023-08-27T11:29:19.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.301+0000 7fa6e27fc700 1 -- 172.21.15.112:0/2056761172 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d8011990 con 0x7fa6e4146c20 2023-08-27T11:29:19.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.301+0000 7fa6e27fc700 1 -- 172.21.15.112:0/2056761172 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8011d20 con 0x7fa6e4146c20 2023-08-27T11:29:19.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.301+0000 7fa6e9e25700 1 -- 172.21.15.112:0/2056761172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e4146c20 msgr2=0x7fa6e4160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.301+0000 7fa6e9e25700 1 --2- 172.21.15.112:0/2056761172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e4146c20 0x7fa6e4160f60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009840 tx=0x7fa6d8009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:19.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.301+0000 7fa6e9e25700 1 -- 172.21.15.112:0/2056761172 shutdown_connections 2023-08-27T11:29:19.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.301+0000 7fa6e9e25700 1 --2- 172.21.15.112:0/2056761172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e406f790 0x7fa6e406fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.301+0000 7fa6e9e25700 1 --2- 172.21.15.112:0/2056761172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e4146c20 0x7fa6e4160f60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.301+0000 7fa6e9e25700 1 -- 172.21.15.112:0/2056761172 >> 172.21.15.112:0/2056761172 conn(0x7fa6e40b1e90 msgr2=0x7fa6e40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.301+0000 7fa6e9e25700 1 -- 172.21.15.112:0/2056761172 shutdown_connections 2023-08-27T11:29:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.301+0000 7fa6e9e25700 1 -- 172.21.15.112:0/2056761172 wait complete. 2023-08-27T11:29:19.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.303+0000 7fa6e9e25700 1 Processor -- start 2023-08-27T11:29:19.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.303+0000 7fa6e9e25700 1 -- start start 2023-08-27T11:29:19.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e9e25700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e406f790 0x7fa6e41f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e9e25700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e40b3440 0x7fa6e415f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e9e25700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4146c20 0x7fa6e415f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e9e25700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e4146490 con 0x7fa6e4146c20 2023-08-27T11:29:19.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e9e25700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e4146000 con 0x7fa6e40b3440 2023-08-27T11:29:19.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e9e25700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e41f2c60 con 0x7fa6e406f790 2023-08-27T11:29:19.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4146c20 0x7fa6e415f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4146c20 0x7fa6e415f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37956/0 (socket says 172.21.15.112:37956) 2023-08-27T11:29:19.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e3fff700 1 -- 172.21.15.112:0/3202415515 learned_addr learned my addr 172.21.15.112:0/3202415515 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:19.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e37fe700 1 --2- 172.21.15.112:0/3202415515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e406f790 0x7fa6e41f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e3fff700 1 -- 172.21.15.112:0/3202415515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e406f790 msgr2=0x7fa6e41f2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e3fff700 1 --2- 172.21.15.112:0/3202415515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e406f790 0x7fa6e41f2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e3fff700 1 -- 172.21.15.112:0/3202415515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e40b3440 msgr2=0x7fa6e415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e3fff700 1 --2- 172.21.15.112:0/3202415515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e40b3440 0x7fa6e415f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e3fff700 1 -- 172.21.15.112:0/3202415515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d80094a0 con 0x7fa6e4146c20 2023-08-27T11:29:19.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.304+0000 7fa6e3fff700 1 --2- 172.21.15.112:0/3202415515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4146c20 0x7fa6e415f580 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009dd0 tx=0x7fa6dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:19.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.305+0000 7fa6cbfff700 1 -- 172.21.15.112:0/3202415515 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc009c40 con 0x7fa6e4146c20 2023-08-27T11:29:19.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.305+0000 7fa6cbfff700 1 -- 172.21.15.112:0/3202415515 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6dc00f040 con 0x7fa6e4146c20 2023-08-27T11:29:19.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.305+0000 7fa6e9e25700 1 -- 172.21.15.112:0/3202415515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e41453f0 con 0x7fa6e4146c20 2023-08-27T11:29:19.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.305+0000 7fa6cbfff700 1 -- 172.21.15.112:0/3202415515 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc014480 con 0x7fa6e4146c20 2023-08-27T11:29:19.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.305+0000 7fa6e9e25700 1 -- 172.21.15.112:0/3202415515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e41459b0 con 0x7fa6e4146c20 2023-08-27T11:29:19.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.306+0000 7fa6e9e25700 1 -- 172.21.15.112:0/3202415515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e41461b0 con 0x7fa6e4146c20 2023-08-27T11:29:19.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.306+0000 7fa6cbfff700 1 -- 172.21.15.112:0/3202415515 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa6dc0145e0 con 0x7fa6e4146c20 2023-08-27T11:29:19.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.306+0000 7fa6cbfff700 1 --2- 172.21.15.112:0/3202415515 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa6d006a4d0 0x7fa6d006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.307+0000 7fa6cbfff700 1 -- 172.21.15.112:0/3202415515 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa6dc089270 con 0x7fa6e4146c20 2023-08-27T11:29:19.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.307+0000 7fa6e37fe700 1 --2- 172.21.15.112:0/3202415515 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa6d006a4d0 0x7fa6d006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.307+0000 7fa6e37fe700 1 --2- 172.21.15.112:0/3202415515 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa6d006a4d0 0x7fa6d006c990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8000c00 tx=0x7fa6d801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:19.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.311+0000 7fa6cbfff700 1 -- 172.21.15.112:0/3202415515 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa6e41461b0 con 0x7fa6e4146c20 2023-08-27T11:29:19.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.498+0000 7fa6e9e25700 1 -- 172.21.15.112:0/3202415515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa6e41461b0 con 0x7fa6e4146c20 2023-08-27T11:29:19.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.498+0000 7fa6cbfff700 1 -- 172.21.15.112:0/3202415515 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v163) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa6e41461b0 con 0x7fa6e4146c20 2023-08-27T11:29:19.500 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:29:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.500+0000 7fa6e9e25700 1 -- 172.21.15.112:0/3202415515 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa6d006a4d0 msgr2=0x7fa6d006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.500+0000 7fa6e9e25700 1 --2- 172.21.15.112:0/3202415515 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa6d006a4d0 0x7fa6d006c990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8000c00 tx=0x7fa6d801c040 comp rx=0 tx=0).stop 2023-08-27T11:29:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.500+0000 7fa6e9e25700 1 -- 172.21.15.112:0/3202415515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4146c20 msgr2=0x7fa6e415f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.500+0000 7fa6e9e25700 1 --2- 172.21.15.112:0/3202415515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4146c20 0x7fa6e415f580 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009dd0 tx=0x7fa6dc00c3a0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.500+0000 7fa6e9e25700 1 -- 172.21.15.112:0/3202415515 shutdown_connections 2023-08-27T11:29:19.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.500+0000 7fa6e9e25700 1 --2- 172.21.15.112:0/3202415515 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa6d006a4d0 0x7fa6d006c990 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.500+0000 7fa6e9e25700 1 --2- 172.21.15.112:0/3202415515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4146c20 0x7fa6e415f580 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.500+0000 7fa6e9e25700 1 --2- 172.21.15.112:0/3202415515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e406f790 0x7fa6e41f2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.500+0000 7fa6e9e25700 1 --2- 172.21.15.112:0/3202415515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e40b3440 0x7fa6e415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.500+0000 7fa6e9e25700 1 -- 172.21.15.112:0/3202415515 >> 172.21.15.112:0/3202415515 conn(0x7fa6e40b1e90 msgr2=0x7fa6e40710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.500+0000 7fa6e9e25700 1 -- 172.21.15.112:0/3202415515 shutdown_connections 2023-08-27T11:29:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.500+0000 7fa6e9e25700 1 -- 172.21.15.112:0/3202415515 wait complete. 2023-08-27T11:29:19.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:29:19.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.636+0000 7fa0194d3700 1 Processor -- start 2023-08-27T11:29:19.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.636+0000 7fa0194d3700 1 -- start start 2023-08-27T11:29:19.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.637+0000 7fa0194d3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa014146fd0 0x7fa0140b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.637+0000 7fa0194d3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa014070eb0 0x7fa014071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.637+0000 7fa0194d3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0140d1500 con 0x7fa014146bf0 2023-08-27T11:29:19.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.637+0000 7fa0194d3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa014071880 con 0x7fa014146fd0 2023-08-27T11:29:19.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.637+0000 7fa0194d3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140d0460 con 0x7fa014070eb0 2023-08-27T11:29:19.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.637+0000 7fa0127fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa014146fd0 0x7fa0140b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.637+0000 7fa012ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa014146bf0 0x7fa0140cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60574/0 (socket says 172.21.15.112:60574) 2023-08-27T11:29:19.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.637+0000 7fa012ffd700 1 -- 172.21.15.112:0/2464561256 learned_addr learned my addr 172.21.15.112:0/2464561256 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:19.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa0127fc700 1 -- 172.21.15.112:0/2464561256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa014070eb0 msgr2=0x7fa014071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:19.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa0137fe700 1 --2- 172.21.15.112:0/2464561256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa014070eb0 0x7fa014071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa0127fc700 1 --2- 172.21.15.112:0/2464561256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa014070eb0 0x7fa014071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa0127fc700 1 -- 172.21.15.112:0/2464561256 >> v1:172.21.15.112:6789/0 conn(0x7fa014146bf0 legacy=0x7fa0140cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa011ffb700 1 -- 172.21.15.112:0/2464561256 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073251969 0 0) 0x7fa0140d1500 con 0x7fa014146bf0 2023-08-27T11:29:19.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa0127fc700 1 -- 172.21.15.112:0/2464561256 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0140d0710 con 0x7fa014146fd0 2023-08-27T11:29:19.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa0137fe700 1 --2- 172.21.15.112:0/2464561256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa014070eb0 0x7fa014071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:19.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa0127fc700 1 --2- 172.21.15.112:0/2464561256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa014146fd0 0x7fa0140b5db0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fa004009840 tx=0x7fa004009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd063f5ed6694f3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:19.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa011ffb700 1 -- 172.21.15.112:0/2464561256 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004011830 con 0x7fa014146fd0 2023-08-27T11:29:19.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa011ffb700 1 -- 172.21.15.112:0/2464561256 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa004011990 con 0x7fa014146fd0 2023-08-27T11:29:19.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa011ffb700 1 -- 172.21.15.112:0/2464561256 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004011d20 con 0x7fa014146fd0 2023-08-27T11:29:19.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa0194d3700 1 -- 172.21.15.112:0/2464561256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa014146fd0 msgr2=0x7fa0140b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa0194d3700 1 --2- 172.21.15.112:0/2464561256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa014146fd0 0x7fa0140b5db0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fa004009840 tx=0x7fa004009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:19.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa0194d3700 1 -- 172.21.15.112:0/2464561256 shutdown_connections 2023-08-27T11:29:19.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa0194d3700 1 --2- 172.21.15.112:0/2464561256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa014070eb0 0x7fa014071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa0194d3700 1 --2- 172.21.15.112:0/2464561256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa014146fd0 0x7fa0140b5db0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.638+0000 7fa0194d3700 1 -- 172.21.15.112:0/2464561256 >> 172.21.15.112:0/2464561256 conn(0x7fa01406c450 msgr2=0x7fa01406f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.639+0000 7fa0194d3700 1 -- 172.21.15.112:0/2464561256 shutdown_connections 2023-08-27T11:29:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.639+0000 7fa0194d3700 1 -- 172.21.15.112:0/2464561256 wait complete. 2023-08-27T11:29:19.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.640+0000 7fa0194d3700 1 Processor -- start 2023-08-27T11:29:19.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.640+0000 7fa0194d3700 1 -- start start 2023-08-27T11:29:19.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa0194d3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa014070eb0 0x7fa0141587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa0194d3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa014146bf0 0x7fa014160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa0194d3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa014146fd0 0x7fa01413d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa0194d3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa014158d20 con 0x7fa014070eb0 2023-08-27T11:29:19.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa0194d3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0141f49f0 con 0x7fa014146fd0 2023-08-27T11:29:19.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa0194d3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0141f4cd0 con 0x7fa014146bf0 2023-08-27T11:29:19.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa012ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa014070eb0 0x7fa0141587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa012ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa014070eb0 0x7fa0141587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37964/0 (socket says 172.21.15.112:37964) 2023-08-27T11:29:19.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa012ffd700 1 -- 172.21.15.112:0/1138269564 learned_addr learned my addr 172.21.15.112:0/1138269564 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:19.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa0127fc700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa014146bf0 0x7fa014160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa0137fe700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa014146fd0 0x7fa01413d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa012ffd700 1 -- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa014146bf0 msgr2=0x7fa014160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa012ffd700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa014146bf0 0x7fa014160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa012ffd700 1 -- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa014146fd0 msgr2=0x7fa01413d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa012ffd700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa014146fd0 0x7fa01413d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa012ffd700 1 -- 172.21.15.112:0/1138269564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0040094a0 con 0x7fa014070eb0 2023-08-27T11:29:19.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa012ffd700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa014070eb0 0x7fa0141587e0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fa0080011e0 tx=0x7fa00800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:19.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7f9ffb7fe700 1 -- 172.21.15.112:0/1138269564 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008004c80 con 0x7fa014070eb0 2023-08-27T11:29:19.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7f9ffb7fe700 1 -- 172.21.15.112:0/1138269564 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa008004e00 con 0x7fa014070eb0 2023-08-27T11:29:19.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa0194d3700 1 -- 172.21.15.112:0/1138269564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0141f4fb0 con 0x7fa014070eb0 2023-08-27T11:29:19.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7f9ffb7fe700 1 -- 172.21.15.112:0/1138269564 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008007310 con 0x7fa014070eb0 2023-08-27T11:29:19.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.641+0000 7fa0194d3700 1 -- 172.21.15.112:0/1138269564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0141f5570 con 0x7fa014070eb0 2023-08-27T11:29:19.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.642+0000 7fa0194d3700 1 -- 172.21.15.112:0/1138269564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa014067e90 con 0x7fa014070eb0 2023-08-27T11:29:19.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.643+0000 7f9ffb7fe700 1 -- 172.21.15.112:0/1138269564 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa008007470 con 0x7fa014070eb0 2023-08-27T11:29:19.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.643+0000 7f9ffb7fe700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa00006a4d0 0x7fa00006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.643+0000 7f9ffb7fe700 1 -- 172.21.15.112:0/1138269564 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa008088440 con 0x7fa014070eb0 2023-08-27T11:29:19.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.643+0000 7fa0127fc700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa00006a4d0 0x7fa00006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.644+0000 7fa0127fc700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa00006a4d0 0x7fa00006c990 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa0040090b0 tx=0x7fa004008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:19.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.647+0000 7f9ffb7fe700 1 -- 172.21.15.112:0/1138269564 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa008053b40 con 0x7fa014070eb0 2023-08-27T11:29:19.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.835+0000 7fa0194d3700 1 -- 172.21.15.112:0/1138269564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa014067a20 con 0x7fa014070eb0 2023-08-27T11:29:19.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.835+0000 7f9ffb7fe700 1 -- 172.21.15.112:0/1138269564 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v163) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa008057160 con 0x7fa014070eb0 2023-08-27T11:29:19.837 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:29:19.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.837+0000 7fa0194d3700 1 -- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa00006a4d0 msgr2=0x7fa00006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.837+0000 7fa0194d3700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa00006a4d0 0x7fa00006c990 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa0040090b0 tx=0x7fa004008030 comp rx=0 tx=0).stop 2023-08-27T11:29:19.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.837+0000 7fa0194d3700 1 -- 172.21.15.112:0/1138269564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa014070eb0 msgr2=0x7fa0141587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.839 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.837+0000 7fa0194d3700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa014070eb0 0x7fa0141587e0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fa0080011e0 tx=0x7fa00800cae0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.837+0000 7fa0194d3700 1 -- 172.21.15.112:0/1138269564 shutdown_connections 2023-08-27T11:29:19.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.837+0000 7fa0194d3700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa00006a4d0 0x7fa00006c990 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.837+0000 7fa0194d3700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa014146fd0 0x7fa01413d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.837+0000 7fa0194d3700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa014070eb0 0x7fa0141587e0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.840 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.837+0000 7fa0194d3700 1 --2- 172.21.15.112:0/1138269564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa014146bf0 0x7fa014160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.837+0000 7fa0194d3700 1 -- 172.21.15.112:0/1138269564 >> 172.21.15.112:0/1138269564 conn(0x7fa01406c450 msgr2=0x7fa01406dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:19.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.837+0000 7fa0194d3700 1 -- 172.21.15.112:0/1138269564 shutdown_connections 2023-08-27T11:29:19.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.837+0000 7fa0194d3700 1 -- 172.21.15.112:0/1138269564 wait complete. 2023-08-27T11:29:19.851 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:29:19.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.969+0000 7fd2c8e2c700 1 Processor -- start 2023-08-27T11:29:19.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.969+0000 7fd2c8e2c700 1 -- start start 2023-08-27T11:29:19.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.970+0000 7fd2c8e2c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c40b1400 0x7fd2c40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.970+0000 7fd2c8e2c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c4161850 con 0x7fd2c4161ce0 2023-08-27T11:29:19.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.970+0000 7fd2c8e2c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c40cfc10 con 0x7fd2c40b17e0 2023-08-27T11:29:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.970+0000 7fd2c8e2c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c4161200 con 0x7fd2c40b1400 2023-08-27T11:29:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.970+0000 7fd2c2d9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd2c4161ce0 0x7fd2c406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60588/0 (socket says 172.21.15.112:60588) 2023-08-27T11:29:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.970+0000 7fd2c2d9d700 1 -- 172.21.15.112:0/4029570493 learned_addr learned my addr 172.21.15.112:0/4029570493 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483093243 0 0) 0x7fd2c4161850 con 0x7fd2c4161ce0 2023-08-27T11:29:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c259c700 1 --2- 172.21.15.112:0/4029570493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c40b1400 0x7fd2c40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2ac0035b0 con 0x7fd2c4161ce0 2023-08-27T11:29:19.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4293794870 0 0) 0x7fd2ac0035b0 con 0x7fd2c4161ce0 2023-08-27T11:29:19.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c40b1400 msgr2=0x7fd2c40cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:19.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 --2- 172.21.15.112:0/4029570493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c40b1400 0x7fd2c40cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 >> v1:172.21.15.138:6789/0 conn(0x7fd2c40b17e0 legacy=0x7fd2c406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c41614e0 con 0x7fd2c4161ce0 2023-08-27T11:29:19.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fd2bc0019f0 con 0x7fd2c4161ce0 2023-08-27T11:29:19.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 >> v1:172.21.15.112:6789/0 conn(0x7fd2c4161ce0 legacy=0x7fd2c406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 --2- 172.21.15.112:0/4029570493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c40b1400 0x7fd2ac003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 --2- 172.21.15.112:0/4029570493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2ac003f70 0x7fd2ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 --2- 172.21.15.112:0/4029570493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac004d90 0x7fd2ac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c4161200 con 0x7fd2ac004d90 2023-08-27T11:29:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac00bb20 con 0x7fd2ac003f70 2023-08-27T11:29:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac00bce0 con 0x7fd2c40b1400 2023-08-27T11:29:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c2d9d700 1 --2- 172.21.15.112:0/4029570493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac004d90 0x7fd2ac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c259c700 1 --2- 172.21.15.112:0/4029570493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c40b1400 0x7fd2ac003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c1d9b700 1 --2- 172.21.15.112:0/4029570493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2ac003f70 0x7fd2ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c2d9d700 1 -- 172.21.15.112:0/4029570493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c40b1400 msgr2=0x7fd2ac003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:19.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c2d9d700 1 --2- 172.21.15.112:0/4029570493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c40b1400 0x7fd2ac003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c2d9d700 1 -- 172.21.15.112:0/4029570493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2ac003f70 msgr2=0x7fd2ac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:19.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c2d9d700 1 --2- 172.21.15.112:0/4029570493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2ac003f70 0x7fd2ac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.971+0000 7fd2c2d9d700 1 -- 172.21.15.112:0/4029570493 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac0035b0 con 0x7fd2ac004d90 2023-08-27T11:29:19.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.972+0000 7fd2c2d9d700 1 --2- 172.21.15.112:0/4029570493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac004d90 0x7fd2ac005210 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00a830 tx=0x7fd2ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6ce15acccc1dc9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:19.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.972+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc012600 con 0x7fd2ac004d90 2023-08-27T11:29:19.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.972+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc012780 con 0x7fd2ac004d90 2023-08-27T11:29:19.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.972+0000 7fd2c159a700 1 -- 172.21.15.112:0/4029570493 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc012b30 con 0x7fd2ac004d90 2023-08-27T11:29:19.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.972+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/4029570493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac004d90 msgr2=0x7fd2ac005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.972+0000 7fd2c8e2c700 1 --2- 172.21.15.112:0/4029570493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac004d90 0x7fd2ac005210 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00a830 tx=0x7fd2ac005880 comp rx=0 tx=0).stop 2023-08-27T11:29:19.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.972+0000 7fd2c2d9d700 1 -- 172.21.15.112:0/4029570493 reap_dead start 2023-08-27T11:29:19.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.972+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/4029570493 shutdown_connections 2023-08-27T11:29:19.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.972+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/4029570493 >> 172.21.15.112:0/4029570493 conn(0x7fd2c40d0000 msgr2=0x7fd2c40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.972+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/4029570493 shutdown_connections 2023-08-27T11:29:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.972+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/4029570493 wait complete. 2023-08-27T11:29:19.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.974+0000 7fd2c8e2c700 1 Processor -- start 2023-08-27T11:29:19.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.974+0000 7fd2c8e2c700 1 -- start start 2023-08-27T11:29:19.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.974+0000 7fd2c8e2c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac004d90 0x7fd2c41f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c8e2c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2c414eba0 0x7fd2c40cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c8e2c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c414f020 0x7fd2c40cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c8e2c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c41f3950 con 0x7fd2ac004d90 2023-08-27T11:29:19.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c8e2c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c41f3c30 con 0x7fd2c414eba0 2023-08-27T11:29:19.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c8e2c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c41f6600 con 0x7fd2c414f020 2023-08-27T11:29:19.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac004d90 0x7fd2c41f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac004d90 0x7fd2c41f6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37984/0 (socket says 172.21.15.112:37984) 2023-08-27T11:29:19.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c259c700 1 -- 172.21.15.112:0/1964735075 learned_addr learned my addr 172.21.15.112:0/1964735075 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:19.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c1d9b700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2c414eba0 0x7fd2c40cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c2d9d700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c414f020 0x7fd2c40cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c259c700 1 -- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c414f020 msgr2=0x7fd2c40cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c259c700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c414f020 0x7fd2c40cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c259c700 1 -- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2c414eba0 msgr2=0x7fd2c40cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:19.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c259c700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2c414eba0 0x7fd2c40cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:19.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c259c700 1 -- 172.21.15.112:0/1964735075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c41f68e0 con 0x7fd2ac004d90 2023-08-27T11:29:19.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c2d9d700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c414f020 0x7fd2c40cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:19.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c259c700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac004d90 0x7fd2c41f6e00 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8011aa0 tx=0x7fd2b8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:19.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2b2ffd700 1 -- 172.21.15.112:0/1964735075 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b800f940 con 0x7fd2ac004d90 2023-08-27T11:29:19.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2b2ffd700 1 -- 172.21.15.112:0/1964735075 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b800faa0 con 0x7fd2ac004d90 2023-08-27T11:29:19.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/1964735075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c41f7340 con 0x7fd2ac004d90 2023-08-27T11:29:19.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2b2ffd700 1 -- 172.21.15.112:0/1964735075 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b800fe50 con 0x7fd2ac004d90 2023-08-27T11:29:19.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.975+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/1964735075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c41f2d10 con 0x7fd2ac004d90 2023-08-27T11:29:19.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.976+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/1964735075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c40cf560 con 0x7fd2ac004d90 2023-08-27T11:29:19.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.977+0000 7fd2b2ffd700 1 -- 172.21.15.112:0/1964735075 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fd2b8014440 con 0x7fd2ac004d90 2023-08-27T11:29:19.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.977+0000 7fd2b2ffd700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd2a406a4d0 0x7fd2a406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:19.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.977+0000 7fd2b2ffd700 1 -- 172.21.15.112:0/1964735075 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd2b80890a0 con 0x7fd2ac004d90 2023-08-27T11:29:19.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.977+0000 7fd2c1d9b700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd2a406a4d0 0x7fd2a406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:19.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.978+0000 7fd2c1d9b700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd2a406a4d0 0x7fd2a406c990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40c6cf0 tx=0x7fd2b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:19.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:19.981+0000 7fd2b2ffd700 1 -- 172.21.15.112:0/1964735075 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd2b8058050 con 0x7fd2ac004d90 2023-08-27T11:29:20.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.173+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/1964735075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd2c4067a20 con 0x7fd2ac004d90 2023-08-27T11:29:20.217 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.173+0000 7fd2b2ffd700 1 -- 172.21.15.112:0/1964735075 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v163) v1 ==== 84+0+126 (secure 0 0 0) 0x7fd2b8057be0 con 0x7fd2ac004d90 2023-08-27T11:29:20.218 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:20.218 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:29:20.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:20.218 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2023-08-27T11:29:20.219 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "rbd" 2023-08-27T11:29:20.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:29:20.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:20.219 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 2, 2023-08-27T11:29:20.220 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2023-08-27T11:29:20.220 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:20.220 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:29:20.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.175+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd2a406a4d0 msgr2=0x7fd2a406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:20.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.175+0000 7fd2c8e2c700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd2a406a4d0 0x7fd2a406c990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40c6cf0 tx=0x7fd2b4009040 comp rx=0 tx=0).stop 2023-08-27T11:29:20.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.175+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/1964735075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac004d90 msgr2=0x7fd2c41f6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:20.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.175+0000 7fd2c8e2c700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac004d90 0x7fd2c41f6e00 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8011aa0 tx=0x7fd2b8011db0 comp rx=0 tx=0).stop 2023-08-27T11:29:20.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.175+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/1964735075 shutdown_connections 2023-08-27T11:29:20.222 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.175+0000 7fd2c8e2c700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd2a406a4d0 0x7fd2a406c990 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:20.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.175+0000 7fd2c8e2c700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c414f020 0x7fd2c40cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:20.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.175+0000 7fd2c8e2c700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac004d90 0x7fd2c41f6e00 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:20.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.175+0000 7fd2c8e2c700 1 --2- 172.21.15.112:0/1964735075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2c414eba0 0x7fd2c40cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:20.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.175+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/1964735075 >> 172.21.15.112:0/1964735075 conn(0x7fd2c40d0000 msgr2=0x7fd2c40cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:20.223 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.175+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/1964735075 shutdown_connections 2023-08-27T11:29:20.224 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.175+0000 7fd2c8e2c700 1 -- 172.21.15.112:0/1964735075 wait complete. 2023-08-27T11:29:20.224 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:29:20.225 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:29:20.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.349+0000 7f4aea828700 1 Processor -- start 2023-08-27T11:29:20.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.349+0000 7f4aea828700 1 -- start start 2023-08-27T11:29:20.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4aea828700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4ae40b5d60 0x7f4ae40ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:20.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4aea828700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4ae415f8a0 0x7f4ae415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:20.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4aea828700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae40ced80 con 0x7f4ae4144b10 2023-08-27T11:29:20.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4aea828700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae40cefd0 con 0x7f4ae40b5d60 2023-08-27T11:29:20.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4aea828700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae40cf2b0 con 0x7f4ae415f8a0 2023-08-27T11:29:20.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4ae37fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4ae40b5d60 0x7f4ae40ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:20.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4ae37fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4ae40b5d60 0x7f4ae40ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37994/0 (socket says 172.21.15.112:37994) 2023-08-27T11:29:20.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4ae37fe700 1 -- 172.21.15.112:0/3096599831 learned_addr learned my addr 172.21.15.112:0/3096599831 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:20.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4ae8dc5700 1 --2- 172.21.15.112:0/3096599831 >> v2:172.21.15.138:3300/0 conn(0x7f4ae415f8a0 0x7f4ae415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:20.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4ae37fe700 1 -- 172.21.15.112:0/3096599831 >> v2:172.21.15.138:3300/0 conn(0x7f4ae415f8a0 msgr2=0x7f4ae415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:20.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4ae37fe700 1 --2- 172.21.15.112:0/3096599831 >> v2:172.21.15.138:3300/0 conn(0x7f4ae415f8a0 0x7f4ae415baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:20.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4ae37fe700 1 -- 172.21.15.112:0/3096599831 >> v1:172.21.15.138:6789/0 conn(0x7f4ae4144b10 legacy=0x7f4ae40b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:20.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4ae37fe700 1 -- 172.21.15.112:0/3096599831 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae40cf590 con 0x7f4ae40b5d60 2023-08-27T11:29:20.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.350+0000 7f4ae37fe700 1 --2- 172.21.15.112:0/3096599831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae40b5d60 0x7f4ae40ce840 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4ae406e7c0 tx=0x7f4ad4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01143555a754b12 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:20.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.351+0000 7f4ae2ffd700 1 -- 172.21.15.112:0/3096599831 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4004230 con 0x7f4ae40b5d60 2023-08-27T11:29:20.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.351+0000 7f4ae2ffd700 1 -- 172.21.15.112:0/3096599831 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad4004390 con 0x7f4ae40b5d60 2023-08-27T11:29:20.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.351+0000 7f4ae2ffd700 1 -- 172.21.15.112:0/3096599831 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40036a0 con 0x7f4ae40b5d60 2023-08-27T11:29:20.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.351+0000 7f4aea828700 1 -- 172.21.15.112:0/3096599831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae40b5d60 msgr2=0x7f4ae40ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:20.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.351+0000 7f4aea828700 1 --2- 172.21.15.112:0/3096599831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae40b5d60 0x7f4ae40ce840 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4ae406e7c0 tx=0x7f4ad4009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:20.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.351+0000 7f4aea828700 1 -- 172.21.15.112:0/3096599831 shutdown_connections 2023-08-27T11:29:20.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.351+0000 7f4aea828700 1 --2- 172.21.15.112:0/3096599831 >> v2:172.21.15.138:3300/0 conn(0x7f4ae415f8a0 0x7f4ae415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:20.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.351+0000 7f4aea828700 1 --2- 172.21.15.112:0/3096599831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae40b5d60 0x7f4ae40ce840 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:20.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.351+0000 7f4aea828700 1 -- 172.21.15.112:0/3096599831 >> 172.21.15.112:0/3096599831 conn(0x7f4ae40b1be0 msgr2=0x7f4ae40b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:20.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.351+0000 7f4aea828700 1 -- 172.21.15.112:0/3096599831 shutdown_connections 2023-08-27T11:29:20.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.351+0000 7f4aea828700 1 -- 172.21.15.112:0/3096599831 wait complete. 2023-08-27T11:29:20.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.352+0000 7f4aea828700 1 Processor -- start 2023-08-27T11:29:20.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.352+0000 7f4aea828700 1 -- start start 2023-08-27T11:29:20.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.353+0000 7f4aea828700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae40b5d60 0x7f4ae41410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:20.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.353+0000 7f4aea828700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ae4144b10 0x7f4ae41f08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:20.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.353+0000 7f4aea828700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ae415f8a0 0x7f4ae415d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:20.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.353+0000 7f4aea828700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae40c9140 con 0x7f4ae40b5d60 2023-08-27T11:29:20.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.353+0000 7f4aea828700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae40c9fa0 con 0x7f4ae4144b10 2023-08-27T11:29:20.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.353+0000 7f4aea828700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae40c9390 con 0x7f4ae415f8a0 2023-08-27T11:29:20.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.353+0000 7f4ae3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae40b5d60 0x7f4ae41410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:20.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.353+0000 7f4ae3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae40b5d60 0x7f4ae41410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37998/0 (socket says 172.21.15.112:37998) 2023-08-27T11:29:20.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.353+0000 7f4ae3fff700 1 -- 172.21.15.112:0/3078129900 learned_addr learned my addr 172.21.15.112:0/3078129900 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:20.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.353+0000 7f4ae37fe700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ae4144b10 0x7f4ae41f08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:20.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.353+0000 7f4ae8dc5700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ae415f8a0 0x7f4ae415d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:20.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.354+0000 7f4ae3fff700 1 -- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ae415f8a0 msgr2=0x7f4ae415d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:20.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.354+0000 7f4ae3fff700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ae415f8a0 0x7f4ae415d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:20.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.354+0000 7f4ae3fff700 1 -- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ae4144b10 msgr2=0x7f4ae41f08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:20.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.354+0000 7f4ae3fff700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ae4144b10 0x7f4ae41f08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:20.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.354+0000 7f4ae3fff700 1 -- 172.21.15.112:0/3078129900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad4009550 con 0x7f4ae40b5d60 2023-08-27T11:29:20.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.354+0000 7f4ae3fff700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae40b5d60 0x7f4ae41410b0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c500 tx=0x7f4ad800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:20.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.354+0000 7f4ae0ff9700 1 -- 172.21.15.112:0/3078129900 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8011790 con 0x7f4ae40b5d60 2023-08-27T11:29:20.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.354+0000 7f4ae0ff9700 1 -- 172.21.15.112:0/3078129900 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad80118f0 con 0x7f4ae40b5d60 2023-08-27T11:29:20.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.354+0000 7f4aea828700 1 -- 172.21.15.112:0/3078129900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae40c9670 con 0x7f4ae40b5d60 2023-08-27T11:29:20.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.354+0000 7f4ae0ff9700 1 -- 172.21.15.112:0/3078129900 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8011ca0 con 0x7f4ae40b5d60 2023-08-27T11:29:20.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.354+0000 7f4aea828700 1 -- 172.21.15.112:0/3078129900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae40c9c30 con 0x7f4ae40b5d60 2023-08-27T11:29:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.355+0000 7f4aea828700 1 -- 172.21.15.112:0/3078129900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae4067a20 con 0x7f4ae40b5d60 2023-08-27T11:29:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.355+0000 7f4ae0ff9700 1 -- 172.21.15.112:0/3078129900 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4ad801c460 con 0x7f4ae40b5d60 2023-08-27T11:29:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.356+0000 7f4ae0ff9700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4ad006a4d0 0x7f4ad006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.356+0000 7f4ae0ff9700 1 -- 172.21.15.112:0/3078129900 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4ad80887f0 con 0x7f4ae40b5d60 2023-08-27T11:29:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.356+0000 7f4ae37fe700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4ad006a4d0 0x7f4ad006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.357+0000 7f4ae37fe700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4ad006a4d0 0x7f4ad006c990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009840 tx=0x7f4ad4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.360+0000 7f4ae0ff9700 1 -- 172.21.15.112:0/3078129900 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4ad8053c50 con 0x7f4ae40b5d60 2023-08-27T11:29:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:20.550+0000 7f4aea828700 1 -- 172.21.15.112:0/3078129900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4ae40ce940 con 0x7f4ae40b5d60 2023-08-27T11:29:21.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.539+0000 7f4ae0ff9700 1 -- 172.21.15.112:0/3078129900 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v164) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4ad8057270 con 0x7f4ae40b5d60 2023-08-27T11:29:21.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.542+0000 7f4aea828700 1 -- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4ad006a4d0 msgr2=0x7f4ad006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:21.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.542+0000 7f4aea828700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4ad006a4d0 0x7f4ad006c990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009840 tx=0x7f4ad4008030 comp rx=0 tx=0).stop 2023-08-27T11:29:21.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.543+0000 7f4aea828700 1 -- 172.21.15.112:0/3078129900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae40b5d60 msgr2=0x7f4ae41410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:21.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.543+0000 7f4aea828700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae40b5d60 0x7f4ae41410b0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c500 tx=0x7f4ad800c8c0 comp rx=0 tx=0).stop 2023-08-27T11:29:21.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.543+0000 7f4aea828700 1 -- 172.21.15.112:0/3078129900 shutdown_connections 2023-08-27T11:29:21.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.543+0000 7f4aea828700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4ad006a4d0 0x7f4ad006c990 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:21.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.543+0000 7f4aea828700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ae415f8a0 0x7f4ae415d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:21.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.543+0000 7f4aea828700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae40b5d60 0x7f4ae41410b0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:21.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.543+0000 7f4aea828700 1 --2- 172.21.15.112:0/3078129900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ae4144b10 0x7f4ae41f08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:21.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.543+0000 7f4aea828700 1 -- 172.21.15.112:0/3078129900 >> 172.21.15.112:0/3078129900 conn(0x7f4ae40b1be0 msgr2=0x7f4ae4073380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:21.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.543+0000 7f4aea828700 1 -- 172.21.15.112:0/3078129900 shutdown_connections 2023-08-27T11:29:21.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.543+0000 7f4aea828700 1 -- 172.21.15.112:0/3078129900 wait complete. 2023-08-27T11:29:21.549 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2023-08-27T11:29:21.558 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:29:21.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.685+0000 7f591c162700 1 Processor -- start 2023-08-27T11:29:21.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.686+0000 7f591c162700 1 -- start start 2023-08-27T11:29:21.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.686+0000 7f591c162700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5914146c20 0x7f5914160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:21.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.686+0000 7f591c162700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f591406f790 0x7f591406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:21.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.686+0000 7f591c162700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59140b5070 con 0x7f59140b3440 2023-08-27T11:29:21.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.686+0000 7f591c162700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5914070180 con 0x7f591406f790 2023-08-27T11:29:21.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.686+0000 7f591c162700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5914147000 con 0x7f5914146c20 2023-08-27T11:29:21.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.687+0000 7f591a6ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f591406f790 0x7f591406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:21.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.687+0000 7f591a6ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f591406f790 0x7f591406fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38010/0 (socket says 172.21.15.112:38010) 2023-08-27T11:29:21.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.687+0000 7f591a6ff700 1 -- 172.21.15.112:0/3811732182 learned_addr learned my addr 172.21.15.112:0/3811732182 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:21.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.687+0000 7f5919efe700 1 --2- 172.21.15.112:0/3811732182 >> v2:172.21.15.138:3300/0 conn(0x7f5914146c20 0x7f5914160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:21.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.687+0000 7f591a6ff700 1 -- 172.21.15.112:0/3811732182 >> v2:172.21.15.138:3300/0 conn(0x7f5914146c20 msgr2=0x7f5914160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:21.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.687+0000 7f591a6ff700 1 --2- 172.21.15.112:0/3811732182 >> v2:172.21.15.138:3300/0 conn(0x7f5914146c20 0x7f5914160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:21.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.687+0000 7f591a6ff700 1 -- 172.21.15.112:0/3811732182 >> v1:172.21.15.138:6789/0 conn(0x7f59140b3440 legacy=0x7f591415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:21.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.687+0000 7f591a6ff700 1 -- 172.21.15.112:0/3811732182 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59141614a0 con 0x7f591406f790 2023-08-27T11:29:21.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.687+0000 7f591a6ff700 1 --2- 172.21.15.112:0/3811732182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591406f790 0x7f591406fc40 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f59140b2d00 tx=0x7f590c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f9772a2a1894ad server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:21.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.687+0000 7f5918efc700 1 -- 172.21.15.112:0/3811732182 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c00e070 con 0x7f591406f790 2023-08-27T11:29:21.692 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.687+0000 7f5918efc700 1 -- 172.21.15.112:0/3811732182 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c003a20 con 0x7f591406f790 2023-08-27T11:29:21.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.687+0000 7f5918efc700 1 -- 172.21.15.112:0/3811732182 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c003dd0 con 0x7f591406f790 2023-08-27T11:29:21.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.688+0000 7f591c162700 1 -- 172.21.15.112:0/3811732182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591406f790 msgr2=0x7f591406fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:21.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.688+0000 7f591c162700 1 --2- 172.21.15.112:0/3811732182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591406f790 0x7f591406fc40 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f59140b2d00 tx=0x7f590c00b330 comp rx=0 tx=0).stop 2023-08-27T11:29:21.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.688+0000 7f591c162700 1 -- 172.21.15.112:0/3811732182 shutdown_connections 2023-08-27T11:29:21.693 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.688+0000 7f591c162700 1 --2- 172.21.15.112:0/3811732182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591406f790 0x7f591406fc40 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:21.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.688+0000 7f591c162700 1 --2- 172.21.15.112:0/3811732182 >> v2:172.21.15.138:3300/0 conn(0x7f5914146c20 0x7f5914160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:21.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.688+0000 7f591c162700 1 -- 172.21.15.112:0/3811732182 >> 172.21.15.112:0/3811732182 conn(0x7f59140b1e90 msgr2=0x7f59140b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:21.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.688+0000 7f591c162700 1 -- 172.21.15.112:0/3811732182 shutdown_connections 2023-08-27T11:29:21.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.688+0000 7f591c162700 1 -- 172.21.15.112:0/3811732182 wait complete. 2023-08-27T11:29:21.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.689+0000 7f591c162700 1 Processor -- start 2023-08-27T11:29:21.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.689+0000 7f591c162700 1 -- start start 2023-08-27T11:29:21.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f591c162700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591406f790 0x7f5914135a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:21.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f591c162700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59140b3440 0x7f59140c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:21.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f591c162700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5914146c20 0x7f59140c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:21.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f591c162700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591415f390 con 0x7f591406f790 2023-08-27T11:29:21.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f591c162700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591415f670 con 0x7f5914146c20 2023-08-27T11:29:21.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f591c162700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591415f950 con 0x7f59140b3440 2023-08-27T11:29:21.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f5919efe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591406f790 0x7f5914135a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:21.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f5919efe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591406f790 0x7f5914135a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38012/0 (socket says 172.21.15.112:38012) 2023-08-27T11:29:21.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f5919efe700 1 -- 172.21.15.112:0/1028356364 learned_addr learned my addr 172.21.15.112:0/1028356364 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:21.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f591a6ff700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5914146c20 0x7f59140c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:21.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f59196fd700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59140b3440 0x7f59140c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:21.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f5919efe700 1 -- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59140b3440 msgr2=0x7f59140c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:21.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f5919efe700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59140b3440 0x7f59140c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:21.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f5919efe700 1 -- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5914146c20 msgr2=0x7f59140c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:21.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f5919efe700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5914146c20 0x7f59140c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f5919efe700 1 -- 172.21.15.112:0/1028356364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c009b30 con 0x7f591406f790 2023-08-27T11:29:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f5919efe700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591406f790 0x7f5914135a40 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f590400c7f0 tx=0x7f590400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f590a7fc700 1 -- 172.21.15.112:0/1028356364 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904011800 con 0x7f591406f790 2023-08-27T11:29:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f590a7fc700 1 -- 172.21.15.112:0/1028356364 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5904011960 con 0x7f591406f790 2023-08-27T11:29:21.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f591c162700 1 -- 172.21.15.112:0/1028356364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591415fc30 con 0x7f591406f790 2023-08-27T11:29:21.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f590a7fc700 1 -- 172.21.15.112:0/1028356364 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904011cd0 con 0x7f591406f790 2023-08-27T11:29:21.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.690+0000 7f591c162700 1 -- 172.21.15.112:0/1028356364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5914135f80 con 0x7f591406f790 2023-08-27T11:29:21.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.691+0000 7f591c162700 1 -- 172.21.15.112:0/1028356364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5914067e90 con 0x7f591406f790 2023-08-27T11:29:21.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.692+0000 7f590a7fc700 1 -- 172.21.15.112:0/1028356364 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5904010350 con 0x7f591406f790 2023-08-27T11:29:21.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.692+0000 7f590a7fc700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f58f806a4d0 0x7f58f806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:21.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.692+0000 7f590a7fc700 1 -- 172.21.15.112:0/1028356364 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f59040885d0 con 0x7f591406f790 2023-08-27T11:29:21.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.692+0000 7f59196fd700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f58f806a4d0 0x7f58f806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:21.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.693+0000 7f59196fd700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f58f806a4d0 0x7f58f806c990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f59140b5680 tx=0x7f5910009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:21.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.697+0000 7f590a7fc700 1 -- 172.21.15.112:0/1028356364 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f59040538c0 con 0x7f591406f790 2023-08-27T11:29:21.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:21.886+0000 7f591c162700 1 -- 172.21.15.112:0/1028356364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5914067a20 con 0x7f591406f790 2023-08-27T11:29:22.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.544+0000 7f590a7fc700 1 -- 172.21.15.112:0/1028356364 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v165) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5904056ee0 con 0x7f591406f790 2023-08-27T11:29:22.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.549+0000 7f591c162700 1 -- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f58f806a4d0 msgr2=0x7f58f806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:22.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.549+0000 7f591c162700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f58f806a4d0 0x7f58f806c990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f59140b5680 tx=0x7f5910009040 comp rx=0 tx=0).stop 2023-08-27T11:29:22.553 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.549+0000 7f591c162700 1 -- 172.21.15.112:0/1028356364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591406f790 msgr2=0x7f5914135a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:22.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.549+0000 7f591c162700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591406f790 0x7f5914135a40 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f590400c7f0 tx=0x7f590400cb00 comp rx=0 tx=0).stop 2023-08-27T11:29:22.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.549+0000 7f591c162700 1 -- 172.21.15.112:0/1028356364 shutdown_connections 2023-08-27T11:29:22.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.549+0000 7f591c162700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f58f806a4d0 0x7f58f806c990 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:22.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.549+0000 7f591c162700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5914146c20 0x7f59140c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:22.554 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.549+0000 7f591c162700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591406f790 0x7f5914135a40 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:22.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.549+0000 7f591c162700 1 --2- 172.21.15.112:0/1028356364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59140b3440 0x7f59140c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:22.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.549+0000 7f591c162700 1 -- 172.21.15.112:0/1028356364 >> 172.21.15.112:0/1028356364 conn(0x7f59140b1e90 msgr2=0x7f5914144210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:22.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.549+0000 7f591c162700 1 -- 172.21.15.112:0/1028356364 shutdown_connections 2023-08-27T11:29:22.555 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.551+0000 7f591c162700 1 -- 172.21.15.112:0/1028356364 wait complete. 2023-08-27T11:29:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2023-08-27T11:29:22.570 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:29:22.694 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.693+0000 7f181ccb0700 1 Processor -- start 2023-08-27T11:29:22.695 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.693+0000 7f181ccb0700 1 -- start start 2023-08-27T11:29:22.697 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.694+0000 7f181ccb0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1818144b80 0x7f18180b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:22.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.694+0000 7f181ccb0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18180cef20 con 0x7f18180b3de0 2023-08-27T11:29:22.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.694+0000 7f181ccb0700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18180b42a0 con 0x7f181806c6e0 2023-08-27T11:29:22.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.694+0000 7f181ccb0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18180b4b40 con 0x7f1818144b80 2023-08-27T11:29:22.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.694+0000 7f1815d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f18180b3de0 0x7f181806bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60634/0 (socket says 172.21.15.112:60634) 2023-08-27T11:29:22.698 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.694+0000 7f1815d9b700 1 -- 172.21.15.112:0/3705417450 learned_addr learned my addr 172.21.15.112:0/3705417450 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:22.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.694+0000 7f181659c700 1 --2- 172.21.15.112:0/3705417450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1818144b80 0x7f18180b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:22.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.695+0000 7f181559a700 1 -- 172.21.15.112:0/3705417450 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742698376 0 0) 0x7f18180cef20 con 0x7f18180b3de0 2023-08-27T11:29:22.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.695+0000 7f181559a700 1 -- 172.21.15.112:0/3705417450 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18000035b0 con 0x7f18180b3de0 2023-08-27T11:29:22.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.695+0000 7f181559a700 1 -- 172.21.15.112:0/3705417450 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862475513 0 0) 0x7f18180b42a0 con 0x7f181806c6e0 2023-08-27T11:29:22.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.695+0000 7f181559a700 1 -- 172.21.15.112:0/3705417450 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18180cef20 con 0x7f181806c6e0 2023-08-27T11:29:22.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.695+0000 7f181659c700 1 -- 172.21.15.112:0/3705417450 >> v1:172.21.15.138:6789/0 conn(0x7f181806c6e0 legacy=0x7f18180cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:22.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.695+0000 7f181659c700 1 -- 172.21.15.112:0/3705417450 >> v1:172.21.15.112:6789/0 conn(0x7f18180b3de0 legacy=0x7f181806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:22.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.695+0000 7f181659c700 1 -- 172.21.15.112:0/3705417450 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18180b4580 con 0x7f1818144b80 2023-08-27T11:29:22.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.695+0000 7f181659c700 1 --2- 172.21.15.112:0/3705417450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1818144b80 0x7f18180b6c00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f180c009870 tx=0x7f180c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f965d4acb347d2f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:22.700 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.695+0000 7f181559a700 1 -- 172.21.15.112:0/3705417450 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c011830 con 0x7f1818144b80 2023-08-27T11:29:22.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.695+0000 7f181559a700 1 -- 172.21.15.112:0/3705417450 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c011990 con 0x7f1818144b80 2023-08-27T11:29:22.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.695+0000 7f181559a700 1 -- 172.21.15.112:0/3705417450 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c011d00 con 0x7f1818144b80 2023-08-27T11:29:22.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.696+0000 7f181ccb0700 1 -- 172.21.15.112:0/3705417450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1818144b80 msgr2=0x7f18180b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:22.701 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.696+0000 7f181ccb0700 1 --2- 172.21.15.112:0/3705417450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1818144b80 0x7f18180b6c00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f180c009870 tx=0x7f180c009b80 comp rx=0 tx=0).stop 2023-08-27T11:29:22.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.696+0000 7f181ccb0700 1 -- 172.21.15.112:0/3705417450 shutdown_connections 2023-08-27T11:29:22.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.696+0000 7f181ccb0700 1 --2- 172.21.15.112:0/3705417450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1818144b80 0x7f18180b6c00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:22.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.696+0000 7f181ccb0700 1 -- 172.21.15.112:0/3705417450 >> 172.21.15.112:0/3705417450 conn(0x7f18180cdc50 msgr2=0x7f18180ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:22.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.696+0000 7f181ccb0700 1 -- 172.21.15.112:0/3705417450 shutdown_connections 2023-08-27T11:29:22.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.696+0000 7f181ccb0700 1 -- 172.21.15.112:0/3705417450 wait complete. 2023-08-27T11:29:22.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.697+0000 7f181ccb0700 1 Processor -- start 2023-08-27T11:29:22.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.697+0000 7f181ccb0700 1 -- start start 2023-08-27T11:29:22.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f181ccb0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f181806c6e0 0x7f1818143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:22.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f181ccb0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18180b3de0 0x7f18180716c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:22.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f181ccb0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1818144b80 0x7f18181f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:22.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f181ccb0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180cc220 con 0x7f181806c6e0 2023-08-27T11:29:22.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f181ccb0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180cb610 con 0x7f18180b3de0 2023-08-27T11:29:22.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f181ccb0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18180cb8f0 con 0x7f1818144b80 2023-08-27T11:29:22.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f181659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f181806c6e0 0x7f1818143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:22.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f181659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f181806c6e0 0x7f1818143b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38026/0 (socket says 172.21.15.112:38026) 2023-08-27T11:29:22.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f181659c700 1 -- 172.21.15.112:0/2799966730 learned_addr learned my addr 172.21.15.112:0/2799966730 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:22.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f181659c700 1 -- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1818144b80 msgr2=0x7f18181f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:22.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f1816d9d700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1818144b80 0x7f18181f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f181659c700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1818144b80 0x7f18181f60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f181659c700 1 -- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18180b3de0 msgr2=0x7f18180716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.698+0000 7f1815d9b700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18180b3de0 0x7f18180716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.699+0000 7f181659c700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18180b3de0 0x7f18180716c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.699+0000 7f181659c700 1 -- 172.21.15.112:0/2799966730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0094a0 con 0x7f181806c6e0 2023-08-27T11:29:22.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.699+0000 7f181659c700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f181806c6e0 0x7f1818143b70 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f180c014040 tx=0x7f180c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:22.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.699+0000 7f1806ffd700 1 -- 172.21.15.112:0/2799966730 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c0045c0 con 0x7f181806c6e0 2023-08-27T11:29:22.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.699+0000 7f1806ffd700 1 -- 172.21.15.112:0/2799966730 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c01e070 con 0x7f181806c6e0 2023-08-27T11:29:22.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.699+0000 7f181ccb0700 1 -- 172.21.15.112:0/2799966730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18180cbbd0 con 0x7f181806c6e0 2023-08-27T11:29:22.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.699+0000 7f1806ffd700 1 -- 172.21.15.112:0/2799966730 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c003900 con 0x7f181806c6e0 2023-08-27T11:29:22.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.699+0000 7f181ccb0700 1 -- 172.21.15.112:0/2799966730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1818071c00 con 0x7f181806c6e0 2023-08-27T11:29:22.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.700+0000 7f181ccb0700 1 -- 172.21.15.112:0/2799966730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1818071df0 con 0x7f181806c6e0 2023-08-27T11:29:22.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.700+0000 7f1806ffd700 1 -- 172.21.15.112:0/2799966730 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f180c003a60 con 0x7f181806c6e0 2023-08-27T11:29:22.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.701+0000 7f1806ffd700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f17f806a4d0 0x7f17f806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:22.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.701+0000 7f1806ffd700 1 -- 172.21.15.112:0/2799966730 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f180c089a50 con 0x7f181806c6e0 2023-08-27T11:29:22.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.701+0000 7f1815d9b700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f17f806a4d0 0x7f17f806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:22.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.701+0000 7f1815d9b700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f17f806a4d0 0x7f17f806c990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f181815b420 tx=0x7f1808009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:22.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.705+0000 7f1806ffd700 1 -- 172.21.15.112:0/2799966730 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f180c054d60 con 0x7f181806c6e0 2023-08-27T11:29:22.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:22.936+0000 7f181ccb0700 1 -- 172.21.15.112:0/2799966730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1818067a20 con 0x7f181806c6e0 2023-08-27T11:29:23.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-08-27T11:29:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.547+0000 7f1806ffd700 1 -- 172.21.15.112:0/2799966730 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 29 and data pool 30 v109) v1 ==== 174+0+0 (secure 0 0 0) 0x7f180c0150c0 con 0x7f181806c6e0 2023-08-27T11:29:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.554+0000 7f181ccb0700 1 -- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f17f806a4d0 msgr2=0x7f17f806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.554+0000 7f181ccb0700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f17f806a4d0 0x7f17f806c990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f181815b420 tx=0x7f1808009040 comp rx=0 tx=0).stop 2023-08-27T11:29:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.555+0000 7f181ccb0700 1 -- 172.21.15.112:0/2799966730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f181806c6e0 msgr2=0x7f1818143b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.555+0000 7f181ccb0700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f181806c6e0 0x7f1818143b70 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f180c014040 tx=0x7f180c004750 comp rx=0 tx=0).stop 2023-08-27T11:29:23.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.555+0000 7f181ccb0700 1 -- 172.21.15.112:0/2799966730 shutdown_connections 2023-08-27T11:29:23.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.555+0000 7f181ccb0700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f17f806a4d0 0x7f17f806c990 secure :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f181815b420 tx=0x7f1808009040 comp rx=0 tx=0).stop 2023-08-27T11:29:23.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.555+0000 7f181ccb0700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1818144b80 0x7f18181f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:23.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.555+0000 7f181ccb0700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f181806c6e0 0x7f1818143b70 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:23.557 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.555+0000 7f181ccb0700 1 --2- 172.21.15.112:0/2799966730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18180b3de0 0x7f18180716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:23.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.555+0000 7f181ccb0700 1 -- 172.21.15.112:0/2799966730 >> 172.21.15.112:0/2799966730 conn(0x7f18180cdc50 msgr2=0x7f181815f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:23.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.556+0000 7f181ccb0700 1 -- 172.21.15.112:0/2799966730 shutdown_connections 2023-08-27T11:29:23.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.556+0000 7f181ccb0700 1 -- 172.21.15.112:0/2799966730 wait complete. 2023-08-27T11:29:23.561 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 29 and data pool 30 2023-08-27T11:29:23.578 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:23.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.698+0000 7fa573b19700 1 Processor -- start 2023-08-27T11:29:23.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.698+0000 7fa573b19700 1 -- start start 2023-08-27T11:29:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.699+0000 7fa573b19700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa56c0cf030 0x7fa56c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.699+0000 7fa573b19700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56c13ff50 con 0x7fa56c13f990 2023-08-27T11:29:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.699+0000 7fa573b19700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56c140230 con 0x7fa56c06e150 2023-08-27T11:29:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.699+0000 7fa573b19700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c0cecb0 con 0x7fa56c0cf030 2023-08-27T11:29:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.699+0000 7fa5710b4700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa56c13f990 0x7fa56c0cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60662/0 (socket says 172.21.15.112:60662) 2023-08-27T11:29:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.699+0000 7fa5710b4700 1 -- 172.21.15.112:0/118655587 learned_addr learned my addr 172.21.15.112:0/118655587 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.699+0000 7fa5720b6700 1 --2- 172.21.15.112:0/118655587 >> v2:172.21.15.112:3300/0 conn(0x7fa56c0cf030 0x7fa56c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.699+0000 7fa5708b3700 1 -- 172.21.15.112:0/118655587 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 157798185 0 0) 0x7fa56c13ff50 con 0x7fa56c13f990 2023-08-27T11:29:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.699+0000 7fa5708b3700 1 -- 172.21.15.112:0/118655587 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5580035b0 con 0x7fa56c13f990 2023-08-27T11:29:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.699+0000 7fa5720b6700 1 -- 172.21.15.112:0/118655587 >> v1:172.21.15.138:6789/0 conn(0x7fa56c06e150 legacy=0x7fa56c06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.699+0000 7fa5720b6700 1 -- 172.21.15.112:0/118655587 >> v1:172.21.15.112:6789/0 conn(0x7fa56c13f990 legacy=0x7fa56c0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.699+0000 7fa5720b6700 1 -- 172.21.15.112:0/118655587 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c140510 con 0x7fa56c0cf030 2023-08-27T11:29:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.699+0000 7fa5720b6700 1 --2- 172.21.15.112:0/118655587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c0cf030 0x7fa56c15ad20 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fa564009d90 tx=0x7fa56400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf1f70f256126f5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.700+0000 7fa5708b3700 1 -- 172.21.15.112:0/118655587 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56400e070 con 0x7fa56c0cf030 2023-08-27T11:29:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.700+0000 7fa5708b3700 1 -- 172.21.15.112:0/118655587 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56400bb20 con 0x7fa56c0cf030 2023-08-27T11:29:23.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.700+0000 7fa5708b3700 1 -- 172.21.15.112:0/118655587 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564012430 con 0x7fa56c0cf030 2023-08-27T11:29:23.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.700+0000 7fa573b19700 1 -- 172.21.15.112:0/118655587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c0cf030 msgr2=0x7fa56c15ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:23.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.700+0000 7fa573b19700 1 --2- 172.21.15.112:0/118655587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c0cf030 0x7fa56c15ad20 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fa564009d90 tx=0x7fa56400b330 comp rx=0 tx=0).stop 2023-08-27T11:29:23.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.700+0000 7fa573b19700 1 -- 172.21.15.112:0/118655587 shutdown_connections 2023-08-27T11:29:23.705 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.700+0000 7fa573b19700 1 --2- 172.21.15.112:0/118655587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c0cf030 0x7fa56c15ad20 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:23.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.700+0000 7fa573b19700 1 -- 172.21.15.112:0/118655587 >> 172.21.15.112:0/118655587 conn(0x7fa56c06cd30 msgr2=0x7fa56c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:23.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.700+0000 7fa573b19700 1 -- 172.21.15.112:0/118655587 shutdown_connections 2023-08-27T11:29:23.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.700+0000 7fa573b19700 1 -- 172.21.15.112:0/118655587 wait complete. 2023-08-27T11:29:23.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.701+0000 7fa573b19700 1 Processor -- start 2023-08-27T11:29:23.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.701+0000 7fa573b19700 1 -- start start 2023-08-27T11:29:23.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa573b19700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c06e150 0x7fa56c0cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:23.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa573b19700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa56c0cf030 0x7fa56c0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:23.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa573b19700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa56c13f990 0x7fa56c13c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:23.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa573b19700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c0ae4b0 con 0x7fa56c06e150 2023-08-27T11:29:23.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa573b19700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c0b1730 con 0x7fa56c13f990 2023-08-27T11:29:23.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa573b19700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c0ae700 con 0x7fa56c0cf030 2023-08-27T11:29:23.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa5718b5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c06e150 0x7fa56c0cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:23.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa5718b5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c06e150 0x7fa56c0cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38052/0 (socket says 172.21.15.112:38052) 2023-08-27T11:29:23.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa5718b5700 1 -- 172.21.15.112:0/1050812236 learned_addr learned my addr 172.21.15.112:0/1050812236 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:23.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa5718b5700 1 -- 172.21.15.112:0/1050812236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa56c0cf030 msgr2=0x7fa56c0af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:23.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa5718b5700 1 --2- 172.21.15.112:0/1050812236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa56c0cf030 0x7fa56c0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:23.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa5718b5700 1 -- 172.21.15.112:0/1050812236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa56c13f990 msgr2=0x7fa56c13c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:23.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa5718b5700 1 --2- 172.21.15.112:0/1050812236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa56c13f990 0x7fa56c13c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:23.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.702+0000 7fa5718b5700 1 -- 172.21.15.112:0/1050812236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa564009a70 con 0x7fa56c06e150 2023-08-27T11:29:23.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.703+0000 7fa5718b5700 1 --2- 172.21.15.112:0/1050812236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c06e150 0x7fa56c0cc3d0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fa56800c4e0 tx=0x7fa56800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:23.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.703+0000 7fa5627fc700 1 -- 172.21.15.112:0/1050812236 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680052d0 con 0x7fa56c06e150 2023-08-27T11:29:23.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.703+0000 7fa5627fc700 1 -- 172.21.15.112:0/1050812236 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa568005430 con 0x7fa56c06e150 2023-08-27T11:29:23.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.703+0000 7fa573b19700 1 -- 172.21.15.112:0/1050812236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56c0ae9e0 con 0x7fa56c06e150 2023-08-27T11:29:23.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.703+0000 7fa5627fc700 1 -- 172.21.15.112:0/1050812236 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680038c0 con 0x7fa56c06e150 2023-08-27T11:29:23.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.703+0000 7fa573b19700 1 -- 172.21.15.112:0/1050812236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c0aefa0 con 0x7fa56c06e150 2023-08-27T11:29:23.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.704+0000 7fa573b19700 1 -- 172.21.15.112:0/1050812236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56c067a20 con 0x7fa56c06e150 2023-08-27T11:29:23.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.704+0000 7fa5627fc700 1 -- 172.21.15.112:0/1050812236 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa568003a20 con 0x7fa56c06e150 2023-08-27T11:29:23.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.705+0000 7fa5627fc700 1 --2- 172.21.15.112:0/1050812236 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa55006a4d0 0x7fa55006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:23.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.705+0000 7fa5627fc700 1 -- 172.21.15.112:0/1050812236 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa568088170 con 0x7fa56c06e150 2023-08-27T11:29:23.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.705+0000 7fa5710b4700 1 --2- 172.21.15.112:0/1050812236 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa55006a4d0 0x7fa55006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:23.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.705+0000 7fa5710b4700 1 --2- 172.21.15.112:0/1050812236 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa55006a4d0 0x7fa55006c990 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa55c001360 tx=0x7fa55c007850 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:23.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.709+0000 7fa5627fc700 1 -- 172.21.15.112:0/1050812236 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa568053280 con 0x7fa56c06e150 2023-08-27T11:29:23.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.897+0000 7fa573b19700 1 -- 172.21.15.112:0/1050812236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa56c0b2320 con 0x7fa56c06e150 2023-08-27T11:29:23.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.898+0000 7fa5627fc700 1 -- 172.21.15.112:0/1050812236 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+18272 (secure 0 0 0) 0x7fa5680568a0 con 0x7fa56c06e150 2023-08-27T11:29:23.900 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:23.900 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":166,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:22.937997+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:29:20.569324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-08-27T11:29:21.905812+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:23.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.900+0000 7fa573b19700 1 -- 172.21.15.112:0/1050812236 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa55006a4d0 msgr2=0x7fa55006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:23.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.900+0000 7fa573b19700 1 --2- 172.21.15.112:0/1050812236 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa55006a4d0 0x7fa55006c990 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa55c001360 tx=0x7fa55c007850 comp rx=0 tx=0).stop 2023-08-27T11:29:23.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.900+0000 7fa573b19700 1 -- 172.21.15.112:0/1050812236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c06e150 msgr2=0x7fa56c0cc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:23.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.900+0000 7fa573b19700 1 --2- 172.21.15.112:0/1050812236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c06e150 0x7fa56c0cc3d0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fa56800c4e0 tx=0x7fa56800c8a0 comp rx=0 tx=0).stop 2023-08-27T11:29:23.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.900+0000 7fa573b19700 1 -- 172.21.15.112:0/1050812236 shutdown_connections 2023-08-27T11:29:23.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.900+0000 7fa573b19700 1 --2- 172.21.15.112:0/1050812236 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa55006a4d0 0x7fa55006c990 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:23.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.900+0000 7fa573b19700 1 --2- 172.21.15.112:0/1050812236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa56c13f990 0x7fa56c13c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:23.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.900+0000 7fa573b19700 1 --2- 172.21.15.112:0/1050812236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c06e150 0x7fa56c0cc3d0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:23.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.900+0000 7fa573b19700 1 --2- 172.21.15.112:0/1050812236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa56c0cf030 0x7fa56c0af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:23.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.900+0000 7fa573b19700 1 -- 172.21.15.112:0/1050812236 >> 172.21.15.112:0/1050812236 conn(0x7fa56c06cd30 msgr2=0x7fa56c071830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:23.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.900+0000 7fa573b19700 1 -- 172.21.15.112:0/1050812236 shutdown_connections 2023-08-27T11:29:23.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:23.900+0000 7fa573b19700 1 -- 172.21.15.112:0/1050812236 wait complete. 2023-08-27T11:29:23.914 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:24.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.033+0000 7fa987c71700 1 Processor -- start 2023-08-27T11:29:24.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.033+0000 7fa987c71700 1 -- start start 2023-08-27T11:29:24.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa987c71700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa980151060 0x7fa98015e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:24.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa987c71700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa98015e990 0x7fa98006adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:24.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa987c71700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9800b02f0 con 0x7fa98006b2f0 2023-08-27T11:29:24.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa987c71700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98006b780 con 0x7fa98015e990 2023-08-27T11:29:24.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa987c71700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800b05d0 con 0x7fa980151060 2023-08-27T11:29:24.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa98520c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa98015e990 0x7fa98006adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:24.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa98620e700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa98006b2f0 0x7fa98006d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60678/0 (socket says 172.21.15.112:60678) 2023-08-27T11:29:24.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa98620e700 1 -- 172.21.15.112:0/3803285512 learned_addr learned my addr 172.21.15.112:0/3803285512 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:24.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa985a0d700 1 --2- 172.21.15.112:0/3803285512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa980151060 0x7fa98015e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:24.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa98520c700 1 -- 172.21.15.112:0/3803285512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa980151060 msgr2=0x7fa98015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:24.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa98520c700 1 --2- 172.21.15.112:0/3803285512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa980151060 0x7fa98015e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:24.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa98520c700 1 -- 172.21.15.112:0/3803285512 >> v1:172.21.15.112:6789/0 conn(0x7fa98006b2f0 legacy=0x7fa98006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:24.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa98520c700 1 -- 172.21.15.112:0/3803285512 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9800b08b0 con 0x7fa98015e990 2023-08-27T11:29:24.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.034+0000 7fa984a0b700 1 -- 172.21.15.112:0/3803285512 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109006463 0 0) 0x7fa9800b02f0 con 0x7fa98006b2f0 2023-08-27T11:29:24.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.035+0000 7fa98520c700 1 --2- 172.21.15.112:0/3803285512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa98015e990 0x7fa98006adb0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009840 tx=0x7fa97c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3c070114f24933 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:24.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.035+0000 7fa984a0b700 1 -- 172.21.15.112:0/3803285512 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c011830 con 0x7fa98015e990 2023-08-27T11:29:24.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.035+0000 7fa984a0b700 1 -- 172.21.15.112:0/3803285512 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c011990 con 0x7fa98015e990 2023-08-27T11:29:24.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.035+0000 7fa984a0b700 1 -- 172.21.15.112:0/3803285512 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c011d20 con 0x7fa98015e990 2023-08-27T11:29:24.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.035+0000 7fa987c71700 1 -- 172.21.15.112:0/3803285512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa98015e990 msgr2=0x7fa98006adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:24.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.035+0000 7fa987c71700 1 --2- 172.21.15.112:0/3803285512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa98015e990 0x7fa98006adb0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009840 tx=0x7fa97c009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:24.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.035+0000 7fa987c71700 1 -- 172.21.15.112:0/3803285512 shutdown_connections 2023-08-27T11:29:24.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.035+0000 7fa987c71700 1 --2- 172.21.15.112:0/3803285512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa980151060 0x7fa98015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:24.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.035+0000 7fa987c71700 1 --2- 172.21.15.112:0/3803285512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa98015e990 0x7fa98006adb0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:24.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.035+0000 7fa987c71700 1 -- 172.21.15.112:0/3803285512 >> 172.21.15.112:0/3803285512 conn(0x7fa9800d0000 msgr2=0x7fa9800d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:24.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.035+0000 7fa987c71700 1 -- 172.21.15.112:0/3803285512 shutdown_connections 2023-08-27T11:29:24.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.035+0000 7fa987c71700 1 -- 172.21.15.112:0/3803285512 wait complete. 2023-08-27T11:29:24.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.036+0000 7fa987c71700 1 Processor -- start 2023-08-27T11:29:24.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.037+0000 7fa987c71700 1 -- start start 2023-08-27T11:29:24.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.037+0000 7fa987c71700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa98006b2f0 0x7fa9801594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:24.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.037+0000 7fa987c71700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa980151060 0x7fa98014eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:24.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.037+0000 7fa987c71700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa98015e990 0x7fa98014f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:24.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.037+0000 7fa987c71700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98015ae40 con 0x7fa98015e990 2023-08-27T11:29:24.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.037+0000 7fa987c71700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98015a9b0 con 0x7fa98006b2f0 2023-08-27T11:29:24.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.037+0000 7fa987c71700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98015c970 con 0x7fa980151060 2023-08-27T11:29:24.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.037+0000 7fa98620e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa98015e990 0x7fa98014f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:24.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.037+0000 7fa98620e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa98015e990 0x7fa98014f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38070/0 (socket says 172.21.15.112:38070) 2023-08-27T11:29:24.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.037+0000 7fa98620e700 1 -- 172.21.15.112:0/345379762 learned_addr learned my addr 172.21.15.112:0/345379762 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:24.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.038+0000 7fa98620e700 1 -- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa980151060 msgr2=0x7fa98014eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:24.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.038+0000 7fa98620e700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa980151060 0x7fa98014eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:24.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.038+0000 7fa98620e700 1 -- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa98006b2f0 msgr2=0x7fa9801594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:24.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.038+0000 7fa985a0d700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa98006b2f0 0x7fa9801594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:24.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.038+0000 7fa98620e700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa98006b2f0 0x7fa9801594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:24.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.038+0000 7fa98620e700 1 -- 172.21.15.112:0/345379762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c0094a0 con 0x7fa98015e990 2023-08-27T11:29:24.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.038+0000 7fa985a0d700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa98006b2f0 0x7fa9801594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:24.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.038+0000 7fa98620e700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa98015e990 0x7fa98014f3f0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fa97800cba0 tx=0x7fa97800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:24.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.038+0000 7fa9767fc700 1 -- 172.21.15.112:0/345379762 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97800ab80 con 0x7fa98015e990 2023-08-27T11:29:24.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.038+0000 7fa9767fc700 1 -- 172.21.15.112:0/345379762 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97800ad00 con 0x7fa98015e990 2023-08-27T11:29:24.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.038+0000 7fa987c71700 1 -- 172.21.15.112:0/345379762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa980159da0 con 0x7fa98015e990 2023-08-27T11:29:24.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.038+0000 7fa9767fc700 1 -- 172.21.15.112:0/345379762 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa978007310 con 0x7fa98015e990 2023-08-27T11:29:24.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.038+0000 7fa987c71700 1 -- 172.21.15.112:0/345379762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98015a360 con 0x7fa98015e990 2023-08-27T11:29:24.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.039+0000 7fa987c71700 1 -- 172.21.15.112:0/345379762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa980067a20 con 0x7fa98015e990 2023-08-27T11:29:24.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.039+0000 7fa9767fc700 1 -- 172.21.15.112:0/345379762 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fa978007470 con 0x7fa98015e990 2023-08-27T11:29:24.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.040+0000 7fa9767fc700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa96406a4d0 0x7fa96406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:24.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.040+0000 7fa9767fc700 1 -- 172.21.15.112:0/345379762 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa978088740 con 0x7fa98015e990 2023-08-27T11:29:24.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.040+0000 7fa985a0d700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa96406a4d0 0x7fa96406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.040+0000 7fa985a0d700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa96406a4d0 0x7fa96406c990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa970009a00 tx=0x7fa970008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.044+0000 7fa9767fc700 1 -- 172.21.15.112:0/345379762 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa978053850 con 0x7fa98015e990 2023-08-27T11:29:24.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.233+0000 7fa987c71700 1 -- 172.21.15.112:0/345379762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa98015aaf0 con 0x7fa98015e990 2023-08-27T11:29:24.235 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.234+0000 7fa9767fc700 1 -- 172.21.15.112:0/345379762 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+18272 (secure 0 0 0) 0x7fa978056e70 con 0x7fa98015e990 2023-08-27T11:29:24.236 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:24.237 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":166,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:22.937997+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:29:20.569324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-08-27T11:29:21.905812+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:24.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.236+0000 7fa987c71700 1 -- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa96406a4d0 msgr2=0x7fa96406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:24.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.236+0000 7fa987c71700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa96406a4d0 0x7fa96406c990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa970009a00 tx=0x7fa970008040 comp rx=0 tx=0).stop 2023-08-27T11:29:24.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.236+0000 7fa987c71700 1 -- 172.21.15.112:0/345379762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa98015e990 msgr2=0x7fa98014f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:24.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.236+0000 7fa987c71700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa98015e990 0x7fa98014f3f0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fa97800cba0 tx=0x7fa97800cf60 comp rx=0 tx=0).stop 2023-08-27T11:29:24.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.236+0000 7fa987c71700 1 -- 172.21.15.112:0/345379762 shutdown_connections 2023-08-27T11:29:24.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.236+0000 7fa987c71700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fa96406a4d0 0x7fa96406c990 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:24.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.236+0000 7fa987c71700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa98015e990 0x7fa98014f3f0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:24.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.236+0000 7fa987c71700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa98006b2f0 0x7fa9801594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:24.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.236+0000 7fa987c71700 1 --2- 172.21.15.112:0/345379762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa980151060 0x7fa98014eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:24.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.236+0000 7fa987c71700 1 -- 172.21.15.112:0/345379762 >> 172.21.15.112:0/345379762 conn(0x7fa9800d0000 msgr2=0x7fa98006f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:24.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.236+0000 7fa987c71700 1 -- 172.21.15.112:0/345379762 shutdown_connections 2023-08-27T11:29:24.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.236+0000 7fa987c71700 1 -- 172.21.15.112:0/345379762 wait complete. 2023-08-27T11:29:24.250 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:29:24.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.373+0000 7f9a6c042700 1 Processor -- start 2023-08-27T11:29:24.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.374+0000 7f9a6c042700 1 -- start start 2023-08-27T11:29:24.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.374+0000 7f9a6c042700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9a6406e4c0 0x7f9a640acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:24.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.374+0000 7f9a6c042700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a64160fa0 con 0x7f9a640ad000 2023-08-27T11:29:24.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.374+0000 7f9a6c042700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a64146f40 con 0x7f9a64146b60 2023-08-27T11:29:24.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.374+0000 7f9a6c042700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a64161b70 con 0x7f9a6406e4c0 2023-08-27T11:29:24.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.375+0000 7f9a695dd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9a6406e4c0 0x7f9a640acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:24.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.375+0000 7f9a6a5df700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9a640ad000 0x7f9a6415d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60692/0 (socket says 172.21.15.112:60692) 2023-08-27T11:29:24.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.375+0000 7f9a6a5df700 1 -- 172.21.15.112:0/1399179094 learned_addr learned my addr 172.21.15.112:0/1399179094 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:24.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.375+0000 7f9a695dd700 1 -- 172.21.15.112:0/1399179094 >> v1:172.21.15.138:6789/0 conn(0x7f9a64146b60 legacy=0x7f9a6406ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:24.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.375+0000 7f9a695dd700 1 -- 172.21.15.112:0/1399179094 >> v1:172.21.15.112:6789/0 conn(0x7f9a640ad000 legacy=0x7f9a6415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:24.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.375+0000 7f9a695dd700 1 -- 172.21.15.112:0/1399179094 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a641611d0 con 0x7f9a6406e4c0 2023-08-27T11:29:24.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.375+0000 7f9a695dd700 1 --2- 172.21.15.112:0/1399179094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a6406e4c0 0x7f9a640acac0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f9a60009840 tx=0x7f9a60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e55f0911f17bfa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:24.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.375+0000 7f9a68ddc700 1 -- 172.21.15.112:0/1399179094 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60011830 con 0x7f9a6406e4c0 2023-08-27T11:29:24.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.375+0000 7f9a68ddc700 1 -- 172.21.15.112:0/1399179094 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a60011990 con 0x7f9a6406e4c0 2023-08-27T11:29:24.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.375+0000 7f9a68ddc700 1 -- 172.21.15.112:0/1399179094 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60011d20 con 0x7f9a6406e4c0 2023-08-27T11:29:24.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.376+0000 7f9a6c042700 1 -- 172.21.15.112:0/1399179094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a6406e4c0 msgr2=0x7f9a640acac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:24.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.376+0000 7f9a6c042700 1 --2- 172.21.15.112:0/1399179094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a6406e4c0 0x7f9a640acac0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f9a60009840 tx=0x7f9a60009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:24.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.376+0000 7f9a6c042700 1 -- 172.21.15.112:0/1399179094 shutdown_connections 2023-08-27T11:29:24.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.376+0000 7f9a6c042700 1 --2- 172.21.15.112:0/1399179094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a6406e4c0 0x7f9a640acac0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:24.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.376+0000 7f9a6c042700 1 -- 172.21.15.112:0/1399179094 >> 172.21.15.112:0/1399179094 conn(0x7f9a640cc240 msgr2=0x7f9a640cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:24.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.376+0000 7f9a6c042700 1 -- 172.21.15.112:0/1399179094 shutdown_connections 2023-08-27T11:29:24.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.376+0000 7f9a6c042700 1 -- 172.21.15.112:0/1399179094 wait complete. 2023-08-27T11:29:24.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.377+0000 7f9a6c042700 1 Processor -- start 2023-08-27T11:29:24.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.377+0000 7f9a6c042700 1 -- start start 2023-08-27T11:29:24.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a6c042700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a6406e4c0 0x7f9a641f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:24.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a6c042700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a640ad000 0x7f9a640702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:24.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a6c042700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a64146b60 0x7f9a640707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:24.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a6c042700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a64141390 con 0x7f9a6406e4c0 2023-08-27T11:29:24.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a6c042700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a641f5f10 con 0x7f9a640ad000 2023-08-27T11:29:24.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a6c042700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a64140750 con 0x7f9a64146b60 2023-08-27T11:29:24.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a69dde700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a6406e4c0 0x7f9a641f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:24.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a69dde700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a6406e4c0 0x7f9a641f59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38080/0 (socket says 172.21.15.112:38080) 2023-08-27T11:29:24.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a69dde700 1 -- 172.21.15.112:0/2917653784 learned_addr learned my addr 172.21.15.112:0/2917653784 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:24.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a695dd700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a640ad000 0x7f9a640702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:24.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a6a5df700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a64146b60 0x7f9a640707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:24.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a69dde700 1 -- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a64146b60 msgr2=0x7f9a640707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:24.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a69dde700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a64146b60 0x7f9a640707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:24.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a69dde700 1 -- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a640ad000 msgr2=0x7f9a640702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:24.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a69dde700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a640ad000 0x7f9a640702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:24.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a69dde700 1 -- 172.21.15.112:0/2917653784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a600094a0 con 0x7f9a6406e4c0 2023-08-27T11:29:24.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a69dde700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a6406e4c0 0x7f9a641f59d0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400c450 tx=0x7f9a5400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:24.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a5a7fc700 1 -- 172.21.15.112:0/2917653784 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a540052d0 con 0x7f9a6406e4c0 2023-08-27T11:29:24.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a5a7fc700 1 -- 172.21.15.112:0/2917653784 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a54005430 con 0x7f9a6406e4c0 2023-08-27T11:29:24.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.378+0000 7f9a5a7fc700 1 -- 172.21.15.112:0/2917653784 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a540038c0 con 0x7f9a6406e4c0 2023-08-27T11:29:24.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.379+0000 7f9a6c042700 1 -- 172.21.15.112:0/2917653784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a641409f0 con 0x7f9a6406e4c0 2023-08-27T11:29:24.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.379+0000 7f9a6c042700 1 -- 172.21.15.112:0/2917653784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a64140fb0 con 0x7f9a6406e4c0 2023-08-27T11:29:24.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.379+0000 7f9a6c042700 1 -- 172.21.15.112:0/2917653784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a64067e90 con 0x7f9a6406e4c0 2023-08-27T11:29:24.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.380+0000 7f9a5a7fc700 1 -- 172.21.15.112:0/2917653784 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f9a54003a20 con 0x7f9a6406e4c0 2023-08-27T11:29:24.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.380+0000 7f9a5a7fc700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9a4806a4d0 0x7f9a4806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:24.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.380+0000 7f9a5a7fc700 1 -- 172.21.15.112:0/2917653784 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9a54087e60 con 0x7f9a6406e4c0 2023-08-27T11:29:24.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.381+0000 7f9a695dd700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9a4806a4d0 0x7f9a4806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:24.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.381+0000 7f9a695dd700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9a4806a4d0 0x7f9a4806c990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9a600090b0 tx=0x7f9a60008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:24.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.385+0000 7f9a5a7fc700 1 -- 172.21.15.112:0/2917653784 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9a54052f70 con 0x7f9a6406e4c0 2023-08-27T11:29:24.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:24.620+0000 7f9a6c042700 1 -- 172.21.15.112:0/2917653784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9a64146f40 con 0x7f9a6406e4c0 2023-08-27T11:29:25.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.569+0000 7f9a5a7fc700 1 -- 172.21.15.112:0/2917653784 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v112) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9a64146f40 con 0x7f9a6406e4c0 2023-08-27T11:29:25.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.572+0000 7f9a6c042700 1 -- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9a4806a4d0 msgr2=0x7f9a4806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:25.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.572+0000 7f9a6c042700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9a4806a4d0 0x7f9a4806c990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9a600090b0 tx=0x7f9a60008030 comp rx=0 tx=0).stop 2023-08-27T11:29:25.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.572+0000 7f9a6c042700 1 -- 172.21.15.112:0/2917653784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a6406e4c0 msgr2=0x7f9a641f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:25.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.572+0000 7f9a6c042700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a6406e4c0 0x7f9a641f59d0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400c450 tx=0x7f9a5400c810 comp rx=0 tx=0).stop 2023-08-27T11:29:25.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.572+0000 7f9a6c042700 1 -- 172.21.15.112:0/2917653784 shutdown_connections 2023-08-27T11:29:25.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.572+0000 7f9a6c042700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9a4806a4d0 0x7f9a4806c990 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:25.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.572+0000 7f9a6c042700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a64146b60 0x7f9a640707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:25.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.572+0000 7f9a6c042700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a6406e4c0 0x7f9a641f59d0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:25.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.572+0000 7f9a6c042700 1 --2- 172.21.15.112:0/2917653784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a640ad000 0x7f9a640702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:25.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.572+0000 7f9a6c042700 1 -- 172.21.15.112:0/2917653784 >> 172.21.15.112:0/2917653784 conn(0x7f9a640cc240 msgr2=0x7f9a640b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:25.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.572+0000 7f9a6c042700 1 -- 172.21.15.112:0/2917653784 shutdown_connections 2023-08-27T11:29:25.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.572+0000 7f9a6c042700 1 -- 172.21.15.112:0/2917653784 wait complete. 2023-08-27T11:29:25.588 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:29:25.588 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:29:25.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.716+0000 7f27e072d700 1 Processor -- start 2023-08-27T11:29:25.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.716+0000 7f27e072d700 1 -- start start 2023-08-27T11:29:25.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27e072d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d80b3440 0x7f27d815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:25.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27e072d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d80b5070 con 0x7f27d815e6e0 2023-08-27T11:29:25.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27e072d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d8147000 con 0x7f27d8146c20 2023-08-27T11:29:25.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27e072d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d8161740 con 0x7f27d80b3440 2023-08-27T11:29:25.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27decca700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f27d815e6e0 0x7f27d806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60706/0 (socket says 172.21.15.112:60706) 2023-08-27T11:29:25.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27decca700 1 -- 172.21.15.112:0/3575765094 learned_addr learned my addr 172.21.15.112:0/3575765094 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:25.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27ddcc8700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d80b3440 0x7f27d815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:25.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178795395 0 0) 0x7f27d0001490 con 0x7f27d815e6e0 2023-08-27T11:29:25.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c40035b0 con 0x7f27d815e6e0 2023-08-27T11:29:25.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586034035 0 0) 0x7f27d8147000 con 0x7f27d8146c20 2023-08-27T11:29:25.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27d0001490 con 0x7f27d8146c20 2023-08-27T11:29:25.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4265609532 0 0) 0x7f27c40035b0 con 0x7f27d815e6e0 2023-08-27T11:29:25.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d80b3440 msgr2=0x7f27d815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:25.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27dd4c7700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d80b3440 0x7f27d815e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:25.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 >> v1:172.21.15.138:6789/0 conn(0x7f27d8146c20 legacy=0x7f27d8160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:25.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d8161a20 con 0x7f27d815e6e0 2023-08-27T11:29:25.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.717+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f27d0001b80 con 0x7f27d815e6e0 2023-08-27T11:29:25.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 >> v1:172.21.15.112:6789/0 conn(0x7f27d815e6e0 legacy=0x7f27d806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:25.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27dd4c7700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27c4003bc0 0x7f27c4003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:25.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27dd4c7700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27c4004850 0x7f27c4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:25.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27dd4c7700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4005210 0x7f27c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:25.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d8161740 con 0x7f27c4003bc0 2023-08-27T11:29:25.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c400c080 con 0x7f27c4005210 2023-08-27T11:29:25.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c400c240 con 0x7f27c4004850 2023-08-27T11:29:25.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27ddcc8700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d80b3440 0x7f27d815e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:25.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27de4c9700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27c4003bc0 0x7f27c4003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:25.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27decca700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4005210 0x7f27c400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:25.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27ddcc8700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27c4004850 0x7f27c4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:25.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27de4c9700 1 -- 172.21.15.112:0/3575765094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27c4004850 msgr2=0x7f27c4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:25.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27de4c9700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27c4004850 0x7f27c4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:25.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27de4c9700 1 -- 172.21.15.112:0/3575765094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4005210 msgr2=0x7f27c400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:25.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27de4c9700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4005210 0x7f27c400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:25.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27de4c9700 1 -- 172.21.15.112:0/3575765094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c40035b0 con 0x7f27c4003bc0 2023-08-27T11:29:25.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27decca700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4005210 0x7f27c400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:25.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27de4c9700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27c4003bc0 0x7f27c4003fc0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f27d4001930 tx=0x7f27c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afde5fcba165611 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:25.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d40052d0 con 0x7f27c4003bc0 2023-08-27T11:29:25.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d4005430 con 0x7f27c4003bc0 2023-08-27T11:29:25.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.718+0000 7f27dd4c7700 1 -- 172.21.15.112:0/3575765094 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d40038c0 con 0x7f27c4003bc0 2023-08-27T11:29:25.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.719+0000 7f27e072d700 1 -- 172.21.15.112:0/3575765094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27c4003bc0 msgr2=0x7f27c4003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:25.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.719+0000 7f27e072d700 1 --2- 172.21.15.112:0/3575765094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27c4003bc0 0x7f27c4003fc0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f27d4001930 tx=0x7f27c4005880 comp rx=0 tx=0).stop 2023-08-27T11:29:25.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.719+0000 7f27decca700 1 -- 172.21.15.112:0/3575765094 reap_dead start 2023-08-27T11:29:25.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.719+0000 7f27e072d700 1 -- 172.21.15.112:0/3575765094 shutdown_connections 2023-08-27T11:29:25.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.719+0000 7f27e072d700 1 -- 172.21.15.112:0/3575765094 >> 172.21.15.112:0/3575765094 conn(0x7f27d80b1e90 msgr2=0x7f27d80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:25.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.719+0000 7f27e072d700 1 -- 172.21.15.112:0/3575765094 shutdown_connections 2023-08-27T11:29:25.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.719+0000 7f27e072d700 1 -- 172.21.15.112:0/3575765094 wait complete. 2023-08-27T11:29:25.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.720+0000 7f27e072d700 1 Processor -- start 2023-08-27T11:29:25.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.720+0000 7f27e072d700 1 -- start start 2023-08-27T11:29:25.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.721+0000 7f27e072d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4003bc0 0x7f27d815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:25.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.721+0000 7f27e072d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d80ad480 0x7f27d815cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:25.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.721+0000 7f27e072d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d813fd30 0x7f27d81401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:25.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.721+0000 7f27e072d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d815ae30 con 0x7f27d80ad480 2023-08-27T11:29:25.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.721+0000 7f27e072d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d81406f0 con 0x7f27c4003bc0 2023-08-27T11:29:25.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.721+0000 7f27e072d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d8143e70 con 0x7f27d813fd30 2023-08-27T11:29:25.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.721+0000 7f27ddcc8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d80ad480 0x7f27d815cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:25.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.721+0000 7f27ddcc8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d80ad480 0x7f27d815cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38102/0 (socket says 172.21.15.112:38102) 2023-08-27T11:29:25.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.721+0000 7f27ddcc8700 1 -- 172.21.15.112:0/658212902 learned_addr learned my addr 172.21.15.112:0/658212902 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:25.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27de4c9700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4003bc0 0x7f27d815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:25.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27decca700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d813fd30 0x7f27d81401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:25.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27ddcc8700 1 -- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d813fd30 msgr2=0x7f27d81401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:25.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27ddcc8700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d813fd30 0x7f27d81401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:25.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27ddcc8700 1 -- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4003bc0 msgr2=0x7f27d815a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:25.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27ddcc8700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4003bc0 0x7f27d815a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:25.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27ddcc8700 1 -- 172.21.15.112:0/658212902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d80ad9d0 con 0x7f27d80ad480 2023-08-27T11:29:25.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27decca700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d813fd30 0x7f27d81401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:25.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27ddcc8700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d80ad480 0x7f27d815cf20 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f27c8011800 tx=0x7f27c8011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:25.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27ceffd700 1 -- 172.21.15.112:0/658212902 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8015870 con 0x7f27d80ad480 2023-08-27T11:29:25.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27ceffd700 1 -- 172.21.15.112:0/658212902 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c80159d0 con 0x7f27d80ad480 2023-08-27T11:29:25.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27e072d700 1 -- 172.21.15.112:0/658212902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d80adcb0 con 0x7f27d80ad480 2023-08-27T11:29:25.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27ceffd700 1 -- 172.21.15.112:0/658212902 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8015d80 con 0x7f27d80ad480 2023-08-27T11:29:25.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.722+0000 7f27e072d700 1 -- 172.21.15.112:0/658212902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d80ae270 con 0x7f27d80ad480 2023-08-27T11:29:25.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.723+0000 7f27e072d700 1 -- 172.21.15.112:0/658212902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d8067e90 con 0x7f27d80ad480 2023-08-27T11:29:25.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.723+0000 7f27ceffd700 1 -- 172.21.15.112:0/658212902 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f27c8013430 con 0x7f27d80ad480 2023-08-27T11:29:25.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.724+0000 7f27ceffd700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27bc06a4d0 0x7f27bc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:25.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.724+0000 7f27ceffd700 1 -- 172.21.15.112:0/658212902 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f27c8089540 con 0x7f27d80ad480 2023-08-27T11:29:25.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.724+0000 7f27de4c9700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27bc06a4d0 0x7f27bc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:25.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.724+0000 7f27de4c9700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27bc06a4d0 0x7f27bc06c990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f27d4015040 tx=0x7f27d4009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:25.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.728+0000 7f27ceffd700 1 -- 172.21.15.112:0/658212902 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f27c8054650 con 0x7f27d80ad480 2023-08-27T11:29:25.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:25.960+0000 7f27e072d700 1 -- 172.21.15.112:0/658212902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f27d8067a20 con 0x7f27d80ad480 2023-08-27T11:29:26.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.573+0000 7f27ceffd700 1 -- 172.21.15.112:0/658212902 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v113) v1 ==== 122+0+0 (secure 0 0 0) 0x7f27c8057c70 con 0x7f27d80ad480 2023-08-27T11:29:26.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.576+0000 7f27e072d700 1 -- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27bc06a4d0 msgr2=0x7f27bc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:26.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.576+0000 7f27e072d700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27bc06a4d0 0x7f27bc06c990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f27d4015040 tx=0x7f27d4009030 comp rx=0 tx=0).stop 2023-08-27T11:29:26.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.576+0000 7f27e072d700 1 -- 172.21.15.112:0/658212902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d80ad480 msgr2=0x7f27d815cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:26.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.576+0000 7f27e072d700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d80ad480 0x7f27d815cf20 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f27c8011800 tx=0x7f27c8011bc0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.576+0000 7f27e072d700 1 -- 172.21.15.112:0/658212902 shutdown_connections 2023-08-27T11:29:26.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.576+0000 7f27e072d700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27bc06a4d0 0x7f27bc06c990 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.576+0000 7f27e072d700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d813fd30 0x7f27d81401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.576+0000 7f27e072d700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4003bc0 0x7f27d815a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.576+0000 7f27e072d700 1 --2- 172.21.15.112:0/658212902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d80ad480 0x7f27d815cf20 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.576+0000 7f27e072d700 1 -- 172.21.15.112:0/658212902 >> 172.21.15.112:0/658212902 conn(0x7f27d80b1e90 msgr2=0x7f27d806ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:26.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.576+0000 7f27e072d700 1 -- 172.21.15.112:0/658212902 shutdown_connections 2023-08-27T11:29:26.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.576+0000 7f27e072d700 1 -- 172.21.15.112:0/658212902 wait complete. 2023-08-27T11:29:26.590 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:29:26.591 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.711+0000 7fd6dfba1700 1 Processor -- start 2023-08-27T11:29:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.711+0000 7fd6dfba1700 1 -- start start 2023-08-27T11:29:26.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.712+0000 7fd6dfba1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd6d81466c0 0x7fd6d806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:26.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.712+0000 7fd6dfba1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd6d80b4760 0x7fd6d806fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:26.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.712+0000 7fd6dfba1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d8146bb0 0x7fd6d806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:26.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.712+0000 7fd6dfba1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d80b57e0 con 0x7fd6d81466c0 2023-08-27T11:29:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.712+0000 7fd6dfba1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d8161030 con 0x7fd6d80b4760 2023-08-27T11:29:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.712+0000 7fd6dfba1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d8161310 con 0x7fd6d8146bb0 2023-08-27T11:29:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.712+0000 7fd6dd93d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd6d81466c0 0x7fd6d806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.712+0000 7fd6dd93d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd6d81466c0 0x7fd6d806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38108/0 (socket says 172.21.15.112:38108) 2023-08-27T11:29:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.712+0000 7fd6dd93d700 1 -- 172.21.15.112:0/1067299341 learned_addr learned my addr 172.21.15.112:0/1067299341 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:26.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.713+0000 7fd6dd13c700 1 --2- 172.21.15.112:0/1067299341 >> v2:172.21.15.138:3300/0 conn(0x7fd6d80b4760 0x7fd6d806fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:26.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.713+0000 7fd6dd93d700 1 -- 172.21.15.112:0/1067299341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d8146bb0 msgr2=0x7fd6d806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:26.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.713+0000 7fd6dd93d700 1 --2- 172.21.15.112:0/1067299341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d8146bb0 0x7fd6d806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.713+0000 7fd6dd93d700 1 -- 172.21.15.112:0/1067299341 >> v2:172.21.15.138:3300/0 conn(0x7fd6d80b4760 msgr2=0x7fd6d806fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:26.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.713+0000 7fd6dd93d700 1 --2- 172.21.15.112:0/1067299341 >> v2:172.21.15.138:3300/0 conn(0x7fd6d80b4760 0x7fd6d806fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.713+0000 7fd6dd93d700 1 -- 172.21.15.112:0/1067299341 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d81615f0 con 0x7fd6d81466c0 2023-08-27T11:29:26.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.713+0000 7fd6dd93d700 1 --2- 172.21.15.112:0/1067299341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d81466c0 0x7fd6d806f790 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80affe0 tx=0x7fd6c8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0c2f2721aa4b26 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:26.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.713+0000 7fd6dc93b700 1 -- 172.21.15.112:0/1067299341 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c800b870 con 0x7fd6d81466c0 2023-08-27T11:29:26.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.713+0000 7fd6dc93b700 1 -- 172.21.15.112:0/1067299341 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c800b9d0 con 0x7fd6d81466c0 2023-08-27T11:29:26.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.713+0000 7fd6dc93b700 1 -- 172.21.15.112:0/1067299341 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c800bd60 con 0x7fd6d81466c0 2023-08-27T11:29:26.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.713+0000 7fd6dfba1700 1 -- 172.21.15.112:0/1067299341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d81466c0 msgr2=0x7fd6d806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:26.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.713+0000 7fd6dfba1700 1 --2- 172.21.15.112:0/1067299341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d81466c0 0x7fd6d806f790 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80affe0 tx=0x7fd6c8009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.714+0000 7fd6dfba1700 1 -- 172.21.15.112:0/1067299341 shutdown_connections 2023-08-27T11:29:26.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.714+0000 7fd6dfba1700 1 --2- 172.21.15.112:0/1067299341 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d8146bb0 0x7fd6d806d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.714+0000 7fd6dfba1700 1 --2- 172.21.15.112:0/1067299341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d81466c0 0x7fd6d806f790 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.714+0000 7fd6dfba1700 1 --2- 172.21.15.112:0/1067299341 >> v2:172.21.15.138:3300/0 conn(0x7fd6d80b4760 0x7fd6d806fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.714+0000 7fd6dfba1700 1 -- 172.21.15.112:0/1067299341 >> 172.21.15.112:0/1067299341 conn(0x7fd6d80cff80 msgr2=0x7fd6d80d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:26.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.714+0000 7fd6dfba1700 1 -- 172.21.15.112:0/1067299341 shutdown_connections 2023-08-27T11:29:26.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.714+0000 7fd6dfba1700 1 -- 172.21.15.112:0/1067299341 wait complete. 2023-08-27T11:29:26.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.715+0000 7fd6dfba1700 1 Processor -- start 2023-08-27T11:29:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.715+0000 7fd6dfba1700 1 -- start start 2023-08-27T11:29:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.716+0000 7fd6dfba1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80b4760 0x7fd6d8159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.716+0000 7fd6dfba1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d81466c0 0x7fd6d80af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.716+0000 7fd6dfba1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d8146bb0 0x7fd6d813c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.716+0000 7fd6dfba1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d8159860 con 0x7fd6d81466c0 2023-08-27T11:29:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.716+0000 7fd6dfba1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80af7f0 con 0x7fd6d8146bb0 2023-08-27T11:29:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.716+0000 7fd6dfba1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d815b7d0 con 0x7fd6d80b4760 2023-08-27T11:29:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.716+0000 7fd6dd13c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d81466c0 0x7fd6d80af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.716+0000 7fd6dd13c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d81466c0 0x7fd6d80af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38116/0 (socket says 172.21.15.112:38116) 2023-08-27T11:29:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.716+0000 7fd6dd13c700 1 -- 172.21.15.112:0/3986539536 learned_addr learned my addr 172.21.15.112:0/3986539536 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.716+0000 7fd6dd13c700 1 -- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80b4760 msgr2=0x7fd6d8159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.716+0000 7fd6dd13c700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80b4760 0x7fd6d8159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.716+0000 7fd6dd13c700 1 -- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d8146bb0 msgr2=0x7fd6d813c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.716+0000 7fd6de13e700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d8146bb0 0x7fd6d813c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.717+0000 7fd6dd13c700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d8146bb0 0x7fd6d813c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.717+0000 7fd6dd13c700 1 -- 172.21.15.112:0/3986539536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c8009550 con 0x7fd6d81466c0 2023-08-27T11:29:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.717+0000 7fd6de13e700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d8146bb0 0x7fd6d813c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.717+0000 7fd6dd13c700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d81466c0 0x7fd6d80af2b0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400c820 tx=0x7fd6d400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.717+0000 7fd6ce7fc700 1 -- 172.21.15.112:0/3986539536 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4011800 con 0x7fd6d81466c0 2023-08-27T11:29:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.717+0000 7fd6ce7fc700 1 -- 172.21.15.112:0/3986539536 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d4011960 con 0x7fd6d81466c0 2023-08-27T11:29:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.717+0000 7fd6ce7fc700 1 -- 172.21.15.112:0/3986539536 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4011cf0 con 0x7fd6d81466c0 2023-08-27T11:29:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.717+0000 7fd6dfba1700 1 -- 172.21.15.112:0/3986539536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d815bab0 con 0x7fd6d81466c0 2023-08-27T11:29:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.717+0000 7fd6dfba1700 1 -- 172.21.15.112:0/3986539536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d815c070 con 0x7fd6d81466c0 2023-08-27T11:29:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.718+0000 7fd6dfba1700 1 -- 172.21.15.112:0/3986539536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d815c3c0 con 0x7fd6d81466c0 2023-08-27T11:29:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.718+0000 7fd6ce7fc700 1 -- 172.21.15.112:0/3986539536 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fd6d4018360 con 0x7fd6d81466c0 2023-08-27T11:29:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.719+0000 7fd6ce7fc700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6bc06a500 0x7fd6bc06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.719+0000 7fd6ce7fc700 1 -- 172.21.15.112:0/3986539536 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd6d4088c40 con 0x7fd6d81466c0 2023-08-27T11:29:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.719+0000 7fd6dd93d700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6bc06a500 0x7fd6bc06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.719+0000 7fd6dd93d700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6bc06a500 0x7fd6bc06c9c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009c10 tx=0x7fd6c8010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.723+0000 7fd6ce7fc700 1 -- 172.21.15.112:0/3986539536 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd6d4053cd0 con 0x7fd6d81466c0 2023-08-27T11:29:26.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.950+0000 7fd6dfba1700 1 -- 172.21.15.112:0/3986539536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6d8146270 con 0x7fd6d81466c0 2023-08-27T11:29:26.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.951+0000 7fd6ce7fc700 1 -- 172.21.15.112:0/3986539536 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4894 (secure 0 0 0) 0x7fd6d40572f0 con 0x7fd6d81466c0 2023-08-27T11:29:26.953 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:26.953 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":113,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/685371648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":685371648},{"type":"v1","addr":"172.21.15.138:6836","nonce":685371648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":7214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3585152138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3585152138},{"type":"v1","addr":"172.21.15.138:6837","nonce":3585152138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":8203,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2190275630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2190275630},{"type":"v1","addr":"172.21.15.112:6835","nonce":2190275630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:29:22.938324+0000","modified":"2023-08-27T11:29:26.571479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6833/4086629952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4086629952},{"type":"v1","addr":"172.21.15.112:6833","nonce":4086629952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-08-27T11:29:26.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.953+0000 7fd6dfba1700 1 -- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6bc06a500 msgr2=0x7fd6bc06c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:26.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.953+0000 7fd6dfba1700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6bc06a500 0x7fd6bc06c9c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009c10 tx=0x7fd6c8010000 comp rx=0 tx=0).stop 2023-08-27T11:29:26.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.953+0000 7fd6dfba1700 1 -- 172.21.15.112:0/3986539536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d81466c0 msgr2=0x7fd6d80af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:26.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.953+0000 7fd6dfba1700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d81466c0 0x7fd6d80af2b0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400c820 tx=0x7fd6d400cbe0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.953+0000 7fd6dfba1700 1 -- 172.21.15.112:0/3986539536 shutdown_connections 2023-08-27T11:29:26.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.953+0000 7fd6dfba1700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd6bc06a500 0x7fd6bc06c9c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.953+0000 7fd6dfba1700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d8146bb0 0x7fd6d813c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.953+0000 7fd6dfba1700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80b4760 0x7fd6d8159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.953+0000 7fd6dfba1700 1 --2- 172.21.15.112:0/3986539536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d81466c0 0x7fd6d80af2b0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:26.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.953+0000 7fd6dfba1700 1 -- 172.21.15.112:0/3986539536 >> 172.21.15.112:0/3986539536 conn(0x7fd6d80cff80 msgr2=0x7fd6d815f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:26.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.953+0000 7fd6dfba1700 1 -- 172.21.15.112:0/3986539536 shutdown_connections 2023-08-27T11:29:26.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:26.953+0000 7fd6dfba1700 1 -- 172.21.15.112:0/3986539536 wait complete. 2023-08-27T11:29:26.957 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 113 2023-08-27T11:29:26.967 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:27.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.086+0000 7f361aef0700 1 Processor -- start 2023-08-27T11:29:27.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.086+0000 7f361aef0700 1 -- start start 2023-08-27T11:29:27.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f361aef0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f361406e150 0x7f36140cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f361aef0700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f361413f990 0x7f361406e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f361aef0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36140cd270 0x7f361415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f361aef0700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36140cecb0 con 0x7f361406e150 2023-08-27T11:29:27.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f361aef0700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36140cf570 con 0x7f361413f990 2023-08-27T11:29:27.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f361aef0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361406ea70 con 0x7f36140cd270 2023-08-27T11:29:27.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f3618c8c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f361406e150 0x7f36140cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f3618c8c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f361406e150 0x7f36140cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38122/0 (socket says 172.21.15.112:38122) 2023-08-27T11:29:27.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f3618c8c700 1 -- 172.21.15.112:0/656155816 learned_addr learned my addr 172.21.15.112:0/656155816 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:27.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f3613fff700 1 --2- 172.21.15.112:0/656155816 >> v2:172.21.15.138:3300/0 conn(0x7f361413f990 0x7f361406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f361948d700 1 --2- 172.21.15.112:0/656155816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36140cd270 0x7f361415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f3618c8c700 1 -- 172.21.15.112:0/656155816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36140cd270 msgr2=0x7f361415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:27.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f3618c8c700 1 --2- 172.21.15.112:0/656155816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36140cd270 0x7f361415ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f3618c8c700 1 -- 172.21.15.112:0/656155816 >> v2:172.21.15.138:3300/0 conn(0x7f361413f990 msgr2=0x7f361406e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:27.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f3618c8c700 1 --2- 172.21.15.112:0/656155816 >> v2:172.21.15.138:3300/0 conn(0x7f361413f990 0x7f361406e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f3618c8c700 1 -- 172.21.15.112:0/656155816 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361413ff50 con 0x7f361406e150 2023-08-27T11:29:27.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.087+0000 7f3618c8c700 1 --2- 172.21.15.112:0/656155816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361406e150 0x7f36140cf030 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f361406cb80 tx=0x7f3608009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a486bef282c24f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:27.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.088+0000 7f36137fe700 1 -- 172.21.15.112:0/656155816 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800b870 con 0x7f361406e150 2023-08-27T11:29:27.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.088+0000 7f36137fe700 1 -- 172.21.15.112:0/656155816 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360800b9d0 con 0x7f361406e150 2023-08-27T11:29:27.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.088+0000 7f36137fe700 1 -- 172.21.15.112:0/656155816 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800bd60 con 0x7f361406e150 2023-08-27T11:29:27.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.088+0000 7f361aef0700 1 -- 172.21.15.112:0/656155816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361406e150 msgr2=0x7f36140cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.088+0000 7f361aef0700 1 --2- 172.21.15.112:0/656155816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361406e150 0x7f36140cf030 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f361406cb80 tx=0x7f3608009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.088+0000 7f361aef0700 1 -- 172.21.15.112:0/656155816 shutdown_connections 2023-08-27T11:29:27.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.088+0000 7f361aef0700 1 --2- 172.21.15.112:0/656155816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36140cd270 0x7f361415ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.088+0000 7f361aef0700 1 --2- 172.21.15.112:0/656155816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361406e150 0x7f36140cf030 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.088+0000 7f361aef0700 1 --2- 172.21.15.112:0/656155816 >> v2:172.21.15.138:3300/0 conn(0x7f361413f990 0x7f361406e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.088+0000 7f361aef0700 1 -- 172.21.15.112:0/656155816 >> 172.21.15.112:0/656155816 conn(0x7f361406cd30 msgr2=0x7f36140717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:27.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.088+0000 7f361aef0700 1 -- 172.21.15.112:0/656155816 shutdown_connections 2023-08-27T11:29:27.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.088+0000 7f361aef0700 1 -- 172.21.15.112:0/656155816 wait complete. 2023-08-27T11:29:27.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.089+0000 7f361aef0700 1 Processor -- start 2023-08-27T11:29:27.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.089+0000 7f361aef0700 1 -- start start 2023-08-27T11:29:27.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.090+0000 7f361aef0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f361406e150 0x7f36140c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.099 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.090+0000 7f361aef0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36140cd270 0x7f36140b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.090+0000 7f361aef0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f361413f990 0x7f36141f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.090+0000 7f361aef0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361414a3d0 con 0x7f36140cd270 2023-08-27T11:29:27.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.090+0000 7f361aef0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36140b1760 con 0x7f361406e150 2023-08-27T11:29:27.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.090+0000 7f361aef0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36140b11a0 con 0x7f361413f990 2023-08-27T11:29:27.100 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.090+0000 7f3613fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36140cd270 0x7f36140b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.090+0000 7f3613fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36140cd270 0x7f36140b06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38130/0 (socket says 172.21.15.112:38130) 2023-08-27T11:29:27.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.090+0000 7f3613fff700 1 -- 172.21.15.112:0/882055420 learned_addr learned my addr 172.21.15.112:0/882055420 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:27.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.091+0000 7f3618c8c700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f361406e150 0x7f36140c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.091+0000 7f3613fff700 1 -- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f361413f990 msgr2=0x7f36141f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.091+0000 7f3613fff700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f361413f990 0x7f36141f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.091+0000 7f3613fff700 1 -- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f361406e150 msgr2=0x7f36140c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.091+0000 7f3613fff700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f361406e150 0x7f36140c2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.091+0000 7f3613fff700 1 -- 172.21.15.112:0/882055420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3608009550 con 0x7f36140cd270 2023-08-27T11:29:27.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.091+0000 7f3618c8c700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f361406e150 0x7f36140c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:27.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.091+0000 7f3613fff700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36140cd270 0x7f36140b06e0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f360000d7d0 tx=0x7f360000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:27.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.091+0000 7f36117fa700 1 -- 172.21.15.112:0/882055420 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36000125f0 con 0x7f36140cd270 2023-08-27T11:29:27.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.091+0000 7f36117fa700 1 -- 172.21.15.112:0/882055420 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3600012750 con 0x7f36140cd270 2023-08-27T11:29:27.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.091+0000 7f361aef0700 1 -- 172.21.15.112:0/882055420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36140b1480 con 0x7f36140cd270 2023-08-27T11:29:27.103 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.091+0000 7f36117fa700 1 -- 172.21.15.112:0/882055420 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600012b00 con 0x7f36140cd270 2023-08-27T11:29:27.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.091+0000 7f361aef0700 1 -- 172.21.15.112:0/882055420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3614150280 con 0x7f36140cd270 2023-08-27T11:29:27.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.092+0000 7f361aef0700 1 -- 172.21.15.112:0/882055420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3614149e50 con 0x7f36140cd270 2023-08-27T11:29:27.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.093+0000 7f36117fa700 1 -- 172.21.15.112:0/882055420 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f3600012c60 con 0x7f36140cd270 2023-08-27T11:29:27.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.093+0000 7f36117fa700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f35f806a4d0 0x7f35f806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.093+0000 7f36117fa700 1 -- 172.21.15.112:0/882055420 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f36000894b0 con 0x7f36140cd270 2023-08-27T11:29:27.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.094+0000 7f3618c8c700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f35f806a4d0 0x7f35f806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.094+0000 7f3618c8c700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f35f806a4d0 0x7f35f806c990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3608009130 tx=0x7f3608008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:27.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.097+0000 7f36117fa700 1 -- 172.21.15.112:0/882055420 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3600054750 con 0x7f36140cd270 2023-08-27T11:29:27.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.290+0000 7f361aef0700 1 -- 172.21.15.112:0/882055420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3614149e50 con 0x7f36140cd270 2023-08-27T11:29:27.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.292+0000 7f36117fa700 1 -- 172.21.15.112:0/882055420 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+18259 (secure 0 0 0) 0x7f3614149e50 con 0x7f36140cd270 2023-08-27T11:29:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:27.294 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":167,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:24.544980+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:29:20.569324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-08-27T11:29:21.905812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:27.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.294+0000 7f361aef0700 1 -- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f35f806a4d0 msgr2=0x7f35f806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.295 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.294+0000 7f361aef0700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f35f806a4d0 0x7f35f806c990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3608009130 tx=0x7f3608008030 comp rx=0 tx=0).stop 2023-08-27T11:29:27.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.294+0000 7f361aef0700 1 -- 172.21.15.112:0/882055420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36140cd270 msgr2=0x7f36140b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.294+0000 7f361aef0700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36140cd270 0x7f36140b06e0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f360000d7d0 tx=0x7f360000db90 comp rx=0 tx=0).stop 2023-08-27T11:29:27.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.294+0000 7f361aef0700 1 -- 172.21.15.112:0/882055420 shutdown_connections 2023-08-27T11:29:27.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.294+0000 7f361aef0700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f35f806a4d0 0x7f35f806c990 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.296 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.294+0000 7f361aef0700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f361413f990 0x7f36141f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.294+0000 7f361aef0700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f361406e150 0x7f36140c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.294+0000 7f361aef0700 1 --2- 172.21.15.112:0/882055420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36140cd270 0x7f36140b06e0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.294+0000 7f361aef0700 1 -- 172.21.15.112:0/882055420 >> 172.21.15.112:0/882055420 conn(0x7f361406cd30 msgr2=0x7f36140cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.294+0000 7f361aef0700 1 -- 172.21.15.112:0/882055420 shutdown_connections 2023-08-27T11:29:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.294+0000 7f361aef0700 1 -- 172.21.15.112:0/882055420 wait complete. 2023-08-27T11:29:27.308 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:29:27.308 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:29:27.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.427+0000 7fb7afc39700 1 Processor -- start 2023-08-27T11:29:27.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.428+0000 7fb7afc39700 1 -- start start 2023-08-27T11:29:27.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.428+0000 7fb7afc39700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb7a80b1400 0x7fb7a80cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.428+0000 7fb7afc39700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb7a80b17e0 0x7fb7a806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.428+0000 7fb7afc39700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7a8161ce0 0x7fb7a806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.428+0000 7fb7afc39700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a8161850 con 0x7fb7a80b1400 2023-08-27T11:29:27.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.428+0000 7fb7afc39700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a806e7d0 con 0x7fb7a80b17e0 2023-08-27T11:29:27.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.428+0000 7fb7afc39700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80cfc10 con 0x7fb7a8161ce0 2023-08-27T11:29:27.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.428+0000 7fb7ad9d5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb7a80b1400 0x7fb7a80cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.429+0000 7fb7ad9d5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb7a80b1400 0x7fb7a80cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38136/0 (socket says 172.21.15.112:38136) 2023-08-27T11:29:27.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.429+0000 7fb7ad9d5700 1 -- 172.21.15.112:0/3808100067 learned_addr learned my addr 172.21.15.112:0/3808100067 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:27.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.429+0000 7fb7ad9d5700 1 -- 172.21.15.112:0/3808100067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7a8161ce0 msgr2=0x7fb7a806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:27.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.429+0000 7fb7ad9d5700 1 --2- 172.21.15.112:0/3808100067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7a8161ce0 0x7fb7a806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.429+0000 7fb7ad9d5700 1 -- 172.21.15.112:0/3808100067 >> v2:172.21.15.138:3300/0 conn(0x7fb7a80b17e0 msgr2=0x7fb7a806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:27.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.429+0000 7fb7ad1d4700 1 --2- 172.21.15.112:0/3808100067 >> v2:172.21.15.138:3300/0 conn(0x7fb7a80b17e0 0x7fb7a806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.429+0000 7fb7ad9d5700 1 --2- 172.21.15.112:0/3808100067 >> v2:172.21.15.138:3300/0 conn(0x7fb7a80b17e0 0x7fb7a806e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.429+0000 7fb7ad9d5700 1 -- 172.21.15.112:0/3808100067 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a8161200 con 0x7fb7a80b1400 2023-08-27T11:29:27.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.429+0000 7fb7ad9d5700 1 --2- 172.21.15.112:0/3808100067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a80b1400 0x7fb7a80cf5a0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80cc420 tx=0x7fb798009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a78eb9cc78631a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:27.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.429+0000 7fb7ac9d3700 1 -- 172.21.15.112:0/3808100067 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79800b870 con 0x7fb7a80b1400 2023-08-27T11:29:27.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.429+0000 7fb7ac9d3700 1 -- 172.21.15.112:0/3808100067 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79800b9d0 con 0x7fb7a80b1400 2023-08-27T11:29:27.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.429+0000 7fb7ac9d3700 1 -- 172.21.15.112:0/3808100067 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79800bd60 con 0x7fb7a80b1400 2023-08-27T11:29:27.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.430+0000 7fb7afc39700 1 -- 172.21.15.112:0/3808100067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a80b1400 msgr2=0x7fb7a80cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.430+0000 7fb7afc39700 1 --2- 172.21.15.112:0/3808100067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a80b1400 0x7fb7a80cf5a0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80cc420 tx=0x7fb798009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.430+0000 7fb7afc39700 1 -- 172.21.15.112:0/3808100067 shutdown_connections 2023-08-27T11:29:27.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.430+0000 7fb7afc39700 1 --2- 172.21.15.112:0/3808100067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7a8161ce0 0x7fb7a806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.430+0000 7fb7afc39700 1 --2- 172.21.15.112:0/3808100067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a80b1400 0x7fb7a80cf5a0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.430+0000 7fb7afc39700 1 --2- 172.21.15.112:0/3808100067 >> v2:172.21.15.138:3300/0 conn(0x7fb7a80b17e0 0x7fb7a806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.430+0000 7fb7afc39700 1 -- 172.21.15.112:0/3808100067 >> 172.21.15.112:0/3808100067 conn(0x7fb7a80d0000 msgr2=0x7fb7a80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:27.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.430+0000 7fb7afc39700 1 -- 172.21.15.112:0/3808100067 shutdown_connections 2023-08-27T11:29:27.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.430+0000 7fb7afc39700 1 -- 172.21.15.112:0/3808100067 wait complete. 2023-08-27T11:29:27.439 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.431+0000 7fb7afc39700 1 Processor -- start 2023-08-27T11:29:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.431+0000 7fb7afc39700 1 -- start start 2023-08-27T11:29:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7afc39700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7a80b1400 0x7fb7a8160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7afc39700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a80b17e0 0x7fb7a814cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7afc39700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a8161ce0 0x7fb7a8152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7afc39700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80c5070 con 0x7fb7a8161ce0 2023-08-27T11:29:27.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7afc39700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a81f2b80 con 0x7fb7a80b17e0 2023-08-27T11:29:27.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7afc39700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a81f6770 con 0x7fb7a80b1400 2023-08-27T11:29:27.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7ae1d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a8161ce0 0x7fb7a8152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7ae1d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a8161ce0 0x7fb7a8152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38146/0 (socket says 172.21.15.112:38146) 2023-08-27T11:29:27.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7ae1d6700 1 -- 172.21.15.112:0/3789887900 learned_addr learned my addr 172.21.15.112:0/3789887900 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:27.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7ad1d4700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a80b17e0 0x7fb7a814cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7ad9d5700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7a80b1400 0x7fb7a8160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7ae1d6700 1 -- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7a80b1400 msgr2=0x7fb7a8160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7ae1d6700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7a80b1400 0x7fb7a8160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7ae1d6700 1 -- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a80b17e0 msgr2=0x7fb7a814cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7ae1d6700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a80b17e0 0x7fb7a814cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7ae1d6700 1 -- 172.21.15.112:0/3789887900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798009550 con 0x7fb7a8161ce0 2023-08-27T11:29:27.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7ae1d6700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a8161ce0 0x7fb7a8152060 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000b970 tx=0x7fb7a000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:27.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb79e7fc700 1 -- 172.21.15.112:0/3789887900 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a000d860 con 0x7fb7a8161ce0 2023-08-27T11:29:27.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb79e7fc700 1 -- 172.21.15.112:0/3789887900 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a000d9e0 con 0x7fb7a8161ce0 2023-08-27T11:29:27.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7afc39700 1 -- 172.21.15.112:0/3789887900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a81f6a50 con 0x7fb7a8161ce0 2023-08-27T11:29:27.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb79e7fc700 1 -- 172.21.15.112:0/3789887900 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a000dd90 con 0x7fb7a8161ce0 2023-08-27T11:29:27.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.432+0000 7fb7afc39700 1 -- 172.21.15.112:0/3789887900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a81f7010 con 0x7fb7a8161ce0 2023-08-27T11:29:27.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.433+0000 7fb7afc39700 1 -- 172.21.15.112:0/3789887900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a80c51e0 con 0x7fb7a8161ce0 2023-08-27T11:29:27.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.434+0000 7fb79e7fc700 1 -- 172.21.15.112:0/3789887900 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fb7a000e430 con 0x7fb7a8161ce0 2023-08-27T11:29:27.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.434+0000 7fb79e7fc700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb78c06a4d0 0x7fb78c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.434+0000 7fb79e7fc700 1 -- 172.21.15.112:0/3789887900 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb7a0088f10 con 0x7fb7a8161ce0 2023-08-27T11:29:27.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.434+0000 7fb7ad9d5700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb78c06a4d0 0x7fb78c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.435+0000 7fb7ad9d5700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb78c06a4d0 0x7fb78c06c990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb798000c00 tx=0x7fb798008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:27.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.438+0000 7fb79e7fc700 1 -- 172.21.15.112:0/3789887900 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb7a80c51e0 con 0x7fb7a8161ce0 2023-08-27T11:29:27.518 INFO:tasks.ceph.mds.c.smithi112.stderr:2023-08-27T11:29:27.517+0000 7f89ac94b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:29:27.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.622+0000 7fb7afc39700 1 -- 172.21.15.112:0/3789887900 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7a81614e0 con 0x7fb78c06a4d0 2023-08-27T11:29:27.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.630+0000 7fb79e7fc700 1 -- 172.21.15.112:0/3789887900 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299938 (secure 0 0 0) 0x7fb79800bc70 con 0x7fb78c06a4d0 2023-08-27T11:29:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:27.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.633+0000 7fb7afc39700 1 -- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb78c06a4d0 msgr2=0x7fb78c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.633+0000 7fb7afc39700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb78c06a4d0 0x7fb78c06c990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb798000c00 tx=0x7fb798008030 comp rx=0 tx=0).stop 2023-08-27T11:29:27.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.633+0000 7fb7afc39700 1 -- 172.21.15.112:0/3789887900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a8161ce0 msgr2=0x7fb7a8152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.633+0000 7fb7afc39700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a8161ce0 0x7fb7a8152060 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000b970 tx=0x7fb7a000bc80 comp rx=0 tx=0).stop 2023-08-27T11:29:27.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.633+0000 7fb7afc39700 1 -- 172.21.15.112:0/3789887900 shutdown_connections 2023-08-27T11:29:27.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.634+0000 7fb7afc39700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb78c06a4d0 0x7fb78c06c990 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.634+0000 7fb7afc39700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a8161ce0 0x7fb7a8152060 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.634+0000 7fb7afc39700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7a80b1400 0x7fb7a8160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.634+0000 7fb7afc39700 1 --2- 172.21.15.112:0/3789887900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a80b17e0 0x7fb7a814cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.634+0000 7fb7afc39700 1 -- 172.21.15.112:0/3789887900 >> 172.21.15.112:0/3789887900 conn(0x7fb7a80d0000 msgr2=0x7fb7a806ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:27.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.634+0000 7fb7afc39700 1 -- 172.21.15.112:0/3789887900 shutdown_connections 2023-08-27T11:29:27.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.634+0000 7fb7afc39700 1 -- 172.21.15.112:0/3789887900 wait complete. 2023-08-27T11:29:27.640 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:29:27.647 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":535,"stamp":"2023-08-27T11:29:26.415957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266828,"kb_used_data":17388,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754707892,"statfs":{"total":773094113280,"available":772820881408,"internally_reserved":0,"allocated":17805312,"data_stored":9073147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.878758"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541996+0000","last_change":"2023-08-27T11:29:21.541996+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541996+0000","last_undegraded":"2023-08-27T11:29:21.541996+0000","last_fullsized":"2023-08-27T11:29:21.541996+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.538975+0000","last_change":"2023-08-27T11:29:21.538975+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.538975+0000","last_undegraded":"2023-08-27T11:29:21.538975+0000","last_fullsized":"2023-08-27T11:29:21.538975+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541380+0000","last_change":"2023-08-27T11:29:21.541380+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541380+0000","last_undegraded":"2023-08-27T11:29:21.541380+0000","last_fullsized":"2023-08-27T11:29:21.541380+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.543003+0000","last_change":"2023-08-27T11:29:21.543003+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.543003+0000","last_undegraded":"2023-08-27T11:29:21.543003+0000","last_fullsized":"2023-08-27T11:29:21.543003+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.540914+0000","last_change":"2023-08-27T11:29:21.540914+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.540914+0000","last_undegraded":"2023-08-27T11:29:21.540914+0000","last_fullsized":"2023-08-27T11:29:21.540914+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541230+0000","last_change":"2023-08-27T11:29:21.541230+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541230+0000","last_undegraded":"2023-08-27T11:29:21.541230+0000","last_fullsized":"2023-08-27T11:29:21.541230+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.542983+0000","last_change":"2023-08-27T11:29:21.542983+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.542983+0000","last_undegraded":"2023-08-27T11:29:21.542983+0000","last_fullsized":"2023-08-27T11:29:21.542983+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.540240+0000","last_change":"2023-08-27T11:29:21.540240+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.540240+0000","last_undegraded":"2023-08-27T11:29:21.540240+0000","last_fullsized":"2023-08-27T11:29:21.540240+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.545172+0000","last_change":"2023-08-27T11:29:21.545172+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.545172+0000","last_undegraded":"2023-08-27T11:29:21.545172+0000","last_fullsized":"2023-08-27T11:29:21.545172+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541880+0000","last_change":"2023-08-27T11:29:21.541880+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541880+0000","last_undegraded":"2023-08-27T11:29:21.541880+0000","last_fullsized":"2023-08-27T11:29:21.541880+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539440+0000","last_change":"2023-08-27T11:29:21.539440+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539440+0000","last_undegraded":"2023-08-27T11:29:21.539440+0000","last_fullsized":"2023-08-27T11:29:21.539440+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539983+0000","last_change":"2023-08-27T11:29:21.539983+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539983+0000","last_undegraded":"2023-08-27T11:29:21.539983+0000","last_fullsized":"2023-08-27T11:29:21.539983+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.540500+0000","last_change":"2023-08-27T11:29:21.540500+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.540500+0000","last_undegraded":"2023-08-27T11:29:21.540500+0000","last_fullsized":"2023-08-27T11:29:21.540500+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.548472+0000","last_change":"2023-08-27T11:29:21.548472+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.548472+0000","last_undegraded":"2023-08-27T11:29:21.548472+0000","last_fullsized":"2023-08-27T11:29:21.548472+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.548235+0000","last_change":"2023-08-27T11:29:21.548235+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.548235+0000","last_undegraded":"2023-08-27T11:29:21.548235+0000","last_fullsized":"2023-08-27T11:29:21.548235+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541912+0000","last_change":"2023-08-27T11:29:21.541912+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541912+0000","last_undegraded":"2023-08-27T11:29:21.541912+0000","last_fullsized":"2023-08-27T11:29:21.541912+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539729+0000","last_change":"2023-08-27T11:29:21.539729+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539729+0000","last_undegraded":"2023-08-27T11:29:21.539729+0000","last_fullsized":"2023-08-27T11:29:21.539729+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.548184+0000","last_change":"2023-08-27T11:29:21.548184+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.548184+0000","last_undegraded":"2023-08-27T11:29:21.548184+0000","last_fullsized":"2023-08-27T11:29:21.548184+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.547888+0000","last_change":"2023-08-27T11:29:21.547888+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.547888+0000","last_undegraded":"2023-08-27T11:29:21.547888+0000","last_fullsized":"2023-08-27T11:29:21.547888+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539022+0000","last_change":"2023-08-27T11:29:21.539022+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539022+0000","last_undegraded":"2023-08-27T11:29:21.539022+0000","last_fullsized":"2023-08-27T11:29:21.539022+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539979+0000","last_change":"2023-08-27T11:29:21.539979+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539979+0000","last_undegraded":"2023-08-27T11:29:21.539979+0000","last_fullsized":"2023-08-27T11:29:21.539979+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539105+0000","last_change":"2023-08-27T11:29:21.539105+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539105+0000","last_undegraded":"2023-08-27T11:29:21.539105+0000","last_fullsized":"2023-08-27T11:29:21.539105+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541342+0000","last_change":"2023-08-27T11:29:21.541342+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541342+0000","last_undegraded":"2023-08-27T11:29:21.541342+0000","last_fullsized":"2023-08-27T11:29:21.541342+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.538994+0000","last_change":"2023-08-27T11:29:21.538994+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.538994+0000","last_undegraded":"2023-08-27T11:29:21.538994+0000","last_fullsized":"2023-08-27T11:29:21.538994+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.547851+0000","last_change":"2023-08-27T11:29:21.547851+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.547851+0000","last_undegraded":"2023-08-27T11:29:21.547851+0000","last_fullsized":"2023-08-27T11:29:21.547851+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":312,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542018+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:29:21.542018+0000","last_peered":"2023-08-27T11:29:21.542018+0000","last_clean":"2023-08-27T11:29:21.542018+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:29:21.542018+0000","last_undegraded":"2023-08-27T11:29:21.542018+0000","last_fullsized":"2023-08-27T11:29:21.542018+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:29.185394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541676+0000","last_change":"2023-08-27T11:29:21.541676+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541676+0000","last_undegraded":"2023-08-27T11:29:21.541676+0000","last_fullsized":"2023-08-27T11:29:21.541676+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":312,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542017+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:29:21.542017+0000","last_peered":"2023-08-27T11:29:21.542017+0000","last_clean":"2023-08-27T11:29:21.542017+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:29:21.542017+0000","last_undegraded":"2023-08-27T11:29:21.542017+0000","last_fullsized":"2023-08-27T11:29:21.542017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:28:59.073750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.543238+0000","last_change":"2023-08-27T11:29:21.543238+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.543238+0000","last_undegraded":"2023-08-27T11:29:21.543238+0000","last_fullsized":"2023-08-27T11:29:21.543238+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":266,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.543589+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:29:21.543589+0000","last_peered":"2023-08-27T11:29:21.543589+0000","last_clean":"2023-08-27T11:29:21.543589+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:29:21.543589+0000","last_undegraded":"2023-08-27T11:29:21.543589+0000","last_fullsized":"2023-08-27T11:29:21.543589+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:51:31.227060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541675+0000","last_change":"2023-08-27T11:29:21.541675+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541675+0000","last_undegraded":"2023-08-27T11:29:21.541675+0000","last_fullsized":"2023-08-27T11:29:21.541675+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":263,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542368+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:29:21.542368+0000","last_peered":"2023-08-27T11:29:21.542368+0000","last_clean":"2023-08-27T11:29:21.542368+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:29:21.542368+0000","last_undegraded":"2023-08-27T11:29:21.542368+0000","last_fullsized":"2023-08-27T11:29:21.542368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:07:02.932605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539991+0000","last_change":"2023-08-27T11:29:21.539991+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539991+0000","last_undegraded":"2023-08-27T11:29:21.539991+0000","last_fullsized":"2023-08-27T11:29:21.539991+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":262,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542492+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:29:21.542492+0000","last_peered":"2023-08-27T11:29:21.542492+0000","last_clean":"2023-08-27T11:29:21.542492+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:29:21.542492+0000","last_undegraded":"2023-08-27T11:29:21.542492+0000","last_fullsized":"2023-08-27T11:29:21.542492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:09.589028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541913+0000","last_change":"2023-08-27T11:29:21.541913+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541913+0000","last_undegraded":"2023-08-27T11:29:21.541913+0000","last_fullsized":"2023-08-27T11:29:21.541913+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":413,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542254+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:29:21.542254+0000","last_peered":"2023-08-27T11:29:21.542254+0000","last_clean":"2023-08-27T11:29:21.542254+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:29:21.542254+0000","last_undegraded":"2023-08-27T11:29:21.542254+0000","last_fullsized":"2023-08-27T11:29:21.542254+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:16:18.080065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":272,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.548899+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:29:21.548899+0000","last_peered":"2023-08-27T11:29:21.548899+0000","last_clean":"2023-08-27T11:29:21.548899+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:29:21.548899+0000","last_undegraded":"2023-08-27T11:29:21.548899+0000","last_fullsized":"2023-08-27T11:29:21.548899+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:26.388589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.545172+0000","last_change":"2023-08-27T11:29:21.545172+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.545172+0000","last_undegraded":"2023-08-27T11:29:21.545172+0000","last_fullsized":"2023-08-27T11:29:21.545172+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":264,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.545688+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:29:21.545688+0000","last_peered":"2023-08-27T11:29:21.545688+0000","last_clean":"2023-08-27T11:29:21.545688+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:29:21.545688+0000","last_undegraded":"2023-08-27T11:29:21.545688+0000","last_fullsized":"2023-08-27T11:29:21.545688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:34:50.302463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.538971+0000","last_change":"2023-08-27T11:29:21.538971+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.538971+0000","last_undegraded":"2023-08-27T11:29:21.538971+0000","last_fullsized":"2023-08-27T11:29:21.538971+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":298,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.539590+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:29:21.539590+0000","last_peered":"2023-08-27T11:29:21.539590+0000","last_clean":"2023-08-27T11:29:21.539590+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:29:21.539590+0000","last_undegraded":"2023-08-27T11:29:21.539590+0000","last_fullsized":"2023-08-27T11:29:21.539590+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:21:23.483284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33792,"kb_used_data":2612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338048,"statfs":{"total":96636764160,"available":96602161152,"internally_reserved":0,"allocated":2674688,"data_stored":1578632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.22700000000000001},"max":{"1min":0.60399999999999998,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.44},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.51500000000000001},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.65800000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.83399999999999996,"15min":0.872},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.58799999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.42799999999999999,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.67800000000000005,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.90200000000000002},"last":0.53400000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":0.996,"15min":0.996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.53000000000000003,"5min":0.42899999999999999,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.67600000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.503},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.72199999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":1.224,"15min":1.224},"last":0.499}]},{"osd":5,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79700000000000004,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.84799999999999998},"last":0.70599999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83799999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.69299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":988264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.84599999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.39800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.89800000000000002},"last":0.54100000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.53900000000000003},"min":{"1min":0.35599999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.81799999999999995,"15min":0.873},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.49399999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.499,"15min":0.51000000000000001},"min":{"1min":0.40799999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":2.0830000000000002},"last":0.56299999999999994}]},{"osd":3,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.83199999999999996,"15min":0.84499999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.34000000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.93799999999999994},"last":0.51600000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":0.71599999999999997,"5min":0.753,"15min":0.83099999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.67600000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75700000000000001,"5min":0.85499999999999998,"15min":0.875},"last":0.68500000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79200000000000004,"5min":1.365,"15min":1.365},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.34899999999999998},"max":{"1min":0.77700000000000002,"5min":0.82999999999999996,"15min":0.91400000000000003},"last":0.65800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33780,"kb_used_data":2600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338060,"statfs":{"total":96636764160,"available":96602173440,"internally_reserved":0,"allocated":2662400,"data_stored":1572462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.312,"15min":0.51400000000000001},"min":{"1min":0.221,"5min":0.191,"15min":0.191},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":11.971},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.72799999999999998},"min":{"1min":0.24299999999999999,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":23.045999999999999},"last":0.44900000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.54200000000000004},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.221},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":11.928000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.316,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.224},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.58599999999999997},"last":0.38900000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39000000000000001,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":10.99},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.76500000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49299999999999999,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.503}]},{"osd":4,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.41499999999999998,"5min":0.40400000000000003,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.90700000000000003,"15min":1.26},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.74099999999999999},"min":{"1min":0.44900000000000001,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.81899999999999995,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.96399999999999997},"min":{"1min":0.46700000000000003,"5min":0.40999999999999998,"15min":0.36199999999999999},"max":{"1min":0.76200000000000001,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.88700000000000001,"15min":1.296},"last":0.81899999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.76700000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.371},"max":{"1min":0.77800000000000002,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60099999999999998,"15min":0.98199999999999998},"min":{"1min":0.437,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":22.952000000000002},"last":0.88}]},{"osd":7,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59599999999999997,"15min":0.97799999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.84199999999999997,"15min":22.988},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.627},"min":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.83799999999999997,"5min":0.86399999999999999,"15min":1.327},"last":0.60999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33208,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338632,"statfs":{"total":96636764160,"available":96602759168,"internally_reserved":0,"allocated":2076672,"data_stored":986903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.215,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.497,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.34799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36199999999999999,"15min":0.34399999999999997},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47699999999999998,"5min":0.628,"15min":0.63200000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34799999999999998,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45800000000000002,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.36299999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35799999999999998,"15min":0.34599999999999997},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.53700000000000003,"15min":0.59599999999999997},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.376},"min":{"1min":0.30499999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.53000000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.47399999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.34399999999999997},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.214},"last":0.44},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.499},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.603},"last":0.48699999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79600000000000004,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.878,"15min":0.878},"last":0.61399999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83399999999999996,"5min":3.79,"15min":3.79},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":988264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32100000000000001,"15min":0.318},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.19700000000000001},"max":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.629},"last":0.313}]},{"osd":1,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41699999999999998,"5min":0.50800000000000001,"15min":0.54400000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34699999999999998},"min":{"1min":0.26200000000000001,"5min":0.23000000000000001,"15min":0.22600000000000001},"max":{"1min":0.49299999999999999,"5min":0.56399999999999995,"15min":0.749},"last":0.34799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.378},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":0.51800000000000002,"5min":0.57099999999999995,"15min":0.73699999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.32600000000000001,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.55100000000000005,"15min":0.69499999999999995},"last":0.39500000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.755},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.879},"last":0.65900000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36699999999999999},"max":{"1min":0.83799999999999997,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.40000000000000002},"max":{"1min":0.78600000000000003,"5min":0.82699999999999996,"15min":0.83599999999999997},"last":0.79900000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.58499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575002,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2068480,"data_stored":982094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.29999999999999999,"15min":0.30299999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.52500000000000002},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.307,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.223,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40899999999999997,"5min":0.755,"15min":0.755},"last":0.35399999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33500000000000002,"15min":0.32300000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":0.40300000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":0.48999999999999999,"5min":2.73,"15min":2.73},"last":0.309}]},{"osd":2,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.372,"15min":0.36399999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.46400000000000002,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.375,"15min":0.37},"min":{"1min":0.27700000000000002,"5min":0.26700000000000002,"15min":0.26400000000000001},"max":{"1min":0.51000000000000001,"5min":1.179,"15min":1.179},"last":0.34200000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.28599999999999998},"max":{"1min":0.748,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.71799999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.621}]},{"osd":5,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.57699999999999996},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.373},"max":{"1min":0.84299999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.497,"5min":0.38,"15min":0.38},"max":{"1min":0.83199999999999996,"5min":0.86899999999999999,"15min":1.111},"last":0.74399999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":1.236},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70799999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":988264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68200000000000005,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.502},"min":{"1min":0.33900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55500000000000005}]},{"osd":1,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66100000000000003,"5min":0.746,"15min":0.76500000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.75,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.56999999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.40400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.96999999999999997,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63300000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71499999999999997,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.38},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":1.865,"15min":1.865},"last":0.39900000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90600000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.624,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82799999999999996,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.49199999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.90100000000000002,"5min":2.714,"15min":2.714},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":988264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.376,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61099999999999999,"5min":0.78900000000000003,"15min":0.96899999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.44700000000000001,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.621},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67500000000000004,"5min":0.68400000000000005,"15min":1.075},"last":0.47699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.373,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.0580000000000001},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":1.03},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.42199999999999999,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.77800000000000002,"15min":1.026},"last":0.58099999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.27000000000000002},"max":{"1min":0.63900000000000001,"5min":0.78700000000000003,"15min":0.86899999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.73399999999999999,"5min":0.89400000000000002,"15min":0.91400000000000003},"last":0.374}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.31},"max":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.88200000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.873,"15min":0.94199999999999995},"last":0.40200000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.34799999999999998},"max":{"1min":0.753,"5min":0.85999999999999999,"15min":0.95899999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.84399999999999997,"15min":0.92000000000000004},"last":0.45100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:29:27.653 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:29:27.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.806+0000 7f2d0f355700 1 Processor -- start 2023-08-27T11:29:27.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.806+0000 7f2d0f355700 1 -- start start 2023-08-27T11:29:27.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.807+0000 7f2d0f355700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2d0806adb0 0x7f2d0806b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.807+0000 7f2d0f355700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d080cdbd0 0x7f2d080ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.807+0000 7f2d0f355700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0815eb60 con 0x7f2d08151060 2023-08-27T11:29:27.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.807+0000 7f2d0f355700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0806b770 con 0x7f2d0806adb0 2023-08-27T11:29:27.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.807+0000 7f2d0f355700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080b02f0 con 0x7f2d080cdbd0 2023-08-27T11:29:27.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.807+0000 7f2d0d0f1700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2d08151060 0x7f2d0815e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60764/0 (socket says 172.21.15.112:60764) 2023-08-27T11:29:27.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.807+0000 7f2d0d0f1700 1 -- 172.21.15.112:0/2283177997 learned_addr learned my addr 172.21.15.112:0/2283177997 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:27.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.807+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185404638 0 0) 0x7f2d0815eb60 con 0x7f2d08151060 2023-08-27T11:29:27.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.807+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf40035b0 con 0x7f2d08151060 2023-08-27T11:29:27.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.807+0000 7f2d0d8f2700 1 --2- 172.21.15.112:0/2283177997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d080cdbd0 0x7f2d080ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.807+0000 7f2d0c8f0700 1 --2- 172.21.15.112:0/2283177997 >> v2:172.21.15.138:3300/0 conn(0x7f2d0806adb0 0x7f2d0806b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2038653529 0 0) 0x7f2cf40035b0 con 0x7f2d08151060 2023-08-27T11:29:27.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d080cdbd0 msgr2=0x7f2d080ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:27.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 --2- 172.21.15.112:0/2283177997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d080cdbd0 0x7f2d080ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 >> v2:172.21.15.138:3300/0 conn(0x7f2d0806adb0 msgr2=0x7f2d0806b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:27.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 --2- 172.21.15.112:0/2283177997 >> v2:172.21.15.138:3300/0 conn(0x7f2d0806adb0 0x7f2d0806b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080b05d0 con 0x7f2d08151060 2023-08-27T11:29:27.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f2cf8001670 con 0x7f2d08151060 2023-08-27T11:29:27.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 >> v1:172.21.15.112:6789/0 conn(0x7f2d08151060 legacy=0x7f2d0815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 --2- 172.21.15.112:0/2283177997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d080cdbd0 0x7f2cf4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 --2- 172.21.15.112:0/2283177997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf4003f80 0x7f2cf4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 --2- 172.21.15.112:0/2283177997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cf4004d90 0x7f2cf4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.815 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0806b770 con 0x7f2cf4003f80 2023-08-27T11:29:27.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080b02f0 con 0x7f2cf4004d90 2023-08-27T11:29:27.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf400ba10 con 0x7f2d080cdbd0 2023-08-27T11:29:27.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2d0c8f0700 1 --2- 172.21.15.112:0/2283177997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf4003f80 0x7f2cf4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2d0d0f1700 1 --2- 172.21.15.112:0/2283177997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d080cdbd0 0x7f2cf4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2d0d8f2700 1 --2- 172.21.15.112:0/2283177997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cf4004d90 0x7f2cf4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2d0c8f0700 1 -- 172.21.15.112:0/2283177997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d080cdbd0 msgr2=0x7f2cf4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:27.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2d0c8f0700 1 --2- 172.21.15.112:0/2283177997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d080cdbd0 0x7f2cf4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2d0c8f0700 1 -- 172.21.15.112:0/2283177997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cf4004d90 msgr2=0x7f2cf4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:27.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2d0c8f0700 1 --2- 172.21.15.112:0/2283177997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cf4004d90 0x7f2cf4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2d0c8f0700 1 -- 172.21.15.112:0/2283177997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40035b0 con 0x7f2cf4003f80 2023-08-27T11:29:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2d0c8f0700 1 --2- 172.21.15.112:0/2283177997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf4003f80 0x7f2cf4004850 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009b60 tx=0x7f2cf4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f68a84ea2627c4d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400e660 con 0x7f2cf4003f80 2023-08-27T11:29:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0400e7c0 con 0x7f2cf4003f80 2023-08-27T11:29:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.808+0000 7f2cfffff700 1 -- 172.21.15.112:0/2283177997 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400eb30 con 0x7f2cf4003f80 2023-08-27T11:29:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.809+0000 7f2d0f355700 1 -- 172.21.15.112:0/2283177997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf4003f80 msgr2=0x7f2cf4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.809+0000 7f2d0f355700 1 --2- 172.21.15.112:0/2283177997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf4003f80 0x7f2cf4004850 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009b60 tx=0x7f2cf4005880 comp rx=0 tx=0).stop 2023-08-27T11:29:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.809+0000 7f2d0d8f2700 1 -- 172.21.15.112:0/2283177997 reap_dead start 2023-08-27T11:29:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.809+0000 7f2d0f355700 1 -- 172.21.15.112:0/2283177997 shutdown_connections 2023-08-27T11:29:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.809+0000 7f2d0f355700 1 -- 172.21.15.112:0/2283177997 >> 172.21.15.112:0/2283177997 conn(0x7f2d080d0000 msgr2=0x7f2d080d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.809+0000 7f2d0f355700 1 -- 172.21.15.112:0/2283177997 shutdown_connections 2023-08-27T11:29:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.809+0000 7f2d0f355700 1 -- 172.21.15.112:0/2283177997 wait complete. 2023-08-27T11:29:27.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.810+0000 7f2d0f355700 1 Processor -- start 2023-08-27T11:29:27.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.810+0000 7f2d0f355700 1 -- start start 2023-08-27T11:29:27.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0f355700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf4003f80 0x7f2d080c3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0f355700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d080c5470 0x7f2d080c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0f355700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d080c3f30 0x7f2d0814e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0f355700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080c3640 con 0x7f2cf4003f80 2023-08-27T11:29:27.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0f355700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080c4fe0 con 0x7f2d080c3f30 2023-08-27T11:29:27.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0f355700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080c43d0 con 0x7f2d080c5470 2023-08-27T11:29:27.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0d0f1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf4003f80 0x7f2d080c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0d0f1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf4003f80 0x7f2d080c3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38162/0 (socket says 172.21.15.112:38162) 2023-08-27T11:29:27.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0d0f1700 1 -- 172.21.15.112:0/3833214591 learned_addr learned my addr 172.21.15.112:0/3833214591 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:27.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0d0f1700 1 -- 172.21.15.112:0/3833214591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d080c5470 msgr2=0x7f2d080c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0d0f1700 1 --2- 172.21.15.112:0/3833214591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d080c5470 0x7f2d080c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0d0f1700 1 -- 172.21.15.112:0/3833214591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d080c3f30 msgr2=0x7f2d0814e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0d0f1700 1 --2- 172.21.15.112:0/3833214591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d080c3f30 0x7f2d0814e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.811+0000 7f2d0d0f1700 1 -- 172.21.15.112:0/3833214591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080c46b0 con 0x7f2cf4003f80 2023-08-27T11:29:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.812+0000 7f2d0d0f1700 1 --2- 172.21.15.112:0/3833214591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf4003f80 0x7f2d080c3100 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8007310 tx=0x7f2cf8007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.812+0000 7f2cfdffb700 1 -- 172.21.15.112:0/3833214591 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800f640 con 0x7f2cf4003f80 2023-08-27T11:29:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.812+0000 7f2cfdffb700 1 -- 172.21.15.112:0/3833214591 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf800f7a0 con 0x7f2cf4003f80 2023-08-27T11:29:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.812+0000 7f2d0f355700 1 -- 172.21.15.112:0/3833214591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d080c4990 con 0x7f2cf4003f80 2023-08-27T11:29:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.812+0000 7f2cfdffb700 1 -- 172.21.15.112:0/3833214591 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800fb50 con 0x7f2cf4003f80 2023-08-27T11:29:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.812+0000 7f2d0f355700 1 -- 172.21.15.112:0/3833214591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d080c58f0 con 0x7f2cf4003f80 2023-08-27T11:29:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.813+0000 7f2d0f355700 1 -- 172.21.15.112:0/3833214591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d080c5a80 con 0x7f2cf4003f80 2023-08-27T11:29:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.813+0000 7f2cfdffb700 1 -- 172.21.15.112:0/3833214591 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f2cf800fcb0 con 0x7f2cf4003f80 2023-08-27T11:29:27.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.813+0000 7f2cfdffb700 1 --2- 172.21.15.112:0/3833214591 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2cec06a4d0 0x7f2cec06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:27.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.814+0000 7f2cfdffb700 1 -- 172.21.15.112:0/3833214591 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2cf8088ae0 con 0x7f2cf4003f80 2023-08-27T11:29:27.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.814+0000 7f2d0c8f0700 1 --2- 172.21.15.112:0/3833214591 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2cec06a4d0 0x7f2cec06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:27.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.814+0000 7f2d0c8f0700 1 --2- 172.21.15.112:0/3833214591 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2cec06a4d0 0x7f2cec06c990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2d0814f070 tx=0x7f2d04013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:27.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:27.818+0000 7f2cfdffb700 1 -- 172.21.15.112:0/3833214591 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2cf8053d50 con 0x7f2cf4003f80 2023-08-27T11:29:28.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.002+0000 7f2d0f355700 1 -- 172.21.15.112:0/3833214591 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d080b0b90 con 0x7f2cec06a4d0 2023-08-27T11:29:28.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.010+0000 7f2cfdffb700 1 -- 172.21.15.112:0/3833214591 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299938 (secure 0 0 0) 0x7f2d0400f9c0 con 0x7f2cec06a4d0 2023-08-27T11:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.013+0000 7f2d0f355700 1 -- 172.21.15.112:0/3833214591 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2cec06a4d0 msgr2=0x7f2cec06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.013+0000 7f2d0f355700 1 --2- 172.21.15.112:0/3833214591 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2cec06a4d0 0x7f2cec06c990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2d0814f070 tx=0x7f2d04013000 comp rx=0 tx=0).stop 2023-08-27T11:29:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.013+0000 7f2d0f355700 1 -- 172.21.15.112:0/3833214591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf4003f80 msgr2=0x7f2d080c3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.013+0000 7f2d0f355700 1 --2- 172.21.15.112:0/3833214591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf4003f80 0x7f2d080c3100 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8007310 tx=0x7f2cf8007620 comp rx=0 tx=0).stop 2023-08-27T11:29:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.014+0000 7f2d0f355700 1 -- 172.21.15.112:0/3833214591 shutdown_connections 2023-08-27T11:29:28.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.014+0000 7f2d0f355700 1 --2- 172.21.15.112:0/3833214591 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2cec06a4d0 0x7f2cec06c990 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.014+0000 7f2d0f355700 1 --2- 172.21.15.112:0/3833214591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d080c3f30 0x7f2d0814e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.014+0000 7f2d0f355700 1 --2- 172.21.15.112:0/3833214591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf4003f80 0x7f2d080c3100 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.014+0000 7f2d0f355700 1 --2- 172.21.15.112:0/3833214591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d080c5470 0x7f2d080c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.014+0000 7f2d0f355700 1 -- 172.21.15.112:0/3833214591 >> 172.21.15.112:0/3833214591 conn(0x7f2d080d0000 msgr2=0x7f2d0814f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:28.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.014+0000 7f2d0f355700 1 -- 172.21.15.112:0/3833214591 shutdown_connections 2023-08-27T11:29:28.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.014+0000 7f2d0f355700 1 -- 172.21.15.112:0/3833214591 wait complete. 2023-08-27T11:29:28.020 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":535,"stamp":"2023-08-27T11:29:26.415957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266828,"kb_used_data":17388,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754707892,"statfs":{"total":773094113280,"available":772820881408,"internally_reserved":0,"allocated":17805312,"data_stored":9073147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.878758"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541996+0000","last_change":"2023-08-27T11:29:21.541996+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541996+0000","last_undegraded":"2023-08-27T11:29:21.541996+0000","last_fullsized":"2023-08-27T11:29:21.541996+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.538975+0000","last_change":"2023-08-27T11:29:21.538975+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.538975+0000","last_undegraded":"2023-08-27T11:29:21.538975+0000","last_fullsized":"2023-08-27T11:29:21.538975+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541380+0000","last_change":"2023-08-27T11:29:21.541380+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541380+0000","last_undegraded":"2023-08-27T11:29:21.541380+0000","last_fullsized":"2023-08-27T11:29:21.541380+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.543003+0000","last_change":"2023-08-27T11:29:21.543003+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.543003+0000","last_undegraded":"2023-08-27T11:29:21.543003+0000","last_fullsized":"2023-08-27T11:29:21.543003+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.540914+0000","last_change":"2023-08-27T11:29:21.540914+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.540914+0000","last_undegraded":"2023-08-27T11:29:21.540914+0000","last_fullsized":"2023-08-27T11:29:21.540914+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541230+0000","last_change":"2023-08-27T11:29:21.541230+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541230+0000","last_undegraded":"2023-08-27T11:29:21.541230+0000","last_fullsized":"2023-08-27T11:29:21.541230+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.542983+0000","last_change":"2023-08-27T11:29:21.542983+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.542983+0000","last_undegraded":"2023-08-27T11:29:21.542983+0000","last_fullsized":"2023-08-27T11:29:21.542983+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.540240+0000","last_change":"2023-08-27T11:29:21.540240+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.540240+0000","last_undegraded":"2023-08-27T11:29:21.540240+0000","last_fullsized":"2023-08-27T11:29:21.540240+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.545172+0000","last_change":"2023-08-27T11:29:21.545172+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.545172+0000","last_undegraded":"2023-08-27T11:29:21.545172+0000","last_fullsized":"2023-08-27T11:29:21.545172+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541880+0000","last_change":"2023-08-27T11:29:21.541880+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541880+0000","last_undegraded":"2023-08-27T11:29:21.541880+0000","last_fullsized":"2023-08-27T11:29:21.541880+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539440+0000","last_change":"2023-08-27T11:29:21.539440+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539440+0000","last_undegraded":"2023-08-27T11:29:21.539440+0000","last_fullsized":"2023-08-27T11:29:21.539440+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539983+0000","last_change":"2023-08-27T11:29:21.539983+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539983+0000","last_undegraded":"2023-08-27T11:29:21.539983+0000","last_fullsized":"2023-08-27T11:29:21.539983+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.540500+0000","last_change":"2023-08-27T11:29:21.540500+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.540500+0000","last_undegraded":"2023-08-27T11:29:21.540500+0000","last_fullsized":"2023-08-27T11:29:21.540500+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.548472+0000","last_change":"2023-08-27T11:29:21.548472+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.548472+0000","last_undegraded":"2023-08-27T11:29:21.548472+0000","last_fullsized":"2023-08-27T11:29:21.548472+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.548235+0000","last_change":"2023-08-27T11:29:21.548235+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.548235+0000","last_undegraded":"2023-08-27T11:29:21.548235+0000","last_fullsized":"2023-08-27T11:29:21.548235+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541912+0000","last_change":"2023-08-27T11:29:21.541912+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541912+0000","last_undegraded":"2023-08-27T11:29:21.541912+0000","last_fullsized":"2023-08-27T11:29:21.541912+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539729+0000","last_change":"2023-08-27T11:29:21.539729+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539729+0000","last_undegraded":"2023-08-27T11:29:21.539729+0000","last_fullsized":"2023-08-27T11:29:21.539729+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.548184+0000","last_change":"2023-08-27T11:29:21.548184+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.548184+0000","last_undegraded":"2023-08-27T11:29:21.548184+0000","last_fullsized":"2023-08-27T11:29:21.548184+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.547888+0000","last_change":"2023-08-27T11:29:21.547888+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.547888+0000","last_undegraded":"2023-08-27T11:29:21.547888+0000","last_fullsized":"2023-08-27T11:29:21.547888+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539022+0000","last_change":"2023-08-27T11:29:21.539022+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539022+0000","last_undegraded":"2023-08-27T11:29:21.539022+0000","last_fullsized":"2023-08-27T11:29:21.539022+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539979+0000","last_change":"2023-08-27T11:29:21.539979+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539979+0000","last_undegraded":"2023-08-27T11:29:21.539979+0000","last_fullsized":"2023-08-27T11:29:21.539979+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539105+0000","last_change":"2023-08-27T11:29:21.539105+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539105+0000","last_undegraded":"2023-08-27T11:29:21.539105+0000","last_fullsized":"2023-08-27T11:29:21.539105+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541342+0000","last_change":"2023-08-27T11:29:21.541342+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541342+0000","last_undegraded":"2023-08-27T11:29:21.541342+0000","last_fullsized":"2023-08-27T11:29:21.541342+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.538994+0000","last_change":"2023-08-27T11:29:21.538994+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.538994+0000","last_undegraded":"2023-08-27T11:29:21.538994+0000","last_fullsized":"2023-08-27T11:29:21.538994+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.547851+0000","last_change":"2023-08-27T11:29:21.547851+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.547851+0000","last_undegraded":"2023-08-27T11:29:21.547851+0000","last_fullsized":"2023-08-27T11:29:21.547851+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":312,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542018+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:29:21.542018+0000","last_peered":"2023-08-27T11:29:21.542018+0000","last_clean":"2023-08-27T11:29:21.542018+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:29:21.542018+0000","last_undegraded":"2023-08-27T11:29:21.542018+0000","last_fullsized":"2023-08-27T11:29:21.542018+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:29.185394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541676+0000","last_change":"2023-08-27T11:29:21.541676+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541676+0000","last_undegraded":"2023-08-27T11:29:21.541676+0000","last_fullsized":"2023-08-27T11:29:21.541676+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":312,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542017+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:29:21.542017+0000","last_peered":"2023-08-27T11:29:21.542017+0000","last_clean":"2023-08-27T11:29:21.542017+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:29:21.542017+0000","last_undegraded":"2023-08-27T11:29:21.542017+0000","last_fullsized":"2023-08-27T11:29:21.542017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:28:59.073750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.543238+0000","last_change":"2023-08-27T11:29:21.543238+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.543238+0000","last_undegraded":"2023-08-27T11:29:21.543238+0000","last_fullsized":"2023-08-27T11:29:21.543238+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":266,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.543589+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:29:21.543589+0000","last_peered":"2023-08-27T11:29:21.543589+0000","last_clean":"2023-08-27T11:29:21.543589+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:29:21.543589+0000","last_undegraded":"2023-08-27T11:29:21.543589+0000","last_fullsized":"2023-08-27T11:29:21.543589+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:51:31.227060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541675+0000","last_change":"2023-08-27T11:29:21.541675+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541675+0000","last_undegraded":"2023-08-27T11:29:21.541675+0000","last_fullsized":"2023-08-27T11:29:21.541675+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":263,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542368+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:29:21.542368+0000","last_peered":"2023-08-27T11:29:21.542368+0000","last_clean":"2023-08-27T11:29:21.542368+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:29:21.542368+0000","last_undegraded":"2023-08-27T11:29:21.542368+0000","last_fullsized":"2023-08-27T11:29:21.542368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:07:02.932605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539991+0000","last_change":"2023-08-27T11:29:21.539991+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539991+0000","last_undegraded":"2023-08-27T11:29:21.539991+0000","last_fullsized":"2023-08-27T11:29:21.539991+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":262,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542492+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:29:21.542492+0000","last_peered":"2023-08-27T11:29:21.542492+0000","last_clean":"2023-08-27T11:29:21.542492+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:29:21.542492+0000","last_undegraded":"2023-08-27T11:29:21.542492+0000","last_fullsized":"2023-08-27T11:29:21.542492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:09.589028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541913+0000","last_change":"2023-08-27T11:29:21.541913+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541913+0000","last_undegraded":"2023-08-27T11:29:21.541913+0000","last_fullsized":"2023-08-27T11:29:21.541913+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":413,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542254+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:29:21.542254+0000","last_peered":"2023-08-27T11:29:21.542254+0000","last_clean":"2023-08-27T11:29:21.542254+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:29:21.542254+0000","last_undegraded":"2023-08-27T11:29:21.542254+0000","last_fullsized":"2023-08-27T11:29:21.542254+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:16:18.080065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":272,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.548899+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:29:21.548899+0000","last_peered":"2023-08-27T11:29:21.548899+0000","last_clean":"2023-08-27T11:29:21.548899+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:29:21.548899+0000","last_undegraded":"2023-08-27T11:29:21.548899+0000","last_fullsized":"2023-08-27T11:29:21.548899+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:26.388589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.545172+0000","last_change":"2023-08-27T11:29:21.545172+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.545172+0000","last_undegraded":"2023-08-27T11:29:21.545172+0000","last_fullsized":"2023-08-27T11:29:21.545172+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":264,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.545688+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:29:21.545688+0000","last_peered":"2023-08-27T11:29:21.545688+0000","last_clean":"2023-08-27T11:29:21.545688+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:29:21.545688+0000","last_undegraded":"2023-08-27T11:29:21.545688+0000","last_fullsized":"2023-08-27T11:29:21.545688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:34:50.302463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.538971+0000","last_change":"2023-08-27T11:29:21.538971+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.538971+0000","last_undegraded":"2023-08-27T11:29:21.538971+0000","last_fullsized":"2023-08-27T11:29:21.538971+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":298,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.539590+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:29:21.539590+0000","last_peered":"2023-08-27T11:29:21.539590+0000","last_clean":"2023-08-27T11:29:21.539590+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:29:21.539590+0000","last_undegraded":"2023-08-27T11:29:21.539590+0000","last_fullsized":"2023-08-27T11:29:21.539590+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:21:23.483284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33792,"kb_used_data":2612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338048,"statfs":{"total":96636764160,"available":96602161152,"internally_reserved":0,"allocated":2674688,"data_stored":1578632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.22700000000000001},"max":{"1min":0.60399999999999998,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.44},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.51500000000000001},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.65800000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.83399999999999996,"15min":0.872},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.58799999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.42799999999999999,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.67800000000000005,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.90200000000000002},"last":0.53400000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":0.996,"15min":0.996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.53000000000000003,"5min":0.42899999999999999,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.67600000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.503},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.72199999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":1.224,"15min":1.224},"last":0.499}]},{"osd":5,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79700000000000004,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.84799999999999998},"last":0.70599999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83799999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.69299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":988264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.84599999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.39800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.89800000000000002},"last":0.54100000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.53900000000000003},"min":{"1min":0.35599999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.81799999999999995,"15min":0.873},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.49399999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.499,"15min":0.51000000000000001},"min":{"1min":0.40799999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":2.0830000000000002},"last":0.56299999999999994}]},{"osd":3,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.83199999999999996,"15min":0.84499999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.34000000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.93799999999999994},"last":0.51600000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":0.71599999999999997,"5min":0.753,"15min":0.83099999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.67600000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75700000000000001,"5min":0.85499999999999998,"15min":0.875},"last":0.68500000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79200000000000004,"5min":1.365,"15min":1.365},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.34899999999999998},"max":{"1min":0.77700000000000002,"5min":0.82999999999999996,"15min":0.91400000000000003},"last":0.65800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33780,"kb_used_data":2600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338060,"statfs":{"total":96636764160,"available":96602173440,"internally_reserved":0,"allocated":2662400,"data_stored":1572462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.312,"15min":0.51400000000000001},"min":{"1min":0.221,"5min":0.191,"15min":0.191},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":11.971},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.72799999999999998},"min":{"1min":0.24299999999999999,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":23.045999999999999},"last":0.44900000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.54200000000000004},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.221},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":11.928000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.316,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.224},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.58599999999999997},"last":0.38900000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39000000000000001,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":10.99},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.76500000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49299999999999999,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.503}]},{"osd":4,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.41499999999999998,"5min":0.40400000000000003,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.90700000000000003,"15min":1.26},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.74099999999999999},"min":{"1min":0.44900000000000001,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.81899999999999995,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.96399999999999997},"min":{"1min":0.46700000000000003,"5min":0.40999999999999998,"15min":0.36199999999999999},"max":{"1min":0.76200000000000001,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.88700000000000001,"15min":1.296},"last":0.81899999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.76700000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.371},"max":{"1min":0.77800000000000002,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60099999999999998,"15min":0.98199999999999998},"min":{"1min":0.437,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":22.952000000000002},"last":0.88}]},{"osd":7,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59599999999999997,"15min":0.97799999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.84199999999999997,"15min":22.988},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.627},"min":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.83799999999999997,"5min":0.86399999999999999,"15min":1.327},"last":0.60999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33208,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338632,"statfs":{"total":96636764160,"available":96602759168,"internally_reserved":0,"allocated":2076672,"data_stored":986903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.215,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.497,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.34799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36199999999999999,"15min":0.34399999999999997},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47699999999999998,"5min":0.628,"15min":0.63200000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34799999999999998,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45800000000000002,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.36299999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35799999999999998,"15min":0.34599999999999997},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.53700000000000003,"15min":0.59599999999999997},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.376},"min":{"1min":0.30499999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.53000000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.47399999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.34399999999999997},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.214},"last":0.44},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.499},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.603},"last":0.48699999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79600000000000004,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.878,"15min":0.878},"last":0.61399999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83399999999999996,"5min":3.79,"15min":3.79},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":988264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32100000000000001,"15min":0.318},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.19700000000000001},"max":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.629},"last":0.313}]},{"osd":1,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41699999999999998,"5min":0.50800000000000001,"15min":0.54400000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34699999999999998},"min":{"1min":0.26200000000000001,"5min":0.23000000000000001,"15min":0.22600000000000001},"max":{"1min":0.49299999999999999,"5min":0.56399999999999995,"15min":0.749},"last":0.34799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.378},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":0.51800000000000002,"5min":0.57099999999999995,"15min":0.73699999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.32600000000000001,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.55100000000000005,"15min":0.69499999999999995},"last":0.39500000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.755},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.879},"last":0.65900000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36699999999999999},"max":{"1min":0.83799999999999997,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.40000000000000002},"max":{"1min":0.78600000000000003,"5min":0.82699999999999996,"15min":0.83599999999999997},"last":0.79900000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.58499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575002,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2068480,"data_stored":982094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.29999999999999999,"15min":0.30299999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.52500000000000002},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.307,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.223,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40899999999999997,"5min":0.755,"15min":0.755},"last":0.35399999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33500000000000002,"15min":0.32300000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":0.40300000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":0.48999999999999999,"5min":2.73,"15min":2.73},"last":0.309}]},{"osd":2,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.372,"15min":0.36399999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.46400000000000002,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.375,"15min":0.37},"min":{"1min":0.27700000000000002,"5min":0.26700000000000002,"15min":0.26400000000000001},"max":{"1min":0.51000000000000001,"5min":1.179,"15min":1.179},"last":0.34200000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.28599999999999998},"max":{"1min":0.748,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.71799999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.621}]},{"osd":5,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.57699999999999996},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.373},"max":{"1min":0.84299999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.497,"5min":0.38,"15min":0.38},"max":{"1min":0.83199999999999996,"5min":0.86899999999999999,"15min":1.111},"last":0.74399999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":1.236},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70799999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":988264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68200000000000005,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.502},"min":{"1min":0.33900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55500000000000005}]},{"osd":1,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66100000000000003,"5min":0.746,"15min":0.76500000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.75,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.56999999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.40400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.96999999999999997,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63300000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71499999999999997,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.38},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":1.865,"15min":1.865},"last":0.39900000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90600000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.624,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82799999999999996,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.49199999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.90100000000000002,"5min":2.714,"15min":2.714},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":988264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.376,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61099999999999999,"5min":0.78900000000000003,"15min":0.96899999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.44700000000000001,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.621},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67500000000000004,"5min":0.68400000000000005,"15min":1.075},"last":0.47699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.373,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.0580000000000001},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":1.03},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.42199999999999999,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.77800000000000002,"15min":1.026},"last":0.58099999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.27000000000000002},"max":{"1min":0.63900000000000001,"5min":0.78700000000000003,"15min":0.86899999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.73399999999999999,"5min":0.89400000000000002,"15min":0.91400000000000003},"last":0.374}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.31},"max":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.88200000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.873,"15min":0.94199999999999995},"last":0.40200000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.34799999999999998},"max":{"1min":0.753,"5min":0.85999999999999999,"15min":0.95899999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.84399999999999997,"15min":0.92000000000000004},"last":0.45100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:29:28.033 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.185+0000 7f3edd98f700 1 Processor -- start 2023-08-27T11:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.185+0000 7f3edd98f700 1 -- start start 2023-08-27T11:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.186+0000 7f3edd98f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3ed806e150 0x7f3ed806e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.186+0000 7f3edd98f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3ed80a8b60 0x7f3ed80a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.186+0000 7f3edd98f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed80cc200 con 0x7f3ed8143190 2023-08-27T11:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.186+0000 7f3edd98f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed806eb40 con 0x7f3ed806e150 2023-08-27T11:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.186+0000 7f3edd98f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed80cc4e0 con 0x7f3ed80a8b60 2023-08-27T11:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.186+0000 7f3ed67fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3ed806e150 0x7f3ed806e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.186+0000 7f3ed6ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3ed8143190 0x7f3ed80ceca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60784/0 (socket says 172.21.15.112:60784) 2023-08-27T11:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.186+0000 7f3ed6ffd700 1 -- 172.21.15.112:0/1420636037 learned_addr learned my addr 172.21.15.112:0/1420636037 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3ed77fe700 1 --2- 172.21.15.112:0/1420636037 >> v2:172.21.15.138:3300/0 conn(0x7f3ed80a8b60 0x7f3ed80a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3ed67fc700 1 -- 172.21.15.112:0/1420636037 >> v2:172.21.15.138:3300/0 conn(0x7f3ed80a8b60 msgr2=0x7f3ed80a8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:28.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3ed67fc700 1 --2- 172.21.15.112:0/1420636037 >> v2:172.21.15.138:3300/0 conn(0x7f3ed80a8b60 0x7f3ed80a8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3ed67fc700 1 -- 172.21.15.112:0/1420636037 >> v1:172.21.15.112:6789/0 conn(0x7f3ed8143190 legacy=0x7f3ed80ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3ed67fc700 1 -- 172.21.15.112:0/1420636037 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed80cc7c0 con 0x7f3ed806e150 2023-08-27T11:29:28.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3ed5ffb700 1 -- 172.21.15.112:0/1420636037 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116366205 0 0) 0x7f3ed80cc200 con 0x7f3ed8143190 2023-08-27T11:29:28.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3ed67fc700 1 --2- 172.21.15.112:0/1420636037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed806e150 0x7f3ed806e600 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80b2510 tx=0x7f3ec800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccde09f4574a60cf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:28.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3ed5ffb700 1 -- 172.21.15.112:0/1420636037 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8005c20 con 0x7f3ed806e150 2023-08-27T11:29:28.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3ed5ffb700 1 -- 172.21.15.112:0/1420636037 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec8005d80 con 0x7f3ed806e150 2023-08-27T11:29:28.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3ed5ffb700 1 -- 172.21.15.112:0/1420636037 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800dba0 con 0x7f3ed806e150 2023-08-27T11:29:28.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3edd98f700 1 -- 172.21.15.112:0/1420636037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed806e150 msgr2=0x7f3ed806e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3edd98f700 1 --2- 172.21.15.112:0/1420636037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed806e150 0x7f3ed806e600 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80b2510 tx=0x7f3ec800bda0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3edd98f700 1 -- 172.21.15.112:0/1420636037 shutdown_connections 2023-08-27T11:29:28.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3edd98f700 1 --2- 172.21.15.112:0/1420636037 >> v2:172.21.15.138:3300/0 conn(0x7f3ed80a8b60 0x7f3ed80a8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3edd98f700 1 --2- 172.21.15.112:0/1420636037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed806e150 0x7f3ed806e600 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.187+0000 7f3edd98f700 1 -- 172.21.15.112:0/1420636037 >> 172.21.15.112:0/1420636037 conn(0x7f3ed806cd30 msgr2=0x7f3ed80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:28.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.188+0000 7f3edd98f700 1 -- 172.21.15.112:0/1420636037 shutdown_connections 2023-08-27T11:29:28.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.188+0000 7f3edd98f700 1 -- 172.21.15.112:0/1420636037 wait complete. 2023-08-27T11:29:28.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.189+0000 7f3edd98f700 1 Processor -- start 2023-08-27T11:29:28.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.189+0000 7f3edd98f700 1 -- start start 2023-08-27T11:29:28.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3edd98f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed806e150 0x7f3ed80c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3edd98f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ed80a8b60 0x7f3ed80c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3edd98f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ed8143190 0x7f3ed80cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3edd98f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed813d200 con 0x7f3ed806e150 2023-08-27T11:29:28.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3edd98f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed80c23f0 con 0x7f3ed8143190 2023-08-27T11:29:28.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3edd98f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed813dfd0 con 0x7f3ed80a8b60 2023-08-27T11:29:28.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ed6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed806e150 0x7f3ed80c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ed6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed806e150 0x7f3ed80c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38176/0 (socket says 172.21.15.112:38176) 2023-08-27T11:29:28.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ed6ffd700 1 -- 172.21.15.112:0/1015737960 learned_addr learned my addr 172.21.15.112:0/1015737960 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:28.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ed77fe700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ed8143190 0x7f3ed80cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ed67fc700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ed80a8b60 0x7f3ed80c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ed6ffd700 1 -- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ed80a8b60 msgr2=0x7f3ed80c2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ed6ffd700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ed80a8b60 0x7f3ed80c2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ed6ffd700 1 -- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ed8143190 msgr2=0x7f3ed80cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ed6ffd700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ed8143190 0x7f3ed80cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ed6ffd700 1 -- 172.21.15.112:0/1015737960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec800b6d0 con 0x7f3ed806e150 2023-08-27T11:29:28.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ed77fe700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ed8143190 0x7f3ed80cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:28.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ed67fc700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ed80a8b60 0x7f3ed80c2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:28.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ed6ffd700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed806e150 0x7f3ed80c1eb0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00c580 tx=0x7f3ecc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:28.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ebffff700 1 -- 172.21.15.112:0/1015737960 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc004c80 con 0x7f3ed806e150 2023-08-27T11:29:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ebffff700 1 -- 172.21.15.112:0/1015737960 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc004e00 con 0x7f3ed806e150 2023-08-27T11:29:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3edd98f700 1 -- 172.21.15.112:0/1015737960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed80c3230 con 0x7f3ed806e150 2023-08-27T11:29:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3ebffff700 1 -- 172.21.15.112:0/1015737960 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc007310 con 0x7f3ed806e150 2023-08-27T11:29:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.190+0000 7f3edd98f700 1 -- 172.21.15.112:0/1015737960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed813d730 con 0x7f3ed806e150 2023-08-27T11:29:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.191+0000 7f3edd98f700 1 -- 172.21.15.112:0/1015737960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed813ad60 con 0x7f3ed806e150 2023-08-27T11:29:28.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.191+0000 7f3ebffff700 1 -- 172.21.15.112:0/1015737960 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f3ecc007470 con 0x7f3ed806e150 2023-08-27T11:29:28.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.192+0000 7f3ebffff700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3ec406a4d0 0x7f3ec406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.192+0000 7f3ebffff700 1 -- 172.21.15.112:0/1015737960 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3ecc088740 con 0x7f3ed806e150 2023-08-27T11:29:28.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.192+0000 7f3ed67fc700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3ec406a4d0 0x7f3ec406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.202 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.193+0000 7f3ed67fc700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3ec406a4d0 0x7f3ec406c990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80b0fb0 tx=0x7f3ec8014000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:28.203 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.196+0000 7f3ebffff700 1 -- 172.21.15.112:0/1015737960 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3ed813ad60 con 0x7f3ed806e150 2023-08-27T11:29:28.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.380+0000 7f3edd98f700 1 -- 172.21.15.112:0/1015737960 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ed80ccd80 con 0x7f3ec406a4d0 2023-08-27T11:29:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.389+0000 7f3ebffff700 1 -- 172.21.15.112:0/1015737960 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299938 (secure 0 0 0) 0x7f3ec8005ba0 con 0x7f3ec406a4d0 2023-08-27T11:29:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.392+0000 7f3edd98f700 1 -- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3ec406a4d0 msgr2=0x7f3ec406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.392+0000 7f3edd98f700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3ec406a4d0 0x7f3ec406c990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80b0fb0 tx=0x7f3ec8014000 comp rx=0 tx=0).stop 2023-08-27T11:29:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.392+0000 7f3edd98f700 1 -- 172.21.15.112:0/1015737960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed806e150 msgr2=0x7f3ed80c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.392+0000 7f3edd98f700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed806e150 0x7f3ed80c1eb0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00c580 tx=0x7f3ecc00c940 comp rx=0 tx=0).stop 2023-08-27T11:29:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.392+0000 7f3edd98f700 1 -- 172.21.15.112:0/1015737960 shutdown_connections 2023-08-27T11:29:28.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.392+0000 7f3edd98f700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3ec406a4d0 0x7f3ec406c990 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.392+0000 7f3edd98f700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ed8143190 0x7f3ed80cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.392+0000 7f3edd98f700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed806e150 0x7f3ed80c1eb0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.392+0000 7f3edd98f700 1 --2- 172.21.15.112:0/1015737960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ed80a8b60 0x7f3ed80c2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.392+0000 7f3edd98f700 1 -- 172.21.15.112:0/1015737960 >> 172.21.15.112:0/1015737960 conn(0x7f3ed806cd30 msgr2=0x7f3ed81415c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:28.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.393+0000 7f3edd98f700 1 -- 172.21.15.112:0/1015737960 shutdown_connections 2023-08-27T11:29:28.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.393+0000 7f3edd98f700 1 -- 172.21.15.112:0/1015737960 wait complete. 2023-08-27T11:29:28.401 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:29:28.406 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":535,"stamp":"2023-08-27T11:29:26.415957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266828,"kb_used_data":17388,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754707892,"statfs":{"total":773094113280,"available":772820881408,"internally_reserved":0,"allocated":17805312,"data_stored":9073147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.878758"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541996+0000","last_change":"2023-08-27T11:29:21.541996+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541996+0000","last_undegraded":"2023-08-27T11:29:21.541996+0000","last_fullsized":"2023-08-27T11:29:21.541996+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.538975+0000","last_change":"2023-08-27T11:29:21.538975+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.538975+0000","last_undegraded":"2023-08-27T11:29:21.538975+0000","last_fullsized":"2023-08-27T11:29:21.538975+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541380+0000","last_change":"2023-08-27T11:29:21.541380+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541380+0000","last_undegraded":"2023-08-27T11:29:21.541380+0000","last_fullsized":"2023-08-27T11:29:21.541380+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.543003+0000","last_change":"2023-08-27T11:29:21.543003+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.543003+0000","last_undegraded":"2023-08-27T11:29:21.543003+0000","last_fullsized":"2023-08-27T11:29:21.543003+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.540914+0000","last_change":"2023-08-27T11:29:21.540914+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.540914+0000","last_undegraded":"2023-08-27T11:29:21.540914+0000","last_fullsized":"2023-08-27T11:29:21.540914+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541230+0000","last_change":"2023-08-27T11:29:21.541230+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541230+0000","last_undegraded":"2023-08-27T11:29:21.541230+0000","last_fullsized":"2023-08-27T11:29:21.541230+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.542983+0000","last_change":"2023-08-27T11:29:21.542983+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.542983+0000","last_undegraded":"2023-08-27T11:29:21.542983+0000","last_fullsized":"2023-08-27T11:29:21.542983+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.540240+0000","last_change":"2023-08-27T11:29:21.540240+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.540240+0000","last_undegraded":"2023-08-27T11:29:21.540240+0000","last_fullsized":"2023-08-27T11:29:21.540240+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.545172+0000","last_change":"2023-08-27T11:29:21.545172+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.545172+0000","last_undegraded":"2023-08-27T11:29:21.545172+0000","last_fullsized":"2023-08-27T11:29:21.545172+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541880+0000","last_change":"2023-08-27T11:29:21.541880+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541880+0000","last_undegraded":"2023-08-27T11:29:21.541880+0000","last_fullsized":"2023-08-27T11:29:21.541880+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539440+0000","last_change":"2023-08-27T11:29:21.539440+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539440+0000","last_undegraded":"2023-08-27T11:29:21.539440+0000","last_fullsized":"2023-08-27T11:29:21.539440+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539983+0000","last_change":"2023-08-27T11:29:21.539983+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539983+0000","last_undegraded":"2023-08-27T11:29:21.539983+0000","last_fullsized":"2023-08-27T11:29:21.539983+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.540500+0000","last_change":"2023-08-27T11:29:21.540500+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.540500+0000","last_undegraded":"2023-08-27T11:29:21.540500+0000","last_fullsized":"2023-08-27T11:29:21.540500+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.548472+0000","last_change":"2023-08-27T11:29:21.548472+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.548472+0000","last_undegraded":"2023-08-27T11:29:21.548472+0000","last_fullsized":"2023-08-27T11:29:21.548472+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.548235+0000","last_change":"2023-08-27T11:29:21.548235+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.548235+0000","last_undegraded":"2023-08-27T11:29:21.548235+0000","last_fullsized":"2023-08-27T11:29:21.548235+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541912+0000","last_change":"2023-08-27T11:29:21.541912+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541912+0000","last_undegraded":"2023-08-27T11:29:21.541912+0000","last_fullsized":"2023-08-27T11:29:21.541912+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539729+0000","last_change":"2023-08-27T11:29:21.539729+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539729+0000","last_undegraded":"2023-08-27T11:29:21.539729+0000","last_fullsized":"2023-08-27T11:29:21.539729+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.548184+0000","last_change":"2023-08-27T11:29:21.548184+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.548184+0000","last_undegraded":"2023-08-27T11:29:21.548184+0000","last_fullsized":"2023-08-27T11:29:21.548184+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.547888+0000","last_change":"2023-08-27T11:29:21.547888+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.547888+0000","last_undegraded":"2023-08-27T11:29:21.547888+0000","last_fullsized":"2023-08-27T11:29:21.547888+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539022+0000","last_change":"2023-08-27T11:29:21.539022+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539022+0000","last_undegraded":"2023-08-27T11:29:21.539022+0000","last_fullsized":"2023-08-27T11:29:21.539022+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539979+0000","last_change":"2023-08-27T11:29:21.539979+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539979+0000","last_undegraded":"2023-08-27T11:29:21.539979+0000","last_fullsized":"2023-08-27T11:29:21.539979+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539105+0000","last_change":"2023-08-27T11:29:21.539105+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539105+0000","last_undegraded":"2023-08-27T11:29:21.539105+0000","last_fullsized":"2023-08-27T11:29:21.539105+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541342+0000","last_change":"2023-08-27T11:29:21.541342+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541342+0000","last_undegraded":"2023-08-27T11:29:21.541342+0000","last_fullsized":"2023-08-27T11:29:21.541342+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.538994+0000","last_change":"2023-08-27T11:29:21.538994+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.538994+0000","last_undegraded":"2023-08-27T11:29:21.538994+0000","last_fullsized":"2023-08-27T11:29:21.538994+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.547851+0000","last_change":"2023-08-27T11:29:21.547851+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.547851+0000","last_undegraded":"2023-08-27T11:29:21.547851+0000","last_fullsized":"2023-08-27T11:29:21.547851+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":312,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542018+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:29:21.542018+0000","last_peered":"2023-08-27T11:29:21.542018+0000","last_clean":"2023-08-27T11:29:21.542018+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:29:21.542018+0000","last_undegraded":"2023-08-27T11:29:21.542018+0000","last_fullsized":"2023-08-27T11:29:21.542018+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:29.185394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541676+0000","last_change":"2023-08-27T11:29:21.541676+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541676+0000","last_undegraded":"2023-08-27T11:29:21.541676+0000","last_fullsized":"2023-08-27T11:29:21.541676+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":312,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542017+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:29:21.542017+0000","last_peered":"2023-08-27T11:29:21.542017+0000","last_clean":"2023-08-27T11:29:21.542017+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:29:21.542017+0000","last_undegraded":"2023-08-27T11:29:21.542017+0000","last_fullsized":"2023-08-27T11:29:21.542017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:28:59.073750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.543238+0000","last_change":"2023-08-27T11:29:21.543238+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.543238+0000","last_undegraded":"2023-08-27T11:29:21.543238+0000","last_fullsized":"2023-08-27T11:29:21.543238+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":266,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.543589+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:29:21.543589+0000","last_peered":"2023-08-27T11:29:21.543589+0000","last_clean":"2023-08-27T11:29:21.543589+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:29:21.543589+0000","last_undegraded":"2023-08-27T11:29:21.543589+0000","last_fullsized":"2023-08-27T11:29:21.543589+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:51:31.227060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541675+0000","last_change":"2023-08-27T11:29:21.541675+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541675+0000","last_undegraded":"2023-08-27T11:29:21.541675+0000","last_fullsized":"2023-08-27T11:29:21.541675+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":263,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542368+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:29:21.542368+0000","last_peered":"2023-08-27T11:29:21.542368+0000","last_clean":"2023-08-27T11:29:21.542368+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:29:21.542368+0000","last_undegraded":"2023-08-27T11:29:21.542368+0000","last_fullsized":"2023-08-27T11:29:21.542368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:07:02.932605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.539991+0000","last_change":"2023-08-27T11:29:21.539991+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.539991+0000","last_undegraded":"2023-08-27T11:29:21.539991+0000","last_fullsized":"2023-08-27T11:29:21.539991+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":262,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542492+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:29:21.542492+0000","last_peered":"2023-08-27T11:29:21.542492+0000","last_clean":"2023-08-27T11:29:21.542492+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:29:21.542492+0000","last_undegraded":"2023-08-27T11:29:21.542492+0000","last_fullsized":"2023-08-27T11:29:21.542492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:09.589028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.541913+0000","last_change":"2023-08-27T11:29:21.541913+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.541913+0000","last_undegraded":"2023-08-27T11:29:21.541913+0000","last_fullsized":"2023-08-27T11:29:21.541913+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":413,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.542254+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:29:21.542254+0000","last_peered":"2023-08-27T11:29:21.542254+0000","last_clean":"2023-08-27T11:29:21.542254+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:29:21.542254+0000","last_undegraded":"2023-08-27T11:29:21.542254+0000","last_fullsized":"2023-08-27T11:29:21.542254+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:16:18.080065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":272,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.548899+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:29:21.548899+0000","last_peered":"2023-08-27T11:29:21.548899+0000","last_clean":"2023-08-27T11:29:21.548899+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:29:21.548899+0000","last_undegraded":"2023-08-27T11:29:21.548899+0000","last_fullsized":"2023-08-27T11:29:21.548899+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:26.388589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.545172+0000","last_change":"2023-08-27T11:29:21.545172+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.545172+0000","last_undegraded":"2023-08-27T11:29:21.545172+0000","last_fullsized":"2023-08-27T11:29:21.545172+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":264,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.545688+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:29:21.545688+0000","last_peered":"2023-08-27T11:29:21.545688+0000","last_clean":"2023-08-27T11:29:21.545688+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:29:21.545688+0000","last_undegraded":"2023-08-27T11:29:21.545688+0000","last_fullsized":"2023-08-27T11:29:21.545688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:34:50.302463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:29:22.536503+0000","last_change":"2023-08-27T11:29:22.536503+0000","last_active":"2023-08-27T11:29:22.536503+0000","last_peered":"2023-08-27T11:29:22.536503+0000","last_clean":"2023-08-27T11:29:22.536503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:22.536503+0000","last_undegraded":"2023-08-27T11:29:22.536503+0000","last_fullsized":"2023-08-27T11:29:22.536503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2023-08-27T11:29:21.538971+0000","last_change":"2023-08-27T11:29:21.538971+0000","last_active":"2023-08-27T11:29:21.525498+0000","last_peered":"2023-08-27T11:29:21.525498+0000","last_clean":"2023-08-27T11:29:21.525498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:29:21.538971+0000","last_undegraded":"2023-08-27T11:29:21.538971+0000","last_fullsized":"2023-08-27T11:29:21.538971+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":298,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-27T11:29:21.539590+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:29:21.539590+0000","last_peered":"2023-08-27T11:29:21.539590+0000","last_clean":"2023-08-27T11:29:21.539590+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:29:21.539590+0000","last_undegraded":"2023-08-27T11:29:21.539590+0000","last_fullsized":"2023-08-27T11:29:21.539590+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:21:23.483284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33792,"kb_used_data":2612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338048,"statfs":{"total":96636764160,"available":96602161152,"internally_reserved":0,"allocated":2674688,"data_stored":1578632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.22700000000000001},"max":{"1min":0.60399999999999998,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.44},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.51500000000000001},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.65800000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.83399999999999996,"15min":0.872},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.58799999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.42799999999999999,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.67800000000000005,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.90200000000000002},"last":0.53400000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":0.996,"15min":0.996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.53000000000000003,"5min":0.42899999999999999,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.67600000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.503},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.72199999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":1.224,"15min":1.224},"last":0.499}]},{"osd":5,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79700000000000004,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.84799999999999998},"last":0.70599999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83799999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.69299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":988264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.84599999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.39800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.89800000000000002},"last":0.54100000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.53900000000000003},"min":{"1min":0.35599999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.81799999999999995,"15min":0.873},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.49399999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.499,"15min":0.51000000000000001},"min":{"1min":0.40799999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":2.0830000000000002},"last":0.56299999999999994}]},{"osd":3,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.83199999999999996,"15min":0.84499999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.34000000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.93799999999999994},"last":0.51600000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":0.71599999999999997,"5min":0.753,"15min":0.83099999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.67600000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75700000000000001,"5min":0.85499999999999998,"15min":0.875},"last":0.68500000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79200000000000004,"5min":1.365,"15min":1.365},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.34899999999999998},"max":{"1min":0.77700000000000002,"5min":0.82999999999999996,"15min":0.91400000000000003},"last":0.65800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33780,"kb_used_data":2600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338060,"statfs":{"total":96636764160,"available":96602173440,"internally_reserved":0,"allocated":2662400,"data_stored":1572462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.312,"15min":0.51400000000000001},"min":{"1min":0.221,"5min":0.191,"15min":0.191},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":11.971},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.72799999999999998},"min":{"1min":0.24299999999999999,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":23.045999999999999},"last":0.44900000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.54200000000000004},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.221},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":11.928000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.316,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.224},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.58599999999999997},"last":0.38900000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39000000000000001,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":10.99},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.76500000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49299999999999999,"5min":0.58999999999999997,"15min":22.893999999999998},"last":0.503}]},{"osd":4,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.41499999999999998,"5min":0.40400000000000003,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.90700000000000003,"15min":1.26},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.74099999999999999},"min":{"1min":0.44900000000000001,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.81899999999999995,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.96399999999999997},"min":{"1min":0.46700000000000003,"5min":0.40999999999999998,"15min":0.36199999999999999},"max":{"1min":0.76200000000000001,"5min":0.79700000000000004,"15min":23.016999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.88700000000000001,"15min":1.296},"last":0.81899999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.76700000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.371},"max":{"1min":0.77800000000000002,"5min":0.92300000000000004,"15min":10.925000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60099999999999998,"15min":0.98199999999999998},"min":{"1min":0.437,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":22.952000000000002},"last":0.88}]},{"osd":7,"last update":"Sun Aug 27 11:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59599999999999997,"15min":0.97799999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.84199999999999997,"15min":22.988},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.627},"min":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.83799999999999997,"5min":0.86399999999999999,"15min":1.327},"last":0.60999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33208,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338632,"statfs":{"total":96636764160,"available":96602759168,"internally_reserved":0,"allocated":2076672,"data_stored":986903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.215,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.497,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.34799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36199999999999999,"15min":0.34399999999999997},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47699999999999998,"5min":0.628,"15min":0.63200000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34799999999999998,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45800000000000002,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.36299999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35799999999999998,"15min":0.34599999999999997},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.53700000000000003,"15min":0.59599999999999997},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.376},"min":{"1min":0.30499999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.53000000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.47399999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.34399999999999997},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.214},"last":0.44},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.499},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.603},"last":0.48699999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79600000000000004,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.878,"15min":0.878},"last":0.61399999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83399999999999996,"5min":3.79,"15min":3.79},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":988264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32100000000000001,"15min":0.318},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.19700000000000001},"max":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.629},"last":0.313}]},{"osd":1,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41699999999999998,"5min":0.50800000000000001,"15min":0.54400000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34699999999999998},"min":{"1min":0.26200000000000001,"5min":0.23000000000000001,"15min":0.22600000000000001},"max":{"1min":0.49299999999999999,"5min":0.56399999999999995,"15min":0.749},"last":0.34799999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.378},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":0.51800000000000002,"5min":0.57099999999999995,"15min":0.73699999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.32600000000000001,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.55100000000000005,"15min":0.69499999999999995},"last":0.39500000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.755},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.879},"last":0.65900000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36699999999999999},"max":{"1min":0.83799999999999997,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.40000000000000002},"max":{"1min":0.78600000000000003,"5min":0.82699999999999996,"15min":0.83599999999999997},"last":0.79900000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.58499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575002,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2068480,"data_stored":982094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.29999999999999999,"15min":0.30299999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.52500000000000002},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.307,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.223,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40899999999999997,"5min":0.755,"15min":0.755},"last":0.35399999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33500000000000002,"15min":0.32300000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":0.40300000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":0.48999999999999999,"5min":2.73,"15min":2.73},"last":0.309}]},{"osd":2,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.372,"15min":0.36399999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.46400000000000002,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.375,"15min":0.37},"min":{"1min":0.27700000000000002,"5min":0.26700000000000002,"15min":0.26400000000000001},"max":{"1min":0.51000000000000001,"5min":1.179,"15min":1.179},"last":0.34200000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.28599999999999998},"max":{"1min":0.748,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.71799999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.621}]},{"osd":5,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.57699999999999996},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.373},"max":{"1min":0.84299999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.497,"5min":0.38,"15min":0.38},"max":{"1min":0.83199999999999996,"5min":0.86899999999999999,"15min":1.111},"last":0.74399999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":1.236},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70799999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":988264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68200000000000005,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.502},"min":{"1min":0.33900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55500000000000005}]},{"osd":1,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66100000000000003,"5min":0.746,"15min":0.76500000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.75,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.56999999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.40400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.96999999999999997,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63300000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71499999999999997,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.38},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":1.865,"15min":1.865},"last":0.39900000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90600000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.624,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82799999999999996,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.49199999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.90100000000000002,"5min":2.714,"15min":2.714},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":988264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.376,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61099999999999999,"5min":0.78900000000000003,"15min":0.96899999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.44700000000000001,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.621},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67500000000000004,"5min":0.68400000000000005,"15min":1.075},"last":0.47699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.373,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.0580000000000001},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":1.03},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.42199999999999999,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.77800000000000002,"15min":1.026},"last":0.58099999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.27000000000000002},"max":{"1min":0.63900000000000001,"5min":0.78700000000000003,"15min":0.86899999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.73399999999999999,"5min":0.89400000000000002,"15min":0.91400000000000003},"last":0.374}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.31},"max":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.88200000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.873,"15min":0.94199999999999995},"last":0.40200000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.34799999999999998},"max":{"1min":0.753,"5min":0.85999999999999999,"15min":0.95899999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.84399999999999997,"15min":0.92000000000000004},"last":0.45100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:29:28.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3f is not active+clean 2023-08-27T11:29:28.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3e is not active+clean 2023-08-27T11:29:28.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3d is not active+clean 2023-08-27T11:29:28.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3c is not active+clean 2023-08-27T11:29:28.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3b is not active+clean 2023-08-27T11:29:28.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3a is not active+clean 2023-08-27T11:29:28.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.39 is not active+clean 2023-08-27T11:29:28.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.38 is not active+clean 2023-08-27T11:29:28.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.37 is not active+clean 2023-08-27T11:29:28.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.36 is not active+clean 2023-08-27T11:29:28.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.35 is not active+clean 2023-08-27T11:29:28.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.34 is not active+clean 2023-08-27T11:29:28.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.33 is not active+clean 2023-08-27T11:29:28.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.32 is not active+clean 2023-08-27T11:29:28.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.31 is not active+clean 2023-08-27T11:29:28.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.30 is not active+clean 2023-08-27T11:29:28.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2f is not active+clean 2023-08-27T11:29:28.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2e is not active+clean 2023-08-27T11:29:28.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2d is not active+clean 2023-08-27T11:29:28.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2c is not active+clean 2023-08-27T11:29:28.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2b is not active+clean 2023-08-27T11:29:28.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2a is not active+clean 2023-08-27T11:29:28.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.29 is not active+clean 2023-08-27T11:29:28.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.28 is not active+clean 2023-08-27T11:29:28.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.27 is not active+clean 2023-08-27T11:29:28.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.26 is not active+clean 2023-08-27T11:29:28.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.25 is not active+clean 2023-08-27T11:29:28.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.24 is not active+clean 2023-08-27T11:29:28.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.23 is not active+clean 2023-08-27T11:29:28.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.22 is not active+clean 2023-08-27T11:29:28.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.21 is not active+clean 2023-08-27T11:29:28.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.20 is not active+clean 2023-08-27T11:29:28.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.14 is not active+clean 2023-08-27T11:29:28.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.17 is not active+clean 2023-08-27T11:29:28.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.541996+0000', 'last_change': '2023-08-27T11:29:21.541996+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.541996+0000', 'last_undegraded': '2023-08-27T11:29:21.541996+0000', 'last_fullsized': '2023-08-27T11:29:21.541996+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:29:28.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.15 is not active+clean 2023-08-27T11:29:28.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.16 is not active+clean 2023-08-27T11:29:28.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.538975+0000', 'last_change': '2023-08-27T11:29:21.538975+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.538975+0000', 'last_undegraded': '2023-08-27T11:29:21.538975+0000', 'last_fullsized': '2023-08-27T11:29:21.538975+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:29:28.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.16 is not active+clean 2023-08-27T11:29:28.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.15 is not active+clean 2023-08-27T11:29:28.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.541380+0000', 'last_change': '2023-08-27T11:29:21.541380+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.541380+0000', 'last_undegraded': '2023-08-27T11:29:21.541380+0000', 'last_fullsized': '2023-08-27T11:29:21.541380+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:29:28.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.17 is not active+clean 2023-08-27T11:29:28.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.14 is not active+clean 2023-08-27T11:29:28.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.543003+0000', 'last_change': '2023-08-27T11:29:21.543003+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.543003+0000', 'last_undegraded': '2023-08-27T11:29:21.543003+0000', 'last_fullsized': '2023-08-27T11:29:21.543003+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:29:28.443 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.10 is not active+clean 2023-08-27T11:29:28.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.444 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.13 is not active+clean 2023-08-27T11:29:28.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.540914+0000', 'last_change': '2023-08-27T11:29:21.540914+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.540914+0000', 'last_undegraded': '2023-08-27T11:29:21.540914+0000', 'last_fullsized': '2023-08-27T11:29:21.540914+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:29:28.444 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.11 is not active+clean 2023-08-27T11:29:28.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.12 is not active+clean 2023-08-27T11:29:28.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.541230+0000', 'last_change': '2023-08-27T11:29:21.541230+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.541230+0000', 'last_undegraded': '2023-08-27T11:29:21.541230+0000', 'last_fullsized': '2023-08-27T11:29:21.541230+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:29:28.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.12 is not active+clean 2023-08-27T11:29:28.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.11 is not active+clean 2023-08-27T11:29:28.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.542983+0000', 'last_change': '2023-08-27T11:29:21.542983+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.542983+0000', 'last_undegraded': '2023-08-27T11:29:21.542983+0000', 'last_fullsized': '2023-08-27T11:29:21.542983+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:29:28.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.13 is not active+clean 2023-08-27T11:29:28.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.10 is not active+clean 2023-08-27T11:29:28.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.540240+0000', 'last_change': '2023-08-27T11:29:21.540240+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.540240+0000', 'last_undegraded': '2023-08-27T11:29:21.540240+0000', 'last_fullsized': '2023-08-27T11:29:21.540240+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:29:28.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.c is not active+clean 2023-08-27T11:29:28.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.f is not active+clean 2023-08-27T11:29:28.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.545172+0000', 'last_change': '2023-08-27T11:29:21.545172+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.545172+0000', 'last_undegraded': '2023-08-27T11:29:21.545172+0000', 'last_fullsized': '2023-08-27T11:29:21.545172+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:29:28.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.d is not active+clean 2023-08-27T11:29:28.451 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.451 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.e is not active+clean 2023-08-27T11:29:28.452 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.541880+0000', 'last_change': '2023-08-27T11:29:21.541880+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.541880+0000', 'last_undegraded': '2023-08-27T11:29:21.541880+0000', 'last_fullsized': '2023-08-27T11:29:21.541880+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:29:28.452 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.e is not active+clean 2023-08-27T11:29:28.452 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.453 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.d is not active+clean 2023-08-27T11:29:28.453 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.539440+0000', 'last_change': '2023-08-27T11:29:21.539440+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.539440+0000', 'last_undegraded': '2023-08-27T11:29:21.539440+0000', 'last_fullsized': '2023-08-27T11:29:21.539440+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:29:28.454 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.f is not active+clean 2023-08-27T11:29:28.454 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.454 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.c is not active+clean 2023-08-27T11:29:28.455 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.539983+0000', 'last_change': '2023-08-27T11:29:21.539983+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.539983+0000', 'last_undegraded': '2023-08-27T11:29:21.539983+0000', 'last_fullsized': '2023-08-27T11:29:21.539983+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:29:28.455 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.8 is not active+clean 2023-08-27T11:29:28.455 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.b is not active+clean 2023-08-27T11:29:28.456 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.540500+0000', 'last_change': '2023-08-27T11:29:21.540500+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.540500+0000', 'last_undegraded': '2023-08-27T11:29:21.540500+0000', 'last_fullsized': '2023-08-27T11:29:21.540500+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:29:28.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.9 is not active+clean 2023-08-27T11:29:28.457 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.a is not active+clean 2023-08-27T11:29:28.458 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.548472+0000', 'last_change': '2023-08-27T11:29:21.548472+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.548472+0000', 'last_undegraded': '2023-08-27T11:29:21.548472+0000', 'last_fullsized': '2023-08-27T11:29:21.548472+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:29:28.458 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.a is not active+clean 2023-08-27T11:29:28.458 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.459 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.9 is not active+clean 2023-08-27T11:29:28.459 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.548235+0000', 'last_change': '2023-08-27T11:29:21.548235+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.548235+0000', 'last_undegraded': '2023-08-27T11:29:21.548235+0000', 'last_fullsized': '2023-08-27T11:29:21.548235+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:29:28.460 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.b is not active+clean 2023-08-27T11:29:28.460 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.460 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.8 is not active+clean 2023-08-27T11:29:28.461 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.541912+0000', 'last_change': '2023-08-27T11:29:21.541912+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.541912+0000', 'last_undegraded': '2023-08-27T11:29:21.541912+0000', 'last_fullsized': '2023-08-27T11:29:21.541912+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:29:28.461 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.4 is not active+clean 2023-08-27T11:29:28.462 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.462 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.7 is not active+clean 2023-08-27T11:29:28.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.539729+0000', 'last_change': '2023-08-27T11:29:21.539729+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.539729+0000', 'last_undegraded': '2023-08-27T11:29:21.539729+0000', 'last_fullsized': '2023-08-27T11:29:21.539729+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:29:28.463 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.5 is not active+clean 2023-08-27T11:29:28.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.6 is not active+clean 2023-08-27T11:29:28.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.548184+0000', 'last_change': '2023-08-27T11:29:21.548184+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.548184+0000', 'last_undegraded': '2023-08-27T11:29:21.548184+0000', 'last_fullsized': '2023-08-27T11:29:21.548184+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:29:28.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.6 is not active+clean 2023-08-27T11:29:28.465 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.5 is not active+clean 2023-08-27T11:29:28.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.547888+0000', 'last_change': '2023-08-27T11:29:21.547888+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.547888+0000', 'last_undegraded': '2023-08-27T11:29:21.547888+0000', 'last_fullsized': '2023-08-27T11:29:21.547888+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:29:28.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.7 is not active+clean 2023-08-27T11:29:28.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.4 is not active+clean 2023-08-27T11:29:28.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.539022+0000', 'last_change': '2023-08-27T11:29:21.539022+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.539022+0000', 'last_undegraded': '2023-08-27T11:29:21.539022+0000', 'last_fullsized': '2023-08-27T11:29:21.539022+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:29:28.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.0 is not active+clean 2023-08-27T11:29:28.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.3 is not active+clean 2023-08-27T11:29:28.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.539979+0000', 'last_change': '2023-08-27T11:29:21.539979+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.539979+0000', 'last_undegraded': '2023-08-27T11:29:21.539979+0000', 'last_fullsized': '2023-08-27T11:29:21.539979+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:29:28.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1 is not active+clean 2023-08-27T11:29:28.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.2 is not active+clean 2023-08-27T11:29:28.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.539105+0000', 'last_change': '2023-08-27T11:29:21.539105+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.539105+0000', 'last_undegraded': '2023-08-27T11:29:21.539105+0000', 'last_fullsized': '2023-08-27T11:29:21.539105+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:29:28.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2 is not active+clean 2023-08-27T11:29:28.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1 is not active+clean 2023-08-27T11:29:28.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.541342+0000', 'last_change': '2023-08-27T11:29:21.541342+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.541342+0000', 'last_undegraded': '2023-08-27T11:29:21.541342+0000', 'last_fullsized': '2023-08-27T11:29:21.541342+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:29:28.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3 is not active+clean 2023-08-27T11:29:28.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.0 is not active+clean 2023-08-27T11:29:28.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.538994+0000', 'last_change': '2023-08-27T11:29:21.538994+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.538994+0000', 'last_undegraded': '2023-08-27T11:29:21.538994+0000', 'last_fullsized': '2023-08-27T11:29:21.538994+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:29:28.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.18 is not active+clean 2023-08-27T11:29:28.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1b is not active+clean 2023-08-27T11:29:28.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.547851+0000', 'last_change': '2023-08-27T11:29:21.547851+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.547851+0000', 'last_undegraded': '2023-08-27T11:29:21.547851+0000', 'last_fullsized': '2023-08-27T11:29:21.547851+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:29:28.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.19 is not active+clean 2023-08-27T11:29:28.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1a is not active+clean 2023-08-27T11:29:28.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.541676+0000', 'last_change': '2023-08-27T11:29:21.541676+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.541676+0000', 'last_undegraded': '2023-08-27T11:29:21.541676+0000', 'last_fullsized': '2023-08-27T11:29:21.541676+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:29:28.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1e is not active+clean 2023-08-27T11:29:28.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1d is not active+clean 2023-08-27T11:29:28.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.543238+0000', 'last_change': '2023-08-27T11:29:21.543238+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.543238+0000', 'last_undegraded': '2023-08-27T11:29:21.543238+0000', 'last_fullsized': '2023-08-27T11:29:21.543238+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:29:28.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1f is not active+clean 2023-08-27T11:29:28.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1c is not active+clean 2023-08-27T11:29:28.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.541675+0000', 'last_change': '2023-08-27T11:29:21.541675+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.541675+0000', 'last_undegraded': '2023-08-27T11:29:21.541675+0000', 'last_fullsized': '2023-08-27T11:29:21.541675+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:29:28.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1d is not active+clean 2023-08-27T11:29:28.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1e is not active+clean 2023-08-27T11:29:28.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.539991+0000', 'last_change': '2023-08-27T11:29:21.539991+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.539991+0000', 'last_undegraded': '2023-08-27T11:29:21.539991+0000', 'last_fullsized': '2023-08-27T11:29:21.539991+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:29:28.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1c is not active+clean 2023-08-27T11:29:28.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.1f is not active+clean 2023-08-27T11:29:28.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.541913+0000', 'last_change': '2023-08-27T11:29:21.541913+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.541913+0000', 'last_undegraded': '2023-08-27T11:29:21.541913+0000', 'last_fullsized': '2023-08-27T11:29:21.541913+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:29:28.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1b is not active+clean 2023-08-27T11:29:28.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.18 is not active+clean 2023-08-27T11:29:28.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.545172+0000', 'last_change': '2023-08-27T11:29:21.545172+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.545172+0000', 'last_undegraded': '2023-08-27T11:29:21.545172+0000', 'last_fullsized': '2023-08-27T11:29:21.545172+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:29:28.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1a is not active+clean 2023-08-27T11:29:28.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:29:22.536503+0000', 'last_change': '2023-08-27T11:29:22.536503+0000', 'last_active': '2023-08-27T11:29:22.536503+0000', 'last_peered': '2023-08-27T11:29:22.536503+0000', 'last_clean': '2023-08-27T11:29:22.536503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:22.536503+0000', 'last_undegraded': '2023-08-27T11:29:22.536503+0000', 'last_fullsized': '2023-08-27T11:29:22.536503+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:22.536503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:29:28.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 29.19 is not active+clean 2023-08-27T11:29:28.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '29.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 164, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:29:21.538971+0000', 'last_change': '2023-08-27T11:29:21.538971+0000', 'last_active': '2023-08-27T11:29:21.525498+0000', 'last_peered': '2023-08-27T11:29:21.525498+0000', 'last_clean': '2023-08-27T11:29:21.525498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:29:21.538971+0000', 'last_undegraded': '2023-08-27T11:29:21.538971+0000', 'last_fullsized': '2023-08-27T11:29:21.538971+0000', 'mapping_epoch': 164, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 164, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'last_clean_scrub_stamp': '2023-08-27T11:29:21.525498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:29:28.486 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-27T11:29:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.607+0000 7f76a3ca9700 1 Processor -- start 2023-08-27T11:29:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.607+0000 7f76a3ca9700 1 -- start start 2023-08-27T11:29:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a3ca9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c0cddb0 0x7f769c15f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a3ca9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f769c15bf90 0x7f769c0cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a3ca9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769c15eed0 con 0x7f769c0ce190 2023-08-27T11:29:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a3ca9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769c15f500 con 0x7f769c15bf90 2023-08-27T11:29:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a3ca9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c15fe60 con 0x7f769c0cddb0 2023-08-27T11:29:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a1244700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f769c0ce190 0x7f769c15b880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60798/0 (socket says 172.21.15.112:60798) 2023-08-27T11:29:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a1244700 1 -- 172.21.15.112:0/3176320968 learned_addr learned my addr 172.21.15.112:0/3176320968 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:28.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292738136 0 0) 0x7f769c15eed0 con 0x7f769c0ce190 2023-08-27T11:29:28.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76880035b0 con 0x7f769c0ce190 2023-08-27T11:29:28.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a1a45700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c0cddb0 0x7f769c15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a2246700 1 --2- 172.21.15.112:0/3176320968 >> v2:172.21.15.138:3300/0 conn(0x7f769c15bf90 0x7f769c0cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2904122747 0 0) 0x7f76880035b0 con 0x7f769c0ce190 2023-08-27T11:29:28.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c0cddb0 msgr2=0x7f769c15f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:28.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c0cddb0 0x7f769c15f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 >> v2:172.21.15.138:3300/0 conn(0x7f769c15bf90 msgr2=0x7f769c0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:28.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 --2- 172.21.15.112:0/3176320968 >> v2:172.21.15.138:3300/0 conn(0x7f769c15bf90 0x7f769c0cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c15c450 con 0x7f769c0ce190 2023-08-27T11:29:28.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f768c001670 con 0x7f769c0ce190 2023-08-27T11:29:28.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 >> v1:172.21.15.112:6789/0 conn(0x7f769c0ce190 legacy=0x7f769c15b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7688003a00 0x7f7688003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c0cddb0 0x7f7688004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7688004d90 0x7f7688005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c15f500 con 0x7f7688004d90 2023-08-27T11:29:28.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c15fe60 con 0x7f7688003a00 2023-08-27T11:29:28.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768800ba10 con 0x7f769c0cddb0 2023-08-27T11:29:28.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.608+0000 7f76a2246700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7688004d90 0x7f7688005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a1a45700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7688003a00 0x7f7688003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a1244700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c0cddb0 0x7f7688004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a2246700 1 -- 172.21.15.112:0/3176320968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c0cddb0 msgr2=0x7f7688004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:28.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a2246700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c0cddb0 0x7f7688004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a2246700 1 -- 172.21.15.112:0/3176320968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7688003a00 msgr2=0x7f7688003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:28.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a2246700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7688003a00 0x7f7688003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a2246700 1 -- 172.21.15.112:0/3176320968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76880035b0 con 0x7f7688004d90 2023-08-27T11:29:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a1a45700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7688003a00 0x7f7688003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a1244700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c0cddb0 0x7f7688004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a2246700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7688004d90 0x7f7688005210 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f7694009fc0 tx=0x7f7688005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb93d6455f6554c8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769400ec30 con 0x7f7688004d90 2023-08-27T11:29:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769400f040 con 0x7f7688004d90 2023-08-27T11:29:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a0a43700 1 -- 172.21.15.112:0/3176320968 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7694014480 con 0x7f7688004d90 2023-08-27T11:29:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a3ca9700 1 -- 172.21.15.112:0/3176320968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7688004d90 msgr2=0x7f7688005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a3ca9700 1 --2- 172.21.15.112:0/3176320968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7688004d90 0x7f7688005210 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f7694009fc0 tx=0x7f7688005880 comp rx=0 tx=0).stop 2023-08-27T11:29:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a2246700 1 -- 172.21.15.112:0/3176320968 reap_dead start 2023-08-27T11:29:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a3ca9700 1 -- 172.21.15.112:0/3176320968 shutdown_connections 2023-08-27T11:29:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.609+0000 7f76a3ca9700 1 -- 172.21.15.112:0/3176320968 >> 172.21.15.112:0/3176320968 conn(0x7f769c06f970 msgr2=0x7f769c06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.610+0000 7f76a3ca9700 1 -- 172.21.15.112:0/3176320968 shutdown_connections 2023-08-27T11:29:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.610+0000 7f76a3ca9700 1 -- 172.21.15.112:0/3176320968 wait complete. 2023-08-27T11:29:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.611+0000 7f76a3ca9700 1 Processor -- start 2023-08-27T11:29:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.611+0000 7f76a3ca9700 1 -- start start 2023-08-27T11:29:28.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.611+0000 7f76a3ca9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7688004d90 0x7f769c0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.611+0000 7f76a3ca9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f769c1f0a20 0x7f769c0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.611+0000 7f76a3ca9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c15a580 0x7f769c15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.611+0000 7f76a3ca9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0ca1d0 con 0x7f769c1f0a20 2023-08-27T11:29:28.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.611+0000 7f76a3ca9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0c95c0 con 0x7f7688004d90 2023-08-27T11:29:28.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.611+0000 7f76a3ca9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0c98a0 con 0x7f769c15a580 2023-08-27T11:29:28.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76a1244700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f769c1f0a20 0x7f769c0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76a1244700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f769c1f0a20 0x7f769c0b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38196/0 (socket says 172.21.15.112:38196) 2023-08-27T11:29:28.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76a1244700 1 -- 172.21.15.112:0/1852205303 learned_addr learned my addr 172.21.15.112:0/1852205303 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:28.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76a1a45700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7688004d90 0x7f769c0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76a2246700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c15a580 0x7f769c15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76a1244700 1 -- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c15a580 msgr2=0x7f769c15aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76a1244700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c15a580 0x7f769c15aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76a1244700 1 -- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7688004d90 msgr2=0x7f769c0c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76a1244700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7688004d90 0x7f769c0c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76a1244700 1 -- 172.21.15.112:0/1852205303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c0c9b80 con 0x7f769c1f0a20 2023-08-27T11:29:28.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76a1244700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f769c1f0a20 0x7f769c0b4cc0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f768c000fb0 tx=0x7f768c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:28.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76927fc700 1 -- 172.21.15.112:0/1852205303 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c012590 con 0x7f769c1f0a20 2023-08-27T11:29:28.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76927fc700 1 -- 172.21.15.112:0/1852205303 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c0126f0 con 0x7f769c1f0a20 2023-08-27T11:29:28.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76a3ca9700 1 -- 172.21.15.112:0/1852205303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c0c9e60 con 0x7f769c1f0a20 2023-08-27T11:29:28.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76927fc700 1 -- 172.21.15.112:0/1852205303 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c012a80 con 0x7f769c1f0a20 2023-08-27T11:29:28.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.612+0000 7f76a3ca9700 1 -- 172.21.15.112:0/1852205303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769c15af40 con 0x7f769c1f0a20 2023-08-27T11:29:28.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.613+0000 7f76a3ca9700 1 -- 172.21.15.112:0/1852205303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769c067a20 con 0x7f769c1f0a20 2023-08-27T11:29:28.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.613+0000 7f76927fc700 1 -- 172.21.15.112:0/1852205303 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f768c012be0 con 0x7f769c1f0a20 2023-08-27T11:29:28.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.614+0000 7f76927fc700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f768006a4d0 0x7f768006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:28.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.614+0000 7f76927fc700 1 -- 172.21.15.112:0/1852205303 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f768c088b80 con 0x7f769c1f0a20 2023-08-27T11:29:28.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.614+0000 7f76a1a45700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f768006a4d0 0x7f768006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:28.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.615+0000 7f76a1a45700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f768006a4d0 0x7f768006c990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7698009b40 tx=0x7f7698008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:28.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.618+0000 7f76927fc700 1 -- 172.21.15.112:0/1852205303 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f768c053c90 con 0x7f769c1f0a20 2023-08-27T11:29:28.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.881+0000 7f76a3ca9700 1 -- 172.21.15.112:0/1852205303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f769c0cf1c0 con 0x7f769c1f0a20 2023-08-27T11:29:28.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.882+0000 7f76927fc700 1 -- 172.21.15.112:0/1852205303 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1438 (secure 0 0 0) 0x7f768c088e60 con 0x7f769c1f0a20 2023-08-27T11:29:28.884 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:28.884 INFO:teuthology.orchestra.run.smithi112.stdout:{"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":762,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":167,"num_osds":8,"num_up_osds":8,"osd_up_since":1693135017,"num_in_osds":8,"osd_in_since":1693135009,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":24,"data_bytes":592778,"bytes_used":273616896,"bytes_avail":772820496384,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":113,"id":14,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":14,"rank":0,"name":"c","status":"up:active","gid":9393}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2023-08-27T11:28:02.395383+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-08-27T11:29:28.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.884+0000 7f76a3ca9700 1 -- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f768006a4d0 msgr2=0x7f768006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.884+0000 7f76a3ca9700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f768006a4d0 0x7f768006c990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7698009b40 tx=0x7f7698008040 comp rx=0 tx=0).stop 2023-08-27T11:29:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.884+0000 7f76a3ca9700 1 -- 172.21.15.112:0/1852205303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f769c1f0a20 msgr2=0x7f769c0b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.884+0000 7f76a3ca9700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f769c1f0a20 0x7f769c0b4cc0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f768c000fb0 tx=0x7f768c00eb80 comp rx=0 tx=0).stop 2023-08-27T11:29:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.884+0000 7f76a3ca9700 1 -- 172.21.15.112:0/1852205303 shutdown_connections 2023-08-27T11:29:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.884+0000 7f76a3ca9700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f768006a4d0 0x7f768006c990 secure :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7698009b40 tx=0x7f7698008040 comp rx=0 tx=0).stop 2023-08-27T11:29:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.884+0000 7f76a3ca9700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f769c15a580 0x7f769c15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.884+0000 7f76a3ca9700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7688004d90 0x7f769c0c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.884+0000 7f76a3ca9700 1 --2- 172.21.15.112:0/1852205303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f769c1f0a20 0x7f769c0b4cc0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.884+0000 7f76a3ca9700 1 -- 172.21.15.112:0/1852205303 >> 172.21.15.112:0/1852205303 conn(0x7f769c06f970 msgr2=0x7f769c153b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.884+0000 7f76a3ca9700 1 -- 172.21.15.112:0/1852205303 shutdown_connections 2023-08-27T11:29:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:28.884+0000 7f76a3ca9700 1 -- 172.21.15.112:0/1852205303 wait complete. 2023-08-27T11:29:28.897 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-27T11:29:28.897 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:29:29.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.016+0000 7f8763659700 1 Processor -- start 2023-08-27T11:29:29.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.016+0000 7f8763659700 1 -- start start 2023-08-27T11:29:29.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.017+0000 7f8763659700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c146990 0x7f875c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:29.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.017+0000 7f8763659700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f875c06d510 0x7f875c06d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:29.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.017+0000 7f8763659700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875c06f930 con 0x7f875c160a00 2023-08-27T11:29:29.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.017+0000 7f8763659700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f875c06dec0 con 0x7f875c06d510 2023-08-27T11:29:29.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.017+0000 7f8763659700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c06fc10 con 0x7f875c146990 2023-08-27T11:29:29.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.017+0000 7f8761bf6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f875c06d510 0x7f875c06d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.017+0000 7f8761bf6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f875c06d510 0x7f875c06d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38206/0 (socket says 172.21.15.112:38206) 2023-08-27T11:29:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.017+0000 7f8761bf6700 1 -- 172.21.15.112:0/581334130 learned_addr learned my addr 172.21.15.112:0/581334130 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.018+0000 7f8760bf4700 1 --2- 172.21.15.112:0/581334130 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c146990 0x7f875c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.018+0000 7f8761bf6700 1 -- 172.21.15.112:0/581334130 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c146990 msgr2=0x7f875c0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.018+0000 7f8761bf6700 1 --2- 172.21.15.112:0/581334130 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c146990 0x7f875c0b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:29.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.018+0000 7f8761bf6700 1 -- 172.21.15.112:0/581334130 >> v1:172.21.15.138:6789/0 conn(0x7f875c160a00 legacy=0x7f875c15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:29.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.018+0000 7f8761bf6700 1 -- 172.21.15.112:0/581334130 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c06fef0 con 0x7f875c06d510 2023-08-27T11:29:29.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.018+0000 7f8760bf4700 1 --2- 172.21.15.112:0/581334130 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c146990 0x7f875c0b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:29.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.018+0000 7f8761bf6700 1 --2- 172.21.15.112:0/581334130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875c06d510 0x7f875c06d980 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f8754009e50 tx=0x7f875400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31472f822be3f76c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:29.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.018+0000 7f8753fff700 1 -- 172.21.15.112:0/581334130 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875400e070 con 0x7f875c06d510 2023-08-27T11:29:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.018+0000 7f8753fff700 1 -- 172.21.15.112:0/581334130 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8754003a20 con 0x7f875c06d510 2023-08-27T11:29:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.018+0000 7f8753fff700 1 -- 172.21.15.112:0/581334130 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754003dd0 con 0x7f875c06d510 2023-08-27T11:29:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.019+0000 7f8763659700 1 -- 172.21.15.112:0/581334130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875c06d510 msgr2=0x7f875c06d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.019+0000 7f8763659700 1 --2- 172.21.15.112:0/581334130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875c06d510 0x7f875c06d980 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f8754009e50 tx=0x7f875400b330 comp rx=0 tx=0).stop 2023-08-27T11:29:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.019+0000 7f8763659700 1 -- 172.21.15.112:0/581334130 shutdown_connections 2023-08-27T11:29:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.019+0000 7f8763659700 1 --2- 172.21.15.112:0/581334130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875c06d510 0x7f875c06d980 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.019+0000 7f8763659700 1 --2- 172.21.15.112:0/581334130 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c146990 0x7f875c0b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.019+0000 7f8763659700 1 -- 172.21.15.112:0/581334130 >> 172.21.15.112:0/581334130 conn(0x7f875c0cfe50 msgr2=0x7f875c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.019+0000 7f8763659700 1 -- 172.21.15.112:0/581334130 shutdown_connections 2023-08-27T11:29:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.019+0000 7f8763659700 1 -- 172.21.15.112:0/581334130 wait complete. 2023-08-27T11:29:29.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.020+0000 7f8763659700 1 Processor -- start 2023-08-27T11:29:29.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.020+0000 7f8763659700 1 -- start start 2023-08-27T11:29:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8763659700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c06d510 0x7f875c145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8763659700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875c146990 0x7f875c1e6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8763659700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875c160a00 0x7f875c1e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8763659700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c0ad3d0 con 0x7f875c146990 2023-08-27T11:29:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8763659700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c1463b0 con 0x7f875c160a00 2023-08-27T11:29:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8763659700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c0aca30 con 0x7f875c06d510 2023-08-27T11:29:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8760bf4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875c146990 0x7f875c1e6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8760bf4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875c146990 0x7f875c1e6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38210/0 (socket says 172.21.15.112:38210) 2023-08-27T11:29:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8760bf4700 1 -- 172.21.15.112:0/2293485487 learned_addr learned my addr 172.21.15.112:0/2293485487 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f87613f5700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c06d510 0x7f875c145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8761bf6700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875c160a00 0x7f875c1e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8760bf4700 1 -- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c06d510 msgr2=0x7f875c145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8760bf4700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c06d510 0x7f875c145b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8760bf4700 1 -- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875c160a00 msgr2=0x7f875c1e65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:29.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8760bf4700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875c160a00 0x7f875c1e65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:29.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8760bf4700 1 -- 172.21.15.112:0/2293485487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8754009b30 con 0x7f875c146990 2023-08-27T11:29:29.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8761bf6700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875c160a00 0x7f875c1e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:29.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8760bf4700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875c146990 0x7f875c1e6080 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f875800c780 tx=0x7f875800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:29.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8751ffb700 1 -- 172.21.15.112:0/2293485487 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758011800 con 0x7f875c146990 2023-08-27T11:29:29.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8751ffb700 1 -- 172.21.15.112:0/2293485487 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8758011960 con 0x7f875c146990 2023-08-27T11:29:29.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8763659700 1 -- 172.21.15.112:0/2293485487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875c0acd10 con 0x7f875c146990 2023-08-27T11:29:29.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.021+0000 7f8751ffb700 1 -- 172.21.15.112:0/2293485487 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758011cd0 con 0x7f875c146990 2023-08-27T11:29:29.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.022+0000 7f8763659700 1 -- 172.21.15.112:0/2293485487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875c1460a0 con 0x7f875c146990 2023-08-27T11:29:29.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.022+0000 7f8763659700 1 -- 172.21.15.112:0/2293485487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875c0ad510 con 0x7f875c146990 2023-08-27T11:29:29.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.023+0000 7f8751ffb700 1 -- 172.21.15.112:0/2293485487 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f8758021350 con 0x7f875c146990 2023-08-27T11:29:29.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.023+0000 7f8751ffb700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f874006a4d0 0x7f874006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:29.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.023+0000 7f8751ffb700 1 -- 172.21.15.112:0/2293485487 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8758088cd0 con 0x7f875c146990 2023-08-27T11:29:29.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.024+0000 7f87613f5700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f874006a4d0 0x7f874006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:29.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.024+0000 7f87613f5700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f874006a4d0 0x7f874006c990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f874c007fd0 tx=0x7f874c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:29.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.027+0000 7f8751ffb700 1 -- 172.21.15.112:0/2293485487 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f875c0ad510 con 0x7f875c146990 2023-08-27T11:29:29.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.211+0000 7f8763659700 1 -- 172.21.15.112:0/2293485487 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f875c0704b0 con 0x7f874006a4d0 2023-08-27T11:29:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.220+0000 7f8751ffb700 1 -- 172.21.15.112:0/2293485487 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312069 (secure 0 0 0) 0x7f874c0123d0 con 0x7f874006a4d0 2023-08-27T11:29:29.225 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:29.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.223+0000 7f8763659700 1 -- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f874006a4d0 msgr2=0x7f874006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:29.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.223+0000 7f8763659700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f874006a4d0 0x7f874006c990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f874c007fd0 tx=0x7f874c009040 comp rx=0 tx=0).stop 2023-08-27T11:29:29.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.223+0000 7f8763659700 1 -- 172.21.15.112:0/2293485487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875c146990 msgr2=0x7f875c1e6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:29.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.223+0000 7f8763659700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875c146990 0x7f875c1e6080 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f875800c780 tx=0x7f875800cb40 comp rx=0 tx=0).stop 2023-08-27T11:29:29.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.223+0000 7f8763659700 1 -- 172.21.15.112:0/2293485487 shutdown_connections 2023-08-27T11:29:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.223+0000 7f8763659700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f874006a4d0 0x7f874006c990 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.223+0000 7f8763659700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875c160a00 0x7f875c1e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.223+0000 7f8763659700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c06d510 0x7f875c145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.223+0000 7f8763659700 1 --2- 172.21.15.112:0/2293485487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875c146990 0x7f875c1e6080 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.224+0000 7f8763659700 1 -- 172.21.15.112:0/2293485487 >> 172.21.15.112:0/2293485487 conn(0x7f875c0cfe50 msgr2=0x7f875c0d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:29.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.224+0000 7f8763659700 1 -- 172.21.15.112:0/2293485487 shutdown_connections 2023-08-27T11:29:29.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:29.224+0000 7f8763659700 1 -- 172.21.15.112:0/2293485487 wait complete. 2023-08-27T11:29:29.230 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:29:29.237 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2023-08-27T11:29:28.416537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267204,"kb_used_data":17764,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754707516,"statfs":{"total":773094113280,"available":772820496384,"internally_reserved":0,"allocated":18190336,"data_stored":9253579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001434"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134944+0000","last_change":"2023-08-27T11:29:23.571985+0000","last_active":"2023-08-27T11:29:25.134944+0000","last_peered":"2023-08-27T11:29:25.134944+0000","last_clean":"2023-08-27T11:29:25.134944+0000","last_became_active":"2023-08-27T11:29:23.571737+0000","last_became_peered":"2023-08-27T11:29:23.571737+0000","last_unstale":"2023-08-27T11:29:25.134944+0000","last_undegraded":"2023-08-27T11:29:25.134944+0000","last_fullsized":"2023-08-27T11:29:25.134944+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:21.305005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978202+0000","last_change":"2023-08-27T11:29:23.571830+0000","last_active":"2023-08-27T11:29:25.978202+0000","last_peered":"2023-08-27T11:29:25.978202+0000","last_clean":"2023-08-27T11:29:25.978202+0000","last_became_active":"2023-08-27T11:29:23.571566+0000","last_became_peered":"2023-08-27T11:29:23.571566+0000","last_unstale":"2023-08-27T11:29:25.978202+0000","last_undegraded":"2023-08-27T11:29:25.978202+0000","last_fullsized":"2023-08-27T11:29:25.978202+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:02:34.211624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978208+0000","last_change":"2023-08-27T11:29:23.561965+0000","last_active":"2023-08-27T11:29:25.978208+0000","last_peered":"2023-08-27T11:29:25.978208+0000","last_clean":"2023-08-27T11:29:25.978208+0000","last_became_active":"2023-08-27T11:29:23.561687+0000","last_became_peered":"2023-08-27T11:29:23.561687+0000","last_unstale":"2023-08-27T11:29:25.978208+0000","last_undegraded":"2023-08-27T11:29:25.978208+0000","last_fullsized":"2023-08-27T11:29:25.978208+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:00:12.715079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978740+0000","last_change":"2023-08-27T11:29:23.570451+0000","last_active":"2023-08-27T11:29:25.978740+0000","last_peered":"2023-08-27T11:29:25.978740+0000","last_clean":"2023-08-27T11:29:25.978740+0000","last_became_active":"2023-08-27T11:29:23.570094+0000","last_became_peered":"2023-08-27T11:29:23.570094+0000","last_unstale":"2023-08-27T11:29:25.978740+0000","last_undegraded":"2023-08-27T11:29:25.978740+0000","last_fullsized":"2023-08-27T11:29:25.978740+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:13:37.637896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134822+0000","last_change":"2023-08-27T11:29:23.569584+0000","last_active":"2023-08-27T11:29:25.134822+0000","last_peered":"2023-08-27T11:29:25.134822+0000","last_clean":"2023-08-27T11:29:25.134822+0000","last_became_active":"2023-08-27T11:29:23.569288+0000","last_became_peered":"2023-08-27T11:29:23.569288+0000","last_unstale":"2023-08-27T11:29:25.134822+0000","last_undegraded":"2023-08-27T11:29:25.134822+0000","last_fullsized":"2023-08-27T11:29:25.134822+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:38:57.352131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553562+0000","last_change":"2023-08-27T11:29:23.573122+0000","last_active":"2023-08-27T11:29:24.553562+0000","last_peered":"2023-08-27T11:29:24.553562+0000","last_clean":"2023-08-27T11:29:24.553562+0000","last_became_active":"2023-08-27T11:29:23.572836+0000","last_became_peered":"2023-08-27T11:29:23.572836+0000","last_unstale":"2023-08-27T11:29:24.553562+0000","last_undegraded":"2023-08-27T11:29:24.553562+0000","last_fullsized":"2023-08-27T11:29:24.553562+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:49:20.104822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977858+0000","last_change":"2023-08-27T11:29:23.564383+0000","last_active":"2023-08-27T11:29:25.977858+0000","last_peered":"2023-08-27T11:29:25.977858+0000","last_clean":"2023-08-27T11:29:25.977858+0000","last_became_active":"2023-08-27T11:29:23.564129+0000","last_became_peered":"2023-08-27T11:29:23.564129+0000","last_unstale":"2023-08-27T11:29:25.977858+0000","last_undegraded":"2023-08-27T11:29:25.977858+0000","last_fullsized":"2023-08-27T11:29:25.977858+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:52.604143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553353+0000","last_change":"2023-08-27T11:29:23.571058+0000","last_active":"2023-08-27T11:29:24.553353+0000","last_peered":"2023-08-27T11:29:24.553353+0000","last_clean":"2023-08-27T11:29:24.553353+0000","last_became_active":"2023-08-27T11:29:23.570776+0000","last_became_peered":"2023-08-27T11:29:23.570776+0000","last_unstale":"2023-08-27T11:29:24.553353+0000","last_undegraded":"2023-08-27T11:29:24.553353+0000","last_fullsized":"2023-08-27T11:29:24.553353+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:39.692761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477610+0000","last_change":"2023-08-27T11:29:23.568596+0000","last_active":"2023-08-27T11:29:25.477610+0000","last_peered":"2023-08-27T11:29:25.477610+0000","last_clean":"2023-08-27T11:29:25.477610+0000","last_became_active":"2023-08-27T11:29:23.568324+0000","last_became_peered":"2023-08-27T11:29:23.568324+0000","last_unstale":"2023-08-27T11:29:25.477610+0000","last_undegraded":"2023-08-27T11:29:25.477610+0000","last_fullsized":"2023-08-27T11:29:25.477610+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:32:36.495461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978268+0000","last_change":"2023-08-27T11:29:23.570426+0000","last_active":"2023-08-27T11:29:25.978268+0000","last_peered":"2023-08-27T11:29:25.978268+0000","last_clean":"2023-08-27T11:29:25.978268+0000","last_became_active":"2023-08-27T11:29:23.570162+0000","last_became_peered":"2023-08-27T11:29:23.570162+0000","last_unstale":"2023-08-27T11:29:25.978268+0000","last_undegraded":"2023-08-27T11:29:25.978268+0000","last_fullsized":"2023-08-27T11:29:25.978268+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:28:11.052067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978899+0000","last_change":"2023-08-27T11:29:23.569056+0000","last_active":"2023-08-27T11:29:25.978899+0000","last_peered":"2023-08-27T11:29:25.978899+0000","last_clean":"2023-08-27T11:29:25.978899+0000","last_became_active":"2023-08-27T11:29:23.568749+0000","last_became_peered":"2023-08-27T11:29:23.568749+0000","last_unstale":"2023-08-27T11:29:25.978899+0000","last_undegraded":"2023-08-27T11:29:25.978899+0000","last_fullsized":"2023-08-27T11:29:25.978899+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:25:27.682286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551907+0000","last_change":"2023-08-27T11:29:23.574176+0000","last_active":"2023-08-27T11:29:24.551907+0000","last_peered":"2023-08-27T11:29:24.551907+0000","last_clean":"2023-08-27T11:29:24.551907+0000","last_became_active":"2023-08-27T11:29:23.571907+0000","last_became_peered":"2023-08-27T11:29:23.571907+0000","last_unstale":"2023-08-27T11:29:24.551907+0000","last_undegraded":"2023-08-27T11:29:24.551907+0000","last_fullsized":"2023-08-27T11:29:24.551907+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:41:35.223678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553737+0000","last_change":"2023-08-27T11:29:23.573811+0000","last_active":"2023-08-27T11:29:24.553737+0000","last_peered":"2023-08-27T11:29:24.553737+0000","last_clean":"2023-08-27T11:29:24.553737+0000","last_became_active":"2023-08-27T11:29:23.573536+0000","last_became_peered":"2023-08-27T11:29:23.573536+0000","last_unstale":"2023-08-27T11:29:24.553737+0000","last_undegraded":"2023-08-27T11:29:24.553737+0000","last_fullsized":"2023-08-27T11:29:24.553737+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:44:18.567183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551908+0000","last_change":"2023-08-27T11:29:23.564844+0000","last_active":"2023-08-27T11:29:24.551908+0000","last_peered":"2023-08-27T11:29:24.551908+0000","last_clean":"2023-08-27T11:29:24.551908+0000","last_became_active":"2023-08-27T11:29:23.564533+0000","last_became_peered":"2023-08-27T11:29:23.564533+0000","last_unstale":"2023-08-27T11:29:24.551908+0000","last_undegraded":"2023-08-27T11:29:24.551908+0000","last_fullsized":"2023-08-27T11:29:24.551908+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:34:27.688653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548282+0000","last_change":"2023-08-27T11:29:23.563226+0000","last_active":"2023-08-27T11:29:24.548282+0000","last_peered":"2023-08-27T11:29:24.548282+0000","last_clean":"2023-08-27T11:29:24.548282+0000","last_became_active":"2023-08-27T11:29:23.562934+0000","last_became_peered":"2023-08-27T11:29:23.562934+0000","last_unstale":"2023-08-27T11:29:24.548282+0000","last_undegraded":"2023-08-27T11:29:24.548282+0000","last_fullsized":"2023-08-27T11:29:24.548282+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:45:22.565179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553339+0000","last_change":"2023-08-27T11:29:23.577445+0000","last_active":"2023-08-27T11:29:24.553339+0000","last_peered":"2023-08-27T11:29:24.553339+0000","last_clean":"2023-08-27T11:29:24.553339+0000","last_became_active":"2023-08-27T11:29:23.577153+0000","last_became_peered":"2023-08-27T11:29:23.577153+0000","last_unstale":"2023-08-27T11:29:24.553339+0000","last_undegraded":"2023-08-27T11:29:24.553339+0000","last_fullsized":"2023-08-27T11:29:24.553339+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:18:03.892673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552061+0000","last_change":"2023-08-27T11:29:23.571984+0000","last_active":"2023-08-27T11:29:24.552061+0000","last_peered":"2023-08-27T11:29:24.552061+0000","last_clean":"2023-08-27T11:29:24.552061+0000","last_became_active":"2023-08-27T11:29:23.571661+0000","last_became_peered":"2023-08-27T11:29:23.571661+0000","last_unstale":"2023-08-27T11:29:24.552061+0000","last_undegraded":"2023-08-27T11:29:24.552061+0000","last_fullsized":"2023-08-27T11:29:24.552061+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:59:47.321676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978320+0000","last_change":"2023-08-27T11:29:23.558818+0000","last_active":"2023-08-27T11:29:25.978320+0000","last_peered":"2023-08-27T11:29:25.978320+0000","last_clean":"2023-08-27T11:29:25.978320+0000","last_became_active":"2023-08-27T11:29:23.558518+0000","last_became_peered":"2023-08-27T11:29:23.558518+0000","last_unstale":"2023-08-27T11:29:25.978320+0000","last_undegraded":"2023-08-27T11:29:25.978320+0000","last_fullsized":"2023-08-27T11:29:25.978320+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:16:46.158709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134856+0000","last_change":"2023-08-27T11:29:23.569571+0000","last_active":"2023-08-27T11:29:25.134856+0000","last_peered":"2023-08-27T11:29:25.134856+0000","last_clean":"2023-08-27T11:29:25.134856+0000","last_became_active":"2023-08-27T11:29:23.569232+0000","last_became_peered":"2023-08-27T11:29:23.569232+0000","last_unstale":"2023-08-27T11:29:25.134856+0000","last_undegraded":"2023-08-27T11:29:25.134856+0000","last_fullsized":"2023-08-27T11:29:25.134856+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:42.128995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553746+0000","last_change":"2023-08-27T11:29:23.568862+0000","last_active":"2023-08-27T11:29:24.553746+0000","last_peered":"2023-08-27T11:29:24.553746+0000","last_clean":"2023-08-27T11:29:24.553746+0000","last_became_active":"2023-08-27T11:29:23.568590+0000","last_became_peered":"2023-08-27T11:29:23.568590+0000","last_unstale":"2023-08-27T11:29:24.553746+0000","last_undegraded":"2023-08-27T11:29:24.553746+0000","last_fullsized":"2023-08-27T11:29:24.553746+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:01:06.953586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977584+0000","last_change":"2023-08-27T11:29:23.562119+0000","last_active":"2023-08-27T11:29:25.977584+0000","last_peered":"2023-08-27T11:29:25.977584+0000","last_clean":"2023-08-27T11:29:25.977584+0000","last_became_active":"2023-08-27T11:29:23.561799+0000","last_became_peered":"2023-08-27T11:29:23.561799+0000","last_unstale":"2023-08-27T11:29:25.977584+0000","last_undegraded":"2023-08-27T11:29:25.977584+0000","last_fullsized":"2023-08-27T11:29:25.977584+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:05:09.501842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548385+0000","last_change":"2023-08-27T11:29:23.569700+0000","last_active":"2023-08-27T11:29:24.548385+0000","last_peered":"2023-08-27T11:29:24.548385+0000","last_clean":"2023-08-27T11:29:24.548385+0000","last_became_active":"2023-08-27T11:29:23.568767+0000","last_became_peered":"2023-08-27T11:29:23.568767+0000","last_unstale":"2023-08-27T11:29:24.548385+0000","last_undegraded":"2023-08-27T11:29:24.548385+0000","last_fullsized":"2023-08-27T11:29:24.548385+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:15:09.273200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477367+0000","last_change":"2023-08-27T11:29:23.569362+0000","last_active":"2023-08-27T11:29:25.477367+0000","last_peered":"2023-08-27T11:29:25.477367+0000","last_clean":"2023-08-27T11:29:25.477367+0000","last_became_active":"2023-08-27T11:29:23.569111+0000","last_became_peered":"2023-08-27T11:29:23.569111+0000","last_unstale":"2023-08-27T11:29:25.477367+0000","last_undegraded":"2023-08-27T11:29:25.477367+0000","last_fullsized":"2023-08-27T11:29:25.477367+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:37:50.003802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134482+0000","last_change":"2023-08-27T11:29:23.570472+0000","last_active":"2023-08-27T11:29:25.134482+0000","last_peered":"2023-08-27T11:29:25.134482+0000","last_clean":"2023-08-27T11:29:25.134482+0000","last_became_active":"2023-08-27T11:29:23.568244+0000","last_became_peered":"2023-08-27T11:29:23.568244+0000","last_unstale":"2023-08-27T11:29:25.134482+0000","last_undegraded":"2023-08-27T11:29:25.134482+0000","last_fullsized":"2023-08-27T11:29:25.134482+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:21:54.469916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134995+0000","last_change":"2023-08-27T11:29:23.576730+0000","last_active":"2023-08-27T11:29:25.134995+0000","last_peered":"2023-08-27T11:29:25.134995+0000","last_clean":"2023-08-27T11:29:25.134995+0000","last_became_active":"2023-08-27T11:29:23.576168+0000","last_became_peered":"2023-08-27T11:29:23.576168+0000","last_unstale":"2023-08-27T11:29:25.134995+0000","last_undegraded":"2023-08-27T11:29:25.134995+0000","last_fullsized":"2023-08-27T11:29:25.134995+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:56:52.509141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978405+0000","last_change":"2023-08-27T11:29:23.567015+0000","last_active":"2023-08-27T11:29:25.978405+0000","last_peered":"2023-08-27T11:29:25.978405+0000","last_clean":"2023-08-27T11:29:25.978405+0000","last_became_active":"2023-08-27T11:29:23.566662+0000","last_became_peered":"2023-08-27T11:29:23.566662+0000","last_unstale":"2023-08-27T11:29:25.978405+0000","last_undegraded":"2023-08-27T11:29:25.978405+0000","last_fullsized":"2023-08-27T11:29:25.978405+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:54:19.977743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.979077+0000","last_change":"2023-08-27T11:29:23.575522+0000","last_active":"2023-08-27T11:29:25.979077+0000","last_peered":"2023-08-27T11:29:25.979077+0000","last_clean":"2023-08-27T11:29:25.979077+0000","last_became_active":"2023-08-27T11:29:23.569334+0000","last_became_peered":"2023-08-27T11:29:23.569334+0000","last_unstale":"2023-08-27T11:29:25.979077+0000","last_undegraded":"2023-08-27T11:29:25.979077+0000","last_fullsized":"2023-08-27T11:29:25.979077+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:02:15.157138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552143+0000","last_change":"2023-08-27T11:29:23.572875+0000","last_active":"2023-08-27T11:29:24.552143+0000","last_peered":"2023-08-27T11:29:24.552143+0000","last_clean":"2023-08-27T11:29:24.552143+0000","last_became_active":"2023-08-27T11:29:23.569074+0000","last_became_peered":"2023-08-27T11:29:23.569074+0000","last_unstale":"2023-08-27T11:29:24.552143+0000","last_undegraded":"2023-08-27T11:29:24.552143+0000","last_fullsized":"2023-08-27T11:29:24.552143+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:48:38.362418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977695+0000","last_change":"2023-08-27T11:29:23.573735+0000","last_active":"2023-08-27T11:29:25.977695+0000","last_peered":"2023-08-27T11:29:25.977695+0000","last_clean":"2023-08-27T11:29:25.977695+0000","last_became_active":"2023-08-27T11:29:23.573427+0000","last_became_peered":"2023-08-27T11:29:23.573427+0000","last_unstale":"2023-08-27T11:29:25.977695+0000","last_undegraded":"2023-08-27T11:29:25.977695+0000","last_fullsized":"2023-08-27T11:29:25.977695+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:47:27.112051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553667+0000","last_change":"2023-08-27T11:29:23.580405+0000","last_active":"2023-08-27T11:29:24.553667+0000","last_peered":"2023-08-27T11:29:24.553667+0000","last_clean":"2023-08-27T11:29:24.553667+0000","last_became_active":"2023-08-27T11:29:23.580101+0000","last_became_peered":"2023-08-27T11:29:23.580101+0000","last_unstale":"2023-08-27T11:29:24.553667+0000","last_undegraded":"2023-08-27T11:29:24.553667+0000","last_fullsized":"2023-08-27T11:29:24.553667+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:32.765196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477476+0000","last_change":"2023-08-27T11:29:23.556890+0000","last_active":"2023-08-27T11:29:25.477476+0000","last_peered":"2023-08-27T11:29:25.477476+0000","last_clean":"2023-08-27T11:29:25.477476+0000","last_became_active":"2023-08-27T11:29:23.556609+0000","last_became_peered":"2023-08-27T11:29:23.556609+0000","last_unstale":"2023-08-27T11:29:25.477476+0000","last_undegraded":"2023-08-27T11:29:25.477476+0000","last_fullsized":"2023-08-27T11:29:25.477476+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:04:52.635785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553569+0000","last_change":"2023-08-27T11:29:23.580349+0000","last_active":"2023-08-27T11:29:24.553569+0000","last_peered":"2023-08-27T11:29:24.553569+0000","last_clean":"2023-08-27T11:29:24.553569+0000","last_became_active":"2023-08-27T11:29:23.580041+0000","last_became_peered":"2023-08-27T11:29:23.580041+0000","last_unstale":"2023-08-27T11:29:24.553569+0000","last_undegraded":"2023-08-27T11:29:24.553569+0000","last_fullsized":"2023-08-27T11:29:24.553569+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:14:10.823406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548762+0000","last_change":"2023-08-27T11:29:23.569596+0000","last_active":"2023-08-27T11:29:24.548762+0000","last_peered":"2023-08-27T11:29:24.548762+0000","last_clean":"2023-08-27T11:29:24.548762+0000","last_became_active":"2023-08-27T11:29:23.569330+0000","last_became_peered":"2023-08-27T11:29:23.569330+0000","last_unstale":"2023-08-27T11:29:24.548762+0000","last_undegraded":"2023-08-27T11:29:24.548762+0000","last_fullsized":"2023-08-27T11:29:24.548762+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:05:09.501842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.135033+0000","last_change":"2023-08-27T11:29:22.562070+0000","last_active":"2023-08-27T11:29:25.135033+0000","last_peered":"2023-08-27T11:29:25.135033+0000","last_clean":"2023-08-27T11:29:25.135033+0000","last_became_active":"2023-08-27T11:29:22.561802+0000","last_became_peered":"2023-08-27T11:29:22.561802+0000","last_unstale":"2023-08-27T11:29:25.135033+0000","last_undegraded":"2023-08-27T11:29:25.135033+0000","last_fullsized":"2023-08-27T11:29:25.135033+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:01:07.716603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548600+0000","last_change":"2023-08-27T11:29:23.568672+0000","last_active":"2023-08-27T11:29:24.548600+0000","last_peered":"2023-08-27T11:29:24.548600+0000","last_clean":"2023-08-27T11:29:24.548600+0000","last_became_active":"2023-08-27T11:29:23.568409+0000","last_became_peered":"2023-08-27T11:29:23.568409+0000","last_unstale":"2023-08-27T11:29:24.548600+0000","last_undegraded":"2023-08-27T11:29:24.548600+0000","last_fullsized":"2023-08-27T11:29:24.548600+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:22:38.147718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978997+0000","last_change":"2023-08-27T11:29:22.562667+0000","last_active":"2023-08-27T11:29:25.978997+0000","last_peered":"2023-08-27T11:29:25.978997+0000","last_clean":"2023-08-27T11:29:25.978997+0000","last_became_active":"2023-08-27T11:29:22.562345+0000","last_became_peered":"2023-08-27T11:29:22.562345+0000","last_unstale":"2023-08-27T11:29:25.978997+0000","last_undegraded":"2023-08-27T11:29:25.978997+0000","last_fullsized":"2023-08-27T11:29:25.978997+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:41:24.143018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978446+0000","last_change":"2023-08-27T11:29:23.561542+0000","last_active":"2023-08-27T11:29:25.978446+0000","last_peered":"2023-08-27T11:29:25.978446+0000","last_clean":"2023-08-27T11:29:25.978446+0000","last_became_active":"2023-08-27T11:29:23.561273+0000","last_became_peered":"2023-08-27T11:29:23.561273+0000","last_unstale":"2023-08-27T11:29:25.978446+0000","last_undegraded":"2023-08-27T11:29:25.978446+0000","last_fullsized":"2023-08-27T11:29:25.978446+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:04:36.834335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"166'3","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134981+0000","last_change":"2023-08-27T11:29:22.565766+0000","last_active":"2023-08-27T11:29:25.134981+0000","last_peered":"2023-08-27T11:29:25.134981+0000","last_clean":"2023-08-27T11:29:25.134981+0000","last_became_active":"2023-08-27T11:29:22.565461+0000","last_became_peered":"2023-08-27T11:29:22.565461+0000","last_unstale":"2023-08-27T11:29:25.134981+0000","last_undegraded":"2023-08-27T11:29:25.134981+0000","last_fullsized":"2023-08-27T11:29:25.134981+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:32:10.556679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.979145+0000","last_change":"2023-08-27T11:29:23.570552+0000","last_active":"2023-08-27T11:29:25.979145+0000","last_peered":"2023-08-27T11:29:25.979145+0000","last_clean":"2023-08-27T11:29:25.979145+0000","last_became_active":"2023-08-27T11:29:23.568978+0000","last_became_peered":"2023-08-27T11:29:23.568978+0000","last_unstale":"2023-08-27T11:29:25.979145+0000","last_undegraded":"2023-08-27T11:29:25.979145+0000","last_fullsized":"2023-08-27T11:29:25.979145+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:48.791526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"166'6","reported_seq":19,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553803+0000","last_change":"2023-08-27T11:29:22.568408+0000","last_active":"2023-08-27T11:29:24.553803+0000","last_peered":"2023-08-27T11:29:24.553803+0000","last_clean":"2023-08-27T11:29:24.553803+0000","last_became_active":"2023-08-27T11:29:22.567952+0000","last_became_peered":"2023-08-27T11:29:22.567952+0000","last_unstale":"2023-08-27T11:29:24.553803+0000","last_undegraded":"2023-08-27T11:29:24.553803+0000","last_fullsized":"2023-08-27T11:29:24.553803+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:18:50.478300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977493+0000","last_change":"2023-08-27T11:29:23.557170+0000","last_active":"2023-08-27T11:29:25.977493+0000","last_peered":"2023-08-27T11:29:25.977493+0000","last_clean":"2023-08-27T11:29:25.977493+0000","last_became_active":"2023-08-27T11:29:23.556886+0000","last_became_peered":"2023-08-27T11:29:23.556886+0000","last_unstale":"2023-08-27T11:29:25.977493+0000","last_undegraded":"2023-08-27T11:29:25.977493+0000","last_fullsized":"2023-08-27T11:29:25.977493+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:26:27.741746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"166'2","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552726+0000","last_change":"2023-08-27T11:29:22.560057+0000","last_active":"2023-08-27T11:29:24.552726+0000","last_peered":"2023-08-27T11:29:24.552726+0000","last_clean":"2023-08-27T11:29:24.552726+0000","last_became_active":"2023-08-27T11:29:22.559575+0000","last_became_peered":"2023-08-27T11:29:22.559575+0000","last_unstale":"2023-08-27T11:29:24.552726+0000","last_undegraded":"2023-08-27T11:29:24.552726+0000","last_fullsized":"2023-08-27T11:29:24.552726+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:28:16.343767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551664+0000","last_change":"2023-08-27T11:29:23.572143+0000","last_active":"2023-08-27T11:29:24.551664+0000","last_peered":"2023-08-27T11:29:24.551664+0000","last_clean":"2023-08-27T11:29:24.551664+0000","last_became_active":"2023-08-27T11:29:23.571833+0000","last_became_peered":"2023-08-27T11:29:23.571833+0000","last_unstale":"2023-08-27T11:29:24.551664+0000","last_undegraded":"2023-08-27T11:29:24.551664+0000","last_fullsized":"2023-08-27T11:29:24.551664+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:23:16.433248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977651+0000","last_change":"2023-08-27T11:29:22.559033+0000","last_active":"2023-08-27T11:29:25.977651+0000","last_peered":"2023-08-27T11:29:25.977651+0000","last_clean":"2023-08-27T11:29:25.977651+0000","last_became_active":"2023-08-27T11:29:22.558779+0000","last_became_peered":"2023-08-27T11:29:22.558779+0000","last_unstale":"2023-08-27T11:29:25.977651+0000","last_undegraded":"2023-08-27T11:29:25.977651+0000","last_fullsized":"2023-08-27T11:29:25.977651+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:22:17.805705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977688+0000","last_change":"2023-08-27T11:29:23.569437+0000","last_active":"2023-08-27T11:29:25.977688+0000","last_peered":"2023-08-27T11:29:25.977688+0000","last_clean":"2023-08-27T11:29:25.977688+0000","last_became_active":"2023-08-27T11:29:23.569195+0000","last_became_peered":"2023-08-27T11:29:23.569195+0000","last_unstale":"2023-08-27T11:29:25.977688+0000","last_undegraded":"2023-08-27T11:29:25.977688+0000","last_fullsized":"2023-08-27T11:29:25.977688+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:11:00.606578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553272+0000","last_change":"2023-08-27T11:29:22.561162+0000","last_active":"2023-08-27T11:29:24.553272+0000","last_peered":"2023-08-27T11:29:24.553272+0000","last_clean":"2023-08-27T11:29:24.553272+0000","last_became_active":"2023-08-27T11:29:22.560845+0000","last_became_peered":"2023-08-27T11:29:22.560845+0000","last_unstale":"2023-08-27T11:29:24.553272+0000","last_undegraded":"2023-08-27T11:29:24.553272+0000","last_fullsized":"2023-08-27T11:29:24.553272+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:41:17.191787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548331+0000","last_change":"2023-08-27T11:29:23.565070+0000","last_active":"2023-08-27T11:29:24.548331+0000","last_peered":"2023-08-27T11:29:24.548331+0000","last_clean":"2023-08-27T11:29:24.548331+0000","last_became_active":"2023-08-27T11:29:23.564803+0000","last_became_peered":"2023-08-27T11:29:23.564803+0000","last_unstale":"2023-08-27T11:29:24.548331+0000","last_undegraded":"2023-08-27T11:29:24.548331+0000","last_fullsized":"2023-08-27T11:29:24.548331+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:12:57.090796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"166'8","reported_seq":21,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551678+0000","last_change":"2023-08-27T11:29:22.558198+0000","last_active":"2023-08-27T11:29:24.551678+0000","last_peered":"2023-08-27T11:29:24.551678+0000","last_clean":"2023-08-27T11:29:24.551678+0000","last_became_active":"2023-08-27T11:29:22.557898+0000","last_became_peered":"2023-08-27T11:29:22.557898+0000","last_unstale":"2023-08-27T11:29:24.551678+0000","last_undegraded":"2023-08-27T11:29:24.551678+0000","last_fullsized":"2023-08-27T11:29:24.551678+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:20:35.852006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977572+0000","last_change":"2023-08-27T11:29:23.568447+0000","last_active":"2023-08-27T11:29:25.977572+0000","last_peered":"2023-08-27T11:29:25.977572+0000","last_clean":"2023-08-27T11:29:25.977572+0000","last_became_active":"2023-08-27T11:29:23.567576+0000","last_became_peered":"2023-08-27T11:29:23.567576+0000","last_unstale":"2023-08-27T11:29:25.977572+0000","last_undegraded":"2023-08-27T11:29:25.977572+0000","last_fullsized":"2023-08-27T11:29:25.977572+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:24:56.575880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"166'1","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978391+0000","last_change":"2023-08-27T11:29:22.560159+0000","last_active":"2023-08-27T11:29:25.978391+0000","last_peered":"2023-08-27T11:29:25.978391+0000","last_clean":"2023-08-27T11:29:25.978391+0000","last_became_active":"2023-08-27T11:29:22.559773+0000","last_became_peered":"2023-08-27T11:29:22.559773+0000","last_unstale":"2023-08-27T11:29:25.978391+0000","last_undegraded":"2023-08-27T11:29:25.978391+0000","last_fullsized":"2023-08-27T11:29:25.978391+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:49:28.746872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134879+0000","last_change":"2023-08-27T11:29:23.581911+0000","last_active":"2023-08-27T11:29:25.134879+0000","last_peered":"2023-08-27T11:29:25.134879+0000","last_clean":"2023-08-27T11:29:25.134879+0000","last_became_active":"2023-08-27T11:29:23.581566+0000","last_became_peered":"2023-08-27T11:29:23.581566+0000","last_unstale":"2023-08-27T11:29:25.134879+0000","last_undegraded":"2023-08-27T11:29:25.134879+0000","last_fullsized":"2023-08-27T11:29:25.134879+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:16:47.512485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"166'2","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477630+0000","last_change":"2023-08-27T11:29:22.556687+0000","last_active":"2023-08-27T11:29:25.477630+0000","last_peered":"2023-08-27T11:29:25.477630+0000","last_clean":"2023-08-27T11:29:25.477630+0000","last_became_active":"2023-08-27T11:29:22.556430+0000","last_became_peered":"2023-08-27T11:29:22.556430+0000","last_unstale":"2023-08-27T11:29:25.477630+0000","last_undegraded":"2023-08-27T11:29:25.477630+0000","last_fullsized":"2023-08-27T11:29:25.477630+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:33:37.647840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978504+0000","last_change":"2023-08-27T11:29:23.563095+0000","last_active":"2023-08-27T11:29:25.978504+0000","last_peered":"2023-08-27T11:29:25.978504+0000","last_clean":"2023-08-27T11:29:25.978504+0000","last_became_active":"2023-08-27T11:29:23.562921+0000","last_became_peered":"2023-08-27T11:29:23.562921+0000","last_unstale":"2023-08-27T11:29:25.978504+0000","last_undegraded":"2023-08-27T11:29:25.978504+0000","last_fullsized":"2023-08-27T11:29:25.978504+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:23:16.433248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"166'1","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551920+0000","last_change":"2023-08-27T11:29:22.567394+0000","last_active":"2023-08-27T11:29:24.551920+0000","last_peered":"2023-08-27T11:29:24.551920+0000","last_clean":"2023-08-27T11:29:24.551920+0000","last_became_active":"2023-08-27T11:29:22.566951+0000","last_became_peered":"2023-08-27T11:29:22.566951+0000","last_unstale":"2023-08-27T11:29:24.551920+0000","last_undegraded":"2023-08-27T11:29:24.551920+0000","last_fullsized":"2023-08-27T11:29:24.551920+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:55:21.740217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552974+0000","last_change":"2023-08-27T11:29:23.577363+0000","last_active":"2023-08-27T11:29:24.552974+0000","last_peered":"2023-08-27T11:29:24.552974+0000","last_clean":"2023-08-27T11:29:24.552974+0000","last_became_active":"2023-08-27T11:29:23.576949+0000","last_became_peered":"2023-08-27T11:29:23.576949+0000","last_unstale":"2023-08-27T11:29:24.552974+0000","last_undegraded":"2023-08-27T11:29:24.552974+0000","last_fullsized":"2023-08-27T11:29:24.552974+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:52:46.803848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552131+0000","last_change":"2023-08-27T11:29:22.567951+0000","last_active":"2023-08-27T11:29:24.552131+0000","last_peered":"2023-08-27T11:29:24.552131+0000","last_clean":"2023-08-27T11:29:24.552131+0000","last_became_active":"2023-08-27T11:29:22.567656+0000","last_became_peered":"2023-08-27T11:29:22.567656+0000","last_unstale":"2023-08-27T11:29:24.552131+0000","last_undegraded":"2023-08-27T11:29:24.552131+0000","last_fullsized":"2023-08-27T11:29:24.552131+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:14:30.868414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551721+0000","last_change":"2023-08-27T11:29:23.570168+0000","last_active":"2023-08-27T11:29:24.551721+0000","last_peered":"2023-08-27T11:29:24.551721+0000","last_clean":"2023-08-27T11:29:24.551721+0000","last_became_active":"2023-08-27T11:29:23.569894+0000","last_became_peered":"2023-08-27T11:29:23.569894+0000","last_unstale":"2023-08-27T11:29:24.551721+0000","last_undegraded":"2023-08-27T11:29:24.551721+0000","last_fullsized":"2023-08-27T11:29:24.551721+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:20:03.029838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"166'2","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552815+0000","last_change":"2023-08-27T11:29:22.565006+0000","last_active":"2023-08-27T11:29:24.552815+0000","last_peered":"2023-08-27T11:29:24.552815+0000","last_clean":"2023-08-27T11:29:24.552815+0000","last_became_active":"2023-08-27T11:29:22.564725+0000","last_became_peered":"2023-08-27T11:29:22.564725+0000","last_unstale":"2023-08-27T11:29:24.552815+0000","last_undegraded":"2023-08-27T11:29:24.552815+0000","last_fullsized":"2023-08-27T11:29:24.552815+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:40:58.304229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134571+0000","last_change":"2023-08-27T11:29:23.575183+0000","last_active":"2023-08-27T11:29:25.134571+0000","last_peered":"2023-08-27T11:29:25.134571+0000","last_clean":"2023-08-27T11:29:25.134571+0000","last_became_active":"2023-08-27T11:29:23.574877+0000","last_became_peered":"2023-08-27T11:29:23.574877+0000","last_unstale":"2023-08-27T11:29:25.134571+0000","last_undegraded":"2023-08-27T11:29:25.134571+0000","last_fullsized":"2023-08-27T11:29:25.134571+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:00:00.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548523+0000","last_change":"2023-08-27T11:29:22.567705+0000","last_active":"2023-08-27T11:29:24.548523+0000","last_peered":"2023-08-27T11:29:24.548523+0000","last_clean":"2023-08-27T11:29:24.548523+0000","last_became_active":"2023-08-27T11:29:22.567411+0000","last_became_peered":"2023-08-27T11:29:22.567411+0000","last_unstale":"2023-08-27T11:29:24.548523+0000","last_undegraded":"2023-08-27T11:29:24.548523+0000","last_fullsized":"2023-08-27T11:29:24.548523+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:36:31.122935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977567+0000","last_change":"2023-08-27T11:29:23.566077+0000","last_active":"2023-08-27T11:29:25.977567+0000","last_peered":"2023-08-27T11:29:25.977567+0000","last_clean":"2023-08-27T11:29:25.977567+0000","last_became_active":"2023-08-27T11:29:23.565704+0000","last_became_peered":"2023-08-27T11:29:23.565704+0000","last_unstale":"2023-08-27T11:29:25.977567+0000","last_undegraded":"2023-08-27T11:29:25.977567+0000","last_fullsized":"2023-08-27T11:29:25.977567+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:27:46.166160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548268+0000","last_change":"2023-08-27T11:29:22.557367+0000","last_active":"2023-08-27T11:29:24.548268+0000","last_peered":"2023-08-27T11:29:24.548268+0000","last_clean":"2023-08-27T11:29:24.548268+0000","last_became_active":"2023-08-27T11:29:22.557097+0000","last_became_peered":"2023-08-27T11:29:22.557097+0000","last_unstale":"2023-08-27T11:29:24.548268+0000","last_undegraded":"2023-08-27T11:29:24.548268+0000","last_fullsized":"2023-08-27T11:29:24.548268+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:50:53.719358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978071+0000","last_change":"2023-08-27T11:29:23.571420+0000","last_active":"2023-08-27T11:29:25.978071+0000","last_peered":"2023-08-27T11:29:25.978071+0000","last_clean":"2023-08-27T11:29:25.978071+0000","last_became_active":"2023-08-27T11:29:23.571101+0000","last_became_peered":"2023-08-27T11:29:23.571101+0000","last_unstale":"2023-08-27T11:29:25.978071+0000","last_undegraded":"2023-08-27T11:29:25.978071+0000","last_fullsized":"2023-08-27T11:29:25.978071+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:59:57.307275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477238+0000","last_change":"2023-08-27T11:29:22.556610+0000","last_active":"2023-08-27T11:29:25.477238+0000","last_peered":"2023-08-27T11:29:25.477238+0000","last_clean":"2023-08-27T11:29:25.477238+0000","last_became_active":"2023-08-27T11:29:22.556349+0000","last_became_peered":"2023-08-27T11:29:22.556349+0000","last_unstale":"2023-08-27T11:29:25.477238+0000","last_undegraded":"2023-08-27T11:29:25.477238+0000","last_fullsized":"2023-08-27T11:29:25.477238+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:48:58.865671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977943+0000","last_change":"2023-08-27T11:29:23.567799+0000","last_active":"2023-08-27T11:29:25.977943+0000","last_peered":"2023-08-27T11:29:25.977943+0000","last_clean":"2023-08-27T11:29:25.977943+0000","last_became_active":"2023-08-27T11:29:23.567510+0000","last_became_peered":"2023-08-27T11:29:23.567510+0000","last_unstale":"2023-08-27T11:29:25.977943+0000","last_undegraded":"2023-08-27T11:29:25.977943+0000","last_fullsized":"2023-08-27T11:29:25.977943+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:44:23.684915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"166'2","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552366+0000","last_change":"2023-08-27T11:29:22.559447+0000","last_active":"2023-08-27T11:29:24.552366+0000","last_peered":"2023-08-27T11:29:24.552366+0000","last_clean":"2023-08-27T11:29:24.552366+0000","last_became_active":"2023-08-27T11:29:22.559069+0000","last_became_peered":"2023-08-27T11:29:22.559069+0000","last_unstale":"2023-08-27T11:29:24.552366+0000","last_undegraded":"2023-08-27T11:29:24.552366+0000","last_fullsized":"2023-08-27T11:29:24.552366+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:56:08.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134998+0000","last_change":"2023-08-27T11:29:23.576119+0000","last_active":"2023-08-27T11:29:25.134998+0000","last_peered":"2023-08-27T11:29:25.134998+0000","last_clean":"2023-08-27T11:29:25.134998+0000","last_became_active":"2023-08-27T11:29:23.575825+0000","last_became_peered":"2023-08-27T11:29:23.575825+0000","last_unstale":"2023-08-27T11:29:25.134998+0000","last_undegraded":"2023-08-27T11:29:25.134998+0000","last_fullsized":"2023-08-27T11:29:25.134998+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:35.075986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"167'1","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548724+0000","last_change":"2023-08-27T11:29:22.557095+0000","last_active":"2023-08-27T11:29:24.548724+0000","last_peered":"2023-08-27T11:29:24.548724+0000","last_clean":"2023-08-27T11:29:24.548724+0000","last_became_active":"2023-08-27T11:29:22.556840+0000","last_became_peered":"2023-08-27T11:29:22.556840+0000","last_unstale":"2023-08-27T11:29:24.548724+0000","last_undegraded":"2023-08-27T11:29:24.548724+0000","last_fullsized":"2023-08-27T11:29:24.548724+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:38:06.716876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553686+0000","last_change":"2023-08-27T11:29:23.579985+0000","last_active":"2023-08-27T11:29:24.553686+0000","last_peered":"2023-08-27T11:29:24.553686+0000","last_clean":"2023-08-27T11:29:24.553686+0000","last_became_active":"2023-08-27T11:29:23.579711+0000","last_became_peered":"2023-08-27T11:29:23.579711+0000","last_unstale":"2023-08-27T11:29:24.553686+0000","last_undegraded":"2023-08-27T11:29:24.553686+0000","last_fullsized":"2023-08-27T11:29:24.553686+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:25.316042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548618+0000","last_change":"2023-08-27T11:29:22.559052+0000","last_active":"2023-08-27T11:29:24.548618+0000","last_peered":"2023-08-27T11:29:24.548618+0000","last_clean":"2023-08-27T11:29:24.548618+0000","last_became_active":"2023-08-27T11:29:22.558768+0000","last_became_peered":"2023-08-27T11:29:22.558768+0000","last_unstale":"2023-08-27T11:29:24.548618+0000","last_undegraded":"2023-08-27T11:29:24.548618+0000","last_fullsized":"2023-08-27T11:29:24.548618+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:54.602268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477675+0000","last_change":"2023-08-27T11:29:23.558027+0000","last_active":"2023-08-27T11:29:25.477675+0000","last_peered":"2023-08-27T11:29:25.477675+0000","last_clean":"2023-08-27T11:29:25.477675+0000","last_became_active":"2023-08-27T11:29:23.557746+0000","last_became_peered":"2023-08-27T11:29:23.557746+0000","last_unstale":"2023-08-27T11:29:25.477675+0000","last_undegraded":"2023-08-27T11:29:25.477675+0000","last_fullsized":"2023-08-27T11:29:25.477675+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:22.816411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552272+0000","last_change":"2023-08-27T11:29:22.565139+0000","last_active":"2023-08-27T11:29:24.552272+0000","last_peered":"2023-08-27T11:29:24.552272+0000","last_clean":"2023-08-27T11:29:24.552272+0000","last_became_active":"2023-08-27T11:29:22.564849+0000","last_became_peered":"2023-08-27T11:29:22.564849+0000","last_unstale":"2023-08-27T11:29:24.552272+0000","last_undegraded":"2023-08-27T11:29:24.552272+0000","last_fullsized":"2023-08-27T11:29:24.552272+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:24:24.316772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977852+0000","last_change":"2023-08-27T11:29:23.572206+0000","last_active":"2023-08-27T11:29:25.977852+0000","last_peered":"2023-08-27T11:29:25.977852+0000","last_clean":"2023-08-27T11:29:25.977852+0000","last_became_active":"2023-08-27T11:29:23.571041+0000","last_became_peered":"2023-08-27T11:29:23.571041+0000","last_unstale":"2023-08-27T11:29:25.977852+0000","last_undegraded":"2023-08-27T11:29:25.977852+0000","last_fullsized":"2023-08-27T11:29:25.977852+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:02:39.792746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552837+0000","last_change":"2023-08-27T11:29:22.559902+0000","last_active":"2023-08-27T11:29:24.552837+0000","last_peered":"2023-08-27T11:29:24.552837+0000","last_clean":"2023-08-27T11:29:24.552837+0000","last_became_active":"2023-08-27T11:29:22.559202+0000","last_became_peered":"2023-08-27T11:29:22.559202+0000","last_unstale":"2023-08-27T11:29:24.552837+0000","last_undegraded":"2023-08-27T11:29:24.552837+0000","last_fullsized":"2023-08-27T11:29:24.552837+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:10:12.587313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978128+0000","last_change":"2023-08-27T11:29:23.558180+0000","last_active":"2023-08-27T11:29:25.978128+0000","last_peered":"2023-08-27T11:29:25.978128+0000","last_clean":"2023-08-27T11:29:25.978128+0000","last_became_active":"2023-08-27T11:29:23.557905+0000","last_became_peered":"2023-08-27T11:29:23.557905+0000","last_unstale":"2023-08-27T11:29:25.978128+0000","last_undegraded":"2023-08-27T11:29:25.978128+0000","last_fullsized":"2023-08-27T11:29:25.978128+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:14:41.834602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978675+0000","last_change":"2023-08-27T11:29:22.562609+0000","last_active":"2023-08-27T11:29:25.978675+0000","last_peered":"2023-08-27T11:29:25.978675+0000","last_clean":"2023-08-27T11:29:25.978675+0000","last_became_active":"2023-08-27T11:29:22.562290+0000","last_became_peered":"2023-08-27T11:29:22.562290+0000","last_unstale":"2023-08-27T11:29:25.978675+0000","last_undegraded":"2023-08-27T11:29:25.978675+0000","last_fullsized":"2023-08-27T11:29:25.978675+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:06:30.698615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477524+0000","last_change":"2023-08-27T11:29:23.557077+0000","last_active":"2023-08-27T11:29:25.477524+0000","last_peered":"2023-08-27T11:29:25.477524+0000","last_clean":"2023-08-27T11:29:25.477524+0000","last_became_active":"2023-08-27T11:29:23.556809+0000","last_became_peered":"2023-08-27T11:29:23.556809+0000","last_unstale":"2023-08-27T11:29:25.477524+0000","last_undegraded":"2023-08-27T11:29:25.477524+0000","last_fullsized":"2023-08-27T11:29:25.477524+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:47:47.795537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134582+0000","last_change":"2023-08-27T11:29:22.561794+0000","last_active":"2023-08-27T11:29:25.134582+0000","last_peered":"2023-08-27T11:29:25.134582+0000","last_clean":"2023-08-27T11:29:25.134582+0000","last_became_active":"2023-08-27T11:29:22.561350+0000","last_became_peered":"2023-08-27T11:29:22.561350+0000","last_unstale":"2023-08-27T11:29:25.134582+0000","last_undegraded":"2023-08-27T11:29:25.134582+0000","last_fullsized":"2023-08-27T11:29:25.134582+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:37.246059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552646+0000","last_change":"2023-08-27T11:29:23.577215+0000","last_active":"2023-08-27T11:29:24.552646+0000","last_peered":"2023-08-27T11:29:24.552646+0000","last_clean":"2023-08-27T11:29:24.552646+0000","last_became_active":"2023-08-27T11:29:23.576584+0000","last_became_peered":"2023-08-27T11:29:23.576584+0000","last_unstale":"2023-08-27T11:29:24.552646+0000","last_undegraded":"2023-08-27T11:29:24.552646+0000","last_fullsized":"2023-08-27T11:29:24.552646+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:55:40.017114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551830+0000","last_change":"2023-08-27T11:29:22.557686+0000","last_active":"2023-08-27T11:29:24.551830+0000","last_peered":"2023-08-27T11:29:24.551830+0000","last_clean":"2023-08-27T11:29:24.551830+0000","last_became_active":"2023-08-27T11:29:22.557312+0000","last_became_peered":"2023-08-27T11:29:22.557312+0000","last_unstale":"2023-08-27T11:29:24.551830+0000","last_undegraded":"2023-08-27T11:29:24.551830+0000","last_fullsized":"2023-08-27T11:29:24.551830+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:06:49.976537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977719+0000","last_change":"2023-08-27T11:29:23.567811+0000","last_active":"2023-08-27T11:29:25.977719+0000","last_peered":"2023-08-27T11:29:25.977719+0000","last_clean":"2023-08-27T11:29:25.977719+0000","last_became_active":"2023-08-27T11:29:23.567559+0000","last_became_peered":"2023-08-27T11:29:23.567559+0000","last_unstale":"2023-08-27T11:29:25.977719+0000","last_undegraded":"2023-08-27T11:29:25.977719+0000","last_fullsized":"2023-08-27T11:29:25.977719+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:25:13.211321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"166'1","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548343+0000","last_change":"2023-08-27T11:29:22.560833+0000","last_active":"2023-08-27T11:29:24.548343+0000","last_peered":"2023-08-27T11:29:24.548343+0000","last_clean":"2023-08-27T11:29:24.548343+0000","last_became_active":"2023-08-27T11:29:22.560555+0000","last_became_peered":"2023-08-27T11:29:22.560555+0000","last_unstale":"2023-08-27T11:29:24.548343+0000","last_undegraded":"2023-08-27T11:29:24.548343+0000","last_fullsized":"2023-08-27T11:29:24.548343+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:43:12.351160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":318,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552500+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:29:24.552500+0000","last_peered":"2023-08-27T11:29:24.552500+0000","last_clean":"2023-08-27T11:29:24.552500+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:29:24.552500+0000","last_undegraded":"2023-08-27T11:29:24.552500+0000","last_fullsized":"2023-08-27T11:29:24.552500+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:45:18.352414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552269+0000","last_change":"2023-08-27T11:29:23.565347+0000","last_active":"2023-08-27T11:29:24.552269+0000","last_peered":"2023-08-27T11:29:24.552269+0000","last_clean":"2023-08-27T11:29:24.552269+0000","last_became_active":"2023-08-27T11:29:23.564270+0000","last_became_peered":"2023-08-27T11:29:23.564270+0000","last_unstale":"2023-08-27T11:29:24.552269+0000","last_undegraded":"2023-08-27T11:29:24.552269+0000","last_fullsized":"2023-08-27T11:29:24.552269+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:52:37.724239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"166'1","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134715+0000","last_change":"2023-08-27T11:29:22.560916+0000","last_active":"2023-08-27T11:29:25.134715+0000","last_peered":"2023-08-27T11:29:25.134715+0000","last_clean":"2023-08-27T11:29:25.134715+0000","last_became_active":"2023-08-27T11:29:22.560605+0000","last_became_peered":"2023-08-27T11:29:22.560605+0000","last_unstale":"2023-08-27T11:29:25.134715+0000","last_undegraded":"2023-08-27T11:29:25.134715+0000","last_fullsized":"2023-08-27T11:29:25.134715+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:25:46.102892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":318,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552399+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:29:24.552399+0000","last_peered":"2023-08-27T11:29:24.552399+0000","last_clean":"2023-08-27T11:29:24.552399+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:29:24.552399+0000","last_undegraded":"2023-08-27T11:29:24.552399+0000","last_fullsized":"2023-08-27T11:29:24.552399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:51:58.956267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552053+0000","last_change":"2023-08-27T11:29:23.574637+0000","last_active":"2023-08-27T11:29:24.552053+0000","last_peered":"2023-08-27T11:29:24.552053+0000","last_clean":"2023-08-27T11:29:24.552053+0000","last_became_active":"2023-08-27T11:29:23.573019+0000","last_became_peered":"2023-08-27T11:29:23.573019+0000","last_unstale":"2023-08-27T11:29:24.552053+0000","last_undegraded":"2023-08-27T11:29:24.552053+0000","last_fullsized":"2023-08-27T11:29:24.552053+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:47:58.802669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"166'2","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553735+0000","last_change":"2023-08-27T11:29:22.564661+0000","last_active":"2023-08-27T11:29:24.553735+0000","last_peered":"2023-08-27T11:29:24.553735+0000","last_clean":"2023-08-27T11:29:24.553735+0000","last_became_active":"2023-08-27T11:29:22.564370+0000","last_became_peered":"2023-08-27T11:29:22.564370+0000","last_unstale":"2023-08-27T11:29:24.553735+0000","last_undegraded":"2023-08-27T11:29:24.553735+0000","last_fullsized":"2023-08-27T11:29:24.553735+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:33:45.954366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":272,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553392+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:29:24.553392+0000","last_peered":"2023-08-27T11:29:24.553392+0000","last_clean":"2023-08-27T11:29:24.553392+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:29:24.553392+0000","last_undegraded":"2023-08-27T11:29:24.553392+0000","last_fullsized":"2023-08-27T11:29:24.553392+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:17:42.278588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978133+0000","last_change":"2023-08-27T11:29:23.565144+0000","last_active":"2023-08-27T11:29:25.978133+0000","last_peered":"2023-08-27T11:29:25.978133+0000","last_clean":"2023-08-27T11:29:25.978133+0000","last_became_active":"2023-08-27T11:29:23.564519+0000","last_became_peered":"2023-08-27T11:29:23.564519+0000","last_unstale":"2023-08-27T11:29:25.978133+0000","last_undegraded":"2023-08-27T11:29:25.978133+0000","last_fullsized":"2023-08-27T11:29:25.978133+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:46:49.330993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"166'2","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134724+0000","last_change":"2023-08-27T11:29:22.562502+0000","last_active":"2023-08-27T11:29:25.134724+0000","last_peered":"2023-08-27T11:29:25.134724+0000","last_clean":"2023-08-27T11:29:25.134724+0000","last_became_active":"2023-08-27T11:29:22.562213+0000","last_became_peered":"2023-08-27T11:29:22.562213+0000","last_unstale":"2023-08-27T11:29:25.134724+0000","last_undegraded":"2023-08-27T11:29:25.134724+0000","last_fullsized":"2023-08-27T11:29:25.134724+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:14.795810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":269,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134584+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:29:25.134584+0000","last_peered":"2023-08-27T11:29:25.134584+0000","last_clean":"2023-08-27T11:29:25.134584+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:29:25.134584+0000","last_undegraded":"2023-08-27T11:29:25.134584+0000","last_fullsized":"2023-08-27T11:29:25.134584+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:08:45.075087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978235+0000","last_change":"2023-08-27T11:29:23.565075+0000","last_active":"2023-08-27T11:29:25.978235+0000","last_peered":"2023-08-27T11:29:25.978235+0000","last_clean":"2023-08-27T11:29:25.978235+0000","last_became_active":"2023-08-27T11:29:23.564808+0000","last_became_peered":"2023-08-27T11:29:23.564808+0000","last_unstale":"2023-08-27T11:29:25.978235+0000","last_undegraded":"2023-08-27T11:29:25.978235+0000","last_fullsized":"2023-08-27T11:29:25.978235+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:56:47.175672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"166'1","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552246+0000","last_change":"2023-08-27T11:29:22.565149+0000","last_active":"2023-08-27T11:29:24.552246+0000","last_peered":"2023-08-27T11:29:24.552246+0000","last_clean":"2023-08-27T11:29:24.552246+0000","last_became_active":"2023-08-27T11:29:22.564666+0000","last_became_peered":"2023-08-27T11:29:22.564666+0000","last_unstale":"2023-08-27T11:29:24.552246+0000","last_undegraded":"2023-08-27T11:29:24.552246+0000","last_fullsized":"2023-08-27T11:29:24.552246+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:31.502049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":268,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134728+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:29:25.134728+0000","last_peered":"2023-08-27T11:29:25.134728+0000","last_clean":"2023-08-27T11:29:25.134728+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:29:25.134728+0000","last_undegraded":"2023-08-27T11:29:25.134728+0000","last_fullsized":"2023-08-27T11:29:25.134728+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:37:36.873832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978212+0000","last_change":"2023-08-27T11:29:23.569098+0000","last_active":"2023-08-27T11:29:25.978212+0000","last_peered":"2023-08-27T11:29:25.978212+0000","last_clean":"2023-08-27T11:29:25.978212+0000","last_became_active":"2023-08-27T11:29:23.568826+0000","last_became_peered":"2023-08-27T11:29:23.568826+0000","last_unstale":"2023-08-27T11:29:25.978212+0000","last_undegraded":"2023-08-27T11:29:25.978212+0000","last_fullsized":"2023-08-27T11:29:25.978212+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:11:58.076108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"166'4","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477685+0000","last_change":"2023-08-27T11:29:22.565482+0000","last_active":"2023-08-27T11:29:25.477685+0000","last_peered":"2023-08-27T11:29:25.477685+0000","last_clean":"2023-08-27T11:29:25.477685+0000","last_became_active":"2023-08-27T11:29:22.565109+0000","last_became_peered":"2023-08-27T11:29:22.565109+0000","last_unstale":"2023-08-27T11:29:25.477685+0000","last_undegraded":"2023-08-27T11:29:25.477685+0000","last_fullsized":"2023-08-27T11:29:25.477685+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:53:54.762552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":419,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134524+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:29:25.134524+0000","last_peered":"2023-08-27T11:29:25.134524+0000","last_clean":"2023-08-27T11:29:25.134524+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:29:25.134524+0000","last_undegraded":"2023-08-27T11:29:25.134524+0000","last_fullsized":"2023-08-27T11:29:25.134524+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:17.130505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":278,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548413+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:29:24.548413+0000","last_peered":"2023-08-27T11:29:24.548413+0000","last_clean":"2023-08-27T11:29:24.548413+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:29:24.548413+0000","last_undegraded":"2023-08-27T11:29:24.548413+0000","last_fullsized":"2023-08-27T11:29:24.548413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:45.832462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477591+0000","last_change":"2023-08-27T11:29:23.568490+0000","last_active":"2023-08-27T11:29:25.477591+0000","last_peered":"2023-08-27T11:29:25.477591+0000","last_clean":"2023-08-27T11:29:25.477591+0000","last_became_active":"2023-08-27T11:29:23.568217+0000","last_became_peered":"2023-08-27T11:29:23.568217+0000","last_unstale":"2023-08-27T11:29:25.477591+0000","last_undegraded":"2023-08-27T11:29:25.477591+0000","last_fullsized":"2023-08-27T11:29:25.477591+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:20.409859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977838+0000","last_change":"2023-08-27T11:29:22.556485+0000","last_active":"2023-08-27T11:29:25.977838+0000","last_peered":"2023-08-27T11:29:25.977838+0000","last_clean":"2023-08-27T11:29:25.977838+0000","last_became_active":"2023-08-27T11:29:22.555930+0000","last_became_peered":"2023-08-27T11:29:22.555930+0000","last_unstale":"2023-08-27T11:29:25.977838+0000","last_undegraded":"2023-08-27T11:29:25.977838+0000","last_fullsized":"2023-08-27T11:29:25.977838+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:17:45.141474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":270,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978275+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:29:25.978275+0000","last_peered":"2023-08-27T11:29:25.978275+0000","last_clean":"2023-08-27T11:29:25.978275+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:29:25.978275+0000","last_undegraded":"2023-08-27T11:29:25.978275+0000","last_fullsized":"2023-08-27T11:29:25.978275+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:03:27.587573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548529+0000","last_change":"2023-08-27T11:29:23.568812+0000","last_active":"2023-08-27T11:29:24.548529+0000","last_peered":"2023-08-27T11:29:24.548529+0000","last_clean":"2023-08-27T11:29:24.548529+0000","last_became_active":"2023-08-27T11:29:23.568324+0000","last_became_peered":"2023-08-27T11:29:23.568324+0000","last_unstale":"2023-08-27T11:29:24.548529+0000","last_undegraded":"2023-08-27T11:29:24.548529+0000","last_fullsized":"2023-08-27T11:29:24.548529+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:53.650499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978638+0000","last_change":"2023-08-27T11:29:22.558632+0000","last_active":"2023-08-27T11:29:25.978638+0000","last_peered":"2023-08-27T11:29:25.978638+0000","last_clean":"2023-08-27T11:29:25.978638+0000","last_became_active":"2023-08-27T11:29:22.558040+0000","last_became_peered":"2023-08-27T11:29:22.558040+0000","last_unstale":"2023-08-27T11:29:25.978638+0000","last_undegraded":"2023-08-27T11:29:25.978638+0000","last_fullsized":"2023-08-27T11:29:25.978638+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:59:15.707841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":304,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.979016+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:29:25.979016+0000","last_peered":"2023-08-27T11:29:25.979016+0000","last_clean":"2023-08-27T11:29:25.979016+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:29:25.979016+0000","last_undegraded":"2023-08-27T11:29:25.979016+0000","last_fullsized":"2023-08-27T11:29:25.979016+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:31:38.716999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575002,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33836,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338004,"statfs":{"total":96636764160,"available":96602116096,"internally_reserved":0,"allocated":2719744,"data_stored":1599713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.22700000000000001},"max":{"1min":0.60399999999999998,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.51500000000000001},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.56200000000000006}]},{"osd":1,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.83399999999999996,"15min":0.872},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.50900000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.42799999999999999,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.67800000000000005,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.90200000000000002},"last":0.61299999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":0.996,"15min":0.996},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.53000000000000003,"5min":0.42899999999999999,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.68500000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.503},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.72199999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":1.224,"15min":1.224},"last":0.752}]},{"osd":5,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79700000000000004,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.84799999999999998},"last":0.67400000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83799999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.71099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575003,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33272,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338568,"statfs":{"total":96636764160,"available":96602693632,"internally_reserved":0,"allocated":2142208,"data_stored":1010411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.84599999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.39800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.89800000000000002},"last":0.46100000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.53900000000000003},"min":{"1min":0.35599999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.81799999999999995,"15min":0.873},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.499,"15min":0.51000000000000001},"min":{"1min":0.40799999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.503},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":2.0830000000000002},"last":0.48499999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.83199999999999996,"15min":0.84499999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.34000000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.93799999999999994},"last":0.56799999999999995}]},{"osd":4,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":0.71599999999999997,"5min":0.753,"15min":0.83099999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.627}]},{"osd":5,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.436},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75700000000000001,"5min":0.85499999999999998,"15min":0.875},"last":0.65800000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79200000000000004,"5min":1.365,"15min":1.365},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.34899999999999998},"max":{"1min":0.77700000000000002,"5min":0.82999999999999996,"15min":0.91400000000000003},"last":0.59599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33828,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338012,"statfs":{"total":96636764160,"available":96602124288,"internally_reserved":0,"allocated":2711552,"data_stored":1599041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.315,"15min":0.47399999999999998},"min":{"1min":0.21199999999999999,"5min":0.19500000000000001,"15min":0.191},"max":{"1min":0.45300000000000001,"5min":0.53300000000000003,"15min":11.971},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.315,"5min":0.32700000000000001,"15min":0.64700000000000002},"min":{"1min":0.23000000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.44900000000000001,"5min":0.74399999999999999,"15min":23.045999999999999},"last":0.25600000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34000000000000002,"15min":0.502},"min":{"1min":0.28199999999999997,"5min":0.23300000000000001,"15min":0.221},"max":{"1min":0.42399999999999999,"5min":0.57699999999999996,"15min":11.928000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.33700000000000002},"min":{"1min":0.249,"5min":0.22800000000000001,"15min":0.224},"max":{"1min":0.46899999999999997,"5min":0.52200000000000002,"15min":0.58599999999999997},"last":0.38500000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38400000000000001,"15min":0.52800000000000002},"min":{"1min":0.309,"5min":0.27500000000000002,"15min":0.26300000000000001},"max":{"1min":0.47599999999999998,"5min":0.64100000000000001,"15min":10.99},"last":0.439},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36799999999999999,"15min":0.68500000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.503,"5min":0.54700000000000004,"15min":22.893999999999998},"last":0.34300000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.33400000000000002},"max":{"1min":0.84999999999999998,"5min":0.90700000000000003,"15min":1.26},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58099999999999996,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.745,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.88600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":23.016999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.81899999999999995,"5min":0.88700000000000001,"15min":1.296},"last":0.498}]},{"osd":6,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58199999999999996,"15min":0.72799999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.371},"max":{"1min":0.72999999999999998,"5min":0.84899999999999998,"15min":10.925000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60199999999999998,"15min":0.90400000000000003},"min":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88,"5min":0.88,"15min":22.952000000000002},"last":0.45300000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59299999999999997,"15min":0.89800000000000002},"min":{"1min":0.433,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":22.988},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":1.327},"last":0.54700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575002,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2113536,"data_stored":1007312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.215,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.377},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.497,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36199999999999999,"15min":0.34399999999999997},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47699999999999998,"5min":0.628,"15min":0.63200000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34799999999999998,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45800000000000002,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.40600000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35799999999999998,"15min":0.34599999999999997},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.53700000000000003,"15min":0.59599999999999997},"last":0.44},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.376},"min":{"1min":0.30499999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.53000000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.42299999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.34399999999999997},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.214},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.66900000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.603},"last":0.59499999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79600000000000004,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.878,"15min":0.878},"last":0.68300000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83399999999999996,"5min":3.79,"15min":3.79},"last":0.70599999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575003,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33256,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2125824,"data_stored":1009289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32100000000000001,"15min":0.318},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.19700000000000001},"max":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.629},"last":0.46500000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41699999999999998,"5min":0.50800000000000001,"15min":0.54400000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34699999999999998},"min":{"1min":0.26200000000000001,"5min":0.23000000000000001,"15min":0.22600000000000001},"max":{"1min":0.49299999999999999,"5min":0.56399999999999995,"15min":0.749},"last":0.44400000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.378},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":0.51800000000000002,"5min":0.57099999999999995,"15min":0.73699999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.32600000000000001,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.55100000000000005,"15min":0.69499999999999995},"last":0.50600000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.879},"last":0.89500000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36699999999999999},"max":{"1min":0.83799999999999997,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.64900000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.40000000000000002},"max":{"1min":0.78600000000000003,"5min":0.82699999999999996,"15min":0.83599999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.73499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2121728,"data_stored":1008695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.29999999999999999,"15min":0.30299999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.52500000000000002},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.307,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.223,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40899999999999997,"5min":0.755,"15min":0.755},"last":0.29399999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33500000000000002,"15min":0.32300000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":0.40300000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":0.48999999999999999,"5min":2.73,"15min":2.73},"last":0.32200000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.372,"15min":0.36399999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.46400000000000002,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.375,"15min":0.37},"min":{"1min":0.27700000000000002,"5min":0.26700000000000002,"15min":0.26400000000000001},"max":{"1min":0.51000000000000001,"5min":1.179,"15min":1.179},"last":0.36799999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.28599999999999998},"max":{"1min":0.748,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.71799999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52400000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.57699999999999996},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.373},"max":{"1min":0.84299999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.498},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.497,"5min":0.38,"15min":0.38},"max":{"1min":0.83199999999999996,"5min":0.86899999999999999,"15min":1.111},"last":0.46899999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":1.236},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.44800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575003,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2121728,"data_stored":1008763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68200000000000005,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.502},"min":{"1min":0.33900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51500000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66100000000000003,"5min":0.746,"15min":0.76500000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.75,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.53800000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.40400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.40699999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.96999999999999997,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.73099999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71499999999999997,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":1.865,"15min":1.865},"last":0.65600000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90600000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.624,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82799999999999996,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.61099999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.90100000000000002,"5min":2.714,"15min":2.714},"last":0.625},{"interface":"front","average":{"1min":0.63,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.78700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575002,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33264,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338576,"statfs":{"total":96636764160,"available":96602701824,"internally_reserved":0,"allocated":2134016,"data_stored":1010355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.376,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61099999999999999,"5min":0.78900000000000003,"15min":0.96899999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.44700000000000001,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.48899999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67500000000000004,"5min":0.68400000000000005,"15min":1.075},"last":0.61099999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.373,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.0580000000000001},"last":0.52100000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":1.03},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.42199999999999999,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.77800000000000002,"15min":1.026},"last":0.57199999999999995}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.27000000000000002},"max":{"1min":0.63900000000000001,"5min":0.78700000000000003,"15min":0.86899999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.73399999999999999,"5min":0.89400000000000002,"15min":0.91400000000000003},"last":0.63800000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.31},"max":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.88200000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.873,"15min":0.94199999999999995},"last":0.64700000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.34799999999999998},"max":{"1min":0.753,"5min":0.85999999999999999,"15min":0.95899999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.84399999999999997,"15min":0.92000000000000004},"last":0.627}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:29:32.245 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:29:32.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.377+0000 7fb756e65700 1 Processor -- start 2023-08-27T11:29:32.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.377+0000 7fb756e65700 1 -- start start 2023-08-27T11:29:32.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.378+0000 7fb756e65700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb750144570 0x7fb75015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.378+0000 7fb756e65700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b4ad0 0x7fb7500b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.378+0000 7fb756e65700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7500b45f0 con 0x7fb7500b3e50 2023-08-27T11:29:32.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.378+0000 7fb756e65700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7500cf7a0 con 0x7fb750144570 2023-08-27T11:29:32.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.378+0000 7fb756e65700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb750144950 con 0x7fb7500b4ad0 2023-08-27T11:29:32.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.378+0000 7fb74ffff700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb7500b3e50 0x7fb75015f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:40124/0 (socket says 172.21.15.112:40124) 2023-08-27T11:29:32.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.378+0000 7fb74ffff700 1 -- 172.21.15.112:0/3044269902 learned_addr learned my addr 172.21.15.112:0/3044269902 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:32.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.378+0000 7fb755402700 1 --2- 172.21.15.112:0/3044269902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b4ad0 0x7fb7500b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.378+0000 7fb754c01700 1 --2- 172.21.15.112:0/3044269902 >> v2:172.21.15.138:3300/0 conn(0x7fb750144570 0x7fb75015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.379+0000 7fb74f7fe700 1 -- 172.21.15.112:0/3044269902 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978380905 0 0) 0x7fb7500b45f0 con 0x7fb7500b3e50 2023-08-27T11:29:32.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.379+0000 7fb74f7fe700 1 -- 172.21.15.112:0/3044269902 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7400035b0 con 0x7fb7500b3e50 2023-08-27T11:29:32.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.379+0000 7fb754c01700 1 -- 172.21.15.112:0/3044269902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b4ad0 msgr2=0x7fb7500b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:32.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.379+0000 7fb754c01700 1 --2- 172.21.15.112:0/3044269902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b4ad0 0x7fb7500b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.379+0000 7fb754c01700 1 -- 172.21.15.112:0/3044269902 >> v1:172.21.15.138:6789/0 conn(0x7fb7500b3e50 legacy=0x7fb75015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:32.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.379+0000 7fb754c01700 1 -- 172.21.15.112:0/3044269902 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7500b4310 con 0x7fb750144570 2023-08-27T11:29:32.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.379+0000 7fb755402700 1 --2- 172.21.15.112:0/3044269902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b4ad0 0x7fb7500b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:32.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.379+0000 7fb754c01700 1 --2- 172.21.15.112:0/3044269902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb750144570 0x7fb75015ee20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb744009870 tx=0x7fb744009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5557c901973d3d12 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:32.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.379+0000 7fb74f7fe700 1 -- 172.21.15.112:0/3044269902 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744011830 con 0x7fb750144570 2023-08-27T11:29:32.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.379+0000 7fb74f7fe700 1 -- 172.21.15.112:0/3044269902 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb744011990 con 0x7fb750144570 2023-08-27T11:29:32.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.379+0000 7fb74f7fe700 1 -- 172.21.15.112:0/3044269902 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744011d00 con 0x7fb750144570 2023-08-27T11:29:32.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.380+0000 7fb756e65700 1 -- 172.21.15.112:0/3044269902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb750144570 msgr2=0x7fb75015ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:32.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.380+0000 7fb756e65700 1 --2- 172.21.15.112:0/3044269902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb750144570 0x7fb75015ee20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb744009870 tx=0x7fb744009b80 comp rx=0 tx=0).stop 2023-08-27T11:29:32.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.380+0000 7fb756e65700 1 -- 172.21.15.112:0/3044269902 shutdown_connections 2023-08-27T11:29:32.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.380+0000 7fb756e65700 1 --2- 172.21.15.112:0/3044269902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b4ad0 0x7fb7500b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.380+0000 7fb756e65700 1 --2- 172.21.15.112:0/3044269902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb750144570 0x7fb75015ee20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.380+0000 7fb756e65700 1 -- 172.21.15.112:0/3044269902 >> 172.21.15.112:0/3044269902 conn(0x7fb75006f970 msgr2=0x7fb75006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:32.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.380+0000 7fb756e65700 1 -- 172.21.15.112:0/3044269902 shutdown_connections 2023-08-27T11:29:32.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.380+0000 7fb756e65700 1 -- 172.21.15.112:0/3044269902 wait complete. 2023-08-27T11:29:32.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.381+0000 7fb756e65700 1 Processor -- start 2023-08-27T11:29:32.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.381+0000 7fb756e65700 1 -- start start 2023-08-27T11:29:32.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.382+0000 7fb756e65700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7500b3e50 0x7fb7500cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.382+0000 7fb756e65700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7500b4ad0 0x7fb750159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.382+0000 7fb756e65700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb750144570 0x7fb7500c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.382+0000 7fb756e65700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75013a270 con 0x7fb7500b4ad0 2023-08-27T11:29:32.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.382+0000 7fb756e65700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75015a950 con 0x7fb7500b3e50 2023-08-27T11:29:32.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.382+0000 7fb756e65700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7501598b0 con 0x7fb750144570 2023-08-27T11:29:32.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.382+0000 7fb74ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7500b4ad0 0x7fb750159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.382+0000 7fb74ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7500b4ad0 0x7fb750159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38224/0 (socket says 172.21.15.112:38224) 2023-08-27T11:29:32.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.382+0000 7fb74ffff700 1 -- 172.21.15.112:0/3421583574 learned_addr learned my addr 172.21.15.112:0/3421583574 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:32.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.382+0000 7fb754c01700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7500b3e50 0x7fb7500cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.383+0000 7fb755402700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb750144570 0x7fb7500c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.383+0000 7fb74ffff700 1 -- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb750144570 msgr2=0x7fb7500c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:32.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.383+0000 7fb74ffff700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb750144570 0x7fb7500c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.383+0000 7fb74ffff700 1 -- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7500b3e50 msgr2=0x7fb7500cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:32.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.383+0000 7fb74ffff700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7500b3e50 0x7fb7500cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.383+0000 7fb74ffff700 1 -- 172.21.15.112:0/3421583574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb748009a70 con 0x7fb7500b4ad0 2023-08-27T11:29:32.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.383+0000 7fb755402700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb750144570 0x7fb7500c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:32.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.383+0000 7fb74ffff700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7500b4ad0 0x7fb750159020 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00c420 tx=0x7fb73c00c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:32.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.383+0000 7fb74d7fa700 1 -- 172.21.15.112:0/3421583574 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c005d70 con 0x7fb7500b4ad0 2023-08-27T11:29:32.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.383+0000 7fb74d7fa700 1 -- 172.21.15.112:0/3421583574 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c0070d0 con 0x7fb7500b4ad0 2023-08-27T11:29:32.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.383+0000 7fb756e65700 1 -- 172.21.15.112:0/3421583574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7440094a0 con 0x7fb7500b4ad0 2023-08-27T11:29:32.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.383+0000 7fb74d7fa700 1 -- 172.21.15.112:0/3421583574 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c007440 con 0x7fb7500b4ad0 2023-08-27T11:29:32.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.383+0000 7fb756e65700 1 -- 172.21.15.112:0/3421583574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb750159e70 con 0x7fb7500b4ad0 2023-08-27T11:29:32.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.384+0000 7fb756e65700 1 -- 172.21.15.112:0/3421583574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7500cdf70 con 0x7fb7500b4ad0 2023-08-27T11:29:32.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.385+0000 7fb74d7fa700 1 -- 172.21.15.112:0/3421583574 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fb73c0075a0 con 0x7fb7500b4ad0 2023-08-27T11:29:32.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.385+0000 7fb74d7fa700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb73406a4d0 0x7fb73406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.385+0000 7fb74d7fa700 1 -- 172.21.15.112:0/3421583574 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb73c087f40 con 0x7fb7500b4ad0 2023-08-27T11:29:32.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.385+0000 7fb754c01700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb73406a4d0 0x7fb73406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.386+0000 7fb754c01700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb73406a4d0 0x7fb73406c990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb7440097c0 tx=0x7fb744008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.389+0000 7fb74d7fa700 1 -- 172.21.15.112:0/3421583574 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb7500cdf70 con 0x7fb7500b4ad0 2023-08-27T11:29:32.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.574+0000 7fb756e65700 1 -- 172.21.15.112:0/3421583574 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7500cdcb0 con 0x7fb73406a4d0 2023-08-27T11:29:32.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.583+0000 7fb74d7fa700 1 -- 172.21.15.112:0/3421583574 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311916 (secure 0 0 0) 0x7fb744011cd0 con 0x7fb73406a4d0 2023-08-27T11:29:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:32.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.586+0000 7fb756e65700 1 -- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb73406a4d0 msgr2=0x7fb73406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:32.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.586+0000 7fb756e65700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb73406a4d0 0x7fb73406c990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb7440097c0 tx=0x7fb744008030 comp rx=0 tx=0).stop 2023-08-27T11:29:32.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.586+0000 7fb756e65700 1 -- 172.21.15.112:0/3421583574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7500b4ad0 msgr2=0x7fb750159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:32.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.586+0000 7fb756e65700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7500b4ad0 0x7fb750159020 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00c420 tx=0x7fb73c00c730 comp rx=0 tx=0).stop 2023-08-27T11:29:32.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.587+0000 7fb756e65700 1 -- 172.21.15.112:0/3421583574 shutdown_connections 2023-08-27T11:29:32.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.587+0000 7fb756e65700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fb73406a4d0 0x7fb73406c990 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.587+0000 7fb756e65700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb750144570 0x7fb7500c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.587+0000 7fb756e65700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7500b3e50 0x7fb7500cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.587+0000 7fb756e65700 1 --2- 172.21.15.112:0/3421583574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7500b4ad0 0x7fb750159020 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.587+0000 7fb756e65700 1 -- 172.21.15.112:0/3421583574 >> 172.21.15.112:0/3421583574 conn(0x7fb75006f970 msgr2=0x7fb750141e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.587+0000 7fb756e65700 1 -- 172.21.15.112:0/3421583574 shutdown_connections 2023-08-27T11:29:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.587+0000 7fb756e65700 1 -- 172.21.15.112:0/3421583574 wait complete. 2023-08-27T11:29:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:29:32.600 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":538,"stamp":"2023-08-27T11:29:32.417582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267204,"kb_used_data":17764,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754707516,"statfs":{"total":773094113280,"available":772820496384,"internally_reserved":0,"allocated":18190336,"data_stored":9253579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.866502"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134944+0000","last_change":"2023-08-27T11:29:23.571985+0000","last_active":"2023-08-27T11:29:25.134944+0000","last_peered":"2023-08-27T11:29:25.134944+0000","last_clean":"2023-08-27T11:29:25.134944+0000","last_became_active":"2023-08-27T11:29:23.571737+0000","last_became_peered":"2023-08-27T11:29:23.571737+0000","last_unstale":"2023-08-27T11:29:25.134944+0000","last_undegraded":"2023-08-27T11:29:25.134944+0000","last_fullsized":"2023-08-27T11:29:25.134944+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:21.305005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978202+0000","last_change":"2023-08-27T11:29:23.571830+0000","last_active":"2023-08-27T11:29:25.978202+0000","last_peered":"2023-08-27T11:29:25.978202+0000","last_clean":"2023-08-27T11:29:25.978202+0000","last_became_active":"2023-08-27T11:29:23.571566+0000","last_became_peered":"2023-08-27T11:29:23.571566+0000","last_unstale":"2023-08-27T11:29:25.978202+0000","last_undegraded":"2023-08-27T11:29:25.978202+0000","last_fullsized":"2023-08-27T11:29:25.978202+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:02:34.211624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978208+0000","last_change":"2023-08-27T11:29:23.561965+0000","last_active":"2023-08-27T11:29:25.978208+0000","last_peered":"2023-08-27T11:29:25.978208+0000","last_clean":"2023-08-27T11:29:25.978208+0000","last_became_active":"2023-08-27T11:29:23.561687+0000","last_became_peered":"2023-08-27T11:29:23.561687+0000","last_unstale":"2023-08-27T11:29:25.978208+0000","last_undegraded":"2023-08-27T11:29:25.978208+0000","last_fullsized":"2023-08-27T11:29:25.978208+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:00:12.715079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978740+0000","last_change":"2023-08-27T11:29:23.570451+0000","last_active":"2023-08-27T11:29:25.978740+0000","last_peered":"2023-08-27T11:29:25.978740+0000","last_clean":"2023-08-27T11:29:25.978740+0000","last_became_active":"2023-08-27T11:29:23.570094+0000","last_became_peered":"2023-08-27T11:29:23.570094+0000","last_unstale":"2023-08-27T11:29:25.978740+0000","last_undegraded":"2023-08-27T11:29:25.978740+0000","last_fullsized":"2023-08-27T11:29:25.978740+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:13:37.637896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134822+0000","last_change":"2023-08-27T11:29:23.569584+0000","last_active":"2023-08-27T11:29:25.134822+0000","last_peered":"2023-08-27T11:29:25.134822+0000","last_clean":"2023-08-27T11:29:25.134822+0000","last_became_active":"2023-08-27T11:29:23.569288+0000","last_became_peered":"2023-08-27T11:29:23.569288+0000","last_unstale":"2023-08-27T11:29:25.134822+0000","last_undegraded":"2023-08-27T11:29:25.134822+0000","last_fullsized":"2023-08-27T11:29:25.134822+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:38:57.352131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553562+0000","last_change":"2023-08-27T11:29:23.573122+0000","last_active":"2023-08-27T11:29:24.553562+0000","last_peered":"2023-08-27T11:29:24.553562+0000","last_clean":"2023-08-27T11:29:24.553562+0000","last_became_active":"2023-08-27T11:29:23.572836+0000","last_became_peered":"2023-08-27T11:29:23.572836+0000","last_unstale":"2023-08-27T11:29:24.553562+0000","last_undegraded":"2023-08-27T11:29:24.553562+0000","last_fullsized":"2023-08-27T11:29:24.553562+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:49:20.104822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977858+0000","last_change":"2023-08-27T11:29:23.564383+0000","last_active":"2023-08-27T11:29:25.977858+0000","last_peered":"2023-08-27T11:29:25.977858+0000","last_clean":"2023-08-27T11:29:25.977858+0000","last_became_active":"2023-08-27T11:29:23.564129+0000","last_became_peered":"2023-08-27T11:29:23.564129+0000","last_unstale":"2023-08-27T11:29:25.977858+0000","last_undegraded":"2023-08-27T11:29:25.977858+0000","last_fullsized":"2023-08-27T11:29:25.977858+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:52.604143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553353+0000","last_change":"2023-08-27T11:29:23.571058+0000","last_active":"2023-08-27T11:29:24.553353+0000","last_peered":"2023-08-27T11:29:24.553353+0000","last_clean":"2023-08-27T11:29:24.553353+0000","last_became_active":"2023-08-27T11:29:23.570776+0000","last_became_peered":"2023-08-27T11:29:23.570776+0000","last_unstale":"2023-08-27T11:29:24.553353+0000","last_undegraded":"2023-08-27T11:29:24.553353+0000","last_fullsized":"2023-08-27T11:29:24.553353+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:39.692761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477610+0000","last_change":"2023-08-27T11:29:23.568596+0000","last_active":"2023-08-27T11:29:25.477610+0000","last_peered":"2023-08-27T11:29:25.477610+0000","last_clean":"2023-08-27T11:29:25.477610+0000","last_became_active":"2023-08-27T11:29:23.568324+0000","last_became_peered":"2023-08-27T11:29:23.568324+0000","last_unstale":"2023-08-27T11:29:25.477610+0000","last_undegraded":"2023-08-27T11:29:25.477610+0000","last_fullsized":"2023-08-27T11:29:25.477610+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:32:36.495461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978268+0000","last_change":"2023-08-27T11:29:23.570426+0000","last_active":"2023-08-27T11:29:25.978268+0000","last_peered":"2023-08-27T11:29:25.978268+0000","last_clean":"2023-08-27T11:29:25.978268+0000","last_became_active":"2023-08-27T11:29:23.570162+0000","last_became_peered":"2023-08-27T11:29:23.570162+0000","last_unstale":"2023-08-27T11:29:25.978268+0000","last_undegraded":"2023-08-27T11:29:25.978268+0000","last_fullsized":"2023-08-27T11:29:25.978268+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:28:11.052067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978899+0000","last_change":"2023-08-27T11:29:23.569056+0000","last_active":"2023-08-27T11:29:25.978899+0000","last_peered":"2023-08-27T11:29:25.978899+0000","last_clean":"2023-08-27T11:29:25.978899+0000","last_became_active":"2023-08-27T11:29:23.568749+0000","last_became_peered":"2023-08-27T11:29:23.568749+0000","last_unstale":"2023-08-27T11:29:25.978899+0000","last_undegraded":"2023-08-27T11:29:25.978899+0000","last_fullsized":"2023-08-27T11:29:25.978899+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:25:27.682286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551907+0000","last_change":"2023-08-27T11:29:23.574176+0000","last_active":"2023-08-27T11:29:24.551907+0000","last_peered":"2023-08-27T11:29:24.551907+0000","last_clean":"2023-08-27T11:29:24.551907+0000","last_became_active":"2023-08-27T11:29:23.571907+0000","last_became_peered":"2023-08-27T11:29:23.571907+0000","last_unstale":"2023-08-27T11:29:24.551907+0000","last_undegraded":"2023-08-27T11:29:24.551907+0000","last_fullsized":"2023-08-27T11:29:24.551907+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:41:35.223678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553737+0000","last_change":"2023-08-27T11:29:23.573811+0000","last_active":"2023-08-27T11:29:24.553737+0000","last_peered":"2023-08-27T11:29:24.553737+0000","last_clean":"2023-08-27T11:29:24.553737+0000","last_became_active":"2023-08-27T11:29:23.573536+0000","last_became_peered":"2023-08-27T11:29:23.573536+0000","last_unstale":"2023-08-27T11:29:24.553737+0000","last_undegraded":"2023-08-27T11:29:24.553737+0000","last_fullsized":"2023-08-27T11:29:24.553737+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:44:18.567183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551908+0000","last_change":"2023-08-27T11:29:23.564844+0000","last_active":"2023-08-27T11:29:24.551908+0000","last_peered":"2023-08-27T11:29:24.551908+0000","last_clean":"2023-08-27T11:29:24.551908+0000","last_became_active":"2023-08-27T11:29:23.564533+0000","last_became_peered":"2023-08-27T11:29:23.564533+0000","last_unstale":"2023-08-27T11:29:24.551908+0000","last_undegraded":"2023-08-27T11:29:24.551908+0000","last_fullsized":"2023-08-27T11:29:24.551908+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:34:27.688653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548282+0000","last_change":"2023-08-27T11:29:23.563226+0000","last_active":"2023-08-27T11:29:24.548282+0000","last_peered":"2023-08-27T11:29:24.548282+0000","last_clean":"2023-08-27T11:29:24.548282+0000","last_became_active":"2023-08-27T11:29:23.562934+0000","last_became_peered":"2023-08-27T11:29:23.562934+0000","last_unstale":"2023-08-27T11:29:24.548282+0000","last_undegraded":"2023-08-27T11:29:24.548282+0000","last_fullsized":"2023-08-27T11:29:24.548282+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:45:22.565179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553339+0000","last_change":"2023-08-27T11:29:23.577445+0000","last_active":"2023-08-27T11:29:24.553339+0000","last_peered":"2023-08-27T11:29:24.553339+0000","last_clean":"2023-08-27T11:29:24.553339+0000","last_became_active":"2023-08-27T11:29:23.577153+0000","last_became_peered":"2023-08-27T11:29:23.577153+0000","last_unstale":"2023-08-27T11:29:24.553339+0000","last_undegraded":"2023-08-27T11:29:24.553339+0000","last_fullsized":"2023-08-27T11:29:24.553339+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:18:03.892673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552061+0000","last_change":"2023-08-27T11:29:23.571984+0000","last_active":"2023-08-27T11:29:24.552061+0000","last_peered":"2023-08-27T11:29:24.552061+0000","last_clean":"2023-08-27T11:29:24.552061+0000","last_became_active":"2023-08-27T11:29:23.571661+0000","last_became_peered":"2023-08-27T11:29:23.571661+0000","last_unstale":"2023-08-27T11:29:24.552061+0000","last_undegraded":"2023-08-27T11:29:24.552061+0000","last_fullsized":"2023-08-27T11:29:24.552061+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:59:47.321676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978320+0000","last_change":"2023-08-27T11:29:23.558818+0000","last_active":"2023-08-27T11:29:25.978320+0000","last_peered":"2023-08-27T11:29:25.978320+0000","last_clean":"2023-08-27T11:29:25.978320+0000","last_became_active":"2023-08-27T11:29:23.558518+0000","last_became_peered":"2023-08-27T11:29:23.558518+0000","last_unstale":"2023-08-27T11:29:25.978320+0000","last_undegraded":"2023-08-27T11:29:25.978320+0000","last_fullsized":"2023-08-27T11:29:25.978320+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:16:46.158709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134856+0000","last_change":"2023-08-27T11:29:23.569571+0000","last_active":"2023-08-27T11:29:25.134856+0000","last_peered":"2023-08-27T11:29:25.134856+0000","last_clean":"2023-08-27T11:29:25.134856+0000","last_became_active":"2023-08-27T11:29:23.569232+0000","last_became_peered":"2023-08-27T11:29:23.569232+0000","last_unstale":"2023-08-27T11:29:25.134856+0000","last_undegraded":"2023-08-27T11:29:25.134856+0000","last_fullsized":"2023-08-27T11:29:25.134856+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:42.128995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553746+0000","last_change":"2023-08-27T11:29:23.568862+0000","last_active":"2023-08-27T11:29:24.553746+0000","last_peered":"2023-08-27T11:29:24.553746+0000","last_clean":"2023-08-27T11:29:24.553746+0000","last_became_active":"2023-08-27T11:29:23.568590+0000","last_became_peered":"2023-08-27T11:29:23.568590+0000","last_unstale":"2023-08-27T11:29:24.553746+0000","last_undegraded":"2023-08-27T11:29:24.553746+0000","last_fullsized":"2023-08-27T11:29:24.553746+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:01:06.953586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977584+0000","last_change":"2023-08-27T11:29:23.562119+0000","last_active":"2023-08-27T11:29:25.977584+0000","last_peered":"2023-08-27T11:29:25.977584+0000","last_clean":"2023-08-27T11:29:25.977584+0000","last_became_active":"2023-08-27T11:29:23.561799+0000","last_became_peered":"2023-08-27T11:29:23.561799+0000","last_unstale":"2023-08-27T11:29:25.977584+0000","last_undegraded":"2023-08-27T11:29:25.977584+0000","last_fullsized":"2023-08-27T11:29:25.977584+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:05:09.501842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548385+0000","last_change":"2023-08-27T11:29:23.569700+0000","last_active":"2023-08-27T11:29:24.548385+0000","last_peered":"2023-08-27T11:29:24.548385+0000","last_clean":"2023-08-27T11:29:24.548385+0000","last_became_active":"2023-08-27T11:29:23.568767+0000","last_became_peered":"2023-08-27T11:29:23.568767+0000","last_unstale":"2023-08-27T11:29:24.548385+0000","last_undegraded":"2023-08-27T11:29:24.548385+0000","last_fullsized":"2023-08-27T11:29:24.548385+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:15:09.273200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477367+0000","last_change":"2023-08-27T11:29:23.569362+0000","last_active":"2023-08-27T11:29:25.477367+0000","last_peered":"2023-08-27T11:29:25.477367+0000","last_clean":"2023-08-27T11:29:25.477367+0000","last_became_active":"2023-08-27T11:29:23.569111+0000","last_became_peered":"2023-08-27T11:29:23.569111+0000","last_unstale":"2023-08-27T11:29:25.477367+0000","last_undegraded":"2023-08-27T11:29:25.477367+0000","last_fullsized":"2023-08-27T11:29:25.477367+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:37:50.003802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134482+0000","last_change":"2023-08-27T11:29:23.570472+0000","last_active":"2023-08-27T11:29:25.134482+0000","last_peered":"2023-08-27T11:29:25.134482+0000","last_clean":"2023-08-27T11:29:25.134482+0000","last_became_active":"2023-08-27T11:29:23.568244+0000","last_became_peered":"2023-08-27T11:29:23.568244+0000","last_unstale":"2023-08-27T11:29:25.134482+0000","last_undegraded":"2023-08-27T11:29:25.134482+0000","last_fullsized":"2023-08-27T11:29:25.134482+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:21:54.469916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134995+0000","last_change":"2023-08-27T11:29:23.576730+0000","last_active":"2023-08-27T11:29:25.134995+0000","last_peered":"2023-08-27T11:29:25.134995+0000","last_clean":"2023-08-27T11:29:25.134995+0000","last_became_active":"2023-08-27T11:29:23.576168+0000","last_became_peered":"2023-08-27T11:29:23.576168+0000","last_unstale":"2023-08-27T11:29:25.134995+0000","last_undegraded":"2023-08-27T11:29:25.134995+0000","last_fullsized":"2023-08-27T11:29:25.134995+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:56:52.509141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978405+0000","last_change":"2023-08-27T11:29:23.567015+0000","last_active":"2023-08-27T11:29:25.978405+0000","last_peered":"2023-08-27T11:29:25.978405+0000","last_clean":"2023-08-27T11:29:25.978405+0000","last_became_active":"2023-08-27T11:29:23.566662+0000","last_became_peered":"2023-08-27T11:29:23.566662+0000","last_unstale":"2023-08-27T11:29:25.978405+0000","last_undegraded":"2023-08-27T11:29:25.978405+0000","last_fullsized":"2023-08-27T11:29:25.978405+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:54:19.977743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.979077+0000","last_change":"2023-08-27T11:29:23.575522+0000","last_active":"2023-08-27T11:29:25.979077+0000","last_peered":"2023-08-27T11:29:25.979077+0000","last_clean":"2023-08-27T11:29:25.979077+0000","last_became_active":"2023-08-27T11:29:23.569334+0000","last_became_peered":"2023-08-27T11:29:23.569334+0000","last_unstale":"2023-08-27T11:29:25.979077+0000","last_undegraded":"2023-08-27T11:29:25.979077+0000","last_fullsized":"2023-08-27T11:29:25.979077+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:02:15.157138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552143+0000","last_change":"2023-08-27T11:29:23.572875+0000","last_active":"2023-08-27T11:29:24.552143+0000","last_peered":"2023-08-27T11:29:24.552143+0000","last_clean":"2023-08-27T11:29:24.552143+0000","last_became_active":"2023-08-27T11:29:23.569074+0000","last_became_peered":"2023-08-27T11:29:23.569074+0000","last_unstale":"2023-08-27T11:29:24.552143+0000","last_undegraded":"2023-08-27T11:29:24.552143+0000","last_fullsized":"2023-08-27T11:29:24.552143+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:48:38.362418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977695+0000","last_change":"2023-08-27T11:29:23.573735+0000","last_active":"2023-08-27T11:29:25.977695+0000","last_peered":"2023-08-27T11:29:25.977695+0000","last_clean":"2023-08-27T11:29:25.977695+0000","last_became_active":"2023-08-27T11:29:23.573427+0000","last_became_peered":"2023-08-27T11:29:23.573427+0000","last_unstale":"2023-08-27T11:29:25.977695+0000","last_undegraded":"2023-08-27T11:29:25.977695+0000","last_fullsized":"2023-08-27T11:29:25.977695+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:47:27.112051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553667+0000","last_change":"2023-08-27T11:29:23.580405+0000","last_active":"2023-08-27T11:29:24.553667+0000","last_peered":"2023-08-27T11:29:24.553667+0000","last_clean":"2023-08-27T11:29:24.553667+0000","last_became_active":"2023-08-27T11:29:23.580101+0000","last_became_peered":"2023-08-27T11:29:23.580101+0000","last_unstale":"2023-08-27T11:29:24.553667+0000","last_undegraded":"2023-08-27T11:29:24.553667+0000","last_fullsized":"2023-08-27T11:29:24.553667+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:32.765196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477476+0000","last_change":"2023-08-27T11:29:23.556890+0000","last_active":"2023-08-27T11:29:25.477476+0000","last_peered":"2023-08-27T11:29:25.477476+0000","last_clean":"2023-08-27T11:29:25.477476+0000","last_became_active":"2023-08-27T11:29:23.556609+0000","last_became_peered":"2023-08-27T11:29:23.556609+0000","last_unstale":"2023-08-27T11:29:25.477476+0000","last_undegraded":"2023-08-27T11:29:25.477476+0000","last_fullsized":"2023-08-27T11:29:25.477476+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:04:52.635785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553569+0000","last_change":"2023-08-27T11:29:23.580349+0000","last_active":"2023-08-27T11:29:24.553569+0000","last_peered":"2023-08-27T11:29:24.553569+0000","last_clean":"2023-08-27T11:29:24.553569+0000","last_became_active":"2023-08-27T11:29:23.580041+0000","last_became_peered":"2023-08-27T11:29:23.580041+0000","last_unstale":"2023-08-27T11:29:24.553569+0000","last_undegraded":"2023-08-27T11:29:24.553569+0000","last_fullsized":"2023-08-27T11:29:24.553569+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:14:10.823406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548762+0000","last_change":"2023-08-27T11:29:23.569596+0000","last_active":"2023-08-27T11:29:24.548762+0000","last_peered":"2023-08-27T11:29:24.548762+0000","last_clean":"2023-08-27T11:29:24.548762+0000","last_became_active":"2023-08-27T11:29:23.569330+0000","last_became_peered":"2023-08-27T11:29:23.569330+0000","last_unstale":"2023-08-27T11:29:24.548762+0000","last_undegraded":"2023-08-27T11:29:24.548762+0000","last_fullsized":"2023-08-27T11:29:24.548762+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:05:09.501842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.135033+0000","last_change":"2023-08-27T11:29:22.562070+0000","last_active":"2023-08-27T11:29:25.135033+0000","last_peered":"2023-08-27T11:29:25.135033+0000","last_clean":"2023-08-27T11:29:25.135033+0000","last_became_active":"2023-08-27T11:29:22.561802+0000","last_became_peered":"2023-08-27T11:29:22.561802+0000","last_unstale":"2023-08-27T11:29:25.135033+0000","last_undegraded":"2023-08-27T11:29:25.135033+0000","last_fullsized":"2023-08-27T11:29:25.135033+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:01:07.716603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548600+0000","last_change":"2023-08-27T11:29:23.568672+0000","last_active":"2023-08-27T11:29:24.548600+0000","last_peered":"2023-08-27T11:29:24.548600+0000","last_clean":"2023-08-27T11:29:24.548600+0000","last_became_active":"2023-08-27T11:29:23.568409+0000","last_became_peered":"2023-08-27T11:29:23.568409+0000","last_unstale":"2023-08-27T11:29:24.548600+0000","last_undegraded":"2023-08-27T11:29:24.548600+0000","last_fullsized":"2023-08-27T11:29:24.548600+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:22:38.147718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978997+0000","last_change":"2023-08-27T11:29:22.562667+0000","last_active":"2023-08-27T11:29:25.978997+0000","last_peered":"2023-08-27T11:29:25.978997+0000","last_clean":"2023-08-27T11:29:25.978997+0000","last_became_active":"2023-08-27T11:29:22.562345+0000","last_became_peered":"2023-08-27T11:29:22.562345+0000","last_unstale":"2023-08-27T11:29:25.978997+0000","last_undegraded":"2023-08-27T11:29:25.978997+0000","last_fullsized":"2023-08-27T11:29:25.978997+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:41:24.143018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978446+0000","last_change":"2023-08-27T11:29:23.561542+0000","last_active":"2023-08-27T11:29:25.978446+0000","last_peered":"2023-08-27T11:29:25.978446+0000","last_clean":"2023-08-27T11:29:25.978446+0000","last_became_active":"2023-08-27T11:29:23.561273+0000","last_became_peered":"2023-08-27T11:29:23.561273+0000","last_unstale":"2023-08-27T11:29:25.978446+0000","last_undegraded":"2023-08-27T11:29:25.978446+0000","last_fullsized":"2023-08-27T11:29:25.978446+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:04:36.834335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"166'3","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134981+0000","last_change":"2023-08-27T11:29:22.565766+0000","last_active":"2023-08-27T11:29:25.134981+0000","last_peered":"2023-08-27T11:29:25.134981+0000","last_clean":"2023-08-27T11:29:25.134981+0000","last_became_active":"2023-08-27T11:29:22.565461+0000","last_became_peered":"2023-08-27T11:29:22.565461+0000","last_unstale":"2023-08-27T11:29:25.134981+0000","last_undegraded":"2023-08-27T11:29:25.134981+0000","last_fullsized":"2023-08-27T11:29:25.134981+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:32:10.556679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.979145+0000","last_change":"2023-08-27T11:29:23.570552+0000","last_active":"2023-08-27T11:29:25.979145+0000","last_peered":"2023-08-27T11:29:25.979145+0000","last_clean":"2023-08-27T11:29:25.979145+0000","last_became_active":"2023-08-27T11:29:23.568978+0000","last_became_peered":"2023-08-27T11:29:23.568978+0000","last_unstale":"2023-08-27T11:29:25.979145+0000","last_undegraded":"2023-08-27T11:29:25.979145+0000","last_fullsized":"2023-08-27T11:29:25.979145+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:48.791526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"166'6","reported_seq":19,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553803+0000","last_change":"2023-08-27T11:29:22.568408+0000","last_active":"2023-08-27T11:29:24.553803+0000","last_peered":"2023-08-27T11:29:24.553803+0000","last_clean":"2023-08-27T11:29:24.553803+0000","last_became_active":"2023-08-27T11:29:22.567952+0000","last_became_peered":"2023-08-27T11:29:22.567952+0000","last_unstale":"2023-08-27T11:29:24.553803+0000","last_undegraded":"2023-08-27T11:29:24.553803+0000","last_fullsized":"2023-08-27T11:29:24.553803+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:18:50.478300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977493+0000","last_change":"2023-08-27T11:29:23.557170+0000","last_active":"2023-08-27T11:29:25.977493+0000","last_peered":"2023-08-27T11:29:25.977493+0000","last_clean":"2023-08-27T11:29:25.977493+0000","last_became_active":"2023-08-27T11:29:23.556886+0000","last_became_peered":"2023-08-27T11:29:23.556886+0000","last_unstale":"2023-08-27T11:29:25.977493+0000","last_undegraded":"2023-08-27T11:29:25.977493+0000","last_fullsized":"2023-08-27T11:29:25.977493+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:26:27.741746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"166'2","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552726+0000","last_change":"2023-08-27T11:29:22.560057+0000","last_active":"2023-08-27T11:29:24.552726+0000","last_peered":"2023-08-27T11:29:24.552726+0000","last_clean":"2023-08-27T11:29:24.552726+0000","last_became_active":"2023-08-27T11:29:22.559575+0000","last_became_peered":"2023-08-27T11:29:22.559575+0000","last_unstale":"2023-08-27T11:29:24.552726+0000","last_undegraded":"2023-08-27T11:29:24.552726+0000","last_fullsized":"2023-08-27T11:29:24.552726+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:28:16.343767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551664+0000","last_change":"2023-08-27T11:29:23.572143+0000","last_active":"2023-08-27T11:29:24.551664+0000","last_peered":"2023-08-27T11:29:24.551664+0000","last_clean":"2023-08-27T11:29:24.551664+0000","last_became_active":"2023-08-27T11:29:23.571833+0000","last_became_peered":"2023-08-27T11:29:23.571833+0000","last_unstale":"2023-08-27T11:29:24.551664+0000","last_undegraded":"2023-08-27T11:29:24.551664+0000","last_fullsized":"2023-08-27T11:29:24.551664+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:23:16.433248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977651+0000","last_change":"2023-08-27T11:29:22.559033+0000","last_active":"2023-08-27T11:29:25.977651+0000","last_peered":"2023-08-27T11:29:25.977651+0000","last_clean":"2023-08-27T11:29:25.977651+0000","last_became_active":"2023-08-27T11:29:22.558779+0000","last_became_peered":"2023-08-27T11:29:22.558779+0000","last_unstale":"2023-08-27T11:29:25.977651+0000","last_undegraded":"2023-08-27T11:29:25.977651+0000","last_fullsized":"2023-08-27T11:29:25.977651+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:22:17.805705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977688+0000","last_change":"2023-08-27T11:29:23.569437+0000","last_active":"2023-08-27T11:29:25.977688+0000","last_peered":"2023-08-27T11:29:25.977688+0000","last_clean":"2023-08-27T11:29:25.977688+0000","last_became_active":"2023-08-27T11:29:23.569195+0000","last_became_peered":"2023-08-27T11:29:23.569195+0000","last_unstale":"2023-08-27T11:29:25.977688+0000","last_undegraded":"2023-08-27T11:29:25.977688+0000","last_fullsized":"2023-08-27T11:29:25.977688+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:11:00.606578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553272+0000","last_change":"2023-08-27T11:29:22.561162+0000","last_active":"2023-08-27T11:29:24.553272+0000","last_peered":"2023-08-27T11:29:24.553272+0000","last_clean":"2023-08-27T11:29:24.553272+0000","last_became_active":"2023-08-27T11:29:22.560845+0000","last_became_peered":"2023-08-27T11:29:22.560845+0000","last_unstale":"2023-08-27T11:29:24.553272+0000","last_undegraded":"2023-08-27T11:29:24.553272+0000","last_fullsized":"2023-08-27T11:29:24.553272+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:41:17.191787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548331+0000","last_change":"2023-08-27T11:29:23.565070+0000","last_active":"2023-08-27T11:29:24.548331+0000","last_peered":"2023-08-27T11:29:24.548331+0000","last_clean":"2023-08-27T11:29:24.548331+0000","last_became_active":"2023-08-27T11:29:23.564803+0000","last_became_peered":"2023-08-27T11:29:23.564803+0000","last_unstale":"2023-08-27T11:29:24.548331+0000","last_undegraded":"2023-08-27T11:29:24.548331+0000","last_fullsized":"2023-08-27T11:29:24.548331+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:12:57.090796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"166'8","reported_seq":21,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551678+0000","last_change":"2023-08-27T11:29:22.558198+0000","last_active":"2023-08-27T11:29:24.551678+0000","last_peered":"2023-08-27T11:29:24.551678+0000","last_clean":"2023-08-27T11:29:24.551678+0000","last_became_active":"2023-08-27T11:29:22.557898+0000","last_became_peered":"2023-08-27T11:29:22.557898+0000","last_unstale":"2023-08-27T11:29:24.551678+0000","last_undegraded":"2023-08-27T11:29:24.551678+0000","last_fullsized":"2023-08-27T11:29:24.551678+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:20:35.852006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977572+0000","last_change":"2023-08-27T11:29:23.568447+0000","last_active":"2023-08-27T11:29:25.977572+0000","last_peered":"2023-08-27T11:29:25.977572+0000","last_clean":"2023-08-27T11:29:25.977572+0000","last_became_active":"2023-08-27T11:29:23.567576+0000","last_became_peered":"2023-08-27T11:29:23.567576+0000","last_unstale":"2023-08-27T11:29:25.977572+0000","last_undegraded":"2023-08-27T11:29:25.977572+0000","last_fullsized":"2023-08-27T11:29:25.977572+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:24:56.575880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"166'1","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978391+0000","last_change":"2023-08-27T11:29:22.560159+0000","last_active":"2023-08-27T11:29:25.978391+0000","last_peered":"2023-08-27T11:29:25.978391+0000","last_clean":"2023-08-27T11:29:25.978391+0000","last_became_active":"2023-08-27T11:29:22.559773+0000","last_became_peered":"2023-08-27T11:29:22.559773+0000","last_unstale":"2023-08-27T11:29:25.978391+0000","last_undegraded":"2023-08-27T11:29:25.978391+0000","last_fullsized":"2023-08-27T11:29:25.978391+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:49:28.746872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134879+0000","last_change":"2023-08-27T11:29:23.581911+0000","last_active":"2023-08-27T11:29:25.134879+0000","last_peered":"2023-08-27T11:29:25.134879+0000","last_clean":"2023-08-27T11:29:25.134879+0000","last_became_active":"2023-08-27T11:29:23.581566+0000","last_became_peered":"2023-08-27T11:29:23.581566+0000","last_unstale":"2023-08-27T11:29:25.134879+0000","last_undegraded":"2023-08-27T11:29:25.134879+0000","last_fullsized":"2023-08-27T11:29:25.134879+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:16:47.512485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"166'2","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477630+0000","last_change":"2023-08-27T11:29:22.556687+0000","last_active":"2023-08-27T11:29:25.477630+0000","last_peered":"2023-08-27T11:29:25.477630+0000","last_clean":"2023-08-27T11:29:25.477630+0000","last_became_active":"2023-08-27T11:29:22.556430+0000","last_became_peered":"2023-08-27T11:29:22.556430+0000","last_unstale":"2023-08-27T11:29:25.477630+0000","last_undegraded":"2023-08-27T11:29:25.477630+0000","last_fullsized":"2023-08-27T11:29:25.477630+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:33:37.647840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978504+0000","last_change":"2023-08-27T11:29:23.563095+0000","last_active":"2023-08-27T11:29:25.978504+0000","last_peered":"2023-08-27T11:29:25.978504+0000","last_clean":"2023-08-27T11:29:25.978504+0000","last_became_active":"2023-08-27T11:29:23.562921+0000","last_became_peered":"2023-08-27T11:29:23.562921+0000","last_unstale":"2023-08-27T11:29:25.978504+0000","last_undegraded":"2023-08-27T11:29:25.978504+0000","last_fullsized":"2023-08-27T11:29:25.978504+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:23:16.433248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"166'1","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551920+0000","last_change":"2023-08-27T11:29:22.567394+0000","last_active":"2023-08-27T11:29:24.551920+0000","last_peered":"2023-08-27T11:29:24.551920+0000","last_clean":"2023-08-27T11:29:24.551920+0000","last_became_active":"2023-08-27T11:29:22.566951+0000","last_became_peered":"2023-08-27T11:29:22.566951+0000","last_unstale":"2023-08-27T11:29:24.551920+0000","last_undegraded":"2023-08-27T11:29:24.551920+0000","last_fullsized":"2023-08-27T11:29:24.551920+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:55:21.740217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552974+0000","last_change":"2023-08-27T11:29:23.577363+0000","last_active":"2023-08-27T11:29:24.552974+0000","last_peered":"2023-08-27T11:29:24.552974+0000","last_clean":"2023-08-27T11:29:24.552974+0000","last_became_active":"2023-08-27T11:29:23.576949+0000","last_became_peered":"2023-08-27T11:29:23.576949+0000","last_unstale":"2023-08-27T11:29:24.552974+0000","last_undegraded":"2023-08-27T11:29:24.552974+0000","last_fullsized":"2023-08-27T11:29:24.552974+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:52:46.803848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552131+0000","last_change":"2023-08-27T11:29:22.567951+0000","last_active":"2023-08-27T11:29:24.552131+0000","last_peered":"2023-08-27T11:29:24.552131+0000","last_clean":"2023-08-27T11:29:24.552131+0000","last_became_active":"2023-08-27T11:29:22.567656+0000","last_became_peered":"2023-08-27T11:29:22.567656+0000","last_unstale":"2023-08-27T11:29:24.552131+0000","last_undegraded":"2023-08-27T11:29:24.552131+0000","last_fullsized":"2023-08-27T11:29:24.552131+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:14:30.868414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551721+0000","last_change":"2023-08-27T11:29:23.570168+0000","last_active":"2023-08-27T11:29:24.551721+0000","last_peered":"2023-08-27T11:29:24.551721+0000","last_clean":"2023-08-27T11:29:24.551721+0000","last_became_active":"2023-08-27T11:29:23.569894+0000","last_became_peered":"2023-08-27T11:29:23.569894+0000","last_unstale":"2023-08-27T11:29:24.551721+0000","last_undegraded":"2023-08-27T11:29:24.551721+0000","last_fullsized":"2023-08-27T11:29:24.551721+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:20:03.029838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"166'2","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552815+0000","last_change":"2023-08-27T11:29:22.565006+0000","last_active":"2023-08-27T11:29:24.552815+0000","last_peered":"2023-08-27T11:29:24.552815+0000","last_clean":"2023-08-27T11:29:24.552815+0000","last_became_active":"2023-08-27T11:29:22.564725+0000","last_became_peered":"2023-08-27T11:29:22.564725+0000","last_unstale":"2023-08-27T11:29:24.552815+0000","last_undegraded":"2023-08-27T11:29:24.552815+0000","last_fullsized":"2023-08-27T11:29:24.552815+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:40:58.304229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134571+0000","last_change":"2023-08-27T11:29:23.575183+0000","last_active":"2023-08-27T11:29:25.134571+0000","last_peered":"2023-08-27T11:29:25.134571+0000","last_clean":"2023-08-27T11:29:25.134571+0000","last_became_active":"2023-08-27T11:29:23.574877+0000","last_became_peered":"2023-08-27T11:29:23.574877+0000","last_unstale":"2023-08-27T11:29:25.134571+0000","last_undegraded":"2023-08-27T11:29:25.134571+0000","last_fullsized":"2023-08-27T11:29:25.134571+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:00:00.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548523+0000","last_change":"2023-08-27T11:29:22.567705+0000","last_active":"2023-08-27T11:29:24.548523+0000","last_peered":"2023-08-27T11:29:24.548523+0000","last_clean":"2023-08-27T11:29:24.548523+0000","last_became_active":"2023-08-27T11:29:22.567411+0000","last_became_peered":"2023-08-27T11:29:22.567411+0000","last_unstale":"2023-08-27T11:29:24.548523+0000","last_undegraded":"2023-08-27T11:29:24.548523+0000","last_fullsized":"2023-08-27T11:29:24.548523+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:36:31.122935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977567+0000","last_change":"2023-08-27T11:29:23.566077+0000","last_active":"2023-08-27T11:29:25.977567+0000","last_peered":"2023-08-27T11:29:25.977567+0000","last_clean":"2023-08-27T11:29:25.977567+0000","last_became_active":"2023-08-27T11:29:23.565704+0000","last_became_peered":"2023-08-27T11:29:23.565704+0000","last_unstale":"2023-08-27T11:29:25.977567+0000","last_undegraded":"2023-08-27T11:29:25.977567+0000","last_fullsized":"2023-08-27T11:29:25.977567+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:27:46.166160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548268+0000","last_change":"2023-08-27T11:29:22.557367+0000","last_active":"2023-08-27T11:29:24.548268+0000","last_peered":"2023-08-27T11:29:24.548268+0000","last_clean":"2023-08-27T11:29:24.548268+0000","last_became_active":"2023-08-27T11:29:22.557097+0000","last_became_peered":"2023-08-27T11:29:22.557097+0000","last_unstale":"2023-08-27T11:29:24.548268+0000","last_undegraded":"2023-08-27T11:29:24.548268+0000","last_fullsized":"2023-08-27T11:29:24.548268+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:50:53.719358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978071+0000","last_change":"2023-08-27T11:29:23.571420+0000","last_active":"2023-08-27T11:29:25.978071+0000","last_peered":"2023-08-27T11:29:25.978071+0000","last_clean":"2023-08-27T11:29:25.978071+0000","last_became_active":"2023-08-27T11:29:23.571101+0000","last_became_peered":"2023-08-27T11:29:23.571101+0000","last_unstale":"2023-08-27T11:29:25.978071+0000","last_undegraded":"2023-08-27T11:29:25.978071+0000","last_fullsized":"2023-08-27T11:29:25.978071+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:59:57.307275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477238+0000","last_change":"2023-08-27T11:29:22.556610+0000","last_active":"2023-08-27T11:29:25.477238+0000","last_peered":"2023-08-27T11:29:25.477238+0000","last_clean":"2023-08-27T11:29:25.477238+0000","last_became_active":"2023-08-27T11:29:22.556349+0000","last_became_peered":"2023-08-27T11:29:22.556349+0000","last_unstale":"2023-08-27T11:29:25.477238+0000","last_undegraded":"2023-08-27T11:29:25.477238+0000","last_fullsized":"2023-08-27T11:29:25.477238+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:48:58.865671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977943+0000","last_change":"2023-08-27T11:29:23.567799+0000","last_active":"2023-08-27T11:29:25.977943+0000","last_peered":"2023-08-27T11:29:25.977943+0000","last_clean":"2023-08-27T11:29:25.977943+0000","last_became_active":"2023-08-27T11:29:23.567510+0000","last_became_peered":"2023-08-27T11:29:23.567510+0000","last_unstale":"2023-08-27T11:29:25.977943+0000","last_undegraded":"2023-08-27T11:29:25.977943+0000","last_fullsized":"2023-08-27T11:29:25.977943+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:44:23.684915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"166'2","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552366+0000","last_change":"2023-08-27T11:29:22.559447+0000","last_active":"2023-08-27T11:29:24.552366+0000","last_peered":"2023-08-27T11:29:24.552366+0000","last_clean":"2023-08-27T11:29:24.552366+0000","last_became_active":"2023-08-27T11:29:22.559069+0000","last_became_peered":"2023-08-27T11:29:22.559069+0000","last_unstale":"2023-08-27T11:29:24.552366+0000","last_undegraded":"2023-08-27T11:29:24.552366+0000","last_fullsized":"2023-08-27T11:29:24.552366+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:56:08.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134998+0000","last_change":"2023-08-27T11:29:23.576119+0000","last_active":"2023-08-27T11:29:25.134998+0000","last_peered":"2023-08-27T11:29:25.134998+0000","last_clean":"2023-08-27T11:29:25.134998+0000","last_became_active":"2023-08-27T11:29:23.575825+0000","last_became_peered":"2023-08-27T11:29:23.575825+0000","last_unstale":"2023-08-27T11:29:25.134998+0000","last_undegraded":"2023-08-27T11:29:25.134998+0000","last_fullsized":"2023-08-27T11:29:25.134998+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:35.075986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"167'1","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548724+0000","last_change":"2023-08-27T11:29:22.557095+0000","last_active":"2023-08-27T11:29:24.548724+0000","last_peered":"2023-08-27T11:29:24.548724+0000","last_clean":"2023-08-27T11:29:24.548724+0000","last_became_active":"2023-08-27T11:29:22.556840+0000","last_became_peered":"2023-08-27T11:29:22.556840+0000","last_unstale":"2023-08-27T11:29:24.548724+0000","last_undegraded":"2023-08-27T11:29:24.548724+0000","last_fullsized":"2023-08-27T11:29:24.548724+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:38:06.716876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553686+0000","last_change":"2023-08-27T11:29:23.579985+0000","last_active":"2023-08-27T11:29:24.553686+0000","last_peered":"2023-08-27T11:29:24.553686+0000","last_clean":"2023-08-27T11:29:24.553686+0000","last_became_active":"2023-08-27T11:29:23.579711+0000","last_became_peered":"2023-08-27T11:29:23.579711+0000","last_unstale":"2023-08-27T11:29:24.553686+0000","last_undegraded":"2023-08-27T11:29:24.553686+0000","last_fullsized":"2023-08-27T11:29:24.553686+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:00:25.316042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548618+0000","last_change":"2023-08-27T11:29:22.559052+0000","last_active":"2023-08-27T11:29:24.548618+0000","last_peered":"2023-08-27T11:29:24.548618+0000","last_clean":"2023-08-27T11:29:24.548618+0000","last_became_active":"2023-08-27T11:29:22.558768+0000","last_became_peered":"2023-08-27T11:29:22.558768+0000","last_unstale":"2023-08-27T11:29:24.548618+0000","last_undegraded":"2023-08-27T11:29:24.548618+0000","last_fullsized":"2023-08-27T11:29:24.548618+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:54.602268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477675+0000","last_change":"2023-08-27T11:29:23.558027+0000","last_active":"2023-08-27T11:29:25.477675+0000","last_peered":"2023-08-27T11:29:25.477675+0000","last_clean":"2023-08-27T11:29:25.477675+0000","last_became_active":"2023-08-27T11:29:23.557746+0000","last_became_peered":"2023-08-27T11:29:23.557746+0000","last_unstale":"2023-08-27T11:29:25.477675+0000","last_undegraded":"2023-08-27T11:29:25.477675+0000","last_fullsized":"2023-08-27T11:29:25.477675+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:22.816411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552272+0000","last_change":"2023-08-27T11:29:22.565139+0000","last_active":"2023-08-27T11:29:24.552272+0000","last_peered":"2023-08-27T11:29:24.552272+0000","last_clean":"2023-08-27T11:29:24.552272+0000","last_became_active":"2023-08-27T11:29:22.564849+0000","last_became_peered":"2023-08-27T11:29:22.564849+0000","last_unstale":"2023-08-27T11:29:24.552272+0000","last_undegraded":"2023-08-27T11:29:24.552272+0000","last_fullsized":"2023-08-27T11:29:24.552272+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:24:24.316772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977852+0000","last_change":"2023-08-27T11:29:23.572206+0000","last_active":"2023-08-27T11:29:25.977852+0000","last_peered":"2023-08-27T11:29:25.977852+0000","last_clean":"2023-08-27T11:29:25.977852+0000","last_became_active":"2023-08-27T11:29:23.571041+0000","last_became_peered":"2023-08-27T11:29:23.571041+0000","last_unstale":"2023-08-27T11:29:25.977852+0000","last_undegraded":"2023-08-27T11:29:25.977852+0000","last_fullsized":"2023-08-27T11:29:25.977852+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:02:39.792746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552837+0000","last_change":"2023-08-27T11:29:22.559902+0000","last_active":"2023-08-27T11:29:24.552837+0000","last_peered":"2023-08-27T11:29:24.552837+0000","last_clean":"2023-08-27T11:29:24.552837+0000","last_became_active":"2023-08-27T11:29:22.559202+0000","last_became_peered":"2023-08-27T11:29:22.559202+0000","last_unstale":"2023-08-27T11:29:24.552837+0000","last_undegraded":"2023-08-27T11:29:24.552837+0000","last_fullsized":"2023-08-27T11:29:24.552837+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:10:12.587313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978128+0000","last_change":"2023-08-27T11:29:23.558180+0000","last_active":"2023-08-27T11:29:25.978128+0000","last_peered":"2023-08-27T11:29:25.978128+0000","last_clean":"2023-08-27T11:29:25.978128+0000","last_became_active":"2023-08-27T11:29:23.557905+0000","last_became_peered":"2023-08-27T11:29:23.557905+0000","last_unstale":"2023-08-27T11:29:25.978128+0000","last_undegraded":"2023-08-27T11:29:25.978128+0000","last_fullsized":"2023-08-27T11:29:25.978128+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:14:41.834602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978675+0000","last_change":"2023-08-27T11:29:22.562609+0000","last_active":"2023-08-27T11:29:25.978675+0000","last_peered":"2023-08-27T11:29:25.978675+0000","last_clean":"2023-08-27T11:29:25.978675+0000","last_became_active":"2023-08-27T11:29:22.562290+0000","last_became_peered":"2023-08-27T11:29:22.562290+0000","last_unstale":"2023-08-27T11:29:25.978675+0000","last_undegraded":"2023-08-27T11:29:25.978675+0000","last_fullsized":"2023-08-27T11:29:25.978675+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:06:30.698615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477524+0000","last_change":"2023-08-27T11:29:23.557077+0000","last_active":"2023-08-27T11:29:25.477524+0000","last_peered":"2023-08-27T11:29:25.477524+0000","last_clean":"2023-08-27T11:29:25.477524+0000","last_became_active":"2023-08-27T11:29:23.556809+0000","last_became_peered":"2023-08-27T11:29:23.556809+0000","last_unstale":"2023-08-27T11:29:25.477524+0000","last_undegraded":"2023-08-27T11:29:25.477524+0000","last_fullsized":"2023-08-27T11:29:25.477524+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:47:47.795537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134582+0000","last_change":"2023-08-27T11:29:22.561794+0000","last_active":"2023-08-27T11:29:25.134582+0000","last_peered":"2023-08-27T11:29:25.134582+0000","last_clean":"2023-08-27T11:29:25.134582+0000","last_became_active":"2023-08-27T11:29:22.561350+0000","last_became_peered":"2023-08-27T11:29:22.561350+0000","last_unstale":"2023-08-27T11:29:25.134582+0000","last_undegraded":"2023-08-27T11:29:25.134582+0000","last_fullsized":"2023-08-27T11:29:25.134582+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:37.246059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552646+0000","last_change":"2023-08-27T11:29:23.577215+0000","last_active":"2023-08-27T11:29:24.552646+0000","last_peered":"2023-08-27T11:29:24.552646+0000","last_clean":"2023-08-27T11:29:24.552646+0000","last_became_active":"2023-08-27T11:29:23.576584+0000","last_became_peered":"2023-08-27T11:29:23.576584+0000","last_unstale":"2023-08-27T11:29:24.552646+0000","last_undegraded":"2023-08-27T11:29:24.552646+0000","last_fullsized":"2023-08-27T11:29:24.552646+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:55:40.017114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.551830+0000","last_change":"2023-08-27T11:29:22.557686+0000","last_active":"2023-08-27T11:29:24.551830+0000","last_peered":"2023-08-27T11:29:24.551830+0000","last_clean":"2023-08-27T11:29:24.551830+0000","last_became_active":"2023-08-27T11:29:22.557312+0000","last_became_peered":"2023-08-27T11:29:22.557312+0000","last_unstale":"2023-08-27T11:29:24.551830+0000","last_undegraded":"2023-08-27T11:29:24.551830+0000","last_fullsized":"2023-08-27T11:29:24.551830+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:06:49.976537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977719+0000","last_change":"2023-08-27T11:29:23.567811+0000","last_active":"2023-08-27T11:29:25.977719+0000","last_peered":"2023-08-27T11:29:25.977719+0000","last_clean":"2023-08-27T11:29:25.977719+0000","last_became_active":"2023-08-27T11:29:23.567559+0000","last_became_peered":"2023-08-27T11:29:23.567559+0000","last_unstale":"2023-08-27T11:29:25.977719+0000","last_undegraded":"2023-08-27T11:29:25.977719+0000","last_fullsized":"2023-08-27T11:29:25.977719+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:25:13.211321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"166'1","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548343+0000","last_change":"2023-08-27T11:29:22.560833+0000","last_active":"2023-08-27T11:29:24.548343+0000","last_peered":"2023-08-27T11:29:24.548343+0000","last_clean":"2023-08-27T11:29:24.548343+0000","last_became_active":"2023-08-27T11:29:22.560555+0000","last_became_peered":"2023-08-27T11:29:22.560555+0000","last_unstale":"2023-08-27T11:29:24.548343+0000","last_undegraded":"2023-08-27T11:29:24.548343+0000","last_fullsized":"2023-08-27T11:29:24.548343+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:43:12.351160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":318,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552500+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:29:24.552500+0000","last_peered":"2023-08-27T11:29:24.552500+0000","last_clean":"2023-08-27T11:29:24.552500+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:29:24.552500+0000","last_undegraded":"2023-08-27T11:29:24.552500+0000","last_fullsized":"2023-08-27T11:29:24.552500+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:45:18.352414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552269+0000","last_change":"2023-08-27T11:29:23.565347+0000","last_active":"2023-08-27T11:29:24.552269+0000","last_peered":"2023-08-27T11:29:24.552269+0000","last_clean":"2023-08-27T11:29:24.552269+0000","last_became_active":"2023-08-27T11:29:23.564270+0000","last_became_peered":"2023-08-27T11:29:23.564270+0000","last_unstale":"2023-08-27T11:29:24.552269+0000","last_undegraded":"2023-08-27T11:29:24.552269+0000","last_fullsized":"2023-08-27T11:29:24.552269+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:52:37.724239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"166'1","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134715+0000","last_change":"2023-08-27T11:29:22.560916+0000","last_active":"2023-08-27T11:29:25.134715+0000","last_peered":"2023-08-27T11:29:25.134715+0000","last_clean":"2023-08-27T11:29:25.134715+0000","last_became_active":"2023-08-27T11:29:22.560605+0000","last_became_peered":"2023-08-27T11:29:22.560605+0000","last_unstale":"2023-08-27T11:29:25.134715+0000","last_undegraded":"2023-08-27T11:29:25.134715+0000","last_fullsized":"2023-08-27T11:29:25.134715+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:25:46.102892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":318,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552399+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:29:24.552399+0000","last_peered":"2023-08-27T11:29:24.552399+0000","last_clean":"2023-08-27T11:29:24.552399+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:29:24.552399+0000","last_undegraded":"2023-08-27T11:29:24.552399+0000","last_fullsized":"2023-08-27T11:29:24.552399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:51:58.956267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552053+0000","last_change":"2023-08-27T11:29:23.574637+0000","last_active":"2023-08-27T11:29:24.552053+0000","last_peered":"2023-08-27T11:29:24.552053+0000","last_clean":"2023-08-27T11:29:24.552053+0000","last_became_active":"2023-08-27T11:29:23.573019+0000","last_became_peered":"2023-08-27T11:29:23.573019+0000","last_unstale":"2023-08-27T11:29:24.552053+0000","last_undegraded":"2023-08-27T11:29:24.552053+0000","last_fullsized":"2023-08-27T11:29:24.552053+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:47:58.802669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"166'2","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553735+0000","last_change":"2023-08-27T11:29:22.564661+0000","last_active":"2023-08-27T11:29:24.553735+0000","last_peered":"2023-08-27T11:29:24.553735+0000","last_clean":"2023-08-27T11:29:24.553735+0000","last_became_active":"2023-08-27T11:29:22.564370+0000","last_became_peered":"2023-08-27T11:29:22.564370+0000","last_unstale":"2023-08-27T11:29:24.553735+0000","last_undegraded":"2023-08-27T11:29:24.553735+0000","last_fullsized":"2023-08-27T11:29:24.553735+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:33:45.954366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":273,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.553437+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:29:24.553437+0000","last_peered":"2023-08-27T11:29:24.553437+0000","last_clean":"2023-08-27T11:29:24.553437+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:29:24.553437+0000","last_undegraded":"2023-08-27T11:29:24.553437+0000","last_fullsized":"2023-08-27T11:29:24.553437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:17:42.278588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978133+0000","last_change":"2023-08-27T11:29:23.565144+0000","last_active":"2023-08-27T11:29:25.978133+0000","last_peered":"2023-08-27T11:29:25.978133+0000","last_clean":"2023-08-27T11:29:25.978133+0000","last_became_active":"2023-08-27T11:29:23.564519+0000","last_became_peered":"2023-08-27T11:29:23.564519+0000","last_unstale":"2023-08-27T11:29:25.978133+0000","last_undegraded":"2023-08-27T11:29:25.978133+0000","last_fullsized":"2023-08-27T11:29:25.978133+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:46:49.330993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"166'2","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134724+0000","last_change":"2023-08-27T11:29:22.562502+0000","last_active":"2023-08-27T11:29:25.134724+0000","last_peered":"2023-08-27T11:29:25.134724+0000","last_clean":"2023-08-27T11:29:25.134724+0000","last_became_active":"2023-08-27T11:29:22.562213+0000","last_became_peered":"2023-08-27T11:29:22.562213+0000","last_unstale":"2023-08-27T11:29:25.134724+0000","last_undegraded":"2023-08-27T11:29:25.134724+0000","last_fullsized":"2023-08-27T11:29:25.134724+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:14.795810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":269,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134584+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:29:25.134584+0000","last_peered":"2023-08-27T11:29:25.134584+0000","last_clean":"2023-08-27T11:29:25.134584+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:29:25.134584+0000","last_undegraded":"2023-08-27T11:29:25.134584+0000","last_fullsized":"2023-08-27T11:29:25.134584+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:08:45.075087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978235+0000","last_change":"2023-08-27T11:29:23.565075+0000","last_active":"2023-08-27T11:29:25.978235+0000","last_peered":"2023-08-27T11:29:25.978235+0000","last_clean":"2023-08-27T11:29:25.978235+0000","last_became_active":"2023-08-27T11:29:23.564808+0000","last_became_peered":"2023-08-27T11:29:23.564808+0000","last_unstale":"2023-08-27T11:29:25.978235+0000","last_undegraded":"2023-08-27T11:29:25.978235+0000","last_fullsized":"2023-08-27T11:29:25.978235+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:56:47.175672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"166'1","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.552246+0000","last_change":"2023-08-27T11:29:22.565149+0000","last_active":"2023-08-27T11:29:24.552246+0000","last_peered":"2023-08-27T11:29:24.552246+0000","last_clean":"2023-08-27T11:29:24.552246+0000","last_became_active":"2023-08-27T11:29:22.564666+0000","last_became_peered":"2023-08-27T11:29:22.564666+0000","last_unstale":"2023-08-27T11:29:24.552246+0000","last_undegraded":"2023-08-27T11:29:24.552246+0000","last_fullsized":"2023-08-27T11:29:24.552246+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:31.502049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":269,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134781+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:29:25.134781+0000","last_peered":"2023-08-27T11:29:25.134781+0000","last_clean":"2023-08-27T11:29:25.134781+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:29:25.134781+0000","last_undegraded":"2023-08-27T11:29:25.134781+0000","last_fullsized":"2023-08-27T11:29:25.134781+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:37:36.873832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978212+0000","last_change":"2023-08-27T11:29:23.569098+0000","last_active":"2023-08-27T11:29:25.978212+0000","last_peered":"2023-08-27T11:29:25.978212+0000","last_clean":"2023-08-27T11:29:25.978212+0000","last_became_active":"2023-08-27T11:29:23.568826+0000","last_became_peered":"2023-08-27T11:29:23.568826+0000","last_unstale":"2023-08-27T11:29:25.978212+0000","last_undegraded":"2023-08-27T11:29:25.978212+0000","last_fullsized":"2023-08-27T11:29:25.978212+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:11:58.076108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"166'4","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477685+0000","last_change":"2023-08-27T11:29:22.565482+0000","last_active":"2023-08-27T11:29:25.477685+0000","last_peered":"2023-08-27T11:29:25.477685+0000","last_clean":"2023-08-27T11:29:25.477685+0000","last_became_active":"2023-08-27T11:29:22.565109+0000","last_became_peered":"2023-08-27T11:29:22.565109+0000","last_unstale":"2023-08-27T11:29:25.477685+0000","last_undegraded":"2023-08-27T11:29:25.477685+0000","last_fullsized":"2023-08-27T11:29:25.477685+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:53:54.762552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":419,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.134524+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:29:25.134524+0000","last_peered":"2023-08-27T11:29:25.134524+0000","last_clean":"2023-08-27T11:29:25.134524+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:29:25.134524+0000","last_undegraded":"2023-08-27T11:29:25.134524+0000","last_fullsized":"2023-08-27T11:29:25.134524+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:17.130505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":279,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548447+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:29:24.548447+0000","last_peered":"2023-08-27T11:29:24.548447+0000","last_clean":"2023-08-27T11:29:24.548447+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:29:24.548447+0000","last_undegraded":"2023-08-27T11:29:24.548447+0000","last_fullsized":"2023-08-27T11:29:24.548447+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:45.832462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.477591+0000","last_change":"2023-08-27T11:29:23.568490+0000","last_active":"2023-08-27T11:29:25.477591+0000","last_peered":"2023-08-27T11:29:25.477591+0000","last_clean":"2023-08-27T11:29:25.477591+0000","last_became_active":"2023-08-27T11:29:23.568217+0000","last_became_peered":"2023-08-27T11:29:23.568217+0000","last_unstale":"2023-08-27T11:29:25.477591+0000","last_undegraded":"2023-08-27T11:29:25.477591+0000","last_fullsized":"2023-08-27T11:29:25.477591+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:58:20.409859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.977838+0000","last_change":"2023-08-27T11:29:22.556485+0000","last_active":"2023-08-27T11:29:25.977838+0000","last_peered":"2023-08-27T11:29:25.977838+0000","last_clean":"2023-08-27T11:29:25.977838+0000","last_became_active":"2023-08-27T11:29:22.555930+0000","last_became_peered":"2023-08-27T11:29:22.555930+0000","last_unstale":"2023-08-27T11:29:25.977838+0000","last_undegraded":"2023-08-27T11:29:25.977838+0000","last_fullsized":"2023-08-27T11:29:25.977838+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:17:45.141474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":271,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978298+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:29:25.978298+0000","last_peered":"2023-08-27T11:29:25.978298+0000","last_clean":"2023-08-27T11:29:25.978298+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:29:25.978298+0000","last_undegraded":"2023-08-27T11:29:25.978298+0000","last_fullsized":"2023-08-27T11:29:25.978298+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:03:27.587573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:24.548529+0000","last_change":"2023-08-27T11:29:23.568812+0000","last_active":"2023-08-27T11:29:24.548529+0000","last_peered":"2023-08-27T11:29:24.548529+0000","last_clean":"2023-08-27T11:29:24.548529+0000","last_became_active":"2023-08-27T11:29:23.568324+0000","last_became_peered":"2023-08-27T11:29:23.568324+0000","last_unstale":"2023-08-27T11:29:24.548529+0000","last_undegraded":"2023-08-27T11:29:24.548529+0000","last_fullsized":"2023-08-27T11:29:24.548529+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:22.536503+0000","last_clean_scrub_stamp":"2023-08-27T11:29:22.536503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:53.650499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.978638+0000","last_change":"2023-08-27T11:29:22.558632+0000","last_active":"2023-08-27T11:29:25.978638+0000","last_peered":"2023-08-27T11:29:25.978638+0000","last_clean":"2023-08-27T11:29:25.978638+0000","last_became_active":"2023-08-27T11:29:22.558040+0000","last_became_peered":"2023-08-27T11:29:22.558040+0000","last_unstale":"2023-08-27T11:29:25.978638+0000","last_undegraded":"2023-08-27T11:29:25.978638+0000","last_fullsized":"2023-08-27T11:29:25.978638+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:29:21.525498+0000","last_clean_scrub_stamp":"2023-08-27T11:29:21.525498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:59:15.707841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":305,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-08-27T11:29:25.979069+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:29:25.979069+0000","last_peered":"2023-08-27T11:29:25.979069+0000","last_clean":"2023-08-27T11:29:25.979069+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:29:25.979069+0000","last_undegraded":"2023-08-27T11:29:25.979069+0000","last_fullsized":"2023-08-27T11:29:25.979069+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:31:38.716999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33836,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338004,"statfs":{"total":96636764160,"available":96602116096,"internally_reserved":0,"allocated":2719744,"data_stored":1599713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.22700000000000001},"max":{"1min":0.60399999999999998,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.51500000000000001},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.52100000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.83399999999999996,"15min":0.872},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.59299999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.42799999999999999,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.67800000000000005,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.90200000000000002},"last":0.60599999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":0.996,"15min":0.996},"last":0.629},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.53000000000000003,"5min":0.42899999999999999,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.68400000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.503},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.72199999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":1.224,"15min":1.224},"last":0.499}]},{"osd":5,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79700000000000004,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.84799999999999998},"last":0.64400000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83799999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.66500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575004,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33272,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338568,"statfs":{"total":96636764160,"available":96602693632,"internally_reserved":0,"allocated":2142208,"data_stored":1010411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.84599999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.39800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.89800000000000002},"last":0.51900000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.53900000000000003},"min":{"1min":0.35599999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.81799999999999995,"15min":0.873},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.441}]},{"osd":2,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.499,"15min":0.51000000000000001},"min":{"1min":0.40799999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":2.0830000000000002},"last":0.55100000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.83199999999999996,"15min":0.84499999999999997},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.34000000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.93799999999999994},"last":0.58399999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":0.71599999999999997,"5min":0.753,"15min":0.83099999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.72699999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75700000000000001,"5min":0.85499999999999998,"15min":0.875},"last":0.69399999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79200000000000004,"5min":1.365,"15min":1.365},"last":0.624},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.34899999999999998},"max":{"1min":0.77700000000000002,"5min":0.82999999999999996,"15min":0.91400000000000003},"last":0.64400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33828,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338012,"statfs":{"total":96636764160,"available":96602124288,"internally_reserved":0,"allocated":2711552,"data_stored":1599041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.315,"15min":0.47399999999999998},"min":{"1min":0.21199999999999999,"5min":0.19500000000000001,"15min":0.191},"max":{"1min":0.45300000000000001,"5min":0.53300000000000003,"15min":11.971},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.315,"5min":0.32700000000000001,"15min":0.64700000000000002},"min":{"1min":0.23000000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.44900000000000001,"5min":0.74399999999999999,"15min":23.045999999999999},"last":0.28899999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34000000000000002,"15min":0.502},"min":{"1min":0.28199999999999997,"5min":0.23300000000000001,"15min":0.221},"max":{"1min":0.42399999999999999,"5min":0.57699999999999996,"15min":11.928000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.33700000000000002},"min":{"1min":0.249,"5min":0.22800000000000001,"15min":0.224},"max":{"1min":0.46899999999999997,"5min":0.52200000000000002,"15min":0.58599999999999997},"last":0.39900000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38400000000000001,"15min":0.52800000000000002},"min":{"1min":0.309,"5min":0.27500000000000002,"15min":0.26300000000000001},"max":{"1min":0.47599999999999998,"5min":0.64100000000000001,"15min":10.99},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36799999999999999,"15min":0.68500000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.503,"5min":0.54700000000000004,"15min":22.893999999999998},"last":0.437}]},{"osd":4,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.33400000000000002},"max":{"1min":0.84999999999999998,"5min":0.90700000000000003,"15min":1.26},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58099999999999996,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.745,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.48199999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.88600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":23.016999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.81899999999999995,"5min":0.88700000000000001,"15min":1.296},"last":0.59099999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58199999999999996,"15min":0.72799999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.371},"max":{"1min":0.72999999999999998,"5min":0.84899999999999998,"15min":10.925000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60199999999999998,"15min":0.90400000000000003},"min":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88,"5min":0.88,"15min":22.952000000000002},"last":0.53300000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59299999999999997,"15min":0.89800000000000002},"min":{"1min":0.433,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":22.988},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":1.327},"last":0.61599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575003,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2113536,"data_stored":1007312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.215,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.497,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.42799999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36199999999999999,"15min":0.34399999999999997},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47699999999999998,"5min":0.628,"15min":0.63200000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34799999999999998,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45800000000000002,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.38100000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35799999999999998,"15min":0.34599999999999997},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.53700000000000003,"15min":0.59599999999999997},"last":0.442},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.376},"min":{"1min":0.30499999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.53000000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.47099999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.34399999999999997},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.214},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.603},"last":0.69899999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79600000000000004,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.878,"15min":0.878},"last":0.73499999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83399999999999996,"5min":3.79,"15min":3.79},"last":0.749}]}]},{"osd":2,"up_from":13,"seq":55834575004,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33256,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2125824,"data_stored":1009289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32100000000000001,"15min":0.318},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.19700000000000001},"max":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.629},"last":0.44500000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41699999999999998,"5min":0.50800000000000001,"15min":0.54400000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34699999999999998},"min":{"1min":0.26200000000000001,"5min":0.23000000000000001,"15min":0.22600000000000001},"max":{"1min":0.49299999999999999,"5min":0.56399999999999995,"15min":0.749},"last":0.311}]},{"osd":3,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.378},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":0.51800000000000002,"5min":0.57099999999999995,"15min":0.73699999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.32600000000000001,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.55100000000000005,"15min":0.69499999999999995},"last":0.42899999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.879},"last":0.66700000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36699999999999999},"max":{"1min":0.83799999999999997,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.65400000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.40000000000000002},"max":{"1min":0.78600000000000003,"5min":0.82699999999999996,"15min":0.83599999999999997},"last":0.68300000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.622}]}]},{"osd":3,"up_from":13,"seq":55834575004,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2121728,"data_stored":1008695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.29999999999999999,"15min":0.30299999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.52500000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.307,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.223,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40899999999999997,"5min":0.755,"15min":0.755},"last":0.48299999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33500000000000002,"15min":0.32300000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":0.40300000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":0.48999999999999999,"5min":2.73,"15min":2.73},"last":0.54300000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.372,"15min":0.36399999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.46400000000000002,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.375,"15min":0.37},"min":{"1min":0.27700000000000002,"5min":0.26700000000000002,"15min":0.26400000000000001},"max":{"1min":0.51000000000000001,"5min":1.179,"15min":1.179},"last":0.58599999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.28599999999999998},"max":{"1min":0.748,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.71799999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.66300000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.57699999999999996},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.745}]},{"osd":6,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.373},"max":{"1min":0.84299999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.497,"5min":0.38,"15min":0.38},"max":{"1min":0.83199999999999996,"5min":0.86899999999999999,"15min":1.111},"last":0.61499999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":1.236},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.75600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2121728,"data_stored":1008763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68200000000000005,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.372},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.502},"min":{"1min":0.33900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66100000000000003,"5min":0.746,"15min":0.76500000000000001},"last":0.497},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.75,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.432}]},{"osd":2,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.40400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.46899999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.96999999999999997,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64500000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71499999999999997,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":1.865,"15min":1.865},"last":0.59099999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90600000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.624,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82799999999999996,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.60599999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.90100000000000002,"5min":2.714,"15min":2.714},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575003,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33264,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338576,"statfs":{"total":96636764160,"available":96602701824,"internally_reserved":0,"allocated":2134016,"data_stored":1010355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.376,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61099999999999999,"5min":0.78900000000000003,"15min":0.96899999999999997},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.44700000000000001,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.77700000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67500000000000004,"5min":0.68400000000000005,"15min":1.075},"last":0.63500000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.373,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.0580000000000001},"last":0.52500000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":1.03},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.42199999999999999,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.77800000000000002,"15min":1.026},"last":0.73199999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.27000000000000002},"max":{"1min":0.63900000000000001,"5min":0.78700000000000003,"15min":0.86899999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.73399999999999999,"5min":0.89400000000000002,"15min":0.91400000000000003},"last":0.47899999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.31},"max":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.88200000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.873,"15min":0.94199999999999995},"last":0.46400000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.34799999999999998},"max":{"1min":0.753,"5min":0.85999999999999999,"15min":0.95899999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.84399999999999997,"15min":0.92000000000000004},"last":0.443}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:29:32.606 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:29:32.607 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:29:32.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.766+0000 7f975a181700 1 Processor -- start 2023-08-27T11:29:32.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.766+0000 7f975a181700 1 -- start start 2023-08-27T11:29:32.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f975a181700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9754146c20 0x7f9754160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f975a181700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97540b5070 con 0x7f975406f790 2023-08-27T11:29:32.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f975a181700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9754147000 con 0x7f97540b3440 2023-08-27T11:29:32.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f975a181700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97541614a0 con 0x7f9754146c20 2023-08-27T11:29:32.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f9753fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f975406f790 0x7f97540b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60848/0 (socket says 172.21.15.112:60848) 2023-08-27T11:29:32.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f9753fff700 1 -- 172.21.15.112:0/3425644326 learned_addr learned my addr 172.21.15.112:0/3425644326 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:32.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129940579 0 0) 0x7f974c001110 con 0x7f975406f790 2023-08-27T11:29:32.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f973c0035b0 con 0x7f975406f790 2023-08-27T11:29:32.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f97537fe700 1 --2- 172.21.15.112:0/3425644326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9754146c20 0x7f9754160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1440530981 0 0) 0x7f973c0035b0 con 0x7f975406f790 2023-08-27T11:29:32.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9754146c20 msgr2=0x7f9754160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:32.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f97527fc700 1 --2- 172.21.15.112:0/3425644326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9754146c20 0x7f9754160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 >> v1:172.21.15.138:6789/0 conn(0x7f97540b3440 legacy=0x7f975415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:32.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9754161740 con 0x7f975406f790 2023-08-27T11:29:32.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f974c001800 con 0x7f975406f790 2023-08-27T11:29:32.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.767+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 >> v1:172.21.15.112:6789/0 conn(0x7f975406f790 legacy=0x7f97540b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:32.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f97527fc700 1 --2- 172.21.15.112:0/3425644326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c003a30 0x7f973c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f97527fc700 1 --2- 172.21.15.112:0/3425644326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9754146c20 0x7f973c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f97527fc700 1 --2- 172.21.15.112:0/3425644326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f973c004d90 0x7f973c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97541614a0 con 0x7f973c004d90 2023-08-27T11:29:32.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c00bb40 con 0x7f973c003a30 2023-08-27T11:29:32.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c00bd00 con 0x7f9754146c20 2023-08-27T11:29:32.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f9753fff700 1 --2- 172.21.15.112:0/3425644326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f973c004d90 0x7f973c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f97537fe700 1 --2- 172.21.15.112:0/3425644326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c003a30 0x7f973c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f9752ffd700 1 --2- 172.21.15.112:0/3425644326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9754146c20 0x7f973c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f9753fff700 1 -- 172.21.15.112:0/3425644326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9754146c20 msgr2=0x7f973c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f9753fff700 1 --2- 172.21.15.112:0/3425644326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9754146c20 0x7f973c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f9753fff700 1 -- 172.21.15.112:0/3425644326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c003a30 msgr2=0x7f973c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f9753fff700 1 --2- 172.21.15.112:0/3425644326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c003a30 0x7f973c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f9753fff700 1 -- 172.21.15.112:0/3425644326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0035b0 con 0x7f973c004d90 2023-08-27T11:29:32.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f9753fff700 1 --2- 172.21.15.112:0/3425644326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f973c004d90 0x7f973c005210 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f974c00c240 tx=0x7f973c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cd7029f9ca9f5a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:32.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c007db0 con 0x7f973c004d90 2023-08-27T11:29:32.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c00cde0 con 0x7f973c004d90 2023-08-27T11:29:32.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.768+0000 7f97527fc700 1 -- 172.21.15.112:0/3425644326 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c005b80 con 0x7f973c004d90 2023-08-27T11:29:32.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.769+0000 7f975a181700 1 -- 172.21.15.112:0/3425644326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f973c004d90 msgr2=0x7f973c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:32.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.769+0000 7f975a181700 1 --2- 172.21.15.112:0/3425644326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f973c004d90 0x7f973c005210 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f974c00c240 tx=0x7f973c005880 comp rx=0 tx=0).stop 2023-08-27T11:29:32.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.769+0000 7f9753fff700 1 -- 172.21.15.112:0/3425644326 reap_dead start 2023-08-27T11:29:32.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.769+0000 7f975a181700 1 -- 172.21.15.112:0/3425644326 shutdown_connections 2023-08-27T11:29:32.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.769+0000 7f975a181700 1 -- 172.21.15.112:0/3425644326 >> 172.21.15.112:0/3425644326 conn(0x7f97540b1e90 msgr2=0x7f97540b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:32.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.769+0000 7f975a181700 1 -- 172.21.15.112:0/3425644326 shutdown_connections 2023-08-27T11:29:32.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.769+0000 7f975a181700 1 -- 172.21.15.112:0/3425644326 wait complete. 2023-08-27T11:29:32.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.770+0000 7f975a181700 1 Processor -- start 2023-08-27T11:29:32.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.770+0000 7f975a181700 1 -- start start 2023-08-27T11:29:32.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f975a181700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f973c004d90 0x7f975413fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f975a181700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97540cf4c0 0x7f97541402d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f975a181700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9754140810 0x7f9754140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f975a181700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9754143cd0 con 0x7f97540cf4c0 2023-08-27T11:29:32.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f975a181700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9754141940 con 0x7f9754140810 2023-08-27T11:29:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f975a181700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9754070fd0 con 0x7f973c004d90 2023-08-27T11:29:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f9752ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97540cf4c0 0x7f97541402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f9752ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97540cf4c0 0x7f97541402d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38244/0 (socket says 172.21.15.112:38244) 2023-08-27T11:29:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f9752ffd700 1 -- 172.21.15.112:0/250749258 learned_addr learned my addr 172.21.15.112:0/250749258 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f9753fff700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9754140810 0x7f9754140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f97537fe700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f973c004d90 0x7f975413fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f9752ffd700 1 -- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f973c004d90 msgr2=0x7f975413fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:32.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f9752ffd700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f973c004d90 0x7f975413fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f9752ffd700 1 -- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9754140810 msgr2=0x7f9754140ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:32.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f9752ffd700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9754140810 0x7f9754140ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:32.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f9752ffd700 1 -- 172.21.15.112:0/250749258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97540cf920 con 0x7f97540cf4c0 2023-08-27T11:29:32.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.771+0000 7f9753fff700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9754140810 0x7f9754140ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:32.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.772+0000 7f9752ffd700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97540cf4c0 0x7f97541402d0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f97440020e0 tx=0x7f974400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:32.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.772+0000 7f973bfff700 1 -- 172.21.15.112:0/250749258 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974400dd10 con 0x7f97540cf4c0 2023-08-27T11:29:32.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.772+0000 7f973bfff700 1 -- 172.21.15.112:0/250749258 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974400de70 con 0x7f97540cf4c0 2023-08-27T11:29:32.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.772+0000 7f975a181700 1 -- 172.21.15.112:0/250749258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9754141380 con 0x7f97540cf4c0 2023-08-27T11:29:32.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.772+0000 7f973bfff700 1 -- 172.21.15.112:0/250749258 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9744003c20 con 0x7f97540cf4c0 2023-08-27T11:29:32.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.772+0000 7f975a181700 1 -- 172.21.15.112:0/250749258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97541438f0 con 0x7f97540cf4c0 2023-08-27T11:29:32.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.773+0000 7f975a181700 1 -- 172.21.15.112:0/250749258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9754143ab0 con 0x7f97540cf4c0 2023-08-27T11:29:32.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.773+0000 7f973bfff700 1 -- 172.21.15.112:0/250749258 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f97440050d0 con 0x7f97540cf4c0 2023-08-27T11:29:32.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.773+0000 7f973bfff700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f974006a4d0 0x7f974006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:32.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.773+0000 7f973bfff700 1 -- 172.21.15.112:0/250749258 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9744089290 con 0x7f97540cf4c0 2023-08-27T11:29:32.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.774+0000 7f97537fe700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f974006a4d0 0x7f974006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:32.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.774+0000 7f97537fe700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f974006a4d0 0x7f974006c990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9748009ba0 tx=0x7f9748008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:32.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:32.778+0000 7f973bfff700 1 -- 172.21.15.112:0/250749258 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9744057e30 con 0x7f97540cf4c0 2023-08-27T11:29:33.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.045+0000 7f975a181700 1 -- 172.21.15.112:0/250749258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9754143e10 con 0x7f97540cf4c0 2023-08-27T11:29:33.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.049+0000 7f973bfff700 1 -- 172.21.15.112:0/250749258 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7f9754143e10 con 0x7f97540cf4c0 2023-08-27T11:29:33.051 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-08-27T11:29:33.051 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA== 2023-08-27T11:29:33.051 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2023-08-27T11:29:33.052 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2023-08-27T11:29:33.052 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:29:33.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.052+0000 7f975a181700 1 -- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f974006a4d0 msgr2=0x7f974006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:33.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.052+0000 7f975a181700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f974006a4d0 0x7f974006c990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9748009ba0 tx=0x7f9748008040 comp rx=0 tx=0).stop 2023-08-27T11:29:33.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.052+0000 7f975a181700 1 -- 172.21.15.112:0/250749258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97540cf4c0 msgr2=0x7f97541402d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:33.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.052+0000 7f975a181700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97540cf4c0 0x7f97541402d0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f97440020e0 tx=0x7f974400d430 comp rx=0 tx=0).stop 2023-08-27T11:29:33.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.052+0000 7f975a181700 1 -- 172.21.15.112:0/250749258 shutdown_connections 2023-08-27T11:29:33.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.052+0000 7f975a181700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f974006a4d0 0x7f974006c990 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.052+0000 7f975a181700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9754140810 0x7f9754140ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.052+0000 7f975a181700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f973c004d90 0x7f975413fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.052+0000 7f975a181700 1 --2- 172.21.15.112:0/250749258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97540cf4c0 0x7f97541402d0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.052+0000 7f975a181700 1 -- 172.21.15.112:0/250749258 >> 172.21.15.112:0/250749258 conn(0x7f97540b1e90 msgr2=0x7f97540b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:33.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.052+0000 7f975a181700 1 -- 172.21.15.112:0/250749258 shutdown_connections 2023-08-27T11:29:33.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.052+0000 7f975a181700 1 -- 172.21.15.112:0/250749258 wait complete. 2023-08-27T11:29:33.056 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2023-08-27T11:29:33.066 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:33.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.184+0000 7fcf9074b700 1 Processor -- start 2023-08-27T11:29:33.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.184+0000 7fcf9074b700 1 -- start start 2023-08-27T11:29:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf9074b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcf880b3de0 0x7fcf8806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf9074b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcf8806c510 0x7fcf880cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf9074b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf880cef20 con 0x7fcf88144b80 2023-08-27T11:29:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf9074b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf880b4b40 con 0x7fcf880b3de0 2023-08-27T11:29:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf9074b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf880b7310 con 0x7fcf8806c510 2023-08-27T11:29:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf8dce6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcf880b3de0 0x7fcf8806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf8dce6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcf880b3de0 0x7fcf8806bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38252/0 (socket says 172.21.15.112:38252) 2023-08-27T11:29:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf8dce6700 1 -- 172.21.15.112:0/3868132465 learned_addr learned my addr 172.21.15.112:0/3868132465 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf8ece8700 1 --2- 172.21.15.112:0/3868132465 >> v2:172.21.15.138:3300/0 conn(0x7fcf8806c510 0x7fcf880cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf8dce6700 1 -- 172.21.15.112:0/3868132465 >> v2:172.21.15.138:3300/0 conn(0x7fcf8806c510 msgr2=0x7fcf880cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf8dce6700 1 --2- 172.21.15.112:0/3868132465 >> v2:172.21.15.138:3300/0 conn(0x7fcf8806c510 0x7fcf880cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf8dce6700 1 -- 172.21.15.112:0/3868132465 >> v1:172.21.15.138:6789/0 conn(0x7fcf88144b80 legacy=0x7fcf880b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf8dce6700 1 -- 172.21.15.112:0/3868132465 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8806c910 con 0x7fcf880b3de0 2023-08-27T11:29:33.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.185+0000 7fcf8dce6700 1 --2- 172.21.15.112:0/3868132465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf880b3de0 0x7fcf8806bfd0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fcf88074170 tx=0x7fcf84009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d428892011f9676 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:33.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.186+0000 7fcf8d4e5700 1 -- 172.21.15.112:0/3868132465 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84004230 con 0x7fcf880b3de0 2023-08-27T11:29:33.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.186+0000 7fcf8d4e5700 1 -- 172.21.15.112:0/3868132465 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf84004390 con 0x7fcf880b3de0 2023-08-27T11:29:33.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.186+0000 7fcf8d4e5700 1 -- 172.21.15.112:0/3868132465 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf840036a0 con 0x7fcf880b3de0 2023-08-27T11:29:33.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.186+0000 7fcf9074b700 1 -- 172.21.15.112:0/3868132465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf880b3de0 msgr2=0x7fcf8806bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:33.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.186+0000 7fcf9074b700 1 --2- 172.21.15.112:0/3868132465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf880b3de0 0x7fcf8806bfd0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fcf88074170 tx=0x7fcf84009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.186+0000 7fcf9074b700 1 -- 172.21.15.112:0/3868132465 shutdown_connections 2023-08-27T11:29:33.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.186+0000 7fcf9074b700 1 --2- 172.21.15.112:0/3868132465 >> v2:172.21.15.138:3300/0 conn(0x7fcf8806c510 0x7fcf880cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.186+0000 7fcf9074b700 1 --2- 172.21.15.112:0/3868132465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf880b3de0 0x7fcf8806bfd0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.186+0000 7fcf9074b700 1 -- 172.21.15.112:0/3868132465 >> 172.21.15.112:0/3868132465 conn(0x7fcf880cdc50 msgr2=0x7fcf880ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:33.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.186+0000 7fcf9074b700 1 -- 172.21.15.112:0/3868132465 shutdown_connections 2023-08-27T11:29:33.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.186+0000 7fcf9074b700 1 -- 172.21.15.112:0/3868132465 wait complete. 2023-08-27T11:29:33.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.188+0000 7fcf9074b700 1 Processor -- start 2023-08-27T11:29:33.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.188+0000 7fcf9074b700 1 -- start start 2023-08-27T11:29:33.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf9074b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf8806c510 0x7fcf881576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf9074b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf880b3de0 0x7fcf8815b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.196 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf9074b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf88144b80 0x7fcf88136080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf9074b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf88157290 con 0x7fcf8806c510 2023-08-27T11:29:33.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf9074b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf88156610 con 0x7fcf88144b80 2023-08-27T11:29:33.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf9074b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf881568f0 con 0x7fcf880b3de0 2023-08-27T11:29:33.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf8e4e7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf8806c510 0x7fcf881576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf8e4e7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf8806c510 0x7fcf881576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38256/0 (socket says 172.21.15.112:38256) 2023-08-27T11:29:33.197 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf8e4e7700 1 -- 172.21.15.112:0/2620264703 learned_addr learned my addr 172.21.15.112:0/2620264703 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:33.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf8dce6700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf880b3de0 0x7fcf8815b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf8ece8700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf88144b80 0x7fcf88136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf8e4e7700 1 -- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf880b3de0 msgr2=0x7fcf8815b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:33.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf8e4e7700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf880b3de0 0x7fcf8815b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.198 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf8e4e7700 1 -- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf88144b80 msgr2=0x7fcf88136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:33.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf8e4e7700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf88144b80 0x7fcf88136080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf8e4e7700 1 -- 172.21.15.112:0/2620264703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf84009550 con 0x7fcf8806c510 2023-08-27T11:29:33.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf8e4e7700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf8806c510 0x7fcf881576b0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800b4a0 tx=0x7fcf7800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:33.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf7effd700 1 -- 172.21.15.112:0/2620264703 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf78011830 con 0x7fcf8806c510 2023-08-27T11:29:33.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf7effd700 1 -- 172.21.15.112:0/2620264703 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf78011990 con 0x7fcf8806c510 2023-08-27T11:29:33.199 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf9074b700 1 -- 172.21.15.112:0/2620264703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf88156bd0 con 0x7fcf8806c510 2023-08-27T11:29:33.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.189+0000 7fcf7effd700 1 -- 172.21.15.112:0/2620264703 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf78011d40 con 0x7fcf8806c510 2023-08-27T11:29:33.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.190+0000 7fcf9074b700 1 -- 172.21.15.112:0/2620264703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf881365c0 con 0x7fcf8806c510 2023-08-27T11:29:33.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.190+0000 7fcf9074b700 1 -- 172.21.15.112:0/2620264703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf88157500 con 0x7fcf8806c510 2023-08-27T11:29:33.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.191+0000 7fcf7effd700 1 -- 172.21.15.112:0/2620264703 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fcf7801a400 con 0x7fcf8806c510 2023-08-27T11:29:33.200 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.191+0000 7fcf7effd700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcf6c06a4d0 0x7fcf6c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.191+0000 7fcf7effd700 1 -- 172.21.15.112:0/2620264703 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcf780889a0 con 0x7fcf8806c510 2023-08-27T11:29:33.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.191+0000 7fcf8dce6700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcf6c06a4d0 0x7fcf6c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.192+0000 7fcf8dce6700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcf6c06a4d0 0x7fcf6c06c990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009130 tx=0x7fcf84008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:33.201 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.195+0000 7fcf7effd700 1 -- 172.21.15.112:0/2620264703 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcf78053b40 con 0x7fcf8806c510 2023-08-27T11:29:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.427+0000 7fcf9074b700 1 -- 172.21.15.112:0/2620264703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf88067a20 con 0x7fcf8806c510 2023-08-27T11:29:33.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.427+0000 7fcf7effd700 1 -- 172.21.15.112:0/2620264703 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4894 (secure 0 0 0) 0x7fcf78057160 con 0x7fcf8806c510 2023-08-27T11:29:33.429 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:33.430 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":113,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/685371648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":685371648},{"type":"v1","addr":"172.21.15.138:6836","nonce":685371648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":7214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3585152138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3585152138},{"type":"v1","addr":"172.21.15.138:6837","nonce":3585152138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":8203,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2190275630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2190275630},{"type":"v1","addr":"172.21.15.112:6835","nonce":2190275630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:29:22.938324+0000","modified":"2023-08-27T11:29:26.571479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6833/4086629952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4086629952},{"type":"v1","addr":"172.21.15.112:6833","nonce":4086629952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-08-27T11:29:33.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.429+0000 7fcf9074b700 1 -- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcf6c06a4d0 msgr2=0x7fcf6c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:33.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.429+0000 7fcf9074b700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcf6c06a4d0 0x7fcf6c06c990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009130 tx=0x7fcf84008030 comp rx=0 tx=0).stop 2023-08-27T11:29:33.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.429+0000 7fcf9074b700 1 -- 172.21.15.112:0/2620264703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf8806c510 msgr2=0x7fcf881576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:33.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.429+0000 7fcf9074b700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf8806c510 0x7fcf881576b0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800b4a0 tx=0x7fcf7800b860 comp rx=0 tx=0).stop 2023-08-27T11:29:33.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.429+0000 7fcf9074b700 1 -- 172.21.15.112:0/2620264703 shutdown_connections 2023-08-27T11:29:33.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.429+0000 7fcf9074b700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcf6c06a4d0 0x7fcf6c06c990 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.429+0000 7fcf9074b700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf88144b80 0x7fcf88136080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.429+0000 7fcf9074b700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf8806c510 0x7fcf881576b0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.429+0000 7fcf9074b700 1 --2- 172.21.15.112:0/2620264703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf880b3de0 0x7fcf8815b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.429+0000 7fcf9074b700 1 -- 172.21.15.112:0/2620264703 >> 172.21.15.112:0/2620264703 conn(0x7fcf880cdc50 msgr2=0x7fcf880c6170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:33.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.429+0000 7fcf9074b700 1 -- 172.21.15.112:0/2620264703 shutdown_connections 2023-08-27T11:29:33.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.429+0000 7fcf9074b700 1 -- 172.21.15.112:0/2620264703 wait complete. 2023-08-27T11:29:33.434 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 113 2023-08-27T11:29:33.845 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:29:22.938324+0000', 'modified': '2023-08-27T11:29:26.571479+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9393}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9393': {'gid': 9393, 'name': 'c', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6833/4086629952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6832', 'nonce': 4086629952}, {'type': 'v1', 'addr': '172.21.15.112:6833', 'nonce': 4086629952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:29:33.846 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:29:33.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-27T11:29:33.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.967+0000 7f4e5ffba700 1 Processor -- start 2023-08-27T11:29:33.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.967+0000 7f4e5ffba700 1 -- start start 2023-08-27T11:29:33.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5ffba700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58144570 0x7f4e5815ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5ffba700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e580b45f0 con 0x7f4e580b3e50 2023-08-27T11:29:33.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5ffba700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e580b4310 con 0x7f4e580b4ad0 2023-08-27T11:29:33.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5ffba700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580cf7a0 con 0x7f4e58144570 2023-08-27T11:29:33.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5d555700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4e580b3e50 0x7f4e5815f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60878/0 (socket says 172.21.15.112:60878) 2023-08-27T11:29:33.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5d555700 1 -- 172.21.15.112:0/364828553 learned_addr learned my addr 172.21.15.112:0/364828553 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:33.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/364828553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58144570 0x7f4e5815ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896885920 0 0) 0x7f4e580b45f0 con 0x7f4e580b3e50 2023-08-27T11:29:33.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e440035b0 con 0x7f4e580b3e50 2023-08-27T11:29:33.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2503613152 0 0) 0x7f4e440035b0 con 0x7f4e580b3e50 2023-08-27T11:29:33.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58144570 msgr2=0x7f4e5815ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:33.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5cd54700 1 --2- 172.21.15.112:0/364828553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58144570 0x7f4e5815ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 >> v1:172.21.15.138:6789/0 conn(0x7f4e580b4ad0 legacy=0x7f4e580b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:33.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e580cdcb0 con 0x7f4e580b3e50 2023-08-27T11:29:33.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.968+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f4e48001670 con 0x7f4e580b3e50 2023-08-27T11:29:33.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 >> v1:172.21.15.112:6789/0 conn(0x7f4e580b3e50 legacy=0x7f4e5815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:33.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5cd54700 1 --2- 172.21.15.112:0/364828553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e44003a30 0x7f4e44003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5cd54700 1 --2- 172.21.15.112:0/364828553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58144570 0x7f4e44004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5cd54700 1 --2- 172.21.15.112:0/364828553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e44004d90 0x7f4e44005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580cf7a0 con 0x7f4e44003a30 2023-08-27T11:29:33.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4400bb40 con 0x7f4e44004d90 2023-08-27T11:29:33.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4400bd00 con 0x7f4e58144570 2023-08-27T11:29:33.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/364828553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e44003a30 0x7f4e44003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5d555700 1 --2- 172.21.15.112:0/364828553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58144570 0x7f4e44004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5e557700 1 --2- 172.21.15.112:0/364828553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e44004d90 0x7f4e44005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5dd56700 1 -- 172.21.15.112:0/364828553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58144570 msgr2=0x7f4e44004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:33.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/364828553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58144570 0x7f4e44004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5dd56700 1 -- 172.21.15.112:0/364828553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e44004d90 msgr2=0x7f4e44005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:33.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/364828553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e44004d90 0x7f4e44005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5dd56700 1 -- 172.21.15.112:0/364828553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e440035b0 con 0x7f4e44003a30 2023-08-27T11:29:33.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5d555700 1 --2- 172.21.15.112:0/364828553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58144570 0x7f4e44004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:33.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/364828553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e44003a30 0x7f4e44003e50 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009b60 tx=0x7f4e44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a90d11a08dd7ee3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:33.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400e6c0 con 0x7f4e44003a30 2023-08-27T11:29:33.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5400e820 con 0x7f4e44003a30 2023-08-27T11:29:33.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.969+0000 7f4e5cd54700 1 -- 172.21.15.112:0/364828553 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400eb90 con 0x7f4e44003a30 2023-08-27T11:29:33.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.970+0000 7f4e5ffba700 1 -- 172.21.15.112:0/364828553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e44003a30 msgr2=0x7f4e44003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:33.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.970+0000 7f4e5ffba700 1 --2- 172.21.15.112:0/364828553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e44003a30 0x7f4e44003e50 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009b60 tx=0x7f4e44005880 comp rx=0 tx=0).stop 2023-08-27T11:29:33.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.970+0000 7f4e5e557700 1 -- 172.21.15.112:0/364828553 reap_dead start 2023-08-27T11:29:33.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.970+0000 7f4e5ffba700 1 -- 172.21.15.112:0/364828553 shutdown_connections 2023-08-27T11:29:33.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.970+0000 7f4e5ffba700 1 -- 172.21.15.112:0/364828553 >> 172.21.15.112:0/364828553 conn(0x7f4e5806f970 msgr2=0x7f4e5806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:33.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.970+0000 7f4e5ffba700 1 -- 172.21.15.112:0/364828553 shutdown_connections 2023-08-27T11:29:33.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.970+0000 7f4e5ffba700 1 -- 172.21.15.112:0/364828553 wait complete. 2023-08-27T11:29:33.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.971+0000 7f4e5ffba700 1 Processor -- start 2023-08-27T11:29:33.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.971+0000 7f4e5ffba700 1 -- start start 2023-08-27T11:29:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5ffba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e44003a30 0x7f4e581407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5ffba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e5813cbb0 0x7f4e581421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5ffba700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58142710 0x7f4e581f0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5ffba700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58141d40 con 0x7f4e44003a30 2023-08-27T11:29:33.989 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5ffba700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58141130 con 0x7f4e5813cbb0 2023-08-27T11:29:33.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5ffba700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58141410 con 0x7f4e58142710 2023-08-27T11:29:33.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5dd56700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e44003a30 0x7f4e581407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5dd56700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e44003a30 0x7f4e581407c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38276/0 (socket says 172.21.15.112:38276) 2023-08-27T11:29:33.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5dd56700 1 -- 172.21.15.112:0/3869509604 learned_addr learned my addr 172.21.15.112:0/3869509604 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:33.990 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5d555700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e5813cbb0 0x7f4e581421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5e557700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58142710 0x7f4e581f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5dd56700 1 -- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58142710 msgr2=0x7f4e581f0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:33.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58142710 0x7f4e581f0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5dd56700 1 -- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e5813cbb0 msgr2=0x7f4e581421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:33.991 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e5813cbb0 0x7f4e581421d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5dd56700 1 -- 172.21.15.112:0/3869509604 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e581416f0 con 0x7f4e44003a30 2023-08-27T11:29:33.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5e557700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58142710 0x7f4e581f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:33.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e44003a30 0x7f4e581407c0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400bb10 tx=0x7f4e540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:33.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e4e7fc700 1 -- 172.21.15.112:0/3869509604 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400e6c0 con 0x7f4e44003a30 2023-08-27T11:29:33.992 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e4e7fc700 1 -- 172.21.15.112:0/3869509604 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e54009ed0 con 0x7f4e44003a30 2023-08-27T11:29:33.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5ffba700 1 -- 172.21.15.112:0/3869509604 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e581419d0 con 0x7f4e44003a30 2023-08-27T11:29:33.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e4e7fc700 1 -- 172.21.15.112:0/3869509604 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54003ae0 con 0x7f4e44003a30 2023-08-27T11:29:33.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.972+0000 7f4e5ffba700 1 -- 172.21.15.112:0/3869509604 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e58140d00 con 0x7f4e44003a30 2023-08-27T11:29:33.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.974+0000 7f4e4e7fc700 1 -- 172.21.15.112:0/3869509604 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4e54003c40 con 0x7f4e44003a30 2023-08-27T11:29:33.993 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.974+0000 7f4e4e7fc700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e3c06a4d0 0x7f4e3c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.974+0000 7f4e4e7fc700 1 -- 172.21.15.112:0/3869509604 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e5408a550 con 0x7f4e44003a30 2023-08-27T11:29:33.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.974+0000 7f4e5d555700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e3c06a4d0 0x7f4e3c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.975+0000 7f4e5d555700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e3c06a4d0 0x7f4e3c06c990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4e48001c90 tx=0x7f4e48007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:33.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.976+0000 7f4e6c17cb80 1 Processor -- start 2023-08-27T11:29:33.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.976+0000 7f4e6c17cb80 1 -- start start 2023-08-27T11:29:33.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e6c17cb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971e1b0 0x55baf971e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e6c17cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf97651d0 0x55baf971a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e6c17cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf971c3d0 0x55baf9745520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:33.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e6c17cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf9677730 con 0x55baf97651d0 2023-08-27T11:29:33.995 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e6c17cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf9675050 con 0x55baf971c3d0 2023-08-27T11:29:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e6c17cb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55baf9670940 con 0x55baf971e1b0 2023-08-27T11:29:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e5d555700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf97651d0 0x55baf971a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e5d555700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf97651d0 0x55baf971a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38286/0 (socket says 172.21.15.112:38286) 2023-08-27T11:29:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e5d555700 1 -- 172.21.15.112:0/1253641114 learned_addr learned my addr 172.21.15.112:0/1253641114 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e5e557700 1 --2- 172.21.15.112:0/1253641114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf971c3d0 0x55baf9745520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/1253641114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971e1b0 0x55baf971e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e5d555700 1 -- 172.21.15.112:0/1253641114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971e1b0 msgr2=0x55baf971e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e5d555700 1 --2- 172.21.15.112:0/1253641114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971e1b0 0x55baf971e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e5d555700 1 -- 172.21.15.112:0/1253641114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf971c3d0 msgr2=0x55baf9745520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e5d555700 1 --2- 172.21.15.112:0/1253641114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf971c3d0 0x55baf9745520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e5d555700 1 -- 172.21.15.112:0/1253641114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55baf94f05e0 con 0x55baf97651d0 2023-08-27T11:29:33.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e5e557700 1 --2- 172.21.15.112:0/1253641114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf971c3d0 0x55baf9745520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:33.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.977+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/1253641114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971e1b0 0x55baf971e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:33.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.978+0000 7f4e5d555700 1 --2- 172.21.15.112:0/1253641114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf97651d0 0x55baf971a060 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f4e48011b60 tx=0x7f4e48014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930c87e9eb37e8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:33.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.978+0000 7f4e3b7fe700 1 -- 172.21.15.112:0/1253641114 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48015b10 con 0x55baf97651d0 2023-08-27T11:29:33.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.978+0000 7f4e3b7fe700 1 -- 172.21.15.112:0/1253641114 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e48015c70 con 0x55baf97651d0 2023-08-27T11:29:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.978+0000 7f4e3b7fe700 1 -- 172.21.15.112:0/1253641114 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480143d0 con 0x55baf97651d0 2023-08-27T11:29:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.978+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/1253641114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf97651d0 msgr2=0x55baf971a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.978+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/1253641114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf97651d0 0x55baf971a060 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f4e48011b60 tx=0x7f4e48014000 comp rx=0 tx=0).stop 2023-08-27T11:29:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.978+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/1253641114 shutdown_connections 2023-08-27T11:29:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.978+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/1253641114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf971c3d0 0x55baf9745520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.978+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/1253641114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971e1b0 0x55baf971e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.978+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/1253641114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf97651d0 0x55baf971a060 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.978+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/1253641114 >> 172.21.15.112:0/1253641114 conn(0x55baf967cc30 msgr2=0x55baf9763d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.978+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/1253641114 shutdown_connections 2023-08-27T11:29:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.978+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/1253641114 wait complete. 2023-08-27T11:29:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.979+0000 7f4e6c17cb80 1 -- start start 2023-08-27T11:29:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.979+0000 7f4e6c17cb80 1 Processor -- start 2023-08-27T11:29:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e6c17cb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971c3d0 0x55baf974c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e6c17cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf9744380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e6c17cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf97651d0 0x55baf9767490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e6c17cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf960eb60 con 0x55baf971e1b0 2023-08-27T11:29:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e6c17cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf960e860 con 0x55baf97651d0 2023-08-27T11:29:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e6c17cb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55baf94f28d0 con 0x55baf971c3d0 2023-08-27T11:29:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e6c17cb80 10 client.0 _ll_register_callbacks cb 0x55baf960c630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:29:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e5d555700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf9744380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e5d555700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf9744380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38292/0 (socket says 172.21.15.112:38292) 2023-08-27T11:29:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e5d555700 1 -- 172.21.15.112:0/184941750 learned_addr learned my addr 172.21.15.112:0/184941750 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e5e557700 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf97651d0 0x55baf9767490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971c3d0 0x55baf974c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e5d555700 1 -- 172.21.15.112:0/184941750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971c3d0 msgr2=0x55baf974c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:34.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e5d555700 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971c3d0 0x55baf974c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e5d555700 1 -- 172.21.15.112:0/184941750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf97651d0 msgr2=0x55baf9767490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:34.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e5d555700 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf97651d0 0x55baf9767490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e5d555700 1 -- 172.21.15.112:0/184941750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4800f040 con 0x55baf971e1b0 2023-08-27T11:29:34.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.980+0000 7f4e5e557700 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf97651d0 0x55baf9767490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:34.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e5d555700 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf9744380 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f4e48011b00 tx=0x7f4e48001c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ddccd71bf2a809 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:34.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:29:34.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e38ff9700 1 -- 172.21.15.112:0/184941750 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48015b10 con 0x55baf971e1b0 2023-08-27T11:29:34.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e38ff9700 1 -- 172.21.15.112:0/184941750 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e4801d050 con 0x55baf971e1b0 2023-08-27T11:29:34.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/184941750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55baf97114f0 con 0x55baf971e1b0 2023-08-27T11:29:34.005 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e38ff9700 1 -- 172.21.15.112:0/184941750 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e480059a0 con 0x55baf971e1b0 2023-08-27T11:29:34.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e6c17cb80 10 client.9570 fetch_fsmap learned FSMap version 113 2023-08-27T11:29:34.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/184941750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x55baf9688750 con 0x55baf971e1b0 2023-08-27T11:29:34.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e38ff9700 1 -- 172.21.15.112:0/184941750 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55baf97114f0 con 0x55baf971e1b0 2023-08-27T11:29:34.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e38ff9700 1 -- 172.21.15.112:0/184941750 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e48024d20 con 0x55baf971e1b0 2023-08-27T11:29:34.006 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e6c17cb80 10 client.9570 fetch_fsmap finished waiting for FSMap version 113 2023-08-27T11:29:34.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e6c17cb80 10 client.9570 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:29:34.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e6c17cb80 20 client.9570 populate_metadata read hostname 'smithi112' 2023-08-27T11:29:34.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.112:6832/4086629952,v1:172.21.15.112:6833/4086629952] conn(0x55baf974a480 0x55baf974a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e6c17cb80 4 client.9570 mds_command: new command op to 9393 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:29:34.007 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/184941750 --> [v2:172.21.15.112:6832/4086629952,v1:172.21.15.112:6833/4086629952] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55baf9669920 con 0x55baf974a480 2023-08-27T11:29:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.981+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.112:6832/4086629952,v1:172.21.15.112:6833/4086629952] conn(0x55baf974a480 0x55baf974a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e38ff9700 10 client.9570 ms_handle_connect on v2:172.21.15.112:6832/4086629952 2023-08-27T11:29:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.112:6832/4086629952,v1:172.21.15.112:6833/4086629952] conn(0x55baf974a480 0x55baf974a860 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c250828d5e217d8 server_cookie=a80f042fc37e3dce in_seq=0 out_seq=0 2023-08-27T11:29:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e38ff9700 1 -- 172.21.15.112:0/184941750 <== mds.0 v2:172.21.15.112:6832/4086629952 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55baf9669920 con 0x55baf974a480 2023-08-27T11:29:34.008 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e38ff9700 10 client.9570 handle_command_reply: tid=0 2023-08-27T11:29:34.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e6c17cb80 1 client.9570 shutdown 2023-08-27T11:29:34.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/184941750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 msgr2=0x55baf9744380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:34.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf9744380 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f4e48011b00 tx=0x7f4e48001c00 comp rx=0 tx=0).stop 2023-08-27T11:29:34.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/184941750 shutdown_connections 2023-08-27T11:29:34.009 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.112:6832/4086629952,v1:172.21.15.112:6833/4086629952] conn(0x55baf974a480 0x55baf974a860 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf97651d0 0x55baf9767490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971c3d0 0x55baf974c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e38ff9700 0 client.9570 ms_handle_reset on v2:172.21.15.112:6832/4086629952 2023-08-27T11:29:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/184941750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf9744380 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.010 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.982+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/184941750 >> 172.21.15.112:0/184941750 conn(0x55baf967cc30 msgr2=0x55baf9719070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:34.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.983+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/184941750 shutdown_connections 2023-08-27T11:29:34.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.983+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/184941750 wait complete. 2023-08-27T11:29:34.011 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:33.983+0000 7f4e6c17cb80 20 client.9570 trim_cache size 0 max 16384 2023-08-27T11:29:34.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.000+0000 7f4e6c17cb80 1 Processor -- start 2023-08-27T11:29:34.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.000+0000 7f4e6c17cb80 1 -- start start 2023-08-27T11:29:34.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e6c17cb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971c3d0 0x55baf9719150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e6c17cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf9717d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.017 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e6c17cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf97651d0 0x55baf97182d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e6c17cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf9781960 con 0x55baf971e1b0 2023-08-27T11:29:34.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e6c17cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf971aa60 con 0x55baf97651d0 2023-08-27T11:29:34.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e6c17cb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55baf9763b00 con 0x55baf971c3d0 2023-08-27T11:29:34.018 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e5d555700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf9717d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e5d555700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf9717d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38300/0 (socket says 172.21.15.112:38300) 2023-08-27T11:29:34.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e5d555700 1 -- 172.21.15.112:0/2329703323 learned_addr learned my addr 172.21.15.112:0/2329703323 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:34.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e5e557700 1 --2- 172.21.15.112:0/2329703323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf97651d0 0x55baf97182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e5d555700 1 -- 172.21.15.112:0/2329703323 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971c3d0 msgr2=0x55baf9719150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:34.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e5d555700 1 --2- 172.21.15.112:0/2329703323 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971c3d0 0x55baf9719150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.019 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e5d555700 1 -- 172.21.15.112:0/2329703323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf97651d0 msgr2=0x55baf97182d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:34.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e5d555700 1 --2- 172.21.15.112:0/2329703323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf97651d0 0x55baf97182d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e5d555700 1 -- 172.21.15.112:0/2329703323 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55baf949cd90 con 0x55baf971e1b0 2023-08-27T11:29:34.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.001+0000 7f4e5d555700 1 --2- 172.21.15.112:0/2329703323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf9717d90 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f4e480017a0 tx=0x7f4e4802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef6a56e29f8dfe server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:34.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.002+0000 7f4e3bfff700 1 -- 172.21.15.112:0/2329703323 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48015b10 con 0x55baf971e1b0 2023-08-27T11:29:34.020 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.002+0000 7f4e3bfff700 1 -- 172.21.15.112:0/2329703323 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e480242a0 con 0x55baf971e1b0 2023-08-27T11:29:34.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.002+0000 7f4e3bfff700 1 -- 172.21.15.112:0/2329703323 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48024da0 con 0x55baf971e1b0 2023-08-27T11:29:34.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.002+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/2329703323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 msgr2=0x55baf9717d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:34.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.002+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/2329703323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf9717d90 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f4e480017a0 tx=0x7f4e4802d000 comp rx=0 tx=0).stop 2023-08-27T11:29:34.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.002+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/2329703323 shutdown_connections 2023-08-27T11:29:34.021 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.002+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/2329703323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf97651d0 0x55baf97182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.002+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/2329703323 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf971c3d0 0x55baf9719150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.002+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/2329703323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf9717d90 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.002+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/2329703323 >> 172.21.15.112:0/2329703323 conn(0x55baf967cc30 msgr2=0x55baf975d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.002+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/2329703323 shutdown_connections 2023-08-27T11:29:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.002+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/2329703323 wait complete. 2023-08-27T11:29:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.003+0000 7f4e6c17cb80 1 -- start start 2023-08-27T11:29:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.003+0000 7f4e6c17cb80 1 Processor -- start 2023-08-27T11:29:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e6c17cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf971c3d0 0x55baf9717ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e6c17cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf974a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e6c17cb80 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf97651d0 0x55baf974ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e6c17cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf97493b0 con 0x55baf971e1b0 2023-08-27T11:29:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e6c17cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf97621e0 con 0x55baf971c3d0 2023-08-27T11:29:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e6c17cb80 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x55baf97484e0 con 0x55baf97651d0 2023-08-27T11:29:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e6c17cb80 10 client.0 _ll_register_callbacks cb 0x55baf960c630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:29:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e5d555700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf974a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e5d555700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf974a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38306/0 (socket says 172.21.15.112:38306) 2023-08-27T11:29:34.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e5d555700 1 -- 172.21.15.112:0/1417714440 learned_addr learned my addr 172.21.15.112:0/1417714440 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:34.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/1417714440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf971c3d0 0x55baf9717ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e5e557700 1 --2- 172.21.15.112:0/1417714440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf97651d0 0x55baf974ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.025 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e5d555700 1 -- 172.21.15.112:0/1417714440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf97651d0 msgr2=0x55baf974ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:34.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e5d555700 1 --2- 172.21.15.112:0/1417714440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf97651d0 0x55baf974ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e5d555700 1 -- 172.21.15.112:0/1417714440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf971c3d0 msgr2=0x55baf9717ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:34.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e5d555700 1 --2- 172.21.15.112:0/1417714440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf971c3d0 0x55baf9717ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e5d555700 1 -- 172.21.15.112:0/1417714440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55baf9688750 con 0x55baf971e1b0 2023-08-27T11:29:34.026 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e5d555700 1 --2- 172.21.15.112:0/1417714440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf974a860 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f4e48024410 tx=0x7f4e4800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524cb8dc0cc75e9c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:34.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-08-27T11:29:34.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e38ff9700 1 -- 172.21.15.112:0/1417714440 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48015750 con 0x55baf971e1b0 2023-08-27T11:29:34.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e38ff9700 1 -- 172.21.15.112:0/1417714440 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e4801d070 con 0x55baf971e1b0 2023-08-27T11:29:34.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.004+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/1417714440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e48015c70 con 0x55baf971e1b0 2023-08-27T11:29:34.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e38ff9700 1 -- 172.21.15.112:0/1417714440 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e48049650 con 0x55baf971e1b0 2023-08-27T11:29:34.027 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e6c17cb80 10 client.9576 fetch_fsmap learned FSMap version 113 2023-08-27T11:29:34.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/1417714440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x55baf9485c10 con 0x55baf971e1b0 2023-08-27T11:29:34.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e38ff9700 1 -- 172.21.15.112:0/1417714440 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e48015c70 con 0x55baf971e1b0 2023-08-27T11:29:34.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e38ff9700 1 -- 172.21.15.112:0/1417714440 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e48046050 con 0x55baf971e1b0 2023-08-27T11:29:34.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e6c17cb80 10 client.9576 fetch_fsmap finished waiting for FSMap version 113 2023-08-27T11:29:34.028 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e6c17cb80 10 client.9576 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:29:34.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e6c17cb80 20 client.9576 populate_metadata read hostname 'smithi112' 2023-08-27T11:29:34.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/1417714440 >> [v2:172.21.15.112:6832/4086629952,v1:172.21.15.112:6833/4086629952] conn(0x55baf9742280 0x55baf9742660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e6c17cb80 4 client.9576 mds_command: new command op to 9393 tid=0[{"prefix": "status"}] 2023-08-27T11:29:34.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/1417714440 --> [v2:172.21.15.112:6832/4086629952,v1:172.21.15.112:6833/4086629952] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55baf962a000 con 0x55baf9742280 2023-08-27T11:29:34.029 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/1417714440 >> [v2:172.21.15.112:6832/4086629952,v1:172.21.15.112:6833/4086629952] conn(0x55baf9742280 0x55baf9742660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e38ff9700 10 client.9576 ms_handle_connect on v2:172.21.15.112:6832/4086629952 2023-08-27T11:29:34.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.005+0000 7f4e5dd56700 1 --2- 172.21.15.112:0/1417714440 >> [v2:172.21.15.112:6832/4086629952,v1:172.21.15.112:6833/4086629952] conn(0x55baf9742280 0x55baf9742660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64639badfd76b984 server_cookie=db5a153180e5e64c in_seq=0 out_seq=0 2023-08-27T11:29:34.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e38ff9700 1 -- 172.21.15.112:0/1417714440 <== mds.0 v2:172.21.15.112:6832/4086629952 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x55baf962a000 con 0x55baf9742280 2023-08-27T11:29:34.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e38ff9700 10 client.9576 handle_command_reply: tid=0 2023-08-27T11:29:34.030 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e6c17cb80 1 client.9576 shutdown 2023-08-27T11:29:34.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/1417714440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 msgr2=0x55baf974a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:34.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/1417714440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf974a860 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f4e48024410 tx=0x7f4e4800c000 comp rx=0 tx=0).stop 2023-08-27T11:29:34.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/1417714440 shutdown_connections 2023-08-27T11:29:34.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/1417714440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55baf97651d0 0x55baf974ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.031 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/1417714440 >> [v2:172.21.15.112:6832/4086629952,v1:172.21.15.112:6833/4086629952] conn(0x55baf9742280 0x55baf9742660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/1417714440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baf971c3d0 0x55baf9717ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e38ff9700 0 client.9576 ms_handle_reset on v2:172.21.15.112:6832/4086629952 2023-08-27T11:29:34.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e6c17cb80 1 --2- 172.21.15.112:0/1417714440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55baf971e1b0 0x55baf974a860 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/1417714440 >> 172.21.15.112:0/1417714440 conn(0x55baf967cc30 msgr2=0x55baf9743540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:34.032 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/1417714440 shutdown_connections 2023-08-27T11:29:34.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e6c17cb80 1 -- 172.21.15.112:0/1417714440 wait complete. 2023-08-27T11:29:34.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e6c17cb80 20 client.9576 trim_cache size 0 max 16384 2023-08-27T11:29:34.033 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:29:34.033 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e099a24f-c1cd-4f11-ac4f-ecacc10e5e98", 2023-08-27T11:29:34.034 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2023-08-27T11:29:34.034 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 9393, 2023-08-27T11:29:34.034 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2023-08-27T11:29:34.034 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2023-08-27T11:29:34.034 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2023-08-27T11:29:34.035 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 10.461649374, 2023-08-27T11:29:34.035 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 113, 2023-08-27T11:29:34.035 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 167, 2023-08-27T11:29:34.035 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 167, 2023-08-27T11:29:34.035 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 17.40173476 2023-08-27T11:29:34.036 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:29:34.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e5ffba700 1 -- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e3c06a4d0 msgr2=0x7f4e3c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:34.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e5ffba700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e3c06a4d0 0x7f4e3c06c990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4e48001c90 tx=0x7f4e48007040 comp rx=0 tx=0).stop 2023-08-27T11:29:34.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e5ffba700 1 -- 172.21.15.112:0/3869509604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e44003a30 msgr2=0x7f4e581407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:34.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e5ffba700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e44003a30 0x7f4e581407c0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400bb10 tx=0x7f4e540047c0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e5ffba700 1 -- 172.21.15.112:0/3869509604 shutdown_connections 2023-08-27T11:29:34.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e5ffba700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e3c06a4d0 0x7f4e3c06c990 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e5ffba700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e58142710 0x7f4e581f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e5ffba700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e44003a30 0x7f4e581407c0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e5ffba700 1 --2- 172.21.15.112:0/3869509604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e5813cbb0 0x7f4e581421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e5ffba700 1 -- 172.21.15.112:0/3869509604 >> 172.21.15.112:0/3869509604 conn(0x7f4e5806f970 msgr2=0x7f4e58143210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:34.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e5ffba700 1 -- 172.21.15.112:0/3869509604 shutdown_connections 2023-08-27T11:29:34.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.006+0000 7f4e5ffba700 1 -- 172.21.15.112:0/3869509604 wait complete. 2023-08-27T11:29:34.040 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:29:34.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.159+0000 7fdf83804700 1 Processor -- start 2023-08-27T11:29:34.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.159+0000 7fdf83804700 1 -- start start 2023-08-27T11:29:34.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.160+0000 7fdf83804700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdf7c1578e0 0x7fdf7c0b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.160+0000 7fdf83804700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7c0cfb40 con 0x7fdf7c0cf170 2023-08-27T11:29:34.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.160+0000 7fdf83804700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7c0cfde0 con 0x7fdf7c160f90 2023-08-27T11:29:34.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.160+0000 7fdf83804700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7c0d00c0 con 0x7fdf7c1578e0 2023-08-27T11:29:34.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.160+0000 7fdf815a0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdf7c1578e0 0x7fdf7c0b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.160+0000 7fdf80d9f700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fdf7c0cf170 0x7fdf7c0d08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60926/0 (socket says 172.21.15.112:60926) 2023-08-27T11:29:34.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.160+0000 7fdf80d9f700 1 -- 172.21.15.112:0/110380221 learned_addr learned my addr 172.21.15.112:0/110380221 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:34.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.160+0000 7fdf815a0700 1 -- 172.21.15.112:0/110380221 >> v1:172.21.15.138:6789/0 conn(0x7fdf7c160f90 legacy=0x7fdf7c0708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:34.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.160+0000 7fdf815a0700 1 -- 172.21.15.112:0/110380221 >> v1:172.21.15.112:6789/0 conn(0x7fdf7c0cf170 legacy=0x7fdf7c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:34.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.160+0000 7fdf815a0700 1 -- 172.21.15.112:0/110380221 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c0d03a0 con 0x7fdf7c1578e0 2023-08-27T11:29:34.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.161+0000 7fdf815a0700 1 --2- 172.21.15.112:0/110380221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf7c1578e0 0x7fdf7c0b0710 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009840 tx=0x7fdf6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884b5526f25be99f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:34.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.161+0000 7fdf73fff700 1 -- 172.21.15.112:0/110380221 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c011830 con 0x7fdf7c1578e0 2023-08-27T11:29:34.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.161+0000 7fdf73fff700 1 -- 172.21.15.112:0/110380221 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c011990 con 0x7fdf7c1578e0 2023-08-27T11:29:34.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.161+0000 7fdf73fff700 1 -- 172.21.15.112:0/110380221 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c011d20 con 0x7fdf7c1578e0 2023-08-27T11:29:34.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.161+0000 7fdf83804700 1 -- 172.21.15.112:0/110380221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf7c1578e0 msgr2=0x7fdf7c0b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:34.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.161+0000 7fdf83804700 1 --2- 172.21.15.112:0/110380221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf7c1578e0 0x7fdf7c0b0710 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009840 tx=0x7fdf6c009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:34.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.161+0000 7fdf83804700 1 -- 172.21.15.112:0/110380221 shutdown_connections 2023-08-27T11:29:34.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.161+0000 7fdf83804700 1 --2- 172.21.15.112:0/110380221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf7c1578e0 0x7fdf7c0b0710 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.161+0000 7fdf83804700 1 -- 172.21.15.112:0/110380221 >> 172.21.15.112:0/110380221 conn(0x7fdf7c06f970 msgr2=0x7fdf7c06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:34.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.161+0000 7fdf83804700 1 -- 172.21.15.112:0/110380221 shutdown_connections 2023-08-27T11:29:34.166 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.161+0000 7fdf83804700 1 -- 172.21.15.112:0/110380221 wait complete. 2023-08-27T11:29:34.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.162+0000 7fdf83804700 1 Processor -- start 2023-08-27T11:29:34.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.163+0000 7fdf83804700 1 -- start start 2023-08-27T11:29:34.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.163+0000 7fdf83804700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0cf170 0x7fdf7c146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.163+0000 7fdf83804700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf7c1578e0 0x7fdf7c065fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.163+0000 7fdf83804700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf7c160f90 0x7fdf7c066520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.163+0000 7fdf83804700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c158aa0 con 0x7fdf7c1578e0 2023-08-27T11:29:34.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.163+0000 7fdf83804700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c15c590 con 0x7fdf7c0cf170 2023-08-27T11:29:34.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.163+0000 7fdf83804700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c158100 con 0x7fdf7c160f90 2023-08-27T11:29:34.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.163+0000 7fdf80d9f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf7c1578e0 0x7fdf7c065fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.163+0000 7fdf80d9f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf7c1578e0 0x7fdf7c065fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38320/0 (socket says 172.21.15.112:38320) 2023-08-27T11:29:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.163+0000 7fdf80d9f700 1 -- 172.21.15.112:0/1173975833 learned_addr learned my addr 172.21.15.112:0/1173975833 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.163+0000 7fdf815a0700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0cf170 0x7fdf7c146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.164+0000 7fdf81da1700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf7c160f90 0x7fdf7c066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.164+0000 7fdf80d9f700 1 -- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf7c160f90 msgr2=0x7fdf7c066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.164+0000 7fdf80d9f700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf7c160f90 0x7fdf7c066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.164+0000 7fdf80d9f700 1 -- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0cf170 msgr2=0x7fdf7c146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:34.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.164+0000 7fdf80d9f700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0cf170 0x7fdf7c146a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.164+0000 7fdf80d9f700 1 -- 172.21.15.112:0/1173975833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c0094a0 con 0x7fdf7c1578e0 2023-08-27T11:29:34.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.164+0000 7fdf815a0700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0cf170 0x7fdf7c146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:34.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.164+0000 7fdf80d9f700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf7c1578e0 0x7fdf7c065fe0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c520 tx=0x7fdf7800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.164+0000 7fdf71ffb700 1 -- 172.21.15.112:0/1173975833 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78005d70 con 0x7fdf7c1578e0 2023-08-27T11:29:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.164+0000 7fdf71ffb700 1 -- 172.21.15.112:0/1173975833 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf780070d0 con 0x7fdf7c1578e0 2023-08-27T11:29:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.164+0000 7fdf83804700 1 -- 172.21.15.112:0/1173975833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7c157e40 con 0x7fdf7c1578e0 2023-08-27T11:29:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.164+0000 7fdf71ffb700 1 -- 172.21.15.112:0/1173975833 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78007440 con 0x7fdf7c1578e0 2023-08-27T11:29:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.164+0000 7fdf83804700 1 -- 172.21.15.112:0/1173975833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7c1586c0 con 0x7fdf7c1578e0 2023-08-27T11:29:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.165+0000 7fdf83804700 1 -- 172.21.15.112:0/1173975833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7c158be0 con 0x7fdf7c1578e0 2023-08-27T11:29:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.165+0000 7fdf71ffb700 1 -- 172.21.15.112:0/1173975833 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fdf780075a0 con 0x7fdf7c1578e0 2023-08-27T11:29:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.166+0000 7fdf71ffb700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdf6006a4d0 0x7fdf6006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.166+0000 7fdf71ffb700 1 -- 172.21.15.112:0/1173975833 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdf78087fd0 con 0x7fdf7c1578e0 2023-08-27T11:29:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.166+0000 7fdf815a0700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdf6006a4d0 0x7fdf6006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:34.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.167+0000 7fdf815a0700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdf6006a4d0 0x7fdf6006c990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0090b0 tx=0x7fdf6c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:34.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.170+0000 7fdf71ffb700 1 -- 172.21.15.112:0/1173975833 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdf7c158be0 con 0x7fdf7c1578e0 2023-08-27T11:29:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.401+0000 7fdf83804700 1 -- 172.21.15.112:0/1173975833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdf7c158be0 con 0x7fdf7c1578e0 2023-08-27T11:29:34.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.402+0000 7fdf71ffb700 1 -- 172.21.15.112:0/1173975833 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v113) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdf78017060 con 0x7fdf7c1578e0 2023-08-27T11:29:34.403 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:29:34.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.404+0000 7fdf83804700 1 -- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdf6006a4d0 msgr2=0x7fdf6006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:34.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.404+0000 7fdf83804700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdf6006a4d0 0x7fdf6006c990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0090b0 tx=0x7fdf6c008030 comp rx=0 tx=0).stop 2023-08-27T11:29:34.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.404+0000 7fdf83804700 1 -- 172.21.15.112:0/1173975833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf7c1578e0 msgr2=0x7fdf7c065fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.404+0000 7fdf83804700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf7c1578e0 0x7fdf7c065fe0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c520 tx=0x7fdf7800c8e0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.404+0000 7fdf83804700 1 -- 172.21.15.112:0/1173975833 shutdown_connections 2023-08-27T11:29:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.404+0000 7fdf83804700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdf6006a4d0 0x7fdf6006c990 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.404+0000 7fdf83804700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf7c160f90 0x7fdf7c066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.404+0000 7fdf83804700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0cf170 0x7fdf7c146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.404+0000 7fdf83804700 1 --2- 172.21.15.112:0/1173975833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf7c1578e0 0x7fdf7c065fe0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.404+0000 7fdf83804700 1 -- 172.21.15.112:0/1173975833 >> 172.21.15.112:0/1173975833 conn(0x7fdf7c06f970 msgr2=0x7fdf7c071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.404+0000 7fdf83804700 1 -- 172.21.15.112:0/1173975833 shutdown_connections 2023-08-27T11:29:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:34.404+0000 7fdf83804700 1 -- 172.21.15.112:0/1173975833 wait complete. 2023-08-27T11:29:34.417 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:29:34.418 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:29:34.418 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-08-27T11:29:34.418 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:29:34.419 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:29:34.419 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:29:34.419 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:29:34.420 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:29:34.420 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:29:34.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:34.421 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-08-27T11:29:34.438 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:29:34.438 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:29:34.439 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:29:34.439 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:34.439 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-08-27T11:29:34.439 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:34.439 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:29:34.440 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:34.440 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:29:34.440 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:34.440 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:29:34.440 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:34.441 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:29:34.441 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:34.441 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:29:34.441 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 41668sec preferred_lft 41668sec 2023-08-27T11:29:34.441 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8d6c:4fe6:6a84:8222/64 scope link noprefixroute 2023-08-27T11:29:34.442 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:34.442 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:29:34.442 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:34.442 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:29:34.442 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:34.443 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::d87c:a1ff:fe1d:b635/64 scope link 2023-08-27T11:29:34.443 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:34.443 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:29:34.443 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ce:14:79:b3:45:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:29:34.443 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::cc14:79ff:feb3:452f/64 scope link 2023-08-27T11:29:34.445 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:34.445 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:29:34.445 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether e6:51:f0:78:04:02 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:29:34.445 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::e451:f0ff:fe78:402/64 scope link 2023-08-27T11:29:34.446 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:34.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:34.446 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:29:34.503 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-08-27T11:29:34.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:34.504 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-08-27T11:29:34.563 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-08-27T11:29:34.563 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:29:34.564 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:29:34.564 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:29:34.564 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:29:34.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:34.565 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-08-27T11:29:34.565 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-08-27T11:29:34.565 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:29:34.566 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:29:34.566 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:29:34.566 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-08-27T11:29:34.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:34.667 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-08-27T11:29:34.684 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:29:34.684 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:29:34.684 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:29:34.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:34.685 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-08-27T11:29:34.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:29:34.751 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:29:34.807 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:29:34.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:29:34.808 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:29:34.864 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:29:34.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:34.865 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:29:34.949 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:29:34.949 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:29:34.949 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:29:34.957 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:29:34.958 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:29:34.958 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:29:34.958 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:29:34.958 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:29:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:29:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:29:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2023-08-27T11:29:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:29:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:29:35.000 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:29:35.000 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:29:35.000 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:29:35.012 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:29:35.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:35.012 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:29:35.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:35.039 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:29:35.122 INFO:teuthology.orchestra.run.smithi112.stdout:client9588 2023-08-27T11:29:35.122 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:29:35.123 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:35.179 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:29:35.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.338+0000 7fce5a4ee700 1 Processor -- start 2023-08-27T11:29:35.339 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.338+0000 7fce5a4ee700 1 -- start start 2023-08-27T11:29:35.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce5a4ee700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fce540b1400 0x7fce540cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:35.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce5a4ee700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540b17e0 0x7fce5406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:35.343 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce5a4ee700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce54161850 con 0x7fce54161ce0 2023-08-27T11:29:35.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce5a4ee700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5406e7d0 con 0x7fce540b1400 2023-08-27T11:29:35.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce5a4ee700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce540cfc10 con 0x7fce540b17e0 2023-08-27T11:29:35.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce58a8b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fce54161ce0 0x7fce5406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60948/0 (socket says 172.21.15.112:60948) 2023-08-27T11:29:35.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce58a8b700 1 -- 172.21.15.112:0/3983497565 learned_addr learned my addr 172.21.15.112:0/3983497565 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:35.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce53fff700 1 --2- 172.21.15.112:0/3983497565 >> v2:172.21.15.138:3300/0 conn(0x7fce540b1400 0x7fce540cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:35.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787915747 0 0) 0x7fce54161850 con 0x7fce54161ce0 2023-08-27T11:29:35.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce537fe700 1 --2- 172.21.15.112:0/3983497565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540b17e0 0x7fce5406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:35.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce3c0035b0 con 0x7fce54161ce0 2023-08-27T11:29:35.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3685993911 0 0) 0x7fce3c0035b0 con 0x7fce54161ce0 2023-08-27T11:29:35.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540b17e0 msgr2=0x7fce5406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:35.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 --2- 172.21.15.112:0/3983497565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540b17e0 0x7fce5406e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:35.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 >> v2:172.21.15.138:3300/0 conn(0x7fce540b1400 msgr2=0x7fce540cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:35.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 --2- 172.21.15.112:0/3983497565 >> v2:172.21.15.138:3300/0 conn(0x7fce540b1400 0x7fce540cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:35.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce54161200 con 0x7fce54161ce0 2023-08-27T11:29:35.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fce4c0019f0 con 0x7fce54161ce0 2023-08-27T11:29:35.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 >> v1:172.21.15.112:6789/0 conn(0x7fce54161ce0 legacy=0x7fce5406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:35.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 --2- 172.21.15.112:0/3983497565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce3c003a00 0x7fce3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:35.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 --2- 172.21.15.112:0/3983497565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540b17e0 0x7fce3c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:35.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 --2- 172.21.15.112:0/3983497565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce3c004d90 0x7fce3c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:35.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5406e7d0 con 0x7fce3c004d90 2023-08-27T11:29:35.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540cfc10 con 0x7fce3c003a00 2023-08-27T11:29:35.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c00bc20 con 0x7fce540b17e0 2023-08-27T11:29:35.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.339+0000 7fce58a8b700 1 --2- 172.21.15.112:0/3983497565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce3c004d90 0x7fce3c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:35.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.340+0000 7fce53fff700 1 --2- 172.21.15.112:0/3983497565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce3c003a00 0x7fce3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:35.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.340+0000 7fce537fe700 1 --2- 172.21.15.112:0/3983497565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540b17e0 0x7fce3c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:35.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.340+0000 7fce58a8b700 1 -- 172.21.15.112:0/3983497565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540b17e0 msgr2=0x7fce3c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:35.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.340+0000 7fce58a8b700 1 --2- 172.21.15.112:0/3983497565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540b17e0 0x7fce3c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:35.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.340+0000 7fce58a8b700 1 -- 172.21.15.112:0/3983497565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce3c003a00 msgr2=0x7fce3c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:35.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.340+0000 7fce58a8b700 1 --2- 172.21.15.112:0/3983497565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce3c003a00 0x7fce3c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:35.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.340+0000 7fce58a8b700 1 -- 172.21.15.112:0/3983497565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c0035b0 con 0x7fce3c004d90 2023-08-27T11:29:35.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.340+0000 7fce58a8b700 1 --2- 172.21.15.112:0/3983497565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce3c004d90 0x7fce3c005210 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00a860 tx=0x7fce3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375b28e4a647bd47 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:35.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.340+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c012600 con 0x7fce3c004d90 2023-08-27T11:29:35.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.340+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c012780 con 0x7fce3c004d90 2023-08-27T11:29:35.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.340+0000 7fce52ffd700 1 -- 172.21.15.112:0/3983497565 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c012b30 con 0x7fce3c004d90 2023-08-27T11:29:35.350 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.341+0000 7fce5a4ee700 1 -- 172.21.15.112:0/3983497565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce3c004d90 msgr2=0x7fce3c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:35.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.341+0000 7fce5a4ee700 1 --2- 172.21.15.112:0/3983497565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce3c004d90 0x7fce3c005210 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00a860 tx=0x7fce3c005880 comp rx=0 tx=0).stop 2023-08-27T11:29:35.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.341+0000 7fce58a8b700 1 -- 172.21.15.112:0/3983497565 reap_dead start 2023-08-27T11:29:35.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.341+0000 7fce5a4ee700 1 -- 172.21.15.112:0/3983497565 shutdown_connections 2023-08-27T11:29:35.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.341+0000 7fce5a4ee700 1 -- 172.21.15.112:0/3983497565 >> 172.21.15.112:0/3983497565 conn(0x7fce540d0000 msgr2=0x7fce540d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:35.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.341+0000 7fce5a4ee700 1 -- 172.21.15.112:0/3983497565 shutdown_connections 2023-08-27T11:29:35.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.341+0000 7fce5a4ee700 1 -- 172.21.15.112:0/3983497565 wait complete. 2023-08-27T11:29:35.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.342+0000 7fce5a4ee700 1 Processor -- start 2023-08-27T11:29:35.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.342+0000 7fce5a4ee700 1 -- start start 2023-08-27T11:29:35.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce5a4ee700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce540c9940 0x7fce540c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:35.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce5a4ee700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce540c70d0 0x7fce541f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:35.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce5a4ee700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540c9d20 0x7fce541f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:35.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce5a4ee700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce541f2cb0 con 0x7fce540c9940 2023-08-27T11:29:35.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce5a4ee700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce541516f0 con 0x7fce540c70d0 2023-08-27T11:29:35.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce5a4ee700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce540c7f80 con 0x7fce540c9d20 2023-08-27T11:29:35.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce53fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce540c9940 0x7fce540c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:35.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce53fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce540c9940 0x7fce540c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38342/0 (socket says 172.21.15.112:38342) 2023-08-27T11:29:35.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce53fff700 1 -- 172.21.15.112:0/4290009727 learned_addr learned my addr 172.21.15.112:0/4290009727 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:35.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce537fe700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce540c70d0 0x7fce541f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:35.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce58a8b700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540c9d20 0x7fce541f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:35.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce53fff700 1 -- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540c9d20 msgr2=0x7fce541f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:35.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce53fff700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540c9d20 0x7fce541f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:35.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce53fff700 1 -- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce540c70d0 msgr2=0x7fce541f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:35.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce53fff700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce540c70d0 0x7fce541f2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:35.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce53fff700 1 -- 172.21.15.112:0/4290009727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce54151940 con 0x7fce540c9940 2023-08-27T11:29:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce58a8b700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540c9d20 0x7fce541f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce53fff700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce540c9940 0x7fce540c7a40 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fce440116d0 tx=0x7fce44011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce50ff9700 1 -- 172.21.15.112:0/4290009727 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4400c070 con 0x7fce540c9940 2023-08-27T11:29:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce50ff9700 1 -- 172.21.15.112:0/4290009727 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce44015870 con 0x7fce540c9940 2023-08-27T11:29:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce50ff9700 1 -- 172.21.15.112:0/4290009727 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44015be0 con 0x7fce540c9940 2023-08-27T11:29:35.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce5a4ee700 1 -- 172.21.15.112:0/4290009727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce54151c20 con 0x7fce540c9940 2023-08-27T11:29:35.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.343+0000 7fce5a4ee700 1 -- 172.21.15.112:0/4290009727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce541521e0 con 0x7fce540c9940 2023-08-27T11:29:35.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.344+0000 7fce5a4ee700 1 -- 172.21.15.112:0/4290009727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce54067a20 con 0x7fce540c9940 2023-08-27T11:29:35.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.345+0000 7fce50ff9700 1 -- 172.21.15.112:0/4290009727 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fce44015d40 con 0x7fce540c9940 2023-08-27T11:29:35.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.345+0000 7fce50ff9700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fce4006a4d0 0x7fce4006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:35.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.346+0000 7fce50ff9700 1 -- 172.21.15.112:0/4290009727 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fce440890c0 con 0x7fce540c9940 2023-08-27T11:29:35.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.346+0000 7fce537fe700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fce4006a4d0 0x7fce4006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:35.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.346+0000 7fce537fe700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fce4006a4d0 0x7fce4006c990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fce48009950 tx=0x7fce48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:35.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.350+0000 7fce50ff9700 1 -- 172.21.15.112:0/4290009727 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fce44054040 con 0x7fce540c9940 2023-08-27T11:29:35.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.579+0000 7fce5a4ee700 1 -- 172.21.15.112:0/4290009727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fce5406db60 con 0x7fce540c9940 2023-08-27T11:29:35.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.579+0000 7fce50ff9700 1 -- 172.21.15.112:0/4290009727 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v113) v1 ==== 53+0+73 (secure 0 0 0) 0x7fce44057660 con 0x7fce540c9940 2023-08-27T11:29:35.581 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:29:35.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.581+0000 7fce5a4ee700 1 -- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fce4006a4d0 msgr2=0x7fce4006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:35.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.581+0000 7fce5a4ee700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fce4006a4d0 0x7fce4006c990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fce48009950 tx=0x7fce48008040 comp rx=0 tx=0).stop 2023-08-27T11:29:35.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.581+0000 7fce5a4ee700 1 -- 172.21.15.112:0/4290009727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce540c9940 msgr2=0x7fce540c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:35.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.581+0000 7fce5a4ee700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce540c9940 0x7fce540c7a40 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fce440116d0 tx=0x7fce44011a90 comp rx=0 tx=0).stop 2023-08-27T11:29:35.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.581+0000 7fce5a4ee700 1 -- 172.21.15.112:0/4290009727 shutdown_connections 2023-08-27T11:29:35.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.581+0000 7fce5a4ee700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fce4006a4d0 0x7fce4006c990 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:35.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.581+0000 7fce5a4ee700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce540c9d20 0x7fce541f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:35.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.581+0000 7fce5a4ee700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce540c9940 0x7fce540c7a40 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:35.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.581+0000 7fce5a4ee700 1 --2- 172.21.15.112:0/4290009727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce540c70d0 0x7fce541f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:35.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.581+0000 7fce5a4ee700 1 -- 172.21.15.112:0/4290009727 >> 172.21.15.112:0/4290009727 conn(0x7fce540d0000 msgr2=0x7fce5415ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:35.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.581+0000 7fce5a4ee700 1 -- 172.21.15.112:0/4290009727 shutdown_connections 2023-08-27T11:29:35.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:35.581+0000 7fce5a4ee700 1 -- 172.21.15.112:0/4290009727 wait complete. 2023-08-27T11:29:35.594 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:29:35.595 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:29:35.595 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2023-08-27T11:29:35.595 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-27T11:29:35.595 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:29:35.596 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:29:35.596 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:29:35.596 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:29:35.597 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:29:35.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:35.598 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2023-08-27T11:29:35.618 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:29:35.618 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:29:35.618 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:29:35.618 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:35.618 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2023-08-27T11:29:35.618 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:35.619 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:29:35.619 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:35.619 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:29:35.619 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:35.619 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:29:35.620 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:35.620 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:29:35.620 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:35.620 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:29:35.620 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 41684sec preferred_lft 41684sec 2023-08-27T11:29:35.620 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ac17:49c5:cb39:801a/64 scope link noprefixroute 2023-08-27T11:29:35.621 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:35.621 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:29:35.621 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:35.621 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:29:35.621 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:35.622 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c4b4:94ff:fe58:4e35/64 scope link 2023-08-27T11:29:35.622 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:35.622 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:29:35.622 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5a:06:6a:04:ef:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:29:35.622 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::5806:6aff:fe04:ef32/64 scope link 2023-08-27T11:29:35.623 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:35.623 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:29:35.623 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 8a:21:ba:15:69:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-27T11:29:35.623 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::8821:baff:fe15:6919/64 scope link 2023-08-27T11:29:35.623 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:35.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:35.624 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:29:35.686 INFO:teuthology.orchestra.run.smithi138.stdout:1 2023-08-27T11:29:35.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:35.688 DEBUG:teuthology.orchestra.run.smithi138:> route 2023-08-27T11:29:35.748 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2023-08-27T11:29:35.749 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:29:35.749 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:29:35.749 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:29:35.749 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:29:35.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:35.750 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2023-08-27T11:29:35.750 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2023-08-27T11:29:35.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:29:35.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:29:35.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:29:35.752 DEBUG:teuthology.orchestra.run.smithi138:> ') 2023-08-27T11:29:35.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:35.858 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2023-08-27T11:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-27T11:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-27T11:29:35.875 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:29:35.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:35.876 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2023-08-27T11:29:35.942 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:29:35.942 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-27T11:29:35.998 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:29:35.998 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:29:35.999 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:29:36.054 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:29:36.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:36.055 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:29:36.141 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:29:36.141 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:29:36.141 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:29:36.149 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:29:36.150 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:29:36.150 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:29:36.150 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:29:36.150 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:29:36.176 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:29:36.177 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:29:36.177 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2023-08-27T11:29:36.177 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.138:3300,172.21.15.138:3301" 2023-08-27T11:29:36.177 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:29:36.178 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790" 2023-08-27T11:29:36.178 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2023-08-27T11:29:36.178 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e099a24f-c1cd-4f11-ac4f-ecacc10e5e98" will pass to kernel 2023-08-27T11:29:36.179 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:29:36.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:36.180 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:29:36.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:36.215 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:29:36.287 INFO:teuthology.orchestra.run.smithi138.stdout:client8341 2023-08-27T11:29:36.288 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:29:36.288 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:36.346 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-08-27T11:29:36.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.475+0000 7f619c213700 1 Processor -- start 2023-08-27T11:29:36.476 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.475+0000 7f619c213700 1 -- start start 2023-08-27T11:29:36.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f619c213700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6194144bd0 0x7f619415b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:36.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f619c213700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f619406bfd0 0x7f619406c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:36.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f619c213700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61940b6560 con 0x7f61940b57a0 2023-08-27T11:29:36.479 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f619c213700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619406c990 con 0x7f619406bfd0 2023-08-27T11:29:36.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f619c213700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61940b6340 con 0x7f6194144bd0 2023-08-27T11:29:36.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f61997ae700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f619406bfd0 0x7f619406c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:36.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f6199faf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6194144bd0 0x7f619415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:36.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f61997ae700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f619406bfd0 0x7f619406c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38354/0 (socket says 172.21.15.112:38354) 2023-08-27T11:29:36.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f619a7b0700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f61940b57a0 0x7f61940b5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60966/0 (socket says 172.21.15.112:60966) 2023-08-27T11:29:36.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f61997ae700 1 -- 172.21.15.112:0/3088258707 learned_addr learned my addr 172.21.15.112:0/3088258707 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:36.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f61997ae700 1 -- 172.21.15.112:0/3088258707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6194144bd0 msgr2=0x7f619415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:36.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f61997ae700 1 --2- 172.21.15.112:0/3088258707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6194144bd0 0x7f619415b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:36.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f61997ae700 1 -- 172.21.15.112:0/3088258707 >> v1:172.21.15.112:6789/0 conn(0x7f61940b57a0 legacy=0x7f61940b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:36.481 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f61997ae700 1 -- 172.21.15.112:0/3088258707 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61940b6840 con 0x7f619406bfd0 2023-08-27T11:29:36.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.476+0000 7f6198fad700 1 -- 172.21.15.112:0/3088258707 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041382220 0 0) 0x7f61940b6560 con 0x7f61940b57a0 2023-08-27T11:29:36.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.477+0000 7f61997ae700 1 --2- 172.21.15.112:0/3088258707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f619406bfd0 0x7f619406c450 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f61900098f0 tx=0x7f6190009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac481712b0a7e0c1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:36.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.477+0000 7f6198fad700 1 -- 172.21.15.112:0/3088258707 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190011830 con 0x7f619406bfd0 2023-08-27T11:29:36.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.477+0000 7f6198fad700 1 -- 172.21.15.112:0/3088258707 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6190011990 con 0x7f619406bfd0 2023-08-27T11:29:36.482 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.477+0000 7f6198fad700 1 -- 172.21.15.112:0/3088258707 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190011d20 con 0x7f619406bfd0 2023-08-27T11:29:36.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.477+0000 7f619c213700 1 -- 172.21.15.112:0/3088258707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f619406bfd0 msgr2=0x7f619406c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:36.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.477+0000 7f619c213700 1 --2- 172.21.15.112:0/3088258707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f619406bfd0 0x7f619406c450 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f61900098f0 tx=0x7f6190009c00 comp rx=0 tx=0).stop 2023-08-27T11:29:36.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.477+0000 7f619c213700 1 -- 172.21.15.112:0/3088258707 shutdown_connections 2023-08-27T11:29:36.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.477+0000 7f619c213700 1 --2- 172.21.15.112:0/3088258707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6194144bd0 0x7f619415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:36.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.477+0000 7f619c213700 1 --2- 172.21.15.112:0/3088258707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f619406bfd0 0x7f619406c450 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:36.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.477+0000 7f619c213700 1 -- 172.21.15.112:0/3088258707 >> 172.21.15.112:0/3088258707 conn(0x7f6194066010 msgr2=0x7f6194066420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:36.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.477+0000 7f619c213700 1 -- 172.21.15.112:0/3088258707 shutdown_connections 2023-08-27T11:29:36.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.477+0000 7f619c213700 1 -- 172.21.15.112:0/3088258707 wait complete. 2023-08-27T11:29:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.479+0000 7f619c213700 1 Processor -- start 2023-08-27T11:29:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.479+0000 7f619c213700 1 -- start start 2023-08-27T11:29:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.479+0000 7f619c213700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f619406bfd0 0x7f619413faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.479+0000 7f619c213700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61940b57a0 0x7f61940a89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:36.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.479+0000 7f619c213700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6194144bd0 0x7f61940ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:36.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.479+0000 7f619c213700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61940a9f40 con 0x7f61940b57a0 2023-08-27T11:29:36.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.479+0000 7f619c213700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6194140030 con 0x7f6194144bd0 2023-08-27T11:29:36.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.479+0000 7f619c213700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61940a9330 con 0x7f619406bfd0 2023-08-27T11:29:36.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.479+0000 7f61997ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61940b57a0 0x7f61940a89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:36.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.479+0000 7f61997ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61940b57a0 0x7f61940a89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38358/0 (socket says 172.21.15.112:38358) 2023-08-27T11:29:36.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.479+0000 7f61997ae700 1 -- 172.21.15.112:0/1172234709 learned_addr learned my addr 172.21.15.112:0/1172234709 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:36.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.479+0000 7f6199faf700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f619406bfd0 0x7f619413faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:36.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.480+0000 7f619a7b0700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6194144bd0 0x7f61940ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:36.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.480+0000 7f61997ae700 1 -- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f619406bfd0 msgr2=0x7f619413faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:36.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.480+0000 7f61997ae700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f619406bfd0 0x7f619413faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:36.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.480+0000 7f61997ae700 1 -- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6194144bd0 msgr2=0x7f61940ab2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:36.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.480+0000 7f61997ae700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6194144bd0 0x7f61940ab2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:36.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.480+0000 7f61997ae700 1 -- 172.21.15.112:0/1172234709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6190009550 con 0x7f61940b57a0 2023-08-27T11:29:36.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.480+0000 7f61997ae700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61940b57a0 0x7f61940a89c0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f6190009c40 tx=0x7f6190004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.480+0000 7f618a7fc700 1 -- 172.21.15.112:0/1172234709 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61900043c0 con 0x7f61940b57a0 2023-08-27T11:29:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.480+0000 7f618a7fc700 1 -- 172.21.15.112:0/1172234709 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619001f070 con 0x7f61940b57a0 2023-08-27T11:29:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.480+0000 7f619c213700 1 -- 172.21.15.112:0/1172234709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61940a9610 con 0x7f61940b57a0 2023-08-27T11:29:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.480+0000 7f618a7fc700 1 -- 172.21.15.112:0/1172234709 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61900036a0 con 0x7f61940b57a0 2023-08-27T11:29:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.480+0000 7f619c213700 1 -- 172.21.15.112:0/1172234709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61940a9bd0 con 0x7f61940b57a0 2023-08-27T11:29:36.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.481+0000 7f619c213700 1 -- 172.21.15.112:0/1172234709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61940aa0b0 con 0x7f61940b57a0 2023-08-27T11:29:36.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.481+0000 7f618a7fc700 1 -- 172.21.15.112:0/1172234709 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f6190021030 con 0x7f61940b57a0 2023-08-27T11:29:36.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.482+0000 7f618a7fc700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f617806a4d0 0x7f617806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:36.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.482+0000 7f618a7fc700 1 -- 172.21.15.112:0/1172234709 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6190089f70 con 0x7f61940b57a0 2023-08-27T11:29:36.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.482+0000 7f6199faf700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f617806a4d0 0x7f617806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:36.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.482+0000 7f6199faf700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f617806a4d0 0x7f617806c990 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6184009b40 tx=0x7f6184009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:36.493 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.486+0000 7f618a7fc700 1 -- 172.21.15.112:0/1172234709 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f61900552c0 con 0x7f61940b57a0 2023-08-27T11:29:36.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:36.718+0000 7f619c213700 1 -- 172.21.15.112:0/1172234709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6194067a20 con 0x7f61940b57a0 2023-08-27T11:29:37.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.358+0000 7f618a7fc700 1 -- 172.21.15.112:0/1172234709 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v114) v1 ==== 136+0+0 (secure 0 0 0) 0x7f61900588e0 con 0x7f61940b57a0 2023-08-27T11:29:37.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.360+0000 7f619c213700 1 -- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f617806a4d0 msgr2=0x7f617806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:37.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.360+0000 7f619c213700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f617806a4d0 0x7f617806c990 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6184009b40 tx=0x7f6184009210 comp rx=0 tx=0).stop 2023-08-27T11:29:37.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.360+0000 7f619c213700 1 -- 172.21.15.112:0/1172234709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61940b57a0 msgr2=0x7f61940a89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:37.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.360+0000 7f619c213700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61940b57a0 0x7f61940a89c0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f6190009c40 tx=0x7f6190004750 comp rx=0 tx=0).stop 2023-08-27T11:29:37.362 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.360+0000 7f619c213700 1 -- 172.21.15.112:0/1172234709 shutdown_connections 2023-08-27T11:29:37.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.360+0000 7f619c213700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f617806a4d0 0x7f617806c990 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.360+0000 7f619c213700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6194144bd0 0x7f61940ab2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.360+0000 7f619c213700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f619406bfd0 0x7f619413faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.360+0000 7f619c213700 1 --2- 172.21.15.112:0/1172234709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61940b57a0 0x7f61940a89c0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.360+0000 7f619c213700 1 -- 172.21.15.112:0/1172234709 >> 172.21.15.112:0/1172234709 conn(0x7f6194066010 msgr2=0x7f61940b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:37.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.360+0000 7f619c213700 1 -- 172.21.15.112:0/1172234709 shutdown_connections 2023-08-27T11:29:37.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.360+0000 7f619c213700 1 -- 172.21.15.112:0/1172234709 wait complete. 2023-08-27T11:29:37.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:29:37.431 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:29:37.433 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:29:37.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.503+0000 7f86eff05700 1 Processor -- start 2023-08-27T11:29:37.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.503+0000 7f86eff05700 1 -- start start 2023-08-27T11:29:37.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86eff05700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f86e806e150 0x7f86e80cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:37.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86eff05700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f86e813f990 0x7f86e806e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:37.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86eff05700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e806ea70 con 0x7f86e80cd270 2023-08-27T11:29:37.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86eff05700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e80cecb0 con 0x7f86e806e150 2023-08-27T11:29:37.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86eff05700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e80cf570 con 0x7f86e813f990 2023-08-27T11:29:37.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86edca1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f86e806e150 0x7f86e80cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:37.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86ee4a2700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f86e80cd270 0x7f86e815ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60982/0 (socket says 172.21.15.112:60982) 2023-08-27T11:29:37.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86edca1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f86e806e150 0x7f86e80cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38364/0 (socket says 172.21.15.112:38364) 2023-08-27T11:29:37.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86ee4a2700 1 -- 172.21.15.112:0/2254758702 learned_addr learned my addr 172.21.15.112:0/2254758702 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:37.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86ed4a0700 1 --2- 172.21.15.112:0/2254758702 >> v2:172.21.15.138:3300/0 conn(0x7f86e813f990 0x7f86e806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:37.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86edca1700 1 -- 172.21.15.112:0/2254758702 >> v2:172.21.15.138:3300/0 conn(0x7f86e813f990 msgr2=0x7f86e806e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:37.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86edca1700 1 --2- 172.21.15.112:0/2254758702 >> v2:172.21.15.138:3300/0 conn(0x7f86e813f990 0x7f86e806e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86edca1700 1 -- 172.21.15.112:0/2254758702 >> v1:172.21.15.112:6789/0 conn(0x7f86e80cd270 legacy=0x7f86e815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:37.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86ecc9f700 1 -- 172.21.15.112:0/2254758702 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23852207 0 0) 0x7f86e806ea70 con 0x7f86e80cd270 2023-08-27T11:29:37.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.504+0000 7f86edca1700 1 -- 172.21.15.112:0/2254758702 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e813ff50 con 0x7f86e806e150 2023-08-27T11:29:37.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.505+0000 7f86edca1700 1 --2- 172.21.15.112:0/2254758702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e806e150 0x7f86e80cf030 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f86e806cb80 tx=0x7f86d8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3309eb8ad490fd3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:37.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.505+0000 7f86ecc9f700 1 -- 172.21.15.112:0/2254758702 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8011830 con 0x7f86e806e150 2023-08-27T11:29:37.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.505+0000 7f86ecc9f700 1 -- 172.21.15.112:0/2254758702 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d8011990 con 0x7f86e806e150 2023-08-27T11:29:37.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.505+0000 7f86ecc9f700 1 -- 172.21.15.112:0/2254758702 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8011d20 con 0x7f86e806e150 2023-08-27T11:29:37.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.505+0000 7f86eff05700 1 -- 172.21.15.112:0/2254758702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e806e150 msgr2=0x7f86e80cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:37.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.505+0000 7f86eff05700 1 --2- 172.21.15.112:0/2254758702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e806e150 0x7f86e80cf030 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f86e806cb80 tx=0x7f86d8009b20 comp rx=0 tx=0).stop 2023-08-27T11:29:37.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.505+0000 7f86eff05700 1 -- 172.21.15.112:0/2254758702 shutdown_connections 2023-08-27T11:29:37.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.505+0000 7f86eff05700 1 --2- 172.21.15.112:0/2254758702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e806e150 0x7f86e80cf030 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.505+0000 7f86eff05700 1 --2- 172.21.15.112:0/2254758702 >> v2:172.21.15.138:3300/0 conn(0x7f86e813f990 0x7f86e806e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.505+0000 7f86eff05700 1 -- 172.21.15.112:0/2254758702 >> 172.21.15.112:0/2254758702 conn(0x7f86e806cd30 msgr2=0x7f86e80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.505+0000 7f86eff05700 1 -- 172.21.15.112:0/2254758702 shutdown_connections 2023-08-27T11:29:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.505+0000 7f86eff05700 1 -- 172.21.15.112:0/2254758702 wait complete. 2023-08-27T11:29:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.507+0000 7f86eff05700 1 Processor -- start 2023-08-27T11:29:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.507+0000 7f86eff05700 1 -- start start 2023-08-27T11:29:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.507+0000 7f86eff05700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e806e150 0x7f86e813d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.507+0000 7f86eff05700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e80cd270 0x7f86e80afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.507+0000 7f86eff05700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e813f990 0x7f86e80b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.507+0000 7f86eff05700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e8155800 con 0x7f86e813f990 2023-08-27T11:29:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.507+0000 7f86eff05700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80b0310 con 0x7f86e806e150 2023-08-27T11:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.507+0000 7f86eff05700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80b05f0 con 0x7f86e80cd270 2023-08-27T11:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.507+0000 7f86ee4a2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e813f990 0x7f86e80b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.507+0000 7f86ee4a2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e813f990 0x7f86e80b13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38374/0 (socket says 172.21.15.112:38374) 2023-08-27T11:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.507+0000 7f86ee4a2700 1 -- 172.21.15.112:0/445662250 learned_addr learned my addr 172.21.15.112:0/445662250 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86edca1700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e806e150 0x7f86e813d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:37.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86ee4a2700 1 -- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e80cd270 msgr2=0x7f86e80afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:37.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86ed4a0700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e80cd270 0x7f86e80afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:37.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86ee4a2700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e80cd270 0x7f86e80afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86ee4a2700 1 -- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e806e150 msgr2=0x7f86e813d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:37.517 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86ee4a2700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e806e150 0x7f86e813d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86ee4a2700 1 -- 172.21.15.112:0/445662250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d80094a0 con 0x7f86e813f990 2023-08-27T11:29:37.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86edca1700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e806e150 0x7f86e813d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:37.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86ed4a0700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e80cd270 0x7f86e80afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:37.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86ee4a2700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e813f990 0x7f86e80b13b0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f86e0002230 tx=0x7f86e000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:37.518 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86de7fc700 1 -- 172.21.15.112:0/445662250 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e000ac20 con 0x7f86e813f990 2023-08-27T11:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86de7fc700 1 -- 172.21.15.112:0/445662250 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e000ada0 con 0x7f86e813f990 2023-08-27T11:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86eff05700 1 -- 172.21.15.112:0/445662250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e80b08d0 con 0x7f86e813f990 2023-08-27T11:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86de7fc700 1 -- 172.21.15.112:0/445662250 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0007310 con 0x7f86e813f990 2023-08-27T11:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.508+0000 7f86eff05700 1 -- 172.21.15.112:0/445662250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e80b0e90 con 0x7f86e813f990 2023-08-27T11:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.509+0000 7f86eff05700 1 -- 172.21.15.112:0/445662250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e80b10b0 con 0x7f86e813f990 2023-08-27T11:29:37.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.509+0000 7f86de7fc700 1 -- 172.21.15.112:0/445662250 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f86e0007470 con 0x7f86e813f990 2023-08-27T11:29:37.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.510+0000 7f86de7fc700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f86cc06a4d0 0x7f86cc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:37.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.510+0000 7f86de7fc700 1 -- 172.21.15.112:0/445662250 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f86e0088bc0 con 0x7f86e813f990 2023-08-27T11:29:37.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.510+0000 7f86edca1700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f86cc06a4d0 0x7f86cc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:37.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.510+0000 7f86edca1700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f86cc06a4d0 0x7f86cc06c990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f86d80090b0 tx=0x7f86d8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:37.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.514+0000 7f86de7fc700 1 -- 172.21.15.112:0/445662250 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f86e0053cd0 con 0x7f86e813f990 2023-08-27T11:29:37.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.729+0000 7f86eff05700 1 -- 172.21.15.112:0/445662250 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f86e8140510 con 0x7f86cc06a4d0 2023-08-27T11:29:37.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.731+0000 7f86de7fc700 1 -- 172.21.15.112:0/445662250 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f86d8011630 con 0x7f86cc06a4d0 2023-08-27T11:29:37.732 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:29:37.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:37.733 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephfs" 2023-08-27T11:29:37.733 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:37.733 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:29:37.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.733+0000 7f86eff05700 1 -- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f86cc06a4d0 msgr2=0x7f86cc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:37.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.733+0000 7f86eff05700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f86cc06a4d0 0x7f86cc06c990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f86d80090b0 tx=0x7f86d8008030 comp rx=0 tx=0).stop 2023-08-27T11:29:37.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.733+0000 7f86eff05700 1 -- 172.21.15.112:0/445662250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e813f990 msgr2=0x7f86e80b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:37.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.733+0000 7f86eff05700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e813f990 0x7f86e80b13b0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f86e0002230 tx=0x7f86e000a060 comp rx=0 tx=0).stop 2023-08-27T11:29:37.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.733+0000 7f86eff05700 1 -- 172.21.15.112:0/445662250 shutdown_connections 2023-08-27T11:29:37.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.733+0000 7f86eff05700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f86cc06a4d0 0x7f86cc06c990 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.733+0000 7f86eff05700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e813f990 0x7f86e80b13b0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.733+0000 7f86eff05700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e806e150 0x7f86e813d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.733+0000 7f86eff05700 1 --2- 172.21.15.112:0/445662250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e80cd270 0x7f86e80afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.733+0000 7f86eff05700 1 -- 172.21.15.112:0/445662250 >> 172.21.15.112:0/445662250 conn(0x7f86e806cd30 msgr2=0x7f86e8158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:37.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.733+0000 7f86eff05700 1 -- 172.21.15.112:0/445662250 shutdown_connections 2023-08-27T11:29:37.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.733+0000 7f86eff05700 1 -- 172.21.15.112:0/445662250 wait complete. 2023-08-27T11:29:37.746 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:29:37.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.908+0000 7fc82a625700 1 Processor -- start 2023-08-27T11:29:37.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.908+0000 7fc82a625700 1 -- start start 2023-08-27T11:29:37.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.908+0000 7fc82a625700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc82415d3f0 0x7fc8240ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:37.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc82a625700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc82406fea0 0x7fc8240b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:37.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc82a625700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8240b0c30 con 0x7fc8240b1350 2023-08-27T11:29:37.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc82a625700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8240b3a10 con 0x7fc82415d3f0 2023-08-27T11:29:37.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc82a625700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc824161d60 con 0x7fc82406fea0 2023-08-27T11:29:37.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc823fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc82415d3f0 0x7fc8240ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:37.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc823fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc82415d3f0 0x7fc8240ad5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38378/0 (socket says 172.21.15.112:38378) 2023-08-27T11:29:37.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc8237fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc8240b1350 0x7fc82406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60992/0 (socket says 172.21.15.112:60992) 2023-08-27T11:29:37.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc823fff700 1 -- 172.21.15.112:0/2933819000 learned_addr learned my addr 172.21.15.112:0/2933819000 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:37.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc828bc2700 1 --2- 172.21.15.112:0/2933819000 >> v2:172.21.15.138:3300/0 conn(0x7fc82406fea0 0x7fc8240b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:37.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc823fff700 1 -- 172.21.15.112:0/2933819000 >> v2:172.21.15.138:3300/0 conn(0x7fc82406fea0 msgr2=0x7fc8240b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:37.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc823fff700 1 --2- 172.21.15.112:0/2933819000 >> v2:172.21.15.138:3300/0 conn(0x7fc82406fea0 0x7fc8240b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc823fff700 1 -- 172.21.15.112:0/2933819000 >> v1:172.21.15.112:6789/0 conn(0x7fc8240b1350 legacy=0x7fc82406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:37.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc823fff700 1 -- 172.21.15.112:0/2933819000 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240b3cb0 con 0x7fc82415d3f0 2023-08-27T11:29:37.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc822ffd700 1 -- 172.21.15.112:0/2933819000 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548263080 0 0) 0x7fc8240b0c30 con 0x7fc8240b1350 2023-08-27T11:29:37.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc823fff700 1 --2- 172.21.15.112:0/2933819000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82415d3f0 0x7fc8240ad5f0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fc8140098f0 tx=0x7fc814009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d77638a7ca703b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:37.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc822ffd700 1 -- 172.21.15.112:0/2933819000 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81400b860 con 0x7fc82415d3f0 2023-08-27T11:29:37.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc822ffd700 1 -- 172.21.15.112:0/2933819000 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81400b9c0 con 0x7fc82415d3f0 2023-08-27T11:29:37.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.909+0000 7fc822ffd700 1 -- 172.21.15.112:0/2933819000 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81400bd50 con 0x7fc82415d3f0 2023-08-27T11:29:37.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.910+0000 7fc82a625700 1 -- 172.21.15.112:0/2933819000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82415d3f0 msgr2=0x7fc8240ad5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:37.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.910+0000 7fc82a625700 1 --2- 172.21.15.112:0/2933819000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82415d3f0 0x7fc8240ad5f0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fc8140098f0 tx=0x7fc814009c00 comp rx=0 tx=0).stop 2023-08-27T11:29:37.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.910+0000 7fc82a625700 1 -- 172.21.15.112:0/2933819000 shutdown_connections 2023-08-27T11:29:37.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.910+0000 7fc82a625700 1 --2- 172.21.15.112:0/2933819000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82415d3f0 0x7fc8240ad5f0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.910+0000 7fc82a625700 1 --2- 172.21.15.112:0/2933819000 >> v2:172.21.15.138:3300/0 conn(0x7fc82406fea0 0x7fc8240b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.910+0000 7fc82a625700 1 -- 172.21.15.112:0/2933819000 >> 172.21.15.112:0/2933819000 conn(0x7fc8240b1e90 msgr2=0x7fc8240b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:37.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.910+0000 7fc82a625700 1 -- 172.21.15.112:0/2933819000 shutdown_connections 2023-08-27T11:29:37.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.910+0000 7fc82a625700 1 -- 172.21.15.112:0/2933819000 wait complete. 2023-08-27T11:29:37.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.911+0000 7fc82a625700 1 Processor -- start 2023-08-27T11:29:37.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.911+0000 7fc82a625700 1 -- start start 2023-08-27T11:29:37.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc82a625700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82406fea0 0x7fc824143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:37.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc82a625700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8240b1350 0x7fc82415aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:37.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc82a625700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415d3f0 0x7fc82415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:37.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc82a625700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824143ea0 con 0x7fc82406fea0 2023-08-27T11:29:37.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc82a625700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824152f00 con 0x7fc8240b1350 2023-08-27T11:29:37.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc82a625700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8241531e0 con 0x7fc82415d3f0 2023-08-27T11:29:37.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc823fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82406fea0 0x7fc824143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:37.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc823fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82406fea0 0x7fc824143960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38382/0 (socket says 172.21.15.112:38382) 2023-08-27T11:29:37.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc823fff700 1 -- 172.21.15.112:0/3055504926 learned_addr learned my addr 172.21.15.112:0/3055504926 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:37.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc8237fe700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8240b1350 0x7fc82415aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:37.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc828bc2700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415d3f0 0x7fc82415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:37.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc823fff700 1 -- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415d3f0 msgr2=0x7fc82415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:37.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc823fff700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415d3f0 0x7fc82415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc823fff700 1 -- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8240b1350 msgr2=0x7fc82415aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:37.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc823fff700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8240b1350 0x7fc82415aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:37.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc823fff700 1 -- 172.21.15.112:0/3055504926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc814009550 con 0x7fc82406fea0 2023-08-27T11:29:37.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.912+0000 7fc823fff700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82406fea0 0x7fc824143960 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fc82406e8b0 tx=0x7fc814004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:37.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.913+0000 7fc820ff9700 1 -- 172.21.15.112:0/3055504926 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814004400 con 0x7fc82406fea0 2023-08-27T11:29:37.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.913+0000 7fc820ff9700 1 -- 172.21.15.112:0/3055504926 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81401f070 con 0x7fc82406fea0 2023-08-27T11:29:37.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.913+0000 7fc82a625700 1 -- 172.21.15.112:0/3055504926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8241534c0 con 0x7fc82406fea0 2023-08-27T11:29:37.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.913+0000 7fc820ff9700 1 -- 172.21.15.112:0/3055504926 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8140038e0 con 0x7fc82406fea0 2023-08-27T11:29:37.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.913+0000 7fc82a625700 1 -- 172.21.15.112:0/3055504926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc824153a80 con 0x7fc82406fea0 2023-08-27T11:29:37.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.914+0000 7fc82a625700 1 -- 172.21.15.112:0/3055504926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc824067a20 con 0x7fc82406fea0 2023-08-27T11:29:37.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.914+0000 7fc820ff9700 1 -- 172.21.15.112:0/3055504926 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fc814021030 con 0x7fc82406fea0 2023-08-27T11:29:37.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.915+0000 7fc820ff9700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc81006a4d0 0x7fc81006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:37.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.915+0000 7fc820ff9700 1 -- 172.21.15.112:0/3055504926 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc814089fc0 con 0x7fc82406fea0 2023-08-27T11:29:37.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.915+0000 7fc8237fe700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc81006a4d0 0x7fc81006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:37.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.915+0000 7fc8237fe700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc81006a4d0 0x7fc81006c990 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc8241f5df0 tx=0x7fc818007be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:37.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:37.919+0000 7fc820ff9700 1 -- 172.21.15.112:0/3055504926 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc814055270 con 0x7fc82406fea0 2023-08-27T11:29:38.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.109+0000 7fc82a625700 1 -- 172.21.15.112:0/3055504926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc8240b0470 con 0x7fc82406fea0 2023-08-27T11:29:38.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.109+0000 7fc820ff9700 1 -- 172.21.15.112:0/3055504926 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc814006d80 con 0x7fc82406fea0 2023-08-27T11:29:38.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.112+0000 7fc82a625700 1 -- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc81006a4d0 msgr2=0x7fc81006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.113 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.112+0000 7fc82a625700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc81006a4d0 0x7fc81006c990 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc8241f5df0 tx=0x7fc818007be0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.112+0000 7fc82a625700 1 -- 172.21.15.112:0/3055504926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82406fea0 msgr2=0x7fc824143960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.112+0000 7fc82a625700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82406fea0 0x7fc824143960 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fc82406e8b0 tx=0x7fc814004750 comp rx=0 tx=0).stop 2023-08-27T11:29:38.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.112+0000 7fc82a625700 1 -- 172.21.15.112:0/3055504926 shutdown_connections 2023-08-27T11:29:38.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.112+0000 7fc82a625700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc81006a4d0 0x7fc81006c990 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.112+0000 7fc82a625700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415d3f0 0x7fc82415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.112+0000 7fc82a625700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc82406fea0 0x7fc824143960 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.112+0000 7fc82a625700 1 --2- 172.21.15.112:0/3055504926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8240b1350 0x7fc82415aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.112+0000 7fc82a625700 1 -- 172.21.15.112:0/3055504926 >> 172.21.15.112:0/3055504926 conn(0x7fc8240b1e90 msgr2=0x7fc8240b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:38.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.112+0000 7fc82a625700 1 -- 172.21.15.112:0/3055504926 shutdown_connections 2023-08-27T11:29:38.115 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.112+0000 7fc82a625700 1 -- 172.21.15.112:0/3055504926 wait complete. 2023-08-27T11:29:38.125 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:29:38.125 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:38.142 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:29:38.143 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:38.200 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:38.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.359+0000 7f889ce78700 1 Processor -- start 2023-08-27T11:29:38.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.359+0000 7f889ce78700 1 -- start start 2023-08-27T11:29:38.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.360+0000 7f889ce78700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f889806dcc0 0x7f88980b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.360+0000 7f889ce78700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f88980b3440 0x7f88980b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.360+0000 7f889ce78700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889815b540 con 0x7f889815a120 2023-08-27T11:29:38.363 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.360+0000 7f889ce78700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88980af830 con 0x7f88980b3440 2023-08-27T11:29:38.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.360+0000 7f889ce78700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88980afb10 con 0x7f889806dcc0 2023-08-27T11:29:38.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.360+0000 7f8896d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f88980b3440 0x7f88980b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.360+0000 7f8896d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f88980b3440 0x7f88980b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38394/0 (socket says 172.21.15.112:38394) 2023-08-27T11:29:38.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.360+0000 7f8896d9d700 1 -- 172.21.15.112:0/2447606967 learned_addr learned my addr 172.21.15.112:0/2447606967 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:38.364 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.360+0000 7f8895d9b700 1 --2- 172.21.15.112:0/2447606967 >> v2:172.21.15.138:3300/0 conn(0x7f889806dcc0 0x7f88980b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.360+0000 7f8896d9d700 1 -- 172.21.15.112:0/2447606967 >> v2:172.21.15.138:3300/0 conn(0x7f889806dcc0 msgr2=0x7f88980b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:38.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.360+0000 7f8896d9d700 1 --2- 172.21.15.112:0/2447606967 >> v2:172.21.15.138:3300/0 conn(0x7f889806dcc0 0x7f88980b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.360+0000 7f8896d9d700 1 -- 172.21.15.112:0/2447606967 >> v1:172.21.15.138:6789/0 conn(0x7f889815a120 legacy=0x7f889806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.365 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.360+0000 7f8896d9d700 1 -- 172.21.15.112:0/2447606967 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88980afdf0 con 0x7f88980b3440 2023-08-27T11:29:38.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.361+0000 7f8896d9d700 1 --2- 172.21.15.112:0/2447606967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980b3440 0x7f88980b38b0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f88980cd2f0 tx=0x7f889000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b2be301500f176 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:38.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.361+0000 7f889559a700 1 -- 172.21.15.112:0/2447606967 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889000e070 con 0x7f88980b3440 2023-08-27T11:29:38.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.361+0000 7f889559a700 1 -- 172.21.15.112:0/2447606967 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889000b910 con 0x7f88980b3440 2023-08-27T11:29:38.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.361+0000 7f889559a700 1 -- 172.21.15.112:0/2447606967 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889000bcc0 con 0x7f88980b3440 2023-08-27T11:29:38.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.361+0000 7f889ce78700 1 -- 172.21.15.112:0/2447606967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980b3440 msgr2=0x7f88980b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.366 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.361+0000 7f889ce78700 1 --2- 172.21.15.112:0/2447606967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980b3440 0x7f88980b38b0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f88980cd2f0 tx=0x7f889000b330 comp rx=0 tx=0).stop 2023-08-27T11:29:38.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.361+0000 7f889ce78700 1 -- 172.21.15.112:0/2447606967 shutdown_connections 2023-08-27T11:29:38.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.361+0000 7f889ce78700 1 --2- 172.21.15.112:0/2447606967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88980b3440 0x7f88980b38b0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.361+0000 7f889ce78700 1 --2- 172.21.15.112:0/2447606967 >> v2:172.21.15.138:3300/0 conn(0x7f889806dcc0 0x7f88980b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.367 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.361+0000 7f889ce78700 1 -- 172.21.15.112:0/2447606967 >> 172.21.15.112:0/2447606967 conn(0x7f88980cff80 msgr2=0x7f88980d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:38.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.361+0000 7f889ce78700 1 -- 172.21.15.112:0/2447606967 shutdown_connections 2023-08-27T11:29:38.368 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.361+0000 7f889ce78700 1 -- 172.21.15.112:0/2447606967 wait complete. 2023-08-27T11:29:38.370 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.363+0000 7f889ce78700 1 Processor -- start 2023-08-27T11:29:38.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.363+0000 7f889ce78700 1 -- start start 2023-08-27T11:29:38.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.363+0000 7f889ce78700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889806dcc0 0x7f8898158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.363+0000 7f889ce78700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88980b3440 0x7f8898148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.363+0000 7f889ce78700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889815a120 0x7f889814ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.371 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.363+0000 7f889ce78700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8898158660 con 0x7f889815a120 2023-08-27T11:29:38.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.363+0000 7f889ce78700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88981590e0 con 0x7f88980b3440 2023-08-27T11:29:38.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.363+0000 7f889ce78700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88980cb050 con 0x7f889806dcc0 2023-08-27T11:29:38.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f8896d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889815a120 0x7f889814ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f8896d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889815a120 0x7f889814ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38400/0 (socket says 172.21.15.112:38400) 2023-08-27T11:29:38.372 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f8896d9d700 1 -- 172.21.15.112:0/2948948448 learned_addr learned my addr 172.21.15.112:0/2948948448 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:38.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f8895d9b700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88980b3440 0x7f8898148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f889659c700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889806dcc0 0x7f8898158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f8896d9d700 1 -- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889806dcc0 msgr2=0x7f8898158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.373 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f8896d9d700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889806dcc0 0x7f8898158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f8896d9d700 1 -- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88980b3440 msgr2=0x7f8898148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f8896d9d700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88980b3440 0x7f8898148f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f8896d9d700 1 -- 172.21.15.112:0/2948948448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8890009b30 con 0x7f889815a120 2023-08-27T11:29:38.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f8896d9d700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889815a120 0x7f889814ceb0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f88980d0d40 tx=0x7f8890004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:38.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f8886ffd700 1 -- 172.21.15.112:0/2948948448 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890004880 con 0x7f889815a120 2023-08-27T11:29:38.374 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f8886ffd700 1 -- 172.21.15.112:0/2948948448 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88900099b0 con 0x7f889815a120 2023-08-27T11:29:38.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f889ce78700 1 -- 172.21.15.112:0/2948948448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88980cb330 con 0x7f889815a120 2023-08-27T11:29:38.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f8886ffd700 1 -- 172.21.15.112:0/2948948448 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889001c680 con 0x7f889815a120 2023-08-27T11:29:38.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.364+0000 7f889ce78700 1 -- 172.21.15.112:0/2948948448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88980cb8f0 con 0x7f889815a120 2023-08-27T11:29:38.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.365+0000 7f889ce78700 1 -- 172.21.15.112:0/2948948448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88980cbe10 con 0x7f889815a120 2023-08-27T11:29:38.375 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.366+0000 7f8886ffd700 1 -- 172.21.15.112:0/2948948448 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f889001c7e0 con 0x7f889815a120 2023-08-27T11:29:38.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.366+0000 7f8886ffd700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f887806a4d0 0x7f887806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.366+0000 7f8886ffd700 1 -- 172.21.15.112:0/2948948448 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f889008a940 con 0x7f889815a120 2023-08-27T11:29:38.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.366+0000 7f889659c700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f887806a4d0 0x7f887806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.367+0000 7f889659c700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f887806a4d0 0x7f887806c990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8898072780 tx=0x7f8888009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:38.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.371+0000 7f8886ffd700 1 -- 172.21.15.112:0/2948948448 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f88900559d0 con 0x7f889815a120 2023-08-27T11:29:38.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.562+0000 7f889ce78700 1 -- 172.21.15.112:0/2948948448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88980b38b0 con 0x7f889815a120 2023-08-27T11:29:38.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.563+0000 7f8886ffd700 1 -- 172.21.15.112:0/2948948448 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+18259 (secure 0 0 0) 0x7f8890058ff0 con 0x7f889815a120 2023-08-27T11:29:38.565 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:38.565 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":167,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:24.544980+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:29:20.569324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-08-27T11:29:21.905812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.565+0000 7f889ce78700 1 -- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f887806a4d0 msgr2=0x7f887806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.565+0000 7f889ce78700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f887806a4d0 0x7f887806c990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8898072780 tx=0x7f8888009040 comp rx=0 tx=0).stop 2023-08-27T11:29:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.565+0000 7f889ce78700 1 -- 172.21.15.112:0/2948948448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889815a120 msgr2=0x7f889814ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.565+0000 7f889ce78700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889815a120 0x7f889814ceb0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f88980d0d40 tx=0x7f8890004ad0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.565+0000 7f889ce78700 1 -- 172.21.15.112:0/2948948448 shutdown_connections 2023-08-27T11:29:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.565+0000 7f889ce78700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f887806a4d0 0x7f887806c990 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.565+0000 7f889ce78700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889815a120 0x7f889814ceb0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.565+0000 7f889ce78700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889806dcc0 0x7f8898158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.565+0000 7f889ce78700 1 --2- 172.21.15.112:0/2948948448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88980b3440 0x7f8898148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.565+0000 7f889ce78700 1 -- 172.21.15.112:0/2948948448 >> 172.21.15.112:0/2948948448 conn(0x7f88980cff80 msgr2=0x7f889815a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.565+0000 7f889ce78700 1 -- 172.21.15.112:0/2948948448 shutdown_connections 2023-08-27T11:29:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.565+0000 7f889ce78700 1 -- 172.21.15.112:0/2948948448 wait complete. 2023-08-27T11:29:38.579 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:29:20.569324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:29:21.905812+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:29:38.579 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:29:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.701+0000 7efc895dd700 1 Processor -- start 2023-08-27T11:29:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.701+0000 7efc895dd700 1 -- start start 2023-08-27T11:29:38.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc895dd700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efc84146bf0 0x7efc840b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.706 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc895dd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc84146fd0 0x7efc840cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc895dd700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc840d1500 con 0x7efc840d0290 2023-08-27T11:29:38.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc895dd700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc840d0710 con 0x7efc84146bf0 2023-08-27T11:29:38.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc895dd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc840d09f0 con 0x7efc84146fd0 2023-08-27T11:29:38.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc837fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7efc840d0290 0x7efc84070eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32790/0 (socket says 172.21.15.112:32790) 2023-08-27T11:29:38.707 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc837fe700 1 -- 172.21.15.112:0/1617324270 learned_addr learned my addr 172.21.15.112:0/1617324270 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:38.708 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc82ffd700 1 --2- 172.21.15.112:0/1617324270 >> v2:172.21.15.138:3300/0 conn(0x7efc84146bf0 0x7efc840b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892552634 0 0) 0x7efc840d1500 con 0x7efc840d0290 2023-08-27T11:29:38.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc827fc700 1 --2- 172.21.15.112:0/1617324270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc84146fd0 0x7efc840cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc6c0035b0 con 0x7efc840d0290 2023-08-27T11:29:38.709 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 456558295 0 0) 0x7efc6c0035b0 con 0x7efc840d0290 2023-08-27T11:29:38.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc84146fd0 msgr2=0x7efc840cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:38.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc81ffb700 1 --2- 172.21.15.112:0/1617324270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc84146fd0 0x7efc840cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 >> v2:172.21.15.138:3300/0 conn(0x7efc84146bf0 msgr2=0x7efc840b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:38.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc81ffb700 1 --2- 172.21.15.112:0/1617324270 >> v2:172.21.15.138:3300/0 conn(0x7efc84146bf0 0x7efc840b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc840d0cd0 con 0x7efc840d0290 2023-08-27T11:29:38.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7efc7c0019f0 con 0x7efc840d0290 2023-08-27T11:29:38.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 >> v1:172.21.15.112:6789/0 conn(0x7efc840d0290 legacy=0x7efc84070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.702+0000 7efc81ffb700 1 --2- 172.21.15.112:0/1617324270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc6c003a00 0x7efc6c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc81ffb700 1 --2- 172.21.15.112:0/1617324270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc6c004850 0x7efc6c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc81ffb700 1 --2- 172.21.15.112:0/1617324270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc84146fd0 0x7efc6c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840d0710 con 0x7efc6c003a00 2023-08-27T11:29:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840d09f0 con 0x7efc6c004850 2023-08-27T11:29:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c00ba10 con 0x7efc84146fd0 2023-08-27T11:29:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc82ffd700 1 --2- 172.21.15.112:0/1617324270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc6c003a00 0x7efc6c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc827fc700 1 --2- 172.21.15.112:0/1617324270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc6c004850 0x7efc6c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc837fe700 1 --2- 172.21.15.112:0/1617324270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc84146fd0 0x7efc6c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc82ffd700 1 -- 172.21.15.112:0/1617324270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc84146fd0 msgr2=0x7efc6c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc82ffd700 1 --2- 172.21.15.112:0/1617324270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc84146fd0 0x7efc6c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc82ffd700 1 -- 172.21.15.112:0/1617324270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc6c004850 msgr2=0x7efc6c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc82ffd700 1 --2- 172.21.15.112:0/1617324270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc6c004850 0x7efc6c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc82ffd700 1 -- 172.21.15.112:0/1617324270 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6c0035b0 con 0x7efc6c003a00 2023-08-27T11:29:38.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc837fe700 1 --2- 172.21.15.112:0/1617324270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc84146fd0 0x7efc6c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:38.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc82ffd700 1 --2- 172.21.15.112:0/1617324270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc6c003a00 0x7efc6c003e20 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7efc78009b90 tx=0x7efc6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbde34e9a352299a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:38.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7800e6c0 con 0x7efc6c003a00 2023-08-27T11:29:38.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7800e820 con 0x7efc6c003a00 2023-08-27T11:29:38.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.703+0000 7efc81ffb700 1 -- 172.21.15.112:0/1617324270 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7800eb90 con 0x7efc6c003a00 2023-08-27T11:29:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.704+0000 7efc895dd700 1 -- 172.21.15.112:0/1617324270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc6c003a00 msgr2=0x7efc6c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.704+0000 7efc895dd700 1 --2- 172.21.15.112:0/1617324270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc6c003a00 0x7efc6c003e20 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7efc78009b90 tx=0x7efc6c005880 comp rx=0 tx=0).stop 2023-08-27T11:29:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.704+0000 7efc837fe700 1 -- 172.21.15.112:0/1617324270 reap_dead start 2023-08-27T11:29:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.704+0000 7efc895dd700 1 -- 172.21.15.112:0/1617324270 shutdown_connections 2023-08-27T11:29:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.704+0000 7efc895dd700 1 -- 172.21.15.112:0/1617324270 >> 172.21.15.112:0/1617324270 conn(0x7efc8406c450 msgr2=0x7efc8406f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:38.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.704+0000 7efc895dd700 1 -- 172.21.15.112:0/1617324270 shutdown_connections 2023-08-27T11:29:38.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.704+0000 7efc895dd700 1 -- 172.21.15.112:0/1617324270 wait complete. 2023-08-27T11:29:38.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.705+0000 7efc895dd700 1 Processor -- start 2023-08-27T11:29:38.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.705+0000 7efc895dd700 1 -- start start 2023-08-27T11:29:38.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc895dd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc6c003a00 0x7efc840b9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc895dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc840b8ad0 0x7efc840ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc895dd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc8413f0e0 0x7efc8415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc895dd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840ad6a0 con 0x7efc840b8ad0 2023-08-27T11:29:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc895dd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840b7d10 con 0x7efc8413f0e0 2023-08-27T11:29:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc895dd700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc840b7ff0 con 0x7efc6c003a00 2023-08-27T11:29:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc827fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc840b8ad0 0x7efc840ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc827fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc840b8ad0 0x7efc840ad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38418/0 (socket says 172.21.15.112:38418) 2023-08-27T11:29:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc827fc700 1 -- 172.21.15.112:0/1226521740 learned_addr learned my addr 172.21.15.112:0/1226521740 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc837fe700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc8413f0e0 0x7efc8415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc82ffd700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc6c003a00 0x7efc840b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc827fc700 1 -- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc6c003a00 msgr2=0x7efc840b9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc827fc700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc6c003a00 0x7efc840b9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc827fc700 1 -- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc8413f0e0 msgr2=0x7efc8415b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc827fc700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc8413f0e0 0x7efc8415b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc827fc700 1 -- 172.21.15.112:0/1226521740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc840b82d0 con 0x7efc840b8ad0 2023-08-27T11:29:38.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc82ffd700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc6c003a00 0x7efc840b9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:38.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc827fc700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc840b8ad0 0x7efc840ad160 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7efc740116a0 tx=0x7efc740119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:38.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.706+0000 7efc6b7fe700 1 -- 172.21.15.112:0/1226521740 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74015870 con 0x7efc840b8ad0 2023-08-27T11:29:38.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.707+0000 7efc6b7fe700 1 -- 172.21.15.112:0/1226521740 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc740159d0 con 0x7efc840b8ad0 2023-08-27T11:29:38.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.707+0000 7efc895dd700 1 -- 172.21.15.112:0/1226521740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc840b85b0 con 0x7efc840b8ad0 2023-08-27T11:29:38.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.707+0000 7efc6b7fe700 1 -- 172.21.15.112:0/1226521740 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74015d80 con 0x7efc840b8ad0 2023-08-27T11:29:38.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.707+0000 7efc895dd700 1 -- 172.21.15.112:0/1226521740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8413f590 con 0x7efc840b8ad0 2023-08-27T11:29:38.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.707+0000 7efc895dd700 1 -- 172.21.15.112:0/1226521740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc84067e90 con 0x7efc840b8ad0 2023-08-27T11:29:38.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.708+0000 7efc6b7fe700 1 -- 172.21.15.112:0/1226521740 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7efc74013400 con 0x7efc840b8ad0 2023-08-27T11:29:38.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.708+0000 7efc6b7fe700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc7006a4d0 0x7efc7006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:38.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.708+0000 7efc6b7fe700 1 -- 172.21.15.112:0/1226521740 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efc74089000 con 0x7efc840b8ad0 2023-08-27T11:29:38.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.709+0000 7efc82ffd700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc7006a4d0 0x7efc7006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:38.723 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.709+0000 7efc82ffd700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc7006a4d0 0x7efc7006c990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7efc780090b0 tx=0x7efc78008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.713+0000 7efc6b7fe700 1 -- 172.21.15.112:0/1226521740 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efc74057ba0 con 0x7efc840b8ad0 2023-08-27T11:29:38.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.899+0000 7efc895dd700 1 -- 172.21.15.112:0/1226521740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efc84067a20 con 0x7efc840b8ad0 2023-08-27T11:29:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.899+0000 7efc6b7fe700 1 -- 172.21.15.112:0/1226521740 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v167) v1 ==== 92+0+10 (secure 0 0 0) 0x7efc74057730 con 0x7efc840b8ad0 2023-08-27T11:29:38.901 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:29:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.901+0000 7efc895dd700 1 -- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc7006a4d0 msgr2=0x7efc7006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.901+0000 7efc895dd700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc7006a4d0 0x7efc7006c990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7efc780090b0 tx=0x7efc78008030 comp rx=0 tx=0).stop 2023-08-27T11:29:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.901+0000 7efc895dd700 1 -- 172.21.15.112:0/1226521740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc840b8ad0 msgr2=0x7efc840ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:38.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.901+0000 7efc895dd700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc840b8ad0 0x7efc840ad160 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7efc740116a0 tx=0x7efc740119b0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.901+0000 7efc895dd700 1 -- 172.21.15.112:0/1226521740 shutdown_connections 2023-08-27T11:29:38.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.901+0000 7efc895dd700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc7006a4d0 0x7efc7006c990 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.901+0000 7efc895dd700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc8413f0e0 0x7efc8415b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.904 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.901+0000 7efc895dd700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc6c003a00 0x7efc840b9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.901+0000 7efc895dd700 1 --2- 172.21.15.112:0/1226521740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc840b8ad0 0x7efc840ad160 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:38.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.901+0000 7efc895dd700 1 -- 172.21.15.112:0/1226521740 >> 172.21.15.112:0/1226521740 conn(0x7efc8406c450 msgr2=0x7efc840d1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:38.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.901+0000 7efc895dd700 1 -- 172.21.15.112:0/1226521740 shutdown_connections 2023-08-27T11:29:38.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:38.901+0000 7efc895dd700 1 -- 172.21.15.112:0/1226521740 wait complete. 2023-08-27T11:29:38.914 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:29:39.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.036+0000 7feb9a9cf700 1 Processor -- start 2023-08-27T11:29:39.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.036+0000 7feb9a9cf700 1 -- start start 2023-08-27T11:29:39.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.037+0000 7feb9a9cf700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feb94144b80 0x7feb940b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.037+0000 7feb9a9cf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb940b3de0 0x7feb9406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.040 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.037+0000 7feb9a9cf700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb940cef20 con 0x7feb9406c510 2023-08-27T11:29:39.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.037+0000 7feb9a9cf700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb940b4b40 con 0x7feb94144b80 2023-08-27T11:29:39.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.037+0000 7feb9a9cf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9406c940 con 0x7feb940b3de0 2023-08-27T11:29:39.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.037+0000 7feb93fff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feb94144b80 0x7feb940b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.037+0000 7feb937fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb940b3de0 0x7feb9406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.041 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.037+0000 7feb937fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb940b3de0 0x7feb9406bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:50078/0 (socket says 172.21.15.112:50078) 2023-08-27T11:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.037+0000 7feb93fff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feb94144b80 0x7feb940b6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:38852/0 (socket says 172.21.15.112:38852) 2023-08-27T11:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.037+0000 7feb937fe700 1 -- 172.21.15.112:0/1572259438 learned_addr learned my addr 172.21.15.112:0/1572259438 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.037+0000 7feb98f6c700 1 --1- 172.21.15.112:0/1572259438 >> v1:172.21.15.138:6789/0 conn(0x7feb9406c510 0x7feb9415eeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:40334/0 (socket says 172.21.15.112:40334) 2023-08-27T11:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.038+0000 7feb92ffd700 1 -- 172.21.15.112:0/1572259438 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292774225 0 0) 0x7feb940cef20 con 0x7feb9406c510 2023-08-27T11:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.038+0000 7feb92ffd700 1 -- 172.21.15.112:0/1572259438 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb7c0035b0 con 0x7feb9406c510 2023-08-27T11:29:39.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.038+0000 7feb93fff700 1 -- 172.21.15.112:0/1572259438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb940b3de0 msgr2=0x7feb9406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:39.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.038+0000 7feb93fff700 1 --2- 172.21.15.112:0/1572259438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb940b3de0 0x7feb9406bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.038+0000 7feb93fff700 1 -- 172.21.15.112:0/1572259438 >> v1:172.21.15.138:6789/0 conn(0x7feb9406c510 legacy=0x7feb9415eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.038+0000 7feb93fff700 1 -- 172.21.15.112:0/1572259438 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb940b42a0 con 0x7feb94144b80 2023-08-27T11:29:39.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.038+0000 7feb937fe700 1 --2- 172.21.15.112:0/1572259438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb940b3de0 0x7feb9406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.038+0000 7feb93fff700 1 --2- 172.21.15.112:0/1572259438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94144b80 0x7feb940b6c00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7feb88009840 tx=0x7feb88009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f800763c6ca6f383 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.038+0000 7feb92ffd700 1 -- 172.21.15.112:0/1572259438 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88011830 con 0x7feb94144b80 2023-08-27T11:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.038+0000 7feb92ffd700 1 -- 172.21.15.112:0/1572259438 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb88011990 con 0x7feb94144b80 2023-08-27T11:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.038+0000 7feb92ffd700 1 -- 172.21.15.112:0/1572259438 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88011d20 con 0x7feb94144b80 2023-08-27T11:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.039+0000 7feb9a9cf700 1 -- 172.21.15.112:0/1572259438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94144b80 msgr2=0x7feb940b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.039+0000 7feb9a9cf700 1 --2- 172.21.15.112:0/1572259438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94144b80 0x7feb940b6c00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7feb88009840 tx=0x7feb88009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.039+0000 7feb9a9cf700 1 -- 172.21.15.112:0/1572259438 shutdown_connections 2023-08-27T11:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.039+0000 7feb9a9cf700 1 --2- 172.21.15.112:0/1572259438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94144b80 0x7feb940b6c00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.039+0000 7feb9a9cf700 1 --2- 172.21.15.112:0/1572259438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb940b3de0 0x7feb9406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.039+0000 7feb9a9cf700 1 -- 172.21.15.112:0/1572259438 >> 172.21.15.112:0/1572259438 conn(0x7feb940cdc50 msgr2=0x7feb940ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.039+0000 7feb9a9cf700 1 -- 172.21.15.112:0/1572259438 shutdown_connections 2023-08-27T11:29:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.039+0000 7feb9a9cf700 1 -- 172.21.15.112:0/1572259438 wait complete. 2023-08-27T11:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.040+0000 7feb9a9cf700 1 Processor -- start 2023-08-27T11:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.040+0000 7feb9a9cf700 1 -- start start 2023-08-27T11:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb9a9cf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb9406c510 0x7feb94144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb9a9cf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb940b3de0 0x7feb94159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb9a9cf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94144b80 0x7feb94156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb9a9cf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb941561a0 con 0x7feb9406c510 2023-08-27T11:29:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb9a9cf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb94155590 con 0x7feb94144b80 2023-08-27T11:29:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb9a9cf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb94155870 con 0x7feb940b3de0 2023-08-27T11:29:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb93fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb9406c510 0x7feb94144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb93fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb9406c510 0x7feb94144020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38430/0 (socket says 172.21.15.112:38430) 2023-08-27T11:29:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb93fff700 1 -- 172.21.15.112:0/3678556220 learned_addr learned my addr 172.21.15.112:0/3678556220 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb937fe700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb940b3de0 0x7feb94159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb98f6c700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94144b80 0x7feb94156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb93fff700 1 -- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb940b3de0 msgr2=0x7feb94159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb93fff700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb940b3de0 0x7feb94159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb93fff700 1 -- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94144b80 msgr2=0x7feb94156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb93fff700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94144b80 0x7feb94156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb93fff700 1 -- 172.21.15.112:0/3678556220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb84009550 con 0x7feb9406c510 2023-08-27T11:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb93fff700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb9406c510 0x7feb94144020 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7feb88009fd0 tx=0x7feb88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb90ff9700 1 -- 172.21.15.112:0/3678556220 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb880036a0 con 0x7feb9406c510 2023-08-27T11:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb90ff9700 1 -- 172.21.15.112:0/3678556220 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8801e070 con 0x7feb9406c510 2023-08-27T11:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb9a9cf700 1 -- 172.21.15.112:0/3678556220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb880094a0 con 0x7feb9406c510 2023-08-27T11:29:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.041+0000 7feb90ff9700 1 -- 172.21.15.112:0/3678556220 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88003a60 con 0x7feb9406c510 2023-08-27T11:29:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.042+0000 7feb9a9cf700 1 -- 172.21.15.112:0/3678556220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb94155e30 con 0x7feb9406c510 2023-08-27T11:29:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.043+0000 7feb9a9cf700 1 -- 172.21.15.112:0/3678556220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb94156480 con 0x7feb9406c510 2023-08-27T11:29:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.043+0000 7feb90ff9700 1 -- 172.21.15.112:0/3678556220 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7feb88003bc0 con 0x7feb9406c510 2023-08-27T11:29:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.043+0000 7feb90ff9700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feb8006a4d0 0x7feb8006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.043+0000 7feb90ff9700 1 -- 172.21.15.112:0/3678556220 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feb88089a20 con 0x7feb9406c510 2023-08-27T11:29:39.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.044+0000 7feb937fe700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feb8006a4d0 0x7feb8006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.044+0000 7feb937fe700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feb8006a4d0 0x7feb8006c990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7feb94144560 tx=0x7feb84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:39.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.048+0000 7feb90ff9700 1 -- 172.21.15.112:0/3678556220 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7feb88054cd0 con 0x7feb9406c510 2023-08-27T11:29:39.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.236+0000 7feb9a9cf700 1 -- 172.21.15.112:0/3678556220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb940b3100 con 0x7feb9406c510 2023-08-27T11:29:39.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.237+0000 7feb90ff9700 1 -- 172.21.15.112:0/3678556220 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb88015020 con 0x7feb9406c510 2023-08-27T11:29:39.238 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:29:39.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.238+0000 7feb9a9cf700 1 -- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feb8006a4d0 msgr2=0x7feb8006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.239+0000 7feb9a9cf700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feb8006a4d0 0x7feb8006c990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7feb94144560 tx=0x7feb84008040 comp rx=0 tx=0).stop 2023-08-27T11:29:39.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.239+0000 7feb9a9cf700 1 -- 172.21.15.112:0/3678556220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb9406c510 msgr2=0x7feb94144020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.239+0000 7feb9a9cf700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb9406c510 0x7feb94144020 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7feb88009fd0 tx=0x7feb88004750 comp rx=0 tx=0).stop 2023-08-27T11:29:39.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.239+0000 7feb9a9cf700 1 -- 172.21.15.112:0/3678556220 shutdown_connections 2023-08-27T11:29:39.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.239+0000 7feb9a9cf700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7feb8006a4d0 0x7feb8006c990 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.239+0000 7feb9a9cf700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94144b80 0x7feb94156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.239+0000 7feb9a9cf700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb9406c510 0x7feb94144020 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.239+0000 7feb9a9cf700 1 --2- 172.21.15.112:0/3678556220 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb940b3de0 0x7feb94159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.239+0000 7feb9a9cf700 1 -- 172.21.15.112:0/3678556220 >> 172.21.15.112:0/3678556220 conn(0x7feb940cdc50 msgr2=0x7feb940c6170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.239+0000 7feb9a9cf700 1 -- 172.21.15.112:0/3678556220 shutdown_connections 2023-08-27T11:29:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.239+0000 7feb9a9cf700 1 -- 172.21.15.112:0/3678556220 wait complete. 2023-08-27T11:29:39.252 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:29:39.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.374+0000 7f29cb082700 1 Processor -- start 2023-08-27T11:29:39.377 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.375+0000 7f29cb082700 1 -- start start 2023-08-27T11:29:39.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29cb082700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29c41445b0 0x7f29c40ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29cb082700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f29c406fea0 0x7f29c415b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29cb082700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c40b08e0 con 0x7f29c40ccad0 2023-08-27T11:29:39.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29cb082700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c40ccf80 con 0x7f29c41445b0 2023-08-27T11:29:39.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29cb082700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c415bc70 con 0x7f29c406fea0 2023-08-27T11:29:39.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29c8e1e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29c41445b0 0x7f29c40ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29c8e1e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29c41445b0 0x7f29c40ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38438/0 (socket says 172.21.15.112:38438) 2023-08-27T11:29:39.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29c8e1e700 1 -- 172.21.15.112:0/251300305 learned_addr learned my addr 172.21.15.112:0/251300305 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:39.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29c961f700 1 --2- 172.21.15.112:0/251300305 >> v2:172.21.15.138:3300/0 conn(0x7f29c406fea0 0x7f29c415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29c8e1e700 1 -- 172.21.15.112:0/251300305 >> v2:172.21.15.138:3300/0 conn(0x7f29c406fea0 msgr2=0x7f29c415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:39.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29c8e1e700 1 --2- 172.21.15.112:0/251300305 >> v2:172.21.15.138:3300/0 conn(0x7f29c406fea0 0x7f29c415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29c8e1e700 1 -- 172.21.15.112:0/251300305 >> v1:172.21.15.138:6789/0 conn(0x7f29c40ccad0 legacy=0x7f29c406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29c8e1e700 1 -- 172.21.15.112:0/251300305 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c40b0b30 con 0x7f29c41445b0 2023-08-27T11:29:39.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29c8e1e700 1 --2- 172.21.15.112:0/251300305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c41445b0 0x7f29c40ce210 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f29c40b0680 tx=0x7f29b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5aaf0fa5cf4b81 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:39.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29c37fe700 1 -- 172.21.15.112:0/251300305 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b800b870 con 0x7f29c41445b0 2023-08-27T11:29:39.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29c37fe700 1 -- 172.21.15.112:0/251300305 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b800b9d0 con 0x7f29c41445b0 2023-08-27T11:29:39.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.376+0000 7f29c37fe700 1 -- 172.21.15.112:0/251300305 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b800bd60 con 0x7f29c41445b0 2023-08-27T11:29:39.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.377+0000 7f29cb082700 1 -- 172.21.15.112:0/251300305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c41445b0 msgr2=0x7f29c40ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.377+0000 7f29cb082700 1 --2- 172.21.15.112:0/251300305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c41445b0 0x7f29c40ce210 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f29c40b0680 tx=0x7f29b8009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.377+0000 7f29cb082700 1 -- 172.21.15.112:0/251300305 shutdown_connections 2023-08-27T11:29:39.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.377+0000 7f29cb082700 1 --2- 172.21.15.112:0/251300305 >> v2:172.21.15.138:3300/0 conn(0x7f29c406fea0 0x7f29c415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.377+0000 7f29cb082700 1 --2- 172.21.15.112:0/251300305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c41445b0 0x7f29c40ce210 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.377+0000 7f29cb082700 1 -- 172.21.15.112:0/251300305 >> 172.21.15.112:0/251300305 conn(0x7f29c40b1e90 msgr2=0x7f29c40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:39.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.377+0000 7f29cb082700 1 -- 172.21.15.112:0/251300305 shutdown_connections 2023-08-27T11:29:39.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.377+0000 7f29cb082700 1 -- 172.21.15.112:0/251300305 wait complete. 2023-08-27T11:29:39.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.378+0000 7f29cb082700 1 Processor -- start 2023-08-27T11:29:39.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.378+0000 7f29cb082700 1 -- start start 2023-08-27T11:29:39.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29cb082700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c406fea0 0x7f29c40aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29cb082700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29c40ccad0 0x7f29c41efcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29cb082700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29c41445b0 0x7f29c41f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29cb082700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40abf40 con 0x7f29c406fea0 2023-08-27T11:29:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29cb082700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40aaea0 con 0x7f29c40ccad0 2023-08-27T11:29:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29cb082700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40ab180 con 0x7f29c41445b0 2023-08-27T11:29:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29c8e1e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c406fea0 0x7f29c40aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29c8e1e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c406fea0 0x7f29c40aeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38444/0 (socket says 172.21.15.112:38444) 2023-08-27T11:29:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29c8e1e700 1 -- 172.21.15.112:0/2992740064 learned_addr learned my addr 172.21.15.112:0/2992740064 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29c8e1e700 1 -- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29c41445b0 msgr2=0x7f29c41f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29c8e1e700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29c41445b0 0x7f29c41f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29c8e1e700 1 -- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29c40ccad0 msgr2=0x7f29c41efcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29c3fff700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29c40ccad0 0x7f29c41efcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29c8e1e700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29c40ccad0 0x7f29c41efcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29c8e1e700 1 -- 172.21.15.112:0/2992740064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b8009550 con 0x7f29c406fea0 2023-08-27T11:29:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.379+0000 7f29c3fff700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29c40ccad0 0x7f29c41efcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.380+0000 7f29c8e1e700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c406fea0 0x7f29c40aeab0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f29b8000c00 tx=0x7f29b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:39.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.380+0000 7f29c17fa700 1 -- 172.21.15.112:0/2992740064 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8004520 con 0x7f29c406fea0 2023-08-27T11:29:39.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.380+0000 7f29c17fa700 1 -- 172.21.15.112:0/2992740064 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b801e070 con 0x7f29c406fea0 2023-08-27T11:29:39.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.380+0000 7f29cb082700 1 -- 172.21.15.112:0/2992740064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c40ab460 con 0x7f29c406fea0 2023-08-27T11:29:39.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.380+0000 7f29c17fa700 1 -- 172.21.15.112:0/2992740064 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b80038a0 con 0x7f29c406fea0 2023-08-27T11:29:39.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.380+0000 7f29cb082700 1 -- 172.21.15.112:0/2992740064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c40aba20 con 0x7f29c406fea0 2023-08-27T11:29:39.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.381+0000 7f29cb082700 1 -- 172.21.15.112:0/2992740064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c4067e90 con 0x7f29c406fea0 2023-08-27T11:29:39.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.381+0000 7f29c17fa700 1 -- 172.21.15.112:0/2992740064 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f29b8003a00 con 0x7f29c406fea0 2023-08-27T11:29:39.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.382+0000 7f29c17fa700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f29a806a4d0 0x7f29a806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.382+0000 7f29c17fa700 1 -- 172.21.15.112:0/2992740064 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29b8020070 con 0x7f29c406fea0 2023-08-27T11:29:39.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.382+0000 7f29c3fff700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f29a806a4d0 0x7f29a806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.382+0000 7f29c3fff700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f29a806a4d0 0x7f29a806c990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f29c415ba60 tx=0x7f29b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:39.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.386+0000 7f29c17fa700 1 -- 172.21.15.112:0/2992740064 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f29b8055400 con 0x7f29c406fea0 2023-08-27T11:29:39.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.575+0000 7f29cb082700 1 -- 172.21.15.112:0/2992740064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29c40cd170 con 0x7f29c406fea0 2023-08-27T11:29:39.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.575+0000 7f29c17fa700 1 -- 172.21.15.112:0/2992740064 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v167) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29b8006ce0 con 0x7f29c406fea0 2023-08-27T11:29:39.577 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:29:39.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.577+0000 7f29cb082700 1 -- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f29a806a4d0 msgr2=0x7f29a806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.577+0000 7f29cb082700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f29a806a4d0 0x7f29a806c990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f29c415ba60 tx=0x7f29b0009040 comp rx=0 tx=0).stop 2023-08-27T11:29:39.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.577+0000 7f29cb082700 1 -- 172.21.15.112:0/2992740064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c406fea0 msgr2=0x7f29c40aeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.577+0000 7f29cb082700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c406fea0 0x7f29c40aeab0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f29b8000c00 tx=0x7f29b8004750 comp rx=0 tx=0).stop 2023-08-27T11:29:39.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.577+0000 7f29cb082700 1 -- 172.21.15.112:0/2992740064 shutdown_connections 2023-08-27T11:29:39.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.577+0000 7f29cb082700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f29a806a4d0 0x7f29a806c990 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.577+0000 7f29cb082700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29c41445b0 0x7f29c41f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.577+0000 7f29cb082700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c406fea0 0x7f29c40aeab0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.577+0000 7f29cb082700 1 --2- 172.21.15.112:0/2992740064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29c40ccad0 0x7f29c41efcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.577+0000 7f29cb082700 1 -- 172.21.15.112:0/2992740064 >> 172.21.15.112:0/2992740064 conn(0x7f29c40b1e90 msgr2=0x7f29c40cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:39.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.577+0000 7f29cb082700 1 -- 172.21.15.112:0/2992740064 shutdown_connections 2023-08-27T11:29:39.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.577+0000 7f29cb082700 1 -- 172.21.15.112:0/2992740064 wait complete. 2023-08-27T11:29:39.591 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:29:39.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.713+0000 7f8979de1700 1 Processor -- start 2023-08-27T11:29:39.714 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.713+0000 7f8979de1700 1 -- start start 2023-08-27T11:29:39.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.713+0000 7f8979de1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f89740cca90 0x7f897415f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.713+0000 7f8979de1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f897406f790 0x7f897406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.716 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f8979de1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897415ee90 con 0x7f8974144900 2023-08-27T11:29:39.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f8979de1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8974070140 con 0x7f897406f790 2023-08-27T11:29:39.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f8979de1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89740b11d0 con 0x7f89740cca90 2023-08-27T11:29:39.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f89737fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8974144900 0x7f89740b09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32834/0 (socket says 172.21.15.112:32834) 2023-08-27T11:29:39.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f8973fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f897406f790 0x7f897406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.717 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f89737fe700 1 -- 172.21.15.112:0/264909668 learned_addr learned my addr 172.21.15.112:0/264909668 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:39.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f8972ffd700 1 --2- 172.21.15.112:0/264909668 >> v2:172.21.15.138:3300/0 conn(0x7f89740cca90 0x7f897415f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f8973fff700 1 -- 172.21.15.112:0/264909668 >> v2:172.21.15.138:3300/0 conn(0x7f89740cca90 msgr2=0x7f897415f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:39.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f8973fff700 1 --2- 172.21.15.112:0/264909668 >> v2:172.21.15.138:3300/0 conn(0x7f89740cca90 0x7f897415f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f8973fff700 1 -- 172.21.15.112:0/264909668 >> v1:172.21.15.112:6789/0 conn(0x7f8974144900 legacy=0x7f89740b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.718 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f8973fff700 1 -- 172.21.15.112:0/264909668 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89740b14b0 con 0x7f897406f790 2023-08-27T11:29:39.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f89727fc700 1 -- 172.21.15.112:0/264909668 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656068870 0 0) 0x7f897415ee90 con 0x7f8974144900 2023-08-27T11:29:39.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f8973fff700 1 --2- 172.21.15.112:0/264909668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897406f790 0x7f897406fc00 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f89740703e0 tx=0x7f896c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737b8de41b546e9b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:39.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f89727fc700 1 -- 172.21.15.112:0/264909668 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c00e070 con 0x7f897406f790 2023-08-27T11:29:39.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f89727fc700 1 -- 172.21.15.112:0/264909668 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896c00bbd0 con 0x7f897406f790 2023-08-27T11:29:39.719 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.714+0000 7f89727fc700 1 -- 172.21.15.112:0/264909668 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c012430 con 0x7f897406f790 2023-08-27T11:29:39.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.715+0000 7f8979de1700 1 -- 172.21.15.112:0/264909668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897406f790 msgr2=0x7f897406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.715+0000 7f8979de1700 1 --2- 172.21.15.112:0/264909668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897406f790 0x7f897406fc00 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f89740703e0 tx=0x7f896c00b330 comp rx=0 tx=0).stop 2023-08-27T11:29:39.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.715+0000 7f8979de1700 1 -- 172.21.15.112:0/264909668 shutdown_connections 2023-08-27T11:29:39.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.715+0000 7f8979de1700 1 --2- 172.21.15.112:0/264909668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897406f790 0x7f897406fc00 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.715+0000 7f8979de1700 1 --2- 172.21.15.112:0/264909668 >> v2:172.21.15.138:3300/0 conn(0x7f89740cca90 0x7f897415f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.720 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.715+0000 7f8979de1700 1 -- 172.21.15.112:0/264909668 >> 172.21.15.112:0/264909668 conn(0x7f89740b1e90 msgr2=0x7f89740b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:39.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.715+0000 7f8979de1700 1 -- 172.21.15.112:0/264909668 shutdown_connections 2023-08-27T11:29:39.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.715+0000 7f8979de1700 1 -- 172.21.15.112:0/264909668 wait complete. 2023-08-27T11:29:39.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.716+0000 7f8979de1700 1 Processor -- start 2023-08-27T11:29:39.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.716+0000 7f8979de1700 1 -- start start 2023-08-27T11:29:39.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8979de1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897406f790 0x7f89741340f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8979de1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89740cca90 0x7f8974143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8979de1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8974144900 0x7f89740cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8979de1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89740ca550 con 0x7f8974144900 2023-08-27T11:29:39.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8979de1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8974134940 con 0x7f89740cca90 2023-08-27T11:29:39.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8979de1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8974134c20 con 0x7f897406f790 2023-08-27T11:29:39.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8973fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8974144900 0x7f89740cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8973fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8974144900 0x7f89740cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38460/0 (socket says 172.21.15.112:38460) 2023-08-27T11:29:39.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8973fff700 1 -- 172.21.15.112:0/3275757834 learned_addr learned my addr 172.21.15.112:0/3275757834 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:39.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f89737fe700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897406f790 0x7f89741340f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8972ffd700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89740cca90 0x7f8974143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8973fff700 1 -- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897406f790 msgr2=0x7f89741340f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8973fff700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897406f790 0x7f89741340f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8973fff700 1 -- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89740cca90 msgr2=0x7f8974143790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8973fff700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89740cca90 0x7f8974143790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.717+0000 7f8973fff700 1 -- 172.21.15.112:0/3275757834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c009a70 con 0x7f8974144900 2023-08-27T11:29:39.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.718+0000 7f8973fff700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8974144900 0x7f89740cad10 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f89740b5830 tx=0x7f896c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:39.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.718+0000 7f895bfff700 1 -- 172.21.15.112:0/3275757834 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c012430 con 0x7f8974144900 2023-08-27T11:29:39.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.718+0000 7f895bfff700 1 -- 172.21.15.112:0/3275757834 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896c020070 con 0x7f8974144900 2023-08-27T11:29:39.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.718+0000 7f895bfff700 1 -- 172.21.15.112:0/3275757834 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c004280 con 0x7f8974144900 2023-08-27T11:29:39.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.718+0000 7f8979de1700 1 -- 172.21.15.112:0/3275757834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8974134f00 con 0x7f8974144900 2023-08-27T11:29:39.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.718+0000 7f8979de1700 1 -- 172.21.15.112:0/3275757834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8974134630 con 0x7f8974144900 2023-08-27T11:29:39.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.719+0000 7f8979de1700 1 -- 172.21.15.112:0/3275757834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89740caa30 con 0x7f8974144900 2023-08-27T11:29:39.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.719+0000 7f895bfff700 1 -- 172.21.15.112:0/3275757834 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f896c0043e0 con 0x7f8974144900 2023-08-27T11:29:39.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.719+0000 7f895bfff700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f896006a4d0 0x7f896006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.720+0000 7f895bfff700 1 -- 172.21.15.112:0/3275757834 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f896c022070 con 0x7f8974144900 2023-08-27T11:29:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.720+0000 7f89737fe700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f896006a4d0 0x7f896006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.720+0000 7f89737fe700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f896006a4d0 0x7f896006c990 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8964000f60 tx=0x7f8964007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:39.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.724+0000 7f895bfff700 1 -- 172.21.15.112:0/3275757834 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f89740caa30 con 0x7f8974144900 2023-08-27T11:29:39.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.913+0000 7f8979de1700 1 -- 172.21.15.112:0/3275757834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89740caa30 con 0x7f8974144900 2023-08-27T11:29:39.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.914+0000 7f895bfff700 1 -- 172.21.15.112:0/3275757834 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v167) v1 ==== 100+0+11 (secure 0 0 0) 0x7f89740caa30 con 0x7f8974144900 2023-08-27T11:29:39.916 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:29:39.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.916+0000 7f8979de1700 1 -- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f896006a4d0 msgr2=0x7f896006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.916+0000 7f8979de1700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f896006a4d0 0x7f896006c990 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8964000f60 tx=0x7f8964007e70 comp rx=0 tx=0).stop 2023-08-27T11:29:39.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.916+0000 7f8979de1700 1 -- 172.21.15.112:0/3275757834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8974144900 msgr2=0x7f89740cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:39.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.916+0000 7f8979de1700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8974144900 0x7f89740cad10 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f89740b5830 tx=0x7f896c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.916+0000 7f8979de1700 1 -- 172.21.15.112:0/3275757834 shutdown_connections 2023-08-27T11:29:39.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.916+0000 7f8979de1700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f896006a4d0 0x7f896006c990 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.916+0000 7f8979de1700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8974144900 0x7f89740cad10 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.916+0000 7f8979de1700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f897406f790 0x7f89741340f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.916+0000 7f8979de1700 1 --2- 172.21.15.112:0/3275757834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89740cca90 0x7f8974143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:39.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.916+0000 7f8979de1700 1 -- 172.21.15.112:0/3275757834 >> 172.21.15.112:0/3275757834 conn(0x7f89740b1e90 msgr2=0x7f89741419f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:39.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.916+0000 7f8979de1700 1 -- 172.21.15.112:0/3275757834 shutdown_connections 2023-08-27T11:29:39.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:39.916+0000 7f8979de1700 1 -- 172.21.15.112:0/3275757834 wait complete. 2023-08-27T11:29:39.929 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:29:40.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.050+0000 7f280697e700 1 Processor -- start 2023-08-27T11:29:40.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.050+0000 7f280697e700 1 -- start start 2023-08-27T11:29:40.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f280697e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f28000b1400 0x7f28000cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f280697e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2800161ce0 0x7f280006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f280697e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2800161850 con 0x7f28000b17e0 2023-08-27T11:29:40.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f280697e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28000cfc10 con 0x7f28000b1400 2023-08-27T11:29:40.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f280697e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2800161200 con 0x7f2800161ce0 2023-08-27T11:29:40.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f27fffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f28000b1400 0x7f28000cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f27fffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f28000b1400 0x7f28000cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38466/0 (socket says 172.21.15.112:38466) 2023-08-27T11:29:40.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f27fffff700 1 -- 172.21.15.112:0/1523964789 learned_addr learned my addr 172.21.15.112:0/1523964789 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:40.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f27ff7fe700 1 --1- 172.21.15.112:0/1523964789 >> v1:172.21.15.112:6789/0 conn(0x7f28000b17e0 0x7f280006e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32848/0 (socket says 172.21.15.112:32848) 2023-08-27T11:29:40.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f2804f1b700 1 --2- 172.21.15.112:0/1523964789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2800161ce0 0x7f280006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f27fffff700 1 -- 172.21.15.112:0/1523964789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2800161ce0 msgr2=0x7f280006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:40.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f27fffff700 1 --2- 172.21.15.112:0/1523964789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2800161ce0 0x7f280006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f27fffff700 1 -- 172.21.15.112:0/1523964789 >> v1:172.21.15.112:6789/0 conn(0x7f28000b17e0 legacy=0x7f280006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f27fffff700 1 -- 172.21.15.112:0/1523964789 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28001614e0 con 0x7f28000b1400 2023-08-27T11:29:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f27feffd700 1 -- 172.21.15.112:0/1523964789 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521121668 0 0) 0x7f2800161850 con 0x7f28000b17e0 2023-08-27T11:29:40.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.051+0000 7f2804f1b700 1 --2- 172.21.15.112:0/1523964789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2800161ce0 0x7f280006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:40.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.052+0000 7f27fffff700 1 --2- 172.21.15.112:0/1523964789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28000b1400 0x7f28000cf5a0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f27f40098f0 tx=0x7f27f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11799abac754bea2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:40.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.052+0000 7f27feffd700 1 -- 172.21.15.112:0/1523964789 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f400b8a0 con 0x7f28000b1400 2023-08-27T11:29:40.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.052+0000 7f27feffd700 1 -- 172.21.15.112:0/1523964789 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f400ba00 con 0x7f28000b1400 2023-08-27T11:29:40.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.052+0000 7f27feffd700 1 -- 172.21.15.112:0/1523964789 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f400bd90 con 0x7f28000b1400 2023-08-27T11:29:40.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.052+0000 7f280697e700 1 -- 172.21.15.112:0/1523964789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28000b1400 msgr2=0x7f28000cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.052+0000 7f280697e700 1 --2- 172.21.15.112:0/1523964789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28000b1400 0x7f28000cf5a0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f27f40098f0 tx=0x7f27f4009c00 comp rx=0 tx=0).stop 2023-08-27T11:29:40.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.052+0000 7f280697e700 1 -- 172.21.15.112:0/1523964789 shutdown_connections 2023-08-27T11:29:40.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.052+0000 7f280697e700 1 --2- 172.21.15.112:0/1523964789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2800161ce0 0x7f280006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.052+0000 7f280697e700 1 --2- 172.21.15.112:0/1523964789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28000b1400 0x7f28000cf5a0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.052+0000 7f280697e700 1 -- 172.21.15.112:0/1523964789 >> 172.21.15.112:0/1523964789 conn(0x7f28000d0000 msgr2=0x7f28000d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:40.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.052+0000 7f280697e700 1 -- 172.21.15.112:0/1523964789 shutdown_connections 2023-08-27T11:29:40.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.052+0000 7f280697e700 1 -- 172.21.15.112:0/1523964789 wait complete. 2023-08-27T11:29:40.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.054+0000 7f280697e700 1 Processor -- start 2023-08-27T11:29:40.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.054+0000 7f280697e700 1 -- start start 2023-08-27T11:29:40.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.054+0000 7f280697e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28000b1400 0x7f28000c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.054+0000 7f280697e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28000b17e0 0x7f28000c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f280697e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2800161ce0 0x7f2800154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f280697e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28000ca3b0 con 0x7f28000b17e0 2023-08-27T11:29:40.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f280697e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28000c9850 con 0x7f2800161ce0 2023-08-27T11:29:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f280697e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28000c3b30 con 0x7f28000b1400 2023-08-27T11:29:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f27ff7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28000b17e0 0x7f28000c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f27ff7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28000b17e0 0x7f28000c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38474/0 (socket says 172.21.15.112:38474) 2023-08-27T11:29:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f27ff7fe700 1 -- 172.21.15.112:0/154943091 learned_addr learned my addr 172.21.15.112:0/154943091 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f2804f1b700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2800161ce0 0x7f2800154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f27fffff700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28000b1400 0x7f28000c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f27ff7fe700 1 -- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28000b1400 msgr2=0x7f28000c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f27ff7fe700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28000b1400 0x7f28000c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f27ff7fe700 1 -- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2800161ce0 msgr2=0x7f2800154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f27ff7fe700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2800161ce0 0x7f2800154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f27ff7fe700 1 -- 172.21.15.112:0/154943091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f4009550 con 0x7f28000b17e0 2023-08-27T11:29:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f27ff7fe700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28000b17e0 0x7f28000c9e70 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f27f000c540 tx=0x7f27f000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f27fcff9700 1 -- 172.21.15.112:0/154943091 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0004c80 con 0x7f28000b17e0 2023-08-27T11:29:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f280697e700 1 -- 172.21.15.112:0/154943091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28000c3e10 con 0x7f28000b17e0 2023-08-27T11:29:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f280697e700 1 -- 172.21.15.112:0/154943091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28000c43d0 con 0x7f28000b17e0 2023-08-27T11:29:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f27fcff9700 1 -- 172.21.15.112:0/154943091 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f0004de0 con 0x7f28000b17e0 2023-08-27T11:29:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.055+0000 7f27fcff9700 1 -- 172.21.15.112:0/154943091 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f00072d0 con 0x7f28000b17e0 2023-08-27T11:29:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.056+0000 7f280697e700 1 -- 172.21.15.112:0/154943091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e00052f0 con 0x7f28000b17e0 2023-08-27T11:29:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.057+0000 7f27fcff9700 1 -- 172.21.15.112:0/154943091 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f27f0007430 con 0x7f28000b17e0 2023-08-27T11:29:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.057+0000 7f27fcff9700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27ec06a4d0 0x7f27ec06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.057+0000 7f27fcff9700 1 -- 172.21.15.112:0/154943091 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f27f0088630 con 0x7f28000b17e0 2023-08-27T11:29:40.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.057+0000 7f27fffff700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27ec06a4d0 0x7f27ec06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.058+0000 7f27fffff700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27ec06a4d0 0x7f27ec06c990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009c40 tx=0x7f27f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:40.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.061+0000 7f27fcff9700 1 -- 172.21.15.112:0/154943091 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f27f00537d0 con 0x7f28000b17e0 2023-08-27T11:29:40.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.266+0000 7f280697e700 1 -- 172.21.15.112:0/154943091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f27e00055a0 con 0x7f28000b17e0 2023-08-27T11:29:40.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.267+0000 7f27fcff9700 1 -- 172.21.15.112:0/154943091 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v167) v1 ==== 80+0+0 (secure 0 0 0) 0x7f27f0056df0 con 0x7f28000b17e0 2023-08-27T11:29:40.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.269+0000 7f280697e700 1 -- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27ec06a4d0 msgr2=0x7f27ec06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.269+0000 7f280697e700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27ec06a4d0 0x7f27ec06c990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009c40 tx=0x7f27f4008030 comp rx=0 tx=0).stop 2023-08-27T11:29:40.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.269+0000 7f280697e700 1 -- 172.21.15.112:0/154943091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28000b17e0 msgr2=0x7f28000c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.269+0000 7f280697e700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28000b17e0 0x7f28000c9e70 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f27f000c540 tx=0x7f27f000c850 comp rx=0 tx=0).stop 2023-08-27T11:29:40.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.269+0000 7f280697e700 1 -- 172.21.15.112:0/154943091 shutdown_connections 2023-08-27T11:29:40.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.269+0000 7f280697e700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f27ec06a4d0 0x7f27ec06c990 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.269+0000 7f280697e700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2800161ce0 0x7f2800154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.269+0000 7f280697e700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28000b1400 0x7f28000c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.269+0000 7f280697e700 1 --2- 172.21.15.112:0/154943091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28000b17e0 0x7f28000c9e70 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.269+0000 7f280697e700 1 -- 172.21.15.112:0/154943091 >> 172.21.15.112:0/154943091 conn(0x7f28000d0000 msgr2=0x7f28001556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:40.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.269+0000 7f280697e700 1 -- 172.21.15.112:0/154943091 shutdown_connections 2023-08-27T11:29:40.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.269+0000 7f280697e700 1 -- 172.21.15.112:0/154943091 wait complete. 2023-08-27T11:29:40.273 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:29:40.282 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:29:40.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:40.283 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:29:40.323 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:29:40.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:40.324 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:29:40.379 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:29:40.380 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:40.399 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:29:40.400 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:40.460 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:40.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.579+0000 7f95c5c6a700 1 Processor -- start 2023-08-27T11:29:40.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.579+0000 7f95c5c6a700 1 -- start start 2023-08-27T11:29:40.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.580+0000 7f95c5c6a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f95c00b17e0 0x7f95c00cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.580+0000 7f95c5c6a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c0161850 con 0x7f95c00b1400 2023-08-27T11:29:40.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.580+0000 7f95c5c6a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c00cfc10 con 0x7f95c0161ce0 2023-08-27T11:29:40.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.580+0000 7f95c5c6a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c0161200 con 0x7f95c00b17e0 2023-08-27T11:29:40.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.580+0000 7f95beffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f95c00b17e0 0x7f95c00cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.580+0000 7f95bf7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f95c00b1400 0x7f95c006e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32862/0 (socket says 172.21.15.112:32862) 2023-08-27T11:29:40.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.580+0000 7f95bf7fe700 1 -- 172.21.15.112:0/3858447042 learned_addr learned my addr 172.21.15.112:0/3858447042 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:40.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.581+0000 7f95beffd700 1 -- 172.21.15.112:0/3858447042 >> v1:172.21.15.138:6789/0 conn(0x7f95c0161ce0 legacy=0x7f95c006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:40.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.581+0000 7f95beffd700 1 -- 172.21.15.112:0/3858447042 >> v1:172.21.15.112:6789/0 conn(0x7f95c00b1400 legacy=0x7f95c006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.581+0000 7f95be7fc700 1 -- 172.21.15.112:0/3858447042 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394087110 0 0) 0x7f95c0161850 con 0x7f95c00b1400 2023-08-27T11:29:40.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.581+0000 7f95beffd700 1 -- 172.21.15.112:0/3858447042 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c01614e0 con 0x7f95c00b17e0 2023-08-27T11:29:40.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.581+0000 7f95beffd700 1 --2- 172.21.15.112:0/3858447042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b17e0 0x7f95c00cf5a0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009840 tx=0x7f95b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e10b4545d438e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:40.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.581+0000 7f95be7fc700 1 -- 172.21.15.112:0/3858447042 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b0004230 con 0x7f95c00b17e0 2023-08-27T11:29:40.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.581+0000 7f95be7fc700 1 -- 172.21.15.112:0/3858447042 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b0004390 con 0x7f95c00b17e0 2023-08-27T11:29:40.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.581+0000 7f95be7fc700 1 -- 172.21.15.112:0/3858447042 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b00036a0 con 0x7f95c00b17e0 2023-08-27T11:29:40.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.581+0000 7f95c5c6a700 1 -- 172.21.15.112:0/3858447042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b17e0 msgr2=0x7f95c00cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.581+0000 7f95c5c6a700 1 --2- 172.21.15.112:0/3858447042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b17e0 0x7f95c00cf5a0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009840 tx=0x7f95b0009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:40.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.581+0000 7f95c5c6a700 1 -- 172.21.15.112:0/3858447042 shutdown_connections 2023-08-27T11:29:40.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.581+0000 7f95c5c6a700 1 --2- 172.21.15.112:0/3858447042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b17e0 0x7f95c00cf5a0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.582+0000 7f95c5c6a700 1 -- 172.21.15.112:0/3858447042 >> 172.21.15.112:0/3858447042 conn(0x7f95c00d0000 msgr2=0x7f95c00d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:40.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.582+0000 7f95c5c6a700 1 -- 172.21.15.112:0/3858447042 shutdown_connections 2023-08-27T11:29:40.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.582+0000 7f95c5c6a700 1 -- 172.21.15.112:0/3858447042 wait complete. 2023-08-27T11:29:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.583+0000 7f95c5c6a700 1 Processor -- start 2023-08-27T11:29:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.583+0000 7f95c5c6a700 1 -- start start 2023-08-27T11:29:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95c5c6a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95c00b1400 0x7f95c00c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95c5c6a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b17e0 0x7f95c01505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95c5c6a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95c0161ce0 0x7f95c014e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95c5c6a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00c84d0 con 0x7f95c00b17e0 2023-08-27T11:29:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95c5c6a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00c6970 con 0x7f95c0161ce0 2023-08-27T11:29:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95c5c6a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c0150b20 con 0x7f95c00b1400 2023-08-27T11:29:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95beffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b17e0 0x7f95c01505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95beffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b17e0 0x7f95c01505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38488/0 (socket says 172.21.15.112:38488) 2023-08-27T11:29:40.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95beffd700 1 -- 172.21.15.112:0/1596053376 learned_addr learned my addr 172.21.15.112:0/1596053376 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:40.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95bffff700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95c0161ce0 0x7f95c014e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95bf7fe700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95c00b1400 0x7f95c00c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95beffd700 1 -- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95c00b1400 msgr2=0x7f95c00c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95beffd700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95c00b1400 0x7f95c00c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95beffd700 1 -- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95c0161ce0 msgr2=0x7f95c014e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95beffd700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95c0161ce0 0x7f95c014e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95beffd700 1 -- 172.21.15.112:0/1596053376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b00094a0 con 0x7f95c00b17e0 2023-08-27T11:29:40.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95bffff700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95c0161ce0 0x7f95c014e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:40.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95beffd700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b17e0 0x7f95c01505e0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f95b00090b0 tx=0x7f95b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:40.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95b7fff700 1 -- 172.21.15.112:0/1596053376 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b0011830 con 0x7f95c00b17e0 2023-08-27T11:29:40.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95b7fff700 1 -- 172.21.15.112:0/1596053376 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b001e070 con 0x7f95c00b17e0 2023-08-27T11:29:40.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95c5c6a700 1 -- 172.21.15.112:0/1596053376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c00c9e40 con 0x7f95c00b17e0 2023-08-27T11:29:40.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95b7fff700 1 -- 172.21.15.112:0/1596053376 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b0011bb0 con 0x7f95c00b17e0 2023-08-27T11:29:40.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.584+0000 7f95c5c6a700 1 -- 172.21.15.112:0/1596053376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c014fde0 con 0x7f95c00b17e0 2023-08-27T11:29:40.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.585+0000 7f95c5c6a700 1 -- 172.21.15.112:0/1596053376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c0067a20 con 0x7f95c00b17e0 2023-08-27T11:29:40.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.586+0000 7f95b7fff700 1 -- 172.21.15.112:0/1596053376 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f95b0011d10 con 0x7f95c00b17e0 2023-08-27T11:29:40.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.586+0000 7f95b7fff700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95a006a4d0 0x7f95a006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.586+0000 7f95b7fff700 1 -- 172.21.15.112:0/1596053376 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f95b0089e90 con 0x7f95c00b17e0 2023-08-27T11:29:40.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.586+0000 7f95bf7fe700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95a006a4d0 0x7f95a006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.587+0000 7f95bf7fe700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95a006a4d0 0x7f95a006c990 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f95a8000f60 tx=0x7f95a8007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:40.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.590+0000 7f95b7fff700 1 -- 172.21.15.112:0/1596053376 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f95b0055140 con 0x7f95c00b17e0 2023-08-27T11:29:40.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.781+0000 7f95c5c6a700 1 -- 172.21.15.112:0/1596053376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95c006db60 con 0x7f95c00b17e0 2023-08-27T11:29:40.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.782+0000 7f95b7fff700 1 -- 172.21.15.112:0/1596053376 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+18259 (secure 0 0 0) 0x7f95b0020090 con 0x7f95c00b17e0 2023-08-27T11:29:40.784 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:40.784 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":167,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:24.544980+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:29:20.569324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-08-27T11:29:21.905812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:40.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.784+0000 7f95c5c6a700 1 -- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95a006a4d0 msgr2=0x7f95a006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.784+0000 7f95c5c6a700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95a006a4d0 0x7f95a006c990 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f95a8000f60 tx=0x7f95a8007c00 comp rx=0 tx=0).stop 2023-08-27T11:29:40.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.784+0000 7f95c5c6a700 1 -- 172.21.15.112:0/1596053376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b17e0 msgr2=0x7f95c01505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.786 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.784+0000 7f95c5c6a700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b17e0 0x7f95c01505e0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f95b00090b0 tx=0x7f95b0004750 comp rx=0 tx=0).stop 2023-08-27T11:29:40.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.784+0000 7f95c5c6a700 1 -- 172.21.15.112:0/1596053376 shutdown_connections 2023-08-27T11:29:40.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.784+0000 7f95c5c6a700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f95a006a4d0 0x7f95a006c990 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.784+0000 7f95c5c6a700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95c0161ce0 0x7f95c014e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.784+0000 7f95c5c6a700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95c00b1400 0x7f95c00c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.787 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.784+0000 7f95c5c6a700 1 --2- 172.21.15.112:0/1596053376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95c00b17e0 0x7f95c01505e0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.784+0000 7f95c5c6a700 1 -- 172.21.15.112:0/1596053376 >> 172.21.15.112:0/1596053376 conn(0x7f95c00d0000 msgr2=0x7f95c015d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:40.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.784+0000 7f95c5c6a700 1 -- 172.21.15.112:0/1596053376 shutdown_connections 2023-08-27T11:29:40.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.785+0000 7f95c5c6a700 1 -- 172.21.15.112:0/1596053376 wait complete. 2023-08-27T11:29:40.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:29:20.569324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:29:21.905812+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:29:40.799 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:29:40.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.921+0000 7f1e661b8700 1 Processor -- start 2023-08-27T11:29:40.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.921+0000 7f1e661b8700 1 -- start start 2023-08-27T11:29:40.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.922+0000 7f1e661b8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e60146c20 0x7f1e60160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.922+0000 7f1e661b8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1e600b3440 0x7f1e6015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.922+0000 7f1e661b8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e600b5070 con 0x7f1e6015e6e0 2023-08-27T11:29:40.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.922+0000 7f1e661b8700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e60147000 con 0x7f1e600b3440 2023-08-27T11:29:40.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.922+0000 7f1e661b8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e601614a0 con 0x7f1e60146c20 2023-08-27T11:29:40.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.922+0000 7f1e5ffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1e6015e6e0 0x7f1e6006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32880/0 (socket says 172.21.15.112:32880) 2023-08-27T11:29:40.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.922+0000 7f1e5ffff700 1 -- 172.21.15.112:0/2729817562 learned_addr learned my addr 172.21.15.112:0/2729817562 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:40.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5f7fe700 1 --2- 172.21.15.112:0/2729817562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e60146c20 0x7f1e60160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5effd700 1 --2- 172.21.15.112:0/2729817562 >> v2:172.21.15.138:3300/0 conn(0x7f1e600b3440 0x7f1e6015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218794580 0 0) 0x7f1e58001490 con 0x7f1e6015e6e0 2023-08-27T11:29:40.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e4c0035b0 con 0x7f1e6015e6e0 2023-08-27T11:29:40.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3948476278 0 0) 0x7f1e4c0035b0 con 0x7f1e6015e6e0 2023-08-27T11:29:40.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e60146c20 msgr2=0x7f1e60160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:40.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 --2- 172.21.15.112:0/2729817562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e60146c20 0x7f1e60160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 >> v2:172.21.15.138:3300/0 conn(0x7f1e600b3440 msgr2=0x7f1e6015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:40.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 --2- 172.21.15.112:0/2729817562 >> v2:172.21.15.138:3300/0 conn(0x7f1e600b3440 0x7f1e6015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e60161740 con 0x7f1e6015e6e0 2023-08-27T11:29:40.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f1e58001b80 con 0x7f1e6015e6e0 2023-08-27T11:29:40.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 >> v1:172.21.15.112:6789/0 conn(0x7f1e6015e6e0 legacy=0x7f1e6006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 --2- 172.21.15.112:0/2729817562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c003a00 0x7f1e4c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.935 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 --2- 172.21.15.112:0/2729817562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e4c004850 0x7f1e4c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 --2- 172.21.15.112:0/2729817562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e60146c20 0x7f1e4c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e60147000 con 0x7f1e4c003a00 2023-08-27T11:29:40.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e601614a0 con 0x7f1e4c004850 2023-08-27T11:29:40.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c00ba10 con 0x7f1e60146c20 2023-08-27T11:29:40.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5f7fe700 1 --2- 172.21.15.112:0/2729817562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c003a00 0x7f1e4c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5effd700 1 --2- 172.21.15.112:0/2729817562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e4c004850 0x7f1e4c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5ffff700 1 --2- 172.21.15.112:0/2729817562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e60146c20 0x7f1e4c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5f7fe700 1 -- 172.21.15.112:0/2729817562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e60146c20 msgr2=0x7f1e4c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:40.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5f7fe700 1 --2- 172.21.15.112:0/2729817562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e60146c20 0x7f1e4c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5f7fe700 1 -- 172.21.15.112:0/2729817562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e4c004850 msgr2=0x7f1e4c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:40.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5f7fe700 1 --2- 172.21.15.112:0/2729817562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e4c004850 0x7f1e4c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5f7fe700 1 -- 172.21.15.112:0/2729817562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c0035b0 con 0x7f1e4c003a00 2023-08-27T11:29:40.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.923+0000 7f1e5f7fe700 1 --2- 172.21.15.112:0/2729817562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c003a00 0x7f1e4c003e20 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009b90 tx=0x7f1e4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6e86da5af52b87 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:40.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.924+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800e6c0 con 0x7f1e4c003a00 2023-08-27T11:29:40.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.924+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4800e820 con 0x7f1e4c003a00 2023-08-27T11:29:40.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.924+0000 7f1e5e7fc700 1 -- 172.21.15.112:0/2729817562 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800eb90 con 0x7f1e4c003a00 2023-08-27T11:29:40.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.924+0000 7f1e661b8700 1 -- 172.21.15.112:0/2729817562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c003a00 msgr2=0x7f1e4c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.924+0000 7f1e661b8700 1 --2- 172.21.15.112:0/2729817562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c003a00 0x7f1e4c003e20 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009b90 tx=0x7f1e4c005880 comp rx=0 tx=0).stop 2023-08-27T11:29:40.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.924+0000 7f1e5ffff700 1 -- 172.21.15.112:0/2729817562 reap_dead start 2023-08-27T11:29:40.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.924+0000 7f1e661b8700 1 -- 172.21.15.112:0/2729817562 shutdown_connections 2023-08-27T11:29:40.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.924+0000 7f1e661b8700 1 -- 172.21.15.112:0/2729817562 >> 172.21.15.112:0/2729817562 conn(0x7f1e600b1e90 msgr2=0x7f1e600b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:40.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.924+0000 7f1e661b8700 1 -- 172.21.15.112:0/2729817562 shutdown_connections 2023-08-27T11:29:40.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.924+0000 7f1e661b8700 1 -- 172.21.15.112:0/2729817562 wait complete. 2023-08-27T11:29:40.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.925+0000 7f1e661b8700 1 Processor -- start 2023-08-27T11:29:40.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.926+0000 7f1e661b8700 1 -- start start 2023-08-27T11:29:40.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.926+0000 7f1e661b8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c003a00 0x7f1e601f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.926+0000 7f1e661b8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e600aeac0 0x7f1e60133db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.926+0000 7f1e661b8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e601f62c0 0x7f1e6015a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.926+0000 7f1e661b8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e601345e0 con 0x7f1e4c003a00 2023-08-27T11:29:40.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.926+0000 7f1e661b8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e601348c0 con 0x7f1e601f62c0 2023-08-27T11:29:40.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.926+0000 7f1e661b8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e60134ba0 con 0x7f1e600aeac0 2023-08-27T11:29:40.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.926+0000 7f1e5f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c003a00 0x7f1e601f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.926+0000 7f1e5f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c003a00 0x7f1e601f2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38506/0 (socket says 172.21.15.112:38506) 2023-08-27T11:29:40.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.926+0000 7f1e5f7fe700 1 -- 172.21.15.112:0/2532812746 learned_addr learned my addr 172.21.15.112:0/2532812746 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:40.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e5effd700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e600aeac0 0x7f1e60133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e5ffff700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e601f62c0 0x7f1e6015a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e5f7fe700 1 -- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e600aeac0 msgr2=0x7f1e60133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e5f7fe700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e600aeac0 0x7f1e60133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e5f7fe700 1 -- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e601f62c0 msgr2=0x7f1e6015a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:40.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e5f7fe700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e601f62c0 0x7f1e6015a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:40.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e5f7fe700 1 -- 172.21.15.112:0/2532812746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e60134e80 con 0x7f1e4c003a00 2023-08-27T11:29:40.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e5ffff700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e601f62c0 0x7f1e6015a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:40.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e5f7fe700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c003a00 0x7f1e601f2ae0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c004740 tx=0x7f1e480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:40.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e57fff700 1 -- 172.21.15.112:0/2532812746 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800e6c0 con 0x7f1e4c003a00 2023-08-27T11:29:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e57fff700 1 -- 172.21.15.112:0/2532812746 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e48003ab0 con 0x7f1e4c003a00 2023-08-27T11:29:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e57fff700 1 -- 172.21.15.112:0/2532812746 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e48003e60 con 0x7f1e4c003a00 2023-08-27T11:29:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e661b8700 1 -- 172.21.15.112:0/2532812746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e60135160 con 0x7f1e4c003a00 2023-08-27T11:29:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.927+0000 7f1e661b8700 1 -- 172.21.15.112:0/2532812746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e601f6750 con 0x7f1e4c003a00 2023-08-27T11:29:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.928+0000 7f1e661b8700 1 -- 172.21.15.112:0/2532812746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e60067e90 con 0x7f1e4c003a00 2023-08-27T11:29:40.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.929+0000 7f1e57fff700 1 -- 172.21.15.112:0/2532812746 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f1e48013470 con 0x7f1e4c003a00 2023-08-27T11:29:40.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.929+0000 7f1e57fff700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e4006a4d0 0x7f1e4006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:40.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.929+0000 7f1e57fff700 1 -- 172.21.15.112:0/2532812746 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1e4808a550 con 0x7f1e4c003a00 2023-08-27T11:29:40.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.929+0000 7f1e5effd700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e4006a4d0 0x7f1e4006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:40.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.930+0000 7f1e5effd700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e4006a4d0 0x7f1e4006c990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009b80 tx=0x7f1e50009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:40.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:40.933+0000 7f1e57fff700 1 -- 172.21.15.112:0/2532812746 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1e48055810 con 0x7f1e4c003a00 2023-08-27T11:29:41.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.119+0000 7f1e661b8700 1 -- 172.21.15.112:0/2532812746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1e601f6970 con 0x7f1e4c003a00 2023-08-27T11:29:41.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.120+0000 7f1e57fff700 1 -- 172.21.15.112:0/2532812746 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v167) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1e48058e30 con 0x7f1e4c003a00 2023-08-27T11:29:41.122 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:29:41.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.122+0000 7f1e661b8700 1 -- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e4006a4d0 msgr2=0x7f1e4006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.122+0000 7f1e661b8700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e4006a4d0 0x7f1e4006c990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009b80 tx=0x7f1e50009210 comp rx=0 tx=0).stop 2023-08-27T11:29:41.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.122+0000 7f1e661b8700 1 -- 172.21.15.112:0/2532812746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c003a00 msgr2=0x7f1e601f2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.124 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.122+0000 7f1e661b8700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c003a00 0x7f1e601f2ae0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c004740 tx=0x7f1e480047c0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.122+0000 7f1e661b8700 1 -- 172.21.15.112:0/2532812746 shutdown_connections 2023-08-27T11:29:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.122+0000 7f1e661b8700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f1e4006a4d0 0x7f1e4006c990 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.122+0000 7f1e661b8700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e601f62c0 0x7f1e6015a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.122+0000 7f1e661b8700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c003a00 0x7f1e601f2ae0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.122+0000 7f1e661b8700 1 --2- 172.21.15.112:0/2532812746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e600aeac0 0x7f1e60133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.122+0000 7f1e661b8700 1 -- 172.21.15.112:0/2532812746 >> 172.21.15.112:0/2532812746 conn(0x7f1e600b1e90 msgr2=0x7f1e600b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.122+0000 7f1e661b8700 1 -- 172.21.15.112:0/2532812746 shutdown_connections 2023-08-27T11:29:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.122+0000 7f1e661b8700 1 -- 172.21.15.112:0/2532812746 wait complete. 2023-08-27T11:29:41.135 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:29:41.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.257+0000 7f086fc1f700 1 Processor -- start 2023-08-27T11:29:41.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.257+0000 7f086fc1f700 1 -- start start 2023-08-27T11:29:41.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086fc1f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f086815d3f0 0x7f08680ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086fc1f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08680b1350 0x7f086806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086fc1f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08680b0c30 con 0x7f086815d7d0 2023-08-27T11:29:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086fc1f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08680b3a10 con 0x7f086815d3f0 2023-08-27T11:29:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086fc1f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0868161d60 con 0x7f08680b1350 2023-08-27T11:29:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086e1bc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f086815d7d0 0x7f08680b44b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32900/0 (socket says 172.21.15.112:32900) 2023-08-27T11:29:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086e1bc700 1 -- 172.21.15.112:0/949290549 learned_addr learned my addr 172.21.15.112:0/949290549 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086d9bb700 1 --2- 172.21.15.112:0/949290549 >> v2:172.21.15.138:3300/0 conn(0x7f086815d3f0 0x7f08680ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086d1ba700 1 --2- 172.21.15.112:0/949290549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08680b1350 0x7f086806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086c9b9700 1 -- 172.21.15.112:0/949290549 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530524968 0 0) 0x7f08680b0c30 con 0x7f086815d7d0 2023-08-27T11:29:41.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086c9b9700 1 -- 172.21.15.112:0/949290549 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08540035b0 con 0x7f086815d7d0 2023-08-27T11:29:41.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086d9bb700 1 -- 172.21.15.112:0/949290549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08680b1350 msgr2=0x7f086806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:41.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086d9bb700 1 --2- 172.21.15.112:0/949290549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08680b1350 0x7f086806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086d9bb700 1 -- 172.21.15.112:0/949290549 >> v1:172.21.15.112:6789/0 conn(0x7f086815d7d0 legacy=0x7f08680b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086d9bb700 1 -- 172.21.15.112:0/949290549 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08680b3cb0 con 0x7f086815d3f0 2023-08-27T11:29:41.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.258+0000 7f086d1ba700 1 --2- 172.21.15.112:0/949290549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08680b1350 0x7f086806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:41.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.259+0000 7f086d9bb700 1 --2- 172.21.15.112:0/949290549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086815d3f0 0x7f08680ad5f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0864009840 tx=0x7f0864009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6a6241c308482d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:41.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.259+0000 7f086c9b9700 1 -- 172.21.15.112:0/949290549 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864011830 con 0x7f086815d3f0 2023-08-27T11:29:41.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.259+0000 7f086c9b9700 1 -- 172.21.15.112:0/949290549 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0864011990 con 0x7f086815d3f0 2023-08-27T11:29:41.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.259+0000 7f086c9b9700 1 -- 172.21.15.112:0/949290549 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864011d00 con 0x7f086815d3f0 2023-08-27T11:29:41.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.259+0000 7f086fc1f700 1 -- 172.21.15.112:0/949290549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086815d3f0 msgr2=0x7f08680ad5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.259+0000 7f086fc1f700 1 --2- 172.21.15.112:0/949290549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086815d3f0 0x7f08680ad5f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0864009840 tx=0x7f0864009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:41.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.259+0000 7f086fc1f700 1 -- 172.21.15.112:0/949290549 shutdown_connections 2023-08-27T11:29:41.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.259+0000 7f086fc1f700 1 --2- 172.21.15.112:0/949290549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086815d3f0 0x7f08680ad5f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.259+0000 7f086fc1f700 1 --2- 172.21.15.112:0/949290549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08680b1350 0x7f086806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.259+0000 7f086fc1f700 1 -- 172.21.15.112:0/949290549 >> 172.21.15.112:0/949290549 conn(0x7f08680b1e90 msgr2=0x7f08680b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:41.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.259+0000 7f086fc1f700 1 -- 172.21.15.112:0/949290549 shutdown_connections 2023-08-27T11:29:41.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.259+0000 7f086fc1f700 1 -- 172.21.15.112:0/949290549 wait complete. 2023-08-27T11:29:41.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.261+0000 7f086fc1f700 1 Processor -- start 2023-08-27T11:29:41.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.261+0000 7f086fc1f700 1 -- start start 2023-08-27T11:29:41.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.261+0000 7f086fc1f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08680b1350 0x7f08681581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.261+0000 7f086fc1f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f086815d3f0 0x7f0868154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.261+0000 7f086fc1f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086815d7d0 0x7f086815a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.261+0000 7f086fc1f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086815f6c0 con 0x7f08680b1350 2023-08-27T11:29:41.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.261+0000 7f086fc1f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680af170 con 0x7f086815d7d0 2023-08-27T11:29:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.261+0000 7f086fc1f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680af430 con 0x7f086815d3f0 2023-08-27T11:29:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.261+0000 7f086d9bb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08680b1350 0x7f08681581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.261+0000 7f086d9bb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08680b1350 0x7f08681581a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38520/0 (socket says 172.21.15.112:38520) 2023-08-27T11:29:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.262+0000 7f086d9bb700 1 -- 172.21.15.112:0/1717482576 learned_addr learned my addr 172.21.15.112:0/1717482576 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.262+0000 7f086d9bb700 1 -- 172.21.15.112:0/1717482576 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f086815d3f0 msgr2=0x7f0868154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.262+0000 7f086d9bb700 1 --2- 172.21.15.112:0/1717482576 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f086815d3f0 0x7f0868154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.262+0000 7f086d9bb700 1 -- 172.21.15.112:0/1717482576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086815d7d0 msgr2=0x7f086815a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.262+0000 7f086d9bb700 1 --2- 172.21.15.112:0/1717482576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086815d7d0 0x7f086815a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.262+0000 7f086d9bb700 1 -- 172.21.15.112:0/1717482576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08580094a0 con 0x7f08680b1350 2023-08-27T11:29:41.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.262+0000 7f086d9bb700 1 --2- 172.21.15.112:0/1717482576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08680b1350 0x7f08681581a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f0864009fd0 tx=0x7f0864004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:41.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.262+0000 7f085e7fc700 1 -- 172.21.15.112:0/1717482576 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08640036a0 con 0x7f08680b1350 2023-08-27T11:29:41.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.262+0000 7f085e7fc700 1 -- 172.21.15.112:0/1717482576 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086401e070 con 0x7f08680b1350 2023-08-27T11:29:41.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.262+0000 7f086fc1f700 1 -- 172.21.15.112:0/1717482576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08640094a0 con 0x7f08680b1350 2023-08-27T11:29:41.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.262+0000 7f085e7fc700 1 -- 172.21.15.112:0/1717482576 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864003a60 con 0x7f08680b1350 2023-08-27T11:29:41.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.262+0000 7f086fc1f700 1 -- 172.21.15.112:0/1717482576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0868149e40 con 0x7f08680b1350 2023-08-27T11:29:41.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.263+0000 7f086fc1f700 1 -- 172.21.15.112:0/1717482576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086815fb80 con 0x7f08680b1350 2023-08-27T11:29:41.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.263+0000 7f085e7fc700 1 -- 172.21.15.112:0/1717482576 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f0864003bc0 con 0x7f08680b1350 2023-08-27T11:29:41.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.264+0000 7f085e7fc700 1 --2- 172.21.15.112:0/1717482576 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f084c06a4d0 0x7f084c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.264+0000 7f085e7fc700 1 -- 172.21.15.112:0/1717482576 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0864089a50 con 0x7f08680b1350 2023-08-27T11:29:41.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.264+0000 7f086d1ba700 1 --2- 172.21.15.112:0/1717482576 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f084c06a4d0 0x7f084c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.265+0000 7f086d1ba700 1 --2- 172.21.15.112:0/1717482576 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f084c06a4d0 0x7f084c06c990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0858006f10 tx=0x7f0858006e60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:41.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.268+0000 7f085e7fc700 1 -- 172.21.15.112:0/1717482576 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f086815fb80 con 0x7f08680b1350 2023-08-27T11:29:41.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.458+0000 7f086fc1f700 1 -- 172.21.15.112:0/1717482576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f086815fb80 con 0x7f08680b1350 2023-08-27T11:29:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.458+0000 7f085e7fc700 1 -- 172.21.15.112:0/1717482576 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08640200a0 con 0x7f08680b1350 2023-08-27T11:29:41.460 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:29:41.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.460+0000 7f086fc1f700 1 -- 172.21.15.112:0/1717482576 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f084c06a4d0 msgr2=0x7f084c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.460+0000 7f086fc1f700 1 --2- 172.21.15.112:0/1717482576 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f084c06a4d0 0x7f084c06c990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0858006f10 tx=0x7f0858006e60 comp rx=0 tx=0).stop 2023-08-27T11:29:41.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.460+0000 7f086fc1f700 1 -- 172.21.15.112:0/1717482576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08680b1350 msgr2=0x7f08681581a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.460+0000 7f086fc1f700 1 --2- 172.21.15.112:0/1717482576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08680b1350 0x7f08681581a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f0864009fd0 tx=0x7f0864004750 comp rx=0 tx=0).stop 2023-08-27T11:29:41.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.460+0000 7f086fc1f700 1 -- 172.21.15.112:0/1717482576 shutdown_connections 2023-08-27T11:29:41.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.460+0000 7f086fc1f700 1 --2- 172.21.15.112:0/1717482576 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f084c06a4d0 0x7f084c06c990 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.460+0000 7f086fc1f700 1 --2- 172.21.15.112:0/1717482576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086815d7d0 0x7f086815a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.460+0000 7f086fc1f700 1 --2- 172.21.15.112:0/1717482576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08680b1350 0x7f08681581a0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.460+0000 7f086fc1f700 1 --2- 172.21.15.112:0/1717482576 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f086815d3f0 0x7f0868154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.460+0000 7f086fc1f700 1 -- 172.21.15.112:0/1717482576 >> 172.21.15.112:0/1717482576 conn(0x7f08680b1e90 msgr2=0x7f086806ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:41.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.460+0000 7f086fc1f700 1 -- 172.21.15.112:0/1717482576 shutdown_connections 2023-08-27T11:29:41.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.460+0000 7f086fc1f700 1 -- 172.21.15.112:0/1717482576 wait complete. 2023-08-27T11:29:41.474 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:29:41.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.595+0000 7f9a20ea3700 1 Processor -- start 2023-08-27T11:29:41.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.595+0000 7f9a20ea3700 1 -- start start 2023-08-27T11:29:41.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.596+0000 7f9a20ea3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c146c20 0x7f9a1c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.596+0000 7f9a20ea3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9a1c06f790 0x7f9a1c06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.596+0000 7f9a20ea3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1c0b5070 con 0x7f9a1c0b3440 2023-08-27T11:29:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.596+0000 7f9a20ea3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1c070160 con 0x7f9a1c06f790 2023-08-27T11:29:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.596+0000 7f9a20ea3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c147000 con 0x7f9a1c146c20 2023-08-27T11:29:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.596+0000 7f9a1ad9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9a1c06f790 0x7f9a1c06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.596+0000 7f9a19d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9a1c0b3440 0x7f9a1c15e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32912/0 (socket says 172.21.15.112:32912) 2023-08-27T11:29:41.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.596+0000 7f9a19d9b700 1 -- 172.21.15.112:0/2911606522 learned_addr learned my addr 172.21.15.112:0/2911606522 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:41.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.597+0000 7f9a1ad9d700 1 -- 172.21.15.112:0/2911606522 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c146c20 msgr2=0x7f9a1c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:41.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.597+0000 7f9a1ad9d700 1 --2- 172.21.15.112:0/2911606522 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c146c20 0x7f9a1c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.597+0000 7f9a1ad9d700 1 -- 172.21.15.112:0/2911606522 >> v1:172.21.15.112:6789/0 conn(0x7f9a1c0b3440 legacy=0x7f9a1c15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.597+0000 7f9a1ad9d700 1 -- 172.21.15.112:0/2911606522 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c1614a0 con 0x7f9a1c06f790 2023-08-27T11:29:41.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.597+0000 7f9a1959a700 1 -- 172.21.15.112:0/2911606522 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239830023 0 0) 0x7f9a04001110 con 0x7f9a1c0b3440 2023-08-27T11:29:41.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.597+0000 7f9a1ad9d700 1 --2- 172.21.15.112:0/2911606522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a1c06f790 0x7f9a1c06fc20 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009d90 tx=0x7f9a1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4da364b30dfbf26 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:41.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.597+0000 7f9a1959a700 1 -- 172.21.15.112:0/2911606522 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1400e070 con 0x7f9a1c06f790 2023-08-27T11:29:41.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.597+0000 7f9a1959a700 1 -- 172.21.15.112:0/2911606522 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1400bbd0 con 0x7f9a1c06f790 2023-08-27T11:29:41.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.597+0000 7f9a1959a700 1 -- 172.21.15.112:0/2911606522 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14012430 con 0x7f9a1c06f790 2023-08-27T11:29:41.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.597+0000 7f9a20ea3700 1 -- 172.21.15.112:0/2911606522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a1c06f790 msgr2=0x7f9a1c06fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.597+0000 7f9a20ea3700 1 --2- 172.21.15.112:0/2911606522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a1c06f790 0x7f9a1c06fc20 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009d90 tx=0x7f9a1400b330 comp rx=0 tx=0).stop 2023-08-27T11:29:41.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.598+0000 7f9a20ea3700 1 -- 172.21.15.112:0/2911606522 shutdown_connections 2023-08-27T11:29:41.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.598+0000 7f9a20ea3700 1 --2- 172.21.15.112:0/2911606522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a1c06f790 0x7f9a1c06fc20 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.598+0000 7f9a20ea3700 1 --2- 172.21.15.112:0/2911606522 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c146c20 0x7f9a1c160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.598+0000 7f9a20ea3700 1 -- 172.21.15.112:0/2911606522 >> 172.21.15.112:0/2911606522 conn(0x7f9a1c0b1e90 msgr2=0x7f9a1c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:41.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.598+0000 7f9a20ea3700 1 -- 172.21.15.112:0/2911606522 shutdown_connections 2023-08-27T11:29:41.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.598+0000 7f9a20ea3700 1 -- 172.21.15.112:0/2911606522 wait complete. 2023-08-27T11:29:41.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.599+0000 7f9a20ea3700 1 Processor -- start 2023-08-27T11:29:41.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.599+0000 7f9a20ea3700 1 -- start start 2023-08-27T11:29:41.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a20ea3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c06f790 0x7f9a1c142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a20ea3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a1c0b3440 0x7f9a1c0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a20ea3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c146c20 0x7f9a1c0c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a20ea3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c15c610 con 0x7f9a1c0b3440 2023-08-27T11:29:41.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a20ea3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c15c180 con 0x7f9a1c06f790 2023-08-27T11:29:41.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a20ea3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c143100 con 0x7f9a1c146c20 2023-08-27T11:29:41.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a19d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a1c0b3440 0x7f9a1c0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a19d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a1c0b3440 0x7f9a1c0c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38536/0 (socket says 172.21.15.112:38536) 2023-08-27T11:29:41.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a19d9b700 1 -- 172.21.15.112:0/1823882688 learned_addr learned my addr 172.21.15.112:0/1823882688 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:41.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a1a59c700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c06f790 0x7f9a1c142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a1ad9d700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c146c20 0x7f9a1c0c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a19d9b700 1 -- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c146c20 msgr2=0x7f9a1c0c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a19d9b700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c146c20 0x7f9a1c0c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a19d9b700 1 -- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c06f790 msgr2=0x7f9a1c142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a19d9b700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c06f790 0x7f9a1c142bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a19d9b700 1 -- 172.21.15.112:0/1823882688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a14009a70 con 0x7f9a1c0b3440 2023-08-27T11:29:41.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a19d9b700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a1c0b3440 0x7f9a1c0c6840 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c730 tx=0x7f9a0400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:41.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a12ffd700 1 -- 172.21.15.112:0/1823882688 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04004c80 con 0x7f9a1c0b3440 2023-08-27T11:29:41.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a12ffd700 1 -- 172.21.15.112:0/1823882688 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a04004e00 con 0x7f9a1c0b3440 2023-08-27T11:29:41.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a12ffd700 1 -- 172.21.15.112:0/1823882688 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04007310 con 0x7f9a1c0b3440 2023-08-27T11:29:41.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.600+0000 7f9a20ea3700 1 -- 172.21.15.112:0/1823882688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c15b570 con 0x7f9a1c0b3440 2023-08-27T11:29:41.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.601+0000 7f9a20ea3700 1 -- 172.21.15.112:0/1823882688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c15bb30 con 0x7f9a1c0b3440 2023-08-27T11:29:41.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.601+0000 7f9a20ea3700 1 -- 172.21.15.112:0/1823882688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c067e90 con 0x7f9a1c0b3440 2023-08-27T11:29:41.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.602+0000 7f9a12ffd700 1 -- 172.21.15.112:0/1823882688 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f9a04007470 con 0x7f9a1c0b3440 2023-08-27T11:29:41.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.602+0000 7f9a12ffd700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f99fc06a4d0 0x7f99fc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.602+0000 7f9a12ffd700 1 -- 172.21.15.112:0/1823882688 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9a04088680 con 0x7f9a1c0b3440 2023-08-27T11:29:41.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.603+0000 7f9a1a59c700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f99fc06a4d0 0x7f99fc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.603+0000 7f9a1a59c700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f99fc06a4d0 0x7f99fc06c990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0c6700 tx=0x7f9a0c009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:41.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.607+0000 7f9a12ffd700 1 -- 172.21.15.112:0/1823882688 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9a040538a0 con 0x7f9a1c0b3440 2023-08-27T11:29:41.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.794+0000 7f9a20ea3700 1 -- 172.21.15.112:0/1823882688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a1c0b5070 con 0x7f9a1c0b3440 2023-08-27T11:29:41.796 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.795+0000 7f9a12ffd700 1 -- 172.21.15.112:0/1823882688 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v167) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9a1c0b5070 con 0x7f9a1c0b3440 2023-08-27T11:29:41.796 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:29:41.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.797+0000 7f9a20ea3700 1 -- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f99fc06a4d0 msgr2=0x7f99fc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.798 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.797+0000 7f9a20ea3700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f99fc06a4d0 0x7f99fc06c990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0c6700 tx=0x7f9a0c009110 comp rx=0 tx=0).stop 2023-08-27T11:29:41.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.797+0000 7f9a20ea3700 1 -- 172.21.15.112:0/1823882688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a1c0b3440 msgr2=0x7f9a1c0c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.797+0000 7f9a20ea3700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a1c0b3440 0x7f9a1c0c6840 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c730 tx=0x7f9a0400ca40 comp rx=0 tx=0).stop 2023-08-27T11:29:41.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.797+0000 7f9a20ea3700 1 -- 172.21.15.112:0/1823882688 shutdown_connections 2023-08-27T11:29:41.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.797+0000 7f9a20ea3700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f99fc06a4d0 0x7f99fc06c990 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.799 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.797+0000 7f9a20ea3700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c146c20 0x7f9a1c0c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.797+0000 7f9a20ea3700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c06f790 0x7f9a1c142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.797+0000 7f9a20ea3700 1 --2- 172.21.15.112:0/1823882688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a1c0b3440 0x7f9a1c0c6840 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.797+0000 7f9a20ea3700 1 -- 172.21.15.112:0/1823882688 >> 172.21.15.112:0/1823882688 conn(0x7f9a1c0b1e90 msgr2=0x7f9a1c071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.797+0000 7f9a20ea3700 1 -- 172.21.15.112:0/1823882688 shutdown_connections 2023-08-27T11:29:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.797+0000 7f9a20ea3700 1 -- 172.21.15.112:0/1823882688 wait complete. 2023-08-27T11:29:41.810 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:29:41.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.931+0000 7fe6876c4700 1 Processor -- start 2023-08-27T11:29:41.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.931+0000 7fe6876c4700 1 -- start start 2023-08-27T11:29:41.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.932+0000 7fe6876c4700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe68006e4c0 0x7fe6800acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.932+0000 7fe6876c4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe680160fa0 con 0x7fe680146b60 2023-08-27T11:29:41.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.932+0000 7fe6876c4700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe680146f40 con 0x7fe6800ad000 2023-08-27T11:29:41.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.932+0000 7fe6876c4700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe680161b70 con 0x7fe68006e4c0 2023-08-27T11:29:41.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.932+0000 7fe685460700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe680146b60 0x7fe68006ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32924/0 (socket says 172.21.15.112:32924) 2023-08-27T11:29:41.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.932+0000 7fe685460700 1 -- 172.21.15.112:0/3421213094 learned_addr learned my addr 172.21.15.112:0/3421213094 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:41.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175733449 0 0) 0x7fe680160fa0 con 0x7fe680146b60 2023-08-27T11:29:41.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe66c0035b0 con 0x7fe680146b60 2023-08-27T11:29:41.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe684c5f700 1 --2- 172.21.15.112:0/3421213094 >> v2:172.21.15.138:3300/0 conn(0x7fe68006e4c0 0x7fe6800acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1903482639 0 0) 0x7fe66c0035b0 con 0x7fe680146b60 2023-08-27T11:29:41.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 >> v2:172.21.15.138:3300/0 conn(0x7fe68006e4c0 msgr2=0x7fe6800acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:41.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 --2- 172.21.15.112:0/3421213094 >> v2:172.21.15.138:3300/0 conn(0x7fe68006e4c0 0x7fe6800acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 >> v1:172.21.15.138:6789/0 conn(0x7fe6800ad000 legacy=0x7fe68015d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6801611d0 con 0x7fe680146b60 2023-08-27T11:29:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fe670001670 con 0x7fe680146b60 2023-08-27T11:29:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe684c5f700 1 --2- 172.21.15.112:0/3421213094 >> v2:172.21.15.138:3300/0 conn(0x7fe68006e4c0 0x7fe6800acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 >> v1:172.21.15.112:6789/0 conn(0x7fe680146b60 legacy=0x7fe68006ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 --2- 172.21.15.112:0/3421213094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c003a00 0x7fe66c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 --2- 172.21.15.112:0/3421213094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe66c004850 0x7fe66c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 --2- 172.21.15.112:0/3421213094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe66c005210 0x7fe66c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe680161b70 con 0x7fe66c005210 2023-08-27T11:29:41.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c00c160 con 0x7fe66c003a00 2023-08-27T11:29:41.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c00c320 con 0x7fe66c004850 2023-08-27T11:29:41.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe685c61700 1 --2- 172.21.15.112:0/3421213094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe66c005210 0x7fe66c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe685460700 1 --2- 172.21.15.112:0/3421213094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c003a00 0x7fe66c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe684c5f700 1 --2- 172.21.15.112:0/3421213094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe66c004850 0x7fe66c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe685c61700 1 -- 172.21.15.112:0/3421213094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe66c004850 msgr2=0x7fe66c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:41.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe685c61700 1 --2- 172.21.15.112:0/3421213094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe66c004850 0x7fe66c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe685c61700 1 -- 172.21.15.112:0/3421213094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c003a00 msgr2=0x7fe66c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:41.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe685c61700 1 --2- 172.21.15.112:0/3421213094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c003a00 0x7fe66c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe685c61700 1 -- 172.21.15.112:0/3421213094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c0035b0 con 0x7fe66c005210 2023-08-27T11:29:41.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe685c61700 1 -- 172.21.15.112:0/3421213094 reap_dead start 2023-08-27T11:29:41.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.933+0000 7fe685c61700 1 --2- 172.21.15.112:0/3421213094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe66c005210 0x7fe66c00ba10 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fe678002150 tx=0x7fe66c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670391921a877c1d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:41.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.934+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678001db0 con 0x7fe66c005210 2023-08-27T11:29:41.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.934+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe678003c40 con 0x7fe66c005210 2023-08-27T11:29:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.934+0000 7fe677fff700 1 -- 172.21.15.112:0/3421213094 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800c930 con 0x7fe66c005210 2023-08-27T11:29:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.934+0000 7fe6876c4700 1 -- 172.21.15.112:0/3421213094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe66c005210 msgr2=0x7fe66c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.934+0000 7fe6876c4700 1 --2- 172.21.15.112:0/3421213094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe66c005210 0x7fe66c00ba10 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fe678002150 tx=0x7fe66c005880 comp rx=0 tx=0).stop 2023-08-27T11:29:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.934+0000 7fe6876c4700 1 -- 172.21.15.112:0/3421213094 shutdown_connections 2023-08-27T11:29:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.934+0000 7fe6876c4700 1 --2- 172.21.15.112:0/3421213094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe66c005210 0x7fe66c00ba10 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.934+0000 7fe6876c4700 1 -- 172.21.15.112:0/3421213094 >> 172.21.15.112:0/3421213094 conn(0x7fe6800cc240 msgr2=0x7fe6800cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.934+0000 7fe6876c4700 1 -- 172.21.15.112:0/3421213094 shutdown_connections 2023-08-27T11:29:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.934+0000 7fe6876c4700 1 -- 172.21.15.112:0/3421213094 wait complete. 2023-08-27T11:29:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.935+0000 7fe6876c4700 1 Processor -- start 2023-08-27T11:29:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.936+0000 7fe6876c4700 1 -- start start 2023-08-27T11:29:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.936+0000 7fe6876c4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c005210 0x7fe68006f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.936+0000 7fe6876c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6800cec90 0x7fe680136a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.936+0000 7fe6876c4700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe68015f6b0 0x7fe68015fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.936+0000 7fe6876c4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe680070860 con 0x7fe6800cec90 2023-08-27T11:29:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.936+0000 7fe6876c4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6800cf110 con 0x7fe66c005210 2023-08-27T11:29:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.936+0000 7fe6876c4700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe680160070 con 0x7fe68015f6b0 2023-08-27T11:29:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.936+0000 7fe684c5f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6800cec90 0x7fe680136a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.936+0000 7fe684c5f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6800cec90 0x7fe680136a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38556/0 (socket says 172.21.15.112:38556) 2023-08-27T11:29:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.936+0000 7fe684c5f700 1 -- 172.21.15.112:0/3460158780 learned_addr learned my addr 172.21.15.112:0/3460158780 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.937+0000 7fe685c61700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe68015f6b0 0x7fe68015fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.937+0000 7fe685460700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c005210 0x7fe68006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.937+0000 7fe684c5f700 1 -- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe68015f6b0 msgr2=0x7fe68015fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.937+0000 7fe684c5f700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe68015f6b0 0x7fe68015fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.937+0000 7fe684c5f700 1 -- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c005210 msgr2=0x7fe68006f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.937+0000 7fe684c5f700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c005210 0x7fe68006f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.937+0000 7fe684c5f700 1 -- 172.21.15.112:0/3460158780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68006fbe0 con 0x7fe6800cec90 2023-08-27T11:29:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.937+0000 7fe684c5f700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6800cec90 0x7fe680136a60 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0096b0 tx=0x7fe67c009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.937+0000 7fe675ffb700 1 -- 172.21.15.112:0/3460158780 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c015880 con 0x7fe6800cec90 2023-08-27T11:29:41.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.937+0000 7fe675ffb700 1 -- 172.21.15.112:0/3460158780 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c0159e0 con 0x7fe6800cec90 2023-08-27T11:29:41.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.937+0000 7fe6876c4700 1 -- 172.21.15.112:0/3460158780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68006fec0 con 0x7fe6800cec90 2023-08-27T11:29:41.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.937+0000 7fe675ffb700 1 -- 172.21.15.112:0/3460158780 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c015d70 con 0x7fe6800cec90 2023-08-27T11:29:41.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.937+0000 7fe6876c4700 1 -- 172.21.15.112:0/3460158780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe680070480 con 0x7fe6800cec90 2023-08-27T11:29:41.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.938+0000 7fe6876c4700 1 -- 172.21.15.112:0/3460158780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6800709a0 con 0x7fe6800cec90 2023-08-27T11:29:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.939+0000 7fe675ffb700 1 -- 172.21.15.112:0/3460158780 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe67c01e400 con 0x7fe6800cec90 2023-08-27T11:29:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.939+0000 7fe675ffb700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe66406a4d0 0x7fe66406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:41.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.939+0000 7fe675ffb700 1 -- 172.21.15.112:0/3460158780 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe67c089360 con 0x7fe6800cec90 2023-08-27T11:29:41.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.939+0000 7fe685460700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe66406a4d0 0x7fe66406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:41.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.940+0000 7fe685460700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe66406a4d0 0x7fe66406c990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe670007cd0 tx=0x7fe670009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:41.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:41.943+0000 7fe675ffb700 1 -- 172.21.15.112:0/3460158780 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe6800709a0 con 0x7fe6800cec90 2023-08-27T11:29:42.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.130+0000 7fe6876c4700 1 -- 172.21.15.112:0/3460158780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe680070640 con 0x7fe6800cec90 2023-08-27T11:29:42.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.130+0000 7fe675ffb700 1 -- 172.21.15.112:0/3460158780 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v167) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe67c057f90 con 0x7fe6800cec90 2023-08-27T11:29:42.132 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:29:42.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.132+0000 7fe6876c4700 1 -- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe66406a4d0 msgr2=0x7fe66406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:42.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.132+0000 7fe6876c4700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe66406a4d0 0x7fe66406c990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe670007cd0 tx=0x7fe670009040 comp rx=0 tx=0).stop 2023-08-27T11:29:42.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.132+0000 7fe6876c4700 1 -- 172.21.15.112:0/3460158780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6800cec90 msgr2=0x7fe680136a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:42.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.132+0000 7fe6876c4700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6800cec90 0x7fe680136a60 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0096b0 tx=0x7fe67c009a70 comp rx=0 tx=0).stop 2023-08-27T11:29:42.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.132+0000 7fe6876c4700 1 -- 172.21.15.112:0/3460158780 shutdown_connections 2023-08-27T11:29:42.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.132+0000 7fe6876c4700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe66406a4d0 0x7fe66406c990 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.132+0000 7fe6876c4700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe68015f6b0 0x7fe68015fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.132+0000 7fe6876c4700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c005210 0x7fe68006f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.132+0000 7fe6876c4700 1 --2- 172.21.15.112:0/3460158780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6800cec90 0x7fe680136a60 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.132+0000 7fe6876c4700 1 -- 172.21.15.112:0/3460158780 >> 172.21.15.112:0/3460158780 conn(0x7fe6800cc240 msgr2=0x7fe680142330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:42.135 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.132+0000 7fe6876c4700 1 -- 172.21.15.112:0/3460158780 shutdown_connections 2023-08-27T11:29:42.136 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.132+0000 7fe6876c4700 1 -- 172.21.15.112:0/3460158780 wait complete. 2023-08-27T11:29:42.146 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:29:42.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.266+0000 7ff1dd62e700 1 Processor -- start 2023-08-27T11:29:42.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.266+0000 7ff1dd62e700 1 -- start start 2023-08-27T11:29:42.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.266+0000 7ff1dd62e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1d806b230 0x7ff1d80ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.266+0000 7ff1dd62e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d815eb60 con 0x7ff1d8151060 2023-08-27T11:29:42.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.266+0000 7ff1dd62e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d806b6c0 con 0x7ff1d806adb0 2023-08-27T11:29:42.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.266+0000 7ff1dd62e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80b02f0 con 0x7ff1d806b230 2023-08-27T11:29:42.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d6ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff1d8151060 0x7ff1d815e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32944/0 (socket says 172.21.15.112:32944) 2023-08-27T11:29:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d6ffd700 1 -- 172.21.15.112:0/2358524363 learned_addr learned my addr 172.21.15.112:0/2358524363 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d77fe700 1 --2- 172.21.15.112:0/2358524363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1d806b230 0x7ff1d80ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828815296 0 0) 0x7ff1d815eb60 con 0x7ff1d8151060 2023-08-27T11:29:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1c00057c0 con 0x7ff1d8151060 2023-08-27T11:29:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315050031 0 0) 0x7ff1d806b6c0 con 0x7ff1d806adb0 2023-08-27T11:29:42.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1d815eb60 con 0x7ff1d806adb0 2023-08-27T11:29:42.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2753392376 0 0) 0x7ff1c00057c0 con 0x7ff1d8151060 2023-08-27T11:29:42.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1d806b230 msgr2=0x7ff1d80ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:42.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 --2- 172.21.15.112:0/2358524363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1d806b230 0x7ff1d80ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.274 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 >> v1:172.21.15.138:6789/0 conn(0x7ff1d806adb0 legacy=0x7ff1d80cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:42.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d80b05d0 con 0x7ff1d8151060 2023-08-27T11:29:42.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7ff1c8001670 con 0x7ff1d8151060 2023-08-27T11:29:42.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 >> v1:172.21.15.112:6789/0 conn(0x7ff1d8151060 legacy=0x7ff1d815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:42.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 --2- 172.21.15.112:0/2358524363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1d806b230 0x7ff1c0005dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.275 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 --2- 172.21.15.112:0/2358524363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0006330 0x7ff1c0006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 --2- 172.21.15.112:0/2358524363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c0006fa0 0x7ff1c0007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80b02f0 con 0x7ff1c0006330 2023-08-27T11:29:42.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c000dd00 con 0x7ff1c0006fa0 2023-08-27T11:29:42.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c000dec0 con 0x7ff1d806b230 2023-08-27T11:29:42.276 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d67fc700 1 --2- 172.21.15.112:0/2358524363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0006330 0x7ff1c0006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d6ffd700 1 --2- 172.21.15.112:0/2358524363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1d806b230 0x7ff1c0005dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d77fe700 1 --2- 172.21.15.112:0/2358524363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c0006fa0 0x7ff1c0007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d67fc700 1 -- 172.21.15.112:0/2358524363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1d806b230 msgr2=0x7ff1c0005dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:42.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d67fc700 1 --2- 172.21.15.112:0/2358524363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1d806b230 0x7ff1c0005dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d67fc700 1 -- 172.21.15.112:0/2358524363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c0006fa0 msgr2=0x7ff1c0007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:42.277 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.267+0000 7ff1d67fc700 1 --2- 172.21.15.112:0/2358524363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c0006fa0 0x7ff1c0007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.268+0000 7ff1d67fc700 1 -- 172.21.15.112:0/2358524363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c00057c0 con 0x7ff1c0006330 2023-08-27T11:29:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.268+0000 7ff1d77fe700 1 --2- 172.21.15.112:0/2358524363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c0006fa0 0x7ff1c0007420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.268+0000 7ff1d67fc700 1 --2- 172.21.15.112:0/2358524363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0006330 0x7ff1c0006a60 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc001940 tx=0x7ff1c0007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffff16b7887971d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.268+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc001430 con 0x7ff1c0006330 2023-08-27T11:29:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.268+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc001d70 con 0x7ff1c0006330 2023-08-27T11:29:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.268+0000 7ff1d5ffb700 1 -- 172.21.15.112:0/2358524363 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc005510 con 0x7ff1c0006330 2023-08-27T11:29:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.268+0000 7ff1dd62e700 1 -- 172.21.15.112:0/2358524363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0006330 msgr2=0x7ff1c0006a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.268+0000 7ff1dd62e700 1 --2- 172.21.15.112:0/2358524363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0006330 0x7ff1c0006a60 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc001940 tx=0x7ff1c0007a90 comp rx=0 tx=0).stop 2023-08-27T11:29:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.268+0000 7ff1d77fe700 1 -- 172.21.15.112:0/2358524363 reap_dead start 2023-08-27T11:29:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.268+0000 7ff1dd62e700 1 -- 172.21.15.112:0/2358524363 shutdown_connections 2023-08-27T11:29:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.268+0000 7ff1dd62e700 1 -- 172.21.15.112:0/2358524363 >> 172.21.15.112:0/2358524363 conn(0x7ff1d80d0000 msgr2=0x7ff1d80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.268+0000 7ff1dd62e700 1 -- 172.21.15.112:0/2358524363 shutdown_connections 2023-08-27T11:29:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.268+0000 7ff1dd62e700 1 -- 172.21.15.112:0/2358524363 wait complete. 2023-08-27T11:29:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.270+0000 7ff1dd62e700 1 Processor -- start 2023-08-27T11:29:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.270+0000 7ff1dd62e700 1 -- start start 2023-08-27T11:29:42.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.270+0000 7ff1dd62e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0006330 0x7ff1d814cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.283 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.270+0000 7ff1dd62e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1d80c70d0 0x7ff1d814dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.270+0000 7ff1dd62e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1d80c7550 0x7ff1d814e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.270+0000 7ff1dd62e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80c39f0 con 0x7ff1c0006330 2023-08-27T11:29:42.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.270+0000 7ff1dd62e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80c3c40 con 0x7ff1d80c7550 2023-08-27T11:29:42.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.270+0000 7ff1dd62e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80c3f20 con 0x7ff1d80c70d0 2023-08-27T11:29:42.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1d6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0006330 0x7ff1d814cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1d6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0006330 0x7ff1d814cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38574/0 (socket says 172.21.15.112:38574) 2023-08-27T11:29:42.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1d6ffd700 1 -- 172.21.15.112:0/1355010634 learned_addr learned my addr 172.21.15.112:0/1355010634 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:42.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1d67fc700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1d80c70d0 0x7ff1d814dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1d77fe700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1d80c7550 0x7ff1d814e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1d6ffd700 1 -- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1d80c70d0 msgr2=0x7ff1d814dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:42.285 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1d6ffd700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1d80c70d0 0x7ff1d814dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1d6ffd700 1 -- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1d80c7550 msgr2=0x7ff1d814e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:42.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1d6ffd700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1d80c7550 0x7ff1d814e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1d6ffd700 1 -- 172.21.15.112:0/1355010634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d80c4200 con 0x7ff1c0006330 2023-08-27T11:29:42.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1d6ffd700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0006330 0x7ff1d814cb80 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8007340 tx=0x7ff1c8007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:42.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1bf7fe700 1 -- 172.21.15.112:0/1355010634 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800f870 con 0x7ff1c0006330 2023-08-27T11:29:42.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1bf7fe700 1 -- 172.21.15.112:0/1355010634 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c800fa10 con 0x7ff1c0006330 2023-08-27T11:29:42.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1dd62e700 1 -- 172.21.15.112:0/1355010634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d80c44e0 con 0x7ff1c0006330 2023-08-27T11:29:42.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1bf7fe700 1 -- 172.21.15.112:0/1355010634 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800fdc0 con 0x7ff1c0006330 2023-08-27T11:29:42.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.271+0000 7ff1dd62e700 1 -- 172.21.15.112:0/1355010634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d814d0c0 con 0x7ff1c0006330 2023-08-27T11:29:42.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.272+0000 7ff1dd62e700 1 -- 172.21.15.112:0/1355010634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d8067a20 con 0x7ff1c0006330 2023-08-27T11:29:42.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.272+0000 7ff1bf7fe700 1 -- 172.21.15.112:0/1355010634 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7ff1c801d460 con 0x7ff1c0006330 2023-08-27T11:29:42.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.273+0000 7ff1bf7fe700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff1c406a4d0 0x7ff1c406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.273+0000 7ff1bf7fe700 1 -- 172.21.15.112:0/1355010634 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff1c8088c60 con 0x7ff1c0006330 2023-08-27T11:29:42.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.273+0000 7ff1d67fc700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff1c406a4d0 0x7ff1c406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.273+0000 7ff1d67fc700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff1c406a4d0 0x7ff1c406c990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff1d815c500 tx=0x7ff1cc00a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:42.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.277+0000 7ff1bf7fe700 1 -- 172.21.15.112:0/1355010634 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff1c8053f40 con 0x7ff1c0006330 2023-08-27T11:29:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.484+0000 7ff1dd62e700 1 -- 172.21.15.112:0/1355010634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff1d80b0880 con 0x7ff1c0006330 2023-08-27T11:29:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.484+0000 7ff1bf7fe700 1 -- 172.21.15.112:0/1355010634 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v167) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff1c8057560 con 0x7ff1c0006330 2023-08-27T11:29:42.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.486+0000 7ff1dd62e700 1 -- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff1c406a4d0 msgr2=0x7ff1c406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:42.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.486+0000 7ff1dd62e700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff1c406a4d0 0x7ff1c406c990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff1d815c500 tx=0x7ff1cc00a000 comp rx=0 tx=0).stop 2023-08-27T11:29:42.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.486+0000 7ff1dd62e700 1 -- 172.21.15.112:0/1355010634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0006330 msgr2=0x7ff1d814cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:42.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.486+0000 7ff1dd62e700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0006330 0x7ff1d814cb80 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8007340 tx=0x7ff1c8007650 comp rx=0 tx=0).stop 2023-08-27T11:29:42.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.486+0000 7ff1dd62e700 1 -- 172.21.15.112:0/1355010634 shutdown_connections 2023-08-27T11:29:42.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.486+0000 7ff1dd62e700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff1c406a4d0 0x7ff1c406c990 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.486+0000 7ff1dd62e700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1d80c7550 0x7ff1d814e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.486+0000 7ff1dd62e700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0006330 0x7ff1d814cb80 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.486+0000 7ff1dd62e700 1 --2- 172.21.15.112:0/1355010634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1d80c70d0 0x7ff1d814dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.486+0000 7ff1dd62e700 1 -- 172.21.15.112:0/1355010634 >> 172.21.15.112:0/1355010634 conn(0x7ff1d80d0000 msgr2=0x7ff1d80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:42.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.486+0000 7ff1dd62e700 1 -- 172.21.15.112:0/1355010634 shutdown_connections 2023-08-27T11:29:42.490 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.486+0000 7ff1dd62e700 1 -- 172.21.15.112:0/1355010634 wait complete. 2023-08-27T11:29:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2023-08-27T11:29:42.500 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-27T11:29:42.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:42.501 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-08-27T11:29:42.545 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:29:42.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:42.546 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-27T11:29:42.605 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:29:42.605 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:42.622 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:29:42.623 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-08-27T11:29:42.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:42.624 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-27T11:29:42.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:29:42.679 INFO:teuthology.orchestra.run.smithi112.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-27T11:29:42.680 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:29:42.680 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:42.700 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:29:42.700 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2023-08-27T11:29:42.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:42.701 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-27T11:29:42.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:29:42.759 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-27T11:29:42.759 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create another-data-pool --pg_num_min 64 2023-08-27T11:29:42.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.882+0000 7f40f1d10700 1 Processor -- start 2023-08-27T11:29:42.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.882+0000 7f40f1d10700 1 -- start start 2023-08-27T11:29:42.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.883+0000 7f40f1d10700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40ec1439e0 0x7f40ec0b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.886 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.883+0000 7f40f1d10700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40ec0b1420 0x7f40ec15a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.883+0000 7f40f1d10700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ec0b0b10 con 0x7f40ec143dc0 2023-08-27T11:29:42.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.883+0000 7f40f1d10700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ec06d850 con 0x7f40ec1439e0 2023-08-27T11:29:42.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.883+0000 7f40f1d10700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec15b460 con 0x7f40ec0b1420 2023-08-27T11:29:42.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.883+0000 7f40eb7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40ec1439e0 0x7f40ec0b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.883+0000 7f40eb7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40ec1439e0 0x7f40ec0b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38582/0 (socket says 172.21.15.112:38582) 2023-08-27T11:29:42.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.883+0000 7f40eb7fe700 1 -- 172.21.15.112:0/4005100499 learned_addr learned my addr 172.21.15.112:0/4005100499 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:42.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.884+0000 7f40ebfff700 1 --2- 172.21.15.112:0/4005100499 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40ec0b1420 0x7f40ec15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.884+0000 7f40eb7fe700 1 -- 172.21.15.112:0/4005100499 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40ec0b1420 msgr2=0x7f40ec15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:42.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.884+0000 7f40eb7fe700 1 --2- 172.21.15.112:0/4005100499 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40ec0b1420 0x7f40ec15a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.884+0000 7f40eb7fe700 1 -- 172.21.15.112:0/4005100499 >> v1:172.21.15.138:6789/0 conn(0x7f40ec143dc0 legacy=0x7f40ec0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:42.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.884+0000 7f40eb7fe700 1 -- 172.21.15.112:0/4005100499 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec15b740 con 0x7f40ec1439e0 2023-08-27T11:29:42.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.884+0000 7f40ebfff700 1 --2- 172.21.15.112:0/4005100499 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40ec0b1420 0x7f40ec15a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:42.889 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.884+0000 7f40eb7fe700 1 --2- 172.21.15.112:0/4005100499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec1439e0 0x7f40ec0b05d0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f40d40098f0 tx=0x7f40d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26ca5720c14e358 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:42.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.884+0000 7f40ea7fc700 1 -- 172.21.15.112:0/4005100499 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d400b8a0 con 0x7f40ec1439e0 2023-08-27T11:29:42.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.884+0000 7f40ea7fc700 1 -- 172.21.15.112:0/4005100499 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d400ba00 con 0x7f40ec1439e0 2023-08-27T11:29:42.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.884+0000 7f40ea7fc700 1 -- 172.21.15.112:0/4005100499 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d400bd90 con 0x7f40ec1439e0 2023-08-27T11:29:42.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.884+0000 7f40f1d10700 1 -- 172.21.15.112:0/4005100499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec1439e0 msgr2=0x7f40ec0b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:42.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.884+0000 7f40f1d10700 1 --2- 172.21.15.112:0/4005100499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec1439e0 0x7f40ec0b05d0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f40d40098f0 tx=0x7f40d4009c00 comp rx=0 tx=0).stop 2023-08-27T11:29:42.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.885+0000 7f40f1d10700 1 -- 172.21.15.112:0/4005100499 shutdown_connections 2023-08-27T11:29:42.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.885+0000 7f40f1d10700 1 --2- 172.21.15.112:0/4005100499 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40ec0b1420 0x7f40ec15a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.885+0000 7f40f1d10700 1 --2- 172.21.15.112:0/4005100499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec1439e0 0x7f40ec0b05d0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.891 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.885+0000 7f40f1d10700 1 -- 172.21.15.112:0/4005100499 >> 172.21.15.112:0/4005100499 conn(0x7f40ec0b1be0 msgr2=0x7f40ec0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:42.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.885+0000 7f40f1d10700 1 -- 172.21.15.112:0/4005100499 shutdown_connections 2023-08-27T11:29:42.892 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.885+0000 7f40f1d10700 1 -- 172.21.15.112:0/4005100499 wait complete. 2023-08-27T11:29:42.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.886+0000 7f40f1d10700 1 Processor -- start 2023-08-27T11:29:42.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.886+0000 7f40f1d10700 1 -- start start 2023-08-27T11:29:42.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.887+0000 7f40f1d10700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec0b1420 0x7f40ec1f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.887+0000 7f40f1d10700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40ec1439e0 0x7f40ec1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.887+0000 7f40f1d10700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec0a9970 0x7f40ec0a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.887+0000 7f40f1d10700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec1f36c0 con 0x7f40ec0b1420 2023-08-27T11:29:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.887+0000 7f40f1d10700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0a88d0 con 0x7f40ec0a9970 2023-08-27T11:29:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.887+0000 7f40f1d10700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0a8bb0 con 0x7f40ec1439e0 2023-08-27T11:29:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.887+0000 7f40eb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec0b1420 0x7f40ec1f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.887+0000 7f40eb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec0b1420 0x7f40ec1f3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38588/0 (socket says 172.21.15.112:38588) 2023-08-27T11:29:42.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.887+0000 7f40eb7fe700 1 -- 172.21.15.112:0/3562084419 learned_addr learned my addr 172.21.15.112:0/3562084419 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:42.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.888+0000 7f40eb7fe700 1 -- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40ec1439e0 msgr2=0x7f40ec1591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:42.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.888+0000 7f40eaffd700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40ec1439e0 0x7f40ec1591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.897 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.888+0000 7f40ebfff700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec0a9970 0x7f40ec0a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.888+0000 7f40eb7fe700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40ec1439e0 0x7f40ec1591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.888+0000 7f40eb7fe700 1 -- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec0a9970 msgr2=0x7f40ec0a9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:42.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.888+0000 7f40eb7fe700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec0a9970 0x7f40ec0a9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:42.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.888+0000 7f40eb7fe700 1 -- 172.21.15.112:0/3562084419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d4009550 con 0x7f40ec0b1420 2023-08-27T11:29:42.898 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.888+0000 7f40eb7fe700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec0b1420 0x7f40ec1f3180 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f40d4000c00 tx=0x7f40d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:42.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.888+0000 7f40e3fff700 1 -- 172.21.15.112:0/3562084419 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4004570 con 0x7f40ec0b1420 2023-08-27T11:29:42.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.888+0000 7f40e3fff700 1 -- 172.21.15.112:0/3562084419 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d401f070 con 0x7f40ec0b1420 2023-08-27T11:29:42.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.888+0000 7f40f1d10700 1 -- 172.21.15.112:0/3562084419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ec0a8e90 con 0x7f40ec0b1420 2023-08-27T11:29:42.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.888+0000 7f40e3fff700 1 -- 172.21.15.112:0/3562084419 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d40038e0 con 0x7f40ec0b1420 2023-08-27T11:29:42.899 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.888+0000 7f40f1d10700 1 -- 172.21.15.112:0/3562084419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ec0a9450 con 0x7f40ec0b1420 2023-08-27T11:29:42.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.889+0000 7f40f1d10700 1 -- 172.21.15.112:0/3562084419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ec067a20 con 0x7f40ec0b1420 2023-08-27T11:29:42.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.889+0000 7f40e3fff700 1 -- 172.21.15.112:0/3562084419 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f40d4021030 con 0x7f40ec0b1420 2023-08-27T11:29:42.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.890+0000 7f40e3fff700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f40cc06a4d0 0x7f40cc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:42.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.890+0000 7f40e3fff700 1 -- 172.21.15.112:0/3562084419 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40d408a060 con 0x7f40ec0b1420 2023-08-27T11:29:42.900 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.890+0000 7f40eaffd700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f40cc06a4d0 0x7f40cc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.890+0000 7f40eaffd700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f40cc06a4d0 0x7f40cc06c990 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0af6f0 tx=0x7f40dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:42.895+0000 7f40e3fff700 1 -- 172.21.15.112:0/3562084419 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f40d40bc990 con 0x7f40ec0b1420 2023-08-27T11:29:43.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.081+0000 7f40f1d10700 1 -- 172.21.15.112:0/3562084419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "another-data-pool", "pg_num_min": 64} v 0) v1 -- 0x7f40ec159ca0 con 0x7f40ec0b1420 2023-08-27T11:29:43.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.376+0000 7f40e3fff700 1 -- 172.21.15.112:0/3562084419 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "another-data-pool", "pg_num_min": 64}]=0 pool 'another-data-pool' created v168) v1 ==== 142+0+0 (secure 0 0 0) 0x7f40d40150b0 con 0x7f40ec0b1420 2023-08-27T11:29:43.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.379+0000 7f40f1d10700 1 -- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f40cc06a4d0 msgr2=0x7f40cc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.379+0000 7f40f1d10700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f40cc06a4d0 0x7f40cc06c990 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0af6f0 tx=0x7f40dc009040 comp rx=0 tx=0).stop 2023-08-27T11:29:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.379+0000 7f40f1d10700 1 -- 172.21.15.112:0/3562084419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec0b1420 msgr2=0x7f40ec1f3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.379+0000 7f40f1d10700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec0b1420 0x7f40ec1f3180 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f40d4000c00 tx=0x7f40d4004750 comp rx=0 tx=0).stop 2023-08-27T11:29:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.380+0000 7f40f1d10700 1 -- 172.21.15.112:0/3562084419 shutdown_connections 2023-08-27T11:29:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.380+0000 7f40f1d10700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f40cc06a4d0 0x7f40cc06c990 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.380+0000 7f40f1d10700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40ec0a9970 0x7f40ec0a9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.380+0000 7f40f1d10700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec0b1420 0x7f40ec1f3180 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.380+0000 7f40f1d10700 1 --2- 172.21.15.112:0/3562084419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40ec1439e0 0x7f40ec1591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.380+0000 7f40f1d10700 1 -- 172.21.15.112:0/3562084419 >> 172.21.15.112:0/3562084419 conn(0x7f40ec0b1be0 msgr2=0x7f40ec142b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.380+0000 7f40f1d10700 1 -- 172.21.15.112:0/3562084419 shutdown_connections 2023-08-27T11:29:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.380+0000 7f40f1d10700 1 -- 172.21.15.112:0/3562084419 wait complete. 2023-08-27T11:29:43.386 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'another-data-pool' created 2023-08-27T11:29:43.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs another-data-pool 2023-08-27T11:29:43.520 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.520+0000 7fe2f5cf9700 1 Processor -- start 2023-08-27T11:29:43.521 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.520+0000 7fe2f5cf9700 1 -- start start 2023-08-27T11:29:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.521+0000 7fe2f5cf9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe2f015e190 0x7fe2f015e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.521+0000 7fe2f5cf9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe2f00cc980 0x7fe2f00cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:43.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.521+0000 7fe2f5cf9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f00b46c0 con 0x7fe2f0161dd0 2023-08-27T11:29:43.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.521+0000 7fe2f5cf9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f00cd370 con 0x7fe2f00cc980 2023-08-27T11:29:43.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.521+0000 7fe2f5cf9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f00b44d0 con 0x7fe2f015e190 2023-08-27T11:29:43.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.521+0000 7fe2effff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe2f00cc980 0x7fe2f00cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:43.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.521+0000 7fe2effff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe2f00cc980 0x7fe2f00cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38600/0 (socket says 172.21.15.112:38600) 2023-08-27T11:29:43.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.521+0000 7fe2effff700 1 -- 172.21.15.112:0/3894504367 learned_addr learned my addr 172.21.15.112:0/3894504367 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:43.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.521+0000 7fe2effff700 1 -- 172.21.15.112:0/3894504367 >> v2:172.21.15.138:3300/0 conn(0x7fe2f015e190 msgr2=0x7fe2f015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:43.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.521+0000 7fe2effff700 1 --2- 172.21.15.112:0/3894504367 >> v2:172.21.15.138:3300/0 conn(0x7fe2f015e190 0x7fe2f015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:43.525 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.521+0000 7fe2effff700 1 -- 172.21.15.112:0/3894504367 >> v1:172.21.15.138:6789/0 conn(0x7fe2f0161dd0 legacy=0x7fe2f00b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:43.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.521+0000 7fe2effff700 1 -- 172.21.15.112:0/3894504367 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f00b49a0 con 0x7fe2f00cc980 2023-08-27T11:29:43.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.522+0000 7fe2effff700 1 --2- 172.21.15.112:0/3894504367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f00cc980 0x7fe2f00cce30 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00d0fb0 tx=0x7fe2e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfb1642bcfb97e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:43.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.522+0000 7fe2ee7fc700 1 -- 172.21.15.112:0/3894504367 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e800e070 con 0x7fe2f00cc980 2023-08-27T11:29:43.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.522+0000 7fe2ee7fc700 1 -- 172.21.15.112:0/3894504367 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e8003a20 con 0x7fe2f00cc980 2023-08-27T11:29:43.526 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.522+0000 7fe2ee7fc700 1 -- 172.21.15.112:0/3894504367 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8003dd0 con 0x7fe2f00cc980 2023-08-27T11:29:43.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.522+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3894504367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f00cc980 msgr2=0x7fe2f00cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:43.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.522+0000 7fe2f5cf9700 1 --2- 172.21.15.112:0/3894504367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f00cc980 0x7fe2f00cce30 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00d0fb0 tx=0x7fe2e800b330 comp rx=0 tx=0).stop 2023-08-27T11:29:43.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.522+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3894504367 shutdown_connections 2023-08-27T11:29:43.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.522+0000 7fe2f5cf9700 1 --2- 172.21.15.112:0/3894504367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f00cc980 0x7fe2f00cce30 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:43.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.522+0000 7fe2f5cf9700 1 --2- 172.21.15.112:0/3894504367 >> v2:172.21.15.138:3300/0 conn(0x7fe2f015e190 0x7fe2f015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:43.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.522+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3894504367 >> 172.21.15.112:0/3894504367 conn(0x7fe2f006bbf0 msgr2=0x7fe2f00cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:43.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.522+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3894504367 shutdown_connections 2023-08-27T11:29:43.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.522+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3894504367 wait complete. 2023-08-27T11:29:43.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.524+0000 7fe2f5cf9700 1 Processor -- start 2023-08-27T11:29:43.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.524+0000 7fe2f5cf9700 1 -- start start 2023-08-27T11:29:43.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.524+0000 7fe2f5cf9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f00cc980 0x7fe2f00af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:43.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.524+0000 7fe2f5cf9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2f015e190 0x7fe2f00ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:43.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2f5cf9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2f0161dd0 0x7fe2f015f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:43.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2f5cf9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00ad9c0 con 0x7fe2f00cc980 2023-08-27T11:29:43.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2f5cf9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00ac890 con 0x7fe2f015e190 2023-08-27T11:29:43.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2f5cf9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00ad760 con 0x7fe2f0161dd0 2023-08-27T11:29:43.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2ef7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f00cc980 0x7fe2f00af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:43.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2ef7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f00cc980 0x7fe2f00af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38602/0 (socket says 172.21.15.112:38602) 2023-08-27T11:29:43.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2ef7fe700 1 -- 172.21.15.112:0/3079665303 learned_addr learned my addr 172.21.15.112:0/3079665303 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:43.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2eeffd700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2f015e190 0x7fe2f00ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:43.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2effff700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2f0161dd0 0x7fe2f015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:43.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2ef7fe700 1 -- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2f0161dd0 msgr2=0x7fe2f015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:43.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2ef7fe700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2f0161dd0 0x7fe2f015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:43.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2ef7fe700 1 -- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2f015e190 msgr2=0x7fe2f00ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:43.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2ef7fe700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2f015e190 0x7fe2f00ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:43.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2ef7fe700 1 -- 172.21.15.112:0/3079665303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e8009b30 con 0x7fe2f00cc980 2023-08-27T11:29:43.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2eeffd700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2f015e190 0x7fe2f00ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:43.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2ef7fe700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f00cc980 0x7fe2f00af8a0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800b530 tx=0x7fe2d800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:43.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2e7fff700 1 -- 172.21.15.112:0/3079665303 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8011830 con 0x7fe2f00cc980 2023-08-27T11:29:43.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2e7fff700 1 -- 172.21.15.112:0/3079665303 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d80119b0 con 0x7fe2f00cc980 2023-08-27T11:29:43.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3079665303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f00acaa0 con 0x7fe2f00cc980 2023-08-27T11:29:43.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2e7fff700 1 -- 172.21.15.112:0/3079665303 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8011d60 con 0x7fe2f00cc980 2023-08-27T11:29:43.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.525+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3079665303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f015fd70 con 0x7fe2f00cc980 2023-08-27T11:29:43.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.526+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3079665303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f0066fa0 con 0x7fe2f00cc980 2023-08-27T11:29:43.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.527+0000 7fe2e7fff700 1 -- 172.21.15.112:0/3079665303 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe2d801a400 con 0x7fe2f00cc980 2023-08-27T11:29:43.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.527+0000 7fe2e7fff700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe2d006a4d0 0x7fe2d006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:43.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.527+0000 7fe2e7fff700 1 -- 172.21.15.112:0/3079665303 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6434+0+0 (secure 0 0 0) 0x7fe2d8088e30 con 0x7fe2f00cc980 2023-08-27T11:29:43.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.527+0000 7fe2eeffd700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe2d006a4d0 0x7fe2d006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:43.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.528+0000 7fe2eeffd700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe2d006a4d0 0x7fe2d006c990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe2f01556e0 tx=0x7fe2e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:43.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.532+0000 7fe2e7fff700 1 -- 172.21.15.112:0/3079665303 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe2d8053e50 con 0x7fe2f00cc980 2023-08-27T11:29:43.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:43.767+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3079665303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "another-data-pool"} v 0) v1 -- 0x7fe2f0070aa0 con 0x7fe2f00cc980 2023-08-27T11:29:44.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.387+0000 7fe2e7fff700 1 -- 172.21.15.112:0/3079665303 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "another-data-pool"}]=0 added data pool 31 to fsmap v115) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe2f0070aa0 con 0x7fe2f00cc980 2023-08-27T11:29:44.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.392+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe2d006a4d0 msgr2=0x7fe2d006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:44.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.392+0000 7fe2f5cf9700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe2d006a4d0 0x7fe2d006c990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe2f01556e0 tx=0x7fe2e0008040 comp rx=0 tx=0).stop 2023-08-27T11:29:44.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.392+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3079665303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f00cc980 msgr2=0x7fe2f00af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:44.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.392+0000 7fe2f5cf9700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f00cc980 0x7fe2f00af8a0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800b530 tx=0x7fe2d800b8f0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.393+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3079665303 shutdown_connections 2023-08-27T11:29:44.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.393+0000 7fe2f5cf9700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe2d006a4d0 0x7fe2d006c990 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.393+0000 7fe2f5cf9700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2f0161dd0 0x7fe2f015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.394 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.393+0000 7fe2f5cf9700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f00cc980 0x7fe2f00af8a0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.393+0000 7fe2f5cf9700 1 --2- 172.21.15.112:0/3079665303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2f015e190 0x7fe2f00ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.393+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3079665303 >> 172.21.15.112:0/3079665303 conn(0x7fe2f006bbf0 msgr2=0x7fe2f0157000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:44.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.393+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3079665303 shutdown_connections 2023-08-27T11:29:44.395 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.393+0000 7fe2f5cf9700 1 -- 172.21.15.112:0/3079665303 wait complete. 2023-08-27T11:29:44.396 INFO:teuthology.orchestra.run.smithi112.stderr:added data pool 31 to fsmap 2023-08-27T11:29:44.407 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:44.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.527+0000 7f49c9730700 1 Processor -- start 2023-08-27T11:29:44.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.528+0000 7f49c9730700 1 -- start start 2023-08-27T11:29:44.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.528+0000 7f49c9730700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49c415a120 0x7f49c40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.528+0000 7f49c9730700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f49c406d5b0 0x7f49c406da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.528+0000 7f49c9730700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c406df50 con 0x7f49c40b3440 2023-08-27T11:29:44.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.528+0000 7f49c9730700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c415b540 con 0x7f49c406d5b0 2023-08-27T11:29:44.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.528+0000 7f49c9730700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40af830 con 0x7f49c415a120 2023-08-27T11:29:44.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c37fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f49c40b3440 0x7f49c40b38b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32996/0 (socket says 172.21.15.112:32996) 2023-08-27T11:29:44.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c37fe700 1 -- 172.21.15.112:0/1161543010 learned_addr learned my addr 172.21.15.112:0/1161543010 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:44.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c27fc700 1 --2- 172.21.15.112:0/1161543010 >> v2:172.21.15.138:3300/0 conn(0x7f49c406d5b0 0x7f49c406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c2ffd700 1 --2- 172.21.15.112:0/1161543010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49c415a120 0x7f49c40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637481907 0 0) 0x7f49c406df50 con 0x7f49c40b3440 2023-08-27T11:29:44.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b00035b0 con 0x7f49c40b3440 2023-08-27T11:29:44.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1565984128 0 0) 0x7f49b00035b0 con 0x7f49c40b3440 2023-08-27T11:29:44.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49c415a120 msgr2=0x7f49c40b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:44.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 --2- 172.21.15.112:0/1161543010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49c415a120 0x7f49c40b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 >> v2:172.21.15.138:3300/0 conn(0x7f49c406d5b0 msgr2=0x7f49c406da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:44.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 --2- 172.21.15.112:0/1161543010 >> v2:172.21.15.138:3300/0 conn(0x7f49c406d5b0 0x7f49c406da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40afb10 con 0x7f49c40b3440 2023-08-27T11:29:44.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f49bc0019f0 con 0x7f49c40b3440 2023-08-27T11:29:44.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c27fc700 1 --2- 172.21.15.112:0/1161543010 >> v2:172.21.15.138:3300/0 conn(0x7f49c406d5b0 0x7f49c406da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:44.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 >> v1:172.21.15.112:6789/0 conn(0x7f49c40b3440 legacy=0x7f49c40b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:44.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 --2- 172.21.15.112:0/1161543010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b0003a00 0x7f49b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 --2- 172.21.15.112:0/1161543010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49b0004850 0x7f49b0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 --2- 172.21.15.112:0/1161543010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49c415a120 0x7f49b0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c415b540 con 0x7f49b0004850 2023-08-27T11:29:44.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40af830 con 0x7f49b0003a00 2023-08-27T11:29:44.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b000ba10 con 0x7f49c415a120 2023-08-27T11:29:44.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c27fc700 1 --2- 172.21.15.112:0/1161543010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49b0004850 0x7f49b0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c2ffd700 1 --2- 172.21.15.112:0/1161543010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b0003a00 0x7f49b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.529+0000 7f49c37fe700 1 --2- 172.21.15.112:0/1161543010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49c415a120 0x7f49b0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c27fc700 1 -- 172.21.15.112:0/1161543010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49c415a120 msgr2=0x7f49b0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:44.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c27fc700 1 --2- 172.21.15.112:0/1161543010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49c415a120 0x7f49b0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c27fc700 1 -- 172.21.15.112:0/1161543010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b0003a00 msgr2=0x7f49b0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:44.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c27fc700 1 --2- 172.21.15.112:0/1161543010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b0003a00 0x7f49b0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c27fc700 1 -- 172.21.15.112:0/1161543010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b00035b0 con 0x7f49b0004850 2023-08-27T11:29:44.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c2ffd700 1 --2- 172.21.15.112:0/1161543010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b0003a00 0x7f49b0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:44.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c27fc700 1 --2- 172.21.15.112:0/1161543010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49b0004850 0x7f49b0004cd0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009d30 tx=0x7f49b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ddb408c737af24 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:44.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac009e80 con 0x7f49b0004850 2023-08-27T11:29:44.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac00e9a0 con 0x7f49b0004850 2023-08-27T11:29:44.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c1ffb700 1 -- 172.21.15.112:0/1161543010 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00ed30 con 0x7f49b0004850 2023-08-27T11:29:44.539 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c9730700 1 -- 172.21.15.112:0/1161543010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49b0004850 msgr2=0x7f49b0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:44.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c9730700 1 --2- 172.21.15.112:0/1161543010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49b0004850 0x7f49b0004cd0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009d30 tx=0x7f49b0005880 comp rx=0 tx=0).stop 2023-08-27T11:29:44.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c37fe700 1 -- 172.21.15.112:0/1161543010 reap_dead start 2023-08-27T11:29:44.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c9730700 1 -- 172.21.15.112:0/1161543010 shutdown_connections 2023-08-27T11:29:44.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c9730700 1 -- 172.21.15.112:0/1161543010 >> 172.21.15.112:0/1161543010 conn(0x7f49c40cff80 msgr2=0x7f49c40d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:44.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c9730700 1 -- 172.21.15.112:0/1161543010 shutdown_connections 2023-08-27T11:29:44.540 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.530+0000 7f49c9730700 1 -- 172.21.15.112:0/1161543010 wait complete. 2023-08-27T11:29:44.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.532+0000 7f49c9730700 1 Processor -- start 2023-08-27T11:29:44.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.532+0000 7f49c9730700 1 -- start start 2023-08-27T11:29:44.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.532+0000 7f49c9730700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49b0004850 0x7f49c414b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.532+0000 7f49c9730700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49c4070e70 0x7f49c40712f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.532+0000 7f49c9730700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49c413d810 0x7f49c413dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.532+0000 7f49c9730700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c413e200 con 0x7f49c413d810 2023-08-27T11:29:44.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.532+0000 7f49c9730700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c41434d0 con 0x7f49c4070e70 2023-08-27T11:29:44.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.532+0000 7f49c9730700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c41437b0 con 0x7f49b0004850 2023-08-27T11:29:44.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49c413d810 0x7f49c413dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49c413d810 0x7f49c413dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38626/0 (socket says 172.21.15.112:38626) 2023-08-27T11:29:44.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c37fe700 1 -- 172.21.15.112:0/3824253417 learned_addr learned my addr 172.21.15.112:0/3824253417 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:44.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c27fc700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49c4070e70 0x7f49c40712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c2ffd700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49b0004850 0x7f49c414b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c37fe700 1 -- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49b0004850 msgr2=0x7f49c414b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:44.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c37fe700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49b0004850 0x7f49c414b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c37fe700 1 -- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49c4070e70 msgr2=0x7f49c40712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:44.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c37fe700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49c4070e70 0x7f49c40712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c37fe700 1 -- 172.21.15.112:0/3824253417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c41424d0 con 0x7f49c413d810 2023-08-27T11:29:44.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c27fc700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49c4070e70 0x7f49c40712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:44.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c37fe700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49c413d810 0x7f49c413dcc0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f49c40b5cd0 tx=0x7f49bc00a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:44.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49bb7fe700 1 -- 172.21.15.112:0/3824253417 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0019e0 con 0x7f49c413d810 2023-08-27T11:29:44.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49bb7fe700 1 -- 172.21.15.112:0/3824253417 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc00ad80 con 0x7f49c413d810 2023-08-27T11:29:44.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c9730700 1 -- 172.21.15.112:0/3824253417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c41427b0 con 0x7f49c413d810 2023-08-27T11:29:44.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49bb7fe700 1 -- 172.21.15.112:0/3824253417 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc013630 con 0x7f49c413d810 2023-08-27T11:29:44.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.533+0000 7f49c9730700 1 -- 172.21.15.112:0/3824253417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c4143040 con 0x7f49c413d810 2023-08-27T11:29:44.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.534+0000 7f49c9730700 1 -- 172.21.15.112:0/3824253417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c4067a20 con 0x7f49c413d810 2023-08-27T11:29:44.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.535+0000 7f49bb7fe700 1 -- 172.21.15.112:0/3824253417 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f49bc013790 con 0x7f49c413d810 2023-08-27T11:29:44.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.535+0000 7f49bb7fe700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f49a406a500 0x7f49a406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.535+0000 7f49bb7fe700 1 -- 172.21.15.112:0/3824253417 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f49bc089420 con 0x7f49c413d810 2023-08-27T11:29:44.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.535+0000 7f49c2ffd700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f49a406a500 0x7f49a406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.536+0000 7f49c2ffd700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f49a406a500 0x7f49a406c9c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009b00 tx=0x7f49b4009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:44.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.539+0000 7f49bb7fe700 1 -- 172.21.15.112:0/3824253417 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f49bc054420 con 0x7f49c413d810 2023-08-27T11:29:44.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.768+0000 7f49c9730700 1 -- 172.21.15.112:0/3824253417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49c41438f0 con 0x7f49c413d810 2023-08-27T11:29:44.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.769+0000 7f49bb7fe700 1 -- 172.21.15.112:0/3824253417 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4897 (secure 0 0 0) 0x7f49bc057a40 con 0x7f49c413d810 2023-08-27T11:29:44.769 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":115,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/685371648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":685371648},{"type":"v1","addr":"172.21.15.138:6836","nonce":685371648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":7214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3585152138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3585152138},{"type":"v1","addr":"172.21.15.138:6837","nonce":3585152138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":8203,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2190275630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2190275630},{"type":"v1","addr":"172.21.15.112:6835","nonce":2190275630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:29:22.938324+0000","modified":"2023-08-27T11:29:44.376275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6833/4086629952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4086629952},{"type":"v1","addr":"172.21.15.112:6833","nonce":4086629952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-08-27T11:29:44.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.771+0000 7f49c9730700 1 -- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f49a406a500 msgr2=0x7f49a406c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:44.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.771+0000 7f49c9730700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f49a406a500 0x7f49a406c9c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009b00 tx=0x7f49b4009210 comp rx=0 tx=0).stop 2023-08-27T11:29:44.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.771+0000 7f49c9730700 1 -- 172.21.15.112:0/3824253417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49c413d810 msgr2=0x7f49c413dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:44.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.771+0000 7f49c9730700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49c413d810 0x7f49c413dcc0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f49c40b5cd0 tx=0x7f49bc00a640 comp rx=0 tx=0).stop 2023-08-27T11:29:44.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.771+0000 7f49c9730700 1 -- 172.21.15.112:0/3824253417 shutdown_connections 2023-08-27T11:29:44.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.771+0000 7f49c9730700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f49a406a500 0x7f49a406c9c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.771+0000 7f49c9730700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49c413d810 0x7f49c413dcc0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.771+0000 7f49c9730700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49b0004850 0x7f49c414b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.771+0000 7f49c9730700 1 --2- 172.21.15.112:0/3824253417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49c4070e70 0x7f49c40712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.771+0000 7f49c9730700 1 -- 172.21.15.112:0/3824253417 >> 172.21.15.112:0/3824253417 conn(0x7f49c40cff80 msgr2=0x7f49c40d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:44.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.771+0000 7f49c9730700 1 -- 172.21.15.112:0/3824253417 shutdown_connections 2023-08-27T11:29:44.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.771+0000 7f49c9730700 1 -- 172.21.15.112:0/3824253417 wait complete. 2023-08-27T11:29:44.774 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 115 2023-08-27T11:29:44.785 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:44.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.904+0000 7fc2cdbf0700 1 Processor -- start 2023-08-27T11:29:44.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.905+0000 7fc2cdbf0700 1 -- start start 2023-08-27T11:29:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.905+0000 7fc2cdbf0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806fea0 0x7fc2c80cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.905+0000 7fc2cdbf0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c8161850 con 0x7fc2c80b17e0 2023-08-27T11:29:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.905+0000 7fc2cdbf0700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c80cfc10 con 0x7fc2c80b1400 2023-08-27T11:29:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.905+0000 7fc2cdbf0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8161200 con 0x7fc2c806fea0 2023-08-27T11:29:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.905+0000 7fc2c6ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc2c80b17e0 0x7fc2c806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33014/0 (socket says 172.21.15.112:33014) 2023-08-27T11:29:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.905+0000 7fc2c6ffd700 1 -- 172.21.15.112:0/4005532989 learned_addr learned my addr 172.21.15.112:0/4005532989 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.906+0000 7fc2c7fff700 1 --2- 172.21.15.112:0/4005532989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806fea0 0x7fc2c80cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.906+0000 7fc2c67fc700 1 -- 172.21.15.112:0/4005532989 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727421713 0 0) 0x7fc2c8161850 con 0x7fc2c80b17e0 2023-08-27T11:29:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.906+0000 7fc2c67fc700 1 -- 172.21.15.112:0/4005532989 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2b40035b0 con 0x7fc2c80b17e0 2023-08-27T11:29:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.906+0000 7fc2c67fc700 1 -- 172.21.15.112:0/4005532989 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761730380 0 0) 0x7fc2c80cfc10 con 0x7fc2c80b1400 2023-08-27T11:29:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.906+0000 7fc2c67fc700 1 -- 172.21.15.112:0/4005532989 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2c8161850 con 0x7fc2c80b1400 2023-08-27T11:29:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.906+0000 7fc2c67fc700 1 -- 172.21.15.112:0/4005532989 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 870547110 0 0) 0x7fc2b40035b0 con 0x7fc2c80b17e0 2023-08-27T11:29:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.906+0000 7fc2c7fff700 1 -- 172.21.15.112:0/4005532989 >> v1:172.21.15.138:6789/0 conn(0x7fc2c80b1400 legacy=0x7fc2c806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.906+0000 7fc2c7fff700 1 -- 172.21.15.112:0/4005532989 >> v1:172.21.15.112:6789/0 conn(0x7fc2c80b17e0 legacy=0x7fc2c806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.906+0000 7fc2c7fff700 1 -- 172.21.15.112:0/4005532989 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c81614e0 con 0x7fc2c806fea0 2023-08-27T11:29:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.906+0000 7fc2c7fff700 1 --2- 172.21.15.112:0/4005532989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806fea0 0x7fc2c80cf5a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009dc0 tx=0x7fc2c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=778395156154db8d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.906+0000 7fc2c67fc700 1 -- 172.21.15.112:0/4005532989 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c000e070 con 0x7fc2c806fea0 2023-08-27T11:29:44.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.906+0000 7fc2c67fc700 1 -- 172.21.15.112:0/4005532989 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c0003a20 con 0x7fc2c806fea0 2023-08-27T11:29:44.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.906+0000 7fc2c67fc700 1 -- 172.21.15.112:0/4005532989 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8001670 con 0x7fc2c806fea0 2023-08-27T11:29:44.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.907+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/4005532989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806fea0 msgr2=0x7fc2c80cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:44.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.907+0000 7fc2cdbf0700 1 --2- 172.21.15.112:0/4005532989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806fea0 0x7fc2c80cf5a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009dc0 tx=0x7fc2c000b330 comp rx=0 tx=0).stop 2023-08-27T11:29:44.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.907+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/4005532989 shutdown_connections 2023-08-27T11:29:44.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.907+0000 7fc2cdbf0700 1 --2- 172.21.15.112:0/4005532989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806fea0 0x7fc2c80cf5a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.907+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/4005532989 >> 172.21.15.112:0/4005532989 conn(0x7fc2c80d0000 msgr2=0x7fc2c80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:44.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.907+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/4005532989 shutdown_connections 2023-08-27T11:29:44.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.907+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/4005532989 wait complete. 2023-08-27T11:29:44.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.909+0000 7fc2cdbf0700 1 Processor -- start 2023-08-27T11:29:44.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.909+0000 7fc2cdbf0700 1 -- start start 2023-08-27T11:29:44.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2cdbf0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c806fea0 0x7fc2c81f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2cdbf0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2c80b1400 0x7fc2c8148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2cdbf0700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c80b17e0 0x7fc2c81f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2cdbf0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8148690 con 0x7fc2c80b1400 2023-08-27T11:29:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2cdbf0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80c8d10 con 0x7fc2c806fea0 2023-08-27T11:29:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2cdbf0700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80c8ff0 con 0x7fc2c80b17e0 2023-08-27T11:29:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2c6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2c80b1400 0x7fc2c8148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2c6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2c80b1400 0x7fc2c8148150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38638/0 (socket says 172.21.15.112:38638) 2023-08-27T11:29:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2c6ffd700 1 -- 172.21.15.112:0/1489834380 learned_addr learned my addr 172.21.15.112:0/1489834380 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2c77fe700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c806fea0 0x7fc2c81f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2c7fff700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c80b17e0 0x7fc2c81f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2c6ffd700 1 -- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c80b17e0 msgr2=0x7fc2c81f04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2c6ffd700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c80b17e0 0x7fc2c81f04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2c6ffd700 1 -- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c806fea0 msgr2=0x7fc2c81f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2c6ffd700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c806fea0 0x7fc2c81f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2c6ffd700 1 -- 172.21.15.112:0/1489834380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c0009a70 con 0x7fc2c80b1400 2023-08-27T11:29:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2c7fff700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c80b17e0 0x7fc2c81f04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2c6ffd700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2c80b1400 0x7fc2c8148150 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80011e0 tx=0x7fc2b800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:44.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2bffff700 1 -- 172.21.15.112:0/1489834380 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8004c80 con 0x7fc2c80b1400 2023-08-27T11:29:44.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2bffff700 1 -- 172.21.15.112:0/1489834380 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b8004de0 con 0x7fc2c80b1400 2023-08-27T11:29:44.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/1489834380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c80c92d0 con 0x7fc2c80b1400 2023-08-27T11:29:44.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2bffff700 1 -- 172.21.15.112:0/1489834380 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8007310 con 0x7fc2c80b1400 2023-08-27T11:29:44.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.910+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/1489834380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c80c89c0 con 0x7fc2c80b1400 2023-08-27T11:29:44.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.911+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/1489834380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c8067a20 con 0x7fc2c80b1400 2023-08-27T11:29:44.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.912+0000 7fc2bffff700 1 -- 172.21.15.112:0/1489834380 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fc2b8007470 con 0x7fc2c80b1400 2023-08-27T11:29:44.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.912+0000 7fc2bffff700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc2a806a4d0 0x7fc2a806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:44.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.912+0000 7fc2bffff700 1 -- 172.21.15.112:0/1489834380 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fc2b8088990 con 0x7fc2c80b1400 2023-08-27T11:29:44.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.912+0000 7fc2c77fe700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc2a806a4d0 0x7fc2a806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:44.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.913+0000 7fc2c77fe700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc2a806a4d0 0x7fc2a806c990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00011e0 tx=0x7fc2b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:44.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:44.916+0000 7fc2bffff700 1 -- 172.21.15.112:0/1489834380 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc2b8053aa0 con 0x7fc2c80b1400 2023-08-27T11:29:45.106 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.104+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/1489834380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2c806db60 con 0x7fc2c80b1400 2023-08-27T11:29:45.107 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.106+0000 7fc2bffff700 1 -- 172.21.15.112:0/1489834380 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+20058 (secure 0 0 0) 0x7fc2b80570c0 con 0x7fc2c80b1400 2023-08-27T11:29:45.107 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:45.108 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":169,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:43.767765+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:29:20.569324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-08-27T11:29:21.905812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"another-data-pool","create_time":"2023-08-27T11:29:43.100842+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:45.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.108+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc2a806a4d0 msgr2=0x7fc2a806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.108+0000 7fc2cdbf0700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc2a806a4d0 0x7fc2a806c990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00011e0 tx=0x7fc2b0009040 comp rx=0 tx=0).stop 2023-08-27T11:29:45.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.108+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/1489834380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2c80b1400 msgr2=0x7fc2c8148150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.108+0000 7fc2cdbf0700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2c80b1400 0x7fc2c8148150 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80011e0 tx=0x7fc2b800c800 comp rx=0 tx=0).stop 2023-08-27T11:29:45.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.108+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/1489834380 shutdown_connections 2023-08-27T11:29:45.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.108+0000 7fc2cdbf0700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc2a806a4d0 0x7fc2a806c990 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.108+0000 7fc2cdbf0700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c80b17e0 0x7fc2c81f04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.110 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.108+0000 7fc2cdbf0700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c806fea0 0x7fc2c81f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.108+0000 7fc2cdbf0700 1 --2- 172.21.15.112:0/1489834380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2c80b1400 0x7fc2c8148150 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.108+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/1489834380 >> 172.21.15.112:0/1489834380 conn(0x7fc2c80d0000 msgr2=0x7fc2c8155830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:45.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.108+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/1489834380 shutdown_connections 2023-08-27T11:29:45.111 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.108+0000 7fc2cdbf0700 1 -- 172.21.15.112:0/1489834380 wait complete. 2023-08-27T11:29:45.121 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:45.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.240+0000 7f0e8af21700 1 Processor -- start 2023-08-27T11:29:45.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.240+0000 7f0e8af21700 1 -- start start 2023-08-27T11:29:45.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.241+0000 7f0e8af21700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0e840b1400 0x7f0e840cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.241+0000 7f0e8af21700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e84161ce0 0x7f0e8406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.241+0000 7f0e8af21700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e84161850 con 0x7f0e840b17e0 2023-08-27T11:29:45.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.241+0000 7f0e8af21700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e840cfc10 con 0x7f0e840b1400 2023-08-27T11:29:45.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.241+0000 7f0e8af21700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e84161200 con 0x7f0e84161ce0 2023-08-27T11:29:45.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.241+0000 7f0e88cbd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0e840b1400 0x7f0e840cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.241+0000 7f0e88cbd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0e840b1400 0x7f0e840cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38644/0 (socket says 172.21.15.112:38644) 2023-08-27T11:29:45.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.241+0000 7f0e88cbd700 1 -- 172.21.15.112:0/4032569881 learned_addr learned my addr 172.21.15.112:0/4032569881 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:45.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.241+0000 7f0e83fff700 1 --1- 172.21.15.112:0/4032569881 >> v1:172.21.15.112:6789/0 conn(0x7f0e840b17e0 0x7f0e8406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33026/0 (socket says 172.21.15.112:33026) 2023-08-27T11:29:45.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.242+0000 7f0e894be700 1 --2- 172.21.15.112:0/4032569881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e84161ce0 0x7f0e8406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.242+0000 7f0e88cbd700 1 -- 172.21.15.112:0/4032569881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e84161ce0 msgr2=0x7f0e8406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:45.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.242+0000 7f0e88cbd700 1 --2- 172.21.15.112:0/4032569881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e84161ce0 0x7f0e8406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.242+0000 7f0e88cbd700 1 -- 172.21.15.112:0/4032569881 >> v1:172.21.15.112:6789/0 conn(0x7f0e840b17e0 legacy=0x7f0e8406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.242+0000 7f0e837fe700 1 -- 172.21.15.112:0/4032569881 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212795615 0 0) 0x7f0e84161850 con 0x7f0e840b17e0 2023-08-27T11:29:45.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.242+0000 7f0e88cbd700 1 -- 172.21.15.112:0/4032569881 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e841614e0 con 0x7f0e840b1400 2023-08-27T11:29:45.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.242+0000 7f0e88cbd700 1 --2- 172.21.15.112:0/4032569881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e840b1400 0x7f0e840cf5a0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f0e780098f0 tx=0x7f0e78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b75d4853bf1f69 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:45.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.242+0000 7f0e837fe700 1 -- 172.21.15.112:0/4032569881 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7800b8a0 con 0x7f0e840b1400 2023-08-27T11:29:45.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.242+0000 7f0e837fe700 1 -- 172.21.15.112:0/4032569881 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7800ba00 con 0x7f0e840b1400 2023-08-27T11:29:45.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.242+0000 7f0e837fe700 1 -- 172.21.15.112:0/4032569881 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7800bd90 con 0x7f0e840b1400 2023-08-27T11:29:45.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.242+0000 7f0e8af21700 1 -- 172.21.15.112:0/4032569881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e840b1400 msgr2=0x7f0e840cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.242+0000 7f0e8af21700 1 --2- 172.21.15.112:0/4032569881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e840b1400 0x7f0e840cf5a0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f0e780098f0 tx=0x7f0e78009c00 comp rx=0 tx=0).stop 2023-08-27T11:29:45.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.243+0000 7f0e8af21700 1 -- 172.21.15.112:0/4032569881 shutdown_connections 2023-08-27T11:29:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.243+0000 7f0e8af21700 1 --2- 172.21.15.112:0/4032569881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e84161ce0 0x7f0e8406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.243+0000 7f0e8af21700 1 --2- 172.21.15.112:0/4032569881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e840b1400 0x7f0e840cf5a0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.243+0000 7f0e8af21700 1 -- 172.21.15.112:0/4032569881 >> 172.21.15.112:0/4032569881 conn(0x7f0e840d0000 msgr2=0x7f0e840d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.243+0000 7f0e8af21700 1 -- 172.21.15.112:0/4032569881 shutdown_connections 2023-08-27T11:29:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.243+0000 7f0e8af21700 1 -- 172.21.15.112:0/4032569881 wait complete. 2023-08-27T11:29:45.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.244+0000 7f0e8af21700 1 Processor -- start 2023-08-27T11:29:45.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.244+0000 7f0e8af21700 1 -- start start 2023-08-27T11:29:45.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e8af21700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e840b1400 0x7f0e8406b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e8af21700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e840b17e0 0x7f0e84151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e8af21700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e84161ce0 0x7f0e840c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e8af21700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840c9b10 con 0x7f0e84161ce0 2023-08-27T11:29:45.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e8af21700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840c9d60 con 0x7f0e840b17e0 2023-08-27T11:29:45.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e8af21700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840c80a0 con 0x7f0e840b1400 2023-08-27T11:29:45.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e894be700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e84161ce0 0x7f0e840c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e894be700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e84161ce0 0x7f0e840c3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38652/0 (socket says 172.21.15.112:38652) 2023-08-27T11:29:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e894be700 1 -- 172.21.15.112:0/1390740870 learned_addr learned my addr 172.21.15.112:0/1390740870 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e88cbd700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e840b1400 0x7f0e8406b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e83fff700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e840b17e0 0x7f0e84151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e894be700 1 -- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e840b1400 msgr2=0x7f0e8406b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e894be700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e840b1400 0x7f0e8406b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e894be700 1 -- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e840b17e0 msgr2=0x7f0e84151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e894be700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e840b17e0 0x7f0e84151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e894be700 1 -- 172.21.15.112:0/1390740870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e78009550 con 0x7f0e84161ce0 2023-08-27T11:29:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e894be700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e84161ce0 0x7f0e840c3b30 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c007e60 tx=0x7f0e7c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e817fa700 1 -- 172.21.15.112:0/1390740870 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c00ede0 con 0x7f0e84161ce0 2023-08-27T11:29:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e817fa700 1 -- 172.21.15.112:0/1390740870 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c014420 con 0x7f0e84161ce0 2023-08-27T11:29:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e8af21700 1 -- 172.21.15.112:0/1390740870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e840c8380 con 0x7f0e84161ce0 2023-08-27T11:29:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e817fa700 1 -- 172.21.15.112:0/1390740870 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0147d0 con 0x7f0e84161ce0 2023-08-27T11:29:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.245+0000 7f0e8af21700 1 -- 172.21.15.112:0/1390740870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e840c7d50 con 0x7f0e84161ce0 2023-08-27T11:29:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.246+0000 7f0e8af21700 1 -- 172.21.15.112:0/1390740870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e840c8570 con 0x7f0e84161ce0 2023-08-27T11:29:45.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.247+0000 7f0e817fa700 1 -- 172.21.15.112:0/1390740870 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f0e7c014930 con 0x7f0e84161ce0 2023-08-27T11:29:45.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.247+0000 7f0e817fa700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e6806a4d0 0x7f0e6806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.247+0000 7f0e817fa700 1 -- 172.21.15.112:0/1390740870 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f0e7c089470 con 0x7f0e84161ce0 2023-08-27T11:29:45.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.248+0000 7f0e88cbd700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e6806a4d0 0x7f0e6806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.248+0000 7f0e88cbd700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e6806a4d0 0x7f0e6806c990 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0e78000c00 tx=0x7f0e78008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:45.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.251+0000 7f0e817fa700 1 -- 172.21.15.112:0/1390740870 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0e7c054430 con 0x7f0e84161ce0 2023-08-27T11:29:45.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.484+0000 7f0e8af21700 1 -- 172.21.15.112:0/1390740870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e84067a20 con 0x7f0e84161ce0 2023-08-27T11:29:45.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.484+0000 7f0e817fa700 1 -- 172.21.15.112:0/1390740870 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4897 (secure 0 0 0) 0x7f0e7c057a50 con 0x7f0e84161ce0 2023-08-27T11:29:45.485 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:45.485 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":115,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/685371648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":685371648},{"type":"v1","addr":"172.21.15.138:6836","nonce":685371648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":7214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3585152138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3585152138},{"type":"v1","addr":"172.21.15.138:6837","nonce":3585152138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":8203,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2190275630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2190275630},{"type":"v1","addr":"172.21.15.112:6835","nonce":2190275630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:29:22.938324+0000","modified":"2023-08-27T11:29:44.376275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6833/4086629952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4086629952},{"type":"v1","addr":"172.21.15.112:6833","nonce":4086629952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-08-27T11:29:45.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.486+0000 7f0e8af21700 1 -- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e6806a4d0 msgr2=0x7f0e6806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.486+0000 7f0e8af21700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e6806a4d0 0x7f0e6806c990 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0e78000c00 tx=0x7f0e78008030 comp rx=0 tx=0).stop 2023-08-27T11:29:45.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.486+0000 7f0e8af21700 1 -- 172.21.15.112:0/1390740870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e84161ce0 msgr2=0x7f0e840c3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.486+0000 7f0e8af21700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e84161ce0 0x7f0e840c3b30 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c007e60 tx=0x7f0e7c00e3c0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.486+0000 7f0e8af21700 1 -- 172.21.15.112:0/1390740870 shutdown_connections 2023-08-27T11:29:45.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.486+0000 7f0e8af21700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e6806a4d0 0x7f0e6806c990 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.486+0000 7f0e8af21700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e84161ce0 0x7f0e840c3b30 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.486+0000 7f0e8af21700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e840b1400 0x7f0e8406b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.486+0000 7f0e8af21700 1 --2- 172.21.15.112:0/1390740870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e840b17e0 0x7f0e84151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.486+0000 7f0e8af21700 1 -- 172.21.15.112:0/1390740870 >> 172.21.15.112:0/1390740870 conn(0x7f0e840d0000 msgr2=0x7f0e840cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:45.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.486+0000 7f0e8af21700 1 -- 172.21.15.112:0/1390740870 shutdown_connections 2023-08-27T11:29:45.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.486+0000 7f0e8af21700 1 -- 172.21.15.112:0/1390740870 wait complete. 2023-08-27T11:29:45.489 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 115 2023-08-27T11:29:45.498 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:45.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.618+0000 7f552c2a1700 1 Processor -- start 2023-08-27T11:29:45.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.618+0000 7f552c2a1700 1 -- start start 2023-08-27T11:29:45.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.619+0000 7f552c2a1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5524158c90 0x7f552406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.619+0000 7f552c2a1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5524159070 0x7f552406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.619+0000 7f552c2a1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f55240b1270 0x7f552406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.619+0000 7f552c2a1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552415f5d0 con 0x7f5524158c90 2023-08-27T11:29:45.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.619+0000 7f552c2a1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55240ca990 con 0x7f55240b1270 2023-08-27T11:29:45.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.619+0000 7f552c2a1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552406e870 con 0x7f5524159070 2023-08-27T11:29:45.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.619+0000 7f552a03d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5524158c90 0x7f552406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.620+0000 7f552a03d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5524158c90 0x7f552406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38656/0 (socket says 172.21.15.112:38656) 2023-08-27T11:29:45.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.620+0000 7f552a03d700 1 -- 172.21.15.112:0/1903440185 learned_addr learned my addr 172.21.15.112:0/1903440185 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:45.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.620+0000 7f552a83e700 1 --2- 172.21.15.112:0/1903440185 >> v2:172.21.15.138:3300/0 conn(0x7f55240b1270 0x7f552406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.620+0000 7f552983c700 1 --2- 172.21.15.112:0/1903440185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5524159070 0x7f552406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.620+0000 7f552a03d700 1 -- 172.21.15.112:0/1903440185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5524159070 msgr2=0x7f552406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:45.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.620+0000 7f552a03d700 1 --2- 172.21.15.112:0/1903440185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5524159070 0x7f552406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.620+0000 7f552a03d700 1 -- 172.21.15.112:0/1903440185 >> v2:172.21.15.138:3300/0 conn(0x7f55240b1270 msgr2=0x7f552406fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:45.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.620+0000 7f552a03d700 1 --2- 172.21.15.112:0/1903440185 >> v2:172.21.15.138:3300/0 conn(0x7f55240b1270 0x7f552406fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.620+0000 7f552a03d700 1 -- 172.21.15.112:0/1903440185 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55240b1a10 con 0x7f5524158c90 2023-08-27T11:29:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.620+0000 7f552a03d700 1 --2- 172.21.15.112:0/1903440185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5524158c90 0x7f552406e330 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f55240cabb0 tx=0x7f5514009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f447333b0ca6f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.620+0000 7f552903b700 1 -- 172.21.15.112:0/1903440185 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514011830 con 0x7f5524158c90 2023-08-27T11:29:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.620+0000 7f552903b700 1 -- 172.21.15.112:0/1903440185 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5514011990 con 0x7f5524158c90 2023-08-27T11:29:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.620+0000 7f552903b700 1 -- 172.21.15.112:0/1903440185 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514011d20 con 0x7f5524158c90 2023-08-27T11:29:45.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.621+0000 7f552c2a1700 1 -- 172.21.15.112:0/1903440185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5524158c90 msgr2=0x7f552406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.621+0000 7f552c2a1700 1 --2- 172.21.15.112:0/1903440185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5524158c90 0x7f552406e330 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f55240cabb0 tx=0x7f5514009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.622+0000 7f552c2a1700 1 -- 172.21.15.112:0/1903440185 shutdown_connections 2023-08-27T11:29:45.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.622+0000 7f552c2a1700 1 --2- 172.21.15.112:0/1903440185 >> v2:172.21.15.138:3300/0 conn(0x7f55240b1270 0x7f552406fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.622+0000 7f552c2a1700 1 --2- 172.21.15.112:0/1903440185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5524158c90 0x7f552406e330 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.622+0000 7f552c2a1700 1 --2- 172.21.15.112:0/1903440185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5524159070 0x7f552406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.622+0000 7f552c2a1700 1 -- 172.21.15.112:0/1903440185 >> 172.21.15.112:0/1903440185 conn(0x7f55240d0000 msgr2=0x7f55240d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:45.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.623+0000 7f552c2a1700 1 -- 172.21.15.112:0/1903440185 shutdown_connections 2023-08-27T11:29:45.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.623+0000 7f552c2a1700 1 -- 172.21.15.112:0/1903440185 wait complete. 2023-08-27T11:29:45.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.624+0000 7f552c2a1700 1 Processor -- start 2023-08-27T11:29:45.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.624+0000 7f552c2a1700 1 -- start start 2023-08-27T11:29:45.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552c2a1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55240b1270 0x7f5524149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552c2a1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5524158c90 0x7f55241562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552c2a1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5524159070 0x7f5524154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552c2a1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524155230 con 0x7f55240b1270 2023-08-27T11:29:45.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552c2a1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524155510 con 0x7f5524158c90 2023-08-27T11:29:45.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552c2a1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55241557f0 con 0x7f5524159070 2023-08-27T11:29:45.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552a03d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55240b1270 0x7f5524149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552a03d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55240b1270 0x7f5524149300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38662/0 (socket says 172.21.15.112:38662) 2023-08-27T11:29:45.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552a03d700 1 -- 172.21.15.112:0/1620680157 learned_addr learned my addr 172.21.15.112:0/1620680157 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:45.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552983c700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5524158c90 0x7f55241562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552a83e700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5524159070 0x7f5524154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552a03d700 1 -- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5524159070 msgr2=0x7f5524154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552a03d700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5524159070 0x7f5524154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552a03d700 1 -- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5524158c90 msgr2=0x7f55241562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552a03d700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5524158c90 0x7f55241562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552a03d700 1 -- 172.21.15.112:0/1620680157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5514009550 con 0x7f55240b1270 2023-08-27T11:29:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.625+0000 7f552a83e700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5524159070 0x7f5524154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.626+0000 7f552a03d700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55240b1270 0x7f5524149300 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f5514000c00 tx=0x7f5514004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.626+0000 7f551affd700 1 -- 172.21.15.112:0/1620680157 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514004300 con 0x7f55240b1270 2023-08-27T11:29:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.626+0000 7f551affd700 1 -- 172.21.15.112:0/1620680157 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551401f070 con 0x7f55240b1270 2023-08-27T11:29:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.626+0000 7f552c2a1700 1 -- 172.21.15.112:0/1620680157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5524155ad0 con 0x7f55240b1270 2023-08-27T11:29:45.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.626+0000 7f551affd700 1 -- 172.21.15.112:0/1620680157 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55140036a0 con 0x7f55240b1270 2023-08-27T11:29:45.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.626+0000 7f552c2a1700 1 -- 172.21.15.112:0/1620680157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5524149840 con 0x7f55240b1270 2023-08-27T11:29:45.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.627+0000 7f552c2a1700 1 -- 172.21.15.112:0/1620680157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55241584a0 con 0x7f55240b1270 2023-08-27T11:29:45.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.627+0000 7f551affd700 1 -- 172.21.15.112:0/1620680157 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5514021030 con 0x7f55240b1270 2023-08-27T11:29:45.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.628+0000 7f551affd700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f550806a4d0 0x7f550806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.628+0000 7f551affd700 1 -- 172.21.15.112:0/1620680157 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f5514089e20 con 0x7f55240b1270 2023-08-27T11:29:45.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.628+0000 7f552983c700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f550806a4d0 0x7f550806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.628+0000 7f552983c700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f550806a4d0 0x7f550806c990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f55240c63c0 tx=0x7f5520009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:45.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.632+0000 7f551affd700 1 -- 172.21.15.112:0/1620680157 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5514054f30 con 0x7f55240b1270 2023-08-27T11:29:45.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.822+0000 7f552c2a1700 1 -- 172.21.15.112:0/1620680157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5524067a20 con 0x7f55240b1270 2023-08-27T11:29:45.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.823+0000 7f551affd700 1 -- 172.21.15.112:0/1620680157 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+20045 (secure 0 0 0) 0x7f5514015090 con 0x7f55240b1270 2023-08-27T11:29:45.824 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:45.825 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":170,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:45.384614+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:29:20.569324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-08-27T11:29:21.905812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"another-data-pool","create_time":"2023-08-27T11:29:43.100842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:45.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.825+0000 7f552c2a1700 1 -- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f550806a4d0 msgr2=0x7f550806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.825+0000 7f552c2a1700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f550806a4d0 0x7f550806c990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f55240c63c0 tx=0x7f5520009210 comp rx=0 tx=0).stop 2023-08-27T11:29:45.826 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.825+0000 7f552c2a1700 1 -- 172.21.15.112:0/1620680157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55240b1270 msgr2=0x7f5524149300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.825+0000 7f552c2a1700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55240b1270 0x7f5524149300 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f5514000c00 tx=0x7f5514004750 comp rx=0 tx=0).stop 2023-08-27T11:29:45.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.825+0000 7f552c2a1700 1 -- 172.21.15.112:0/1620680157 shutdown_connections 2023-08-27T11:29:45.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.825+0000 7f552c2a1700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f550806a4d0 0x7f550806c990 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.825+0000 7f552c2a1700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5524159070 0x7f5524154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.825+0000 7f552c2a1700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55240b1270 0x7f5524149300 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.825+0000 7f552c2a1700 1 --2- 172.21.15.112:0/1620680157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5524158c90 0x7f55241562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.825+0000 7f552c2a1700 1 -- 172.21.15.112:0/1620680157 >> 172.21.15.112:0/1620680157 conn(0x7f55240d0000 msgr2=0x7f55240d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:45.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.826+0000 7f552c2a1700 1 -- 172.21.15.112:0/1620680157 shutdown_connections 2023-08-27T11:29:45.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.826+0000 7f552c2a1700 1 -- 172.21.15.112:0/1620680157 wait complete. 2023-08-27T11:29:45.838 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000001022841 --yes-i-really-mean-it 2023-08-27T11:29:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.961+0000 7fe964c5d700 1 Processor -- start 2023-08-27T11:29:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.961+0000 7fe964c5d700 1 -- start start 2023-08-27T11:29:45.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.962+0000 7fe964c5d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe96013cfb0 0x7fe96006dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.962+0000 7fe964c5d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe96015e9a0 0x7fe96006e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.962+0000 7fe964c5d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe96015a630 0x7fe96015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.962+0000 7fe964c5d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96015dc70 con 0x7fe96015e9a0 2023-08-27T11:29:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.962+0000 7fe964c5d700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96015b460 con 0x7fe96015a630 2023-08-27T11:29:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.962+0000 7fe964c5d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96015dec0 con 0x7fe96013cfb0 2023-08-27T11:29:45.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.962+0000 7fe95dd9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe96015e9a0 0x7fe96006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.962+0000 7fe95dd9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe96015e9a0 0x7fe96006e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38670/0 (socket says 172.21.15.112:38670) 2023-08-27T11:29:45.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.962+0000 7fe95dd9b700 1 -- 172.21.15.112:0/2309331566 learned_addr learned my addr 172.21.15.112:0/2309331566 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:45.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.963+0000 7fe95ed9d700 1 --2- 172.21.15.112:0/2309331566 >> v2:172.21.15.138:3300/0 conn(0x7fe96015a630 0x7fe96015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.963+0000 7fe95e59c700 1 --2- 172.21.15.112:0/2309331566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe96013cfb0 0x7fe96006dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.963+0000 7fe95dd9b700 1 -- 172.21.15.112:0/2309331566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe96013cfb0 msgr2=0x7fe96006dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.963+0000 7fe95dd9b700 1 --2- 172.21.15.112:0/2309331566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe96013cfb0 0x7fe96006dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.963+0000 7fe95dd9b700 1 -- 172.21.15.112:0/2309331566 >> v2:172.21.15.138:3300/0 conn(0x7fe96015a630 msgr2=0x7fe96015aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.963+0000 7fe95dd9b700 1 --2- 172.21.15.112:0/2309331566 >> v2:172.21.15.138:3300/0 conn(0x7fe96015a630 0x7fe96015aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.963+0000 7fe95dd9b700 1 -- 172.21.15.112:0/2309331566 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96015e1a0 con 0x7fe96015e9a0 2023-08-27T11:29:45.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.963+0000 7fe95dd9b700 1 --2- 172.21.15.112:0/2309331566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96015e9a0 0x7fe96006e160 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fe96006d2d0 tx=0x7fe950009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4194c7861a781f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:45.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.963+0000 7fe95d59a700 1 -- 172.21.15.112:0/2309331566 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95000b870 con 0x7fe96015e9a0 2023-08-27T11:29:45.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.963+0000 7fe95d59a700 1 -- 172.21.15.112:0/2309331566 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95000b9d0 con 0x7fe96015e9a0 2023-08-27T11:29:45.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.963+0000 7fe95d59a700 1 -- 172.21.15.112:0/2309331566 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95000bd60 con 0x7fe96015e9a0 2023-08-27T11:29:45.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.964+0000 7fe964c5d700 1 -- 172.21.15.112:0/2309331566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96015e9a0 msgr2=0x7fe96006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.964+0000 7fe964c5d700 1 --2- 172.21.15.112:0/2309331566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96015e9a0 0x7fe96006e160 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fe96006d2d0 tx=0x7fe950009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.964+0000 7fe964c5d700 1 -- 172.21.15.112:0/2309331566 shutdown_connections 2023-08-27T11:29:45.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.964+0000 7fe964c5d700 1 --2- 172.21.15.112:0/2309331566 >> v2:172.21.15.138:3300/0 conn(0x7fe96015a630 0x7fe96015aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.964+0000 7fe964c5d700 1 --2- 172.21.15.112:0/2309331566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe96013cfb0 0x7fe96006dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.964+0000 7fe964c5d700 1 --2- 172.21.15.112:0/2309331566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96015e9a0 0x7fe96006e160 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.964+0000 7fe964c5d700 1 -- 172.21.15.112:0/2309331566 >> 172.21.15.112:0/2309331566 conn(0x7fe96006bc60 msgr2=0x7fe96006d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:45.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.964+0000 7fe964c5d700 1 -- 172.21.15.112:0/2309331566 shutdown_connections 2023-08-27T11:29:45.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.964+0000 7fe964c5d700 1 -- 172.21.15.112:0/2309331566 wait complete. 2023-08-27T11:29:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.965+0000 7fe964c5d700 1 Processor -- start 2023-08-27T11:29:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.965+0000 7fe964c5d700 1 -- start start 2023-08-27T11:29:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe964c5d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe96013cfb0 0x7fe96015bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe964c5d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96015a630 0x7fe96014ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe964c5d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe96015e9a0 0x7fe96014b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe964c5d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96006f710 con 0x7fe96015a630 2023-08-27T11:29:45.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe964c5d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe960155df0 con 0x7fe96015e9a0 2023-08-27T11:29:45.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe964c5d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9601560d0 con 0x7fe96013cfb0 2023-08-27T11:29:45.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe95dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96015a630 0x7fe96014ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe95dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96015a630 0x7fe96014ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38678/0 (socket says 172.21.15.112:38678) 2023-08-27T11:29:45.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe95dd9b700 1 -- 172.21.15.112:0/915982144 learned_addr learned my addr 172.21.15.112:0/915982144 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:45.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe95ed9d700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe96015e9a0 0x7fe96014b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe95e59c700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe96013cfb0 0x7fe96015bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe95dd9b700 1 -- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe96013cfb0 msgr2=0x7fe96015bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe95dd9b700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe96013cfb0 0x7fe96015bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe95dd9b700 1 -- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe96015e9a0 msgr2=0x7fe96014b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:45.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe95dd9b700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe96015e9a0 0x7fe96014b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:45.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe95dd9b700 1 -- 172.21.15.112:0/915982144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950009550 con 0x7fe96015a630 2023-08-27T11:29:45.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe95dd9b700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96015a630 0x7fe96014ae80 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fe950000c00 tx=0x7fe950004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:45.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe94effd700 1 -- 172.21.15.112:0/915982144 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950004310 con 0x7fe96015a630 2023-08-27T11:29:45.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe94effd700 1 -- 172.21.15.112:0/915982144 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95001e070 con 0x7fe96015a630 2023-08-27T11:29:45.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe964c5d700 1 -- 172.21.15.112:0/915982144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9601563b0 con 0x7fe96015a630 2023-08-27T11:29:45.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.966+0000 7fe94effd700 1 -- 172.21.15.112:0/915982144 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9500036a0 con 0x7fe96015a630 2023-08-27T11:29:45.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.967+0000 7fe964c5d700 1 -- 172.21.15.112:0/915982144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe960156970 con 0x7fe96015a630 2023-08-27T11:29:45.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.967+0000 7fe964c5d700 1 -- 172.21.15.112:0/915982144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe960067a20 con 0x7fe96015a630 2023-08-27T11:29:45.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.969+0000 7fe94effd700 1 -- 172.21.15.112:0/915982144 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe950020030 con 0x7fe96015a630 2023-08-27T11:29:45.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.969+0000 7fe94effd700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe94006a4d0 0x7fe94006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:45.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.969+0000 7fe94effd700 1 -- 172.21.15.112:0/915982144 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fe950089f60 con 0x7fe96015a630 2023-08-27T11:29:45.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.969+0000 7fe95e59c700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe94006a4d0 0x7fe94006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:45.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.970+0000 7fe95e59c700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe94006a4d0 0x7fe94006c990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe960072260 tx=0x7fe954008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:45.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:45.972+0000 7fe94effd700 1 -- 172.21.15.112:0/915982144 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe950055170 con 0x7fe96015a630 2023-08-27T11:29:46.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:46.187+0000 7fe964c5d700 1 -- 172.21.15.112:0/915982144 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000001022841", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe960156240 con 0x7fe94006a4d0 2023-08-27T11:29:47.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.412+0000 7fe94effd700 1 -- 172.21.15.112:0/915982144 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+127 (secure 0 0 0) 0x7fe960156240 con 0x7fe94006a4d0 2023-08-27T11:29:47.414 INFO:teuthology.orchestra.run.smithi112.stdout:FS volume 'cephfs' renamed to 'volume_0000000001022841'. But failed to rename data pools as more than one data pool was found. 2023-08-27T11:29:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.415+0000 7fe964c5d700 1 -- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe94006a4d0 msgr2=0x7fe94006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.415+0000 7fe964c5d700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe94006a4d0 0x7fe94006c990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe960072260 tx=0x7fe954008040 comp rx=0 tx=0).stop 2023-08-27T11:29:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.415+0000 7fe964c5d700 1 -- 172.21.15.112:0/915982144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96015a630 msgr2=0x7fe96014ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.415+0000 7fe964c5d700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96015a630 0x7fe96014ae80 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fe950000c00 tx=0x7fe950004750 comp rx=0 tx=0).stop 2023-08-27T11:29:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.415+0000 7fe964c5d700 1 -- 172.21.15.112:0/915982144 shutdown_connections 2023-08-27T11:29:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.415+0000 7fe964c5d700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe94006a4d0 0x7fe94006c990 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.415+0000 7fe964c5d700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe96015e9a0 0x7fe96014b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.415+0000 7fe964c5d700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe96013cfb0 0x7fe96015bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.415+0000 7fe964c5d700 1 --2- 172.21.15.112:0/915982144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96015a630 0x7fe96014ae80 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.415+0000 7fe964c5d700 1 -- 172.21.15.112:0/915982144 >> 172.21.15.112:0/915982144 conn(0x7fe96006bc60 msgr2=0x7fe96015d480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:47.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.415+0000 7fe964c5d700 1 -- 172.21.15.112:0/915982144 shutdown_connections 2023-08-27T11:29:47.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.415+0000 7fe964c5d700 1 -- 172.21.15.112:0/915982144 wait complete. 2023-08-27T11:29:47.431 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:29:47.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.557+0000 7f59d8806700 1 Processor -- start 2023-08-27T11:29:47.558 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.557+0000 7f59d8806700 1 -- start start 2023-08-27T11:29:47.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d8806700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59d40cfd00 0x7f59d40b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:47.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d8806700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f59d414aeb0 0x7f59d40cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:47.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d8806700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d4165f30 con 0x7f59d414a770 2023-08-27T11:29:47.561 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d8806700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d4166210 con 0x7f59d414aeb0 2023-08-27T11:29:47.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d8806700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4071190 con 0x7f59d40cfd00 2023-08-27T11:29:47.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d2d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f59d414aeb0 0x7f59d40cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:47.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d259c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f59d414a770 0x7f59d40cf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33066/0 (socket says 172.21.15.112:33066) 2023-08-27T11:29:47.562 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d259c700 1 -- 172.21.15.112:0/3520241591 learned_addr learned my addr 172.21.15.112:0/3520241591 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:47.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d1d9b700 1 --2- 172.21.15.112:0/3520241591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59d40cfd00 0x7f59d40b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:47.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d2d9d700 1 -- 172.21.15.112:0/3520241591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59d40cfd00 msgr2=0x7f59d40b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:47.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d2d9d700 1 --2- 172.21.15.112:0/3520241591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59d40cfd00 0x7f59d40b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.563 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d2d9d700 1 -- 172.21.15.112:0/3520241591 >> v1:172.21.15.112:6789/0 conn(0x7f59d414a770 legacy=0x7f59d40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:47.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d2d9d700 1 -- 172.21.15.112:0/3520241591 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d4071470 con 0x7f59d414aeb0 2023-08-27T11:29:47.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d159a700 1 -- 172.21.15.112:0/3520241591 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140994182 0 0) 0x7f59d4165f30 con 0x7f59d414a770 2023-08-27T11:29:47.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d2d9d700 1 --2- 172.21.15.112:0/3520241591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d414aeb0 0x7f59d40cbe90 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009d90 tx=0x7f59cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8986e274c6e16bc9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:47.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d159a700 1 -- 172.21.15.112:0/3520241591 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00e070 con 0x7f59d414aeb0 2023-08-27T11:29:47.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d159a700 1 -- 172.21.15.112:0/3520241591 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc00bb20 con 0x7f59d414aeb0 2023-08-27T11:29:47.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.558+0000 7f59d159a700 1 -- 172.21.15.112:0/3520241591 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc012430 con 0x7f59d414aeb0 2023-08-27T11:29:47.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.559+0000 7f59d8806700 1 -- 172.21.15.112:0/3520241591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d414aeb0 msgr2=0x7f59d40cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:47.565 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.559+0000 7f59d8806700 1 --2- 172.21.15.112:0/3520241591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d414aeb0 0x7f59d40cbe90 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009d90 tx=0x7f59cc00b330 comp rx=0 tx=0).stop 2023-08-27T11:29:47.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.559+0000 7f59d8806700 1 -- 172.21.15.112:0/3520241591 shutdown_connections 2023-08-27T11:29:47.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.559+0000 7f59d8806700 1 --2- 172.21.15.112:0/3520241591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d414aeb0 0x7f59d40cbe90 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.559+0000 7f59d8806700 1 --2- 172.21.15.112:0/3520241591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59d40cfd00 0x7f59d40b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.559+0000 7f59d8806700 1 -- 172.21.15.112:0/3520241591 >> 172.21.15.112:0/3520241591 conn(0x7f59d406f970 msgr2=0x7f59d406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:47.566 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.559+0000 7f59d8806700 1 -- 172.21.15.112:0/3520241591 shutdown_connections 2023-08-27T11:29:47.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.559+0000 7f59d8806700 1 -- 172.21.15.112:0/3520241591 wait complete. 2023-08-27T11:29:47.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.560+0000 7f59d8806700 1 Processor -- start 2023-08-27T11:29:47.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.561+0000 7f59d8806700 1 -- start start 2023-08-27T11:29:47.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.561+0000 7f59d8806700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d40cfd00 0x7f59d415bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:47.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.561+0000 7f59d8806700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59d414a770 0x7f59d40b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:47.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.561+0000 7f59d8806700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d414aeb0 0x7f59d40b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:47.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.561+0000 7f59d8806700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41460e0 con 0x7f59d414aeb0 2023-08-27T11:29:47.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.561+0000 7f59d8806700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d415c1e0 con 0x7f59d40cfd00 2023-08-27T11:29:47.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d8806700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41451f0 con 0x7f59d414a770 2023-08-27T11:29:47.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d414aeb0 0x7f59d40b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:47.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d414aeb0 0x7f59d40b0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38694/0 (socket says 172.21.15.112:38694) 2023-08-27T11:29:47.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d2d9d700 1 -- 172.21.15.112:0/1288596904 learned_addr learned my addr 172.21.15.112:0/1288596904 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:47.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d1d9b700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59d414a770 0x7f59d40b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:47.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d259c700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d40cfd00 0x7f59d415bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:47.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d2d9d700 1 -- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59d414a770 msgr2=0x7f59d40b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:47.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d2d9d700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59d414a770 0x7f59d40b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d2d9d700 1 -- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d40cfd00 msgr2=0x7f59d415bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:47.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d2d9d700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d40cfd00 0x7f59d415bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d2d9d700 1 -- 172.21.15.112:0/1288596904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc009a70 con 0x7f59d414aeb0 2023-08-27T11:29:47.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d259c700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d40cfd00 0x7f59d415bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:47.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d2d9d700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d414aeb0 0x7f59d40b0c50 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f59d40ceab0 tx=0x7f59cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:47.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59c2ffd700 1 -- 172.21.15.112:0/1288596904 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc012430 con 0x7f59d414aeb0 2023-08-27T11:29:47.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59c2ffd700 1 -- 172.21.15.112:0/1288596904 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc020070 con 0x7f59d414aeb0 2023-08-27T11:29:47.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d8806700 1 -- 172.21.15.112:0/1288596904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d41454d0 con 0x7f59d414aeb0 2023-08-27T11:29:47.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59c2ffd700 1 -- 172.21.15.112:0/1288596904 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc004280 con 0x7f59d414aeb0 2023-08-27T11:29:47.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.562+0000 7f59d8806700 1 -- 172.21.15.112:0/1288596904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d4145a90 con 0x7f59d414aeb0 2023-08-27T11:29:47.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.563+0000 7f59d8806700 1 -- 172.21.15.112:0/1288596904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d4146290 con 0x7f59d414aeb0 2023-08-27T11:29:47.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.564+0000 7f59c2ffd700 1 -- 172.21.15.112:0/1288596904 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f59cc0043e0 con 0x7f59d414aeb0 2023-08-27T11:29:47.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.564+0000 7f59c2ffd700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59b406a4d0 0x7f59b406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:47.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.564+0000 7f59c2ffd700 1 -- 172.21.15.112:0/1288596904 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f59cc022070 con 0x7f59d414aeb0 2023-08-27T11:29:47.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.564+0000 7f59d259c700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59b406a4d0 0x7f59b406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:47.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.565+0000 7f59d259c700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59b406a4d0 0x7f59b406c990 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f59c4000f60 tx=0x7f59c4007ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:47.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.569+0000 7f59c2ffd700 1 -- 172.21.15.112:0/1288596904 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f59d4146290 con 0x7f59d414aeb0 2023-08-27T11:29:47.788 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.787+0000 7f59d8806700 1 -- 172.21.15.112:0/1288596904 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f59d414ab50 con 0x7f59b406a4d0 2023-08-27T11:29:47.790 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.789+0000 7f59c2ffd700 1 -- 172.21.15.112:0/1288596904 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7f59c4005ba0 con 0x7f59b406a4d0 2023-08-27T11:29:47.790 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:29:47.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:29:47.790 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volume_0000000001022841" 2023-08-27T11:29:47.791 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:29:47.791 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:29:47.792 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.791+0000 7f59d8806700 1 -- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59b406a4d0 msgr2=0x7f59b406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:47.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.791+0000 7f59d8806700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59b406a4d0 0x7f59b406c990 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f59c4000f60 tx=0x7f59c4007ca0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.791+0000 7f59d8806700 1 -- 172.21.15.112:0/1288596904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d414aeb0 msgr2=0x7f59d40b0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:47.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.791+0000 7f59d8806700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d414aeb0 0x7f59d40b0c50 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f59d40ceab0 tx=0x7f59cc004ad0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.791+0000 7f59d8806700 1 -- 172.21.15.112:0/1288596904 shutdown_connections 2023-08-27T11:29:47.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.791+0000 7f59d8806700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f59b406a4d0 0x7f59b406c990 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.791+0000 7f59d8806700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59d414aeb0 0x7f59d40b0c50 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.791+0000 7f59d8806700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d40cfd00 0x7f59d415bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.791+0000 7f59d8806700 1 --2- 172.21.15.112:0/1288596904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59d414a770 0x7f59d40b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.794 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.791+0000 7f59d8806700 1 -- 172.21.15.112:0/1288596904 >> 172.21.15.112:0/1288596904 conn(0x7f59d406f970 msgr2=0x7f59d4149a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:47.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.791+0000 7f59d8806700 1 -- 172.21.15.112:0/1288596904 shutdown_connections 2023-08-27T11:29:47.795 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.791+0000 7f59d8806700 1 -- 172.21.15.112:0/1288596904 wait complete. 2023-08-27T11:29:47.804 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:47.925 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.924+0000 7f8c6ad20700 1 Processor -- start 2023-08-27T11:29:47.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.924+0000 7f8c6ad20700 1 -- start start 2023-08-27T11:29:47.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c6ad20700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8c6406e860 0x7f8c640acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:47.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c6ad20700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c640ad150 0x7f8c640cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:47.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c6ad20700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c64140810 con 0x7f8c64140430 2023-08-27T11:29:47.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c6ad20700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c6415db70 con 0x7f8c6406e860 2023-08-27T11:29:47.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c6ad20700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6415de50 con 0x7f8c640ad150 2023-08-27T11:29:47.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c692bd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c640ad150 0x7f8c640cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:47.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c63fff700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8c6406e860 0x7f8c640acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:47.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c692bd700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c640ad150 0x7f8c640cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:50356/0 (socket says 172.21.15.112:50356) 2023-08-27T11:29:47.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c692bd700 1 -- 172.21.15.112:0/3851547582 learned_addr learned my addr 172.21.15.112:0/3851547582 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:47.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c68abc700 1 --1- 172.21.15.112:0/3851547582 >> v1:172.21.15.138:6789/0 conn(0x7f8c64140430 0x7f8c6406e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:40604/0 (socket says 172.21.15.112:40604) 2023-08-27T11:29:47.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c692bd700 1 -- 172.21.15.112:0/3851547582 >> v2:172.21.15.138:3300/0 conn(0x7f8c6406e860 msgr2=0x7f8c640acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:47.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c692bd700 1 --2- 172.21.15.112:0/3851547582 >> v2:172.21.15.138:3300/0 conn(0x7f8c6406e860 0x7f8c640acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c692bd700 1 -- 172.21.15.112:0/3851547582 >> v1:172.21.15.138:6789/0 conn(0x7f8c64140430 legacy=0x7f8c6406e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:47.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c692bd700 1 -- 172.21.15.112:0/3851547582 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6415e130 con 0x7f8c640ad150 2023-08-27T11:29:47.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.925+0000 7f8c63fff700 1 --2- 172.21.15.112:0/3851547582 >> v2:172.21.15.138:3300/0 conn(0x7f8c6406e860 0x7f8c640acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:47.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.926+0000 7f8c692bd700 1 --2- 172.21.15.112:0/3851547582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c640ad150 0x7f8c640cd740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009e50 tx=0x7f8c5c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fccce2bb6b435225 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:47.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.926+0000 7f8c637fe700 1 -- 172.21.15.112:0/3851547582 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c00e070 con 0x7f8c640ad150 2023-08-27T11:29:47.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.926+0000 7f8c637fe700 1 -- 172.21.15.112:0/3851547582 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c00b910 con 0x7f8c640ad150 2023-08-27T11:29:47.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.926+0000 7f8c637fe700 1 -- 172.21.15.112:0/3851547582 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c00bcc0 con 0x7f8c640ad150 2023-08-27T11:29:47.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.926+0000 7f8c6ad20700 1 -- 172.21.15.112:0/3851547582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c640ad150 msgr2=0x7f8c640cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:47.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.926+0000 7f8c6ad20700 1 --2- 172.21.15.112:0/3851547582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c640ad150 0x7f8c640cd740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009e50 tx=0x7f8c5c00b330 comp rx=0 tx=0).stop 2023-08-27T11:29:47.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.926+0000 7f8c6ad20700 1 -- 172.21.15.112:0/3851547582 shutdown_connections 2023-08-27T11:29:47.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.926+0000 7f8c6ad20700 1 --2- 172.21.15.112:0/3851547582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c640ad150 0x7f8c640cd740 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.926+0000 7f8c6ad20700 1 --2- 172.21.15.112:0/3851547582 >> v2:172.21.15.138:3300/0 conn(0x7f8c6406e860 0x7f8c640acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.926+0000 7f8c6ad20700 1 -- 172.21.15.112:0/3851547582 >> 172.21.15.112:0/3851547582 conn(0x7f8c6406cd30 msgr2=0x7f8c640717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:47.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.926+0000 7f8c6ad20700 1 -- 172.21.15.112:0/3851547582 shutdown_connections 2023-08-27T11:29:47.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.926+0000 7f8c6ad20700 1 -- 172.21.15.112:0/3851547582 wait complete. 2023-08-27T11:29:47.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.928+0000 7f8c6ad20700 1 Processor -- start 2023-08-27T11:29:47.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.928+0000 7f8c6ad20700 1 -- start start 2023-08-27T11:29:47.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.928+0000 7f8c6ad20700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6406e860 0x7f8c6413d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:47.936 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.928+0000 7f8c6ad20700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c640ad150 0x7f8c640b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:47.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.928+0000 7f8c6ad20700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c64140430 0x7f8c64070650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:47.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.928+0000 7f8c6ad20700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c640ce3a0 con 0x7f8c64140430 2023-08-27T11:29:47.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.928+0000 7f8c6ad20700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c64150420 con 0x7f8c640ad150 2023-08-27T11:29:47.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.928+0000 7f8c6ad20700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c64150700 con 0x7f8c6406e860 2023-08-27T11:29:47.937 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c692bd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c64140430 0x7f8c64070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:47.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c692bd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c64140430 0x7f8c64070650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38708/0 (socket says 172.21.15.112:38708) 2023-08-27T11:29:47.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c692bd700 1 -- 172.21.15.112:0/815603969 learned_addr learned my addr 172.21.15.112:0/815603969 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:47.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c63fff700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c640ad150 0x7f8c640b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:47.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c68abc700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6406e860 0x7f8c6413d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:47.938 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c692bd700 1 -- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6406e860 msgr2=0x7f8c6413d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:47.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c692bd700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6406e860 0x7f8c6413d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c692bd700 1 -- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c640ad150 msgr2=0x7f8c640b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:47.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c692bd700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c640ad150 0x7f8c640b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:47.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c692bd700 1 -- 172.21.15.112:0/815603969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c50009430 con 0x7f8c64140430 2023-08-27T11:29:47.939 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c692bd700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c64140430 0x7f8c64070650 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f8c6406ee60 tx=0x7f8c5c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:47.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c617fa700 1 -- 172.21.15.112:0/815603969 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c003ea0 con 0x7f8c64140430 2023-08-27T11:29:47.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c617fa700 1 -- 172.21.15.112:0/815603969 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c0048e0 con 0x7f8c64140430 2023-08-27T11:29:47.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c6ad20700 1 -- 172.21.15.112:0/815603969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5c009b30 con 0x7f8c64140430 2023-08-27T11:29:47.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c617fa700 1 -- 172.21.15.112:0/815603969 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c01c680 con 0x7f8c64140430 2023-08-27T11:29:47.940 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.929+0000 7f8c6ad20700 1 -- 172.21.15.112:0/815603969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c64150cc0 con 0x7f8c64140430 2023-08-27T11:29:47.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.930+0000 7f8c6ad20700 1 -- 172.21.15.112:0/815603969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c640cdc80 con 0x7f8c64140430 2023-08-27T11:29:47.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.931+0000 7f8c617fa700 1 -- 172.21.15.112:0/815603969 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f8c5c01c7e0 con 0x7f8c64140430 2023-08-27T11:29:47.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.931+0000 7f8c617fa700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8c4806a4d0 0x7f8c4806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:47.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.931+0000 7f8c617fa700 1 -- 172.21.15.112:0/815603969 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f8c5c08ab20 con 0x7f8c64140430 2023-08-27T11:29:47.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.931+0000 7f8c68abc700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8c4806a4d0 0x7f8c4806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:47.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.932+0000 7f8c68abc700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8c4806a4d0 0x7f8c4806c990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8c58000f60 tx=0x7f8c58007f10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:47.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:47.935+0000 7f8c617fa700 1 -- 172.21.15.112:0/815603969 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8c5c0559d0 con 0x7f8c64140430 2023-08-27T11:29:48.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.166+0000 7f8c6ad20700 1 -- 172.21.15.112:0/815603969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c64067a20 con 0x7f8c64140430 2023-08-27T11:29:48.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.166+0000 7f8c617fa700 1 -- 172.21.15.112:0/815603969 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4914 (secure 0 0 0) 0x7f8c5c058ff0 con 0x7f8c64140430 2023-08-27T11:29:48.168 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:48.168 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/685371648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":685371648},{"type":"v1","addr":"172.21.15.138:6836","nonce":685371648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":7214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3585152138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3585152138},{"type":"v1","addr":"172.21.15.138:6837","nonce":3585152138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":8203,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2190275630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2190275630},{"type":"v1","addr":"172.21.15.112:6835","nonce":2190275630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:29:22.938324+0000","modified":"2023-08-27T11:29:46.388631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6833/4086629952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4086629952},{"type":"v1","addr":"172.21.15.112:6833","nonce":4086629952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"volume_0000000001022841","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-08-27T11:29:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.168+0000 7f8c6ad20700 1 -- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8c4806a4d0 msgr2=0x7f8c4806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.168+0000 7f8c6ad20700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8c4806a4d0 0x7f8c4806c990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8c58000f60 tx=0x7f8c58007f10 comp rx=0 tx=0).stop 2023-08-27T11:29:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.168+0000 7f8c6ad20700 1 -- 172.21.15.112:0/815603969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c64140430 msgr2=0x7f8c64070650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.168+0000 7f8c6ad20700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c64140430 0x7f8c64070650 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f8c6406ee60 tx=0x7f8c5c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.168+0000 7f8c6ad20700 1 -- 172.21.15.112:0/815603969 shutdown_connections 2023-08-27T11:29:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.168+0000 7f8c6ad20700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8c4806a4d0 0x7f8c4806c990 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.168+0000 7f8c6ad20700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c64140430 0x7f8c64070650 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.168+0000 7f8c6ad20700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6406e860 0x7f8c6413d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.168+0000 7f8c6ad20700 1 --2- 172.21.15.112:0/815603969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c640ad150 0x7f8c640b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.168+0000 7f8c6ad20700 1 -- 172.21.15.112:0/815603969 >> 172.21.15.112:0/815603969 conn(0x7f8c6406cd30 msgr2=0x7f8c6406d110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.169+0000 7f8c6ad20700 1 -- 172.21.15.112:0/815603969 shutdown_connections 2023-08-27T11:29:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.169+0000 7f8c6ad20700 1 -- 172.21.15.112:0/815603969 wait complete. 2023-08-27T11:29:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 116 2023-08-27T11:29:48.182 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:48.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.301+0000 7f84a5448700 1 Processor -- start 2023-08-27T11:29:48.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.301+0000 7f84a5448700 1 -- start start 2023-08-27T11:29:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.302+0000 7f84a5448700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f84a00b3e50 0x7f84a015f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.302+0000 7f84a5448700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84a00b4ad0 0x7f84a00b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.302+0000 7f84a5448700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a00b45f0 con 0x7f84a0144570 2023-08-27T11:29:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.302+0000 7f84a5448700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a00cf7a0 con 0x7f84a00b3e50 2023-08-27T11:29:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.302+0000 7f84a5448700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a0144950 con 0x7f84a00b4ad0 2023-08-27T11:29:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.303+0000 7f849f7fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84a00b4ad0 0x7f84a00b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.303+0000 7f849e7fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f84a00b3e50 0x7f84a015f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.303+0000 7f849e7fc700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f84a00b3e50 0x7f84a015f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:39140/0 (socket says 172.21.15.112:39140) 2023-08-27T11:29:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.303+0000 7f849f7fe700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84a00b4ad0 0x7f84a00b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:50370/0 (socket says 172.21.15.112:50370) 2023-08-27T11:29:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.303+0000 7f849e7fc700 1 -- 172.21.15.112:0/3526715906 learned_addr learned my addr 172.21.15.112:0/3526715906 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.303+0000 7f849effd700 1 --1- 172.21.15.112:0/3526715906 >> v1:172.21.15.138:6789/0 conn(0x7f84a0144570 0x7f84a015ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:40618/0 (socket says 172.21.15.112:40618) 2023-08-27T11:29:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.303+0000 7f849f7fe700 1 -- 172.21.15.112:0/3526715906 >> v2:172.21.15.138:3300/0 conn(0x7f84a00b3e50 msgr2=0x7f84a015f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.303+0000 7f849dffb700 1 -- 172.21.15.112:0/3526715906 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540373247 0 0) 0x7f84a00b45f0 con 0x7f84a0144570 2023-08-27T11:29:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.303+0000 7f849f7fe700 1 --2- 172.21.15.112:0/3526715906 >> v2:172.21.15.138:3300/0 conn(0x7f84a00b3e50 0x7f84a015f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.303+0000 7f849f7fe700 1 -- 172.21.15.112:0/3526715906 >> v1:172.21.15.138:6789/0 conn(0x7f84a0144570 legacy=0x7f84a015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:48.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.303+0000 7f849f7fe700 1 -- 172.21.15.112:0/3526715906 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a00b4310 con 0x7f84a00b4ad0 2023-08-27T11:29:48.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.303+0000 7f849f7fe700 1 --2- 172.21.15.112:0/3526715906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84a00b4ad0 0x7f84a00b4f60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8498009d90 tx=0x7f849800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88ba22c0c598e4ef server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:48.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.304+0000 7f849dffb700 1 -- 172.21.15.112:0/3526715906 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849800e070 con 0x7f84a00b4ad0 2023-08-27T11:29:48.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.304+0000 7f849dffb700 1 -- 172.21.15.112:0/3526715906 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849800bb20 con 0x7f84a00b4ad0 2023-08-27T11:29:48.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.304+0000 7f849dffb700 1 -- 172.21.15.112:0/3526715906 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498012430 con 0x7f84a00b4ad0 2023-08-27T11:29:48.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.304+0000 7f84a5448700 1 -- 172.21.15.112:0/3526715906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84a00b4ad0 msgr2=0x7f84a00b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:48.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.304+0000 7f84a5448700 1 --2- 172.21.15.112:0/3526715906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84a00b4ad0 0x7f84a00b4f60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8498009d90 tx=0x7f849800b330 comp rx=0 tx=0).stop 2023-08-27T11:29:48.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.304+0000 7f84a5448700 1 -- 172.21.15.112:0/3526715906 shutdown_connections 2023-08-27T11:29:48.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.304+0000 7f84a5448700 1 --2- 172.21.15.112:0/3526715906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84a00b4ad0 0x7f84a00b4f60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.304+0000 7f84a5448700 1 --2- 172.21.15.112:0/3526715906 >> v2:172.21.15.138:3300/0 conn(0x7f84a00b3e50 0x7f84a015f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.304+0000 7f84a5448700 1 -- 172.21.15.112:0/3526715906 >> 172.21.15.112:0/3526715906 conn(0x7f84a006f970 msgr2=0x7f84a006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:48.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.304+0000 7f84a5448700 1 -- 172.21.15.112:0/3526715906 shutdown_connections 2023-08-27T11:29:48.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.304+0000 7f84a5448700 1 -- 172.21.15.112:0/3526715906 wait complete. 2023-08-27T11:29:48.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.305+0000 7f84a5448700 1 Processor -- start 2023-08-27T11:29:48.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.305+0000 7f84a5448700 1 -- start start 2023-08-27T11:29:48.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.306+0000 7f84a5448700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84a00b3e50 0x7f84a00c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:48.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.306+0000 7f84a5448700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84a00b4ad0 0x7f84a015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:48.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.306+0000 7f84a5448700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84a0144570 0x7f84a01f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:48.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.306+0000 7f84a5448700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a015aef0 con 0x7f84a00b4ad0 2023-08-27T11:29:48.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.306+0000 7f84a5448700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a015dbb0 con 0x7f84a00b3e50 2023-08-27T11:29:48.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.306+0000 7f84a5448700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a01f3790 con 0x7f84a0144570 2023-08-27T11:29:48.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.306+0000 7f849e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84a00b4ad0 0x7f84a015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:48.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.306+0000 7f849e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84a00b4ad0 0x7f84a015a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38720/0 (socket says 172.21.15.112:38720) 2023-08-27T11:29:48.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.306+0000 7f849e7fc700 1 -- 172.21.15.112:0/2394524951 learned_addr learned my addr 172.21.15.112:0/2394524951 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:48.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.306+0000 7f849f7fe700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84a0144570 0x7f84a01f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:48.315 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.306+0000 7f849effd700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84a00b3e50 0x7f84a00c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:48.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.307+0000 7f849e7fc700 1 -- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84a0144570 msgr2=0x7f84a01f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:48.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.307+0000 7f849e7fc700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84a0144570 0x7f84a01f3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.307+0000 7f849e7fc700 1 -- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84a00b3e50 msgr2=0x7f84a00c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:48.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.307+0000 7f849e7fc700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84a00b3e50 0x7f84a00c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.316 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.307+0000 7f849e7fc700 1 -- 172.21.15.112:0/2394524951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8498009a70 con 0x7f84a00b4ad0 2023-08-27T11:29:48.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.307+0000 7f849e7fc700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84a00b4ad0 0x7f84a015a9b0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f849000e8b0 tx=0x7f849000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:48.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.307+0000 7f84877fe700 1 -- 172.21.15.112:0/2394524951 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8490009600 con 0x7f84a00b4ad0 2023-08-27T11:29:48.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.307+0000 7f84877fe700 1 -- 172.21.15.112:0/2394524951 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8490013680 con 0x7f84a00b4ad0 2023-08-27T11:29:48.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.307+0000 7f84877fe700 1 -- 172.21.15.112:0/2394524951 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8490013a30 con 0x7f84a00b4ad0 2023-08-27T11:29:48.317 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.307+0000 7f84a5448700 1 -- 172.21.15.112:0/2394524951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a015d280 con 0x7f84a00b4ad0 2023-08-27T11:29:48.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.307+0000 7f84a5448700 1 -- 172.21.15.112:0/2394524951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a015d840 con 0x7f84a00b4ad0 2023-08-27T11:29:48.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.308+0000 7f84a5448700 1 -- 172.21.15.112:0/2394524951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a015dd20 con 0x7f84a00b4ad0 2023-08-27T11:29:48.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.308+0000 7f84877fe700 1 -- 172.21.15.112:0/2394524951 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f8490013b90 con 0x7f84a00b4ad0 2023-08-27T11:29:48.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.309+0000 7f84877fe700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f848c06a4d0 0x7f848c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:48.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.309+0000 7f84877fe700 1 -- 172.21.15.112:0/2394524951 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f8490088be0 con 0x7f84a00b4ad0 2023-08-27T11:29:48.318 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.309+0000 7f849effd700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f848c06a4d0 0x7f848c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:48.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.309+0000 7f849effd700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f848c06a4d0 0x7f848c06c990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8494000f60 tx=0x7f8494007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:48.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.313+0000 7f84877fe700 1 -- 172.21.15.112:0/2394524951 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8490057630 con 0x7f84a00b4ad0 2023-08-27T11:29:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.505+0000 7f84a5448700 1 -- 172.21.15.112:0/2394524951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84a0067a20 con 0x7f84a00b4ad0 2023-08-27T11:29:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.506+0000 7f84877fe700 1 -- 172.21.15.112:0/2394524951 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+20116 (secure 0 0 0) 0x7f8490021020 con 0x7f84a00b4ad0 2023-08-27T11:29:48.508 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:48.508 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":172,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:47.398430+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs.volume_0000000001022841.meta","create_time":"2023-08-27T11:29:20.569324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000001022841"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-08-27T11:29:21.905812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000001022841"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"another-data-pool","create_time":"2023-08-27T11:29:43.100842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000001022841"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:48.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.508+0000 7f84a5448700 1 -- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f848c06a4d0 msgr2=0x7f848c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:48.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.508+0000 7f84a5448700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f848c06a4d0 0x7f848c06c990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8494000f60 tx=0x7f8494007de0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.508+0000 7f84a5448700 1 -- 172.21.15.112:0/2394524951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84a00b4ad0 msgr2=0x7f84a015a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:48.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.508+0000 7f84a5448700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84a00b4ad0 0x7f84a015a9b0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f849000e8b0 tx=0x7f849000ebc0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.509+0000 7f84a5448700 1 -- 172.21.15.112:0/2394524951 shutdown_connections 2023-08-27T11:29:48.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.509+0000 7f84a5448700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f848c06a4d0 0x7f848c06c990 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.509+0000 7f84a5448700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84a0144570 0x7f84a01f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.509+0000 7f84a5448700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84a00b3e50 0x7f84a00c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.509+0000 7f84a5448700 1 --2- 172.21.15.112:0/2394524951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84a00b4ad0 0x7f84a015a9b0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.509+0000 7f84a5448700 1 -- 172.21.15.112:0/2394524951 >> 172.21.15.112:0/2394524951 conn(0x7f84a006f970 msgr2=0x7f84a00b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:48.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.509+0000 7f84a5448700 1 -- 172.21.15.112:0/2394524951 shutdown_connections 2023-08-27T11:29:48.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.509+0000 7f84a5448700 1 -- 172.21.15.112:0/2394524951 wait complete. 2023-08-27T11:29:48.522 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:29:48.523 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:29:48.562 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:29:48.611 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:29:48.611 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:29:48.612 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:48.629 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:29:48.630 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:29:48.630 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:48.650 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:29:48.651 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:29:48.651 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:48.686 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:29:48.686 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:29:48.687 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:48.711 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:29:48.711 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:48.845 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.844+0000 7f850721b700 1 Processor -- start 2023-08-27T11:29:48.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.844+0000 7f850721b700 1 -- start start 2023-08-27T11:29:48.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.845+0000 7f850721b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8500144570 0x7f850015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:48.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.845+0000 7f850721b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f85000b4ad0 0x7f85000b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:48.848 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.845+0000 7f850721b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85000b45f0 con 0x7f85000b3e50 2023-08-27T11:29:48.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.845+0000 7f850721b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85000cf7a0 con 0x7f85000b4ad0 2023-08-27T11:29:48.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.845+0000 7f850721b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8500144950 con 0x7f8500144570 2023-08-27T11:29:48.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.845+0000 7f85057b8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f85000b4ad0 0x7f85000b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:48.849 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.845+0000 7f84fffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f85000b3e50 0x7f850015f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33108/0 (socket says 172.21.15.112:33108) 2023-08-27T11:29:48.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.845+0000 7f85057b8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f85000b4ad0 0x7f85000b4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38730/0 (socket says 172.21.15.112:38730) 2023-08-27T11:29:48.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.845+0000 7f84fffff700 1 -- 172.21.15.112:0/1017436348 learned_addr learned my addr 172.21.15.112:0/1017436348 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:48.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.845+0000 7f8504fb7700 1 --2- 172.21.15.112:0/1017436348 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8500144570 0x7f850015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:48.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f85057b8700 1 -- 172.21.15.112:0/1017436348 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8500144570 msgr2=0x7f850015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:48.850 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f85057b8700 1 --2- 172.21.15.112:0/1017436348 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8500144570 0x7f850015ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f85057b8700 1 -- 172.21.15.112:0/1017436348 >> v1:172.21.15.112:6789/0 conn(0x7f85000b3e50 legacy=0x7f850015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:48.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f85057b8700 1 -- 172.21.15.112:0/1017436348 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85000b4310 con 0x7f85000b4ad0 2023-08-27T11:29:48.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f84ff7fe700 1 -- 172.21.15.112:0/1017436348 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090148412 0 0) 0x7f85000b45f0 con 0x7f85000b3e50 2023-08-27T11:29:48.851 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f85057b8700 1 --2- 172.21.15.112:0/1017436348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85000b4ad0 0x7f85000b4f80 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009dc0 tx=0x7f84f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271684e45283c14c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:48.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f84ff7fe700 1 -- 172.21.15.112:0/1017436348 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f800e070 con 0x7f85000b4ad0 2023-08-27T11:29:48.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f84ff7fe700 1 -- 172.21.15.112:0/1017436348 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f800bb90 con 0x7f85000b4ad0 2023-08-27T11:29:48.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f84ff7fe700 1 -- 172.21.15.112:0/1017436348 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8012430 con 0x7f85000b4ad0 2023-08-27T11:29:48.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f850721b700 1 -- 172.21.15.112:0/1017436348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85000b4ad0 msgr2=0x7f85000b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:48.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f850721b700 1 --2- 172.21.15.112:0/1017436348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85000b4ad0 0x7f85000b4f80 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009dc0 tx=0x7f84f800b330 comp rx=0 tx=0).stop 2023-08-27T11:29:48.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f850721b700 1 -- 172.21.15.112:0/1017436348 shutdown_connections 2023-08-27T11:29:48.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f850721b700 1 --2- 172.21.15.112:0/1017436348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85000b4ad0 0x7f85000b4f80 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f850721b700 1 --2- 172.21.15.112:0/1017436348 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8500144570 0x7f850015ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f850721b700 1 -- 172.21.15.112:0/1017436348 >> 172.21.15.112:0/1017436348 conn(0x7f850006f970 msgr2=0x7f850006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:48.853 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f850721b700 1 -- 172.21.15.112:0/1017436348 shutdown_connections 2023-08-27T11:29:48.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.846+0000 7f850721b700 1 -- 172.21.15.112:0/1017436348 wait complete. 2023-08-27T11:29:48.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.848+0000 7f850721b700 1 Processor -- start 2023-08-27T11:29:48.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.848+0000 7f850721b700 1 -- start start 2023-08-27T11:29:48.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.848+0000 7f850721b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85000b3e50 0x7f85001359f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:48.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.848+0000 7f850721b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85000b4ad0 0x7f8500139f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:48.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.848+0000 7f850721b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8500144570 0x7f8500138ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:48.857 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.848+0000 7f850721b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850013b4b0 con 0x7f85000b4ad0 2023-08-27T11:29:48.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.848+0000 7f850721b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85001f0b20 con 0x7f8500144570 2023-08-27T11:29:48.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.848+0000 7f850721b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850013a830 con 0x7f85000b3e50 2023-08-27T11:29:48.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f84fffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85000b4ad0 0x7f8500139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:48.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f84fffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85000b4ad0 0x7f8500139f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38734/0 (socket says 172.21.15.112:38734) 2023-08-27T11:29:48.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f84fffff700 1 -- 172.21.15.112:0/4267962116 learned_addr learned my addr 172.21.15.112:0/4267962116 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:48.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f8504fb7700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85000b3e50 0x7f85001359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:48.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f85057b8700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8500144570 0x7f8500138ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:48.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f84fffff700 1 -- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85000b3e50 msgr2=0x7f85001359f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:48.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f84fffff700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85000b3e50 0x7f85001359f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f84fffff700 1 -- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8500144570 msgr2=0x7f8500138ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:48.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f84fffff700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8500144570 0x7f8500138ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:48.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f84fffff700 1 -- 172.21.15.112:0/4267962116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f8009a70 con 0x7f85000b4ad0 2023-08-27T11:29:48.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f85057b8700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8500144570 0x7f8500138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:48.860 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f84fffff700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85000b4ad0 0x7f8500139f50 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0011e0 tx=0x7f84ec00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:48.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f84fd7fa700 1 -- 172.21.15.112:0/4267962116 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec004c80 con 0x7f85000b4ad0 2023-08-27T11:29:48.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f84fd7fa700 1 -- 172.21.15.112:0/4267962116 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec004e00 con 0x7f85000b4ad0 2023-08-27T11:29:48.861 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f850721b700 1 -- 172.21.15.112:0/4267962116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850013ab10 con 0x7f85000b4ad0 2023-08-27T11:29:48.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f84fd7fa700 1 -- 172.21.15.112:0/4267962116 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec007310 con 0x7f85000b4ad0 2023-08-27T11:29:48.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.849+0000 7f850721b700 1 -- 172.21.15.112:0/4267962116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850013b0d0 con 0x7f85000b4ad0 2023-08-27T11:29:48.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.850+0000 7f850721b700 1 -- 172.21.15.112:0/4267962116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850013b5f0 con 0x7f85000b4ad0 2023-08-27T11:29:48.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.851+0000 7f84fd7fa700 1 -- 172.21.15.112:0/4267962116 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f84ec007470 con 0x7f85000b4ad0 2023-08-27T11:29:48.862 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.851+0000 7f84fd7fa700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f84e406a4d0 0x7f84e406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:48.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.851+0000 7f84fd7fa700 1 -- 172.21.15.112:0/4267962116 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f84ec088a30 con 0x7f85000b4ad0 2023-08-27T11:29:48.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.851+0000 7f8504fb7700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f84e406a4d0 0x7f84e406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:48.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.852+0000 7f8504fb7700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f84e406a4d0 0x7f84e406c990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f84f400b9b0 tx=0x7f84f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:48.863 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:48.855+0000 7f84fd7fa700 1 -- 172.21.15.112:0/4267962116 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f850013b5f0 con 0x7f85000b4ad0 2023-08-27T11:29:49.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.084+0000 7f850721b700 1 -- 172.21.15.112:0/4267962116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f850013b5f0 con 0x7f85000b4ad0 2023-08-27T11:29:49.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.085+0000 7f84fd7fa700 1 -- 172.21.15.112:0/4267962116 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4914 (secure 0 0 0) 0x7f850013b5f0 con 0x7f85000b4ad0 2023-08-27T11:29:49.087 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:49.087 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/685371648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":685371648},{"type":"v1","addr":"172.21.15.138:6836","nonce":685371648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":7214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3585152138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3585152138},{"type":"v1","addr":"172.21.15.138:6837","nonce":3585152138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":8203,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2190275630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2190275630},{"type":"v1","addr":"172.21.15.112:6835","nonce":2190275630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:29:22.938324+0000","modified":"2023-08-27T11:29:46.388631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6833/4086629952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4086629952},{"type":"v1","addr":"172.21.15.112:6833","nonce":4086629952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"volume_0000000001022841","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-08-27T11:29:49.088 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.087+0000 7f850721b700 1 -- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f84e406a4d0 msgr2=0x7f84e406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.087+0000 7f850721b700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f84e406a4d0 0x7f84e406c990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f84f400b9b0 tx=0x7f84f4008040 comp rx=0 tx=0).stop 2023-08-27T11:29:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.087+0000 7f850721b700 1 -- 172.21.15.112:0/4267962116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85000b4ad0 msgr2=0x7f8500139f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.087+0000 7f850721b700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85000b4ad0 0x7f8500139f50 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0011e0 tx=0x7f84ec00cae0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.087+0000 7f850721b700 1 -- 172.21.15.112:0/4267962116 shutdown_connections 2023-08-27T11:29:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.087+0000 7f850721b700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f84e406a4d0 0x7f84e406c990 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.087+0000 7f850721b700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8500144570 0x7f8500138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.087+0000 7f850721b700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85000b3e50 0x7f85001359f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.087+0000 7f850721b700 1 --2- 172.21.15.112:0/4267962116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85000b4ad0 0x7f8500139f50 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.087+0000 7f850721b700 1 -- 172.21.15.112:0/4267962116 >> 172.21.15.112:0/4267962116 conn(0x7f850006f970 msgr2=0x7f850013ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.087+0000 7f850721b700 1 -- 172.21.15.112:0/4267962116 shutdown_connections 2023-08-27T11:29:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.087+0000 7f850721b700 1 -- 172.21.15.112:0/4267962116 wait complete. 2023-08-27T11:29:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 116 2023-08-27T11:29:49.101 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:49.220 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.219+0000 7f832ab44700 1 Processor -- start 2023-08-27T11:29:49.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.219+0000 7f832ab44700 1 -- start start 2023-08-27T11:29:49.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.220+0000 7f832ab44700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8324158e20 0x7f83240b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.220+0000 7f832ab44700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832406e3a0 0x7f832406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.220+0000 7f832ab44700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832415c190 con 0x7f832406fcd0 2023-08-27T11:29:49.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.220+0000 7f832ab44700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83240b1730 con 0x7f8324158e20 2023-08-27T11:29:49.225 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.220+0000 7f832ab44700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8324070160 con 0x7f832406e3a0 2023-08-27T11:29:49.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.220+0000 7f83290e1700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f832406fcd0 0x7f832415ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33126/0 (socket says 172.21.15.112:33126) 2023-08-27T11:29:49.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.220+0000 7f83290e1700 1 -- 172.21.15.112:0/2324880856 learned_addr learned my addr 172.21.15.112:0/2324880856 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:49.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.220+0000 7f83288e0700 1 --2- 172.21.15.112:0/2324880856 >> v2:172.21.15.138:3300/0 conn(0x7f8324158e20 0x7f83240b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.220+0000 7f8323fff700 1 --2- 172.21.15.112:0/2324880856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832406e3a0 0x7f832406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.226 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.220+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957653160 0 0) 0x7f832415c190 con 0x7f832406fcd0 2023-08-27T11:29:49.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.220+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83140035b0 con 0x7f832406fcd0 2023-08-27T11:29:49.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1632257706 0 0) 0x7f83140035b0 con 0x7f832406fcd0 2023-08-27T11:29:49.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832406e3a0 msgr2=0x7f832406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:49.227 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 --2- 172.21.15.112:0/2324880856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832406e3a0 0x7f832406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 >> v2:172.21.15.138:3300/0 conn(0x7f8324158e20 msgr2=0x7f83240b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:49.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 --2- 172.21.15.112:0/2324880856 >> v2:172.21.15.138:3300/0 conn(0x7f8324158e20 0x7f83240b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240b09d0 con 0x7f832406fcd0 2023-08-27T11:29:49.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f831c0019f0 con 0x7f832406fcd0 2023-08-27T11:29:49.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 >> v1:172.21.15.112:6789/0 conn(0x7f832406fcd0 legacy=0x7f832415ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.228 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 --2- 172.21.15.112:0/2324880856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8314003a00 0x7f8314003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 --2- 172.21.15.112:0/2324880856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832406e3a0 0x7f8314004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 --2- 172.21.15.112:0/2324880856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8314004d90 0x7f8314005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240b1730 con 0x7f8314003a00 2023-08-27T11:29:49.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8324070160 con 0x7f8314004d90 2023-08-27T11:29:49.229 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831400ba10 con 0x7f832406e3a0 2023-08-27T11:29:49.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83288e0700 1 --2- 172.21.15.112:0/2324880856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8314003a00 0x7f8314003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f8323fff700 1 --2- 172.21.15.112:0/2324880856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832406e3a0 0x7f8314004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83290e1700 1 --2- 172.21.15.112:0/2324880856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8314004d90 0x7f8314005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83288e0700 1 -- 172.21.15.112:0/2324880856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832406e3a0 msgr2=0x7f8314004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:49.230 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83288e0700 1 --2- 172.21.15.112:0/2324880856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832406e3a0 0x7f8314004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83288e0700 1 -- 172.21.15.112:0/2324880856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8314004d90 msgr2=0x7f8314005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:49.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83288e0700 1 --2- 172.21.15.112:0/2324880856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8314004d90 0x7f8314005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83288e0700 1 -- 172.21.15.112:0/2324880856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83140035b0 con 0x7f8314003a00 2023-08-27T11:29:49.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83288e0700 1 --2- 172.21.15.112:0/2324880856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8314003a00 0x7f8314003e20 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f8318009b60 tx=0x7f8314005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c2fad650bb0d23 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:49.231 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831800e6c0 con 0x7f8314003a00 2023-08-27T11:29:49.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831800e820 con 0x7f8314003a00 2023-08-27T11:29:49.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.221+0000 7f83237fe700 1 -- 172.21.15.112:0/2324880856 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831800eb90 con 0x7f8314003a00 2023-08-27T11:29:49.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.222+0000 7f832ab44700 1 -- 172.21.15.112:0/2324880856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8314003a00 msgr2=0x7f8314003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.232 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.222+0000 7f832ab44700 1 --2- 172.21.15.112:0/2324880856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8314003a00 0x7f8314003e20 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f8318009b60 tx=0x7f8314005880 comp rx=0 tx=0).stop 2023-08-27T11:29:49.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.222+0000 7f83290e1700 1 -- 172.21.15.112:0/2324880856 reap_dead start 2023-08-27T11:29:49.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.222+0000 7f832ab44700 1 -- 172.21.15.112:0/2324880856 shutdown_connections 2023-08-27T11:29:49.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.222+0000 7f832ab44700 1 -- 172.21.15.112:0/2324880856 >> 172.21.15.112:0/2324880856 conn(0x7f83240d0000 msgr2=0x7f83240d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:49.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.222+0000 7f832ab44700 1 -- 172.21.15.112:0/2324880856 shutdown_connections 2023-08-27T11:29:49.233 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.222+0000 7f832ab44700 1 -- 172.21.15.112:0/2324880856 wait complete. 2023-08-27T11:29:49.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.224+0000 7f832ab44700 1 Processor -- start 2023-08-27T11:29:49.234 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.224+0000 7f832ab44700 1 -- start start 2023-08-27T11:29:49.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f832ab44700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8314003a00 0x7f8324151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.236 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f832ab44700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832415d800 0x7f83240708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f832ab44700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832415dc80 0x7f8324070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f832ab44700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832415c5f0 con 0x7f832415dc80 2023-08-27T11:29:49.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f832ab44700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832415c8d0 con 0x7f832415d800 2023-08-27T11:29:49.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f832ab44700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832415cbb0 con 0x7f8314003a00 2023-08-27T11:29:49.237 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f83290e1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832415dc80 0x7f8324070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f83290e1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832415dc80 0x7f8324070de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38756/0 (socket says 172.21.15.112:38756) 2023-08-27T11:29:49.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f83290e1700 1 -- 172.21.15.112:0/4238504365 learned_addr learned my addr 172.21.15.112:0/4238504365 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:49.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f83288e0700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8314003a00 0x7f8324151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f8323fff700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832415d800 0x7f83240708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f83290e1700 1 -- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8314003a00 msgr2=0x7f8324151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f83290e1700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8314003a00 0x7f8324151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f83290e1700 1 -- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832415d800 msgr2=0x7f83240708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f83290e1700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832415d800 0x7f83240708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f83290e1700 1 -- 172.21.15.112:0/4238504365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832415ce90 con 0x7f832415dc80 2023-08-27T11:29:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f83290e1700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832415dc80 0x7f8324070de0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f83240d0e20 tx=0x7f831c007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f83217fa700 1 -- 172.21.15.112:0/4238504365 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00ed90 con 0x7f832415dc80 2023-08-27T11:29:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f83217fa700 1 -- 172.21.15.112:0/4238504365 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c013420 con 0x7f832415dc80 2023-08-27T11:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f832ab44700 1 -- 172.21.15.112:0/4238504365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832415d170 con 0x7f832415dc80 2023-08-27T11:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.225+0000 7f83217fa700 1 -- 172.21.15.112:0/4238504365 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c0137d0 con 0x7f832415dc80 2023-08-27T11:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.226+0000 7f832ab44700 1 -- 172.21.15.112:0/4238504365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832415e5b0 con 0x7f832415dc80 2023-08-27T11:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.226+0000 7f832ab44700 1 -- 172.21.15.112:0/4238504365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8324067a20 con 0x7f832415dc80 2023-08-27T11:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.227+0000 7f83217fa700 1 -- 172.21.15.112:0/4238504365 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f831c013930 con 0x7f832415dc80 2023-08-27T11:29:49.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.227+0000 7f83217fa700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f830806a4d0 0x7f830806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.227+0000 7f83217fa700 1 -- 172.21.15.112:0/4238504365 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f831c0895a0 con 0x7f832415dc80 2023-08-27T11:29:49.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.227+0000 7f83288e0700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f830806a4d0 0x7f830806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.228+0000 7f83288e0700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f830806a4d0 0x7f830806c990 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f83180090b0 tx=0x7f8318008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:49.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.232+0000 7f83217fa700 1 -- 172.21.15.112:0/4238504365 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f831c054490 con 0x7f832415dc80 2023-08-27T11:29:49.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.423+0000 7f832ab44700 1 -- 172.21.15.112:0/4238504365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83240b20e0 con 0x7f832415dc80 2023-08-27T11:29:49.425 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.424+0000 7f83217fa700 1 -- 172.21.15.112:0/4238504365 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+20116 (secure 0 0 0) 0x7f831c05d020 con 0x7f832415dc80 2023-08-27T11:29:49.426 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:49.426 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":172,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:47.398430+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs.volume_0000000001022841.meta","create_time":"2023-08-27T11:29:20.569324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000001022841"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-08-27T11:29:21.905812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000001022841"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"another-data-pool","create_time":"2023-08-27T11:29:43.100842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000001022841"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:49.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.426+0000 7f832ab44700 1 -- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f830806a4d0 msgr2=0x7f830806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.426+0000 7f832ab44700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f830806a4d0 0x7f830806c990 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f83180090b0 tx=0x7f8318008030 comp rx=0 tx=0).stop 2023-08-27T11:29:49.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.426+0000 7f832ab44700 1 -- 172.21.15.112:0/4238504365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832415dc80 msgr2=0x7f8324070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.426+0000 7f832ab44700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832415dc80 0x7f8324070de0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f83240d0e20 tx=0x7f831c007c90 comp rx=0 tx=0).stop 2023-08-27T11:29:49.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.426+0000 7f832ab44700 1 -- 172.21.15.112:0/4238504365 shutdown_connections 2023-08-27T11:29:49.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.426+0000 7f832ab44700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f830806a4d0 0x7f830806c990 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.426+0000 7f832ab44700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832415dc80 0x7f8324070de0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.426+0000 7f832ab44700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8314003a00 0x7f8324151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.426+0000 7f832ab44700 1 --2- 172.21.15.112:0/4238504365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832415d800 0x7f83240708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.426+0000 7f832ab44700 1 -- 172.21.15.112:0/4238504365 >> 172.21.15.112:0/4238504365 conn(0x7f83240d0000 msgr2=0x7f832415a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:49.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.426+0000 7f832ab44700 1 -- 172.21.15.112:0/4238504365 shutdown_connections 2023-08-27T11:29:49.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.426+0000 7f832ab44700 1 -- 172.21.15.112:0/4238504365 wait complete. 2023-08-27T11:29:49.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs.volume_0000000001022841.meta', 'create_time': '2023-08-27T11:29:20.569324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000001022841'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:29:21.905812+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000001022841'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'another-data-pool', 'create_time': '2023-08-27T11:29:43.100842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'volume_0000000001022841'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:29:49.441 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:29:49.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.563+0000 7fbbaf94c700 1 Processor -- start 2023-08-27T11:29:49.564 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.563+0000 7fbbaf94c700 1 -- start start 2023-08-27T11:29:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbbaf94c700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbba81437c0 0x7fbba80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbbaf94c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbba8143ba0 0x7fbba80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbbaf94c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba80cfd50 con 0x7fbba806d5b0 2023-08-27T11:29:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbbaf94c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba80cf140 con 0x7fbba8143ba0 2023-08-27T11:29:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbbaf94c700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba80cf420 con 0x7fbba81437c0 2023-08-27T11:29:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbbacee7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbba8143ba0 0x7fbba80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbbadee9700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbba806d5b0 0x7fbba815dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33146/0 (socket says 172.21.15.112:33146) 2023-08-27T11:29:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbbadee9700 1 -- 172.21.15.112:0/3393168323 learned_addr learned my addr 172.21.15.112:0/3393168323 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbbad6e8700 1 --2- 172.21.15.112:0/3393168323 >> v2:172.21.15.138:3300/0 conn(0x7fbba81437c0 0x7fbba80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbb9ffff700 1 -- 172.21.15.112:0/3393168323 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289323723 0 0) 0x7fbba80cfd50 con 0x7fbba806d5b0 2023-08-27T11:29:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbb9ffff700 1 -- 172.21.15.112:0/3393168323 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb940035b0 con 0x7fbba806d5b0 2023-08-27T11:29:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbbacee7700 1 -- 172.21.15.112:0/3393168323 >> v2:172.21.15.138:3300/0 conn(0x7fbba81437c0 msgr2=0x7fbba80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbbacee7700 1 --2- 172.21.15.112:0/3393168323 >> v2:172.21.15.138:3300/0 conn(0x7fbba81437c0 0x7fbba80b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbbacee7700 1 -- 172.21.15.112:0/3393168323 >> v1:172.21.15.112:6789/0 conn(0x7fbba806d5b0 legacy=0x7fbba815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.564+0000 7fbbacee7700 1 -- 172.21.15.112:0/3393168323 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba80cf700 con 0x7fbba8143ba0 2023-08-27T11:29:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.565+0000 7fbbacee7700 1 --2- 172.21.15.112:0/3393168323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba8143ba0 0x7fbba80ce860 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fbba806e3d0 tx=0x7fbb98009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bdc3ecbc1f4a8c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.565+0000 7fbb9ffff700 1 -- 172.21.15.112:0/3393168323 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98011830 con 0x7fbba8143ba0 2023-08-27T11:29:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.565+0000 7fbb9ffff700 1 -- 172.21.15.112:0/3393168323 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb98011990 con 0x7fbba8143ba0 2023-08-27T11:29:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.565+0000 7fbb9ffff700 1 -- 172.21.15.112:0/3393168323 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98011d20 con 0x7fbba8143ba0 2023-08-27T11:29:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.565+0000 7fbbaf94c700 1 -- 172.21.15.112:0/3393168323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba8143ba0 msgr2=0x7fbba80ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.565+0000 7fbbaf94c700 1 --2- 172.21.15.112:0/3393168323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba8143ba0 0x7fbba80ce860 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fbba806e3d0 tx=0x7fbb98009b20 comp rx=0 tx=0).stop 2023-08-27T11:29:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.565+0000 7fbbaf94c700 1 -- 172.21.15.112:0/3393168323 shutdown_connections 2023-08-27T11:29:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.565+0000 7fbbaf94c700 1 --2- 172.21.15.112:0/3393168323 >> v2:172.21.15.138:3300/0 conn(0x7fbba81437c0 0x7fbba80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.565+0000 7fbbaf94c700 1 --2- 172.21.15.112:0/3393168323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba8143ba0 0x7fbba80ce860 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.565+0000 7fbbaf94c700 1 -- 172.21.15.112:0/3393168323 >> 172.21.15.112:0/3393168323 conn(0x7fbba80b1e90 msgr2=0x7fbba80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.565+0000 7fbbaf94c700 1 -- 172.21.15.112:0/3393168323 shutdown_connections 2023-08-27T11:29:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.565+0000 7fbbaf94c700 1 -- 172.21.15.112:0/3393168323 wait complete. 2023-08-27T11:29:49.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.567+0000 7fbbaf94c700 1 Processor -- start 2023-08-27T11:29:49.574 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.567+0000 7fbbaf94c700 1 -- start start 2023-08-27T11:29:49.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbaf94c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba806d5b0 0x7fbba81f0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbaf94c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbba81437c0 0x7fbba81f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbaf94c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbba8143ba0 0x7fbba813a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.575 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbaf94c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba80c5300 con 0x7fbba806d5b0 2023-08-27T11:29:49.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbaf94c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba813b770 con 0x7fbba8143ba0 2023-08-27T11:29:49.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbaf94c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba813ba50 con 0x7fbba81437c0 2023-08-27T11:29:49.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbad6e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba806d5b0 0x7fbba81f0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbad6e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba806d5b0 0x7fbba81f0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38766/0 (socket says 172.21.15.112:38766) 2023-08-27T11:29:49.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbad6e8700 1 -- 172.21.15.112:0/2018579421 learned_addr learned my addr 172.21.15.112:0/2018579421 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:49.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbacee7700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbba81437c0 0x7fbba81f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbadee9700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbba8143ba0 0x7fbba813a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbad6e8700 1 -- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbba81437c0 msgr2=0x7fbba81f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbad6e8700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbba81437c0 0x7fbba81f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbad6e8700 1 -- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbba8143ba0 msgr2=0x7fbba813a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbad6e8700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbba8143ba0 0x7fbba813a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbad6e8700 1 -- 172.21.15.112:0/2018579421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb980094a0 con 0x7fbba806d5b0 2023-08-27T11:29:49.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbadee9700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbba8143ba0 0x7fbba813a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:49.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbacee7700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbba81437c0 0x7fbba81f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:49.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.568+0000 7fbbad6e8700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba806d5b0 0x7fbba81f0530 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fbba400e9b0 tx=0x7fbba400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:49.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.569+0000 7fbb9dffb700 1 -- 172.21.15.112:0/2018579421 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4009710 con 0x7fbba806d5b0 2023-08-27T11:29:49.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.569+0000 7fbb9dffb700 1 -- 172.21.15.112:0/2018579421 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba400c850 con 0x7fbba806d5b0 2023-08-27T11:29:49.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.569+0000 7fbb9dffb700 1 -- 172.21.15.112:0/2018579421 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400cbc0 con 0x7fbba806d5b0 2023-08-27T11:29:49.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.569+0000 7fbbaf94c700 1 -- 172.21.15.112:0/2018579421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba813bd30 con 0x7fbba806d5b0 2023-08-27T11:29:49.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.569+0000 7fbbaf94c700 1 -- 172.21.15.112:0/2018579421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba813c2f0 con 0x7fbba806d5b0 2023-08-27T11:29:49.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.570+0000 7fbbaf94c700 1 -- 172.21.15.112:0/2018579421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba813c660 con 0x7fbba806d5b0 2023-08-27T11:29:49.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.570+0000 7fbb9dffb700 1 -- 172.21.15.112:0/2018579421 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fbba400cd20 con 0x7fbba806d5b0 2023-08-27T11:29:49.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.570+0000 7fbb9dffb700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbb8c06a4d0 0x7fbb8c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.571+0000 7fbb9dffb700 1 -- 172.21.15.112:0/2018579421 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7fbba4089030 con 0x7fbba806d5b0 2023-08-27T11:29:49.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.571+0000 7fbbacee7700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbb8c06a4d0 0x7fbb8c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.571+0000 7fbbacee7700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbb8c06a4d0 0x7fbb8c06c990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fbba815d830 tx=0x7fbb98008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:49.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.575+0000 7fbb9dffb700 1 -- 172.21.15.112:0/2018579421 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbba40bb990 con 0x7fbba806d5b0 2023-08-27T11:29:49.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.764+0000 7fbbaf94c700 1 -- 172.21.15.112:0/2018579421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbba8067e90 con 0x7fbba806d5b0 2023-08-27T11:29:49.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.764+0000 7fbb9dffb700 1 -- 172.21.15.112:0/2018579421 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v172) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbba4021020 con 0x7fbba806d5b0 2023-08-27T11:29:49.766 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:29:49.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.766+0000 7fbbaf94c700 1 -- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbb8c06a4d0 msgr2=0x7fbb8c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.766+0000 7fbbaf94c700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbb8c06a4d0 0x7fbb8c06c990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fbba815d830 tx=0x7fbb98008030 comp rx=0 tx=0).stop 2023-08-27T11:29:49.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.766+0000 7fbbaf94c700 1 -- 172.21.15.112:0/2018579421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba806d5b0 msgr2=0x7fbba81f0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.766+0000 7fbbaf94c700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba806d5b0 0x7fbba81f0530 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fbba400e9b0 tx=0x7fbba400ed70 comp rx=0 tx=0).stop 2023-08-27T11:29:49.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.766+0000 7fbbaf94c700 1 -- 172.21.15.112:0/2018579421 shutdown_connections 2023-08-27T11:29:49.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.766+0000 7fbbaf94c700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbb8c06a4d0 0x7fbb8c06c990 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.766+0000 7fbbaf94c700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbba8143ba0 0x7fbba813a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.766+0000 7fbbaf94c700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba806d5b0 0x7fbba81f0530 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.766+0000 7fbbaf94c700 1 --2- 172.21.15.112:0/2018579421 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbba81437c0 0x7fbba81f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.766+0000 7fbbaf94c700 1 -- 172.21.15.112:0/2018579421 >> 172.21.15.112:0/2018579421 conn(0x7fbba80b1e90 msgr2=0x7fbba8159980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:49.769 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.766+0000 7fbbaf94c700 1 -- 172.21.15.112:0/2018579421 shutdown_connections 2023-08-27T11:29:49.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.766+0000 7fbbaf94c700 1 -- 172.21.15.112:0/2018579421 wait complete. 2023-08-27T11:29:49.779 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:29:49.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.900+0000 7f4e50de1700 1 Processor -- start 2023-08-27T11:29:49.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.900+0000 7f4e50de1700 1 -- start start 2023-08-27T11:29:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.901+0000 7f4e50de1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4e4c144900 0x7f4e4c0b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.901+0000 7f4e50de1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4e4c0cca90 0x7f4e4c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.901+0000 7f4e50de1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c15ee90 con 0x7f4e4c06f790 2023-08-27T11:29:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.901+0000 7f4e50de1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c0b11d0 con 0x7f4e4c0cca90 2023-08-27T11:29:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.901+0000 7f4e50de1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c0b14b0 con 0x7f4e4c144900 2023-08-27T11:29:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e49d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4e4c0cca90 0x7f4e4c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e4ad9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4e4c06f790 0x7f4e4c15b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33156/0 (socket says 172.21.15.112:33156) 2023-08-27T11:29:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e4ad9d700 1 -- 172.21.15.112:0/2537154214 learned_addr learned my addr 172.21.15.112:0/2537154214 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e4a59c700 1 --2- 172.21.15.112:0/2537154214 >> v2:172.21.15.138:3300/0 conn(0x7f4e4c144900 0x7f4e4c0b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e49d9b700 1 -- 172.21.15.112:0/2537154214 >> v2:172.21.15.138:3300/0 conn(0x7f4e4c144900 msgr2=0x7f4e4c0b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:49.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e49d9b700 1 --2- 172.21.15.112:0/2537154214 >> v2:172.21.15.138:3300/0 conn(0x7f4e4c144900 0x7f4e4c0b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e49d9b700 1 -- 172.21.15.112:0/2537154214 >> v1:172.21.15.112:6789/0 conn(0x7f4e4c06f790 legacy=0x7f4e4c15b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e4959a700 1 -- 172.21.15.112:0/2537154214 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72208823 0 0) 0x7f4e4c15ee90 con 0x7f4e4c06f790 2023-08-27T11:29:49.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e49d9b700 1 -- 172.21.15.112:0/2537154214 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c15f130 con 0x7f4e4c0cca90 2023-08-27T11:29:49.907 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e4a59c700 1 --2- 172.21.15.112:0/2537154214 >> v2:172.21.15.138:3300/0 conn(0x7f4e4c144900 0x7f4e4c0b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e49d9b700 1 --2- 172.21.15.112:0/2537154214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e4c0cca90 0x7f4e4c15f650 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c06ea50 tx=0x7f4e3c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836505dead36e023 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e4959a700 1 -- 172.21.15.112:0/2537154214 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c011830 con 0x7f4e4c0cca90 2023-08-27T11:29:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e4959a700 1 -- 172.21.15.112:0/2537154214 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c011990 con 0x7f4e4c0cca90 2023-08-27T11:29:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.902+0000 7f4e4959a700 1 -- 172.21.15.112:0/2537154214 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c011d20 con 0x7f4e4c0cca90 2023-08-27T11:29:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.903+0000 7f4e50de1700 1 -- 172.21.15.112:0/2537154214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e4c0cca90 msgr2=0x7f4e4c15f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.903+0000 7f4e50de1700 1 --2- 172.21.15.112:0/2537154214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e4c0cca90 0x7f4e4c15f650 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c06ea50 tx=0x7f4e3c009b20 comp rx=0 tx=0).stop 2023-08-27T11:29:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.903+0000 7f4e50de1700 1 -- 172.21.15.112:0/2537154214 shutdown_connections 2023-08-27T11:29:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.903+0000 7f4e50de1700 1 --2- 172.21.15.112:0/2537154214 >> v2:172.21.15.138:3300/0 conn(0x7f4e4c144900 0x7f4e4c0b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.903+0000 7f4e50de1700 1 --2- 172.21.15.112:0/2537154214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e4c0cca90 0x7f4e4c15f650 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.903+0000 7f4e50de1700 1 -- 172.21.15.112:0/2537154214 >> 172.21.15.112:0/2537154214 conn(0x7f4e4c0b1e90 msgr2=0x7f4e4c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:49.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.903+0000 7f4e50de1700 1 -- 172.21.15.112:0/2537154214 shutdown_connections 2023-08-27T11:29:49.910 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.903+0000 7f4e50de1700 1 -- 172.21.15.112:0/2537154214 wait complete. 2023-08-27T11:29:49.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.904+0000 7f4e50de1700 1 Processor -- start 2023-08-27T11:29:49.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.904+0000 7f4e50de1700 1 -- start start 2023-08-27T11:29:49.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e50de1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e4c06f790 0x7f4e4c15a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e50de1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e4c0cca90 0x7f4e4c15cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e50de1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e4c144900 0x7f4e4c15d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e50de1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c15acf0 con 0x7f4e4c0cca90 2023-08-27T11:29:49.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e50de1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0cb120 con 0x7f4e4c144900 2023-08-27T11:29:49.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e50de1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0cb400 con 0x7f4e4c06f790 2023-08-27T11:29:49.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e49d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e4c0cca90 0x7f4e4c15cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e49d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e4c0cca90 0x7f4e4c15cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38782/0 (socket says 172.21.15.112:38782) 2023-08-27T11:29:49.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e49d9b700 1 -- 172.21.15.112:0/1753947911 learned_addr learned my addr 172.21.15.112:0/1753947911 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:49.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e4ad9d700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e4c144900 0x7f4e4c15d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.916 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e4a59c700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e4c06f790 0x7f4e4c15a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e49d9b700 1 -- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e4c06f790 msgr2=0x7f4e4c15a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e49d9b700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e4c06f790 0x7f4e4c15a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e49d9b700 1 -- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e4c144900 msgr2=0x7f4e4c15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:49.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e49d9b700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e4c144900 0x7f4e4c15d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:49.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e49d9b700 1 -- 172.21.15.112:0/1753947911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c0094a0 con 0x7f4e4c0cca90 2023-08-27T11:29:49.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e4a59c700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e4c06f790 0x7f4e4c15a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:49.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.905+0000 7f4e4ad9d700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e4c144900 0x7f4e4c15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:49.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.906+0000 7f4e49d9b700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e4c0cca90 0x7f4e4c15cfe0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c000c00 tx=0x7f4e3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:49.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.906+0000 7f4e3affd700 1 -- 172.21.15.112:0/1753947911 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c0036a0 con 0x7f4e4c0cca90 2023-08-27T11:29:49.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.906+0000 7f4e3affd700 1 -- 172.21.15.112:0/1753947911 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c01f070 con 0x7f4e4c0cca90 2023-08-27T11:29:49.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.906+0000 7f4e3affd700 1 -- 172.21.15.112:0/1753947911 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c003a60 con 0x7f4e4c0cca90 2023-08-27T11:29:49.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.906+0000 7f4e50de1700 1 -- 172.21.15.112:0/1753947911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c0cb6e0 con 0x7f4e4c0cca90 2023-08-27T11:29:49.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.906+0000 7f4e50de1700 1 -- 172.21.15.112:0/1753947911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c0cbee0 con 0x7f4e4c0cca90 2023-08-27T11:29:49.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.907+0000 7f4e50de1700 1 -- 172.21.15.112:0/1753947911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c067e90 con 0x7f4e4c0cca90 2023-08-27T11:29:49.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.908+0000 7f4e3affd700 1 -- 172.21.15.112:0/1753947911 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4e3c003bc0 con 0x7f4e4c0cca90 2023-08-27T11:29:49.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.908+0000 7f4e3affd700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e2c06a4d0 0x7f4e2c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:49.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.908+0000 7f4e3affd700 1 -- 172.21.15.112:0/1753947911 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f4e3c021070 con 0x7f4e4c0cca90 2023-08-27T11:29:49.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.908+0000 7f4e4a59c700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e2c06a4d0 0x7f4e2c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:49.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.909+0000 7f4e4a59c700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e2c06a4d0 0x7f4e2c06c990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009960 tx=0x7f4e40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:49.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:49.912+0000 7f4e3affd700 1 -- 172.21.15.112:0/1753947911 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4e3c0557e0 con 0x7f4e4c0cca90 2023-08-27T11:29:50.101 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.100+0000 7f4e50de1700 1 -- 172.21.15.112:0/1753947911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e4c0cb260 con 0x7f4e4c0cca90 2023-08-27T11:29:50.102 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.100+0000 7f4e3affd700 1 -- 172.21.15.112:0/1753947911 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v172) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e4c0cb260 con 0x7f4e4c0cca90 2023-08-27T11:29:50.102 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:29:50.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.102+0000 7f4e50de1700 1 -- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e2c06a4d0 msgr2=0x7f4e2c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.102+0000 7f4e50de1700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e2c06a4d0 0x7f4e2c06c990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009960 tx=0x7f4e40008040 comp rx=0 tx=0).stop 2023-08-27T11:29:50.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.102+0000 7f4e50de1700 1 -- 172.21.15.112:0/1753947911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e4c0cca90 msgr2=0x7f4e4c15cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.104 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.102+0000 7f4e50de1700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e4c0cca90 0x7f4e4c15cfe0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c000c00 tx=0x7f4e3c004750 comp rx=0 tx=0).stop 2023-08-27T11:29:50.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.102+0000 7f4e50de1700 1 -- 172.21.15.112:0/1753947911 shutdown_connections 2023-08-27T11:29:50.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.102+0000 7f4e50de1700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4e2c06a4d0 0x7f4e2c06c990 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.102+0000 7f4e50de1700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e4c144900 0x7f4e4c15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.105 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.102+0000 7f4e50de1700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e4c06f790 0x7f4e4c15a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.108 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.102+0000 7f4e50de1700 1 --2- 172.21.15.112:0/1753947911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e4c0cca90 0x7f4e4c15cfe0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.102+0000 7f4e50de1700 1 -- 172.21.15.112:0/1753947911 >> 172.21.15.112:0/1753947911 conn(0x7f4e4c0b1e90 msgr2=0x7f4e4c1437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:50.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.102+0000 7f4e50de1700 1 -- 172.21.15.112:0/1753947911 shutdown_connections 2023-08-27T11:29:50.109 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.102+0000 7f4e50de1700 1 -- 172.21.15.112:0/1753947911 wait complete. 2023-08-27T11:29:50.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000001022841.meta pg_num 2023-08-27T11:29:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.238+0000 7f7e9ab2a700 1 Processor -- start 2023-08-27T11:29:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.238+0000 7f7e9ab2a700 1 -- start start 2023-08-27T11:29:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e9ab2a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e94146c20 0x7f7e94160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e9ab2a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7e9406f790 0x7f7e9406fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e9ab2a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e940b5070 con 0x7f7e940b3440 2023-08-27T11:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e9ab2a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e94070160 con 0x7f7e9406f790 2023-08-27T11:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e9ab2a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e94147000 con 0x7f7e94146c20 2023-08-27T11:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e93fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7e940b3440 0x7f7e9415e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33172/0 (socket says 172.21.15.112:33172) 2023-08-27T11:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e93fff700 1 -- 172.21.15.112:0/2530401539 learned_addr learned my addr 172.21.15.112:0/2530401539 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e988c6700 1 --2- 172.21.15.112:0/2530401539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e94146c20 0x7f7e94160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e990c7700 1 --2- 172.21.15.112:0/2530401539 >> v2:172.21.15.138:3300/0 conn(0x7f7e9406f790 0x7f7e9406fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e937fe700 1 -- 172.21.15.112:0/2530401539 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510272382 0 0) 0x7f7e80001110 con 0x7f7e940b3440 2023-08-27T11:29:50.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e937fe700 1 -- 172.21.15.112:0/2530401539 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e840035b0 con 0x7f7e940b3440 2023-08-27T11:29:50.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e988c6700 1 -- 172.21.15.112:0/2530401539 >> v2:172.21.15.138:3300/0 conn(0x7f7e9406f790 msgr2=0x7f7e9406fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:50.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e988c6700 1 --2- 172.21.15.112:0/2530401539 >> v2:172.21.15.138:3300/0 conn(0x7f7e9406f790 0x7f7e9406fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e988c6700 1 -- 172.21.15.112:0/2530401539 >> v1:172.21.15.112:6789/0 conn(0x7f7e940b3440 legacy=0x7f7e9415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e988c6700 1 -- 172.21.15.112:0/2530401539 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e941614a0 con 0x7f7e94146c20 2023-08-27T11:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.239+0000 7f7e990c7700 1 --2- 172.21.15.112:0/2530401539 >> v2:172.21.15.138:3300/0 conn(0x7f7e9406f790 0x7f7e9406fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.240+0000 7f7e988c6700 1 --2- 172.21.15.112:0/2530401539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e94146c20 0x7f7e94160f60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009840 tx=0x7f7e88009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5acfe2a0a13bdaf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.240+0000 7f7e937fe700 1 -- 172.21.15.112:0/2530401539 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e88011830 con 0x7f7e94146c20 2023-08-27T11:29:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.240+0000 7f7e937fe700 1 -- 172.21.15.112:0/2530401539 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e88011990 con 0x7f7e94146c20 2023-08-27T11:29:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.240+0000 7f7e937fe700 1 -- 172.21.15.112:0/2530401539 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e88011d00 con 0x7f7e94146c20 2023-08-27T11:29:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.240+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/2530401539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e94146c20 msgr2=0x7f7e94160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.240+0000 7f7e9ab2a700 1 --2- 172.21.15.112:0/2530401539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e94146c20 0x7f7e94160f60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009840 tx=0x7f7e88009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.240+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/2530401539 shutdown_connections 2023-08-27T11:29:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.240+0000 7f7e9ab2a700 1 --2- 172.21.15.112:0/2530401539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e94146c20 0x7f7e94160f60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.240+0000 7f7e9ab2a700 1 --2- 172.21.15.112:0/2530401539 >> v2:172.21.15.138:3300/0 conn(0x7f7e9406f790 0x7f7e9406fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.240+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/2530401539 >> 172.21.15.112:0/2530401539 conn(0x7f7e940b1e90 msgr2=0x7f7e940b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.240+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/2530401539 shutdown_connections 2023-08-27T11:29:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.240+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/2530401539 wait complete. 2023-08-27T11:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.242+0000 7f7e9ab2a700 1 Processor -- start 2023-08-27T11:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.242+0000 7f7e9ab2a700 1 -- start start 2023-08-27T11:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.242+0000 7f7e9ab2a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e9406f790 0x7f7e94142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.242+0000 7f7e9ab2a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e940b3440 0x7f7e940719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.242+0000 7f7e9ab2a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e94146c20 0x7f7e94071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.242+0000 7f7e9ab2a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e94143370 con 0x7f7e940b3440 2023-08-27T11:29:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.242+0000 7f7e9ab2a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9406e360 con 0x7f7e94146c20 2023-08-27T11:29:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.242+0000 7f7e9ab2a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e94144b00 con 0x7f7e9406f790 2023-08-27T11:29:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e93fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e940b3440 0x7f7e940719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e93fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e940b3440 0x7f7e940719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38796/0 (socket says 172.21.15.112:38796) 2023-08-27T11:29:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e93fff700 1 -- 172.21.15.112:0/1927129359 learned_addr learned my addr 172.21.15.112:0/1927129359 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e990c7700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e94146c20 0x7f7e94071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e988c6700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e9406f790 0x7f7e94142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e93fff700 1 -- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e9406f790 msgr2=0x7f7e94142e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e93fff700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e9406f790 0x7f7e94142e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e93fff700 1 -- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e94146c20 msgr2=0x7f7e94071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e93fff700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e94146c20 0x7f7e94071f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e93fff700 1 -- 172.21.15.112:0/1927129359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c009a70 con 0x7f7e940b3440 2023-08-27T11:29:50.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e93fff700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e940b3440 0x7f7e940719c0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000c4d0 tx=0x7f7e8000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:50.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e917fa700 1 -- 172.21.15.112:0/1927129359 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80005d70 con 0x7f7e940b3440 2023-08-27T11:29:50.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e917fa700 1 -- 172.21.15.112:0/1927129359 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e800070d0 con 0x7f7e940b3440 2023-08-27T11:29:50.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/1927129359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e880094a0 con 0x7f7e940b3440 2023-08-27T11:29:50.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e917fa700 1 -- 172.21.15.112:0/1927129359 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80007480 con 0x7f7e940b3440 2023-08-27T11:29:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.243+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/1927129359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e941450c0 con 0x7f7e940b3440 2023-08-27T11:29:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.244+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/1927129359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e94067e90 con 0x7f7e940b3440 2023-08-27T11:29:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.244+0000 7f7e917fa700 1 -- 172.21.15.112:0/1927129359 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f7e800075e0 con 0x7f7e940b3440 2023-08-27T11:29:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.245+0000 7f7e917fa700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7e7806a4d0 0x7f7e7806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.245+0000 7f7e917fa700 1 -- 172.21.15.112:0/1927129359 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f7e80088270 con 0x7f7e940b3440 2023-08-27T11:29:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.245+0000 7f7e988c6700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7e7806a4d0 0x7f7e7806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.246+0000 7f7e988c6700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7e7806a4d0 0x7f7e7806c990 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f7e88014040 tx=0x7f7e88008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.249+0000 7f7e917fa700 1 -- 172.21.15.112:0/1927129359 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7e800531a0 con 0x7f7e940b3440 2023-08-27T11:29:50.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.439+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/1927129359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000001022841.meta", "var": "pg_num"} v 0) v1 -- 0x7f7e940b5070 con 0x7f7e940b3440 2023-08-27T11:29:50.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.440+0000 7f7e917fa700 1 -- 172.21.15.112:0/1927129359 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000001022841.meta", "var": "pg_num"}]=0 v172) v1 ==== 124+0+11 (secure 0 0 0) 0x7f7e940b5070 con 0x7f7e940b3440 2023-08-27T11:29:50.441 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:29:50.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.442+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7e7806a4d0 msgr2=0x7f7e7806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.442+0000 7f7e9ab2a700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7e7806a4d0 0x7f7e7806c990 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f7e88014040 tx=0x7f7e88008030 comp rx=0 tx=0).stop 2023-08-27T11:29:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.442+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/1927129359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e940b3440 msgr2=0x7f7e940719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.442+0000 7f7e9ab2a700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e940b3440 0x7f7e940719c0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000c4d0 tx=0x7f7e8000c890 comp rx=0 tx=0).stop 2023-08-27T11:29:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.442+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/1927129359 shutdown_connections 2023-08-27T11:29:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.442+0000 7f7e9ab2a700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f7e7806a4d0 0x7f7e7806c990 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.442+0000 7f7e9ab2a700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e94146c20 0x7f7e94071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.442+0000 7f7e9ab2a700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e9406f790 0x7f7e94142e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.442+0000 7f7e9ab2a700 1 --2- 172.21.15.112:0/1927129359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e940b3440 0x7f7e940719c0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.442+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/1927129359 >> 172.21.15.112:0/1927129359 conn(0x7f7e940b1e90 msgr2=0x7f7e941602d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.442+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/1927129359 shutdown_connections 2023-08-27T11:29:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.442+0000 7f7e9ab2a700 1 -- 172.21.15.112:0/1927129359 wait complete. 2023-08-27T11:29:50.455 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:29:50.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.577+0000 7fe0c5c8e700 1 Processor -- start 2023-08-27T11:29:50.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.577+0000 7fe0c5c8e700 1 -- start start 2023-08-27T11:29:50.580 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0c5c8e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe0c01437c0 0x7fe0c00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0c5c8e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0c006d5b0 0x7fe0c006da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0c5c8e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c00cfd50 con 0x7fe0c0143ba0 2023-08-27T11:29:50.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0c5c8e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c006df80 con 0x7fe0c01437c0 2023-08-27T11:29:50.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0c5c8e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00cf140 con 0x7fe0c006d5b0 2023-08-27T11:29:50.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0bffff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0c006d5b0 0x7fe0c006da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0bf7fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe0c01437c0 0x7fe0c00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0bffff700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0c006d5b0 0x7fe0c006da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:50460/0 (socket says 172.21.15.112:50460) 2023-08-27T11:29:50.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0bf7fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe0c01437c0 0x7fe0c00b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:39228/0 (socket says 172.21.15.112:39228) 2023-08-27T11:29:50.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0beffd700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe0c0143ba0 0x7fe0c00ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:40708/0 (socket says 172.21.15.112:40708) 2023-08-27T11:29:50.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0bffff700 1 -- 172.21.15.112:0/1546626660 learned_addr learned my addr 172.21.15.112:0/1546626660 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:50.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0bf7fe700 1 -- 172.21.15.112:0/1546626660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0c006d5b0 msgr2=0x7fe0c006da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:50.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0bf7fe700 1 --2- 172.21.15.112:0/1546626660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0c006d5b0 0x7fe0c006da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0bf7fe700 1 -- 172.21.15.112:0/1546626660 >> v1:172.21.15.138:6789/0 conn(0x7fe0c0143ba0 legacy=0x7fe0c00ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0bf7fe700 1 -- 172.21.15.112:0/1546626660 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00cf420 con 0x7fe0c01437c0 2023-08-27T11:29:50.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0be7fc700 1 -- 172.21.15.112:0/1546626660 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042316399 0 0) 0x7fe0c00cfd50 con 0x7fe0c0143ba0 2023-08-27T11:29:50.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.578+0000 7fe0bffff700 1 --2- 172.21.15.112:0/1546626660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0c006d5b0 0x7fe0c006da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:50.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.579+0000 7fe0bf7fe700 1 --2- 172.21.15.112:0/1546626660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c01437c0 0x7fe0c00b4670 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009840 tx=0x7fe0b4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcc2bf78b84912ca server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:50.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.579+0000 7fe0be7fc700 1 -- 172.21.15.112:0/1546626660 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4011830 con 0x7fe0c01437c0 2023-08-27T11:29:50.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.579+0000 7fe0be7fc700 1 -- 172.21.15.112:0/1546626660 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b4011990 con 0x7fe0c01437c0 2023-08-27T11:29:50.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.579+0000 7fe0be7fc700 1 -- 172.21.15.112:0/1546626660 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4011d20 con 0x7fe0c01437c0 2023-08-27T11:29:50.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.579+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/1546626660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c01437c0 msgr2=0x7fe0c00b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.579+0000 7fe0c5c8e700 1 --2- 172.21.15.112:0/1546626660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c01437c0 0x7fe0c00b4670 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009840 tx=0x7fe0b4009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:50.585 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.579+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/1546626660 shutdown_connections 2023-08-27T11:29:50.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.579+0000 7fe0c5c8e700 1 --2- 172.21.15.112:0/1546626660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0c006d5b0 0x7fe0c006da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.579+0000 7fe0c5c8e700 1 --2- 172.21.15.112:0/1546626660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c01437c0 0x7fe0c00b4670 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.579+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/1546626660 >> 172.21.15.112:0/1546626660 conn(0x7fe0c00b1e90 msgr2=0x7fe0c00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:50.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.579+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/1546626660 shutdown_connections 2023-08-27T11:29:50.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.579+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/1546626660 wait complete. 2023-08-27T11:29:50.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.581+0000 7fe0c5c8e700 1 Processor -- start 2023-08-27T11:29:50.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.581+0000 7fe0c5c8e700 1 -- start start 2023-08-27T11:29:50.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.581+0000 7fe0c5c8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0c006d5b0 0x7fe0c015c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.581+0000 7fe0c5c8e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c01437c0 0x7fe0c015d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.581+0000 7fe0c5c8e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0c0143ba0 0x7fe0c015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0c5c8e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c015b2e0 con 0x7fe0c006d5b0 2023-08-27T11:29:50.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0c5c8e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c013c8a0 con 0x7fe0c01437c0 2023-08-27T11:29:50.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0c5c8e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c015b720 con 0x7fe0c0143ba0 2023-08-27T11:29:50.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0bf7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0c006d5b0 0x7fe0c015c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0bf7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0c006d5b0 0x7fe0c015c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38810/0 (socket says 172.21.15.112:38810) 2023-08-27T11:29:50.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0bf7fe700 1 -- 172.21.15.112:0/4223459309 learned_addr learned my addr 172.21.15.112:0/4223459309 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:50.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0beffd700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c01437c0 0x7fe0c015d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0bffff700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0c0143ba0 0x7fe0c015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0bf7fe700 1 -- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0c0143ba0 msgr2=0x7fe0c015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0bf7fe700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0c0143ba0 0x7fe0c015ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0bf7fe700 1 -- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c01437c0 msgr2=0x7fe0c015d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0bf7fe700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c01437c0 0x7fe0c015d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0bf7fe700 1 -- 172.21.15.112:0/4223459309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b8009b30 con 0x7fe0c006d5b0 2023-08-27T11:29:50.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0bf7fe700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0c006d5b0 0x7fe0c015c200 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009fd0 tx=0x7fe0b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:50.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0a7fff700 1 -- 172.21.15.112:0/4223459309 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40036a0 con 0x7fe0c006d5b0 2023-08-27T11:29:50.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0a7fff700 1 -- 172.21.15.112:0/4223459309 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b401f070 con 0x7fe0c006d5b0 2023-08-27T11:29:50.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/4223459309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b40094a0 con 0x7fe0c006d5b0 2023-08-27T11:29:50.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0a7fff700 1 -- 172.21.15.112:0/4223459309 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4003a60 con 0x7fe0c006d5b0 2023-08-27T11:29:50.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.582+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/4223459309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c015bce0 con 0x7fe0c006d5b0 2023-08-27T11:29:50.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.583+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/4223459309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c015a970 con 0x7fe0c006d5b0 2023-08-27T11:29:50.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.584+0000 7fe0a7fff700 1 -- 172.21.15.112:0/4223459309 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe0b4003bc0 con 0x7fe0c006d5b0 2023-08-27T11:29:50.593 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.584+0000 7fe0a7fff700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0ac06a4d0 0x7fe0ac06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.584+0000 7fe0a7fff700 1 -- 172.21.15.112:0/4223459309 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7fe0b4021070 con 0x7fe0c006d5b0 2023-08-27T11:29:50.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.584+0000 7fe0beffd700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0ac06a4d0 0x7fe0ac06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.585+0000 7fe0beffd700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0ac06a4d0 0x7fe0ac06c990 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe0c015cb60 tx=0x7fe0b0007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:50.594 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.589+0000 7fe0a7fff700 1 -- 172.21.15.112:0/4223459309 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe0c015a970 con 0x7fe0c006d5b0 2023-08-27T11:29:50.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.775+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/4223459309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0c015a970 con 0x7fe0c006d5b0 2023-08-27T11:29:50.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.776+0000 7fe0a7fff700 1 -- 172.21.15.112:0/4223459309 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0c015a970 con 0x7fe0c006d5b0 2023-08-27T11:29:50.777 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2023-08-27T11:29:50.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.778+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0ac06a4d0 msgr2=0x7fe0ac06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.778+0000 7fe0c5c8e700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0ac06a4d0 0x7fe0ac06c990 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe0c015cb60 tx=0x7fe0b0007e70 comp rx=0 tx=0).stop 2023-08-27T11:29:50.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.778+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/4223459309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0c006d5b0 msgr2=0x7fe0c015c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.778+0000 7fe0c5c8e700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0c006d5b0 0x7fe0c015c200 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009fd0 tx=0x7fe0b4004750 comp rx=0 tx=0).stop 2023-08-27T11:29:50.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.778+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/4223459309 shutdown_connections 2023-08-27T11:29:50.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.778+0000 7fe0c5c8e700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe0ac06a4d0 0x7fe0ac06c990 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.778+0000 7fe0c5c8e700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0c0143ba0 0x7fe0c015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.778+0000 7fe0c5c8e700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0c006d5b0 0x7fe0c015c200 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.778+0000 7fe0c5c8e700 1 --2- 172.21.15.112:0/4223459309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c01437c0 0x7fe0c015d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.778+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/4223459309 >> 172.21.15.112:0/4223459309 conn(0x7fe0c00b1e90 msgr2=0x7fe0c0070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:50.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.778+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/4223459309 shutdown_connections 2023-08-27T11:29:50.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.778+0000 7fe0c5c8e700 1 -- 172.21.15.112:0/4223459309 wait complete. 2023-08-27T11:29:50.791 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get another-data-pool pg_num 2023-08-27T11:29:50.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.913+0000 7fd26d538700 1 Processor -- start 2023-08-27T11:29:50.915 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.913+0000 7fd26d538700 1 -- start start 2023-08-27T11:29:50.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.914+0000 7fd26d538700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd26806fea0 0x7fd2680ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.914+0000 7fd26d538700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd2680ccad0 0x7fd26815b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.914+0000 7fd26d538700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2680b08e0 con 0x7fd2681445b0 2023-08-27T11:29:50.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.914+0000 7fd26d538700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26815bc70 con 0x7fd2680ccad0 2023-08-27T11:29:50.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.914+0000 7fd26d538700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2680cceb0 con 0x7fd26806fea0 2023-08-27T11:29:50.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.914+0000 7fd2677fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd2680ccad0 0x7fd26815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.914+0000 7fd2677fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd2680ccad0 0x7fd26815b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38820/0 (socket says 172.21.15.112:38820) 2023-08-27T11:29:50.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.914+0000 7fd2677fe700 1 -- 172.21.15.112:0/498364584 learned_addr learned my addr 172.21.15.112:0/498364584 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:50.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.915+0000 7fd2667fc700 1 --2- 172.21.15.112:0/498364584 >> v2:172.21.15.138:3300/0 conn(0x7fd26806fea0 0x7fd2680ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.915+0000 7fd2677fe700 1 -- 172.21.15.112:0/498364584 >> v2:172.21.15.138:3300/0 conn(0x7fd26806fea0 msgr2=0x7fd2680ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:50.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.915+0000 7fd2677fe700 1 --2- 172.21.15.112:0/498364584 >> v2:172.21.15.138:3300/0 conn(0x7fd26806fea0 0x7fd2680ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.915+0000 7fd2677fe700 1 -- 172.21.15.112:0/498364584 >> v1:172.21.15.138:6789/0 conn(0x7fd2681445b0 legacy=0x7fd26806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:50.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.915+0000 7fd2677fe700 1 -- 172.21.15.112:0/498364584 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2680b0b30 con 0x7fd2680ccad0 2023-08-27T11:29:50.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.915+0000 7fd2667fc700 1 --2- 172.21.15.112:0/498364584 >> v2:172.21.15.138:3300/0 conn(0x7fd26806fea0 0x7fd2680ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:50.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.915+0000 7fd2677fe700 1 --2- 172.21.15.112:0/498364584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2680ccad0 0x7fd26815b730 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fd268070400 tx=0x7fd26000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1326177a07068c8f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:50.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.915+0000 7fd265ffb700 1 -- 172.21.15.112:0/498364584 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26000e070 con 0x7fd2680ccad0 2023-08-27T11:29:50.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.915+0000 7fd265ffb700 1 -- 172.21.15.112:0/498364584 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd260003a20 con 0x7fd2680ccad0 2023-08-27T11:29:50.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.915+0000 7fd265ffb700 1 -- 172.21.15.112:0/498364584 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260003dd0 con 0x7fd2680ccad0 2023-08-27T11:29:50.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.916+0000 7fd26d538700 1 -- 172.21.15.112:0/498364584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2680ccad0 msgr2=0x7fd26815b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.916+0000 7fd26d538700 1 --2- 172.21.15.112:0/498364584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2680ccad0 0x7fd26815b730 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fd268070400 tx=0x7fd26000b330 comp rx=0 tx=0).stop 2023-08-27T11:29:50.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.916+0000 7fd26d538700 1 -- 172.21.15.112:0/498364584 shutdown_connections 2023-08-27T11:29:50.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.916+0000 7fd26d538700 1 --2- 172.21.15.112:0/498364584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2680ccad0 0x7fd26815b730 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.916+0000 7fd26d538700 1 --2- 172.21.15.112:0/498364584 >> v2:172.21.15.138:3300/0 conn(0x7fd26806fea0 0x7fd2680ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.916+0000 7fd26d538700 1 -- 172.21.15.112:0/498364584 >> 172.21.15.112:0/498364584 conn(0x7fd2680b1e90 msgr2=0x7fd2680b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:50.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.916+0000 7fd26d538700 1 -- 172.21.15.112:0/498364584 shutdown_connections 2023-08-27T11:29:50.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.916+0000 7fd26d538700 1 -- 172.21.15.112:0/498364584 wait complete. 2023-08-27T11:29:50.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.917+0000 7fd26d538700 1 Processor -- start 2023-08-27T11:29:50.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.917+0000 7fd26d538700 1 -- start start 2023-08-27T11:29:50.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.918+0000 7fd26d538700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd26806fea0 0x7fd2680aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.918+0000 7fd26d538700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2680ccad0 0x7fd26806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.918+0000 7fd26d538700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2681445b0 0x7fd26806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.918+0000 7fd26d538700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26813ac10 con 0x7fd2681445b0 2023-08-27T11:29:50.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.918+0000 7fd26d538700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26813aeb0 con 0x7fd2680ccad0 2023-08-27T11:29:50.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.918+0000 7fd26d538700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26813b190 con 0x7fd26806fea0 2023-08-27T11:29:50.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.918+0000 7fd2677fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2681445b0 0x7fd26806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.918+0000 7fd2677fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2681445b0 0x7fd26806d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38824/0 (socket says 172.21.15.112:38824) 2023-08-27T11:29:50.929 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.918+0000 7fd2677fe700 1 -- 172.21.15.112:0/613937858 learned_addr learned my addr 172.21.15.112:0/613937858 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:50.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd2667fc700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2680ccad0 0x7fd26806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd2677fe700 1 -- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd26806fea0 msgr2=0x7fd2680aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd266ffd700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd26806fea0 0x7fd2680aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd2677fe700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd26806fea0 0x7fd2680aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.930 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd2677fe700 1 -- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2680ccad0 msgr2=0x7fd26806d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:50.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd2677fe700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2680ccad0 0x7fd26806d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:50.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd2677fe700 1 -- 172.21.15.112:0/613937858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd260009b30 con 0x7fd2681445b0 2023-08-27T11:29:50.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd2667fc700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2680ccad0 0x7fd26806d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:50.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd266ffd700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd26806fea0 0x7fd2680aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:50.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd2677fe700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2681445b0 0x7fd26806d9f0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fd260015040 tx=0x7fd260004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:50.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd24f7fe700 1 -- 172.21.15.112:0/613937858 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260004440 con 0x7fd2681445b0 2023-08-27T11:29:50.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd24f7fe700 1 -- 172.21.15.112:0/613937858 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2600045a0 con 0x7fd2681445b0 2023-08-27T11:29:50.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd26d538700 1 -- 172.21.15.112:0/613937858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26813b470 con 0x7fd2681445b0 2023-08-27T11:29:50.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd24f7fe700 1 -- 172.21.15.112:0/613937858 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260004950 con 0x7fd2681445b0 2023-08-27T11:29:50.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.919+0000 7fd26d538700 1 -- 172.21.15.112:0/613937858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26814dcd0 con 0x7fd2681445b0 2023-08-27T11:29:50.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.920+0000 7fd26d538700 1 -- 172.21.15.112:0/613937858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26814de60 con 0x7fd2681445b0 2023-08-27T11:29:50.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.920+0000 7fd24f7fe700 1 -- 172.21.15.112:0/613937858 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fd260012430 con 0x7fd2681445b0 2023-08-27T11:29:50.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.921+0000 7fd24f7fe700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd25406a4d0 0x7fd25406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:50.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.921+0000 7fd24f7fe700 1 -- 172.21.15.112:0/613937858 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7fd26008a580 con 0x7fd2681445b0 2023-08-27T11:29:50.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.921+0000 7fd266ffd700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd25406a4d0 0x7fd25406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:50.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.921+0000 7fd266ffd700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd25406a4d0 0x7fd25406c990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd25c003f40 tx=0x7fd25c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:50.934 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:50.925+0000 7fd24f7fe700 1 -- 172.21.15.112:0/613937858 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd260055370 con 0x7fd2681445b0 2023-08-27T11:29:51.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.114+0000 7fd26d538700 1 -- 172.21.15.112:0/613937858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "another-data-pool", "var": "pg_num"} v 0) v1 -- 0x7fd268067e90 con 0x7fd2681445b0 2023-08-27T11:29:51.116 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.115+0000 7fd24f7fe700 1 -- 172.21.15.112:0/613937858 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "another-data-pool", "var": "pg_num"}]=0 v172) v1 ==== 106+0+11 (secure 0 0 0) 0x7fd260058990 con 0x7fd2681445b0 2023-08-27T11:29:51.117 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2023-08-27T11:29:51.118 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.117+0000 7fd26d538700 1 -- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd25406a4d0 msgr2=0x7fd25406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.117+0000 7fd26d538700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd25406a4d0 0x7fd25406c990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd25c003f40 tx=0x7fd25c009040 comp rx=0 tx=0).stop 2023-08-27T11:29:51.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.117+0000 7fd26d538700 1 -- 172.21.15.112:0/613937858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2681445b0 msgr2=0x7fd26806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.117+0000 7fd26d538700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2681445b0 0x7fd26806d9f0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fd260015040 tx=0x7fd260004ad0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.119 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.117+0000 7fd26d538700 1 -- 172.21.15.112:0/613937858 shutdown_connections 2023-08-27T11:29:51.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.117+0000 7fd26d538700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fd25406a4d0 0x7fd25406c990 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.117+0000 7fd26d538700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2681445b0 0x7fd26806d9f0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.117+0000 7fd26d538700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd26806fea0 0x7fd2680aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.117+0000 7fd26d538700 1 --2- 172.21.15.112:0/613937858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2680ccad0 0x7fd26806d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.117+0000 7fd26d538700 1 -- 172.21.15.112:0/613937858 >> 172.21.15.112:0/613937858 conn(0x7fd2680b1e90 msgr2=0x7fd26806f660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:51.120 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.117+0000 7fd26d538700 1 -- 172.21.15.112:0/613937858 shutdown_connections 2023-08-27T11:29:51.121 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.117+0000 7fd26d538700 1 -- 172.21.15.112:0/613937858 wait complete. 2023-08-27T11:29:51.130 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:51.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.249+0000 7efc7c210700 1 Processor -- start 2023-08-27T11:29:51.250 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.249+0000 7efc7c210700 1 -- start start 2023-08-27T11:29:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc7c210700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efc74144570 0x7efc7415ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc7c210700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc740b4ad0 0x7efc740b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc7c210700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc740b45f0 con 0x7efc740b3e50 2023-08-27T11:29:51.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc7c210700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc740cf7a0 con 0x7efc74144570 2023-08-27T11:29:51.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc7c210700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc74144950 con 0x7efc740b4ad0 2023-08-27T11:29:51.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc797ab700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7efc740b3e50 0x7efc7415f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33214/0 (socket says 172.21.15.112:33214) 2023-08-27T11:29:51.254 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc797ab700 1 -- 172.21.15.112:0/4237301553 learned_addr learned my addr 172.21.15.112:0/4237301553 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:51.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc7a7ad700 1 --2- 172.21.15.112:0/4237301553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc740b4ad0 0x7efc740b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc79fac700 1 --2- 172.21.15.112:0/4237301553 >> v2:172.21.15.138:3300/0 conn(0x7efc74144570 0x7efc7415ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc78faa700 1 -- 172.21.15.112:0/4237301553 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328965826 0 0) 0x7efc740b45f0 con 0x7efc740b3e50 2023-08-27T11:29:51.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc78faa700 1 -- 172.21.15.112:0/4237301553 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc600035b0 con 0x7efc740b3e50 2023-08-27T11:29:51.255 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc78faa700 1 -- 172.21.15.112:0/4237301553 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1959595013 0 0) 0x7efc600035b0 con 0x7efc740b3e50 2023-08-27T11:29:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc79fac700 1 -- 172.21.15.112:0/4237301553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc740b4ad0 msgr2=0x7efc740b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc79fac700 1 --2- 172.21.15.112:0/4237301553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc740b4ad0 0x7efc740b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc79fac700 1 -- 172.21.15.112:0/4237301553 >> v1:172.21.15.112:6789/0 conn(0x7efc740b3e50 legacy=0x7efc7415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc79fac700 1 -- 172.21.15.112:0/4237301553 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc740b4310 con 0x7efc74144570 2023-08-27T11:29:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.250+0000 7efc7a7ad700 1 --2- 172.21.15.112:0/4237301553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc740b4ad0 0x7efc740b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.251+0000 7efc79fac700 1 --2- 172.21.15.112:0/4237301553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc74144570 0x7efc7415ee20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efc70009870 tx=0x7efc70009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68bdf453faf14ff7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.251+0000 7efc78faa700 1 -- 172.21.15.112:0/4237301553 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc64001670 con 0x7efc74144570 2023-08-27T11:29:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.251+0000 7efc78faa700 1 -- 172.21.15.112:0/4237301553 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc70004030 con 0x7efc74144570 2023-08-27T11:29:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.251+0000 7efc78faa700 1 -- 172.21.15.112:0/4237301553 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc700043c0 con 0x7efc74144570 2023-08-27T11:29:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.251+0000 7efc7c210700 1 -- 172.21.15.112:0/4237301553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc74144570 msgr2=0x7efc7415ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.251+0000 7efc7c210700 1 --2- 172.21.15.112:0/4237301553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc74144570 0x7efc7415ee20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efc70009870 tx=0x7efc70009b80 comp rx=0 tx=0).stop 2023-08-27T11:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.251+0000 7efc7c210700 1 -- 172.21.15.112:0/4237301553 shutdown_connections 2023-08-27T11:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.251+0000 7efc7c210700 1 --2- 172.21.15.112:0/4237301553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc740b4ad0 0x7efc740b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.251+0000 7efc7c210700 1 --2- 172.21.15.112:0/4237301553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc74144570 0x7efc7415ee20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.251+0000 7efc7c210700 1 -- 172.21.15.112:0/4237301553 >> 172.21.15.112:0/4237301553 conn(0x7efc7406f970 msgr2=0x7efc7406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:51.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.251+0000 7efc7c210700 1 -- 172.21.15.112:0/4237301553 shutdown_connections 2023-08-27T11:29:51.259 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.251+0000 7efc7c210700 1 -- 172.21.15.112:0/4237301553 wait complete. 2023-08-27T11:29:51.261 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.253+0000 7efc7c210700 1 Processor -- start 2023-08-27T11:29:51.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.253+0000 7efc7c210700 1 -- start start 2023-08-27T11:29:51.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.253+0000 7efc7c210700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc740b3e50 0x7efc741f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.253+0000 7efc7c210700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc740b4ad0 0x7efc7415a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.253+0000 7efc7c210700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc74144570 0x7efc7415a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.262 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.253+0000 7efc7c210700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc741f07e0 con 0x7efc740b4ad0 2023-08-27T11:29:51.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.253+0000 7efc7c210700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7413abf0 con 0x7efc740b3e50 2023-08-27T11:29:51.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.253+0000 7efc7c210700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc7413b080 con 0x7efc74144570 2023-08-27T11:29:51.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.253+0000 7efc797ab700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc740b4ad0 0x7efc7415a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.253+0000 7efc797ab700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc740b4ad0 0x7efc7415a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38838/0 (socket says 172.21.15.112:38838) 2023-08-27T11:29:51.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.253+0000 7efc797ab700 1 -- 172.21.15.112:0/4038173608 learned_addr learned my addr 172.21.15.112:0/4038173608 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:51.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.254+0000 7efc797ab700 1 -- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc74144570 msgr2=0x7efc7415a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.254+0000 7efc79fac700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc740b3e50 0x7efc741f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.254+0000 7efc7a7ad700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc74144570 0x7efc7415a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.254+0000 7efc797ab700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc74144570 0x7efc7415a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.254+0000 7efc797ab700 1 -- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc740b3e50 msgr2=0x7efc741f02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.254+0000 7efc797ab700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc740b3e50 0x7efc741f02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.254+0000 7efc797ab700 1 -- 172.21.15.112:0/4038173608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6c009a70 con 0x7efc740b4ad0 2023-08-27T11:29:51.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.254+0000 7efc797ab700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc740b4ad0 0x7efc7415a390 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7efc6400c780 tx=0x7efc6400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:51.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.254+0000 7efc6a7fc700 1 -- 172.21.15.112:0/4038173608 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc64007560 con 0x7efc740b4ad0 2023-08-27T11:29:51.265 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.254+0000 7efc6a7fc700 1 -- 172.21.15.112:0/4038173608 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc640076e0 con 0x7efc740b4ad0 2023-08-27T11:29:51.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.254+0000 7efc6a7fc700 1 -- 172.21.15.112:0/4038173608 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc64004ab0 con 0x7efc740b4ad0 2023-08-27T11:29:51.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.254+0000 7efc7c210700 1 -- 172.21.15.112:0/4038173608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc700094a0 con 0x7efc740b4ad0 2023-08-27T11:29:51.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.254+0000 7efc7c210700 1 -- 172.21.15.112:0/4038173608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc74139fe0 con 0x7efc740b4ad0 2023-08-27T11:29:51.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.255+0000 7efc7c210700 1 -- 172.21.15.112:0/4038173608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc74067a20 con 0x7efc740b4ad0 2023-08-27T11:29:51.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.255+0000 7efc6a7fc700 1 -- 172.21.15.112:0/4038173608 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7efc64004c10 con 0x7efc740b4ad0 2023-08-27T11:29:51.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.256+0000 7efc6a7fc700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc5806a4d0 0x7efc5806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.256+0000 7efc6a7fc700 1 -- 172.21.15.112:0/4038173608 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7efc64088040 con 0x7efc740b4ad0 2023-08-27T11:29:51.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.256+0000 7efc79fac700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc5806a4d0 0x7efc5806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.256+0000 7efc79fac700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc5806a4d0 0x7efc5806c990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7efc700090b0 tx=0x7efc70013040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:51.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.260+0000 7efc6a7fc700 1 -- 172.21.15.112:0/4038173608 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efc64053000 con 0x7efc740b4ad0 2023-08-27T11:29:51.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.493+0000 7efc7c210700 1 -- 172.21.15.112:0/4038173608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc740b2fb0 con 0x7efc740b4ad0 2023-08-27T11:29:51.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.494+0000 7efc6a7fc700 1 -- 172.21.15.112:0/4038173608 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4914 (secure 0 0 0) 0x7efc64018020 con 0x7efc740b4ad0 2023-08-27T11:29:51.496 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:51.497 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/685371648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":685371648},{"type":"v1","addr":"172.21.15.138:6836","nonce":685371648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":7214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3585152138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3585152138},{"type":"v1","addr":"172.21.15.138:6837","nonce":3585152138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":8203,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2190275630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2190275630},{"type":"v1","addr":"172.21.15.112:6835","nonce":2190275630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:29:22.938324+0000","modified":"2023-08-27T11:29:46.388631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6833/4086629952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4086629952},{"type":"v1","addr":"172.21.15.112:6833","nonce":4086629952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"volume_0000000001022841","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-08-27T11:29:51.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.496+0000 7efc7c210700 1 -- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc5806a4d0 msgr2=0x7efc5806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.496+0000 7efc7c210700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc5806a4d0 0x7efc5806c990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7efc700090b0 tx=0x7efc70013040 comp rx=0 tx=0).stop 2023-08-27T11:29:51.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.496+0000 7efc7c210700 1 -- 172.21.15.112:0/4038173608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc740b4ad0 msgr2=0x7efc7415a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.496+0000 7efc7c210700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc740b4ad0 0x7efc7415a390 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7efc6400c780 tx=0x7efc6400cb40 comp rx=0 tx=0).stop 2023-08-27T11:29:51.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.496+0000 7efc7c210700 1 -- 172.21.15.112:0/4038173608 shutdown_connections 2023-08-27T11:29:51.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.496+0000 7efc7c210700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7efc5806a4d0 0x7efc5806c990 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.496+0000 7efc7c210700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc74144570 0x7efc7415a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.496+0000 7efc7c210700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc740b3e50 0x7efc741f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.496+0000 7efc7c210700 1 --2- 172.21.15.112:0/4038173608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc740b4ad0 0x7efc7415a390 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.496+0000 7efc7c210700 1 -- 172.21.15.112:0/4038173608 >> 172.21.15.112:0/4038173608 conn(0x7efc7406f970 msgr2=0x7efc74140990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:51.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.496+0000 7efc7c210700 1 -- 172.21.15.112:0/4038173608 shutdown_connections 2023-08-27T11:29:51.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.496+0000 7efc7c210700 1 -- 172.21.15.112:0/4038173608 wait complete. 2023-08-27T11:29:51.500 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 116 2023-08-27T11:29:51.509 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:51.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.627+0000 7f4bd7b77700 1 Processor -- start 2023-08-27T11:29:51.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.627+0000 7f4bd7b77700 1 -- start start 2023-08-27T11:29:51.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd7b77700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4bd0146c00 0x7f4bd015e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd7b77700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4bd00cec70 0x7f4bd0066010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd7b77700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bd0066550 0x7f4bd0161110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd7b77700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd00d0b60 con 0x7f4bd0146c00 2023-08-27T11:29:51.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd7b77700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd00669a0 con 0x7f4bd00cec70 2023-08-27T11:29:51.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd7b77700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0072250 con 0x7f4bd0066550 2023-08-27T11:29:51.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd5913700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4bd0146c00 0x7f4bd015e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd5913700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4bd0146c00 0x7f4bd015e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38844/0 (socket says 172.21.15.112:38844) 2023-08-27T11:29:51.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd5913700 1 -- 172.21.15.112:0/2889663107 learned_addr learned my addr 172.21.15.112:0/2889663107 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:51.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd5112700 1 --2- 172.21.15.112:0/2889663107 >> v2:172.21.15.138:3300/0 conn(0x7f4bd00cec70 0x7f4bd0066010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd6114700 1 --2- 172.21.15.112:0/2889663107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bd0066550 0x7f4bd0161110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd5913700 1 -- 172.21.15.112:0/2889663107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bd0066550 msgr2=0x7f4bd0161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:51.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd5913700 1 --2- 172.21.15.112:0/2889663107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bd0066550 0x7f4bd0161110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd5913700 1 -- 172.21.15.112:0/2889663107 >> v2:172.21.15.138:3300/0 conn(0x7f4bd00cec70 msgr2=0x7f4bd0066010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:51.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd5913700 1 --2- 172.21.15.112:0/2889663107 >> v2:172.21.15.138:3300/0 conn(0x7f4bd00cec70 0x7f4bd0066010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.628+0000 7f4bd5913700 1 -- 172.21.15.112:0/2889663107 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd00724a0 con 0x7f4bd0146c00 2023-08-27T11:29:51.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.629+0000 7f4bd5913700 1 --2- 172.21.15.112:0/2889663107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0146c00 0x7f4bd015e550 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00b30c0 tx=0x7f4bc0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c3280805ea617 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:51.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.629+0000 7f4bd4911700 1 -- 172.21.15.112:0/2889663107 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc000b870 con 0x7f4bd0146c00 2023-08-27T11:29:51.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.629+0000 7f4bd4911700 1 -- 172.21.15.112:0/2889663107 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc000b9d0 con 0x7f4bd0146c00 2023-08-27T11:29:51.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.629+0000 7f4bd4911700 1 -- 172.21.15.112:0/2889663107 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc000bd60 con 0x7f4bd0146c00 2023-08-27T11:29:51.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.629+0000 7f4bd7b77700 1 -- 172.21.15.112:0/2889663107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0146c00 msgr2=0x7f4bd015e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.629+0000 7f4bd7b77700 1 --2- 172.21.15.112:0/2889663107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0146c00 0x7f4bd015e550 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00b30c0 tx=0x7f4bc0009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.629+0000 7f4bd7b77700 1 -- 172.21.15.112:0/2889663107 shutdown_connections 2023-08-27T11:29:51.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.629+0000 7f4bd7b77700 1 --2- 172.21.15.112:0/2889663107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bd0066550 0x7f4bd0161110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.629+0000 7f4bd7b77700 1 --2- 172.21.15.112:0/2889663107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0146c00 0x7f4bd015e550 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.629+0000 7f4bd7b77700 1 --2- 172.21.15.112:0/2889663107 >> v2:172.21.15.138:3300/0 conn(0x7f4bd00cec70 0x7f4bd0066010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.629+0000 7f4bd7b77700 1 -- 172.21.15.112:0/2889663107 >> 172.21.15.112:0/2889663107 conn(0x7f4bd006f970 msgr2=0x7f4bd006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:51.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.629+0000 7f4bd7b77700 1 -- 172.21.15.112:0/2889663107 shutdown_connections 2023-08-27T11:29:51.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.629+0000 7f4bd7b77700 1 -- 172.21.15.112:0/2889663107 wait complete. 2023-08-27T11:29:51.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.631+0000 7f4bd7b77700 1 Processor -- start 2023-08-27T11:29:51.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.631+0000 7f4bd7b77700 1 -- start start 2023-08-27T11:29:51.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.631+0000 7f4bd7b77700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bd0066550 0x7f4bd00ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.631+0000 7f4bd7b77700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00cec70 0x7f4bd013b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.631+0000 7f4bd7b77700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0146c00 0x7f4bd015ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.631+0000 7f4bd7b77700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd015ee60 con 0x7f4bd0146c00 2023-08-27T11:29:51.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.631+0000 7f4bd7b77700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd015f140 con 0x7f4bd00cec70 2023-08-27T11:29:51.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.631+0000 7f4bd7b77700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd015f420 con 0x7f4bd0066550 2023-08-27T11:29:51.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.631+0000 7f4bd6114700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0146c00 0x7f4bd015ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.631+0000 7f4bd6114700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0146c00 0x7f4bd015ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38854/0 (socket says 172.21.15.112:38854) 2023-08-27T11:29:51.641 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.631+0000 7f4bd6114700 1 -- 172.21.15.112:0/1477849359 learned_addr learned my addr 172.21.15.112:0/1477849359 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:51.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.631+0000 7f4bd5112700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00cec70 0x7f4bd013b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bd5913700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bd0066550 0x7f4bd00ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bd6114700 1 -- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bd0066550 msgr2=0x7f4bd00ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bd6114700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bd0066550 0x7f4bd00ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.642 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bd6114700 1 -- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00cec70 msgr2=0x7f4bd013b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bd6114700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00cec70 0x7f4bd013b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bd6114700 1 -- 172.21.15.112:0/1477849359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc0009550 con 0x7f4bd0146c00 2023-08-27T11:29:51.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bd5112700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00cec70 0x7f4bd013b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:51.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bd5913700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bd0066550 0x7f4bd00ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:51.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bd6114700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0146c00 0x7f4bd015ff00 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8007d20 tx=0x7f4bc800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:51.643 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bc67fc700 1 -- 172.21.15.112:0/1477849359 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800ede0 con 0x7f4bd0146c00 2023-08-27T11:29:51.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bc67fc700 1 -- 172.21.15.112:0/1477849359 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc8014420 con 0x7f4bd0146c00 2023-08-27T11:29:51.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bd7b77700 1 -- 172.21.15.112:0/1477849359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd015f700 con 0x7f4bd0146c00 2023-08-27T11:29:51.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bc67fc700 1 -- 172.21.15.112:0/1477849359 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80147d0 con 0x7f4bd0146c00 2023-08-27T11:29:51.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.632+0000 7f4bd7b77700 1 -- 172.21.15.112:0/1477849359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd0160440 con 0x7f4bd0146c00 2023-08-27T11:29:51.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.633+0000 7f4bd7b77700 1 -- 172.21.15.112:0/1477849359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd015fc20 con 0x7f4bd0146c00 2023-08-27T11:29:51.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.633+0000 7f4bc67fc700 1 -- 172.21.15.112:0/1477849359 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f4bc8014930 con 0x7f4bd0146c00 2023-08-27T11:29:51.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.634+0000 7f4bc67fc700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bb406a4d0 0x7f4bb406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.634+0000 7f4bc67fc700 1 -- 172.21.15.112:0/1477849359 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f4bc80895b0 con 0x7f4bd0146c00 2023-08-27T11:29:51.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.634+0000 7f4bd5913700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bb406a4d0 0x7f4bb406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.634+0000 7f4bd5913700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bb406a4d0 0x7f4bb406c990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0000c00 tx=0x7f4bc0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:51.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.638+0000 7f4bc67fc700 1 -- 172.21.15.112:0/1477849359 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4bd015fc20 con 0x7f4bd0146c00 2023-08-27T11:29:51.827 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.826+0000 7f4bd7b77700 1 -- 172.21.15.112:0/1477849359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bd015fc20 con 0x7f4bd0146c00 2023-08-27T11:29:51.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.828+0000 7f4bc67fc700 1 -- 172.21.15.112:0/1477849359 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+20116 (secure 0 0 0) 0x7f4bd015fc20 con 0x7f4bd0146c00 2023-08-27T11:29:51.829 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:51.830 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":172,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:47.398430+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs.volume_0000000001022841.meta","create_time":"2023-08-27T11:29:20.569324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000001022841"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-08-27T11:29:21.905812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000001022841"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"another-data-pool","create_time":"2023-08-27T11:29:43.100842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000001022841"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:51.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.830+0000 7f4bd7b77700 1 -- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bb406a4d0 msgr2=0x7f4bb406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.830+0000 7f4bd7b77700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bb406a4d0 0x7f4bb406c990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0000c00 tx=0x7f4bc0008030 comp rx=0 tx=0).stop 2023-08-27T11:29:51.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.830+0000 7f4bd7b77700 1 -- 172.21.15.112:0/1477849359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0146c00 msgr2=0x7f4bd015ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.830+0000 7f4bd7b77700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0146c00 0x7f4bd015ff00 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8007d20 tx=0x7f4bc800e3c0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.830+0000 7f4bd7b77700 1 -- 172.21.15.112:0/1477849359 shutdown_connections 2023-08-27T11:29:51.832 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.830+0000 7f4bd7b77700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f4bb406a4d0 0x7f4bb406c990 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.830+0000 7f4bd7b77700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0146c00 0x7f4bd015ff00 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.830+0000 7f4bd7b77700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bd0066550 0x7f4bd00ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.830+0000 7f4bd7b77700 1 --2- 172.21.15.112:0/1477849359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00cec70 0x7f4bd013b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.830+0000 7f4bd7b77700 1 -- 172.21.15.112:0/1477849359 >> 172.21.15.112:0/1477849359 conn(0x7f4bd006f970 msgr2=0x7f4bd0144520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:51.833 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.830+0000 7f4bd7b77700 1 -- 172.21.15.112:0/1477849359 shutdown_connections 2023-08-27T11:29:51.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.830+0000 7f4bd7b77700 1 -- 172.21.15.112:0/1477849359 wait complete. 2023-08-27T11:29:51.843 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000001022841 and related pools 2023-08-27T11:29:51.844 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:51.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.962+0000 7f8fb89e1700 1 Processor -- start 2023-08-27T11:29:51.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.962+0000 7f8fb89e1700 1 -- start start 2023-08-27T11:29:51.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.963+0000 7f8fb89e1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8fb40b17e0 0x7f8fb40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.963+0000 7f8fb89e1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb4161850 con 0x7f8fb40b1400 2023-08-27T11:29:51.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.963+0000 7f8fb89e1700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb40cfc10 con 0x7f8fb4161ce0 2023-08-27T11:29:51.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.963+0000 7f8fb89e1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb4161200 con 0x7f8fb40b17e0 2023-08-27T11:29:51.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.963+0000 7f8fb259c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8fb40b1400 0x7f8fb406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33240/0 (socket says 172.21.15.112:33240) 2023-08-27T11:29:51.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.963+0000 7f8fb259c700 1 -- 172.21.15.112:0/2635426182 learned_addr learned my addr 172.21.15.112:0/2635426182 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:51.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.963+0000 7f8fb1d9b700 1 --2- 172.21.15.112:0/2635426182 >> v2:172.21.15.138:3300/0 conn(0x7f8fb40b17e0 0x7f8fb40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.963+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686950331 0 0) 0x7f8fb4161850 con 0x7f8fb40b1400 2023-08-27T11:29:51.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f9c0035b0 con 0x7f8fb40b1400 2023-08-27T11:29:51.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4112022947 0 0) 0x7f8f9c0035b0 con 0x7f8fb40b1400 2023-08-27T11:29:51.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 >> v2:172.21.15.138:3300/0 conn(0x7f8fb40b17e0 msgr2=0x7f8fb40cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:51.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 --2- 172.21.15.112:0/2635426182 >> v2:172.21.15.138:3300/0 conn(0x7f8fb40b17e0 0x7f8fb40cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 >> v1:172.21.15.138:6789/0 conn(0x7f8fb4161ce0 legacy=0x7f8fb406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb41614e0 con 0x7f8fb40b1400 2023-08-27T11:29:51.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f8fa4001670 con 0x7f8fb40b1400 2023-08-27T11:29:51.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 >> v1:172.21.15.112:6789/0 conn(0x7f8fb40b1400 legacy=0x7f8fb406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 --2- 172.21.15.112:0/2635426182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f9c003a00 0x7f8f9c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 --2- 172.21.15.112:0/2635426182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f9c004850 0x7f8f9c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 --2- 172.21.15.112:0/2635426182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f9c005330 0x7f8f9c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb4161200 con 0x7f8f9c005330 2023-08-27T11:29:51.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c00c200 con 0x7f8f9c003a00 2023-08-27T11:29:51.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c00c3c0 con 0x7f8f9c004850 2023-08-27T11:29:51.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb2d9d700 1 --2- 172.21.15.112:0/2635426182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f9c005330 0x7f8f9c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb259c700 1 --2- 172.21.15.112:0/2635426182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f9c003a00 0x7f8f9c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb1d9b700 1 --2- 172.21.15.112:0/2635426182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f9c004850 0x7f8f9c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb2d9d700 1 -- 172.21.15.112:0/2635426182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f9c004850 msgr2=0x7f8f9c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:51.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb2d9d700 1 --2- 172.21.15.112:0/2635426182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f9c004850 0x7f8f9c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb2d9d700 1 -- 172.21.15.112:0/2635426182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f9c003a00 msgr2=0x7f8f9c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:51.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb2d9d700 1 --2- 172.21.15.112:0/2635426182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f9c003a00 0x7f8f9c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb2d9d700 1 -- 172.21.15.112:0/2635426182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c0035b0 con 0x7f8f9c005330 2023-08-27T11:29:51.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb1d9b700 1 --2- 172.21.15.112:0/2635426182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f9c004850 0x7f8f9c004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:51.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb2d9d700 1 -- 172.21.15.112:0/2635426182 reap_dead start 2023-08-27T11:29:51.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb2d9d700 1 --2- 172.21.15.112:0/2635426182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f9c005330 0x7f8f9c00ba10 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00c160 tx=0x7f8f9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d327f91c7fba14 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:51.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac001db0 con 0x7f8f9c005330 2023-08-27T11:29:51.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac00cca0 con 0x7f8f9c005330 2023-08-27T11:29:51.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.964+0000 7f8fb159a700 1 -- 172.21.15.112:0/2635426182 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac003c00 con 0x7f8f9c005330 2023-08-27T11:29:51.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.965+0000 7f8fb89e1700 1 -- 172.21.15.112:0/2635426182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f9c005330 msgr2=0x7f8f9c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.965+0000 7f8fb89e1700 1 --2- 172.21.15.112:0/2635426182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f9c005330 0x7f8f9c00ba10 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00c160 tx=0x7f8f9c005880 comp rx=0 tx=0).stop 2023-08-27T11:29:51.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.965+0000 7f8fb89e1700 1 -- 172.21.15.112:0/2635426182 shutdown_connections 2023-08-27T11:29:51.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.965+0000 7f8fb89e1700 1 --2- 172.21.15.112:0/2635426182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f9c005330 0x7f8f9c00ba10 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.965+0000 7f8fb89e1700 1 -- 172.21.15.112:0/2635426182 >> 172.21.15.112:0/2635426182 conn(0x7f8fb40d0000 msgr2=0x7f8fb40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:51.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.965+0000 7f8fb89e1700 1 -- 172.21.15.112:0/2635426182 shutdown_connections 2023-08-27T11:29:51.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.965+0000 7f8fb89e1700 1 -- 172.21.15.112:0/2635426182 wait complete. 2023-08-27T11:29:51.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.966+0000 7f8fb89e1700 1 Processor -- start 2023-08-27T11:29:51.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.966+0000 7f8fb89e1700 1 -- start start 2023-08-27T11:29:51.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.967+0000 7f8fb89e1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f9c005330 0x7f8fb40c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.967+0000 7f8fb89e1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fb4144ec0 0x7f8fb4145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.967+0000 7f8fb89e1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb40c70d0 0x7f8fb40c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.967+0000 7f8fb89e1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb40c88c0 con 0x7f8fb40c70d0 2023-08-27T11:29:51.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.967+0000 7f8fb89e1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb40c6d00 con 0x7f8fb4144ec0 2023-08-27T11:29:51.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.967+0000 7f8fb89e1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb40b06f0 con 0x7f8f9c005330 2023-08-27T11:29:51.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.967+0000 7f8fb2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb40c70d0 0x7f8fb40c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.967+0000 7f8fb2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb40c70d0 0x7f8fb40c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38872/0 (socket says 172.21.15.112:38872) 2023-08-27T11:29:51.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.967+0000 7f8fb2d9d700 1 -- 172.21.15.112:0/128156958 learned_addr learned my addr 172.21.15.112:0/128156958 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:51.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.967+0000 7f8fb1d9b700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fb4144ec0 0x7f8fb4145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.967+0000 7f8fb259c700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f9c005330 0x7f8fb40c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.968+0000 7f8fb2d9d700 1 -- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f9c005330 msgr2=0x7f8fb40c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.968+0000 7f8fb2d9d700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f9c005330 0x7f8fb40c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.968+0000 7f8fb2d9d700 1 -- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fb4144ec0 msgr2=0x7f8fb4145340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:51.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.968+0000 7f8fb2d9d700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fb4144ec0 0x7f8fb4145340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:51.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.968+0000 7f8fb2d9d700 1 -- 172.21.15.112:0/128156958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb4147d50 con 0x7f8fb40c70d0 2023-08-27T11:29:51.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.968+0000 7f8fb1d9b700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fb4144ec0 0x7f8fb4145340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:51.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.968+0000 7f8fb2d9d700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb40c70d0 0x7f8fb40c7560 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f8fb406c130 tx=0x7f8fac0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:51.982 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.968+0000 7f8fa2ffd700 1 -- 172.21.15.112:0/128156958 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac005070 con 0x7f8fb40c70d0 2023-08-27T11:29:51.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.968+0000 7f8fa2ffd700 1 -- 172.21.15.112:0/128156958 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac0051d0 con 0x7f8fb40c70d0 2023-08-27T11:29:51.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.968+0000 7f8fa2ffd700 1 -- 172.21.15.112:0/128156958 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac005540 con 0x7f8fb40c70d0 2023-08-27T11:29:51.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.968+0000 7f8fb89e1700 1 -- 172.21.15.112:0/128156958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb40c7c40 con 0x7f8fb40c70d0 2023-08-27T11:29:51.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.968+0000 7f8fb89e1700 1 -- 172.21.15.112:0/128156958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb40c8200 con 0x7f8fb40c70d0 2023-08-27T11:29:51.983 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.969+0000 7f8fb89e1700 1 -- 172.21.15.112:0/128156958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb40c8a00 con 0x7f8fb40c70d0 2023-08-27T11:29:51.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.969+0000 7f8fa2ffd700 1 -- 172.21.15.112:0/128156958 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f8fac0056a0 con 0x7f8fb40c70d0 2023-08-27T11:29:51.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.970+0000 7f8fa2ffd700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f9406a4d0 0x7f8f9406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:51.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.970+0000 7f8fa2ffd700 1 -- 172.21.15.112:0/128156958 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f8fac089e10 con 0x7f8fb40c70d0 2023-08-27T11:29:51.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.970+0000 7f8fb259c700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f9406a4d0 0x7f8f9406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:51.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.970+0000 7f8fb259c700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f9406a4d0 0x7f8f9406c990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4007c10 tx=0x7f8fa4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:51.984 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:51.974+0000 7f8fa2ffd700 1 -- 172.21.15.112:0/128156958 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8fb40c8a00 con 0x7f8fb40c70d0 2023-08-27T11:29:52.207 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.206+0000 7f8fb89e1700 1 -- 172.21.15.112:0/128156958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fb40c8a00 con 0x7f8fb40c70d0 2023-08-27T11:29:52.208 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.207+0000 7f8fa2ffd700 1 -- 172.21.15.112:0/128156958 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4914 (secure 0 0 0) 0x7f8fb40c8a00 con 0x7f8fb40c70d0 2023-08-27T11:29:52.208 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:52.208 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/685371648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":685371648},{"type":"v1","addr":"172.21.15.138:6836","nonce":685371648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":7214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3585152138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3585152138},{"type":"v1","addr":"172.21.15.138:6837","nonce":3585152138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":8203,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2190275630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2190275630},{"type":"v1","addr":"172.21.15.112:6835","nonce":2190275630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:29:22.938324+0000","modified":"2023-08-27T11:29:46.388631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6833/4086629952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4086629952},{"type":"v1","addr":"172.21.15.112:6833","nonce":4086629952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"volume_0000000001022841","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-08-27T11:29:52.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.208+0000 7f8fb89e1700 1 -- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f9406a4d0 msgr2=0x7f8f9406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.208+0000 7f8fb89e1700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f9406a4d0 0x7f8f9406c990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4007c10 tx=0x7f8fa4009040 comp rx=0 tx=0).stop 2023-08-27T11:29:52.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.209+0000 7f8fb89e1700 1 -- 172.21.15.112:0/128156958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb40c70d0 msgr2=0x7f8fb40c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.210 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.209+0000 7f8fb89e1700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb40c70d0 0x7f8fb40c7560 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f8fb406c130 tx=0x7f8fac0014a0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.209+0000 7f8fb89e1700 1 -- 172.21.15.112:0/128156958 shutdown_connections 2023-08-27T11:29:52.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.209+0000 7f8fb89e1700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8f9406a4d0 0x7f8f9406c990 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.209+0000 7f8fb89e1700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb40c70d0 0x7f8fb40c7560 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.211 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.209+0000 7f8fb89e1700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f9c005330 0x7f8fb40c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.209+0000 7f8fb89e1700 1 --2- 172.21.15.112:0/128156958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fb4144ec0 0x7f8fb4145340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.209+0000 7f8fb89e1700 1 -- 172.21.15.112:0/128156958 >> 172.21.15.112:0/128156958 conn(0x7f8fb40d0000 msgr2=0x7f8fb40cb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:52.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.209+0000 7f8fb89e1700 1 -- 172.21.15.112:0/128156958 shutdown_connections 2023-08-27T11:29:52.212 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.209+0000 7f8fb89e1700 1 -- 172.21.15.112:0/128156958 wait complete. 2023-08-27T11:29:52.213 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 116 2023-08-27T11:29:52.222 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:29:52.341 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.340+0000 7f80335c9700 1 Processor -- start 2023-08-27T11:29:52.342 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.340+0000 7f80335c9700 1 -- start start 2023-08-27T11:29:52.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.341+0000 7f80335c9700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f802c0cddb0 0x7f802c15f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:52.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.341+0000 7f80335c9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0ce190 0x7f802c15b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:52.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.341+0000 7f80335c9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c15eed0 con 0x7f802c15bdc0 2023-08-27T11:29:52.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.341+0000 7f80335c9700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802c15f500 con 0x7f802c0cddb0 2023-08-27T11:29:52.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.341+0000 7f80335c9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c15c200 con 0x7f802c0ce190 2023-08-27T11:29:52.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.341+0000 7f8030b64700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0ce190 0x7f802c15b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:52.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.341+0000 7f8031365700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f802c0cddb0 0x7f802c15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:52.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.341+0000 7f8030b64700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0ce190 0x7f802c15b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:50530/0 (socket says 172.21.15.112:50530) 2023-08-27T11:29:52.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.341+0000 7f8031b66700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f802c15bdc0 0x7f802c0cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:40786/0 (socket says 172.21.15.112:40786) 2023-08-27T11:29:52.345 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.341+0000 7f8030b64700 1 -- 172.21.15.112:0/2322566027 learned_addr learned my addr 172.21.15.112:0/2322566027 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:52.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.341+0000 7f8031365700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f802c0cddb0 0x7f802c15f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:39302/0 (socket says 172.21.15.112:39302) 2023-08-27T11:29:52.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.341+0000 7f8031365700 1 -- 172.21.15.112:0/2322566027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0ce190 msgr2=0x7f802c15b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:52.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.341+0000 7f8031365700 1 --2- 172.21.15.112:0/2322566027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0ce190 0x7f802c15b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f8031365700 1 -- 172.21.15.112:0/2322566027 >> v1:172.21.15.138:6789/0 conn(0x7f802c15bdc0 legacy=0x7f802c0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.346 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f8023fff700 1 -- 172.21.15.112:0/2322566027 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680929106 0 0) 0x7f802c15eed0 con 0x7f802c15bdc0 2023-08-27T11:29:52.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f8031365700 1 -- 172.21.15.112:0/2322566027 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c15fe60 con 0x7f802c0cddb0 2023-08-27T11:29:52.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f8030b64700 1 --2- 172.21.15.112:0/2322566027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0ce190 0x7f802c15b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:52.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f8031365700 1 --2- 172.21.15.112:0/2322566027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0cddb0 0x7f802c15f920 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8024009840 tx=0x7f8024009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b786af97f8cc5361 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:52.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f8023fff700 1 -- 172.21.15.112:0/2322566027 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024011830 con 0x7f802c0cddb0 2023-08-27T11:29:52.347 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f8023fff700 1 -- 172.21.15.112:0/2322566027 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8024011990 con 0x7f802c0cddb0 2023-08-27T11:29:52.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f8023fff700 1 -- 172.21.15.112:0/2322566027 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024011d20 con 0x7f802c0cddb0 2023-08-27T11:29:52.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f80335c9700 1 -- 172.21.15.112:0/2322566027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0cddb0 msgr2=0x7f802c15f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f80335c9700 1 --2- 172.21.15.112:0/2322566027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0cddb0 0x7f802c15f920 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8024009840 tx=0x7f8024009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:52.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f80335c9700 1 -- 172.21.15.112:0/2322566027 shutdown_connections 2023-08-27T11:29:52.348 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f80335c9700 1 --2- 172.21.15.112:0/2322566027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0cddb0 0x7f802c15f920 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f80335c9700 1 --2- 172.21.15.112:0/2322566027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0ce190 0x7f802c15b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.342+0000 7f80335c9700 1 -- 172.21.15.112:0/2322566027 >> 172.21.15.112:0/2322566027 conn(0x7f802c06f970 msgr2=0x7f802c06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:52.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.343+0000 7f80335c9700 1 -- 172.21.15.112:0/2322566027 shutdown_connections 2023-08-27T11:29:52.349 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.343+0000 7f80335c9700 1 -- 172.21.15.112:0/2322566027 wait complete. 2023-08-27T11:29:52.351 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.344+0000 7f80335c9700 1 Processor -- start 2023-08-27T11:29:52.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.344+0000 7f80335c9700 1 -- start start 2023-08-27T11:29:52.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.344+0000 7f80335c9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0cddb0 0x7f802c152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:52.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.344+0000 7f80335c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0ce190 0x7f802c151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:52.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.344+0000 7f80335c9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c15bdc0 0x7f802c0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:52.352 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.344+0000 7f80335c9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c15bc80 con 0x7f802c15bdc0 2023-08-27T11:29:52.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.344+0000 7f80335c9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c156bd0 con 0x7f802c0ce190 2023-08-27T11:29:52.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.344+0000 7f80335c9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c1533e0 con 0x7f802c0cddb0 2023-08-27T11:29:52.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8031b66700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c15bdc0 0x7f802c0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:52.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8031b66700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c15bdc0 0x7f802c0ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38886/0 (socket says 172.21.15.112:38886) 2023-08-27T11:29:52.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8031b66700 1 -- 172.21.15.112:0/1801555608 learned_addr learned my addr 172.21.15.112:0/1801555608 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:52.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8030b64700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0ce190 0x7f802c151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:52.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8031365700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0cddb0 0x7f802c152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:52.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8031b66700 1 -- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0cddb0 msgr2=0x7f802c152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8031b66700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0cddb0 0x7f802c152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.354 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8031b66700 1 -- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0ce190 msgr2=0x7f802c151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8031b66700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0ce190 0x7f802c151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8031b66700 1 -- 172.21.15.112:0/1801555608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c009550 con 0x7f802c15bdc0 2023-08-27T11:29:52.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8031b66700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c15bdc0 0x7f802c0ca510 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f802800cba0 tx=0x7f802800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:52.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8021ffb700 1 -- 172.21.15.112:0/1801555608 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802800aaa0 con 0x7f802c15bdc0 2023-08-27T11:29:52.355 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8021ffb700 1 -- 172.21.15.112:0/1801555608 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802800ac20 con 0x7f802c15bdc0 2023-08-27T11:29:52.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f8021ffb700 1 -- 172.21.15.112:0/1801555608 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80280070d0 con 0x7f802c15bdc0 2023-08-27T11:29:52.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f80335c9700 1 -- 172.21.15.112:0/1801555608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80240094a0 con 0x7f802c15bdc0 2023-08-27T11:29:52.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.345+0000 7f80335c9700 1 -- 172.21.15.112:0/1801555608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c152a40 con 0x7f802c15bdc0 2023-08-27T11:29:52.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.346+0000 7f80335c9700 1 -- 172.21.15.112:0/1801555608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c1e7f10 con 0x7f802c15bdc0 2023-08-27T11:29:52.356 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.347+0000 7f8021ffb700 1 -- 172.21.15.112:0/1801555608 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f8028007230 con 0x7f802c15bdc0 2023-08-27T11:29:52.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.347+0000 7f8021ffb700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f801006a4d0 0x7f801006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:52.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.347+0000 7f8021ffb700 1 -- 172.21.15.112:0/1801555608 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f8028088870 con 0x7f802c15bdc0 2023-08-27T11:29:52.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.347+0000 7f8031365700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f801006a4d0 0x7f801006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:52.357 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.348+0000 7f8031365700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f801006a4d0 0x7f801006c990 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f80240090b0 tx=0x7f8024008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:52.358 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.351+0000 7f8021ffb700 1 -- 172.21.15.112:0/1801555608 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8028053830 con 0x7f802c15bdc0 2023-08-27T11:29:52.586 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.585+0000 7f80335c9700 1 -- 172.21.15.112:0/1801555608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f802c067a20 con 0x7f802c15bdc0 2023-08-27T11:29:52.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.586+0000 7f8021ffb700 1 -- 172.21.15.112:0/1801555608 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4914 (secure 0 0 0) 0x7f8028056e50 con 0x7f802c15bdc0 2023-08-27T11:29:52.587 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:52.587 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/685371648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":685371648},{"type":"v1","addr":"172.21.15.138:6836","nonce":685371648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":7214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3585152138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3585152138},{"type":"v1","addr":"172.21.15.138:6837","nonce":3585152138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":8203,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2190275630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2190275630},{"type":"v1","addr":"172.21.15.112:6835","nonce":2190275630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:29:22.938324+0000","modified":"2023-08-27T11:29:46.388631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6833/4086629952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4086629952},{"type":"v1","addr":"172.21.15.112:6833","nonce":4086629952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"volume_0000000001022841","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-08-27T11:29:52.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.588+0000 7f80335c9700 1 -- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f801006a4d0 msgr2=0x7f801006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.588+0000 7f80335c9700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f801006a4d0 0x7f801006c990 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f80240090b0 tx=0x7f8024008030 comp rx=0 tx=0).stop 2023-08-27T11:29:52.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.588+0000 7f80335c9700 1 -- 172.21.15.112:0/1801555608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c15bdc0 msgr2=0x7f802c0ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.588+0000 7f80335c9700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c15bdc0 0x7f802c0ca510 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f802800cba0 tx=0x7f802800ceb0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.588+0000 7f80335c9700 1 -- 172.21.15.112:0/1801555608 shutdown_connections 2023-08-27T11:29:52.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.588+0000 7f80335c9700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f801006a4d0 0x7f801006c990 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.588+0000 7f80335c9700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c15bdc0 0x7f802c0ca510 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.588+0000 7f80335c9700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0cddb0 0x7f802c152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.588+0000 7f80335c9700 1 --2- 172.21.15.112:0/1801555608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0ce190 0x7f802c151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.588+0000 7f80335c9700 1 -- 172.21.15.112:0/1801555608 >> 172.21.15.112:0/1801555608 conn(0x7f802c06f970 msgr2=0x7f802c0719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:52.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.588+0000 7f80335c9700 1 -- 172.21.15.112:0/1801555608 shutdown_connections 2023-08-27T11:29:52.591 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.588+0000 7f80335c9700 1 -- 172.21.15.112:0/1801555608 wait complete. 2023-08-27T11:29:52.592 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 116 2023-08-27T11:29:52.601 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:52.721 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.720+0000 7f6827ee2700 1 Processor -- start 2023-08-27T11:29:52.722 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.720+0000 7f6827ee2700 1 -- start start 2023-08-27T11:29:52.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.721+0000 7f6827ee2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6820151060 0x7f682015e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:52.724 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.721+0000 7f6827ee2700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f682015e990 0x7f682006adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:52.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.721+0000 7f6827ee2700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68200b02f0 con 0x7f682006b2f0 2023-08-27T11:29:52.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.721+0000 7f6827ee2700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682006b780 con 0x7f682015e990 2023-08-27T11:29:52.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.721+0000 7f6827ee2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68200b05d0 con 0x7f6820151060 2023-08-27T11:29:52.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.721+0000 7f6825c7e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6820151060 0x7f682015e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:52.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.721+0000 7f682547d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f682015e990 0x7f682006adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:52.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.721+0000 7f682647f700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f682006b2f0 0x7f682006d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:40800/0 (socket says 172.21.15.112:40800) 2023-08-27T11:29:52.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.721+0000 7f6825c7e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6820151060 0x7f682015e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.112:50544/0 (socket says 172.21.15.112:50544) 2023-08-27T11:29:52.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.721+0000 7f682547d700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f682015e990 0x7f682006adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:39316/0 (socket says 172.21.15.112:39316) 2023-08-27T11:29:52.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.721+0000 7f682647f700 1 -- 172.21.15.112:0/2199368330 learned_addr learned my addr 172.21.15.112:0/2199368330 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:52.726 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.722+0000 7f6824c7c700 1 -- 172.21.15.112:0/2199368330 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184348679 0 0) 0x7f68200b02f0 con 0x7f682006b2f0 2023-08-27T11:29:52.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.722+0000 7f6824c7c700 1 -- 172.21.15.112:0/2199368330 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f680c0035b0 con 0x7f682006b2f0 2023-08-27T11:29:52.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.722+0000 7f6825c7e700 1 -- 172.21.15.112:0/2199368330 >> v2:172.21.15.138:3300/0 conn(0x7f682015e990 msgr2=0x7f682006adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:52.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.722+0000 7f6825c7e700 1 --2- 172.21.15.112:0/2199368330 >> v2:172.21.15.138:3300/0 conn(0x7f682015e990 0x7f682006adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.722+0000 7f6825c7e700 1 -- 172.21.15.112:0/2199368330 >> v1:172.21.15.138:6789/0 conn(0x7f682006b2f0 legacy=0x7f682006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.722+0000 7f6825c7e700 1 -- 172.21.15.112:0/2199368330 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68200b08b0 con 0x7f6820151060 2023-08-27T11:29:52.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.722+0000 7f682547d700 1 --2- 172.21.15.112:0/2199368330 >> v2:172.21.15.138:3300/0 conn(0x7f682015e990 0x7f682006adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:52.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.722+0000 7f6825c7e700 1 --2- 172.21.15.112:0/2199368330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6820151060 0x7f682015e450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6810009840 tx=0x7f6810009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c69ce459b253a19 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:52.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.722+0000 7f6824c7c700 1 -- 172.21.15.112:0/2199368330 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6810011830 con 0x7f6820151060 2023-08-27T11:29:52.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.722+0000 7f6824c7c700 1 -- 172.21.15.112:0/2199368330 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6810011990 con 0x7f6820151060 2023-08-27T11:29:52.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.722+0000 7f6824c7c700 1 -- 172.21.15.112:0/2199368330 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6810011d20 con 0x7f6820151060 2023-08-27T11:29:52.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.723+0000 7f6827ee2700 1 -- 172.21.15.112:0/2199368330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6820151060 msgr2=0x7f682015e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.723+0000 7f6827ee2700 1 --2- 172.21.15.112:0/2199368330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6820151060 0x7f682015e450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6810009840 tx=0x7f6810009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:52.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.723+0000 7f6827ee2700 1 -- 172.21.15.112:0/2199368330 shutdown_connections 2023-08-27T11:29:52.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.723+0000 7f6827ee2700 1 --2- 172.21.15.112:0/2199368330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6820151060 0x7f682015e450 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.723+0000 7f6827ee2700 1 --2- 172.21.15.112:0/2199368330 >> v2:172.21.15.138:3300/0 conn(0x7f682015e990 0x7f682006adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.723+0000 7f6827ee2700 1 -- 172.21.15.112:0/2199368330 >> 172.21.15.112:0/2199368330 conn(0x7f68200d0000 msgr2=0x7f68200d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:52.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.723+0000 7f6827ee2700 1 -- 172.21.15.112:0/2199368330 shutdown_connections 2023-08-27T11:29:52.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.723+0000 7f6827ee2700 1 -- 172.21.15.112:0/2199368330 wait complete. 2023-08-27T11:29:52.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.724+0000 7f6827ee2700 1 Processor -- start 2023-08-27T11:29:52.732 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.724+0000 7f6827ee2700 1 -- start start 2023-08-27T11:29:52.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f6827ee2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f682006b2f0 0x7f68200c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:52.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f6827ee2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6820151060 0x7f68200c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:52.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f6827ee2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f682015e990 0x7f682014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:52.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f6827ee2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68200c4da0 con 0x7f682015e990 2023-08-27T11:29:52.733 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f6827ee2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68200c4220 con 0x7f682006b2f0 2023-08-27T11:29:52.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f6827ee2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68200c4500 con 0x7f6820151060 2023-08-27T11:29:52.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f682647f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f682015e990 0x7f682014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:52.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f682647f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f682015e990 0x7f682014f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38900/0 (socket says 172.21.15.112:38900) 2023-08-27T11:29:52.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f682647f700 1 -- 172.21.15.112:0/223320100 learned_addr learned my addr 172.21.15.112:0/223320100 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:52.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f6825c7e700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f682006b2f0 0x7f68200c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:52.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f682547d700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6820151060 0x7f68200c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:52.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f682647f700 1 -- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6820151060 msgr2=0x7f68200c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f682647f700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6820151060 0x7f68200c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f682647f700 1 -- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f682006b2f0 msgr2=0x7f68200c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f682647f700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f682006b2f0 0x7f68200c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f682647f700 1 -- 172.21.15.112:0/223320100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c00b740 con 0x7f682015e990 2023-08-27T11:29:52.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f682647f700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f682015e990 0x7f682014f3e0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f681800caa0 tx=0x7f681800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:52.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f68167fc700 1 -- 172.21.15.112:0/223320100 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681800a9e0 con 0x7f682015e990 2023-08-27T11:29:52.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f68167fc700 1 -- 172.21.15.112:0/223320100 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681800ab40 con 0x7f682015e990 2023-08-27T11:29:52.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f6827ee2700 1 -- 172.21.15.112:0/223320100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68100094a0 con 0x7f682015e990 2023-08-27T11:29:52.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.725+0000 7f68167fc700 1 -- 172.21.15.112:0/223320100 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818012400 con 0x7f682015e990 2023-08-27T11:29:52.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.726+0000 7f6827ee2700 1 -- 172.21.15.112:0/223320100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68200c4ac0 con 0x7f682015e990 2023-08-27T11:29:52.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.726+0000 7f6827ee2700 1 -- 172.21.15.112:0/223320100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68200c5010 con 0x7f682015e990 2023-08-27T11:29:52.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.727+0000 7f68167fc700 1 -- 172.21.15.112:0/223320100 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f681801b3b0 con 0x7f682015e990 2023-08-27T11:29:52.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.727+0000 7f68167fc700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f680406a4d0 0x7f680406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:52.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.728+0000 7f68167fc700 1 -- 172.21.15.112:0/223320100 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f68180889b0 con 0x7f682015e990 2023-08-27T11:29:52.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.728+0000 7f6825c7e700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f680406a4d0 0x7f680406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:52.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.728+0000 7f6825c7e700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f680406a4d0 0x7f680406c990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6810000c00 tx=0x7f6810008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:52.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.732+0000 7f68167fc700 1 -- 172.21.15.112:0/223320100 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f68200c5010 con 0x7f682015e990 2023-08-27T11:29:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.921+0000 7f6827ee2700 1 -- 172.21.15.112:0/223320100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68200c5010 con 0x7f682015e990 2023-08-27T11:29:52.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.923+0000 7f68167fc700 1 -- 172.21.15.112:0/223320100 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+20116 (secure 0 0 0) 0x7f68200c5010 con 0x7f682015e990 2023-08-27T11:29:52.924 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:52.924 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":172,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:47.398430+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs.volume_0000000001022841.meta","create_time":"2023-08-27T11:29:20.569324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000001022841"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-08-27T11:29:21.905812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000001022841"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"another-data-pool","create_time":"2023-08-27T11:29:43.100842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000001022841"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:52.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.925+0000 7f6827ee2700 1 -- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f680406a4d0 msgr2=0x7f680406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.925+0000 7f6827ee2700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f680406a4d0 0x7f680406c990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6810000c00 tx=0x7f6810008030 comp rx=0 tx=0).stop 2023-08-27T11:29:52.926 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.925+0000 7f6827ee2700 1 -- 172.21.15.112:0/223320100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f682015e990 msgr2=0x7f682014f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:52.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.925+0000 7f6827ee2700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f682015e990 0x7f682014f3e0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f681800caa0 tx=0x7f681800cdb0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.925+0000 7f6827ee2700 1 -- 172.21.15.112:0/223320100 shutdown_connections 2023-08-27T11:29:52.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.925+0000 7f6827ee2700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f680406a4d0 0x7f680406c990 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.925+0000 7f6827ee2700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f682015e990 0x7f682014f3e0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.925+0000 7f6827ee2700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f682006b2f0 0x7f68200c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.925+0000 7f6827ee2700 1 --2- 172.21.15.112:0/223320100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6820151060 0x7f68200c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:52.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.925+0000 7f6827ee2700 1 -- 172.21.15.112:0/223320100 >> 172.21.15.112:0/223320100 conn(0x7f68200d0000 msgr2=0x7f68200b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:52.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.925+0000 7f6827ee2700 1 -- 172.21.15.112:0/223320100 shutdown_connections 2023-08-27T11:29:52.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:52.925+0000 7f6827ee2700 1 -- 172.21.15.112:0/223320100 wait complete. 2023-08-27T11:29:52.938 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000001022841 2023-08-27T11:29:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.057+0000 7f0ec084e700 1 Processor -- start 2023-08-27T11:29:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.057+0000 7f0ec084e700 1 -- start start 2023-08-27T11:29:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.058+0000 7f0ec084e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ebc066450 0x7f0ebc0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.058+0000 7f0ec084e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0ebc06e150 0x7f0ebc0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:53.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.058+0000 7f0ec084e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ebc0ad580 con 0x7f0ebc15d490 2023-08-27T11:29:53.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.058+0000 7f0ec084e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ebc0ad260 con 0x7f0ebc06e150 2023-08-27T11:29:53.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.058+0000 7f0ec084e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc06f790 con 0x7f0ebc066450 2023-08-27T11:29:53.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0eba59c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0ebc15d490 0x7f0ebc15d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33286/0 (socket says 172.21.15.112:33286) 2023-08-27T11:29:53.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0eba59c700 1 -- 172.21.15.112:0/1067402138 learned_addr learned my addr 172.21.15.112:0/1067402138 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:53.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0ebad9d700 1 --2- 172.21.15.112:0/1067402138 >> v2:172.21.15.112:3300/0 conn(0x7f0ebc06e150 0x7f0ebc0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:53.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0eb9d9b700 1 --2- 172.21.15.112:0/1067402138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ebc066450 0x7f0ebc0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:53.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0eb959a700 1 -- 172.21.15.112:0/1067402138 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837269054 0 0) 0x7f0ebc0ad580 con 0x7f0ebc15d490 2023-08-27T11:29:53.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0eb959a700 1 -- 172.21.15.112:0/1067402138 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ea40035b0 con 0x7f0ebc15d490 2023-08-27T11:29:53.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0ebad9d700 1 -- 172.21.15.112:0/1067402138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ebc066450 msgr2=0x7f0ebc0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:53.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0ebad9d700 1 --2- 172.21.15.112:0/1067402138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ebc066450 0x7f0ebc0701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0ebad9d700 1 -- 172.21.15.112:0/1067402138 >> v1:172.21.15.112:6789/0 conn(0x7f0ebc15d490 legacy=0x7f0ebc15d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:53.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0ebad9d700 1 -- 172.21.15.112:0/1067402138 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc06fa70 con 0x7f0ebc06e150 2023-08-27T11:29:53.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0ebad9d700 1 --2- 172.21.15.112:0/1067402138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ebc06e150 0x7f0ebc0706e0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009d90 tx=0x7f0eb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c476c9739629f9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:53.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0eb959a700 1 -- 172.21.15.112:0/1067402138 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb400e070 con 0x7f0ebc06e150 2023-08-27T11:29:53.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0eb959a700 1 -- 172.21.15.112:0/1067402138 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb400bb20 con 0x7f0ebc06e150 2023-08-27T11:29:53.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.059+0000 7f0eb959a700 1 -- 172.21.15.112:0/1067402138 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4012430 con 0x7f0ebc06e150 2023-08-27T11:29:53.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.060+0000 7f0ec084e700 1 -- 172.21.15.112:0/1067402138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ebc06e150 msgr2=0x7f0ebc0706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:53.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.060+0000 7f0ec084e700 1 --2- 172.21.15.112:0/1067402138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ebc06e150 0x7f0ebc0706e0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009d90 tx=0x7f0eb400b330 comp rx=0 tx=0).stop 2023-08-27T11:29:53.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.060+0000 7f0ec084e700 1 -- 172.21.15.112:0/1067402138 shutdown_connections 2023-08-27T11:29:53.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.060+0000 7f0ec084e700 1 --2- 172.21.15.112:0/1067402138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ebc06e150 0x7f0ebc0706e0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.060+0000 7f0ec084e700 1 --2- 172.21.15.112:0/1067402138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ebc066450 0x7f0ebc0701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.060+0000 7f0ec084e700 1 -- 172.21.15.112:0/1067402138 >> 172.21.15.112:0/1067402138 conn(0x7f0ebc0b1380 msgr2=0x7f0ebc0b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.060+0000 7f0ec084e700 1 -- 172.21.15.112:0/1067402138 shutdown_connections 2023-08-27T11:29:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.060+0000 7f0ec084e700 1 -- 172.21.15.112:0/1067402138 wait complete. 2023-08-27T11:29:53.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.061+0000 7f0ec084e700 1 Processor -- start 2023-08-27T11:29:53.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.061+0000 7f0ec084e700 1 -- start start 2023-08-27T11:29:53.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0ec084e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ebc066450 0x7f0ebc1401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:53.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0ec084e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ebc06e150 0x7f0ebc14fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:53.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0ec084e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ebc15d490 0x7f0ebc1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:53.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0ec084e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc15cb70 con 0x7f0ebc066450 2023-08-27T11:29:53.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0ec084e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc1406e0 con 0x7f0ebc15d490 2023-08-27T11:29:53.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0ec084e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc15bad0 con 0x7f0ebc06e150 2023-08-27T11:29:53.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0eba59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ebc066450 0x7f0ebc1401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:53.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0eba59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ebc066450 0x7f0ebc1401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38910/0 (socket says 172.21.15.112:38910) 2023-08-27T11:29:53.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0eba59c700 1 -- 172.21.15.112:0/1442682314 learned_addr learned my addr 172.21.15.112:0/1442682314 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:53.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0eb9d9b700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ebc06e150 0x7f0ebc14fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:53.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0eba59c700 1 -- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ebc06e150 msgr2=0x7f0ebc14fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:53.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0ebad9d700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ebc15d490 0x7f0ebc1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:53.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0eba59c700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ebc06e150 0x7f0ebc14fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0eba59c700 1 -- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ebc15d490 msgr2=0x7f0ebc1500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:53.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0eba59c700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ebc15d490 0x7f0ebc1500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0eba59c700 1 -- 172.21.15.112:0/1442682314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb4009a70 con 0x7f0ebc066450 2023-08-27T11:29:53.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.062+0000 7f0eba59c700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ebc066450 0x7f0ebc1401a0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f0eac001eb0 tx=0x7f0eac00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:53.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.063+0000 7f0eaaffd700 1 -- 172.21.15.112:0/1442682314 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac005d70 con 0x7f0ebc066450 2023-08-27T11:29:53.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.063+0000 7f0eaaffd700 1 -- 172.21.15.112:0/1442682314 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac0070d0 con 0x7f0ebc066450 2023-08-27T11:29:53.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.063+0000 7f0ec084e700 1 -- 172.21.15.112:0/1442682314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ebc15bdb0 con 0x7f0ebc066450 2023-08-27T11:29:53.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.063+0000 7f0eaaffd700 1 -- 172.21.15.112:0/1442682314 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac007460 con 0x7f0ebc066450 2023-08-27T11:29:53.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.063+0000 7f0ec084e700 1 -- 172.21.15.112:0/1442682314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ebc15c370 con 0x7f0ebc066450 2023-08-27T11:29:53.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.064+0000 7f0ec084e700 1 -- 172.21.15.112:0/1442682314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ebc067a20 con 0x7f0ebc066450 2023-08-27T11:29:53.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.064+0000 7f0eaaffd700 1 -- 172.21.15.112:0/1442682314 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f0eac0075c0 con 0x7f0ebc066450 2023-08-27T11:29:53.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.064+0000 7f0eaaffd700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e9c06a4d0 0x7f0e9c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:53.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.065+0000 7f0eaaffd700 1 -- 172.21.15.112:0/1442682314 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6537+0+0 (secure 0 0 0) 0x7f0eac088420 con 0x7f0ebc066450 2023-08-27T11:29:53.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.065+0000 7f0eb9d9b700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e9c06a4d0 0x7f0e9c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:53.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.065+0000 7f0eb9d9b700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e9c06a4d0 0x7f0e9c06c990 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc161e90 tx=0x7f0eb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:53.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.069+0000 7f0eaaffd700 1 -- 172.21.15.112:0/1442682314 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0eac053310 con 0x7f0ebc066450 2023-08-27T11:29:53.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.298+0000 7f0ec084e700 1 -- 172.21.15.112:0/1442682314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000001022841"} v 0) v1 -- 0x7f0ebc066230 con 0x7f0ebc066450 2023-08-27T11:29:53.300 INFO:tasks.ceph.mon.a.smithi112.stderr:2023-08-27T11:29:53.298+0000 7f4d6178c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:29:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.303+0000 7f0eaaffd700 1 -- 172.21.15.112:0/1442682314 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000001022841"}]=0 volume_0000000001022841 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v117) v1 ==== 195+0+0 (secure 0 0 0) 0x7f0eac056930 con 0x7f0ebc066450 2023-08-27T11:29:53.306 INFO:tasks.ceph.mds.c.smithi112.stderr: -3540> 2023-08-27T11:29:27.517+0000 7f89ac94b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:29:53.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.306+0000 7f0ec084e700 1 -- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e9c06a4d0 msgr2=0x7f0e9c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:53.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.306+0000 7f0ec084e700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e9c06a4d0 0x7f0e9c06c990 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc161e90 tx=0x7f0eb0008040 comp rx=0 tx=0).stop 2023-08-27T11:29:53.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.306+0000 7f0ec084e700 1 -- 172.21.15.112:0/1442682314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ebc066450 msgr2=0x7f0ebc1401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:53.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.306+0000 7f0ec084e700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ebc066450 0x7f0ebc1401a0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f0eac001eb0 tx=0x7f0eac00c860 comp rx=0 tx=0).stop 2023-08-27T11:29:53.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.306+0000 7f0ec084e700 1 -- 172.21.15.112:0/1442682314 shutdown_connections 2023-08-27T11:29:53.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.306+0000 7f0ec084e700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0e9c06a4d0 0x7f0e9c06c990 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.306+0000 7f0ec084e700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ebc15d490 0x7f0ebc1500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.306+0000 7f0ec084e700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ebc066450 0x7f0ebc1401a0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.306+0000 7f0ec084e700 1 --2- 172.21.15.112:0/1442682314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ebc06e150 0x7f0ebc14fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.306+0000 7f0ec084e700 1 -- 172.21.15.112:0/1442682314 >> 172.21.15.112:0/1442682314 conn(0x7f0ebc0b1380 msgr2=0x7f0ebc151610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:53.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.306+0000 7f0ec084e700 1 -- 172.21.15.112:0/1442682314 shutdown_connections 2023-08-27T11:29:53.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.306+0000 7f0ec084e700 1 -- 172.21.15.112:0/1442682314 wait complete. 2023-08-27T11:29:53.310 INFO:teuthology.orchestra.run.smithi112.stderr:volume_0000000001022841 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:29:53.320 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000001022841 --yes-i-really-mean-it 2023-08-27T11:29:53.334 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:29:53.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.442+0000 7f630f577700 1 Processor -- start 2023-08-27T11:29:53.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.442+0000 7f630f577700 1 -- start start 2023-08-27T11:29:53.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.443+0000 7f630f577700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6308071980 0x7f63080cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:53.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.443+0000 7f630f577700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63080ccea0 0x7f630815ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:53.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.443+0000 7f630f577700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63080cd2e0 con 0x7f630815ea00 2023-08-27T11:29:53.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.443+0000 7f630f577700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630815e5e0 con 0x7f6308071980 2023-08-27T11:29:53.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.443+0000 7f630f577700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630815b510 con 0x7f63080ccea0 2023-08-27T11:29:53.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.443+0000 7f630d313700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6308071980 0x7f63080cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:53.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.443+0000 7f630db14700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63080ccea0 0x7f630815ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:53.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.443+0000 7f630cb12700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f630815ea00 0x7f630815a760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:40836/0 (socket says 172.21.15.112:40836) 2023-08-27T11:29:53.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.443+0000 7f630d313700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6308071980 0x7f63080cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:39356/0 (socket says 172.21.15.112:39356) 2023-08-27T11:29:53.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.443+0000 7f630cb12700 1 -- 172.21.15.112:0/1295006331 learned_addr learned my addr 172.21.15.112:0/1295006331 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:53.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.444+0000 7f630d313700 1 -- 172.21.15.112:0/1295006331 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63080ccea0 msgr2=0x7f630815ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:53.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.444+0000 7f630d313700 1 --2- 172.21.15.112:0/1295006331 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63080ccea0 0x7f630815ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.444+0000 7f630d313700 1 -- 172.21.15.112:0/1295006331 >> v1:172.21.15.138:6789/0 conn(0x7f630815ea00 legacy=0x7f630815a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:53.448 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.444+0000 7f630d313700 1 -- 172.21.15.112:0/1295006331 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630815df20 con 0x7f6308071980 2023-08-27T11:29:53.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.444+0000 7f630db14700 1 --2- 172.21.15.112:0/1295006331 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63080ccea0 0x7f630815ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:53.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.444+0000 7f630d313700 1 --2- 172.21.15.112:0/1295006331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6308071980 0x7f63080cc960 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f62f800bae0 tx=0x7f62f800bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25e161f5484759f9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:53.449 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.444+0000 7f62fffff700 1 -- 172.21.15.112:0/1295006331 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f800dc70 con 0x7f6308071980 2023-08-27T11:29:53.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.444+0000 7f62fffff700 1 -- 172.21.15.112:0/1295006331 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f800ddd0 con 0x7f6308071980 2023-08-27T11:29:53.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.444+0000 7f62fffff700 1 -- 172.21.15.112:0/1295006331 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8010690 con 0x7f6308071980 2023-08-27T11:29:53.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.445+0000 7f630f577700 1 -- 172.21.15.112:0/1295006331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6308071980 msgr2=0x7f63080cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:53.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.445+0000 7f630f577700 1 --2- 172.21.15.112:0/1295006331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6308071980 0x7f63080cc960 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f62f800bae0 tx=0x7f62f800bdf0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.450 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.445+0000 7f630f577700 1 -- 172.21.15.112:0/1295006331 shutdown_connections 2023-08-27T11:29:53.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.445+0000 7f630f577700 1 --2- 172.21.15.112:0/1295006331 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63080ccea0 0x7f630815ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.445+0000 7f630f577700 1 --2- 172.21.15.112:0/1295006331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6308071980 0x7f63080cc960 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.445+0000 7f630f577700 1 -- 172.21.15.112:0/1295006331 >> 172.21.15.112:0/1295006331 conn(0x7f630806bbf0 msgr2=0x7f63080cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:53.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.445+0000 7f630f577700 1 -- 172.21.15.112:0/1295006331 shutdown_connections 2023-08-27T11:29:53.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.445+0000 7f630f577700 1 -- 172.21.15.112:0/1295006331 wait complete. 2023-08-27T11:29:53.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.446+0000 7f630f577700 1 Processor -- start 2023-08-27T11:29:53.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.446+0000 7f630f577700 1 -- start start 2023-08-27T11:29:53.454 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.446+0000 7f630f577700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6308071980 0x7f63081547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:53.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.446+0000 7f630f577700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63080ccea0 0x7f630806d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:53.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630f577700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f630815ea00 0x7f630806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:53.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630f577700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63081551e0 con 0x7f630815ea00 2023-08-27T11:29:53.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630f577700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63081554c0 con 0x7f63080ccea0 2023-08-27T11:29:53.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630f577700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63081557a0 con 0x7f6308071980 2023-08-27T11:29:53.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630db14700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f630815ea00 0x7f630806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:53.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630db14700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f630815ea00 0x7f630806dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38940/0 (socket says 172.21.15.112:38940) 2023-08-27T11:29:53.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630db14700 1 -- 172.21.15.112:0/958273603 learned_addr learned my addr 172.21.15.112:0/958273603 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:53.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630cb12700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63080ccea0 0x7f630806d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:53.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630d313700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6308071980 0x7f63081547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:53.456 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630db14700 1 -- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6308071980 msgr2=0x7f63081547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:53.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630db14700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6308071980 0x7f63081547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630db14700 1 -- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63080ccea0 msgr2=0x7f630806d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:53.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630db14700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63080ccea0 0x7f630806d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:53.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630db14700 1 -- 172.21.15.112:0/958273603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6300009a70 con 0x7f630815ea00 2023-08-27T11:29:53.457 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630db14700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f630815ea00 0x7f630806dd60 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f630000efc0 tx=0x7f630000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:53.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f62fdffb700 1 -- 172.21.15.112:0/958273603 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630000ca90 con 0x7f630815ea00 2023-08-27T11:29:53.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f62fdffb700 1 -- 172.21.15.112:0/958273603 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630000f040 con 0x7f630815ea00 2023-08-27T11:29:53.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f62fdffb700 1 -- 172.21.15.112:0/958273603 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630000ce10 con 0x7f630815ea00 2023-08-27T11:29:53.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.447+0000 7f630f577700 1 -- 172.21.15.112:0/958273603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f800b740 con 0x7f630815ea00 2023-08-27T11:29:53.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.448+0000 7f630f577700 1 -- 172.21.15.112:0/958273603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6308155d60 con 0x7f630815ea00 2023-08-27T11:29:53.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.448+0000 7f630f577700 1 -- 172.21.15.112:0/958273603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6308066fa0 con 0x7f630815ea00 2023-08-27T11:29:53.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.449+0000 7f62fdffb700 1 -- 172.21.15.112:0/958273603 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f6300014480 con 0x7f630815ea00 2023-08-27T11:29:53.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.449+0000 7f62fdffb700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f62ec06a4d0 0x7f62ec06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:53.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.449+0000 7f62fdffb700 1 -- 172.21.15.112:0/958273603 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f6300089660 con 0x7f630815ea00 2023-08-27T11:29:53.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.449+0000 7f630d313700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f62ec06a4d0 0x7f62ec06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:53.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.450+0000 7f630d313700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f62ec06a4d0 0x7f62ec06c990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f62f800be30 tx=0x7f62f801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:53.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.454+0000 7f62fdffb700 1 -- 172.21.15.112:0/958273603 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6300057e80 con 0x7f630815ea00 2023-08-27T11:29:53.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:53.685+0000 7f630f577700 1 -- 172.21.15.112:0/958273603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000001022841", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6308155f80 con 0x7f630815ea00 2023-08-27T11:29:54.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.303+0000 7f62fdffb700 1 -- 172.21.15.112:0/958273603 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000001022841", "yes_i_really_mean_it": true}]=0 v118) v1 ==== 121+0+0 (secure 0 0 0) 0x7f6300009d70 con 0x7f630815ea00 2023-08-27T11:29:54.307 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.306+0000 7f630f577700 1 -- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f62ec06a4d0 msgr2=0x7f62ec06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:54.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.306+0000 7f630f577700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f62ec06a4d0 0x7f62ec06c990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f62f800be30 tx=0x7f62f801c040 comp rx=0 tx=0).stop 2023-08-27T11:29:54.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.306+0000 7f630f577700 1 -- 172.21.15.112:0/958273603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f630815ea00 msgr2=0x7f630806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:54.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.306+0000 7f630f577700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f630815ea00 0x7f630806dd60 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f630000efc0 tx=0x7f630000c3a0 comp rx=0 tx=0).stop 2023-08-27T11:29:54.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.306+0000 7f630f577700 1 -- 172.21.15.112:0/958273603 shutdown_connections 2023-08-27T11:29:54.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.306+0000 7f630f577700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f62ec06a4d0 0x7f62ec06c990 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:54.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.306+0000 7f630f577700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f630815ea00 0x7f630806dd60 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:54.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.306+0000 7f630f577700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6308071980 0x7f63081547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:54.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.306+0000 7f630f577700 1 --2- 172.21.15.112:0/958273603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63080ccea0 0x7f630806d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:54.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.306+0000 7f630f577700 1 -- 172.21.15.112:0/958273603 >> 172.21.15.112:0/958273603 conn(0x7f630806bbf0 msgr2=0x7f63081583d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:54.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.306+0000 7f630f577700 1 -- 172.21.15.112:0/958273603 shutdown_connections 2023-08-27T11:29:54.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.306+0000 7f630f577700 1 -- 172.21.15.112:0/958273603 wait complete. 2023-08-27T11:29:54.322 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000001022841.meta cephfs.volume_0000000001022841.meta --yes-i-really-really-mean-it 2023-08-27T11:29:54.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.495+0000 7f8f1ef17700 1 Processor -- start 2023-08-27T11:29:54.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.495+0000 7f8f1ef17700 1 -- start start 2023-08-27T11:29:54.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f1ef17700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f18157a70 0x7f8f1815da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:54.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f1ef17700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f181613d0 con 0x7f8f180b3d50 2023-08-27T11:29:54.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f1ef17700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1815e310 con 0x7f8f1806f790 2023-08-27T11:29:54.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f1ef17700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180b1a80 con 0x7f8f18157a70 2023-08-27T11:29:54.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f17fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8f180b3d50 0x7f8f18161860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33330/0 (socket says 172.21.15.112:33330) 2023-08-27T11:29:54.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f17fff700 1 -- 172.21.15.112:0/1908253553 learned_addr learned my addr 172.21.15.112:0/1908253553 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:54.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f1ccb3700 1 --2- 172.21.15.112:0/1908253553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f18157a70 0x7f8f1815da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:54.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f177fe700 1 -- 172.21.15.112:0/1908253553 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205656271 0 0) 0x7f8f181613d0 con 0x7f8f180b3d50 2023-08-27T11:29:54.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f177fe700 1 -- 172.21.15.112:0/1908253553 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f080035b0 con 0x7f8f180b3d50 2023-08-27T11:29:54.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f177fe700 1 -- 172.21.15.112:0/1908253553 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231132363 0 0) 0x7f8f1815e310 con 0x7f8f1806f790 2023-08-27T11:29:54.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f177fe700 1 -- 172.21.15.112:0/1908253553 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f181613d0 con 0x7f8f1806f790 2023-08-27T11:29:54.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f1ccb3700 1 -- 172.21.15.112:0/1908253553 >> v1:172.21.15.138:6789/0 conn(0x7f8f1806f790 legacy=0x7f8f180b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:54.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f177fe700 1 -- 172.21.15.112:0/1908253553 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1163444716 0 0) 0x7f8f080035b0 con 0x7f8f180b3d50 2023-08-27T11:29:54.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f1ccb3700 1 -- 172.21.15.112:0/1908253553 >> v1:172.21.15.112:6789/0 conn(0x7f8f180b3d50 legacy=0x7f8f18161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:54.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.496+0000 7f8f1ccb3700 1 -- 172.21.15.112:0/1908253553 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1815e5f0 con 0x7f8f18157a70 2023-08-27T11:29:54.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.497+0000 7f8f1ccb3700 1 --2- 172.21.15.112:0/1908253553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f18157a70 0x7f8f1815da70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009870 tx=0x7f8f0c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50761b8f323f7d77 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:54.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.497+0000 7f8f177fe700 1 -- 172.21.15.112:0/1908253553 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04001670 con 0x7f8f18157a70 2023-08-27T11:29:54.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.497+0000 7f8f177fe700 1 -- 172.21.15.112:0/1908253553 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c004030 con 0x7f8f18157a70 2023-08-27T11:29:54.502 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.497+0000 7f8f177fe700 1 -- 172.21.15.112:0/1908253553 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0043c0 con 0x7f8f18157a70 2023-08-27T11:29:54.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.497+0000 7f8f1ef17700 1 -- 172.21.15.112:0/1908253553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f18157a70 msgr2=0x7f8f1815da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:54.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.497+0000 7f8f1ef17700 1 --2- 172.21.15.112:0/1908253553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f18157a70 0x7f8f1815da70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009870 tx=0x7f8f0c009b80 comp rx=0 tx=0).stop 2023-08-27T11:29:54.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.497+0000 7f8f1ef17700 1 -- 172.21.15.112:0/1908253553 shutdown_connections 2023-08-27T11:29:54.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.497+0000 7f8f1ef17700 1 --2- 172.21.15.112:0/1908253553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f18157a70 0x7f8f1815da70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:54.503 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.497+0000 7f8f1ef17700 1 -- 172.21.15.112:0/1908253553 >> 172.21.15.112:0/1908253553 conn(0x7f8f180b1be0 msgr2=0x7f8f180b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:54.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.497+0000 7f8f1ef17700 1 -- 172.21.15.112:0/1908253553 shutdown_connections 2023-08-27T11:29:54.504 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.497+0000 7f8f1ef17700 1 -- 172.21.15.112:0/1908253553 wait complete. 2023-08-27T11:29:54.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.498+0000 7f8f1ef17700 1 Processor -- start 2023-08-27T11:29:54.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.499+0000 7f8f1ef17700 1 -- start start 2023-08-27T11:29:54.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.499+0000 7f8f1ef17700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f1806f790 0x7f8f1813f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:54.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.499+0000 7f8f1ef17700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f180b3d50 0x7f8f181601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:54.507 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.499+0000 7f8f1ef17700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f18157a70 0x7f8f1813b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:54.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.499+0000 7f8f1ef17700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1813f560 con 0x7f8f180b3d50 2023-08-27T11:29:54.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.499+0000 7f8f1ef17700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f181f6960 con 0x7f8f1806f790 2023-08-27T11:29:54.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.499+0000 7f8f1ef17700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f181f6c40 con 0x7f8f18157a70 2023-08-27T11:29:54.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.499+0000 7f8f17fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f180b3d50 0x7f8f181601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:54.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.499+0000 7f8f17fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f180b3d50 0x7f8f181601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38954/0 (socket says 172.21.15.112:38954) 2023-08-27T11:29:54.508 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.499+0000 7f8f17fff700 1 -- 172.21.15.112:0/929266364 learned_addr learned my addr 172.21.15.112:0/929266364 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:54.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f1ccb3700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f1806f790 0x7f8f1813f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:54.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f17fff700 1 -- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f18157a70 msgr2=0x7f8f1813b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:54.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f1d4b4700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f18157a70 0x7f8f1813b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:54.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f17fff700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f18157a70 0x7f8f1813b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:54.509 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f17fff700 1 -- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f1806f790 msgr2=0x7f8f1813f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:54.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f17fff700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f1806f790 0x7f8f1813f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:54.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f17fff700 1 -- 172.21.15.112:0/929266364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c0094a0 con 0x7f8f180b3d50 2023-08-27T11:29:54.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f1ccb3700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f1806f790 0x7f8f1813f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:54.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f17fff700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f180b3d50 0x7f8f181601e0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400c770 tx=0x7f8f0400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:54.510 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f157fa700 1 -- 172.21.15.112:0/929266364 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04007560 con 0x7f8f180b3d50 2023-08-27T11:29:54.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f157fa700 1 -- 172.21.15.112:0/929266364 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f040076c0 con 0x7f8f180b3d50 2023-08-27T11:29:54.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f1ef17700 1 -- 172.21.15.112:0/929266364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f181f6f20 con 0x7f8f180b3d50 2023-08-27T11:29:54.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f157fa700 1 -- 172.21.15.112:0/929266364 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04004a70 con 0x7f8f180b3d50 2023-08-27T11:29:54.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.500+0000 7f8f1ef17700 1 -- 172.21.15.112:0/929266364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1813c620 con 0x7f8f180b3d50 2023-08-27T11:29:54.511 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.501+0000 7f8f1ef17700 1 -- 172.21.15.112:0/929266364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1813c840 con 0x7f8f180b3d50 2023-08-27T11:29:54.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.502+0000 7f8f157fa700 1 -- 172.21.15.112:0/929266364 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f8f04004bd0 con 0x7f8f180b3d50 2023-08-27T11:29:54.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.502+0000 7f8f157fa700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8efc06a4d0 0x7f8efc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:54.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.502+0000 7f8f157fa700 1 -- 172.21.15.112:0/929266364 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f8f04088440 con 0x7f8f180b3d50 2023-08-27T11:29:54.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.502+0000 7f8f1ccb3700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8efc06a4d0 0x7f8efc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:54.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.503+0000 7f8f1ccb3700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8efc06a4d0 0x7f8efc06c990 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0090b0 tx=0x7f8f0c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:54.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.506+0000 7f8f157fa700 1 -- 172.21.15.112:0/929266364 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8f040533b0 con 0x7f8f180b3d50 2023-08-27T11:29:54.704 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:54.702+0000 7f8f1ef17700 1 -- 172.21.15.112:0/929266364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000001022841.meta", "pool2": "cephfs.volume_0000000001022841.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8f18067a20 con 0x7f8f180b3d50 2023-08-27T11:29:55.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.446+0000 7f8f157fa700 1 -- 172.21.15.112:0/929266364 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000001022841.meta", "pool2": "cephfs.volume_0000000001022841.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000001022841.meta' removed v174) v1 ==== 241+0+0 (secure 0 0 0) 0x7f8f04018100 con 0x7f8f180b3d50 2023-08-27T11:29:55.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.449+0000 7f8f1ef17700 1 -- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8efc06a4d0 msgr2=0x7f8efc06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:55.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.449+0000 7f8f1ef17700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8efc06a4d0 0x7f8efc06c990 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0090b0 tx=0x7f8f0c008030 comp rx=0 tx=0).stop 2023-08-27T11:29:55.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.449+0000 7f8f1ef17700 1 -- 172.21.15.112:0/929266364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f180b3d50 msgr2=0x7f8f181601e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:55.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.449+0000 7f8f1ef17700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f180b3d50 0x7f8f181601e0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400c770 tx=0x7f8f0400cb30 comp rx=0 tx=0).stop 2023-08-27T11:29:55.451 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.449+0000 7f8f1ef17700 1 -- 172.21.15.112:0/929266364 shutdown_connections 2023-08-27T11:29:55.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.449+0000 7f8f1ef17700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f8efc06a4d0 0x7f8efc06c990 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:55.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.449+0000 7f8f1ef17700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f18157a70 0x7f8f1813b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:55.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.449+0000 7f8f1ef17700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f1806f790 0x7f8f1813f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:55.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.449+0000 7f8f1ef17700 1 --2- 172.21.15.112:0/929266364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f180b3d50 0x7f8f181601e0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:55.452 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.449+0000 7f8f1ef17700 1 -- 172.21.15.112:0/929266364 >> 172.21.15.112:0/929266364 conn(0x7f8f180b1be0 msgr2=0x7f8f180b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:55.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.449+0000 7f8f1ef17700 1 -- 172.21.15.112:0/929266364 shutdown_connections 2023-08-27T11:29:55.453 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.449+0000 7f8f1ef17700 1 -- 172.21.15.112:0/929266364 wait complete. 2023-08-27T11:29:55.453 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs.volume_0000000001022841.meta' removed 2023-08-27T11:29:55.464 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:29:55.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.591+0000 7f99a48d3700 1 Processor -- start 2023-08-27T11:29:55.592 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.591+0000 7f99a48d3700 1 -- start start 2023-08-27T11:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f99a48d3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a00b3de0 0x7f99a015e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f99a48d3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f99a006f790 0x7f99a006fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f99a48d3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a0161390 con 0x7f99a0157ae0 2023-08-27T11:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f99a48d3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a00b1aa0 con 0x7f99a006f790 2023-08-27T11:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f99a48d3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a0070160 con 0x7f99a00b3de0 2023-08-27T11:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f999ed9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f99a006f790 0x7f99a006fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f999e59c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f99a0157ae0 0x7f99a0161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33342/0 (socket says 172.21.15.112:33342) 2023-08-27T11:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f999e59c700 1 -- 172.21.15.112:0/4128203714 learned_addr learned my addr 172.21.15.112:0/4128203714 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f999ed9d700 1 -- 172.21.15.112:0/4128203714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a00b3de0 msgr2=0x7f99a015e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f999dd9b700 1 --2- 172.21.15.112:0/4128203714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a00b3de0 0x7f99a015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:55.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f999ed9d700 1 --2- 172.21.15.112:0/4128203714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a00b3de0 0x7f99a015e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:55.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f999ed9d700 1 -- 172.21.15.112:0/4128203714 >> v1:172.21.15.112:6789/0 conn(0x7f99a0157ae0 legacy=0x7f99a0161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:55.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f999d59a700 1 -- 172.21.15.112:0/4128203714 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094721062 0 0) 0x7f99a0161390 con 0x7f99a0157ae0 2023-08-27T11:29:55.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f999ed9d700 1 -- 172.21.15.112:0/4128203714 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a00b35f0 con 0x7f99a006f790 2023-08-27T11:29:55.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.592+0000 7f999ed9d700 1 --2- 172.21.15.112:0/4128203714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a006f790 0x7f99a006fc20 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f9998009d90 tx=0x7f999800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fa39f0e7fc1c51 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:55.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.593+0000 7f999d59a700 1 -- 172.21.15.112:0/4128203714 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999800e070 con 0x7f99a006f790 2023-08-27T11:29:55.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.593+0000 7f999d59a700 1 -- 172.21.15.112:0/4128203714 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999800bbd0 con 0x7f99a006f790 2023-08-27T11:29:55.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.593+0000 7f999d59a700 1 -- 172.21.15.112:0/4128203714 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998012430 con 0x7f99a006f790 2023-08-27T11:29:55.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.593+0000 7f99a48d3700 1 -- 172.21.15.112:0/4128203714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a006f790 msgr2=0x7f99a006fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:55.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.593+0000 7f99a48d3700 1 --2- 172.21.15.112:0/4128203714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a006f790 0x7f99a006fc20 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f9998009d90 tx=0x7f999800b330 comp rx=0 tx=0).stop 2023-08-27T11:29:55.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.593+0000 7f99a48d3700 1 -- 172.21.15.112:0/4128203714 shutdown_connections 2023-08-27T11:29:55.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.593+0000 7f99a48d3700 1 --2- 172.21.15.112:0/4128203714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a006f790 0x7f99a006fc20 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:55.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.593+0000 7f99a48d3700 1 --2- 172.21.15.112:0/4128203714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a00b3de0 0x7f99a015e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:55.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.593+0000 7f99a48d3700 1 -- 172.21.15.112:0/4128203714 >> 172.21.15.112:0/4128203714 conn(0x7f99a00b1be0 msgr2=0x7f99a00b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.593+0000 7f99a48d3700 1 -- 172.21.15.112:0/4128203714 shutdown_connections 2023-08-27T11:29:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.593+0000 7f99a48d3700 1 -- 172.21.15.112:0/4128203714 wait complete. 2023-08-27T11:29:55.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.595+0000 7f99a48d3700 1 Processor -- start 2023-08-27T11:29:55.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.595+0000 7f99a48d3700 1 -- start start 2023-08-27T11:29:55.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.595+0000 7f99a48d3700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a006f790 0x7f99a0160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:55.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.595+0000 7f99a48d3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99a00b3de0 0x7f99a00ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:55.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.595+0000 7f99a48d3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0157ae0 0x7f99a00acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:55.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.595+0000 7f99a48d3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a0159e70 con 0x7f99a0157ae0 2023-08-27T11:29:55.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.595+0000 7f99a48d3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a0158dd0 con 0x7f99a00b3de0 2023-08-27T11:29:55.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.595+0000 7f99a48d3700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a01590b0 con 0x7f99a006f790 2023-08-27T11:29:55.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.595+0000 7f999ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0157ae0 0x7f99a00acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:55.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f999ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0157ae0 0x7f99a00acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38970/0 (socket says 172.21.15.112:38970) 2023-08-27T11:29:55.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f999ed9d700 1 -- 172.21.15.112:0/2393296540 learned_addr learned my addr 172.21.15.112:0/2393296540 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:55.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f999e59c700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a006f790 0x7f99a0160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f999dd9b700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99a00b3de0 0x7f99a00ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f999ed9d700 1 -- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a006f790 msgr2=0x7f99a0160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f999ed9d700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a006f790 0x7f99a0160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f999ed9d700 1 -- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99a00b3de0 msgr2=0x7f99a00ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f999ed9d700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99a00b3de0 0x7f99a00ac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f999ed9d700 1 -- 172.21.15.112:0/2393296540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9998009a70 con 0x7f99a0157ae0 2023-08-27T11:29:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f999ed9d700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0157ae0 0x7f99a00acf00 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f99a006e930 tx=0x7f9998004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f998effd700 1 -- 172.21.15.112:0/2393296540 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998012430 con 0x7f99a0157ae0 2023-08-27T11:29:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f998effd700 1 -- 172.21.15.112:0/2393296540 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9998020070 con 0x7f99a0157ae0 2023-08-27T11:29:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f99a48d3700 1 -- 172.21.15.112:0/2393296540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a0159390 con 0x7f99a0157ae0 2023-08-27T11:29:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f998effd700 1 -- 172.21.15.112:0/2393296540 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998004280 con 0x7f99a0157ae0 2023-08-27T11:29:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.596+0000 7f99a48d3700 1 -- 172.21.15.112:0/2393296540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a0159950 con 0x7f99a0157ae0 2023-08-27T11:29:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.597+0000 7f99a48d3700 1 -- 172.21.15.112:0/2393296540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a0067a20 con 0x7f99a0157ae0 2023-08-27T11:29:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.597+0000 7f998effd700 1 -- 172.21.15.112:0/2393296540 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f99980043e0 con 0x7f99a0157ae0 2023-08-27T11:29:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.598+0000 7f998effd700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f998006a4d0 0x7f998006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.598+0000 7f998effd700 1 -- 172.21.15.112:0/2393296540 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f9998022070 con 0x7f99a0157ae0 2023-08-27T11:29:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.598+0000 7f999e59c700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f998006a4d0 0x7f998006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.599+0000 7f999e59c700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f998006a4d0 0x7f998006c990 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9990000f60 tx=0x7f9990007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.602+0000 7f998effd700 1 -- 172.21.15.112:0/2393296540 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9998056190 con 0x7f99a0157ae0 2023-08-27T11:29:55.793 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:55.792+0000 7f99a48d3700 1 -- 172.21.15.112:0/2393296540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99a0160e60 con 0x7f99a0157ae0 2023-08-27T11:29:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.453+0000 7f998effd700 1 -- 172.21.15.112:0/2393296540 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v175) v1 ==== 169+0+0 (secure 0 0 0) 0x7f999801ae60 con 0x7f99a0157ae0 2023-08-27T11:29:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.457+0000 7f99a48d3700 1 -- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f998006a4d0 msgr2=0x7f998006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.457+0000 7f99a48d3700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f998006a4d0 0x7f998006c990 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9990000f60 tx=0x7f9990007c90 comp rx=0 tx=0).stop 2023-08-27T11:29:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.457+0000 7f99a48d3700 1 -- 172.21.15.112:0/2393296540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0157ae0 msgr2=0x7f99a00acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.457+0000 7f99a48d3700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0157ae0 0x7f99a00acf00 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f99a006e930 tx=0x7f9998004ad0 comp rx=0 tx=0).stop 2023-08-27T11:29:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.457+0000 7f99a48d3700 1 -- 172.21.15.112:0/2393296540 shutdown_connections 2023-08-27T11:29:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.457+0000 7f99a48d3700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f998006a4d0 0x7f998006c990 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.457+0000 7f99a48d3700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0157ae0 0x7f99a00acf00 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.457+0000 7f99a48d3700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a006f790 0x7f99a0160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.457+0000 7f99a48d3700 1 --2- 172.21.15.112:0/2393296540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99a00b3de0 0x7f99a00ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.457+0000 7f99a48d3700 1 -- 172.21.15.112:0/2393296540 >> 172.21.15.112:0/2393296540 conn(0x7f99a00b1be0 msgr2=0x7f99a01457b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:56.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.457+0000 7f99a48d3700 1 -- 172.21.15.112:0/2393296540 shutdown_connections 2023-08-27T11:29:56.461 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.457+0000 7f99a48d3700 1 -- 172.21.15.112:0/2393296540 wait complete. 2023-08-27T11:29:56.461 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2023-08-27T11:29:56.473 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm another-data-pool another-data-pool --yes-i-really-really-mean-it 2023-08-27T11:29:56.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.602+0000 7f02264c6700 1 Processor -- start 2023-08-27T11:29:56.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.602+0000 7f02264c6700 1 -- start start 2023-08-27T11:29:56.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.603+0000 7f02264c6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0220144bd0 0x7f022015b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:56.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.603+0000 7f02264c6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02200b5eb0 0x7f022006c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:56.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.603+0000 7f02264c6700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02200b6560 con 0x7f022006bfd0 2023-08-27T11:29:56.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.603+0000 7f02264c6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022006c9c0 con 0x7f0220144bd0 2023-08-27T11:29:56.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.603+0000 7f02264c6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02200b6340 con 0x7f02200b5eb0 2023-08-27T11:29:56.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.603+0000 7f021ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0220144bd0 0x7f022015b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:56.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.603+0000 7f021ffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0220144bd0 0x7f022015b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38974/0 (socket says 172.21.15.112:38974) 2023-08-27T11:29:56.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.603+0000 7f021ffff700 1 -- 172.21.15.112:0/2476657384 learned_addr learned my addr 172.21.15.112:0/2476657384 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:56.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.604+0000 7f0224a63700 1 --2- 172.21.15.112:0/2476657384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02200b5eb0 0x7f022006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:56.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.604+0000 7f021ffff700 1 -- 172.21.15.112:0/2476657384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02200b5eb0 msgr2=0x7f022006c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.604+0000 7f021ffff700 1 --2- 172.21.15.112:0/2476657384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02200b5eb0 0x7f022006c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.604+0000 7f021ffff700 1 -- 172.21.15.112:0/2476657384 >> v1:172.21.15.138:6789/0 conn(0x7f022006bfd0 legacy=0x7f02200b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.604+0000 7f021ffff700 1 -- 172.21.15.112:0/2476657384 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02200b6840 con 0x7f0220144bd0 2023-08-27T11:29:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.604+0000 7f0224a63700 1 --2- 172.21.15.112:0/2476657384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02200b5eb0 0x7f022006c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.604+0000 7f021ffff700 1 --2- 172.21.15.112:0/2476657384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0220144bd0 0x7f022015b910 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f0214009920 tx=0x7f0214009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71e27123cb94068 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:56.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.604+0000 7f021effd700 1 -- 172.21.15.112:0/2476657384 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214004230 con 0x7f0220144bd0 2023-08-27T11:29:56.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.604+0000 7f021effd700 1 -- 172.21.15.112:0/2476657384 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0214004390 con 0x7f0220144bd0 2023-08-27T11:29:56.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.604+0000 7f021effd700 1 -- 172.21.15.112:0/2476657384 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214009e70 con 0x7f0220144bd0 2023-08-27T11:29:56.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.604+0000 7f02264c6700 1 -- 172.21.15.112:0/2476657384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0220144bd0 msgr2=0x7f022015b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:56.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.604+0000 7f02264c6700 1 --2- 172.21.15.112:0/2476657384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0220144bd0 0x7f022015b910 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f0214009920 tx=0x7f0214009c30 comp rx=0 tx=0).stop 2023-08-27T11:29:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.605+0000 7f02264c6700 1 -- 172.21.15.112:0/2476657384 shutdown_connections 2023-08-27T11:29:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.605+0000 7f02264c6700 1 --2- 172.21.15.112:0/2476657384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02200b5eb0 0x7f022006c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.605+0000 7f02264c6700 1 --2- 172.21.15.112:0/2476657384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0220144bd0 0x7f022015b910 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.605+0000 7f02264c6700 1 -- 172.21.15.112:0/2476657384 >> 172.21.15.112:0/2476657384 conn(0x7f0220066010 msgr2=0x7f0220066420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.605+0000 7f02264c6700 1 -- 172.21.15.112:0/2476657384 shutdown_connections 2023-08-27T11:29:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.605+0000 7f02264c6700 1 -- 172.21.15.112:0/2476657384 wait complete. 2023-08-27T11:29:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.606+0000 7f02264c6700 1 Processor -- start 2023-08-27T11:29:56.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.606+0000 7f02264c6700 1 -- start start 2023-08-27T11:29:56.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f02264c6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f022006bfd0 0x7f0220156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:56.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f02264c6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02200b5eb0 0x7f02201f3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:56.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f02264c6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0220144bd0 0x7f0220157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:56.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f02264c6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02201586b0 con 0x7f0220144bd0 2023-08-27T11:29:56.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f02264c6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0220157790 con 0x7f02200b5eb0 2023-08-27T11:29:56.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f02264c6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0220157a30 con 0x7f022006bfd0 2023-08-27T11:29:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f0224a63700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0220144bd0 0x7f0220157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f0224a63700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0220144bd0 0x7f0220157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38984/0 (socket says 172.21.15.112:38984) 2023-08-27T11:29:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f0224a63700 1 -- 172.21.15.112:0/2250508985 learned_addr learned my addr 172.21.15.112:0/2250508985 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f021ffff700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f022006bfd0 0x7f0220156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f021f7fe700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02200b5eb0 0x7f02201f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f0224a63700 1 -- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f022006bfd0 msgr2=0x7f0220156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f0224a63700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f022006bfd0 0x7f0220156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f0224a63700 1 -- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02200b5eb0 msgr2=0x7f02201f3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f0224a63700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02200b5eb0 0x7f02201f3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f0224a63700 1 -- 172.21.15.112:0/2250508985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0214009550 con 0x7f0220144bd0 2023-08-27T11:29:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f0224a63700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0220144bd0 0x7f0220157250 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f021800cef0 tx=0x7f021800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f021cff9700 1 -- 172.21.15.112:0/2250508985 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021800ae50 con 0x7f0220144bd0 2023-08-27T11:29:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f021cff9700 1 -- 172.21.15.112:0/2250508985 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0218015440 con 0x7f0220144bd0 2023-08-27T11:29:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f02264c6700 1 -- 172.21.15.112:0/2250508985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0220157d10 con 0x7f0220144bd0 2023-08-27T11:29:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f021cff9700 1 -- 172.21.15.112:0/2250508985 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02180157f0 con 0x7f0220144bd0 2023-08-27T11:29:56.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.607+0000 7f02264c6700 1 -- 172.21.15.112:0/2250508985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02201582d0 con 0x7f0220144bd0 2023-08-27T11:29:56.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.608+0000 7f02264c6700 1 -- 172.21.15.112:0/2250508985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02201587f0 con 0x7f0220144bd0 2023-08-27T11:29:56.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.609+0000 7f021cff9700 1 -- 172.21.15.112:0/2250508985 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f021800d430 con 0x7f0220144bd0 2023-08-27T11:29:56.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.609+0000 7f021cff9700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f020c06a4d0 0x7f020c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:56.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.609+0000 7f021cff9700 1 -- 172.21.15.112:0/2250508985 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f02180890e0 con 0x7f0220144bd0 2023-08-27T11:29:56.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.609+0000 7f021ffff700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f020c06a4d0 0x7f020c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:56.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.610+0000 7f021ffff700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f020c06a4d0 0x7f020c06c990 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f0214009130 tx=0x7f0214008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:56.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.613+0000 7f021cff9700 1 -- 172.21.15.112:0/2250508985 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f02180543d0 con 0x7f0220144bd0 2023-08-27T11:29:56.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:56.803+0000 7f02264c6700 1 -- 172.21.15.112:0/2250508985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "another-data-pool", "pool2": "another-data-pool", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0220067a20 con 0x7f0220144bd0 2023-08-27T11:29:57.459 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.458+0000 7f021cff9700 1 -- 172.21.15.112:0/2250508985 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "another-data-pool", "pool2": "another-data-pool", "yes_i_really_really_mean_it": true}]=0 pool 'another-data-pool' removed v176) v1 ==== 187+0+0 (secure 0 0 0) 0x7f02180579f0 con 0x7f0220144bd0 2023-08-27T11:29:57.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.461+0000 7f02264c6700 1 -- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f020c06a4d0 msgr2=0x7f020c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:57.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.461+0000 7f02264c6700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f020c06a4d0 0x7f020c06c990 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f0214009130 tx=0x7f0214008030 comp rx=0 tx=0).stop 2023-08-27T11:29:57.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.461+0000 7f02264c6700 1 -- 172.21.15.112:0/2250508985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0220144bd0 msgr2=0x7f0220157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:57.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.461+0000 7f02264c6700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0220144bd0 0x7f0220157250 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f021800cef0 tx=0x7f021800a340 comp rx=0 tx=0).stop 2023-08-27T11:29:57.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.461+0000 7f02264c6700 1 -- 172.21.15.112:0/2250508985 shutdown_connections 2023-08-27T11:29:57.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.461+0000 7f02264c6700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f020c06a4d0 0x7f020c06c990 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.461+0000 7f02264c6700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0220144bd0 0x7f0220157250 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.461+0000 7f02264c6700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f022006bfd0 0x7f0220156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.461+0000 7f02264c6700 1 --2- 172.21.15.112:0/2250508985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02200b5eb0 0x7f02201f3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.461+0000 7f02264c6700 1 -- 172.21.15.112:0/2250508985 >> 172.21.15.112:0/2250508985 conn(0x7f0220066010 msgr2=0x7f02200663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:57.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.461+0000 7f02264c6700 1 -- 172.21.15.112:0/2250508985 shutdown_connections 2023-08-27T11:29:57.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.461+0000 7f02264c6700 1 -- 172.21.15.112:0/2250508985 wait complete. 2023-08-27T11:29:57.465 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'another-data-pool' removed 2023-08-27T11:29:57.476 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:29:57.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.604+0000 7f13e3086700 1 Processor -- start 2023-08-27T11:29:57.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.604+0000 7f13e3086700 1 -- start start 2023-08-27T11:29:57.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e3086700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13dc15e190 0x7f13dc15e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e3086700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13dc161dd0 0x7f13dc0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e3086700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f13dc0cc980 0x7f13dc0ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e3086700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13dc0cd330 con 0x7f13dc0cc980 2023-08-27T11:29:57.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e3086700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13dc0b46c0 con 0x7f13dc15e190 2023-08-27T11:29:57.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e3086700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc0b49a0 con 0x7f13dc161dd0 2023-08-27T11:29:57.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e1623700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f13dc0cc980 0x7f13dc0ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e1623700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f13dc0cc980 0x7f13dc0ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38992/0 (socket says 172.21.15.112:38992) 2023-08-27T11:29:57.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e1623700 1 -- 172.21.15.112:0/457799949 learned_addr learned my addr 172.21.15.112:0/457799949 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e0e22700 1 --2- 172.21.15.112:0/457799949 >> v2:172.21.15.138:3300/0 conn(0x7f13dc15e190 0x7f13dc15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13dbfff700 1 --2- 172.21.15.112:0/457799949 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13dc161dd0 0x7f13dc0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e1623700 1 -- 172.21.15.112:0/457799949 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13dc161dd0 msgr2=0x7f13dc0b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e1623700 1 --2- 172.21.15.112:0/457799949 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13dc161dd0 0x7f13dc0b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e1623700 1 -- 172.21.15.112:0/457799949 >> v2:172.21.15.138:3300/0 conn(0x7f13dc15e190 msgr2=0x7f13dc15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e1623700 1 --2- 172.21.15.112:0/457799949 >> v2:172.21.15.138:3300/0 conn(0x7f13dc15e190 0x7f13dc15e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e1623700 1 -- 172.21.15.112:0/457799949 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc1612f0 con 0x7f13dc0cc980 2023-08-27T11:29:57.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13e1623700 1 --2- 172.21.15.112:0/457799949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc0cc980 0x7f13dc0ccdf0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0d0fb0 tx=0x7f13d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc999a759e73d6c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:57.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.605+0000 7f13db7fe700 1 -- 172.21.15.112:0/457799949 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d400e070 con 0x7f13dc0cc980 2023-08-27T11:29:57.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.606+0000 7f13db7fe700 1 -- 172.21.15.112:0/457799949 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d400b910 con 0x7f13dc0cc980 2023-08-27T11:29:57.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.606+0000 7f13db7fe700 1 -- 172.21.15.112:0/457799949 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d400bcc0 con 0x7f13dc0cc980 2023-08-27T11:29:57.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.606+0000 7f13e3086700 1 -- 172.21.15.112:0/457799949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc0cc980 msgr2=0x7f13dc0ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:57.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.606+0000 7f13e3086700 1 --2- 172.21.15.112:0/457799949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc0cc980 0x7f13dc0ccdf0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0d0fb0 tx=0x7f13d400b330 comp rx=0 tx=0).stop 2023-08-27T11:29:57.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.606+0000 7f13e3086700 1 -- 172.21.15.112:0/457799949 shutdown_connections 2023-08-27T11:29:57.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.606+0000 7f13e3086700 1 --2- 172.21.15.112:0/457799949 >> v2:172.21.15.138:3300/0 conn(0x7f13dc15e190 0x7f13dc15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.606+0000 7f13e3086700 1 --2- 172.21.15.112:0/457799949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc0cc980 0x7f13dc0ccdf0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.606+0000 7f13e3086700 1 --2- 172.21.15.112:0/457799949 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13dc161dd0 0x7f13dc0b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.606+0000 7f13e3086700 1 -- 172.21.15.112:0/457799949 >> 172.21.15.112:0/457799949 conn(0x7f13dc06bbf0 msgr2=0x7f13dc0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:57.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.606+0000 7f13e3086700 1 -- 172.21.15.112:0/457799949 shutdown_connections 2023-08-27T11:29:57.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.606+0000 7f13e3086700 1 -- 172.21.15.112:0/457799949 wait complete. 2023-08-27T11:29:57.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.607+0000 7f13e3086700 1 Processor -- start 2023-08-27T11:29:57.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.607+0000 7f13e3086700 1 -- start start 2023-08-27T11:29:57.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.608+0000 7f13e3086700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc0cc980 0x7f13dc140b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.608+0000 7f13e3086700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13dc15e190 0x7f13dc071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.608+0000 7f13e3086700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13dc161dd0 0x7f13dc0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.608+0000 7f13e3086700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc06fe50 con 0x7f13dc0cc980 2023-08-27T11:29:57.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.608+0000 7f13e3086700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc06fb70 con 0x7f13dc161dd0 2023-08-27T11:29:57.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.608+0000 7f13e3086700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc0b1700 con 0x7f13dc15e190 2023-08-27T11:29:57.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.608+0000 7f13e0e22700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc0cc980 0x7f13dc140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.608+0000 7f13e0e22700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc0cc980 0x7f13dc140b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38994/0 (socket says 172.21.15.112:38994) 2023-08-27T11:29:57.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.608+0000 7f13e0e22700 1 -- 172.21.15.112:0/926387609 learned_addr learned my addr 172.21.15.112:0/926387609 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:57.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.608+0000 7f13e1623700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13dc161dd0 0x7f13dc0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.608+0000 7f13dbfff700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13dc15e190 0x7f13dc071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.609+0000 7f13e0e22700 1 -- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13dc15e190 msgr2=0x7f13dc071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:57.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.609+0000 7f13e0e22700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13dc15e190 0x7f13dc071e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.609+0000 7f13e0e22700 1 -- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13dc161dd0 msgr2=0x7f13dc0b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:57.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.609+0000 7f13e0e22700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13dc161dd0 0x7f13dc0b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.609+0000 7f13e0e22700 1 -- 172.21.15.112:0/926387609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4009b30 con 0x7f13dc0cc980 2023-08-27T11:29:57.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.609+0000 7f13e0e22700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc0cc980 0x7f13dc140b30 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f13d000e930 tx=0x7f13d000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:57.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.609+0000 7f13d97fa700 1 -- 172.21.15.112:0/926387609 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0013680 con 0x7f13dc0cc980 2023-08-27T11:29:57.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.609+0000 7f13d97fa700 1 -- 172.21.15.112:0/926387609 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d00137e0 con 0x7f13dc0cc980 2023-08-27T11:29:57.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.609+0000 7f13e3086700 1 -- 172.21.15.112:0/926387609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13dc0b19e0 con 0x7f13dc0cc980 2023-08-27T11:29:57.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.609+0000 7f13d97fa700 1 -- 172.21.15.112:0/926387609 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0013b70 con 0x7f13dc0cc980 2023-08-27T11:29:57.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.609+0000 7f13e3086700 1 -- 172.21.15.112:0/926387609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13dc141070 con 0x7f13dc0cc980 2023-08-27T11:29:57.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.610+0000 7f13e3086700 1 -- 172.21.15.112:0/926387609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13dc066fa0 con 0x7f13dc0cc980 2023-08-27T11:29:57.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.610+0000 7f13d97fa700 1 -- 172.21.15.112:0/926387609 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f13d0013cd0 con 0x7f13dc0cc980 2023-08-27T11:29:57.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.611+0000 7f13d97fa700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13c006a4d0 0x7f13c006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.611+0000 7f13d97fa700 1 -- 172.21.15.112:0/926387609 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f13d00888e0 con 0x7f13dc0cc980 2023-08-27T11:29:57.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.611+0000 7f13dbfff700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13c006a4d0 0x7f13c006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.611+0000 7f13dbfff700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13c006a4d0 0x7f13c006c990 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f13dc15f790 tx=0x7f13c8009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:57.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.615+0000 7f13d97fa700 1 -- 172.21.15.112:0/926387609 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f13d0053cf0 con 0x7f13dc0cc980 2023-08-27T11:29:57.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.807+0000 7f13e3086700 1 -- 172.21.15.112:0/926387609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f13dc0b1560 con 0x7f13dc0cc980 2023-08-27T11:29:57.811 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.810+0000 7f13d97fa700 1 -- 172.21.15.112:0/926387609 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f13dc0b1560 con 0x7f13dc0cc980 2023-08-27T11:29:57.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.812+0000 7f13e3086700 1 -- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13c006a4d0 msgr2=0x7f13c006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:57.816 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.812+0000 7f13e3086700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13c006a4d0 0x7f13c006c990 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f13dc15f790 tx=0x7f13c8009210 comp rx=0 tx=0).stop 2023-08-27T11:29:57.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.812+0000 7f13e3086700 1 -- 172.21.15.112:0/926387609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc0cc980 msgr2=0x7f13dc140b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:57.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.812+0000 7f13e3086700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc0cc980 0x7f13dc140b30 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f13d000e930 tx=0x7f13d000ec40 comp rx=0 tx=0).stop 2023-08-27T11:29:57.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.813+0000 7f13e3086700 1 -- 172.21.15.112:0/926387609 shutdown_connections 2023-08-27T11:29:57.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.813+0000 7f13e3086700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f13c006a4d0 0x7f13c006c990 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.817 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.813+0000 7f13e3086700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13dc161dd0 0x7f13dc0b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.813+0000 7f13e3086700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc0cc980 0x7f13dc140b30 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.813+0000 7f13e3086700 1 --2- 172.21.15.112:0/926387609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13dc15e190 0x7f13dc071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.813+0000 7f13e3086700 1 -- 172.21.15.112:0/926387609 >> 172.21.15.112:0/926387609 conn(0x7f13dc06bbf0 msgr2=0x7f13dc0cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:57.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.813+0000 7f13e3086700 1 -- 172.21.15.112:0/926387609 shutdown_connections 2023-08-27T11:29:57.818 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.813+0000 7f13e3086700 1 -- 172.21.15.112:0/926387609 wait complete. 2023-08-27T11:29:57.826 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool' 2023-08-27T11:29:57.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.946+0000 7f49324d8700 1 Processor -- start 2023-08-27T11:29:57.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.946+0000 7f49324d8700 1 -- start start 2023-08-27T11:29:57.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.946+0000 7f49324d8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c144080 0x7f492c0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.946+0000 7f49324d8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f492c0b5d60 0x7f492c15ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.946+0000 7f49324d8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c15fc00 con 0x7f492c144690 2023-08-27T11:29:57.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.946+0000 7f49324d8700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492c15b660 con 0x7f492c0b5d60 2023-08-27T11:29:57.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.946+0000 7f49324d8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c15f080 con 0x7f492c144080 2023-08-27T11:29:57.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f492c144690 0x7f492c0b5650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33386/0 (socket says 172.21.15.112:33386) 2023-08-27T11:29:57.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492b7fe700 1 -- 172.21.15.112:0/3216693823 learned_addr learned my addr 172.21.15.112:0/3216693823 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:57.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492bfff700 1 --2- 172.21.15.112:0/3216693823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c144080 0x7f492c0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f4930a75700 1 --2- 172.21.15.112:0/3216693823 >> v2:172.21.15.138:3300/0 conn(0x7f492c0b5d60 0x7f492c15ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051823427 0 0) 0x7f492c15fc00 con 0x7f492c144690 2023-08-27T11:29:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49180035b0 con 0x7f492c144690 2023-08-27T11:29:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1341969704 0 0) 0x7f49180035b0 con 0x7f492c144690 2023-08-27T11:29:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c144080 msgr2=0x7f492c0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:57.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 --2- 172.21.15.112:0/3216693823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c144080 0x7f492c0ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 >> v2:172.21.15.138:3300/0 conn(0x7f492c0b5d60 msgr2=0x7f492c15ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:57.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 --2- 172.21.15.112:0/3216693823 >> v2:172.21.15.138:3300/0 conn(0x7f492c0b5d60 0x7f492c15ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c15f360 con 0x7f492c144690 2023-08-27T11:29:57.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f4914001670 con 0x7f492c144690 2023-08-27T11:29:57.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 >> v1:172.21.15.112:6789/0 conn(0x7f492c144690 legacy=0x7f492c0b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:57.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 --2- 172.21.15.112:0/3216693823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4918003a00 0x7f4918003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 --2- 172.21.15.112:0/3216693823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c144080 0x7f4918004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 --2- 172.21.15.112:0/3216693823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4918004d90 0x7f4918005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c15f080 con 0x7f4918004d90 2023-08-27T11:29:57.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491800ba10 con 0x7f4918003a00 2023-08-27T11:29:57.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491800bb50 con 0x7f492c144080 2023-08-27T11:29:57.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f4930a75700 1 --2- 172.21.15.112:0/3216693823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4918004d90 0x7f4918005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492bfff700 1 --2- 172.21.15.112:0/3216693823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4918003a00 0x7f4918003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.947+0000 7f492b7fe700 1 --2- 172.21.15.112:0/3216693823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c144080 0x7f4918004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f4930a75700 1 -- 172.21.15.112:0/3216693823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c144080 msgr2=0x7f4918004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:57.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f4930a75700 1 --2- 172.21.15.112:0/3216693823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c144080 0x7f4918004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f4930a75700 1 -- 172.21.15.112:0/3216693823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4918003a00 msgr2=0x7f4918003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:57.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f4930a75700 1 --2- 172.21.15.112:0/3216693823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4918003a00 0x7f4918003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f4930a75700 1 -- 172.21.15.112:0/3216693823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49180035b0 con 0x7f4918004d90 2023-08-27T11:29:57.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f4930a75700 1 --2- 172.21.15.112:0/3216693823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4918004d90 0x7f4918005210 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f4924009fc0 tx=0x7f4918005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af0f5fb97876c22 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:57.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492400ec30 con 0x7f4918004d90 2023-08-27T11:29:57.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492400f040 con 0x7f4918004d90 2023-08-27T11:29:57.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f492affd700 1 -- 172.21.15.112:0/3216693823 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924014480 con 0x7f4918004d90 2023-08-27T11:29:57.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f49324d8700 1 -- 172.21.15.112:0/3216693823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4918004d90 msgr2=0x7f4918005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f49324d8700 1 --2- 172.21.15.112:0/3216693823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4918004d90 0x7f4918005210 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f4924009fc0 tx=0x7f4918005880 comp rx=0 tx=0).stop 2023-08-27T11:29:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f4930a75700 1 -- 172.21.15.112:0/3216693823 reap_dead start 2023-08-27T11:29:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f49324d8700 1 -- 172.21.15.112:0/3216693823 shutdown_connections 2023-08-27T11:29:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.948+0000 7f49324d8700 1 -- 172.21.15.112:0/3216693823 >> 172.21.15.112:0/3216693823 conn(0x7f492c0cff80 msgr2=0x7f492c0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.949+0000 7f49324d8700 1 -- 172.21.15.112:0/3216693823 shutdown_connections 2023-08-27T11:29:57.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.949+0000 7f49324d8700 1 -- 172.21.15.112:0/3216693823 wait complete. 2023-08-27T11:29:57.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.950+0000 7f49324d8700 1 Processor -- start 2023-08-27T11:29:57.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.950+0000 7f49324d8700 1 -- start start 2023-08-27T11:29:57.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f49324d8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4918004d90 0x7f492c0b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f49324d8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c0ce950 0x7f492c1f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f49324d8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f492c0cedb0 0x7f492c13fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f49324d8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0b3a30 con 0x7f4918004d90 2023-08-27T11:29:57.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f49324d8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0b3440 con 0x7f492c0cedb0 2023-08-27T11:29:57.964 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f49324d8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c1f3750 con 0x7f492c0ce950 2023-08-27T11:29:57.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f492bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4918004d90 0x7f492c0b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f492bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4918004d90 0x7f492c0b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39014/0 (socket says 172.21.15.112:39014) 2023-08-27T11:29:57.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f492bfff700 1 -- 172.21.15.112:0/3627079862 learned_addr learned my addr 172.21.15.112:0/3627079862 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:57.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f492b7fe700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c0ce950 0x7f492c1f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.965 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f4930a75700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f492c0cedb0 0x7f492c13fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f492bfff700 1 -- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c0ce950 msgr2=0x7f492c1f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:57.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f492bfff700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c0ce950 0x7f492c1f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f492bfff700 1 -- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f492c0cedb0 msgr2=0x7f492c13fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:57.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f492bfff700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f492c0cedb0 0x7f492c13fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:57.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f492bfff700 1 -- 172.21.15.112:0/3627079862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c1f3a30 con 0x7f4918004d90 2023-08-27T11:29:57.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f492b7fe700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c0ce950 0x7f492c1f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:57.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f492bfff700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4918004d90 0x7f492c0b3e50 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f491c0119a0 tx=0x7f491c011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:57.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f4928ff9700 1 -- 172.21.15.112:0/3627079862 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c00f940 con 0x7f4918004d90 2023-08-27T11:29:57.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f4928ff9700 1 -- 172.21.15.112:0/3627079862 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c00faa0 con 0x7f4918004d90 2023-08-27T11:29:57.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f49324d8700 1 -- 172.21.15.112:0/3627079862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c0b3650 con 0x7f4918004d90 2023-08-27T11:29:57.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f4928ff9700 1 -- 172.21.15.112:0/3627079862 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c00fe30 con 0x7f4918004d90 2023-08-27T11:29:57.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.951+0000 7f49324d8700 1 -- 172.21.15.112:0/3627079862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c0b4ce0 con 0x7f4918004d90 2023-08-27T11:29:57.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.952+0000 7f49324d8700 1 -- 172.21.15.112:0/3627079862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c0b3b70 con 0x7f4918004d90 2023-08-27T11:29:57.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.953+0000 7f4928ff9700 1 -- 172.21.15.112:0/3627079862 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f491c014440 con 0x7f4918004d90 2023-08-27T11:29:57.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.953+0000 7f4928ff9700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f490c06a500 0x7f490c06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:57.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.953+0000 7f4928ff9700 1 -- 172.21.15.112:0/3627079862 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f491c089050 con 0x7f4918004d90 2023-08-27T11:29:57.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.953+0000 7f492b7fe700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f490c06a500 0x7f490c06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:57.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.954+0000 7f492b7fe700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f490c06a500 0x7f490c06c9c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4914000c00 tx=0x7f4914009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:57.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:57.958+0000 7f4928ff9700 1 -- 172.21.15.112:0/3627079862 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f491c0bb990 con 0x7f4918004d90 2023-08-27T11:29:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.220+0000 7f49324d8700 1 -- 172.21.15.112:0/3627079862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]} v 0) v1 -- 0x7f492c067a20 con 0x7f4918004d90 2023-08-27T11:29:58.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.459+0000 7f4928ff9700 1 -- 172.21.15.112:0/3627079862 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]}]=0 v605) v1 ==== 163+0+0 (secure 0 0 0) 0x7f491c057f80 con 0x7f4918004d90 2023-08-27T11:29:58.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.461+0000 7f49324d8700 1 -- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f490c06a500 msgr2=0x7f490c06c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:58.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.461+0000 7f49324d8700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f490c06a500 0x7f490c06c9c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4914000c00 tx=0x7f4914009040 comp rx=0 tx=0).stop 2023-08-27T11:29:58.463 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.461+0000 7f49324d8700 1 -- 172.21.15.112:0/3627079862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4918004d90 msgr2=0x7f492c0b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.461+0000 7f49324d8700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4918004d90 0x7f492c0b3e50 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f491c0119a0 tx=0x7f491c011d60 comp rx=0 tx=0).stop 2023-08-27T11:29:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.461+0000 7f49324d8700 1 -- 172.21.15.112:0/3627079862 shutdown_connections 2023-08-27T11:29:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.461+0000 7f49324d8700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f490c06a500 0x7f490c06c9c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.461+0000 7f49324d8700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f492c0cedb0 0x7f492c13fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.461+0000 7f49324d8700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4918004d90 0x7f492c0b3e50 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.461+0000 7f49324d8700 1 --2- 172.21.15.112:0/3627079862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f492c0ce950 0x7f492c1f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.461+0000 7f49324d8700 1 -- 172.21.15.112:0/3627079862 >> 172.21.15.112:0/3627079862 conn(0x7f492c0cff80 msgr2=0x7f492c15e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.462+0000 7f49324d8700 1 -- 172.21.15.112:0/3627079862 shutdown_connections 2023-08-27T11:29:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.462+0000 7f49324d8700 1 -- 172.21.15.112:0/3627079862 wait complete. 2023-08-27T11:29:58.475 INFO:tasks.cephfs_test_runner:That renaming a volume with more than one data pool does not change ... ok 2023-08-27T11:29:58.475 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_idempotency (tasks.cephfs.test_volumes.TestVolumes) That volume rename is idempotent. 2023-08-27T11:29:58.476 INFO:tasks.cephfs_test_runner:test_volume_rename_idempotency (tasks.cephfs.test_volumes.TestVolumes) 2023-08-27T11:29:58.476 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency' 2023-08-27T11:29:58.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.595+0000 7f3fa35c1700 1 Processor -- start 2023-08-27T11:29:58.596 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.595+0000 7f3fa35c1700 1 -- start start 2023-08-27T11:29:58.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3fa35c1700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3f9c153e70 0x7f3f9c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:58.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3fa35c1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f9c159e00 0x7f3f9c15a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:58.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3fa35c1700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f9c154250 con 0x7f3f9c06e150 2023-08-27T11:29:58.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3fa35c1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f9c15a970 con 0x7f3f9c159e00 2023-08-27T11:29:58.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3fa35c1700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f9c15ac50 con 0x7f3f9c153e70 2023-08-27T11:29:58.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3fa0b5c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f9c159e00 0x7f3f9c15a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:58.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3fa0b5c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f9c159e00 0x7f3f9c15a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39026/0 (socket says 172.21.15.112:39026) 2023-08-27T11:29:58.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3fa0b5c700 1 -- 172.21.15.112:0/4031679346 learned_addr learned my addr 172.21.15.112:0/4031679346 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:58.600 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3fa0b5c700 1 -- 172.21.15.112:0/4031679346 >> v2:172.21.15.138:3300/0 conn(0x7f3f9c153e70 msgr2=0x7f3f9c15de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:58.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3fa0b5c700 1 --2- 172.21.15.112:0/4031679346 >> v2:172.21.15.138:3300/0 conn(0x7f3f9c153e70 0x7f3f9c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:58.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3fa0b5c700 1 -- 172.21.15.112:0/4031679346 >> v1:172.21.15.138:6789/0 conn(0x7f3f9c06e150 legacy=0x7f3f9c0cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:58.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3fa0b5c700 1 -- 172.21.15.112:0/4031679346 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c15d380 con 0x7f3f9c159e00 2023-08-27T11:29:58.601 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3fa0b5c700 1 --2- 172.21.15.112:0/4031679346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f9c159e00 0x7f3f9c15a290 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c066f70 tx=0x7f3f8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bc2dcf89da39a7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:58.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3f93fff700 1 -- 172.21.15.112:0/4031679346 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c004230 con 0x7f3f9c159e00 2023-08-27T11:29:58.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3f93fff700 1 -- 172.21.15.112:0/4031679346 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8c004390 con 0x7f3f9c159e00 2023-08-27T11:29:58.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.596+0000 7f3f93fff700 1 -- 172.21.15.112:0/4031679346 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c0036a0 con 0x7f3f9c159e00 2023-08-27T11:29:58.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.597+0000 7f3fa35c1700 1 -- 172.21.15.112:0/4031679346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f9c159e00 msgr2=0x7f3f9c15a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:58.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.597+0000 7f3fa35c1700 1 --2- 172.21.15.112:0/4031679346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f9c159e00 0x7f3f9c15a290 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c066f70 tx=0x7f3f8c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:58.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.597+0000 7f3fa35c1700 1 -- 172.21.15.112:0/4031679346 shutdown_connections 2023-08-27T11:29:58.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.597+0000 7f3fa35c1700 1 --2- 172.21.15.112:0/4031679346 >> v2:172.21.15.138:3300/0 conn(0x7f3f9c153e70 0x7f3f9c15de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:58.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.597+0000 7f3fa35c1700 1 --2- 172.21.15.112:0/4031679346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f9c159e00 0x7f3f9c15a290 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:58.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.597+0000 7f3fa35c1700 1 -- 172.21.15.112:0/4031679346 >> 172.21.15.112:0/4031679346 conn(0x7f3f9c06cd30 msgr2=0x7f3f9c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:58.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.597+0000 7f3fa35c1700 1 -- 172.21.15.112:0/4031679346 shutdown_connections 2023-08-27T11:29:58.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.597+0000 7f3fa35c1700 1 -- 172.21.15.112:0/4031679346 wait complete. 2023-08-27T11:29:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.598+0000 7f3fa35c1700 1 Processor -- start 2023-08-27T11:29:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.598+0000 7f3fa35c1700 1 -- start start 2023-08-27T11:29:58.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa35c1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f9c06e150 0x7f3f9c0716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:58.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa35c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f9c153e70 0x7f3f9c1411b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:58.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa35c1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f9c159e00 0x7f3f9c154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:58.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa35c1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c1416f0 con 0x7f3f9c06e150 2023-08-27T11:29:58.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa35c1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c155860 con 0x7f3f9c153e70 2023-08-27T11:29:58.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa35c1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c154c50 con 0x7f3f9c159e00 2023-08-27T11:29:58.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa135d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f9c06e150 0x7f3f9c0716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:58.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa135d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f9c06e150 0x7f3f9c0716f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39028/0 (socket says 172.21.15.112:39028) 2023-08-27T11:29:58.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa135d700 1 -- 172.21.15.112:0/1775477584 learned_addr learned my addr 172.21.15.112:0/1775477584 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:58.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa0b5c700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f9c153e70 0x7f3f9c1411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa1b5e700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f9c159e00 0x7f3f9c154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa135d700 1 -- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f9c159e00 msgr2=0x7f3f9c154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa135d700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f9c159e00 0x7f3f9c154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa135d700 1 -- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f9c153e70 msgr2=0x7f3f9c1411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa135d700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f9c153e70 0x7f3f9c1411b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.599+0000 7f3fa135d700 1 -- 172.21.15.112:0/1775477584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8c009550 con 0x7f3f9c06e150 2023-08-27T11:29:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.600+0000 7f3fa135d700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f9c06e150 0x7f3f9c0716f0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800b530 tx=0x7f3f9800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.600+0000 7f3f91ffb700 1 -- 172.21.15.112:0/1775477584 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98011830 con 0x7f3f9c06e150 2023-08-27T11:29:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.600+0000 7f3f91ffb700 1 -- 172.21.15.112:0/1775477584 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f980119b0 con 0x7f3f9c06e150 2023-08-27T11:29:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.600+0000 7f3fa35c1700 1 -- 172.21.15.112:0/1775477584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9c154f30 con 0x7f3f9c06e150 2023-08-27T11:29:58.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.600+0000 7f3f91ffb700 1 -- 172.21.15.112:0/1775477584 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98011d60 con 0x7f3f9c06e150 2023-08-27T11:29:58.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.600+0000 7f3fa35c1700 1 -- 172.21.15.112:0/1775477584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9c1554f0 con 0x7f3f9c06e150 2023-08-27T11:29:58.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.601+0000 7f3fa35c1700 1 -- 172.21.15.112:0/1775477584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f9c067a20 con 0x7f3f9c06e150 2023-08-27T11:29:58.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.601+0000 7f3f91ffb700 1 -- 172.21.15.112:0/1775477584 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f3f9801a400 con 0x7f3f9c06e150 2023-08-27T11:29:58.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.601+0000 7f3f91ffb700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3f8006a4d0 0x7f3f8006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:58.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.602+0000 7f3f91ffb700 1 -- 172.21.15.112:0/1775477584 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3f98088a40 con 0x7f3f9c06e150 2023-08-27T11:29:58.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.602+0000 7f3fa0b5c700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3f8006a4d0 0x7f3f8006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:58.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.602+0000 7f3fa0b5c700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3f8006a4d0 0x7f3f8006c990 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009130 tx=0x7f3f8c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:58.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.606+0000 7f3f91ffb700 1 -- 172.21.15.112:0/1775477584 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3f98053e50 con 0x7f3f9c06e150 2023-08-27T11:29:58.873 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:58.872+0000 7f3fa35c1700 1 -- 172.21.15.112:0/1775477584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]} v 0) v1 -- 0x7f3f9c071c90 con 0x7f3f9c06e150 2023-08-27T11:29:59.462 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.461+0000 7f3f91ffb700 1 -- 172.21.15.112:0/1775477584 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]}]=0 v606) v1 ==== 150+0+0 (secure 0 0 0) 0x7f3f980150d0 con 0x7f3f9c06e150 2023-08-27T11:29:59.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.463+0000 7f3fa35c1700 1 -- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3f8006a4d0 msgr2=0x7f3f8006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.463+0000 7f3fa35c1700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3f8006a4d0 0x7f3f8006c990 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009130 tx=0x7f3f8c008030 comp rx=0 tx=0).stop 2023-08-27T11:29:59.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.463+0000 7f3fa35c1700 1 -- 172.21.15.112:0/1775477584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f9c06e150 msgr2=0x7f3f9c0716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.463+0000 7f3fa35c1700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f9c06e150 0x7f3f9c0716f0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800b530 tx=0x7f3f9800b8f0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.465 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.463+0000 7f3fa35c1700 1 -- 172.21.15.112:0/1775477584 shutdown_connections 2023-08-27T11:29:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.463+0000 7f3fa35c1700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f3f8006a4d0 0x7f3f8006c990 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.463+0000 7f3fa35c1700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f9c159e00 0x7f3f9c154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.463+0000 7f3fa35c1700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f9c06e150 0x7f3f9c0716f0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.463+0000 7f3fa35c1700 1 --2- 172.21.15.112:0/1775477584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f9c153e70 0x7f3f9c1411b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.463+0000 7f3fa35c1700 1 -- 172.21.15.112:0/1775477584 >> 172.21.15.112:0/1775477584 conn(0x7f3f9c06cd30 msgr2=0x7f3f9c06e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:59.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.463+0000 7f3fa35c1700 1 -- 172.21.15.112:0/1775477584 shutdown_connections 2023-08-27T11:29:59.467 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.463+0000 7f3fa35c1700 1 -- 172.21.15.112:0/1775477584 wait complete. 2023-08-27T11:29:59.477 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:29:59.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.597+0000 7ff8bb8e8700 1 Processor -- start 2023-08-27T11:29:59.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.597+0000 7ff8bb8e8700 1 -- start start 2023-08-27T11:29:59.602 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.598+0000 7ff8bb8e8700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff8b406fea0 0x7ff8b40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.598+0000 7ff8bb8e8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b4161850 con 0x7ff8b40b1400 2023-08-27T11:29:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.598+0000 7ff8bb8e8700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b40cfc10 con 0x7ff8b40b17e0 2023-08-27T11:29:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.598+0000 7ff8bb8e8700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b4161200 con 0x7ff8b406fea0 2023-08-27T11:29:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.598+0000 7ff8b9684700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff8b40b1400 0x7ff8b406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33418/0 (socket says 172.21.15.112:33418) 2023-08-27T11:29:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.598+0000 7ff8b9684700 1 -- 172.21.15.112:0/2998884957 learned_addr learned my addr 172.21.15.112:0/2998884957 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.598+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228984288 0 0) 0x7ff8b4161850 con 0x7ff8b40b1400 2023-08-27T11:29:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8a00035b0 con 0x7ff8b40b1400 2023-08-27T11:29:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8b9e85700 1 --2- 172.21.15.112:0/2998884957 >> v2:172.21.15.138:3300/0 conn(0x7ff8b406fea0 0x7ff8b40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2528833066 0 0) 0x7ff8a00035b0 con 0x7ff8b40b1400 2023-08-27T11:29:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 >> v2:172.21.15.138:3300/0 conn(0x7ff8b406fea0 msgr2=0x7ff8b40cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:59.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 --2- 172.21.15.112:0/2998884957 >> v2:172.21.15.138:3300/0 conn(0x7ff8b406fea0 0x7ff8b40cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 >> v1:172.21.15.138:6789/0 conn(0x7ff8b40b17e0 legacy=0x7ff8b406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b41614e0 con 0x7ff8b40b1400 2023-08-27T11:29:59.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7ff8a4001670 con 0x7ff8b40b1400 2023-08-27T11:29:59.605 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 >> v1:172.21.15.112:6789/0 conn(0x7ff8b40b1400 legacy=0x7ff8b406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 --2- 172.21.15.112:0/2998884957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a0003a00 0x7ff8a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 --2- 172.21.15.112:0/2998884957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a0004850 0x7ff8a0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 --2- 172.21.15.112:0/2998884957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8a0005210 0x7ff8a000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b4161200 con 0x7ff8a0004850 2023-08-27T11:29:59.606 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a000c0b0 con 0x7ff8a0003a00 2023-08-27T11:29:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a000c270 con 0x7ff8a0005210 2023-08-27T11:29:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8b8e83700 1 --2- 172.21.15.112:0/2998884957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a0004850 0x7ff8a0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8b9684700 1 --2- 172.21.15.112:0/2998884957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a0003a00 0x7ff8a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8b9e85700 1 --2- 172.21.15.112:0/2998884957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8a0005210 0x7ff8a000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8b8e83700 1 -- 172.21.15.112:0/2998884957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8a0005210 msgr2=0x7ff8a000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8b8e83700 1 --2- 172.21.15.112:0/2998884957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8a0005210 0x7ff8a000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8b8e83700 1 -- 172.21.15.112:0/2998884957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a0003a00 msgr2=0x7ff8a0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8b8e83700 1 --2- 172.21.15.112:0/2998884957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a0003a00 0x7ff8a0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8b8e83700 1 -- 172.21.15.112:0/2998884957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a00035b0 con 0x7ff8a0004850 2023-08-27T11:29:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8b9e85700 1 --2- 172.21.15.112:0/2998884957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8a0005210 0x7ff8a000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8b9e85700 1 -- 172.21.15.112:0/2998884957 reap_dead start 2023-08-27T11:29:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.599+0000 7ff8b8e83700 1 --2- 172.21.15.112:0/2998884957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a0004850 0x7ff8a0004cd0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40b1bf0 tx=0x7ff8a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffdc11319272c9b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.600+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00052d0 con 0x7ff8a0004850 2023-08-27T11:29:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.600+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b0005430 con 0x7ff8a0004850 2023-08-27T11:29:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.600+0000 7ff8abfff700 1 -- 172.21.15.112:0/2998884957 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00038c0 con 0x7ff8a0004850 2023-08-27T11:29:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.600+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/2998884957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a0004850 msgr2=0x7ff8a0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.600+0000 7ff8bb8e8700 1 --2- 172.21.15.112:0/2998884957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a0004850 0x7ff8a0004cd0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40b1bf0 tx=0x7ff8a0005880 comp rx=0 tx=0).stop 2023-08-27T11:29:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.600+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/2998884957 shutdown_connections 2023-08-27T11:29:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.600+0000 7ff8bb8e8700 1 --2- 172.21.15.112:0/2998884957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a0004850 0x7ff8a0004cd0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.600+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/2998884957 >> 172.21.15.112:0/2998884957 conn(0x7ff8b40d0000 msgr2=0x7ff8b40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.600+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/2998884957 shutdown_connections 2023-08-27T11:29:59.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.600+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/2998884957 wait complete. 2023-08-27T11:29:59.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.601+0000 7ff8bb8e8700 1 Processor -- start 2023-08-27T11:29:59.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.601+0000 7ff8bb8e8700 1 -- start start 2023-08-27T11:29:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8bb8e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a0004850 0x7ff8b40c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8bb8e8700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8b41f6e60 0x7ff8b40c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8bb8e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b41f62c0 0x7ff8b41f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8bb8e8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b4152d70 con 0x7ff8b41f62c0 2023-08-27T11:29:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8bb8e8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b41f6c60 con 0x7ff8a0004850 2023-08-27T11:29:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8bb8e8700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b40c7610 con 0x7ff8b41f6e60 2023-08-27T11:29:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8b9e85700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b41f62c0 0x7ff8b41f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8b9e85700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b41f62c0 0x7ff8b41f6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39052/0 (socket says 172.21.15.112:39052) 2023-08-27T11:29:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8b9e85700 1 -- 172.21.15.112:0/3492219518 learned_addr learned my addr 172.21.15.112:0/3492219518 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8b9684700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a0004850 0x7ff8b40c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8b8e83700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8b41f6e60 0x7ff8b40c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8b9e85700 1 -- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8b41f6e60 msgr2=0x7ff8b40c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8b9e85700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8b41f6e60 0x7ff8b40c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8b9e85700 1 -- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a0004850 msgr2=0x7ff8b40c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8b9e85700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a0004850 0x7ff8b40c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8b9e85700 1 -- 172.21.15.112:0/3492219518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b40c9a00 con 0x7ff8b41f62c0 2023-08-27T11:29:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.602+0000 7ff8b8e83700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8b41f6e60 0x7ff8b40c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.603+0000 7ff8b9e85700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b41f62c0 0x7ff8b41f6720 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7ff8b406c130 tx=0x7ff8ac0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.603+0000 7ff8a9ffb700 1 -- 172.21.15.112:0/3492219518 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac00e050 con 0x7ff8b41f62c0 2023-08-27T11:29:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.603+0000 7ff8a9ffb700 1 -- 172.21.15.112:0/3492219518 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac00f040 con 0x7ff8b41f62c0 2023-08-27T11:29:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.603+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/3492219518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b40c9ce0 con 0x7ff8b41f62c0 2023-08-27T11:29:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.603+0000 7ff8a9ffb700 1 -- 172.21.15.112:0/3492219518 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac010e30 con 0x7ff8b41f62c0 2023-08-27T11:29:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.603+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/3492219518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b40c83f0 con 0x7ff8b41f62c0 2023-08-27T11:29:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.604+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/3492219518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b40c7750 con 0x7ff8b41f62c0 2023-08-27T11:29:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.604+0000 7ff8a9ffb700 1 -- 172.21.15.112:0/3492219518 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7ff8ac015440 con 0x7ff8b41f62c0 2023-08-27T11:29:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.605+0000 7ff8a9ffb700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff89806a4d0 0x7ff89806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.605+0000 7ff8a9ffb700 1 -- 172.21.15.112:0/3492219518 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff8ac088d40 con 0x7ff8b41f62c0 2023-08-27T11:29:59.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.605+0000 7ff8b9684700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff89806a4d0 0x7ff89806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.605+0000 7ff8b9684700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff89806a4d0 0x7ff89806c990 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4007cd0 tx=0x7ff8a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:59.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.609+0000 7ff8a9ffb700 1 -- 172.21.15.112:0/3492219518 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff8ac054180 con 0x7ff8b41f62c0 2023-08-27T11:29:59.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.803+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/3492219518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8b4067a20 con 0x7ff8b41f62c0 2023-08-27T11:29:59.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.803+0000 7ff8a9ffb700 1 -- 172.21.15.112:0/3492219518 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+14746 (secure 0 0 0) 0x7ff8ac0577a0 con 0x7ff8b41f62c0 2023-08-27T11:29:59.805 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:29:59.805 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":176,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:29:57.452754+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6833/4086629952":"2023-08-28T11:29:53.299683+0000","172.21.15.112:6832/4086629952":"2023-08-28T11:29:53.299683+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:29:59.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.806+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff89806a4d0 msgr2=0x7ff89806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.806+0000 7ff8bb8e8700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff89806a4d0 0x7ff89806c990 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4007cd0 tx=0x7ff8a4009040 comp rx=0 tx=0).stop 2023-08-27T11:29:59.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.806+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/3492219518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b41f62c0 msgr2=0x7ff8b41f6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.806+0000 7ff8bb8e8700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b41f62c0 0x7ff8b41f6720 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7ff8b406c130 tx=0x7ff8ac0104a0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.806+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/3492219518 shutdown_connections 2023-08-27T11:29:59.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.806+0000 7ff8bb8e8700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff89806a4d0 0x7ff89806c990 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.806+0000 7ff8bb8e8700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b41f62c0 0x7ff8b41f6720 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.806+0000 7ff8bb8e8700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a0004850 0x7ff8b40c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.806+0000 7ff8bb8e8700 1 --2- 172.21.15.112:0/3492219518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8b41f6e60 0x7ff8b40c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.806+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/3492219518 >> 172.21.15.112:0/3492219518 conn(0x7ff8b40d0000 msgr2=0x7ff8b415eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.806+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/3492219518 shutdown_connections 2023-08-27T11:29:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.806+0000 7ff8bb8e8700 1 -- 172.21.15.112:0/3492219518 wait complete. 2023-08-27T11:29:59.819 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:29:59.820 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:29:59.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.941+0000 7f559e8f2700 1 Processor -- start 2023-08-27T11:29:59.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.941+0000 7f559e8f2700 1 -- start start 2023-08-27T11:29:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.942+0000 7f559e8f2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55980af2b0 0x7f559815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.942+0000 7f559e8f2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5598161190 con 0x7f55980aee70 2023-08-27T11:29:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.942+0000 7f559e8f2700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5598161470 con 0x7f559815af20 2023-08-27T11:29:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.942+0000 7f559e8f2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559806bce0 con 0x7f55980af2b0 2023-08-27T11:29:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.942+0000 7f55977fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f55980aee70 0x7f559815d9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33438/0 (socket says 172.21.15.112:33438) 2023-08-27T11:29:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.942+0000 7f55977fe700 1 -- 172.21.15.112:0/1032464564 learned_addr learned my addr 172.21.15.112:0/1032464564 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:29:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.942+0000 7f559ce8f700 1 --2- 172.21.15.112:0/1032464564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55980af2b0 0x7f559815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713812630 0 0) 0x7f5598161190 con 0x7f55980aee70 2023-08-27T11:29:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55800035b0 con 0x7f55980aee70 2023-08-27T11:29:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409666832 0 0) 0x7f5598161470 con 0x7f559815af20 2023-08-27T11:29:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5598161190 con 0x7f559815af20 2023-08-27T11:29:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2594749846 0 0) 0x7f55800035b0 con 0x7f55980aee70 2023-08-27T11:29:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55980af2b0 msgr2=0x7f559815e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 --2- 172.21.15.112:0/1032464564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55980af2b0 0x7f559815e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 >> v1:172.21.15.138:6789/0 conn(0x7f559815af20 legacy=0x7f559806b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5598161750 con 0x7f55980aee70 2023-08-27T11:29:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f5588001670 con 0x7f55980aee70 2023-08-27T11:29:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 >> v1:172.21.15.112:6789/0 conn(0x7f55980aee70 legacy=0x7f559815d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 --2- 172.21.15.112:0/1032464564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5580003bc0 0x7f5580003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 --2- 172.21.15.112:0/1032464564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55980af2b0 0x7f5580004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 --2- 172.21.15.112:0/1032464564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5580004d90 0x7f5580005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559806bce0 con 0x7f5580004d90 2023-08-27T11:29:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558000bc00 con 0x7f5580003bc0 2023-08-27T11:29:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558000bdc0 con 0x7f55980af2b0 2023-08-27T11:29:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f559ce8f700 1 --2- 172.21.15.112:0/1032464564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5580004d90 0x7f5580005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f55977fe700 1 --2- 172.21.15.112:0/1032464564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55980af2b0 0x7f5580004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f5597fff700 1 --2- 172.21.15.112:0/1032464564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5580003bc0 0x7f5580003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f559ce8f700 1 -- 172.21.15.112:0/1032464564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55980af2b0 msgr2=0x7f5580004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f559ce8f700 1 --2- 172.21.15.112:0/1032464564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55980af2b0 0x7f5580004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f559ce8f700 1 -- 172.21.15.112:0/1032464564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5580003bc0 msgr2=0x7f5580003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f559ce8f700 1 --2- 172.21.15.112:0/1032464564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5580003bc0 0x7f5580003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f559ce8f700 1 -- 172.21.15.112:0/1032464564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55800035b0 con 0x7f5580004d90 2023-08-27T11:29:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.943+0000 7f559ce8f700 1 --2- 172.21.15.112:0/1032464564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5580004d90 0x7f5580005210 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f5590009fc0 tx=0x7f5580005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7131b61ba89072cd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.944+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559000ec30 con 0x7f5580004d90 2023-08-27T11:29:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.944+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559000f040 con 0x7f5580004d90 2023-08-27T11:29:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.944+0000 7f5596ffd700 1 -- 172.21.15.112:0/1032464564 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590014480 con 0x7f5580004d90 2023-08-27T11:29:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.944+0000 7f559e8f2700 1 -- 172.21.15.112:0/1032464564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5580004d90 msgr2=0x7f5580005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.944+0000 7f559e8f2700 1 --2- 172.21.15.112:0/1032464564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5580004d90 0x7f5580005210 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f5590009fc0 tx=0x7f5580005880 comp rx=0 tx=0).stop 2023-08-27T11:29:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.944+0000 7f559ce8f700 1 -- 172.21.15.112:0/1032464564 reap_dead start 2023-08-27T11:29:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.944+0000 7f559e8f2700 1 -- 172.21.15.112:0/1032464564 shutdown_connections 2023-08-27T11:29:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.944+0000 7f559e8f2700 1 -- 172.21.15.112:0/1032464564 >> 172.21.15.112:0/1032464564 conn(0x7f55980d03e0 msgr2=0x7f55980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.944+0000 7f559e8f2700 1 -- 172.21.15.112:0/1032464564 shutdown_connections 2023-08-27T11:29:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.944+0000 7f559e8f2700 1 -- 172.21.15.112:0/1032464564 wait complete. 2023-08-27T11:29:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.945+0000 7f559e8f2700 1 Processor -- start 2023-08-27T11:29:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.946+0000 7f559e8f2700 1 -- start start 2023-08-27T11:29:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.946+0000 7f559e8f2700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55981f26d0 0x7f55981f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.946+0000 7f559e8f2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559815d330 0x7f55981f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.946+0000 7f559e8f2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55980ad8e0 0x7f55980add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.946+0000 7f559e8f2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55981f32f0 con 0x7f55980ad8e0 2023-08-27T11:29:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.946+0000 7f559e8f2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55980ae2d0 con 0x7f559815d330 2023-08-27T11:29:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.946+0000 7f559e8f2700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55981f7290 con 0x7f55981f26d0 2023-08-27T11:29:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f559ce8f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55980ad8e0 0x7f55980add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f559ce8f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55980ad8e0 0x7f55980add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39070/0 (socket says 172.21.15.112:39070) 2023-08-27T11:29:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f559ce8f700 1 -- 172.21.15.112:0/1531811502 learned_addr learned my addr 172.21.15.112:0/1531811502 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:29:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f5597fff700 1 --2- 172.21.15.112:0/1531811502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55981f26d0 0x7f55981f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f559ce8f700 1 -- 172.21.15.112:0/1531811502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55981f26d0 msgr2=0x7f55981f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f559ce8f700 1 --2- 172.21.15.112:0/1531811502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55981f26d0 0x7f55981f3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f559ce8f700 1 -- 172.21.15.112:0/1531811502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559815d330 msgr2=0x7f55981f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f559ce8f700 1 --2- 172.21.15.112:0/1531811502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559815d330 0x7f55981f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f559ce8f700 1 -- 172.21.15.112:0/1531811502 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55981f6cd0 con 0x7f55980ad8e0 2023-08-27T11:29:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f559ce8f700 1 --2- 172.21.15.112:0/1531811502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55980ad8e0 0x7f55980add90 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f5590016000 tx=0x7f5590009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f5594ff9700 1 -- 172.21.15.112:0/1531811502 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590007000 con 0x7f55980ad8e0 2023-08-27T11:29:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f5594ff9700 1 -- 172.21.15.112:0/1531811502 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5590015b30 con 0x7f55980ad8e0 2023-08-27T11:29:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f559e8f2700 1 -- 172.21.15.112:0/1531811502 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55981f6fb0 con 0x7f55980ad8e0 2023-08-27T11:29:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f5594ff9700 1 -- 172.21.15.112:0/1531811502 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559001d420 con 0x7f55980ad8e0 2023-08-27T11:29:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.947+0000 7f559e8f2700 1 -- 172.21.15.112:0/1531811502 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55981f3c50 con 0x7f55980ad8e0 2023-08-27T11:29:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.948+0000 7f559e8f2700 1 -- 172.21.15.112:0/1531811502 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55981f3430 con 0x7f55980ad8e0 2023-08-27T11:29:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.948+0000 7f5594ff9700 1 -- 172.21.15.112:0/1531811502 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f559001d580 con 0x7f55980ad8e0 2023-08-27T11:29:59.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.949+0000 7f5594ff9700 1 --2- 172.21.15.112:0/1531811502 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f558406a4d0 0x7f558406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:59.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.949+0000 7f5594ff9700 1 -- 172.21.15.112:0/1531811502 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f559008a1c0 con 0x7f55980ad8e0 2023-08-27T11:29:59.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.949+0000 7f5597fff700 1 --2- 172.21.15.112:0/1531811502 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f558406a4d0 0x7f558406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:59.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.950+0000 7f5597fff700 1 --2- 172.21.15.112:0/1531811502 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f558406a4d0 0x7f558406c990 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f558c0010e0 tx=0x7f558c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:59.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:29:59.953+0000 7f5594ff9700 1 -- 172.21.15.112:0/1531811502 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f55981f3430 con 0x7f55980ad8e0 2023-08-27T11:30:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.142+0000 7f559e8f2700 1 -- 172.21.15.112:0/1531811502 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f55981f3430 con 0x7f55980ad8e0 2023-08-27T11:30:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.142+0000 7f5594ff9700 1 -- 172.21.15.112:0/1531811502 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v176) v1 ==== 92+0+10 (secure 0 0 0) 0x7f55981f3430 con 0x7f55980ad8e0 2023-08-27T11:30:00.144 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:30:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.144+0000 7f559e8f2700 1 -- 172.21.15.112:0/1531811502 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f558406a4d0 msgr2=0x7f558406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.144+0000 7f559e8f2700 1 --2- 172.21.15.112:0/1531811502 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f558406a4d0 0x7f558406c990 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f558c0010e0 tx=0x7f558c009040 comp rx=0 tx=0).stop 2023-08-27T11:30:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.144+0000 7f559e8f2700 1 -- 172.21.15.112:0/1531811502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55980ad8e0 msgr2=0x7f55980add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.144+0000 7f559e8f2700 1 --2- 172.21.15.112:0/1531811502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55980ad8e0 0x7f55980add90 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f5590016000 tx=0x7f5590009750 comp rx=0 tx=0).stop 2023-08-27T11:30:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.144+0000 7f559e8f2700 1 -- 172.21.15.112:0/1531811502 shutdown_connections 2023-08-27T11:30:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.144+0000 7f559e8f2700 1 --2- 172.21.15.112:0/1531811502 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f558406a4d0 0x7f558406c990 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.144+0000 7f559e8f2700 1 --2- 172.21.15.112:0/1531811502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55980ad8e0 0x7f55980add90 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.144+0000 7f559e8f2700 1 --2- 172.21.15.112:0/1531811502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55981f26d0 0x7f55981f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.144+0000 7f559e8f2700 1 --2- 172.21.15.112:0/1531811502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559815d330 0x7f55981f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.144+0000 7f559e8f2700 1 -- 172.21.15.112:0/1531811502 >> 172.21.15.112:0/1531811502 conn(0x7f55980d03e0 msgr2=0x7f559806b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.144+0000 7f559e8f2700 1 -- 172.21.15.112:0/1531811502 shutdown_connections 2023-08-27T11:30:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.144+0000 7f559e8f2700 1 -- 172.21.15.112:0/1531811502 wait complete. 2023-08-27T11:30:00.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:30:00.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.280+0000 7fc2ccc4e700 1 Processor -- start 2023-08-27T11:30:00.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.280+0000 7fc2ccc4e700 1 -- start start 2023-08-27T11:30:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.281+0000 7fc2ccc4e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806f790 0x7fc2c806fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.281+0000 7fc2ccc4e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c80b5070 con 0x7fc2c8146c20 2023-08-27T11:30:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.281+0000 7fc2ccc4e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c8147000 con 0x7fc2c80b3440 2023-08-27T11:30:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.281+0000 7fc2ccc4e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8070160 con 0x7fc2c806f790 2023-08-27T11:30:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c659c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc2c8146c20 0x7fc2c8160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33458/0 (socket says 172.21.15.112:33458) 2023-08-27T11:30:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c659c700 1 -- 172.21.15.112:0/1585922536 learned_addr learned my addr 172.21.15.112:0/1585922536 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692513400 0 0) 0x7fc2b0001110 con 0x7fc2c8146c20 2023-08-27T11:30:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c6d9d700 1 --2- 172.21.15.112:0/1585922536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806f790 0x7fc2c806fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2b40035b0 con 0x7fc2c8146c20 2023-08-27T11:30:00.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2106230194 0 0) 0x7fc2b40035b0 con 0x7fc2c8146c20 2023-08-27T11:30:00.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806f790 msgr2=0x7fc2c806fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:00.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 --2- 172.21.15.112:0/1585922536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806f790 0x7fc2c806fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 >> v1:172.21.15.138:6789/0 conn(0x7fc2c80b3440 legacy=0x7fc2c815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c8161740 con 0x7fc2c8146c20 2023-08-27T11:30:00.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fc2b0001800 con 0x7fc2c8146c20 2023-08-27T11:30:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 >> v1:172.21.15.112:6789/0 conn(0x7fc2c8146c20 legacy=0x7fc2c8160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 --2- 172.21.15.112:0/1585922536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2b4003a30 0x7fc2b4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 --2- 172.21.15.112:0/1585922536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806f790 0x7fc2b4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 --2- 172.21.15.112:0/1585922536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b4004d90 0x7fc2b4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8070160 con 0x7fc2b4004d90 2023-08-27T11:30:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b400bb70 con 0x7fc2b4003a30 2023-08-27T11:30:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b400bd30 con 0x7fc2c806f790 2023-08-27T11:30:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c6d9d700 1 --2- 172.21.15.112:0/1585922536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b4004d90 0x7fc2b4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c659c700 1 --2- 172.21.15.112:0/1585922536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2b4003a30 0x7fc2b4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.282+0000 7fc2c5d9b700 1 --2- 172.21.15.112:0/1585922536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806f790 0x7fc2b4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2c6d9d700 1 -- 172.21.15.112:0/1585922536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806f790 msgr2=0x7fc2b4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2c6d9d700 1 --2- 172.21.15.112:0/1585922536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c806f790 0x7fc2b4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2c6d9d700 1 -- 172.21.15.112:0/1585922536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2b4003a30 msgr2=0x7fc2b4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2c6d9d700 1 --2- 172.21.15.112:0/1585922536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2b4003a30 0x7fc2b4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2c6d9d700 1 -- 172.21.15.112:0/1585922536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b40035b0 con 0x7fc2b4004d90 2023-08-27T11:30:00.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2c6d9d700 1 --2- 172.21.15.112:0/1585922536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b4004d90 0x7fc2b4005210 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009fc0 tx=0x7fc2b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b9722563048640 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:00.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c000ec30 con 0x7fc2b4004d90 2023-08-27T11:30:00.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c000f040 con 0x7fc2b4004d90 2023-08-27T11:30:00.292 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2c559a700 1 -- 172.21.15.112:0/1585922536 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0014480 con 0x7fc2b4004d90 2023-08-27T11:30:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/1585922536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b4004d90 msgr2=0x7fc2b4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2ccc4e700 1 --2- 172.21.15.112:0/1585922536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b4004d90 0x7fc2b4005210 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009fc0 tx=0x7fc2b4005880 comp rx=0 tx=0).stop 2023-08-27T11:30:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2c6d9d700 1 -- 172.21.15.112:0/1585922536 reap_dead start 2023-08-27T11:30:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/1585922536 shutdown_connections 2023-08-27T11:30:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/1585922536 >> 172.21.15.112:0/1585922536 conn(0x7fc2c80b1e90 msgr2=0x7fc2c80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.283+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/1585922536 shutdown_connections 2023-08-27T11:30:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.284+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/1585922536 wait complete. 2023-08-27T11:30:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.285+0000 7fc2ccc4e700 1 Processor -- start 2023-08-27T11:30:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.285+0000 7fc2ccc4e700 1 -- start start 2023-08-27T11:30:00.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.285+0000 7fc2ccc4e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b4004d90 0x7fc2c815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.285+0000 7fc2ccc4e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c81420a0 0x7fc2c8142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.297 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2ccc4e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c813f620 0x7fc2c813fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2ccc4e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8140010 con 0x7fc2b4004d90 2023-08-27T11:30:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2ccc4e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c81f2ea0 con 0x7fc2c813f620 2023-08-27T11:30:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2ccc4e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c81f3180 con 0x7fc2c81420a0 2023-08-27T11:30:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2c659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b4004d90 0x7fc2c815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2c659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b4004d90 0x7fc2c815c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39088/0 (socket says 172.21.15.112:39088) 2023-08-27T11:30:00.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2c659c700 1 -- 172.21.15.112:0/3272396177 learned_addr learned my addr 172.21.15.112:0/3272396177 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:00.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2c5d9b700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c81420a0 0x7fc2c8142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2c6d9d700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c813f620 0x7fc2c813fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2c659c700 1 -- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c81420a0 msgr2=0x7fc2c8142500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.299 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2c659c700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c81420a0 0x7fc2c8142500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2c659c700 1 -- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c813f620 msgr2=0x7fc2c813fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2c659c700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c813f620 0x7fc2c813fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2c659c700 1 -- 172.21.15.112:0/3272396177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c81f3460 con 0x7fc2b4004d90 2023-08-27T11:30:00.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2c6d9d700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c813f620 0x7fc2c813fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:00.300 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2c659c700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b4004d90 0x7fc2c815c3e0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0000fb0 tx=0x7fc2b00075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:00.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2beffd700 1 -- 172.21.15.112:0/3272396177 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b000f660 con 0x7fc2b4004d90 2023-08-27T11:30:00.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2beffd700 1 -- 172.21.15.112:0/3272396177 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b000f7c0 con 0x7fc2b4004d90 2023-08-27T11:30:00.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/3272396177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c81f3740 con 0x7fc2b4004d90 2023-08-27T11:30:00.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2beffd700 1 -- 172.21.15.112:0/3272396177 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b000fb50 con 0x7fc2b4004d90 2023-08-27T11:30:00.301 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.286+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/3272396177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c81f2b00 con 0x7fc2b4004d90 2023-08-27T11:30:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.287+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/3272396177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c80cf9a0 con 0x7fc2b4004d90 2023-08-27T11:30:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.288+0000 7fc2beffd700 1 -- 172.21.15.112:0/3272396177 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fc2b000fcb0 con 0x7fc2b4004d90 2023-08-27T11:30:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.288+0000 7fc2beffd700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc2a806a4d0 0x7fc2a806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.288+0000 7fc2beffd700 1 -- 172.21.15.112:0/3272396177 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc2b00886e0 con 0x7fc2b4004d90 2023-08-27T11:30:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.288+0000 7fc2c5d9b700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc2a806a4d0 0x7fc2a806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.289+0000 7fc2c5d9b700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc2a806a4d0 0x7fc2a806c990 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80b30a0 tx=0x7fc2b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.292+0000 7fc2beffd700 1 -- 172.21.15.112:0/3272396177 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc2b0053c00 con 0x7fc2b4004d90 2023-08-27T11:30:00.484 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.483+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/3272396177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2c8067e90 con 0x7fc2b4004d90 2023-08-27T11:30:00.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.483+0000 7fc2beffd700 1 -- 172.21.15.112:0/3272396177 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v176) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc2b0057220 con 0x7fc2b4004d90 2023-08-27T11:30:00.485 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:30:00.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.486+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc2a806a4d0 msgr2=0x7fc2a806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.486+0000 7fc2ccc4e700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc2a806a4d0 0x7fc2a806c990 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80b30a0 tx=0x7fc2b8009040 comp rx=0 tx=0).stop 2023-08-27T11:30:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.486+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/3272396177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b4004d90 msgr2=0x7fc2c815c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.486+0000 7fc2ccc4e700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b4004d90 0x7fc2c815c3e0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0000fb0 tx=0x7fc2b00075f0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.486+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/3272396177 shutdown_connections 2023-08-27T11:30:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.486+0000 7fc2ccc4e700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc2a806a4d0 0x7fc2a806c990 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.486+0000 7fc2ccc4e700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c813f620 0x7fc2c813fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.486+0000 7fc2ccc4e700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b4004d90 0x7fc2c815c3e0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.486+0000 7fc2ccc4e700 1 --2- 172.21.15.112:0/3272396177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2c81420a0 0x7fc2c8142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.486+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/3272396177 >> 172.21.15.112:0/3272396177 conn(0x7fc2c80b1e90 msgr2=0x7fc2c8070750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.486+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/3272396177 shutdown_connections 2023-08-27T11:30:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.486+0000 7fc2ccc4e700 1 -- 172.21.15.112:0/3272396177 wait complete. 2023-08-27T11:30:00.499 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:30:00.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.621+0000 7ff168c5f700 1 Processor -- start 2023-08-27T11:30:00.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.621+0000 7ff168c5f700 1 -- start start 2023-08-27T11:30:00.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.622+0000 7ff168c5f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff164146c20 0x7ff164160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.622+0000 7ff168c5f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1640b3440 0x7ff16415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.622+0000 7ff168c5f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1640b5070 con 0x7ff16415e6e0 2023-08-27T11:30:00.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.622+0000 7ff168c5f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff164147000 con 0x7ff164146c20 2023-08-27T11:30:00.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.622+0000 7ff168c5f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1641614a0 con 0x7ff1640b3440 2023-08-27T11:30:00.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.622+0000 7ff16259c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff164146c20 0x7ff164160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.623+0000 7ff162d9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff16415e6e0 0x7ff16406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33482/0 (socket says 172.21.15.112:33482) 2023-08-27T11:30:00.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.623+0000 7ff162d9d700 1 -- 172.21.15.112:0/458067805 learned_addr learned my addr 172.21.15.112:0/458067805 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.623+0000 7ff161d9b700 1 --2- 172.21.15.112:0/458067805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1640b3440 0x7ff16415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.623+0000 7ff16259c700 1 -- 172.21.15.112:0/458067805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1640b3440 msgr2=0x7ff16415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.623+0000 7ff16259c700 1 --2- 172.21.15.112:0/458067805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1640b3440 0x7ff16415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.623+0000 7ff16259c700 1 -- 172.21.15.112:0/458067805 >> v1:172.21.15.112:6789/0 conn(0x7ff16415e6e0 legacy=0x7ff16406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.623+0000 7ff16259c700 1 -- 172.21.15.112:0/458067805 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff164161740 con 0x7ff164146c20 2023-08-27T11:30:00.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.623+0000 7ff16159a700 1 -- 172.21.15.112:0/458067805 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203879049 0 0) 0x7ff15c001490 con 0x7ff16415e6e0 2023-08-27T11:30:00.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.623+0000 7ff16259c700 1 --2- 172.21.15.112:0/458067805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff164146c20 0x7ff164160f60 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7ff158009840 tx=0x7ff158009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d813063431b5797 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:00.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.623+0000 7ff16159a700 1 -- 172.21.15.112:0/458067805 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158011830 con 0x7ff164146c20 2023-08-27T11:30:00.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.623+0000 7ff16159a700 1 -- 172.21.15.112:0/458067805 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff158011990 con 0x7ff164146c20 2023-08-27T11:30:00.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.623+0000 7ff16159a700 1 -- 172.21.15.112:0/458067805 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158011d20 con 0x7ff164146c20 2023-08-27T11:30:00.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.624+0000 7ff168c5f700 1 -- 172.21.15.112:0/458067805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff164146c20 msgr2=0x7ff164160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.624+0000 7ff168c5f700 1 --2- 172.21.15.112:0/458067805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff164146c20 0x7ff164160f60 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7ff158009840 tx=0x7ff158009b50 comp rx=0 tx=0).stop 2023-08-27T11:30:00.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.624+0000 7ff168c5f700 1 -- 172.21.15.112:0/458067805 shutdown_connections 2023-08-27T11:30:00.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.624+0000 7ff168c5f700 1 --2- 172.21.15.112:0/458067805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff164146c20 0x7ff164160f60 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.624+0000 7ff168c5f700 1 --2- 172.21.15.112:0/458067805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1640b3440 0x7ff16415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.624+0000 7ff168c5f700 1 -- 172.21.15.112:0/458067805 >> 172.21.15.112:0/458067805 conn(0x7ff1640b1e90 msgr2=0x7ff1640b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.624+0000 7ff168c5f700 1 -- 172.21.15.112:0/458067805 shutdown_connections 2023-08-27T11:30:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.624+0000 7ff168c5f700 1 -- 172.21.15.112:0/458067805 wait complete. 2023-08-27T11:30:00.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.625+0000 7ff168c5f700 1 Processor -- start 2023-08-27T11:30:00.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.625+0000 7ff168c5f700 1 -- start start 2023-08-27T11:30:00.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff168c5f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1640b3440 0x7ff16413af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff168c5f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff164146c20 0x7ff1641387a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff168c5f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff16415e6e0 0x7ff164070e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff168c5f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16413a240 con 0x7ff164146c20 2023-08-27T11:30:00.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff168c5f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16414d240 con 0x7ff16415e6e0 2023-08-27T11:30:00.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff168c5f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16414cf60 con 0x7ff1640b3440 2023-08-27T11:30:00.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff161d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff164146c20 0x7ff1641387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff161d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff164146c20 0x7ff1641387a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39104/0 (socket says 172.21.15.112:39104) 2023-08-27T11:30:00.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff161d9b700 1 -- 172.21.15.112:0/1349889615 learned_addr learned my addr 172.21.15.112:0/1349889615 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:00.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff161d9b700 1 -- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1640b3440 msgr2=0x7ff16413af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff16259c700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1640b3440 0x7ff16413af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff162d9d700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff16415e6e0 0x7ff164070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff161d9b700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1640b3440 0x7ff16413af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff161d9b700 1 -- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff16415e6e0 msgr2=0x7ff164070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff161d9b700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff16415e6e0 0x7ff164070e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff161d9b700 1 -- 172.21.15.112:0/1349889615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1580094a0 con 0x7ff164146c20 2023-08-27T11:30:00.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff162d9d700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff16415e6e0 0x7ff164070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:00.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff161d9b700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff164146c20 0x7ff1641387a0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7ff15400e9b0 tx=0x7ff15400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:00.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff152ffd700 1 -- 172.21.15.112:0/1349889615 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154009710 con 0x7ff164146c20 2023-08-27T11:30:00.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff152ffd700 1 -- 172.21.15.112:0/1349889615 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15400c850 con 0x7ff164146c20 2023-08-27T11:30:00.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff168c5f700 1 -- 172.21.15.112:0/1349889615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16413a470 con 0x7ff164146c20 2023-08-27T11:30:00.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.626+0000 7ff152ffd700 1 -- 172.21.15.112:0/1349889615 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15400cbc0 con 0x7ff164146c20 2023-08-27T11:30:00.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.627+0000 7ff168c5f700 1 -- 172.21.15.112:0/1349889615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16413aa30 con 0x7ff164146c20 2023-08-27T11:30:00.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.627+0000 7ff168c5f700 1 -- 172.21.15.112:0/1349889615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff164067e90 con 0x7ff164146c20 2023-08-27T11:30:00.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.628+0000 7ff152ffd700 1 -- 172.21.15.112:0/1349889615 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7ff15400cd20 con 0x7ff164146c20 2023-08-27T11:30:00.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.628+0000 7ff152ffd700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff14406a500 0x7ff14406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:00.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.628+0000 7ff152ffd700 1 -- 172.21.15.112:0/1349889615 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff154088b90 con 0x7ff164146c20 2023-08-27T11:30:00.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.629+0000 7ff16259c700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff14406a500 0x7ff14406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:00.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.629+0000 7ff16259c700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff14406a500 0x7ff14406c9c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff1580090b0 tx=0x7ff158008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:00.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.633+0000 7ff152ffd700 1 -- 172.21.15.112:0/1349889615 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff1540579a0 con 0x7ff164146c20 2023-08-27T11:30:00.822 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.821+0000 7ff168c5f700 1 -- 172.21.15.112:0/1349889615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff164161cd0 con 0x7ff164146c20 2023-08-27T11:30:00.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.825+0000 7ff152ffd700 1 -- 172.21.15.112:0/1349889615 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff154021020 con 0x7ff164146c20 2023-08-27T11:30:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.827+0000 7ff168c5f700 1 -- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff14406a500 msgr2=0x7ff14406c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.827+0000 7ff168c5f700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff14406a500 0x7ff14406c9c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff1580090b0 tx=0x7ff158008030 comp rx=0 tx=0).stop 2023-08-27T11:30:00.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.827+0000 7ff168c5f700 1 -- 172.21.15.112:0/1349889615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff164146c20 msgr2=0x7ff1641387a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:00.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.827+0000 7ff168c5f700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff164146c20 0x7ff1641387a0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7ff15400e9b0 tx=0x7ff15400ed70 comp rx=0 tx=0).stop 2023-08-27T11:30:00.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.827+0000 7ff168c5f700 1 -- 172.21.15.112:0/1349889615 shutdown_connections 2023-08-27T11:30:00.829 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.827+0000 7ff168c5f700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff14406a500 0x7ff14406c9c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.827+0000 7ff168c5f700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff16415e6e0 0x7ff164070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.827+0000 7ff168c5f700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1640b3440 0x7ff16413af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.827+0000 7ff168c5f700 1 --2- 172.21.15.112:0/1349889615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff164146c20 0x7ff1641387a0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:00.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.827+0000 7ff168c5f700 1 -- 172.21.15.112:0/1349889615 >> 172.21.15.112:0/1349889615 conn(0x7ff1640b1e90 msgr2=0x7ff16406ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:00.830 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.827+0000 7ff168c5f700 1 -- 172.21.15.112:0/1349889615 shutdown_connections 2023-08-27T11:30:00.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:00.827+0000 7ff168c5f700 1 -- 172.21.15.112:0/1349889615 wait complete. 2023-08-27T11:30:00.840 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:30:00.841 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:30:00.882 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:30:00.932 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:30:00.932 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:30:00.933 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:30:00.952 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:30:00.953 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:30:00.953 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:30:00.973 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:30:00.973 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-08-27T11:30:00.974 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:30:01.009 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:30:01.010 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-08-27T11:30:01.010 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:30:01.031 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16881 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:30:01.032 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:30:01.032 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:30:01.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.168+0000 7fe3de18c700 1 Processor -- start 2023-08-27T11:30:01.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.168+0000 7fe3de18c700 1 -- start start 2023-08-27T11:30:01.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.169+0000 7fe3de18c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d8159070 0x7fe3d806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.173 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.169+0000 7fe3de18c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d815f5d0 con 0x7fe3d8158c90 2023-08-27T11:30:01.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.169+0000 7fe3de18c700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d80ca990 con 0x7fe3d80b1270 2023-08-27T11:30:01.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.169+0000 7fe3de18c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d80b1a10 con 0x7fe3d8159070 2023-08-27T11:30:01.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.169+0000 7fe3d77fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe3d8158c90 0x7fe3d806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33492/0 (socket says 172.21.15.112:33492) 2023-08-27T11:30:01.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.169+0000 7fe3d77fe700 1 -- 172.21.15.112:0/2612006979 learned_addr learned my addr 172.21.15.112:0/2612006979 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:01.174 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d6ffd700 1 --2- 172.21.15.112:0/2612006979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d8159070 0x7fe3d806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:01.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325716700 0 0) 0x7fe3d815f5d0 con 0x7fe3d8158c90 2023-08-27T11:30:01.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3c00035b0 con 0x7fe3d8158c90 2023-08-27T11:30:01.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2304200870 0 0) 0x7fe3c00035b0 con 0x7fe3d8158c90 2023-08-27T11:30:01.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d8159070 msgr2=0x7fe3d806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:01.175 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 --2- 172.21.15.112:0/2612006979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d8159070 0x7fe3d806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 >> v1:172.21.15.138:6789/0 conn(0x7fe3d80b1270 legacy=0x7fe3d815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:01.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d80b1730 con 0x7fe3d8158c90 2023-08-27T11:30:01.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7fe3cc001670 con 0x7fe3d8158c90 2023-08-27T11:30:01.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 >> v1:172.21.15.112:6789/0 conn(0x7fe3d8158c90 legacy=0x7fe3d806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:01.176 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 --2- 172.21.15.112:0/2612006979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d8159070 0x7fe3c0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 --2- 172.21.15.112:0/2612006979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c0003f70 0x7fe3c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 --2- 172.21.15.112:0/2612006979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c0004d90 0x7fe3c0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d80b1a10 con 0x7fe3c0004d90 2023-08-27T11:30:01.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c000bb20 con 0x7fe3c0003f70 2023-08-27T11:30:01.177 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c000bce0 con 0x7fe3d8159070 2023-08-27T11:30:01.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d7fff700 1 --2- 172.21.15.112:0/2612006979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c0004d90 0x7fe3c0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:01.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d6ffd700 1 --2- 172.21.15.112:0/2612006979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c0003f70 0x7fe3c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:01.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d77fe700 1 --2- 172.21.15.112:0/2612006979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d8159070 0x7fe3c0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:01.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d7fff700 1 -- 172.21.15.112:0/2612006979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d8159070 msgr2=0x7fe3c0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:01.178 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d7fff700 1 --2- 172.21.15.112:0/2612006979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d8159070 0x7fe3c0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d7fff700 1 -- 172.21.15.112:0/2612006979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c0003f70 msgr2=0x7fe3c0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:01.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d7fff700 1 --2- 172.21.15.112:0/2612006979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c0003f70 0x7fe3c0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d7fff700 1 -- 172.21.15.112:0/2612006979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c00035b0 con 0x7fe3c0004d90 2023-08-27T11:30:01.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.170+0000 7fe3d7fff700 1 --2- 172.21.15.112:0/2612006979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c0004d90 0x7fe3c0005210 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80b0c70 tx=0x7fe3c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aebc5412e18492c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:01.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.171+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0001e70 con 0x7fe3c0004d90 2023-08-27T11:30:01.179 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.171+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d000cdf0 con 0x7fe3c0004d90 2023-08-27T11:30:01.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.171+0000 7fe3d67fc700 1 -- 172.21.15.112:0/2612006979 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0012660 con 0x7fe3c0004d90 2023-08-27T11:30:01.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.171+0000 7fe3de18c700 1 -- 172.21.15.112:0/2612006979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c0004d90 msgr2=0x7fe3c0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:01.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.171+0000 7fe3de18c700 1 --2- 172.21.15.112:0/2612006979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3c0004d90 0x7fe3c0005210 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80b0c70 tx=0x7fe3c0005880 comp rx=0 tx=0).stop 2023-08-27T11:30:01.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.171+0000 7fe3d7fff700 1 -- 172.21.15.112:0/2612006979 reap_dead start 2023-08-27T11:30:01.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.171+0000 7fe3de18c700 1 -- 172.21.15.112:0/2612006979 shutdown_connections 2023-08-27T11:30:01.180 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.171+0000 7fe3de18c700 1 -- 172.21.15.112:0/2612006979 >> 172.21.15.112:0/2612006979 conn(0x7fe3d80d0000 msgr2=0x7fe3d80d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:01.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.171+0000 7fe3de18c700 1 -- 172.21.15.112:0/2612006979 shutdown_connections 2023-08-27T11:30:01.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.171+0000 7fe3de18c700 1 -- 172.21.15.112:0/2612006979 wait complete. 2023-08-27T11:30:01.181 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.172+0000 7fe3de18c700 1 Processor -- start 2023-08-27T11:30:01.183 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.173+0000 7fe3de18c700 1 -- start start 2023-08-27T11:30:01.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.173+0000 7fe3de18c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3d8157090 0x7fe3d80c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.173+0000 7fe3de18c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d8157470 0x7fe3d80c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.173+0000 7fe3de18c700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d80cdee0 0x7fe3d80ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.173+0000 7fe3de18c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d80cec30 con 0x7fe3d8157090 2023-08-27T11:30:01.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.173+0000 7fe3de18c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d80cef10 con 0x7fe3d8157470 2023-08-27T11:30:01.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.173+0000 7fe3de18c700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d80cf1f0 con 0x7fe3d80cdee0 2023-08-27T11:30:01.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.173+0000 7fe3d77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3d8157090 0x7fe3d80c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:01.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.173+0000 7fe3d77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3d8157090 0x7fe3d80c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39118/0 (socket says 172.21.15.112:39118) 2023-08-27T11:30:01.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.173+0000 7fe3d77fe700 1 -- 172.21.15.112:0/3455066172 learned_addr learned my addr 172.21.15.112:0/3455066172 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:01.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.174+0000 7fe3d77fe700 1 -- 172.21.15.112:0/3455066172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d80cdee0 msgr2=0x7fe3d80ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:01.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.174+0000 7fe3d77fe700 1 --2- 172.21.15.112:0/3455066172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d80cdee0 0x7fe3d80ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.174+0000 7fe3d77fe700 1 -- 172.21.15.112:0/3455066172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d8157470 msgr2=0x7fe3d80c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:01.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.174+0000 7fe3d77fe700 1 --2- 172.21.15.112:0/3455066172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d8157470 0x7fe3d80c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.174+0000 7fe3d77fe700 1 -- 172.21.15.112:0/3455066172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d80cf4d0 con 0x7fe3d8157090 2023-08-27T11:30:01.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.174+0000 7fe3d77fe700 1 --2- 172.21.15.112:0/3455066172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3d8157090 0x7fe3d80c7ae0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0072c0 tx=0x7fe3cc007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:01.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.174+0000 7fe3bffff700 1 -- 172.21.15.112:0/3455066172 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc00f850 con 0x7fe3d8157090 2023-08-27T11:30:01.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.174+0000 7fe3bffff700 1 -- 172.21.15.112:0/3455066172 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc00f9b0 con 0x7fe3d8157090 2023-08-27T11:30:01.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.174+0000 7fe3de18c700 1 -- 172.21.15.112:0/3455066172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d815ba80 con 0x7fe3d8157090 2023-08-27T11:30:01.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.174+0000 7fe3bffff700 1 -- 172.21.15.112:0/3455066172 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc00fd60 con 0x7fe3d8157090 2023-08-27T11:30:01.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.174+0000 7fe3de18c700 1 -- 172.21.15.112:0/3455066172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d815bfa0 con 0x7fe3d8157090 2023-08-27T11:30:01.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.175+0000 7fe3de18c700 1 -- 172.21.15.112:0/3455066172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d815bcc0 con 0x7fe3d8157090 2023-08-27T11:30:01.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.175+0000 7fe3bffff700 1 -- 172.21.15.112:0/3455066172 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe3cc01d460 con 0x7fe3d8157090 2023-08-27T11:30:01.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.176+0000 7fe3bffff700 1 --2- 172.21.15.112:0/3455066172 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3c406a500 0x7fe3c406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.176+0000 7fe3bffff700 1 -- 172.21.15.112:0/3455066172 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe3cc0888b0 con 0x7fe3d8157090 2023-08-27T11:30:01.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.176+0000 7fe3d6ffd700 1 --2- 172.21.15.112:0/3455066172 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3c406a500 0x7fe3c406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:01.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.177+0000 7fe3d6ffd700 1 --2- 172.21.15.112:0/3455066172 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3c406a500 0x7fe3c406c9c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80cfaf0 tx=0x7fe3c8009670 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:01.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.180+0000 7fe3bffff700 1 -- 172.21.15.112:0/3455066172 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe3d815bcc0 con 0x7fe3d8157090 2023-08-27T11:30:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.412+0000 7fe3de18c700 1 -- 172.21.15.112:0/3455066172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3d815bcc0 con 0x7fe3d8157090 2023-08-27T11:30:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.412+0000 7fe3bffff700 1 -- 172.21.15.112:0/3455066172 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+3713 (secure 0 0 0) 0x7fe3cc053dd0 con 0x7fe3d8157090 2023-08-27T11:30:01.414 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:30:01.414 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":118,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/685371648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":685371648},{"type":"v1","addr":"172.21.15.138:6836","nonce":685371648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":7214,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3585152138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3585152138},{"type":"v1","addr":"172.21.15.138:6837","nonce":3585152138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":8203,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2190275630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2190275630},{"type":"v1","addr":"172.21.15.112:6835","nonce":2190275630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":9849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1666895907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1666895907},{"type":"v1","addr":"172.21.15.112:6833","nonce":1666895907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[]} 2023-08-27T11:30:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.414+0000 7fe3de18c700 1 -- 172.21.15.112:0/3455066172 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3c406a500 msgr2=0x7fe3c406c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.414+0000 7fe3de18c700 1 --2- 172.21.15.112:0/3455066172 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3c406a500 0x7fe3c406c9c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80cfaf0 tx=0x7fe3c8009670 comp rx=0 tx=0).stop 2023-08-27T11:30:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.414+0000 7fe3de18c700 1 -- 172.21.15.112:0/3455066172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3d8157090 msgr2=0x7fe3d80c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:01.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.414+0000 7fe3de18c700 1 --2- 172.21.15.112:0/3455066172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3d8157090 0x7fe3d80c7ae0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0072c0 tx=0x7fe3cc007680 comp rx=0 tx=0).stop 2023-08-27T11:30:01.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.415+0000 7fe3de18c700 1 -- 172.21.15.112:0/3455066172 shutdown_connections 2023-08-27T11:30:01.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.415+0000 7fe3de18c700 1 --2- 172.21.15.112:0/3455066172 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe3c406a500 0x7fe3c406c9c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.415+0000 7fe3de18c700 1 --2- 172.21.15.112:0/3455066172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d80cdee0 0x7fe3d80ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.415+0000 7fe3de18c700 1 --2- 172.21.15.112:0/3455066172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3d8157090 0x7fe3d80c7ae0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.415+0000 7fe3de18c700 1 --2- 172.21.15.112:0/3455066172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d8157470 0x7fe3d80c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.415+0000 7fe3de18c700 1 -- 172.21.15.112:0/3455066172 >> 172.21.15.112:0/3455066172 conn(0x7fe3d80d0000 msgr2=0x7fe3d806ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:01.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.415+0000 7fe3de18c700 1 -- 172.21.15.112:0/3455066172 shutdown_connections 2023-08-27T11:30:01.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.415+0000 7fe3de18c700 1 -- 172.21.15.112:0/3455066172 wait complete. 2023-08-27T11:30:01.419 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 118 2023-08-27T11:30:01.428 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:30:01.428 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:30:01.428 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:30:01.429 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:30:01.429 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:30:01.429 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:30:01.430 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:30:01.430 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:30:01.430 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:30:01.430 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:30:01.431 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:30:01.431 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:30:01.431 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:30:01.431 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:30:01.432 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:30:01.432 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:30:01.437 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:30:01.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:30:01.439 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:30:01.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:30:01.441 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:30:01.442 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:30:01.443 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:30:01.443 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:30:01.481 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:30:01.481 DEBUG:teuthology.parallel:result is None 2023-08-27T11:30:01.482 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:30:01.483 DEBUG:teuthology.parallel:result is None 2023-08-27T11:30:01.485 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:30:01.485 DEBUG:teuthology.parallel:result is None 2023-08-27T11:30:01.486 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:30:01.486 DEBUG:teuthology.parallel:result is None 2023-08-27T11:30:01.487 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:30:01.560 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2023-08-27T11:30:01.561 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2023-08-27T11:30:01.574 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2023-08-27T11:30:01.574 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2023-08-27T11:30:01.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.614+0000 7f53456bf700 1 Processor -- start 2023-08-27T11:30:01.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.614+0000 7f53456bf700 1 -- start start 2023-08-27T11:30:01.618 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f53456bf700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5340160a00 0x7f53400b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f53456bf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f534006d510 0x7f534006d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f53456bf700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534006dea0 con 0x7f5340146990 2023-08-27T11:30:01.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f53456bf700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534006f930 con 0x7f5340160a00 2023-08-27T11:30:01.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f53456bf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534006fc10 con 0x7f534006d510 2023-08-27T11:30:01.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f533e7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5340146990 0x7f534015dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33530/0 (socket says 172.21.15.112:33530) 2023-08-27T11:30:01.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f533e7fc700 1 -- 172.21.15.112:0/3661735594 learned_addr learned my addr 172.21.15.112:0/3661735594 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:01.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f533f7fe700 1 --2- 172.21.15.112:0/3661735594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f534006d510 0x7f534006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:01.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f533effd700 1 --2- 172.21.15.112:0/3661735594 >> v2:172.21.15.138:3300/0 conn(0x7f5340160a00 0x7f53400b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:01.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f533dffb700 1 -- 172.21.15.112:0/3661735594 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946885109 0 0) 0x7f5334001110 con 0x7f5340146990 2023-08-27T11:30:01.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f533dffb700 1 -- 172.21.15.112:0/3661735594 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53280035b0 con 0x7f5340146990 2023-08-27T11:30:01.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f533f7fe700 1 -- 172.21.15.112:0/3661735594 >> v2:172.21.15.138:3300/0 conn(0x7f5340160a00 msgr2=0x7f53400b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:01.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f533f7fe700 1 --2- 172.21.15.112:0/3661735594 >> v2:172.21.15.138:3300/0 conn(0x7f5340160a00 0x7f53400b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f533f7fe700 1 -- 172.21.15.112:0/3661735594 >> v1:172.21.15.112:6789/0 conn(0x7f5340146990 legacy=0x7f534015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:01.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f533f7fe700 1 -- 172.21.15.112:0/3661735594 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534006fef0 con 0x7f534006d510 2023-08-27T11:30:01.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.615+0000 7f533effd700 1 --2- 172.21.15.112:0/3661735594 >> v2:172.21.15.138:3300/0 conn(0x7f5340160a00 0x7f53400b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:30:01.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.616+0000 7f533f7fe700 1 --2- 172.21.15.112:0/3661735594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f534006d510 0x7f534006d960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5330009840 tx=0x7f5330009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d44302b9a7f8fd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:01.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.616+0000 7f533dffb700 1 -- 172.21.15.112:0/3661735594 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330011830 con 0x7f534006d510 2023-08-27T11:30:01.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.616+0000 7f533dffb700 1 -- 172.21.15.112:0/3661735594 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5330011990 con 0x7f534006d510 2023-08-27T11:30:01.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.616+0000 7f533dffb700 1 -- 172.21.15.112:0/3661735594 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330011d00 con 0x7f534006d510 2023-08-27T11:30:01.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.616+0000 7f53456bf700 1 -- 172.21.15.112:0/3661735594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f534006d510 msgr2=0x7f534006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:01.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.616+0000 7f53456bf700 1 --2- 172.21.15.112:0/3661735594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f534006d510 0x7f534006d960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5330009840 tx=0x7f5330009b50 comp rx=0 tx=0).stop 2023-08-27T11:30:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.616+0000 7f53456bf700 1 -- 172.21.15.112:0/3661735594 shutdown_connections 2023-08-27T11:30:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.616+0000 7f53456bf700 1 --2- 172.21.15.112:0/3661735594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f534006d510 0x7f534006d960 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.616+0000 7f53456bf700 1 --2- 172.21.15.112:0/3661735594 >> v2:172.21.15.138:3300/0 conn(0x7f5340160a00 0x7f53400b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.616+0000 7f53456bf700 1 -- 172.21.15.112:0/3661735594 >> 172.21.15.112:0/3661735594 conn(0x7f53400cfe50 msgr2=0x7f53400d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.616+0000 7f53456bf700 1 -- 172.21.15.112:0/3661735594 shutdown_connections 2023-08-27T11:30:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.616+0000 7f53456bf700 1 -- 172.21.15.112:0/3661735594 wait complete. 2023-08-27T11:30:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.618+0000 7f53456bf700 1 Processor -- start 2023-08-27T11:30:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.618+0000 7f53456bf700 1 -- start start 2023-08-27T11:30:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.618+0000 7f53456bf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534006d510 0x7f534015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.618+0000 7f53456bf700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5340146990 0x7f53400ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.618+0000 7f53456bf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5340160a00 0x7f53400b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.618+0000 7f53456bf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53401395e0 con 0x7f534006d510 2023-08-27T11:30:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.618+0000 7f53456bf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53400b9720 con 0x7f5340160a00 2023-08-27T11:30:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.618+0000 7f53456bf700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5340138ff0 con 0x7f5340146990 2023-08-27T11:30:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.618+0000 7f533effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534006d510 0x7f534015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.618+0000 7f533effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534006d510 0x7f534015cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39154/0 (socket says 172.21.15.112:39154) 2023-08-27T11:30:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.618+0000 7f533effd700 1 -- 172.21.15.112:0/2641678718 learned_addr learned my addr 172.21.15.112:0/2641678718 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.619+0000 7f533e7fc700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5340146990 0x7f53400ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.619+0000 7f533f7fe700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5340160a00 0x7f53400b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.619+0000 7f533effd700 1 -- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5340146990 msgr2=0x7f53400ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.619+0000 7f533effd700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5340146990 0x7f53400ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.619+0000 7f533effd700 1 -- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5340160a00 msgr2=0x7f53400b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.619+0000 7f533effd700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5340160a00 0x7f53400b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.619+0000 7f533effd700 1 -- 172.21.15.112:0/2641678718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5338009a70 con 0x7f534006d510 2023-08-27T11:30:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.619+0000 7f533effd700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534006d510 0x7f534015cf30 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f5338009ec0 tx=0x7f533800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.619+0000 7f53277fe700 1 -- 172.21.15.112:0/2641678718 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533800ce80 con 0x7f534006d510 2023-08-27T11:30:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.619+0000 7f53456bf700 1 -- 172.21.15.112:0/2641678718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53300094a0 con 0x7f534006d510 2023-08-27T11:30:01.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.619+0000 7f53277fe700 1 -- 172.21.15.112:0/2641678718 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533800f040 con 0x7f534006d510 2023-08-27T11:30:01.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.619+0000 7f53277fe700 1 -- 172.21.15.112:0/2641678718 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53380146a0 con 0x7f534006d510 2023-08-27T11:30:01.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.619+0000 7f53456bf700 1 -- 172.21.15.112:0/2641678718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53400d2090 con 0x7f534006d510 2023-08-27T11:30:01.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.620+0000 7f53456bf700 1 -- 172.21.15.112:0/2641678718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5340067a20 con 0x7f534006d510 2023-08-27T11:30:01.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.621+0000 7f53277fe700 1 -- 172.21.15.112:0/2641678718 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f5338014800 con 0x7f534006d510 2023-08-27T11:30:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.621+0000 7f53277fe700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f532c06a4d0 0x7f532c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.621+0000 7f53277fe700 1 -- 172.21.15.112:0/2641678718 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f53380898d0 con 0x7f534006d510 2023-08-27T11:30:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.621+0000 7f533e7fc700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f532c06a4d0 0x7f532c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.622+0000 7f533e7fc700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f532c06a4d0 0x7f532c06c990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f53400d29d0 tx=0x7f5334009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.625+0000 7f53277fe700 1 -- 172.21.15.112:0/2641678718 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5338054b50 con 0x7f534006d510 2023-08-27T11:30:01.828 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:01.827+0000 7f53456bf700 1 -- 172.21.15.112:0/2641678718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f534006e740 con 0x7f534006d510 2023-08-27T11:30:02.483 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.482+0000 7f53277fe700 1 -- 172.21.15.112:0/2641678718 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v178) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5338058170 con 0x7f534006d510 2023-08-27T11:30:02.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.484+0000 7f53456bf700 1 -- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f532c06a4d0 msgr2=0x7f532c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:02.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.484+0000 7f53456bf700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f532c06a4d0 0x7f532c06c990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f53400d29d0 tx=0x7f5334009040 comp rx=0 tx=0).stop 2023-08-27T11:30:02.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.484+0000 7f53456bf700 1 -- 172.21.15.112:0/2641678718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534006d510 msgr2=0x7f534015cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:02.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.484+0000 7f53456bf700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534006d510 0x7f534015cf30 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f5338009ec0 tx=0x7f533800c3a0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.486 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.484+0000 7f53456bf700 1 -- 172.21.15.112:0/2641678718 shutdown_connections 2023-08-27T11:30:02.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.484+0000 7f53456bf700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f532c06a4d0 0x7f532c06c990 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.484+0000 7f53456bf700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5340160a00 0x7f53400b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.484+0000 7f53456bf700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534006d510 0x7f534015cf30 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.484+0000 7f53456bf700 1 --2- 172.21.15.112:0/2641678718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5340146990 0x7f53400ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.487 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.484+0000 7f53456bf700 1 -- 172.21.15.112:0/2641678718 >> 172.21.15.112:0/2641678718 conn(0x7f53400cfe50 msgr2=0x7f53400d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:02.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.484+0000 7f53456bf700 1 -- 172.21.15.112:0/2641678718 shutdown_connections 2023-08-27T11:30:02.488 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.484+0000 7f53456bf700 1 -- 172.21.15.112:0/2641678718 wait complete. 2023-08-27T11:30:02.488 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2023-08-27T11:30:02.500 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:30:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.622+0000 7fe2abaf6700 1 Processor -- start 2023-08-27T11:30:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.623+0000 7fe2abaf6700 1 -- start start 2023-08-27T11:30:02.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.623+0000 7fe2abaf6700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe2a406fea0 0x7fe2a40b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:02.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.623+0000 7fe2abaf6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe2a406e3a0 0x7fe2a40af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:02.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.623+0000 7fe2abaf6700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a415c190 con 0x7fe2a4158e20 2023-08-27T11:30:02.626 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.623+0000 7fe2abaf6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a40b1730 con 0x7fe2a406e3a0 2023-08-27T11:30:02.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.623+0000 7fe2abaf6700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a4159200 con 0x7fe2a406fea0 2023-08-27T11:30:02.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.623+0000 7fe2aa093700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe2a406e3a0 0x7fe2a40af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:02.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.623+0000 7fe2a9892700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe2a4158e20 0x7fe2a406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33546/0 (socket says 172.21.15.112:33546) 2023-08-27T11:30:02.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.623+0000 7fe2a9892700 1 -- 172.21.15.112:0/655069339 learned_addr learned my addr 172.21.15.112:0/655069339 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:02.627 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.624+0000 7fe2aa093700 1 -- 172.21.15.112:0/655069339 >> v2:172.21.15.138:3300/0 conn(0x7fe2a406fea0 msgr2=0x7fe2a40b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:02.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.624+0000 7fe2a9091700 1 --2- 172.21.15.112:0/655069339 >> v2:172.21.15.138:3300/0 conn(0x7fe2a406fea0 0x7fe2a40b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:02.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.624+0000 7fe2a8890700 1 -- 172.21.15.112:0/655069339 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466801504 0 0) 0x7fe2a415c190 con 0x7fe2a4158e20 2023-08-27T11:30:02.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.624+0000 7fe2aa093700 1 --2- 172.21.15.112:0/655069339 >> v2:172.21.15.138:3300/0 conn(0x7fe2a406fea0 0x7fe2a40b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.624+0000 7fe2aa093700 1 -- 172.21.15.112:0/655069339 >> v1:172.21.15.112:6789/0 conn(0x7fe2a4158e20 legacy=0x7fe2a406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:02.628 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.624+0000 7fe2aa093700 1 -- 172.21.15.112:0/655069339 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a40b1910 con 0x7fe2a406e3a0 2023-08-27T11:30:02.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.624+0000 7fe2aa093700 1 --2- 172.21.15.112:0/655069339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2a406e3a0 0x7fe2a40af9c0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40cc9b0 tx=0x7fe29c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a80154493ea8ce9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:02.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.624+0000 7fe2a8890700 1 -- 172.21.15.112:0/655069339 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c00e070 con 0x7fe2a406e3a0 2023-08-27T11:30:02.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.624+0000 7fe2a8890700 1 -- 172.21.15.112:0/655069339 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c00bbd0 con 0x7fe2a406e3a0 2023-08-27T11:30:02.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.624+0000 7fe2a8890700 1 -- 172.21.15.112:0/655069339 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c012430 con 0x7fe2a406e3a0 2023-08-27T11:30:02.629 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.625+0000 7fe2abaf6700 1 -- 172.21.15.112:0/655069339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2a406e3a0 msgr2=0x7fe2a40af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:02.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.625+0000 7fe2abaf6700 1 --2- 172.21.15.112:0/655069339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2a406e3a0 0x7fe2a40af9c0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40cc9b0 tx=0x7fe29c00b330 comp rx=0 tx=0).stop 2023-08-27T11:30:02.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.625+0000 7fe2abaf6700 1 -- 172.21.15.112:0/655069339 shutdown_connections 2023-08-27T11:30:02.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.625+0000 7fe2abaf6700 1 --2- 172.21.15.112:0/655069339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2a406e3a0 0x7fe2a40af9c0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.625+0000 7fe2abaf6700 1 --2- 172.21.15.112:0/655069339 >> v2:172.21.15.138:3300/0 conn(0x7fe2a406fea0 0x7fe2a40b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.630 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.625+0000 7fe2abaf6700 1 -- 172.21.15.112:0/655069339 >> 172.21.15.112:0/655069339 conn(0x7fe2a40d0000 msgr2=0x7fe2a40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:02.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.625+0000 7fe2abaf6700 1 -- 172.21.15.112:0/655069339 shutdown_connections 2023-08-27T11:30:02.631 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.625+0000 7fe2abaf6700 1 -- 172.21.15.112:0/655069339 wait complete. 2023-08-27T11:30:02.633 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.626+0000 7fe2abaf6700 1 Processor -- start 2023-08-27T11:30:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.626+0000 7fe2abaf6700 1 -- start start 2023-08-27T11:30:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2abaf6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2a406e3a0 0x7fe2a4149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2abaf6700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2a406fea0 0x7fe2a40c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2abaf6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2a4158e20 0x7fe2a4155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2abaf6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a4156570 con 0x7fe2a4158e20 2023-08-27T11:30:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2abaf6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40c78c0 con 0x7fe2a406e3a0 2023-08-27T11:30:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2abaf6700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40c9fb0 con 0x7fe2a406fea0 2023-08-27T11:30:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2aa093700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2a4158e20 0x7fe2a4155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2aa093700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2a4158e20 0x7fe2a4155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39174/0 (socket says 172.21.15.112:39174) 2023-08-27T11:30:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2aa093700 1 -- 172.21.15.112:0/3052756489 learned_addr learned my addr 172.21.15.112:0/3052756489 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:02.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2a9892700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2a406e3a0 0x7fe2a4149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:02.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2a9091700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2a406fea0 0x7fe2a40c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:02.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2aa093700 1 -- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2a406fea0 msgr2=0x7fe2a40c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:02.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2aa093700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2a406fea0 0x7fe2a40c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.636 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2aa093700 1 -- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2a406e3a0 msgr2=0x7fe2a4149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:02.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2aa093700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2a406e3a0 0x7fe2a4149f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2aa093700 1 -- 172.21.15.112:0/3052756489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c009a70 con 0x7fe2a4158e20 2023-08-27T11:30:02.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2aa093700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2a4158e20 0x7fe2a4155a40 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40d0f90 tx=0x7fe29c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:02.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe29a7fc700 1 -- 172.21.15.112:0/3052756489 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c012430 con 0x7fe2a4158e20 2023-08-27T11:30:02.637 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe29a7fc700 1 -- 172.21.15.112:0/3052756489 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c020070 con 0x7fe2a4158e20 2023-08-27T11:30:02.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2abaf6700 1 -- 172.21.15.112:0/3052756489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a4156290 con 0x7fe2a4158e20 2023-08-27T11:30:02.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe29a7fc700 1 -- 172.21.15.112:0/3052756489 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c004280 con 0x7fe2a4158e20 2023-08-27T11:30:02.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.627+0000 7fe2abaf6700 1 -- 172.21.15.112:0/3052756489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a40c7550 con 0x7fe2a4158e20 2023-08-27T11:30:02.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.628+0000 7fe2abaf6700 1 -- 172.21.15.112:0/3052756489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a40c70d0 con 0x7fe2a4158e20 2023-08-27T11:30:02.638 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.629+0000 7fe29a7fc700 1 -- 172.21.15.112:0/3052756489 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe29c0043e0 con 0x7fe2a4158e20 2023-08-27T11:30:02.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.629+0000 7fe29a7fc700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe28806a4d0 0x7fe28806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:02.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.629+0000 7fe29a7fc700 1 -- 172.21.15.112:0/3052756489 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe29c022070 con 0x7fe2a4158e20 2023-08-27T11:30:02.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.630+0000 7fe2a9892700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe28806a4d0 0x7fe28806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:02.639 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.630+0000 7fe2a9892700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe28806a4d0 0x7fe28806c990 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe294007fd0 tx=0x7fe294007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.633+0000 7fe29a7fc700 1 -- 172.21.15.112:0/3052756489 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe2a40c70d0 con 0x7fe2a4158e20 2023-08-27T11:30:02.901 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.900+0000 7fe2abaf6700 1 -- 172.21.15.112:0/3052756489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe2a40c70d0 con 0x7fe2a4158e20 2023-08-27T11:30:02.902 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.901+0000 7fe29a7fc700 1 -- 172.21.15.112:0/3052756489 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe2a40c70d0 con 0x7fe2a4158e20 2023-08-27T11:30:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:30:02.903 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-08-27T11:30:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2023-08-27T11:30:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2023-08-27T11:30:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk2bnkLBAA+vxcTeJzucKk9iSF/Zw3Lg==", 2023-08-27T11:30:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:30:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:30:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:30:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2023-08-27T11:30:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtk3zD5OBAASVnwG6yn/YQzfYyeMakFpA==", 2023-08-27T11:30:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:30:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:30:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:30:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2023-08-27T11:30:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtki7wJNBAAsIGgDLM82sWNcKnCDlSBMw==", 2023-08-27T11:30:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:30:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:30:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:30:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2023-08-27T11:30:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkCsW/BBAAFTPiTOpBo1L+rLo9px+AjQ==", 2023-08-27T11:30:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:30:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:30:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:30:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2023-08-27T11:30:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB7MOtkR/cSHhAAR8e8VRd+e0ESJkqQDYWbEw==", 2023-08-27T11:30:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:30:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:30:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2023-08-27T11:30:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB+MOtksBF3HRAA1OIXiIkhgPDYpQyYPaWOXg==", 2023-08-27T11:30:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:30:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:30:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2023-08-27T11:30:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCBMOtkWsrdHBAAkfWbiE6wAFzpt9+BxRh8fw==", 2023-08-27T11:30:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:30:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:30:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2023-08-27T11:30:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCEMOtkIkiKHRAARDoh/FH9ZdqUlZZ29Mc/Dw==", 2023-08-27T11:30:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:30:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:30:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2023-08-27T11:30:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCKMOtkPgDzGxAA+U4kbwD0d4lS8aNPG2CgHw==", 2023-08-27T11:30:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:30:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:30:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2023-08-27T11:30:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCNMOtkKomSGxAAGAQUcIH2H/uyZIU8vlq1pw==", 2023-08-27T11:30:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:30:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:30:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2023-08-27T11:30:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCQMOtkHI2MGhAA7sgbi8JXO8P9wnVvMy3PqA==", 2023-08-27T11:30:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:30:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:30:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2023-08-27T11:30:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQCTMOtkVHpiGhAA4C4YJMWbQ0mmGgB4i8G/AA==", 2023-08-27T11:30:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2023-08-27T11:30:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2023-08-27T11:30:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2023-08-27T11:30:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkeR0zCRAADIxFUGskWc7QDA8CS50eIA==", 2023-08-27T11:30:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:30:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2023-08-27T11:30:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:30:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2023-08-27T11:30:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtk66DbDxAAWZ5StELux0lriJ2gEN9icg==", 2023-08-27T11:30:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:30:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:30:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:30:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:30:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2023-08-27T11:30:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkViKJDBAAF2ofwO+78er9OyFfn2bO+g==", 2023-08-27T11:30:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:30:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:30:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:30:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:30:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2023-08-27T11:30:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB4MOtkwt+QFRAAowWE69VqeRhQz0896rxx9Q==", 2023-08-27T11:30:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2023-08-27T11:30:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2023-08-27T11:30:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2023-08-27T11:30:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2023-08-27T11:30:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2023-08-27T11:30:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkjDA+DxAAjmOn82AC2OMpKlAVCD6hlg==", 2023-08-27T11:30:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:30:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2023-08-27T11:30:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2023-08-27T11:30:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2023-08-27T11:30:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkXguHKRAALHwzk8diVXqsMvTKr+AzLQ==", 2023-08-27T11:30:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:30:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:30:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2023-08-27T11:30:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQB3MOtkrRjBJRAAq3ftmIh81LKadpliVUiJ6g==", 2023-08-27T11:30:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2023-08-27T11:30:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2023-08-27T11:30:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2023-08-27T11:30:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2023-08-27T11:30:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:02.941 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-08-27T11:30:02.941 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-08-27T11:30:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.903+0000 7fe2abaf6700 1 -- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe28806a4d0 msgr2=0x7fe28806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.903+0000 7fe2abaf6700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe28806a4d0 0x7fe28806c990 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe294007fd0 tx=0x7fe294007de0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.903+0000 7fe2abaf6700 1 -- 172.21.15.112:0/3052756489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2a4158e20 msgr2=0x7fe2a4155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.903+0000 7fe2abaf6700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2a4158e20 0x7fe2a4155a40 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40d0f90 tx=0x7fe29c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.903+0000 7fe2abaf6700 1 -- 172.21.15.112:0/3052756489 shutdown_connections 2023-08-27T11:30:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.903+0000 7fe2abaf6700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe28806a4d0 0x7fe28806c990 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.903+0000 7fe2abaf6700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2a4158e20 0x7fe2a4155a40 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.903+0000 7fe2abaf6700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2a406e3a0 0x7fe2a4149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.903+0000 7fe2abaf6700 1 --2- 172.21.15.112:0/3052756489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2a406fea0 0x7fe2a40c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.903+0000 7fe2abaf6700 1 -- 172.21.15.112:0/3052756489 >> 172.21.15.112:0/3052756489 conn(0x7fe2a40d0000 msgr2=0x7fe2a40d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.903+0000 7fe2abaf6700 1 -- 172.21.15.112:0/3052756489 shutdown_connections 2023-08-27T11:30:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:02.903+0000 7fe2abaf6700 1 -- 172.21.15.112:0/3052756489 wait complete. 2023-08-27T11:30:02.945 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:30:03.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.064+0000 7fe01424e700 1 Processor -- start 2023-08-27T11:30:03.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.065+0000 7fe01424e700 1 -- start start 2023-08-27T11:30:03.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.065+0000 7fe01424e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe00c151060 0x7fe00c15e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.065+0000 7fe01424e700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe00c15e990 0x7fe00c06adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe01424e700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe00c06b710 con 0x7fe00c06b2f0 2023-08-27T11:30:03.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe01424e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00c0b02f0 con 0x7fe00c151060 2023-08-27T11:30:03.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe01424e700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00c0b05d0 con 0x7fe00c15e990 2023-08-27T11:30:03.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe011fea700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe00c151060 0x7fe00c15e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe011fea700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe00c151060 0x7fe00c15e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39178/0 (socket says 172.21.15.112:39178) 2023-08-27T11:30:03.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe011fea700 1 -- 172.21.15.112:0/835162365 learned_addr learned my addr 172.21.15.112:0/835162365 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:03.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe0117e9700 1 --2- 172.21.15.112:0/835162365 >> v2:172.21.15.138:3300/0 conn(0x7fe00c15e990 0x7fe00c06adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe011fea700 1 -- 172.21.15.112:0/835162365 >> v2:172.21.15.138:3300/0 conn(0x7fe00c15e990 msgr2=0x7fe00c06adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:03.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe011fea700 1 --2- 172.21.15.112:0/835162365 >> v2:172.21.15.138:3300/0 conn(0x7fe00c15e990 0x7fe00c06adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe011fea700 1 -- 172.21.15.112:0/835162365 >> v1:172.21.15.138:6789/0 conn(0x7fe00c06b2f0 legacy=0x7fe00c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:03.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe011fea700 1 -- 172.21.15.112:0/835162365 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00c0b08b0 con 0x7fe00c151060 2023-08-27T11:30:03.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe0117e9700 1 --2- 172.21.15.112:0/835162365 >> v2:172.21.15.138:3300/0 conn(0x7fe00c15e990 0x7fe00c06adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:03.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe011fea700 1 --2- 172.21.15.112:0/835162365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe00c151060 0x7fe00c15e450 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fe00c06ccb0 tx=0x7fdffc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83c180001aac0c3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:03.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe010fe8700 1 -- 172.21.15.112:0/835162365 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc004230 con 0x7fe00c151060 2023-08-27T11:30:03.071 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe010fe8700 1 -- 172.21.15.112:0/835162365 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc004390 con 0x7fe00c151060 2023-08-27T11:30:03.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.066+0000 7fe010fe8700 1 -- 172.21.15.112:0/835162365 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc0036a0 con 0x7fe00c151060 2023-08-27T11:30:03.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.067+0000 7fe01424e700 1 -- 172.21.15.112:0/835162365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe00c151060 msgr2=0x7fe00c15e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.067+0000 7fe01424e700 1 --2- 172.21.15.112:0/835162365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe00c151060 0x7fe00c15e450 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fe00c06ccb0 tx=0x7fdffc009bd0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.067+0000 7fe01424e700 1 -- 172.21.15.112:0/835162365 shutdown_connections 2023-08-27T11:30:03.072 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.067+0000 7fe01424e700 1 --2- 172.21.15.112:0/835162365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe00c151060 0x7fe00c15e450 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.067+0000 7fe01424e700 1 --2- 172.21.15.112:0/835162365 >> v2:172.21.15.138:3300/0 conn(0x7fe00c15e990 0x7fe00c06adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.067+0000 7fe01424e700 1 -- 172.21.15.112:0/835162365 >> 172.21.15.112:0/835162365 conn(0x7fe00c0d0000 msgr2=0x7fe00c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:03.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.067+0000 7fe01424e700 1 -- 172.21.15.112:0/835162365 shutdown_connections 2023-08-27T11:30:03.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.067+0000 7fe01424e700 1 -- 172.21.15.112:0/835162365 wait complete. 2023-08-27T11:30:03.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.068+0000 7fe01424e700 1 Processor -- start 2023-08-27T11:30:03.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.068+0000 7fe01424e700 1 -- start start 2023-08-27T11:30:03.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe01424e700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe00c06b2f0 0x7fe00c0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe01424e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe00c151060 0x7fe00c14e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe01424e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe00c15e990 0x7fe00c0a8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe01424e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c0c69b0 con 0x7fe00c15e990 2023-08-27T11:30:03.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe01424e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c0c53d0 con 0x7fe00c151060 2023-08-27T11:30:03.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe01424e700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c0c56b0 con 0x7fe00c06b2f0 2023-08-27T11:30:03.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe0127eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe00c15e990 0x7fe00c0a8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe0127eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe00c15e990 0x7fe00c0a8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39188/0 (socket says 172.21.15.112:39188) 2023-08-27T11:30:03.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe0127eb700 1 -- 172.21.15.112:0/767422631 learned_addr learned my addr 172.21.15.112:0/767422631 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:03.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe011fea700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe00c06b2f0 0x7fe00c0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe0117e9700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe00c151060 0x7fe00c14e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe0127eb700 1 -- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe00c06b2f0 msgr2=0x7fe00c0c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe0127eb700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe00c06b2f0 0x7fe00c0c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe0127eb700 1 -- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe00c151060 msgr2=0x7fe00c14e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe0127eb700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe00c151060 0x7fe00c14e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe0127eb700 1 -- 172.21.15.112:0/767422631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc009550 con 0x7fe00c15e990 2023-08-27T11:30:03.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.069+0000 7fe0127eb700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe00c15e990 0x7fe00c0a8e20 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fe00400b970 tx=0x7fe00400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:03.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.070+0000 7fe0027fc700 1 -- 172.21.15.112:0/767422631 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00400c9b0 con 0x7fe00c15e990 2023-08-27T11:30:03.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.070+0000 7fe0027fc700 1 -- 172.21.15.112:0/767422631 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00400cb10 con 0x7fe00c15e990 2023-08-27T11:30:03.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.070+0000 7fe0027fc700 1 -- 172.21.15.112:0/767422631 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004014440 con 0x7fe00c15e990 2023-08-27T11:30:03.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.070+0000 7fe01424e700 1 -- 172.21.15.112:0/767422631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00c0c5990 con 0x7fe00c15e990 2023-08-27T11:30:03.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.070+0000 7fe01424e700 1 -- 172.21.15.112:0/767422631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00c0c5f50 con 0x7fe00c15e990 2023-08-27T11:30:03.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.070+0000 7fe01424e700 1 -- 172.21.15.112:0/767422631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00c070eb0 con 0x7fe00c15e990 2023-08-27T11:30:03.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.071+0000 7fe0027fc700 1 -- 172.21.15.112:0/767422631 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe0040145a0 con 0x7fe00c15e990 2023-08-27T11:30:03.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.071+0000 7fe0027fc700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdff006a4d0 0x7fdff006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.071+0000 7fe0027fc700 1 -- 172.21.15.112:0/767422631 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe004088450 con 0x7fe00c15e990 2023-08-27T11:30:03.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.072+0000 7fe011fea700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdff006a4d0 0x7fdff006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.081 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.072+0000 7fe011fea700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdff006a4d0 0x7fdff006c990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009130 tx=0x7fdffc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:03.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.076+0000 7fe0027fc700 1 -- 172.21.15.112:0/767422631 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe004053a40 con 0x7fe00c15e990 2023-08-27T11:30:03.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.265+0000 7fe01424e700 1 -- 172.21.15.112:0/767422631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe00c067a20 con 0x7fe00c15e990 2023-08-27T11:30:03.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.266+0000 7fe0027fc700 1 -- 172.21.15.112:0/767422631 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+14615 (secure 0 0 0) 0x7fe004057060 con 0x7fe00c15e990 2023-08-27T11:30:03.267 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:30:03.268 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":180,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:30:02.491501+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:30:03.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.268+0000 7fe01424e700 1 -- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdff006a4d0 msgr2=0x7fdff006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.268+0000 7fe01424e700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdff006a4d0 0x7fdff006c990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009130 tx=0x7fdffc008030 comp rx=0 tx=0).stop 2023-08-27T11:30:03.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.268+0000 7fe01424e700 1 -- 172.21.15.112:0/767422631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe00c15e990 msgr2=0x7fe00c0a8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.270 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.268+0000 7fe01424e700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe00c15e990 0x7fe00c0a8e20 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fe00400b970 tx=0x7fe00400bd30 comp rx=0 tx=0).stop 2023-08-27T11:30:03.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.268+0000 7fe01424e700 1 -- 172.21.15.112:0/767422631 shutdown_connections 2023-08-27T11:30:03.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.268+0000 7fe01424e700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fdff006a4d0 0x7fdff006c990 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.268+0000 7fe01424e700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe00c15e990 0x7fe00c0a8e20 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.268+0000 7fe01424e700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe00c06b2f0 0x7fe00c0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.268+0000 7fe01424e700 1 --2- 172.21.15.112:0/767422631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe00c151060 0x7fe00c14e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.268+0000 7fe01424e700 1 -- 172.21.15.112:0/767422631 >> 172.21.15.112:0/767422631 conn(0x7fe00c0d0000 msgr2=0x7fe00c0b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:03.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.268+0000 7fe01424e700 1 -- 172.21.15.112:0/767422631 shutdown_connections 2023-08-27T11:30:03.272 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.268+0000 7fe01424e700 1 -- 172.21.15.112:0/767422631 wait complete. 2023-08-27T11:30:03.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:16:57.927938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:16:58.266800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:30:03.284 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:30:03.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.406+0000 7f9e79935700 1 Processor -- start 2023-08-27T11:30:03.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.406+0000 7f9e79935700 1 -- start start 2023-08-27T11:30:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e79935700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e7415d440 0x7f9e740cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e79935700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9e740cdbd0 0x7f9e7406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e79935700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e740cfae0 con 0x7f9e7406fcd0 2023-08-27T11:30:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e79935700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e74070160 con 0x7f9e7415d440 2023-08-27T11:30:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e79935700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e740cfd10 con 0x7f9e740cdbd0 2023-08-27T11:30:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e72ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e7415d440 0x7f9e740cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e737fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9e7406fcd0 0x7f9e74161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33578/0 (socket says 172.21.15.112:33578) 2023-08-27T11:30:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e737fe700 1 -- 172.21.15.112:0/349239617 learned_addr learned my addr 172.21.15.112:0/349239617 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e727fc700 1 --2- 172.21.15.112:0/349239617 >> v2:172.21.15.138:3300/0 conn(0x7f9e740cdbd0 0x7f9e7406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e72ffd700 1 -- 172.21.15.112:0/349239617 >> v2:172.21.15.138:3300/0 conn(0x7f9e740cdbd0 msgr2=0x7f9e7406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e72ffd700 1 --2- 172.21.15.112:0/349239617 >> v2:172.21.15.138:3300/0 conn(0x7f9e740cdbd0 0x7f9e7406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e72ffd700 1 -- 172.21.15.112:0/349239617 >> v1:172.21.15.112:6789/0 conn(0x7f9e7406fcd0 legacy=0x7f9e74161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e72ffd700 1 -- 172.21.15.112:0/349239617 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e740b1ed0 con 0x7f9e7415d440 2023-08-27T11:30:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e71ffb700 1 -- 172.21.15.112:0/349239617 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893527629 0 0) 0x7f9e740cfae0 con 0x7f9e7406fcd0 2023-08-27T11:30:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e72ffd700 1 --2- 172.21.15.112:0/349239617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7415d440 0x7f9e740cf5a0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f9e740d1100 tx=0x7f9e68009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1407fea5e19fa93 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e71ffb700 1 -- 172.21.15.112:0/349239617 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68011830 con 0x7f9e7415d440 2023-08-27T11:30:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e71ffb700 1 -- 172.21.15.112:0/349239617 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e68011990 con 0x7f9e7415d440 2023-08-27T11:30:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.407+0000 7f9e71ffb700 1 -- 172.21.15.112:0/349239617 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68011d20 con 0x7f9e7415d440 2023-08-27T11:30:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.408+0000 7f9e79935700 1 -- 172.21.15.112:0/349239617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7415d440 msgr2=0x7f9e740cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.408+0000 7f9e79935700 1 --2- 172.21.15.112:0/349239617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7415d440 0x7f9e740cf5a0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f9e740d1100 tx=0x7f9e68009b20 comp rx=0 tx=0).stop 2023-08-27T11:30:03.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.408+0000 7f9e79935700 1 -- 172.21.15.112:0/349239617 shutdown_connections 2023-08-27T11:30:03.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.408+0000 7f9e79935700 1 --2- 172.21.15.112:0/349239617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7415d440 0x7f9e740cf5a0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.408+0000 7f9e79935700 1 --2- 172.21.15.112:0/349239617 >> v2:172.21.15.138:3300/0 conn(0x7f9e740cdbd0 0x7f9e7406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.408+0000 7f9e79935700 1 -- 172.21.15.112:0/349239617 >> 172.21.15.112:0/349239617 conn(0x7f9e740cca80 msgr2=0x7f9e740cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:03.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.408+0000 7f9e79935700 1 -- 172.21.15.112:0/349239617 shutdown_connections 2023-08-27T11:30:03.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.408+0000 7f9e79935700 1 -- 172.21.15.112:0/349239617 wait complete. 2023-08-27T11:30:03.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.410+0000 7f9e79935700 1 Processor -- start 2023-08-27T11:30:03.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.410+0000 7f9e79935700 1 -- start start 2023-08-27T11:30:03.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e79935700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7406fcd0 0x7f9e741f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e79935700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e740cdbd0 0x7f9e741595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e79935700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7415d440 0x7f9e741f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e79935700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e740c9fe0 con 0x7f9e7415d440 2023-08-27T11:30:03.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e79935700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e740ae2a0 con 0x7f9e7406fcd0 2023-08-27T11:30:03.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e79935700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e741f6ef0 con 0x7f9e740cdbd0 2023-08-27T11:30:03.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e737fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7415d440 0x7f9e741f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e737fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7415d440 0x7f9e741f2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39202/0 (socket says 172.21.15.112:39202) 2023-08-27T11:30:03.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e737fe700 1 -- 172.21.15.112:0/3325439415 learned_addr learned my addr 172.21.15.112:0/3325439415 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:03.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e72ffd700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7406fcd0 0x7f9e741f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e727fc700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e740cdbd0 0x7f9e741595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e737fe700 1 -- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e740cdbd0 msgr2=0x7f9e741595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e737fe700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e740cdbd0 0x7f9e741595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e737fe700 1 -- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7406fcd0 msgr2=0x7f9e741f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e737fe700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7406fcd0 0x7f9e741f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e737fe700 1 -- 172.21.15.112:0/3325439415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e680094a0 con 0x7f9e7415d440 2023-08-27T11:30:03.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e727fc700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e740cdbd0 0x7f9e741595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:03.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e737fe700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7415d440 0x7f9e741f2a60 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00cca0 tx=0x7f9e6c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:03.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e5bfff700 1 -- 172.21.15.112:0/3325439415 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c00ac20 con 0x7f9e7415d440 2023-08-27T11:30:03.421 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e5bfff700 1 -- 172.21.15.112:0/3325439415 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6c00ad80 con 0x7f9e7415d440 2023-08-27T11:30:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e79935700 1 -- 172.21.15.112:0/3325439415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e74159ae0 con 0x7f9e7415d440 2023-08-27T11:30:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e5bfff700 1 -- 172.21.15.112:0/3325439415 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c007310 con 0x7f9e7415d440 2023-08-27T11:30:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.411+0000 7f9e79935700 1 -- 172.21.15.112:0/3325439415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e740c8970 con 0x7f9e7415d440 2023-08-27T11:30:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.412+0000 7f9e79935700 1 -- 172.21.15.112:0/3325439415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e740ae430 con 0x7f9e7415d440 2023-08-27T11:30:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.413+0000 7f9e5bfff700 1 -- 172.21.15.112:0/3325439415 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f9e6c007470 con 0x7f9e7415d440 2023-08-27T11:30:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.413+0000 7f9e5bfff700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e6006a4d0 0x7f9e6006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.413+0000 7f9e5bfff700 1 -- 172.21.15.112:0/3325439415 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9e6c088460 con 0x7f9e7415d440 2023-08-27T11:30:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.414+0000 7f9e72ffd700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e6006a4d0 0x7f9e6006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.414+0000 7f9e72ffd700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e6006a4d0 0x7f9e6006c990 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9e68000c00 tx=0x7f9e6801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.418+0000 7f9e5bfff700 1 -- 172.21.15.112:0/3325439415 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9e6c0539d0 con 0x7f9e7415d440 2023-08-27T11:30:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.609+0000 7f9e79935700 1 -- 172.21.15.112:0/3325439415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9e74067a20 con 0x7f9e7415d440 2023-08-27T11:30:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.609+0000 7f9e5bfff700 1 -- 172.21.15.112:0/3325439415 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v180) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9e6c056ff0 con 0x7f9e7415d440 2023-08-27T11:30:03.611 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 8 2023-08-27T11:30:03.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.612+0000 7f9e79935700 1 -- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e6006a4d0 msgr2=0x7f9e6006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.613 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.612+0000 7f9e79935700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e6006a4d0 0x7f9e6006c990 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9e68000c00 tx=0x7f9e6801c040 comp rx=0 tx=0).stop 2023-08-27T11:30:03.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.612+0000 7f9e79935700 1 -- 172.21.15.112:0/3325439415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7415d440 msgr2=0x7f9e741f2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.612+0000 7f9e79935700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7415d440 0x7f9e741f2a60 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00cca0 tx=0x7f9e6c00a060 comp rx=0 tx=0).stop 2023-08-27T11:30:03.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.612+0000 7f9e79935700 1 -- 172.21.15.112:0/3325439415 shutdown_connections 2023-08-27T11:30:03.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.612+0000 7f9e79935700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f9e6006a4d0 0x7f9e6006c990 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.612+0000 7f9e79935700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7415d440 0x7f9e741f2a60 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.612+0000 7f9e79935700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7406fcd0 0x7f9e741f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.612+0000 7f9e79935700 1 --2- 172.21.15.112:0/3325439415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e740cdbd0 0x7f9e741595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.612+0000 7f9e79935700 1 -- 172.21.15.112:0/3325439415 >> 172.21.15.112:0/3325439415 conn(0x7f9e740cca80 msgr2=0x7f9e7415a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:03.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.612+0000 7f9e79935700 1 -- 172.21.15.112:0/3325439415 shutdown_connections 2023-08-27T11:30:03.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.612+0000 7f9e79935700 1 -- 172.21.15.112:0/3325439415 wait complete. 2023-08-27T11:30:03.625 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:30:03.748 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.747+0000 7fbc9cb3d700 1 Processor -- start 2023-08-27T11:30:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.747+0000 7fbc9cb3d700 1 -- start start 2023-08-27T11:30:03.756 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.747+0000 7fbc9cb3d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbc980b3440 0x7fbc9815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc9cb3d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc980b5070 con 0x7fbc9815e6e0 2023-08-27T11:30:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc9cb3d700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc98147000 con 0x7fbc98146c20 2023-08-27T11:30:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc9cb3d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc98161740 con 0x7fbc980b3440 2023-08-27T11:30:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc95d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbc980b3440 0x7fbc9815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc96d9d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbc9815e6e0 0x7fbc9806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33590/0 (socket says 172.21.15.112:33590) 2023-08-27T11:30:03.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc96d9d700 1 -- 172.21.15.112:0/3508521698 learned_addr learned my addr 172.21.15.112:0/3508521698 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:03.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc95d9b700 1 -- 172.21.15.112:0/3508521698 >> v1:172.21.15.138:6789/0 conn(0x7fbc98146c20 legacy=0x7fbc98160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc95d9b700 1 -- 172.21.15.112:0/3508521698 >> v1:172.21.15.112:6789/0 conn(0x7fbc9815e6e0 legacy=0x7fbc9806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc9559a700 1 -- 172.21.15.112:0/3508521698 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188610880 0 0) 0x7fbc90001490 con 0x7fbc9815e6e0 2023-08-27T11:30:03.758 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc95d9b700 1 -- 172.21.15.112:0/3508521698 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc98161a20 con 0x7fbc980b3440 2023-08-27T11:30:03.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc95d9b700 1 --2- 172.21.15.112:0/3508521698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc980b3440 0x7fbc9815e1a0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009840 tx=0x7fbc8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897b5eb9df24e855 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:03.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc9559a700 1 -- 172.21.15.112:0/3508521698 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c011830 con 0x7fbc980b3440 2023-08-27T11:30:03.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc9559a700 1 -- 172.21.15.112:0/3508521698 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8c011990 con 0x7fbc980b3440 2023-08-27T11:30:03.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.748+0000 7fbc9559a700 1 -- 172.21.15.112:0/3508521698 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c011d20 con 0x7fbc980b3440 2023-08-27T11:30:03.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.749+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/3508521698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc980b3440 msgr2=0x7fbc9815e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.749+0000 7fbc9cb3d700 1 --2- 172.21.15.112:0/3508521698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc980b3440 0x7fbc9815e1a0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009840 tx=0x7fbc8c009b50 comp rx=0 tx=0).stop 2023-08-27T11:30:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.749+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/3508521698 shutdown_connections 2023-08-27T11:30:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.749+0000 7fbc9cb3d700 1 --2- 172.21.15.112:0/3508521698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc980b3440 0x7fbc9815e1a0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.749+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/3508521698 >> 172.21.15.112:0/3508521698 conn(0x7fbc980b1e90 msgr2=0x7fbc980b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:03.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.749+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/3508521698 shutdown_connections 2023-08-27T11:30:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.749+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/3508521698 wait complete. 2023-08-27T11:30:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.750+0000 7fbc9cb3d700 1 Processor -- start 2023-08-27T11:30:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.750+0000 7fbc9cb3d700 1 -- start start 2023-08-27T11:30:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.751+0000 7fbc9cb3d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc980b3440 0x7fbc981f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.761 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.751+0000 7fbc9cb3d700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc98146c20 0x7fbc98145db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.751+0000 7fbc9cb3d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc9815e6e0 0x7fbc98144210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.751+0000 7fbc9cb3d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc98144750 con 0x7fbc980b3440 2023-08-27T11:30:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.751+0000 7fbc9cb3d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc981449f0 con 0x7fbc9815e6e0 2023-08-27T11:30:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.751+0000 7fbc9cb3d700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc98144cd0 con 0x7fbc98146c20 2023-08-27T11:30:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.751+0000 7fbc9659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc980b3440 0x7fbc981f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.751+0000 7fbc9659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc980b3440 0x7fbc981f44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39214/0 (socket says 172.21.15.112:39214) 2023-08-27T11:30:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.751+0000 7fbc9659c700 1 -- 172.21.15.112:0/2256803960 learned_addr learned my addr 172.21.15.112:0/2256803960 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.751+0000 7fbc9659c700 1 -- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc98146c20 msgr2=0x7fbc98145db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.751+0000 7fbc9659c700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc98146c20 0x7fbc98145db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.751+0000 7fbc9659c700 1 -- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc9815e6e0 msgr2=0x7fbc98144210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.751+0000 7fbc96d9d700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc9815e6e0 0x7fbc98144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.752+0000 7fbc9659c700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc9815e6e0 0x7fbc98144210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.752+0000 7fbc9659c700 1 -- 172.21.15.112:0/2256803960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c0094a0 con 0x7fbc980b3440 2023-08-27T11:30:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.752+0000 7fbc96d9d700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc9815e6e0 0x7fbc98144210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.752+0000 7fbc9659c700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc980b3440 0x7fbc981f44f0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800c4e0 tx=0x7fbc8800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.752+0000 7fbc86ffd700 1 -- 172.21.15.112:0/2256803960 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc880052d0 con 0x7fbc980b3440 2023-08-27T11:30:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.752+0000 7fbc86ffd700 1 -- 172.21.15.112:0/2256803960 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc88005430 con 0x7fbc980b3440 2023-08-27T11:30:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.752+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/2256803960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc98144fb0 con 0x7fbc980b3440 2023-08-27T11:30:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.752+0000 7fbc86ffd700 1 -- 172.21.15.112:0/2256803960 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc880038c0 con 0x7fbc980b3440 2023-08-27T11:30:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.752+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/2256803960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc980ae5e0 con 0x7fbc980b3440 2023-08-27T11:30:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.753+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/2256803960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc981454d0 con 0x7fbc980b3440 2023-08-27T11:30:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.753+0000 7fbc86ffd700 1 -- 172.21.15.112:0/2256803960 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fbc88003a20 con 0x7fbc980b3440 2023-08-27T11:30:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.754+0000 7fbc86ffd700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbc7806a500 0x7fbc7806c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.754+0000 7fbc86ffd700 1 -- 172.21.15.112:0/2256803960 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbc88087e30 con 0x7fbc980b3440 2023-08-27T11:30:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.754+0000 7fbc95d9b700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbc7806a500 0x7fbc7806c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.754+0000 7fbc95d9b700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbc7806a500 0x7fbc7806c9c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009b90 tx=0x7fbc8c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.758+0000 7fbc86ffd700 1 -- 172.21.15.112:0/2256803960 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbc88053290 con 0x7fbc980b3440 2023-08-27T11:30:03.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.945+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/2256803960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc9815d8e0 con 0x7fbc980b3440 2023-08-27T11:30:03.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.945+0000 7fbc86ffd700 1 -- 172.21.15.112:0/2256803960 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v180) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc88017020 con 0x7fbc980b3440 2023-08-27T11:30:03.947 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2023-08-27T11:30:03.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.947+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbc7806a500 msgr2=0x7fbc7806c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.947+0000 7fbc9cb3d700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbc7806a500 0x7fbc7806c9c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009b90 tx=0x7fbc8c010000 comp rx=0 tx=0).stop 2023-08-27T11:30:03.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.947+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/2256803960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc980b3440 msgr2=0x7fbc981f44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:03.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.947+0000 7fbc9cb3d700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc980b3440 0x7fbc981f44f0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800c4e0 tx=0x7fbc8800c8a0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.947+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/2256803960 shutdown_connections 2023-08-27T11:30:03.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.947+0000 7fbc9cb3d700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbc7806a500 0x7fbc7806c9c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.947+0000 7fbc9cb3d700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc9815e6e0 0x7fbc98144210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.947+0000 7fbc9cb3d700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc980b3440 0x7fbc981f44f0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.947+0000 7fbc9cb3d700 1 --2- 172.21.15.112:0/2256803960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc98146c20 0x7fbc98145db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:03.950 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.947+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/2256803960 >> 172.21.15.112:0/2256803960 conn(0x7fbc980b1e90 msgr2=0x7fbc980b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:03.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.947+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/2256803960 shutdown_connections 2023-08-27T11:30:03.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:03.947+0000 7fbc9cb3d700 1 -- 172.21.15.112:0/2256803960 wait complete. 2023-08-27T11:30:03.961 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:30:04.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.079+0000 7fc1bb895700 1 Processor -- start 2023-08-27T11:30:04.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.079+0000 7fc1bb895700 1 -- start start 2023-08-27T11:30:04.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1bb895700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc1b4144570 0x7fc1b415ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1bb895700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc1b40b3e50 0x7fc1b415f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1bb895700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b40b5ae0 con 0x7fc1b415f8a0 2023-08-27T11:30:04.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1bb895700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b415fcf0 con 0x7fc1b4144570 2023-08-27T11:30:04.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1bb895700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b4144950 con 0x7fc1b40b3e50 2023-08-27T11:30:04.083 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1b9631700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc1b4144570 0x7fc1b415ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1b9e32700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc1b415f8a0 0x7fc1b40ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33606/0 (socket says 172.21.15.112:33606) 2023-08-27T11:30:04.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1b9e32700 1 -- 172.21.15.112:0/2214132290 learned_addr learned my addr 172.21.15.112:0/2214132290 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:04.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1b8e30700 1 --2- 172.21.15.112:0/2214132290 >> v2:172.21.15.138:3300/0 conn(0x7fc1b40b3e50 0x7fc1b415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1b9631700 1 -- 172.21.15.112:0/2214132290 >> v2:172.21.15.138:3300/0 conn(0x7fc1b40b3e50 msgr2=0x7fc1b415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:04.084 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1b9631700 1 --2- 172.21.15.112:0/2214132290 >> v2:172.21.15.138:3300/0 conn(0x7fc1b40b3e50 0x7fc1b415f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1b9631700 1 -- 172.21.15.112:0/2214132290 >> v1:172.21.15.112:6789/0 conn(0x7fc1b415f8a0 legacy=0x7fc1b40ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:04.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1b9631700 1 -- 172.21.15.112:0/2214132290 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b40b45f0 con 0x7fc1b4144570 2023-08-27T11:30:04.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1b9631700 1 --2- 172.21.15.112:0/2214132290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b4144570 0x7fc1b415ee20 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40b33f0 tx=0x7fc1a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5923188eb9ec7c0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:04.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1abfff700 1 -- 172.21.15.112:0/2214132290 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4011830 con 0x7fc1b4144570 2023-08-27T11:30:04.085 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1abfff700 1 -- 172.21.15.112:0/2214132290 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a4011990 con 0x7fc1b4144570 2023-08-27T11:30:04.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.080+0000 7fc1abfff700 1 -- 172.21.15.112:0/2214132290 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4011d20 con 0x7fc1b4144570 2023-08-27T11:30:04.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.081+0000 7fc1bb895700 1 -- 172.21.15.112:0/2214132290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b4144570 msgr2=0x7fc1b415ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:04.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.081+0000 7fc1bb895700 1 --2- 172.21.15.112:0/2214132290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b4144570 0x7fc1b415ee20 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40b33f0 tx=0x7fc1a4009b50 comp rx=0 tx=0).stop 2023-08-27T11:30:04.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.081+0000 7fc1bb895700 1 -- 172.21.15.112:0/2214132290 shutdown_connections 2023-08-27T11:30:04.086 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.081+0000 7fc1bb895700 1 --2- 172.21.15.112:0/2214132290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b4144570 0x7fc1b415ee20 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.081+0000 7fc1bb895700 1 --2- 172.21.15.112:0/2214132290 >> v2:172.21.15.138:3300/0 conn(0x7fc1b40b3e50 0x7fc1b415f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.081+0000 7fc1bb895700 1 -- 172.21.15.112:0/2214132290 >> 172.21.15.112:0/2214132290 conn(0x7fc1b406f970 msgr2=0x7fc1b406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:04.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.081+0000 7fc1bb895700 1 -- 172.21.15.112:0/2214132290 shutdown_connections 2023-08-27T11:30:04.087 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.081+0000 7fc1bb895700 1 -- 172.21.15.112:0/2214132290 wait complete. 2023-08-27T11:30:04.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.082+0000 7fc1bb895700 1 Processor -- start 2023-08-27T11:30:04.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.082+0000 7fc1bb895700 1 -- start start 2023-08-27T11:30:04.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1bb895700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b3e50 0x7fc1b4142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1bb895700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b4144570 0x7fc1b41f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.090 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1bb895700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1b415f8a0 0x7fc1b41f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1bb895700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b4143000 con 0x7fc1b40b3e50 2023-08-27T11:30:04.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1bb895700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40a8520 con 0x7fc1b4144570 2023-08-27T11:30:04.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1bb895700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40a8800 con 0x7fc1b415f8a0 2023-08-27T11:30:04.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1b9631700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b3e50 0x7fc1b4142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.091 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1b9631700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b3e50 0x7fc1b4142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39226/0 (socket says 172.21.15.112:39226) 2023-08-27T11:30:04.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1b9631700 1 -- 172.21.15.112:0/1839676450 learned_addr learned my addr 172.21.15.112:0/1839676450 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:04.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1b8e30700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b4144570 0x7fc1b41f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1b9e32700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1b415f8a0 0x7fc1b41f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1b9631700 1 -- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1b415f8a0 msgr2=0x7fc1b41f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:04.092 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1b9631700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1b415f8a0 0x7fc1b41f6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1b9631700 1 -- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b4144570 msgr2=0x7fc1b41f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:04.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1b9631700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b4144570 0x7fc1b41f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1b9631700 1 -- 172.21.15.112:0/1839676450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a40094a0 con 0x7fc1b40b3e50 2023-08-27T11:30:04.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.083+0000 7fc1b9631700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b3e50 0x7fc1b4142ac0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40097c0 tx=0x7fc1a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:04.093 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.084+0000 7fc1a9ffb700 1 -- 172.21.15.112:0/1839676450 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a40038a0 con 0x7fc1b40b3e50 2023-08-27T11:30:04.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.084+0000 7fc1a9ffb700 1 -- 172.21.15.112:0/1839676450 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a401f070 con 0x7fc1b40b3e50 2023-08-27T11:30:04.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.084+0000 7fc1bb895700 1 -- 172.21.15.112:0/1839676450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b40a8ae0 con 0x7fc1b40b3e50 2023-08-27T11:30:04.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.084+0000 7fc1a9ffb700 1 -- 172.21.15.112:0/1839676450 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4003c40 con 0x7fc1b40b3e50 2023-08-27T11:30:04.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.084+0000 7fc1bb895700 1 -- 172.21.15.112:0/1839676450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b40a90a0 con 0x7fc1b40b3e50 2023-08-27T11:30:04.094 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.085+0000 7fc1bb895700 1 -- 172.21.15.112:0/1839676450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b4067a20 con 0x7fc1b40b3e50 2023-08-27T11:30:04.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.085+0000 7fc1a9ffb700 1 -- 172.21.15.112:0/1839676450 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fc1a4026480 con 0x7fc1b40b3e50 2023-08-27T11:30:04.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.086+0000 7fc1a9ffb700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc19806a4d0 0x7fc19806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.086+0000 7fc1a9ffb700 1 -- 172.21.15.112:0/1839676450 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc1a4021070 con 0x7fc1b40b3e50 2023-08-27T11:30:04.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.086+0000 7fc1b8e30700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc19806a4d0 0x7fc19806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.095 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.086+0000 7fc1b8e30700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc19806a4d0 0x7fc19806c990 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009930 tx=0x7fc1b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:04.096 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.090+0000 7fc1a9ffb700 1 -- 172.21.15.112:0/1839676450 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc1a40557f0 con 0x7fc1b40b3e50 2023-08-27T11:30:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.282+0000 7fc1bb895700 1 -- 172.21.15.112:0/1839676450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc1b406bb00 con 0x7fc1b40b3e50 2023-08-27T11:30:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.283+0000 7fc1a9ffb700 1 -- 172.21.15.112:0/1839676450 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v180) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc1a4058e10 con 0x7fc1b40b3e50 2023-08-27T11:30:04.284 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:30:04.284 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2023-08-27T11:30:04.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:04.285 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2023-08-27T11:30:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "rbd" 2023-08-27T11:30:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-08-27T11:30:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-08-27T11:30:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 2, 2023-08-27T11:30:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2023-08-27T11:30:04.287 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-08-27T11:30:04.287 INFO:teuthology.orchestra.run.smithi112.stdout:] 2023-08-27T11:30:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.285+0000 7fc1bb895700 1 -- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc19806a4d0 msgr2=0x7fc19806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.285+0000 7fc1bb895700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc19806a4d0 0x7fc19806c990 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009930 tx=0x7fc1b0008040 comp rx=0 tx=0).stop 2023-08-27T11:30:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.285+0000 7fc1bb895700 1 -- 172.21.15.112:0/1839676450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b3e50 msgr2=0x7fc1b4142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.285+0000 7fc1bb895700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b3e50 0x7fc1b4142ac0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40097c0 tx=0x7fc1a4004750 comp rx=0 tx=0).stop 2023-08-27T11:30:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.285+0000 7fc1bb895700 1 -- 172.21.15.112:0/1839676450 shutdown_connections 2023-08-27T11:30:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.285+0000 7fc1bb895700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fc19806a4d0 0x7fc19806c990 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.285+0000 7fc1bb895700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1b415f8a0 0x7fc1b41f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.285+0000 7fc1bb895700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b3e50 0x7fc1b4142ac0 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.285+0000 7fc1bb895700 1 --2- 172.21.15.112:0/1839676450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b4144570 0x7fc1b41f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.285+0000 7fc1bb895700 1 -- 172.21.15.112:0/1839676450 >> 172.21.15.112:0/1839676450 conn(0x7fc1b406f970 msgr2=0x7fc1b40cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:04.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.285+0000 7fc1bb895700 1 -- 172.21.15.112:0/1839676450 shutdown_connections 2023-08-27T11:30:04.291 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.285+0000 7fc1bb895700 1 -- 172.21.15.112:0/1839676450 wait complete. 2023-08-27T11:30:04.298 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:30:04.298 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:30:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.421+0000 7f1395951700 1 Processor -- start 2023-08-27T11:30:04.423 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.421+0000 7f1395951700 1 -- start start 2023-08-27T11:30:04.426 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.422+0000 7f1395951700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1390146d50 0x7f13900b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.422+0000 7f1395951700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13900b5750 0x7f13900d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.422+0000 7f1395951700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13900b7f60 con 0x7f13900b69b0 2023-08-27T11:30:04.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.422+0000 7f1395951700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13900b6ec0 con 0x7f1390146d50 2023-08-27T11:30:04.427 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.422+0000 7f1395951700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13900b71a0 con 0x7f13900b5750 2023-08-27T11:30:04.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.422+0000 7f138e7fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f13900b69b0 0x7f13900b5040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33614/0 (socket says 172.21.15.112:33614) 2023-08-27T11:30:04.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.422+0000 7f138e7fc700 1 -- 172.21.15.112:0/3908920739 learned_addr learned my addr 172.21.15.112:0/3908920739 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:04.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.422+0000 7f138effd700 1 --2- 172.21.15.112:0/3908920739 >> v2:172.21.15.138:3300/0 conn(0x7f1390146d50 0x7f13900b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.422+0000 7f138f7fe700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13900b5750 0x7f13900d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.428 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.422+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129968877 0 0) 0x7f13900b7f60 con 0x7f13900b69b0 2023-08-27T11:30:04.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.422+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13780035b0 con 0x7f13900b69b0 2023-08-27T11:30:04.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1783204122 0 0) 0x7f13780035b0 con 0x7f13900b69b0 2023-08-27T11:30:04.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13900b5750 msgr2=0x7f13900d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:04.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13900b5750 0x7f13900d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.429 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 >> v2:172.21.15.138:3300/0 conn(0x7f1390146d50 msgr2=0x7f13900b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 --2- 172.21.15.112:0/3908920739 >> v2:172.21.15.138:3300/0 conn(0x7f1390146d50 0x7f13900b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13900b7480 con 0x7f13900b69b0 2023-08-27T11:30:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f1380001670 con 0x7f13900b69b0 2023-08-27T11:30:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 >> v1:172.21.15.112:6789/0 conn(0x7f13900b69b0 legacy=0x7f13900b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1378003a00 0x7f1378003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1378004850 0x7f1378004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13900b5750 0x7f1378005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.431 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13900b6ec0 con 0x7f1378004850 2023-08-27T11:30:04.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13900b71a0 con 0x7f1378003a00 2023-08-27T11:30:04.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137800ba10 con 0x7f13900b5750 2023-08-27T11:30:04.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138e7fc700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1378004850 0x7f1378004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.432 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138effd700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1378003a00 0x7f1378003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138f7fe700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13900b5750 0x7f1378005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138e7fc700 1 -- 172.21.15.112:0/3908920739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13900b5750 msgr2=0x7f1378005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138e7fc700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13900b5750 0x7f1378005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138e7fc700 1 -- 172.21.15.112:0/3908920739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1378003a00 msgr2=0x7f1378003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138e7fc700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1378003a00 0x7f1378003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138e7fc700 1 -- 172.21.15.112:0/3908920739 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13780035b0 con 0x7f1378004850 2023-08-27T11:30:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138effd700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1378003a00 0x7f1378003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138f7fe700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13900b5750 0x7f1378005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138e7fc700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1378004850 0x7f1378004cd0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f13900b6df0 tx=0x7f1378005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58aebf650e7563c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380007c60 con 0x7f1378004850 2023-08-27T11:30:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138000ce80 con 0x7f1378004850 2023-08-27T11:30:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.423+0000 7f138dffb700 1 -- 172.21.15.112:0/3908920739 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380005ba0 con 0x7f1378004850 2023-08-27T11:30:04.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.424+0000 7f1395951700 1 -- 172.21.15.112:0/3908920739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1378004850 msgr2=0x7f1378004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:04.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.424+0000 7f1395951700 1 --2- 172.21.15.112:0/3908920739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1378004850 0x7f1378004cd0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f13900b6df0 tx=0x7f1378005880 comp rx=0 tx=0).stop 2023-08-27T11:30:04.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.424+0000 7f138f7fe700 1 -- 172.21.15.112:0/3908920739 reap_dead start 2023-08-27T11:30:04.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.424+0000 7f1395951700 1 -- 172.21.15.112:0/3908920739 shutdown_connections 2023-08-27T11:30:04.436 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.424+0000 7f1395951700 1 -- 172.21.15.112:0/3908920739 >> 172.21.15.112:0/3908920739 conn(0x7f139006d330 msgr2=0x7f13900717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:04.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.424+0000 7f1395951700 1 -- 172.21.15.112:0/3908920739 shutdown_connections 2023-08-27T11:30:04.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.424+0000 7f1395951700 1 -- 172.21.15.112:0/3908920739 wait complete. 2023-08-27T11:30:04.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.425+0000 7f1395951700 1 Processor -- start 2023-08-27T11:30:04.437 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.425+0000 7f1395951700 1 -- start start 2023-08-27T11:30:04.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f1395951700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1378004850 0x7f139015c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f1395951700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1390160600 0x7f139013d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f1395951700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13901f27c0 0x7f13901f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.441 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f1395951700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139013cf70 con 0x7f1390160600 2023-08-27T11:30:04.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f1395951700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139015cf20 con 0x7f13901f27c0 2023-08-27T11:30:04.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f1395951700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139013c8b0 con 0x7f1378004850 2023-08-27T11:30:04.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f138e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1390160600 0x7f139013d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f138e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1390160600 0x7f139013d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39246/0 (socket says 172.21.15.112:39246) 2023-08-27T11:30:04.442 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f138e7fc700 1 -- 172.21.15.112:0/343868877 learned_addr learned my addr 172.21.15.112:0/343868877 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:04.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f138effd700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1378004850 0x7f139015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f138f7fe700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13901f27c0 0x7f13901f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f138e7fc700 1 -- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1378004850 msgr2=0x7f139015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:04.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f138e7fc700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1378004850 0x7f139015c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.443 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f138e7fc700 1 -- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13901f27c0 msgr2=0x7f13901f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:04.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f138e7fc700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13901f27c0 0x7f13901f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:04.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f138e7fc700 1 -- 172.21.15.112:0/343868877 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139013cb90 con 0x7f1390160600 2023-08-27T11:30:04.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.426+0000 7f138effd700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1378004850 0x7f139015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:04.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.427+0000 7f138e7fc700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1390160600 0x7f139013d390 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f13780046d0 tx=0x7f1380000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:04.444 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.427+0000 7f1377fff700 1 -- 172.21.15.112:0/343868877 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13800070d0 con 0x7f1390160600 2023-08-27T11:30:04.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.427+0000 7f1377fff700 1 -- 172.21.15.112:0/343868877 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1380007230 con 0x7f1390160600 2023-08-27T11:30:04.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.427+0000 7f1395951700 1 -- 172.21.15.112:0/343868877 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1390160a80 con 0x7f1390160600 2023-08-27T11:30:04.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.427+0000 7f1377fff700 1 -- 172.21.15.112:0/343868877 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13800075e0 con 0x7f1390160600 2023-08-27T11:30:04.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.427+0000 7f1395951700 1 -- 172.21.15.112:0/343868877 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13901f2c40 con 0x7f1390160600 2023-08-27T11:30:04.445 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.428+0000 7f1395951700 1 -- 172.21.15.112:0/343868877 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f139013c710 con 0x7f1390160600 2023-08-27T11:30:04.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.428+0000 7f1377fff700 1 -- 172.21.15.112:0/343868877 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f138001e430 con 0x7f1390160600 2023-08-27T11:30:04.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.428+0000 7f1377fff700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f137c06a4d0 0x7f137c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:04.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.429+0000 7f1377fff700 1 -- 172.21.15.112:0/343868877 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1380089c90 con 0x7f1390160600 2023-08-27T11:30:04.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.429+0000 7f138effd700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f137c06a4d0 0x7f137c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:04.446 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.429+0000 7f138effd700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f137c06a4d0 0x7f137c06c990 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1384009bd0 tx=0x7f1384008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:04.447 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.433+0000 7f1377fff700 1 -- 172.21.15.112:0/343868877 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f139013c710 con 0x7f1390160600 2023-08-27T11:30:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:04.624+0000 7f1395951700 1 -- 172.21.15.112:0/343868877 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f139013c710 con 0x7f1390160600 2023-08-27T11:30:05.489 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.488+0000 7f1377fff700 1 -- 172.21.15.112:0/343868877 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v181) v1 ==== 138+0+0 (secure 0 0 0) 0x7f139013c710 con 0x7f1390160600 2023-08-27T11:30:05.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.492+0000 7f1395951700 1 -- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f137c06a4d0 msgr2=0x7f137c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:05.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.492+0000 7f1395951700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f137c06a4d0 0x7f137c06c990 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1384009bd0 tx=0x7f1384008040 comp rx=0 tx=0).stop 2023-08-27T11:30:05.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.492+0000 7f1395951700 1 -- 172.21.15.112:0/343868877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1390160600 msgr2=0x7f139013d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:05.494 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.492+0000 7f1395951700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1390160600 0x7f139013d390 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f13780046d0 tx=0x7f1380000fe0 comp rx=0 tx=0).stop 2023-08-27T11:30:05.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.492+0000 7f1395951700 1 -- 172.21.15.112:0/343868877 shutdown_connections 2023-08-27T11:30:05.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.492+0000 7f1395951700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f137c06a4d0 0x7f137c06c990 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:05.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.492+0000 7f1395951700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13901f27c0 0x7f13901f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.492+0000 7f1395951700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1378004850 0x7f139015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.492+0000 7f1395951700 1 --2- 172.21.15.112:0/343868877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1390160600 0x7f139013d390 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.492+0000 7f1395951700 1 -- 172.21.15.112:0/343868877 >> 172.21.15.112:0/343868877 conn(0x7f139006d330 msgr2=0x7f1390140500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:05.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.492+0000 7f1395951700 1 -- 172.21.15.112:0/343868877 shutdown_connections 2023-08-27T11:30:05.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.493+0000 7f1395951700 1 -- 172.21.15.112:0/343868877 wait complete. 2023-08-27T11:30:05.498 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2023-08-27T11:30:05.512 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:30:05.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.645+0000 7f0cc953b700 1 Processor -- start 2023-08-27T11:30:05.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.645+0000 7f0cc953b700 1 -- start start 2023-08-27T11:30:05.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.646+0000 7f0cc953b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0cc4146bf0 0x7f0cc40b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:05.654 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.646+0000 7f0cc953b700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0cc4070eb0 0x7f0cc4071320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:05.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.646+0000 7f0cc953b700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc40d1500 con 0x7f0cc4146fd0 2023-08-27T11:30:05.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.646+0000 7f0cc953b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc4071860 con 0x7f0cc4146bf0 2023-08-27T11:30:05.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.646+0000 7f0cc953b700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc40d0460 con 0x7f0cc4070eb0 2023-08-27T11:30:05.655 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.646+0000 7f0cc2ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0cc4146bf0 0x7f0cc40b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:05.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.646+0000 7f0cc2ffd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0cc4146bf0 0x7f0cc40b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39252/0 (socket says 172.21.15.112:39252) 2023-08-27T11:30:05.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.646+0000 7f0cc2ffd700 1 -- 172.21.15.112:0/2943367655 learned_addr learned my addr 172.21.15.112:0/2943367655 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:05.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.646+0000 7f0cc37fe700 1 --2- 172.21.15.112:0/2943367655 >> v2:172.21.15.138:3300/0 conn(0x7f0cc4070eb0 0x7f0cc4071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:05.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc2ffd700 1 -- 172.21.15.112:0/2943367655 >> v2:172.21.15.138:3300/0 conn(0x7f0cc4070eb0 msgr2=0x7f0cc4071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:05.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc2ffd700 1 --2- 172.21.15.112:0/2943367655 >> v2:172.21.15.138:3300/0 conn(0x7f0cc4070eb0 0x7f0cc4071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:05.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc2ffd700 1 -- 172.21.15.112:0/2943367655 >> v1:172.21.15.138:6789/0 conn(0x7f0cc4146fd0 legacy=0x7f0cc40cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:05.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc2ffd700 1 -- 172.21.15.112:0/2943367655 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc40d0710 con 0x7f0cc4146bf0 2023-08-27T11:30:05.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc2ffd700 1 --2- 172.21.15.112:0/2943367655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cc4146bf0 0x7f0cc40b5db0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40b58f0 tx=0x7f0cb4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2570ec899f0ed5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:05.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc1ffb700 1 -- 172.21.15.112:0/2943367655 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb400b870 con 0x7f0cc4146bf0 2023-08-27T11:30:05.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc1ffb700 1 -- 172.21.15.112:0/2943367655 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb400b9d0 con 0x7f0cc4146bf0 2023-08-27T11:30:05.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc1ffb700 1 -- 172.21.15.112:0/2943367655 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb400bd60 con 0x7f0cc4146bf0 2023-08-27T11:30:05.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc953b700 1 -- 172.21.15.112:0/2943367655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cc4146bf0 msgr2=0x7f0cc40b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:05.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc953b700 1 --2- 172.21.15.112:0/2943367655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cc4146bf0 0x7f0cc40b5db0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40b58f0 tx=0x7f0cb4009bd0 comp rx=0 tx=0).stop 2023-08-27T11:30:05.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc953b700 1 -- 172.21.15.112:0/2943367655 shutdown_connections 2023-08-27T11:30:05.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc953b700 1 --2- 172.21.15.112:0/2943367655 >> v2:172.21.15.138:3300/0 conn(0x7f0cc4070eb0 0x7f0cc4071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc953b700 1 --2- 172.21.15.112:0/2943367655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cc4146bf0 0x7f0cc40b5db0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.647+0000 7f0cc953b700 1 -- 172.21.15.112:0/2943367655 >> 172.21.15.112:0/2943367655 conn(0x7f0cc406c450 msgr2=0x7f0cc406f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.648+0000 7f0cc953b700 1 -- 172.21.15.112:0/2943367655 shutdown_connections 2023-08-27T11:30:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.648+0000 7f0cc953b700 1 -- 172.21.15.112:0/2943367655 wait complete. 2023-08-27T11:30:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.649+0000 7f0cc953b700 1 Processor -- start 2023-08-27T11:30:05.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.649+0000 7f0cc953b700 1 -- start start 2023-08-27T11:30:05.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc953b700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0cc4070eb0 0x7f0cc415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:05.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc953b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cc4146bf0 0x7f0cc4141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:05.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc953b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cc4146fd0 0x7f0cc4142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:05.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc953b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40ad470 con 0x7f0cc4146fd0 2023-08-27T11:30:05.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc953b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4140fe0 con 0x7f0cc4146bf0 2023-08-27T11:30:05.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc953b700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4140360 con 0x7f0cc4070eb0 2023-08-27T11:30:05.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cc4146fd0 0x7f0cc4142390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:05.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc27fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cc4146bf0 0x7f0cc4141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:05.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cc4146fd0 0x7f0cc4142390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39262/0 (socket says 172.21.15.112:39262) 2023-08-27T11:30:05.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc27fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cc4146bf0 0x7f0cc4141e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.112:39686/0 (socket says 172.21.15.112:39686) 2023-08-27T11:30:05.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc37fe700 1 -- 172.21.15.112:0/2159245746 learned_addr learned my addr 172.21.15.112:0/2159245746 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:05.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc2ffd700 1 --2- 172.21.15.112:0/2159245746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0cc4070eb0 0x7f0cc415ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:05.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc37fe700 1 -- 172.21.15.112:0/2159245746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0cc4070eb0 msgr2=0x7f0cc415ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:05.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc37fe700 1 --2- 172.21.15.112:0/2159245746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0cc4070eb0 0x7f0cc415ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:05.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc37fe700 1 -- 172.21.15.112:0/2159245746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cc4146bf0 msgr2=0x7f0cc4141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:05.663 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc37fe700 1 --2- 172.21.15.112:0/2159245746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cc4146bf0 0x7f0cc4141e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:05.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc37fe700 1 -- 172.21.15.112:0/2159245746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb4009550 con 0x7f0cc4146fd0 2023-08-27T11:30:05.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc37fe700 1 --2- 172.21.15.112:0/2159245746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cc4146fd0 0x7f0cc4142390 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc007e70 tx=0x7f0cbc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:05.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cab7fe700 1 -- 172.21.15.112:0/2159245746 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00ed30 con 0x7f0cc4146fd0 2023-08-27T11:30:05.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cab7fe700 1 -- 172.21.15.112:0/2159245746 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc00eeb0 con 0x7f0cc4146fd0 2023-08-27T11:30:05.664 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc953b700 1 -- 172.21.15.112:0/2159245746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc4140640 con 0x7f0cc4146fd0 2023-08-27T11:30:05.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cab7fe700 1 -- 172.21.15.112:0/2159245746 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc014640 con 0x7f0cc4146fd0 2023-08-27T11:30:05.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.650+0000 7f0cc953b700 1 -- 172.21.15.112:0/2159245746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc4140c00 con 0x7f0cc4146fd0 2023-08-27T11:30:05.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.651+0000 7f0cc953b700 1 -- 172.21.15.112:0/2159245746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc4067e90 con 0x7f0cc4146fd0 2023-08-27T11:30:05.665 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.652+0000 7f0cab7fe700 1 -- 172.21.15.112:0/2159245746 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f0cbc0147a0 con 0x7f0cc4146fd0 2023-08-27T11:30:05.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.652+0000 7f0cab7fe700 1 --2- 172.21.15.112:0/2159245746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0cb006a4d0 0x7f0cb006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:05.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.652+0000 7f0cab7fe700 1 -- 172.21.15.112:0/2159245746 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f0cbc0891a0 con 0x7f0cc4146fd0 2023-08-27T11:30:05.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.653+0000 7f0cc2ffd700 1 --2- 172.21.15.112:0/2159245746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0cb006a4d0 0x7f0cb006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:05.666 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.653+0000 7f0cc2ffd700 1 --2- 172.21.15.112:0/2159245746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0cb006a4d0 0x7f0cb006c990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4000c00 tx=0x7f0cb4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:05.667 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.657+0000 7f0cab7fe700 1 -- 172.21.15.112:0/2159245746 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0cbc057f20 con 0x7f0cc4146fd0 2023-08-27T11:30:05.846 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:05.845+0000 7f0cc953b700 1 -- 172.21.15.112:0/2159245746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0cc4067a20 con 0x7f0cc4146fd0 2023-08-27T11:30:06.492 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.491+0000 7f0cab7fe700 1 -- 172.21.15.112:0/2159245746 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v182) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0cbc05d0d0 con 0x7f0cc4146fd0 2023-08-27T11:30:06.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.496+0000 7f0cc953b700 1 -- 172.21.15.112:0/2159245746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0cb006a4d0 msgr2=0x7f0cb006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:06.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.496+0000 7f0cc953b700 1 --2- 172.21.15.112:0/2159245746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0cb006a4d0 0x7f0cb006c990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4000c00 tx=0x7f0cb4008030 comp rx=0 tx=0).stop 2023-08-27T11:30:06.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.496+0000 7f0cc953b700 1 -- 172.21.15.112:0/2159245746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cc4146fd0 msgr2=0x7f0cc4142390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:06.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.496+0000 7f0cc953b700 1 --2- 172.21.15.112:0/2159245746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cc4146fd0 0x7f0cc4142390 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc007e70 tx=0x7f0cbc00e3c0 comp rx=0 tx=0).stop 2023-08-27T11:30:06.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.497+0000 7f0cc953b700 1 -- 172.21.15.112:0/2159245746 shutdown_connections 2023-08-27T11:30:06.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.497+0000 7f0cc953b700 1 --2- 172.21.15.112:0/2159245746 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f0cb006a4d0 0x7f0cb006c990 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:06.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.497+0000 7f0cc953b700 1 --2- 172.21.15.112:0/2159245746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cc4146fd0 0x7f0cc4142390 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:06.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.497+0000 7f0cc953b700 1 --2- 172.21.15.112:0/2159245746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0cc4070eb0 0x7f0cc415ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:06.499 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.497+0000 7f0cc953b700 1 --2- 172.21.15.112:0/2159245746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cc4146bf0 0x7f0cc4141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.497+0000 7f0cc953b700 1 -- 172.21.15.112:0/2159245746 >> 172.21.15.112:0/2159245746 conn(0x7f0cc406c450 msgr2=0x7f0cc4143c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.499+0000 7f0cc953b700 1 -- 172.21.15.112:0/2159245746 shutdown_connections 2023-08-27T11:30:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.499+0000 7f0cc953b700 1 -- 172.21.15.112:0/2159245746 wait complete. 2023-08-27T11:30:06.504 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2023-08-27T11:30:06.521 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:30:06.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.644+0000 7fcee2560700 1 Processor -- start 2023-08-27T11:30:06.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.645+0000 7fcee2560700 1 -- start start 2023-08-27T11:30:06.648 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.645+0000 7fcee2560700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcedc146c20 0x7fcedc160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:06.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.645+0000 7fcee2560700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcedc0b3440 0x7fcedc15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:06.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.645+0000 7fcee2560700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcedc0b5070 con 0x7fcedc15e6e0 2023-08-27T11:30:06.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.645+0000 7fcee2560700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc147000 con 0x7fcedc146c20 2023-08-27T11:30:06.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.645+0000 7fcee2560700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc1614a0 con 0x7fcedc0b3440 2023-08-27T11:30:06.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.645+0000 7fcedbfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcedc146c20 0x7fcedc160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:06.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.646+0000 7fcedbfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcedc146c20 0x7fcedc160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39266/0 (socket says 172.21.15.112:39266) 2023-08-27T11:30:06.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.646+0000 7fcedbfff700 1 -- 172.21.15.112:0/3843422566 learned_addr learned my addr 172.21.15.112:0/3843422566 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:06.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.646+0000 7fcedb7fe700 1 --2- 172.21.15.112:0/3843422566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcedc0b3440 0x7fcedc15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:06.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.646+0000 7fcedbfff700 1 -- 172.21.15.112:0/3843422566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcedc0b3440 msgr2=0x7fcedc15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:06.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.646+0000 7fcedbfff700 1 --2- 172.21.15.112:0/3843422566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcedc0b3440 0x7fcedc15e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.646+0000 7fcedbfff700 1 -- 172.21.15.112:0/3843422566 >> v1:172.21.15.138:6789/0 conn(0x7fcedc15e6e0 legacy=0x7fcedc06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.646+0000 7fcedbfff700 1 -- 172.21.15.112:0/3843422566 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc161740 con 0x7fcedc146c20 2023-08-27T11:30:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.646+0000 7fcedb7fe700 1 --2- 172.21.15.112:0/3843422566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcedc0b3440 0x7fcedc15e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.646+0000 7fcedbfff700 1 --2- 172.21.15.112:0/3843422566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcedc146c20 0x7fcedc160f60 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fced00098f0 tx=0x7fced0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f0359500dd1414 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.646+0000 7fcedaffd700 1 -- 172.21.15.112:0/3843422566 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced000b8a0 con 0x7fcedc146c20 2023-08-27T11:30:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.646+0000 7fcedaffd700 1 -- 172.21.15.112:0/3843422566 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced000ba00 con 0x7fcedc146c20 2023-08-27T11:30:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.646+0000 7fcedaffd700 1 -- 172.21.15.112:0/3843422566 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced000bd90 con 0x7fcedc146c20 2023-08-27T11:30:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.647+0000 7fcee2560700 1 -- 172.21.15.112:0/3843422566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcedc146c20 msgr2=0x7fcedc160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.647+0000 7fcee2560700 1 --2- 172.21.15.112:0/3843422566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcedc146c20 0x7fcedc160f60 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fced00098f0 tx=0x7fced0009c00 comp rx=0 tx=0).stop 2023-08-27T11:30:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.647+0000 7fcee2560700 1 -- 172.21.15.112:0/3843422566 shutdown_connections 2023-08-27T11:30:06.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.647+0000 7fcee2560700 1 --2- 172.21.15.112:0/3843422566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcedc146c20 0x7fcedc160f60 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:06.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.647+0000 7fcee2560700 1 --2- 172.21.15.112:0/3843422566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcedc0b3440 0x7fcedc15e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:06.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.647+0000 7fcee2560700 1 -- 172.21.15.112:0/3843422566 >> 172.21.15.112:0/3843422566 conn(0x7fcedc0b1e90 msgr2=0x7fcedc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:06.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.647+0000 7fcee2560700 1 -- 172.21.15.112:0/3843422566 shutdown_connections 2023-08-27T11:30:06.653 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.647+0000 7fcee2560700 1 -- 172.21.15.112:0/3843422566 wait complete. 2023-08-27T11:30:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.648+0000 7fcee2560700 1 Processor -- start 2023-08-27T11:30:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.649+0000 7fcee2560700 1 -- start start 2023-08-27T11:30:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.649+0000 7fcee2560700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcedc0b3440 0x7fcedc145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.649+0000 7fcee2560700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcedc146c20 0x7fcedc1f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.649+0000 7fcee2560700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcedc15e6e0 0x7fcedc1f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.649+0000 7fcee2560700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc138ac0 con 0x7fcedc0b3440 2023-08-27T11:30:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.649+0000 7fcee2560700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc138d10 con 0x7fcedc146c20 2023-08-27T11:30:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.649+0000 7fcee2560700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc138ff0 con 0x7fcedc15e6e0 2023-08-27T11:30:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fcedbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcedc0b3440 0x7fcedc145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:06.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fcedbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcedc0b3440 0x7fcedc145dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39272/0 (socket says 172.21.15.112:39272) 2023-08-27T11:30:06.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fcedbfff700 1 -- 172.21.15.112:0/1511618174 learned_addr learned my addr 172.21.15.112:0/1511618174 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:06.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fcedb7fe700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcedc146c20 0x7fcedc1f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:06.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fcee0afd700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcedc15e6e0 0x7fcedc1f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:06.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fcedbfff700 1 -- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcedc15e6e0 msgr2=0x7fcedc1f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fcedbfff700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcedc15e6e0 0x7fcedc1f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fcedbfff700 1 -- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcedc146c20 msgr2=0x7fcedc1f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fcedbfff700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcedc146c20 0x7fcedc1f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fcedbfff700 1 -- 172.21.15.112:0/1511618174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced0009550 con 0x7fcedc0b3440 2023-08-27T11:30:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fcedbfff700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcedc0b3440 0x7fcedc145dc0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fced0000c00 tx=0x7fced0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:06.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fced8ff9700 1 -- 172.21.15.112:0/1511618174 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0004480 con 0x7fcedc0b3440 2023-08-27T11:30:06.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fced8ff9700 1 -- 172.21.15.112:0/1511618174 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced001f070 con 0x7fcedc0b3440 2023-08-27T11:30:06.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fcee2560700 1 -- 172.21.15.112:0/1511618174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc1392d0 con 0x7fcedc0b3440 2023-08-27T11:30:06.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fced8ff9700 1 -- 172.21.15.112:0/1511618174 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced00038c0 con 0x7fcedc0b3440 2023-08-27T11:30:06.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.650+0000 7fcee2560700 1 -- 172.21.15.112:0/1511618174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc1f6350 con 0x7fcedc0b3440 2023-08-27T11:30:06.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.651+0000 7fcee2560700 1 -- 172.21.15.112:0/1511618174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc1f61b0 con 0x7fcedc0b3440 2023-08-27T11:30:06.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.652+0000 7fced8ff9700 1 -- 172.21.15.112:0/1511618174 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fced0021030 con 0x7fcedc0b3440 2023-08-27T11:30:06.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.652+0000 7fced8ff9700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcec806a4d0 0x7fcec806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:06.661 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.652+0000 7fced8ff9700 1 -- 172.21.15.112:0/1511618174 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fced008a160 con 0x7fcedc0b3440 2023-08-27T11:30:06.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.652+0000 7fcedb7fe700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcec806a4d0 0x7fcec806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:06.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.653+0000 7fcedb7fe700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcec806a4d0 0x7fcec806c990 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fcedc140d40 tx=0x7fcecc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:06.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.657+0000 7fced8ff9700 1 -- 172.21.15.112:0/1511618174 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcedc1f61b0 con 0x7fcedc0b3440 2023-08-27T11:30:06.890 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:06.889+0000 7fcee2560700 1 -- 172.21.15.112:0/1511618174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcedc1f61b0 con 0x7fcedc0b3440 2023-08-27T11:30:07.503 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:30:07.506 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:30:07.506 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.504+0000 7fced8ff9700 1 -- 172.21.15.112:0/1511618174 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 32 and data pool 33 v121) v1 ==== 174+0+0 (secure 0 0 0) 0x7fcedc1f61b0 con 0x7fcedc0b3440 2023-08-27T11:30:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.511+0000 7fcee2560700 1 -- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcec806a4d0 msgr2=0x7fcec806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.511+0000 7fcee2560700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcec806a4d0 0x7fcec806c990 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fcedc140d40 tx=0x7fcecc008040 comp rx=0 tx=0).stop 2023-08-27T11:30:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.511+0000 7fcee2560700 1 -- 172.21.15.112:0/1511618174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcedc0b3440 msgr2=0x7fcedc145dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:07.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.511+0000 7fcee2560700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcedc0b3440 0x7fcedc145dc0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fced0000c00 tx=0x7fced0004750 comp rx=0 tx=0).stop 2023-08-27T11:30:07.513 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.511+0000 7fcee2560700 1 -- 172.21.15.112:0/1511618174 shutdown_connections 2023-08-27T11:30:07.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.511+0000 7fcee2560700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fcec806a4d0 0x7fcec806c990 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.511+0000 7fcee2560700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcedc15e6e0 0x7fcedc1f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.511+0000 7fcee2560700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcedc0b3440 0x7fcedc145dc0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.511+0000 7fcee2560700 1 --2- 172.21.15.112:0/1511618174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcedc146c20 0x7fcedc1f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.514 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.511+0000 7fcee2560700 1 -- 172.21.15.112:0/1511618174 >> 172.21.15.112:0/1511618174 conn(0x7fcedc0b1e90 msgr2=0x7fcedc1410a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:07.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.511+0000 7fcee2560700 1 -- 172.21.15.112:0/1511618174 shutdown_connections 2023-08-27T11:30:07.515 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.512+0000 7fcee2560700 1 -- 172.21.15.112:0/1511618174 wait complete. 2023-08-27T11:30:07.517 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 32 and data pool 33 2023-08-27T11:30:07.529 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:30:07.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.724+0000 7f6fc2b58700 1 Processor -- start 2023-08-27T11:30:07.725 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.724+0000 7f6fc2b58700 1 -- start start 2023-08-27T11:30:07.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fc2b58700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc15c5d0 0x7f6fbc15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:07.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fc2b58700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6fbc0ccdc0 0x7f6fbc0cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:07.727 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fc2b58700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc15fb00 con 0x7f6fbc0cc9e0 2023-08-27T11:30:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fc2b58700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc15caa0 con 0x7f6fbc0ccdc0 2023-08-27T11:30:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fc2b58700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc15cd80 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fc08f4700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6fbc0cc9e0 0x7f6fbc15bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33662/0 (socket says 172.21.15.112:33662) 2023-08-27T11:30:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fc08f4700 1 -- 172.21.15.112:0/1726116114 learned_addr learned my addr 172.21.15.112:0/1726116114 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fbbfff700 1 --2- 172.21.15.112:0/1726116114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc15c5d0 0x7f6fbc15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fc10f5700 1 --2- 172.21.15.112:0/1726116114 >> v2:172.21.15.138:3300/0 conn(0x7f6fbc0ccdc0 0x7f6fbc0cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fbb7fe700 1 -- 172.21.15.112:0/1726116114 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1775334370 0 0) 0x7f6fbc15fb00 con 0x7f6fbc0cc9e0 2023-08-27T11:30:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fbb7fe700 1 -- 172.21.15.112:0/1726116114 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fac0035b0 con 0x7f6fbc0cc9e0 2023-08-27T11:30:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fbbfff700 1 -- 172.21.15.112:0/1726116114 >> v2:172.21.15.138:3300/0 conn(0x7f6fbc0ccdc0 msgr2=0x7f6fbc0cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fbbfff700 1 --2- 172.21.15.112:0/1726116114 >> v2:172.21.15.138:3300/0 conn(0x7f6fbc0ccdc0 0x7f6fbc0cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fbbfff700 1 -- 172.21.15.112:0/1726116114 >> v1:172.21.15.112:6789/0 conn(0x7f6fbc0cc9e0 legacy=0x7f6fbc15bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.725+0000 7f6fbbfff700 1 -- 172.21.15.112:0/1726116114 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc15f4b0 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.726+0000 7f6fbbfff700 1 --2- 172.21.15.112:0/1726116114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc15c5d0 0x7f6fbc15ff90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009840 tx=0x7f6fa8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2000a2a49369d52d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.726+0000 7f6fbb7fe700 1 -- 172.21.15.112:0/1726116114 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8011830 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.726+0000 7f6fbb7fe700 1 -- 172.21.15.112:0/1726116114 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa8011990 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.731 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.726+0000 7f6fbb7fe700 1 -- 172.21.15.112:0/1726116114 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8011d20 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.726+0000 7f6fc2b58700 1 -- 172.21.15.112:0/1726116114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc15c5d0 msgr2=0x7f6fbc15ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:07.734 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.726+0000 7f6fc2b58700 1 --2- 172.21.15.112:0/1726116114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc15c5d0 0x7f6fbc15ff90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009840 tx=0x7f6fa8009b50 comp rx=0 tx=0).stop 2023-08-27T11:30:07.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.727+0000 7f6fc2b58700 1 -- 172.21.15.112:0/1726116114 shutdown_connections 2023-08-27T11:30:07.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.727+0000 7f6fc2b58700 1 --2- 172.21.15.112:0/1726116114 >> v2:172.21.15.138:3300/0 conn(0x7f6fbc0ccdc0 0x7f6fbc0cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.727+0000 7f6fc2b58700 1 --2- 172.21.15.112:0/1726116114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc15c5d0 0x7f6fbc15ff90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.727+0000 7f6fc2b58700 1 -- 172.21.15.112:0/1726116114 >> 172.21.15.112:0/1726116114 conn(0x7f6fbc066010 msgr2=0x7f6fbc0663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:07.735 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.727+0000 7f6fc2b58700 1 -- 172.21.15.112:0/1726116114 shutdown_connections 2023-08-27T11:30:07.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.727+0000 7f6fc2b58700 1 -- 172.21.15.112:0/1726116114 wait complete. 2023-08-27T11:30:07.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.728+0000 7f6fc2b58700 1 Processor -- start 2023-08-27T11:30:07.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.728+0000 7f6fc2b58700 1 -- start start 2023-08-27T11:30:07.736 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc2b58700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc0cc9e0 0x7f6fbc147920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:07.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc2b58700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fbc0ccdc0 0x7f6fbc14cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:07.737 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc2b58700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fbc15c5d0 0x7f6fbc14d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:07.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc2b58700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc151390 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc2b58700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc150780 con 0x7f6fbc0ccdc0 2023-08-27T11:30:07.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc2b58700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc150a60 con 0x7f6fbc0cc9e0 2023-08-27T11:30:07.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc10f5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fbc15c5d0 0x7f6fbc14d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:07.738 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc10f5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fbc15c5d0 0x7f6fbc14d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39290/0 (socket says 172.21.15.112:39290) 2023-08-27T11:30:07.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fbbfff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fbc0ccdc0 0x7f6fbc14cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:07.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc10f5700 1 -- 172.21.15.112:0/4147600039 learned_addr learned my addr 172.21.15.112:0/4147600039 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:07.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc08f4700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc0cc9e0 0x7f6fbc147920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:07.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc10f5700 1 -- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc0cc9e0 msgr2=0x7f6fbc147920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:07.739 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc10f5700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc0cc9e0 0x7f6fbc147920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc10f5700 1 -- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fbc0ccdc0 msgr2=0x7f6fbc14cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:07.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc10f5700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fbc0ccdc0 0x7f6fbc14cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc10f5700 1 -- 172.21.15.112:0/4147600039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa80094a0 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fbbfff700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fbc0ccdc0 0x7f6fbc14cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:30:07.740 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc10f5700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fbc15c5d0 0x7f6fbc14d080 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400efc0 tx=0x7f6fb400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:07.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fb97fa700 1 -- 172.21.15.112:0/4147600039 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4009ea0 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc2b58700 1 -- 172.21.15.112:0/4147600039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc150d40 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fc2b58700 1 -- 172.21.15.112:0/4147600039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc147e60 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.729+0000 7f6fb97fa700 1 -- 172.21.15.112:0/4147600039 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb400f040 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.741 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.730+0000 7f6fb97fa700 1 -- 172.21.15.112:0/4147600039 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400ce90 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.730+0000 7f6fc2b58700 1 -- 172.21.15.112:0/4147600039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc06a4d0 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.731+0000 7f6fb97fa700 1 -- 172.21.15.112:0/4147600039 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f6fb4014480 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.731+0000 7f6fb97fa700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6fa006a4d0 0x7f6fa006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:07.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.731+0000 7f6fb97fa700 1 -- 172.21.15.112:0/4147600039 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6fb4089620 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.742 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.731+0000 7f6fc08f4700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6fa006a4d0 0x7f6fa006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:07.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.732+0000 7f6fc08f4700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6fa006a4d0 0x7f6fa006c990 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00010e0 tx=0x7f6fb0007c70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:07.743 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.736+0000 7f6fb97fa700 1 -- 172.21.15.112:0/4147600039 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6fb40545a0 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.927 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.926+0000 7f6fc2b58700 1 -- 172.21.15.112:0/4147600039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fbc067a20 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.928 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.927+0000 7f6fb97fa700 1 -- 172.21.15.112:0/4147600039 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+18284 (secure 0 0 0) 0x7f6fb4057bc0 con 0x7f6fbc15c5d0 2023-08-27T11:30:07.929 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:30:07.929 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":183,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:30:06.890688+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:30:04.643937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-08-27T11:30:05.864973+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:30:07.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.929+0000 7f6fc2b58700 1 -- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6fa006a4d0 msgr2=0x7f6fa006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:07.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.929+0000 7f6fc2b58700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6fa006a4d0 0x7f6fa006c990 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00010e0 tx=0x7f6fb0007c70 comp rx=0 tx=0).stop 2023-08-27T11:30:07.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.929+0000 7f6fc2b58700 1 -- 172.21.15.112:0/4147600039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fbc15c5d0 msgr2=0x7f6fbc14d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:07.931 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.929+0000 7f6fc2b58700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fbc15c5d0 0x7f6fbc14d080 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400efc0 tx=0x7f6fb400c3a0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.929+0000 7f6fc2b58700 1 -- 172.21.15.112:0/4147600039 shutdown_connections 2023-08-27T11:30:07.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.929+0000 7f6fc2b58700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f6fa006a4d0 0x7f6fa006c990 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.929+0000 7f6fc2b58700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fbc15c5d0 0x7f6fbc14d080 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.930+0000 7f6fc2b58700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc0cc9e0 0x7f6fbc147920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.932 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.930+0000 7f6fc2b58700 1 --2- 172.21.15.112:0/4147600039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fbc0ccdc0 0x7f6fbc14cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:07.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.930+0000 7f6fc2b58700 1 -- 172.21.15.112:0/4147600039 >> 172.21.15.112:0/4147600039 conn(0x7f6fbc066010 msgr2=0x7f6fbc0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:07.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.930+0000 7f6fc2b58700 1 -- 172.21.15.112:0/4147600039 shutdown_connections 2023-08-27T11:30:07.933 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:07.930+0000 7f6fc2b58700 1 -- 172.21.15.112:0/4147600039 wait complete. 2023-08-27T11:30:07.943 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:30:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.061+0000 7f6888f92700 1 Processor -- start 2023-08-27T11:30:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.061+0000 7f6888f92700 1 -- start start 2023-08-27T11:30:08.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.062+0000 7f6888f92700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6884158e20 0x7f68840b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:08.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.062+0000 7f6888f92700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f688406fea0 0x7f68840af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.062+0000 7f6888f92700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f688415c190 con 0x7f688406e3a0 2023-08-27T11:30:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.062+0000 7f6888f92700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68840b1730 con 0x7f6884158e20 2023-08-27T11:30:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.062+0000 7f6888f92700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6884159200 con 0x7f688406fea0 2023-08-27T11:30:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.062+0000 7f688259c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6884158e20 0x7f68840b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.063+0000 7f688259c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6884158e20 0x7f68840b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39294/0 (socket says 172.21.15.112:39294) 2023-08-27T11:30:08.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.063+0000 7f688259c700 1 -- 172.21.15.112:0/2437946804 learned_addr learned my addr 172.21.15.112:0/2437946804 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:08.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.063+0000 7f6882d9d700 1 --2- 172.21.15.112:0/2437946804 >> v2:172.21.15.138:3300/0 conn(0x7f688406fea0 0x7f68840af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:08.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.063+0000 7f688259c700 1 -- 172.21.15.112:0/2437946804 >> v2:172.21.15.138:3300/0 conn(0x7f688406fea0 msgr2=0x7f68840af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:08.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.063+0000 7f688259c700 1 --2- 172.21.15.112:0/2437946804 >> v2:172.21.15.138:3300/0 conn(0x7f688406fea0 0x7f68840af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:08.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.063+0000 7f688259c700 1 -- 172.21.15.112:0/2437946804 >> v1:172.21.15.138:6789/0 conn(0x7f688406e3a0 legacy=0x7f688406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.063+0000 7f688259c700 1 -- 172.21.15.112:0/2437946804 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68840b1910 con 0x7f6884158e20 2023-08-27T11:30:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.063+0000 7f688259c700 1 --2- 172.21.15.112:0/2437946804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6884158e20 0x7f68840b11f0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f688406dba0 tx=0x7f6878009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effcd5214029887d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.063+0000 7f688159a700 1 -- 172.21.15.112:0/2437946804 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687800b870 con 0x7f6884158e20 2023-08-27T11:30:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.063+0000 7f688159a700 1 -- 172.21.15.112:0/2437946804 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687800b9d0 con 0x7f6884158e20 2023-08-27T11:30:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.063+0000 7f688159a700 1 -- 172.21.15.112:0/2437946804 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687800bd60 con 0x7f6884158e20 2023-08-27T11:30:08.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.064+0000 7f6888f92700 1 -- 172.21.15.112:0/2437946804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6884158e20 msgr2=0x7f68840b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:08.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.064+0000 7f6888f92700 1 --2- 172.21.15.112:0/2437946804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6884158e20 0x7f68840b11f0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f688406dba0 tx=0x7f6878009bd0 comp rx=0 tx=0).stop 2023-08-27T11:30:08.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.064+0000 7f6888f92700 1 -- 172.21.15.112:0/2437946804 shutdown_connections 2023-08-27T11:30:08.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.064+0000 7f6888f92700 1 --2- 172.21.15.112:0/2437946804 >> v2:172.21.15.138:3300/0 conn(0x7f688406fea0 0x7f68840af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:08.069 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.064+0000 7f6888f92700 1 --2- 172.21.15.112:0/2437946804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6884158e20 0x7f68840b11f0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.064+0000 7f6888f92700 1 -- 172.21.15.112:0/2437946804 >> 172.21.15.112:0/2437946804 conn(0x7f68840d0000 msgr2=0x7f68840d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.064+0000 7f6888f92700 1 -- 172.21.15.112:0/2437946804 shutdown_connections 2023-08-27T11:30:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.064+0000 7f6888f92700 1 -- 172.21.15.112:0/2437946804 wait complete. 2023-08-27T11:30:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.065+0000 7f6888f92700 1 Processor -- start 2023-08-27T11:30:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.065+0000 7f6888f92700 1 -- start start 2023-08-27T11:30:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f6888f92700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688406e3a0 0x7f68840ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f6888f92700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f688406fea0 0x7f68840c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f6888f92700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6884158e20 0x7f6884154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f6888f92700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68840c9720 con 0x7f688406e3a0 2023-08-27T11:30:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f6888f92700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68840c8310 con 0x7f6884158e20 2023-08-27T11:30:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f6888f92700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68840c9f30 con 0x7f688406fea0 2023-08-27T11:30:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f688259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688406e3a0 0x7f68840ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f688259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688406e3a0 0x7f68840ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39302/0 (socket says 172.21.15.112:39302) 2023-08-27T11:30:08.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f688259c700 1 -- 172.21.15.112:0/2188528455 learned_addr learned my addr 172.21.15.112:0/2188528455 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:08.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f6882d9d700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6884158e20 0x7f6884154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:08.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f6881d9b700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f688406fea0 0x7f68840c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:08.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f688259c700 1 -- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f688406fea0 msgr2=0x7f68840c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:08.075 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f688259c700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f688406fea0 0x7f68840c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:08.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f688259c700 1 -- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6884158e20 msgr2=0x7f6884154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:08.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f688259c700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6884158e20 0x7f6884154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:08.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f688259c700 1 -- 172.21.15.112:0/2188528455 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6878009550 con 0x7f688406e3a0 2023-08-27T11:30:08.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f6881d9b700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f688406fea0 0x7f68840c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:08.076 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f688259c700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688406e3a0 0x7f68840ac8a0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f68840d0f90 tx=0x7f6878004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.066+0000 7f6872ffd700 1 -- 172.21.15.112:0/2188528455 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878012940 con 0x7f688406e3a0 2023-08-27T11:30:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.067+0000 7f6872ffd700 1 -- 172.21.15.112:0/2188528455 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687801e070 con 0x7f688406e3a0 2023-08-27T11:30:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.067+0000 7f6888f92700 1 -- 172.21.15.112:0/2188528455 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6884154ab0 con 0x7f688406e3a0 2023-08-27T11:30:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.067+0000 7f6872ffd700 1 -- 172.21.15.112:0/2188528455 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878012cc0 con 0x7f688406e3a0 2023-08-27T11:30:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.067+0000 7f6888f92700 1 -- 172.21.15.112:0/2188528455 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68840c9c50 con 0x7f688406e3a0 2023-08-27T11:30:08.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.068+0000 7f6888f92700 1 -- 172.21.15.112:0/2188528455 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6884067a20 con 0x7f688406e3a0 2023-08-27T11:30:08.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.068+0000 7f6872ffd700 1 -- 172.21.15.112:0/2188528455 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f687801a420 con 0x7f688406e3a0 2023-08-27T11:30:08.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.069+0000 7f6872ffd700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f686406a4d0 0x7f686406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:08.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.069+0000 7f6872ffd700 1 -- 172.21.15.112:0/2188528455 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6878089c40 con 0x7f688406e3a0 2023-08-27T11:30:08.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.069+0000 7f6881d9b700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f686406a4d0 0x7f686406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.069+0000 7f6881d9b700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f686406a4d0 0x7f686406c990 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f68840c3d10 tx=0x7f6874009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.073+0000 7f6872ffd700 1 -- 172.21.15.112:0/2188528455 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6878054ef0 con 0x7f688406e3a0 2023-08-27T11:30:08.263 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.262+0000 7f6888f92700 1 -- 172.21.15.112:0/2188528455 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6884073c10 con 0x7f688406e3a0 2023-08-27T11:30:08.264 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.263+0000 7f6872ffd700 1 -- 172.21.15.112:0/2188528455 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+18284 (secure 0 0 0) 0x7f6878020090 con 0x7f688406e3a0 2023-08-27T11:30:08.264 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:30:08.265 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":183,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:30:06.890688+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:30:04.643937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-08-27T11:30:05.864973+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:30:08.266 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.265+0000 7f6888f92700 1 -- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f686406a4d0 msgr2=0x7f686406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:08.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.265+0000 7f6888f92700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f686406a4d0 0x7f686406c990 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f68840c3d10 tx=0x7f6874009040 comp rx=0 tx=0).stop 2023-08-27T11:30:08.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.265+0000 7f6888f92700 1 -- 172.21.15.112:0/2188528455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688406e3a0 msgr2=0x7f68840ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:08.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.265+0000 7f6888f92700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688406e3a0 0x7f68840ac8a0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f68840d0f90 tx=0x7f6878004750 comp rx=0 tx=0).stop 2023-08-27T11:30:08.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.265+0000 7f6888f92700 1 -- 172.21.15.112:0/2188528455 shutdown_connections 2023-08-27T11:30:08.267 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.265+0000 7f6888f92700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f686406a4d0 0x7f686406c990 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:08.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.265+0000 7f6888f92700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6884158e20 0x7f6884154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:08.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.265+0000 7f6888f92700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688406e3a0 0x7f68840ac8a0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:08.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.265+0000 7f6888f92700 1 --2- 172.21.15.112:0/2188528455 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f688406fea0 0x7f68840c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:08.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.265+0000 7f6888f92700 1 -- 172.21.15.112:0/2188528455 >> 172.21.15.112:0/2188528455 conn(0x7f68840d0000 msgr2=0x7f688406b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:08.268 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.265+0000 7f6888f92700 1 -- 172.21.15.112:0/2188528455 shutdown_connections 2023-08-27T11:30:08.269 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.265+0000 7f6888f92700 1 -- 172.21.15.112:0/2188528455 wait complete. 2023-08-27T11:30:08.279 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:30:08.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.402+0000 7f297b6a1700 1 Processor -- start 2023-08-27T11:30:08.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.402+0000 7f297b6a1700 1 -- start start 2023-08-27T11:30:08.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f297b6a1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2974146c20 0x7f2974160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:08.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f297b6a1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29740b5070 con 0x7f297406f790 2023-08-27T11:30:08.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f297b6a1700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2974147000 con 0x7f29740b3440 2023-08-27T11:30:08.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f297b6a1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29741614a0 con 0x7f2974146c20 2023-08-27T11:30:08.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f2979c3e700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f297406f790 0x7f29740b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33694/0 (socket says 172.21.15.112:33694) 2023-08-27T11:30:08.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f2979c3e700 1 -- 172.21.15.112:0/3202658172 learned_addr learned my addr 172.21.15.112:0/3202658172 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:08.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f297943d700 1 --2- 172.21.15.112:0/3202658172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2974146c20 0x7f2974160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:08.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f296bfff700 1 -- 172.21.15.112:0/3202658172 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347153148 0 0) 0x7f296c001490 con 0x7f297406f790 2023-08-27T11:30:08.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f296bfff700 1 -- 172.21.15.112:0/3202658172 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29600035b0 con 0x7f297406f790 2023-08-27T11:30:08.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f296bfff700 1 -- 172.21.15.112:0/3202658172 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738423929 0 0) 0x7f2974147000 con 0x7f29740b3440 2023-08-27T11:30:08.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f296bfff700 1 -- 172.21.15.112:0/3202658172 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f296c001490 con 0x7f29740b3440 2023-08-27T11:30:08.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f297943d700 1 -- 172.21.15.112:0/3202658172 >> v1:172.21.15.138:6789/0 conn(0x7f29740b3440 legacy=0x7f297415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:08.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f297943d700 1 -- 172.21.15.112:0/3202658172 >> v1:172.21.15.112:6789/0 conn(0x7f297406f790 legacy=0x7f29740b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:08.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f297943d700 1 -- 172.21.15.112:0/3202658172 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2974161740 con 0x7f2974146c20 2023-08-27T11:30:08.408 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.403+0000 7f297943d700 1 --2- 172.21.15.112:0/3202658172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2974146c20 0x7f2974160f60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2964009870 tx=0x7f2964009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c283345d5dbc11e4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:08.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.404+0000 7f296bfff700 1 -- 172.21.15.112:0/3202658172 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964011830 con 0x7f2974146c20 2023-08-27T11:30:08.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.404+0000 7f296bfff700 1 -- 172.21.15.112:0/3202658172 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2964011990 con 0x7f2974146c20 2023-08-27T11:30:08.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.404+0000 7f296bfff700 1 -- 172.21.15.112:0/3202658172 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964011d20 con 0x7f2974146c20 2023-08-27T11:30:08.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.404+0000 7f297b6a1700 1 -- 172.21.15.112:0/3202658172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2974146c20 msgr2=0x7f2974160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:08.409 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.404+0000 7f297b6a1700 1 --2- 172.21.15.112:0/3202658172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2974146c20 0x7f2974160f60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2964009870 tx=0x7f2964009b80 comp rx=0 tx=0).stop 2023-08-27T11:30:08.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.404+0000 7f297b6a1700 1 -- 172.21.15.112:0/3202658172 shutdown_connections 2023-08-27T11:30:08.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.404+0000 7f297b6a1700 1 --2- 172.21.15.112:0/3202658172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2974146c20 0x7f2974160f60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:08.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.404+0000 7f297b6a1700 1 -- 172.21.15.112:0/3202658172 >> 172.21.15.112:0/3202658172 conn(0x7f29740b1e90 msgr2=0x7f29740b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:08.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.404+0000 7f297b6a1700 1 -- 172.21.15.112:0/3202658172 shutdown_connections 2023-08-27T11:30:08.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.404+0000 7f297b6a1700 1 -- 172.21.15.112:0/3202658172 wait complete. 2023-08-27T11:30:08.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.405+0000 7f297b6a1700 1 Processor -- start 2023-08-27T11:30:08.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.406+0000 7f297b6a1700 1 -- start start 2023-08-27T11:30:08.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.406+0000 7f297b6a1700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f297406f790 0x7f29740ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:08.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.406+0000 7f297b6a1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29740b3440 0x7f2974133fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:08.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.406+0000 7f297b6a1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2974146c20 0x7f29741f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:08.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.406+0000 7f297b6a1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29741335c0 con 0x7f2974146c20 2023-08-27T11:30:08.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.406+0000 7f297b6a1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29740ceff0 con 0x7f29740b3440 2023-08-27T11:30:08.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.406+0000 7f297b6a1700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2974134510 con 0x7f297406f790 2023-08-27T11:30:08.414 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.406+0000 7f2979c3e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2974146c20 0x7f29741f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:08.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.406+0000 7f2979c3e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2974146c20 0x7f29741f2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39318/0 (socket says 172.21.15.112:39318) 2023-08-27T11:30:08.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.406+0000 7f2979c3e700 1 -- 172.21.15.112:0/3581573612 learned_addr learned my addr 172.21.15.112:0/3581573612 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:08.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f297943d700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f297406f790 0x7f29740ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:08.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f2978c3c700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29740b3440 0x7f2974133fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:08.415 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f2979c3e700 1 -- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f297406f790 msgr2=0x7f29740ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:08.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f2979c3e700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f297406f790 0x7f29740ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:08.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f2979c3e700 1 -- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29740b3440 msgr2=0x7f2974133fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:08.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f2979c3e700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29740b3440 0x7f2974133fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:08.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f2979c3e700 1 -- 172.21.15.112:0/3581573612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29640094a0 con 0x7f2974146c20 2023-08-27T11:30:08.416 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f2978c3c700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29740b3440 0x7f2974133fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f2979c3e700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2974146c20 0x7f29741f2f20 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f296c001f50 tx=0x7f296c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f2969ffb700 1 -- 172.21.15.112:0/3581573612 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c00ad80 con 0x7f2974146c20 2023-08-27T11:30:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f2969ffb700 1 -- 172.21.15.112:0/3581573612 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c012400 con 0x7f2974146c20 2023-08-27T11:30:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f297b6a1700 1 -- 172.21.15.112:0/3581573612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29741f39c0 con 0x7f2974146c20 2023-08-27T11:30:08.417 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f2969ffb700 1 -- 172.21.15.112:0/3581573612 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c012770 con 0x7f2974146c20 2023-08-27T11:30:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.407+0000 7f297b6a1700 1 -- 172.21.15.112:0/3581573612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29741337d0 con 0x7f2974146c20 2023-08-27T11:30:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.408+0000 7f297b6a1700 1 -- 172.21.15.112:0/3581573612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2974067e90 con 0x7f2974146c20 2023-08-27T11:30:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.408+0000 7f2969ffb700 1 -- 172.21.15.112:0/3581573612 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f296c01b3b0 con 0x7f2974146c20 2023-08-27T11:30:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.409+0000 7f2969ffb700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f295806a4d0 0x7f295806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.409+0000 7f2969ffb700 1 -- 172.21.15.112:0/3581573612 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f296c0887b0 con 0x7f2974146c20 2023-08-27T11:30:08.418 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.409+0000 7f297943d700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f295806a4d0 0x7f295806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:08.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.410+0000 7f297943d700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f295806a4d0 0x7f295806c990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f29740720d0 tx=0x7f296401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:08.419 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.413+0000 7f2969ffb700 1 -- 172.21.15.112:0/3581573612 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f296c053a70 con 0x7f2974146c20 2023-08-27T11:30:08.650 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:08.649+0000 7f297b6a1700 1 -- 172.21.15.112:0/3581573612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f29740b5070 con 0x7f2974146c20 2023-08-27T11:30:09.524 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.523+0000 7f2969ffb700 1 -- 172.21.15.112:0/3581573612 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v124) v1 ==== 118+0+0 (secure 0 0 0) 0x7f29740b5070 con 0x7f2974146c20 2023-08-27T11:30:09.527 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.526+0000 7f297b6a1700 1 -- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f295806a4d0 msgr2=0x7f295806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:09.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.526+0000 7f297b6a1700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f295806a4d0 0x7f295806c990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f29740720d0 tx=0x7f296401c040 comp rx=0 tx=0).stop 2023-08-27T11:30:09.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.526+0000 7f297b6a1700 1 -- 172.21.15.112:0/3581573612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2974146c20 msgr2=0x7f29741f2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:09.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.526+0000 7f297b6a1700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2974146c20 0x7f29741f2f20 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f296c001f50 tx=0x7f296c00a340 comp rx=0 tx=0).stop 2023-08-27T11:30:09.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.526+0000 7f297b6a1700 1 -- 172.21.15.112:0/3581573612 shutdown_connections 2023-08-27T11:30:09.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.526+0000 7f297b6a1700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f295806a4d0 0x7f295806c990 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:09.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.526+0000 7f297b6a1700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2974146c20 0x7f29741f2f20 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:09.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.526+0000 7f297b6a1700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f297406f790 0x7f29740ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:09.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.526+0000 7f297b6a1700 1 --2- 172.21.15.112:0/3581573612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29740b3440 0x7f2974133fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:09.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.526+0000 7f297b6a1700 1 -- 172.21.15.112:0/3581573612 >> 172.21.15.112:0/3581573612 conn(0x7f29740b1e90 msgr2=0x7f297415d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.526+0000 7f297b6a1700 1 -- 172.21.15.112:0/3581573612 shutdown_connections 2023-08-27T11:30:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.526+0000 7f297b6a1700 1 -- 172.21.15.112:0/3581573612 wait complete. 2023-08-27T11:30:09.542 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:30:09.543 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:30:09.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.671+0000 7ff6e738a700 1 Processor -- start 2023-08-27T11:30:09.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.671+0000 7ff6e738a700 1 -- start start 2023-08-27T11:30:09.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.672+0000 7ff6e738a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6e0158da0 0x7ff6e015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:09.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.672+0000 7ff6e738a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e00b64a0 con 0x7ff6e015f2a0 2023-08-27T11:30:09.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.672+0000 7ff6e738a700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e0159180 con 0x7ff6e00b6930 2023-08-27T11:30:09.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.672+0000 7ff6e738a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e015f820 con 0x7ff6e0158da0 2023-08-27T11:30:09.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.672+0000 7ff6e5126700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6e0158da0 0x7ff6e015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:09.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.672+0000 7ff6e4925700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff6e015f2a0 0x7ff6e015c930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33704/0 (socket says 172.21.15.112:33704) 2023-08-27T11:30:09.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.672+0000 7ff6e4925700 1 -- 172.21.15.112:0/2483848156 learned_addr learned my addr 172.21.15.112:0/2483848156 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:09.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.672+0000 7ff6e5126700 1 -- 172.21.15.112:0/2483848156 >> v1:172.21.15.138:6789/0 conn(0x7ff6e00b6930 legacy=0x7ff6e00b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:09.676 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.672+0000 7ff6e5126700 1 -- 172.21.15.112:0/2483848156 >> v1:172.21.15.112:6789/0 conn(0x7ff6e015f2a0 legacy=0x7ff6e015c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:09.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.672+0000 7ff6d7fff700 1 -- 172.21.15.112:0/2483848156 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200829100 0 0) 0x7ff6e00b64a0 con 0x7ff6e015f2a0 2023-08-27T11:30:09.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.672+0000 7ff6e5126700 1 -- 172.21.15.112:0/2483848156 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e015fb00 con 0x7ff6e0158da0 2023-08-27T11:30:09.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.673+0000 7ff6e5126700 1 --2- 172.21.15.112:0/2483848156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e0158da0 0x7ff6e015ed60 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009840 tx=0x7ff6d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b63d355e0016cca server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:09.677 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.673+0000 7ff6d7fff700 1 -- 172.21.15.112:0/2483848156 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0004230 con 0x7ff6e0158da0 2023-08-27T11:30:09.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.673+0000 7ff6d7fff700 1 -- 172.21.15.112:0/2483848156 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d0004390 con 0x7ff6e0158da0 2023-08-27T11:30:09.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.673+0000 7ff6d7fff700 1 -- 172.21.15.112:0/2483848156 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d00036a0 con 0x7ff6e0158da0 2023-08-27T11:30:09.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.673+0000 7ff6e738a700 1 -- 172.21.15.112:0/2483848156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e0158da0 msgr2=0x7ff6e015ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:09.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.673+0000 7ff6e738a700 1 --2- 172.21.15.112:0/2483848156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e0158da0 0x7ff6e015ed60 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009840 tx=0x7ff6d0009b50 comp rx=0 tx=0).stop 2023-08-27T11:30:09.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.673+0000 7ff6e738a700 1 -- 172.21.15.112:0/2483848156 shutdown_connections 2023-08-27T11:30:09.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.673+0000 7ff6e738a700 1 --2- 172.21.15.112:0/2483848156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e0158da0 0x7ff6e015ed60 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:09.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.673+0000 7ff6e738a700 1 -- 172.21.15.112:0/2483848156 >> 172.21.15.112:0/2483848156 conn(0x7ff6e0066010 msgr2=0x7ff6e00663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:09.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.673+0000 7ff6e738a700 1 -- 172.21.15.112:0/2483848156 shutdown_connections 2023-08-27T11:30:09.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.673+0000 7ff6e738a700 1 -- 172.21.15.112:0/2483848156 wait complete. 2023-08-27T11:30:09.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.674+0000 7ff6e738a700 1 Processor -- start 2023-08-27T11:30:09.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.675+0000 7ff6e738a700 1 -- start start 2023-08-27T11:30:09.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.675+0000 7ff6e738a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e00b6930 0x7ff6e0149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:09.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.675+0000 7ff6e738a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6e0158da0 0x7ff6e00c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:09.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.675+0000 7ff6e738a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6e015f2a0 0x7ff6e00c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:09.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.675+0000 7ff6e738a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e00c7f30 con 0x7ff6e00b6930 2023-08-27T11:30:09.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.675+0000 7ff6e738a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0149650 con 0x7ff6e0158da0 2023-08-27T11:30:09.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.675+0000 7ff6e738a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0157740 con 0x7ff6e015f2a0 2023-08-27T11:30:09.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.675+0000 7ff6e5126700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e00b6930 0x7ff6e0149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:09.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.675+0000 7ff6e5126700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e00b6930 0x7ff6e0149110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39328/0 (socket says 172.21.15.112:39328) 2023-08-27T11:30:09.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.675+0000 7ff6e5126700 1 -- 172.21.15.112:0/1330247252 learned_addr learned my addr 172.21.15.112:0/1330247252 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.675+0000 7ff6e4925700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6e0158da0 0x7ff6e00c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.676+0000 7ff6e5927700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6e015f2a0 0x7ff6e00c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.676+0000 7ff6e5126700 1 -- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6e015f2a0 msgr2=0x7ff6e00c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.676+0000 7ff6e5126700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6e015f2a0 0x7ff6e00c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.676+0000 7ff6e5126700 1 -- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6e0158da0 msgr2=0x7ff6e00c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.676+0000 7ff6e5126700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6e0158da0 0x7ff6e00c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.676+0000 7ff6e5126700 1 -- 172.21.15.112:0/1330247252 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d00094a0 con 0x7ff6e00b6930 2023-08-27T11:30:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.676+0000 7ff6e5927700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6e015f2a0 0x7ff6e00c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.676+0000 7ff6e5126700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e00b6930 0x7ff6e0149110 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0000c00 tx=0x7ff6d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.676+0000 7ff6d5ffb700 1 -- 172.21.15.112:0/1330247252 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0011630 con 0x7ff6e00b6930 2023-08-27T11:30:09.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.676+0000 7ff6d5ffb700 1 -- 172.21.15.112:0/1330247252 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d001e070 con 0x7ff6e00b6930 2023-08-27T11:30:09.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.676+0000 7ff6e738a700 1 -- 172.21.15.112:0/1330247252 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e01566a0 con 0x7ff6e00b6930 2023-08-27T11:30:09.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.676+0000 7ff6d5ffb700 1 -- 172.21.15.112:0/1330247252 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d00119f0 con 0x7ff6e00b6930 2023-08-27T11:30:09.688 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.676+0000 7ff6e738a700 1 -- 172.21.15.112:0/1330247252 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e0156c60 con 0x7ff6e00b6930 2023-08-27T11:30:09.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.677+0000 7ff6e738a700 1 -- 172.21.15.112:0/1330247252 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e0067a20 con 0x7ff6e00b6930 2023-08-27T11:30:09.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.678+0000 7ff6d5ffb700 1 -- 172.21.15.112:0/1330247252 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7ff6d0011b50 con 0x7ff6e00b6930 2023-08-27T11:30:09.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.678+0000 7ff6d5ffb700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff6c406a4d0 0x7ff6c406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:09.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.678+0000 7ff6d5ffb700 1 -- 172.21.15.112:0/1330247252 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff6d0089f10 con 0x7ff6e00b6930 2023-08-27T11:30:09.689 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.678+0000 7ff6e4925700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff6c406a4d0 0x7ff6c406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:09.690 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.679+0000 7ff6e4925700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff6c406a4d0 0x7ff6c406c990 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00d1310 tx=0x7ff6dc007d90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.682+0000 7ff6d5ffb700 1 -- 172.21.15.112:0/1330247252 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff6d00551c0 con 0x7ff6e00b6930 2023-08-27T11:30:09.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:09.913+0000 7ff6e738a700 1 -- 172.21.15.112:0/1330247252 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff6e00cf1b0 con 0x7ff6e00b6930 2023-08-27T11:30:10.529 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.528+0000 7ff6d5ffb700 1 -- 172.21.15.112:0/1330247252 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v125) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff6d0020020 con 0x7ff6e00b6930 2023-08-27T11:30:10.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.531+0000 7ff6e738a700 1 -- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff6c406a4d0 msgr2=0x7ff6c406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.531+0000 7ff6e738a700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff6c406a4d0 0x7ff6c406c990 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00d1310 tx=0x7ff6dc007d90 comp rx=0 tx=0).stop 2023-08-27T11:30:10.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.531+0000 7ff6e738a700 1 -- 172.21.15.112:0/1330247252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e00b6930 msgr2=0x7ff6e0149110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.531+0000 7ff6e738a700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e00b6930 0x7ff6e0149110 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0000c00 tx=0x7ff6d0004750 comp rx=0 tx=0).stop 2023-08-27T11:30:10.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.531+0000 7ff6e738a700 1 -- 172.21.15.112:0/1330247252 shutdown_connections 2023-08-27T11:30:10.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.531+0000 7ff6e738a700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7ff6c406a4d0 0x7ff6c406c990 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.531+0000 7ff6e738a700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6e015f2a0 0x7ff6e00c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.531+0000 7ff6e738a700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e00b6930 0x7ff6e0149110 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.531+0000 7ff6e738a700 1 --2- 172.21.15.112:0/1330247252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6e0158da0 0x7ff6e00c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.531+0000 7ff6e738a700 1 -- 172.21.15.112:0/1330247252 >> 172.21.15.112:0/1330247252 conn(0x7ff6e0066010 msgr2=0x7ff6e00b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.531+0000 7ff6e738a700 1 -- 172.21.15.112:0/1330247252 shutdown_connections 2023-08-27T11:30:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.531+0000 7ff6e738a700 1 -- 172.21.15.112:0/1330247252 wait complete. 2023-08-27T11:30:10.545 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:30:10.545 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:30:10.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.667+0000 7fec5d63a700 1 Processor -- start 2023-08-27T11:30:10.668 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.667+0000 7fec5d63a700 1 -- start start 2023-08-27T11:30:10.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.667+0000 7fec5d63a700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fec580ce260 0x7fec5806fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.670 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.667+0000 7fec5d63a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fec5806ff70 0x7fec5806cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.667+0000 7fec5d63a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec5806cb00 con 0x7fec5813f910 2023-08-27T11:30:10.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.667+0000 7fec5d63a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec580707a0 con 0x7fec5806ff70 2023-08-27T11:30:10.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.667+0000 7fec5d63a700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec58070a80 con 0x7fec580ce260 2023-08-27T11:30:10.671 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.667+0000 7fec577fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fec5806ff70 0x7fec5806cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.668+0000 7fec56ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fec5813f910 0x7fec580cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33718/0 (socket says 172.21.15.112:33718) 2023-08-27T11:30:10.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.668+0000 7fec56ffd700 1 -- 172.21.15.112:0/1760049522 learned_addr learned my addr 172.21.15.112:0/1760049522 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:10.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.668+0000 7fec567fc700 1 --2- 172.21.15.112:0/1760049522 >> v2:172.21.15.138:3300/0 conn(0x7fec580ce260 0x7fec5806fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.668+0000 7fec577fe700 1 -- 172.21.15.112:0/1760049522 >> v2:172.21.15.138:3300/0 conn(0x7fec580ce260 msgr2=0x7fec5806fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:10.672 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.668+0000 7fec577fe700 1 --2- 172.21.15.112:0/1760049522 >> v2:172.21.15.138:3300/0 conn(0x7fec580ce260 0x7fec5806fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.668+0000 7fec577fe700 1 -- 172.21.15.112:0/1760049522 >> v1:172.21.15.112:6789/0 conn(0x7fec5813f910 legacy=0x7fec580cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.668+0000 7fec577fe700 1 -- 172.21.15.112:0/1760049522 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec58070d60 con 0x7fec5806ff70 2023-08-27T11:30:10.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.668+0000 7fec567fc700 1 --2- 172.21.15.112:0/1760049522 >> v2:172.21.15.138:3300/0 conn(0x7fec580ce260 0x7fec5806fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:10.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.668+0000 7fec577fe700 1 --2- 172.21.15.112:0/1760049522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5806ff70 0x7fec5806cf90 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fec58072460 tx=0x7fec5000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9ee241a7b694bc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:10.673 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.668+0000 7fec55ffb700 1 -- 172.21.15.112:0/1760049522 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec50010070 con 0x7fec5806ff70 2023-08-27T11:30:10.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.668+0000 7fec55ffb700 1 -- 172.21.15.112:0/1760049522 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5000b2e0 con 0x7fec5806ff70 2023-08-27T11:30:10.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.668+0000 7fec55ffb700 1 -- 172.21.15.112:0/1760049522 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5000dcb0 con 0x7fec5806ff70 2023-08-27T11:30:10.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.669+0000 7fec5d63a700 1 -- 172.21.15.112:0/1760049522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5806ff70 msgr2=0x7fec5806cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.669+0000 7fec5d63a700 1 --2- 172.21.15.112:0/1760049522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5806ff70 0x7fec5806cf90 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fec58072460 tx=0x7fec5000d330 comp rx=0 tx=0).stop 2023-08-27T11:30:10.674 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.669+0000 7fec5d63a700 1 -- 172.21.15.112:0/1760049522 shutdown_connections 2023-08-27T11:30:10.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.669+0000 7fec5d63a700 1 --2- 172.21.15.112:0/1760049522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5806ff70 0x7fec5806cf90 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.669+0000 7fec5d63a700 1 --2- 172.21.15.112:0/1760049522 >> v2:172.21.15.138:3300/0 conn(0x7fec580ce260 0x7fec5806fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.669+0000 7fec5d63a700 1 -- 172.21.15.112:0/1760049522 >> 172.21.15.112:0/1760049522 conn(0x7fec580b0bf0 msgr2=0x7fec580b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:10.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.669+0000 7fec5d63a700 1 -- 172.21.15.112:0/1760049522 shutdown_connections 2023-08-27T11:30:10.675 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.669+0000 7fec5d63a700 1 -- 172.21.15.112:0/1760049522 wait complete. 2023-08-27T11:30:10.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.670+0000 7fec5d63a700 1 Processor -- start 2023-08-27T11:30:10.678 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.670+0000 7fec5d63a700 1 -- start start 2023-08-27T11:30:10.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec5d63a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5806ff70 0x7fec58158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec5d63a700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec580ce260 0x7fec581e3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.679 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec5d63a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5813f910 0x7fec581394a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec5d63a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec581e49f0 con 0x7fec5806ff70 2023-08-27T11:30:10.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec5d63a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec58158ad0 con 0x7fec5813f910 2023-08-27T11:30:10.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec5d63a700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec581e3ed0 con 0x7fec580ce260 2023-08-27T11:30:10.680 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec56ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5806ff70 0x7fec58158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec56ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5806ff70 0x7fec58158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39342/0 (socket says 172.21.15.112:39342) 2023-08-27T11:30:10.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec56ffd700 1 -- 172.21.15.112:0/1800376599 learned_addr learned my addr 172.21.15.112:0/1800376599 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:10.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec577fe700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5813f910 0x7fec581394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec567fc700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec580ce260 0x7fec581e3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.681 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec56ffd700 1 -- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec580ce260 msgr2=0x7fec581e3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec56ffd700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec580ce260 0x7fec581e3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec56ffd700 1 -- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5813f910 msgr2=0x7fec581394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec56ffd700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5813f910 0x7fec581394a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec56ffd700 1 -- 172.21.15.112:0/1800376599 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5000bc80 con 0x7fec5806ff70 2023-08-27T11:30:10.682 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec567fc700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec580ce260 0x7fec581e3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:10.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec56ffd700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5806ff70 0x7fec58158590 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00c520 tx=0x7fec4c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:10.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.671+0000 7fec3f7fe700 1 -- 172.21.15.112:0/1800376599 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c005d70 con 0x7fec5806ff70 2023-08-27T11:30:10.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.672+0000 7fec3f7fe700 1 -- 172.21.15.112:0/1800376599 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4c0070d0 con 0x7fec5806ff70 2023-08-27T11:30:10.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.672+0000 7fec5d63a700 1 -- 172.21.15.112:0/1800376599 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5815ab70 con 0x7fec5806ff70 2023-08-27T11:30:10.683 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.672+0000 7fec3f7fe700 1 -- 172.21.15.112:0/1800376599 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c007440 con 0x7fec5806ff70 2023-08-27T11:30:10.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.672+0000 7fec5d63a700 1 -- 172.21.15.112:0/1800376599 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec581e4430 con 0x7fec5806ff70 2023-08-27T11:30:10.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.672+0000 7fec5d63a700 1 -- 172.21.15.112:0/1800376599 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec58159b10 con 0x7fec5806ff70 2023-08-27T11:30:10.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.673+0000 7fec3f7fe700 1 -- 172.21.15.112:0/1800376599 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fec4c0075a0 con 0x7fec5806ff70 2023-08-27T11:30:10.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.673+0000 7fec3f7fe700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fec4406a4d0 0x7fec4406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.673+0000 7fec3f7fe700 1 -- 172.21.15.112:0/1800376599 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fec4c087f60 con 0x7fec5806ff70 2023-08-27T11:30:10.684 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.674+0000 7fec567fc700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fec4406a4d0 0x7fec4406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.674+0000 7fec567fc700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fec4406a4d0 0x7fec4406c990 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fec5806ca40 tx=0x7fec48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:10.685 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.678+0000 7fec3f7fe700 1 -- 172.21.15.112:0/1800376599 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fec4c0531b0 con 0x7fec5806ff70 2023-08-27T11:30:10.913 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.912+0000 7fec5d63a700 1 -- 172.21.15.112:0/1800376599 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec5815a9d0 con 0x7fec5806ff70 2023-08-27T11:30:10.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.913+0000 7fec3f7fe700 1 -- 172.21.15.112:0/1800376599 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4897 (secure 0 0 0) 0x7fec5815a9d0 con 0x7fec5806ff70 2023-08-27T11:30:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:30:10.915 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":125,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1535178300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1535178300},{"type":"v1","addr":"172.21.15.138:6836","nonce":1535178300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9933,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3076680959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3076680959},{"type":"v1","addr":"172.21.15.112:6834","nonce":3076680959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9936,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3090607154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3090607154},{"type":"v1","addr":"172.21.15.112:6835","nonce":3090607154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:30:06.890996+0000","modified":"2023-08-27T11:30:10.523324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9948":{"gid":9948,"name":"d","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/3884278029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3884278029},{"type":"v1","addr":"172.21.15.138:6837","nonce":3884278029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-08-27T11:30:10.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.915+0000 7fec5d63a700 1 -- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fec4406a4d0 msgr2=0x7fec4406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.917 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.915+0000 7fec5d63a700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fec4406a4d0 0x7fec4406c990 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fec5806ca40 tx=0x7fec48008040 comp rx=0 tx=0).stop 2023-08-27T11:30:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.915+0000 7fec5d63a700 1 -- 172.21.15.112:0/1800376599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5806ff70 msgr2=0x7fec58158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.915+0000 7fec5d63a700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5806ff70 0x7fec58158590 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00c520 tx=0x7fec4c00c830 comp rx=0 tx=0).stop 2023-08-27T11:30:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.915+0000 7fec5d63a700 1 -- 172.21.15.112:0/1800376599 shutdown_connections 2023-08-27T11:30:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.915+0000 7fec5d63a700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fec4406a4d0 0x7fec4406c990 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.915+0000 7fec5d63a700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5813f910 0x7fec581394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.915+0000 7fec5d63a700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5806ff70 0x7fec58158590 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.915+0000 7fec5d63a700 1 --2- 172.21.15.112:0/1800376599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec580ce260 0x7fec581e3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.915+0000 7fec5d63a700 1 -- 172.21.15.112:0/1800376599 >> 172.21.15.112:0/1800376599 conn(0x7fec580b0bf0 msgr2=0x7fec58140c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:10.919 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.915+0000 7fec5d63a700 1 -- 172.21.15.112:0/1800376599 shutdown_connections 2023-08-27T11:30:10.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:10.915+0000 7fec5d63a700 1 -- 172.21.15.112:0/1800376599 wait complete. 2023-08-27T11:30:10.920 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 125 2023-08-27T11:30:10.928 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:30:11.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.047+0000 7f825d37f700 1 Processor -- start 2023-08-27T11:30:11.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.047+0000 7f825d37f700 1 -- start start 2023-08-27T11:30:11.050 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.048+0000 7f825d37f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8258146990 0x7f82580b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.048+0000 7f825d37f700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f825806d510 0x7f825806d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.048+0000 7f825d37f700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825806f930 con 0x7f8258160a00 2023-08-27T11:30:11.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.048+0000 7f825d37f700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825806dec0 con 0x7f825806d510 2023-08-27T11:30:11.051 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.048+0000 7f825d37f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825806fc10 con 0x7f8258146990 2023-08-27T11:30:11.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.048+0000 7f82577fe700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f825806d510 0x7f825806d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.048+0000 7f82567fc700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8258146990 0x7f82580b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.048+0000 7f8256ffd700 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8258160a00 0x7f825815dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.112:41256/0 (socket says 172.21.15.112:41256) 2023-08-27T11:30:11.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.048+0000 7f8256ffd700 1 -- 172.21.15.112:0/3938918954 learned_addr learned my addr 172.21.15.112:0/3938918954 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:11.052 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.048+0000 7f82577fe700 1 -- 172.21.15.112:0/3938918954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8258146990 msgr2=0x7f82580b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:11.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.048+0000 7f82577fe700 1 --2- 172.21.15.112:0/3938918954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8258146990 0x7f82580b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.048+0000 7f82577fe700 1 -- 172.21.15.112:0/3938918954 >> v1:172.21.15.138:6789/0 conn(0x7f8258160a00 legacy=0x7f825815dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.048+0000 7f82577fe700 1 -- 172.21.15.112:0/3938918954 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825806fef0 con 0x7f825806d510 2023-08-27T11:30:11.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.049+0000 7f82577fe700 1 --2- 172.21.15.112:0/3938918954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f825806d510 0x7f825806d980 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8250009d90 tx=0x7f825000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bfc7e45cb9bb730 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:11.053 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.049+0000 7f8255ffb700 1 -- 172.21.15.112:0/3938918954 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000e070 con 0x7f825806d510 2023-08-27T11:30:11.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.049+0000 7f8255ffb700 1 -- 172.21.15.112:0/3938918954 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825000bb20 con 0x7f825806d510 2023-08-27T11:30:11.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.049+0000 7f8255ffb700 1 -- 172.21.15.112:0/3938918954 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250012430 con 0x7f825806d510 2023-08-27T11:30:11.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.049+0000 7f825d37f700 1 -- 172.21.15.112:0/3938918954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f825806d510 msgr2=0x7f825806d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.049+0000 7f825d37f700 1 --2- 172.21.15.112:0/3938918954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f825806d510 0x7f825806d980 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8250009d90 tx=0x7f825000b330 comp rx=0 tx=0).stop 2023-08-27T11:30:11.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.049+0000 7f825d37f700 1 -- 172.21.15.112:0/3938918954 shutdown_connections 2023-08-27T11:30:11.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.049+0000 7f825d37f700 1 --2- 172.21.15.112:0/3938918954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f825806d510 0x7f825806d980 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.049+0000 7f825d37f700 1 --2- 172.21.15.112:0/3938918954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8258146990 0x7f82580b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.049+0000 7f825d37f700 1 -- 172.21.15.112:0/3938918954 >> 172.21.15.112:0/3938918954 conn(0x7f82580cfe50 msgr2=0x7f82580d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:11.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.049+0000 7f825d37f700 1 -- 172.21.15.112:0/3938918954 shutdown_connections 2023-08-27T11:30:11.055 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.049+0000 7f825d37f700 1 -- 172.21.15.112:0/3938918954 wait complete. 2023-08-27T11:30:11.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.051+0000 7f825d37f700 1 Processor -- start 2023-08-27T11:30:11.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.051+0000 7f825d37f700 1 -- start start 2023-08-27T11:30:11.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.051+0000 7f825d37f700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f825806d510 0x7f82580afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.058 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.051+0000 7f825d37f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8258146990 0x7f8258142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.051+0000 7f825d37f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8258160a00 0x7f82580ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.051+0000 7f825d37f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82580cfc20 con 0x7f8258160a00 2023-08-27T11:30:11.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.051+0000 7f825d37f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82580b01e0 con 0x7f8258146990 2023-08-27T11:30:11.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.051+0000 7f825d37f700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8258142fa0 con 0x7f825806d510 2023-08-27T11:30:11.059 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f82577fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8258160a00 0x7f82580ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f82577fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8258160a00 0x7f82580ad330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39360/0 (socket says 172.21.15.112:39360) 2023-08-27T11:30:11.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f82577fe700 1 -- 172.21.15.112:0/103199492 learned_addr learned my addr 172.21.15.112:0/103199492 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:11.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f82567fc700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8258146990 0x7f8258142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f8256ffd700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f825806d510 0x7f82580afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.060 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f82577fe700 1 -- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f825806d510 msgr2=0x7f82580afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f82577fe700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f825806d510 0x7f82580afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f82577fe700 1 -- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8258146990 msgr2=0x7f8258142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f82577fe700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8258146990 0x7f8258142a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f82577fe700 1 -- 172.21.15.112:0/103199492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8250009a70 con 0x7f8258160a00 2023-08-27T11:30:11.061 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f8256ffd700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f825806d510 0x7f82580afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:11.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f82577fe700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8258160a00 0x7f82580ad330 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f8250014040 tx=0x7f8250004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:11.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f823f7fe700 1 -- 172.21.15.112:0/103199492 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250012430 con 0x7f8258160a00 2023-08-27T11:30:11.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f823f7fe700 1 -- 172.21.15.112:0/103199492 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8250020070 con 0x7f8258160a00 2023-08-27T11:30:11.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f825d37f700 1 -- 172.21.15.112:0/103199492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82580ac890 con 0x7f8258160a00 2023-08-27T11:30:11.062 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f823f7fe700 1 -- 172.21.15.112:0/103199492 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250003ea0 con 0x7f8258160a00 2023-08-27T11:30:11.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.052+0000 7f825d37f700 1 -- 172.21.15.112:0/103199492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82580b06d0 con 0x7f8258160a00 2023-08-27T11:30:11.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.053+0000 7f825d37f700 1 -- 172.21.15.112:0/103199492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8258067a20 con 0x7f8258160a00 2023-08-27T11:30:11.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.053+0000 7f823f7fe700 1 -- 172.21.15.112:0/103199492 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f8250004280 con 0x7f8258160a00 2023-08-27T11:30:11.063 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.054+0000 7f823f7fe700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f824406a4d0 0x7f824406c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.054+0000 7f823f7fe700 1 -- 172.21.15.112:0/103199492 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8250022070 con 0x7f8258160a00 2023-08-27T11:30:11.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.054+0000 7f8256ffd700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f824406a4d0 0x7f824406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.055+0000 7f8256ffd700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f824406a4d0 0x7f824406c990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8248005910 tx=0x7f8248009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:11.064 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.058+0000 7f823f7fe700 1 -- 172.21.15.112:0/103199492 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8250055fd0 con 0x7f8258160a00 2023-08-27T11:30:11.247 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.246+0000 7f825d37f700 1 -- 172.21.15.112:0/103199492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8258160570 con 0x7f8258160a00 2023-08-27T11:30:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.247+0000 7f823f7fe700 1 -- 172.21.15.112:0/103199492 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+18271 (secure 0 0 0) 0x7f82500595f0 con 0x7f8258160a00 2023-08-27T11:30:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:30:11.249 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":184,"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","created":"2023-08-27T11:16:46.503819+0000","modified":"2023-08-27T11:30:08.358293+0000","last_up_change":"2023-08-27T11:16:57.262608+0000","last_in_change":"2023-08-27T11:16:49.027963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-27T11:16:57.927938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-27T11:16:58.266800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:30:04.643937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-08-27T11:30:05.864973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"913e8c21-bec3-440c-915a-0b3155c4c1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6817","nonce":3936386115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6819","nonce":3936386115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6823","nonce":3936386115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3936386115},{"type":"v1","addr":"172.21.15.112:6821","nonce":3936386115}]},"public_addr":"172.21.15.112:6817/3936386115","cluster_addr":"172.21.15.112:6819/3936386115","heartbeat_back_addr":"172.21.15.112:6823/3936386115","heartbeat_front_addr":"172.21.15.112:6821/3936386115","state":["exists","up"]},{"osd":1,"uuid":"28afd136-6896-4ad4-a7ba-3d9ff30bf460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6801","nonce":3136809237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6803","nonce":3136809237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6807","nonce":3136809237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3136809237},{"type":"v1","addr":"172.21.15.112:6805","nonce":3136809237}]},"public_addr":"172.21.15.112:6801/3136809237","cluster_addr":"172.21.15.112:6803/3136809237","heartbeat_back_addr":"172.21.15.112:6807/3136809237","heartbeat_front_addr":"172.21.15.112:6805/3136809237","state":["exists","up"]},{"osd":2,"uuid":"ae5cac36-4cb0-4286-83ee-18eac4623cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6809","nonce":742922092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6811","nonce":742922092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6815","nonce":742922092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":742922092},{"type":"v1","addr":"172.21.15.112:6813","nonce":742922092}]},"public_addr":"172.21.15.112:6809/742922092","cluster_addr":"172.21.15.112:6811/742922092","heartbeat_back_addr":"172.21.15.112:6815/742922092","heartbeat_front_addr":"172.21.15.112:6813/742922092","state":["exists","up"]},{"osd":3,"uuid":"fea7bd81-941f-413d-be1a-54a4578417b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6825","nonce":3108899647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3108899647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6831","nonce":3108899647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3108899647},{"type":"v1","addr":"172.21.15.112:6829","nonce":3108899647}]},"public_addr":"172.21.15.112:6825/3108899647","cluster_addr":"172.21.15.112:6827/3108899647","heartbeat_back_addr":"172.21.15.112:6831/3108899647","heartbeat_front_addr":"172.21.15.112:6829/3108899647","state":["exists","up"]},{"osd":4,"uuid":"d260a2ae-e869-4f76-a0cd-1289be7e12e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6825","nonce":547050660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6827","nonce":547050660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6831","nonce":547050660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":547050660},{"type":"v1","addr":"172.21.15.138:6829","nonce":547050660}]},"public_addr":"172.21.15.138:6825/547050660","cluster_addr":"172.21.15.138:6827/547050660","heartbeat_back_addr":"172.21.15.138:6831/547050660","heartbeat_front_addr":"172.21.15.138:6829/547050660","state":["exists","up"]},{"osd":5,"uuid":"a1d24950-2c63-4da5-88b4-b694c0650c80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6801","nonce":2981217086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6803","nonce":2981217086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6807","nonce":2981217086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2981217086},{"type":"v1","addr":"172.21.15.138:6805","nonce":2981217086}]},"public_addr":"172.21.15.138:6801/2981217086","cluster_addr":"172.21.15.138:6803/2981217086","heartbeat_back_addr":"172.21.15.138:6807/2981217086","heartbeat_front_addr":"172.21.15.138:6805/2981217086","state":["exists","up"]},{"osd":6,"uuid":"0dc87f47-0452-42e0-bc6e-48d04491f63a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6809","nonce":570347350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6811","nonce":570347350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6815","nonce":570347350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":570347350},{"type":"v1","addr":"172.21.15.138:6813","nonce":570347350}]},"public_addr":"172.21.15.138:6809/570347350","cluster_addr":"172.21.15.138:6811/570347350","heartbeat_back_addr":"172.21.15.138:6815/570347350","heartbeat_front_addr":"172.21.15.138:6813/570347350","state":["exists","up"]},{"osd":7,"uuid":"23ba72a6-c505-4abf-9362-dba0e87146c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6817","nonce":2837763972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6819","nonce":2837763972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6823","nonce":2837763972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2837763972},{"type":"v1","addr":"172.21.15.138:6821","nonce":2837763972}]},"public_addr":"172.21.15.138:6817/2837763972","cluster_addr":"172.21.15.138:6819/2837763972","heartbeat_back_addr":"172.21.15.138:6823/2837763972","heartbeat_front_addr":"172.21.15.138:6821/2837763972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.209991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.189530+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.166414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.172115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.242246+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.208087+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.170335+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:16:54.228431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:30:11.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.250+0000 7f825d37f700 1 -- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f824406a4d0 msgr2=0x7f824406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.250+0000 7f825d37f700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f824406a4d0 0x7f824406c990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8248005910 tx=0x7f8248009040 comp rx=0 tx=0).stop 2023-08-27T11:30:11.251 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.250+0000 7f825d37f700 1 -- 172.21.15.112:0/103199492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8258160a00 msgr2=0x7f82580ad330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.250+0000 7f825d37f700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8258160a00 0x7f82580ad330 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f8250014040 tx=0x7f8250004ad0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.250+0000 7f825d37f700 1 -- 172.21.15.112:0/103199492 shutdown_connections 2023-08-27T11:30:11.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.250+0000 7f825d37f700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f824406a4d0 0x7f824406c990 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.250+0000 7f825d37f700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8258160a00 0x7f82580ad330 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.252 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.250+0000 7f825d37f700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f825806d510 0x7f82580afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.250+0000 7f825d37f700 1 --2- 172.21.15.112:0/103199492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8258146990 0x7f8258142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.250+0000 7f825d37f700 1 -- 172.21.15.112:0/103199492 >> 172.21.15.112:0/103199492 conn(0x7f82580cfe50 msgr2=0x7f8258145d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:11.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.250+0000 7f825d37f700 1 -- 172.21.15.112:0/103199492 shutdown_connections 2023-08-27T11:30:11.253 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.250+0000 7f825d37f700 1 -- 172.21.15.112:0/103199492 wait complete. 2023-08-27T11:30:11.263 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:30:11.264 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:30:11.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.382+0000 7fbdc07ef700 1 Processor -- start 2023-08-27T11:30:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.382+0000 7fbdc07ef700 1 -- start start 2023-08-27T11:30:11.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.383+0000 7fbdc07ef700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbdb806dcc0 0x7fbdb80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.383+0000 7fbdc07ef700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbdb80b3440 0x7fbdb80b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.383+0000 7fbdc07ef700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb815b540 con 0x7fbdb815a120 2023-08-27T11:30:11.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.383+0000 7fbdc07ef700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb80af830 con 0x7fbdb80b3440 2023-08-27T11:30:11.387 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.383+0000 7fbdc07ef700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb80afb10 con 0x7fbdb806dcc0 2023-08-27T11:30:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.384+0000 7fbdbed8c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbdb80b3440 0x7fbdb80b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.384+0000 7fbdbed8c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbdb80b3440 0x7fbdb80b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39368/0 (socket says 172.21.15.112:39368) 2023-08-27T11:30:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.384+0000 7fbdbed8c700 1 -- 172.21.15.112:0/2444127749 learned_addr learned my addr 172.21.15.112:0/2444127749 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.384+0000 7fbdbdd8a700 1 --2- 172.21.15.112:0/2444127749 >> v2:172.21.15.138:3300/0 conn(0x7fbdb806dcc0 0x7fbdb80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.384+0000 7fbdbed8c700 1 -- 172.21.15.112:0/2444127749 >> v2:172.21.15.138:3300/0 conn(0x7fbdb806dcc0 msgr2=0x7fbdb80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:11.389 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.384+0000 7fbdbed8c700 1 --2- 172.21.15.112:0/2444127749 >> v2:172.21.15.138:3300/0 conn(0x7fbdb806dcc0 0x7fbdb80b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.384+0000 7fbdbed8c700 1 -- 172.21.15.112:0/2444127749 >> v1:172.21.15.138:6789/0 conn(0x7fbdb815a120 legacy=0x7fbdb806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.384+0000 7fbdbed8c700 1 -- 172.21.15.112:0/2444127749 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb80afdf0 con 0x7fbdb80b3440 2023-08-27T11:30:11.390 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.384+0000 7fbdbed8c700 1 --2- 172.21.15.112:0/2444127749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb80b3440 0x7fbdb80b38b0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80cd2f0 tx=0x7fbdb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a1f0e37517c8eb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:11.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.384+0000 7fbdbd589700 1 -- 172.21.15.112:0/2444127749 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb000e070 con 0x7fbdb80b3440 2023-08-27T11:30:11.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.384+0000 7fbdbd589700 1 -- 172.21.15.112:0/2444127749 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb000b9c0 con 0x7fbdb80b3440 2023-08-27T11:30:11.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.384+0000 7fbdbd589700 1 -- 172.21.15.112:0/2444127749 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb000bd70 con 0x7fbdb80b3440 2023-08-27T11:30:11.391 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.385+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2444127749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb80b3440 msgr2=0x7fbdb80b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.385+0000 7fbdc07ef700 1 --2- 172.21.15.112:0/2444127749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb80b3440 0x7fbdb80b38b0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80cd2f0 tx=0x7fbdb000b330 comp rx=0 tx=0).stop 2023-08-27T11:30:11.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.385+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2444127749 shutdown_connections 2023-08-27T11:30:11.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.385+0000 7fbdc07ef700 1 --2- 172.21.15.112:0/2444127749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb80b3440 0x7fbdb80b38b0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.385+0000 7fbdc07ef700 1 --2- 172.21.15.112:0/2444127749 >> v2:172.21.15.138:3300/0 conn(0x7fbdb806dcc0 0x7fbdb80b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.392 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.385+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2444127749 >> 172.21.15.112:0/2444127749 conn(0x7fbdb80cff80 msgr2=0x7fbdb80d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:11.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.385+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2444127749 shutdown_connections 2023-08-27T11:30:11.393 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.385+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2444127749 wait complete. 2023-08-27T11:30:11.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.386+0000 7fbdc07ef700 1 Processor -- start 2023-08-27T11:30:11.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.386+0000 7fbdc07ef700 1 -- start start 2023-08-27T11:30:11.396 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdc07ef700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdb806dcc0 0x7fbdb80cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdc07ef700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb80b3440 0x7fbdb81e51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdc07ef700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdb815a120 0x7fbdb80cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdc07ef700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb81e5730 con 0x7fbdb80b3440 2023-08-27T11:30:11.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdc07ef700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80ce4b0 con 0x7fbdb806dcc0 2023-08-27T11:30:11.397 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdc07ef700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80cdc10 con 0x7fbdb815a120 2023-08-27T11:30:11.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdbdd8a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb80b3440 0x7fbdb81e51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdbdd8a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb80b3440 0x7fbdb81e51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39372/0 (socket says 172.21.15.112:39372) 2023-08-27T11:30:11.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdbdd8a700 1 -- 172.21.15.112:0/2580636342 learned_addr learned my addr 172.21.15.112:0/2580636342 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:11.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdbe58b700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdb806dcc0 0x7fbdb80cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdbed8c700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdb815a120 0x7fbdb80cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdbdd8a700 1 -- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdb815a120 msgr2=0x7fbdb80cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdbdd8a700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdb815a120 0x7fbdb80cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdbdd8a700 1 -- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdb806dcc0 msgr2=0x7fbdb80cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdbdd8a700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdb806dcc0 0x7fbdb80cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdbdd8a700 1 -- 172.21.15.112:0/2580636342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb0009b30 con 0x7fbdb80b3440 2023-08-27T11:30:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdbdd8a700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb80b3440 0x7fbdb81e51f0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400e810 tx=0x7fbdb400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:11.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdaeffd700 1 -- 172.21.15.112:0/2580636342 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb400c070 con 0x7fbdb80b3440 2023-08-27T11:30:11.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdaeffd700 1 -- 172.21.15.112:0/2580636342 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb4013610 con 0x7fbdb80b3440 2023-08-27T11:30:11.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2580636342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb80cdef0 con 0x7fbdb80b3440 2023-08-27T11:30:11.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdaeffd700 1 -- 172.21.15.112:0/2580636342 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40139c0 con 0x7fbdb80b3440 2023-08-27T11:30:11.400 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.387+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2580636342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb80c9630 con 0x7fbdb80b3440 2023-08-27T11:30:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.388+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2580636342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb8067a20 con 0x7fbdb80b3440 2023-08-27T11:30:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.389+0000 7fbdaeffd700 1 -- 172.21.15.112:0/2580636342 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fbdb4013b20 con 0x7fbdb80b3440 2023-08-27T11:30:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.389+0000 7fbdaeffd700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd9c06a4d0 0x7fbd9c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.389+0000 7fbdaeffd700 1 -- 172.21.15.112:0/2580636342 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbdb4088710 con 0x7fbdb80b3440 2023-08-27T11:30:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.389+0000 7fbdbe58b700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd9c06a4d0 0x7fbd9c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.390+0000 7fbdbe58b700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd9c06a4d0 0x7fbd9c06c990 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fbda8007fd0 tx=0x7fbda8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:11.402 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.393+0000 7fbdaeffd700 1 -- 172.21.15.112:0/2580636342 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbdb4057330 con 0x7fbdb80b3440 2023-08-27T11:30:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.580+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2580636342 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbdb80b00d0 con 0x7fbd9c06a4d0 2023-08-27T11:30:11.590 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.589+0000 7fbdaeffd700 1 -- 172.21.15.112:0/2580636342 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310726 (secure 0 0 0) 0x7fbda8001cc0 con 0x7fbd9c06a4d0 2023-08-27T11:30:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:30:11.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.593+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd9c06a4d0 msgr2=0x7fbd9c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.597 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.593+0000 7fbdc07ef700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd9c06a4d0 0x7fbd9c06c990 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fbda8007fd0 tx=0x7fbda8009040 comp rx=0 tx=0).stop 2023-08-27T11:30:11.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.593+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2580636342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb80b3440 msgr2=0x7fbdb81e51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.593+0000 7fbdc07ef700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb80b3440 0x7fbdb81e51f0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400e810 tx=0x7fbdb400eb20 comp rx=0 tx=0).stop 2023-08-27T11:30:11.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.593+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2580636342 shutdown_connections 2023-08-27T11:30:11.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.593+0000 7fbdc07ef700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fbd9c06a4d0 0x7fbd9c06c990 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.598 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.593+0000 7fbdc07ef700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdb815a120 0x7fbdb80cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.593+0000 7fbdc07ef700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdb806dcc0 0x7fbdb80cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.593+0000 7fbdc07ef700 1 --2- 172.21.15.112:0/2580636342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb80b3440 0x7fbdb81e51f0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.593+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2580636342 >> 172.21.15.112:0/2580636342 conn(0x7fbdb80cff80 msgr2=0x7fbdb81592a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:11.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.593+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2580636342 shutdown_connections 2023-08-27T11:30:11.599 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.593+0000 7fbdc07ef700 1 -- 172.21.15.112:0/2580636342 wait complete. 2023-08-27T11:30:11.600 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:30:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":574,"stamp":"2023-08-27T11:30:10.427283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268692,"kb_used_data":19248,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754706028,"statfs":{"total":773094113280,"available":772818972672,"internally_reserved":0,"allocated":19709952,"data_stored":10028015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.937212"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375737+0000","last_change":"2023-08-27T11:30:07.373935+0000","last_active":"2023-08-27T11:30:07.375737+0000","last_peered":"2023-08-27T11:30:07.375737+0000","last_clean":"2023-08-27T11:30:07.375737+0000","last_became_active":"2023-08-27T11:30:07.373619+0000","last_became_peered":"2023-08-27T11:30:07.373619+0000","last_unstale":"2023-08-27T11:30:07.375737+0000","last_undegraded":"2023-08-27T11:30:07.375737+0000","last_fullsized":"2023-08-27T11:30:07.375737+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:16.699227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.384847+0000","last_change":"2023-08-27T11:30:07.380950+0000","last_active":"2023-08-27T11:30:07.384847+0000","last_peered":"2023-08-27T11:30:07.384847+0000","last_clean":"2023-08-27T11:30:07.384847+0000","last_became_active":"2023-08-27T11:30:07.380668+0000","last_became_peered":"2023-08-27T11:30:07.380668+0000","last_unstale":"2023-08-27T11:30:07.384847+0000","last_undegraded":"2023-08-27T11:30:07.384847+0000","last_fullsized":"2023-08-27T11:30:07.384847+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:27:15.104758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.387295+0000","last_change":"2023-08-27T11:30:07.383032+0000","last_active":"2023-08-27T11:30:07.387295+0000","last_peered":"2023-08-27T11:30:07.387295+0000","last_clean":"2023-08-27T11:30:07.387295+0000","last_became_active":"2023-08-27T11:30:07.382798+0000","last_became_peered":"2023-08-27T11:30:07.382798+0000","last_unstale":"2023-08-27T11:30:07.387295+0000","last_undegraded":"2023-08-27T11:30:07.387295+0000","last_fullsized":"2023-08-27T11:30:07.387295+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:06:13.527429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.379831+0000","last_change":"2023-08-27T11:30:07.378918+0000","last_active":"2023-08-27T11:30:07.379831+0000","last_peered":"2023-08-27T11:30:07.379831+0000","last_clean":"2023-08-27T11:30:07.379831+0000","last_became_active":"2023-08-27T11:30:07.378583+0000","last_became_peered":"2023-08-27T11:30:07.378583+0000","last_unstale":"2023-08-27T11:30:07.379831+0000","last_undegraded":"2023-08-27T11:30:07.379831+0000","last_fullsized":"2023-08-27T11:30:07.379831+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:17:10.066132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376893+0000","last_change":"2023-08-27T11:30:07.375355+0000","last_active":"2023-08-27T11:30:07.376893+0000","last_peered":"2023-08-27T11:30:07.376893+0000","last_clean":"2023-08-27T11:30:07.376893+0000","last_became_active":"2023-08-27T11:30:07.375064+0000","last_became_peered":"2023-08-27T11:30:07.375064+0000","last_unstale":"2023-08-27T11:30:07.376893+0000","last_undegraded":"2023-08-27T11:30:07.376893+0000","last_fullsized":"2023-08-27T11:30:07.376893+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:11:07.185199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.378756+0000","last_change":"2023-08-27T11:30:07.378756+0000","last_active":"2023-08-27T11:30:07.378756+0000","last_peered":"2023-08-27T11:30:07.378756+0000","last_clean":"2023-08-27T11:30:07.378756+0000","last_became_active":"2023-08-27T11:30:07.378495+0000","last_became_peered":"2023-08-27T11:30:07.378495+0000","last_unstale":"2023-08-27T11:30:07.378756+0000","last_undegraded":"2023-08-27T11:30:07.378756+0000","last_fullsized":"2023-08-27T11:30:07.378756+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:01:44.098335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380828+0000","last_change":"2023-08-27T11:30:07.377440+0000","last_active":"2023-08-27T11:30:07.380828+0000","last_peered":"2023-08-27T11:30:07.380828+0000","last_clean":"2023-08-27T11:30:07.380828+0000","last_became_active":"2023-08-27T11:30:07.376823+0000","last_became_peered":"2023-08-27T11:30:07.376823+0000","last_unstale":"2023-08-27T11:30:07.380828+0000","last_undegraded":"2023-08-27T11:30:07.380828+0000","last_fullsized":"2023-08-27T11:30:07.380828+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:38:41.760242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380236+0000","last_change":"2023-08-27T11:30:07.377813+0000","last_active":"2023-08-27T11:30:07.380236+0000","last_peered":"2023-08-27T11:30:07.380236+0000","last_clean":"2023-08-27T11:30:07.380236+0000","last_became_active":"2023-08-27T11:30:07.377471+0000","last_became_peered":"2023-08-27T11:30:07.377471+0000","last_unstale":"2023-08-27T11:30:07.380236+0000","last_undegraded":"2023-08-27T11:30:07.380236+0000","last_fullsized":"2023-08-27T11:30:07.380236+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:35:22.802502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.368157+0000","last_change":"2023-08-27T11:30:07.368157+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.368157+0000","last_undegraded":"2023-08-27T11:30:07.368157+0000","last_fullsized":"2023-08-27T11:30:07.368157+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:48.531209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.382327+0000","last_change":"2023-08-27T11:30:07.378494+0000","last_active":"2023-08-27T11:30:07.382327+0000","last_peered":"2023-08-27T11:30:07.382327+0000","last_clean":"2023-08-27T11:30:07.382327+0000","last_became_active":"2023-08-27T11:30:07.378196+0000","last_became_peered":"2023-08-27T11:30:07.378196+0000","last_unstale":"2023-08-27T11:30:07.382327+0000","last_undegraded":"2023-08-27T11:30:07.382327+0000","last_fullsized":"2023-08-27T11:30:07.382327+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:01:59.361767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.387354+0000","last_change":"2023-08-27T11:30:07.382784+0000","last_active":"2023-08-27T11:30:07.387354+0000","last_peered":"2023-08-27T11:30:07.387354+0000","last_clean":"2023-08-27T11:30:07.387354+0000","last_became_active":"2023-08-27T11:30:07.382436+0000","last_became_peered":"2023-08-27T11:30:07.382436+0000","last_unstale":"2023-08-27T11:30:07.387354+0000","last_undegraded":"2023-08-27T11:30:07.387354+0000","last_fullsized":"2023-08-27T11:30:07.387354+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:15:13.361952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375762+0000","last_change":"2023-08-27T11:30:07.372273+0000","last_active":"2023-08-27T11:30:07.375762+0000","last_peered":"2023-08-27T11:30:07.375762+0000","last_clean":"2023-08-27T11:30:07.375762+0000","last_became_active":"2023-08-27T11:30:07.371943+0000","last_became_peered":"2023-08-27T11:30:07.371943+0000","last_unstale":"2023-08-27T11:30:07.375762+0000","last_undegraded":"2023-08-27T11:30:07.375762+0000","last_fullsized":"2023-08-27T11:30:07.375762+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:06:45.150286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.363359+0000","last_change":"2023-08-27T11:30:07.363359+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.363359+0000","last_undegraded":"2023-08-27T11:30:07.363359+0000","last_fullsized":"2023-08-27T11:30:07.363359+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:05:25.871181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370483+0000","last_change":"2023-08-27T11:30:07.367703+0000","last_active":"2023-08-27T11:30:07.370483+0000","last_peered":"2023-08-27T11:30:07.370483+0000","last_clean":"2023-08-27T11:30:07.370483+0000","last_became_active":"2023-08-27T11:30:07.367443+0000","last_became_peered":"2023-08-27T11:30:07.367443+0000","last_unstale":"2023-08-27T11:30:07.370483+0000","last_undegraded":"2023-08-27T11:30:07.370483+0000","last_fullsized":"2023-08-27T11:30:07.370483+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:40:42.999295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376506+0000","last_change":"2023-08-27T11:30:07.376506+0000","last_active":"2023-08-27T11:30:07.376506+0000","last_peered":"2023-08-27T11:30:07.376506+0000","last_clean":"2023-08-27T11:30:07.376506+0000","last_became_active":"2023-08-27T11:30:07.376242+0000","last_became_peered":"2023-08-27T11:30:07.376242+0000","last_unstale":"2023-08-27T11:30:07.376506+0000","last_undegraded":"2023-08-27T11:30:07.376506+0000","last_fullsized":"2023-08-27T11:30:07.376506+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:41:53.411101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377343+0000","last_change":"2023-08-27T11:30:07.377343+0000","last_active":"2023-08-27T11:30:07.377343+0000","last_peered":"2023-08-27T11:30:07.377343+0000","last_clean":"2023-08-27T11:30:07.377343+0000","last_became_active":"2023-08-27T11:30:07.377032+0000","last_became_peered":"2023-08-27T11:30:07.377032+0000","last_unstale":"2023-08-27T11:30:07.377343+0000","last_undegraded":"2023-08-27T11:30:07.377343+0000","last_fullsized":"2023-08-27T11:30:07.377343+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:53:51.803335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.496034+0000","last_change":"2023-08-27T11:30:06.496034+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.496034+0000","last_undegraded":"2023-08-27T11:30:06.496034+0000","last_fullsized":"2023-08-27T11:30:06.496034+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.382296+0000","last_change":"2023-08-27T11:30:07.376652+0000","last_active":"2023-08-27T11:30:07.382296+0000","last_peered":"2023-08-27T11:30:07.382296+0000","last_clean":"2023-08-27T11:30:07.382296+0000","last_became_active":"2023-08-27T11:30:07.376381+0000","last_became_peered":"2023-08-27T11:30:07.376381+0000","last_unstale":"2023-08-27T11:30:07.382296+0000","last_undegraded":"2023-08-27T11:30:07.382296+0000","last_fullsized":"2023-08-27T11:30:07.382296+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:47:41.397698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380879+0000","last_change":"2023-08-27T11:30:07.377266+0000","last_active":"2023-08-27T11:30:07.380879+0000","last_peered":"2023-08-27T11:30:07.380879+0000","last_clean":"2023-08-27T11:30:07.380879+0000","last_became_active":"2023-08-27T11:30:07.376986+0000","last_became_peered":"2023-08-27T11:30:07.376986+0000","last_unstale":"2023-08-27T11:30:07.380879+0000","last_undegraded":"2023-08-27T11:30:07.380879+0000","last_fullsized":"2023-08-27T11:30:07.380879+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:46:44.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376491+0000","last_change":"2023-08-27T11:30:07.376491+0000","last_active":"2023-08-27T11:30:07.376491+0000","last_peered":"2023-08-27T11:30:07.376491+0000","last_clean":"2023-08-27T11:30:07.376491+0000","last_became_active":"2023-08-27T11:30:07.376247+0000","last_became_peered":"2023-08-27T11:30:07.376247+0000","last_unstale":"2023-08-27T11:30:07.376491+0000","last_undegraded":"2023-08-27T11:30:07.376491+0000","last_fullsized":"2023-08-27T11:30:07.376491+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:09:11.582134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.368242+0000","last_change":"2023-08-27T11:30:07.368242+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.368242+0000","last_undegraded":"2023-08-27T11:30:07.368242+0000","last_fullsized":"2023-08-27T11:30:07.368242+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:46:21.277652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.369379+0000","last_change":"2023-08-27T11:30:07.369379+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.369379+0000","last_undegraded":"2023-08-27T11:30:07.369379+0000","last_fullsized":"2023-08-27T11:30:07.369379+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:18:50.569923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380789+0000","last_change":"2023-08-27T11:30:07.377088+0000","last_active":"2023-08-27T11:30:07.380789+0000","last_peered":"2023-08-27T11:30:07.380789+0000","last_clean":"2023-08-27T11:30:07.380789+0000","last_became_active":"2023-08-27T11:30:07.376059+0000","last_became_peered":"2023-08-27T11:30:07.376059+0000","last_unstale":"2023-08-27T11:30:07.380789+0000","last_undegraded":"2023-08-27T11:30:07.380789+0000","last_fullsized":"2023-08-27T11:30:07.380789+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:57.640378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.495396+0000","last_change":"2023-08-27T11:30:06.495396+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.495396+0000","last_undegraded":"2023-08-27T11:30:06.495396+0000","last_fullsized":"2023-08-27T11:30:06.495396+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375518+0000","last_change":"2023-08-27T11:30:07.370865+0000","last_active":"2023-08-27T11:30:07.375518+0000","last_peered":"2023-08-27T11:30:07.375518+0000","last_clean":"2023-08-27T11:30:07.375518+0000","last_became_active":"2023-08-27T11:30:07.370571+0000","last_became_peered":"2023-08-27T11:30:07.370571+0000","last_unstale":"2023-08-27T11:30:07.375518+0000","last_undegraded":"2023-08-27T11:30:07.375518+0000","last_fullsized":"2023-08-27T11:30:07.375518+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:45:40.891269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.511830+0000","last_change":"2023-08-27T11:30:06.511151+0000","last_active":"2023-08-27T11:30:06.511830+0000","last_peered":"2023-08-27T11:30:06.511830+0000","last_clean":"2023-08-27T11:30:06.511830+0000","last_became_active":"2023-08-27T11:30:06.510830+0000","last_became_peered":"2023-08-27T11:30:06.510830+0000","last_unstale":"2023-08-27T11:30:06.511830+0000","last_undegraded":"2023-08-27T11:30:06.511830+0000","last_fullsized":"2023-08-27T11:30:06.511830+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:26:30.060104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.378015+0000","last_change":"2023-08-27T11:30:07.378015+0000","last_active":"2023-08-27T11:30:07.378015+0000","last_peered":"2023-08-27T11:30:07.378015+0000","last_clean":"2023-08-27T11:30:07.378015+0000","last_became_active":"2023-08-27T11:30:07.377749+0000","last_became_peered":"2023-08-27T11:30:07.377749+0000","last_unstale":"2023-08-27T11:30:07.378015+0000","last_undegraded":"2023-08-27T11:30:07.378015+0000","last_fullsized":"2023-08-27T11:30:07.378015+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:40.421055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.365033+0000","last_change":"2023-08-27T11:30:06.506992+0000","last_active":"2023-08-27T11:30:07.365033+0000","last_peered":"2023-08-27T11:30:07.365033+0000","last_clean":"2023-08-27T11:30:07.365033+0000","last_became_active":"2023-08-27T11:30:06.506671+0000","last_became_peered":"2023-08-27T11:30:06.506671+0000","last_unstale":"2023-08-27T11:30:07.365033+0000","last_undegraded":"2023-08-27T11:30:07.365033+0000","last_fullsized":"2023-08-27T11:30:07.365033+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:07:13.889636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.387319+0000","last_change":"2023-08-27T11:30:07.383160+0000","last_active":"2023-08-27T11:30:07.387319+0000","last_peered":"2023-08-27T11:30:07.387319+0000","last_clean":"2023-08-27T11:30:07.387319+0000","last_became_active":"2023-08-27T11:30:07.382823+0000","last_became_peered":"2023-08-27T11:30:07.382823+0000","last_unstale":"2023-08-27T11:30:07.387319+0000","last_undegraded":"2023-08-27T11:30:07.387319+0000","last_fullsized":"2023-08-27T11:30:07.387319+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:13.020805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377069+0000","last_change":"2023-08-27T11:30:06.509202+0000","last_active":"2023-08-27T11:30:07.377069+0000","last_peered":"2023-08-27T11:30:07.377069+0000","last_clean":"2023-08-27T11:30:07.377069+0000","last_became_active":"2023-08-27T11:30:06.508960+0000","last_became_peered":"2023-08-27T11:30:06.508960+0000","last_unstale":"2023-08-27T11:30:07.377069+0000","last_undegraded":"2023-08-27T11:30:07.377069+0000","last_fullsized":"2023-08-27T11:30:07.377069+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:41:59.638609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375837+0000","last_change":"2023-08-27T11:30:07.373389+0000","last_active":"2023-08-27T11:30:07.375837+0000","last_peered":"2023-08-27T11:30:07.375837+0000","last_clean":"2023-08-27T11:30:07.375837+0000","last_became_active":"2023-08-27T11:30:07.373083+0000","last_became_peered":"2023-08-27T11:30:07.373083+0000","last_unstale":"2023-08-27T11:30:07.375837+0000","last_undegraded":"2023-08-27T11:30:07.375837+0000","last_fullsized":"2023-08-27T11:30:07.375837+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:01:00.108886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.374018+0000","last_change":"2023-08-27T11:30:06.509017+0000","last_active":"2023-08-27T11:30:07.374018+0000","last_peered":"2023-08-27T11:30:07.374018+0000","last_clean":"2023-08-27T11:30:07.374018+0000","last_became_active":"2023-08-27T11:30:06.508619+0000","last_became_peered":"2023-08-27T11:30:06.508619+0000","last_unstale":"2023-08-27T11:30:07.374018+0000","last_undegraded":"2023-08-27T11:30:07.374018+0000","last_fullsized":"2023-08-27T11:30:07.374018+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:01:51.459390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.368141+0000","last_change":"2023-08-27T11:30:07.368141+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.368141+0000","last_undegraded":"2023-08-27T11:30:07.368141+0000","last_fullsized":"2023-08-27T11:30:07.368141+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:28:48.644407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.362633+0000","last_change":"2023-08-27T11:30:06.503177+0000","last_active":"2023-08-27T11:30:07.362633+0000","last_peered":"2023-08-27T11:30:07.362633+0000","last_clean":"2023-08-27T11:30:07.362633+0000","last_became_active":"2023-08-27T11:30:06.502915+0000","last_became_peered":"2023-08-27T11:30:06.502915+0000","last_unstale":"2023-08-27T11:30:07.362633+0000","last_undegraded":"2023-08-27T11:30:07.362633+0000","last_fullsized":"2023-08-27T11:30:07.362633+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:04:47.203315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":4,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.377944+0000","last_change":"2023-08-27T11:30:07.372096+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.377944+0000","last_undegraded":"2023-08-27T11:30:07.377944+0000","last_fullsized":"2023-08-27T11:30:07.377944+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:49:33.070564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.508333+0000","last_change":"2023-08-27T11:30:06.503408+0000","last_active":"2023-08-27T11:30:06.508333+0000","last_peered":"2023-08-27T11:30:06.508333+0000","last_clean":"2023-08-27T11:30:06.508333+0000","last_became_active":"2023-08-27T11:30:06.503111+0000","last_became_peered":"2023-08-27T11:30:06.503111+0000","last_unstale":"2023-08-27T11:30:06.508333+0000","last_undegraded":"2023-08-27T11:30:06.508333+0000","last_fullsized":"2023-08-27T11:30:06.508333+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:54:21.203272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.369922+0000","last_change":"2023-08-27T11:30:07.369922+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.369922+0000","last_undegraded":"2023-08-27T11:30:07.369922+0000","last_fullsized":"2023-08-27T11:30:07.369922+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:34.484460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.371984+0000","last_change":"2023-08-27T11:30:06.506988+0000","last_active":"2023-08-27T11:30:07.371984+0000","last_peered":"2023-08-27T11:30:07.371984+0000","last_clean":"2023-08-27T11:30:07.371984+0000","last_became_active":"2023-08-27T11:30:06.506687+0000","last_became_peered":"2023-08-27T11:30:06.506687+0000","last_unstale":"2023-08-27T11:30:07.371984+0000","last_undegraded":"2023-08-27T11:30:07.371984+0000","last_fullsized":"2023-08-27T11:30:07.371984+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:55:51.579443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380804+0000","last_change":"2023-08-27T11:30:07.377249+0000","last_active":"2023-08-27T11:30:07.380804+0000","last_peered":"2023-08-27T11:30:07.380804+0000","last_clean":"2023-08-27T11:30:07.380804+0000","last_became_active":"2023-08-27T11:30:07.376448+0000","last_became_peered":"2023-08-27T11:30:07.376448+0000","last_unstale":"2023-08-27T11:30:07.380804+0000","last_undegraded":"2023-08-27T11:30:07.380804+0000","last_fullsized":"2023-08-27T11:30:07.380804+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:02:14.837924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.363301+0000","last_change":"2023-08-27T11:30:06.507787+0000","last_active":"2023-08-27T11:30:07.363301+0000","last_peered":"2023-08-27T11:30:07.363301+0000","last_clean":"2023-08-27T11:30:07.363301+0000","last_became_active":"2023-08-27T11:30:06.507390+0000","last_became_peered":"2023-08-27T11:30:06.507390+0000","last_unstale":"2023-08-27T11:30:07.363301+0000","last_undegraded":"2023-08-27T11:30:07.363301+0000","last_fullsized":"2023-08-27T11:30:07.363301+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:00:38.629013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380650+0000","last_change":"2023-08-27T11:30:07.376530+0000","last_active":"2023-08-27T11:30:07.380650+0000","last_peered":"2023-08-27T11:30:07.380650+0000","last_clean":"2023-08-27T11:30:07.380650+0000","last_became_active":"2023-08-27T11:30:07.376221+0000","last_became_peered":"2023-08-27T11:30:07.376221+0000","last_unstale":"2023-08-27T11:30:07.380650+0000","last_undegraded":"2023-08-27T11:30:07.380650+0000","last_fullsized":"2023-08-27T11:30:07.380650+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:41.856918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.510282+0000","last_change":"2023-08-27T11:30:06.508059+0000","last_active":"2023-08-27T11:30:06.510282+0000","last_peered":"2023-08-27T11:30:06.510282+0000","last_clean":"2023-08-27T11:30:06.510282+0000","last_became_active":"2023-08-27T11:30:06.507677+0000","last_became_peered":"2023-08-27T11:30:06.507677+0000","last_unstale":"2023-08-27T11:30:06.510282+0000","last_undegraded":"2023-08-27T11:30:06.510282+0000","last_fullsized":"2023-08-27T11:30:06.510282+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:50:44.051625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.495697+0000","last_change":"2023-08-27T11:30:06.495697+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.495697+0000","last_undegraded":"2023-08-27T11:30:06.495697+0000","last_fullsized":"2023-08-27T11:30:06.495697+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375338+0000","last_change":"2023-08-27T11:30:06.509700+0000","last_active":"2023-08-27T11:30:07.375338+0000","last_peered":"2023-08-27T11:30:07.375338+0000","last_clean":"2023-08-27T11:30:07.375338+0000","last_became_active":"2023-08-27T11:30:06.509455+0000","last_became_peered":"2023-08-27T11:30:06.509455+0000","last_unstale":"2023-08-27T11:30:07.375338+0000","last_undegraded":"2023-08-27T11:30:07.375338+0000","last_fullsized":"2023-08-27T11:30:07.375338+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:12:09.969178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.373978+0000","last_change":"2023-08-27T11:30:07.373333+0000","last_active":"2023-08-27T11:30:07.373978+0000","last_peered":"2023-08-27T11:30:07.373978+0000","last_clean":"2023-08-27T11:30:07.373978+0000","last_became_active":"2023-08-27T11:30:07.373009+0000","last_became_peered":"2023-08-27T11:30:07.373009+0000","last_unstale":"2023-08-27T11:30:07.373978+0000","last_undegraded":"2023-08-27T11:30:07.373978+0000","last_fullsized":"2023-08-27T11:30:07.373978+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:24:21.667407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377251+0000","last_change":"2023-08-27T11:30:06.509723+0000","last_active":"2023-08-27T11:30:07.377251+0000","last_peered":"2023-08-27T11:30:07.377251+0000","last_clean":"2023-08-27T11:30:07.377251+0000","last_became_active":"2023-08-27T11:30:06.509418+0000","last_became_peered":"2023-08-27T11:30:06.509418+0000","last_unstale":"2023-08-27T11:30:07.377251+0000","last_undegraded":"2023-08-27T11:30:07.377251+0000","last_fullsized":"2023-08-27T11:30:07.377251+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:23:19.197311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.494625+0000","last_change":"2023-08-27T11:30:06.494625+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.494625+0000","last_undegraded":"2023-08-27T11:30:06.494625+0000","last_fullsized":"2023-08-27T11:30:06.494625+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.508364+0000","last_change":"2023-08-27T11:30:06.503031+0000","last_active":"2023-08-27T11:30:06.508364+0000","last_peered":"2023-08-27T11:30:06.508364+0000","last_clean":"2023-08-27T11:30:06.508364+0000","last_became_active":"2023-08-27T11:30:06.502780+0000","last_became_peered":"2023-08-27T11:30:06.502780+0000","last_unstale":"2023-08-27T11:30:06.508364+0000","last_undegraded":"2023-08-27T11:30:06.508364+0000","last_fullsized":"2023-08-27T11:30:06.508364+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:39:40.298338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.496930+0000","last_change":"2023-08-27T11:30:06.496930+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.496930+0000","last_undegraded":"2023-08-27T11:30:06.496930+0000","last_fullsized":"2023-08-27T11:30:06.496930+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.365524+0000","last_change":"2023-08-27T11:30:06.511539+0000","last_active":"2023-08-27T11:30:07.365524+0000","last_peered":"2023-08-27T11:30:07.365524+0000","last_clean":"2023-08-27T11:30:07.365524+0000","last_became_active":"2023-08-27T11:30:06.511200+0000","last_became_peered":"2023-08-27T11:30:06.511200+0000","last_unstale":"2023-08-27T11:30:07.365524+0000","last_undegraded":"2023-08-27T11:30:07.365524+0000","last_fullsized":"2023-08-27T11:30:07.365524+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:01.722653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.371737+0000","last_change":"2023-08-27T11:30:07.371737+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.371737+0000","last_undegraded":"2023-08-27T11:30:07.371737+0000","last_fullsized":"2023-08-27T11:30:07.371737+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:39:16.696934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370740+0000","last_change":"2023-08-27T11:30:06.505422+0000","last_active":"2023-08-27T11:30:07.370740+0000","last_peered":"2023-08-27T11:30:07.370740+0000","last_clean":"2023-08-27T11:30:07.370740+0000","last_became_active":"2023-08-27T11:30:06.505120+0000","last_became_peered":"2023-08-27T11:30:06.505120+0000","last_unstale":"2023-08-27T11:30:07.370740+0000","last_undegraded":"2023-08-27T11:30:07.370740+0000","last_fullsized":"2023-08-27T11:30:07.370740+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:21:11.370238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.494948+0000","last_change":"2023-08-27T11:30:06.494948+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.494948+0000","last_undegraded":"2023-08-27T11:30:06.494948+0000","last_fullsized":"2023-08-27T11:30:06.494948+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.373236+0000","last_change":"2023-08-27T11:30:06.506911+0000","last_active":"2023-08-27T11:30:07.373236+0000","last_peered":"2023-08-27T11:30:07.373236+0000","last_clean":"2023-08-27T11:30:07.373236+0000","last_became_active":"2023-08-27T11:30:06.506657+0000","last_became_peered":"2023-08-27T11:30:06.506657+0000","last_unstale":"2023-08-27T11:30:07.373236+0000","last_undegraded":"2023-08-27T11:30:07.373236+0000","last_fullsized":"2023-08-27T11:30:07.373236+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:05.904960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.497037+0000","last_change":"2023-08-27T11:30:06.497037+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.497037+0000","last_undegraded":"2023-08-27T11:30:06.497037+0000","last_fullsized":"2023-08-27T11:30:06.497037+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.374141+0000","last_change":"2023-08-27T11:30:06.516142+0000","last_active":"2023-08-27T11:30:07.374141+0000","last_peered":"2023-08-27T11:30:07.374141+0000","last_clean":"2023-08-27T11:30:07.374141+0000","last_became_active":"2023-08-27T11:30:06.515761+0000","last_became_peered":"2023-08-27T11:30:06.515761+0000","last_unstale":"2023-08-27T11:30:07.374141+0000","last_undegraded":"2023-08-27T11:30:07.374141+0000","last_fullsized":"2023-08-27T11:30:07.374141+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:12:05.165692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376818+0000","last_change":"2023-08-27T11:30:07.376818+0000","last_active":"2023-08-27T11:30:07.376818+0000","last_peered":"2023-08-27T11:30:07.376818+0000","last_clean":"2023-08-27T11:30:07.376818+0000","last_became_active":"2023-08-27T11:30:07.376575+0000","last_became_peered":"2023-08-27T11:30:07.376575+0000","last_unstale":"2023-08-27T11:30:07.376818+0000","last_undegraded":"2023-08-27T11:30:07.376818+0000","last_fullsized":"2023-08-27T11:30:07.376818+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:14:01.759128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.366408+0000","last_change":"2023-08-27T11:30:06.504618+0000","last_active":"2023-08-27T11:30:07.366408+0000","last_peered":"2023-08-27T11:30:07.366408+0000","last_clean":"2023-08-27T11:30:07.366408+0000","last_became_active":"2023-08-27T11:30:06.504354+0000","last_became_peered":"2023-08-27T11:30:06.504354+0000","last_unstale":"2023-08-27T11:30:07.366408+0000","last_undegraded":"2023-08-27T11:30:07.366408+0000","last_fullsized":"2023-08-27T11:30:07.366408+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:09:49.437363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.384834+0000","last_change":"2023-08-27T11:30:07.380702+0000","last_active":"2023-08-27T11:30:07.384834+0000","last_peered":"2023-08-27T11:30:07.384834+0000","last_clean":"2023-08-27T11:30:07.384834+0000","last_became_active":"2023-08-27T11:30:07.380378+0000","last_became_peered":"2023-08-27T11:30:07.380378+0000","last_unstale":"2023-08-27T11:30:07.384834+0000","last_undegraded":"2023-08-27T11:30:07.384834+0000","last_fullsized":"2023-08-27T11:30:07.384834+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:06:41.915453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375394+0000","last_change":"2023-08-27T11:30:06.511005+0000","last_active":"2023-08-27T11:30:07.375394+0000","last_peered":"2023-08-27T11:30:07.375394+0000","last_clean":"2023-08-27T11:30:07.375394+0000","last_became_active":"2023-08-27T11:30:06.510712+0000","last_became_peered":"2023-08-27T11:30:06.510712+0000","last_unstale":"2023-08-27T11:30:07.375394+0000","last_undegraded":"2023-08-27T11:30:07.375394+0000","last_fullsized":"2023-08-27T11:30:07.375394+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:53:54.707523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377430+0000","last_change":"2023-08-27T11:30:07.377430+0000","last_active":"2023-08-27T11:30:07.377430+0000","last_peered":"2023-08-27T11:30:07.377430+0000","last_clean":"2023-08-27T11:30:07.377430+0000","last_became_active":"2023-08-27T11:30:07.376459+0000","last_became_peered":"2023-08-27T11:30:07.376459+0000","last_unstale":"2023-08-27T11:30:07.377430+0000","last_undegraded":"2023-08-27T11:30:07.377430+0000","last_fullsized":"2023-08-27T11:30:07.377430+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:51:11.653496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372581+0000","last_change":"2023-08-27T11:30:06.510880+0000","last_active":"2023-08-27T11:30:07.372581+0000","last_peered":"2023-08-27T11:30:07.372581+0000","last_clean":"2023-08-27T11:30:07.372581+0000","last_became_active":"2023-08-27T11:30:06.510602+0000","last_became_peered":"2023-08-27T11:30:06.510602+0000","last_unstale":"2023-08-27T11:30:07.372581+0000","last_undegraded":"2023-08-27T11:30:07.372581+0000","last_fullsized":"2023-08-27T11:30:07.372581+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:12.746422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377066+0000","last_change":"2023-08-27T11:30:07.374685+0000","last_active":"2023-08-27T11:30:07.377066+0000","last_peered":"2023-08-27T11:30:07.377066+0000","last_clean":"2023-08-27T11:30:07.377066+0000","last_became_active":"2023-08-27T11:30:07.374131+0000","last_became_peered":"2023-08-27T11:30:07.374131+0000","last_unstale":"2023-08-27T11:30:07.377066+0000","last_undegraded":"2023-08-27T11:30:07.377066+0000","last_fullsized":"2023-08-27T11:30:07.377066+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:22:46.104350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.364655+0000","last_change":"2023-08-27T11:30:06.515673+0000","last_active":"2023-08-27T11:30:07.364655+0000","last_peered":"2023-08-27T11:30:07.364655+0000","last_clean":"2023-08-27T11:30:07.364655+0000","last_became_active":"2023-08-27T11:30:06.515364+0000","last_became_peered":"2023-08-27T11:30:06.515364+0000","last_unstale":"2023-08-27T11:30:07.364655+0000","last_undegraded":"2023-08-27T11:30:07.364655+0000","last_fullsized":"2023-08-27T11:30:07.364655+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:05:05.778823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.378734+0000","last_change":"2023-08-27T11:30:07.378734+0000","last_active":"2023-08-27T11:30:07.378734+0000","last_peered":"2023-08-27T11:30:07.378734+0000","last_clean":"2023-08-27T11:30:07.378734+0000","last_became_active":"2023-08-27T11:30:07.378472+0000","last_became_peered":"2023-08-27T11:30:07.378472+0000","last_unstale":"2023-08-27T11:30:07.378734+0000","last_undegraded":"2023-08-27T11:30:07.378734+0000","last_fullsized":"2023-08-27T11:30:07.378734+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:46:20.644887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.369874+0000","last_change":"2023-08-27T11:30:06.504079+0000","last_active":"2023-08-27T11:30:07.369874+0000","last_peered":"2023-08-27T11:30:07.369874+0000","last_clean":"2023-08-27T11:30:07.369874+0000","last_became_active":"2023-08-27T11:30:06.503811+0000","last_became_peered":"2023-08-27T11:30:06.503811+0000","last_unstale":"2023-08-27T11:30:07.369874+0000","last_undegraded":"2023-08-27T11:30:07.369874+0000","last_fullsized":"2023-08-27T11:30:07.369874+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:27:05.776026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":4,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.376957+0000","last_change":"2023-08-27T11:30:07.371363+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.376957+0000","last_undegraded":"2023-08-27T11:30:07.376957+0000","last_fullsized":"2023-08-27T11:30:07.376957+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:49:27.497988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.365622+0000","last_change":"2023-08-27T11:30:06.503387+0000","last_active":"2023-08-27T11:30:07.365622+0000","last_peered":"2023-08-27T11:30:07.365622+0000","last_clean":"2023-08-27T11:30:07.365622+0000","last_became_active":"2023-08-27T11:30:06.503081+0000","last_became_peered":"2023-08-27T11:30:06.503081+0000","last_unstale":"2023-08-27T11:30:07.365622+0000","last_undegraded":"2023-08-27T11:30:07.365622+0000","last_fullsized":"2023-08-27T11:30:07.365622+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:25:44.805880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.494479+0000","last_change":"2023-08-27T11:30:06.494479+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.494479+0000","last_undegraded":"2023-08-27T11:30:06.494479+0000","last_fullsized":"2023-08-27T11:30:06.494479+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372445+0000","last_change":"2023-08-27T11:30:06.511539+0000","last_active":"2023-08-27T11:30:07.372445+0000","last_peered":"2023-08-27T11:30:07.372445+0000","last_clean":"2023-08-27T11:30:07.372445+0000","last_became_active":"2023-08-27T11:30:06.511248+0000","last_became_peered":"2023-08-27T11:30:06.511248+0000","last_unstale":"2023-08-27T11:30:07.372445+0000","last_undegraded":"2023-08-27T11:30:07.372445+0000","last_fullsized":"2023-08-27T11:30:07.372445+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:22:48.182930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376018+0000","last_change":"2023-08-27T11:30:07.373745+0000","last_active":"2023-08-27T11:30:07.376018+0000","last_peered":"2023-08-27T11:30:07.376018+0000","last_clean":"2023-08-27T11:30:07.376018+0000","last_became_active":"2023-08-27T11:30:07.373460+0000","last_became_peered":"2023-08-27T11:30:07.373460+0000","last_unstale":"2023-08-27T11:30:07.376018+0000","last_undegraded":"2023-08-27T11:30:07.376018+0000","last_fullsized":"2023-08-27T11:30:07.376018+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:47:30.525294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.366107+0000","last_change":"2023-08-27T11:30:06.502148+0000","last_active":"2023-08-27T11:30:07.366107+0000","last_peered":"2023-08-27T11:30:07.366107+0000","last_clean":"2023-08-27T11:30:07.366107+0000","last_became_active":"2023-08-27T11:30:06.501869+0000","last_became_peered":"2023-08-27T11:30:06.501869+0000","last_unstale":"2023-08-27T11:30:07.366107+0000","last_undegraded":"2023-08-27T11:30:07.366107+0000","last_fullsized":"2023-08-27T11:30:07.366107+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:19:08.944028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376696+0000","last_change":"2023-08-27T11:30:07.376696+0000","last_active":"2023-08-27T11:30:07.376696+0000","last_peered":"2023-08-27T11:30:07.376696+0000","last_clean":"2023-08-27T11:30:07.376696+0000","last_became_active":"2023-08-27T11:30:07.376266+0000","last_became_peered":"2023-08-27T11:30:07.376266+0000","last_unstale":"2023-08-27T11:30:07.376696+0000","last_undegraded":"2023-08-27T11:30:07.376696+0000","last_fullsized":"2023-08-27T11:30:07.376696+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:50:58.773261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.364173+0000","last_change":"2023-08-27T11:30:06.504227+0000","last_active":"2023-08-27T11:30:07.364173+0000","last_peered":"2023-08-27T11:30:07.364173+0000","last_clean":"2023-08-27T11:30:07.364173+0000","last_became_active":"2023-08-27T11:30:06.503935+0000","last_became_peered":"2023-08-27T11:30:06.503935+0000","last_unstale":"2023-08-27T11:30:07.364173+0000","last_undegraded":"2023-08-27T11:30:07.364173+0000","last_fullsized":"2023-08-27T11:30:07.364173+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:44:29.308537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.495279+0000","last_change":"2023-08-27T11:30:06.495279+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.495279+0000","last_undegraded":"2023-08-27T11:30:06.495279+0000","last_fullsized":"2023-08-27T11:30:06.495279+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370211+0000","last_change":"2023-08-27T11:30:06.515077+0000","last_active":"2023-08-27T11:30:07.370211+0000","last_peered":"2023-08-27T11:30:07.370211+0000","last_clean":"2023-08-27T11:30:07.370211+0000","last_became_active":"2023-08-27T11:30:06.514812+0000","last_became_peered":"2023-08-27T11:30:06.514812+0000","last_unstale":"2023-08-27T11:30:07.370211+0000","last_undegraded":"2023-08-27T11:30:07.370211+0000","last_fullsized":"2023-08-27T11:30:07.370211+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:56:41.368080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377075+0000","last_change":"2023-08-27T11:30:07.375476+0000","last_active":"2023-08-27T11:30:07.377075+0000","last_peered":"2023-08-27T11:30:07.377075+0000","last_clean":"2023-08-27T11:30:07.377075+0000","last_became_active":"2023-08-27T11:30:07.375167+0000","last_became_peered":"2023-08-27T11:30:07.375167+0000","last_unstale":"2023-08-27T11:30:07.377075+0000","last_undegraded":"2023-08-27T11:30:07.377075+0000","last_fullsized":"2023-08-27T11:30:07.377075+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:18:51.080195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.363503+0000","last_change":"2023-08-27T11:30:06.507172+0000","last_active":"2023-08-27T11:30:07.363503+0000","last_peered":"2023-08-27T11:30:07.363503+0000","last_clean":"2023-08-27T11:30:07.363503+0000","last_became_active":"2023-08-27T11:30:06.506885+0000","last_became_peered":"2023-08-27T11:30:06.506885+0000","last_unstale":"2023-08-27T11:30:07.363503+0000","last_undegraded":"2023-08-27T11:30:07.363503+0000","last_fullsized":"2023-08-27T11:30:07.363503+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:03.298758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.387326+0000","last_change":"2023-08-27T11:30:07.383116+0000","last_active":"2023-08-27T11:30:07.387326+0000","last_peered":"2023-08-27T11:30:07.387326+0000","last_clean":"2023-08-27T11:30:07.387326+0000","last_became_active":"2023-08-27T11:30:07.382838+0000","last_became_peered":"2023-08-27T11:30:07.382838+0000","last_unstale":"2023-08-27T11:30:07.387326+0000","last_undegraded":"2023-08-27T11:30:07.387326+0000","last_fullsized":"2023-08-27T11:30:07.387326+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:48:13.826189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.510301+0000","last_change":"2023-08-27T11:30:06.508063+0000","last_active":"2023-08-27T11:30:06.510301+0000","last_peered":"2023-08-27T11:30:06.510301+0000","last_clean":"2023-08-27T11:30:06.510301+0000","last_became_active":"2023-08-27T11:30:06.507412+0000","last_became_peered":"2023-08-27T11:30:06.507412+0000","last_unstale":"2023-08-27T11:30:06.510301+0000","last_undegraded":"2023-08-27T11:30:06.510301+0000","last_fullsized":"2023-08-27T11:30:06.510301+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:58.753023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.374655+0000","last_change":"2023-08-27T11:30:07.374655+0000","last_active":"2023-08-27T11:30:07.374655+0000","last_peered":"2023-08-27T11:30:07.374655+0000","last_clean":"2023-08-27T11:30:07.374655+0000","last_became_active":"2023-08-27T11:30:07.374306+0000","last_became_peered":"2023-08-27T11:30:07.374306+0000","last_unstale":"2023-08-27T11:30:07.374655+0000","last_undegraded":"2023-08-27T11:30:07.374655+0000","last_fullsized":"2023-08-27T11:30:07.374655+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:23.749135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370729+0000","last_change":"2023-08-27T11:30:06.500047+0000","last_active":"2023-08-27T11:30:07.370729+0000","last_peered":"2023-08-27T11:30:07.370729+0000","last_clean":"2023-08-27T11:30:07.370729+0000","last_became_active":"2023-08-27T11:30:06.499775+0000","last_became_peered":"2023-08-27T11:30:06.499775+0000","last_unstale":"2023-08-27T11:30:07.370729+0000","last_undegraded":"2023-08-27T11:30:07.370729+0000","last_fullsized":"2023-08-27T11:30:07.370729+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:13.320294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376417+0000","last_change":"2023-08-27T11:30:07.376417+0000","last_active":"2023-08-27T11:30:07.376417+0000","last_peered":"2023-08-27T11:30:07.376417+0000","last_clean":"2023-08-27T11:30:07.376417+0000","last_became_active":"2023-08-27T11:30:07.376144+0000","last_became_peered":"2023-08-27T11:30:07.376144+0000","last_unstale":"2023-08-27T11:30:07.376417+0000","last_undegraded":"2023-08-27T11:30:07.376417+0000","last_fullsized":"2023-08-27T11:30:07.376417+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:07:51.665760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370869+0000","last_change":"2023-08-27T11:30:06.505423+0000","last_active":"2023-08-27T11:30:07.370869+0000","last_peered":"2023-08-27T11:30:07.370869+0000","last_clean":"2023-08-27T11:30:07.370869+0000","last_became_active":"2023-08-27T11:30:06.505138+0000","last_became_peered":"2023-08-27T11:30:06.505138+0000","last_unstale":"2023-08-27T11:30:07.370869+0000","last_undegraded":"2023-08-27T11:30:07.370869+0000","last_fullsized":"2023-08-27T11:30:07.370869+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:12:32.588899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375665+0000","last_change":"2023-08-27T11:30:07.375665+0000","last_active":"2023-08-27T11:30:07.375665+0000","last_peered":"2023-08-27T11:30:07.375665+0000","last_clean":"2023-08-27T11:30:07.375665+0000","last_became_active":"2023-08-27T11:30:07.375184+0000","last_became_peered":"2023-08-27T11:30:07.375184+0000","last_unstale":"2023-08-27T11:30:07.375665+0000","last_undegraded":"2023-08-27T11:30:07.375665+0000","last_fullsized":"2023-08-27T11:30:07.375665+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:42.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.363224+0000","last_change":"2023-08-27T11:30:06.504990+0000","last_active":"2023-08-27T11:30:07.363224+0000","last_peered":"2023-08-27T11:30:07.363224+0000","last_clean":"2023-08-27T11:30:07.363224+0000","last_became_active":"2023-08-27T11:30:06.504768+0000","last_became_peered":"2023-08-27T11:30:06.504768+0000","last_unstale":"2023-08-27T11:30:07.363224+0000","last_undegraded":"2023-08-27T11:30:07.363224+0000","last_fullsized":"2023-08-27T11:30:07.363224+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:27:04.823447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377092+0000","last_change":"2023-08-27T11:30:07.376099+0000","last_active":"2023-08-27T11:30:07.377092+0000","last_peered":"2023-08-27T11:30:07.377092+0000","last_clean":"2023-08-27T11:30:07.377092+0000","last_became_active":"2023-08-27T11:30:07.375823+0000","last_became_peered":"2023-08-27T11:30:07.375823+0000","last_unstale":"2023-08-27T11:30:07.377092+0000","last_undegraded":"2023-08-27T11:30:07.377092+0000","last_fullsized":"2023-08-27T11:30:07.377092+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:39.725351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.510303+0000","last_change":"2023-08-27T11:30:06.508106+0000","last_active":"2023-08-27T11:30:06.510303+0000","last_peered":"2023-08-27T11:30:06.510303+0000","last_clean":"2023-08-27T11:30:06.510303+0000","last_became_active":"2023-08-27T11:30:06.507721+0000","last_became_peered":"2023-08-27T11:30:06.507721+0000","last_unstale":"2023-08-27T11:30:06.510303+0000","last_undegraded":"2023-08-27T11:30:06.510303+0000","last_fullsized":"2023-08-27T11:30:06.510303+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:21:52.252880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375020+0000","last_change":"2023-08-27T11:30:07.372999+0000","last_active":"2023-08-27T11:30:07.375020+0000","last_peered":"2023-08-27T11:30:07.375020+0000","last_clean":"2023-08-27T11:30:07.375020+0000","last_became_active":"2023-08-27T11:30:07.372727+0000","last_became_peered":"2023-08-27T11:30:07.372727+0000","last_unstale":"2023-08-27T11:30:07.375020+0000","last_undegraded":"2023-08-27T11:30:07.375020+0000","last_fullsized":"2023-08-27T11:30:07.375020+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:53:39.155707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":351,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.362862+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:30:07.362862+0000","last_peered":"2023-08-27T11:30:07.362862+0000","last_clean":"2023-08-27T11:30:07.362862+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:30:07.362862+0000","last_undegraded":"2023-08-27T11:30:07.362862+0000","last_fullsized":"2023-08-27T11:30:07.362862+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:54:33.147243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.382079+0000","last_change":"2023-08-27T11:30:07.379099+0000","last_active":"2023-08-27T11:30:07.382079+0000","last_peered":"2023-08-27T11:30:07.382079+0000","last_clean":"2023-08-27T11:30:07.382079+0000","last_became_active":"2023-08-27T11:30:07.378802+0000","last_became_peered":"2023-08-27T11:30:07.378802+0000","last_unstale":"2023-08-27T11:30:07.382079+0000","last_undegraded":"2023-08-27T11:30:07.382079+0000","last_fullsized":"2023-08-27T11:30:07.382079+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:26.058597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":351,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.364140+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:30:07.364140+0000","last_peered":"2023-08-27T11:30:07.364140+0000","last_clean":"2023-08-27T11:30:07.364140+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:30:07.364140+0000","last_undegraded":"2023-08-27T11:30:07.364140+0000","last_fullsized":"2023-08-27T11:30:07.364140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:59:17.376710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375699+0000","last_change":"2023-08-27T11:30:07.372568+0000","last_active":"2023-08-27T11:30:07.375699+0000","last_peered":"2023-08-27T11:30:07.375699+0000","last_clean":"2023-08-27T11:30:07.375699+0000","last_became_active":"2023-08-27T11:30:07.372289+0000","last_became_peered":"2023-08-27T11:30:07.372289+0000","last_unstale":"2023-08-27T11:30:07.375699+0000","last_undegraded":"2023-08-27T11:30:07.375699+0000","last_fullsized":"2023-08-27T11:30:07.375699+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:10:25.361282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":299,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372055+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:30:07.372055+0000","last_peered":"2023-08-27T11:30:07.372055+0000","last_clean":"2023-08-27T11:30:07.372055+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:30:07.372055+0000","last_undegraded":"2023-08-27T11:30:07.372055+0000","last_fullsized":"2023-08-27T11:30:07.372055+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:32:49.299155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.373549+0000","last_change":"2023-08-27T11:30:07.366404+0000","last_active":"2023-08-27T11:30:07.373549+0000","last_peered":"2023-08-27T11:30:07.373549+0000","last_clean":"2023-08-27T11:30:07.373549+0000","last_became_active":"2023-08-27T11:30:07.366051+0000","last_became_peered":"2023-08-27T11:30:07.366051+0000","last_unstale":"2023-08-27T11:30:07.373549+0000","last_undegraded":"2023-08-27T11:30:07.373549+0000","last_fullsized":"2023-08-27T11:30:07.373549+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:53:30.711567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":297,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372131+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:30:07.372131+0000","last_peered":"2023-08-27T11:30:07.372131+0000","last_clean":"2023-08-27T11:30:07.372131+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:30:07.372131+0000","last_undegraded":"2023-08-27T11:30:07.372131+0000","last_fullsized":"2023-08-27T11:30:07.372131+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:36:26.335579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370464+0000","last_change":"2023-08-27T11:30:07.367607+0000","last_active":"2023-08-27T11:30:07.370464+0000","last_peered":"2023-08-27T11:30:07.370464+0000","last_clean":"2023-08-27T11:30:07.370464+0000","last_became_active":"2023-08-27T11:30:07.367368+0000","last_became_peered":"2023-08-27T11:30:07.367368+0000","last_unstale":"2023-08-27T11:30:07.370464+0000","last_undegraded":"2023-08-27T11:30:07.370464+0000","last_fullsized":"2023-08-27T11:30:07.370464+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:39.373129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":297,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372428+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:30:07.372428+0000","last_peered":"2023-08-27T11:30:07.372428+0000","last_clean":"2023-08-27T11:30:07.372428+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:30:07.372428+0000","last_undegraded":"2023-08-27T11:30:07.372428+0000","last_fullsized":"2023-08-27T11:30:07.372428+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:55:02.343378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.496946+0000","last_change":"2023-08-27T11:30:06.496946+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.496946+0000","last_undegraded":"2023-08-27T11:30:06.496946+0000","last_fullsized":"2023-08-27T11:30:06.496946+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":447,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372029+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:30:07.372029+0000","last_peered":"2023-08-27T11:30:07.372029+0000","last_clean":"2023-08-27T11:30:07.372029+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:30:07.372029+0000","last_undegraded":"2023-08-27T11:30:07.372029+0000","last_fullsized":"2023-08-27T11:30:07.372029+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:04.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":305,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.374061+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:30:07.374061+0000","last_peered":"2023-08-27T11:30:07.374061+0000","last_clean":"2023-08-27T11:30:07.374061+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:30:07.374061+0000","last_undegraded":"2023-08-27T11:30:07.374061+0000","last_fullsized":"2023-08-27T11:30:07.374061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:46:35.266946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376873+0000","last_change":"2023-08-27T11:30:07.376873+0000","last_active":"2023-08-27T11:30:07.376873+0000","last_peered":"2023-08-27T11:30:07.376873+0000","last_clean":"2023-08-27T11:30:07.376873+0000","last_became_active":"2023-08-27T11:30:07.376104+0000","last_became_peered":"2023-08-27T11:30:07.376104+0000","last_unstale":"2023-08-27T11:30:07.376873+0000","last_undegraded":"2023-08-27T11:30:07.376873+0000","last_fullsized":"2023-08-27T11:30:07.376873+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:48:39.739503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":295,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.497876+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:30:06.497876+0000","last_peered":"2023-08-27T11:30:06.497876+0000","last_clean":"2023-08-27T11:30:06.497876+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:30:06.497876+0000","last_undegraded":"2023-08-27T11:30:06.497876+0000","last_fullsized":"2023-08-27T11:30:06.497876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:17.897409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380860+0000","last_change":"2023-08-27T11:30:07.375200+0000","last_active":"2023-08-27T11:30:07.380860+0000","last_peered":"2023-08-27T11:30:07.380860+0000","last_clean":"2023-08-27T11:30:07.380860+0000","last_became_active":"2023-08-27T11:30:07.374915+0000","last_became_peered":"2023-08-27T11:30:07.374915+0000","last_unstale":"2023-08-27T11:30:07.380860+0000","last_undegraded":"2023-08-27T11:30:07.380860+0000","last_fullsized":"2023-08-27T11:30:07.380860+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:32:43.401182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":335,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.366365+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:30:07.366365+0000","last_peered":"2023-08-27T11:30:07.366365+0000","last_clean":"2023-08-27T11:30:07.366365+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:30:07.366365+0000","last_undegraded":"2023-08-27T11:30:07.366365+0000","last_fullsized":"2023-08-27T11:30:07.366365+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:39:06.252428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34036,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337804,"statfs":{"total":96636764160,"available":96601911296,"internally_reserved":0,"allocated":2920448,"data_stored":1702685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.22700000000000001},"max":{"1min":0.60399999999999998,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.753},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.51500000000000001},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.68300000000000005}]},{"osd":1,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.83399999999999996,"15min":0.872},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.72399999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.42799999999999999,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.67800000000000005,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.623},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.90200000000000002},"last":0.66500000000000004}]},{"osd":3,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":0.996,"15min":0.996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.53000000000000003,"5min":0.42899999999999999,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.71099999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.503},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.72199999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":1.224,"15min":1.224},"last":0.55000000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79700000000000004,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.84799999999999998},"last":0.64100000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83799999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.73399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575011,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.84599999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.39800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.89800000000000002},"last":0.48799999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.53900000000000003},"min":{"1min":0.35599999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.81799999999999995,"15min":0.873},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55400000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.499,"15min":0.51000000000000001},"min":{"1min":0.40799999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":2.0830000000000002},"last":0.72599999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.83199999999999996,"15min":0.84499999999999997},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.34000000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.93799999999999994},"last":0.626}]},{"osd":4,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":0.71599999999999997,"5min":0.753,"15min":0.83099999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.77000000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75700000000000001,"5min":0.85499999999999998,"15min":0.875},"last":0.78200000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79200000000000004,"5min":1.365,"15min":1.365},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.34899999999999998},"max":{"1min":0.77700000000000002,"5min":0.82999999999999996,"15min":0.91400000000000003},"last":0.753}]}]},{"osd":1,"up_from":13,"seq":55834575010,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34020,"kb_used_data":2840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337820,"statfs":{"total":96636764160,"available":96601927680,"internally_reserved":0,"allocated":2908160,"data_stored":1695557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.315,"15min":0.47399999999999998},"min":{"1min":0.21199999999999999,"5min":0.19500000000000001,"15min":0.191},"max":{"1min":0.45300000000000001,"5min":0.53300000000000003,"15min":11.971},"last":0.254},{"interface":"front","average":{"1min":0.315,"5min":0.32700000000000001,"15min":0.64700000000000002},"min":{"1min":0.23000000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.44900000000000001,"5min":0.74399999999999999,"15min":23.045999999999999},"last":0.313}]},{"osd":2,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34000000000000002,"15min":0.502},"min":{"1min":0.28199999999999997,"5min":0.23300000000000001,"15min":0.221},"max":{"1min":0.42399999999999999,"5min":0.57699999999999996,"15min":11.928000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.33700000000000002},"min":{"1min":0.249,"5min":0.22800000000000001,"15min":0.224},"max":{"1min":0.46899999999999997,"5min":0.52200000000000002,"15min":0.58599999999999997},"last":0.29599999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38400000000000001,"15min":0.52800000000000002},"min":{"1min":0.309,"5min":0.27500000000000002,"15min":0.26300000000000001},"max":{"1min":0.47599999999999998,"5min":0.64100000000000001,"15min":10.99},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36799999999999999,"15min":0.68500000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.503,"5min":0.54700000000000004,"15min":22.893999999999998},"last":0.35799999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.33400000000000002},"max":{"1min":0.84999999999999998,"5min":0.90700000000000003,"15min":1.26},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58099999999999996,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.745,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.62}]},{"osd":5,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.88600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":23.016999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.81899999999999995,"5min":0.88700000000000001,"15min":1.296},"last":0.66500000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58199999999999996,"15min":0.72799999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.371},"max":{"1min":0.72999999999999998,"5min":0.84899999999999998,"15min":10.925000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60199999999999998,"15min":0.90400000000000003},"min":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88,"5min":0.88,"15min":22.952000000000002},"last":0.70999999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59299999999999997,"15min":0.89800000000000002},"min":{"1min":0.433,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":22.988},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":1.327},"last":0.56999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2310144,"data_stored":1103828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.215,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.497,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35999999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36199999999999999,"15min":0.34399999999999997},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47699999999999998,"5min":0.628,"15min":0.63200000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34799999999999998,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45800000000000002,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.34599999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35799999999999998,"15min":0.34599999999999997},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.53700000000000003,"15min":0.59599999999999997},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.376},"min":{"1min":0.30499999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.53000000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.315}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.34399999999999997},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.214},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.65700000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.603},"last":0.62}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79600000000000004,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.878,"15min":0.878},"last":0.60399999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83399999999999996,"5min":3.79,"15min":3.79},"last":0.63800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32100000000000001,"15min":0.318},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.19700000000000001},"max":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.629},"last":0.32300000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41699999999999998,"5min":0.50800000000000001,"15min":0.54400000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34699999999999998},"min":{"1min":0.26200000000000001,"5min":0.23000000000000001,"15min":0.22600000000000001},"max":{"1min":0.49299999999999999,"5min":0.56399999999999995,"15min":0.749},"last":0.36099999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.378},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":0.51800000000000002,"5min":0.57099999999999995,"15min":0.73699999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.32600000000000001,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.55100000000000005,"15min":0.69499999999999995},"last":0.39900000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.879},"last":0.45700000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36699999999999999},"max":{"1min":0.83799999999999997,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.52800000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.40000000000000002},"max":{"1min":0.78600000000000003,"5min":0.82699999999999996,"15min":0.83599999999999997},"last":0.47399999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.46600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.29999999999999999,"15min":0.30299999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.52500000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.307,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.223,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40899999999999997,"5min":0.755,"15min":0.755},"last":0.36499999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33500000000000002,"15min":0.32300000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":0.40300000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":0.48999999999999999,"5min":2.73,"15min":2.73},"last":0.34799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.372,"15min":0.36399999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.46400000000000002,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.375,"15min":0.37},"min":{"1min":0.27700000000000002,"5min":0.26700000000000002,"15min":0.26400000000000001},"max":{"1min":0.51000000000000001,"5min":1.179,"15min":1.179},"last":0.44900000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.28599999999999998},"max":{"1min":0.748,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.437},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.71799999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54100000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.57699999999999996},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.54900000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.373},"max":{"1min":0.84299999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.497,"5min":0.38,"15min":0.38},"max":{"1min":0.83199999999999996,"5min":0.86899999999999999,"15min":1.111},"last":0.52600000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":1.236},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.499}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68200000000000005,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.502},"min":{"1min":0.33900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57399999999999995}]},{"osd":1,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66100000000000003,"5min":0.746,"15min":0.76500000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.75,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.376}]},{"osd":2,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.40400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.55700000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.96999999999999997,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59899999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71499999999999997,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":1.865,"15min":1.865},"last":0.61799999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90600000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.624,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82799999999999996,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.58799999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.90100000000000002,"5min":2.714,"15min":2.714},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.64500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575010,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.376,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61099999999999999,"5min":0.78900000000000003,"15min":0.96899999999999997},"last":0.38},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.44700000000000001,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.437}]},{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67500000000000004,"5min":0.68400000000000005,"15min":1.075},"last":0.33500000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.373,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.0580000000000001},"last":0.35399999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":1.03},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.42199999999999999,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.77800000000000002,"15min":1.026},"last":0.39400000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.27000000000000002},"max":{"1min":0.63900000000000001,"5min":0.78700000000000003,"15min":0.86899999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.73399999999999999,"5min":0.89400000000000002,"15min":0.91400000000000003},"last":0.51500000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.31},"max":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.88200000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.873,"15min":0.94199999999999995},"last":0.46000000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.34799999999999998},"max":{"1min":0.753,"5min":0.85999999999999999,"15min":0.95899999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.84399999999999997,"15min":0.92000000000000004},"last":0.63400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:30:11.613 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:30:11.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.766+0000 7f625a3b9700 1 Processor -- start 2023-08-27T11:30:11.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.766+0000 7f625a3b9700 1 -- start start 2023-08-27T11:30:11.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.766+0000 7f625a3b9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6254158e20 0x7f62540b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.766+0000 7f625a3b9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f625406e3a0 0x7f625406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.770 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.766+0000 7f625a3b9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f625415c190 con 0x7f625406fcd0 2023-08-27T11:30:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.766+0000 7f625a3b9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62540b1730 con 0x7f6254158e20 2023-08-27T11:30:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.766+0000 7f625a3b9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6254070160 con 0x7f625406e3a0 2023-08-27T11:30:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f6253fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6254158e20 0x7f62540b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f6258956700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f625406fcd0 0x7f625415ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33760/0 (socket says 172.21.15.112:33760) 2023-08-27T11:30:11.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f6258956700 1 -- 172.21.15.112:0/1833716563 learned_addr learned my addr 172.21.15.112:0/1833716563 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:11.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f62537fe700 1 --2- 172.21.15.112:0/1833716563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f625406e3a0 0x7f625406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f6253fff700 1 -- 172.21.15.112:0/1833716563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f625406e3a0 msgr2=0x7f625406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:11.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f6253fff700 1 --2- 172.21.15.112:0/1833716563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f625406e3a0 0x7f625406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f6253fff700 1 -- 172.21.15.112:0/1833716563 >> v1:172.21.15.112:6789/0 conn(0x7f625406fcd0 legacy=0x7f625415ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f6252ffd700 1 -- 172.21.15.112:0/1833716563 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130400731 0 0) 0x7f625415c190 con 0x7f625406fcd0 2023-08-27T11:30:11.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f6253fff700 1 -- 172.21.15.112:0/1833716563 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62540b09d0 con 0x7f6254158e20 2023-08-27T11:30:11.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f62537fe700 1 --2- 172.21.15.112:0/1833716563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f625406e3a0 0x7f625406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:11.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f6253fff700 1 --2- 172.21.15.112:0/1833716563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254158e20 0x7f62540b11f0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f6244009840 tx=0x7f6244009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cdc5de0fbf996a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:11.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f6252ffd700 1 -- 172.21.15.112:0/1833716563 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6244011830 con 0x7f6254158e20 2023-08-27T11:30:11.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f6252ffd700 1 -- 172.21.15.112:0/1833716563 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6244011990 con 0x7f6254158e20 2023-08-27T11:30:11.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.767+0000 7f6252ffd700 1 -- 172.21.15.112:0/1833716563 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6244011d20 con 0x7f6254158e20 2023-08-27T11:30:11.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.768+0000 7f625a3b9700 1 -- 172.21.15.112:0/1833716563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254158e20 msgr2=0x7f62540b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.768+0000 7f625a3b9700 1 --2- 172.21.15.112:0/1833716563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254158e20 0x7f62540b11f0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f6244009840 tx=0x7f6244009b50 comp rx=0 tx=0).stop 2023-08-27T11:30:11.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.768+0000 7f625a3b9700 1 -- 172.21.15.112:0/1833716563 shutdown_connections 2023-08-27T11:30:11.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.768+0000 7f625a3b9700 1 --2- 172.21.15.112:0/1833716563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254158e20 0x7f62540b11f0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.768+0000 7f625a3b9700 1 --2- 172.21.15.112:0/1833716563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f625406e3a0 0x7f625406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.768+0000 7f625a3b9700 1 -- 172.21.15.112:0/1833716563 >> 172.21.15.112:0/1833716563 conn(0x7f62540d0000 msgr2=0x7f62540d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:11.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.768+0000 7f625a3b9700 1 -- 172.21.15.112:0/1833716563 shutdown_connections 2023-08-27T11:30:11.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.768+0000 7f625a3b9700 1 -- 172.21.15.112:0/1833716563 wait complete. 2023-08-27T11:30:11.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.769+0000 7f625a3b9700 1 Processor -- start 2023-08-27T11:30:11.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.769+0000 7f625a3b9700 1 -- start start 2023-08-27T11:30:11.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f625a3b9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f625406e3a0 0x7f62541efd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.779 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f625a3b9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f625406fcd0 0x7f62541f0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f625a3b9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6254158e20 0x7f6254146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f625a3b9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62541f0780 con 0x7f625406e3a0 2023-08-27T11:30:11.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f625a3b9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62540c64f0 con 0x7f6254158e20 2023-08-27T11:30:11.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f625a3b9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625414a4e0 con 0x7f625406fcd0 2023-08-27T11:30:11.780 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f6253fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f625406e3a0 0x7f62541efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f6253fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f625406e3a0 0x7f62541efd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39384/0 (socket says 172.21.15.112:39384) 2023-08-27T11:30:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f6253fff700 1 -- 172.21.15.112:0/3483126859 learned_addr learned my addr 172.21.15.112:0/3483126859 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f6258956700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6254158e20 0x7f6254146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f62537fe700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f625406fcd0 0x7f62541f0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f6253fff700 1 -- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f625406fcd0 msgr2=0x7f62541f0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f6253fff700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f625406fcd0 0x7f62541f0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f6253fff700 1 -- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6254158e20 msgr2=0x7f6254146240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f6253fff700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6254158e20 0x7f6254146240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f6253fff700 1 -- 172.21.15.112:0/3483126859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440094a0 con 0x7f625406e3a0 2023-08-27T11:30:11.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f6258956700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6254158e20 0x7f6254146240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:11.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.770+0000 7f6253fff700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f625406e3a0 0x7f62541efd00 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f6244000c00 tx=0x7f6244004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:11.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.771+0000 7f6250ff9700 1 -- 172.21.15.112:0/3483126859 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62440038a0 con 0x7f625406e3a0 2023-08-27T11:30:11.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.771+0000 7f6250ff9700 1 -- 172.21.15.112:0/3483126859 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624401e070 con 0x7f625406e3a0 2023-08-27T11:30:11.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.771+0000 7f625a3b9700 1 -- 172.21.15.112:0/3483126859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62540c5730 con 0x7f625406e3a0 2023-08-27T11:30:11.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.771+0000 7f6250ff9700 1 -- 172.21.15.112:0/3483126859 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6244003c40 con 0x7f625406e3a0 2023-08-27T11:30:11.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.771+0000 7f625a3b9700 1 -- 172.21.15.112:0/3483126859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62540c5cf0 con 0x7f625406e3a0 2023-08-27T11:30:11.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.772+0000 7f625a3b9700 1 -- 172.21.15.112:0/3483126859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6254067a20 con 0x7f625406e3a0 2023-08-27T11:30:11.784 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.772+0000 7f6250ff9700 1 -- 172.21.15.112:0/3483126859 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f624401a420 con 0x7f625406e3a0 2023-08-27T11:30:11.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.773+0000 7f6250ff9700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f624006a4d0 0x7f624006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.773+0000 7f6250ff9700 1 -- 172.21.15.112:0/3483126859 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6244089cc0 con 0x7f625406e3a0 2023-08-27T11:30:11.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.773+0000 7f62537fe700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f624006a4d0 0x7f624006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.773+0000 7f62537fe700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f624006a4d0 0x7f624006c990 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f6248006fd0 tx=0x7f6248008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:11.785 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.777+0000 7f6250ff9700 1 -- 172.21.15.112:0/3483126859 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6244090080 con 0x7f625406e3a0 2023-08-27T11:30:11.961 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.960+0000 7f625a3b9700 1 -- 172.21.15.112:0/3483126859 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62540b1910 con 0x7f624006a4d0 2023-08-27T11:30:11.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.969+0000 7f6250ff9700 1 -- 172.21.15.112:0/3483126859 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310726 (secure 0 0 0) 0x7f624800c040 con 0x7f624006a4d0 2023-08-27T11:30:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:30:11.976 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.972+0000 7f625a3b9700 1 -- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f624006a4d0 msgr2=0x7f624006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.972+0000 7f625a3b9700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f624006a4d0 0x7f624006c990 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f6248006fd0 tx=0x7f6248008040 comp rx=0 tx=0).stop 2023-08-27T11:30:11.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.972+0000 7f625a3b9700 1 -- 172.21.15.112:0/3483126859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f625406e3a0 msgr2=0x7f62541efd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.972+0000 7f625a3b9700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f625406e3a0 0x7f62541efd00 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f6244000c00 tx=0x7f6244004750 comp rx=0 tx=0).stop 2023-08-27T11:30:11.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.973+0000 7f625a3b9700 1 -- 172.21.15.112:0/3483126859 shutdown_connections 2023-08-27T11:30:11.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.973+0000 7f625a3b9700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f624006a4d0 0x7f624006c990 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.973+0000 7f625a3b9700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6254158e20 0x7f6254146240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.973+0000 7f625a3b9700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f625406e3a0 0x7f62541efd00 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.973+0000 7f625a3b9700 1 --2- 172.21.15.112:0/3483126859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f625406fcd0 0x7f62541f0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.973+0000 7f625a3b9700 1 -- 172.21.15.112:0/3483126859 >> 172.21.15.112:0/3483126859 conn(0x7f62540d0000 msgr2=0x7f625406ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:11.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.973+0000 7f625a3b9700 1 -- 172.21.15.112:0/3483126859 shutdown_connections 2023-08-27T11:30:11.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:11.973+0000 7f625a3b9700 1 -- 172.21.15.112:0/3483126859 wait complete. 2023-08-27T11:30:11.979 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:30:11.986 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":574,"stamp":"2023-08-27T11:30:10.427283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268692,"kb_used_data":19248,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754706028,"statfs":{"total":773094113280,"available":772818972672,"internally_reserved":0,"allocated":19709952,"data_stored":10028015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.937212"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375737+0000","last_change":"2023-08-27T11:30:07.373935+0000","last_active":"2023-08-27T11:30:07.375737+0000","last_peered":"2023-08-27T11:30:07.375737+0000","last_clean":"2023-08-27T11:30:07.375737+0000","last_became_active":"2023-08-27T11:30:07.373619+0000","last_became_peered":"2023-08-27T11:30:07.373619+0000","last_unstale":"2023-08-27T11:30:07.375737+0000","last_undegraded":"2023-08-27T11:30:07.375737+0000","last_fullsized":"2023-08-27T11:30:07.375737+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:16.699227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.384847+0000","last_change":"2023-08-27T11:30:07.380950+0000","last_active":"2023-08-27T11:30:07.384847+0000","last_peered":"2023-08-27T11:30:07.384847+0000","last_clean":"2023-08-27T11:30:07.384847+0000","last_became_active":"2023-08-27T11:30:07.380668+0000","last_became_peered":"2023-08-27T11:30:07.380668+0000","last_unstale":"2023-08-27T11:30:07.384847+0000","last_undegraded":"2023-08-27T11:30:07.384847+0000","last_fullsized":"2023-08-27T11:30:07.384847+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:27:15.104758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.387295+0000","last_change":"2023-08-27T11:30:07.383032+0000","last_active":"2023-08-27T11:30:07.387295+0000","last_peered":"2023-08-27T11:30:07.387295+0000","last_clean":"2023-08-27T11:30:07.387295+0000","last_became_active":"2023-08-27T11:30:07.382798+0000","last_became_peered":"2023-08-27T11:30:07.382798+0000","last_unstale":"2023-08-27T11:30:07.387295+0000","last_undegraded":"2023-08-27T11:30:07.387295+0000","last_fullsized":"2023-08-27T11:30:07.387295+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:06:13.527429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.379831+0000","last_change":"2023-08-27T11:30:07.378918+0000","last_active":"2023-08-27T11:30:07.379831+0000","last_peered":"2023-08-27T11:30:07.379831+0000","last_clean":"2023-08-27T11:30:07.379831+0000","last_became_active":"2023-08-27T11:30:07.378583+0000","last_became_peered":"2023-08-27T11:30:07.378583+0000","last_unstale":"2023-08-27T11:30:07.379831+0000","last_undegraded":"2023-08-27T11:30:07.379831+0000","last_fullsized":"2023-08-27T11:30:07.379831+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:17:10.066132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376893+0000","last_change":"2023-08-27T11:30:07.375355+0000","last_active":"2023-08-27T11:30:07.376893+0000","last_peered":"2023-08-27T11:30:07.376893+0000","last_clean":"2023-08-27T11:30:07.376893+0000","last_became_active":"2023-08-27T11:30:07.375064+0000","last_became_peered":"2023-08-27T11:30:07.375064+0000","last_unstale":"2023-08-27T11:30:07.376893+0000","last_undegraded":"2023-08-27T11:30:07.376893+0000","last_fullsized":"2023-08-27T11:30:07.376893+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:11:07.185199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.378756+0000","last_change":"2023-08-27T11:30:07.378756+0000","last_active":"2023-08-27T11:30:07.378756+0000","last_peered":"2023-08-27T11:30:07.378756+0000","last_clean":"2023-08-27T11:30:07.378756+0000","last_became_active":"2023-08-27T11:30:07.378495+0000","last_became_peered":"2023-08-27T11:30:07.378495+0000","last_unstale":"2023-08-27T11:30:07.378756+0000","last_undegraded":"2023-08-27T11:30:07.378756+0000","last_fullsized":"2023-08-27T11:30:07.378756+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:01:44.098335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380828+0000","last_change":"2023-08-27T11:30:07.377440+0000","last_active":"2023-08-27T11:30:07.380828+0000","last_peered":"2023-08-27T11:30:07.380828+0000","last_clean":"2023-08-27T11:30:07.380828+0000","last_became_active":"2023-08-27T11:30:07.376823+0000","last_became_peered":"2023-08-27T11:30:07.376823+0000","last_unstale":"2023-08-27T11:30:07.380828+0000","last_undegraded":"2023-08-27T11:30:07.380828+0000","last_fullsized":"2023-08-27T11:30:07.380828+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:38:41.760242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380236+0000","last_change":"2023-08-27T11:30:07.377813+0000","last_active":"2023-08-27T11:30:07.380236+0000","last_peered":"2023-08-27T11:30:07.380236+0000","last_clean":"2023-08-27T11:30:07.380236+0000","last_became_active":"2023-08-27T11:30:07.377471+0000","last_became_peered":"2023-08-27T11:30:07.377471+0000","last_unstale":"2023-08-27T11:30:07.380236+0000","last_undegraded":"2023-08-27T11:30:07.380236+0000","last_fullsized":"2023-08-27T11:30:07.380236+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:35:22.802502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.368157+0000","last_change":"2023-08-27T11:30:07.368157+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.368157+0000","last_undegraded":"2023-08-27T11:30:07.368157+0000","last_fullsized":"2023-08-27T11:30:07.368157+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:48.531209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.382327+0000","last_change":"2023-08-27T11:30:07.378494+0000","last_active":"2023-08-27T11:30:07.382327+0000","last_peered":"2023-08-27T11:30:07.382327+0000","last_clean":"2023-08-27T11:30:07.382327+0000","last_became_active":"2023-08-27T11:30:07.378196+0000","last_became_peered":"2023-08-27T11:30:07.378196+0000","last_unstale":"2023-08-27T11:30:07.382327+0000","last_undegraded":"2023-08-27T11:30:07.382327+0000","last_fullsized":"2023-08-27T11:30:07.382327+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:01:59.361767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.387354+0000","last_change":"2023-08-27T11:30:07.382784+0000","last_active":"2023-08-27T11:30:07.387354+0000","last_peered":"2023-08-27T11:30:07.387354+0000","last_clean":"2023-08-27T11:30:07.387354+0000","last_became_active":"2023-08-27T11:30:07.382436+0000","last_became_peered":"2023-08-27T11:30:07.382436+0000","last_unstale":"2023-08-27T11:30:07.387354+0000","last_undegraded":"2023-08-27T11:30:07.387354+0000","last_fullsized":"2023-08-27T11:30:07.387354+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:15:13.361952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375762+0000","last_change":"2023-08-27T11:30:07.372273+0000","last_active":"2023-08-27T11:30:07.375762+0000","last_peered":"2023-08-27T11:30:07.375762+0000","last_clean":"2023-08-27T11:30:07.375762+0000","last_became_active":"2023-08-27T11:30:07.371943+0000","last_became_peered":"2023-08-27T11:30:07.371943+0000","last_unstale":"2023-08-27T11:30:07.375762+0000","last_undegraded":"2023-08-27T11:30:07.375762+0000","last_fullsized":"2023-08-27T11:30:07.375762+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:06:45.150286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.363359+0000","last_change":"2023-08-27T11:30:07.363359+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.363359+0000","last_undegraded":"2023-08-27T11:30:07.363359+0000","last_fullsized":"2023-08-27T11:30:07.363359+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:05:25.871181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370483+0000","last_change":"2023-08-27T11:30:07.367703+0000","last_active":"2023-08-27T11:30:07.370483+0000","last_peered":"2023-08-27T11:30:07.370483+0000","last_clean":"2023-08-27T11:30:07.370483+0000","last_became_active":"2023-08-27T11:30:07.367443+0000","last_became_peered":"2023-08-27T11:30:07.367443+0000","last_unstale":"2023-08-27T11:30:07.370483+0000","last_undegraded":"2023-08-27T11:30:07.370483+0000","last_fullsized":"2023-08-27T11:30:07.370483+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:40:42.999295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376506+0000","last_change":"2023-08-27T11:30:07.376506+0000","last_active":"2023-08-27T11:30:07.376506+0000","last_peered":"2023-08-27T11:30:07.376506+0000","last_clean":"2023-08-27T11:30:07.376506+0000","last_became_active":"2023-08-27T11:30:07.376242+0000","last_became_peered":"2023-08-27T11:30:07.376242+0000","last_unstale":"2023-08-27T11:30:07.376506+0000","last_undegraded":"2023-08-27T11:30:07.376506+0000","last_fullsized":"2023-08-27T11:30:07.376506+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:41:53.411101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377343+0000","last_change":"2023-08-27T11:30:07.377343+0000","last_active":"2023-08-27T11:30:07.377343+0000","last_peered":"2023-08-27T11:30:07.377343+0000","last_clean":"2023-08-27T11:30:07.377343+0000","last_became_active":"2023-08-27T11:30:07.377032+0000","last_became_peered":"2023-08-27T11:30:07.377032+0000","last_unstale":"2023-08-27T11:30:07.377343+0000","last_undegraded":"2023-08-27T11:30:07.377343+0000","last_fullsized":"2023-08-27T11:30:07.377343+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:53:51.803335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.496034+0000","last_change":"2023-08-27T11:30:06.496034+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.496034+0000","last_undegraded":"2023-08-27T11:30:06.496034+0000","last_fullsized":"2023-08-27T11:30:06.496034+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.382296+0000","last_change":"2023-08-27T11:30:07.376652+0000","last_active":"2023-08-27T11:30:07.382296+0000","last_peered":"2023-08-27T11:30:07.382296+0000","last_clean":"2023-08-27T11:30:07.382296+0000","last_became_active":"2023-08-27T11:30:07.376381+0000","last_became_peered":"2023-08-27T11:30:07.376381+0000","last_unstale":"2023-08-27T11:30:07.382296+0000","last_undegraded":"2023-08-27T11:30:07.382296+0000","last_fullsized":"2023-08-27T11:30:07.382296+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:47:41.397698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380879+0000","last_change":"2023-08-27T11:30:07.377266+0000","last_active":"2023-08-27T11:30:07.380879+0000","last_peered":"2023-08-27T11:30:07.380879+0000","last_clean":"2023-08-27T11:30:07.380879+0000","last_became_active":"2023-08-27T11:30:07.376986+0000","last_became_peered":"2023-08-27T11:30:07.376986+0000","last_unstale":"2023-08-27T11:30:07.380879+0000","last_undegraded":"2023-08-27T11:30:07.380879+0000","last_fullsized":"2023-08-27T11:30:07.380879+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:46:44.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376491+0000","last_change":"2023-08-27T11:30:07.376491+0000","last_active":"2023-08-27T11:30:07.376491+0000","last_peered":"2023-08-27T11:30:07.376491+0000","last_clean":"2023-08-27T11:30:07.376491+0000","last_became_active":"2023-08-27T11:30:07.376247+0000","last_became_peered":"2023-08-27T11:30:07.376247+0000","last_unstale":"2023-08-27T11:30:07.376491+0000","last_undegraded":"2023-08-27T11:30:07.376491+0000","last_fullsized":"2023-08-27T11:30:07.376491+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:09:11.582134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.368242+0000","last_change":"2023-08-27T11:30:07.368242+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.368242+0000","last_undegraded":"2023-08-27T11:30:07.368242+0000","last_fullsized":"2023-08-27T11:30:07.368242+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:46:21.277652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.369379+0000","last_change":"2023-08-27T11:30:07.369379+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.369379+0000","last_undegraded":"2023-08-27T11:30:07.369379+0000","last_fullsized":"2023-08-27T11:30:07.369379+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:18:50.569923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380789+0000","last_change":"2023-08-27T11:30:07.377088+0000","last_active":"2023-08-27T11:30:07.380789+0000","last_peered":"2023-08-27T11:30:07.380789+0000","last_clean":"2023-08-27T11:30:07.380789+0000","last_became_active":"2023-08-27T11:30:07.376059+0000","last_became_peered":"2023-08-27T11:30:07.376059+0000","last_unstale":"2023-08-27T11:30:07.380789+0000","last_undegraded":"2023-08-27T11:30:07.380789+0000","last_fullsized":"2023-08-27T11:30:07.380789+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:57.640378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.495396+0000","last_change":"2023-08-27T11:30:06.495396+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.495396+0000","last_undegraded":"2023-08-27T11:30:06.495396+0000","last_fullsized":"2023-08-27T11:30:06.495396+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375518+0000","last_change":"2023-08-27T11:30:07.370865+0000","last_active":"2023-08-27T11:30:07.375518+0000","last_peered":"2023-08-27T11:30:07.375518+0000","last_clean":"2023-08-27T11:30:07.375518+0000","last_became_active":"2023-08-27T11:30:07.370571+0000","last_became_peered":"2023-08-27T11:30:07.370571+0000","last_unstale":"2023-08-27T11:30:07.375518+0000","last_undegraded":"2023-08-27T11:30:07.375518+0000","last_fullsized":"2023-08-27T11:30:07.375518+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:45:40.891269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.511830+0000","last_change":"2023-08-27T11:30:06.511151+0000","last_active":"2023-08-27T11:30:06.511830+0000","last_peered":"2023-08-27T11:30:06.511830+0000","last_clean":"2023-08-27T11:30:06.511830+0000","last_became_active":"2023-08-27T11:30:06.510830+0000","last_became_peered":"2023-08-27T11:30:06.510830+0000","last_unstale":"2023-08-27T11:30:06.511830+0000","last_undegraded":"2023-08-27T11:30:06.511830+0000","last_fullsized":"2023-08-27T11:30:06.511830+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:26:30.060104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.378015+0000","last_change":"2023-08-27T11:30:07.378015+0000","last_active":"2023-08-27T11:30:07.378015+0000","last_peered":"2023-08-27T11:30:07.378015+0000","last_clean":"2023-08-27T11:30:07.378015+0000","last_became_active":"2023-08-27T11:30:07.377749+0000","last_became_peered":"2023-08-27T11:30:07.377749+0000","last_unstale":"2023-08-27T11:30:07.378015+0000","last_undegraded":"2023-08-27T11:30:07.378015+0000","last_fullsized":"2023-08-27T11:30:07.378015+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:40.421055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.365033+0000","last_change":"2023-08-27T11:30:06.506992+0000","last_active":"2023-08-27T11:30:07.365033+0000","last_peered":"2023-08-27T11:30:07.365033+0000","last_clean":"2023-08-27T11:30:07.365033+0000","last_became_active":"2023-08-27T11:30:06.506671+0000","last_became_peered":"2023-08-27T11:30:06.506671+0000","last_unstale":"2023-08-27T11:30:07.365033+0000","last_undegraded":"2023-08-27T11:30:07.365033+0000","last_fullsized":"2023-08-27T11:30:07.365033+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:07:13.889636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.387319+0000","last_change":"2023-08-27T11:30:07.383160+0000","last_active":"2023-08-27T11:30:07.387319+0000","last_peered":"2023-08-27T11:30:07.387319+0000","last_clean":"2023-08-27T11:30:07.387319+0000","last_became_active":"2023-08-27T11:30:07.382823+0000","last_became_peered":"2023-08-27T11:30:07.382823+0000","last_unstale":"2023-08-27T11:30:07.387319+0000","last_undegraded":"2023-08-27T11:30:07.387319+0000","last_fullsized":"2023-08-27T11:30:07.387319+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:13.020805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377069+0000","last_change":"2023-08-27T11:30:06.509202+0000","last_active":"2023-08-27T11:30:07.377069+0000","last_peered":"2023-08-27T11:30:07.377069+0000","last_clean":"2023-08-27T11:30:07.377069+0000","last_became_active":"2023-08-27T11:30:06.508960+0000","last_became_peered":"2023-08-27T11:30:06.508960+0000","last_unstale":"2023-08-27T11:30:07.377069+0000","last_undegraded":"2023-08-27T11:30:07.377069+0000","last_fullsized":"2023-08-27T11:30:07.377069+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:41:59.638609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375837+0000","last_change":"2023-08-27T11:30:07.373389+0000","last_active":"2023-08-27T11:30:07.375837+0000","last_peered":"2023-08-27T11:30:07.375837+0000","last_clean":"2023-08-27T11:30:07.375837+0000","last_became_active":"2023-08-27T11:30:07.373083+0000","last_became_peered":"2023-08-27T11:30:07.373083+0000","last_unstale":"2023-08-27T11:30:07.375837+0000","last_undegraded":"2023-08-27T11:30:07.375837+0000","last_fullsized":"2023-08-27T11:30:07.375837+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:01:00.108886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.374018+0000","last_change":"2023-08-27T11:30:06.509017+0000","last_active":"2023-08-27T11:30:07.374018+0000","last_peered":"2023-08-27T11:30:07.374018+0000","last_clean":"2023-08-27T11:30:07.374018+0000","last_became_active":"2023-08-27T11:30:06.508619+0000","last_became_peered":"2023-08-27T11:30:06.508619+0000","last_unstale":"2023-08-27T11:30:07.374018+0000","last_undegraded":"2023-08-27T11:30:07.374018+0000","last_fullsized":"2023-08-27T11:30:07.374018+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:01:51.459390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.368141+0000","last_change":"2023-08-27T11:30:07.368141+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.368141+0000","last_undegraded":"2023-08-27T11:30:07.368141+0000","last_fullsized":"2023-08-27T11:30:07.368141+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:28:48.644407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.362633+0000","last_change":"2023-08-27T11:30:06.503177+0000","last_active":"2023-08-27T11:30:07.362633+0000","last_peered":"2023-08-27T11:30:07.362633+0000","last_clean":"2023-08-27T11:30:07.362633+0000","last_became_active":"2023-08-27T11:30:06.502915+0000","last_became_peered":"2023-08-27T11:30:06.502915+0000","last_unstale":"2023-08-27T11:30:07.362633+0000","last_undegraded":"2023-08-27T11:30:07.362633+0000","last_fullsized":"2023-08-27T11:30:07.362633+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:04:47.203315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":4,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.377944+0000","last_change":"2023-08-27T11:30:07.372096+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.377944+0000","last_undegraded":"2023-08-27T11:30:07.377944+0000","last_fullsized":"2023-08-27T11:30:07.377944+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:49:33.070564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.508333+0000","last_change":"2023-08-27T11:30:06.503408+0000","last_active":"2023-08-27T11:30:06.508333+0000","last_peered":"2023-08-27T11:30:06.508333+0000","last_clean":"2023-08-27T11:30:06.508333+0000","last_became_active":"2023-08-27T11:30:06.503111+0000","last_became_peered":"2023-08-27T11:30:06.503111+0000","last_unstale":"2023-08-27T11:30:06.508333+0000","last_undegraded":"2023-08-27T11:30:06.508333+0000","last_fullsized":"2023-08-27T11:30:06.508333+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:54:21.203272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.369922+0000","last_change":"2023-08-27T11:30:07.369922+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.369922+0000","last_undegraded":"2023-08-27T11:30:07.369922+0000","last_fullsized":"2023-08-27T11:30:07.369922+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:34.484460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.371984+0000","last_change":"2023-08-27T11:30:06.506988+0000","last_active":"2023-08-27T11:30:07.371984+0000","last_peered":"2023-08-27T11:30:07.371984+0000","last_clean":"2023-08-27T11:30:07.371984+0000","last_became_active":"2023-08-27T11:30:06.506687+0000","last_became_peered":"2023-08-27T11:30:06.506687+0000","last_unstale":"2023-08-27T11:30:07.371984+0000","last_undegraded":"2023-08-27T11:30:07.371984+0000","last_fullsized":"2023-08-27T11:30:07.371984+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:55:51.579443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380804+0000","last_change":"2023-08-27T11:30:07.377249+0000","last_active":"2023-08-27T11:30:07.380804+0000","last_peered":"2023-08-27T11:30:07.380804+0000","last_clean":"2023-08-27T11:30:07.380804+0000","last_became_active":"2023-08-27T11:30:07.376448+0000","last_became_peered":"2023-08-27T11:30:07.376448+0000","last_unstale":"2023-08-27T11:30:07.380804+0000","last_undegraded":"2023-08-27T11:30:07.380804+0000","last_fullsized":"2023-08-27T11:30:07.380804+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:02:14.837924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.363301+0000","last_change":"2023-08-27T11:30:06.507787+0000","last_active":"2023-08-27T11:30:07.363301+0000","last_peered":"2023-08-27T11:30:07.363301+0000","last_clean":"2023-08-27T11:30:07.363301+0000","last_became_active":"2023-08-27T11:30:06.507390+0000","last_became_peered":"2023-08-27T11:30:06.507390+0000","last_unstale":"2023-08-27T11:30:07.363301+0000","last_undegraded":"2023-08-27T11:30:07.363301+0000","last_fullsized":"2023-08-27T11:30:07.363301+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:00:38.629013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380650+0000","last_change":"2023-08-27T11:30:07.376530+0000","last_active":"2023-08-27T11:30:07.380650+0000","last_peered":"2023-08-27T11:30:07.380650+0000","last_clean":"2023-08-27T11:30:07.380650+0000","last_became_active":"2023-08-27T11:30:07.376221+0000","last_became_peered":"2023-08-27T11:30:07.376221+0000","last_unstale":"2023-08-27T11:30:07.380650+0000","last_undegraded":"2023-08-27T11:30:07.380650+0000","last_fullsized":"2023-08-27T11:30:07.380650+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:41.856918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.510282+0000","last_change":"2023-08-27T11:30:06.508059+0000","last_active":"2023-08-27T11:30:06.510282+0000","last_peered":"2023-08-27T11:30:06.510282+0000","last_clean":"2023-08-27T11:30:06.510282+0000","last_became_active":"2023-08-27T11:30:06.507677+0000","last_became_peered":"2023-08-27T11:30:06.507677+0000","last_unstale":"2023-08-27T11:30:06.510282+0000","last_undegraded":"2023-08-27T11:30:06.510282+0000","last_fullsized":"2023-08-27T11:30:06.510282+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:50:44.051625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.495697+0000","last_change":"2023-08-27T11:30:06.495697+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.495697+0000","last_undegraded":"2023-08-27T11:30:06.495697+0000","last_fullsized":"2023-08-27T11:30:06.495697+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375338+0000","last_change":"2023-08-27T11:30:06.509700+0000","last_active":"2023-08-27T11:30:07.375338+0000","last_peered":"2023-08-27T11:30:07.375338+0000","last_clean":"2023-08-27T11:30:07.375338+0000","last_became_active":"2023-08-27T11:30:06.509455+0000","last_became_peered":"2023-08-27T11:30:06.509455+0000","last_unstale":"2023-08-27T11:30:07.375338+0000","last_undegraded":"2023-08-27T11:30:07.375338+0000","last_fullsized":"2023-08-27T11:30:07.375338+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:12:09.969178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.373978+0000","last_change":"2023-08-27T11:30:07.373333+0000","last_active":"2023-08-27T11:30:07.373978+0000","last_peered":"2023-08-27T11:30:07.373978+0000","last_clean":"2023-08-27T11:30:07.373978+0000","last_became_active":"2023-08-27T11:30:07.373009+0000","last_became_peered":"2023-08-27T11:30:07.373009+0000","last_unstale":"2023-08-27T11:30:07.373978+0000","last_undegraded":"2023-08-27T11:30:07.373978+0000","last_fullsized":"2023-08-27T11:30:07.373978+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:24:21.667407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377251+0000","last_change":"2023-08-27T11:30:06.509723+0000","last_active":"2023-08-27T11:30:07.377251+0000","last_peered":"2023-08-27T11:30:07.377251+0000","last_clean":"2023-08-27T11:30:07.377251+0000","last_became_active":"2023-08-27T11:30:06.509418+0000","last_became_peered":"2023-08-27T11:30:06.509418+0000","last_unstale":"2023-08-27T11:30:07.377251+0000","last_undegraded":"2023-08-27T11:30:07.377251+0000","last_fullsized":"2023-08-27T11:30:07.377251+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:23:19.197311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.494625+0000","last_change":"2023-08-27T11:30:06.494625+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.494625+0000","last_undegraded":"2023-08-27T11:30:06.494625+0000","last_fullsized":"2023-08-27T11:30:06.494625+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.508364+0000","last_change":"2023-08-27T11:30:06.503031+0000","last_active":"2023-08-27T11:30:06.508364+0000","last_peered":"2023-08-27T11:30:06.508364+0000","last_clean":"2023-08-27T11:30:06.508364+0000","last_became_active":"2023-08-27T11:30:06.502780+0000","last_became_peered":"2023-08-27T11:30:06.502780+0000","last_unstale":"2023-08-27T11:30:06.508364+0000","last_undegraded":"2023-08-27T11:30:06.508364+0000","last_fullsized":"2023-08-27T11:30:06.508364+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:39:40.298338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.496930+0000","last_change":"2023-08-27T11:30:06.496930+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.496930+0000","last_undegraded":"2023-08-27T11:30:06.496930+0000","last_fullsized":"2023-08-27T11:30:06.496930+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.365524+0000","last_change":"2023-08-27T11:30:06.511539+0000","last_active":"2023-08-27T11:30:07.365524+0000","last_peered":"2023-08-27T11:30:07.365524+0000","last_clean":"2023-08-27T11:30:07.365524+0000","last_became_active":"2023-08-27T11:30:06.511200+0000","last_became_peered":"2023-08-27T11:30:06.511200+0000","last_unstale":"2023-08-27T11:30:07.365524+0000","last_undegraded":"2023-08-27T11:30:07.365524+0000","last_fullsized":"2023-08-27T11:30:07.365524+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:01.722653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.371737+0000","last_change":"2023-08-27T11:30:07.371737+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.371737+0000","last_undegraded":"2023-08-27T11:30:07.371737+0000","last_fullsized":"2023-08-27T11:30:07.371737+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:39:16.696934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370740+0000","last_change":"2023-08-27T11:30:06.505422+0000","last_active":"2023-08-27T11:30:07.370740+0000","last_peered":"2023-08-27T11:30:07.370740+0000","last_clean":"2023-08-27T11:30:07.370740+0000","last_became_active":"2023-08-27T11:30:06.505120+0000","last_became_peered":"2023-08-27T11:30:06.505120+0000","last_unstale":"2023-08-27T11:30:07.370740+0000","last_undegraded":"2023-08-27T11:30:07.370740+0000","last_fullsized":"2023-08-27T11:30:07.370740+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:21:11.370238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.494948+0000","last_change":"2023-08-27T11:30:06.494948+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.494948+0000","last_undegraded":"2023-08-27T11:30:06.494948+0000","last_fullsized":"2023-08-27T11:30:06.494948+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.373236+0000","last_change":"2023-08-27T11:30:06.506911+0000","last_active":"2023-08-27T11:30:07.373236+0000","last_peered":"2023-08-27T11:30:07.373236+0000","last_clean":"2023-08-27T11:30:07.373236+0000","last_became_active":"2023-08-27T11:30:06.506657+0000","last_became_peered":"2023-08-27T11:30:06.506657+0000","last_unstale":"2023-08-27T11:30:07.373236+0000","last_undegraded":"2023-08-27T11:30:07.373236+0000","last_fullsized":"2023-08-27T11:30:07.373236+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:05.904960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.497037+0000","last_change":"2023-08-27T11:30:06.497037+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.497037+0000","last_undegraded":"2023-08-27T11:30:06.497037+0000","last_fullsized":"2023-08-27T11:30:06.497037+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.374141+0000","last_change":"2023-08-27T11:30:06.516142+0000","last_active":"2023-08-27T11:30:07.374141+0000","last_peered":"2023-08-27T11:30:07.374141+0000","last_clean":"2023-08-27T11:30:07.374141+0000","last_became_active":"2023-08-27T11:30:06.515761+0000","last_became_peered":"2023-08-27T11:30:06.515761+0000","last_unstale":"2023-08-27T11:30:07.374141+0000","last_undegraded":"2023-08-27T11:30:07.374141+0000","last_fullsized":"2023-08-27T11:30:07.374141+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:12:05.165692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376818+0000","last_change":"2023-08-27T11:30:07.376818+0000","last_active":"2023-08-27T11:30:07.376818+0000","last_peered":"2023-08-27T11:30:07.376818+0000","last_clean":"2023-08-27T11:30:07.376818+0000","last_became_active":"2023-08-27T11:30:07.376575+0000","last_became_peered":"2023-08-27T11:30:07.376575+0000","last_unstale":"2023-08-27T11:30:07.376818+0000","last_undegraded":"2023-08-27T11:30:07.376818+0000","last_fullsized":"2023-08-27T11:30:07.376818+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:14:01.759128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.366408+0000","last_change":"2023-08-27T11:30:06.504618+0000","last_active":"2023-08-27T11:30:07.366408+0000","last_peered":"2023-08-27T11:30:07.366408+0000","last_clean":"2023-08-27T11:30:07.366408+0000","last_became_active":"2023-08-27T11:30:06.504354+0000","last_became_peered":"2023-08-27T11:30:06.504354+0000","last_unstale":"2023-08-27T11:30:07.366408+0000","last_undegraded":"2023-08-27T11:30:07.366408+0000","last_fullsized":"2023-08-27T11:30:07.366408+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:09:49.437363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.384834+0000","last_change":"2023-08-27T11:30:07.380702+0000","last_active":"2023-08-27T11:30:07.384834+0000","last_peered":"2023-08-27T11:30:07.384834+0000","last_clean":"2023-08-27T11:30:07.384834+0000","last_became_active":"2023-08-27T11:30:07.380378+0000","last_became_peered":"2023-08-27T11:30:07.380378+0000","last_unstale":"2023-08-27T11:30:07.384834+0000","last_undegraded":"2023-08-27T11:30:07.384834+0000","last_fullsized":"2023-08-27T11:30:07.384834+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:06:41.915453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375394+0000","last_change":"2023-08-27T11:30:06.511005+0000","last_active":"2023-08-27T11:30:07.375394+0000","last_peered":"2023-08-27T11:30:07.375394+0000","last_clean":"2023-08-27T11:30:07.375394+0000","last_became_active":"2023-08-27T11:30:06.510712+0000","last_became_peered":"2023-08-27T11:30:06.510712+0000","last_unstale":"2023-08-27T11:30:07.375394+0000","last_undegraded":"2023-08-27T11:30:07.375394+0000","last_fullsized":"2023-08-27T11:30:07.375394+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:53:54.707523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377430+0000","last_change":"2023-08-27T11:30:07.377430+0000","last_active":"2023-08-27T11:30:07.377430+0000","last_peered":"2023-08-27T11:30:07.377430+0000","last_clean":"2023-08-27T11:30:07.377430+0000","last_became_active":"2023-08-27T11:30:07.376459+0000","last_became_peered":"2023-08-27T11:30:07.376459+0000","last_unstale":"2023-08-27T11:30:07.377430+0000","last_undegraded":"2023-08-27T11:30:07.377430+0000","last_fullsized":"2023-08-27T11:30:07.377430+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:51:11.653496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372581+0000","last_change":"2023-08-27T11:30:06.510880+0000","last_active":"2023-08-27T11:30:07.372581+0000","last_peered":"2023-08-27T11:30:07.372581+0000","last_clean":"2023-08-27T11:30:07.372581+0000","last_became_active":"2023-08-27T11:30:06.510602+0000","last_became_peered":"2023-08-27T11:30:06.510602+0000","last_unstale":"2023-08-27T11:30:07.372581+0000","last_undegraded":"2023-08-27T11:30:07.372581+0000","last_fullsized":"2023-08-27T11:30:07.372581+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:12.746422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377066+0000","last_change":"2023-08-27T11:30:07.374685+0000","last_active":"2023-08-27T11:30:07.377066+0000","last_peered":"2023-08-27T11:30:07.377066+0000","last_clean":"2023-08-27T11:30:07.377066+0000","last_became_active":"2023-08-27T11:30:07.374131+0000","last_became_peered":"2023-08-27T11:30:07.374131+0000","last_unstale":"2023-08-27T11:30:07.377066+0000","last_undegraded":"2023-08-27T11:30:07.377066+0000","last_fullsized":"2023-08-27T11:30:07.377066+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:22:46.104350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.364655+0000","last_change":"2023-08-27T11:30:06.515673+0000","last_active":"2023-08-27T11:30:07.364655+0000","last_peered":"2023-08-27T11:30:07.364655+0000","last_clean":"2023-08-27T11:30:07.364655+0000","last_became_active":"2023-08-27T11:30:06.515364+0000","last_became_peered":"2023-08-27T11:30:06.515364+0000","last_unstale":"2023-08-27T11:30:07.364655+0000","last_undegraded":"2023-08-27T11:30:07.364655+0000","last_fullsized":"2023-08-27T11:30:07.364655+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:05:05.778823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.378734+0000","last_change":"2023-08-27T11:30:07.378734+0000","last_active":"2023-08-27T11:30:07.378734+0000","last_peered":"2023-08-27T11:30:07.378734+0000","last_clean":"2023-08-27T11:30:07.378734+0000","last_became_active":"2023-08-27T11:30:07.378472+0000","last_became_peered":"2023-08-27T11:30:07.378472+0000","last_unstale":"2023-08-27T11:30:07.378734+0000","last_undegraded":"2023-08-27T11:30:07.378734+0000","last_fullsized":"2023-08-27T11:30:07.378734+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:46:20.644887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.369874+0000","last_change":"2023-08-27T11:30:06.504079+0000","last_active":"2023-08-27T11:30:07.369874+0000","last_peered":"2023-08-27T11:30:07.369874+0000","last_clean":"2023-08-27T11:30:07.369874+0000","last_became_active":"2023-08-27T11:30:06.503811+0000","last_became_peered":"2023-08-27T11:30:06.503811+0000","last_unstale":"2023-08-27T11:30:07.369874+0000","last_undegraded":"2023-08-27T11:30:07.369874+0000","last_fullsized":"2023-08-27T11:30:07.369874+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:27:05.776026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":4,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.376957+0000","last_change":"2023-08-27T11:30:07.371363+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.376957+0000","last_undegraded":"2023-08-27T11:30:07.376957+0000","last_fullsized":"2023-08-27T11:30:07.376957+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:49:27.497988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.365622+0000","last_change":"2023-08-27T11:30:06.503387+0000","last_active":"2023-08-27T11:30:07.365622+0000","last_peered":"2023-08-27T11:30:07.365622+0000","last_clean":"2023-08-27T11:30:07.365622+0000","last_became_active":"2023-08-27T11:30:06.503081+0000","last_became_peered":"2023-08-27T11:30:06.503081+0000","last_unstale":"2023-08-27T11:30:07.365622+0000","last_undegraded":"2023-08-27T11:30:07.365622+0000","last_fullsized":"2023-08-27T11:30:07.365622+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:25:44.805880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.494479+0000","last_change":"2023-08-27T11:30:06.494479+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.494479+0000","last_undegraded":"2023-08-27T11:30:06.494479+0000","last_fullsized":"2023-08-27T11:30:06.494479+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372445+0000","last_change":"2023-08-27T11:30:06.511539+0000","last_active":"2023-08-27T11:30:07.372445+0000","last_peered":"2023-08-27T11:30:07.372445+0000","last_clean":"2023-08-27T11:30:07.372445+0000","last_became_active":"2023-08-27T11:30:06.511248+0000","last_became_peered":"2023-08-27T11:30:06.511248+0000","last_unstale":"2023-08-27T11:30:07.372445+0000","last_undegraded":"2023-08-27T11:30:07.372445+0000","last_fullsized":"2023-08-27T11:30:07.372445+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:22:48.182930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376018+0000","last_change":"2023-08-27T11:30:07.373745+0000","last_active":"2023-08-27T11:30:07.376018+0000","last_peered":"2023-08-27T11:30:07.376018+0000","last_clean":"2023-08-27T11:30:07.376018+0000","last_became_active":"2023-08-27T11:30:07.373460+0000","last_became_peered":"2023-08-27T11:30:07.373460+0000","last_unstale":"2023-08-27T11:30:07.376018+0000","last_undegraded":"2023-08-27T11:30:07.376018+0000","last_fullsized":"2023-08-27T11:30:07.376018+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:47:30.525294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.366107+0000","last_change":"2023-08-27T11:30:06.502148+0000","last_active":"2023-08-27T11:30:07.366107+0000","last_peered":"2023-08-27T11:30:07.366107+0000","last_clean":"2023-08-27T11:30:07.366107+0000","last_became_active":"2023-08-27T11:30:06.501869+0000","last_became_peered":"2023-08-27T11:30:06.501869+0000","last_unstale":"2023-08-27T11:30:07.366107+0000","last_undegraded":"2023-08-27T11:30:07.366107+0000","last_fullsized":"2023-08-27T11:30:07.366107+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:19:08.944028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376696+0000","last_change":"2023-08-27T11:30:07.376696+0000","last_active":"2023-08-27T11:30:07.376696+0000","last_peered":"2023-08-27T11:30:07.376696+0000","last_clean":"2023-08-27T11:30:07.376696+0000","last_became_active":"2023-08-27T11:30:07.376266+0000","last_became_peered":"2023-08-27T11:30:07.376266+0000","last_unstale":"2023-08-27T11:30:07.376696+0000","last_undegraded":"2023-08-27T11:30:07.376696+0000","last_fullsized":"2023-08-27T11:30:07.376696+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:50:58.773261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.364173+0000","last_change":"2023-08-27T11:30:06.504227+0000","last_active":"2023-08-27T11:30:07.364173+0000","last_peered":"2023-08-27T11:30:07.364173+0000","last_clean":"2023-08-27T11:30:07.364173+0000","last_became_active":"2023-08-27T11:30:06.503935+0000","last_became_peered":"2023-08-27T11:30:06.503935+0000","last_unstale":"2023-08-27T11:30:07.364173+0000","last_undegraded":"2023-08-27T11:30:07.364173+0000","last_fullsized":"2023-08-27T11:30:07.364173+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:44:29.308537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.495279+0000","last_change":"2023-08-27T11:30:06.495279+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.495279+0000","last_undegraded":"2023-08-27T11:30:06.495279+0000","last_fullsized":"2023-08-27T11:30:06.495279+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370211+0000","last_change":"2023-08-27T11:30:06.515077+0000","last_active":"2023-08-27T11:30:07.370211+0000","last_peered":"2023-08-27T11:30:07.370211+0000","last_clean":"2023-08-27T11:30:07.370211+0000","last_became_active":"2023-08-27T11:30:06.514812+0000","last_became_peered":"2023-08-27T11:30:06.514812+0000","last_unstale":"2023-08-27T11:30:07.370211+0000","last_undegraded":"2023-08-27T11:30:07.370211+0000","last_fullsized":"2023-08-27T11:30:07.370211+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:56:41.368080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377075+0000","last_change":"2023-08-27T11:30:07.375476+0000","last_active":"2023-08-27T11:30:07.377075+0000","last_peered":"2023-08-27T11:30:07.377075+0000","last_clean":"2023-08-27T11:30:07.377075+0000","last_became_active":"2023-08-27T11:30:07.375167+0000","last_became_peered":"2023-08-27T11:30:07.375167+0000","last_unstale":"2023-08-27T11:30:07.377075+0000","last_undegraded":"2023-08-27T11:30:07.377075+0000","last_fullsized":"2023-08-27T11:30:07.377075+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:18:51.080195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.363503+0000","last_change":"2023-08-27T11:30:06.507172+0000","last_active":"2023-08-27T11:30:07.363503+0000","last_peered":"2023-08-27T11:30:07.363503+0000","last_clean":"2023-08-27T11:30:07.363503+0000","last_became_active":"2023-08-27T11:30:06.506885+0000","last_became_peered":"2023-08-27T11:30:06.506885+0000","last_unstale":"2023-08-27T11:30:07.363503+0000","last_undegraded":"2023-08-27T11:30:07.363503+0000","last_fullsized":"2023-08-27T11:30:07.363503+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:03.298758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.387326+0000","last_change":"2023-08-27T11:30:07.383116+0000","last_active":"2023-08-27T11:30:07.387326+0000","last_peered":"2023-08-27T11:30:07.387326+0000","last_clean":"2023-08-27T11:30:07.387326+0000","last_became_active":"2023-08-27T11:30:07.382838+0000","last_became_peered":"2023-08-27T11:30:07.382838+0000","last_unstale":"2023-08-27T11:30:07.387326+0000","last_undegraded":"2023-08-27T11:30:07.387326+0000","last_fullsized":"2023-08-27T11:30:07.387326+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:48:13.826189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.510301+0000","last_change":"2023-08-27T11:30:06.508063+0000","last_active":"2023-08-27T11:30:06.510301+0000","last_peered":"2023-08-27T11:30:06.510301+0000","last_clean":"2023-08-27T11:30:06.510301+0000","last_became_active":"2023-08-27T11:30:06.507412+0000","last_became_peered":"2023-08-27T11:30:06.507412+0000","last_unstale":"2023-08-27T11:30:06.510301+0000","last_undegraded":"2023-08-27T11:30:06.510301+0000","last_fullsized":"2023-08-27T11:30:06.510301+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:58.753023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.374655+0000","last_change":"2023-08-27T11:30:07.374655+0000","last_active":"2023-08-27T11:30:07.374655+0000","last_peered":"2023-08-27T11:30:07.374655+0000","last_clean":"2023-08-27T11:30:07.374655+0000","last_became_active":"2023-08-27T11:30:07.374306+0000","last_became_peered":"2023-08-27T11:30:07.374306+0000","last_unstale":"2023-08-27T11:30:07.374655+0000","last_undegraded":"2023-08-27T11:30:07.374655+0000","last_fullsized":"2023-08-27T11:30:07.374655+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:23.749135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370729+0000","last_change":"2023-08-27T11:30:06.500047+0000","last_active":"2023-08-27T11:30:07.370729+0000","last_peered":"2023-08-27T11:30:07.370729+0000","last_clean":"2023-08-27T11:30:07.370729+0000","last_became_active":"2023-08-27T11:30:06.499775+0000","last_became_peered":"2023-08-27T11:30:06.499775+0000","last_unstale":"2023-08-27T11:30:07.370729+0000","last_undegraded":"2023-08-27T11:30:07.370729+0000","last_fullsized":"2023-08-27T11:30:07.370729+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:13.320294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376417+0000","last_change":"2023-08-27T11:30:07.376417+0000","last_active":"2023-08-27T11:30:07.376417+0000","last_peered":"2023-08-27T11:30:07.376417+0000","last_clean":"2023-08-27T11:30:07.376417+0000","last_became_active":"2023-08-27T11:30:07.376144+0000","last_became_peered":"2023-08-27T11:30:07.376144+0000","last_unstale":"2023-08-27T11:30:07.376417+0000","last_undegraded":"2023-08-27T11:30:07.376417+0000","last_fullsized":"2023-08-27T11:30:07.376417+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:07:51.665760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370869+0000","last_change":"2023-08-27T11:30:06.505423+0000","last_active":"2023-08-27T11:30:07.370869+0000","last_peered":"2023-08-27T11:30:07.370869+0000","last_clean":"2023-08-27T11:30:07.370869+0000","last_became_active":"2023-08-27T11:30:06.505138+0000","last_became_peered":"2023-08-27T11:30:06.505138+0000","last_unstale":"2023-08-27T11:30:07.370869+0000","last_undegraded":"2023-08-27T11:30:07.370869+0000","last_fullsized":"2023-08-27T11:30:07.370869+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:12:32.588899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375665+0000","last_change":"2023-08-27T11:30:07.375665+0000","last_active":"2023-08-27T11:30:07.375665+0000","last_peered":"2023-08-27T11:30:07.375665+0000","last_clean":"2023-08-27T11:30:07.375665+0000","last_became_active":"2023-08-27T11:30:07.375184+0000","last_became_peered":"2023-08-27T11:30:07.375184+0000","last_unstale":"2023-08-27T11:30:07.375665+0000","last_undegraded":"2023-08-27T11:30:07.375665+0000","last_fullsized":"2023-08-27T11:30:07.375665+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:42.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.363224+0000","last_change":"2023-08-27T11:30:06.504990+0000","last_active":"2023-08-27T11:30:07.363224+0000","last_peered":"2023-08-27T11:30:07.363224+0000","last_clean":"2023-08-27T11:30:07.363224+0000","last_became_active":"2023-08-27T11:30:06.504768+0000","last_became_peered":"2023-08-27T11:30:06.504768+0000","last_unstale":"2023-08-27T11:30:07.363224+0000","last_undegraded":"2023-08-27T11:30:07.363224+0000","last_fullsized":"2023-08-27T11:30:07.363224+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:27:04.823447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377092+0000","last_change":"2023-08-27T11:30:07.376099+0000","last_active":"2023-08-27T11:30:07.377092+0000","last_peered":"2023-08-27T11:30:07.377092+0000","last_clean":"2023-08-27T11:30:07.377092+0000","last_became_active":"2023-08-27T11:30:07.375823+0000","last_became_peered":"2023-08-27T11:30:07.375823+0000","last_unstale":"2023-08-27T11:30:07.377092+0000","last_undegraded":"2023-08-27T11:30:07.377092+0000","last_fullsized":"2023-08-27T11:30:07.377092+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:39.725351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.510303+0000","last_change":"2023-08-27T11:30:06.508106+0000","last_active":"2023-08-27T11:30:06.510303+0000","last_peered":"2023-08-27T11:30:06.510303+0000","last_clean":"2023-08-27T11:30:06.510303+0000","last_became_active":"2023-08-27T11:30:06.507721+0000","last_became_peered":"2023-08-27T11:30:06.507721+0000","last_unstale":"2023-08-27T11:30:06.510303+0000","last_undegraded":"2023-08-27T11:30:06.510303+0000","last_fullsized":"2023-08-27T11:30:06.510303+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:21:52.252880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375020+0000","last_change":"2023-08-27T11:30:07.372999+0000","last_active":"2023-08-27T11:30:07.375020+0000","last_peered":"2023-08-27T11:30:07.375020+0000","last_clean":"2023-08-27T11:30:07.375020+0000","last_became_active":"2023-08-27T11:30:07.372727+0000","last_became_peered":"2023-08-27T11:30:07.372727+0000","last_unstale":"2023-08-27T11:30:07.375020+0000","last_undegraded":"2023-08-27T11:30:07.375020+0000","last_fullsized":"2023-08-27T11:30:07.375020+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:53:39.155707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":351,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.362862+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:30:07.362862+0000","last_peered":"2023-08-27T11:30:07.362862+0000","last_clean":"2023-08-27T11:30:07.362862+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:30:07.362862+0000","last_undegraded":"2023-08-27T11:30:07.362862+0000","last_fullsized":"2023-08-27T11:30:07.362862+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:54:33.147243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.382079+0000","last_change":"2023-08-27T11:30:07.379099+0000","last_active":"2023-08-27T11:30:07.382079+0000","last_peered":"2023-08-27T11:30:07.382079+0000","last_clean":"2023-08-27T11:30:07.382079+0000","last_became_active":"2023-08-27T11:30:07.378802+0000","last_became_peered":"2023-08-27T11:30:07.378802+0000","last_unstale":"2023-08-27T11:30:07.382079+0000","last_undegraded":"2023-08-27T11:30:07.382079+0000","last_fullsized":"2023-08-27T11:30:07.382079+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:26.058597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":351,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.364140+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:30:07.364140+0000","last_peered":"2023-08-27T11:30:07.364140+0000","last_clean":"2023-08-27T11:30:07.364140+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:30:07.364140+0000","last_undegraded":"2023-08-27T11:30:07.364140+0000","last_fullsized":"2023-08-27T11:30:07.364140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:59:17.376710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375699+0000","last_change":"2023-08-27T11:30:07.372568+0000","last_active":"2023-08-27T11:30:07.375699+0000","last_peered":"2023-08-27T11:30:07.375699+0000","last_clean":"2023-08-27T11:30:07.375699+0000","last_became_active":"2023-08-27T11:30:07.372289+0000","last_became_peered":"2023-08-27T11:30:07.372289+0000","last_unstale":"2023-08-27T11:30:07.375699+0000","last_undegraded":"2023-08-27T11:30:07.375699+0000","last_fullsized":"2023-08-27T11:30:07.375699+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:10:25.361282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":299,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372055+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:30:07.372055+0000","last_peered":"2023-08-27T11:30:07.372055+0000","last_clean":"2023-08-27T11:30:07.372055+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:30:07.372055+0000","last_undegraded":"2023-08-27T11:30:07.372055+0000","last_fullsized":"2023-08-27T11:30:07.372055+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:32:49.299155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.373549+0000","last_change":"2023-08-27T11:30:07.366404+0000","last_active":"2023-08-27T11:30:07.373549+0000","last_peered":"2023-08-27T11:30:07.373549+0000","last_clean":"2023-08-27T11:30:07.373549+0000","last_became_active":"2023-08-27T11:30:07.366051+0000","last_became_peered":"2023-08-27T11:30:07.366051+0000","last_unstale":"2023-08-27T11:30:07.373549+0000","last_undegraded":"2023-08-27T11:30:07.373549+0000","last_fullsized":"2023-08-27T11:30:07.373549+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:53:30.711567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":297,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372131+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:30:07.372131+0000","last_peered":"2023-08-27T11:30:07.372131+0000","last_clean":"2023-08-27T11:30:07.372131+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:30:07.372131+0000","last_undegraded":"2023-08-27T11:30:07.372131+0000","last_fullsized":"2023-08-27T11:30:07.372131+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:36:26.335579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370464+0000","last_change":"2023-08-27T11:30:07.367607+0000","last_active":"2023-08-27T11:30:07.370464+0000","last_peered":"2023-08-27T11:30:07.370464+0000","last_clean":"2023-08-27T11:30:07.370464+0000","last_became_active":"2023-08-27T11:30:07.367368+0000","last_became_peered":"2023-08-27T11:30:07.367368+0000","last_unstale":"2023-08-27T11:30:07.370464+0000","last_undegraded":"2023-08-27T11:30:07.370464+0000","last_fullsized":"2023-08-27T11:30:07.370464+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:39.373129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":297,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372428+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:30:07.372428+0000","last_peered":"2023-08-27T11:30:07.372428+0000","last_clean":"2023-08-27T11:30:07.372428+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:30:07.372428+0000","last_undegraded":"2023-08-27T11:30:07.372428+0000","last_fullsized":"2023-08-27T11:30:07.372428+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:55:02.343378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.496946+0000","last_change":"2023-08-27T11:30:06.496946+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.496946+0000","last_undegraded":"2023-08-27T11:30:06.496946+0000","last_fullsized":"2023-08-27T11:30:06.496946+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":447,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372029+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:30:07.372029+0000","last_peered":"2023-08-27T11:30:07.372029+0000","last_clean":"2023-08-27T11:30:07.372029+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:30:07.372029+0000","last_undegraded":"2023-08-27T11:30:07.372029+0000","last_fullsized":"2023-08-27T11:30:07.372029+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:04.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":305,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.374061+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:30:07.374061+0000","last_peered":"2023-08-27T11:30:07.374061+0000","last_clean":"2023-08-27T11:30:07.374061+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:30:07.374061+0000","last_undegraded":"2023-08-27T11:30:07.374061+0000","last_fullsized":"2023-08-27T11:30:07.374061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:46:35.266946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376873+0000","last_change":"2023-08-27T11:30:07.376873+0000","last_active":"2023-08-27T11:30:07.376873+0000","last_peered":"2023-08-27T11:30:07.376873+0000","last_clean":"2023-08-27T11:30:07.376873+0000","last_became_active":"2023-08-27T11:30:07.376104+0000","last_became_peered":"2023-08-27T11:30:07.376104+0000","last_unstale":"2023-08-27T11:30:07.376873+0000","last_undegraded":"2023-08-27T11:30:07.376873+0000","last_fullsized":"2023-08-27T11:30:07.376873+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:48:39.739503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":295,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.497876+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:30:06.497876+0000","last_peered":"2023-08-27T11:30:06.497876+0000","last_clean":"2023-08-27T11:30:06.497876+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:30:06.497876+0000","last_undegraded":"2023-08-27T11:30:06.497876+0000","last_fullsized":"2023-08-27T11:30:06.497876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:17.897409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380860+0000","last_change":"2023-08-27T11:30:07.375200+0000","last_active":"2023-08-27T11:30:07.380860+0000","last_peered":"2023-08-27T11:30:07.380860+0000","last_clean":"2023-08-27T11:30:07.380860+0000","last_became_active":"2023-08-27T11:30:07.374915+0000","last_became_peered":"2023-08-27T11:30:07.374915+0000","last_unstale":"2023-08-27T11:30:07.380860+0000","last_undegraded":"2023-08-27T11:30:07.380860+0000","last_fullsized":"2023-08-27T11:30:07.380860+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:32:43.401182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":335,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.366365+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:30:07.366365+0000","last_peered":"2023-08-27T11:30:07.366365+0000","last_clean":"2023-08-27T11:30:07.366365+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:30:07.366365+0000","last_undegraded":"2023-08-27T11:30:07.366365+0000","last_fullsized":"2023-08-27T11:30:07.366365+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:39:06.252428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34036,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337804,"statfs":{"total":96636764160,"available":96601911296,"internally_reserved":0,"allocated":2920448,"data_stored":1702685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.22700000000000001},"max":{"1min":0.60399999999999998,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.753},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.51500000000000001},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.68300000000000005}]},{"osd":1,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.83399999999999996,"15min":0.872},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.72399999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.42799999999999999,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.67800000000000005,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.623},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.90200000000000002},"last":0.66500000000000004}]},{"osd":3,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":0.996,"15min":0.996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.53000000000000003,"5min":0.42899999999999999,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.71099999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.503},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.72199999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":1.224,"15min":1.224},"last":0.55000000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79700000000000004,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.84799999999999998},"last":0.64100000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83799999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.73399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575011,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.84599999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.39800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.89800000000000002},"last":0.48799999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.53900000000000003},"min":{"1min":0.35599999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.81799999999999995,"15min":0.873},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55400000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.499,"15min":0.51000000000000001},"min":{"1min":0.40799999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":2.0830000000000002},"last":0.72599999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.83199999999999996,"15min":0.84499999999999997},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.34000000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.93799999999999994},"last":0.626}]},{"osd":4,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":0.71599999999999997,"5min":0.753,"15min":0.83099999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.77000000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75700000000000001,"5min":0.85499999999999998,"15min":0.875},"last":0.78200000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79200000000000004,"5min":1.365,"15min":1.365},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.34899999999999998},"max":{"1min":0.77700000000000002,"5min":0.82999999999999996,"15min":0.91400000000000003},"last":0.753}]}]},{"osd":1,"up_from":13,"seq":55834575010,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34020,"kb_used_data":2840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337820,"statfs":{"total":96636764160,"available":96601927680,"internally_reserved":0,"allocated":2908160,"data_stored":1695557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.315,"15min":0.47399999999999998},"min":{"1min":0.21199999999999999,"5min":0.19500000000000001,"15min":0.191},"max":{"1min":0.45300000000000001,"5min":0.53300000000000003,"15min":11.971},"last":0.254},{"interface":"front","average":{"1min":0.315,"5min":0.32700000000000001,"15min":0.64700000000000002},"min":{"1min":0.23000000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.44900000000000001,"5min":0.74399999999999999,"15min":23.045999999999999},"last":0.313}]},{"osd":2,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34000000000000002,"15min":0.502},"min":{"1min":0.28199999999999997,"5min":0.23300000000000001,"15min":0.221},"max":{"1min":0.42399999999999999,"5min":0.57699999999999996,"15min":11.928000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.33700000000000002},"min":{"1min":0.249,"5min":0.22800000000000001,"15min":0.224},"max":{"1min":0.46899999999999997,"5min":0.52200000000000002,"15min":0.58599999999999997},"last":0.29599999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38400000000000001,"15min":0.52800000000000002},"min":{"1min":0.309,"5min":0.27500000000000002,"15min":0.26300000000000001},"max":{"1min":0.47599999999999998,"5min":0.64100000000000001,"15min":10.99},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36799999999999999,"15min":0.68500000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.503,"5min":0.54700000000000004,"15min":22.893999999999998},"last":0.35799999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.33400000000000002},"max":{"1min":0.84999999999999998,"5min":0.90700000000000003,"15min":1.26},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58099999999999996,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.745,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.62}]},{"osd":5,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.88600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":23.016999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.81899999999999995,"5min":0.88700000000000001,"15min":1.296},"last":0.66500000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58199999999999996,"15min":0.72799999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.371},"max":{"1min":0.72999999999999998,"5min":0.84899999999999998,"15min":10.925000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60199999999999998,"15min":0.90400000000000003},"min":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88,"5min":0.88,"15min":22.952000000000002},"last":0.70999999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59299999999999997,"15min":0.89800000000000002},"min":{"1min":0.433,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":22.988},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":1.327},"last":0.56999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2310144,"data_stored":1103828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.215,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.497,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35999999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36199999999999999,"15min":0.34399999999999997},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47699999999999998,"5min":0.628,"15min":0.63200000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34799999999999998,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45800000000000002,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.34599999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35799999999999998,"15min":0.34599999999999997},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.53700000000000003,"15min":0.59599999999999997},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.376},"min":{"1min":0.30499999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.53000000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.315}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.34399999999999997},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.214},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.65700000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.603},"last":0.62}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79600000000000004,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.878,"15min":0.878},"last":0.60399999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83399999999999996,"5min":3.79,"15min":3.79},"last":0.63800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32100000000000001,"15min":0.318},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.19700000000000001},"max":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.629},"last":0.32300000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41699999999999998,"5min":0.50800000000000001,"15min":0.54400000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34699999999999998},"min":{"1min":0.26200000000000001,"5min":0.23000000000000001,"15min":0.22600000000000001},"max":{"1min":0.49299999999999999,"5min":0.56399999999999995,"15min":0.749},"last":0.36099999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.378},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":0.51800000000000002,"5min":0.57099999999999995,"15min":0.73699999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.32600000000000001,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.55100000000000005,"15min":0.69499999999999995},"last":0.39900000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.879},"last":0.45700000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36699999999999999},"max":{"1min":0.83799999999999997,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.52800000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.40000000000000002},"max":{"1min":0.78600000000000003,"5min":0.82699999999999996,"15min":0.83599999999999997},"last":0.47399999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.46600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.29999999999999999,"15min":0.30299999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.52500000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.307,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.223,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40899999999999997,"5min":0.755,"15min":0.755},"last":0.36499999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33500000000000002,"15min":0.32300000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":0.40300000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":0.48999999999999999,"5min":2.73,"15min":2.73},"last":0.34799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.372,"15min":0.36399999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.46400000000000002,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.375,"15min":0.37},"min":{"1min":0.27700000000000002,"5min":0.26700000000000002,"15min":0.26400000000000001},"max":{"1min":0.51000000000000001,"5min":1.179,"15min":1.179},"last":0.44900000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.28599999999999998},"max":{"1min":0.748,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.437},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.71799999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54100000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.57699999999999996},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.54900000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.373},"max":{"1min":0.84299999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.497,"5min":0.38,"15min":0.38},"max":{"1min":0.83199999999999996,"5min":0.86899999999999999,"15min":1.111},"last":0.52600000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":1.236},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.499}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68200000000000005,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.502},"min":{"1min":0.33900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57399999999999995}]},{"osd":1,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66100000000000003,"5min":0.746,"15min":0.76500000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.75,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.376}]},{"osd":2,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.40400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.55700000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.96999999999999997,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59899999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71499999999999997,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":1.865,"15min":1.865},"last":0.61799999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90600000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.624,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82799999999999996,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.58799999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.90100000000000002,"5min":2.714,"15min":2.714},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.64500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575010,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.376,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61099999999999999,"5min":0.78900000000000003,"15min":0.96899999999999997},"last":0.38},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.44700000000000001,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.437}]},{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67500000000000004,"5min":0.68400000000000005,"15min":1.075},"last":0.33500000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.373,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.0580000000000001},"last":0.35399999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":1.03},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.42199999999999999,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.77800000000000002,"15min":1.026},"last":0.39400000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.27000000000000002},"max":{"1min":0.63900000000000001,"5min":0.78700000000000003,"15min":0.86899999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.73399999999999999,"5min":0.89400000000000002,"15min":0.91400000000000003},"last":0.51500000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.31},"max":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.88200000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.873,"15min":0.94199999999999995},"last":0.46000000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.34799999999999998},"max":{"1min":0.753,"5min":0.85999999999999999,"15min":0.95899999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.84399999999999997,"15min":0.92000000000000004},"last":0.63400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:30:11.992 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:30:12.146 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.145+0000 7f027af95700 1 Processor -- start 2023-08-27T11:30:12.147 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.145+0000 7f027af95700 1 -- start start 2023-08-27T11:30:12.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.146+0000 7f027af95700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0274158c90 0x7f027406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.146+0000 7f027af95700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0274159070 0x7f027406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.146+0000 7f027af95700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027415f5d0 con 0x7f02740b1270 2023-08-27T11:30:12.150 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.146+0000 7f027af95700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02740ca990 con 0x7f0274158c90 2023-08-27T11:30:12.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.146+0000 7f027af95700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f027406e870 con 0x7f0274159070 2023-08-27T11:30:12.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.146+0000 7f0278d31700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0274158c90 0x7f027406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.146+0000 7f0278d31700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0274158c90 0x7f027406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39392/0 (socket says 172.21.15.112:39392) 2023-08-27T11:30:12.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.146+0000 7f0278d31700 1 -- 172.21.15.112:0/984271485 learned_addr learned my addr 172.21.15.112:0/984271485 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:12.151 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.146+0000 7f0273fff700 1 --2- 172.21.15.112:0/984271485 >> v2:172.21.15.138:3300/0 conn(0x7f0274159070 0x7f027406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.146+0000 7f0278d31700 1 -- 172.21.15.112:0/984271485 >> v2:172.21.15.138:3300/0 conn(0x7f0274159070 msgr2=0x7f027406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:12.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.146+0000 7f0278d31700 1 --2- 172.21.15.112:0/984271485 >> v2:172.21.15.138:3300/0 conn(0x7f0274159070 0x7f027406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.146+0000 7f0278d31700 1 -- 172.21.15.112:0/984271485 >> v1:172.21.15.138:6789/0 conn(0x7f02740b1270 legacy=0x7f027415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:12.152 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.146+0000 7f0278d31700 1 -- 172.21.15.112:0/984271485 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02740b1a10 con 0x7f0274158c90 2023-08-27T11:30:12.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.147+0000 7f0273fff700 1 --2- 172.21.15.112:0/984271485 >> v2:172.21.15.138:3300/0 conn(0x7f0274159070 0x7f027406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:12.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.147+0000 7f0278d31700 1 --2- 172.21.15.112:0/984271485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0274158c90 0x7f027406e330 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f02740b2ce0 tx=0x7f0268009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d92e448b762a53 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:12.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.147+0000 7f02737fe700 1 -- 172.21.15.112:0/984271485 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268004230 con 0x7f0274158c90 2023-08-27T11:30:12.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.147+0000 7f02737fe700 1 -- 172.21.15.112:0/984271485 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0268004390 con 0x7f0274158c90 2023-08-27T11:30:12.153 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.147+0000 7f02737fe700 1 -- 172.21.15.112:0/984271485 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02680036a0 con 0x7f0274158c90 2023-08-27T11:30:12.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.147+0000 7f027af95700 1 -- 172.21.15.112:0/984271485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0274158c90 msgr2=0x7f027406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.147+0000 7f027af95700 1 --2- 172.21.15.112:0/984271485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0274158c90 0x7f027406e330 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f02740b2ce0 tx=0x7f0268009bd0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.147+0000 7f027af95700 1 -- 172.21.15.112:0/984271485 shutdown_connections 2023-08-27T11:30:12.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.147+0000 7f027af95700 1 --2- 172.21.15.112:0/984271485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0274158c90 0x7f027406e330 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.147+0000 7f027af95700 1 --2- 172.21.15.112:0/984271485 >> v2:172.21.15.138:3300/0 conn(0x7f0274159070 0x7f027406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.147+0000 7f027af95700 1 -- 172.21.15.112:0/984271485 >> 172.21.15.112:0/984271485 conn(0x7f02740d0000 msgr2=0x7f02740d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:12.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.147+0000 7f027af95700 1 -- 172.21.15.112:0/984271485 shutdown_connections 2023-08-27T11:30:12.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.147+0000 7f027af95700 1 -- 172.21.15.112:0/984271485 wait complete. 2023-08-27T11:30:12.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.148+0000 7f027af95700 1 Processor -- start 2023-08-27T11:30:12.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.149+0000 7f027af95700 1 -- start start 2023-08-27T11:30:12.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.149+0000 7f027af95700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02740b1270 0x7f02741524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.149+0000 7f027af95700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0274158c90 0x7f02740ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.149+0000 7f027af95700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0274159070 0x7f02740717d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.159 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.149+0000 7f027af95700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02740ac8a0 con 0x7f02740b1270 2023-08-27T11:30:12.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.149+0000 7f027af95700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02740ad420 con 0x7f0274159070 2023-08-27T11:30:12.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.149+0000 7f027af95700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02740acaf0 con 0x7f0274158c90 2023-08-27T11:30:12.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.149+0000 7f0278d31700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02740b1270 0x7f02741524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f0278d31700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02740b1270 0x7f02741524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39398/0 (socket says 172.21.15.112:39398) 2023-08-27T11:30:12.160 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f0278d31700 1 -- 172.21.15.112:0/2136221373 learned_addr learned my addr 172.21.15.112:0/2136221373 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:12.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f0273fff700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0274158c90 0x7f02740ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f0279532700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0274159070 0x7f02740717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f0278d31700 1 -- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0274158c90 msgr2=0x7f02740ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f0278d31700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0274158c90 0x7f02740ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f0278d31700 1 -- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0274159070 msgr2=0x7f02740717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f0278d31700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0274159070 0x7f02740717d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f0278d31700 1 -- 172.21.15.112:0/2136221373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0268009550 con 0x7f02740b1270 2023-08-27T11:30:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f0279532700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0274159070 0x7f02740717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f0273fff700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0274158c90 0x7f02740ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:12.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f0278d31700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02740b1270 0x7f02741524c0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f02740cff60 tx=0x7f0268004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:12.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f02717fa700 1 -- 172.21.15.112:0/2136221373 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268003d80 con 0x7f02740b1270 2023-08-27T11:30:12.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f02717fa700 1 -- 172.21.15.112:0/2136221373 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026801f070 con 0x7f02740b1270 2023-08-27T11:30:12.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f027af95700 1 -- 172.21.15.112:0/2136221373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02740acdd0 con 0x7f02740b1270 2023-08-27T11:30:12.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f02717fa700 1 -- 172.21.15.112:0/2136221373 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268011870 con 0x7f02740b1270 2023-08-27T11:30:12.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.150+0000 7f027af95700 1 -- 172.21.15.112:0/2136221373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0274152a00 con 0x7f02740b1270 2023-08-27T11:30:12.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.151+0000 7f027af95700 1 -- 172.21.15.112:0/2136221373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0274067a20 con 0x7f02740b1270 2023-08-27T11:30:12.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.152+0000 7f02717fa700 1 -- 172.21.15.112:0/2136221373 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f0268021030 con 0x7f02740b1270 2023-08-27T11:30:12.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.152+0000 7f02717fa700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f025806a4d0 0x7f025806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.164 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.152+0000 7f02717fa700 1 -- 172.21.15.112:0/2136221373 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f026808a1b0 con 0x7f02740b1270 2023-08-27T11:30:12.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.152+0000 7f0273fff700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f025806a4d0 0x7f025806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.153+0000 7f0273fff700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f025806a4d0 0x7f025806c990 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f02740caec0 tx=0x7f0260008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:12.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.156+0000 7f02717fa700 1 -- 172.21.15.112:0/2136221373 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0268055460 con 0x7f02740b1270 2023-08-27T11:30:12.344 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.343+0000 7f027af95700 1 -- 172.21.15.112:0/2136221373 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f027415ef80 con 0x7f025806a4d0 2023-08-27T11:30:12.353 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.352+0000 7f02717fa700 1 -- 172.21.15.112:0/2136221373 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310726 (secure 0 0 0) 0x7f026000a6f0 con 0x7f025806a4d0 2023-08-27T11:30:12.356 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:30:12.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.355+0000 7f027af95700 1 -- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f025806a4d0 msgr2=0x7f025806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.355+0000 7f027af95700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f025806a4d0 0x7f025806c990 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f02740caec0 tx=0x7f0260008040 comp rx=0 tx=0).stop 2023-08-27T11:30:12.359 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.355+0000 7f027af95700 1 -- 172.21.15.112:0/2136221373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02740b1270 msgr2=0x7f02741524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.355+0000 7f027af95700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02740b1270 0x7f02741524c0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f02740cff60 tx=0x7f0268004750 comp rx=0 tx=0).stop 2023-08-27T11:30:12.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.355+0000 7f027af95700 1 -- 172.21.15.112:0/2136221373 shutdown_connections 2023-08-27T11:30:12.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.355+0000 7f027af95700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f025806a4d0 0x7f025806c990 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.355+0000 7f027af95700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0274159070 0x7f02740717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.360 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.355+0000 7f027af95700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02740b1270 0x7f02741524c0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.355+0000 7f027af95700 1 --2- 172.21.15.112:0/2136221373 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0274158c90 0x7f02740ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.355+0000 7f027af95700 1 -- 172.21.15.112:0/2136221373 >> 172.21.15.112:0/2136221373 conn(0x7f02740d0000 msgr2=0x7f02740c3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:12.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.355+0000 7f027af95700 1 -- 172.21.15.112:0/2136221373 shutdown_connections 2023-08-27T11:30:12.361 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.355+0000 7f027af95700 1 -- 172.21.15.112:0/2136221373 wait complete. 2023-08-27T11:30:12.361 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:30:12.368 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":574,"stamp":"2023-08-27T11:30:10.427283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268692,"kb_used_data":19248,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754706028,"statfs":{"total":773094113280,"available":772818972672,"internally_reserved":0,"allocated":19709952,"data_stored":10028015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.937212"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375737+0000","last_change":"2023-08-27T11:30:07.373935+0000","last_active":"2023-08-27T11:30:07.375737+0000","last_peered":"2023-08-27T11:30:07.375737+0000","last_clean":"2023-08-27T11:30:07.375737+0000","last_became_active":"2023-08-27T11:30:07.373619+0000","last_became_peered":"2023-08-27T11:30:07.373619+0000","last_unstale":"2023-08-27T11:30:07.375737+0000","last_undegraded":"2023-08-27T11:30:07.375737+0000","last_fullsized":"2023-08-27T11:30:07.375737+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:16.699227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.384847+0000","last_change":"2023-08-27T11:30:07.380950+0000","last_active":"2023-08-27T11:30:07.384847+0000","last_peered":"2023-08-27T11:30:07.384847+0000","last_clean":"2023-08-27T11:30:07.384847+0000","last_became_active":"2023-08-27T11:30:07.380668+0000","last_became_peered":"2023-08-27T11:30:07.380668+0000","last_unstale":"2023-08-27T11:30:07.384847+0000","last_undegraded":"2023-08-27T11:30:07.384847+0000","last_fullsized":"2023-08-27T11:30:07.384847+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:27:15.104758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.387295+0000","last_change":"2023-08-27T11:30:07.383032+0000","last_active":"2023-08-27T11:30:07.387295+0000","last_peered":"2023-08-27T11:30:07.387295+0000","last_clean":"2023-08-27T11:30:07.387295+0000","last_became_active":"2023-08-27T11:30:07.382798+0000","last_became_peered":"2023-08-27T11:30:07.382798+0000","last_unstale":"2023-08-27T11:30:07.387295+0000","last_undegraded":"2023-08-27T11:30:07.387295+0000","last_fullsized":"2023-08-27T11:30:07.387295+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:06:13.527429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.379831+0000","last_change":"2023-08-27T11:30:07.378918+0000","last_active":"2023-08-27T11:30:07.379831+0000","last_peered":"2023-08-27T11:30:07.379831+0000","last_clean":"2023-08-27T11:30:07.379831+0000","last_became_active":"2023-08-27T11:30:07.378583+0000","last_became_peered":"2023-08-27T11:30:07.378583+0000","last_unstale":"2023-08-27T11:30:07.379831+0000","last_undegraded":"2023-08-27T11:30:07.379831+0000","last_fullsized":"2023-08-27T11:30:07.379831+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:17:10.066132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376893+0000","last_change":"2023-08-27T11:30:07.375355+0000","last_active":"2023-08-27T11:30:07.376893+0000","last_peered":"2023-08-27T11:30:07.376893+0000","last_clean":"2023-08-27T11:30:07.376893+0000","last_became_active":"2023-08-27T11:30:07.375064+0000","last_became_peered":"2023-08-27T11:30:07.375064+0000","last_unstale":"2023-08-27T11:30:07.376893+0000","last_undegraded":"2023-08-27T11:30:07.376893+0000","last_fullsized":"2023-08-27T11:30:07.376893+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:11:07.185199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.378756+0000","last_change":"2023-08-27T11:30:07.378756+0000","last_active":"2023-08-27T11:30:07.378756+0000","last_peered":"2023-08-27T11:30:07.378756+0000","last_clean":"2023-08-27T11:30:07.378756+0000","last_became_active":"2023-08-27T11:30:07.378495+0000","last_became_peered":"2023-08-27T11:30:07.378495+0000","last_unstale":"2023-08-27T11:30:07.378756+0000","last_undegraded":"2023-08-27T11:30:07.378756+0000","last_fullsized":"2023-08-27T11:30:07.378756+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:01:44.098335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380828+0000","last_change":"2023-08-27T11:30:07.377440+0000","last_active":"2023-08-27T11:30:07.380828+0000","last_peered":"2023-08-27T11:30:07.380828+0000","last_clean":"2023-08-27T11:30:07.380828+0000","last_became_active":"2023-08-27T11:30:07.376823+0000","last_became_peered":"2023-08-27T11:30:07.376823+0000","last_unstale":"2023-08-27T11:30:07.380828+0000","last_undegraded":"2023-08-27T11:30:07.380828+0000","last_fullsized":"2023-08-27T11:30:07.380828+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:38:41.760242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380236+0000","last_change":"2023-08-27T11:30:07.377813+0000","last_active":"2023-08-27T11:30:07.380236+0000","last_peered":"2023-08-27T11:30:07.380236+0000","last_clean":"2023-08-27T11:30:07.380236+0000","last_became_active":"2023-08-27T11:30:07.377471+0000","last_became_peered":"2023-08-27T11:30:07.377471+0000","last_unstale":"2023-08-27T11:30:07.380236+0000","last_undegraded":"2023-08-27T11:30:07.380236+0000","last_fullsized":"2023-08-27T11:30:07.380236+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:35:22.802502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.368157+0000","last_change":"2023-08-27T11:30:07.368157+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.368157+0000","last_undegraded":"2023-08-27T11:30:07.368157+0000","last_fullsized":"2023-08-27T11:30:07.368157+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:54:48.531209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.382327+0000","last_change":"2023-08-27T11:30:07.378494+0000","last_active":"2023-08-27T11:30:07.382327+0000","last_peered":"2023-08-27T11:30:07.382327+0000","last_clean":"2023-08-27T11:30:07.382327+0000","last_became_active":"2023-08-27T11:30:07.378196+0000","last_became_peered":"2023-08-27T11:30:07.378196+0000","last_unstale":"2023-08-27T11:30:07.382327+0000","last_undegraded":"2023-08-27T11:30:07.382327+0000","last_fullsized":"2023-08-27T11:30:07.382327+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:01:59.361767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.387354+0000","last_change":"2023-08-27T11:30:07.382784+0000","last_active":"2023-08-27T11:30:07.387354+0000","last_peered":"2023-08-27T11:30:07.387354+0000","last_clean":"2023-08-27T11:30:07.387354+0000","last_became_active":"2023-08-27T11:30:07.382436+0000","last_became_peered":"2023-08-27T11:30:07.382436+0000","last_unstale":"2023-08-27T11:30:07.387354+0000","last_undegraded":"2023-08-27T11:30:07.387354+0000","last_fullsized":"2023-08-27T11:30:07.387354+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:15:13.361952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375762+0000","last_change":"2023-08-27T11:30:07.372273+0000","last_active":"2023-08-27T11:30:07.375762+0000","last_peered":"2023-08-27T11:30:07.375762+0000","last_clean":"2023-08-27T11:30:07.375762+0000","last_became_active":"2023-08-27T11:30:07.371943+0000","last_became_peered":"2023-08-27T11:30:07.371943+0000","last_unstale":"2023-08-27T11:30:07.375762+0000","last_undegraded":"2023-08-27T11:30:07.375762+0000","last_fullsized":"2023-08-27T11:30:07.375762+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:06:45.150286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.363359+0000","last_change":"2023-08-27T11:30:07.363359+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.363359+0000","last_undegraded":"2023-08-27T11:30:07.363359+0000","last_fullsized":"2023-08-27T11:30:07.363359+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:05:25.871181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370483+0000","last_change":"2023-08-27T11:30:07.367703+0000","last_active":"2023-08-27T11:30:07.370483+0000","last_peered":"2023-08-27T11:30:07.370483+0000","last_clean":"2023-08-27T11:30:07.370483+0000","last_became_active":"2023-08-27T11:30:07.367443+0000","last_became_peered":"2023-08-27T11:30:07.367443+0000","last_unstale":"2023-08-27T11:30:07.370483+0000","last_undegraded":"2023-08-27T11:30:07.370483+0000","last_fullsized":"2023-08-27T11:30:07.370483+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:40:42.999295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376506+0000","last_change":"2023-08-27T11:30:07.376506+0000","last_active":"2023-08-27T11:30:07.376506+0000","last_peered":"2023-08-27T11:30:07.376506+0000","last_clean":"2023-08-27T11:30:07.376506+0000","last_became_active":"2023-08-27T11:30:07.376242+0000","last_became_peered":"2023-08-27T11:30:07.376242+0000","last_unstale":"2023-08-27T11:30:07.376506+0000","last_undegraded":"2023-08-27T11:30:07.376506+0000","last_fullsized":"2023-08-27T11:30:07.376506+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:41:53.411101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377343+0000","last_change":"2023-08-27T11:30:07.377343+0000","last_active":"2023-08-27T11:30:07.377343+0000","last_peered":"2023-08-27T11:30:07.377343+0000","last_clean":"2023-08-27T11:30:07.377343+0000","last_became_active":"2023-08-27T11:30:07.377032+0000","last_became_peered":"2023-08-27T11:30:07.377032+0000","last_unstale":"2023-08-27T11:30:07.377343+0000","last_undegraded":"2023-08-27T11:30:07.377343+0000","last_fullsized":"2023-08-27T11:30:07.377343+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:53:51.803335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.496034+0000","last_change":"2023-08-27T11:30:06.496034+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.496034+0000","last_undegraded":"2023-08-27T11:30:06.496034+0000","last_fullsized":"2023-08-27T11:30:06.496034+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.382296+0000","last_change":"2023-08-27T11:30:07.376652+0000","last_active":"2023-08-27T11:30:07.382296+0000","last_peered":"2023-08-27T11:30:07.382296+0000","last_clean":"2023-08-27T11:30:07.382296+0000","last_became_active":"2023-08-27T11:30:07.376381+0000","last_became_peered":"2023-08-27T11:30:07.376381+0000","last_unstale":"2023-08-27T11:30:07.382296+0000","last_undegraded":"2023-08-27T11:30:07.382296+0000","last_fullsized":"2023-08-27T11:30:07.382296+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:47:41.397698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380879+0000","last_change":"2023-08-27T11:30:07.377266+0000","last_active":"2023-08-27T11:30:07.380879+0000","last_peered":"2023-08-27T11:30:07.380879+0000","last_clean":"2023-08-27T11:30:07.380879+0000","last_became_active":"2023-08-27T11:30:07.376986+0000","last_became_peered":"2023-08-27T11:30:07.376986+0000","last_unstale":"2023-08-27T11:30:07.380879+0000","last_undegraded":"2023-08-27T11:30:07.380879+0000","last_fullsized":"2023-08-27T11:30:07.380879+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:46:44.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376491+0000","last_change":"2023-08-27T11:30:07.376491+0000","last_active":"2023-08-27T11:30:07.376491+0000","last_peered":"2023-08-27T11:30:07.376491+0000","last_clean":"2023-08-27T11:30:07.376491+0000","last_became_active":"2023-08-27T11:30:07.376247+0000","last_became_peered":"2023-08-27T11:30:07.376247+0000","last_unstale":"2023-08-27T11:30:07.376491+0000","last_undegraded":"2023-08-27T11:30:07.376491+0000","last_fullsized":"2023-08-27T11:30:07.376491+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:09:11.582134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.368242+0000","last_change":"2023-08-27T11:30:07.368242+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.368242+0000","last_undegraded":"2023-08-27T11:30:07.368242+0000","last_fullsized":"2023-08-27T11:30:07.368242+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:46:21.277652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.369379+0000","last_change":"2023-08-27T11:30:07.369379+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.369379+0000","last_undegraded":"2023-08-27T11:30:07.369379+0000","last_fullsized":"2023-08-27T11:30:07.369379+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:18:50.569923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380789+0000","last_change":"2023-08-27T11:30:07.377088+0000","last_active":"2023-08-27T11:30:07.380789+0000","last_peered":"2023-08-27T11:30:07.380789+0000","last_clean":"2023-08-27T11:30:07.380789+0000","last_became_active":"2023-08-27T11:30:07.376059+0000","last_became_peered":"2023-08-27T11:30:07.376059+0000","last_unstale":"2023-08-27T11:30:07.380789+0000","last_undegraded":"2023-08-27T11:30:07.380789+0000","last_fullsized":"2023-08-27T11:30:07.380789+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:57.640378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.495396+0000","last_change":"2023-08-27T11:30:06.495396+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.495396+0000","last_undegraded":"2023-08-27T11:30:06.495396+0000","last_fullsized":"2023-08-27T11:30:06.495396+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375518+0000","last_change":"2023-08-27T11:30:07.370865+0000","last_active":"2023-08-27T11:30:07.375518+0000","last_peered":"2023-08-27T11:30:07.375518+0000","last_clean":"2023-08-27T11:30:07.375518+0000","last_became_active":"2023-08-27T11:30:07.370571+0000","last_became_peered":"2023-08-27T11:30:07.370571+0000","last_unstale":"2023-08-27T11:30:07.375518+0000","last_undegraded":"2023-08-27T11:30:07.375518+0000","last_fullsized":"2023-08-27T11:30:07.375518+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:45:40.891269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.511830+0000","last_change":"2023-08-27T11:30:06.511151+0000","last_active":"2023-08-27T11:30:06.511830+0000","last_peered":"2023-08-27T11:30:06.511830+0000","last_clean":"2023-08-27T11:30:06.511830+0000","last_became_active":"2023-08-27T11:30:06.510830+0000","last_became_peered":"2023-08-27T11:30:06.510830+0000","last_unstale":"2023-08-27T11:30:06.511830+0000","last_undegraded":"2023-08-27T11:30:06.511830+0000","last_fullsized":"2023-08-27T11:30:06.511830+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:26:30.060104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.378015+0000","last_change":"2023-08-27T11:30:07.378015+0000","last_active":"2023-08-27T11:30:07.378015+0000","last_peered":"2023-08-27T11:30:07.378015+0000","last_clean":"2023-08-27T11:30:07.378015+0000","last_became_active":"2023-08-27T11:30:07.377749+0000","last_became_peered":"2023-08-27T11:30:07.377749+0000","last_unstale":"2023-08-27T11:30:07.378015+0000","last_undegraded":"2023-08-27T11:30:07.378015+0000","last_fullsized":"2023-08-27T11:30:07.378015+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:40.421055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.365033+0000","last_change":"2023-08-27T11:30:06.506992+0000","last_active":"2023-08-27T11:30:07.365033+0000","last_peered":"2023-08-27T11:30:07.365033+0000","last_clean":"2023-08-27T11:30:07.365033+0000","last_became_active":"2023-08-27T11:30:06.506671+0000","last_became_peered":"2023-08-27T11:30:06.506671+0000","last_unstale":"2023-08-27T11:30:07.365033+0000","last_undegraded":"2023-08-27T11:30:07.365033+0000","last_fullsized":"2023-08-27T11:30:07.365033+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:07:13.889636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.387319+0000","last_change":"2023-08-27T11:30:07.383160+0000","last_active":"2023-08-27T11:30:07.387319+0000","last_peered":"2023-08-27T11:30:07.387319+0000","last_clean":"2023-08-27T11:30:07.387319+0000","last_became_active":"2023-08-27T11:30:07.382823+0000","last_became_peered":"2023-08-27T11:30:07.382823+0000","last_unstale":"2023-08-27T11:30:07.387319+0000","last_undegraded":"2023-08-27T11:30:07.387319+0000","last_fullsized":"2023-08-27T11:30:07.387319+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:13.020805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377069+0000","last_change":"2023-08-27T11:30:06.509202+0000","last_active":"2023-08-27T11:30:07.377069+0000","last_peered":"2023-08-27T11:30:07.377069+0000","last_clean":"2023-08-27T11:30:07.377069+0000","last_became_active":"2023-08-27T11:30:06.508960+0000","last_became_peered":"2023-08-27T11:30:06.508960+0000","last_unstale":"2023-08-27T11:30:07.377069+0000","last_undegraded":"2023-08-27T11:30:07.377069+0000","last_fullsized":"2023-08-27T11:30:07.377069+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:41:59.638609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375837+0000","last_change":"2023-08-27T11:30:07.373389+0000","last_active":"2023-08-27T11:30:07.375837+0000","last_peered":"2023-08-27T11:30:07.375837+0000","last_clean":"2023-08-27T11:30:07.375837+0000","last_became_active":"2023-08-27T11:30:07.373083+0000","last_became_peered":"2023-08-27T11:30:07.373083+0000","last_unstale":"2023-08-27T11:30:07.375837+0000","last_undegraded":"2023-08-27T11:30:07.375837+0000","last_fullsized":"2023-08-27T11:30:07.375837+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:01:00.108886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.374018+0000","last_change":"2023-08-27T11:30:06.509017+0000","last_active":"2023-08-27T11:30:07.374018+0000","last_peered":"2023-08-27T11:30:07.374018+0000","last_clean":"2023-08-27T11:30:07.374018+0000","last_became_active":"2023-08-27T11:30:06.508619+0000","last_became_peered":"2023-08-27T11:30:06.508619+0000","last_unstale":"2023-08-27T11:30:07.374018+0000","last_undegraded":"2023-08-27T11:30:07.374018+0000","last_fullsized":"2023-08-27T11:30:07.374018+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:01:51.459390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.368141+0000","last_change":"2023-08-27T11:30:07.368141+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.368141+0000","last_undegraded":"2023-08-27T11:30:07.368141+0000","last_fullsized":"2023-08-27T11:30:07.368141+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:28:48.644407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.362633+0000","last_change":"2023-08-27T11:30:06.503177+0000","last_active":"2023-08-27T11:30:07.362633+0000","last_peered":"2023-08-27T11:30:07.362633+0000","last_clean":"2023-08-27T11:30:07.362633+0000","last_became_active":"2023-08-27T11:30:06.502915+0000","last_became_peered":"2023-08-27T11:30:06.502915+0000","last_unstale":"2023-08-27T11:30:07.362633+0000","last_undegraded":"2023-08-27T11:30:07.362633+0000","last_fullsized":"2023-08-27T11:30:07.362633+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:04:47.203315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":4,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.377944+0000","last_change":"2023-08-27T11:30:07.372096+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.377944+0000","last_undegraded":"2023-08-27T11:30:07.377944+0000","last_fullsized":"2023-08-27T11:30:07.377944+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:49:33.070564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.508333+0000","last_change":"2023-08-27T11:30:06.503408+0000","last_active":"2023-08-27T11:30:06.508333+0000","last_peered":"2023-08-27T11:30:06.508333+0000","last_clean":"2023-08-27T11:30:06.508333+0000","last_became_active":"2023-08-27T11:30:06.503111+0000","last_became_peered":"2023-08-27T11:30:06.503111+0000","last_unstale":"2023-08-27T11:30:06.508333+0000","last_undegraded":"2023-08-27T11:30:06.508333+0000","last_fullsized":"2023-08-27T11:30:06.508333+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:54:21.203272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.369922+0000","last_change":"2023-08-27T11:30:07.369922+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.369922+0000","last_undegraded":"2023-08-27T11:30:07.369922+0000","last_fullsized":"2023-08-27T11:30:07.369922+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:34.484460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.371984+0000","last_change":"2023-08-27T11:30:06.506988+0000","last_active":"2023-08-27T11:30:07.371984+0000","last_peered":"2023-08-27T11:30:07.371984+0000","last_clean":"2023-08-27T11:30:07.371984+0000","last_became_active":"2023-08-27T11:30:06.506687+0000","last_became_peered":"2023-08-27T11:30:06.506687+0000","last_unstale":"2023-08-27T11:30:07.371984+0000","last_undegraded":"2023-08-27T11:30:07.371984+0000","last_fullsized":"2023-08-27T11:30:07.371984+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:55:51.579443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380804+0000","last_change":"2023-08-27T11:30:07.377249+0000","last_active":"2023-08-27T11:30:07.380804+0000","last_peered":"2023-08-27T11:30:07.380804+0000","last_clean":"2023-08-27T11:30:07.380804+0000","last_became_active":"2023-08-27T11:30:07.376448+0000","last_became_peered":"2023-08-27T11:30:07.376448+0000","last_unstale":"2023-08-27T11:30:07.380804+0000","last_undegraded":"2023-08-27T11:30:07.380804+0000","last_fullsized":"2023-08-27T11:30:07.380804+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:02:14.837924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.363301+0000","last_change":"2023-08-27T11:30:06.507787+0000","last_active":"2023-08-27T11:30:07.363301+0000","last_peered":"2023-08-27T11:30:07.363301+0000","last_clean":"2023-08-27T11:30:07.363301+0000","last_became_active":"2023-08-27T11:30:06.507390+0000","last_became_peered":"2023-08-27T11:30:06.507390+0000","last_unstale":"2023-08-27T11:30:07.363301+0000","last_undegraded":"2023-08-27T11:30:07.363301+0000","last_fullsized":"2023-08-27T11:30:07.363301+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:00:38.629013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380650+0000","last_change":"2023-08-27T11:30:07.376530+0000","last_active":"2023-08-27T11:30:07.380650+0000","last_peered":"2023-08-27T11:30:07.380650+0000","last_clean":"2023-08-27T11:30:07.380650+0000","last_became_active":"2023-08-27T11:30:07.376221+0000","last_became_peered":"2023-08-27T11:30:07.376221+0000","last_unstale":"2023-08-27T11:30:07.380650+0000","last_undegraded":"2023-08-27T11:30:07.380650+0000","last_fullsized":"2023-08-27T11:30:07.380650+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:41.856918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.510282+0000","last_change":"2023-08-27T11:30:06.508059+0000","last_active":"2023-08-27T11:30:06.510282+0000","last_peered":"2023-08-27T11:30:06.510282+0000","last_clean":"2023-08-27T11:30:06.510282+0000","last_became_active":"2023-08-27T11:30:06.507677+0000","last_became_peered":"2023-08-27T11:30:06.507677+0000","last_unstale":"2023-08-27T11:30:06.510282+0000","last_undegraded":"2023-08-27T11:30:06.510282+0000","last_fullsized":"2023-08-27T11:30:06.510282+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:50:44.051625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.495697+0000","last_change":"2023-08-27T11:30:06.495697+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.495697+0000","last_undegraded":"2023-08-27T11:30:06.495697+0000","last_fullsized":"2023-08-27T11:30:06.495697+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375338+0000","last_change":"2023-08-27T11:30:06.509700+0000","last_active":"2023-08-27T11:30:07.375338+0000","last_peered":"2023-08-27T11:30:07.375338+0000","last_clean":"2023-08-27T11:30:07.375338+0000","last_became_active":"2023-08-27T11:30:06.509455+0000","last_became_peered":"2023-08-27T11:30:06.509455+0000","last_unstale":"2023-08-27T11:30:07.375338+0000","last_undegraded":"2023-08-27T11:30:07.375338+0000","last_fullsized":"2023-08-27T11:30:07.375338+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:12:09.969178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.373978+0000","last_change":"2023-08-27T11:30:07.373333+0000","last_active":"2023-08-27T11:30:07.373978+0000","last_peered":"2023-08-27T11:30:07.373978+0000","last_clean":"2023-08-27T11:30:07.373978+0000","last_became_active":"2023-08-27T11:30:07.373009+0000","last_became_peered":"2023-08-27T11:30:07.373009+0000","last_unstale":"2023-08-27T11:30:07.373978+0000","last_undegraded":"2023-08-27T11:30:07.373978+0000","last_fullsized":"2023-08-27T11:30:07.373978+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:24:21.667407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377251+0000","last_change":"2023-08-27T11:30:06.509723+0000","last_active":"2023-08-27T11:30:07.377251+0000","last_peered":"2023-08-27T11:30:07.377251+0000","last_clean":"2023-08-27T11:30:07.377251+0000","last_became_active":"2023-08-27T11:30:06.509418+0000","last_became_peered":"2023-08-27T11:30:06.509418+0000","last_unstale":"2023-08-27T11:30:07.377251+0000","last_undegraded":"2023-08-27T11:30:07.377251+0000","last_fullsized":"2023-08-27T11:30:07.377251+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:23:19.197311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.494625+0000","last_change":"2023-08-27T11:30:06.494625+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.494625+0000","last_undegraded":"2023-08-27T11:30:06.494625+0000","last_fullsized":"2023-08-27T11:30:06.494625+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.508364+0000","last_change":"2023-08-27T11:30:06.503031+0000","last_active":"2023-08-27T11:30:06.508364+0000","last_peered":"2023-08-27T11:30:06.508364+0000","last_clean":"2023-08-27T11:30:06.508364+0000","last_became_active":"2023-08-27T11:30:06.502780+0000","last_became_peered":"2023-08-27T11:30:06.502780+0000","last_unstale":"2023-08-27T11:30:06.508364+0000","last_undegraded":"2023-08-27T11:30:06.508364+0000","last_fullsized":"2023-08-27T11:30:06.508364+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:39:40.298338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.496930+0000","last_change":"2023-08-27T11:30:06.496930+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.496930+0000","last_undegraded":"2023-08-27T11:30:06.496930+0000","last_fullsized":"2023-08-27T11:30:06.496930+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.365524+0000","last_change":"2023-08-27T11:30:06.511539+0000","last_active":"2023-08-27T11:30:07.365524+0000","last_peered":"2023-08-27T11:30:07.365524+0000","last_clean":"2023-08-27T11:30:07.365524+0000","last_became_active":"2023-08-27T11:30:06.511200+0000","last_became_peered":"2023-08-27T11:30:06.511200+0000","last_unstale":"2023-08-27T11:30:07.365524+0000","last_undegraded":"2023-08-27T11:30:07.365524+0000","last_fullsized":"2023-08-27T11:30:07.365524+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:01.722653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.371737+0000","last_change":"2023-08-27T11:30:07.371737+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.371737+0000","last_undegraded":"2023-08-27T11:30:07.371737+0000","last_fullsized":"2023-08-27T11:30:07.371737+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:39:16.696934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370740+0000","last_change":"2023-08-27T11:30:06.505422+0000","last_active":"2023-08-27T11:30:07.370740+0000","last_peered":"2023-08-27T11:30:07.370740+0000","last_clean":"2023-08-27T11:30:07.370740+0000","last_became_active":"2023-08-27T11:30:06.505120+0000","last_became_peered":"2023-08-27T11:30:06.505120+0000","last_unstale":"2023-08-27T11:30:07.370740+0000","last_undegraded":"2023-08-27T11:30:07.370740+0000","last_fullsized":"2023-08-27T11:30:07.370740+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:21:11.370238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.494948+0000","last_change":"2023-08-27T11:30:06.494948+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.494948+0000","last_undegraded":"2023-08-27T11:30:06.494948+0000","last_fullsized":"2023-08-27T11:30:06.494948+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.373236+0000","last_change":"2023-08-27T11:30:06.506911+0000","last_active":"2023-08-27T11:30:07.373236+0000","last_peered":"2023-08-27T11:30:07.373236+0000","last_clean":"2023-08-27T11:30:07.373236+0000","last_became_active":"2023-08-27T11:30:06.506657+0000","last_became_peered":"2023-08-27T11:30:06.506657+0000","last_unstale":"2023-08-27T11:30:07.373236+0000","last_undegraded":"2023-08-27T11:30:07.373236+0000","last_fullsized":"2023-08-27T11:30:07.373236+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:05.904960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.497037+0000","last_change":"2023-08-27T11:30:06.497037+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.497037+0000","last_undegraded":"2023-08-27T11:30:06.497037+0000","last_fullsized":"2023-08-27T11:30:06.497037+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.374141+0000","last_change":"2023-08-27T11:30:06.516142+0000","last_active":"2023-08-27T11:30:07.374141+0000","last_peered":"2023-08-27T11:30:07.374141+0000","last_clean":"2023-08-27T11:30:07.374141+0000","last_became_active":"2023-08-27T11:30:06.515761+0000","last_became_peered":"2023-08-27T11:30:06.515761+0000","last_unstale":"2023-08-27T11:30:07.374141+0000","last_undegraded":"2023-08-27T11:30:07.374141+0000","last_fullsized":"2023-08-27T11:30:07.374141+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:12:05.165692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376818+0000","last_change":"2023-08-27T11:30:07.376818+0000","last_active":"2023-08-27T11:30:07.376818+0000","last_peered":"2023-08-27T11:30:07.376818+0000","last_clean":"2023-08-27T11:30:07.376818+0000","last_became_active":"2023-08-27T11:30:07.376575+0000","last_became_peered":"2023-08-27T11:30:07.376575+0000","last_unstale":"2023-08-27T11:30:07.376818+0000","last_undegraded":"2023-08-27T11:30:07.376818+0000","last_fullsized":"2023-08-27T11:30:07.376818+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:14:01.759128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.366408+0000","last_change":"2023-08-27T11:30:06.504618+0000","last_active":"2023-08-27T11:30:07.366408+0000","last_peered":"2023-08-27T11:30:07.366408+0000","last_clean":"2023-08-27T11:30:07.366408+0000","last_became_active":"2023-08-27T11:30:06.504354+0000","last_became_peered":"2023-08-27T11:30:06.504354+0000","last_unstale":"2023-08-27T11:30:07.366408+0000","last_undegraded":"2023-08-27T11:30:07.366408+0000","last_fullsized":"2023-08-27T11:30:07.366408+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:09:49.437363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.384834+0000","last_change":"2023-08-27T11:30:07.380702+0000","last_active":"2023-08-27T11:30:07.384834+0000","last_peered":"2023-08-27T11:30:07.384834+0000","last_clean":"2023-08-27T11:30:07.384834+0000","last_became_active":"2023-08-27T11:30:07.380378+0000","last_became_peered":"2023-08-27T11:30:07.380378+0000","last_unstale":"2023-08-27T11:30:07.384834+0000","last_undegraded":"2023-08-27T11:30:07.384834+0000","last_fullsized":"2023-08-27T11:30:07.384834+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:06:41.915453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375394+0000","last_change":"2023-08-27T11:30:06.511005+0000","last_active":"2023-08-27T11:30:07.375394+0000","last_peered":"2023-08-27T11:30:07.375394+0000","last_clean":"2023-08-27T11:30:07.375394+0000","last_became_active":"2023-08-27T11:30:06.510712+0000","last_became_peered":"2023-08-27T11:30:06.510712+0000","last_unstale":"2023-08-27T11:30:07.375394+0000","last_undegraded":"2023-08-27T11:30:07.375394+0000","last_fullsized":"2023-08-27T11:30:07.375394+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:53:54.707523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377430+0000","last_change":"2023-08-27T11:30:07.377430+0000","last_active":"2023-08-27T11:30:07.377430+0000","last_peered":"2023-08-27T11:30:07.377430+0000","last_clean":"2023-08-27T11:30:07.377430+0000","last_became_active":"2023-08-27T11:30:07.376459+0000","last_became_peered":"2023-08-27T11:30:07.376459+0000","last_unstale":"2023-08-27T11:30:07.377430+0000","last_undegraded":"2023-08-27T11:30:07.377430+0000","last_fullsized":"2023-08-27T11:30:07.377430+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:51:11.653496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372581+0000","last_change":"2023-08-27T11:30:06.510880+0000","last_active":"2023-08-27T11:30:07.372581+0000","last_peered":"2023-08-27T11:30:07.372581+0000","last_clean":"2023-08-27T11:30:07.372581+0000","last_became_active":"2023-08-27T11:30:06.510602+0000","last_became_peered":"2023-08-27T11:30:06.510602+0000","last_unstale":"2023-08-27T11:30:07.372581+0000","last_undegraded":"2023-08-27T11:30:07.372581+0000","last_fullsized":"2023-08-27T11:30:07.372581+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:12.746422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377066+0000","last_change":"2023-08-27T11:30:07.374685+0000","last_active":"2023-08-27T11:30:07.377066+0000","last_peered":"2023-08-27T11:30:07.377066+0000","last_clean":"2023-08-27T11:30:07.377066+0000","last_became_active":"2023-08-27T11:30:07.374131+0000","last_became_peered":"2023-08-27T11:30:07.374131+0000","last_unstale":"2023-08-27T11:30:07.377066+0000","last_undegraded":"2023-08-27T11:30:07.377066+0000","last_fullsized":"2023-08-27T11:30:07.377066+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:22:46.104350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.364655+0000","last_change":"2023-08-27T11:30:06.515673+0000","last_active":"2023-08-27T11:30:07.364655+0000","last_peered":"2023-08-27T11:30:07.364655+0000","last_clean":"2023-08-27T11:30:07.364655+0000","last_became_active":"2023-08-27T11:30:06.515364+0000","last_became_peered":"2023-08-27T11:30:06.515364+0000","last_unstale":"2023-08-27T11:30:07.364655+0000","last_undegraded":"2023-08-27T11:30:07.364655+0000","last_fullsized":"2023-08-27T11:30:07.364655+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:05:05.778823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.378734+0000","last_change":"2023-08-27T11:30:07.378734+0000","last_active":"2023-08-27T11:30:07.378734+0000","last_peered":"2023-08-27T11:30:07.378734+0000","last_clean":"2023-08-27T11:30:07.378734+0000","last_became_active":"2023-08-27T11:30:07.378472+0000","last_became_peered":"2023-08-27T11:30:07.378472+0000","last_unstale":"2023-08-27T11:30:07.378734+0000","last_undegraded":"2023-08-27T11:30:07.378734+0000","last_fullsized":"2023-08-27T11:30:07.378734+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:46:20.644887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.369874+0000","last_change":"2023-08-27T11:30:06.504079+0000","last_active":"2023-08-27T11:30:07.369874+0000","last_peered":"2023-08-27T11:30:07.369874+0000","last_clean":"2023-08-27T11:30:07.369874+0000","last_became_active":"2023-08-27T11:30:06.503811+0000","last_became_peered":"2023-08-27T11:30:06.503811+0000","last_unstale":"2023-08-27T11:30:07.369874+0000","last_undegraded":"2023-08-27T11:30:07.369874+0000","last_fullsized":"2023-08-27T11:30:07.369874+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:27:05.776026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":4,"reported_epoch":183,"state":"creating+activating","last_fresh":"2023-08-27T11:30:07.376957+0000","last_change":"2023-08-27T11:30:07.371363+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:07.376957+0000","last_undegraded":"2023-08-27T11:30:07.376957+0000","last_fullsized":"2023-08-27T11:30:07.376957+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:49:27.497988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.365622+0000","last_change":"2023-08-27T11:30:06.503387+0000","last_active":"2023-08-27T11:30:07.365622+0000","last_peered":"2023-08-27T11:30:07.365622+0000","last_clean":"2023-08-27T11:30:07.365622+0000","last_became_active":"2023-08-27T11:30:06.503081+0000","last_became_peered":"2023-08-27T11:30:06.503081+0000","last_unstale":"2023-08-27T11:30:07.365622+0000","last_undegraded":"2023-08-27T11:30:07.365622+0000","last_fullsized":"2023-08-27T11:30:07.365622+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:25:44.805880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.494479+0000","last_change":"2023-08-27T11:30:06.494479+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.494479+0000","last_undegraded":"2023-08-27T11:30:06.494479+0000","last_fullsized":"2023-08-27T11:30:06.494479+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372445+0000","last_change":"2023-08-27T11:30:06.511539+0000","last_active":"2023-08-27T11:30:07.372445+0000","last_peered":"2023-08-27T11:30:07.372445+0000","last_clean":"2023-08-27T11:30:07.372445+0000","last_became_active":"2023-08-27T11:30:06.511248+0000","last_became_peered":"2023-08-27T11:30:06.511248+0000","last_unstale":"2023-08-27T11:30:07.372445+0000","last_undegraded":"2023-08-27T11:30:07.372445+0000","last_fullsized":"2023-08-27T11:30:07.372445+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:22:48.182930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376018+0000","last_change":"2023-08-27T11:30:07.373745+0000","last_active":"2023-08-27T11:30:07.376018+0000","last_peered":"2023-08-27T11:30:07.376018+0000","last_clean":"2023-08-27T11:30:07.376018+0000","last_became_active":"2023-08-27T11:30:07.373460+0000","last_became_peered":"2023-08-27T11:30:07.373460+0000","last_unstale":"2023-08-27T11:30:07.376018+0000","last_undegraded":"2023-08-27T11:30:07.376018+0000","last_fullsized":"2023-08-27T11:30:07.376018+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:47:30.525294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.366107+0000","last_change":"2023-08-27T11:30:06.502148+0000","last_active":"2023-08-27T11:30:07.366107+0000","last_peered":"2023-08-27T11:30:07.366107+0000","last_clean":"2023-08-27T11:30:07.366107+0000","last_became_active":"2023-08-27T11:30:06.501869+0000","last_became_peered":"2023-08-27T11:30:06.501869+0000","last_unstale":"2023-08-27T11:30:07.366107+0000","last_undegraded":"2023-08-27T11:30:07.366107+0000","last_fullsized":"2023-08-27T11:30:07.366107+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:19:08.944028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376696+0000","last_change":"2023-08-27T11:30:07.376696+0000","last_active":"2023-08-27T11:30:07.376696+0000","last_peered":"2023-08-27T11:30:07.376696+0000","last_clean":"2023-08-27T11:30:07.376696+0000","last_became_active":"2023-08-27T11:30:07.376266+0000","last_became_peered":"2023-08-27T11:30:07.376266+0000","last_unstale":"2023-08-27T11:30:07.376696+0000","last_undegraded":"2023-08-27T11:30:07.376696+0000","last_fullsized":"2023-08-27T11:30:07.376696+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:50:58.773261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.364173+0000","last_change":"2023-08-27T11:30:06.504227+0000","last_active":"2023-08-27T11:30:07.364173+0000","last_peered":"2023-08-27T11:30:07.364173+0000","last_clean":"2023-08-27T11:30:07.364173+0000","last_became_active":"2023-08-27T11:30:06.503935+0000","last_became_peered":"2023-08-27T11:30:06.503935+0000","last_unstale":"2023-08-27T11:30:07.364173+0000","last_undegraded":"2023-08-27T11:30:07.364173+0000","last_fullsized":"2023-08-27T11:30:07.364173+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:44:29.308537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.495279+0000","last_change":"2023-08-27T11:30:06.495279+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.495279+0000","last_undegraded":"2023-08-27T11:30:06.495279+0000","last_fullsized":"2023-08-27T11:30:06.495279+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370211+0000","last_change":"2023-08-27T11:30:06.515077+0000","last_active":"2023-08-27T11:30:07.370211+0000","last_peered":"2023-08-27T11:30:07.370211+0000","last_clean":"2023-08-27T11:30:07.370211+0000","last_became_active":"2023-08-27T11:30:06.514812+0000","last_became_peered":"2023-08-27T11:30:06.514812+0000","last_unstale":"2023-08-27T11:30:07.370211+0000","last_undegraded":"2023-08-27T11:30:07.370211+0000","last_fullsized":"2023-08-27T11:30:07.370211+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:56:41.368080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377075+0000","last_change":"2023-08-27T11:30:07.375476+0000","last_active":"2023-08-27T11:30:07.377075+0000","last_peered":"2023-08-27T11:30:07.377075+0000","last_clean":"2023-08-27T11:30:07.377075+0000","last_became_active":"2023-08-27T11:30:07.375167+0000","last_became_peered":"2023-08-27T11:30:07.375167+0000","last_unstale":"2023-08-27T11:30:07.377075+0000","last_undegraded":"2023-08-27T11:30:07.377075+0000","last_fullsized":"2023-08-27T11:30:07.377075+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:18:51.080195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.363503+0000","last_change":"2023-08-27T11:30:06.507172+0000","last_active":"2023-08-27T11:30:07.363503+0000","last_peered":"2023-08-27T11:30:07.363503+0000","last_clean":"2023-08-27T11:30:07.363503+0000","last_became_active":"2023-08-27T11:30:06.506885+0000","last_became_peered":"2023-08-27T11:30:06.506885+0000","last_unstale":"2023-08-27T11:30:07.363503+0000","last_undegraded":"2023-08-27T11:30:07.363503+0000","last_fullsized":"2023-08-27T11:30:07.363503+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:03.298758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.387326+0000","last_change":"2023-08-27T11:30:07.383116+0000","last_active":"2023-08-27T11:30:07.387326+0000","last_peered":"2023-08-27T11:30:07.387326+0000","last_clean":"2023-08-27T11:30:07.387326+0000","last_became_active":"2023-08-27T11:30:07.382838+0000","last_became_peered":"2023-08-27T11:30:07.382838+0000","last_unstale":"2023-08-27T11:30:07.387326+0000","last_undegraded":"2023-08-27T11:30:07.387326+0000","last_fullsized":"2023-08-27T11:30:07.387326+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:48:13.826189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.510301+0000","last_change":"2023-08-27T11:30:06.508063+0000","last_active":"2023-08-27T11:30:06.510301+0000","last_peered":"2023-08-27T11:30:06.510301+0000","last_clean":"2023-08-27T11:30:06.510301+0000","last_became_active":"2023-08-27T11:30:06.507412+0000","last_became_peered":"2023-08-27T11:30:06.507412+0000","last_unstale":"2023-08-27T11:30:06.510301+0000","last_undegraded":"2023-08-27T11:30:06.510301+0000","last_fullsized":"2023-08-27T11:30:06.510301+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:58.753023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.374655+0000","last_change":"2023-08-27T11:30:07.374655+0000","last_active":"2023-08-27T11:30:07.374655+0000","last_peered":"2023-08-27T11:30:07.374655+0000","last_clean":"2023-08-27T11:30:07.374655+0000","last_became_active":"2023-08-27T11:30:07.374306+0000","last_became_peered":"2023-08-27T11:30:07.374306+0000","last_unstale":"2023-08-27T11:30:07.374655+0000","last_undegraded":"2023-08-27T11:30:07.374655+0000","last_fullsized":"2023-08-27T11:30:07.374655+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:31:23.749135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370729+0000","last_change":"2023-08-27T11:30:06.500047+0000","last_active":"2023-08-27T11:30:07.370729+0000","last_peered":"2023-08-27T11:30:07.370729+0000","last_clean":"2023-08-27T11:30:07.370729+0000","last_became_active":"2023-08-27T11:30:06.499775+0000","last_became_peered":"2023-08-27T11:30:06.499775+0000","last_unstale":"2023-08-27T11:30:07.370729+0000","last_undegraded":"2023-08-27T11:30:07.370729+0000","last_fullsized":"2023-08-27T11:30:07.370729+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:13.320294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376417+0000","last_change":"2023-08-27T11:30:07.376417+0000","last_active":"2023-08-27T11:30:07.376417+0000","last_peered":"2023-08-27T11:30:07.376417+0000","last_clean":"2023-08-27T11:30:07.376417+0000","last_became_active":"2023-08-27T11:30:07.376144+0000","last_became_peered":"2023-08-27T11:30:07.376144+0000","last_unstale":"2023-08-27T11:30:07.376417+0000","last_undegraded":"2023-08-27T11:30:07.376417+0000","last_fullsized":"2023-08-27T11:30:07.376417+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:07:51.665760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370869+0000","last_change":"2023-08-27T11:30:06.505423+0000","last_active":"2023-08-27T11:30:07.370869+0000","last_peered":"2023-08-27T11:30:07.370869+0000","last_clean":"2023-08-27T11:30:07.370869+0000","last_became_active":"2023-08-27T11:30:06.505138+0000","last_became_peered":"2023-08-27T11:30:06.505138+0000","last_unstale":"2023-08-27T11:30:07.370869+0000","last_undegraded":"2023-08-27T11:30:07.370869+0000","last_fullsized":"2023-08-27T11:30:07.370869+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:12:32.588899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375665+0000","last_change":"2023-08-27T11:30:07.375665+0000","last_active":"2023-08-27T11:30:07.375665+0000","last_peered":"2023-08-27T11:30:07.375665+0000","last_clean":"2023-08-27T11:30:07.375665+0000","last_became_active":"2023-08-27T11:30:07.375184+0000","last_became_peered":"2023-08-27T11:30:07.375184+0000","last_unstale":"2023-08-27T11:30:07.375665+0000","last_undegraded":"2023-08-27T11:30:07.375665+0000","last_fullsized":"2023-08-27T11:30:07.375665+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:42.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.363224+0000","last_change":"2023-08-27T11:30:06.504990+0000","last_active":"2023-08-27T11:30:07.363224+0000","last_peered":"2023-08-27T11:30:07.363224+0000","last_clean":"2023-08-27T11:30:07.363224+0000","last_became_active":"2023-08-27T11:30:06.504768+0000","last_became_peered":"2023-08-27T11:30:06.504768+0000","last_unstale":"2023-08-27T11:30:07.363224+0000","last_undegraded":"2023-08-27T11:30:07.363224+0000","last_fullsized":"2023-08-27T11:30:07.363224+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:27:04.823447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.377092+0000","last_change":"2023-08-27T11:30:07.376099+0000","last_active":"2023-08-27T11:30:07.377092+0000","last_peered":"2023-08-27T11:30:07.377092+0000","last_clean":"2023-08-27T11:30:07.377092+0000","last_became_active":"2023-08-27T11:30:07.375823+0000","last_became_peered":"2023-08-27T11:30:07.375823+0000","last_unstale":"2023-08-27T11:30:07.377092+0000","last_undegraded":"2023-08-27T11:30:07.377092+0000","last_fullsized":"2023-08-27T11:30:07.377092+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:39.725351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.510303+0000","last_change":"2023-08-27T11:30:06.508106+0000","last_active":"2023-08-27T11:30:06.510303+0000","last_peered":"2023-08-27T11:30:06.510303+0000","last_clean":"2023-08-27T11:30:06.510303+0000","last_became_active":"2023-08-27T11:30:06.507721+0000","last_became_peered":"2023-08-27T11:30:06.507721+0000","last_unstale":"2023-08-27T11:30:06.510303+0000","last_undegraded":"2023-08-27T11:30:06.510303+0000","last_fullsized":"2023-08-27T11:30:06.510303+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:21:52.252880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375020+0000","last_change":"2023-08-27T11:30:07.372999+0000","last_active":"2023-08-27T11:30:07.375020+0000","last_peered":"2023-08-27T11:30:07.375020+0000","last_clean":"2023-08-27T11:30:07.375020+0000","last_became_active":"2023-08-27T11:30:07.372727+0000","last_became_peered":"2023-08-27T11:30:07.372727+0000","last_unstale":"2023-08-27T11:30:07.375020+0000","last_undegraded":"2023-08-27T11:30:07.375020+0000","last_fullsized":"2023-08-27T11:30:07.375020+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:53:39.155707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":351,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.362862+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:30:07.362862+0000","last_peered":"2023-08-27T11:30:07.362862+0000","last_clean":"2023-08-27T11:30:07.362862+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:30:07.362862+0000","last_undegraded":"2023-08-27T11:30:07.362862+0000","last_fullsized":"2023-08-27T11:30:07.362862+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:54:33.147243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.382079+0000","last_change":"2023-08-27T11:30:07.379099+0000","last_active":"2023-08-27T11:30:07.382079+0000","last_peered":"2023-08-27T11:30:07.382079+0000","last_clean":"2023-08-27T11:30:07.382079+0000","last_became_active":"2023-08-27T11:30:07.378802+0000","last_became_peered":"2023-08-27T11:30:07.378802+0000","last_unstale":"2023-08-27T11:30:07.382079+0000","last_undegraded":"2023-08-27T11:30:07.382079+0000","last_fullsized":"2023-08-27T11:30:07.382079+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:26.058597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":351,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.364140+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:30:07.364140+0000","last_peered":"2023-08-27T11:30:07.364140+0000","last_clean":"2023-08-27T11:30:07.364140+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:30:07.364140+0000","last_undegraded":"2023-08-27T11:30:07.364140+0000","last_fullsized":"2023-08-27T11:30:07.364140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:59:17.376710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.375699+0000","last_change":"2023-08-27T11:30:07.372568+0000","last_active":"2023-08-27T11:30:07.375699+0000","last_peered":"2023-08-27T11:30:07.375699+0000","last_clean":"2023-08-27T11:30:07.375699+0000","last_became_active":"2023-08-27T11:30:07.372289+0000","last_became_peered":"2023-08-27T11:30:07.372289+0000","last_unstale":"2023-08-27T11:30:07.375699+0000","last_undegraded":"2023-08-27T11:30:07.375699+0000","last_fullsized":"2023-08-27T11:30:07.375699+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:10:25.361282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":299,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372055+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:30:07.372055+0000","last_peered":"2023-08-27T11:30:07.372055+0000","last_clean":"2023-08-27T11:30:07.372055+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:30:07.372055+0000","last_undegraded":"2023-08-27T11:30:07.372055+0000","last_fullsized":"2023-08-27T11:30:07.372055+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:32:49.299155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.373549+0000","last_change":"2023-08-27T11:30:07.366404+0000","last_active":"2023-08-27T11:30:07.373549+0000","last_peered":"2023-08-27T11:30:07.373549+0000","last_clean":"2023-08-27T11:30:07.373549+0000","last_became_active":"2023-08-27T11:30:07.366051+0000","last_became_peered":"2023-08-27T11:30:07.366051+0000","last_unstale":"2023-08-27T11:30:07.373549+0000","last_undegraded":"2023-08-27T11:30:07.373549+0000","last_fullsized":"2023-08-27T11:30:07.373549+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:53:30.711567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":297,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372131+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:30:07.372131+0000","last_peered":"2023-08-27T11:30:07.372131+0000","last_clean":"2023-08-27T11:30:07.372131+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:30:07.372131+0000","last_undegraded":"2023-08-27T11:30:07.372131+0000","last_fullsized":"2023-08-27T11:30:07.372131+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:36:26.335579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.370464+0000","last_change":"2023-08-27T11:30:07.367607+0000","last_active":"2023-08-27T11:30:07.370464+0000","last_peered":"2023-08-27T11:30:07.370464+0000","last_clean":"2023-08-27T11:30:07.370464+0000","last_became_active":"2023-08-27T11:30:07.367368+0000","last_became_peered":"2023-08-27T11:30:07.367368+0000","last_unstale":"2023-08-27T11:30:07.370464+0000","last_undegraded":"2023-08-27T11:30:07.370464+0000","last_fullsized":"2023-08-27T11:30:07.370464+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:39.373129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":297,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372428+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:30:07.372428+0000","last_peered":"2023-08-27T11:30:07.372428+0000","last_clean":"2023-08-27T11:30:07.372428+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:30:07.372428+0000","last_undegraded":"2023-08-27T11:30:07.372428+0000","last_fullsized":"2023-08-27T11:30:07.372428+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:55:02.343378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2023-08-27T11:30:06.496946+0000","last_change":"2023-08-27T11:30:06.496946+0000","last_active":"2023-08-27T11:30:06.487864+0000","last_peered":"2023-08-27T11:30:06.487864+0000","last_clean":"2023-08-27T11:30:06.487864+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:30:06.496946+0000","last_undegraded":"2023-08-27T11:30:06.496946+0000","last_fullsized":"2023-08-27T11:30:06.496946+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":447,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.372029+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:30:07.372029+0000","last_peered":"2023-08-27T11:30:07.372029+0000","last_clean":"2023-08-27T11:30:07.372029+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:30:07.372029+0000","last_undegraded":"2023-08-27T11:30:07.372029+0000","last_fullsized":"2023-08-27T11:30:07.372029+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:04.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":305,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.374061+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:30:07.374061+0000","last_peered":"2023-08-27T11:30:07.374061+0000","last_clean":"2023-08-27T11:30:07.374061+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:30:07.374061+0000","last_undegraded":"2023-08-27T11:30:07.374061+0000","last_fullsized":"2023-08-27T11:30:07.374061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:46:35.266946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.376873+0000","last_change":"2023-08-27T11:30:07.376873+0000","last_active":"2023-08-27T11:30:07.376873+0000","last_peered":"2023-08-27T11:30:07.376873+0000","last_clean":"2023-08-27T11:30:07.376873+0000","last_became_active":"2023-08-27T11:30:07.376104+0000","last_became_peered":"2023-08-27T11:30:07.376104+0000","last_unstale":"2023-08-27T11:30:07.376873+0000","last_undegraded":"2023-08-27T11:30:07.376873+0000","last_fullsized":"2023-08-27T11:30:07.376873+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:48:39.739503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":295,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-27T11:30:06.497876+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:30:06.497876+0000","last_peered":"2023-08-27T11:30:06.497876+0000","last_clean":"2023-08-27T11:30:06.497876+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:30:06.497876+0000","last_undegraded":"2023-08-27T11:30:06.497876+0000","last_fullsized":"2023-08-27T11:30:06.497876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:17.897409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.380860+0000","last_change":"2023-08-27T11:30:07.375200+0000","last_active":"2023-08-27T11:30:07.380860+0000","last_peered":"2023-08-27T11:30:07.380860+0000","last_clean":"2023-08-27T11:30:07.380860+0000","last_became_active":"2023-08-27T11:30:07.374915+0000","last_became_peered":"2023-08-27T11:30:07.374915+0000","last_unstale":"2023-08-27T11:30:07.380860+0000","last_undegraded":"2023-08-27T11:30:07.380860+0000","last_fullsized":"2023-08-27T11:30:07.380860+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:32:43.401182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":335,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-08-27T11:30:07.366365+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:30:07.366365+0000","last_peered":"2023-08-27T11:30:07.366365+0000","last_clean":"2023-08-27T11:30:07.366365+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:30:07.366365+0000","last_undegraded":"2023-08-27T11:30:07.366365+0000","last_fullsized":"2023-08-27T11:30:07.366365+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:39:06.252428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34036,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337804,"statfs":{"total":96636764160,"available":96601911296,"internally_reserved":0,"allocated":2920448,"data_stored":1702685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.22700000000000001},"max":{"1min":0.60399999999999998,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.753},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.51500000000000001},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.68300000000000005}]},{"osd":1,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.83399999999999996,"15min":0.872},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.72399999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.42799999999999999,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.67800000000000005,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.623},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.90200000000000002},"last":0.66500000000000004}]},{"osd":3,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":0.996,"15min":0.996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.53000000000000003,"5min":0.42899999999999999,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.71099999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.503},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.72199999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":1.224,"15min":1.224},"last":0.55000000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79700000000000004,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.84799999999999998},"last":0.64100000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83799999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.73399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575011,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.84599999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.39800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.89800000000000002},"last":0.48799999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.53900000000000003},"min":{"1min":0.35599999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.81799999999999995,"15min":0.873},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55400000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.499,"15min":0.51000000000000001},"min":{"1min":0.40799999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":2.0830000000000002},"last":0.72599999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.83199999999999996,"15min":0.84499999999999997},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.34000000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.93799999999999994},"last":0.626}]},{"osd":4,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":0.71599999999999997,"5min":0.753,"15min":0.83099999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.77000000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75700000000000001,"5min":0.85499999999999998,"15min":0.875},"last":0.78200000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79200000000000004,"5min":1.365,"15min":1.365},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.34899999999999998},"max":{"1min":0.77700000000000002,"5min":0.82999999999999996,"15min":0.91400000000000003},"last":0.753}]}]},{"osd":1,"up_from":13,"seq":55834575010,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34020,"kb_used_data":2840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337820,"statfs":{"total":96636764160,"available":96601927680,"internally_reserved":0,"allocated":2908160,"data_stored":1695557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.315,"15min":0.47399999999999998},"min":{"1min":0.21199999999999999,"5min":0.19500000000000001,"15min":0.191},"max":{"1min":0.45300000000000001,"5min":0.53300000000000003,"15min":11.971},"last":0.254},{"interface":"front","average":{"1min":0.315,"5min":0.32700000000000001,"15min":0.64700000000000002},"min":{"1min":0.23000000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.44900000000000001,"5min":0.74399999999999999,"15min":23.045999999999999},"last":0.313}]},{"osd":2,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34000000000000002,"15min":0.502},"min":{"1min":0.28199999999999997,"5min":0.23300000000000001,"15min":0.221},"max":{"1min":0.42399999999999999,"5min":0.57699999999999996,"15min":11.928000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.33700000000000002},"min":{"1min":0.249,"5min":0.22800000000000001,"15min":0.224},"max":{"1min":0.46899999999999997,"5min":0.52200000000000002,"15min":0.58599999999999997},"last":0.29599999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38400000000000001,"15min":0.52800000000000002},"min":{"1min":0.309,"5min":0.27500000000000002,"15min":0.26300000000000001},"max":{"1min":0.47599999999999998,"5min":0.64100000000000001,"15min":10.99},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36799999999999999,"15min":0.68500000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.503,"5min":0.54700000000000004,"15min":22.893999999999998},"last":0.35799999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.33400000000000002},"max":{"1min":0.84999999999999998,"5min":0.90700000000000003,"15min":1.26},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58099999999999996,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.745,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.62}]},{"osd":5,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.88600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":23.016999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.81899999999999995,"5min":0.88700000000000001,"15min":1.296},"last":0.66500000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58199999999999996,"15min":0.72799999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.371},"max":{"1min":0.72999999999999998,"5min":0.84899999999999998,"15min":10.925000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60199999999999998,"15min":0.90400000000000003},"min":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88,"5min":0.88,"15min":22.952000000000002},"last":0.70999999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59299999999999997,"15min":0.89800000000000002},"min":{"1min":0.433,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":22.988},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":1.327},"last":0.56999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2310144,"data_stored":1103828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.215,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.497,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35999999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36199999999999999,"15min":0.34399999999999997},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47699999999999998,"5min":0.628,"15min":0.63200000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34799999999999998,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45800000000000002,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.34599999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35799999999999998,"15min":0.34599999999999997},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.53700000000000003,"15min":0.59599999999999997},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.376},"min":{"1min":0.30499999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.53000000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.315}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.34399999999999997},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.214},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.65700000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.603},"last":0.62}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79600000000000004,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.878,"15min":0.878},"last":0.60399999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83399999999999996,"5min":3.79,"15min":3.79},"last":0.63800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32100000000000001,"15min":0.318},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.19700000000000001},"max":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.629},"last":0.32300000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41699999999999998,"5min":0.50800000000000001,"15min":0.54400000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34699999999999998},"min":{"1min":0.26200000000000001,"5min":0.23000000000000001,"15min":0.22600000000000001},"max":{"1min":0.49299999999999999,"5min":0.56399999999999995,"15min":0.749},"last":0.36099999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.378},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":0.51800000000000002,"5min":0.57099999999999995,"15min":0.73699999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.32600000000000001,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.55100000000000005,"15min":0.69499999999999995},"last":0.39900000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.879},"last":0.45700000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36699999999999999},"max":{"1min":0.83799999999999997,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.52800000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.40000000000000002},"max":{"1min":0.78600000000000003,"5min":0.82699999999999996,"15min":0.83599999999999997},"last":0.47399999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.46600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.29999999999999999,"15min":0.30299999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.52500000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.307,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.223,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.40899999999999997,"5min":0.755,"15min":0.755},"last":0.36499999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33500000000000002,"15min":0.32300000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":0.40300000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.35299999999999998},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":0.48999999999999999,"5min":2.73,"15min":2.73},"last":0.34799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.372,"15min":0.36399999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.46400000000000002,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.375,"15min":0.37},"min":{"1min":0.27700000000000002,"5min":0.26700000000000002,"15min":0.26400000000000001},"max":{"1min":0.51000000000000001,"5min":1.179,"15min":1.179},"last":0.44900000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.28599999999999998},"max":{"1min":0.748,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.437},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.71799999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54100000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.57699999999999996},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.54900000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.373},"max":{"1min":0.84299999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.497,"5min":0.38,"15min":0.38},"max":{"1min":0.83199999999999996,"5min":0.86899999999999999,"15min":1.111},"last":0.52600000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":1.236},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.499}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68200000000000005,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.502},"min":{"1min":0.33900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57399999999999995}]},{"osd":1,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66100000000000003,"5min":0.746,"15min":0.76500000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.75,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.376}]},{"osd":2,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.40400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.55700000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.96999999999999997,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59899999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71499999999999997,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":1.865,"15min":1.865},"last":0.61799999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90600000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.624,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82799999999999996,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.58799999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.90100000000000002,"5min":2.714,"15min":2.714},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.64500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575010,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2314240,"data_stored":1105189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.376,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61099999999999999,"5min":0.78900000000000003,"15min":0.96899999999999997},"last":0.38},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.44700000000000001,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.437}]},{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67500000000000004,"5min":0.68400000000000005,"15min":1.075},"last":0.33500000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.373,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.0580000000000001},"last":0.35399999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":1.03},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.42199999999999999,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.77800000000000002,"15min":1.026},"last":0.39400000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.27000000000000002},"max":{"1min":0.63900000000000001,"5min":0.78700000000000003,"15min":0.86899999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.73399999999999999,"5min":0.89400000000000002,"15min":0.91400000000000003},"last":0.51500000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.31},"max":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.88200000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.873,"15min":0.94199999999999995},"last":0.46000000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.34799999999999998},"max":{"1min":0.753,"5min":0.85999999999999999,"15min":0.95899999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.84399999999999997,"15min":0.92000000000000004},"last":0.63400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:30:12.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.37 is not active+clean 2023-08-27T11:30:12.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.37', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2023-08-27T11:30:07.368157+0000', 'last_change': '2023-08-27T11:30:07.368157+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:07.368157+0000', 'last_undegraded': '2023-08-27T11:30:07.368157+0000', 'last_fullsized': '2023-08-27T11:30:07.368157+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-28T14:54:48.531209+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': ['0', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:30:12.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.33 is not active+clean 2023-08-27T11:30:12.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.33', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2023-08-27T11:30:07.363359+0000', 'last_change': '2023-08-27T11:30:07.363359+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:07.363359+0000', 'last_undegraded': '2023-08-27T11:30:07.363359+0000', 'last_fullsized': '2023-08-27T11:30:07.363359+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-28T17:05:25.871181+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:30:12.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2f is not active+clean 2023-08-27T11:30:12.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:30:06.496034+0000', 'last_change': '2023-08-27T11:30:06.496034+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:06.496034+0000', 'last_undegraded': '2023-08-27T11:30:06.496034+0000', 'last_fullsized': '2023-08-27T11:30:06.496034+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:30:12.378 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2b is not active+clean 2023-08-27T11:30:12.378 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2023-08-27T11:30:07.368242+0000', 'last_change': '2023-08-27T11:30:07.368242+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:07.368242+0000', 'last_undegraded': '2023-08-27T11:30:07.368242+0000', 'last_fullsized': '2023-08-27T11:30:07.368242+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-28T21:46:21.277652+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': ['0', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:30:12.378 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2a is not active+clean 2023-08-27T11:30:12.379 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2023-08-27T11:30:07.369379+0000', 'last_change': '2023-08-27T11:30:07.369379+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:07.369379+0000', 'last_undegraded': '2023-08-27T11:30:07.369379+0000', 'last_fullsized': '2023-08-27T11:30:07.369379+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-28T18:18:50.569923+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': ['0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:30:12.379 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.28 is not active+clean 2023-08-27T11:30:12.380 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:30:06.495396+0000', 'last_change': '2023-08-27T11:30:06.495396+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:06.495396+0000', 'last_undegraded': '2023-08-27T11:30:06.495396+0000', 'last_fullsized': '2023-08-27T11:30:06.495396+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:30:12.380 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1a is not active+clean 2023-08-27T11:30:12.380 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2023-08-27T11:30:07.368141+0000', 'last_change': '2023-08-27T11:30:07.368141+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:07.368141+0000', 'last_undegraded': '2023-08-27T11:30:07.368141+0000', 'last_fullsized': '2023-08-27T11:30:07.368141+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-28T20:28:48.644407+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': ['0', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:30:12.381 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1b is not active+clean 2023-08-27T11:30:12.381 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1b', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2023-08-27T11:30:07.377944+0000', 'last_change': '2023-08-27T11:30:07.372096+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:07.377944+0000', 'last_undegraded': '2023-08-27T11:30:07.377944+0000', 'last_fullsized': '2023-08-27T11:30:07.377944+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-28T17:49:33.070564+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': ['1', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:30:12.382 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.18 is not active+clean 2023-08-27T11:30:12.382 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.18', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2023-08-27T11:30:07.369922+0000', 'last_change': '2023-08-27T11:30:07.369922+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:07.369922+0000', 'last_undegraded': '2023-08-27T11:30:07.369922+0000', 'last_fullsized': '2023-08-27T11:30:07.369922+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-28T19:30:34.484460+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': ['1', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:30:12.382 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.17 is not active+clean 2023-08-27T11:30:12.383 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:30:06.495697+0000', 'last_change': '2023-08-27T11:30:06.495697+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:06.495697+0000', 'last_undegraded': '2023-08-27T11:30:06.495697+0000', 'last_fullsized': '2023-08-27T11:30:06.495697+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:30:12.383 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.15 is not active+clean 2023-08-27T11:30:12.383 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:30:06.494625+0000', 'last_change': '2023-08-27T11:30:06.494625+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:06.494625+0000', 'last_undegraded': '2023-08-27T11:30:06.494625+0000', 'last_fullsized': '2023-08-27T11:30:06.494625+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:30:12.384 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.12 is not active+clean 2023-08-27T11:30:12.384 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:30:06.496930+0000', 'last_change': '2023-08-27T11:30:06.496930+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:06.496930+0000', 'last_undegraded': '2023-08-27T11:30:06.496930+0000', 'last_fullsized': '2023-08-27T11:30:06.496930+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:30:12.385 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.13 is not active+clean 2023-08-27T11:30:12.385 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.13', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2023-08-27T11:30:07.371737+0000', 'last_change': '2023-08-27T11:30:07.371737+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:07.371737+0000', 'last_undegraded': '2023-08-27T11:30:07.371737+0000', 'last_fullsized': '2023-08-27T11:30:07.371737+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-28T21:39:16.696934+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': ['1', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:30:12.385 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.10 is not active+clean 2023-08-27T11:30:12.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:30:06.494948+0000', 'last_change': '2023-08-27T11:30:06.494948+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:06.494948+0000', 'last_undegraded': '2023-08-27T11:30:06.494948+0000', 'last_fullsized': '2023-08-27T11:30:06.494948+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:30:12.386 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.11 is not active+clean 2023-08-27T11:30:12.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:30:06.497037+0000', 'last_change': '2023-08-27T11:30:06.497037+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:06.497037+0000', 'last_undegraded': '2023-08-27T11:30:06.497037+0000', 'last_fullsized': '2023-08-27T11:30:06.497037+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:30:12.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.b is not active+clean 2023-08-27T11:30:12.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.b', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2023-08-27T11:30:07.376957+0000', 'last_change': '2023-08-27T11:30:07.371363+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:07.376957+0000', 'last_undegraded': '2023-08-27T11:30:07.376957+0000', 'last_fullsized': '2023-08-27T11:30:07.376957+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-28T14:49:27.497988+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': ['1', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:30:12.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.8 is not active+clean 2023-08-27T11:30:12.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:30:06.494479+0000', 'last_change': '2023-08-27T11:30:06.494479+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:06.494479+0000', 'last_undegraded': '2023-08-27T11:30:06.494479+0000', 'last_fullsized': '2023-08-27T11:30:06.494479+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:30:12.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.7 is not active+clean 2023-08-27T11:30:12.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:30:06.495279+0000', 'last_change': '2023-08-27T11:30:06.495279+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:06.495279+0000', 'last_undegraded': '2023-08-27T11:30:06.495279+0000', 'last_fullsized': '2023-08-27T11:30:06.495279+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:30:12.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.23 is not active+clean 2023-08-27T11:30:12.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:30:06.496946+0000', 'last_change': '2023-08-27T11:30:06.496946+0000', 'last_active': '2023-08-27T11:30:06.487864+0000', 'last_peered': '2023-08-27T11:30:06.487864+0000', 'last_clean': '2023-08-27T11:30:06.487864+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:30:06.496946+0000', 'last_undegraded': '2023-08-27T11:30:06.496946+0000', 'last_fullsized': '2023-08-27T11:30:06.496946+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'last_clean_scrub_stamp': '2023-08-27T11:30:06.487864+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:30:12.391 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-27T11:30:12.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.527+0000 7f2ccfb61700 1 Processor -- start 2023-08-27T11:30:12.528 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.527+0000 7f2ccfb61700 1 -- start start 2023-08-27T11:30:12.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccfb61700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cc806e150 0x7f2cc80cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccfb61700 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2cc80cd270 0x7f2cc815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.531 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccfb61700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc813ff50 con 0x7f2cc813f990 2023-08-27T11:30:12.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccfb61700 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc80cecb0 con 0x7f2cc80cd270 2023-08-27T11:30:12.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccfb61700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc80cf570 con 0x7f2cc806e150 2023-08-27T11:30:12.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccd0fc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2cc813f990 0x7f2cc806e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33788/0 (socket says 172.21.15.112:33788) 2023-08-27T11:30:12.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccd0fc700 1 -- 172.21.15.112:0/498458705 learned_addr learned my addr 172.21.15.112:0/498458705 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:12.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccd8fd700 1 --2- 172.21.15.112:0/498458705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cc806e150 0x7f2cc80cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2cce0fe700 1 --2- 172.21.15.112:0/498458705 >> v2:172.21.15.138:3300/0 conn(0x7f2cc80cd270 0x7f2cc815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854177259 0 0) 0x7f2cc813ff50 con 0x7f2cc813f990 2023-08-27T11:30:12.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cb40035b0 con 0x7f2cc813f990 2023-08-27T11:30:12.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 165879783 0 0) 0x7f2cb40035b0 con 0x7f2cc813f990 2023-08-27T11:30:12.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cc806e150 msgr2=0x7f2cc80cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:12.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 --2- 172.21.15.112:0/498458705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cc806e150 0x7f2cc80cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 >> v2:172.21.15.138:3300/0 conn(0x7f2cc80cd270 msgr2=0x7f2cc815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:12.534 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 --2- 172.21.15.112:0/498458705 >> v2:172.21.15.138:3300/0 conn(0x7f2cc80cd270 0x7f2cc815ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc8140230 con 0x7f2cc813f990 2023-08-27T11:30:12.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 977097423 0 0) 0x7f2cb80017f0 con 0x7f2cc813f990 2023-08-27T11:30:12.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 >> v1:172.21.15.112:6789/0 conn(0x7f2cc813f990 legacy=0x7f2cc806e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 --2- 172.21.15.112:0/498458705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cc806e150 0x7f2cb4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 --2- 172.21.15.112:0/498458705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cb4003f80 0x7f2cb4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 --2- 172.21.15.112:0/498458705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb4004d90 0x7f2cb4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc80cf570 con 0x7f2cb4004d90 2023-08-27T11:30:12.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb400ba10 con 0x7f2cb4003f80 2023-08-27T11:30:12.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.528+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb400bb50 con 0x7f2cc806e150 2023-08-27T11:30:12.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.529+0000 7f2cce0fe700 1 --2- 172.21.15.112:0/498458705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb4004d90 0x7f2cb4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.529+0000 7f2ccd8fd700 1 --2- 172.21.15.112:0/498458705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cc806e150 0x7f2cb4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.529+0000 7f2ccd0fc700 1 --2- 172.21.15.112:0/498458705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cb4003f80 0x7f2cb4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.529+0000 7f2cce0fe700 1 -- 172.21.15.112:0/498458705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cc806e150 msgr2=0x7f2cb4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:12.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.529+0000 7f2cce0fe700 1 --2- 172.21.15.112:0/498458705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cc806e150 0x7f2cb4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.529+0000 7f2cce0fe700 1 -- 172.21.15.112:0/498458705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cb4003f80 msgr2=0x7f2cb4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:12.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.529+0000 7f2cce0fe700 1 --2- 172.21.15.112:0/498458705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cb4003f80 0x7f2cb4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.529+0000 7f2cce0fe700 1 -- 172.21.15.112:0/498458705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb40035b0 con 0x7f2cb4004d90 2023-08-27T11:30:12.538 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.529+0000 7f2cce0fe700 1 --2- 172.21.15.112:0/498458705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb4004d90 0x7f2cb4005210 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009fc0 tx=0x7f2cb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb12fefecbfcffcb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.531+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc000ebd0 con 0x7f2cb4004d90 2023-08-27T11:30:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.531+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc000f040 con 0x7f2cb4004d90 2023-08-27T11:30:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.531+0000 7f2ccc8fb700 1 -- 172.21.15.112:0/498458705 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0012430 con 0x7f2cb4004d90 2023-08-27T11:30:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.531+0000 7f2ccfb61700 1 -- 172.21.15.112:0/498458705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb4004d90 msgr2=0x7f2cb4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.531+0000 7f2ccfb61700 1 --2- 172.21.15.112:0/498458705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb4004d90 0x7f2cb4005210 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009fc0 tx=0x7f2cb4005880 comp rx=0 tx=0).stop 2023-08-27T11:30:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.531+0000 7f2cce0fe700 1 -- 172.21.15.112:0/498458705 reap_dead start 2023-08-27T11:30:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.531+0000 7f2ccfb61700 1 -- 172.21.15.112:0/498458705 shutdown_connections 2023-08-27T11:30:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.531+0000 7f2ccfb61700 1 -- 172.21.15.112:0/498458705 >> 172.21.15.112:0/498458705 conn(0x7f2cc806cd30 msgr2=0x7f2cc80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.531+0000 7f2ccfb61700 1 -- 172.21.15.112:0/498458705 shutdown_connections 2023-08-27T11:30:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.531+0000 7f2ccfb61700 1 -- 172.21.15.112:0/498458705 wait complete. 2023-08-27T11:30:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.533+0000 7f2ccfb61700 1 Processor -- start 2023-08-27T11:30:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.533+0000 7f2ccfb61700 1 -- start start 2023-08-27T11:30:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.533+0000 7f2ccfb61700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cc813bc80 0x7f2cc81519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.533+0000 7f2ccfb61700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cc813c060 0x7f2cc814fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.533+0000 7f2ccfb61700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cc81597f0 0x7f2cc8159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.533+0000 7f2ccfb61700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc815a1b0 con 0x7f2cc813c060 2023-08-27T11:30:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.533+0000 7f2ccfb61700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc8152ee0 con 0x7f2cc813bc80 2023-08-27T11:30:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.533+0000 7f2ccfb61700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc8152280 con 0x7f2cc81597f0 2023-08-27T11:30:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2ccd0fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cc813c060 0x7f2cc814fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2ccd0fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cc813c060 0x7f2cc814fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39420/0 (socket says 172.21.15.112:39420) 2023-08-27T11:30:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2ccd0fc700 1 -- 172.21.15.112:0/3943619224 learned_addr learned my addr 172.21.15.112:0/3943619224 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2ccd8fd700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cc813bc80 0x7f2cc81519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2cce0fe700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cc81597f0 0x7f2cc8159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2ccd0fc700 1 -- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cc81597f0 msgr2=0x7f2cc8159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2ccd0fc700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cc81597f0 0x7f2cc8159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2ccd0fc700 1 -- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cc813bc80 msgr2=0x7f2cc81519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2ccd0fc700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cc813bc80 0x7f2cc81519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.546 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2ccd0fc700 1 -- 172.21.15.112:0/3943619224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc81528d0 con 0x7f2cc813c060 2023-08-27T11:30:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2ccd0fc700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cc813c060 0x7f2cc814fe40 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8001130 tx=0x7f2cb8001890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2cbe7fc700 1 -- 172.21.15.112:0/3943619224 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb80072e0 con 0x7f2cc813c060 2023-08-27T11:30:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2cbe7fc700 1 -- 172.21.15.112:0/3943619224 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb8007440 con 0x7f2cc813c060 2023-08-27T11:30:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2ccfb61700 1 -- 172.21.15.112:0/3943619224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc8151fa0 con 0x7f2cc813c060 2023-08-27T11:30:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2cbe7fc700 1 -- 172.21.15.112:0/3943619224 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8005830 con 0x7f2cc813c060 2023-08-27T11:30:12.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.534+0000 7f2ccfb61700 1 -- 172.21.15.112:0/3943619224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc8150380 con 0x7f2cc813c060 2023-08-27T11:30:12.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.535+0000 7f2ccfb61700 1 -- 172.21.15.112:0/3943619224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc81590a0 con 0x7f2cc813c060 2023-08-27T11:30:12.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.536+0000 7f2cbe7fc700 1 -- 172.21.15.112:0/3943619224 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f2cb8005990 con 0x7f2cc813c060 2023-08-27T11:30:12.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.536+0000 7f2cbe7fc700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2cac06a4d0 0x7f2cac06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.548 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.536+0000 7f2cbe7fc700 1 -- 172.21.15.112:0/3943619224 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2cb8088ca0 con 0x7f2cc813c060 2023-08-27T11:30:12.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.536+0000 7f2ccd8fd700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2cac06a4d0 0x7f2cac06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.537+0000 7f2ccd8fd700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2cac06a4d0 0x7f2cac06c990 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2cc813f470 tx=0x7f2cc4009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:12.549 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.540+0000 7f2cbe7fc700 1 -- 172.21.15.112:0/3943619224 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2cb8053e40 con 0x7f2cc813c060 2023-08-27T11:30:12.807 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.806+0000 7f2ccfb61700 1 -- 172.21.15.112:0/3943619224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2cc8067a20 con 0x7f2cc813c060 2023-08-27T11:30:12.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.807+0000 7f2cbe7fc700 1 -- 172.21.15.112:0/3943619224 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1438 (secure 0 0 0) 0x7f2cb8057460 con 0x7f2cc813c060 2023-08-27T11:30:12.808 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:30:12.808 INFO:teuthology.orchestra.run.smithi112.stdout:{"fsid":"e099a24f-c1cd-4f11-ac4f-ecacc10e5e98","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":806,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":184,"num_osds":8,"num_up_osds":8,"osd_up_since":1693135017,"num_in_osds":8,"osd_in_since":1693135009,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":24,"data_bytes":592778,"bytes_used":275451904,"bytes_avail":772818661376,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":125,"id":15,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":15,"rank":0,"name":"d","status":"up:active","gid":9948}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2023-08-27T11:28:02.395383+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-08-27T11:30:12.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.808+0000 7f2ccfb61700 1 -- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2cac06a4d0 msgr2=0x7f2cac06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.812 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.808+0000 7f2ccfb61700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2cac06a4d0 0x7f2cac06c990 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2cc813f470 tx=0x7f2cc4009210 comp rx=0 tx=0).stop 2023-08-27T11:30:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.808+0000 7f2ccfb61700 1 -- 172.21.15.112:0/3943619224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cc813c060 msgr2=0x7f2cc814fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.808+0000 7f2ccfb61700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cc813c060 0x7f2cc814fe40 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8001130 tx=0x7f2cb8001890 comp rx=0 tx=0).stop 2023-08-27T11:30:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.809+0000 7f2ccfb61700 1 -- 172.21.15.112:0/3943619224 shutdown_connections 2023-08-27T11:30:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.809+0000 7f2ccfb61700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2cac06a4d0 0x7f2cac06c990 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.809+0000 7f2ccfb61700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cc81597f0 0x7f2cc8159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.809+0000 7f2ccfb61700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cc813bc80 0x7f2cc81519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.809+0000 7f2ccfb61700 1 --2- 172.21.15.112:0/3943619224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cc813c060 0x7f2cc814fe40 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.809+0000 7f2ccfb61700 1 -- 172.21.15.112:0/3943619224 >> 172.21.15.112:0/3943619224 conn(0x7f2cc806cd30 msgr2=0x7f2cc80a8480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:12.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.809+0000 7f2ccfb61700 1 -- 172.21.15.112:0/3943619224 shutdown_connections 2023-08-27T11:30:12.814 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.809+0000 7f2ccfb61700 1 -- 172.21.15.112:0/3943619224 wait complete. 2023-08-27T11:30:12.822 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-27T11:30:12.822 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:30:12.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.941+0000 7f2baad70700 1 Processor -- start 2023-08-27T11:30:12.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.941+0000 7f2baad70700 1 -- start start 2023-08-27T11:30:12.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2baad70700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2ba40b3e50 0x7f2ba415f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2baad70700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba40b45f0 con 0x7f2ba40b4ad0 2023-08-27T11:30:12.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2baad70700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba40b4310 con 0x7f2ba4144570 2023-08-27T11:30:12.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2baad70700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba40cf7a0 con 0x7f2ba40b3e50 2023-08-27T11:30:12.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2ba3fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2ba40b3e50 0x7f2ba415f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2ba930d700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2ba40b4ad0 0x7f2ba40b4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:33812/0 (socket says 172.21.15.112:33812) 2023-08-27T11:30:12.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2ba930d700 1 -- 172.21.15.112:0/508212104 learned_addr learned my addr 172.21.15.112:0/508212104 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-08-27T11:30:12.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2ba3fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2ba40b3e50 0x7f2ba415f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39426/0 (socket says 172.21.15.112:39426) 2023-08-27T11:30:12.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2ba3fff700 1 -- 172.21.15.112:0/508212104 >> v1:172.21.15.138:6789/0 conn(0x7f2ba4144570 legacy=0x7f2ba415ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:12.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2ba3fff700 1 -- 172.21.15.112:0/508212104 >> v1:172.21.15.112:6789/0 conn(0x7f2ba40b4ad0 legacy=0x7f2ba40b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2ba37fe700 1 -- 172.21.15.112:0/508212104 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725075357 0 0) 0x7f2ba40b45f0 con 0x7f2ba40b4ad0 2023-08-27T11:30:12.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2ba3fff700 1 -- 172.21.15.112:0/508212104 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba40cdcb0 con 0x7f2ba40b3e50 2023-08-27T11:30:12.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2ba3fff700 1 --2- 172.21.15.112:0/508212104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ba40b3e50 0x7f2ba415f530 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009840 tx=0x7f2b90009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f487a06e9650b867 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:12.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2ba37fe700 1 -- 172.21.15.112:0/508212104 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90004230 con 0x7f2ba40b3e50 2023-08-27T11:30:12.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2ba37fe700 1 -- 172.21.15.112:0/508212104 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b90004390 con 0x7f2ba40b3e50 2023-08-27T11:30:12.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.942+0000 7f2ba37fe700 1 -- 172.21.15.112:0/508212104 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b900036a0 con 0x7f2ba40b3e50 2023-08-27T11:30:12.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.943+0000 7f2baad70700 1 -- 172.21.15.112:0/508212104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ba40b3e50 msgr2=0x7f2ba415f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.943+0000 7f2baad70700 1 --2- 172.21.15.112:0/508212104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ba40b3e50 0x7f2ba415f530 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009840 tx=0x7f2b90009b50 comp rx=0 tx=0).stop 2023-08-27T11:30:12.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.943+0000 7f2baad70700 1 -- 172.21.15.112:0/508212104 shutdown_connections 2023-08-27T11:30:12.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.943+0000 7f2baad70700 1 --2- 172.21.15.112:0/508212104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ba40b3e50 0x7f2ba415f530 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.943+0000 7f2baad70700 1 -- 172.21.15.112:0/508212104 >> 172.21.15.112:0/508212104 conn(0x7f2ba406f970 msgr2=0x7f2ba406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:12.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.943+0000 7f2baad70700 1 -- 172.21.15.112:0/508212104 shutdown_connections 2023-08-27T11:30:12.949 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.943+0000 7f2baad70700 1 -- 172.21.15.112:0/508212104 wait complete. 2023-08-27T11:30:12.951 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.944+0000 7f2baad70700 1 Processor -- start 2023-08-27T11:30:12.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.944+0000 7f2baad70700 1 -- start start 2023-08-27T11:30:12.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2baad70700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ba40b3e50 0x7f2ba40cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2baad70700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ba40b4ad0 0x7f2ba414e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.952 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2baad70700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ba4144570 0x7f2ba414ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2baad70700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba4071630 con 0x7f2ba4144570 2023-08-27T11:30:12.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2baad70700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba4071910 con 0x7f2ba40b4ad0 2023-08-27T11:30:12.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2baad70700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba4071bf0 con 0x7f2ba40b3e50 2023-08-27T11:30:12.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2ba930d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ba4144570 0x7f2ba414ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.953 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2ba930d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ba4144570 0x7f2ba414ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39436/0 (socket says 172.21.15.112:39436) 2023-08-27T11:30:12.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2ba930d700 1 -- 172.21.15.112:0/1687681276 learned_addr learned my addr 172.21.15.112:0/1687681276 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:12.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2ba3fff700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ba40b4ad0 0x7f2ba414e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2ba8b0c700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ba40b3e50 0x7f2ba40cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2ba930d700 1 -- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ba40b3e50 msgr2=0x7f2ba40cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.954 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2ba930d700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ba40b3e50 0x7f2ba40cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2ba930d700 1 -- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ba40b4ad0 msgr2=0x7f2ba414e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2ba930d700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ba40b4ad0 0x7f2ba414e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.945+0000 7f2ba930d700 1 -- 172.21.15.112:0/1687681276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b900094a0 con 0x7f2ba4144570 2023-08-27T11:30:12.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.946+0000 7f2ba930d700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ba4144570 0x7f2ba414ea50 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00cca0 tx=0x7f2b9c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:12.955 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.946+0000 7f2ba17fa700 1 -- 172.21.15.112:0/1687681276 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c00ab70 con 0x7f2ba4144570 2023-08-27T11:30:12.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.946+0000 7f2ba17fa700 1 -- 172.21.15.112:0/1687681276 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9c00acd0 con 0x7f2ba4144570 2023-08-27T11:30:12.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.946+0000 7f2baad70700 1 -- 172.21.15.112:0/1687681276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba4071ed0 con 0x7f2ba4144570 2023-08-27T11:30:12.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.946+0000 7f2ba17fa700 1 -- 172.21.15.112:0/1687681276 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c007310 con 0x7f2ba4144570 2023-08-27T11:30:12.956 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.946+0000 7f2baad70700 1 -- 172.21.15.112:0/1687681276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba40cd570 con 0x7f2ba4144570 2023-08-27T11:30:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.947+0000 7f2baad70700 1 -- 172.21.15.112:0/1687681276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba40cd790 con 0x7f2ba4144570 2023-08-27T11:30:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.947+0000 7f2ba17fa700 1 -- 172.21.15.112:0/1687681276 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7f2b9c007470 con 0x7f2ba4144570 2023-08-27T11:30:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.947+0000 7f2ba17fa700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b8806a4d0 0x7f2b8806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.948+0000 7f2ba17fa700 1 -- 172.21.15.112:0/1687681276 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b9c088740 con 0x7f2ba4144570 2023-08-27T11:30:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.948+0000 7f2ba8b0c700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b8806a4d0 0x7f2b8806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.948+0000 7f2ba8b0c700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b8806a4d0 0x7f2b8806c990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009960 tx=0x7f2b980090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:12.958 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:12.952+0000 7f2ba17fa700 1 -- 172.21.15.112:0/1687681276 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2b9c0538e0 con 0x7f2ba4144570 2023-08-27T11:30:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.137+0000 7f2baad70700 1 -- 172.21.15.112:0/1687681276 --> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ba40cdf90 con 0x7f2b8806a4d0 2023-08-27T11:30:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.147+0000 7f2ba17fa700 1 -- 172.21.15.112:0/1687681276 <== mgr.4101 v2:172.21.15.138:6832/702326761 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312025 (secure 0 0 0) 0x7f2b98009e30 con 0x7f2b8806a4d0 2023-08-27T11:30:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-08-27T11:30:13.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.150+0000 7f2baad70700 1 -- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b8806a4d0 msgr2=0x7f2b8806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:13.154 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.150+0000 7f2baad70700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b8806a4d0 0x7f2b8806c990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009960 tx=0x7f2b980090e0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.150+0000 7f2baad70700 1 -- 172.21.15.112:0/1687681276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ba4144570 msgr2=0x7f2ba414ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:13.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.150+0000 7f2baad70700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ba4144570 0x7f2ba414ea50 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00cca0 tx=0x7f2b9c00a060 comp rx=0 tx=0).stop 2023-08-27T11:30:13.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.150+0000 7f2baad70700 1 -- 172.21.15.112:0/1687681276 shutdown_connections 2023-08-27T11:30:13.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.150+0000 7f2baad70700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7f2b8806a4d0 0x7f2b8806c990 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.155 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.150+0000 7f2baad70700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ba4144570 0x7f2ba414ea50 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.150+0000 7f2baad70700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ba40b3e50 0x7f2ba40cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.150+0000 7f2baad70700 1 --2- 172.21.15.112:0/1687681276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ba40b4ad0 0x7f2ba414e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.150+0000 7f2baad70700 1 -- 172.21.15.112:0/1687681276 >> 172.21.15.112:0/1687681276 conn(0x7f2ba406f970 msgr2=0x7f2ba406ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:13.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.150+0000 7f2baad70700 1 -- 172.21.15.112:0/1687681276 shutdown_connections 2023-08-27T11:30:13.156 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:13.150+0000 7f2baad70700 1 -- 172.21.15.112:0/1687681276 wait complete. 2023-08-27T11:30:13.156 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2023-08-27T11:30:13.164 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":575,"stamp":"2023-08-27T11:30:12.427690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268996,"kb_used_data":19492,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754705724,"statfs":{"total":773094113280,"available":772818661376,"internally_reserved":0,"allocated":19959808,"data_stored":10135955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001203"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365708+0000","last_change":"2023-08-27T11:30:07.373935+0000","last_active":"2023-08-27T11:30:08.365708+0000","last_peered":"2023-08-27T11:30:08.365708+0000","last_clean":"2023-08-27T11:30:08.365708+0000","last_became_active":"2023-08-27T11:30:07.373619+0000","last_became_peered":"2023-08-27T11:30:07.373619+0000","last_unstale":"2023-08-27T11:30:08.365708+0000","last_undegraded":"2023-08-27T11:30:08.365708+0000","last_fullsized":"2023-08-27T11:30:08.365708+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:15:55.853691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425092+0000","last_change":"2023-08-27T11:30:07.380950+0000","last_active":"2023-08-27T11:30:08.425092+0000","last_peered":"2023-08-27T11:30:08.425092+0000","last_clean":"2023-08-27T11:30:08.425092+0000","last_became_active":"2023-08-27T11:30:07.380668+0000","last_became_peered":"2023-08-27T11:30:07.380668+0000","last_unstale":"2023-08-27T11:30:08.425092+0000","last_undegraded":"2023-08-27T11:30:08.425092+0000","last_fullsized":"2023-08-27T11:30:08.425092+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:28.629882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425118+0000","last_change":"2023-08-27T11:30:07.383032+0000","last_active":"2023-08-27T11:30:08.425118+0000","last_peered":"2023-08-27T11:30:08.425118+0000","last_clean":"2023-08-27T11:30:08.425118+0000","last_became_active":"2023-08-27T11:30:07.382798+0000","last_became_peered":"2023-08-27T11:30:07.382798+0000","last_unstale":"2023-08-27T11:30:08.425118+0000","last_undegraded":"2023-08-27T11:30:08.425118+0000","last_fullsized":"2023-08-27T11:30:08.425118+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:12:12.344287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424716+0000","last_change":"2023-08-27T11:30:07.378918+0000","last_active":"2023-08-27T11:30:08.424716+0000","last_peered":"2023-08-27T11:30:08.424716+0000","last_clean":"2023-08-27T11:30:08.424716+0000","last_became_active":"2023-08-27T11:30:07.378583+0000","last_became_peered":"2023-08-27T11:30:07.378583+0000","last_unstale":"2023-08-27T11:30:08.424716+0000","last_undegraded":"2023-08-27T11:30:08.424716+0000","last_fullsized":"2023-08-27T11:30:08.424716+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:28:50.138349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424600+0000","last_change":"2023-08-27T11:30:07.375355+0000","last_active":"2023-08-27T11:30:08.424600+0000","last_peered":"2023-08-27T11:30:08.424600+0000","last_clean":"2023-08-27T11:30:08.424600+0000","last_became_active":"2023-08-27T11:30:07.375064+0000","last_became_peered":"2023-08-27T11:30:07.375064+0000","last_unstale":"2023-08-27T11:30:08.424600+0000","last_undegraded":"2023-08-27T11:30:08.424600+0000","last_fullsized":"2023-08-27T11:30:08.424600+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:32:25.097136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424683+0000","last_change":"2023-08-27T11:30:07.378756+0000","last_active":"2023-08-27T11:30:08.424683+0000","last_peered":"2023-08-27T11:30:08.424683+0000","last_clean":"2023-08-27T11:30:08.424683+0000","last_became_active":"2023-08-27T11:30:07.378495+0000","last_became_peered":"2023-08-27T11:30:07.378495+0000","last_unstale":"2023-08-27T11:30:08.424683+0000","last_undegraded":"2023-08-27T11:30:08.424683+0000","last_fullsized":"2023-08-27T11:30:08.424683+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:09:36.930041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365217+0000","last_change":"2023-08-27T11:30:07.377440+0000","last_active":"2023-08-27T11:30:08.365217+0000","last_peered":"2023-08-27T11:30:08.365217+0000","last_clean":"2023-08-27T11:30:08.365217+0000","last_became_active":"2023-08-27T11:30:07.376823+0000","last_became_peered":"2023-08-27T11:30:07.376823+0000","last_unstale":"2023-08-27T11:30:08.365217+0000","last_undegraded":"2023-08-27T11:30:08.365217+0000","last_fullsized":"2023-08-27T11:30:08.365217+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:59:30.066335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.366343+0000","last_change":"2023-08-27T11:30:07.377813+0000","last_active":"2023-08-27T11:30:08.366343+0000","last_peered":"2023-08-27T11:30:08.366343+0000","last_clean":"2023-08-27T11:30:08.366343+0000","last_became_active":"2023-08-27T11:30:07.377471+0000","last_became_peered":"2023-08-27T11:30:07.377471+0000","last_unstale":"2023-08-27T11:30:08.366343+0000","last_undegraded":"2023-08-27T11:30:08.366343+0000","last_fullsized":"2023-08-27T11:30:08.366343+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:39:35.099705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.426123+0000","last_change":"2023-08-27T11:30:07.380127+0000","last_active":"2023-08-27T11:30:08.426123+0000","last_peered":"2023-08-27T11:30:08.426123+0000","last_clean":"2023-08-27T11:30:08.426123+0000","last_became_active":"2023-08-27T11:30:07.379869+0000","last_became_peered":"2023-08-27T11:30:07.379869+0000","last_unstale":"2023-08-27T11:30:08.426123+0000","last_undegraded":"2023-08-27T11:30:08.426123+0000","last_fullsized":"2023-08-27T11:30:08.426123+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:41:38.670917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365642+0000","last_change":"2023-08-27T11:30:07.378494+0000","last_active":"2023-08-27T11:30:08.365642+0000","last_peered":"2023-08-27T11:30:08.365642+0000","last_clean":"2023-08-27T11:30:08.365642+0000","last_became_active":"2023-08-27T11:30:07.378196+0000","last_became_peered":"2023-08-27T11:30:07.378196+0000","last_unstale":"2023-08-27T11:30:08.365642+0000","last_undegraded":"2023-08-27T11:30:08.365642+0000","last_fullsized":"2023-08-27T11:30:08.365642+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:59:50.925310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365396+0000","last_change":"2023-08-27T11:30:07.382784+0000","last_active":"2023-08-27T11:30:08.365396+0000","last_peered":"2023-08-27T11:30:08.365396+0000","last_clean":"2023-08-27T11:30:08.365396+0000","last_became_active":"2023-08-27T11:30:07.382436+0000","last_became_peered":"2023-08-27T11:30:07.382436+0000","last_unstale":"2023-08-27T11:30:08.365396+0000","last_undegraded":"2023-08-27T11:30:08.365396+0000","last_fullsized":"2023-08-27T11:30:08.365396+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:40:28.471028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.366284+0000","last_change":"2023-08-27T11:30:07.372273+0000","last_active":"2023-08-27T11:30:08.366284+0000","last_peered":"2023-08-27T11:30:08.366284+0000","last_clean":"2023-08-27T11:30:08.366284+0000","last_became_active":"2023-08-27T11:30:07.371943+0000","last_became_peered":"2023-08-27T11:30:07.371943+0000","last_unstale":"2023-08-27T11:30:08.366284+0000","last_undegraded":"2023-08-27T11:30:08.366284+0000","last_fullsized":"2023-08-27T11:30:08.366284+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:01:05.196608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424834+0000","last_change":"2023-08-27T11:30:07.380648+0000","last_active":"2023-08-27T11:30:08.424834+0000","last_peered":"2023-08-27T11:30:08.424834+0000","last_clean":"2023-08-27T11:30:08.424834+0000","last_became_active":"2023-08-27T11:30:07.380371+0000","last_became_peered":"2023-08-27T11:30:07.380371+0000","last_unstale":"2023-08-27T11:30:08.424834+0000","last_undegraded":"2023-08-27T11:30:08.424834+0000","last_fullsized":"2023-08-27T11:30:08.424834+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:40:06.522629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424659+0000","last_change":"2023-08-27T11:30:07.367703+0000","last_active":"2023-08-27T11:30:08.424659+0000","last_peered":"2023-08-27T11:30:08.424659+0000","last_clean":"2023-08-27T11:30:08.424659+0000","last_became_active":"2023-08-27T11:30:07.367443+0000","last_became_peered":"2023-08-27T11:30:07.367443+0000","last_unstale":"2023-08-27T11:30:08.424659+0000","last_undegraded":"2023-08-27T11:30:08.424659+0000","last_fullsized":"2023-08-27T11:30:08.424659+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:45:54.074313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.426197+0000","last_change":"2023-08-27T11:30:07.376506+0000","last_active":"2023-08-27T11:30:08.426197+0000","last_peered":"2023-08-27T11:30:08.426197+0000","last_clean":"2023-08-27T11:30:08.426197+0000","last_became_active":"2023-08-27T11:30:07.376242+0000","last_became_peered":"2023-08-27T11:30:07.376242+0000","last_unstale":"2023-08-27T11:30:08.426197+0000","last_undegraded":"2023-08-27T11:30:08.426197+0000","last_fullsized":"2023-08-27T11:30:08.426197+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:14:59.862534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424342+0000","last_change":"2023-08-27T11:30:07.377343+0000","last_active":"2023-08-27T11:30:08.424342+0000","last_peered":"2023-08-27T11:30:08.424342+0000","last_clean":"2023-08-27T11:30:08.424342+0000","last_became_active":"2023-08-27T11:30:07.377032+0000","last_became_peered":"2023-08-27T11:30:07.377032+0000","last_unstale":"2023-08-27T11:30:08.424342+0000","last_undegraded":"2023-08-27T11:30:08.424342+0000","last_fullsized":"2023-08-27T11:30:08.424342+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:10:32.797709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.364185+0000","last_change":"2023-08-27T11:30:07.378507+0000","last_active":"2023-08-27T11:30:08.364185+0000","last_peered":"2023-08-27T11:30:08.364185+0000","last_clean":"2023-08-27T11:30:08.364185+0000","last_became_active":"2023-08-27T11:30:07.378238+0000","last_became_peered":"2023-08-27T11:30:07.378238+0000","last_unstale":"2023-08-27T11:30:08.364185+0000","last_undegraded":"2023-08-27T11:30:08.364185+0000","last_fullsized":"2023-08-27T11:30:08.364185+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:44:01.239503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365829+0000","last_change":"2023-08-27T11:30:07.376652+0000","last_active":"2023-08-27T11:30:08.365829+0000","last_peered":"2023-08-27T11:30:08.365829+0000","last_clean":"2023-08-27T11:30:08.365829+0000","last_became_active":"2023-08-27T11:30:07.376381+0000","last_became_peered":"2023-08-27T11:30:07.376381+0000","last_unstale":"2023-08-27T11:30:08.365829+0000","last_undegraded":"2023-08-27T11:30:08.365829+0000","last_fullsized":"2023-08-27T11:30:08.365829+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:13:43.845755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365231+0000","last_change":"2023-08-27T11:30:07.377266+0000","last_active":"2023-08-27T11:30:08.365231+0000","last_peered":"2023-08-27T11:30:08.365231+0000","last_clean":"2023-08-27T11:30:08.365231+0000","last_became_active":"2023-08-27T11:30:07.376986+0000","last_became_peered":"2023-08-27T11:30:07.376986+0000","last_unstale":"2023-08-27T11:30:08.365231+0000","last_undegraded":"2023-08-27T11:30:08.365231+0000","last_fullsized":"2023-08-27T11:30:08.365231+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:48:33.006466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424971+0000","last_change":"2023-08-27T11:30:07.376491+0000","last_active":"2023-08-27T11:30:08.424971+0000","last_peered":"2023-08-27T11:30:08.424971+0000","last_clean":"2023-08-27T11:30:08.424971+0000","last_became_active":"2023-08-27T11:30:07.376247+0000","last_became_peered":"2023-08-27T11:30:07.376247+0000","last_unstale":"2023-08-27T11:30:08.424971+0000","last_undegraded":"2023-08-27T11:30:08.424971+0000","last_fullsized":"2023-08-27T11:30:08.424971+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:02:41.445601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425567+0000","last_change":"2023-08-27T11:30:07.379943+0000","last_active":"2023-08-27T11:30:08.425567+0000","last_peered":"2023-08-27T11:30:08.425567+0000","last_clean":"2023-08-27T11:30:08.425567+0000","last_became_active":"2023-08-27T11:30:07.379668+0000","last_became_peered":"2023-08-27T11:30:07.379668+0000","last_unstale":"2023-08-27T11:30:08.425567+0000","last_undegraded":"2023-08-27T11:30:08.425567+0000","last_fullsized":"2023-08-27T11:30:08.425567+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:43.082626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424937+0000","last_change":"2023-08-27T11:30:07.382316+0000","last_active":"2023-08-27T11:30:08.424937+0000","last_peered":"2023-08-27T11:30:08.424937+0000","last_clean":"2023-08-27T11:30:08.424937+0000","last_became_active":"2023-08-27T11:30:07.382025+0000","last_became_peered":"2023-08-27T11:30:07.382025+0000","last_unstale":"2023-08-27T11:30:08.424937+0000","last_undegraded":"2023-08-27T11:30:08.424937+0000","last_fullsized":"2023-08-27T11:30:08.424937+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:59:30.271420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365402+0000","last_change":"2023-08-27T11:30:07.377088+0000","last_active":"2023-08-27T11:30:08.365402+0000","last_peered":"2023-08-27T11:30:08.365402+0000","last_clean":"2023-08-27T11:30:08.365402+0000","last_became_active":"2023-08-27T11:30:07.376059+0000","last_became_peered":"2023-08-27T11:30:07.376059+0000","last_unstale":"2023-08-27T11:30:08.365402+0000","last_undegraded":"2023-08-27T11:30:08.365402+0000","last_fullsized":"2023-08-27T11:30:08.365402+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:41:53.616003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.363845+0000","last_change":"2023-08-27T11:30:07.374739+0000","last_active":"2023-08-27T11:30:08.363845+0000","last_peered":"2023-08-27T11:30:08.363845+0000","last_clean":"2023-08-27T11:30:08.363845+0000","last_became_active":"2023-08-27T11:30:07.374411+0000","last_became_peered":"2023-08-27T11:30:07.374411+0000","last_unstale":"2023-08-27T11:30:08.363845+0000","last_undegraded":"2023-08-27T11:30:08.363845+0000","last_fullsized":"2023-08-27T11:30:08.363845+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:49:14.394917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.366637+0000","last_change":"2023-08-27T11:30:07.370865+0000","last_active":"2023-08-27T11:30:08.366637+0000","last_peered":"2023-08-27T11:30:08.366637+0000","last_clean":"2023-08-27T11:30:08.366637+0000","last_became_active":"2023-08-27T11:30:07.370571+0000","last_became_peered":"2023-08-27T11:30:07.370571+0000","last_unstale":"2023-08-27T11:30:08.366637+0000","last_undegraded":"2023-08-27T11:30:08.366637+0000","last_fullsized":"2023-08-27T11:30:08.366637+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:36:02.646876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"183'4","reported_seq":17,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.364274+0000","last_change":"2023-08-27T11:30:06.511151+0000","last_active":"2023-08-27T11:30:08.364274+0000","last_peered":"2023-08-27T11:30:08.364274+0000","last_clean":"2023-08-27T11:30:08.364274+0000","last_became_active":"2023-08-27T11:30:06.510830+0000","last_became_peered":"2023-08-27T11:30:06.510830+0000","last_unstale":"2023-08-27T11:30:08.364274+0000","last_undegraded":"2023-08-27T11:30:08.364274+0000","last_fullsized":"2023-08-27T11:30:08.364274+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:31:25.210331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424763+0000","last_change":"2023-08-27T11:30:07.378015+0000","last_active":"2023-08-27T11:30:08.424763+0000","last_peered":"2023-08-27T11:30:08.424763+0000","last_clean":"2023-08-27T11:30:08.424763+0000","last_became_active":"2023-08-27T11:30:07.377749+0000","last_became_peered":"2023-08-27T11:30:07.377749+0000","last_unstale":"2023-08-27T11:30:08.424763+0000","last_undegraded":"2023-08-27T11:30:08.424763+0000","last_fullsized":"2023-08-27T11:30:08.424763+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:55:58.126815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"183'1","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425918+0000","last_change":"2023-08-27T11:30:06.506992+0000","last_active":"2023-08-27T11:30:08.425918+0000","last_peered":"2023-08-27T11:30:08.425918+0000","last_clean":"2023-08-27T11:30:08.425918+0000","last_became_active":"2023-08-27T11:30:06.506671+0000","last_became_peered":"2023-08-27T11:30:06.506671+0000","last_unstale":"2023-08-27T11:30:08.425918+0000","last_undegraded":"2023-08-27T11:30:08.425918+0000","last_fullsized":"2023-08-27T11:30:08.425918+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:19.135969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.366492+0000","last_change":"2023-08-27T11:30:07.383160+0000","last_active":"2023-08-27T11:30:08.366492+0000","last_peered":"2023-08-27T11:30:08.366492+0000","last_clean":"2023-08-27T11:30:08.366492+0000","last_became_active":"2023-08-27T11:30:07.382823+0000","last_became_peered":"2023-08-27T11:30:07.382823+0000","last_unstale":"2023-08-27T11:30:08.366492+0000","last_undegraded":"2023-08-27T11:30:08.366492+0000","last_fullsized":"2023-08-27T11:30:08.366492+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:15:08.785349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"183'2","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425705+0000","last_change":"2023-08-27T11:30:06.509202+0000","last_active":"2023-08-27T11:30:08.425705+0000","last_peered":"2023-08-27T11:30:08.425705+0000","last_clean":"2023-08-27T11:30:08.425705+0000","last_became_active":"2023-08-27T11:30:06.508960+0000","last_became_peered":"2023-08-27T11:30:06.508960+0000","last_unstale":"2023-08-27T11:30:08.425705+0000","last_undegraded":"2023-08-27T11:30:08.425705+0000","last_fullsized":"2023-08-27T11:30:08.425705+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:16:24.999328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365233+0000","last_change":"2023-08-27T11:30:07.373389+0000","last_active":"2023-08-27T11:30:08.365233+0000","last_peered":"2023-08-27T11:30:08.365233+0000","last_clean":"2023-08-27T11:30:08.365233+0000","last_became_active":"2023-08-27T11:30:07.373083+0000","last_became_peered":"2023-08-27T11:30:07.373083+0000","last_unstale":"2023-08-27T11:30:08.365233+0000","last_undegraded":"2023-08-27T11:30:08.365233+0000","last_fullsized":"2023-08-27T11:30:08.365233+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:57:19.587525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"183'2","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425576+0000","last_change":"2023-08-27T11:30:06.509017+0000","last_active":"2023-08-27T11:30:08.425576+0000","last_peered":"2023-08-27T11:30:08.425576+0000","last_clean":"2023-08-27T11:30:08.425576+0000","last_became_active":"2023-08-27T11:30:06.508619+0000","last_became_peered":"2023-08-27T11:30:06.508619+0000","last_unstale":"2023-08-27T11:30:08.425576+0000","last_undegraded":"2023-08-27T11:30:08.425576+0000","last_fullsized":"2023-08-27T11:30:08.425576+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:07:48.560390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425370+0000","last_change":"2023-08-27T11:30:07.379970+0000","last_active":"2023-08-27T11:30:08.425370+0000","last_peered":"2023-08-27T11:30:08.425370+0000","last_clean":"2023-08-27T11:30:08.425370+0000","last_became_active":"2023-08-27T11:30:07.379696+0000","last_became_peered":"2023-08-27T11:30:07.379696+0000","last_unstale":"2023-08-27T11:30:08.425370+0000","last_undegraded":"2023-08-27T11:30:08.425370+0000","last_fullsized":"2023-08-27T11:30:08.425370+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:02:14.535256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"183'1","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365955+0000","last_change":"2023-08-27T11:30:06.503177+0000","last_active":"2023-08-27T11:30:08.365955+0000","last_peered":"2023-08-27T11:30:08.365955+0000","last_clean":"2023-08-27T11:30:08.365955+0000","last_became_active":"2023-08-27T11:30:06.502915+0000","last_became_peered":"2023-08-27T11:30:06.502915+0000","last_unstale":"2023-08-27T11:30:08.365955+0000","last_undegraded":"2023-08-27T11:30:08.365955+0000","last_fullsized":"2023-08-27T11:30:08.365955+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:51:32.411141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424568+0000","last_change":"2023-08-27T11:30:07.381589+0000","last_active":"2023-08-27T11:30:08.424568+0000","last_peered":"2023-08-27T11:30:08.424568+0000","last_clean":"2023-08-27T11:30:08.424568+0000","last_became_active":"2023-08-27T11:30:07.381357+0000","last_became_peered":"2023-08-27T11:30:07.381357+0000","last_unstale":"2023-08-27T11:30:08.424568+0000","last_undegraded":"2023-08-27T11:30:08.424568+0000","last_fullsized":"2023-08-27T11:30:08.424568+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:20:12.326185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"183'1","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.364208+0000","last_change":"2023-08-27T11:30:06.503408+0000","last_active":"2023-08-27T11:30:08.364208+0000","last_peered":"2023-08-27T11:30:08.364208+0000","last_clean":"2023-08-27T11:30:08.364208+0000","last_became_active":"2023-08-27T11:30:06.503111+0000","last_became_peered":"2023-08-27T11:30:06.503111+0000","last_unstale":"2023-08-27T11:30:08.364208+0000","last_undegraded":"2023-08-27T11:30:08.364208+0000","last_fullsized":"2023-08-27T11:30:08.364208+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:06:56.765817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425096+0000","last_change":"2023-08-27T11:30:07.379712+0000","last_active":"2023-08-27T11:30:08.425096+0000","last_peered":"2023-08-27T11:30:08.425096+0000","last_clean":"2023-08-27T11:30:08.425096+0000","last_became_active":"2023-08-27T11:30:07.379419+0000","last_became_peered":"2023-08-27T11:30:07.379419+0000","last_unstale":"2023-08-27T11:30:08.425096+0000","last_undegraded":"2023-08-27T11:30:08.425096+0000","last_fullsized":"2023-08-27T11:30:08.425096+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:03.053303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424573+0000","last_change":"2023-08-27T11:30:06.506988+0000","last_active":"2023-08-27T11:30:08.424573+0000","last_peered":"2023-08-27T11:30:08.424573+0000","last_clean":"2023-08-27T11:30:08.424573+0000","last_became_active":"2023-08-27T11:30:06.506687+0000","last_became_peered":"2023-08-27T11:30:06.506687+0000","last_unstale":"2023-08-27T11:30:08.424573+0000","last_undegraded":"2023-08-27T11:30:08.424573+0000","last_fullsized":"2023-08-27T11:30:08.424573+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:42:59.767368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365550+0000","last_change":"2023-08-27T11:30:07.377249+0000","last_active":"2023-08-27T11:30:08.365550+0000","last_peered":"2023-08-27T11:30:08.365550+0000","last_clean":"2023-08-27T11:30:08.365550+0000","last_became_active":"2023-08-27T11:30:07.376448+0000","last_became_peered":"2023-08-27T11:30:07.376448+0000","last_unstale":"2023-08-27T11:30:08.365550+0000","last_undegraded":"2023-08-27T11:30:08.365550+0000","last_fullsized":"2023-08-27T11:30:08.365550+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:41.937707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365901+0000","last_change":"2023-08-27T11:30:06.507787+0000","last_active":"2023-08-27T11:30:08.365901+0000","last_peered":"2023-08-27T11:30:08.365901+0000","last_clean":"2023-08-27T11:30:08.365901+0000","last_became_active":"2023-08-27T11:30:06.507390+0000","last_became_peered":"2023-08-27T11:30:06.507390+0000","last_unstale":"2023-08-27T11:30:08.365901+0000","last_undegraded":"2023-08-27T11:30:08.365901+0000","last_fullsized":"2023-08-27T11:30:08.365901+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:27:30.543933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.366071+0000","last_change":"2023-08-27T11:30:07.376530+0000","last_active":"2023-08-27T11:30:08.366071+0000","last_peered":"2023-08-27T11:30:08.366071+0000","last_clean":"2023-08-27T11:30:08.366071+0000","last_became_active":"2023-08-27T11:30:07.376221+0000","last_became_peered":"2023-08-27T11:30:07.376221+0000","last_unstale":"2023-08-27T11:30:08.366071+0000","last_undegraded":"2023-08-27T11:30:08.366071+0000","last_fullsized":"2023-08-27T11:30:08.366071+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:07:27.091236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.364377+0000","last_change":"2023-08-27T11:30:06.508059+0000","last_active":"2023-08-27T11:30:08.364377+0000","last_peered":"2023-08-27T11:30:08.364377+0000","last_clean":"2023-08-27T11:30:08.364377+0000","last_became_active":"2023-08-27T11:30:06.507677+0000","last_became_peered":"2023-08-27T11:30:06.507677+0000","last_unstale":"2023-08-27T11:30:08.364377+0000","last_undegraded":"2023-08-27T11:30:08.364377+0000","last_fullsized":"2023-08-27T11:30:08.364377+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:41:54.151347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.364182+0000","last_change":"2023-08-27T11:30:07.377975+0000","last_active":"2023-08-27T11:30:08.364182+0000","last_peered":"2023-08-27T11:30:08.364182+0000","last_clean":"2023-08-27T11:30:08.364182+0000","last_became_active":"2023-08-27T11:30:07.377527+0000","last_became_peered":"2023-08-27T11:30:07.377527+0000","last_unstale":"2023-08-27T11:30:08.364182+0000","last_undegraded":"2023-08-27T11:30:08.364182+0000","last_fullsized":"2023-08-27T11:30:08.364182+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:23:07.356838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425661+0000","last_change":"2023-08-27T11:30:06.509700+0000","last_active":"2023-08-27T11:30:08.425661+0000","last_peered":"2023-08-27T11:30:08.425661+0000","last_clean":"2023-08-27T11:30:08.425661+0000","last_became_active":"2023-08-27T11:30:06.509455+0000","last_became_peered":"2023-08-27T11:30:06.509455+0000","last_unstale":"2023-08-27T11:30:08.425661+0000","last_undegraded":"2023-08-27T11:30:08.425661+0000","last_fullsized":"2023-08-27T11:30:08.425661+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:55:56.798351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425258+0000","last_change":"2023-08-27T11:30:07.373333+0000","last_active":"2023-08-27T11:30:08.425258+0000","last_peered":"2023-08-27T11:30:08.425258+0000","last_clean":"2023-08-27T11:30:08.425258+0000","last_became_active":"2023-08-27T11:30:07.373009+0000","last_became_peered":"2023-08-27T11:30:07.373009+0000","last_unstale":"2023-08-27T11:30:08.425258+0000","last_undegraded":"2023-08-27T11:30:08.425258+0000","last_fullsized":"2023-08-27T11:30:08.425258+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:19:17.668035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"183'3","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425819+0000","last_change":"2023-08-27T11:30:06.509723+0000","last_active":"2023-08-27T11:30:08.425819+0000","last_peered":"2023-08-27T11:30:08.425819+0000","last_clean":"2023-08-27T11:30:08.425819+0000","last_became_active":"2023-08-27T11:30:06.509418+0000","last_became_peered":"2023-08-27T11:30:06.509418+0000","last_unstale":"2023-08-27T11:30:08.425819+0000","last_undegraded":"2023-08-27T11:30:08.425819+0000","last_fullsized":"2023-08-27T11:30:08.425819+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:56:24.794927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.364014+0000","last_change":"2023-08-27T11:30:07.382878+0000","last_active":"2023-08-27T11:30:08.364014+0000","last_peered":"2023-08-27T11:30:08.364014+0000","last_clean":"2023-08-27T11:30:08.364014+0000","last_became_active":"2023-08-27T11:30:07.382596+0000","last_became_peered":"2023-08-27T11:30:07.382596+0000","last_unstale":"2023-08-27T11:30:08.364014+0000","last_undegraded":"2023-08-27T11:30:08.364014+0000","last_fullsized":"2023-08-27T11:30:08.364014+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:59.609565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"183'6","reported_seq":19,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.363916+0000","last_change":"2023-08-27T11:30:06.503031+0000","last_active":"2023-08-27T11:30:08.363916+0000","last_peered":"2023-08-27T11:30:08.363916+0000","last_clean":"2023-08-27T11:30:08.363916+0000","last_became_active":"2023-08-27T11:30:06.502780+0000","last_became_peered":"2023-08-27T11:30:06.502780+0000","last_unstale":"2023-08-27T11:30:08.363916+0000","last_undegraded":"2023-08-27T11:30:08.363916+0000","last_fullsized":"2023-08-27T11:30:08.363916+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:41:11.834261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.364159+0000","last_change":"2023-08-27T11:30:07.377846+0000","last_active":"2023-08-27T11:30:08.364159+0000","last_peered":"2023-08-27T11:30:08.364159+0000","last_clean":"2023-08-27T11:30:08.364159+0000","last_became_active":"2023-08-27T11:30:07.377425+0000","last_became_peered":"2023-08-27T11:30:07.377425+0000","last_unstale":"2023-08-27T11:30:08.364159+0000","last_undegraded":"2023-08-27T11:30:08.364159+0000","last_fullsized":"2023-08-27T11:30:08.364159+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:38.178500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"183'2","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.366223+0000","last_change":"2023-08-27T11:30:06.511539+0000","last_active":"2023-08-27T11:30:08.366223+0000","last_peered":"2023-08-27T11:30:08.366223+0000","last_clean":"2023-08-27T11:30:08.366223+0000","last_became_active":"2023-08-27T11:30:06.511200+0000","last_became_peered":"2023-08-27T11:30:06.511200+0000","last_unstale":"2023-08-27T11:30:08.366223+0000","last_undegraded":"2023-08-27T11:30:08.366223+0000","last_fullsized":"2023-08-27T11:30:08.366223+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:07:18.165171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424881+0000","last_change":"2023-08-27T11:30:07.381396+0000","last_active":"2023-08-27T11:30:08.424881+0000","last_peered":"2023-08-27T11:30:08.424881+0000","last_clean":"2023-08-27T11:30:08.424881+0000","last_became_active":"2023-08-27T11:30:07.381069+0000","last_became_peered":"2023-08-27T11:30:07.381069+0000","last_unstale":"2023-08-27T11:30:08.424881+0000","last_undegraded":"2023-08-27T11:30:08.424881+0000","last_fullsized":"2023-08-27T11:30:08.424881+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:11:17.241040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425133+0000","last_change":"2023-08-27T11:30:06.505422+0000","last_active":"2023-08-27T11:30:08.425133+0000","last_peered":"2023-08-27T11:30:08.425133+0000","last_clean":"2023-08-27T11:30:08.425133+0000","last_became_active":"2023-08-27T11:30:06.505120+0000","last_became_peered":"2023-08-27T11:30:06.505120+0000","last_unstale":"2023-08-27T11:30:08.425133+0000","last_undegraded":"2023-08-27T11:30:08.425133+0000","last_fullsized":"2023-08-27T11:30:08.425133+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:05:48.997387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.363992+0000","last_change":"2023-08-27T11:30:07.375008+0000","last_active":"2023-08-27T11:30:08.363992+0000","last_peered":"2023-08-27T11:30:08.363992+0000","last_clean":"2023-08-27T11:30:08.363992+0000","last_became_active":"2023-08-27T11:30:07.374443+0000","last_became_peered":"2023-08-27T11:30:07.374443+0000","last_unstale":"2023-08-27T11:30:08.363992+0000","last_undegraded":"2023-08-27T11:30:08.363992+0000","last_fullsized":"2023-08-27T11:30:08.363992+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:19:52.248669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424934+0000","last_change":"2023-08-27T11:30:06.506911+0000","last_active":"2023-08-27T11:30:08.424934+0000","last_peered":"2023-08-27T11:30:08.424934+0000","last_clean":"2023-08-27T11:30:08.424934+0000","last_became_active":"2023-08-27T11:30:06.506657+0000","last_became_peered":"2023-08-27T11:30:06.506657+0000","last_unstale":"2023-08-27T11:30:08.424934+0000","last_undegraded":"2023-08-27T11:30:08.424934+0000","last_fullsized":"2023-08-27T11:30:08.424934+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:29:21.716122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.364300+0000","last_change":"2023-08-27T11:30:07.374211+0000","last_active":"2023-08-27T11:30:08.364300+0000","last_peered":"2023-08-27T11:30:08.364300+0000","last_clean":"2023-08-27T11:30:08.364300+0000","last_became_active":"2023-08-27T11:30:07.373726+0000","last_became_peered":"2023-08-27T11:30:07.373726+0000","last_unstale":"2023-08-27T11:30:08.364300+0000","last_undegraded":"2023-08-27T11:30:08.364300+0000","last_fullsized":"2023-08-27T11:30:08.364300+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:34:27.743609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"183'8","reported_seq":21,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425179+0000","last_change":"2023-08-27T11:30:06.516142+0000","last_active":"2023-08-27T11:30:08.425179+0000","last_peered":"2023-08-27T11:30:08.425179+0000","last_clean":"2023-08-27T11:30:08.425179+0000","last_became_active":"2023-08-27T11:30:06.515761+0000","last_became_peered":"2023-08-27T11:30:06.515761+0000","last_unstale":"2023-08-27T11:30:08.425179+0000","last_undegraded":"2023-08-27T11:30:08.425179+0000","last_fullsized":"2023-08-27T11:30:08.425179+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:42:19.730924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425675+0000","last_change":"2023-08-27T11:30:07.376818+0000","last_active":"2023-08-27T11:30:08.425675+0000","last_peered":"2023-08-27T11:30:08.425675+0000","last_clean":"2023-08-27T11:30:08.425675+0000","last_became_active":"2023-08-27T11:30:07.376575+0000","last_became_peered":"2023-08-27T11:30:07.376575+0000","last_unstale":"2023-08-27T11:30:08.425675+0000","last_undegraded":"2023-08-27T11:30:08.425675+0000","last_fullsized":"2023-08-27T11:30:08.425675+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:31:04.591271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"183'1","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365842+0000","last_change":"2023-08-27T11:30:06.504618+0000","last_active":"2023-08-27T11:30:08.365842+0000","last_peered":"2023-08-27T11:30:08.365842+0000","last_clean":"2023-08-27T11:30:08.365842+0000","last_became_active":"2023-08-27T11:30:06.504354+0000","last_became_peered":"2023-08-27T11:30:06.504354+0000","last_unstale":"2023-08-27T11:30:08.365842+0000","last_undegraded":"2023-08-27T11:30:08.365842+0000","last_fullsized":"2023-08-27T11:30:08.365842+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:00:24.118021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424872+0000","last_change":"2023-08-27T11:30:07.380702+0000","last_active":"2023-08-27T11:30:08.424872+0000","last_peered":"2023-08-27T11:30:08.424872+0000","last_clean":"2023-08-27T11:30:08.424872+0000","last_became_active":"2023-08-27T11:30:07.380378+0000","last_became_peered":"2023-08-27T11:30:07.380378+0000","last_unstale":"2023-08-27T11:30:08.424872+0000","last_undegraded":"2023-08-27T11:30:08.424872+0000","last_fullsized":"2023-08-27T11:30:08.424872+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:20:23.181055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"183'2","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425782+0000","last_change":"2023-08-27T11:30:06.511005+0000","last_active":"2023-08-27T11:30:08.425782+0000","last_peered":"2023-08-27T11:30:08.425782+0000","last_clean":"2023-08-27T11:30:08.425782+0000","last_became_active":"2023-08-27T11:30:06.510712+0000","last_became_peered":"2023-08-27T11:30:06.510712+0000","last_unstale":"2023-08-27T11:30:08.425782+0000","last_undegraded":"2023-08-27T11:30:08.425782+0000","last_fullsized":"2023-08-27T11:30:08.425782+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:30:59.153445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425371+0000","last_change":"2023-08-27T11:30:07.377430+0000","last_active":"2023-08-27T11:30:08.425371+0000","last_peered":"2023-08-27T11:30:08.425371+0000","last_clean":"2023-08-27T11:30:08.425371+0000","last_became_active":"2023-08-27T11:30:07.376459+0000","last_became_peered":"2023-08-27T11:30:07.376459+0000","last_unstale":"2023-08-27T11:30:08.425371+0000","last_undegraded":"2023-08-27T11:30:08.425371+0000","last_fullsized":"2023-08-27T11:30:08.425371+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:45:06.457076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"183'1","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424987+0000","last_change":"2023-08-27T11:30:06.510880+0000","last_active":"2023-08-27T11:30:08.424987+0000","last_peered":"2023-08-27T11:30:08.424987+0000","last_clean":"2023-08-27T11:30:08.424987+0000","last_became_active":"2023-08-27T11:30:06.510602+0000","last_became_peered":"2023-08-27T11:30:06.510602+0000","last_unstale":"2023-08-27T11:30:08.424987+0000","last_undegraded":"2023-08-27T11:30:08.424987+0000","last_fullsized":"2023-08-27T11:30:08.424987+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:10.436569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425410+0000","last_change":"2023-08-27T11:30:07.374685+0000","last_active":"2023-08-27T11:30:08.425410+0000","last_peered":"2023-08-27T11:30:08.425410+0000","last_clean":"2023-08-27T11:30:08.425410+0000","last_became_active":"2023-08-27T11:30:07.374131+0000","last_became_peered":"2023-08-27T11:30:07.374131+0000","last_unstale":"2023-08-27T11:30:08.425410+0000","last_undegraded":"2023-08-27T11:30:08.425410+0000","last_fullsized":"2023-08-27T11:30:08.425410+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:56:41.607457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425480+0000","last_change":"2023-08-27T11:30:06.515673+0000","last_active":"2023-08-27T11:30:08.425480+0000","last_peered":"2023-08-27T11:30:08.425480+0000","last_clean":"2023-08-27T11:30:08.425480+0000","last_became_active":"2023-08-27T11:30:06.515364+0000","last_became_peered":"2023-08-27T11:30:06.515364+0000","last_unstale":"2023-08-27T11:30:08.425480+0000","last_undegraded":"2023-08-27T11:30:08.425480+0000","last_fullsized":"2023-08-27T11:30:08.425480+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:59:46.468146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424682+0000","last_change":"2023-08-27T11:30:07.378734+0000","last_active":"2023-08-27T11:30:08.424682+0000","last_peered":"2023-08-27T11:30:08.424682+0000","last_clean":"2023-08-27T11:30:08.424682+0000","last_became_active":"2023-08-27T11:30:07.378472+0000","last_became_peered":"2023-08-27T11:30:07.378472+0000","last_unstale":"2023-08-27T11:30:08.424682+0000","last_undegraded":"2023-08-27T11:30:08.424682+0000","last_fullsized":"2023-08-27T11:30:08.424682+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:46.396300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"183'2","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424678+0000","last_change":"2023-08-27T11:30:06.504079+0000","last_active":"2023-08-27T11:30:08.424678+0000","last_peered":"2023-08-27T11:30:08.424678+0000","last_clean":"2023-08-27T11:30:08.424678+0000","last_became_active":"2023-08-27T11:30:06.503811+0000","last_became_peered":"2023-08-27T11:30:06.503811+0000","last_unstale":"2023-08-27T11:30:08.424678+0000","last_undegraded":"2023-08-27T11:30:08.424678+0000","last_fullsized":"2023-08-27T11:30:08.424678+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:49.975301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425005+0000","last_change":"2023-08-27T11:30:07.381559+0000","last_active":"2023-08-27T11:30:08.425005+0000","last_peered":"2023-08-27T11:30:08.425005+0000","last_clean":"2023-08-27T11:30:08.425005+0000","last_became_active":"2023-08-27T11:30:07.381031+0000","last_became_peered":"2023-08-27T11:30:07.381031+0000","last_unstale":"2023-08-27T11:30:08.425005+0000","last_undegraded":"2023-08-27T11:30:08.425005+0000","last_fullsized":"2023-08-27T11:30:08.425005+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:51.108980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365450+0000","last_change":"2023-08-27T11:30:06.503387+0000","last_active":"2023-08-27T11:30:08.365450+0000","last_peered":"2023-08-27T11:30:08.365450+0000","last_clean":"2023-08-27T11:30:08.365450+0000","last_became_active":"2023-08-27T11:30:06.503081+0000","last_became_peered":"2023-08-27T11:30:06.503081+0000","last_unstale":"2023-08-27T11:30:08.365450+0000","last_undegraded":"2023-08-27T11:30:08.365450+0000","last_fullsized":"2023-08-27T11:30:08.365450+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:48:02.339257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.363883+0000","last_change":"2023-08-27T11:30:07.374775+0000","last_active":"2023-08-27T11:30:08.363883+0000","last_peered":"2023-08-27T11:30:08.363883+0000","last_clean":"2023-08-27T11:30:08.363883+0000","last_became_active":"2023-08-27T11:30:07.374172+0000","last_became_peered":"2023-08-27T11:30:07.374172+0000","last_unstale":"2023-08-27T11:30:08.363883+0000","last_undegraded":"2023-08-27T11:30:08.363883+0000","last_fullsized":"2023-08-27T11:30:08.363883+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:51:49.138249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425170+0000","last_change":"2023-08-27T11:30:06.511539+0000","last_active":"2023-08-27T11:30:08.425170+0000","last_peered":"2023-08-27T11:30:08.425170+0000","last_clean":"2023-08-27T11:30:08.425170+0000","last_became_active":"2023-08-27T11:30:06.511248+0000","last_became_peered":"2023-08-27T11:30:06.511248+0000","last_unstale":"2023-08-27T11:30:08.425170+0000","last_undegraded":"2023-08-27T11:30:08.425170+0000","last_fullsized":"2023-08-27T11:30:08.425170+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:05:17.464366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424478+0000","last_change":"2023-08-27T11:30:07.373745+0000","last_active":"2023-08-27T11:30:08.424478+0000","last_peered":"2023-08-27T11:30:08.424478+0000","last_clean":"2023-08-27T11:30:08.424478+0000","last_became_active":"2023-08-27T11:30:07.373460+0000","last_became_peered":"2023-08-27T11:30:07.373460+0000","last_unstale":"2023-08-27T11:30:08.424478+0000","last_undegraded":"2023-08-27T11:30:08.424478+0000","last_fullsized":"2023-08-27T11:30:08.424478+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:45:13.577999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.366432+0000","last_change":"2023-08-27T11:30:06.502148+0000","last_active":"2023-08-27T11:30:08.366432+0000","last_peered":"2023-08-27T11:30:08.366432+0000","last_clean":"2023-08-27T11:30:08.366432+0000","last_became_active":"2023-08-27T11:30:06.501869+0000","last_became_peered":"2023-08-27T11:30:06.501869+0000","last_unstale":"2023-08-27T11:30:08.366432+0000","last_undegraded":"2023-08-27T11:30:08.366432+0000","last_fullsized":"2023-08-27T11:30:08.366432+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:02:20.956445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425798+0000","last_change":"2023-08-27T11:30:07.376696+0000","last_active":"2023-08-27T11:30:08.425798+0000","last_peered":"2023-08-27T11:30:08.425798+0000","last_clean":"2023-08-27T11:30:08.425798+0000","last_became_active":"2023-08-27T11:30:07.376266+0000","last_became_peered":"2023-08-27T11:30:07.376266+0000","last_unstale":"2023-08-27T11:30:08.425798+0000","last_undegraded":"2023-08-27T11:30:08.425798+0000","last_fullsized":"2023-08-27T11:30:08.425798+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:20:25.840373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"183'2","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424858+0000","last_change":"2023-08-27T11:30:06.504227+0000","last_active":"2023-08-27T11:30:08.424858+0000","last_peered":"2023-08-27T11:30:08.424858+0000","last_clean":"2023-08-27T11:30:08.424858+0000","last_became_active":"2023-08-27T11:30:06.503935+0000","last_became_peered":"2023-08-27T11:30:06.503935+0000","last_unstale":"2023-08-27T11:30:08.424858+0000","last_undegraded":"2023-08-27T11:30:08.424858+0000","last_fullsized":"2023-08-27T11:30:08.424858+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:59:09.438636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.364340+0000","last_change":"2023-08-27T11:30:07.377800+0000","last_active":"2023-08-27T11:30:08.364340+0000","last_peered":"2023-08-27T11:30:08.364340+0000","last_clean":"2023-08-27T11:30:08.364340+0000","last_became_active":"2023-08-27T11:30:07.377173+0000","last_became_peered":"2023-08-27T11:30:07.377173+0000","last_unstale":"2023-08-27T11:30:08.364340+0000","last_undegraded":"2023-08-27T11:30:08.364340+0000","last_fullsized":"2023-08-27T11:30:08.364340+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:37:38.754968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"184'1","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425146+0000","last_change":"2023-08-27T11:30:06.515077+0000","last_active":"2023-08-27T11:30:08.425146+0000","last_peered":"2023-08-27T11:30:08.425146+0000","last_clean":"2023-08-27T11:30:08.425146+0000","last_became_active":"2023-08-27T11:30:06.514812+0000","last_became_peered":"2023-08-27T11:30:06.514812+0000","last_unstale":"2023-08-27T11:30:08.425146+0000","last_undegraded":"2023-08-27T11:30:08.425146+0000","last_fullsized":"2023-08-27T11:30:08.425146+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:59:47.841918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424847+0000","last_change":"2023-08-27T11:30:07.375476+0000","last_active":"2023-08-27T11:30:08.424847+0000","last_peered":"2023-08-27T11:30:08.424847+0000","last_clean":"2023-08-27T11:30:08.424847+0000","last_became_active":"2023-08-27T11:30:07.375167+0000","last_became_peered":"2023-08-27T11:30:07.375167+0000","last_unstale":"2023-08-27T11:30:08.424847+0000","last_undegraded":"2023-08-27T11:30:08.424847+0000","last_fullsized":"2023-08-27T11:30:08.424847+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:02:43.732426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365425+0000","last_change":"2023-08-27T11:30:06.507172+0000","last_active":"2023-08-27T11:30:08.365425+0000","last_peered":"2023-08-27T11:30:08.365425+0000","last_clean":"2023-08-27T11:30:08.365425+0000","last_became_active":"2023-08-27T11:30:06.506885+0000","last_became_peered":"2023-08-27T11:30:06.506885+0000","last_unstale":"2023-08-27T11:30:08.365425+0000","last_undegraded":"2023-08-27T11:30:08.365425+0000","last_fullsized":"2023-08-27T11:30:08.365425+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:37.621762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425264+0000","last_change":"2023-08-27T11:30:07.383116+0000","last_active":"2023-08-27T11:30:08.425264+0000","last_peered":"2023-08-27T11:30:08.425264+0000","last_clean":"2023-08-27T11:30:08.425264+0000","last_became_active":"2023-08-27T11:30:07.382838+0000","last_became_peered":"2023-08-27T11:30:07.382838+0000","last_unstale":"2023-08-27T11:30:08.425264+0000","last_undegraded":"2023-08-27T11:30:08.425264+0000","last_fullsized":"2023-08-27T11:30:08.425264+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:57:34.968767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.364034+0000","last_change":"2023-08-27T11:30:06.508063+0000","last_active":"2023-08-27T11:30:08.364034+0000","last_peered":"2023-08-27T11:30:08.364034+0000","last_clean":"2023-08-27T11:30:08.364034+0000","last_became_active":"2023-08-27T11:30:06.507412+0000","last_became_peered":"2023-08-27T11:30:06.507412+0000","last_unstale":"2023-08-27T11:30:08.364034+0000","last_undegraded":"2023-08-27T11:30:08.364034+0000","last_fullsized":"2023-08-27T11:30:08.364034+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:44:06.964057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424905+0000","last_change":"2023-08-27T11:30:07.374655+0000","last_active":"2023-08-27T11:30:08.424905+0000","last_peered":"2023-08-27T11:30:08.424905+0000","last_clean":"2023-08-27T11:30:08.424905+0000","last_became_active":"2023-08-27T11:30:07.374306+0000","last_became_peered":"2023-08-27T11:30:07.374306+0000","last_unstale":"2023-08-27T11:30:08.424905+0000","last_undegraded":"2023-08-27T11:30:08.424905+0000","last_fullsized":"2023-08-27T11:30:08.424905+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:07:23.177143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425677+0000","last_change":"2023-08-27T11:30:06.500047+0000","last_active":"2023-08-27T11:30:08.425677+0000","last_peered":"2023-08-27T11:30:08.425677+0000","last_clean":"2023-08-27T11:30:08.425677+0000","last_became_active":"2023-08-27T11:30:06.499775+0000","last_became_peered":"2023-08-27T11:30:06.499775+0000","last_unstale":"2023-08-27T11:30:08.425677+0000","last_undegraded":"2023-08-27T11:30:08.425677+0000","last_fullsized":"2023-08-27T11:30:08.425677+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:16:01.820893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424951+0000","last_change":"2023-08-27T11:30:07.376417+0000","last_active":"2023-08-27T11:30:08.424951+0000","last_peered":"2023-08-27T11:30:08.424951+0000","last_clean":"2023-08-27T11:30:08.424951+0000","last_became_active":"2023-08-27T11:30:07.376144+0000","last_became_peered":"2023-08-27T11:30:07.376144+0000","last_unstale":"2023-08-27T11:30:08.424951+0000","last_undegraded":"2023-08-27T11:30:08.424951+0000","last_fullsized":"2023-08-27T11:30:08.424951+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:40.400619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425246+0000","last_change":"2023-08-27T11:30:06.505423+0000","last_active":"2023-08-27T11:30:08.425246+0000","last_peered":"2023-08-27T11:30:08.425246+0000","last_clean":"2023-08-27T11:30:08.425246+0000","last_became_active":"2023-08-27T11:30:06.505138+0000","last_became_peered":"2023-08-27T11:30:06.505138+0000","last_unstale":"2023-08-27T11:30:08.425246+0000","last_undegraded":"2023-08-27T11:30:08.425246+0000","last_fullsized":"2023-08-27T11:30:08.425246+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:37:18.305837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424488+0000","last_change":"2023-08-27T11:30:07.375665+0000","last_active":"2023-08-27T11:30:08.424488+0000","last_peered":"2023-08-27T11:30:08.424488+0000","last_clean":"2023-08-27T11:30:08.424488+0000","last_became_active":"2023-08-27T11:30:07.375184+0000","last_became_peered":"2023-08-27T11:30:07.375184+0000","last_unstale":"2023-08-27T11:30:08.424488+0000","last_undegraded":"2023-08-27T11:30:08.424488+0000","last_fullsized":"2023-08-27T11:30:08.424488+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:34.136155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.426297+0000","last_change":"2023-08-27T11:30:06.504990+0000","last_active":"2023-08-27T11:30:08.426297+0000","last_peered":"2023-08-27T11:30:08.426297+0000","last_clean":"2023-08-27T11:30:08.426297+0000","last_became_active":"2023-08-27T11:30:06.504768+0000","last_became_peered":"2023-08-27T11:30:06.504768+0000","last_unstale":"2023-08-27T11:30:08.426297+0000","last_undegraded":"2023-08-27T11:30:08.426297+0000","last_fullsized":"2023-08-27T11:30:08.426297+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:58:10.013135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424550+0000","last_change":"2023-08-27T11:30:07.376099+0000","last_active":"2023-08-27T11:30:08.424550+0000","last_peered":"2023-08-27T11:30:08.424550+0000","last_clean":"2023-08-27T11:30:08.424550+0000","last_became_active":"2023-08-27T11:30:07.375823+0000","last_became_peered":"2023-08-27T11:30:07.375823+0000","last_unstale":"2023-08-27T11:30:08.424550+0000","last_undegraded":"2023-08-27T11:30:08.424550+0000","last_fullsized":"2023-08-27T11:30:08.424550+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:37:54.123909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.364004+0000","last_change":"2023-08-27T11:30:06.508106+0000","last_active":"2023-08-27T11:30:08.364004+0000","last_peered":"2023-08-27T11:30:08.364004+0000","last_clean":"2023-08-27T11:30:08.364004+0000","last_became_active":"2023-08-27T11:30:06.507721+0000","last_became_peered":"2023-08-27T11:30:06.507721+0000","last_unstale":"2023-08-27T11:30:08.364004+0000","last_undegraded":"2023-08-27T11:30:08.364004+0000","last_fullsized":"2023-08-27T11:30:08.364004+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:05.482710+0000","last_clean_scrub_stamp":"2023-08-27T11:30:05.482710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:17:06.958682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424732+0000","last_change":"2023-08-27T11:30:07.372999+0000","last_active":"2023-08-27T11:30:08.424732+0000","last_peered":"2023-08-27T11:30:08.424732+0000","last_clean":"2023-08-27T11:30:08.424732+0000","last_became_active":"2023-08-27T11:30:07.372727+0000","last_became_peered":"2023-08-27T11:30:07.372727+0000","last_unstale":"2023-08-27T11:30:08.424732+0000","last_undegraded":"2023-08-27T11:30:08.424732+0000","last_fullsized":"2023-08-27T11:30:08.424732+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:59:50.783749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":353,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.366558+0000","last_change":"2023-08-27T11:16:59.324481+0000","last_active":"2023-08-27T11:30:08.366558+0000","last_peered":"2023-08-27T11:30:08.366558+0000","last_clean":"2023-08-27T11:30:08.366558+0000","last_became_active":"2023-08-27T11:16:59.324111+0000","last_became_peered":"2023-08-27T11:16:59.324111+0000","last_unstale":"2023-08-27T11:30:08.366558+0000","last_undegraded":"2023-08-27T11:30:08.366558+0000","last_fullsized":"2023-08-27T11:30:08.366558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:07:10.060782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425002+0000","last_change":"2023-08-27T11:30:07.379099+0000","last_active":"2023-08-27T11:30:08.425002+0000","last_peered":"2023-08-27T11:30:08.425002+0000","last_clean":"2023-08-27T11:30:08.425002+0000","last_became_active":"2023-08-27T11:30:07.378802+0000","last_became_peered":"2023-08-27T11:30:07.378802+0000","last_unstale":"2023-08-27T11:30:08.425002+0000","last_undegraded":"2023-08-27T11:30:08.425002+0000","last_fullsized":"2023-08-27T11:30:08.425002+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:33:03.662936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":353,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.366301+0000","last_change":"2023-08-27T11:17:00.246209+0000","last_active":"2023-08-27T11:30:08.366301+0000","last_peered":"2023-08-27T11:30:08.366301+0000","last_clean":"2023-08-27T11:30:08.366301+0000","last_became_active":"2023-08-27T11:17:00.245859+0000","last_became_peered":"2023-08-27T11:17:00.245859+0000","last_unstale":"2023-08-27T11:30:08.366301+0000","last_undegraded":"2023-08-27T11:30:08.366301+0000","last_fullsized":"2023-08-27T11:30:08.366301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:37:47.589842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.366526+0000","last_change":"2023-08-27T11:30:07.372568+0000","last_active":"2023-08-27T11:30:08.366526+0000","last_peered":"2023-08-27T11:30:08.366526+0000","last_clean":"2023-08-27T11:30:08.366526+0000","last_became_active":"2023-08-27T11:30:07.372289+0000","last_became_peered":"2023-08-27T11:30:07.372289+0000","last_unstale":"2023-08-27T11:30:08.366526+0000","last_undegraded":"2023-08-27T11:30:08.366526+0000","last_fullsized":"2023-08-27T11:30:08.366526+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:27:52.609579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":301,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425257+0000","last_change":"2023-08-27T11:17:00.246421+0000","last_active":"2023-08-27T11:30:08.425257+0000","last_peered":"2023-08-27T11:30:08.425257+0000","last_clean":"2023-08-27T11:30:08.425257+0000","last_became_active":"2023-08-27T11:17:00.245871+0000","last_became_peered":"2023-08-27T11:17:00.245871+0000","last_unstale":"2023-08-27T11:30:08.425257+0000","last_undegraded":"2023-08-27T11:30:08.425257+0000","last_fullsized":"2023-08-27T11:30:08.425257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:10.953704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365751+0000","last_change":"2023-08-27T11:30:07.366404+0000","last_active":"2023-08-27T11:30:08.365751+0000","last_peered":"2023-08-27T11:30:08.365751+0000","last_clean":"2023-08-27T11:30:08.365751+0000","last_became_active":"2023-08-27T11:30:07.366051+0000","last_became_peered":"2023-08-27T11:30:07.366051+0000","last_unstale":"2023-08-27T11:30:08.365751+0000","last_undegraded":"2023-08-27T11:30:08.365751+0000","last_fullsized":"2023-08-27T11:30:08.365751+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:46:32.465399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":299,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424735+0000","last_change":"2023-08-27T11:17:00.246652+0000","last_active":"2023-08-27T11:30:08.424735+0000","last_peered":"2023-08-27T11:30:08.424735+0000","last_clean":"2023-08-27T11:30:08.424735+0000","last_became_active":"2023-08-27T11:17:00.246300+0000","last_became_peered":"2023-08-27T11:17:00.246300+0000","last_unstale":"2023-08-27T11:30:08.424735+0000","last_undegraded":"2023-08-27T11:30:08.424735+0000","last_fullsized":"2023-08-27T11:30:08.424735+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:36:37.425474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424773+0000","last_change":"2023-08-27T11:30:07.367607+0000","last_active":"2023-08-27T11:30:08.424773+0000","last_peered":"2023-08-27T11:30:08.424773+0000","last_clean":"2023-08-27T11:30:08.424773+0000","last_became_active":"2023-08-27T11:30:07.367368+0000","last_became_peered":"2023-08-27T11:30:07.367368+0000","last_unstale":"2023-08-27T11:30:08.424773+0000","last_undegraded":"2023-08-27T11:30:08.424773+0000","last_fullsized":"2023-08-27T11:30:08.424773+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:37:16.543089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":299,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425147+0000","last_change":"2023-08-27T11:17:00.246364+0000","last_active":"2023-08-27T11:30:08.425147+0000","last_peered":"2023-08-27T11:30:08.425147+0000","last_clean":"2023-08-27T11:30:08.425147+0000","last_became_active":"2023-08-27T11:17:00.245906+0000","last_became_peered":"2023-08-27T11:17:00.245906+0000","last_unstale":"2023-08-27T11:30:08.425147+0000","last_undegraded":"2023-08-27T11:30:08.425147+0000","last_fullsized":"2023-08-27T11:30:08.425147+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:16:56.943889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.364267+0000","last_change":"2023-08-27T11:30:07.382609+0000","last_active":"2023-08-27T11:30:08.364267+0000","last_peered":"2023-08-27T11:30:08.364267+0000","last_clean":"2023-08-27T11:30:08.364267+0000","last_became_active":"2023-08-27T11:30:07.382139+0000","last_became_peered":"2023-08-27T11:30:07.382139+0000","last_unstale":"2023-08-27T11:30:08.364267+0000","last_undegraded":"2023-08-27T11:30:08.364267+0000","last_fullsized":"2023-08-27T11:30:08.364267+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:39:06.125320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":449,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.424588+0000","last_change":"2023-08-27T11:16:59.324564+0000","last_active":"2023-08-27T11:30:08.424588+0000","last_peered":"2023-08-27T11:30:08.424588+0000","last_clean":"2023-08-27T11:30:08.424588+0000","last_became_active":"2023-08-27T11:16:59.324206+0000","last_became_peered":"2023-08-27T11:16:59.324206+0000","last_unstale":"2023-08-27T11:30:08.424588+0000","last_undegraded":"2023-08-27T11:30:08.424588+0000","last_fullsized":"2023-08-27T11:30:08.424588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:03:55.723493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":307,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425228+0000","last_change":"2023-08-27T11:16:59.323602+0000","last_active":"2023-08-27T11:30:08.425228+0000","last_peered":"2023-08-27T11:30:08.425228+0000","last_clean":"2023-08-27T11:30:08.425228+0000","last_became_active":"2023-08-27T11:16:59.323157+0000","last_became_peered":"2023-08-27T11:16:59.323157+0000","last_unstale":"2023-08-27T11:30:08.425228+0000","last_undegraded":"2023-08-27T11:30:08.425228+0000","last_fullsized":"2023-08-27T11:30:08.425228+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:26.358331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.425132+0000","last_change":"2023-08-27T11:30:07.376873+0000","last_active":"2023-08-27T11:30:08.425132+0000","last_peered":"2023-08-27T11:30:08.425132+0000","last_clean":"2023-08-27T11:30:08.425132+0000","last_became_active":"2023-08-27T11:30:07.376104+0000","last_became_peered":"2023-08-27T11:30:07.376104+0000","last_unstale":"2023-08-27T11:30:08.425132+0000","last_undegraded":"2023-08-27T11:30:08.425132+0000","last_fullsized":"2023-08-27T11:30:08.425132+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:02:48.132543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":299,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.364071+0000","last_change":"2023-08-27T11:16:59.323219+0000","last_active":"2023-08-27T11:30:08.364071+0000","last_peered":"2023-08-27T11:30:08.364071+0000","last_clean":"2023-08-27T11:30:08.364071+0000","last_became_active":"2023-08-27T11:16:59.322873+0000","last_became_peered":"2023-08-27T11:16:59.322873+0000","last_unstale":"2023-08-27T11:30:08.364071+0000","last_undegraded":"2023-08-27T11:30:08.364071+0000","last_fullsized":"2023-08-27T11:30:08.364071+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:00:24.358263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365296+0000","last_change":"2023-08-27T11:30:07.375200+0000","last_active":"2023-08-27T11:30:08.365296+0000","last_peered":"2023-08-27T11:30:08.365296+0000","last_clean":"2023-08-27T11:30:08.365296+0000","last_became_active":"2023-08-27T11:30:07.374915+0000","last_became_peered":"2023-08-27T11:30:07.374915+0000","last_unstale":"2023-08-27T11:30:08.365296+0000","last_undegraded":"2023-08-27T11:30:08.365296+0000","last_fullsized":"2023-08-27T11:30:08.365296+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:30:06.487864+0000","last_clean_scrub_stamp":"2023-08-27T11:30:06.487864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:43.907376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":337,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-08-27T11:30:08.365390+0000","last_change":"2023-08-27T11:17:00.186083+0000","last_active":"2023-08-27T11:30:08.365390+0000","last_peered":"2023-08-27T11:30:08.365390+0000","last_clean":"2023-08-27T11:30:08.365390+0000","last_became_active":"2023-08-27T11:17:00.185797+0000","last_became_peered":"2023-08-27T11:17:00.185797+0000","last_unstale":"2023-08-27T11:30:08.365390+0000","last_undegraded":"2023-08-27T11:30:08.365390+0000","last_fullsized":"2023-08-27T11:30:08.365390+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:16:58.304918+0000","last_clean_scrub_stamp":"2023-08-27T11:16:58.304918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:16:52.155055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575011,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34056,"kb_used_data":2868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337784,"statfs":{"total":96636764160,"available":96601890816,"internally_reserved":0,"allocated":2936832,"data_stored":1709408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.22700000000000001},"max":{"1min":0.60399999999999998,"5min":0.79500000000000004,"15min":0.93500000000000005},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.51500000000000001},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.252},"max":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.81499999999999995},"last":0.55300000000000005}]},{"osd":1,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.28999999999999998,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.83399999999999996,"15min":0.872},"last":0.623},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.32500000000000001,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.84999999999999998},"last":0.48899999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.42799999999999999,"5min":0.32400000000000001,"15min":0.28299999999999997},"max":{"1min":0.67800000000000005,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.90200000000000002},"last":0.66800000000000004}]},{"osd":3,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":0.996,"15min":0.996},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.53000000000000003,"5min":0.42899999999999999,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.82899999999999996,"15min":0.91700000000000004},"last":0.753}]},{"osd":4,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.503},"min":{"1min":0.32600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.72199999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.32700000000000001},"max":{"1min":0.76600000000000001,"5min":1.224,"15min":1.224},"last":0.69599999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79700000000000004,"5min":0.86399999999999999,"15min":0.86599999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.84799999999999998},"last":0.71499999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83799999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.73899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33480,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338360,"statfs":{"total":96636764160,"available":96602480640,"internally_reserved":0,"allocated":2347008,"data_stored":1119082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.84599999999999997},"last":0.623},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.39800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.89800000000000002},"last":0.41399999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.53900000000000003},"min":{"1min":0.35599999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.81799999999999995,"15min":0.873},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.27400000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63800000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.499,"15min":0.51000000000000001},"min":{"1min":0.40799999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":2.0830000000000002},"last":0.45700000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.83199999999999996,"15min":0.84499999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.34000000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.93799999999999994},"last":0.58299999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":0.71599999999999997,"5min":0.753,"15min":0.83099999999999996},"last":0.307},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.85599999999999998,"15min":0.92800000000000005},"last":0.59699999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75700000000000001,"5min":0.85499999999999998,"15min":0.875},"last":0.66100000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:29:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79200000000000004,"5min":1.365,"15min":1.365},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.34899999999999998},"max":{"1min":0.77700000000000002,"5min":0.82999999999999996,"15min":0.91400000000000003},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575011,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34068,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337772,"statfs":{"total":96636764160,"available":96601878528,"internally_reserved":0,"allocated":2949120,"data_stored":1710114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.315,"15min":0.47399999999999998},"min":{"1min":0.21199999999999999,"5min":0.19500000000000001,"15min":0.191},"max":{"1min":0.45300000000000001,"5min":0.53300000000000003,"15min":11.971},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.315,"5min":0.32700000000000001,"15min":0.64700000000000002},"min":{"1min":0.23000000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.44900000000000001,"5min":0.74399999999999999,"15min":23.045999999999999},"last":0.33400000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34000000000000002,"15min":0.502},"min":{"1min":0.28199999999999997,"5min":0.23300000000000001,"15min":0.221},"max":{"1min":0.42399999999999999,"5min":0.57699999999999996,"15min":11.928000000000001},"last":0.316},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.33700000000000002},"min":{"1min":0.249,"5min":0.22800000000000001,"15min":0.224},"max":{"1min":0.46899999999999997,"5min":0.52200000000000002,"15min":0.58599999999999997},"last":0.29799999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38400000000000001,"15min":0.52800000000000002},"min":{"1min":0.309,"5min":0.27500000000000002,"15min":0.26300000000000001},"max":{"1min":0.47599999999999998,"5min":0.64100000000000001,"15min":10.99},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36799999999999999,"15min":0.68500000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.503,"5min":0.54700000000000004,"15min":22.893999999999998},"last":0.40899999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.33400000000000002},"max":{"1min":0.84999999999999998,"5min":0.90700000000000003,"15min":1.26},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58099999999999996,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.29099999999999998},"max":{"1min":0.745,"5min":0.88900000000000001,"15min":10.957000000000001},"last":0.69099999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.88600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":23.016999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.40999999999999998,"15min":0.39700000000000002},"max":{"1min":0.81899999999999995,"5min":0.88700000000000001,"15min":1.296},"last":0.66100000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58199999999999996,"15min":0.72799999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.371},"max":{"1min":0.72999999999999998,"5min":0.84899999999999998,"15min":10.925000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60199999999999998,"15min":0.90400000000000003},"min":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88,"5min":0.88,"15min":22.952000000000002},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59299999999999997,"15min":0.89800000000000002},"min":{"1min":0.433,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":22.988},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":1.327},"last":0.61799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575011,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33468,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338372,"statfs":{"total":96636764160,"available":96602492928,"internally_reserved":0,"allocated":2334720,"data_stored":1117609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.215,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.22800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.497,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35099999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36199999999999999,"15min":0.34399999999999997},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47699999999999998,"5min":0.628,"15min":0.63200000000000001},"last":0.373},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34799999999999998,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45800000000000002,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.29099999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35799999999999998,"15min":0.34599999999999997},"min":{"1min":0.27200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.53700000000000003,"15min":0.59599999999999997},"last":0.31},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.376},"min":{"1min":0.30499999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.53000000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.32200000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.34399999999999997},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.214},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.90200000000000002,"15min":1.2569999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.91400000000000003},"last":0.497},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":1.603},"last":0.72799999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79600000000000004,"5min":0.97099999999999997,"15min":0.97599999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.878,"15min":0.878},"last":0.59499999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83399999999999996,"5min":3.79,"15min":3.79},"last":0.749}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2338816,"data_stored":1118970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.51700000000000002},"last":0.31},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32100000000000001,"15min":0.318},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.19700000000000001},"max":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.629},"last":0.33300000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.252,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41699999999999998,"5min":0.50800000000000001,"15min":0.54400000000000004},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34699999999999998},"min":{"1min":0.26200000000000001,"5min":0.23000000000000001,"15min":0.22600000000000001},"max":{"1min":0.49299999999999999,"5min":0.56399999999999995,"15min":0.749},"last":0.35699999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.378},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":0.51800000000000002,"5min":0.57099999999999995,"15min":0.73699999999999999},"last":0.37},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.32600000000000001,"5min":0.27500000000000002,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.55100000000000005,"15min":0.69499999999999995},"last":0.38200000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.879},"last":0.63600000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36699999999999999},"max":{"1min":0.83799999999999997,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.53400000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.40000000000000002},"max":{"1min":0.78600000000000003,"5min":0.82699999999999996,"15min":0.83599999999999997},"last":0.59099999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:29:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.64500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33488,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338352,"statfs":{"total":96636764160,"available":96602472448,"internally_reserved":0,"allocated":2355200,"data_stored":1121164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.22,"5min":0.193,"15min":0.193},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.32800000000000001,"15min":0.32200000000000001},"min":{"1min":0.23300000000000001,"5min":0.223,"15min":0.20000000000000001},"max":{"1min":0.58299999999999996,"5min":0.755,"15min":0.755},"last":0.29999999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.34300000000000003,"15min":0.32700000000000001},"min":{"1min":0.245,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":0.67200000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.496},{"interface":"front","average":{"1min":0.371,"5min":0.38300000000000001,"15min":0.35499999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23400000000000001},"max":{"1min":0.60999999999999999,"5min":2.73,"15min":2.73},"last":0.40999999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36599999999999999},"min":{"1min":0.28899999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64200000000000002,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.372},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.69099999999999995,"5min":1.179,"15min":1.179},"last":0.377}]},{"osd":4,"last update":"Sun Aug 27 11:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.28599999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.56799999999999995},"min":{"1min":0.378,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68300000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.57699999999999996},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.314},"max":{"1min":1.006,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.41299999999999998,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.85999999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71499999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.40999999999999998,"15min":0.373},"max":{"1min":0.99399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.38},"max":{"1min":0.871,"5min":0.871,"15min":1.111},"last":0.70199999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59399999999999997,"15min":0.60099999999999998},"min":{"1min":0.42499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.236},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60299999999999998,"15min":0.58899999999999997},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.36499999999999999},"max":{"1min":0.88200000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575012,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33492,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338348,"statfs":{"total":96636764160,"available":96602468352,"internally_reserved":0,"allocated":2359296,"data_stored":1120638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68200000000000005,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.502},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.502},"min":{"1min":0.33900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58799999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66100000000000003,"5min":0.746,"15min":0.76500000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.75,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.53300000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.40400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72299999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.47299999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.96999999999999997,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.628}]},{"osd":5,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51300000000000001,"15min":0.52000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71499999999999997,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":1.865,"15min":1.865},"last":0.432}]},{"osd":6,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90600000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.624,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82799999999999996,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.57199999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:29:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.90100000000000002,"5min":2.714,"15min":2.714},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34200000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.45300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575011,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2338816,"data_stored":1118970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.376,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61099999999999999,"5min":0.78900000000000003,"15min":0.96899999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.44700000000000001,"5min":0.311,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":0.81200000000000006,"15min":0.98499999999999999},"last":0.41699999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67500000000000004,"5min":0.68400000000000005,"15min":1.075},"last":0.34300000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.95699999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.373,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.0580000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":1.03},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.42199999999999999,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.77800000000000002,"15min":1.026},"last":0.67800000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.27000000000000002},"max":{"1min":0.63900000000000001,"5min":0.78700000000000003,"15min":0.86899999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.29799999999999999,"15min":0.26900000000000002},"max":{"1min":0.73399999999999999,"5min":0.89400000000000002,"15min":0.91400000000000003},"last":0.64400000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.31},"max":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.88200000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.873,"15min":0.94199999999999995},"last":0.625}]},{"osd":7,"last update":"Sun Aug 27 11:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.34799999999999998},"max":{"1min":0.753,"5min":0.85999999999999999,"15min":0.95899999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.84399999999999997,"15min":0.92000000000000004},"last":0.74299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:30:13.481 INFO:tasks.ceph.mds.d.smithi138.stderr:2023-08-27T11:30:13.480+0000 7f2b37482700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:30:16.172 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:30:16.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.304+0000 7fe154ff9700 1 Processor -- start 2023-08-27T11:30:16.305 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.304+0000 7fe154ff9700 1 -- start start 2023-08-27T11:30:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe154ff9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe150146c00 0x7fe15015e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe154ff9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe1500cec70 0x7fe150066010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe154ff9700 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe150072250 con 0x7fe150066550 2023-08-27T11:30:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe154ff9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1500d0b60 con 0x7fe1500cec70 2023-08-27T11:30:16.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe154ff9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500669c0 con 0x7fe150146c00 2023-08-27T11:30:16.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe14dd9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe1500cec70 0x7fe150066010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.309 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe14dd9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe1500cec70 0x7fe150066010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39442/0 (socket says 172.21.15.112:39442) 2023-08-27T11:30:16.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe14dd9b700 1 -- 172.21.15.112:0/76239607 learned_addr learned my addr 172.21.15.112:0/76239607 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:16.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe14e59c700 1 --2- 172.21.15.112:0/76239607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe150146c00 0x7fe15015e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe14dd9b700 1 -- 172.21.15.112:0/76239607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe150146c00 msgr2=0x7fe15015e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:16.310 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe14dd9b700 1 --2- 172.21.15.112:0/76239607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe150146c00 0x7fe15015e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe14dd9b700 1 -- 172.21.15.112:0/76239607 >> v1:172.21.15.138:6789/0 conn(0x7fe150066550 legacy=0x7fe150161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe14dd9b700 1 -- 172.21.15.112:0/76239607 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1500724a0 con 0x7fe1500cec70 2023-08-27T11:30:16.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.305+0000 7fe14dd9b700 1 --2- 172.21.15.112:0/76239607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1500cec70 0x7fe150066010 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fe144009920 tx=0x7fe144009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38d527a1810fa38 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:16.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.306+0000 7fe14d59a700 1 -- 172.21.15.112:0/76239607 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400ba20 con 0x7fe1500cec70 2023-08-27T11:30:16.311 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.306+0000 7fe14d59a700 1 -- 172.21.15.112:0/76239607 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14400bb80 con 0x7fe1500cec70 2023-08-27T11:30:16.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.306+0000 7fe14d59a700 1 -- 172.21.15.112:0/76239607 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144009e70 con 0x7fe1500cec70 2023-08-27T11:30:16.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.306+0000 7fe154ff9700 1 -- 172.21.15.112:0/76239607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1500cec70 msgr2=0x7fe150066010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.306+0000 7fe154ff9700 1 --2- 172.21.15.112:0/76239607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1500cec70 0x7fe150066010 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fe144009920 tx=0x7fe144009c30 comp rx=0 tx=0).stop 2023-08-27T11:30:16.312 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.306+0000 7fe154ff9700 1 -- 172.21.15.112:0/76239607 shutdown_connections 2023-08-27T11:30:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.306+0000 7fe154ff9700 1 --2- 172.21.15.112:0/76239607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe150146c00 0x7fe15015e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.306+0000 7fe154ff9700 1 --2- 172.21.15.112:0/76239607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1500cec70 0x7fe150066010 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.306+0000 7fe154ff9700 1 -- 172.21.15.112:0/76239607 >> 172.21.15.112:0/76239607 conn(0x7fe15006f970 msgr2=0x7fe15006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.306+0000 7fe154ff9700 1 -- 172.21.15.112:0/76239607 shutdown_connections 2023-08-27T11:30:16.314 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.306+0000 7fe154ff9700 1 -- 172.21.15.112:0/76239607 wait complete. 2023-08-27T11:30:16.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.308+0000 7fe154ff9700 1 Processor -- start 2023-08-27T11:30:16.319 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.308+0000 7fe154ff9700 1 -- start start 2023-08-27T11:30:16.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.308+0000 7fe154ff9700 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe150066550 0x7fe150140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.308+0000 7fe154ff9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1500cec70 0x7fe15013d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.320 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.308+0000 7fe154ff9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe150146c00 0x7fe15015fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.308+0000 7fe154ff9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe150143340 con 0x7fe1500cec70 2023-08-27T11:30:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.308+0000 7fe154ff9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe150144180 con 0x7fe150146c00 2023-08-27T11:30:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.308+0000 7fe154ff9700 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1501411b0 con 0x7fe150066550 2023-08-27T11:30:16.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe14dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1500cec70 0x7fe15013d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe14dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1500cec70 0x7fe15013d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39448/0 (socket says 172.21.15.112:39448) 2023-08-27T11:30:16.322 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe14dd9b700 1 -- 172.21.15.112:0/2647381773 learned_addr learned my addr 172.21.15.112:0/2647381773 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-08-27T11:30:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe14e59c700 1 --2- 172.21.15.112:0/2647381773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe150066550 0x7fe150140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe14ed9d700 1 --2- 172.21.15.112:0/2647381773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe150146c00 0x7fe15015fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe14dd9b700 1 -- 172.21.15.112:0/2647381773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe150066550 msgr2=0x7fe150140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe14dd9b700 1 --2- 172.21.15.112:0/2647381773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe150066550 0x7fe150140c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe14dd9b700 1 -- 172.21.15.112:0/2647381773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe150146c00 msgr2=0x7fe15015fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe14dd9b700 1 --2- 172.21.15.112:0/2647381773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe150146c00 0x7fe15015fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe14dd9b700 1 -- 172.21.15.112:0/2647381773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe144009550 con 0x7fe1500cec70 2023-08-27T11:30:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe14ed9d700 1 --2- 172.21.15.112:0/2647381773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe150146c00 0x7fe15015fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe14dd9b700 1 --2- 172.21.15.112:0/2647381773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1500cec70 0x7fe15013d7d0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fe144009c70 tx=0x7fe144004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe13effd700 1 -- 172.21.15.112:0/2647381773 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144004550 con 0x7fe1500cec70 2023-08-27T11:30:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe13effd700 1 -- 172.21.15.112:0/2647381773 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14401e070 con 0x7fe1500cec70 2023-08-27T11:30:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe154ff9700 1 -- 172.21.15.112:0/2647381773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe150143570 con 0x7fe1500cec70 2023-08-27T11:30:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe13effd700 1 -- 172.21.15.112:0/2647381773 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1440038e0 con 0x7fe1500cec70 2023-08-27T11:30:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.309+0000 7fe154ff9700 1 -- 172.21.15.112:0/2647381773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe150143b30 con 0x7fe1500cec70 2023-08-27T11:30:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.310+0000 7fe154ff9700 1 -- 172.21.15.112:0/2647381773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe150067a20 con 0x7fe1500cec70 2023-08-27T11:30:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.311+0000 7fe13effd700 1 -- 172.21.15.112:0/2647381773 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 88698+0+0 (secure 0 0 0) 0x7fe144003a40 con 0x7fe1500cec70 2023-08-27T11:30:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.311+0000 7fe13effd700 1 --2- 172.21.15.112:0/2647381773 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe13006a4d0 0x7fe13006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.311+0000 7fe13effd700 1 -- 172.21.15.112:0/2647381773 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe144020070 con 0x7fe1500cec70 2023-08-27T11:30:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.311+0000 7fe14e59c700 1 --2- 172.21.15.112:0/2647381773 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe13006a4d0 0x7fe13006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.312+0000 7fe14e59c700 1 --2- 172.21.15.112:0/2647381773 >> [v2:172.21.15.138:6832/702326761,v1:172.21.15.138:6833/702326761] conn(0x7fe13006a4d0 0x7fe13006c990 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe1500ce760 tx=0x7fe140008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:2023-08-27T11:30:16.316+0000 7fe13effd700 1 -- 172.21.15.112:0/2647381773 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe144055310 con 0x7fe1500cec70